To help industrial intelligent production quality inspection, develop and build a hot-rolled steel defect detection and recognition system based on yolov5n/s/m series models with different precisions, and compare and analyze performance differences

The combination of intelligent detection applications and deep learning with defects first closed has a very good application prospect, such as: PCB defect detection, cloth defect detection, tile defect detection, etc., in my previous blog post, I have practiced in the defect field There are also quite a few project development practices, but if you are emotional, you can read them on your own.

The core purpose of this article is to compare and analyze the performance differences of models with different precision series for steel defect detection. First, look at the renderings:

Just look at the dataset:

 The hot-rolled steel defect data set open sourced by Northeastern University is used here.

The data set contains 6 different types of defect data.

The detailed data configuration is as follows:

# Dataset
path: ./dataset
train:
  - images/train
val:
  - images/test
test:
  - images/test



# Classes
names:
  0: crazing
  1: inclusion
  2: patches
  3: pittedSurface
  4: rolledInScale
  5: scratches

For the model, three models of n/s/m series with different precision are used to complete the development of the model.

By default, the iterative calculation of 100 epochs is performed. Next, look at the result details:
[n series]

 【S series】

 【m series】

In order to compare and analyze the three models as a whole, here is a comprehensive comparison and visualization as follows:

The first is accuracy:

Precision is an indicator used to measure the performance of classification models, especially for binary classification problems. It measures how many of the samples predicted by the model to be positive are actually positive. Accuracy can be expressed by the following formula:

Precision = True Positives / (True Positives + False Positives)

Among them, the number of true positives is the number of samples that the model correctly predicts as positives and is actually positive, and the number of false positives is the number of samples that the model incorrectly predicts as positives but are actually negatives.

The value range of the accuracy rate is between 0 and 1. The higher the value, the higher the proportion of true positive cases in the classification results of the model, that is, the model will misjudge negative cases as positive cases less.

 Next is the recall rate:

Recall, also known as Sensitivity or True Positive Rate, is an indicator used to measure the performance of a classification model, especially for binary classification problems. It measures the model's ability to correctly identify all true positive examples. The recall rate can be expressed by the following formula:

Recall = True Positives / (True Positives + False Negatives)

Among them, the number of true positive cases is the number of samples that the model correctly predicts as positive and is actually positive, and the number of false negative cases is the number of samples that the model incorrectly predicts as negative but is actually positive.

The value range of the recall rate is between 0 and 1. The higher the value, the more accurately the model can identify the real positive samples, that is, the model will miss the positive samples less (wrongly judged as negative samples). .

 After that is the F1 value:

The F1 value is an evaluation index that comprehensively considers the precision rate (Precision) and the recall rate (Recall), and is used to measure the performance of the binary classification model. The F1 value is the harmonic mean of the precision rate and the recall rate, which can comprehensively evaluate the model's ability to correctly predict positive cases and minimize false negatives. The F1 value can be calculated using the following formula:

F1 value = 2 * (precision rate * recall rate) / (precision rate + recall rate)

Both the precision rate and the recall rate range from 0 to 1, and the F1 value also ranges from 0 to 1. The closer the F1 value is to 1, the better the performance of the model is when both precision and recall are considered.

The F1 value is very useful when dealing with imbalanced datasets and evaluating the overall performance of the model. For example, in the task of spam classification, we hope that the model can accurately classify spam as spam and miss real spam as little as possible. The F1 value combines these two indicators to provide a comprehensive performance evaluation indicator.

It should be noted that the F1 value regards the precision rate and the recall rate as equally important, and is suitable for tasks that do not have a clear priority relationship between the precision rate and the recall rate. If tasks have different importance for precision or recall, you can choose an appropriate evaluation indicator or adjust the threshold of the model according to your needs.

 Finally, the loss curve comparison:

 It can be seen that the model effect of the m series is better than that of n and s.