Target detection: yolov5 version 6.0 ncnn environment installation to + Android deployment one-stop tutorial
Article Directory
background
The subject of the topic stepped on a lot of pitfalls when deploying the model, and also saw that there are not many reference materials for yolov5 deployment, so Ben Xiaomengxin recorded the process of stepping on pitfalls, which is also convenient for everyone to refer to. yolov5->pt->onnx->ncnn float16-bit quantization
1. Preparation stage
1. Reference articles
1. YoloV5 official github
YoloV5 official TFLite, ONNX, CoreML, TensorRT Export implementation
2. The article written by nihui (the author of ncnn), thanks to nihui boss for enjoying the meal and eating
ncnn github address
ncnn-android-yolov5 github address
Detailed record u version YOLOv5 target detection ncnn implementation
detailed record u version YOLOv5 target detection ncnn implementation (PNNX version)
3. The reference article when I stepped on the pit is
the latest yolov5 to ncnn tutorial under windows.
The YOLOv5 to NCNN process
yolov5 to onnx to ncnn
is basically an article of 21 years. Some steps are actually unnecessary now. You can refer to it
4. ncnn deployment process under Linux– for reference
ncnn compile deployment (Linux)
#2. Process
1. To prepare the trained model, it is recommended to use the official yolov5.pt model to go through the whole process by yourself, and then replace it with your own model structure. Yolov5 official pt model link (link is version 6.1)
2.pt model->onnx 3.Detailed
explanation of ncnn environment configuration tutorial under Windows
4.onnx->ncnn
5.Android deployment of ncnn
2. pt model -> onnx
Download the yolov5 project on github, follow the yolov5 README guide, call detect.py to see if there is any problem with the model. When ready, call export.py for model conversion.
I use the yolov5-6.1 project , and copy yolov5s.pt to the file directory
python export.py --weights yolov5s.pt --include onnx --img 640 --train --simplify
Version 6.1 has been integrated, it is very convenient
to add –include to specify the export type as torchscript/onnx, and readers can convert it as needed.
Adding –train is to remove post-processing, that is, instead of directly outputting a classification result output, 3 feature maps are used as output.
Adding –simplify is to simplify the model. It does not need to be simplified onnxsim separately like other posts, which saves trouble.
If you are modifying your own model, remember --data xxx.yaml
and finally get the file
yolov5s.onnx
3. Detailed explanation of ncnn environment configuration tutorial under Windows
At first, I was learning the pnnx version of yolo deployment written by nihui, but it failed.
In the process of trying onnx later, I found that many reference materials and tutorials were not added to the ncnn environment deployment part. Most of them directly talk about the Android deployment of ncnn-android-yolov5.
In fact, for Mengxin ( at least I am ), I can’t go on following the tutorials and materials of station B. This is because they were out of gear when they shared the experience of converting the onnx model to the ncnn model, and ignored the ncnn configuration link.
1. Reference
Article: ncnn environment configuration
This reference article is very helpful to me, thanks to the author. Here I move over, invade and delete
Preparation items: VS, cmake, protobuf
are reasonable, the versions of these 3 should be fine, I think the latest version can be used [狗头]
2. VS installation
I'm using version 19, 19 and 22 on this projectshouldnot much differences.
Download address: https://visualstudio.microsoft.com/zh-hans/vs/
After downloading VS, install the following 3 workloads.
Readers who already have VS can supplement the tool load: Tools -> Get Tools and Functions
3. cmake installation
I have cmake myself, so I skipped it at the time, here is a reference to other articles
cmake-3.16 download address: Extraction code: zme6
cmake needs to change the environment variable,
click to open the decompressed cmake-3.16.5-win64-x64 folder and enterbinAdd the path to the environment variable,
such as the path: D:\software\cmake-3.16.5-win64-x64\bin
4. protobuf installation
protobuf-3.4.0 download address: extraction code:
decompress after downloading hd3z, then open the x64 command prompt of vs2019
or
jump to the newly decompressedprotobufroot directory, then create build-vs2019 files and perform cmake operations
Note: protobuf-root-dir is the download location of protobuf
> cd <protobuf-root-dir>
> mkdir build-vs2019
> cd build-vs2019
> cmake -G"NMake Makefiles" -DCMAKE_BUILD_TYPE=Release -DCMAKE_INSTALL_PREFIX=%cd%/install -Dprotobuf_BUILD_TESTS=OFF -Dprotobuf_MSVC_STATIC_RUNTIME=OFF ../cmake
> nmake
> nmake install
After success, several files will be generated under .\protobuf\build-vs2019
5. ncnn compile
Download the project directly on github, and decompress
the ncnn github address
Next we usex64 commands for vs2019, similar to the installation of protobuf, we operate according to the following code,
> cd <ncnn-root-dir>
> mkdir -p build-vs2019
> cd build-vs2019
> cmake -G"NMake Makefiles" -DCMAKE_BUILD_TYPE=Release -DCMAKE_INSTALL_PREFIX=%cd%/install -DProtobuf_INCLUDE_DIR=<protobuf-root-dir>/build-vs2019/install/include -DProtobuf_LIBRARIES=<protobuf-root-dir>/build-vs2019/install/lib/libprotobuf.lib -DProtobuf_PROTOC_EXECUTABLE=<protobuf-root-dir>/build-vs2019/install/bin/protoc.exe -DNCNN_VULKAN=OFF ..
> nmake
> nmake install
Note:ncnn-root-dirDownload the location for your ncnn, and all (3) in the fourth command cmakeprotobuf-root-dirReplace them with the reader's own protobuf path, pay attention toabsolute path。
After success, several files will be generated under .\nuxnn\build-vs2019.
The tools folder is the most commonly used by us, and the tools inside are basically integrated.
The install also contains commonly used libraries such as lib, bin, and include
Four, onnx->ncnn
In the second chapter, we got yolov5s.onnx.
In the third chapter, the ncnn environment is configured.
In this chapter, we perform model conversion.
1. ncnn conversion
The next step is to useThe x64 command line of vs2019, cd to .\ncnn\build-vs2019\tools\onnx
There will be onnx2ncnn.exe application, we will copy yolov5s.onnx to the directory, and then enter
.\onnx2ncnn yolov5s.onnx yolov5s.param yolov5s.bin
At this point we complete the ncnn conversion and get yolov5s.param and yolov5s.bin.
The model used by the subject isyolov5s.pt model before version 6.0, and ncnn does not support the focus structure, so the focus module needs to be manually repaired.
But after version 6.0, the focus has been officially removed. It is recommended that you check whether you are using the pt model of version 6.0. Yolov5 official pt model link (link is version 6.1)
Readers who use version 6.0 or above can jump to Chapter 5. Readers who use version 6.0 or below continue to deal with the error reporting of the focus structure:
at the same time, many Unsupported slice steps are reported, which is an error report for focus module conversion
Unsupported slice step !
Unsupported slice step !
Unsupported slice step !
Unsupported slice step !
Unsupported slice step !
Unsupported slice step !
Unsupported slice step !
Unsupported slice step !
If the following error is reported, is it because you did not export the model without –simpliffy?
Unsupported slice step !
Unsupported slice step !
... ....
Unsupported slice step !
Unsupported slice step !
Shape not supported yet
2. param explanation
Let's first analyze the param file of NCNN together, for example:
7767517 # 文件头 魔数
75 83 # 层数量 输入输出blob数量
# 下面有75行
Input data 0 1 data 0=227 1=227 2=3
Convolution conv1 1 1 data conv1 0=64 1=3 2=1 3=2 4=0 5=1 6=1728
ReLU relu_conv1 1 1 conv1 conv1_relu_conv1 0=0.000000
Pooling pool1 1 1 conv1_relu_conv1 pool1 0=0 1=3 2=2 3=0 4=0
Convolution fire2/squeeze1x1 1 1 pool1 fire2/squeeze1x1 0=16 1=1 2=1 3=1 4=0 5=1 6=1024
...
层类型 层名字 输入blob数量 输出blob数量 输入blob名字 输出blob名字 参数字典
参数字典，每一层的意义不一样：
数据输入层 Input data 0 1 data 0=227 1=227 2=3 图像宽度×图像高度×通道数量
卷积层 Convolution ... 0=64 1=3 2=1 3=2 4=0 5=1 6=1728
0输出通道数 num_output() ; 1卷积核尺寸 kernel_size(); 2空洞卷积参数 dilation(); 3卷积步长 stride();
4卷积填充pad_size(); 5卷积偏置有无bias_term(); 6卷积核参数数量 weight_blob.data_size()；
C_OUT * C_in * W_h * W_w = 64*3*3*3 = 1728
池化层 Pooling 0=0 1=3 2=2 3=0 4=0
0池化方式:最大值、均值、随机 1池化核大小 kernel_size(); 2池化核步长 stride();
3池化核填充 pad(); 4是否为全局池化 global_pooling();
激活层 ReLU 0=0.000000 下限阈值 negative_slope();
ReLU6 0=0.000000 1=6.000000 上下限
综合示例：
0=1 1=2.5 -23303=2,2.0,3.0
数组关键字 : -23300
-(-23303) - 23300 = 3 表示该参数在参数数组中的index
后面的第一个参数表示数组元素数量，2表示包含两个元素
3. Implement the focus module
After ncnn conversion we get yolov5s.param and yolov5s.bin
withnotebookWays to open yolov5s.param
Modify Split to Concat this10 floors, use the YoloV5Focus layer defined by ncnn to replace these 10 layers.
The input layer of the YoloV5Focus layer is connected to the output of the previous layer (orange box), and the output layer is connected to the input of the next layer (blue box).
In the second line at the beginning of param, the layer_count should be modified accordingly, but the blob_count only needs to be greater than or equal to the actual number. Tier 1 replaces Tier 10, so instead185-10+1=176
You must remember to change this. I was stuck here for a long time. At that time, I wondered why the App always crashed.
Then, drag to the final output layer, change the parameters of reshape, and modify the detection frame to a dynamic detection frame.
OK, after we save yolov5s.param, yolov5s.param and yolov5s.bin can be used normally at this time.
The obtained bin and param are deployed to Linux/win/android/ios according to your own needs. If you need to deploy to Android, continue to read
5. Android deployment of ncnn
Nihui's tutorial has been written very clearly, we just follow the steps. nihui YYDS!
ncnn-android-yolov5 github address
In addition, it is recommended that you use Netron to view our models
OK to get to the topic, assuming that the reader has installed Android Studio, deployed it and used the project of the nihui master, and the test can be used without any problem. our next step isReplace your own model on the app。
1. Model Replacement
Put the bin and param generated in Chapter 4 into
The directory where the project is located is F:\ncnn-android-yolov5\app\src\main\assets
Modify the names of param and bin at cpp to correspond to your own
Use Netron to view the model structure, and then modify the input (1 place) and output (3 places)
Change model class
Finally, if you are using a model trained by yourself, you need to modify the static const char* class_names[] under cpp to change it to the category of your own model.
Finally, the model can be quantified, the accuracy will not drop much, and the memory will be reduced by half.
CD to the ncnn directory and run with the command console, which is similar to the steps of converting ncnn from onnx
ncnnoptimize yolov5s.param yolov5s.bin yolov5s-opt.param yolov5s-opt.bin 65536