I. Introduction
Recently, the project stopped due to some reasons. When I was free, I found something to do for myself, so I set my sights on pedestrian detection. This model was sent by one of my target detection group leaders. He has accounts on Zhihu and git. , but I forgot. . . Don't say anything else, let's start. demo address: demo address
2. Model selection and conversion
The model is the basis of everything. The reason why this model was chosen is that its op HiSilicon supports it, and there is no very new op, which makes it possible to switch models. I won’t look at the model floor plan, it’s too big, I’ll provide the address of the gitee warehouse later, if you need it, go down and see it yourself.
1. Model conversion
The original model is the pytorch version, so it needs to be converted into onnx first, then into caffe, and finally into nnie-wk format. It took a little time here, and it’s been a long time, so I try to remember it (sorry, sorry).
The conversion route is pytorch->onnx->caffe, where onnx->caffe demo: https://gitee.com/huiwei13/onnx2caffe?_from=gitee_search , such demos are basically the same online, I don’t know who wrote the initial version . My pytorch version is 1.5, and there are demos on the Internet that convert pytorch directly to caffe, but they all require pytorch version 0.4 and below, which is not easy to download, and it is easy to report errors when opening the pytorch1.5 model of such a version, and the two frameworks of pytorch and caffe The implementation of maxpool is different. Even if the conversion is successful, the final model may still not be available. Therefore, it is still recommended to use onnx as a transfer. After all, pytorch directly provides an official interface for converting pt to onnx, which saves worry and effort, and then uses the above demo to convert to caffe format.
But one more thing to say, the converted onnx model cannot be used directly, and it needs to be simplified before it can be converted to the caffe format. onnx simplified demo: https://gitee.com/rsmeng/onnx-simplifier.
There is no error when converting pytorch to onnx, but there is some trouble when converting onnx to caffe, as shown in the figure below: the
two outputs of onnx in the above figure are reshaped and then output, but they become flatten in caffe, making the final result output Very weird. But it doesn't matter, for this kind of layer without parameters, just change it directly. 
The picture above is the original prototxt file, and the picture below is the changed and finally usable prototxt file. Basically changed the following:
(1) Change the "type:Flatten" at "reshape_274" to "type:Reshape". The parameter configuration refers to "reshape_272"
. It is very small, like 10e-5. After testing, it is suspected that the axis parameter is the reason, because the output dimension of the previous layer is three-dimensional, and axis = 1 corresponds to the output line in the original prototxt file. From this inference, the softmax output is indeed very small. . Changed to axis = 2, the test found that the problem was successfully solved;
(3) In the original prototxt file, the first dimension of the parameter under reshape is dim:1, check the HiSilicon development document, they suggest changing to dim:0, such a change It has no effect on the caffe model;
after the above changes, the caffe model is successfully fixed. The following is to convert the caffe to the wk model. When the original prototxt file was not modified at the beginning, I tried to convert it directly to the wk model, and failed without exception. . Therefore, it is better to fix the caffe model first, and then convert it to the wk model. And when I converted the available caffe, no error was reported, and the conversion was successful at one time, so I won’t say much.
caffe->wk: https://zhuanlan.zhihu.com/p/107548509, you can take a look at this old man, it is very helpful.
2. Model comparison
Anyone who has worked on HiSilicon knows that vector comparison is required after model conversion to improve accuracy. But at this stage, I encountered some very puzzling problems, so I will say it specially, hoping to help everyone.
The specific performance is that ruyi's comparison result is outrageous. Why it is outrageous and not very bad is because the comparison result has drifted far away from the input, which makes people confused. Quantification is still not enough, and the input image comparison is far from the same. It stands to reason that I understand the deviation between the middle layer and the output layer. The input is so exciting. After communicating with colleagues, I think it is the reason of ruyi, so I ignore this problem and run directly on the board, and finally found that the result is good.
Therefore, sometimes the results of ruyi can't explain anything. According to the experience of the big guys in my HiSilicon development group, run a data set on the board, save the results, and then calculate the map or accuracy.
3. Results
Finally, I ran it on the board. The 3516d can achieve real-time. The ssd-based model is fast inference, but the performance is not very strong, and it is not very accurate for long-distance recognition. Post the code address, take it yourself if you need it, it’s written in a hurry, if you don’t understand it, tell me in the comments. As for the pytorch version and the caffe version, I will look for the author's git, and you can download it yourself. I can provide the caffe model. The author's original model is from pytorch, without caffe.
demo: https://gitee.com/shopping-tang/pedestrain_detec_ssd.