Perform yolov5 TensorRT acceleration practice on vs2015 under win10

---

1. Installation environment

• CUDA10.2
• TensorRT7.2
• OpenCV3.4 (given in the project, no need to install)
• vs2015
  download related projects: https://github.com/wang-xinyu/tensorrtx.git

2. Generate yolov5s.wts file

Before generating yolov5s.wts, you first need to download the model. At the same time, we need to install the ultralytics/yolov5 environment. Here you can refer to other articles on the Internet or github tutorials for configuration and installation, and no details are given here.

• Copy the tensorrtx-master\yolov5folder under the folder gen_wts.pyto the ultralytics/yolov5folder

Execute in the current directory:

python gen_wts.py

Eventually we will get a yolov5s.wtsfile in the current directory .

3. Build the vs2015 environment

Here we use the library compiled by others, download link: tensorrtx . (Note: Some header files and lib files have been given in my future project, students who need it can download it directly)

• Create a vs project, name it yolov5_Trt, and tensorrtx-master\yolov5copy the files in the folder to the created project project

• Add header file

C:\Users\Administrator\Documents\Visual Studio 2015\Projects\yolov5_Trt\yolov5_Trt\include
C:\Users\Administrator\Documents\Visual Studio 2015\Projects\yolov5_Trt\yolov5_Trt\include\tensorrt
C:\Users\Administrator\Documents\Visual Studio 2015\Projects\yolov5_Trt\yolov5_Trt\include\cudnn
C:\Program Files\NVIDIA GPU Computing Toolkit\CUDA\v10.2\include
C:\Users\Administrator\Documents\Visual Studio 2015\Projects\yolov5_Trt\yolov5_Trt\include\opencv\opencv2
C:\Users\Administrator\Documents\Visual Studio 2015\Projects\yolov5_Trt\yolov5_Trt\include\opencv

• Add lib

C:\Program Files\NVIDIA GPU Computing Toolkit\CUDA\v10.2\lib\x64
C:\Users\Administrator\Documents\Visual Studio 2015\Projects\yolov5_Trt\yolov5_Trt\lib\trt
C:\Users\Administrator\Documents\Visual Studio 2015\Projects\yolov5_Trt\yolov5_Trt\lib\opencv
C:\Users\Administrator\Documents\Visual Studio 2015\Projects\yolov5_Trt\yolov5_Trt\lib\cudnn

• Add dependencies

cudart.lib
cublas.lib
cudnn.lib
cudnn64_8.lib
myelin64_1.lib
nvinfer.lib
nvinfer_plugin.lib
nvonnxparser.lib
nvparsers.lib
opencv_world340.lib

4. TensorRt accelerated implementation

• Modify the yolov5.cppsource code

Since the source code needs to be executed under the command line, we need to change it and be able to execute it directly

Change the code from lines 421 to 449:

    if (argc == 2 && std::string(argv[1]) == "-s") {
        IHostMemory* modelStream{ nullptr };
        APIToModel(BATCH_SIZE, &modelStream);
        assert(modelStream != nullptr);
        std::ofstream p(engine_name, std::ios::binary);
        if (!p) {
            std::cerr << "could not open plan output file" << std::endl;
            return -1;
        }
        p.write(reinterpret_cast<const char*>(modelStream->data()), modelStream->size());
        modelStream->destroy();
        return 0;
    } else if (argc == 3 && std::string(argv[1]) == "-d") {
        std::ifstream file(engine_name, std::ios::binary);
        if (file.good()) {
            file.seekg(0, file.end);
            size = file.tellg();
            file.seekg(0, file.beg);
            trtModelStream = new char[size];
            assert(trtModelStream);
            file.read(trtModelStream, size);
            file.close();
        }
    } else {
        std::cerr << "arguments not right!" << std::endl;
        std::cerr << "./yolov5 -s  // serialize model to plan file" << std::endl;
        std::cerr << "./yolov5 -d ../samples  // deserialize plan file and run inference" << std::endl;
        return -1;
    }

Replace with:

if (true) {
		IHostMemory* modelStream{ nullptr };
		APIToModel(BATCH_SIZE, &modelStream);
		assert(modelStream != nullptr);
		std::ofstream p(engine_name, std::ios::binary);
		if (!p) {
			std::cerr << "could not open plan output file" << std::endl;
			return -1;
		}
		p.write(reinterpret_cast<const char*>(modelStream->data()), modelStream->size());
		modelStream->destroy();
		return 0;
	}
	else {
		std::ifstream file(engine_name, std::ios::binary);
		if (file.good()) {
			file.seekg(0, file.end);
			size = file.tellg();
			file.seekg(0, file.beg);
			trtModelStream = new char[size];
			assert(trtModelStream);
			file.read(trtModelStream, size);
			file.close();
		}
	}

• Generate yolov5s.enginefile

Run the code directly, the following problems will occur

1. fatal error C1083: Cannot open include file: "dirent.h": No such file or directory

Solution: VS2017/2019 cannot open include file: "dirent.h": No such file or directory

• Run TensorRT under yolov5s

Change line 423 if(true)to if(false)run directly. Here we tested the "./test_data/3.jpg"picture, the picture size is: 640*480

The final test result is:

Excluding the time to load the model for the first time, it takes about 30ms to perform an inference on my own notebook GeForce MX250.

My project link: link: https://pan.baidu.com/s/1SkJUrSOWQf6v8WkubFIA-Q extraction code: v37i