Table of contents
1 Introduction
Recently, I learned about the Hough line transform while studying opencv. The Hough line transform is an algorithm for finding straight lines in images. One of its application scenarios is to identify lanes. This article takes lane recognition as an example to introduce the simple usage of Hough lines.
2. Hough line transformation
2.1. What is Hough line transformation?
Here are the instructions given by chatGPT:
Hough Line Transform is an image processing technique that can be used to detect straight lines in images. The basic idea is to find these straight lines by converting them into parameter space and finding points in the parameter space that correspond to edges in the image. Hough line transform is commonly used in the field of computer vision, such as lane line detection, image splicing, and face recognition.
I won’t talk about the principle stuff here, because it’s a bit complicated and I’m a bit confused.
2.2. Basic usage in opencv
2.2.1. HoughLinesP function definition
The function that opencv implements Hough line transformation is HoughLinesP, which is defined as follows.
void HoughLinesP( InputArray image, OutputArray lines,
double rho, double theta, int threshold,
double minLineLength = 0, double maxLineGap = 0 );
The meanings of its parameters are as follows:
image: 8-bit, single-channel binary source image.
lines: vector of output lines. Each line is represented by a 4-element vector, and the vector<cv::Vec4i> type can be passed in.
Control precision:
rho: distance resolution of the accumulator in pixels.
theta: angular resolution of the accumulator (radians).
Filtering:
threshold: Accumulator threshold parameter.
minLineLength: Minimum line length. Line segments smaller than this length will be rejected.
maxLineGap: The maximum allowed gap between link points on the same line.
2.2.2. Usage
Because the image passed in by HoughLinesP must be an 8-bit, single-channel binary source image, before passing in the image, you need to perform the operation of converting to grayscale image -> converting to binary image.
opencv provides some methods for converting binary images. Because the purpose of HoughLinesP is to find straight lines, and straight lines are actually part of the outline, so generally we use the Canny algorithm to convert grayscale images into binary images.
Routine:
#include <opencv2/core.hpp>
#include <opencv2/imgcodecs.hpp>
#include <opencv2/highgui.hpp>
#include <opencv2/imgproc.hpp>
using namespace cv;
int main() {
Mat src = imread("road.png");
imshow("src", src);
Mat gray;
cvtColor(src, gray, COLOR_BGR2GRAY);
imshow("gray", gray);
// Apply Canny edge detection
Mat edges;
Canny(gray, edges, 50, 150);
imshow("canny", edges);
// Perform Hough transform to find lines
std::vector<Vec4i> lines;
HoughLinesP(gray, lines, 1, CV_PI / 180, 50, 50, 10);
// Draw lines on output image
Mat dst = src.clone();
for (size_t i = 0; i < lines.size(); i++) {
Vec4i vline = lines[i];
line(dst, Point(vline[0], vline[1]), Point(vline[2], vline[3]), Scalar(0, 0, 255), 2);
}
imshow("dst", dst);
waitKey(0);
}
3. Identify lanes
First prepare a picture, as shown in the picture below, and identify its white lane lines.
We directly use the routine from the previous section, and the effect is as follows.
It was found that although the lane was recognized, the texture in the environment was also mistaken for the lane, so further optimization was required.
3.1. Optimization
3.1.1. Noise reduction
As you can see from the Canny diagram above, the trees in the environment form dense textures, which are one of the factors that affect the effect.
After testing, I chose the method of "Binarization-"Corrosion-"Expansion" to complete noise reduction. The optimized code is as follows:
#include <opencv2/core.hpp>
#include <opencv2/imgcodecs.hpp>
#include <opencv2/highgui.hpp>
#include <opencv2/imgproc.hpp>
using namespace cv;
int main() {
Mat src = imread("/road.png");
imshow("src", src);
Mat gray;
cvtColor(src, gray, COLOR_BGR2GRAY);
imshow("gray", gray);
//二值化
Mat thr;
threshold(gray, thr, 100, 255, THRESH_BINARY);
imshow("threshold", thr);
// 腐蚀
Mat eroded;
Mat element = getStructuringElement(MORPH_RECT, Size(5, 5));
erode(thr, eroded, element);
// 膨胀
Mat dilated;
dilate(eroded, dilated, element);
imshow("dilated", dilated);
// Apply Canny edge detection
Mat edges;
Canny(dilated, edges, 50, 150);
imshow("canny", edges);
// Perform Hough transform to find lines
std::vector<Vec4i> lines;
HoughLinesP(edges, lines, 1, CV_PI / 180, 50, 50, 10);
// Draw lines on output image
Mat dst = src.clone();
for (size_t i = 0; i < lines.size(); i++) {
Vec4i vline = lines[i];
line(dst, Point(vline[0], vline[1]), Point(vline[2], vline[3]), Scalar(0, 0, 255), 2);
}
imshow("dst", dst);
waitKey(0);
}
The optimized effect is as follows:
It can be clearly seen from Canny that the environment texture is much less.
3.1.2. Filtering direction
In the picture above, you can see that there are some horizontal textures that affect the effect. We can further filter through the direction of straight lines.
From the perspective of the car, the lane is inclined toward the middle, and the lanes on both sides form a figure-eight shape, as shown in the picture.
In other words, the lane line will not have a small inclination angle on the image, so we can add a filter condition when obtaining the final result: straight lines with an inclination angle less than 20 degrees do not meet the conditions.
Modify the code as follows:
.....
// Perform Hough transform to find lines
std::vector<Vec4i> lines;
HoughLinesP(edges, lines, 1, CV_PI / 180, 50, 50, 10);
// Draw lines on output image
Mat dst = src.clone();
for (size_t i = 0; i < lines.size(); i++) {
Vec4i vline = lines[i];
/* 过滤倾斜45度及以下的斜线 */
float tanVal = (float)(vline[3] - vline[1]) / (vline[2] - vline[0]);
if (abs(tanVal) < tan(CV_PI / 18)) continue;
line(dst, Point(vline[0], vline[1]), Point(vline[2], vline[3]), Scalar(0, 0, 255), 2);
}
.......
Effect:
3.1.3. Interception area
When identifying the lane, because the lane is at the foot of the car, the only image that needs to be recognized is the lower half captured by the camera, so an additional layer of optimization can be added here: cut off the upper half of the image and only process the lower half of the image.
Modify the code:
int main() {
Mat src = imread("road.png");
Rect vaildRect(0, src.rows / 2, src.cols, src.rows / 2);
Mat src = src(vaildRect);
imshow("src", src);
......
Effect:
3.2. Test other pictures
3.2.1. Code
After the previous optimization, the following code is obtained:
int main() {
Mat src = imread("road.png");
Rect vaildRect(0, src.rows / 2, src.cols, src.rows / 2);
src = src(vaildRect);
imshow("src", src);
Mat gray;
cvtColor(src, gray, COLOR_BGR2GRAY);
imshow("gray", gray);
Mat thr;
threshold(gray, thr, 150, 255, THRESH_BINARY);
imshow("threshold", thr);
// 腐蚀
Mat eroded;
Mat element = getStructuringElement(MORPH_RECT, Size(5, 5));
erode(thr, eroded, element);
// 膨胀
Mat dilated;
dilate(eroded, dilated, element);
imshow("dilated", dilated);
// Apply Canny edge detection
Mat edges;
Canny(dilated, edges, 50, 150);
imshow("canny", edges);
// Perform Hough transform to find lines
std::vector<Vec4i> lines;
HoughLinesP(edges, lines, 1, CV_PI / 180, 50, 50, 10);
// Draw lines on output image
Mat dst = src.clone();
for (size_t i = 0; i < lines.size(); i++) {
Vec4i vline = lines[i];
float tanVal = (float)(vline[3] - vline[1]) / (vline[2] - vline[0]);
if (abs(tanVal) < tan(CV_PI / 18)) {
continue;
}
line(dst, Point(vline[0], vline[1]), Point(vline[2], vline[3]), Scalar(0, 0, 255), 2);
}
imshow("dst", dst);
waitKey(0);
}
Let's use this code to test other examples.
3.2.2, Picture 1
Because the brightness when taking pictures is different, the threshold during binarization needs to be adjusted according to the brightness. In this example,
threshold(gray, thr, 170, 255, THRESH_BINARY) is used;
3.2.3, Picture 2
3.2.4, Picture 3
threshold(gray, gray, 150, 255, THRESH_BINARY);
