Chapter 16: The Hough Transform

• The Hough transform is a method of finding lines, circles, and other simple shapes in an image.
• The Hough transform uses a method similar to voting to obtain the shape set in the current image. This transform was first proposed by Paul Hough (Hough) in 1962.

The original Hough transform can only be used to detect straight lines. After development, the Hough transform can not only recognize straight lines, but also recognize other simple graphic structures, such as circles and ellipses. In fact, as long as it is an object that can be expressed by a parameter equation, it is suitable for detection by Hough transform.

The following mainly introduces the Hough line transform and the Hough circle transform .

• The Hough line transform is used to find lines in an image. In OpenCV, it can be realized with function cv2.HoughLines() and function cv2.HoughLinesP().
• The Hough circle transform is used to find circles within an image. To use the function cv2.HoughCircles () to achieve.

1. Hough linear transformation:

OpenCV provides the function cv2.HoughLines() and the function cv2.HoughLinesP() to realize the Hough line transformation. The following first introduces the basic principle of the Hough transform, and then introduces the basic usage of these two functions.

1. Hough linear transformation principle:

​ In order to facilitate the explanation of the problem, first take the familiar Cartesian coordinate system (that is, the plane Cartesian coordinate system, corresponding to the Cartesian space) as an example to illustrate the basic principle of the Hough transform. Corresponding to the Cartesian coordinate system, we construct a Hough coordinate system (corresponding to the Hough space). In the Hough coordinate system, the abscissa uses the slope k of the straight line in the Cartesian coordinate system, and the ordinate uses the intercept b of the straight line in the Cartesian coordinate system.

​ First, we observe the mapping of a straight line in Cartesian space in Hough space. For example, in the figure below, the left picture is the Cartesian xy coordinate system (Cartesian space), and the right picture is the Hough kb coordinate system (Hough space). In Cartesian space, there is a straight line y=k0x+b0, the intercept k0 of this line is a known constant, and the intercept b0 is also a known constant. Map the straight line into the Hough space, find the known point (k0, b0), and complete the mapping.

​ From the above analysis, it can be seen that a straight line in Cartesian space has a slope of k and an intercept of b, which is mapped to a point (k,b) in Hough space. Or, it can be understood that a point (k0, b0) in Hough space is mapped to Cartesian space, which is a straight line y=k0x+b0.

​ Here, we use the word "mapping" to express the correspondence between different spaces (coordinate systems), which can also be expressed as "determined". For example, the above relationship can be expressed as:

• A line in Cartesian space defines a point in Hough space.
• A point in Hough space defines a line in Cartesian space.

​ Next, observe the mapping of a point in Cartesian space in Hough space. As shown in the figure below, there is a point (x0, y0) in Cartesian space, and the straight line passing through this point can be expressed as y0=kx0+b. Among them, (x0, y0) are known constants, and (k, b) are variables.

​ For the expression y0=kx0+b, through the left and right shift of the arithmetic operation, it can be expressed as b=−x0k+y0. When the point (x0, y0) is mapped to the Hough space, it can be considered that the slope of the corresponding line is −x0 and the intercept is y0, that is, b=−x0k+y0, as shown in the straight line on the right in the figure below.

From the above analysis we can see that:

• The point (x0, y0) in the Cartesian space is mapped to the Hough space, which is the straight line b=−x0k+y0.
• The straight line b=−x0k+y0 in Hough space is mapped to Cartesian space, which is the point (x0, y0).

Let's look at the situation where two points in Cartesian space are mapped to Hough space. For example, in the figure below, there are two points (x0, y0) and (x1, y1) in the Cartesian space of the left figure, and analyze the situation where these two points are mapped to the Hough space.

For the convenience of understanding, we analyze the mapping of these two points in Cartesian space to Hough space from different angles.

• Angle 1: A point in Cartesian space maps to a line in Hough space.

  In Cartesian space, there are any two points (x0, y0), (x1, y1). In Hough space, these two points correspond to two different straight lines. Of course, it can be seen through analysis that one straight line is b=−x0k+y0, and the other straight line is b=−x1k+y1.

• Angle 2: A line in Cartesian space maps to a point in Hough space

  In Cartesian space, there are any two points (x0, y0), (x1, y1). These two points must be connected by a straight line, and the straight line connecting them is marked as y=k1x+b1, then the intercept and slope of the line are (k1, b1). That is, the straight line is mapped to the point (k1, b1) in the Hough space.

From the above analysis we can see that:

• Two points in Cartesian space are mapped to two lines intersecting at (k1,b1) in Hough space.
• The straight line corresponding to these two points will be mapped to the point (k1, b1) in the Hough space.

In other words, angle 1 determines the number of lines, and angle 2 determines the point where the two lines intersect.

This shows that if there are two points A and B in Cartesian space, they can be connected into a straight line y=k1x+b1, then there will be two straight lines on the point (k1,b1) in Hough space, Corresponding to two points A and B in Cartesian space respectively.

Let's look at the situation where three points in Cartesian space are mapped to Hough space. In the figure below, the left picture is a Cartesian space, in which there are three points (0,1), (1,2), (2,3).

The following analyzes the mapping of these three points in Cartesian space to Hough space from different angles.

• Angle 1: A point in Cartesian space is mapped to a line in Hough space.

  For example, when three points (0,1), (1,2), and (2,3) in Cartesian space are mapped to Hough space, each point corresponds to a straight line, and the corresponding relationship is shown in the table.

  According to the corresponding relationship, it can be known that:

  • The point (0,1) in Cartesian space corresponds to the straight line b=1 in Hough space.
  • The point (1,2) in Cartesian space corresponds to the straight line b=-k+2 in Hough space.
  • The point (2,3) in Cartesian space corresponds to the straight line b=-2k+3 in Hough space.

From the above analysis, we can see that three points in Cartesian space are mapped to three straight lines in Hough space.

• Angle 2: A line in Cartesian space maps to a point in Hough space.

  For example, the three points (0,1), (1,2), and (2,3) in Cartesian space correspond to the straight line y=x+1, the slope k is 1, and the intercept b is 1. The line y=x+1 maps to the point (1,1) in Hough space.

From the above analysis of angle 1 and angle 2, we can know that:

• The three points (0,1), (1,2), and (2,3) in Cartesian space are mapped to three straight lines intersecting at point (1,1) in Hough space.
• The straight line formed by the three points (0,1), (1,2), and (2,3) in Cartesian space is mapped to the point (1,1) in Hough space

​ This shows that if there are three points in Cartesian space, and they can be connected into a straight line y=k1x+b1, then in Hough space, there will be three straight lines on the corresponding point (k1, b1), correspond to three points in Cartesian space, respectively.

​ So far, we have found that if in Cartesian space, there are N points that can be connected into a straight line y=k1x+b1, then there will be N straight lines passing through the corresponding points in Hough space (k1 ,b1). Or conversely, if in the Hough space, there are more straight lines passing through the point (k1, b1), it means that in the Cartesian space, there are more points located on a straight line with a slope of k1 and an intercept of b1 On y=k1x+b1.

​ Now, let's look at an example where more points in Cartesian space are mapped to Hough space, and verify the above point of view. In the figure below, the left figure shows the Cartesian space, in which there are 6 points. Let’s look at the mapping of these 6 points in the Hough space of the right figure from different angles.

• Angle 1: A point in Cartesian space maps to a line in Hough space.

6 points in Cartesian space: (0,1), (1,2), (2,3), (3,4), (3,2), (1,4), mapped to Hough In space, each point corresponds to a straight line, and the corresponding relationship is shown in the table

According to the corresponding relationship, it can be known that:

• The point (0,1) in Cartesian space corresponds to the straight line b=1 in Hough space.
• The point (1,2) in Cartesian space corresponds to the straight line b=-k+2 in Hough space.
• The point (2,3) in Cartesian space corresponds to the straight line b=-2*k+3 in Hough space.
• The point (3,4) in Cartesian space corresponds to the straight line b=-3*k+4 in Hough space.
• The point (3,2) in Cartesian space corresponds to the straight line b=-3*k+2 in Hough space.
• The point (1,4) in Cartesian space corresponds to the straight line b=-1*k+4 in Hough space.

From the above analysis, we can see that the 6 points in Cartesian space are mapped to 6 straight lines in Hough space

• Angle 2: A line in Cartesian space maps to a point in Hough space.

  Here, for the convenience of observation, the lines connected with more points in the Cartesian space are drawn: the line LineA connecting the points (0,1), (1,2), (2,3), (3,4), The line LineB connecting the points (2,3), (3,2), (1,4), the line LineC connecting the points (0,1), (3,2)

  

It should be noted that in Cartesian space, there are many straight lines between each point. For example, there are straight lines between points (1,2) and (3,2), between points (3,2) and (3,4), and between points (1,4) and (3,4) , the simplification is made here, and the above-mentioned straight lines are not drawn.

The following analyzes the mapping of the three straight lines LineA, LineB, and LineC in Cartesian space in Hough space.

• The straight line LineA passes through 4 points, the expression is y=1 * x+1, the slope k is 1, and the intercept b is 1, which corresponds to point A(1,1) in the Hough space.
• The straight line LineB passes through 3 points, the expression is y=-1 * x+5, the slope k is -1, and the intercept b is 5, which corresponds to point B(-1,5) in the Hough space.
• The straight line LineC has passed through 2 points, the expression is y=-1/3 * x+1, the slope k is -1/3, and the intercept b is 1, which corresponds to point C (-1/3 ,1).

As can be seen in the figure above, in the Hough space shown on the right, there are 4 straight lines passing through point A, 3 straight lines passing through point B, and 2 straight lines passing through point C. Analyzing the above relationship:

• There are 4 straight lines passing through point A in Hough space. Point A determines a straight line in Cartesian space, and the line passes through 4 points at the same time, that is, point A in Hough space determines LineA in Cartesian space, which contains (0,1), (1,2), (2,3), (3,4) a total of 4 points.
• There are 3 straight lines passing through point B in Hough space. Point B determines a straight line in Cartesian space, and the line passes through 3 points at the same time, that is, point B in Hough space determines LineB in Cartesian space, which contains (2,3), There are 3 points (3,2) and (1,4).
• There are 2 straight lines passing through point C in Hough space. Point C determines a straight line in Cartesian space, and the line passes through two points at the same time, that is, point C in Hough space determines LineC in Cartesian space, which contains (0,1), (2,3) has 2 points in total.

​ To sum up, in the Hough space, the more straight lines passing through a point, it means that the straight line mapped in the Cartesian space is composed of (passes through) more points. We know that two points can form a straight line. However, if a point is generated due to a calculation error, then it and another point will also form a straight line. At this time, a straight line that does not actually exist will be constructed out of thin air. This situation is to be avoided as much as possible.

Therefore, in the calculation, we hope to construct a straight line with more points to improve the reliability of the straight line. That is to say, if a straight line is composed of more points, then the possibility of its actual existence is greater, and its reliability is also higher.

​Therefore , the basic idea of ​​Hough transform to select a straight line is to select a point where as many straight lines as possible meet.

The above are all illustrated by taking the familiar Cartesian space as an example. In Cartesian space, there may be a vertical line LineA such as x=x0 of the form

At this time, the slope k is infinite, and the intercept b cannot take a value. Therefore, the vertical line LineA in the above figure cannot be mapped into the Hough space. In order to solve the above problems, it can be considered to map the Cartesian coordinate system to the polar coordinate system, as shown in the figure.

In the Cartesian coordinate system, the slope k and the intercept b are used, that is, a straight line is represented by (k,b). In the polar coordinate system, it is represented by the polar radius r (sometimes also expressed by ρ) and the polar angle θ, that is, (r, θ). A straight line in polar coordinates can be expressed as:

• r=xcosθ+ysinθ

​ For example, the straight line LineA in the above figure can be represented by the polar radius r and polar angle θ in polar coordinates. Among them, r is the distance between the straight line LineA and the image origin O, and the parameter θ is the angle between the vertical line LineB of the straight line LineA and the x-axis. In this representation, the straight line in the image has an angle θ of (0~π), and the maximum value of r is the length of the diagonal of the image. Using this representation method, it is very convenient to represent the straight line formed by the three points in the figure above.

Similar to the mapping relationship between Cartesian space and Hough space:

• A point in the polar coordinate system is mapped to a line (curve) in the Hough coordinate system (Hough space).
• A line in polar coordinates maps to a point in Hough coordinates.

​In general , a straight line in polar coordinates can be evaluated by the number of lines that intersect at a point in Hough coordinates. In the Hough coordinate system, the more lines that pass through a point, it means that the straight line mapped in the polar coordinate system is formed (passed) by more points. Therefore, the basic idea of ​​Hough transform to select a straight line is to select the points formed by as many lines as possible.

​ Usually, a threshold is set, and when the curve intersecting at a certain point in the Hough coordinate system reaches the threshold, it is considered that a straight line exists (detected) in the corresponding polar coordinate system.

​ The above content is the principle of Hough transform. Even if we don't understand the above principles at all, it will not affect us to use the Hough transform function provided by OpenCV to perform Hough transform. OpenCV itself is a black box, which provides us with interfaces (parameters, return values). We only need to master the correct use of the interface, and we can correctly handle image problems without mastering its internal working principle.

​ In some cases, image processing software such as OpenCV library and Photoshop are similar, as long as you master how to use them, you can get correct processing results. When performing image processing, we do not need to pay attention to technical details such as its realization principle. However, if we further understand how it works, it will also be of great benefit to our work.

2. HoughLines function:

OpenCV provides the function cv2.HoughLines() to implement the Hough line transform. This function requires that the source image to be operated is a binary image, so the source image must be binarized before the Hough transform, or Canny edge detection.

The syntax format of the function cv2.HoughLines() is:

• lines=cv2.HoughLines（image,rho,theta,threshold）
  • image: The input image, that is, the source image, must be an 8-bit single-channel binary image. If it is another type of image, it needs to be modified to the specified format before performing the Hough transform.
  • rho: the precision of the distance r in pixels. Typically, a precision of 1 is used.
  • theta: the accuracy of the angle θ. Typically, the precision used is π/180, meaning that all possible angles are to be searched.
  • threshold: is the threshold. The smaller the value, the more straight lines are judged. According to the analysis in the previous section, when identifying a straight line, it is necessary to determine how many points are located on the straight line. When determining whether a line exists, the number of points passed by the line is evaluated. If the number of points passed by the line is less than a threshold, these points are considered to happen to (by chance) form the line algorithmically, but in the source image The line does not exist; if it is greater than the threshold, the line is considered to exist. So, if the threshold is small, more straight lines will be obtained; if the threshold is large, fewer straight lines will be obtained.
  • lines: The return value, each element is a pair of floating point numbers, representing the parameters of the detected straight line, ie (r, θ), which is of type numpy.ndarray.

Notice:

​ Using the function cv2.HoughLines() detects straight lines in the image instead of line segments, so the detected straight lines have no endpoints. Therefore, the straight lines we draw when performing the Hough line transformation pass through the entire image.

​ The method of drawing a straight line is, for a straight line in the vertical direction (not a vertical line, but a straight line at various angles in the vertical direction), calculate the distance between it and the horizontal boundary of the image (that is, the first and last lines in the image) intersection, and draw a line between those two intersections. For straight lines in the horizontal direction, it is done in a similar way, except that the first and last columns of the image are used.

​ When drawing a line, the function used is cv2.line(). The convenience of this function is that it draws the line correctly even if the coordinates of the points are outside the bounds of the image, so there is no need to check whether the intersection point is inside the image. Traverse the return value lines of the function cv2.HoughLines()

import cv2
import numpy as np
import matplotlib.pyplot as plt

img = cv2.imread('./buliding.jpg')
gray_img = cv2.cvtColor(img, cv2.COLOR_BGR2GRAY)
edges = cv2.Canny(gray_img, 50, 150, apertureSize=3)

rgb_img = cv2.cvtColor(img, cv2.COLOR_BGR2RGB)
new_img = rgb_img.copy()
print(lines)
for line in lines:
    rho, theta = line[0]
    a = np.cos(theta)
    b = np.sin(theta)

    x0 = a * rho
    y0 = b * rho
    x1 = int(x0 + 1000 * (-b))
    y1 = int(y0 + 1000 * (a))
    x2 = int(x0 - 1000 * (-b))
    y2 = int(y0 - 1000 * (a))
    cv2.line(new_img, (x1, y1), (x2, y2), (0, 0, 255), 2)

plt.subplot(131)
plt.imshow(rgb_img)
plt.axis('off')

plt.subplot(132)
plt.imshow(edges)
plt.axis('off')

plt.subplot(133)
plt.imshow(new_img)
plt.axis('off')

plt.show()

Supplement: How to calculate the two points (x1, y1) and (x2, y2) of HoughLines in OpenCV.

The figure above shows how to enlarge the line segment to 1000, that is, the calculation method when r=1000. It can be 300 or 500 or 800.

3. HoughLinesP function:

The probabilistic Hough transform makes some modifications to the basic Hough transform algorithm, and is an optimization of the Hough transform algorithm. It doesn't consider all points. Instead, it only needs a random subset of points sufficient for line detection.

In order to better judge straight lines (line segments), the probabilistic Hough transform algorithm also makes two improvements to the method of selecting straight lines:

• **Minimum length of line accepted. **If there are more than the threshold number of pixels to form a straight line, but the straight line is very short, then the straight line will not be accepted as the judgment result, and the straight line is considered to be just a few pixels in the image that happen to be random It constitutes an algorithmic straight-line relationship, but in fact, this straight line does not exist in the original picture.
• **Maximum pixel spacing allowed when accepting straight lines. **If there are more than the threshold number of pixels to form a straight line, but the distance between the group of pixels is very far, the straight line will not be accepted as the judgment result, and the straight line is considered to be only a few in the image The pixels just randomly constitute an algorithmic straight line relationship, but in fact this straight line does not exist in the original image.

The probability Hough transform is implemented in OpenCV through the cv2.HoughLinesP() function:

• lines = cv2.HoughLiesP(image, rho, theta, threshold, minLineLength, maxLineGap)
  • image: The input image, that is, the original image, must be an 8-bit single-channel binary image. For other types of images, they need to be modified to this specified format before performing the Hough transform.
  • rho: the precision of the distance r in pixels. Typically, a precision of 1 is used.
  • theta: is the accuracy of the angle θ. In general, the precision used is np.pi/180, indicating possible angles to search for.
  • threshold: is the threshold. The smaller the value, the more straight lines are judged; the larger the value, the fewer straight lines are judged.
  • minLineLength: used to control the value of "minimum length of accepted line", the default value is 0.
  • maxLineGap: Used to control the minimum interval between accepted collinear line segments, that is, the maximum interval between two points in a line.
    If the distance between two points exceeds the value of the parameter maxLineGap, the two points are not considered to be on a line. The default value is 0.
  • lines: return value. It is composed of elements of type numpy.ndarray, each of which is a pair of floating point numbers, representing the parameters of the detected straight line, namely (x1, y1), (x2, y2).

import cv2
import numpy as np
import matplotlib.pyplot as plt

img = cv2.imread('buliding.jpg')
grey_img = cv2.cvtColor(img, cv2.COLOR_BGR2GRAY)
rgb_img = cv2.cvtColor(img, cv2.COLOR_BGR2RGB)
new_img = rgb_img.copy()

edges = cv2.Canny(grey_img, 150, 200, apertureSize=3)

lines = cv2.HoughLinesP(edges, 1, np.pi / 180, 100, minLineLength=100, maxLineGap=30)
for line in lines:
    x1, y1, x2, y2 = line[0]
    cv2.line(new_img, (x1, y1), (x2, y2), (255, 0, 0), 2)

plt.subplot(131)
plt.imshow(rgb_img)
plt.title('img')
plt.axis('off')

plt.subplot(132)
plt.imshow(edges)
plt.title('edges')
plt.axis('off')

plt.subplot(133)
plt.imshow(new_img)
plt.title('rst')
plt.axis('off')

plt.show()

2. Hough ring transformation:

In addition to being used to detect straight lines, the Hough transform can also be used to detect other geometric objects. In fact, as long as it is an object that can be expressed by a parameter equation, it is suitable for detection by Hough transform.

​Using Hough circle transform to detect circles in images is similar to the principle of using Hough line transform to detect straight lines. In the Hough circle transformation, three parameters, the radius of the circle and the center of the circle (x coordinate, y coordinate), need to be considered. In OpenCV, the strategy adopted is two rounds of screening. The first round of screening finds the position (center) of the possible circle; the second round screens out the radius based on the results of the first round.

​Similar to the two parameters used to decide whether to accept a straight line, "minimum length of accepted line (minLineLength)" and "maximum pixel spacing allowed when accepting a straight line (MaxLineGap)", there are several Hough circle transforms used to decide whether Accepts parameters for circles: the minimum distance between centers, the minimum radius of circles, and the maximum radius of circles.

In OpenCV, the Hough circle transformation is realized by the function cv2.HoughCircles(), **This function combines Canny edge detection and Hough transformation. ** Its grammatical format is:

• circles = cv2.HoughCircles(image, method, dp, minDist, param1, param2, minRadius, maxradius)
  • image: The input image, that is, the source image, is an 8-bit single-channel grayscale image.
  • method: detection method. As of OpenCV 4.0.0-pre version, HOUGH_GRADIENT is the only available parameter value. This parameter represents the method used for two rounds of detection in Hough circle detection.
  • dp: accumulator resolution, which is a division ratio, used to specify the ratio of the image resolution to the center accumulator resolution. For example, if dp=1, the input image and the accumulator have the same resolution.
  • minDist: The minimum distance between circle centers. This value is used as a threshold, if there are multiple circles whose centers are less than this value, only one will be detected. Therefore, if the value is too small, multiple adjacent circles will be detected; if the value is too large, some circles may be missed during detection.
  • param1: This parameter is default, and the default value is 100. It corresponds to the high threshold of the Canny edge detector (the low threshold is half of the high threshold).
  • param2: The number of votes that must be received at the center of the circle. Only the circles whose votes exceed this value in the first round of screening are eligible to enter the second round of screening. Therefore, the larger the value, the fewer circles are detected; the smaller the value, the more circles are detected. This parameter is the default and has a default value of 100 by default.
  • minRadius: The minimum value of the radius of the circle, circles smaller than this value will not be detected. This parameter is default, and has a default value of 0 by default, and this parameter has no effect at this time.
  • maxRadius: The maximum value of the radius of the circle, circles larger than this value will not be detected. This parameter is default, and has a default value of 0 by default, and this parameter has no effect at this time.
  • circles: The return value, a numpy.ndarray composed of the coordinates of the center of the circle and the radius.

Special attention should be paid to smoothing the source image before calling the function cv2.HoughLinesCircles() to reduce the noise in the image and avoid misjudgment.

import cv2
import numpy as np
import matplotlib.pyplot as plt

img = cv2.imread('xiangqi.jpg')
gray_img = cv2.cvtColor(img, cv2.COLOR_BGR2GRAY)
rgb_img = cv2.cvtColor(img, cv2.COLOR_BGR2RGB)
new_img = rgb_img.copy()

circles = cv2.HoughCircles(gray_img, cv2.HOUGH_GRADIENT, 1, 100, param1=100, param2=30, minRadius=50, maxRadius=100)
print(circles)
circles = np.uint16(np.around(circles))
print(circles)
for i in circles[0, :]:
    cv2.circle(new_img, (i[0], i[1]), i[2], (255, 0, 0), 10)
    # 圆心
    cv2.circle(new_img, (i[0], i[1]), 2, (255, 0, 0), 10)

plt.subplot(121)
plt.imshow(rgb_img)
plt.title('img')
plt.axis('off')

plt.subplot(122)
plt.imshow(new_img)
plt.title('rst')
plt.axis('off')
plt.show()