Notes for the course "3D Reconstruction of Computer Vision (Sfm and SLAM Core Algorithms)" taught by Beiyou teacher Lu Peng
3. Single View Geometry
3.1 2D
straight line
Use l = ( a , b , c ) T l=(a,b,c)^T for a straight line on the planel=(a,b,c)T to represent, pointx = ( x , y , 1 ) T x=(x,y,1)^Tx=(x,y,1)T , the point on the line:
l T x = 0 l^Tx=0lTx=The normal vector of the 0
line: n ⃗ = ( b , − a ) \vec n=(b,-a)n=(b,− a ) The direction vector of the straight line:
d ⃗ = ( a , b ) \vec d=(a,b)d=(a,b ) Two straight linesl , l ′ l,l'l,l′ intersection point:
x = l × l ′ x=l\times l'x=l×l′
point line at infinity
Two parallel lines intersect at the point at infinity, the straight line ll can be obtainedInfinity point of l
: x ∞ = ( b , − a , 0 ) T x_\infty=(b,-a,0)^Tx∞=(b,−a,0)T
x ∞ = l × l ′ = ( a , b , c 1 ) × ( a , b , c 2 ) = ( b c 2 − b c 1 , − ( a c 2 − a c 1 ) , a b − a b ) = ( b , − a , 0 ) x_\infty=l\times l'=(a,b,c_1)\times(a,b,c_2)=(bc_2-bc_1,-(ac_2-ac_1),ab-ab)=(b,-a,0) x∞=l×l′=(a,b,c1)×(a,b,c2)=(bc2−bc1,−(ac2−ac1),ab−ab)=(b,−a,0)
All points at infinity have the shape ( x , y , 0 ) T (x,y,0)^T(x,y,0)T , the resulting infinite line is:
l ∞ = ( 0 , 0 , 1 ) T l_\infty=(0,0,1)^Tl∞=(0,0,1)T
3.2 3D
flat
Plane Π = ( a , b , c , d ) T \Pi=(a,b,c,d)^T in 3-dimensional spacePi=(a,b,c,d)T,点 x = ( x , y , z , 1 ) T x=(x,y,z,1)^T x=(x,y,z,1)T , a point on the plane:
Π T x = 0 \Pi^Tx=0PiTx=The normal vector of the 0
plane: n = ( a , b , c ) T n=(a,b,c)^Tn=(a,b,c)T
straight line
A straight line in three-dimensional space can be obtained by intersecting two planes. This representation is a bit cumbersome. Here we only care about the direction of the straight line. Use the direction of the straight line ddd to represent straight linelll:
d = ( a , b , c ) T d=(a,b,c)^T d=(a,b,c)T
infinity
straight line llThe infinity point of l
is: l ∞ = ( a , b , c , 0 ) T l_\infty=(a,b,c,0)^Tl∞=(a,b,c,0)T
hePoints on l can be expressed as x = ( a λ , b λ , c λ , 1 ) T x=(a\lambda,b\lambda,c\lambda,1)^Tx=( a λ ,b l ,c l ,1)T,即 x = ( a , b , c , 1 λ ) T x=(a,b,c,\frac{1}{\lambda})^T x=(a,b,c,l1)T
x ∞ = lim λ → ∞ ( a , b , c , 1 λ ) T = ( a , b , c , 0 ) T x_\infty=\lim_{\lambda\rightarrow\infty}(a,b,c,\frac{1}{\lambda})^T=(a,b,c,0)^T x∞=limλ→∞(a,b,c,l1)T=(a,b,c,0)T
The infinite surface formed by all infinite points:
Π ∞ = ( 0 , 0 , 0 , 1 ) T \Pi_\infty=(0,0,0,1)^TPi∞=(0,0,0,1)T
3.3 Vanishing points and vanishing lines
In the camera coordinate system, the line llThe direction of l d = ( a , b , c ) T d=(a,b,c)^Td=(a,b,c)T , whose infinity point isx ∞ = ( a , b , c , 0 ) T x_\infty=(a,b,c,0)^Tx∞=(a,b,c,0)T , the point projected on the image becomesthevanishing pointvvv:
v = K d v=Kd v=K d d = K − 1 v ∥ K − 1 v ∥ d=\frac{K^{-1}v}{\|K^{-1}v\|}d=∥K−1v∥K- 1 v
v = M x ∞ = K [ I , 0 ] [ a b c 0 ] = K [ a b c ] = K d v=Mx_\infty=K[I,0]\left[\begin{array}{c}a\\b\\c\\0\end{array}\right]=K\left[\begin{array}{c}a\\b\\c\end{array}\right]=Kd v=Mx∞=K[I,0]⎣⎢⎢⎡abc0⎦⎥⎥⎤=K⎣⎡abc⎦⎤=Kd
Plane Π \PiThe normal vector of Π isnnn , its infinity linel ∞ l_\inftyl∞The projection on the image is the shadow line lh l_hlh:
n = K T l h n=K^Tl_h n=KTlhThe vanishing line is composed of vanishing points.
l ∞ l_\inftyl∞Point on P = ( a , b , c , 0 ) TP=(a,b,c,0)^TP=(a,b,c,0)The vanishing point vv of T on the imagev, l h T v = 0 l_h^Tv=0 lhTv=0
所以lh TK d = 0 l_h^TKd=0lhTKd=0
planeΠ \PiDirection ddof a line on Πd are both perpendicular tonnn,有 n T d = 0 n^Td=0 nTd=0
⇒ n = K T l h □ \Rightarrow n=K^Tl_h\square ⇒n=KTlh□
3.4 Single View Reconstruction
Two straight lines d 1 , d 2 d_1,d_2d1,d2The included angle is θ \thetaθ , shadow vanishing pointsv 1 , v 2 v_1,v_2v1,v2,令ω = ( KKT ) − 1 \omega=(KK^T)^{-1}oh=(KKT)− 1:
cos θ = v 1 T ω v 2 v 1 T ω v 1 v 2 T ω v 2 \cos\theta=\frac{v_1^T\omega v_2}{\sqrt{v_1^T\omega v_1 }\sqrt{v_2^T\omega v_2}}cosi=v1Tωv1v2Tωv2v1Tωv2
cos θ = d 1 T d 2 ∥ d 1 T d 2 ∥ = ( K − 1 v 1 ) T ∥ K − 1 v 1 ∥ K − 1 v 2 ∥ K − 1 v 2 ∥ = v 1 K − TK − 1 v 2 v 1 K − TK − 1 v 1 v 2 K − TK − 1 v 2 = v 1 T ω v 2 v 1 T ω v 1 v 2 T ω v 2 \cos\theta=\frac{d_1 ^Td_2}{\|d_1^Td_2\|}=\frac{(K^{-1}v_1)^T}{\|K^{-1}v_1\|}\frac{K^{-1} v_2}{\|K^{-1}v_2\|}=\frac{v_1K^{-T}K^{-1}v_2}{\sqrt{v_1K^{-T}K^{-1}v_1 }\sqrt{v_2K^{-T}K^{-1}v_2}}=\frac{v_1^T\omega v_2}{\sqrt{v_1^T\omega v_1}\sqrt{v_2^T\omega v_2 }}cosi=∥d1Td2∥d1Td2=∥K- 1 v1∥(K- 1 v1)T∥K- 1 v2∥K- 1 v2=v1K−TK- 1 v1v2K−TK- 1 v2v1K−TK- 1 v2=v1Tωv1v2Tωv2v1Tωv2
When the line is vertical, cos θ = 0 ⇒ \cos\theta=0\Rightarrowcosi=0⇒
v 1 T ω v 2 = 0 v_1^T\omega v_2=0 v1Tωv2=0
single vision calibration
Find the image of three mutually perpendicular planes in space (general houses have this property), and find the shadow vanishing points v 1 , v 2 , v 3 of three mutually perpendicular lines on the image v_1,v_2,v_3v1,v2,v3,则得到方程:
{ v 1 T ω v 2 = 0 v 2 T ω v 3 = 0 v 3 T ω v 1 = 0 \left\{\begin{array}{l}v_1^T\omega v_2=0\\ v_2^T\omega v_3=0\\ v_3^T\omega v_1=0 \end{array}\right. ⎩⎨⎧v1Tωv2=0v2Tωv3=0v3Tωv1=0If the camera has zero tilt and square pixels, then KKK has three degrees of freedom, soω \omegaω has three degrees of freedom, then the equations can be solved to obtain the internal parameters of the camera.
single vision reconstruction
Get KKAfter K , you can find a planar shadow linelh l_hlh, by n = KT lhn=K^Tl_hn=KTlhThe three-dimensional information of the plane can be recovered.
Single-view reconstruction needs to know some prior information, such as parallel lines and vertical lines, and these need to be manually marked on the image, which is suitable for scenes with a large number of parallel lines.