Original | Wen BFT robot
01
Color mode with full frame space variation
The main disadvantages of sequential projection techniques include the inability to capture 3D objects in dynamic motion or living objects such as human body parts. We have now proposed several single-lens 3D surface imaging techniques that exploit color information in projection mode or unique encoding schemes to obtain a full-frame 3D image of the (x, y, z) coordinates of every visible point in the scene by acquiring only one image of the object under color-mode illumination.
Figure 1 Rainbow 3D camera
1.1 Rainbow 3D Camera
Unlike conventional stereo images, which must extract corresponding features from a pair of stereo images to calculate depth values, rainbow 3D cameras project spatially varying wavelength illumination onto object surfaces. The fixed geometry of the rainbow light projector establishes a one-to-one correspondence between the projection angle θ of the light plane and a specific spectral wavelength λ, providing easily identifiable landmarks at each surface point. In the case of known baseline B and known viewing angle α, the 3D distance value corresponding to each pixel can be calculated using a simple triangulation principle, and a full-frame 3D distance image can be obtained in a single snapshot at the frame rate of the camera (30 frames/second or faster).
Figure 2 A continuously changing color coding scheme: three additive primary color sawtooth
1.2 Continuously changing color coding
Various continuously changing color patterns can be combined to encode spatial location information. For example, we could build a pattern of intensity variations for each color channel of a projector so that, when added together, the patterns in the individual color channels form a continuously varying color pattern.
Figure 2 shows an example of intensity variation patterns for three additive primary color channels. When added together, they create a rainbow-like pattern of color projections. Note that this type of color pattern does not necessarily follow a linear variation of the color spectrum (wavelength). However, since the contribution ratio of each color channel is known, the decoding scheme is easy to derive and implement.
02
Stripe index (single)
The fringe index is necessary to achieve a robust 3D surface reconstruction because the order in which the fringes are observed is not necessarily the same as the order in which the fringes are projected. This is due to the inherent parallax in triangulation-based 3D surface imaging systems, and the acquired image may be missing fringes due to the occlusion of object 3D surface features. We now present some representative stripe indexing techniques.
2.1 Strip index using color
Color image sensors typically have three independent acquisition channels, one for each spectral band. A linear combination of the values of these color components can produce an infinite number of colors. Three 8-bit channels can represent 224 different colors. Such rich color information can be used to improve 3D imaging accuracy and reduce acquisition time.
For example, using color for fringe indexing in projected mode (Figure 3) can help alleviate the ambiguity problems faced by phase-shifting or multi-stripe techniques using monochrome mode. This color-coding system enables real-time 3D surface imaging capabilities.
It is also possible to encode multiple patterns into a single color projected image, with each pattern having a unique color value in the color space. To reduce the decoding error rate, you can choose a set of colors where each color has the greatest distance from any other color in the set. The maximum number of colors in this set is limited to the distance between colors that produces the least crosstalk in the acquired image.
Figure 3 Strip index using color
2.2 Segmented index using segmented mode
In order to distinguish different stripes, some unique fragment patterns can be added in each stripe (Fig. 4), so that when doing 3D reconstruction, the algorithm can use the unique fragment patterns of each stripe to distinguish them.
The indexing method proposed in [2] is interesting and clever, but it is only suitable for 3D objects with smooth and continuous surfaces, when the pattern distortion caused by the surface shape is not severe. Otherwise, it may be difficult to recover unique fragment patterns due to pattern deformations and/or discontinuities in the object surface.
Figure 4 Striped index using segmented mode
2.3 Stripe indexing using repeating grayscale patterns
If more than two intensity levels are used, the intensity levels of the stripes can be arranged such that any set of stripes (a sliding window of N bars) has a unique intensity pattern over a length. For example, if using three gray levels (black, gray and white), the pattern can be designed as (Fig. 5)
Figure 5 Stripe indexing using a repeating grayscale pattern
The pattern matching process starts with the correlation of acquired image intensities with projected intensity patterns. Once a match is found, a further search is performed for sub-grayscale sequence matches, such as the three-letter sequence WGB, GWB, etc.
2.4 Stripe index based on De Bruijn sequence
On an alphabet of size k, a De Bruijn sequence of rank n is a cycle of words where each of kn words of length n occurs exactly once as we progress along the cycle.
Figure 6 shows a simple example of a De Bruijn circle with n = 3, k = 2 (letter is {0,1}). As we run along this loop (either clockwise or counterclockwise), we will encounter 23 = 8 three-digit patterns 000, 001, 010, 011, 100, 101, 110, 111 each of them exactly once.
There are no repeated three-digit patterns in the sequence. In other words, no subsequence is related to any subsequence in the De Bruijn sequence. This unique feature of De Bruijn sequences can be used to construct sequences of stripe patterns with unique local variation patterns and no repetitions. This uniqueness makes pattern decoding easier. A graph associated with a De Bruijn sequence is called a De Bruijn graph.
Figure 6 A simple example of the De Bruyne sequence
We now show an example of using binary combinations of (R, G, B) colors to generate color-indexed stripes based on the De Bruijn sequence.
The maximum number of combinations of three colors is 8 (=23). Since (0,0,0) is not going to be used, there are only seven possible colors.
This problem can be solved by constructing a De Bruijn sequence with k = 7, n = 3. This results in a sequence of 343 stripes. If the number of stripes is too large, the reduced set of De Bruijn sequence can be used by setting k = 5, n = 3. In this case, the number of stripes is reduced to 125.
When constructing a sequence of color-indexed stripes using the De Bruijn technique, there is an important constraint: all adjacent stripes must have different colors. Otherwise, double or triple width streaks appear, confusing the 3D reconstruction algorithm. This constraint can be easily applied by using the exclusive-or operator.
Figure 7 shows a set of results with actual color-indexed fringe patterns. In this sequence of stripes, all adjacent stripes have different colors. Various variants of the De Bruijn technique implementation can be used to generate unique color-indexed, grayscale-indexed or other types of projection modes for 3D surface imaging applications.
Figure 7 Example of color bar index based on De Bruijn sequence (k = 5, n = 3)
references:
[1] Geng J .Structured-light 3D surface imaging: atutorial[J].Advances in Optics & Photonics, 2011, 3(2):128-160.DOI:10.1364/AOP.3.000128.
[2] M. Maruyama and S. Abe, “Range sensing by projecting multiple slits with random cuts,” IEEE Trans. Pattern Anal. Mach. Intell. 15(6), 647–651 (1993)
For more exciting content, please pay attention to the official account: BFT Robot
This article is an original article, and the copyright belongs to BFT Robot. If you need to reprint, please contact us. If you have any questions about the content of this article, please contact us and we will respond promptly.