IROS2022 | Laser Point Cloud Processing in Snow Environment

原文：LiSnowNet: Real-time Snow Removal for LiDAR Point Cloud

作者：Ming-Yuan Yu , Ram Vasudevan , and Matthew Johnson-Roberson

Conference: IROS2022

LiDAR is an active sensor that emits pulses of light and uses the returned pulses to calculate the distance to surrounding objects. This feature enables lidar to perform detailed 3D measurements of the surrounding scene no matter it is day or night. Such measurements are readily found in many datasets today, such as frame-by-frame point clouds, which are typically sampled at 10Hz and have been used for 3D object detection, semantic segmentation, and mapping.

Although lidar provides more accurate 3D measurements compared to mmWave radar, the measurements are easily degraded in harsh weather conditions. Unlike millimeter-wave radar, which can see through fog, snow, and raindrops, lidar is greatly affected by particles in the air. Especially during snowfall, the pulsed signal hits the snowflakes and gets a ghosted measurement back to the sensor, as shown in Figures 1a and 1b. For applications such as map building, it is critical to remove these measurements and reveal the underlying geometry of the scene.

This article proposes a real-time processing network for lidar point clouds in snowy environments, which has improved performance in denoising and is 52 times faster than the current SOTA algorithm.

01 method

1.1 Preprocessing

In the first step, the point cloud is converted into a distance image. Given a point in the kth point cloud in the dataset $p=[x\ \ y\ \ z]^T \in \mathbb{R}^3$ and its corresponding intensity value i∈[0,1], we can obtain the following values.

d is the distance from the point to the lidar center, ϕ is the inclination, and ψ is the azimuth.

Through the inclination and azimuth angles in the discrete lidar field of view (FOV), we can project each point in the point cloud onto spherical coordinates, and a distance image can be obtained, where is the index $I_k \in \mathbb {R}_{+}^{h\times w \times 2}$ frame $k \in \mathbb{Z}$ , $h$ is the vertical resolution, and $w$ is the horizontal resolution.

The second step is to compress the distance image to an appropriate ratio. As shown in Figure 1, a very critical issue is that ghost measurements are mainly concentrated within a distance of 25 meters, but the maximum distance of lidar can easily exceed 150 meters.

We need to enhance the importance of the points around the vehicle while maintaining the order of the points, because the intensity value of the snowflakes is almost 0, and the scene usually has a positive intensity value, so the contrast between the noise and the scene can be improved .

Finally, to be clear, not every pixel has a corresponding value, because some direction points are missing, such as the sky and some transparent points.

1.2 Network structure

The network structure of this paper is based on the MWCNN of reference [12] and some key points have been modified.

First, replace all convolutional layers with two ring convolutional layers with residual modules; then place a dropout layer after the first ReLU activation function of each residual module to normalize the network.

Finally, drastically reduce the number of channels. Finally, the designed network can generate the corresponding residual graph $\Delta _{k}:=f_{\theta }(\widetilde{I}_k)$ , which can satisfy

1.3 Loss function

Set to $\mathcal{F}:\mathbb{R}^{h \times w \times 2} \rightarrow \mathbb{R}^{h \times w \times 2}$ the truth value of the FFT, $\mathcal{W}:\mathbb{R}^{h \times w \times 2} \rightarrow \mathbb{R}^{h \times w \times 2}$ which is based on Haar's DWT. Three new loss functions can then be designed:

The total loss function is

02 results

As shown in the table, compared to the baseline, the network proposed in this paper can have the same or even better performance in denoising; when the recall rate is 0.0310 lower than DSOR, the accuracy and IoU are significantly higher than all other methods. . The speed is 52 times faster than the DSOR method and 158 times faster than the DROR method. Considering that the sampling frequency of lidar is generally 10Hz, this method should be the best solution for real-time work.

03 Conclusion

This paper proposes a deep convolutional neural network - LiSnowNet - specially designed for point cloud denoising under severe weather conditions. This network can be trained without any labeled data, can process 100,000 points in 7ms compared to the state-of-the-art method, and works well in denoising. Can improve the quality of downstream tasks, such as map building in snowy weather.

recommend:

1. Download | 101 NeRF high-quality paper recommendations (including project codes)

2. How does NeRF quickly build a 3D scene?

3. NeRF-based 3D scene reconstruction and understanding