ICLR2023 Oral | Noise2NoiseMapping: Learning signed distance functions from noisy point clouds

Title: Learning Signed Distance Functions from Noisy 3D Point Clouds via Noise to Noise Mapping

Paper: https://github.com/mabaorui/Noise2NoiseMapping/blob/master/pdf/learning_signed_distance_funct.pdf

guide

Learning signed distance functions (SDFs) from 3D point clouds is an important task in 3D computer vision. However, current methods still struggle to learn SDFs from noisy point clouds in the absence of ground truth signed distances, point normals, or clean point clouds. To overcome this challenge, researchers propose to learn SDFs via noise-to-noise mapping, which does not require any clean point cloud or ground truth supervision for training. The innovation of the paper lies in the noise-to-noise mapping , which can infer a highly accurate SDF of a single object or scene from multiple or even a single noisy point cloud observation . The paper's new learning approach is supported by modern lidar systems that capture multiple noisy observations per second. The paper achieves this goal through a new loss function that performs statistical inference on point clouds and maintains geometric consistency despite point clouds being irregular, unordered, and between noisy observations. There is no dot correspondence . Evaluations under widely used benchmarks show the advantages of the proposed method over existing methods in surface reconstruction , point cloud denoising and upsampling .

background

The essence of the Signed Distance Function (SDF) is to store the closest distance from each point to the surface of the geometry . A positive value of SDF indicates that the point is outside the surface of the geometry, and a negative value indicates that the point is inside the surface of the geometry. Through SDF, the surface of the geometry can be divided, so that the value of the point outside the model surface is greater than 0, and the value of the point inside the model surface is less than 0:

Raw point clouds are discrete and noisy, which is not friendly to downstream applications such as virtual reality and augmented reality where clear surfaces are required. Existing deep learning-based methods usually require additional constraints or noise-free assumptions when learning the SDF of noisy point clouds , which limits their accuracy and applicability. These requirements have a significant impact on learning accurate SDFs from noisy point clouds, either due to poor generalization or inability to denoise. Therefore, learning SDFs from noisy point clouds without clean or ground-truth supervision remains challenging.

contribute

The contributions of this paper are as follows:

1. The paper proposes a method for learning signed distance functions (SDFs) from noisy point clouds without requiring accurate signed distances, point normals, or clean point clouds as training data.

2. The paper proves the effectiveness of statistical inference using Earth Mover's Distance (EMD) through noise-to-noise mapping , and proves it through the proposed new loss function, even if the 3D point cloud is irregular, disordered and does not have point correspondence.

3. On widely used benchmarks, the proposed method achieves SOTA in surface reconstruction, point cloud denoising and upsampling for various shapes and scenes.

method

The goal of the paper is to **learn a signed distance function (SDF) of a denoised 3D shape or scene from a set of noisy 3D point cloud observations,** without requiring accurate signed distances, point normals or Clear point cloud. The paper learns SDF by training a neural network that takes as input a noisy point cloud and performs statistical inference through a noise-to-noise mapping. The paper's loss function not only minimizes the distance between the denoised point cloud and other noisy point clouds, but also constrains the accuracy of the learned SDF through regularization of geometric consistency. Through this learning, downstream tasks such as surface reconstruction, point cloud denoising, and point cloud upsampling can be performed.

Reducing Noise

Drawing on the conclusion of "Noise2Noise" (Lehtinen et al., 2018) in 2D image denoising: the paper proposes to use the noise point cloud ( $N_i$) to estimate the point cloud without noise , according to a loss function $L$ looks for the target with the smallest average deviation from the noisy point cloud, so thatthe input does not require the ground truth without noise:

For 2D images, multiple corrupted observations have pixel correspondences. This leads to an assumption that all noisy observations at the same pixel location are random realizations of the distribution around clean pixel values . However, this assumption is not valid for point clouds. This is also the reason why TotalDenoising (Casajus et al., 2019) believes that the above formula cannot be used for point cloud denoising, because the noise in 3D point cloud is global . In contrast, the paper found the exact opposite. The paper believes that the above formula can still be used to reduce the noise in the 3D point cloud, the key lies in how to define the distance measure $L$ , **This is considered one of the contributions of this paper.

Another problem the paper faces is how to learn SDFs through point cloud denoising.

Denoising Function F

In order to learn point cloud $NSDFs$ f θ f_{\theta}$f_{}$, the paper hopes that the denoising process can also perceive NNA signed distance field around$N.$The paper defines a denoising function$F$ , inspired by the NeuralPull method. This function utilizes the signed distance$d=f_{}(\mathbf{n},c)$and gradient$\nabla f_{}(n,c)$to point cloud$N$ 's noise$n$ pulls towards the surface of the point cloud, thereby denoising the point cloud:

So as to get a point cloud $N^{\prime }=F(N,f_{})$ . But one remaining problem is how to constrain$N^{\text{'}}$ converges to the undamaged (noise-free) surface.

Distance Metric L

Papers on Distance Metrics $L$ conducted research and summarized the following theorems:

Theorem 1: Suppose there exists a clean point cloud $G$ , passed to$Each\; point\; of\; G$ samples noise and corrupts it into noisy observationsS = { N i } S = \{N_i\}S={ Ni​} . If EMD is taken as the distance metric LLdefined in$L$ , and byminimizing$G^{\text{'}}$ withSSEMD between each noisy observation in$S$ to learn a point cloud$G^{\prime }$ , then$G^{\prime }$ will converge to a clean point cloud$G$，即$L(G,G0)=0$。

The paper believes that the one-to-one correspondence $\phi$ plays an important role in the denoising process in statistical inference. This is very similar to the pixel correspondence between noise images in Noise2Noise, although point clouds are irregular, unordered, and have no spatial correspondence between points in different observations. Figure 4(a) below is the noise point cloud Ni, and Figure 4(b) shows that the CD-optimized point cloud$G^{\prime }$ is still noisy, while the EMD-optimized point cloud$G^{\text{'}}$ Very clean.

Geometric Consistency

In order to solve the problem of 3D point cloud denoising, the author introduces the concept of geometric consistency. Near a query point, through the signed distance function (SDF) $f_{}$The shortest distance for prediction should be consistent with the shortest distance calculated from the denoised point cloud . Therefore, they proposed a geometric consistency inequality, requiring that the absolute value of the predicted signed distance at the query point should be less than or equal to the minimum distance between the query point and the denoised point cloud.

The introduction of geometric consistency enables the adjustment of SDF predictions by geometric constraints to ensure that the point cloud is correctly pulled back to the surface. This approach improves point cloud denoising and provides more accurate SDFs in surface reconstruction. By introducing geometric consistency, the authors address the insensitivity to correct surfaces in previous methods, resulting in improved point cloud denoising and reconstruction performance.

Loss Function

By geometric consistency, the wrong signed distance field shown in Figure 3(b) can be penalized and the correct distance field in Figure 3© can be encouraged. Therefore, the paper regards geometric consistency as a regularization term R, and combines it with Equation (5) and Equation (6) to form our objective function as follows:

experiment

Point Cloud Denoising

As shown in Table 1, the paper uses two benchmark datasets (PUNet and PointCleanNet) for evaluation and compares with a variety of other methods, including traditional point cloud denoising methods and deep learning-based methods. The paper uses L2 Chamfer distance and point-to-grid distance as evaluation indicators to evaluate the denoising performance at different noise levels. Experimental results show that the method of the ** paper significantly outperforms other methods in both supervised and unsupervised settings, exhibiting state-of-the-art denoising performance. **As shown in the figure above, the comparison of error maps further proves the superiority of the paper's method .

Point Cloud Upsampling

As shown in the table above, the authors evaluate their learning method on point cloud upsampling using the PU dataset. They compare with PU-Net and SBP methods by adding noise and denoising the sparse point cloud. The results show that their method is able to more accurately restore surfaces on point clouds with good results.

Surface Reconstruction for Shapes

The paper first reports the surface reconstruction performance of the paper's method on the test set of 13 categories of ShapeNet. The numerical comparison in Table 3 shows the surface reconstruction accuracy of the paper's method on 13 categories, indicating that it achieves SOTA in surface reconstruction . Although the paper method does not require ground truth supervision, the paper method outperforms supervised methods such as SAP, COcc and IMLS in surface reconstruction . In the visual comparison in Figure 8, the paper further demonstrates our superiority in complex geometry reconstruction.

The comparison in Table 4 shows that the paper method can extract more accurate surfaces from noisy point clouds without training set, ground truth supervision or even multiple noisy point clouds. By comparing the error maps under FAMOUS, it can be seen that the statistical inference and geometric regularization on the point cloud make the generated surface more accurate.

A comparison of Figure 10 and Figure 11 shows that our method produces more accurate surfaces on both rigid and non-rigid shapes without missing parts.

The paper evaluates the paper's method under the real scene scan dataset . The numerical comparison results in Table 7 show that our method significantly outperforms the existing state-of-the-art methods. Figure 12 further demonstrates that our method can generate smoother, more geometrically detailed surfaces.

Summarize

The paper introduces a method for learning directed distance fields (SDFs) from noisy point clouds via noise-to-noise mapping. The paper explores the feasibility of learning SDFs from multiple noisy point clouds or even a single noisy point cloud without ground truth directed distances, point normals or clean point clouds. The paper's noise-to-noise mapping enables statistical inference in point clouds, even if there is no spatial correspondence between different noisy point clouds. The key insight of the paper in statistical inference is the use of EMD as a metric. Through the ability of statistical reasoning, the paper successfully recovers surfaces from noisy point clouds by learning highly accurate SDFs. We evaluate our method on synthetic datasets or real scan datasets, for different applications of shapes or scenes. The performance of the paper in different applications demonstrates the effectiveness of our method, reaching the state-of-the-art.