# Neural Poisson Surface Reconstruction:
Resolution-Agnostic Shape Reconstruction from Point Clouds

Hector Andrade-Loarca∗
Ludwig-Maximilians-Universität in Munich
<EMAIL_ADDRESS>Julius Hege∗
Ludwig-Maximilians-Universität in Munich
<EMAIL_ADDRESS>Daniel Cremers
Technical University of Munich
<EMAIL_ADDRESS>Gitta Kutyniok
Ludwig-Maximilians-Universität in Munich
<EMAIL_ADDRESS>
###### Abstract

We introduce Neural Poisson Surface Reconstruction (nPSR), an architecture for
shape reconstruction that addresses the challenge of recovering 3D shapes from
points. Traditional deep neural networks face challenges with common 3D shape
discretization techniques due to their computational complexity at higher
resolutions. To overcome this, we leverage Fourier Neural Operators to solve
the Poisson equation and reconstruct a mesh from oriented point cloud
measurements. nPSR exhibits two main advantages: First, it enables efficient
training on low-resolution data while achieving comparable performance at
high-resolution evaluation, thanks to the resolution-agnostic nature of FNOs.
This feature allows for one-shot super-resolution. Second, our method
surpasses existing approaches in reconstruction quality while being
differentiable and robust with respect to point sampling rates. Overall, the
neural Poisson surface reconstruction not only improves upon the limitations
of classical deep neural networks in shape reconstruction but also achieves
superior results in terms of reconstruction quality, running time, and
resolution agnosticism.

††$*$ Equal contribution.

##  1 Introduction

Shape reconstruction, the process of creating a three-dimensional mesh of an
object from 2D images or point clouds plays a crucial role in computer vision.
It is an active research area with significant applications in robotics,
virtual reality, and autonomous driving. A common subproblem in this field is
the task of reconstructing a surface from noisy point clouds, which are
typically acquired using techniques such as photogrammetry (multiple images)
or depth sensors.

A plethora of shape reconstruction algorithms have been developed, as surveyed
in [1]. Recently, deep learning approaches trained on large datasets of 3D
objects, such as Convolutional Occupancy Networks [16] and POCO [2], have
emerged, promising improved reconstruction quality. However, these methods
typically do not generalize well beyond the training distribution [20].

Another challenge faced by deep learning approaches is scalability, especially
when dealing with complex models at high resolutions. Convolutional neural
networks have achieved widespread success in computer vision and can
straightforwardly generalize to three dimensions. However, they operate on
regular grids which limits their ability to represent fine details accurately.

To overcome this limitation, various strategies have been proposed. Some
solutions involve working with irregular grids [16], representing shapes as
implicit functions [4], or employing multi-grid approaches in a coarse-to-fine
manner [19] or sliding-window fashion [12].

Our aim in this work is to enhance reconstruction quality by integrating deep
learning into the well-established Poisson surface reconstruction method,
thereby leveraging robustness and generalization capabilities. In order to
achieve this, we introduce a neural network architecture whose backbone is a
_Fourier Neural Operator_ [10]. The choice is motivated by Fourier Neural
Operators’ ability to approximate partial differential equation solutions.

###  1.1 Poisson Surface Reconstruction

We begin by introducing the problem of Poisson surface reconstruction. In this
setting, we are given a point cloud sampled from the surface of a (three-
dimensional) object and the corresponding normal vectors, and we want to
recover the indicator function that implicitly represents the shape of such an
object. Assuming that we have access to the full normal vector field
$V:\Omega\subset\mathbbm{R}^{3}\rightarrow{\mathbbm{R}^{3}}$, our goal is to
reconstruct/learn a function $\chi:\Omega\rightarrow\\{0,1\\}$ such that

$\displaystyle\nabla\chi(x)=V(x)\quad\text{for all }x\in\Omega,$ (1.1)

where $\chi$ denotes the indicator function representing the object. Taking
the divergence of (1.1), we obtain the Poisson equation

$\displaystyle\Delta\chi(x)=\nabla\cdot V(x)\quad\text{for all }x\in\Omega.$
(1.2)

Notice that the indicator function is unique up to a constant. The uniqueness
of a solution can be ensured by imposing sparse constraints. Here, the domain
$\Omega$ can be chosen to be Lipschitz and large enough so that the object is
fully contained in $\Omega$, i.e., $\chi(x)=0$ on $\partial\Omega$. Thus, we
face a boundary value problem given by (1.2) supplemented with homogeneous
Dirichlet boundary conditions. This is an ill-posed inverse problem since for
any given finite point cloud there exists multiple possible solutions.

###  1.2 Fourier Neural Operator

Fourier Neural Operators (FNOs) were first introduced by Li et al. in [10] as
a technique to learn the solution of PDEs in the form of operators. In general
terms, the aim of an FNO is to learn a mapping
$\mathcal{N}:\mathcal{A}\rightarrow\mathcal{U}$ between two infinite-
dimensional function spaces by using a finite collection of observations
$\\{(a_{i},u_{i})\\}_{i=1}^{N}$, where $\mathcal{N}(a_{i})=u_{i}$ for all $i$.

######  Definition 1.1 (Fourier Neural Operator, [10]).

A _Fourier Neural Operator_ is given by the iterative architecture

$\displaystyle a\rightarrow Pa$ $\displaystyle=:v_{0}\mapsto\cdots\mapsto
v_{T}\mapsto Qv_{T}=:u,$ (1.3) $\displaystyle v_{t+1}(x)$
$\displaystyle:=\sigma(Wv_{t}(x)+\mathcal{F}^{-1}\left(R\cdot(\mathcal{F}v_{t})\right))(x)$

for $t=0,\dots,T-1,$ where $\mathcal{F}$ denotes the Fourier transform,
$R\in\mathbb{C}^{k_{\text{max}}{\times}d_{v}{\times}d_{v}}$ represents the
weight tensor, ${P:\mathcal{A}\rightarrow\mathbb{R}^{d_{v}}}$ is a _lifting
operator_ , $Q:\mathbb{R}^{d_{v}}\rightarrow\mathcal{U}$ is a _projection
operator_ , $W:\mathbb{R}^{d_{v}}\rightarrow\mathbb{R}^{d_{v}}$ is a linear
transformation, and ${\sigma:\mathbb{R}\rightarrow\mathbb{R}}$ is a nonlinear
activation function applied component-wise.

Fig. 1 shows the schematic representation of the architecture of a Fourier
Neural Operator (FNO) for four Fourier blocks. The aim of the FNO is to learn
the weights $R$. One can interpret $R$ as the Fourier representation of a
convolutional kernel, where the formula
$\mathcal{F}^{-1}\left(R\cdot(\mathcal{F}v_{t})\right)$ in (1.3) is obtained
by the _Fourier convolution theorem_. Now, in order to implement this method,
we need to discretize the layer (1.3).

###  1.3 Discretization

We assume that our domain $\Omega\subset\mathbb{R}^{3}$, where the input
function $a\in\mathcal{A}$ is evaluated at, is discretized into
$d_{a}\in\mathbb{N}$ points and we wish to evaluate the output function at a
higher resolution with $d_{u}\in\mathbb{N}$ points. Thus, the lifting and
projection operator can be represented by the matrices
$P\in\mathbb{R}^{d_{v}{\times}d_{a}}$ and
$Q\in\mathbb{R}^{d_{u}{\times}d_{v}}$, respectively. Furthermore, in (1.3) we
have, $v_{t}\in\mathbb{R}^{d_{a}{\times}d_{v}}$ and
$\mathcal{F}(v_{t})\in\mathbb{C}^{d_{a}{\times}d_{v}}$. The key advantage of
Fourier layers (over traditional convnets) is that they are discretization-
invariant since parameters are learned directly in Fourier space. Resolving
the functions in physical space simply amounts to projecting on the basis of
wave functions which are well-defined everywhere in the space. This allows us
to transfer the learned network among resolutions. If implemented with
standard FFT, then it will be restricted to uniform mesh, but still
independent of discretization, therefore _resolution-agnostic_. This feature
of the Fourier Neural Operator is crucial in the context of shape since 3D
data requires a lot of memory.

In this work, we also assume the discretization to be uniform, thus
$\mathcal{F}$ can be replaced by the Fast Fourier Transform (FFT).
Furthermore, following [10], we truncate the higher Fourier modes of the FFT
by keeping the first $k_{\text{max}}$ Fourier modes. The multiplication by the
weight tensor $R\in\mathbb{C}^{k_{\text{max}}{\times}d_{v}{\times}d_{v}}$ is
then given by

$\displaystyle\left(R\cdot(\mathcal{F}v_{t})\right)_{k,l}=\sum_{j=1}^{d_{v}}R_{k,l,j}(\mathcal{F}v_{t})_{k,j},$
(1.4)

$k=1,\ldots,k_{\text{max}},j=1,\ldots,k_{\text{max}}.$

##  2 Related Work

###  2.1 Poisson Surface Reconstruction

The use of the Poisson equation to solve a 3D shape reconstruction problem was
first introduced by Kazhdan et al. in 2006 [9]. Later, the same authors
proposed an improvement by adding additional boundary conditions named
_screened Poisson surface reconstruction (SPSR)_ [8].

More recently, Poisson surface reconstruction has been further explored, first
in the form of a differentiable Poisson solver (Peng et al. [17]) which can be
used to either optimize a Poisson surface reconstruction or learn it. In
addition, Hue et al. introduced in 2022 an iterative method that can perform
the reconstruction without the need of normals, the _iterative Poisson surface
reconstruction (iPSR)_ [7]. Their method initializes the normals of a given
point cloud randomly and denoises them by iteratively applying the Poisson
surface reconstruction. Sellán et al. [18] have recently proposed solving a
stochastic form of the Poisson equation using Physics-Informed Neural
Networks, optimizing a neural network to represent an object.

These techniques have shown amazing performance in recovering high-resolution
close-to-perfect reconstructions in high-sampling regimes ($25\,0000$ points).
Yet, in the low sampling regime (around $3\,000$ to $25\,000$ points) they
often generate very sub-optimal reconstructions, lacking the ability to
correctly interpolate surfaces. Furthermore, their computational expense is
significantly influenced by the input resolution, making them inefficient for
high-resolution applications.

###  2.2 Fourier Neural Operators

In the past years, _Fourier Neural Operators (FNOs)_ have emerged as a potent
tool for differentiably solving complex, high-dimensional PDEs [10], thanks to
their training convenience and resolution-agnostic attributes. Given these
advantages, FNOs have been employed in tasks demanding high resolution,
pertaining to traditional PDEs, such as weather modeling [15] and multiphase
flow [21].

Motivated by the poor performance of existing methods in the low-sampling
regime, we introduce an approach that addresses this drawback by using the
ability of FNOs to solve PDEs in challenging scenarios such as low-sampling
and high-resolutions.

##  3 Main Contribution

We present a new approach for solving the problem of 3D shape Poisson surface
reconstruction. Our method outperforms existing methods, particularly in low
sampling scenarios where the input point cloud consists of approximately
$3\,000$ to $25\,000$ points. In addition, in high-sampling regimes, it
performs as well as the classical method which presents close-to-perfect
reconstruction given sufficiently many points (around $25\,0000$ points in our
case).

Our approach is based on a deep neural network architecture formed by Fourier
blocks and can thus be considered a Fourier Neural Operator, which is used to
solve the underlying Poisson equation. It takes a directed point cloud as
input and outputs a 3D reconstruction in the form of a voxel grid. We further
threshold the output using Otsu’s method [14] and finally perform marching
cubes [11] to extract a mesh.

In addition to outperforming existing approaches, our method also inherits the
one-shot super-resolution capabilities of Fourier Neural Operators. We are
able to train the method on voxel grids of size $64{\times}64{\times}64$ while
evaluating it on grids of size $128{\times}128{\times}128$ with similar
performance, in contrast to classical convolutional neural networks where the
error grows exponentially with the resolution of the input. In essence, our
technique consistently produces superior-quality reconstructions and
demonstrates enhanced robustness when considering variations in point sampling
rate and resolution, surpassing the performance of current methods.

Finally, notably, this study pioneers the application of a Fourier Neural
Operator in computer vision tasks, highlighting its prospective significance
in this field.

##  4 Method

In this section, we will explain in detail all the steps that our method
follows to learn how to compute a watertight mesh from a given oriented point
cloud.

###  4.1 Preparation of the Training Data

To train our model, we utilize the ShapeNet dataset [3] (see
https://shapenet.org/). Since the standard output of a Poisson surface
reconstruction algorithm is a voxel grid of a given dimension, we use the
voxelized meshes provided in ShapeNet to obtain clean ground truths.
Additionally, we generate ground truth data for the oriented point cloud
samples by performing marching cubes [11] on the voxel grids and sampling
points on the resulting surface mesh using the Poisson disk sampling technique
proposed by Yuksel in [23].

Since we extract the meshes using marching cubes, we have access to the
normals of their faces. All sampled points are on some triangle of the ground
truth mesh. We calculate the normals of the sampled points using the average
over the neighboring faces, weighted by their barycentric coordinates.

We trained our method using four different sampling scenarios, with $3\,000$,
$10\,000$, $25\,000$, and $250\,000$ sampling points, all of them with
resolution $64\times 64\times 64$. These experiments demonstrate how our
method outperforms other methods in the low sampling regime
($3\,000$-$25\,000$) while producing close-to-perfect reconstructions in the
high sampling regime ($250\,000$), comparable to other methods, including the
classical Poisson surface reconstruction [9]. In addition, we also demonstrate
resolution-agnosticism by comparing the reconstruction quality when the
resolution is changed. One can find the numerical results and reconstruction
images corresponding to these experiments in Sec. 5.

###  4.2 Architecture and Training Procedure

The full pipeline of our approach is composed of a Fourier Neural Operator
with four 3D Fourier blocks (see Fig. 1) as well as a pre-processing and a
post-processing step. Following each Fourier block, we incorporate GELU
(Gaussian Error Linear Unit, [6]) non-linearities and dropout layers after the
first three Fourier blocks. Furthermore, we employ $20$ frequency modes on the
Fourier transform.

nPSR is trained on a representative subset of $10\,000$ randomly selected
shapes from the ShapeNet dataset, which accounts for a fifth of the full
dataset.The preparation of the data is detailed in Sec. 4.1, outlining our
comprehensive approach. The complete pipeline is visualized in Fig. 2.

To assess our model’s performance, we use the mean squared error on the voxel
grid as the loss function, consistent with prior works on 3D FNOs [10, 22].
Furthermore, we evaluate and test the network on two distinct sets, each
containing $2\,000$ examples. Throughout training, we employ batches of 20
data points for approximately $10\,000$ training steps.

Figure 1: Fourier Neural Operator with four Fourier Blocks, the backbone of
nPSR, where $\mathcal{F}$ represents the Fourier transform.

###  4.3 Data pre-Processing and Prediction post-Processing

As discussed in the previous sections, the input of our method is regarded as
a directed point cloud, consisting of point coordinates and normal directions
associated with each point. The Fourier block (see Fig. 1) we use to construct
our Fourier Neural Operator architecture, nPSR, assumes the input to be
defined in a uniform grid. In order to transform our input point cloud into a
uniform grid representation we perform point rasterization (see [13]) of the
divergence field $\nabla\cdot V$ in (1.2). The point rasterization is done in
two steps, we first rasterize the point locations in a voxel grid of a given
resolution (e.g. $64\times 64\times 64$), where we perform a smoothing step
using a Gaussian filter of standard deviation $\sigma=2$. The second step
consists of incorporating the normal information of every given point to
compute the divergence field with finite differences. Finally, we perform a
smoothing step on the resulting divergence field using the filter from the
first step. The resulting voxel grid representing the divergence field is then
fed into our Fourier neural architecture.

To refine nPSR’s reconstruction, we employ a thresholding step as a post-
processing technique. The threshold value is computed through Otsu’s
thresholding method [14]. Finally, we generate a surface mesh through the
marching cubes algorithm [11]. The complete pipeline is illustrated in Fig. 2.

Figure 2: nPSR, full pipeline.

##  5 Experiments

In this section, we present the numerical results as well as some
reconstruction examples. In addition, we also perform a number of ablation
studies, which are presented in this section as well. All code and
instructions necessary to reproduce our experiments can be found in the
supplementary material.

N. Sampled Points | Method | Chamfer-L1 ($\downarrow$) | F-Score ($\uparrow$) | Norm. Consist. ($\uparrow$) | Runtime (s)
---|---|---|---|---|---
$3\,000$ | SPSR [8] | $0.430$ | $0.552$ | $0.525$ | $0.31$
iPSR [7] | $0.554$ | $0.488$ | $0.465$ | $325.25$
Point2Mesh [5] | $0.644$ | $0.392$ | $0.381$ | $2\,215.64$
SAP [17] | $0.492$ | $0.516$ | $0.495$ | $0.45$
nPSR | $0.081$ | $0.895$ | $0.875$ | $0.52$
$10\,000$ | SPSR [8] | $0.265$ | $0.722$ | $0.715$ | $0.31$
iPSR [7] | $0.372$ | $0.548$ | $0.625$ | $410.41$
Point2Mesh [5] | $0.485$ | $0.522$ | $0.578$ | $3\,015.91$
SAP [17] | $0.338$ | $0.852$ | $0.681$ | $0.45$
nPSR | $0.042$ | $0.954$ | $0.912$ | $0.52$
$25\,000$ | SPSR [8] | $0.230$ | $0.751$ | $0.753$ | $0.31$
iPSR [7] | $0.348$ | $0.583$ | $0.688$ | $452.33$
Point2Mesh [5] | $0.412$ | $0.512$ | $0.623$ | $3\,890.82$
SAP [17] | $0.315$ | $0.88$ | $0.715$ | $0.45$
nPSR | $0.029$ | $0.983$ | $0.976$ | $0.52$
$250\,000$ | SPSR [8] | $0.022$ | $0.954$ | $0.925$ | $0.31$
iPSR [7] | $0.031$ | $0.961$ | $0.930$ | $11\,015.84$
Point2Mesh [5] | $0.649$ | $0.614$ | $0.622$ | $43\,850.11$
SAP [17] | $0.015$ | $0.974$ | $0.971$ | $0.45$
nPSR | $0.008$ | $0.99$ | $0.991$ | $0.52$
Table 1: Reconstruction metrics averaged over all ShapeNet voxel models. Comparisons against the baseline models SPSR [8], iPSR [7], Point2Mesh [5] and SAP [17]. For each metric, the best result is highlighted. nPSR achieves the best performance with a marginal increase in runtime. The performance improvement is around an order of magnitude in terms of Chamfer-L$1$ distance. Resolution (voxels) | Chamfer-L1 ($\downarrow$) | F-Score ($\uparrow$) | Norm. Consist. ($\uparrow$)
---|---|---|---
$32\times 32\times 32$ | 0.024 | 0.992 | 0.983
$64\times 64\times 64$ | 0.027 | 0.986 | 0.980
$128\times 128\times 128$ | 0.029 | 0.983 | 0.976
Table 2: Reconstruction metrics averaged over all ShapeNet voxel models, with
$25\,000$ sampled points. Comparison of reconstruction performance along
different resolutions Figure 3: While all methods work well in high-sampling
regimes, the proposed nPSR significantly outperforms in low-sampling regimes.
Figure 4: Our method is resolution agnostic, performing much better than
existing methods when the resolution increases in inference time.

###  5.1 Results

We conduct experiments with four different sampling rates, $3\,000$,
$10\,000$, $25\,000$, and $250\,000$ sampled points (and normals). In
addition, the performance of our method is also compared on the same data with
other four existing methods, namely, Screened Poisson Surface Reconstruction
(SPSR) [8], Iterative Poisson Surface Reconstruction (iPSR) [7], Point2Mesh
[5] and Shape as Points (SAP) [17]. To assess the quality of reconstructions,
we utilized metrics introduced by Peng et al. [17], which include Chamfer L1
loss (measuring point cloud distances), F-Score (evaluating point estimation
classification accuracy), and Normal Consistency (accounting for normal
orientation). All experiments were performed on the ShapeNet Core dataset
(https://shapenet.org/).

Our method was trained for approximately $1\,000$ epochs on shapes with
resolution $64{\times}64{\times}64$ and evaluated on shapes with resolution
$128{\times}128{\times}128$. The numerical results for the different sampling
rates are presented in Tab. 1. Notably, nPSR consistently outperformed the
alternatives in all sampling rates, with particularly significant improvements
observed in the very low sampling regime ($3\,000$). In contrast, other
methods exhibited poorer performance in such scenarios.

While our approach may not be the fastest (classical SPSR holds that
distinction), it maintains a constant running time relative to the sampling
rate. This is because most of the point cloud processing, such as
rasterization, occurs during pre-processing at the training stage, enabling
straightforward scalability.

We also perform reconstructions for different resolutions, in which the model
was trained on resolution $64{\times}64{\times}64$ with $25\,000$ sampled
points and evaluated on resolutions $32{\times}32{\times}32$ and
$128{\times}128{\times}128$. LABEL:{table:resolution_agnostic} depicts the
performance metrics corresponding to the different resolutions, showing the
robustness of the method against different resolutions (resolution-
agnosticism).

Finally, Fig. 3 showcases three examples of reconstructions at different
sampling rates, visually confirming the superior performance of our method
over the others. In addition, Fig. 4 shows a reconstruction example with
different resolutions, confirming the resolution-agnosticism of the method and
the superiority with respect to other methods.

###  5.2 Ablation Studies

We conducted various ablation studies to optimize our pipeline and model
setup. Initially, the input was computed without incorporating the normal
information, relying solely on rasterization and smoothing of the point cloud,
which led to unsatisfactory performance. This highlighted the crucial role of
normal field information in our model. Subsequently, we experimented with
different parameters for the smoothing Gaussian used in the input, ultimately
discovering that a standard deviation of $\sigma=2$ yields the best results
across all sampling rates.

In the pursuit of identifying the optimal Fourier modes, we tested different
values, finding that 20 Fourier modes achieved the best performance.
Considering the model architecture design, we trained models with varying
numbers of layers. Notably, the configuration with four Fourier blocks and
GELU activation function demonstrated superior performance.

Overall, our ablation studies confirmed the relative optimality of our
pipeline regarding hyperparameter changes, emphasizing the significance of
incorporating normal field information and utilizing specific parameter
settings for the smoothing Gaussian and Fourier modes. The chosen architecture
with four Fourier blocks and GELU activation function exhibited the most
favorable performance, contributing to the effectiveness of our approach.

##  6 Conclusion

This paper presents nPSR, a groundbreaking approach to 3D shape reconstruction
that significantly outperforms state-of-the-art techniques, particularly in
low-sampling scenarios with point clouds ranging from $3\,000$ to $25\,000$
points. Even in high-sampling domains, nPSR competes favorably with other
methods, demonstrating superior capability in generating nearly precise
reconstructions with a substantial data set, encompassing around $250\,000$
points.

The core strength of nPSR originates from its sophisticated deep neural
network architecture that seamlessly integrates Fourier layers, recognized as
Fourier Neural Operators. These layers adeptly address the Poisson equation,
enabling the generation of a voxel grid, which serves as the 3D reconstruction
output. The strategic implementation of Otsu’s thresholding on the
reconstruction, coupled with the utilization of marching cubes to extract a
mesh representation, underscores the robustness and versatility inherent in
nPSR.

One of the standout advantages of nPSR is its one-shot super-resolution
capabilities inherited from Fourier Neural Operators. This innovative feature
facilitates training on low-resolution voxel grids ($64{\times}64{\times}64$)
while ensuring comparable performance during evaluation on higher-resolution
grids ($128{\times}128{\times}128$). This is a marked advancement over
traditional convolutional neural networks, where errors tend to escalate
exponentially with the augmentation of the input dimension.

The contributions of this paper extend beyond 3D shape reconstruction,
encouraging further investigation of Fourier Neural Operators across various
computer vision tasks. The observed resolution agnosticism, though challenged
by the need for a predetermined resolution of training data, presents an
avenue for additional exploration. The requirement to specify the resolution
of training data in advance often leads to a reduction of all training data to
a set resolution, resulting in the loss of fine details within images. This is
a significant challenge shared by similar methods aimed at solving partial
differential equations. However, with sufficient computational resources,
training at a higher resolution using more capable GPUs becomes a feasible
option.

This work alludes to the potential of delving deeper into optimizing nPSR to
enhance its performance and generalization further. The prospect of exploring
alternative architectures, incorporating additional regularization techniques,
or investigating different loss functions appears promising in tackling
specific challenges inherent in shape reconstruction. Particularly captivating
is the notion of extending our approach beyond the confinement of regular grid
representations by employing graph neural operators to manage highly detailed
meshes.

In summary, the results presented in this thesis not only provide a robust
methodology for 3D shape reconstruction but also lay the foundation for a
broader application of Fourier Neural Operators in the broad field of computer
vision. The resolution agnosticism demonstrated here, coupled with the
suggestion of further optimization of nPSR or exploration of alternative
architectures, suggests a positive path forward for advancing shape
reconstruction and other areas of computer vision.

##  7 Acknowledgments

The work of H. Andrade-Loarca was supported by the German Research Foundation
under Grant DFG-SFB/TR 109, Project C09. J. Hege was supported by in part by
the Konrad Zuse School of Excellence in Reliable AI (DAAD) as well as the
German Research Foundation under Grant DFG-SFB/TR 109, Project C09. This work
of D. Cremers was supported by the ERC Advanced Grant SIMULACRON. G. Kutyniok
acknowledges support by the Konrad Zuse School of Excellence in Reliable AI
(DAAD), the Munich Center for Machine Learning (BMBF) as well as the German
Research Foundation under Grant DFG-SFB/TR 109, Project C09.

## References

  * Berger et al. [2017] Matthew Berger, Andrea Tagliasacchi, Lee M Seversky, Pierre Alliez, Gael Guennebaud, Joshua A Levine, Andrei Sharf, and Claudio T Silva.  A survey of surface reconstruction from point clouds.  In _Computer graphics forum_ , pages 301–329. Wiley Online Library, 2017.
  * Boulch and Marlet [2022] Alexandre Boulch and Renaud Marlet.  Poco: Point convolution for surface reconstruction.  In _Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition_ , pages 6302–6314, 2022.
  * Chang et al. [2015] Angel X. Chang, Thomas Funkhouser, Leonidas Guibas, Pat Hanrahan, Qixing Huang, Zimo Li, Silvio Savarese, Manolis Savva, Shuran Song, Hao Su, Jianxiong Xiao, Li Yi, and Fisher Yu.  ShapeNet: An Information-Rich 3D Model Repository.  Technical Report arXiv:1512.03012 [cs.GR], Stanford University — Princeton University — Toyota Technological Institute at Chicago, 2015.
  * Chibane et al. [2020] Julian Chibane, Thiemo Alldieck, and Gerard Pons-Moll.  Implicit functions in feature space for 3d shape reconstruction and completion.  In _Proceedings of the IEEE/CVF conference on computer vision and pattern recognition_ , pages 6970–6981, 2020.
  * Hanocka et al. [2020] Rana Hanocka, Gal Metzer, Raja Giryes, and Daniel Cohen-Or.  Point2mesh: A self-prior for deformable meshes.  _ACM Trans. Graph._ , 39(4), 2020.
  * Hendrycks and Gimpel [2016] Dan Hendrycks and Kevin Gimpel.  Gaussian error linear units (gelus).  _arXiv preprint arXiv:1606.08415_ , 2016.
  * Hou et al. [2022] Fei Hou, Chiyu Wang, Wencheng Wang, Hong Qin, Chen Qian, and Ying He.  Iterative poisson surface reconstruction (ipsr) for unoriented points.  _ACM Trans. Graph._ , 41(4), 2022.
  * Kazhdan and Hoppe [2013] Michael Kazhdan and Hugues Hoppe.  Screened poisson surface reconstruction.  _ACM Trans. Graph._ , 32(3), 2013.
  * Kazhdan et al. [2006] Michael Kazhdan, Matthew Bolitho, and Hugues Hoppe.  Poisson surface reconstruction.  In _Proceedings of the Fourth Eurographics Symposium on Geometry Processing_ , page 61–70, Goslar, DEU, 2006. Eurographics Association.
  * Li et al. [2021] Zongyi Li, Nikola Borislavov Kovachki, Kamyar Azizzadenesheli, Burigede liu, Kaushik Bhattacharya, Andrew Stuart, and Anima Anandkumar.  Fourier neural operator for parametric partial differential equations.  In _International Conference on Learning Representations_ , 2021.
  * Lorensen and Cline [1987] William E. Lorensen and Harvey E. Cline.  Marching cubes: A high resolution 3d surface construction algorithm.  _SIGGRAPH Comput. Graph._ , 21(4):163–169, 1987\.
  * Nguyen and Wang [2021] Andrew-Hieu Nguyen and Zhaoyang Wang.  Accurate 3d shape reconstruction from single structured-light image via fringe-to-fringe network.  _Photonics_ , 8:459, 2021.
  * Noll [1971] A. Michael Noll.  Scanned-display computer graphics.  _Communications of the ACM_ , 14:143 – 150, 1971.
  * Otsu [1979] Nobuyuki Otsu.  A threshold selection method from gray-level histograms.  _IEEE Transactions on Systems, Man, and Cybernetics_ , 9(1):62–66, 1979.
  * Pathak et al. [2022] Jaideep Pathak, Shashank Subramanian, Peter Harrington, Sanjeev Raja, Ashesh Chattopadhyay, Morteza Mardani, Thorsten Kurth, David Hall, Zongyi Li, Kamyar Azizzadenesheli, Pedram Hassanzadeh, Karthik Kashinath, and Animashree Anandkumar.  FourCastNet: A global data-driven high-resolution weather model using adaptive fourier neural operators.  2022\.
  * Peng et al. [2020] Songyou Peng, Michael Niemeyer, Lars Mescheder, Marc Pollefeys, and Andreas Geiger.  Convolutional occupancy networks.  In _Computer Vision–ECCV 2020: 16th European Conference, Glasgow, UK, August 23–28, 2020, Proceedings, Part III 16_ , pages 523–540. Springer, 2020.
  * Peng et al. [2021] Songyou Peng, Chiyu Jiang, Yiyi Liao, Michael Niemeyer, Marc Pollefeys, and Andreas Geiger.  Shape as points: A differentiable poisson solver.  In _Advances in Neural Information Processing Systems_ , pages 13032–13044, 2021.
  * Sellán and Jacobson [2023] Silvia Sellán and Alec Jacobson.  Neural stochastic screened poisson reconstruction.  _arXiv preprint arXiv:2309.11993_ , 2023.
  * Sharf et al. [2006] Andrei Sharf, Thomas Lewiner, Ariel Shamir, Leif Kobbelt, and Daniel Cohen-Or.  Competing fronts for coarse–to–fine surface reconstruction.  _Comput. Graph. Forum_ , 25:389–398, 2006.
  * Sulzer et al. [2023] Raphael Sulzer, Loic Landrieu, Renaud Marlet, and Bruno Vallet.  A survey and benchmark of automatic surface reconstruction from point clouds, 2023.
  * Wen et al. [2022a] Gege Wen, Zongyi Li, Kamyar Azizzadenesheli, Anima Anandkumar, and Sally M. Benson.  U-fno—an enhanced fourier neural operator-based deep-learning model for multiphase flow.  _Advances in Water Resources_ , 163:104180, 2022a.
  * Wen et al. [2022b] Gege Wen, Zongyi Li, Kamyar Azizzadenesheli, Anima Anandkumar, and Sally M. Benson.  U-fno—an enhanced fourier neural operator-based deep-learning model for multiphase flow.  _Advances in Water Resources_ , 163:104180, 2022b.
  * Yuksel [2015] Cem Yuksel.  Sample elimination for generating poisson disk sample sets.  _Computer Graphics Forum_ , 34, 2015.