# Iterative Approach to Reconstructing Neural Disparity Fields from Light-
field Data

Ligen Shi, Chang Liu, Xing Zhao*, and Jun Qiu Ligen Shi and Xing Zhao are with
the School of Mathematical Sciences, Capital Normal University, Beijing
100048, China<EMAIL_ADDRESS>zhaoxing_1999@126.com)Chang Liu and
Jun Qiu are with the Institute of Applied Mathematics, Beijing Information
Science and Technology University, Beijing 100101, China,
<EMAIL_ADDRESS>qiujun_sub@126.com)Corresponding author: Xing Zhao and
Jun Qiu.

###### Abstract

This study proposes a neural disparity field (NDF) that establishes an
implicit, continuous representation of scene disparity based on a neural field
and an iterative approach to address the inverse problem of NDF reconstruction
from light-field data. NDF enables seamless and precise characterization of
disparity variations in three-dimensional scenes and can discretize disparity
at any arbitrary resolution, overcoming the limitations of traditional
disparity maps that are prone to sampling errors and interpolation
inaccuracies. The proposed NDF network architecture utilizes hash encoding
combined with multilayer perceptrons to capture detailed disparities in
texture levels, thereby enhancing its ability to represent the geometric
information of complex scenes. By leveraging the spatial-angular consistency
inherent in light- field data, a differentiable forward model to generate a
central view image from the light-field data is developed. Based on the
forward model, an optimization scheme for the inverse problem of NDF
reconstruction using differentiable propagation operators is established.
Furthermore, an iterative solution method is adopted to reconstruct the NDF in
the optimization scheme, which does not require training datasets and applies
to light-field data captured by various acquisition methods. Experimental
results demonstrate that high-quality NDF can be reconstructed from light-
field data using the proposed method. High-resolution disparity can be
effectively recovered by NDF, demonstrating its capability for the implicit,
continuous representation of scene disparities.

###### Index Terms:

Neural Disparity Field, Disparity estimation, Light Field

##  I Introduction

Computational imaging innovatively incorporates computer vision and signal
processing techniques into imaging systems to reconstruct high-dimensional
optical signals that carry scene information through novel imaging mechanisms,
optical paths, and reconstruction methods, thereby achieving efficient and
high-precision observation of scenes [1]. Computational imaging surpasses the
limitations of classical imaging in terms of signal dimensions, scale, and
resolution.

From the perspective of imaging methodology, light-field imaging operates as a
close-proximity passive direct line-of-sight imaging technology that relies on
ambient light to capture scene content within its visual ambit. The
characteristic features included narrower baselines, increased uniformity in
viewpoint sampling, and diminished occlusions. Notably, substantial domain
discrepancies emerge in the light-field data generated by different cameras
owing to inherent variations in their imaging modalities, structural designs,
and internal parameters.

Deep-learning-based reconstruction methods rely on labeled training data and
lack exploration of the inherent consistency within light-field data.
Consequently, networks trained using data from one camera model may not be
generalizable to other camera models. Given that individual light-field data
contain abundant information, exploiting the full potential of their spatial-
angular consistency empowers a network to reduce its reliance on specific
camera models, thereby promoting generalization across different camera
models.

The neural field [2] represents the scene to be reconstructed as a continuous
implicit function, achieving advanced visual quality by the neural network
parameterization. The continuous implicit function representation ensures
continuity and is not limited by resolution constraints. By employing neural
network parameterization of signal values, this representation method adopts a
”discrete-to-continuous” computational approach and utilizes differentiability
for solving inverse problems. Compared with traditional methods, the neural
field exhibits superior precision, lower complexity, and increased robustness.
Furthermore, compared with deep learning approaches, their advantages include
not requiring extensive training datasets, being unrestricted by data formats,
having fewer parameters, and exhibiting high robustness.

Therefore, we utilize a fully connected neural network to parameterize the
disparity. The network constructs a neural disparity field (NDF) by
incorporating hash encoding and multilayer perceptrons (MLPs) with activation
functions. This approach implicitly represents the disparity contained in the
light field and exploits the spatial-angular consistency within it. The
inverse problem-solving model based on the NDF was established to reconstruct
the disparity from the light field. This model was trained and iteratively
solved through differentiability, effectively transforming the disparity
estimation problem into an optimization problem for the NDF. The framework for
the implicit representation of an NDF is shown in Fig.1.

Figure 1: Framework for the implicit representation of neural disparity
fields. Red ray bundles indicate the propagation of disparities from the
visible views towards the central view.

By exploiting the spatial-angular consistency within individual light-field
data, the proposed method demonstrates exceptional performance in
reconstructing the NDF. This reconstructed field can restore high-quality
explicit discrete disparity maps. Unlike depth-learning-based methods for
disparity estimation, the NDF reconstruction method does not depend on
training datasets. It can be applied to two-plane parameterized light-field
data using any collection method. By utilizing the implicit representation of
the neural field, this disparity reconstruction method departs from the
traditional approach of representing disparity as discrete points, overcoming
the inevitable sampling, interpolation, and other discretization calculation
issues of explicit discrete disparity and establishing a continuous disparity
surface with implicit expression. The innovations of this study are as
follows:

  1. 1.

Disparity is parameterized using neural networks to establish an NDF, that
creates an implicit continuous representation of scene disparity functions.
This NDF enhances the expressive capability of complex details of the
disparity surface and effectively reduces computational errors introduced by
discretization. Unlike traditional hierarchical disparity search estimation
methods, a representation based on the NDF allows the formation of a
differentiable propagation forward problem model, inverse problem optimization
modeling, and iterative solving of the reconstruction algorithm for continuous
implicit disparity functions.

  2. 2.

From the perspective of spatial-angular consistency, a forward problem model
for the disparity propagation of the NDF under non-ideal scene conditions
(with occlusion and noise) is constructed. This forward-problem model,
described by differentiable propagation operators, can propagate any viewpoint
image to a central view via the NDF.

  3. 3.

Based on the forward problem model, an optimization model for the inverse
problem of the NDF was established using differentiable propagation operators.
An iterative approach was proposed to solve the disparity reconstruction from
the light-field. This method offers a novel solution to the disparity
reconstruction problem. The iterative reconstruction method based on the
inverse problem does not rely on training datasets and can be applied to two-
plane parameterized light-field data obtained by any collection method.
Furthermore, it allows the application of inverse problem-solving strategies
in disparity reconstruction.

##  II Related work

The 7D plenoptic function[3] was initially used to characterize the light
fields. Over time, this function has been transformed into a two-plane four-
parameter light field model[4], with the majority of commonly used methods
relying on four-dimensional(4D) two-plane light fields[5]. The utilization of
light field technology is crucial for providing depth information. To ensure
generality, the proposed method is demonstrated using the central viewpoint
depth and possesses the ability to reconstruct depth information from
alternative viewpoints. Additionally, for light fields, there exists a one-to-
one correspondence between the scene depth and disparity, which accurately
represents the three-dimensional(3D) spatial structure of the scene. Existing
methods can be broadly classified into two categories: rule-based traditional
methods and deep-learning-based methods.

###  II-A Rule-based traditional methods

Disparity estimation algorithms that are rule-based traditional methods adhere
to a computational approach known as ”discrete-to-discrete”. They aimed to
estimate the corresponding discrete disparity images from discretized light-
field data. However, this computational approach is influenced by the
resolution limitations of disparity and unavoidably introduces various errors.
A common source of error is sub-pixel interpolation, given that the accuracy
of interpolation influences the estimation precision of the algorithm. For
instance, Jeon et al. [6]proposed the use of the Fourier phase-shifting theory
to replace bilinear interpolation, resulting in improved estimation accuracy.
Currently, researchers have not noted advancements in the mitigation of
discretization errors.

Early disparity estimation followed rule-based estimation methods, exploring
the structural information of light-field data to assess the consistency among
different views. Wanner et al. [7] introduced structural tensors into depth
estimation from light-field data, enhancing the global consistency of the
reconstructed depth by constraining the continuity of depth from different
subview images of the light field. However, owing to the insufficient
utilization of angular information in light-field data, the estimation
performance of this algorithm requires further improvement. Heber et al. [8]
introduced principal component analysis for the computation of stereo
matching. They focused on subview images that impacted matching errors while
disregarding those with minor influences. This approach improves the
robustness of the algorithm and to some extent, its precision for stereo
matching. To overcome the challenges of matching small baselines between
subview images in light fields, Jeon et al. [6] applied Fourier phase-shifting
theory to improve the accuracy of the matching computation. They adaptively
aggregated the gradient costs based on the angular coordinates of the light
fields and incorporated feature correspondences between subaperture images as
additional constraints. Tao et al. [9]examined the properties of two depth
cues derived from the correspondence and defocus in light-field data. Their
findings demonstrate that these cues yield complementary depth information.
Consequently, researchers have suggested merging the two depth cues to
incorporate both angular and spatial information from the light field, thereby
enhancing the performance of the algorithm for disparity estimation. However,
this algorithm requires manual parameter adjustment to blend different
results, thereby affecting its robustness across different scenes. In
addition, it does not account for occlusion effects, leading to estimation
errors in occluded regions.

To mitigate the errors arising from occlusions, Wang et al. [10] expanded the
work of [9] by developing a model for the occlusion effect in light fields.
They proposed an anti-occlusion algorithm by studying the relative positional
relationships between different subview images in a light field and
incorporating prior knowledge of object occlusion into stereo-matching
algorithms. Although this algorithm partially addresses the challenges
associated with occlusion in depth estimation by modeling occlusion effects in
light fields, its straightforward occlusion detection criteria result in
limited robustness across different scenes. To further mitigate the impact of
occlusion on depth estimation performance, Sheng et al. [11] devised an
estimation algorithm based on epipolar plane images (EPIs) combined with the
local consistency of scene depth to address occlusion issues. Zhang et al.
[12] considered the structure of EPIs and proposed a local depth estimation
method based on a spinning parallelogram operator(SPO). Utilizing the regions
segmented by the SPO in EPIs, they located lines representing depth
information by maximizing the distribution distances between areas. Liu et al.
[13] proposed an iterative scheme for scene depth reconstruction based on a 4D
light field. The objective function of their optimization problem consisted of
a fidelity term representing matching, along with penalty terms comprising
gradient and classification terms. It iteratively corrects mismatched pixels
to minimize the objective function, ultimately achieving accurate depth
estimation. Subsequently, Liu et al. [14] mapped the pixel differences between
matching regions to radial basis functions, where pixels with larger
differences in occluded and noisy areas were mapped to smaller values in the
radial basis functions, thereby effectively achieving anti-occlusion
resistance and antinoise.

Although rule-based traditional disparity estimation algorithms achieve
relatively high precision in depth estimation by exploiting the
interrelationships among sub-aperture images, optimizing stereo matching
computations, and addressing occlusion effects, their performance is still
limited owing to the insufficient exploitation of angular and spatial
information within the light field. Additionally, these methods, which use a
”discrete-to-discrete” approach, fail to overcome the resolution limitations
inherent in the current representation of disparity and remain vulnerable to
errors caused by data discretization. By contrast, neural fields with implicit
representations, characterized by their inherent continuity, overcome the
limitations imposed by finite-resolution discrete disparities and offer
enhanced adaptability to complex variations in disparity and precision in
their depiction.

###  II-B Deep learning-based methods

With the advancements made in deep learning techniques for diverse computer
vision tasks, there has been increasing interest in depth estimation from
light-field data. Deep-learning-based algorithms primarily depend on
supervised learning approaches, which require a large amount of training data
annotated with accurate depth labels.

Heber et al. [15] introduced convolutional neural networks (CNNs) to address
the task of depth estimation from light field data. Their approach utilizes
supervised learning by providing a network with subaperture images in the
horizontal and vertical directions and the corresponding depth labels. Data-
augmentation techniques involving rotation, translation, brightness, and color
adjustments have been proposed to improve the training process. Although the
algorithm demonstrated superior performance on synthetic datasets compared to
traditional algorithms, it required a large amount of depth-labeled training
data, resulting in substantial data generation costs. In additionally, the
domain gap between synthetic and real data poses challenges to the
applicability of the algorithm in practical imaging systems. Subsequently,
Shin et al. [16] proposed a multi-branch network structure with four branches
dedicated to extracting features from light-field images in horizontal,
vertical, and diagonally tilted directions. These features were then combined
using CNNs to reconstruct the scene depth. Although the method showed improved
estimation performance on synthetic data compared with traditional disparity
estimation algorithms, it exhibited limitations in occluded and specular
regions.

Alperovich et al. [17] developed a fully convolutional autoencoder for
simultaneous depth and reflectance estimations. Although this supervised
approach partially addresses the occlusion effects, its generalization ability
is limited. Peng et al. [18] introduced unsupervised learning for depth
estimation using light-field data. They proposed a combined loss function that
imposed compliance and divergence constraints on the disparity between the
warped sub-aperture images and the central view. This approach alleviates the
challenge of acquiring labeled data and improves the generalizability of the
algorithm across different imaging systems. Zhou et al. [19] utilized
geometrically constrained EPIs to train networks and reconstruct depth
information without requiring ground-truth depth labels. Li et al. [20]
introduced an Oriented Relation Module that captured spatial relationships in
vertical and horizontal EPIs. They enhanced training effectiveness and depth-
estimation performance using refocusing-based data augmentation. Tsai et al.
[21] presented a viewpoint-selection module that generated attention maps to
determine the importance of each view for accurate depth estimation. This
attention-driven approach improves the effective utilization of light field
data. Li et al. [22] developed a lightweight model to estimate the disparity.
The model employed physics-based multiscale cost-volume convolutional
aggregation and an edge-guided subnetwork to enhance the geometric detail
recovery near the edges and the overall performance. Huang et al. [23]
designed an end-to-end trainable lightweight convolutional network that
integrated cost volumes and attention modules. The designed network
architecture enables the inference of light-field inputs at varying angles and
facilitates detailed recovery in occluded regions.

The acquisition of the training and labeling data required by existing deep-
learning algorithms poses a challenge, thus limiting their precision and
applicability. Most deep learning-based approaches depend on data sourced
exclusively from particular camera models during training, impeding their
capacity to acclimate to various camera models and diminishing their
generalization ability.

###  II-C Neural fields

The neural radiance field [24] represents the scene to be reconstructed as a
continuous implicit function parameterized by a neural network, achieving
advanced visual quality and inspiring numerous subsequent works [25, 26, 27,
28, 29, 30]. Among these, mip-NeRF [25] reduces the multiscale representation
of image aliasing in the neural radiance field. Mip-NeRF 360 [26] extended
mip-NeRF to address poor performance in unbounded scene representations. Ref-
NeRF [27] addresses specular and reflection issues in novel view synthesis.
The instant-NGP [28] model enhances the training speed of neural radiance
field models. TensorRF [29] decomposes the radiance field tensor into multiple
lower-order tensor components to obtain a more compact scene representation.
Factor Fields [30] is a unified framework based on coordinate-input networks,
allowing the creation of powerful new signal representations and achieving
improvements in approximation quality, compactness, and training time.

Continuous implicit functions parameterized by neural fields exhibit
continuity and are not subject to limitations imposed by finite resolution,
showing advantages over discrete representations (e.g., pixel color [31, 32],
light field [33, 34, 35], signed distance [36], and occupancy [37]) in certain
applications[38]. The core idea involves training MLPs with activation
functions to map the input coordinates to the desired target values. Similar
to existing coordinate-based representation methods, our method learns to map
the coordinates of the central view to the corresponding disparities within
the central view scene.

##  III Method

Traditional methods for disparity estimation from light field data follow a
”discrete-to-discrete” approach, which fails to overcome the resolution
limitations inherent in disparity representation. By contrast, with their
continuous nature, neural field implicit representations provide superior
expression capabilities compared to traditional discrete disparity
representations, as they are not bound by finite resolution constraints, thus
enabling robust modeling of complex disparity variations.

###  III-A Neural disparity field representation

In contrast to traditional implicit representations of 3D shapes, disparity
refers to a two-dimensional (2D) geometric manifold in 3D space that is
associated with the depth of the scene. It describes the differences between
viewpoints instead of directly representing 3D geometric shapes. The implicit
continuous parameterization of NDFs allows the modeling of continuous
disparities, making it more appropriate for describing continuous variations
in disparity within light fields compared to the conventional 2D matrix
representations.

The disparity inherent in a scene within a light field is parameterized as a
continuous 2D implicit function using neural fields, termed NDFs. This is
achieved by approximating the continuous 2D implicit function using a
multiresolution grid and the MLPs network $\mathbf{F}_{\mathbf{\Theta}}$ with
activation functions, defined as

$\mathbf{F}_{\mathbf{\Theta}}:\mathbf{x}\rightarrow{d(\mathbf{x})},$ (1)

where the network takes 2D coordinate positions $\mathbf{x}=(x,y)$ as input
and outputs the disparity ${d(\mathbf{x})}$ at that point. By optimizing the
network weights $\mathbf{\Theta}$, this representation maps each input 2D
coordinate $\mathbf{x}$ to its corresponding disparity ${d(\mathbf{x})}$.

###  III-B Inverse problem solving for disparity reconstruction

In the light field, disparity information is shared across various subviews,
allowing viewpoint transitions through disparity propagation. In the two-plane
four-parameter light field, assuming that occlusion and camera noise are
disregarded, the relationship between image $L_{(u_{0},v_{0})}(\mathbf{x})$ at
the central viewpoint $(u_{0},v_{0})$ and image
$L_{(u_{i},v_{j})}(\mathbf{x})$ at viewpoint $(u_{i},v_{j})$ can be described
as follows:

$L_{(u_{0},v_{0})}\left(\mathbf{x}\right)=L_{(u_{i},v_{j})}\left(\mathbf{x}+\Delta\cdot
d(\mathbf{x})\right),$ (2)

where $\Delta=\left(u_{i}-u_{0},v_{j}-v_{0}\right)$ represents the distance
between viewpoints $(u_{i},v_{j})$ and $(u_{0},v_{0})$ in the $u$ and $v$
directions.

Certain factors such as occlusion and camera noise render Eq.(2) invalid for
real-world scenarios. Therefore, this study introduces the propagation mask
$m_{(u,v)}(\mathbf{x})$, with $m_{(u,v)}(\mathbf{x})=0$ in cases of occlusion
or noise, and $m_{(u,v)}(\mathbf{x})=1$ otherwise. Based on this, the
relationship between the central view image
$L_{(u_{0},v_{0})}\left(\mathbf{x}\right)$ and the view image
$L_{(u_{i},v_{j})}\left(\mathbf{x}\right)$ in the real scenes is adjusted as
follows:

$\begin{split}L_{(u_{0},v_{0})}(\mathbf{x})=m_{(u_{i},v_{j})}(\mathbf{x})\cdot
L_{(u_{i},v_{j})}\left(\mathbf{x}+\Delta\cdot
d(\mathbf{x})\right).\end{split}$ (3)

####  III-B1 Forward problem modeling

The light field is regularly sampled on the viewpoint plane, ensuring that the
pixels in the central view image are visible in at least half of the
viewpoints, resulting in a stronger coupling between the central view and
other views. Therefore, this study employs the disparity propagation process
from other views to the central view in the light field to model disparity
reconstruction. This process effectively utilizes the constraints of regular
sampling on the viewpoint plane, while preserving the characteristics of the
light field. This provides a robust forward problem model for disparity
reconstruction.

To account for occlusion and camera noise in real scenes with a light field, a
forward problem model for disparity reconstruction under angular consistency
was established as follows:

$L(d(\mathbf{x}))=\frac{\sum_{(u_{i},v_{j})\in\phi}\left(m_{(u_{i},v_{j})}(\mathbf{x})\cdot
L_{(u_{i},v_{j})}\left(\mathbf{x}+\Delta\cdot
d(\mathbf{x})\right)\right)}{\sum_{(u_{i},v_{j})\in\phi}m_{(u_{i},v_{j})}(\mathbf{x})}.$
(4)

where $\phi$ is the set of viewpoints.

####  III-B2 Inverse problem solving

##### Loss functions for selected viewpoints.

Traditional stereo-matching algorithms often utilize region-based matching
windows instead of individual pixels to enhance the robustness of the matching
results[39]. To exploit the local structural information of the light field
fully, this study utilized a combination of L1 and mean structural similarity
(MSSIM) losses [40]. Total variation (TV) [41] regularization is introduced as
a loss term to exploit the sparsity of disparities in the gradient domain.

Given the light field $L_{(u,v)}(\mathbf{x})$ and the coordinate $\mathbf{x}$
of the central view, this study aims to determine the optimal parameters
$\Theta^{*}$ by minimizing the objective function:

$\begin{split}&{\Theta^{*}}=\underset{\Theta}{\operatorname{arg}\operatorname{min}}\left|L_{(u_{0},v_{0})}\left(\mathbf{x}\right)-L\left(\hat{d}_{\Theta}\left(\mathbf{x}\right)\right)\right|+\\\
&\alpha\cdot\left(1-\text{MSSIM}\left(L_{(u_{0},v_{0})}\left(\mathbf{x}\right),L\left(\hat{d}_{\Theta}\left(\mathbf{x}\right)\right)\right)\right)+\beta\cdot\Psi\left({\hat{d}_{\Theta}\left(\mathbf{x}\right)}\right),\end{split}$
(5)

where $\hat{d}_{\Theta}\left(\mathbf{x}\right)$ represents the disparity
predicted at point $\mathbf{x}$ by the network and
$L\left(\hat{d}_{\Theta}\left(\mathbf{x}\right)\right)$ is computed value
using Eq.(4). $\Psi(\mathbf{x})$ denotes the regularization term for the
parameterized disparities, where TV regularization is employed. $\alpha$ and
$\beta$ are the two hyperparameters.

This study utilized the L1 + $\alpha\cdot\left(\text{MSSIM}\right)$ metric to
evaluate the disparity between viewpoints that inadequately convey information
and the central view image. This metric, denoted as distance
$E_{(u_{i},v_{j})}\left(\mathbf{x}\right)$, is defined as follows:

$\begin{split}&E_{(u_{i},v_{j})}\left(\mathbf{x}\right)=\left|L_{(u_{0},v_{0})}\left(\mathbf{x}\right)-L_{(u_{i},v_{j})}\left(\mathbf{x}+\Delta\cdot\hat{d}_{\Theta}(\mathbf{x})\right)\right|\\\
&+\alpha\cdot\left(1-\text{MSSIM}\left(L_{(u_{0},v_{0})}\left(\mathbf{x}\right),L_{(u_{i},v_{j})}\left(\mathbf{x}+\Delta\cdot\hat{d}_{\Theta}(\mathbf{x})\right)\right)\right).\end{split}$
(6)

Given that occluded pixels can effectively transmit information from at least
half of the viewpoints, this study proposes a simplified strategy for
computing an occlusion mask during disparity propagation. Assuming that all
viewpoint masks hold a value of 1, signifying that all viewpoints contribute
valuable information, this study proceeds by selecting half of the viewpoints
based on a comparison of the ray distances $E_{(u_{i},v_{j})}(\mathbf{x})$. It
then selects viewpoints $\Phi_{\text{selected}}$ from which the ray distance
is smaller than that of the remaining viewpoints; these selected viewpoints
are subsequently utilized for conducting backward propagation of rays. In
general, this loss function is expressed by

$\mathcal{L}=\sum_{(u_{i},v_{j})\in\Phi_{\text{selected}}}\left(E_{(u_{i},v_{j})}\left(\mathbf{x}\right)\right)+\Psi\left({\hat{d}_{\Theta}\left(\mathbf{x}\right)}\right).$
(7)

This approach effectively avoids the complex computation of propagation masks
during disparity propagation, while simultaneously reducing interference from
occlusion and noise factors. Therefore, we term this approach the viewpoint
selection loss strategy, which improves the accuracy of depth estimation in
occluded scenes by giving precedence to the efficacy of information
transmission.

##### Network architecture

To enhance the computational efficiency and expressive power of neural
networks, recent research [42, 43, 28, 29, 30] has explored the possibility of
the structured storage of latent features. While fully connected networks
combined with positional encoding can record data information through weights,
they reduce computational efficiency and limit the network’s capacity to
represent complex patterns. Previous studies explored different approaches for
storing latent features, such as 3D grids [42]. However, this grid structure
suffers from resource wastage because a large portion of the internal units
remain empty, resulting in the effective utilization of only surface
information. To address this issue and reduce memory and training data
requirements, prior work has explored the use of tree-like data structures
similar to those of Acorn [43]; however, this approach incurs a larger
overhead in dynamically maintaining the tree structure during training. Thus,
to overcome these limitations, [28] proposed the use of multi-resolution grids
for feature storage.

Considering the disparity characteristics, adjacent pixels often have similar
values, reflecting the smoothness of scene depth variations. This necessitates
the precise capture of subtle changes using the NDF model. Additionally, the
distribution of gradients in the disparity exhibits sparsity, with most pixels
having gradients close to zero, which is significant only at the scene or
object edges. This sparsity implies that the local pixels share the same
features, obviating the need to learn unique feature descriptions for each
independent pixel, thereby reducing the training time. Thus, this study
constructed an NDF network inspired by the approach proposed in [28]. Its
structure comprises a multiresolution grid feature module and a MLPs module,
as depicted in Fig. 2. The multiresolution grid feature module, a trainable
component, encompasses $L$ grids with resolutions linearly increasing from 32
to 128, and is utilized for feature vector storage. The MLPs module consisted
of fully connected layers and LeakyReLU activation functions, comprising two
hidden layers, each with 256 neurons.

Figure 2: Neural-disparity-field network architecture. The network
architecture comprises multiresolution grid-feature and MLPs modules. The
multiresolution grid-feature module is a trainable module that stores Hash
encoding feature vectors. MLPs module comprises fully connected layers and
LeakyReLU activation functions.

To train the NDF, the following steps are executed:

  1. 1.

A set of 2D sampling points $\mathbf{x}$ was extracted from the coordinate
system of the reference view.

  2. 2.

The multiresolution grid feature module was utilized to extract the
corresponding features of the sampling points. These features were input into
the MLPs module to predict the corresponding disparity
$\hat{d}_{\Theta}\left(\mathbf{x}\right)$.

  3. 3.

The distance $E_{(u_{i},v_{j})}\left(\mathbf{x}\right)$ is calculated using
Eq.(6) and retain those with $E_{(u_{i},v_{j})}\left(\mathbf{x}\right)$
smaller than the corresponding rays of the other half of the viewpoints.

  4. 4.

The loss of the retained rays is computed using Eq.(7). Subsequently, backward
propagation is performed to endow the network with the ability to learn the
disparity.

##  IV Experiments

To comprehensively assess the performance of the proposed method, we conducted
comparative experiments on several light field datasets, including the HCI 4D
light field dataset [44], 4D Light Field Benchmark (CVIA Konstanz & HCI
Heidelberg) [45], and Stanford Lytro dataset [46].

We evaluated the proposed method in synthetic environments of the HCI 4D light
field dataset and 4D Light Field Benchmark to verify its fundamental
performance and generalization. We then applied it to real Lytro-captured
light-field data for a practical disparity estimation assessment. The ability
of this method to directly reconstruct disparities from a network at diverse
spatial resolutions was also demonstrated. Finally, an ablation study
investigated the impact of combined losses, offering deep theoretical insights
and practical guidance for algorithm understanding and optimization.

Experiments were conducted on hardware featuring an NVIDIA GeForce RTX 3080
with 10GB memory. The experimental parameter settings in this section are as
follows: a grid feature module with six resolutions linearly increasing within
the range from 32 to 128, an MSSIM loss window size of 11 pixels, and
hyperparameter values $\alpha$ and $\beta$ ranging between $1\text{e}^{-1}$
and $1\text{e}^{1}$, with slight variations for different scenes. During
network training, Gaussian noise with intensities ranging from
$1\text{e}^{-2}$ to $1\text{e}^{0}$ was introduced.

###  IV-A Algorithm analysis and evaluation using synthetic data experiments

####  IV-A1 Complex scene experiments

In this section, disparity estimation is conducted using synthetic light-field
datasets, and compared with the MV [47], LF_OCC [10], SPO [12], and AMN [14]
algorithms for evaluation. Experimental results are illustrated in Fig. 3,
which shows four scenes from the two datasets.

Figure 3: Disparity results for complex synthetic scenes, with odd rows
showing estimated disparity maps and even rows displaying profile curves (red
line in reference view) comparing actual (red) and algorithm-estimated (blue)
disparities. Scenes include from HCI [44] (Buddha and Papillon) and 4D Light
Field Benchmark [45] (Pens and Tomb).

In the Buddha scene, the prominent elements include a Buddha statue, slanted
wooden planks, stone pillars, and dice. The intricate textures of Buddha
statues and stone pillars pose challenges to the robustness of these
algorithms. However, as shown in Fig. 3, all of the aforementioned algorithms
effectively estimated the disparities of these elements. Slanted wooden planks
tested the ability of the algorithms to handle inclined surfaces, where the
proposed method outperformed MV, producing smoother disparity predictions on
slanted surfaces. Additionally, the black smooth regions on the dice present
challenges to the algorithm performance, where the proposed method
demonstrates notable advantages over SPO in handling textureless smooth areas.
The profile graphs in the second row of Fig. 3 show that the proposed method
exhibited minimal oscillations, except for slight oscillations at the
boundaries. In the Papillon scene, the primary elements are butterflies and
leaves. The butterfly tail features a weakly textured black area and handling
such a region is difficult. The proposed method successfully computed
disparities in this area compared with MV, SPO, and AMN. Furthermore, for the
boundary areas of the leaves, LF_OCC generates considerable noise owing to
occlusion, whereas the proposed method exhibits advantages in handling such
occluded regions. The Pens scene comprised multiple Pens arranged at various
angles and depths. LF_OCC introduces substantial noise at pen boundaries,
whereas the proposed method and other comparative algorithms effectively
reconstruct the disparities in this scene. The Tomb scene contains several
stone statues, and the results reconstructed using the proposed method and SPO
exhibit smoothness closer to the ground truth.

This section provides detailed analyses of the Buddha, Papillon, Pens, and
Tomb scenes, covering common scenarios of inclined surfaces, smooth areas,
occluded regions, and texture structures. Through comparative analysis, the
proposed method demonstrated advantages in handling smooth and weakly textured
areas. From the profiles, it is evident that the proposed method maintains
minimal deviation across the entire curve, although it exhibits oscillations,
particularly at the edges of the scene.

####  IV-A2 Specific scene experiments

To gain a comprehensive understanding of the performance of the proposed
method, we set up isolated challenges in specific scenarios: backgammons,
dots, pyramids, and stripes. The Backgammon scene evaluated the algorithm’s
capability in handling fine structures and occlusion boundaries. The Dots
scene investigated the impact of different levels of camera noise on the
reconstruction of objects of various sizes. The Pyramids scene tested the
algorithm’s performance on convex, concave, circular, and planar geometrical
structures, and the Stripes scene specifically assessed the influence of
texture and contrast on occlusion boundary recognition. We conducted
comparative experiments using the proposed method, MV, LF_OCC, SPO, AMN,
EPI_ORM[20], LLFNet[22], and FastLFnet[23]. The results are presented in Fig.
4.

Figure 4: Visual comparison of disparity estimation methods across different scenes. From top to bottom, rows correspond to Backgammon, Dots, Pyramids, and Stripes scene. For each scene, the top row shows the estimated disparity, and the bottom row shows the corresponding MSE. Results (b)-(h) are derived from original data in [47]. TABLE I: Qualitative comparisons of results on synthetic datasets, presented with BP01(BadPix0.01), BP03(BadPix0.03), BP07(BadPix0.07) and MSE ($\times$ 100) metrics. |  | MV | LF_OCC | SPO | AMN | EPI_ORM | LLFNet | FastLFNet | Ours
---|---|---|---|---|---|---|---|---|---
Backgammon | BP01 | 65.842 | 88.839 | 49.940 | 43.599 | 34.316 | 65.975 | 39.839 | 54.878
BP03 | 18.547 | 39.826 | 8.639 | 13.354 | 7.238 | 34.337 | 11.409 | 16.378
BP07 | 8.924 | 19.006 | 3.782 | 6.913 | 3.988 | 14.853 | 5.138 | 6.656
MSE | 13.225 | 21.587 | 4.587 | 12.287 | 3.411 | 8.058 | 3.986 | 8.929
Dots | BP01 | 83.519 | 65.762 | 58.079 | 21.658 | 65.707 | 67.417 | 68.146 | 4.538
BP03 | 59.654 | 23.552 | 35.068 | 18.428 | 47.932 | 41.971 | 41.109 | 4.485
BP07 | 45.587 | 5.822 | 16.274 | 18.428 | 36.098 | 20.134 | 21.169 | 4.380
MSE | 7.262 | 3.301 | 5.238 | 2.936 | 14.480 | 6.159 | 3.407 | 5.433
Pyramids | BP01 | 85.392 | 74.144 | 79.206 | 32.485 | 19.063 | 49.817 | 22.188 | 32.675
BP03 | 9.824 | 20.495 | 6.263 | 5.643 | 1.301 | 6.495 | 2.193 | 2.853
BP07 | 0.776 | 3.172 | 0.861 | 0.449 | 0.324 | 1.059 | 0.620 | 0.129
MSE | 0.051 | 0.098 | 0.043 | 0.023 | 0.016 | 0.035 | 0.018 | 0.016
Stripes | BP01 | 76.372 | 53.669 | 21.879 | 13.499 | 55.135 | 79.080 | 63.400 | 8.389
BP03 | 62.417 | 21.257 | 15.460 | 7.966 | 13.943 | 47.050 | 32.594 | 8.053
BP07 | 46.192 | 18.408 | 14.987 | 5.835 | 6.871 | 23.837 | 9.442 | 7.518
MSE | 12.173 | 8.131 | 6.955 | 2.128 | 1.744 | 2.390 | 0.984 | 2.518
Average | BP01 | 77.781 | 70.604 | 52.276 | 27.810 | 43.556 | 65.572 | 48.393 | 25.12
BP03 | 37.611 | 26.283 | 16.358 | 11.348 | 17.604 | 32.463 | 21.826 | 7.942
BP07 | 25.370 | 11.602 | 8.976 | 7.906 | 11.820 | 14.971 | 9.092 | 4.671
MSE | 8.178 | 8.279 | 4.206 | 4.344 | 4.913 | 4.161 | 2.099 | 4.224
Paremeteres (M) | — | — | — | — | 9.1 | 1.8 | 1.4 | 0.5

Figures 4 (b)-(h) correspond to the different algorithms. From the
experimental results, it is evident that the proposed method successfully
reconstructs disparities in various challenging scenarios. In the Backgammon
scene, the proposed method performed remarkably well in handling complex
textures, tilted surfaces, and textureless smooth regions, particularly when
compared with MV, LF_OCC, and SPO, although it failed to accurately estimate
disparities at the upper-left crevice. In the Dots scene, the proposed method
reconstructs more foreground points in the upper-left corner compared to the
lower-right corner, indicating a certain robustness to noise; however, as the
noise increases, the proposed method loses its advantage. In the Pyramids
scene, all algorithms accurately reconstructed the convex and concave
surfaces. The proposed method demonstrated some advantages in Stripe scene
with lower demands for texture contrast. Table I presents the quantitative
analyses of the scenes in Fig. 4 based on the BadPix(BP) and MSE metrics,
along with the size of the network model parameters. Across various scenarios,
the proposed method exhibited competitive advantages based on average
quantitative metrics. Regarding the Average BP01, BP03, BP07 metrics, the
proposed method shows lower misjudgment rates. Regarding the Network
parameters metric, the proposed method is particularly efficient in
lightweight design, having only 0.5M parameters, which makes it more suitable
for resource-constrained environments than methods such as LLFNet and
FastLFnet.

Although the proposed method maintains relatively low levels of BP and MSE
metrics in specific scenes, such as Stripes, there is still room for further
optimization, especially in handling complex textures or special structures.
The proposed method demonstrates stable performance across different scenes,
making it applicable to various scenarios.

###  IV-B Algorithm analysis and evaluation using real data experiments

Real light field data typically contain more noise, non-Lambertian radiance
data, and more complex occlusion scenarios. The Stanford Lytro light field
dataset [46] was captured using a Lytro Illum camera. In this study, the
proposed method was compared with the built-in algorithms of the Lytro camera,
MV, LF_OCC, SPO, and AMN, as shown in Fig. 5.

Figure 5: Disparity estimation results for data acquired using the Lytro
Illum, with zoomed-in regions shown in close-up mode.

In Fig. 5, the first two rows depict scenes with intricate occlusion
structures suitable for evaluating the performance of the algorithm in
handling complex scenarios. The proposed and other methods accurately
predicted the intricate branches in the foreground. LF_OCC exhibited
significant errors in the background region, SPO showed multiple protrusion
errors at the boundaries of the branches, and the AMN and MV algorithms were
smoothed out in detail, which can be observed in the magnified area indicated
by the red box in the second row. In Fig. 5, the foreground in the last two
rows is mainly comprised of iron fences. Apart from the built-in algorithm of
the Lytro camera, both the proposed and other algorithms accurately
reconstructed the iron fences in the foreground. However, in the background
region, the proposed algorithm predicted the least noise, which was
particularly notable in the scene reconstructed in the last row.

The experimental results demonstrate that the proposed method accurately
predicts the disparity structure in real-world scenes and reconstructs more
details in edge-occlusion areas.

###  IV-C Spatial super-resolution experiments

The proposed method parameterizes the continuous disparity, which provides a
unique advantage because a discrete system matrix does not limit it. This
allows the extraction of high-resolution disparity images from a well-trained
network, thereby satisfying the need for super-resolution disparity image
reconstruction. Figure 6 illustrates the differently sized reconstructed
images extracted from the trained network using the Dino scene. This indicates
that the quality of the disparity images was not compromised when extracting
high-resolution images from the network.

Figure 6: Experimental results on super-resolution: Using the Dino from the 4D
Light Field Benchmark [45]. (a) presents the reference view of the scene, and
(b) displays the ground truth. (c) exhibits disparity reconstructed at
resolutions of $512\times 512$, with (d) showing its profile curves at the
300th line. Panels (e) and (f) exhibit the disparity reconstructed at
resolutions of $1024\times 1024$ and $2048\times 2048$, respectively.

###  IV-D Ablation studies

This section validates the design and parameter selection through an ablation
study of the combination of loss functions and viewpoint selection loss
strategy.

The ablation study results presented in Fig. 7 and Table II provides evidence
for the rationale behind the proposed method of combining loss functions and
designing loss functions for viewpoint selection. Table II presents the
quantitative analysis results for the scenes depicted in Fig. 7. These scenes
included a grid-cloth background, a pear with a bee pattern, and a wooden
sphere with a black pattern. Analysis of the sphere’s disparity in the images
reveals that using only the L1 loss leads to noticeable layering and rough
surfaces on continuous surfaces. However, the predictions using both L1 and
MSSIM losses resulted in smoother surfaces. Furthermore, the inclusion of TV
regularization, which imposes sparse constraints on the gradient domain of the
disparity, leads to smoother surfaces.

Figure 7: Ablation experiments using the Dino from HCI [44]. The first row is the disparity, whereas the second row (excluding (a)) is the profile curves. The ground truth curves are marked in red in the profile plots, and those reconstructed by the proposed method are shown in blue. In column (a), the first row is the ground truth and the second row is the reference view of the scene. (b) The results using the L1 norm; (c) the results using both L1 and MSSIM losses. (d) and (e) show the results using L1 and MSSIM losses along with TV regularization. Panels (b)-(d) use only half of the viewpoints for backpropagation, whereas panel (e) utilizes all viewpoints. TABLE II: Quantitative analysis of ablation experiment, presented with BadPix0.03, BadPix0.05, BadPix0.07 and MSE ($\times$ 100) metrics (smaller scores are better). StillLife | MSE | BadPix0.07 | BadPix0.05 | BadPix0.03
---|---|---|---|---
L1 | 0.87 | 10.42 | 21.07 | 43.92
L1+MSSIM | 0.56 | 8.58 | 14.87 | 29.93
TV | 0.52 | 5.01 | 8.46 | 21.50
Full view | 1.55 | 7.28 | 9.60 | 16.32

As shown in Table II, the combination of MSSIM loss and TV constraint yielded
the lowest MSE and BadPixel values, indicating a higher reconstruction
accuracy. Figure 7 (d) applies a loss function based on viewpoint selection,
transmitting only half of the viewpoints during each backpropagation step,
whereas Fig. 7 (e) utilizes all the viewpoints. The profile curves in Fig. 7
demonstrate that, when using the complete set of viewpoints for
backpropagation, the occlusion caused by the wings of the bee cannot be
effectively removed. According to Table II, employing all viewpoints for
backpropagation (full view row) results in higher values for MSE, BadPix0.07,
and BadPix0.05 metrics compared to propagating only half of the viewpoints (TV
row), indicating poorer reconstruction quality. In disparity reconstruction
from light fields, occlusions and noise within a scene are the main sources of
error, although these occluded and noisy regions account for a relatively
small fraction of the overall scene. During the backpropagation process, the
use of more viewpoints when propagating unoccluded areas improves the
reconstruction accuracy. Consequently, regarding the BadPix0.03 metric,
employing all available viewpoints for backpropagation (full view row) yielded
lower values than using only half of the viewpoints (TV row), indicating
superior reconstruction quality.

##  V Discussion

We conducted comparative experiments on widely used light-field datasets to
validate the performance of the proposed method in disparity estimation. The
results demonstrate the exceptional proficiency of the proposed method in
addressing inclined surfaces, smooth textureless regions, and occlusions in
intricate scenes. When confronted with the weakly textured tail area of the
butterfly and occluded leaf boundaries in the Papillon scene, the proposed
method delivered a higher accuracy and smoother disparity estimates than MV,
LF_OCC, SPO, and AMN. Further validation of the method’s resilience and
adaptability was sought through tailored challenges in scenes such as
backgammons, dots, pyramids, and stripes, which tested its ability to handle
fine structures, noise interference, convex-concave geometries, and low-
texture contrast scenarios. Despite a slight decline in the performance under
high-noise conditions, the proposed method achieved effective disparity
reconstruction and detail preservation in most cases. Additionally,
experiments on the Stanford Lytro light field dataset demonstrated the success
of the proposed method in accurately discerning disparity structures, even in
highly noisy and complex occluded real-world scenes, particularly in
suppressing background noise and reconstructing foreground edge details
relative to contrastive algorithms. Moreover, by extracting disparities at
varying resolutions directly from a pre-trained NDF, the proposed approach
transcends the conventional limitations imposed by the reliance on discrete
matrix-based disparity representations. Finally, the ablation studies shed
light on the influence of combination strategies and selected viewpoints on
the loss function. The MSSIM loss curbed the spread of local pixel errors. In
contrast, including total variation regularization enforces constraints on the
disparity gradient domain, ensuring smooth and coherent estimation outcomes.
The capability to propagate with only half of the viewpoints bolsters the
resistance to occlusion and noise.

Although the proposed method demonstrated excellent performance in the above
experiments, the disparity map exhibited higher frequencies than the natural
images, particularly in edge regions with disparity differences, resulting in
oscillations in the predicted disparities. Thus, further research is required
to investigate parameterized network models for disparities to enhance
algorithm performance and applicability.

In summary, the proposed compact network model demonstrates efficiency and
robustness in handling weak textures, varying noise levels, and occlusion
scenarios. This could potentially provide more precise and reliable solutions
for light field depth estimation in practical applications.

##  VI Conclusions

We proposed a novel iterative approach to reconstruct the NDF that utilized
neural network parameterization to achieve an implicit, continuous disparity
function. The essential feature of this method is the implicit
parameterization of the disparity by neural fields, which overcomes the
resolution limitations inherent in traditional light-field disparity
estimation. Compared to conventional methods based on discrete disparity maps,
the proposed approach significantly reduced computational complexity while
maintaining high precision and demonstrating superior robustness, particularly
in handling complex scenes where occlusion and noise affect accurate disparity
reconstruction. The proposed method employed a differentiable framework to
construct a neural network model capable of modeling disparity continuously,
thereby effectively avoiding the loss of detail caused by discrete
representations. During the backpropagation optimization process, only a
subset of rays was selected for propagation computation in the viewpoint
direction, helping to mitigate occlusion effects and noise interference in the
disparity estimation. Experimental validation indicates that the reconstructed
NDF achieved accurate disparity on both synthetic and real datasets, enabling
the effective recovery of high-resolution disparity.

## References

  * [1] M. Levoy, “Light fields and computational imaging,” _Computer_ , vol. 39, no. 8, pp. 46–55, 2006.
  * [2] A. Mittal, “Neural radiance fields: Past, present, and future,” _arXiv preprint arXiv:2304.10050_ , 2023.
  * [3] A. Gershun, “The light field,” _Journal of Mathematics and Physics_ , vol. 18, no. 1-4, pp. 51–151, 1939.
  * [4] R. Ng, M. Levoy, M. Brédif, G. Duval, M. Horowitz, and P. Hanrahan, “Light field photography with a hand-held plenoptic camera,” Ph.D. dissertation, Stanford university, 2005.
  * [5] X. Xiao, B. Javidi, M. Martinez-Corral, and A. Stern, “Advances in three-dimensional integral imaging: sensing, display, and applications,” _Applied optics_ , vol. 52, no. 4, pp. 546–560, 2013.
  * [6] H.-G. Jeon, J. Park, G. Choe, J. Park, Y. Bok, Y.-W. Tai, and I. So Kweon, “Accurate depth map estimation from a lenslet light field camera,” in _Proceedings of the IEEE conference on computer vision and pattern recognition_ , 2015, pp. 1547–1555.
  * [7] S. Wanner and B. Goldluecke, “Variational light field analysis for disparity estimation and super-resolution,” _IEEE transactions on pattern analysis and machine intelligence_ , vol. 36, no. 3, pp. 606–619, 2013.
  * [8] S. Heber and T. Pock, “Shape from light field meets robust pca,” in _European Conference on Computer Vision_. Springer, 2014, pp. 751–767.
  * [9] M. W. Tao, S. Hadap, J. Malik, and R. Ramamoorthi, “Depth from combining defocus and correspondence using light-field cameras,” in _Proceedings of the IEEE International Conference on Computer Vision_ , 2013, pp. 673–680.
  * [10] T.-C. Wang, A. A. Efros, and R. Ramamoorthi, “Depth estimation with occlusion modeling using light-field cameras,” _IEEE transactions on pattern analysis and machine intelligence_ , vol. 38, no. 11, pp. 2170–2181, 2016.
  * [11] H. Sheng, P. Zhao, S. Zhang, J. Zhang, and D. Yang, “Occlusion-aware depth estimation for light field using multi-orientation epis,” _Pattern Recognition_ , vol. 74, pp. 587–599, 2018.
  * [12] S. Zhang, H. Sheng, C. Li, J. Zhang, and Z. Xiong, “Robust depth estimation for light field via spinning parallelogram operator,” _Computer Vision and Image Understanding_ , vol. 145, pp. 148–159, 2016.
  * [13] C. Liu, J. Qiu, and S. Zhao, “Iterative reconstruction of scene depth with fidelity based on light field data,” _Applied Optics_ , vol. 56, no. 11, pp. 3185–3192, 2017.
  * [14] C. Liu, L. Shi, X. Zhao, and J. Qiu, “Adaptive matching norm based disparity estimation from light field data,” _Signal Processing_ , vol. 209, p. 109042, 2023. [Online]. Available: https://www.sciencedirect.com/science/article/pii/S0165168423001160
  * [15] S. Heber and T. Pock, “Convolutional networks for shape from light field,” in _Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition_ , 2016, pp. 3746–3754.
  * [16] C. Shin, H.-G. Jeon, Y. Yoon, I. S. Kweon, and S. J. Kim, “Epinet: A fully-convolutional neural network using epipolar geometry for depth from light field images,” in _Proceedings of the IEEE conference on computer vision and pattern recognition_ , 2018, pp. 4748–4757.
  * [17] A. Alperovich, O. Johannsen, M. Strecke, and B. Goldluecke, “Light field intrinsics with a deep encoder-decoder network,” in _Proceedings of the IEEE conference on computer vision and pattern recognition_ , 2018, pp. 9145–9154.
  * [18] J. Peng, Z. Xiong, D. Liu, and X. Chen, “Unsupervised depth estimation from light field using a convolutional neural network,” in _2018 International Conference on 3D Vision (3DV)_. IEEE, 2018, pp. 295–303.
  * [19] W. Zhou, E. Zhou, G. Liu, L. Lin, and A. Lumsdaine, “Unsupervised monocular depth estimation from light field image,” _IEEE Transactions on Image Processing_ , vol. 29, pp. 1606–1617, 2020.
  * [20] K. Li, J. Zhang, R. Sun, X. Zhang, and J. Gao, “Epi-based oriented relation networks for light field depth estimation,” in _British Machine Vision Conference (BMVC)_ , 2020.
  * [21] Y.-J. Tsai, Y.-L. Liu, M. Ouhyoung, and Y.-Y. Chuang, “Attention-based view selection networks for light-field disparity estimation,” in _Proceedings of the 34th Conference on Artificial Intelligence (AAAI)_ , 2020\.
  * [22] Y. Li, Q. Wang, L. Zhang, and G. Lafruit, “A lightweight depth estimation network for wide-baseline light fields,” _IEEE Transactions on Image Processing_ , vol. 30, pp. 2288–2300, 2021.
  * [23] Z. Huang, X. Hu, Z. Xue, W. Xu, and T. Yue, “Fast light-field disparity estimation with multi-disparity-scale cost aggregation,” in _Proceedings of the IEEE/CVF International Conference on Computer Vision_ , 2021, pp. 6320–6329.
  * [24] B. Mildenhall, P. P. Srinivasan, M. Tancik, J. T. Barron, R. Ramamoorthi, and R. Ng, “Nerf: Representing scenes as neural radiance fields for view synthesis,” _Communications of the ACM_ , vol. 65, no. 1, pp. 99–106, 2021\.
  * [25] J. T. Barron, B. Mildenhall, M. Tancik, P. Hedman, R. Martin-Brualla, and P. P. Srinivasan, “Mip-nerf: A multiscale representation for anti-aliasing neural radiance fields,” in _Proceedings of the IEEE/CVF International Conference on Computer Vision_ , 2021, pp. 5855–5864.
  * [26] J. T. Barron, B. Mildenhall, D. Verbin, P. P. Srinivasan, and P. Hedman, “Mip-nerf 360: Unbounded anti-aliased neural radiance fields,” in _Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition_ , 2022, pp. 5470–5479.
  * [27] D. Verbin, P. Hedman, B. Mildenhall, T. Zickler, J. T. Barron, and P. P. Srinivasan, “Ref-nerf: Structured view-dependent appearance for neural radiance fields,” in _2022 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR)_. IEEE, 2022, pp. 5481–5490.
  * [28] T. Müller, A. Evans, C. Schied, and A. Keller, “Instant neural graphics primitives with a multiresolution hash encoding,” _ACM Transactions on Graphics (ToG)_ , vol. 41, no. 4, pp. 1–15, 2022.
  * [29] A. Chen, Z. Xu, A. Geiger, J. Yu, and H. Su, “Tensorf: Tensorial radiance fields,” in _European Conference on Computer Vision_. Springer, 2022, pp. 333–350.
  * [30] A. Chen, Z. Xu, X. Wei, S. Tang, H. Su, and A. Geiger, “Factor fields: A unified framework for neural fields and beyond,” _arXiv preprint arXiv:2302.01226_ , 2023.
  * [31] V. Sitzmann, J. Martel, A. Bergman, D. Lindell, and G. Wetzstein, “Implicit neural representations with periodic activation functions,” _Advances in neural information processing systems_ , vol. 33, pp. 7462–7473, 2020.
  * [32] S. Xie, H. Zhu, Z. Liu, Q. Zhang, Y. Zhou, X. Cao, and Z. Ma, “Diner: Disorder-invariant implicit neural representation,” in _Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition_ , 2023, pp. 6143–6152.
  * [33] B. Attal, J.-B. Huang, M. Zollhöfer, J. Kopf, and C. Kim, “Learning neural light fields with ray-space embedding,” in _Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition_ , 2022, pp. 19 819–19 829.
  * [34] Z. Li, L. Song, C. Liu, J. Yuan, and Y. Xu, “Neulf: Efficient novel view synthesis with neural 4d light field,” _arXiv preprint arXiv:2105.07112_ , 2021.
  * [35] B. Y. Feng and A. Varshney, “Signet: Efficient neural representation for light fields,” in _Proceedings of the IEEE/CVF International Conference on Computer Vision_ , 2021, pp. 14 224–14 233.
  * [36] J. J. Park, P. Florence, J. Straub, R. Newcombe, and S. Lovegrove, “Deepsdf: Learning continuous signed distance functions for shape representation,” in _Proceedings of the IEEE/CVF conference on computer vision and pattern recognition_ , 2019, pp. 165–174.
  * [37] L. Mescheder, M. Oechsle, M. Niemeyer, S. Nowozin, and A. Geiger, “Occupancy networks: Learning 3d reconstruction in function space,” in _Proceedings of the IEEE/CVF conference on computer vision and pattern recognition_ , 2019, pp. 4460–4470.
  * [38] Y. Xie, T. Takikawa, S. Saito, O. Litany, S. Yan, N. Khan, F. Tombari, J. Tompkin, V. Sitzmann, and S. Sridhar, “Neural fields in visual computing and beyond,” in _Computer Graphics Forum_. Wiley Online Library, 2022, pp. 641–676.
  * [39] L. Shi, C. Liu, D. He, X. Zhao, and J. Qiu, “Matching entropy based disparity estimation from light field data,” _Optics Express_ , vol. 31, no. 4, pp. 6111–6131, 2023.
  * [40] Z. Wang, A. C. Bovik, H. R. Sheikh, and E. P. Simoncelli, “Image quality assessment: from error visibility to structural similarity,” _IEEE transactions on image processing_ , vol. 13, no. 4, pp. 600–612, 2004.
  * [41] L. I. Rudin, S. Osher, and E. Fatemi, “Nonlinear total variation based noise removal algorithms,” _Physica D: nonlinear phenomena_ , vol. 60, no. 1-4, pp. 259–268, 1992.
  * [42] S. Fridovich-Keil, A. Yu, M. Tancik, Q. Chen, B. Recht, and A. Kanazawa, “Plenoxels: Radiance fields without neural networks,” in _Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition_ , 2022, pp. 5501–5510.
  * [43] J. N. Martel, D. B. Lindell, C. Z. Lin, E. R. Chan, M. Monteiro, and G. Wetzstein, “Acorn: Adaptive coordinate networks for neural scene representation,” _arXiv preprint arXiv:2105.02788_ , 2021.
  * [44] S. Wanner, S. Meister, and B. Goldluecke, “Datasets and benchmarks for densely sampled 4d light fields.” in _VMV_ , vol. 13. Citeseer, 2013, pp. 225–226.
  * [45] K. Honauer, O. Johannsen, D. Kondermann, and B. Goldluecke, “A dataset and evaluation methodology for depth estimation on 4d light fields,” in _Asian conference on computer vision_. Springer, 2016, pp. 19–34.
  * [46] S. R. Abhilash, L. Michael, S. Raj, and W. Gordon, “Stanford lytro light field database,” http://lightfields.stanford.edu/LF2016.html.
  * [47] O. Johannsen, K. Honauer, B. Goldluecke, A. Alperovich, F. Battisti, Y. Bok, M. Brizzi, M. Carli, G. Choe, M. Diebold, M. Gutsche, H. Jeon, I. S. Kweon, J. Park, J. Park, H. Schilling, H. Sheng, L. Si, M. Strecke, A. Sulc, Y. Tai, Q. Wang, T. Wang, S. Wanner, Z. Xiong, J. Yu, S. Zhang, and H. Zhu, “A taxonomy and evaluation of dense light field depth estimation algorithms,” in _2017 IEEE Conference on Computer Vision and Pattern Recognition Workshops (CVPRW)_ , 2017, pp. 1795–1812.