# MixNeRF: Modeling a Ray with Mixture Density
for Novel View Synthesis from Sparse Inputs

Seunghyeon Seo Donghoon Han∗ Yeonjin Chang∗ Nojun Kwak
Seoul National Univeristy
<EMAIL_ADDRESS>
###### Abstract

Neural Radiance Field (NeRF) has broken new ground in the novel view synthesis
due to its simple concept and state-of-the-art quality. However, it suffers
from severe performance degradation unless trained with a dense set of images
with different camera poses, which hinders its practical applications.
Although previous methods addressing this problem achieved promising results,
they relied heavily on the additional training resources, which goes against
the philosophy of sparse-input novel-view synthesis pursuing the training
efficiency. In this work, we propose MixNeRF, an effective training strategy
for novel view synthesis from sparse inputs by modeling a ray with a mixture
density model. Our MixNeRF estimates the joint distribution of RGB colors
along the ray samples by modeling it with mixture of distributions. We also
propose a new task of ray depth estimation as a useful training objective,
which is highly correlated with 3D scene geometry. Moreover, we remodel the
colors with regenerated blending weights based on the estimated ray depth and
further improves the robustness for colors and viewpoints. Our MixNeRF
outperforms other state-of-the-art methods in various standard benchmarks with
superior efficiency of training and inference.

##  1 Introduction

††footnotetext: ∗D. Han and Y. Chang equally contributed to this work. Figure
1: Comparison with the vanilla mip-NeRF [1] and other regularization methods.
Given the same number of training batch and iterations, our MixNeRF
outperforms mip-NeRF and DietNeRF [12] by a large margin with comparable or
shorter training time. Compared to RegNeRF [30], ours achieves superior
performance with about 42% shortened training time. The size of the circles
are proportional to the number of input views, indicating 3/6/9-view,
respectively. More details are provided in Sec. 4.2.

A photo-realistic view synthesis is one of the major research topics in
computer vision. Recently, the coordinate-based neural representation [6, 26,
27, 31] has gained much popularity for the novel view synthesis task. Among
them, Neural Radiance Field (NeRF) [29], which models a 3D scene by learning
from a dense set of 2D images, enabled high-quality view synthesis with a
simple concept and has become the prevailing mainstream. However, NeRF suffers
from severe performance degradation in real-world applications, _e.g_. AR/VR,
autonomous driving, and so on, where only a sparse set of views are available
due to the burdensome task of collecting dense training images.

One of the key factors for a model’s high-quality rendering with limited input
views is its robustness in 3D geometry learning, _i.e_. accurate depth
estimation for a scene. There are several works to address this problem and it
can be classified into two major paradigms: pre-training and regularization
approaches. For the pre-training approach [43, 5, 7, 38, 23, 13, 21, 32, 35,
16], a general 3D geometry is trained by the multi-view images from a large-
scale dataset and per-scene finetuning is optionally conducted in the test
time. Although it has achieved promising results, it still requires the
expensive cost for collecting a large-scale dataset across different scenes
for pre-training and is not well-generalized for a novel domain in the test
time.

Another line of research, the regularization approach [30, 9, 33, 12, 18, 40],
performs per-scene optimization from scratch by applying regularization to
prevent being overfitted from the limited training views. Most existing
methods of this kind depend heavily on the extra training resources for
compensating a lack of supervisory signals, _e.g_. depth-map generation by
running SfM [9, 33], unseen ray generation with arbitrary camera poses [18,
30], leveraging external modules to exploit additional features [12, 40, 30],
or so on. However, the additional training data might not always be available
and the external modules should be pre-trained with a large-scale dataset.
This is against the philosophy of novel view synthesis from sparse inputs,
pursuing training efficiency.

In this work, we propose MixNeRF, an effective regularization approach for
novel view synthesis from sparse inputs, modeling the colors along a ray with
a mixture density model which represents a complex distribution with a mixture
of component distributions. By exploiting the blending weights as mixing
coefficients for our mixture model, we are able to regularize effectively both
the colors and the densities of the samples along a ray. Furthermore, we
propose a new auxiliary task of ray depth estimation for learning the 3D
geometry which is crucial for the rendering quality. Since the estimated 3D
geometry is highly correlated with the scene depth estimation, our proposed
training objective acts as a useful supervisory signal. Finally, we regenerate
the blending weights based on the estimated ray depth and remodel a ray. Since
the estimated depth is not exactly the same, but nearly identical to the
ground truth, it can play a role of pseudo geometry for adjacent points of the
sample, like an unseen viewpoint. By remodeling the samples with the mixing
coefficients based on the regenerated blending weights, we can further improve
the robustness for shift of colors and viewpoints. Our main contributions are
summarized as follows:

  * •

Our method estimates the joint distribution of RGB color values along the ray
samples by a mixture of distributions, learning the 3D geometry successfully
with sparse views.

  * •

We propose a ray depth estimation as an effective auxiliary task for few-shot
novel view synthesis, playing a role of useful training objective.

  * •

We use the regenerated blending weights based on the estimated ray depths for
improving the robustness with negligible extra training cost.

  * •

Our MixNeRF outperforms other state-of-the-art methods in the different
standard benchmarks, showing much improved training and inference efficiency.

##  2 Related Works

###  2.1 Neural Scene Representations

Recently, coordinate-based neural representations [6, 26, 27, 31] have gained
a lot of popularity in the field of neural scene rendering [1, 2, 11, 15, 17,
29, 22, 24]. Among them, Neural Radiance Fields (NeRF) [29] have broken new
ground in the novel view synthesis research due to its wide possibility with
the simple concept and state-of-the-art quality. Since NeRF, several works
have been followed to ameliorate its drawbacks and improve the performance.
Mip-NeRF [1] tackled the problem of aliasing in NeRF by introducing cone
tracing method. Ref-NeRF [37] reparameterized NeRF from the view-dependent
outgoing radiance to reflected radiance, leading to significant improvement
for specular reflections.

However, these methods suffer from severe performance degradation unless
trained with a set of dense images with different camera poses, which hinders
their practical applications. In this work, we address the sparse input
scenario which is closer to the real-world condition. We are able to perform
high-quality view synthesis from sparse inputs by modeling a ray with a
mixture density model and improve both the training and the inference
efficiency.

Figure 2: Overview of MixNeRF. Our method models the color and depth of a ray
with a mixture of distributions, and remodels the color based on the estimated
ray depth $\mu^{d}$. The dotted rays indicate the imaginary views
corresponding to $\mu^{d}$. See Sec. 3 for more details.

###  2.2 Sparse Input Novel View Synthesis

One of the fundamental causes of performance degradation is the lack of 3D
geometry information from training images, resulting in an inaccurate depth
estimation. There are two major paradigms to tackle this problem in the novel
view synthesis from sparse inputs: pre-training and regularization approaches.
The former approach [43, 5, 7, 38, 23, 13, 21, 32, 35, 16] provides prior
knowledges to conditional models through pre-training. The image features
extracted by a CNN feature extractor [43, 7] or a 3D cost volume obtained by
image warping [5, 16] are used for training a generalizable model. Although
they achieved promising performances under the sparse input setting, a large-
scale dataset of multi-view images with different scenes is required for pre-
training, which is burdensome to collect. Furthermore, despite the lengthy
pre-training phase, most of these methods require additional test-time fine-
tuning and are apt to suffer from quality degradation on different data
domains.

The regularization approach [30, 9, 33, 12, 18, 40] introduces extra
supervision to regularize the color and the geometry without an expensive pre-
training process. Additional training resources, _e.g_. external modules such
as CLIP [12] or a pre-trained normalizing flow model [10], extra depth inputs
obtained by running structure-from-motion (SfM), and additional rays of unseen
viewpoints, are often used to provide abundant supervisory signals. However,
the existing methods are overly dependent on the extra training resources
which might not always be available, hampering data/time efficiency. Moreover,
it goes against the philosophy of the sparse-input novel-view synthesis which
pursues the training efficiency.

Our proposed method requires neither an external module nor an additional
inference of extra supervisory signals, such as additional depth inputs or
pre-generated rays from unobserved viewpoints, resulting in a more efficient
training framework.

###  2.3 Mixture Density Model

There exists a line of research utilizing a mixture density model in different
tasks of computer vision [36, 41, 42, 20, 8, 34]. Among 3D vision tasks, Tosi
_et al_. [34] proposed a novel stereo-matching framework, SMD-Nets, tackling
the over-smoothing problem of output representations by leveraging a mixture
density network [3]. Choi _et al_. [8] reformulated 3D bounding box regression
as a density estimation problem using a Gaussian Mixture Model (GMM),
achieving a more efficient 3D object detection framework with few heuristic
design factors.

Although the mixture density model shows a great potential in 3D vision tasks,
there has not been an attempt to utilize it in the NeRF framework for novel
view synthesis. Our MixNeRF is able to learn the 3D geometry successfully,
which is a critical factor for rendering quality under the sparse input
setting, by modeling a ray with a mixture of distributions.

##  3 Method

In this work, we propose a novel training framework of neural radiance fields
for novel view synthesis from sparse inputs. We build our MixNeRF upon mip-
NeRF [1] which uses a multiscale scene representation (Sec. 3.1). Moreover, we
leverage the mixture density model framework to learn 3D geometry efficiently.
More specifically, we model the colors of samples along a ray by a mixture of
Laplace distributions with the predicted weights as mixing coefficients, which
contributes to learning a scene’s geometry effectively with limited input
views (Sec. 3.2). Furthermore, we estimate the depths of input rays as an
auxiliary task and reuse it for producing blending weights once again as
supplemental training resources, which enables robust rendering from unseen
viewpoints with little additional burden for training (Sec. 3.3). In the
training phase, our MixNeRF is not only trained to minimize the mean squared
error (MSE) between predictions and GT colors, but also to maximize the
likelihood of colors and depths for each ray (Sec. 3.4). Fig. 2 demonstrates
an overview of our MixNeRF.

###  3.1 Preliminary: Neural Radiance Field

NeRF [29] represents a 3D scene with a continuous function, where a neural
network $f(\cdot,\cdot)$ consisting of an MLP maps a 3D location
$\mathbf{x}=(x,y,z)$ and viewing direction $(\theta,\phi)$, which is expressed
as a 3D Cartesian unit vector $\mathbf{\bar{d}}$ in practice, along rays to
colors $\mathbf{c}=(r,g,b)$ and volume density $\sigma$:

$f(\gamma(\mathbf{x}),\gamma(\bar{\mathbf{d}}))\to(\mathbf{c},\sigma),$ (1)

where $\gamma(\cdot)$ indicates the positional encoding applied to the inputs
$(\mathbf{x},\bar{\mathbf{d}})$. Following the volume rendering theory [25], a
pixel on an image is rendered by alpha compositing the colors and densities
along the ray $\mathbf{r}(t)=\mathbf{o}+t\mathbf{d}$ cast from the camera
origin $\mathbf{o}$, where $\mathbf{d}$ is the unnormalized direction vector,
_i.e_. $\mathbf{d}=\lVert\mathbf{d}\rVert_{2}\cdot\bar{\mathbf{d}}$. The
volume rendering integrals are approximated by the quadrature rule in practice
[29] as follows:

$\begin{split}\mathbf{\hat{c}(r)}=\sum_{i=1}^{N}T_{i}(1-\exp(-\sigma_{i}\delta_{i}))\mathbf{c}_{i},\\\
\text{where}\quad
T_{i}=\exp(-\sum_{j=1}^{i-1}\sigma_{j}\delta_{j}).\end{split}$ (2)

Note that $N$ and $\delta_{i}=\lVert\mathbf{d}\rVert_{2}\cdot(t_{i+1}-t_{i})$
denote the number of samples and the interval between the $i$-th sample and
its adjacent one, respectively. To improve rendering efficiency, the two-stage
hierarchical sampling is performed: coarse and fine stage. The points are
sampled uniformly along a ray in the coarse stage, and then more informed
samples are generated in the fine stage based on the density estimated from
the coarse stage. Finally, the radiance field is optimized by minimizing the
MSE between the rendered color and ground truth color over the input images:

$\mathcal{L}_{\text{MSE}}=\sum_{\mathbf{r}\in\mathcal{R}}||\mathbf{\hat{c}(r)}-\mathbf{c^{\text{GT}}(r)}||^{2}_{2}\,,$
(3)

where $\mathcal{R}$ indicates a set of input rays.

Following RegNeRF [30], we adopt the mip-NeRF [1] representation for our
MixNeRF. Mip-NeRF effectively alleviates the aliasing problem of NeRF by
introducing a cone tracing method and an integrated positional encoding.

###  3.2 Modeling a Ray with Mixture Density Model

Given a set of input rays
$\mathcal{R}=\\{\mathbf{r}_{1},\cdots,\mathbf{r}_{K}\\}$ on training images
with the ground truths $G=\\{G_{1},\cdots,G_{K}\\}$ for each of $K$ pixels,
the $i$-th ground truth $G_{i}$ consists of the RGB color values
$\mathbf{c}_{i}^{\text{GT}}=\\{r_{i},g_{i},b_{i}\\}$ and the unnormalized 3D
ray vector $\mathbf{d}_{i}^{\text{GT}}$, _i.e_.
$G_{i}\triangleq\\{\mathbf{c}_{i}^{\text{GT}},\mathbf{d}_{i}^{\text{GT}}\\}$.
Note that $\mathbf{d}^{\text{GT}}$ is the direction vector corresponding to
$t=1$ from the camera center. First, our MixNeRF estimates the distribution of
the RGB color values $\mathbf{c}_{i}$ along the samples of the ray
$\mathbf{r}_{i}$ on a pixel with a mixture model, which is derived from a
weighted combination of component distributions. As shown in Fig. 2, in our
model, $(\mathbf{c},\sigma)$, the conventional outputs of NeRF for each
sampled point $\mathbf{r}(t)$, are used as a location parameter
$\mu^{\mathbf{c}}$ and to compute a mixing coefficient $\pi$, respectively. In
addition to these, a scale parameter
$\beta=\\{\beta^{r},\beta^{g},\beta^{b}\\}$ is also estimated in our model.

We assume that every element of $\mathbf{c}_{i}$ is independent of each other
to simplify our mixture model formulation. Therefore, the $j$-th component’s
probability density function (pdf) corresponding to the $j$-th sampled point
for the $i$-th ray $\mathbf{r}_{i}$ is as follows:

$\displaystyle\begin{split}\mathcal{F}(\mathbf{c};\mu^{\mathbf{c}}_{ij},\beta_{ij})&=\prod_{c\in\\{r,g,b\\}}\mathcal{F}(c;\mu^{c}_{ij},\beta^{c}_{ij})\\\
&=\prod_{c\in\\{r,g,b\\}}\frac{1}{2\beta^{c}_{ij}}\exp\left(-\frac{|c-\mu^{c}_{ij}|}{\beta^{c}_{ij}}\right),\end{split}$
(4)

where $\mathcal{F}$ denotes the Laplacian pdf. The pdf of our mixture model
formed by the component distributions above is defined as:

$p(\mathbf{c}|\mathbf{r}_{i})=\sum_{j=1}^{M}\pi_{ij}\mathcal{F}(\mathbf{c};\mu_{ij}^{\mathbf{c}},\beta_{ij}),$
(5)

where $M$ denotes the number of mixture components which is the same as the
number of samples along a ray. The mixture coefficient $\pi_{ij}$ is derived
from the density output $\sigma_{ij}$ as follows:

$\pi_{j}=\frac{w_{j}}{\sum_{m=1}^{M}w_{m}}=\frac{T_{j}(1-\exp(-\sigma_{j}\delta_{j}))}{\sum_{m=1}^{M}T_{m}(1-\exp(-\sigma_{m}\delta_{m}))}.$
(6)

Note that we omitted ray index $i$ for simplicity. Here, $w_{j}$ and
$\delta_{j}$ indicate the weight for the alpha compositing and the sample
interval, respectively. Since the mixture components corresponding to the
samples with higher weights, which contribute more to the alpha composition of
the color than other samples, are likely to have higher $\pi$, we use the
normalized weight as a mixing coefficient $\pi$ so that
$\sum_{j=1}^{M}\pi_{j}=1$.

The concept of a mixture model corresponds to that of alpha compositing in
that a complex multimodal distribution is able to be represented by the
weighted combination of component distributions with mixing coefficients
$\pi$, like a pixel value derived from the weighted combination of estimated
RGB values along ray samples with blending weights $w$. Motivated by this
conceptual similarity, we are able to model a ray with a mixture of
distributions successfully without any heuristic factors. The mixing
coefficients derived from the blending weights provide effective supervisory
signals toward the densities, which are the core factor for successfully
learning 3D scene geometry with limited input views.

|  mip-NeRF [1] |  RegNeRF [30] |  MixNeRF (Ours) |  mip-NeRF [1] (All-view)
---|---|---|---|---

Figure 3: Comparison of blending weight distributions. Compared to the
baselines, ours estimates the modes of weight distributions more accurately,
leading to the precise 3D geometry. The ideal distributions on the blue and
red points are unimodal and bimodal, respectively.

###  3.3 Depth Estimation by Mixture Density Model

We propose a scene’s depth estimation as an effective auxiliary task for
training our MixNeRF with sparse inputs. As demonstrated in Fig. 2, our
MixNeRF estimates $d$, the ray’s depth, which is defined as the length of the
unnormalized ray direction vector $\mathbf{d}$, _i.e_.
$d\triangleq\lVert\mathbf{d}\rVert$, along the ray samples. The ground truth
$G_{i}$ contains the ray direction values $\mathbf{d}_{i}$ as well, which are
used in the form of 3D Cartesian unit vectors
$\mathbf{\bar{d}}_{i}=\mathbf{d}_{i}/\lVert\mathbf{d}_{i}\rVert_{2}$ as an
input viewing direction in practice. Like the RGB color values, the depths for
each ray are modeled by our mixture model consisting of the Laplace
distributions with the same scale parameters $\beta$ and mixing coefficients
$\pi$ used above. The pdf of our mixture model for the depth of the $i$-th ray
is as follows:

$p(d|\mathbf{r}_{i})=\sum_{j=1}^{M}\pi_{ij}\mathcal{F}(d;\mu^{d}_{ij},\beta_{ij}).$
(7)

Since the mixing coefficient $\pi$ and parameter $\beta$ are optimized through
the supervision of the depth as well as the color values, it improves the
robustness of our MixNeRF for slight changes of geometry. Also, considering
that the successful depth estimation is crucial to the rendered images’
quality in a NeRF model [30, 9, 33, 18], our direct estimation of the scene’s
depth benefits a lot.

#### Blending weight regeneration.

In addition, we exploit the estimated depth to regenerate the blending weights
along the samples and model the RGB color values by a mixture of distributions
once again. Since the estimated depth of each sample is trained to be nearly
identical to the ground truth depth, but not exactly the same, it can play a
role of pseudo geometry for adjacent points of the sample without any
additional pre-generation process of extra training data, _e.g_. depth inputs
made by SfM or rays from unobserved viewpoints. The new blending weight
$\hat{w}_{j}$ of the $j$-th sample along a ray based on the estimated depth
$\mu_{{\color[rgb]{1,0,0}\definecolor[named]{pgfstrokecolor}{rgb}{1,0,0}j}}^{d}$
are defined as follow:

$\displaystyle\hat{w}_{j}=\hat{T}_{j}(1-\exp(-\sigma_{j}\hat{\delta}_{j})),\
\hat{\delta}_{j}=\mu^{d}_{j}(t_{j+1}-t_{j}),$ (8)

in which we replace $\lVert\mathbf{d}\rVert_{2}$ in $\delta_{j}$ formulation
with $\mu_{j}^{d}$. Finally, we model the color values along a ray based on
the new mixing coefficients $\hat{\pi}$ derived from $\hat{w}$ and the
corresponding pdf is as follows:

$\hat{p}(\mathbf{c}|\mathbf{r}_{i})=\sum_{j=1}^{M}\hat{\pi}_{ij}\mathcal{F}(\mathbf{c};\mu_{ij}^{\mathbf{c}},\beta_{ij}).$
(9)

Since the estimated ray depths are likely to be close enough to those of the
ground truths, we use the same GT color values of input rays for modeling the
mixture distribution based on the newly generated $\hat{\pi}$. It further
improves the robustness for shift of colors and ray viewpoints by simply
modeling a ray once again with regenerated blending weights, eliminating pre-
generation and extra inference of unseen views without much computational
overhead.

###  3.4 Total Loss

Our MixNeRF is trained to maximize the likelihood of $\mathbf{c}^{\text{GT}}$
and $d^{\text{GT}}$ for a set of input rays $\mathcal{R}$ as well as to
minimize the $\mathcal{L}_{\text{MSE}}$. Therefore, the loss functions can be
simply defined to minimize the negative log-likelihood (NLL) of the ground
truths as follows:

$\begin{array}[]{cl}\mathcal{L}^{C}_{\text{NLL}}=-\sum_{\mathbf{r}\in\mathcal{R}}\log
p(\mathbf{c}^{\text{GT}}|\mathbf{r}),\\\
\mathcal{L}^{D}_{\text{NLL}}=-\sum_{\mathbf{r}\in\mathcal{R}}\log
p(d^{\text{GT}}|\mathbf{r}),\\\
\hat{\mathcal{L}}^{C}_{\text{NLL}}=-\sum_{\mathbf{r}\in\mathcal{R}}\log\hat{p}(\mathbf{c}^{\text{GT}}|\mathbf{r}),\end{array}$
(10)

each of which corresponds to the NLL form of Eq. 5, Eq. 7 and Eq. 9,
respectively. As a result, we define our total loss as:

$\mathcal{L}_{\text{total}}=\mathcal{L}_{\text{MSE}}+\lambda_{C}\mathcal{L}_{\text{NLL}}^{C}+\lambda_{D}\mathcal{L}_{\text{NLL}}^{D}+\hat{\lambda}_{C}\hat{\mathcal{L}}_{\text{NLL}}^{C},$
(11)

where $\lambda_{C}$, $\lambda_{D}$ and $\hat{\lambda}_{C}$ are balancing terms
for the losses. More details about training and implementation are provided in
the supplementary material.

##  4 Experiments

###  4.1 Experimental Details

#### Datasets and metrics.

We evaluate MixNeRF on the multiple standard benchmarks: DTU [14], LLFF [28]
and Realistic Synthetic 360∘ [29]. DTU consists of images containing objects
located on a white table with a black background. LLFF contains real forward-
facing scenes and is usually used as an out-of-distribution test set for pre-
training methods. We also compare our MixNeRF against other regularization
methods on the Realistic Synthetic 360∘, which provides 8 synthetic scenes
each consisting of 400 images rendered from inward-facing cameras with various
viewpoints. We follow the overall experimental protocols of [43, 29, 18, 12]
for these datasets.

For the evaluation metrics, we adopt the mean of PSNR, structural similarity
index (SSIM) [39], LPIPS perceptual metric [44], and the geometric average
[1]. Kindly refer to the supp. mat. for more details about datasets and
metrics.

#### Baselines.

We compare our method against several representative pre-training and
regularization approaches [43, 7, 5, 12, 30, 18] as well as the vanilla mip-
NeRF [1]. We report the evaluation results from [30] for DTU and LLFF, which
are superior to those from their original papers due to the improved training
curriculum. The DTU dataset is used as a pre-training resources for PixelNeRF
[43], MVSNeRF [5], and SRF [7], and the LLFF dataset serves as an out-of-
domain test set. The regularization approaches and mip-NeRF are trained for
each scene without pre-training. For Realistic Synthetic 360∘, we train other
regularization approaches [18, 12, 30] by their training schemes. Note that
the pre-trained RealNVP [10] for training RegNeRF [30] is not publicly
available and we report the results of RegNeRF trained without it on the
Realistic Synthetic 360∘. For the analysis of MixNeRF (Sec. 4.2), all models
including ours are trained with the same batch size and iterations.

RegNeRF [30] |  MixNeRF (Ours)
---|---

(a) LLFF 3-view

(b) DTU 3-view

(c) Realistic Synthetic 360∘ 4-view

Figure 4:  Comparison of estimated depth map. We compare MixNeRF against
RegNeRF, the state-of-the-art regularization approach. Our MixNeRF estimates
more accurate depth maps and captures fine details better, leading to high-
quality rendering with more distinct edges and less artifacts.

###  4.2 Analysis of MixNeRF

#### Benefit of mixture density model.

We leverage a mixture density model, which represents a complex multimodal
distribution with a weighted combination of component distributions, to learn
the distribution of density and colors along the ray samples effectively. Fig.
3 demonstrates the comparison of the blending weight distributions of cast
rays on the LLFF fern scene of 3-view scenario. We compare ours against mip-
NeRF and RegNeRF, with mip-NeRF trained from all training views as an ideal
distribution. For the unimodal distribution in blue, mip-NeRF does not
estimate the mode well and achieves degenerate geometry. However, RegNeRF and
our MixNeRF show the unimodal weight distributions leading to higher-quality
novel views, and especially our MixNeRF achieves the distribution with sharper
mode than RegNeRF, which is more similar to that of mip-NeRF (All-view). In
case of the bimodal-shaped distribution in red, our MixNeRF estimates the
weight distribution successfully while both mip-NeRF and RegNeRF fail to
estimate the accurate modes. Since the predicted 3D geometry is directly
correlated with how well the density is estimated, our MixNeRF is able to
learn the geometry more efficiently with limited input views through mixture
density modeling.

Depth map estimation.  We compare our MixNeRF against RegNeRF, which utilizes
the prior of depth smoothness to learn 3D geometry, on the multiple
benchmarks. As illustrated in Fig. 4, our MixNeRF estimates more accurate
depth maps with distinct edges while RegNeRF generates both RGB images and
depth maps with blurry fine details. Especially for Realistic Synthetic 360∘,
we observe that RegNeRF fails to learn the geometry with its smoothing
strategy and achieves degenerate results due to the overly strong prior of
depth smoothness. However, since our MixNeRF learns the depth of a ray by
leveraging a mixture density model without smoothing from additional unseen
rays, the depth maps are predicted much more efficiently and precisely.

# of samples | 128 | 64 | 32 | 16 | 8
---|---|---|---|---|---
mip-NeRF [1] | 0.332 | 0.331 | 0.329 | 0.308 | 0.251
RegNeRF [30] | 0.587 | 0.585 | 0.576 | 0.522 | 0.379
MixNeRF (Ours) | 0.629 | 0.629 | 0.620 | 0.580 | 0.468

Table 1:  Comparison with baselines by the number of ray samples. Our MixNeRF
with 75% fewer samples (32-sample) outperforms RegNeRF with default
128-sample, and still achieves comparable results with only 16-sample
($\times$8 reduction).

| Approach | PSNR $\uparrow$ | SSIM $\uparrow$ | LPIPS $\downarrow$ | Average Error $\downarrow$
---|---|---|---|---|---
| 3-view | 6-view | 9-view | 3-view | 6-view | 9-view | 3-view | 6-view | 9-view | 3-view | 6-view | 9-view
LLFF.
mip-NeRF [1] | N/A | 14.62 | 20.87 | 24.26 | 0.351 | 0.692 |  yellow!250.805 | 0.495 | 0.255 |  yellow!250.172 | 0.246 | 0.114 |  yellow!250.073
PixelNeRF [43] | Pre-training | 7.93 | 8.74 | 8.61 | 0.272 | 0.280 | 0.274 | 0.682 | 0.676 | 0.665 | 0.461 | 0.433 | 0.432
PixelNeRF ft [43] | 16.17 | 17.03 | 18.92 | 0.438 | 0.473 | 0.535 | 0.512 | 0.477 | 0.430 | 0.217 | 0.196 | 0.163
SRF [7] | 12.34 | 13.10 | 13.00 | 0.250 | 0.293 | 0.297 | 0.591 | 0.594 | 0.605 | 0.313 | 0.293 | 0.296
SRF ft [7] | 17.07 | 16.75 | 17.39 | 0.436 | 0.438 | 0.465 | 0.529 | 0.521 | 0.503 | 0.203 | 0.207 | 0.193
MVSNeRF [5] | 17.25 | 19.79 | 20.47 | 0.557 | 0.656 | 0.689 | 0.356 | 0.269 | 0.242 | 0.171 | 0.125 | 0.111
MVSNeRF ft [5] |  yellow!2517.88 | 19.99 | 20.47 |  yellow!250.584 | 0.660 | 0.695 |  orange!250.327 | 0.264 | 0.244 |  yellow!250.157 | 0.122 | 0.111
DietNeRF [12] |  Regularization | 14.94 |  yellow!2521.75 |  yellow!2524.28 | 0.370 |  yellow!250.717 | 0.801 | 0.496 |  yellow!250.248 | 0.183 | 0.240 |  yellow!250.105 |  yellow!250.073
RegNeRF [30] |  orange!2519.08 |  orange!2523.10 |  orange!2524.86 |  orange!250.587 |  orange!250.760 |  orange!250.820 |  yellow!250.336 |  orange!250.206 |  orange!250.161 |  orange!250.146 |  orange!250.086 |  orange!250.067
MixNeRF (Ours) |  red!2519.27 |  red!2523.76 |  red!2525.20 |  red!250.629 |  red!250.791 |  red!250.833 |  red!250.236 |  red!250.115 |  red!250.087 |  red!250.124 |  red!250.066 |  red!250.052
DTU.
mip-NeRF [1] | N/A | 8.68 | 16.54 | 23.58 | 0.571 | 0.741 |  orange!250.879 | 0.353 | 0.198 |  yellow!250.092 | 0.323 | 0.148 |  yellow!250.056
PixelNeRF [43] | Pre-training | 16.82 | 19.11 | 20.40 | 0.695 | 0.745 | 0.768 | 0.270 | 0.232 | 0.220 | 0.147 | 0.115 | 0.100
PixelNeRF ft [43] |  red!2518.95 | 20.56 | 21.83 | 0.710 | 0.753 | 0.781 | 0.269 | 0.223 | 0.203 |  yellow!250.125 | 0.104 | 0.090
SRF [7] | 15.32 | 17.54 | 18.35 | 0.671 | 0.730 | 0.752 | 0.304 | 0.250 | 0.232 | 0.171 | 0.132 | 0.120
SRF ft [7] | 15.68 | 18.87 | 20.75 | 0.698 | 0.757 | 0.785 | 0.281 | 0.225 | 0.205 | 0.162 | 0.114 | 0.093
MVSNeRF [5] |  yellow!2518.63 |  yellow!2520.70 | 22.40 |  red!250.769 |  yellow!250.823 |  yellow!250.853 |  orange!250.197 | 0.156 | 0.135 |  orange!250.113 |  yellow!250.088 | 0.068
MVSNeRF ft [5] | 18.54 | 20.49 | 22.22 |  red!250.769 | 0.822 |  yellow!250.853 |  orange!250.197 |  yellow!250.155 | 0.135 |  orange!250.113 | 0.089 | 0.069
DietNeRF [12] |  Regularization | 11.85 | 20.63 |  yellow!2523.83 | 0.633 | 0.778 | 0.823 | 0.314 | 0.201 | 0.173 | 0.243 | 0.101 | 0.068
RegNeRF [30] |  orange!2518.89 |  orange!2522.20 |  orange!2524.93 |  orange!250.745 |  red!250.841 |  red!250.884 |  red!250.190 |  orange!250.117 |  orange!250.089 |  red!250.112 |  orange!250.071 |  orange!250.047
MixNeRF (Ours) |  red!2518.95 |  red!2522.30 |  red!2525.03 |  yellow!250.744 |  orange!250.835 |  orange!250.879 |  yellow!250.203 |  red!250.102 |  red!250.065 |  orange!250.113 |  red!250.066 |  red!250.042

Table 2:  Quantitative results on LLFF and DTU. Our method achieves comparable
or state-of-the-art performance across all scenarios in LLFF and DTU. For
LLFF, our MixNeRF outperforms both pre-training and regularization baselines
without any burdensome extra training resources. Likewise, MixNeRF achieves
competitive results against the state-of-the-art methods on DTU dataset. ft
indicates fine-tuning.

Efficiency in training and inference.  Our MixNeRF improves the efficiency for
both the training and the inference phases by learning the 3D geometry
effectively without burdensome extra training resources. Fig. 1 illustrates
that MixNeRF achieves superior performance with reduced training time among
the vanilla mip-NeRF and two representative regularization methods on the
LLFF. For a fair comparison, we compare the methods based on the identical JAX
codebase [4] using the same batch size and iterations on 2 NVIDIA TITAN RTX.
Although it takes a similar amount of time to train DietNeRF as MixNeRF, its
performance is inferior significantly to ours in 3 and 6-view scenario.
Compared to RegNeRF, ours outperforms it with about 42% shorter training time
per scene under the same number of input view scenario, resulting from the
elimination of extra inference for additional unseen rays. Furthermore, we
also observe that our MixNeRF shows better data efficiency requiring up to
about 60% fewer inputs than mip-NeRF to achieve comparable results, and
outperforms mip-NeRF consistently in more than 9-view scenarios. It indicates
that our proposed training strategy is effective in general scenarios as well
as the sparse input setting. The related experimental results are provided in
the suppl. material. For the inference efficiency, Tab. 1 demonstrates the
SSIM results by the number of samples along a ray on the LLFF under the 3-view
scenario. Our MixNeRF with 32-sample outperforms RegNeRF with default
128-sample, and still achieves comparable results with only 16-sample thanks
to the capacity of our mixture model for representing the blending weight
distributions successfully.

| $\mathcal{L}_{\text{NLL}}^{C}$ | $\mathcal{L}_{\text{NLL}}^{D}$ | $\hat{\mathcal{L}}_{\text{NLL}}^{C}$ | PSNR $\uparrow$ | SSIM $\uparrow$ | LPIPS $\downarrow$ | Average $\downarrow$
---|---|---|---|---|---|---|---
(1) | Baseline (mip-NeRF† [1]) | 13.72 | 0.665 | 0.273 | 0.209
(2) | ✓ |  |  | 17.44 |  yellow!250.729 |  red!250.187 |  yellow!250.130
(3) | ✓ | ✓ |  |  orange!2518.35 |  orange!250.739 | 0.209 |  orange!250.121
(4) | ✓ |  | ✓ |  yellow!2517.45 | 0.728 |  orange!250.191 | 0.134
(5) | ✓ | ✓ | ✓ |  red!2518.95 |  red!250.744 |  yellow!250.203 |  red!250.113

(1) |  (2) |  (3) |  (4) |  (5)
---|---|---|---|---

Figure 5:  Ablation study. $\dagger$ uses a scene annealing strategy.

###  4.3 Ablation Study

We report the quantitative and qualitative results of our ablation study in
Fig. 5. We observe that modeling a ray with mixture of distributions is
helpful for improving performance under the sparse view setting ((1)
$\rightarrow$ (2)). Also, our proposed ray depth estimation task contributes
to further improving the rendering quality by generating more accurate depth
maps ((2) $\rightarrow$ (3)). However, despite the well-estimated depth map,
the RGB image suffers from the foggy artifacts upon the objects as shown in
(3). By remodeling a ray through the weight regeneration process, our MixNeRF
achieves high-quality of both RGB image and depth map ((3) $\rightarrow$ (5)).
Since the regenerated weights are not helpful without a supervision toward the
ray depth estimation task ((2) $\rightarrow$ (4)), our proposed auxiliary task
of ray depth estimation is useful for learning 3D geometry and playing a role
of additional training resources by weight regeneration process on-the-fly.

###  4.4 Comparison with other SOTA methods

| PSNR $\uparrow$ | SSIM $\uparrow$ | LPIPS $\downarrow$ | Average Err. $\downarrow$
---|---|---|---|---
4-view | 8-view | 4-view | 8-view | 4-view | 8-view | 4-view | 8-view
mip-NeRF [1] | 14.12 | 18.74 | 0.720 | 0.830 | 0.380 | 0.240 | 0.220 | 0.120
DietNeRF [12] |  yellow!2515.42 |  yellow!2521.31 | 0.730 |  yellow!250.847 |  yellow!250.314 |  yellow!250.153 |  yellow!250.201 |  yellow!250.086
InfoNeRF [18] |  orange!2518.44 |  orange!2522.01 |  orange!250.792 |  orange!250.852 |  orange!250.223 |  orange!250.133 |  orange!250.119 |  orange!250.073
RegNeRF [30] | 13.71 | 19.11 |  yellow!250.786 | 0.841 | 0.346 | 0.200 | 0.210 | 0.122
MixNeRF (Ours) |  red!2518.99 |  red!2523.84 |  red!250.807 |  red!250.878 |  red!250.199 |  red!250.103 |  red!250.113 |  red!250.060

Table 3:  Quantitative results on Realistic Synthetic 360∘. MixNeRF
outperforms other state-of-the-art regularization methods by a large margin.

|  mip-NeRF [1] |  DietNeRF [12] |  RegNeRF [30] |  MixNeRF (Ours) |  Ground Truth |
---|---|---|---|---|---|---

(a) LLFF 3-view

(b) LLFF 6-view

|  mip-NeRF [1] |  PixelNeRF [43] |  RegNeRF [30] |  MixNeRF (Ours) |  Ground Truth |
---|---|---|---|---|---|---

(c) DTU 3-view

(d) DTU 6-view

|  mip-NeRF [1] |  DietNeRF [12] |  InfoNeRF [18] |  RegNeRF [30] |  MixNeRF (Ours) |  Ground Truth |
---|---|---|---|---|---|---|---

(e) Realistic Synthetic 360∘ 4-view

Figure 6:  Qualitative results on LLFF (a,b), DTU (c,d), and Realistic
Synthetic 360∘ (e). More results are provided in suppl. material.

#### LLFF.

As shown in Tab. 2, the pre-training approaches except MVSNeRF are not able to
achieve comparable results without fine-tuning for the 3-view scenario. The
regularization approaches and vanilla mip-NeRF outperform the pre-training
approaches in 6 and 9-view settings. Especially, RegNeRF improves the
rendering quality by a large margin compared to mip-NeRF and DietNeRF, thanks
to the regularization strategy using pre-generated rays from unseen
viewpoints. Our MixNeRF achieves state-of-the-art results across all scenarios
and metrics without any pre-training process using a large-scale dataset or
extra inference for pre-generated training resources. It improves the training
efficiency, _e.g_. requiring about 42% shorter training time than RegNeRF in
the same input view setting (see Fig. 1). Furthermore, as demonstrated in Fig.
6, we observe that our method achieves more realistic results with fine
details than RegNeRF since MixNeRF learns the 3D geometry successfully without
an explicit regularization for smoothing the depth.

DTU.  Since the pre-training approaches are conditioned from the prior
knowledge of DTU, they achieve competitive results with and without fine-
tuning as demonstrated in Tab. 2. Our MixNeRF achieves comparable or better
results than other baselines. Compared to the PSNR, our MixNeRF achieves worse
quantitative results than RegNeRF for SSIM. We conjecture that since RegNeRF
uses an explicit smoothing term for regularizing the depth, it can achieve
slightly better quantitative results of SSIM which is a patch-wise evaluation
metric. However, as shown in Fig. 6, our MixNeRF renders clearer images,
capturing fine details better than other baselines despite a little worse
quantitative results for some metrics. More qualitative results of LLFF and
DTU are provided in the suppl. material.

Realistic Synthetic 360∘.  As shown in Tab. 3, our MixNeRF outperforms other
regularization baselines across all settings and metrics by a large margin.
Fig. 6(e) illustrates that other methods suffer from severe floating artifacts
and degenerate colors in the 4-view scenario. Especially, the depth smoothing
strategy of RegNeRF rather brings about the significant performance
degradation. It implies that smoothing is not a fundamental solution
universally effective for different datasets. Compared to the baselines, our
MixNeRF achieves superior rendering quality with much less artifacts and more
accurate geometry in both 4 and 8-view (see the suppl. material) scenarios.

##  5 Conclusion

We have introduced MixNeRF, a novel regularization approach for training NeRF
in the limited data scenario. However, previous approaches heavily depend on
the extra training resources, which goes against the philosophy of sparse-
input novel-view synthesis pursuing the efficiency of training. To overcome
this bottleneck, we propose modeling a ray with mixture density, which enables
effective learning of 3D geometry with sparse inputs. Furthermore, our novel
training strategy, consisting of an auxiliary ray depth estimation and the
following weight regeneration, further improves the rendering quality and
better reconstructs 3D geometry by more accurate depth estimation without any
extra training resources that should be prepared in advance. Our proposed
MixNeRF outperforms both pre-training and regularization approaches across the
multiple benchmarks with an enhanced efficiency of training and inference.

## References

  * [1] Jonathan T Barron, Ben Mildenhall, Matthew Tancik, Peter Hedman, Ricardo Martin-Brualla, and Pratul P Srinivasan.  Mip-nerf: A multiscale representation for anti-aliasing neural radiance fields.  In Proceedings of the IEEE/CVF International Conference on Computer Vision, pages 5855–5864, 2021.
  * [2] Alexander Bergman, Petr Kellnhofer, and Gordon Wetzstein.  Fast training of neural lumigraph representations using meta learning.  Advances in Neural Information Processing Systems, 34:172–186, 2021\.
  * [3] Christopher M Bishop.  Mixture density networks.  1994\.
  * [4] James Bradbury, Roy Frostig, Peter Hawkins, Matthew James Johnson, Chris Leary, Dougal Maclaurin, George Necula, Adam Paszke, Jake VanderPlas, Skye Wanderman-Milne, et al.  Jax: composable transformations of python+ numpy programs.  Version 0.2, 5:14–24, 2018.
  * [5] Anpei Chen, Zexiang Xu, Fuqiang Zhao, Xiaoshuai Zhang, Fanbo Xiang, Jingyi Yu, and Hao Su.  Mvsnerf: Fast generalizable radiance field reconstruction from multi-view stereo.  In Proceedings of the IEEE/CVF International Conference on Computer Vision, pages 14124–14133, 2021.
  * [6] Zhiqin Chen and Hao Zhang.  Learning implicit fields for generative shape modeling.  In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 5939–5948, 2019.
  * [7] Julian Chibane, Aayush Bansal, Verica Lazova, and Gerard Pons-Moll.  Stereo radiance fields (srf): Learning view synthesis for sparse views of novel scenes.  In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 7911–7920, 2021.
  * [8] Jaeseok Choi, Yeji Song, Yerim Kim, Jaeyoung Yoo, and Nojun Kwak.  Md3d: Mixture-density-based 3d object detection in point clouds.  IEEE Access, 2022.
  * [9] Kangle Deng, Andrew Liu, Jun-Yan Zhu, and Deva Ramanan.  Depth-supervised nerf: Fewer views and faster training for free.  In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 12882–12891, 2022.
  * [10] Laurent Dinh, Jascha Sohl-Dickstein, and Samy Bengio.  Density estimation using real nvp.  arXiv preprint arXiv:1605.08803, 2016.
  * [11] Chen Gao, Yichang Shih, Wei-Sheng Lai, Chia-Kai Liang, and Jia-Bin Huang.  Portrait neural radiance fields from a single image.  arXiv preprint arXiv:2012.05903, 2020.
  * [12] Ajay Jain, Matthew Tancik, and Pieter Abbeel.  Putting nerf on a diet: Semantically consistent few-shot view synthesis.  In Proceedings of the IEEE/CVF International Conference on Computer Vision, pages 5885–5894, 2021.
  * [13] Wonbong Jang and Lourdes Agapito.  Codenerf: Disentangled neural radiance fields for object categories.  In Proceedings of the IEEE/CVF International Conference on Computer Vision, pages 12949–12958, 2021.
  * [14] Rasmus Jensen, Anders Dahl, George Vogiatzis, Engin Tola, and Henrik Aanæs.  Large scale multi-view stereopsis evaluation.  In Proceedings of the IEEE conference on computer vision and pattern recognition, pages 406–413, 2014.
  * [15] Yue Jiang, Dantong Ji, Zhizhong Han, and Matthias Zwicker.  Sdfdiff: Differentiable rendering of signed distance fields for 3d shape optimization.  In Proceedings of the IEEE/CVF conference on computer vision and pattern recognition, pages 1251–1261, 2020.
  * [16] Mohammad Mahdi Johari, Yann Lepoittevin, and François Fleuret.  Geonerf: Generalizing nerf with geometry priors.  In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 18365–18375, 2022.
  * [17] Petr Kellnhofer, Lars C Jebe, Andrew Jones, Ryan Spicer, Kari Pulli, and Gordon Wetzstein.  Neural lumigraph rendering.  In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 4287–4297, 2021.
  * [18] Mijeong Kim, Seonguk Seo, and Bohyung Han.  Infonerf: Ray entropy minimization for few-shot neural volume rendering.  In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 12912–12921, 2022.
  * [19] Diederik P Kingma and Jimmy Ba.  Adam: A method for stochastic optimization.  arXiv preprint arXiv:1412.6980, 2014.
  * [20] Chen Li and Gim Hee Lee.  Generating multiple hypotheses for 3d human pose estimation with mixture density network.  In Proceedings of the IEEE/CVF conference on computer vision and pattern recognition, pages 9887–9895, 2019.
  * [21] Jiaxin Li, Zijian Feng, Qi She, Henghui Ding, Changhu Wang, and Gim Hee Lee.  Mine: Towards continuous depth mpi with nerf for novel view synthesis.  In Proceedings of the IEEE/CVF International Conference on Computer Vision, pages 12578–12588, 2021.
  * [22] Lingjie Liu, Jiatao Gu, Kyaw Zaw Lin, Tat-Seng Chua, and Christian Theobalt.  Neural sparse voxel fields.  Advances in Neural Information Processing Systems, 33:15651–15663, 2020.
  * [23] Yuan Liu, Sida Peng, Lingjie Liu, Qianqian Wang, Peng Wang, Christian Theobalt, Xiaowei Zhou, and Wenping Wang.  Neural rays for occlusion-aware image-based rendering.  In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 7824–7833, 2022.
  * [24] Ricardo Martin-Brualla, Noha Radwan, Mehdi SM Sajjadi, Jonathan T Barron, Alexey Dosovitskiy, and Daniel Duckworth.  Nerf in the wild: Neural radiance fields for unconstrained photo collections.  In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 7210–7219, 2021.
  * [25] Nelson Max.  Optical models for direct volume rendering.  IEEE Transactions on Visualization and Computer Graphics, 1(2):99–108, 1995.
  * [26] Lars Mescheder, Michael Oechsle, Michael Niemeyer, Sebastian Nowozin, and Andreas Geiger.  Occupancy networks: Learning 3d reconstruction in function space.  In Proceedings of the IEEE/CVF conference on computer vision and pattern recognition, pages 4460–4470, 2019.
  * [27] Mateusz Michalkiewicz, Jhony K Pontes, Dominic Jack, Mahsa Baktashmotlagh, and Anders Eriksson.  Implicit surface representations as layers in neural networks.  In Proceedings of the IEEE/CVF International Conference on Computer Vision, pages 4743–4752, 2019.
  * [28] Ben Mildenhall, Pratul P Srinivasan, Rodrigo Ortiz-Cayon, Nima Khademi Kalantari, Ravi Ramamoorthi, Ren Ng, and Abhishek Kar.  Local light field fusion: Practical view synthesis with prescriptive sampling guidelines.  ACM Transactions on Graphics (TOG), 38(4):1–14, 2019.
  * [29] Ben Mildenhall, Pratul P Srinivasan, Matthew Tancik, Jonathan T Barron, Ravi Ramamoorthi, and Ren Ng.  Nerf: Representing scenes as neural radiance fields for view synthesis.  Communications of the ACM, 65(1):99–106, 2021.
  * [30] Michael Niemeyer, Jonathan T Barron, Ben Mildenhall, Mehdi SM Sajjadi, Andreas Geiger, and Noha Radwan.  Regnerf: Regularizing neural radiance fields for view synthesis from sparse inputs.  In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 5480–5490, 2022.
  * [31] Jeong Joon Park, Peter Florence, Julian Straub, Richard Newcombe, and Steven Lovegrove.  Deepsdf: Learning continuous signed distance functions for shape representation.  In Proceedings of the IEEE/CVF conference on computer vision and pattern recognition, pages 165–174, 2019.
  * [32] Konstantinos Rematas, Ricardo Martin-Brualla, and Vittorio Ferrari.  Sharf: Shape-conditioned radiance fields from a single view.  In International Conference on Machine Learning, pages 8948–8958. PMLR, 2021.
  * [33] Barbara Roessle, Jonathan T Barron, Ben Mildenhall, Pratul P Srinivasan, and Matthias Nießner.  Dense depth priors for neural radiance fields from sparse input views.  In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 12892–12901, 2022.
  * [34] Fabio Tosi, Yiyi Liao, Carolin Schmitt, and Andreas Geiger.  Smd-nets: Stereo mixture density networks.  In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 8942–8952, 2021.
  * [35] Alex Trevithick and Bo Yang.  Grf: Learning a general radiance field for 3d representation and rendering.  In Proceedings of the IEEE/CVF International Conference on Computer Vision, pages 15182–15192, 2021.
  * [36] Prune Truong, Martin Danelljan, Luc Van Gool, and Radu Timofte.  Learning accurate dense correspondences and when to trust them.  In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 5714–5724, 2021.
  * [37] Dor Verbin, Peter Hedman, Ben Mildenhall, Todd Zickler, Jonathan T Barron, and Pratul P Srinivasan.  Ref-nerf: Structured view-dependent appearance for neural radiance fields.  In 2022 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), pages 5481–5490. IEEE, 2022.
  * [38] Qianqian Wang, Zhicheng Wang, Kyle Genova, Pratul P Srinivasan, Howard Zhou, Jonathan T Barron, Ricardo Martin-Brualla, Noah Snavely, and Thomas Funkhouser.  Ibrnet: Learning multi-view image-based rendering.  In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 4690–4699, 2021.
  * [39] Zhou Wang, Alan C Bovik, Hamid R Sheikh, and Eero P Simoncelli.  Image quality assessment: from error visibility to structural similarity.  IEEE transactions on image processing, 13(4):600–612, 2004.
  * [40] Dejia Xu, Yifan Jiang, Peihao Wang, Zhiwen Fan, Humphrey Shi, and Zhangyang Wang.  Sinnerf: Training neural radiance fields on complex scenes from a single image.  arXiv preprint arXiv:2204.00928, 2022.
  * [41] Jaeyoung Yoo, Hojun Lee, Inseop Chung, Geonseok Seo, and Nojun Kwak.  Training multi-object detector by estimating bounding box distribution for input image.  In Proceedings of the IEEE/CVF International Conference on Computer Vision, pages 3437–3446, 2021.
  * [42] Jaeyoung Yoo, Hojun Lee, Seunghyeon Seo, Inseop Chung, and Nojun Kwak.  Sparse mdod: Training end-to-end multi-object detector without bipartite matching.  arXiv preprint arXiv:2205.08714, 2022.
  * [43] Alex Yu, Vickie Ye, Matthew Tancik, and Angjoo Kanazawa.  pixelnerf: Neural radiance fields from one or few images.  In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 4578–4587, 2021.
  * [44] Richard Zhang, Phillip Isola, Alexei A Efros, Eli Shechtman, and Oliver Wang.  The unreasonable effectiveness of deep features as a perceptual metric.  In Proceedings of the IEEE conference on computer vision and pattern recognition, pages 586–595, 2018.

| Anneal. | LLFF [28] | DTU [14] | Real. Syn. 360∘ [29]
---|---|---|---|---
| 3-view | 6-view | 9-view | 3-view | 6-view | 9-view | 4-view | 8-view
$\lambda_{C}$ | ✓ | $[4.0,1\mathrm{e}{-3}]$
$\lambda_{D}$ |  | $1\mathrm{e}{-4}$ | $1\mathrm{e}{-5}$ | $1\mathrm{e}{-6}$ | $1\mathrm{e}{-3}$ | $1\mathrm{e}{-4}$ | $1\mathrm{e}{-5}$ | $1\mathrm{e}{-3}$ | $1\mathrm{e}{-4}$
$\hat{\lambda}_{C}$ |  | $1\mathrm{e}{-5}$ | $1\mathrm{e}{-6}$ | $1\mathrm{e}{-7}$ | $1\mathrm{e}{-4}$ | $1\mathrm{e}{-5}$ | $1\mathrm{e}{-6}$ | $1\mathrm{e}{-4}$ | $1\mathrm{e}{-5}$

Table A:  Overview of our loss balancing weights. We apply a linear annealing
strategy for $\lambda_{C}$ to stabilize the training. We divide $\lambda_{D}$
and $\hat{\lambda}_{C}$ by a factor of 10 as more input views are provided for
training.

##  A Implementation Details

###  A.1 Hyperparameters

Following RegNeRF [30], we adopt a scene space annealing during the early
training stage, an exponential learning rate decay from $2\mathrm{e}{-3}$ to
$2\mathrm{e}{-5}$, and 512 steps of warm up [1] with a delay multiplier of
$1\mathrm{e}{-2}$. For the Realistic Synthetic 360∘ [29], we set the initial
learning rate as $1\mathrm{e}{-3}$ and apply an exponential decay to
$1\mathrm{e}{-5}$. The Adam [19] optimizer is used and the gradient clippings
are applied by value at 0.1 and norm at 0.1 in order. We train our MixNeRF for
500 pixel epochs with 4096 batch size on 2 NVIDIA TITAN RTX, and the training
time is measured on the same hardware. For the balancing hyperparameters for
our loss terms, we anneal $\lambda_{C}$ from $4.0$ to $1\mathrm{e}{-3}$ over
the first 512 iterations, while setting $\lambda_{D}$ and $\hat{\lambda}_{C}$
as different values by the datasets. Tab. A shows the overview of balancing
terms by the datasets and the number of input views.

###  A.2 Architecture

Figure A: MixNeRF Network Architecture. The architecture of MixNeRF is
implemented based upon the mip-NeRF [1]. It additionally outputs the scale
parameter $\beta$ using softplus activation and the ray depths
$\mu^{d}=\lVert\tilde{\mu}^{d}\rVert_{2}$, which are denoted in red.
$\mathbf{b}$ indicates a bottleneck vector.

Our MixNeRF is based on the architecture of mip-NeRF [1]. As illustrated in
Fig. A, our MixNeRF additionally outputs the scale parameters $\beta$ using
softplus activation and the ray depths $\mu^{d}$ for our mixture model. In
practice, we estimate the unnormalized ray directions
$\tilde{\mu}^{d}\in\mathbb{R}^{\text{N}\times 3}$, where $N$ indicates the
number of samples, and we use its Euclidean norm
$\mu^{d}=\lVert\tilde{\mu}^{d}\rVert_{2}$ as the estimated ray depths for the
training stability.

##  B Experimental Details

###  B.1 Datasets

We evaluate MixNeRF on the different standard benchmarks: LLFF [28], DTU [14],
and Realistic Synthetic 360∘ [29].

#### LLFF:

It contains realistic forward-facing scenes and is generally used as an out-
of-domain test set for pre-training methods. Following the protocol of [29],
every 8-th image is used as a held-out test set and input views are chosen
evenly from the remaining images. We report the results under the scenarios of
3, 6, and 9 input views following [43].

#### DTU:

It consists of images containing objects placed on a white table with a black
background. We follow the experimental protocol of [43] and conduct
experiments on their designated 15 scenes. As with the LLFF dataset, we
conduct the experiments under the scenarios of 3, 6, and 9-view.

#### Realistic Synthetic 360∘:

It consists of 8 inward-facing synthetic scenes with different viewpoints,
each containing 400 images. Following previous works [18, 12], we conduct the
experiments for the scenarios of 4 and 8 views. For a fair comparison with
other regularization methods, we sample the first 4 and 8 images from the
training set for the scenario of 4 and 8 input views, respectively, for all
models and use the 200 test set images for evaluation. Note that the images of
the training set are arranged randomly in the first place, and we do not
choose the training input views carefully for improving the performance.

###  B.2 Evaluation Metrics

We adopt a set of evaluation metrics including the mean of PSNR, structural
similarity index (SSIM) [39], and LPIPS perceptual metric [44]. Additionally,
we report its geometric average [1]: $\text{MSE}=10^{-\text{PSNR}/10}$,
$\sqrt{1-\text{SSIM}}$, and LPIPS. Following [30], we adopt masked metrics to
avoid background bias for DTU.

##  C Data Efficiency Experiment

Figure B:  Comparison with baseline by the number of input views. Our MixNeRF
requires up to about 60% fewer input views than mip-NeRF to achieve comparable
performance, and outperforms mip-NeRF consistently even when more input views
are used for training. Since the reduced test set is used for the experiment
following [30], the results can be slightly different from the main table.

As demonstrated in Fig. B, we observe that our MixNeRF achieves superior data
efficiency to the vanilla mip-NeRF. Our MixNeRF requires up to about 60% fewer
input views to mip-NeRF to achieve comparable results. Moreover, ours
outperforms mip-NeRF consistently even when more than 9 input views are
provided. It indicates that our proposed mixture modeling strategy is
effective in general scenarios as well as the sparse input setting.

##  D Additional Qualitative Results

|  mip-NeRF [1] |  DietNeRF [12] |  RegNeRF [30] |  MixNeRF (Ours) |  Ground Truth |
---|---|---|---|---|---|---

(a) 3-view

(b) 6-view

(c) 9-view

Figure C:  Additional qualitative comparisons on LLFF.

|  mip-NeRF [1] |  PixelNeRF [43] |  RegNeRF [30] |  MixNeRF (Ours) |  Ground Truth |
---|---|---|---|---|---|---

(a) 3-view

(b) 6-view

(c) 9-view

Figure D:  Additional qualitative comparisons on DTU.

|  mip-NeRF [1] |  DietNeRF [12] |  InfoNeRF [18] |  RegNeRF [30] |  MixNeRF (Ours) |  Ground Truth |
---|---|---|---|---|---|---|---

(a) 4-view

(b) 8-view

Figure E:  Additional qualitative comparisons on Realistic Synthetic 360∘.

(a) 3-view

(b) 6-view

(c) 9-view

Figure F:  Additional qualitative results of our MixNeRF on LLFF.

(a) 3-view

(b) 6-view

(c) 9-view

Figure G:  Additional qualitative results of our MixNeRF on DTU.

(a) 4-view

(b) 8-view

Figure H:  Additional qualitative results of our MixNeRF on Realistic
Synthetic 360∘.

We demonstrate the additional qualitative comparisons in Fig. C, Fig. D, and
Fig. E. Moreover, we show the additional qualitative results of our MixNeRF in
Fig. F, Fig. G, and Fig. H.

##  E Limitations and Future Work

Our MixNeRF achieves the state-of-the-art performance without any extra
training resources, _e.g_. additional inference for pre-generated rays from
unseen viewpoints, external modules for providing supplemental supervisory
signals, or so on. However, it still shows a few degenerate parts in the
rendered images under the very sparse scenario as few as 3-view, due to the
disturbance from the non-objects, _e.g_. a background or a table, especially
on the DTU dataset. To eliminate the artifacts more effectively, developing an
algorithm for classifying the pixels into an object or non-object can be a
promising future work.

##  F Potential Negative Societal Impact

Our method is able to synthesize a photo-realistic image from novel view from
the limited training resources. Although it provides much benefits for
practical applications where the dense training resources are hard to collect,
there exists a possibility of negative consequences with malicious intents,
_e.g_. a misleading content made with an intent to either conceal or show some
specific views. Therefore, the effort to prevent the malicious usage should be
made, _e.g_. strictly checking on the permission to use sensitive data, deep
fake detection, and so on.


2}\;\left(\;\frac{\textstyle\pi}{\textstyle
2}\;-\;|\epsilon|\;\right)\;\sin|\epsilon|}\;=\;\frac{\tan|\epsilon|}{1\;-\;\left(\;\frac{\textstyle\pi}{\textstyle
2}\;-\;|\epsilon|\;\right)\;\tan|\epsilon|}\;\;\;.~{}~{}~{}~{}~{}$ (117)

As can be seen from (117), both the product $\,\sin\epsilon\,\cos\epsilon\,$
and the appropriate component of the torque attain their maxima when
$\,Q\,=\,1\,-\,\pi/4\,$.

Usually, $\,|\epsilon|\,$ is small, and we arrive at the customary expression

$\displaystyle Q^{-1}\,=\,\tan|\epsilon|\;+\;O(\epsilon^{2})\;\;\;.$ (118)

In the opposite case, when $Q\rightarrow 0$ and $|\epsilon|\rightarrow\pi/2$,
it is convenient to employ the small difference

$\displaystyle\xi\;\equiv\;\frac{\pi}{2}\;-\;|\epsilon|\;\;\;,$ (119)

in terms whereof the inverse quality factor will read:

$\displaystyle
Q^{-1}\,=\,\frac{\cot\xi}{1\;-\;\xi\;\cot\xi}\;=\;\frac{1}{\tan\xi\;-\;\xi}\;=\;\frac{1}{z\;-\;\arctan
z}\;=\;\frac{1}{\frac{\textstyle 1}{\textstyle
3}\;z^{3}\;\left[\,1\;-\;\frac{\textstyle 3}{\textstyle
5}\;z^{2}\,+\;O(z^{4})\,\right]}\;\;\;,\;\;\;$ (120)

where $\;z\,\equiv\,\tan\xi\;$ and, accordingly, $\;\xi\;=\;\arctan
z\;=\;z\,-\frac{\textstyle 1}{\textstyle 3}\,z^{3}\,+\,\frac{\textstyle
1}{\textstyle 5}\,z^{5}\,+\,O(z^{7})\;\,.\;$ Formula (120) may, of course, be
rewritten as

$\displaystyle
z^{3}\;\left[\,1\;-\;\frac{3}{5}\;z^{2}\;+\;O(z^{4})\;\right]\;=\;3\;Q\;\;\;$
(121)

or, the same, as

$\displaystyle
z\;=\;(3\,Q)^{1/3}\;\left[\,1\;+\;\frac{1}{5}\;z^{2}\;+\;O(z^{4})\,\right]\;\;\;.$
(122)

While the zeroth approximation is simply $\;z\,=\,(3Q)^{1/3}\,+\,O(Q)\;$, the
first iteration gives:

$\displaystyle\tan\xi\;\equiv\;z\;=\;(3Q)^{1/3}\,\left[\,1\;+\;\frac{1}{5}\;(3Q)^{2/3}\;+\;O(Q^{4/3})\,\right]\;=\;q\;\left[\,1\;+\;\frac{1}{5}\;q^{2}\;+\;O(q^{4})\,\right]\;\;\;,~{}~{}~{}$
(123)

with $\,q\,=\,(3Q)^{1/3}\,$ playing the role of a small parameter.

We now see that the customary relation (118) should be substituted, for large
lags, i.e., for small262626 The afore-employed expansion of $\,\arctan z\,$ is
valid for $\,|z|\,<\,1\,$. This inequality, along with (120), entails:
$\,Q\,=\,z\,-\,\arctan z\,<\,1\,-\,\pi/4\,$. values of $\,Q\,$, with:

$\displaystyle\tan|\epsilon|\;=\;(3Q)^{-1/3}\,\left[\,1\;-\;\frac{1}{5}\;(3Q)^{2/3}\;+\;O(Q^{4/3})\,\right]$
(124)

The formula for the tidal torque contains a multiplier
$\,\sin\epsilon\,\cos\epsilon\,$, whose absolute value can, for our purposes,
be written down as

$\displaystyle\sin|\epsilon|\,\cos|\epsilon|=\cos\xi\,\sin\xi=\frac{\tan\xi}{1+\tan^{2}\xi}\,=\,\frac{q\,\left[1+\frac{\textstyle
1}{\textstyle
5}\,q^{2}+O(q^{4})\right]}{1+q^{2}\left[1+O(q^{2})\right]}=(3Q)^{1/3}\left[1-\frac{4}{5}(3Q)^{2/3}+O(Q^{4/3})\right]\;,~{}~{}~{}$
(125)

whence

$\displaystyle\sin\epsilon\;\cos\epsilon\;=\;\pm\;(3Q)^{1/3}\left[1-\frac{4}{5}(3Q)^{2/3}+O(Q^{4/3})\right]\;,~{}~{}~{}$
(126)

an expression vanishing for $\,Q\,\rightarrow\,0\;$. Be mindful that both
$\,\epsilon_{\textstyle{{}_{2200}}}\,$ and the appropriate component of the
torque change their sign on the satellite crossing the synchronous orbit.

## References

  * [1] Alexander, M. E. 1973. “The weak-friction approximation and tidal evolution in close binary systems.” _Astrophysics and Space Sciences_ , Vol. 23, pp. 459 - 510
  * [2] Bills, B. G., Neumann, G. A., Smith, D.E., & Zuber, M.T. 2005\. “Improved estimate of tidal dissipation within Mars from MOLA observations of the shadow of Phobos.” Journal of Geophysical Research, Vol. 110, pp. 2376 - 2406. doi:10.1029/2004JE002376, 2005
  * [3] Churkin, V. A. 1998. “The Love numbers for the models of inelastic Earth.” Preprint No 121. Institute of Applied Astronomy. St.Petersburg, Russia. /in Russian/
  * [4] Correia, A. C. M., & Laskar, J. 2004. “Mercury’s capture into the $\,3/2\,$ spin-orbit resonance as a result of its chaotic dynamics.” _Nature_ , Vol. 429, pp. 848 - 850
  * [5] Darwin, G. H. 1879. “On the precession of a viscous spheroid and on the remote history of the Earth” Philosophical Transactions of the Royal Society of London, Vol.170, pp 447 -530
http://www.jstor.org/view/02610523/ap000081/00a00010/

  * [6] Darwin, G. H. 1880. “On the secular change in the elements of the orbit of a satellite revolving about a tidally distorted planet.” Philosophical Transactions of the Royal Society of London, Vol. 171, pp. 713 - 891
http://www.jstor.org/view/02610523/ap000082/00a00200

  * [7] Darwin, G. H. 1908. “Tidal friction and cosmogony.” In: Darwin, G. H., _Scientific Papers,_ Vol.2. Cambridge University Press, NY 1908.
  * [8] Dobrovolskis, A. 1995. “Chaotic rotation of Nereid?” _Icarus_ , Vol. 118, pp. 181 - 195
  * [9] Dobrovolskis, A. 2007. “Spin states and climates of eccentric exoplanets.” _Icarus_ , Vol. 192, pp. 1 - 23
  * [10] Efroimsky, M. 2006. The theory of bodily tides. The models and the physics.
astro-ph/0605521

  * [11] Efroimsky, M., & V. Lainey. 2007. “The Physics of Bodily Tides in Terrestrial Planets, and the Appropriate Scales of Dynamical Evolution.” _Journal of Geophysical Research – Planets_ , Vol. 112, p. E12003
doi:10.1029/2007JE002908

  * [12] Efroimsky, M. 2008. “Can the tidal quality factors of terrestrial planets and moons scale as positive powers of the tidal frequency?”
arXiv:0712.1056

  * [13] Ferraz-Mello, S., Rodríguez, A., & Hussmann, H. 2008. “Tidal friction in close-in satellites and exoplanets: The Darwin theory re-visited.” _Celestial mechanics and Dynamical Astronomy,_ Vol. 101, pp. 171 - 201
  * [14] Gerstenkorn, H. 1955. “Über Gezeitenreibung beim Zweikörperproblem.” Zeitschrift für Astrophysik, Vol. 36, pp. 245 - 274
  * [15] Getino, J., Escapa, A., & García, A. 2003. “Spheroidal and Toroidal Modes for Tidal Kinetic Energy in Axisymmetric, Slightly Elliptical, Elastic Bodies.” _Romanian Astronomical Journal_ , Vol. , pp. 143 - 161
  * [16] Goldreich, P. 1966a. “History of the Lunar Orbit.” _Reviews of Geophysics_. Vol. 4, pp. 411 - 439
  * [17] Goldreich, P. 1966b. “Final spin states of planets and satellites.” _The Astronomical Journal_. Vol. 4, pp. 411 - 439
  * [18] Goldreich, P., & Peale, S. 1966. “Spin-orbit coupling in the Solar System.” _The Astronomical Journal_. Vol. 71, pp. 425 - 438
  * [19] Gooding, R. H., & Wagner, C. A. 2008. “On the inclination functions and a rapid stable procedure for their evaluation together with derivatives.” _Celestial Mechanics and Dynamical Astronomy_ , Vol. 101, pp. 247 - 272
  * [20] Gurfil, P., Lainey, V., & Efroimsky, M. 2007. “Long-term evolution of orbits about a precessing oblate planet: 3. A semianalytical and a purely numerical approach.” _Celestial Mechanics and Dynamical Astronomy_ , Vol. 99, pp. 261 - 292
  * [21] Herschel, J. F. W. 1863. “About Volcanoes and Earthquakes.” _Good Words_ , 4 February 1863, pp. 53 - 58.
Reprinted in: Herschel, J. F. W. 1866. _Familiar Lectures on Scientific
Subjects,_ pp. 1 - 46. Alexander Strahan Publishers, London & NY, 1866

  * [22] Hut, P. 1981. “Tidal evolution in close binary systems.” _Astronomy & Astrophysics_, Vol. 99, pp. 126 - 140
  * [23] Innanen, K. A., Zheng, J. Q., Mikkola, S., & Valtonen, M. J. 1997, “The Kozai Mechanism and the Stability of Planetary Orbits in Binary Star Systems”, The Astronomical Journal, Vol. 113, pp. 1915 - 1919.
  * [24] Johnson, Samuel. 1755. _A Dictionary of the English Language, in which the words are deduced from their originals, and illustrated with their different significations by examples from the best writers; to which are prefixed a history of the language, and an English grammar._ London. Printed by W. Strahan, for J. and P. Knapton, T. and T. Longman, C. Hitch, L. Hawes, A. Millar, R. and J. Dodsley. 1755. Folio.
  * [25] Kant, I. 1754. “Untersuchung der Frage, ob die Erde in ihrer Umdrehung um die Achse, wodurch sie die Abwechselung des Tages und der Nacht hervorbringt, einige Veränderung seit den ersten Zeiten ihres Ursprungs erlitten habe und woraus man sich ihrer versichern könne, welche von der Königl.” Akademie der Wissenschaften zu Berlin zum Preise für das jetztlaufende Jahr aufgegeben worden. In: _Kant’s gesammelte Schriften._ , Vol. I, pp. 183 - 191. Ed by the Royal Prussian Academy of Sciences, Georg Reimer Publishers, Berlin 1900
http://www.ikp.uni-bonn.de/Kant/aa01/Inhalt1.html
English translations:
Kant, I. 1754. _“Essay on the Retardation of the Rotation of the Earth.”_
Translation by William Hastie, in: Hastie, W. 1900. “Kant’s Cosmogony, as in
his _Essay on the Retardation of the Rotation of the Earth_ and his _Natural
History and Theory of the Heavens._ pp. 157 - 165. J Maclehose Publishers,
Glasgow, 1900\. Reprinted: 1968, ed. Willy Ley (Greenwood Publishers, NY), and
1969, ed. Milton K. Munitz (University of Michigan Press).

  * [26] Karato, S.-i. 2007. _Deformation of Earth Materials. An Introduction to the Rheology of Solid Earth._ Cambridge University Press, UK.
  * [27] Kaula, W. M. 1961. “Analysis of gravitational and geometric aspects of geodetic utilisation of satellites.” The Geophysical Journal, Vol. 5, pp. 104 - 133
  * [28] Kaula, W. M. 1964. “Tidal Dissipation by Solid Friction and the Resulting Orbital Evolution.” Reviews of Geophysics, Vol. 2, pp. 661 - 684
  * [29] Kaula, W. M. 1966. _Theory of Satellite Geodesy: Applications of Satellites to Geodesy._ Blaisdell Publishing Co, Waltham MA. (Re-published in 2006 by Dover. ISBN: 0486414655.)
  * [30] Kaula, W. M. 1968. An Introduction to Planetary Physics. John Wiley & Sons, NY.
  * [31] Kozai, Y. 1959a. “The motion of a close earth satellite.” The Astronomical Journal, Vol. 64, pp. 367 - 377.
  * [32] Kozai, Y. 1959b. “On the effects of the Sun and the Moon upon the motion of a close Earth satellite.” SAO Special Report, Vol. 22, pp. 7 - 10.
  * [33] Kozai, Y. 1962. “Secular perturbations of asteroids with high inclination and eccentricity.” The Astronomical Journal, Vol. 67, pp. 591 - 598.
  * [34] Krasinsky, G. A. 2002. “Dynamical History of the Earth-Moon System.” _Celestial Mechanics & Dynamical Astronomy_, Vol. 84, pp. 27 - 55
  * [35] Krasinsky, G. A. 2006. “Numerical theory of rotation of the deformable Earth with the two-layer fluid core. Part 1: Mathematical model.” _Celestial Mechanics & Dynamical Astronomy_, Vol. 96, pp. 169 - 217
  * [36] Lambeck, K. 1980. The Earth’s Variable Rotation: Geophysical Causes and Consequences. Cambridge University Press, Cambridge UK
  * [37] MacDonald, G. J. F. 1964. “Tidal Friction.” Reviews of Geophysics. Vol. 2, pp. 467 - 541
  * [38] Mignard, F. 1979. “The Evolution of the Lunar Orbit Revisited. I.” The Moon and the Planets. Vol. 20, pp. 301 - 315.
  * [39] Mignard, F. 1980. “The Evolution of the Lunar Orbit Revisited. II.” The Moon and the Planets. Vol. 23, pp. 185 - 201
  * [40] Neron de Surgy, O., and Laskar, J. 1997. “On the long term evolution of the spin of the Earth.” _Astronomy & Astrophysics_, Vol. 318, pp. 975 - 989
  * [41] Peale, S. 2005. “The free precession and libration of Mercury.” Icarus, Vol 178, pp 4 - 18
  * [42] Roche, E. A. 1849. “Mémorie sur la figure d’une masse fluide, soumise a l’attraction d’un point éloingné.” _Académie des Sciences et Lettres de Montpellier. Mémories de la Section des Sciences._ , Tome 1, No 3, pp. 243 - 262
http://gallica.bnf.fr/ark:/12148/bpt6k209711r

  * [43] Singer, S. F. 1968. “The Origin of the Moon and Geophysical Consequences.” The Geophysical Journal of the Royal Astronomical Society, Vol. 15, pp. 205 - 226
  * [44] Taff, L. G. 1985. _Celestial Mechanics. A Computational Guide for the Practitioner._ John Wiley & Sons, NY 1985, pp. 332 - 340
  * [45] Thomson, W. 1863. “On the rigidity of the Earth.” _Philosophical Transactions of the Royal Society of London._ Vol. 153, pp. 573 - 582
http://www.jstor.org/view/02610523/ap000064/00a00270

  * [46] Tisserand, F.-F. 1896. _Traité de Mécanique Céleste_. Tome I. _Perturbations des planètes d’après la méthode de la variation des constantes arbitraires._ Gauthier Villars, Paris 1896. Chapitre X.
  * [47] Touma, J., & Wisdom, J. 1994. “Evolution of the Earth-Moon system.” _The Astronomical Journal._ Vol. 108, pp. 1943 - 1961.
  * [48] Williams, J. G., & Benson, G. S. 1971. “Resonances in the Neptune-Pluto System.” _The Astronomical Journal,_ Vol. 71, pp. 167 - 176
  * [49] Williams, J. G., Boggs, D. H., Yoder, C. F., Ratcliff, J. T. & Dickey, J. O. 2001. “Lunar rotational dissipation in solid body and molten core.” _The Journal of Geophysical Research - Planets_ , Vol. 106, No E11, pp. 27933 - 27968.