(eccv) Package eccv Warning: Package ‘hyperref’ is loaded with option
‘pagebackref’, which is *not* recommended for camera-ready version

11institutetext: Technical University of Munich, Germany

# Gaussian Splatting in Style

Abhishek Saroha 11 Mariia Gladkova 11 Cecilia Curreli 11 Tarun Yenamandra 11
Daniel Cremers 11

###### Abstract

Scene stylization extends the work of neural style transfer to three spatial
dimensions. A vital challenge in this problem is to maintain the uniformity of
the stylized appearance across a multi-view setting. A vast majority of the
previous works achieve this by optimizing the scene with a specific style
image. In contrast, we propose a novel architecture trained on a collection of
style images, that at test time produces high quality stylized novel views.
Our work builds up on the framework of 3D Gaussian splatting. For a given
scene, we take the pretrained Gaussians and process them using a multi
resolution hash grid and a tiny MLP to obtain the conditional stylised views.
The explicit nature of 3D Gaussians give us inherent advantages over NeRF-
based methods including geometric consistency, along with having a fast
training and rendering regime. This enables our method to be useful for vast
practical use cases such as in augmented or virtual reality applications.
Through our experiments, we show our methods achieve state-of-the-art
performance with superior visual quality on various indoor and outdoor real-
world data.

###### Keywords:

Scene Stylization Gaussian Splatting

Figure 1: Given multi-view images of a real-world scene, we perform the task
of scene stylization. We train a network that, provided with style image at
test time, generates stylized novel views of the scene conditioned on the
input style in real-time that are consistent in 3D space. In contrary to
popular scene stylization approaches of fitting a scene with each new style,
we learn this mapping via a neural network and therefore, can generate novel
views of the scene even for unseen styles at a rate of roughly 150FPS.

Figure 2: The motivation from our work stems from the requirement of a
specialized method that while stylizing a scene considers the spatial
information into account. We show that to generate stylized novel views of a
scene, it is not sufficient to stylize the rendered views or train a scene
representation model on stylized 2D images. It leads to loss of information
such as deformity in solid truck’s body shown above.

##  1 Introduction

Driven by the curiosity of how a particular image would appear if painted by
Van Gogh, Picasso or any artist, style transfer has been at the heart of the
image processing community for a long time. It can be seen as a challenge of
texture transfer. Earlier works of [13, 12, 81, 37] devised powerful
algorithms for texture synthesis. Further advancements in deep learning gave
rise to works such as [16, 31, 27] that approach the problem by suitably
trained neural networks. The core idea of neural style transfer is to apply
the features of a style image such as colors, texture, and aesthetics to the
content image while keeping information inside the content image intact. The
ability of neural networks to grasp the features of an image at multiple
hierarchical levels made them a preferred choice over non-learning based
algorithms[16]. Fuelled by the availability of 3D data, this phenomenon of
style transfer was extended to the stylization of scenes[28, 26, 43, 87], and
is referred to as scene stylization.

In this work, we propose a solution for the task of scene stylization given a
style image. The ability to control the visual properties of a scene is
central to many practical applications such as avatar modeling or scene
editing [60, 51, 64, 91] to name a few. With the wide adaptability of various
augmented and virtual reality appliances gaining momentum, it is not only
necessary that these techniques are correct, but also need to execute in real-
time for maximum consumer satisfaction.

Over the past few years, various different methods have been proposed to
efficiently represent a complex 3D scene. [59, 82, 77, 71] were some of the
methods leveraging an explicit pointcloud representation. The domain witnessed
an incredible pace of growth following the pioneering work of Neural Radiance
Fields(NeRFs)[50, 49]. NeRFs gained large popularity, primarily due to its
ability to store the information of a complex scene as the weights of a multi-
layer perceptron(MLP), while maintaining great detailing in the resulting
novel view synthesis. One drawback of radiance fields was the slow speed of
training and at test time. This was followed up by many variants of NeRFs such
as [66, 86, 1, 54, 57, 3] that improved the seminal work significantly.
Recently, the work of 3D Gaussian Splatting(3DGS)[33] further pushed the
boundaries of novel view synthesis, especially in terms of consistency and
speed. 3DGS is, by design, more explicit in terms of storing scene information
in the form of 3D Gaussians, with each Gaussian having certain properties
attached to it. Additionally, 3DGS renders views in real-time, thus making it
highly suitable for practical applications.

In this paper, we tackle the problem of scene stylization as that of stylizing
a set of pre-trained Gaussians given a set of style images. Therefore, we name
our approach as Gaussian Splatting in Style(GSS). We then use these Gaussians
conditioned on a style image to obtain stylized views of the complex 3D scene.
Utilizing these Gaussians as the backbone of our approach ensures spatial
consistency. Furthermore, our approach does not add any overhead to the
existing real-time rendering speed of 3DGS, making it a preferred choice for
applications in AR/VR ecosystems. We support our method by obtaining the most
consistent short-term and long-term views as a quantitative metric. To further
strengthen our approach, we also compare the superior quality of stylized
novel views generated by our method in a qualitative fashion against other
methods. To summarise, our contribution in this paper are as follows:

  * •

We develop a novel state-of-the-art method, GSS, to perform neural scene
stylization in real-time based on 3D Gaussian splatting. To the best of our
knowledge, we are the first method to perform scene stylization using 3D
Gaussians.

  * •

We demonstrate the effectiveness of our method by comparing against various
types of baselines both quantitatively and qualitatively across various real-
world datasets across different settings.

##  2 Related Work

###  2.1 3D Scene representation

Complex 3D scenes can be represented in many ways. One of the most common and
well known ways is that of using point clouds[26, 2]. Similar to point clouds,
some works such as [69, 85] use a voxel grid for representing a 3D scene.
Voxel grids are not the most ideal choice for scene representation due to
their high memory footprint. Neural implicit scene based methods, led by the
works of [62, 46, 63] making use of signed distance fields(SDFs) and
occupancies respectively, helped solve this memory issue. In principle,
implicit neural representations can generate meshes and surfaces upto an
arbitrarily high resolution. Recently, work of NeuralAngelo [42] set a new
benchmark as it could recover a highly detailed 3D surface of a large-scale
scene from a set of input images. Following a similar direction neural
radiance fields, also known as NeRFs[50] are also an implicit method that is
described in further detail below.

###  2.2 Novel View Synthesis

The task of generating unseen views of an object/scene given a collection of
input images is known as novel view synthesis. Classical works, such as [39,
5, 10, 92, 22, 70, 17] aimed at generating novel views directly from the given
set of multi-view images. With the onset of deep learning, new approaches such
as [21, 74, 73, 67] paved the way for neural network-based novel view
synthesis(NVS). All these methods represented the scene in their own ways. A
major milestone in the domain of NVS was with the advent of Neural Radiance
Fields(NeRFs) [50]. NeRFs represent the scene as the weights of a multi-layer
perceptron(MLP). The input to these MLPs are points in space and the viewing
direction, which then gives out the density and color for each queried point.
Despite its success, NeRF suffered from many drawbacks. A slow training
regime, followed by the need to optimize it for each scene made it less viable
for many use cases. It was also observed that NeRFs performed extremely well
when the input views were dense and covered many angles of the scene. It often
failed to get the finer details from views far-away from those used in
training. Follow up works such as [57, 9, 18, 86] focused specifically on
solving NVS for a sparse set of input views. Meanwhile, [86, 83, 55] focused
on a more generalized framework to avoid fitting a MLP to each new scene,
while [1, 85, 66, 54] improved on the original work in terms of training and
inference speed. Recently, 3D Gaussian Splatting(3DGS) [33] also tackled the
task of novel view synthesis in real-time. We explain the idea of 3DGS in
Section 3 and can recommend the readers to visit [33] for more details.

###  2.3 Style Transfer

Neural style transfer is the task of transferring the style of a source image
onto the target(or content) image[16]. It is a complex task as the transfer
needs to preserve the information present in the target image. This problem
can also be formulated as that of texture transfer, and was tackled in the
works of [13, 12, 81, 37] before the widespread adoption of neural networks.
Later, the pioneering work of [16] successfully applied the use of neural
networks to the task of style transfer. The work made use of a pretrained
VGG[72] to extract the semantic information from an image at different
hierarchical levels. [16] optimized a loss function composed of two terms,
content loss and style loss, which represented the similarity of the generated
image with the content image and the style image respectively. The style loss
was optimized by using the Gram matrix. The primary objective of Gram matrix
was to encapsulate the correlations between the different features. Since [16]
worked by running an optimization for each target and style image, it was not
suitable for real-time applications. This was further improved by [40, 31, 78,
27] by being real-time and providing the flexibility of general style
conditioning. Additionally, the works of [44, 30] added the sophistication of
monocular depth estimation for preserving the depth information present in the
source image. Furthermore, works of [25, 19, 15, 4, 6, 80] extended the work
of 2D style transfer to a video sequence by making use of advancements in
methods like optical flow[11]. However, [26] and our experiments show that
simply extending 2D style transfer to 3D scenes result in visual artifacts
such as blurriness and inconsistency across different views, as also depicted
in Figure 2.

###  2.4 Scene Stylization

Stylizing scenes given a style image has gained prominence in recent years,
especially with the arrival of AR/VR headsets in the mainstream. One way to
distinguish between different methods is based on their way of scene
representation. Works of [26, 52, 2] work with a point cloud based
representation. [26] for instance, project the style features from 2D into 3D
and transform them into that of the style image before re-projecting them back
into 2D for getting stylized views. Similarly, [23] performs scene stylization
of indoor scenes in the form of a mesh. Recently, radiance fields [50] gained
huge popularity due to their ability to generate novel views with a very high
quality. Due to this, a lot of works made use of a NeRF backbone to perform
scene stylization. One direction of approaching the problem is by focusing to
optimize a radiance for each given style image. Methods such as [32, 87, 41,
8, 56, 84, 89, 61, 90] follow this outline. Methods such as [7, 8] make use of
a hypernetwork to impart the style image features onto the rendered views of a
complex 3D scene. On the other hand, works such as [28, 43] provide a
generalizable scene stylization framework, i.e a stylized scene can be
generated at inference time given a conditioned style image input. Our work
also follows in this category. Another direction in this domain lies making
use of additional information such as depth maps [32] since their primary
focus lies in geometry preservation. In this work, instead of a radiance
field, we make use of a Gaussian splatting[33] framework. To the best of our
knowledge, we are the first method that builds up on a 3DGS backbone for
image-based scene stylization.

Apart from image based conditioning for stylization, one direction is to input
the conditioning in the form of text. Leveraging the use of large language
models(LLMs), [48, 47, 29, 79, 20, 75, 14] perform scene editing, and
stylization in some cases, via text-based inputs. In our work, we do not
compare with any text-based input methods and focus entirely on methods that
condition the scene via style images only. While textual information can be
informative and descriptive, the challenges of image-based conditioning can be
far more demanding as it provides information in terms of abstraction,
strokes, and so on.

##  3 Preliminaries

###  3.1 3D Gaussian Splatting

3D Gaussian Splatting(3DGS)[33] is an explicit method for scene
representation, that is especially useful for high quality real-time
rendering. It can be considered as a form based on point cloud representation.
Each scene in 3DGS is represented as a bunch of 3D Gaussians, each of which
has certain properties. Each 3D Gaussian is defined by its mean position
$\mu\in\mathbb{R}^{3}$ , covariance matrix $\Sigma$ as the following:

$G(X)=e^{-\frac{1}{2}\mu^{T}\Sigma^{-1}\mu}.$ (1)

Since the covariance matrix has to be positive semi-definite for
differentiable optimization, it is decomposed into a rotation matrix
$\mathbf{R}$, expressed in quaternions and a scaling matrix
$\mathbf{S}\in\mathbb{R}^{3}$ as:

$\Sigma=\mathbf{R}\mathbf{S}\mathbf{S}^{T}\mathbf{R}^{T}.$ (2)

Additionally, each Gaussian also contains the opacity $\alpha$ and view-
dependent color values, represented by spherical harmonics coefficients. Once
initialised, these Gaussians are projected onto 2D for rendering. This
rendering process is done by a differentiable tile rasterizer. During the
process of optimization, these Gaussians undergo density control, i.e they are
constantly removed and new ones are added to the scene in order to maintain
high image quality. This is done in order to ensure that the density of
Gaussians is more or less consistent in all areas of the scene, especially
ones which were empty during the SfM initialization. The 3D scene is optimized
for the following loss function:

$\mathcal{L}=(1-\lambda)\mathcal{L}_{1}+\lambda\mathcal{L}_{D-SSIM}$ (3)

where $\mathcal{L}_{1}$ and $\mathcal{L}_{D-SSIM}$ are computed between the
generated image and the ground truth view. For thorough explanation, we refer
the reader to [33].

###  3.2 Multi-resolution Hash Grid

Encoding neural network inputs into higher dimensional space has been crucial
in achieving scene representations of high quality with compact models [50].
[54] furthers fidelity and efficiency by outsourcing the representation power
to an auxiliary data structure, namely a multi-resolution hash grid, which
allows fast access to learned features at each grid cell on multiple spatial
resolutions. Specifically, an input 3D point is first parameterized by a hash
code $(\boldsymbol{x}_{1},\ldots,\boldsymbol{x}_{l})$, which corresponds to
the corner indices of a set of grids with different spatial resolution
$\\{V_{1},\ldots,V_{L}\\}$. Given those indices, feature vectors are obtained
and linearly interpolated according to the relative position of
$\boldsymbol{x}$ across all $L$ spatial resolutions. Formally, the final
multi-resolution hash encoding of a 3D point $\boldsymbol{x}$ becomes
$\gamma(\boldsymbol{x})=(\gamma_{1}(\boldsymbol{x}_{1}),\ldots,\gamma_{l}(\boldsymbol{x}_{l}))$,
where $\gamma_{l}(\boldsymbol{x}_{l})$ is a feature vector obtained by means
of interpolation of hash entries at the grid corners of the $l$th resolution
level.

Multi-resolution hash encoding is independent at each level and highly
adaptive to the desired finest resolution. Thus, it enables faster convergence
during training, does not impede inference speed, and promotes a higher level
of detail, which is beneficial for the stylization task using 3DGS
representation.

###  3.3 Stylized NeRF

StylizedNeRF [28] set up a new benchmark in the field of neural scene
stylization. Built upon the framework of NeRF and Nerf-W [50, 45], this work
makes use of a mutual learning strategy to combine 2D stylization method AdaIN
[27] and a pre-trained NeRF architecture. The high-level idea is to predict a
new stylized color conditioned on the style image. This is achieved by
replacing the color component with a new style module while keeping the
density prediction from the pre-trained NeRF constant to maintain geometric
consistency. To facilitate mutual learning between 2D-3D stylization,
StylizedNeRF makes the decoder of AdaIN trainable for fine-tuning the final
results. Inspired by Nerf-W, they also employ a pre-trained VAE that provides
learnable latent codes for the conditioned style inputs. Motivated by the
positive results of StylizedNeRF, GSS also makes use of pre-trained Gaussians
to better represent the geometric details of the complex scenes.

(a)

(b)

Figure 3: Here we diagramatically show the overall architecture of our
pipeline. (a) contains the overview of our method. (b) shows in detail the
top-right part of (a) where we have a deeper look into how the 3D color module
combines with 3DGS to produce stylized novel views.

##  4 Gaussian Splatting in Style

Our goal is to stylize a complex 3D scene. The first step is to learn the
scene information from a given collection of images in a multi-view setting.
We achieve this by using a 3DGS backbone. For this objective, we take
pretrained models of learnt scenes for the next step. The second stage
includes learning the scene information conditioned on various style images.
During inference time, we generate novel views conditioned on an input style
image. In the following section, we elaborate on the various constituents of
our pipeline.

2D Stylization Module The first component of our pipeline is a 2D stylization
module. It is based on AdaIN [27]. It consists of three components, a
pretrained VGG [72] as the encoder, a convolutional decoder, and adaptive
instance normalisation layer. For details, please refer to [28]. As shown in
Figure 3, we pass the style image and the rendered view from 3DGS through a
pretrained VGG to obtain their features. These features are then passed
through the adaptive instance normalization layer and the decoder to obtain
the 2D styled image of that particular view. This 2D stylized image acts as a
guide for our 3D color module, and gives a rough indication as to how the
scene should appear from that specific viewpoint. Therefore, this image,
represented by $\mathcal{I}_{style2d}$ forms on what we call it as a Guide
Loss $\mathcal{L}_{g}$, which is detailed in Section 4.

3D Color Module This module is the second component in our overall
architecture. The input to this module are the mean positions $\mu$ of the 3D
Gaussians and a latent code representing the style image obtained through the
use of a pretrained encoder[65, 28]. The input positions are fed into a multi-
resolution hash grid(MHG) [54] while the latent codes are fed into an FC layer
before concatenating them together. This resulting tensor is then passed
through a tiny MLP [53, 38] to obtain the new RGB color for each input
Gaussian. The use of MHG and tiny MLP was motivated by the fact that each
scene has a large number of Gaussians, and could also be in the order of
millions for large scenes such as those in the TnT[35] dataset. The predicted
RGB from the 3D color module is fed into the 3DGS pipeline, and the Gaussians,
with their new colors are rendered to generate a 3D consistent stylised view
$\mathcal{I}_{gen}$. To preserve important details in the scene, we compare
$\mathcal{I}_{gen}$ with the unstyled view $\mathcal{I}_{i}$ as shown in
Equation 5. Since our color module is view dependent, it does not affect the
runtime during inference time. Since the geometry is conserved, one forward
pass through GSS with the conditioned style input allows us to obtain the
stylized RGB for any arbitrary viewpoint.

Loss function As shown in Figure 3, we apply the loss functions on the views
generated by the 3D color module. Our loss term is composed as a weighted sum
of two terms, the  guide loss $\mathcal{L}_{g}$ and the content loss
$\mathcal{L}_{c}$, elaborated as follows:

$\mathcal{L}_{g}=||\mathcal{I}_{style2d}-\mathcal{I}_{gen}||$ (4)

$\mathcal{L}_{c}=||\mathcal{I}_{gen}-\mathcal{I}_{i}||$ (5)

where $\mathcal{I}_{style2d},\mathcal{I}_{i},\mathcal{I}_{gen}$ are the 2D
stylized image from the 2D stylization module, the unstyled view generated
from 3D Gaussian splatting, and the stylized view from our pipeline
respectively.

Finally, our total loss $\mathcal{L}_{total}$ can be written as:

$\mathcal{L}_{c}=\lambda_{g}\mathcal{L}_{g}+\lambda_{c}\mathcal{L}_{c}$ (6)

where $\lambda_{g}$ and $\lambda_{c}$ are the weights of the guide loss and
content loss respectively.

##  5 Experiments

| Dataset
---
↓
| Method
---
→
| GS →
---
AdaIN
| AdaIN
---
→ GS
LSNV | ARF |  | Stylised-
---
NeRF
StyleRF | GSS(Ours)
| RMSE ↓ | 0.098 | 0.083 | 0.030 | 0.036 | 0.002 | 0.038 | 0.034
Flower | LPIPS ↓ | 0.106 | 0.080 | 0.011 | 0.015 | 0.001 | 0.013 | 0.009
| RMSE ↓ | 0.127 | 0.116 | 0.075 | 0.038 | 0.041 | 0.037 | 0.026
Fern | LPIPS ↓ | 0.151 | 0.134 | 0.049 | 0.020 | 0.012 | 0.019 | 0.008
| RMSE ↓ | 0.104 | 0.103 | 0.044 | 0.037 | 0.029 | 0.035 | 0.021
Room | LPIPS ↓ | 0.109 | 0.105 | 0.014 | 0.014 | 0.011 | 0.014 | 0.006
| RMSE ↓ | 0.136 | 0.099 | 0.089 | 0.043 | 0.015 | 0.045 | 0.010
Trex | LPIPS ↓ | 0.136 | 0.087 | 0.033 | 0.022 | 0.004 | 0.017 | 0.003
Table 1: Short Term Consistency Here, we demonstrate the superior performance
of our method using the short-term consistency metric. We compute the metric
using both, RMSE and LPIPS[88] using a VGG backbone. As shown in Figure 4, the
novel views rendered by StylizedNerf are highly blurry and misses crucial
details. Therefore, due to the over-smooth and blurry output, the metric
obtained for them is really low and does not correctly reflect the adequate
stylization qualities.

Implementation Details We build our work on top of the 3DGS[33] framework and
use it to pretrain the Gaussians for each scene. For this, we followed the
original training regime as suggested by the authors. For the 2D stylization
module, we use a pretrained VGG [72, 16] and AdaIN[27] decoder. For obtaining
the latent code of input style images, we use a pretrained image encoder used
in [65]. As for the hyperparameters, we set $\lambda_{g}$ and $\lambda_{c}$ as
1 and 0.1 respectively. We train the tiny MLP for the second stage as per the
training regime described in Figure 3 for 50k iterations. For optimization, we
use the Adam [34] optimizer with a learning rate of $1e^{-4}$. We run all our
experiments on a Nvidia A4500 GPU.

Baselines Since no previous approach performs scene stylization based on
Gaussian splatting, we compare our method with NeRF-based approaches. We
primarily compare against StylizedNerf[28, 24], and LSNV[26] as their problem
statement resembles most closely to that of ours. We also compare against a
newer method StyleRF[43], along with artistic radiance fields(ARF)[87],
although ARF optimizes every scene for each given style image. For
completeness, we also create two other baselines, ADA-GS and GS-ADA. In ADA-
GS, we train 3DGS on 2D stylized images for each style. This setting resembles
that of ARF. In GS-ADA, we apply AdaIN[27] on the novel view renders obtained
from pretrained 3DGS.

Datasets Similar to the baselines [28, 43, 26] we perform stylization on
various real-world datasets. For real-world, we use the LLFF dataset[49]. It
contains high-resolution front-facing scenes, out of which we select the most
common ones used in the baselines, namely $trex,fern,flowers,room$. We also
demonstrate the effectiveness of our method on more challenging Tanks and
Temples(TnT)[36] dataset. For training, we use a dataset of 120 diverse style
images sampled from WikiArt dataset[68], which include the paintings and
popular artworks of verious artists.

| Dataset
---
↓
| Method
---
→
| GS →
---
AdaIN
| AdaIN
---
→ GS
LSNV | ARF |  | Stylised-
---
NeRF
StyleRF | GSS(Ours)
| RMSE ↓ | 0.147 | 0.141 | 0.040 | 0.123 | 0.002 | 0.123 | 0.090
Flower | LPIPS ↓ | 0.151 | 0.135 | 0.018 | 0.112 | 0.001 | 0.081 | 0.057
| RMSE ↓ | 0.151 | 0.132 | 0.076 | 0.109 | 0.044 | 0.115 | 0.042
Fern | LPIPS ↓ | 0.193 | 0.162 | 0.051 | 0.107 | 0.019 | 0.099 | 0.021
| RMSE ↓ | 0.143 | 0.150 | 0.045 | 0.084 | 0.045 | 0.078 | 0.033
Room | LPIPS ↓ | 0.182 | 0.184 | 0.019 | 0.061 | 0.028 | 0.046 | 0.015
| RMSE ↓ | 0.164 | 0.131 | 0.091 | 0.092 | 0.030 | 0.091 | 0.020
Trex | LPIPS ↓ | 0.194 | 0.137 | 0.050 | 0.081 | 0.016 | 0.057 | 0.007
Table 2: Long Term Consistency In this table, we measure the performance of
GSS against the said baselines via long-term consistency. Similar to [43], for
long-term consistency, the two frames should be far away from each other.
Having a long-term consistent method ensures that the produced novel views of
the scene are consistent in the stylized color space. Similar to short-term
consistency in Table 1, StylizedNerf numbers does not accurately depict the
stylization quality as compared to other methods.

###  5.1 Quantitative Results

Since the task of scene stylization is relatively new, there are not many ways
to quantitatively compare methods. However, following [26, 28, 43], we measure
the quantitative result by measure short-term and long-term consistency.
Similar to the evaluation in [43], we first take two stylized views. By using
[76] and [58], we then warp one view into another before computing the masked
LPIPS[88] and RMSE score. We use adjacent views for short-term and far-away
views for long-term consistency. Many works such as [26] also refer to it as
warped LPIPS and warped RMSE. Mathematically, they can be summarised as
follows:

$\mathbb{E}_{wlpips}(\mathcal{O}_{v},\mathcal{O}_{v^{{}^{\prime}}})=LPIPS(\mathcal{O}_{v},\mathcal{M}_{v}(\mathcal{W}(\mathcal{O}_{v^{{}^{\prime}}})))$
(7)

$\mathbb{E}_{wrmse}(\mathcal{O}_{v},\mathcal{O}_{v^{{}^{\prime}}})=RMSE(\mathcal{O}_{v},\mathcal{M}_{v}(\mathcal{W}(\mathcal{O}_{v^{{}^{\prime}}})))$
(8)

where $\mathcal{M}_{v}$, $\mathcal{O}_{v}$, and $\mathcal{W}$ are the masking,
view ${v}$, and the warping function respectively.

We quantitatively compare our method with the baselines as shown in Tables 1
and 2. It can be clearly seen that our method outperforms all other baselines
in both short-term and long-term consistency. We can see that simply applying
2D stylization to rendered novel views is not enough to ensure consistency, in
accordance with [26], and hence Ada-GS is almost always more consistent than
GS-ADA for both short-term and long-term views. Furthermore, simply adding 2D
stylization to the input or outputs of existing scene representation
approaches is not sufficient, as all other dedicated baselines perform better
on average than GS-ADA and ADA-GS. We also find that methods utilising
explicit scene representations, such as GSS and LSNV[26] perform better than
radiance field based baselines. It is also important to observe that for
StylizedNerf, the generated novel views suffer from high blur, over-smoothing
and loss of details. Due to this, they obtain a very high consistency despite
not being of reasonable quality as compared to all other methods.

| LSNV | ARF | StylizedNerf | StyleRF | GSS
---|---|---|---|---|---
Rendered FPS | 40 | 1 | 0.004 | 0.04 | 157
Table 3: We perform a runtime analysis of our method vs other baselines. Due
to our 3DGS backbone, we are able to generate stylized novel views
approximately 4x faster than the second method, achieving an average of 157
FPS(Frames per second). Radiance fields in general have a high rendering time
and suffer from slow training and testing time.

On running on the same hardware, GSS is also the fastest. Whereas all other
methods take in the order of a few seconds to minutes to generate a single
view, we obtain a rendering speed of roughly averaging 157 FPS. Our method is
also very efficient in training and it requires roughly 2 hours to train for
each scene on a standard Nvidia A4500 GPU.

Figure 4: We provide a detailed qualitative comparison of our method against
the baselines detailed in Section 5. It can be seen that GSS achieves a highly
accurate stylization based on the input style. While methods such as ARF
obtain a better texture, it is attributed to the fact that it is optimized
separately for each style. GS-Ada and Ada-GS are also able to get high quality
texture, however, it is not accurate at geometry conservation since these
methods are based on pure 2D stylization. Also, methods such as StyleRF fail
to grasp the accurate style color such as that in room, fern or flower scene.

###  5.2 Qualitative Results

Along with the quantitative results, we demonstrate the performance of our
method with the baselines with a qualitative evaluation as shown in Figure 4.
It can be seen that not only our method preserves content details better, it
also is more faithful to the style images. For instance, while StyleRF[43] is
great at preserving geometric details, it does not adequately transfer the
style image features onto the novel views. On the other hand, ARF[87] is
aggressive at transferring the style image features onto the scene, and hence
often leads to a loss in the information. It also has an affinity to focus
more on the central object and does not transfer the style to far-away objects
in the scene. It is worth noting that ARF optimizes each scene for every style
image. We also observe that StylisedNerf[28] produces blurry views while LSNV
also struggles to retain fine details in the stylized views.

##  6 Ablation Studies

To study the effects of various components of our method, we perform several
experiments and report their findings below.

Figure 5: Here we show the effect of varying $\lambda_{c}$ in our loss
function as described in Equation 5. We observe that setting $\lambda_{c}=0$
results in loss of details such as the anthers and filaments of the flower as
shown above. However, further increasing $\lambda_{c}$ results in loss of
style, and the generated view looks similar to that of the unstylized view.

###  6.1 Content Loss

We first ablate the content loss $\mathcal{L}_{c}$, described in Equation 5,
by varying the value of $\lambda_{c}$. We vary the value in the discrete set
of $\\{0,0.1,1,10,100\\}$ to cover a wide range of possible values.

Setting the value of $\lambda_{c}=0$ leads to over-smoothing of the generated
view. While the generated view is still consistent and stylized, it loses some
of the finer details. On increasing the value beyond 1, the rendered view
resembles that of being unstylized.

Figure 6: We ablate the effect of multi-resolution hash grid(MHG) in our
setup. We find that having MHG in our model enforces a much better texture and
color transfer from the style image onto the scene. It can be attributed to
its high capability of learning features, while keeping the model fast and
small.

###  6.2 Multi-resolution Hash Grid

We also study the effect of having a multi-resolution hash grid(MHG). On
removing the MHG from our 3D color module, we find that the resulting stylized
scene is not adequately transferred onto the scene as shown in Figure 6.
Instead, the scene loosely adopts the color-palette from the style image
without grasping any textual information, thus verifying our decision to use
MHG.

Figure 7: We interpolate between the latent vectors of the style images. The
four style images are shown at the four corners of the image above and were
chosen so they’re highly diverse and accurately depict the differences in the
predicted RGBs for each Gaussian from our model.

###  6.3 Style Interpolation

We perform style interpolation by taking four different style images at test
time, and obtaining the resulting style latent by linearly interpolating
between the different styles. We show the results in Figure 7. The ablation
shows the capability of our model to generate results for not only the given
style images, but also a mixture of such styles. It demonstrates that our
model learns the mapping from style inputs to predicted RGB values for each 3D
Gaussian in a meaningful way, and thus provides increased generalisability.

##  7 Conclusion

In this paper, we presented a novel method to stylize complex 3D scenes that
are spatially consistent. Contrary to a majority of existing works, once
trained, our method is capable to take unseen input scenes at inference time
and produce novel views in real-time. By leveraging a multi-resolution hash
grid and a tiny MLP, we are able to accurately generate the stylized colors of
each 3D Gaussian present in a scene. Since we only do one forward pass through
the 3D color module, we are able to generate novel views at around 150FPS. We
exhibit that GSS produces superior results by the use of quantitative and
qualitative results, thus making GSS suitable for many practical applications.

## References

  * [1] Barron, J.T., Mildenhall, B., Tancik, M., Hedman, P., Martin-Brualla, R., Srinivasan, P.P.: Mip-nerf: A multiscale representation for anti-aliasing neural radiance fields (2021)
  * [2] Cao, X., Wang, W., Nagao, K., Nakamura, R.: Psnet: A style transfer network for point cloud stylization on geometry and color. In: Proceedings of the IEEE/CVF Winter Conference on Applications of Computer vision. pp. 3337–3345 (2020)
  * [3] Chen, A., Xu, Z., Geiger, A., Yu, J., Su, H.: Tensorf: Tensorial radiance fields. In: European Conference on Computer Vision (ECCV) (2022)
  * [4] Chen, D., Liao, J., Yuan, L., Yu, N., Hua, G.: Coherent online video style transfer. In: Proceedings of the IEEE International Conference on Computer Vision. pp. 1105–1114 (2017)
  * [5] Chen, S.E., Williams, L.: View Interpolation for Image Synthesis. In: SIGGRAPH (1993)
  * [6] Chen, X., Zhang, Y., Wang, Y., Shu, H., Xu, C., Xu, C.: Optical flow distillation: Towards efficient and stable video style transfer. In: Computer Vision–ECCV 2020: 16th European Conference, Glasgow, UK, August 23–28, 2020, Proceedings, Part VI 16. pp. 614–630. Springer (2020)
  * [7] Chen, Y., Yuan, Q., Li, Z., Liu, Y., Wang, W., Xie, C., Wen, X., Yu, Q.: Upst-nerf: Universal photorealistic style transfer of neural radiance fields for 3d scene (2022)
  * [8] Chiang, P.Z., Tsai, M.S., Tseng, H.Y., sheng Lai, W., Chiu, W.C.: Stylizing 3d scene via implicit representation and hypernetwork (2022)
  * [9] Chibane, J., Bansal, A., Lazova, V., Pons-Moll, G.: Stereo radiance fields (srf): Learning view synthesis for sparse views of novel scenes. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition. pp. 7911–7920 (2021)
  * [10] Debevec, P.E., Taylor, C.J., Malik, J.: Modeling and rendering architecture from photographs: A hybrid geometry-and image-based approach. In: SIGGRAPH (1996)
  * [11] Dosovitskiy, A., Fischer, P., Ilg, E., Hausser, P., Hazirbas, C., Golkov, V., Van Der Smagt, P., Cremers, D., Brox, T.: Flownet: Learning optical flow with convolutional networks. In: Proceedings of the IEEE international conference on computer vision. pp. 2758–2766 (2015)
  * [12] Efros, A.A., Freeman, W.T.: Image quilting for texture synthesis and transfer. In: Seminal Graphics Papers: Pushing the Boundaries, Volume 2, pp. 571–576 (2023)
  * [13] Efros, A.A., Leung, T.K.: Texture synthesis by non-parametric sampling. In: Proceedings of the seventh IEEE international conference on computer vision. vol. 2, pp. 1033–1038. IEEE (1999)
  * [14] Fang, J., Wang, J., Zhang, X., Xie, L., Tian, Q.: Gaussianeditor: Editing 3d gaussians delicately with text instructions (2023)
  * [15] Gao, C., Gu, D., Zhang, F., Yu, Y.: Reconet: Real-time coherent video style transfer network. In: Computer Vision–ACCV 2018: 14th Asian Conference on Computer Vision, Perth, Australia, December 2–6, 2018, Revised Selected Papers, Part VI 14. pp. 637–653. Springer (2019)
  * [16] Gatys, L.A., Ecker, A.S., Bethge, M.: Image style transfer using convolutional neural networks. In: Proceedings of the IEEE conference on computer vision and pattern recognition. pp. 2414–2423 (2016)
  * [17] Gortler, S.J., Grzeszczuk, R., Szeliski, R., Cohen, M.F.: The Lumigraph. SIGGRAPH 96(30) (1996)
  * [18] Guangcong, Chen, Z., Loy, C.C., Liu, Z.: Sparsenerf: Distilling depth ranking for few-shot novel view synthesis. IEEE/CVF International Conference on Computer Vision (ICCV) (2023)
  * [19] Gupta, A., Johnson, J., Alahi, A., Fei-Fei, L.: Characterizing and improving stability in neural style transfer. In: Proceedings of the IEEE International Conference on Computer Vision. pp. 4067–4076 (2017)
  * [20] Haque, A., Tancik, M., Efros, A.A., Holynski, A., Kanazawa, A.: Instruct-nerf2nerf: Editing 3d scenes with instructions. arXiv preprint arXiv:2303.12789 (2023)
  * [21] Hedman, P., Philip, J., Price, T., Frahm, J.M., Drettakis, G., Brostow, G.: Deep Blending for Free-Viewpoint Image-Based Rendering. In: SIGGRAPH Asia (2018)
  * [22] Heigl, B., Koch, R., Pollefeys, M., Denzler, J., Van Gool, L.: Plenoptic Modeling and Rendering from Image Sequences taken by a Hand Held Camera. In: GCPR (1999)
  * [23] Höllein, L., Johnson, J., Nießner, M.: Stylemesh: Style transfer for indoor 3d scene reconstructions. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition. pp. 6198–6208 (2022)
  * [24] Hu, S.M., Liang, D., Yang, G.Y., Yang, G.W., Zhou, W.Y.: Jittor: a novel deep learning framework with meta-operators and unified graph execution. Science China Information Sciences 63(222103), 1–21 (2020)
  * [25] Huang, H., Wang, H., Luo, W., Ma, L., Jiang, W., Zhu, X., Li, Z., Liu, W.: Real-time neural style transfer for videos. In: Proceedings of the IEEE conference on computer vision and pattern recognition. pp. 783–791 (2017)
  * [26] Huang, H.P., Tseng, H.Y., Saini, S., Singh, M., Yang, M.H.: Learning to stylize novel views. In: Proceedings of the IEEE/CVF International Conference on Computer Vision. pp. 13869–13878 (2021)
  * [27] Huang, X., Belongie, S.: Arbitrary style transfer in real-time with adaptive instance normalization. In: Proceedings of the IEEE international conference on computer vision. pp. 1501–1510 (2017)
  * [28] Huang, Y.H., He, Y., Yuan, Y.J., Lai, Y.K., Gao, L.: Stylizednerf: consistent 3d scene stylization as stylized nerf via 2d-3d mutual learning. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition. pp. 18342–18352 (2022)
  * [29] Hwang, I., Kim, H., Kim, Y.M.: Text2scene: Text-driven indoor scene stylization with part-aware details (2023)
  * [30] Ioannou, E., Maddock, S.: Depth-aware neural style transfer using instance normalization. arXiv preprint arXiv:2203.09242 (2022)
  * [31] Johnson, J., Alahi, A., Fei-Fei, L.: Perceptual losses for real-time style transfer and super-resolution. In: Computer Vision–ECCV 2016: 14th European Conference, Amsterdam, The Netherlands, October 11-14, 2016, Proceedings, Part II 14. pp. 694–711. Springer (2016)
  * [32] Jung, H., Nam, S., Sarafianos, N., Yoo, S., Sorkine-Hornung, A., Ranjan, R.: Geometry transfer for stylizing radiance fields (2024)
  * [33] Kerbl, B., Kopanas, G., Leimkühler, T., Drettakis, G.: 3d gaussian splatting for real-time radiance field rendering. ACM Transactions on Graphics 42(4) (July 2023), https://repo-sam.inria.fr/fungraph/3d-gaussian-splatting/
  * [34] Kingma, D.P., Ba, J.: Adam: A method for stochastic optimization (2017)
  * [35] Knapitsch, A., Park, J., Zhou, Q.Y., Koltun, V.: Tanks and temples: Benchmarking large-scale scene reconstruction. ACM Transactions on Graphics 36(4) (2017)
  * [36] Knapitsch, A., Park, J., Zhou, Q.Y., Koltun, V.: Tanks and temples: Benchmarking large-scale scene reconstruction. ACM Transactions on Graphics (ToG) 36(4), 1–13 (2017)
  * [37] Kwatra, V., Schödl, A., Essa, I., Turk, G., Bobick, A.: Graphcut textures: Image and video synthesis using graph cuts. Acm transactions on graphics (tog) 22(3), 277–286 (2003)
  * [38] Lee, J.C., Rho, D., Sun, X., Ko, J.H., Park, E.: Compact 3d gaussian representation for radiance field. arXiv preprint arXiv:2311.13681 (2023)
  * [39] Levoy, M., Hanrahan, P.: Light field rendering. In: SIGGRAPH (1996)
  * [40] Li, C., Wand, M.: Precomputed real-time texture synthesis with markovian generative adversarial networks. In: Computer Vision–ECCV 2016: 14th European Conference, Amsterdam, The Netherlands, October 11-14, 2016, Proceedings, Part III 14. pp. 702–716. Springer (2016)
  * [41] Li, W., Wu, T., Zhong, F., Oztireli, C.: Arf-plus: Controlling perceptual factors in artistic radiance fields for 3d scene stylization (2023)
  * [42] Li, Z., Müller, T., Evans, A., Taylor, R.H., Unberath, M., Liu, M.Y., Lin, C.H.: Neuralangelo: High-fidelity neural surface reconstruction. In: IEEE Conference on Computer Vision and Pattern Recognition (CVPR) (2023)
  * [43] Liu, K., Zhan, F., Chen, Y., Zhang, J., Yu, Y., Saddik, A.E., Lu, S., Xing, E.: Stylerf: Zero-shot 3d style transfer of neural radiance fields (2023)
  * [44] Liu, X.C., Cheng, M.M., Lai, Y.K., Rosin, P.L.: Depth-aware neural style transfer. In: Proceedings of the symposium on non-photorealistic animation and rendering. pp. 1–10 (2017)
  * [45] Martin-Brualla, R., Radwan, N., Sajjadi, M.S., Barron, J.T., Dosovitskiy, A., Duckworth, D.: Nerf in the wild: Neural radiance fields for unconstrained photo collections. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition. pp. 7210–7219 (2021)
  * [46] Mescheder, L., Oechsle, M., Niemeyer, M., Nowozin, S., Geiger, A.: Occupancy networks: Learning 3d reconstruction in function space. In: Proceedings IEEE Conf. on Computer Vision and Pattern Recognition (CVPR) (2019)
  * [47] Metzer, G., Richardson, E., Patashnik, O., Giryes, R., Cohen-Or, D.: Latent-nerf for shape-guided generation of 3d shapes and textures (2022)
  * [48] Michel, O., Bar-On, R., Liu, R., Benaim, S., Hanocka, R.: Text2mesh: Text-driven neural stylization for meshes (2021)
  * [49] Mildenhall, B., Srinivasan, P.P., Ortiz-Cayon, R., Kalantari, N.K., Ramamoorthi, R., Ng, R., Kar, A.: Local light field fusion: Practical view synthesis with prescriptive sampling guidelines. ACM Transactions on Graphics (TOG) 38(4), 1–14 (2019)
  * [50] Mildenhall, B., Srinivasan, P.P., Tancik, M., Barron, J.T., Ramamoorthi, R., Ng, R.: Nerf: Representing scenes as neural radiance fields for view synthesis. Communications of the ACM 65(1), 99–106 (2021)
  * [51] Moreau, A., Song, J., Dhamo, H., Shaw, R., Zhou, Y., Pérez-Pellitero, E.: Human gaussian splatting: Real-time rendering of animatable avatars (2023)
  * [52] Mu, F., Wang, J., Wu, Y., Li, Y.: 3d photo stylization: Learning to generate stylized novel views from a single image. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition. pp. 16273–16282 (2022)
  * [53] Müller, T.: tiny-cuda-nn (4 2021), https://github.com/NVlabs/tiny-cuda-nn
  * [54] Müller, T., Evans, A., Schied, C., Keller, A.: Instant neural graphics primitives with a multiresolution hash encoding. ACM Transactions on Graphics (ToG) 41(4), 1–15 (2022)
  * [55] Nguyen-Ha, P., Huynh, L., Rahtu, E., Matas, J., Heikkil, J., et al.: Cascaded and generalizable neural radiance fields for fast view synthesis. IEEE Transactions on Pattern Analysis and Machine Intelligence (2023)
  * [56] Nguyen-Phuoc, T., Liu, F., Xiao, L.: Snerf: Stylized neural implicit representations for 3d scenes (2022)
  * [57] Niemeyer, M., Barron, J.T., Mildenhall, B., Sajjadi, M.S.M., Geiger, A., Radwan, N.: Regnerf: Regularizing neural radiance fields for view synthesis from sparse inputs (2021)
  * [58] Niklaus, S., Liu, F.: Softmax splatting for video frame interpolation (2020)
  * [59] Niklaus, S., Mai, L., Yang, J., Liu, F.: 3d ken burns effect from a single image (2019)
  * [60] Pang, H., Zhu, H., Kortylewski, A., Theobalt, C., Habermann, M.: Ash: Animatable gaussian splats for efficient and photoreal human rendering (2023)
  * [61] Pang, H.W., Hua, B.S., Yeung, S.K.: Locally stylized neural radiance fields (2023)
  * [62] Park, J.J., Florence, P., Straub, J., Newcombe, R., Lovegrove, S.: Deepsdf: Learning continuous signed distance functions for shape representation (2019)
  * [63] Peng, S., Niemeyer, M., Mescheder, L., Pollefeys, M., Geiger, A.: Convolutional occupancy networks. In: Computer Vision – ECCV 2020. Lecture Notes in Computer Science, 12348, vol. 3, pp. 523–540. Springer, Cham (Aug 2020). https://doi.org/10.1007/978-3-030-58580-8_31
  * [64] Qian, S., Kirschstein, T., Schoneveld, L., Davoli, D., Giebenhain, S., Nießner, M.: Gaussianavatars: Photorealistic head avatars with rigged 3d gaussians (2023)
  * [65] Radford, A., Kim, J.W., Hallacy, C., Ramesh, A., Goh, G., Agarwal, S., Sastry, G., Askell, A., Mishkin, P., Clark, J., et al.: Learning transferable visual models from natural language supervision. In: International conference on machine learning. pp. 8748–8763. PMLR (2021)
  * [66] Reiser, C., Peng, S., Liao, Y., Geiger, A.: Kilonerf: Speeding up neural radiance fields with thousands of tiny mlps (2021)
  * [67] Riegler, G., Koltun, V.: Free view synthesis. In: Computer Vision–ECCV 2020: 16th European Conference, Glasgow, UK, August 23–28, 2020, Proceedings, Part XIX 16. pp. 623–640. Springer (2020)
  * [68] Saleh, B., Elgammal, A.: Large-scale classification of fine-art paintings: Learning the right metric on the right feature (2015)
  * [69] Schwarz, K., Sauer, A., Niemeyer, M., Liao, Y., Geiger, A.: Voxgraf: Fast 3d-aware image synthesis with sparse voxel grids (2022)
  * [70] Seitz, S.M., Dyer, C.R.: View Morphing. SIGGRAPH (1996)
  * [71] Shin, D., Ren, Z., Sudderth, E.B., Fowlkes, C.C.: 3d scene reconstruction with multi-layer depth and epipolar transformers (2019)
  * [72] Simonyan, K., Zisserman, A.: Very deep convolutional networks for large-scale image recognition. arXiv preprint arXiv:1409.1556 (2014)
  * [73] Sitzmann, V., Thies, J., Heide, F., Nießner, M., Wetzstein, G., Zollhöfer, M.: Deepvoxels: Learning persistent 3d feature embeddings. In: CVPR (2019)
  * [74] Sitzmann, V., Zollhöfer, M., Wetzstein, G.: Scene Representation Networks: Continuous 3D-Structure-Aware Neural Scene Representations. In: NeurIPS (2019)
  * [75] Song, H., Choi, S., Do, H., Lee, C., Kim, T.: Blending-nerf: Text-driven localized editing in neural radiance fields (2023)
  * [76] Teed, Z., Deng, J.: Raft: Recurrent all-pairs field transforms for optical flow (2020)
  * [77] Tulsiani, S., Gupta, S., Fouhey, D., Efros, A.A., Malik, J.: Factoring shape, pose, and layout from the 2d image of a 3d scene (2018)
  * [78] Ulyanov, D., Vedaldi, A., Lempitsky, V.: Improved texture networks: Maximizing quality and diversity in feed-forward stylization and texture synthesis. In: Proceedings of the IEEE conference on computer vision and pattern recognition. pp. 6924–6932 (2017)
  * [79] Wang, C., Jiang, R., Chai, M., He, M., Chen, D., Liao, J.: Nerf-art: Text-driven neural radiance fields stylization (2022)
  * [80] Wang, W., Yang, S., Xu, J., Liu, J.: Consistent video style transfer via relaxation and regularization. IEEE Transactions on Image Processing 29, 9125–9139 (2020)
  * [81] Wei, L.Y., Levoy, M.: Fast texture synthesis using tree-structured vector quantization. In: Proceedings of the 27th annual conference on Computer graphics and interactive techniques. pp. 479–488 (2000)
  * [82] Wiles, O., Gkioxari, G., Szeliski, R., Johnson, J.: Synsin: End-to-end view synthesis from a single image (2020)
  * [83] Wimbauer, F., Yang, N., Rupprecht, C., Cremers, D.: Behind the scenes: Density fields for single view reconstruction (2023)
  * [84] Xu, S., Li, L., Shen, L., Lian, Z.: Desrf: Deformable stylized radiance field. 2023 IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops (CVPRW) pp. 709–718 (2023), https://api.semanticscholar.org/CorpusID:259237546
  * [85] Yu, A., Fridovich-Keil, S., Tancik, M., Chen, Q., Recht, B., Kanazawa, A.: Plenoxels: Radiance fields without neural networks (2021)
  * [86] Yu, A., Ye, V., Tancik, M., Kanazawa, A.: pixelnerf: Neural radiance fields from one or few images (2021)
  * [87] Zhang, K., Kolkin, N., Bi, S., Luan, F., Xu, Z., Shechtman, E., Snavely, N.: Arf: Artistic radiance fields (2022)
  * [88] Zhang, R., Isola, P., Efros, A.A., Shechtman, E., Wang, O.: The unreasonable effectiveness of deep features as a perceptual metric. In: CVPR (2018)
  * [89] Zhang, Y., He, Z., Xing, J., Yao, X., Jia, J.: Ref-npr: Reference-based non-photorealistic radiance fields for controllable scene stylization (2023)
  * [90] Zhang, Z., Liu, Y., Han, C., Pan, Y., Guo, T., Yao, T.: Transforming radiance field with lipschitz network for photorealistic 3d scene stylization (2023)
  * [91] Zheng, S., Zhou, B., Shao, R., Liu, B., Zhang, S., Nie, L., Liu, Y.: Gps-gaussian: Generalizable pixel-wise 3d gaussian splatting for real-time human novel view synthesis (2023)
  * [92] Zitnick, C.L., Kang, S.B., Uyttendaele, M., Winder, S., Szeliski, R.: High-quality video view interpolation using a layered representation. SIGGRAPH 23(3) (2004)