Ours + Denoising

Offline Reference

5c[width=, trim=0 260 0 90, clip]figures/fig_teaser/combined_greek_villa.jpg

(38.54166666666667, 35.15625)(0,0)[red,ultra thick] (0,0) rectangle (0.10312500000000001, 0.041249999999999995);
(35.104166666666664, 22.916666666666664)(0,0)[orange,ultra thick] (0,0) rectangle (0.10312500000000001, 0.041249999999999995);

Path tracing + ReSTIR

+ Secondary DDGI (ours)

+ DDGI Resampling (ours)

+ Denoising (ours)

22.5 ms
12.8 ms
18.4 ms
26.6 ms

The indirectly lit  scene rendered at $1920\times1080$ with 1 sample per pixel (spp) on an i7 6800k CPU and RTX 3090 GPU.
Direct illumination resampling, such as ReSTIR <cit.> (leftmost inset), reduces noise at the primary path vertex but does not affect the remainder of the path which is both noisy and expensive to trace.
Therefore, we propose replacing the remainder of the path, starting from the secondary vertex, with a cheap, noise-free approximation: an extended variant of DDGI <cit.> (second inset).
Our key observation is that DDGI thereby acts as a light source, allowing us to include it in the resampling algorithm (third inset).
This unifies the sampling of direct and indirect illumination.
Combined with denoising (fourth inset), scenes with complex indirect illumination such as this one can be rendered in real time from 1 spp, with quality approaching offline path tracing with hundreds of samples per pixel (fifth inset).

Interactive global illumination remains a challenge in radiometrically- and geometrically-complex scenes. Specialized sampling strategies are effective for specular and near-specular transport because the scattering has relatively low directional variance per scattering event. In contrast, the high variance from transport paths comprising multiple rough glossy or diffuse scattering events remains notoriously difficult to resolve with a small number of samples.
We extend unidirectional path tracing to address this by combining screen-space reservoir resampling and sparse world-space probes, significantly improving sample efficiency for transport contributions that terminate on diffuse scattering events.  Our experiments demonstrate a clear improvement – at equal time and equal quality – over purely path traced and purely probe-based baselines. Moreover, when combined with commodity denoisers, we are able to interactively render global illumination in complex scenes.

<concept_desc>Computing methodologies Ray tracing</concept_desc>

[500]Computing methodologies Ray tracing

§ INTRODUCTION

Modern physically-based production renderers rely primarily on variants of the unidirectional path tracing algorithm <cit.>, in which paths are traced from the camera and scattered in the scene until they reach an emitter. Path tracing scenes with high geometric and radiometric complexity can result in high-variance (i.e., noisy) images unless they take a prohibitively large number of samples.
To alleviate this, many Monte Carlo variance reduction-based solutions have been proposed, such as variants of importance sampling <cit.> and path guiding <cit.>.
However, even with advanced sampling techniques in place, hundreds of samples per pixel are typically required to produce a converged image, which is prohibitive for real-time applications.

Even though recent improvements in GPU ray tracing and effective denoising have greatly improved path tracing performance for complex scenes, real-time path tracing budgets remain limited to only few indirect scattering events (often just one) and scenes with only modestly complex illumination. As such, interactive graphics methods necessarily resort to approximate global illumination methods in addition to importance sampling. One such recent approach combines world-space irradiance probes with visibility-aware interpolation to rapidly approximate multi-bounce indirect illumination (DDGI) <cit.>.
Crucially, the normal-dependent irradiance can also be interpreted as cosine-prefiltered radiance <cit.> in the normal direction, allowing its use in both rough glossy as well as Lambertian diffuse transport.
As with most approximate solutions, these dynamic probes introduce bias – in the form of the transport that is computed – in exchange for a noise-free and fast result.

In contrast to interactive indirect illumination sampling, the sample efficiency and approachable scene complexity for ray-traced direct illumination has been greatly improved by recent a spatio-temporal resampling scheme (ReSTIR) <cit.>.
In the graphics pipeline, sampling is followed by full-screen post-processing, including denoising <cit.>.
So, the goal of a modern real-time sampling algorithm is not a fully converged image, but rather one with sufficiently low noise that after post-processing it is acceptably close to a converged image for the application. We address the efficient sampling problem in this work, assuming a downstream denoiser. We note that today's commodity denoisers both desirably conceal sampling noise and also undesirably exhibit various residual temporal errors that are beyond the scope of this work. Hence, we show the impact of denoising in Fig. <ref> for a static image to validate the unconverged targets, and then show the pre-denoising output of our method in all other results.

We present a new real-time global illumination method that combines the advantages of spatio-temporal resampling (improved sample efficiency for stochastic estimators) and dynamic diffuse probe volumes (smooth, interpolation-friendly transport proxies). Our method is efficient, treats direct and multi-bounce indirect effects, and is robust to radiometrically- and geometrically-complex settings.

Naïvely combining DDGI with ReSTIR (see Fig. <ref>, “Primary DDGI” column) will neither eliminate the probe grid artifacts nor address the discrepancy in their respective artifact/noise characteristics: DDGI's noise-free and biased indirect illumination, and ReSTIR's noisy and unbiased direct illumination.

We alleviate this discrepancy by postponing DDGI queries by one bounce (from the eye) – where sample contributions are evaluated during ReSTIR
(see Fig. <ref>, “DDGI in ReSTIR” column) – similarly to final gathering with photon mapping <cit.>. This hides a large portion of DDGI's bias and allows us to combine direct and indirect samples during resampling, selecting each sample proportional to its global transport contribution.
Where the original ReSTIR approach would normally resample direct illumination, it now treats both direct and indirect samples, each with similar noise characteristics due to the final gathering-like postponed DDGI sample, resulting in a sample efficient global shading estimate that is amenable to commodity denoising methods (see Fig. <ref>).

Our method takes a principled approach to balancing the performance-quality trade-off inherent to any combination of DDGI and ReSTIR, as evidenced in our analysis and benchmark against
purely path traced and purely probe-based
variants (Sec. <ref>). Concisely, our contributions are as follows:

* a categorization of contributions from each constituent technique,

* an in-depth analysis of sampling-based methods applied to these contributions,

* an interactive global illumination algorithm, motivated by our analysis, that combines resampling with a global transport approximation.

§ BACKGROUND AND RELATED WORK

*(Ir)radiance Caching.
Most modern real-time global illumination caches have their roots in Ward's seminal irradiance cache <cit.> and other early approaches to precomputed light transport <cit.>.
While initial work focused on adapting precomputed light transport to the harsh constraints of real-time rendering in static settings <cit.>, many recent techniques offer at least partial dynamics, such as dynamic lighting through precomputed radiance transfer <cit.>.
Orthogonally, radiance caching techniques overcome the Lambertian assumption by including the directional domain <cit.>.
Although today's selection of high-quality partially dynamic techniques is vast <cit.>, our goal is fully dynamic global illumination with as few constraints as possible.

To this end, we build upon the DDGI volume <cit.>, which consists of a 3D grid of directionally resolved irradiance probes that is updated in real-time through hardware ray-tracing.
Crucially, DDGI contains visibility information to prevent light leaking and it can approximate sufficiently rough glossy transport by re-purposing its directional dependence <cit.>.
These features make it – with mild modifications (Sec. <ref>) – a reasonably versatile approximation of global illumination for most phenomena other than specular reflections.
By querying DDGI at secondary path vertices, we employ a similar strategy to final gathering to conceal DDGI's limitations (e.g., coarse spatial discretization) behind the blurring effect of the primary scattering interaction at the cost of sampling noise. We then address the sampling noise by applying spatio-temporal reservoir sampling <cit.>.

Müller et al. <cit.> use a real-time trained neural radiance cache to approximate fully-dynamic global illumination.
They evaluate their cache at later path vertices to hide the artifacts of their neural network, but unlike this paper, they do not employ specialized importance sampling.
Crucially, their method is compatible with ours in that the caches can be interchanged in future investigations.

*Importance Sampling.
In offline rendering, importance sampling is the dominant technique for reducing sampling noise: the closer the sampling distribution matches the distribution of light, the less noise there is <cit.>.
With the advent of hardware ray tracing, it is now important to adapt established techniques to the real-time setting <cit.>.

Like with caching techniques, spatio-temporal reuse is key for high-quality importance sampling, as evidenced by bidirectional techniques <cit.>, resampling and mutation strategies <cit.>, as well as learned distributions via neural networks <cit.> or path guiding <cit.>.
Unfortunately, most reuse strategies come with a significant performance penalty, meaning that while they handle difficult illumination well, they are outperformed by naïve unidirectional path tracing under simple illumination.
This disqualifies them from the real-time setting that we strive for.
Path guiding has generally little overhead, which makes it a good choice for production rendering <cit.>, however adapting its underlying data structures to animated content in real-time is non-trivial ongoing work <cit.>.

Our importance sampling method of choice is ReSTIR <cit.>: a recent combination of importance resampling <cit.> and classic weighted reservoir sampling <cit.> that permits the reuse of a massive number of samples in constant time.
For estimating direct lighting, ReSTIR generates many candidate samples on the light sources in the scene, resamples those candidates across space and time proportional to their predicted contribution, and then traces a shadow ray to determine the visibility of the selected sample(s).
This scheme not only has negligible overhead compared to simple path tracing, but also gracefully handles dynamic content.

In contrast, we apply ReSTIR to global illumination by targeting it at the sum of direct illumination from light sources and indirect illumination stored in DDGI probes <cit.> of irradiance (for diffuse reflection) and angularly-filtered radiance <cit.> (for rough glossy reflection).
This converts the global illumination problem into a purely direct illumination problem for the high variance case of low-frequency angular scattering, by treating DDGI as a light source.
In contrast to multiple importance sampling <cit.>, this means that we draw a single sample approximately proportional to the total transported light rather than multiple samples that are heuristically combined.
Note that the result has two symbiotic levels of reuse: (i) screen-space resampling by ReSTIR for both direct and indirect light as well as (ii) world space DDGI probes which contain spatio-temporal aggregates of global illumination.

Concurrent work of Ouyang et al. <cit.> proposes an alternative mechanism for spatio-temporal resampling of global illumination: they target ReSTIR at single-sample Monte Carlo estimates – rather than a cache – which can be likened to importance sampling of virtual point lights.
Compared with our approach, theirs is capable of less biased (optionally unbiased) rendering, but lacks world-space spatio-temporal reuse and early path termination, which in our case is handled by the DDGI volume.

As we show in our results, combining the underlying ideas of ReSTIR, DDGI, and path tracing into a new sampling strategy produces a less-biased and less noisy result than naïvely compositing the results of separately-computed ReSTIR direct, DDGI diffuse/rough-glossy indirect, and path traced near-specular contributions.

§ ALGORITHM

In order to render images with global illumination in
real-time, we strive to efficiently simulate radiance transport.
The rendering equation <cit.>

\begin{equation} \label{Eqn:REQ}
L = L_e + T_f \; L
\end{equation}

describes the outgoing radiance $L : \R^3 \times \mathbb{S}^2 \rightarrow \R^3$
as the sum of the radiance $L_e$ emitted by the light sources
and the transported radiance $T_f L$, where $T_f$ is a higher order operator that maps between two functions (similar to notation
defined by Veach <cit.>).
The symbol $f$ represents
the bidirectional scattering distribution function (BSDF),
describing how the surfaces in the scene transport radiance.
For our derivation, it is useful to partition radiance transport

\[
T_f = T_{\bsdfDiff} + T_{\bsdfGlossy}
\]

into diffuse $T_{\bsdfDiff}$ and glossy $T_{\bsdfGlossy}$ transport, the latter of which including specular transport as the special case of small surface roughness.

With this operator notation at hand,
the dynamic diffuse global illumination (DDGI)
algorithm <cit.> is

\begin{equation}
\approx L_e
+ T_{\bsdfDiff} \; L_e
+ L_{\text{DDGI}}
+ T_{\bsdfGlossy} \; L \;, \label{Eqn:DDGI}
\end{equation}

in which $\Lddgi \approx \sum_{i=2}^\infty (T_{\bsdfDiff})^i \; L_e$ is an approximation of the diffuse transported radiance without the directly visible light sources ($i$ starts at 2).
The other terms are estimated by sampling.
Inserting the DDGI approximation into equation (<ref>) and assuming equality results almost in the rendering equation (<ref>), missing only higher order glossy transport.

Majercik et al. <cit.> extended DDGI to include this missing contribution for the special case of specular reflection, which we will build on in <ref>.
The “Primary DDGI” column in Fig. <ref> and <ref>
shows results of this extended DDGI algorithm. As compared to path tracing
<cit.>, the images are smooth and noise only results
from sampling the direct diffuse illumination $T_{\bsdfDiff} \; L_e$ and the recursion for glossy transport $T_{\bsdfGlossy} \; L$.
Yet, the irradiance probe approximation of $L_{\text{DDGI}}$
shows visible artifacts that we resolve by postponing its evaluation to secondary path vertices.
We develop our new algorithm in the following sections.

(-62.5, -2) $\pos$
(-55.5, 11.5) $\posy$
(-13, -2) $\pos$
(1, 30.5) $\posz$

We compute direct and indirect illumination in a unified way by targeting the ReSTIR algorithm at the sum of emitted light $L_e$ and the DDGI approximation $\Lddgi$.
Left: two points are sampled. One (red) is sampled on the emissive surfaces and another (green, $\posy$) by BSDF sampling at the primary vertex $\pos$.
Right: with all samples across pixels and past frames converted to the area measure, a single sample $\posz$ is selected with probability proportional to ${L_e+\Lddgi}$
and a shadow ray (red) is traced.
The radiance leaving $\pos$ towards the camera is estimated with low variance by combining the reflected light from $\posy$ and $\posz$ through the method of Kollig and Keller <cit.>.
(Near-)specular transport at $\posy$ and $\posz$ is estimated through recursion (blue) and the remaining transport by $\Lddgi$.

§.§ Sampling all Surfaces with ReSTIR

As the DDGI approximation can be queried at any point
of the scene surface, we consider the whole scene surface as a
light source rather than only the actual light sources.
Our new sampling algorithm performs the following steps:

* Generate candidate samples on all surfaces of the scene.
To this end, we uniformly sample a small number of positions on emissive surfaces and generate additional samples by tracing secondary rays in directions sampled from the BSDF.

* Select one candidate by first sampling from a weighted reservoir proportional to the sample contribution, then
resample spatially among neighboring pixels, and resample temporally among previous frames, proportional to the sum ${L_e + \Lddgi}$.

* Trace one shadow ray to the selected sample point.

* If visible, shade from that sample point, using the emissive contribution $L_e$ and adding the contribution
from the DDGI volume.

As with original ReSTIR, our unified sampling scheme traces only a single ray to the selected sample (see Figure <ref>).
However, as the original DDGI volume only contains the diffuse
indirectly reflected light without direct reflections as shown in equation (<ref>),
we need to add these, as well as glossy contributions, which we describe next.

§.§ Augmenting the DDGI Approximation

As mentioned in Sec. <ref>,
integrating $L_{\text{DDGI}}$ from secondary vertices over the hemisphere at the primary vertex
would result in too dark images, because
the DDGI approximation lacks the diffuse direct illumination
and the radiance from non-diffuse transport.
In fact, adding these missing terms to the DDGI approximation recovers an approximation of the transport operator

\begin{equation} \label{Eqn:DGI}
L_{\text{DDGI}} + T_{\bsdfDiff} \; L_e + T_{\bsdfGlossy} \; L
\approx T_{\bsdfDiff} \; L + T_{\bsdfGlossy} \; L
= T_f \; L .
\end{equation}

Primary DDGI     Secondary DDGI

(26.5625, 28.125)(0,0)[orange,ultra thick] (0,0) rectangle (0.1546875, 0.061875);
(61.979166666666664, 23.177083333333332)(0,0)[red,ultra thick] (0,0) rectangle (0.1546875, 0.061875);

Primary DDGI

Secondary DDGI

Using the DDGI approximation $\Lddgi$ at the primary vertex (left) shows light leaking artifacts beneath the couch and does not correctly reproduce indirect contact shadows on the table, vases, and plate. Using our augmented $\LddgiP$ at the secondary vertex (middle) exhibits no leaks, recovers some of the indirect contact shadows and is closer to the path traced reference (right).

Adding the missing terms
by recursive path tracing would increase the noise and involve
further shading and tracing cost.

Instead, we augment the DDGI approximation in two steps, which are most easily explained by examining the underlying irradiance approximation that DDGI makes.
Given a 3D position and 2D normal vector, the DDGI probe volume stores irradiance from indirect diffuse reflections, which is approximated by repeatedly evaluating the following double-bounce transport <cit.>

\begin{align}
\Eddgi(\pos, \normal) &\approx \int (\Lddgi + T_{\bsdfDiff} \; L_e)(h(\pos, \diri), -\diri) \langle \normal, \diri \rangle \,\Diff{\diri} \,, \nonumber \\
\Lddgi(\pos, \diri) &:= \frac{\rho(\pos)}{\pi} \; \Eddgi(\pos, \normal) \,,
\end{align}

where $h$ is the ray tracing operation and $\rho$ is the diffuse albedo.
Repeated Monte Carlo estimation of this equation at the centers $\pos$ of all DDGI probes converges to the aforementioned approximation
$$\Lddgi \approx \sum_{i=2}^\infty (T_{\bsdfDiff})^i \; L_e \,.$$

To include direct reflections and non-diffuse transport, we replace the double-bounce transport with simpler single-bounce recursion

\begin{align}
\EddgiP(\pos, \normal) \approx \int (\LddgiP + L_e)(h(\pos, \diri), -\diri) \langle \normal, \diri \rangle \,\Diff{\diri} \,,
\label{eq:probeUpdate}
\end{align}

where $L_e$ no longer undergoes a diffuse interaction and $\LddgiP$ will be adapted to include glossy transport in the following section.
The approximation thus converges to the full transport operator

$$\LddgiP \approx T_{\bsdf} \; L \,.$$

Including direct illumination (i.e. untransported $L_e$) in the probe update equation (<ref>) produces results that are too bright whenever probe centers are closer to light sources than the surfaces that they shade.
This causes surfaces to receive more (secondary) direct illumination than they should – a common problem in probe-based caches.
To alleviate this, we approximate the true distance to the surface by querying the average visibility already computed by the probes in the backwards direction <cit.> (dashed red lines), clipping that value to the probe boundary (pink), and adding it to the length of the corresponding probe update rays (solid red arrows). Because the update rays are traced per probe, this costs just a texture read without any probe weight computation.

We then use the inverse square of the total distance, i.e., solid + dashed, to attenuate $L_e$.
In contrast to direct illumination computed at probe centers, this heuristic slightly underestimates direct illumination.
We therefore expose a user-tunable slider to scale the additional attenuation per scene to correct for brightness differences that would otherwise be a significant source of bias.

(25.3125, 33.854166666666664)(0,0)[red,ultra thick] (0,0) rectangle (0.10312500000000001, 0.041249999999999995);
(31.354166666666668, 16.145833333333332)(0,0)[orange,ultra thick] (0,0) rectangle (0.10312500000000001, 0.041249999999999995);

Secondary DDGI

+Specular Recursion

For shading, we recursively trace (near-)specular interactions until their footprint grows sufficiently large to blur away small-scale artifacts of the DDGI approximation (middle).
This produces less biased results than always querying the DDGI approximation at the secondary vertex (left).

§.§ Including Glossy Illumination

Recall that the reflected light from a diffuse material with albedo $\rho$ can be cheaply approximated by looking up $\frac{\rho(\pos)}{\pi} \;\EddgiP(\pos, \normal)$.

Majercik et al. <cit.> make the observation that the irradiance approximation can additionally be used
to very coarsely approximate specular transport by substituting the normal $\normal$ with the direction of specular reflection $\dirr$.
The irradiance $\Eddgi(\pos, \dirr)$ can then be re-interpreted as a prefiltered (by the cosine term) coarse approximation of incident radiance up to a normalization factor of $2\pi$ which must be added.
In the illustration, the green lobes visualize cosine-weighted prefiltering of incident radiance and the red arrows indicate the query direction of $\Eddgi$.

(-73, 4) $\pos$
(-70, 30) $\normal$
(-28, 4) $\pos$
(-25, 30) $\normal$
(-68, 10) $\diri$
(-8, 10) $\diri$
(-2, 20) $\dirr$
(-90, 20) $\diro$
(-44, 20) $\diro$
(-82, -5)  Diffuse
(-37, -5)  Specular

While Majercik et al. use this strategy for just specular transport, we generalize it to arbitrary glossy BSDFs – in our case microfacet models – by Monte Carlo estimation.
More specifically, we approximate reflected radiance by our augmented DDGI model

\begin{align}
\LddgiP(\pos, \diro) \label{Eqn:lddgiplus}
&= \underbrace{\frac{\rho(\pos)}{\pi} \; \EddgiP(\pos, \normal)}_{\text{Diffuse}}\\
&+ \underbrace{\frac{1}{2\pi} \int \EddgiP(\pos, \diri) \bsdfGlossy(\pos, \diro, \diri) \langle \normal, \diri \rangle \,\Diff{\diri} }_{\text{Glossy}} \nonumber
\end{align}

as the decomposition into diffuse and glossy transport, where we estimate the glossy integral with a single Monte Carlo sample drawn proportionally to the glossy BSDF component $\bsdfGlossy$.

*Specular recursion for shading.
We use our radiance approximaton $\LddgiP$ in three parts of the algorithm: (i) in the update rule of $\EddgiP$ (<ref>), (ii) to resample proportional to ${(L_e + \LddgiP)}$, and (iii) for shading.
The latter use case – shading – requires special care to avoid exposing too much of the inherent bias of $\LddgiP$ through (near-)specular interactions.

We thus trace shading paths recursively until their scattering footprint is sufficiently spread out to blur DDGI's visual artifacts.
To this end, we adopt the path termination strategy of Müller et al. <cit.>, which is based on the isotropic path footprint approximation of Bekaert et al. <cit.>.

More specifically, we approximate the footprint of a path with vertices $\pos_1\cdots\pos_n$ as

\begin{align}
a(\pos_1\cdots\pos_n) &= {\left( \sum_{i=1}^{n-1} \sqrt{\frac{\|\pos_{i} - \pos_{i+1}\|^2}{p_i(\dir_{\mathrm{i},i} \,|\,\pos_{i}, \dir_{\mathrm{o},i}) \, \langle \normal_{i+1}, -\dir_{\mathrm{i},i} \rangle}} \right)}^2 \,,
\end{align}

where $p_i$ is the BSDF sampling density at the $i$-th vertex.
At the primary vertex, $p_1$ is proportional to the entire BSDF $\bsdf$, whereas ${p_i; \, i>1}$ is proportional to only the glossy portion of the BSDF $\bsdfGlossy$, because we simply terminate the recursion early and query $\frac{\rho(\pos_n)}{\pi}\;\EddgiP(\pos_n, \normal_n)$ when the diffuse portion is sampled.
When the glossy portion is sampled, we terminate the recursion by estimating the glossy integral of Eq. (<ref>) as soon as $a(\pos_1\cdots\pos_n) > a_0 \cdot c$, where

\begin{align}
\AreaHeuristic_0 := \frac{\|\pos_0 - \pos_1\|^2}{4\pi \; \langle \normal_1, -\dir_{\mathrm{i},i} \rangle}
\end{align}

is an approximation (up to constant factors) of the pixel footprint projected into the scene, $\pos_0$ is the camera position, and $c$ is a user-chosen threshold.
See Bekaert et al. <cit.> and Müller et al. <cit.> for details.

We empirically set the threshold ${c = 0.2}$ to obtain a satisfactory noise versus bias trade-off.
Fig. <ref> confirms that this helps reduce artifacts on highly glossy surfaces without affecting rough glossy and diffuse surfaces.

§.§ Correcting Self-shadow Bias

Default SS Bias
View Path Bias

(43, 23) $d$
(93.5, 21.5) $d$

Default DDGI self-shadow bias

New view path bias

DDGI combats self-shadowing by pushing the evaluation point away from surfaces (violet arrow) by a fixed distance $d$ that is proportional to the probe size. Left: DDGI uses a linear combination of the view direction and the surface normal.
This may push the evaluation point through geometry, leading to light leaking.
Right: we instead move the evaluation point a fixed distance back along the path, which is guaranteed to not penetrate surfaces.

The original DDGI algorithm applies an offset to the positions at which $\Lddgi$ is queried to avoid computing visibility weights exactly at a surface where their variance is high. This offset is called the “self-shadow” bias, and is computed as a weighted combination of the view and normal vector at the queried point.

While the self-shadow bias was sufficient for the original primary-vertex DDGI algorithm, using it unmodified in our indirect sampling strategy showed light leaking artifacts (Figure <ref>). These artifacts arise from very short distances between the view point and the query point, as might arise from sharp corners. To address these artifacts, we bias the query point along the view path as opposed to the surface normal and view direction. Results using our view path bias against the original self-shadow bias are shown in Figure <ref>.

§.§ Taming the Weak Singularity in the Geometry Term

In order to perform spatio-temporal resampling among different shading locations, the geometry term is introduced as part of the change of the integration measure.
Unfortunately, the weak singularity in the geometry term may lead to unbounded variance in the resampling procedure in geometric creases; see Fig. <ref> (middle).

In typical path tracing applications, uniform multiple importance sampling <cit.> of light sources would bound the variance when light sources (almost) touch the surfaces that they shade.
But this is an impractical solution in our case, where the entire scene is considered a light source.
It may be feasible to non-uniformly sample all relevant scene surfaces according to a data structure that is constructed online, but we instead take a simpler approach that does not require multiple importance sampling: we partition the integrand to bound the variance <cit.>.

To avoid the weak singularity caused by samples very close to the primary vertex, we clamp the computed geometry term to a maximum value of $G_\text{max} = 1$ and use the notation
$\lceil T_{\bsdf} \rceil$ for the
transport with this bounded geometry term.
The residual transport is denoted by $\lfloor T_{\bsdf} \rfloor$,
such that

\[
T_{\bsdf} = \lceil T_{\bsdf} \rceil + \lfloor T_{\bsdf} \rfloor .
\]

$\lfloor T_{\bsdf} \rfloor$ is evaluated by
an auxiliary BSDF sample in the sampling step described in Sec. <ref>.
The improved variance is shown in Fig. <ref>.

(28.072916666666664, 14.0625)(0,0)[orange,ultra thick] (0,0) rectangle (0.10312500000000001, 0.041249999999999995);
(58.90625000000001, 34.895833333333336)(0,0)[red,ultra thick] (0,0) rectangle (0.10312500000000001, 0.041249999999999995);

PT + ReSTIR

DDGI Resampling

DDGI Resampling

without bounding

with bounding (ours)

Even though spatio-temporal resampling of DDGI (middle) improves the noise of path tracing (left), outliers (“fireflies”) cause distracting artifacts in geometric concavities.
These fireflies are due to the weak singularity in the geometry term of the rendering equation. Using the method of Kollig and Keller <cit.>, we bound the geometry term during resampling and then trace an auxiliary ray to estimate the residual transport (right). This eliminates fireflies and preserves a significant noise reduction. The images were rendered at 128 samples per pixel.

(38.54166666666667, 48.17708333333333)(0,0)[red,ultra thick] (0,0) rectangle (0.10312500000000001, 0.041249999999999995);
(35.104166666666664, 35.416666666666664)(0,0)[orange,ultra thick] (0,0) rectangle (0.10312500000000001, 0.041249999999999995);

DDGI Resampling

+ Query Caching

MAPE: 0.63
MAPE: 0.64

50.3 ms
18.4 ms

Caching the DDGI query for resampling is more efficient: while the increase in noise level and MAPE is negligible in query caching, the reduction in frame rendering time is substantial.

In summary, our new algorithm to approximate the rendering equation
amounts to summing separate estimators of each term of

\begin{equation}
\approx L_e
+ \underbrace{\lceil T_{\bsdf} \rceil \; (L_e + \LddgiP)}_{\text{bounded geometry term}}
+ \underbrace{\lfloor T_{\bsdf} \rfloor \; (L_e + \LddgiP)}_{\text{residual transport}} , \label{Eqn:Algorithm}
\end{equation}

* $L_e$ is evaluated at the primary vertex,

* $\lfloor T_{\bsdf} \rfloor$ by importance sampling the BSDF, and

* $\lceil T_{\bsdf} \rceil$ by spatio-temporal reservoir resampling.

§.§ Improved Performance by Caching DDGI Queries

During the spatio-temporal resampling step, our algorithm requires numerous DDGI queries:
one for each candidate's contribution to the shaded pixel, which, in our implementation, amounts to 1 temporal and 3 spatial candidates per pixel.

(-63, 2) $\pos_2$
(-48, 2) $\pos_1$
(-33, 2) $\pos_3$
(-10, 42) $\posy_2$
(-20, 42) $\posy_1$
(-40, 42) $\posy_3$

In the illustration, we depict the simpler case of resampling among just 2 spatial candidates $\posy_2$ and $\posy_3$ for shading at the center vertex $\pos_1$.
For each candidate, we must query its contribution $\LddgiP(\posy_i, \posy_i\!\rightarrow\!\pos_1)$ to the center vertex, where ${\posy\!\rightarrow\!\pos}$ is the direction vector pointing from $\posy$ towards $\pos$ (dashed arrows).

These ${\text{\#vertices} \times \text{\#candidates}}$ queries can be avoided by approximating them with the $\text{\#vertices}$ values $\LddgiP(\posy_i, \posy_i\!\rightarrow\!\pos_i)$, which have to be evaluated for shading each vertex $\pos_i$ in any case.
This saves a factor of $\text{\#candidates}$ queries.
Importantly, using approximate values in the resampling step does not introduce additional bias in the rendered image – merely additional noise.

Nonetheless, we analyze the error of this approximation by considering its three sources:

* the scene and the DDGI volume may change each frame,

* the shading location $\pos$ is different, and

* the view-path bias (Sec. <ref>) depends on the path prefix

each of which become small if we apply the assumptions of our method: local pixel neighborhoods, near-diffuse scattering, and short frame times.
The approximation error is thus reasonably small in practice (see Fig. <ref>) and we use this optimization in all our results.

§ RESULTS

<cit.>     <cit.>     3cOurs

Primary DDGI
PT + ReSTIR
Secondary DDGI
+DDGI in ReSTIR

MAPE / Frametime (ms):
0.44 / 10.2 ms
1.35 / 49.1 ms
0.79 / 10.8 ms
0.45 / 15.8 ms
0.22 / 24.0 ms

MAPE / Frametime (ms):
0.80 / 12.2 ms
1.10 / 22.5 ms
0.78 / 12.8 ms
0.64 / 18.4 ms
0.38 / 26.6 ms

MAPE / Frametime (ms):
0.07 / 8.62 ms
1.73 / 18.9 ms
0.52 / 9.05 ms
0.23 / 13.5 ms
0.08 / 22.0 ms

MAPE / Frametime (ms):
0.22 / 6.73 ms
0.76 / 12.4 ms
0.38 / 7.00 ms
0.18 / 10.9 ms
0.12 / 18.3 ms

MAPE / Frametime (ms):
0.18 / 7.89 ms
0.88 / 12.7 ms
0.54 / 8.15 ms
0.17 / 13.1 ms
0.05 / 21.2 ms

MAPE / Frametime (ms):
0.27 / 10.0 ms
1.18 / 54.5 ms
0.45 / 10.8 ms
0.46 / 18.4 ms
0.19 / 26.3 ms

Comparison of rendering algorithms at 1 sample per pixel with respect to noise and bias.
All algorithms are based on an unbiased path tracer that uses ReSTIR <cit.> to estimate direct illumination at the primary vertex.
As compared to this baseline, dynamic diffuse global illumination (Primary DDGI)
renders smooth indirect illumination. Bias artifacts inherent with the DDGI
approximation can be ameliorated by pushing the query of DDGI one bounce further into the light path (Secondary DDGI).
Facilitating the ReSTIR sampling mechanism to jointly importance sample these DDGI queries with direct illumination, the induced noise is reduced (+DDGI in ReSTIR), and is further amenable to commodity denoising methods (+Denoising).
We also show a failure case () where the outer walls consist of flat planes that receive strong sunlight from the outside.
In this difficult scene, our method is unable to hide DDGI light leaking artifacts (red inset) and BSDF importance sampling (Secondary DDGI) performs slightly better than ReSTIR sampling.

We implemented DDGI resampling in Direct3D 12 using the Falcor rendering framework <cit.>.
All results were generated on a high-end desktop machine with an Intel i7-6800K CPU and an NVIDIA RTX 3090 GPU.

Fig. <ref> summarizes our results, comparing visuals, render time, and mean absolute percentage error (MAPE) of 5 different rendering techniques at 1 sample per pixel.
MAPE is defined as $\frac{1}{N} \sum_{i=1}^N |v_i - \hat{v}_i| / (\hat{v}_i + \epsilon)$, where $\hat{v}_i$ is the value of the $i$-th pixel in the reference image, $v_i$ is the value of the $i$-th rendered pixel, and $\epsilon = 0.01$ prevents near-black pixels from dominating the metric.

All rendering techniques are based on an unbiased path tracer that uses ReSTIR <cit.> to estimate direct illumination at the primary vertex.

* PT + ReSTIR: The aforementioned baseline path tracer.

* Primary DDGI: The original DDGI algorithm <cit.>, which approximates indirect illumination at the primary vertex of the path.

* Secondary DDGI: Our augmented DDGI algorithm approximates global illumination at the secondary vertex of the path (Sec. <ref>–<ref>). Near-specular transport is traced recursively.

* + DDGI in ReSTIR: The same as before, but the DDGI approximation is included in the ReSTIR algorithm (Sec. <ref>). The diffuse weak singularity is separately sampled (Sec. <ref>).

* + Denoising: The same as before, but with OptiX denoising.

As expected, the DDGI algorithm <cit.> has the least noise with the greatest performance (“Primary DDGI” column).
However, it also exhibits significant visual artifacts such as flat shading (e.g., in the ), a lack of ambient occlusion (e.g., in the ), and light leaking (e.g., in the ).
At the other extreme, an unbiased path tracer produces highly accurate results but requires a long time for its noise to converge away in globally illuminated scenes, even when using ReSTIR <cit.> for direct lighting (“PT + ReSTIR” column).

Time to converge to equal MAPE

Our proposed indirect use of DDGI probes – provided that the probes contain the energy of the full transport (Sec. <ref> and <ref>) – reduces noise while introducing only little bias, because the inaccuracies of the DDGI probes are hidden behind the first scattering event (“Secondary DDGI” column).
Additionally, the performance is improved, because paths are terminated early into DDGI probes.
Incorporating the indirect lighting injected by the DDGI probes in the resampling step further reduces noise (“DDGI in ReSTIR” column).
Overall, the noise is now low enough that adding the OptiX denoiser results in more visually pleasing images than the original DDGI algorithm, albeit at an on average 1.5-2$\times$ greater cost.

We also show a failure case () where the outer walls consist of flat planes that receive strong sunlight from the outside.
In this difficult scene, secondary DDGI queries are unable to hide DDGI's light leaking artifacts (red inset) and BSDF importance sampling (“Secondary DDGI” column) performs slightly better than ReSTIR sampling.
This shows that further work is needed in making DDGI more resilient to thin geometry as well as in understanding the circumstances in which ReSTIR can be outperformed by its candidate generation strategies.

*Quantitative evaluation.
To gauge the convergence improvements offered by using DDGI at the secondary path vertex (with or without resampling), we list the time it takes for all noisy methods to reach equal error in Tab. <ref>.
In all scenes, except for the , DDGI resampling converges the fastest – in some scenes by a significant margin – even when taking into account the slightly increased cost of rendering.

To better understand this cost, we break it down by component in Tab. <ref>.
As expected, terminating paths into DDGI reduces the ray tracing cost because paths are shorter, even when taking into account the additional cost of querying DDGI during the shading step, as well as the cost of updating the DDGI volume every frame.
However, perhaps unexpectedly, resampling only becomes marginally more expensive by incorporating DDGI in it (by $1.7$ ms on average) thanks to the caching of previous DDGI queries (Sec. <ref>).
The remaining overhead of DDGI resampling arises in tracing & shading of (near-)specular path suffixes, which must now be performed twice per pixel: once for the BSDF sample that estimates $\lfloor T_{\bsdf} \rfloor$ and once for the resampled vertex that estimates $\lceil T_{\bsdf} \rceil$ (see Sec. <ref>).

Rendering cost by component

§ DISCUSSION AND FUTURE WORK

The following discussion focuses on future avenues for potential performance and noise improvements.

To reduce the remaining resampling cost, it might be feasible to subsample the generation of new candidates: for example, only every $n$-th pixel could generate novel samples from the BSDF and from light sources.
All other pixels would merely resample among spatio-temporal neighbors.
Since this strategy is rather orthogonal to our work – it would benefit all ReSTIR-type algorithms – we did not include it in our technique.
However, we feel that it warrants a detailed analysis in the future.

*Alternative Caching Strategies.
While we focused on the specific use of DDGI, we believe other caching strategies, such as the recent neural radiance caching <cit.>, can similarly benefit from importance sampling through ReSTIR.
Another example is the concurrent work of Ouyang et al. <cit.> who apply ReSTIR to a cache that is effectively a collection of spatio-directional point light sources.
Their approach represents a different point on the Pareto front that trades off noise, bias, and performance – one that has lower bias (and can even be unbiased) at the cost of tracing longer paths, leading to extra noise.

§.§ Alternative Variance Reduction Strategies

In real-time rendering, sampling with low variance is key due to the limited budget of rays to be traced.
In this context, we also analyzed the efficiency of utilizing DDGI as a control variate, as well as that of using it for path guiding instead of resampling.
Both approaches were overall less efficient than the presented DDGI resampling, but we will nonetheless describe them in the hope that future algorithms may build on top of them.

DDGI Resampling

+ Control Variate

(78.38541666666666, 31.510416666666664)(0,0)[orange,ultra thick] (0,0) rectangle (0.10312500000000001, 0.041249999999999995);
(31.614583333333336, 32.86458333333333)(0,0)[red,ultra thick] (0,0) rectangle (0.10312500000000001, 0.041249999999999995);

DDGI Resampling
+ Control Variate

of DDGI

Analysis of DDGI Resampling (left split) augmented by DDGI used as a control variate at the primary vertex (right split).
The transient artifacts visible at low sampling rates are due to the underlying probe volume as can be seen by the visualization of DDGI.
At 1 spp, these are too salient to ignore, although with higher sample counts both DDGI Resampling and DDGI Resampling with control variate converge to the same result.

*DDGI as Control Variate.
For scenes with simple occlusion, the original DDGI data structure provides a low-bias, noise-free result. This can sometimes be desired over our new resampling, which will introduce noise even in cases where the resulting lower bias is not visually noticeable. To combat this,
we explored extending our solution by using primary hitpoint irradiance $\EddgiP$ as a control variate of diffuse transport.
That is, we assumed that ${\frac{\rho}{\pi}\,\EddgiP}$ at primary vertices yields the correct indirect illumination and we resampled proportional to the remaining non-diffuse transport plus (neglecting constants for brevity) the absolute difference between ${(\LddgiP + L_e)}$ at the secondary vertex and the spatio-directionally nearest DDGI update ray among those that were traced while rendering the last frame.
This amounts to using the DDGI update rays as a spatio-directionally piecewise constant control variate and their average, $\EddgiP$, as its exact integral.

When shading, we then applied DDGI to the primary vertex (as described in <cit.>) and added the difference (not the absolute value) between ${L_e + \LddgiP}$ at the secondary vertex and the nearest DDGI update ray.

Adding this difference in the shading step may result in negative radiance values, which frequently manifest as negative fireflies that cause black pixels.
We omitted spatio-temporal probe interpolation for
the DDGI control variate, as otherwise the value of $\LddgiP$ at the primary vertex
would not be an exact integral of the control variate.

Fig. <ref> shows our results.
Though this algorithm converges to the same (biased) solution as plain DDGI resampling, at one sample per pixel, the
control variate exposes light leaks similar to the naïve
ReSTIR + DDGI combination. At the same time, the noise level does not improve upon plain DDGI resampling, which produces less prominent
light leaks.

While using DDGI as a control variate may be beneficial
in some settings,
in general, it appears not sufficiently efficient and remains an avenue for future research.

*DDGI for Path Guiding.
Based on the DDGI data structure, guiding rays towards where
the radiance comes from is straightforward. Given a position
in space, we can query the closest DDGI probe and build the
cumulative distribution function (CDF) from the results of
its update rays to sample a guiding direction.
While the results of the probe rays are stored, building
the CDF and sampling from it was too slow
to increase the overall efficiency of our algorithm.

In addition, using $\LddgiP$ to normalize the PDF and sample the guiding direction
in an unbiased way requires the same modifications to be made to the DDGI volume that were
made to the control variate. These make the volume less stable and thus less effective for path guiding.
An alternative would be porting efficient path guiding data structures <cit.>
to the GPU.

*Combining Primary and Secondary DDGI.
Even though we focus on hiding the bias of DDGI behind scattering interactions, there are situations in which DDGI's bias is very low in the first place and does not warrant the extra noise caused by secondary queries; see the orange inset of the scene in Fig. <ref>.
It likely pays off to design an automatic mechanism for detecting such situations and to select among primary and secondary DDGI queries accordingly.

*Path Space Filtering.
Lastly, alternatively to screen-space denoising, we believe that additional spatio-temporal reuse through path space filtering could be beneficial.
While ReSTIR amounts to spatio-temporal filtering of sampling probabilities and DDGI amounts to spatio-temporal filtering of radiance at secondary vertices, path space filtering would provide additional spatio-temporal filtering of radiance at primary vertices.
Recent hash-based implementations <cit.> can run with little overhead on modern GPUs.

§.§ Limitations

The limitations of our approach are largely inherited from its components: ReSTIR and DDGI.
ReSTIR is a screen-space technique whose reuse capability depends highly on the availability of motion vectors, slow camera motion, and transparency.
These downsides can be mitigated by transitioning to a world-space representation <cit.>, but another limitation remains: through resampling, samples become spatio-temporally correlated, which reduces the total amount of information available to modern denoisers.
Effective denoising in the presence of ReSTIR samples is still ongoing work.

The limitations induced by DDGI are two-fold.
First, being grid-based, DDGI does not scale easily to expansive scenes where the camera may only focus on small regions.
And second, DDGI is incapable of resolving fine spatio-directional detail such as intricate caustics, shadows, ambient occlusion or near-specular reflections.
These limitations are hidden to a degree when DDGI is queried after a primary scattering interaction that is sufficiently diffuse or after undergoing sufficiently many secondary interactions, but better quality can be achieved by using a more accurate cache <cit.>.

§ CONCLUSION

We have combined DDGI <cit.> and ReSTIR <cit.>, which, at first glance, might seem orthogonal.
DDGI renders indirect illumination and ReSTIR direct illumination.
However, the appearance is deceiving and a small modification to the DDGI algorithm – querying the probe volume at secondary path vertices rather than primary ones – unifies the two approaches into an algorithm that performs much better than its constituent parts.
By querying DDGI at secondary vertices, it acts as a scene-spanning light source, effectively turning the global illumination problem into a much simpler direct illumination problem, which ReSTIR can holistically attack.
This observation is a general one that has inspired rendering algorithms since the 90s, making us excited about future work in this direction.

Quantitatively, the combined algorithm is about 60% more expensive than either of the two prior works, but makes up for its cost in terms of low noise and bias.
It achieves equal error over an order of magnitude more quickly.
Together with denoising, real-time rendering of global illumination in dynamic scenes is feasible.

§ ACKNOWLEDGEMENTS

We would like to thank David Luebke and Aaron Lefohn for supporting this work. We also thank Benedikt Bitterli and Chris Wyman for providing the initial codebase in Falcor <cit.>, Chris Wyman and Peter Shirley for many early ideas and fruitful discussion, and Nikolaus Binder and Christoph Schied for proofreading the article and many valuable comments. The Pink Room scene was modified from “The Modern Living Room" by Wig42 and the Living Room scene was modified from “The White Room" by Jay-Artist. The Greek Villa was modified from “3D Greek Villa" by holy_diver.

Missing 'biblatex' package
The bibliography requires the 'biblatex' package.

Coriolis Group

booktitleGraphics Programming Black Book
titleQuake’s lighting model: Surface caching

booktitleIn ACM SIGGRAPH ’86 Course Notes - Developments in Ray Tracing
titleBackward Ray Tracing

Association for Computing Machinery

journaltitleACM Trans. Graph.
titleSpecial Issue on Production Rendering

journaltitleComputer Graphics Forum
titleBayesian Collaborative Denoising for Monte Carlo Rendering
CCS Concepts,•Computing methodologies → Ray tracing,Image processing

booktitleMonte Carlo and Quasi-Monte Carlo Methods 2020
titleMassively Parallel Path Space Filtering
pagesto appear

Chichester, GBR

The Eurographs Association    amp; John Wiley    amp; Sons, Ltd.

journaltitleComput. Graph. Forum
titleNonlinearly Weighted First-Order Regression for Denoising Monte Carlo Renderings
I.4.3 [Computer Graphics]: Enhancement-Filtering,I.3.7 [Computer Graphics]: Three-Dimensional Graphics and Realism-Raytracing,Categories and Subject Descriptors according to ACM CCS

Max-Planck-Institut für Informatik, Saarbrücken, Germany

titleA custom designed Density Estimation Method for Light Transport

New York, NY, USA

Association for Computing Machinery

journaltitleACM Trans. Graph.
titleKernel-Predicting Convolutional Networks for Denoising Monte Carlo Renderings
Monte Carlo denoising,Monte Carlo rendering,global illumination

abstractEfficiently rendering direct lighting from millions of dynamic light sources using Monte Carlo integration remains a challenging problem, even for off-line rendering systems. We introduce a new algorithm—ReSTIR—that renders such lighting interactively, at high quality, and without needing to maintain complex data structures. We repeatedly resample a set of candidate light samples and apply further spatial and temporal resampling to leverage information from relevant nearby samples. We derive an unbiased Monte Carlo estimator for this approach, and show that it achieves equal-error 6×-60× faster than state-of-the-art methods. A biased estimator reduces noise further and is 35×-65× faster, at the cost of some energy loss. We implemented our approach on the GPU, rendering complex scenes containing up to 3.4 million dynamic, emissive triangles in under 50 ms per frame while tracing at most 8 rays per pixel.
journaltitleACM Transactions on Graphics (Proceedings of SIGGRAPH)
titleSpatiotemporal reservoir resampling for real-time ray tracing with dynamic direct lighting
reservoir sampling,streaming algorithm,resampled importance sampling,RIS,rendering

titleThe Falcor Rendering Framework

abstractWe present a general purpose unequal probability without replacement sampling plan with fixed sample size. In contrast to existing such plans, our scheme keeps the sample size fixed and lets the population units enter the sample one at a time through a carefully designed random mechanism. Consequently, all high-order inclusion probabilities can be easily computed.
titleA general purpose unequal probability sampling plan

New York, NY, USA

Association for Computing Machinery

journaltitleACM Trans. Graph.
titleInteractive Reconstruction of Monte Carlo Image Sequences Using a Recurrent Denoising Autoencoder
Monte Carlo denoising,interactive global illumination,image reconstruction,machine learning

The Eurographics Association

booktitleEurographics Symposium on Rendering - DL-only Track
titleTemporal Sample Reuse for Next Event Estimation and Path Guiding for Real-Time Path Tracing

Springer International Publishing

booktitleMonte Carlo and Quasi-Monte Carlo Methods
titleLearning Light Transport the Reinforced Way

journaltitleInformation Processing Letters
titleWeighted random sampling with a reservoir

New York, NY, USA

journaltitleACM Trans. Graph.
titleLight transport simulation with vertex connection and merging
bidirectional path tracing,density estimation,global illumination,importance sampling,light transport,photon mapping

booktitleGame Developers Conference
titleDeferred radiance transfer volumes, global illumination in far cry 3

New York, NY, USA

Association for Computing Machinery

abstractWe present a rendering method designed to provide accurate, general simulation of global illumination for realistic image synthesis. Separating surface interaction into diffuse plus specular, we compute the specular component on the fly, as in ray tracing, and store the diffuse component (the radiosity) for later-reuse, similar to a radiosity algorithm. Radiosities are stored in adaptive radiosity textures (rexes)1 that record the pattern of light and shadow on every diffuse surface in the scene. They adaptively subdivide themselves to the appropriate level of detail for the picture being made, resolving sharp shadow edges automatically.We use a three-pass, bidirectional ray tracing algorithm that traces rays from both the lights and the eye. The "size pass" records visibility information on diffuse surfaces; the "light pass" progressively traces rays from lights and bright surfaces to deposit photons on diffuse surfaces to construct the radiosity textures; and the "eye pass" traces rays from the eye, collecting light from diffuse surfaces to make a picture.
journaltitleSIGGRAPH Comput. Graph.
titleAdaptive Radiosity Textures for Bidirectional Ray Tracing

New York, NY, USA

journaltitleACM Trans. Graph.
titleMultiplexed Metropolis Light Transport
Markov chain Monte Carlo,global illumination,multiple importance sampling

journaltitleComputational Visual Media
titleA survey on deep learning-based Monte Carlo denoising

seriesI3D ’21 Posters
titleRendering of Many Lights with Grid-Based Reservoirs

Springer Vienna

titleImportance Driven Path Tracing using the Photon Map

Porto, Portugal

booktitleProceedings of the Eurographics Workshop on Rendering Techniques '96
titleGlobal Illumination Using Photon Maps
ray tracing,Monte Carlo,photon maps,global illumination

booktitleComputer Graphics (Proc. SIGGRAPH 1986)
titleThe Rendering Equation

journaltitleMathematics and Computers in Simulation
titleIntegral Equations and Machine Learning

booktitleSIGGRAPH '97: Proceedings of the 24th Annual Conference on Computer Graphics and Interactive Techniques
titleInstant Radiosity

journaltitleIEEE Transactions on Visualization and Computer Graphics
titleRadiance caching for efficient global illumination computation

abstractApproximating illumination by point light sources, as done in many professional applications, allows for efficient algorithms, but suffers from the problem of the weak singularity: Besides avoiding numerical exceptions caused by the division by the squared distance between the point light source and the point to be illuminated, the estimator should be unbiased and of finite variance. We first illustrate that the common practice of clipping weak singularities to a reasonable value yields clearly visible bias. Then we present a new global illumination algorithm that is unbiased and as simple as a path tracer, but elegantly avoids the problem of the weak singularity. In order to demonstrate its performance, the algorithm has been integrated in an interactive global illumination system.
booktitleMonte Carlo and Quasi-Monte Carlo Methods 2004
titleIllumination in the Presence of Weak Singularities

New York, NY, USA

Association for Computing Machinery

abstractWe introduce gradient-domain rendering for Monte Carlo image synthesis. While previous gradient-domain Metropolis Light Transport sought to distribute more samples in areas of high gradients, we show, in contrast, that estimating image gradients is also possible using standard (non-Metropolis) Monte Carlo algorithms, and furthermore, that even without changing the sample distribution, this often leads to significant error reduction. This broadens the applicability of gradient rendering considerably. To gain insight into the conditions under which gradient-domain sampling is beneficial, we present a frequency analysis that compares Monte Carlo sampling of gradients followed by Poisson reconstruction to traditional Monte Carlo sampling. Finally, we describe Gradient-Domain Path Tracing (G-PT), a relatively simple modification of the standard path tracing algorithm that can yield far superior results.
journaltitleACM Trans. Graph.
titleGradient-Domain Path Tracing
gradient-domain,global illumination,light transport,path tracing

Blackwell Publishing, Inc

journaltitleComputer Graphics Forum
titleA Simple and Robust Mutation Strategy for the Metropolis Light Transport Algorithm

Los Angeles, California

Association for Computing Machinery

abstractReal-time graphics has come a long way since the "brute-force approach" of rasterization had been classified "ridiculously expensive" in 1974. Henceforth the promise "Ray tracing is the future and ever will be" drove the development of ray tracing algorithms and hardware, and resulted in a major revolution of image synthesis. This course will take a look at how far out the future is, review the state of the art, and identify the current challenges for research. Not surprisingly, it looks like we are not done with ray tracing, yet.
booktitleACM SIGGRAPH 2019 Courses
seriesSIGGRAPH '19
titleAre We Done with Ray Tracing?

Alvor, Portugal

booktitleCompugraphics '93
titleBi-Directional Path Tracing

booktitleRendering Techniques '95 (Proc. of the 6th Eurographics Workshop on Rendering)
titleA 5D tree to reduce the variance of Monte Carlo ray tracing

howpublishedSIGGRAPH 2010 Course: Advances in Real-Time Rendering in 3D Graphics and Games
titleA Real-Time Radiosity Architecture for Video Games

journaltitleComputer Graphics Forum
titlePractical Path Guiding for Efficient Light-Transport Simulation
global illumination,ray tracing

journaltitleJournal of Computer Graphics Techniques (JCGT)
titleDynamic Diffuse Global Illumination with Ray-Traced Irradiance Fields

booktitleACM SIGGRAPH / Eurographics High Performance Graphics
noteHPG 2017
titleAn Efficient Denoising Algorithm for Global Illumination

New York, NY, USA

journaltitleACM Trans. Graph.
titleNeural Importance Sampling
Monte Carlo,deep learning,importance sampling,normalizing flows,path guiding,rendering

journaltitleJournal of Computer Graphics Techniques (JCGT)
titleScaling Probe-Based Real-Time Dynamic Global Illumination for Production

New York, NY, USA

journaltitleACM Trans. Graph.
titleNeural Control Variates
variance reduction,path tracing,control variates,normalizing flows,neural networks,deep learning

New York, NY, USA

journaltitleACM Trans. Graph.
titleReal-time Neural Radiance Caching for Path Tracing
real-time,rendering,deep learning,neural networks,path tracing,radiance caching

booktitleGame Developers Conference
titleIrradiance Volumes for Games

The Eurographics Association
John Wiley & Sons Ltd.

journaltitleComputer Graphics Forum
titleReSTIR GI: Path Resampling for Real-Time Path Tracing

journaltitleJournal of the American Statistical Association
titleSafe and Effective Importance Sampling

Swansea, Wales, United Kingdom

Eurographics Association

abstractWe present a scalable method for rendering indirect illumination in diffuse and glossy scenes. Our method builds on pre-convolved radiance caching (RC), which enables reusing the incident radiance computed at a surface point for its neighborhood. Our contributions include efficient and robust generation of these RCs based on a pre-filtered voxel representation that stores scene-geometry and surface illumination. In addition, we describe a distribution strategy that places the RCs according to screen-space clusters to ensure all pixels have valid radiance data when evaluating indirect illumination. The results demonstrate the scalability of our method and analyze the relation between render quality, surface glossiness and computation time, which depends on the number of caches and their resolution.
booktitleProceedings of the 14th Eurographics Symposium on Parallel Graphics and Visualization
seriesPGV '14
titleClustered Pre-Convolved Radiance Caching

New York, NY, USA

Association for Computing Machinery

journaltitleACM Trans. Graph.
titlePractical Hessian-Based Error Control for Irradiance Caching
Monte Carlo ray tracing,irradiance caching,global illumination

San Antonio, Texas

Association for Computing Machinery

booktitleProceedings of the 29th Annual Conference on Computer Graphics and Interactive Techniques
seriesSIGGRAPH ’02
titlePrecomputed Radiance Transfer for Real-Time Rendering in Dynamic, Low-Frequency Lighting Environments
shadow algorithms,illumination,rendering,Monte Carlo techniques,graphics hardware

Los Angeles, California

Association for Computing Machinery

booktitleProceedings of High Performance Graphics
seriesHPG '17
titleSpatiotemporal Variance-Guided Filtering: Real-Time Reconstruction for Path-Traced Global Illumination
global illumination,real-time rendering,reconstruction

New York, NY, USA

Association for Computing Machinery

journaltitleACM Trans. Graph.
titleReal-Time Global Illumination by Precomputed Local Reconstruction from Sparse Radiance Probes
global illumination,real-time rendering

Chichester, GBR

The Eurographs Association & John Wiley & Sons, Ltd.

abstractThe incident indirect light over a range of image pixels is often coherent. Two common approaches to exploit this inter-pixel coherence to improve rendering performance are Irradiance Caching and Radiance Caching. Both compute incident indirect light only for a small subset of pixels (the cache), and later interpolate between pixels. Irradiance Caching uses scalar values that can be interpolated efficiently, but cannot account for shading variations caused by normal and reflectance variation between cache items. Radiance Caching maintains directional information, e.g., to allow highlights between cache items, but at the cost of storing and evaluating a Spherical Harmonics (SH) function per pixel. The arithmetic and bandwidth cost for this evaluation is linear in the number of coefficients and can be substantial. In this paper, we propose a method to replace it by an efficient per-cache item pre-filtering based on MIP maps — such as previously done for environment maps — leading to a single constant-time lookup per pixel. Additionally, per-cache item geometry statistics stored in distance-MIP maps are used to improve the quality of each pixel's lookup. Our approximate interactive global illumination approach is an order of magnitude faster than Radiance Caching with Phong BRDFs and can be combined with Monte Carlo-raytracing, Point-based Global Illumination or Instant Radiosity. © 2012 Wiley Periodicals, Inc.
journaltitleComput. Graph. Forum
titlePre-Convolved Radiance Caching
shading,and texture,I.3.7 [Computer Graphics]: Three-Dimensional Graphics and Realism—Color,shadowing

abstractWe describe the design and evolution of UberBake, a global illumination system developed by Activision, which supports limited lighting changes in response to certain player interactions. Instead of relying on a fully dynamic solution, we use a traditional static light baking pipeline and extend it with a small set of features that allow us to dynamically update the precomputed lighting at run-time while introducing little to no overhead. This means that our system works on the complete set of target hardware, ranging from high-end PCs to previous generation gaming consoles, allowing the use of lighting changes for gameplay purposes. In particular, we show how to efficiently precompute lighting changes due to individual lights being enabled and disabled and doors opening and closing. Finally, we provide a detailed performance evaluation of our system using a set of production levels and discuss how to extend its dynamic capabilities in the future.
journaltitleACM Transactions on Graphics (Proceedings of SIGGRAPH)
pubstateawaiting publication
titleThe design and evolution of the UberBake light baking system
realtime rendering

Konstanz, Germany

Eurographics Association

booktitleProceedings of the Sixteenth Eurographics Conference on Rendering Techniques
seriesEGSR '05
titleImportance Resampling for Global Illumination

Stanford University

titleRobust Monte Carlo Methods for Light Transport Simulation

booktitleEG Rendering Workshop
titleBidirectional estimators for light transport
Monte Carlo,distribution ray tracing,global illumination,lighting simulation,rendering,variance reduction

New York, NY, USA

Association for Computing Machinery

booktitleProceedings of the 22nd Annual Conference on Computer Graphics and Interactive Techniques
seriesSIGGRAPH '95
titleOptimally Combining Sampling Techniques for Monte Carlo Rendering
Monte Carlo,global illumination,distribution ray tracing,variance reduction,rendering,lighting simulation

New York, NY, USA

ACM Press/Addison-Wesley Publishing Co.

booktitleProc. SIGGRAPH 1997
seriesAnnual Conference Series
titleMetropolis Light Transport

Los Angeles, California

booktitleACM SIGGRAPH 2019 Courses
seriesSIGGRAPH '19
titlePath Guiding in Production

New York, NY, USA

Association for Computing Machinery

abstractWe introduce fast algorithms for selecting a random sample of n records without replacement from a pool of N records, where the value of N is unknown beforehand. The main result of the paper is the design and analysis of Algorithm Z; it does the sampling in one pass using constant space and in O(n(1 + log(N/n))) expected time, which is optimum, up to a constant factor. Several optimizations are studied that collectively improve the speed of the naive version of the algorithm by an order of magnitude. We give an efficient Pascal-like implementation that incorporates these modifications and that is suitable for general use. Theoretical and empirical results indicate that Algorithm Z outperforms current methods by a significant margin.
journaltitleACM Trans. Math. Softw.
titleRandom Sampling with a Reservoir

journaltitleACM Trans. Graph.
titleOn-line Learning of Parametric Mixture Models for Light Transport Simulation
light transport simulation,importance sampling,parametric density estimation,on-line expectation maximization

journaltitleJournal of Computer Graphics Techniques (JCGT)
titleReal-time Radiance Caching using Chrominance Compression

New York, NY, USA

Association for Computing Machinery

journaltitleACM Trans. Graph.
titleDenoising with Kernel Prediction and Asymmetric Loss Functions
multi-scale,asymmetric loss,kernel prediction,neural networks,monte carlo denoising,temporal

New York, NY, USA

Association for Computing Machinery

abstractAn efficient ray tracing method is presented for calculating interreflections between surfaces with both diffuse and specular components. A Monte Carlo technique computes the indirect contributions to illuminance at locations chosen by the rendering process. The indirect illuminance values are averaged over surfaces and used in place of a constant "ambient" term. Illuminance calculations are made only for those areas participating in the selected view, and the results are stored so that subsequent views can reuse common values. The density of the calculation is adjusted to maintain a constant accuracy, permitting less populated portions of the scene to be computed quickly. Successive reflections use proportionally fewer samples, which speeds the process and provides a natural limit to recursion. The technique can also model diffuse transmission and illumination from large area sources, such as the sky.
journaltitleSIGGRAPH Comput. Graph.
titleA Ray Tracing Solution for Diffuse Interreflection
Monte Carlo technique,specular,ray tracing,luminance,rendering,illuminance,radiosity,caching,interreflection,diffuse

New York, NY, USA

Association for Computing Machinery

journaltitleACM Trans. Graph.
titleAdversarial Monte Carlo Denoising with Conditioned Auxiliary Feature Modulation
Monte Carlo denoising,path tracing,adversarial learning,feature modulation

abstractAbstract Importance sampling is one of the most widely used variance reduction strategies in Monte Carlo rendering. We propose a novel importance sampling technique that uses a neural network to learn how to sample from a desired density represented by a set of samples. Our approach considers an existing Monte Carlo rendering algorithm as a black box. During a scene-dependent training phase, we learn to generate samples with a desired density in the primary sample space of the renderer using maximum likelihood estimation. We leverage a recent neural network architecture that was designed to represent real-valued non-volume preserving (“Real NVP”) transformations in high dimensional spaces. We use Real NVP to non-linearly warp primary sample space and obtain desired densities. In addition, Real NVP efficiently computes the determinant of the Jacobian of the warp, which is required to implement the change of integration variables implied by the warp. A main advantage of our approach is that it is agnostic of underlying light transport effects, and can be combined with an existing rendering technique by treating it as a black box. We show that our approach leads to effective variance reduction in several practical scenarios.
journaltitleComputer Graphics Forum
titleLearning to Importance Sample in Primary Sample Space
CCS Concepts,• Computing methodologies → Ray tracing,Neural networks,Importance sampling,Global illumination

Missing 'biblatex' package
The bibliography requires the 'biblatex' package.

Coriolis Group

booktitleGraphics Programming Black Book
titleQuake’s lighting model: Surface caching

booktitleIn ACM SIGGRAPH ’86 Course Notes - Developments in Ray Tracing
titleBackward Ray Tracing

Association for Computing Machinery

journaltitleACM Trans. Graph.
titleSpecial Issue on Production Rendering

journaltitleComputer Graphics Forum
titleBayesian Collaborative Denoising for Monte Carlo Rendering
CCS Concepts,•Computing methodologies → Ray tracing,Image processing

booktitleMonte Carlo and Quasi-Monte Carlo Methods 2020
titleMassively Parallel Path Space Filtering
pagesto appear

Chichester, GBR

The Eurographs Association    amp; John Wiley    amp; Sons, Ltd.

journaltitleComput. Graph. Forum
titleNonlinearly Weighted First-Order Regression for Denoising Monte Carlo Renderings
I.4.3 [Computer Graphics]: Enhancement-Filtering,I.3.7 [Computer Graphics]: Three-Dimensional Graphics and Realism-Raytracing,Categories and Subject Descriptors according to ACM CCS

Max-Planck-Institut für Informatik, Saarbrücken, Germany

titleA custom designed Density Estimation Method for Light Transport

New York, NY, USA

Association for Computing Machinery

journaltitleACM Trans. Graph.
titleKernel-Predicting Convolutional Networks for Denoising Monte Carlo Renderings
Monte Carlo denoising,Monte Carlo rendering,global illumination

abstractEfficiently rendering direct lighting from millions of dynamic light sources using Monte Carlo integration remains a challenging problem, even for off-line rendering systems. We introduce a new algorithm—ReSTIR—that renders such lighting interactively, at high quality, and without needing to maintain complex data structures. We repeatedly resample a set of candidate light samples and apply further spatial and temporal resampling to leverage information from relevant nearby samples. We derive an unbiased Monte Carlo estimator for this approach, and show that it achieves equal-error 6×-60× faster than state-of-the-art methods. A biased estimator reduces noise further and is 35×-65× faster, at the cost of some energy loss. We implemented our approach on the GPU, rendering complex scenes containing up to 3.4 million dynamic, emissive triangles in under 50 ms per frame while tracing at most 8 rays per pixel.
journaltitleACM Transactions on Graphics (Proceedings of SIGGRAPH)
titleSpatiotemporal reservoir resampling for real-time ray tracing with dynamic direct lighting
reservoir sampling,streaming algorithm,resampled importance sampling,RIS,rendering

titleThe Falcor Rendering Framework

abstractWe present a general purpose unequal probability without replacement sampling plan with fixed sample size. In contrast to existing such plans, our scheme keeps the sample size fixed and lets the population units enter the sample one at a time through a carefully designed random mechanism. Consequently, all high-order inclusion probabilities can be easily computed.
titleA general purpose unequal probability sampling plan

New York, NY, USA

Association for Computing Machinery

journaltitleACM Trans. Graph.
titleInteractive Reconstruction of Monte Carlo Image Sequences Using a Recurrent Denoising Autoencoder
Monte Carlo denoising,interactive global illumination,image reconstruction,machine learning

The Eurographics Association

booktitleEurographics Symposium on Rendering - DL-only Track
titleTemporal Sample Reuse for Next Event Estimation and Path Guiding for Real-Time Path Tracing

Springer International Publishing

booktitleMonte Carlo and Quasi-Monte Carlo Methods
titleLearning Light Transport the Reinforced Way

journaltitleInformation Processing Letters
titleWeighted random sampling with a reservoir

New York, NY, USA

journaltitleACM Trans. Graph.
titleLight transport simulation with vertex connection and merging
bidirectional path tracing,density estimation,global illumination,importance sampling,light transport,photon mapping

booktitleGame Developers Conference
titleDeferred radiance transfer volumes, global illumination in far cry 3

New York, NY, USA

Association for Computing Machinery

abstractWe present a rendering method designed to provide accurate, general simulation of global illumination for realistic image synthesis. Separating surface interaction into diffuse plus specular, we compute the specular component on the fly, as in ray tracing, and store the diffuse component (the radiosity) for later-reuse, similar to a radiosity algorithm. Radiosities are stored in adaptive radiosity textures (rexes)1 that record the pattern of light and shadow on every diffuse surface in the scene. They adaptively subdivide themselves to the appropriate level of detail for the picture being made, resolving sharp shadow edges automatically.We use a three-pass, bidirectional ray tracing algorithm that traces rays from both the lights and the eye. The "size pass" records visibility information on diffuse surfaces; the "light pass" progressively traces rays from lights and bright surfaces to deposit photons on diffuse surfaces to construct the radiosity textures; and the "eye pass" traces rays from the eye, collecting light from diffuse surfaces to make a picture.
journaltitleSIGGRAPH Comput. Graph.
titleAdaptive Radiosity Textures for Bidirectional Ray Tracing

New York, NY, USA

journaltitleACM Trans. Graph.
titleMultiplexed Metropolis Light Transport
Markov chain Monte Carlo,global illumination,multiple importance sampling

journaltitleComputational Visual Media
titleA survey on deep learning-based Monte Carlo denoising

seriesI3D ’21 Posters
titleRendering of Many Lights with Grid-Based Reservoirs

Springer Vienna

titleImportance Driven Path Tracing using the Photon Map

Porto, Portugal

booktitleProceedings of the Eurographics Workshop on Rendering Techniques '96
titleGlobal Illumination Using Photon Maps
ray tracing,Monte Carlo,photon maps,global illumination

booktitleComputer Graphics (Proc. SIGGRAPH 1986)
titleThe Rendering Equation

journaltitleMathematics and Computers in Simulation
titleIntegral Equations and Machine Learning

booktitleSIGGRAPH '97: Proceedings of the 24th Annual Conference on Computer Graphics and Interactive Techniques
titleInstant Radiosity

journaltitleIEEE Transactions on Visualization and Computer Graphics
titleRadiance caching for efficient global illumination computation

abstractApproximating illumination by point light sources, as done in many professional applications, allows for efficient algorithms, but suffers from the problem of the weak singularity: Besides avoiding numerical exceptions caused by the division by the squared distance between the point light source and the point to be illuminated, the estimator should be unbiased and of finite variance. We first illustrate that the common practice of clipping weak singularities to a reasonable value yields clearly visible bias. Then we present a new global illumination algorithm that is unbiased and as simple as a path tracer, but elegantly avoids the problem of the weak singularity. In order to demonstrate its performance, the algorithm has been integrated in an interactive global illumination system.
booktitleMonte Carlo and Quasi-Monte Carlo Methods 2004
titleIllumination in the Presence of Weak Singularities

New York, NY, USA

Association for Computing Machinery

abstractWe introduce gradient-domain rendering for Monte Carlo image synthesis. While previous gradient-domain Metropolis Light Transport sought to distribute more samples in areas of high gradients, we show, in contrast, that estimating image gradients is also possible using standard (non-Metropolis) Monte Carlo algorithms, and furthermore, that even without changing the sample distribution, this often leads to significant error reduction. This broadens the applicability of gradient rendering considerably. To gain insight into the conditions under which gradient-domain sampling is beneficial, we present a frequency analysis that compares Monte Carlo sampling of gradients followed by Poisson reconstruction to traditional Monte Carlo sampling. Finally, we describe Gradient-Domain Path Tracing (G-PT), a relatively simple modification of the standard path tracing algorithm that can yield far superior results.
journaltitleACM Trans. Graph.
titleGradient-Domain Path Tracing
gradient-domain,global illumination,light transport,path tracing

Blackwell Publishing, Inc

journaltitleComputer Graphics Forum
titleA Simple and Robust Mutation Strategy for the Metropolis Light Transport Algorithm

Los Angeles, California

Association for Computing Machinery

abstractReal-time graphics has come a long way since the "brute-force approach" of rasterization had been classified "ridiculously expensive" in 1974. Henceforth the promise "Ray tracing is the future and ever will be" drove the development of ray tracing algorithms and hardware, and resulted in a major revolution of image synthesis. This course will take a look at how far out the future is, review the state of the art, and identify the current challenges for research. Not surprisingly, it looks like we are not done with ray tracing, yet.
booktitleACM SIGGRAPH 2019 Courses
seriesSIGGRAPH '19
titleAre We Done with Ray Tracing?

Alvor, Portugal

booktitleCompugraphics '93
titleBi-Directional Path Tracing

booktitleRendering Techniques '95 (Proc. of the 6th Eurographics Workshop on Rendering)
titleA 5D tree to reduce the variance of Monte Carlo ray tracing

howpublishedSIGGRAPH 2010 Course: Advances in Real-Time Rendering in 3D Graphics and Games
titleA Real-Time Radiosity Architecture for Video Games

journaltitleComputer Graphics Forum
titlePractical Path Guiding for Efficient Light-Transport Simulation
global illumination,ray tracing

journaltitleJournal of Computer Graphics Techniques (JCGT)
titleDynamic Diffuse Global Illumination with Ray-Traced Irradiance Fields

booktitleACM SIGGRAPH / Eurographics High Performance Graphics
noteHPG 2017
titleAn Efficient Denoising Algorithm for Global Illumination

New York, NY, USA

journaltitleACM Trans. Graph.
titleNeural Importance Sampling
Monte Carlo,deep learning,importance sampling,normalizing flows,path guiding,rendering

journaltitleJournal of Computer Graphics Techniques (JCGT)
titleScaling Probe-Based Real-Time Dynamic Global Illumination for Production

New York, NY, USA

journaltitleACM Trans. Graph.
titleNeural Control Variates
variance reduction,path tracing,control variates,normalizing flows,neural networks,deep learning

New York, NY, USA

journaltitleACM Trans. Graph.
titleReal-time Neural Radiance Caching for Path Tracing
real-time,rendering,deep learning,neural networks,path tracing,radiance caching

booktitleGame Developers Conference
titleIrradiance Volumes for Games

The Eurographics Association
John Wiley & Sons Ltd.

journaltitleComputer Graphics Forum
titleReSTIR GI: Path Resampling for Real-Time Path Tracing

journaltitleJournal of the American Statistical Association
titleSafe and Effective Importance Sampling

Swansea, Wales, United Kingdom

Eurographics Association

abstractWe present a scalable method for rendering indirect illumination in diffuse and glossy scenes. Our method builds on pre-convolved radiance caching (RC), which enables reusing the incident radiance computed at a surface point for its neighborhood. Our contributions include efficient and robust generation of these RCs based on a pre-filtered voxel representation that stores scene-geometry and surface illumination. In addition, we describe a distribution strategy that places the RCs according to screen-space clusters to ensure all pixels have valid radiance data when evaluating indirect illumination. The results demonstrate the scalability of our method and analyze the relation between render quality, surface glossiness and computation time, which depends on the number of caches and their resolution.
booktitleProceedings of the 14th Eurographics Symposium on Parallel Graphics and Visualization
seriesPGV '14
titleClustered Pre-Convolved Radiance Caching

New York, NY, USA

Association for Computing Machinery

journaltitleACM Trans. Graph.
titlePractical Hessian-Based Error Control for Irradiance Caching
Monte Carlo ray tracing,irradiance caching,global illumination

San Antonio, Texas

Association for Computing Machinery

booktitleProceedings of the 29th Annual Conference on Computer Graphics and Interactive Techniques
seriesSIGGRAPH ’02
titlePrecomputed Radiance Transfer for Real-Time Rendering in Dynamic, Low-Frequency Lighting Environments
shadow algorithms,illumination,rendering,Monte Carlo techniques,graphics hardware

Los Angeles, California

Association for Computing Machinery

booktitleProceedings of High Performance Graphics
seriesHPG '17
titleSpatiotemporal Variance-Guided Filtering: Real-Time Reconstruction for Path-Traced Global Illumination
global illumination,real-time rendering,reconstruction

New York, NY, USA

Association for Computing Machinery

journaltitleACM Trans. Graph.
titleReal-Time Global Illumination by Precomputed Local Reconstruction from Sparse Radiance Probes
global illumination,real-time rendering

Chichester, GBR

The Eurographs Association & John Wiley & Sons, Ltd.

abstractThe incident indirect light over a range of image pixels is often coherent. Two common approaches to exploit this inter-pixel coherence to improve rendering performance are Irradiance Caching and Radiance Caching. Both compute incident indirect light only for a small subset of pixels (the cache), and later interpolate between pixels. Irradiance Caching uses scalar values that can be interpolated efficiently, but cannot account for shading variations caused by normal and reflectance variation between cache items. Radiance Caching maintains directional information, e.g., to allow highlights between cache items, but at the cost of storing and evaluating a Spherical Harmonics (SH) function per pixel. The arithmetic and bandwidth cost for this evaluation is linear in the number of coefficients and can be substantial. In this paper, we propose a method to replace it by an efficient per-cache item pre-filtering based on MIP maps — such as previously done for environment maps — leading to a single constant-time lookup per pixel. Additionally, per-cache item geometry statistics stored in distance-MIP maps are used to improve the quality of each pixel's lookup. Our approximate interactive global illumination approach is an order of magnitude faster than Radiance Caching with Phong BRDFs and can be combined with Monte Carlo-raytracing, Point-based Global Illumination or Instant Radiosity. © 2012 Wiley Periodicals, Inc.
journaltitleComput. Graph. Forum
titlePre-Convolved Radiance Caching
shading,and texture,I.3.7 [Computer Graphics]: Three-Dimensional Graphics and Realism—Color,shadowing

abstractWe describe the design and evolution of UberBake, a global illumination system developed by Activision, which supports limited lighting changes in response to certain player interactions. Instead of relying on a fully dynamic solution, we use a traditional static light baking pipeline and extend it with a small set of features that allow us to dynamically update the precomputed lighting at run-time while introducing little to no overhead. This means that our system works on the complete set of target hardware, ranging from high-end PCs to previous generation gaming consoles, allowing the use of lighting changes for gameplay purposes. In particular, we show how to efficiently precompute lighting changes due to individual lights being enabled and disabled and doors opening and closing. Finally, we provide a detailed performance evaluation of our system using a set of production levels and discuss how to extend its dynamic capabilities in the future.
journaltitleACM Transactions on Graphics (Proceedings of SIGGRAPH)
pubstateawaiting publication
titleThe design and evolution of the UberBake light baking system
realtime rendering

Konstanz, Germany

Eurographics Association

booktitleProceedings of the Sixteenth Eurographics Conference on Rendering Techniques
seriesEGSR '05
titleImportance Resampling for Global Illumination

Stanford University

titleRobust Monte Carlo Methods for Light Transport Simulation

booktitleEG Rendering Workshop
titleBidirectional estimators for light transport
Monte Carlo,distribution ray tracing,global illumination,lighting simulation,rendering,variance reduction

New York, NY, USA

Association for Computing Machinery

booktitleProceedings of the 22nd Annual Conference on Computer Graphics and Interactive Techniques
seriesSIGGRAPH '95
titleOptimally Combining Sampling Techniques for Monte Carlo Rendering
Monte Carlo,global illumination,distribution ray tracing,variance reduction,rendering,lighting simulation

New York, NY, USA

ACM Press/Addison-Wesley Publishing Co.

booktitleProc. SIGGRAPH 1997
seriesAnnual Conference Series
titleMetropolis Light Transport

Los Angeles, California

booktitleACM SIGGRAPH 2019 Courses
seriesSIGGRAPH '19
titlePath Guiding in Production

New York, NY, USA

Association for Computing Machinery

abstractWe introduce fast algorithms for selecting a random sample of n records without replacement from a pool of N records, where the value of N is unknown beforehand. The main result of the paper is the design and analysis of Algorithm Z; it does the sampling in one pass using constant space and in O(n(1 + log(N/n))) expected time, which is optimum, up to a constant factor. Several optimizations are studied that collectively improve the speed of the naive version of the algorithm by an order of magnitude. We give an efficient Pascal-like implementation that incorporates these modifications and that is suitable for general use. Theoretical and empirical results indicate that Algorithm Z outperforms current methods by a significant margin.
journaltitleACM Trans. Math. Softw.
titleRandom Sampling with a Reservoir

journaltitleACM Trans. Graph.
titleOn-line Learning of Parametric Mixture Models for Light Transport Simulation
light transport simulation,importance sampling,parametric density estimation,on-line expectation maximization

journaltitleJournal of Computer Graphics Techniques (JCGT)
titleReal-time Radiance Caching using Chrominance Compression

New York, NY, USA

Association for Computing Machinery

journaltitleACM Trans. Graph.
titleDenoising with Kernel Prediction and Asymmetric Loss Functions
multi-scale,asymmetric loss,kernel prediction,neural networks,monte carlo denoising,temporal

New York, NY, USA

Association for Computing Machinery

abstractAn efficient ray tracing method is presented for calculating interreflections between surfaces with both diffuse and specular components. A Monte Carlo technique computes the indirect contributions to illuminance at locations chosen by the rendering process. The indirect illuminance values are averaged over surfaces and used in place of a constant "ambient" term. Illuminance calculations are made only for those areas participating in the selected view, and the results are stored so that subsequent views can reuse common values. The density of the calculation is adjusted to maintain a constant accuracy, permitting less populated portions of the scene to be computed quickly. Successive reflections use proportionally fewer samples, which speeds the process and provides a natural limit to recursion. The technique can also model diffuse transmission and illumination from large area sources, such as the sky.
journaltitleSIGGRAPH Comput. Graph.
titleA Ray Tracing Solution for Diffuse Interreflection
Monte Carlo technique,specular,ray tracing,luminance,rendering,illuminance,radiosity,caching,interreflection,diffuse

New York, NY, USA

Association for Computing Machinery

journaltitleACM Trans. Graph.
titleAdversarial Monte Carlo Denoising with Conditioned Auxiliary Feature Modulation
Monte Carlo denoising,path tracing,adversarial learning,feature modulation

abstractAbstract Importance sampling is one of the most widely used variance reduction strategies in Monte Carlo rendering. We propose a novel importance sampling technique that uses a neural network to learn how to sample from a desired density represented by a set of samples. Our approach considers an existing Monte Carlo rendering algorithm as a black box. During a scene-dependent training phase, we learn to generate samples with a desired density in the primary sample space of the renderer using maximum likelihood estimation. We leverage a recent neural network architecture that was designed to represent real-valued non-volume preserving (“Real NVP”) transformations in high dimensional spaces. We use Real NVP to non-linearly warp primary sample space and obtain desired densities. In addition, Real NVP efficiently computes the determinant of the Jacobian of the warp, which is required to implement the change of integration variables implied by the warp. A main advantage of our approach is that it is agnostic of underlying light transport effects, and can be combined with an existing rendering technique by treating it as a black box. We show that our approach leads to effective variance reduction in several practical scenarios.
journaltitleComputer Graphics Forum
titleLearning to Importance Sample in Primary Sample Space
CCS Concepts,• Computing methodologies → Ray tracing,Neural networks,Importance sampling,Global illumination