# An Idiosyncrasy of Time-discretization in
Reinforcement Learning

Kris De Asis
<EMAIL_ADDRESS>
Department of Computing Science
University of Alberta &Richard S. Sutton
<EMAIL_ADDRESS>
Department of Computing Science
University of Alberta

###### Abstract

Many reinforcement learning algorithms are built on an assumption that an
agent interacts with an environment over fixed-duration, discrete time steps.
However, physical systems are continuous in time, requiring a choice of time-
discretization granularity when digitally controlling them. Furthermore, such
systems do not wait for decisions to be made before advancing the environment
state, necessitating the study of how the choice of discretization may affect
a reinforcement learning algorithm. In this work, we consider the relationship
between the definitions of the continuous-time and discrete-time returns.
Specifically, we acknowledge an idiosyncrasy with naively applying a discrete-
time algorithm to a discretized continuous-time environment, and note how a
simple modification can better align the return definitions. This observation
is of practical consideration when dealing with environments where time-
discretization granularity is a choice, or situations where such granularity
is inherently stochastic.

##  1 Introduction

Reinforcement learning provides a framework for solving sequential decision
making problems based on evaluative feedback (Sutton & Barto, 2018). It
remains a promising approach for robot learning as it can allow for real-time
adaptation of behavior. Many reinforcement learning algorithms assume that the
agent-environment interaction occurs at synchronous, discrete time steps,
where the environment waits for an action before advancing. In contrast, real-
world physical systems are continuous in time, and do not wait for an agent’s
input. As such, time-discretization becomes a necessary and important
consideration (Mahmood et al., 2018a).

Prior work suggests that current reinforcement learning algorithms are
sensitive to the choice of discretization. Tallec et al. (2019) emphasizes
that action-values converge to state-values as the discretization interval
approaches zero, creating degenerate cases for algorithms like Q-learning.
Similarly, Munos (2006) shows that the variance of policy gradients can
explode under the same limit. Mahmood et al. (2018a) details the trade-off
between having fine-grained control and being able to discern the changes
between subsequent states. Finally, Farrahi & Mahmood (2023) provides
guidelines for time-discretization-aware parameter selection by acknowledging
how changes in discrete-time parameters influence the underlying continuous-
time objective.

In this work, we explicitly view the discrete-time objective as a discrete
approximation of the continuous-time objective. By considering when rewards
occur, particularly in existing continuous-control environment setups, we
identify an idiosyncratic dependence on the choice of discretization beyond
those listed in Tallec et al. (2019) and Farrahi & Mahmood (2023).
Specifically, the discrete-time return can be viewed as a mixture of two
Riemann sums. We characterize and demonstrate that this is a relatively poor
integral approximation in comparison with a conventional Riemann sum, and
provide a simple modification to the definition of the return to better align
the objectives.

The contributions of this work are as follows:

  * •

Acknowledgement and characterization of a discrepancy when naively applying a
discrete-time reinforcement learning algorithm to a discretized continuous-
time environment.

  * •

A simple modification to the definition of the return to avoid a nuanced
dependence on time-discretization, based on an integral approximation
perspective.

  * •

Characterization of when the modification is most impactful, supported by
empirical results demonstrating improved alignment with an underlying
continuous time objective.

##  2 Definitions of the Return

Reinforcement learning (Sutton & Barto, 2018) is a framework for sequential
decision making from evaluative feedback. A learning system, denoted an agent,
observes its current situation and selects an action, after which it observes
a new situation while receiving a reward. The agent’s objective is to learn to
act so as to maximize its expected return\- a discounted sum of future
rewards. In discrete-time (Sutton & Barto, 2018), the return from a time step
$t$ onward is defined to be:

$\ddot{G}_{t}=\sum_{k=t}^{T-1}{\gamma^{k-t}R_{k+1}}$ (1)

where $T$ is the final time step in an episodic task, or $\infty$ in a
continuing, infinite-horizon one. In continuous-time reinforcement learning
(Doya, 2000; Mehta & Meyn, 2009; Frémaux et al., 2013; Lee & Sutton, 2021;
Tallec et al., 2019), we instead define the integral return from time step $t$
onward:

$G_{t}=\int_{t}^{T}{\gamma^{\tau-t}R_{\tau}d\tau}$ (2)

Such a formulation is pertinent to applications involving real-time
interaction, such as robotics. Despite being continuous in time, they are
often digitally controlled, and as such time-discretization becomes a
necessary consideration (Mahmood et al., 2018a).

##  3 When Rewards Occur

There are notation inconsistencies in the literature with respect to time
indices in the discrete time return (Equation 1). Some define it to start from
$R_{t+1}$ (Sutton, 1988; Precup et al., 2000; van Seijen et al., 2009; Barreto
et al., 2017), as presented in this document, while some would start from
$R_{t}$ (Watkins, 1989; van Hasselt, 2010; Mnih et al., 2015; Wang et al.,
2016). This inconsistency is inconsequential when solely considering the
discrete-time setting, as the rewards occur at the same locations in an
agent’s stream of experience. However, it has implications when viewed as a
discrete approximation to an underlying integral return. Thus, it is worth
considering when rewards occur.

We emphasize the focus on a setting where there is an underlying continuous-
time objective, of which a digital learning agent samples at an arbitrary (and
potentially variable) frequency. Despite the discrete-time notation
inconsistencies, it is often agreed upon that from the agent’s perspective,
the reward and next state are jointly observed. This is reflected in
environment step calls in relatively standard reinforcement learning APIs
(Brockman et al., 2016), agent-environment interaction diagrams (Sutton &
Barto, 2018), or explicit acknowledgement that reward can be a function of
state, action, and next state (Puterman, 1994). In real-time settings which do
not wait for an agent’s input, actions typically take time to execute and have
an influence, and so meaningful evaluative feedback must come after time $t$.
Hardware limitations on sampling rates further impose an inherent delay in
when a system can receive feedback for an action. In many existing robotics
environments, where the considered setting is especially pertinent, rewards
are often explicitly computed based on the next time step’s state information
(Todorov et al., 2012; Brockman et al., 2016; Mahmood et al., 2018b), for
example, rewards based on distance traveled in some direction between two time
steps, or distance between an end-effector and a desired setpoint at the
subsequent time step.

Of note, semi-MDPs and options (Sutton et al., 1999; Precup, 2000) address the
problem of when rewards occur, but under the assumption that one has access to
higher-frequency interaction with the environment to integrate the discounted
sum of rewards within the discretization interval. It is akin to the agent
being aware of and able to time when each component of a temporally-extended
reward occurs. In this work, we consider when one does not have access to
these higher-frequency samples, but is aware of how much time has elapsed
between discrete decision points. Acquiring such information may not be
possible due to hardware limitations, and highlights a nuance that arises when
naively applying a discrete-time algorithm to a discretized continuous-time
environment.

##  4 Implications for Time Discretization

If we consider rewards jointly arriving with the next state, at least from the
agent’s perspective, this results in an idiosyncrasy with respect to
approximating an underlying integral return. While the discrete-time returns
may use inconsistent reward time-indices, they are consistent on when
discounting begins: the first reward is given weight $\gamma^{0}=1$, with
subsequent rewards weighted by increasing powers of $\gamma$. We can view the
integral return in Equation 2 to be of the form:

$\int_{t}^{T}{f(\tau)g(\tau)d\tau}$ (3)

where $f(\tau)$ is the discounting term, and $g(\tau)$ is the reward signal. A
right-point Riemann sum approximation to this would yield:

$\sum_{i=0}^{n-1}{f(\tau_{i})g(\tau_{i})\Delta}$ (4)

where $\Delta=\frac{T-t}{n}$ and $\tau=\\{t+\Delta,t+2\Delta,...,T\\}$. The
right-point Riemann sum beginning with $t+\Delta$ aligns with an agent jointly
receiving a reward with the observation of the next state. However, this sum
would weight the first reward by $\gamma^{\Delta}\neq\gamma^{0}$. This
highlights that if one naively applies a discrete-time reinforcement learning
algorithm to a discretized continuous-time environment, it is akin to a left-
point Riemann sum for discounting, and a right-point Riemann sum for rewards:

$\sum_{i=0}^{n-1}{f(\tau_{i})g(\tau_{i+1})\Delta}$ (5)

where $\tau\in\\{t,t+\Delta,t+2\Delta,...,T\\}$. See Figure 1 for a
visualization of this Riemann sum. This sum still converges to the correct
integral as $n\rightarrow\infty$, as Bliss’s Theorem (1914) allows each
function to be evaluated at any point within the interval. However, for the
specific case where a left-point Riemann sum is used for discounting, we
expect this to perform worse than committing to a right-point Riemann sum. Due
to the curvature of exponential decay, if one drew a rectangle with opposite
corners at any two points, there will always be more area above the curve than
below, implying an underestimate has strictly lower error than an
overestimate. This is visualized in Figure 2.

Figure 1: The resulting sum when applying a discrete-time algorithm to a
discretized continuous-time domain. Note how rectangle heights may fall out of
the function’s range within an interval. Figure 2: A visualization of the
left-point and right-point Riemann sum approximation errors for an exponential
decay. Due to curvature, a right-point Riemann sum will always have lower
error.

To rectify this discrepancy and commit to a right-point Riemann sum
approximation, one would simply multiply the discrete-time return by a factor
of $\gamma$ (assuming $\Delta=1$):

$\gamma\ddot{G}_{t}=\gamma R_{t+1}+\gamma^{2}R_{t+2}+\cdots$ (6)

For a fixed, pre-specified action cycle-time $\Delta$, there is no loss of
generality as the discrete-time return is proportional by a factor of
$\gamma^{\Delta}\Delta$. However, this is not the case when $\Delta$ may vary
over time, e.g., due to an adaptive algorithm (Karimi et al., 2023) or
inherent stochasticity. These concerns similarly apply to a variable $\gamma$
and may extend toward tuning fixed-$\Delta$ and/or $\gamma$ in practice in
terms of a nuanced and unintuitive dependence on discretization. See appendix
A for a discrete-time return visualization with variable interval sizes. To
emphasize the dependence on $\Delta$, we note the explicit right-point Riemann
sum return:

$\displaystyle\ddot{G}_{t}^{RP}$
$\displaystyle\overset{\textrm{def}}{=}\sum_{k=t}^{T-1}\gamma^{\sum_{i=t}^{k}{\Delta_{i+1}}}R_{k+1}\Delta_{k+1}$
(7)
$\displaystyle=\gamma^{\Delta_{t+1}}R_{t+1}\Delta_{t+1}+\gamma^{\Delta_{t+1}+\Delta_{t+2}}R_{t+2}\Delta_{t+2}+\cdots$

Prior work has acknowledged the modifications of (1) scaling rewards by
$\Delta$, and (2) exponentiating $\gamma$ by $\Delta$ (Tallec et al., 2019;
Farrahi & Mahmood, 2023) in improving robustness to time-discretization. The
key difference and contribution in Equation 7 being the earlier discounting.

##  5 Comparison with Standard Riemann Sums

To see how the discrete-time return (DTR) in Equation 5 fares against a right-
point Riemann sum, we evaluate them on randomly generated continuous-time
signals. Inspired by robotics, we consider periodic signals, and Gaussian
mixtures. Periodic signals are comparable to signals pertaining to robot
locomotion, while Gaussian mixtures instead resemble both sparse and distance-
based rewards, depending on the spread of each Gaussian. We fix the signal
length to 3 seconds, with no loss of generality due to being continuous in
time. Each signal generator is detailed below:

Random Periodic Signals \- This signal sums 6 sinusoids
$\sum_{i=0}^{5}{A_{i}\sin(\omega_{i}t+\phi_{i})}$ with angular frequencies
$\omega\in\\{\frac{2\pi}{4},\frac{2\pi}{2},2\pi,4\pi,8\pi,16\pi\\}$,
amplitudes $A_{i}\sim\mathcal{N}(0,1)$, and phase shifts
$\phi_{i}\sim\mathcal{U}(0,2\pi)$.

Random Gaussian Mixtures \- This signal sums 6 Gaussians
$\sum_{i=0}^{5}{\mathcal{N}(\mu_{i},\sigma_{i})}$ with means
$\mu_{i}\sim\mathcal{U}(0,3)$, and standard deviations
$\sigma_{i}\sim\mathcal{U}(0,\frac{3}{2})$.

For each method, we varied the number of intervals $n\in\\{5,10,25,50,100\\}$,
the discount factor $\gamma\in\\{0.5,0.75,0.875\\}$, and measured the absolute
error of the integral approximation- compared against a mid-point Riemann sum
with $10^{4}$ intervals. The values of $\gamma$ used may appear small and
unrepresentative of typical values. We however note that the discount is per
second, and that for a robot sampling every 30 ms, $\gamma=0.5$ is effectively
$\gamma^{\Delta}=0.5^{0.03}\approx 0.98$ per discrete time step. Averaged
across $10^{6}$ randomly generated signals of each type, the results can be
seen in Figure 3.

Figure 3: Numerical integration approximation error on discounted random
signals. Results are averaged over $10^{6}$ signals and shaded regions
represent one standard error.

As expected, the errors generally increase as $\Delta\propto\frac{1}{n}$
increases. There’s a consistent dip in error with the periodic signals, likely
due to the intervals coincidentally aligning with the pre-specified
frequencies. Across all settings, DTR had larger absolute error, consistent
with our hypothesis that DTR would perform worse than right-point on
discounted signals. The gap closes as $\gamma\rightarrow 1$, as the sums are
equivalent at this extreme.

We then considered stochastic intervals to simulate variable time-
discretization. This was implemented by sampling, sorting, and re-scaling a
set of $n+1$ uniform random points representing interval endpoints. This is
particularly pertinent as DTR is no longer proportional to right-point, and
reflects the variability in applications on real-time systems. Fixing
$\gamma=0.75$, Figure 4 shows results averaged across $10^{6}$ randomly
generated signals of each type, plotted against average $\Delta$. Errors
generally increased, with DTR maintaining larger approximation error across
every setting.

Figure 4: Numerical integration approximation error on discounted random
signals, with stochastic discretization intervals. Results are averaged over
$10^{6}$ signals and shaded regions represent one standard error.

Lastly, to see whether results hold beyond exponential discounting, we
considered the product of each pair of the signal generators. This evaluates
each sum in a more general numerical integration setting, while having
implications for variable, transition-dependent $\gamma$ in reinforcement
learning. Averaged across $10^{6}$ randomly generated signal pairs, the
results can be seen in Figure 5. Perhaps surprisingly, the gap between DTR and
the right-point Riemann sum widens dramatically. These results suggest that
beyond the structure of discounting, DTR is a generally worse integral
approximation.

Figure 5: Numerical integration approximation error on undiscounted products
of random signals. Results are averaged over $10^{6}$ signals and shaded
regions represent one standard error.

A key takeaway from these results is that shifting the discount factor in the
discrete-time return yields a better prediction target (e.g., in value-based
methods) in terms of error between the integral return. To reiterate, in the
fixed $\Delta$ case, the sums are proportional despite the gaps in
approximation error. This suggests that the improvement is inconsequential for
control. However, such improved alignment is expected to improve control
performance in the variable $\Delta$ setting, in terms of maximizing the
underlying integral return. We explore this further in the next section.

##  6 Discretized Continuous-time Control

To evaluate the right-point Riemann sum in a continuous-time control setting,
we build off of the REINFORCE (Williams, 1992) algorithm. Such a choice is due
to the algorithm’s simplicity, allowing for more confidence in attributing
differences in performance. We specifically use online REINFORCE with
eligiblity traces (Kimura et al., 1995) and dropped $\gamma^{t}$ term,
summarized by:

$\displaystyle\mathbf{z}$
$\displaystyle\leftarrow\mathbf{z}+\nabla_{\theta}\log{\pi(A_{t}|S_{t})}$
$\displaystyle\theta$ $\displaystyle\leftarrow\theta+\alpha R_{eff}\mathbf{z}$
$\displaystyle\mathbf{z}$
$\displaystyle\leftarrow\gamma^{\Delta_{t+1}}\mathbf{z}$

where $\Delta_{t+1}$ is the elapsed time between time steps $t$ and $t+1$,
$R_{eff}=R_{t+1}\Delta_{t+1}$ for the discrete-time return, and
$R_{eff}=\gamma^{\Delta_{t+1}}R_{t+1}\Delta_{t+1}$ for the right-point Riemann
sum. The above algorithm employs the recommendations of (Farrahi & Mahmood,
2023) for making algorithms more robust to time-discretization, emphasizing
that the proposed right-point modification is complimentary. Each agent’s
policy used a two-hidden-layer fully-connected network with $tanh$
activations, with its output being treated as the mean of a Gaussian with an
initial (bias unit) standard deviation of 1.

We designed a simulated Servo Reacher environment based on the setup in
Mahmood et al. (2018b), with physical parameters sourced from a Dynamixel
MX-28AT data sheet. This custom environment allows for fine-grained
computation of the integral return, and flexibility in the discretization
intervals an agent can sample at. Full environment specification can be found
in Appendix B. To simulate the inherent stochasticity of a real robot,
Gaussian noise was added to the target discretization interval,
$\Delta_{t}\sim\mathcal{N}(\Delta_{\mu},10\textrm{ ms})$, with a minimum of 1
ms. We additionally included a 1% chance to sample the interval from
$\mathcal{N}(1000\textrm{ ms},10\textrm{ ms})$ to simulate “catastrophic”
events akin to communication errors.

The environment fixed $\gamma=0.25$, which when using an interval of 40 ms,
corresponds with discrete-time $\gamma^{0.04}\approx 0.95$. We considered
target discretization intervals $\Delta_{\mu}\in\\{40,80,120\\}$ ms with a 4
second time limit, and measured the episodic integral return. Averaged over
100 25-minute runs, Figure 6 shows parameter sensitivity curves, as well as
learning curves under the best parameters.

(a) Parameter Sensitivity ($\Delta_{\mu}=40\textrm{ ms}$)

(b) Parameter Sensitivity ($\Delta_{\mu}=80\textrm{ ms}$)

(c) Parameter Sensitivity ($\Delta_{\mu}=120\textrm{ ms}$)

(d) Learning Curves ($\Delta_{\mu}=120\textrm{ ms}$)

Figure 6: Servo Reacher results for REINFORCE using the discrete-time return
(DTR) and right-point Riemann sum (RP), averaged over 100 runs. Shaded regions
represent one standard error.

An initial observation is a systematic “lag” between the two algorithms in the
sesntivity curves at low $\alpha$. This is due the return magnitudes being
roughly proportional by a factor of $\mathbb{E}[\gamma^{\Delta_{t}}]$. If one
absorbs this factor into the step-size, the right-point Riemann sum can be
viewed as using a smaller effective $\alpha$ in the policy gradient update.
Scaling the figure to use this effective $\alpha$ can be found to well-align
the curves at low $\alpha$. Nevertheless, we find that after accounting for
this shift, REINFORCE with the right-point Riemann sum performs better for
large $\alpha$, and can significantly outperform the discrete-time return with
both algorithms properly tuned. The right-point Riemann sum is seen to improve
with increasing $\Delta_{\mu}$, in line with the approximation error results
in Section 5. Acknowledging that the two returns are roughly proportional by
$\mathbb{E}[\gamma^{\Delta_{t}}]$, improvements are expected as this term
deviates from 1, i.e., decreasing $\gamma$ or increasing $\Delta_{\mu}$.

##  7 Conclusions and Future Work

In this work, we identified and characterized an idiosyncrasy of time-
discretization in reinforcement learning. Specifically, a nuance between the
definitions of the discrete-time and continuous-time returns when viewing one
as a discretization of the other. Our results suggest that when one does not
have access to evaluating the integral return via options, one can better
align the objectives by shifting the discount factor to begin discounting
sooner. This provides unification in that the discrete-time return becomes a
relatively straight-forward discretization of the integral return. We strongly
emphasize the simplicity of the modification, and how apart from the
$\gamma=0$ extreme, such a modification has no loss of generality in discrete-
time or with fixed discretization intervals due to proportionality with the
conventional discrete-time return. The returns are equivalent as
$\gamma^{\Delta}\rightarrow 1$, but as it deviates, it results in better
prediction targets in terms integral approximation error, and improves control
performance with variable time-discretization. Beyond the integral
approximation perspective, the modification has intuitive appeal in that
results from catastrophically long delays are attenuated in the return, rather
than fully crediting an action for that outcome.

This work was built on an assumption that the reward better aligns with the
subsequent time-step, which is often the case in how existing continuous-time
environments are set up. The right-point Riemann sum can be viewed as
evaluating the integral within the interval for an impulse reward at the
subsequent time-step, as would be done by the options framework in a semi-MDP.
However, we emphasize that this work still considers problems with arbitrarily
dense rewards, but due to discrete sampling, rewards appear as delayed impulse
rewards from the agent’s perspective. Should there be additional information
about when a reward occurs within an interval, the ideas still generalize in
that discounting can be shifted to reflect this information.

Regarding avenues for future work, the integral approximation perspective
suggests opportunity to explore return modifications corresponding with other
integral approximation techniques. For example, if one were to additionally
track predecessor rewards, it opens up the possibility of interpolation-based
approximations (e.g., trapezoidal rule). For the case of exponential
discounting, we could further leverage a closed-form integral for that term.

### Acknowledgements

The authors were generously supported by Amii, NSERC, and CIFAR, and would
like to thank Alan Chan and Sungsu Lim for insights and discussions
contributing to the results presented in this paper. The authors further thank
the reviewers for valuable feedback during the review process.

## References

  * Barreto et al. (2017) A. Barreto, W. Dabney, R. Munos, J. J. Hunt, T. Schaul, D. Silver, and H. van Hasselt.  Successor features for transfer in reinforcement learning.  In _Proceedings of the 30th Annual Conference on Neural Information Processing Systems_ , pp. 4055–4065, 2017.
  * Bliss (1914) Gilbert Ames Bliss.  A substitute for duhamel’s theorem.  _Annals of Mathematics_ , 16(1/4):45–49, 1914.
  * Brockman et al. (2016) G. Brockman, V. Cheung, L. Pettersson, J. Schneider, J. Schulman, J. Tang, and W. Zaremba.  OpenAI gym.  _CoRR_ , abs/1606.01540, 2016.
  * Doya (2000) K. Doya.  Reinforcement learning in continuous time and space.  _Neural Computation_ , 12:219–245, 2000.
  * Farrahi & Mahmood (2023) Homayoon Farrahi and A. Rupam Mahmood.  Reducing the cost of cycle-time tuning for real-world policy optimization, 2023.
  * Frémaux et al. (2013) N. Frémaux, H. Sprekeler, and W. Gerstner.  Reinforcement learning using a continuous time actor-critic framework with spiking neurons.  _PLOS Computational Biology_ , 9:1–21, 2013.
  * Karimi et al. (2023) A. Karimi, J. Jin, J. Luo, A. R. Mahmood, M. Jagersand, and S. Tosatto.  Dynamic decision frequency with continuous options, 2023.
  * Kimura et al. (1995) Hajime Kimura, Masayuki Yamamura, and Shigenobu Kobayashi.  Reinforcement learning by stochastic hill climbing on discounted reward.  In _Proceedings of the 12th International Conference on Machine Learning_ , 1995.
  * Lee & Sutton (2021) J. Lee and R. S. Sutton.  Policy iterations for reinforcement learning problems in continuous time and space — fundamental theory and methods.  _Automatica_ , 126:109421, 2021.  ISSN 0005-1098.
  * Mahmood et al. (2018a) A. R. Mahmood, D. Korenkevych, B. J. Komer, and J. Bergstra.  Setting up a reinforcement learning task with a real-world robot.  In _Proceedings of the 2018 IEEE/RSJ International Conference on Intelligent Robots and Systems_ , pp. 4635–4640, 2018a.
  * Mahmood et al. (2018b) A. R. Mahmood, D. Korenkevych, G. Vasan, W. Ma, and J. Bergstra.  Benchmarking reinforcement learning algorithms on real-world robots.  In _Conference on Robot Learning_ , 2018b.
  * Mehta & Meyn (2009) P. G. Mehta and S. P. Meyn.  Q-learning and Pontryagin’s minimum principle.  In _Proceedings of the 48th IEEE Conference on Decision and Control. Held jointly with the 2009 28th Chinese Control Conference. CDC/CCC 2009._ , pp. 3598–3605, 2009.
  * Mnih et al. (2015) V. Mnih, K. Kavukcuoglu, D. Silver, A. A. Rusu, J. Veness, M. G. Bellemare, A. Graves, M. A. Riedmiller, A. Fidjeland, G. Ostrovski, S. Petersen, C. Beattie, A. Sadik, I. Antonoglou, H. King, D. Kumaran, D. Wierstra, S. Legg, and D. Hassabis.  Human-level control through deep reinforcement learning.  _Nature_ , 518(7540):529–533, 2015.
  * Munos (2006) R. Munos.  Policy gradient in continuous time.  _Journal of Machine Learning Research_ , 7:771–791, 2006.
  * Precup et al. (2000) D. Precup, R. S. Sutton, and S. Singh.  Eligibility traces for off-policy policy evaluation.  In _Proceedings of the 17th International Conference on Machine Learning, ICML 2000_ , pp. 759–766, 2000.
  * Precup (2000) Doina Precup.  _Temporal abstraction in reinforcement learning_.  PhD thesis, 2000.
  * Puterman (1994) M. L. Puterman.  _Markov Decision Processes: Discrete Stochastic Dynamic Programming_.  John Wiley & Sons, Inc., USA, 1st edition, 1994.  ISBN 0471619779.
  * Sutton (1988) R. S. Sutton.  Learning to predict by the methods of temporal differences.  _Machine Learning_ , 3:9–44, 1988.
  * Sutton & Barto (2018) R. S. Sutton and A. G Barto.  _Reinforcement learning: An introduction_.  MIT press, 2018.
  * Sutton et al. (1999) Richard S. Sutton, Doina Precup, and Satinder Singh.  Between mdps and semi-mdps: A framework for temporal abstraction in reinforcement learning.  _Artificial Intelligence_ , 112(1):181–211, 1999.  ISSN 0004-3702.
  * Tallec et al. (2019) C. Tallec, L. Blier, and Y. Ollivier.  Making deep q-learning methods robust to time discretization.  In _Proceedings of the 36th International Conference on Machine Learning_ , volume 97, pp. 6096–6104, 2019.
  * Todorov et al. (2012) E. Todorov, T. Erez, and Y. Tassa.  Mujoco: A physics engine for model-based control.  In _IROS_ , pp. 5026–5033. IEEE, 2012.
  * van Hasselt (2010) H. van Hasselt.  Double q-learning.  In _Advances in Neural Information Processing Systems_ , 2010.
  * van Seijen et al. (2009) H. van Seijen, H. van Hasselt, S. Whiteson, and M. A. Wiering.  A theoretical and empirical analysis of expected sarsa.  In _IEEE Symposium on Adaptive Dynamic Programming and Reinforcement Learning, ADPRL 2009_ , pp. 177–184, 2009.
  * Wang et al. (2016) Z. Wang, T. Schaul, M. Hessel, H. van Hasselt, M. Lanctot, and N. de Freitas.  Dueling network architectures for deep reinforcement learning.  In _Proceedings of the 33nd International Conference on Machine Learning, ICML 2016_ , pp. 1995–2003, 2016.
  * Watkins (1989) C. J. C. H. Watkins.  _Learning from Delayed Rewards_.  PhD thesis, King’s College, 1989.
  * Williams (1992) Ronald J Williams.  Simple statistical gradient-following algorithms for connectionist reinforcement learning.  _Machine Learning_ , 8(3-4):229–256, 1992.

##  Appendix A Discrete-time Return Visualization with Variable Interval Sizes

Below we provide a visualization akin to Figure 1, but with variable
discretization intervals.

Figure 7: The resulting sum when applying a discrete-time algorithm to a
discretized continuous-time domain with variable interval sizes. Note how
rectangle heights may fall out of the function’s range within an interval.

##  Appendix B Servo Reacher Environment Details

The environment state $\mathbf{x}$ is a column vector containing the DC
motor’s angular velocity [rad/s], the DC motor’s current [A], the output
shaft’s angle [rad], the output shaft’s angular velocity [rad/s], and the
output shaft’s target angle [rad], respectively. The state vector is updated
as follows:

$\dot{\mathbf{x}}_{t}\leftarrow\begin{bmatrix}-\frac{b_{m}}{J_{m}}&\frac{K_{t}}{J_{m}}&0&0&0\\\
-\frac{K_{t}}{L_{a}}&-\frac{R_{a}}{L_{a}}&0&0&0\\\ 0&0&0&1&0\\\
-\frac{b_{m}}{J_{m}N\eta}&\frac{K_{t}}{J_{m}N\eta}&0&0&0\\\
0&0&0&0&0\end{bmatrix}\mathbf{x}_{t}+\begin{bmatrix}0\\\ \frac{1}{L_{a}}\\\
0\\\ 0\\\ 0\end{bmatrix}A_{t}$
$\mathbf{x}_{t+1}\leftarrow\mathbf{x}_{t}+\dot{\mathbf{x}}_{t}\Delta_{s}$

where $\Delta_{s}=10^{-4}$ [s] is the simulation discretization granularity,
and $A_{t}$ is an input voltage with built-in saturation limits of
$\in[-12,12]$ [V]. The output shaft angle is clamped $\in[-1.306,1.306]$ [rad]
in accordance with Mahmood et al. (2018b). The physical parameters used are
detailed below:

$L_{a}$ | Armature Inductance |  $2.05\times 10^{-3}$ [H]
---|---|---
$R_{a}$ | Armature Resistance | 8.29 [Ohm]
$J_{m}$ | Rotor Inertia |  $8.67\times 10^{-8}$ [$\textrm{kg}\cdot\textrm{m}^{2}$]
$b_{m}$ | Rotor Friction |  $8.87\times 10^{-8}$ [$\textrm{N}\cdot\textrm{m}\cdot\textrm{s}$]
$K_{t}$ | Torque Constant | 0.0107 [$\frac{\textrm{N}\cdot\textrm{m}}{\textrm{A}}$]
$N$ | Gear Ratio | 200
$\eta$ | Gear Efficiency | 0.836

Given a target discretization interval $>10^{-4}$ [s], the above updates are
repeated until the target elapsed time is reached, keeping track of any
overshoot and compensating accordingly in the next time interval. As a
reinforcement learning environment, an agent observes the output shaft’s
angle, angular velocity, and target angle. The initial output shaft angle,
$\theta_{0}$, and target angle, $\theta_{target}$, are uniformly sampled
$\in[-1.306,1.306]$ at the start of each episode, and an episode terminates
when $|\theta_{t+1}-\theta_{target}|<0.1$ [rad] with angular velocity
$\dot{\theta}_{t+1}<0.1$ [rad/s]. An agent provides a continuous-valued action
as a voltage, and receives a reward $|\theta_{t+1}-\theta_{target}|$, computed
and received jointly with the next observation.


# Learning Dynamic Interpolation for Extremely Sparse Light Fields with Wide
Baselines

Mantang Guo 111Equal Contributions Jing Jin 111Equal Contributions Hui Liu
Junhui Hou  Department of Computer Science, City University of Hong Kong, Hong
Kong SAR
<EMAIL_ADDRESS><EMAIL_ADDRESS>
###### Abstract

In this paper, we tackle the problem of dense light field (LF) reconstruction
from sparsely-sampled ones with wide baselines and propose a learnable model,
namely dynamic interpolation, to replace the commonly-used geometry warping
operation. Specifically, with the estimated geometric relation between input
views, we first construct a lightweight neural network to dynamically learn
weights for interpolating neighbouring pixels from input views to synthesize
each pixel of novel views independently. In contrast to the fixed and content-
independent weights employed in the geometry warping operation, the learned
interpolation weights implicitly incorporate the correspondences between the
source and novel views and adapt to different image content information. Then,
we recover the spatial correlation between the independently synthesized
pixels of each novel view by referring to that of input views using a
geometry-based spatial refinement module. We also constrain the angular
correlation between the novel views through a disparity-oriented LF structure
loss. Experimental results on LF datasets with wide baselines show that the
reconstructed LFs achieve much higher PSNR/SSIM and preserve the LF parallax
structure better than state-of-the-art methods. The source code is publicly
available at https://github.com/MantangGuo/DI4SLF.

††footnotetext: This work was supported by the Hong Kong RGC under grants
CityU 21211518 and 11218121. Corresponding author: Junhui Hou

##  1 Introduction

Figure 1: Comparison of the commonly-used warping operation and the proposed
dynamic interpolation. In contrast to the fixed and content-independent
weights employed in the warping operation (taking bilinear interpolation
weights as an example), we propose to dynamically learn geometry-aware and
content-adaptive interpolation weights from carefully constructed embeddings.

Densely-sampled light field (LF) images record not only appearance but also
geometry information of 3D scenes, which enable wide applications, such as 3D
reconstruction [30, 24, 3], image post-refocusing [20], and virtual reality
[7, 38]. However, densely-sampled LFs raise great challenges for the
acquisition. For example, camera array [33] or computer-controlled gantry [29]
are either bulky and expensive or limited in capturing static scenes, while
cost-effective commercial LF cameras [17, 21] suffer from a trade-off between
the spatial and angular resolution due to the limited sensor resolution [8,
9].

Although many computational methods have been proposed to reconstruct densely-
sampled LFs from sparsely-sampled ones, the wide baseline between input views
remains a great challenge. To be more specific, non-depth-based methods [23,
37, 35, 31, 36, 4, 5] investigate the implicit signal distribution of LF data
to learn the mapping from sparse to dense LFs. These methods inevitably suffer
from the aliasing problem and lead to artifacts when the LF is extremely
under-sampled. In comparison, depth-based methods [32, 12, 34, 11, 10] perform
much better by employing the explicit geometry information. These methods
follow the general pipeline of warping-based view synthesis, and mainly focus
on improving the disparity estimation and post-processing refinement. However,
the reconstruction quality is still limited.

In this paper, we tackle the challenging problem of LF reconstruction from
extremely sparse and wide-baseline inputs, based on an insight that commonly-
used warping operation confronts with natural limitations. Specifically, the
warping operation synthesizes pixels of the novel view by performing
interpolation using their neighboring pixels from input views. The employed
interpolation weights are determined by fitting a simple and smooth curve
using a small set of neighbors, which inevitably impacts the reconstruction
quality as the content information is not considered. To this end, we propose
a learnable module, namely dynamic interpolation, to replace the commonly-used
warping operation. As shown in Fig. 1, dynamic interpolation uses a
lightweight neural network to dynamically predict geometry-aware and content-
adaptive interpolation weights for novel view synthesis. As the pixels of the
novel views are independently synthesized, we subsequently recover the spatial
correlation between them by referring to that of input views using a geometry-
based refinement module. We also constrain the angular correlation between the
novel views through a disparity-oriented LF structure loss. Extensive
experimental results demonstrate the significant superiority of the proposed
model on LF datasets with wide baselines over warping-based methods as well as
other state-of-the-art ones.

In summary, the main contributions of this paper are as follows:

  * •

we deeply analyze the geometry warping operation for handling the challenge of
LF reconstruction from wide-baseline inputs, and figure out the essential
limitation lies in the weakness of the interpolation weights; and

  * •

we reformulate the LF reconstruction from a new perspective and propose
dynamic interpolation, which is capable of overcoming the limitation of the
geometry warping operation.

##  2 Related Work

The existing LF reconstruction methods could be roughly divided into two
categories: non-learning-based methods and learning-based methods.

Non-learning-based methods usually adopt various prior assumptions to
regularize the LF data, i.e., Gaussian-based priors [16, 15, 19], sparse
priors [18, 23, 28], and low-rank [13]. These methods either require many
sparse samplings, or have high computational complexity. Explicitly estimating
the scene depth information, and then using it to warp input sub-aperture
images (SAIs) to novel ones is another kind of methods for LF reconstruction.
Wanner and Goldluecke [32] estimated disparity maps at input view by
calculating the structure tensor of epipolar plane images (EPIs), and then
used the estimated disparity maps to warp input SAIs to the novel viewpoints.
This method makes the reconstruction quality rely heavily on the accuracy of
the depth estimation. Zhang et al. [39] proposed a disparity-assisted phase-
based method that can iteratively refine the disparity map to minimize the
phase difference between the warped novel SAI and the input SAI. However, the
angular positions of synthesized SAIs are restricted to the neighbor of input
views, which cannot reconstruct LFs with large baselines.

Recently, many deep learning-based methods have been proposed to reconstruct
dense LFs from sparse samplings. Yoon et al. [37] reconstructed novel SAIs
from spatially up-sampled horizontal, vertical and surrounding SAI-pairs by
using three separate networks. This method can only regress novel SAIs from
adjacent ones, and could not process sparse LFs with large disparities. Wu et
al. [35] used a 2-D image super-resolution network to recover high-frequency
details along the angular dimension of the interpolated EPI. Analogously, Wang
et al. [31] restored the high-frequency details of EPI stacks by using 3-D
convolutional neural networks (CNNs). These methods process 2-D or 3-D slices
of a 4-D sparse LF, which cannot fully explore the spatial-angular
correlations implied in the LF. Yeung et al. [36] proposed the computational
efficient spatial-angular separable convolution for reconstructing a dense LF
from a sparse one in a single forward pass.

The pipeline of warping-based non-learning LF reconstruction methods is also
employed by several deep learning-based ones. Kalantari et al. [12] used two
sequential networks to separately estimate the disparity map at the novel
view, and predicted the color of novel SAI from warped images, respectively.
Wu et al. [34] extracted depth information from the sheared EPI volume, and
then used it to reconstruct high angular-resolution EPIs. These methods either
ignore the angular relations between synthesized SAIs, or underuse the spatial
information of the input SAIs during the reconstruction. Srinivasan et al.
[25] reconstructed an LF from a single 2-D image with predicting 4-D ray
depths. This method only works on dataset with small disparities, and is
restricted by its generalization ability. Jin et al. [10] explicitly learned
the disparity map at the novel viewpoint from input SAIs. They synthesized the
coarse novel SAIs individually by fusing the warped input SAIs with confidence
maps. Then they used a refinement network to recover the parallax structure by
exploring the complementary information from the coarse LF. Zhou et al. [40]
predicted the multiplane image at a reference view by using a CNN to represent
the scene’s content. Then the novel view can be synthesized from the
multiplane image representation with homography and alpha compositing.

Figure 2:  The flowchart of the proposed dynamic interpolation model for LF
reconstruction from extremely sparse (taking $M=2$ as an example) and wide-
baseline inputs. Our proposed model consists of three components: dynamic
weight learning, confidence-based blending, and geometry-based spatial
refinement.

##  3 Problem Analysis

Denote by $\mathcal{L}(u,x,y)\in\mathbb{R}^{U\times H\times W}$ a 3-D LF
containing $U$ sub-aperture images (SAIs) of each spatial resolution $H\times
W$, which are sampled along a 1-D straight line. The SAI at the angular
position of $u$ is denoted as $\mathbf{I}_{u}$. Given an extremely sparse LF
with $M$ SAIs, denoted by
$\mathcal{S}=\\{\mathbf{I}_{s_{1}},\cdots,\mathbf{I}_{s_{M}}\\}$, where $M\ll
U$, our goal is to synthesize the unsampled SAIs, denoted by
$\mathcal{T}=\\{\mathbf{I}_{t_{1}},\cdots,\mathbf{I}_{t_{N}}\\}$, where
$N=U-M$, so that the densely-sampled LF denoted by $\widetilde{\mathcal{L}}$
can be reconstructed as close to $\mathcal{L}$ as possible. This problem can
be implicitly formulated as:

$\widetilde{\mathcal{L}}=\mathcal{S}\bigcup\widetilde{\mathcal{T}}=f(\mathcal{S}),\vspace{-0.25cm}$
(1)

where $\widetilde{\mathcal{T}}$ is the set of synthesized novel SAIs, and $f$
denotes the mapping function to be learned. Not that we also denote the input
and target SAI as $\mathbf{I}_{s}$ and $\mathbf{I}_{t}$, respectively, in the
rest of the paper for simplification.

The SAIs of a 3-D LF image are the observations of the same scene from
different viewpoints. Under the assumption of Lambertian and non-occlusion,
projections of the same scene point will have the same intensity at different
SAIs. This relation can be described as:

$\mathbf{I}_{t}(\mathbf{x}_{t})=\mathbf{I}_{s}(\mathbf{x}^{\prime}_{t}),\vspace{-0.25cm}$
(2)

where $\mathbf{x}_{t}=(x_{t},y_{t})$ is the spatial coordinate, and
$\mathbf{x}^{\prime}_{t}$ is the location of the corresponding pixel of
$\mathbf{I}_{t}(\mathbf{x}_{t})$ at $\mathbf{I}_{s}$. Given the disparity
value of $\mathbf{I}_{t}(\mathbf{x}_{t})$, denoted by $d$,
$\mathbf{x}^{\prime}_{t}$ can be easily computed as
$\mathbf{x}^{\prime}_{t}=(x^{\prime}_{t},y^{\prime}_{t})=(x_{t}+d(s-t),y_{t})$.
Based on this relation, the pixels of $\mathbf{I}_{t}$ can be estimated by
collecting their corresponding pixels on $\mathbf{I}_{s}$. However, as the
values of $\mathbf{x}^{\prime}_{t}$ are always fractional, interpolation is
required to computed the intensity of the corresponding pixel by the weighted
sum of the neighboring pixels. The interpolation process can be formulated as:

$\mathbf{I}_{t}(\mathbf{x}_{t})=\sum_{\mathbf{x}_{s}\in\mathcal{P}_{\mathbf{x}^{\prime}_{t}}}w(\mathbf{x}_{s}-\mathbf{x}^{\prime}_{t};\phi_{w})\mathbf{I}_{s}(\mathbf{x}_{s}),\vspace{-0.25cm}$
(3)

where $\mathcal{P}_{\mathbf{x}^{\prime}_{t}}$ is the set of neighbors of
$\mathbf{x}^{\prime}_{t}$, and $w$ is the function with the parameter
$\phi_{w}$ which defines the interpolation weights for the pixels of
$\mathbf{I}_{s}$.

The above mentioned procedure is adopted by the commonly-used warping
operation. However, we provide an insight that this procedure has natural
limitations from two aspects:

(1) It requires to estimate $d$ to locate $\mathbf{x}^{\prime}_{t}$. However,
estimating the disparities of unsampled SAIs from the input SAIs is
challenging. Moreover, as $\mathcal{P}_{\mathbf{x}^{\prime}_{t}}$ is always a
small set of pixels surrounding $\mathbf{x}^{\prime}_{t}$, e.g., $2$ (or
$2\times 2$) neighbors for linear (or bilinear) interpolation [25] and $4$ (or
$4\times 4$) neighbors for cubic (or bicubic) interpolation [12], the
reconstruction results greatly rely on the accuracy of the disparity
estimation.

(2) The weight function $w(\cdot;\phi_{w})$ is defined by fitting a simple and
smooth curve using the small set $\mathcal{P}_{\mathbf{x}^{\prime}_{t}}$,
which neglects the content information. Thus, even with an accurate estimation
of $d$, it is difficult to produce high-quality results, especially on areas
with texture edges, occlusion boundaries, and non-Lambertian objects.

Base on these observations, we propose a novel model, namely dynamic
interpolation, to synthesize $\widetilde{\mathcal{T}}$ from $\mathcal{S}$,
which overcomes the limitations of the commonly-used warping operation by
learning geometry-aware and content-adaptive interpolation weights.

##  4 Proposed Method

Overview. As shown in Fig. 2, the proposed model, namely dynamic
interpolation, mainly consists of three components, i.e., dynamic weight
learning, confidence-based blending, and geometry-based spatial refinement.
Specifically, we first independently synthesize each pixel of
$\widetilde{\mathbf{I}}_{t}$ by applying interpolation over its neighboring
pixels from $\mathbf{I}_{s}$. In contrast to commonly-used warping operation,
the interpolation weights used in our model are dynamically learned. We also
estimate confidence maps to blend the pixels interpolated from different input
SAIs, which further handles the occlusion problems. Then, we recover the
spatial correlation between the independently synthesized pixels by referring
to that of $\mathbf{I}_{s}$.

In this paper, we set $M=2$. It is worth noting that our framework could be
straightforwardly extended to 4-D LFs with larger $M$. In what follows, we
will introduce the technical details of the proposed method.

###  4.1 Dynamic Weight Learning

This module aims at learning the interpolation weights to independently
synthesize each pixel of a novel SAI, denoted by
$\mathbf{I}_{t}(\mathbf{x}_{t})$, from $\mathbf{I}_{s}$. The interpolation
weight for each pixel $\mathbf{I}_{s}(\mathbf{x}_{s})$ is predicted by a
multilayer perceptron (MLP), and the following information is embedded in the
MLP:

(1) The correspondence relation between $\mathbf{I}_{t}$ and $\mathbf{I}_{s}$,
which helps to implicitly locate the corresponding pixel of
$\mathbf{I}_{t}(\mathbf{x_{t}})$ in $\mathbf{I}_{s}$. The correspondence
embedding consists of three components, i.e., a geometric code
$E_{\mathbf{x}_{t},\mathbf{x}_{s}}^{geo}$, a spatial code
$\mathbf{E}_{\mathbf{x}_{t},\mathbf{x}_{s}}^{spa}$, and an angular code
$E_{\mathbf{x}_{t},\mathbf{x}_{s}}^{ang}$. Specifically,
$E_{\mathbf{x}_{t},\mathbf{x}_{s}}^{geo}$ is the disparity value of
$\mathbf{I}_{s}(\mathbf{x}_{s})$, i.e.,

$\displaystyle
E_{\mathbf{x}_{t},\mathbf{x}_{s}}^{geo}=\mathbf{D}_{s}(\mathbf{x}_{s}),$ (4)

where $\mathbf{D}_{s}$ is the disparity map of $\mathbf{I}_{s}$, which is
estimated from $\mathcal{S}$ using a pre-trained optical-flow model.
$E_{\mathbf{x}_{t},\mathbf{x}_{s}}^{spa}$ and
$E_{\mathbf{x}_{t},\mathbf{x}_{s}}^{ang}$ describe the the spatial and angular
distance between $\mathbf{I}_{t}(\mathbf{x}_{t})$ and
$\mathbf{I}_{s}(\mathbf{x}_{s})$, i.e.,

$\displaystyle\mathbf{E}_{\mathbf{x}_{t},\mathbf{x}_{s}}^{spa}$
$\displaystyle=\mathbf{x}_{s}-\mathbf{x}_{t},\
E_{\mathbf{x}_{t},\mathbf{x}_{s}}^{ang}$ $\displaystyle=s-t.$ (5)

These information can directly determine whether
$\mathbf{I}_{s}(\mathbf{x}_{s})$ corresponds to
$\mathbf{I}_{t}(\mathbf{x}_{t})$ under the estimated geometric relation, and
thus, greatly helps the MLP to locate informative pixels in $\mathbf{I}_{s}$
and allocate larger weights to them.

(2) The content information around $\mathbf{I}_{s}(\mathbf{x}_{s})$, which
helps to understand complicated scenarios, such as texture edges, occlusion
boundaries, and non-Lambertian objects. To construct the content embedding,
denoted by $\mathbf{E}_{\mathbf{x}_{t},\mathbf{x}_{s}}^{ctt}$, we first
backward warp the other SAI in $\mathcal{S}$ to $\mathbf{I}_{s}$ based on
$\mathbf{D}_{s}$, and the resulting image is denoted by
$\widehat{\mathbf{I}}_{s^{\prime}\rightarrow s}$. Then, we employ a sub-CNN
$f_{c}(\cdot)$ to learn the content information, i.e.,

$\displaystyle\mathbf{E}_{\mathbf{x}_{t},\mathbf{x}_{s}}^{ctt}$
$\displaystyle=f_{c}\left(\mathbf{x}_{s},\mathbf{x}_{t},\mathbf{I}_{s},\widehat{\mathbf{I}}_{s^{\prime}\rightarrow
s},\mathbf{D}_{s}\right).$ (6)

It is expected that $f_{c}$ is able to detect the texture edges of
$\mathbf{I}_{s}$ and understand the occlusion and non-Lambertian relations by
comparing $\mathbf{I}_{s}$ and $\widehat{\mathbf{I}}_{s^{\prime}\rightarrow
s}$ with the assistance of $\mathbf{D}_{s}$.

Finally, the geometry and content embedding, denoted by
$\mathbf{E}_{\mathbf{x}_{t},\mathbf{x}_{s}}$, is constructed as:

$\displaystyle\mathbf{E}_{\mathbf{x}_{t},\mathbf{x}_{s}}=\textsf{CAT}\left(E_{\mathbf{x}_{t},\mathbf{x}_{s}}^{geo},\mathbf{E}_{\mathbf{x}_{t},\mathbf{x}_{s}}^{spa},E_{\mathbf{x}_{t},\mathbf{x}_{s}}^{ang},\mathbf{E}_{\mathbf{x}_{t},\mathbf{x}_{s}}^{ctt}\right),$
(7)

where $\textsf{CAT}(\cdot)$ is the concatenation operation, and the
interpolation weights for $\mathbf{I}_{s}(\mathbf{x}_{s})$ to synthesize
$\mathbf{I}_{t}(\mathbf{x}_{t})$, denoted by
$W_{\mathbf{x}_{t},\mathbf{x}_{s}}$, is predicted as:

$\displaystyle
W_{\mathbf{x}_{t},\mathbf{x}_{s}}=f_{w}\left(\mathbf{E}_{\mathbf{x}_{t},\mathbf{x}_{s}}\right),$
(8)

where $f_{w}(\cdot)$ is the learnable MLP.

To reduce the computational cost, the interpolation is performed over the
neighborhood of $\mathbf{I}_{t}(\mathbf{x}_{t})$ in $\mathbf{I}_{s}$, instead
of the whole range of $\mathbf{I}_{s}$. Suppose the disparity range of
$\mathbf{I}_{t}$ is $[-d_{max},d_{max}]$, the neighborhood of
$\mathbf{I}_{t}(\mathbf{x}_{t})$ in $\mathbf{I}_{s}$ is defined as:
$\mathcal{P}_{\mathbf{x}_{t}}=\\{\mathbf{x}=(x,y)|x_{t}-d_{max}(s-t)\leq x\leq
x_{t}+d_{max}(s-t),y=y_{t}\\}$. Then, we predict the intensity of
$\mathbf{I}_{t}(\mathbf{x}_{t})$ by applying interpolation on
$\mathcal{P}_{\mathbf{x}_{t}}$ based on the learned weights, and the predicted
result is denoted by $\widetilde{\mathbf{I}}_{s\rightarrow
t}(\mathbf{x}_{t})$, i.e.,

$\displaystyle\widetilde{\mathbf{I}}_{s\rightarrow
t}(\mathbf{x}_{t})=\sum_{\mathbf{x}_{s}\in\mathcal{P}_{\mathbf{x}_{t}}}W_{\mathbf{x}_{t},\mathbf{x}_{s}}\mathbf{I}_{s}(\mathbf{x}_{s}).$
(9)

Remark: Compared with the commonly-used warping operation, our dynamic
interpolation has the following advantages:

(1) Instead of relying on the disparity estimation accuracy of the novel SAI,
we utilize the disparity map directly estimated between input SAIs, which is
much more reliable. Moreover, the geometry information is implicitly
incorporated by learning the weights for each pixel in the possibly maximum
neighborhood, which might improve the tolerance of the disparity estimation
error.

(2) Instead of fitting a simple curve using a small set of pixels for
interpolation, we learn the weights using an MLP and provide content
information over a relatively large field to make the weights adaptive to
various and complicated neighboring correlations.

###  4.2 Confidence-based Blending

Although the weight learning module has the ability of handling the problem of
occlusion boundaries by embedding the content information, it is still
difficult to synthesize the pixels whose correspondences are completely
occluded in $\mathbf{I}_{s}$ by interpolation on only one of the input SAIs.
Fortunately, the object occluded from one viewpoint might be visible from
another one. Therefore, we blend the images synthesized from different input
SAIs under the guidance of their confidence maps, which indicate the non-
occlusion pixels with higher values.

To predict the confidence value for each pixel position $\mathbf{x}_{t}$ in
the synthesized SAI, we first aggregate the geometry and content embeddings
for each neighbors of $\mathbf{x}_{t}$ in $\mathbf{I}_{s}$ by concatenation,
and then apply another MLP, denoted by $f_{b}(\cdot)$, on the aggregated
feature, i.e.,

$\displaystyle\widetilde{\mathbf{C}}_{s\rightarrow
t}(\mathbf{x_{t}})=f_{b}\left(\textsf{CAT}\\{\mathbf{E}_{\mathbf{x}_{t},\mathbf{x}_{s}}|\mathbf{x}_{s}\in\mathcal{P}_{\mathbf{x}_{t}}\\}\right),$
(10)

where $\widetilde{\mathbf{C}}_{s\rightarrow t}$ is the confidence map for
$\widetilde{\mathbf{I}}_{s\rightarrow t}$. Based on the learned confidence
map, the SAIs synthesized from different input SAIs are combined to produce
the intermediate result of the novel SAI, denoted by
$\widetilde{\mathbf{I}}_{t}^{b}$, i.e.,

$\displaystyle\widetilde{\mathbf{I}}_{t}^{b}=\sum_{s\in\\{s_{1},\cdots,s_{M}\\}}\widetilde{\mathbf{C}}_{s\rightarrow
t}\odot\widetilde{\mathbf{I}}_{s\rightarrow t},$ (11)

where $\odot$ is the element-wise multiplication operator.

###  4.3 Geometry-based Spatial Refinement

As pixels in $\widetilde{\mathbf{I}}_{t}^{b}$ are independently synthesized,
the spatial correlations among them are not considered. To further enhance the
quality of $\widetilde{\mathbf{I}}_{t}^{b}$, we propose a refinement module to
recover its spatial correlation by inferring that from the input SAIs using a
sub-CNN. Considering the wide baseline between
$\widetilde{\mathbf{I}}_{t}^{b}$ and $\mathbf{I}_{s}$, directly applying a
network will have difficulties to perceive the corresponding information from
$\mathbf{I}_{s}$. Therefore, we adopt a geometry-based spatial refinement,
which first explicitly locates the correspondences in $\mathbf{I}_{s}$ at the
patch level, and then learn the spatial correlations from $\mathbf{I}_{s}$ to
refine $\widetilde{\mathbf{I}}_{t}^{b}$.

Let $\widetilde{\mathbf{H}}_{t}^{\mathbf{x}_{t}^{o}}$ denote a patch of
$\widetilde{\mathbf{I}}_{t}^{b}$ centered at
$\mathbf{x}_{t}^{o}=(x_{t}^{o},y_{t}^{o})$. To locate its similar patch in
$\mathbf{I}_{s}$, we first estimate the disparity map of
$\widetilde{\mathbf{I}}_{t}^{b}$ by forward warping $\mathbf{D}_{s}$,
resulting in $\widetilde{\mathbf{D}}_{t}$, and then calculate the patch-level
disparity of $\widetilde{\mathbf{H}}_{t}^{\mathbf{x}_{t}^{o}}$ by averaging
the disparity over all of its contained pixels, leading to
$\widetilde{d}_{h}$. Then, the central position of the corresponding patch of
$\widetilde{\mathbf{H}}_{t}^{\mathbf{x}_{t}^{o}}$ at $\mathbf{I}_{s}$, denoted
by $\mathbf{x}_{s}^{o}=(x_{s}^{o},y_{s}^{o})$, can be estimated as:

$\displaystyle x_{s}^{o}=x_{t}^{o}+\widetilde{d}_{h}(s-t).$ (12)

Based on $\mathbf{x}_{s}^{o}$, we can collect the corresponding patch of
$\widetilde{\mathbf{H}}_{t}^{\mathbf{x}_{t}^{o}}$ at $\mathbf{I}_{s}$, which
is denoted by $\mathbf{H}_{s}^{\mathbf{x}_{s}^{o}}$.

To recover the spatial correlation among pixels in
$\widetilde{\mathbf{H}}_{t}^{\mathbf{x}_{t}^{o}}$, we feed the concatenation
of $\widetilde{\mathbf{H}}_{t}^{\mathbf{x}_{t}^{o}}$ and its corresponding
patches in all the input SAIs, i.e.,
$\\{\mathbf{H}_{s}^{\mathbf{x}_{s}^{o}}|s\in\\{s_{1},\cdots,s_{M}\\}\\}$, into
a sub-CNN to predict a residual map for refinement. We then merge the refined
patches to produce the final prediction of the novel SAI, i.e.,

$\displaystyle\widetilde{\mathbf{I}}_{t}=f_{r}(\widetilde{\mathbf{I}}_{t}^{b}),$
(13)

where $f_{r}(\cdot)$ denotes the geometry-based spatial refinement module.

###  4.4 Disparity-oriented Loss

The final and intermediate predictions of the novel SAIs are supervised by the
ground-truth one, i.e., the loss function for the reconstruction of
$\mathbf{I}_{t}$ is defined as:

$\displaystyle\ell^{r}_{t}=\left\|\widetilde{\mathbf{I}}_{t}-\mathbf{I}_{t}\right\|_{1}+\left\|\widetilde{\mathbf{I}}_{t}^{b}-\mathbf{I}_{t}\right\|_{1}+\sum_{s\in\\{s_{1},\cdots,s_{M}\\}}\left\|\widetilde{\mathbf{I}}_{s\rightarrow
t}-\mathbf{I}_{t}\right\|_{1}.$ (14)

Moreover, as each novel SAI is reconstructed individually, we propose a
disparity-oriented LF structure loss to constrain the angular correlation
between them. The relation described in Eq. (2) can be constrained by
minimizing the gradients along the directions of the straight lines in EPIs of
the LF. Considering the existence of occlusions and non-Lambertian, we instead
minimizing the distance between the gradients of predicted EPIs and the
ground-truth ones. Note that the directions of EPI lines are located under the
guidance of the ground-truth disparity of the LFs, which are easily available
in the training datasets. Such a disparity-oriented LF structure loss can be
formulated as

$\displaystyle\ell^{d}=\left\|\nabla_{d}\widetilde{\mathcal{E}}-\nabla_{d}\mathcal{E}\right\|_{1},$
(15)

where $\nabla_{d}$ is the gradient operator along the direction defined by the
ground-truth disparity $d$ of each pixel, and $\widetilde{\mathcal{E}}$ and
$\mathcal{E}$ are the EPIs of $\widetilde{\mathcal{L}}$ and $\mathcal{L}$,
respectively.

Our framework is end-to-end trained using the final objective function defined
as: $\ell=\sum_{t\in\\{t_{1},\cdots,t_{N}\\}}\ell_{t}^{r}+\lambda\ell^{d}$,
where $\lambda\geq 0$ is the weight factor for the disparity-oriented loss.

Table 1: Quantitative comparisons (PSNR/SSIM) of different methods over the Inria Sparse LF dataset [22]. Light Field | Disparity range | Baseline |  Kalantari et al. [12] |  Wu et al. [34] |  Jin et al. [10] |  Ours (PWCNet) |  Ours (RAFT)
---|---|---|---|---|---|---|---
Electro_devices | [-19.6, 32.8] | 28.49/0.871 | 24.66/0.691 | 28.51/0.866 | 32.77/0.936 | 33.04/0.941 | 35.43/0.960
Flying_furniture | [-34.0, 62.4] | 28.39/0.838 | 28.83/0.784 | 27.38/0.783 | 31.69/0.896 | 30.06/0.881 | 33.93/0.935
Coffee_beans_vases | [10.8, 58.4] | 27.17/0.886 | 21.54/0.579 | 23.04/0.836 | 28.08/0.927 |  29.63/0.936 | 29.55/0.943
Dinosaur | [-57.6, 72.8] | 23.00/0.773 | 22.21/0.731 | 23.07/0.788 | 26.61/0.897 | 24.94/0.861 | 27.50/0.904
Flowers | [-40.4, 66.0] | 23.05/0.757 | 21.96/0.667 | 23.52/0.767 | 24.36/0.842 | 25.24/0.860 | 24.86/0.849
Rooster_clock | [-34.4, 21.2] | 31.43/0.904 | 22.71/0.710 | 29.05/0.887 | 27.69/0.929 | 35.90/0.946 | 38.16/0.966
Smiling_crowd | [-40.4, 64.8] | 18.87/0.722 | 17.01/0.596 | 19.52/0.729 | 21.01/0.822 | 20.90/0.824 | 22.87/0.877
Average | 25.77/0.821 | 22.70/0.680 | 25.05/0.802 | 27.46/0.893 | 28.53/0.893 | 30.33/0.919

##  5 Experimental Results

###  5.1 Training Details and Datasets

Both the content embedding network $f_{c}(\cdot)$ and the spatial refinement
network $f_{r}(\cdot)$ are 2-D CNNs that consist of $4$ residual blocks [6]
with the kernel of size $3\times 3$. Zeros-padding was applied to keep the
spatial size unchanged. We refer readers to the Supplementary Material for the
detailed network architecture. At each iteration of the training phase, a
$32\times 32$ patch randomly cropped from the LF image was synthesized. We
used the unfold function in PyTorch to efficiently locate the neighborhood of
each synthesized pixel.

The batch size was empirically set to $1$. The learning rate was initially set
to $1e^{-4}$ and reduced to $1e^{-5}$ after $8000$ epochs. We used Adam [14]
with $\beta_{1}=0.9$ and $\beta_{2}=0.999$ as the optimizer.

We trained our framework using $29$ LF images from the Inria Sparse LF dataset
[22]. Each LF image contains $9\times 9$ SAIs with a disparity range of
$[-20,20]$ between adjacent SAIs. We took the 3rd and 7th SAIs at the same row
as inputs, which have a wide baseline up to $80$ pixels, to train the
framework. The test dataset consists of $7$ LF images from the Inria Sparse LF
dataset [22] and $14$ LF images from the MPI LF archive [1]. Note that MPI [1]
is a high angular-resolution LF dataset where each LF image contains $101$
SAIs distributed on a scanline. Thus, we can construct testing LFs with
different baselines by sampling SAIs with different intervals (see details in
Section 5.2 ).

###  5.2 Comparison with State-of-the-art Methods

We compared the proposed method with three state-of-the-art deep learning-
based methods for LF reconstruction, including Kalantari et al. [12], Wu et
al. [34], and Jin et al. [10]. All the methods were retrained on the same
dataset with the officially released codes and suggested configurations. Note
that the 2-D angular convolutional layers used by Jin et al. [10] were
degenerated to 1-D convolutional layers to adapt to the 3-D LF.

To verify the advantage of the proposed dynamic interpolation in comparison to
the commonly-used warping operation, we developed a baseline model by
replacing the dynamic weight learning module with a disparity-based warping
operation while leaving the confidence-based blending and the geometry-based
refinement unchanged. Note that to ensure fair comparisons, the disparity maps
of the novel views used for warping were estimated from the same inputs as
Ours, i.e., the disparity maps of the input views from a pre-trained optical-
flow model.

Moreover, to demonstrate the effects of the input disparity estimation
accuracy on the performance of our model, we adopted two different optical-
flow methods, namely RAFT [27] and PWCNet [26], and used them to separately
train two models, denoted as Ours (RAFT) and Ours (PWCNet), respectively.

(a)  (b)

Figure 3: Quantitative comparisons (PSNR/SSIM) of different methods under
different disparity ranges (pixels) between input SAIs on the MPI LF dataset
[1]. All subfigures share the same legend shown in the first one. Figure 4:
Visual comparisons of reconstructed LFs from different methods over the Inria
Sparse LF dataset [22]. The disparity range between input SAIs of each LF is
shown on the left. Figure 5: Visual comparisons of reconstructed LFs from
different methods over the MPI LF dataset [1]. The disparity range between
input SAIs reaches $80$ pixels for each reconstructed LF.

Quantitative comparisons on the Inria Sparse LF dataset. We reconstructed
$5\times 5$ LF images in the Inria dataset [22] in a row-by-row manner, and
calculated the PSNR and SSIM values between the reconstructed LFs and ground-
truth ones in Y channel to quantitatively evaluate different methods. Table 1
lists the results, where it can be observed that:

  * •

Ours (RAFT) achieves significantly higher PSNR and SSIM than Baseline.
Although the disparity maps between input SAIs are also provided to Baseline,
its performance is still limited by the fixed and content-independent weights
employed in the commonly-used warping operation, which demonstrates the
advantage of learned dynamic weights in our method;

  * •

Ours (PWCNet) is worse than Ours (RAFT) but still much better than all the
compared methods. Although the estimated disparity maps with different levels
of quality are involved, our method may adaptively aggregate content features
to provide gains for learning dynamic weights, which can demonstrate the
advantages of learning dynamic weights over commonly-used ones. Besides, it is
highly expected that our framework will be further improved with more powerful
and advanced optical flow estimation proposed in future;

  * •

our method achieves higher performance than Wu et al. [34]. The reason maybe
that Wu et al. [34] reconstructed LFs in the perspective of reconstructing 2-D
EPIs, and neglected modeling the correlations between two spatial dimensions,
which limits the performance. On the contrary, our method employs a geometry-
based refinement module to refine the correlations among pixels of novel SAIs,
and further improves the reconstruction quality; and

  * •

both Kalantari et al. [12] and Jin et al. [10] achieve worse performances than
our method. In addition to that they cannot estimate disparities well from
extremely sparse LFs with a limited receptive field of CNNs, they also cannot
handle severe artifacts brought by the warping operation. By contrast, our
method can effectively mitigate the warping errors in reconstructing extremely
sparse LFs by dynamically learning content-adaptive weights for each pixel of
the novel SAI. Besides, the geometry-based refinement module can further
refine novel SAIs to improve the quality.

Figure 6: Visual comparisons of estimated depth maps from different methods
over the Inria Sparse LF dataset [22]. The disparity range between input SAIs
of each LF is shown on the left.

Quantitative comparisons on the MPI LF dataset. Furthermore, we also evaluated
the robustness of different methods under different disparity ranges on the
MPI dataset [1]. Each scene contains a high-angular densely-sampled LF image
composed of $101$ SAIs distributed on a scanline with spatial resolution
$720\times 960$. The disparity between adjacent SAI is around $1$ pixel. We
can sample SAIs with different intervals along the angular dimension to
construct LFs with different disparity ranges. Specifically, we separately set
$10$ disparity ranges from $8$ to $80$ pixels between two input SAIs. For each
disparity range, we evenly sampled $3$ SAIs between two input SAIs as ground
truth. The PSNR/SSIM shown in Fig. 3 indicates that although the
reconstruction quality of all methods decreases along with the increase of the
disparity range, Ours consistently achieves better reconstruction quality than
the other methods under all disparity ranges, demonstrating the robustness of
our method towards different disparity ranges.

Table 2: Comparisons of running time (in seconds per view) and model parameter size (M) of different methods on the Inria Sparse LF dataset [22]. | Baseline |  Kalantari et al. [12] |  Wu et al. [34] |  Jin et al. [10] | Ours
---|---|---|---|---|---
Time | 0.65 | 6.32 | 27.77 | 0.69 | 2.78
# Params | 0.95 | 2.55 | 0.24 | 2.22 | 0.69

Qualitative comparisons. We visually compared reconstructed wide-baseline LFs
by different methods. One of the challenges for the wide-baseline LF
reconstruction is fairly reconstructing a large number of occlusion regions
while are not so many in narrow-baseline ones. From reconstructed SAIs and
zoomed-in regions shown in Fig. 4 and Fig. 5, it can be observed that our
method can produce sharp edges at the occlusion boundaries, while other
methods produce either severe distortions or heavy blurry effects at these
regions. Moreover, our method can produce better high-frequency details at
texture regions than other methods, which demonstrate that our method can
still achieve high-quality LF reconstructions even under such large input
disparity range. Please refer to the Supplementary Material for more visual
results.

Comparisons of the LF parallax structure. Moreover, as the parallax structure
is one of the most important values of LF data, we thus managed to compare the
parallax structures of LFs reconstructed by different methods. On the one
hand, comparisons of EPIs in Figs. 4 and 5 indicate that our method can
preserve clearer linear structures than other methods, even for lines
corresponding to large-disparity regions, validating the strong ability of our
method in preserving parallax structure on extremely sparse LFs. On the other
hand, it is expected that depth/disparity maps estimated from high-quality LFs
shall be close to those are estimated from ground-truth ones. Thus, we
compared the depth maps estimated from reconstructed LFs of different methods
by a identical LF-based depth estimation method [2]. As shown in Fig. 6, our
method can produce sharper edges at occlusion boundaries and preserve
smoothness at regions with uniform depth, which are closest to the ground
truth. Such observations also demonstrate the advantage of our method on
preserving the LF parallax structure.

Efficiency comparisons. We compared the efficiency and model size of different
methods. All the methods were implemented on a Linux server with Intel CPU
E5-2699 @ 2.20GHz, 128GB RAM and Tesla V100. As listed in Table 2, we can see
that Ours is much faster than Kalantari et al. [12] and Wu et al. [34] but
slower than Baseline and Jin et al. [10]. Besides, our model size is smaller
than Baseline, Kalantari et al. [12], and Jin et al. [10], but larger than Wu
et al. [34]. Taking the reconstruction accuracy, efficiency, and model size
together, we believe our method is the best.

###  5.3 Ablation Study

We carried out comprehensive ablation studies to validate the effectiveness of
three key components involved in our framework, i.e., content embedding, the
geometry-based refinement module, and the disparity-oriented loss term.
Specifically, each component was sequentially added to the base model until
all the three components were included to form the complete model. As shown in
Table 3 and Fig. 7, it can be seen that there is a significant increase of
performance when adding the content embedding to the base model, which
verifies the advantage brought by detecting the texture edges of input views,
and understanding the occlusion and non-Lambertian relations between input
views. The visual comparisons shown in Fig. 7 (examples 1-3) also verify the
advantage. Moreover, the geometry-based refinement module can also bring
around $0.3$ dB increase of PSNR based on model with the content embedding.
The examples 4-5 in Fig. 7 also shown that some fine structures such as
delicate objects and textures are obviously broken without this module. It
verifies the effectiveness of our refining pixel correlations in novel SAI
through being guided by the correct ones from input SAIs. By comparing results
in the last two rows in Table 3 and the EPIs in examples 6-7 in Fig. 7, we can
see that supervising the parallax structure by the ground-truth disparity
during training is helpful to reconstruct high-quality LFs. We also provide
the intermediate visual results before and after confidence-based blending in
Fig. 8, where it can be observed that the confidence-based blending handles
the occlusion regions by leveraging the advantages of the left and right
results under the guidance of their confidence maps.

Table 3: Ablation study. “$\times$” denotes that the corresponding component is not included, while “$\surd$” denotes being included. Content embedding |  Geometry- based refinement |  Disparity- oriented loss | PSNR | SSIM
---|---|---|---|---
$\times$ | $\times$ | $\times$ | 29.05 | 0.901
$\surd$ | $\times$ | $\times$ | 29.83 | 0.915
$\surd$ | $\surd$ | $\times$ | 30.14 | 0.920
$\surd$ | $\surd$ | $\surd$ | 30.33 | 0.919
Figure 7: Effectiveness of the three key modules. Figure 8: Effectiveness of
confidence-based blending.

##  6 Conclusion

We have presented a learning-based method for densely-sampled LF
reconstruction from extremely sparse ones. More precisely, we focused on
addressing the challenging problem of wide-baseline inputs and proposed a
novel dynamic interpolation model. By learning geometry-aware and content-
adaptive interpolation weights via a lightweight neural network, our method
overcomes the limitations of commonly-used warping operation, and efficiently
reconstructs LFs with much higher quality, compared with state-of-the-art
methods.

## References

  * [1] Vamsi Kiran Adhikarla, Marek Vinkler, Denis Sumin, Rafał Mantiuk, Karol Myszkowski, Hans-Peter Seidel, and Piotr Didyk.  Towards a quality metric for dense light fields.  In IEEE Conference on Computer Vision and Pattern Recognition (CVPR), pages 58–67, 2017.
  * [2] Jie Chen, Junhui Hou, Yun Ni, and Lap-Pui Chau.  Accurate light field depth estimation with superpixel regularization over partially occluded regions.  IEEE Transactions on Image Processing, 27(10):4889–4900, 2018.
  * [3] Chunle Guo, Jing Jin, Junhui Hou, and Jie Chen.  Accurate light field depth estimation via an occlusion-aware network.  In IEEE International Conference on Multimedia and Expo (ICME), pages 1–6, 2020.
  * [4] Mantang Guo, Junhui Hou, Jing Jin, Jie Chen, and Lap-Pui Chau.  Deep spatial-angular regularization for compressive light field reconstruction over coded apertures.  In European Conference on Computer Vision (ECCV), pages 278–294. Springer, 2020.
  * [5] Mantang Guo, Junhui Hou, Jing Jin, Jie Chen, and Lap-Pui Chau.  Deep spatial-angular regularization for light field imaging, denoising, and super-resolution.  IEEE Transactions on Pattern Analysis and Machine Intelligence, 2021\.
  * [6] Kaiming He, Xiangyu Zhang, Shaoqing Ren, and Jian Sun.  Deep residual learning for image recognition.  In IEEE Conference on Computer Vision and Pattern Recognition (CVPR), pages 770–778, 2016.
  * [7] Fu-Chung Huang, Kevin Chen, and Gordon Wetzstein.  The light field stereoscope: immersive computer graphics via factored near-eye light field displays with focus cues.  ACM Transactions on Graphics, 34(4):60, 2015.
  * [8] Jing Jin, Junhui Hou, Jie Chen, and Sam Kwong.  Light field spatial super-resolution via deep combinatorial geometry embedding and structural consistency regularization.  In IEEE Conference on Computer Vision and Pattern Recognition (CVPR), pages 2260–2269, 2020.
  * [9] Jing Jin, Junhui Hou, Jie Chen, Sam Kwong, and Jingyi Yu.  Light field super-resolution via attention-guided fusion of hybrid lenses.  In ACM International Conference on Multimedia (ACM MM), pages 193–201, 2020.
  * [10] Jing Jin, Junhui Hou, Jie Chen, Huanqiang Zeng, Sam Kwong, and Jingyi Yu.  Deep coarse-to-fine dense light field reconstruction with flexible sampling and geometry-aware fusion.  IEEE Transactions on Pattern Analysis and Machine Intelligence, 2020\.
  * [11] Jing Jin, Junhui Hou, Hui Yuan, and Sam Kwong.  Learning light field angular super-resolution via a geometry-aware network.  In AAAI Conference on Artificial Intelligence (AAAI), volume 34, pages 11141–11148, 2020.
  * [12] Nima Khademi Kalantari, Ting-Chun Wang, and Ravi Ramamoorthi.  Learning-based view synthesis for light field cameras.  ACM Transactions on Graphics, 35(6):1–10, 2016.
  * [13] Mahdad Hosseini Kamal, Barmak Heshmat, Ramesh Raskar, Pierre Vandergheynst, and Gordon Wetzstein.  Tensor low-rank and sparse light field photography.  Computer Vision and Image Understanding, 145:172–181, 2016.
  * [14] Diederik P Kingma and Jimmy Lei Ba.  Adam: A method for stochastic gradient descent.  In International Conference on Learning Representations (ICLR), pages 1–15, 2015.
  * [15] Anat Levin and Fredo Durand.  Linear view synthesis using a dimensionality gap light field prior.  In IEEE Computer Society Conference on Computer Vision and Pattern Recognition (CVPR), pages 1831–1838. IEEE, 2010.
  * [16] Anat Levin, William T Freeman, and Frédo Durand.  Understanding camera trade-offs through a bayesian analysis of light field projections.  In European Conference on Computer Vision (ECCV), pages 88–101. Springer, 2008.
  * [17] Lytro.  http://lytro.com, 2016.
  * [18] Kshitij Marwah, Gordon Wetzstein, Yosuke Bando, and Ramesh Raskar.  Compressive light field photography using overcomplete dictionaries and optimized projections.  ACM Transactions on Graphics, 32(4):1–12, 2013.
  * [19] Kaushik Mitra and Ashok Veeraraghavan.  Light field denoising, light field superresolution and stereo camera based refocussing using a gmm light field patch prior.  In IEEE Computer Society Conference on Computer Vision and Pattern Recognition Workshops (CVPRW), pages 22–28. IEEE, 2012.
  * [20] Ren Ng et al.  Digital light field photography.  Stanford University, 2006.
  * [21] RayTrix.  3d light field camera technology.  https://raytrix.de/.
  * [22] Jinglei Shi, Xiaoran Jiang, and Christine Guillemot.  A framework for learning depth from a flexible subset of dense and sparse light field views.  IEEE Transactions on Image Processing, 28(12):5867–5880, 2019.
  * [23] Lixin Shi, Haitham Hassanieh, Abe Davis, Dina Katabi, and Fredo Durand.  Light field reconstruction using sparsity in the continuous fourier domain.  ACM Transactions on Graphics, 34(1):1–13, 2014.
  * [24] Changha Shin, Hae-Gon Jeon, Youngjin Yoon, In So Kweon, and Seon Joo Kim.  Epinet: A fully-convolutional neural network using epipolar geometry for depth from light field images.  In IEEE Conference on Computer Vision and Pattern Recognition (CVPR), pages 4748–4757, 2018.
  * [25] Pratul P Srinivasan, Tongzhou Wang, Ashwin Sreelal, Ravi Ramamoorthi, and Ren Ng.  Learning to synthesize a 4d rgbd light field from a single image.  In IEEE International Conference on Computer Vision (ICCV), pages 2243–2251, 2017.
  * [26] Deqing Sun, Xiaodong Yang, Ming-Yu Liu, and Jan Kautz.  Pwc-net: Cnns for optical flow using pyramid, warping, and cost volume.  In IEEE Conference on Computer Vision and Pattern Recognition (CVPR), pages 8934–8943, 2018.
  * [27] Zachary Teed and Jia Deng.  Raft: Recurrent all-pairs field transforms for optical flow.  In European Conference on Computer Vision (ECCV), pages 402–419. Springer, 2020.
  * [28] Suren Vagharshakyan, Robert Bregovic, and Atanas Gotchev.  Light field reconstruction using shearlet transform.  IEEE Transactions on Pattern Analysis and Machine Intelligence, 40(1):133–147, 2017.
  * [29] Vaish Vaibhav and Adams Andrew.  The (new) stanford light field archive.  http://lightfield.stanford.edu/acq.html.
  * [30] Tingchun Wang, Alexei A. Efros, and Ravi Ramamoorthi.  Depth estimation with occlusion modeling using light-field cameras.  IEEE Transactions on Pattern Analysis and Machine Intelligence, 38(11):2170–2181, 2016.
  * [31] Yunlong Wang, Fei Liu, Zilei Wang, Guangqi Hou, Zhenan Sun, and Tieniu Tan.  End-to-end view synthesis for light field imaging with pseudo 4dcnn.  In European Conference on Computer Vision (ECCV), pages 333–348, 2018.
  * [32] Sven Wanner and Bastian Goldluecke.  Variational light field analysis for disparity estimation and super-resolution.  IEEE Transactions on Pattern Analysis and Machine Intelligence, 36(3):606–619, 2013.
  * [33] Bennett Wilburn, Neel Joshi, Vaibhav Vaish, Eino-Ville Talvala, Emilio Antunez, Adam Barth, Andrew Adams, Mark Horowitz, and Marc Levoy.  High performance imaging using large camera arrays.  In ACM Transactions on Graphics, pages 765–776. 2005.
  * [34] Gaochang Wu, Yebin Liu, Qionghai Dai, and Tianyou Chai.  Learning sheared epi structure for light field reconstruction.  IEEE Transactions on Image Processing, 28(7):3261–3273, 2019.
  * [35] Gaochang Wu, Mandan Zhao, Liangyong Wang, Qionghai Dai, Tianyou Chai, and Yebin Liu.  Light field reconstruction using deep convolutional network on epi.  In IEEE Conference on Computer Vision and Pattern Recognition (CVPR), pages 6319–6327. IEEE, 2017.
  * [36] Henry Wing Fung Yeung, Junhui Hou, Jie Chen, Yuk Ying Chung, and Xiaoming Chen.  Fast light field reconstruction with deep coarse-to-fine modeling of spatial-angular clues.  In European Conference on Computer Vision (ECCV), pages 137–152, 2018.
  * [37] Youngjin Yoon, Hae-Gon Jeon, Donggeun Yoo, Joon-Young Lee, and In So Kweon.  Learning a deep convolutional network for light-field image super-resolution.  In IEEE International Conference on Computer Vision Workshops (ICCVW), pages 24–32. IEEE, 2015.
  * [38] Jingyi Yu.  A light-field journey to virtual reality.  IEEE MultiMedia, 24(2):104–112, 2017.
  * [39] Zhoutong Zhang, Yebin Liu, and Qionghai Dai.  Light field from micro-baseline image pair.  In IEEE Conference on Computer Vision and Pattern Recognition (CVPR), pages 3800–3809. IEEE, 2015.
  * [40] Tinghui Zhou, Richard Tucker, John Flynn, Graham Fyffe, and Noah Snavely.  Stereo magnification: learning view synthesis using multiplane images.  ACM Transactions on Graphics, 37(4):1–12, 2018.