1

Prime and Modulate Learning: Generation of forward models with signed back-
propagation and environmental cues

Sama Daryanavard${}^{\displaystyle 1},$ Bernd Porr1
1Biomedical Engineering Division, School of Engineering, University of
Glasgow, Glasgow G12 8QQ, UK.

Abstract

Deep neural networks employing error back-propagation for learning can suffer
from exploding and vanishing gradient problems. Numerous solutions have been
proposed such as normalisation techniques or limiting activation functions to
linear rectifying units. In this work we follow a different approach which is
particularly applicable to closed-loop learning of forward models where back-
propagation makes exclusive use of the sign of the error signal to prime the
learning, whilst a global relevance signal modulates the rate of learning.
This is inspired by the interaction between local plasticity and a global
neuromodulation. For example, whilst driving on an empty road, one can allow
for slow step-wise optimisation of actions, whereas, at a busy junction, an
error must be corrected at once. Hence, the error is the priming signal and
the intensity of the experience is a modulating factor in the weight change.
The advantages of this Prime and Modulate paradigm is twofold: it is free from
normalisation and it makes use of relevant cues from the environment to enrich
the learning. We present a mathematical derivation of the learning rule in
z-space and demonstrate the real-time performance with a robotic platform. The
results show a significant improvement in the speed of convergence compared to
that of the conventional back-propagation.

##  1 Introduction

Since its inception, deep learning has proven remarkably successful in a wide
variety of areas, such as: image classification, speech recognition, and
reinforcement learning Bahri et al. (2020). Deep Neural Networks (DNNs) employ
activation functions, such as the logistic or the hyperbolic tangent
($\tanh$), and are most commonly trained using an error signal with the
Gradient Descent Method (GDM) for optimisation. The derivative of such
functions have a narrow range of practical values and a limit of zero outwith,
therefore the propagation of the error signal through this non-linearity often
suffers from the exploding and vanishing gradient problem (EVGP). This has
ignited a significant research effort into addressing this issue; amongst the
solutions are the network architectures such as long short-term memory (LSTM),
precise weight initialisations, and specific non-linear activation functions
(Hanin, 2018). Notably, linear rectifying units are used to remedy this
problem which effectively remove the derivative and thus, drastically alter
the behaviour of the network and the nature of learning. On the other hand,
the logistic function allows for a smooth saturation of signals, along with
$\tanh$. This is in accordance with neuroscience where nearly all psychometric
functions or internal neuronal processing follow a sigmoid activation pattern.

From a neurophysiological standpoint, learning is driven by local and global
mechanisms changing synaptic plasticity Reynolds and Wickens (2002). In
particular, in closed-loop learning an interplay of local learning and global
learning which has been advantageous, for example, improving the stability of
learning when generating a forward model of a reflex Porr and Wörgötter
(2007). Nonetheless, to date, this class of closed-loop learning has only been
used in shallow networks, often with a single learning units or shallow
networks (Porr and Wörgötter, 2007; Kulvicius et al., 2007; Maffei et al.,
2017).

In this work, we present a learning paradigm that combines local error back-
propagation and global modulation to create a robust learning scheme for the
generation of forward models. More specifically, only the sign of the
propagating signal is used to prime the nature of weight change in the context
of their local connections, whilst a global “relevance” signal acts as a third
factor to excite the weight changes across the network. This offers not only a
robust, nearly one-shot real-time learning, free of exploding and vanishing
gradient problem (EVGP), but also a more comparable learning model to
neuroscience than the conventional back-propagation. In this paper, we
implement this novel algorithm on a physical robot that learns to improve the
performance of a closed-loop feedback controller (a reflex) by calculating its
forward model, and thus prevents the triggering of the controller.

##  2 The closed-loop learning platform

### The reflex and predictive loops:

Figure 1 shows the learning platform. The so called “reflex” is a reactive
closed loop controller which aims to stay as close to its desired state
$I_{d}$ as possible. This fixed reflex controller acts against the disturbance
$D$ which travels through the environmental transfer function $R_{E}$. This
leads to a new state $S$ that is picked up by $R_{S}$ and causes a sensor
signal $I$. This signal is compared to the desired state $I_{d}$ at node
\small{1}⃝ and in turn creates the error signal $E$. This error signal is
translated into an appropriate action $A$ via the motor transfer function
$R_{M}$ to counteract the disturbance $D$ at node \small{2}⃝. Here, the
crucial aspect of the error signal $E$ is that it can be used for realtime
learning that enables the learner to generate a forward model of the reflex
loop.

Figure 1: The Prime and Modulate (PaM) closed-loop learning platform
consisting of an inner reflex loop (highlighted in blue) and an outer learning
loop (highlighted in green). The signal and transfer function notations are as
follow: Disturbance (D), delay function ($z^{-T}$), reflex environment
($R_{E}$), state of reflex ($S$), reflex sensory unit ($R_{S}$), reflex
sensory input ($I$), desired sensory input ($I_{d}$), error signal ($E$),
reflex motor unit ($R_{M}$), reflex motor action ($A$), learner environment
($L_{E}$), state of learner ($S^{\prime}$), learner sensory unit ($L_{S}$),
leaner sensory input ($I^{\prime}$), learner (PaM network), learner motor
action ($A^{\prime}$). The priming pathway is highlighted with red where
$F_{P}$ is the priming factor that propagates through the network. Modulating
pathway is highlighted with yellow, where, reflex cue function ($R_{C}$) and
learner cue function ($L_{C}$) lead to the modulating factor ($F_{M}$) and
drives the learning.

Learning of the forward model is performed by the novel PaM network placed in
the learning (predictive) loop. The disturbance travels through the learner’s
environment $L_{E}$, leading to a new state $S^{\prime}$ which is picked up by
learner’s sensory unit $L_{S}$ and causes a predictive sensory signal
$I^{\prime}$. This is fed into the learning algorithm which in turn generates
the output $P$. This is translated into the predictive action $A^{\prime}$,
executed by the learner’s motor unit $L_{M}$, to eliminate the Disturbance $D$
at node \small{3}⃝, before it can enter the reflex loop.

### z-space:

The signals in Figure 1 are discrete time and real-valued physical
measurements, therefore are more accurately referred to as sequences. Due to
the recursive nature of closed-loop systems it is beneficial to analyse their
behaviour in z-space where discrete time-domain sequences are transformed into
complex frequency-domain representations (Oppenheim, 1999). We use the
unilateral z-transform representation of these sequences in this work, which,
for an arbitrary sequence $x[n]$, is defined as:

$\displaystyle X(z)=\mathcal{Z}\\{x[n]\\}=\Sigma_{n=0}^{\infty}(x[n]z^{-n})$
(1)

Where $\mathcal{Z}\\{.\\}$ is z-transform operator and $z$ is a complex
variable.

In this work we harness three properties of the z-transform: linearity, time
shifting, and convolution of sequences (Oppenheim, 1999). This converts the
recursive nature of the derivations into simple algebraic operations, for
example considering the reflex loop and assuming that $D$ and $A^{\prime}$ are
zero, solving for the error signal yields:

time-domaine[n] $\displaystyle=i_{d}(t)-r_{S}(r_{E}(r_{M}(e[n-1])))$ (2)
z-spaceE(z) $\displaystyle=I_{d}(z)-R_{S}R_{E}R_{M}\cdot
z^{-1}E(z)=\frac{I_{d}}{1+R_{S}R_{E}R_{M}z^{-1}}$ (3)

This shows that the error signal is function in time-domain, whereas, with
z-transformation this translates into multiplication of transfer functions and
sequences which allows for expression of $E(z)$ to be derived. In this work
the $(z)$ symbol, representing the complex variable, is omitted for brevity.

### The prime and modulate pathways:

As described in the introduction, only the sign of the error signal $E$ is
used to generate the priming factor ($F_{P}$) for the back-propagation (BP),
while the global modulating factor ($F_{M}$) acts as a third factor where a
filtered version of the sensor inputs is used as a novelty or relevance signal
which modulates the learning with its amplitude. This is in line with the
claim that in particular dorsal striatal dopamine is more of a novelty or
salience detector Prescott et al. (2006) than an error signal, or serotonin
being a rectified version of the reward prediction error Li et al. (2016).
These signals and their functionalities are described in more details in the
future sections.

### The learning goal:

The aim of the learner is to produce a predictive signal $P$ such that the
error signal $E$ is kept at zero persistently. In mathematical terms, this is
analogous to minimising the absolute value of the error $|E|$; since this is a
non differentiable function at $E=0$ the quadratic of $E$ is minimised
instead:

$\displaystyle\textit{Learning goal: }\quad P=\underset{p}{arg\,min}\textit{
}E^{2}$ (4)

This is achieved by adjusting the internal parameters of the network that are
introduced below.

### The neural network:

This paradigm employs a feed-forward neural network with fully connected
layers. Figure 2 shows the internal connections of two neurons in this
network.

Figure 2: Neuron connections in PaM network. Forward propagation of inputs in
shown with the left-to-right solid lines highlighted in blue. $\sigma$ is the
sigmoid activation function and $A_{j}^{\ell}$ denotes the activation of
$j^{th}$ neuron in layer $\ell$. $[\omega]^{\ell}_{ij}$ is the weight matrix
associated with inputs $I$ inputs to this layer. The summation node \small{a}⃝
corresponds to Equation 5. Backpropagation pathway is shown with right-to-left
dashed lines highlighted in green. The summation at node \small{b}⃝ and
product at node \small{c}⃝ correspond to Equation 10. The priming pathway is
shown with short dashed lines highlighted in red. This is the sign of the
resulting value from the backpropagation pathway, see Equation 11. The
modulating pathway is shown in long dashed lines that enter each neuron from
the environment, see Equation 12. The priming and modulating factors join at
node \small{d}⃝, together with the learning rate $\eta$ and the relevant input
to the neuron $A_{i}^{\ell-1}$, to drive the learning rule, corresponding to
Equation 3.

The forward propagation of activations $A$ is shown with solid right-arrows
and is calculated as:111The superscripts denote the layer index, the first
subscripts denote the neuron index, and the second subscripts denote the input
(or the associated weight) index.

$\displaystyle A_{j}^{\ell}$
$\displaystyle=\sigma(v_{j}^{\ell})=\sigma(\Sigma_{i=1}^{I}\omega^{\ell}_{ij}\sigma(v_{i}^{\ell-1}))=\sigma(\Sigma_{i=1}^{I}\omega^{\ell}_{ij}A^{\ell-1}_{i})\textit{
Where: }\ell=1\rightarrow L$ (5)

Where $\omega$ denotes the weights, $v$ is referred to as the sum output, and
the $\sigma$ is the logistic function that maps the sum output of the neuron
$v$ to the activation. The weighted sum takes place at summation points
\small{a}⃝. Note that for $\ell=1$, the predictive input information
substitute for the input activations: $A^{0}_{i}=I^{\prime}_{i}$ and, for
$\ell=L$, the weighted sum of activations in the last layer yields the
predictive signal in the closed-loop platform:
$P=\Sigma_{x=0}^{X}(g_{x}A_{x}^{L})$, where $g_{x}$ is a weighting constant.

##  3 Derivation of the learning rule

The learning goal in Equation 4 is achieved through adjustments of weights
$\omega$ in Equation 5. At each iteration, the error signal $E$ provides the
network with constructive feedback on the adequacy of the predictive signal
$P$. Conventionally, the GDM is employed for weight optimisation; where the
change to an arbitrary weight is proportional to the sensitivity of $E^{2}$
with respect to the sum output $v$ of the neuron containing that weight:

$\displaystyle\textit{Gradient descent
method:}\quad\Delta\omega_{ij}^{\ell}\propto\frac{\partial E^{2}}{\partial
v_{j}^{\ell}}$ (6)

When differentiating in z-space we assume that the weight changes in time-
domain are constant or significantly slower than the changes in the closed-
loop system. (reference???). We seek an expression of this gradient in the
context of closed-loop applications. With a similar approach to Daryanavard
and Porr (2020), this gradient is unravelled using the chain rule:

$\displaystyle\frac{\partial E^{2}}{\partial
v_{j}^{\ell}}=\overbrace{\frac{\partial E^{2}}{\partial P}}^{Closed-
loop}\cdot\overbrace{\frac{\partial P}{\partial v_{j}^{\ell}}}^{Network}$ (7)

The former partial derivative solely relates to the dynamics of the closed-
loop platform, whilst, the latter partial derivative relates to the inner
connections of the network.

### The closed-loop gradient ($\frac{\partial E^{2}}{\partial P}$):

Referring to Figure 1 (summation points \small{1}⃝ and \small{2}⃝), the
closed-loop expression of the error signal in z-space is derived as:

E
$\displaystyle=I_{d}-I=I_{d}-R_{S}R_{E}(\overbrace{\textit{{E}}R_{M}}^{=A}+\overbrace{PL_{M}}^{=A^{\prime}}+Dz^{-T})$
$\displaystyle=\frac{I_{d}-R_{S}R_{E}(PL_{M}+Dz^{-T})}{1+R_{S}R_{E}R_{M}}$ (8)

Therefore, differentiation of quadratic of $E$ with respect to $P$ yields:

$\displaystyle\textit{Closed-loop gradient: }\quad\frac{\partial
E^{2}}{\partial P}=2E\frac{\partial E}{\partial
P}=2E\frac{-R_{S}R_{E}L_{M}}{1+R_{S}R_{E}R_{M}}$ (9)

The value of the resulting fraction can be found experimentally by
substituting $I_{d}=0$, $D=0$, and $P=1$ in Equation 3 and measuring $E$; this
is the closed loop gain of the system.

### The network gradient ($\frac{\partial P}{\partial v}$):

From Equation 5, differentiating with respect to an arbitrary sum output
$v^{\ell}_{j}$ results in a recursive expression:

$\displaystyle\textit{Network gradient:}\quad\frac{\partial P}{\partial
v^{\ell}_{j}}=\sigma^{-1}(v^{\ell}_{j})\cdot\Sigma_{k=0}^{K}(\omega^{\ell+1}_{jk}\frac{\partial
P}{\partial v^{\ell+1}_{k}})$ (10)

Where $\sigma^{-1}$ is the inverse logistic function. For the neurons in the
final layer we have: $P=A^{L}_{x}$, therefore, this gradient is simply
calculated as: $\sigma^{-1}(v^{L}_{x})$. This is fed through the network using
the back-propagation technique. This chain of events is shown by dashed left-
arrows in Figure 2, with the weighted sum and the product indicated at points
\small{b}⃝ and \small{c}⃝, respectively.

With that, we have an expression of the gradient found in Equation 6 which can
be used for closed-loop learning applications. However, depending on the
distribution of the weights and the topology of the neural network, the
propagation can suffer from exploding and vanishing gradient problem (Pascanu
et al., 2013; Bengio et al., 1994, 1993). In the following section we derive
the PaM learning rule which is free from this issue.

###  Prime and Modulate learning

In this work merely the sign of the gradient $\frac{\partial E^{2}}{\partial
v}$, found in Equation 6, is used for learning. This serves to ’prime’ the
weights to later undergo an increase, a decrease, or remain unchanged. Hence,
it is referred to as the Priming Factor $F_{P}$ as seen in Figures 1 and 2:

$\displaystyle F_{P}=\delta(E)=\frac{\frac{\partial|E|^{2}}{\partial
v}}{|\frac{\partial|E|^{2}}{\partial v}|}=\begin{cases}+1&\text{{primes
$\omega$ to be increased}}\\\ 0&\text{{primes $\omega$ to remain
unchanged}}\\\ -1&\text{{primes $\omega$ to be decreased}}\end{cases}$ (11)

Once primed, the magnitude of weight change is dictated by a secondary signal
that contains collective cues gathered from the environment informing the
significance and/or relevance of the learning experience at any given instance
of time. $R_{C}$ and $L_{C}$, shown in Figure 1, are functions designed to
extract relevant cues from the reflex and predictive loops, respectively. The
correlation of these cues indicated at product point \small{4}⃝, modulates the
magnitude of weight changes. Hence, this signal is referred to as the
Modulating Factor $F_{M}$ (Figures 1 and 2):

$\displaystyle F_{M}=|ER_{C}\cdot I^{\prime}L_{C}|$ (12)

Finally, we redefine the weight change proportionality in Equation 6, and with
the introduction of the learning rate $\eta$, we establish the update rule for
this paradigm (refer to Equations 5, 7, and 9):

$\displaystyle\Delta\omega_{ij}^{\ell}$
$\displaystyle\propto\overbrace{\frac{2E\cdot\frac{-R_{S}R_{E}L_{M}}{1+R_{S}R_{E}R_{M}}\cdot\frac{\partial
P}{\partial
v^{\ell}_{i}}}{|2E\cdot\frac{-R_{S}R_{E}L_{M}}{1+R_{S}R_{E}R_{M}}\cdot\frac{\partial
P}{\partial v^{\ell}_{i}}|}}^{\textit{priming
factor}}\cdot\overbrace{|ER_{C}\cdot I^{\prime}L_{C}|}^{\textit{modulating
factor}}$ $\displaystyle\Delta\omega_{ij}^{\ell}$ $\displaystyle\coloneqq\eta
A^{\ell-1}_{i}{F_{P}}_{j}^{\ell}F_{M}$ (13)

As explained above, ${F_{P}}_{j}^{\ell}$ is back-propagated through the
layers, whilst, $F_{M}$ is available to all neurons in the network globally;
Figure 2 point \small{d}⃝ illustrates the weight changes. This finalises the
derivation of the learning rule for Prime and Modulate (PaM) paradigm.

##  4 Experimentation platform: robotic navigation

Figure 3 illustrates the experimental setup where a robot is placed on a
canvas with the task of following a path. The robot is fitted with a Raspberry
Pi 3B+ (RPi) that hosts the learning algorithm as an external C++ library. The
network is initialised with 10 hidden layers for the following set of
experiments.

Figure 3: Schematic of the experimental setup: A robot placed on a canvas
which navigates the path

The chassis houses an array of Light Dependent Resistors (LDRs), a camera, two
wheels with servo motors, and a battery bank to power the system. The
equivalent closed-loop symbol of these components, as introduced in Figure 1,
are shown here.

###  Light Dependent Resistors:

These sensors measure the Grey-Scale Values $I_{i}$ of the surface underneath
them and thus, monitor the position of the robot with respect to the path. The
desired state is a symmetrical stand, therefore, a nonsymmetrical alignment
results in the generation of a non-zero error signal as:

$\displaystyle
E=a(I_{1}-I_{6})+b(I_{2}-I_{5})+c(I_{3}-I_{4})\quad\textit{where: }a>b>c$ (14)

The weighting registers the degree of deviation and enables sharp, medium, or
gentle steering.

### Servo motors:

This error signal is used to recover the desired symmetrical state by altering
the speed of motors:

$\displaystyle
V_{left}=V_{0}-\overbrace{dE}^{Reflex}-\overbrace{eP}^{Network}$
$\displaystyle=V_{0}-\Delta V\quad$ (15) $\displaystyle\quad V_{right}$
$\displaystyle=V_{0}+\Delta V$

Where $V_{0}=7\textit{\footnotesize{$[\frac{cm}{s}]$}}$ and $d$ is an
experimental constant. This describes the functionality of the reflex loop in
this application.

### The camera

The view of the road ahead is captured by the camera in the form of a matrix
of GSVs $[I]^{\prime}_{8*12}$ and is fed into the neural network.

In addition to the error signal, the predictive output of the network $P$ is
used for steering as in Equation 15, where $e$ is a scaling factor found
experimentally. The error signal in Equation 14 is used to train the network.
Below, we derive the learning rule for this application.

### The priming factor:

The closed loop gain derived in Equation 9 is found to be $\approx 0.97$ for
this setup. Therefore, the Priming Factor for this learning is:
$F_{P}=1.94E\frac{\partial P}{\partial v}$. Where the error signal and the
partial derivative are calculated form Equations 14 and 10, respectively.

### The modulating factor:

When $E>0$, a positive change indicates further deviation whilst a negative
change implies recentering. In contrary, when $E<0$, a negative change
indicates further deviation from the path whilst a positive change implies
recentering. Therefore, the product of the two, $E\frac{\partial E}{\partial
t}$, signals the worsening or bettering of the performance when its value is
positive or negative, respectively. The exponential of this product is the cue
extracted from the reflex inputs $[I]_{6}$ signifying the learning at each
iteration: $E\cdot R_{C}=e^{E\frac{\partial E}{\partial t}}$

Thus the learning is weakly modulated when the navigation is improving and is
strongly modulated as the navigation worsens. The cue extracted from the
predictive inputs $[I]^{\prime}_{8*12}$ is the angle of deviation. The
vertical distance of the bottom row of this matrix to the sensors’ location
$v$ and the horizontal spread $h$ of each pixel are measured. By having the
index $q$ of the pixel that captures the path the angle of deviation is found
as: $I^{\prime}\cdot L_{C}=\arctan{\frac{h\cdot|6.5-q|}{v}}$

The modulating factor is defined as the correlation of the aforementioned
cues. Therefore, the update rule for this application is:

$\displaystyle\Delta\omega_{ij}^{\ell}$ $\displaystyle\coloneqq\eta
A^{\ell-1}_{i}\cdot 1.94E\frac{\partial P}{\partial v_{j}^{\ell}}\cdot
e^{E\frac{\partial E}{\partial t}}\cdot\arctan{\frac{h\cdot|6.5-q|}{v}}$ (16)

##  5 Results: comparison of GDM and PaM

The performance of PaM paradigm is compared to that of the closed-loop GDM
bench-marked in Daryanavard and Porr (2020). Successful learning is defined as
the state where the error’s moving average over $10$ seconds,
$0.1\int_{t-10}^{t}|E|dt$, falls below $1\%$ of its maximum value during the
trial.

Figure 4: A pair of comparative learning trials with learning rate of
$\eta=e^{-5}$

The dashed traces in Figure 4A and B show the error signal $E$ and its moving
average during a GDM trial with the learning rate222This is the natural number
$e=2.71828$ of $\eta=e^{-5}$; the success condition is reached at time
$t=266.3[s]$. The black traces in these figures show the error signal $E$ and
its moving average during a trial with PaM; the success condition is achieved
at time $t=23.1[s]$. The modulating factor $F_{M}$ for this trial is shown in
figure 4C. This pair of trials shows a significant improvement in the speed of
learning and navigational performance of the robot.

Figure 5: A pair of comparative learning trials with learning rate of
$\eta=e^{-1}$

A second pair of trials with higher learning rate of $\eta=e^{-1}$ is shown in
figure 5; this shows a close to one-shot learning performance for PaM. Such
comparative trial pairs were repeated $50$ times across learning rates of:
$\eta=[e^{-5},e^{-4},e^{-3},e^{-2},e^{-1}]$.

Figure 6: comparison of learning speed and total error over a range of 5
different learning rates

Figure 6A shows the time taken for the robot to obtain the success state
during the trials with PaM (black trace) compared to that of the GDM (dashed
trace). This shows consistency in the trend observed in Figures 4 and 5, in
that, the PaM paradigm is significantly faster than its GDM counterpart. Both
methods demonstrate a faster learning with higher learning rates, however, the
performance of GDM is influenced by changes in the learning rate to a greater
degree, given by the deflection of the fitted curve. Figure 6B show the
corresponding error integrals, $\int_{0}^{t}|E|dt$, for these trials. As
anticipated, the accumulation of the error signal is greater in trials with
slower learning rate. Although the total accumulation of error is
significantly smaller in PaM trials, in both methods this parameter is
influenced by the learning rates to the same degree; inferred from the slope
of the fitted curves.

##  6 Discussion

In this work we presented a novel closed loop algorithm which learns the
forward model of a reflex Porr and Wörgötter (2003). These models play an
important role in robotic and biological motor control (Wolpert and Kawato,
1998; Wolpert et al., 2001; Haruno et al., 2001; Nakanishi and Schaal, 2004)
where they guarantee, for example, an optimal trajectory. Previous work in
this area used shallow networks (Kulvicius et al., 2007), filter banks Porr
and Wörgötter (2003) or single layers to perform predictive control (Nakanishi
and Schaal, 2004; Maffei et al., 2017) and it was not possible to employ
deeper structures. On the other hand, model free closed-loop learning has been
using more complex network structures such as deep learning in combination
with Q-learning (Guo et al., 2014; Bansal et al., 2016). Here, we demonstrate
that model based closed-loop learning can also benefit from deep learning and
thus a combination of both is more powerful (Botvinick et al., 2019). However,
given the fast learning of this model it can be prone to converging to a local
minima. Whether or not deep learning is biologically realistic has been
debated for many years where the main issue is the requirement of weight
symmetry for forward- and backward-pass which limits its plausibility to only
a few layers Lillicrap et al. (2016). However, if the error is merely
transmitted as a sign, this weight symmetry can be relaxed as long as there
are interconnections between the top/down and bottom/up pathways guaranteeing
the correct sign of the learning Larkum (2013). In the context of
neuroscience, this means that the bottom up pathway just controls long term
potentiation (LTP) or long term depression (LTD) while neuromodulators, in
particular serotonin as a rectified reward prediction error, can control the
speed of the learning Iigaya et al. (2018) as a third factor (Li et al.,
2016). Thus, in particular for cortical processing where serotonin is more
prominent than dopamine and deep neuronal structures exist, a combination of
local and global learning is a compelling fit for neuroscience, in addition to
its application in machine learning and robotic navigation.

## Acknowledgements

We would like to acknowledge Jarez Patel for his valuable intellectual and
technical input for the making of the robotic platform.

## References

  * Bahri et al. (2020) Bahri, Y., Kadmon, J., Pennington, J., Schoenholz, S. S., Sohl-Dickstein, J., and Ganguli, S. (2020).  Statistical mechanics of deep learning.  Annual Review of Condensed Matter Physics.
  * Bansal et al. (2016) Bansal, S., Akametalu, A. K., Jiang, F. J., Laine, F., and Tomlin, C. J. (2016).  Learning quadrotor dynamics using neural network for flight control.  In 2016 IEEE 55th Conference on Decision and Control (CDC), pages 4653–4660. IEEE.
  * Bengio et al. (1993) Bengio, Y., Frasconi, P., and Simard, P. (1993).  The problem of learning long-term dependencies in recurrent networks.  In IEEE international conference on neural networks, pages 1183–1188. IEEE.
  * Bengio et al. (1994) Bengio, Y., Simard, P., and Frasconi, P. (1994).  Learning long-term dependencies with gradient descent is difficult.  IEEE transactions on neural networks, 5(2):157–166.
  * Botvinick et al. (2019) Botvinick, M., Ritter, S., Wang, J. X., Kurth-Nelson, Z., Blundell, C., and Hassabis, D. (2019).  Reinforcement learning, fast and slow.  Trends in cognitive sciences.
  * Daryanavard and Porr (2020) Daryanavard, S. and Porr, B. (2020).  Closed-loop deep learning: Generating forward models with backpropagation.  Neural Computation, 32(11):2122–2144.
  * Guo et al. (2014) Guo, X., Singh, S., Lee, H., Lewis, R. L., and Wang, X. (2014).  Deep learning for real-time atari game play using offline monte-carlo tree search planning.  In Ghahramani, Z., Welling, M., Cortes, C., Lawrence, N. D., and Weinberger, K. Q., editors, Advances in Neural Information Processing Systems 27, pages 3338–3346. Curran Associates, Inc.
  * Hanin (2018) Hanin, B. (2018).  Which neural net architectures give rise to exploding and vanishing gradients?  arXiv preprint arXiv:1801.03744.
  * Haruno et al. (2001) Haruno, M., Wolpert, D. M., and Kawato, M. (2001).  Mosaic model for sensorimotor learning and control.  Neural computation, 13(10):2201–2220.
  * Iigaya et al. (2018) Iigaya, K., Fonseca, M. S., Murakami, M., Mainen, Z. F., and Dayan, P. (2018).  An effect of serotonergic stimulation on learning rates for rewards apparent after long intertrial intervals.  Nature Communications, 9(1):2477.
  * Kulvicius et al. (2007) Kulvicius, T., Porr, B., and Wörgötter, F. (2007).  Chained learning architectures in a simple closed-loop behavioural context.  Biological Cybernetics, 97(5-6):363–378.
  * Larkum (2013) Larkum, M. (2013).  A cellular mechanism for cortical associations: an organizing principle for the cerebral cortex.  Trends in neurosciences, 36(3):141–151.
  * Li et al. (2016) Li, Y., Zhong, W., Wang, D., Feng, Q., Liu, Z., Zhou, J., Jia, C., Hu, F., Zeng, J., Guo, Q., Fu, L., and Luo, M. (2016).  Serotonin neurons in the dorsal raphe nucleus encode reward signals.  Nature Communications, 7(1):10503.
  * Lillicrap et al. (2016) Lillicrap, T. P., Cownden, D., Tweed, D. B., and Akerman, C. J. (2016).  Random synaptic feedback weights support error backpropagation for deep learning.  Nature communications, 7:13276.
  * Maffei et al. (2017) Maffei, G., Herreros, I., Sanchez-Fibla, M., Friston, K. J., and Verschure, P. F. (2017).  The perceptual shaping of anticipatory actions.  Proceedings of the Royal Society B: Biological Sciences, 284(1869):20171780.
  * Nakanishi and Schaal (2004) Nakanishi, J. and Schaal, S. (2004).  Feedback error learning and nonlinear adaptive control.  Neural Networks, 17(10):1453–1465.
  * Oppenheim (1999) Oppenheim, A. V. (1999).  Discrete-time signal processing.  Pearson Education India.
  * Pascanu et al. (2013) Pascanu, R., Mikolov, T., and Bengio, Y. (2013).  On the difficulty of training recurrent neural networks.  In International conference on machine learning, pages 1310–1318.
  * Porr and Wörgötter (2003) Porr, B. and Wörgötter, F. (2003).  Learning a forward model of a reflex.  In Becker, S., Thrun, S., and Obermayer, K., editors, Advances in Neural Information Processing Systems, volume 15. MIT Press.
  * Porr and Wörgötter (2007) Porr, B. and Wörgötter, F. (2007).  Learning with “Relevance”: Using a Third Factor to Stabilize Hebbian Learning.  Neural Computation, 19(10):2694–2719.
  * Prescott et al. (2006) Prescott, T. J., González, F. M. M., Gurney, K., Humpries, M. D., and Redgrave, P. (2006).  A robot model of the basal ganglia: Behaviour and intrinsic processing.  Neural Networks, In Press.
  * Reynolds and Wickens (2002) Reynolds, J. N. and Wickens, J. R. (2002).  Dopamine dependent plasticity of corticostriatal synapses.  Neural Networks, 15:507–521.
  * Riedmiller and Braun (1993) Riedmiller, M. and Braun, H. (1993).  A direct adaptive method for faster backpropagation learning: The rprop algorithm.  In IEEE international conference on neural networks, pages 586–591. IEEE.
  * Wolpert et al. (2001) Wolpert, D. M., Ghahramani, Z., and Flanagan, J. R. (2001).  Perspectives and problems in motor learning.  Trends in cognitive sciences, 5(11):487–494.
  * Wolpert and Kawato (1998) Wolpert, D. M. and Kawato, M. (1998).  Multiple paired forward and inverse models for motor control.  Neural networks, 11(7-8):1317–1329.

  *[DNNs]: Deep Neural Network
  *[GDM]: Gradient Descent Method
  *[EVGP]: exploding and vanishing gradient problem
  *[LSTM]: long short-term memory
  *[PaM]: Prime and Modulate
  *[BP]: back-propagation
  *[RPi]: Raspberry Pi 3B+
  *[LDRs]: Light Dependent Resistor
  *[GSVs]: Grey-Scale Value


# FCDSN-DC: An Accurate and Lightweight Convolutional Neural Network for
Stereo Estimation with Depth Completion

Dominik Hirner  Graz University of Technology, Austria
Institute for Computer Graphics and Vision
Email<EMAIL_ADDRESS>Friedrich Fraundorfer  Graz University of
Technology, Austria
Institute for Computer Graphics and Vision
Email<EMAIL_ADDRESS>
###### Abstract

We propose an accurate and lightweight convolutional neural network for stereo
estimation with depth completion. We name this method fully-convolutional
deformable similarity network with depth completion (FCDSN-DC). This method
extends FC-DCNN by improving the feature extractor, adding a network structure
for training highly accurate similarity functions and a network structure for
filling inconsistent disparity estimates. The whole method consists of three
parts. The first part consists of fully-convolutional densely connected layers
that computes expressive features of rectified image pairs. The second part of
our network learns highly accurate similarity functions between this learned
features. It consists of densely-connected convolution layers with a
deformable convolution block at the end to further improve the accuracy of the
results. After this step an initial disparity map is created and the left-
right consistency check is performed in order to remove inconsistent points.
The last part of the network then uses this input together with the
corresponding left RGB image in order to train a network that fills in the
missing measurements. Consistent depth estimations are gathered around invalid
points and are parsed together with the RGB points into a shallow CNN network
structure in order to recover the missing values. We evaluate our method on
challenging real world indoor and outdoor scenes, in particular Middlebury,
KITTI and ETH3D were it produces competitive results. We furthermore show that
this method generalizes well and is well suited for many applications without
the need of further training. The code of our full framework is available at:
https://github.com/thedodo/FCDSN-DC

Figure 1: FCDSN-DC network structure. The Feature Extraction part learns
expressive, deep features for the left and right image. Afterwards the
Similarity Function part takes the concatenated input of the feature extractor
and learns accurate similarity functions. Then the LR check is performed in
order to get rid of inconsistent points. The Depth Completion part then takes
the incomplete disparity map together with the corresponding RGB image in
order to fill the missing disparity measurements.

##  I INTRODUCTION

Stereo vision has been a core problem of computer vision for many years. In
stereo vision a pair of rectified images of the same scene but with different
camera positions is used in order to extract 3D information. The retrieval of
3D information in such a manner is used in many important applications such as
robotics, autonomous driving and 3D scene reconstruction. A traditional stereo
method consists of four steps, namely: feature extraction, matching cost
calculation, disparity estimation and disparity refinement. In the past all of
this steps have been done using hand-crafted features and functions, however
recent publications have shown real improvements upon this traditional
methods, like SGM [1] or MGM [2] by replacing one or several steps using deep
learning approaches.

The method FC-DCNN [3] by D. Hirner and F. Fraundorfer is used as a baseline
implementation for this method. There, it has been shown that replacing the
feature extraction step with a deep learning approach in order to learn
highly-dimensional and expressive features already outperforms traditional
stereo methods such as SGM. We chose this method because of the lightweight
and accurate foundation of the densely connected scheme that is easy to build
upon. In this work we extend upon this method in two ways: Adding a network
structure to learn a better similarity function between image patches and
adding a network structure that learns to complete the sparse disparity map
instead of using handcrafted post-processing steps. We end up with a fully
trainable method that outperforms the baseline method of FC-DCNN [3] in all
evaluated datasets and is comparable with other state-of-the art deep-learning
methods. The whole method split into every step is illustrated in Fig. 1. We
further show that this improvement in accuracy does not impact the generality
of the method. We achieve all of this without the need of costly
3D-convolutions or fully-connected layers that are used by many popular state-
of-the art methods such as GC-Net [4], PSMNet [5] or MC-CNN-acrt [6]. That
3D-convolutions are a major bottleneck for stereo estimation networks has been
shown by R. Rahim et al. in their recent work [41].

The feature extraction part of our network consists of a densely-connected
siamese CNN structure with shared weights. These highly-dimensional trained
features of the left and right image are then concatenated and passed to the
next part of the network in order to train a more accurate similarity
function. This part of our method consists of five densely-connected layers
with a deformable convolution block at the end in order to further improve the
results. The feature extraction and the similarity function are trained
jointly using a hinge-loss. These two trained parts are then used in order to
create a cost-volume by writing the similarity measurement for each possible
candidate at every possible image location along the predefined search
direction and stacking them along the third dimension. This will lead to a
cost-volume with the dimensions $H\times W\times D$, where $H$ and $W$ are the
spatial dimensions of the image and $D$ is the maximum search range. Then a
winner-takes-all approach is used by taking the argmax along the $D$ dimension
in order to get the final disparity estimation. The disparity map for the left
and the right image is created in such a manner and the left-right consistency
check [7] is performed in order to remove inconsistent points.

The last part of the network then fills these previously removed inconsistent
points. For each invalid point a set of consistent points is gathered along
predefined cardinal directions. This gathered information together with the
left RGB image is parsed into a shallow CNN network structure in order to
train a new disparity label at this location. Instead of directly training on
the integer ground-truth disparity we re-formulate the problem by choosing the
closest disparity in the gathered information and using its position in the
input vector as the class label.

This leads to a very lightweight network with a total of $0.385$ million
trainable parameters, which is only $15$ thousand more than the baseline
method FC-DCNN [3] and significantly less parameters than other machine
learning stereo methods, which often need millions of trainable parameters. In
summary, our contributions are as follows:

  * •

We improve upon the baseline method FC-DCNN by adding two new parts, leading
to a more accurate trainable method with two stages that outperforms the
baseline method in every evaluated dataset.

  * •

We introduce a novel yet simple machine learning method for depth completion.
This method learns new disparities for previously considered inconsistent
points with only the need of a shallow CNN structure. The output of this
method is a completely dense and more accurate disparity map.

  * •

We show that this method does not only outperform the baseline method on the
most challenging and well-known stereo vision benchmarks, namely the
Middlebury, Kitti and ETH3D benchmark, but can also compete with state-of-the
art methods well known in the field.

  * •

We show that our method is not only accurate for the trained datasets, but
also applicable for a wide array of different domains without the need of
retraining.

##  II Related Work

Our work is based on previous works on deep learning based stereo estimation
networks, disparity refinement and depth completion.

Learning based stereo estimation has lead to some major advances in the field
in recent years. J. Zbontar and Y. LeCun popularized the shared-weigths
siamese network structure for stereo estimation in their work named MC-CNN
[6]. In their work they extract small grayscale image patches from the left
and the corresponding image patches from the right image. As their learning
goal is to increase the distance of similarity between corresponding and non-
corresponding image patches, they furthermore extract non-corresponding
patches from the right image used for the training loss. J. Zbontar and Y.
LeCun created two different versions of their method called MC-CNN-fast and
MC-CNN-accurate. The first version uses the dot product as similarity function
while the latter trains it using fully-connected layers. Most state-of the art
learning based stereo methods use variations of the shared-weights siamese
network structure [3][4][5][6][10][11][12][13][14].

In D. Hirner and F. Fraundorfers work called FC-DCNN [3] the siamese network
structure was extended by using densely connected layers for the feature
extractor as well as building a novel handcrafted post-processing method. In
their paper they have shown that this hybrid method already works better than
traditional methods such as SGM [1] while remaining lightweight in terms of
the total number of trainable parameters.

H. Xu and J. Zhang introduced a network called AANet [13]. In this work they
get completely rid of the 3D convolutions to achieve faster inference speed
while maintaining comparable accuracy to other state-of the art methods. In
their method they first use a feature extractor on multiple scales to create
multiple cost-volumes on different scales. These cost-volumes are then passed
to Intra- and Cross-Scale Aggregation modules. These modules use deformable
convolutions [15][16] to get rid of edge-flattening issues. Afterwards the
resulting disparity maps at different resolutions are used for upsampling and
refinement.

Disparity refinement is used in order to further improve the disparity
prediction. In this step the often noisy and outlier-prone initial disparity
map is taken and improved via optimization. One of the most popular
traditional disparity refinement methods is semi-global matching (SGM) by H.
Hirschmueller [1]. In his paper he uses Mutual Information [17] as the
similarity function in order to get the initial noisy disparity map.
Afterwards the matching costs from all 16 cardinal direction for each pixel is
aggregated and used in order to update the disparity value. This is done by
viewing the aggregation of each direction separately as a 1D optimization
problem which is then combined to get the updated value. G. Facciolo et al.
[2] improved upon this method by using more evolved structures for the
matching cost aggregation. In his paper he shows that these evolved structures
can improve upon some artefacts of the belief update such as streaking
artefacts that are often present when using the SGM [1] method.

Depth completion is the process of taking a sparse disparity input and
assigning new values to the missing measurements. F. Aleotti et al. used
monocular cues in their work [18] called Monocular Completion Network (MCN) in
order to fill unreliable points. They argue that since monocular depth
estimation does not rely on matching, it does not suffer from occlusion
artefacts like traditional stereo methods. They leverage reliable disparity
points gotten from a traditional stereo method and train a monocular disparity
completion network on this. There exists a number of methods aiming to fill
sparse depth maps based on the output of LIDAR scanners or SLAM algorithms
[19][20][21][42][43]. These methods however are not directly comparable to our
work, as they use depth or point cloud data directly as their input and
therefore compete in different benchmarks.

##  III Network

The network consists of three sequentially dependent parts. First, rich and
deep features are trained, then a similarity function for these new features
is learned. In the last step, the left-right consistency check [7] is
performed in order to get rid of inconsistent depth predictions and a novel
trainable depth-completion task is performed to fill in this missing values.
In our experiments, the feature extraction and similarity estimation part is
trained jointly, while the depth-completion task is trained afterwards.

Our method differs from the baseline implementation FC-DCNN [3] in the
following ways:

  * •

The feature extractor remains the same, using densely connected fully-
convolutional layers as proposed by G. Huang et al. [8]. Our feature extractor
consists of four such densely connected convolutional layers training a 60
dimensional feature vector per image point. This is a reduction of one layer
in comparison to the baseline method FC-DCNN [3]. We found this network
configuration to work best through multiple empirical trial-and-error
evaluations. The same training scheme is used as described in the baseline
method implementation FC-DCNN [3], where the distance of the similarity score
between matching and close non-matching image patches of the left and right
image are increased by using a hinge-loss.

  * •

The handcrafted cosine similarity function is replaced by our similarity
function network. By fitting the similarity function on the data by training,
a better accuracy score can be achieved.

  * •

The handcrafted post-processing step to fill in the inconsistent points is
replaced by our novel shallow network for depth-completion.

Our whole method has been implemented using Python3, pytorch 1.2.0 [22] and
Cuda 10.0. Furthermore, we use the OpenCV 4.2.0 [23] library for image
manipulation. The feature extraction and similarity measurement part are
trained jointly using the Adam optimizer [24] with a learning rate of
$6.0\times 10^{-5}$, a batch-size of $100$ and a patch-size of $21$. The depth
completion part is trained separately, with the weights of the feature
extractor and similarity measurement network being frozen. We use Adam
optimizer [24] with a learning rate of $6.0\times 10^{-6}$ and the Cross-
entropy loss $CE$ for the training of the depth completion network as seen in
Eq. 1. For the depth-completion part a batch-size of $1000$ and a patch-size
of $7$ is used for training.

$CE=-\sum P(X)log(P(X))$ (1)

###  III-A Similarity Measurement

The goal of the similarity measurement network is to learn better matching
costs for the dataset than for example the cosine similarity or sum of
absolute difference/sum of squared difference (SAD/SSD) [9]. It consists of
five densely connected fully convolutional layers and one deformable
convolution layer at the end of the network.

In contrast to other popular methods we do not use fully connected layers or
3D-convolutions. Instead it uses a fully convolutional, densely connected
network structure which leads to a less complex yet accurate network
structure. Furthermore, this network structure allows for varying input image
size.

The network is trained jointly with the feature extraction network, getting as
input the concatenated trained features for $s_{+}=\\{p,q_{pos}\\}$ and
$s_{-}=\\{p,q_{neg}\\}$, where $p$ denotes the image point of the left image
and $q_{pos}$ and $q_{neg}$ denotes the correct and and incorrect match of the
right image respectively. Therefore the same hyperparameters, such as patch
size, batch size or optimizer are used in order to train both the feature
extraction as well as the similarity measurement part of the method. Both
similarities $s_{+}$ and $s_{-}$ are then used in each training step, using
Eq. 2 as loss.

$loss=max(0,0.2+s_{-}-s_{+}).$ (2)

###  III-B Depth Completion

In order to find new labels for the missing points we train a shallow CNN
network. To this end, finding new labels is defined as a classification
problem. However, instead of defining the integer disparities of the ground-
truth as the class labels and training on that directly, we instead re-
formulate the problem. In our method, this consistent disparity map with often
large holes of missing data is taken and for every point marked invalid, a set
number of valid points from the neighbourhood is gathered. The amount of valid
points gathered is a hyperparameter. We empirically found that 10 valid points
per invalid pixel lead to good results. The valid points are always gathered
along the same directions, the left and the right side of the inconsistent
point consecutively, however the first valid point along any given direction
could be further or closer depending on how many invalid points are next in
that direction. Afterwards the ground truth disparity value at that position
is taken and compared with the gathered information. The position of the
closest disparity within the range of $[-2,2]$ in the so created vector is
then taken and recorded as the class-label for the training task. This is
illustrated in the first example of Fig. 2. The first line, class, shows the
position of the input vector as the class label which will be used for
training. The second line, input, shows a dummy example of gathered valid
disparities for a given invalid point. The third line shows the true disparity
of this invalid point and the last line shows the found class label used for
training. The input vector is then searched for the occurrence of this true
disparity. If no value is found within this range of the ground-truth, the
point is discarded for the training process.

If the found class-label has multiple entries in the vector, the first
occurrence is taken as the label. This is shown in the second example of Fig.
2. This however has the drawback, that lower classes are favoured and
therefore are more likely to appear. To counter this class imbalance that can
occur, the class weights are normalized previous to training.

Figure 2: First: Example on how the ground-truth label for the depth-
completion network is created given the collected valid disparities and the
corresponding known disparity of the training dataset.
Second: Example on how the ground-truth label for the depth-completion network
is created if the input vector has multiple valid entries.

A patch of the so created vectors together with the patch of RGB values of the
left image at the same position is then taken as input for a shallow CNN. The
network consists of three convolution layers and a softmax output layer.

This method has two main advantages over training the integer disparity labels
directly: One, it strongly limits the number of classes needed for the
classification task therefore also strongly limiting the needed resources and
complexity of the network. Two, it helps with generalization as the disparity
range of the data does not effect the input data, as only the position of the
closest point in the neighbourhood is learned. One drawback of this method is
that the data preparation for training and inference is more time-consuming as
training directly on integer labels.

##  IV Experiments

We test our whole framework on a number of challenging real life indoor and
outdoor scenes and compare our methods to the results of other state-of-the
art methods from the official benchmarks with similar scores. For our overall
ranking and further comparisons with more state-of-the art methods the online
benchmarks can be visited.

###  IV-A Middlebury

The Middlebury stereo dataset [26] is a challenging indoor dataset with dense
and highly accurate subpixel ground-truth data and an online leaderboard. All
our experiments were done using the half (H) resolution.

Table I shows that we more than doubled the accuracy of the baseline method
FC-DCNN on the training dataset in regards to the $4-PE$ and the $2-PE$ and
strongly improved the $1-PE$. Furthermore we compare our results to other
popular methods such as MC-CNN-acrt [6] or HSM-Net [29] and show that our
method performs either better or is on-par with their results. The last row of
Tab. I shows our results on the 13 additional samples of the 2014 Middlebury
dataset that were not used in the training process.

TABLE I: Accuracy comparison on the Middlebury training dataset Method | 4-PE | 2-PE | 1-PE | 0.5 PE
---|---|---|---|---
|  | Train |  |
FCDSN-DC (ours) | 5.08 | 9.47 | 26.6 | 60.6
LBPS [28] | 4.97 | 9.63 | 21.2 | 51.5
MC-CNN-acrt [6] | 6.34 | 10.1 | 18.4 | 39.8
HSM-Net_RVC [29] | 4.52 | 10.2 | 22.8 | 49.2
FC-DCNN (baseline) [3] | 12.3 | 17.9 | 34.7 | 65.1
|  | Test |  |
FCDSN-DC (ours) | 10.2 | 13.0 | 19.7 | 39.9

###  IV-B KITTI

KITTI stereo [32] is an outdoor street image dataset created for autonomous
driving. There are two different KITTI datasets, namely KITTI2012 and
KITTI2015 captured in different years which can be viewed identical for the
stereo estimation task.

Table II shows that we improved the accuracy of the baseline method with
exception of the $2-PE$ for the KITTI2012 dataset. Although the method was
optimized with the Middlebury dataset in mind, the method produces reasonable
results that are on-par with, or better than other recently released learning
based stereo methods and widely used non-learning methods such as the SGM
implementation of OpenCV [23].

TABLE II: Accuracy comparison on the KITTI Testing dataset Method | 5-PE | 4-PE | 3-PE | 2-PE
---|---|---|---|---
|  | KITTI2012 |  |
FCDSN-DC (ours) | 3.16 | 3.80 | 5.11 | 9.11
FC-DCNN (baseline) [3] | 3.71 | 4.40 | 5.61 | 8.81
OASM-Net [39] | 4.32 | 5.11 | 6.39 | 9.01
AAFS [34] | 3.28 | 4.28 | 6.10 | 10.64
HSMA [35] | 5.13 | 6.20 | 8.15 | 13.44
|  | KITTI2015 |  |
FCDSN-DC (ours) | - | - | 7.09 | -
PASMnet [40] | - | - | 7.23 | -
AAFS [34] | - | - | 7.54 | -
FC-DCNN (baseline) [3] | - | - | 7.71 | -
OASM-Net [39] | - | - | 8.98 | -

###  IV-C ETH3D

The ETH3D stereo dataset [33] consists of a wide range of different indoor as
well as outdoor scenes. Despite the fact, that this dataset is not the best
fit for our method, as it has a small baseline and therefore less integer-
valued disparities for training, Tab. III shows that we produce competitive
results, often outperforming or being on-par with well-known machine learning
based networks on the training dataset. The difference between the accuracy of
the train and test dataset can be explained by looking closer at the
individual samples. While the method works well for most test samples, a few
samples produce high errors. This, in fact, is not due to overfitting of the
method but rather these samples should be seen as failure cases for our
method. The failure and success cases can be viewed at the official benchmark
site of ETH3D. Despite that, Tab. III shows that our method outperforms the
baseline method in all categories except the $0.5-PE$.

TABLE III: Accuracy comparison on the ETH dataset Method | 4-PE | 2-PE | 1-PE | 0.5 PE
---|---|---|---|---
|  | Train |  |
FCDSN-DC (ours) | 0.45 | 0.70 | 1.58 | 11.37
HSM-Net_RVC [29] | 0.37 | 0.88 | 2.86 | 10.31
RAFT-Stereo [30] | 0.50 | 0.88 | 2.86 | 7.06
iResNet [31] | 0.09 | 1.17 | 4.14 | 12.61
FC-DCNN (baseline) [3] | 0.75 | 1.41 | 3.82 | 16.94
|  | Test |  |
FCDSN-DC (ours) | 2.66 | 5.04 | 10.24 | 25.59
HSM-Net_RVC [29] | 0.52 | 1.40 | 4.20 | 10.88
RAFT-Stereo [30] | 0.15 | 0.44 | 2.44 | 7.04
iResNet [31] | 0.25 | 1.00 | 3.68 | 10.26
FC-DCNN (baseline) [3] | 3.42 | 6.09 | 10.72 | 24.37

###  IV-D Ablation Study

In this section we show the validity and impact of our method by performing a
number of ablation experiments. For the sake of consistency, all the following
experiments are done using the same data, namely the Middlebury stereo
dataset. To show the generality of the method, 13 image pairs from the 2014
Middlebury dataset were omitted from the training process and are used as the
test split for all evaluations. The structure is as follows: First, we compare
the accuracy of our trained similarity with the accuracy of the handcrafted
cosine similarity cost. Next, we compare the accuracy of our trained
similarity function with or without the deformable convolution layer. Last, we
compare the accuracy of our method with and without the depth completion part.

###  IV-E Trained Similarity Function vs. Cosine

We show correctness and improvement of our trained similarity estimation
function by conducting the following experiment. As the feature extraction and
similarity estimation part are trained jointly, we train our feature extractor
from scratch for one day using the cosine similarity. Afterwards the feature
extraction and similarity estimation is trained jointly for the same amount of
time in order to ensure fairness and correctness of the comparison.

TABLE IV: Ablation study trained similarity function on the test data | 4-PE | 2-PE | 1-PE | 0.5-PE
---|---|---|---|---
|  | Train |  |
Cosine | 29.1795 | 32.946 | 39.251 | 57.328
Trained Similarity | 15.793 | 19.350 | 30.234 | 55.584
|  | Test |  |
Cosine | 28.984 | 31.499 | 36.885 | 53.005
Trained Similarity | 20.399 | 23.157 | 32.274 | 53.611

As Tab. IV shows, the accuracy of the method increases considerably when the
similarity function is trained as opposed to using a handcrafted function such
as cosine, except for the subpixel error on the test set.

###  IV-F Deformable Convolution Layer Ablation Study

We conduct an ablation study for the deformable convolution layer by running
two experiments. First, we omit the last deformable convolution layer in the
similarity estimation part of the network. For a more correct comparison we
put a convolution-layer at the end with the same amount of trainable
parameters as the deformable convolution layer has and train it for the same
amount of training steps. Then, we repeat the experiment, only now with the
deformable convolution layer in place. As seen in Tab. V, the deformable
convolution layer improves the accuracy of the $4-PE$ and $2-PE$ but lowers
the accuracy of the $1-PE$ and $0.5-PE$. This means that in our experiments,
using a deformable convolutions decreases the lower end-point accuracy.
However, our method is not build with subpixel accuracy in mind, instead
focusing on optimizing the higher end-point errors. As the experiment shows an
improvement in the higher end-point errors this does not speak against the use
of deformable convolution layers.

TABLE V: Ablation study deformable convolution layer | 4-PE | 2-PE | 1-PE | 0.5-PE
---|---|---|---|---
|  | Train |  |
Without DConv | 17.427 | 20.277 | 26.248 | 46.387
With DConv | 15.793 | 19.350 | 30.234 | 55.584
|  | Test |  |
Without DConv | 21.717 | 23.991 | 29.161 | 46.258
With DConv | 20.399 | 23.157 | 32.274 | 53.611

###  IV-G Depth Completion Ablation Study

In this section the validity and correctness of our depth completion part is
shown. To this end we report on the disparity map result of the left frame
with the depth completion part omitted and compare it with the depth
completion in place.

TABLE VI: Ablation study depth completion (DC) | 4-PE | 2-PE | 1-PE | 0.5-PE
---|---|---|---|---
|  | Train |  |
Without DC | 8.867 | 10.743 | 16.041 | 38.208
With DC | 6.971 | 9.793 | 16.192 | 38.858
|  | Test |  |
Without DC | 16.272 | 18.288 | 23.858 | 42.533
With DC | 10.240 | 13.025 | 19.681 | 39.860

Tab. VI shows that using our depth completion method improves upon the overall
accuracy, especially for non-trained data.

###  IV-H Generalization Test

We show that our method generalizes well and is well suited for many different
scenarios. We demonstrate this by using the hyperparameter and trained weights
from the Middlebury dataset and do inference on true in-the-wild datasets that
lack ground-truth or knowledge about the camera intrinsics. To this end we
test our framework on two different publicly available datasets, namely the
Holopix50k dataset [36] and the Flickr1024 dataset [37]. As Fig. 3 shows, our
method produces useful results without the need of retraining for many
different domains, such as indoor scenes with clutter, outdoor scenes and
architecture. Furthermore, a quantitative study was performed to show the
generality of the trained depth completion. To this end, we use the weights
trained on one dataset, to do inference on the other datasets and report on
the 2-point error. As input the disparity maps with removed inconsistencies
produced by the previous parts of our method were taken.

TABLE VII: Generalization Test Depth Completion | Middlebury | Kitti2012 | Kitti2015 | ETH3D
---|---|---|---|---
| (trained) | (trained) | (trained) | (trained)
Middlebury | 9.47 | 10.254 | 11.02 | 10.256
Kitti2012 | 13.56 | 13.16 | 13.24 | 13.16
Kitti2015 | 15.58 | 15.21 | 15.21 | 15.21
ETH3D | 0.93 | 0.87 | 0.98 | 0.87

Tab. VII shows the 2-PE of the quantitative generalization test of the depth
completion part of our method. It shows that our method generalizes well and
that the end-point error stays stable, even if the network is trained with
different data.

Figure 3: Qualitative results of selected stereo pair of our framework from
different domains. From top to bottom: 2 samples of Holopix and 2 samples of
Flickr.

##  V Conclusion

In this work we have presented a fully trainable stereo estimation method that
produces completely dense disparity maps. We have shown that our method
improved upon the baseline method of FC-DCNN in all evaluated challenging
datasets. We have introduced a novel learning based depth-completion method.
By reformulating the classification task for the missing disparity labels we
were able to limit the amount of classes needed for inference and training for
this task. We have argued that this leads to a shallow and effective network
that improved the overall accuracy of the method. Furthermore, we have shown
that our method is able to generalize well to previously unseen data from
different domains, producing reasonable qualitative results.

## References

  * [1] H. Hirschmueller. _Accurate and efficient stereo processing by semi-global matching and mutual information._ IEEE Computer Society Conference on Computer Vision and Pattern Recognition (CVPR’05), Vol. 2., p. 807-814, 2005.
  * [2] G. Facciolo, et al. _MGM: A significantly more global matching for stereovision._ BMVC 2015, 2015.
  * [3] D. Hirner and F. Fraundorfer. _FC-DCNN: A densely connected neural network for stereo estimation._ 2020 25th International Conference on Pattern Recognition (ICPR). IEEE, 2021.
  * [4] A. Kendall et al. _End-to-end learning of geometry and context for deep stereo regression._ Proceedings of the IEEE International Conference on Computer Vision, 2017.
  * [5] J.R. Chang and Y.S. Chen. _Pyramid stereo matching network._ Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, 2018.
  * [6] J. Zbontar and Y. LeCun. _Stereo matching by training a convolutional neural network to compare image patches._ Journal of Machine Learning Research, Vol. 17, p. 1-32, 2016.
  * [7] C. Chang et al. _On an analysis of static occlusion in stereo vision._ IEEE Computer Society Conference on Computer Vision and Pattern Recognition, p.722-723, 1991.
  * [8] G. Huang et al. _Densely connected convolutional networks._ Proceedings of the IEEE conference on computer vision and pattern recognition, 2017.
  * [9] D. Kong and H. Tao. _A Method for Learning Matching Errors in Stereo Computation._ BMVC, p. 2, 2004.
  * [10] P. Knobelreiter et al. _End-to-end training of hybrid CNN-CRF models for stereo._ Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, 2017.
  * [11] F. Zhang et al. _Ga-net: Guided aggregation net for end-to-end stereo matching._ Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, 2019.
  * [12] W. Luo et al. _Efficient deep learning for stereo matching._ Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, 2016.
  * [13] H. Xu and J. Zhang. _Aanet: Adaptive aggregation network for efficient stereo matching._ Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, p.1959–1968, 2020.
  * [14] L. Lipson et al. _RAFT-Stereo: Multilevel Recurrent Field Transforms for Stereo Matching._ arXiv preprint arXiv:2109.07547, 2021.
  * [15] J. Dai et al._Deformable convolutional networks._ Proceedings of the IEEE international conference on computer vision, p.764–773,2017
  * [16] X. Zhu et al. _Deformable convnets v2: More deformable, better results._ Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, p. 9308–9316, 2019.
  * [17] P. Viola and W.M. Wells III. _Alignment by maximization of mutual information._ International journal of computer vision 24.2, p. 137-154, 1997.
  * [18] F. Aleotti et al. _Reversing the cycle: self-supervised deep stereo through enhanced monocular distillation._ European Conference on Computer Vision, p.614–632, 2020.
  * [19] A. Eldesokey et al. _Propagating confidences through cnns for sparse data regression._ arXiv preprint arXiv:1805.11913, 2018.
  * [20] J. Ku et al. _In defense of classical image processing: Fast depth completion on the cpu._ 2018 15th Conference on Computer and Robot Vision (CRV). IEEE, 2018.
  * [21] L.K. Liu et al. _Depth reconstruction from sparse samples: Representation, algorithm, and sampling._ IEEE Transactions on Image Processing 24.6 , p.1983-1996, 2015.
  * [22] A. Paszke et al. _PyTorch: An Imperative Style, High-Performance Deep Learning Library._ Advances in Neural Information Processing Systems 32, p. 8024–8035, 2019.
  * [23] G. Bradski. _Open Source Computer Vision Library._ Dr. Dobb’s journal of software tools 3, 2000.
  * [24] D.P. Kingma and J.Ba. _Adam: A method for stochastic optimization._ arXiv preprint arXiv:1412.6980, 2014.
  * [25] H. Wu et al. _Fast end-to-end trainable guided filter._ Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, 2018.
  * [26] D. Scharstein et al. _High-resolution stereo datasets with subpixel-accurate ground truth._ German conference on pattern recognition. Springer, Cham, 2014.
  * [27] A. Wong et al. _An Adaptive Framework for Learning Unsupervised Depth Completion._ IEEE Robotics and Automation Letters, 6.2, p.3120–3127, 2021.
  * [28] P.Knöbelreiter et al. _Belief Propagation Reloaded: Learning BP-Layers for Labeling Problems._ Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, p.7900–7909, 2020.
  * [29] G. Yang et al. _Hierarchical deep stereo matching on high-resolution images._ Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, p.5515–5524, 2019.
  * [30] L. Lipson et al. _Raft-stereo: Multilevel recurrent field transforms for stereo matching._ arXiv preprint arXiv:2109.07547, 2021.
  * [31] Z. Liang et al. _Learning for disparity estimation through feature constancy._ Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, p.2811–2820, 2018.
  * [32] M. Menze and A. Geiger. _Object scene flow for autonomous vehicles._ Proceedings of the IEEE conference on computer vision and pattern recognition., 2015.
  * [33] Thomas Schöps et al. _A Multi-View Stereo Benchmark with High-Resolution Images and Multi-Camera Videos._ Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, 2017.
  * [34] J. Chang et al. _Attention-Aware Feature Aggregation for Real-time Stereo Matching on Edge Devices._ Proceedings of the Asian Conference on Computer Vision, 2020.
  * [35] O. Zeglazi et al. _A hierarchical stereo matching algorithm based on adaptive support region aggregation method._ Pattern Recognition Letters 112, p.205-211, 2018.
  * [36] Y. Hua et al. _Holopix50k: A large-scale in-the-wild stereo image dataset._ arXiv preprint arXiv:2003.11172, 2020.
  * [37] Y. Wang et al. _Flickr1024: A large-scale dataset for stereo image super-resolution._ Proceedings of the IEEE/CVF International Conference on Computer Vision Workshops, 2019.
  * [38] J. Zhang et al. _Dispsegnet: Leveraging semantics for end-to-end learning of disparity estimation from stereo imagery._ IEEE Robotics and Automation Letters 4.2, p.1162-1169, 2019.
  * [39] L. Ang and Y. Zejian. _Occlusion Aware Stereo Matching via Cooperative Unsupervised Learning._ Proceedings of the Asian Conference on Computer Vision, ACCV, 2018.
  * [40] L. Wang et al. _Parallax attention for unsupervised stereo correspondence learning._ IEEE transactions on pattern analysis and machine intelligence, 2020.
  * [41] R. Rahim et al. _Separable convolutions for optimizing 3d stereo networks._ , 2021 IEEE International Conference on Image Processing, ICIP, 2021.
  * [42] F. Ma and S. Karaman. _Sparse-to-dense: Depth prediction from sparse depth samples and a single image._ , 2018 IEEE international conference on robotics and automation (ICRA), 2018.
  * [43] F. Ma et al. _Self-supervised sparse-to-dense: Self-supervised depth completion from lidar and monocular camera._ , 2019 International Conference on Robotics and Automation (ICRA), 2019.