# Obstacle Avoidance Using a Monocular Camera

Kyle Hatch111Undergraduate Student, Department of Computer Science,
<EMAIL_ADDRESS>Stanford University, Stanford, CA, 94305  John
Mern222Ph.D. Candidate, Department of Aeronautics and Astronautics, AIAA
Student Member Stanford University, Stanford, CA, 94305  and Mykel
Kochenderfer333Associate Professor, Department of Aeronautics and
Astronautics, AIAA Associate Fellow Stanford University, Stanford, CA, 94305

###### Abstract

A collision avoidance system based on simple digital cameras would help enable
the safe integration of small UAVs into crowded, low-altitude environments. In
this work, we present an obstacle avoidance system for small UAVs that uses a
monocular camera with a hybrid neural network and path planner controller. The
system is comprised of a vision network for estimating depth from camera
images, a high-level control network, a collision prediction network, and a
contingency policy. This system is evaluated on a simulated UAV navigating an
obstacle course in a constrained flight pattern. Results show the proposed
system achieves low collision rates while maintaining operationally relevant
flight speeds.

##  1 Introduction

The safe introduction of unmanned vehicles into congested urban environments
will require robust obstacle sensing and avoidance systems. Small unmanned
aerial vehicles (UAVs) often have size, weight, and power consumption (SWaP)
constraints on the payload, making traditional obstacle avoidance sensors such
as radar infeasible [1]. Cameras are light-weight and low-power and are
commonly found on UAVs. However, cameras do not directly detect the distance
of observed objects, making their use in conventional obstacle avoidance
systems challenging. Methods to estimate depth from images and provide robust
obstacle avoidance commands are needed for low-SWaP systems.

Vision-based obstacle avoidance is challenging due to the difficulties of
estimating obstacle depth from 2D images [2, 3]. Deep neural network vision
systems have been able to improve upon conventional depth estimation methods,
however the resulting estimates remain noisy [4]. Deep reinforcement learning
has been shown to be effective for high-level control with noisy, high-
dimensional inputs [5]. Training a network to pilot a vehicle to a destination
without collisions has challenged standard learning approaches [6].

The purpose of this work is to develop a camera-based neural network guidance
and collision avoidance system. Obstacle detection systems estimate the
distance of objects in a vehicle’s field of view (FoV) and predict the
likelihood of a collision based on those estimates and the currently planned
vehicle trajectory. Monocular camera images do not explicitly contain range
information, making their direct use in obstacle detection challenging.

Several methods exist to estimate depth from single images or image sequences
[2, 7]. Conventional techniques such as depth-from-motion or context-based
estimation are highly sensitive to scene content and may produce large errors
[8]. Convolutional Neural Networks (CNNs) may be more robust to scene type,
but typically result in noisy estimates [4], making them difficult to use with
conventional path planning methods.

Deep Reinforcement Learning (RL) can effectively solve high-dimensional
control problems but it can be ineffective for multi-objective or constrained
problems [9]. This is in part because neural network behavior is highly
sensitive to the choice of reward function, which serves as the optimization
target during training. The task of piloting a UAV to a goal destination while
avoiding obstacles is inherently a multi-objective problem: the task of
reaching the goal destination in as little time as possible must be balanced
with the task of minimizing the risk of collision. Tuning the reward such that
the network learns to solve the target task while avoiding failures is
difficult and incorrect parameters can lead to undesirable behavior.

Path planning algorithms such as $A^{*}$ [10] can safely navigate UAVs around
obstacles while minimizing the risk of collision given an accurate map. With
map uncertainty, path planners often rely on reachability analysis with large
error margins to ensure safe operation [11]. This can lead to overly
conservative behavior where generated vehicle paths are much longer than
optimal. One approach to overcome these problems is to develop a hybrid system
in which specialized sub-systems solve the path planning and obstacle
avoidance sub-tasks and enforce behavior constraints [12].

We propose a hybrid neural network, path planner control system that can solve
the multi-objective collision avoidance problem. The system is composed of
three neural networks and a contingency policy. Each of the neural networks
performs a distinct task: one network is used for depth estimation, one for
collision prediction, and one for vehicle control. The contingency policy is
either a path planning based algorithm such as $A^{*}$ or an expert policy.
During unobstructed flight, the vehicle control neural network pilots the
aircraft quickly towards a goal destination. At each timestep, the collision
detection network estimates the probability that the vehicle will collide with
an obstacle within a receding horizon window. If the collision probability
exceeds a threshold, control of the aircraft is temporarily given to the
contingency policy. The contingency policy safely pilots the aircraft around
the obstacle and then returns control of the aircraft to the learned neural
network controller. The trained system is evaluated on its ability to
efficiently reach a goal destination while avoiding stationary obstacles.

By decomposing the multi-objective collision avoidance problem into these two
modes of operation, our hybrid system leverages the advantages offered by both
the neural network controller and the contingency policy. The neural network
controller is trained using RL and can learn a wide range of behaviors that
enable it to efficiently pilot the aircraft towards a destination. Using the
vehicle control network as the default controller of the aircraft avoids using
a highly conservative path planning algorithm when such caution is
unnecessary. This enables the hybrid system pilot the aircraft to its
destination more quickly than if the contingency policy were to control the
aircraft the entire time. However, in high risk situations a conservative
contingency policy can pilot the aircraft around nearby obstacles more safely
than the learned RL policy can. Because the contingency policy is used
exclusively for avoiding obstacles that are very close, it can be optimized to
perform this single, specific purpose. In this way, our system balances the
dual objectives of quickly flying to a goal destination and avoiding
collisions without requiring extensive tuning of the RL reward function.

##  2 Prior Work

Camera-based collision avoidance systems have been developed using a variety
of methods [13]. Some systems rely on visual cues to estimate depth from
monocular images. Michels et al. [2] analyze texture queues from monocular
images to estimate the depth of the closest object in a series of discrete
regions within the image. They then train a model-based RL agent to drive a
small remote control car around stationary obstacles at speeds of up to 5
meters per second. Mori and Scherer [3] estimate depth by analyzing the change
in relative size of image patches between frames. Identifying object features,
such as corners and edges, and comparing the size and location of these
features between frames is also a common approach for monocular depth
estimation [7, 8, 14, 15]. While approaches using visual cues can be
successful at estimating depth in environments where specific visual cues are
present, they tend to generalize poorly to other types of environments.

Stereoscopic techniques have also been used to estimate depth in images for
obstacle avoidance systems. In these approaches, two cameras located at a
known distance apart from each other are used, and the disparity between the
two images is used to estimate depth [16]. Such approaches have been
demonstrated to work with collision avoidance systems for autonomous aircraft
[17]. However, the distance at which they are effective is limited by how far
apart the two cameras are positioned from each other [18], limiting their
utility on small platforms.

More recently, CNNs have been shown to be effective at estimating depth from
monocular images in a variety of environments [19]. Several different CNN
architectures have been developed specifically for the purpose of being used
by obstacle avoidance systems for autonomous aircraft [20, 21, 4].

In addition to obtaining accurate depth mappings of their surroundings,
obstacle avoidance systems must be able to use these maps to avoid collisions
with obstacles. Path planning algorithms are often used with CNN depth
estimators to maneuver aircraft around detected obstacles. Although path
planning algorithms can be effective, they typically require tuning a large
number of parameters and do not learn new behaviors from experience [22].

Reinforcement learning methods that learn to avoid obstacles through trial and
error without supervision or expert knowledge have also been proposed. An
agent learns from experience to find mappings from states to actions that
maximize a long term, accumulated reward. RL has been shown to be successful
in a wide variety of areas, including video games [23] and robotic control
[24]. Similar simulation-based methods have been successfully applied to
specific obstacle avoidance tasks as well [13, 25].

Several previous works have addressed the issue of developing camera based
obstacle avoidance systems using RL. Xie et al. [22] use a convolutional
residual neural network to produce pixel-wise depth estimates from RGB images.
They then train a deep double Q-network to pilot a small wheeled robot around
obstacles. Anwar and Raychowdhury [26] also use double deep Q-networks to
train an agent to pilot a small quadrotor around stationary obstacles. For
depth estimation, they use the network architecture developed by Laina et al.
[19]. Similarly, Singla et al. [27] use deep Q-learning with attention to
pilot a quadrotor around obstacles in a simulator using discrete directional
commands.

Recently, hybrid systems that utilize a combination of learning and planning
methods have been proposed for general vehicle guidance [28]. These systems
are designed to use the efficient route planning of a learned system during
nominal operation and use more conservative contingency planning during high-
risk situations. At the time of this work, no hybrid guidance system has yet
been proposed using only monocular images for obstacle detection.

These previously approaches have some important limitations. Previously
proposed systems have been trained and tested in indoor environments. Indoor
environments are visually much simpler than outdoor environments, which makes
depth estimation easier. The use of an action space consisting of moving
forward a fixed distance or rotating right or left by a fixed amount, as used
by Singla et al. [27] and Anwar and Raychowdhury [26], limit the potential
maneuvers the aircraft can perform. These obstacle avoidance systems must
operate at relatively slow speeds because of their reliance on stop-and-go
flight maneuvers. These limitations decrease the usefulness of these systems
on aircraft flying practical missions. In contrast, our system operates in a
visually complex outdoor environment at comparatively high speeds without the
use of stop-and-go flight maneuvers.

##  3 Methods

Figure 1: System diagram: This figure shows the information flow through the
collision avoidance system. The RGB input images are processed by the depth
CNN and the reduced-scale output is flattened into a 144-length vector. The
depth vector is concatenated with the vehicle odometry vector and input into
the collision prediction network, which outputs the probability of collision.
If the collision probability is below a target threshold and the contingency
policy is not currently executing, the input vector is passed to the policy
network, which outputs a velocity command. Otherwise, the contingency policy
is queried. Once activated, the contingency policy pilots the aircraft for a
short period of time until it terminates. Control of the aircraft is then
returned to the policy network.

The inputs to the proposed collision avoidance system are images from an on-
board visual-spectrum camera and filtered kinematic estimates from the vehicle
inertial sensors. The system outputs a velocity command, which is used as
input to a low-level controller. The collision avoidance system is comprised
of three neural networks–a depth map estimator CNN, a control policy
Q-network, and a collision prediction network–and a non-learned contingency
policy. An architecture diagram is shown in fig. 1.

The depth CNN estimates a depth map from the on-board camera images. A depth
map is an array of distance estimates where each value in the array
corresponds to a location in the original field-of-view. In a full resolution
depth map (where the size of the depth map is equal to the size of the input
image), the value of each depth map pixel is the estimated distance from the
camera to the object seen at the input image pixel. The depth CNN outputs a
1/16 resolution depth map, where each value is an estimate of the minimum
distance of the objects in the corresponding $16\times 16$ pixel block in the
original image. An example of depth maps at full and reduced resolution is
shown in fig. 2. The system is run at 10 Hz.

The depth map estimates are flattened into vectors and concatenated with a
kinematics estimate provided by the on-board odometry sensors. This joint
vector is input to the policy network. Based on this input, the policy network
chooses an action from a discrete set of predefined velocity commands. It does
so by estimating the expected value of taking each action in this set and
recommending the action with the highest expectation.

The joint input vector is also provided to the collision prediction network as
input. The collision prediction network estimates the probability that a
collision will occur in the next 10 time-steps (corresponding to 1 second at
the frequency used in this work). If the predicted collision probability
exceeds some specified threshold (nominally $50\%$), a contingency policy is
enacted. This contingency policy temporarily takes control of the aircraft
from the RL policy in order to pilot the aircraft around the sensed
obstacle(s). Once the contingency policy terminates, control of the aircraft
is returned to the RL policy.

In this work, we tested both a rules-based policy and an online path planner
as the contingency policy. The first policy is an online path planner based on
the $A^{*}$ algorithm. As an input to $A^{*}$, a 2D obstacle map was generated
using the last depth estimate frame. $A^{*}$ was then executed until a route
around the obstacle was found. The details of how the obstacle maps were
generated and how $A^{*}$ was used are described in the Appendix. The second
policy is a simple expert policy designed to pilot the aircraft around an
obstacle in front of it. The expert policy is also described in the Appendix.
These contingency policies worked well for the experiments we conducted,
however, the policies can be replaced by any suitable policy or planner.

Figure 2: Depth map example: (Left) Input RGB image. (Center) Full resolution
depth map. (Right) 1/16 resolution depth map. Pixel intensity is proportional
to the estimated distance. Darker pixels indicate closer objects.

A prototype system was developed to control a small UAV in Microsoft’s AirSim
[29] simulator. The host vehicle was a simple quadrotor with a forward mounted
monocular camera. The training images and true depth maps were gathered as
matched-pairs from AirSim. Velocity commands were input to the AirSim velocity
controller API. The testing was conducted in the “Neighborhood” simulated
environment.

###  3.1 Depth Network

The depth network is a down-scaling convolutional neural network. Inputs to
the network are images represented as 3D arrays of dimension $H\times W\times
3$, where 3 is the number of color channels in the RGB representation. The
network architecture is detailed in table 1. Each layer consists of a standard
affine 2D convolution by the layer filters, followed by a non-linear
operation. In this case, we used the leaky rectified linear unit (Leaky-ReLU)
as the non linear activation function. The leaky-ReLU tends to suffer fewer
issues with vanishing gradients than the basic ReLU operation [30].

We explored using sequences of images as input to the network to enable depth-
from-motion estimation within the graph. In this setting, to estimate a depth
map $y_{t}$, the network would have the sequence of $b$ prior images
$(x_{t-b},\ldots,x_{t})$, as an input, concatenated by the channel axis such
that the complete input would have dimensions $H\times W\times 3b$. Our
initial results show that there are no significant performance benefits to the
complete system using this approach. The remaining work was conducted using
single image inputs to reduce computation.

Table 1: Depth Network Architecture. Input dimensions are shown for input sequences of $b$ images, where $b=1$ corresponds to single image inputs. Layer | Input Dims | Transform | Kernel Size | Stride | Activation | Output Dims
---|---|---|---|---|---|---
Input | $H\times W\times 3b$ | Conv2D | 5 | 2 | Leaky ReLU | $H/2\times W/2\times 32$
H1 | $H/2\times W/2\times 32$ | Conv2D | 5 | 2 | Leaky ReLU | $H/4\times W/4\times 64$
H2 | $H/4\times W/4\times 64$ | Conv2D | 3 | 2 | Leaky ReLU | $H/8\times W/8\times 128$
H3 | $H/8\times W/8\times 128$ | Conv2D | 3 | 2 | Leaky ReLU | $H/16\times W/16\times 256$
H4 | $H/16\times W/16\times 256$ | Conv2D | 3 | 1 | Leaky ReLU | $H/16\times W/16\times 128$
H5 | $H/16\times W/16\times 128$ | Conv2D | 3 | 1 | Leaky ReLU | $H/16\times W/16\times 32$
Output | $H/16\times W/16\times 32$ | Conv2D | 3 | 1 | None | $H/2\times W/2\times 1$

The network was initially trained using a data set of matched image and depth
map pairs. The training data set was gathered by manually piloting the
aircraft through the AirSim environment and recording the camera view and
depth map at a 10 Hz. The network was trained to minimize the expected Huber
loss between the true depth map $y_{i}$ and the depth map estimate produced by
the network $\hat{y}_{i}$. To produce the depth map targets, the full-
resolution depth maps were down-scaled using a min-pooling operation, where
the value of each element in the $H/16\times W/16$ array was set to the
minimum value of the corresponding $16\times 16$ block in the full resolution
map. The expected loss $L(x_{i},y_{i})$ was estimated by the mean loss over a
batch of $m$ training images as

$\hat{L}(x_{i},y_{i})=\frac{1}{m}\sum_{i}^{m}\textsc{Huber}(\phi(x_{i}),y_{i})$
(1)

where $\phi(x_{i})=\hat{y}_{i}$ denotes the depth network output from the
input image $x_{i}$. The Huber loss for a given image pair is defined as

$\textsc{Huber}(\hat{y}_{i},y_{i})=\frac{1}{n}\sum_{j}^{n}\begin{cases}\frac{1}{2}(y_{i}^{j}-\hat{y}_{i}^{j}),&\text{for}|y_{i}^{j}-\hat{y}_{i}^{j}|\leq\delta\\\
\delta|y_{i}^{j}-\hat{y}_{i}^{j}|-\frac{1}{2}\delta^{2},&\text{otherwise}\end{cases}$
(2)

where $j$ is the array element index and $\delta$ is some constant hyper-
parameter. In this work, $\delta$ was set to $1$. The input and output images
were normalized prior to training such that all values were within
$[-1.0,1.0]$.

###  3.2 Policy Network

The policy network was implemented as a Deep Q-Network (DQN) using the
extensions recommended in the RAINBOW architecture [31]. The network takes as
input a vector of the flattened depth map estimate concatenated with a
kinematic estimate from the on-board vehicle sensors. The kinematic state
estimate is the vector
$(p_{y},v_{x},v_{y},v_{z},a_{x},a_{y},a_{z},\phi,\gamma,\theta,\dot{\phi},\dot{\gamma},\dot{\theta},\ddot{\phi},\ddot{\gamma},\ddot{\theta})$,
which correspond to the lateral position, linear velocity, linear
accelerations, angular orientations, angular rates, and angular accelerations
respectively. The lateral position $p_{y}$ was included in this vector so that
the policy network would be able to learn not to pilot the quadrotor out of
bounds. The inclusion of $p_{y}$ cannot aid in obstacle avoidance, as the
policy network was trained on a course in which obstacles are randomly
positioned at the beginning of each episode. The network outputs the Q-values
of each action in the action-set. The Q-value is defined as the expected sum
of discounted rewards when taking a given action and following a fixed policy
for the subsequent steps.

The network architecture is a dual-stream network. Inputs are pre-processed
through three hidden-layers with 128 hidden units each and ReLU activation
functions. This output is then passed through two parallel streams, each with
two hidden layers of 128 units. One stream estimates the state value and the
other estimates the state-action advantage. The outputs from each stream are
combined and the Q-value for each action is then estimated using a discrete
atomized estimator process. To facilitate exploration during training, a noise
term is added to the output of the final layer. For more details on this
architecture and training process, please see the RAINBOW paper [31].

The action set was defined as the set of four actions
$\\{\textsc{Left},\textsc{Forward},\textsc{Right},\textsc{Reverse}\\}$, where
forward corresponds to commanding $3.0\text{\,}\mathrm{m}\text{/}\mathrm{s}$
along the vehicle longitudinal axis, left and right correspond to commanding
$3.0\text{\,}\mathrm{m}\text{/}\mathrm{s}$ with a
$15\text{\,}\mathrm{\SIUnitSymbolDegree}\text{/}$ yaw to the left or right of
the vehicle longitudinal axis, and reverse corresponds to commanding
$-0.5\text{\,}\mathrm{m}\text{/}\mathrm{s}$ along the longitudinal direction.
The UAV was restricted to 2D-flight at a fixed altitude above ground. These
constraints made solving the obstacle avoidance problem much more difficult
and provides a better test case for the learned system. While this small
action set may seem limited, the $10\text{\,}\mathrm{Hz}\text{/}$ update rate
and the use of a low-level controller enable a rich variety of maneuvers to be
conducted.

Early in this work, we tested a stochastic policy neural network that directly
output actions from a continuous domain. We trained this network with Proximal
Policy Optimization [32] and found that training was much more difficult and
resulted in trajectories with frequent small changes in commanded velocity.
The performance of the selected Q-network architecture was higher and the
resulting trajectories were observed to be much smoother.

The network was trained to reach a goal location as quickly as possible
without collision. The reward function was defined as

$r_{t}(s_{t},a_{t})=\begin{cases}x_{t}-x_{t-1}&\textsc{NoCollision}(s_{t})\\\
-\lambda&\textsc{Collision}(s_{t})\end{cases}$ (3)

where $s_{t}$ is the complete vehicle state, $a_{t}$ is the action commanded,
$x_{t}$ is the vehicle’s x-position at time-step $t$ (with the x-axis as the
road direction), and $\lambda$ is a penalty for collision or out-of-bounds
states. If the vehicle has not crashed or flown out-of-bounds, the reward is
equal to the forward progress made since the previous time-step. If the
vehicle has crashed or flown out of bounds, the reward is equal to the penalty
$-\lambda$.

The policy was trained to navigate an obstacle course we created in the AirSim
Neighborhood environment. The UAV’s goal was to fly from one end of a street
to the other. Cars were placed as obstacles in the street and the UAV was
constrained to fly at an altitude approximately equal to half the height of
the cars. Images of this obstacle course are shown in fig. 3.

The course is 100 meters long with six cars placed as obstacles. The cars were
spaced evenly from the 20 meter mark to the 100 meter mark at 15 meter
intervals. At the beginning of each episode, the lateral position of each car
was randomly assigned. The quadrotor was positioned at the 0 meter mark at the
beginning of each episode. When the quadrotor safely reached the end of the
$100\text{\,}\mathrm{m}\text{/}$ long street, crashed into an obstacle, flew
out of bounds, or exceeded 600 time-steps (1 minute) of flight time, the
episode was terminated.

Figure 3: UAV obstacle course. The left image shows the obstacle course from
just behind the UAV point-of-view. The right shows a top-down view of the
course. The UAV must fly forward while avoiding the cars and staying above the
roadway to successfully navigate the course.

The policy was trained using estimates from the depth map network and
simulated odometry measurements from AirSim as input. The velocity of the
maximum Q-value action was commanded at each time-step. The network was
optimized using the RAINBOW algorithm.

###  3.3 Collision Prediction Network

The input to the first layer of the collision prediction network is the
concatenation of the policy network input. This input is processed through
four fully-connected layers and the resulting hidden vector is concatenated
with the selected action index. This intermediate layer is further processed
through four additional layers, returning a $2\times 1$ output. This output
vector represents the un-normalized probabilities of collision versus non-
collision in the next $d$ time-steps. The normalized probabilities are
calculated using the softmax logistic transform. The architecture is
summarized in table 2.

Table 2: Collision Network Architecture. Layer | Input Length | Activation | Output Units
---|---|---|---
Input | 10 | Leaky ReLU | 256
H1 | 256 | Leaky ReLU | 256
H2 | 256 | Leaky ReLU | 128
H3 | 128 | Leaky ReLU | 32
Concatenate | 32 + 4 | None | 36
H4 | 36 | Leaky ReLU | 32
H5 | 32 | Leaky ReLU | 32
H6 | 32 | Leaky ReLU | 32
Output | 32 | None | 2

The collision network was trained by supervised learning. The training data
was gathered by running the depth estimation network and policy network in the
obstacle course for 1,000 episodes without the collision prediction network.
The depth map estimates and odometry vectors were collected as the training
inputs at each time step. The AirSim collision flag was collected each time
step as well. The training labels were set to $(1,0)$ for each time step in
which a collision was observed in any of the subsequent ten frames, and
$(0,0)$ otherwise.

The training process minimized the cross entropy loss for a binary
distribution between the target distribution and the predicted distribution,
as in

$L=-(y_{t}\log(\hat{y}_{t})+(1-y_{t})\log(1-\hat{y}_{t}))$ (4)

where $y_{t}$ is the true probability of collision and $\hat{y}_{t}$ is the
prediction.

The training data set had far more examples of non-collision frames than
collision frames, with 295,183 total frames and just 1,890 with collisions
occurring in the next ten frames. Sampling from this set directly would often
result in training batches that incorrectly estimate an expected crash rate of
zero, which could bias the training process. To offset this, we constructed
each training batch by sampling $25\%$ of the batch to be collision examples
and $75\%$ of the batch to be non-collision examples. Sampling in this way
could also bias the training to be overly conservative. To offset this
potential bias, we considered using an importance sampling weighting on each
loss component. Initial testing, however, showed this bias not to be an issue
and importance sampling was not used for our results.

##  4 Experiments and Results

The depth network, policy network, and collision prediction network were each
trained separately. The performance of each network was also evaluated
independently using appropriate metrics. The best performing instance of each
of the three networks was then integrated into the hybrid obstacle avoidance
system and the performance of the complete system evaluated. Systems that only
used either the RL policy or the contingency policy were also tested as
baselines for comparison.

###  4.1 Depth Network

A data set of $14,048$ (RGB image, reduced depth map) pairs was collected, of
which ninety percent were used to train the depth network and ten percent of
were held out for performance validation. The performance of the trained
network on the validation set was measured using several different loss
functions.

The mean average error (MAE) is defined as

$\text{MAE}=\sum_{i=1}^{n}\|y_{i}-\hat{y}_{i}\|_{1}$ (5)

and evaluates the general fit of the estimates to the data. The mean square
error (MSE) is defined as

$\text{MSE}=\sum_{i=1}^{n}\|y_{i}-\hat{y}_{i}\|^{2}_{2}$ (6)

and similarly evaluates the data agreement, while putting relatively more
emphasis on larger magnitude errors than MAE. The root mean-square logistic
error (RMSLE) is defined as

$\text{RMSLE}=\sum_{i=1}^{n}\|\text{log}(2-y_{i})-\text{log}(2-\hat{y}_{i})\|^{2}_{2}$
(7)

and evaluates the accuracy of estimates relative to the regional intensity of
image pixels. This means that errors are penalized more heavily in regions
with high pixel values than errors of equal magnitude in regions with low
pixel values.

The mean validation loss of the trained depth estimation network for these
metrics and the Huber loss are shown along with standard error bounds in table
3.

Table 3: Depth estimation network performance. Disagreement loss calculated by four metrics: mean average error (MAE), mean square error (MSE), root mean-square logistic error (RMSLE) , and Huber loss. MAE | MSE | RMSLE | Huber
---|---|---|---
$0.147\pm 1.41\times 10^{-3}$ | $0.043\pm 9.01\times 10^{-4}$ | $0.102\pm 6.60\times 10^{-4}$ | $0.022\pm 4.49\times 10^{-4}$

The observed validation error is reasonable based on the performance of
similar depth-estimation CNNs reported in prior works [4]. We tested the
effectiveness of the depth network as part of the complete system by comparing
the performance with the depth network estimates to the performance of the
system using the true depth map. The result of this testing is discussed in
the following sections.

###  4.2 Policy Network

The policy network was initially trained with a collision penalty
$\lambda=20.0$ for 5,199 episodes totalling 983,751 time-steps. Its
performance was evaluated by running the network through the obstacle course
for 1,000 trials. Since the cars were positioned randomly at the beginning of
each episode, it was not necessary to change the obstacle course for
evaluation. During evaluation, the noise factor in the final layers of the
policy network used for exploration was set to zero. The policy was evaluated
on the course completion rate, the collision rate, and average episode length.

Evaluating the initial network showed a higher collision rate than acceptable.
We first tried to address this by tuning the collision loss parameter. The
crash penalty was increased to 30 and the policy network was trained for a
further 1,720 episodes. This additional training led to a slight increase in
the fraction of episodes completed during evaluation. However, the collision
rate remained unacceptably high. The crash penalty was also increased to 40
and the policy network was trained for a further 1,234 episodes (336,202 total
time-steps). This further training decreased the fraction of episodes
completed during evaluation. These results are summarized in table 4.

In order to evaluate the impact of depth estimation performance on the policy
network performance, we also trained and tested a policy network using the
true depth maps provide directly by AirSim. This network was trained for 4,397
episodes with a collision penalty of $\lambda=20$ and evaluated for 1,000
trials. These results are also included in table 4. The comparable completion
rate when using the true depth network suggests that the depth estimation
network performance is sufficient for the collision avoidance task. The
increased out-of-bounds rate with the true depth maps suggests the true maps
may have caused difficulties with training the policy, resulting in
convergence to a lower-performing local optimum.

Table 4: Policy network evaluation results. Results from 1,000 test runs of the policy network without using the collision detection network. The collision, out-of-bounds (OOB), and timeout rates are shown. The mean number of time-steps per episode and one standard error bounds are also shown. Each time-step is 0.1 seconds in simulation time. $\lambda$ | Depth Map | Collision Rate | OOB Rate | Timeout Rate | Completion Rate | Episode Length
---|---|---|---|---|---|---
20.0 | Estimated | $17.7\%$ | $2.4\%$ | $0\%$ | $79.9\%$ | $286.5\pm 3.6$
30.0 | Estimated | $18\%$ | $0.09\%$ | $0\%$ | $81.1\%$ | $295.8\pm 2.7$
40.0 | Estimated | $18.5\%$ | $17.7\%$ | $4.0\%$ | $59.7\%$ | $321.4\pm 3.6$
20.0 | True | $9.4\%$ | $24.5\%$ | $0.0\%$ | $66.0\%$ | $308.2\pm 2.4$

###  4.3 Collision Prediction Network

Results from the policy network evaluation showed that tuning the collision
loss parameter was not sufficient to achieve a low collision rate. To evaluate
the effectiveness of the collision prediction network, we assessed its error
rates on trajectories generated from the UAV flown by the depth estimation and
policy networks. In this way, the test set was generated in the same way as
the training set. The data set contained 189 collision-labeled frames and
29,000 collision-free frames. A total of 25,000 batches were analyzed.

Because there were disproportionately more non-collision frames, the
evaluation was run in batches where each batch was sampled to contain 24 non-
collision frames and 8 collision-labeled frames. From the results, we
calculated the accuracy, precision, recall, and $\text{F}_{1}$ score. The
results are summarized in table 2. The false positive and false negative
errors are both on the order of $1\%$.

Table 5: Collision Prediction Network Validation Results. Metric | Value
---|---
Accuracy | 0.982
Precision | 0.976
Recall | 0.954
F1-score | 0.963
Cross Entropy | 0.057
True Positive Rate | 0.239
True Negative Rate | 0.743
False Positive Rate | 0.006
False Negative Rate | 0.012

###  4.4 Hybrid System

The best performing instance of each of the three networks was integrated into
the complete obstacle avoidance system. In the case of the policy network,
this is the network that was trained with the collision penalty $\lambda=40$.
If the collision prediction network predicted that a collision would occur
within the next timesteps with greater than 50% probability, then the
contingency policy would take control of the aircraft. The contingency policy
would then pilot the aircraft until it terminated, at which point control of
the aircraft would be returned to the RL controller. Systems using both
$A^{*}$ and the expert policy were each tested.

As a baseline, we compare the performance of our hybrid system with the
performance of the best performing policy network alone (without activating
the contingency policy). These results are shown in 4 and are reproduced in 6
for convenience. We also compare our hybrid system against a system that only
uses the contingency policy without the learned RL component. In this case,
the aircraft simply flies forward in a straight line until the collision
detection network goes off. At this point, the expert contingency policy takes
control of the aircraft. After the contingency policy finishes executing, the
aircraft resumes flying forward in a straight line.

Because the dynamics of the aircraft differ when the it is being flown forward
in a straight line than when it is piloted by the RL policy, the collision
network trained with RL policy data would perform more poorly during straight
line flight. Because of this, we trained a separate collision prediction
network for use when the aircraft is flown forward in a straight line. This
network was trained in the same manner to the original collision prediction
network using data gathered from straight line flight. This network achieved
similar validation results to the RL policy prediction network.

The performance of each system was evaluated on the obstacle course for 1,000
episodes.

Table 6: Hybrid System Evaluation Results.

| Hybrid ($A*$) | Hybrid (expert) | RL policy | Expert Only
---|---|---|---|---
Collision Rate | $5.5\%$ | $1.8\%$ | $17.2\%$ | $24.5\%$
Contingency Policy Collision Rate | $10.1\%$ | $3.3\%$ | – | $36.1\%$
Out-of-Bounds Rate | $0.1\%$ | $0\%$ | $0.9\%$ | $0\%$
Timeout Rate | $0.1\%$ | $0\%$ | $0\%$ | $0.2\%$
Completion Rate | $84.2\%$ | $94.9\%$ | $81.9\%$ | $39.2\%$
Episode Length* | $468.2\pm 5.4$ |  $440.2\pm$ $3.2$ | $332.8\pm 0.1$ | $617.2\pm 7.8$

  * •

*Episode lengths are calculated only for episodes that end in completion.

The hybrid system using the expert contingency policy achieved a completion
rate of $94.9\%$. This is significantly higher than the completion rates
achieved by the RL policy alone or the expert contingency policy alone. The
hybrid system using $A^{*}$ also achieved a higher completion rate than those
of RL policy alone and the expert contingency policy alone. However, its
completion rate was significantly lower than that of the hybrid system using
the expert policy.

The hybrid system using the expert contingency policy achieved an average
completed episode length of $440.2$ timesteps. Although this is longer than
average completed episode length of $332.8$ achieved by the RL policy alone,
it is significantly shorter than the average episode length of $617.2$
achieved by the expert contingency policy alone. This shows that our hybrid
approach is able to balance the objective of quickly reaching its destination
with the objective of safely avoiding collisions.

Qualitative analysis of the hybrid system using $A^{*}$ shows that many of its
collisions occurred when the $A^{*}$ planner planned a path that ran too close
to the obstacle it was trying to avoid. This is likely due to the fact that
the $A^{*}$ planner relies on having an accurate map of its surroundings in
order to plan a safe route. In our system, these maps are generated using the
depth estimation network. Noise in these estimates can have significant
negative impacts on the paths planned by the $A^{*}$ planner.

In contrast, the expert contingency policy relies very little on the depth
estimates generated by the depth estimation network. This is likely a major
reason why the hybrid system using the expert policy achieved much a higher
completion rate than that achieved by the hybrid system using $A^{*}$.
Although the expert policy we designed was very effective in this environment,
it is a less general approach than using the $A^{*}$ planner.

Examination of the hybrid system using the expert contingency policy shows
that, of all the collisions that occurred while the RL policy was piloting the
aircraft, the vast majority took place immediately after control of the
aircraft was returned to the RL policy from the contingency policy. Almost no
collisions occurred as a result of the collision prediction network failing to
activate the contingency policy. This indicates that the majority of the
collisions from our hybrid system are due to weaknesses in the contingency
policy. In many instances of collision, the contingency policy would pilot the
aircraft nearly around an obstacle but not quite clear it. The RL policy would
then hit the corner of the obstacle that the contingency policy failed to
completely clear. Presumably, if this expert contingency policy were replaced
with a more carefully designed expert policy, then these collisions would be
avoided. However, the objective of this paper is not to design an optimal
contingency policy for use on this environment. Instead, we show that our
hybrid system, when used with an appropriate contingency policy, achieves much
better performance than do either the RL policy or the contingency policy
alone.

##  5 Conclusions

This work presented an obstacle avoidance system for small UAVs based on a
hybrid neural network and path planner controller using monocular image
inputs. The proposed hybrid system combines the strength of a learned RL
policy and a path planning or expert contingency policy. By decomposing
collision avoidance in this way, our hybrid system achieves significantly
lower collision rates than either the RL policy or the contingency policy do
alone. Our hybrid system also reaches its destination faster than the
contingency policy alone does, demonstrating that it is able to quickly reach
its destination while also safely avoiding collisions. Our system balances
these dual objectives without relying on extensive tuning of the RL reward
function.

The complete system is composed of three specialized neural networks for depth
estimation, piloting, and collision detection. Experiments showed that neural
network piloting performed as well with estimated depth as with the true depth
maps. This suggests that the proposed depth network was sufficiently accurate
for the guidance task. However, the $A^{*}$ planner used as a contingency
policy seemed to perform poorly due to the estimate noise. In practical
applications, even small UAVs may have short range obstacle sensors that would
be able to provide better map data for contingency systems. Future work will
model this data and investigate its effect on system performance.

The reinforcement learning trained neural network controller was able to reach
the goal more quickly than the expert policy on completed episodes.
Qualitative inspection suggests this is due to the RL policy following much
smoother trajectories than the expert. However, the collision rate of the RL
policy alone was unacceptably high. Increasing the penalty for collisions
during RL training was not sufficient to resolve this.

The collision detection network performed well, with type I and type II errors
each on the order of $1\%$. These errors may be higher when used on control
polices that differ for the one on which it was trained. Future work will
investigate this and robust training solutions using curricular replay
buffers.

The focus of this work was to demonstrate that a hybrid approach to collision
avoidance for small UAVs can outperform approaches that rely on RL, expert
policies, or path planning algorithms alone. Replacing the contingency
policies that we used in our experiments with more sophisticated path planning
algorithms will likely further reduce the collision rate. Future work will
explore more robust contingency policies, for example using uncertainty-aware
Monte Carlo tree search or Hamilton-Jacobi reachability.

## References

  * Sahawneh [2016] Sahawneh, L. R., “Airborne Collision Detection and Avoidance for Small UAS Sense and Avoid Systems,” Ph.D. thesis, Brigham Young University, 2016.
  * Michels et al. [2005] Michels, J., Saxena, A., and Ng, A. Y., “High speed obstacle avoidance using monocular vision and reinforcement learning,” _International Conference on Machine Learning (ICML)_ , Vol. 119, 2005, pp. 593–600.  10.1145/1102351.1102426.
  * Mori and Scherer [2013] Mori, T., and Scherer, S. A., “First results in detecting and avoiding frontal obstacles from a monocular camera for micro unmanned aerial vehicles,” _IEEE International Conference on Robotics and Automation (ICRA)_ , 2013, pp. 1750–1757.  10.1109/ICRA.2013.6630807.
  * Mern et al. [2019] Mern, J. M., Julian, K. D., Tompa, R. E., and Kochenderfer, M. J., “Visual Depth Mapping from Monocular Images using Recurrent Convolutional Neural Networks,” _AIAA SciTech Forum_ , 2019, pp. 1189–1197.  10.2514/6.2019-1189.
  * Levine et al. [2016] Levine, S., Finn, C., Darrell, T., and Abbeel, P., “End-to-End Training of Deep Visuomotor Policies,” _Journal of Machine Learning Research_ , Vol. 17, 2016, pp. 39:1–39:40.
  * Pirotta et al. [2015] Pirotta, M., Parisi, S., and Restelli, M., “Multi-Objective Reinforcement Learning with Continuous Pareto Frontier Approximation,” _AAAI Conference on Artificial Intelligence (AAAI)_ , 2015, pp. 2928–2934.
  * Lee et al. [2011] Lee, J.-O., Lee, K.-H., Park, S.-H., Im, S.-G., and Park, J., “Obstacle avoidance for small UAVs using monocular vision,” _Aircraft Engineering and Aerospace Technology_ , Vol. 83, 2011, pp. 397–406.
  * Saha et al. [2014] Saha, S., Natraj, A., and Waharte, S., “A real-time monocular vision-based frontal obstacle detection and avoidance for low cost UAVs in GPS denied environment,” _IEEE International Conference on Aerospace Electronics and Remote Sensing Technology_ , 2014, pp. 189–195.
  * Dewey [2014] Dewey, D., “Reinforcement Learning and the Reward Engineering Principle,” _AAAI Conference on Artificial Intelligence (AAAI)_ , 2014.
  * Hart et al. [1968] Hart, P. E., Nilsson, N. J., and Raphael, B., “A Formal Basis for the Heuristic Determination of Minimum Cost Paths,” _IEEE Transactions on Systems Science and Cybernetics_ , Vol. 4, No. 2, 1968, pp. 100–107.  10.1109/TSSC.1968.300136.
  * Herbert et al. [2019] Herbert, S. L., Bansal, S., Ghosh, S., and Tomlin, C. J., “Reachability-Based Safety Guarantees using Efficient Initializations,” _IEEE Conference on Decision and Control (CDC)_ , 2019, pp. 4810–4816.  10.1109/CDC40024.2019.9029575.
  * Ding et al. [2011] Ding, J., Gillula, J. H., Huang, H., Vitus, M. P., Zhang, W., and Tomlin, C. J., “Hybrid Systems in Robotics,” _IEEE Robotics Automation Magazine_ , Vol. 18, No. 3, 2011, pp. 33–43.
  * Kochenderfer et al. [2008] Kochenderfer, M., Griffith, J., and Kuchar, J., “Hazard Alerting Using Line-of-Sight Rate,” _AIAA Guidance, Navigation, and Control Conference (GNC)_ , 2008.  10.2514/6.2008-6630.
  * Al-Kaff et al. [2016] Al-Kaff, A., Meng, Q., Martín, D., de la Escalera, A., and Armingol, J. M., “Monocular vision-based obstacle detection/avoidance for unmanned aerial vehicles,” _IEEE Intelligent Vehicles Symposium (IV)_ , 2016, pp. 92–97.  10.1109/IVS.2016.7535370.
  * Kovács [2016] Kovács, L., “Visual Monocular Obstacle Avoidance for Small Unmanned Vehicles,” _IEEE Computer Society Conference on Computer Vision and Pattern Recognition (CVPR)_ , 2016, pp. 877–884.  10.1109/CVPRW.2016.114.
  * Lucas and Kanade [1981] Lucas, B. D., and Kanade, T., “An Iterative Image Registration Technique with an Application to Stereo Vision,” _International Joint Conference on Artificial Intelligence (IJCAI)_ , 1981, pp. 674–679.
  * Ruf et al. [2018] Ruf, B., Monka, S., Kollmann, M., and Grinberg, M., “Real-time on-board obstacle avoidance for UAVs based on embedded stereo vision,” _Computing Research Repository_ , 2018.
  * Alvertos [1988] Alvertos, N., “Resolution limitations and error analysis for stereo camera models,” _IEEE Southeastcon_ , 1988, pp. 220–224.
  * Laina et al. [2016] Laina, I., Rupprecht, C., Belagiannis, V., Tombari, F., and Navab, N., “Deeper Depth Prediction with Fully Convolutional Residual Networks,” _International Conference on 3D Vision (3DV)_ , 2016, pp. 239–248.  10.1109/3DV.2016.32.
  * Mancini et al. [2017] Mancini, M., Costante, G., Valigi, P., and Ciarfuglia, T. A., “J-MOD2: Joint Monocular Obstacle Detection and Depth Estimation,” _Computing Research Repository_ , 2017.
  * Chakravarty et al. [2017] Chakravarty, P., Kelchtermans, K., Roussel, T., Wellens, S., Tuytelaars, T., and Eycken, L. V., “CNN-based single image obstacle avoidance on a quadrotor,” _IEEE International Conference on Robotics and Automation (ICRA)_ , 2017, pp. 6369–6374.  10.1109/ICRA.2017.7989752.
  * Xie et al. [2017] Xie, L., Wang, S., Markham, A., and Trigoni, N., “Towards Monocular Vision based Obstacle Avoidance through Deep Reinforcement Learning,” _Computing Research Repository_ , 2017.
  * Mnih et al. [2013] Mnih, V., Kavukcuoglu, K., Silver, D., Graves, A., Antonoglou, I., Wierstra, D., and Riedmiller, M. A., “Playing Atari with Deep Reinforcement Learning,” _Computing Research Repository_ , 2013.
  * Amarjyoti [2017] Amarjyoti, S., “Deep Reinforcement Learning for Robotic Manipulation - The state of the art,” _Computing Research Repository_ , Vol. abs/1701.08878, 2017.
  * Temizer et al. [2010] Temizer, S., Kochenderfer, M., Kaelbling, L., Lozano-Perez, T., and Kuchar, J., “Collision Avoidance for Unmanned Aircraft using Markov Decision Processes,” _AIAA Guidance, Navigation, and Control Conference (GNC)_ , 2010\.  10.2514/6.2010-8040.
  * Anwar and Raychowdhury [2018] Anwar, M. A., and Raychowdhury, A., “NavREn-Rl: Learning to fly in real environment via end-to-end deep reinforcement learning using monocular images,” _International Conference on Mechatronics and Machine Vision in Practice, M2VIP_ , IEEE, 2018, pp. 1–6.  10.1109/M2VIP.2018.8600838.
  * Singla et al. [2018] Singla, A., Padakandla, S., and Bhatnagar, S., “Memory-based Deep Reinforcement Learning for Obstacle Avoidance in UAV with Limited Environment Knowledge,” _Computing Research Repository_ , 2018.
  * Dhinakaran et al. [2017] Dhinakaran, A., Chen, M., Chou, G., Shih, J. C., and Tomlin, C. J., “A hybrid framework for multi-vehicle collision avoidance,” _IEEE Conference on Decision and Control (CDC)_ , 2017, pp. 2979–2984.  10.1109/CDC.2017.8264092.
  * Shah et al. [2017] Shah, S., Dey, D., Lovett, C., and Kapoor, A., “AirSim: High-Fidelity Visual and Physical Simulation for Autonomous Vehicles,” _Field and Service Robotics (FSR)_ , Springer Proceedings in Advanced Robotics, Vol. 5, 2017, pp. 621–635.
  * Xu et al. [2015] Xu, B., Wang, N., Chen, T., and Li, M., “Empirical Evaluation of Rectified Activations in Convolutional Network,” _Computing Research Repository_ , Vol. abs/1505.00853, 2015.
  * Hessel et al. [2018] Hessel, M., Modayil, J., van Hasselt, H., Schaul, T., Ostrovski, G., Dabney, W., Horgan, D., Piot, B., Azar, M. G., and Silver, D., “Rainbow: Combining Improvements in Deep Reinforcement Learning,” _AAAI Conference on Artificial Intelligence (AAAI)_ , 2018, pp. 3215–3222.
  * Schulman et al. [2017] Schulman, J., Wolski, F., Dhariwal, P., Radford, A., and Klimov, O., “Proximal Policy Optimization Algorithms,” _Computing Research Repository_ , Vol. abs/1707.06347, 2017.

## Appendix

###  .1 Expert Contingency Policy

The expert contingency policy we used in these experiments operates as
follows:

Take control of the aircraft from the RL policy.

Slow quadrotor to a stop.

Take an RGB image with the on-board camera and generate a depth map using the
depth estimation network.

if  _the aircraft is to the right of the middle of the track_ then

if  _the left half of the middle three rows of the depth map do not contain
any cells closer than 10 meters_ then

Fly the aircraft right until it reaches within 0.5 meters of the right
boundary of the track.

else

Fly the aircraft left until it reaches within 0.5 meters of the left boundary
of the track.

end if

else

if  _the right half of the middle three rows of the depth map do not contain
any cells closer than 10 meters_ then

Fly the aircraft left until it reaches within 0.5 meters of the left boundary
of the track.

else

Fly the aircraft right until it reaches within 0.5 meters of the right
boundary of the track.

end if

end if

Return control of the aircraft to the RL policy.

Algorithm 1 Expert Contingency Policy

###  .2 $A^{*}$ Planner Contingency Policy

The $A^{*}$ contingency policy used the $A^{*}$ path planning algorithm to
route the UAV around detected obstacles. A rules-based policy was used to
orient the vehicle before executing $A^{*}$ and to handle the input and output
from the search. The policy procedures are described here.

First, the contingency policy slows the quadrotor to a stop. Procedure A is
then executed. If Procedure A returns that there are no obstacles detected,
then the aircraft is flown forward for 4 meters and control is then returned
the RL policy. If Procedure A returns a path, it will pilot the aircraft along
this path, and then return control of the aircraft to the RL policy.

If Procedure A finds that there are no paths around the obstacle, then the
aircraft is rotated right by 30 degrees. Procedure A is then called again. If
Procedure A returns that there are no obstacles detected, then the aircraft is
flown forward in the current direction its facing for 4 meters or until the
aircraft reaches within 0.5 meters of the boundary of the track. Control of
the aircraft is then returned to the RL policy. If Procedure A returns a path,
it will pilot the aircraft along this path, and then return control of the
aircraft to the RL policy.

If Procedure A again finds that there are no paths around the obstacle, then
the aircraft is rotated back to its original orientation, and then is rotated
left by 30 degrees and the above process is repeated.

If Procedure A finds that there are no paths around the obstacle, then the
aircraft is rotated back to its original orientation, and backed up for 1
meter. The steps described in the previous three paragraphs are then repeated.
If still no path is found at this point, then control of the aircraft is
returned to the RL policy.

####  .2.1 Planning Procedures

Procedure A: First, an RGB image is taken with the on-board camera and a depth
map is generated using the depth estimation network. A $(1\times 16)$
occupancy map is then created using the depth map via the process described in
Procedure B. If no cells in the occupancy map are marked as occupied, the
procedure returns that no obstacles are detected. Otherwise, it will generate
an obstacle map using the process described in Procedure C. Once an obstacle
map has been generated, it will run $A^{*}$ to try to find a route past the
obstacle using the process described in Procedure D. If a path is found, this
path is returned. If no path is found, then "no path found" is returned.

Procedure B: The occupancy map is created by looking at the middle three rows
of the depth estimate (recall that the depth estimates have the shape
$(9\times 16)$. If any column in the three middle rows of the depth estimate
contains a cell closer than 10 meters, then the corresponding column in the
occupancy map is marked as occupied. Otherwise, it is marked as vacant.

Procedure C: First, the lines of sight from the on-board camera are projected
out as lines going forward at 45 degrees from either side of the aircraft. For
each contiguous block of occupied cells in the occupancy map, the minimum
distanced contained in the corresponding columns of the three middle rows of
the depth map is found. The width of each of these cells is found by
calculating how wide the camera’s field of vision is at the distance found
previously, and then dividing this width by 16 (the number of cells in the
occupancy map). An obstacle is then represented as a square, with its nearest
face being located at the minimum distance found earlier. The width of the
square is equal to total the width of the contiguous block of occupied cells
in the occupancy map at the aforementioned distance. The square extends back
to a depth equal to its width.

This process of generating obstacles is repeated for each contiguous block of
occupied cells in the occupancy map. The positions and orientations of the
squares are adjusted according to the angle at which the aircraft was facing
when the obstacle map was generated.

Procedure D: First, a rectangle is defined, the width of which is equal to the
width of the flight track and the depth of which is equal to either 5 meters
or distance of the furthest corner of any obstacle on the obstacle map,
whichever is greater. Then, the lines of sight from the on-board camera are
projected out going forward at 45 degrees from either side of the aircraft at
the angle at which the aircraft is facing.

The goal location that $A^{*}$ is trying to reach is defined as any point with
a distance equal to or greater than the furthest end of the rectangle. In
other words, any point that $A^{*}$ finds that is past the furthest end of the
rectangle counts as a goal location. Additionally, $A^{*}$ is constrained plan
paths which do not go past the left or right lines of the field of vision, and
which do not go outside of any of the sides of the rectangle (except for the
goal side, which it may pass).

The $A^{*}$ algorithm is then used to find a path within the allowable flight
space, avoiding any obstacles, to the goal edge. The $A^{*}$ planner can use
any of the following steps for the aircraft when planning a path: fly forward
1 meter, fly 45 degrees forward and to the right 1 meter, fly 90 degrees to
the right 1 meter, fly 45 degrees back and to the right 1 meter, fly back one
meter, fly 45 degrees back and to the left 1 meter, fly 90 degrees to the left
1 meter, fly 45 degrees forward and to the left one meter.


# ADMM-DAD NET: a deep unfolding network
for analysis compressed sensing

###### Abstract

In this paper, we propose a new deep unfolding neural network based on the
ADMM algorithm for analysis Compressed Sensing. The proposed network jointly
learns a redundant analysis operator for sparsification and reconstructs the
signal of interest. We compare our proposed network with a state-of-the-art
unfolded ISTA decoder, that also learns an orthogonal sparsifier. Moreover, we
consider not only image, but also speech datasets as test examples.
Computational experiments demonstrate that our proposed network outperforms
the state-of-the-art deep unfolding network, consistently for both real-world
image and speech datasets.

Index Terms—  Analysis Compressed Sensing, ADMM, deep neural network, deep
unfolding.

##  1 Introduction

Compressed Sensing (CS) [1] is a modern technique to recover signals of
interest $x\in\mathbb{R}^{n}$ from few linear and possibly corrupted
measurements $y=Ax+e\in\mathbb{R}^{m}$, $m<n$. Iterative optimization
algorithms applied on CS are by now widely used [2], [3], [4]. Recently,
approaches based on deep learning were introduced [5], [6]. It seems promising
to merge these two areas by considering what is called _deep unfolding_. The
latter pertains to unfolding the iterations of well-known optimization
algorithms into layers of a deep neural network (DNN), which reconstructs the
signal of interest.
Related work: Deep unfolding networks have gained much attention in the last
few years [7], [8], [9], because of some advantages they have compared to
traditional DNNs: they are interpretable, integrate prior knowledge about the
signal structure [10], and have a relatively small number of trainable
parameters [11]. Especially in the case of CS, many unfolding networks have
proven to work particularly well. The authors in [12], [13], [14], [15], [16]
propose deep unfolding networks that learn a decoder, which aims at
reconstructing $x$ from $y$. Additionally, most of these networks jointly
learn a dictionary that sparsely represents $x$, along with thresholds used by
the original optimization algorithms.
Motivation: Our work is inspired by the articles [14] and [16], which propose
unfolded versions of the iterative soft thresholding algorithm (ISTA), with
learnable parameters being the sparsifying (orthogonal) basis and/or the
thresholds involved in ISTA. The authors then test their frameworks on
synthetic data and/or real-world image datasets. In a similar spirit, we
derive a decoder by interpreting the iterations of the alternating direction
method of multipliers algorithm [17] (ADMM) as a DNN and call it ADMM Deep
Analysis Decoding (ADMM-DAD) network. We differentiate our approach by
learning a redundant analysis operator as a sparsifier for $x$, i.e. we employ
_analysis sparsity_ in CS. The reason for choosing analysis sparsity over its
synthesis counterpart is due to some advantages the former has. For example,
analysis sparsity provides flexibility in modeling sparse signals, since it
leverages the redundancy of the involved analysis operators. We choose to
unfold ADMM into a DNN, since most of the optimization-based CS algorithms
cannot treat analysis sparsity, while ADMM solves the generalized LASSO
problem [18] which resembles analysis CS. Moreover, we test our decoder on
speech datasets, not only on image ones. To the best of our knowledge, an
unfolded CS decoder has not yet been used on speech datasets. We compare
numerically our proposed network111code available at
www.github.com/vicky-k-19/ADMM-DAD to the state-of-the-art learnable ISTA of
[16], on real-world image and speech data. In all datasets, our proposed
neural architecture outperforms the baseline, in terms of both test and
generalization error.
Key results: Our novelty is twofold: a) we introduce a new ADMM-based deep
unfolding network that solves the analysis CS problem, namely ADMM-DAD net,
that jointly learns an analysis sparsifying operator b) we test ADMM-DAD net
on image and speech datasets (while state-of-the-art deep unfolding networks
are only tested on synthetic data and images so far). Experimental results
demonstrate that ADMM-DAD outperforms the baseline ISTA-net on speech and
images, indicating that the redundancy of the learned analysis operator leads
to a smaller test MSE and generalization error as well.

Notation: For matrices $A_{1},A_{2}\in\mathbb{R}^{N\times N}$, we denote by
$[A_{1};A_{2}]\in\mathbb{R}^{2N\times N}$ their concatenation with respect to
the first dimension, while we denote by
$[A_{1}\,|\,A_{2}]\in\mathbb{R}^{N\times 2N}$ their concatenation with respect
to the second dimension. We denote by $O_{N\times N}$ a square matrix filled
with zeros. We write $I_{N\times N}$ for the real $N\times N$ identity matrix.
For $x\in\mathbb{R},\,\tau>0$, the soft thresholding operator
$\mathcal{S}_{\tau}:\mathbb{R}\mapsto\mathbb{R}$ is defined in closed form as
$\mathcal{S}_{\tau}(x)=\mathrm{sign}(x)\max(0,|x|-\tau)$. For
$x\in\mathbb{R}^{n}$, the soft thresholding operator acts componentwise, i.e.
$(\mathcal{S}_{\tau}(x))_{i}=\mathcal{S}_{\tau}(x_{i})$. For two functions
$f,g:\mathbb{R}^{n}\mapsto\mathbb{R}^{n}$, we write their composition as
$f\circ g:\mathbb{R}^{n}\mapsto\mathbb{R}^{n}$.

##  2 Main Results

Optimization-based analysis CS: As we mentioned in Section 1, the main idea of
CS is to reconstruct a vector $x\in\mathbb{R}^{n}$ from
$y=Ax+e\in\mathbb{R}^{m}$, $m<n$, where $A$ is the so-called measurement
matrix and $e\in\mathbb{R}^{m}$, with $\|e\|_{2}\leq\eta$, corresponds to
noise. To do so, we assume there exists a redundant sparsifying transform
$\Phi\in\mathbb{R}^{N\times n}$ ($N>n$) called the analysis operator, such
that $\Phi x$ is (approximately) sparse. Using analysis sparsity in CS, we
wish to recover $x$ from $y$. A common approach is the analysis
$l_{1}$-minimization problem

$\min_{x\in\mathbb{R}^{n}}\|\Phi x\|_{1}\quad\text{subject
to}\quad\|Ax-y\|_{2}\leq\eta,$ (1)

A well-known algorithm that solves (1) is ADMM, which considers an equivalent
generalized LASSO form of (1), i.e.,

$\min_{x\in\mathbb{R}^{n}}\frac{1}{2}\|Ax-y\|_{2}^{2}+\lambda\|\Phi x\|_{1},$
(2)

with $\lambda>0$ being a scalar regularization parameter. ADMM introduces the
dual variables $z,u\in\mathbb{R}^{N}$, so that (2) is equivalent to

$\min_{x\in\mathbb{R}^{n}}\frac{1}{2}\|Ax-y\|_{2}^{2}+\lambda\|z\|_{1}\quad\text{subject
to}\quad\Phi x-z=0.$ (3)

Now, for $\rho>0$ (penalty parameter), $k\in\mathbb{N}$ and initial points
$(x^{0},z^{0},u^{0})=(0,0,0)$, the optimization problem in (3) can be solved
by the iterative scheme of ADMM:

$\displaystyle
x^{k+1}=(A^{T}A+\rho\Phi^{T}\Phi)^{-1}(A^{T}y+\rho\Phi^{T}(z^{k}-u^{k}))$ (4)
$\displaystyle z^{k+1}=\mathcal{S}_{\lambda/\rho}(\Phi x^{k+1}-u^{k})$ (5)
$\displaystyle u^{k+1}=u^{k}+\Phi x^{k+1}-z^{k+1}.$ (6)

The iterates (4) – (6) are known [17] to converge to a solution $p^{\star}$ of
(3), i.e., $\|Ax^{k}-y\|_{2}^{2}+\|z^{k}\|_{1}\rightarrow p^{\star}$ and $\Phi
x^{k}-z^{k}\rightarrow 0$ as $k\rightarrow\infty$.
Neural network formulation: Our goal is to formulate the previous iterative
scheme as a neural network. We substitute first (4) into the update rules (5)
and (6) and second (5) into (6), yielding

$\displaystyle u^{k+1}=$ $\displaystyle(I-W)u^{k}+Wz^{k}+b$
$\displaystyle-\mathcal{S}_{\lambda/\rho}((-I-W)u^{k}+Wz^{k}+b)$ (7)
$\displaystyle z^{k+1}=$
$\displaystyle\mathcal{S}_{\lambda/\rho}((-I-W)u^{k}+Wz^{k}+b),$

where

$\displaystyle W=$
$\displaystyle\rho\Phi(A^{T}A+\rho\Phi^{T}\Phi)^{-1}\Phi^{T}\in\mathbb{R}^{N\times
N}$ (8) $\displaystyle b=b(y)=$
$\displaystyle\Phi(A^{T}A+\rho\Phi^{T}\Phi)^{-1}A^{T}y\in\mathbb{R}^{N\times
1}.$ (9)

We introduce $v_{k}=[u^{k};z^{k}]\in\mathbb{R}^{2N\times 1}$ and set
$\Theta=(-I-W\,|\,W)\in\mathbb{R}^{N\times 2N}$,
$\Lambda=(I-W\,|\,W)\in\mathbb{R}^{N\times 2N}$ to obtain

$v_{k+1}=\begin{pmatrix}\Lambda\\\ O_{N\times
2N}\end{pmatrix}v_{k}+\begin{pmatrix}b\\\ 0\end{pmatrix}\\\
+\begin{pmatrix}-\mathcal{S}_{\lambda/\rho}(\Theta v_{k}+b)\\\
\mathcal{S}_{\lambda/\rho}(\Theta v_{k}+b)\end{pmatrix}.$ (10)

Now, we set $\tilde{\Theta}=[\Lambda;O_{N\times 2N}]\in\mathbb{R}^{2N\times
2N}$ and $I_{1}=[I_{N\times N};O_{N\times N}]\in\mathbb{R}^{2N\times N}$,
$I_{2}=[-I_{N\times N};I_{N\times N}]\in\mathbb{R}^{2N\times N}$, so that (10)
is transformed into

$v_{k+1}=\tilde{\Theta}v_{k}+I_{1}b+I_{2}\mathcal{S}_{\lambda/\rho}(\Theta
v_{k}+b).$ (11)

Based on (11), we formulate ADMM as a neural network with $L$
layers/iterations, defined as

$\displaystyle f_{1}(y)$
$\displaystyle=I_{1}b(y)+I_{2}\mathcal{S}_{\lambda/\rho}(b(y)),$
$\displaystyle f_{k}(v)$
$\displaystyle=\tilde{\Theta}v+I_{1}b+I_{2}\mathcal{S}_{\lambda/\rho}(\Theta
v+b),\quad k=2,\ldots,L.$

The trainable parameters are the entries of $\Phi$ (or more generally, the
parameters in a parameterization of $\Phi$). We denote the concatenation of
$L$ such layers (all having the same $\Phi$) as

$f^{L}_{\Phi}(y)=f_{L}\circ\dots\circ f_{1}(y).$ (12)

The final output $\hat{x}$ is obtained after applying an affine map $T$
motivated by (4) to the final layer $L$, so that

$\begin{split}\hat{x}=&T(f^{L}_{\Phi}(y))\\\
=&(A^{T}A+\rho\Phi^{T}\Phi)^{-1}(A^{T}y+\rho\Phi^{T}(z^{L}-u^{L})),\end{split}$
(13)

where $[u^{L};z^{L}]=v_{L}$. In order to clip the output in case its norm
falls out of a reasonable range, we add an extra function
$\sigma:\mathbb{R}^{n}\to\mathbb{R}^{n}$ defined as $\sigma(x)=x$ if
$\|x\|_{2}\leq B_{\mathrm{out}}$ and $\sigma(x)=B_{\mathrm{out}}x/\|x\|_{2}$
otherwise, for some fixed constant $B_{\mathrm{out}}>0$. We introduce the
hypothesis class

$\begin{split}\mathcal{H}^{L}=\\{\sigma\circ
h:\,&\mathbb{R}^{m}\mapsto\mathbb{R}^{n}:h(y)=T(f^{L}_{\Phi}(y)),\\\
&\Phi\in\mathbb{R}^{N\times n},N>n\\}\end{split}$ (14)

5 layers |  $25\%$ CS ratio
---|---
Decoder Dataset | SpeechCommands | TIMIT | MNIST | CIFAR10
| test MSE | gen. error | test MSE | gen. error | test MSE | gen. error | test MSE | gen. error
ISTA-net | $0.58\cdot 10^{-2}$ | $0.13\cdot 10^{-2}$ | $0.22\cdot 10^{-3}$ | $0.24\cdot 10^{-4}$ | $0.67\cdot 10^{-1}$ | $0.17\cdot 10^{-1}$ | $0.22\cdot 10^{-1}$ | $0.12\cdot 10^{-1}$
ADMM-DAD | $\bf 0.25\cdot 10^{-2}$ | $\bf 0.16\cdot 10^{-3}$ | $\bf 0.79\cdot 10^{-4}$ | $\bf 0.90\cdot 10^{-5}$ | $\bf 0.23\cdot 10^{-1}$ | $\bf 0.16\cdot 10^{-3}$ | $\bf 0.15\cdot 10^{-1}$ | $\bf 0.11\cdot 10^{-3}$

10 layers | 40% CS ratio | 50% CS ratio
---|---|---
Decoder Dataset | SpeechCommands | TIMIT | SpeechCommands | TIMIT
| test MSE | gen. error | test MSE | gen. error | test MSE | gen. error | test MSE | gen. error
ISTA-net | $0.46\cdot 10^{-2}$ | $0.18\cdot 10^{-2}$ | $0.20\cdot 10^{-3}$ | $0.25\cdot 10^{-4}$ | $0.45\cdot 10^{-2}$ | $0.20\cdot 10^{-2}$ | $0.20\cdot 10^{-3}$ | $0.25\cdot 10^{-4}$
ADMM-DAD | $\bf 0.13\cdot 10^{-2}$ | $\bf 0.58\cdot 10^{-4}$ | $\bf 0.42\cdot 10^{-4}$ | $\bf 0.47\cdot 10^{-5}$ | $\bf 0.87\cdot 10^{-3}$ | $\bf 0.10\cdot 10^{-4}$ | $\bf 0.29\cdot 10^{-4}$ | $\bf 0.30\cdot 10^{-5}$

Table 1: Average test MSE and generalization error for 5-layer decoders (all
datasets) and 10-layer decoders (speech datasets). Bold letters indicate the
best performance between the two decoders.

consisting of all the functions that ADMM-DAD can implement. Then, given the
aforementioned class and a set $\mathcal{S}=\\{(y_{i},x_{i})\\}_{i=1}^{s}$ of
$s$ training samples, ADMM-DAD yields a function/decoder
$h_{\mathcal{S}}\in\mathcal{H}^{L}$ that aims at reconstructing $x$ from
$y=Ax$. In order to measure the difference between $x_{i}$ and
$\hat{x}_{i}=h_{\mathcal{S}}(y_{i})$, $i=1,\dots,s$, we choose the training
mean squared error (train MSE)

$\mathcal{L}_{train}=\frac{1}{s}\sum_{i=1}^{s}\|h(y_{i})-x_{i}\|_{2}^{2}$ (15)

as loss function. The test mean squared error (test MSE) is defined as

$\mathcal{L}_{test}=\frac{1}{d}\sum_{i=1}^{d}\|h(\tilde{y}_{i})-\tilde{x}_{i}\|_{2}^{2},$
(16)

where $\mathcal{D}=\\{(\tilde{y}_{i},\tilde{x}_{i})\\}_{i=1}^{d}$ is a set of
$d$ test data, not used in the training phase. We examine the generalization
ability of the network by considering the difference between the average train
MSE and the average test MSE, i.e.,

$\mathcal{L}_{gen}=|\mathcal{L}_{test}-\mathcal{L}_{train}|.$ (17)

##  3 Experimental Setup

Datasets and pre-processing: We train and test the proposed ADMM-DAD network
on two speech datasets, i.e., SpeechCommands [19] (85511 training and 4890
test speech examples, sampled at 16kHz) and TIMIT [20] (phonemes sampled at
16kHz; we take 70% of the dataset for training and the 30% for testing) and
two image datasets, i.e. MNIST [21] (60000 training and 10000 test $28\times
28$ image examples) and CIFAR10 [22] (50000 training and 10000 test $32\times
32$ coloured image examples). For the CIFAR10 dataset, we transform the images
into grayscale ones. We preprocess the raw speech data, before feeding them to
both our ADMM-DAD and ISTA-net: we downsample each .wav file from 16000 to
8000 samples and segment each downsampled .wav into 10 segments.

Experimental settings: We choose a random Gaussian measurement matrix
$A\in\mathbb{R}^{m\times n}$ and appropriately normalize it, i.e.,
$\tilde{A}=A/\sqrt{m}$. We consider three CS ratios
$m/n\in\\{25\%,40\%,50\%\\}$. We add zero-mean Gaussian noise with standard
deviation $\mathrm{std}=10^{-4}$ to the measurements, set the redundancy ratio
$N/n=5$ for the trainable analysis operator $\Phi\in\mathbb{R}^{N\times n}$,
perform He (normal) initialization for $\Phi$ and choose
$(\lambda,\rho)=(10^{-4},1)$. We also examined different values for
$\lambda,\rho$, as well as treating $\lambda,\rho$ as trainable parameters,
but both settings yielded identical performance. We evaluate ADMM-DAD for $5$
and $10$ layers. All networks are trained using the _Adam_ optimizer [23] and
batch size $128$. For the image datasets, we set the learning rate
$\eta=10^{-4}$ and train the $5$\- and $10$-layer ADMM-DAD for $50$ and $100$
epochs, respectively. For the audio datasets, we set $\eta=10^{-5}$ and train
the $5$\- and $10$-layer ADMM-DAD for $40$ and $50$ epochs, respectively. We
compare ADMM-DAD to the ISTA-net proposed in [16]. For ISTA-net, we set the
best hyper-parameters proposed by the original authors and experiment with $5$
and $10$ layers. All networks are implemented in PyTorch [24]. For our
experiments, we report the average test MSE and generalization error as
defined in (16) and (17) respectively.

(a) Original

(b) 5-layer ADMM-DAD reconstruction

(c) 5-layer ISTA-net reconstruction

Fig. 1: Spectrograms of reconstructed test raw audio file from TIMIT for
$25\%$ CS ratio (top), as well as $50\%$ CS ratio (bottom).

(a) $25\%$ CS ratio

(b) $40\%$ CS ratio

Fig. 2: Average test MSE for increasing $\mathrm{std}$ levels of additive
Gaussian noise. Blue: 10-layer ADMM-DAD, orange: 10-layer ISTA-net.

##  4 Experiments and Results

We compare our decoder to the baseline of the ISTA-net decoder, for $5$ layers
on all datasets with a fixed $25\%$ CS ratio, and for $10$ layers and both
$40\%$ and $50\%$ CS ratios on the speech datasets and report the
corresponding average test MSE and generalization error in Table 1. Both the
test errors and the generalization errors are always lower for our ADMM-DAD
net than for ISTA-net. Overall, the results from Table 1 indicate that the
redundancy of the learned analysis operator improves the performance of ADMM-
DAD net, especially when tested on the speech datasets. Furthermore, we
extract the spectrograms of an example test raw audio file of TIMIT
reconstructed by either of the 5-layer decoders. We use $1024$ FFT points. The
resulting spectrograms for $25\%$ and $50\%$ CS ratio are illustrated in Fig.
1. Both figures indicate that our decoder outperforms the baseline, since the
former distinguishes many more frequencies than the latter. Naturally, the
quality of the reconstructed raw audio file by both decoders increases, as the
CS ratio also increases from $25\%$ to $50\%$. However, ADMM-DAD reconstructs
–even for the $25\%$ CS ratio– a clearer version of the signal compared to
ISTA-net; the latter recovers a significant part of noise, even for the $50\%$
CS ratio. Finally, we examine the robustness of both 10-layer decoders. We
consider noisy measurements in the test set of TIMIT, taken at $25\%$ and
$40\%$ CS ratio, with varying $\mathrm{std}$ of the additive Gaussian noise.
Fig. 2 shows how the average test MSE scales as the noise’s $\mathrm{std}$
increases. Our decoder outperforms the baseline by an order of magnitude and
is robust to increasing levels of noise. This behaviour confirms improved
robustness when learning a redundant sparsifying dictionary instead of an
orthogonal one.

##  5 Conclusion and future directions

In this paper we derived ADMM-DAD, a new deep unfolding network for solving
the analysis Compressed Sensing problem, by interpreting the iterations of the
ADMM algorithm as layers of the network. Our decoder jointly reconstructs the
signal of interest and learns a redundant analysis operator, serving as
sparsifier for the signal. We compared our framework with a state-of-the-art
ISTA-based unfolded network on speech and image datasets. Our experiments
confirm improved performance: the redundancy provided by the learned analysis
operator yields a lower average test MSE and generalization error of our
method compared to the ISTA-net. Future work will include the derivation of
generalization bounds for the hypothesis class defined in (14) similar to
[16]. Additionally, it would be interesting to examine the performance of
ADMM-DAD, when constraining $\Phi$ to a particular class of operators, e.g.,
for $\Phi$ being a tight frame.

## References

  * [1] Emmanuel J. Candès, Justin Romberg, and Terence Tao,  “Robust uncertainty principles: Exact signal reconstruction from highly incomplete frequency information,”  IEEE Transactions on information theory, vol. 52, no. 2, pp. 489–509, 2006.
  * [2] Amir Beck and Marc Teboulle,  “A fast iterative shrinkage-thresholding algorithm for linear inverse problems,”  SIAM journal on imaging sciences, vol. 2, no. 1, pp. 183–202, 2009\.
  * [3] Sundeep Rangan, Philip Schniter, and Alyson K. Fletcher,  “Vector approximate message passing,”  IEEE Transactions on Information Theory, vol. 65, no. 10, pp. 6664–6684, 2019.
  * [4] Elaine T. Hale, Wotao Yin, and Yin Zhang,  “Fixed-point continuation applied to compressed sensing: implementation and numerical experiments,”  Journal of Computational Mathematics, pp. 170–194, 2010.
  * [5] Guang Yang et al.,  “DAGAN: Deep de-aliasing generative adversarial networks for fast compressed sensing MRI reconstruction,”  IEEE transactions on medical imaging, vol. 37, no. 6, pp. 1310–1321, 2017.
  * [6] Ali Mousavi, Gautam Dasarathy, and Richard G. Baraniuk,  “Deepcodec: Adaptive sensing and recovery via deep convolutional neural networks,”  arXiv preprint arXiv:1707.03386, 2017.
  * [7] Karol Gregor and Yann LeCun,  “Learning fast approximations of sparse coding,”  in Proceedings of the 27th international conference on machine learning, 2010, pp. 399–406.
  * [8] Scott Wisdom, John Hershey, Jonathan Le Roux, and Shinji Watanabe,  “Deep unfolding for multichannel source separation,”  in 2016 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP), 2016, pp. 121–125.
  * [9] Carla Bertocchi, Emilie Chouzenoux, Marie-Caroline Corbineau, Jean-Christophe Pesquet, and Marco Prato,  “Deep unfolding of a proximal interior point method for image restoration,”  Inverse Problems, vol. 36, no. 3, pp. 034005, 2020.
  * [10] Yuqing Yang, Peng Xiao, Bin Liao, and Nikos Deligiannis,  “A robust deep unfolded network for sparse signal recovery from noisy binary measurements,”  in 2020 28th European Signal Processing Conference (EUSIPCO). IEEE, 2021, pp. 2060–2064.
  * [11] Anand P. Sabulal and Srikrishna Bhashyam,  “Joint sparse recovery using deep unfolding with application to massive random access,”  in 2020 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP), 2020, pp. 5050–5054.
  * [12] Zhonghao Zhang, Yipeng Liu, Jiani Liu, Fei Wen, and Ce Zhu,  “AMP-Net: Denoising-based deep unfolding for compressive image sensing,”  IEEE Transactions on Image Processing, vol. 30, pp. 1487–1500, 2021\.
  * [13] Daisuke Ito, Satoshi Takabe, and Tadashi Wadayama,  “Trainable ISTA for sparse signal recovery,”  IEEE Transactions on Signal Processing, vol. 67, no. 12, pp. 3113–3125, 2019.
  * [14] Jian Zhang and Bernard Ghanem,  “ISTA-Net: Interpretable optimization-inspired deep network for image compressive sensing,”  in Proceedings of the IEEE conference on computer vision and pattern recognition, 2018, pp. 1828–1837.
  * [15] Jian Sun, Huibin Li, Zongben Xu, et al.,  “Deep ADMM-net for compressive sensing MRI,”  Advances in neural information processing systems, vol. 29, 2016\.
  * [16] Arash Behboodi, Holger Rauhut, and Ekkehard Schnoor,  “Compressive sensing and neural networks from a statistical learning perspective,”  arXiv preprint arXiv:2010.15658, 2020.
  * [17] Stephen Boyd, Neal Parikh, and Eric Chu,  Distributed optimization and statistical learning via the alternating direction method of multipliers,  Now Publishers Inc, 2011.
  * [18] Yunzhang Zhu,  “An augmented ADMM algorithm with application to the generalized lasso problem,”  Journal of Computational and Graphical Statistics, vol. 26, no. 1, pp. 195–204, 2017.
  * [19] Pete Warden,  “Speech commands: A dataset for limited-vocabulary speech recognition,”  arXiv preprint arXiv:1804.03209, 2018.
  * [20] John S. Garofolo,  “Timit acoustic phonetic continuous speech corpus,”  Linguistic Data Consortium, 1993.
  * [21] Yann LeCun, Léon Bottou, Yoshua Bengio, and Patrick Haffner,  “Gradient-based learning applied to document recognition,”  Proceedings of the IEEE, vol. 86, no. 11, pp. 2278–2324, 1998.
  * [22] Alex Krizhevsky, Geoffrey Hinton, et al.,  “Learning multiple layers of features from tiny images,”  2009\.
  * [23] Diederik P. Kingma and Jimmy Ba,  “Adam: A method for stochastic optimization,”  arXiv preprint arXiv:1412.6980, 2014.
  * [24] Nikhil Ketkar,  “Introduction to pytorch,”  in Deep learning with python, pp. 195–208. Springer, 2017.