# Towards Robust Neural Networks via Close-loop Control

Zhuotong Chen1§, Qianxiao Li2,3§, Zheng Zhang1
1Department of Electrical & Computer Engineering, University of California,
Santa Barbara, CA 93106
2Department of Mathematics, National University of Singapore, Singapore
3Institute of High Performance Computing, A*STAR, Singapore
<EMAIL_ADDRESS><EMAIL_ADDRESS><EMAIL_ADDRESS>

###### Abstract

Despite their success in massive engineering applications, deep neural
networks are vulnerable to various perturbations due to their black-box
nature. Recent study has shown that a deep neural network can misclassify the
data even if the input data is perturbed by an imperceptible amount. In this
paper, we address the robustness issue of neural networks by a novel close-
loop control method from the perspective of dynamic systems. Instead of
modifying the parameters in a fixed neural network architecture, a close-loop
control process is added to generate control signals adaptively for the
perturbed or corrupted data. We connect the robustness of neural networks with
optimal control using the geometrical information of underlying data to design
the control objective. The detailed analysis shows how the embedding manifolds
of state trajectory affect error estimation of the proposed method. Our
approach can simultaneously maintain the performance on clean data and improve
the robustness against many types of data perturbations. It can also further
improve the performance of robustly trained neural networks against different
perturbations. To the best of our knowledge, this is the first work that
improves the robustness of neural networks with close-loop control 111A
Pytorch implementation can be found
in:https://github.com/zhuotongchen/Towards-Robust-Neural-Networks-via-Close-
loop-Control.git.

§§footnotetext: Equal contributing authors.

##  1 Introduction

Due to the increasing data and computing power, deep neural networks have
achieved state-of-the-art performance in many applications such as computer
vision, natural language processing and recommendation systems. However, many
deep neural networks are vulnerable to various malicious perturbations due to
their black-box nature: a small (even imperceptible) perturbation of input
data may lead to completely wrong predictions (Szegedy et al., 2013; Nguyen et
al., 2015). This has been a major concern in some safety-critical applications
such as autonomous driving (Grigorescu et al., 2020) and medical image
analysis (Lundervold & Lundervold, 2019). Various perturbations have been
reported, including the $\ell_{p}$ norm based attack (Madry et al., 2017;
Moosavi-Dezfooli et al., 2016; Carlini & Wagner, 2017), semantic perturbation
(Engstrom et al., 2017) etc. On the other side, some algorithms to improve the
robustness against those perturbations have shown great success (Madry et al.,
2017). However, most robustly trained models are tailored for certain types of
perturbations, and they do not work well for other types of perturbations.
Khoury & Hadfield-Menell (2018) showed the non-existence of optimal decision
boundary for any $\ell_{p}$-norm perturbation.

Recent works (E, 2017; Haber & Ruthotto, 2017) have shown the connection
between dynamical systems and neural networks. This dynamic system perspective
provides some interesting theoretical insights about the robustness issue.
Given a set of data $\mathbf{x}_{0}\in\mathbb{R}^{d}$ and its labels
$\mathbf{y}\in\mathbb{R}^{l}$ with a joint distribution $\mathcal{D}$,
training a neural network can be considered as following

$\min\limits_{\bm{\theta}}\mathop{\mathbb{E}}_{(\mathbf{x}_{0},\mathbf{y})\sim\mathcal{D}}[\Phi(\mathbf{x}_{T},\mathbf{y})],\hskip
8.5359pt\text{s.t.}~{}~{}\mathbf{x}_{t+1}=f(\mathbf{x}_{t},\bm{\theta}_{t}),$

where $\bm{\theta}$ are the unknown parameters to train, and $f$, $\Phi$
represent the forward propagation rule and loss function (e.g. cross-entropy)
respectively. The dynamical system perspective interprets the vulnerability of
neural networks as a system instability issue, which addresses the state
trajectory variation under small perturbations applied on initial conditions.
The optimal control theory focuses on developing a control model to adjust the
system state trajectory in an optimal manner. The first work that links and
extends the classical back-propagation algorithm using optimal control theory
was presented in Li et al. (2017), where the direct relationship between the
Pontryagin’s Maximum Principle (Kirk, 1970) and the gradient based network
training was established. Ye et al. (2019) used control theory to adjust the
hyperparameters in the adversarial training algorithm. Han et al. (2018)
established the mathematical basis of the optimal control viewpoint of deep
learning. These existing works on algorithm development are open-loop control
methods since they commonly treat the network weights $\bm{\theta}$ as control
parameters and keep them fixed once the training is done. The fixed control
parameters $\bm{\theta}$ operate optimally for data sampled from the data
distribution $\mathcal{D}$. However, various perturbation methods cause data
distributions to deviate from the true distribution $\mathcal{D}$ (Song et
al., 2017) and cause poor performance with the fixed open-loop control
parameters.

###  1.1 Paper Contributions

$\mathbf{x}_{0}$$\color[rgb]{0,0,1}\definecolor[named]{pgfstrokecolor}{rgb}{0,0,1}\mathcal{E}_{0}(\mathbf{x}_{0})$Cont0$\color[rgb]{0,0,1}\definecolor[named]{pgfstrokecolor}{rgb}{0,0,1}\mathbf{u}_{0}$Layer0$\mathbf{x}_{1}$$\color[rgb]{0,0,1}\definecolor[named]{pgfstrokecolor}{rgb}{0,0,1}\mathcal{E}_{1}(\mathbf{x}_{1})$Cont1$\color[rgb]{0,0,1}\definecolor[named]{pgfstrokecolor}{rgb}{0,0,1}\mathbf{u}_{1}$Layer1$\cdots$LayerT
Figure 1: The structures of feed-forward neural network (black) and the
proposed method (blue).

To address the limitation of using open-loop control methods, we propose the
Close- Loop Control Neural Network (CLC-NN), the first close-loop control
method to improve the robustness of neural networks. As shown in Fig. 1, our
method adds additional blocks to a given $T$-layer neural network: embedding
functions $\mathcal{E}_{t}$, which induce running losses in all layers that
measure the discrepancies between true features and observed features under
input perturbation, then control processes generate control variables
$\mathbf{u}_{t}$ to minimize the total running loss under various data
perturbations. The original neural network can be designed by either standard
training or robust training. In the latter case, our CLC-NN framework can
achieve extra robustness against different perturbations. The forward
propagation rule is thus modified with an extra control parameter
$\mathbf{u}_{t}\in\mathbb{R}^{d^{\prime}}$

$\mathbf{x}_{t+1}=f(\mathbf{x}_{t},\bm{\theta}_{t},\mathbf{u}_{t}).$

Fig. 1 should not be misunderstood as an open-loop control. From the
perspective of dynamic systems, $\mathbf{x}_{0}$ is an initial condition, and
the excitation input signal is $\mathbf{u}_{t}$ (which is $0$ in a standard
feed-forward network). Therefore, the forward signal path is from
$\mathbf{u}_{t}$ to the internal states $\mathbf{x}_{t}$ and then to the
output label $\mathbf{y}$. The path from $\mathbf{x}_{t}$ to the embedding
function $\mathcal{E}_{t}(\mathbf{x}_{t})$ and then to the excitation signal
$\mathbf{u}_{t}$ forms a feedback and closes the whole loop.

The technical contributions of this paper are summarized below:

  * •

The proposed method relies on the well accepted assumption that the data and
hidden state manifolds are low dimensional compared to the ambient dimension
(Fefferman et al., 2016). We study the geometrical information of the data and
hidden layers to define the objective function for control. Given a trained
$T$-layer neural network, a set of embedding functions $\mathcal{E}_{t}$ are
trained off-line by minimizing the reconstruction loss
$\lVert\mathcal{E}(\mathbf{x}_{t})-\mathbf{x}_{t}\rVert$ over some clean data
from $\mathcal{D}$ only. The embedding functions support defining a running
loss required in our control method.

  * •

We define the control problem by dynamic programming and implement the online
iterative solver based on the Pontryagin’s Maximum Principle to avoid the
curse of dimensionality. The proposed close-loop control formulation does not
require prior information of the perturbation.

  * •

We provide a theoretical error bound of the controlled system for the
simplified case with linear activation functions and linear embedding. This
error bound reveals how the close-loop control improves neural network
robustness in the simplest setting.

##  2 Related Works

Many techniques have been reported to improve the robustness of neural
networks, such as data augmentation (Shorten & Khoshgoftaar, 2019), gradient
masking (Liu et al., 2018), etc. We review adversarial training and reactive
defense which are most relevant to this work.

#### Adversarial Training.

Adversarial training is (possibly) the most popular robust training method,
and it solves a min-max robust optimization problem to minimize the worse-case
loss with perturbed data. Adversarial training effectively regularizes the
network’s local Lipschitz constants of the loss surface around the data
manifold (Liu et al., 2018). Zhang et al. (2019) formulated the robustness
training using the Pontryagon’s Maximum Principle, such open-loop control
methods result in a set of fixed parameters that operates optimally on the
considered perturbation. Liu et al. (2020a; b) considered a close-loop
formulation from the differential dynamic programming perspective, this
algorithm is categorized as a open-loop control method because it utilizes the
state feedback information to boost the training convergence and results in a
set of fixed controls for any unseen data. On the contrary, the proposed CLC-
NN formulation adaptively targets on the inputs with different control
parameters and is capable of distinguishing clean data by generating no
control.

#### Reactive Defense.

A reactive defense method tries to reject or pre-process the input data that
may cause mis-classifications. Metzen et al. (2017) rejected perturbed data by
using adversarial detectors that are trained with adversarial data to detect
abnormal data during forward propagation. Song et al. (2017) estimated the
input data distribution $\mathcal{D}$ with a generative model (Oord et al.,
2016) to detect data that does not belong to $\mathcal{D}$, it applies a
greedy method to search the local neighbour of input data for a more
statistically plausible counterpart. This purification process has shown
improved accuracy with adversarial data contaminated by various types of
perturbations. Purification can be considered as a one-step method to solve
the optimal control problem that has the objective function defined over the
initial condition only. On the contrary, the proposed CLC-NN solves the
control problem by the dynamic programming principle and its objective
function is defined over the entire state trajectory, which guarantees the
optimality for the resulted controls.

##  3 The Close-loop Control Framework for Neural Networks

Now we present a close-loop optimal control formulation to address the
robustness issue of deep learning. Consider a neural network consisting of
model parameters $\bm{\theta}$ equipped with external control policy
$\bm{\pi}$, where $\bm{\pi}\in\bm{\Pi}$ is a collection of functions
$\mathbb{R}^{d}\rightarrow\mathbb{R}^{d^{\prime}}$ acting on the state and
outputting the control signal. The feed-forward propagation in a $T$-layer
neural network can be represented as

$\mathbf{x}_{t+1}=f(\mathbf{x}_{t},\bm{\theta}_{t},\bm{\pi}_{t}(\mathbf{x}_{t})),\hskip
8.5359ptt=0,\cdots,T-1.$ (1)

Given a trained network, we solve the following optimization problem

$\min\limits_{\overline{\bm{\pi}}}\mathbb{E}_{(\mathbf{x}_{0},\mathbf{y})\sim\mathcal{D}}\left[J(\mathbf{x}_{0},\mathbf{y},\overline{\bm{\pi}})\right]\vcentcolon=\min\limits_{\overline{\bm{\pi}}}\mathbb{E}_{(\mathbf{x}_{0},\mathbf{y})\sim\mathcal{D}}\left[\Phi(\mathbf{x}_{T},\mathbf{y})+\sum_{s=0}^{T-1}{\cal
L}(\mathbf{x}_{s},\bm{\pi}_{s}(\mathbf{x}_{s}))\right],\;\;s.t.~{}\text{Eq.}~{}(\ref{eq:difference_equation}),$
(2)

where $\overline{\bm{\pi}}$ collects the control policies
$\bm{\pi}_{0},\cdots,\bm{\pi}_{T-1}$ for all layers. Note that (2) differs
from the open-loop control used in standard training. An open-loop control
that treats the network parameters as control variables seeks for a set of
fixed parameters $\bm{\theta}$ to match the output with true label
$\mathbf{y}$ by minimizing the terminal loss $\Phi$, and the running loss
${\cal L}$ defines a regularization for $\bm{\theta}$. However, the terminal
and running losses play different roles when our goal is to improve the
robustness of a neural network by generating some adaptive controls for
different inputs.

#### Challenge of Close-loop Control for Neural Networks.

Optimal control has been well studied in the control community for trajectory
optimization, where one defines the running loss as the error between the
actual state $\mathbf{x}_{t}$ and a reference state
$\mathbf{x}_{t,\text{ref}}$ over time interval $[0,T]$. The resulting control
policy adjusts $\mathbf{x}_{t}$ and makes it approach
$\mathbf{x}_{t,\text{ref}}$. In this paper, we apply the idea of trajectory
optimization to improve the robustness of a neural network via adjusting the
undesired state of $\mathbf{x}_{t}$. However, the formulation is more
challenging in neural networks: we do not have a “reference” state during the
inference process, therefore it is unclear how to define the running loss
${\cal L}$.

In the following, we investigate manifold embedding of the state trajectory to
precisely define the loss functions $\Phi$ and ${\cal L}$ of Eq. (2) required
for the control objective function of a neural network.

###  3.1 Manifold Learning for State Trajectories

#### State Manifold.

Our controller design is based on the “manifold hypothesis”: real-world high
dimensional data can often be embedded in a lower dimensional manifold
$\mathcal{M}$ (Fefferman et al., 2016). Indeed, neural networks extract the
embedded features from $\mathcal{M}$. To fool a well-trained neural network,
the perturbed data often stays away from the data manifold $\mathcal{M}$
(Khoury & Hadfield-Menell, 2018). We consider the data space $Z$
($\mathbf{x}\in Z,\forall\mathbf{x}\sim\mathcal{D}$) as:
$Z=Z_{\parallel}\bigoplus Z_{\perp}$, where $Z_{\parallel}$ contains the
embedded manifold $\mathcal{M}$ and $Z_{\perp}$ is the orthogonal complement
of $Z_{\parallel}$. During forward propagation, the state manifold embedded in
$Z_{\parallel}$ varies at different layers due to both the nonlinear
activation function $f$ and state dimensionality variation. Therefore, we
denote $Z^{t}=Z_{\parallel}^{t}\bigoplus Z_{\perp}^{t}$ as the state space
decomposition at layer $t$ and $\mathcal{M}_{t}\in Z_{\parallel}^{t}$. Once an
input data is perturbed, the main effects of causing misclassifications are in
$Z_{\perp}$. Therefore, it is important to measure how far the possibly
perturbed state $\mathbf{x}_{t}$ deviates from the state manifold
$\mathcal{M}_{t}$.

#### Embedding Function.

Given an embedding function $\mathcal{E}_{t}$ that encodes $\mathbf{x}_{t}$
onto the lower-dimensional manifold $\mathcal{M}_{t}$ and decodes the result
back to the full state space $Z_{t}$, the reconstruction loss
$\lVert\mathcal{E}_{t}(\mathbf{x}_{t})-\mathbf{x}_{t}\rVert$ measures the
deviation of the possibly perturbed state $\mathbf{x}_{t}$ from the manifold
$\mathcal{M}_{t}$. The reconstruction loss is nonzero as long as
$\mathbf{x}_{t}$ has components in $Z_{\perp}^{t}$. The embedding functions
are constructed offline by minimizing the total reconstruction losses over a
clean training data set.

  * •

Linear Case: $\mathcal{E}_{t}(\cdot)$ can be considered as
$\mathbf{V}_{t}^{r}(\mathbf{V}_{t}^{r})^{T}$ where $\mathbf{V}_{t}^{r}$ forms
an orthonormal basis for $Z_{\parallel}^{t}$. Specifically one can first
perform a principle component analysis over a collection of hidden states at
layer $t$, then $\mathbf{V}_{t}^{r}$ can be obtained as the first $r$ columns
of the resulting eigenvectors.

  * •

Nonlinear Case: we choose a convolutional auto-encoder (detailed in Appendix
B) to obtain a representative manifold embedding function ${\cal E}_{t}$ due
to its ease of implementation. Based on the assumption that most perturbations
are in the $Z_{\perp}$ subspace, the embeddings are effective to detect the
perturbations as long as the target manifold is of a low dimension.
Alternative manifold learning methods such as Izenman (2012) may also be
employed.

###  3.2 Formulation for the Close-Loop Control of Neural Networks

#### Control Objectives.

The above embedding function allows us to define a running loss ${\cal L}$:

${\cal
L}(\mathbf{x}_{t},\bm{\pi}_{t}(\mathbf{x}_{t}),\mathcal{E}_{t}(\cdot))=\lVert\mathcal{E}_{t}(\mathbf{x}_{t})-\mathbf{x}_{t}\rVert_{2}^{2}+(\bm{\pi}_{t}(\mathbf{x}_{t}))^{T}\mathbf{R}\bm{\pi}_{t}(\mathbf{x}_{t}).$
(3)

Here the matrix $\mathbf{R}$ defines a regularization term promoting controls
of small magnitudes. In practical implementations, using a diagonal matrix
$\mathbf{R}$ with small elements often helps to improve the performance. Now
we are ready to design the control objective function of CLC-NN. Different
from a standard open-loop control, this work sets the terminal loss $\Phi$ as
zero because no true label is given during inference. Consequently, the close-
loop control formulation in Eq. (2) becomes

$\displaystyle\min\limits_{\overline{\bm{\pi}}}\mathbb{E}_{(\mathbf{x}_{0},\mathbf{y})\sim\mathcal{D}}\left[J(\mathbf{x}_{0},\mathbf{y},\overline{\bm{\pi}})\right]\vcentcolon=\min\limits_{\overline{\bm{\pi}}}\mathbb{E}_{(\mathbf{x}_{0},\mathbf{y})\sim\mathcal{D}}\sum_{t=0}^{T-1}\left[{\cal
L}(\mathbf{x}_{t},\bm{\pi}_{t}(\mathbf{x}_{t}),\mathcal{E}_{t}(\cdot))\right],\hskip
8.5359pts.t.~{}Eq.~{}(\ref{eq:difference_equation}).$ (4)

Assume that the input data is perturbed by a bounded and small amount, i.e.,

$\mathbf{x}_{\epsilon,0}=\mathbf{x}_{0}+\epsilon\cdot\mathbf{z},$

where $\mathbf{z}$ can be either random or adversarial. The proposed CLC-NN
adjusts the perturbed state trajectory $\mathbf{x}_{\epsilon,t}$ such that it
stays at a minimum distance from the desired manifold $\mathcal{M}_{t}$ while
promoting small magnitudes of controls.

#### Intuition.

We use an intuitive example to show how CLC-NN controls the state trajectory
of unseen data samples. We create a synthetic binary classification data set
with $1500$ samples. We train a residual neural network with one hidden layer
of dimension $2$, and adopt the fast gradient sign method (Goodfellow et al.,
2014) to generate adversarial data. Fig. 2 (a) and (b) show the states of
clean data (red and blue) and of perturbed data (black and gray) at $t=0$ and
$t=1$, respectively. The CLC-NN adjusts the state trajectory to reduce the
reconstruction loss as shown in Fig. 2 (c) and (d), where lighter background
color represents lower reconstruction loss. Comparing Fig. 2 (a) with (c), and
Fig. 2 (b) with (d), we see that the perturbed states in Fig. 2 (a) and (b)
deviate from the desired state manifold (light green region) and has a high
reconstruction loss. Running $1000$ iterations of Alg. 1 adjusts the perturbed
states and improves the classification accuracy from $86\%$ to $100\%$.

(a) $\mathbf{x}_{0},\mathbf{x}_{\epsilon,0}$

(b) $\mathbf{x}_{1},\mathbf{x}_{\epsilon,1}$

(c) Controlled $\overline{\mathbf{x}}_{\epsilon,0}$

(d) Controlled $\overline{\mathbf{x}}_{\epsilon,1}$

Figure 2: (a) and (b) show the states of clean data (red and blue) and of
perturbed data (black and gray) at the initial and hidden layers respectively.
The yellow and green backgrounds represent the two classes predicted by the
network. Some of the perturbed data are mis-classified. (c) and (d) show the
adjusted states after the proposed close-loop control. The background
lightness from light to dark represent the increasing reconstruction loss
$\lVert\mathcal{E}_{t}(\mathbf{x}_{t})-\mathbf{x}_{t}\rVert_{2}^{2}$.

##  4 Implementation via the Pontryagin’s Maximum Principle

#### Dynamic Programming for Close-Loop Control (4).

The control problem in Eq. (4) can be solved by the dynamical programming
principle (Bellman, 1952). For simplicity we consider one input data sample,
and define a value function
$V:\mathcal{T}\times\mathbb{R}^{d}\rightarrow\mathbb{R}$ (where
$\mathcal{T}:=\\{0,1,\dots,T-1\\}$). Here $V(t,\mathbf{x})$ represents the
optimal cost-to-go function of Eq. (4) incurred from time $t$ at state
$\mathbf{x}$. One can show that $V(t,\mathbf{x})$ satisfies the dynamic
programming principle

$V(t,\mathbf{x})=\inf\limits_{\bm{\pi}\in\bm{\Pi}}\left[V(t+1,\mathbf{x}+f(\mathbf{x},\bm{\theta}_{t},\bm{\pi}(\mathbf{x})))+{\cal
L}(\mathbf{x},\bm{\pi}(\mathbf{x}),\mathcal{E}_{t}(\cdot))\right].$ (5)

Eq. (5) gives a necessary and sufficient condition for optimality of Eq. (4),
and it is often solved backward in time by discretizing the entire state
space. The state dimension of a modern neural network is at the order of
thousands or even higher, therefore, discretizing the state space and directly
solving Eq. (5) is intractable for real-world applications due to the curse of
dimensionality.

#### Solving (5) via the Pontryagin’s Maximum Principle.

To overcome the computational challenge, the Pontryagin’s Maximum Principle
(Kirk, 1970) converts the intractable dynamical programming into two ordinary
differential equations and a maximization condition. Instead of computing the
control policy $\bm{\pi}$ of Eq. (5), the Pontryagin’s Maximum Principle
provides a necessary condition for the optimality with a set of control
parameters [$\mathbf{u}_{0}^{\ast},\cdots,\mathbf{u}_{T}^{\ast}$]. The mean-
field Pontryagin’s Maximum Principle can be considered when the initial
condition is a batch of i.i.d. samples drawn from $\mathcal{D}$. Specifically,
we trade the intractable computational complexity with processing time for
solving the Hamilton equations and its maximization condition for every newly
observed data. To begin with, we define the Hamiltonian
$H:\mathcal{T}\times\mathbb{R}^{d}\times\mathbb{R}^{d}\times\mathbb{R}^{l}\times\mathbb{R}^{m}\rightarrow\mathbb{R}$
as

$H(t,\mathbf{x}_{t},\mathbf{p}_{t+1},\bm{\theta}_{t},\mathbf{u}_{t})\vcentcolon=\mathbf{p}_{t+1}^{T}\cdot
f(\mathbf{x}_{t},\bm{\theta}_{t},\mathbf{u}_{t})-{\cal
L}(\mathbf{x}_{t},\mathbf{u}_{t},\mathcal{E}_{t}(\cdot)).$ (6)

Let $\mathbf{x}^{\ast}$ denote the corresponding optimally controlled state
trajectory. There exists a co-state process
$\mathbf{p}^{\ast}:[0,T]\rightarrow\mathbb{R}^{d}$ such that the Hamilton’s
equations

$\displaystyle\mathbf{x}_{t+1}^{\ast}=\nabla_{p}H(t,\mathbf{x}_{t}^{\ast},\mathbf{p}_{t}^{\ast},\bm{\theta}_{t},\mathbf{u}_{t}^{\ast}),$
$\displaystyle(\mathbf{x}_{0}^{\ast},\mathbf{y})\sim\mathcal{D},$ (7)
$\displaystyle\mathbf{p}_{t}^{\ast}=\nabla_{x}H(t,\mathbf{x}_{t}^{\ast},\mathbf{p}_{t+1}^{\ast},\bm{\theta}_{t},\mathbf{u}_{t}^{\ast}),$
$\displaystyle\mathbf{p}_{T}^{\ast}=\mathbf{0},$ (8)

are satisfied. The terminal co-state $\mathbf{p}_{T}=\mathbf{0}$, since we do
not consider the terminal loss $\Phi(\mathbf{x}_{T},\mathbf{y})$. Moreover, we
have the Hamiltonian maximization condition

$H(t,\mathbf{x}_{t}^{\ast},\mathbf{p}_{t}^{\ast},\bm{\theta}_{t},\mathbf{u}_{t}^{\ast})\geq
H(t,\mathbf{x}_{t}^{\ast},\mathbf{p}_{t}^{\ast},\bm{\theta}_{t},\mathbf{u}_{t}),\forall\mathbf{u}\in\mathbb{R}^{d^{\prime}}\text{
and }\forall t\in\mathcal{T}.$ (9)

Instead of solving Eq. (5) for the optimal control policy
$\bm{\pi}^{\ast}(\mathbf{x}_{t})$, for a given initial condition, the
Pontryagin’s Maximum Principle seeks for a open-loop optimal solution such
that the global optimum of Eq. (5) is satisfied. The limitation of using the
maximum principle is that the control parameters $\mathbf{u}_{t}^{\ast}$ need
to be solved for every unseen data to achieve the optimal solution.

#### Algorithm Flow.

The numerical implementation of CLC-NN is summarized in Alg. 1. Given a
trained network (either from standard or adversarial training) and a set of
embedding functions, the controls are initialized as
$\mathbf{u}_{t}=\mathbf{0},\forall t\in\mathcal{T}$, because adding random
initialization weakens the robustness performance in general, and clean
trajectory often does not result in any running loss for the gradient update
on the control parameters. In every iteration, a given input $\mathbf{x}_{0}$
is propagated forwardly with Eq. (7) to obtain all the intermediate hidden
states $\mathbf{x}_{t}$ for all $t$ and to accumulate cost $J$. Eq. (8)
backward propagates the co-state $\mathbf{p}_{t}$ and Eq. (9) maximizes the
$t^{th}$ Hamiltonian with current $\mathbf{x}_{t}$ and $\mathbf{p}_{t}$ to
compute the optimal control parameters $\mathbf{u}_{t}^{\ast}$.

Input : Possibly perturbed data $\mathbf{x}_{\epsilon}$, a trained neural
network, embedding functions [$\mathcal{E}_{1},\cdots,\mathcal{E}_{T-1}]$,
${\rm maxItr}$ (maximum number of iterations).

Output : A set of optimal control parameters
$\mathbf{u}_{0}^{\ast},\cdots,\mathbf{u}_{T-1}^{\ast}$.

1 for  _$k=0$ to ${\rm maxItr}$_ do

2 $J_{k}=0$,

3 for  _$t=0$ to $T-1$_ do

4 $\mathbf{x}_{t+1,k}=f(\mathbf{x}_{t,k},\bm{\theta}_{t},\mathbf{u}_{t,k})$,
where $\mathbf{x}_{0,k}=\mathbf{x}_{\epsilon},$ $\triangleright$ Forward
propagation Eq. (7),

5 $J_{k}=J_{k}+{\cal
L}(\mathbf{x}_{t,k},\mathbf{u}_{t,k},\mathcal{E}_{t}(\mathbf{x}_{t,k}))$,
$\triangleright$ Objective function Eq. (4),

6 end for

7 for  _$t=T$ to $1$_ do

8
$\mathbf{p}_{t,k}=\mathbf{p}_{t+1}^{T}\cdot\nabla_{\mathbf{x}_{t}}f(\mathbf{x}_{t,k},\bm{\theta}_{t},\mathbf{u}_{t,k})-\nabla_{\mathbf{x}_{t}}{\cal
L}(\mathbf{x}_{t,k},\mathbf{u}_{t,k},\mathcal{E}_{t}(\mathbf{x}_{t,k}))$,

9 where $\mathbf{p}_{T,k}=\mathbf{0},$ $\triangleright$ Backward propagation
Eq. (8)

10 end for

11 for  _$t=0$ to $T-1$_ do

12
$\mathbf{u}_{t,k+1}=\mathbf{u}_{t,k}+\big{(}\mathbf{p}_{t+1,k}^{T}\cdot\nabla_{\mathbf{u}_{t}}f(\mathbf{x}_{t,k},\bm{\theta}_{t},\mathbf{u}_{t,k})-\nabla_{\mathbf{u}_{t}}{\cal
L}(\mathbf{x}_{t,k},\mathbf{u}_{t,k},\mathcal{E}_{t}(\mathbf{x}_{t,k}))\big{)}$,

13 $\triangleright$ Maximization of Hamiltonian Eq. (9) based on Eq. (6) and
gradient ascent.

14 end for

15

16 end for

Algorithm 1 CLC-NN with the Pontryagin’s Maximum Principle.

##  5 Error Analysis for Simplified Linear Cases

For the ease of analysis, we consider a simplified neural network with linear
activation functions:

$\mathbf{x}_{t+1}=\bm{\theta}_{t}(\mathbf{x}_{t}+\mathbf{u}_{t}),$

and reveal why our proposed method can improve robustness in the simplest
setting. Given a perturbed data sample $\overline{\mathbf{x}}_{\epsilon,0}$,
we denote its perturbation-free counterpart as $\mathbf{x}_{0}$ so that
$\mathbf{z}=\overline{\mathbf{x}}_{\epsilon,0}-\mathbf{x}_{0}$. We consider a
general perturbation where $\mathbf{z}$ is the direct sum of two orthogonal
contributions: $\mathbf{z}^{\parallel}$, which is a perturbation within the
data manifold (subspace), and $\mathbf{z}^{\perp}$, which is a perturbation in
the orthogonal complement of the data manifold. This case is general: if we
consider adversarial attacks, then the perturbation along the orthogonal
complement dominates. In contrast, if we consider random perturbations, then
the two perturbations are on the same scale. Our formulation covers both such
extreme scenarios, together with intermediate cases.

We use an orthogonal projection as the embedding function such that
$\mathcal{E}_{t}=\mathbf{V}_{t}^{r}(\mathbf{V}_{t}^{r})^{T}$, where
$\mathbf{V}_{t}^{r}$ is the first $r$ columns of the eigenvectors computed by
the Principle Component Analysis on a collection of states $\mathbf{x}_{t}$.
The proposed CLC-NN minimizes
$\lVert\overline{\mathbf{x}}_{\epsilon,t}-\mathbf{x}_{t}\rVert_{2}^{2}$ by
reducing the components of $\mathbf{x}_{\epsilon,t}$ that lie in the the
orthogonal complement of $Z_{\parallel}^{t}$. The following theorem provides
an error estimation between $\overline{\mathbf{x}}_{\epsilon,t}$ and
$\mathbf{x}_{t}$.

######  Theorem 1.

For $t\geq 1$, we have the error estimation

$\lVert\overline{\mathbf{x}}_{\epsilon,t}-\mathbf{x}_{t}\rVert_{2}^{2}\leq\lVert\bm{\theta}_{t-1}\cdots\bm{\theta}_{0}\rVert_{2}^{2}\cdot\bigg{(}\alpha^{2t}\lVert\mathbf{z}^{\perp}\rVert_{2}^{2}+\lVert\mathbf{z}^{\parallel}\rVert_{2}^{2}+\gamma_{t}\lVert\mathbf{z}\rVert_{2}^{2}\big{(}\gamma_{t}\alpha^{2}(1-\alpha^{t-1})^{2}+2(\alpha-\alpha^{t})\big{)}\bigg{)},$
(10)

where $\gamma_{t}\vcentcolon=\max\limits_{s\leq
t}\big{(}1+\kappa(\overline{\bm{\theta}}_{s})^{2}\big{)}\lVert\mathbf{I}-\overline{\bm{\theta}}_{s}^{T}\overline{\bm{\theta}}_{s}\rVert_{2}$,
and $\alpha=\frac{c}{1+c}$, $c$ represents the control regularization. In
particular, the equality

$\lVert\overline{\mathbf{x}}_{\epsilon,t}-\mathbf{x}_{t}\rVert_{2}^{2}=\alpha^{2t}\lVert\mathbf{z}^{\perp}\rVert_{2}^{2}+\lVert\mathbf{z}^{\parallel}\rVert_{2}^{2},$
(11)

holds when all $\bm{\theta}_{t}$ are orthogonal.

The detailed derivation is presented in Appendix A. Let us summarize the
insights from Theorem 1.

  * •

The above error estimation is general for any input perturbation. It shows the
working principle behind the proposed CLC-NN on controlling the perturbation
that lies in the orthogonal complement of input subspace
($\mathbf{z}^{\perp}$).

  * •

The above error estimation improves as the control regularization $c$ goes to
$0$ (so $\alpha\rightarrow 0$). It is not the sharpest possible as it relies
on a greedily optimal control at each layer. The globally optimal control
defined by the Ricatti equation may achieve a lower loss when $c\neq 0$.

  * •

When the dimension of embedding subspace $r$ decreases, our control becomes
more effective in reducing
$\lVert\overline{\mathbf{x}}_{\epsilon,t}-\mathbf{x}_{t}\rVert_{2}^{2}$. This
means that the control approach works the best when the data is constrained on
a low dimensional manifold, which is consistent with the manifold hypothesis.
In particular, observe that as $r\rightarrow 0$,
$\lVert\mathbf{z}^{\parallel}\rVert_{2}^{2}\rightarrow 0$

  * •

The obtained upper bound is tight: the estimated upper bound becomes the
actual error if all the forward propagation layers are orthogonal matrices.

Table 1: Experimental results on ResNet-20 from standard training. |  | Accuracy: original model without CLC / CLC-NN + Linear / CLC-NN + Nonlinear
---|---|---
Dataset |  $\epsilon$ | Type of input perturbations
|  |  None |  Manifold |  FGSM |  PGD |  CW
CIFAR-10 |  2 |  92 / 88 / 89 |  24 / 79 / 82 |  21 / 56 / 56 |  0 / 50 / 50 |  8 / 75 / 79
|  4 |  |  5 / 78 / 81 |  11 / 40 / 30 |  0 / 31 / 19 |  0 / 75 / 79
|  8 |  |  1 / 78 / 81 |  8 / 20 / 12 |  0 / 11 / 2 |  0 / 76 / 79
CIFAR-100 |  2 |  69 / 60 / 58 |  9 / 51 / 52 |  9 / 25 / 23 |  0 / 17 / 22 |  4 / 47 / 49
|  4 |  |  3 / 50 / 52 |  5 / 15 / 9 |  0 / 6 / 4 |  1 / 47 / 49
|  8 |  |  2 / 50 / 52 |  4 / 9 / 5 |  0 / 1 / 0 |  0 / 47 / 49

##  6 Numerical Experiments

We test our proposed CLC-NN framework under various input data perturbations.
Here we briefly summarize our experimental settings, and we refer readers to
Appendix B for the details.

  * •

Original Networks without Close-Loop Control. We choose residual neural
networks (He et al., 2016) with ReLU activation functions as our target for
close-loop control. In order to show that CLC-NN can improve the robustness in
various settings, we consider networks from both standard and adversarial
trainings. We consider multiple adversarial training methods: fast gradient
sign method (FGSM) (Goodfellow et al., 2014), project gradient descent (PGD)
(Madry et al., 2017), and the Label smoothing training (Label Smooth) (Hazan
et al., 2017).

  * •

Input Perturbations. In order to test our CLC-NN framework, we perturb the
input data within a radius of $\epsilon$ with $\epsilon=2,4$ and $8$
respectively. We consider various perturbations, including non-adversarial
perturbations with the manifold-based attack (Jalal et al., 2017) (Manifold),
as well as some adversarial attacks such as FGSM, PGD and the CW methods
(Carlini & Wagner, 2017).

  * •

CLC-NN Implementations. We consider both linear and nonlinear embedding in our
close-loop control. Specifically, we employ a principal component analysis
with a $1\%$ truncation error for linear embedding, and convolutional auto-
encoders for nonlinear embedding. We use Adam (Kingma & Ba, 2014) to maximize
the Hamiltonian function (9) and keep the same hyper-parameters (learning
rate, maximum iterations) for each model against all perturbations.

#### Result Summary:

Table 1 and Table 2 show the results for both CIFAR-$10$ and CIFAR-$100$
datasets on some neural networks from both standard training and adversarial
training respectively.

  * •

CLC-NN significantly improves the robustness of neural networks from standard
training. Table 1 shows that the baseline network trained on a clean data set
becomes completely vulnerable (with almost $0\%$ accuracy) under PGD and CW
attacks. Our CLC-NN improves its accuracy to nearly $40\%$ and $80\%$ under
PGD and CW attacks respectively. The accuracy under FGSM attacks has almost
been doubled by our CLC-NN method. The accuracy on clean data is slightly
decreased because the lower-dimensional embedding functions cannot exactly
capture $Z_{\parallel}$ or $\mathcal{M}$.

  * •

CLC-NN further improves the robustness of adversarially trained networks.
Table 2 shows that while an adversarially trained network is inherently robust
against certain types of perturbations, CLC-NN strengthens its robustness
significantly against various perturbations. For instance, CLC-NN improves the
accuracy of an FGSM trained network under PGD and CW attacks by a maximum of
$59\%$ and $81\%$, respectively.

  * •

The robustness improvement of adversarially trained networks is less
significant. This is expected because the trajectory of perturbed data lies on
the embedding subspace $Z_{\parallel}$ if that data sample has been used in
adversarial training. However, our experiments show that applying CLC-NN to
adversarially trained networks can achieve the best performance under most
attacks.

#### Comparison with PixelDefend (Song et al., 2017).

Our method achieves similar performance on CIFAR-10 with slightly different
experimental setting. Specifically, PixelDefend improved the robustness of a
normally trained $62$-layer ResNet from $0\%$ to $78\%$ against CW attack. Our
proposed CLC-NN improves the robustness of a $20$-layer ResNet from $0\%$ to
$81\%$ against CW attacks. Furthermore, we show that CLC-NN is robust against
the manifold-based attack. No result was reported for CIFAR-100 in Song et al.
(2017).

Table 2: Experimental results on robustly trained networks. |  |  | Accuracy: Robustly trained models / CLC-NN + Linear Embedding
---|---|---|---
Dataset |  Training |  $\epsilon$ | Type of input perturbations
|  methods |  |  None |  Manifold |  FGSM |  PGD |  CW
CIFAR-10 |  FGSM |  2 |  92 / 90 |  43 / 82 |  90 / 87 |  1 / 60 |  14 / 81
|  |  4 |  |  17 / 81 |  95 / 90 |  0 / 36 |  2 / 81
|  |  8 |  |  3 / 80 |  96 / 93 |  0 / 10 |  0 / 81
|  Label Smooth |  2 |  93 / 89 |  42 / 78 |  57 / 69 |  10 / 60 |  29 / 76
|  |  4 |  |  20 / 77 |  51 / 61 |  1 / 49 |  11 / 76
|  |  8 |  |  6 / 77 |  40 / 47 |  0 / 26 |  2 / 76
|  PGD |  2 |  83 / 80 |  80 / 78 |  75 / 73 |  74 / 73 |  75 / 75
|  |  4 |  |  78 / 76 |  66 / 66 |  63 / 65 |  66 / 71
|  |  8 |  |  74 / 73 |  49 / 51 |  40 / 46 |  48 / 64
CIFAR-100 |  FGSM |  2 |  69 / 66 |  21 / 55 |  59 / 54 |  1 / 27 |  9 / 57
|  |  4 |  |  8 / 54 |  66 / 55 |  0 / 9 |  2 / 57
|  |  8 |  |  3 / 53 |  67 / 56 |  0 / 1 |  0 / 57
|  Label Smooth |  2 |  69 / 62 |  10 / 43 |  16 / 28 |  1 / 19 |  5 / 46
|  |  4 |  |  3 / 42 |  12 / 19 |  0 / 9 |  1 / 46
|  |  8 |  |  2 / 42 |  8 / 11 |  0 / 2 |  0 / 46
|  PGD |  2 |  58 / 52 |  52 / 50 |  46 / 43 |  45 / 43 |  45 / 47
|  |  4 |  |  49 / 48 |  36 / 36 |  34 / 35 |  35 / 43
|  |  8 |  |  43 / 45 |  22 / 24 |  16 / 21 |  21 / 40

#### Comparison with Reactive Defense

Reactive defenses can be understood as only applying a control at the initial
condition of a dynamical system. Specifically, reactive defense equipped with
linear embedding admits the following dynamics:

$\overline{\mathbf{x}}_{t+1}=f(\overline{\mathbf{x}}_{t},\bm{\theta}_{t}),\hskip
8.5359pts.t.~{}\overline{\mathbf{x}}_{0}=\mathbf{V}_{0}^{r}(\mathbf{V}_{0}^{r})^{T}\mathbf{x}_{\epsilon,0}.$
(12)

By contrast, CLC-NN controls all hidden states and results in a decreasing
error as the number of layers $T$ increases (c.f. Theorem 1). To
quantitatively compare CLC-NN with reactive defense, we implement them with
the same linear embedding functions and against all perturbations. In Table 3,
CLC-NN outperforms reactive defense in almost all cases except that their
performances are case-dependent on clean data.

Table 3: Accuracy comparision of CLC-NN and reactive defense in Eq. (12), with $\epsilon_{attack}$ = $2$ / $4$ / $8$. Here “+” (or “-”) indicates how much CLC-NN outperforms (or underperforms) reactive defense. Method | Type of input perturbations
---|---
|  None |  Manifold |  FGSM |  PGD |  CW
CIFAR-10 |  -3 |  +47 / +63 / +66 |  +27 / +20 / +13 |  +43 / +35 / +25 |  +66 / +76 / +77
CIFAR-100 |  +1 |  +34 / +37 / +38 |  +22 / 0 / +9 |  +44 / +30 / +11 |  +37 / +30 / +16

##  7 Conclusion

We have proposed a close-loop control formulation to improve the robustness of
neural networks. We have studied the embedding of state trajectory during
forward propagation to define the optimal control objective function. The
numerical experiments have shown that our method can improve the robustness of
a trained neural network against various perturbations. We have provided an
error estimation for the proposed method in the linear case. Our current
implementation uses the Pontryagin’s Maximum Principle and an online iterative
algorithm to overcome the intractability of solving a dynamical programming.
This online process adds extra inference time. In the future, we plan to show
the theoretical analysis for the nonlinear embedding case.

#### Acknowledgement

Zhuotong Chen and Zheng Zhang are supported by NSF CAREER Award No. 1846476
and NSF CCF No. 1817037. Qianxiao Li is supported by the start-up grant under
the NUS PYP programme.

## References

  * Bellman (1952) Richard Bellman.  On the theory of dynamic programming.  _Proceedings of the National Academy of Sciences of the United States of America_ , 38(8):716, 1952.
  * Carlini & Wagner (2017) Nicholas Carlini and David Wagner.  Towards evaluating the robustness of neural networks.  In _2017 ieee symposium on security and privacy (sp)_ , pp. 39–57. IEEE, 2017.
  * E (2017) E.  A proposal on machine learning via dynamical systems.  _Communications in Mathematics and Statistics_ , 5(1):1–11, 2017.
  * Engstrom et al. (2017) Logan Engstrom, Dimitris Tsipras, Ludwig Schmidt, and Aleksander Madry.  A rotation and a translation suffice: Fooling cnns with simple transformations.  _arXiv preprint arXiv:1712.02779_ , 1(2):3, 2017\.
  * Fefferman et al. (2016) Charles Fefferman, Sanjoy Mitter, and Hariharan Narayanan.  Testing the manifold hypothesis.  _Journal of the American Mathematical Society_ , 29(4):983–1049, 2016.
  * Goodfellow et al. (2014) Ian J Goodfellow, Jonathon Shlens, and Christian Szegedy.  Explaining and harnessing adversarial examples.  _arXiv preprint arXiv:1412.6572_ , 2014.
  * Grigorescu et al. (2020) Sorin Grigorescu, Bogdan Trasnea, Tiberiu Cocias, and Gigel Macesanu.  A survey of deep learning techniques for autonomous driving.  _Journal of Field Robotics_ , 37(3):362–386, 2020\.
  * Haber & Ruthotto (2017) Eldad Haber and Lars Ruthotto.  Stable architectures for deep neural networks.  _Inverse Problems_ , 34(1):014004, 2017.
  * Han et al. (2018) Jiequn Han, Qianxiao Li, et al.  A mean-field optimal control formulation of deep learning.  _arXiv preprint arXiv:1807.01083_ , 2018.
  * Hazan et al. (2017) Tamir Hazan, George Papandreou, and Daniel Tarlow.  Adversarial perturbations of deep neural networks.  2017\.
  * He et al. (2016) Kaiming He, Xiangyu Zhang, Shaoqing Ren, and Jian Sun.  Deep residual learning for image recognition.  In _Proceedings of the IEEE conference on computer vision and pattern recognition_ , pp. 770–778, 2016.
  * Izenman (2012) Alan Julian Izenman.  Introduction to manifold learning.  _Wiley Interdisciplinary Reviews: Computational Statistics_ , 4(5):439–446, 2012.
  * Jalal et al. (2017) Ajil Jalal, Andrew Ilyas, Constantinos Daskalakis, and Alexandros G Dimakis.  The robust manifold defense: Adversarial training using generative models.  _arXiv preprint arXiv:1712.09196_ , 2017.
  * Khoury & Hadfield-Menell (2018) Marc Khoury and Dylan Hadfield-Menell.  On the geometry of adversarial examples.  _arXiv preprint arXiv:1811.00525_ , 2018.
  * Kingma & Ba (2014) Diederik P Kingma and Jimmy Ba.  Adam: A method for stochastic optimization.  _arXiv preprint arXiv:1412.6980_ , 2014.
  * Kirk (1970) Donald E Kirk.  _Optimal control theory: an introduction_.  Springer, 1970.
  * Li et al. (2017) Qianxiao Li, Long Chen, Cheng Tai, and E Weinan.  Maximum principle based algorithms for deep learning.  _The Journal of Machine Learning Research_ , 18(1):5998–6026, 2017.
  * Liu et al. (2020a) Guan-Horng Liu, Tianrong Chen, and Evangelos A Theodorou.  Differential dynamic programming neural optimizer.  _arXiv preprint arXiv:2002.08809_ , 2020a.
  * Liu et al. (2020b) Guan-Horng Liu, Tianrong Chen, and Evangelos A Theodorou.  A differential game theoretic neural optimizer for training residual networks.  _arXiv preprint arXiv:2007.08880_ , 2020b.
  * Liu et al. (2018) Xuanqing Liu, Minhao Cheng, Huan Zhang, and Cho-Jui Hsieh.  Towards robust neural networks via random self-ensemble.  In _Proceedings of the European Conference on Computer Vision (ECCV)_ , pp. 369–385, 2018.
  * Lundervold & Lundervold (2019) Alexander Selvikvåg Lundervold and Arvid Lundervold.  An overview of deep learning in medical imaging focusing on mri.  _Zeitschrift für Medizinische Physik_ , 29(2):102–127, 2019.
  * Madry et al. (2017) Aleksander Madry, Aleksandar Makelov, Ludwig Schmidt, Dimitris Tsipras, and Adrian Vladu.  Towards deep learning models resistant to adversarial attacks.  _arXiv preprint arXiv:1706.06083_ , 2017.
  * Metzen et al. (2017) Jan Hendrik Metzen, Tim Genewein, Volker Fischer, and Bastian Bischoff.  On detecting adversarial perturbations.  _arXiv preprint arXiv:1702.04267_ , 2017.
  * Moosavi-Dezfooli et al. (2016) Seyed-Mohsen Moosavi-Dezfooli, Alhussein Fawzi, and Pascal Frossard.  Deepfool: a simple and accurate method to fool deep neural networks.  In _Proceedings of the IEEE conference on computer vision and pattern recognition_ , pp. 2574–2582, 2016.
  * Nguyen et al. (2015) Anh Nguyen, Jason Yosinski, and Jeff Clune.  Deep neural networks are easily fooled: High confidence predictions for unrecognizable images.  In _Proceedings of the IEEE conference on computer vision and pattern recognition_ , pp. 427–436, 2015.
  * Oord et al. (2016) Aaron van den Oord, Nal Kalchbrenner, and Koray Kavukcuoglu.  Pixel recurrent neural networks.  _arXiv preprint arXiv:1601.06759_ , 2016.
  * Samangouei et al. (2018) Pouya Samangouei, Maya Kabkab, and Rama Chellappa.  Defense-gan: Protecting classifiers against adversarial attacks using generative models.  _arXiv preprint arXiv:1805.06605_ , 2018.
  * Shorten & Khoshgoftaar (2019) Connor Shorten and Taghi M Khoshgoftaar.  A survey on image data augmentation for deep learning.  _Journal of Big Data_ , 6(1):60, 2019.
  * Song et al. (2017) Yang Song, Taesup Kim, Sebastian Nowozin, Stefano Ermon, and Nate Kushman.  Pixeldefend: Leveraging generative models to understand and defend against adversarial examples.  _arXiv preprint arXiv:1710.10766_ , 2017.
  * Szegedy et al. (2013) Christian Szegedy, Wojciech Zaremba, Ilya Sutskever, Joan Bruna, Dumitru Erhan, Ian Goodfellow, and Rob Fergus.  Intriguing properties of neural networks.  _arXiv preprint arXiv:1312.6199_ , 2013.
  * Ye et al. (2019) Nanyang Ye, Qianxiao Li, and Zhanxing Zhu.  Amata: An annealing mechanism for adversarial training acceleration.  2019\.
  * Zhang et al. (2019) Dinghuai Zhang, Tianyuan Zhang, Yiping Lu, Zhanxing Zhu, and Bin Dong.  You only propagate once: Accelerating adversarial training via maximal principle.  In _Advances in Neural Information Processing Systems_ , pp. 227–238, 2019.

##  Appendix A Appendix A Error Estimation for the Proposed CLC-NN

#### Preliminaries

We define the performance index at time $t$ as

$J(\mathbf{x}_{t},\mathbf{u}_{t})=\frac{1}{2}\lVert\mathbf{Q}_{t}(\mathbf{x}_{t}+\mathbf{u}_{t})\rVert_{2}^{2}+\frac{c}{2}\lVert\mathbf{u}_{t}\rVert_{2}^{2},$
(13)

where $\mathbf{Q}_{t}=\mathbf{I}-\mathbf{V}_{t}^{r}(\mathbf{V}_{t}^{r})^{T}$,
$\mathbf{V}_{t}^{r}$ is the linear projection matrix at time $t$ with only its
first $r$ principle components corresponding to the largest $r$ eigenvalues.
The optimal feedback control is defined as

$\mathbf{u}_{t}^{\ast}(\mathbf{x}_{t})=\arg\min\limits_{\mathbf{u}_{t}}J(\mathbf{x}_{t},\mathbf{u}_{t}),$

due to the linear system and quadratic performance index, the optimal feedback
control admits an analytic solution by taking the gradient of performance
index (Eq. (13)) and setting it to $\mathbf{0}$.

$\displaystyle\nabla_{\mathbf{u}}J(\mathbf{x}_{t},\mathbf{u}_{t})$
$\displaystyle=\nabla_{\mathbf{u}}\bigg{(}\frac{1}{2}\lVert\mathbf{Q}_{t}(\mathbf{x}_{t}+\mathbf{u}_{t})\rVert_{2}^{2}+\frac{c}{2}\lVert\mathbf{u}_{t}\rVert_{2}^{2}\bigg{)},$
$\displaystyle=\mathbf{Q}_{t}^{T}\mathbf{Q}_{t}\mathbf{x}_{t}+\mathbf{Q}_{t}^{T}\mathbf{Q}_{t}\mathbf{u}_{t}+c\cdot\mathbf{u}_{t},$

which leads to the analytic solution of
$\mathbf{u}_{t}^{\ast}(\mathbf{x}_{t})$ as

$\mathbf{u}_{t}^{\ast}(\mathbf{x}_{t})=-(c\cdot\mathbf{I}+\mathbf{Q}_{t}^{T}\mathbf{Q}_{t})^{-1}\mathbf{Q}_{t}^{T}\mathbf{Q}_{t}\mathbf{x}_{t}.$
(14)

The above analytic control solution $\mathbf{u}_{t}^{\ast}$ optimizes the
performance index instantly at time step $t$, the error measured by Eq. (13)
for the dynamical programming solution $\overline{\mathbf{x}}_{\epsilon,t}$
must be smaller or equal to the state trajectory equipped with
$\mathbf{u}_{t}^{\ast}$ define by Eq. (14), which gives a guaranteed upper
bound for the error estimation of the dynamic programming solution.

We define the feedback gain matrix
$\mathbf{K}_{t}=(c\cdot\mathbf{I}+\mathbf{Q}_{t}^{T}\mathbf{Q}_{t})^{-1}\mathbf{Q}_{t}^{T}\mathbf{Q}_{t}$.
Thus, the one-step optimal feedback control can be represented as
$\mathbf{u}_{t}^{\ast}=-\mathbf{K}_{t}\mathbf{x}_{t}$.

The difference between the controlled system applied with perturbation at
initial condition and the uncontrolled system without perturbation is shown

$\displaystyle\overline{\mathbf{x}}_{\epsilon,t+1}-\mathbf{x}_{t+1}$
$\displaystyle=\bm{\theta}_{t}(\overline{\mathbf{x}}_{\epsilon,t}+\mathbf{u}_{t}-\mathbf{x}_{t}),$
$\displaystyle=\bm{\theta}_{t}(\overline{\mathbf{x}}_{\epsilon,t}-\mathbf{K}_{t}\overline{\mathbf{x}}_{\epsilon,t}-\mathbf{x}_{t}).$
(15)

The control objective is to minimize the state components that span the
orthogonal complement of the data manifold
($\mathbf{I}-\mathbf{V}_{t}^{r}(\mathbf{V}_{t}^{r})^{T}$), when the input data
to feedback control only stays in the state manifold, such that
$\lVert(\mathbf{I}-\mathbf{V}_{t}^{r}(\mathbf{V}_{t}^{r})^{T})\mathbf{x}_{t}\rVert_{2}^{2}=0$,
the feedback control $\mathbf{K}_{t}\mathbf{x}_{t}=\mathbf{0}$. The state
difference of Eq. (15) can be further shown by adding a $\mathbf{0}$ term of
$(\bm{\theta}_{t}\mathbf{K}_{t}\mathbf{x}_{t})$

$\displaystyle\overline{\mathbf{x}}_{\epsilon,t+1}-\mathbf{x}_{t+1}$
$\displaystyle=\bm{\theta}_{t}(\mathbf{I}-\mathbf{K}_{t})\overline{\mathbf{x}}_{\epsilon,t}-\bm{\theta}_{t}\mathbf{x}_{t}+\bm{\theta}_{t}\mathbf{K}_{t}\mathbf{x}_{t},$
$\displaystyle=\bm{\theta}_{t}(\mathbf{I}-\mathbf{K}_{t})(\overline{\mathbf{x}}_{\epsilon,t}-\mathbf{x}_{t}).$
(16)

In the following, we show a transformation on the control dynamic term
$(\mathbf{I}-\mathbf{K}_{t})$ based on its definition.

######  Lemma 1.

For $t\geq 0$, we have

$\mathbf{I}-\mathbf{K}_{t}=\alpha\cdot\mathbf{I}+(1-\alpha)\cdot\mathbf{P}_{t},$

where $\mathbf{P}_{t}\vcentcolon=\mathbf{V}_{t}^{r}(\mathbf{V}_{t}^{r})^{T}$,
which is the orthogonal projection onto $Z_{\parallel}^{t}$, and
$\alpha\vcentcolon=\frac{c}{1+c}$ such that $\alpha\in[0,1]$.

###### Proof.

Recall that
$\mathbf{K}_{t}=(c\cdot\mathbf{I}+\mathbf{Q}_{t}^{T}\mathbf{Q}_{t})^{-1}\mathbf{Q}_{t}^{T}\mathbf{Q}_{t}$,
and $\mathbf{Q}_{t}=\mathbf{I}-\mathbf{V}_{t}^{r}(\mathbf{V}_{t}^{r})^{T}$,
$\mathbf{Q}_{t}$ can be diagonalized as following

$\displaystyle\mathbf{Q}_{t}=\mathbf{V}_{t}\begin{bmatrix}0&0&\cdots&0&0\\\
0&0&\cdots&0&0\\\ \vdots&\vdots&\ddots&0&0\\\ 0&0&\cdots&1&0\\\
0&0&\cdots&0&1\\\ \end{bmatrix}\mathbf{V}_{t}^{T},$

where the first $r$ diagonal elements have common value of $0$ and the last
($d-r$) diagonal elements have common value of $1$. Furthermore, the feedback
gain matrix $\mathbf{K}_{t}$ can be diagonalized as

$\displaystyle\mathbf{K}_{t}=\mathbf{V}_{t}\begin{bmatrix}0&0&\cdots&0&0\\\
0&0&\cdots&0&0\\\ \vdots&\vdots&\ddots&0&0\\\ 0&0&\cdots&\frac{1}{1+c}&0\\\
0&0&\cdots&0&\frac{1}{1+c}\\\ \end{bmatrix}\mathbf{V}_{t}^{T},$

where the last ($d-r$) diagonal elements have common value of $\frac{1}{1+c}$.
The control term $(\mathbf{I}-\mathbf{K}_{t})$ thus can be represented as

$\displaystyle\mathbf{I}-\mathbf{K}_{t}=\mathbf{V}_{t}\begin{bmatrix}1&0&\cdots&0&0\\\
0&1&\cdots&0&0\\\ \vdots&\vdots&\ddots&0&0\\\ 0&0&\cdots&\frac{c}{1+c}&0\\\
0&0&\cdots&0&\frac{1}{1+c}\\\ \end{bmatrix}\mathbf{V}_{t}^{T},$

where the first $r$ diagonal elements have common value of $1$ and the last
($d-r$) diagonal elements have common value of $\frac{c}{1+c}$. By denoting
the projection of first $r$ columns as $\mathbf{V}_{t}^{r}$ and last $(d-r)$
columns as $\hat{\mathbf{V}}_{t}^{r}$, it can be further shown as

$\displaystyle\mathbf{I}-\mathbf{K}_{t}$
$\displaystyle=\mathbf{V}_{t}^{r}(\mathbf{V}_{t}^{r})^{T}+\frac{c}{1+c}\big{(}\hat{\mathbf{V}}_{t}^{r}(\hat{\mathbf{V}}_{t}^{r})^{T}\big{)},$
$\displaystyle=\mathbf{P}_{t}+\alpha\big{(}\mathbf{I}-\mathbf{P}_{t}\big{)},$
$\displaystyle=\alpha\cdot\mathbf{I}+(1-\alpha)\cdot\mathbf{P}_{t}.$

∎

#### Oblique Projections.

Let $\mathbf{P}$ be a linear operator on $\mathbb{R}^{d}$,

  * •

We say that $\mathbf{P}$ is an projection if $\mathbf{P}^{2}=\mathbf{P}$.

  * •

$\mathbf{P}$ is an orthogonal projection if
$\mathbf{P}=\mathbf{P}^{T}=\mathbf{P}^{2}$.

  * •

If $\mathbf{P}^{2}=\mathbf{P}$ but $\mathbf{P}\neq\mathbf{P}^{T}$, it is
called an oblique projection.

######  Proposition 2.

For a projection $\mathbf{P}$,

  1. 1.

If $\mathbf{P}$ is an orthogonal projection, then
$\lVert\mathbf{P}\rVert_{2}=1$.

  2. 2.

If $\mathbf{P}$ is an oblique projection, then $\lVert\mathbf{P}\rVert_{2}>1$.

  3. 3.

If $\mathbf{P}$, $\mathbf{Q}$ are two projections such that
$range(\mathbf{P})=range(\mathbf{Q})$, then $\mathbf{P}\mathbf{Q}=\mathbf{Q}$
and $\mathbf{Q}\mathbf{P}=\mathbf{P}$.

  4. 4.

If $\mathbf{P}$ is a projection, then $rank(\mathbf{P})=Tr(\mathbf{P})$.
Furthermore, if $\mathbf{P}$ is an orthogonal projection, then
$rank(\mathbf{P})=\lVert\mathbf{P}\rVert_{F}^{2}=Tr(\mathbf{P}\mathbf{P^{T}})$.

Define for $t\geq 0$

$\begin{cases}\mathbf{P}_{t}^{0}\vcentcolon=\mathbf{P}_{t},\\\
\mathbf{P}_{t}^{(s+1)}\vcentcolon=\bm{\theta}_{t-s-1}^{-1}\mathbf{P}_{t}^{s}\bm{\theta}_{t-s-1},\hskip
8.5359pts=0,1,\ldots,t-1,\end{cases}$

######  Lemma 3.

Let $\mathbf{P}_{t}^{s}$ be defined as above for $0\leq s\leq t$. Then

  1. 1.

$\mathbf{P}_{t}^{s}$ is a projection.

  2. 2.

$\mathbf{P}_{t}^{s}$ is a projection onto $Z_{\parallel}^{t-s}$, i.e.
$range(\mathbf{P}_{t}^{s})=Z_{\parallel}^{t-s}$.

  3. 3.

$\lVert\mathbf{P}_{t}^{s}\rVert_{F}^{2}\leq\kappa(\bm{\theta}_{t-1}\bm{\theta}_{t-2}\ldots\bm{\theta}_{t-s})^{2}\cdot
r,$

where $\kappa(\mathbf{A})$ is the condition number of $\mathbf{A}$, i.e.
$\kappa(\mathbf{A})=\lVert\mathbf{A}\rVert_{2}\cdot\lVert\mathbf{A}^{-1}\rVert_{2}$,
and
$r=rank(Z_{\parallel}^{0})=rank(Z_{\parallel}^{1})=\ldots=rank(Z_{\parallel}^{t})$.

###### Proof.

  1. 1.

We prove it by induction on $s$ for each $t$. For $s=0$,
$\mathbf{P}_{t}^{0}=\mathbf{P}_{t}$, which is a projection by its definition.
Suppose it is true for $s$ such that
$\mathbf{P}_{t}^{s}=\mathbf{P}_{t}^{s}\mathbf{P}_{t}^{s}$, then for $(s+1)$,

$\displaystyle(\mathbf{P}_{t}^{s+1})^{2}$
$\displaystyle=\big{(}\bm{\theta}_{t-s-1}^{-1}\mathbf{P}_{t}^{s}\bm{\theta}_{t-s-1}\big{)}^{2},$
$\displaystyle=\bm{\theta}_{t-s-1}^{-1}\big{(}\mathbf{P}_{t}^{s}\big{)}^{2}\bm{\theta}_{t-s-1},$
$\displaystyle=\bm{\theta}_{t-s-1}^{-1}\mathbf{P}_{t}^{s}\bm{\theta}_{t-s-1},$
$\displaystyle=\mathbf{P}_{t}^{s+1}.$

  2. 2.

We prove it by induction on $s$ for each $t$. For $s=0$,
$\mathbf{P}_{t}^{0}=\mathbf{P}_{t}$, which is the orthogonal projection onto
$Z_{\parallel}^{t}$. Suppose that it is true for $s$ such that
$\mathbf{P}_{t}^{s}$ is a projection onto $Z_{\parallel}^{t-s}$, then for
$(s+1)$,
$\mathbf{P}_{t}^{s+1}=\bm{\theta}_{t-s-1}^{-1}\mathbf{P}_{t}^{s}\bm{\theta}_{t-s-1}$,
which implies

$\displaystyle range(\mathbf{P}_{t}^{s+1})$
$\displaystyle=range(\bm{\theta}_{t-s-1}^{-1}\mathbf{P}_{t}^{s}),$
$\displaystyle=\\{\bm{\theta}_{t-s-1}^{-1}\mathbf{x}:\mathbf{x}\in
Z_{\parallel}^{t-s}\\},$ $\displaystyle=Z_{\parallel}^{t-s-1}.$

  3. 3.

We use the inequalities
$\lVert\mathbf{A}\mathbf{B}\rVert_{F}\leq\lVert\mathbf{A}\rVert_{2}\lVert\mathbf{B}\rVert_{F}$,
and
$\lVert\mathbf{A}\mathbf{B}\rVert_{F}\leq\lVert\mathbf{A}\rVert_{F}\lVert\mathbf{B}\rVert_{2}$.
By the definition of $\mathbf{P}_{t}^{s}$,

$\mathbf{P}_{t}^{s}=\big{(}\bm{\theta}_{t-1}\bm{\theta}_{t-2}\cdots\bm{\theta}_{t-s}\big{)}^{-1}\mathbf{P}_{t}^{0}\big{(}\bm{\theta}_{t-1}\bm{\theta}_{t-2}\cdots\bm{\theta}_{t-s}\big{)},$

we have the following

$\displaystyle\lVert\mathbf{P}_{t}^{s}\rVert_{F}^{2}$
$\displaystyle\leq\lVert\big{(}\bm{\theta}_{t-1}\bm{\theta}_{t-2}\cdots\bm{\theta}_{t-s}\big{)}^{-1}\rVert_{2}^{2}\cdot\lVert\big{(}\bm{\theta}_{t-1}\bm{\theta}_{t-2}\cdots\bm{\theta}_{t-s}\big{)}\rVert_{2}^{2}\cdot\lVert\mathbf{P}_{t}^{0}\rVert_{F}^{2},$
$\displaystyle\leq\kappa(\bm{\theta}_{t-1}\bm{\theta}_{t-2}\cdots\bm{\theta}_{t-s})^{2}\cdot
r,$ $\displaystyle\textnormal{Lemma}~{}\ref{proposition:projections}(4).$

∎

The following Lemma uses the concept of oblique projection to show a recursive
relationship to project any $t^{th}$ state space of Eq. (A) back to the input
data space.

######  Lemma 4.

Define for $0\leq s\leq t$,

$\mathbf{G}_{t}^{s}\vcentcolon=\alpha\cdot\mathbf{I}+(1-\alpha)\mathbf{P}_{t}^{s}.$

Then, Eq. (A) can be written as

$\overline{\mathbf{x}}_{\epsilon,t}-\mathbf{x}_{t}=(\bm{\theta}_{t-1}\bm{\theta}_{t-2}\cdots\bm{\theta}_{0})(\mathbf{G}_{t-1}^{t-1}\mathbf{G}_{t-2}^{t-2}\cdots\mathbf{G}_{0}^{0})(\overline{\mathbf{x}}_{\epsilon,0}-\mathbf{x}_{0}),\hskip
8.5359ptt\geq 1.$

###### Proof.

We prove it by induction on $t$. For $t=1$, by the definition of
$\mathbf{G}_{t}^{s}$ and transformation from Lemma 1,

$\displaystyle\overline{\mathbf{x}}_{\epsilon,1}-\mathbf{x}_{1}$
$\displaystyle=\bm{\theta}_{0}(\mathbf{I}-\mathbf{K}_{0})(\overline{\mathbf{x}}_{\epsilon,0}-\mathbf{x}_{0}),$
$\displaystyle\textnormal{Eq.}~{}(\ref{eq:controlled_dynamics}),$
$\displaystyle=\bm{\theta}_{0}(\alpha\cdot\mathbf{I}+(1-\alpha)\cdot\mathbf{P}_{0})(\overline{\mathbf{x}}_{\epsilon,0}-\mathbf{x}_{0}),$
$\displaystyle\textnormal{Lemma}~{}\ref{lemma:control_matrix},$
$\displaystyle=\bm{\theta}_{0}\mathbf{G}_{0}^{0}(\overline{\mathbf{x}}_{\epsilon,0}-\mathbf{x}_{0}).$

Suppose that it is true for
$(\overline{\mathbf{x}}_{\epsilon,t}-\mathbf{x}_{t})$, by using Eq. (A) and
Lemma 1, we have

$\displaystyle\overline{\mathbf{x}}_{\epsilon,t+1}-\mathbf{x}_{t+1}$
$\displaystyle=\bm{\theta}_{t}(\mathbf{I}-\mathbf{K}_{t})(\overline{\mathbf{x}}_{\epsilon,t}-\mathbf{x}_{t}),$
$\displaystyle\textnormal{Eq.}~{}(\ref{eq:controlled_dynamics}),$
$\displaystyle=\bm{\theta}_{t}(\alpha\cdot\mathbf{I}-(1-\alpha)\cdot\mathbf{P}_{t})(\overline{\mathbf{x}}_{\epsilon,t}-\mathbf{x}_{t}),$
$\displaystyle\textnormal{Lemma}~{}\ref{lemma:control_matrix},$
$\displaystyle=\bm{\theta}_{t}\mathbf{G}_{t}^{0}(\bm{\theta}_{t-1}\bm{\theta}_{t-2}\cdots\bm{\theta}_{0})(\mathbf{G}_{t-1}^{t-1}\mathbf{G}_{t-2}^{t-2}\cdots\mathbf{G}_{0}^{0})(\overline{\mathbf{x}}_{\epsilon,0}-\mathbf{x}_{0}).$
(17)

Recall the definitions of
$\mathbf{P}_{t}^{(s+1)}\vcentcolon=\bm{\theta}_{t-s-1}^{-1}\mathbf{P}_{t}^{s}\bm{\theta}_{t-s-1}$,
and
$\mathbf{G}_{t}^{s}\vcentcolon=\alpha\cdot\mathbf{I}+(1-\alpha)\mathbf{P}_{t}^{s}$,
we have the following

$\displaystyle\mathbf{G}_{t}^{s+1}$
$\displaystyle=\alpha\cdot\mathbf{I}+(1-\alpha)\cdot\mathbf{P}_{t}^{(s+1)},$
$\displaystyle=\alpha\cdot\mathbf{I}+(1-\alpha)\cdot\bm{\theta}_{t-s-1}^{-1}\mathbf{P}_{t}^{s}\bm{\theta}_{t-s-1},$
$\displaystyle=\bm{\theta}_{t-s-1}^{-1}\big{(}\alpha\cdot\mathbf{I}+(1-\alpha)\cdot\mathbf{P}_{t}^{s}\big{)}\bm{\theta}_{t-s-1},$
$\displaystyle=\bm{\theta}_{t-s-1}^{-1}\mathbf{G}_{t}^{s}\bm{\theta}_{t-s-1},$

which results in the equality for the oblique projections. Furthermore,

$\bm{\theta}_{t-s-1}\mathbf{G}_{t}^{(s+1)}=\mathbf{G}_{t}^{s}\bm{\theta}_{t-s-1}.$

Applying the above to Eq. (A) results in

$\displaystyle\overline{\mathbf{x}}_{\epsilon,t+1}-\mathbf{x}_{t+1}$
$\displaystyle=\bm{\theta}_{t}\mathbf{G}_{t}^{0}(\bm{\theta}_{t-1}\bm{\theta}_{t-2}\cdots\bm{\theta}_{0})(\mathbf{G}_{t-1}^{t-1}\mathbf{G}_{t-2}^{t-2}\cdots\mathbf{G}_{0}^{0})(\overline{\mathbf{x}}_{\epsilon,0}-\mathbf{x}_{0}),$
$\displaystyle=(\bm{\theta}_{t}\bm{\theta}_{t-1})\mathbf{G}_{t}^{1}(\bm{\theta}_{t-2}\bm{\theta}_{t-3}\cdots\bm{\theta}_{0})(\mathbf{G}_{t-1}^{t-1}\mathbf{G}_{t-2}^{t-2}\cdots\mathbf{G}_{0}^{0})(\overline{\mathbf{x}}_{\epsilon,0}-\mathbf{x}_{0}),$
$\displaystyle=(\bm{\theta}_{t}\bm{\theta}_{t-1}\bm{\theta}_{t-2})\mathbf{G}_{t}^{2}(\bm{\theta}_{t-3}\bm{\theta}_{t-4}\cdots\bm{\theta}_{0})(\mathbf{G}_{t-1}^{t-1}\mathbf{G}_{t-2}^{t-2}\cdots\mathbf{G}_{0}^{0})(\overline{\mathbf{x}}_{\epsilon,0}-\mathbf{x}_{0}),$
$\displaystyle=(\bm{\theta}_{t}\bm{\theta}_{t-1}\cdots\bm{\theta}_{0})(\mathbf{G}_{t}^{t}\mathbf{G}_{t-1}^{t-1}\cdots\mathbf{G}_{0}^{0})(\overline{\mathbf{x}}_{\epsilon,0}-\mathbf{x}_{0}).$

∎

######  Lemma 5.

Let

$\mathbf{F}_{t}\vcentcolon=\mathbf{G}_{t-1}^{(t-1)}\mathbf{G}_{t-2}^{(t-2)}\cdots\mathbf{G}_{0}^{0},\hskip
8.5359ptt\geq 1.$

Then,

$\mathbf{F}_{t}=\alpha^{t}\cdot\mathbf{I}+(1-\alpha)\sum_{s=0}^{t-1}\alpha^{s}\mathbf{P}_{s}^{s}.$

###### Proof.

We prove it by induction on $t$. Recall the definition of
$\mathbf{G}_{t}^{s}\vcentcolon=\alpha\cdot\mathbf{I}+(1-\alpha)\cdot\mathbf{P}_{t}^{s}$.
When $t=1$,

$\mathbf{F}_{1}=\mathbf{G}_{0}^{0}=\alpha\cdot\mathbf{I}+(1-\alpha)\cdot\mathbf{P}_{0}^{0}.$

Suppose that it is true for $t$ such that

$\mathbf{F}_{t}=\mathbf{G}_{t-1}^{(t-1)}\mathbf{G}_{t-2}^{(t-2)}\cdots\mathbf{G}_{0}^{0}=\alpha^{t}\cdot\mathbf{I}+(1-\alpha)\sum_{s=0}^{t-1}\alpha^{s}\mathbf{P}_{s}^{s},$

for $(t+1)$,

$\displaystyle\mathbf{F}_{t+1}$ $\displaystyle=\mathbf{G}_{t}^{t}F(t),$
$\displaystyle=(\alpha\cdot\mathbf{I}+(1-\alpha)\cdot\mathbf{P}_{t}^{t})\mathbf{F}_{t},$
$\displaystyle=(\alpha\cdot\mathbf{I}+(1-\alpha)\cdot\mathbf{P}_{t}^{t})(\alpha^{t}\cdot\mathbf{I}+(1-\alpha)\sum_{s=0}^{t-1}\alpha^{s}\mathbf{P}_{s}^{s}),$
$\displaystyle=\alpha^{t+1}\cdot\mathbf{I}+\alpha^{t}(1-\alpha)\mathbf{P}_{t}^{t}+(1-\alpha)^{2}\sum_{s=0}^{t-1}\alpha^{s}\cdot\mathbf{P}_{t}^{t}\mathbf{P}_{s}^{s}+\alpha(1-\alpha)\sum_{s=0}^{t-1}\alpha^{s}\cdot\mathbf{P}_{s}^{s}.$

Recall Lemma 3,
$range(\mathbf{P}_{t}^{t})=range(\mathbf{P}_{s}^{s})=Z_{\parallel}^{0}$.
According to Proposition 2 (3),
$\mathbf{P}_{t}^{t}\mathbf{P}_{s}^{s}=\mathbf{P}_{s}^{s}$. Hence,

$\displaystyle\mathbf{F}_{t+1}$
$\displaystyle=\alpha^{t+1}\cdot\mathbf{I}+\alpha^{t}(1-\alpha)\cdot\mathbf{P}_{t}^{t}+(1-\alpha)\sum_{s=0}^{t-1}\alpha^{s}\cdot\mathbf{P}_{s}^{s},$
$\displaystyle=\alpha^{t+1}\cdot\mathbf{I}+(1-\alpha)\sum_{s=0}^{t}\alpha^{s}\cdot\mathbf{P}_{s}^{s}.$

∎

######  Lemma 6.

Let $\mathbf{V}\in\mathbb{R}^{d\times r}$ be a matrix whose columns are an
orthogonal basis for a subspace $\mathcal{D}$, and
$\bm{\theta}\in\mathbb{R}^{d\times d}$ be invertible. Let
$\mathbf{P}=\mathbf{V}\mathbf{V}^{T}$ be the orthogonal projection onto
$\mathcal{D}$. Denote by $\hat{\mathbf{P}}$ the orthogonal projection onto
$\bm{\theta}\mathcal{D}\vcentcolon=\\{\bm{\theta}\mathbf{x}:~{}\mathbf{x}\in\mathcal{D}\\}$.
Then

  1. 1.

$\bm{\theta}^{-1}\hat{\mathbf{P}}\bm{\theta}$ is an oblique projection onto
$\mathcal{D}$.

  2. 2.

$\lVert\bm{\theta}^{-1}\hat{\mathbf{P}}\bm{\theta}-\mathbf{P}\rVert_{2}\leq\big{(}1+\kappa(\bm{\theta})^{2}\big{)}\cdot\lVert\mathbf{I}-\bm{\theta}^{T}\bm{\theta}\rVert_{2}$.

In general, the last inequality shows that
$\bm{\theta}^{-1}\hat{\mathbf{P}}\bm{\theta}=\mathbf{P}$, if $\bm{\theta}$ is
orthogonal.

###### Proof.

  1. 1.

$(\bm{\theta}^{-1}\hat{\mathbf{P}}\bm{\theta})^{2}=\bm{\theta}^{-1}\hat{\mathbf{P}}^{2}\bm{\theta}=\bm{\theta}^{-1}\hat{\mathbf{P}}\bm{\theta}$,
therefore, $\bm{\theta}^{-1}\hat{\mathbf{P}}\bm{\theta}$ is an projection.

  2. 2.

Since $\hat{\mathbf{P}}$ is orthogonal projection onto the row space of
$\bm{\theta}\mathbf{V}$, then

$\displaystyle\hat{\mathbf{P}}$
$\displaystyle=\bm{\theta}\mathbf{V}\big{[}(\bm{\theta}\mathbf{V})^{T}(\bm{\theta}\mathbf{V})\big{]}^{-1}(\bm{\theta}\mathbf{V})^{T},$
$\displaystyle=\bm{\theta}\mathbf{V}\big{[}\mathbf{V}^{T}\bm{\theta}^{T}\bm{\theta}\mathbf{V}\big{]}^{-1}\mathbf{V}^{T}\bm{\theta}^{T}.$
$\displaystyle\bm{\theta}^{-1}\hat{\mathbf{P}}\bm{\theta}=\mathbf{V}\big{[}\mathbf{V}^{T}\bm{\theta}^{T}\bm{\theta}\mathbf{V}\big{]}^{-1}\mathbf{V}^{T}\bm{\theta}^{T}\bm{\theta}.$

Furthermore,

$\displaystyle\lVert\bm{\theta}^{-1}\hat{\mathbf{P}}\bm{\theta}-\mathbf{P}\rVert_{2}$
$\displaystyle=\lVert\mathbf{V}\big{[}\mathbf{V}^{T}\bm{\theta}^{T}\bm{\theta}\mathbf{V}\big{]}^{-1}\mathbf{V}^{T}\bm{\theta}^{T}\bm{\theta}-\mathbf{V}\mathbf{V}^{T}\rVert_{2},$
$\displaystyle\leq\lVert\mathbf{V}\big{[}\mathbf{V}^{T}\bm{\theta}^{T}\bm{\theta}\mathbf{V}\big{]}^{-1}\mathbf{V}^{T}\bm{\theta}^{T}\bm{\theta}-\mathbf{V}\mathbf{V}^{T}\bm{\theta}^{T}\bm{\theta}\rVert_{2}+\lVert\mathbf{V}\mathbf{V}^{T}\bm{\theta}^{T}\bm{\theta}-\mathbf{V}\mathbf{V}^{T}\rVert_{2},$
$\displaystyle\leq\lVert\mathbf{V}\big{(}[\mathbf{V}^{T}\bm{\theta}^{T}\bm{\theta}\mathbf{V}]^{-1}-\mathbf{I}\big{)}\mathbf{V}^{T}\rVert_{2}\cdot\lVert\bm{\theta}^{T}\bm{\theta}\rVert_{2}+\lVert\bm{\theta}^{T}\bm{\theta}-\mathbf{I}\rVert_{2},$
$\displaystyle\leq\lVert[\mathbf{V}^{T}\bm{\theta}^{T}\bm{\theta}\mathbf{V}]^{-1}\rVert_{2}\cdot\lVert\mathbf{I}-\mathbf{V}^{T}\bm{\theta}^{T}\bm{\theta}\mathbf{V}\rVert_{2}\cdot\lVert\bm{\theta}^{T}\bm{\theta}\rVert_{2}+\lVert\bm{\theta}^{T}\bm{\theta}-\mathbf{I}\rVert_{2},$
$\displaystyle\leq\lVert[\mathbf{V}^{T}\bm{\theta}^{T}\bm{\theta}\mathbf{V}]^{-1}\rVert_{2}\cdot\lVert\mathbf{I}-\bm{\theta}^{T}\bm{\theta}\rVert_{2}\cdot\lVert\bm{\theta}^{T}\bm{\theta}\rVert_{2}+\lVert\bm{\theta}^{T}\bm{\theta}-\mathbf{I}\rVert_{2}.$

We further bound
$\lVert[\mathbf{V}^{T}\bm{\theta}^{T}\bm{\theta}\mathbf{V}]^{-1}\rVert_{2}$.

$\displaystyle\lVert[\mathbf{V}^{T}\bm{\theta}^{T}\bm{\theta}\mathbf{V}]^{-1}\rVert_{2}$
$\displaystyle=\big{(}\lambda_{min}(\mathbf{V}^{T}\bm{\theta}^{T}\bm{\theta}\mathbf{V})\big{)}^{-1},$
$\displaystyle=\big{(}\inf\limits_{\lVert\mathbf{x}\rVert_{2}=1}\mathbf{x}^{T}\mathbf{V}^{T}\bm{\theta}^{T}\bm{\theta}\mathbf{V}\mathbf{x}\big{)}^{-1},$
$\displaystyle\leq\big{(}\inf\limits_{\lVert\mathbf{x}^{\prime}\rVert_{2}=1}(\mathbf{x}^{\prime})^{T}\bm{\theta}^{T}\bm{\theta}\mathbf{x}^{\prime}\big{)}^{-1},$
$\displaystyle=\big{(}\lambda_{min}(\bm{\theta}^{T}\bm{\theta})\big{)}^{-1},$
$\displaystyle=\lVert(\bm{\theta}^{T}\bm{\theta})^{-1}\rVert_{2}.$

Hence, we have

$\displaystyle\lVert\bm{\theta}^{-1}\hat{\mathbf{P}}\bm{\theta}-\mathbf{P}\rVert_{2}$
$\displaystyle\leq\big{(}1+\lVert\bm{\theta}^{T}\bm{\theta}\rVert_{2}\cdot\lVert(\bm{\theta}^{T}\bm{\theta})^{-1}\rVert_{2}\big{)}\cdot\lVert\mathbf{I}-\bm{\theta}^{T}\bm{\theta}\rVert_{2},$
$\displaystyle=\big{(}1+\kappa(\bm{\theta})^{2}\big{)}\cdot\lVert\mathbf{I}-\bm{\theta}^{T}\bm{\theta}\rVert_{2}.$

∎

######  Corollary 1.

Let $t\geq 1$. Then for each $s=0,1,\cdots,t$, we have

$\lVert\mathbf{P}_{s}^{s}-\mathbf{P}_{0}\rVert_{2}\leq\big{(}1+\kappa(\overline{\bm{\theta}}_{s})^{2}\big{)}\cdot\lVert\mathbf{I}-\overline{\bm{\theta}}_{s}^{T}\overline{\bm{\theta}}_{s}\rVert_{2},$

where

  * •

$\overline{\theta}\vcentcolon=\bm{\theta}_{s-1}\cdots\bm{\theta}_{0},~{}s\geq
1$,

  * •

$\overline{\theta}\vcentcolon=\mathbf{I},~{}s=0$.

Observe that
$\mathbf{P}_{s}^{s}=(\overline{\bm{\theta}}_{s})^{-1}\mathbf{P}_{s}\overline{\bm{\theta}}_{s}$.
Using Lemma 6, we arrive at the main theorem.

######  Theorem 1.

For $t\geq 1$, we have the error estimation

$\lVert\overline{\mathbf{x}}_{\epsilon,t}-\mathbf{x}_{t}\rVert_{2}^{2}\leq\lVert\bm{\theta}_{t-1}\cdots\bm{\theta}_{0}\rVert_{2}^{2}\cdot\bigg{(}\alpha^{2t}\lVert\mathbf{z}^{\perp}\rVert_{2}^{2}+\lVert\mathbf{z}^{\parallel}\rVert_{2}^{2}+\gamma_{t}\lVert\mathbf{z}\rVert_{2}^{2}\big{(}\gamma_{t}\alpha^{2}(1-\alpha^{t-1})^{2}+2(\alpha-\alpha^{t})\big{)}\bigg{)}.$

where $\gamma_{t}\vcentcolon=\max\limits_{s\leq
t}\big{(}1+\kappa(\overline{\bm{\theta}}_{s})^{2}\big{)}\lVert\mathbf{I}-\overline{\bm{\theta}}_{s}^{T}\overline{\bm{\theta}}_{s}\rVert_{2}$,
and $\alpha=\frac{c}{1+c}$, $c$ represents the control regularization. In
particular, the equality

$\lVert\overline{\mathbf{x}}_{\epsilon,t}-\mathbf{x}_{t}\rVert_{2}^{2}=\alpha^{2t}\lVert\mathbf{z}^{\perp}\rVert_{2}^{2}+\lVert\mathbf{z}^{\parallel}\rVert_{2}^{2}.$

holds when all $\bm{\theta}_{t}$ are orthogonal.

###### Proof.

The input perturbation
$\mathbf{z}=\overline{\mathbf{x}}_{\epsilon,0}-\mathbf{x}_{0}$ can be written
as $\mathbf{z}=\mathbf{z}^{\parallel}+\cdot\mathbf{z}^{\perp}$, where
$\mathbf{z}^{\parallel}\in Z_{\parallel}$ and $\mathbf{z}^{\perp}\in
Z_{\perp}$, where $\mathbf{z}^{\parallel}$ and $\mathbf{z}^{\perp}$ are
vectors such that

  * •

$\mathbf{z}^{\parallel}\cdot\mathbf{z}^{\perp}=0$ almost surely.

  * •

$\mathbf{z}^{\parallel}$, $\mathbf{z}^{\perp}$ have uncorrelated components.

  * •

$\mathbf{z}^{\parallel}\in\mathcal{D}$, and
$\mathbf{z}^{\perp}\in\mathcal{D}^{\perp}$.

Since $\mathbf{z}^{\parallel}$ and $\mathbf{z}^{\perp}$ are orthogonal almost
surely, recall Lemma 4,

$\displaystyle\lVert\overline{\mathbf{x}}_{\epsilon,t}-\mathbf{x}_{t}\rVert_{2}^{2}$
$\displaystyle=\lVert(\bm{\theta}_{t-1}\bm{\theta}_{t-2}\cdots\bm{\theta}_{0})(\mathbf{G}_{t-1}^{t-1}\cdots\mathbf{G}_{0}^{0})z\rVert_{2}^{2},$
$\displaystyle\leq\lVert\bm{\theta}_{t-1}\bm{\theta}_{t-2}\cdots\bm{\theta}_{0}\rVert_{2}^{2}\cdot\lVert(\mathbf{G}_{t-1}^{t-1}\cdots\mathbf{G}_{0}^{0})z\rVert_{2}^{2},$
(18)

For the term
$\lVert(\mathbf{G}_{t-1}^{t-1}\cdots\mathbf{G}_{0}^{0})z\rVert_{2}^{2}$,
recall Lemma 5,

$\displaystyle\lVert(\mathbf{G}_{t-1}^{t-1}\cdots\mathbf{G}_{0}^{0})z\rVert_{2}^{2}$
$\displaystyle=\lVert\Big{(}\alpha^{t}\cdot\mathbf{I}+(1-\alpha)\sum_{s=0}^{t-1}\alpha^{s}\cdot\mathbf{P}_{s}^{s}\Big{)}\mathbf{z}\rVert_{2}^{2},$
$\displaystyle=\lVert\alpha^{t}\mathbf{z}+(1-\alpha)\sum_{s=0}^{t-1}\alpha^{s}\mathbf{P}_{0}\mathbf{z}+(1-\alpha)\sum_{s=0}^{t-1}\alpha^{s}(\mathbf{P}_{s}^{s}-\mathbf{P}_{0})\mathbf{z}\rVert_{2}^{2},$
$\displaystyle=\lVert\alpha^{t}\mathbf{z}+(1-\alpha^{t})\mathbf{z}^{\parallel}+(1-\alpha)\sum_{s=0}^{t-1}\alpha^{s}(\mathbf{P}_{s}^{s}-\mathbf{P}_{0})\mathbf{z}\rVert_{2}^{2},$

in the above, $\mathbf{P}_{0}$ is an orthogonal projection on $t=0$ (input
data space), therefore, $\mathbf{P}_{0}\mathbf{z}=\mathbf{z}^{\parallel}$.
Furthermore, when $s=0$, $\mathbf{P}_{s}^{s}-\mathbf{P}_{0}=\mathbf{0}$. Thus,

$\displaystyle\hskip
11.38092pt\lVert(\mathbf{G}_{t-1}^{t-1}\cdots\mathbf{G}_{0}^{0})z\rVert_{2}^{2}$
$\displaystyle=\alpha^{2t}\lVert\mathbf{z}\rVert_{2}^{2}+(1-\alpha^{t})^{2}\lVert\mathbf{z}^{\parallel}\rVert_{2}^{2}+(1-\alpha)^{2}\sum_{s,q=1}^{t-1}\alpha^{s}\alpha^{q}\mathbf{z}^{T}(\mathbf{P}_{s}^{s}-\mathbf{P}_{0})^{T}(\mathbf{P}_{q}^{q}-\mathbf{P}_{0})\mathbf{z}$
$\displaystyle\hskip
14.22636pt+2\alpha^{t}(1-\alpha^{t})\lVert\mathbf{z}^{\parallel}\rVert_{2}^{2}+2\alpha^{t}(1-\alpha)\sum_{s=1}^{t-1}\alpha^{s}\mathbf{z}^{T}(\mathbf{P}_{s}^{s}-\mathbf{P}_{0})\mathbf{z}$
$\displaystyle\hskip
14.22636pt+2(1-\alpha^{t})(1-\alpha)\sum_{s=1}^{t-1}\alpha^{s}(\mathbf{z}^{\parallel})^{T}(\mathbf{P}_{s}^{s}-\mathbf{P}_{0})\mathbf{z},$
$\displaystyle=\alpha^{2t}\lVert\mathbf{z}^{\perp}\rVert_{2}^{2}+\big{(}\alpha^{2t}+2\alpha^{t}(1-\alpha^{t})+(1-\alpha^{t})^{2}\big{)}\lVert\mathbf{z}^{\parallel}\rVert_{2}^{2}$
$\displaystyle\hskip
14.22636pt+(1-\alpha)^{2}\sum_{s,q=1}^{t-1}\alpha^{s}\alpha^{q}\mathbf{z}^{T}(\mathbf{P}_{s}^{s}-\mathbf{P}_{0})^{T}(\mathbf{P}_{q}^{q}-\mathbf{P}_{0})\mathbf{z}+2\alpha^{t}(1-\alpha)\sum_{s=1}^{t-1}\alpha^{s}\mathbf{z}^{T}(\mathbf{P}_{s}^{s}-\mathbf{P}_{0})\mathbf{z}$
$\displaystyle\hskip
14.22636pt+2(1-\alpha^{t})(1-\alpha)\sum_{s=1}^{t-1}\alpha^{s}(\mathbf{z}^{\parallel})^{T}(\mathbf{P}_{s}^{s}-\mathbf{P}_{0})\mathbf{z},$
$\displaystyle=\alpha^{2t}\lVert\mathbf{z}^{\perp}\rVert_{2}^{2}+\lVert\mathbf{z}^{\parallel}\rVert_{2}^{2}+(1-\alpha)^{2}\sum_{s,q=1}^{t-1}\alpha^{s}\alpha^{q}\mathbf{z}^{T}(\mathbf{P}_{s}^{s}-\mathbf{P}_{0})^{T}(\mathbf{P}_{q}^{q}-\mathbf{P}_{0})\mathbf{z}$
$\displaystyle\hskip
14.22636pt+2\alpha^{t}(1-\alpha)\sum_{s=1}^{t-1}\alpha^{s}\mathbf{z}^{T}(\mathbf{P}_{s}^{s}-\mathbf{P}_{0})\mathbf{z}+2(1-\alpha^{t})(1-\alpha)\sum_{s=1}^{t-1}\alpha^{s}(\mathbf{z}^{\parallel})^{T}(\mathbf{P}_{s}^{s}-\mathbf{P}_{0})\mathbf{z}.$

Using Corollary 1, we have

  * •

$\displaystyle\mathbf{z}^{T}(\mathbf{P}_{s}^{s}-\mathbf{P}_{0})\mathbf{z}$
$\displaystyle\leq\lVert\mathbf{z}\rVert_{2}^{2}\cdot\lVert\mathbf{P}_{s}^{s}-\mathbf{P}_{0}\rVert_{2},$
$\displaystyle\leq\gamma_{t}\lVert\mathbf{z}\rVert_{2}^{2}.$

  * •

$\displaystyle\mathbf{z}^{T}(\mathbf{P}_{s}^{s}-\mathbf{P}_{0})^{T}(\mathbf{P}_{q}^{q}-\mathbf{P}_{0})\mathbf{z}$
$\displaystyle\leq\lVert\mathbf{z}\rVert_{2}^{2}\cdot\lVert\mathbf{P}_{s}^{s}-\mathbf{P}_{0}\rVert_{2}\cdot\lVert\mathbf{P}_{q}^{q}-\mathbf{P}_{0}\rVert_{2},$
$\displaystyle\leq\gamma_{t}^{2}\lVert\mathbf{z}\rVert_{2}^{2}.$

  * •

$\displaystyle(\mathbf{z}^{\parallel})^{T}(\mathbf{P}_{s}^{s}-\mathbf{P}_{0})\mathbf{z}$
$\displaystyle\leq\gamma_{t}\lVert\mathbf{z}^{\parallel}\rVert_{2}\cdot\lVert\mathbf{z}\rVert_{2},$
$\displaystyle\leq\gamma_{t}\lVert\mathbf{z}\rVert_{2}^{2}.$

Thus, we have

$\displaystyle\lVert(\mathbf{G}_{t-1}^{t-1}\cdots\mathbf{G}_{0}^{0})z\rVert_{2}^{2}$
$\displaystyle\leq\alpha^{2t}\lVert\mathbf{z}^{\perp}\rVert_{2}^{2}+\lVert\mathbf{z}^{\parallel}\rVert_{2}^{2}+\alpha^{2}(1-\alpha^{t-1})^{2}\gamma_{t}^{2}\lVert\mathbf{z}\rVert_{2}^{2}+2\alpha^{t+1}(1-\alpha^{t-1})\gamma_{t}\lVert\mathbf{z}\rVert_{2}^{2}$
$\displaystyle\hskip
14.22636pt+2\alpha(1-\alpha^{t})(1-\alpha^{t-1})\gamma_{t}\lVert\mathbf{z}\rVert_{2}^{2},$
$\displaystyle=\alpha^{2t}\lVert\mathbf{z}^{\perp}\rVert_{2}^{2}+\lVert\mathbf{z}^{\parallel}\rVert_{2}^{2}+\gamma_{t}\lVert\mathbf{z}\rVert_{2}^{2}\big{(}\gamma_{t}\alpha^{2}(1-\alpha^{t-1})^{2}+2(\alpha-\alpha^{t})\big{)}.$

Recall the error estimation in Eq. (18),

$\displaystyle\lVert\overline{\mathbf{x}}_{\epsilon,t}-\mathbf{x}_{t}\rVert_{2}^{2}$
$\displaystyle\leq\lVert\bm{\theta}_{t-1}\bm{\theta}_{t-2}\cdots\bm{\theta}_{0}\rVert_{2}^{2}\cdot\lVert(\mathbf{G}_{t-1}^{t-1}\cdots\mathbf{G}_{0}^{0})z\rVert_{2}^{2},$
$\displaystyle\leq\lVert\bm{\theta}_{t-1}\cdots\bm{\theta}_{0}\rVert_{2}^{2}\cdot\bigg{(}\alpha^{2t}\lVert\mathbf{z}^{\perp}\rVert_{2}^{2}+\lVert\mathbf{z}^{\parallel}\rVert_{2}^{2}+\gamma_{t}\lVert\mathbf{z}\rVert_{2}^{2}\big{(}\gamma_{t}\alpha^{2}(1-\alpha^{t-1})^{2}+2(\alpha-\alpha^{t})\big{)}\bigg{)}.$

In the specific case, when all $\bm{\theta}_{t}$ are orthogonal,

$\displaystyle\gamma_{t}\vcentcolon$ $\displaystyle=\max\limits_{s\leq
t}\big{(}1+\kappa(\overline{\bm{\theta}}_{s})^{2}\big{)}\lVert\mathbf{I}-\overline{\bm{\theta}}_{s}^{T}\overline{\bm{\theta}}_{s}\rVert_{2}$
$\displaystyle=0.$

Thus,

$\displaystyle\lVert\overline{\mathbf{x}}_{\epsilon,t}-\mathbf{x}_{t}\rVert_{2}^{2}=\alpha^{2t}\lVert\mathbf{z}^{\perp}\rVert_{2}^{2}+\lVert\mathbf{z}^{\parallel}\rVert_{2}^{2}.$

∎

##  Appendix B Appendix B Details of Experimental Setting

###  B.1 Network Configurations

Since the proposed CLC-NN optimizes the entire state trajectory, it is
important to have a relatively smooth state trajectory, in which case, when
the reconstruction loss
$\lVert\mathcal{E}_{t}(\mathbf{x}_{t})-\mathbf{x}_{t}\rVert_{2}^{2}$ at layer
$t$ is small, the reconstruction losses at its adjacent layers should be
small. For this reason, we use residual neural network (He et al., 2016) as
network candidate to retain smoother dynamics. The configuration of the
residual neural network used for both CIFAR-$10$ and CIFAR-$100$ is shown in
Tab. 4.

Table 4: ResNet for Both CIFAR-10 and CIFAR-100 Structure |  Configuration
---|---
Initial Layer |  Conv2d (input channel = $3$, output channel = $16$, kernel size = $3\times 3$), BatchNorm2d(channel = $16$), Relu()
Residual Block 1 |  $\\{$Conv2d (input channel = $16$, output channel = $16$, kernel size = $3\times 3$), BatchNorm2d(channel = $16$), Relu(), Shortcut()$\\}$ $\times 6$
Residual Block 2 |  Conv2d (input channel = $16$, output channel = $32$, kernel size = $3\times 3$), BatchNorm2d(channel = $16$), Relu(), Shortcut(), $\\{$Conv2d (input channel = $32$, output channel = $32$, kernel size = $3\times 3$), BatchNorm2d(channel = $32$), Relu(), Shortcut()$\\}$ $\times 5$
Residual Block 3 |  Conv2d (input channel = $32$, output channel = $64$, kernel size = $3\times 3$), BatchNorm2d(channel = $64$), Relu(), Shortcut(), $\\{$Conv2d (input channel = $64$, output channel = $64$, kernel size = $3\times 3$), BatchNorm2d(channel = $64$), Relu(), Shortcut()$\\}$ $\times 5$
Final Layer |  Fully Connected $(64,10)$

Based on the configuration of residual neural network shown in Tab. 4, we
construct $4$ embedding functions applied at input space, outputs of initial
layer, residual block $1$ and residual block $2$. The output of residual block
$3$ is embedded with a linear orthogonal projection. We randomly select $5000$
clean training data to collect state trajectories at all $5$ locations.

  * •

For the linear orthogonal projections: we apply the Principle Component
Analysis on each of the state collections. We retain the first $r$ columns of
the resulted basis, such that
$r=\arg\min\\{i:\frac{\lambda_{1}+\ldots+\lambda_{i}}{\lambda_{1}\ldots+\lambda_{d}}\geq
1-\delta\\}$, where $\delta=0.1$.

  * •

For the nonlinear embedding: we train $4$ convolutional auto-encoders for the
input space, outputs of the initial layer and residual blocks $1,2$. All of
the embedding functions are trained individually. We adopt shallow
convolutional auto-encoder structure to gain fast inference speed, in which
case, CLC-NN equipped with linear embedding often outperform the nonlinear
embedding as shown in Tab. 1. The configuration of all $4$ convolutional auto-
encoders are shown in Tab. 5.

Table 5: Convolutional Auto-Encoders

Structure |  Configuration
---|---
Encoder |  Conv2d (input channel = $c_{1}$, output channel = $c_{2}$, kernel size = $4\times 4$, stride = $2\times 2$, padding = $1\times 1$ ), ELU(alpha=$1$), BatchNorm2d(channel = $c_{2}$), Conv2d (input channel = $c_{2}$, output channel = $c_{3}$, kernel size = $4\times 4$, stride = $2\times 2$, padding = $1\times 1$ ), ELU(alpha=$1$).
Decoder |  ConvTranspose2d (input channel = $c_{3}$, output channel = $c_{2}$, kernel size = $4\times 4$, stride = $2\times 2$, padding = $1\times 1$), ELU(alpha=$1$), ConvTranspose2d (input channel = $c_{2}$, output channel = $c_{1}$, kernel size = $4\times 4$, stride = $2\times 2$, padding = $1\times 1$),

Auto-encoder Index |  0 |  1 |  2 |  3
---|---|---|---|---
Channel Dimensions [$c_{1}$, $c_{2}$, $c_{3}$] |  [$3,18,36$] |  [$16,36,72$] |  [$16,36,72$] |  [$32,128,256$]

###  B.2 Perturbations and Defensive Training

In this section, we show details about the perturbations and robust networks
that have been considered in this work. For the adversarial training objective
function,

$\min\limits_{\bm{\theta}\in\Theta}\max\limits_{\mathbf{x}_{\epsilon,0}=\Delta(\mathbf{x}_{0},\epsilon)}\mathop{\mathbb{E}}_{(\mathbf{x}_{0},\mathbf{y})\sim\mathcal{D}}[(1-\lambda)\cdot\Phi_{i}(\mathbf{x}_{\epsilon,T},\mathbf{y},\bm{\theta})+\lambda\cdot\Phi_{i}(\mathbf{x}_{T},\mathbf{y},\bm{\theta})],$

where $\Delta(\mathbf{x}_{0},\epsilon)$ generates a perturbed data from given
input $\mathbf{x}_{0}$ within the range of $\epsilon$. $\lambda$ balances
between standard accuracy and robustness. We choose $\lambda=0.5$ in all
adversarial training.

#### For robust networks,

we consider both perturbation agnostic and non-agnostic methods. For the
perturbation agnostic adversarial training algorithms equipped
$\Delta(\mathbf{x}_{0},\epsilon)$, the resulted network that is the most
robust against the $\Delta(\mathbf{x}_{0},\epsilon)$ perturbation. On the
contrary, perturbation non-agnostic robust training methods are often robust
against many types of perturbations.

  * •

Adversarial training with the fast gradient sign method (FGSM) (Goodfellow et
al., 2014) considers perturbed data as follows.

$\mathbf{x}_{\epsilon,0}=\mathbf{x}_{0}+sign(\nabla_{\mathbf{x}_{0}}\Phi(\mathbf{x}_{T},\mathbf{y})),\hskip
8.5359pt(\mathbf{x}_{0},y)\sim\mathcal{D},$

where $sign(\cdot)$ outputs the sign of the input. In which case, FGSM
considers the worse case within the range of $\epsilon$ along the gradient
$\nabla_{\mathbf{x}_{0}}\Phi(\mathbf{x}_{T},\mathbf{y})$ increasing direction.
Due to the worse case consideration, it does not scale well for deep networks,
for this reason, we adversarially train the network with FGSM with
$\epsilon=4$, which is half of the maximum perturbation considered in this
paper.

  * •

The label smoothing training (Label Smooth) (Hazan et al., 2017) does not
utilize any perturbation information $\Delta(\mathbf{x}_{0},\epsilon)$. It
converts one-hot labels into soft targets by setting the correct class as
$1-\epsilon$, while other classes have value of $\frac{\epsilon}{N-1}$, where
$\epsilon$ is a small constant and $N$ is number of classes. Specifically, we
choose $\epsilon=0.9$ in this paper.

  * •

Adversarial training with the project gradient descent (PGD) (Madry et al.,
2017) generates adversarial data by iteratively run FGSM with small step size,
which results in stronger perturbations compared with FGSM within the same
range $\epsilon$. We use $7$-step of $\epsilon=2$ to generate adversarial data
for robust training.

#### For Perturbations,

we consider the maximum range of $\epsilon=2,4,8$ to test the performance the
network robustness against both strong and weak perturbations. For this work,
we test network robustness with the manifold-based attack (Jalal et al.,
2017), FGSM (Goodfellow et al., 2014), $20$-step of PGD (Madry et al., 2017)
and the CW attack (Carlini & Wagner, 2017).

###  B.3 Online Optimization

#### Optimization Methods.

we use Adam (Kingma & Ba, 2014) to maximize the Hamiltonian Eq. (9) with
default setting. In which case, solving the PMP brings in extra computational
cost for inference.

Each online iteration of solving the PMP requires a combination of forward
propagation (Eq. (7)), backward propagation (Eq. (8)) and a maximization
w.r.t. the control parameters (Eq. (9)), which has computational cost
approximately the same as performing gradient descent on training a neural
network for one iteration. For the numerical results presented in the paper,
we choose the maximum iteration that gives the best performance from one of
$[5,10,20,30,50]$.

##  Appendix C More Numerical Experiments

The proposed CLC-NN is designed to be compatible with existing open-loop
trained. We show extra experiments by employing the proposed CLC-NN on two
baseline models, DenseNet-$40$ (Table 6).

Table 6: Experimental results on DenseNet-40 from standard training. |  | Accuracy: original without CLC / CLC-NN + Nonlinear Embedding
---|---|---
Dataset |  $\epsilon$ | Type of input perturbations
|  |  None |  Manifold |  FGSM |  PGD |  CW
CIFAR-10 |  2 |  92 / 89 |  19 / 67 |  27 / 57 |  0 / 52 |  5 / 79
|  4 |  |  4 / 56 |  20 / 48 |  0 / 37 |  0 / 79
|  8 |  |  1 / 59 |  15 / 33 |  0 / 17 |  0 / 79
CIFAR-100 |  2 |  70 / 64 |  8 / 25 |  8 / 24 |  0 / 25 |  3 / 47
|  4 |  |  2 / 24 |  4 / 15 |  0 / 10 |  0 / 45
|  8 |  |  1 / 24 |  3 / 7 |  0 / 5 |  0 / 45

The layer-wise projection performs orthogonal projection on the hidden state.
We define the local cost function at the $t^{th}$ layer as follows

$J(\mathbf{x}_{t},\mathbf{u}_{t})=\frac{1}{2}\lVert\mathbf{Q}_{t}(\mathbf{x}_{t}+\mathbf{u}_{t})\rVert_{2}^{2}+\frac{c}{2}\lVert\mathbf{u}_{t}\rVert_{2}^{2},$

the layer-wise achieves the optimal solution at local time $t$,

$\mathbf{u}_{t}^{\ast}(\mathbf{x}_{t})=\arg\min\limits_{\mathbf{u}_{t}}J(\mathbf{x}_{t},\mathbf{u}_{t}).$

However, the layer-wise optimal control solution does not guarantee the
optimum across all layers. In Table 7, we launch comparisons between the
proposed CLC-NN with layer-wise projection. Specifically, under all
perturbations the proposed CLC-NN outperforms layer-wise projection.

Table 7: Comparison between CLC-NN and layer-wise projection. |  | Accuracy: Layer-wise projection / CLC-NN + Linear embedding
---|---|---
Dataset |  $\epsilon$ | Type of input perturbations
|  |  None |  Manifold |  FGSM |  PGD |  CW
CIFAR-10 |  2 |  92 / 88 |  68 / 79 |  29 / 56 |  4 / 50 |  51 / 75
|  4 |  |  64 / 78 |  15 / 40 |  0 / 31 |  50 / 75
|  8 |  |  64 / 78 |  10 / 20 |  0 / 11 |  49 / 76
CIFAR-100 |  2 |  67 / 60 |  45 / 51 |  13 / 25 |  2 / 17 |  35 / 47
|  4 |  |  44 / 50 |  8 / 15 |  0 / 6 |  34 / 47
|  8 |  |  44 / 50 |  5 / 9 |  0 / 1 |  34 / 47

##  Appendix D Robustness Against Manifold-based Attack

The manifold-based attack (Jalal et al., 2017) (denoted as Manifold) has shown
great success on breaking down the manifold-based defenses (Samangouei et al.,
2018). The proposed CLC-NN can successfully defend such specifically design
adversarial attack for manifold-based defenses and improves the robustness
accuracy from $1\%$ to $81\%$ for the standard trained model in Cifar-$10$,
and $2\%$ to $52\%$ in Cifar-$100$.

We provide detailed explanation for the successful defense of the proposed
CLC-NN against such strong adversarial attack. Exsiting manifold-based defense
(Samangouei et al., 2018) focuses on detecting and de-noising the input
components that do not lie within the underlying manifold. The overpowered
attack proposed in Jalal et al. (2017) searches adversarial attack with in the
embedded latent space, which is undetectable for the manifold-based defenses
and caused complete failure defense.

In the real implementation, the manifold-based attack (Jalal et al., 2017) is
detectable and controllable under the proposed framework due to the following
reason. The numerically generated manifold embedding functions are not ideal.
The error sources of non-ideal embedding functions are mainly due to the
algorithm that used to compute the manifold, the architecture of embedding
function, and the distribution shift between training and testing data
(embedding functions of training data do not perfectly agree with testing
data). In which case, even the perturbation is undetectable and non-
controllable at initial layer, as it is propagated into hidden layers, each
layer amplifies such perturbation, therefore, the perturbation becomes
detectable and controllable in hidden layers.

We randomly select the batch of testing data to generate the manifold-based
attack following the same procedure proposed in Jalal et al. (2017). The
proposed method improves the attacked accuracy from $1\%$ to $78\%$. More
specifically, we compare the differences of all hidden states spanning the
orthogonal complement between a perturbed testing data and its unperturbed
counterpart,
$\|\mathbf{P}_{t}^{\perp}\mathbf{x}_{\epsilon,t}-\mathbf{P}_{t}^{\perp}\overline{\mathbf{x}}_{\epsilon,t}\rVert$,
where $\mathbf{P}_{t}^{\perp}$ is a projection onto the orthogonal complement.
The difference is growing such as $0,0.016,0.0438,0.0107,0.0552$ for hidden
states at layer $0,1,2,3,4$ respectively. This validates the argument for how
the proposed method is able to detect such perturbation and controls the
perturbation in hidden layers.

Furthermore, we provide some insights about the reasons behind the success of
such an adversarial attack. This follows the same concept of the existence of
adversarial attack in neural networks. The highly nonlinear behaviour of
neural networks preserves complex representative ability, meanwhile, its
powerful representation results in its vulnerability. For example, a constant
function has $50\%$ chance to make a correct prediction in binary
classification problem under any perturbation, but its performance is limited.
Therefore, we propose to use a linear embedding function that compensates
between the embedding accuracy and robustness.

##  Appendix E Definition of threat model

Generally, an attacker should not have access to the hidden states during
inference, in which case, an attacker is not allowed to inject extra noise
during inference. To define the threat model of the proposed method, for the
white-box setting, an attacker has access to both network and all embedding
functions. The condition that the perturbation $\epsilon\cdot\mathbf{z}$ makes
our method vulnerable is defined as follows,

$\sum_{t=0}^{T-1}\lVert\mathcal{E}_{t}(\mathbf{x}_{\epsilon,t})-\mathbf{x}_{\epsilon,t}\rVert_{2}^{2}=0,\hskip
8.5359pt\mathbf{x}_{\epsilon,0}=\mathbf{x}_{0}+\epsilon\cdot\mathbf{z}.$

In words, the perturbation $\epsilon\cdot\mathbf{z}$ applied on the input data
must result in $0$ reconstruction losses across all hidden layers, which means
its corresponding state trajectory does not span any of all orthogonal
complements of all hidden state spaces. To obtain an effective attack
satisfying the above equation, conventional gradient-based attackers cannot
guarantee to find an perfect attack. A possible way is to perform grid-search
backward in layers to find such an adversarial attack satisfying the thread
model condition, which is extremely costly.