# Lyapunov-Based Dropout Deep Neural Network (Lb-DDNN) Controller

Saiedeh Akbari∗, Emily J. Griffis∗, Omkar Sudhir Patil∗, Warren E. Dixon∗
∗Saiedeh Akbari, Emily J. Griffis, Omkar Sudhir Patil, and Warren E. Dixon are
with the Department of Mechanical and Aerospace Engineering, University of
Florida, Gainesville, FL, 32611-6250 USA. Email: {akbaris, emilygriffis00,
patilomkarsudhir<EMAIL_ADDRESS>research is supported in part by AFRL
grant FA8651-21- F-1027, AFRL grant FA8651-21-F-1025, AFOSR grant FA9550-19-
1-0169, and Office of Naval Research grant N00014-21-1-2481. Any opinions,
findings, and conclusions or recommendations expressed in this material are
those of the author(s) and do not necessarily reflect the views of the
sponsoring agencies.

###### Abstract

Deep neural network (DNN)-based adaptive controllers can be used to compensate
for unstructured uncertainties in nonlinear dynamic systems. However, DNNs are
also very susceptible to overfitting and co-adaptation. Dropout regularization
is an approach where nodes are randomly dropped during training to alleviate
issues such as overfitting and co-adaptation. In this paper, a dropout DNN-
based adaptive controller is developed. The developed dropout technique allows
the deactivation of weights that are stochastically selected for each
individual layer within the DNN. Simultaneously, a Lyapunov-based real-time
weight adaptation law is introduced to update the weights of all layers of the
DNN for online unsupervised learning. A non-smooth Lyapunov-based stability
analysis is performed to ensure asymptotic convergence of the tracking error.
Simulation results of the developed dropout DNN-based adaptive controller
indicate a $\boldsymbol{38.32\%}$ improvement in the tracking error, a
$\boldsymbol{53.67\%}$ improvement in the function approximation error, and
$\boldsymbol{50.44\%}$ lower control effort when compared to a baseline
adaptive DNN-based controller without dropout regularization.

###### Index Terms:

Deep neural network, dropout, adaptive control, Lyapunov methods, nonlinear
control systems.

##  I Introduction

Empirical evidence indicates that deep neural networks (DNNs) can provide
better function approximation than single layer neural networks [1].
Traditionally, DNN-based controllers are trained using offline training
methods based on prior collected datasets. [2, Section 6.6]. Recent
developments in [3, 4, 5, 6, 7] use Lyapunov-based methods to develop
unsupervised online learning for all weights of a deep neural network (i.e.,
Lb-DNNs).

Unfortunately, both offline and Lb-DNNs can exhibit significantly degraded
performance due to data overfitting. Another challenge that decreases
generalization and performance of the trained DNN is co-adaptation, where
multiple neurons, or even entire layers, become overly reliant on each other
during the training process[8]. One effective approach to address these issues
is through dropout regularization. Dropout was originally introduced by G.
Hinton in [9] to prevent co-adaptation of feature detectors and improve the
generalization performance of DNNs. Dropout regularization involves
stochastically dropping out neurons during training, which helps prevent over-
fitting, enhances the overall function approximation performance, and
efficiently allocates computational resources while updating the network’s
weights [10, 11]. By setting the activation of certain individual weights to
zero, dropout induces sparse representation in the network which reduces co-
dependency in neurons. Moreover, dropouts can be viewed as training an
ensemble of multiple DNNs with smaller width that are trained independently.
Independence in the training has a regularizing effect and provides better
generalization to new information. This intuitive reasoning is also applicable
for using dropout in DNN-based adaptive control, since dropouts mitigate co-
adaptation by reducing the number of weights influencing an adaptation law.

Although dropout regularization has been used for offline training of DNNs in
results such as [12, 13], its application has been limited in real-time
adaptive control settings. In [10], the dropout method is employed on a DNN to
improve the training performance of inner layers in pseudo real-time, and
through simulations, the study demonstrates the improved performance of a DNN-
based adaptive controller with dropout. However, the pseudo real-time
adaptation laws in [10] are not stability-driven but are rather based on a
modular design where the stability analysis is primarily facilitated using
robust control techniques.

This paper introduces a novel dropout technique aimed at enhancing the
function approximation performance of a DNN-based adaptive controller that
updates the weights of all layers of the DNN using the Lyapunov-based update
law in [3] (i.e., a Lyapunov-based Dropout Deep Neural Network (Lb-DDNN)). The
proposed technique involves the selective inactivation, or dropout, of weights
associated with randomly selected neurons within each DNN layer. To
incorporate dropout regularization, a new recursive DNN representation and
stability-driven weight adaptation laws are constructed by considering the
effect of randomization matrices on the closed-loop error system. Through a
non-smooth Lyapunov-based stability analysis, the designed controller is
guaranteed to stabilize the system in the sense that the tracking error
asymptotically converges to zero. Simulation experiments are performed to
compare the Lb-DDNN adaptive controller with the baseline adaptive DNN
controller developed in [3]. The simulation results show a $35.56\%$
improvement in the tracking error, a $49.94\%$ improvement in the function
approximation error, and $48.56\%$ lower control effort in the proposed
controller when compared to the baseline controller.

##  II Problem Formulation

###  II-A Notation

The space of essentially bounded Lebesgue measurable functions is denoted by
$\mathcal{L}_{\infty}$. Given two functions $f:A\to B$ and $g:B\to C$, where
$A$, $B$, and $C$ are sets, the composition of $f$ and $g$, denoted as $g\circ
f$, is a new function $h:A\to C$ defined as $h\triangleq g\circ
f=g\left(f\left(x\right)\right)$, for all $x\in A$. Let $\mathbf{0}_{m\times
n}$ denote a zero matrix with the dimension of $m\times n$. Let $I_{n\times
n}$ denote an identity matrix with the dimension of $n$. For matrices
$A\in\mathbb{R}^{m\times n}$ and $B\in\mathbb{R}^{p\times q}$, the Kronecker
product is denoted as $A\otimes B$. Given a matrix
$A\triangleq\left[a_{i,j}\right]\in\mathbb{R}^{n\times m}$, where $a_{i,j}$
denotes the element in the $i^{th}$ row and $j^{th}$ column of $A$, the
vectorization operator is defined as
$\mathcal{\text{vec}}\left(A\right)\triangleq\left[a_{1,1},\ldots,a_{1,m},\ldots,a_{n,1},\ldots,a_{n,m}\right]^{\top}\in\mathbb{R}^{nm}$.
From [14, Proposition 7.1.9] and given matrices $A\in\mathbb{R}^{p\times a}$,
$B\in\mathbb{R}^{a\times r}$, and $C\in\mathbb{R}^{r\times s}$, the
vectorization operator satisfies the property
$\mathcal{\text{vec}}\left(ABC\right)=\left(C^{\top}\otimes
A\right)\mathcal{\text{vec}}\left(B\right).$ Differentiating
$\mathcal{\text{vec}}\left(ABC\right)$ on both sides with respect to
$\mathcal{\text{vec}}\left(B\right)$ yields the property
$\frac{\partial}{\partial\mathcal{\text{vec}}\left(B\right)}\mathcal{\text{vec}}\left(ABC\right)=C^{\top}\otimes
A.$ The right-to-left matrix product operator is represented by
$\stackrel{{\scriptstyle\curvearrowleft}}{{\prod}}$, i.e.,
$\stackrel{{\scriptstyle\curvearrowleft}}{{\stackrel{{\scriptstyle[}}{{p}}=1]{m}{\prod}}}A_{p}=A_{m}\ldots
A_{2}A_{1}$, and
$\stackrel{{\scriptstyle\curvearrowleft}}{{\stackrel{{\scriptstyle[}}{{p}}=a]{m}{\prod}}}A_{p}=1$
if $a>m$. The Filippov set-valued map defined in [15, Equation 2b] is denoted
by $\text{K}\left[\cdot\right]$. The notation
$\overset{\text{a.a.t.}}{\left(\cdot\right)}$ denotes that the relation
$\left(\cdot\right)$ holds for almost all time (a.a.t.). Consider a Lebesgue
measurable and locally essentially bounded function
$h:\mathbb{R}^{n}\times\mathbb{R}_{\geq 0}\to\mathbb{R}^{n}$. Then, the
function $y:\mathcal{I}\to\mathbb{R}^{n}$ is called a Filippov solution of
$\dot{y}=h\left(y,t\right)$ on the interval
$\mathcal{I}\subseteq\mathbb{R}_{\geq 0}$ if $y$ is absolutely continuous on
$\mathcal{I}$ and
$\dot{y}\overset{\text{a.a.t.}}{\in}\text{K}\left[h\right]\left(y,t\right)$.
Given some functions $f$ and $g$, the notation
$f\left(y\right)=\mathcal{O}^{m}\left(g\left(y\right)\right)$ means that there
exists some constants $M\in\mathbb{R}_{>0}$ and $y_{0}\in\mathbb{R}$ such that
$\left\|f(y)\right\|\leq M\left\|g(y)\right\|^{m}$ for all $y\geq y_{0}$. The
operator $\text{proj}\left(\cdot\right)$ denotes the projection operator
defined in [16, Appendix E, Eq. E.4].

###  II-B Dynamic Model and Control Objective

Consider a control-affine nonlinear system modeled as

$\dot{x}\left(t\right)=f\left(x\left(t\right)\right)+u\left(t\right),$ (1)

where $t\in\mathbb{R}_{\geq 0}$, $x:\mathbb{R}_{\geq 0}\to\mathbb{R}^{n}$,
$f:\mathbb{R}^{n}\to\mathbb{R}^{n}$, and $u:\mathbb{R}_{\geq
0}\to\mathbb{R}^{n}$ denote continuous time, the state, the unknown
differentiable drift vector field, and the control input, respectively. The
control objective is to design a controller $u\left(t\right)$ such that the
state tracks the desired trajectory $x_{d}$. To achieve the control objective,
an adaptive Lb-DNN architecture and a controller are designed to learn the
unknown drift vector field and to achieve asymptotic convergence on the
tracking error, respectively. To quantify the control objective, the tracking
error $e:\mathbb{R}_{\geq 0}\to\mathbb{R}^{n}$, is defined as

$e\left(t\right)\triangleq x\left(t\right)-x_{d}\left(t\right),$ (2)

where $x_{d}:\mathbb{R}_{\geq 0}\to\mathbb{R}^{n}$ denotes a continuously
differentiable desired trajectory.

######  Assumption 1.

The desired trajectory is designed such that for all $t\in\mathbb{R}_{\geq
0}$, $x_{d}\left(t\right)\in\Omega$, and $\dot{x}_{d}\in\mathcal{L}_{\infty}$,
where $\Omega\subset\mathbb{R}^{n}$ is a known compact set. Hence, the desired
trajectory can be bounded as $\left\|x_{d}\right\|\leq\overline{x_{d}}$, where
$\overline{x_{d}}\in\mathbb{R}_{>0}$ is a known constant.

##  III Control Design

###  III-A Deep Neural Network Architecture

To estimate the unknown nonlinear drift vector field $f\left(x\right)$, a Lb-
DNN architecture is developed using dropout. Dropout randomly omits neurons
while training, which helps mitigate over-fitting and co-adaptation, thus
improving the overall performance and function approximation capabilities of
the DNN [10, 11]. Leveraging the Lyapunov stability-driven weight adaptation
laws developed in [3], the dropout DNN is designed such that randomization
matrices are used to incorporate dropout techniques into the online,
stability-driven weight adaptation. Through the randomization matrices,
weights associated with a batch of randomly selected neurons are inactivated,
i.e., dropped out, to reduce the interdependency and excessive reliance on
specific weights and neurons.

Figure 1: The structure of a DNN with three hidden layers, where the dashed
and solid lines respectively represent the randomly dropped out and selected
weights.

As shown in Figure 1, let the dropout DNN architecture,
$\Phi:\mathbb{R}^{n}\times\left\\{0,1\right\\}^{\stackrel{{\scriptstyle[}}{{m}}=0]{k}{\sum}L_{m}\times\stackrel{{\scriptstyle[}}{{m}}=0]{k}{\sum}L_{m}}\times\mathbb{R}^{\stackrel{{\scriptstyle[}}{{j}}=0]{k}{\sum}L_{j}L_{j+1}}\to\mathbb{R}^{L_{k+1}}$,
be defined as

$\Phi\left(x,R_{i},\theta\right)=\left(R_{i,k}V_{k}\right)^{\top}\phi_{k}\circ\cdots\circ\left(R_{i,1}V_{1}\right)^{\top}\phi_{1}\circ\left(R_{i,0}V_{0}\right)^{\top}x,$
(3)

where $k\in\mathbb{Z}_{>0}$ denotes the number of the layers in
$\Phi\left(x,R_{i},\theta\right)$, and
$\phi_{j}:\mathbb{R}^{L_{j}}\to\mathbb{R}^{L_{j}}$ denotes the vector of
smooth activation functions in the $j^{\text{th}}$ layer, for all
$j\in\left\\{1,\cdots,k\right\\}$.111Although $\phi_{j}$ is defined as a
smooth function, the subsequent analysis allows the inclusion of non-smooth
activation functions by using the switched systems analysis in [3]. For
$j\in\left\\{0,\cdots,k\right\\}$, $V_{j}\in\mathbb{R}^{L_{j}\times L_{j+1}}$
and $L_{j}\in\mathbb{Z}_{>0}$ represent the weight matrix and the number of
nodes in the $j^{\text{th}}$ layer of $\Phi$, respectively. For notation
simplicity, the weights can be represented in a vector
$\theta\in\mathbb{R}^{\stackrel{{\scriptstyle[}}{{j}}=0]{k}{\sum}L_{j}L_{j+1}}$
as
$\theta\triangleq\left[\mathcal{\text{vec}}\left(V_{0}\right)^{\top},\mathcal{\text{vec}}\left(V_{1}\right)^{\top},\cdots,\mathcal{\text{vec}}\left(V_{k}\right)^{\top}\right]^{\top}$.
Let
$R_{i}\in\left\\{0,1\right\\}^{\stackrel{{\scriptstyle[}}{{m}}=0]{k}{\sum}L_{m}\times\stackrel{{\scriptstyle[}}{{m}}=0]{k}{\sum}L_{m}}$
denote the $i^{\text{th}}$ instance of the randomization matrix, for all
$i\in\mathcal{I}\triangleq\left\\{1,\cdots,J\right\\}$, where $\mathcal{I}$
denotes the set of all possible switching instances.

After every user-selected constant time period of $\delta t\in\mathbb{R}_{>0}$
seconds, the randomization matrix $R_{i}$ switches to $R_{i+1}$, where
$R_{i+1}$ is randomly selected from all possible permutations of randomization
matrices. Each permutation is defined as

$R_{i}\triangleq\left[\begin{array}[]{cccc}R_{i,0}&\mathbf{0}_{L_{0}\times
L_{1}}&\cdots&\mathbf{0}_{L_{0}\times L_{k}}\\\ \mathbf{0}_{L_{1}\times
L_{0}}&R_{i,1}&\cdots&\mathbf{0}_{L_{1}\times L_{k}}\\\
\vdots&\vdots&\ddots&\vdots\\\ \mathbf{0}_{L_{k}\times
L_{0}}&\mathbf{0}_{L_{k}\times
L_{1}}&\cdots&R_{i,k}\end{array}\right],\quad\forall i\in\mathcal{I}.$

For $i\in\left\\{1,\cdots,J\right\\}$ and $j\in\left\\{1,\cdots,k\right\\}$,
$R_{i,j}\in\left\\{0,1\right\\}^{L_{j}\times L_{j}}$ is designed to be a
diagonal matrix, and the matrix $R_{i,0}$ is an identity matrix. The number of
ones on the diagonal of each $R_{i,j}$ is equal to a user-selected constant
number $n_{j}$, and the placement of non-zero elements on the diagonal of the
matrix $R_{i,j}$ randomly changes after every $\delta t$ seconds.222Once the
system reaches the steady state, the randomization can be stopped in the sense
that $R_{i}$ is replaced with identity matrices. This can be considered as the
final permutation of $R_{i}$ for all $i\in\mathcal{I}$. To illustrate the
design of the randomization matrix $R_{i,j}$ and the effect of dropout on the
DNN architecture, the following example is provided.

######  Example 1.

Consider $R_{i,2}\in\left\\{0,1\right\\}^{3\times 3}$,
$V_{2}\in\mathbb{R}^{3\times 2}$, and let $n_{2}=1$. Therefore, every $\delta
t$ seconds, $n_{2}=1$ of the elements on the diagonal of $R_{i,2}$ are
randomly set to 1 and the others are zeroed. The considered permutations of
$R_{i,2}$ for $i\in\left\\{1,2,3\right\\}$ are

$\displaystyle R_{1,2}=\left[\begin{array}[]{ccc}1&0&0\\\ 0&0&0\\\
0&0&0\end{array}\right],\,R_{2,2}=\left[\begin{array}[]{ccc}0&0&0\\\ 0&1&0\\\
0&0&0\end{array}\right],\,R_{3,2}=\left[\begin{array}[]{ccc}0&0&0\\\ 0&0&0\\\
0&0&1\end{array}\right].$

Let $v_{p\times q}\in\mathbb{R}$ denote to each individual weight of the
weight matrix $V_{2}$, for rows $p\in\left\\{1,2,3\right\\}$ and columns
$q\in\left\\{1,2\right\\}$. For $p\in\left\\{1,2,3\right\\}$, let
$\phi_{2,p}:\mathbb{R}^{L_{2}}\to\mathbb{R}^{L_{2}}$ denote to the activation
functions of the second layer such that the activation vector is
$\phi_{2}=\left[\phi_{2,1},\,\phi_{2,2},\,\phi_{2,3}\right]^{\top}$.
Therefore, in the presence and absence of dropout matrix
$\left(R_{3,2}V_{2}\right)^{\top}\phi_{2}$ and $V_{2}^{\top}\phi_{2}$ are
respectively obtained as

$\displaystyle\left(R_{3,2}V_{2}\right)^{\top}\phi_{2}$
$\displaystyle=\left[\begin{array}[]{cc}v_{1,1}&v_{1,2}\\\ v_{2,1}&v_{2,2}\\\
v_{3,1}&v_{3,2}\end{array}\right]^{\top}\left[\begin{array}[]{ccc}0&0&0\\\
0&0&0\\\ 0&0&1\end{array}\right]^{\top}\left[\begin{array}[]{c}\phi_{2,1}\\\
\phi_{2,2}\\\ \phi_{2,3}\end{array}\right]$ (13)
$\displaystyle=\left[\begin{array}[]{c}v_{3,1}\phi_{2,3}\\\
v_{3,2}\phi_{2,3}\end{array}\right].$ (16) $\displaystyle
V_{2}^{\top}\phi_{2}$
$\displaystyle=\left[\begin{array}[]{cc}v_{1,1}&v_{1,2}\\\ v_{2,1}&v_{2,2}\\\
v_{3,1}&v_{3,2}\end{array}\right]^{\top}\left[\begin{array}[]{c}\phi_{2,1}\\\
\phi_{2,2}\\\ \phi_{2,3}\end{array}\right]$ (23)
$\displaystyle=\left[\begin{array}[]{c}v_{1,1}\phi_{2,1}+v_{2,1}\phi_{2,2}+v_{3,1}\phi_{2,3}\\\
v_{1,2}\phi_{2,1}+v_{2,2}\phi_{2,2}+v_{3,2}\phi_{2,3}\end{array}\right].$ (26)

Comparing (16) and (26) suggests how the dropout method deactivates the
activation functions associated with the zeros on the diagonal of the
randomization matrix. Since the dropout matrix is randomly generated, a new
batch of weights are selected every $\delta t$ seconds.

The universal function approximation property states that the function space
of (3) is dense in $\mathcal{C}\left(\mathcal{X}\right)$, where
$\mathcal{C}\left(\mathcal{X}\right)$ denotes the space of continuous
functions over the compact set $\mathcal{X}\subseteq\mathbb{R}^{n}$, where
$x\in\mathcal{X}$ [17, Theorem 1.1]. Therefore, for all
$j\in\left\\{0,\cdots,k\right\\}$, there exists a corresponding vector of
ideal weights
$\theta^{*}\in\mathbb{R}^{\stackrel{{\scriptstyle[}}{{j}}=0]{k}{\sum}L_{j}L_{j+1}}$
such that
$\sup_{x_{d}\in\Omega}\left\|f\left(x\right)-\Phi\left(x,R_{i},\theta^{*}\right)\right\|\leq\overline{\varepsilon}$.
Thus, the drift vector field can be modeled as

$\displaystyle f\left(x\right)$ $\displaystyle=$
$\displaystyle\Phi\left(x,R_{i},\theta^{*}\right)+\varepsilon\left(x\right),$
(27)

where $\varepsilon:\mathbb{R}^{n}\to\mathbb{R}^{n}$ denotes an unknown
function reconstruction error that can be bounded as
$\sup\left\|\varepsilon\left(x\right)\right\|\leq\overline{\varepsilon}$.

######  Assumption 2.

The vector of ideal weights can be bounded by a known constant
$\overline{\theta}\in\mathbb{R}_{>0}$ as
$\left\|\theta^{*}\right\|\leq\overline{\theta}$, [18, Assumption 1].

###  III-B Adaptation Law

To fulfill the tracking objective, the DNN model in (27) is used to estimate
the unknown drift dynamics in (1). Since the ideal weights of the modeled DNN
are unknown, adaptive estimates of the weight matrices are developed to learn
the unknown drift dynamics $f\left(x\right)$. Let
$\hat{\theta}:\mathbb{R}_{\geq
0}\to\mathbb{R}^{\stackrel{{\scriptstyle[}}{{j}}=0]{k}{\sum}L_{j}L_{j+1}}$ be
defined as
$\hat{\theta}\left(t\right)\triangleq\left[\mathcal{\text{vec}}\left(\widehat{V}_{0}\right)^{\top},\mathcal{\text{vec}}\left(\widehat{V}_{1}\right)^{\top},\cdots,\mathcal{\text{vec}}\left(\widehat{V}_{k}\right)^{\top}\right]^{\top}$,
where $\widehat{V}_{j}:\mathbb{R}^{L_{j}\times L_{j+1}}$, for all
$j\in\left\\{0,\cdots,k\right\\}$, denote the weight estimates. The
corresponding weight estimation error $\tilde{\theta}:\mathbb{R}_{\geq
0}\to\mathbb{R}^{\stackrel{{\scriptstyle[}}{{j}}=0]{k}{\sum}L_{j}L_{j+1}}$ is
defined as
$\tilde{\theta}\left(t\right)\triangleq\theta^{*}-\hat{\theta}\left(t\right)$.
Using the weight estimates $\hat{\theta}$, the adaptive estimate of
$f\left(x\right)$ can be represented as
$\Phi\left(x,R_{i},\hat{\theta}\right)\triangleq\left(R_{i,k}\widehat{V}_{k}\right)^{\top}\phi_{k}\circ\cdots\circ\left(R_{i,1}\widehat{V}_{1}\right)^{\top}\phi_{1}\circ\left(R_{i,0}\widehat{V}_{0}\right)^{\top}x$.
The estimated DNN architecture can be written in a recursive relation as

$\widehat{\Phi}_{j}=\begin{cases}\left(R_{i,j}\widehat{V}_{j}\right)^{\top}\phi_{j}\left(\widehat{\Phi}_{j-1}\right),&j=\left\\{1,...,k\right\\},\\\
\left(R_{i,0}\widehat{V}_{0}\right)^{\top}x,&j=0,\end{cases}$ (28)

where $\widehat{\Phi}_{j}$ is the shorthand notation for
$\widehat{\Phi}_{j}\triangleq\Phi_{j}\left(x,R_{i},\hat{\theta}\right)$, and
$\widehat{\Phi}=\widehat{\Phi}_{k}$. Based on the subsequent stability
analysis, the adaptation law for DNN weight estimates is designed as

$\dot{\hat{\theta}}\triangleq\text{proj}\left(\Gamma_{\theta}\widehat{\Phi}^{\prime\top}e\right),$
(29)

where
$\Gamma_{\theta}\in\mathbb{R}^{\stackrel{{\scriptstyle[}}{{j}}=0]{k}{\sum}L_{j}L_{j+1}\times\stackrel{{\scriptstyle[}}{{j}}=0]{k}{\sum}L_{j}L_{j+1}}$
denotes a positive-definite adaptation gain matrix, and
$\widehat{\Phi}^{\prime}$ is a shorthand notation for the Jacobian
$\widehat{\Phi}^{\prime}\triangleq\frac{\partial\Phi\left(x,R_{i},\hat{\theta}\right)}{\partial\hat{\theta}}$.
The Jacobian $\widehat{\Phi}^{\prime}$ can be represented as
$\widehat{\Phi}^{\prime}\triangleq\left[\widehat{\Phi}_{0}^{\prime},...,\widehat{\Phi}_{k}^{\prime}\right],$
where the shorthand notation $\widehat{\Phi}_{j}^{\prime}$ is defined as
$\widehat{\Phi}_{j}^{\prime}\triangleq\frac{\partial\Phi_{j}\left(x,R_{i},\hat{\theta}\right)}{\partial\hat{\theta}}$,
for all $j\in\left\\{0,...,k\right\\}$. The projection operator is
incorporated in the update law to ensure that
$\hat{\theta}\left(t\right)\in\Lambda\triangleq\bigg{\\{}\theta\in\mathbb{R}^{\stackrel{{\scriptstyle[}}{{j}}=0]{k}{\sum}L_{j}L_{j+1}}:\left\|\theta\right\|\leq\overline{\theta}\bigg{\\}}$,
for all $t\geq 0$. Since $\left\|\theta^{*}\right\|\leq\overline{\theta}$ and
$\left\|\hat{\theta}\right\|\leq\overline{\theta}$, using the definition of
$\tilde{\theta}$, it can be shown that $\left\|\tilde{\theta}\right\|\leq
2\overline{\theta}$. Using (28), the chain rule, and the properties of the
vectorization operator, the Jacobians $\widehat{\Phi}_{0}^{\prime}$ and
$\widehat{\Phi}_{j}^{\prime}$, for all $j=\left\\{1,\cdots,k\right\\}$ and
$i\in\mathcal{I}$, can respectively be calculated as

$\displaystyle\widehat{\Phi}_{0}^{\prime}$
$\displaystyle\triangleq\left(\stackrel{{\scriptstyle[}}{{l}}=1]{\overset{\curvearrowleft}{k}}{\prod}\left(R_{i,l}\widehat{V}_{l}\right)^{\top}\widehat{\phi}_{l}^{\prime}\right)\left(\left(x^{\top}R_{i,0}\right)\otimes
I_{L_{1}}\right),$ $\displaystyle\widehat{\Phi}_{j}^{\prime}$
$\displaystyle\triangleq\left(\stackrel{{\scriptstyle[}}{{l}}=j+1]{\overset{\curvearrowleft}{k}}{\prod}\left(R_{i,l}\widehat{V}_{l}\right)^{\top}\widehat{\phi}_{l}^{\prime}\right)\left(\left(\widehat{\phi}_{j}^{\top}R_{i,j}\right)\otimes
I_{L_{j+1}}\right),$ (30)

where $\hat{\phi}_{j}$ and the Jacobian $\hat{\phi}_{j}^{\prime}$ are the
short-hand notations for
$\hat{\phi}_{j}\triangleq\phi_{j}\left(\widehat{\Phi}_{j-1}\right)$ and
$\hat{\phi}_{j}^{\prime}\triangleq\phi_{j}^{\prime}\left(\widehat{\Phi}_{j-1}\right)=\frac{\partial\widehat{\Phi}_{j}}{\partial\hat{\theta}}$,
respectively.

######  Remark 1.

The presence of matrix $R_{i}$ for all $i\in\mathcal{I}$ in (30) mitigates co-
adaptation by reducing the interdependency of weights in the adaptation law.

###  III-C Closed-Loop Error System

The designed DNN estimate is used in the developed controller to approximate
the unknown drift vector field in (1). By incorporating the developed adaptive
DNN estimate, the controller in (31) is designed such that the state $x$
tracks the desired trajectory $x_{d}$ despite inactivation of weights
associated with a randomly selected batch of neurons. Based on the subsequent
stability analysis, the control input is designed as

$u\left(t\right)\triangleq\dot{x}_{d}-\widehat{\Phi}-k_{e}e-k_{s}\text{sgn}\left(e\right),$
(31)

where $k_{e},k_{s}\in\mathbb{R}_{>0}$ are constant control gains. Taking the
time-derivative of (2) and substituting (1), (27), and the designed controller
in (31) and canceling cross-terms yields the closed-loop error system as

$\displaystyle\dot{e}\left(t\right)=\Phi\left(x,R_{i},\theta^{*}\right)-\widehat{\Phi}+\varepsilon\left(x\right)-k_{s}\text{sgn}\left(e\right)-k_{e}e.$
(32)

To address the technical challenges in deriving adaptation the DNN weights,
many results use Taylor series approximation based techniques [18, 3, 5, 7,
Eq. 22]. Applying a first-order Taylor series approximation-based error model
on $\Phi\left(x,R_{i},\theta^{*}\right)-\widehat{\Phi}$ yields

$\displaystyle\Phi\left(x,R_{i},\theta^{*}\right)-\widehat{\Phi}$
$\displaystyle=\widehat{\Phi}^{\prime}\tilde{\theta}+\mathcal{O}^{2}\left(\left\|\tilde{\theta}\right\|\right),$
(33)

where $\mathcal{O}\left(\left\|\tilde{\theta}\right\|\right)$ denotes the
higher-order terms that can be bounded as
$\left\|\mathcal{O}^{2}\left(\left\|\tilde{\theta}\right\|\right)\right\|\leq\Delta$,
where $\Delta\in\mathbb{R}_{>0}$ denotes a known constant [5, Eq. 18].
Substituting (33) into (32) yields

$\displaystyle\dot{e}=\widehat{\Phi}^{\prime}\tilde{\theta}+\mathcal{O}^{2}\left(\left\|\tilde{\theta}\right\|\right)+\varepsilon\left(x\right)-k_{s}\text{sgn}\left(e\right)-k_{e}e.$
(34)

To facilitate the subsequent stability analysis, let $z:\mathbb{R}_{\geq
0}\to\mathbb{R}^{\psi}$ denote the concatenated error system defined as

$z\left(t\right)\triangleq\left[e^{\top}\left(t\right),\,\tilde{\theta}^{\top}\left(t\right)\right]^{\top},$
(35)

where $\psi\triangleq
n+\stackrel{{\scriptstyle[}}{{j}}=0]{k}{\sum}L_{j}L_{j+1}$. Additionally, let
$\dot{z}=h\left(z,t\right)$, where $h:\mathbb{R}^{\psi}\times\mathbb{R}_{\geq
0}\to\mathbb{R}^{\psi}$ is as

$h\left(z,t\right)\triangleq\left[\begin{array}[]{c}\left(\begin{array}[]{c}\widehat{\Phi}^{\prime}\tilde{\theta}+\mathcal{O}^{2}\left(\left\|\tilde{\theta}\right\|\right)\\\
+\varepsilon\left(x\right)-k_{s}\text{sgn}\left(e\right)-k_{e}e\end{array}\right)\\\
-\text{proj}\left(\Gamma_{\theta}\widehat{\Phi}^{\prime\top}e\right)\end{array}\right].$
(36)

##  IV Stability Analysis

Let $V_{L}:\mathbb{R}^{\psi}\to\mathbb{R}_{\geq 0}$ denote the Lyapunov
function candidate defined as

$V_{L}\left(z\right)\triangleq\frac{1}{2}e^{\top}e+\frac{1}{2}\tilde{\theta}^{\top}\Gamma_{\theta}^{-1}\tilde{\theta}.$
(37)

Given the known constants
$\underline{\alpha},\overline{\alpha}\in\mathbb{R}_{>0}$, the Lyapunov
function candidate satisfies the following inequality:

$\underline{\alpha}\left\|z\right\|^{2}\leq
V_{L}\left(z\right)\leq\overline{\alpha}\left\|z\right\|^{2}.$ (38)

Let the open and connected sets $\mathcal{B}\in\mathbb{R}^{\psi}$ and
$\Upsilon\subseteq\mathcal{X}$ be defined as
$\mathcal{B}\triangleq\left\\{\varsigma\in\mathbb{R}^{\psi}:\lVert\varsigma\rVert\leq\omega\sqrt{\nicefrac{{\underline{\alpha}}}{{\overline{\alpha}}}}\right\\}$
and
$\Upsilon\triangleq\left\\{\varsigma\in\mathcal{X}:\lVert\varsigma\rVert<\overline{x_{d}}+\omega\right\\}$.
Theorem 1 uses the non-smooth analysis technique in [19] to establish the
invariance properties of Fillipov solutions to $\dot{z}$ and to guarantee
asymptotic convergence of the tracking error, $e$.

######  Theorem 1.

The controller designed in (31) and the DNN update law developed in (29)
guarantee asymptotic tracking error convergence for the dynamical system in
(1) in the sense that
$\underset{t\to\infty}{\lim}\left\|e\left(t\right)\right\|=0$, given
$\left\|z\left(t_{0}\right)\right\|\in\mathcal{B}$ and that the gain condition
$k_{s}>\overline{\varepsilon}+\Delta$ is satisfied.

###### Proof:

Let $\partial V_{L}$ denote the Clarke gradient of $V_{L}$ defined in [20, p.
39]. Since the Lyapunov function candidate is continuously differentiable,
$\partial V_{L}(z)=\\{\nabla V_{L}(z)\\}$, where $\nabla$ denotes the standard
gradient operator. From (36), it can be concluded that for all
$i\in\mathcal{I}$, $V_{L}$ satisfies the following differential equation

$\displaystyle\dot{V}_{L}$
$\displaystyle\overset{\text{a.a.t.}}{\in}\overset{\sigma\in\partial
V_{L}\left(z\right)}{\bigcap}\sigma^{\top}\text{K}\left[h\right]\left(z,t\right)$
$\displaystyle=\nabla
V_{L}^{\top}\left(z\right)\text{K}\left[h\right]\left(z,t\right)$
$\displaystyle=e^{\top}\bigg{(}\widehat{\Phi}^{\prime}\tilde{\theta}+\mathcal{O}^{2}\left(\left\|\tilde{\theta}\right\|\right)+\varepsilon\left(x\right)-k_{e}e$
$\displaystyle\quad-
k_{s}\text{K}\left[\text{sgn}\right]\left(e\right)\bigg{)}-\tilde{\theta}^{\top}\Gamma_{\theta}^{-1}\text{K}\left[\text{proj}\right]\big{(}\Gamma_{\theta}\widehat{\Phi}^{\prime\top}e\big{)}.$
(39)

Using [16, Lemma E.1.IV], the bounds on
$\mathcal{O}^{2}\left(\left\|\tilde{\theta}\right\|\right)$ and
$\varepsilon\left(x\right)$, the fact that
$\text{K}\left[\text{proj}\right]\left(\cdot\right)$ is the set of convex
combinations of $\text{proj}\left(\cdot\right)$ and $\left(\cdot\right)$, and
therefore,
$-\tilde{\theta}^{\top}\Gamma_{\theta}^{-1}\text{K}\left[\text{proj}\right]\left(\Gamma_{\theta}\widehat{\Phi}^{\prime\top}e\right)\leq-\tilde{\theta}^{\top}\widehat{\Phi}^{\prime\top}e$,
and canceling cross terms, (39) can be upper-bounded as

$\displaystyle\dot{V}_{L}$
$\displaystyle\overset{\text{a.a.t.}}{\leq}-k_{e}\left\|e\right\|^{2}-k_{s}\left\|e\right\|+\left\|e\right\|\left(\Delta+\overline{\varepsilon}\right),\quad\forall
i\in\mathcal{I}.$

Selecting the gain $k_{s}$ according to the gain condition in Theorem 1 yields

$\dot{V}_{L}\overset{\text{a.a.t.}}{\leq}-k_{e}\left\|e\right\|^{2}.$ (40)

From the inequality obtained in (40), [19, Corollary 1] can be invoked to
conclude that $z\in\mathcal{L}_{\infty}$ and
$\underset{t\to\infty}{\lim}\left\|e\left(t\right)\right\|=0$. Due to the
facts that $\widehat{\Phi}$ is smooth for all $i\in\mathcal{I}$, $x\in\Omega$,
and $\left\|\hat{\theta}\right\|\leq\overline{\theta}$,
$\widehat{\Phi}\in\mathcal{L}_{\infty}$. Since
$\dot{x}_{d}\in\mathcal{L}_{\infty}$, $e\in\mathcal{L}_{\infty}$, and
$\widehat{\Phi}\in\mathcal{L}_{\infty}$, $u\in\mathcal{L}_{\infty}$. To show
that $x\in\mathcal{X}$, and therefore the universal function approximation
property holds, let $\left\|z\left(t_{0}\right)\right\|\in\mathcal{B}$. Since
$\lVert
z\left(t_{0}\right)\rVert\leq\omega\sqrt{\nicefrac{{\underline{\alpha}}}{{\overline{\alpha}}}}$,
using (38), $\lVert e\left(t\right)\rVert\leq\omega$. Hence, using (2),
$\left\|x\right\|$ can be bounded as
$\left\|x\right\|\leq\overline{x_{d}}+\omega$. Therefore, if
$z\left(t_{0}\right)\in\mathcal{B}$, then $x\in\Upsilon\subseteq\mathcal{X}$.
∎

##  V Simulation

To demonstrate the efficacy of the Lb-DDNN adaptive controller, simulations
are performed on a three-dimensional nonlinear system, and $f$ in (1) is
modeled as

$f=\left[\begin{array}[]{c}x_{1}x_{2}^{2}\tanh\left(x_{2}\right)+\sin\left(x_{1}\right)^{2}\\\
\cos\left(x_{1}+x_{2}+x_{3}\right)^{3}-\exp\left(x_{2}\right)^{2}+x_{1}x_{2}\\\
x_{3}^{2}\log\left(1+\text{abs}\left(x_{1}-x_{2}\right)\right)\end{array}\right],$

where $x\triangleq\left[x_{1},x_{2},x_{3}\right]^{\top}:\mathbb{R}_{\geq
0}\to\mathbb{R}^{3}$ denotes the system state. Three simulation experiments
are performed for $10\,\text{sec}$ with initial condition
$x\left(0\right)=\left[5,1,-5\right]^{\top}$. The desired trajectory is
selected as
$x_{d}\left(t\right)=\left[\sin\left(2t\right),-\cos\left(t\right),\sin\left(3t\right)+\cos\left(-2t\right)\right]^{\top}$.
The DNN used in the simulations has $k=7$ inner layers with $L=10$ neurons in
each hidden layer and contained hyperbolic tangent activation functions. The
first set of simulations are performed to compare the baseline DNN-based
adaptive controller in [3] and the Lb-DDNN adaptive controller in (29) and
(31). The second set of simulations are performed to examine the effect of
$\delta t$ on the performance of the propose method. The third set of
simulations are performed to compare the performance in the absence and
presence of dropout deactivation after the transient period. In all
simulations, the DNN weight estimates are initialized randomly from the normal
distribution $\mathcal{N}\left(0,10\right)$. The control gains in (31) are
selected as $k_{e}=10.5$ and $k_{s}=1.5$. The learning gain for the baseline
DNN is selected as $\Gamma_{\theta}=100I_{670\times 670}$. In the first two
sets of simulations, the randomization is activated for the first
$2\,\text{sec}$ where the system is in the transient stage. After
$2\,\text{sec}$, all the randomization matrices change to identity matrices.
For the first $2\,\text{sec}$, the learning gain of the dropout DNN update law
in (29) is selected as $\Gamma_{\theta}=100I_{670\times 670}$, and after
$2\,\text{sec}$, the learning gain changes to $\Gamma_{\theta}=40I_{670\times
670}$. In the transient stage of the first and third set of simulations, the
matrices $R_{i}$ change to $R_{i+1}$ every $\delta t=0.1\,\text{sec}$.

The performance results of the simulations are presented in Table I. As shown
in the first subplot of Figure 2, the tracking error for the dropout DNN
converges significantly faster than the baseline DNN. Specifically, the
dropout DNN results in convergence to the final error after approximately
$0.5\,\text{sec}$, roughly four times faster than that of the baseline
controller. Despite the jump in the tracking error after the deactivation of
the dropout of the DNN weights, the dropout DNN still yields the norm of the
root mean square tracking error of $0.81$, which shows a $38.32\%$ improvement
when compared to the baseline DNN adaptive controller. Moreover, the baseline
DNN controller presents more oscillatory behavior within the transient period
than the dropout DNN controller. The oscillatory behavior in the baseline DNN
is due to interdependency of weights in the adaptation. However, as stated in
Remark 1, the dropout DNN mitigates co-adaptation in the adaptation law, thus
yielding less oscillatory behavior. As shown in the second subplot of Figure
2, the function approximation error for the dropout DNN controller rapidly
converges after less than $0.2\,\text{sec}$ but takes approximately
$2\,\text{sec}$ to converge with the baseline DNN controller. Although there
is a jump in the function approximation error after the deactivation of the
DNN, the dropout DNN controller demonstrated a $53.67\%$ improvement in
function approximation with the norm of the root mean square function
approximation error of $19.44$. Thus, the developed dropout adaptive DNN
architecture resulted in better transient behavior and improved tracking and
function approximation performance with a $50.44\%$ lower control effort when
compared to the baseline adaptive DNN controller developed in [3].

To examine the effect of selecting different $\delta t$, simulations are
performed with $\delta t=0.2\,\text{sec}$ and $\delta t=0.05\,\text{sec}$
using the Lb-DDNN controller. As shown in Figure 3, reducing $\delta t$ causes
more spikes in the tracking and function approximation performances. Although
the differences between the the tracking and function approximation errors are
not significant, reducing $\delta t$ is found to cause more spikes in the
plots, as shown in Figure 3.

The third set of simulations examine the performance of the developed dropout
DNN controller under two cases; dropping out the neurons for the entire
duration of the simulation, and deactivating dropout after $1\,\text{sec}$. As
shown in Figure 4, for both cases, the difference between the tracking and
function approximation performances are insignificant during the first second,
as expected. Once the dropout is deactivated after $1\,\text{sec}$, there is
an overshoot in both tracking and function approximation errors which does not
occur when dropout is maintained throughout the simulation duration. However,
not deactivating the dropout after the transient period leads to more spikes
in both tracking and function approximation error in the steady state stage.
Despite the increase in the the tracking error, deactivation of the dropout in
the steady state leads to lower control input and function approximation error
as shown in Table I.

Table I: Performance comparison results DNN Architecture | $\left\|e\right\|$ | $\left\|f-\widehat{\Phi}\right\|$ | $\left\|u\right\|$
---|---|---|---
DNN | $1.32$ | $41.81$ | $780.19$
Dropout DNN | $0.81$ | $19.44$ | $386.67$
Dropout DNN, $\delta t=0.2\,\text{sec}$ | $0.88$ | $23.87$ | $461.83$
Dropout DNN, $\delta t=0.05\,\text{sec}$ | $0.90$ | $22.64$ | $430.79$
Dropout DNN, | $0.78$ | $20.40$ | $399.93$
no dropout deactivation
Dropout DNN, | $0.86$ | $20.00$ | $387.32$
dropout deactivation after $1\,\text{sec}$
Figure 2: Performance of the tracking error (top) and function approximation
error (bottom) over time comparing the Lb-DDNN controller and the adaptive DNN
controller developed in [3]. Figure 3: Performance of the tracking error (top)
and function approximation error (bottom) over time for dropout DNN with two
different $\delta t$ values. Figure 4: Performance of the tracking error (top)
and function approximation error (bottom) over time in the presence and
absence of dropout deactivation in the steady state (after $1\,\text{sec}$).

##  VI Conclusion

A dropout DNN-based adaptive controller is developed for general continuous
nonlinear systems. Leveraging the stability-derived DNN update law in [3] and
inspired by the dropout technique, the developed dropout DNN controller
improves function approximation performance and yields faster learning when
compared to the DNN controllers without dropout. A Lyapunov-based stability
analysis is performed to guarantee stability in the sense that the tracking
error asymptotically converges to zero. Simulation results show $38.32\%$ and
$53.67\%$ improvement in the tracking error and function approximation error,
respectively, with a $50.44\%$ reduced control effort when compared to the
baseline adaptive DNN controller. Additional simulations showed the effect of
dropout during both transient and steady state periods and how modifying
dropout parameters, i.e., $\delta t$, can effect system performance. Using the
established Lb-DDNN framework, future work can explore implementation
questions related to the dropout regularization such as changes in $\delta t$,
the number of neurons that are randomly selected, and dropout deactivation
strategies.

## References

  * [1] S. Liang and R. Srikant, “Why deep neural networks for function approximation?,” arXiv preprint arXiv:1610.04161, 2016.
  * [2] S. L. Brunton and J. N. Kutz, Data-driven science and engineering: Machine learning, dynamical systems, and control.  Cambridge University Press, 2019.
  * [3] O. S. Patil, D. M. Le, E. Griffis, and W. E. Dixon, “Deep residual neural network (ResNet)-based adaptive control: A Lyapunov-based approach,” in Proc. IEEE Conf. Decis. Control, 2022.
  * [4] R. Sun, M. Greene, D. Le, Z. Bell, G. Chowdhary, and W. E. Dixon, “Lyapunov-based real-time and iterative adjustment of deep neural networks,” IEEE Control Syst. Lett., vol. 6, pp. 193–198, 2022.
  * [5] O. Patil, D. Le, M. Greene, and W. E. Dixon, “Lyapunov-derived control and adaptive update laws for inner and outer layer weights of a deep neural network,” IEEE Control Syst Lett., vol. 6, pp. 1855–1860, 2022.
  * [6] G. Joshi and G. Chowdhary, “Deep model reference adaptive control,” in Proc. IEEE Conf. Decis. Control, pp. 4601–4608, 2019.
  * [7] E. Griffis, O. Patil, Z. Bell, and W. E. Dixon, “Lyapunov-based long short-term memory (Lb-LSTM) neural network-based control,” IEEE Control Syst. Lett., vol. 7, pp. 2976–2981, 2023.
  * [8] C. A. Charu, Neural networks and deep learning: a textbook.  Spinger, 2018.
  * [9] G. E. Hinton, N. Srivastava, A. Krizhevsky, I. Sutskever, and R. R. Salakhutdinov, “Improving neural networks by preventing co-adaptation of feature detectors,” arXiv preprint arXiv:1207.0580, 2012.
  * [10] D. Le, M. Greene, W. Makumi, and W. E. Dixon, “Real-time modular deep neural network-based adaptive control of nonlinear systems,” IEEE Control Syst. Lett., vol. 6, pp. 476–481, 2022.
  * [11] N. Srivastava, G. Hinton, A. Krizhevsky, I. Sutskever, and R. Salakhutdinov, “Dropout: a simple way to prevent neural networks from overfitting,” J. Mach. Learn. Res., vol. 15, no. 1, pp. 1929–1958, 2014.
  * [12] J. Ba and B. Frey, “Adaptive dropout for training deep neural networks,” Advances in neural inf. processing systems, vol. 26, 2013.
  * [13] G. E. Dahl, T. N. Sainath, and G. E. Hinton, “Improving deep neural networks for LVCSR using rectified linear units and dropout,” in Proc. IEEE Int. Conf. Acoust. Speech Signal Process., pp. 8609–8613, 2013.
  * [14] D. S. Bernstein, Matrix mathematics.  Princeton university press, 2009.
  * [15] B. E. Paden and S. S. Sastry, “A calculus for computing Filippov’s differential inclusion with application to the variable structure control of robot manipulators,” IEEE Trans. Circuits Syst., vol. 34, pp. 73–82, Jan. 1987.
  * [16] M. Krstic, I. Kanellakopoulos, and P. V. Kokotovic, Nonlinear and Adaptive Control Design.  New York: John Wiley & Sons, 1995.
  * [17] P. Kidger and T. Lyons, “Universal approximation with deep narrow networks,” in Conf. Learn. Theory, pp. 2306–2327, 2020.
  * [18] F. L. Lewis, A. Yesildirek, and K. Liu, “Multilayer neural-net robot controller with guaranteed tracking performance,” IEEE Trans. on Neural Netw., vol. 7, no. 2, pp. 388–399, 1996.
  * [19] N. Fischer, R. Kamalapurkar, and W. E. Dixon, “LaSalle-Yoshizawa corollaries for nonsmooth systems,” IEEE Trans. Autom. Control, vol. 58, pp. 2333–2338, Sep. 2013.
  * [20] F. H. Clarke, Optimization and nonsmooth analysis.  SIAM, 1990.