remarkRemark hypothesisHypothesis claimClaim Composing MPC with LQR and Neural
NetworksF. Wu, G. Wang, S. Zhuang, K. Wang, A. Keimer, I. Stoica, and A. Bayen

# Composing MPC with LQR and Neural Networks
for Efficient and Stable Control††thanks: Submitted to the editors .

Fangyu Wu Department of EECS, UC Berkeley, Berkeley, CA. For correspondence,
please contact Fangyu Wu<EMAIL_ADDRESS>Guanhua Wang22footnotemark:
2 Siyuan Zhuang22footnotemark: 2 Kehan Wang22footnotemark: 2 Alexander
Keimer22footnotemark: 2 Ion Stoica22footnotemark: 2 Alexandre
Bayen22footnotemark: 2

###### Abstract

Model predictive control (MPC) is a powerful control method that handles
dynamical systems with constraints. However, solving MPC iteratively in real
time, i.e., implicit MPC, has been a challenge for 1) systems with low-latency
requirements, 2) systems with limited computational resources, and 3) systems
with fast and complex dynamics. To address this challenge, for low-dimensional
linear systems, a classical approach is explicit MPC; for high-dimensional or
nonlinear systems, a common approach is function approximation using neural
networks. Both methods, whenever applicable, may improve the computational
efficiency of the original MPC by several orders of magnitude. The existing
methods have the following disadvantages: 1) explicit MPC does not apply to
higher-dimensional problems or most of the problems with nonlinear
constraints; and 2) function approximation is not guaranteed to find an
accurate surrogate policy, the failure of which may lead to closed-loop
instability. To address these issues, we propose a triple-mode hybrid control
scheme, named Memory-Augmented MPC, by combining an efficient linear quadratic
regulator, an efficient neural network, and a costly, fail-safe MPC. From its
standard form, we derive two variants of such hybrid control scheme: one
customized for chaotic systems and the other for slow systems. We prove
stability of the control scheme with any arbitrary neural networks and test
its computational performance in simulated numerical experiments.

###### keywords:

Model predictive control, computing methodologies, robotics

93B45, 68U99, 68T40

##  1 Introduction

Model predictive control (MPC) is a powerful method for controlling dynamical
systems with constraints [25, 4]. It has been widely applied in robotics,
ranging from ground [5] and aerial [1] vehicle control to humanoid [16] and
quadruped [3] robot control.

To implement MPC, one has two conventional approaches: 1) implicit MPC, which
solves an optimization problem in real time with a preferably efficient
problem formulation and numerical scheme, 2) explicit MPC, which caches every
solution of the optimization problem offline and looks up the cached solutions
online.

Explicit MPC tends to have faster running time than implicit MPC. However,
existing explicit methods have limited use cases. Common explicit MPC methods
only handle MPC with quadratic cost and linear constraints. Within the set of
applicable problems, explicit MPC does not work well for MPC of many variables
and complex constraints, since caching solutions of these problems tends to
demand a prohibitively large amount of memory.

Alternatively, one can approximate an MPC with a neural network (NN) through
supervised learning. It develops a surrogate model of MPC by fitting a neural
network into a trajectory dataset generated using implicit MPC. A well-trained
neural network model behaves similarly to the original MPC model and can
achieve similar control effectiveness at a fraction of the MPC’s computational
budget. Nevertheless, to our best knowledge, we find the NN function
approximation approach has yet to fully address the following open problems:
1) it is difficult, if not impossible, to guarantee that the approximation
will always converge to a solution of least approximation errors; 2) it is
challenging to verify that the approximation will produce a stable closed-loop
dynamics without any knowledge of approximation errors between the original
and the approximated control laws.

Motivated by the shortcomings of explicit MPC and NN function approximation,
we propose a triple-mode hybrid control scheme, named Memory-Augmented MPC
(MAMPC). The core idea is to mix a costly MPC controller with an efficient
linear quadratic regulator (LQR) controller and an efficient NN controller,
whenever stability permits. We summarize the main contributions of our work
below.

  * •

We present a novel controller design, i.e., Memory-Augmented MPC, in its
standard form and two modified forms.

  * •

We prove stability of Memory-Augmented MPC without imposing any condition on
the approximation errors of the neural network.

  * •

We demonstrate computational performance of Memory-Augmented MPC via four
simulated robotic control applications.

The rest of the article is organized as follows. A brief literature review is
provided in Section 2 to situate the work in context. Terminology and
notations of MPC are introduced in Section 3. Standard and modified versions
of MAMPC are presented in Section 4. Theoretical properties of MAMPC,
including stability, are proven in Section 5. Four numerical experiments are
presented in Section 6 and discussed in Section 7. Lastly, major findings and
future work are briefed in Section 8.

##  2 Related Works

A major limitation of MPC has been a lack of computational efficiency as
evident in many robotic applications such as the Atlas humanoid robot [16] and
the MIT Cheetah robot [3]. To overcome this shortcoming, we have three basic
approaches: 1) developing an efficient implicit MPC policy, 2) precompiling an
explicit MPC policy, and 3) approximating the MPC policy by an efficient
surrogate policy such as a NN.

Implicit MPC approach implements the MPC policy by developing an efficient
numerical scheme for solving the optimization program. This is perhaps the
most common approach of implementing a MPC policy. Efficient implicit MPC
controller often depends on custom optimization solvers that leverage
structures specific to the problems they solve, such as [8, 18].
Implementation of such implicit MPC is usually done in a low-level programming
language such as C [10, 19]. Despite of being computationally expensive,
recent development in algorithms and hardware enables some implicit MPC
controllers to achieve high-frequency performance of approximately 1000 Hz [9,
15].

When existing implicit MPC methods do not provide satisfactory latency, one
sometimes resorts to explicit MPC, whenever such alternative is applicable.
Explicit MPC approach implements the MPC policy by precomputing the solution
of the optimization program and looking up the solution in real time. At its
core, explicit methods achieve reduced running time at the cost of increased
memory footprint. To this end, numerous works has been developed to reduce its
space complexity [27, 17]. Nevertheless, due to poor scalability and lack of
theory on nonlinear systems, explicit MPC has limited use cases in large-scale
robotic applications.

When the explicit MPC method fails to apply to a problem due to scalability or
nonlinearity, one may consider another alternative, namely, function
approximation. Function approximation approach implements a MPC policy by
approximating offline that MPC control law with an efficient surrogate model,
such as a neural network, and uses that surrogate model for online deployment.
Early works in this direction include [23, 22], which has described a
procedure to design a surrogate NN controller and has proposed conditions to
guarantee its closed-loop stability. Later development has focused on
guaranteeing different theoretical properties of the NN-controlled closed-loop
system. For example, [30] proposes a NN method with probabilistic optimality
bounds; [24] has designed a projection operator that can modify any arbitrary
NN controller to a stabilizing one; [12] proposes a mixed-integer linear
program to certify stability and feasibility of ReLU-activated NN controllers;
and [11] has developed a method that guarantees certain performance indicator
of a NN-controlled system using probabilistic validation methods. Besides,
rather than replacing the original MPC controller, a very recent work [7] uses
a NN to warm start an optimization solver, which can then solve the MPC policy
faster in real time. For a comprehensive study, readers may consult [31].
Applications of function approximation methods have found successes in
practice, such as [21].

It is worth noting that closed-loop stability of the function approximation
methods above, except [24], requires reasonable convergence of NN model to the
original control law. Nevertheless, training a NN is commonly a high-
dimensional nonlinear program. This nonlinear program is solved with faith by
a black-box solver, such as [14], which in general does not come with any
guarantee on convergence or optimality. Hence, for most of the existing works,
their closed-loop stability replies in large part on the faithful assumption
that there exists a black-box optimization procedure that can somehow find a
good approximation to the original MPC.

In contrary, we propose a novel approach named Memory-Augmented MPC, which can
work without any condition on the convergence and optimality of the NN.
Compared to the technique in [24], which only applies to discrete-time linear
time-invariant plants, our method also works for general nonlinear plants.
Concretely, the hybrid controller combines a NN with a costly MPC and an
efficient LQR. This hybrid design is an extension of the early dual-mode MPC
design [20], which just combines a MPC with a LQR. An advantage of our
approach over [20] is that it may provide more reduction in latency owing to
the addition of a fast intermediate NN mode. As presented in Section 7, with
parallelization, MAMPC is at worse equivalent to [20].

We continue the study by next introducing a basic set of terminology and
notations in Section 3.

##  3 A Primer on MPC

We consider the discrete-time time-invariant constrained dynamical systems of
the form

(1) $\mathbf{x}[i+1]=f(\mathbf{x}[i],\mathbf{u}[i]),\quad i\in\mathbb{Z}_{\geq
0}$

where

$\displaystyle\mathbf{x}[0]=\mathbf{x}_{0}\quad\text{ for some
}\mathbf{x}_{0}\in\mathbb{R}^{n},$
$\displaystyle(\mathbf{x}[i],\mathbf{u}[i])\in\mathbb{A},\quad\forall i>0,$

where $\mathbb{A}\subset\mathbb{R}^{n}\times\mathbb{R}^{m}$ is a closed set
defining the system constraints, $\mathbf{x}[i]$ is the state at time index
$i$, $\mathbf{u}[i]$ is the input at time index $i$,
$f:\mathbb{R}^{n}\times\mathbb{R}^{m}\rightarrow\mathbb{R}^{n}$ is
continuously differentiable in both its components, with an equilibrium point
at ($\mathbf{0}_{n}$, $\mathbf{0}_{m}$), i.e.,
$f(\mathbf{0}_{n},\mathbf{0}_{m})=\mathbf{0}_{n}$, and the linearized time-
invariant system of $f(\cdot,\cdot)$ around the equilibrium is stabilizable.

Let $\mathbb{X}$ be state constraint set and $\mathbb{U}$ be input constraint
set. We consider system constraints of the following form
$\mathbb{A}\coloneqq\mathbb{X}\times\mathbb{U}$. The system dynamics is a map
$f:\mathbb{A}\rightarrow\mathbb{X}$ and any control law is a map
$u:\mathbb{X}\rightarrow\mathbb{U}$.

Consider a finite planning and control horizon $N$. Let
$\mathcal{X}^{0:N}\coloneqq(\mathbf{x}[0],\dots,\mathbf{x}[N])$ be a
trajectory of the system and
$\mathcal{U}^{0:N-1}\coloneqq(\mathbf{u}[0],\dots,\mathbf{u}[N-1])$ be the
corresponding ordered control sequence.

Provided with the dynamical system (1), a MPC is an optimal control law
implicitly defined through the following set of optimization problems

(2) $\displaystyle\min_{\mathcal{X}_{i}^{1:N},\mathcal{U}_{i}^{0:N-1}}$
$\displaystyle\sum_{k=0}^{N-1}{c(\mathbf{x}[k|i],\mathbf{u}[k|i])}+c_{f}(\mathbf{x}[N|i])$
s.t.: $\displaystyle\mathbf{x}[0|i]=\mathbf{x}[i],$
$\displaystyle\mathbf{x}[k+1|i]=f(\mathbf{x}[k|i],\mathbf{u}[k|i]),$
$\displaystyle(\mathbf{x}[k|i],\mathbf{u}[k|i])\in\mathbb{A},$
$\displaystyle\mathbf{x}[N|i]\in\mathbb{X}_{f},$ $\displaystyle
k=0,\dots,N-1,$

where $N$ is planning and control horizon,
$c:\mathbb{A}\rightarrow\mathbb{R}_{+}$ is a continuous stage cost function,
$c_{f}:\mathbb{X}_{f}\rightarrow\mathbb{R}_{+}$ is a continuous terminal cost
function, $\mathbf{x}[k|i]$ is the predicted state $k$ steps ahead of present
state $\mathbf{x}[i]$, $\mathbf{u}[k|i]$ is the anticipated input that
generates $\mathbf{x}[k+1|i]$ from $\mathbf{x}[k|i]$,
$\mathcal{X}_{i}^{1:N}\coloneqq(\mathbf{x}[1|i],\dots,\mathbf{x}[N|i])$ and
$\mathcal{U}_{i}^{0:N-1}\coloneqq(\mathbf{u}[0|i],\dots,\mathbf{u}[N-1|i])$
are two sets of optimization variables corresponding to predicted states and
anticipated inputs, $\mathbb{X}_{f}$ is a terminal constraint set containing
the origin usually designed to guarantee asymptotic stability. If optimization
problem (2) is feasible and has an optimal solution
$(\mathcal{X}_{i}^{1:N*},\mathcal{U}_{i}^{0:N-1*})$, then the MPC control law
selects the first element of $\mathcal{U}_{i}^{0:N-1*}$ as the input at time
index $i$, that is,

(3)
$u_{\textnormal{MPC}}(\mathbf{x}[i])\coloneqq\mathbf{u}^{*}[0|i]=\mathcal{U}_{i}^{0:N-1*}[0].$

This process is repeated at the next time index $i+1$, until some termination
criterion is met. As a result, we obtain a closed-loop system

(4)
$\mathbf{x}[i+1]=f(\mathbf{x}[i],u_{\textnormal{MPC}}(\mathbf{x}[i])),\quad
i\in\mathbb{Z}_{\geq 0},$

with initial condition
$\mathbf{x}[0]=\mathbf{x}_{0}\in\mathbb{X}_{0}\coloneqq\\{\mathbf{x}_{0}\in\mathbb{R}^{n}\mid\text{problem
\eqref{eq:mpc} is feasible}\\}$ is the set of all admissible initial states.
For well-posedness of MPC, we require $\mathbb{X}\subseteq\mathbb{X}_{0}$. Let
$J(\mathbf{x}[i])\coloneqq\sum_{k=0}^{N-1}{c(\mathbf{x}[k|i],\mathbf{u}[k|i])}+c_{f}(\mathbf{x}[N|i])$
be the cumulative cost and
$J^{*}(\mathbf{x}[i])\coloneqq\sum_{k=0}^{N-1}{c(\mathbf{x}^{*}[k|i],\mathbf{u}^{*}[k|i])}+c_{f}(\mathbf{x}^{*}[N|i])$
be the optimal cumulative cost. By Lyapunov stability theorem, we can derive
the following sufficient condition for asymptotic stability of MPC as in
Theorem 12.2 in [4].

######  Theorem 3.1 (Local Asymptotic Stability of MPC).

Assume that

  * •

The stage cost and terminal cost are continuous and positive definite.

  * •

$\mathbb{X}_{f}$ is control invariant.

  * •

For all $\mathbf{x}\in\mathbb{X}_{f}$,

$\displaystyle\min_{\mathbf{u}:\mathbf{u}\in\mathbb{U},f(\mathbf{x},\mathbf{u})\in\mathbb{X}_{f}}\quad
c(\mathbf{x},\mathbf{u})-c_{f}(\mathbf{x})+c_{f}(f(\mathbf{x},\mathbf{u}))\leq
0.$

then the MPC problem (2) is persistently feasible and the closed-loop system
(4) is locally asymptotically stable with respect to the origin. In addition,
$J^{*}(\cdot)$ is a Lyapunov function and $\mathbb{X}_{0}$ is a positively
invariant region of attraction.

With the above definitions and theorem, we are ready to describe the
composition rules of MPC, LQR, and NN in Section 4.

##  4 Composing MPC, LQR, and NN

Provided with a computationally costly implicit MPC $u_{\textnormal{MPC}}$
that satisfies Theorem (3.1), we propose a hybrid control scheme, namely,
Memory-Augmented Model Predictive Control (MAMPC),
$u_{\textnormal{MAMPC}}:\mathbb{X}\rightarrow\mathbb{U}$, by augmenting
$u_{\textnormal{MPC}}$ with a LQR controller $u_{\textnormal{LQR}}$ and a NN
controller $u_{\textnormal{NN}}$, where the LQR controller and the NN
controller are independently developed from the implicit MPC. In addition, we
present two variants of MAMPC specifically tailored for chaotic systems and
slow systems, respectively.

###  4.1 LQR from MPC

Provided with a MPC, we derive an infinite-time LQR controller by 1)
linearizing the system dynamics around the equilibrium, 2) removing stage
constraints, terminal constraint, and terminal cost, 3) taking planning and
control horizon $N$ to $\infty$, and 4) replacing stage cost $c(\cdot)$ with a
positive definite quadratic cost, if it is not already so in the original MPC.

Formally, the MPC-induced LQR problem is defined as follows

(5)
$\displaystyle\min_{\mathcal{X}_{i}^{1:\infty},\mathcal{U}_{i}^{0:\infty}}$
$\displaystyle\sum_{k=0}^{\infty}\mathbf{x}^{\top}[k|i]\bm{Q}\mathbf{x}[k|i]+\mathbf{u}^{\top}[k|i]\bm{R}\mathbf{u}[k|i]$
s.t.: $\displaystyle\mathbf{x}[0|i]=\mathbf{x}[i],$
$\displaystyle\mathbf{x}[k+1|i]=\bm{A}\mathbf{x}[k|i]+\bm{B}\mathbf{u}[k|i],$
$\displaystyle k\in\mathbb{Z}_{+},$

where $\bm{A}\coloneqq\left.\frac{\partial
f}{\partial\mathbf{x}}\right\rvert_{(\mathbf{0},\mathbf{0})}\in\mathbb{R}^{n\times
n},\bm{B}\coloneqq\left.\frac{\partial
f}{\partial\mathbf{u}}\right\rvert_{(\mathbf{0},\mathbf{0})}\in\mathbb{R}^{n\times
m}$ are linearized system dynamics around the equilibrium, and
$\bm{Q}\in\\{\bm{N}\in\mathbb{R}^{n\times n}\mid\bm{N}\succeq
0\\},\bm{R}\in\\{\bm{M}\in\mathbb{R}^{m\times m}\mid\bm{M}\succeq 0\\}$ are
weighting matrices that measure significance of state deviations and control
costs, respectively.

It can be shown that the optimal LQR control law for an initial state
$\mathbf{x}$ is a linear map

(6) $u_{\textnormal{LQR}}(\mathbf{x})=-\bm{K}\mathbf{x}$

for some $\bm{K}\in\mathbb{R}^{m\times n}$ [2]. Furthermore, if the linearized
system $(\bm{A},\bm{B})$ is stabilizable and if $\bm{A}-\bm{B}\bm{K}$ is
Hurwitz, then the closed-loop system

(7) $\mathbf{x}[i+1]=f(\mathbf{x}[i],u_{\textnormal{LQR}}(\mathbf{x}[i])),$

is locally asymptotically stable near the equilibrium with a positively
invariant region of attraction $\mathcal{R}_{\textnormal{LQR}}$, as shown in
Theorem 4.7 in [13]. Specifically, $\mathcal{R}_{\textnormal{LQR}}$ is a set
that if $\mathbf{x}_{0}\in\mathcal{R}_{\textnormal{LQR}}$ and
$\mathbf{x}[0]=\mathbf{x}_{0}$, then
$\mathbf{x}[i]\in\mathcal{R}_{\textnormal{LQR}},\forall i\in\mathbb{Z}_{>0}$
and $\lim_{i\rightarrow\infty}\mathbf{x}[i]=\mathbf{0}$. As implied by local
asymptotic stability, the largest possible $\mathcal{R}_{\textnormal{LQR}}$ is
nonempty.

Because the LQR controller only requires one matrix multiplication within a
single controller step, it is usually significantly more efficient than the
implicit MPC in terms of per-step computation. Hence, it is usually economical
to switch to the LQR controller when the state is in the region of attraction
of the LQR, that is, $\mathbf{x}[i]\in\mathcal{R}_{\textnormal{LQR}}$.

###  4.2 NN from MPC

Provided with a MPC, we derive a NN controller 1) by imitating the MPC policy
$u_{\textnormal{MPC}}(\mathbf{x})$ through supervised learning and 2)
optionally by interacting with the environment through reinforcement learning,
as illustrated in Figure 1.

$u_{\textnormal{MPC}}(\mathbf{x})$$u_{\textnormal{NN}}(\mathbf{x})$(a)
Imitation phase
$f(\mathbf{x},u_{\textnormal{NN}}(\mathbf{x}))$$u_{\textnormal{NN}}(\mathbf{x})$(b)
Adaptation phase Figure 1:  Schematics of the learning processes. (a)
Imitation phase: $(\mathbf{x},u_{\textnormal{MPC}}(\mathbf{x}))$ provides data
to train the NN controller $u_{\textnormal{NN}}$ in a supervised learning
setting. (b) Adaptation phase (optional): $u_{\textnormal{NN}}$ interacts with
the environment $f(\cdot,\cdot)$ to enhance control effectiveness in a
reinforcement learning setting.

In the imitation phase, the NN training is a supervised learning problem
usually of the following mean squared form

(8)
$\min_{\mathcal{W}}\quad\frac{1}{M}\sum_{\mathbf{x}\in\mathcal{D}}\left\|u_{\textnormal{MPC}}(\mathbf{x})-\phi(\mathbf{x}\mid\mathcal{W})\right\|_{2}^{2},$

where $\phi:\mathbb{R}^{n}\rightarrow\mathbb{R}^{m}$ is a NN model with
weights $\mathcal{W}$ and $\mathcal{D}$ is a set of $M$ states randomly
sampled from $\mathbb{X}_{0}$.

Therefore, the resulting NN control law is

(9)
$u_{\textnormal{NN}}(\mathbf{x})\coloneqq\phi(\mathbf{x}\mid\overline{\mathcal{W}}),$

where $\overline{\mathcal{W}}$ is a suboptimal solution to problem (8),
generally estimated via a gradient method such as Adam [14].

In the adaptation phase, it is possible to fine tune the NN further through a
reinforcement learning (RL) algorithm. The training setup varies based on the
specific choice of the RL algorithm. A comprehensive review of reinforcement
learning is clearly beyond the scope of this work. Interested readers may
refer to the standard text on this topic [28]. Lastly, we remark that this
phase may be skipped if the NN trained in the imitation phase is sufficiently
effective for the target control application or if the RL algorithm does not
bring any improvement to the trained NN.

In practice, people find that implicit MPC controllers can be well
approximated by a more efficient NN controller [23]. However, it is not easy
to certify stability of the closed-loop system without bounding approximation
errors between the NN controller and the original MPC controller.

(10) $\mathbf{x}[i+1]=f(\mathbf{x}[i],u_{\textnormal{NN}}(\mathbf{x}[i])).$

To solve the problem, we show that by composing the NN controller with the MPC
and the LQR in a hybrid control scheme, we can prove the local asymptotic
stability of the closed-loop system at an acceptable cost of computational
efficiency, even if the NN is random. Next, we present the basic form of our
method, i.e., the standard MAMPC.

###  4.3 Standard Memory-Augmented MPC

The hybrid control scheme combines the LQR controller and the NN controller
with the original MPC controller. At every control step, if the state is in
the region of attraction of the LQR, apply the LQR controller; else, we
simulate the closed-loop system (10) for up to $N_{\textnormal{LQR}}$ steps:
if there exists a step $j\leq N_{\textnormal{LQR}}$ such that the state
reaches within the region of attraction of LQR and that up until the
$j^{\text{th}}$ step the simulated system does not violate any stage
constraint of the MPC, apply the NN controller; otherwise, if the state is
within the admissible initial states of the MPC, apply the implicit MPC.

Formally, the hybrid controller is defined as follows

(11) $\displaystyle
u_{\textnormal{MAMPC}}(\mathbf{x})\coloneqq\begin{cases}u_{\textnormal{LQR}}(\mathbf{x}),&\textnormal{if
}\mathbf{x}\in\mathcal{R}_{\textnormal{LQR}},\\\ \hline\cr&\textnormal{if
}\mathbf{x}\in\mathbb{X}_{0}\setminus\mathcal{R}_{\textnormal{LQR}}\quad\textnormal{and}\\\
u_{\textnormal{NN}}(\mathbf{x}),&\quad\exists
i=1,\dots,N_{\textnormal{LQR}},\quad\mathbf{y}[i]\in\mathcal{R}_{\textnormal{LQR}}\quad\textnormal{and}\\\
&\quad\forall
j=0,\dots,i,\quad(\mathbf{y}[j],u_{\textnormal{NN}}(\mathbf{y}[j]))\in\mathbb{A},\\\
\hline\cr u_{\textnormal{MPC}}(\mathbf{x}),&\text{otherwise},\end{cases}$

where $\mathbf{y}$ is the simulated state by numerically stepping:
$\mathbf{y}[i]=f(\mathbf{y}[i-1],u_{\textnormal{NN}}(\mathbf{y}[i-1]))$ with
$\mathbf{y}[0]=\mathbf{x}$, $\mathcal{R}_{\textnormal{LQR}}$ is designed to be
a subset of $\mathbb{X}_{0}$, and $N_{\textnormal{LQR}}\in\mathbb{Z}_{>0}$ is
the verification horizon of $\mathcal{R}_{\textnormal{LQR}}$.

A pseudocode of the $u_{\textnormal{MAMPC}}(\mathbf{x})$ is described in
Algorithm 1.

0: State $\mathbf{x}$

0: Control $\mathbf{u}$

1: if $\mathbf{x}\in\mathcal{R}_{\textnormal{LQR}}$ then

2: $\mathbf{u}\leftarrow u_{\textnormal{LQR}}(\mathbf{x})$

3: else if $u_{\textnormal{NN}}$ steers $f$ into
$\mathcal{R}_{\textnormal{LQR}}$ within $N_{\textnormal{LQR}}$ steps without
violating $\mathbb{A}$ then

4: $\mathbf{u}\leftarrow u_{\textnormal{NN}}(\mathbf{x})$

5: else

6: $\mathbf{u}\leftarrow u_{\textnormal{MPC}}(\mathbf{x})$

7: end if

8: return $\mathbf{u}$

Algorithm 1 Standard MAMPC

As we show in Section 5, the stability of NN relies on verifying whether the
system reaches within $\mathcal{R}_{\textnormal{LQR}}$ through forward
simulation. However, this approach is ineffective for systems that are
sensitive to initial conditions and systems that require a large number of
steps to converge to the origin. To address these two challenges, we introduce
the following two variants of MAMPC.

###  4.4 Alternating-Authority Memory-Augmented MPC

The first variant of MAMPC is designed specifically for chaotic systems. A
chaotic system is loosely defined as a system that is very sensitive to
initial condition and control input. Qualitatively speaking, the same chaotic
system that begins with two slightly different initial conditions and control
sequences will arrive at significantly different terminal states. As a result,
it is very challenging to stabilize a chaotic system with any NN controller
because the function approximator will inevitably produce random control
errors which can easily deter the system from its stabilizing trajectory. In
this case, it is useful to modify the hybrid control scheme to bound error
accumulation induced by the NN by periodically alternating between the NN
controller and the MPC controller. We name this modified version of MAMPC as
alternating-authority MAMPC, which is formally described below.

(12) $\displaystyle
u_{\textnormal{MAMPC}}^{\textnormal{AA}}(\mathbf{x})\coloneqq\begin{cases}u_{\textnormal{LQR}}(\mathbf{x}),&\textnormal{if
}\mathbf{x}\in\mathcal{R}_{\textnormal{LQR}},\\\ \hline\cr&\textnormal{if
}\mathbf{x}\in\mathbb{X}_{0}\setminus\mathcal{R}_{\textnormal{LQR}}\quad\textnormal{and}\\\
u_{\textnormal{NN}}(\mathbf{x}),&\quad
u_{\textnormal{NN}}(\mathbf{x})\in\mathbb{U}\land\mathbf{y}[1]\in\mathbb{X}\quad\textnormal{and}\\\
&\quad i\ (\mathrm{mod}\ i_{d})\neq 0,\\\ \hline\cr
u_{\textnormal{MPC}}(\mathbf{x}),&\text{otherwise},\end{cases}$

where $i_{d}\in\mathbb{Z}_{\geq 1}$ is the period of MPC defaulting and
$\mathcal{R}_{\textnormal{LQR}}\subset\mathbb{X}_{0}$.

A pseudocode of the $u_{\textnormal{MAMPC}}^{\textnormal{AA}}(\mathbf{x})$ is
described in Algorithm 2.

0: State $\mathbf{x}$

0: Control $\mathbf{u}$

1: if $\mathbf{x}\in\mathcal{R}_{\textnormal{LQR}}$ then

2: $\mathbf{u}\leftarrow u_{\textnormal{LQR}}(\mathbf{x})$

3: else if $i\ (\mathrm{mod}\ i_{d})\neq 0$ and
$u_{\textnormal{NN}}(\mathbf{x})$ keeps $f$ inside $\mathbb{X}$ without
violating $\mathbb{U}$ then

4: $\mathbf{u}\leftarrow u_{\textnormal{NN}}(\mathbf{x})$

5: else

6: $\mathbf{u}\leftarrow u_{\textnormal{MPC}}(\mathbf{x})$

7: end if

Algorithm 2 Alternating-Authority MAMPC

The main variation from the standard MAMPC is that we do not verify if the
system falls within $\mathcal{R}_{\textnormal{LQR}}$ in $N_{\textnormal{LQR}}$
steps, as forward simulation is not as reliable for chaotic systems. This
implies that $i_{d}$ should be chosen as a small positive number. Otherwise,
prediction produced by forward simulation will be a reliable estimate of the
future. However, note that reducing $i_{d}$ will prolong running time.
Therefore, a rule of thumb of designing $i_{d}$ should be to choose it as
large as the stability permits.

In addition, we remark that the closed-loop system has a smaller region of
attaction compared to standard MAMPC, since we do not verify whether the
system enters $\mathcal{R}_{\textnormal{LQR}}$ through forward simulation.

###  4.5 Way-Point Memory-Augmented MPC

Another variant of MAMPC is designed specifically for slow systems. A slow
system is loosely defined as a system that requires a substantial number of
steps to steer close to the origin. For slow systems, it is difficult to
choose an effective verification horizon $N_{\textnormal{LQR}}$: a small
$N_{\textnormal{LQR}}$ may be too short to predict convergence of the NN
controller; a large $N_{\textnormal{LQR}}$ may incur computational overhead
and thus render the hybrid control inefficient. To address this problem, we
propose a second variant of MAMPC by introducing a “way-point” set,
$\mathcal{D}_{\textnormal{WP}}$, and enabling the NN controller to be applied
as long as the anticipated trajectory falls with that way-point set. We name
this variant of MAMPC as way-point MAMPC, which is formulated as follows.

(13) $\displaystyle
u_{\textnormal{MAMPC}}^{\textnormal{WP}}(\mathbf{x})\begin{cases}u_{\textnormal{LQR}}(\mathbf{x}),&\textnormal{if
}\mathbf{x}\in\mathcal{R}_{\textnormal{LQR}},\\\ \hline\cr&\textnormal{if
}\mathbf{x}\in\mathcal{D}_{\textnormal{WP}}\setminus\mathcal{R}_{\textnormal{LQR}}\quad\textnormal{and}\\\
u_{\textnormal{NN}}(\mathbf{x}),&\quad\exists i=1,\dots
N_{\textnormal{LQR}},\quad\mathbf{y}[i]\in\mathcal{R}_{\textnormal{LQR}}\quad\textnormal{and}\\\
&\quad\forall j=0,\dots
i,\quad(\mathbf{y}[j],u_{\textnormal{NN}}(\mathbf{y}[j]))\in\mathbb{A}_{\textnormal{WP}},\\\
\hline\cr&\textnormal{if
}\mathbf{x}\in\mathbb{X}_{0}\setminus\mathcal{D}_{\textnormal{WP}}\quad\textnormal{and}\\\
u_{\textnormal{NN}}(\mathbf{x}),&\quad\exists i\in
1,\dots,N_{\textnormal{WP}},\quad\mathbf{y}[i]\in\mathcal{D}_{\textnormal{WP}}\quad\textnormal{and}\\\
&\quad\forall j\in
0,\dots,i,\quad(\mathbf{y}[j],u_{\textnormal{NN}}(\mathbf{y}[j]))\in\mathbb{A},\\\
\hline\cr u_{\textnormal{MPC}}(\mathbf{x}),&\text{otherwise},\end{cases}$

where $\mathcal{D}_{\textnormal{WP}}$ is a compact set that contains
$\mathcal{R}_{\textnormal{LQR}}$,
$\mathbb{A}_{\textnormal{WP}}\coloneqq\mathcal{D}_{\textnormal{WP}}\times\mathbb{U}$,
and $N_{\textnormal{WP}}\in\mathbb{Z}_{>0}$ is the verification horizon of
$\mathcal{D}_{\textnormal{WP}}$.

0: State $\mathbf{x}$

0: Control $\mathbf{u}$

1: if $\mathbf{x}\in\mathcal{R}_{\textnormal{LQR}}$ then

2: $\mathbf{u}\leftarrow u_{\textnormal{LQR}}(\mathbf{x})$

3: else if
$\mathbf{x}\in\mathcal{D}_{\textnormal{WP}}\setminus\mathcal{R}_{\textnormal{LQR}}$
then

4: if $u_{\textnormal{NN}}$ steers $f$ into $\mathcal{R}_{\textnormal{LQR}}$
within $N_{\textnormal{LQR}}$ steps without violating
$\mathbb{A}_{\textnormal{WP}}$ then

5: $\mathbf{u}\leftarrow u_{\textnormal{NN}}(\mathbf{x})$

6: else

7: $\mathbf{u}\leftarrow u_{\textnormal{MPC}}(\mathbf{x})$

8: end if

9: else

10: if $u_{\textnormal{NN}}$ steers $f$ into $\mathcal{D}_{\textnormal{WP}}$
within $N_{\textnormal{WP}}$ steps without violating $\mathbb{A}$ then

11: $\mathbf{u}\leftarrow u_{\textnormal{NN}}(\mathbf{x})$

12: else

13: $\mathbf{u}\leftarrow u_{\textnormal{MPC}}(\mathbf{x})$

14: end if

15: end if

16: return $\mathbf{u}$

Algorithm 3 Way-Point MAMPC (a) Standard MAMPC(b) A.A. MAMPC(c) W.P. MAMPC
Figure 2:  An illustrative example of how different variants of MAMPC could
steer a system toward equilibrium. (a) Standard MAMPC: a triple-mode hybrid
control composed of a MPC, a NN, and a LQR. (b) Alternating-authority MAMPC: a
variant of the standard MAMPC, modified for chaotic systems. (c) Way-point
MAMPC: a variant of the standard MAMPC, modified for slow systems. Solid red
regions represent $\mathbb{X}_{0}$; solid green regions represent
$\mathcal{R}_{\textnormal{NN}}$; green dots indicate periodic controller
switching region in alternating-authority MAMPC; and solid blue regions
represent $\mathcal{R}_{\textnormal{LQR}}$. A black dot represents a state and
an arrow represents a controlled state transition. The color of arrows
indicates which sub-controller is invoked during that control step: red is
MPC; green is NN; and blue is LQR. The yellow line represents
$\partial{\mathcal{D}_{\textnormal{WP}}}$. Note that the above examples by no
means exhaustive. For instance, it is possible that
$\mathcal{R}_{\textnormal{NN}}=\emptyset$.

At minimum, we require that the way-point set contains the region of
attraction of the LQR, that is,
$\mathcal{R}_{\textnormal{LQR}}\subset\mathcal{D}_{\textnormal{WP}}$.
Additional improvement of the way-point set design can be performed through
search. For example, one can parameterize $\mathcal{D}_{\textnormal{WP}}$ with
one or more parameters and search for a set of parameters that produces the
most efficient closed-loop performance.

Besides, note that introduction of a way-point set may reduce the area of the
region of attraction of the closed-loop system.

With the three MAMPC hybrid control schemes defined above, we are ready to
analyze their theoretical properties in Section 5.

##  5 Theoretical Analysis

In this section, we prove the stability of the three MAMPC methods proposed
above. In addition, we remark on robustness under model uncertainties and
necessity of the fail-safe mode.

###  5.1 Stability of MAMPC

We show that the three MAMPC methods are locally asymptotically stable with
slightly different stability properties.

We first prove that standard MAMPC is locally asymptotically stable in
$\mathbb{X}_{0}$, as stated in the theorem below.

######  Theorem 5.1 (Stability of Standard MAMPC).

For every MPC problem of the form (2), there always exists a
$u_{\textnormal{MAMPC}}$ of the form (11). Furthermore, the closed-loop system

(14)
$\mathbf{x}[i+1]=f\left(\mathbf{x}[i],u_{\textnormal{MAMPC}}(\mathbf{x}[i])\right),\quad
i\in\mathbb{Z}_{\geq 0},$

with $\mathbf{x}[0]=\mathbf{x}_{0}\in\mathbb{X}_{0}$ is locally asymptotically
stable in $\mathbb{X}_{0}$.

######  Proof 5.2.

Existence of MAMPC depends on existence of LQR and NN. A LQR controller of the
form (5) can always be derived from (2) because 1) $f$ is continuously
differentiable, so it can always be linearized to produce the $\bm{A},\bm{B}$
matrices in (5); 2) the linearized system $(\bm{A},\bm{B})$ is by assumption
stabilizable, so region of attraction of LQR is nonempty, i.e.,
$\mathcal{R}_{\textnormal{LQR}}\neq\emptyset$; 3) as for $\bm{Q},\bm{R}$ we
only require $\bm{Q}\succeq 0,\bm{R}\succ 0$, which is always possible; 4)
lastly, optimization problem (5) is just a relaxation of problem (2), which
can always be solved.

Meanwhile, a NN controller trivially exists because we do not require NN to
satisfy any properties other than the basic definition of a NN. This completes
the first part of the proof on existence.

To prove local asymptotic stability of standard MAMPC in $\mathbb{X}_{0}$, we
partition $\mathbb{X}_{0}$ into three regions as follows

$\mathbb{X}_{0}=\mathcal{R}_{\textnormal{LQR}}\cup\mathcal{R}_{\textnormal{NN}}\cup\mathcal{R}_{\textnormal{MPC}},$

where $\mathcal{R}_{\textnormal{NN}}$ is the set of states where the NN is
invoked and $\mathcal{R}_{\textnormal{MPC}}$ is the rest of states in
$\mathbb{X}_{0}$, i.e.,
$\mathcal{R}_{\textnormal{MPC}}\coloneqq\mathbb{X}_{0}\setminus(\mathcal{R}_{\textnormal{LQR}}\cup\mathcal{R}_{\textnormal{NN}})$.
By construction, $\mathcal{R}_{\textnormal{LQR}}$,
$\mathcal{R}_{\textnormal{NN}}$, and $\mathcal{R}_{\textnormal{MPC}}$ are
mutually disjoint.

For every state $\mathbf{x}[0]\in\mathcal{R}_{\textnormal{LQR}}$,
$u_{\textnormal{LQR}}$ is invoked and the closed-loop system (14) will be
equivalent to (7), which is locally asymptotically stable.

For every state $\mathbf{x}[0]\in\mathcal{R}_{\textnormal{NN}}$,
$u_{\textnormal{NN}}$ is invoked and the closed-loop system (14) will be
equivalent to (10) until the state enters $\mathcal{R}_{\textnormal{LQR}}$ in
no more than $N_{\textnormal{LQR}}$ steps. The system will stay in
$\mathcal{R}_{\textnormal{LQR}}$ and will be taken to the origin by
$u_{\textnormal{LQR}}$ as $i\rightarrow\infty$. Consequently, the closed-loop
system (14) is locally asymptotically stable in
$\mathcal{R}_{\textnormal{MPC}}\cup\mathcal{R}_{\textnormal{NN}}$.

For every state $\mathbf{x}[0]\in\mathcal{R}_{\textnormal{MPC}}$,
$u_{\textnormal{MPC}}$ is invoked and the closed-loop system (14) will be
equivalent to (4) for at least one time step. In the next time step, if the
state enters
$\mathcal{R}_{\textnormal{LQR}}\cup\mathcal{R}_{\textnormal{NN}}$, then the
closed-loop system will converge to the origin as it is locally asymptotically
stable in $\mathcal{R}_{\textnormal{LQR}}\cup\mathcal{R}_{\textnormal{NN}}$;
otherwise, $\mathcal{R}_{\textnormal{MPC}}$ will be invoked again. Because the
closed-loop system (4) is locally asymptotically stable, even if MPC never
brings the state into $\mathcal{R}_{\textnormal{NN}}$, it will eventually
steer the system into $\mathcal{R}_{\textnormal{LQR}}$ in finite time, which
will then be taken to the origin by $u_{\textnormal{LQR}}$. Consequently, the
closed-loop system (14) is locally asymptotically stable in $\mathbb{X}_{0}$.

This completes the second part of the proof on stability.

To provide geometric intuition on how a MAMPC achieves local asymptotic
stability, we illustrate schematically in Figure 2a how a MAMPC could steer a
system from an initial state to the equilibrium at zero. A state in
$\mathcal{R}_{\textnormal{MPC}}$ is first steered by $u_{\textnormal{MPC}}$
into $\mathcal{R}_{\textnormal{NN}}$, which is then steered by
$u_{\textnormal{NN}}$ into $\mathcal{R}_{\textnormal{LQR}}$, after which is
steered to zero by $u_{\textnormal{LQR}}$. Note that the example demonstrated
in Figure 2a is only a hypothetical particular case of MAMPC closed-loop
behaviors.

Next, for alternating-authority MAMPC, the closed-loop system no longer has
guaranteed local asymptotic stability in $\mathbb{X}_{0}$, because it does not
check whether NN shoots inside $\mathcal{R}_{\textnormal{LQR}}$. Rather, we
can only show that the system is locally asymptotically stable in
$\mathcal{R}_{\textnormal{LQR}}$ and will never escape $\mathbb{X}_{0}$, as
stated the theorem below.

######  Theorem 5.3 (Stability of Alternating-Authority MAMPC).

For every MPC problem of the form (2), there always exists a
$u_{\textnormal{MAMPC}}^{\textnormal{AA}}$ of the form (12). Furthermore, the
closed-loop system

(15)
$\mathbf{x}[i+1]=f\left(\mathbf{x}[i],u_{\textnormal{MAMPC}}^{\textnormal{AA}}(\mathbf{x}[i])\right),\quad
i\in\mathbb{Z}_{\geq 0},$

with $\mathbf{x}[0]=\mathbf{x}_{0}\in\mathbb{X}_{0}$ is locally asymptotically
stable in $\mathcal{R}_{\textnormal{LQR}}$. Besides, for every initial
condition $\mathbf{x}[0]\in\mathbb{X}_{0}$, $\mathbf{x}[i]\in\mathbb{X}_{0}$
for all $i\in\mathbb{Z}_{+}$.

######  Proof 5.4.

The first part of the proof on existence is identical to the one in Theorem
5.1.

To prove stability of alternating-authority MAMPC, we partition
$\mathbb{X}_{0}$ into two disjoint regions as follows

$\mathbb{X}_{0}=\mathcal{R}_{\textnormal{LQR}}\cup\mathcal{R}_{\textnormal{AA}},$

where
$\mathcal{R}_{\textnormal{AA}}\coloneqq\mathbb{X}_{0}\setminus\mathcal{R}_{\textnormal{LQR}}$.

For every state $\mathbf{x}[0]\in\mathcal{R}_{\textnormal{LQR}}$, the closed-
loop system (15) is identical to closed-loop system (14), which is locally
asymptotically stable, as shown in Theorem (5.1).

Next, we prove by contradiction that the system will never escape
$\mathbb{X}_{0}$. Suppose there exits an escaping control $\bar{\mathbf{u}}$
such that $\bar{\mathbf{x}}\in\mathbb{X}_{0}$ but
$f(\bar{\mathbf{x}},\bar{\mathbf{u}})\notin\mathbb{X}_{0}$. This escaping
control $\bar{\mathbf{u}}$ must be produced by either $u_{\textnormal{MPC}}$,
$u_{\textnormal{NN}}$, or $u_{\textnormal{LQR}}$, that is,
$\bar{\mathbf{u}}=u_{\textnormal{MPC}}(\bar{\mathbf{x}})$,
$\bar{\mathbf{u}}=u_{\textnormal{NN}}(\bar{\mathbf{x}})$, or
$\bar{\mathbf{u}}=u_{\textnormal{LQR}}(\bar{\mathbf{x}})$. If
$\bar{\mathbf{u}}=u_{\textnormal{MPC}}(\bar{\mathbf{x}})$, then
$u_{\textnormal{MPC}}$ is not persistently feasible in $\mathbb{X}_{0}$, which
is not possible because we assume that $u_{\textnormal{MPC}}$ satisfies
Theorem (3.1). If $\bar{\mathbf{u}}=u_{\textnormal{NN}}(\bar{\mathbf{x}})$,
then the following switch condition must hold:
$f(\bar{\mathbf{x}},\bar{\mathbf{u}})\in\mathbb{X}\subseteq\mathbb{X}_{0}$.
However, this is not possible since $\bar{\mathbf{u}}$ is assumed to be an
escaping control, that is,
$f(\bar{\mathbf{x}},\bar{\mathbf{u}})\notin\mathbb{X}_{0}$. If
$\bar{\mathbf{u}}=u_{\textnormal{LQR}}(\bar{\mathbf{x}})$, then there must be
$\bar{\mathbf{x}}\in\mathcal{R}_{\textnormal{LQR}}$. Since
$\mathcal{R}_{\textnormal{LQR}}$ is positively invariant, we must have
$f(\bar{\mathbf{x}},\bar{\mathbf{u}})\in\mathcal{R}_{\textnormal{LQR}}\subset\mathbb{X}_{0}$,
which is not possible since
$f(\bar{\mathbf{x}},\bar{\mathbf{u}})\notin\mathbb{X}_{0}$. Therefore, the
system will never escape $\mathbb{X}_{0}$.

This completes the second part of the proof on stability.

An illustration of how the alternating-authority MAMPC could steer a system to
the equilibrium is provided in Figure 2b. Note that marginal stability may be
an understatement in practice: a well-trained NN, combined with a modest
alternation period $i_{d}$, could very well result in a closed-loop system
that is asymptotically stable.

If analytical guarantee of asymptotic stability is absolutely required, one
can replace the admissible control set $\mathbb{X}_{0}$ with a time-varying
set $\mathbb{S}[i]$, where

(16)
$\mathcal{R}_{\textnormal{LQR}}\subset\mathbb{S}[i+1]\subset\mathbb{S}[i]\subset\mathbb{X}_{0},\quad\forall
i=1,\dots,n-1,$

for some $n\in\mathbb{Z}_{>1}$ with $\mathbb{S}[0]=\mathbb{X}_{0}$ and
$\mathbb{S}[n]=\mathcal{R}_{\textnormal{LQR}}$. Geometrically, $\mathbb{S}$
can be viewed as a set that gets successively shrunk from $\mathbb{X}_{0}$
toward $\mathcal{R}_{\textnormal{LQR}}$. Such design will invoke the fail-safe
MPC if the systems is “stuck” outside of $\mathcal{R}_{\textnormal{LQR}}$.

Lastly, similar to alternating-authority MAMPC, the closed-loop system of a
way-point MAMPC is locally asymptotically stable in
$\mathcal{R}_{\textnormal{LQR}}$ and will eventually enter and stay within a
level set that escribes $\mathcal{D}_{\textnormal{WP}}$, as stated in the
following theorem.

######  Theorem 5.5 (Stability of Way-Point MAMPC).

For every MPC problem of the form (2), there always exists a
$u_{\textnormal{MAMPC}}^{\textnormal{WP}}$ of the form (13). Furthermore, the
closed-loop system

(17)
$\mathbf{x}[i+1]=f\left(\mathbf{x}[i],u_{\textnormal{MAMPC}}^{\textnormal{WP}}(\mathbf{x}[i])\right),\quad
i\in\mathbb{Z}_{\geq 0},$

with $\mathbf{x}[0]=\mathbf{x}_{0}\in\mathbb{X}_{0}$ is locally asymptotically
stable in $\mathcal{R}_{\textnormal{LQR}}$. Besides, there exists a time index
$j\in\mathbb{Z}_{\geq 0}$ such that for all $i\geq j$

$\mathbf{x}[i]\in\mathcal{D}_{\textnormal{WP}}^{+}\coloneqq\\{\mathbf{x}\in\mathbb{R}^{n}\mid
V(\mathbf{x})<\max_{q\in\partial{\mathcal{D}_{\textnormal{WP}}}}V(\mathbf{x})\\},$

where $V:\mathbb{R}^{n}\rightarrow\mathbb{R}_{+}$ is a Lyapunov function of
the closed-loop system (4) and $\partial{\mathcal{D}_{\textnormal{WP}}}$ is
the boundary of the compact set $\mathcal{D}_{\textnormal{WP}}$ which by
construction contains $\mathcal{R}_{\textnormal{LQR}}$.

######  Proof 5.6.

The first part of the proof on existence is identical to the one in Theorem
5.1.

To prove stability of way-point MAMPC, we partition $\mathbb{X}_{0}$ into five
disjoint regions as follows

$\mathbb{X}_{0}=\mathcal{R}_{\textnormal{LQR}}\cup\mathcal{R}_{\textnormal{NN}}^{i}\cup\mathcal{R}_{\textnormal{MPC}}^{i}\cup\mathcal{R}_{\textnormal{NN}}^{o}\cup\mathcal{R}_{\textnormal{MPC}}^{o},$

where $\mathcal{R}_{\textnormal{NN}}^{i}$ is the set of states in or on the
boundary of $\mathcal{D}_{\textnormal{WP}}$ where the NN is invoked;
$\mathcal{R}_{\textnormal{NN}}^{o}$ is the set of states in
$\mathbb{X}_{0}\setminus\mathcal{D}_{\textnormal{WP}}$ where the NN is
invoked;
$\mathcal{R}_{\textnormal{MPC}}^{i}\coloneqq\mathcal{D}_{\textnormal{WP}}\setminus(\mathcal{R}_{\textnormal{LQR}}\cup\mathcal{R}_{\textnormal{NN}}^{i})$;
and
$\mathcal{R}_{\textnormal{MPC}}^{o}\coloneqq(\mathbb{X}_{0}\setminus\mathcal{D}_{\textnormal{WP}})\setminus\mathcal{R}_{\textnormal{NN}}^{o}$.

For every state $\mathbf{x}\in\mathcal{R}_{\textnormal{LQR}}$, the closed-loop
system (17) is identical to closed-loop system (14), which is locally
asymptotically stable, as shown in Theorem (5.1).

Next, we prove that the system will eventually enter and stay within
$\mathcal{D}_{\textnormal{WP}}^{+}$.

For every state
$\mathbf{x}[0]\in\mathbb{X}_{0}\setminus\mathcal{D}_{\textnormal{WP}}$, either
NN or MPC will steer the system into $\mathcal{D}_{\textnormal{WP}}^{+}$ in
finite time. If NN is ever invoked, the system (17) will be taken into
$\mathcal{D}_{\textnormal{WP}}$ in no more than $N_{\textnormal{WP}}$ steps.
Otherwise, MPC will bring the system (17) into $\mathcal{D}_{\textnormal{WP}}$
in finite time since the MPC is locally asymptotically stable in
$\mathbb{X}_{0}$.

For every state $\mathbf{x}[0]\in\mathcal{D}_{\textnormal{WP}}$, we prove by
contradiction that there exists a $j\geq 0$ such that for all $i\geq
j,\mathbf{x}[i]\in\mathcal{D}_{\textnormal{WP}}^{+}$. Suppose there exists a
$\mathbf{x}[0]=\bar{\mathbf{x}}\in\mathcal{D}_{\textnormal{WP}}$ such that for
every $k\geq 0$, there exists an $i\geq k$ such that
$\mathbf{x}[i]\notin\mathcal{D}_{\textnormal{WP}}^{+}$. Without loss of
generality, define a control $\bar{\mathbf{u}}$ such that
$\mathbf{x}[i-1]\in\mathcal{D}_{\textnormal{WP}}^{+}$ but
$\mathbf{x}[i]=f(\mathbf{x}[i-1],\bar{\mathbf{u}})\notin\mathcal{D}_{\textnormal{WP}}^{+}$.

This escaping control $\bar{\mathbf{u}}$ must be produced by either
$u_{\textnormal{MPC}}$, $u_{\textnormal{NN}}$, or $u_{\textnormal{LQR}}$, that
is, $\bar{\mathbf{u}}=u_{\textnormal{MPC}}(\mathbf{x}[i-1])$,
$\bar{\mathbf{u}}=u_{\textnormal{NN}}(\mathbf{x}[i-1])$, or
$\bar{\mathbf{u}}=u_{\textnormal{LQR}}(\mathbf{x}[i-1])$. If
$\bar{\mathbf{u}}=u_{\textnormal{MPC}}(\mathbf{x}[i-1])$, then the close-loop
system (17) is equivalent to system (4). Because
$\mathbf{x}[i]\notin\mathcal{D}_{\textnormal{WP}}^{+}$, we have

$V(\mathbf{x}[i])=V(f(\mathbf{x}[i-1],u_{\textnormal{MPC}}(\mathbf{x}[i-1]))>V(\mathbf{x}[i-1]),$

but this is not possible because $V$ is a Lyapunov function of the system (4),
i.e.,

$V(f(\mathbf{x}[i-1],u_{\textnormal{MPC}}(\mathbf{x}[i-1]))\leq
V(\mathbf{x}[i-1]).$

If $\bar{\mathbf{u}}=u_{\textnormal{NN}}(\mathbf{x}[i-1])$, then
$f(\mathbf{x}[i-1],u_{\textnormal{NN}}(\mathbf{x}[i-1]))\notin\mathcal{D}_{\textnormal{WP}}$,
which is not possible because invocation of NN implies that
$f(\mathbf{x}[i-1],u_{\textnormal{NN}}(\mathbf{x}[i-1]))\in\mathcal{D}_{\textnormal{WP}}$.
If $\bar{\mathbf{u}}=u_{\textnormal{LQR}}(\mathbf{x}[i-1])$, then by
assumption $\lim_{i\rightarrow\infty}\mathbf{x}[i]$ does not exist. However,
this is impossible because $\lim_{i\rightarrow\infty}\mathbf{x}[i]=\mathbf{0}$
by the local asymptotic stability of the system (17) in
$\mathcal{R}_{\textnormal{LQR}}$. Consequently, the assumption must be false.

This completes the second part of the proof on stability.

An illustration of how the way-point MAMPC could steer a system to the
equilibrium is provided in Figure 2c. As in the alternating-authority MAMPC,
to always achieve asymptotic stability, one can successively shrink
$\mathcal{D}_{\textnormal{WP}}$ until
$\mathcal{D}_{\textnormal{WP}}=\mathcal{R}_{\textnormal{LQR}}$ in the same way
as presented in equation (16).

###  5.2 Remark on Robustness

In some cases, MAMPC may be robustified to account for model uncertainties. To
this end, we propose a robustification procedure to improve the robustness of
MAMPC using the idea of bounding error margin. We demonstrate the procedure by
applying it to the standard MAMPC.

Provided with the true system dynamics as $f$, we distinguish $\bar{f}$ as the
model of the dynamical system. Define the model uncertainties of a controller
$u$ with initial condition $\mathbf{x}[i]$ at time step $i+k$ as

$\Delta\mathbf{x}_{u}[k|i]\coloneqq\mathbf{y}[k]-\bar{\mathbf{y}}[k],$

where

$\displaystyle\mathbf{y}[0]$
$\displaystyle=\mathbf{x}[i],\quad\mathbf{y}[k]=f(\mathbf{y}[k-1],u(\mathbf{y}[k-1])),$
$\displaystyle\bar{\mathbf{y}}[0]$
$\displaystyle=\mathbf{x}[i],\quad\bar{\mathbf{y}}[k]=\bar{f}(\bar{\mathbf{y}}[k-1],u(\bar{\mathbf{y}}[k-1])).$

When $\|\Delta\mathbf{x}_{u}[k|i]\|=\alpha>0$, it is possible that
$\mathbf{y}[k]\notin\mathcal{R}_{\textnormal{LQR}}$ but
$\bar{\mathbf{y}}[k]\in\mathcal{R}_{\textnormal{LQR}}$, potentially leading to
a failure of MAMPC. To address this problem, we define a set operator
$\operatorname{Ero}_{\delta}$ as follows

$\operatorname{Ero}_{\delta}(\mathcal{X})\coloneqq\\{\mathbf{x}\in\mathcal{X}\mid\|\mathbf{x}-\mathbf{q}\|\geq\delta,\forall\mathbf{q}\in\partial\mathcal{X}\\}\textnormal{
for some }\delta>0,$

where $\mathcal{X}$ is an arbitrary set and $\partial\mathcal{X}$ denotes the
boundary of the set $\mathcal{X}$. We claim that for
$\|\Delta\mathbf{x}_{u}[k|i]\|=\alpha>0$, if
$\bar{\mathbf{y}}[k]\in\operatorname{Ero}_{\delta}(\mathcal{R}_{\textnormal{LQR}})$
with $\delta\geq\alpha$, then there must be
$\mathbf{y}[k]\in\mathcal{R}_{\textnormal{LQR}}$. Proof of the claim is a
direct application of the triangle inequality of norm. We can therefore
incorporate the set operator $\operatorname{Ero}_{\delta}$ to enhance the
robustness of MAMPC with respect to model uncertainties.

We can apply the above procedure to robustify the standard MAMPC as follows

$\displaystyle u_{\textnormal{MAMPC}}^{\delta}(\mathbf{x})\coloneqq$
$\displaystyle\begin{cases}u_{\textnormal{LQR}}(\mathbf{x}),&\mathbf{x}\in\mathcal{R}_{\textnormal{LQR}},\\\
\hline\cr&\mathbf{x}\in\mathbb{X}_{0}\setminus\mathcal{R}_{\textnormal{LQR}},\\\
u_{\textnormal{NN}}(\mathbf{x}),&\exists
i=1,\dots,N_{\textnormal{LQR}},\quad\bar{\mathbf{y}}[i]\in\mathcal{R}_{\textnormal{LQR}}^{\delta},\\\
&\forall
j=0,\dots,j,\quad(\bar{\mathbf{y}}[j],u_{\textnormal{NN}}(\bar{\mathbf{y}}[j]))\in\mathbb{A}^{\delta}\\\
\hline\cr u_{\textnormal{MPC}}(\mathbf{x}),&\text{otherwise},\end{cases}$

where
$\mathcal{R}_{\textnormal{LQR}}^{\delta}\coloneqq\operatorname{Ero}_{\delta}(\mathcal{R}_{\textnormal{LQR}})$
and
$\mathbb{A}^{\delta}\coloneqq\operatorname{Ero}_{\delta}(\mathbb{X})\times\mathbb{U}$,
for some $\delta>0$. The stability of the above robustified MAMPC is stated in
the following theorem.

######  Theorem 5.7 (Robustification of Standard MAMPC).

Suppose the model uncertainties are upper bounded by some $\alpha>0$, that is,

$\|\Delta\mathbf{x}_{u}[k|i]\|\leq\alpha,\quad\forall i\in\mathbb{Z}_{\geq
0},\forall k=0,\dots,N_{\textnormal{LQR}}.$

Then the following closed-loop system

$\mathbf{x}[i+1]=f(\mathbf{x}[i],u_{\textnormal{MAMPC}}^{\alpha}(\mathbf{x}[i])),\quad
i\in\mathbb{Z}_{\geq 0},$

with $\mathbf{x}[0]=\mathbf{x}_{0}\in\mathbb{X}_{0}$ is locally asymptotically
stable in $\mathbb{X}_{0}$.

######  Proof 5.8.

The proof is identical to that of Theorem 5.1. Note that it possible that
$\mathcal{R}_{\textnormal{LQR}}^{\delta}=\emptyset$. In this case, the hybrid
control is equivalent to a dual-mode MPC-LQR controller.

Similar to the above example, the robustification procedure can also be
applied to enhance the alternating-authority MAMPC and way-point MAMPC.

###  5.3 Remark on Necessity of Fail-Safe MPC

The hybrid control scheme can clearly be more efficient if we can skip the
forward verification and remove the fail-safe MPC mode. In light of this
observation, we describe a condition, where the fail-safe MPC mode will never
be invoked and thus can be safely removed from the hybrid control scheme.

Define the following “fail-free” MAMPC

$\displaystyle
u_{\textnormal{MAMPC}}^{+}(\mathbf{x})\coloneqq\begin{cases}u_{\textnormal{LQR}}(\mathbf{x}),&\mathbf{x}\in\mathcal{R}_{\textnormal{LQR}},\\\
u_{\textnormal{NN}}(\mathbf{x})&\mathbf{x}\in\mathbb{X}_{0}\setminus\mathcal{R}_{\textnormal{LQR}}.\end{cases}$

Define $L:\mathbb{X}\rightarrow\mathbb{R}_{+}$ as an augmented cumulative cost
function of NN, that is,

$L(\mathbf{x})\coloneqq\sum_{k=0}^{N-1}{c(\mathbf{y}[k],u_{\textnormal{NN}}(\mathbf{y}[k]))}+c_{f}(\mathbf{y}[N]),$

with

$\mathbf{y}[0]=\mathbf{x},\quad\mathbf{y}[k]=f(\mathbf{y}[k-1],u_{\textnormal{NN}}(\mathbf{y}[k-1])).$

From Theorem 13.1 in [4], we are ready to state the condition in the following
theorem.

######  Theorem 5.9 (Fail-Free MAMPC).

If there exists some nonnegative function
$\gamma:\mathbb{X}_{0}\rightarrow\mathbb{R}_{+}$ such that

(18) $\displaystyle(\mathbf{x},u_{\textnormal{NN}}(\mathbf{x}))\in\mathbb{A},$
$\displaystyle\forall\mathbf{x}\in\mathbb{X}_{0},$ $\displaystyle
L(\mathbf{x})\leq J^{*}(\mathbf{x})+\gamma(\mathbf{x}),$
$\displaystyle\forall\mathbf{x}\in\mathbb{X}_{0},$
$\displaystyle\gamma(\mathbf{x})-c(\mathbf{x},\mathbf{0})<0,$
$\displaystyle\forall\mathbf{x}\in\mathbb{X}_{0}\setminus\mathbf{0},$

then the closed-loop system

$\mathbf{x}[i+1]=f(\mathbf{x}[i],u_{\textnormal{MAMPC}}^{+}(\mathbf{x}[i])),\quad
i\in\mathbb{Z}_{\geq 0},$

with $\mathbf{x}[0]=\mathbf{x}_{0}\in\mathbb{X}_{0}$ is locally asymptotically
stable in $\mathbb{X}_{0}$ without violating any constraints in the original
MPC problem (2).

######  Proof 5.10.

The proof of the theorem is a direct application of Theorem 13.1 in [4].

Note that inequality conditions (18) are typically verified through sampling
and interpolation [4], which makes this approach only suitable for relatively
simple, moderately sized problems. Besides, the above condition is only a
sufficient condition, for which we can safely discard the fail-safe MPC mode.
Other methods are possible. For example, [26, 6] have proposed methods to
verify the stability of a NN controller using Lyapunov approaches.

Last but not least, we emphasize that the above analysis does not impose any
condition on the approximation error of the NN. In fact, so long as the NN has
the right input and output dimension, the corresponding MAMPC will be closed-
loop stable, even if the weights of the NN are random. In the worst case,
MAMPC reduces to a hybrid controller consisting of just the MPC and the LQR,
where the NN mode will never be invoked. This reduced hybrid controller is
identical to one proposed in [20].

Next, we evaluate the computational performance of the proposed method in
Section 6.

##  6 Numerical Experiments

To evaluate the running time performance of MAMPC, we conduct numerical
experiments on four models in robotics, i.e., pendulum, triple pendulum,
bicopter, and quadcopter. The models are selected for their significance in
robotic applications: pendulum models are the building blocks for robot arm
and leg control; copter models are an important model class in unmanned aerial
vehicle control. We highlight key parameters and results of the four
experiments in this section. For implementation details, please refer to the
source code [29].

###  6.1 Pendulum

The first model is a single arm pendulum as shown in Figure 3. The goal is to
maintain the pendulum at the position of the highest potential energy as
marked by the dashed line.

The state is defined as
$\mathbf{x}\coloneqq[\theta\quad\dot{\theta}]^{\top}\in[-\pi,\pi)\times\mathbb{R}$,
where $\theta$ is the angular displacement from the inverted position. The
input is a scalar $u\in\mathbb{R}$, which is the applied torque at the joint.
The equation of motion is described by the following nonlinear dynamics

(19) $\ddot{\theta}=\frac{3g}{2l}\sin{\theta}+\frac{3}{ml^{2}}u,$

where definitions and values of parameters are specified in Table 1. By
convention, we assign $\theta$ to be the clockwise angular displacement
between the rod and the dashed line.

Figure 3: Diagram of a pendulum model. A torque between $[-0.05,0.05]\hskip 5.0pt$\text{\,}\mathrm{N}\text{⋅}\mathrm{m}$$ is being applied to the hinge to keep the pendulum at its inverted position. Parameter | Description | Value
---|---|---
$m$ | Mass of the rod | $0.1\text{\,}\mathrm{kg}$
$l$ | Length of the rod | $0.1\text{\,}\mathrm{m}$
$g$ | Gravitational acceleration | $9.8\text{\,}\mathrm{m}\text{⋅}{\mathrm{s}}^{-2}$
Table 1: Specifications of the pendulum model.

The control objective is to steer the pendulum to the origin by applying a
limited torque $u\in[-0.05,0.05]$ in $\text{\,}\mathrm{N}\text{⋅}\mathrm{m}$
at the joint. The MPC design highlights a sampling interval of
$0.1\text{\,}\mathrm{s}$, a planning horizon $N=5$, and three optimization
variables per step, resulting in a linearly-constrained quadratic programming
(LCQP) of $0.5\text{\,}\mathrm{s}$ prediction horizon and 15 variables. Plant
nonlinearity is handled through linearizing around the equilibrium, leading to
a linear time-invariant system. We apply a standard MAMPC to control the
pendulum with

$\mathcal{R}_{\textnormal{LQR}}=\\{\mathbf{x}\in\mathbb{R}^{2}\mid\|\mathbf{x}\|_{2}\leq
0.5\\},\quad N_{\textnormal{LQR}}=5.$

The NN is a two-layer, 20-neuron multilayer perceptron (MLP) trained through
supervised learning with data randomly sampled from
$\theta\in[\pi,\pi],\dot{\theta}\in[-1,1]$.

To evaluate the computational efficiency of MAMPC, we compare its per-step
running time and total running time with implicit and explicit MPC. The system
is simulated with the same initial condition at $[\pi/2\quad 1/2]$ with the
three different controllers. The per-step running times of the three
controllers are shown in Figure 4 with the corresponding statistics tabulated
in the first column of Table 5. The total running times of the three
controllers are summarized in the first column of Table 6. In addition, An up
time analysis of the three control modes within the MAMPC is tabulated in the
first column of Table 7.

Figure 4: Pendulum per-step running time comparison. The pendulum evolves from
$\mathbf{x}=[\pi/2\quad 1/2]$ to the origin. Simulation stops when
$\|\mathbf{x}\|_{2}\leq 0.01$. The dashed line represents the running time of
NN mode without forward simulation check.

###  6.2 Triple Pendulum

The second model is a triple pendulum as shown in Figure 5, which extends the
above pendulum model to a more complex use case. The goal is to maintain the
triple pendulum at the position of the highest potential energy.

The state is defined as
$\mathbf{x}\coloneqq[\theta_{1}\quad\dot{\theta}_{1}\quad\theta_{2}\quad\dot{\theta}_{2}\quad\theta_{3}\quad\dot{\theta}_{3}]^{\top}\in([-\pi,\pi)\times\mathbb{R})^{3}$,
where $\theta_{1},\theta_{2},\theta_{3}$ are the angular displacements of the
three links respectively. The input is defined as
$\mathbf{u}\coloneqq[u_{1}\quad u_{2}\quad u_{3}]\in\mathbb{R}^{3}$, which is
the applied torques at the three joints respectively. The equation of motion
of the triple pendulum is obtained through symbolic computation using the
principle of Lagrangian mechanics:

(20) $\frac{d}{dt}\frac{\partial{L}}{\partial\dot{\theta}_{i}}-\frac{\partial
L}{\partial\theta_{i}}=u_{i},\quad\forall i=1,2,3,$

where $L\coloneqq T-V$ is the Lagrangian of the system, $T$ is the kinematic
energy, and $V$ is the potential energy.

The kinematic energy and potential energy of the triple pendulum are

$\displaystyle T$
$\displaystyle=0.5m_{1}\left(\dot{x}_{1}^{2}+\dot{y}_{1}^{2}\right)+0.5m_{2}\left(\dot{x}_{2}^{2}+\dot{y}_{2}^{2}\right)+0.5m_{3}\left(\dot{x}_{3}^{2}+\dot{y}_{3}^{2}\right),$
$\displaystyle V$ $\displaystyle=m_{1}gy_{1}+m_{2}gy_{2}+m_{3}gy_{3},$

where the coordinates of the three joints are calculated as follows

$\displaystyle x_{1}$ $\displaystyle=l_{1}\cos\left(\pi/2-\theta_{1}\right),$
$\displaystyle y_{1}$ $\displaystyle=l_{1}\sin\left(\pi/2-\theta_{1}\right),$
$\displaystyle x_{2}$
$\displaystyle=x_{1}+l_{2}\cos\left(\pi/2-\theta_{1}+\theta_{2}\right),$
$\displaystyle y_{2}$
$\displaystyle=y_{1}+l_{2}\sin\left(\pi/2-\theta_{1}+\theta_{2}\right),$
$\displaystyle x_{3}$
$\displaystyle=x_{2}+l_{3}\cos\left(\pi/2-\theta_{1}+\theta_{2}+\theta_{3}\right),$
$\displaystyle y_{3}$
$\displaystyle=y_{2}+l_{3}\sin\left(\pi/2-\theta_{1}+\theta_{2}+\theta_{3}\right),$

where definitions and values of parameters are specified in Table 2. For
simplicity, we assign $\theta_{1},\theta_{2},\theta_{3}$ to be the
counterclockwise angular displacements between the respective links and the
perpendicular directions of the corresponding dashed lines. Explicit form of
the system dynamics takes pages to write and is therefore omitted here.

Figure 5: Diagram of a triple pendulum model. Note that the centers of mass of the first and second torques coincide with the second and third joints. A torque between $[-1,1]\hskip 5.0pt$\text{\,}\mathrm{N}\text{⋅}\mathrm{m}$$ is being applied to each joint to keep the triple pendulum at its inverted position. Parameter | Description | Value
---|---|---
$m_{1}$ | Mass of link 1 | $0.1\text{\,}\mathrm{kg}$
$l_{1}$ | Length of link 1 | $0.1\text{\,}\mathrm{m}$
$m_{2}$ | Mass of link 2 | $0.1\text{\,}\mathrm{kg}$
$l_{2}$ | Length of link 2 | $0.1\text{\,}\mathrm{m}$
$m_{3}$ | Mass of link 3 | $0.1\text{\,}\mathrm{kg}$
$l_{3}$ | Length of link 3 | $0.1\text{\,}\mathrm{m}$
$g$ | Gravitational acceleration | $9.8\text{\,}\mathrm{m}\text{⋅}{\mathrm{s}}^{-2}$
Table 2: Specifications of the triple pendulum model.

The control objective is to steer the triple pendulum to the origin by
applying three limited torque inputs $u_{1},u_{2},u_{3}\in[-1,1]$ in
$\text{\,}\mathrm{N}\text{⋅}\mathrm{m}$ at the three joints, respectively. The
MPC design highlights a sampling interval of $0.1\text{\,}\mathrm{s}$, a
planning horizon $N=5$, and nine optimization variables per step, resulting in
a LCQP of $0.5\text{\,}\mathrm{s}$ prediction horizon and 45 variables.
Linearization is applied as before to handle nonlinearity. Because triple
pendulum is a chaotic system, we apply an alternating-authority MAMPC instead
of standard MAMPC with

$\mathcal{R}_{\textnormal{LQR}}=\\{\mathbf{x}\in\mathbb{R}^{6}\mid\|\mathbf{x}\|_{2}\leq
0.4\\},\quad N_{\textnormal{LQR}}=5,\quad i_{d}=2.$

The NN is a three-layer, 50-neuron MLP trained through supervised learning
with data randomly sampled from

$\theta_{1},\theta_{2},\theta_{3}\in[-\pi/6,\pi/6],\quad\dot{\theta_{1}},\dot{\theta_{2}},\dot{\theta_{3}}\in[-1,1].$

Evaluation of computational efficiency is also identical to the single
pendulum experiment. The system is simulated three times with alternating-
authority MAMPC, implicit MPC, and explicit MPC at
$[\pi/6\;1\;\pi/6\;1\;\pi/6\;1]$. The per-step running times of the three
controllers are shown in Figure 6 with the corresponding statistics tabulated
in the second column of Table 5. The total running times of the three
controllers are summarized in the second column of Table 6. An up time
analysis of the three control modes within the MAMPC is tabulated in the
second column of Table 7.

Figure 6: Triple pendulum per-step running time comparison. The triple
pendulum evolves from $\mathbf{x}=[\pi/6\quad 1\quad\pi/6\quad
1\quad\pi/6\quad 1]$ to the origin. Simulation stops when
$\|\mathbf{x}\|_{2}\leq 0.01$. The dashed line represents the running time of
NN mode without forward simulation check.

###  6.3 Bicopter

As a parallel robotic application to the pendulum models, the third model is a
bicopter as shown in Figure 7. The goal is to hover the bicopter in air.

The state is defined as $\mathbf{x}\coloneqq[x\quad\dot{x}\quad
y\quad\dot{y}\quad\theta\quad\dot{\theta}]^{\top}\in\mathbb{R}^{4}\times([-\pi,\pi)\times\mathbb{R})$,
where $x,y$ are horizontal and vertical translations and $\theta$ is the angle
of tilting. The input is defined as $\mathbf{u}\coloneqq[u_{1}\quad
u_{2}]^{\top}\in\mathbb{R}^{2}$, where $u_{1},u_{2}$ are the thrust exerted by
the left and right propellers, respectively. The equation of motion is
described in the following:

$\displaystyle\begin{bmatrix}\ddot{x}\\\ \ddot{y}\\\
\ddot{\theta}\end{bmatrix}=\begin{bmatrix}-\frac{1}{m}(u_{1}+u_{2})\sin\theta\\\
\frac{1}{m}(u_{1}+u_{2})\cos\theta-g\\\
\frac{1}{I}(u_{1}-u_{2})\end{bmatrix},$

where definitions and values of parameters are specified in Table 3.

Figure 7: Diagram of a bicopter model. Each of the two propellers is capable of generating a thrust between $0.1\text{\,}\mathrm{N}$ and $9.1572\text{\,}\mathrm{N}$ to keep the copter hover in air. Parameter | Description | Value
---|---|---
$m$ | Mass of the bicopter | $1.1\text{\,}\mathrm{kg}$
$l$ | Arm length of the bicopter | $0.21\text{\,}\mathrm{m}$
$I$ | Moment of inertia of the bicopter | $0.0196\text{\,}\mathrm{kg}\text{⋅}{\mathrm{m}}^{2}$
$g$ | Gravitational acceleration | $9.8\text{\,}\mathrm{m}\text{⋅}{\mathrm{s}}^{-2}$
Table 3: Specifications of the bicopter model.

The control objective is to steer the bicopter to the origin by applying
limited thrusts $u_{1},u_{2}\in[0.1,9.1572]$ in $\text{\,}\mathrm{N}$ at the
two propellers. The thrust limits are chosen such that hovering thrusts are
approximately in the center of the thrust limit. The MPC design highlights a
sampling interval of $0.1\text{\,}\mathrm{s}$, a planning horizon $N=20$, and
eight optimization variables per step, resulting in a LCQP of
$2.0\text{\,}\mathrm{s}$ prediction horizon and 160 variables. To handle plant
nonlinearity, we linearize the system around the equilibrium. We apply a
standard MAMPC to control the bicopter with

$\mathcal{R}_{\textnormal{LQR}}=\\{\mathbf{x}\in\mathbb{R}^{6}\mid\|\mathbf{x}\|_{2}\leq
0.5\\},\quad N_{\textnormal{LQR}}=10.$

The NN is a three-layer, 50-neuron MLP trained through supervised learning
with data randomly sampled from

$x,y,\theta\in[-\pi/2,\pi/2],\quad\dot{x},\dot{y},\dot{\theta}\in[-1,1].$

Evaluation of computational efficiency is identical as before. The system is
simulated with the same initial condition at $[\pi/2\quad 1\quad\pi/2\quad
1\quad\pi/2\quad 1]$ with standard MAMPC, implicit MPC, and explicit MPC. The
per-step running times of the three controllers are shown in Figure 8 with the
corresponding statistics tabulated in the third column of Table 5. The total
running times of the three controllers are summarized in the third column of
Table 6. An up time analysis of the three control modes within the MAMPC is
tabulated in the third column of Table 7.

Figure 8: Bicopter per-step running time comparison. The triple pendulum
evolves from $\mathbf{x}=[\pi/2\quad 1\quad\pi/2\quad 1\quad\pi/2\quad 1]$ to
the origin. Simulation stops when $\|\mathbf{x}\|_{2}\leq 0.01$. The dashed
line represents the running time of NN mode without forward simulation check.

###  6.4 Quadcopter

As an extension to the ideal bicopter model, the last model is a quadcopter as
shown in Figure 9 [1]. The goal is also to hover the copter in air but with
more realistic system dynamics.

The state is defined as
$\mathbf{x}\coloneqq[x\;\dot{x}\;y\;\dot{y}\;z\;\dot{z}\;\phi\;\dot{\phi}\;\theta\;\dot{\theta}\;\psi\;\dot{\psi}]^{\top}\in\mathbb{R}^{6}\times([-\pi,\pi)\times\mathbb{R})^{3}$,
where $x,y,z$ represent translations and $\phi,\theta,\psi$ represent
rotations corresponding to the three ZYX Euler angles roll, pitch, and yaw.
The input is $\mathbf{u}=[u_{1}\quad u_{2}\quad u_{3}\quad
u_{4}]^{\top}\in\mathbb{R}^{4}$, where $u_{i}$ is the rotational speeds of the
$i^{\text{th}}$ propeller for $i=1,2,3,4$. The equation of motion of the
quadcopter is developed by [1], as described below:

(21) $\begin{bmatrix}\ddot{x}\\\ \ddot{y}\\\ \ddot{z}\\\ \ddot{\phi}\\\
\ddot{\theta}\\\
\ddot{\psi}\end{bmatrix}=\begin{bmatrix}(\cos{\phi}\sin{\theta}\cos{\psi}+\sin{\phi}\sin{\psi})U_{1}/m\\\
(\cos{\phi}\sin{\theta}\sin{\psi}-\sin{\phi}\cos{\psi})U_{1}/m\\\
-g+\cos{\phi}\cos{\theta}U_{1}/m\\\
\dot{\theta}\dot{\psi}\frac{I_{yy}-I_{zz}}{I_{xx}}+\dot{\theta}\frac{I_{p}}{I_{xx}}U_{5}+\frac{l}{I_{xx}}U_{2}\\\
\dot{\phi}\dot{\psi}\frac{I_{zz}-I_{xx}}{I_{yy}}-\dot{\phi}\frac{I_{p}}{I_{yy}}U_{5}+\frac{l}{I_{yy}}U_{3}\\\
\dot{\phi}\dot{\theta}\frac{I_{xx}-I_{yy}}{I_{zz}}+\frac{1}{I_{zz}}U_{4}\end{bmatrix},\begin{bmatrix}U_{1}\\\
U_{2}\\\ U_{3}\\\ U_{4}\\\
U_{5}\end{bmatrix}=\begin{bmatrix}b(u^{2}_{1}+u^{2}_{2}+u^{2}_{3}+u^{2}_{4})\\\
b(u^{2}_{2}-u^{2}_{4})\\\ b(-u^{2}_{1}+u^{2}_{3})\\\
d(-u^{2}_{1}+u^{2}_{2}-u^{2}_{3}+u^{2}_{4})\\\
-u_{1}+u_{2}-u_{3}+u_{4}\end{bmatrix},$

where definitions and values of parameters are specified in Table 4.

Figure 9: Diagram of a quadcopter model. Each of the four propellers is capable of rotating at speed between $0\text{\,}\mathrm{rad}\text{⋅}{\mathrm{s}}^{-1}$ and $313.96\text{\,}\mathrm{rad}\text{⋅}{\mathrm{s}}^{-1}$ to keep the copter hover in air. Parameter | Description | Value
---|---|---
$m$ | Mass of the quadcopter | $1.1\text{\,}\mathrm{kg}$
$l$ | Arm length of the quadcopter | $0.21\text{\,}\mathrm{m}$
$I_{xx}$ | Moment of inertia about $x$ axis | $0.0196\text{\,}\mathrm{kg}\text{⋅}{\mathrm{m}}^{2}$
$I_{yy}$ | Moment of inertia about $y$ axis | $0.0196\text{\,}\mathrm{kg}\text{⋅}{\mathrm{m}}^{2}$
$I_{zz}$ | Moment of inertia about $z$ axis | $0.0264\text{\,}\mathrm{kg}\text{⋅}{\mathrm{m}}^{2}$
$I_{p}$ | Moment of inertia of a propeller | $8.5\text{\times}{10}^{-4}\text{\,}\mathrm{kg}\text{⋅}{\mathrm{m}}^{2}$
$b$ | Buoyancy coefficient | $9.29\text{\times}{10}^{-5}\text{\,}\mathrm{N}\text{⋅}{\mathrm{s}}^{2}$
$d$ | Drag coefficient | $1.1\text{\times}{10}^{-6}\text{\,}\mathrm{N}\text{⋅}\mathrm{m}\text{⋅}{\mathrm{s}}^{2}$
$g$ | Gravitational acceleration | $9.8\text{\,}\mathrm{m}\text{⋅}{\mathrm{s}}^{-2}$
Table 4: Specifications of the quadcopter model.

The control objective is to steer the quadcopter to the origin by applying
bounded rotations $u_{1},u_{2},u_{3},u_{4}\in[0,313.96]$ in
$\text{\,}\mathrm{rad}\text{⋅}{\mathrm{s}}^{-1}$ at the four propellers. The
rotor limits are chosen such that hovering rotor rotations are approximately
in the center of the rotor limit. The MPC design highlights a sampling
interval of $0.1\text{\,}\mathrm{s}$, a planning horizon $N=20$, and 16
optimization variables per step, resulting in a LCQP of
$2.0\text{\,}\mathrm{s}$ prediction horizon and 320 variables. Nonlinearity is
handled by linearization around the equilibrium. Because the closed-loop
quadcopter dynamics with MPC takes many steps to converge, we choose a way-
point MAMPC with

$\displaystyle\mathcal{D}_{\textnormal{WP}}=\\{\mathbf{x}\in\mathbb{R}^{12}\mid\|\mathbf{x}\|_{2}\leq
2\\},\quad N_{\textnormal{WP}}=10,$
$\displaystyle\mathcal{R}_{\textnormal{LQR}}=\\{\mathbf{x}\in\mathbb{R}^{12}\mid\|\mathbf{x}\|_{2}\leq
0.5\\},\quad N_{\textnormal{LQR}}=10.$

The NN is a four-layer, 60-neuron MLP trained through supervised learning with
data randomly sampled from

$\displaystyle x,y,z\in[-0.5,0.5],\quad\dot{x},\dot{y},\dot{z}\in[-0.1,0.1],$
$\displaystyle\phi,\theta\in[-\pi/6,\pi/6],\psi\in[-\pi/4,\pi/4],\dot{\phi},\dot{\theta},\dot{\psi}\in[-0.1,0.1].$

Evaluation of computational efficiency is identical as before. The system is
simulated with the same initial condition at
$[0.5\;0.1\;0.5\;0.1\;0.5\;0.1\;\pi/6\;0.1\;\pi/6\;0.1\;\pi/4\;0.1]$ with way-
point MAMPC, implicit MPC, and explicit MPC. The per-step running times of the
three controllers are shown in Figure 10 with the corresponding statistics
tabulated in the last column of Table 5. The total running times of the three
controllers are summarized in the last column of Table 6. An up time analysis
of the three control modes within the MAMPC is tabulated in the last column of
Table 7.

Figure 10: Quadcopter per-step running time comparison. The triple pendulum
evolves from
$\mathbf{x}=[0.5\;0.1\;0.5\;0.1\;0.5\;0.1\;\pi/6\;0.1\;\pi/6\;0.1\;\pi/4\;0.1]$
to the origin. Simulation stops when $\|\mathbf{x}\|_{2}\leq 0.01$. The dashed
line represents the running time of NN mode without forward simulation check.

Note that this experiment is already at the limit of explicit MPC method. The
explicit MPC offline caching takes over 10 hours to complete with the
resulting explicit control law occupies around 100 MB of hard disk storage. In
comparison, NN supervised learning takes less than one hour and only requires
less than 20 KB of storage. As explicit MPC method scales exponentially with
state dimension, it will not likely work well for problems of higher
dimensions.

| Pendulum ($\text{\,}\mathrm{\SIUnitSymbolMicro s}$) | Triple Pend. ($\text{\,}\mathrm{\SIUnitSymbolMicro s}$) | Bicopter ($\text{\,}\mathrm{\SIUnitSymbolMicro s}$) | Quadcopter ($\text{\,}\mathrm{\SIUnitSymbolMicro s}$)
---|---|---|---|---
Imp. MPC | $438\pm 252$ | $555\pm 366$ | $436\pm 221$ | $474\pm 184$
Exp. MPC | $257\pm 5.38$ | $263\pm 3.07$ | $245\pm 3.25$ | $246\pm 2.01$
MAMPC-MPC | $1110\pm 397$ | $597\pm 406$ | $1300\pm 615$ | $1070\pm 404$
MAMPC-NN | $82.3\pm 66.8$ | $158\pm 95.4$ | $252\pm 136$ | $247\pm 126$
MAMPC-LQR | $1.60\pm 0.259$ | $2.31\pm 0.480$ | $1.79\pm 0.198$ | $1.68\pm 0.102$
NN | $32.2\pm 0.445$ | $35.8\pm 0.500$ | $41.7\pm 11.7$ | $37.3\pm 0.771$

Table 5: Summary of per-step running time in
$\text{\,}\mathrm{\SIUnitSymbolMicro s}$. MAMPC-MPC, MAMPC-NN, MAMPC-LQR refer
to the MPC mode, NN mode, and LQR mode of the MAMPC respectively. NN is the NN
mode of MAMPC without forward simulation. The numerical results indicate that
MAMPC has better amortized running time if the system is mostly running in the
NN and LQR mode. However, the trade-off is that MAMPC has worse worst-case
latency.

| Pendulum ($\text{\,}\mathrm{ms}$) | Triple Pend. ($\text{\,}\mathrm{ms}$) | Bicopter ($\text{\,}\mathrm{ms}$) | Quadcopter ($\text{\,}\mathrm{ms}$)
---|---|---|---|---
Imp. MPC. | $12.7\text{\,}$ | $5.55\text{\,}$ | $17.0\text{\,}$ | $52.6\text{\,}$
Exp. MPC | $7.46\text{\,}$ | $2.63\text{\,}$ | $9.57\text{\,}$ | $27.3\text{\,}$
MAMPC | $7.18\text{\,}$ | $3.78\text{\,}$ | $11.4\text{\,}$ | $42.1\text{\,}$

Table 6: Summary of total running time in $\text{\,}\mathrm{ms}$. The implicit
MPC method is slower than the explicit MPC and MAMPC method. MAMPC method is
mostly on par with the explicit MPC, except in the case of quadcopter.

| Pendulum (%) | Triple Pend. (%) | Bicopter (%) | Quadcopter (%)
---|---|---|---|---
MPC Time Div. | 91.831 | 78.812 | 79.762 | 89.270
MPC Step Div. | 24.000 | 33.333 | 17.949 | 37.634
NN Time Div. | 6.879 | 20.883 | 19.877 | 10.570
NN Step Div. | 24.000 | 33.333 | 23.077 | 19.355
LQR Time Div. | 0.290 | 0.305 | 0.361 | 0.160
LQR Step Div. | 52.000 | 33.333 | 58.974 | 43.011

Table 7: Summary of division of up time in percentage. Even though most of the
compute steps the controller is running in the NN or LQR mode, it spends most
of it computational time in the MPC mode.

###  6.5 Timing Method

The numerical experiments are conducted on a macOS with a 3.2 GHz 6-Core Intel
Core i7 processor. All codes are implemented and timed using MATLAB. The
running time data in Figure 4 to Figure 10 are measured by repeating and
timing the same experiment 50 times and then taking the median of the 50
measurements. Due to initial loading and logging performed by MATLAB MPC
package, the first control step of each experiment is very slow and thus
omitted from data. The per-step running times are obtained by taking the means
and one standard deviations of relevant data along a trajectory. The total
running times are obtained by summing the running time of every data point
along a trajectory. Up time division is obtained by dividing the running time
and the number of control steps each mode of MAMPC takes by the total amount.

We remark that the explicit MPC solver in MATLAB is likely not very optimized.
There should be room for improvement at least for the simpler systems like
pendulum and bicopter. Nevertheless, we decide to benchmark in MATLAB as it is
one of the standard software that the community use for robotic control. We
believe that although the exact values of running time in the Section may vary
with a different setup of hardware and software, the comparative performance
of each control methods should approximately remain the same.

With the numerical experiments presented above, we continue to provide a few
key interpretations of the experimental results and controller design in
Section 7.

##  7 Discussions

The numerical experiments above indicates that MAMPC may have superior
amortized running time but inferior worst-case running time in common robotic
applications. As data shows, the LQR mode and NN mode of MAMPC is faster than
implicit and explicit MPC methods. Moreover, we will be able to further reduce
the running time of the NN mode by approximately an order of magnitude, if we
can safely skip the forward verification and apply the NN directly. However,
the MPC mode of MAMPC is slower than the implicit and explicit methods due to
overhead from checking switch conditions, leading to increased latency in the
worst-case scenario. Despite of the drawback, experiments show that the MPC
mode is seldom invoked, which enables MAMPC to have smaller average latency
than the implicit and explicit methods.

Beyond the direct implications of the numerical experiments, we make following
remarks on set design, forward verification, additional running time
optimizations, and limitations.

###  7.1 Set Design

Identification and design of feasibility set $\mathbb{X}_{0}$ and attraction
set $\mathcal{R}_{\textnormal{LQR}}$ is very challenging. One one hand, it is
easier to just approximate them by conservative set estimates for robustness
measures. For example, one can replace $\mathbb{X}_{0}$ and
$\mathcal{R}_{\textnormal{LQR}}$ with $\mathbb{X}_{0}^{-}$ and
$\mathcal{R}_{\textnormal{LQR}}^{-}$, where
$\mathbb{X}_{0}^{-}\subset\mathbb{X}_{0}$ and
$\mathcal{R}_{\textnormal{LQR}}^{-}\subset\mathcal{R}_{\textnormal{LQR}}$. On
the other hand, however, to guarantee computational efficiency, one should
make sets $\mathcal{R}_{\textnormal{LQR}},\mathcal{R}_{\textnormal{NN}}$ as
large as possible. This way, faster modes of MAMPC get invoked more frequently
than the slow, default mode of MAMPC, leading to a more efficient amortized
running time performance. Conservative set design makes the system more robust
to model uncertainties but also slower in running time. Balancing the trade-
off between latency and robustness is therefore a design process.

###  7.2 Forward verification

The horizon used in forward verification immediately affects the maximum
possible running time of the MAMPC. A valid verification horizon should not
make the running time of the MPC mode longer than the maximum allowable
computational latency. Moreover, an additional rule of thumb is to choose a
verification horizon such that the maximum possible running time of NN mode is
on par with the running time of the implicit MPC.

The simulation method used for forward verification also directly impacts the
worst-case latency of the MAMPC. For continuous plants, we recommend to use
forward Euler method with a sampling time that is as large as possible and to
use a conservative set design to absorb integration errors. Like designing
sets, choosing the right sampling time is a balancing act between latency and
robustness.

###  7.3 Additional Optimizations

For small to moderately sized problems, we may consider developing a fail-free
MAMPC as specified in Theorem 5.9, since it will further speed up computation.
Fail-free MAMPC prevents controller from ever having to default to fail-safe
mode and therefore skips the routine computation of forward verification in NN
mode. It significantly improves the amortized and worst-case running time.

If the computational device is capable of executing independent threads in
parallel, it may be advantageous to distribute computation across multiple
threads to further reduce latency. For example, a possible parallelization
strategy for standard MAMPC is as follows: 1) compute the three control modes
of MAMPC in three parallel threads, 2) among those that pass their
corresponding switch conditions, execute whichever that finishes first, and 3)
reset threads for next control step and repeat. This way, the worst-case
running time will be roughly identical to that of the original MPC. As a
result, the parallelized MAMPC scheme is at worst equivalent to the hybrid
strategy in [20].

###  7.4 Limitations

The foremost limitation of the MAMPC method is that without removing the fail-
safe controller it can never reduce the worst-case running time to be below
that of the original MPC. This shortcoming limits the applicability of the
hyrbid control scheme. For instance, if the original MPC breaks closed-loop
stability due to exceedingly high latency, the MAMPC counterpart will also
fail to make the closed-loop system stable. In the worst-case scenario, the
MAMPC controller is at best equivalent to the original MPC controller.
Therefore, MAMPC is only suited for applications where the existing MPC
controller works but users demand additional improvement in amortized running
time. Examples of such applications include routine repetitive manipulator
tasks, where one can acquire proficiency through practice, such as pick-and-
place and item assembly.

Another limitation of MAMPC is that the MPC designs presented in the four
numercial experiments are neither optimal nor certified. For example, instead
of linearization around the equilibrium, nonlinearity may be handled directly
by a nonlinear solver. Also, no verification has been done to check that the
MPC controllers obtained through plant linearization are persistent feasible
and closed-loop stable with respect to the original nonlinear plants. Hence,
the controllers in the numerical experiments are not solutions for deployment.
Instead, they are academic prototypes designed specifically to investigate the
computational aspects of the proposed method.

As the final remark, we conclude our work in Section 8.

##  8 Conclusion

To improve average computational efficiency of MPC, we have developed a
triple-mode hybrid control named MAMPC. Stability of MAMPC is guaranteed via
forward simulation, while efficiency is achieved by replacing MPC with a more
efficient NN or LQR, whenever stability permits. Results indicate that MAMPC
often has a better amortized running time but a slightly prolonged worst-case
running time.

As future work, one may generalize the hybrid control pattern in MAMPC to
other controllers beyond MPC. The hybrid control pattern consists of three
basic ingredients: an efficient equilibrium policy, a frugal heuristic policy,
and an expensive default policy. When the state is close to its equilibrium,
we may effectively approximate the system dynamics with a linear system and
control it with a linear controller. Developing such linear controller can be
achieved through LQR or other linear control methods in either frequency or
time domain. Away from the equilibrium, first-order approximation will no
longer be effective, so a new controller is required to maintain system
stability. In this case, one can attempt to leverage machine learning methods,
such as supervised learning or reinforcement learning, to train a heuristic
controller that is empirically effective and efficient. Lastly, when the
heuristic policy is expected to fail, we should invoke a default policy to
maintain the system in control, or at least to prevent the system from
catastrophic failures. In this work, the default policy is MPC but this can be
other nonlinear controllers too.

## Acknowledgments

This material is based upon work supported by the National Science Foundation
under Grant Numbers CNS-1837244. Any opinions, findings, and conclusions or
recommendations expressed in this material are those of the author(s) and do
not necessarily reflect the views of the National Science Foundation. This
material is based upon work supported by the U.S. Department of Energy’s
Office of Energy Efficiency and Renewable Energy (EERE) award number CID DE-
EE0008872. The views expressed herein do not necessarily represent the views
of the U.S. Department of Energy or the United States Government.

## References

  * [1] K. Alexis, G. Nikolakopoulos, and A. Tzes, Model predictive quadrotor control: attitude, altitude and position experimental studies, IET Control Theory and Applications, 6 (2012), pp. 1812–1827.
  * [2] B. D. O. Anderson and J. B. Moore, Optimal Control: Linear Quadratic Methods, Dover Publications, 2018, pp. 53–54.
  * [3] G. Bledt, M. J. Powell, B. Katz, J. Di Carlo, P. M. Wensing, and S. Kim, MIT Cheetah 3: Design and Control of a Robust, Dynamic Quadruped Robot, in 2018 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS), IEEE, 2018, pp. 2245–2252.
  * [4] F. Borrelli, A. Bemporad, and M. Morari, Predictive Control for Linear and Hybrid Systems, Cambridge University Press, 2017.
  * [5] F. Borrelli, P. Falcone, T. Keviczky, J. Asgari, and D. Hrovat, MPC-based approach to active steering for autonomous vehicle systems, International Journal of Vehicle Autonomous Systems, 3 (2005), pp. 265–291.
  * [6] S. Chen, M. Fazlyab, M. Morari, G. J. Pappas, and V. M. Preciado, Learning Lyapunov functions for hybrid systems, in Proceedings of the 24th International Conference on Hybrid Systems: Computation and Control, 2021, pp. 1–11.
  * [7] S. W. Chen, T. Wang, N. Atanasov, V. Kumar, and M. Morari, Large Scale Model Predictive Control with Neural Networks and Primal Active Sets, Automatica, 135 (2022), p. 109947.
  * [8] M. Diehl, H. G. Bock, and J. P. Schlöder, A Real-Time Iteration Scheme for Nonlinear Optimization in Optimal Feedback Control, SIAM Journal on Control and Optimization, 43 (2005), pp. 1714–1736.
  * [9] T. Faulwasser, T. Weber, P. Zometa, and R. Findeisen, Implementation of Nonlinear Model Predictive Path-Following Control for an Industrial Robot, IEEE Transactions on Control Systems Technology, 25 (2016), pp. 1505–1511.
  * [10] B. Houska, H. J. Ferreau, and M. Diehl, An Auto-Generated Real-Time Iteration Algorithm for Nonlinear MPC in the Microsecond Range, Automatica, 47 (2011), pp. 2279–2285.
  * [11] B. Karg, T. Alamo, and S. Lucia, Probabilistic performance validation of deep learning-based robust nmpc controllers, International Journal of Robust and Nonlinear Control, (2021).
  * [12] B. Karg and S. Lucia, Stability and feasibility of neural network-based controllers via output range analysis, in IEEE Conference on Decision and Control, IEEE, 2020, pp. 4947–4954.
  * [13] H. K. Khalil, Nonlinear Systems, Prentice Hall, 3rd ed., 2002.
  * [14] D. P. Kingma and J. Ba, Adam: A Method for Stochastic Optimization, in International Conference on Learning Representations, 2015\.
  * [15] S. Kleff, A. Meduri, R. Budhiraja, N. Mansard, and L. Righetti, High-Frequency Nonlinear Model Predictive Control of a Manipulator, in IEEE International Conference on Robotics and Automation, 2021\.
  * [16] S. Kuindersma, R. Deits, M. Fallon, A. Valenzuela, H. Dai, F. Permenter, T. Koolen, P. Marion, and R. Tedrake, Optimization-based locomotion planning, estimation, and control design for the atlas humanoid robot, Autonomous robots, 40 (2016), pp. 429–455.
  * [17] M. Kvasnica, J. Hledík, I. Rauová, and M. Fikar, Complexity reduction of explicit model predictive control via separation, Automatica, 49 (2013), pp. 1776–1781.
  * [18] C. Mastalli, R. Budhiraja, W. Merkt, G. Saurel, B. Hammoud, M. Naveau, J. Carpentier, L. Righetti, S. Vijayakumar, and N. Mansard, Crocoddyl: An Efficient and Versatile Framework for Multi-Contact Optimal Control, in IEEE International Conference on Robotics and Automation, IEEE, 2020, pp. 2536–2542.
  * [19] J. Mattingley and S. Boyd, CVXGEN: a code generator for embedded convex optimization, Optimization and Engineering, 13 (2012), pp. 1–27.
  * [20] H. Michalska and D. Q. Mayne, Robust Receding Horizon Control of Constrained Nonlinear Systems, IEEE transactions on automatic control, 38 (1993), pp. 1623–1633.
  * [21] J. Nubert, J. Köhler, V. Berenz, F. Allgöwer, and S. Trimpe, Safe and Fast Tracking on a Robot Manipulator: Robust MPC and Neural Network Control, IEEE Robotics and Automation Letters, 5 (2020), pp. 3050–3057.
  * [22] T. Parisini, M. Sanguineti, and R. Zoppoli, Nonlinear Stabilization by Receding-Horizon Neural Regulators, International Journal of Control, 70 (1998), pp. 341–362.
  * [23] T. Parisini and R. Zoppoli, A Receding-horizon Regulator for Nonlinear Systems and a Neural Approximation, Automatica, 31 (1995), pp. 1443–1451.
  * [24] J. A. Paulson and A. Mesbah, Approximate Closed-Loop Robust Model Predictive Control with Guaranteed Stability and Constraint Satisfaction, IEEE Control Systems Letters, 4 (2020), pp. 719–724.
  * [25] J. B. Rawlings, D. Q. Mayne, and M. M. Diehl, Model Predictive Control: Theory, Computation and Design, Nob Hill Publishing, 2017.
  * [26] S. M. Richards, F. Berkenkamp, and A. Krause, The Lyapunov Neural Network: Adaptive Stability Certification for Safe Learning of Dynamical Systems, in Conference on Robot Learning, PMLR, 2018, pp. 466–476.
  * [27] S. Summers, C. N. Jones, J. Lygeros, and M. Morari, A Multiresolution Approximation Method for Fast Explicit Model Predictive Control, IEEE Transactions on Automatic Control, 56 (2011), pp. 2530–2541.
  * [28] R. S. Sutton and A. G. Barto, Reinforcement Learning: An Introduction, MIT press, 2018.
  * [29] F. Wu, Memory-Augmented Model Predictive Control: Numerical Experiments.  https://github.com/fywu85/mampc, 2021.
  * [30] X. Zhang, M. Bujarbaruah, and F. Borrelli, Safe and Near-Optimal Policy Learning for Model Predictive Control using Primal-Dual Neural Networks, in 2019 American Control Conference, IEEE, 2019, pp. 354–359.
  * [31] R. Zoppoli, M. Sanguineti, G. Gnecco, and T. Parisini, Neural Approximations for Optimal Control and Decision, Springer, 2020.