# Oracle-based economic predictive control

José María Manzano111 Corresponding author<EMAIL_ADDRESS>David Muñoz de
la Peña2<EMAIL_ADDRESS>Daniel Limon2<EMAIL_ADDRESS>1 Departamento de Ingeniería,
Universidad Loyola Andalucía
Avda. de las Universidades s/n 41071 Dos Hermanas, Sevilla, Spain
2Departamento de Ingeniería de Sistemas y Automática, Universidad de Sevilla
Camino de los Descubrimientos s/n 41092 Sevilla, Spain

###### Abstract

This paper presents an economic model predictive controller, under the
assumption that the only measurable signal of the plant is the economic cost
to be minimized. In order to forecast the evolution of this economic cost for
a given input trajectory, a prediction model with a NARX structure, the so-
called _oracle_ , is proposed. Sufficient conditions to ensure the existence
of such oracle are studied, proving that it can be derived for a general
nonlinear system if the economic cost function is a Morse function. Based on
this oracle, economic model predictive controllers are proposed, and their
stability is demonstrated in nominal conditions under a standard dissipativity
assumption. The viability of these controllers in practical settings (where
the oracle may provide imperfect predictions for generic inputs) is proven by
means of input-to-state stability. These properties have been illustrated in a
case study based on a continuously stirred tank reactor.

††journal: Computers & Chemical Engineering

##  1 Introduction

Often, control systems have to simultaneously consider both performance and
safety requirements. This double objective has been typically addressed by
means of a hierarchical structure, where a real time optimization layer
calculates the equilibrium point that minimizes the operation cost, while a
lower control layer regulates the system to this equilibrium point. Recently,
in the model predictive control (MPC) framework, this hierarchical control
structure has been united in a single layer [1], aimed to minimize the
operation cost during the transient, instead of a tracking cost, often
designed to provide robustness and stability properties. This is the so-called
economic MPC, whose properties have been studied in several works [2, 3]. The
main difference between economic MPC and regulation MPC is that the former
relaxes the architecture of the optimization problem, in order to minimize any
_economic_ cost function, which may not be positive definite. Two recent
overviews on EMPC can be found in [4, 5]. Further, extensions to robust and
multistage or output-feedback EMPC can be found in [6, 7, 8], among others.

Predictive controllers are based on the availability of a model of the plant,
in order to predict the evolution of the states of the system, and based on
these predictions, the cost to be minimized is calculated. Recently, data-
based MPCs have been proposed to address applications in which an appropriate
model of the plant is not available, and hence the predictions have to be
obtained from historical data sets, as reviewed in [9]. Accordingly, this
data-based approach is also being applied to economic MPCs. For example, in
[10], a Lyapunov-based economic MPC (LB-EMPC) that integrates a linear
prediction model is presented, updated online from the measurements of the
plant. In [11], a LB-EMPC for nonlinear systems aimed to maintain excitation
on the system in order to obtain an state-space model from the measured inputs
was presented. In [12], the authors propose a recurrent neural network to
learn the model of the plant that is controlled by an LB-EMPC, and they extend
this approach to take into account constraints by means of barrier functions
in [13]. In [14], a data-driven EMPC based on reinforcement learning in which
the EMPC is used as approximator of the value function of the reinforced
learning policy is proposed.

These data-based approaches are based on both inputs and outputs historical
data sets. However, there may exist situations in which no measurements of any
inner variable of the plant are available, for example, to maintain privacy of
operation, or due to security reasons. Consider for instance a data center in
which the operation cost accounts for the cost of the electric consumption of
the refrigeration system and the consumption of the servers. In order to
design a controller to optimize the operation cost, sharing inner information
of the state of the servers could be limited due to security reasons, while
sharing only the operation cost may not jeopardize the security of the system.

In this paper, we study the case in which the only available measurement from
the plant is the value of the economic cost to be optimized. The prediction of
the behaviour of the plant is carried out by an oracle that forecasts the
economic performance of the plant, from a historical data set of the tuple
inputs-economic costs, using a nonlinear autoregressive exogenous model (NARX)
structure, which is obtained using nonlinear identification or machine
learning methods [15]. The existence of this class of oracles for the
prediction of the economic cost is studied, proving that they can be derived
for general nonlinear plants under mild assumptions on the economic cost
function. Based on this oracle, several economic model predictive controllers
are proposed, proving that in the case of exact predictions, they inherit the
properties of the economic MPC based on the process model. In addition, in the
practical case of an imperfect oracle, it is proven that the oracle-based
economic MPC with terminal cost function is input-to-state stable with respect
to the estimation error of the oracle.

To the best of the authors’ knowledge, this is the first work in which an
economic MPC based only on the measure of the economic cost is studied,
proving that for a general nonlinear system and under mild assumptions on the
economic cost function, it suffices to measure the economic cost function to
design a stabilizing economic predictive controller using a data-based oracle.
The proposed controller has been applied in simulation to the economic
operation of a continuously stirred tank reactor, using kinky inference
processes [16] to learn the oracle. A preliminary version of this work was
presented in [17].

The rest of the paper is structured as follows: Section 2 presents the problem
formulation and the standard economic MPC. Section 3 states the conditions
under which it is possible to define an oracle to predict the future evolution
of the economic cost. Section 4 describes the proposed oracle-based economic
predictive controllers and Section 5 addresses practical aspects of the
problem. Finally, Section 6 presents the case study.

### Notation

Given two column vectors $v$ and $w$, $(v,w)$ stands for $[v^{T},w^{T}]^{T}$.
The set $\mathbb{I}_{a}^{b}$ stands for the set of integers from $a$ to $b$. A
function $\alpha:\mathbb{R}_{\geq 0}\rightarrow\mathbb{R}_{\geq 0}$ is a
$\mathcal{K}$-function if it is strictly increasing and $\alpha(0)=0$.
Besides, if a $\mathcal{K}$-function is such that
$\lim\limits_{s\rightarrow\infty}\alpha(s)=\infty$, then it is called a
$\mathcal{K}_{\infty}$-function. Given a set
$\mathcal{X}\subseteq{\mathbb{R}}^{n}$, $\mathcal{X}^{M}$ denotes the
cartesian product of the set $M$ times, i.e.
$\mathcal{X}^{M}=\mathcal{X}^{M-1}\times\mathcal{X}$ with
$\mathcal{X}^{1}=\mathcal{X}$.

##  2 Problem formulation

In this paper, we consider that the system to be controlled is a sampled
continuous-time system described by an unknown discrete time model

$\displaystyle x(k+1)$ $\displaystyle=$ $\displaystyle f(x(k),u(k)),$ (1)

where $x(k)\in{\mathbb{R}}^{n}$ is the state of the plant and
$u(k)\in{\mathbb{R}}^{m}$ is the control input. It is assumed that the inputs
are subject to (hard) constraints $u(k)\in\mathcal{U}$, where
$\mathcal{U}\subset{\mathbb{R}}^{m}$ is a compact set.

The objective of the control strategy to be designed is to guarantee that the
closed-loop system is stable, while a cost function is minimized during the
transient. This cost function is said to be economic because it measures the
performance of the evolution of the system according to a generic function,
that does not necessarily penalize the tracking error w.r.t. a given target.

The economic cost function to be considered in this paper is defined by a
function of the form $\ell(x,u)$. The model $f$ and the economic cost function
$\ell$ must satisfy the following condition:

######  Assumption 1.

The function $f(x,u)$ is smooth and state invertible, i.e., for a given $u$,
$f$ defines a diffeomorphism in $x$. The function
$\ell(\cdot,\cdot):{\mathbb{R}}^{n}\times\mathcal{U}\mapsto\mathcal{L}$ is
smooth and its image $\mathcal{L}\subset{\mathbb{R}}$ is a compact set.

Note that as stated in [18], in the general case in which the model function
(1) is derived from sampling a continuous-time system controlled using a zero-
order holder, such that it is described by a finite-dimensional differential
equation with an unique solution, the resulting model function $f$ is state
invertible. Moreover, since the value of the cost function $\ell(x,u)$ is
assumed to be measured, the assumption that its image is bounded is not
limiting.

######  Remark 1 (Soft constraints).

There may exist a collection of variables of the system
$y_{c}(k)\in{\mathbb{R}}^{n_{y}}$ which are subject to (soft) constraints
$y_{c}(k)\in\mathcal{Y}_{c}$, being $\mathcal{Y}_{c}$ a closed set. To cope
with this case, the stage cost function $\ell(x,u)$ can be used to take into
account these constraints by adding a term that penalizes their violation.
This term can be thought of as the economic cost of not fulfilling them.

###  2.1 Stabilizing economic MPC

According to the given economic cost function, the optimal equilibrium point
is obtained from the solution of the following optimization problem:

$\displaystyle(x_{s},u_{s})$ $\displaystyle=$
$\displaystyle\arg\min_{x,u\in\mathcal{U},}\ell(x,u)$ (2a) s.t. $\displaystyle
x=f(x,u).$ (2b)

The economic optimal operation of a system by means of a model predictive
control law is a very complex problem that has been thoroughly studied
recently. See for instance the excellent survey papers [2, 5] and the
references therein. For the asymptotic stabilization of economic optimal
controllers, the dissipativity property plays an important role. This property
has been related to the turnpike property, see [19] for their relation within
continuous-time models in optimal control, and [20, 21] for discrete-time
models. In this work, this condition is stated in the following assumption:

######  Assumption 2.

The system $f$ is strictly dissipative with respect to the supply rate
$s(x,u)=\ell(x,u)-\ell(x_{s},u_{s})$, i.e. there exists a storage function
$\lambda:{\mathbb{R}}^{n}\rightarrow{\mathbb{R}}$ such that

$\displaystyle\lambda(f(x,u))-\lambda(x)\leq$
$\displaystyle-\rho(\|x-x_{s}\|)-\rho(\|u-u_{s}\|)$
$\displaystyle+\ell(x,u)-\ell(x_{s},u_{s}),$ (3)

for certain $\mathcal{K}$ function $\rho(\cdot)$. It is also assumed that the
storage function is locally Lipschitz continuous and bounded below for any
admissible trajectory of the system and that $u_{s}$ lies in the relative
interior of $\mathcal{U}$.

Notice that if this assumption holds, then the optimization problem (2) has an
unique solution.

Two different stabilizing economic model predictive control formulations are
considered in this paper: with and without terminal equality constraint. An
economic MPC with terminal equality constraint is derived from the solution of
the following optimization problem $P^{e}_{N}(x(k))$ [22]:

$\displaystyle\min_{\hat{\mathbf{u}}}$ $\displaystyle
V_{N,e}(x(k),\hat{\mathbf{u}})=$ (4a)
$\displaystyle\sum_{j=0}^{N-1}\ell(\hat{x}(j|k),\hat{u}(j))$ s.t.
$\displaystyle\hat{x}(0|k)=x(k)$ (4e)
$\displaystyle\hat{x}(j+1|k)=f(\hat{x}(j|k),\hat{u}(j)),j\in{\mathbb{I}}_{0}^{N}\quad$
$\displaystyle\hat{u}(j)\in\mathcal{U}$ $\displaystyle\hat{x}(N|k)=x_{s}.$

The optimum of this problem is denoted $V_{N,e}^{*}(x(k))$.

A more general formulation of the economic MPC without terminal equality
constraint can be obtained adding a relaxed terminal constraint and a terminal
cost function, leading to the following problem $P_{N}^{t}(x(k))$ [2]:

$\displaystyle\min_{\hat{\mathbf{u}}}$ $\displaystyle
V_{N,t}(x(k),\hat{\mathbf{u}})$ (5b)
$\displaystyle=\sum_{j=0}^{N-1}\ell(\hat{x}(j|k),\hat{u}(j))+V_{f}(x(N|k))\quad$
s.t. $\displaystyle\eqref{eq:PNe2}-\eqref{eq:PNe4}$ (5c)
$\displaystyle\hat{x}(N|k)\in X_{f}.$

The optimum is denoted $V_{N,t}^{*}(x(k))$.

In both cases, a state feedback control law (either
$u(k)=\kappa_{\textup{eco}}^{e}(x(k))$ or $\kappa_{\textup{eco}}^{t}(\cdot)$)
is obtained applying the solution of the corresponding optimization problem in
a receding horizon manner, i.e. $u(k)=u^{*}(0;x(k))$.

In this paper, it is assumed that the economic MPC optimization problems
satisfy the following condition:

######  Assumption 3.

The optimal solution of the problem $P_{N}^{e}(x(k))$ ( or $P_{N}^{t}(x(k))$)
is unique, and the optimal cost function $V_{N,e}^{*}(\cdot)$ (or
$V_{N,t}^{*}(\cdot)$) is continuous at the reference $x=x_{s}$.

The asymptotic stability of the closed-loop system, controlled by the EMPC law
with terminal equality constraint, $\kappa_{eco}^{e}(x)$, was proven in [23].
In the case of the EMPC with a relaxed terminal constraint, under a suitable
design of the terminal ingredients $V_{f}(\cdot)$ and $X_{f}$, the resulting
control law also stabilizes the system to the optimal equilibrium point.
Following [24], these conditions are categorized in three different cases as
follows:

  * TC1

Terminal inequality constraint [22]. Considering a locally stabilizing control
law, a suitable function $V_{f}(\cdot)$ and an invariant set $X_{f}$
satisfying a set of conditions. These ingredients can be determined from the
linearized model of the plant at the optimal equilibrium point.

  * TC2

Terminal cost function and no terminal constraint [19]. Taking
$X_{f}={\mathbb{R}}^{n}$, a linear terminal cost function
$V_{f}(x)=\eta_{f}^{T}x$, where $\eta_{f}$ is the Lagrange multiplier
corresponding to constraint (2b) in optimization problem (2), and a
sufficiently long prediction horizon $N$.

  * TC3

No terminal ingredients [25]. Taking $X_{f}={\mathbb{R}}^{n}$, $V_{f}(x)=0$
and a sufficiently long prediction horizon, the resulting controller is
practically asymptotically stable and the ultimately bound depends inversely
on the prediction horizon.

For the cases TC2 and TC3, additional technical assumptions are necessary,
such as exponential reachability of the equilibrium point, regularity of the
steady-state optimization problem and some controllability conditions of the
linearized model at the equilibrium point [26].

All these controllers are recursively feasible and stabilize the closed-loop
system providing an economically optimal closed-loop trajectory, such that

$\lim_{T\rightarrow\infty}\frac{1}{T}\sum_{k=0}^{T-1}\ell(x(k),\kappa_{\textup{eco}}(x(k)))\leq\ell(x_{s},u_{s}).$
(6)

Note that in addition to the averaged performance defined here, the literature
also contains estimates for transient or non-averaged optimality of economic
MPC schemes (see, e.g., [25]), which may be interesting to consider in the
oracle-based approach proposed in this paper.

###  2.2 Oracle-based economic MPC

The standard economic MPC formulations presented in this section require
knowledge of the prediction model of the plant and the measurement or
estimation of the current state. However, there may exists scenarios in which
this information is not accessible, due e.g. to privacy or security conditions
of the plant.

The main objective of this work is to design an economic predictive controller
that stabilizes the plant and minimizes its economic performance satisfying
(6), under the assumption that the model of the plant is not known and that
the only measurement of the plant is the value of the economic cost at each
sampling time.

Using a database of past inputs and economic cost trajectories, a function
used to predict the evolution of the cost will be obtained. This function is
called an oracle, since it allows us to forecast the economic performance of
the plant. Once that this oracle is obtained, a suitable predictive controller
will be designed based only in the available measurement of the current
economic cost function.

In the following section, the existence of an oracle is studied. Then, oracle-
based economic predictive controllers will be presented, and their properties
analyzed.

##  3 The oracle

In this section, the procedure to obtain an oracle from past input and
performance trajectories, and the conditions under which such an oracle exists
are presented. The oracle proposed has the form of a nonlinear auto-regressive
model with exogenous signals (NARX), which has been extensively used in
nonlinear systems identification [27], defined by the following nonlinear
difference equation 222 We may sometimes aggregate the notation of the cost as
$\ell(k)=\ell(x(k),u(k))$.

$\hat{\ell}(k)=\mathcal{O}(z(k),u(k)),$ (7)

where $\hat{\ell}(k)$ is the estimated cost at sampling time $k$, and $z(k)$
is a vector given by the following collection of past inputs and costs:

$\displaystyle z(k)$ $\displaystyle=$
$\displaystyle(\ell(k-1),\cdots,\ell(k-n_{a}),$ (8)
$\displaystyle\;u(k-1),\cdots,u(k-n_{b})),$

for some memory horizons $n_{a},n_{b}\in\mathbb{N}$. The vector $z(k)$ can be
regarded as the state of the oracle, and its dimension is $n_{z}=n_{a}+m\cdot
n_{b}$. It follows that the oracle is a function
$\mathcal{O}:{\mathbb{R}}^{n_{z}}\times{\mathbb{R}}^{m}\rightarrow{\mathbb{R}}$,
since the economic cost is a real number.

Notice that the value of the economic cost function at time instant $k$,
$\ell(k)$, depends in general on the value of the input at the same time
instant, $u(k)$, leading to an inner feed-forward structure. This implies that
the state vector $z(k)$ can only depend on the sequence of past costs up to
$k-1$, that is, $\ell(k-1),\cdots,\ell(k-n_{a})$. Then, the state feedback
controller to be designed with the form $u(k)=\kappa_{\textup{MPC}}(z(k))$ is
such that the current control action $u(k)$ depends on the information of the
plant available up to $k-1$, given the set-up defined by (7) and (8).

The conditions under which a system can be described as a NARX have been
widely studied during the last 30 years. One of the first results on this
topic was given by Sontag [28], relating the existence of this model to the
observability property of the system. Later, Chen and Billings [29] proved
that local NARX models can be obtained if the system is locally observable. A
comprehensive study of this problem, for local and global estimators, was
presented by Levin and Narendra [30].

In [30, Theorem 3] it was proven that if the linearised model at the
equilibrium point $(x_{s},u_{s})$ is observable, then the dynamics of the
system can be locally described by a NARX model. When a global NARX model is
required, the strong observability property must be ensured, from the
linearised system, for instance. However, in [30, Theorem 6], the authors
proved that only generic observability is necessary, which is a much weaker
property. Since these results are a keystone of this paper, the main results
used in [30] have been summarized in the appendix, including an extra
corollary to relate the approach to the case presented in this paper.

Next, for the sake of clarity, we introduce the definition of Morse functions.

######  Definition 1 (Morse function).

A function $m(x,u)$ is said to be Morse in $x$ if for all
$(\tilde{x},\tilde{u})$ where the gradient
$\nabla_{x}m(\tilde{x},\tilde{u})=0$, the Hessian matrix
$\nabla_{xx}m(\tilde{x},\tilde{u})$ is non-singular [31].

In order to derive the existence of an oracle of the economic cost, the
following assumption must be fulfilled.

######  Assumption 4.

The economic cost function $\ell(x,u)$ is a Morse function in $x$.

In virtue of Corollary 2 (presented in the appendix), the conditions required
to ensure the existence of the oracle can be derived:

######  Theorem 1 (Existence of the oracle).

Consider that Assumptions 1 and 4 hold and that the horizons $n_{a}$ and
$n_{b}$ are larger than or equal to $2n$. Let
$\mathbf{u}_{z}(k)\in\mathcal{U}^{n_{b}+1}$ be the sequence of $n_{b}+1$ last
inputs applied to the system (1) up to sample time $k$. Then, for every
$\epsilon>0$ there exists a set of sequences of $n_{b}+1$ inputs
$\mathcal{U}_{no}\subseteq\mathcal{U}^{n_{b}+1}$ of measure
$\mu(\mathcal{U}_{no})<\epsilon$ such that:

  1. 1.

There exists a continuous oracle function (7) that describes the system (1),
i.e.

$\ell(k)=\mathcal{O}(z(k),u(k)),$

for any
$\mathbf{u}_{z}(k)\in\mathcal{U}_{o}=\mathcal{U}^{n_{b}+1}\setminus\mathcal{U}_{no}$.

  2. 2.

There exists a continuous oracle function (7) such that for any admissible
state $z(k)$ and any admissible inputs $u(k)\in\mathcal{U}$,

$|\ell(k)-\mathcal{O}(z(k),u(k))|\leq\epsilon$

From this theorem it follows that under mild conditions on the measured
economic cost, an oracle can be constructed to precisely predict the expected
evolution of the cost for almost every sequence of control inputs. Besides, it
is also proved that assuming certain (arbitrarily small) description error, an
oracle can be found for any admissible input.

The procedure to derive the oracle can be obtained using estimation and
learning theory methods. There exists a number of methods capable of
approximating the real function (the so-called ground truth function) from
possibly noisy sampled data, such as support vector machines, neural networks
or direct weight optimization [32]. Recently, other non-parametric methods
such as Gaussian processes [33] or kinky inference [16, 34] have gained a lot
of attention, thanks to their capability to provide estimations of the
prediction error.

As the model order $n$ may not be known a priori, the parameters of the chosen
estimator, including the memory horizons $n_{a}$ and $n_{b}$, can be
calculated from a database of historical inputs and costs, namely the
_training data set_. In addition, a different collection of data points is
used for validation of the proposed estimator. This cross-validation
methodology allows one to derive the best structure of the estimator, as well
as the best horizons $n_{a}$ and $n_{b}$, from the real data.

Notice that the requirement for the horizons $n_{a},n_{b}$ to be lager than
$2n$ can be theoretically addressed by considering that the oracle function
depends on a sequence of at least $2n$ past inputs and outputs, where some of
its components may not affect to the output of the oracle.

##  4 Oracle-based economic predictive control

In this section, the proposed oracle-based predictive controller is presented,
and its stability and optimality properties are studied, under the assumption
of perfect estimation.

Assuming that an oracle is available, the system can be described by an state-
space prediction model as follows:

$\displaystyle\hat{z}(j+1|k)$ $\displaystyle=$
$\displaystyle\hat{F}(\hat{z}(j|k),\hat{u}(j)),$ (9a)
$\displaystyle\hat{\ell}(j|k)$ $\displaystyle=$
$\displaystyle\mathcal{O}(\hat{z}(j|k),\hat{u}(j)),$ (9b)

where the predicted state $\hat{z}(j|k)\in{\mathbb{R}}^{n_{z}}$ is given by

$\displaystyle\hat{z}(j|k)$ $\displaystyle=$
$\displaystyle\big{(}\hat{\ell}(j-1|k),\cdots,\hat{\ell}(k),\cdots,$ (10)
$\displaystyle\;\ell(k+j-n_{a}),\;\hat{u}(j-1),\cdots,$
$\displaystyle\;\hat{u}(0),\cdots,u(j-n_{b})\big{)},$

for $j\geq 1$. This state includes measured costs $\ell$ and inputs $u$ if
$n_{a}\geq j$ or $n_{b}\geq j$ respectively, and only estimated values
$\hat{\ell}$ or $\hat{u}$ otherwise.

Thus, the prediction model is

$\displaystyle\hat{F}(\hat{z}(j|k),\hat{u}(j))$ $\displaystyle=$
$\displaystyle\big{(}\mathcal{O}(\hat{z}(j|k),\hat{u}(j)),\hat{\ell}(j-1|k),$
$\displaystyle\;\cdots,\hat{\ell}(1|k),\ell(k),\cdots,$
$\displaystyle\;\ell(k+j-n_{a}+1),\hat{u}(j),$
$\displaystyle\;\cdots,\hat{u}(j-n_{b}+1)\big{)}.$

Using this oracle-based prediction model, economic predictive controllers will
be presented, based on both terminal equality constraint and relaxed terminal
inequality constraint designs. The stability of the proposed controllers will
be analyzed in two steps, as it is customary in the predictive control field.
First, nominal stability is studied, assuming that the predictions (provided
by the oracle) are exact. Next, in the following section, stability under
prediction mismatches is analyzed.

######  Assumption 5 (Exact oracle).

The sequence of the last $n_{b}+1$ inputs applied to the system at every
sample time $k$, $\mathbf{u}_{z}(k)$, is contained in the set
$\mathcal{U}_{o}$ defined in Theorem 1 and the oracle exactly forecasts the
cost, i.e. $\mathcal{O}(z(k),u(k))=\ell(k)$.

From Theorem 1, we have that in practice this hypothesis could hold true for
any sequence of inputs with probability practically equal to 1, since the set
of non-observable sequence of inputs is of measure zero.

In order to derive the economic MPC based on the oracle, the economically
optimal equilibrium point (also based on the oracle) must be calculated first.
To this end, the following optimization problem, similar to (2), has to be
solved:

$\displaystyle(u_{s},\ell_{s})$ $\displaystyle=$
$\displaystyle\arg\min_{u\in\mathcal{U},\ell}\ell$ (11a)
$\displaystyle\text{s.t}.$ $\displaystyle\ell=\mathcal{O}(z,u)$ (11c)
$\displaystyle z=(\ell,\ldots,\ell,u,\ldots,u).$

In the following theorem, it is proven that the optimal equilibrium point
derived from this optimization problem is equivalent to the solution of (2).

######  Theorem 2.

If Assumptions 1-5 hold, then the optimal equilibrium point derived from the
solution of (11), based on the oracle, is equal to the optimal equilibrium
point of the plant, derived from (2).

###### Proof.

Since the state $z(k)$ contains the last $2n$ measures of the measured input
and output, from statement 2 of Theorem 6 and Corollary 2 (both presented in
the appendix), there exists a continuous bijective function $\Phi_{z}(\cdot)$
such that

$(x(k-2n),\mathbf{u}_{z}(k))=\Phi_{z}(z(k)),$ (12)

where $\mathbf{u}_{z}(k)$ denotes the sequence of past inputs contained in
$z(k)$, such that $\mathbf{u}_{z}(k)=(u(k-1),\cdots,u(k-2n))$. Let denote
$\Phi(\cdot)$ the continuous function $x(k-2n)=\Phi(z(k))$, given by the first
$n$ components of the map $\Phi_{z}(\cdot)$.

Let $(\bar{x}_{s},\bar{u}_{s})$ be the optimizer of (2), and $\bar{\ell}_{s}$
its optimal economic cost function. Let $(\ell_{s},u_{s})$ be the optimizer of
(11) and let $z_{s}$ be the optimal steady state of the oracle-based model
given by (11c). Since ($\bar{x}_{s},\bar{u}_{s})$ is an equilibrium point of
(1), then defining

$\bar{z}_{s}=(\bar{\ell}_{s},\ldots,\bar{\ell}_{s},\bar{u}_{s},\ldots,\bar{u}_{s}),$

we have that $(\bar{z}_{s},\bar{u}_{s})$ is an equilibrium point of (9). Thus,
$(\bar{\ell}_{s},\bar{u}_{s})$ is a feasible solution of (11), which means
that $\bar{\ell}_{s}\geq\ell_{s}$.

Since the map $\Phi(\cdot)$ is continuous, the condition $z(k)=z(k+1)$ implies
that $x(k-2n)=x(k-2n+1)$. That is, an equilibrium point of (9) corresponds to
an equilibrium of (1), and vice versa. Indeed, for a given $u_{s}$, both
equilibrium points are related by the continuous map $x_{s}=\Phi(z_{s})$.

Therefore, the pair $(\Phi(z_{s}),u_{s})$ is a feasible solution of (2), and
its optimal cost function is $\ell_{s}$, which leads to
$\ell_{s}\geq\bar{\ell}_{s}$, and consequently, $\ell_{s}=\bar{\ell}_{s}$.

From Assumption 2, we derive that the solution of (2) is also unique, which
means that $\bar{u}_{s}=u_{s}$, and $\bar{x}_{s}=\Phi(z_{s})=x_{s}$. ∎

In what follows, the economic model predictive controllers based on the
proposed oracle are presented.

###  4.1 Oracle-based economic MPC with terminal equality constraint

This formulation is one of the simplest stabilizing designs of the economic
MPC. If the state-space system model were available, the control law could be
derived from the optimization problem $P_{N}^{e}(x(k))$. In case that the
oracle is used as prediction model, we present a formulation based uniquely in
the available information, whose control law is proven to be equivalent to the
control law derived from the state-space prediction model.

The proposed oracle-based economic MPC with terminal equality constraint is
derived from the solution of the following optimization problem
$P_{\mathcal{O}}^{e}(z(k))$:

$\displaystyle\min_{\hat{\mathbf{u}}}$ $\displaystyle
V_{\mathcal{O},e}(z(k),\hat{\mathbf{u}})=\sum_{j=0}^{N-1}\hat{\ell}(j|k)$
(13a) s.t. $\displaystyle\hat{z}(0|k)=z(k)$ (13g)
$\displaystyle\hat{z}(j+1|k)=\hat{F}(\hat{z}(j|k),\hat{u}(j)),j\in{\mathbb{I}}_{0}^{N_{p}-1}\quad$
$\displaystyle\hat{\ell}(j|k)=\mathcal{O}(z(j|k),u(j))$
$\displaystyle\hat{u}(j)\in\mathcal{U}$
$\displaystyle\hat{u}(j)=u_{s},\,\forall j\in{\mathbb{I}}_{N}^{N_{p}-2}$
$\displaystyle\hat{\ell}(j|k)=\ell_{s},\,\forall
j\in{\mathbb{I}}_{N}^{N_{p}-1},$

where $N_{p}=N+\max(n_{a},n_{b})$ and $(\ell_{s},u_{s})$ is the optimal
solution of (11).

Note that the terminal constraint, defined by (13g) and (13g), requires that
both the input and the cost maintain certain constant value for a given period
of time, defined by the memory horizons of the NARX model. The control law is
then given by

$u(k)=\kappa_{\mathcal{O}}^{e}(z(k))=\hat{u}^{*}(0;z(k)).$

In order to compare the oracle-based controller and its state-space model
counterpart, we include the following definition.

######  Definition 2 (Consistent states).

For a given trajectory of the system, we say that the state $z(k)$ of the
model (9) is _consistent_ with the state $x(k)$ of the model (1) if (8) holds,
i.e., if

$\displaystyle z(k)=\big{(}\ell(x(k-1),u(k-1)),\cdots,\ell(x(k-n_{a}),$
$\displaystyle u(k-n_{a})),u(k-1),\cdots,u(k-n_{b})\big{)}.$

In the following theorem, it is stated that the oracle-based economic MPC is
equivalent to the model-based economic MPC under nominal conditions, and hence
it renders the controlled system asymptotically stable and minimizes the
economic performance.

######  Theorem 3.

Consider that Assumptions 1-4 hold and assume that the initial state $x(0)$ is
such that $P_{N}^{e}(x(0))$ is feasible. Then, under Assumption 5 the
optimization problem $P_{\mathcal{O}}^{e}(z(0))$ is feasible, where $z(0)$ is
the state of the oracle consistent with $x(0)$, and the evolution of the
system controlled by the economic control law $\kappa_{\mathcal{O}}^{e}(z(k))$
is equal to the one resulting from the control law derived from
$P_{N}^{e}(x(k))$, that is,
$\kappa_{\mathcal{O}}^{e}(z(k))=\kappa_{\textup{eco}}^{e}(x(k))$.

###### Proof.

First, take into account that in virtue of Theorem 1 and Assumption 5, if
$x(k)$ and $z(k)$ are consistent, then for a certain sequence of future
control inputs $\mathbf{u}=(u(0),\cdots,u(N-1))$, the predicted trajectories
using the plant model (1) and the oracle (9) are such that

$\ell(x(j|k),u(j))=\hat{\ell}(j|k)=\mathcal{O}(z(j|k),u(j)).$ (14)

Assume that $x(k)$ is such that $P_{N}^{e}(x(k))$ is feasible, then there
exists a sequence of $N$ inputs $\tilde{\mathbf{u}}$ that steers the system to
the equilibrium point $(x_{s},u_{s})$. From Theorem 2, we have that the
optimal solution of (11), $\ell_{s}$, is equal to $\ell(x_{s},u_{s})$. From
the consistency between the models (14), it is inferred that
$\hat{\ell}(N|k)=\ell_{s}$. Since $x(N|k)=x_{s}$, applying subsequently
$u_{s}$, the system remains in the equilibrium point $(x_{s},u_{s})$ and then
$\hat{\ell}(j|k)=\ell(x_{s},u_{s})=\ell_{s}$ for $j\in\mathbb{I}_{N}^{N_{p}}$.
Therefore, $P_{\mathcal{O}}^{e}(z(k))$ is feasible.

From this, we can infer that $P_{\mathcal{O}}^{e}(z(0))$ is feasible given
that $P_{N}^{e}(x(0))$ is assumed to be feasible.

In order to prove the equivalence between the control laws, it is demonstrated
that (i) the optimal solution of $P_{N}^{e}(x(k))$ is a solution of
$P_{\mathcal{O}}^{e}(z(k))$ and (ii) the opposite.

(i) It was proven that if $P_{N}^{e}(x(k))$ is feasible, then the optimal
solution of $P_{N}^{e}(x(k))$, $\mathbf{u}^{*}_{x}$, is a feasible solution of
$P_{\mathcal{O}}^{e}(z(k))$, and then the costs are such that
$V_{\mathcal{O},e}(z(k),\mathbf{u}^{*}_{x})\geq V_{\mathcal{O},e}^{*}(z(k))$.
Taking into account (14), we have that
$V_{\mathcal{O},e}(z(k),\mathbf{u}^{*}_{x})=V_{N,e}^{*}(x(k))$, and then
$V_{N,e}^{*}(x(k))\geq V_{\mathcal{O},e}^{*}(z(k))$.

(ii) On the other hand, in order to prove the opposite, notice that the set of
constraints (13g) and (13g) is equivalent to force that $z(N_{p}|k)=z_{s}$.

Hence, as the optimal solution of $P_{\mathcal{O}}^{e}(z(k))$,
$\mathbf{u}^{*}_{z}$ leads to $z(N_{p}|k)=z_{s}$, recalling function
$\Phi(\cdot)$ defined in the proof of Theorem 2, we have that

$x(N|k)=x(N_{p}-2n|k)=\Phi(z(N_{p}|k))=\Phi(z_{s})=x_{s},$

and therefore $\mathbf{u}^{*}_{z}$ is a suboptimal solution of
$P_{N}^{e}(x(k))$. From (14), we have that
$V_{N,e}(x(k),\mathbf{u}^{*}_{z})=V_{\mathcal{O},e}^{*}(z(k))$, so
$V_{\mathcal{O},e}^{*}(z(k))=V_{N,e}(x(k),\mathbf{u}^{*}_{z})\geq
V^{*}_{N,e}(x(k))$.

Then, it is straightforward that
$V^{*}_{N,e}(x(k))=V^{*}_{\mathcal{O},e}(z(k))$, so the solution is the same
for both problems, i.e. $\mathbf{u}^{*}_{x}=\mathbf{u}^{*}_{z}$, provided that
Assumption 3 holds. ∎

From this theorem we conclude that the proposed oracle-based economic control
law $\kappa_{\mathcal{O},e}(z(k))$ provides the same domain of attraction, the
same optimality properties and the same evolution that the state-space law
$\kappa_{N,e}(x(k))$. This demonstrates that an oracle of the economic cost
function suffices to derive economic predictive controllers under nominal
conditions.

###  4.2 Oracle-based economic MPC without terminal equality constraint

A stabilizing economic model predictive control design resorting on terminal
equality constraint is simple and direct, but the resulting controller might
exhibit a small domain of attraction and lack of inherent robustness [35]. In
this section, stabilizing oracle-based economic predictive control laws are
derived extending the existing terminal conditions design summarized in
Section 2.1 to the oracle-based framework. This design is only based on the
input-measured cost available information, and the stability of the resulting
controller is proven.

Taking a stabilizing design based on terminal ingredients (cf. TC1 in Section
2.1), the optimization control problem $P_{\mathcal{O}}^{t}(z(k))$ is
formulated as follows, where a prediction horizon $N_{p}$ larger than the
control horizon $N_{c}$ can be considered:

$\displaystyle\min_{\hat{\mathbf{u}}}$ $\displaystyle
V_{\mathcal{O},t}(z(k),\hat{\mathbf{u}})=$ (15a)
$\displaystyle\sum_{j=0}^{N_{p}-1}\hat{\ell}(j|k)+V_{\mathcal{O},f}(\hat{z}(N_{p}|k))$
s.t. $\displaystyle\hat{z}(0|k)=z(k)$ (15i)
$\displaystyle\hat{z}(j+1|k)=\hat{F}(\hat{z}(j|k),\hat{u}(j)),j\in{\mathbb{I}}_{0}^{N_{p}-1}$
$\displaystyle\hat{\ell}(j|k)=\mathcal{O}(z(j|k),\hat{u}(j))$ $\displaystyle
u(j)\in\mathcal{U}$
$\displaystyle\hat{z}(i+1|k)=\hat{F}(\hat{z}(i|k),\kappa_{f}(z(i|k))),$
$\displaystyle i\in{\mathbb{I}}_{N_{c}}^{N_{p}-1}$
$\displaystyle\hat{\ell}(i|k)=\mathcal{O}(z(i|k),\kappa_{t}(z(i|k)))$
$\displaystyle\kappa_{f}(z(i|k))\in\mathcal{U}$ $\displaystyle
z(N_{p}|k)\in\mathcal{Z}_{f}.$

The control law is implicitly obtained applying the optimal sequence of
control inputs in a receding horizon manner
$u(k)=\kappa_{\mathcal{O}}^{t}(z(k))=\hat{u}^{*}(0|k)$.

In order to derive asymptotic stability of the economic MPC, the following
assumption is made [22], in which the terminal cost function
$V_{\mathcal{O},f}$ and the terminal set $\mathcal{Z}_{f}$ must satisfy the
following conditions:

######  Assumption 6.

$V_{\mathcal{O},f}$ is continuous at $z=z_{s}$ and there exists a terminal
control law $u=\kappa_{f}(z)$ such that for all $z\in\mathcal{Z}_{f}$,
$\kappa_{f}(z)\in\mathcal{U}$,
$z^{+}=\hat{F}(z,\kappa_{f}(z))\in\mathcal{Z}_{f}$ and

$V_{\mathcal{O},f}(z^{+})-V_{\mathcal{O},f}(z)\leq-\mathcal{O}(z,\kappa_{f}(z))+\ell(x_{s},u_{s}).$

If the linearization of the oracle-based autoregressive model (7) at the
equilibrium point given by (11) is stabilizable, then the terminal ingredients
can be calculated from the linearization, as proposed in [22], yielding a
linear terminal control law $\kappa_{f}(z)=K(z-z_{s})+u_{s}$, a quadratic
terminal cost $V_{\mathcal{O},f}(z)=\frac{1}{2}z^{T}Q_{f}z+q_{f}^{T}z$, and a
terminal constraint
$\mathcal{Z}_{f}=\\{\bar{z}:\bar{(}z-z_{s})^{T}P\bar{(}z-z_{s})\leq\alpha\\}$.

We are now in position to study the stability of the proposed control law.

######  Theorem 4.

Consider that Assumptions 1-6 hold. If the initial state $z(0)$ is such that
$P_{\mathcal{O}}^{t}(z(0))$ is feasible, then the equilibrium point given by
(11) is asymptotically stable for the system (1) controlled by the economic
control law $\kappa_{\mathcal{O}}^{t}(z(k))$ and its trajectory satisfies the
constraints.

###### Proof.

First, we show that there exists a map that relates the real state of the
system $x(k)$ and the state of the oracle-based model $z(k)$. From Assumption
1 we have that the model function $x\mapsto f(x,u)$ defines a diffeomorphism
in $x$ for a given $u$. Therefore, there exists a continuous and bijective map
$\mathcal{F}$ such that

$(x(k),\mathbf{u}_{z}(k))=\mathcal{F}(x(k-2n),\mathbf{u}_{z}(k)).$ (16)

Taking into account that $z(k)$ contains the sequence $\mathbf{u}_{z}(k)$ and
the maps (12) and (16), we can state that
$\mathcal{G}=\mathcal{F}\circ\Phi_{z}$ is a continuous and bijective map such
that

$(x(k),\mathbf{u}_{z}(k))=\mathcal{G}(z(k)).$ (17)

Based on this map, it is shown that the system (7) is dissipative with respect
to the output of the oracle. Consider the continuous function
$x(k)=G_{x}(z(k))$ given by the first $n$ components of map
$\mathcal{G}(\cdot)$ introduced in (17), and define the storage function
$\lambda_{z}(z)=\lambda(G_{x}(z))$.

Then, from Assumptions 2 and 5 and Theorem 2, we have that

$\displaystyle\lambda_{z}(z(k+1))$
$\displaystyle-\lambda_{z}(z(k))=\lambda(x(k+1))-\lambda(x(k))$
$\displaystyle\leq$ $\displaystyle\,\ell(x(k),u(k))-\ell(x_{s},u_{s})$
$\displaystyle+\rho(\|x(k)-x_{s}\|)+\rho(\|u(k)-u_{s}\|)$ $\displaystyle=\,$
$\displaystyle\mathcal{O}(z(k),u(k))-\ell_{s}$
$\displaystyle+\rho(\|x(k)-x_{s}\|)+\rho(\|u(k)-u_{s}\|).$

Stability of the proposed controller can be stated using the rotated stage
cost function and terminal cost function, as proposed in [22]:

$\displaystyle L_{r}(z,u)$ $\displaystyle=$
$\displaystyle\mathcal{O}(z,u)+\lambda_{z}(z)-\lambda_{z}(\hat{F}(z,u))-\ell_{s},$
$\displaystyle V_{\mathcal{O},fr}(z)$ $\displaystyle=$ $\displaystyle
V_{\mathcal{O},f}(z)+\lambda_{z}(z)-V_{\mathcal{O},f}(z_{s})-\lambda_{z}(z_{s}).$

Notice that the rotated stage cost $L_{r}$ satisfies the following condition:

$L_{r}(z,u)\geq\rho(\|x-x_{s}\|)+\rho(\|u-u_{s}\|),$

where $x=G_{x}(z)$.

Now, we define the optimization problem $P_{\mathcal{O},tr}(z(k))$ based on
the rotated ingredients as:

$\displaystyle\min_{\hat{\mathbf{u}}}$ $\displaystyle
V_{\mathcal{O},tr}(z(k),\hat{\mathbf{u}})=$ (18a)
$\displaystyle\;\sum_{j=0}^{N_{p}-1}L_{r}(j|k)+V_{\mathcal{O},fr}(\hat{z}(N_{p}|k))$
$\displaystyle\mathrm{s.t.}$ $\displaystyle\eqref{eq:PO1}-\eqref{eq:PO8}$
(18c) $\displaystyle L_{r}(j|k)=\hat{\ell}(j|k)+\lambda_{z}(\hat{z}(j|k))$
$\displaystyle\;-\lambda_{z}(\hat{z}(j+1|k))-\ell_{s}.$

Following [22], it can be proven that

$\displaystyle V_{\mathcal{O},tr}(z(k),\hat{\mathbf{u}})$ $\displaystyle=$
$\displaystyle\sum_{j=0}^{N_{p}-1}L_{r}(j|k)+V_{\mathcal{O},fr}(\hat{z}(N_{p}|k))$
$\displaystyle=$ $\displaystyle
V_{\mathcal{O},t}(z(k),\hat{\mathbf{u}})+\lambda_{z}(z(k))-N_{p}\ell_{s}$
$\displaystyle\;-V_{\mathcal{O},f}(z_{s})-\lambda_{z}(z_{s}),$

and then the optimal solution of $P_{\mathcal{O},tr}(z(k))$ is equal to the
optimal solution of $P_{\mathcal{O},t}(z(k))$. Therefore, the receding horizon
control laws derived from these two optimization problems are identical.

Next, asymptotic stability of the proposed economic MPC is addressed. First,it
can be shown that the optimal cost function $V^{*}_{\mathcal{O},tr}(z)$ is
such that

$\displaystyle V^{*}_{\mathcal{O},tr}(z(k))$ $\displaystyle\geq
L_{r}(z(k),\hat{u}^{*}(0|k))$
$\displaystyle\geq\rho(\|x(k)-x_{s}\|)+\rho(\|u(k)-u_{s}\|),$ (19)

for all feasible $z(k)$.

In order to prove asymptotic stability, we introduce the following Lyapunov
function candidate:

$W(z(k))=\sum_{j=k-2n}^{k}V^{*}_{\mathcal{O},tr}(z(j)),$ (20)

and next we will prove that it satisfies the sufficient conditions to derive
asymptotic stability of the controlled system.

From (4.2), we have that

$\displaystyle W(z(k))$
$\displaystyle=\sum_{j=k-2n}^{k}V^{*}_{\mathcal{O},tr}(z(j))$
$\displaystyle\geq\sum_{j=k-2n}^{k}\rho(\|x(j)-x_{s}\|)+\rho(\|u(j)-u_{s}\|).$
(21)

Denoting the sequences $\mathbf{x}_{2n}(k-2n)=(x(k),\cdots,x(k-2n))$, and
$\mathbf{u}_{2n}(k-2n)=(u(k),\cdots,u(k-2n))$ and $\mathbf{x}_{2n}^{s}$ and
$\mathbf{u}_{2n}^{s}$ their steady values counterparts at $(x_{s},u_{s})$, we
can see that the right hand side of (21) is a $\mathcal{K}$-function of the
sequence of $\mathbf{x}_{2n}(k-2n)-\mathbf{x}_{2n}^{s}$ and
$\mathbf{u}_{2n}(k-2n)-\mathbf{u}_{2n}^{s}$. From similar arguments to the
derivation of map $\mathcal{F}$ previously presented in (16), there exists a
bijective map $\mathcal{F}_{2n}$ such that

$\displaystyle(\mathbf{x}_{2n}(k-2n),$ $\displaystyle\mathbf{u}_{2n}(k-2n))=$
$\displaystyle\mathcal{F}_{2n}(x(k-2n),\mathbf{u}_{2n}(k-2n)).$ (22)

Taking into account (12), and denoting
$\mathcal{H}_{2n}=\mathcal{F}_{2n}\circ\Phi_{z}$, we have that
$\mathcal{H}_{2n}$ is a bijective map such that

$(\mathbf{x}_{2n}(k-2n),\mathbf{u}_{2n}(k-2n))=\mathcal{H}_{2n}(z(k)),$ (23)

and then
$(\mathbf{x}_{2n}(k-2n),\mathbf{u}_{2n}(k-2n))=(\mathbf{x}_{2n}^{s},\mathbf{u}_{2n}^{s})$
if and only if $z(k)=z_{s}$. Therefore, there exists a $\mathcal{K}$-function
$\beta_{1}$ such that

$\displaystyle W(z(k))\geq$
$\displaystyle\sum_{j=k-2n}^{k}\rho(\|x(j)-x_{s}\|)+\rho(\|u(j)-u_{s}\|)$
$\displaystyle\geq$ $\displaystyle\beta_{1}(\|z(k)-z_{s}\|).$ (24)

On the other hand, we have that $V^{*}_{\mathcal{O},tr}(z)\leq
V_{\mathcal{O},fr}(z)$ for all $z-z_{s}\in\mathcal{Z}_{f}$ (see Proposition
2.18 in [36]), which means that the optimal cost function
$V^{*}_{\mathcal{O},tr}(z)$ is continuous at $z_{s}$ and locally bounded for
all $z$ such that $z-z_{s}\in\mathcal{Z}_{f}$. Consequently $W(z)$ is also
continuos at $z_{s}$ and locally bounded and in virtue of Proposition B.25 in
[37], there exists a $\mathcal{K}$-function $\beta_{2}(\cdot)$ such that for
all $z-z_{s}\in\mathcal{Z}_{f}$

$W(z)\leq\beta_{2}(\|z-z_{s}\|).$

Finally, following [22] we have that

$\displaystyle V^{*}_{\mathcal{O},tr}(z(k$
$\displaystyle+1))-V^{*}_{\mathcal{O},tr}(z(k))\leq-
L_{r}(z(k),\hat{u}^{*}(0|k))$
$\displaystyle\leq-\rho(\|x(k)-x_{s}\|)-\rho(\|u(k)-x_{s}\|).$ (25)

Then, we can state that

$\displaystyle W(z$ $\displaystyle(k+1))-W(z(k))\leq$
$\displaystyle\sum_{j=k-2n}^{k}-\rho(\|x(j)-x_{s}\|)-\rho(\|u(j)-x_{s}\|).$
(26)

From (4.2), we get the last sufficient condition in order to derive asymptotic
stability

$W(z(k+1))-W(z(k))\leq-\beta_{1}(\|z(k)-z_{s}\|).$ (27)

Consequently, the system controlled by the control law derived from (13) is
stable, and it asymptotically steers the state $z(k)$ to the equilibrium point
$z_{s}$. From (12), this implies that the state of system, $x(k)$, is
equivalently steered to the economically optimal steady state $x_{s}$. ∎

######  Remark 2.

(Economic optimality) From equation (25) we can derive that

$V^{*}_{\mathcal{O},t}(z(k+1))-V^{*}_{\mathcal{O},t}(z(k))\leq-\ell(z(k),\kappa_{\mathcal{O}}^{t}(z(k)))-\ell_{s}.$
(28)

and therefore, equation (6) also holds for the proposed controller.

Based on the previous stability proof of the oracle-based economic MPC, the
stabilizing designs without terminal constraint proposed by [19] and [25] (cf.
TC2 and TC3 in Section 2.1) can also be extended to the oracle-based case,
provided that some additional technical conditions are fulfilled. In this
case, the control law is derived from the following optimization problem.

$\displaystyle\min_{\hat{\mathbf{u}}}$ $\displaystyle
V_{\mathcal{O},tc}(z(k),\hat{\mathbf{u}})=\sum_{j=0}^{N-1}\hat{\ell}(j|k)+\eta^{T}\hat{z}(N|k)$
s.t. $\displaystyle\hat{z}(0|k)=z(k)$ (29e)
$\displaystyle\hat{z}(j+1|k)=\hat{F}(\hat{z}(j|k),\hat{u}(j)),j\in{\mathbb{I}}_{0}^{N-1}$
$\displaystyle\hat{\ell}(j|k)=\mathcal{O}(z(j|k),\hat{u}(j))$ $\displaystyle
u(j)\in\mathcal{U}.$

Thus, taking a sufficiently large prediction horizon $N$ and an appropriate
Lagrange multiplier $\eta$, the resulting control law asymptotically
stabilizes the system [19]. Taking $\eta=0$, then only practical stability can
be achieved [25].

##  5 Practical viability of the oracle-based economic MPC

In the previous section, asymptotic stability of the oracle-based economic MPC
was proven under the nominal conditions stated in Assumption 5. The exact
prediction considered in this hypothesis may hold if every applied sequence of
inputs rendered the system observable, and if the available oracle NARX model
were exact.

In this section we study the case in which the available oracle is inexact,
providing approximate predictions with a bounded estimation error. We consider
that the two previous assumptions may not hold, which is likely in practice,
either due to non-observable input sequences or because of oracles with
estimation errors.

In practice, the oracle is determined from input-output historical data,
generated with an appropriate sequence of inputs, for which a bound on the
estimation error signal

$|w(k)|=|\ell(x(k),u(k))-\mathcal{O}(z(k),u(k))|$ (30)

may be estimated. There exists machine learning methods that provide either
guaranteed bounds, such as kinky inference [34, 38], or probabilistic bounds,
such as Gaussian processes [33]. We assume that we are in the (worst-case)
scenario that the obtained bound is only valid for every sequence of inputs
that makes the system observable.

In the following proposition, it is proven that an inexact oracle obtained
from input-output data, possibly derived using observable sequences of inputs,
with a bounded estimation error, also provides estimations with a bounded
error for any sequence of inputs, even if they are not observable.

######  Proposition 1.

Consider that Assumptions 1 and 4 hold and that the oracle function
$\mathcal{O}(\cdot,\cdot)$ is a continuous function such that the estimation
error (30) is bounded by $\tilde{\mu}$ for every sequence of inputs for which
the system is observable. Then, for every admissible sequence of inputs, the
estimation error is bounded by $\mu>\tilde{\mu}$.

###### Proof.

From Theorem 6, derived in the appendix from [30, Theorem 6], we have that for
any (arbitrarily small) constant $\epsilon>0$, the set of sequence of inputs
for which the system is not observable, $\mathcal{U}_{no}$, is of measure
$\epsilon$, i.e. arbitrarily small. Let assume that the state of the system is
$x(k-2n)$ and a sequence of $2n$ inputs
$\mathbf{u}=(u(k-1),\cdots,u(k-2n))\in\mathcal{U}_{no}$ is applied, evolving
the system with a sequence of outputs
$\mathbf{l}=(\ell(k-1),\cdots,\ell(k-2n))$.

Since $\mathbf{u}\in\mathcal{U}_{no}$, there exists a sequence
$\tilde{\mathbf{u}}\not\in\mathcal{U}_{no}$ such that
$\|\tilde{\mathbf{u}}-\mathbf{u}\|\leq\theta_{u}(\epsilon)$, being
$\theta_{u}$ a $\mathcal{K}$-function. Let $\tilde{l}$, $\tilde{x}(k)$ be the
corresponding sequence of outputs and the state derived from the application
of $\tilde{\mathbf{u}}$ at $x(k-2n)$. From the smoothness of the transition
map $f$ and of the output map $\ell$, we can derive that there exists a couple
of $\mathcal{K}$-functions $\theta_{y}$ and $\theta_{x}$ such that
$\|\tilde{x}(k)-x(k)\|\leq\theta_{x}(\epsilon)$ and
$\|\mathbf{\tilde{l}}-\mathbf{l}\|\leq\theta_{y}(\epsilon)$. Let define $z(k)$
as the regressor state that is composed of $\mathbf{l}$ and $\mathbf{u}$, and
$\tilde{z}(k)$ as the one composed of $\mathbf{\tilde{l}}$ and
$\tilde{\mathbf{u}}$. Then, the estimation error can be bounded as follows:

$\displaystyle|w(k)|$ $\displaystyle=$
$\displaystyle|\ell(x(k),u(k))-\mathcal{O}(z(k),u(k))|$ $\displaystyle\leq$
$\displaystyle|\ell(x(k),u(k))-\ell(\tilde{x}(k),u(k))|$
$\displaystyle+|\ell(\tilde{x}(k),u(k))-\mathcal{O}(\tilde{z}(k),u(k))|$
$\displaystyle+|\mathcal{O}(\tilde{z}(k),u(k))-\mathcal{O}(z(k),u(k))|.$

From the smoothness of $\ell(x,u)$, we have that this function is locally
bounded. Then, from [37, Proposition B.25], we have that for all
$x(k)\in\\{x:\|x-\tilde{x}(k)\|\leq\theta_{x}(\epsilon)\\}$ there exists a
$\mathcal{K}$-function $\theta_{\ell}(\cdot)$ such that

$\displaystyle|\ell(x(k),u(k))-\ell(\tilde{x}(k),u(k))|$
$\displaystyle\leq\theta_{\ell}(|x(k)-\tilde{x}(k)|)$
$\displaystyle\leq\theta_{\ell}(\theta_{x}(\epsilon)).$

On the other hand, since $\tilde{\mathbf{u}}$ is a sequence of inputs that
make the system observable, we have that
$|\ell(\tilde{x}(k),u(k))-\mathcal{O}(\tilde{z}(k),u(k))|\leq\tilde{\mu}$.

By construction, $\|z(k)-\tilde{z}(k)\|\leq\theta_{z}(\epsilon)$, where
$\theta_{z}$ is a $\mathcal{K}$-function.In addition, since the oracle
function $\mathcal{O}(z,u)$ is assumed to be continuous in its arguments and
the domains of $z$ and $u$ are compact, the oracle function is uniformly
continuous in its domain. Therefore, we can state that there exists a
$\mathcal{K}$-function $\theta_{\mathcal{O}}$ such that

$\displaystyle\mathcal{O}(\tilde{z}(k),u(k))-\mathcal{O}(z(k),u(k))|$
$\displaystyle\leq\theta_{\mathcal{O}}(z(k)-\tilde{z}(k))$
$\displaystyle\leq\theta_{\mathcal{O}}(\theta_{z}(\epsilon)).$

Recapping, we have demonstrated that

$|w(k)|=|\ell(x(k),u(k))-\mathcal{O}(z(k),u(k))|\leq\tilde{\mu}+\Theta(\epsilon),$

for a $\mathcal{K}$-function $\Theta(\cdot)$. Since $\epsilon$ can be taken
arbitrarily small, for any $\mu>\tilde{\mu}$, there exists a suitable
$\epsilon>0$ for which the bound on the estimation error holds. ∎

######  Remark 3.

Notice that the hypothesis that the oracle function is continuous is a mild
assumption, since the form of the oracle function is chosen by the user, so it
can be designed to satisfy this condition. For instance, oracles derived from
Lipschitz interpolation methods, neural networks or Gaussian processes can be
chosen to guarantee the satisfaction of this assumption.

Next, we will prove the practical viability of the proposed controllers,
demonstrating that the closed-loop system is input-to-state stable (ISS)
w.r.t. the model mismatch signal $w(k)$.

It is well known that a nominally stabilizing MPC may exhibit zero-robustness
due to the the discontinuity of the optimal solution [35]. In [39, Theorem 4],
it is proven that if the ingredients of the MPC are uniformly continuous and
the constraints on the state in the optimization problem are not active, then
the resulting control law is ISS in a region where the constraints on the
states are not active.

Although the system to be controlled does not have constraints on the states,
the optimization problems of the proposed oracle-based economic predictive
controllers ($P_{\mathcal{O}}^{e}$ and $P_{\mathcal{O}}^{t}$) have constraints
on the states, arising from the terminal constraint. .

In the case of $P_{\mathcal{O}}^{e}$, the terminal equality constraint is
instrumental for the stability and hence, it cannot be removed without
compromising the stability (although, as proven in [25], practical stability
can be achieved for a sufficiently large $N$).

Stability of predictive controllers with terminal cost but without terminal
constraint has been analyzed in [40]. In that paper, it was proven that there
exists a level set of the optimal cost function where the MPC without terminal
constraint is stabilizing. This level set can be enlarged if the prediction
horizon is increased, or if the terminal cost function is weighted. This
result was extended in [41] to the tracking case, and a prediction horizon
larger than a control horizon. As analysed in [16], larger prediction horizons
increase the domain of attraction while barely increasing computational
complexity, in contrast to choosing larger control horizons.

Taking these arguments into account, we can derive robust stability of the
proposed oracle-based economic MPC. To this aim, we first need to proof the
following lemma, which is an extension of [40, 41], tailored to this case.

######  Lemma 1.

Consider that Assumptions 1-4 and 6 hold and let
$V^{*}_{\mathcal{O},tr}(\cdot)$ be the optimal cost of the optimization
problem $P_{\mathcal{O},tr}(\cdot)$ given by (18). Then, there exists certain
positive constants $d$ and $\alpha$ such that for any $z$ satisfying

$V^{*}_{\mathcal{O},tr}(z)\leq N_{p}d+\alpha,$

the terminal constraint (15i) in $P_{\mathcal{O},tr}(z)$ is not active.

###### Proof.

Let define the compact region
$\Omega_{f}=\\{z:V_{\mathcal{O},fr}(z)\leq\alpha\\}$ and let $\alpha$ be a
positive constant such that $\Omega_{f}$ is contained in the interior of
$\mathcal{Z}_{f}$. Notice that this constant exists since the constraints are
not active at the equilibrium point, $z=z_{s}$.

Then, for all $z\in\Omega_{f}$ we have that

$V_{\mathcal{O},fr}(\hat{F}(z,\kappa_{f}(z)))-V_{\mathcal{O},fr}(z)\leq-
L_{r}(z,\kappa_{f}(z)).$ (31)

From this inequality we have that for any $M\geq 1$

$\sum_{j=0}^{M-1}L_{r}(z(j),\kappa_{f}(z(j)))+V_{\mathcal{O},fr}(z(M))\leq
V_{\mathcal{O},fr}(z),$ (32)

where $z(j)$ is the trajectory of the system beginning at $z$, and controlled
by the terminal control law $u=\kappa_{f}(z)$. Besides, from [36, Proposition
2.18] we can infer that $V^{*}_{\mathcal{O},tr}(z)\leq V_{\mathcal{O},fr}(z)$,
for all $z\in\Omega_{f}$.

Define the positive constant $d>0$ such that $L_{r}(z,u)>d$ for all
$z\not\in\Omega_{f}$, and for all $u\in\mathcal{U}$. This constant exists
since $L_{r}(\cdot,\cdot)$ is a positive definite function of $(z-z_{s})$, and
$\Omega_{f}$ is a neighborhood of $z_{s}$.

Let $\mathbf{u}^{*}$ and $\mathbf{z}^{*}$ be the optimal sequences of inputs
and states of $P_{\mathcal{O},tr}(z)$. We will prove that if the terminal
state $z^{*}(N_{p})$ is not contained in $\Omega_{f}$, then the whole
predicted trajectory lies out of $\Omega_{f}$.

This will be proved by contradiction. Assume that
$z^{*}(N_{p})\not\in\Omega_{f}$, but that there exists certain instant $i$ in
which $z^{*}(i)\in\Omega_{f}$.

Define the trajectories $\bar{\mathbf{u}}$ and $\bar{\mathbf{z}}$ as follows:
for $j\in\mathbb{I}_{0}^{i-1}$, $\bar{u}(j)=u^{*}(j)$ and
$\bar{z}(j+1)=\hat{F}(\bar{z}(j),u^{*}(j))$, with $z(0)=z$ (which satisfies
that $\bar{z}(j)=z^{*}(j)$); and for $j\geq i$,
$\bar{u}(j)=\kappa_{f}(\bar{z}(j))$ and
$\bar{z}(j+1)=\hat{F}(\bar{z}(j),\kappa_{f}(\bar{z}(j)))$.

Define $\bar{V}_{\mathcal{O},tr}(z)$ as the cost associated to the input
sequence $\bar{\mathbf{u}}$. Then, since $z^{*}(i)\in\Omega_{f}$ and taking
into account inequality (32), we have that

$\displaystyle\bar{V}_{\mathcal{O},tr}(z)$ $\displaystyle=$
$\displaystyle\sum_{j=0}^{i-1}L_{r}(z^{*}(j),u^{*}(j)))$
$\displaystyle+\sum_{j=i}^{N_{p}}L_{r}(\bar{z}(j),\kappa_{f}(\bar{z}(j)))$
$\displaystyle+V_{\mathcal{O},fr}(\bar{z}(N_{p}))$ $\displaystyle\leq$
$\displaystyle\sum_{j=0}^{i-1}L_{r}(z^{*}(j),u^{*}(j)))+V_{\mathcal{O},fr}(z^{*}(i)).$

On the other hand, taking into account that $z^{*}(N_{p})\not\in\Omega_{f}$

$\displaystyle V^{*}_{\mathcal{O},tr}(z)$ $\displaystyle=$
$\displaystyle\sum_{j=0}^{i-1}L_{r}(z^{*}(j),u^{*}(j)))$
$\displaystyle+\sum_{j=i}^{N_{p}}L_{r}(z^{*}(j),u^{*}(j))$
$\displaystyle+V_{\mathcal{O},fr}(z^{*}(N_{p}))$ $\displaystyle>$
$\displaystyle\sum_{j=0}^{i-1}L_{r}(z^{*}(j),u^{*}(j)))+\alpha.$

Moreover, by optimality we have that $\bar{V}_{\mathcal{O},tr}(z)\geq
V^{*}_{\mathcal{O},tr}(z)$, and then

$\displaystyle\sum_{j=0}^{i-1}L_{r}(z^{*}(j),u^{*}(j)))+V_{\mathcal{O},fr}(z^{*}(i))\geq\bar{V}_{\mathcal{O},tr}(z)$
$\displaystyle\geq
V^{*}_{\mathcal{O},tr}(z)>\sum_{j=0}^{i-1}L_{r}(z^{*}(j),u^{*}(j)))+\alpha.$

From this, we can state that $V_{\mathcal{O},fr}(z^{*}(i))>\alpha$, and hence
$z^{*}(i)\not\in\Omega_{f}$, which is a contradiction.

Therefore, we have proved that if $z^{*}(N_{p})\not\in\Omega_{f}$, then
$z^{*}(j)\not\in\Omega_{f}$ for all $j\in\mathbb{I}_{0}^{N_{p}}$, which
implies that $L_{r}(z^{*}(j),u^{*}(j))>d$. Then, the optimal cost function is
such that

$V^{*}_{\mathcal{O},tr}(z)>N_{p}d+\alpha.$

Consequently, it is proven that for any $z$ such that
$V^{*}_{\mathcal{O},tr}(z)\leq N_{p}d+\alpha$, then
$z^{*}(N_{p})\in\Omega_{f}\subset\mathcal{Z}_{f}$, and therefore the terminal
constraint is satisfied. ∎

At this point, we have demonstrated sufficient conditions to derive the robust
stability of the proposed oracle-based predictive controller in presence of
inexact oracles: (i) the control law ensures asymptotic stability in nominal
conditions, i.e. assuming that the oracle provides exact predictions; (ii) the
oracle provides predictions with bounded errors for any input applied on the
system and (iii) the terminal constraint is not active which, as it will be
proven next, ensures uniform continuity of the optimal cost function. This
result is rigorously stated in the following theorem.

######  Theorem 5.

Consider that Assumptions 1-4 and 6 hold, then the system (1) controlled by
the control law derived from $P_{\mathcal{O},t}(z)$ is input-to-state stable
w.r.t. the model mismatch signal $w$ in a neighborhood of the equilibrium
point $z_{s}$, $\Omega_{r}$, for a sufficiently small model mismatch. Besides,
the region $\Omega_{r}$ is enlarged if the prediction horizon increases, i.e.,

$\Omega_{r}(N_{p})\subseteq\Omega_{r}(N_{p}+1).$ (33)

###### Proof.

From Lemma 1, we have that there exists some positive constants $d$ and
$\alpha$ such that for all $z$ satisfying

$V^{*}_{\mathcal{O},tr}(z)\leq N_{p}d+\alpha,$ (34)

the terminal constraint in $P_{\mathcal{O},t}(z)$ is not active.

Define $\Omega_{r}(N_{p})=\\{z:W(z)\leq N_{p}d+\alpha\\}$, which is a level
set for the Lyapunov function, and hence it is a positive invariant set of the
controlled system. Besides, since $W(z)\geq\beta_{1}(|z-z_{s}|)$, $\Omega_{r}$
is compact. From this definition we have that for all $z(k)\in\Omega_{r}$,
$V^{*}_{\mathcal{O},tr}(z(k))\leq W(z(k))\leq N_{p}d+\alpha$, and thus the
terminal constraint is not active.

Since $\Omega_{r}$ is compact and $u$ is bounded, the value of
$\mathcal{O}(z,u)$ is bounded for all $(z,u)\in\Omega_{r}\times\mathcal{U}$.
Therefore, as the function $\mathcal{O}(z,u)$ is continuous, it is also
uniformly continuous in $\Omega_{r}\times\mathcal{U}$, and then the model
function (7) is uniformly continuous. Hence, in virtue of [39, Theorem 4], the
system controlled by $\kappa_{\mathcal{O},t}(z)$ is ISS with respect to the
model mismatch signal $w$ in $\Omega_{r}$, if the model mismatch is
sufficiently small.

Finally, from the definition of $\Omega_{r}(N_{p})$ we have that
$\Omega_{r}(N_{p})\subseteq\Omega_{r}(Np+1)$ since for any feasible $z$, the
optimal cost function decreases with $N_{p}$. ∎

######  Remark 4.

In the previous section it was proven that for a sufficiently large prediction
horizon, the control law without terminal constraint derived from (29) is
nominally stabilizing. Since this optimization problem does not have
constraints on the states, following the arguments of Theorem 5, the closed-
loop system is ISS w.r.t. to the model mismatch.

######  Remark 5 (Online learning).

Input-to-state stability of the oracle-based EMPC can be extended to the case
that the oracle is updated online with fresh data collected during the closed-
loop operation of the system. Notice that occasionally, the update of the data
set might not lead to an enhancement of the estimation error. Under the
assumption that the effect of the update policy on the predictions is bounded
by a sufficiently small value, the possible increment on the predicted cost
function is also bounded and sufficiently small, and then, the optimal cost
function serves also as an ISS-Lyapunov function.
This condition can be relaxed if the update policy filters those data points
collected that worsen the expected cost function, as proposed in [42].

##  6 Case study

This section presents a simulated case study in which the system taken into
consideration is the continuously stirred tank reactor (CSTR) introduced in
[24].

###  6.1 The reactor

Two consecutive reactions take place: $A\to B$ at a rate
$k_{1}=$1\text{\,}{\mathrm{min}}^{-1}$$, and $B\to C$ at a rate
$k_{2}=$0.05\text{\,}{\mathrm{min}}^{-1}$$. The molar concentrations of $A$
and $B$ are denoted $c_{A}$ and $c_{B}$, and measured in
$\mathrm{kmol}\text{\,}{\mathrm{m}}^{-3}$. They evolve according to the
following set of ordinary differential equations:

$\displaystyle\frac{dc_{A}(t)}{dt}$ $\displaystyle=$
$\displaystyle\frac{u(t)}{V}(c_{A0}-c_{A}(t))-k_{1}c_{A}(t),$ (35a)
$\displaystyle\frac{dc_{B}(t)}{dt}$ $\displaystyle=$
$\displaystyle\frac{u(t)}{V}(-c_{A}(t))+k_{1}c_{A}(t)-k_{2}c_{B}(t),$

where $u$ (${\mathrm{m}}^{3}\text{\,}{\mathrm{min}}^{-1}$) is the manipulable
input: the feed flow rate (note that the same flow exits the tank, so the
volume remains constant, $V=$1\text{\,}{\mathrm{m}}^{3}$$), and
$c_{A0}=$1\text{\,}\mathrm{kmol}\text{\,}{\mathrm{m}}^{-3}$$ is concentration
of $A$ in the input flow. The system is discretized with sampling time
$\tau_{s}=$0.25\text{\,}\mathrm{min}$$.

The economic cost function accounts for the use of reactant and the benefits
of the product:

$\ell(x(k),u(k))=u(k)(c_{A0}-\alpha c_{B}(k)),$ (36)

where the weighting term is set to $\alpha=4$ (in the appropriate dimensions).
Existing noise in the measurements follows a normal distribution of zero mean
and standard deviation of $2\%$ of the measurement. The system is subject to
input constraints, such that
$0<u\leq$2\text{\,}{\mathrm{m}}^{3}\text{\,}{\mathrm{min}}^{-1}$$. The set of
equilibrium points is represented in Figure 1. Its minimum is obtained for
$u_{s}=$1.043\text{\,}{\mathrm{m}}^{3}\text{\,}{\mathrm{min}}^{-1}$$,
resulting in $\ell_{s}=$-0.957\text{\,}\$\mathrm{/}\mathrm{min}$$.

Figure 1: Set of equilibrium costs.

###  6.2 Ideal estimation

To replicate the nominal conditions studied in the paper, we seek to obtain an
oracle that provides an exact estimation, in order to compare the control
actions that the standard EMPC and the oracle-based EMPC yield, and to
illustrate that they are the same, as proven in Theorem 3.

To this end, consistent states (as introduced in Definition 2) must be
obtained, and applied to the mentioned controllers. To do so we proceed as
follows. An initial state is chosen randomly, following an uniform
distribution, within the workspace (both concentrations among $0$ and
$1\text{\,}\mathrm{kmol}\text{\,}{\mathrm{m}}^{-3}$). A sequence of control
actions of length $N_{p}=\max(n_{a},n_{b})$, with $n_{a}=3,\,n_{b}=2$ is
chosen, each element drawn randomly from $\mathcal{U}$ using an uniform
distribution. This sequence is applied to the real system, which yields the
initial state $x_{0}$, consistent with the initial regressor $z_{0}$, given
the measured costs. This setup is represented in Figure 2.

Figure 2: One experiment of the procedure to compare standard and oracle-based
economic predictive controllers. Blue values are initial conditions obtained
randomly, which yield the consistent states $x_{0}$ (in yellow, fed to the
standard EMPC) and $z_{0}$ (in green, fed to the oracle-based EMPC). Building
an exact oracle and applying $P^{e}_{N}(x_{0})$ and
$P^{e}_{\mathcal{O}}(z_{0})$ yield the same control action.

Next, the standard EMPC with terminal equality constraint given by the
optimization problem (4) is applied from $x_{0}$, with prediction horizon
$N=5$ and run for a single time step. Every time the solver in the
optimization problem (MATLAB®’s fmincon function) obtains the predicted
trajectory and cost for a candidate sequence $\mathbf{u}$, the resulting
regressors are stored in a data set $\mathcal{D}$.

Once a data set is obtained, the kinky inference (KI) class of learning rules
is applied to build the oracle. Kinky inference [34] is a class of non-
parametric learning methods based on Lipschitz interpolation, which have
proven able to conform a valid model for nonlinear learning-based predictive
controllers [43]. Further details of the functioning of KI can be found in
[16]. Given its non-parametric nature, it is guaranteed that in the absence of
noise the prediction over a given query $q$ already contained in the data set
is its corresponding real output $f(q)$, which is key for the exact estimation
sought in this paper. Besides, there is no tuning process, apart from
identifying the Lipschitz constant of the ground truth function, which is
overweighted to $L=100$ in order to ensure sample consistency.

Once the oracle (7) is built, the oracle-based MPC problem (13) is applied
from $z_{0}$, again with $N=5$ and for a single simulation step. It is
important to remark that despite the way the data set was obtained, the only
information given to the oracle (and to the optimization problem) is
measurements of past costs and input signals, as we assumed in this paper.
Hence, note that neither the states nor the control law of the standard EMPC
have been accessible to the oracle, as represented in Figure 2.

This setup is repeated a hundred times, with different initial random states
and input sequences. The resulting optimal control actions are the same in all
cases, as it was stated in Theorem 3,
$\kappa_{\mathcal{O}}^{e}(z(k))=\kappa_{\textup{eco}}^{e}(x(k))$ for all $k$,
proving the statements of this paper.

###  6.3 Closed-loop example

The previous example illustrates that under perfect estimation, the control
decision of the proposed oracle-based EMPC is the same as the one that would
be obtained with a standard EMPC. However, the conditions to obtain the data
set are artificial and not practical. Next, we practically demonstrate that an
oracle derived from realistic input-output data sets can be obtained, and that
the proposed oracle-based economic MPC stabilizes the real plant in presence
of estimation errors.

To this end, a chirp signal is designed with initial and final frequencies of
$1\text{\times}{10}^{-6}$ and $0.3\text{\,}{\mathrm{min}}^{-1}$, respectively;
and amplitude and center of
$1\text{\,}{\mathrm{m}}^{3}\text{\,}{\mathrm{min}}^{-1}$, of
$40\text{\times}{10}^{3}$ samples of length. This signal is applied as control
input to the plant, yielding the data set of costs-inputs represented in
Figure 3. The regression state is built with $n_{a}=3,\,n_{b}=2$.

Figure 3: Training data set, obtained with a chirp input signal.

A kinky inference predictor is built with these data. In order to asses its
performance, another set of experiments is carried out, applying a
pseudorandom signal to the plant that consists of a sequence of steps of value
$0\leq u\leq 2$ (${\mathrm{m}}^{3}\text{\,}{\mathrm{min}}^{-1}$) and length
among $30\text{\,}\mathrm{s}$ and $5\text{\,}\mathrm{min}$. The validation
test represented in Figure 4 shows the performance and the prediction error.

Then, the oracle-based MPC (13) is applied, from one hundred initial points
consisting of the regressor obtained applying a random input sequence to the
plant on a random state, and measuring costs. The results are shown in Figure
5. Besides, in order to compare the performance to the ideal case, the
standard EMPC (4) is applied to the same 100 initial states. Note that since
the estimation is not perfect, the response differ, although the closed-loop
performance is very similar. Such performance, represented in Figure 5b, is
assessed by the index:

$\Phi=\sum_{i=1}^{t_{\mathrm{sim}}}\ell(i).$ (37)

(a) Validation experiment (b) Histogram of the prediction error

Figure 4: Validation of the predictor

(a) Closed-loop

(b) Performance index

Figure 5: Closed-loop performance of the oracle-based EMPC (blue solid lines)
and the standard ideal state-feedback EMPC (orange dashed lines), for 100
initial random states.

###  6.4 Online learning

Using the same setup as in the previous case, the system is disturbed with an
impulse, periodically every $10\text{\,}\mathrm{min}$, returning to the same
initial state. At that same time step, the data set $\mathcal{D}$ is updated,
including the set of regressors and costs measured during the transient. The
kinky inference technique employed to learn the oracle can be updated
recursively without a cumbersome tuning. The recalculation of the Lipschitz
constant can be done recursively in $\mathcal{O}(n_{\mathcal{D}})$ [44], being
$n_{\mathcal{D}}=40$ the number of new data points.

The performance index (37) is computed for each one of the iterations, that
is, during the interval of $10\text{\,}\mathrm{min}$ after the impulse when
the system is regulated to the optimal equilibrium point rejecting its effect.
The value of this index for every iteration is shown in Figure 6. Note that,
in general, the inclusion of past measurements in an online learning fashion
enhances the performance, yielding better closed-loop responses, closer to the
ideal state-feedback ones.

Figure 6: Performance index each iteration, i.e. during the regulation between
disturbing the system, including past measurements in the oracle online.

##  7 Conclusion

The objective of this paper was to prove that an economic predictive
controller can be designed for a system whose model is unknown, under the
assumption that only the economic cost of the plant is observed. This set-up
can be encountered in many situations in which the client is not willing to
provide further details of the intern operation of his plant, or when the
system is too complex to be modeled.

The evolution of the economic cost is predicted using an oracle, i.e., a data-
based prediction model of the cost that is learnt from a set of historical
input-cost measurements of past trajectories. The structure of the oracle is a
NARX model, and the prediction is done with a chosen machine learning
technique.

First, it is demonstrated that under mild assumptions on the economic cost
function, an oracle can be obtained for generic nonlinear plants. Then, it is
proven that in nominal conditions, an oracle-based economic MPC with terminal
equality constraint provides the same control law as a state-space prediction
model. Besides, stabilizing design of oracle-based economic MPC using terminal
ingredients is presented. Finally, it has been formally proven that the
resulting control law can stabilize the plant in a real setting, where the
oracle may exhibit estimation error.

These results demonstrate that under mild assumptions on the model of the
plant and on the economic cost function, an oracle of the economic cost
suffices to design economic predictive controllers with the same good
properties of the one based on state-space.

These properties have been demonstrated in an realistic illustrative case
study where a CSTR is regulated with an economic predictive control based on
first principle model and on an appropriate oracle derived using kinky
inference from input-output data of the plant.

## Acknowledgments

The authors would like to thank the funds received by the MINECO Spain and
Feder Funds under contract DPI2016-76493-C3-1-R and Ministerio de Ciencia e
Innovación of Spain under project PID2019-106212RB-C41.

##  Appendix A Existence of a NARX model

Consider a discrete-time system described by

$\displaystyle x(k+1)$ $\displaystyle=$ $\displaystyle f(x(k),u(k))$ (38)
$\displaystyle y(k)$ $\displaystyle=$ $\displaystyle h(x(k)),$ (39)

where $x\in\mathcal{X}\subset{\mathbb{R}}^{n}$,
$u\in\mathcal{U}\subset{\mathbb{R}}^{m}$ and
$y\in\mathcal{Y}\subset{\mathbb{R}}^{p}$, being $\mathcal{X}$, $\mathcal{U}$
and $\mathcal{Y}$ compact sets. The functions $f(\cdot,\cdot)$ and $h(\cdot)$
that define the model are considered to be unknown, and the state not
measurable. The only measures available for this system are the input $u(k)$
and the output $y(k)$.

In this section we summarize the results presented by Levin and Narendra 333In
order to present the Levin and Narendra’s results in a form closer to the
notation used in this paper, we present them taking the sequence of inputs and
outputs as a collection of past measurements, contrary to the original paper
where these were defined as sequence of future values. in [30], on the
existence of an input-output description of the system by means of a nonlinear
autoregresive with exogenous signal (NARX) model, of the form

$y(k+1)=\mathcal{F}(\mathbf{y}_{M}(k),\mathbf{u}_{M}(k)),$ (40)

where

$\displaystyle\mathbf{y}_{M}(k)=(y(k),y(k-1),\cdots,y(k-M))\subset\mathcal{Y}^{M+1}$
(41)
$\displaystyle\mathbf{u}_{M}(k)=(u(k),u(k-1),\cdots,u(k-M))\subset\mathcal{U}^{M+1}.$
(42)

The existence of the NARX model is closely related with the notion of global
observability of the system. Levin and Narendra define the observability map
as444Formally, the sequence $\mathbf{y}_{M}(k)$ depends on
$\mathbf{u}_{M-1}(k-1)$, but this is extended by Levin and Narendra w.l.o.g.
to $\mathbf{u}_{M}(k)$, for the sake of simplicity of the presented results.

$\mathbf{y}_{M}(k)=\mathcal{H}_{M}(x(k-M),\mathbf{u}_{M}(k))$ (43)

Then, a system is said to be observable if for any two distinct states $x_{1}$
and $x_{2}$, there exists an input sequence $\mathbf{u}_{M}$ such that the
corresponding output sequences are distinct, i.e.
$\mathcal{H}_{M}(x_{1},\mathbf{u}_{M})\neq\mathcal{H}_{M}(x_{2},\mathbf{u}_{M})$.
This can be read as the map $\mathcal{H}_{M}(x,\mathbf{u}_{M})$ being
injective in $x$.

If this property holds for any input sequence $\mathbf{u}_{M}$, then the
system is said to be strongly observable, and if it holds for almost every
input sequence $\mathbf{u}_{M}$, then the system is called generically
observable. In this case, the set of input sequences for which this property
does not hold $\mathcal{U}_{no}$ is of measure zero.

######  Assumption 7.

The system given by (38) and (39) satisfies the following conditions:

  1. 1.

The model functions $f$ and $h$ are smooth.

  2. 2.

The system (38) is state invertible, i.e. for a given $u$, $f(x,u)$ defines a
diffeomorphism on $x$.

  3. 3.

The function $h$ is a Morse function, i.e. its critical points are non-
degenerate (see Definition 1 on page 1).

The second condition is naturally met by continuous-time sampled systems
controlled by a discrete-time measurable control law function and a zero-order
holder.

Under this assumption, Levin and Narendra derived Theorem 6 in [30], on the
existence of an input-output model. This Theorem is rewritten here as follows:

######  Theorem 6.

Let a system be defined by (38) and (39) satisfying Assumption 7. Then, for
$M\geq 2n$ and for any $\epsilon>0$, there exists a set of sequences of inputs
$\mathcal{\mathcal{U}}_{M}^{\epsilon}\subseteq\mathcal{U}^{M+1}$ of measure
$\mu(\mathcal{\mathcal{U}}_{M}^{\epsilon})<\epsilon$, such that

  1. 1.

The system defined by (38) and (39) is observable for any sequence
$\mathbf{u}_{M}\not\in\mathcal{U}_{M}^{\epsilon}$.

  2. 2.

There exists a continuous and bijective map $\Phi$ such that for all
$x(k-M)\in\mathcal{X}$ and
$\mathbf{u}_{M}(k)\in\mathcal{\mathcal{U}}^{M+1}\setminus\mathcal{\mathcal{U}}_{M}^{\epsilon}$
we have

$(x(k-M),\mathbf{u}_{M}(k))=\Phi(\mathbf{y}_{M}(k),\mathbf{u}_{M}(k)).$ (44)

  3. 3.

There exists a continuous function $\mathcal{F}(\cdot,\cdot)$ such that for
all input sequences
$\mathbf{u}_{M}(k)\in\mathcal{\mathcal{U}}^{M+1}\setminus\mathcal{\mathcal{U}}_{M}^{\epsilon}$
we have

$\displaystyle y(k+1)$ $\displaystyle=$
$\displaystyle\mathcal{F}(\mathbf{y}_{M}(k),\mathbf{u}_{M}(k)).$ (45)

  4. 4.

There exists a continuous function $\mathcal{\tilde{F}}$ such that for all
input sequences $\mathbf{u}_{M}(k)\in\mathcal{\mathcal{U}}^{M+1}$ we have

$\|\mathcal{F}(\mathbf{y}_{M}(k),\mathbf{u}_{M}(k))-\mathcal{\tilde{F}}(\mathbf{y}_{M}(k),\mathbf{u}_{M}(k))\|<\epsilon.$

###### Proof.

First of all, notice that from [30, Theorems 4], it is derived that the
generic observability property holds for almost all systems satisfying
Assumption 7.

The first statement is proven in the first line of the proof of [30, Theorem
6]. The existence of the map $\Phi$ in the second statement is demonstrated in
[30, Theorem 5]. The existence of $\mathcal{F}$ in the third statement
corresponds to statement 1 of [30, Theorem 6]. The last statement is derived
from the proof of the second part of [30, Theorem 6], for the case in which
the continuous function $\mathcal{\tilde{F}}$ is given by a multilayer
feedforward neural network with a sigmoidal function as activation function of
each neuron. ∎

The following corollary proves that there exists an input-output system such
that its estimation error w.r.t. the real output signal is arbitrarily small
for any sequence of inputs, irrespective if they are observable or not.

######  Corollary 1.

Under the assumptions of Theorem 6, there exists a continuous function
$\mathcal{\tilde{F}}$ such that for all input sequences
$\mathbf{u}_{M}(k)\in\mathcal{\mathcal{U}}^{M+1}$,

$\|y(k+1)-\mathcal{\tilde{F}}(\mathbf{y}_{M}(k),\mathbf{u}_{M}(k))\|<\Theta(\epsilon),$

for a $\mathcal{K}$-function $\Theta(\cdot)$.

###### Proof.

This can be proven as a consequence of Proposition 1 (on page 1), taking
$\mathcal{F}(Y_{M},U_{M})$ as a continuous oracle function which provides a
null estimation error (i.e. $\mu=0$). Therefore, from this Proposition we
infer that

$|y(k+1)-\mathcal{F}(\mathbf{y}_{M}(k),\mathbf{u}_{M}(k)|\leq\Theta(\epsilon).$

∎

Last, the following corollary extends the formulation of Theorem 6 to feed-
through systems, as they are considered in this paper.

######  Corollary 2.

Assume that the output of the system $y(k)$ depends explicitly on $u(k)$, i.e.

$y(k)=h(x(k),u(k)),$ (46)

such that $h$ is Morse in its first argument, i.e., for any critic point
$(x_{a},u_{a})$ of $h$ its Hessian in $x$ is nonsingular. Then, Theorem 6
holds considering in the third claim the following NARX model:

$\displaystyle y(k+1)$ $\displaystyle=$
$\displaystyle\mathcal{F}(\mathbf{y}_{M}(k),\mathbf{u}_{M+1}(k+1)).$ (47)

###### Proof.

Notice that in this case, the considered observability map is valid, making
most of the results immediate. In the proof of Lemma 1 in [30], equation (30)
should be modified accordingly, but the subsequent equation (31) holds true in
virtue of the chain rule, and therefore, the proof of the lemma follows. The
NARX model is derived taking into account that $y(k+1)$ can be written as a
continuous function of the state $x(k-M)$ and the sequence
$\mathbf{u}_{M+1}(k+1)$. ∎

## References

  * [1] F. Dörfler, J. W. Simpson-Porco, F. Bullo, Breaking the hierarchy: Distributed control and economic optimality in microgrids, IEEE Transactions on Control of Network Systems 3 (3) (2015) 241–253.
  * [2] J. Rawlings, D. Angeli, C. N. Bates, Fundamentals of economic model predictive control., in: Proceedings of the CDC, 2012, pp. 3851–3861.
  * [3] M. Heidarinejad, J. Liu, P. D. Christofides, Economic model predictive control of nonlinear process systems using Lyapunov techniques, AIChe Journal 58 (3) (2012) 855–870.
  * [4] M. Ellis, J. Liu, P. D. Christofides, Economic model predictive control, Vol. 2, Springer, 2017.
  * [5] T. Faulwasser, L. Grüne, M. A. Müller, et al., Economic nonlinear model predictive control, Foundations and Trends® in Systems and Control 5 (1) (2018) 1–98.
  * [6] S. Subramanian, S. Lucia, S. Engell, Economic multi-stage output feedback nmpc using the unscented kalman filter, IFAC-PapersOnLine 48 (8) (2015) 38–43.
  * [7] F. A. Bayer, M. A. Müller, F. Allgöwer, Tube-based robust economic model predictive control, Journal of Process Control 8 (24) (2014) 1237–1246.
  * [8] S. Lucia, J. A. Andersson, H. Brandt, M. Diehl, S. Engell, Handling uncertainty in economic nonlinear model predictive control: A comparative case study, Journal of Process Control 24 (8) (2014) 1247–1259.
  * [9] L. Hewing, K. P. Wabersich, M. Menner, M. N. Zeilinger, Learning-based model predictive control: Toward safe learning in control, Annual Review of Control, Robotics, and Autonomous Systems 3 (2020) 269–296.
  * [10] M. Kheradmandi, P. Mhaskar, Data driven economic model predictive control, Mathematics 6 (2018) –.
  * [11] L. Giuliani, H. Durand, Data-based nonlinear model identification in economic model predictive control, Smart and Sustainable Manufacturing Systems 2 (2018) 61–109.
  * [12] Z. Wu, P. D. Christofides, Economic machine-learning-based predictive control of nonlinear systems, Mathematics 7 (2019) –.
  * [13] Z. Wu, P. D. Christofides, Optimizing process economics and operational safety via economic MPC using barrier functions and recurrent neural network models, Chemical Engineering Research and Design 152 (2019) 455–465.
  * [14] S. Gros, M. Zanon, Data-driven economic NMPC using reinforcement learning, IEEE Transactions on Automatic Control 65 (2) (2019) 636–648.
  * [15] A. Chiuso, G. Pillonetto, System identification: A machine learning perspective, Annual Review of Control, Robotics, and Autonomous Systems 2 (2019) 281–304.
  * [16] J. M. Manzano, D. Limon, D. Muñoz de la Peña, J. P. Calliess, Output feedback MPC based on smoothed projected kinky inference, IET Control Theory & Applications.
  * [17] J. Manzano, J. Nadales, D. Muñoz de la Peña, D. Limon, Oracle-based economic predictive control, in: 2019 IEEE 58th Conference on Decision and Control (CDC), IEEE, 2019, pp. 4246–4251.
  * [18] B. Jakubczyk, E. D. Sontag, Controllability of nonlinear discrete-time systems: a lie-algebraic approach, SIAM Journal on Control and Optimization 28 (1) (1990) 1–33.
  * [19] T. Faulwasser, M. Korda, C. N. Jones, D. Bonvin, On turnpike and dissipativity properties of continuous-time optimal control problems, Automatica 81 (2017) 297–304.
  * [20] L. Grüne, M. A. Müller, On the relation between strict dissipativity and turnpike properties, Systems & Control Letters 90 (2016) 45–53.
  * [21] M. A. Müller, D. Angeli, F. Allgöwer, On convergence of averagely constrained economic MPC and necessity of dissipativity for optimal steady-state operation, in: 2013 American Control Conference, IEEE, 2013, pp. 3141–3146.
  * [22] R. Amrit, J. B. Rawligns, D. Angeli, Economic optimization using model predictive control with a terminal cost, Annual Reviews in Control 35 (2011) 178–186.
  * [23] D. Angeli, R. Amrit, J. Rawlings, On average performance and stability of economic model predictive control, IEEE Trans. Aut. Control 57 (2012) 1615–1626.
  * [24] T. Faulwasser, G. Pannocchia, Toward a unifying framework blending real-time optimization and economic model predictive control, Industrial & Engineering Chemistry Research 58 (30) (2019) 13583–13598.  doi:10.1021/acs.iecr.9b00782.
  * [25] L. Grüne, J. Pannek, Nonlinear Model Predictive Control: Theory and Algorithms, 2nd Edition, Communication and Control Engineering; Springer Verlag, 2017.
  * [26] T. Faulwasser, T. Zanon, Asymptotic stability of economic nmpc: The importance of adjoints, in: Proceedings of the 6th IFAC Nonlinear Model Predictive Control Conference - NMPC ’18, 2018.
  * [27] I. Leontaritis, S. A. Billings, Input-output parametric models for non-linear systems part i: deterministic non-linear systems, International journal of control 41 (2) (1985) 303–328.
  * [28] E. D. Sontag, On the observability of polynomial systems, I: Finite-time problems, SIAM Journal on Control and Optimization 17 (1) (1979) 139–151.
  * [29] S. Chen, S. A. Billings, Representations of non-linear systems: the NARMAX model, International Journal of Control 49 (3) (1989) 1013–1032.
  * [30] A. Levin, K. Narendra, Identification of nonlinear dynamical systems using neural networks, in: Neural systems for control, Elsevier, 1997, pp. 129–160.
  * [31] H. Edelsbrunner, J. Harer, Computational topology: an introduction, American Mathematical Soc., 2010.
  * [32] J. Roll, A. Nazin, L. Ljung, A general direct weight optimization framework for nonlinear system identification, Linköping University Electronic Press, 2005\.
  * [33] C. E. Rasmussen, C. K. Williams, Gaussian process for machine learning, MIT press, 2006.
  * [34] J.-P. Calliess, Conservative decision-making and inference in uncertain dynamical systems, Ph.D. thesis, University of Oxford (2014).
  * [35] G. Grimm, M. J. Messina, S. Z. Tuna, A. R. Teel, Examples when nonlinear model predictive control is nonrobust, Automatica 40 (2004) 1729–1738.
  * [36] J. B. Rawlings, D. Q. Mayne, M. Diehl, Model Predictive Control: Theory, Computation and Design, 2nd Edition, Nob-Hill Publishing, 2017.
  * [37] J. B. Rawlings, D. Q. Mayne, M. Diehl, Preface of the book Model Predictive Control: Theory, Computation and Design, 2nd Edition, Nob-Hill Publishing, 2017\.
  * [38] M. Canale, L. Fagiano, M. C. Signorile, Nonlinear model predictive control from data: a set membership approach, International Journal of Robust and Nonlinear Control 24 (1) (2014) 123–139.
  * [39] D. Limon, T. Alamo, D. M. Raimondo, D. Muñoz de la Peña, J. M. Bravo, E. F. Camacho, Input-to-state stability: an unifying framework for robust model predictive control, in: Proceedings of the Int. Workshop on Assessment and Future Directions of Nonlinear Model Predictive Control (NMPC08). 5-9 Sept. Pavia, Italy., 2008.
  * [40] D. Limon, T. Alamo, F. Salas, E. F. Camacho, On the stability of MPC without terminal constraint, IEEE Transactions on Automatic Control 42 (2006) 832–836.
  * [41] D. Limon, A. Ferramosca, I. Alvarado, T. Alamo, Nonlinear MPC for tracking piece-wise constant reference signals, IEEE Transactions on Automatic Control.
  * [42] J. Manzano, D. Limon, D. Muñoz de la Peña, J. Calliess, Online learning constrained MPC based on double prediction, International Journal of Robust and Nonlinear Control (Accepted for publication) (2020) –.
  * [43] J. M. Manzano, D. Limon, D. Muñoz de la Peña, J. P. Calliess, Robust learning-based MPC for nonlinear constrained systems, Automatica 117 (2020) 108948\.
  * [44] J.-P. Calliess, S. J. Roberts, C. E. Rasmussen, J. Maciejowski, Lazily adapted constant kinky inference for nonparametric regression and model-reference adaptive control, Automatica 122 (2020) 109216.