# Learning Robust Data-based LQG Controllers from Noisy Data

Wenjie Liu, Jian Sun, Gang Wang, Francesco Bullo, , and Jie Chen The work was
supported in part by the National Natural Science Foundation of China under
Grants 61925303, 62173034, 62088101, the BIT Research and Innovation Promoting
Project under Grant 2022YCXZ007, and the Chongqing Natural Science Foundation
under Grant 2021ZX4100027. W. Liu, J. Sun, and G. Wang are with the National
Key Lab of Autonomous Intelligent Unmanned Systems and School of Automation,
Beijing Institute of Technology, Beijing 100081, China, and also with the
Beijing Institute of Technology Chongqing Innovation Center, Chongqing 401120,
China (e-mail<EMAIL_ADDRESS><EMAIL_ADDRESS>gangwang@bit.edu.cn).
F. Bullo is with the Mechanical Engineering Department and the Center of
Control, Dynamical Systems and Computation, UC Santa Barbara, CA 93106-5070,
USA (e-mail: bullo@ucsb.edu). J. Chen is with the Department of Control
Science and Engineering, Tongji University, Shanghai 201804, China, and also
with the National Key Lab of Autonomous Intelligent Unmanned Systems, Beijing
Institute of Technology, Beijing 100081, China (e-mail: chenjie@bit.edu.cn).

###### Abstract

This paper addresses the joint state estimation and control problems for
unknown linear time-invariant systems subject to both process and measurement
noise. The aim is to redesign the linear quadratic Gaussian (LQG) controller
based solely on data. The LQG controller comprises a linear quadratic
regulator (LQR) and a steady-state Kalman observer; while the data-based LQR
design problem has been previously studied, constructing the Kalman gain and
the LQG controller from noisy data presents a novel challenge.

In this work, a data-based formulation for computing the steady-state Kalman
gain is proposed based on semi-definite programming (SDP) using some noise-
free input-state-output data. Additionally, a data-based LQG controller is
developed, which is shown to be equivalent to the model-based LQG controller.
For cases where offline data are corrupted by noise, a robust data-based
observer gain is constructed by tackling a relaxed SDP. The proposed
controllers are proven to achieve robust global exponential stability (RGES)
for state estimation and input-to-state practical stability (ISpS) under
standard conditions. Finally, numerical tests are conducted to validate the
proposed controllers’ correctness and effectiveness.

###### Index Terms:

Data-driven control, state estimation, noisy data, semi-definite program,
linear quadratic Gaussian.

##  I Introduction

The states of a dynamical system refer to a set of variables that provide a
complete characterization of its internal conditions or status at a given time
[35], which plays a fundamental role in control, monitoring, and high-level
information extraction of real-world systems. Unfortunately, the complete
states are not always fully observable; and even if measurements can be
collected, the inevitable sensor miscalibration and noise may degrade the
accuracy. As such, using the most likely value of a state reconstructed from
raw measurements serves as an alternative, that is what is known as state
estimation. Numerous efforts have contributed to the development of state
estimation methods and theory as well as their applications in networked
control systems, cyber-physical systems, and smart power grids; see e.g., [9,
33, 39, 10].

Most existing results on state estimation capitalize on explicit system models
describing the underlying dynamics. However, in many situations e.g., robotics
[8], biology [25], or human-in-the-loop systems [21], modeling using first-
principles can be challenging or parameters obtained by system identification
methods may be inaccurate or unreliable. On the contrary, data can be often
collected through open-loop experiments. This approach has inspired the recent
literature on data-driven control [20, 14, 32, 28], i.e., on designing
controllers directly from data. In the literature, most related results were
derived based on the _Willems et al._ ’s fundamental lemma [43]. Rather than
capitalizing on an explicit system model, the lemma demonstrates that any
finite-time trajectory of a linear time-invariant system can be expressed as a
linear combination of some past trajectories generated by the system, if the
input signal is sufficiently rich. This offers an effective means for
describing linear systems without explicit models. Although this lemma has led
to diverse data-driven control schemes, e.g., model predictive control [6, 12,
27, 31, 4, 29, 7] and explicit feedback control policies [14, 38, 26, 24, 40,
41], very few works have investigated its application in state estimation.

In the context of data-driven state estimation, it is often assumed that
input-state-output data can be collected offline while only input-output data
are obtained during online operation. For noise-free systems, data-driven
Luenberger observers were designed in [38, 1] by exploiting the separation
principle of estimation and control [16] together with the fundamental lemma.
To handle unknown inputs or noise, unknown input observer (UIO) in [36],
moving horizon estimation (MHE) in [44], and set-based estimation in [3] were
developed. In addition to bounding conditions, [36] posed a more stringent
condition on the noise dimension. Although [44, 3] only constrain the noise
bound, an optimization problem should be solved online at each time, rendering
them less computationally appealing for time-critical applications. It is
worth emphasizing that, to estimate the state, these approaches require
collecting the input data on-the-fly, as they do not tackle the controller
design problem.

The goal of this paper is to address robust data-based state estimation and
control problems for unknown linear time-invariant systems subject to noise.
The idea is using input-state-output data collected offline to redesign the
linear quadratic Gaussian (LQG) controller that returns optimal state
estimates and control inputs simultaneously under certain conditions. To
fulfill this design, robust data-based formulations for the linear quadratic
regulator (LQR) and steady-state Kalman observer are required. The problem of
learning LQR from noisy data has been dealt with in [15]. Our focus is thus
placed on formulating the robust data-based Kalman observer and LQG
controller. We consider two settings depending on whether pre-collected data
are noise-corrupted or not. When noise-free data are collected offline, we
propose a data-based SDP for computing the steady-state Kalman gain. It is
shown that the proposed data-based LQG controller is equivalent to its model-
based counterpart provided the online noise is additive white Gaussian. When
offline data are corrupted by noise, we relax the original data-based SDP by
adding a soft constraint to restore its feasibility as well as ensure the
stability of the resulting Kalman gain. In this setting, although the
controller does not share equivalence with the model-based LQG controller, it
is robust to bounded noise. Finally, we establish, under rigorous conditions,
robust global exponential stability (RGES) for the proposed estimator and
input-to-state practical stability (ISpS) for the system in both cases.

A few setups similar to ours appeared very recently in [5, 2, 17, 42]. To be
specific, de-noising methods for offline data were investigated in [5, 42], to
improve the performance of the data-driven predictive controller of [12].
Precisely, the work of [5] designed a data-based extended Kalman filter for
reducing the effect of noise in outputs, and a low-rank subspace
identification noise estimation algorithm was developed in [42]. By expressing
a linear system equivalently in the behavior space, finding the LQG controller
reduces to finding a static feedback controller which can be obtained using
data [2]. In [17], a near-optimal safe output feedback controller was obtained
using noisy data, whose suboptimality with respect to the model-based LQG
controller was analyzed. It is worth stressing the differences between these
works and ours. First, the results [5, 2, 17, 42] have only considered the
data-driven control problem, whereas data-driven state estimation is
investigated too in the present work. Moreover, [2] requires the pre-collected
data (a.k.a. expert data) be generated from the system that has been
stabilized by an LQG controller, which is stringent and unrealistic. The work
of [17] can only handle input and measurement noise but not the process noise.

In a nutshell, the contributions of this paper are as follows:

  * c1)

When noise-free data are collected, the data-based LQG controller is designed
by solving data-based SDPs, which is shown equivalent to the model-based LQG
controller;

  * c2)

To deal with noise in offline data, robust data-based SDPs are derived by
incorporating a soft constraint to recover the SDP feasibility and stability
of the Kalman gain; and,

  * c3)

For both noise-free and noisy data-based controllers, it is shown that the
estimator is RGES and the closed-loop system is ISpS under standard
conditions.

_Notation:_ Denote the set of real numbers, integers, and positive integers by
$\mathbb{R}$, $\mathbb{N}$, and $\mathbb{N}_{+}$, respectively. For a full
row-rank matrix $M$, its right pseudo-inverse is denoted by $M^{\dagger}$. Let
$\|\cdot\|$ denote the vector Euclidean norm or matrix spectral norm. Given a
measurable time function $f:\mathbb{N}\rightarrow\mathbb{R}^{n}$ and a time
interval $[0,t)$, its $\mathcal{L}_{\infty}$ norm on $[0,t)$ is
$\|f_{t}\|_{\infty}:={\rm ess}\sup_{s\in[0,t)}\|f(s)\|$. For matrices $A$,
$B$, and $C$ with compatible dimensions, we abbreviate $ABC(AB)^{\prime}$ to
$AB\cdot C[\star]^{\prime}$; $\lambda(A)$ represents the collection of all
singular values (or eigenvalues if $A$ is square); $\underline{\lambda}_{A}$
($\bar{\lambda}_{A}$) the minimum (maximum) singular value of $A$. Let
$x_{[t_{1},t_{2}]}:=[x(t_{1})~{}x(t_{1}+1)\,\cdots\,x(t_{2})]$ denote a
stacked window of the signal $x$ in time interval $[t_{1},t_{2}]$.

The Hankel matrix associated with the sequence $\\{x(t)\\}_{t=0}^{N-1}$, is
denoted by

$H_{L}(x_{[0,N-1]}):=\left[\begin{matrix}x(0)&x(1)&\ldots&x(N-L)\\\
\vdots&\vdots&\ddots&\vdots\\\ x(L-1)&x(L)&\ldots&x(N-1)\end{matrix}\right].$

The definition of persistency of excitation is given below.

######  Definition 1 (Persistency of excitation).

A sequence $\\{x(t)\\}_{t=0}^{T-1}$ with $x(t)\in\mathbb{R}^{n_{x}}$ and
$T\geq(n+1)L+1$ is persistently exciting of order $L$ if
${\rm{rank}}(H_{L}(x_{[0,T-1]}))=n_{x}L$.

##  II Preliminaries and Problem Formulation

This section introduces the problem of interest and some relevant results in
the literature.

###  II-A Problem formulation

Consider the following linear system

$\displaystyle x(t+1)$ $\displaystyle=Ax(t)+Bu(t)+w(t)$ (1a) $\displaystyle
y(t)$ $\displaystyle=Cx(t)+v(t)$ (1b)

where $x(t)\in\mathbb{R}^{n_{x}}$ is the state , $u(t)\in\mathbb{R}^{n_{u}}$
the control input, $y(t)\in\mathbb{R}^{n_{y}}$ the output,
$w(t)\in\mathbb{R}^{n_{x}}$ the process noise, and $v(t)\in\mathbb{R}^{n_{y}}$
the measurement noise. Throughout this paper, the following assumptions hold.

######  Assumption 1 (Controllability and observability).

The pair $(A,B)$ is controllable and the pair $(A,C)$ is observable.

######  Assumption 2 (System trajectory).

The matrices $(A,B,C)$ in (1) are unknown. Input-state-output trajectories,
i.e., $x_{[0,T]}$, $u_{[0,T-1]}$ and $y_{[0,T-1]}$ of system (1) can be
collected during offline experiments, and the input sequence $u_{[0,T]}$ is
persistently exciting of order $n_{x}+1$. During online operation, only input-
output data, i.e., $u(t)$ and $y(t)$, are available.

######  Assumption 3 (Bounded noise).

For all $t\in\mathbb{N}$, there exist non-negative constants $\bar{w}$ and
$\bar{v}$ such that $w(t)\in\mathbb{B}_{\bar{w}}:=\\{w|\|w\|\leq\bar{w}\\}$
and $v(t)\in\mathbb{B}_{\bar{v}}:=\\{v|\|v\|\leq\bar{v}\\}$.

######  Remark 1.

Assumption 2 can be fulfilled in a remote networked estimation and control
scenario. Specifically, if the state is of large size (e.g., video),
transmitting via a network can be time-consuming and costly for real-time
applications. In addition, for states containing sensitive information,
eavesdropping or damage may happen during network transmission. As such,
transmitting outputs and recovering the state afterwards appears an appealing
alternative. On the other hand, state can be collected in offline experiments
and transmitted only once to the controller using possibly a different
physical medium. This assumption has appeared in recent data-driven state
estimation results [36, 44]. For the data-driven set-based estimation method
in [3], matrix $C$ was assumed known, which is similar to the assumption of
having partial knowledge of the states during online inference. Instead of
Assumption 3, the results of [36, 44] impose additional constraints on the
noise. In [36], it is required that the dimensions of noise vectors $w(t)$ and
$v(t)$ should not exceed the dimension of the output; see [37, Proposition 1].
In [44], no process noise is considered. Instead of (1a), they deal with
noise-corrupted versions of the state, i.e., $\tilde{x}(t)=x(t)+w(t)$ from the
noise-free system $x(t+1)=Ax(t)+Bu(t)$. In comparison, system (1a) with
process noise is more natural and challenging as discussed in [44, Remark 5].

Existing results on data-driven state estimation have focused on autonomous
systems or systems with known inputs. For open-loop unstable systems, a
stabilizing controller should be separately designed and equipped which
complicates the architecture and implementation. To bypass this drawback, we
are interested in addressing the joint state estimation and control problems
of system (1) under Assumptions 1—3. To this end, some standard definitions on
the desired performance are given. Dealing with noisy data, rather than
exponential stability, we consider RGES and ISpS. The former is extended from
[22, Definition 9] and has been considered in [44].

######  Definition 2 (RGES of an estimator).

Consider system (1) with bounded noise $w(t)\in\mathbb{B}_{\bar{w}}$ and
$v(t)\in\mathbb{B}_{\bar{v}}$. A state estimator $\hat{x}(t)$ is RGES if there
exist functions $\phi\in\mathcal{KL}$ and $\pi_{w},\pi_{v}\in\mathcal{K}$111 A
function $\pi:[0,\infty)\rightarrow[0,\infty)$ is a $\mathcal{K}$-function if
it is continuous and strictly increasing satisfying $\pi(0)=0$. A function
$\phi:[0,\infty)\times[0,\infty)\rightarrow[0,\infty)$ is a
$\mathcal{KL}$-function if $\phi(\cdot,t)$ is of class $\mathcal{K}$ for each
$t\in\mathbb{N}$ and $\phi(s,t)$ decreases to $0$ as $t\rightarrow\infty$ for
any $s\in\mathbb{N}$. such that for any $x(0),\hat{x}(0)\in\mathbb{R}^{n_{x}}$
and $w(t)\in\mathbb{B}_{\bar{v}}$, $v(t)\in\mathbb{B}_{\bar{v}}$, the
following is satisfied for all $t\in\mathbb{N}$

$\displaystyle\|x(t)-\hat{x}(t)\|$
$\displaystyle\leq\phi(\|x(0)-\hat{x}(0)\|,t)$
$\displaystyle\quad+\pi_{w}(\|w_{t}\|_{\infty})+\pi_{v}(\|v_{t}\|_{\infty}).$
(2)

The following definition of ISpS for discrete-time systems is adapted from
[23, Definition 2.2].

######  Definition 3 (ISpS).

A control system $x(t+1)=f(x(t),w(t))$ is called input-to-state practically
stable (ISpS) if there exist $\mathcal{KL}$-function $\phi$,
$\mathcal{K}$-function $\pi_{w}$, and constant $D_{0}$, such that for each
$x(0)\in\mathbb{R}^{n_{x}}$ and each measurable essentially bounded input
$w(t)$ on $[0,+\infty)$, the solution $x(t)$ satisfies

$\|x(t)\|\leq\phi(\|x(0)\|,t)+\pi_{w}(\|w_{t}\|_{\infty})+D_{0},\quad\forall
t\in\mathbb{N}.$

In this setting, we consider the following problem.

######  Problem 1.

Consider the system (1) under Assumptions 1—3. Design a data-based state
estimator $\hat{x}(t)$ achieving RGES and a data-driven controller $u(t)$
ensuring system (1) ISpS.

###  II-B Model-based SDP for LQG control

When matrices $A$, $B$ and $C$ in (1) are known, Problem 1 can be addressed by
the LQG controller. In the following, we briefly review its model-based
design, which plays an important role in deriving the main results of this
paper.

Under Assumption 1, a dynamic measurement feedback controller stabilizing
system (1) exists and is given by [30]

$\displaystyle\hat{x}(t+1)$ $\displaystyle=(A+BK-LC)\hat{x}(t)+Ly(t)$ (3a)
$\displaystyle u(t)$ $\displaystyle=K\hat{x}(t)$ (3b)

where matrices $K$ and $L$ are chosen such that $A+BK$ and $A-LC$ are Schur
stable. The LQG controller which provides optimal control inputs while
returning optimal state estimates is a special case of the controller (3),
when i) $w(t),v(t)$ are mutually independent additive white Gaussian noise
(AWGN) processes obeying $\mathbb{E}[w(t)w^{\prime}(t)]=N_{x}$ and
$\mathbb{E}[v(t)v^{\prime}(t)]=N_{y}$ for all $t\in\mathbb{N}$, and ii) in
addition to Schur stable, the gain matrices $K$ and $L$ are designed such that
the quadratic cost is minimized.

Precisely, considering weight matrices $W_{x}>0$ and $W_{u}>0$, matrix $K$ is
chosen to minimize the cost function
$J=\lim_{T\rightarrow\infty}\mathbb{E}[(1/T)[\sum_{t=0}^{T-1}x(t)W_{x}x^{\prime}(t)+u(t)W_{u}u^{\prime}(t)]$,
which is the classical stochastic LQR problem. This has been shown in [11,
Section 6.4] equivalent to the deterministic $\mathcal{H}_{2}$ problem, i.e.,
finding $K$ such that $A+BK$ is Schur stable and ${\rm tr}(W_{x}P)+{\rm
tr}(W_{u}KPK^{\prime})$ is minimized where $P$ is the controllability Gramian
of system (1a). Such a controller gain matrix $K$ is unique and can be found
as follows

$\bar{K}=-(W_{u}+B^{\prime}PB)^{-1}B^{\prime}PA$ (4)

where $P>0$ is the unique solution to the following discrete-time algebraic
Riccati (DARE) equation

$A^{\prime}PA-P-A^{\prime}PB(W_{u}+B^{\prime}PB)^{-1}B^{\prime}PA+W_{x}=0.$
(5)

It is shown in [15, Section 3.2] that the problem of finding $\bar{K}$ in (4)
can be equivalently posed as a semi-definite program

$\displaystyle\min_{\gamma,K,P,G}\gamma$ (6) $\displaystyle~{}~{}~{}{\rm
s.t.}~{}\begin{cases}(A+BK)P(A+BK)^{\prime}-P+I\leq 0\\\ P\geq I\\\
G-KPK^{\prime}\geq 0\\\ {\rm tr}(W_{x}P)+{\rm
tr}(W_{u}G)\leq\gamma.\end{cases}$

The matrix $L$ in the LQG controller is called the steady-state Kalman gain
and is designed as follows. Let $e(t)=x(t)-\hat{x}(t)$ denote the estimation
error evolving as follows

$e(t+1)=(A-LC)e(t)+w(t)-Lv(t)$ (7)

and $L$ is chosen to minimize the trace of estimation error covariance matrix
$\Sigma=\lim_{T\rightarrow\infty}\mathbb{E}[(1/T)\sum_{t=0}^{T-1}e(t)e^{\prime}(t)]$,
which is the classical Kalman filtering problem. According to [18, Section
7.3], the steady-state Kalman gain can be obtained as follows

$\bar{L}=A\Sigma C^{\prime}(C\Sigma C^{\prime}+N_{y})^{-1}$ (8)

where $\Sigma$ is the unique positive definite solution to the DARE equation

$A\Sigma A^{\prime}-\Sigma+N_{x}-A\Sigma C^{\prime}(C\Sigma
C^{\prime}+N_{y})^{-1}C\Sigma A^{\prime}=0.$ (9)

It can be seen that, upon replacing $(A,B,W_{x},W_{u},K)$ by
$(A^{\prime},C^{\prime},N_{x},N_{y},L^{\prime})$, the DARE (9) boils down to
(5). Therefore, the Kalman filtering problem can be formulated as the dual of
LQR, i.e., finding $L$ such that ${\rm tr}(N_{x}\Sigma)+{\rm
tr}(N_{y}L^{\prime}\Sigma L)$ is minimized and $A-LC$ is Schur stable, where
$\Sigma$ is the observability Gramian of (1). Hence, the problem of finding
$\bar{L}$ in (8) can be equivalently formulated as the following SDP

$\displaystyle\min_{\epsilon,L,\Sigma,\Upsilon}\epsilon$ (10)
$\displaystyle~{}~{}~{}{\rm
s.t.}~{}\begin{cases}(A-LC)^{\prime}\Sigma(A-LC)-\Sigma+I\leq 0\\\ \Sigma\geq
I\\\ \Upsilon-L^{\prime}\Sigma L\geq 0\\\ {\rm tr}(N_{x}\Sigma)+{\rm
tr}(N_{y}\Upsilon)\leq\epsilon\end{cases}$

which can be seen as the discrete-time counterpart of the Kalman filtering
formulation for continuous-time systems in [34, Section 156.6].

In words, the model-based LQG controller is expressed by

$\displaystyle\hat{x}(t+1)$
$\displaystyle=(A+B\bar{K}-\bar{L}C)\hat{x}(t)+\bar{L}y(t)$ (11a)
$\displaystyle u(t)$ $\displaystyle=\bar{K}\hat{x}(t)$ (11b)

where the LQR controller gain $\bar{K}$ is given in (4) and the steady-state
Kalman gain $\bar{L}$ given in (8). The problem of finding the gain matrices
can be converted into two deterministic $\mathcal{H}_{2}$ problems and
addressed by solving SDPs (6) and (10). Therefore, if we can redesign the
controller (11) with matrices $\bar{K}$ and $\bar{L}$ by only using data, then
Problem 1 is solved.

###  II-C Relevant data-driven results

Data-based versions of the formulas above have been reported in the
literature. For example, [38, Section V] provided a data-based framework for
representation and control of linear systems using dynamic output measurement
feedback (3). Data-driven LQR problem equivalent to (6) was formulated in
[15]. A brief review of related results is given as follows.

Consider the $T$-long data $x_{[0,T-1]}$, $x_{[1,T]}$, $u_{[0,T-1]}$,
$w_{[0,T-1]}$, $y_{[0,T-1]}$, $v_{[0,T-1]}$ obtained from an open-loop
experiment on system (1). Rearrange these data into matrices, yielding

$\displaystyle X_{0}=[x(0)~{}x(1)~{}\cdots~{}x(T-1)]$ (12a) $\displaystyle
X_{1}=[x(1)~{}x(2)~{}\cdots~{}x(T)]$ (12b) $\displaystyle
U_{0}=[u(0)~{}u(1)~{}\cdots~{}u(T-1)]$ (12c) $\displaystyle
W_{0}=[w(0)~{}w(1)~{}\cdots~{}w(T-1)]$ (12d) $\displaystyle
Y_{0}=[y(0)~{}y(1)~{}\cdots~{}y(T-1)]$ (12e) $\displaystyle
V_{0}=[v(0)~{}v(1)~{}\cdots~{}v(T-1)]$ (12f)

which satisfy

$\left[\begin{matrix}X_{1}\\\
Y_{0}\end{matrix}\right]=\left[\begin{matrix}A&B\\\
C&0\end{matrix}\right]\left[\begin{matrix}X_{0}\\\
U_{0}\end{matrix}\right]+\left[\begin{matrix}W_{0}\\\
V_{0}\end{matrix}\right].$ (13)

When considering the noise-free case, i.e., $W_{0}=0$ and $V_{0}=0$, the
equation reduces to

$\left[\begin{matrix}X_{1}\\\
Y_{0}\end{matrix}\right]=\left[\begin{matrix}A&B\\\
C&0\end{matrix}\right]\left[\begin{matrix}X_{0}\\\ U_{0}\end{matrix}\right].$
(14)

Let $\Phi_{0}:=[X_{0}^{\prime}~{}U_{0}^{\prime}]^{\prime}$. A data-based
version of controller (3) is constructed in [38], which we outline as follows.

######  Theorem 1 ([38, Theorem 34]).

Consider data $X_{1}$, $X_{0}$, $U_{0}$, $Y_{0}$ generated from system (1)
with noise $w(t)=0$ and $v(t)=0$, $\forall t\in\mathbb{N}$. Under Assumption
1, there exist matrices $\Phi_{1}^{{\dagger}}$, $\Phi_{2}^{{\dagger}}$, $K$,
$L$ such that
$\Phi_{0}[\Phi_{1}^{{\dagger}}~{}\Phi_{2}^{{\dagger}}]=\Phi_{0}\Phi_{0}^{{\dagger}}=I$,
and both $X_{1}(\Phi_{1}^{{\dagger}}+\Phi_{2}^{{\dagger}}K)$ and
$(X_{1}-LY_{0})\Phi_{1}^{{\dagger}}$ are Schur stable if and only if the
following holds

${\rm rank}(\Phi_{0})=n_{x}+n_{u}.$ (15)

A stabilizing measurement feedback controller of the form (3) is given by

$\displaystyle\hat{x}(t\\!+\\!1)$
$\displaystyle\\!=\\!(X_{1}\Phi_{1}^{\dagger}\\!+\\!X_{1}\Phi_{2}^{\dagger}K\\!-\\!LY_{0}\Phi_{1}^{\dagger})\hat{x}(t)\\!+\\!Ly(t)$
(16a) $\displaystyle u(t)$ $\displaystyle=K\hat{x}(t).$ (16b)

The condition (15) is critical for guaranteeing existence of the controller
(16). When there is no noise in input-state data, i.e., $W_{0}=0$, condition
(15) can be met by using a sufficiently exciting input sequence $u_{[0,T-1]}$.

######  Lemma 1 ([43, Corollary 2]).

For controllable system (1) with $w(t)=0$ for all $t\in\mathbb{N}$, if
$u_{[0,T-1]}$ is persistently exciting of order $n_{x}+1$, then condition (15)
holds.

Having formulated the data-based version of controller (3), it can be observed
that if the gain matrices $\bar{K}$ in (4) and $\bar{L}$ in $\eqref{eq:lqg:L}$
can also be derived using data, then a data-based LQG controller is
constructed.

Previous works have addressed the data-based LQR problem in both noise-free
data [14] and noisy data [15] scenarios. Specifically, if $W_{0}=0$, SDP (6)
can be equivalently reformulated using data.

######  Theorem 2 ([15, Theorem 1]).

Let $U_{0}$, $X_{0}$ and $X_{1}$ be data collected from an experiment on
system (1) with $W_{0}=0$ using a persistently exciting input sequence of
order $n_{x}+1$. Then the condition (15) holds and the SDP

$\displaystyle\min_{\gamma,Q,P,G}\gamma$ (17) $\displaystyle~{}~{}~{}{\rm
s.t.}~{}\begin{cases}X_{1}QP^{-1}Q^{\prime}X_{1}^{\prime}-P+I\leq 0\\\ P\geq
I\\\ G-U_{0}QP^{-1}Q^{\prime}U_{0}^{\prime}\geq 0\\\ X_{0}Q=P\\\ {\rm
tr}(W_{x}P)+{\rm tr}(W_{u}G)\leq\gamma\end{cases}$

is feasible. Moreover, let $(\gamma_{*},Q_{*},P_{*},G_{*})$ be any optimal
solution of (17), and $K_{*}=U_{0}Q_{*}P_{*}^{-1}$ recovers the LQR controller
gain in (4), i.e., $\bar{K}=K_{*}$.

When $W_{0}\neq 0$, Lemma 1 does not hold in general and both the feasibility
of SDP (17) and the optimality of the controller gain are deteriorated. To
mitigate this issue, a modified SDP was proposed in [15, Section 5], which
introduces a soft constraint to balance between robustness against noise and
optimality.

######  Theorem 3 ([15, Section 5]).

Let $U_{0}$, $X_{0}$, $X_{1}$ and $W_{0}$ be data collected from an experiment
on system (1) using a persistently exciting input sequence of order $n_{x}+1$.
Consider the SDP

$\displaystyle\min_{\gamma,Q,P,G,M_{1}}\gamma$ (18)
$\displaystyle~{}~{}~{}~{}~{}{\rm
s.t.}~{}~{}~{}\begin{cases}X_{1}QP^{-1}Q^{\prime}X_{1}^{\prime}-P+I\leq 0\\\
P\geq I\\\ G-U_{0}QP^{-1}Q^{\prime}U_{0}^{\prime}\geq 0\\\
M_{1}-QP^{-1}Q^{\prime}\geq 0\\\ X_{0}Q=P\\\ {\rm tr}(W_{x}P)+{\rm
tr}(W_{u}G)+\alpha_{1}{\rm tr}(M_{1})\leq\gamma\end{cases}$

where $\alpha_{1}>0$ is arbitrary. Then there exists some $\delta_{r}>0$ such
that if $\|W_{0}\|\leq\delta_{r}$, then the condition (15) is satisfied and
SDP (18) is feasible. In addition, letting
$(\tilde{\gamma},\tilde{Q},\tilde{P},\tilde{G},\tilde{M}_{1})$ be any optimal
solution of SDP (18), then $u(k)=\tilde{K}x(k)$ with
$\tilde{K}=U_{0}\tilde{Q}\tilde{P}^{-1}$ is stabilizing.

Bearing in mind the results in Subsections II-B and II-C, Problem 1 can be
rephrased as follows.

######  Problem 2.

For system (1) obeying Assumptions 1—3, design a data-based observer and
controller to fulfill the LQG controller (11), which achieves RGES for the
data-driven state estimator and ISpS for the system.

We will focus on Problem 2 in the rest of the paper.

##  III Data-based LQG Control Using Noise-free Data

In this section, we address Problem 2 by considering that noise-free input-
state-output data can be collected offline.

######  Assumption 4 (Noise-free offline data).

Let $X_{1}$, $X_{0}$, $U_{0}$ and $Y_{0}$ be data collected from an offline
experiment on system (1) in the absence of noise, i.e., $W_{0}=0$ and
$V_{0}=0$.

Under this assumption and invoking Theorem 2, the matrix $\bar{K}$ in (4) can
be obtained by resorting to the optimal solution of SDP (17). Building on this
result, a possible solution for Problem 2 reduces to deriving a data-based
version of $\bar{L}$ in (8). Thus, this section proceeds by first constructing
a steady-state Kalman gain $\bar{L}$ based on data. Substituting the data-
based formulations for designing the gain matrices $\bar{K}$ and $\bar{L}$
into (16), we show that (16) is equivalent to (11). Finally, theoretical
guarantees on RGES and ISpS are provided.

Noticing from Assumption 2, input sequence $u_{[0,T-1]}$ is persistently
exciting of order $n_{x}+1$. According to Lemma 1, the condition (15) holds
and there exists matrix
$\Phi_{0}^{{\dagger}}:=[\Phi_{1}^{\dagger}~{}\Phi_{2}^{\dagger}]$ such that
$\Phi_{0}\Phi_{0}^{{\dagger}}=I$. In addition, under Assumption 4, matrices
$X_{1}$, $X_{0}$, $U_{0}$ and $Y_{0}$ adhere to (14). Hence, post-multiplying
both sides of (14) by $\Phi_{0}^{{\dagger}}$ returns

$A=X_{1}\Phi_{1}^{\dagger},~{}B=X_{1}\Phi_{2}^{\dagger},~{}C=Y_{0}\Phi_{1}^{\dagger}.$
(19)

Substituting (19) into (10), we arrive at the following SDP

$\displaystyle\min_{\epsilon,\Pi,\Sigma,\Upsilon}\epsilon$ (20)
$\displaystyle~{}~{}~{}{\rm
s.t.}~{}\begin{cases}\Phi_{1}^{{\dagger}\prime}(\Sigma X_{1}+\Pi
Y_{0})^{\prime}\Sigma^{-1}(\Sigma X_{1}+\Pi
Y_{0})\Phi_{1}^{\dagger}-\Sigma+I\leq 0\\\ \Sigma\geq I\\\
\Upsilon-\Pi^{\prime}\Sigma^{-1}\Pi\geq 0\\\ {\rm tr}(N_{x}\Sigma)+{\rm
tr}(N_{y}\Upsilon)\leq\epsilon.\end{cases}$

The equivalence between SDPs (10) and (20) is formally stated as follows.

######  Theorem 4 (Equivalence between (10) and (20)).

Under Assumptions 1, 2 and 4, SDP (20) is feasible. Moreover, any optimal
solution $(\epsilon_{*},\Pi_{*},\Sigma_{*},\Upsilon_{*})$ of (20) is such that
$L_{*}=\Sigma_{*}^{-1}\Pi_{*}$ is equivalent to the optimal solution $\bar{L}$
obtained from (10).

###### Proof.

It follows from Assumption 2 and Lemma 1 that condition (15) holds, and hence
there exist matrices $\Phi_{1}^{\dagger}$, $\Phi_{2}^{\dagger}$ such that
$\Phi_{0}[\Phi_{1}^{\dagger}\Phi_{2}^{\dagger}]=I$ and SDP (10) is feasible.
Let $(\bar{\epsilon},\bar{L},\bar{\Sigma},\bar{\Upsilon})$ be an optimal
solution of (10) and $(\epsilon_{*},\Pi_{*},\Sigma_{*},\Upsilon_{*})$ be an
optimal solution of (20). The key idea of the proof is to show that a
candidate solution of (10) can be constructed by any optimal solution of (20),
and vice versa. Let
$(\epsilon,L,\Sigma,\Upsilon):=(\epsilon_{*},\Sigma_{*}^{-1}\Pi_{*},\Sigma_{*}\Upsilon_{*})$.
It follows from (14) that $X_{1}\Phi_{1}^{{\dagger}}=A$ and
$Y_{0}\Phi_{1}^{{\dagger}}=C$, hence we have that
$X_{1}\Phi_{1}^{{\dagger}}-\Sigma^{-1}\Pi Y_{0}\Phi_{1}^{{\dagger}}=A-LC$.
This implies that $(L,\Sigma,\Upsilon)$ satisfies the first three constraints
in (10). In addition, noticing that $\epsilon=\epsilon_{*}={\rm
tr}(N_{x}\Sigma_{*})+{\rm tr}(N_{y}\Upsilon_{*})$, one has that
$(\epsilon_{*},\Sigma_{*}^{-1}\Pi_{*},\Sigma_{*}\Upsilon_{*})$ is a feasible
solution of problem (10) and $\epsilon_{*}\geq\bar{\epsilon}={\rm
tr}(N_{x}\bar{\Sigma})+{\rm tr}(N_{y}\bar{\Upsilon})$. Accordingly, let
$\hat{\Pi}=\bar{\Sigma}\bar{L}$, and thus
$(\hat{\epsilon},\hat{\Pi},\hat{\Sigma},\hat{\Upsilon}):=(\bar{\epsilon},\bar{\Sigma}\bar{L},\bar{\Sigma},\bar{\Upsilon})$
is a feasible solution to (20). This implies that
$\bar{\epsilon}\geq\epsilon_{*}$, and consequently,
$\bar{\epsilon}=\epsilon_{*}$. This shows that
$\bar{L}=\Sigma_{*}^{-1}\Pi_{*}$, which completes the proof. ∎

######  Remark 2 (Influence of $\Phi_{1}^{\dagger}$).

To solve (20), the pseudo-inverse $\Phi_{1}^{{\dagger}}$ of matrix $X_{0}$
should be computed first. Different choices of $\Phi_{1}^{{\dagger}}$ may have
an influence on the error between the optimal observer gain obtained using the
model-based SDP (10) and that using the data-based SDP (20).

Notice from Subsection II-C that the LQR controller gain $\bar{K}$ can be
obtained by solving the data-based SDP in (17). In other words, the LQR
controller gain satisfies $\bar{K}=K_{*}=U_{0}Q_{*}P^{-1}_{*}$ for any optimal
solution of (17). Substituting matrices $K_{*}$ and $L_{*}$ derived from data-
based SDPs into (16), a data-driven LQG controller resembling the model-based
one in (11) is given by

$\displaystyle\hat{x}(t\\!+\\!1)$
$\displaystyle=(X_{1}\Phi_{1}^{\dagger}\\!+\\!X_{1}\Phi_{2}^{\dagger}K_{*}\\!-\\!L_{*}Y_{0}\Phi_{1}^{\dagger})\hat{x}(t)\\!+\\!L_{*}y(t)$
(21a) $\displaystyle u(t)$ $\displaystyle=K_{*}\hat{x}(t)$ (21b)

where $\hat{x}(t)$ is the estimated state from the data-driven estimator (21a)
with $\hat{x}(0)=0$ and $u(t)$ is the data-driven control input based on the
estimated state.

######  Theorem 5 (Equivalence between (11) and (21)).

Consider system (1) with mutually independent AWGN processes $w(t)$ and $v(t)$
obeying $\mathbb{E}[w(t)w^{\prime}(t)]=N_{x}$ and
$\mathbb{E}[v(t)v^{\prime}(t)]=N_{y}$, $\forall t$. Under Assumptions 1, 2 and
4, condition (15) is met, and SDPs (17) and (20) are feasible. Let
$(\gamma_{*},Q_{*},P_{*},G_{*})$ and
$(\epsilon_{*},\Pi_{*},\Sigma_{*},\Upsilon_{*})$ denote respectively the
unique optimal solution of SDP (17) and (20). Then the data-based LQG
controller (21) with $K_{*}=U_{0}Q_{*}P_{*}^{-1}$ and
$L_{*}=\Sigma_{*}^{-1}\Pi_{*}$ is equivalent to the model-based LQG controller
(10) in terms of the control input $u(t)$ and state estimate $\hat{x}(t)$.

###### Proof.

It has been shown in Theorem 2 that (17) is equivalent to (6). In addition,
Theorem 4 indicates that (20) is equivalent to (10). It follows from (14) that
(21) is equivalent to (11), which completes the proof. ∎

######  Remark 3 (_LQG control under bounded noise_).

In the linear-quadratic-Gaussian case, the LQG controller gives the optimal
control input. Yet, the motivation of this paper is to fill in the gap in the
literature of data-driven state estimation for open-loop unstable systems.
Hence, we follow tradition by considering bounded noise in Assumption 3. With
slight abuse of terminology, we still refer to the controller (21) as LQG
controller in the rest of the paper.

Building on Theorems 4 and 5, we are ready to present the main result of this
section.

######  Theorem 6 (RGES and ISpS of (21)).

Under Assumptions 1—4, condition (15) is satisfied, and SDPs (17) and (20) are
feasible. Then the state estimator (21a) is RGES and the system (1) with
$u(t)$ given by (21b) treating $w(t)$ as input is ISpS.

###### Proof.

According to the first constraint in (17) and (20), both matrices
$X_{1}\Phi_{1}^{\dagger}+X_{1}\Phi_{2}^{\dagger}K_{*}$ and
$X_{1}\Phi_{1}^{\dagger}-L_{*}Y_{0}\Phi_{1}^{\dagger}$ are Schur stable.
Therefore, there exist constants $c_{1}>1$, $c_{2}>1$, $\beta_{1}\in(0,1)$ and
$\beta_{2}\in(0,1)$ such that
$\|(X_{1}\Phi_{1}^{\dagger}+X_{1}\Phi_{2}^{\dagger}K_{*})^{t}\|\leq
c_{1}\beta_{1}^{t}$ and
$\|(X_{1}\Phi_{1}^{\dagger}-L_{*}Y_{0}\Phi_{1}^{\dagger})^{t}\|\leq
c_{2}\beta_{2}^{t}$. Recalling from Theorem 5 that SDP (20) is equivalent to
(10). Hence, the estimation error $e(t)=x(t)-\hat{x}(t)$ obeys (7) and
recursively

$\displaystyle\|e(t)\|=\Big{\|}(A-L_{*}C)^{t}x(0)+\sum_{i=0}^{t-1}(A-L_{*}C)^{i}$
$\displaystyle\times w(t-i-1)-L_{*}v(t-i-1)\Big{\|}$ $\displaystyle\leq
c_{2}\beta_{2}^{t}\|x(0)\|+\sum_{i=0}^{t-1}c_{2}\beta_{2}^{i}\|w(t-i-1)-L_{*}v(t-i-1)\|.$

where $\sum_{i=0}^{t-1}c_{2}\beta_{2}^{i}\|w(t-i-1)-L_{*}v(t-i-1)\|$ is a
$\mathcal{K}$-function of $w_{[0,t-1]}$ and $v_{[0,t-1]}$. Therefore,
according to Definition 2, the state estimator (21) is RGES.

Recalling from Assumption 3 that $v(t)\in\mathbb{B}_{\bar{v}}$, the estimation
error can be further bounded by

$\|e(t)\|\\!\leq\\!c_{2}\beta_{2}^{t}\|x(0)\|+\\!\sum_{i=0}^{t-1}c_{2}\beta_{2}^{i}(\|w(t-i-1)\|+\bar{v}\|L_{*}\|).$
(22)

Bearing this in mind, it follows from (1) that

$\displaystyle\|x(t)\|$
$\displaystyle=\|(A+BK_{*})x(t-1)-BK_{*}e(t-1)+w(t-1)\|$
$\displaystyle=\|(A+BK_{*})^{t}x(0)-\sum_{i=0}^{t-1}(A+BK_{*})^{i}$
$\displaystyle~{}~{}~{}\times[BK_{*}e(t-i-1)+w(t-i-1)]\|$
$\displaystyle\overset{\eqref{eq:e_bound}}{\leq}c_{1}\beta_{1}^{t}\|x(0)\|+\sum_{i=0}^{t-1}c_{1}\beta_{1}^{i}\Big{[}\|w_{t}\|_{\infty}+BK_{*}$
$\displaystyle~{}~{}~{}\times\Big{(}c_{2}\beta_{2}^{t-i+1}\|x(0)\|+\\!\sum_{j=0}^{i-1}c_{2}\beta_{2}^{j}(\|w_{i}\|_{\infty}+\bar{v}\|L_{*}\|)\Big{)}\\!\Big{]}$
$\displaystyle\leq
c_{1}\beta_{1}^{t}\|x(0)\|+\sum_{i=0}^{t-1}c_{3}\beta_{3}^{t-1}\|x(0)\|$
$\displaystyle~{}~{}~{}+\sum_{i=0}^{t-1}c_{1}\beta_{1}^{i}\Big{[}\|w_{t}\|_{\infty}+\sum_{j=0}^{i-1}c_{4}\beta_{2}^{j}(\|w_{i}\|_{\infty}+\bar{v}\|L_{*}\|)\Big{]}$

where $\beta_{3}:=\max\\{\beta_{1},\beta_{2}\\}$, $c_{3}=c_{1}\|BK_{*}\|$ and
$c_{4}=c_{2}\|BK_{*}\|$. Noticing that there exist constants $\tilde{c}_{3}$
and $\tilde{\beta}_{3}\in(\beta_{3},1)$ such that the second term in the last
inequality satisfies $\sum_{i=0}^{t-1}c_{3}\beta_{3}^{t-1}\leq
tc_{3}\beta_{3}^{t-1}\leq\tilde{c}_{3}\tilde{\beta}_{3}^{t}$, yielding

$\displaystyle\|x(t)\|$ $\displaystyle\leq
c_{1}\tilde{\beta}_{3}^{t}\|x(0)\|+\tilde{c}_{3}\tilde{\beta}_{3}^{t}\|x(0)\|+\frac{c_{1}(1-\beta_{1}^{t})}{1-\beta_{1}}\|w_{t}\|_{\infty}$
$\displaystyle~{}~{}~{}+\frac{c_{1}c_{4}(1-(1-\beta_{2})^{t}\beta_{1}^{t})(\|w_{t}\|_{\infty}+\|L_{*}\|\bar{v})}{(1-\beta_{1}+\beta_{1}\beta_{2})(1-\beta_{2})}$
$\displaystyle\leq
c_{5}\tilde{\beta}_{3}^{t}+\pi_{w}(\|w_{t}\|_{\infty})+\frac{c_{1}c_{4}\|L_{*}\|\bar{v}}{(1-\\!\beta_{1}+\beta_{1}\beta_{2})(1-\beta_{2})}$

where $c_{5}:=c_{1}+\tilde{c}_{3}$ and
$\pi_{w}(\|w_{t}\|_{\infty}):=(c_{4}+1-\beta_{2})c_{1}\|w_{t}\|_{\infty}/(1-\beta_{1}-\beta_{2}+\beta_{1}\beta_{2})$
is a $\mathcal{K}$-function. According to Definition 3, system (1) with the
proposed LQG controller achieves ISpS, which completes the proof. ∎

##  IV Robust Data-driven Control Using Noisy Data

The previous data-driven LQG controller requires clean offline data. This
requirement can be infeasible in real systems and limits practical use. This
motivates us to develop a robust controller based on offline noisy data.
Relaxing the noiseless data Assumption 4, Theorem 5 does not hold. Our results
in the following can be seen as robust versions of those in the previous
section. Considering that a data-based SDP for computing robust controller
gain $\tilde{K}$ has been suggested in Theorem 3, this section addresses
Problem 2 by modifying SDP (20) to compensate for noise in the data, such that
a robust stabilizing observer gain $\tilde{L}$ can be constructed based on its
optimal solution. Finally, leveraging gain matrices $\tilde{K}$ and
$\tilde{L}$, a robust version of controller (31) is devised for which
stability analysis follows.

###  IV-A Certainty-equivalent solution for (20) under noisy data

Due to noise, data matrices satisfy (13). Consequently, if condition (15)
holds, $(A,B,C)$ can be represented by

$\displaystyle
A=(X_{1}-W_{0})\Phi_{1}^{\dagger},~{}B=(X_{1}-W_{0})\Phi_{2}^{\dagger},~{}C=(Y_{0}-V_{0})\Phi_{1}^{\dagger}.$

In this setting, SDP (20) is modified into

$\displaystyle\min_{\epsilon,\Pi,\Sigma,\Upsilon}\ \epsilon$ (23)
$\displaystyle~{}~{}~{}{\rm
s.t.}~{}\begin{cases}\Phi_{1}^{{\dagger}\prime}[\Sigma(X_{1}\\!-\\!W_{0})\\!+\\!\Pi(Y_{0}\\!-\\!V_{0})]^{\prime}\cdot\Sigma^{-1}[\star]^{\prime}\\!-\\!\Sigma\\!+\\!I\\!\leq\\!0\\\
\Sigma\geq I\\\ \Upsilon-\Pi^{\prime}\Sigma^{-1}\Pi\geq 0\\\ {\rm
tr}(N_{x}\Sigma)+{\rm tr}(N_{y}\Upsilon)\leq\epsilon\end{cases}$

whose feasibility is guaranteed by Theorem 4. Denote its optimal solution by
$(\epsilon_{*},\Pi_{*},\Sigma_{*},\Upsilon_{*})$ and compute the robust
observer gain by $L_{*}=\Sigma_{*}^{-1}\Pi_{*}$. Since the noise $W_{0}$ and
$V_{0}$ are unknown, SDP (23) cannot be solved directly. It can be observed
that the difference between SDPs (20) and (23) lies only in the first
constraint. For convenience, define

$\displaystyle\Theta$ $\displaystyle:=\Phi_{1}^{{\dagger}\prime}(\Sigma
X_{1}+\Pi Y_{0})^{\prime}\Sigma^{-1}(\Sigma X_{1}+\Pi
Y_{0})\Phi_{1}^{\dagger}-\Sigma$ (24a) $\displaystyle M$
$\displaystyle:=W_{0}^{\prime}\Sigma(X_{1}-W_{0})-X_{1}^{\prime}\Sigma
W_{0}+W_{0}^{\prime}\Pi(Y_{0}-V_{0})$
$\displaystyle~{}~{}~{}-X_{1}^{\prime}\Pi
V_{0}+Y_{0}^{\prime}\Pi^{\prime}W_{0}-V_{0}^{\prime}\Pi^{\prime}(X_{1}-W_{0})$
$\displaystyle~{}~{}~{}-Y_{0}^{\prime}\Pi^{\prime}\Sigma^{-1}\Pi
V_{0}-V_{0}^{\prime}\Pi^{\prime}\Sigma^{-1}\Pi(Y_{0}-V_{0})$ (24b)
$\displaystyle\Psi$
$\displaystyle:=\Phi_{1}^{{\dagger}\prime}M\Phi_{1}^{\dagger}$ (24c)

which allow us to rewrite the first constraint of (20) as $\Theta+I\leq 0$ and
that of (23) as $\Theta+\Psi+I\leq 0$. Since noise matrices $W_{0}$ and
$V_{0}$ only show up in the first constraint of (23), i.e., in $\Psi$ in
(24c), it is natural to reformulate (23) by discarding the noise term $\Psi$
for tractability, i.e., by using the same SDP as in the noise-free case

$\displaystyle\min_{\epsilon,\Pi,\Sigma,\Upsilon}\ \epsilon$ (25)
$\displaystyle~{}~{}~{}{\rm
s.t.}~{}\begin{cases}\Phi_{1}^{{\dagger}\prime}(\Sigma X_{1}+\Pi
Y_{0})^{\prime}\Sigma^{-1}(\Sigma X_{1}+\Pi
Y_{0})\Phi_{1}^{\dagger}-\Sigma+I\leq 0\\\ \Sigma\geq I\\\
\Upsilon-\Pi^{\prime}\Sigma^{-1}\Pi\geq 0\\\ {\rm tr}(N_{x}\Sigma)+{\rm
tr}(N_{y}\Upsilon)\leq\epsilon.\end{cases}$

However, due to the noise-corrupted matrices $X_{1}$ and $Y_{0}$, two issues
arise with (25).

  * i)

This SDP may not be feasible; and,

  * ii)

Even if it is feasible, the corresponding observer gain $\hat{L}$ may not
guarantee that $A-\hat{L}C$ is Schur stable.

To address these issues, sufficient conditions are provided in the following
results, whose proof can be proceeded following [15, Theorem 2].

Suppose that a solution
$(\hat{\epsilon},\hat{\Pi},\hat{\Sigma},\hat{\Upsilon})$ for (25) is found and
compute the observer gain by $\hat{L}=\hat{\Sigma}^{-1}\hat{\Pi}$. With a
slight abuse of notation, let $(\epsilon_{*},\Pi_{*},\Sigma_{*},\Upsilon_{*})$
denote any optimal solution of (23) and $L_{*}=\Sigma_{*}^{-1}\Pi_{*}$ the
associated observer gain. Matrices
$\hat{\Theta},\,\hat{M},\,\hat{\Psi},\,\Theta_{*},\,M_{*}$ and $\Psi_{*}$ are
similarly defined as in (24).

######  Theorem 7 (Feasibility of SDP (25)).

Let Assumptions 1—3 hold. For any $\eta_{1}\geq 1$, there exists some
$\delta\geq 0$ such that if $\max\\{\|W_{0}\|,\|V_{0}\|\\}\leq\delta$, then
the condition (15) is satisfied and the SDP (25) is feasible with any optimal
solution $(\hat{\epsilon},\hat{\Pi},\hat{\Sigma},\hat{\Upsilon})$. If the
following condition holds

$\hat{\Psi}\leq(1-1/\eta_{1})I,$ (26)

then the observer gain $\hat{L}=\hat{\Sigma}^{-1}\hat{\Pi}$ is stabilizing.

Theorem 7 asserts that feasibility of SDP (25) and stability of the associated
observer gain $\hat{L}$ rely on two conditions: i) sufficiently small noise
and ii) condition (26). The former is standard in data-driven control, and it
does not bring issues when replacing (20) with (25) under small-noise data.
The challenge relates to how to ensure the condition (26), where $\hat{\Psi}$
is the gap between the stability conditions of the ideal formulation (23) and
the certainty-equivalent (25). To fulfill (26), we search for a solution
$\hat{\Phi}$ with small norm. This requirement translates into requiring
$\hat{M}$ to have small norm. However, there is no such constraint in (25).
Thus, even small noise may undermine the condition (26). To tackle this issue,
a new SDP is formulated by incorporating the robustness constraint (26) into
(25).

###  IV-B Robust observer gain

As discussed above, we aim at designing a robust data-based SDP by
incorporating the robustness constraint (26) into (25); that is, the solution
of SDP automatically enforces the norm of $\hat{\Psi}$. Observe from (24c)
that
$\|\hat{\Psi}\|\leq\|\hat{M}\|\|\Phi_{1}^{{\dagger}}\|^{2}=\|\hat{M}\|\|\Phi_{1}^{{\dagger}\prime}\Phi_{1}^{\dagger}\|$,
which implies that smaller $\|\hat{M}\|$ and
$\|\Phi_{1}^{{\dagger}\prime}\Phi_{1}^{\dagger}\|$ leads to smaller
$\|\hat{\Psi}\|$. Since $\Phi_{1}^{\dagger}$ appears in (25), we obtain this
matrix by solving an optimization problem minimizing
$\|\Phi_{1}^{{\dagger}\prime}\Phi_{1}^{\dagger}\|$ as follows

$\displaystyle\min_{\rho,M_{2},\Phi_{1}^{\dagger},\Phi_{2}^{\dagger}}\ \rho$
(27) $\displaystyle~{}~{}~{}~{}~{}{\rm s.t.}~{}\
\begin{cases}\Phi_{0}[\Phi_{1}^{\dagger}~{}\Phi_{2}^{\dagger}]=I\\\
\left[\begin{matrix}M_{2}&\Phi_{1}^{{\dagger}\prime}\\\
\Phi_{1}^{\dagger}&I\end{matrix}\right]\geq 0\\\ {\rm
tr}(M_{2})\leq\rho.\end{cases}$

Feasibility of this SDP can be guaranteed under condition (15). Define its
optimal solution by
$(\tilde{\rho},\tilde{M}_{2},\tilde{\Phi}_{1}^{{\dagger}},\tilde{\Phi}_{2}^{{\dagger}})$.
Based on $\tilde{\Phi}_{1}^{{\dagger}}$, a noise-robust version of SDP (25) is
given by

$\displaystyle\min_{\epsilon,\Pi,\Sigma,\Upsilon}\epsilon$ (28)
$\displaystyle~{}~{}~{}{\rm
s.t.}\,\begin{cases}\tilde{\Phi}_{1}^{{\dagger}\prime}(\Sigma X_{1}+\Pi
Y_{0})^{\prime}\Sigma^{-1}(\Sigma X_{1}+\Pi
Y_{0})\tilde{\Phi}_{1}^{{\dagger}}-\Sigma+I\leq 0\\\ \Sigma\geq I\\\
\Upsilon-\Pi^{\prime}\Sigma^{-1}\Pi\geq 0\\\ {\rm tr}(N_{x}\Sigma)+{\rm
tr}(N_{y}\Upsilon)+\alpha_{2}(\|\Upsilon\|+\|\Sigma\|+\|\Pi\|)\leq\epsilon\end{cases}$

where $\alpha_{2}>0$ is a parameter interpolating performance and robustness.
Specifically, larger $\alpha_{2}$ values favor solutions with smaller
$\|\Sigma\|+\|\Pi\|+\|\Upsilon\|$. Denote its optimal solution by
$(\tilde{\epsilon},\tilde{\Pi},\tilde{\Sigma},\tilde{\Upsilon})$. It follows
from (24b) that

$\displaystyle\|\tilde{M}\|\leq\max\big{\\{}\|W_{0}\|\|X_{1}-W_{0}\|+\|X_{1}\|\|W_{0}\|,~{}\|X_{1}\|\|V_{0}\|$
$\displaystyle+\\!\|W_{0}\|\|Y_{0}\\!-\\!V_{0}\|\\!+\\!\|Y_{0}\|\|W_{0}\|\\!+\\!\|V_{0}\|\|X_{1}\\!-\\!W_{0}\|,~{}\|Y_{0}\|\|V_{0}\|$
$\displaystyle+\|V_{0}\|\|Y_{0}-V_{0}\|\big{\\}}(\|\tilde{\Sigma}\|+\|\tilde{\Pi}\|+\|\tilde{\Pi}^{\prime}\tilde{\Sigma}\tilde{\Pi}^{-1}\|)$
(29)

which asserts that, the larger $\alpha_{2}$, the smaller $\|\tilde{M}\|$. A
sufficient condition for (26) is

$\|\tilde{M}\|\|\tilde{\Phi}_{1}^{{\dagger}\prime}\tilde{\Phi}_{1}^{\dagger}\|\leq
1-1/\eta_{1}.$ (30)

Obviously, larger $\alpha_{2}$ values promote more stabilizing solutions.
Building on this observation, a counterpart of Theorem 7 for (28) is
established.

######  Theorem 8 (Feasibility of SDP (28)).

Under Assumptions 1—3, there exists a value $\delta_{g}\geq 0$ such that if
$\max\\{\|W_{0}\|,\|V_{0}\|\\}\leq\delta_{g}$, then (15) is satisfied and SDP
(28) is feasible. In addition, the observer gain
$\tilde{L}=\tilde{\Sigma}^{-1}\tilde{\Pi}$ is stabilizing.

The proof of Theorem 8 is similar to that of [15, Lemma 4], and is thus
omitted here due to space limitation. Having designed the gain matrices
$\tilde{K}$ and $\tilde{L}$ based on offline noisy data, we proceed to
construct a robust version of the controller (31).

###  IV-C Robust data-based controller using noisy data

Leveraging $(\tilde{\Phi}_{1}^{{\dagger}},\tilde{\Phi}_{2}^{{\dagger}})$ from
solving (27) and matrices $\tilde{K}$, $\tilde{L}$ constructed by optimal
solutions of (18) and (28), respectively, the controller (21) is robustified
as follows

$\displaystyle\hat{x}(t+1)$
$\displaystyle=\big{[}(X_{1}-W_{0})\tilde{\Phi}_{1}^{\dagger}+(X_{1}-W_{0})\tilde{\Phi}_{2}^{\dagger}\tilde{K}$
$\displaystyle~{}~{}~{}-\tilde{L}(Y_{0}-V_{0})\tilde{\Phi}_{1}^{\dagger}\big{]}\hat{x}(t)+\tilde{L}y(t)$
(31a) $\displaystyle u(t)$ $\displaystyle=\tilde{K}\hat{x}(t).$ (31b)

Again, since noise $W_{0}$ and $V_{0}$ are not known, (31) cannot be
implemented. Upon ignoring $W_{0}$ and $V_{0}$, we arrive at an implementable
robust data-driven controller as follows

$\displaystyle\hat{x}(t+1)$
$\displaystyle=(X_{1}\tilde{\Phi}_{1}^{\dagger}+X_{1}\tilde{\Phi}_{2}^{\dagger}\tilde{K}-\tilde{L}Y_{0}\tilde{\Phi}_{1}^{\dagger})\hat{x}(t)+\tilde{L}y(t)$
(32a) $\displaystyle u(t)$ $\displaystyle=\tilde{K}\hat{x}(t).$ (32b)

Stability analysis for the resulting system (1) with this controller is
presented as follows.

$\displaystyle\left[\begin{matrix}x(t+1)\\\
\hat{x}(t+1)\end{matrix}\right]=\left[\begin{matrix}(X_{1}\\!-\\!W_{0})\tilde{\Phi}_{1}^{\dagger}\\!\\!&(X_{1}-W_{0})\tilde{\Phi}_{2}^{\dagger}\tilde{K}\\\
\tilde{L}(Y_{0}-V_{0})\tilde{\Phi}_{1}^{\dagger}\\!\\!&X_{1}\tilde{\Phi}_{1}^{\dagger}\\!+\\!X_{1}\tilde{\Phi}_{2}^{\dagger}\tilde{K}\\!-\\!\tilde{L}Y_{0}\tilde{\Phi}_{1}^{\dagger}\end{matrix}\right]\left[\begin{matrix}x(t)\\\
\hat{x}(t)\end{matrix}\right]+\left[\begin{matrix}w(t)\\\
\tilde{L}v(t)\end{matrix}\right]=\Xi_{0}\left[\begin{matrix}x(t)\\\
\hat{x}(t)\end{matrix}\right]+\left[\begin{matrix}w(t)\\\
\tilde{L}v(t)\end{matrix}\right]$ (33)
$\displaystyle\left[\begin{matrix}x(t+1)\\\
e(t+1)\end{matrix}\right]=\Xi_{3}\left[\begin{matrix}x(t)\\\
e(t)\end{matrix}\right]+\left[\begin{matrix}w(t)\\\
w(t)-\tilde{L}v(t)\end{matrix}\right],$ (34) $\displaystyle~{}{\rm
where}~{}~{}\Xi_{3}=\underbrace{\left[\begin{matrix}(X_{1}-W_{0})\tilde{\Phi}_{1}^{\dagger}+(X_{1}-W_{0})\tilde{\Phi}_{2}^{\dagger}\tilde{K}\\!\\!&\\!\\!(X_{1}-W_{0})\tilde{\Phi}_{2}^{\dagger}\tilde{K}\\\
0\\!\\!&\\!\\!X_{1}\tilde{\Phi}_{1}^{\dagger}-\tilde{L}Y_{0}\tilde{\Phi}_{1}^{\dagger}\end{matrix}\right]}_{\triangleq\Xi_{1}}\\!+\\!\underbrace{\left[\begin{matrix}0\\!\\!&\\!\\!0\\\
W_{0}\tilde{\Phi}_{1}^{\dagger}\\!+\\!W_{0}\tilde{\Phi}_{2}^{\dagger}\tilde{K}\\!-\\!\tilde{L}V_{0}\tilde{\Phi}_{1}^{\dagger}\\!\\!&\\!\\!W_{0}\tilde{\Phi}_{2}^{\dagger}\tilde{K}\end{matrix}\right]}_{\triangleq\Xi_{2}}$
(35)

######  Theorem 9 (RGES and ISpS for (32)).

Under Assumptions 1—3, there exists $\delta_{\bar{n}}>0$ such that for all
$\bar{n}:=\max\\{\bar{w},\bar{v}\\}\leq\delta_{\bar{n}}$ condition (15) is
satisfied and SDPs (18), (27) and (28) are feasible. Consider system (1) with
controller (32), where
$(\tilde{\Phi}_{1}^{{\dagger}},\tilde{\Phi}_{2}^{{\dagger}})$ is an optimal
solution of (27), and $\tilde{K}=U_{0}\tilde{Q}\tilde{P}^{-1}$,
$\tilde{L}=\tilde{\Sigma}^{-1}\tilde{\Pi}$ obtained by solving (18) and (28),
respectively. Then the state estimator (32a) is RGES and the system with
$u(t)$ given by (32b) treating $w(t)$ as input is ISpS.

###### Proof.

System (1) with controller (32) can be described by the composite system (33),
presented at the top of this page. Considering the estimation error
$e(t)=x(t)-\hat{x}(t)$, system (33) can be transformed into the composite
system (34). Since the transformation from
$[x(t)^{\prime}~{}\hat{x}^{\prime}(t)]^{\prime}$ to
$[x(t)^{\prime}~{}e^{\prime}(t)]^{\prime}$ is linear and non-singular, matrix
$\Xi_{3}$ is similar to $\Xi_{0}$. From [13, Section 6.6], this implies that
system (33) has same characteristic polynomial as system (34). Hence, if
$\Xi_{3}$ is Schur stable, so is $\Xi_{0}$. According to (35), one has that
$\|\Xi_{3}\|=\|\Xi_{1}+\Xi_{2}\|\leq\|\Xi_{1}\|+\|\Xi_{2}\|$. In the
following, we analyze $\|\Xi_{1}\|$ and $\|\Xi_{2}\|$ separately.

The eigenvalues of matrix $\Xi_{1}$ can be expressed in terms of those of
matrices
$(X_{1}-W_{0})\tilde{\Phi}_{1}^{\dagger}+(X_{1}-W_{0})\tilde{\Phi}_{2}^{\dagger}\tilde{K}$
and
$X_{1}\tilde{\Phi}_{1}^{\dagger}-\tilde{L}Y_{0}\tilde{\Phi}_{1}^{\dagger}$,
i.e.,
$\lambda(\Xi_{1})=\lambda((X_{1}-W_{0})\tilde{\Phi}_{1}^{\dagger}+(X_{1}-W_{0})\tilde{\Phi}_{2}^{\dagger}\tilde{K})\cup\lambda(X_{1}\tilde{\Phi}_{1}^{\dagger}-\tilde{L}Y_{0}\tilde{\Phi}_{1}^{\dagger})$
[30, Section 1.6]. Theorems 3 and 8 confirm that matrices $\tilde{K}$ and
$\tilde{L}$ are stabilizable. Consequently, $\Xi_{1}$ is Schur stable since
system (1) is controllable and observable under Assumption 1. Since being
Schur stable requires $\|\Xi_{3}\|<1$, this can be met if
$\|\Xi_{1}\|+\|\Xi_{2}\|<1$. Bearing this in mind, we show in the following
that if the noise is small enough, then $\|\Xi_{1}\|+\|\Xi_{2}\|<1$ always
holds.

Let $n(t):=[w(t)^{\prime}\ v(t)^{\prime}]^{\prime}$. It follows that

$\displaystyle\|\Xi_{2}\|\leq\bar{n}\sqrt{T\\!-1}\big{[}\sqrt{n_{x}}(\|\tilde{\Phi}_{0}^{\dagger}\|+\|\tilde{\Phi}_{2}^{\dagger}\|\|\tilde{K}\|)+\sqrt{n_{y}}\|\tilde{L}\|\|\tilde{\Phi}_{1}^{\dagger}\|\big{]}.$

Hence, recalling $\delta_{r}$ defined in Theorem 3 and $\delta_{g}$ defined in
Theorem 8, if
$\bar{n}\leq\delta_{\bar{n}}:=\min\\{\delta_{r},\delta_{g},\tilde{\delta}_{\bar{n}}\\}$
with

$\displaystyle\tilde{\delta}_{\bar{n}}\leq\frac{1-\|\Xi_{1}\|}{\sqrt{T-1}\big{[}\sqrt{n_{x}}(\|\tilde{\Phi}_{0}^{\dagger}\|+\|\tilde{\Phi}_{2}^{\dagger}\|\|\tilde{K}\|)+\sqrt{n_{y}}\|\tilde{L}\|\|\tilde{\Phi}_{1}^{\dagger}\|\big{]}}$

there exist a matrix $P_{z}\geq 0$ and a constant $\bar{\beta}_{1}>0$ such
that

$(\Xi_{1}+\Xi_{2})P_{z}(\Xi_{1}+\Xi_{2})^{\prime}-P_{z}=\Xi_{3}P_{z}\Xi_{3}^{\prime}-P_{z}\leq-\bar{\beta}_{1}I.$
(36)

Define the extended state by $z(t):=[x(t)^{\prime}\ e(t)^{\prime}]^{\prime}$.
Choosing the Lyapunov function $V(z(t))=z(t)^{\prime}P_{z}z(t)$, one gets that

$\displaystyle\ V(z(t\\!+\\!1))-V(z(t))$ $\displaystyle=$
$\displaystyle\,\bigg{(}\Xi_{3}z(t)+\left[\begin{matrix}I&0\\\
I&-\tilde{L}\end{matrix}\right]n(t)\\!\\!\bigg{)}^{\prime}\\!\cdot\\!P_{z}[\star]-z(t)^{\prime}P_{2}z(t)$
$\displaystyle\leq$
$\displaystyle\,-\big{(}\bar{\beta}_{1}-\tau_{1}\|P_{z}\|^{2}\|\Xi_{3}\|^{2}\big{)}\|z(t)\|^{2}$
$\displaystyle\,+\Big{(}\frac{1}{\tau_{1}}+\|P_{z}\|\Big{)}(1+\|\tilde{L}\|)^{2}\|n(t)\|^{2}$
$\displaystyle\leq$
$\displaystyle\,-\bar{\beta}_{2}V(z(t))+\tau_{2}\|n(t)\|^{2}$

where $\tau_{1}>0$ is such that Young’s inequality
$[w(t)^{\prime}~{}w(t)^{\prime}-v(t)^{\prime}\tilde{L}^{\prime}]P_{z}\Xi_{3}z(t)\leq\tau_{1}\|P_{z}\|^{2}\|\Xi_{3}\|^{2}\|z(t)\|^{2}+\|n(t)\|^{2}(1/\tau_{1}+\|P_{z}\|)$
holds. In addition,
$\bar{\beta}_{2}:=\underline{\lambda}_{P_{z}}(\bar{\beta}_{1}-\tau_{1}\|P_{z}\|^{2}\|\Xi_{3}\|^{2})>0$
and $\tau_{2}:=(1/\tau_{1}+\|P_{z}\|)\|n(t)\|^{2}$. Recursively, it holds that

$V(z(t))\leq(1-\bar{\beta}_{2})^{t}V(z(0))\\!+\tau_{2}\\!\sum_{i=0}^{t-1}(1-\bar{\beta}_{2})^{i}\|n(t\\!-i\\!-1)\|^{2}.$

Since
$\underline{\lambda}_{P_{z}}\|e(t)\|^{2}\leq\underline{\lambda}_{P_{z}}\|z(t)\|^{2}\leq
V(z(t))\leq\bar{\lambda}_{P_{z}}\|z(t)\|^{2}$ and $e(0)=x(0)$, the estimation
error obeys

$\displaystyle\|e(t)\|$
$\displaystyle\leq\sqrt{\frac{2\bar{\lambda}_{P_{z}}}{\underline{\lambda}_{P_{z}}}}(1-\bar{\beta})^{\frac{t}{2}}\|e(0)\|$
$\displaystyle~{}~{}~{}+\Big{(}\tau_{2}\sum_{i=0}^{t-1}(1-\bar{\beta}_{2})^{i}\|n(t-i-1)\|^{2}\Big{)}^{\frac{1}{2}}$
(37)

which implies RGES for the estimator (32a). Similarly, the state satisfies

$\|x(t)\|\leq\sqrt{\frac{2\bar{\lambda}_{P_{z}}}{\underline{\lambda}_{P_{z}}}}(1-\bar{\beta})^{\frac{t}{2}}\|x(0)\|+\tilde{\pi}_{w}(\|w_{t}\|_{\infty})+\bar{v}\sqrt{\frac{2\tau_{2}}{\bar{\beta_{2}}}}$

where
$\tilde{\pi}_{w}(\|w_{t}\|_{\infty}):=\sqrt{2\tau_{2}\sum_{i=0}^{t-1}(1-\bar{\beta}_{2})^{i}}\|w_{t}\|_{\infty}$
is a $\mathcal{K}$-function of $\|w_{t}\|_{\infty}$. This establishes ISpS for
the system according to Definition 3. ∎

######  Remark 4 (_Comparison with existing results_).

Compared with the results in [36, 44, 3], the proposed method has five
advantages.

  * 1)

_Less offline data._ The work [36] assumes that the offline data
$\\{[u(i)^{\prime}~{}w(i)^{\prime}]\\}_{i=0}^{T-1}$ collected is persistently
exciting of order $n_{x}+1$; as such, at least $T\geq
n_{x}+(n_{u}+n_{x})(n_{x}+1)$ data samples should be collected. In contrast,
the proposed controller only requires $\\{u(i)\\}_{i=0}^{T-1}$ be persistently
exciting of order $n_{x}+1$, and $T\geq n_{x}+n_{u}(n_{x}+1)$, which saves at
least $n_{x}(n_{x}+1)$ data samples.

  * 2)

_Less conservative assumptions on noise._ As discussed in Remark 1, in [36]
and [44], except for Assumption 3, other constrains on noise are required.

  * 3)

_Less time and computing resources._ In [44, 3], an optimization problem is
solved at each time during online operation, which costs more time and
consumes more computing resources than the proposed method.

  * 4)

_Easy-to-check and less conservative conditions._ In [36], a kernel inclusion
condition on the offline data is posed for stabilization, which is not easy to
check in practice. In contrast, the stability conditions of our method are
Lyapunov-type and simple. Relative to the practical robust exponential
stability in [44], we ensure RGES when both offline and online data are
subject to noise.

  * 5)

_Tackling state estimation and control problems simultaneously._ In [44, 36],
both the input $u(k)$ and output $y(k)$ are used to estimate the state $x(k)$,
and hence an additional controller should be implemented to generate $u(k)$.
Our proposed controller estimates the state $x(k)$ requiring only the previous
output $y(k-1)$, and a simple feedback controller is designed based on the
estimated state. In other words, the state estimation and control are jointly
addressed involving a single dynamics, which simplifies practical
implementations.

##  V Numerical Examples

In this section, we compare the proposed schemes with related results in [36,
3, 44] through two numerical examples. The following simulations were run on a
Lenovo laptop with a 20-core i7-12700H processor at 2.3GHz.

###  V-A Noise-free offline data

In the first experiment, we considered the discretized version of the open-
loop unstable batch reactor system in [14] using a sampling period of $0.1$s,
with matrices $A$, $B$ and $C$ in (1) given by

$\displaystyle A=\left[\begin{matrix}1.178&0.001&0.511&-0.403\\\
-0.051&0.661&-0.011&0.061\\\ 0.076&0.335&0.560&0.382\\\
0&0.335&0.089&0.849\end{matrix}\right],$ $\displaystyle
B=\left[\begin{matrix}0.004&-0.087\\\ 0.467&0.001\\\ 0.213&-0.235\\\
0.213&-0.016\end{matrix}\right],~{}~{}C=\left[\begin{matrix}1&0&1&-1\\\
0&1&0&0\end{matrix}\right].$

All state estimation schemes were implemented in MATLAB. We collected noise-
free trajectories of length $T=15$ from random initial conditions with random
inputs uniformly generated from $[-1,1]$. SDP (20) was solved with
$N_{x}=0.02I_{4}$ and $N_{y}=0.02I_{2}$ by CVX [19], yielding

$L_{*}=\left[\begin{matrix}0.7034&0.0385\\\ -0.0228&0.3812\\\ 0.2753&0.4023\\\
0.0619&0.4066\end{matrix}\right]$

which is the same as the observer $\bar{L}$ found using the MATLAB command
${\rm dare}(A^{\prime},C^{\prime},N_{x},N_{y})$ solving the DARE equation.
Moreover, noise $w(t)$ and $v(t)$ during online operation were generated
uniformly from $[-0.02,0.02]$ and the state and estimated state trajectories
are depicted in Fig. 1, which verifies the effectiveness of the proposed
method in Section III.

Figure 1: State trajectory by the controller (21).

###  V-B Noisy offline data

In the second experiment, we examine the performance of our proposed robust
data-driven control method relative to three recent competing methods,
including the data-based UIO in [36], data-based MHE in [44], and the data-
driven set-based method in [3] on the batch reactor system. As discussed in
Remark 1, the dimension of both process and measurement noise in [36] should
not exceed that of the output, i.e., (1a) is replaced by
$x(t+1)=Ax(t)+Bu(t)+E_{w}w(t)$ with ${\rm rank}(E_{w})\leq n_{y}$. Therefore,
we first compare the performance of the proposed method and the UIO method in
the case of ${\rm rank}(E_{w})=n_{y}=2$, which is then followed by comparison
of the four methods under ${\rm rank}(E_{w})=n_{y}=4$. It is worth clarifying
that the methods in [36, 44, 3] were designed for autonomous systems, and
therefore they were simulated with the input sequence generated from (32).

Figure 2: Performance of the proposed controller (32), data-based UIO state
estimator [36] and data-based MHE [3] with $\bar{n}=0.2$ under ${\rm
rank}(E_{w})=2$ (top panel) and ${\rm rank}(E_{w})=4$ (bottom panel).

Considering ${\rm rank}(E_{w})=n_{y}=2$, noise $w(t)$ and $v(t)$ were
uniformly generated at random from $[-0.02,0.02]$. In addition SDP (18) was
solved with $W_{x}=I_{4}$, $W_{u}=I_{2}$, $\alpha_{1}=0.2$ and SDP (28) was
solved with $N_{x}=0.02I_{4}$, $N_{y}=0.02I_{2}$, $\alpha_{2}=0.2$. The
offline data were collected following the same procedure as described in the
previous experiment. The top panel of Fig. 2 shows the state estimation
performance of the proposed method (32) (dashed line), the data-based UIO
method in [36] (dash-dotted line) and the data-based MHE method (dotted line)
[44]. It can be observed that the state converges under the proposed robust
controller with noisy data and the estimation error remains small. The
averaged state estimation error
$\bar{e}:=(1/100)\sum_{t=1}^{t=100}\|\hat{x}(t)-x(t)\|$ of the proposed method
is $0.489$, which is smaller than $0.545$ that of the data-based UIO and
$1.815$ that of the data-based MHE.

Under ${\rm rank}(E_{w})=4$, we plot the estimated states by the three methods
in the bottom panel of Fig. 2. Evidently, the estimated state by the proposed
method remains convergent, while that by the UIO method diverges. Moreover,
the average estimation error of controller (28) $0.889$ is several times
smaller than that of MHE method $2.311$.

Figure 3:  State estimation performance of the data-driven set-based method
[3], data-based MHE [44], ID-based Kalman filter, and proposed controller
(32).

The working assumption of [3] is slightly different from the others, which
assumes the knowledge of matrix $C$. To fulfill Assumption 2, an extension of
this method is made, which is discussed in Appendix VII. The following
simulation was proceeded by using the implicit intersection with constrained
zonotopes in [3], which was shown to numerically outperform its baseline
methods. Moreover, the set-based method returns a bound on the reachable set
of state $x(t)$ and we use its center as the estimated state. Consider the
same input-driven variant of the rotating target system as in [3]

$\displaystyle A=\left[\begin{matrix}0.9455\\!&-0.2426\\\
0.2486\\!&0.9455\end{matrix}\right],B=\left[\begin{matrix}0.1\\\
0\end{matrix}\right],C=\left[\begin{matrix}1&\\!0.4\\\ 0.9\\!&-1.2\\\
-0.8\\!&0.2\\\ 0\\!&0.7\end{matrix}\right]$

which is an autonomous system and whose control inputs were randomly generated
from $[-1,1]$. Noise $w(t)$ and $v(t)$ were generated obeying a uniform
distribution over $[-1,1]$. SDP (28) was solved with $N_{x}=I_{4}$,
$N_{y}=I_{2}$ and $\alpha_{2}=1$. Fig. 3 compares the state estimation
performance of the proposed method (32) (red solid), the data-driven set-based
method in [3] using constraint zonotopes (green dotted with cross), the data-
based MHE in [44] (orange dashed with circle), and the identification
(ID)-based Kalman method (blue dashed). System ID was performed using the
_n4sid_ toolbox with default option setting based on the same set of offline
data. Clearly, the proposed controller (32) stabilizes the system.
Furthermore, the set-based method achieves better performance at the
beginning, which is because $u(0)$ and $y(0)$ are used to estimate $x(0)$, yet
$\hat{x}(0)$ is randomly generated for the proposed method. Let $\bar{t}$
denote the average computation time per iteration. Comparison of $\bar{e}$ and
$\bar{t}$ between different methods is presented in Table I. The proposed
controller enjoys comparable state estimation performance while being (much)
more computationally efficient that competing data-based methods.

TABLE I: Comparison between different methods with $\bar{n}=1$ Method | $\bar{e}$ |  $\bar{t}$ (sec)
---|---|---
Data-driven controller (32) | $0.577$ | $10^{-4}$
Data-based MHE in[44] | $6.738$ | $0.006$
Data-driven set-based method in [3] | $0.300$ | $5.125$
System ID-based Kalman filter | $0.608$ | $0.005$

##  VI Conclusions

This paper presented a solution to the problem of jointly estimating and
controlling the state of an unknown linear systems subject to both process and
measurement noise. Specifically, this study proposed data-based linear
quadratic Gaussian (LQG) control formulations based on offline data
collection, both in noise-free and noisy settings. For noise-free offline
data, a convex semi-definite programming (SDP) method is used to construct a
steady-state Kalman filter. Building upon existing research on data-driven
linear quadratic regulator (LQR), this work also presented a data-based
version of the LQG controller that is equivalent to its model-based
counterpart. For noisy data, a soft constraint is added to the SDP to develop
a robust SDP method that provides a robust observer gain. The proposed data-
driven design enjoys robust global exponential stability (RGES) of the state
estimator and input-to-state practical stability (ISpS) of the closed-loop
system. Numerical examples illustrate the efficacy of the proposed
controllers.

##  VII Appendix

In [3], two approaches were proposed to estimate a reachable set for the
state, in which a set $\tilde{\mathcal{R}}\subset\mathbb{R}^{n_{x}}$ such that
$x(t)\in\tilde{\mathcal{R}}$ for all $t\in\mathbb{N}$ is constructed using the
offline data and updated on-the-fly to yield a more accurate set
$\hat{\mathcal{R}}_{t}$ using online measurements. The two approaches are
referred to as reverse mapping and implicit intersection. The main idea of the
former is to explicitly recover the state from the output based on the SVD of
matrix $C$. However, since $C$ is unknown in this paper, the first approach
cannot be employed directly. Hence, this section focuses on extending the
implicit intersection approach to fit our setting in Assumption 2 using
zonotopes.

Assumption 3 is restated in terms of zonotopes as follows. Noise
$w(t)\in\mathcal{Z}_{w}$ and $v(t)\in\mathcal{Z}_{v}$ are assumed to belong to
the bounding zonotopes $\mathcal{Z}_{w}=\langle
c_{w},G_{w}\rangle\subset\mathbb{R}^{n_{x}}$ and $\mathcal{Z}_{v}=\langle
c_{v},G_{v}\rangle\subset\mathbb{R}^{n_{y}}$, respectively. Define the matrix
zonotope of pair $(A,B)$ as

$\mathcal{N}_{\Sigma}=\\{[A~{}B]|X_{1}=AX_{0}+BU_{0}+W_{0},\
W_{0}\in\mathcal{M}_{w}\\}$ (38)

where $\mathcal{M}_{w}=\langle C_{\mathcal{M},w},G_{\mathcal{M},w}^{(1:\xi
T)}\rangle$ is the zonotope of process noise sequence $W_{0}$. Similarly, the
matrix zonotope of $C$, i.e., $\mathcal{M}_{C}=\langle
C_{\mathcal{M},C},G_{\mathcal{M},C}^{(1:\xi T)}\rangle$, is given by

$\displaystyle\mathcal{M}_{C}=(Y_{0}\\!-V_{0})X_{0}^{\dagger}\\!=\Big{(}Y_{0}-C_{\mathcal{M},v}\\!-\sum_{i=1}^{\xi_{\mathcal{M},v}}\beta^{(i)}_{\mathcal{M},v}G_{\mathcal{M},v}\Big{)}X_{0}^{\dagger}$

where $X_{0}^{{\dagger}}$ exists under the condition (15), and
$\mathcal{M}_{v}=\langle C_{\mathcal{M},v},G_{\mathcal{M},v}^{(1:\xi
T)}\rangle$ is the matrix zonotope of measurement noise $V_{0}$. The set
$\tilde{R}_{t}=\langle\tilde{c}_{t},\tilde{G}_{t}\rangle$ over-approximates
the exact reachable set with
$\tilde{R}_{t+1}=\mathcal{M}_{\Sigma}(\tilde{R}_{t}\times\mathcal{U}_{t})+\mathcal{Z}_{w}$
and

$\mathcal{M}_{\Sigma}=(X_{1}-\mathcal{M}_{w})\left[\begin{matrix}X_{0}\\\
U_{0}\end{matrix}\right]^{{\dagger}}.$

The true state can be expressed as $x_{t}=\tilde{c}_{t}+\tilde{G}_{t}z_{t}$
for some weight vector $z_{t}$. Proposition 2 in [3] can be restated as
follows.

######  Proposition 1.

The intersection of $\tilde{R}_{t}$ and the region for $x(t)$ corresponding to
$y(t)$ with noise $v(t)\in\mathcal{Z}_{v}$ satisfying (1) can be over-
approximated by the zonotope
$\hat{R}_{t}=\langle\hat{c}_{t},\hat{G}_{t}\rangle$ with

$\displaystyle\hat{c}_{t}$
$\displaystyle=\tilde{c}_{t}+\lambda_{t}^{*}(y(t)-C_{\mathcal{M}_{C}}-C_{v,i})$
$\displaystyle\hat{G}_{t}$
$\displaystyle=\\!\big{[}\begin{matrix}(I\\!-\lambda_{t}^{*}C_{\mathcal{M}_{C}})\tilde{G}_{t}&-\\!\lambda_{t}^{*}G_{\mathcal{M}_{C}}&-\\!\lambda_{t}^{*}G_{v}&-\\!\lambda_{t}^{*}G_{\mathcal{M}_{C}}\tilde{G}_{t}\end{matrix}\big{]}$

where $\lambda_{t}^{*}$ is the optimal solution of the following problem

$\min_{\lambda_{t}}\big{\|}\\!\big{[}\begin{matrix}(\\!I\\!-\\!\lambda_{t}C_{\\!\mathcal{M}_{C}})\tilde{G}_{t}&\\!\\!-\\!\lambda_{t}G_{\\!\mathcal{M}_{C}}&\\!-\\!\lambda_{t}G_{v}&\\!-\\!\lambda_{t}G_{\\!\mathcal{M}_{C}}\\!\tilde{G}_{t}\end{matrix}\big{]}\\!\big{\|}_{F}^{2}$

and $\|\cdot\|_{F}^{2}$ denotes the matrix Frobenius norm.

Based on the above Proposition, the state estimator can be obtained leveraging
[3, Algorithm 1].

## References

  * [1] R. Adachi and Y. Wakasa, “Dual system representation and prediction method for data-driven estimation,” in _Proc. of Annu. Conf. Soc. Inst. Control Eng. Japan_ , Tokyo, Japan, Sept. 8-10 2021, pp. 1245–1250.
  * [2] A. A. Al Makdah, V. Krishnan, V. Katewa, and F. Pasqualetti, “Behavioral feedback for optimal LQG control,” in _Proc. of IEEE Conf. Decis. Control_ , Cancun, Mexico, Dec. 6-9, 2022, pp. 4660–4666.
  * [3] A. Alanwar, A. Berndt, K. H. Johansson, and H. Sandberg, “Data-driven set-based estimation using matrix zonotopes with set containment guarantees,” in _Proc. of Eur. Control Conf._ , London, United Kingdom, July 12-15, 2022, pp. 875–881.
  * [4] A. Allibhoy and J. Cortés, “Data-based receding horizon control of linear network systems,” _IEEE Control Syst. Lett._ , vol. 5, no. 4, pp. 1207–1212, Oct. 2021.
  * [5] D. Alpago, F. Dörfler, and J. Lygeros, “An extended Kalman filter for data-enabled predictive control,” _IEEE Control Syst. Lett._ , vol. 4, no. 4, pp. 994–999, May, 2020.
  * [6] J. Berberich, J. Köhler, M. A. Müller, and F. Allgöwer, “Data-driven model predictive control with stability and robustness guarantees,” _IEEE Trans. Autom. Control_ , vol. 66, no. 4, pp. 1702–1717, June, 2021.
  * [7] V. Breschi, M. Fabris, S. Formentin, and A. Chiuso, “Uncertainty-aware data-driven predictive control in a stochastic setting,” _arXiv:2211.10321_ , Nov. 2022.
  * [8] F. Bullo, J. Cortés, and S. Martínez, _Distributed Control of Robotic Networks_ , ser. Applied Math. Series. Princeton University Press, 2009.
  * [9] D. E. Catlin, _Estimation, Control, and the Discrete Kalman Filter_. New York, USA: Springer Science & Business Media, 2012, vol. 71.
  * [10] J. Chen, J. Sun, and G. Wang, “From unmanned systems to autonomous intelligent systems,” _Eng._ , vol. 12, no. 5, pp. 16–19, May, 2022.
  * [11] T. Chen and B. A. Francis, _Optimal Sampled-data Control Systems_. London, UK: Springer Science & Business Media, 2012.
  * [12] J. Coulson, J. Lygeros, and F. Dörfler, “Regularized and distributionally robust data-enabled predictive control,” in _Proc. of IEEE Conf. Decis. Control_ , Nice, France, France, Dec. 11-13 2019, pp. 2696–2701.
  * [13] C. G. Cullen, _Matrices and Linear Transformations_. New York, USA: Courier Corporation, 2012.
  * [14] C. De Persis and P. Tesi, “Formulas for data-driven control: Stabilization, optimality, and robustness,” _IEEE Trans. Autom. Control_ , vol. 65, no. 3, pp. 909–924, Mar. 2020.
  * [15] C. De Persis and P. Tesi, “Low-complexity learning of linear quadratic regulators from noisy data,” _Automatica_ , vol. 128, p. 109548, Mar. 2021\.
  * [16] T. Duncan and P. Varaiya, “On the solutions of a stochastic control system,” _SIAM J. Control_ , vol. 9, no. 3, pp. 354–371, Aug. 1971.
  * [17] L. Furieri, B. Guo, A. Martin, and G. Ferrari-Trecate, “Near-optimal design of safe output feedback controllers from noisy data,” _IEEE Trans. Autom. Control_ , pp. 1–16, June, 2022, doi = 10.1109/TAC.2022.3180692.
  * [18] G. C. Goodwin and K. S. Sin, _Adaptive Filtering Prediction and Control_. Mineola, New York, USA: Courier Corporation, 2014.
  * [19] M. Grant and S. Boyd, “CVX: Matlab software for disciplined convex programming, version 2.1,” http://cvxr.com/cvx/citing/, 2014.
  * [20] Z. Hou and Z. Wang, “From model-based control to data-driven control: Survey, classification and perspective,” _IEEE Trans. Ind. Inform._ , vol. 235, pp. 3–35, Jun. 2013.
  * [21] M. Inoue and V. Gupta, ““Weak” control for human-in-the-loop systems,” _IEEE Control Syst. Lett._ , vol. 3, no. 2, pp. 440–445, Jan. 2019.
  * [22] L. Ji, J. B. Rawlings, W. Hu, A. Wynn, and M. Diehl, “Robust stability of moving horizon estimation under bounded disturbances,” _IEEE Trans. Autom. Control_ , vol. 61, no. 11, pp. 3509–3514, Nov. 2016.
  * [23] Z.-P. Jiang, A. R. Teel, and L. Praly, “Small-gain theorem for ISS systems and applications,” _Math. Control Signals Syst._ , vol. 7, pp. 95–120, 1994.
  * [24] S. Kang and K. You, “Minimum input design for direct data-driven property identification of unknown linear systems,” _arXiv:2208.13454_ , 2022.
  * [25] H. Kitano, “Systems biology: A brief overview,” _Science_ , vol. 295, no. 5560, pp. 1662–1664, Mar. 2002.
  * [26] Y. Li, X. Wang, J. Sun, G. Wang, and J. Chen, “Data-driven consensus control of fully distributed event-triggered multi-agent systems,” _Sci. CHINA Inf. Sci._ , vol. 66, no. 5, pp. 152 202–, May 2023.
  * [27] W. Liu, J. Sun, G. Wang, F. Bullo, and J. Chen, “Data-driven resilient predictive control under Denial-of-Service,” _IEEE Trans. Autom. Control_ , pp. 1–16, Sept. 2022, doi: 10.1109/TAC.2022.3209399.
  * [28] I. Markovsky, L. Huang, and F. Dörfler, “Data-driven control based on the behavioral approach: From theory to applications in power systems,” _IEEE Control Syst._ , June 2022.
  * [29] D. Müller, J. Feilhauer, J. Wickert, J. Berberich, F. Allgöwer, and O. Sawodny, “Data-driven predictive disturbance observer for quasi continuum manipulators,” in _Proc. of IEEE Conf. Decis. Control_ , Cancún, Mexico, Dec. 6-9 2022, pp. 1816–1822.
  * [30] J. O’Reilly, _Mathematics in Science and Engineering, Observers for Linear Systems_. London, UK: Academic Press, 1983\.
  * [31] G. Pan, R. Ou, and T. Faulwasser, “On a stochastic fundamental lemma and its use for data-driven optimal control,” _IEEE Trans. Autom. Control_ , pp. 1–16, 2022, doi: 10.1109/TAC.2022.3232442.
  * [32] Z.-H. Pang, X.-Y. Zhao, J. Sun, Y. Shi, and G.-P. Liu, “Comparison of three data-driven networked predictive control methods for a class of nonlinear systems,” _IEEE/CAA J. Autom. Sinica_ , vol. 9, no. 9, pp. 1714–1716, Aug. 2022.
  * [33] F. Pasqualetti, R. Carli, and F. Bullo, “Distributed estimation via iterative projections with application to power network monitoring,” _Automatica_ , vol. 48, no. 5, pp. 747–758, May 2012.
  * [34] M. Peet, “Control in LMIs,” https://en.wikibooks.org/wiki/LMIs_in_Control, 2021.
  * [35] D. Simon, _Optimal State Estimation: Kalman, H-infinity, and Nonlinear Approaches_. New York, USA: John Wiley & Sons, 2006.
  * [36] M. S. Turan and G. Ferrari-Trecate, “Data-driven unknown-input observers and state estimation,” _IEEE Control Syst. Lett._ , vol. 6, pp. 1424–1429, Aug. 2022.
  * [37] M. E. Valcher, “State observers for discrete-time linear systems with unknown inputs,” _IEEE Trans. Autom. Control_ , vol. 44, no. 2, pp. 397–401, Feb. 1999.
  * [38] H. J. van Waarde, J. Eising, H. L. Trentelman, and M. K. Camlibel, “Data informativity: A new perspective on data-driven analysis and control,” _IEEE Trans. Autom. Control_ , vol. 65, no. 11, pp. 4753–4768, Jan. 2020.
  * [39] G. Wang, G. B. Giannakis, J. Chen, and J. Sun, “Distribution system state estimation: An overview of recent developments,” _Front. Infor. Technol. Electr. Eng._ , vol. 20, no. 1, pp. 4–17, Jan. 2019.
  * [40] X. Wang, J. Sun, F. Deng, G. Wang, and J. Chen, “Event-triggered consensus control of heterogeneous multi-agent systems: Model- and data-based approaches,” _Sci. CHINA Inf. Sci._ , pp. 1–13, 2022, doi: 10.1007/s11432-022-3683-y.
  * [41] X. Wang, J. Sun, G. Wang, J. Chen, and F. Allgöwer, “Data-driven control of distributed event-triggered network systems,” _IEEE/CAA J. Autom. Sinica_ , vol. 10, no. 2, p. 351–364, Feb. 2023.
  * [42] Y. Wang, C. Shang, and D. Huang, “Data-driven control of stochastic systems: An innovation estimation approach,” _arXiv:2209.08995_ , Sep. 2022.
  * [43] J. C. Willems, I. Markovsky, P. Rapisarda, and B. L. M. De Moor, “A note on persistency of excitation,” _Syst. Control Lett._ , vol. 56, no. 4, pp. 325–329, May, 2005.
  * [44] T. M. Wolff, V. G. Lopez, and M. A. Müller, “Robust data-driven moving horizon estimation for linear discrete-time systems,” _arXiv:2210.09017_ , Oct. 2022.