# Mixing and observation for Markov operator cocycles

Fumihiko NAKAMURA  Faculty of Engineering, Kitami Institute of Technology,
Hokkaido, 090-8507, JAPAN<EMAIL_ADDRESS>,  Yushi NAKANO
Department of Mathematics, Tokai University, Kanagawa 259-1292, JAPAN
<EMAIL_ADDRESS>and  Hisayoshi TOYOKAWA  Department of
Mathematics, Hokkaido University, Hokkaido, 060-0810, JAPAN
<EMAIL_ADDRESS>
###### Abstract.

We consider generalized definitions of mixing and exactness for random
dynamical systems in terms of Markov operator cocycles. We first give six
fundamental definitions of mixing for Markov operator cocycles in view of
observations of the randomness in environments, and show that they can be
reduced into two different groups. Secondly, we give the definition of
exactness for Markov operator cocycles and show that Lin's criterion for
exactness can be naturally extended to the case of Markov operator cocycles.
Finally, in the class of asymptotically periodic Markov operator cocycles, we
show the Lasota-Mackey type equivalence between mixing, exactness and
asymptotic stability.

###### Key words and phrases:

Markov operator cocycles, mixing, exactness, random dynamical systems

###### 2010 Mathematics Subject Classification:

37H05 ; 37A25

##  1\. Introduction

This paper concerns a cocycle generated by Markov operators, called a Markov
operator cocycle. Let $(X,\mathcal{A},m)$ be a probability space, and
$L^{1}(X,m)$ (the quotient by equality $m$-almost everywhere of) the space of
all $m$-integrable functions on $X$, endowed with the usual $L^{1}$-norm
$\|\cdot\|_{L^{1}(X)}$. An operator $P:L^{1}(X,m)\to L^{1}(X,m)$ is called a
_Markov operator_ if $P$ is linear, positive (i.e. $Pf\geq 0$ $m$-almost
everywhere if $f\geq 0$ $m$-almost everywhere) and

$\int_{X}Pfdm=\int_{X}fdm\quad\text{ for all $f\in L^{1}(X,m)$}.$ (1.1)

Markov operators naturally appear in the study of dynamical systems (as
Perron–Frobenius operators; see (1.3)), Markov processes (as integral
operators with the stochastic kernels of the processes), and random dynamical
systems in the annealed regime (as integrations of Perron–Frobenius operators
over environmental parameters). For these deterministic/stochastic dynamics,
$\\{P^{n}f\\}_{n\geq 0}$ is the evolution of density functions of random
variables driven by the system. We refer to [F, LM].

A Markov operator cocycle is given by compositions of different Markov
operators which are provided with according to the environment
$\\{\sigma^{n}(\omega)\\}_{n\geq 0}$ driven by a measure-preserving
transformation $\sigma:\Omega\to\Omega$ on a probability space
$(\Omega,\mathcal{F},\mathbb{P})$,

$\mathbb{N}\times\Omega\times L^{1}(X,m)\to L^{1}(X,m):(n,\omega,f)\mapsto
P_{\sigma^{n-1}(\omega)}\circ P_{\sigma^{n-2}(\omega)}\circ\cdots\circ
P_{\omega}f$

(see Definition 1.1 for more precise description). So, in nature it possess
two kinds of randomness:

  * (i)

The evolution of densities at each time are dominated by Markov operators
$P_{\omega}$,

  * (ii)

The selection of each Markov operators is driven by the base dynamics
$\sigma$.

The aim of this paper is to investigate how the _observation_ of the
randomness of the state space and the environment influences statistical
properties of the system, and to give a step to understanding more complicated
phenomenon in multi-stochastic systems.

Our focus lies on the mixing property. Recall that a Markov operator
$P:L^{1}(X,m)\to L^{1}(X,m)$ is said to be _mixing_ if

$\int_{X}P^{n}fgdm\to\int_{X}fdm\int_{X}gdm\quad\text{as $n\to\infty$}$ (1.2)

for any $f\in L^{1}(X,m)$ and $g\in L^{\infty}(X,m)$ (when $P1_{X}=1_{X}$, see
Remark 1.3 for more general form). Due to (1.1), this means that two random
variables $P^{n}f$ and $g$ are asymptotically independent so that the system
is considered to ``mix'' the state space well. In other words, the randomness
of $P$ in the sense of mixing can be seen through the observables $f$ and $g$.
Hence, for Markov operator cocycles, the strength of the dependence of the
observables on $\omega$ expresses how one observes the randomness of the state
space and the environment. Furthermore, more directly, we can consider
different kinds of mixing properties according to whether the environment
$\omega$ is observed as _a prior event_ to the observation of $f,g$. According
to these viewpoints, we will introduce six definitions of mixing for Markov
operator cocycles (Definition 1.2). In Section 2, we show that four of them
are equivalent when $\Omega$ is a compact topological space, while at least
two of them are different. In the case when the Markov operator cocycle is
generated by a random dynamical system, we also show that all of them imply
the (conventional) mixing property of the skew-product transformation induced
by the random dynamical system.

We further investigate exactness for Markov operator cocycles. Since the
observable $g$ in (1.2) does not appear in the definition of exactness for a
Markov operator $P$ (recall that, when $P1_{X}=1_{X}$, $P$ is said to be exact
if $\displaystyle\lim_{n\to\infty}$ $\left\lVert
P^{n}f-\int_{X}fdm\right\rVert_{L^{1}(X)}=0$ for all $f\in L^{1}(X,m)$; see
also the remark following Definition 1.5). Thus, in contrast to the mixing
property, we only have one definition of exactness for Markov operator
cocycles (Definition 1.5). We will show that Lin's criterion for exactness can
be naturally extended to the case of Markov operator cocycles (Section 3), and
finally, in the class of asymptotically periodic Markov operator cocycles, we
prove Lasota-Mackey type equivalence between mixing, exactness and asymptotic
stability, as well as their relationship with the existence of an invariant
density map (Section 4). See Figure 1 for the summary.

###  1.1. Definitions of mixing and exactness

Let $D(X,m)$ and $L^{1}_{0}(X,m)$ be subsets of $L^{1}(X,m)$ given by

$\displaystyle D(X,m)$ $\displaystyle=$ $\displaystyle\left\\{f\in
L^{1}(X,m):f\geq 0\ \text{$m$-almost everywhere,
$\|f\|_{L^{1}(X)}=1$}\right\\},$ $\displaystyle L^{1}_{0}(X,m)$
$\displaystyle=$ $\displaystyle\left\\{f\in
L^{1}(X,m):\int_{X}fdm=0\right\\}.$

Note that $P:L^{1}(X,m)\to L^{1}(X,m)$ is a Markov operator if and only if
$P(D(X,m))\subset D(X,m)$.

One of the most important example of Markov operators is the _Perron–Frobenius
operator_ induced by a non-singular transformation $T:X\to X$ (that is,
$T_{*}m$ is absolutely continuous with respect to $m$, where $T_{*}m$ is the
pushforward of $m$ given by $T_{*}m(A)=m(T^{-1}A)$ for $A\in\mathcal{A}$). The
Perron–Frobenius operator $L_{T}:L^{1}(X,m)\to L^{1}(X,m)$ of $T$ is defined
by

$L_{T}f=\frac{d[T_{*}(fm)]}{dm}\quad\text{for $f\in L^{1}(X,m)$},$ (1.3)

where $fm$ is a finite signed measure given by $(fm)(A)=\int_{A}fdm$ for
$A\in\mathcal{A}$ and $d\mu/dm$ is the Radon-Nikodym derivative of an
absolutely continuous finite signed measure $\mu$. Note that for each
$X$-valued random variable $\chi$ whose distribution is $fm$ with some $f\in
D(X,m)$, $T(\chi)$ has the distribution $(L_{T}f)m$ (and thus, $L_{T}$ is also
called the transfer operator associated with $T$). It is straightforward to
see that

$\int_{X}L_{T}fgdm=\int_{X}fg\circ Tdm\quad\quad\text{for $f\in L^{1}(X,m)$
and $g\in L^{\infty}(X,m)$},$ (1.4)

and that $L_{T}$ is a Markov operator.

Recall that $(\Omega,\mathcal{F},\mathbb{P})$ is a probability space, and
$\sigma:\Omega\to\Omega$ is a $\mathbb{P}$-preserving transformation. For a
measurable space $\Sigma$, we say that a measurable map
$\Phi:\mathbb{N}_{0}\times\Omega\times\Sigma\to\Sigma$ is a _random dynamical
system_ on $\Sigma$ over the driving system $\sigma$ if

$\varphi^{(0)}_{\omega}=\mathrm{id}_{\Sigma}\quad\text{and}\quad\varphi^{(n+m)}_{\omega}=\varphi^{(n)}_{\sigma^{m}\omega}\circ\varphi^{(m)}_{\omega}$

for each $n,m\in\mathbb{N}_{0}$ and $\omega\in\Omega$, with the notation
$\varphi^{(n)}_{\omega}=\Phi(n,\omega,\cdot)$ and
$\sigma\omega=\sigma(\omega)$, where $\mathbb{N}_{0}=\mathbb{N}\cup\\{0\\}$. A
standard reference for random dynamical systems is the monographs by Arnold
[Ar]. It is easy to check that

$\varphi^{(n)}_{\omega}=\varphi_{\sigma^{n-1}\omega}\circ\varphi_{\sigma^{n-2}\omega}\circ\cdots\circ\varphi_{\omega}$
(1.5)

with the notation $\varphi_{\omega}=\Phi(1,\omega,\cdot)$. Conversely, for
each measurable map
$\varphi:\Omega\times\Sigma\to\Sigma:(\omega,x)\mapsto\varphi_{\omega}(x)$,
the measurable map $(n,\omega,x)\mapsto\varphi_{\omega}^{(n)}(x)$ given by
(1.5) is a random dynamical system. We call it a random dynamical system
induced by $\varphi$ over $\sigma$, and simply denote it by
$(\varphi,\sigma)$. When $\Sigma$ is a Banach space and
$\varphi_{\omega}:\Sigma\to\Sigma$ is $\mathbb{P}$-almost surely linear,
$(\varphi,\sigma)$ is called a _linear operator cocycle_. We give a
formulation of Markov operators in random environments in terms of linear
operator cocycles.

######  Definition 1.1.

We say that a linear operator cocycle $(P,\sigma)$ induced by a measurable map
$P:\Omega\times L^{1}(X,m)\to L^{1}(X,m)$ over $\sigma$ is a _Markov operator
cocycle_ (or a _Markov operator in random environments_) if
$P_{\omega}=P(\omega,\cdot):L^{1}(X,m)\to L^{1}(X,m)$ is a Markov operator for
$\mathbb{P}$-almost every $\omega\in\Omega$.

Let $(n,\omega,f)\mapsto P^{(n)}_{\omega}f$ be a Markov operator cocycle
induced by $P:\Omega\times L^{1}(X,m)\to L^{1}(X,m)$ such that
$P_{\omega}=P(\omega,\cdot)$ is the Perron–Frobenius operator $L_{T_{\omega}}$
associated with a non-singular map $T_{\omega}:X\to X$ for $\mathbb{P}$-almost
every $\omega$. Then, it follows from (1.4) that $\mathbb{P}$-almost surely

$\displaystyle\int_{X}P_{\omega}^{(n)}fgdm=\int_{X}fg\circ
T_{\omega}^{(n)}dm,\quad\text{for }f\in L^{1}(X,m)\text{ and }g\in
L^{\infty}(X,m),$ (1.6)

where $T_{\omega}^{(n)}=T_{\sigma^{n-1}\omega}\circ
T_{\sigma^{n-2}\omega}\circ\cdots\circ T_{\omega}$.

We are now in place to give definitions of mixing for Markov operator
cocycles. Let $K$ be a space consisting of measurable maps from $\Omega$ to
$L^{\infty}(X,m)$.

######  Definition 1.2.

A Markov operator cocycle. $(P,\sigma)$ is called

  1. (1)

prior mixing for homogeneous observables if for $\mathbb{P}$-almost every
$\omega\in\Omega$, any $f\in L^{1}_{0}(X,m)$ and $g\in L^{\infty}(X,m)$, it
holds that

$\displaystyle\lim_{n\to\infty}\int_{X}P^{(n)}_{\omega}fgdm=0;$ (1.7)

  2. (2)

posterior mixing for homogeneous observables if for any $f\in L^{1}_{0}(X,m)$,
$g\in L^{\infty}(X,m)$ and $\mathbb{P}$-almost every $\omega\in\Omega$, (1.7)
holds;

  3. (3)

prior mixing for inhomogeneous observables in $K$ if for $\mathbb{P}$-almost
every $\omega\in\Omega$, any $f\in L^{1}_{0}(X,m)$ and $g\in K$, it holds that

$\displaystyle\lim_{n\to\infty}\int_{X}P^{(n)}_{\omega}fg_{\sigma^{n}\omega}dm=0;$
(1.8)

  4. (4)

posterior mixing for inhomogeneous observables in $K$ if for any $f\in
L^{1}_{0}(X,m)$, $g\in K$ and $\mathbb{P}$-almost every $\omega\in\Omega$,
(1.8) holds.

In the prior case (the posterior case), the observation of the environment
$\omega$ is _a prior event_ (_a posterior event_ , respectively) to the
observation of $f$ and $g$. As the class of inhomogeneous observables $K$ in
Definition 1.2, we will consider the following two fundamental classes.

  * (i)

$B(\Omega,L^{\infty}(X,m))$: the set of all bounded and measurable maps from
$\Omega$ to $L^{\infty}(X,m)$.

  * (ii)

$C(\Omega,L^{\infty}(X,m))$: the set of all bounded and continuous maps from
$\Omega$ to $L^{\infty}(X,m)$ (when $\Omega$ is a topological space and
$\mathcal{F}$ is its Borel $\sigma$-field).

######  Remark 1.3.

The above definitions need not require an invariant density map for the Markov
operator cocycle $(P,\sigma)$. We say that a measurable map $h:\Omega\to
D(X,m)$ is an invariant density map for $(P,\sigma)$ if
$P_{\omega}h_{\omega}=h_{\sigma\omega}$ holds for $\mathbb{P}$-almost every
$\omega\in\Omega$ where $h_{\omega}=h(\omega)$. Now we assume that there exist
an invariant density map $h:\Omega\to D(X,m)$ for $(P,\sigma)$ such that for
$\mathbb{P}$-almost every $\omega\in\Omega$,

$\displaystyle\lim_{n\to\infty}m\left(\operatorname*{supp}P_{\omega}^{(n)}1_{X}\setminus\operatorname*{supp}P_{\omega}^{(n)}h_{\omega}\right)=0.$
(1.9)

Then by (1.9) and the fact that
$P^{(n)}_{\omega}f-h_{\sigma^{n}\omega}=P^{(n)}_{\omega}(f-h_{\omega})\in
L^{1}_{0}(X,m)$ for $f\in D(X,m)$, one can easily check that $(P,\sigma)$ is
prior mixing for homogeneous observables if and only if for
$\mathbb{P}$-almost every $\omega\in\Omega$, any $f\in D(X,m)$ and $g\in
L^{\infty}(X,m)$, it holds that

$\displaystyle\lim_{n\to\infty}\int_{X}\left(P^{(n)}_{\omega}f-h_{\sigma^{n}\omega}\right)gdm=0.$

Furthermore, when $P_{\omega}$ is the Perron–Frobenius operator
$L_{T_{\omega}}$ associated with a non-singular map $T_{\omega}:X\to X$, by
(1.6), it is also equivalent to that for $\mathbb{P}$-almost every
$\omega\in\Omega$, any $f\in L^{1}(X,\mu_{\omega})$ and $g\in
L^{\infty}(X,m)$,

$\displaystyle\int_{X}fg\circ
T_{\omega}^{(n)}d\mu_{\omega}-\int_{X}fd\mu_{\omega}\int_{X}gd\mu_{\sigma^{n}\omega}\to
0\quad\text{as $n\to\infty$},$ (1.10)

where $\mu_{\omega}=h_{\omega}m$. See also Remark 2.7. Moreover, we can
replace ``for any $f\in L^{1}(X,\mu_{\omega})$'' in the previous sentence with
``for any measurable function $f:\Omega\times X\to\mathbb{R}$ such that
$f_{\omega}=f(\omega,\cdot)\in L^{1}(X,\mu_{\omega})$ $\mathbb{P}$-almost
surely'', and ``$f$'' in (1.10) with ``$f_{\omega}$''. Similar equivalent
conditions can be found for other types of mixing in Definition 1.2.

All kinds of mixing in Definition 1.2 were adopted in literature, especially
in the form of (1.10) to discuss mixing for random dynamical systems. For
instance, we refer to Baladi et al. [BKS] for the definition (1), Dragičević
et al. [DFGV] for the definition (2), Bahsoun et al. [BBR] for the definition
(3), and Gundlach [G] for the definition (4). Moreover, in the deterministic
case (i.e. $\Omega$ is a singleton), all the definitions are equivalent to the
usual definition of mixing for a single Markov operator [LM].

######  Remark 1.4.

Another natural candidate for the class of inhomogeneous observable is the
Bochner-Lebesgue space $L^{\infty}(\Omega,L^{\infty}(X,m))$, that is, the
Kolmogorov quotient (by equality $\mathbb{P}$-almost surely) of the space of
all $\mathbb{P}$-essentially bounded and Bochner measurable maps from $\Omega$
to $L^{\infty}(X,m)$ (and (1.8) is interpreted as it holds under the usual
identification between an equivalent class and a representative of the class).
However, in the case $K=L^{\infty}(\Omega,L^{\infty}(X,m))$, the prior version
(3) does not make sense because one can find an equivalent class $[g]\in
L^{\infty}(\Omega,L^{\infty}(X,m))$ and maps $g_{1},g_{2}\in[g]$ such that
(1.8) holds for $g=g_{1}$ while (1.8) does not hold for $g=g_{2}$, see
Subsection 2.2. On the other hand, the posterior version (4) makes sense for
$K=L^{\infty}(\Omega,L^{\infty}(X,m))$, and indeed, its relationship with
posterior mixing for homogeneous observables will be discussed in Theorem 2.3.

By the definitions, we immediately see that the prior mixing implies the
posterior mixing (that is, $(1)\Rightarrow(2)$ and $(3)\Rightarrow(4)$ in
Definition 1.2). It is also obvious that the prior (posterior) mixing for
inhomogeneous observables in $B(\Omega,L^{\infty}(X,m))$ or
$C(\Omega,L^{\infty}(X,m))$ implies the prior (posterior, respectively) mixing
for homogeneous observables. Refer to Figure 1.

We next define exactness for Markov operator cocycles.

######  Definition 1.5.

A Markov operator cocycle $(P,\sigma)$ is called exact if for
$\mathbb{P}$-almost every $\omega\in\Omega$ and any $f\in L^{1}_{0}(X,m)$, it
holds that

$\displaystyle\lim_{n\to\infty}\left\lVert
P^{(n)}_{\omega}f\right\rVert_{L^{1}(X)}=0.$ (1.11)

As in Remark 1.3, we can easily see that the exactness of a Markov operator
cocycle $(P,\sigma)$ is equivalent to that for $\mathbb{P}$-almost every
$\omega\in\Omega$ and any $f\in D(X,m)$,

$\displaystyle\lim_{n\to\infty}\left\lVert
P^{(n)}_{\omega}f-h_{\sigma^{n}\omega}\right\rVert_{L^{1}(X)}=0.$

In Section 3, we will see another equivalent condition of the exactness in the
case when $(P,\sigma)$ is associated with a random dynamical system on $X$.
The relationship between mixing, exactness and asymptotic stability will be
also discussed in Section 4, see again Figure 1 for a summary.

##  2\. Mixing

###  2.1. Equivalence

We show the equivalence between prior/posterior mixing for homogeneous
observables and prior/posterior mixing for inhomogeneous observables in
$C(\Omega,L^{\infty}(X,m))$ when $\Omega$ is a compact topological space.

######  Theorem 2.1.

Assume that $\Omega$ is a compact topological space. Then, the followings are
equivalent:

  1. (1)

$(P,\sigma)$ is prior mixing for homogeneous observables.

  2. (2)

$(P,\sigma)$ is posterior mixing for homogeneous observables.

  3. (3)

$(P,\sigma)$ is prior mixing for inhomogeneous observables in
$C(\Omega,L^{\infty}(X,m))$.

  4. (4)

$(P,\sigma)$ is posterior mixing for inhomogeneous observables in
$C(\Omega,L^{\infty}(X,m))$.

###### Proof.

As mentioned, the implications $(3)\Rightarrow(4)$ and $(4)\Rightarrow(1)$
immediately follow from the definitions. We show $(2)\Rightarrow(1)$. Assume
that $(P,\sigma)$ is posterior mixing for homogeneous observables, that is,
for any $f\in L_{0}^{1}(X,m)$ and $g\in L^{\infty}(X,m)$, there is a
measurable set $\Omega_{0}(f,g)$ such that $\mathbb{P}(\Omega_{0}(f,g))=1$ and
(1.7) holds for each $\omega\in\Omega_{0}(f,g)$. By the simple function
approximation with rational coefficients, we can find countable dense subsets
$\\{f_{k}\\}_{k\in\mathbb{N}}$ of $L_{0}^{1}(X,m)$ and
$\\{g_{l}\\}_{l\in\mathbb{N}}$ of $L^{\infty}(X,m)$. Define a measurable set
$\Omega_{0}$ by

$\Omega_{0}=\bigcap_{k\in\mathbb{N}}\bigcap_{l\in\mathbb{N}}\Omega_{0}(f_{k},g_{l}),$

then it is straightforward to see that $\mathbb{P}(\Omega_{0})=1$ and (1.7)
holds for any $\omega\in\Omega_{0}$, $f\in L_{0}^{1}(X,m)$ and $g\in
L^{\infty}(X,m)$, i.e. $(P,\sigma)$ is prior mixing for homogeneous
observables.

We next show $(1)\Rightarrow(3)$. Assume that $(P,\sigma)$ is prior mixing for
homogeneous observables, that is, there is a measurable set $\Omega_{0}$ with
$\mathbb{P}(\Omega_{0})=1$ such that (1.7) holds for any
$\omega\in\Omega_{0}$, $f\in L_{0}^{1}(X,m)$ and $g\in L^{\infty}(X,m)$. Fix
such an $\Omega_{0}$. Fix also $f\in L_{0}^{1}(X,m)$, $g\in
C(\Omega,L^{\infty}(X,m))$ and $\epsilon>0$. Then, since $\Omega$ is compact,
we get finitely many functions $\\{g_{i}\\}_{i=1}^{I}\subset L^{\infty}(X,m)$
such that, for any $\omega\in\Omega$ there is $1\leq i(\omega)\leq I$
satisfying

$\|g_{\omega}-g_{i(\omega)}\|_{L^{\infty}(X)}<\epsilon.$

(Note that
$\\{\\{\omega\in\Omega:\|g_{\omega}-\tilde{g}\|_{L^{\infty}(X)}<\epsilon\\}:\tilde{g}\in
L^{\infty}(X)\\}$ is an open covering of $\Omega$ by virtue of the continuity
of $g$.) For convenience, let $g_{0}=1_{X}$.

We further fix $\omega\in\Omega_{0}$. By applying (1.7) to $g=g_{i}$ with
$0\leq i\leq I$, one can find $N_{i}\equiv N_{i}(\omega,f)\in\mathbb{N}$ such
that

$\left|\int_{X}P^{(n)}_{\omega}fg_{i}dm\right|<\epsilon\quad\text{for all
$n\geq N_{i}$}.$

Hence, for any $n\geq\max_{0\leq i\leq I}N_{i}$,

$\displaystyle\left|\int_{X}P^{(n)}_{\omega}fg_{\sigma^{n}\omega}dm\right|$
$\displaystyle\leq\left|\int_{X}P^{(n)}_{\omega}fg_{i(\sigma^{n}\omega)}dm\right|+\left\|g_{\sigma^{n}\omega}-g_{i(\sigma^{n}\omega)}\right\|_{L^{\infty}(X)}\left\|P^{(n)}_{\omega}f\right\|_{L^{1}(X)}$
$\displaystyle<(1+\|f\|_{L^{1}(X)})\epsilon.$ (2.1)

Since $\epsilon>0$ is arbitrary, we conclude

$\lim_{n\to\infty}\int_{X}P^{(n)}_{\omega}fg_{\sigma^{n}\omega}dm=0\quad\text{for
all $\omega\in\Omega_{0}$},$

which implies that $(P,\sigma)$ is prior mixing for inhomogeneous observables
in $C(\Omega,L^{\infty}(X,m))$. This completes the proof. ∎

######  Remark 2.2.

As in the proof, the compactness of $\Omega$ in Theorem 2.1 is only needed to
show the implication of prior mixing for inhomogeneous observables in
$C(\Omega,L^{\infty}(X,m))$ from prior mixing for homogeneous observables.

We also can show the following equivalence for observables in
$L^{\infty}(\Omega,L^{\infty}(X,m))$.

######  Theorem 2.3.

If $(P,\sigma)$ is posterior mixing for homogeneous observables, then
$(P,\sigma)$ is posterior mixing for inhomogeneous observables in
$L^{\infty}(\Omega,L^{\infty}(X,m))$.

###### Proof.

Assume that $(P,\sigma)$ is posterior mixing for homogeneous observables, i.e.
for any $f\in L_{0}^{1}(X,m)$ and $g\in L^{\infty}(X,m)$, there is a
measurable set $\Omega_{0}(f,g)$ such that $\mathbb{P}(\Omega_{0}(f,g))=1$ and
(1.7) holds for each $\omega\in\Omega_{0}(f,g)$. Fix $f\in L^{1}_{0}(X,m)$ and
$g\in L^{\infty}(\Omega,L^{\infty}(X,m))$. We only consider the case when
$g_{\omega}(x)$ is positive for $\mathbb{P}\times m$-almost every
$(\omega,x)\in\Omega\times X$. (If not, we consider the usual decomposition
$g=g^{+}-g^{-}$ with $g^{+}_{\omega}(x)=\max\\{g_{\omega}(x),0\\}$ and
$g^{-}_{\omega}(x)=\max\\{-g_{\omega}(x),0\\}$.)

Since $g\in L^{\infty}(\Omega,L^{\infty}(X,m))$ (in particular, $g$ is Bochner
measurable), there is a sequence of simple functions
$\\{g^{k}\\}_{k\in\mathbb{N}}\subset L^{\infty}(\Omega,L^{\infty}(X,m))$ of
the form

$g^{k}_{\omega}(x)=\sum_{i=1}^{I(k)}\tilde{g}_{i}^{k}(x)1_{F_{i}^{k}}(\omega)\quad\left(g_{i}^{k}\in
L^{\infty}(X,m),\ F_{i}^{k}\in\mathcal{F}\right)$

and a $\mathbb{P}$-full measure set $\Omega_{1}$ such that
$\sup_{\omega\in\Omega_{1}}\left\|g_{\omega}-g_{\omega}^{k}\right\|_{L^{\infty}(X)}\to
0$ as $k\to\infty$. Define a $\mathbb{P}$-full measure set $\Omega_{0}$ by

$\Omega_{0}=\bigcap_{k\in\mathbb{N}}\bigcap_{1\leq i\leq
I(k)}\Omega_{0}(f,g_{i}^{k}).$

Let $\Omega_{2}=\Omega_{0}\cap(\bigcap_{n\geq 0}\sigma^{-n}\Omega_{1})$, then
$\mathbb{P}(\Omega_{2})=1$ by the invariance of $\mathbb{P}$ for $\sigma$.

Fix $\omega\in\Omega_{2}$ and $\epsilon>0$. Fix also $k\in\mathbb{N}$ such
that

$\left\|g_{\sigma^{n}\omega}-g_{\sigma^{n}\omega}^{k}\right\|_{L^{\infty}(X)}<\epsilon\quad\text{for
all $n\in\mathbb{N}$}.$

Calculate that

$\left\lvert\int_{X}P_{\omega}^{(n)}fg^{k}_{\sigma^{n}\omega}dm\right\rvert\leq\sum_{i=1}^{I(k)}\left\lvert\int_{X}P_{\omega}^{(n)}f\tilde{g}_{i}^{k}dm\right\rvert
1_{F_{i}^{k}}(\sigma^{n}\omega)\leq I(k)\max_{1\leq i\leq
I(k)}\left\lvert\int_{X}P_{\omega}^{(n)}f\tilde{g}_{i}^{k}dm\right\rvert.$

On the other hand, by the choice of $\omega$, for any $1\leq i\leq I(k)$ one
can find a positive integer $N_{i}=N_{i}(f,\omega,k)$ such that if $n\geq
N_{i}$, then

$\left\lvert\int_{X}P^{(n)}_{\omega}fg_{i}^{k}dm\right\rvert<\frac{\epsilon}{I(k)}.$

Thus, for any $n\geq\max_{1\leq i\leq I(k)}N_{i}$,

$\displaystyle\left|\int_{X}P^{(n)}_{\omega}fg_{\sigma^{n}\omega}dm\right|$
$\displaystyle\leq\left|\int_{X}P^{(n)}_{\omega}fg_{\sigma^{n}\omega}^{k}dm\right|+\left\|g_{\sigma^{n}\omega}-g_{\sigma^{n}\omega}^{k}\right\|_{L^{\infty}(X)}\left\|P^{(n)}_{\omega}f\right\|_{L^{1}(X)}$
$\displaystyle<(1+\|f\|_{L^{1}(X)})\epsilon.$

Since $\epsilon>0$ is arbitrary, we conclude that $(P,\sigma)$ is prior mixing
for inhomogeneous observables in $L^{\infty}(\Omega,L^{\infty}(X,m))$. ∎

###  2.2. Counterexamples

We give an example exhibiting prior mixing for homogeneous observables but not
for inhomogeneous observables in $B(\Omega,L^{\infty}(X,m))$.

Let $T:X\to X$ be a measurably bijective map (up to zero $m$-measure sets)
preserving $m$ such that the Perron–Frobenius operator $L_{T}$ associated with
$T$ is mixing (note that $L_{T}1_{X}=1_{X}$ due to the invariance of $m$ and
recall (1.2)). Note that the baker map is well-known example as such map $T$.
Assume that there is a $\mathbb{P}$-positive measure set $\Omega_{0}$ such
that the forward orbit of $\omega$ is not finite and a measurable set (e.g.
$\Omega=[0,1]$ and $\mathbb{P}$ is the Lebesgue measure on $\Omega$), and that
$P_{\omega}=L_{T}$ for all $\omega\in\Omega_{0}$. By construction, this Markov
operator cocycle $(P,\sigma)$ is prior mixing for homogeneous observables.

######  Theorem 2.4.

The Markov operator cocycle $(P,\sigma)$ given above is not prior mixing for
inhomogeneous observables in $B(\Omega,L^{\infty}(X,m))$.

###### Proof.

We first note that the negation of prior mixing for inhomogeneous observables
in $B(\Omega,L^{\infty}(X,m))$ is that there is a measurable set
$\Gamma\subset\Omega$ with $\mathbb{P}(\Gamma)>0$ such that for any
$\omega\in\Gamma$, there exist $f=f^{\omega}$ in $L^{1}_{0}(X,m)$ and a
bounded measurable map $g=g^{\omega}:\Omega\to
L^{\infty}(X,m):\tilde{\omega}\mapsto g^{\omega}_{\tilde{\omega}}$ satisfying

$\lim_{n\to\infty}\int_{X}P_{\omega}^{(n)}fg_{\sigma^{n}\omega}dm=\lim_{n\to\infty}\int_{X}P_{\omega}^{(n)}f^{\omega}g^{\omega}_{\sigma^{n}\omega}dm\neq
0.$

We emphasize that the observable $g=g^{\omega}$ may depend on
$\omega\in\Gamma$.

Let $\Gamma=\Omega_{0}$ and fix $\omega\in\Gamma$. Fix a measurable set $A$
with $m(A)=1/2$. Let $f=1_{A}-1_{X\setminus A}$. Define
$g=g^{\omega}:\Omega\to L^{\infty}(X,m)$ by

$g_{\tilde{\omega}}:=\begin{cases}L_{T}^{n}1_{A}&(\text{when
$\tilde{\omega}=\sigma^{n}\omega$})\\\ 0&{\rm(otherwise)}.\end{cases}$

Then, by construction, $f\in L^{1}_{0}(X,m)$ and $g:\Omega\to L^{\infty}(X,m)$
is a bounded and measurable map. Furthermore, since $T$ is bijective, for
every $n\in\mathbb{N}$,

$L^{n}_{T}1_{A}\cdot L_{T}^{n}1_{X\backslash A}=0\quad\text{$m$-almost
everywhere}$

(note that $L^{n}_{T}1_{B}=1_{T^{n}(B)}$ for any measurable set $B$).
Therefore, for every $n\in\mathbb{N}$

$\int_{X}P_{\omega}^{(n)}fg_{\sigma^{n}\omega}dm=\int_{X}L_{T}^{n}\left(1_{A}-1_{X\setminus
A}\right)L_{T}^{n}1_{A}dm=\int_{X}\left(L_{T}^{n}1_{A}\right)^{2}dm=m(T^{n}(A))=\frac{1}{2}>0.$

In conclusion, $(P,\sigma)$ is not prior mixing for inhomogeneous observables
in $B(\Omega,L^{\infty}(X,m))$. ∎

###  2.3. Skew-product transformations

In this subsection, we show that our definitions of mixing for Markov operator
cocycles naturally lead the conventional mixing property for skew-product
transformations.

Recall that $(X,\mathcal{A},m)$ and $(\Omega,\mathcal{F},\mathbb{P})$ are
probability spaces, and $\sigma:\Omega\to\Omega$ is a $\mathbb{P}$-preserving
transformation. We further assume that $\sigma$ is invertible and mixing. Let
$(P,\sigma)$ be a Markov operator induced by the Perron–Frobenius operator
$P_{\omega}=L_{T_{\omega}}$ corresponding to a non-singular transformation
$T_{\omega}:X\to X$ for $\mathbb{P}$-almost every $\omega\in\Omega$. Assume
that there is an invariant density map $h:\Omega\to D(X,m)$ of $(P,\sigma)$
and define a measurable family of measures
$\\{\mu_{\omega}\\}_{\omega\in\Omega}$ by
$\mu_{\omega}(A)=\int_{A}h_{\omega}dm$ for $A\in\mathcal{A}$, so that we have
$(T_{\omega})_{*}\mu_{\omega}=\mu_{\sigma\omega}$ due to (1.6).

Consider the skew-product transformation $\Theta:\Omega\times X\to\Omega\times
X$ defined by $\Theta(\omega,x)=(\sigma\omega,T_{\omega}x)$ with the measure
$\nu$ on $\Omega\times X$

$\nu(A)=\int_{\Omega}\mu_{\omega}(A_{\omega})d\mathbb{P}(\omega)\quad\text{for
$A\in\mathcal{F}\otimes\mathcal{A}$},$

where $A_{\omega}:=\\{x\in X:(\omega,x)\in A\\}$ denotes the
$\omega$-sections. Then, $(\Omega\times X,\mathcal{F}\otimes\mathcal{A},\nu)$
becomes a probability space, and $\nu$ is an invariant measure for $\Theta$,
namely the Perron–Frobenius operator $L_{\Theta}$ corresponding to $\Theta$
with respect to $\nu$ satisfies $L_{\Theta}1_{\Omega\times X}=1_{\Omega\times
X}$ $\nu$-almost everywhere.

######  Theorem 2.5.

If $(P,\sigma)$ is prior mixing for inhomogeneous observables in
$B(\Omega,L^{\infty}(X,m))$, then $\Theta$ is mixing, that is, for any
$A,B\in\mathcal{F}\otimes\mathcal{A}$,

$\displaystyle\lim_{n\to\infty}\nu(\Theta^{-n}A\cap B)=\nu(A)\nu(B).$ (2.2)

###### Proof.

Let $1_{B_{\omega}}/\mu_{\omega}(B_{\omega})\in D(X,\mu_{\omega})$ so that
$1_{B_{\omega}}h_{\omega}/\mu_{\omega}(B_{\omega})\in D(X,m)$. Assuming that
$(P,\sigma)$ is prior mixing for inhomogeneous observables in
$B(\Omega,L^{\infty}(X,m))$, we then know

$\displaystyle\lim_{n\to\infty}\int_{X}\left(P_{\omega}^{(n)}\left(\frac{1_{B_{\omega}}h_{\omega}}{\mu_{\omega}(B_{\omega})}\right)-h_{\sigma^{n}\omega}\right)1_{A_{\sigma^{n}\omega}}dm=0.$

Let $\widehat{P}_{\omega}:L^{1}(X,\mu_{\omega})\to
L^{1}(X,\mu_{\sigma\omega})$ be the normalized Markov operator defined by

$\widehat{P}_{\omega}f(x)=\begin{cases}\frac{P_{\omega}(fh_{\omega})(x)}{h_{\sigma\omega}(x)}&(x\in
X^{\sigma\omega})\\\ 0&({\rm otherwise}).\end{cases}$

where $X^{\omega}:={\rm supp}h_{\sigma\omega}$. Note that the relation
$\widehat{P}_{\omega}1_{X^{\omega}}=1_{X^{\sigma\omega}}$ holds for almost
every $\omega\in\Omega$. Then we have,

$\lim_{n\to\infty}\left(\int_{X}\widehat{P}^{(n)}_{\omega}1_{B_{\omega}}\cdot
1_{A_{\sigma^{n}\omega}}d\mu_{\sigma^{n}\omega}-\int_{X}1_{A_{\sigma^{n}\omega}}d\mu_{\sigma^{n}\omega}\int_{X}1_{B_{\omega}}d\mu_{\omega}\right)=0.$

The first term can be calculated as

$\displaystyle\int_{X}\widehat{P}^{(n)}_{\omega}1_{B_{\omega}}\cdot
1_{A_{\sigma^{n}\omega}}d\mu_{\sigma^{n}\omega}$ $\displaystyle=$
$\displaystyle\int_{X}(\widehat{P}^{(n)}_{\omega}1_{B_{\omega}})\cdot
h_{\sigma^{n}\omega}\cdot 1_{A_{\sigma^{n}\omega}}dm$ $\displaystyle=$
$\displaystyle\int_{X}P_{\omega}^{(n)}(1_{B_{\omega}}\cdot h_{\omega})\cdot
1_{A_{\sigma^{n}\omega}}dm$ $\displaystyle=$
$\displaystyle\int_{X}1_{B_{\omega}}\cdot h_{\omega}\cdot
P^{*}_{\sigma\omega}\circ\cdots\circ
P^{*}_{\sigma^{n-1}\omega}1_{A_{\sigma^{n}\omega}}dm$ $\displaystyle=$
$\displaystyle\int_{X}1_{B_{\omega}}\cdot P^{*}_{\omega}\circ\cdots\circ
P^{*}_{\sigma^{n-1}\omega}1_{A_{\sigma^{n}\omega}}d\mu_{\omega}$

Thus,

$\displaystyle\lim_{n\to\infty}\left(\int_{X}1_{B_{\omega}}\cdot
P^{*}_{\omega}\circ\cdots\circ
P^{*}_{\sigma^{n-1}\omega}1_{A_{\sigma^{n}\omega}}d\mu_{\omega}-\int_{X}1_{A_{\sigma^{n}\omega}}d\mu_{\sigma^{n}\omega}\int_{X}1_{B_{\omega}}d\mu_{\omega}\right)=0,$
(2.3)

where $P^{*}_{\omega}:L^{\infty}(X,m)\to L^{\infty}(X,m)$ is the Koopman
operator with respect to $T_{\omega}$. Note that this implies the following
natural mixing property for the random dynamical system
$\\{T_{\omega}\\}_{\omega\in\Omega}$,

$\displaystyle\lim_{n\to\infty}\left(\mu_{\omega}\left(T^{(-n)}_{\omega}A_{\sigma^{n}\omega}\cap
B_{\omega}\right)-\mu_{\sigma^{n}\omega}(A_{\sigma^{n}\omega})\mu_{\omega}(B_{\omega})\right)=0,$
(2.4)

where $T^{(-n)}_{\omega}=T^{-1}_{\omega}\circ\cdots\circ
T^{-1}_{\sigma^{n-1}\omega}$. Since $\mu_{\omega}$ is a probability measure,
by the Lebesgue dominated convergence theorem,

$\displaystyle\lim_{n\to\infty}\left(\int_{\Omega}\mu_{\omega}\left(T^{(-n)}_{\omega}A_{\sigma^{n}\omega}\cap
B_{\omega}\right)d\mathbb{P}(\omega)-\int_{\Omega}\mu_{\sigma^{n}\omega}(A_{\sigma^{n}\omega})\mu_{\omega}(B_{\omega})d\mathbb{P}(\omega)\right)=0.$
(2.5)

By using
$\displaystyle\Theta^{-n}(A)=\bigcup_{\omega\in\Omega}(\sigma^{-n}\omega,T^{-1}_{\sigma^{-n}\omega}\circ\cdots\circ
T^{-1}_{\sigma^{-1}\omega}A_{\omega})$ and $\displaystyle
B=\bigcup_{\omega\in\Omega}(\omega,B_{\omega})=\bigcup_{\omega\in\Omega}(\sigma^{-n}\omega,B_{\sigma^{-n}\omega})$,
we have

$\displaystyle\nu(\Theta^{-n}A\cap B)$ $\displaystyle=$
$\displaystyle\int_{\Omega}\mu_{\sigma^{-n}\omega}(T^{-1}_{\sigma^{-n}\omega}\circ\cdots\circ
T^{-1}_{\sigma^{-1}\omega}A_{\omega}\cap
B_{\sigma^{-n}\omega})d\mathbb{P}(\omega)$ $\displaystyle=$
$\displaystyle\int_{\Omega}\mu_{\omega}\left(T^{(-n)}_{\omega}A_{\sigma^{n}\omega}\cap
B_{\omega}\right)d\mathbb{P}(\omega)$

On the other hand, since $\sigma$ is mixing, invertible and
$\mathbb{P}$-preserving,

$\displaystyle\lim_{n\to\infty}\int_{\Omega}\mu_{\omega}(A_{\omega})d\mathbb{P}\int_{\Omega}\mu_{\omega}(B_{\omega})d\mathbb{P}(\omega)$
$\displaystyle=$
$\displaystyle\lim_{n\to\infty}\int_{\Omega}\mu_{\omega}(A_{\omega})L_{\sigma}^{n}(\mu_{\omega}(B_{\omega}))d\mathbb{P}(\omega)$
$\displaystyle=$
$\displaystyle\int_{\Omega}\mu_{\omega}(A_{\omega})d\mathbb{P}\int_{\Omega}\mu_{\omega}(B_{\omega})d\mathbb{P}(\omega)$
$\displaystyle=$ $\displaystyle\nu(A)\nu(B)$

where $L_{\sigma}:L^{1}(\Omega,\mathbb{P})\to L^{1}(\Omega,\mathbb{P})$ is the
Perron–Frobenius operator of $\sigma$. Therefore we obtain
$\nu(\Theta^{-n}A\cap B)\to\nu(A)\nu(B)$ as $n\to\infty$ for
$A,B\in\mathcal{F}\otimes\mathcal{A}$. ∎

######  Remark 2.6.

In the case of prior mixing for homogeneous observables, as in the proof of
Theorem 2.5, we can derive the convergence

$\displaystyle\nu(\Theta^{-n}(F_{1}\times A_{1})\cap(F_{2}\times
A_{2}))\to\nu(F_{1}\times A_{1})\nu(F_{2}\times A_{2})\quad(n\to\infty)$ (2.6)

for any $F_{1},F_{2}\in\mathcal{F}$ and $A_{1},A_{2}\in\mathcal{A}$.

Consequently, due to the relationship summarized in Figure 1, every mixing
considered in this paper imply the conventional mixing for skew-product
transformations since any measurable set in $\mathcal{F}\otimes\mathcal{A}$ is
approximated by countable rectangle sets in $\mathcal{F}\times\mathcal{A}$.
Therefore, we conclude that prior/posterior mixing for (in)homogeneous
observables are natural definitions of mixing for random dynamical systems.

######  Remark 2.7.

When $\sigma$ is an invertible $\mathbb{P}$-preserving mixing transformation,
by considering a skew-product transformation, the conventional definition of
mixing for a random dynamical system $(T,\sigma)$ can be derived from our
definitions of mixing as follows. From the definition of mixing for
homogeneous observables, for any $A_{1},A_{2}\in\mathcal{A}$ and
$\mathbb{P}$-almost every $\omega$,

$\displaystyle\mu_{\omega}\left(T^{(-n)}_{\omega}A_{1}\cap
A_{2}\right)-\mu_{\sigma^{n}\omega}(A_{1})\mu_{\omega}(A_{2})\to
0\quad(n\to\infty).$

On the other hand, from the definition of mixing for inhomogeneous measurable
observables, for any $A,B\in\mathcal{A}\otimes\mathcal{F}$ and
$\mathbb{P}$-almost every $\omega$,

$\displaystyle\mu_{\omega}\left(T^{(-n)}_{\omega}A_{\sigma^{n}\omega}\cap
B_{\omega}\right)-\mu_{\sigma^{n}\omega}(A_{\sigma^{n}\omega})\mu_{\omega}(B_{\omega})\to
0\quad(n\to\infty).$

where $A_{\omega}$ denotes the $\omega$-section of $A$. One can see that the
above two forms of mixing for a random dynamical systems $(T,\sigma)$ are
equivalent.

###  2.4. Problems

We finally give two related problems. Our definitions of mixing tell us only
the convergence, and one of importance for mixing arguments is to evaluate its
decay rate as seen in the previous works [BKS, DFGV, BBR, G]. Thus, we state
the following problem:

######  Problem 2.8.

Evaluate decay rates of mixing for each type of mixing in Theorem 2.1.

Furthermore, the counter example showed in section 2.2 is applicable to prior
mixing, and we have never seen the difference between posterior mixing for
homogeneous observables and inhomogeneous observables in
$B(\Omega,L^{\infty}(X,m))$, or between prior and posterior mixing for
inhomogeneous observables in $B(\Omega,L^{\infty}(X,m))$ yet. Hence, the
following is an also open question.

######  Problem 2.9.

Prove or disprove that posterior mixing for homogeneous observables implies
posterior mixing for inhomogeneous observables in $B(\Omega,L^{\infty}(X,m))$.
Moreover, prove or disprove posterior mixing for inhomogeneous observables in
$B(\Omega,L^{\infty}(X,m))$ implies prior mixing for inhomogeneous observables
in $B(\Omega,L^{\infty}(X,m))$.

##  3\. Exactness

As a characterization of exactness which is well-known for one non-singular
transformation (see [Aa]), we have the generalization of Lin's theorem [L] as
follows.

######  Theorem 3.1.

Let $(P,\sigma)$ be a Markov operator cocycle and $S=\\{g\in
L^{\infty}(X,m):\left\|g\right\|_{L^{\infty}}\leq 1\\}$ the unit ball in
$L^{\infty}(X,m)$. Then the following are equivalent for each
$\omega\in\Omega$.

  1. (1)

$f\in L^{1}(X,m)$ satisfies $\left\lVert
P_{\omega}^{(n)}f\right\rVert_{L^{1}(X)}\to 0$ as $n\to\infty;$

  2. (2)

$f\in L^{1}(X,m)$ satisfies $\int_{X}fgdm=0$ for any $g\in\bigcap_{n\geq
1}P_{\omega}^{*(n)}S$.

Consequently, $(P,\omega)$ is exact if and only if $\bigcap_{n\geq
1}P_{\omega}^{*(n)}S=\\{c1_{X}:c\in\mathbb{R}\\}$ for $\mathbb{P}$-almost
every $\omega\in\Omega$.

###### Proof.

First of all, notice that for any $\omega\in\Omega$, $P_{\omega}^{*}S\subset
S$ and $P_{\omega}^{*(n)}=P_{\omega}^{*}\circ
P_{\sigma\omega}^{*}\circ\dots\circ P_{\sigma^{n-1}\omega}^{*}$ enable us to
have the decreasing sequence in $L^{\infty}(X,m)$:

$\displaystyle S\supset P_{\omega}^{*}S\supset
P_{\omega}^{*(2)}S\supset\dots\supset\bigcap_{n\geq 1}P_{\omega}^{*(n)}S.$

Now we assume (1) is true. Then for each $g\in\bigcap_{n\geq
1}P_{\omega}^{*(n)}S$, there is a sequence $\\{g_{n}\\}_{n}\subset S$ so that
$P_{\omega}^{*(n)}g_{n}=g$ and for $f$ in the condition (1),

$\displaystyle\int_{X}fgdm=\int_{X}fP_{\omega}^{*(n)}g_{n}dm=\int_{X}P_{\omega}^{(n)}fg_{n}dm\leq\left\lVert
P_{\omega}^{(n)}f\right\rVert_{L^{1}(X)}\to 0$

as $n\to\infty$. Thus (2) is valid.

Next, suppose that (2) holds. By the Banach–Alaoglu theorem and continuity of
$P_{\omega}^{*}$ on the weak-* topology in $L^{\infty}(X,m)$, $S$ is compact
in weak-* and so is $P_{\omega}^{*(n)}S$. For $f$ in the condition (2), taking
$g_{n}=\operatorname*{sgn}\left(P_{\omega}^{(n)}f\right)\in S$ where
$\operatorname*{sgn}(\phi)=1$ on $\\{\phi\geq 0\\}$ and
$\operatorname*{sgn}(\phi)=-1$ otherwise, we have

$\displaystyle\left\lVert
P_{\omega}^{(n)}f\right\rVert_{L^{1}(X)}=\int_{X}P_{\omega}^{(n)}fg_{n}dm=\int_{X}fP_{\omega}^{*(n)}g_{n}dm.$

Let $g$ be an accumulation point of
$\left\\{P_{\omega}^{*(n)}g_{n}\right\\}_{n}$ which belongs to $\bigcap_{n\geq
1}P_{\omega}^{*(n)}S$. Then we have $\int_{X}fgdm=0$ by assumption (2) and for
some subsequence $\\{n_{i}\\}_{i}\subset\mathbb{N}$, we have

$\displaystyle\lim_{i\to\infty}\left\lVert
P_{\omega}^{(n_{i})}f\right\rVert_{L^{1}(X)}=\lim_{i\to\infty}\int_{X}fP_{\omega}^{*(n_{i})}g_{n_{i}}dm=\int_{X}fgdm=0.$

Since $P_{\omega}$ is Markov, $\left\lVert
P_{\omega}^{(n)}f\right\rVert_{L^{1}(X)}\leq\left\lVert
P_{\omega}^{(n_{i})}f\right\rVert_{L^{1}(X)}$ for $n\geq n_{i}$. Therefore we
have the condition (1).

Finally, considering the case when $f\in L^{1}_{0}(X,m)$, we have the
equivalent condition for exactness of $(P,\sigma)$ and the proof is completed.
∎

As an immediate corollary of Theorem 3.1, we have:

######  Corollary 3.2.

If a Markov operator cocycle $(P,\sigma)$ is derived from non-singular
transformations $T_{\omega}$, that is, each $P_{\omega}$ is the
Perron–Frobenius operator associated to $T_{\omega}$. Then $(P,\omega)$ is
exact if and only if for $\mathbb{P}$-almost every $\omega\in\Omega$,

$\displaystyle\bigcap_{n\geq
1}\left(T_{\omega}^{n}\right)^{-1}\mathcal{A}=\\{\emptyset,X\\}\pmod{m}$

where $T_{\omega}^{n}\coloneqq T_{\sigma^{n-1}\omega}\circ
T_{\sigma^{n-2}\omega}\circ\dots\circ T_{\omega}$.

###### Proof.

Since $P_{\omega}^{*}$ is the Koopman operator of $T_{\omega}$, characteristic
functions are mapped to characteristic functions. Thus we can consider
$P_{\omega}^{*(n)}$ on $\\{g\in L^{\infty}(X,m):\lVert
g\rVert_{L^{\infty}(X)}=1\\}$ and we prove the corollary. ∎

##  4\. Asymptotic periodicity

In the arguments of conventional Markov operators, it is known that mixing and
exactness are equivalent properties in the asymptotically periodic class [LM].
In this section, we consider a similar result to the conventional one for our
definitions of mixing and exactness for Markov operator cocycles under the
following definition of asymptotic periodicity, which is studied in [NN].
Moreover, in the sequel of the section, we introduce the relation between the
asymptotic periodicity and exactness from the viewpoint of the existence of an
invariant density.

######  Definition 4.1 (Asymptotic periodicity).

A Markov operator cocycle $(P,\sigma)$ is said to be asymptotically periodic
if there exist an integer $r$, finite collections
$\\{\lambda_{i}\\}_{i=1}^{r}\subset
B\left(\Omega,(L^{1}(X,m))^{\prime}\right)$ and $\\{g_{i}\\}_{i=1}^{r}\subset
B\left(\Omega,D(X,m)\right)$ satisfying that $\\{g_{i}^{\omega}\\}_{i=1}^{r}$
have mutually disjoint supports for $\mathbb{P}$-almost every
$\omega\in\Omega$, and there exists a permutation $\rho_{\omega}$ of
$\\{1,\dots,r\\}$ such that

$\displaystyle
P_{\omega}g_{i}^{\omega}=g_{\rho_{\omega}(i)}^{\sigma\omega}\quad\text{and}\quad\lim_{n\to\infty}\left\lVert
P_{\omega}^{(n)}\left(f-\sum_{i=1}^{r}\lambda_{i}^{\omega}(f)g_{i}^{\omega}\right)\right\rVert_{L^{1}(X)}=0$
(4.1)

for every $f\in L^{1}(X,m)$, $1\leq i\leq r$ and $\mathbb{P}$-almost every
$\omega\in\Omega$, where $\lambda_{i}^{\omega}=\lambda_{i}(\omega)$,
$g_{i}^{\omega}=g_{i}(\omega)$ and
$\rho_{\omega}^{n}:=\rho_{\sigma^{n-1}\omega}\circ\cdots\circ\rho_{\omega}$.

Furthermore, if in addition $r=1$, then $(P,\sigma)$ is said to be
asymptotically stable.

Note that when $(P,\sigma)$ is asymptotically periodic,

$\displaystyle h_{\omega}=\frac{1}{r}\sum_{i=1}^{r}g_{i}^{\omega}$

becomes an invariant density for $(P,\sigma)$.

For an asymptotically periodic single Markov operator, exactness and mixing
coincide with $r=1$ for the representation of asymptotic periodicity (see
Theorem 5.5.2 and 5.5.3 in [LM]). The following theorem and proposition are
Markov operator cocycles version of them.

######  Theorem 4.2.

Let $(P,\sigma)$ be an asymptotically periodic Markov operator cocycle. Then
the followings are equivalent.

  1. (1)

$(P,\sigma)$ is exact;

  2. (2)

$(P,\sigma)$ is prior mixing for inhomogeneous observables in
$B(\Omega,L^{\infty}(X))$;

  3. (3)

$(P,\sigma)$ is posterior mixing for inhomogeneous observables in
$B(\Omega,L^{\infty}(X))$;

  4. (4)

$(P,\sigma)$ is asymptotically stable.

###### Proof.

(1) $\Rightarrow$ (2) $\Rightarrow$ (3): Obvious.

(3) $\Rightarrow$ (4): Suppose $(P,\sigma)$ is posterior mixing for
inhomogeneous observables in $B(\Omega,L^{\infty}(X,m))$ and $r>1$ (recall
that $r$ is the period of the asymptotically periodic Markov operator cocycle
$(P,\sigma)$ given in Definition 4.1). By asymptotic periodicity of
$(P,\sigma)$, we have an invariant density
$h_{\omega}=\frac{1}{r}\sum_{i=1}^{r}g_{i}^{\omega}$. Set
$\varphi_{\omega}=1_{\operatorname*{supp}g_{1}^{\omega}}$ and write

$\displaystyle\int_{X}P_{\omega}^{(n)}\left(h_{\omega}-g_{1}^{\omega}\right)\cdot\varphi_{\sigma^{n}\omega}dm$
$\displaystyle=\int_{\operatorname*{supp}g_{1}^{\sigma^{n}\omega}}\left(h_{\sigma^{n}\omega}-g_{\rho_{\omega}^{n}(1)}^{\sigma^{n}\omega}\right)dm$
$\displaystyle=\frac{1}{r}-\int_{\operatorname*{supp}g_{1}^{\sigma^{n}\omega}}g_{\rho_{\omega}^{n}(1)}^{\sigma^{n}\omega}dm$
$\displaystyle=\begin{cases}\frac{1}{r}&(\rho_{\omega}^{n}(1)\neq 1)\\\
\frac{1}{r}-1&(\rho_{\omega}^{n}(1)=1)\end{cases}$

for each $n\geq 0$. This contradicts posterior mixing for inhomogeneous
observables in $B(\Omega,L^{\infty}(X))$ of $(P,\sigma)$.

(4) $\Rightarrow$ (1): We assume $(P,\sigma)$ is asymptotically periodic with
$r=1$. That is, for any $f\in L^{1}(X,m)$, $\left\lVert
P^{(n)}_{\omega}(f-\lambda^{\omega}(f)g^{\omega})\right\rVert_{L^{1}(X)}\to 0$
as $n\to\infty$. Since $P_{\omega}$ is Markov and $g_{\omega}\in D(X,m)$ for
each $\omega\in\Omega$, for any $f\in D(X,m)$ we have $\lambda^{\omega}(f)=1$.
Thus $(P,\sigma)$ is exact by Remark 1.3. ∎

The following proposition reveals the relationship between two kinds of mixing
and exactness. Namely, in the setting of asymptotically periodic systems
mixing for homogeneous observables, mixing for inhomogeneous measurable
observables and exactness are equivalent under certain topological assumption
of $\Omega$.

######  Proposition 4.3.

Let $(P,\sigma)$ be an asymptotically periodic Markov operator cocycle.
Suppose $\sigma$ preserves a regular probability measure $\mathbb{P}$ on a
metric space $(\Omega,\mathcal{F},\mathbb{P})$ with metric $d_{\Omega}$ and
for $\mathbb{P}$-almost every $\omega\in\Omega$, each component of invariant
densities $g_{i}^{\omega}$ belongs to $C(\Omega,L^{\infty}(X,m))$. Then the
condition that $(P,\sigma)$ is prior mixing for homogeneous observables is
necessary and sufficient for each condition in Theorem 4.2.

###### Proof.

Necessity: obvious.

Sufficiency: we show $r=1$ in the representation of asymptotic periodicity.
Assume $r>1$ contrarily. By our assumption, for almost every $\omega\in\Omega$
and any $\epsilon>0$ there exists $\delta>0$ such that
$d_{\Omega}(\omega,\omega^{\prime})<\delta$ implies $\left\lVert
g_{i}^{\omega}-g_{i}^{\omega^{\prime}}\right\rVert_{L^{1}(X)}<\epsilon$ for
$i=1,\dots,r$. Also, we have that for $i\neq j$, if
$d_{\Omega}(\omega,\omega^{\prime})<\delta$ then

$\displaystyle\int_{\operatorname*{supp}g_{j}^{\omega}}g_{i}^{\omega^{\prime}}dm\leq\left\lVert
g_{i}^{\omega^{\prime}}-g_{i}^{\omega}\right\rVert_{L^{1}(X)}+\int_{\operatorname*{supp}g_{j}^{\omega}}g_{i}^{\omega}dm<\epsilon.$

Set a $\delta$-ball $B_{\delta}(\omega)$ centered at a given $\omega\in\Omega$
which is of positive measure since $\mathbb{P}$ is regular. Poincaré's
recurrence theorem tells us that $\sigma^{n}\omega$ visits
$B_{\delta}(\omega)$ infinitely many times and let $\\{n_{k}\\}_{k}$ satisfy
$\sigma^{n_{k}}\omega\in B_{\delta}(\omega)$. Then for an invariant density
$h_{\omega}=\frac{1}{r}\sum_{i=1}^{r}g_{i}^{\omega}$ and for
$A=\operatorname*{supp}g_{1}^{\omega}$, if $\omega^{\prime}$ satisfies
$d_{\Omega}(\omega,\omega^{\prime})<\delta$,

$\displaystyle\left\lvert\int_{A}h_{\omega^{\prime}}dm\right\rvert$
$\displaystyle=\frac{1}{r}\left\lvert\int_{A}\left(g_{1}^{\omega^{\prime}}-g_{1}^{\omega}+g_{1}^{\omega}\right)dm+\sum_{i\neq
1}\int_{A}g_{i}^{\omega^{\prime}}dm\right\rvert$

and we have

$\displaystyle\frac{1}{r}-\epsilon\leq\left\lvert\int_{A}h_{\omega^{\prime}}dm\right\rvert\leq\frac{1}{r}+\epsilon.$

Therefore, taking $0<\epsilon<\frac{1}{2r}$ we have

$\displaystyle\left\lvert\int_{A}P^{(n_{k})}_{\omega}\left(g_{1}^{\omega}-h_{\omega}\right)dm\right\rvert$
$\displaystyle=\left\lvert\int_{A}\left(g_{\rho_{\omega}^{n_{k}}(1)}^{\sigma^{n_{k}}\omega}-h_{\sigma^{n_{k}}\omega}\right)dm\right\rvert$
$\displaystyle\geq\begin{cases}\frac{1}{r}-2\epsilon&(\rho_{\omega}^{n_{k}}(1)\neq
1)\\\ (1-2\epsilon)-\frac{1}{r}&(\rho_{\omega}^{n_{k}}(1)=1)\end{cases}$
$\displaystyle>0.$

This contradicts prior mixing of $(P,\sigma)$ for homogeneous observables. ∎

######  Remark 4.4.

According to [NN], The assumptions of Proposition 4.3 are automatically
satisfied if $\Omega$ is a compact and connected metric space, $\sigma$ is a
minimal homeomorphism probability preserving, $P$ is strongly continuous
(i.e., $\omega\mapsto P_{\omega}f$ is continuous for any $f\in L^{1}(X,m)$),
and $(P,\sigma)$ is weakly constrictive admitting a bounded continuous
invariant density.

In order to relate asymptotic periodicity and exactness together with the
existence of an invariant density, we give the definition of quasi-
constrictiveness.

######  Definition 4.5.

Let $(P,\sigma)$ be a Markov operator cocycle. Then $(P,\sigma)$ is called
quasi-constrictive if for any $\epsilon>0$ there exists $\delta>0$ such that
for any $E\in\mathcal{A}$ with $m(E)<\delta$, it holds that

$\displaystyle\operatorname*{lim\ sup}_{n\to\infty}\operatorname*{ess\
sup}_{\omega\in\Omega}\int_{E}P_{\omega}^{(n)}fdm<\epsilon\quad\text{for any
}f\in D(X,m).$

For the sake of convenience, for an asymptotically periodic Markov operator
cocycle $(P,\sigma)$ and each component of the invariant density
$g_{i}^{\omega}$, we denote a measurable map $\omega\mapsto
P_{\omega}^{(k)}\mid_{\operatorname*{supp}g_{i}^{\omega}}$ by
$P^{(k)}\mid_{\operatorname*{supp}g_{i}}$ for $k\in\mathbb{N}$ and
$i=1,\dots,r$. In the following two propositions, we can see that (i):
asymptotic periodicity of $(P,\sigma)$ is equivalent to (ii): the existence of
an invariant density and exactness of
$(P^{(k)}\mid_{\operatorname*{supp}g_{i}},\sigma^{k})$ for some
$k\in\mathbb{N}$ and all $i=1,\dots,r$.

######  Proposition 4.6.

Let $(P,\sigma)$ be a Markov operator cocycle such that $P$ is strongly
continuous i.e., $\omega\mapsto P_{\omega}f$ is continuous for each $f\in
L^{1}(X,m)$. Suppose $\Omega$ is compact, $(P,\sigma)$ has an invariant
density $h_{\omega}$ and $\\{\mu_{\omega}\\}_{\omega\in\Omega}$ is uniformly
absolutely continuous with respect to $m$ where $d\mu_{\omega}=h_{\omega}dm$.
If $(P^{(k)},\sigma^{k})$ is exact for some $k\in\mathbb{N}$, then
$(P,\sigma)$ is quasi-constrictive.

###### Proof.

By the assumption, $(P,\sigma)$ admits an invariant density
$\\{h_{\omega}\\}_{\omega}$ and there exists $k\in\mathbb{N}$ such that, for
any $\varphi\in L^{1}_{0}(X,m)$ and almost every $\omega$, $\left\lVert
P^{(kn)}_{\omega}\varphi\right\rVert_{L^{1}(X)}\to 0$ as $n\to\infty$. We show
$(P,\sigma)$ is quasi-constrictive. For any $N$ sufficiently large, $f\in
D(X,m)$ and $E\in\mathcal{A}$,

$\displaystyle\sup_{n>N}\operatorname*{ess\
sup}_{\omega\in\Omega}\int_{E}P_{\omega}^{(n)}fdm$ $\displaystyle=$
$\displaystyle\sup_{n_{0}k\geq N}\operatorname*{ess\
sup}_{\omega\in\Omega}\left\\{\int_{E}P_{\omega}^{(n_{0}k)}fdm,\int_{E}P_{\omega}^{(n_{0}k+1)}fdm,\dots,\int_{E}P_{\omega}^{((n_{0}+1)k-1)}fdm\right\\}.$

For $j\in\\{0,\dots,k-1\\}$, we have

$\displaystyle\int_{E}P_{\omega}^{(n_{0}k+j)}fdm$
$\displaystyle=\int_{E}P_{\sigma^{j}\omega}^{(n_{0}k)}P_{\omega}^{(j)}fdm$
$\displaystyle\leq\left\|P_{\sigma^{j}\omega}^{(n_{0}k)}\left(P_{\omega}^{(j)}f-h_{\sigma^{j}\omega}\right)\right\|_{L^{1}(X)}+\int_{E}P_{\sigma^{j}\omega}^{(n_{0}k)}h_{\sigma^{j}\omega}dm$
$\displaystyle=\left\|P_{\sigma^{j}\omega}^{(n_{0}k)}\left(P_{\omega}^{(j)}f-h_{\sigma^{j}\omega}\right)\right\|_{L^{1}(X)}+\int_{E}h_{\sigma^{n_{0}k+j+1}\omega}dm.$

Thus, since $P_{\omega}^{(j)}f-h_{\sigma^{j}\omega}\in L^{1}_{0}(X,m)$ and
$\Omega$ is compact, we have

$\displaystyle\operatorname*{lim\ sup}_{n\to\infty}\operatorname*{ess\
sup}_{\omega\in\Omega}\int_{E}P_{\omega}^{(n)}fdm$ $\displaystyle\leq$
$\displaystyle\operatorname*{lim\ sup}_{n_{0}\to\infty}\operatorname*{ess\
sup}_{\omega\in\Omega}\max_{0\leq j\leq
k-1}\left\|P_{\sigma^{j}\omega}^{(n_{0}k)}\left(P_{\omega}^{(j)}f-h_{\sigma^{j}\omega}\right)\right\|_{L^{1}(X)}$
(4.2) $\displaystyle\qquad+\operatorname*{lim\
sup}_{n_{0}\to\infty}\operatorname*{ess\ sup}_{\omega\in\Omega}\max_{0\leq
j\leq k-1}\mu_{\sigma^{n_{0}k+j+1}\omega}(E)$ $\displaystyle\leq$
$\displaystyle\operatorname*{ess\ sup}_{\omega\in\Omega}\mu_{\omega}(E).$

Indeed, if $\displaystyle\operatorname*{lim\
sup}_{n_{0}\to\infty}\operatorname*{ess\ sup}_{\omega\in\Omega}\max_{0\leq
j\leq
k-1}\left\|P_{\sigma^{j}\omega}^{(n_{0}k)}\left(P_{\omega}^{(j)}f-h_{\sigma^{j}\omega}\right)\right\|_{L^{1}(X)}\neq
0$, there exist $\varphi\in L^{1}_{0}(X,m)$, $\epsilon_{0}>0$,
$\\{\omega_{l}\\}_{l}\subset\Omega$ and $\\{n_{l}\\}_{l}\subset\mathbb{N}$
such that
$\left\|P_{\omega_{l}}^{(n_{l}k)}\varphi\right\|_{L^{1}(X)}\geq\epsilon_{0}$.
Compactness of $\Omega$ ensures that there exists further subsequences
$\omega^{\prime}_{s}=\omega_{l_{s}}$ and $n^{\prime}_{s}=n_{l_{s}}$ such that
$\omega^{\prime}_{s}\to\bar{\omega}$ for some $\bar{\omega}\in\Omega$ as $s$
tends to $\infty$. Now strong continuity of $P$ and exactness of $P^{k}$ imply
that

$\displaystyle\left\|P_{\omega^{\prime}_{s}}^{(n^{\prime}_{s}k)}\varphi\right\|_{L^{1}(X)}\leq\left\|P_{\omega^{\prime}_{s}}^{(n^{\prime}_{s}k)}\varphi-
P_{\bar{\omega}}^{(n^{\prime}_{s}k)}\varphi\right\|_{L^{1}(X)}+\left\|P_{\bar{\omega}}^{(n^{\prime}_{s}k)}\varphi\right\|_{L^{1}(X)}\to
0$

as $s\to\infty$ and this leads contradiction.

Uniform absolute continuity of $\\{\mu_{\omega}\\}_{\omega}$ with respect to
$m$ implies that for each $\epsilon>0$ there exists $\delta>0$ such that if
$m(E)<\delta$ then (4.2) is less than $\epsilon$. Therefore the desired result
is obtained. ∎

######  Proposition 4.7.

Let $(P,\sigma)$ be an asymptotically periodic Markov operator cocycle such
that the permutation $\rho_{\omega}\equiv\rho$ in Definition 4.1 is constant
$\mathbb{P}$-almost everywhere. Then $(P,\sigma)$ admits an invariant density
and there exists a natural number $k$ such that
$(P^{(k)}\mid_{\operatorname*{supp}g_{i}},\sigma^{k})$ is exact for
$i=1,\dots,r$.

###### Proof.

Obviously $h_{\omega}=\frac{1}{r}\sum_{i=1}^{r}g_{i}^{\omega}$ is an invariant
density for $P_{\omega}$.

Now let $k$ be the smallest number satisfying $\rho^{k}=\mathrm{id}$. Then,
setting $A_{i}^{\omega}=\operatorname*{supp}g_{i}^{\omega}$,
$P^{(k)}_{\omega}\mid_{A_{i}^{\omega}}$ is a Markov operator from
$L^{1}(A_{i}^{\omega},m)$ into $L^{1}(A_{i}^{\sigma^{k}\omega},m)$. By
representation of asymptotic periodicity of $P_{\omega}$, for any $f\in
D(A_{i}^{\omega},m)$ we have that

$\displaystyle\lambda_{j}^{\omega}(f)=\begin{cases}1&(j=i)\\\ 0&(j\neq
i).\end{cases}$

This implies that for any $f\in D(A_{i}^{\omega},m)$,

$\displaystyle\lim_{n\to\infty}\left\lVert
P_{\omega}^{(nk)}\left(f-g_{i}^{\omega}\right)\right\rVert_{L^{1}(X)}=0.$

Therefore we conclude $(P^{(k)}\mid_{\operatorname*{supp}g_{i}},\sigma^{k})$
is exact by Remark 1.3. ∎

######  Remark 4.8.

As a consequence of Proposition 4.6 and Proposition 4.7 together with the
results in [NN], we have the following.

  1. (1)

If $(P,\sigma)$ admits an invariant density bounded below and above and
$(P^{(k)},\sigma^{k})$ is exact for some $k\geq 1$, then $(P,\sigma)$ is
asymptotically periodic.

  2. (2)

Conversely, if $(P,\sigma)$ is asymptotically periodic, then $(P,\sigma)$
admits an invariant density and there exists $k\geq 1$ such that
$(P^{(k)}\mid_{\operatorname*{supp}g_{i}},\sigma^{k})$ is exact for
$i=1,\dots,r$.

In particular, $(P,\sigma)$ is asymptotically periodic with period 1 if and
only if $(P,\sigma)$ admits a unique invariant density and is exact.

## Acknowledgement

The authors would like to express our gratitude to Professor Michiko Yuri
(Hokkaido university) for giving us constructive suggestions and warm
encouragement. This work was partially supported by JSPS KAKENHI Grant Number
19K14575.

## References

  * [Aa] Aaronson, J.,  _An Introduction to Infinite Ergodic Theory_ ,  Mathematical Surveys and Monographs, 50. Amer. Math. Soc., (1997).
  * [Ar] Arnold, L.,  _Random dynamical systems_ ,  Springer, Berlin, Heidelberg, (1995).
  * [B] Buzzi, J.,  Exponential decay of correlations for random Lasota–Yorke maps,  _Communications in mathematical physics_ , 208(1) (1999), 25–54.
  * [BBR] Bahsoun, W., Bose, C. and Ruziboev, M.,  Quenched decay of correlations for slowly mixing systems,  _Trans. Amer. Math. Soc._ , 372 (2019), 6547–6587.
  * [BKS] Baladi, V., Kondah, A., and Schmitt, B.,  Random correlations for small perturbations of expanding maps,  _Random and Computational Dynamics_ 4 (1996), 179–204.
  * [BY] Baladi, V., and Young, L.-S.,  On the spectra of randomly perturbed expanding maps,  _Communications in Mathematical Physics_ 156(2) (1993), 355–385.
  * [DFGV] Dragičević, D., Froyland, G., González-Tokman, C., and Vaienti, S.,  A spectral approach for quenched limit theorems for random expanding dynamical systems,  _Communications in Mathematical Physics_ , 360(3) (2018), 1121–1187.
  * [F] Foguel, Shaul R.,  _The ergodic theory of Markov processes_ ,  Van Nostrand Mathematical Studies, No. 21. Van Nostrand Reinhold Co., New York-Toronto, Ont.-London (1969), v+102 pp.
  * [FLQ] Froyland, G., Lloyd, S., and Quas, A.,  A semi-invertible Oseledets theorem with applications to transfer operator cocycles,  _DYNAMICAL SYSTEMS_ 33.9 (2013), 3835–3860.
  * [GQ] González-Tokman, C., and Quas, A.,  A semi-invertible operator Oseledets theorem,  _Ergodic Theory and Dynamical Systems_ , 34.4 (2014), 1230–1272.
  * [G] Gundlach, V.,  Thermodynamic formalism for random subshifts of finite type,  Report 385, Institut für Dynamische Systeme, Universität Bremen (1996).
  * [L] Lin, M.,  Mixing for Markov operators,  _Z. Wahrscheinlichkeitstheorie und Verw. Gebiete_ 19 (1971), 231–242.
  * [LM] Lasota, A. and Mackey, M. C.,  _Chaos, fractals, and noise. Stochastic aspects of dynamics. Second edition._ Applied Mathematical Sciences, 97. Springer-Verlag, New York, (1994).
  * [NN] Nakamura, F. and Nakano, Y.,  Asymptotic periodicity of Markov operators in random environments,  _in preperation_.

Figure 1. The relations between definitions in this paper. Here $B$ and $C$
are abbreviations of $B(\Omega,L^{\infty}(X,m))$ and
$C(\Omega,L^{\infty}(X,m))$, respectively. The implications by a gray arrow
represent trivial relations by definitions. From the figure, we can see that
the above four definitions are all equivalent. Moreover, the below four
definitions are equivalent for an asymptotically periodic Markov operator
cocycle. Furthermore, if $\Omega$ is connected, then all of them are
equivalent.


# Runtime Monitoring and Fault Detection for Neural Network-Controlled Systems

Jianglin Lan  Siyuan Zhan  Ron Patton  Xianxian Zhao  James Watt School of
Engineering, University of Glasgow, Glasgow G12 8QQ, UK (e-mail:
Jianglin.Lan@glasgow.ac.uk).  Department of Mechanical, Manufacturing and
Biomedical Engineering, Trinity College Dublin, Dublin 2, Ireland (e-mail:
zhans@tcd.ie).  School of Engineering, University of Hull, Hull HU6 7RX, UK
(e-mail: R.J.Patton@hull.ac.uk).  School of Electrical and Electronic
Engineering, University College Dublin, Belfield, D04 V1W8 Dublin, Ireland
(xianxian.zhao@ucd.ie).

###### Abstract

There is an emerging trend in applying deep learning methods to control
complex nonlinear systems. This paper considers enhancing the runtime safety
of nonlinear systems controlled by neural networks in the presence of
disturbance and measurement noise. A robustly stable interval observer is
designed to generate sound and precise lower and upper bounds for the neural
network, nonlinear function, and system state. The obtained interval is
utilised to monitor the real-time system safety and detect faults in the
system’s outputs or actuators. An adaptive cruise control vehicular system is
simulated to demonstrate effectiveness of the proposed design.

###### keywords:

Safety, neural network, observer, fault detection, intelligent autonomous
vehicles.

††thanks: This work was supported in part by the Leverhulme Trust Early Career
Fellowship under Award ECF-2021-517, and in part by the UK Royal Society
International Exchanges Programme under Award
IES$\backslash$R3$\backslash$223168\. Xianxian Zhao was supported by the SEAI
(Sustainable Energy Authority of Ireland) under RD&D Award
22$\backslash$RDD$\backslash$776.

, , ,

##  1 Introduction

Machine learning techniques including deep neural networks (NNs) are powerful
for modelling and controlling complex nonlinear systems such as robots and
autonomous vehicles (Moe et al., 2018; Tang et al., 2023). However, NNs are
vulnerable to input perturbations such as noise and adversarial attacks. This
is even more problematic when NNs are used to generate real-time control
actions for automatic systems such as aircraft (Julian and Kochenderfer,
2021), because uncertainties (or deviations) in the NN will be propagated and
accumulated the closed-loop, leading to degraded performance and safety
concerns (Bensalem et al., 2023). It is thus important to assure real-time
safety of NN-controlled systems.

Safety assurance for NN-controlled autonomous systems has been looked at from
different angles in the literature. Lots of research has been devoted to
formal methods for verifying the robustness of NNs against perturbations (Liu
et al., 2021). The formal methods are normally based on interval bound
propagation and the solving of optimisation problems such as mixed-integer
linear programming (MILP) (Lomuscio and Maganti, 2017), semidefinite
programming (SDP) (Lan et al., 2023), and linear programming (LP) (Bunel et
al., 2020). However, these works consider NNs as stand-alone components for
image classification, natural language processing, etc. Some researchers have
considered verifying safety of the NN-controlled systems through reachability
analysis. Given an initial state set, they use methods, such as MILP (Karg and
Lucia, 2020), SDP (Hu et al., 2020), LP (Everett et al., 2021), or constrained
zonotopes (Zhang and Xu, 2022), to compute the reachable set which contains
all the possible future system trajectories and check whether this set is
within the safe region or not. However, reachability analysis is
computationally expensive and has been conducted offline. This method is thus
unable to account for system changes due to external disturbances and
measurement noise or detect system security issues in a timely manner.

This paper is dedicated to runtime monitoring of NN-controlled autonomous
systems to assure their safety. Cofer et al. (2020) propose a rule-based
method for monitoring NN-based aircraft taxiing, using a set of monitors to
measure the aircraft’s relative position to the runway and NN’s deviation from
its training dataset. Xiang (2021) and Wang et al. (2023) develop observers to
monitor the state of continuous-time NN-controlled nonlinear systems by taking
inspiration from the interval observer technique (Efimov et al., 2013).
However, their works do not consider system disturbance or measurement noise,
and assume the existence of an interval for the nonlinear function without
providing a way to derive it. They adopt the auxiliary network (AN) method to
compute the output bounds of NNs, which is applicable to a wide range of
activation functions including the most widely used Rectified Linear Unit
(ReLU) and tanh (Dubey et al., 2022), but its conservatism results in loose
intervals of the system trajectories.

This paper proposes a new interval observer for discrete-time nonlinear
systems under disturbance and measurement noise, where the controller is a
deep NN with ReLU activations. The proposed observer design is advantageous
over the existing methods (Xiang, 2021; Wang et al., 2023) in three aspects:
(i) it uses an MILP optimisation approach to obtain the tightest NN bounds,
avoiding the conservatism introduced by the AN method. (ii) it provides a
systematic way to construct the upper and lower bounds of the nonlinear
function; (iii) it ensures that the generated state interval is robust against
external disturbance and measurement noise.

##  2 Problem description and preliminaries

Consider a discrete-time system modelled by

$\displaystyle x_{t+1}$ $\displaystyle=Ax_{t}+Bu_{t}+Fg(x_{t})+w_{t},$ (1a)
$\displaystyle y_{t}$ $\displaystyle=Cx_{t}+v_{t},$ (1b)

where $t$ is the sampling time. $x_{t}\in\mathbb{R}^{n}$,
$u_{t}\in\mathbb{R}^{m}$, $w_{t}\in\mathbb{R}^{n}$, $y_{t}\in\mathbb{R}^{p}$,
and $v_{t}\in\mathbb{R}^{p}$ are the vectors of system state, control inputs,
external disturbances, measured outputs, and noise, respectively.
$g(x_{t})\in\mathbb{R}^{s}$ is the system nonlinearity.
$A\in\mathbb{R}^{n\times n}$, $B\in\mathbb{R}^{n\times m}$,
$F\in\mathbb{R}^{n\times s}$, and $C\in\mathbb{R}^{p\times n}$ are known
constant matrices. In this paper, $u_{t}=f_{\text{nn}}(r_{t},y_{t}^{o})$,
where $r_{t}\in\mathbb{R}^{q}$ is the vector of known reference signals and
$y_{t}^{o}=C_{o}y_{t}$ which takes partial $y_{t}$ using
$C_{o}=[\mathbf{0}_{\ell\times p_{1}}~{}I_{\ell}~{}\mathbf{0}_{\ell\times
p-\ell-p_{1}}]$ with the integers $\ell\geq 1$ and $p_{1}\geq 0$.
$f_{\text{nn}}\in\mathbb{R}^{m}$ is a $L$-layer feedforward NN given by:

$\displaystyle z_{0}$
$\displaystyle=[r_{t};y_{t}^{o}],~{}z_{i}=\phi(W_{i}z_{i-1}+b_{i}),~{}i\in[1,L-1],$
$\displaystyle f_{\text{nn}}(z_{0})$ $\displaystyle=W_{L}z_{L-1}+b_{L},$

where $W_{i}$ and $b_{i}$ are the weight and bias of the $i$-th hidden layer,
respectively. $\phi(\cdot)$ is the ReLU activation function with the form
$\phi(\hat{z}_{i})=\max(\hat{z}_{i},0)$, where
$\hat{z}_{i}=W_{i}z_{i-1}+b_{i}$ and the operation $\max$ is applied
elementwise.

This paper aims to design a robustly stable observer that can generate a
precise interval for the state trajectory for runtime safety monitoring and
detection of actuator or sensor faults.

In this paper, the following notations are defined for a matrix $X$:
$X^{+}=\max\\{0,X\\}$, $X^{-}=X^{+}-X$, and $|X|=X^{+}+X^{-}$. The observer
design is based on Assumption 1 and will use Lemma 2.

###### Assumption 1

The pair $(A,C)$ is observable. $x_{0}\in[\underline{x}_{0},\bar{x}_{0}]$ for
some known constant vector $\underline{x}_{0}$ and $\bar{x}_{0}$.
$w_{t}\in[\underline{w}_{t},\bar{w}_{t}]$, and
$v_{t}\in[\underline{v}_{t},\bar{v}_{t}]$ for all $t\geq 0$, where
$\underline{w}_{t}$, $\bar{w}_{t}$, $\underline{v}_{t}$, and $\bar{v}_{t}$ are
known bounded signals.

###### Lemma 2

(Efimov et al., 2013) Given a vector $x\in\mathbb{R}^{n}$ satisfying
$x\in[\underline{x},\bar{x}]$ and a constant matrix $X\in\mathbb{R}^{m\times
n}$, it holds that $X^{+}\underline{x}-X^{-}\bar{x}\leq Xx\leq
X^{+}\bar{x}-X^{-}\underline{x}.$

##  3 Interval Observer Design

The proposed interval observer needs the intervals for $x_{0}$, $w_{t}$,
$v_{t}$, $f_{\text{nn}}(y^{o}_{t},r_{t})$, and $g(x(t))$. The intervals for
$x_{0}$, $w_{t}$ and $v_{t}$ are known from Assumption 1, while those for
$f_{\text{nn}}(r_{t},y^{o}_{t})$ and $g(x(t))$ are computed below.

Interval of the NN Controller. Let the output $y_{t}^{o}$ satisfy
$y_{t}^{o}\in[\underline{y^{o}}_{t},\overline{y^{o}}_{t}]$. It follows from
Assumption 1 and Lemma 2 that
$\underline{y^{o}}_{t}=C_{o}(C^{+}\underline{x}_{t}-C^{-}\overline{x}_{t}+\underline{v}_{t})$
and
$\overline{y^{o}}_{t}=C_{o}(C^{+}\overline{x}_{t}-C^{-}\underline{x}_{t}+\bar{v}_{t}),$
where $[\underline{x}_{t},\overline{x}_{t}]$ is the interval of $x_{t}$ to be
generated by the proposed interval observer. Suppose that the NN controller
$f_{\text{nn}}(z_{0})$ is bounded as
$f_{\text{nn}}(z_{0})\in[\underline{f}_{\text{nn}}(t),\overline{f}_{\text{nn}}(t)]$
for the input region $z_{0}\in[\underline{z}_{0},\overline{z}_{0}]$, where
$\underline{z}_{0}=[r_{t};\underline{y^{o}}_{t}]$ and
$\overline{z}_{0}=[r_{t};\overline{y^{o}}_{t}]$. Two methods are described
below to compute the NN interval
$[\underline{f}_{\text{nn}}(t),\overline{f}_{\text{nn}}(t)]$.

The auxiliary network (AN) method described in (Xiang, 2021, Theorem 1)
computes the NN interval as follows:

$\displaystyle\left\\{\begin{array}[]{l}\underline{z}_{i}=\phi(\underline{W}_{i}\overline{z}_{i-1}+\overline{W}_{i}\underline{z}_{i-1}+b_{i}),~{}i\in[1,L-1]\\\
\underline{f}_{\text{nn}}(t)=\underline{W}_{i}\overline{z}_{i-1}+\overline{W}_{i}\underline{z}_{i-1}+b_{i}\end{array}\right.,$
(2c)
$\displaystyle\left\\{\begin{array}[]{l}\overline{z}_{i}=\phi(\underline{W}_{i}\underline{z}_{i-1}+\overline{W}_{i}\overline{z}_{i-1}+b_{i}),~{}i\in[1,L-1]\\\
\overline{f}_{\text{nn}}(t)=\underline{W}_{i}\underline{z}_{i-1}+\overline{W}_{i}\overline{z}_{i-1}+b_{i}\end{array}\right.,$
(2f)

where $\overline{W}_{i}=W_{i}^{+}$ and $\underline{W}_{i}=-W_{i}^{-}$. This
method ignores the coupling among neurons, resulting in a loose NN interval
that will reduce the precision of the state trajectory intervals to be
generated by the interval observer.

In this paper, an optimisation method (OP) is proposed to compute a tighter
(the best) NN interval by considering the coupling among all neurons. Given
the interval of $z_{0}$, the pre-activation bounds $\hat{l}_{i}$ and
$\hat{u}_{i}$ of the $i$-th hidden layer, $i\in[1,L-1]$, can be computed using
the interval arithmetic method as follows (Liu et al., 2021):

$\displaystyle\begin{split}l_{0}&=\underline{z}_{0},~{}u_{0}=\overline{z}_{0},\\\
\hat{l}_{i}&=\overline{W}l_{i-1}+\underline{W}_{i}u_{i-1}+b_{i},~{}i\in[1,L-1],\\\
\hat{u}_{i}&=\underline{W}_{i}l_{i-1}+\overline{W}_{i}u_{i-1}+b_{i},~{}i\in[1,L-1],\\\
l_{i}&=\phi(\hat{l}_{i}),~{}u_{i}=\phi(\hat{u}_{i}),~{}i\in[1,L-2].\end{split}$
(3)

The $j$-th ReLU neuron at the $i$-th hidden layer
$z_{i,j}=\phi(W_{i}(j,:)z_{i-1}+b_{i}(j))$ is equivalently represented by the
following set of inequalities:

$\displaystyle\begin{split}&\hat{z}_{i,j}=W_{i}(j,:)z_{i-1}+b_{i}(j),\\\
&z_{i,j}\geq\hat{z}_{i,j},~{}z_{i,j}\geq
0,~{}z_{i,j}\leq\sigma_{i,j}\hat{u}_{i,j},\\\
&z_{i,j}\leq\hat{z}_{i,j}-(1-\sigma_{i,j})\hat{l}_{i,j},~{}\sigma_{i,j}\in\\{0,1\\},\end{split}$
(4)

where $\hat{l}_{i,j}$ and $\hat{u}_{i,j}$ are the lower and upper pre-
activation bounds of $\hat{z}_{i,j}$ computed in (3). The binary variable
$\sigma_{i,j}$ indicates the status of the neuron: When $\sigma_{i,j}=1$, the
neuron is active and the above set of inequalities revert to be
$z_{i,j}=\hat{z}_{i,j}$; when $\sigma_{i,j}=0$, the neuron is inactive and the
set of inequalities in (4) revert to be $z_{i,j}=0$.

Based on (4), the $j$-th element of the lower bound
$\underline{f}_{\text{nn}}(t)$ is solved from the following optimisation
problem:

$\displaystyle\underline{f}^{j}_{\text{nn}}(t):=\underset{\\{z_{i}\\}_{i=0}^{L-1},\\{\delta_{i}\\}_{i=1}^{L-1}}{\min}W_{L}(j,:)z_{L-1}+b_{L}(j)$
$\displaystyle\text{s.t.}~{}~{}\underline{z}_{0}\leq
z_{0}\leq\overline{z}_{0},~{}\hat{z}_{i}=W_{i}z_{i-1}+b_{i},~{}i\in[1,L-1],$
(5a) $\displaystyle z_{i}\geq\hat{z}_{i},~{}z_{i}\geq
0,~{}z_{i}\leq\sigma_{i}\odot\hat{u}_{i},~{}i\in[1,L-1],$ (5b) $\displaystyle
z_{i}\leq\hat{z}_{i}-(1-\sigma_{i})\odot\hat{l}_{i},~{}\sigma_{i}\in\\{0,1\\}^{n_{i}},~{}i\in[1,L-1],$
(5c)

where $\odot$ is the elementwise product and $n_{i}$ is the number of neurons
at the $i$-th hidden layer. The $j$-th element of
$\overline{f}_{\text{nn}}(t)$ are solved from an optimisation problem in the
same formulation as (5) but with $\min$ being replaced by $\max$. These
optimisation problems are MILP problems which can be solved efficiently using
off-the-shelf solvers like Gurobi (Gurobi Optimization, LLC, 2023).

By using the computed NN interval
$[\underline{f}_{\text{nn}}(t),\overline{f}_{\text{nn}}(t)]$, it follows from
Assumption 1 and Lemma 2 that

$\displaystyle
B^{+}\underline{f}_{\text{nn}}(t)-B^{-}\overline{f}_{\text{nn}}(t)\leq
Bf_{\text{nn}}(z_{0})$ $\displaystyle\leq
B^{+}\overline{f}_{\text{nn}}(t)-B^{-}\underline{f}_{\text{nn}}(t).$ (6)

Interval of the Nonlinear Function. The OVERT method in (Sidrane et al., 2022)
is adapted to compute the interval $[\underline{g}(t),\overline{g}(t)]$ for
$g(x_{t})$. Its key idea is briefly introduced as below: Let $g_{i}(x_{t})$ be
the $i$-th elementary nonlinear function of $g(x_{t})$, where $i\in[1,s]$.
Without generality, this paper assumes that $g_{i}(x_{t})$ can be expressed by
a conjunction of relations, where each relation is either an elementary
function $e(\chi)$ of a scalar variable $\chi$ (e.g., $\sin(\chi)$,
$\chi^{2}$, etc) or an algebraic operation (addition $+$ or subtraction $-$).
The multiplication ($\times$) and division ($\div$) operations are converted
into an exponential of a sum of logarithms, for which the details are referred
to (Sidrane et al., 2022, Algorithm 1). Hence, the interval of $g(x_{t})$ can
be constructed by stacking the intervals of all elementary functions
$g_{i}(x_{t})$, $i\in[1,s]$. Let the scalar variable $\chi$ satisfy
$\chi\in[\underline{\chi},\overline{\chi}]$. The interval
$[\underline{\chi},\overline{\chi}]$ is divided into $h_{i}+1$ subintervals,
where $h_{i}$ is the number of intermediate points between the two endpoints
$\underline{\chi}$ and $\overline{\chi}$ to be selected. We then form the
upper (or lower) bound by connecting each two neighbouring points using either
the secant line (tangent line for lower bound, resp.) if $g_{i}(\chi)$ is
convex in that region, or the tangent line (secant line for lower bound,
resp.) if $g_{i}(\chi)$ is concave. The obtained bounds are in the form of

$\displaystyle\overline{g}_{i}(\chi)=\sum_{j=0}^{h_{i}}a_{i,j}\varphi_{i,j}(\chi),~{}\underline{g}_{i}(\chi)=\sum_{j=0}^{h_{i}}b_{i,j}\delta_{i,j}(\chi),$
(7)

where $\varphi_{i,j}(y_{t}^{o})$ and $\delta_{i,j}(\chi)$ are linear
functions, and $a_{i,j}$ and $b_{i,j}$ are constant coefficients. The optimal
way of selecting the $h_{i}$ intermediate points is described in Sidrane et
al. (2022). A higher value of $h_{i}$ gives a tighter interval but with a more
complex expression.

Prior physical knowledge of the nonlinear function $g_{i}(\chi)$ can be used
to refine the obtained interval. For example, in the simulation system in
Section 5, $g_{i}(\chi)=\chi^{2}$, where $\chi$ is the vehicle velocity.
Hence, the lower bound of $g_{i}(\chi)$ satisfies $\underline{g}_{i}\geq 0$
and the lower bound in (7) is refined as
$\underline{g}_{i}=\max(\underline{g}_{i},0)$. The refinement can help to
obtain a tighter interval, as exemplified in Fig. 1.

Figure 1: Intervals of $g_{i}(\chi)=\chi^{2}$ with and without refinement when
$h_{i}=2$ and $\chi\in[-10,20]$.

The interval $[\underline{g}_{i}(x_{t}),\overline{g}_{i}(x_{t})]$ for
$g_{i}(x_{t})$ under $x_{t}\in[\underline{x}_{t},\overline{x}_{t}]$ can be
derived using (7), as in (Sidrane et al., 2022, Algorithm 2). But this
interval depends on $x_{t}$, which is unavailable in the system (1). Instead,
the state interval $x_{t}\in[\underline{x}_{t},\overline{x}_{t}]$ will be
provided at each time step by the proposed interval observer. This state
interval can be used to compute $[\underline{g}_{i}(t),\overline{g}_{i}(t)]$
in real time, as described below:

Given $x_{t}\in[\underline{x}_{t},\overline{x}_{t}]$, the lower bound of
$g_{i}(x_{t})$ is redefined as $\underline{g}_{i}(t)$, by solving the
following optimisation problem:

$\displaystyle\underline{g}_{i}(t):=\quad$
$\displaystyle\underset{x_{t},\Omega_{i}}{\min}\quad\Omega_{i}$ s.t.
$\displaystyle\underline{x}_{t}\leq
x_{t}\leq\overline{x}_{t},~{}\underline{g}_{i}(x_{t})\leq\Omega_{i}\leq\overline{g}_{i}(x_{t}).$
(8)

The upper bound is redefined as $\overline{g}_{i}(t)$ and solved from an
optimisation problem with the same form of (3) but with $\min$ being replaced
by $\max$.

In some special cases, the nonlinear function $g_{i}(x_{t})$ is monotonic in
the region $x_{t}\in[\underline{x}_{t},\overline{x}_{t}]$. One example is the
simulation system in Section 5 where $g_{i}(x_{t})=x_{t}^{2}$ and the scalar
variable $x_{t}$ is the vehicle velocity satisfying $x_{t}\geq 0$. A graphic
view of this example is shown in Fig. 1 by regarding $x_{t}=\chi$ and
$\chi\in[0,20]$. In such special cases, solving the optimisation problem (3)
is not needed. $\underline{g}_{i}(t)$ and $\overline{g}_{i}(t)$ can be
directly computed using

$\underline{g}_{i}(t)\\!=\\!\min\\{\underline{g}_{i}(\underline{x}_{t}),\overline{g}_{i}(\underline{x}_{t})\\},\overline{g}_{i}(t)\\!=\\!\max\\{\underline{g}_{i}(\overline{x}_{t}),~{}\overline{g}_{i}(\overline{x}_{t})\\}.\\!\\!\\!\\!$
(9)

By using the computed interval for $g(x_{t})$ and Lemma 2, for
$x_{t}\in[\underline{x}_{t},\overline{x}_{t}]$, the interval of $Fg(x_{t})$ is
derived as

$\displaystyle F^{+}\underline{g}(t)-F^{-}\overline{g}(t)\leq Fg(x_{t})\leq
F^{+}\overline{g}(t)-F^{-}\underline{g}(t).$ (10)

Observer Design. For any design matrix $L_{o}\in\mathbb{R}^{n\times p}$, the
system (1) can be rewritten as

$\displaystyle
x_{t+1}=A_{o}x_{t}+Bf_{\text{nn}}+Fg(x_{t})+L_{o}y_{t}-L_{o}v_{t}+w_{t}.$ (11)

where $A_{o}=A-L_{o}C$. Based on Assumption 1, Lemma 2, (3) and (10), the
unknown terms $Bf_{\text{nn}}(z_{0})$, $Fg(x_{t})$, $L_{o}v_{t}$, and $w_{t}$
in (11) are replaced with their intervals to formulate the interval observer:

$\displaystyle\overline{x}_{t+1}={}$ $\displaystyle
A_{o}\overline{x}_{t}+B^{+}\overline{f}_{\text{nn}}(t)-B^{-}\underline{f}_{\text{nn}}(t)+F^{+}\overline{g}(t)$
$\displaystyle-F^{-}\underline{g}(t)+L_{o}y_{t}-L_{o}^{+}\underline{v}_{t}+L_{o}^{-}\overline{v}_{t}+\overline{w}_{t},$
(12a) $\displaystyle\underline{x}_{t+1}={}$ $\displaystyle
A_{o}\underline{x}_{t}+B^{+}\underline{f}_{\text{nn}}(t)-B^{-}\overline{f}_{\text{nn}}(t)+F^{+}\underline{g}(t)$
$\displaystyle-F^{-}\overline{g}(t)+L_{o}y_{t}-L_{o}^{+}\overline{v}_{t}+L_{o}^{-}\underline{v}_{t}+\underline{w}_{t}.$
(12b)

Define the interval width $\delta x_{t}=\overline{x}_{t}-\underline{x}_{t}$.
Its dynamics are

$\delta x_{t+1}=A_{o}\delta x_{t}+D\xi_{t},$ (13)

where $D=\left[|B|,|F|,|L_{o}|,I\right]$ and $\xi_{t}=[\delta
f_{\text{nn}};\delta g(t);\delta v_{t};\delta w_{t}]$ is regarded as the
disturbance with $\delta v_{t}=\overline{v}_{t}-\underline{v}_{t}$, $\delta
w_{t}=\overline{w}_{t}-\underline{w}_{t}$, $\delta
f_{\text{nn}}(t)=\overline{f}_{\text{nn}}(t)-\underline{f}_{\text{nn}}(t)$,
and $\delta g(t)=\overline{g}(t)-\underline{g}(t)$.

The gain $L_{o}$ is designed to ensure that the interval observer (12) is
sound, i.e., $\overline{x}_{t}\leq x_{t}\leq\underline{x}_{t},\forall t\geq
0$, and the interval width $\delta x_{t}$ is robustly stable against the
disturbance $\xi_{t}$.

###### Theorem 3

For system (1) under Assumption 1, if there is a scalar $\rho$, a diagonal
matrix $P$, and matrices $H_{1},H_{2}$ such that the following optimisation
problem is feasible:

$\displaystyle\hskip 51.21504pt\underset{\rho,P,H_{1},H_{2}}{\min}~{}~{}\rho$
$\displaystyle\text{s.t.}\hskip
14.22636ptP\succ\mathbf{0},~{}PA-(H_{1}-H_{2})C\geq\mathbf{0},$ (14a)
$\displaystyle\hskip
2.84544pt\rho>0,~{}\begin{bmatrix}P-I&\mathbf{0}&[PA-(H_{1}-H_{2})C]^{\top}\\\
\star&\rho I&\hat{D}^{\top}\\\ \star&\star&P\end{bmatrix}\succ\mathbf{0},$
(14b)

where $\hat{D}=[P|B|,P|F|,H_{1}+H_{2},P]$, then (12) is a sound interval
observer, i.e., $x_{t}\in[\underline{x}_{t},\overline{x}_{t}]$, $\forall t\geq
0$, given that $x_{0}\in[\underline{x}_{0},\overline{x}_{0}]$. Moreover, the
interval width $\delta x_{t}$ is robust against the disturbance vector
$\xi_{t}$ with the $\mathrm{H}_{\infty}$ performance gain $\sqrt{\rho}$. The
observer gain is computed as $L_{o}=L_{o}^{+}-L_{o}^{-}$ with
$L_{o}^{+}=P^{-1}H_{1}$ and $L_{o}^{-}=P^{-1}H_{2}$.

Define the errors as $\overline{e}_{t}=\overline{x}_{t}-x_{t}$ and
$\underline{e}_{t}=x_{t}-\underline{x}_{t}$. The error dynamics are derived as

$\overline{e}_{t+1}=A_{o}\overline{e}_{t}+\overline{\Theta}_{t},~{}\underline{e}_{t+1}=A_{o}\underline{e}_{t}+\underline{\Theta}_{t},$
(15)

where
$\overline{\Theta}_{t}=B^{+}(\overline{f}_{\text{nn}}-f_{\text{nn}})+B^{-}(f_{\text{nn}}-\underline{f}_{\text{nn}})+F^{+}(\overline{g}-g)+F^{-}(g-\underline{g})+(\overline{w}_{t}-w_{t})+L_{o}v_{t}-L_{o}^{+}\underline{v}_{t}+L_{o}^{-}\overline{v}_{t}$
and
$\underline{\Theta}_{t}=B^{+}(f_{\text{nn}}-\underline{f}_{\text{nn}})+B^{-}(\overline{f}_{\text{nn}}-f_{\text{nn}})+F^{+}(g-\underline{g})+F^{-}(\overline{g}-g)+(w_{t}-\underline{w}_{t})-L_{o}v_{t}+L_{o}^{+}\overline{v}_{t}-L_{o}^{-}\underline{v}_{t}$.

Under Assumption 1, the following inequalities hold:

$\displaystyle
L_{o}v_{t}-L_{o}^{+}\underline{v}_{t}+L_{o}^{-}\overline{v}_{t}$
$\displaystyle=L_{o}^{+}(v_{t}-\underline{v}_{t})+L_{o}^{-}(\overline{v}_{t}-v_{t})\geq\mathbf{0},$
$\displaystyle-
L_{o}v_{t}+L_{o}^{+}\overline{v}_{t}-L_{o}^{-}\underline{v}_{t}$
$\displaystyle=L_{o}^{-}(v_{t}-\underline{v}_{t})+L_{o}^{+}(\overline{v}_{t}-v_{t})\geq\mathbf{0}.$

Subsequently, it holds that $\overline{\Theta}_{t}\geq\mathbf{0}$ and
$\underline{\Theta}_{t}\geq\mathbf{0}$. Further recalling that
$\overline{e}_{0},\underline{e}_{0}\geq\mathbf{0}$ under Assumption 1, it is
seen from (15) that $\overline{e}_{t},\underline{e}_{t}\geq\mathbf{0}$,
$\forall t\geq 0$, if $A_{o}$ is non-negative (i.e. all its elements are non-
negative). Let $P\in\mathbb{R}^{n\times n}\succ\mathbf{0}$ be a diagonal
matrix, then $A_{o}$ is non-negative if $PA_{o}\geq\mathbf{0}.$ Submitting
$A_{o}=A-(L_{o}^{+}-L_{o}^{-})C$ into the above inequality and introducing
$H_{1}=PL_{o}^{+}$ and $H_{2}=PL_{o}^{-}$ yields (14a).

To ensure robust stability of $\delta x_{t}$ against the disturbance
$\xi_{t}$, the Lyapunov function $V_{t}=\delta x_{t}^{\top}P\delta x_{t}$ is
used. The following relation can be derived from (13):

$\displaystyle\Delta V_{t}=V_{t+1}-V_{t}={}$
$\displaystyle\zeta_{t}^{\top}\begin{bmatrix}A_{o}^{\top}PA_{o}-P&A_{o}^{\top}PD\\\
\star&D^{\top}PD\end{bmatrix}\zeta_{t},$ (16)

where $\zeta_{t}=[\delta x_{t};\xi_{t}]$ and
$D=\left[|B|,|F|,|L_{o}|,I\right]$. Considers the performance index
$J=\sum_{t=0}^{\infty}(\delta x_{t}^{\top}\delta
x_{t}-\gamma^{2}\xi_{t}^{\top}\xi_{t}+\Delta V_{t})$ for a scalar $\gamma>0$.
If $J<0$, the interval width dynamics (13) satisfy the $\mathrm{H}_{\infty}$
performance $\sum_{t=0}^{\infty}\delta x_{t}^{\top}\delta
x_{t}\leq\gamma^{2}\sum_{t=0}^{\infty}\xi_{t}^{\top}\xi_{t}$. By using (16), a
sufficient condition for $J<0$ is $\delta x_{t}^{\top}\delta
x_{t}-\gamma^{2}\xi_{t}^{\top}\xi_{t}+\Delta V_{t}<0$, i.e.,

$\begin{bmatrix}A_{o}^{\top}PA_{o}-P+I&A_{o}^{\top}PD\\\
\star&D^{\top}PD-\gamma^{2}I\end{bmatrix}\prec\mathbf{0}.$ (17)

Rearranging (17) and applying Schur complement to it, with the introduction of
new variables $H_{1}=PL_{o}^{+}$, $H_{2}=PL_{o}^{-}$, and $\rho=\gamma^{2}$,
results in the constraint (14b). ∎

##  4 Runtime Monitoring and Fault Detection

Safety Monitoring. Given the safe state interval $\mathcal{X}_{t}$, it follows
from Theorem 3 that the $i$-th state $x_{t}^{i}$ is safe if

$[\underline{x}_{t}^{i},\overline{x}_{t}^{i}]\subseteq\mathcal{X}_{t}^{i},~{}i\in[1,n].$
(18)

When this condition is false, safety of $x_{t}^{i}$ is undefined. It could be
either $[\underline{x}_{t}^{i},\overline{x}_{t}^{i}]$ is a too coarse outer-
approximation of $x_{t}^{i}$ or the state $x_{t}^{i}$ is indeed unsafe. At
time $t$, the observer (12) can also provide the one-step ahead predicted
state interval $[\underline{x}_{t+1},\overline{x}_{t+1}]$ and output interval
$[\underline{y}_{t+1},\overline{y}_{t+1}]$ as follows:
$\overline{y}_{t+1}=C^{+}\overline{x}_{t+1}-C^{-}\underline{x}_{t+1}+\overline{v}_{t+1}$,
$\underline{y}_{t+1}=C^{+}\underline{x}_{t+1}-C^{-}\overline{x}_{t+1}+\underline{v}_{t+1},$
These one-step ahead predicted intervals can be used to raise alerts of
potential unsafe operations in the future. Denote the one-step ahead predicted
intervals of $x_{t}$ and $y_{t}$ as
$X_{\mathrm{pred},t+1}:=[\underline{x}_{t+1},\overline{x}_{t+1}]$ and
$Y_{\mathrm{pred},t+1}:=[\underline{y}_{t+1},\overline{y}_{t+1}]$,
respectively. Let the safe intervals of $x_{t+1}$ and $y_{t+1}$ be
$\mathcal{X}_{t+1}$ and $\mathcal{Y}_{t+1}$, respectively. At time step $t+1$,
the $i$-th state is safe if

$X_{\mathrm{pred},t+1}^{i}\subseteq\mathcal{X}_{t+1}^{i},~{}i\in[1,n],$ (19)

and the $j$-th output is safe if

$Y_{\mathrm{pred},t+1}^{j}\subseteq\mathcal{Y}_{t+1}^{j},~{}j\in[1,p].$ (20)

When these set inclusions are false, safety of the $i$-th state or $j$-th
output is undefined. It could be either that the predicted intervals are too
coarse or the real state or output are indeed unsafe. Nevertheless, the
information of violation is still practically valuable for alerting unsafe
system operations and triggering appropriate preventions.

Fault Detection. Consider the case when an actuator fault $f^{\text{a}}_{t}$
acting on the control signal $u_{t}$ as follows:

$u_{t}=f_{\text{nn}}(r_{t},y^{o}_{t})+f^{\text{a}}_{t}.$ (21)

By using the computed interval
$[\underline{f}_{\text{nn}}(t),\overline{f}_{\text{nn}}(t)]$, there is a fault
in the $i$-th control input channel if

$u^{i}_{t}\notin[\underline{f}^{i}_{\text{nn}}(t),\overline{f}^{i}_{\text{nn}}(t)],~{}i\in[1,m].$
(22)

The control input $u_{t}$ is also affected by faults occurring at the output
$y^{o}_{t}$ because it is the input to the NN $f_{\text{nn}}$. In the presence
of additive faults, the output $y^{o}_{t}$ is represented by

$y^{o}_{t}=C_{o}(Cx_{t}+F_{\mathrm{s}}f^{\mathrm{s}}_{t}+v_{t}),$ (23)

with the fault distribution matrix $F_{\mathrm{s}}\in\mathbb{R}^{p\times s}$,
where $s\leq p$. This paper focuses on sensor faults, but faults could also be
from cyber attacks when some elements of $y^{o}_{t}$ are transmitted through
communication networks. Presence of $f^{\mathrm{s}}_{t}$ can also be detected
based on the condition (21). But this condition alone does not allow fault
isolation because the fault effects due to $f^{\text{a}}_{t}$ and
$f^{\mathrm{s}}_{t}$ are coupled.

Due to the nonlinear nature of the NN $f_{\text{nn}}$, it may happen that even
a relative large output fault $f^{\mathrm{s}}_{t}$ results in just minor
changes to $u_{t}$, making $u_{t}$ remain within the estimated interval and
the fault “invisible”. To address this issue, another detecting condition is
introduced below: The one-step ahead predicted interval for $y^{o}_{t}$ is
$y^{o}_{t+1}\in[\underline{y}^{o}_{t+1},\overline{y}^{o}_{t+1}]$, where
$\underline{y}^{o}_{t+1}=C_{o}\underline{y}_{t+1}$ and
$\overline{y}^{o}_{t+1}=C_{o}\overline{y}_{t+1}$. At time $t+1$, there is a
fault in the $i$-th output $y^{o,i}_{t+1}$ if

$y^{o,i}_{t}\notin[\underline{y}^{o,i}_{t+1},\overline{y}^{o,i}_{t+1}],~{}i\in[1,\ell].$
(24)

If (21) is false, but (24) is true, then we are still able to know that there
exists faults.

Sufficiency of the checking conditions (18), (19), (20), (22), and (24) relies
on tightness of the generated intervals. Tighter intervals increase the
capability of safety monitoring and fault detection. The tightness is affected
by the priori known intervals for $\omega_{t}$ and $v_{t}$ in Assumption 1 and
the intervals for $f_{\text{nn}}$ and $g(x_{t})$ computed in Section 3. Those
for $\omega_{t}$ and $v_{t}$ can be improved if a better knowledge of them is
available. The proposed OP method has achieved the tightest interval for
$f_{\text{nn}}$ with ReLU activations. For the nonlinear function $g(x_{t})$,
there may exist other ways to get a tighter interval, which is left for future
research.

##  5 Case Study

Consider an adaptive cruise control (ACC) system:

$\displaystyle\dot{x}={}$ $\displaystyle
A_{c}x+B_{c}u_{\mathrm{e}}+F_{c}g(x)+w,$ (25a) $\displaystyle y={}$
$\displaystyle Cx+v,$ (25b)

where
$x=[p_{\mathrm{l}};v_{\mathrm{l}};a_{\mathrm{l}};p_{\mathrm{e}};v_{\mathrm{e}};a_{\mathrm{e}}]$,
$y=[p_{\mathrm{e}};v_{\mathrm{e}};h;\tilde{v}]$,
$h=p_{\mathrm{l}}-p_{\mathrm{e}}$, $\tilde{v}=v_{\mathrm{l}}-v_{\mathrm{e}}$,
$g(x)=[v_{\mathrm{l}}^{2};v_{\mathrm{e}}^{2}]$,
$w=[0;0;2u_{\mathrm{l}};0;0;0]$, and

$\displaystyle A_{c}$ $\displaystyle=\begin{bmatrix}0&1&0&0&0&0\\\
0&0&1&0&0&0\\\ 0&0&-2&0&0&0\\\ 0&0&0&0&1&0\\\ 0&0&0&0&0&1\\\
0&0&0&0&0&-2\end{bmatrix},B_{c}=\begin{bmatrix}0\\\ 0\\\ 0\\\ 0\\\ 0\\\
2\end{bmatrix},F_{c}=\begin{bmatrix}0&0\\\ 0&0\\\ -\mu&0\\\ 0&0\\\ 0&0\\\
0&-\mu\end{bmatrix}.$

The variables $p$, $v$, $a$, and $u$ are the position, velocity, acceleration,
and control command of the lead (with subscript $\mathrm{l}$) and ego vehicles
(with subscript $\mathrm{e}$), respectively. $\mu$ is the friction parameter.
Discretising (25) with sampling time $t_{s}=0.1~{}\mathrm{s}$ gives a system
in the form of (1).

Define the safe relative vehicular distance as
$d_{\mathrm{safe}}=v_{\mathrm{e}}t_{\mathrm{gap}}+d_{\mathrm{still}}$, where
$t_{\mathrm{gap}}$ is the time headway and $d_{\mathrm{still}}$ is the
standstill distance. To achieve safe car following, the ACC controller
$u_{\text{e}}$ is designed for the ego vehicle to achieve two objectives: When
$h\geq d_{\mathrm{safe}}$, $u_{\mathrm{e}}$ takes the speed control mode to
maintain the ego vehicle at the driver-set speed $v_{\mathrm{set}}$; When
$h<d_{\mathrm{safe}}$, $u_{\mathrm{e}}$ takes the spacing control mode to
ensure $h=d_{\mathrm{safe}}$. A 4-layer feedforward NN controller
$u_{\text{e}}=f_{\mathrm{nn}}(r,y^{o})$, each hidden layer having 10 ReLU
neurons, is trained and implemented, where
$r=[v_{\mathrm{set}};t_{\mathrm{gap}}]$ and
$y^{o}=[h;\tilde{v};v_{\mathrm{e}}]=C_{o}y$ with $C_{o}=[\mathbf{0}_{3\times
1},I_{3}]$. The simulation parameters are: $\mu\\!=\\!0.0001$,
$t_{\mathrm{gap}}=1.4~{}\mathrm{s}$,
$d_{\mathrm{still}}\\!=\\!10~{}\mathrm{m}$,
$v_{\mathrm{set}}\\!=\\!30~{}\mathrm{m/s}$,
$a_{\mathrm{min}}\\!=\\!-3~{}\mathrm{m/s^{2}}$,
$a_{\mathrm{max}}\\!=\\!2~{}\mathrm{m/s^{2}}$. $v$ is a zero-mean white noise
with $|v|\\!\leq\\!0.001$. The initial state is
$x_{0}\\!=\\![50;20;0;10;20;0]$. The signal bounds in Assumption 1 are set as
$\underline{x}_{0}\\!=\\![49;19;-1;9;19;-1]$,
$\overline{x}_{0}\\!=\\![51;21;1;11;21;1]$, $\underline{v}_{t}\\!=\\!-0.001$,
and $\overline{v}_{t}\\!=\\!0.001$. Comparison is made for the interval
observers using the auxiliary network (AN) or optimisation (OP) methods to
compute the interval of $f_{\text{nn}}$. At each time step, the MILP problems
are solved using Gurobi and their total computation time is 0.006 s in
average. Vehicle reversing is not considered in the simulation, thus each
element of the nonlinear function
$g(x)=[v_{\mathrm{l}}^{2};v_{\mathrm{e}}^{2}]$ is monotonic and the interval
for $g(x)$ is computed using (9).

Case 1: The results in Figs. 2 \- 5 demonstrate efficacy of the proposed
observer in generating sound and robust runtime intervals for the NN
controller, vehicle positions, velocities, and accelerations. The results also
demonstrate that the intervals based on the OP method are much tighter than
those using the AN method, including $f_{\text{nn}}$ in Fig. 2, $v_{\text{e}}$
in Fig. 4, and $a_{\text{e}}$ in Fig. 5. The two methods result in similarly
tight intervals for the other vehicle states.

Figure 2: NN controller $u_{\text{e}}=f_{\mathrm{nn}}(r,y^{o})$ and its
intervals. Figure 3: Vehicle positions ($p_{\mathrm{l}}$ and $p_{\mathrm{e}}$)
and their intervals. Figure 4: Vehicle velocities ($v_{\mathrm{l}}$ and
$v_{\mathrm{e}}$) and their intervals. Figure 5: Vehicle accelerations
($a_{\mathrm{l}}$ and $a_{\mathrm{e}}$) and their intervals.

Case 2: Consider the case when there is a bias actuator fault
$f_{a}(t)=0.3\sin(0.5\pi t_{s}t),t\geq 0$. Fig. 6 shows that if the interval
of $f_{\text{nn}}$ is generated by the OP method, then applying the condition
(22) detects the fault. If the interval is generated by the AN method, the
fault is not detected. This showcases that the OP method is advantageous over
the AN method in actuator fault detection.

Figure 6: Detection of actuator fault $f_{a}(t)=0.3\sin(0.5\pi t_{s}t)$.

Suppose the lead vehicle sends its real-time position $p_{\mathrm{l}}$ and
velocity $v_{\mathrm{l}}$ to the ego vehicle through a wireless communication
network. These data are used to construct $y^{o}_{t}$ for generating the NN
control law $f_{\text{nn}}$. Consider the case when there is a relative large
fault $f_{s}(t)=5\sin(0.2\pi t_{s}t)$ on the data $p_{\text{l}}$ or
$f_{s}(t)=\sin(0.2\pi t_{s}t)$ on the data $v_{\text{l}}$. These faults may be
due to offsets in the lead vehicle’s sensors (Lan et al., 2020) or cyber
attacks in the communication network (Petit and Shladover, 2014). The top
subplots in Fig. 7 and Fig. 8 show that using the interval of $f_{\text{nn}}$
from either the AN or OP methods are unable to detect the faults, meaning that
the condition (22) alone is not sufficient. The bottom subplots in Fig. 7 and
Fig. 8 show that these faults can be effectively detected by both methods
using the interval $[\underline{h},\overline{h}]$ of the output $h$ based on
the condition (24). If no faults present, the interval observer always
generates the interval satisfying $\underline{h}\leq h\leq\overline{h}$, i.e.,
$h-\underline{h}\geq 0$ and $\overline{h}-h\geq 0$. However, these two
inequalities are violated as shown in the bottom subplots in Fig. 7 and Fig.
8, indicating the existence of faults in the system. This confirms the
benefits of combining the two conditions (22) and (24), as discussed in
Section 4.

Figure 7: Detection of fault $f_{s}(t)=5\sin(0.2\pi t_{s}t)$ on
$p_{\text{l}}$. Figure 8: Detection of fault $f_{s}(t)=\sin(0.2\pi t_{s}t)$ on
$v_{\text{l}}$.

##  6 Conclusions

A robust interval observer is proposed to generate a sound and tight state
interval for nonlinear systems controlled by a deep NN. The obtained interval
is applied to monitor the runtime system safety and detect faults in the
actuators and sensors. Simulation results of the ACC system demonstrate
effectiveness of the interval observer and its advantages over the existing
design. As for future research, appropriate remedial actions will be proposed
to ensure safe operation of NN-controlled systems based on the monitoring and
detection results.

## References

  * Bensalem et al. (2023) Bensalem, S. et al. (2023).  What, indeed, is an achievable provable guarantee for learning-enabled safety critical systems.  _arXiv preprint arXiv:2307.11784_.
  * Bunel et al. (2020) Bunel, R. et al. (2020).  Branch and bound for piecewise linear neural network verification.  _JMLR_ , 21(2020).
  * Cofer et al. (2020) Cofer, D. et al. (2020).  Run-time assurance for learning-enabled systems.  In _Proc. NFM_ , 361–368. Springer.
  * Dubey et al. (2022) Dubey, S.R., Singh, S.K., and Chaudhuri, B.B. (2022).  Activation functions in deep learning: a comprehensive survey and benchmark.  _Neurocomputing_ , 503, 92–108.
  * Efimov et al. (2013) Efimov, D. et al. (2013).  Interval observers for time-varying discrete-time systems.  _IEEE Trans. Automat. Contr._ , 58(12), 3218–3224.
  * Everett et al. (2021) Everett, M. et al. (2021).  Reachability analysis of neural feedback loops.  _IEEE Access_ , 9, 163938–163953.
  * Gurobi Optimization, LLC (2023) Gurobi Optimization, LLC (2023).  Gurobi Optimizer Reference Manual.  URL https://www.gurobi.com.
  * Hu et al. (2020) Hu, H. et al. (2020).  Reach-SDP: Reachability analysis of closed-loop systems with neural network controllers via semidefinite programming.  In _Proc. CDC_ , 5929–5934. IEEE.
  * Julian and Kochenderfer (2021) Julian, K.D. and Kochenderfer, M.J. (2021).  Reachability analysis for neural network aircraft collision avoidance systems.  _J. Guid. Control Dyn._ , 44(6), 1132–1142.
  * Karg and Lucia (2020) Karg, B. and Lucia, S. (2020).  Stability and feasibility of neural network-based controllers via output range analysis.  In _Proc. CDC_ , 4947–4954. IEEE.
  * Lan et al. (2020) Lan, J., Zhao, D., and Tian, D. (2020).  Robust cooperative adaptive cruise control of vehicles on banked and curved roads with sensor bias.  In _Proc. ACC_ , 2276–2281. IEEE.
  * Lan et al. (2023) Lan, J., Zheng, Y., and Lomuscio, A. (2023).  Iteratively enhanced semidefinite relaxations for efficient neural network verification.  In _Proc. AAAI_ , volume 37, 14937–14945.
  * Liu et al. (2021) Liu, C. et al. (2021).  Algorithms for verifying deep neural networks.  _Found. Trends. Optim._ , 4(3-4), 244–404.
  * Lomuscio and Maganti (2017) Lomuscio, A. and Maganti, L. (2017).  An approach to reachability analysis for feed-forward ReLU neural networks.  _arXiv preprint arXiv:1706.07351_.
  * Moe et al. (2018) Moe, S., Rustad, A.M., and Hanssen, K.G. (2018).  Machine learning in control systems: An overview of the state of the art.  In _Proc. Innov. Appl. Artif. Intell. Conf._ , 250–265. Springer.
  * Petit and Shladover (2014) Petit, J. and Shladover, S.E. (2014).  Potential cyberattacks on automated vehicles.  _IEEE Trans. Intell. Transp. Syst._ , 16(2), 546–556.
  * Sidrane et al. (2022) Sidrane, C. et al. (2022).  OVERT: An algorithm for safety verification of neural network control policies for nonlinear systems.  _JMLR_ , 23(1), 5090–5134.
  * Tang et al. (2023) Tang, Y. et al. (2023).  Perception and navigation in autonomous systems in the era of learning: A survey.  _IEEE Trans. Neural Netw. Learn. Syst._ , 34(12), 9604–9624.
  * Wang et al. (2023) Wang, T. et al. (2023).  Observer-based safety monitoring of nonlinear dynamical systems with neural networks via quadratic constraint approach.  _Int. J. Control_ , DOI: 10.1080/00207179.2023.2274924.
  * Xiang (2021) Xiang, W. (2021).  Runtime safety monitoring of neural-network-enabled dynamical systems.  _IEEE Trans. Cybern._ , 52(9), 9587–9596.
  * Zhang and Xu (2022) Zhang, Y. and Xu, X. (2022).  Safety verification of neural feedback systems based on constrained zonotopes.  _arXiv preprint arXiv:2204.00903_.