This article has been accepted for publication by IEEE Transactions on
Automatic Control.

The manuscript included in this file is the open access accepted version.

This open access version is released on arXiv in accordance with the IEEE
copyright agreement.

The final version will be available at IEEE Xplore.

# Stochastic Relative Degree and Path-wise Control of Nonlinear Stochastic
Systems

Alberto Mellone, , and Giordano Scarciotti *This work is partially supported
by Shanghai University under the “Foreign Expert Program” of Shanghai
University (No. 21WZ0109).Alberto Mellone and Giordano Scarciotti are with the
Department of Electrical and Electronic Engineering, Imperial College London,
London, SW7 2AZ, UK<EMAIL_ADDRESS>
###### Abstract

We address the path-wise control of systems described by a set of nonlinear
stochastic differential equations. For this class of systems, we introduce a
notion of stochastic relative degree and a change of coordinates which
transforms the dynamics to a stochastic normal form. The normal form is
instrumental for the design of a state-feedback control which linearises and
makes the dynamics deterministic. We observe that this control is _idealistic_
, _i.e._ it is not practically implementable because it employs a feedback of
the Brownian motion (which is never available) to cancel the noise. Using the
idealistic control as a starting point, we introduce a hybrid control
architecture which achieves _practical_ path-wise control. This hybrid
controller uses measurements of the state to perform periodic compensations
for the noise contribution to the dynamics. We prove that the hybrid
controller retrieves the idealistic performances in the limit as the
compensating period approaches zero. We address the problem of asymptotic
output tracking, solving it in the idealistic and in the practical framework.
We finally validate the theory by means of a numerical example.

###### Index Terms:

Stochastic systems, relative degree, normal form, feedback linearisation,
output tracking.

##  I Introduction

A point of departure in the study of nonlinear deterministic systems is the
definition of the relative degree of the system and, consequently, of a change
of coordinates that is able to transform the differential equations in a so-
called normal form that makes analysis and control easier. These ideas were
first introduced in the seminal work [1], where the authors solved the problem
of static state-feedback non-interacting control. The theory of normal forms
was later addressed in [2] and [3] for the control and observation of time-
varying nonlinear systems, and a systematic overview of normal forms was given
in [4]. The problem of feedback linearisation of single-input single-output
and multi-input systems was introduced in [5] and in [6], respectively, and a
systematic procedure to find the feedback-linearising control was provided in
[7] and [8]. In [9] the notion of zero dynamics was introduced and later
employed in [10] to tackle the problem of asymptotic stabilisation of
nonlinear systems.

In this paper we introduce the notion of stochastic relative degree to develop
a theory of path-wise feedback control for a general class of systems
described by nonlinear stochastic differential equations. The advantage of
using this stochastic framework lies in the fact that it allows to address
uncertainties characterised by probabilistic properties which cannot be
captured by the classical deterministic robust designs. Modelling in the
stochastic framework is flexible and lends itself to mechanical systems
(_e.g._ the quarter-car model), electro-mechanical systems (_e.g._ the
suspended gyro), aerospace systems (_e.g._ the satellite dynamics) and
mathematical finance. A survey of these applications can be found in [11,
Section 1.9] and references therein. Stochastic differential equations are
also at the basis of methodological applications such as stochastic
$H_{\infty}$ control (see [12], [13] and [14]), filtering and optimal control
(see [15] and [16]).

Some notions of normal forms for stochastic systems have been introduced in
the literature. For example, in [17] and [18], Stratonovich calculus was used
to obtain a normal form for purely diffusive processes, while [19] employed
coordinate changes to introduce symmetries for stochastic differential
equations. One of the first works suggesting the convenience of a normal form
for control of stochastic systems is [20], where the change of coordinates
makes the dynamics quasi-linear. In [21] and [22], similar coordinate
transformations are proposed in order to reduce the system dynamics to
canonical forms that are amenable to specific control strategies.
Specifically, strict-feedback noise-prone dynamics are obtained, which allow
achieving optimal globally stabilising back-stepping controllers as proposed
in [23].

The differences between the normal forms mentioned above and the one we
introduce in this paper are both technical and in scope. From a technical
viewpoint, we propose a coordinate projection and a feedback control that
_annihilate_ the noise in the linearised coordinates. From an objective
viewpoint, the stochastic relative degree and the normal form are mere
instrumental means for the design of a practical hybrid controller which
achieves output tracking in a path-wise fashion. In other words, the
controller that we develop compensates for each _specific_ realisation of the
stochastic disturbance.

More generally, by defining a new stochastic normal form, the goal of this
paper is to address the path-wise control of stochastic systems described by a
general class of nonlinear stochastic differential equations. We show that the
implementation of feedback laws that perfectly linearise the system dynamics
in a new set of coordinates comes with insurmountable causality issues, hence
the attribute _idealistic_ with which we refer to these control laws. In fact,
they employ a feedback of the noise, which is not practically available.
However, with these idealistic controls at hand, we introduce hybrid nonlinear
controllers that incorporate a causal estimator of the Brownian motion. This
controller is practically realisable and, although achieving only approximate
feedback linearisation and tracking, its performance can be arbitrarily
improved by tuning an underlying parameter, retrieving the idealistic case as
a limit behaviour.

While preliminary work has been published in [24], [25], [26], in this paper
we present several additional contributions. The main novelties introduced are
as follows. 1) All the results are now proved, which provides a substantial
theoretical contribution. 2) By leveraging the Itô-Stratonovich equivalence,
we are able to obtain sharper results in the theory of the stochastic normal
form. 3) We provide a characterisation of the solvability of the feedback
linearisation problem. 4) The theory of the practical control has been revised
and made sharper. 5) An analysis of the control challenges arising when the
input appears in the diffusion term of the equation has been added. 6)
Practical asymptotic output tracking has been addressed and solved for the
first time. 7) A number of technical results regarding uniform asymptotic
stability of nonlinear time-varying stochastic systems have been added and
proved. For the sake of readability, they have been gathered in the Appendix.
8) A comprehensive example illustrates the path-wise output tracking of a
nonlinear stochastic system both in the idealistic and practical scenarios.

The paper is organised as follows. In Section II we recall some preliminary
notions on stochastic systems. Section III introduces the stochastic relative
degree and normal form. In Section IV we address the problem of feedback
linearisation in the idealistic framework. In Section V we propose a hybrid
controller, which practically approximates the idealistic linearising control.
Section VI addresses the problem of asymptotic output tracking. In Section VII
we validate the theory by means of a numerical example. Section VIII contains
some concluding remarks. Finally, technical lemmas which are instrumental to
prove some of the results of the paper have been collected and proved in the
Appendix.

Notation. The symbol $\mathbb{Z}$ denotes the set of integer numbers, while
$\mathbb{R}$ and $\mathbb{C}$ denote the fields of real and complex numbers,
respectively; by adding the subscript “$<0$” (“$\geq 0$”, “$0$”) to any symbol
indicating a set of numbers, we denote that subset of numbers with negative
(non-negative, zero) real part. Where convenient, the symbol
$\partial^{n}_{x}$ is used as a shorthand for the operator
$\partial^{n}/\partial x^{n}$, while $\alpha^{(n)}$ indicates the $n$-th time
derivative of $\alpha$. The Lie derivative of the smooth scalar function
$h(x)$ along the vector field $f(x)$ is denoted by
$\operatorname{\mathcal{L}}_{f}h(x)$. We use the recursive notation
$\operatorname{\mathcal{L}}_{f}^{k}h(x)=\operatorname{\mathcal{L}}_{f}\operatorname{\mathcal{L}}_{f}^{k-1}h(x)$,
with $\operatorname{\mathcal{L}}^{0}_{f}h(x)=h(x)$. Given two smooth vector
fields $f(x)$ and $g(x)$, we define the operator
$\text{ad}_{f}g(x)=(\partial_{x}g(x))f(x)-(\partial_{x}f(x))g(x)$, and,
recursively, $\text{ad}_{f}^{k}g(x)=\text{ad}_{f}\text{ad}_{f}^{k-1}g(x)$ with
$\text{ad}_{f}^{0}g(x)=g(x)$. $(\nabla,\mathfrak{A},\mathfrak{P})$ is a
probability space given by the set $\nabla$, the $\sigma$-algebra
$\mathfrak{A}$ defined on $\nabla$ and the probability measure $\mathfrak{P}$
on the measurable space $(\nabla,\mathfrak{A})$. A stochastic process with
state space $\mathbb{R}^{n}$ is a family $\\{x_{t},\,t\in\mathbb{R}\\}$ of
$\mathbb{R}^{n}$-valued random variables, i.e. for every fixed
$t\in\mathbb{R}$, $x_{t}(\cdot)$ is an $\mathbb{R}^{n}$-valued random variable
and, for every fixed $w\in\nabla$, $x_{\cdot}(w)$ is an
$\mathbb{R}^{n}$-valued function of time [27, Section 1.8]. For ease of
notation, we often indicate a stochastic process
$\\{x_{t},\,t\in\mathbb{R}\\}$ simply with $x_{t}$ (this is common in the
literature, see e.g. [27]). With a slight abuse of notation, any subscript
different from the symbol “$t$” indicates the corresponding component of the
vector $x_{t}$, _e.g._ $x_{i}$ is the $i$-th component of the vector $x_{t}$.
All mappings appearing as integrands in stochastic integrals are assumed to be
integrable in the corresponding sense, namely Itô’s (see, _e.g._ , [16,
Definition 3.1.4]) or Stratonovich’s (equivalent to deterministic
integrability).

##  II Preliminaries

In this section we shortly recall the theory of generalised stochastic
processes and define differential operators that will be used in the remainder
of the paper.

Let $C^{\infty}_{0}(\mathbb{R})$ be the space of all infinitely differentiable
functions on $\mathbb{R}$ with compact support [28, Definition 1.2.1]. The
following definitions characterise the notions of distribution (also known as
generalised function), distributional derivative and generalised stochastic
process.

######  Definition 1.

[29, Definition 3.1] Let $X$ be an open subset of $\mathbb{R}$. A
_distribution on_ $X$ is a linear form $\psi$ on $C^{\infty}_{0}(\mathbb{R})$
that is also continuous in the sense that

$\lim_{j\rightarrow\infty}\psi(\varphi_{j})=\psi(\varphi)\qquad\text{as}\qquad\lim_{j\rightarrow\infty}\varphi_{j}=\varphi\quad\text{in}\quad
C^{\infty}_{0}(\mathbb{R}).$ (1)

######  Definition 2.

[28, Definition 3.1.1] For any distribution $\psi$, its _distributional
derivative_ $\dot{\psi}$ is defined as the distribution that satisfies

$\dot{\psi}(\varphi)=-\psi(\dot{\varphi}),\quad\forall\varphi\in
C_{0}^{\infty}(\mathbb{R}).$ (2)

Note that generalised functions have derivatives of all order, which are
generalised functions as well.

######  Definition 3.

[27, Section 3.2] A _generalised stochastic process_ is a random generalised
function in the sense that a random variable $\psi(\varphi)$ is assigned to
every $\varphi\in C^{\infty}_{0}$, where $\psi$ is, with probability 1, a
generalised function.

We now look at the Brownian motion as a generalised stochastic process.
Therefore, its distributional derivative is always defined [27, Section 3.2].
In particular, the generalised stochastic process given by such a derivative
has zero mean value and covariance function given by the generalised function
$\delta(t-s),$ $t,s\in\mathbb{R}$, _i.e._ the Dirac delta. Consequently, the
derivative of the generalised Brownian motion is the generalised _white noise_
[27, Section 3.2]. In the remainder, with a slight abuse of notation, we refer
to generalised Brownian motion and generalised white noise omitting the
attribute “generalised” and we denote them by simply $\mathcal{W}_{t}$ and
$\xi_{t}$, respectively, with $\xi_{t}=\dot{\mathcal{W}}_{t}$. It should be
emphasised that the just mentioned time derivative is meant in the sense of
distributions, and not as the limit of the difference quotient as the
increment tends to zero, which instead applies to differentiable functions in
the classical sense.

Consider the nonlinear single-input, single-output stochastic system expressed
in the shorthand integral notation by

$\displaystyle dx_{t}$
$\displaystyle=(f(x_{t})+g(x_{t})u)dt+(l(x_{t})+m(x_{t})u)d\operatorname{\mathcal{W}}_{t},$
(3) $\displaystyle y_{t}$ $\displaystyle=h(x_{t}),$

with $x_{t}\in\mathbb{R}^{n}$, $u\in\mathbb{R}$, $y_{t}\in\mathbb{R}$ and
$f:\mathbb{R}^{n}\rightarrow\mathbb{R}^{n}$,
$g:\mathbb{R}^{n}\rightarrow\mathbb{R}^{n}$,
$l:\mathbb{R}^{n}\rightarrow\mathbb{R}^{n}$,
$m:\mathbb{R}^{n}\rightarrow\mathbb{R}^{n}$,
$h:\mathbb{R}^{n}\rightarrow\mathbb{R}$ _smooth_ functions, _i.e._ they admit
continuous partial derivatives of any order. We assume that, for a fixed
initial condition $x_{t=0}$, the solution of (3) is unique. Note that, in the
light of the previous discussion, system (3) can be rewritten in the following
differential notation

$\dot{x}_{t}=f(x_{t})+g(x_{t})u+(l(x_{t})+m(x_{t})u)\xi_{t},\quad
y_{t}=h(x_{t}).$ (4)

Note that when $\xi_{t}$ is (generalised) white noise, as in this case, then
the differential equation (4) is equivalent to the integral equation (3) if
the latter is interpreted in Itô’s sense [27, Section 10.3]. Given the
equivalence of the two representations in the framework of generalised
stochastic processes, in the remainder of the paper equations (3) and (4) are
used interchangeably, as convenient, to refer to the same nonlinear stochastic
system. We refer the reader to [27, Chapter 3] for a detailed discussion on
the relation between Brownian motion and white noise, and the representations
(3) and (4).

##  III Stochastic Relative Degree and Normal Form

In this section we introduce the concept of stochastic relative degree and
show that a suitable coordinate transformation brings the system into a
simpler form, which is convenient for analysis and control.

We first introduce three new operators, which are fundamental to
systematically define repeated time derivatives of stochastic processes. The
first one, which indicates the second derivative of $h$ along the vector
fields $f$ and $g$, is defined as

$\text{$\sideset{{}^{g}}{{}_{f}}{\operatorname{\mathcal{G}}}$}h(x)\\!=\\!g(x)^{\top}\text{$\partial^{2}_{x}$}[h]\;f(x)=\sum_{j=1}^{n}g_{j}(x)\sum_{i=1}^{n}\frac{\partial^{2}h}{\partial
x_{j}\partial x_{i}}f_{i}(x).$ (5)

Similarly to the Lie derivative, we use the notation
$\text{$\sideset{{}^{b}}{{}_{a}}{\operatorname{\mathcal{G}}}$}\text{$\sideset{{}^{g}}{{}_{f}}{\operatorname{\mathcal{G}}}$}h(x)=b(x)^{\top}\text{$\partial^{2}_{x}$}[\text{$\sideset{{}^{g}}{{}_{f}}{\operatorname{\mathcal{G}}}$}h]\;a(x),$
and
$\text{$\sideset{{}^{g}}{{}_{f}^{k}}{\operatorname{\mathcal{G}}}$}h(x)=g(x)^{\top}\text{$\partial^{2}_{x}$}[\text{$\sideset{{}^{g}}{{}_{f}^{k-1}}{\operatorname{\mathcal{G}}}$}h]\;f(x)$,
to indicate the reiterated operations. The second operator
$\text{$\sideset{{}^{l}}{{}_{f}}{\operatorname{\mathcal{S}}}$}h$, which we
call the _stochastic Lie derivative_ , indicates the derivative of $h$ along
the drift vector field $f$ and diffusion vector field $l$, namely

$\text{$\sideset{{}^{l}}{{}_{f}}{\operatorname{\mathcal{S}}}$}h(\xi_{t},x)=\operatorname{\mathcal{L}}_{f}h(x)+\operatorname{\mathcal{L}}_{l}h(x)\xi_{t}+\frac{1}{2}\text{$\sideset{{}^{l}}{{}_{l}}{\operatorname{\mathcal{G}}}$}h(x).$
(6)

The reiterated application of this operator can be defined if the white noise
does not appear explicitly. That is, if
$\text{$\sideset{{}^{l}}{{}_{f}}{\operatorname{\mathcal{S}}}$}h(\xi_{t},x)=\text{$\sideset{{}^{l}}{{}_{f}}{\operatorname{\mathcal{S}}}$}h(x)$
is a deterministic expression, we use the notation
$\text{$\sideset{{}^{l}}{{}_{f}^{2}}{\operatorname{\mathcal{S}}}$}h(\xi_{t},x)=\text{$\sideset{{}^{l}}{{}_{f}}{\operatorname{\mathcal{S}}}$}\text{$\sideset{{}^{l}}{{}_{f}}{\operatorname{\mathcal{S}}}$}h(\xi_{t},x)$
and, iteratively, if
$\text{$\sideset{{}^{l}}{{}_{f}^{k-1}}{\operatorname{\mathcal{S}}}$}h(\xi_{t},x)=\text{$\sideset{{}^{l}}{{}_{f}^{k-1}}{\operatorname{\mathcal{S}}}$}h(x)$
is deterministic,
$\text{$\sideset{{}^{l}}{{}_{f}^{k}}{\operatorname{\mathcal{S}}}$}h(\xi_{t},x)=\text{$\sideset{{}^{l}}{{}_{f}}{\operatorname{\mathcal{S}}}$}\text{$\sideset{{}^{l}}{{}_{f}^{k-1}}{\operatorname{\mathcal{S}}}$}h(\xi_{t},x)$,
with
$\text{$\sideset{{}^{l}}{{}_{f}^{0}}{\operatorname{\mathcal{S}}}$}h(x)=h(x)$
by definition. Finally, we define the third operator

$\text{$\sideset{{}^{m}_{g}}{{}_{l}}{\operatorname{\mathcal{A}}}$}h(\xi_{t},x)=\operatorname{\mathcal{L}}_{g}h(x)+\operatorname{\mathcal{L}}_{m}h(x)\xi_{t}+\text{$\sideset{{}^{m}}{{}_{l}}{\operatorname{\mathcal{G}}}$}h(x).$
(7)

Having defined the operators $\sideset{}{}{\operatorname{\mathcal{G}}}$,
$\sideset{}{}{\operatorname{\mathcal{S}}}$ and
$\sideset{}{}{\operatorname{\mathcal{A}}}$, it is easy to see that, by using
Itô’s formula, the first derivative of the output of system (4) is given by

$y^{(1)}_{t}=\text{$\sideset{{}^{l}}{{}_{f}}{\operatorname{\mathcal{S}}}$}h(\xi_{t},x_{t})+\text{$\sideset{{}^{m}_{g}}{{}_{l}}{\operatorname{\mathcal{A}}}$}h(\xi_{t},x_{t})u+\frac{1}{2}\text{$\sideset{{}^{m}}{{}_{m}}{\operatorname{\mathcal{G}}}$}h(x_{t})u^{2}.$
(8)

We now define the concept of stochastic relative degree and then point out the
rationale of such a definition.

######  Definition 4.

(Stochastic Relative Degree) System (4) is said to have _stochastic relative
degree_ $r$ at a point $\bar{x}$ if

(ND)

$\operatorname{\mathcal{L}}_{l}\text{$\sideset{{}^{l}}{{}_{f}^{k}}{\operatorname{\mathcal{S}}}$}h(x)=0$
and
$\operatorname{\mathcal{L}}_{m}\text{$\sideset{{}^{l}}{{}_{f}^{k}}{\operatorname{\mathcal{S}}}$}h(x)=0$
for all $x$ in a neighborhood of $\bar{x}$ and for all $k\in\\{0,...,r-2\\}$.

(CD)

$\operatorname{\mathcal{L}}_{g}\text{$\sideset{{}^{l}}{{}_{f}^{k}}{\operatorname{\mathcal{S}}}$}h(x)+\text{$\sideset{{}^{m}}{{}_{l}}{\operatorname{\mathcal{G}}}$}\text{$\sideset{{}^{l}}{{}_{f}^{k}}{\operatorname{\mathcal{S}}}$}h(x)=0$,
$\operatorname{\mathcal{L}}_{m}\text{$\sideset{{}^{l}}{{}_{f}^{k}}{\operatorname{\mathcal{S}}}$}h(x)=0$
and
$\text{$\sideset{{}^{m}}{{}_{m}}{\operatorname{\mathcal{G}}}$}\text{$\sideset{{}^{l}}{{}_{f}^{k}}{\operatorname{\mathcal{S}}}$}h(x)=0$
for all $x$ in a neighborhood of $\bar{x}$ and all $k\in\\{0,...,r-2\\}$.

(RD)

$\operatorname{\mathcal{L}}_{g}\text{$\sideset{{}^{l}}{{}_{f}^{r-1}}{\operatorname{\mathcal{S}}}$}h(\bar{x})+\text{$\sideset{{}^{m}}{{}_{l}}{\operatorname{\mathcal{G}}}$}\text{$\sideset{{}^{l}}{{}_{f}^{r-1}}{\operatorname{\mathcal{S}}}$}h(\bar{x})\neq
0$ or
$\operatorname{\mathcal{L}}_{m}\text{$\sideset{{}^{l}}{{}_{f}^{r-1}}{\operatorname{\mathcal{S}}}$}h(\bar{x})\neq
0$ or
$\text{$\sideset{{}^{m}}{{}_{m}}{\operatorname{\mathcal{G}}}$}\text{$\sideset{{}^{l}}{{}_{f}^{r-1}}{\operatorname{\mathcal{S}}}$}h(\bar{x})\neq
0$.

As it is formally proved in the next proposition, in Definition 4, condition
(ND) (which stands for noise decoupling) ensures that the noise $\xi_{t}$ does
not appear in $y_{t}$ and its first $r-1$ derivatives. In the remainder we
will omit the dependency of the operators
$\sideset{}{}{\operatorname{\mathcal{S}}}$ and
$\sideset{}{}{\operatorname{\mathcal{A}}}$ on the white noise $\xi_{t}$
whenever this does not appear explicitly (for instance because of condition
(ND)). Condition (CD) (for control decoupling) ensures that the control input
$u$ does not appear in $y_{t}$ and its first $r-1$ derivatives. Finally,
condition (RD) (for relative degree) ensures that the control $u$ appears in
the $r$-th derivative of $y_{t}$, thus defining the relative degree of the
system. These observations are formally gathered in the following result, the
proof of which can be found in Appendix A.

######  Proposition 1.

Suppose that system (4) has stochastic relative degree $r>0$ at $\bar{x}$.
Then

$y^{(k)}_{t}=\text{$\sideset{{}^{l}}{{}_{f}^{k}}{\operatorname{\mathcal{S}}}$}h(x_{t})\qquad\forall
k\in\\{0,\dots,r-1\\},$ (9)

_i.e._ the first $r-1$ derivatives of $y_{t}$ do not depend explicitly on
$\xi_{t}$ nor $u$. Moreover, if, at time $\bar{t}$, $x_{\bar{t}}=\bar{x}$,
then

$y^{(r)}_{t=\bar{t}}=\text{$\sideset{{}^{l}}{{}_{f}^{r}}{\operatorname{\mathcal{S}}}$}h(\xi_{\bar{t}},\bar{x})+\text{$\sideset{{}^{m}_{g}}{{}_{l}}{\operatorname{\mathcal{A}}}$}\text{$\sideset{{}^{l}}{{}_{f}^{r-1}}{\operatorname{\mathcal{S}}}$}h(\xi_{\bar{t}},\bar{x})u(\bar{t})+\\\
\frac{1}{2}\text{$\sideset{{}^{m}}{{}_{m}}{\operatorname{\mathcal{G}}}$}\text{$\sideset{{}^{l}}{{}_{f}^{r-1}}{\operatorname{\mathcal{S}}}$}h(\bar{x})u(\bar{t})^{2},$
(10)

where either
$\text{$\sideset{{}^{m}_{g}}{{}_{l}}{\operatorname{\mathcal{A}}}$}\text{$\sideset{{}^{l}}{{}_{f}^{r-1}}{\operatorname{\mathcal{S}}}$}h(\xi_{\bar{t}},\bar{x})$
or
$\text{$\sideset{{}^{m}}{{}_{m}}{\operatorname{\mathcal{G}}}$}\text{$\sideset{{}^{l}}{{}_{f}^{r-1}}{\operatorname{\mathcal{S}}}$}h(\bar{x})$
are nonzero.

The previous result shows that, analogously to the deterministic case, the
stochastic relative degree is equal to the order of the derivative of the
output at time $\bar{t}$ in which the input $u(\bar{t})$ explicitly appears.
Two observations are in order: first, while the white noise does not appear in
all the derivatives up to order $r-1$ because of condition (ND), it may or may
not appear in the $r$-th derivative; second, differently from the
deterministic case, the control $u$ appears linearly and/or quadratically in
(10).

######  Remark 1.

Condition (ND) is a type of _disturbance decoupling_ condition, as we suppose
that the noise does not appear in $y_{t}$ and its successive $r-1$
derivatives. For a deterministic analogous, see _e.g._ , [30, Section 4.6]. If
$\operatorname{\mathcal{L}}_{l}\text{$\sideset{{}^{l}}{{}_{f}^{k}}{\operatorname{\mathcal{S}}}$}h(x)\neq
0$ for a $k<r-1$, the differentiation of $y_{t}$ up to the $r$-th time would
require us to introduce successive derivatives of the white noise, which is
theoretically and practically challenging. We exclude this possibility with
(ND).

######  Remark 2.

The equality
$\operatorname{\mathcal{L}}_{m}\text{$\sideset{{}^{l}}{{}_{f}^{k}}{\operatorname{\mathcal{S}}}$}h(x)=0$
appears in both conditions (ND) and (CD). This repetition is unnecessary.
However, since
$\operatorname{\mathcal{L}}_{m}\text{$\sideset{{}^{l}}{{}_{f}^{k}}{\operatorname{\mathcal{S}}}$}h(x)$
multiplies both $\xi_{t}$ and $u$ in the expression of the derivative
$y_{t}^{(k+1)}$, we believe that for the sake of clarity it is beneficial to
require it to be zero in both the noise and control decoupling conditions.

######  Remark 3.

Consider a linear stochastic system, _i.e._ system (4) with $f(x_{t})=Ax_{t}$,
$g(x_{t})\equiv B$, $l(x_{t})=Fx_{t}$ and $m(x_{t})\equiv G$, where
$A\in\mathbb{R}^{n\times n}$, $B\in\mathbb{R}^{n\times 1}$,
$F\in\mathbb{R}^{n\times n}$ and $G\in\mathbb{R}^{n\times 1}$. By applying
Definition 4, system (4) in the linear case has stochastic relative degree $r$
if

  1. 1.

$CA^{k}B=0$ and $CA^{k}G=0$ for all $k\in\\{0,...,r-2\\}$.

  2. 2.

$CA^{r-1}B\neq 0$ or $CA^{r-1}G\neq 0$.

Note that the conditions are remarkably simple and reminiscent of the
deterministic case. In fact,
$\text{$\sideset{{}^{l}}{{}_{f}^{k}}{\operatorname{\mathcal{S}}}$}h(x)=CA^{k}x$,
_i.e._ it is linear in $x$, for all $k\in\\{0,...,r-1\\}$, hence
$\text{$\sideset{{}^{l}}{{}_{l}}{\operatorname{\mathcal{G}}}$}\text{$\sideset{{}^{l}}{{}_{f}^{k}}{\operatorname{\mathcal{S}}}$}h(x)\equiv\text{$\sideset{{}^{m}}{{}_{l}}{\operatorname{\mathcal{G}}}$}\text{$\sideset{{}^{l}}{{}_{f}^{k}}{\operatorname{\mathcal{S}}}$}h(x)\equiv\text{$\sideset{{}^{m}}{{}_{m}}{\operatorname{\mathcal{G}}}$}\text{$\sideset{{}^{l}}{{}_{f}^{k}}{\operatorname{\mathcal{S}}}$}h(x)\equiv
0$ because the Hessian of a linear function with respect to $x$ is identically
zero. Assuming that the relative degree of this system is, for instance,
$r>2$, it follows that

$\displaystyle y^{(k)}_{t}$ $\displaystyle=CA^{k}x_{t},\qquad\qquad\forall
k\in\\{0,...,r-1\\},$ (11) $\displaystyle y^{(r)}_{t}$
$\displaystyle=CA^{r-1}(A+F\xi_{t})x_{t}+CA^{r-1}(B+G\xi_{t})u.$

Having defined the notion of stochastic relative degree and having discussed
its interpreation, we are now interested in finding a diffeomorphism
$\Phi:\mathbb{R}^{n}\rightarrow\mathbb{R}^{n}$ that locally (_i.e._ in a
neighbourhood $\bar{U}$ of $\bar{x}\in U\subset\mathbb{R}^{n}$) transforms
system (4) in such a way that its dynamics is somewhat “simpler”. The
diffeomorphism we are looking for is a direct consequence of the definition of
the stochastic relative degree previously given. To simplify the exposition,
we make the following assumption on the stochastic Lie derivatives of
$y_{t}=h(x_{t})$ along the drift vector $f$ and the diffusion vector $l$.

######  Assumption 1.

Let $r$ be the stochastic relative degree of system (4) at $\bar{x}$. Then the
row vectors

$\text{$\partial_{x}$}[h]_{x=\bar{x}},\;\text{$\partial_{x}$}[\text{$\sideset{{}^{l}}{{}_{f}}{\operatorname{\mathcal{S}}}$}h]_{x=\bar{x}},\;...\;,\;\text{$\partial_{x}$}[\text{$\sideset{{}^{l}}{{}_{f}^{r-1}}{\operatorname{\mathcal{S}}}$}h]_{x=\bar{x}},$
(12)

are linearly independent.

Observe that if Assumption 1 holds, then necessarily $r\leq n$.

######  Remark 4.

For deterministic nonlinear systems, the linear independence of the gradients
of the first $r-1$ successive derivatives of the output at $\bar{x}$ is a
consequence of the relative degree being defined, see _e.g_ , [30, Lemma
4.1.1]. In Section III-A we prove that this property holds in the present
setting for the case in which $m\equiv 0$ near $\bar{x}$. However, a proof
that a similar result holds in general ($m\neq 0$) is missing. For simplicity,
we use Assumption 1 at this stage to develop the theory of normal form for the
most general class of systems, and we note that no counter-example has been
found for which this property is not satisfied when $m\neq 0$ near $\bar{x}$.

######  Proposition 2.

Suppose that system (4) has stochastic relative degree $r$ at $\bar{x}$ and
let Assumption 1 hold. Set

$\phi_{1}(x)\\!\\!=\\!\\!h(x),\;\;\phi_{2}(x)\\!\\!=\\!\\!\text{$\sideset{{}^{l}}{{}_{f}}{\operatorname{\mathcal{S}}}$}h(x),\;\;\dots,\;\;\phi_{r}(x)\\!\\!=\\!\\!\text{$\sideset{{}^{l}}{{}_{f}^{r-1}}{\operatorname{\mathcal{S}}}$}h(x).$
(13)

If $r<n$, then there exist smooth functions $\phi_{r+1}(x),...,\phi_{n}(x)$,
with $\phi_{j}\in\mathbb{R}$ for all $j\in\\{r+1,..,n\\}$, such that the
Jacobian of the mapping

$\Phi(x)=\begin{bmatrix}\phi_{1}(x)&\phi_{2}(x)&\dots&\phi_{n}(x)\end{bmatrix}^{\top}$
(14)

is invertible at $\bar{x}$ almost surely, thus defining a coordinate
transformation in a neighbourhood of $\bar{x}$. Then the state-space
representation of system (4) in the transformed state $z_{t}=\Phi(x_{t})$ is

$\displaystyle\dot{z}_{i}$ $\displaystyle=z_{i+1},$ $\displaystyle
i=1,...,r-1,$ (15) $\displaystyle\dot{z}_{r}$
$\displaystyle=c(\xi_{t},z_{t})+b(\xi_{t},z_{t})u+a(z_{t})u^{2},$
$\displaystyle\dot{z}_{j}$
$\displaystyle=p_{j}(\xi_{t},z_{t})+q_{j}(\xi_{t},z_{t})u+s_{j}(z_{t})u^{2},\;$
$\displaystyle j=r+1,...,n,$ $\displaystyle y_{t}$ $\displaystyle=z_{1},$

where the mappings111To keep the statement of the proposition concise, the
mappings $a$, $b$, $c$, $p_{j}$, $q_{j}$, $s_{j}$ are defined in the proof.
$c$, $b$, $p_{j}$ and $q_{j}$ are affine in $\xi_{t}$.

###### Proof.

By Assumption 1, the matrix

$\begin{bmatrix}\text{$\partial_{x}$}[h(x)]^{\top}&\dots&\text{$\partial_{x}$}[\text{$\sideset{{}^{l}}{{}_{f}^{r-1}}{\operatorname{\mathcal{S}}}$}h(x)]^{\top}\end{bmatrix}^{\top}$
(16)

has rank $r$ at $\bar{x}$. If $r<n$, let $\gamma_{r+1}(x),...,\gamma_{n}(x)$,
with $\gamma_{j}\in\mathbb{R}^{n}$ for all $j\in\\{r+1,..,n\\}$, be any set of
$n-r$ vectors such that

$\begin{bmatrix}\text{$\partial_{x}$}[h(\bar{x})]^{\top}\dots\;\text{$\partial_{x}$}[\text{$\sideset{{}^{l}}{{}_{f}^{r-1}}{\operatorname{\mathcal{S}}}$}h(\bar{x})]^{\top}\;\;\gamma_{r+1}(\bar{x})\;\dots\;\gamma_{n}(\bar{x})\end{bmatrix}^{\top}$
(17)

has rank $n$. Note that this is possible because there always exist $n-r$
linearly independent vectors $\gamma_{r+1}(\bar{x}),...,\gamma_{n}(\bar{x})$
that complete the first $r$ linearly independent vectors
$\text{$\partial_{x}$}[h(\bar{x})]^{\top},...,\text{$\partial_{x}$}[\text{$\sideset{{}^{l}}{{}_{f}^{r-1}}{\operatorname{\mathcal{S}}}$}h(x)]^{\top}$
to a basis of $\mathbb{R}^{n}$. Let $\phi_{j}(x)$ be any smooth function such
that $\text{$\partial_{x}$}[\phi_{j}(x)]=\gamma_{j}^{\top}(x)$ for
$j=r+1,...,n$. Then $\Phi(x)$ as defined in (14) is a local diffeomorphism in
a neighbourhood of $\bar{x}$ and, therefore, it defines a local change of
coordinates $z_{t}=\Phi(x_{t})$ for the stochastic system (4). Applying Itô’s
lemma and since the system has relative degree $r$, the following holds

$\dot{z}_{i}=\text{$\sideset{{}^{l}}{{}_{f}^{i}}{\operatorname{\mathcal{S}}}$}h(x_{t})=\phi_{i+1}(x_{t})=z_{i+1},\qquad
i=1,...,r-1.$ (18)

Moreover,

$\dot{z}_{r}=\\!\text{$\sideset{{}^{l}}{{}_{f}^{r}}{\operatorname{\mathcal{S}}}$}h(x_{t})+\text{$\sideset{{}^{m}_{g}}{{}_{l}}{\operatorname{\mathcal{A}}}$}\text{$\sideset{{}^{l}}{{}_{f}^{r-1}}{\operatorname{\mathcal{S}}}$}h(x_{t})u+\frac{1}{2}\text{$\sideset{{}^{m}}{{}_{m}}{\operatorname{\mathcal{G}}}$}\text{$\sideset{{}^{l}}{{}_{f}^{r-1}}{\operatorname{\mathcal{S}}}$}h(x_{t})u^{2}.$
(19)

We now set

$\displaystyle c(\xi_{t},z_{t})$
$\displaystyle=\text{$\sideset{{}^{l}}{{}_{f}^{r}}{\operatorname{\mathcal{S}}}$}h(\xi_{t},\Phi^{-1}(z_{t})),$
(20) $\displaystyle b(\xi_{t},z_{t})$
$\displaystyle=\text{$\sideset{{}^{m}_{g}}{{}_{l}}{\operatorname{\mathcal{A}}}$}\text{$\sideset{{}^{l}}{{}_{f}^{r-1}}{\operatorname{\mathcal{S}}}$}h(\xi_{t},\Phi^{-1}(z_{t})),$
$\displaystyle a(z_{t})$
$\displaystyle=\frac{1}{2}\text{$\sideset{{}^{m}}{{}_{m}}{\operatorname{\mathcal{G}}}$}\text{$\sideset{{}^{l}}{{}_{f}^{r-1}}{\operatorname{\mathcal{S}}}$}h(\Phi^{-1}(z_{t})),$

thus obtaining

$\dot{z}_{r}=c(\xi_{t},z_{t})+b(\xi_{t},z_{t})u+a(z_{t})u^{2}.$ (21)

As for the remaining $n-r$ components of $z_{t}$, by applying Itô’s lemma to
the functions $\phi_{j}(x_{t})$ and setting

$\displaystyle p_{j}(\xi_{t},z_{t})$
$\displaystyle=\text{$\sideset{{}^{l}}{{}_{f}}{\operatorname{\mathcal{S}}}$}\phi_{j}(\xi_{t},\Phi^{-1}(z_{t})),$
(22) $\displaystyle q_{j}(\xi_{t},z_{t})$
$\displaystyle=\text{$\sideset{{}^{m}_{g}}{{}_{l}}{\operatorname{\mathcal{A}}}$}\phi_{j}(\xi_{t},\Phi^{-1}(z_{t})),$
$\displaystyle s_{j}(z_{t})$
$\displaystyle=\frac{1}{2}\text{$\sideset{{}^{m}}{{}_{m}}{\operatorname{\mathcal{G}}}$}\phi_{j}(\Phi^{-1}(z_{t})),$

yields

$\dot{z}_{j}=p_{j}(\xi_{t},z_{t})+q_{j}(\xi_{t},z_{t})u+s_{j}(z_{t})u^{2},\qquad
j=r+1,...,n.$ (23)

The proof is completed by observing that $y_{t}=h(x_{t})=z_{1}$ and that, by
the definitions of the operators $\mathcal{S}$ and $\mathcal{A}$, the
coefficients $c$, $b$, $p_{j}$ and $q_{j}$ are affine in $\xi_{t}$. ∎

Note that it might be possible to find smooth functions
$\phi_{r+1},...,\phi_{n}$ such that the dynamics of the last $n-r$ transformed
coordinates is independent of the input $u$, _i.e._ $q_{j}(\cdot,z_{t})\equiv
0$, $s_{j}(\cdot,z_{t})\equiv 0$, for all $j\in\\{r+1,...,n\\}$, in a
neighbourhood of $\Phi(\bar{x})$. This observation motivates the next
definition.

######  Definition 5.

(Stochastic Normal Form) Let $x_{t}$ be the unique solution of (4) and
$z_{t}=\Phi(x_{t})$ be a local diffeomorphism in a subset $U$ of
$\mathbb{R}^{n}$ such that

$\displaystyle\dot{z}_{i}$ $\displaystyle=z_{i+1},\quad$ $\displaystyle
i=1,...,r-1,$ (24) $\displaystyle\dot{z}_{r}$
$\displaystyle=c(\xi_{t},z_{t})+b(\xi_{t},z_{t})u+a(z_{t})u^{2},$
$\displaystyle\dot{z}_{j}$ $\displaystyle=p_{j}(\xi_{t},z_{t}),$
$\displaystyle j=r+1,...,n,$ $\displaystyle y_{t}$ $\displaystyle=z_{1}.$

System (24) is said to be the _stochastic normal form_ of system (4).

######  Remark 5.

The fact that the coefficients $c$, $b$, $p_{j}$ and $q_{j}$ are affine in
$\xi_{t}$ guarantees that system (15), which is written in the differential
notation (as in (4)), can always be equivalently written in the integral
notation (as in (3)).

For compactness, in the remainder we use the definitions
$p=\begin{bmatrix}p_{r+1}&\dots&p_{n}\end{bmatrix}^{\top}$,
$q=\begin{bmatrix}q_{r+1}&\dots&q_{n}\end{bmatrix}^{\top}$ and
$s=\begin{bmatrix}s_{r+1}&\dots&s_{n}\end{bmatrix}^{\top}$. Obviously, if the
stochastic relative degree at $\bar{x}$ is equal to the order of the system,
then the system admits a stochastic normal form in a neighbourhood $U$ of
$\bar{x}$ (because $p$, $q$ and $s$ have dimension zero).

######  Remark 6.

For deterministic systems it can be proved (see, _e.g._ , [30, Proposition
4.1.3]) that functions $\phi_{r+1},...,\phi_{n}$ always exist such that a
normal form exists when $r<n$. While a proof that this property holds in
general in the present setting is missing, in Section III-A we prove that this
property holds for the case in which $m\equiv 0$ near $\bar{x}$.

######  Remark 7.

The notion of stochastic relative degree and normal form presented are
consistent with Itô’s interpretation. This is without loss of generality, as
all the results of this section can be obtained also in other formalisms,
_e.g._ Stratonovich’s formalism [31].

###  III-A Sharper Results for $m\equiv 0$

In the previous section we introduced the concept of relative degree and of
normal form for a class of nonlinear stochastic systems in which the control
input appears both in the drift and in the diffusion terms of the stochastic
differential equation. This allowed us to provide as general as possible
definitions. However, for the remainder of the article, it is beneficial to
make the standing assumption that the control $u$ does not enter the diffusion
term of the stochastic differential equation (_i.e._ $m\equiv 0$) in a
neighbourhood of $\bar{x}$. On the one hand, this sub-class of systems is more
common in the literature (see _e.g._ [21], [22] and references therein). On
the other hand, this assumption allows us to achieve sharper results in terms
of nonlinear control of stochastic systems, both in the idealistic case _i.e._
when the noise process is assumed available) and, in the practically
implementable controller that we develop in Section V-B. A discussion of the
general case (_i.e._ $m\neq 0$) is given in Section V-C. Therefore, for the
time being, we assume $m(x_{t})\equiv 0$ near $\bar{x}$ and we consider
systems of the form

$dx_{t}=(f(x_{t})+g(x_{t})u)dt+l(x_{t})d\mathcal{W}_{t},\qquad
y_{t}=h(x_{t}),$ (25)

or, equivalently,

$\dot{x}_{t}=f(x_{t})+g(x_{t})u+l(x_{t})\xi_{t},\qquad y_{t}=h(x_{t}).$ (26)

In order to prove results in this section it is useful to introduce the
_Stratonovich equivalent_ of system (25), given by

$dx_{t}=(f_{S}(x_{t})+g(x_{t})u)dt+l(x_{t})\circ d\mathcal{W}_{t},\quad
y_{t}=h(x_{t}),$ (27)

where $f_{S}(x)=f(x)-\frac{1}{2}\frac{\partial l(x)}{\partial x}l(x),$ and the
symbol $\circ$ denotes the fact that the stochastic integral is meant in
Stratonovich’s sense. The Itô solution $x_{t}$ of (25) is identical to the
Stratonovich solution of (27) (see, _e.g._ , [32]).

We now discuss what implications follow from restricting ourselves to the
class of systems (26). Firstly, the operator
$\sideset{{}^{m}_{g}}{{}_{l}}{\operatorname{\mathcal{A}}}$, reduces to
$\operatorname{\mathcal{L}}_{g}$, the Lie derivative along the only control
vector field $g$. Secondly, as the control input does not appear in the
diffusion term, the derivative of any function of the state $x_{t}$ is never
quadratic in the control input $u$. In fact one can observe that $u^{2}$
multiplies the operator
$\sideset{{}^{m}}{{}_{m}}{\operatorname{\mathcal{G}}}$, _e.g._ in (8), and
that this operator is identically zero near $\bar{x}$ because so is the vector
field $m$. Having said this, the definition of stochastic relative degree
becomes simpler and for the sake of clarity it is useful to rewrite it.

######  Definition 6.

(Stochastic Relative Degree - $m\equiv 0$) System (26) is said to have
_stochastic relative degree_ $r$ at a point $\bar{x}$ if

(ND)

$\operatorname{\mathcal{L}}_{l}\text{$\sideset{{}^{l}}{{}_{f}^{k}}{\operatorname{\mathcal{S}}}$}h(x)=0$
for all $x$ in a neighborhood of $\bar{x}$ and for all $k\in\\{0,...,r-2\\}$.

(CD)

$\operatorname{\mathcal{L}}_{g}\text{$\sideset{{}^{l}}{{}_{f}^{k}}{\operatorname{\mathcal{S}}}$}h(x)=0$
for all $x$ in a neighborhood of $\bar{x}$ and all $k\in\\{0,...,r-2\\}$.

(RD)

$\operatorname{\mathcal{L}}_{g}\text{$\sideset{{}^{l}}{{}_{f}^{r-1}}{\operatorname{\mathcal{S}}}$}h(\bar{x})\neq
0$.

By considering a class of systems of the form (26), we can achieve sharper
results in terms of definition of a stochastic normal form. Indeed, we can now
prove the claim previously assumed in Assumption 1. To this end, we first
prove a technical result.

######  Lemma 1.

Let $r$ be the stochastic relative degree of system (26) at $\bar{x}$. Then

$\text{$\sideset{{}^{l}}{{}_{f}^{k}}{\operatorname{\mathcal{S}}}$}h=\operatorname{\mathcal{L}}_{f_{S}}^{k}h\qquad\forall
k\in\\{0,...,r-1\\}\quad\text{near $\bar{x}$.}$ (28)

###### Proof.

First observe that by (ND),

$\partial_{x}\left(\operatorname{\mathcal{L}}_{l}\text{$\sideset{{}^{l}}{{}_{f}^{k}}{\operatorname{\mathcal{S}}}$}h\right)=l^{\top}\frac{\partial^{2}\text{$\sideset{{}^{l}}{{}_{f}^{k}}{\operatorname{\mathcal{S}}}$}h}{\partial
x^{2}}+\frac{\partial\text{$\sideset{{}^{l}}{{}_{f}^{k}}{\operatorname{\mathcal{S}}}$}h}{\partial
x}\frac{\partial l}{\partial x}=0,$ (29)

hence

$l^{\top}\frac{\partial^{2}\text{$\sideset{{}^{l}}{{}_{f}^{k}}{\operatorname{\mathcal{S}}}$}h}{\partial
x^{2}}=-\frac{\partial\text{$\sideset{{}^{l}}{{}_{f}^{k}}{\operatorname{\mathcal{S}}}$}h}{\partial
x}\frac{\partial l}{\partial x},$ (30)

for all $k$ in $\\{0,...,r-2\\}$ near $\bar{x}$. Then the claim follows by
induction. In fact, (28) trivially holds for $k=0$. Note that for some
$0<k<r-1$ we have, in view of (30), that

$\displaystyle\text{$\sideset{{}^{l}}{{}_{f}^{k+1}}{\operatorname{\mathcal{S}}}$}h\\!$
$\displaystyle=\\!\operatorname{\mathcal{L}}_{f}\\!\\!\text{$\sideset{{}^{l}}{{}_{f}^{k}}{\operatorname{\mathcal{S}}}$}h+\frac{1}{2}l^{\top}\frac{\partial^{2}\text{$\sideset{{}^{l}}{{}_{f}^{k}}{\operatorname{\mathcal{S}}}$}h}{\partial
x^{2}}l$ (31)
$\displaystyle=\operatorname{\mathcal{L}}_{f}\\!\\!\text{$\sideset{{}^{l}}{{}_{f}^{k}}{\operatorname{\mathcal{S}}}$}h-\frac{1}{2}\frac{\partial\text{$\sideset{{}^{l}}{{}_{f}^{k}}{\operatorname{\mathcal{S}}}$}h}{\partial
x}\frac{\partial l}{\partial
x}l=\operatorname{\mathcal{L}}_{f-\frac{1}{2}\frac{\partial l}{\partial
x}l}\text{$\sideset{{}^{l}}{{}_{f}^{k}}{\operatorname{\mathcal{S}}}$}h$ (32)

Rewriting this equation using in the Lie derivative notation and assuming by
the inductive hypothesis that (28) holds for $0<k<r-1$ yields

$\text{$\sideset{{}^{l}}{{}_{f}^{k+1}}{\operatorname{\mathcal{S}}}$}h=\operatorname{\mathcal{L}}_{f-\frac{1}{2}\frac{\partial
l}{\partial
x}l}\operatorname{\mathcal{L}}_{f_{S}}^{k}h=\operatorname{\mathcal{L}}_{f_{S}}\operatorname{\mathcal{L}}_{f_{S}}^{k}h=\operatorname{\mathcal{L}}_{f_{S}}^{k+1}h.$
(33)

∎

The fact that the stochastic Lie derivatives of $h$ along $f$ and $l$ are
identical to the Lie derivatives of $h$ along $f_{S}$ up to order $r-1$ is
crucial to prove the rest of the results in this and the next section. In
fact, this equivalence allows us to leverage the deterministic techniques,
because the Stratonovich differentiation rule is formally identical to the
deterministic one. We can then use the identities in Lemma 1 to translate the
results to the Itô system (26).

We are now ready to prove the following.

######  Lemma 2.

Let $r$ be the stochastic relative degree of system (26) at $\bar{x}$. Then
the row vectors

$\text{$\partial_{x}$}[h]_{x=\bar{x}},\;\text{$\partial_{x}$}[\text{$\sideset{{}^{l}}{{}_{f}}{\operatorname{\mathcal{S}}}$}h]_{x=\bar{x}},\;...\;,\;\text{$\partial_{x}$}[\text{$\sideset{{}^{l}}{{}_{f}^{r-1}}{\operatorname{\mathcal{S}}}$}h]_{x=\bar{x}},$
(34)

are linearly independent.

###### Proof.

Lemma 1 and the definition of stochastic relative degree of (26) imply that
the set of conditions
$\operatorname{\mathcal{L}}_{g}\operatorname{\mathcal{L}}_{f_{S}}^{k}h=0$ near
$\bar{x}$ for all $k$ in $\\{0,...,r-2\\}$ and
$\operatorname{\mathcal{L}}_{g}\operatorname{\mathcal{L}}_{f_{S}}^{r-1}h\neq
0$ at $\bar{x}$ hold. By [30, Lemma 4.1.1], this set of conditions in turn
implies the linear independence of

$\text{$\partial_{x}$}[h]_{x=\bar{x}},\;\text{$\partial_{x}$}[\operatorname{\mathcal{L}}_{f_{S}}h]_{x=\bar{x}},\;...\;,\;\text{$\partial_{x}$}[\operatorname{\mathcal{L}}_{f_{S}}^{r-1}h]_{x=\bar{x}}.$
(35)

Using again Lemma 1, the claim follows. ∎

For systems of the form (26) the result in Proposition 2 can be specialised to
the existence of a coordinate transformation $z_{t}=\Phi(x_{t})$ such that the
transformed dynamics is given by

$\displaystyle\dot{z}_{i}$ $\displaystyle=z_{i+1},$ $\displaystyle
i=1,...,r-1,$ (36) $\displaystyle\dot{z}_{r}$
$\displaystyle=c(\xi_{t},z_{t})+b(z_{t})u,$ $\displaystyle\dot{z}_{j}$
$\displaystyle=p_{j}(\xi_{t},z_{t})+q_{j}(z_{t})u,\;$ $\displaystyle
j=r+1,...,n,$

where the coefficients $c$ and $p_{j}$ are as before and

$b(z_{t})\\!=\\!\operatorname{\mathcal{L}}_{g}\\!\text{$\sideset{{}^{l}}{{}_{f}^{r-1}}{\operatorname{\mathcal{S}}}$}h(\Phi^{-1}(z_{t})),\quad
q_{j}(z_{t})\\!=\\!\operatorname{\mathcal{L}}_{g}\\!\phi_{j}(\Phi^{-1}(z_{t})).$
(37)

Observe that, unlike the general case in which $m\neq 0$, since now the
coefficients $q_{j}$ are simply the Lie derivatives of $\phi_{j}$ along $g$
for $j=r+1,\dots,n$, it is easy to show that it is always possible to find
$\phi_{r+1},\dots,\phi_{n}$ such that said coefficients are identically zero
for $x$ in a neighbourhood of $\bar{x}$. Exploiting Lemma 1, the proof is
analogous to the one reported in [30, Proposition 4.1.3]. Thus, we can
conclude that it is always possible to find a coordinate transformation
$\Phi(x)$ in a subset $U$ of $\mathbb{R}^{n}$ such that the dynamics in the
transformed state $z_{t}=\Phi(x_{t})$ is

$\displaystyle\dot{z}_{i}$ $\displaystyle=z_{i+1},\quad$ $\displaystyle
i=1,...,r-1,$ (38) $\displaystyle\dot{z}_{r}$
$\displaystyle=c(\xi_{t},z_{t})+b(z_{t})u,$ $\displaystyle\dot{z}_{j}$
$\displaystyle=p_{j}(\xi_{t},z_{t}),$ $\displaystyle j=r+1,...,n,$
$\displaystyle y_{t}$ $\displaystyle=z_{1},$

which is the normal form of system (26).

######  Remark 8.

The notion of normal form which we propose introduces elements of significant
novelty, compared to works on analogous topics like [20], [21] and [22]. By
condition (ND) in Definition 6, the state is projected onto new coordinates,
of which the first $r-1$ have a noise-free dynamics. Not only is this a
fundamental difference with respect to past works, but it also allows us to
introduce, as shown in Sections IV and V, control laws performing path-wise
control of nonlinear stochastic systems. To the best of the authors’
knowledge, this problem has not been systematically addressed in the
literature.

##  IV Idealistic Linearisation via State Feedback

We now turn our attention to the problem of exact feedback linearisation.
Specifically, in this section we formulate and solve an idealistic version of
the problem, by assuming that the control input is a function of the noise
$\xi_{t}$. Although unrealistic because it implies that the noise is available
for feedback, this assumption is necessary to develop the theory of _exact_
linearisation via feedback. This theory is instrumental for the introduction
of a practically implementable controller in Section V which, by approximating
the noise contribution to the dynamics via measurements of the state, achieves
a _practical_ result. The feedback linearisation problem is formally defined
as follows.

######  Problem 1.

(Exact Feedback Linearisation) Consider the nonlinear stochastic system

$\dot{x}_{t}=f(x_{t})+g(x_{t})u+l(x_{t})\xi_{t}.$ (39)

Given a point $\bar{x}$, the problem of _exact feedback linearisation_
consists in finding a neighbourhood $U$ of $\bar{x}$, a feedback law
$u_{t}=k(\xi_{t},x_{t},v)$ affine in $\xi_{t}$, with $v\in\mathbb{R}$, defined
on $U$ and a stochastic coordinate transformation $z_{t}=\Phi(x_{t})$ defined
on $U$ such that the closed-loop system

$\dot{x}_{t}=f(x_{t})+g(x_{t})k(\xi_{t},x_{t},v)+l(x_{t})\xi_{t},$ (40)

in the coordinates $z_{t}=\Phi(x_{t})$, is linear, deterministic and
controllable.

######  Remark 9.

By requiring that the control $u_{t}=k(\xi_{t},x_{t},v)$ is an explicit
function of the white noise, as in the statement of Problem 1, we may be
enlarging the class of systems (26) beyond what could be allowed. In the
general case where $k$ is any nonlinear function of $\xi_{t}$, the resulting
closed-loop system in the derivative notation, _i.e._ (25), is not anymore
equivalent to the one in the differential notation, _i.e._ (26). In fact, this
equivalence is preserved only when the resulting closed-loop dynamics is
affine in the white noise. Therefore, we hereby define an idealistic control
law to be _admissible_ if the dynamics of the closed-loop system is affine in
$\xi_{t}$. This is achieved easily by requiring that $k$ is affine in
$\xi_{t}$ as done in the statement of Problem 1. Note that the idealistic
controls that we introduce in this paper are all admissible by construction.

######  Proposition 3.

Problem 1 is solvable if and only if there exists a real valued function
$h(x_{t})$ such that system (39) with the output $y_{t}=h(x_{t})$ has
stochastic relative degree $n$ at $\bar{x}$.

###### Proof.

_Sufficiency._ Suppose $y_{t}=h(x_{t})$ is such that the stochastic relative
degree of system (39) is $n$ at $\bar{x}$. Then by Proposition 2, there exists
a change of coordinates

$z_{t}=\Phi(x_{t})=\begin{bmatrix}h(x_{t})&\text{$\sideset{{}^{l}}{{}_{f}}{\operatorname{\mathcal{S}}}$}h(x_{t})&\dots&\text{$\sideset{{}^{l}}{{}_{f}^{n-1}}{\operatorname{\mathcal{S}}}$}h(x_{t})\end{bmatrix}^{\top},$
(41)

for all $x$ in a neighbourhood $U$ of $\bar{x}$, such that the normal form of
system (39) is

$\displaystyle\dot{z}_{i}$ $\displaystyle=z_{i+1},\quad i=1,...,n-1,$ (42)
$\displaystyle\dot{z}_{n}$ $\displaystyle=c(\xi_{t},z_{t})+b(z_{t})u.$

By the definition of stochastic relative degree, there exists a neighbourhood
$U$ of $\bar{x}$ such that $b(z_{t})\neq 0$ in $\Phi(U)$. Therefore the
control law

$u_{t}=\tilde{k}(\xi_{t},z_{t},v)=\frac{1}{b(z_{t})}(-c(\xi_{t},z_{t})+v)$
(43)

is well-defined in $\Phi(U)$. Therefore, in the neighbourhood $U$ of $\bar{x}$
the feedback law $u_{t}=\tilde{k}(\xi_{t},\Phi(x_{t}),v)=k(\xi_{t},x_{t},v)$
defined on $U$ brings the transformed system with state $z_{t}=\Phi(x_{t})$
into the form

$\dot{z}_{t}=\begin{bmatrix}0&1&0&\dots&0\\\ 0&0&1&\dots&0\\\
\vdots&\vdots&\vdots&\vdots&\vdots\\\ 0&0&0&\dots&1\\\
0&0&0&\dots&0\end{bmatrix}z_{t}+\begin{bmatrix}0\\\ 0\\\ \vdots\\\ 0\\\
1\end{bmatrix}v=Az_{t}+Bv,$ (44)

where $A$ and $B$ are such that
$\begin{bmatrix}B&AB&\dots&A^{n-1}B\end{bmatrix}$ has rank $n$. The
transformed system is linear, deterministic and controllable, hence Problem 1
is solved.
_Necessity._ This proof is formally identical to the one of [30, Lemma 4.2.1].
In fact, it is possible to show that the stochastic relative degree is
invariant under coordinate change and feedback by using the arguments in [30]
on the Stratonovich equivalent system (27) and then use Lemma 1 to show that
these arguments hold for the Itô system (26) as well. ∎

######  Remark 10.

As the coefficient $c$ is in general a function of $\xi_{t}$, the feedback
linearising control (43) could require the knowledge of the exact value of the
white noise for all $t$, because this is the only way that the noisy dynamics
of the open loop system (39) can be rendered deterministic via feedback. Of
course, the assumption that the noise can be used in the feedback loop is
unrealistic. However, building on the results of this section, in Section V we
design a practical hybrid control scheme. This hybrid scheme implements a
deterministic state feedback law which periodically compensates, in an
approximate way, the noise.

######  Remark 11.

The control $u_{t}=k(\xi_{t},x_{t},v)$ designed in the proof of Proposition 3
is affine in the variable $\xi_{t}$, because in (43) the coefficient $b$ is
not an explicit function of $\xi_{t}$, whilst the coefficient $c$ is affine in
$\xi_{t}$. Therefore, the feedback linearising control is always admissible
because replacing its expression in (26) leaves the closed-loop dynamics
affine in $\xi_{t}$.

To conclude this section, we provide necessary and sufficient conditions for
the existence of an output $y_{t}=h(x_{t})$ which makes the stochastic
relative degree of system (39) equal to $n$ at a point $\bar{x}$.

######  Theorem 1.

Problem 1 is solvable if and only if

  1. 1.

the matrix
$\begin{bmatrix}g(\bar{x})&\text{ad}_{f_{S}}g(\bar{x})&\dots&\text{ad}_{f_{S}}^{n-1}g(\bar{x})\end{bmatrix}$
is invertible and

  2. 2.

the distribution
$\text{span}\\{g,\text{ad}_{f_{S}}g,\dots,\text{ad}_{f_{S}}^{n-2}g\\}$ is
involutive near $\bar{x}$.

###### Proof.

By Proposition 3, Problem 1 is solvable if and only if it is possible to find
an output function $h(x_{t})$ satisfying the conditions in Definition 6. By
Lemma 1, this amounts to the set of conditions

  1. 1.

$\operatorname{\mathcal{L}}_{g}\operatorname{\mathcal{L}}_{f_{S}}^{k}h(x)=0$
for all $x$ in a neighborhood of $\bar{x}$ and all $k\in\\{0,...,r-2\\}$.

  2. 2.

$\operatorname{\mathcal{L}}_{g}\operatorname{\mathcal{L}}_{f_{S}}^{r-1}h(\bar{x})\neq
0$,

which is a set of partial differential equations and a non-triviality
condition. By [30, Lemma 4.1.2] these are equivalent to

  1. 1.

$\operatorname{\mathcal{L}}_{g}\operatorname{\mathcal{L}}_{\text{ad}_{f_{S}}g}^{k}h(x)=0$
for all $x$ in a neighborhood of $\bar{x}$ and all $k\in\\{0,...,r-2\\}$.

  2. 2.

$\operatorname{\mathcal{L}}_{g}\operatorname{\mathcal{L}}_{\text{ad}_{f_{S}}g}^{r-1}h(\bar{x})\neq
0$.

Following the proof of [30, Lemma 4.2.2] it is possible to show that the
previous set of conditions is equivalent to the set of conditions in the
statement of this theorem. ∎

If the stochastic relative degree $r$ of system (26) is strictly less than $n$
at $\bar{x}$, we indicate by
$\zeta_{t}=\begin{bmatrix}z_{1}&\dots&z_{r}\end{bmatrix}^{\top}$ the first $r$
components of the transformed state $z_{t}=\Phi(x_{t})$ and by
$\eta_{t}=\begin{bmatrix}z_{r+1}&\dots&z_{n}\end{bmatrix}^{\top}$ the
remaining $n-r$. Note that if $r<n$ the control law (43) linearises only the
dynamics of $\zeta_{t}$, thus we say that it partially feedback linearises
system (38).

The zero dynamics of a nonlinear stochastic system is, analogously to the
deterministic case, the dynamics of the internal variable $\eta_{t}$ when the
input and the initial conditions are chosen in such a way that the output is
constrained to be identically zero. This is achieved by setting $\zeta_{0}=0$
and $u_{z,t}=-c(\xi_{t},0,\eta_{t})/b(0,\eta_{t})$. Note that the control
$u_{z,t}$ is affine in $\xi_{t}$, hence admissible. We now provide a
definition that extends the concept of zero dynamics to nonlinear stochastic
systems of the form (26).

######  Definition 7.

(_Zero Dynamics_) The stochastic differential equation

$\dot{\eta}_{t}=p(\xi_{t},0,\eta_{t})$ (45)

is called the _zero dynamics_ of system (26).

The properties of the zero dynamics are fundamental in studying the problem of
asymptotic output tracking, which is the topic of Section VI.

##  V Practical Linearisation via State Feedback

In this section we introduce a causal method to estimate the sequence of
variations of the Brownian motion between successive time instants. We then
show that the use of these estimates is beneficial in the design of practical
feedback linearising controls.

###  V-A Estimation of the Brownian motion

The estimation of the Brownian variations is performed by periodically
sampling the state. This method was first introduced in [33] in order to
practically solve the problem of output regulation of _linear_ stochastic
systems. We now extend it to the present context of _nonlinear_ stochastic
systems.

Let $\\{t_{k}\\}_{k\in\mathbb{Z}_{\geq 0}}$ be a sequence of equally-spaced
sampling times, with $t_{k}-t_{k-1}=\varepsilon$ for all
$k\in\mathbb{Z}_{>0}$. Define the differences $\Delta
W_{\varepsilon}(k)=\operatorname{\mathcal{W}}_{t_{k}}-\operatorname{\mathcal{W}}_{t_{k-1}}$
and $\Delta x(k)=x_{t_{k}}-x_{t_{k-1}}$. Our aim is to show that it is
possible to compute a causal estimate $\Delta\widehat{W}_{\varepsilon}(k)$ of
the quantity $\Delta W_{\varepsilon}(k)$ by comparing the samples of the state
of the system at times $t_{k-1}$ and $t_{k}$. In particular, we want this
estimate to “converge”, in a sense to be defined, to the stochastic
differential $d\operatorname{\mathcal{W}}_{t}$ as the sampling period
$\varepsilon$ converges to zero. Let $\mathfrak{L}_{I}$ be the space of
functions that are integrable in Itô’s sense. Then with the notation
$\Delta\widehat{W}_{\varepsilon}\xrightarrow{\varepsilon}d\operatorname{\mathcal{W}}_{t}$
we mean that for all $\alpha\in\mathfrak{L}_{I}$ $\lim_{\varepsilon\rightarrow
0}\sum_{k}\alpha(t_{k-1},w)\Delta\widehat{W}_{\varepsilon}(k)=\int_{0}^{t}\alpha(\tau,w)d\mathcal{W}_{\tau}$
for almost all $w\in\nabla$. For ease of notation, define
$F_{t_{k}}=f(x_{t_{k}})+g(x_{t_{k}})u_{t_{k}}$ and $L_{t_{k}}=l(x_{t_{k}}),$
which are the drift and diffusion coefficients, respectively, of system (25)
evaluated at time $t_{k}$. We make the following assumption.

######  Assumption 2.

There exists $\delta>0$ such that $|L_{t_{k}}|>\delta$ almost surely for all
$k\in\mathbb{Z}_{\geq 0}$.

The rationale of this assumption is explained later, in Remark 12 at the end
of Section V. Under Assumption 2 the Moore-Penrose left pseudo-inverse of
$L_{t_{k}}$, _i.e._
$L_{t_{k}}^{\dagger}=(L_{t_{k}}^{\top}L_{t_{k}})^{-1}L_{t_{k}}^{\top}$, is
well-defined almost surely. The following Lemma extends the results in [33] to
systems with nonlinear drift and diffusion terms.

######  Lemma 3.

Consider system (26) and let Assumption 2 hold. Let
$\\{\Delta\widehat{W}_{\varepsilon}(k)\\}_{k>0}$ be a sequence of scalars
defined as

$\Delta\widehat{W}_{\varepsilon}(k)=L_{t_{k-1}}^{\dagger}\left[\Delta
x(k)-F_{t_{k-1}}\varepsilon\right].$ (46)

Then
$\Delta\widehat{W}_{\varepsilon}(k)\xrightarrow{\varepsilon}d\operatorname{\mathcal{W}}_{t}$
almost surely.

###### Proof.

Let $k\in\mathbb{Z}_{>0}$. By [34, Theorem 7.1]

$\Delta x(k)=F_{t_{k-1}}\varepsilon+L_{t_{k-1}}\Delta
W_{\varepsilon}(k)+o(\varepsilon^{2}),$ (47)

holds, where $o(\varepsilon^{2})$, which is the one-step truncation error of
the forward-Euler scheme, is an infinitesimal of the same order of
$\varepsilon^{2}$. The previous expression can be rewritten as

$L_{t_{k-1}}\Delta W_{\varepsilon}(k)=\Delta
x(k)-F_{t_{k-1}}\varepsilon+o(\varepsilon^{2}).$ (48)

Since $L_{t_{k-1}}$ has full column rank almost surely, the expression

$\Delta W_{\varepsilon}(k)=L_{t_{k-1}}^{\dagger}[\Delta
x(k)-F_{t_{k-1}}\varepsilon+o(\varepsilon^{2})],$ (49)

holds almost surely. Defining $\Delta\widehat{W}_{\varepsilon}(k)$ as in (46)
yields

$\Delta\widehat{W}_{\varepsilon}(k)=\Delta
W_{\varepsilon}(k)+L_{t_{k-1}}^{\dagger}o(\varepsilon^{2}).$ (50)

almost surely. Let $\alpha_{t}\in\mathfrak{L}_{I}$. Then

$\sum_{k}\alpha_{t_{k-1}}\Delta\widehat{W}_{\varepsilon}(k)=\sum_{k}\alpha_{t_{k-1}}(\Delta
W_{\varepsilon}(k)+L_{t_{k-1}}^{\dagger}o(\varepsilon^{2})).$ (51)

Taking the limit of both sides as $\varepsilon$ tends to zero yields
$\Delta\widehat{W}_{\varepsilon}\xrightarrow{\varepsilon}d\mathcal{W}_{t}$,
since for all $\alpha\in\mathfrak{L}_{I}$

$\lim_{\varepsilon\rightarrow
0}\sum_{k}\alpha_{t_{k-1}}L_{t_{k-1}}^{\dagger}o(\varepsilon^{2})=0\quad\text{almost
surely.}$ (52)

∎

###  V-B Hybrid control law

In this section we discuss how the sequence of estimates
$\\{\Delta\widehat{W}_{\varepsilon}(k)\\}_{k}$ can be used to design a
practically implementable hybrid feedback control law that approximates the
idealistic input (43), namely $u_{t}^{lin}=(-c(\xi_{t},z_{t})+v)/b(z_{t})$,
which linearises the dynamics of the first $r$ components of system (38). We
show that the accuracy can be improved by increasing the sampling time
$\varepsilon$. We formally state the problem as follows.

######  Problem 2.

(Practical Feedback Linearisation) Consider system (38) and let $z_{t}^{lin}$
be its solution when $u_{t}=u_{t}^{lin}$. The problem of _practical feedback
linearisation_ consists in finding a control law
$\hat{u}^{lin}_{t}(\varepsilon)$, depending on the sampling rate
$\varepsilon$, such that, if $z_{t}$ is the solution of (38) when
$u_{t}=\hat{u}^{lin}_{t}$, then for every $\sigma>0$

$\lim_{\varepsilon\rightarrow
0}\mathfrak{P}(\left|z_{t}-z_{t}^{lin}\right|\geq\sigma)=0.$ (53)

The meaning of Problem 2 is to find a causal controller for which the closed-
loop system dynamics converges in probability to the idealistic closed-loop
dynamics as the sampling time is made smaller. To begin with, since the
coefficient $c$ is affine in $\xi_{t}$, we can express it as
$c(\xi_{t},z_{t})=c_{d}(z_{t})+c_{s}(z_{t})\xi_{t}$ for some mappings $c_{d}$
and $c_{s}$. Note that since in practice $\xi_{t}$ is not known, a naive
approximation of $\xi_{t}$ boils down to replacing it with its expectation,
_i.e._ zero. Therefore, a first causal approximation of the coefficient $c$ is
given by only the term $c_{d}(z_{t})$, in turn implying that the control
$u_{t}^{lin}$ can be practically approximated by the naive law

$u^{zn,lin}_{t}=\frac{-c_{d}(z_{t})+v}{b(z_{t})}.$ (54)

We call this basic feedback law the _zero-noise control_. By replacing
$c_{s}(z_{t})\xi_{t}$ with zero in the expression of $c(\xi_{t},z_{t})$, the
zero-noise control does not perform any form of stochastic compensation when
performing feedback linearisation. Consequently, there is no guarantee that
the closed-loop behaviour of system (38) with $u_{t}=u_{t}^{zn,lin}$ is any
close to its idealistic behaviour, _i.e._ when $u_{t}=u_{t}^{lin}$.

The goal of this section is to improve the performance of the zero-noise
control $u_{t}^{zn,lin}$ by leveraging the estimated sequence
$\\{\Delta\widehat{W}_{\varepsilon}(k)\\}_{k}$ and show that we can recover
the idealistic behaviour in probability. Therefore, we define
$\hat{u}_{t}^{lin}=u^{zn,lin}_{t}+u^{s}_{t}$, with $u^{s}_{t}$ to be specified
in such a way that $\hat{u}_{t}^{lin}$ is a “better” approximation of
$u_{t}^{lin}$ than $u_{t}^{zn,lin}$. By replacing the control
$\hat{u}_{t}^{lin}$ in (38), the dynamics of the transformed system becomes

$\displaystyle\dot{z}_{i}$ $\displaystyle=z_{i+1},\qquad\qquad i=1,\dots,r-1,$
(55) $\displaystyle\dot{z}_{r}$
$\displaystyle=v+c_{s}(z_{t})\xi_{t}+b(z_{t})u^{s}_{t},$
$\displaystyle\dot{\eta}_{t}$ $\displaystyle=p(\xi_{t},\zeta_{t},\eta_{t}),$
$\displaystyle y_{t}$ $\displaystyle=z_{1},$

where the term $c_{s}(z_{t})\xi_{t}$ in the dynamics of the $r$-th component
is due to the fact that the approximating control $u^{zn,lin}_{t}$ cannot
cancel the noisy dynamics as the idealistic control $u_{t}^{lin}$ does.

We now want to design the control $u_{t}^{s}$ employing the estimates
$\\{\Delta\widehat{W}_{\varepsilon}(k)\\}_{k}$ introduced in Section V-A, to
reduce the contribution of the term $c_{s}(z_{t})\xi_{t}$ onto the dynamics of
the system. Since the quantity $\Delta\widehat{W}_{\varepsilon}(k)$ carries
information on the evolution of the noise between $t_{k-1}$ and $t_{k}$, we
look at the evolution of $z_{r}$ between these two consecutive sampling times.
The value of $z_{r}$ at time $t_{k}$ is given by

$z_{r,t_{k}}=z_{r,t_{k-1}}+\int_{t_{k-1}}^{t_{k}}\\!\\!\\!\\!\\!\\!\\!vd\tau+\beta_{d}(k)+\int_{t_{k-1}}^{t_{k}}\\!\\!\\!\\!\\!\\!\\!b(z_{\tau})u^{s}_{\tau}d\tau,$
(56)

where

$\beta_{d}(k)=\int_{t_{k-1}}^{t_{k}}\\!\\!\\!\\!\\!\\!\\!c_{s}(z_{\tau})d\operatorname{\mathcal{W}}_{\tau}$
(57)

is the contribution of the noise on the dynamics of $z_{r}$ between the two
sampling times. Our goal is to minimise this contribution using $u^{s}_{t}$
and the estimate $\Delta\widehat{W}_{\varepsilon}(k)$ obtained at time
$t_{k}$. The fact that $\Delta\widehat{W}_{\varepsilon}(k)$ is only available
_a posteriori_ , at the end of the sampling period, suggests that $u^{s}_{t}$
should induce a jump variation at time $t_{k}$ in the state $z_{r}$ in order
to compensate for the quantity $\beta_{d}(k)$. In other words, the dynamics of
the closed-loop system should be hybrid. It is then necessary to introduce a
simplified jump notation. At time $t_{k}$, we denote by $z_{t_{k}}$ the state
before the jump, and by $z_{t_{k}^{+}}$ the state after the jump. The flow
dynamics of the closed-loop hybrid system we seek is, therefore, given by

$\displaystyle\dot{z}_{i}$ $\displaystyle=z_{i+1},\qquad\qquad i=1,\dots,r-1,$
(58) $\displaystyle\dot{z}_{r}$ $\displaystyle=v+c_{s}(z_{t})\xi_{t},$
$\displaystyle\dot{\eta}_{t}$ $\displaystyle=p(\xi_{t},\zeta_{t},\eta_{t}),$

for all $t\in\mathbb{R}_{\geq 0}$, while the jump dynamics is given by

$\displaystyle z_{i,t_{k}^{+}}$ $\displaystyle=z_{i,t_{k}},\qquad\qquad
i=1,\dots,r-1,$ (59) $\displaystyle z_{r,t_{k}^{+}}$
$\displaystyle=z_{r,t_{k}}+b(z_{t_{k}})u^{*}(k),$
$\displaystyle\eta_{t_{k}^{+}}$ $\displaystyle=\eta_{t_{k}},$

for all $k\in\mathbb{Z}_{>0}$, where $\\{u^{*}(k)\\}_{k}$ is a yet to be
defined sequence of scalars depending on
$\\{\Delta\widehat{W}_{\varepsilon}(k)\\}_{k}$. Alternatively, the
aforementioned hybrid dynamics (58)-(59) can be equivalently produced by using
in (55) an impulsive control $u_{t}^{s}$ given by

$u^{s}_{t}=\sum_{i=1}^{k}u^{*}(i)\delta(t-t_{i}),\qquad t\leq t_{k},$ (60)

where $\delta(t)$ is a Dirac delta. As a result of the hybrid dynamics induced
by the control $u_{t}^{s}$, the expression of $z_{r}$ before the jump at time
$t_{k}$ is given by

$z_{r,t_{k}}=z_{r,t_{k-1}^{+}}+\int_{t_{k-1}}^{t_{k}}\\!\\!\\!\\!\\!\\!\\!vd\tau+\beta_{d}(k),$
(61)

while after the jump by

$\displaystyle z_{r,t_{k}^{+}}$
$\displaystyle=z_{r,t_{k}}+b(z_{t_{k}})u^{*}(k)$ (62)
$\displaystyle=z_{r,t_{k-1}^{+}}+\int_{t_{k-1}}^{t_{k}}\\!\\!\\!\\!\\!\\!\\!vd\tau+\beta_{d}(k)+b(z_{t_{k}})u^{*}(k).$
(63)

Thus, we have reduced the problem of approximate partial feedback
linearisation to the problem of finding the sequence $\\{u^{*}(k)\\}_{k}$ such
that the contribution of the term $\beta_{d}(k)+b(z_{t_{k}})u^{*}(k)$ is
minimised. In particular, we look for a sequence $\\{u^{*}(k)\\}_{k}$ which
retrieves the exact linearisation of the dynamics of $\zeta_{t}$ as
$\varepsilon$ tends to zero. We are now ready to solve Problem 2.

######  Theorem 2.

Consider system (38) and let Assumption 2 hold. Let the control
$\hat{u}_{t}^{lin}$ be given by

$\hat{u}_{t}^{lin}=\frac{-c_{d}(z_{t})+v}{b(z_{t})}-\sum_{i=1}^{k}\frac{c_{s}(z_{t_{i-1}^{+}})\Delta\widehat{W}_{\varepsilon}(i)}{b(z_{t_{i}})}\delta(t-t_{i}),\quad
t\leq t_{k},$ (64)

with $\Delta\widehat{W}_{\varepsilon}(k)$ given by (46). Then
$\hat{u}_{t}^{lin}$ solves Problem 2.

###### Proof.

It is trivial to observe that, since system (38) is in normal form, (53) holds
if and only if it holds for the $r$-th component of the state $z_{t}$. Thus,
we now focus on the $r$-th component. Under the control $\hat{u}_{t}^{lin}$,
the jump of $z_{r}$ at $t_{k}$ is given by

$z_{r,t_{k}^{+}}=z_{r,t_{k}}-c_{s}(z_{t_{k-1}^{+}})\Delta\widehat{W}_{\varepsilon}(k),$
(65)

thus, in light of the previous discussion,

$z_{r,t_{k}^{+}}=z_{r,t_{k-1}^{+}}+\int_{t_{k-1}}^{t_{k}}\\!\\!\\!\\!\\!\\!\\!vd\tau+\int_{t_{k-1}}^{t_{k}}\\!\\!\\!\\!\\!\\!\\!c_{s}(z_{\tau})d\mathcal{W}_{\tau}-c_{s}(z_{t_{k-1}^{+}})\Delta\widehat{W}_{\varepsilon}(k).$
(66)

This can be rewritten as

$z_{r,t_{k}^{+}}=z_{r,t_{0}}+\int_{t_{0}}^{t_{k}}\\!\\!\\!\\!\\!vd\tau+\int_{t_{0}}^{t_{k}}\\!\\!\\!\\!\\!c_{s}(z_{\tau})d\mathcal{W}_{\tau}-\sum_{i=1}^{k}c_{s}(z_{t_{i-1}^{+}})\Delta\widehat{W}_{\varepsilon}(i).$
(67)

By Lemma 3

$\lim_{\varepsilon\rightarrow
0}\sum_{i=1}^{k}c_{s}(z_{t_{i-1}^{+}})\Delta\widehat{W}_{\varepsilon}(i)=\int_{t_{0}}^{t_{k}}\\!\\!\\!\\!\\!c_{s}(z_{\tau})d\mathcal{W}_{\tau}\quad\text{almost
surely},$ (68)

hence

$\lim_{\varepsilon\rightarrow
0}z_{r,t_{k}^{+}}=z_{r,t_{0}}+\int_{t_{0}}^{t_{k}}\\!\\!\\!\\!\\!vd\tau=z_{r,t_{k}}^{lin}\quad\text{almost
surely.}$ (69)

Now, consider any $t\in(t_{k-1},t_{k})$, for some $k$, and let
$0<\bar{\varepsilon}<\varepsilon$ be given by $\bar{\varepsilon}=t-t_{k-1}$.
Then

$z_{r,t}=z_{r,t_{k-1}^{+}}+\int_{t_{k-1}}^{t}\\!\\!\\!\\!\\!\\!\\!vd\tau+\int_{t_{k-1}}^{t}\\!\\!\\!\\!\\!\\!\\!c_{s}(z_{\tau})d\mathcal{W}_{\tau},$
(70)

which can be discretised with the Euler-Maruyama method as

$z_{r,t}=z_{r,t_{k-1}^{+}}+v(z_{t_{k-1}^{+}})\bar{\varepsilon}+c_{s}(z_{t_{k-1}^{+}})\Delta
W_{\bar{\varepsilon}}(k)+o(\bar{\varepsilon}^{2}),$ (71)

with $\Delta W_{\bar{\varepsilon}}(k)=\mathcal{W}_{t}-\mathcal{W}_{t_{k-1}}$,
and where $o(\bar{\varepsilon}^{2})$ is the one-step truncation error, which
is of order $\bar{\varepsilon}^{2}$. Similarly, the discretised dynamics of
$z_{r,t}^{lin}$ is

$z_{r,t}^{lin}=z_{r,t_{k-1}}^{lin}+v(z_{t_{k-1}}^{lin})\bar{\varepsilon}+o(\bar{\varepsilon}^{2}).$
(72)

The difference $z_{r,t}-z_{r,t}^{lin}$ is therefore

$z_{r,t}-z_{r,t}^{lin}=z_{r,t_{k-1}^{+}}-z_{r,t_{k-1}}^{lin}+(v(z_{t_{k-1}^{+}})-v(z_{t_{k-1}}^{lin}))\bar{\varepsilon}+\\\
c_{s}(z_{t_{k-1}^{+}})\Delta
W_{\bar{\varepsilon}}(k)+o(\bar{\varepsilon}^{2})$ (73)

Observe that, by the properties of the Brownian motion, $\Delta
W_{\bar{\varepsilon}}(k)$ is a normally distributed random variable with zero
expectation and variance $\bar{\varepsilon}$. Moreover $c_{s}$ is bounded near
zero, because it is the Lie derivative of a smooth function. By taking the
limit of (73) as $\varepsilon$ (hence $\bar{\varepsilon}$) goes to zero and
using (69), we have that for every $\sigma$

$\lim_{\varepsilon\rightarrow
0}\mathfrak{P}(\left|z_{r,t}-z_{r,t}^{lin}\right|\geq\sigma)=0,$ (74)

hence the claim follows. ∎

The previous proposition states that, when $u^{s}_{t}$ is selected as

$u^{s}_{t}=-\sum_{i=1}^{k}\frac{c_{s}(z_{t_{i-1}^{+}})\Delta\widehat{W}_{\varepsilon}(i)}{b(z_{t_{i}})}\delta(t-t_{i}),\quad
t\leq t_{k},$ (75)

the control $\hat{u}_{t}^{lin}$ approximates the idealistic control
$u_{t}^{lin}$ as $\varepsilon$ tends to zero, in the sense that the dynamics
of the variable $\zeta_{t}$ can be made approximately linear with an accuracy
increasing as $\varepsilon$ decreases.

######  Remark 12.

The rationale of Assumption 2 is that the noise is persistently exciting, so
it can be estimated by measuring the state of the system. From a technical
viewpoint, this assumption is necessary for the development of the theory
because it ensures the boundedness of $L_{t_{k}}^{\dagger}$. However,
Assumption 2 should not be considered practically restrictive. In fact, if
$|L_{t_{k}}|<\bar{\delta}$, for an arbitrarily small $\bar{\delta}$, this
would imply that the noise contribution to the dynamics of the system is
sufficiently small at time $t_{k}$ to be considered negligible. In that case
it is simply possible to avoid performing the stochastic compensation at time
$t_{k}$ and still obtain good control performance.

###  V-C Remarks on the control input in the diffusion: $m\neq 0$

At the beginning of Section IV we made the standing assumption that
$m(x)\equiv 0$ near $\bar{x}$, thus restricting the class of systems
considered to those with the control input appearing only in the drift term of
the stochastic differential equations. This assumption allowed us to obtain
sharper analytical results. In this section we discuss the rationale of such
assumption from a control viewpoint, which has implications both in the
idealistic and the practical frameworks.

As shown in (24), for general $m$, the dynamics of the $r$-th component of the
transformed state $z_{t}=\Phi(x_{t})$ is a quadratic function of the control
input. This implies that, in the case that
$a(z_{t})=(1/2)\text{$\sideset{{}^{m}}{{}_{m}}{\operatorname{\mathcal{G}}}$}\text{$\sideset{{}^{l}}{{}_{f}^{r-1}}{\operatorname{\mathcal{S}}}$}h(\Phi^{-1}(z_{t}))$
is not zero for $x$ near $\bar{x}$, the idealistic feedback linearising
control would have the form

$u_{t}^{lin}=\frac{-b(\xi_{t},z_{t})\pm\sqrt{b(\xi_{t},z_{t})^{2}-4a(z_{t})(c(\xi_{t},z_{t})-v)}}{2a(z_{t})},$
(76)

which has real values if and only if the input $v$ is such that
$b(\xi_{t},z_{t})^{2}-4a(z_{t})(c(\xi_{t},z_{t})-v)\geq 0$. The corresponding
zero-noise control would have the form

$u_{t}^{zn,lin}=\frac{-b_{d}(z_{t})\pm\sqrt{b_{d}(z_{t})^{2}-4a(z_{t})(c_{d}(z_{t})-v)}}{2a(z_{t})},$
(77)

where the input $v$ should enforce
$b_{d}(z_{t})^{2}-4a(z_{t})(c_{d}(z_{t})-v)\geq 0$. It is evident that,
although in both cases the control input can be forced to be real by an
appropriate choice of $v$, from a practical viewpoint such a choice may not
leave any space for a control law achieving objectives such as, for instance,
output tracking. Moreover, even though we may be able to define a zero-noise
control, the construction of a hybrid controller presents even more
challenges, as the quadratic $u_{t}^{lin}$ is not affine in $\xi_{t}$. Of
course, one might make the standing assumption that $a(z_{t})\equiv 0$ for $x$
near $\bar{x}$. This class of systems would include the systems for which
$m\equiv 0$, which we address in detail in this paper, but also those systems
with $m\neq 0$ with the additional assumption that
$\text{$\sideset{{}^{m}}{{}_{m}}{\operatorname{\mathcal{G}}}$}\text{$\sideset{{}^{l}}{{}_{f}^{r-1}}{\operatorname{\mathcal{S}}}$}h(\Phi^{-1}(z_{t}))=0$.
However, the design of controllers poses substantial technical challenges even
in this case. For the sake of completeness, we briefly give an overview of
these issues. Suppose $m\neq 0$ and $a(z_{t})\equiv 0$ for $x$ near $\bar{x}$.
The coefficient $b$ in (24) is therefore affine in $\xi_{t}$, _i.e._ ,
$b(\xi_{t},z_{t})=b_{d}(z_{t})+b_{s}(z_{t})\xi_{t}$. In the idealistic
framework, a feedback linearising control would require a division by
$b(\xi_{t},z_{t})$. Unless further, possibly restrictive, assumptions are
made, such a division may result in a non-admissible control. Additionally,
the fact that the coefficient $b$ depends on the noise affects impulsive
compensations as well. In fact, the dynamics of $z_{r}$ when the control
$\hat{u}_{t}^{lin}$ is applied is

$\dot{z}_{r}=v+c_{s}(z_{t})\xi_{t}+b_{d}(z_{t})u^{s}_{t}+b_{s}(z_{t})u_{t}^{s}\xi_{t}.$
(78)

Thus any compensating control $u_{t}^{s}$ inevitably introduces noise at time
$t_{k}^{+}$ which cannot be compensated for because $\xi_{t_{k}}$ is a random
variable which is independent of the process $\xi_{t}$, $t<t_{k}$. This makes
it impossible to conclude on the convergence in probability of
$z_{r,t_{k}^{+}}$ to $z_{r,t_{k}}^{lin}$ as $\varepsilon$ approaches zero,
because the random variable $\xi_{t_{k}}$ can take arbitrarily large values in
$\mathbb{R}$ with nonzero probability.

##  VI Asymptotic Output Tracking

In this section we first design an idealistic controller, _i.e._ using the
white noise in the feedback loop, to make the output asymptotically track a
reference signal. Then we show that a practical feedback control law
leveraging the causal stochastic compensations introduced in Section V is able
to retrieve the idealistic result, in the limit of the sampling time
$\varepsilon$ going to zero.

We start by introducing the idealistic control and we show that, under
suitable stability hypotheses on the zero dynamics, it is possible to control
the system so that its output tracks reference trajectories while its internal
variables remain bounded almost surely. First observe that, as long as
$z_{t}=\Phi(x_{t})$ is chosen such that $q\equiv 0$ near zero (which is always
possible, see Section III-A), the zero dynamics of system (38) is affine in
$\xi_{t}$. Let it be expressed as

$\dot{\eta}_{t}=p(\xi_{t},0,\eta_{t})=p_{d}(0,\eta_{t})+p_{s}(0,\eta_{t})\xi_{t}.$
(79)

Consider a reference signal $y_{R}$ which is continuously differentiable $r$
times with values in a neighbourhood of zero. We assume that the initial state
of the transformed system (38) is arbitrary while in a neighbourhood of zero
and we seek a feedback control $u_{t}$ that makes the output $y_{t}$ of the
system asymptotically converge to $y_{R}$. Let

$v(\zeta_{t},y_{R}(t))=y_{R}^{(r)}-\sum_{i=1}^{r}d_{i-1}(\zeta_{i}-y_{R}^{(i-1)}),$
(80)

with $d_{i}\in\mathbb{R}$ for $i=0,\dots,r-1$ to be determined, and the
admissible idealistic feedback control law be given by

$u_{t}^{track}=-\frac{c(\xi_{t},\zeta_{t},\eta_{t})-v(\zeta_{t},y^{R}(t))}{b(\zeta_{t},\eta_{t})}.$
(81)

Define the tracking error $e_{t}:=y_{t}-y_{R}(t)$. Then the control
$u_{t}^{track}$ forces the dynamics of the tracking error to be
$e^{(r)}_{t}+d_{r-1}e^{(r-1)}_{t}+...+d_{1}e^{(1)}_{t}+d_{0}e_{t}$, which can
be made exponentially stable by selecting the coefficients $d_{i}$ such that

$\Lambda(s)=s^{r}+d_{r-1}s^{r-1}+...+d_{1}s+d_{0},$ (82)

which is the characteristic polynomial of the matrix

$A=\begin{bmatrix}0&1&0&\dots&0\\\ 0&0&1&\dots&0\\\
\vdots&\vdots&\vdots&\vdots&\vdots\\\ 0&0&0&\dots&1\\\
-d_{0}&-d_{1}&-d_{2}&\dots&-d_{r-1}\end{bmatrix},$ (83)

has roots with negative real parts. We also study the boundedness of
$\zeta_{t}$ and of the internal variable $\eta_{t}$ under the control
$u_{t}^{track}$, when $y_{R}$ and its first $r-1$ derivatives are bounded.
Define
$\zeta_{R}(t)=\begin{bmatrix}y_{R}(t)&\dots&y_{R}^{(r-1)}(t)\end{bmatrix}^{\top}$
and $\theta_{t}=\begin{bmatrix}e_{t}&\dots&e^{(r-1)}_{t}\end{bmatrix}^{\top}$.
Then the following result, the proof of which relies on the definition of
strict Lyapunov function and some technical lemmas presented in Appendix B,
holds.

######  Theorem 3.

Consider system (38). Suppose $y_{R}(t)$,
$y_{R}^{(1)}(t),\dots,y^{(r-1)}_{R}(t)$ are bounded. Let $\eta_{R,t}$ be the
solution of

$\dot{\eta}_{R,t}=p(\xi_{t},\zeta_{R}(t),\eta_{R,t}),\quad\eta_{R,0}=0$ (84)

and let $p_{d}$ and $p_{s}$ be Lipschitz continuous. Moreover, assume that
there exists a strict Lyapunov function $V(\eta_{R},t)$ for (84) such that
$\frac{\partial V}{\partial\eta_{R,i}}(x,t)$ and
$\frac{\partial^{2}V}{\partial\eta_{R,i}\partial\eta_{R,j}}(x,t)$ are bounded
for all $x$ in a neighbourhood of the origin and $t\geq 0$. Suppose that the
roots of the polynomial $\Lambda(s)$ in (82) have negative real part. Then for
sufficiently small $\epsilon_{R}>0$, if

$|z_{i}(\bar{t})-y_{R}^{(i-1)}(\bar{t})|<\epsilon_{R},\;\;1\leq i\leq
r,\qquad\|\eta_{\bar{t}}-\eta_{R,\bar{t}}\|<\epsilon_{R},$ (85)

then for all $\epsilon>0$ there exists $\delta>0$ such that

$|z_{i}(\bar{t})-y_{R}^{(i-1)}(\bar{t})|<\delta\rightarrow|z_{i}(t)-y_{R}^{(i-1)}(t)|<\epsilon,\\\
1\leq i\leq r,\;\text{for all $t\geq\bar{t}\geq 0$},$ (86)
$\|\eta_{\bar{t}}-\eta_{R,\bar{t}}\|<\delta\rightarrow\|\eta_{t}-\eta_{R,t}\|<\epsilon\;\;\text{for
all $t\geq\bar{t}\geq 0$},$ (87)

almost surely, _i.e._ the response $z_{i}$ and $\eta_{t}$, $t\geq\bar{t}\geq
0$, of system (38) under the control law $u^{track}_{t}$ is bounded almost
surely.

###### Proof.

System (38) under the control law $u^{track}_{t}$ can be rewritten in the form

$\dot{\theta}_{t}=A\theta_{t},\quad\dot{\eta}_{t}=p(\xi_{t},\zeta_{R}(t)+\theta_{t},\eta_{t}),$
(88)

with $A$ given by (83), which has characteristic polynomial $\Lambda(s)$.
Therefore $\theta_{t}$ has asymptotically stable dynamics. Let
$\nu_{t}=\eta_{t}-\eta_{R,t}$ and
$P(\xi_{t},\nu_{t},\theta_{t},t)=p(\xi_{t},\zeta_{R}(t)+\theta_{t},\eta_{R,t}+\nu_{t})-p(\xi_{t},\zeta_{R}(t),\eta_{R,t})$.
Note that the system

$\dot{\nu}_{t}=P(\xi_{t},\nu_{t},\theta_{t},t),\quad\dot{\theta}_{t}=A\theta_{t}$
(89)

is in the form (110) (see Appendix B). Moreover, system (89) additionally
satisfies the hypotheses of Lemma 8 in Appendix B because of the assumption of
Lipschitz continuity of $p_{d}$ and $p_{s}$ and of the existence of $V$ as in
the statement. Then, by Lemma 8, $(0,\eta_{R,t})$ is an almost surely
uniformly stable solution of (88) and the claimed estimates follow. ∎

The previous theorem solves the idealistic local asymptotic output tracking
problem, _i.e._ the output $y_{t}=z_{1}$ asymptotically converges to $y_{R}$
whilst the state $z_{t}$ remains bounded almost surely.

We now focus on practical output tracking. While it can be proved (see
Proposition 4 in Appendix C) that under some technical assumptions a zero-
noise control is sufficient to asymptotically stabilise the equilibrium at the
origin, a controller not performing any sort of compensation for the
stochastic disturbances does not guarantee asymptotic tracking. This is
because if the system tracks a non-zero reference, then the states will not
converge to zero and so the noise will enter the dynamics in a persistent
fashion. Hence the noise might drive the states away from the desired
trajectory, possibly inducing instability. On the contrary, the control law
$\hat{u}_{t}^{track}=u_{t}^{zn,track}+u^{s}_{t}$, with

$u_{t}^{zn,track}=-\frac{c_{d}(\zeta_{t},\eta_{t})-v(\zeta_{t},y^{R}(t))}{b(\zeta_{t},\eta_{t})}$
(90)

and $u_{t}^{s}$ given by (75) is able to prevent this when $\varepsilon$ tends
to zero by approximately compensating for the Brownian-induced disturbances,
as shown in the following result.

######  Corollary 1.

Consider system (38) and suppose that Assumption 2 and the assumptions in
Theorem 3 hold. Then the control law $\hat{u}_{t}^{track}$ is such that
$y_{t}$ converges to $y_{R}$ in probability and the state $z_{t}$,
$t\geq\bar{t}\geq 0$, of system (38) is bounded in probability in the limit as
$\varepsilon$ approaches zero.

###### Proof.

By Theorem 3 the control $u_{t}^{track}$ makes the output $y_{t}$ of (38)
asymptotically converge to $y_{R}$ while keeping the internal states bounded.
Let $z_{t}^{track}$ and $z_{t}$ be the the state of system (38) when
$u_{t}=u_{t}^{track}$ and $u_{t}=\hat{u}_{t}^{track}$ are applied,
respectively. Then, by Theorem 2, for every $\sigma>0$,

$\lim_{\varepsilon\rightarrow
0}\mathfrak{P}(\left|z_{t}-z_{t}^{track}\right|\geq\sigma)=0,$ (91)

and the claim follows. ∎

##  VII Illustrative Example

In this section we illustrate the validity of the theory by means of a
numerical example. Consider the following nonlinear stochastic system in the
form (26) with

$f(x_{t})=\begin{bmatrix}s_{2}(1+x_{1})\\\ -2\tan x_{2}\\\
f_{3}(x_{t})\end{bmatrix},\;g(x_{t})=\begin{bmatrix}e^{x_{3}}\\\ 0\\\
e^{x_{3}}\end{bmatrix},\;l(x_{t})=\begin{bmatrix}x_{1}\vspace{1mm}\\\
\frac{-2x_{1}}{c_{2}}\vspace{1mm}\\\ x_{1}^{2}\end{bmatrix},$ (92)

with $f_{3}(x_{t})=2x_{3}+x_{1}s_{2}-\frac{2s_{2}x_{1}^{2}}{c_{2}^{2}}$ and
$s_{i}$ and $c_{i}$ denoting $\sin{x_{i}}$ and $\cos{x_{i}}$ respectively. Let
the output of the system be $y_{t}=x_{1}+s_{2}-x_{3}$. We are interested in
analysing the system around the origin, _i.e._ we set $\bar{x}=0$. The goal is
to bring the system to its normal form and to perform asymptotic output
tracking. The first step is determining the stochastic relative degree of the
system at zero. We set $z_{1}=y_{t}=x_{1}+s_{2}-x_{3}$ and we compute its
derivative applying Itô’s formula (we omit the procedure for brevity), thus
obtaining $dz_{1}=(-2x_{3}-s_{2})dt$. As neither the input $u$ nor the noise
appear in this expression, the stochastic relative degree, if defined, is
higher than one at the origin. We set $z_{2}=-2x_{3}-s_{2}$ and, by computing
its derivative, we obtain

$dz_{2}=\left(2s_{2}-4x_{3}-2x_{1}s_{2}+\frac{6x_{1}^{2}s_{2}}{c_{2}^{2}}-2e^{x_{3}}u\right)dt+4x_{1}d\mathcal{W}_{t}.$
(93)

The system has therefore stochastic relative degree $r=2$ at the origin.
Moreover,

$\displaystyle\tilde{c}_{d}(x_{t})$
$\displaystyle=c_{d}(\Phi(x_{t}))=2s_{2}-4x_{3}-2x_{1}s_{2}+\frac{6x_{1}^{2}s_{2}}{c_{2}^{2}},$
(94) $\displaystyle\tilde{c}_{s}(x_{t})$
$\displaystyle=c_{s}(\Phi(x_{t}))=-2e^{x_{3}},$ (95)
$\displaystyle\tilde{b}(x_{t})$ $\displaystyle=b(\Phi(x_{t}))=4x_{1}.$ (96)

Setting $z_{3}=x_{1}-x_{3}$ makes the coordinate change $z=\Phi(x)$ a
diffeomorphism in a neighbourhood of the origin, with

$\Phi(x)=\begin{bmatrix}x_{1}+s_{2}-x_{3}\\\ -s_{2}-2x_{3}\\\
x_{1}-x_{3}\end{bmatrix}.$ (98)

In this new set of coordinates the dynamics of the system is given by

$\displaystyle\dot{z}_{1}$ $\displaystyle=z_{2},$ (99)
$\displaystyle\dot{z}_{2}$
$\displaystyle=c_{d}(z_{t})+c_{s}(z_{t})\xi_{t}+b(z_{t})u,$ (100)
$\displaystyle\dot{z}_{3}$
$\displaystyle=s_{2}-2x_{3}+\frac{2x_{1}^{2}s_{2}}{c_{2}^{2}}+2x_{1}\xi_{t}=\tilde{p}(\xi_{t},\Phi(x_{t})),$
(101)

which is in the stochastic normal form. The zero dynamics of the system is
obtained by equating $z_{1}=0$ and $z_{2}=0$, $z_{3}=\eta_{t}$, which yields
$x_{1}=(3/2)\eta_{t}$, $s_{2}=-\eta_{t}$. Replacing these in the third
equation in (99) we get the zero dynamics as follows

$\dot{\eta}_{t}=p(\xi_{t},0,\eta_{t})=-2\eta_{t}+\frac{9\eta_{t}^{3}}{2(\eta_{t}^{2}-1)}+3\eta_{t}\xi_{t}.$
(102)

Its first approximation around the origin is
$\dot{\eta}_{t}=-2\eta_{t}+3\eta_{t}\xi_{t}=A\eta_{t}+F\eta_{t}\xi_{t}$, which
is asymptotically stable almost surely because $A-F^{2}/2<0$. Therefore the
zero dynamics of the system is almost surely asymptotically stable. We now
choose a reference signal of the form $y_{R}(t)=\beta+\alpha\cos(\omega t)$
and illustrate that $u_{t}^{track}$ achieves asymptotic output tracking. To
this end, we first performed a simulation in the idealistic scenario in which
the noise is used in the feedback law. We selected the coefficients $d_{0}=12$
and $d_{1}=7$ in the input $v$ so the characteristic polynomial $\Lambda(s)$
in (82) has roots in $-3$ and $-4$, thus guaranteeing asymptotic stability of
the linearised sub-system in the coordinates $\zeta_{t}$. The reference signal
$y_{R}$ is characterised by $\beta=0.1$, $\alpha=0.01$ and $\omega=5$. The
nonlinear stochastic differential equations were integrated using the Euler-
Maruyama scheme with period $\Delta t=10^{-6}$. In Figure 1 we show the time
history of the state in the coordinates $z_{t}$ when $u_{t}^{track}$ is
applied. Observe that the first two components (blue and red lines) display,
as expected, linear and deterministic behaviours, while the internal variable
$z_{3}$ has noisy dynamics. Because of the properties of the zero dynamics,
$z_{3}$ stays bounded under $u_{t}^{track}$. Moreover, the component $z_{1}$
(blue line), which by the definition of normal form is the output $y_{t}$ of
the system, asymptotically converges to the reference signal $y_{R}$
(purple/dashed line).

Figure 1: Time history of the state $z_{t}$ under the control $u_{t}^{track}$,
achieving idealistic output tracking.

We now turn our attention to practically realisable controllers. To this end,
we first discuss the numerical implementation of the simulations in the cases
where stochastic compensations are performed with a period $\varepsilon$. The
continuous-time dynamics was integrated, as usual, using the Euler-Maruyama
numerical scheme with fixed period $\Delta t=10^{-6}$. The stochastic
compensations were performed with a period $\varepsilon$, which must
necessarily satisfy $\varepsilon>\Delta t$. We performed simulations with
values of $\varepsilon$ of $10^{-3}$, $10^{-4}$ and $10^{-5}$, in order to
illustrate the limit behaviour of the solutions as $\varepsilon$ decreases.
Note that it could be possible to select smaller $\varepsilon$ as long as
$\Delta t$ is decreased accordingly.

We illustrate in Figure 2 that when the control law with compensations is
employed and $\varepsilon$ is decreased, the trajectory of the state under the
idealistic control is retrieved. In Figure 2 we consider again the output
tracking setting and display the time history of the component $z_{r}=z_{2}$
under the controls $u_{t}^{zn,track}$ (blue line), $\hat{u}_{t}^{track}$, with
varying $\varepsilon$ ($10^{-3}$ (red line), $10^{-4}$ (yellow line),
$10^{-5}$ (purple line)), and $u_{t}^{track}$ (green line). Observe that,
since the control $\hat{u}_{t}^{track}$ improves the noise compensations as
$\varepsilon$ decreases, the trajectories of $z_{2}$ under the compensated
control tend to the idealistic trajectory.

Figure 2: Time history of the component $z_{2}$ under the controls
$u_{t}^{zn,track}$ (blue line), $\hat{u}_{t}^{track}$ with
$\varepsilon=10^{-3}$ (red line), $\varepsilon=10^{-4}$ (yellow line),
$\varepsilon=10^{-5}$ (purple line), and $u_{t}^{track}$.

To conclude, Figure 3 shows a comparison between the time histories of the
tracking error $z_{1}-y_{R}=y_{t}-y_{R}$ when the controls applied are,
respectively, $u_{t}^{zn,track}$ (blue line) or $\hat{u}_{t}^{track}$ with
decreasing values of $\varepsilon$, namely $10^{-3}$ (red line), $10^{-4}$
(yellow line), $10^{-5}$ (purple line). While the hybrid controller produces,
for any of the values of $\varepsilon$, strikingly better asymptotic
performances than the zero-noise control, observe that the tracking error is
made smaller and smaller as $\varepsilon$ is decreased. This is in line with
the fact that the component $z_{1}=y_{t}$ under $\hat{u}_{t}^{track}$ tends,
as $\varepsilon$ approaches zero, to $z_{1}$ under the control
$u_{t}^{track}$, which in turn asymptotically approaches $y_{R}$.

Figure 3: Time history of the tracking error $y_{t}-y_{R}$ under the controls
$u_{t}^{zn,track}$ (blue line) and $\hat{u}_{t}^{track}$ with
$\varepsilon=10^{-3}$ (red line), $\varepsilon=10^{-4}$ (yellow line) and
$\varepsilon=10^{-5}$ (purple line).

##  VIII Conclusions

In this paper we have addressed the path-wise control of nonlinear stochastic
systems. First we have introduced a notion of stochastic relative degree and
normal form. Then, leveraging these, we have presented a feedback linearising
controller. We have observed that this controller is not causal, hence
referred to as idealistic, because it requires a feedback of the noise. To
overcome this limitation, we have introduced a hybrid control architecture
that estimates the Brownian motion from measurements of the state and uses
these estimates to periodically compensate, in approximate way, for the noise.
We have proved that the performances of the idealistic control law are
retrieved when compensations at a frequency tending to infinity are performed.
Finally, we have solved the problem of asymptotic output tracking, both in the
idealistic and practical framework, and we have provided an illustrative
example.

## Appendix

###  VIII-A Technical lemmas and proof of Proposition 1

######  Lemma 4.

Let $x\in U\subset\mathbb{R}^{n}$ and $k\in\\{0,...,r-2\\}$. Then
$\text{$\sideset{{}^{m}_{g}}{{}_{l}}{\operatorname{\mathcal{A}}}$}\text{$\sideset{{}^{l}}{{}_{f}^{k}}{\operatorname{\mathcal{S}}}$}h(\xi_{t},x)=0$
for all $\xi_{t}\in\mathbb{R}$ if and only if
$\operatorname{\mathcal{L}}_{g}\text{$\sideset{{}^{l}}{{}_{f}^{k}}{\operatorname{\mathcal{S}}}$}h(x)+\text{$\sideset{{}^{m}}{{}_{l}}{\operatorname{\mathcal{G}}}$}\text{$\sideset{{}^{l}}{{}_{f}^{k}}{\operatorname{\mathcal{S}}}$}h(x)=0$
and
$\operatorname{\mathcal{L}}_{m}\text{$\sideset{{}^{l}}{{}_{f}^{k}}{\operatorname{\mathcal{S}}}$}h(x)=0$.

###### Proof.

Without loss of generality assume $k=0$. Then the expression of
$\text{$\sideset{{}^{m}_{g}}{{}_{l}}{\operatorname{\mathcal{A}}}$}\text{$\sideset{{}^{l}}{{}_{f}^{k}}{\operatorname{\mathcal{S}}}$}h(\xi_{t},x)$
for $k=0$ is given in (7). The sufficiency is trivial. As for the necessity,
observe that if $\operatorname{\mathcal{L}}_{m}h(x)\neq 0$ then the randomness
induced by the white noise implies that
$\text{$\sideset{{}^{m}_{g}}{{}_{l}}{\operatorname{\mathcal{A}}}$}h(\xi_{t},x)\neq
0$ almost surely. Therefore $\operatorname{\mathcal{L}}_{m}h(x)=0$ is a
necessary condition for
$\text{$\sideset{{}^{m}_{g}}{{}_{l}}{\operatorname{\mathcal{A}}}$}h(\xi_{t},x)$
to be zero for all $\xi_{t}\in\mathbb{R}$. As a consequence, also
$\operatorname{\mathcal{L}}_{g}h(x)+\text{$\sideset{{}^{m}}{{}_{l}}{\operatorname{\mathcal{G}}}$}h(x)=0$
is a necessary condition for
$\text{$\sideset{{}^{m}_{g}}{{}_{l}}{\operatorname{\mathcal{A}}}$}h(\xi_{t},x)$
to be zero for all $\xi_{t}\in\mathbb{R}$. ∎

######  Lemma 5.

Let $\bar{x}\in\mathbb{R}^{n}$. Then
$\text{$\sideset{{}^{m}_{g}}{{}_{l}}{\operatorname{\mathcal{A}}}$}\text{$\sideset{{}^{l}}{{}_{f}^{r-1}}{\operatorname{\mathcal{S}}}$}h(\xi_{t},\bar{x})\neq
0$ almost surely if and only if
$\operatorname{\mathcal{L}}_{g}\text{$\sideset{{}^{l}}{{}_{f}^{r-1}}{\operatorname{\mathcal{S}}}$}h(\xi_{t},\bar{x})+\text{$\sideset{{}^{m}}{{}_{l}}{\operatorname{\mathcal{G}}}$}\text{$\sideset{{}^{l}}{{}_{f}^{r-1}}{\operatorname{\mathcal{S}}}$}h(\xi_{t},\bar{x})\neq
0$ or
$\operatorname{\mathcal{L}}_{m}\text{$\sideset{{}^{l}}{{}_{f}^{r-1}}{\operatorname{\mathcal{S}}}$}h(\xi_{t},\bar{x})\neq
0$.

###### Proof.

Observe that
$\text{$\sideset{{}^{m}_{g}}{{}_{l}}{\operatorname{\mathcal{A}}}$}\text{$\sideset{{}^{l}}{{}_{f}^{r-1}}{\operatorname{\mathcal{S}}}$}h(\xi_{t},\bar{x})=\operatorname{\mathcal{L}}_{g}\text{$\sideset{{}^{l}}{{}_{f}^{r-1}}{\operatorname{\mathcal{S}}}$}h(\bar{x})+\operatorname{\mathcal{L}}_{m}\text{$\sideset{{}^{l}}{{}_{f}^{r-1}}{\operatorname{\mathcal{S}}}$}h(\bar{x})\xi_{t}+\text{$\sideset{{}^{m}}{{}_{l}}{\operatorname{\mathcal{G}}}$}\text{$\sideset{{}^{l}}{{}_{f}^{r-1}}{\operatorname{\mathcal{S}}}$}h(\bar{x})$.
Then the necessity is trivial. As for the sufficiency, observe that if
$\operatorname{\mathcal{L}}_{g}\text{$\sideset{{}^{l}}{{}_{f}^{r-1}}{\operatorname{\mathcal{S}}}$}h(\bar{x})+\text{$\sideset{{}^{m}}{{}_{l}}{\operatorname{\mathcal{G}}}$}\text{$\sideset{{}^{l}}{{}_{f}^{r-1}}{\operatorname{\mathcal{S}}}$}h(\bar{x})\neq
0$ or
$\operatorname{\mathcal{L}}_{m}\text{$\sideset{{}^{l}}{{}_{f}^{r-1}}{\operatorname{\mathcal{S}}}$}h(\bar{x})\neq
0$ then the randomness induced by the white noise implies that
$\operatorname{\mathcal{L}}_{g}\text{$\sideset{{}^{l}}{{}_{f}^{r-1}}{\operatorname{\mathcal{S}}}$}h(\bar{x})+\text{$\sideset{{}^{m}}{{}_{l}}{\operatorname{\mathcal{G}}}$}\text{$\sideset{{}^{l}}{{}_{f}^{r-1}}{\operatorname{\mathcal{S}}}$}h(\bar{x})\neq-\operatorname{\mathcal{L}}_{m}\text{$\sideset{{}^{l}}{{}_{f}^{r-1}}{\operatorname{\mathcal{S}}}$}h(\bar{x})\xi_{t}$
almost surely, hence the claim follows. ∎

Proof of Proposition 1. We prove the first part of the proposition by
induction. Equation (9) trivially holds for $k=0$. Now, suppose that it holds
for any $k\in\\{1,\dots,r-2\\}$. Then

$y_{t}^{(k+1)}=\text{$\sideset{{}^{l}}{{}_{f}^{k+1}}{\operatorname{\mathcal{S}}}$}h(\xi_{t},x_{t})+\text{$\sideset{{}^{m}_{g}}{{}_{l}}{\operatorname{\mathcal{A}}}$}\text{$\sideset{{}^{l}}{{}_{f}^{k}}{\operatorname{\mathcal{S}}}$}h(\xi_{t},x_{t})u+\\\
\frac{1}{2}\text{$\sideset{{}^{m}}{{}_{m}}{\operatorname{\mathcal{G}}}$}\text{$\sideset{{}^{l}}{{}_{f}^{k}}{\operatorname{\mathcal{S}}}$}h(\xi_{t},x_{t})u^{2}.$
(103)

The first term on the right-hand side expands to

$\text{$\sideset{{}^{l}}{{}_{f}^{k+1}}{\operatorname{\mathcal{S}}}$}h(\xi_{t},x_{t})=\operatorname{\mathcal{L}}_{f}\text{$\sideset{{}^{l}}{{}_{f}^{k}}{\operatorname{\mathcal{S}}}$}h(x_{t})+\operatorname{\mathcal{L}}_{l}\text{$\sideset{{}^{l}}{{}_{f}^{k}}{\operatorname{\mathcal{S}}}$}h(x_{t})\xi_{t}+\\\
\frac{1}{2}\text{$\sideset{{}^{l}}{{}_{l}}{\operatorname{\mathcal{G}}}$}\text{$\sideset{{}^{l}}{{}_{f}^{k}}{\operatorname{\mathcal{S}}}$}h(x_{t}),$
(104)

which, by the first equality in condition (ND), reduces to

$\text{$\sideset{{}^{l}}{{}_{f}^{k+1}}{\operatorname{\mathcal{S}}}$}h(x_{t})=\operatorname{\mathcal{L}}_{f}\text{$\sideset{{}^{l}}{{}_{f}^{k}}{\operatorname{\mathcal{S}}}$}h(x_{t})+\frac{1}{2}\text{$\sideset{{}^{l}}{{}_{l}}{\operatorname{\mathcal{G}}}$}\text{$\sideset{{}^{l}}{{}_{f}^{k}}{\operatorname{\mathcal{S}}}$}h(x_{t}),$
(105)

_i.e._
$\text{$\sideset{{}^{l}}{{}_{f}^{k+1}}{\operatorname{\mathcal{S}}}$}h(x_{t})$
is independent of $\xi_{t}$ for all $k\in\\{0,\dots,r-2\\}$. Going back to
(103), by Lemma 4, the first two equalities in condition (CD) are equivalent
to
$\text{$\sideset{{}^{m}_{g}}{{}_{l}}{\operatorname{\mathcal{A}}}$}\text{$\sideset{{}^{l}}{{}_{f}^{k}}{\operatorname{\mathcal{S}}}$}h(\xi_{t},x_{t})=0$.
Moreover, by the last equality in condition (CD),
$\text{$\sideset{{}^{m}}{{}_{m}}{\operatorname{\mathcal{G}}}$}\text{$\sideset{{}^{l}}{{}_{f}^{k}}{\operatorname{\mathcal{S}}}$}h(\xi_{t},x_{t})=0$.
In conclusion,
$y_{t}^{(k+1)}=\text{$\sideset{{}^{l}}{{}_{f}^{k+1}}{\operatorname{\mathcal{S}}}$}h(x_{t})$
for all $k\in\\{0,\dots,r-2\\}$, which proves that conditions (ND) and (RD)
imply (9).

To prove (10), observe that by Lemma 5, the first two inequalities in
condition (RD) are equivalent to
$\text{$\sideset{{}^{m}_{g}}{{}_{l}}{\operatorname{\mathcal{A}}}$}\text{$\sideset{{}^{l}}{{}_{f}^{r-1}}{\operatorname{\mathcal{S}}}$}h(\xi_{t},\bar{x})\neq
0$. Therefore condition (RD) implies that either
$\text{$\sideset{{}^{m}_{g}}{{}_{l}}{\operatorname{\mathcal{A}}}$}\text{$\sideset{{}^{l}}{{}_{f}^{r-1}}{\operatorname{\mathcal{S}}}$}h(\xi_{t},\bar{x})\neq
0$ or
$\text{$\sideset{{}^{m}}{{}_{m}}{\operatorname{\mathcal{G}}}$}\text{$\sideset{{}^{l}}{{}_{f}^{r-1}}{\operatorname{\mathcal{S}}}$}h(\bar{x})\neq
0$, thus making the control $u(\bar{t})$ appear in the expression of the
$r$-th derivative of $y_{t}$. $\square$

###  VIII-B Almost sure stability of perturbed stochastic systems

Consider the stochastic time-varying system

$\dot{x}_{t}=f_{d}(x_{t},t)+f_{s}(x_{t},t)\xi_{t}=\tilde{f}(\xi_{t},x_{t},t),$
(106)

for which we introduce the following Lipschitz assumption [35].

######  Assumption 3.

$f_{d}$ and $f_{s}$ are locally Lipschitz continuous for all $t\geq 0$.

We now introduce a concept of almost sure total stability for nonlinear time-
varying stochastic system. Consider the following extension of the definition
of strict Lyapunov function for stationary stochastic systems given in [36] to
the case of time-varying stochastic systems.

######  Definition 8.

Consider the autonomous system (106). A smooth function
$V:U\times\mathbb{R}_{\geq 0}\rightarrow\mathbb{R}$, where $U$ is a bounded
domain, is said to be a _strict Lyapunov function_ for system (106) if

  * •

$W_{1}(x)\leq V(x,t)\leq W_{2}(x)$ for all $x\in U$, $t\geq 0$, $V(0,\cdot)=0$
and $W_{1}$ and $W_{2}$ are continuous positive definite functions on $U$;

  * •

there exists a positive definite function $W_{3}(x)$ such that $\frac{\partial
V}{\partial
t}(x,t)+\operatorname{\mathcal{L}}_{f_{d}}V(x,t)+\frac{1}{2}\text{$\sideset{{}^{f_{s}}}{{}_{f_{s}}}{\operatorname{\mathcal{G}}}$}V(x,t)<-W_{3}(x)$
and $\operatorname{\mathcal{L}}_{f_{s}}V(x,t)=0$ for all $x\in U$, $t\geq 0$.

It is possible to show that the existence of a strict Lyapunov function for
system (4) is sufficient to conclude the almost sure (uniform) asymptotic
stability of its equilibrium point (as defined, _e.g._ , in [37],[38]).
Namely, the following result is an extension of [36, Theorem 2.5].

######  Lemma 6.

Under Assumption 3, if there exists a strict Lyapunov function for system
(106), then the equilibrium point at the origin of system (106) is almost
surely uniformly asymptotically stable.

The proof follows by replacing in the proof of [36, Theorem 2.6] the time-
invariant infinitesimal generator of $V$ with its time-varying version.

######  Remark 13.

By dropping the condition that $\operatorname{\mathcal{L}}_{f_{s}}V(x,t)=0$ in
Definition 8, the results can be reformulated in the context of the weaker
stability in probability. See [27, Theorem 11.2.8] for an equivalent of Lemma
6 in this context.

Consider now a persistently perturbed version of system (106), namely

$dx_{t}=(f_{d}(x_{t},t)+p_{d}(x_{t},t))dt+(f_{s}(x_{t},t)+p_{s}(x_{t},t))d\mathcal{W}_{t}$
(107)

where $p_{d}$ and $p_{s}$ are Lipschitz. In the following we formalise the
idea that if $p_{d}$ and $p_{s}$ are small enough perturbations, then the
stability properties of (106) are analogous to those of (107). In other words,
we provide an extension to stochastic systems of the concept of total
stability presented, _e.g._ , in [39, Definition 56.1]. To this end, let
$\tilde{p}(\xi_{t},x_{t},t)=p_{d}(x_{t},t)+p_{s}(x_{t},t)\xi_{t}$.

######  Definition 9.

Consider the stochastic systems (106) and the perturbed system

$\displaystyle\dot{x}_{t}$
$\displaystyle=\tilde{f}(\xi_{t},x_{t},t)+\tilde{p}(\xi_{t},x_{t},t)$ (108)
$\displaystyle=f_{d}(x_{t},t)+p_{d}(x_{t},t)\\!+(f_{s}(x_{t},t)+p_{s}(x_{t},t))\xi_{t}.$

Suppose that there exists a solution $x_{p,t}(\bar{t},\bar{x})$ of (108). The
equilibrium $x=0$ of (106) is said to be _totally stable almost surely_ if for
each $\epsilon>0$ there exist positive $\delta_{1}(\epsilon)$,
$\delta_{2}(\epsilon)$ and $\delta_{3}(\epsilon)$ such that
$\|x_{p,t}(\bar{t},\bar{x})\|<\epsilon$ for all $t\geq\bar{t}$ almost surely
provided that $\|\bar{x}\|<\delta_{1}$, $\|p_{d}(x,t)\|<\delta_{2}$ and
$\|p_{s}(x,t)\|<\delta_{3}$ for all $(x,t)$ satisfying $t\geq\bar{t}$ and
$\|x\|\leq\epsilon$.

The following lemma provides some properties that a strict Lyapunov function
must satisfy in order for system (106) to have an almost surely totally stable
equilibrium at zero.

######  Lemma 7.

Consider systems (106) and (108). Suppose that Assumption 3 holds. If there
exists a strict Lyapunov function $V$ for system (106) such that

  * •

$\frac{\partial V}{\partial x_{i}}(x,t)$ and $\frac{\partial^{2}V}{\partial
x_{i}\partial x_{j}}(x,t)$ are bounded for all $x\in U$ and $t\geq 0$,

  * •

$\operatorname{\mathcal{L}}_{p_{s}}V(x,t)=0$ for all $x\in U$ and $t\geq 0$,

then system (106) has an almost surely totally stable equilibrium at $x=0$.

###### Proof.

The proof follows from [39, Theorem 56.3] by noticing that the time
derivatives of $V$ for (106) and (108) differ by the term

$\frac{\partial V}{\partial
x}p_{d}+(f_{s}+p_{s})^{\top}\frac{\partial^{2}V}{\partial
x^{2}}(f_{s}+p_{s}),$ (109)

which, by Assumption 3 and by the hypotheses of this lemma, can be made so
small that the time derivative of $V$ for (108) is negative. The rest of the
proof is analogous to that of [39, Theorem 56.3]. ∎

Finally, the following lemma gives important stability properties of nonlinear
time-varying stochastic systems driven by almost surely stable systems.

######  Lemma 8.

Consider the stochastic system

$\displaystyle\dot{y}_{t}$
$\displaystyle=\tilde{a}(\xi_{t},y_{t},z_{t},t)=\tilde{a}_{d}(y_{t},z_{t},t)+\tilde{a}_{s}(y_{t},z_{t},t)\xi_{t},$
(110) $\displaystyle\dot{z}_{t}$
$\displaystyle=\tilde{b}(\xi_{t},z_{t})=\tilde{b}_{d}(z_{t})+\tilde{b}_{s}(z_{t})\xi_{t},$

and assume the following.

  1. A1)

$(y,z)=(0,0)$ is an equilibrium of (110) and $\tilde{a}_{d}$ and
$\tilde{a}_{s}$ are Lipschitz continuous for all $t\geq 0$ in a neighbourhood
of zero.

  2. A2)

There exists a strict Lyapunov function $V(x,t)$ for the first equation of
system (110) such that $\frac{\partial V}{\partial x_{i}}(x,t)$ and
$\frac{\partial^{2}V}{\partial x_{i}\partial x_{j}}(x,t)$ are bounded for all
$x\in U$ and $t\geq 0$.

  3. A3)

The equilibrium $z=0$ of $\dot{z}_{t}=\tilde{b}(\xi_{t},z_{t})$ is stable
almost surely.

Then the equilibrium $(y,z)=(0,0)$ of (110) is (uniformly) stable almost
surely.

###### Proof.

Set $\tilde{f}(\xi_{t},y_{t},t)=\tilde{a}(\xi_{t},y_{t},0,t)$ and
$\tilde{p}(\xi_{t},y_{t},t)=\tilde{a}(\xi_{t},y_{t},z_{t}(\bar{t},\bar{z}),t)-\tilde{a}(\xi_{t},y_{t},0,t)$,
with $z_{t}(\bar{t},\bar{z})$ solution of
$\dot{z}_{t}=\tilde{b}(\xi_{t},z_{t})$ satisfying $z_{\bar{t}}=\bar{z}$. Then
the first equation in (110) has the form (108). If $\|z_{t}\|<\epsilon_{z}$
almost surely for all $t>\bar{t}$, then, by Assumption A1 there exist
$\mu_{1}>0$ and $\mu_{2}>0$ such that

$\begin{gathered}\|p_{d}(y,t)\|=\|a_{d}(y,z_{t}(\bar{t},\bar{z}),t)-a_{d}(y,0,t)\|<\mu_{1}\epsilon_{z},\\\
\|p_{s}(y,t)\|=\|a_{s}(y,z_{t}(\bar{t},\bar{z}),t)-a_{s}(y,0,t)\|<\mu_{2}\epsilon_{z},\end{gathered}$
(111)

almost surely for all $y$ in a neighbourhood of zero and all $t\geq\bar{t}$.
Observe that by Assumption A2 the first and second conditions of Lemma 7 are
satisfied for the first equation in (110). Therefore, the first equation in
(110) is totally stable at $y=0$. By Definition 9, for all $\epsilon_{y}$,
there exist $\delta_{1}>0$, $\delta_{2}>0$ and $\delta_{3}>0$ such that
$\|\bar{y}\|<\delta_{1}$, $\|p_{d}(y,t)\|<\delta_{2}$ and
$\|p_{s}(y,t)\|<\delta_{3}$, for all $(y,t)$ such that $\|y\|<\epsilon_{y}$
and $t\geq\bar{t}\geq 0$, imply

$\|y_{t}(\bar{t},\bar{y})\|<\epsilon_{y}\quad\text{almost surely for all
$t\geq\bar{t}$}.$ (112)

Finally, by Assumption A3, it is possible to find a $\delta_{z}$ such that
$\|\bar{z}\|<\delta_{z}$ yields
$\|z_{t}\|<\bar{\delta}=\min\\{\delta_{2},\delta_{3}\\}/\max\\{\mu_{1},\mu_{2}\\}$
for all $t\geq\bar{t}$, which, for what stated earlier, makes
$\|p_{d}(y,t)\|<\mu_{1}\bar{\delta}\leq\delta_{2}$ and
$\|p_{s}(y,t)\|<\mu_{2}\bar{\delta}\leq\delta_{3}$ be satisfied almost surely.
Therefore, for a bounded initial condition $(\bar{y},\bar{z})$ in a
neighbourhood of the equilibrium $(0,0)$, the trajectory $(y_{t},z_{t})$ is
bounded almost surely in a neighbourhood of $(0,0)$, which concludes the
proof. ∎

###  VIII-C Almost sure stability under zero-noise control

######  Lemma 9.

Consider the system

$\dot{y}_{t}=\hat{A}y_{t}+\gamma(\xi_{t},y_{t},z_{t})\quad\dot{z}_{t}=\hat{f}(\xi_{t},y_{t},z_{t}),$
(113)

with $\gamma$ and $\hat{f}$ affine in $\xi_{t}$ and suppose that
$\gamma(\xi_{t},0,z)=0$ for $z$ in a neighbourhood of zero and that
$\frac{\partial\gamma}{\partial y}(\xi_{t},0,0)=0.$ If
$\dot{z}_{t}=f(\xi_{t},0,z_{t})$ has an almost surely asymptotically stable
equilibrium at $z=0$ and the eigenvalues of $A$ all have negative real part,
then system (113) has an almost surely asymptotically stable equilibrium at
$(y,z)=(0,0)$.

###### Proof.

Consider the stochastic subsystem $\dot{z}_{t}=f(\xi_{t},0,z_{t})$. Since this
has an almost surely asymptotically stable equilibrium at 0 by assumption, by
[40] it is possible to decompose the system in the Oseledec subspaces $E_{c}$
and $E_{s}$ relative to the zero and negative Lyapunov exponents of the
linearised dynamics respectively. Let $z_{t}^{c}\in E_{c}$ be the projection
of $z_{t}$ onto $E_{c}$ and $z_{t}^{s}\in E_{s}$ the projection of $z_{t}$
onto $E_{s}$; then for some $F_{c}$, $F_{s}$, $g_{c}$ and $g_{s}$

$\displaystyle\dot{z}_{t}^{c}$
$\displaystyle=F_{c}(\xi_{t})z_{t}^{c}+g_{c}(\xi_{t},0,z_{t}^{c},z_{t}^{s}),$
(114) $\displaystyle\dot{z}_{t}^{s}$
$\displaystyle=F_{s}(\xi_{t})z_{t}^{s}+g_{s}(\xi_{t},0,z_{t}^{c},z_{t}^{s}),$

where the Lyapunov exponents associated to $F_{c}$ are zero and those
associated to $F_{s}$ are negative. Let $z_{t}^{s}=\pi_{1}(\xi_{t},z_{t}^{c})$
be a stochastic center manifold (see [40]) at zero for (114). Then, by the
reduction principle ([40, Theorem 7.3]), since the subsystem
$\dot{z}_{t}=f(\xi_{t},0,z_{t})$ has an almost surely asymptotically stable
equilibrium at $z=0$ by assumption, then the equilibrium at the origin of

$\dot{z}_{t}^{c}=F_{c}(\xi_{t})z_{t}^{c}+g_{c}(\xi_{t},0,z_{t}^{c},\pi_{1}(\xi_{t},z_{t}^{c}))$
(115)

is asymptotically stable almost surely as well. Consider now the full system
(113), for which a center manifold at zero is described by the pair

$z_{t}^{s}=\chi_{1}(\xi_{t},z_{t}^{c}),\quad
y_{t}=\chi_{2}(\xi_{t},z_{t}^{c}).$ (116)

By the reduction principle, the dynamics (113) has an almost surely
asymptotically stable equilibrium at $(0,0)$ if the reduced system

$\dot{z}_{t}^{c}=F_{c}(\xi_{t})z_{t}^{c}+g_{c}(\xi_{t},\chi_{2}(\xi_{t},z_{t}^{c}),z_{t}^{c},\chi_{1}(\xi_{t},z_{t}^{c}))$
(117)

has an almost surely asymptotically stable equilibrium at 0. It is easy to see
that $\chi_{2}(\xi_{t},z_{t}^{c})=0$, because, by the properties of $\gamma$,
the first approximation of the dynamics of $y_{t}$ reduces to
$\dot{y}_{t}=Ay_{t}$, which is asymptotically stable by assumption. Then,
since $\chi_{1}(\xi_{t},z_{t}^{c})=\pi_{1}(\xi_{t},z_{t}^{c})$, (117) reduces
to (115), which has been proved to have an almost surely asymptotically stable
equilibrium at zero. Hence system (113) has an almost surely asymptotically
stable equilibrium at $(y,z)=(0,0)$. ∎

Consider system (38) and the control

$u_{t}^{zn,stab}=-\frac{c_{d}(\zeta_{t},\eta_{t})-v(\zeta_{t})}{b(\zeta_{t},\eta_{t})},$
(118)

with $c_{d}$ (and $c_{s}$) as defined in Section V-B and
$v(\zeta_{t})=-d_{r-1}\zeta_{r}-d_{r-2}\zeta_{r-1}-\dots-d_{0}\zeta_{1}$, with
$d_{i}\in\mathbb{R}$ for $i=1,\dots,r-1$ such that the polynomial (82) has
roots with negative real part. Then the following holds.

######  Proposition 4.

Consider system (38) and suppose that the equilibrium at $\eta=0$ of the zero
dynamics of system (38) is locally asymptotically stable almost surely.
Additionally, suppose that $c_{s}(0,\eta)\equiv 0$ for $\eta$ in a
neighbourhood of zero and that $\frac{\partial c_{s}}{\partial\zeta}(0,0)=0$.
Then the control law $u_{t}^{zn,stab}$ renders the equilibrium
$(\zeta,\eta)=(0,0)$ asymptotically stable almost surely.

###### Proof.

Observe that, if $u_{t}=u_{t}^{zn,stab}$ in (38) then, the closed-loop
dynamics is given by

$\dot{\zeta}_{t}=A\zeta_{t}+Bc_{s}(\zeta_{t},\eta_{t})\xi_{t},\quad\dot{\eta}_{t}=p(\xi_{t},\zeta_{t},\eta_{t})$
(119)

where $A$ is given by (83), which has characteristic polynomial (82). This
system is in the form (113) with
$\gamma(\xi_{t},\zeta_{t},\eta_{t})=Bc_{s}(\zeta_{t},\eta_{t})\xi_{t}$ and, by
the assumptions on $c_{s}$, $\gamma$ satisfies the hypotheses of Lemma 9.
Therefore, the equilibrium at the origin is asymptotically stable almost
surely. ∎

######  Remark 14.

When studying asymptotic stabilisation, the convergence of the states to the
equilibrium at the origin contradicts the persistence of excitation condition
of Assumption 2. However, following from the discussion in Remark 12, if the
hypotheses of Proposition 4 are satisfied, one might perform practical
asymptotic stabilisation using a control
$\hat{u}_{t}^{stab}=u_{t}^{zn,stab}+u_{t}^{s}$ until the states are in an
arbitrarily small neighbourhood of zero, and then switch to just the zero-
noise control $u_{t}^{zn,stab}$. Doing this has the advantage of obtaining
state trajectories that are closer, as $\varepsilon$ goes to zero, to the
idealistic ones when the system is away from the equilibrium, thus ensuring a
more predictable behaviour of the system.

## References

  * [1] A. Isidori, A. Krener, C. Gori-Giorgi, and S. Monaco, “Nonlinear decoupling via feedback: A differential geometric approach,” _IEEE Transactions on Automatic Control_ , vol. 26, no. 2, pp. 331–345, April 1981.
  * [2] M. Zeitz, “Controllability canonical (phase-variable) form for non-linear time-variable systems,” _International Journal of Control_ , vol. 37, no. 6, pp. 1449–1457, 1983.
  * [3] D. Bestle and M. Zeitz, “Canonical form observer design for non-linear time-variable systems,” _International Journal of Control_ , vol. 38, no. 2, pp. 419–431, 1983.
  * [4] A. J. Krener, “Normal forms for linear and nonlinear systems,” _Contemporary Mathematics_ , vol. 68, pp. 157–189, 1987.
  * [5] R. Brockett, “Feedback invariants for nonlinear systems,” _IFAC Proceedings Volumes_ , vol. 11, no. 1, pp. 1115–1120, 1978.
  * [6] B. Jakubczyk and W. Respondek, “On linearization of control systems,” _Bulletin de l’Académie Polonaise des Sciences. Série des sciences mathématiques_ , vol. 28, pp. 517–522, 1980.
  * [7] R. Su, “On the linear equivalents of nonlinear systems,” _Systems & Control Letters_, vol. 2, no. 1, pp. 48–52, 1982.
  * [8] L. Hunt, , and G. Meyer, “Global transformations of nonlinear systems,” _IEEE Transactions on Automatic Control_ , vol. 28, no. 1, pp. 24–31, January 1983.
  * [9] C. I. Byrnes and A. Isidori, “A frequency domain philosophy for nonlinear systems, with applications to stabilization and to adaptive control,” in _The 23rd IEEE Conference on Decision and Control_ , Dec 1984, pp. 1569–1573.
  * [10] ——, “Local stabilization of minimum-phase nonlinear systems,” _Systems & Control Letters_, vol. 11, no. 1, pp. 9–17, 1988.
  * [11] T. Damm, _Rational Matrix Equations in Stochastic Control_ , ser. Lecture Notes in Control and Information Sciences. Springer, 2004, no. 297.
  * [12] D. Hinrichsen and A. J. Pritchard, “Stochastic $H^{\infty}$,” _SIAM Journal on Control and Optimization_ , vol. 36, no. 5, pp. 1504–1538, 1998.
  * [13] E. Gershon, “Robust reduced-order $H_{\infty}$ output-feedback control of retarded stochastic linear systems,” _IEEE Transactions on Automatic Control_ , vol. 58, no. 11, pp. 2898–2904, 2013.
  * [14] H. Hua, J. Cao, G. Yang, and G. Ren, “Voltage control for uncertain stochastic nonlinear system with application to energy internet: Non-fragile robust $H_{\infty}$ approach,” _Journal of Mathematical Analysis and Applications_ , vol. 463, no. 1, pp. 93 – 110, 2018.
  * [15] J. Yong and X. Y. Zhou, _Stochastic Controls: Hamiltonian Systems and HJB Equations_ , ser. Stochastic Modelling and Applied Probability. Springer New York, 1999.
  * [16] B. Øksendal, _Stochastic Differential Equations (Sixth Edition)_. Springer-Verlag, 2003.
  * [17] L. Arnold and P. Imkeller, “Normal forms for stochastic differential equations,” _Probability Theory and Related Fields_ , vol. 110, no. 4, pp. 559–588, May 1998.
  * [18] L. Arnold, _Random Dynamical Systems_ , ser. Springer Monographs in Mathematics. Springer-Verlag Berlin Heidelberg, 2003.
  * [19] G. Gaeta and N. Rodríguez Quintero, “Lie-point symmetries and stochastic differential equations,” _Journal of Physics A: Mathematical and General_ , vol. 32, no. 48, pp. 8485–8505, 1999.
  * [20] T. Lahdhiri and A. Alouani, “An introduction to the theory of exact stochastic feedback linearization for nonlinear stochastic systems,” _IFAC Proceedings Volumes_ , vol. 29, no. 1, pp. 3904–3909, 1996, 13th World Congress of IFAC, 1996, San Francisco USA, 30 June - 5 July.
  * [21] Z. Pan, “Differential geometric condition for feedback complete linearization of stochastic nonlinear system,” _Automatica_ , vol. 37, no. 1, pp. 145–149, 2001.
  * [22] ——, “Canonical forms for stochastic nonlinear systems,” _Automatica_ , vol. 38, no. 7, pp. 1163–1170, 2002.
  * [23] Z. Pan and T. Basar, “Backstepping controller design for nonlinear stochastic systems under a risk-sensitive cost criterion,” _SIAM Journal on Control and Optimization_ , vol. 37, no. 3, pp. 957–995, 1999.
  * [24] A. Mellone and G. Scarciotti, “Normal Form and Exact Feedback Linearisation of Nonlinear Stochastic Systems: the Ideal Case,” in _2019 IEEE 58th Conference on Decision and Control (CDC)_ , 2019, pp. 3503–3508.
  * [25] A. Mellone and G. Scarciotti, “The Zero Dynamics of Nonlinear Stochastic Systems: Stabilization and Output Tracking in the Ideal Case,” in _21st IFAC World Congress (IFAC 2020)_ , July 2020, pp. 5053–5058.
  * [26] ——, “Approximate Feedback Linearisation and Stabilisation of Nonlinear Stochastic Systems,” in _21st IFAC World Congress (IFAC 2020)_ , July 2020, pp. 5059–5064.
  * [27] L. Arnold, _Stochastic Differential Equations_ , ser. A Wiley-Interscience publication. Wiley, 1974\.
  * [28] L. Hörmander, _The Analysis of Linear Partial Differential Operators I_ , ser. Classics in Mathematics. Springer-Verlag Berlin Heidelberg, 1983.
  * [29] J. J. Duistermaat and J. A. C. Kolk, _Distributions_ , ser. Cornerstones. Birkäuser Basel, 1995\.
  * [30] A. Isidori, _Nonlinear Control Systems_ , ser. Communications and Control Engineering. Springer-Verlag London, 1995\.
  * [31] A. Mellone and G. Scarciotti, “A Note on the Itô and Stratonovich Stochastic Relative Degree and Normal Form,” in _2020 59th IEEE Conference on Decision and Control (CDC)_ , 2020, pp. 4306–4311.
  * [32] E. Wong and M. Zakai, “On the convergence of ordinary integrals to stochastic integrals,” _The Annals of Mathematical Statistics_ , vol. 36, no. 5, pp. 1560–1564, 10 1965.
  * [33] A. Mellone and G. Scarciotti, “Output regulation of linear stochastic systems,” _IEEE Transactions on Automatic Control_ , vol. 67, no. 4, pp. 1728–1743, April 2022.
  * [34] T. Gard, _Introduction to Stochastic Differential Equations_ , ser. Monographs and textbooks in pure and applied mathematics. M. Dekker, 1988.
  * [35] H. K. Khalil, _Nonlinear Systems_ , ser. Pearson Education. Prentice Hall, 2002.
  * [36] M. Bardi and A. Cesaroni, “Almost Sure Stabilizability of Controlled Degenerate Diffusions,” _SIAM Journal on Control and Optimization_ , vol. 44, no. 1, pp. 75–98, 2005.
  * [37] F. Kozin, “On almost sure stability of linear systems with random coefficients,” _Journal of Mathematics and Physics_ , vol. 42, no. 1-4, pp. 59–67, 1963.
  * [38] R. Z. Khas’minskii, “Necessary and Sufficient Conditions for the Asymptotic Stability of Linear Stochastic Systems,” _Theory of Probability & Its Applications_, vol. 12, no. 1, pp. 144–147, 1967.
  * [39] W. Hahn and A. P. Baartz, _Stability of Motion_ , ser. Grundlehren der mathematischen Wissenschaften. Springer Verlag, 1967.
  * [40] P. Boxler, “A stochastic version of center manifold theory,” _Probability Theory and Related Fields_ , vol. 83, no. 4, pp. 509–545, Dec 1989.

|  Alberto Mellone was born in Galatina (Lecce), Italy, in 1993\. He received
the B.S. degree in information engineering from the University of Salento,
Lecce, Italy in 2015 and the M.S. degree in robotics and automation
engineering from the University of Pisa, Pisa, Italy in 2018. In 2018 he
joined the Control and Power Group, Imperial College London, London, UK, where
he he pursued a Ph.D. looking into analysis and control of stochastic systems.
He graduated in 2022 with a thesis on path-wise output regulation and
nonlinear control of stochastic systems.
---|---
|  Giordano Scarciotti (Senior Member, IEEE) was born in Frascati (Rome),
Italy, in 1988. He received his B.Sc. and M.Sc. degrees in Automation
Engineering from the University of Rome “Tor Vergata”, Italy, in 2010 and
2012, respectively. In 2012 he joined the Control and Power Group, Imperial
College London, UK, where he obtained a Ph.D. degree in 2016 with a thesis on
approximation, analysis and control of large-scale systems. He also received
an M.Sc. in Applied Mathematics from Imperial College in 2020. He is currently
a Senior Lecturer in the Control and Power Group. His current research
interests are focused on analysis and control of uncertain systems, model
reduction and optimal control. He was a visiting scholar at New York
University in 2015 and at University of California Santa Barbara in 2016 and
he is currently a Visiting Fellow of Shanghai University. He is the recipient
of an Imperial College Junior Research Fellowship (2016), of the IET Control &
Automation PhD Award (2016), the Eryl Cadwaladr Davies Prize (2017) and an
ItalyMadeMe award (2017). He is a member of the IEEE CSS Conference Editorial
Board, of the IFAC and IEEE CSS Technical Committees on Nonlinear Control
Systems and has served in the International Programme Committee of multiple
conferences. He is the National Organising Committee Chair for the EUCA
European Control Conference 2022 and the Invited Session Chair for IFAC
Symposium on Nonlinear Control Systems 2022.
---|---