# On Consistency of Signatures Using Lasso ††thanks: Ruixun Zhang and Chaoyi
Zhao are corresponding authors. The authors are ordered alphabetically.

Xin Guo
UC Berkeley
<EMAIL_ADDRESS>
&Ruixun Zhang
Peking University
<EMAIL_ADDRESS>
&Chaoyi Zhao
Peking University
<EMAIL_ADDRESS>

###### Abstract

Signature transforms are iterated path integrals of continuous and discrete-
time time series data, and their universal nonlinearity linearizes the problem
of feature selection. This paper revisits the consistency issue of Lasso
regression for the signature transform, both theoretically and numerically.
Our study shows that, for processes and time series that are closer to
Brownian motion or random walk with weaker inter-dimensional correlations, the
Lasso regression is more consistent for their signatures defined by Itô
integrals; for mean reverting processes and time series, their signatures
defined by Stratonovich integrals have more consistency in the Lasso
regression. Our findings highlight the importance of choosing appropriate
definitions of signatures and stochastic models in statistical inference and
machine learning.

##  1 Introduction

##### Signature transform.

Originally introduced and studied in algebraic topology [4, 5], the signature
transform, sometimes referred to as the path signature or simply signature,
has been adopted and further developed in rough path theory [13, 31]. Given
any continuous or discrete time series, the signature transform produces a
vector of real-valued features that extract rich and relevant information [27,
32]. It has been proven an attractive and powerful tool for feature generation
and pattern recognition with state-of-the-art performance in a wide range of
domains, including handwriting recognition [38, 40, 41], action recognition
[25, 39, 42], medical prediction [21, 33, 34], and finance [1, 15, 18, 28,
30]. Comprehensive reviews of successful and potential applications of the
signature transform in machine learning can be found in [6] and [27].

Most of the empirical success and theoretical studies of the signature
transform are built upon its most striking universal nonlinearity property. It
states that every continuous function of the time series may be approximated
arbitrarily well by a linear function of its signature (see Section 2.1 for
details). This nonlinearity property makes the signature a computationally
efficient tool; it also cements the prominent role of regression analysis for
time series data thanks to the linearization of the feature space by the
signature. When learning nonlinear relationships between variables, utilizing
linear regression models after applying the signature transform can yield
significantly improved out-of-sample prediction performance compared to
modeling without the signature transform [24].

Signatures are iterated path integrals of time series, and there are multiple
definitions of integrals adopted for signatures. Given the successful
application of the signature transform and the rapidly growing literature on
its probabilistic characteristics, it is time to understand and systematically
study the statistical implications of these different forms of signatures on a
given time series data.

##### Consistency of Lasso.

In practice, feature selection methods like the Lasso [35] are commonly used
to identify a sparse set of features from a universe of all signatures [6, 24,
26, 27]. One of the well-documented and extensively studied issues concerning
linear models is the consistency in feature selections by Lasso [2, 37, 43].
Consistency is an important metric for out-of-sample model performance. Given
the different definitions of signatures, the natural starting point is the
consistency issue for Lasso regression models under different signature
transforms.

##### Main results.

This paper studies the consistency issue of Lasso for signature transforms. It
focuses on two definitions of signatures: Itô and Stratonovich. It chooses two
representative classes of Gaussian processes: multi-dimensional Brownian
motion and Ornstein–Uhlenbeck (OU) process, and their respective discrete-time
counterparts, i.e., random walk and autoregressive (AR) process. These
processes have been widely applied in a number of domains [1, 19, 24, 27, 36].

To analyze the consistency of Lasso regressions, correlation structures of
signatures are first studied for these processes. For Brownian motions, its
correlation structure is shown to be block diagonal if its signatures are
defined by Itô integrals (Propositions 1–2), and to have a special odd–even
alternating structure if its signatures are defined by Stratonovich integrals
(Propositions 3–4). In contrast, the OU process exhibits this odd–even
alternating structure by either definition of the integral (Proposition 5).

Based on the correlation structures of signatures, we investigate the
consistency of Lasso regressions for different processes (Propositions 6–8).
For time series and processes that are closer to Brownian motion and with
weaker inter-dimensional correlations, the Lasso regression is more consistent
for their feature selection by Itô signatures; for mean reverting time series
and processes, Stratonovich signatures yield more consistency for the Lasso
regression.

##### Contribution.

Our study takes the first step toward understanding the statistical properties
of the signature transform for regression analysis. It fills one of the gaps
between the theory and the practice of signature transforms in machine
learning. Our work highlights the importance of choosing appropriate signature
transforms and stochastic models for feature selections and for general
statistical analysis.

##  2 The framework

In this section, we present the framework for studying the consistency of
feature selections in Lasso via signature. All proofs are given in Appendix F.

###  2.1 Review of signatures and their properties

Consider a $d$-dimensional continuous-time stochastic process
$\mathbf{X}_{t}=(X_{t}^{1},X_{t}^{2},\dots,X_{t}^{d})^{\top}\in\mathbb{R}^{d}$,
$0\leq t\leq T$, and its signature or signature transform defined as follows:

######  Definition 1 (Signature).

For $k\geq 1$ and $i_{1},\dots,i_{k}\in\\{1,2,\dots,d\\}$, the $k$-th order
signature of the process $\mathbf{X}$ with index $(i_{1},\dots,i_{k})$ from
time 0 to $t$ is defined as

$S(\mathbf{X})_{t}^{i_{1},\dots,i_{k}}=\int_{0<t_{1}<\dots<t_{k}<t}\mathrm{d}X_{t_{1}}^{i_{1}}\cdots\mathrm{d}X_{t_{k}}^{i_{k}},\quad
0\leq t\leq T.$ (1)

In addition, the 0-th order signature of $\mathbf{X}$ from time 0 to $t$ is
defined as $S(\mathbf{X})_{t}^{0}=1$ for any $0\leq t\leq T$.

In other words, the $k$-th order signature of $\mathbf{X}$ given by Equation
(1) is its $k$-fold iterated path integral along the indices
$i_{1},\dots,i_{k}$. For a given order $k$, there are $d^{k}$ choices of
indices $(i_{1},\dots,i_{k})$, therefore the number of all $k$-th order
signatures is $d^{k}$.

The integral in Equation (1) can be specified differently. For example, if
$\mathbf{X}$ is a deterministic process, it can be defined via the
Riemann/Lebesgue integral. If $\mathbf{X}$ is a multi-dimensional Brownian
motion, it is a stochastic integral that can be defined either by the Itô
integral or by the Stratonovich integral. For clarity, we write

$S(\mathbf{X})_{t}^{i_{1},\dots,i_{k},I}=\int_{0<t_{1}<\dots<t_{k}<t}\mathrm{d}X_{t_{1}}^{i_{1}}\cdots\mathrm{d}X_{t_{k}}^{i_{k}}=\int_{0<s<t}S(\mathbf{X})_{s}^{i_{1},\dots,i_{k-1},I}\mathrm{d}X_{s}^{i_{k}}$

when considering the Itô integral, and

$S(\mathbf{X})_{t}^{i_{1},\dots,i_{k},S}=\int_{0<t_{1}<\dots<t_{k}<t}\mathrm{d}X_{t_{1}}^{i_{1}}\circ\cdots\circ\mathrm{d}X_{t_{k}}^{i_{k}}=\int_{0<s<t}S(\mathbf{X})_{s}^{i_{1},\dots,i_{k-1},S}\circ\mathrm{d}X_{s}^{i_{k}}$

for the Stratonovich integral.

Throughout the paper, for ease of exposition, we refer to the signature of
$\mathbf{X}$ as the Itô (the Stratonovich) signature if the integral is
defined in the sense of the Itô (the Stratonovich) integral.

Signatures enjoy several nice probabilistic properties. First, all expected
signatures of a stochastic process together can characterize the distribution
of the process [7, 8]. Second, the signatures uniquely determine the path of
the underlying process [3, 16, 22].

One of the most striking properties of the signature transform is its
universal nonlinearity [12, 20, 23, 24, 27]. It is of particular relevance for
feature selections in machine learning or statistical analysis, where one
needs to find or learn a (nonlinear) function $f$ that maps time series data
$\mathbf{X}$ to a target label $y$. By universal nonlinearity, any such
function can be approximately linearized by the signature of $\mathbf{X}$ in
the following sense: for any $\varepsilon>0$, under some technical conditions,
there exists $K\geq 1$ and a _linear_ function $L$ such that

$\|f(\mathbf{X})-L(\mathrm{Sig}^{K}_{T}(\mathbf{X}))\|\leq\varepsilon,$ (2)

where $\mathrm{Sig}^{K}_{T}(\mathbf{X})$ represents all signatures of
$\mathbf{X}$ from time 0 to $T$ truncated to some order $K$. This universal
nonlinearity lays the foundation for learning the relationship between the
time series $\mathbf{X}$ and a target label $y$ using a linear regression
model. Table 1 summarizes different statements of the universal nonlinearity
of signatures in the literature.111Appendix A provides the precise statement
of universal nonlinearity. The time augmentation is discussed in Section 5 and
Appendix E.

Table 1: Universal nonlinearity in the literature. Path | With time augmentation | Integral | Literature
---|---|---|---
Càdlàg rough path | Yes | Rough | [10]
Continuous semimartingale | Yes | Stratonovich | [9]
Continuous rough path | Yes | Stratonovich | [1, 29]
Continuous rough path | No | Itô/Stratonovich | [24]
Bounded variation path | No | Riemann/Lebesgue | [12, 20, 27]

In the next section, we study feature selections via Lasso regression by
signature transform.

###  2.2 Feature selection using Lasso with signatures

Suppose that one is given $N$ pairs of samples,
$(\mathbf{X}_{1},y_{1}),(\mathbf{X}_{2},y_{2}),\dots,(\mathbf{X}_{N},y_{N})$,
where $\mathbf{X}_{n}=\\{\mathbf{X}_{n,t}\\}_{0\leq t\leq T}$ is the $n$-th
time series, for $n=1,2,\dots,N$. Given a fixed order $K\geq 1$, consider the
following regression model:

$y_{n}=\beta_{0}+\sum_{i_{1}=1}^{d}\beta_{i_{1}}S(\mathbf{X}_{n})^{i_{1}}_{T}+\sum_{i_{1},i_{2}=1}^{d}\beta_{i_{1},i_{2}}S(\mathbf{X}_{n})^{i_{1},i_{2}}_{T}+\dots+\sum_{i_{1},\dots,i_{K}=1}^{d}\beta_{i_{1},\dots,i_{K}}S(\mathbf{X}_{n})^{i_{1},\dots,i_{K}}_{T}+\varepsilon_{n},$
(3)

where $n=1,2,\dots,N$ represents $N$ samples, and
$\\{\varepsilon_{n}\\}_{n=1}^{N}$ are independent and identically distributed
errors. Here the number of predictors, i.e., the signature of various orders,
is $\frac{d^{K+1}-1}{d-1}$, including the 0-th order signature
$S(\mathbf{X})^{0}_{T}=1$, whose coefficient is $\beta_{0}$. It has been
documented that including signatures up to a small order $K$ as predictors in
a linear regression model usually suffices to achieve good performances in
practice [27, 32].

The goal of Lasso is to identify the true predictors/features among all the
predictors included in the linear regression model (3). We use $A^{*}_{k}$ to
represent the set of all signatures of order $k$ with nonzero coefficients in
Equation (3). Given any (nonlinear) function $f$ that one needs to learn, let
us define the set of true predictors222True predictors are predictors with
nonzero coefficients in Equation (3).  $A^{*}$ by

$A^{*}=\bigcup_{k=0}^{K}A_{k}^{*}:=\bigcup_{k=0}^{K}\\{(i_{1},\dots,i_{k}):\beta_{i_{1},\dots,i_{k}}\neq
0\\}.$ (4)

Here, we begin the union with $k=0$ to include the 0-th order signature for
notational convenience.

Given a tuning parameter $\lambda>0$ and $N$ samples, we adopt the following
Lasso estimator to identify the true predictors:

$\displaystyle\hat{{\boldsymbol{\beta}}}^{N}(\lambda)=\arg\min_{\tilde{\boldsymbol{\beta}}}\Bigg{[}\sum_{n=1}^{N}\Bigg{(}y_{n}-$
$\displaystyle\tilde{\beta}_{0}-\sum_{i_{1}=1}^{d}\tilde{\beta}_{i_{1}}\tilde{S}(\mathbf{X}_{n})^{i_{1}}_{T}-\sum_{i_{1},i_{2}=1}^{d}\tilde{\beta}_{i_{1},i_{2}}\tilde{S}(\mathbf{X}_{n})^{i_{1},i_{2}}_{T}-\cdots$
$\displaystyle-\sum_{i_{1},\dots,i_{K}=1}^{d}\tilde{\beta}_{i_{1},\dots,i_{K}}\tilde{S}(\mathbf{X}_{n})^{i_{1},\dots,i_{K}}_{T}\Bigg{)}^{2}+\lambda\left\|\tilde{\boldsymbol{\beta}}\right\|_{1}\Bigg{]},$
(5)

where $\tilde{\boldsymbol{\beta}}$ is the vector containing all coefficients
$\tilde{\beta}_{i_{1},\dots,i_{k}}$, and $\left\|\cdot\right\|_{1}$ denotes
the $l_{1}$-norm. Here, $\tilde{S}(\mathbf{X}_{n})$ represents the standarized
version of ${S}(\mathbf{X}_{n})$ across $N$ samples by the $l_{2}$-norm, i.e.,
for any index $(i_{1},\dots,i_{k})$,

$\tilde{S}(\mathbf{X}_{n})^{i_{1},\dots,i_{k}}_{T}=\frac{{S}(\mathbf{X}_{n})^{i_{1},\dots,i_{k}}_{T}}{\sqrt{\sum_{m=1}^{N}\left[{S}(\mathbf{X}_{m})^{i_{1},\dots,i_{k}}_{T}\right]^{2}}},\quad
n=1,2,\dots,N.$

We perform this standardization for two reasons. First, the Lasso estimator is
sensitive to the magnitudes of the predictors, and standardization helps
prevent the domination of predictors with larger magnitudes in the estimation
process [17]. Second, the magnitudes of the signatures vary as the order of
the signature changes [31], therefore standardization is necessary to ensure
that the coefficients of different orders of signatures are on the same scale
and can be compared directly. Furthermore, the covariance matrix is now
equivalent to the correlation matrix, allowing us to focus on the correlation
structure of the signatures in the subsequent analysis.

###  2.3 Consistency and the irrepresentable condition of Lasso

Our goal is to study the consistency of feature selections via signatures
using the Lasso estimator in Equation (5). We use the concept of _(strong)
sign consistency_ , a custom definition of consistency for Lasso proposed in
[43].

######  Definition 2 (Consistency).

Lasso is (strongly) sign consistent if there exists $\lambda_{N}$, a function
of sample number $N$, such that

$\lim_{N\to+\infty}\mathbb{P}\left(\mathrm{sign}\left(\hat{{\boldsymbol{\beta}}}^{N}(\lambda_{N})\right)=\mathrm{sign}({\boldsymbol{\beta}})\right)=1,$

where $\hat{{\boldsymbol{\beta}}}^{N}(\cdot)$ is the Lasso estimator given by
Equation (5), ${\boldsymbol{\beta}}$ is a vector containing all beta
coefficients of the true model, Equation (3), and the function
$\mathrm{sign}(\cdot)$ maps positive entries to $1$, negative entries to $-1$,
and $0$ to $0$.

In other words, sign consistency requires that a pre-selected $\lambda$ can be
used to achieve consistent feature selection via Lasso.

The following irrepresentable condition is nearly a necessary and sufficient
condition for the Lasso to be sign consistent [43].

######  Definition 3 (Irrepresentable condition).

The feature selection in Equation (3) satisfies the (strong) irrepresentable
condition if there exists a positive constant vector ${\boldsymbol{\eta}}$
such that

$\left|{\Sigma}_{A^{*c},A^{*}}{\Sigma}_{A^{*},A^{*}}^{-1}\mathrm{sign}({\boldsymbol{\beta}}_{A^{*}})\right|\leq\mathbf{1}-{\boldsymbol{\eta}},$

where $A^{*}$ is given by Equation (4) and $A^{*c}$ the complement of $A^{*}$,
${\Sigma}_{A^{*c},A^{*}}$ (${\Sigma}_{A^{*},A^{*}}$) represents the covariance
matrix333In this paper, in line with [43], all covariances and correlation
coefficients are defined to be uncentered. Specifically, for random variables
$X$ and $Y$, we define their covariance as $\mathbb{E}[XY]$, and their
correlation coefficient as
$\mathbb{E}[XY]/\sqrt{\mathbb{E}[X^{2}]\mathbb{E}[Y^{2}]}$. One can easily
extend our results to the centered case. between all predictors in $A^{*c}$
and $A^{*}$ ($A^{*}$ and $A^{*}$), ${\boldsymbol{\beta}}_{A^{*}}$ represents a
vector formed by beta coefficients for all predictors in $A^{*}$, $\mathbf{1}$
is an all-one vector, $|\cdot|$ calculates the absolute values of all entries,
and the inequality “$\leq$” holds element-wise.

This irrepresentable condition uses the _population_ covariance matrix instead
of the _sample_ covariance matrix in [43]. Nevertheless, similar to [43], it
means that the irrelevant predictors in $A^{*c}$ cannot be sufficiently
represented by the true predictors in $A^{*}$, implying weak collinearity
between the predictors.

By the signature transform, predictors in our linear regression model (3) are
correlated and have special correlation structures that differ from earlier
studies [2, 37, 43]. We will show in the following section that their
correlation structures vary with the underlying process $\mathbf{X}$, hence
leading to different consistency performances for different processes.
Moreover, these correlation structures depend on the choice of integrals used
in Equation (1).

##  3 Correlation structure of signatures

To study the consistency of Lasso using signatures, let us investigate the
correlation structure of Itô and Stratonovich signatures for two
representative Gaussian processes with different characteristics: the Brownian
motion and the OU process.

###  3.1 Correlation structure for multi-dimensional Brownian motion

######  Definition 4 (Brownian motion).

$\mathbf{X}$ is a $d$-dimensional Brownian motion if it can be expressed as:

$\mathbf{X}_{t}=(X_{t}^{1},X_{t}^{2},\dots,X_{t}^{d})^{\top}=\Gamma(W_{t}^{1},W_{t}^{2},\dots,W_{t}^{d})^{\top},$
(6)

where $W_{t}^{1},W_{t}^{2},\dots,W_{t}^{d}$ are mutually independent standard
Brownian motions, and $\Gamma$ is a matrix independent of $t$. In particular,
$\mathrm{d}X_{t}^{i}\mathrm{d}X_{t}^{j}=\rho_{ij}\sigma_{i}\sigma_{j}\mathrm{d}t$
with $\rho_{ij}\sigma_{i}\sigma_{j}=(\Gamma\Gamma^{\top})_{ij}$, where
$\sigma_{i}$ is the volatility of $X_{t}^{i}$, and $\rho_{ij}\in[-1,1]$ is the
inter-dimensional correlation between $X_{t}^{i}$ and $X_{t}^{j}$.

Now we study the correlation structure of Itô and Stratonovich signatures
respectively.

####  3.1.1 Itô signatures for Brownian motion

The following proposition gives the moments of Itô signatures of a
$d$-dimensional Brownian motion.

######  Proposition 1.

Let $\mathbf{X}$ be a $d$-dimensional Brownian motion given by Equation (6).
For $m,n=1,2,\dots$ and $m\neq n$, we have:

$\displaystyle\mathbb{E}\left[S(\mathbf{X})_{t}^{i_{1},\dots,i_{n},I}\right]$
$\displaystyle=0,$
$\displaystyle\mathbb{E}\left[S(\mathbf{X})_{t}^{i_{1},\dots,i_{n},I}S(\mathbf{X})_{t}^{j_{1},\dots,j_{m},I}\right]$
$\displaystyle=0,$
$\displaystyle\mathbb{E}\left[S(\mathbf{X})_{t}^{i_{1},\dots,i_{n},I}S(\mathbf{X})_{t}^{j_{1},\dots,j_{n},I}\right]$
$\displaystyle=\frac{t^{n}}{n!}\prod_{k=1}^{n}\rho_{i_{k}j_{k}}\sigma_{i_{k}}\sigma_{j_{k}}.$

With Proposition 1, the following result explicitly characterizes the
correlation structure of Itô signatures for Brownian motions.

######  Proposition 2.

Let $\mathbf{X}$ be a $d$-dimensional Brownian motion given by Equation (6).
If we arrange the signatures in recursive order (see Definition A.1 in
Appendix B), the correlation matrix for Itô signatures of $\mathbf{X}$ with
orders truncated to $K$ is a block diagonal matrix:

$\Sigma^{I}=\mathrm{diag}\\{\Omega_{0},\Omega_{1},\Omega_{2},\dots,\Omega_{K}\\},$
(7)

whose diagonal block $\Omega_{k}$ represents the correlation matrix for all
$k$-th order signatures, which is given by:

$\Omega_{k}=\underbrace{\Omega\otimes\Omega\otimes\cdots\otimes\Omega}_{k},\quad
k=1,2,\dots,K,$ (8)

and $\Omega_{0}=1$, where $\otimes$ represents the Kronecker product, and

$\Omega=\begin{pmatrix}1&\rho_{12}&\cdots&\rho_{1d}\\\
\rho_{12}&1&\cdots&\rho_{2d}\\\ \vdots&\vdots&\ddots&\vdots\\\
\rho_{1d}&\rho_{2d}&\cdots&1\end{pmatrix}.$

Proposition 2 reveals several important facts about Itô signatures for
Brownian motions. First, signatures of different orders are mutually
independent, leading to a block diagonal correlation structure. Second, the
correlation between signatures of the same order has a Kronecker product
structure determined by the inter-correlation ($\rho_{ij}$) between different
dimensions of $\mathbf{X}$.

####  3.1.2 Stratonovich signatures for Brownian motion

The moments and correlation structure for Stratonovich signatures of Brownian
motions are more complicated. We first provide the moments of Stratonovich
signatures.

######  Proposition 3.

Let $\mathbf{X}$ be a $d$-dimensional Brownian motion given by Equation (6).
For $m,n=1,2,\dots$, we have

$\displaystyle\mathbb{E}\left[S(\mathbf{X})_{t}^{i_{1},\dots,i_{2n-1},S}\right]$
$\displaystyle=0,$
$\displaystyle\mathbb{E}\left[S(\mathbf{X})_{t}^{i_{1},\dots,i_{2n},S}\right]$
$\displaystyle=\frac{1}{2^{n}}\frac{t^{n}}{n!}\prod_{k=1}^{n}\rho_{i_{2k-1}i_{2k}}\prod_{k=1}^{2n}\sigma_{i_{k}},$
$\displaystyle\mathbb{E}\left[S(\mathbf{X})_{t}^{i_{1},\dots,i_{2n},S}S(\mathbf{X})_{t}^{j_{1},\dots,j_{2m-1},S}\right]$
$\displaystyle=0,$

and
$\mathbb{E}\left[S(\mathbf{X})_{t}^{i_{1},\dots,i_{2n},S}S(\mathbf{X})_{t}^{j_{1},\dots,j_{2m},S}\right]$
and
$\mathbb{E}\left[S(\mathbf{X})_{t}^{i_{1},\dots,i_{2n-1},S}S(\mathbf{X})_{t}^{j_{1},\dots,j_{2m-1},S}\right]$
can be calculated using formulas provided in Proposition A.1 in Appendix B.

The following result explicitly characterizes the correlation structure of
Stratonovich signatures for Brownian motions.

######  Proposition 4.

Let $\mathbf{X}$ be a $d$-dimensional Brownian motion given by Equation (6).
The correlation matrix for all Stratonovich signatures of $\mathbf{X}$ with
orders truncated to $2K$ has the following odd–even alternating structure:

$\Sigma^{S}=\begin{pmatrix}\Psi_{0,0}&0&\Psi_{0,2}&0&\cdots&0&\Psi_{0,2K}\\\
0&\Psi_{1,1}&0&\Psi_{1,3}&\cdots&\Psi_{1,2K-1}&0\\\
\Psi_{2,0}&0&\Psi_{2,2}&0&\cdots&0&\Psi_{2,2K}\\\
0&\Psi_{3,1}&0&\Psi_{3,3}&\cdots&\Psi_{3,2K-1}&0\\\
\vdots&\vdots&\vdots&\vdots&\ddots&\vdots&\vdots\\\
0&\Psi_{2K-1,1}&0&\Psi_{2K-1,3}&\cdots&\Psi_{2K-1,2K-1}&0\\\
\Psi_{2K,0}&0&\Psi_{2K,2}&0&\cdots&0&\Psi_{2K,2K}\end{pmatrix},$ (9)

where $\Psi_{m,n}$ is the correlation matrix between all $m$-th and $n$-th
order signatures, which can be calculated using Proposition 3. In particular,
if we re-arrange the indices of the signatures by putting all odd-order
signatures and all even-order signatures together respectively, the
correlation matrix has the following block diagonal form:

$\tilde{\Sigma}^{S}=\mathrm{diag}\\{\Psi_{\mathrm{odd}},\Psi_{\mathrm{even}}\\},$

where $\Psi_{\mathrm{odd}}$ and $\Psi_{\mathrm{even}}$ are given respectively
by

$\begin{pmatrix}\Psi_{1,1}&\Psi_{1,3}&\cdots&\Psi_{1,2K-1}\\\
\Psi_{3,1}&\Psi_{3,3}&\cdots&\Psi_{3,2K-1}\\\ \vdots&\vdots&\cdots&\vdots\\\
\Psi_{2K-1,1}&\Psi_{2K-1,3}&\cdots&\Psi_{2K-1,2K-1}\end{pmatrix}~{}~{}\text{and}~{}~{}\begin{pmatrix}\Psi_{0,0}&\Psi_{0,2}&\cdots&\Psi_{0,2K}\\\
\Psi_{2,0}&\Psi_{2,2}&\cdots&\Psi_{2,2K}\\\ \vdots&\vdots&\cdots&\vdots\\\
\Psi_{2K,0}&\Psi_{2K,2}&\cdots&\Psi_{2K,2K}\end{pmatrix}.$ (10)

Propositions 2 and 4 reveal a striking difference between Itô and Stratonovich
signatures for Brownian motions. Specifically, Itô signatures of different
orders are uncorrelated, leading to a block diagonal correlation structure;
Stratonovich signatures, in contrast, are uncorrelated only if they have
different parity, leading to an odd–even alternating structure. This
difference has significant implications for the consistency of the two types
of signatures, which will be discussed in Section 4.

###  3.2 Correlation structure for multi-dimensional OU process

######  Definition 5 (OU process).

$\mathbf{X}$ is a $d$-dimensional Ornstein–Uhlenbeck (OU) process if it can be
expressed as:

$\mathbf{X}_{t}=(X_{t}^{1},X_{t}^{2},\dots,X_{t}^{d})^{\top}=\Gamma(Y_{t}^{1},Y_{t}^{2},\dots,Y_{t}^{d})^{\top},$
(11)

where $\Gamma$ is a matrix independent of $t$, and
$Y_{t}^{1},Y_{t}^{2},\dots,Y_{t}^{d}$ are mutually independent OU processes
driven by the following stochastic differential equations:

$\mathrm{d}Y_{t}^{i}=-\kappa_{i}Y_{t}^{i}\mathrm{d}t+\mathrm{d}W_{t}^{i},\quad
Y_{0}^{i}=0,$

for $i=1,2,\ldots,d$. Here $\kappa_{i}>0$, and $W_{t}^{i}$ are independent
standard Brownian motions.

The parameter $\kappa_{i}$ of the OU process controls the speed of mean
reversion of the process $Y_{t}^{i}$, and a higher $\kappa_{i}$ implies a
stronger mean reversion. When $\kappa_{i}=0$, $Y_{t}^{i}$ reduces to a
standard Brownian motion.

The following proposition shows that the odd–even alternating structure we
observe in Proposition 4 holds for both Itô and Stratonovich signatures of the
OU process.

######  Proposition 5.

Let $\mathbf{X}$ be a $d$-dimensional OU process given by Equation (11). The
correlation matrix for all Itô signatures and the correlation matrix for all
Stratonovich signatures of $\mathbf{X}$, with orders truncated to $2K$, both
have the odd–even alternating structure given by Equation (9).

Proposition 5 can be regarded as a generalization of the correlation
structures for Itô and Stratonovich signatures of the Brownian motion in
Propositions 2 and 4. In particular, for Itô signatures of the Brownian
motion, all off-diagonal blocks in the odd–even alternating structure reduce
to zero, as we observe in Proposition 2. However, the calculation of moments
for the OU process is much more complicated than that for the Brownian motion,
which we discuss in Appendix B.

##  4 Consistency of signatures using Lasso

This section investigates the consistency of feature selection in Lasso using
signatures for both classes of Gaussian processes: the Brownian motion and the
OU process. We also provide results for their discrete-time counterparts: the
random walk and the AR process, respectively.

###  4.1 Consistency of signatures for Brownian motion and random walk

The following propositions characterize when the irrepresentable condition
holds for signatures of Brownian motion.

######  Proposition 6.

For a multi-dimensional Brownian motion given by Equation (6), the
irrepresentable condition holds if and only if it holds for each block in the
block-diagonal correlation matrix. In particular, for Itô signatures this is
true when the irrepresentable condition holds for each $\Omega_{k}$ in
Equation (8); for Stratonovich signatures this is true when the
irrepresentable condition holds for both $\Psi_{\mathrm{odd}}$ and
$\Psi_{\mathrm{even}}$ in Equation (10).

######  Proposition 7.

For a multi-dimensional Brownian motion given by Equation (6), the
irrepresentable condition holds for the correlation matrix of Itô signatures
given by Equation (7) if

$|\rho_{ij}|<\frac{1}{2\max_{0\leq k\leq K}\\{\\#A^{*}_{k}\\}-1},$ (12)

where $A^{*}_{k}$ is defined in Equation (4).

Proposition 6 demonstrates both the similarity and difference between Itô
signatures and Stratonovich signatures for Brownian motions. In particular,
the difference in the block structure of their correlation matrices leads to
the difference in the consistency of their feature selections.

Proposition 7 provides a sufficient condition for Itô signatures that can be
easily used in practice: the Lasso is consistent when different dimensions of
the multi-dimensional Brownian motion are not strongly correlated, with a
sufficient bound by Equation (12).

Empirically, it has been documented that a small $K$ suffices to provide a
reasonable approximation in applications [27, 32]. Therefore, this bound can
be fairly easy to satisfy.

The consistency study for Stratonovich signatures reveals a different picture:
the irrepresentable condition may fail even when all dimensions of
$\mathbf{X}$ are mutually independent, as shown in the following example.

######  Example 1.

Consider a two-dimensional process
$\mathbf{X}_{t}=(X_{t}^{1},X_{t}^{2})=(W_{t}^{1},W_{t}^{2})$, where
$W_{t}^{1}$ and $W_{t}^{2}$ are independent standard Brownian motions. Suppose
that one includes all Stratonovich signatures of orders up to $K=4$ in the
Lasso regression given by Equation (5), and the true model given by Equation
(3) has beta coefficients $\beta_{0}=0$, $\beta_{1}>0$, $\beta_{2}>0$,
$\beta_{1,1}>0$, $\beta_{1,2}>0$, $\beta_{2,1}>0$, $\beta_{2,2}<0$, and
$\beta_{i_{1},i_{2},i_{3}}=\beta_{i_{1},i_{2},i_{3},i_{4}}=0$. Then, by
Proposition 3,

$\displaystyle{\Sigma}_{A^{*c},A^{*}}^{S}({\Sigma}_{A^{*},A^{*}}^{S})^{-1}\mathrm{sign}({\boldsymbol{\beta}}_{A^{*}})$
$\displaystyle=(0,0.77,0.5,0,0.5,0.5,0,0.5,0.77,1.01,0.73,0.47,0,$
$\displaystyle~{}~{}~{}~{}0.47,0,0.58,0.73,0.73,-0.58,0,0.47,0,0.47,0.73,-1.01),$

which does not satisfy the irrepresentable condition defined in Definition 3
because $1.01>1$.

Example 1 distinguishes the consistency of Itô and Stratonovich signatures
using Lasso. Since the irrepresentable condition is almost necessary and
sufficient for the consistency of Lasso [43], it suggests that the statistical
properties of Lasso may be worse for Stratonovich signatures.

Simulations further confirm this implication.444Appendix C reports more
details for the simulations including its computational cost and robustness
checks. Consider a two-dimensional ($d=2$) Brownian motion with inter-
dimensional correlation $\rho$; assume that there are $q=\\#A^{*}$ true
predictors in the true model (3), and all of these predictors are signatures
of orders no greater than $K=4$. Now, first randomly choose $q$ true
predictors from all $\frac{d^{K+1}-1}{d-1}=31$ signatures; next randomly set
each beta coefficient of these true predictors from the standard normal
distribution; next generate 100 samples from this true model with error term
$\varepsilon_{n}$ drawn from a normal distribution with mean zero and standard
error 0.01; then run a Lasso regression given by Equation (5) to select
predictors based on these 100 samples; and finally check whether the Lasso is
sign consistent according to Definition 2. Repeat the above procedure by 1,000
times and calculate the _consistency rate_ , which is defined as the
proportion of consistent results among these 1,000 experiments.

(a) Brownian motion.

(b) Random walk.

Figure 1: Consistency rates for the Brownian motion and the random walk with
different values of inter-dimensional correlation, $\rho$, and different
numbers of true predictors, $q$. Solid lines correspond to Itô signatures and
dashed lines correspond to Stratonovich signatures.

Figure 1 shows the consistency rates for different values of inter-dimensional
correlation, $\rho$, and different numbers of true predictors, $q$. Figure
1(a) shows the results for the Brownian motion, and Figure 1(b) for its
discrete version—the random walk. First, signatures for both Brownian motion
and random walk are similar: they both exhibit higher consistency rates when
the absolute value of $\rho$ is small, i.e., when the inter-dimensional
correlations of the Brownian motion (random walk) are weak; as the number of
true predictors $q$ increases, both consistency rates decrease; and finally,
consistency rates for Itô signatures are consistently higher than those for
Stratonovich signatures, holding other variables constant ($\rho$ and $q$).
All these findings are consistent with our theoretical results.

###  4.2 Consistency of signatures for OU processes and AR processes

For both the Itô and the Stratonovich signatures of the OU process, we have
the following necessary and sufficient condition for the irrepresentable
condition. However, it appears difficult to derive the analogue of Proposition
7 for OU processes.

######  Proposition 8.

For a multi-dimensional OU process given by Equation (11), the irrepresentable
condition holds for the correlation matrix of signatures if and only if it
holds for both $\Psi_{\mathrm{odd}}$ and $\Psi_{\mathrm{even}}$ given by
Equation (10). This result holds for both Itô and Stratonovich signatures.

Now we study the impact of different degrees of mean reversion on the
consistency of Lasso, for both the OU process and its discrete version—the
autoregressive AR(1) model with parameter $\phi$. Recall that higher values of
$\kappa$ for the OU process and lower values of $\phi$ for the AR(1) model
imply stronger levels of mean reversion. We consider two-dimensional OU
processes and AR(1) processes, with both dimensions sharing the same
parameters ($\kappa$ and $\phi$). The inter-dimensional correlation matrix
$\Gamma\Gamma^{\top}$ is randomly drawn from the Wishart$(2,2)$ distribution.
Other simulation setups are the same as in Section 4.1.

(a) OU process.

(b) AR(1) model.

Figure 2: Consistency rates for the OU process and the AR(1) model with
different parameters ($\kappa$ and $1-\phi$) and different numbers of true
predictors, $q$. Solid lines correspond to Itô signatures and dashed lines
correspond to Stratonovich signatures.

Figure 2 shows the simulation results for the consistency rates of both
processes. First, the Itô signature reaches the highest consistency rate when
$\kappa$ and $1-\phi$ approach $0$, which correspond respectively to a
Brownian motion and a random walk. Second, when the process is sufficiently
mean reverting, Stratonovich signatures have higher consistency rates than Itô
signatures. Finally, as observed in Section 4.1, Lasso gets less consistent
when the number of true predictors $q$ increases.

These results suggest that, in practice, for processes that are sufficiently
rough or mean reverting [11, 14], using Lasso with Stratonovich signatures
will likely lead to higher statistical consistency compared to Itô signatures.
More theoretical explanations are provided in Appendix B.

##  5 Discussion

##### Consistency and universal nonlinearity.

We have adopted the sign consistency of Lasso [43], defined as whether the
Lasso can select _all_ true predictors with correct signs. This restrictive
notion of consistency may be relaxed in the context of signatures because the
true model is approximated by the linear combination of signatures with some
error according to Equation (2). Extensions of the sign consistency for
signatures are given in Appendix D. Overall, a lower sign consistency implies
poorer performances when using other metrics to measure the performance of
feature selections using Lasso, such as the out-of-sample mean squared error.
This highlights the significant and practical relevance to study the
statistical properties of the signature transform.

##### Time augmentation.

Time augmentation is a widely used technique in signature-based analysis,
which involves adding a time dimension $t$ to the original time series,
$\mathbf{X}_{t}$ [6, 27]. Time augmentation lowers the consistency rate of
Lasso, as presented in Appendix E.

##### Other feature selection techniques.

While Lasso is a popular feature selection technique, there are also other
commonly used techniques, such as the ridge regression [17]. The research on
the consistency of signatures using other feature selection techniques is left
for further investigation.

##  6 Conclusion

This paper studies the statistical consistency of Lasso regression for
signatures. It finds that consistency is highly dependent on the definition of
the signatures and the characteristics of the underlying processes. These
findings call for further statistical studies for signature transform before
its potential for machine learning can be fully realized.

## References

  * [1] Imanol Perez Arribas.  Derivatives pricing using signature payoffs.  arXiv preprint arXiv:1809.09466, 2018.
  * [2] Peter J Bickel, Ya’acov Ritov, and Alexandre B Tsybakov.  Simultaneous analysis of Lasso and Dantzig selector.  The Annals of Statistics, pages 1705–1732, 2009.
  * [3] Horatio Boedihardjo, Hao Ni, and Zhongmin Qian.  Uniqueness of signature for simple curves.  Journal of Functional Analysis, 267(6):1778–1806, 2014.
  * [4] Kuo-Tsai Chen.  Iterated integrals and exponential homomorphisms.  Proceedings of the London Mathematical Society, 3(1):502–512, 1954\.
  * [5] Kuo-Tsai Chen.  Integration of paths, geometric invariants and a generalized Baker–Hausdorff formula.  Annals of Mathematics, pages 163–178, 1957.
  * [6] Ilya Chevyrev and Andrey Kormilitzin.  A primer on the signature method in machine learning.  arXiv preprint arXiv:1603.03788, 2016.
  * [7] Ilya Chevyrev and Terry Lyons.  Characteristic functions of measures on geometric rough paths.  Annals of Probability, 44(6):4049–4082, 2016.
  * [8] Ilya Chevyrev and Harald Oberhauser.  Signature moments to characterize laws of stochastic processes.  Journal of Machine Learning Research, 23(176):1–42, 2022.
  * [9] Christa Cuchiero, Guido Gazzani, and Sara Svaluto-Ferro.  Signature-based models: theory and calibration.  arXiv preprint arXiv:2207.13136, 2022.
  * [10] Christa Cuchiero, Francesca Primavera, and Sara Svaluto-Ferro.  Universal approximation theorems for continuous functions of càdlàg paths and Lévy-type signature models.  arXiv preprint arXiv:2208.02293, 2022.
  * [11] Omar El Euch, Masaaki Fukasawa, and Mathieu Rosenbaum.  The microstructural foundations of leverage effect and rough volatility.  Finance and Stochastics, 22:241–280, 2018.
  * [12] Adeline Fermanian.  Embedding and learning with signatures.  Computational Statistics & Data Analysis, 157:107148, 2021.
  * [13] Peter K Friz and Nicolas B Victoir.  Multidimensional Stochastic Processes as Rough Paths: Theory and Applications, volume 120.  Cambridge University Press, 2010.
  * [14] Jim Gatheral, Thibault Jaisson, and Mathieu Rosenbaum.  Volatility is rough.  Quantitative Finance, 18(6):933–949, 2018.
  * [15] Lajos Gergely Gyurkó, Terry Lyons, Mark Kontkowski, and Jonathan Field.  Extracting information from the signature of a financial data stream.  arXiv preprint arXiv:1307.7244, 2013.
  * [16] Ben Hambly and Terry Lyons.  Uniqueness for the signature of a path of bounded variation and the reduced path group.  Annals of Mathematics, pages 109–167, 2010.
  * [17] Trevor Hastie, Robert Tibshirani, Jerome H Friedman, and Jerome H Friedman.  The Elements of Statistical Learning: Data Mining, Inference, and Prediction, volume 2.  Springer, 2009.
  * [18] Jasdeep Kalsi, Terry Lyons, and Imanol Perez Arribas.  Optimal execution with rough path signatures.  SIAM Journal on Financial Mathematics, 11(2):470–493, 2020.
  * [19] Patrick Kidger, Patric Bonnier, Imanol Perez Arribas, Cristopher Salvi, and Terry Lyons.  Deep signature transforms.  Advances in Neural Information Processing Systems, 32, 2019.
  * [20] Franz J Király and Harald Oberhauser.  Kernels for sequentially ordered data.  Journal of Machine Learning Research, 20, 2019.
  * [21] Andrey Kormilitzin, Kate EA Saunders, Paul J Harrison, John R Geddes, and Terry Lyons.  Detecting early signs of depressive and manic episodes in patients with bipolar disorder using the signature-based model.  arXiv preprint arXiv:1708.01206, 2017.
  * [22] Yves Le Jan and Zhongmin Qian.  Stratonovich’s signatures of Brownian motion determine Brownian sample paths.  Probability Theory and Related Fields, 157(1-2):209–223, 2013.
  * [23] Maud Lemercier, Cristopher Salvi, Theodoros Damoulas, Edwin Bonilla, and Terry Lyons.  Distribution regression for sequential data.  In International Conference on Artificial Intelligence and Statistics, pages 3754–3762. PMLR, 2021.
  * [24] Daniel Levin, Terry Lyons, and Hao Ni.  Learning from the past, predicting the statistics for the future, learning an evolving system.  arXiv preprint arXiv:1309.0260, 2016.
  * [25] Chenyang Li, Xin Zhang, and Lianwen Jin.  LPSNet: a novel log path signature feature based hand gesture recognition framework.  In Proceedings of the IEEE International Conference on Computer Vision Workshops, pages 631–639, 2017.
  * [26] Terry Lyons.  Rough paths, signatures and the modelling of functions on streams.  arXiv preprint arXiv:1405.4537, 2014.
  * [27] Terry Lyons and Andrew D McLeod.  Signature methods in machine learning.  arXiv preprint arXiv:2206.14674, 2022.
  * [28] Terry Lyons, Sina Nejad, and Imanol Perez Arribas.  Numerical method for model-free pricing of exotic derivatives in discrete time using rough path signatures.  Applied Mathematical Finance, 26(6):583–597, 2019.
  * [29] Terry Lyons, Sina Nejad, and Imanol Perez Arribas.  Non-parametric pricing and hedging of exotic derivatives.  Applied Mathematical Finance, 27(6):457–494, 2020.
  * [30] Terry Lyons, Hao Ni, and Harald Oberhauser.  A feature set for streams and an application to high-frequency financial tick data.  In Proceedings of the 2014 International Conference on Big Data Science and Computing, pages 1–8, 2014.
  * [31] Terry J Lyons, Michael Caruana, and Thierry Lévy.  Differential Equations Driven by Rough Paths.  Springer, 2007.
  * [32] James Morrill, Adeline Fermanian, Patrick Kidger, and Terry Lyons.  A generalised signature method for multivariate time series feature extraction.  arXiv preprint arXiv:2006.00873, 2020.
  * [33] James Morrill, Andrey Kormilitzin, Alejo Nevado-Holgado, Sumanth Swaminathan, Sam Howison, and Terry Lyons.  The signature-based model for early detection of sepsis from electronic health records in the intensive care unit.  In 2019 Computing in Cardiology (CinC). IEEE, 2019.
  * [34] James H Morrill, Andrey Kormilitzin, Alejo J Nevado-Holgado, Sumanth Swaminathan, Samuel D Howison, and Terry J Lyons.  Utilization of the signature method to identify the early onset of sepsis from multivariate physiological time series in critical care monitoring.  Critical Care Medicine, 48(10):e976–e981, 2020.
  * [35] Robert Tibshirani.  Regression shrinkage and selection via the lasso.  Journal of the Royal Statistical Society: Series B (Methodological), 58(1):267–288, 1996.
  * [36] George E Uhlenbeck and Leonard S Ornstein.  On the theory of the Brownian motion.  Physical Review, 36(5):823, 1930.
  * [37] Martin J Wainwright.  Sharp thresholds for high-dimensional and noisy sparsity recovery using $l_{1}$-constrained quadratic programming (Lasso).  IEEE Transactions on Information Theory, 55(5):2183–2202, 2009\.
  * [38] Daniel Wilson-Nunn, Terry Lyons, Anastasia Papavasiliou, and Hao Ni.  A path signature approach to online arabic handwriting recognition.  In 2018 IEEE 2nd International Workshop on Arabic and Derived Script Analysis and Recognition (ASAR), pages 135–139. IEEE, 2018.
  * [39] Weixin Yang, Lianwen Jin, and Manfei Liu.  DeepWriterID: An end-to-end online text-independent writer identification system.  IEEE Intelligent Systems, 31(2):45–53, 2016.
  * [40] Weixin Yang, Lianwen Jin, Hao Ni, and Terry Lyons.  Rotation-free online handwritten character recognition using dyadic path signature features, hanging normalization, and deep neural network.  In 2016 23rd International Conference on Pattern Recognition (ICPR), pages 4083–4088. IEEE, 2016.
  * [41] Weixin Yang, Lianwen Jin, Dacheng Tao, Zecheng Xie, and Ziyong Feng.  DropSample: A new training method to enhance deep convolutional neural networks for large-scale unconstrained handwritten Chinese character recognition.  Pattern Recognition, 58:190–203, 2016.
  * [42] Weixin Yang, Terry Lyons, Hao Ni, Cordelia Schmid, and Lianwen Jin.  Developing the path signature methodology and its application to landmark-based human action recognition.  In Stochastic Analysis, Filtering, and Stochastic Optimization: A Commemorative Volume to Honor Mark HA Davis’s Contributions, pages 431–464. Springer, 2022.
  * [43] Peng Zhao and Bin Yu.  On model selection consistency of Lasso.  The Journal of Machine Learning Research, 7:2541–2563, 2006.

## Supplementary Material

##  Appendix A Technical details for universal nonlinearity of signatures

There are different statements of universal nonlinearity of signatures in the
literature, which we summarize in Table 1 in the main paper. The following
theorem gives the precise statement of universal nonlinearity proposed in
[27].

######  Theorem 1 (Universal nonlinearity, Theorem 3.1 of [27]).

Let $\mathbf{X}_{t}$ be a $\mathbb{R}^{d}$-valued continuous path with
$p$-variation, and let $\mathcal{S}$ be a compact subset of signature paths of
${\mathbf{X}}_{t}$ from time 0 to $T$. Assume that
$f:\mathcal{S}\to\mathbb{R}$ is a continuous function. Then, for any
$\varepsilon>0$, there exists some linear functional
$L:\mathbb{R}^{\infty}\to\mathbb{R}$ such that for every $s\in\mathcal{S}$:

$\left|f(s)-L(s)\right|\leq\varepsilon.$

The proof of this theorem can be found in [27]. Other versions of universal
nonlinearity can be found in, for example, [1, 9, 10, 12, 20, 24, 29].

##  Appendix B Technical details and examples for the calculation of
correlation matrices

This appendix provides details and examples for calculating the correlation
structures of signatures. Appendices B.1 and B.2 discuss the Brownian motion
and the OU process, respectively.

###  B.1 Brownian motion

##### Itô signature.

Propositions 1–2 in the main paper give explicit formulas for calculating the
correlation structure of Itô signatures for Brownian motions. The “recursive
order” mentioned in Proposition 2 is defined as follows.

######  Definition A.1 (Recursive Order).

Consider a $d$-dimensional process $\mathbf{X}$. We order the indices of all
of its 1st order signatures as:

$1\quad 2\quad\cdots\quad d.$

Then, if all $k$-th order signatures are ordered as:

$r_{1}\quad r_{2}\quad\cdots\quad r_{d^{k}},$

we define the orders of all $(k+1)$-th order signatures as:

$r_{1},1\quad r_{2},1\quad\cdots\quad r_{d^{k}},1\quad r_{1},2\quad
r_{2},2\quad\cdots\quad r_{d^{k}},2\quad\cdots\cdots\cdots\quad r_{1},d\quad
r_{2},d\quad\cdots\quad r_{d^{k}},d.$

For example, for a $d=3$ dimensional process, the recursive order of its
signatures is:

  * •

1st order: $1\quad 2\quad 3$

  * •

2nd order: $1,1\quad 2,1\quad 3,1\quad 1,2\quad 2,2\quad 3,2\quad 1,3\quad
2,3\quad 3,3$

  * •

3rd order: $1,1,1\quad 2,1,1\quad 3,1,1\quad 1,2,1\quad 2,2,1\quad 3,2,1\quad
1,3,1\quad 2,3,1\quad 3,3,1$
$~{}~{}~{}~{}~{}~{}~{}~{}~{}~{}~{}~{}~{}~{}~{}~{}~{}~{}1,1,2\quad 2,1,2\quad
3,1,2\quad 1,2,2\quad 2,2,2\quad 3,2,2\quad 1,3,2\quad 2,3,2\quad 3,3,2$
$~{}~{}~{}~{}~{}~{}~{}~{}~{}~{}~{}~{}~{}~{}~{}~{}~{}~{}1,1,3\quad 2,1,3\quad
3,1,3\quad 1,2,3\quad 2,2,3\quad 3,2,3\quad 1,3,3\quad 2,3,3\quad 3,3,3$

  * •

…

To provide intuition for Propositions 1–2 in the main paper, the following two
examples show the correlation structures of Itô signatures for 2-dimensional
Brownian motions with inter-dimensional correlations $\rho=0.6$ and $\rho=0$,
respectively.

######  Example A.1.

Consider a 2-dimensional Brownian motion given by Equation (6) with inter-
dimensional correlation $\rho=0.6$. Figure A.1 shows the correlation matrix of
its Itô signatures with orders truncated to 4 calculated using Proposition 1.
The figure illustrates Proposition 2—the correlation matrix has a block
diagonal structure, and each block of the matrix is the Kronecker product of
the inter-dimensional correlation matrix $\begin{pmatrix}1&0.6\\\
0.6&1\end{pmatrix}$.

Figure A.1: Correlation matrix of Itô signatures with orders truncated to 4
for a 2-dimensional Brownian motion with inter-dimensional correlation
$\rho=0.6$.

######  Example A.2.

Consider a 2-dimensional Brownian motion given by Equation (6) with inter-
dimensional correlation $\rho=0$. Figure A.2 shows the correlation matrix of
its Itô signatures with orders truncated to 4 calculated using Proposition 1.
When $\rho=0$, the block diagonal correlation matrix reduces to an identity
matrix, indicating that all of its Itô signatures are mutually uncorrelated.

Figure A.2: Correlation matrix of Itô signatures with orders truncated to 4
for a 2-dimensional Brownian motion with inter-dimensional correlation
$\rho=0$.

##### Stratonovich signature.

Propositions 3–4 in the main paper provide formulas for calculating the
correlation structure of Stratonovich signatures for Brownian motions. The
following proposition gives recursive formulas for calculating
$\mathbb{E}\left[S(\mathbf{X})_{t}^{i_{1},\dots,i_{2n},S}S(\mathbf{X})_{t}^{j_{1},\dots,j_{2m},S}\right]$
and
$\mathbb{E}\left[S(\mathbf{X})_{t}^{i_{1},\dots,i_{2n-1},S}S(\mathbf{X})_{t}^{j_{1},\dots,j_{2m-1},S}\right]$,
which extends Proposition 3 in the main paper.

######  Proposition A.1.

Let $\mathbf{X}$ be a $d$-dimensional Brownian motion given by Equation (6).
For any $l,t\geq 0$ and $m,n=1,2,\dots$, define
$f_{2n,2m}(l,t):=\mathbb{E}\left[S(\mathbf{X})_{l}^{i_{1},\dots,i_{2n},S}S(\mathbf{X})_{t}^{j_{1},\dots,j_{2m},S}\right]$,
we have:

$\displaystyle f_{2n,2m}(l,t)$
$\displaystyle=g_{2n,2m}(l,t)+\frac{1}{2}\rho_{j_{2m-1}j_{2m}}\sigma_{j_{2m-1}}\sigma_{j_{2m}}\int_{0}^{t}f_{2n,2m-2}(l,s)\mathrm{d}s,$
(A.1) $\displaystyle g_{2n,2m}(l,t)$
$\displaystyle=\rho_{i_{2n}j_{2m}}\sigma_{i_{2n}}\sigma_{j_{2m}}\int_{0}^{l\wedge
t}f_{2n-1,2m-1}(s,s)\mathrm{d}s$
$\displaystyle\qquad\qquad+\frac{1}{2}\rho_{i_{2n-1}i_{2n}}\sigma_{i_{2n-1}}\sigma_{i_{2n}}\int_{0}^{l}g_{2n-2,2m}(s,t)\mathrm{d}s,$
(A.2)

with initial conditions

$\displaystyle f_{0,0}(l,t)$ $\displaystyle=1,$ (A.3) $\displaystyle
g_{0,2m}(l,t)$ $\displaystyle=0.$ (A.4)

In addition, define
$f_{2n-1,2m-1}(l,t):=\mathbb{E}\left[S(\mathbf{X})_{l}^{i_{1},\dots,i_{2n-1},S}S(\mathbf{X})_{t}^{j_{1},\dots,j_{2m-1},S}\right]$,
we have:

$\displaystyle f_{2n-1,2m-1}(l,t)$
$\displaystyle=g_{2n-1,2m-1}(l,t)+\frac{1}{2}\rho_{j_{2m-2}j_{2m-1}}\sigma_{j_{2m-2}}\sigma_{j_{2m-1}}\int_{0}^{t}f_{2n-1,2m-3}(l,s)\mathrm{d}s,$
(A.5) $\displaystyle g_{2n-1,2m-1}(l,t)$
$\displaystyle=\rho_{i_{2n-1}j_{2m-1}}\sigma_{i_{2n-1}}\sigma_{j_{2m-1}}\int_{0}^{l\wedge
t}f_{2n-2,2m-2}(s,s)\mathrm{d}s$
$\displaystyle\qquad\qquad+\frac{1}{2}\rho_{i_{2n-2}i_{2n-1}}\sigma_{i_{2n-2}}\sigma_{i_{2n-1}}\int_{0}^{l}g_{2n-3,2m-1}(s,t)\mathrm{d}s,$
(A.6)

with initial conditions

$\displaystyle f_{1,1}(l,t)$
$\displaystyle=\rho_{i_{1}j_{1}}\sigma_{i_{1}}\sigma_{j_{1}}(l\wedge t),$
(A.7) $\displaystyle g_{1,2m-1}(l,t)$
$\displaystyle=\rho_{i_{1}j_{2m-1}}\frac{1}{2^{m-1}}\frac{(l\wedge
t)^{m-1}}{(m-1)!}\sigma_{i_{1}}\prod_{k=1}^{2m-1}\sigma_{j_{k}}\prod_{k=1}^{m-1}\rho_{j_{2k-1}j_{2k}}.$
(A.8)

Here, $x\wedge y$ represents the smaller value between $x$ and $y$.

The following two examples show the correlation structures of Stratonovich
signatures for 2-dimensional Brownian motions with inter-dimensional
correlations $\rho=0.6$ and $\rho=0$, respectively, calculated using
Propositions 3–4 in the main paper and Proposition A.1.

######  Example A.3.

Consider a 2-dimensional Brownian motion given by Equation (6) with inter-
dimensional correlation $\rho=0.6$. Figure A.3 shows the correlation matrix of
its Stratonovich signatures with orders truncated to 4 calculated using
Propositions 3 and A.1. The figure illustrates that the correlation matrix has
an odd–even alternating structure.

Figure A.3: Correlation matrix of Stratonovich signatures with orders
truncated to 4 for a 2-dimensional Brownian motion with inter-dimensional
correlation $\rho=0.6$.

######  Example A.4.

Consider a 2-dimensional Brownian motion given by Equation (6) with inter-
dimensional correlation $\rho=0$. Figure A.4 shows the correlation matrix of
its Stratonovich signatures with orders truncated to 4 calculated using
Propositions 3 and A.1. The figure demonstrates that the correlation matrix
has an odd–even alternating structure, even though different dimensions of the
Brownian motion are mutually independent ($\rho=0$). This is different from
the result for Itô signatures shown in Example A.2, where all Itô signatures
are mutually uncorrelated. Note that this example has the same setup as
Example 1 in the main paper.

Figure A.4: Correlation matrix of Stratonovich signatures with orders
truncated to 4 for a 2-dimensional Brownian motion with inter-dimensional
correlation $\rho=0$.

###  B.2 OU process

Deriving explicit formulas for calculating the exact correlation between
signatures of OU processes (both Itô and Stratonovich) is complicated. Here we
provide an example to show the general approach for calculating the
correlation. The proof of this example is given in Appendix F, and one can use
a similar routine to compute the correlation for other setups of OU processes.

######  Example A.5.

Consider a 1-dimensional OU process $\mathbf{X}_{t}=Y_{t}$ with a mean
reversion speed $\kappa>0$:

$\mathrm{d}Y_{t}=-\kappa Y_{t}\mathrm{d}t+\mathrm{d}W_{t},\quad Y_{0}=0.$
(A.9)

The correlation coefficients between its 0-th order and 2nd order signatures
are

$\displaystyle\frac{\mathbb{E}\left[S(\mathbf{X})_{T}^{0,I}S(\mathbf{X})_{T}^{1,1,I}\right]}{\sqrt{\mathbb{E}\left[S(\mathbf{X})_{T}^{0,I}\right]^{2}\mathbb{E}\left[S(\mathbf{X})_{T}^{1,1,I}\right]^{2}}}$
$\displaystyle=\frac{-2\kappa T-e^{-2\kappa T}+1}{\sqrt{4\kappa Te^{-2\kappa
T}+3e^{-4\kappa T}-6e^{-2\kappa T}-4\kappa T+3+4\kappa^{2}T^{2}}},$
$\displaystyle\frac{\mathbb{E}\left[S(\mathbf{X})_{T}^{0,S}S(\mathbf{X})_{T}^{1,1,S}\right]}{\sqrt{\mathbb{E}\left[S(\mathbf{X})_{T}^{0,S}\right]^{2}\mathbb{E}\left[S(\mathbf{X})_{T}^{1,1,S}\right]^{2}}}$
$\displaystyle=\frac{\sqrt{3}}{3},$

for Itô and Stratonovich signatures, respectively. The proof is provided in
Appendix F.

Figure 5(a) shows the absolute values of correlation coefficients between the
0-th order and 2nd order signatures calculated using the formulas above under
different values of $\kappa$. Notably, the correlation for Itô signatures
rises with $\kappa$, while the correlation for Stratonovich signatures remains
fixed at $\frac{\sqrt{3}}{3}$.

(a) Correlation between the 0-th and the 2nd order signatures.

(b) Correlation between the first four order signatures.

Figure A.5: Absolute values of correlation coefficients between signatures of
the 1-dimensional OU process. Solid lines correspond to Itô signatures and
dashed lines correspond to Stratonovich signatures.

We further perform simulations to estimate the correlation coefficients for
higher-order signatures of the OU process. We generate 10,000 sample paths of
the OU process using the methods discussed in Appendix C. For each path, we
calculate the corresponding signatures and then estimate the sample
correlation matrix based on the 10,000 simulated samples. Figure 5(b) shows
the simulation results for the absolute values of correlation coefficients
between the first four order signatures under different values of $\kappa$.
Consistent with the observation in Figure 5(a), the correlation for Itô
signatures rises with $\kappa$, while the correlation for Stratonovich
signatures remains relatively stable. Notably, the correlations for Itô
signatures are zero when $\kappa=0$, which reduces to a Brownian motion. In
addition, when $\kappa$ is sufficiently large, the absolute values of
correlation coefficients for Itô signatures exceed those for Stratonovich
signatures.

Recall that the irrepresentable condition, as defined in Definition 3,
illustrates that a higher correlation generally leads to poorer consistency.
Therefore, based on Example A.5, we can expect that the Lasso is more
consistent when using Itô signatures for small values of $\kappa$ (weaker mean
reversion), and more consistent when using Stratonovich signatures for large
values of $\kappa$ (stronger mean reversion). This provides a theoretical
explanation for our observations in Section 4.2 of the main paper: When
processes are sufficiently rough or mean reverting [11, 14], using Lasso with
Stratonovich signatures will likely lead to higher statistical consistency
compared to Itô signatures.

##  Appendix C Details for simulations

This appendix provides additional technical details, computational cost, and
robustness checks for the simulations conducted in this paper.

###  C.1 More technical details

##### Simulation of processes.

We simulate the $i$-th dimension of the Brownian motion, $W_{t}^{i}$, and OU
process, $Y_{t}^{i}$, by discretizing the stochastic differential equations of
the processes using the Euler–Maruyama method:

  * •

Brownian motion: $W_{t_{k+1}}^{i}=W_{t_{k}}^{i}+\sqrt{\Delta
t}\varepsilon_{k}^{i}$, $W_{0}^{i}=0$;

  * •

OU process: $Y_{t_{k+1}}^{i}=Y_{t_{k}}^{i}-\kappa_{i}Y_{t_{k}}^{i}\Delta
t+\sqrt{\Delta t}\varepsilon_{k}^{i}$, $Y_{0}^{i}=0$.

Here, $0=t_{0}<t_{1}<\dots<t_{N}=T$, $t_{k+1}-t_{k}=\Delta t=T/N$ for any $k$,
and $\varepsilon_{k}^{i}$ are randomly drawn from the standard normal
distribution. The number of steps is set to $N=100$.

The $i$-th dimension of the random walk and AR(1) model, both denoted by
$Z_{t}^{i}$, are simulated using the following formulas:

  * •

Random walk: $Z_{t_{k+1}}^{i}=Z_{t_{k}}^{i}+e_{k}^{i}$, $Z_{0}^{i}=0$;

  * •

AR(1) model: $Z_{t_{k+1}}^{i}=\phi_{i}Z_{t_{k}}^{i}+\varepsilon_{k}^{i}$,
$Z_{0}^{i}=0$.

Here, $0=t_{0}<t_{1}<\dots<t_{N}=T$, $t_{k+1}-t_{k}=\Delta t=T/N$ for any $k$,
$e_{k}^{i}$ are randomly drawn from the following distribution:

$\mathbb{P}(e_{k}^{i}=+1)=\mathbb{P}(e_{k}^{i}=-1)=0.5,$

and $\varepsilon_{k}^{i}$ are randomly drawn from the standard normal
distribution. The number of steps is set to $N=100$.

After simulating each dimension of the processes, we simulate the inter-
dimensional correlation between different dimensions of the processes using
the Cholesky decomposition. Specifically, we set the inter-dimensional
correlation matrix $\Gamma\Gamma^{\top}$ based on the setups described in the
main paper and calculate $\Gamma$ using the Cholesky decomposition. Finally,
we generate $\mathbf{X}$ using Equations (6) or (11).

In all of our simulations, we set the length of the processes to $T=1$, and
the initial values of the processes to zero. These choices have no impact on
the results because the signatures of a path $\mathbf{X}$ are invariant under
a time reparametrization and a shift of the starting point of $\mathbf{X}$,
see, for example, [6].

##### Calculation of integrals.

The calculation of Itô and Stratonovich signatures requires the calculation of
Itô and Stratonovich integrals. By definition, these integrals are computed
using the following schemes:

  * •

Itô integral:
$\int_{0}^{T}A_{t}\mathrm{d}B_{t}\approx\sum_{k=0}^{N-1}A_{t_{k}}(B_{t_{k+1}}-B_{t_{k}})$;

  * •

Stratonovich integral:
$\int_{0}^{T}A_{t}\circ\mathrm{d}B_{t}\approx\sum_{k=0}^{N-1}\frac{1}{2}(A_{t_{k}}+A_{t_{k+1}})(B_{t_{k+1}}-B_{t_{k}})$.

Here, we set $0=t_{0}<t_{1}<\dots<t_{N}=T$ and $t_{k+1}-t_{k}=\Delta t=T/N$
for any $k$.

###  C.2 Computational details

  * •

The simulations are implemented using Python 3.7.

  * •

The simulations are run on a laptop with an Intel(R) Core(TM) i7-9750H CPU @
2.60GHz.

  * •

The random seed is set to 0 for reproducibility.

  * •

The Lasso regressions are performed using the sklearn.linear_model.lars_path
package.

  * •

Each individual experiment, including generating 100 paths, calculating their
signatures, and performing the Lasso regression, can be completed within one
second.

###  C.3 Robustness checks

To show the robustness of our simulations shown in Figures 1 and 2 in Section
4 of the main paper, we present Figures A.6 and A.7, which include confidence
intervals (shaded regions) for the estimated consistency rates of the Brownian
motion/random walk and OU process/AR(1) model, respectively.

In Figures A.6 and A.7, we estimate the consistency rate by repeating the
procedure described in Section 4 100 times, and this process is repeated 30
times to obtain the confidence interval for the estimation. Thus, these
confidence intervals are based on 30 estimations of the consistency rate, with
each estimation calculated using 100 experiments.

(a) Brownian motion.

(b) Random walk.

Figure A.6: Consistency rates for the Brownian motion and the random walk with
different values of inter-dimensional correlation, $\rho$, and different
numbers of true predictors, $q$. Solid lines correspond to Itô signatures and
dashed lines correspond to Stratonovich signatures. Shaded regions are
confidence intervals of the experiments.

(a) OU process.

(b) AR(1) model.

Figure A.7: Consistency rates for the OU process and the AR(1) model with
different parameters ($\kappa$ and $1-\phi$) and different numbers of true
predictors, $q$. Solid lines correspond to Itô signatures and dashed lines
correspond to Stratonovich signatures. Shaded regions are confidence intervals
of the experiments.

We observe that the confidence intervals of the consistency rates shown in
Figures A.6 and A.7 are narrow. Moreover, the observations made in Figures 1
and 2 are consistent with the results presented here, further confirming the
robustness of our findings.

##  Appendix D Extensions of the definition of consistency

As remarked in Section 5 of the main paper, the sign consistency may be too
restrictive and can be relaxed in the context of signatures. This appendix
provides numerical experiments that explore several extensions of consistency
measures for signatures.

###  D.1 Precision, recall, and F1-score

One possible approach for extending the definition of consistency is to use
precision, recall, and the F1-score to evaluate the performance of the Lasso
regression in selecting true predictors.

In particular, for a given tuning parameter $\lambda$ in the Lasso regression,
Equation (5), let $A(\lambda)$ denote the set of selected predictors based on
the Lasso:

$A(\lambda)=\bigcup_{k=0}^{K}\\{(i_{1},\dots,i_{k}):\tilde{\beta}_{i_{1},\dots,i_{k}}(\lambda)\neq
0\\},$

where $\tilde{\beta}_{i_{1},\dots,i_{k}}(\lambda)$ are beta coefficients
estimated using Equation (5). The true predictor set, $A^{*}$, is defined in
Equation (4). We can calculate the true positive (TP), false positive (FP),
true negative (TN), and false negative (TN) counts as follows:

$\text{TP}(\lambda)=\\#A(\lambda)\cap
A^{*},~{}\text{FP}(\lambda)=\\#A(\lambda)\cap
A^{*c},~{}\text{TN}(\lambda)=\\#A(\lambda)^{c}\cap
A^{*c},~{}\text{FN}(\lambda)=\\#A(\lambda)^{c}\cap A^{*},$

where $A^{*c}$ and $A(\lambda)^{c}$ represent the complements of $A^{*}$ and
$A(\lambda)$, respectively. The precision, recall, and F1-score can then be
defined as:

$\displaystyle\text{precision}(\lambda)$
$\displaystyle=\frac{\text{TP}(\lambda)}{\text{TP}(\lambda)+\text{FP}(\lambda)},$
$\displaystyle\text{recall}(\lambda)$
$\displaystyle=\frac{\text{TP}(\lambda)}{\text{TP}(\lambda)+\text{FN}(\lambda)},$
$\displaystyle\text{F1-score}(\lambda)$
$\displaystyle=\frac{2}{1/\text{recall}(\lambda)+1/\text{precision}(\lambda)}.$

We can then examine the maximum values of precision, recall, and F1-score as
we vary the tuning parameter $\lambda$ in the Lasso. These maximum values
reflect the best performance in terms of feature selection by the Lasso.

To assess these measures of consistency, we conducted simulations similar to
those in Section 4.1 for the Brownian motion. Figure A.8 shows the maximum
values of precision, recall, and F1-score for the Brownian motion with
different inter-dimensional correlation values ($\rho$) and numbers of true
predictors ($q$), averaged over 1,000 experiments. The results demonstrate
that, similar to the findings in Figure 1, the maximum precision (Figure 8(a))
and F1-score (Figure 8(c)) reach their highest values when $\rho=0$, and
decrease as the dimensions of the Brownian motion become more correlated. It
is also observed that the results for Itô signatures consistently outperform
those for Stratonovich signatures. In addition, compared to Figure 1, these
alternative measures of consistency exhibit a less pronounced decrease in
performance as the absolute value of $\rho$ increases.

(a) Maximum precision.

(b) Maximum recall.

(c) Maximum F1-score.

Figure A.8: Maximum values of precision, recall, and F1-score for the Brownian
motion with different values of inter-dimensional correlation, $\rho$, and
different numbers of true predictors, $q$. Solid lines correspond to Itô
signatures and dashed lines correspond to Stratonovich signatures.

It is worth noting that the maximum recall rate (Figure 8(b)) remains close to
1 under different parameters. This occurs because none of the predictors are
selected by the Lasso when the tuning parameter $\lambda$ is extremely large.
As a result, $\text{FN}(\lambda)$ is zero, and $\text{recall}(\lambda)$ is
equal to 1. Consequently, the maximum recall rate is not an appropriate
measure of consistency in the context of signatures when using the Lasso.

###  D.2 Out-of-sample $R^{2}$

The out-of-sample $R^{2}$ is a commonly used measure of model performance in
machine learning. It generalizes the notion of consistency as it considers the
performance of the model on out-of-sample data, rather than solely focusing on
the selection of true predictors by the Lasso. This aligns well with practical
applications. Furthermore, it is consistent with the concept of universal
nonlinearity in signatures, as described by Equation (2). Universal
nonlinearity suggests that the true model can be approximated by a linear
combination of signatures, without requiring the exact selection of true
predictors by the Lasso regression.

We conduct simulations to study the out-of-sample $R^{2}$ in the context of
signatures using Lasso. Consider a two-dimensional ($d=2$) Brownian motion
with inter-dimensional correlation $\rho$; assume that there are $q=\\#A^{*}$
true predictors in the true model (3), and all of these predictors are
signatures of orders no greater than $K=4$. Now, first randomly choose $q$
true predictors from all $\frac{d^{K+1}-1}{d-1}=31$ signatures; next randomly
set each beta coefficient of these true predictors from the standard normal
distribution; next generate 200 samples from this true model with error term
$\varepsilon_{n}$ drawn from a normal distribution with mean zero and standard
error 0.0001. We divide the 200 samples into a training set and a test set,
with 100 samples assigned to each. Then, we run a Lasso regression given by
Equation (5) to select predictors based on the training set. The tuning
parameter $\lambda$ is chosen using 5-fold cross-validation. Finally, we
calculate the out-of-sample $R^{2}$ using the chosen $\lambda$ on the test
set. We repeat the above procedure by 1,000 times and calculate the average
out-of-sample $R^{2}$.

Figure A.9 shows the out-of-sample $R^{2}$ for different values of inter-
dimensional correlation, $\rho$, and different numbers of true predictors,
$q$. We can find that, first, Lasso exhibits lower out-of-sample $R^{2}$ when
the absolute value of $\rho$ is small, i.e., when the inter-dimensional
correlations of the Brownian motion are weak. Second, as the number of true
predictors $q$ increases, the out-of-sample $R^{2}$ increases. Finally, Itô
signatures have lower out-of-sample $R^{2}$ than those for Stratonovich
signatures, holding other variables constant ($\rho$ and $q$).

Figure A.9: Average out-of-sample $R^{2}$ for the Brownian motion with
different values of inter-dimensional correlation, $\rho$, and different
numbers of true predictors, $q$. Solid lines correspond to Itô signatures and
dashed lines correspond to Stratonovich signatures.

All these findings are consistent with our analysis of sign consistency in the
main paper. A higher consistency rate corresponds to a higher precision, a
higher F1-score, and a lower out-of-sample $R^{2}$. This consistency across
different metrics reinforces the applicability of our theoretical results when
using alternative measures to evaluate the performance of Lasso in the context
of signatures. It confirms that the theoretical insights derived from sign
consistency extend to other evaluation metrics, demonstrating the robustness
and broad applicability of our findings.

##  Appendix E Time augmentation

Time augmentation is a widely used technique in signature-based analysis,
which involves adding a time dimension $t$ to the original time series,
$\mathbf{X}_{t}$ [6, 27]. In this section, we consider the time-augmented
Brownian motion defined as follows:

######  Definition A.2 (Time-augmented Brownian motion).

$\hat{\mathbf{X}}$ is a $(d+1)$-dimensional time-augmented Brownian motion if
it can be expressed as:

$\hat{\mathbf{X}}_{t}=\begin{pmatrix}t,{\mathbf{X}}_{t}^{\top}\end{pmatrix}^{\top}=\begin{pmatrix}t,X_{t}^{1},X_{t}^{2},\dots,X_{t}^{d}\end{pmatrix}^{\top},$
(A.10)

where $\mathbf{X}_{t}=(X_{t}^{1},X_{t}^{2},\dots,X_{t}^{d})^{\top}$ is a
$d$-dimensional Brownian motion given by Equation (6) in the main paper. For
notational simplicity, let $X_{t}^{0}=t$, then

$\hat{\mathbf{X}}_{t}=\begin{pmatrix}X_{t}^{0},X_{t}^{1},X_{t}^{2},\dots,X_{t}^{d}\end{pmatrix}^{\top}.$

Now we discuss the correlation structure of signatures and the consistency of
signatures using Lasso for the time-augmented Brownian motion.

###  E.1 Correlation structure of signatures for time-augmented Brownian
motion

The following proposition shows the moments of Itô signatures for the time-
augmented Brownian motion. Note that an index of 0 corresponds to the time
dimension, $X_{t}^{0}=t$.

######  Proposition A.2.

Let $\hat{\mathbf{X}}$ be a $(d+1)$-dimensional time-augmented Brownian motion
given by Equation (A.10). For any $l,t\geq 0$ and $m,n=1,2,\dots$,
$f_{n,m}(l,t):=\mathbb{E}\left[S(\hat{\mathbf{X}})_{l}^{i_{1},\dots,i_{n},I}S(\hat{\mathbf{X}})_{t}^{j_{1},\dots,j_{m},I}\right]$
can be calculated recursively by:

$f_{n,m}(l,t)=\begin{cases}\int_{0}^{l}\int_{0}^{t}f_{n-1,m-1}(s,\tau)\mathrm{d}\tau\mathrm{d}s,&\text{if
}i_{n}=0,j_{m}=0,\\\ \int_{0}^{l}f_{n-1,m}(s,t)\mathrm{d}s,&\text{if
}i_{n}=0,j_{m}\neq 0,\\\ \int_{0}^{t}f_{n,m-1}(l,s)\mathrm{d}s,&\text{if
}i_{n}\neq 0,j_{m}=0,\\\
\rho_{i_{n}j_{m}}\sigma_{i_{n}}\sigma_{j_{m}}\int_{0}^{l\wedge
t}f_{n-1,m-1}(s,s)\mathrm{d}s,&\text{if }i_{n}\neq 0,j_{m}\neq 0,\end{cases}$

with initial conditions

$\displaystyle f_{0,m}(l,t)$
$\displaystyle=\begin{cases}\frac{t^{m}}{m!}&\text{if
}j_{1}=j_{2}=\dots=j_{m}=0,\\\ 0,&\text{otherwise},\end{cases}$ $\displaystyle
f_{n,0}(l,t)$ $\displaystyle=\begin{cases}\frac{t^{n}}{n!}&\text{if
}i_{1}=i_{2}=\dots=i_{n}=0,\\\ 0,&\text{otherwise}.\end{cases}$

The following proposition shows the moments of Stratonovich signatures for the
time-augmented Brownian motion.

######  Proposition A.3.

Let $\hat{\mathbf{X}}$ be a $(d+1)$-dimensional time-augmented Brownian motion
given by Equation (A.10). For any $l,t\geq 0$ and $m,n=1,2,\dots$,
$f_{n,m}(l,t):=\mathbb{E}\left[S(\hat{\mathbf{X}})_{l}^{i_{1},\dots,i_{n},S}S(\hat{\mathbf{X}})_{t}^{j_{1},\dots,j_{m},S}\right]$
can be calculated recursively by:

$\displaystyle f_{n,m}(l,t)=$
$\displaystyle\begin{cases}\int_{0}^{l}\int_{0}^{t}f_{n-1,m-1}(s,\tau)\mathrm{d}\tau\mathrm{d}s,&\text{if
}i_{n}=0,j_{m}=0,\\\ \int_{0}^{l}f_{n-1,m}(s,t)\mathrm{d}s,&\text{if
}i_{n}=0,j_{m}\neq 0,\\\
\rho_{i_{n}j_{m}}\sigma_{i_{n}}\sigma_{j_{m}}\int_{0}^{l\wedge
t}f_{n-1,m-1}(s,s)\mathrm{d}s,&\text{if }i_{n}\neq 0,j_{m}\neq
0,i_{n-1}=0,j_{m-1}=0,\\\
\rho_{i_{n}j_{m}}\sigma_{i_{n}}\sigma_{j_{m}}\int_{0}^{l\wedge
t}f_{n-1,m-1}(s,s)\mathrm{d}s&\\\
~{}~{}~{}+\frac{1}{2}\rho_{i_{n-1}i_{n}}\sigma_{i_{n-1}}\sigma_{i_{n}}\int_{0}^{l}g_{n-2,m}(s,t)\mathrm{d}s,&\text{if
}i_{n}\neq 0,j_{m}\neq 0,i_{n-1}\neq 0,j_{m-1}=0,\\\
\rho_{i_{n}j_{m}}\sigma_{i_{n}}\sigma_{j_{m}}\int_{0}^{l\wedge
t}f_{n-1,m-1}(s,s)\mathrm{d}s&\\\
~{}~{}~{}+\frac{1}{2}\rho_{i_{n-1}i_{n}}\sigma_{i_{n-1}}\sigma_{i_{n}}\int_{0}^{l}g_{n-2,m}(s,t)\mathrm{d}s&\\\
~{}~{}~{}+\frac{1}{2}\rho_{j_{m-1}j_{m}}\sigma_{j_{m-1}}\sigma_{j_{m}}\int_{0}^{t}\tilde{g}_{m-2,n}(s,l)\mathrm{d}s&\\\
~{}~{}~{}+\frac{1}{4}\rho_{i_{n-1}i_{n}}\sigma_{i_{n-1}}\sigma_{i_{n}}\rho_{j_{m-1}j_{m}}\sigma_{j_{m-1}}\sigma_{j_{m}}&\\\
~{}~{}~{}~{}~{}~{}~{}~{}~{}~{}~{}~{}~{}~{}~{}~{}~{}~{}~{}~{}~{}~{}~{}~{}~{}~{}~{}~{}~{}~{}~{}~{}~{}\cdot\int_{0}^{l}\int_{0}^{t}f_{n-2,m-2}(s,\tau)\mathrm{d}\tau\mathrm{d}s,&\text{if
}i_{n}\neq 0,j_{m}\neq 0,i_{n-1}\neq 0,j_{m-1}\neq 0,\\\ \end{cases}$
$\displaystyle g_{n,m}(l,t)=$
$\displaystyle\begin{cases}\int_{0}^{l}g_{n-1,m}(s,t)\mathrm{d}s,&\text{if
}i_{n}=0,\\\ \rho_{i_{n}j_{m}}\sigma_{i_{n}}\sigma_{j_{m}}\int_{0}^{l\wedge
t}f_{n-1,m-1}(s,s)\mathrm{d}s+\frac{1}{2}\rho_{i_{n-1}i_{n}}\sigma_{i_{n-1}}\sigma_{i_{n}}\int_{0}^{l}g_{n-2,m}(s,t)\mathrm{d}s,&\text{if
}i_{n}\neq 0,\end{cases}$ $\displaystyle\tilde{g}_{m,n}(t,l)=$
$\displaystyle\begin{cases}\int_{0}^{t}\tilde{g}_{m-1,n}(s,l)\mathrm{d}s,&\text{if
}j_{m}=0,\\\ \rho_{i_{n}j_{m}}\sigma_{i_{n}}\sigma_{j_{m}}\int_{0}^{l\wedge
t}f_{n-1,m-1}(s,s)\mathrm{d}s+\frac{1}{2}\rho_{j_{m-1}j_{m}}\sigma_{j_{m-1}}\sigma_{j_{m}}\int_{0}^{t}\tilde{g}_{m-2,n}(s,l)\mathrm{d}s,&\text{if
}j_{m}\neq 0,\end{cases}$

with initial conditions

$\displaystyle f_{0,0}(l,t)$ $\displaystyle=1,$ $\displaystyle g_{0,m}(l,t)$
$\displaystyle=\begin{cases}\int_{0}^{t}f_{0,m-1}(l,s)\mathrm{d}s,&\text{if
}j_{m}=0,\\\ 0,&\text{if }j_{m}\neq 0,\end{cases}$
$\displaystyle\tilde{g}_{0,n}(l,t)$
$\displaystyle=\begin{cases}\int_{0}^{l}f_{n-1,0}(s,t)\mathrm{d}s,&\text{if
}i_{n}=0,\\\ 0,&\text{if }i_{n}\neq 0.\end{cases}$

The following example shows the correlation structures of Itô and Stratonovich
signatures for a (1+1)-dimensional time-augmented Brownian motion
$\hat{\textbf{X}}$.

######  Example A.6.

Consider a (1+1)-dimensional time-augmented Brownian motion
$\hat{\textbf{X}}$. Figures A.10 and A.11 show the correlation structures of
Itô and Stratonovich signatures of $\hat{\textbf{X}}$, respectively,
calculated through simulations. In particular, we first simulate 10,000 paths
of $\hat{\textbf{X}}$, then calculate the signatures of each path, and finally
calculate the sample correlation matrix of the signatures.

Figure A.10: Correlation matrix of Itô signatures with orders truncated to 4
for a (1+1)-dimensional time-augmented Brownian motion $\hat{\textbf{X}}$.
Figure A.11: Correlation matrix of Stratonovich signatures with orders
truncated to 4 for a (1+1)-dimensional time-augmented Brownian motion
$\hat{\textbf{X}}$.

From the figures, we can observe that the correlation matrix of the time-
augmented Brownian motion does not exhibit the same special structures (block
diagonal or odd–even alternating) we observe in the main paper. In addition,
the correlation between Stratonovich signatures is generally stronger than
that between Itô signatures.

By comparing Figures A.10 and A.11 with Figures A.2 and A.4 in the main paper,
we can find that the time augmentation generally increases the correlation
between signatures. Therefore, we expect that the time augmentation will lead
to a lower consistency rate for Lasso.

###  E.2 Consistency of signatures for time-augmented Brownian motion

We conduct simulations to study the consistency of signatures using Lasso for
the time-augmented Brownian motion.

Consider a (2+1)-dimensional time-augmented Brownian motion
$\hat{\textbf{X}}$, which is the time-augmented version of a 2-dimensional
Brownian motion X. The inter-dimensional correlation of X is $\rho$. We
perform the same experiment as conducted in Section 4.1 for
$\hat{\textbf{X}}$.

Figure A.12 shows the consistency rates for different values of inter-
dimensional correlation, $\rho$, and different numbers of true predictors,
$q$. The observations are similar to the results for Brownian motion without
time augmentation (Figure 1(a)). In addition, the consistency rates are
generally lower when there exists time augmentation (Figure A.12) compared to
the case without time augmentation (Figure 1(a)). This is consistent with
Example A.6 because time augmentation tends to increase the correlation
between signatures, which results in a lower consistency rate.

Figure A.12: Consistency rates for the time-augmented Brownian motion with
different values of inter-dimensional correlation, $\rho$, and different
numbers of true predictors, $q$. Solid lines correspond to Itô signatures and
dashed lines correspond to Stratonovich signatures.

In summary, our simulation shows that time augmentation lowers the consistency
rate of Lasso.

##  Appendix F Proofs

This appendix provides the proofs of all theoretical results in this paper.

###### Proof of Proposition 1.

For the expectation, we have

$\mathbb{E}\left[S(\mathbf{X})_{t}^{i_{1},\dots,i_{n},I}\right]=\mathbb{E}\left[\int_{0}^{t}S(\mathbf{X})_{s}^{i_{1},\dots,i_{n-1}}\mathrm{d}X_{s}^{i_{n}}\right]=0$
(A.11)

because the expectation of an Itô integral is zero.

Next we prove
$\mathbb{E}\left[S(\mathbf{X})_{t}^{i_{1},\dots,i_{n},I}S(\mathbf{X})_{t}^{j_{1},\dots,j_{m},I}\right]=0$
for $m\neq n$ by induction. Without loss of generality, we assume that $m>n$.
When $n=1$, for any $m>1$, we have

$\displaystyle\mathbb{E}\left[S(\mathbf{X})_{t}^{i_{1},I}S(\mathbf{X})_{t}^{j_{1},\dots,j_{m},I}\right]$
$\displaystyle=\mathbb{E}\left[\left(\int_{0}^{t}\mathrm{d}X_{s}^{i_{1}}\right)\left(\int_{0}^{t}S(\mathbf{X})_{s}^{j_{1},\dots,j_{m-1}}\mathrm{d}X_{s}^{j_{m}}\right)\right]$
$\displaystyle=\int_{0}^{t}\mathbb{E}\left[S(\mathbf{X})_{s}^{j_{1},\dots,j_{m-1},I}\right]\rho_{i_{1}j_{m}}\sigma_{i_{1}}\sigma_{j_{m}}\mathrm{d}s=0,$

where the second equality uses the Itô isometry and the third equality uses
Equation (A.11). Now assume that for $m>n$, we have
$\mathbb{E}\left[S(\mathbf{X})_{t}^{i_{1},\dots,i_{n},I}S(\mathbf{X})_{t}^{j_{1},\dots,j_{m},I}\right]=0$.
Then,

$\displaystyle\mathbb{E}\left[S(\mathbf{X})_{t}^{i_{1},\dots,i_{n+1},I}S(\mathbf{X})_{t}^{j_{1},\dots,j_{m+1},I}\right]$
$\displaystyle=$
$\displaystyle\mathbb{E}\left[\left(\int_{0}^{t}S(\mathbf{X})_{t}^{i_{1},\dots,i_{n},I}\mathrm{d}X_{s}^{i_{n+1}}\right)\left(\int_{0}^{t}S(\mathbf{X})_{s}^{j_{1},\dots,j_{m},I}\mathrm{d}X_{s}^{j_{m+1}}\right)\right]$
$\displaystyle=$
$\displaystyle\int_{0}^{t}\mathbb{E}\left[S(\mathbf{X})_{t}^{i_{1},\dots,i_{k},I}S(\mathbf{X})_{s}^{j_{1},\dots,j_{m},I}\right]\rho_{i_{n+1}j_{m+1}}\sigma_{i_{n+1}}\sigma_{j_{m+1}}\mathrm{d}s=0.$

This proves
$\mathbb{E}\left[S(\mathbf{X})_{t}^{i_{1},\dots,i_{n},I}S(\mathbf{X})_{t}^{j_{1},\dots,j_{m},I}\right]=0$.

We finally prove
$\mathbb{E}\left[S(\mathbf{X})_{t}^{i_{1},\dots,i_{n},I}S(\mathbf{X})_{t}^{j_{1},\dots,j_{n},I}\right]=\frac{t^{n}}{n!}\prod_{k=1}^{n}\rho_{i_{k}j_{k}}\sigma_{i_{k}}\sigma_{j_{k}}$
by induction. When $n=1$, we have

$\displaystyle\mathbb{E}\left[S(\mathbf{X})_{t}^{i_{1},I}S(\mathbf{X})_{t}^{j_{1},I}\right]$
$\displaystyle=$
$\displaystyle\mathbb{E}\left[\left(\int_{0}^{t}\mathrm{d}X_{s}^{i_{1}}\right)\left(\int_{0}^{t}\mathrm{d}X_{s}^{j_{1}}\right)\right]=\int_{0}^{t}\rho_{i_{1}j_{1}}\sigma_{i_{1}}\sigma_{j_{1}}\mathrm{d}s=t\rho_{i_{1}j_{1}}\sigma_{i_{1}}\sigma_{j_{1}}.$

Then, assume that
$\mathbb{E}\left[S(\mathbf{X})_{t}^{i_{1},\dots,i_{n},I}S(\mathbf{X})_{t}^{j_{1},\dots,j_{n},I}\right]=\frac{t^{n}}{n!}\prod_{k=1}^{n}\rho_{i_{k}j_{k}}\sigma_{i_{k}}\sigma_{j_{k}}$.
We have

$\displaystyle\mathbb{E}\left[S(\mathbf{X})_{t}^{i_{1},\dots,i_{n+1},I}S(\mathbf{X})_{t}^{j_{1},\dots,j_{n+1},I}\right]$
$\displaystyle=$
$\displaystyle\mathbb{E}\left[\left(\int_{0}^{t}S(\mathbf{X})_{s}^{i_{1},\dots,i_{n},I}\mathrm{d}X_{s}^{i_{n+1}}\right)\left(\int_{0}^{t}S(\mathbf{X})_{s}^{j_{1},\dots,j_{n},I}\mathrm{d}X_{s}^{j_{n+1}}\right)\right]$
$\displaystyle=$
$\displaystyle\int_{0}^{t}\mathbb{E}\left[S(\mathbf{X})_{s}^{i_{1},\dots,i_{n},I}S(\mathbf{X})_{s}^{j_{1},\dots,j_{n},I}\right]\rho_{i_{n+1}j_{n+1}}\sigma_{i_{n+1}}\sigma_{j_{n+1}}\mathrm{d}s$
$\displaystyle=$
$\displaystyle\int_{0}^{t}\left(\frac{s^{n}}{n!}\prod_{k=1}^{n}\rho_{i_{k}j_{k}}\sigma_{i_{k}}\sigma_{j_{k}}\right)\rho_{i_{n+1}j_{n+1}}\sigma_{i_{n+1}}\sigma_{j_{n+1}}\mathrm{d}s=\frac{t^{n+1}}{(n+1)!}\prod_{k=1}^{n+1}\rho_{i_{k}j_{k}}\sigma_{i_{k}}\sigma_{j_{k}}.$

Therefore,
$\mathbb{E}\left[S(\mathbf{X})_{t}^{i_{1},\dots,i_{n},I}S(\mathbf{X})_{t}^{j_{1},\dots,j_{n},I}\right]=\frac{t^{n}}{n!}\prod_{k=1}^{n}\rho_{i_{k}j_{k}}\sigma_{i_{k}}\sigma_{j_{k}}$.
This completes the proof. ∎

###### Proof of Proposition 2.

By Proposition 1, for any $n$, we have

$\frac{\mathbb{E}\left[S(\mathbf{X})_{t}^{i_{1},\dots,i_{n},I}S(\mathbf{X})_{t}^{j_{1},\dots,j_{n},I}\right]}{\sqrt{\mathbb{E}\left[S(\mathbf{X})_{t}^{i_{1},\dots,i_{n},I}\right]^{2}\mathbb{E}\left[S(\mathbf{X})_{t}^{j_{1},\dots,j_{n},I}\right]^{2}}}=\frac{\frac{t^{n}}{n!}\prod_{k=1}^{n}\rho_{i_{k}j_{k}}\sigma_{i_{k}}\sigma_{j_{k}}}{\sqrt{\frac{t^{n}}{n!}\prod_{k=1}^{n}\sigma_{i_{k}}\sigma_{i_{k}}\cdot\frac{t^{n}}{n!}\prod_{k=1}^{n}\sigma_{j_{k}}\sigma_{j_{k}}}}=\prod_{k=1}^{n}\rho_{i_{k}j_{k}},$

implying that

$\frac{\mathbb{E}\left[S(\mathbf{X})_{t}^{i_{1},I}S(\mathbf{X})_{t}^{j_{1},I}\right]}{\sqrt{\mathbb{E}\left[S(\mathbf{X})_{t}^{i_{1},I}\right]^{2}\mathbb{E}\left[S(\mathbf{X})_{t}^{j_{1},I}\right]^{2}}}=\rho_{i_{1}j_{1}}$

and

$\frac{\mathbb{E}\left[S(\mathbf{X})_{t}^{i_{1},\dots,i_{n},I}S(\mathbf{X})_{t}^{j_{1},\dots,j_{n},I}\right]}{\sqrt{\mathbb{E}\left[S(\mathbf{X})_{t}^{i_{1},\dots,i_{n},I}\right]^{2}\mathbb{E}\left[S(\mathbf{X})_{t}^{j_{1},\dots,j_{n},I}\right]^{2}}}=\rho_{i_{n}j_{n}}\cdot\frac{\mathbb{E}\left[S(\mathbf{X})_{t}^{i_{1},\dots,i_{n-1},I}S(\mathbf{X})_{t}^{j_{1},\dots,j_{n-1},I}\right]}{\sqrt{\mathbb{E}\left[S(\mathbf{X})_{t}^{i_{1},\dots,i_{n-1},I}\right]^{2}\mathbb{E}\left[S(\mathbf{X})_{t}^{j_{1},\dots,j_{n-1},I}\right]^{2}}}.$

This proves the Kronecker product structure given by Equation (8).

Proposition 1 also implies that, for any $m\neq n$,

$\frac{\mathbb{E}\left[S(\mathbf{X})_{t}^{i_{1},\dots,i_{n},I}S(\mathbf{X})_{t}^{j_{1},\dots,j_{m},I}\right]}{\sqrt{\mathbb{E}\left[S(\mathbf{X})_{t}^{i_{1},\dots,i_{n},I}\right]^{2}\mathbb{E}\left[S(\mathbf{X})_{t}^{j_{1},\dots,j_{m},I}\right]^{2}}}=0.$

This proves that Itô signatures with different orders are uncorrelated and,
therefore, the correlation matrix is block diagonal. This completes the proof.
∎

###### Proof of Proposition 3.

We omit the proof of Equations

$\mathbb{E}\left[S(\mathbf{X})_{t}^{i_{1},\dots,i_{2n-1},S}\right]=0$

and

$\mathbb{E}\left[S(\mathbf{X})_{t}^{i_{1},\dots,i_{2n},S}S(\mathbf{X})_{t}^{j_{1},\dots,j_{2m-1},S}\right]=0$

because they can be proven using a similar approach to the proof Proposition
5. Equation

$\mathbb{E}\left[S(\mathbf{X})_{t}^{i_{1},\dots,i_{2n},S}\right]=\frac{1}{2^{n}}\frac{t^{n}}{n!}\prod_{k=1}^{n}\rho_{i_{2k-1}i_{2k}}\prod_{k=1}^{2n}\sigma_{i_{k}}$

is a corollary of Proposition A.1. ∎

###### Proof of Proposition 4.

This is a direct corollary of Proposition 3. ∎

###### Proof of Proposition 5.

We only need to prove that, for an odd number $m$ and an even number $n$, we
have

$\mathbb{E}\left[S(\mathbf{X})_{t}^{i_{1},\dots,i_{m}}S(\mathbf{X})_{t}^{j_{1},\dots,j_{n}}\right]=0$

for any $i_{1},\dots,i_{m}$ and $j_{1},\dots,j_{n}$ taking values in
$\\{1,2,\dots,d\\}$. Here the signatures can be defined in the sense of either
Itô or Stratonovich.

Consider the reflected OU process, $\tilde{\mathbf{X}}_{t}=-\mathbf{X}_{t}$.
By definition, $\tilde{\mathbf{X}}_{t}$ is also an OU process with the same
mean reversion parameter. Therefore, the signatures of
$\tilde{\mathbf{X}}_{t}$ and ${\mathbf{X}}_{t}$ should have the same
distribution. In particular, we have

$\mathbb{E}\left[S(\tilde{\mathbf{X}})_{t}^{i_{1},\dots,i_{m}}S(\tilde{\mathbf{X}})_{t}^{j_{1},\dots,j_{n}}\right]=\mathbb{E}\left[S(\mathbf{X})_{t}^{i_{1},\dots,i_{m}}S({\mathbf{X}})_{t}^{j_{1},\dots,j_{n}}\right].$
(A.12)

Now we consider the definition of the signatures:

$S(\mathbf{X})_{t}^{i_{1},\dots,i_{m}}=\int_{0<t_{1}<\dots<t_{m}<t}\mathrm{d}X_{t_{1}}^{i_{1}}\cdots\mathrm{d}X_{t_{k}}^{i_{m}},$

where the integral can be defined in the sense of either Itô or Stratonovich.
We therefore have

$\displaystyle
S(\tilde{\mathbf{X}})_{t}^{i_{1},\dots,i_{m}}=S(-{\mathbf{X}})_{t}^{i_{1},\dots,i_{m}}$
$\displaystyle=\int_{0<t_{1}<\dots<t_{m}<t}\mathrm{d}(-X_{t_{1}}^{i_{1}})\cdots\mathrm{d}(-X_{t_{k}}^{i_{m}})$
$\displaystyle=(-1)^{m}\int_{0<t_{1}<\dots<t_{m}<t}\mathrm{d}X_{t_{1}}^{i_{1}}\cdots\mathrm{d}X_{t_{k}}^{i_{m}}=(-1)^{m}S(\mathbf{X})_{t}^{i_{1},\dots,i_{m}}.$

Similarly, we have

$S(\tilde{\mathbf{X}})_{t}^{j_{1},\dots,j_{n}}=(-1)^{n}S(\mathbf{X})_{t}^{j_{1},\dots,j_{n}}.$

Therefore,

$\displaystyle\mathbb{E}\left[S(\tilde{\mathbf{X}})_{t}^{i_{1},\dots,i_{m}}S(\tilde{\mathbf{X}})_{t}^{j_{1},\dots,j_{n}}\right]$
$\displaystyle=(-1)^{m+n}\mathbb{E}\left[S(\mathbf{X})_{t}^{i_{1},\dots,i_{m}}S({\mathbf{X}})_{t}^{j_{1},\dots,j_{n}}\right]$
$\displaystyle=-\mathbb{E}\left[S(\mathbf{X})_{t}^{i_{1},\dots,i_{m}}S({\mathbf{X}})_{t}^{j_{1},\dots,j_{n}}\right],$

and combining this with Equation (A.12) leads to the result. ∎

###### Proof of Proposition 6.

This result holds because of Proposition 2 and [43, Corollary 5]. ∎

###### Proof of Proposition 7.

This result holds because of Proposition 2, [43, Corollary 2], and [43,
Corollary 5]. ∎

###### Proof of Proposition 8.

This result holds because of Proposition 4 and [43, Corollary 5]. ∎

###### Proof of Proposition A.1.

By the relationship between the Stratonovich integral and the Itô integral, we
have

$\displaystyle S(\mathbf{X})_{t}^{j_{1},\dots,j_{m},S}$
$\displaystyle=\int_{0}^{t}S(\mathbf{X})_{s}^{j_{1},\dots,j_{m-1},S}\circ\mathrm{d}X_{s}^{j_{m}}$
$\displaystyle=\int_{0}^{t}S(\mathbf{X})_{s}^{j_{1},\dots,j_{m-1},S}\mathrm{d}X_{s}^{j_{m}}+\frac{1}{2}\left[S(\mathbf{X})^{j_{1},\dots,j_{m-1},S},X^{j_{m}}\right]_{t},$

where $[A,B]_{t}$ represents the quadratic covariation between processes $A$
and $B$ from time 0 to $t$. Furthermore, by properties of the quadratic
covariation,

$\displaystyle\left[S(\mathbf{X})^{j_{1},\dots,j_{m-1},S},X^{j_{m}}\right]_{t}$
$\displaystyle=\int_{0}^{t}S(\mathbf{X})_{s}^{j_{1},\dots,j_{m-2},S}\mathrm{d}\left[X^{j_{m-1}},X^{j_{m}}\right]_{s}$
$\displaystyle=\rho_{j_{m-1}j_{m}}\sigma_{j_{m-1}}\sigma_{j_{m}}\int_{0}^{t}S(\mathbf{X})_{s}^{j_{1},\dots,j_{m-2},S}\mathrm{d}s.$

Therefore,

$S(\mathbf{X})_{t}^{j_{1},\dots,j_{m},S}=\int_{0}^{t}S(\mathbf{X})_{s}^{j_{1},\dots,j_{m-1},S}\mathrm{d}X_{s}^{j_{m}}+\frac{1}{2}\rho_{j_{m-1}j_{m}}\sigma_{j_{m-1}}\sigma_{j_{m}}\int_{0}^{t}S(\mathbf{X})_{s}^{j_{1},\dots,j_{m-2},S}\mathrm{d}s.$

For any $l,t\geq 0$ and $m,n=0,1,\dots$, define

$\displaystyle f_{n,m}(l,t)$
$\displaystyle:=\mathbb{E}\left[S(\mathbf{X})_{l}^{i_{1},\dots,i_{n},S}S(\mathbf{X})_{t}^{j_{1},\dots,j_{m},S}\right],$
$\displaystyle g_{n,m}(l,t)$
$\displaystyle:=\mathbb{E}\left[S(\mathbf{X})_{l}^{i_{1},\dots,i_{n},S}\int_{0}^{t}S(\mathbf{X})_{s}^{j_{1},\dots,j_{m-1},S}\mathrm{d}X_{s}^{j_{m}}\right].$

Then, by Fubini’s theorem,

$\displaystyle
f_{n,m}(l,t)=\mathbb{E}\left[S(\mathbf{X})^{i_{1},\dots,i_{n},S}_{l}S(\mathbf{X})^{j_{1},\dots,j_{m},S}_{t}\right]$
$\displaystyle=$
$\displaystyle\mathbb{E}\left[S(\mathbf{X})^{i_{1},\dots,i_{n},S}_{l}\left(\int_{0}^{t}S(\mathbf{X})_{s}^{j_{1},\dots,j_{m-1},S}\mathrm{d}X_{s}^{j_{m}}+\frac{1}{2}\rho_{j_{m-1}j_{m}}\sigma_{j_{m-1}}\sigma_{j_{m}}\int_{0}^{t}S(\mathbf{X})_{s}^{j_{1},\dots,j_{m-2},S}\mathrm{d}s\right)\right]$
$\displaystyle=$ $\displaystyle
g_{n,m}(l,t)+\frac{1}{2}\rho_{j_{m-1}j_{m}}\sigma_{j_{m-1}}\sigma_{j_{m}}\mathbb{E}\left[S(\mathbf{X})^{i_{1},\dots,i_{n},S}_{l}\int_{0}^{t}S(\mathbf{X})_{s}^{j_{1},\dots,j_{m-2},S}\mathrm{d}s\right]$
$\displaystyle=$ $\displaystyle
g_{n,m}(l,t)+\frac{1}{2}\rho_{j_{m-1}j_{m}}\sigma_{j_{m-1}}\sigma_{j_{m}}\int_{0}^{t}\mathbb{E}\left[S(\mathbf{X})^{i_{1},\dots,i_{n},S}_{l}S(\mathbf{X})_{s}^{j_{1},\dots,j_{m-2},S}\right]\mathrm{d}s$
$\displaystyle=$ $\displaystyle
g_{n,m}(l,t)+\frac{1}{2}\rho_{j_{m-1}j_{m}}\sigma_{j_{m-1}}\sigma_{j_{m}}\int_{0}^{t}f_{n,m-2}(l,s)\mathrm{d}s.$

This proves Equations (A.1) and (A.5). In addition, by Itô isometry and
Fubini’s theorem,

$\displaystyle
g_{n,m}(l,t)=\mathbb{E}\left[S(\mathbf{X})_{l}^{i_{1},\dots,i_{n},S}\int_{0}^{t}S(\mathbf{X})_{s}^{j_{1},\dots,j_{m-1},S}\mathrm{d}X_{s}^{j_{m}}\right]$
$\displaystyle=$
$\displaystyle\mathbb{E}\Bigg{[}\left(\int_{0}^{l}S(\mathbf{X})_{s}^{i_{1},\dots,i_{n-1},S}\mathrm{d}X_{s}^{i_{n}}+\frac{1}{2}\rho_{i_{n-1}i_{n}}\sigma_{i_{n-1}}\sigma_{i_{n}}\int_{0}^{l}S(\mathbf{X})_{s}^{i_{1},\dots,i_{n-2},S}\mathrm{d}s\right)$
$\displaystyle\qquad\qquad\qquad\qquad\qquad\qquad\qquad\qquad\qquad\qquad\qquad\qquad\qquad\cdot\int_{0}^{t}S(\mathbf{X})_{s}^{j_{1},\dots,j_{m-1},S}\mathrm{d}X_{s}^{j_{m}}\Bigg{]}$
$\displaystyle=$
$\displaystyle\mathbb{E}\left[\int_{0}^{l}S(\mathbf{X})_{s}^{i_{1},\dots,i_{n-1},S}\mathrm{d}X_{s}^{i_{n}}\int_{0}^{t}S(\mathbf{X})_{s}^{j_{1},\dots,j_{m-1},S}\mathrm{d}X_{s}^{j_{m}}\right]$
$\displaystyle\qquad\qquad+\frac{1}{2}\rho_{i_{n-1}i_{n}}\sigma_{i_{n-1}}\sigma_{i_{n}}\mathbb{E}\left[\int_{0}^{l}S(\mathbf{X})_{s}^{i_{1},\dots,i_{n-2},S}\mathrm{d}s\int_{0}^{t}S(\mathbf{X})_{s}^{j_{1},\dots,j_{m-1},S}\mathrm{d}X_{s}^{j_{m}}\right]$
$\displaystyle=$
$\displaystyle\rho_{i_{n}j_{m}}\sigma_{i_{n}}\sigma_{j_{m}}\int_{0}^{l\wedge
t}\mathbb{E}\left[S(\mathbf{X})_{s}^{i_{1},\dots,i_{n-1},S}S(\mathbf{X})_{s}^{j_{1},\dots,j_{m-1},S}\right]\mathrm{d}s$
$\displaystyle\qquad\qquad+\frac{1}{2}\rho_{i_{n-1}i_{n}}\sigma_{i_{n-1}}\sigma_{i_{n}}\int_{0}^{l}\mathbb{E}\left[S(\mathbf{X})_{s}^{i_{1},\dots,i_{n-2},S}\int_{0}^{t}S(\mathbf{X})_{u}^{j_{1},\dots,j_{m-1},S}\mathrm{d}X_{u}^{j_{m}}\right]\mathrm{d}s$
$\displaystyle=$
$\displaystyle\rho_{i_{n}j_{m}}\sigma_{i_{n}}\sigma_{j_{m}}\int_{0}^{l\wedge
t}f_{n-1,m-1}(s,s)\mathrm{d}s+\frac{1}{2}\rho_{i_{n-1}i_{n}}\sigma_{i_{n-1}}\sigma_{i_{n}}\int_{0}^{l}g_{n-2,m}(s,t)\mathrm{d}s.$

This proves Equations (A.2) and (A.6).

Now we prove the initial conditions. First, by the definition of 0-th order
signatures,
$f_{0,0}(l,t)=\mathbb{E}[S(\mathbf{X})_{l}^{0}S(\mathbf{X})_{t}^{0}]=1$, which
proves Equation (A.3). Second,

$g_{0,2m}(l,t)=\mathbb{E}\left[\int_{0}^{t}S(\mathbf{X})_{s}^{j_{1},\dots,j_{2m-1},S}\mathrm{d}X_{s}^{j_{2m}}\right]=0$

because the expectation of an Itô integral is zero, which proves Equation
(A.4). Third,

$\displaystyle f_{1,1}(l,t)$
$\displaystyle=\mathbb{E}\left[S(\mathbf{X})_{l}^{i_{1},S}S(\mathbf{X})_{t}^{j_{1},S}\right]=\mathbb{E}\left[\int_{0}^{l}1\circ\mathrm{d}X_{s}^{i_{1}}\int_{0}^{t}1\circ\mathrm{d}X_{s}^{j_{1}}\right]$
$\displaystyle=\mathbb{E}\left[X_{l}^{i_{1}}X_{t}^{j_{1}}\right]=\rho_{i_{1}j_{1}}\sigma_{i_{1}}\sigma_{j_{1}}(l\wedge
t),$

which proves Equation (A.7). Fourth, by Itô isometry,

$\displaystyle g_{1,2m-1}(l,t)$
$\displaystyle=\mathbb{E}\left[S(\mathbf{X})_{l}^{i_{1},S}\int_{0}^{t}S(\mathbf{X})_{s}^{j_{1},\dots,j_{2m-2},S}\mathrm{d}X_{s}^{j_{2m-1}}\right]$
$\displaystyle=\mathbb{E}\left[\int_{0}^{l}1\circ\mathrm{d}X_{s}^{i_{1}}\int_{0}^{t}S(\mathbf{X})_{s}^{j_{1},\dots,j_{2m-2},S}\mathrm{d}X_{s}^{j_{2m-1}}\right]$
$\displaystyle=\mathbb{E}\left[\int_{0}^{l}\mathrm{d}X_{s}^{i_{1}}\int_{0}^{t}S(\mathbf{X})_{s}^{j_{1},\dots,j_{2m-2},S}\mathrm{d}X_{s}^{j_{2m-1}}\right]$
$\displaystyle=\int_{0}^{l\wedge
t}\mathbb{E}\left[S(\mathbf{X})_{s}^{j_{1},\dots,j_{2m-2},S}\right]\rho_{i_{1}j_{2m-1}}\sigma_{i_{1}}\sigma_{j_{2m-1}}\mathrm{d}s$
$\displaystyle=\rho_{i_{1}j_{2m-1}}\sigma_{i_{1}}\sigma_{j_{2m-1}}\int_{0}^{l\wedge
t}f_{0,2m-2}(s,s)\mathrm{d}s.$

In addition, by using Equation (A.1) recursively, we can obtain that

$f_{0,2m-2}(s,s)=\frac{1}{2^{m-1}}\frac{s^{m-1}}{(m-1)!}\prod_{k=1}^{m-1}\rho_{j_{2k-1}j_{2k}}\prod_{k=1}^{2m-2}\sigma_{j_{k}}.$

Therefore,

$g_{1,2m-1}(l,t)=\rho_{i_{1}j_{2m-1}}\frac{1}{2^{m-1}}\frac{(l\wedge
t)^{m-1}}{(m-1)!}\sigma_{i_{1}}\prod_{k=1}^{2m-1}\sigma_{j_{k}}\prod_{k=1}^{m-1}\rho_{j_{2k-1}j_{2k}},$

which proves Equation (A.8). This completes the proof. ∎

###### Proof of Proposition A.2.

When $i_{n}=0$ and $j_{m}=0$, by Fubini’s theorem,

$\displaystyle f_{n,m}(l,t)$
$\displaystyle=\mathbb{E}\left[\int_{0}^{l}S(\hat{\mathbf{X}})_{s}^{i_{1},\dots,i_{n-1},I}\mathrm{d}s\int_{0}^{t}S(\hat{\mathbf{X}})_{\tau}^{j_{1},\dots,j_{m-1},I}\mathrm{d}\tau\right]$
$\displaystyle=\mathbb{E}\left[\int_{0}^{l}\int_{0}^{t}S(\hat{\mathbf{X}})_{s}^{i_{1},\dots,i_{n-1},I}S(\hat{\mathbf{X}})_{\tau}^{j_{1},\dots,j_{m-1},I}\mathrm{d}\tau\mathrm{d}s\right]=\int_{0}^{l}\int_{0}^{t}f_{n-1,m-1}(s,\tau)\mathrm{d}\tau\mathrm{d}s.$

When $i_{n}=0$ and $j_{m}\neq 0$, by Fubini’s theorem,

$\displaystyle f_{n,m}(l,t)$
$\displaystyle=\mathbb{E}\left[\int_{0}^{l}S(\hat{\mathbf{X}})_{s}^{i_{1},\dots,i_{n-1},I}\mathrm{d}s\cdot
S(\hat{\mathbf{X}})_{t}^{j_{1},\dots,j_{m},I}\right]$
$\displaystyle=\mathbb{E}\left[\int_{0}^{l}S(\hat{\mathbf{X}})_{s}^{i_{1},\dots,i_{n-1},I}S(\hat{\mathbf{X}})_{t}^{j_{1},\dots,j_{m},I}\mathrm{d}s\right]=\int_{0}^{l}f_{n-1,m}(s,t)\mathrm{d}s.$

When $i_{n}\neq 0$ and $j_{m}=0$, we can similarly obtain

$f_{n,m}(l,t)=\int_{0}^{t}f_{n,m-1}(l,s)\mathrm{d}s.$

When $i_{n}\neq 0$ and $j_{m}\neq 0$, by Itô isometry,

$\displaystyle f_{n,m}(l,t)$
$\displaystyle=\mathbb{E}\left[\int_{0}^{l}S(\hat{\mathbf{X}})_{s}^{i_{1},\dots,i_{n-1},I}\mathrm{d}X_{s}^{i_{n}}\int_{0}^{t}S(\hat{\mathbf{X}})_{s}^{j_{1},\dots,j_{m-1},I}\mathrm{d}X_{s}^{j_{m}}\right]$
$\displaystyle=\int_{0}^{l\wedge
t}\mathbb{E}\left[S(\hat{\mathbf{X}})_{s}^{i_{1},\dots,i_{n-1},I}S(\hat{\mathbf{X}})_{s}^{j_{1},\dots,j_{m-1},I}\right]\rho_{i_{n}j_{m}}\sigma_{i_{n}}\sigma_{j_{m}}\mathrm{d}s$
$\displaystyle=\rho_{i_{n}j_{m}}\sigma_{i_{n}}\sigma_{j_{m}}\int_{0}^{l\wedge
t}\int_{0}^{t}f_{n-1,m-1}(s,s)\mathrm{d}s.$

The initial conditions can be easily verified using Proposition 1. This
completes the proof. ∎

###### Proof of Proposition A.3.

We omit the proof because one can easily use approaches similar to the proofs
of Propositions 3 and A.2 to obtain the result by letting

$\displaystyle f_{n,m}(l,t)$
$\displaystyle:=\mathbb{E}\left[S(\hat{\mathbf{X}})_{l}^{i_{1},\dots,i_{n},S}S(\hat{\mathbf{X}})_{t}^{j_{1},\dots,j_{m},S}\right],$
$\displaystyle g_{n,m}(l,t)$
$\displaystyle:=\mathbb{E}\left[S(\hat{\mathbf{X}})_{l}^{i_{1},\dots,i_{n},S}\int_{0}^{t}S(\hat{\mathbf{X}})_{u}^{j_{1},\dots,j_{m-1},S}\mathrm{d}X^{j_{m}}_{u}\right],$
$\displaystyle\tilde{g}_{m,n}(t,l)$
$\displaystyle:=\mathbb{E}\left[S(\hat{\mathbf{X}})_{t}^{j_{1},\dots,j_{m},S}\int_{0}^{l}S(\hat{\mathbf{X}})_{u}^{i_{1},\dots,i_{n-1},S}\mathrm{d}X^{i_{n}}_{u}\right].$

∎

###### Proof of Example A.5.

The solution to stochastic differential equation (A.9) can be explicitly
expressed as

$Y_{t}=\int_{0}^{t}e^{-\kappa(t-s)}\mathrm{d}W_{s},\quad t\geq 0,$

where $W_{t}$ is a standard Brownian motion. Therefore, by Itô isometry,
$Y_{t}$ is a Gaussian random variable with mean 0 and variance

$\mathrm{Var}(Y_{t})=\mathbb{E}\left[Y_{t}^{2}\right]=\mathbb{E}\left[\int_{0}^{t}e^{-\kappa(t-s)}\mathrm{d}W_{s}\right]^{2}=\int_{0}^{t}\left[e^{-\kappa(t-s)}\right]^{2}\mathrm{d}s=\frac{1-e^{-2\kappa
t}}{2\kappa}.$

Now we calculate the correlation coefficient for its Itô and Stratonovich
signatures, respectively.

##### Itô signatures.

By the definition of signatures and Equation (A.9), we have

$\displaystyle\mathbb{E}\left[S(\mathbf{X})_{T}^{1,1,I}\right]=\mathbb{E}\left[\int_{0}^{T}Y_{t}\mathrm{d}Y_{t}\right]$
$\displaystyle=-\kappa\mathbb{E}\left[\int_{0}^{T}Y_{t}^{2}\mathrm{d}t\right]+\mathbb{E}\left[\int_{0}^{T}Y_{t}\mathrm{d}W_{t}\right]=-\kappa\int_{0}^{T}\mathbb{E}\left[Y_{t}^{2}\right]\mathrm{d}t$
$\displaystyle=-\kappa\int_{0}^{T}\frac{1-e^{-2\kappa
t}}{2\kappa}\mathrm{d}t=-\frac{T}{2}+\frac{1-e^{-2\kappa T}}{4\kappa}.$ (A.13)

For the second moment, by Itô isometry, we have

$\displaystyle\mathbb{E}\left[S(\mathbf{X})_{T}^{1,1,I}\right]^{2}$
$\displaystyle=\mathbb{E}\left[\int_{0}^{T}Y_{t}\mathrm{d}Y_{t}\right]^{2}=\mathbb{E}\left[-\kappa\int_{0}^{T}Y_{t}^{2}\mathrm{d}t+\int_{0}^{T}Y_{t}\mathrm{d}W_{t}\right]^{2}$
$\displaystyle=\kappa^{2}\int_{0}^{T}\int_{0}^{T}\mathbb{E}\left[Y_{t}^{2}Y_{s}^{2}\right]\mathrm{d}t\mathrm{d}s-2\kappa\mathbb{E}\left[\int_{0}^{T}Y_{t}^{2}\mathrm{d}t\int_{0}^{T}Y_{t}\mathrm{d}W_{t}\right]+\int_{0}^{T}\mathbb{E}\left[Y_{t}^{2}\right]\mathrm{d}t$
$\displaystyle=:\text{(a)}-\text{(b)}+\text{(c)}.$

It is easy to calculate Term (c):

$\text{(c)}=\int_{0}^{T}\mathbb{E}\left[Y_{t}^{2}\right]\mathrm{d}t=\int_{0}^{T}\frac{1-e^{-2\kappa
t}}{2\kappa}\mathrm{d}t=\frac{T}{2\kappa}+\frac{e^{-2\kappa
T}-1}{4\kappa^{2}}.$ (A.14)

To derive Term (a), we need to calculate
$\mathbb{E}\left[Y_{t}^{2}Y_{s}^{2}\right]$. Assume that $s<t$ and denote
$M_{t}=\int_{0}^{t}e^{\kappa u}\mathrm{d}W_{u}$, we have $Y_{t}=e^{-\kappa
t}M_{t}$, and therefore

$\displaystyle\mathbb{E}\left[Y_{t}^{2}Y_{s}^{2}\right]$
$\displaystyle=e^{-2\kappa(t+s)}\mathbb{E}\left[M_{t}^{2}M_{s}^{2}\right]=e^{-2\kappa(t+s)}\mathbb{E}\left[(M_{t}-M_{s}+M_{s})^{2}M_{s}^{2}\right]$
$\displaystyle=e^{-2\kappa(t+s)}\left[\mathbb{E}\left[(M_{t}-M_{s})^{2}M_{s}^{2}\right]+2\mathbb{E}\left[(M_{t}-M_{s})M_{s}^{3}\right]+\mathbb{E}\left[M_{s}^{4}\right]\right].$

Because $M_{t}-M_{s}=\int_{s}^{t}e^{\kappa u}\mathrm{d}W_{u}$ is a Gaussian
random variable with mean 0 and variance

$\mathrm{Var}(M_{t}-M_{s})=\mathbb{E}\left[(M_{t}-M_{s})^{2}\right]=\mathbb{E}\left[\int_{s}^{t}e^{\kappa
u}\mathrm{d}W_{u}\right]^{2}=\int_{s}^{t}\left[e^{\kappa
u}\right]^{2}\mathrm{d}u=\frac{e^{2\kappa t}-e^{2\kappa s}}{2\kappa},$

and $M_{t}$ has independent increments, we further have

$\displaystyle\mathbb{E}\left[Y_{t}^{2}Y_{s}^{2}\right]$
$\displaystyle=e^{-2\kappa(t+s)}\left[\mathbb{E}\left[(M_{t}-M_{s})^{2}\right]\mathbb{E}\left[M_{s}^{2}\right]+2\mathbb{E}\left[M_{t}-M_{s}\right]\mathbb{E}\left[M_{s}^{3}\right]+\mathbb{E}\left[M_{s}^{4}\right]\right]$
$\displaystyle=e^{-2\kappa(t+s)}\left[\frac{e^{2\kappa t}-e^{2\kappa
s}}{2\kappa}\cdot\frac{e^{2\kappa s}-1}{2\kappa}+0+3\left(\frac{e^{2\kappa
s}-1}{2\kappa}\right)^{2}\right]$ $\displaystyle=\frac{1+2e^{-2\kappa
t+2\kappa s}-e^{-2\kappa s}-5e^{-2\kappa t}+3e^{-2\kappa t-2\kappa
s}}{4\kappa^{2}}$

when $s<t$. One can similarly write the corresponding formula for the case of
$s>t$ and therefore obtain that

(a)
$\displaystyle=\kappa^{2}\int_{0}^{T}\int_{0}^{T}\mathbb{E}\left[Y_{t}^{2}Y_{s}^{2}\right]\mathrm{d}t\mathrm{d}s$
$\displaystyle=\frac{1}{4}\left(T^{2}+\frac{T}{\kappa}+\frac{10Te^{-2\kappa
T}}{2\kappa}+\frac{3e^{-4\kappa
T}}{4\kappa^{2}}-\frac{9}{4\kappa^{2}}+\frac{3e^{-2\kappa
T}}{2\kappa^{2}}\right).$

For Term (b), note that

$2\kappa\mathbb{E}\left[\int_{0}^{T}Y_{t}^{2}\mathrm{d}t\int_{0}^{T}Y_{t}\mathrm{d}W_{t}\right]=2\kappa\int_{0}^{T}\mathbb{E}\left[Y_{s}^{2}\int_{0}^{T}Y_{t}\mathrm{d}W_{t}\right]\mathrm{d}s,$

we therefore need to calculate
$f(s):=\mathbb{E}\left[Y_{s}^{2}\int_{0}^{T}Y_{t}\mathrm{d}W_{t}\right]$ for
$s<T$. To do this, by Itô’s Lemma, we have

$\mathrm{d}Y_{s}^{2}=2Y_{s}\mathrm{d}Y_{s}+\mathrm{d}[Y,Y]_{s}=-2\kappa
Y_{s}^{2}\mathrm{d}s+2Y_{s}\mathrm{d}W_{s}+\mathrm{d}s,$

which implies that

$Y_{s}^{2}=-2\kappa\int_{0}^{s}Y_{u}^{2}\mathrm{d}u+2\int_{0}^{s}Y_{u}\mathrm{d}W_{u}+\int_{0}^{s}\mathrm{d}u.$

Therefore, for $s<T$, with the help of Itô isometry and Equation (A.14), we
have

$\displaystyle f(s)$
$\displaystyle=\mathbb{E}\left[Y_{s}^{2}\int_{0}^{T}Y_{t}\mathrm{d}W_{t}\right]$
$\displaystyle=\mathbb{E}\left[\left(-2\kappa\int_{0}^{s}Y_{u}^{2}\mathrm{d}u+2\int_{0}^{s}Y_{u}\mathrm{d}W_{u}+\int_{0}^{s}\mathrm{d}u\right)\int_{0}^{T}Y_{t}\mathrm{d}W_{t}\right]$
$\displaystyle=-2\kappa\int_{0}^{s}\mathbb{E}\left(Y_{u}^{2}\int_{0}^{T}Y_{t}\mathrm{d}W_{t}\right)\mathrm{d}u+2\int_{0}^{s}\mathbb{E}\left[Y_{t}^{2}\right]\mathrm{d}t+0$
$\displaystyle=-2\kappa\int_{0}^{s}f(u)\mathrm{d}u+\frac{s}{\kappa}+\frac{e^{-2\kappa
s}-1}{2\kappa^{2}},$

and taking derivatives of both sides leads to

$\frac{\mathrm{d}f}{\mathrm{d}s}=-2\kappa
f(s)+\frac{1}{\kappa}-\frac{e^{-2\kappa s}}{\kappa}.$

By solving this ordinary differential equation with respect to $f$ with
initial condition $f(0)=0$, we obtain that

$f(s)=\frac{1}{2\kappa^{2}}-\frac{se^{-2\kappa s}}{\kappa}-\frac{e^{-2\kappa
s}}{2\kappa^{2}}.$

Therefore,

$\text{(b)}=2\kappa\int_{0}^{T}f(s)\mathrm{d}s=\frac{T}{\kappa}+\frac{Te^{-2\kappa
T}}{\kappa}+\frac{e^{-2\kappa T}-1}{\kappa^{2}}.$

Finally, we obtain that

$\mathbb{E}\left[S(\mathbf{X})_{T}^{1,1,I}\right]^{2}=\text{(a)}-\text{(b)}+\text{(c)}=\frac{Te^{-2\kappa
T}}{4\kappa}+\frac{3e^{-4\kappa T}}{16\kappa^{2}}-\frac{3e^{-2\kappa
T}}{8\kappa^{2}}-\frac{T}{4\kappa}+\frac{3}{16\kappa^{2}}+\frac{T^{2}}{4}.$
(A.15)

Therefore, for Itô signature, we have

$\displaystyle\frac{\mathbb{E}\left[S(\mathbf{X})_{T}^{0,I}S(\mathbf{X})_{T}^{1,1,I}\right]}{\sqrt{\mathbb{E}\left[S(\mathbf{X})_{T}^{0,I}\right]^{2}\mathbb{E}\left[S(\mathbf{X})_{T}^{1,1,I}\right]^{2}}}$
$\displaystyle=\frac{\mathbb{E}\left[S(\mathbf{X})_{T}^{1,1,I}\right]}{\sqrt{\mathbb{E}\left[S(\mathbf{X})_{T}^{1,1,I}\right]^{2}}}$
$\displaystyle=\frac{-2\kappa T-e^{-2\kappa T}+1}{\sqrt{4\kappa Te^{-2\kappa
T}+3e^{-4\kappa T}-6e^{-2\kappa T}-4\kappa T+3+4\kappa^{2}T^{2}}},$

where we use the fact that the 0-th order signature is defined as 1.

##### Stratonovich signatures.

The Stratonovich integral and the Itô integral are related by

$\int_{0}^{t}A_{s}\circ\mathrm{d}B_{s}=\int_{0}^{t}A_{s}\mathrm{d}B_{s}+\frac{1}{2}[A,B]_{t}.$

Therefore, for Stratonovich signatures, we have

$S(\mathbf{X})_{T}^{1,S}=\int_{0}^{T}1\circ\mathrm{d}Y_{t}=\int_{0}^{T}1\mathrm{d}Y_{t}+\frac{1}{2}[1,Y]_{T}=\int_{0}^{T}1\mathrm{d}Y_{t}=S(\mathbf{X})_{T}^{1,I}=Y_{T},$

and

$S(\mathbf{X})_{T}^{1,1,S}=\int_{0}^{T}S(\mathbf{X})_{T}^{1,S}\circ\mathrm{d}Y_{t}=\int_{0}^{T}Y_{t}\circ\mathrm{d}Y_{t}=\int_{0}^{T}Y_{t}\mathrm{d}Y_{t}+\frac{1}{2}[Y,Y]_{T}=S(\mathbf{X})_{T}^{1,1,I}+\frac{T}{2},$

where we use the fact that $[1,Y]_{T}=0$ and $[Y,Y]_{T}=T$. Now we can use our
results for the Itô signatures, Equations (A.13) and (A.15), to obtain that

$\mathbb{E}\left[S(\mathbf{X})_{T}^{1,1,S}\right]=\mathbb{E}\left[S(\mathbf{X})_{T}^{1,1,I}\right]+\frac{T}{2}=\frac{1-e^{-2\kappa
T}}{4\kappa},$

and

$\displaystyle\mathbb{E}\left[S(\mathbf{X})_{T}^{1,1,S}\right]^{2}$
$\displaystyle=\mathbb{E}\left[S(\mathbf{X})_{T}^{1,1,I}+\frac{T}{2}\right]^{2}$
$\displaystyle=\mathbb{E}\left[S(\mathbf{X})_{T}^{1,1,I}\right]^{2}+T\mathbb{E}\left[S(\mathbf{X})_{T}^{1,1,I}\right]+\frac{T^{2}}{4}=\frac{3(1-e^{-2\kappa
T})^{2}}{16\kappa^{2}}.$

Therefore, for the Stratonovich signature, we have

$\frac{\mathbb{E}\left[S(\mathbf{X})_{T}^{0,S}S(\mathbf{X})_{T}^{1,1,S}\right]}{\sqrt{\mathbb{E}\left[S(\mathbf{X})_{T}^{0,S}\right]^{2}\mathbb{E}\left[S(\mathbf{X})_{T}^{1,1,S}\right]^{2}}}=\frac{\sqrt{3}}{3}.$

This completes the proof. ∎