Let $T_{ij}$ be a strictly decreasing, continuous transformation for a fixed
$i\in\\{1,\dots,k\\}$ and a fixed $j\in\\{1,\dots,d_{i}\\}$. Then

$\mathcal{D}\big{(}\mathbf{X}_{1},\dots,T_{i}(\mathbf{X}_{i}),\dots,\mathbf{X}_{k}\big{)}=\mathcal{D}(\mathbf{X}_{1},\dots,\mathbf{X}_{k}),$

where $T_{i}(\mathbf{X}_{i})=(X_{i1},\dots,T_{ij}(X_{ij}),\dots,X_{id_{i}})$.

  8. (A8)

Let $(\mathbf{X}_{n})_{n\in\mathbb{N}}$ be a sequence of $q$-dimensional
random vectors with corresponding copulas $(C_{n})_{n\in\mathbb{N}}$, then

$\lim_{n\to\infty}\mathcal{D}^{d_{1},\dots,d_{k}}(\mathbf{X}_{n})=\mathcal{D}^{d_{1},\dots,d_{k}}(\mathbf{X})$

if $C_{n}\to C$ uniformly, where $C$ denotes the copula of $\mathbf{X}$.

##  Appendix B Proofs of theoretical results of Section 2

###  B.1 Proof of Lemma 1

For (a), let $\mathbf{V}=(\mathbf{V}_{1},\dots,\mathbf{V}_{k})$ with
$\mathbf{V}_{i}=(V_{i1},\dots,V_{id_{i}})$ be a random vector with
distribution $\nu_{1}\times\cdots\times\nu_{k}$ and let
$\mathbf{U}=(\mathbf{U}_{1},\dots,\mathbf{U}_{k})$ with
$\mathbf{U}_{i}=(U_{i1},\dots,U_{id_{i}})$ be a random vector with
distribution $\mu_{C}$, as marginal distribution of an arbitrary coupling
$\gamma\in\Gamma(\mu_{C},\nu_{1}\times\cdots\times\nu_{k})$ of
$(\mathbf{U},\mathbf{V})$, and having marginals
$\mu_{C_{1}},\dots,\mu_{C_{k}}$ itself. Then, the distribution of
$(\mathbf{U}_{i},\mathbf{V}_{i})$ is a coupling of $\mu_{C_{i}}$ and $\nu_{i}$
for all $i=1,\dots,k$ such that

$\mathbb{E}\left(||\mathbf{U}-\mathbf{V}||^{2}\right)=\mathbb{E}\left(\sum_{i=1}^{k}\sum_{j=1}^{d_{i}}(U_{ij}-V_{ij})^{2}\right)=\sum_{i=1}^{k}\mathbb{E}\left(||\mathbf{U}_{i}-\mathbf{V}_{i}||^{2}\right)\geq\sum_{i=1}^{k}W_{2}^{2}(\mu_{C_{i}},\nu_{i}).$
(34)

Taking the infimum over all couplings
$\gamma\in\Gamma(\mu_{C},\nu_{1}\times\cdots\times\nu_{k})$ yields the result.
Part (b) follows immediately from the definition.

Regarding (c), if $\mu_{C_{i}}=\nu_{i}$ for all $i=1,\dots,k$, we have
$T_{d_{1},\dots,d_{k}}(\mu_{C};\nu_{1},\dots,\nu_{k})=W_{2}^{2}(\mu_{C},\mu_{C_{1}}\times\cdots\times\mu_{C_{k}})$,
making the statement trivial since $W_{2}$ defines a metric. Suppose now that
$\nu_{i}$ is absolutely continuous for all $i=1,\dots,k$ and
$T_{d_{1},\dots,d_{k}}(\mu_{C};\nu_{1},\dots,\nu_{k})=0$. The latter means
that, working further on the proof of (a), there exists a
$\gamma\in\Gamma(\mu_{C},\nu_{1}\times\cdots\times\nu_{k})$ minimizing the
left-hand side of (34), and with equality instead of inequality. Hence, the
$2$-Wasserstein distance between $\mu_{C_{i}}$ and $\nu_{i}$ is obtained at
the coupling distribution of $(\mathbf{U}_{i},\mathbf{V}_{i})$ coming from
$\gamma$, for all $i=1,\dots,k$. However, Brenier’s theorem (see, e.g.,
Theorem 2.12 in [48]) tells us that, since $\nu_{i}$ is absolutely continuous,
this optimum is uniquely and deterministically attained, i.e., (denoting
$\nabla$ for the gradient) there must exist convex functions
$\psi_{i}:\mathbb{I}^{d_{i}}\rightarrow\mathbb{R}\cup\\{\infty\\}$ such that
$\mathbf{U}_{i}=\nabla\psi_{i}(\mathbf{V}_{i})$ almost surely for all
$i=1,\dots,k$. Since $\mathbf{V}_{1},\dots,\mathbf{V}_{k}$ are independent,
this implies that $\mathbf{U}_{1},\dots,\mathbf{U}_{k}$ are independent and
thus $\mu_{C}=\mu_{C_{1}}\times\cdots\times\mu_{C_{k}}$.

We next prove statement (d). The $W_{2}$-compactness of
$\Gamma(\mu_{C_{1}},\dots,\mu_{C_{k}})$ follows from the well-known result
that if $(\gamma_{n})_{n}\in\Gamma(\mu_{C_{1}},\dots,\mu_{C_{k}})$ is
$W_{2}$-convergent, then it is also weakly convergent (see, e.g., [10]) and
the limit will again be in $\Gamma(\mu_{C_{1}},\dots,\mu_{C_{k}})$ as the
marginals remain fixed. Finally, let us consider a fixed
$\pi\in\Gamma(\mu_{C_{1}},\dots,\mu_{C_{k}})$ and arbitrary $\epsilon>0$. Take
an arbitrary $\widetilde{\pi}\in\Gamma(\mu_{C_{1}},\dots,\mu_{C_{k}})$ such
that $W_{2}(\pi,\widetilde{\pi})<\delta$, where we pick $\delta>0$ such that
$\delta<\min\\{1,\epsilon/(1+2D_{\pi})\\}$ with
$D_{\pi}=W_{2}(\pi;\nu_{1}\times\cdots\times\nu_{k})$. We also denote
$D_{\widetilde{\pi}}=W_{2}(\widetilde{\pi};\nu_{1}\times\cdots\times\nu_{k})$.
It holds that

$\begin{split}|T_{d_{1},\dots,d_{k}}(\pi;\nu_{1},\dots,\nu_{k})-T_{d_{1},\dots,d_{k}}(\widetilde{\pi};\nu_{1},\dots,\nu_{k})|=|D_{\pi}^{2}-D_{\widetilde{\pi}}^{2}|&=|D_{\widetilde{\pi}}-D_{\pi}|\cdot|D_{\widetilde{\pi}}-D_{\pi}+2D_{\pi}|\\\
&\leq|D_{\widetilde{\pi}}-D_{\pi}|\left(|D_{\widetilde{\pi}}-D_{\pi}|+2D_{\pi}\right)\\\
&<\epsilon,\end{split}$

where we used that $|D_{\widetilde{\pi}}-D_{\pi}|\leq
W_{2}(\pi,\widetilde{\pi})$ because of the triangle inequality (recall that
$W_{2}$ defines a metric). This finishes the proof of statement (d). ∎

###  B.2 Proof of Proposition 1

We start with proving (A4) for $T_{d_{1},\dots,d_{k}}$. Suppose we consider an
additional random vector $\mathbf{X}_{k+1}$ having copula measure
$\mu_{C_{k+1}}$, an additional absolutely continuous reference measure
$\nu_{k+1}$, and let $\mu_{\widetilde{C}}$ be the copula measure of
$(\mathbf{X}_{1},\dots,\mathbf{X}_{k+1})$. If we first assume that
$\mathbf{X}_{k+1}$ is independent of $(\mathbf{X}_{1},\dots,\mathbf{X}_{k})$,
we have $\mu_{\widetilde{C}}=\mu_{C}\times\mu_{C_{k+1}}$ and hence

$\begin{split}T_{d_{1},\dots,d_{k+1}}(\mu_{\widetilde{C}};\nu_{1},\dots,\nu_{k+1})&=W_{2}^{2}(\mu_{C}\times\mu_{C_{k+1}},\nu_{1}\times\cdots\times\nu_{k+1})-\sum_{i=1}^{k+1}W_{2}^{2}(\mu_{C_{i}},\nu_{i})\\\
&=W_{2}^{2}(\mu_{C},\nu_{1}\times\cdots\times\nu_{k})+W_{2}^{2}(\mu_{C_{k+1}},\nu_{k+1})-\sum_{i=1}^{k+1}W_{2}^{2}(\mu_{C_{i}},\nu_{i})\\\
&=W_{2}^{2}(\mu_{C},\nu_{1}\times\cdots\times\nu_{k})-\sum_{i=1}^{k}W_{2}^{2}(\mu_{C_{i}},\nu_{i})\\\
&=T_{d_{1},\dots,d_{k}}(\mu_{C};\nu_{1},\dots,\nu_{k}),\end{split}$

such that the Wasserstein dependence measure remains unchanged when adding
$\mathbf{X}_{k+1}$ into consideration. In general, suppose that
$\gamma\in\Gamma(\mu_{\widetilde{C}},\nu_{1}\times\cdots\times\nu_{k+1})$ is
an optimal transport map from $\mu_{\widetilde{C}}$ to
$\nu_{1}\times\cdots\times\nu_{k+1}$, as joint distribution of
$(\widetilde{\mathbf{U}},\widetilde{\mathbf{V}})$ with
$\widetilde{\mathbf{U}}=(\mathbf{U},\mathbf{U}_{k+1})=(\mathbf{U}_{1},\dots,\mathbf{U}_{k+1})$
and
$\widetilde{\mathbf{V}}=(\mathbf{V},\mathbf{V}_{k+1})=(\mathbf{V}_{1},\dots,\mathbf{V}_{k+1})$,
i.e.,

$\begin{split}T_{d_{1},\dots,d_{k+1}}(\mu_{\widetilde{C}};\nu_{1},\dots,\nu_{k+1})&=\mathbb{E}\left(||\widetilde{\mathbf{U}}-\widetilde{\mathbf{V}}||^{2}\right)-\sum_{i=1}^{k+1}W_{2}^{2}(\mu_{C_{i}},\nu_{i})\\\
&=\mathbb{E}\left(||\mathbf{U}-\mathbf{V}||^{2}\right)+\mathbb{E}\left(||\mathbf{U}_{k+1}-\mathbf{V}_{k+1}||^{2}\right)-\sum_{i=1}^{k+1}W_{2}^{2}(\mu_{C_{i}},\nu_{i})\\\
&\geq
W_{2}^{2}(\mu_{C},\nu_{1}\times\cdots\times\nu_{k})-\sum_{i=1}^{k}W_{2}^{2}(\mu_{C_{i}},\nu_{i})\\\
&=T_{d_{1},\dots,d_{k}}(\mu_{C};\nu_{1},\dots,\nu_{k}),\end{split}$

where the inequality follows from the definition of the Wasserstein distance.
If this inequality is an equality, it must hold that

$\begin{split}W_{2}^{2}(\mu_{C},\nu_{1}\times\cdots\times\nu_{k})&=\mathbb{E}\left(||\mathbf{U}-\mathbf{V}||^{2}\right)+\mathbb{E}\left(||\mathbf{U}_{k+1}-\mathbf{V}_{k+1}||^{2}\right)-W_{2}^{2}(\mu_{C_{k+1}},\nu_{k+1})\geq\mathbb{E}\left(||\mathbf{U}-\mathbf{V}||^{2}\right),\end{split}$

from which

$\mathbb{E}\left(||\mathbf{U}-\mathbf{V}||^{2}\right)=W_{2}^{2}(\mu_{C},\nu_{1}\times\cdots\times\nu_{k})\hskip
5.69046pt\text{and}\hskip
5.69046pt\mathbb{E}\left(||\mathbf{U}_{k+1}-\mathbf{V}_{k+1}||^{2}\right)=W_{2}^{2}(\mu_{C_{k+1}},\nu_{k+1}).$

Because of the absolute continuity of the $\nu_{i}$, the latter implies (using
Brenier’s theorem) that there exists
$T_{k+1}:\mathbb{I}^{d_{k+1}}\rightarrow\mathbb{R}^{d_{k+1}}$ such that
$\mathbf{U}_{k+1}=T_{k+1}(\mathbf{V}_{k+1})$ almost surely, while the former
means that there exist $T_{i}:\mathbb{I}^{q}\rightarrow\mathbb{R}^{d_{i}}$
such that $\mathbf{U}_{i}=T_{i}(\mathbf{V}_{1},\dots,\mathbf{V}_{k})$ almost
surely for all $i=1,\dots,k$. Since $\mathbf{V}_{k+1}$ is independent of
$(\mathbf{V}_{1},\dots,\mathbf{V}_{k})$, this shows that $\mathbf{U}_{k+1}$ is
independent of $(\mathbf{U}_{1},\dots,\mathbf{U}_{k})$, i.e.,
$\mu_{\widetilde{C}}=\mu_{C}\times\mu_{C_{k+1}}$ and thus $\mathbf{X}_{k+1}$
independent from $(\mathbf{X}_{1},\dots,\mathbf{X}_{k})$.

The fact that $\mathcal{D}$ satisfies (A1)-(A3),(A5),(A6) follows from basic
properties of copulas and Lemma 1.

In the context of property (A7), assume without loss of generality that
$X_{11}$ gets transformed to $T_{11}(X_{11})$ for a strictly decreasing
transformation $T_{11}$ and let $\mu_{\widetilde{C}}$ be the copula
distribution of $(T_{1}(\mathbf{X}_{1}),\mathbf{X}_{2},\dots,\mathbf{X}_{k})$
with $T_{1}(\mathbf{X}_{1})=(T_{11}(X_{11}),X_{12},\dots,X_{1d_{1}})$. Then,
$\widetilde{C}(\mathbf{u}_{1},\dots,\mathbf{u}_{k})=C(\overline{\mathbf{u}}_{1},\mathbf{u}_{2},\dots,\mathbf{u}_{k})-C(\widetilde{\mathbf{u}}_{1},\mathbf{u}_{2},\dots,\mathbf{u}_{k})$
with $\overline{\mathbf{u}}_{1}=(1,u_{12},\dots,u_{1d_{1}})$ and
$\widetilde{\mathbf{u}}_{1}=(1-u_{11},u_{12},\dots,u_{1d_{1}})$. Suppose now
that $\gamma\in\Gamma(\mu_{C},\nu_{1}\times\cdots\times\nu_{k})$ is an optimal
transport map from $\mu_{C}$ to $\nu_{1}\times\cdots\times\nu_{k}$, as joint
distribution of $(\mathbf{U},\mathbf{V})$, i.e.,

$W_{2}^{2}(\mu_{C},\nu_{1}\times\cdots\times\nu_{k})=\mathbb{E}\left(||\mathbf{U}-\mathbf{V}||^{2}\right)=\int_{\mathbb{I}^{2q}}||\mathbf{u}-\mathbf{v}||^{2}d\gamma(\mathbf{u},\mathbf{v}).$

Consider then
$\widetilde{\gamma}(\mathbf{u},\mathbf{v})=\gamma(\overline{\mathbf{u}}_{1},\mathbf{u}_{2},\dots,\mathbf{u}_{k},\mathbf{v})-\gamma(\widetilde{\mathbf{u}}_{1},\mathbf{u}_{2},\dots,\mathbf{u}_{k},\mathbf{v})$,
which clearly is a coupling of $\mu_{\widetilde{C}}$ and
$\nu_{1}\times\cdots\times\nu_{k}$, as joint distribution of
$(\widetilde{\mathbf{U}},\mathbf{V})$ with
$\widetilde{\mathbf{U}}=(\widetilde{\mathbf{U}}_{1},\mathbf{U}_{2},\dots,\mathbf{U}_{k})$
and $\widetilde{\mathbf{U}}_{1}=(1-U_{11},U_{12},\dots,U_{1d_{1}})$. Moreover,
putting
$\widetilde{\mathbf{u}}=(\widetilde{\mathbf{u}}_{1},\mathbf{u}_{2},\dots,\mathbf{u}_{2})$,
we have

$\begin{split}W_{2}^{2}(\mu_{\widetilde{C}},\nu_{1}\times\cdots\times\nu_{k})\leq\mathbb{E}\left(||\widetilde{\mathbf{U}}-\mathbf{V}||^{2}\right)=\int_{\mathbb{I}^{2q}}||\widetilde{\mathbf{u}}-\mathbf{v}||^{2}d\widetilde{\gamma}(\mathbf{u},\mathbf{v})&=\int_{\mathbb{I}^{2q}}||\mathbf{u}-\mathbf{v}||^{2}d\gamma(\mathbf{u},\mathbf{v})=W_{2}^{2}(\mu_{C},\nu_{1}\times\cdots\times\nu_{k}),\end{split}$

by simply doing a substitution $t_{11}=1-u_{11}$. Following a reversed
reasoning, we also obtain
$W_{2}^{2}(\mu_{C},\nu_{1}\times\cdots\times\nu_{k})\leq
W_{2}^{2}(\mu_{\widetilde{C}},\nu_{1}\times\cdots\times\nu_{k})$ and hence
$W_{2}^{2}(\mu_{C},\nu_{1}\times\cdots\times\nu_{k})=W_{2}^{2}(\mu_{\widetilde{C}},\nu_{1}\times\cdots\times\nu_{k})$.
Similarly, we can show that
$W_{2}^{2}(\mu_{C_{1}},\nu_{1})=W_{2}^{2}(\mu_{\widetilde{C}_{1}},\nu_{1})$
where $\mu_{\widetilde{C}_{1}}$ is the copula distribution of
$T_{1}(\mathbf{X}_{1})$ and hence conclude (A7).

Finally, for (A8), we want that if $C_{n}\to C$ uniformly for $n\to\infty$, it
is true that

$|T_{d_{1},\dots,d_{k}}(\mu_{C_{n}};\nu_{1},\dots,\nu_{k})-T_{d_{1},\dots,d_{k}}(\mu_{C};\nu_{1},\dots,\nu_{k})|\to
0$ (35)

as $n\to\infty$. Note that if $C_{n}\to C$ uniformly, we also have
$\int_{\mathbb{I}^{q}}||\mathbf{u}||^{p}dC_{n}(\mathbf{u})\to\int_{\mathbb{I}^{q}}||\mathbf{u}||^{p}dC(\mathbf{u})$
as $n\to\infty$ for every $p>0$ (Helly-Bray theorem). Since convergence in
distribution together with convergence of the first two moments implies
$W_{2}$-convergence, equation (35) is proven by using similar arguments as for
(d) of Lemma 1. ∎

##  Appendix C Proofs of theoretical results of Section 3

###  C.1 Proof of Proposition 2

The first step of the proof consists of showing that the eigenvalues of
$\mathbf{R}_{m}$ are indeed given by (8). Therefore, already note that because
of the orthogonality of the matrix

$\mathbf{U}=\begin{pmatrix}\mathbf{U}_{11}&\mathbf{0}_{12}&\cdots&\mathbf{0}_{1k}\\\
\mathbf{0}_{12}^{\text{T}}&\mathbf{U}_{22}&\cdots&\mathbf{0}_{2k}\\\
\vdots&\vdots&\ddots&\vdots\\\
\mathbf{0}_{1k}^{\text{T}}&\mathbf{0}_{2k}^{\text{T}}&\cdots&\mathbf{U}_{kk}\end{pmatrix},$

denoting $\mathbf{0}_{ij}\in\mathbb{R}^{d_{i}\times d_{j}}$ for a matrix of
zeroes, the eigenvalues of $\mathbf{R}_{m}$ are the same as those of

$\boldsymbol{\Lambda}_{m}=\mathbf{U}^{\text{T}}\mathbf{R}_{m}\mathbf{U}=\begin{pmatrix}\vspace{0.1cm}\boldsymbol{\Lambda}_{11}&\boldsymbol{\Lambda}_{11}^{1/2}\boldsymbol{\Pi}_{12}\boldsymbol{\Lambda}_{22}^{1/2}&\cdots&\boldsymbol{\Lambda}_{11}^{1/2}\boldsymbol{\Pi}_{1k}\boldsymbol{\Lambda}_{kk}^{1/2}\\\
\boldsymbol{\Lambda}_{22}^{1/2}\boldsymbol{\Pi}_{12}^{\text{T}}\boldsymbol{\Lambda}_{11}^{1/2}&\boldsymbol{\Lambda}_{22}&\cdots&\boldsymbol{\Lambda}_{22}^{1/2}\boldsymbol{\Pi}_{2k}\boldsymbol{\Lambda}_{kk}^{1/2}\\\
\vdots&\vdots&\ddots&\vdots\\\
\boldsymbol{\Lambda}_{kk}^{1/2}\boldsymbol{\Pi}_{1k}^{\text{T}}\boldsymbol{\Lambda}_{11}^{1/2}&\boldsymbol{\Lambda}_{kk}^{1/2}\boldsymbol{\Pi}_{2k}^{\text{T}}\boldsymbol{\Lambda}_{22}^{1/2}&\cdots&\boldsymbol{\Lambda}_{kk}\end{pmatrix},$
(36)

which we can find explicitly. Let $\mathbf{e}_{r,\ell}$ be the $r$-th
canonical unit (column) vector in $\mathbb{R}^{\ell}$ and put $d_{0}=0$. For
ease of notation, we also write
$\boldsymbol{\Pi}_{r\ell}=\boldsymbol{\Pi}_{\ell r}^{\text{T}}$ if $r>\ell$.
We now list the eigenvalues and eigenvectors of the matrix
$\boldsymbol{\Lambda}_{m}$.

  * 1.

For all $i=0,\dots,k-2$ and all $j=d_{i}+1,\dots,d_{i+1}$, the vector

$\left(\mathbf{0}_{d_{1}}^{\text{T}},\dots,\mathbf{0}_{d_{i}}^{\text{T}},\lambda_{j,(i+1)(i+1)}^{1/2}\mathbf{e}_{j,d_{i+1}}^{\text{T}},\dots,\lambda_{j,kk}^{1/2}\mathbf{e}_{j,d_{k}}^{\text{T}}\right)^{\text{T}},$
(37)

denoting $\mathbf{0}_{d_{i}}\in\mathbb{R}^{d_{i}}$ for a column vector of
zeroes, is an eigenvector of $\boldsymbol{\Lambda}_{m}$ with eigenvalue

$\lambda_{j,(i+1)(i+1)}+\dots+\lambda_{j,kk}.$ (38)

Indeed, when fixing a certain $i\in\\{0,\dots,k-2\\}$ and
$j\in\\{d_{i}+1,\dots,d_{i+1}\\}$, the $r$-th block row of (36) for
$r\in\\{1,\dots,i\\}$, multiplied with (37) equals

$\begin{split}\boldsymbol{\Lambda}_{rr}\mathbf{0}_{d_{r}}+\sum_{\begin{subarray}{c}\ell=1\\\
\ell\neq
r\end{subarray}}^{i}\boldsymbol{\Lambda}_{rr}^{1/2}\boldsymbol{\Pi}_{r\ell}\boldsymbol{\Lambda}_{\ell\ell}^{1/2}\mathbf{0}_{d_{\ell}}+\sum_{\ell=i+1}^{k}\boldsymbol{\Lambda}_{rr}^{1/2}\boldsymbol{\Pi}_{r\ell}\boldsymbol{\Lambda}_{\ell\ell}^{1/2}\lambda_{j,\ell\ell}^{1/2}\mathbf{e}_{j,d_{\ell}}&=\sum_{\ell=i+1}^{k}\lambda_{j,\ell\ell}\boldsymbol{\Lambda}_{rr}^{1/2}\boldsymbol{\Pi}_{r\ell}\mathbf{e}_{j,d_{\ell}}=\mathbf{0}_{d_{r}},\end{split}$

since $\boldsymbol{\Pi}_{r\ell}\mathbf{e}_{j,d_{\ell}}=\mathbf{0}_{d_{r}}$, as
$j>d_{r}$. If for $r\in\\{i+1,\dots,k\\}$, we multiply the $r$-th block row of
(36) with (37), we obtain

$\begin{split}\sum_{\ell=1}^{i}\boldsymbol{\Lambda}_{rr}^{1/2}\boldsymbol{\Pi}_{r\ell}\boldsymbol{\Lambda}_{\ell\ell}^{1/2}\mathbf{0}_{d_{\ell}}+\boldsymbol{\Lambda}_{rr}\lambda_{j,rr}^{1/2}\mathbf{e}_{j,d_{r}}+\sum_{\begin{subarray}{c}\ell=i+1\\\
\ell\neq
r\end{subarray}}^{k}\boldsymbol{\Lambda}_{rr}^{1/2}\boldsymbol{\Pi}_{r\ell}\boldsymbol{\Lambda}_{\ell\ell}^{1/2}\lambda_{j,\ell\ell}^{1/2}\mathbf{e}_{j,d_{\ell}}&=\lambda_{j,rr}^{3/2}\mathbf{e}_{j,d_{r}}+\sum_{\begin{subarray}{c}\ell=i+1\\\
\ell\neq
r\end{subarray}}^{k}\lambda_{j,\ell\ell}\boldsymbol{\Lambda}_{rr}^{1/2}\boldsymbol{\Pi}_{r\ell}\mathbf{e}_{j,d_{\ell}}\\\
&=\lambda_{j,rr}^{3/2}\mathbf{e}_{j,d_{r}}+\lambda_{j,rr}^{1/2}\sum_{\begin{subarray}{c}\ell=i+1\\\
\ell\neq r\end{subarray}}^{k}\lambda_{j,\ell\ell}\hskip
0.28436pt\mathbf{e}_{j,d_{r}}\\\
&=\lambda_{j,rr}^{1/2}\left(\sum_{\ell=i+1}^{k}\lambda_{j,\ell\ell}\right)\mathbf{e}_{j,d_{r}},\end{split}$

since $\boldsymbol{\Pi}_{rl}\mathbf{e}_{j,d_{\ell}}=\mathbf{e}_{j,d_{r}}$, as
$j\leq d_{r}$. Hence, for every $r\in\\{1,\dots,k\\}$, the $r$-th block
element (in $\mathbb{R}^{d_{r}\times 1}$) of the matrix multiplication of
$\boldsymbol{\Lambda}_{m}$ with the vector (37), is equal to (38) multiplied
with the vector (37). This shows what was desired, and delivers
$d_{1}+(d_{2}-d_{1})+\dots+(d_{k-1}-d_{k-2})=d_{k-1}$ eigenvalues.

  * 1.

For all $j=d_{k-1}+1,\dots,d_{k}$, the vector

$\left(\mathbf{0}_{d_{1}}^{\text{T}},\dots,\mathbf{0}_{d_{k-1}}^{\text{T}},\mathbf{e}_{j,d_{k}}^{\text{T}}\right)^{\text{T}}$
(39)

is an eigenvector with eigenvalue

$\lambda_{j,kk}.$

Indeed, following the previous result, it is straightforward to see that for
$r\in\\{1,\dots,k-1\\}$, the multiplication of the $r$-th block row of
$\boldsymbol{\Lambda}_{m}$ with the vector (39) equals $\mathbf{0}_{d_{r}}$,
as $j>d_{r}$, while the last block row leads to
$\boldsymbol{\Lambda}_{kk}\mathbf{e}_{j,d_{k}}=\lambda_{j,kk}\mathbf{e}_{j,d_{k}}$.
This gives an additional amount of $(d_{k}-d_{k-1})$ eigenvalues, resulting in
an intermediate total of $d_{k-1}+(d_{k}-d_{k-1})=d_{k}$ eigenvalues.

  * 1.

For all $i=1,\dots,k-1$ and all $j=1,\dots,d_{i}$, the vector

$\left(\mathbf{0}_{d_{1}}^{\text{T}},\dots,\mathbf{0}_{d_{i-1}}^{\text{T}},\lambda_{j,(i+1)(i+1)}^{1/2}\mathbf{e}_{j,d_{i}}^{\text{T}},-\lambda_{j,ii}^{1/2}\mathbf{e}_{j,d_{i+1}}^{\text{T}},\mathbf{0}_{d_{i+2}}^{\text{T}},\dots,\mathbf{0}_{d_{k}}^{\text{T}}\right)$
(40)

is an eigenvector with eigenvalue $0$.

Indeed, when fixing a certain $i\in\\{1,\dots,k-1\\}$ and
$j\in\\{1,\dots,d_{i}\\}$, the $r$-th block row of (36) for
$r\in\\{1,\dots,i-1\\}$ multiplied with (40) equals

$\begin{split}\boldsymbol{\Lambda}_{rr}\mathbf{0}_{d_{r}}+\sum_{\begin{subarray}{c}\ell=1\\\
\ell\neq
r\end{subarray}}^{i-1}\boldsymbol{\Lambda}_{rr}^{1/2}\boldsymbol{\Pi}_{r\ell}\boldsymbol{\Lambda}_{\ell\ell}^{1/2}\mathbf{0}_{d_{\ell}}&+\boldsymbol{\Lambda}_{rr}^{1/2}\boldsymbol{\Pi}_{ri}\boldsymbol{\Lambda}_{ii}^{1/2}\lambda_{j,(i+1)(i+1)}^{1/2}\mathbf{e}_{j,d_{i}}-\boldsymbol{\Lambda}_{rr}^{1/2}\boldsymbol{\Pi}_{r(i+1)}\boldsymbol{\Lambda}_{(i+1)(i+1)}^{1/2}\lambda_{j,ii}^{1/2}\mathbf{e}_{j,d_{i+1}}+\sum_{\ell=i+2}^{k}\boldsymbol{\Lambda}_{rr}^{1/2}\boldsymbol{\Pi}_{rl}\boldsymbol{\Lambda}_{\ell\ell}^{1/2}\mathbf{0}_{d_{\ell}}\\\
&=\lambda_{j,(i+1)(i+1)}^{1/2}\lambda_{j,ii}^{1/2}\boldsymbol{\Lambda}_{rr}^{1/2}\boldsymbol{\Pi}_{ri}\mathbf{e}_{j,d_{i}}-\lambda_{j,ii}^{1/2}\lambda_{j,(i+1)(i+1)}^{1/2}\boldsymbol{\Lambda}_{rr}^{1/2}\boldsymbol{\Pi}_{r(i+1)}\mathbf{e}_{j,d_{i+1}}\\\
&=\mathbf{0}_{d_{r}},\end{split}$

since
$\boldsymbol{\Pi}_{ri}\mathbf{e}_{j,d_{i}}=\boldsymbol{\Pi}_{r(i+1)}\mathbf{e}_{j,d_{i+1}}$.
For $r=i$, we get

$\begin{split}\sum_{\ell=1}^{i-1}\boldsymbol{\Lambda}_{rr}^{1/2}\boldsymbol{\Pi}_{rl}\boldsymbol{\Lambda}_{\ell\ell}^{1/2}\mathbf{0}_{d_{\ell}}&+\lambda_{j,(i+1)(i+1)}^{1/2}\boldsymbol{\Lambda}_{ii}\mathbf{e}_{j,d_{i}}-\lambda_{j,ii}^{1/2}\boldsymbol{\Lambda}_{ii}^{1/2}\boldsymbol{\Pi}_{i(i+1)}\boldsymbol{\Lambda}_{(i+1)(i+1)}^{1/2}\mathbf{e}_{j,d_{i+1}}+\sum_{\ell=i+2}^{k}\boldsymbol{\Lambda}_{rr}^{1/2}\boldsymbol{\Pi}_{rl}\boldsymbol{\Lambda}_{\ell\ell}^{1/2}\mathbf{0}_{d_{\ell}}\\\
&=\lambda_{j,(i+1)(i+1)}^{1/2}\lambda_{j,ii}\mathbf{e}_{j,d_{i}}-\lambda_{j,ii}^{1/2}\lambda_{j,(i+1)(i+1)}^{1/2}\boldsymbol{\Lambda}_{ii}^{1/2}\mathbf{e}_{j,d_{i}}\\\
&=\lambda_{j,(i+1)(i+1)}^{1/2}\lambda_{j,ii}\mathbf{e}_{j,d_{i}}-\lambda_{j,ii}^{1/2}\lambda_{j,(i+1)(i+1)}^{1/2}\lambda_{j,ii}^{1/2}\mathbf{e}_{j,d_{i}}\\\
&=\mathbf{0}_{d_{i}},\end{split}$

since $\boldsymbol{\Pi}_{i(i+1)}\mathbf{e}_{j,d_{i+1}}=\mathbf{e}_{j,d_{i}}$,
as $j\leq d_{i}$. For $r=i+1$, we get

$\begin{split}\sum_{\ell=1}^{i-1}\boldsymbol{\Lambda}_{rr}^{1/2}\boldsymbol{\Pi}_{r\ell}\boldsymbol{\Lambda}_{\ell\ell}^{1/2}\mathbf{0}_{d_{\ell}}&+\lambda_{j,(i+1)(i+1)}^{1/2}\boldsymbol{\Lambda}_{(i+1)(i+1)}^{1/2}\boldsymbol{\Pi}_{(i+1)i}\boldsymbol{\Lambda}_{ii}^{1/2}\mathbf{e}_{j,d_{i}}-\lambda_{j,ii}^{1/2}\boldsymbol{\Lambda}_{(i+1)(i+1)}\mathbf{e}_{j,d_{i+1}}+\sum_{\ell=i+2}^{k}\boldsymbol{\Lambda}_{rr}^{1/2}\boldsymbol{\Pi}_{rl}\boldsymbol{\Lambda}_{\ell\ell}^{1/2}\mathbf{0}_{d_{\ell}}\\\
&=\lambda_{j,(i+1)(i+1)}^{1/2}\lambda_{j,ii}^{1/2}\boldsymbol{\Lambda}_{(i+1)(i+1)}^{1/2}\mathbf{e}_{j,d_{i+1}}-\lambda_{j,ii}^{1/2}\lambda_{j,(i+1)(i+1)}\mathbf{e}_{j,d_{i+1}}\\\
&=\lambda_{j,(i+1)(i+1)}^{1/2}\lambda_{j,ii}^{1/2}\lambda_{j,(i+1)(i+1)}^{1/2}\mathbf{e}_{j,d_{i+1}}-\lambda_{j,ii}^{1/2}\lambda_{j,(i+1)(i+1)}\mathbf{e}_{j,d_{i+1}}\\\
&=\mathbf{0}_{d_{i+1}},\end{split}$

since $\boldsymbol{\Pi}_{(i+1)i}\mathbf{e}_{j,d_{i}}=\mathbf{e}_{j,d_{i+1}}$,
as $j\leq d_{i}$. Finally, for $r\in\\{i+2,\dots,k\\}$, we obtain

$\begin{split}\sum_{\ell=1}^{i-1}\boldsymbol{\Lambda}_{rr}^{1/2}\boldsymbol{\Pi}_{rl}\boldsymbol{\Lambda}_{\ell\ell}^{1/2}\mathbf{0}_{d_{\ell}}&+\boldsymbol{\Lambda}_{rr}^{1/2}\boldsymbol{\Pi}_{ri}\boldsymbol{\Lambda}_{ii}^{1/2}\lambda_{j,(i+1)(i+1)}^{1/2}\mathbf{e}_{j,d_{i}}-\boldsymbol{\Lambda}_{rr}^{1/2}\boldsymbol{\Pi}_{r(i+1)}\boldsymbol{\Lambda}_{(i+1)(i+1)}^{1/2}\lambda_{j,ii}^{1/2}\mathbf{e}_{j,d_{i+1}}+\boldsymbol{\Lambda}_{rr}\mathbf{0}_{d_{r}}+\sum_{\begin{subarray}{c}\ell=i+2\\\
l\neq
r\end{subarray}}^{k}\boldsymbol{\Lambda}_{rr}^{1/2}\boldsymbol{\Pi}_{r\ell}\boldsymbol{\Lambda}_{\ell\ell}^{1/2}\mathbf{0}_{d_{\ell}}\\\
&=\lambda_{j,(i+1)(i+1)}^{1/2}\lambda_{j,ii}^{1/2}\boldsymbol{\Lambda}_{rr}^{1/2}\boldsymbol{\Pi}_{ri}\mathbf{e}_{j,d_{i}}-\lambda_{j,ii}^{1/2}\lambda_{j,(i+1)(i+1)}^{1/2}\boldsymbol{\Lambda}_{rr}^{1/2}\boldsymbol{\Pi}_{r(i+1)}\mathbf{e}_{j,d_{i+1}}\\\
&=\mathbf{0}_{d_{r}},\end{split}$

since
$\boldsymbol{\Pi}_{ri}\mathbf{e}_{j,d_{i}}=\boldsymbol{\Pi}_{r(i+1)}\mathbf{e}_{j,d_{i+1}}$,
and we come by $d_{1}+\dots+d_{k-1}$ eigenvalues, resulting in the desired
total of $d_{1}+\dots+d_{k}=q$ eigenvalues. These are indeed given by

$\boldsymbol{\lambda}(\mathbf{R}_{m})=(\lambda_{j,11}+\lambda_{j,22}+\dots+\lambda_{j,kk})_{j=1}^{q}.$

Next, we need to show that for an arbitrary
$\mathbf{A}\in\Gamma(\mathbf{R}_{11},\dots,\mathbf{R}_{kk})$ with eigenvalues
$\lambda_{1}\geq\lambda_{2}\geq\dots\geq\lambda_{q}$, it holds that

$\begin{split}\sum_{\ell=1}^{n}\lambda_{\ell}&\leq\sum_{\ell=1}^{w}d_{\ell}+\sum_{\ell=1}^{n}(\lambda_{\ell,(w+1)(w+1)}+\dots+\lambda_{\ell,kk})\hskip
5.69046pt\text{for all}\hskip
5.69046ptw=0,\dots,k-1,n=d_{w}+1,\dots,d_{w+1}.\end{split}$ (41)

For this, we first introduce some notation. We know that $\mathbf{A}$ must be
of the form

$\mathbf{A}=\begin{pmatrix}\mathbf{R}_{11}&\mathbf{Q}_{12}&\cdots&\mathbf{Q}_{1k}\\\
\mathbf{Q}_{12}^{\text{T}}&\mathbf{R}_{22}&\cdots&\mathbf{Q}_{2k}\\\
\vdots&\vdots&\ddots&\vdots\\\
\mathbf{Q}_{1k}^{\text{T}}&\mathbf{Q}_{2k}^{\text{T}}&\cdots&\mathbf{R}_{kk}\end{pmatrix}\in\mathbb{R}^{q\times
q},$

with $\mathbf{Q}_{ij}\in\mathbb{R}^{d_{i}\times d_{j}}$. Based on this, define
the matrices

$\mathbf{B}_{i}=\begin{pmatrix}\mathbf{R}_{(i+1)(i+1)}&\mathbf{Q}_{(i+1)(i+2)}&\cdots&\mathbf{Q}_{(i+1)k}\\\
\mathbf{Q}_{(i+1)(i+2)}^{\text{T}}&\mathbf{R}_{(i+2)(i+2)}&\cdots&\mathbf{Q}_{(i+2)k}\\\
\vdots&\vdots&\ddots&\vdots\\\
\mathbf{Q}_{(i+1)k}^{\text{T}}&\mathbf{Q}_{(i+2)k}^{\text{T}}&\cdots&\mathbf{R}_{kk}\end{pmatrix}\hskip
8.5359pt\text{and}\hskip
8.5359pt\mathbf{C}_{i}=\begin{pmatrix}\mathbf{R}_{ii}&\mathbf{Q}_{i}\\\
\mathbf{Q}_{i}^{\text{T}}&\mathbf{B}_{i}\end{pmatrix},$

where
$\mathbf{Q}_{i}=(\mathbf{Q}_{i(i+1)}\cdots\mathbf{Q}_{ik})\in\mathbb{R}^{d_{i}\times(d_{i+1}\cdots+d_{k})}$,
for all $i=1,\dots,k-2$. We will approach $\mathbf{C}_{i}$ as a block matrix
consisting of four blocks. Note that
$\mathbf{B}_{i}\in\mathbb{R}^{(d_{i+1}+\cdots+d_{k})\times(d_{i+1}+\cdots+d_{k})}$
and
$\mathbf{C}_{i}\in\mathbb{R}^{(d_{i}+\cdots+d_{k})\times(d_{i}+\cdots+d_{k})}$.
Further, put

$\begin{split}\gamma_{1,i}&\geq\gamma_{2,i}\geq\cdots\geq\gamma_{m_{i},i}\hskip
9.95863pt\text{the eigenvalues of}\hskip 5.69046pt\mathbf{C}_{i}\hskip
5.69046pt\text{for}\hskip 5.69046pti=1,\dots,k-2,m_{i}=d_{i}+\dots+d_{k}\\\
\lambda_{1,ii}&\geq\lambda_{2,ii}\geq\cdots\geq\lambda_{d_{i},ii}\hskip
7.11317pt\text{the eigenvalues of}\hskip 5.69046pt\mathbf{R}_{ii}\hskip
3.1298pt\text{for}\hskip 5.69046pti=1,\dots,k\\\
\mu_{1,i}&\geq\mu_{2,i}\geq\cdots\geq\mu_{t_{i},i}\hskip 12.80365pt\text{the
eigenvalues of}\hskip 5.69046pt\mathbf{B}_{i}\hskip 5.69046pt\text{for}\hskip
5.69046pti=1,\dots,k-2,t_{i}=d_{i+1}+\dots+d_{k}.\end{split}$

Then, by applying Theorem 1 in [46] to $\mathbf{C}_{i}$ for an arbitrary
$i\in\\{1,\dots,k-2\\}$ and also to $\mathbf{B}_{k-2}$, we can pick any
$\alpha_{i},\beta_{i}$ satisfying

$\begin{split}0&\leq\alpha_{i}\leq d_{i}\hskip 5.69046pt\text{for}\hskip
5.69046pti=1,\dots,k\\\ 0&\leq\beta_{i}\leq t_{i}\hskip
7.96674pt\text{for}\hskip 5.69046pti=1,\dots,k-2\end{split}$

and integers

$\begin{split}1&\leq r_{1,i}<\cdots<r_{\alpha_{i},i}\leq d_{i},\hskip
28.45274ptr_{\ell,i}=d_{i}-\alpha_{i}+\ell\hskip 5.69046pt\text{for}\hskip
5.69046pt\ell>\alpha_{i},\hskip 28.45274pt\text{for}\hskip
5.69046pti=1,\dots,k\\\ 1&\leq j_{1,i}<\cdots<j_{\beta_{i},i}\leq t_{i},\hskip
29.87547ptj_{\ell,i}=t_{i}-\beta_{i}+\ell\hskip 5.69046pt\text{for}\hskip
5.97527pt\ell>\beta_{i},\hskip 32.72049pt\text{for}\hskip
5.69046pti=1,\dots,k-2,\end{split}$

that, defined as such, will guarantee that

$\begin{split}\sum_{\ell=1}^{\alpha_{i}+\beta_{i}}\gamma_{r_{\ell,i}+j_{\ell,i}-\ell,i}&\leq\sum_{\ell=1}^{\alpha_{i}}\lambda_{r_{\ell,i},ii}+\sum_{\ell=1}^{\beta_{i}}\mu_{j_{\ell,i},i}\hskip
5.69046pt\text{for}\hskip 5.69046pti=1,\dots,k-2\\\
\sum_{\ell=1}^{\alpha_{k-1}+\alpha_{k}}\mu_{r_{\ell,k-1}+r_{\ell,k}-\ell,k-2}&\leq\sum_{\ell=1}^{\alpha_{k-1}}\lambda_{r_{\ell,k-1},(k-1)(k-1)}+\sum_{\ell=1}^{\alpha_{k}}\lambda_{r_{\ell,k},kk}.\end{split}$
(42)

Notice that (42) describes a total of $k-1$ inequalities. We will now prove
(41) by making an adequate choice for the $\alpha_{i},\beta_{i}$ and
corresponding indices $r_{\ell,i},j_{\ell,i}$. In particular, when considering
a fixed $w\in\\{0,\dots,k-1\\}$ and fixed $n\in\\{d_{w}+1,\dots,d_{w+1}\\}$,
take

$\begin{split}&\hskip 113.81102pt\alpha_{i}=d_{i},\hskip
2.84544ptr_{\ell,i}=\ell\hskip 5.69046pt\text{for}\hskip
5.69046pt\ell=1,\dots,d_{i},i=1,\dots,w\\\ &\hskip
116.65646pt\alpha_{i}=n,\hskip 2.84544ptr_{\ell,i}=\ell\hskip
5.69046pt\text{for}\hskip 5.69046pt\ell=1,\dots,n,i=w+1,\dots,k\\\
&\beta_{i}=t_{i}-t_{w}+(k-w)n,\hskip
2.84544ptj_{\ell,i}=\begin{cases}\ell&\mbox{if }\ell\in\\{1,\dots,n\\}\\\
t_{w}-(k-w)n+\ell,&\mbox{if }\ell\in\\{n+1,\dots,\beta_{i}\\}\end{cases}\hskip
5.69046pt\text{for}\hskip 5.69046pti=1,\dots,\min\\{k-2,w\\}\\\ &\hskip
39.83368pt\beta_{i}=(k-i)n,\hskip
2.84544ptj_{\ell,i}=\begin{cases}\ell&\mbox{if }\ell\in\\{1,\dots,n\\}\\\
t_{i}-(k-i)n+\ell,&\mbox{if }\ell\in\\{n+1,\dots,\beta_{i}\\}\end{cases}\hskip
5.69046pt\text{for}\hskip 5.69046pti=w+1,\dots,k-2.\end{split}$

Then, concerning the $k-2$’ th and $k-1$’ th inequality of (42), we see that
if $w=k-1$, we have

$\begin{split}\sum_{\ell=1}^{\beta_{k-2}}\mu_{j_{\ell,k-2},k-2}&=\sum_{\ell=1}^{n}\mu_{\ell,k-2}+\sum_{\ell=n+1}^{\beta_{k-2}}\mu_{d_{k}-n+\ell,k-2}\\\
&=\mu_{1,k-2}+\dots+\mu_{n,k-2}+\mu_{d_{k}+1,k-2}+\dots+\mu_{t_{k-2},k-2},\end{split}$
(43)

and,

$\begin{split}\sum_{\ell=1}^{\alpha_{k-1}+\alpha_{k}}\mu_{r_{\ell,k-1}+r_{\ell,k}-\ell,k-2}&=\sum_{\ell=1}^{d_{k-1}+n}\mu_{r_{\ell,k-1}+r_{\ell,k}-\ell,k-2}\\\
&=\sum_{\ell=1}^{d_{k-1}}\mu_{\ell,k-2}+\sum_{\ell=d_{k-1}+1}^{n}\mu_{\ell,k-2}+\sum_{\ell=n+1}^{n+d_{k-1}}\mu_{d_{k}-n+\ell,k-2}\\\
&=\mu_{1,k-2}+\dots+\mu_{n,k-2}+\mu_{d_{k}+1,k-2}+\dots+\mu_{d_{k-1}+d_{k},k-2}.\end{split}$
(44)

Since $t_{k-2}=d_{k-1}+d_{k}$, we see that (43) and (44) are equal. If on the
other hand $w\in\\{0,\dots,k-2\\}$, we get

$\begin{split}\sum_{\ell=1}^{\beta_{k-2}}\mu_{j_{\ell,k-2},k-2}&=\sum_{\ell=1}^{n}\mu_{\ell,k-2}+\sum_{\ell=n+1}^{\beta_{k-2}}\mu_{t_{k-2}-2n+\ell,k-2}\\\
&=\mu_{1,k-2}+\dots+\mu_{n,k-2}+\mu_{t_{k-2}-n+1,k-2}+\dots+\mu_{t_{k-2},k-2},\end{split}$
(45)

and,

$\begin{split}\sum_{\ell=1}^{\alpha_{k-1}+\alpha_{k}}\mu_{r_{\ell,k-1}+r_{\ell,k}-\ell,k-2}&=\sum_{\ell=1}^{2n}\mu_{r_{\ell,k-1}+r_{\ell,k}-\ell,k-2}\\\
&=\sum_{\ell=1}^{n}\mu_{\ell,k-2}+\sum_{\ell=n+1}^{2n}\mu_{d_{k-1}+d_{k}-2n+\ell,k-2}\\\
&=\mu_{1,k-2}+\dots+\mu_{n,k-2}+\mu_{d_{k-1}+d_{k}-n+1,k-2}+\dots+\mu_{d_{k-1}+d_{k},k-2}.\end{split}$
(46)

Again because $t_{k-2}=d_{k-1}+d_{k}$, we have equality between (45) and (46).
We thus have shown so far that the second term on the right side of the $k-2$’
th inequality of (42) equals the term on the left side of the $k-1$’ th
inequality. Now, concerning the $i$’ th and $i+1$’ th inequality for
$i\in\\{1,\dots,k-3\\}$, we must have either both
$i,i+1\in\\{w+1,\dots,k-2\\}$ or both $i,i+1\in\\{1,\dots,w\\}$, or
$i\in\\{1,\dots,w\\},i+1\in\\{w+1,\dots,k-2\\}$ meaning $i=w$. In the first
case, it holds that

$\displaystyle\sum_{\ell=1}^{\alpha_{i+1}+\beta_{i+1}}\gamma_{r_{\ell,i+1}+j_{\ell,i+1}-\ell,i+1}$
$\displaystyle=\sum_{\ell=1}^{n+\beta_{i+1}}\gamma_{r_{\ell,i+1}+j_{\ell,i+1}-\ell,i+1}$
$\displaystyle=\sum_{\ell=1}^{n}\gamma_{\ell,i+1}+\sum_{\ell=n+1}^{\beta_{i+1}}\gamma_{d_{i+1}-n+t_{i+1}-(k-i-1)n+\ell,i+1}+\sum_{\ell=\beta_{i+1}+1}^{n+\beta_{i+1}}\gamma_{d_{i+1}-n+t_{i+1}-(k-i-1)n+\ell,i+1}$
$\displaystyle=\gamma_{1,i+1}+\dots+\gamma_{n,i+1}+\gamma_{d_{i+1}+t_{i+1}-(k-i-1)n+1,i+1}+\dots+\gamma_{d_{i+1}+t_{i+1},i+1},$

and

$\begin{split}\sum_{\ell=1}^{\beta_{i}}\mu_{j_{\ell,i},i}&=\sum_{\ell=1}^{n}\mu_{\ell,i}+\sum_{\ell=n+1}^{\beta_{i}}\mu_{t_{i}-(k-i)n+\ell,i}=\mu_{1,i}+\dots+\mu_{n,i}+\mu_{t_{i}-(k-i)n+n+1,i}+\dots+\mu_{t_{i},i}.\end{split}$
(48)

Since $t_{i}=d_{i+1}+t_{i+1}$ and $\mathbf{C}_{i+1}=\mathbf{B}_{i}$ (by
construction, hence they also have the same eigenvalues), we see that (C.1) is
equal to (48). For the second case, i.e., both $i,i+1\in\\{1,\dots,w\\}$, we
have

$\displaystyle\sum_{\ell=1}^{\alpha_{i+1}+\beta_{i+1}}\gamma_{r_{\ell,i+1}+j_{\ell,i+1}-\ell,i+1}$
$\displaystyle=\sum_{\ell=1}^{d_{i+1}+\beta_{i+1}}\gamma_{r_{\ell,i+1}+j_{\ell,i+1}-\ell,i+1}$
$\displaystyle=\sum_{\ell=1}^{d_{i+1}}\gamma_{\ell,i+1}+\sum_{\ell=d_{i+1}+1}^{n}\gamma_{\ell,i+1}+\sum_{\ell=n+1}^{\beta_{i+1}}\gamma_{t_{w}-(k-w)n+\ell,i+1}+\sum_{\ell=\beta_{i+1}+1}^{d_{i+1}+\beta_{i+1}}\gamma_{t_{w}-(k-w)n+\ell,i+1}$
(49)
$\displaystyle=\gamma_{1,i+1}+\dots+\gamma_{n,i+1}+\gamma_{t_{w}-(k-w-1)n+1,i+1}+\dots+\gamma_{d_{i+1}+t_{i+1},i+1},$

and

$\begin{split}\sum_{\ell=1}^{\beta_{i}}\mu_{j_{\ell,i},i}&=\sum_{\ell=1}^{n}\mu_{\ell,i}+\sum_{\ell=n+1}^{\beta_{i}}\mu_{t_{w}-(k-w)n+\ell,i}=\mu_{1,i}+\dots+\mu_{n,i}+\mu_{t_{w}-(k-w-1)n+1,i}+\dots+\mu_{t_{i},i}.\end{split}$
(50)

For the same reasons as in the previous case, we clearly see that (49) and
(50) are the same expressions. Finally, if $i=w$,

$\displaystyle\sum_{\ell=1}^{\alpha_{i+1}+\beta_{i+1}}\gamma_{r_{\ell,i+1}+j_{\ell,i+1}-\ell,i+1}$
$\displaystyle=\sum_{\ell=1}^{n+\beta_{w+1}}\gamma_{r_{\ell,w+1}+j_{\ell,w+1}-\ell,w+1}$
(51)
$\displaystyle=\gamma_{1,w+1}+\dots+\gamma_{n,w+1}+\gamma_{d_{w+1}+t_{w+1}-(k-w-1)n+1,w+1}+\dots+\gamma_{d_{w+1}+t_{w+1},w+1},$

and,

$\begin{split}\sum_{\ell=1}^{\beta_{i}}\mu_{j_{\ell,i},i}&=\sum_{\ell=1}^{\beta_{w}}\mu_{j_{\ell,w},w}=\mu_{1,w}+\dots+\mu_{n,w}+\mu_{t_{w}-(k-w-1)n+1,w}+\dots+\mu_{t_{w},w}.\end{split}$
(52)

Since $t_{w}=d_{w+1}+t_{w+1}$ and $\mathbf{C}_{w+1}=\mathbf{B}_{w}$, we also
have equality between (51) and (52). All together, we have proven that the
second term on the right hand side of the $i$’ th inequality of (42) is equal
to the term on the left hand side of the $i+1$’ th inequality for all
$i=1,\dots,k-2$. Hence (42) can be reduced to

$\sum_{\ell=1}^{\alpha_{1}+\beta_{1}}\gamma_{r_{\ell,1}+j_{\ell,1}-\ell,1}\leq\sum_{\ell=1}^{w}d_{\ell}+\sum_{\ell=1}^{n}(\lambda_{\ell,(w+1)(w+1)}+\dots+\lambda_{\ell,kk}).$
(53)

Moreover, since by choice $r_{\ell,1}=\ell$ for all $\ell=1,\dots,n$ if $w=0$,
or $r_{\ell,1}=\ell$ for all $\ell=1,\dots,d_{1}$ and
$r_{\ell,1}=d_{1}-d_{1}+\ell=\ell$ for all $\ell>d_{1}$ if $w>0$, and
$j_{\ell,1}=\ell$ for all $\ell=1,\dots,n$, we definitely have

$\sum_{\ell=1}^{n}\gamma_{\ell,1}\leq\sum_{\ell=1}^{\alpha_{1}+\beta_{1}}\gamma_{r_{\ell,1}+j_{\ell,1}-\ell,1}.$
(54)

The inequality in (54) is an equality if there are $\alpha_{1}+\beta_{1}-n$
eigenvalues equal to zero. If for example $w=k-1$ and $n=d_{k}$, this means
that $d_{1}+t_{1}-t_{k-1}+n-n=q-d_{k}$ eigenvalues should be zero, which is
the case for the matrix $\mathbf{R}_{m}$. Combining (53) with (54) and
realizing that the eigenvalues $\gamma_{\ell,1}$ are the same as the
eigenvalues $\lambda_{\ell}$, since $\mathbf{A}=\mathbf{C}_{1}$, finishes the
proof. ∎

###  C.2 Proof of Proposition 3

Consider an arbitrary
$\mathbf{A}\in\Gamma(\mathbf{R}_{11},\dots,\mathbf{R}_{kk})$ having ordered
eigenvalues $\lambda_{1}\geq\dots\geq\lambda_{q}$. Then, by definition of the
Bures-Wasserstein distance (3),

$d_{W}^{2}(\mathbf{A},\mathbf{I}_{q})=q+\text{tr}(\mathbf{A})-2\text{tr}\left(\mathbf{A}^{1/2}\right)=q+\sum_{j=1}^{q}\lambda_{j}-2\sum_{j=1}^{q}\lambda_{j}^{1/2}.$
(55)

Notice that the function $\lambda\mapsto\lambda-2\lambda^{1/2}$ is convex for
$\lambda\in[0,\infty)$. By Proposition 2, we know that the eigenvalues of
$\mathbf{R}_{m}$ majorize those of $\mathbf{A}$. From Lemma 2, it follows that
(55) is maximal if $\mathbf{A}=\mathbf{R}_{m}$. Secondly, we have

$\mathbf{R}_{0}^{1/2}\mathbf{A}\mathbf{R}_{0}^{1/2}=\begin{pmatrix}\vspace{0.1cm}\mathbf{R}_{11}^{2}&\mathbf{R}_{11}^{1/2}\boldsymbol{\Psi}_{12}\mathbf{R}_{22}^{1/2}&\cdots&\mathbf{R}_{11}^{1/2}\boldsymbol{\Psi}_{1k}\mathbf{R}_{kk}^{1/2}\\\
\mathbf{R}_{22}^{1/2}\boldsymbol{\Psi}_{12}^{\text{T}}\mathbf{R}_{11}^{1/2}&\mathbf{R}_{22}^{2}&\cdots&\mathbf{R}_{22}^{1/2}\boldsymbol{\Psi}_{2k}\mathbf{R}_{kk}^{1/2}\\\
\vdots&\vdots&\ddots&\vdots\\\
\mathbf{R}_{kk}^{1/2}\boldsymbol{\Psi}_{1k}^{\text{T}}\mathbf{R}_{11}^{1/2}&\mathbf{R}_{kk}^{1/2}\boldsymbol{\Psi}_{2k}^{\text{T}}\mathbf{R}_{22}^{1/2}&\cdots&\mathbf{R}_{kk}^{2}\end{pmatrix}.$

Recall that
$\mathbf{R}^{\alpha}_{ii}=\mathbf{U}_{ii}\boldsymbol{\Lambda}_{ii}^{\alpha}\mathbf{U}_{ii}^{\text{T}}$
is the eigendecomposition of $\mathbf{R}_{ii}^{\alpha}$ for $\alpha>0$, and
for $\mathbf{R}_{m}$ we have
$\boldsymbol{\Psi}_{ij}=\mathbf{U}_{ii}\boldsymbol{\Lambda}_{ii}^{1/2}\boldsymbol{\Pi}_{ij}\boldsymbol{\Lambda}_{jj}^{1/2}\mathbf{U}_{jj}^{\text{T}}$.
Hence, we get

$\mathbf{R}_{ii}^{1/2}\boldsymbol{\Psi}_{ij}\mathbf{R}_{jj}^{1/2}=\left(\mathbf{U}_{ii}\boldsymbol{\Lambda}_{ii}^{1/2}\mathbf{U}_{ii}^{\text{T}}\right)\left(\mathbf{U}_{ii}\boldsymbol{\Lambda}_{ii}^{1/2}\boldsymbol{\Pi}_{ij}\boldsymbol{\Lambda}_{jj}^{1/2}\mathbf{U}_{jj}^{\text{T}}\right)\left(\mathbf{U}_{jj}\boldsymbol{\Lambda}_{jj}^{1/2}\mathbf{U}_{jj}^{\text{T}}\right)=\mathbf{U}_{ii}\boldsymbol{\Lambda}_{ii}\boldsymbol{\Pi}_{ij}\boldsymbol{\Lambda}_{jj}\mathbf{U}_{jj}^{\text{T}},$

which is of the same form as $\boldsymbol{\Psi}_{ij}$, but with
$\boldsymbol{\Lambda}_{ii}^{1/2}$ and $\boldsymbol{\Lambda}_{jj}^{1/2}$
replaced by $\boldsymbol{\Lambda}_{ii}$ and $\boldsymbol{\Lambda}_{jj}$.
Applying Proposition 2 with $\mathbf{R}_{ii}^{2}$ instead of $\mathbf{R}_{ii}$
for $i=1,\dots,k$, it follows that the eigenvalues of an arbitrary matrix in
$\Gamma(\mathbf{R}_{11}^{2},\dots,\mathbf{R}_{kk}^{2})$ are majorized by those
of the matrix $\mathbf{R}_{0}^{1/2}\mathbf{R}_{m}\mathbf{R}_{0}^{1/2}$. Hence,

$d_{W}^{2}(\mathbf{A},\mathbf{R}_{0})=2\text{tr}(\mathbf{A})-2\text{tr}\left\\{\left(\mathbf{R}_{0}^{1/2}\mathbf{A}\mathbf{R}_{0}^{1/2}\right)^{1/2}\right\\}=2\text{tr}(\mathbf{A})-2\sum_{j=1}^{q}\kappa_{j}^{1/2}$
(56)

with $\kappa_{1},\dots,\kappa_{q}$ the eigenvalues of
$\mathbf{R}_{0}^{1/2}\mathbf{A}\mathbf{R}_{0}^{1/2}$. The fact that (56) is
maximal if $\mathbf{A}=\mathbf{R}_{m}$ now follows from the convexity of the
function $\kappa\mapsto-\kappa^{1/2}$ on $[0,\infty)$ and Lemma 2. ∎

###  C.3 Additional lemmas and proof of Theorem 1

######  Lemma 3.

Under the conditions of Theorem 1, it holds that, for
$\mathbf{H}_{t}\in\mathbb{S}^{q}$ for $t>0$ and $\mathbf{H}\in\mathbb{S}^{q}$
such that $||\mathbf{H}_{t}-\mathbf{H}||_{\text{F}}\to 0$ as $t\to 0$,

$\lim_{t\,\underset{>}{\to}\,0}\frac{\text{tr}\left\\{(\mathbf{R}+t\mathbf{H}_{t})_{m}^{1/2}\right\\}-\text{tr}\left(\mathbf{R}_{m}^{1/2}\right)}{t}=\frac{1}{2}\text{tr}\left(\boldsymbol{\Upsilon}_{1}\mathbf{H}\right),$

with $(\mathbf{R}+t\mathbf{H}_{t})_{m}$ the matrix in (7) for $\mathbf{R}$
replaced by $\mathbf{R}+t\mathbf{H}_{t}$ and $\boldsymbol{\Upsilon}_{1}$
defined in (16).

###### Proof.

Define the map
$L:(\mathbb{S}^{q},||\cdot||_{\text{F}})\rightarrow(\mathbb{S}^{q},||\cdot||_{\text{F}}):\mathbf{A}\mapsto
L(\mathbf{A})$ via $L(\mathbf{A})$ being the diagonal matrix whose diagonal is
equal to the $q$ eigenvalues (counting multiplicities) of $\mathbf{A}$ in
decreasing order. Then, we have
$L(\mathbf{R}_{ii})=\boldsymbol{\Lambda}_{ii}=\text{diag}(\lambda_{1,ii},\dots,\lambda_{d_{i},ii})$
for $i=1,\dots,k$. Also define the map

$M:(\mathbb{S}^{d_{1}}_{>}\times\cdots\times\mathbb{S}^{d_{k}}_{>},||\cdot||_{\text{F}})\rightarrow(\mathbb{S}^{d_{1}}_{>}\times\cdots\times\mathbb{S}^{d_{k}}_{>},||\cdot||_{\text{F}}):(\mathbf{A}_{1},\dots,\mathbf{A}_{k})\mapsto\left(L(\mathbf{A}_{1}),\dots,L(\mathbf{A}_{k})\right),$

where the Frobenius norm is naturally defined for a certain
$(\mathbf{A}_{1},\dots,\mathbf{A}_{k})\in\mathbb{S}^{d_{1}}\times\cdots\times\mathbb{S}^{d_{k}}$
through
$||(\mathbf{A}_{1},\dots,\mathbf{A}_{k})||_{\text{F}}=(||\mathbf{A}_{1}||_{\text{F}}^{2}+\cdots+||\mathbf{A}_{k}||_{\text{F}}^{2})^{1/2}$.
Using Lemma 3 of [36], it is then quickly seen that the Fréchet derivative of
$M$ at $(\mathbf{R}_{11},\dots,\mathbf{R}_{kk})$ in the direction of a certain
$\mathbf{H}=(\mathbf{H}_{11},\dots,\mathbf{H}_{kk})\in\mathbb{S}^{d_{1}}_{>}\times\cdots\times\mathbb{S}^{d_{k}}_{>}$
is
$\boldsymbol{\Delta}(\mathbf{H})=(\mathbf{D}_{\mathbf{U}_{11}^{\text{T}}\mathbf{H}_{11}\mathbf{U}_{11}},\dots,\mathbf{D}_{\mathbf{U}_{kk}^{\text{T}}\mathbf{H}_{kk}\mathbf{U}_{kk}})$,
with $\mathbf{D}_{\mathbf{U}_{ii}^{\text{T}}\mathbf{H}_{ii}\mathbf{U}_{ii}}$
the diagonal matrix containing the diagonal of
$\mathbf{U}_{ii}^{\text{T}}\mathbf{H}_{ii}\mathbf{U}_{ii}$. Consider now
(recalling that $d_{1}\leq\cdots\leq d_{k}$)

$g:(\mathbb{S}^{d_{1}}_{>}\times\cdots\times\mathbb{S}^{d_{k}}_{>},||\cdot||_{\text{F}})\rightarrow(\mathbb{R},|\cdot|):(\mathbf{B}_{1},\dots,\mathbf{B}_{k})\mapsto\sum_{j=1}^{d_{k}}(\eta_{j,11}+\cdots+\eta_{j,kk})^{1/2},$

with $\eta_{j,ii}$ the eigenvalues of $\mathbf{B}_{i}$ in decreasing order for
$i=1,\dots,k$ and $j=1,\dots,d_{i}$, and putting $\eta_{j,ii}=0$ for
$j=d_{i}+1,\dots,d_{k}$. We see that
$\text{tr}(\mathbf{R}_{m}^{1/2})=g(L(\mathbf{R}_{11}),\dots,L(\mathbf{R}_{kk}))=g(\boldsymbol{\Lambda}_{11},\dots,\boldsymbol{\Lambda}_{kk})$,
also putting $\lambda_{j,ii}=0$ for $j=d_{i}+1,\dots,d_{k}$. The Fréchet
derivative of $g$ at
$(\boldsymbol{\Lambda}_{11},\dots,\boldsymbol{\Lambda}_{kk})$ in the direction
of $\boldsymbol{\Delta}(\mathbf{H})$ equals
$\Psi(\boldsymbol{\Delta}(\mathbf{H}))$, where
$\Psi:\mathbb{S}^{d_{1}}_{>}\times\cdots\times\mathbb{S}^{d_{k}}_{>}\rightarrow\mathbb{R}$
satisfies

$\lim_{||\boldsymbol{\Delta}(\mathbf{H})||_{\text{F}}\to
0}\frac{|g(\boldsymbol{\Lambda}_{11}+\mathbf{D}_{\mathbf{U}_{11}^{\text{T}}\mathbf{H}_{11}\mathbf{U}_{11}},\dots,\boldsymbol{\Lambda}_{kk}+\mathbf{D}_{\mathbf{U}_{kk}^{\text{T}}\mathbf{H}_{kk}\mathbf{U}_{kk}})-g(\boldsymbol{\Lambda}_{11},\dots,\boldsymbol{\Lambda}_{kk})-\Psi(\boldsymbol{\Delta}(\mathbf{H}))|}{||\boldsymbol{\Delta}(\mathbf{H})||_{\text{F}}}=0.$
(57)

Defining
$t_{j,ii}=(\mathbf{U}_{ii}^{\text{T}}\mathbf{H}_{ii}\mathbf{U}_{ii})_{jj}$ for
all $i=1,\dots,k$ and $j=1,\dots,d_{i}$, as well as $t_{j,ii}=0$ for
$j=d_{i}+1,\dots,d_{k}$, it is obvious that the eigenvalues of
$\boldsymbol{\Lambda}_{ii}+\mathbf{D}_{\mathbf{U}_{ii}^{\text{T}}\mathbf{H}_{ii}\mathbf{U}_{ii}}$
are $\lambda_{j,ii}+t_{j,ii}$. Moreover, since we assumed that the eigenvalues
$\lambda_{j,ii}$ are distinct over $j$, it will hold, for
$||\boldsymbol{\Delta}(\mathbf{H})||_{\text{F}}$ small enough, that
$\lambda_{j,ii}+t_{j,ii}$ are in decreasing order again over $j$. Hence, using
our definition of $g$, we see that (57) becomes

$\lim_{||\boldsymbol{\Delta}(\mathbf{H})||_{\text{F}}\to
0}\frac{\left|\sum_{j=1}^{d_{k}}\left\\{\sum_{i=1}^{k}(\lambda_{j,ii}+t_{j,ii})\right\\}^{1/2}-\sum_{j=1}^{d_{k}}\left(\sum_{i=1}^{k}\lambda_{j,ii}\right)^{1/2}-\Psi(\boldsymbol{\Delta}(\mathbf{H}))\right|}{||\boldsymbol{\Delta}(\mathbf{H})||_{\text{F}}}=0.$
(58)

From expression (58), we observe that $\Psi(\boldsymbol{\Delta}(\mathbf{H}))$
is nothing more than the total derivative of the function

$f:\mathbb{R}^{kd_{k}}\rightarrow\mathbb{R}:(x_{1,11},x_{1,22},\dots,x_{d_{k},kk})\mapsto\sum_{j=1}^{d_{k}}\left(\sum_{i=1}^{k}x_{j,ii}\right)^{1/2}$

at $(\lambda_{1,11},\lambda_{1,22},\dots,\lambda_{d_{k},kk})$ evaluated in
$(t_{1,11},t_{1,22},\dots,t_{d_{k},kk})$. Computing the Jacobian matrix of
$f$, we find

$\begin{split}\Psi(\boldsymbol{\Delta}(\mathbf{H}))&=\sum_{j=1}^{d_{k}}\left(\sum_{i=1}^{k}\frac{1}{2(\lambda_{j,11}+\cdots+\lambda_{j,kk})^{1/2}}t_{j,ii}\right)\\\
&=\sum_{i=1}^{k}\left(\sum_{j=1}^{d_{i}}\frac{1}{2(\lambda_{j,11}+\cdots+\lambda_{j,kk})^{1/2}}t_{j,ii}\right)\\\
&=\sum_{i=1}^{k}\left(\sum_{j=1}^{d_{i}}\frac{1}{2(\lambda_{j,11}+\cdots+\lambda_{j,kk})^{1/2}}\left(\mathbf{U}_{ii}^{\text{T}}\mathbf{H}_{ii}\mathbf{U}_{ii}\right)_{jj}\right).\end{split}$

By the chain rule, we have just shown that

$\begin{split}\lim_{t\,\underset{>}{\to}\,0}\frac{\text{tr}\left\\{(\mathbf{R}+t\mathbf{H}_{t})_{m}^{1/2}\right\\}-\text{tr}\left(\mathbf{R}_{m}^{1/2}\right)}{t}&=\lim_{t\,\underset{>}{\to}\,0}\frac{g(M(\mathbf{R}_{11}+t\mathbf{H}_{t,11},\dots,\mathbf{R}_{kk}+t\mathbf{H}_{t,kk}))-g(M(\mathbf{R}_{11},\dots,\mathbf{R}_{kk}))}{t}\\\
&=\sum_{i=1}^{k}\left(\sum_{j=1}^{d_{i}}\frac{1}{2(\lambda_{j,11}+\cdots+\lambda_{j,kk})^{1/2}}\left(\mathbf{U}_{ii}^{\text{T}}\mathbf{H}_{ii}\mathbf{U}_{ii}\right)_{jj}\right),\end{split}$

where $\mathbf{H}_{t,ii}$ is the $d_{i}\times d_{i}$ diagonal block of
$\mathbf{H}_{t}$. Using the notations (11), (12) and (13), we can further
simplify the above expression into

$\begin{split}\frac{1}{2}\sum_{i=1}^{k}\text{tr}\left(\boldsymbol{\Delta}_{i}\mathbf{U}_{ii}^{\text{T}}\mathbf{H}_{ii}\mathbf{U}_{ii}\right)&=\frac{1}{2}\sum_{i=1}^{k}\text{tr}\left(\mathbf{P}_{i}^{\text{T}}\mathbf{U}_{ii}\boldsymbol{\Delta}_{i}\mathbf{U}_{ii}^{\text{T}}\mathbf{P}_{i}\mathbf{H}\right)=\frac{1}{2}\text{tr}(\boldsymbol{\Upsilon}_{1}\mathbf{H}),\end{split}$

where we used that $\text{tr}(\mathbf{A}\hskip
1.13791pt\text{diag}(\mathbf{B}))=\text{tr}(\text{diag}(\mathbf{A})\mathbf{B})$
for square matrices $\mathbf{A},\mathbf{B}$, the cyclic permutation property
of the trace operator, the fact that
$\mathbf{H}_{ii}=\mathbf{P}_{i}\mathbf{H}\mathbf{P}_{i}^{\text{T}}$ and
finally the identity
$\sum_{i=1}^{k}\mathbf{P}_{i}^{\text{T}}\mathbf{A}_{i}\mathbf{P}_{i}=\text{diag}(\mathbf{A}_{1},\dots,\mathbf{A}_{k})$
for $(d_{i}\times d_{i})$ matrices $\mathbf{A}_{i}$. ∎

######  Lemma 4.

The Fréchet derivative of the map

$\eta:\mathbb{S}^{q}_{>}\rightarrow\mathbb{R}:\mathbf{R}\mapsto\text{tr}\left\\{\left(\mathbf{R}_{0}^{1/2}\mathbf{R}\mathbf{R}_{0}^{1/2}\right)^{1/2}\right\\}$

is given by

$\lim_{t\,\underset{>}{\to}\,0}\frac{\eta(\mathbf{R}+t\mathbf{H}_{t})-\eta(\mathbf{R})}{t}=\frac{1}{2}\text{tr}\left\\{\left(\mathbf{J}_{0}+\mathbf{J}^{-1}\right)\mathbf{H}\right\\},$

for $\mathbf{H}_{t},\mathbf{H}\in\mathbb{S}^{q}$ with
$||\mathbf{H}_{t}\to\mathbf{H}||_{\text{F}}\to 0$ as $t\to 0$ and $\mathbf{J}$
and $\mathbf{J}_{0}$ as in (14) and (15) respectively.

###### Proof.

From Lemma 5 in [36], the Fréchet derivative of the map

$\phi:\mathbb{S}^{q}_{>}\times\mathbb{S}^{q}_{>}\rightarrow\mathbb{R}:(\mathbf{A},\mathbf{B})\mapsto
2\text{tr}\left\\{\left(\mathbf{A}^{1/2}\mathbf{B}\mathbf{A}^{1/2}\right)^{1/2}\right\\}$

at $(\mathbf{A},\mathbf{B})\in\mathbb{S}^{q}_{>}\times\mathbb{S}^{q}_{>}$ in
the direction of
$(\mathbf{G},\mathbf{H})\in\mathbb{S}^{q}\times\mathbb{S}^{q}$ equals
$\text{tr}(\mathbf{J}\mathbf{G})+\text{tr}(\mathbf{J}^{-1}\mathbf{H})$ with

$\begin{split}\mathbf{J}&=\mathbf{A}^{-1/2}\left(\mathbf{A}^{1/2}\mathbf{B}\mathbf{A}^{1/2}\right)^{1/2}\mathbf{A}^{-1/2}=\mathbf{B}^{1/2}\left(\mathbf{B}^{1/2}\mathbf{A}\mathbf{B}^{1/2}\right)^{-1/2}\mathbf{B}^{1/2}\\\
\mathbf{J}^{-1}&=\mathbf{A}^{1/2}\left(\mathbf{A}^{1/2}\mathbf{B}\mathbf{A}^{1/2}\right)^{-1/2}\mathbf{A}^{1/2}=\mathbf{B}^{-1/2}\left(\mathbf{B}^{1/2}\mathbf{A}\mathbf{B}^{1/2}\right)^{1/2}\mathbf{B}^{-1/2}.\end{split}$

Applying this to $(\mathbf{A},\mathbf{B})=(\mathbf{R}_{0},\mathbf{R})$ and
$(\mathbf{G},\mathbf{H})=(\mathbf{H}_{0},\mathbf{H})$ with $\mathbf{H}_{0}$
having the same $d_{i}\times d_{i}$ diagonal blocks as $\mathbf{H}$, but zero
off-diagonal blocks , yields
$\frac{1}{2}(\text{tr}(\mathbf{J}\mathbf{H}_{0})+\text{tr}(\mathbf{J}^{-1}\mathbf{H}))$
as the Fréchet derivative of $\eta$ at $\mathbf{R}$ in the direction of
$\mathbf{H}$, with $\mathbf{J}$ given in (14). The result follows from
$\text{tr}(\mathbf{J}\mathbf{H}_{0})=\text{tr}(\mathbf{J}_{0}\mathbf{H})$,
with $\mathbf{J}_{0}$ given in (15). ∎

######  Lemma 5.

Under the conditions of Theorem 1, it holds that, for
$\mathbf{H}_{t}\in\mathbb{S}^{q}$ for $t>0$ and $\mathbf{H}\in\mathbb{S}^{q}$
such that $||\mathbf{H}_{t}-\mathbf{H}||_{\text{F}}\to 0$ as $t\to 0$,

$\lim_{t\,\underset{>}{\to}\,0}\frac{\text{tr}\left[\left\\{(\mathbf{R}+t\mathbf{H}_{t})_{0}^{1/2}(\mathbf{R}+t\mathbf{H}_{t})_{m}(\mathbf{R}+t\mathbf{H}_{t})_{0}^{1/2}\right\\}^{1/2}-\left(\mathbf{R}_{0}^{1/2}\mathbf{R}_{m}\mathbf{R}_{0}^{1/2}\right)^{1/2}\right]}{t}=\text{tr}(\boldsymbol{\Upsilon_{2}\mathbf{H}}),$

with $(\mathbf{R}+t\mathbf{H}_{t})_{0}$ having the same $d_{i}\times d_{i}$
diagonal blocks as $\mathbf{R}+t\mathbf{H}_{t}$ but zero off-diagonal blocks,
$(\mathbf{R}+t\mathbf{H}_{t})_{m}$ the matrix in (7) with $\mathbf{R}$
replaced by $\mathbf{R}+t\mathbf{H}_{t}$, and with $\boldsymbol{\Upsilon}_{2}$
defined in (17).

###### Proof.

Recall the proof of Lemma 3. Keep the same definition for the map $M$ and
define

$g:\left(\mathbb{S}^{d_{1}}_{>}\times\cdots\times\mathbb{S}^{d_{k}}_{>},||\cdot||_{\text{F}}\right)\rightarrow(\mathbb{R},|\cdot|):(\mathbf{B}_{1},\dots,\mathbf{B}_{k})\mapsto\sum_{j=1}^{d_{k}}\left(\eta_{j,11}^{2}+\cdots+\eta_{j,kk}^{2}\right)^{1/2},$

with again $\eta_{j,ii}$ the eigenvalues of $\mathbf{B}_{i}$ in decreasing
order for $i=1,\dots,k$ and $j=1,\dots,d_{i}$, and putting $\eta_{j,ii}=0$ if
$j=d_{i}+1,\dots,d_{k}$. Again applying the chain rule yields, in a very
similar way and using the same notations,

$\begin{split}\lim_{t\,\underset{>}{\to}\,0}\frac{g(M(\mathbf{R}_{11}+t\mathbf{H}_{t,11},\dots,\mathbf{R}_{kk}+t\mathbf{H}_{t,kk}))-g(M(\mathbf{R}_{11},\dots,\mathbf{R}_{kk}))}{t}&=\sum_{i=1}^{k}\left(\sum_{j=1}^{d_{i}}\frac{\lambda_{j,ii}}{\left(\lambda_{j,11}^{2}+\cdots+\lambda_{j,kk}^{2}\right)^{1/2}}\left(\mathbf{U}_{ii}^{\text{T}}\mathbf{H}_{ii}\mathbf{U}_{ii}\right)_{jj}\right)\\\
&=\sum_{i=1}^{k}\text{tr}\left(\widetilde{\boldsymbol{\Delta}}_{ii}\mathbf{U}_{ii}^{\text{T}}\mathbf{H}_{ii}\mathbf{U}_{ii}\right)=\text{tr}\left(\boldsymbol{\Upsilon}_{2}\mathbf{H}\right).\end{split}$

∎

Proof of Theorem 1

We start by showing the Fréchet differentiability of $\mathcal{D}_{1}$ and
$\mathcal{D}_{2}$. To this end, we prove that they are Hadamard differentiable
on $\mathbb{S}^{q}_{>}$ with

$\lim_{t\,\underset{>}{\to}\,0}\frac{\mathcal{D}_{r}(\mathbf{R}+t\mathbf{H}_{t})-\mathcal{D}_{r}(\mathbf{R})}{t}=\text{tr}(\mathbf{M}_{r}\mathbf{H}),$

for $r\in\\{1,2\\}$ and $\mathbf{H}_{t}\in\mathbb{S}^{q}$ for $t>0$ and
$\mathbf{H}\in\mathbb{S}^{q}$ such that
$||\mathbf{H}_{t}-\mathbf{H}||_{\text{F}}\to 0$ as $t\to 0$.

Differentiability of $\mathcal{D}_{1}$

Consider the function

$f:(0,\infty)^{k+2}\to\mathbb{R}:\left(\overline{x}_{1},\dots,\overline{x}_{k},\overline{y},\overline{z}\right)\mapsto\frac{\sum_{i=1}^{k}\overline{x}_{i}-\overline{y}}{\sum_{i=1}^{k}\overline{x}_{i}-\overline{z}}.$

It is then quickly seen that
$\mathcal{D}_{1}(\mathbf{R})=f(x_{1},\dots,x_{k},y,z)$ and
$\mathcal{D}_{1}(\mathbf{R}+t\mathbf{H}_{t})=f(x_{1}^{t},\dots,{x}_{k}^{t},y^{t},z^{t})$
with

$\begin{split}&\hskip 56.9055pty=\text{tr}\left(\mathbf{R}^{1/2}\right),\hskip
14.22636ptx_{i}=\text{tr}\left(\mathbf{R}_{ii}^{1/2}\right),\hskip
14.22636ptz=\text{tr}\left(\mathbf{R}_{m}^{1/2}\right),\\\
y^{t}&=\text{tr}\left\\{(\mathbf{R}+t\mathbf{H}_{t})^{1/2}\right\\},\hskip
14.22636ptx_{i}^{t}=\text{tr}\left\\{(\mathbf{R}+t\mathbf{H}_{t})_{ii}^{1/2}\right\\},\hskip
14.22636ptz^{t}=\text{tr}\left\\{(\mathbf{R}+t\mathbf{H}_{t})_{m}^{1/2}\right\\},\end{split}$

for $i=1,\dots,k$. Here, $(\mathbf{R}+t\mathbf{H}_{t})_{ii}$ is the
$d_{i}\times d_{i}$ diagonal block of $\mathbf{R}+t\mathbf{H}_{t}$ and
$(\mathbf{R}+t\mathbf{H}_{t})_{m}$ the matrix in (7) with $\mathbf{R}$
replaced by $\mathbf{R}+t\mathbf{H}_{t}$. Since the matrix $\mathbf{R}$ is
positive definite, there exists a unique square root matrix $\mathbf{R}^{1/2}$
having eigenvalues on the sector
$\\{z\in\mathbb{C}:-\pi/2<\text{arg}(z)<\pi/2\\}$ of the complex plane, where
$\text{arg}(z)$ denotes the argument of the complex number $z$. Moreover,
since the mapping $z\mapsto z^{1/2}$ is infinitely differentiable on this
sector, it follows from Theorem 3.8 of [25] that the Fréchet derivative of the
matrix function $\mathbf{R}\mapsto\mathbf{R}^{1/2}$ exists. Furthermore, Lemma
2.2 in [7] tells us that this derivative in the direction of $\mathbf{H}$ is
equal to the unique solution $\mathbf{Y}$ of the Sylvester equation

$\mathbf{R}^{1/2}\mathbf{Y}+\mathbf{Y}\mathbf{R}^{1/2}=\mathbf{H}.$ (59)

Hence, the Hadamard derivative in the direction of $\mathbf{H}$ also equals
$\mathbf{Y}$ and from (59) we obtain

$\text{tr}(\mathbf{Y})=\text{tr}\left(\mathbf{R}^{-1/2}\mathbf{H}\right)-\text{tr}\left(\mathbf{R}^{-1/2}\mathbf{Y}\mathbf{R}^{1/2}\right)=\text{tr}\left(\mathbf{R}^{-1/2}\mathbf{H}\right)-\text{tr}(\mathbf{Y}),$

i.e.,
$\text{tr}(\mathbf{Y})=\text{tr}\left(\mathbf{R}^{-1/2}\mathbf{H}\right)/2$.
This (and a similar reasoning with $\mathbf{R}_{ii}$ instead of $\mathbf{R}$)
shows that

$\lim_{t\,\underset{>}{\to}\,0}\frac{y^{t}-y}{t}=\frac{1}{2}\text{tr}\left(\mathbf{R}^{-1/2}\mathbf{H}\right)\hskip
5.69046pt\text{and}\hskip
5.69046pt\lim_{t\,\underset{>}{\to}\,0}\frac{x_{i}^{t}-x_{i}}{t}=\frac{1}{2}\text{tr}\left(\mathbf{R}_{ii}^{-1/2}\mathbf{H}_{ii}\right)$

for $i=1,\dots,k$, where we used the notation $\mathbf{H}_{ii}$ for the
$d_{i}\times d_{i}$ diagonal block of $\mathbf{H}$. Also, Lemma 3 guarantees
that

$\lim_{t\,\underset{>}{\to}\,0}\frac{z^{t}-z}{t}=\frac{1}{2}\text{tr}(\boldsymbol{\Upsilon}_{1}\mathbf{H}).$

Putting this together, we see that the Fréchet derivative of the map

$g:\mathbb{S}^{q}\rightarrow(0,\infty)^{k+2}:\mathbf{A}\mapsto\left(\text{tr}\left(\mathbf{A}_{11}^{1/2}\right),\dots,\text{tr}\left(\mathbf{A}_{kk}^{1/2}\right),\text{tr}\left(\mathbf{A}^{1/2}\right),\text{tr}\left(\mathbf{A}_{m}^{1/2}\right)\right)$

at $\mathbf{R}$ in the direction of $\mathbf{H}$ equals

$\left(\frac{1}{2}\text{tr}\left(\mathbf{R}_{11}^{-1/2}\mathbf{H}_{11}\right),\dots,\frac{1}{2}\text{tr}\left(\mathbf{R}_{kk}^{-1/2}\mathbf{H}_{kk}\right),\frac{1}{2}\text{tr}\left(\mathbf{R}^{-1/2}\mathbf{H}\right),\frac{1}{2}\text{tr}\left(\boldsymbol{\Upsilon}_{1}\mathbf{H}\right)\right),$

which we will call $\boldsymbol{\Delta}(\mathbf{H})$. Next, the Jacobian
matrix of the function $f$ is given by

$\begin{split}\mathbf{J}_{f}&=\begin{pmatrix}\frac{\partial
f}{\partial\overline{x}_{1}}&\cdots&\frac{\partial
f}{\partial\overline{x}_{k}}&\frac{\partial
f}{\partial\overline{y}}&\frac{\partial
f}{\partial\overline{z}}\end{pmatrix}=\begin{pmatrix}\frac{\overline{y}-\overline{z}}{(\overline{x}_{1}+\cdots+\overline{x}_{k}-\overline{z})^{2}}&\cdots&\frac{\overline{y}-\overline{z}}{(\overline{x}_{1}+\cdots+\overline{x}_{k}-\overline{z})^{2}}&\frac{-1}{\overline{x}_{1}+\cdots+\overline{x}_{k}-\overline{z}}&\frac{\overline{x}_{1}+\cdots\overline{x}_{k}-\overline{y}}{(\overline{x}_{1}+\cdots+\overline{x}_{k}-\overline{z})^{2}}\end{pmatrix},\end{split}$

such that the Fréchet derivative (being nothing more than a total derivative)
of the function $f$ at $g(\mathbf{R})$ in the direction of
$\boldsymbol{\Delta}(\mathbf{H})$ is equal to

$\begin{split}\mathbf{J}_{f}|_{g(\mathbf{R})}\boldsymbol{\Delta}(\mathbf{H})^{\text{T}}=\sum_{\ell=1}^{k}\left[\frac{\text{tr}\left(\mathbf{R}^{1/2}\right)-\text{tr}\left(\mathbf{R}_{m}^{1/2}\right)}{\left\\{\sum_{i=1}^{k}\text{tr}\left(\mathbf{R}_{ii}^{1/2}\right)-\text{tr}\left(\mathbf{R}_{m}^{1/2}\right)\right\\}^{2}}\frac{1}{2}\text{tr}\left(\mathbf{R}_{\ell\ell}^{-1/2}\mathbf{H}_{\ell\ell}\right)\right]&-\frac{1}{\sum_{i=1}^{k}\text{tr}\left(\mathbf{R}_{ii}^{1/2}\right)-\text{tr}\left(\mathbf{R}_{m}^{1/2}\right)}\frac{1}{2}\text{tr}\left(\mathbf{R}^{-1/2}\mathbf{H}\right)\\\
&+\frac{\sum_{i=1}^{k}\text{tr}\left(\mathbf{R}_{ii}^{1/2}\right)-\text{tr}\left(\mathbf{R}^{1/2}\right)}{\left\\{\sum_{i=1}^{k}\text{tr}\left(\mathbf{R}_{ii}^{1/2}\right)-\text{tr}\left(\mathbf{R}_{m}^{1/2}\right)\right\\}^{2}}\frac{1}{2}\text{tr}(\boldsymbol{\Upsilon}_{1}\mathbf{H})\\\
&=\frac{1}{2C_{1}}(1-\mathcal{D}_{1}(\mathbf{R}))\text{tr}\left(\mathbf{R}_{0}^{-1/2}\mathbf{H}\right)-\frac{1}{2C_{1}}\text{tr}\left(\mathbf{R}^{-1/2}\mathbf{H}\right)+\frac{1}{2C_{1}}\mathcal{D}_{1}(\mathbf{R})\text{tr}(\boldsymbol{\Upsilon}_{1}\mathbf{H})\\\
&=\frac{1}{2C_{1}}\left(-\text{tr}\left(\mathbf{R}^{-1/2}\mathbf{H}\right)+(1-\mathcal{D}_{1}(\mathbf{R}))\text{tr}\left(\mathbf{R}_{0}^{-1/2}\mathbf{H}\right)+\mathcal{D}_{1}(\mathbf{R})\text{tr}(\boldsymbol{\Upsilon}_{1}\mathbf{H})\right),\end{split}$

where we used the definition of $\mathcal{D}_{1}$ and the fact that
$\sum_{\ell=1}^{k}\text{tr}(\mathbf{R}_{\ell\ell}^{-1/2}\mathbf{H}_{\ell\ell})=\text{tr}(\mathbf{R}_{0}^{-1/2}\mathbf{H})$.
The result follows from the linearity of the trace operator and applying the
chain rule to $f(g(\mathbf{R}))=\mathcal{D}_{1}(\mathbf{R})$.

Differentiability of $\mathcal{D}_{2}$

Consider the function

$f:(0,\infty)^{3}\rightarrow\mathbb{R}:(\overline{x},\overline{y},\overline{z})\mapsto\frac{\overline{z}-\overline{x}}{\overline{z}-\overline{y}}.$

Then, $\mathcal{D}_{2}(\mathbf{R})=f(x,y,z)$ and
$\mathcal{D}_{2}(\mathbf{R}+t\mathbf{H}_{t})=f(x^{t},y^{t},z^{t})$, where

$\begin{split}x=\text{tr}\left\\{\left(\mathbf{R}_{0}^{1/2}\mathbf{R}\mathbf{R}_{0}^{1/2}\right)^{1/2}\right\\},\hskip
5.69046pty=\text{tr}\left\\{\left(\mathbf{R}_{0}^{1/2}\mathbf{R}_{m}\mathbf{R}_{0}^{1/2}\right)^{1/2}\right\\},\hskip
5.69046ptz=\text{tr}(\mathbf{R}),\end{split}$

and similarly for $x^{t}$, $y^{t}$ and $z^{t}$ with
$\mathbf{R}+t\mathbf{H}_{t}$ instead of $\mathbf{R}$. From Lemma 4 and Lemma
5, we have

$\lim_{t\,\underset{>}{\to}\,0}\frac{x^{t}-x}{t}=\frac{1}{2}\text{tr}\left\\{\left(\mathbf{J}_{0}+\mathbf{J}^{-1}\right)\mathbf{H}\right\\}\hskip
5.69046pt\text{and}\hskip
5.69046pt\lim_{t\,\underset{>}{\to}\,0}\frac{y^{t}-y}{t}=\text{tr}(\boldsymbol{\Upsilon}_{2}\mathbf{H}).$

Evidently

$\lim_{t\,\underset{>}{\to}\,0}\frac{z^{t}-z}{t}=\text{tr}(\mathbf{H}).$

Hence, the Fréchet derivative of the map

$g:\mathbb{S}^{q}\rightarrow(0,\infty)^{3}:\mathbf{A}\mapsto\left(\text{tr}\left\\{\left(\mathbf{A}_{0}^{1/2}\mathbf{A}\mathbf{A}_{0}^{1/2}\right)^{1/2}\right\\},\text{tr}\left\\{\left(\mathbf{A}_{0}^{1/2}\mathbf{A}_{m}\mathbf{A}_{0}^{1/2}\right)^{1/2}\right\\},\text{tr}(\mathbf{A})\right)$

at $\mathbf{R}$ in the direction of $\mathbf{H}$ equals

$\left(\frac{1}{2}\text{tr}\left\\{\left(\mathbf{J}_{0}+\mathbf{J}^{-1}\right)\mathbf{H}\right\\},\text{tr}\left(\boldsymbol{\Upsilon}_{2}\mathbf{H}\right),\text{tr}(\mathbf{H})\right),$

which we will call $\boldsymbol{\Delta}(\mathbf{H})$. The Jacobian matrix of
$f$ is given by

$\mathbf{J}_{f}=\begin{pmatrix}\frac{\partial
f}{\partial\overline{x}}&\frac{\partial
f}{\partial\overline{y}}&\frac{\partial
f}{\partial\overline{z}}\end{pmatrix}=\begin{pmatrix}\frac{-1}{\overline{z}-\overline{y}}&\frac{\overline{z}-\overline{x}}{(\overline{z}-\overline{y})^{2}}&\frac{\overline{x}-\overline{y}}{(\overline{z}-\overline{y})^{2}}\end{pmatrix},$

such that the total derivative of $f$ at $g(\mathbf{R})$ in the direction of
$\boldsymbol{\Delta}(\mathbf{H})$ becomes

$\begin{split}&\mathbf{J}_{f}|_{g(\mathbf{R})}\boldsymbol{\Delta}(\mathbf{H})^{\text{T}}=\frac{\frac{1}{2}\text{tr}\left\\{\left(\mathbf{J}_{0}+\mathbf{J}^{-1}\right)\mathbf{H}\right\\}}{\text{tr}\left\\{\left(\mathbf{R}_{0}^{1/2}\mathbf{R}_{m}\mathbf{R}_{0}^{1/2}\right)^{1/2}\right\\}-\text{tr}(\mathbf{R})}+\frac{\text{tr}(\mathbf{R})-\text{tr}\left\\{\left(\mathbf{R}_{0}^{1/2}\mathbf{R}\mathbf{R}_{0}^{1/2}\right)^{1/2}\right\\}}{\left[\text{tr}(\mathbf{R})-\text{tr}\left\\{\left(\mathbf{R}_{0}^{1/2}\mathbf{R}_{m}\mathbf{R}_{0}^{1/2}\right)^{1/2}\right\\}\right]^{2}}\text{tr}(\boldsymbol{\Upsilon}_{2}\mathbf{H})\\\
&\hskip
182.09746pt+\frac{\text{tr}\left\\{\left(\mathbf{R}_{0}^{1/2}\mathbf{R}\mathbf{R}_{0}^{1/2}\right)^{1/2}\right\\}-\text{tr}\left\\{\left(\mathbf{R}_{0}^{1/2}\mathbf{R}_{m}\mathbf{R}_{0}^{1/2}\right)^{1/2}\right\\}}{\left[\text{tr}(\mathbf{R})-\text{tr}\left\\{\left(\mathbf{R}_{0}^{1/2}\mathbf{R}_{m}\mathbf{R}_{0}^{1/2}\right)^{1/2}\right\\}\right]^{2}}\text{tr}(\mathbf{H})\\\
&\hskip
48.36958pt=\frac{1}{C_{2}}\left(-\frac{1}{2}\text{tr}\left\\{\left(\mathbf{J}_{0}+\mathbf{J}^{-1}\right)\mathbf{H}\right\\}+\mathcal{D}_{2}(\mathbf{R})\text{tr}(\boldsymbol{\Upsilon_{2}}\mathbf{H})+(1-\mathcal{D}_{2}(\mathbf{R}))\text{tr}(\mathbf{H})\right).\end{split}$

Applying the delta method

Next, we consider the estimator $\widehat{\mathbf{R}}_{n}$. Theorem 3.1 in
[28] tells us that

$\sqrt{n}(\widehat{\mathbf{R}}_{n}-\mathbf{R})-\frac{1}{\sqrt{n}}\sum_{\ell=1}^{n}\left[\mathbf{Z}^{(\ell)}\left(\mathbf{Z}^{(\ell)}\right)^{\text{T}}-\frac{1}{2}\left\\{\mbox{diag}\left(\mathbf{Z}^{(\ell)}\left(\mathbf{Z}^{(\ell)}\right)^{\text{T}}\right)\mathbf{R}+\mathbf{R}\hskip
1.42271pt\text{diag}\left(\mathbf{Z}^{(\ell)}\left(\mathbf{Z}^{(\ell)}\right)^{\text{T}}\right)\right\\}\right]\xrightarrow{p}\mathbf{0}_{q\times
q}$

as $n\to\infty$, where
$\mathbf{Z}^{(\ell)}=(\mathbf{Z}_{1}^{(\ell)},\dots,\mathbf{Z}_{k}^{(\ell)})^{\text{T}}$,
with $\mathbf{Z}_{i}^{(\ell)}=(Z_{i1}^{(\ell)},\dots,Z_{id_{i}}^{(\ell)})$ for
$i\in\\{1,\dots,k\\}$, for $\ell\in\\{1,\dots,n\\}$ is a sample from the
$\mathcal{N}_{q}(\mathbf{0}_{q},\mathbf{R})$ distribution. The same expansion
holds when $\widehat{\mathbf{R}}_{n}$ is the empirical correlation matrix of
$\mathbf{Z}^{(1)},\dots,\mathbf{Z}^{(n)}$, see, e.g., Lemma 8 in [36]. Hence

$\sqrt{n}(\widehat{\mathbf{R}}_{n}-\mathbf{R})-\sqrt{n}\left(\varphi\left(\frac{1}{n}\sum_{\ell=1}^{n}\mathbf{Z}^{(\ell)}\left(\mathbf{Z}^{(\ell)}\right)^{\text{T}}\right)-\mathbf{R}\right)\xrightarrow{p}\mathbf{0}_{q\times
q}$

as $n\to\infty$, i.e., making use of the empirical correlation matrix based on
a true Gaussian sample or based on a pseudo Gaussian sample, results in the
same asymptotic expansion. Suppose further that
$\mathbf{R}=\mathbf{U}\boldsymbol{\Lambda}\mathbf{U}^{\text{T}}$ is the
eigendecomposition of $\mathbf{R}$. Then
$\mathbf{Z}^{(\ell)}=\mathbf{U}\boldsymbol{\Lambda}^{1/2}\boldsymbol{\epsilon}^{(\ell)}$
for $\ell\in\\{1,\dots,n\\}$ and
$\boldsymbol{\epsilon}^{(1)},\dots,\boldsymbol{\epsilon}^{(n)}$ a sample from
$\mathcal{N}_{q}(\mathbf{0}_{q},\mathbf{I}_{q})$. From Lemma 7 of [36], we
have

$\mathbf{W}_{n}=\frac{1}{\sqrt{n}}\sum_{\ell=1}^{n}\left\\{\boldsymbol{\epsilon}^{(\ell)}\left(\boldsymbol{\epsilon}^{(\ell)}\right)^{\text{T}}-\mathbf{I}_{q}\right\\}\xrightarrow{d}\mathbf{W},$

as $n\to\infty$, where $\mathbf{W}$ is a random symmetric matrix with
$\mathbf{W}_{jk}\sim\mathcal{N}(0,2)$ if $j=k\in\\{1,\dots,q\\}$ and
$\mathbf{W}_{jk}\sim\mathcal{N}(0,1)$ if $1\leq j<k\leq q$ independently (and
similarly for $k<j$). Moreover, for $\mathbf{A},\mathbf{B}\in\mathbb{S}^{q}$,
it holds that

$\text{E}\left(\text{tr}(\mathbf{A}\mathbf{W})\text{tr}(\mathbf{B}\mathbf{W})\right)=2\text{tr}(\mathbf{A}\mathbf{B}).$

We find

$\mathbf{U}\boldsymbol{\Lambda}^{1/2}\mathbf{W}_{n}\boldsymbol{\Lambda}^{1/2}\mathbf{U}^{\text{T}}=\sqrt{n}\left(\frac{1}{n}\sum_{\ell=1}^{n}\left(\mathbf{Z}^{(\ell)}\left(\mathbf{Z}^{(\ell)}\right)^{\text{T}}\right)-\mathbf{R}\right)\xrightarrow{d}\mathbf{U}\boldsymbol{\Lambda}^{1/2}\mathbf{W}\boldsymbol{\Lambda}^{1/2}\mathbf{U}^{\text{T}},$

as $n\to\infty$. Applying the delta method (and using that
$\varphi(\mathbf{R})=\mathbf{D}_{\mathbf{R}}^{-1/2}\mathbf{R}\mathbf{D}_{\mathbf{R}}^{-1/2}=\mathbf{R}$),
we obtain

$\begin{split}\sqrt{n}\left(\mathcal{D}_{r}(\widehat{\mathbf{R}}_{n})-\mathcal{D}_{r}(\mathbf{R})\right)&\xrightarrow{d}\text{tr}\left\\{(\mathbf{M}_{r}-\mathbf{D}_{\mathbf{M}_{r}\mathbf{R}})\mathbf{U}\boldsymbol{\Lambda}^{1/2}\mathbf{W}\boldsymbol{\Lambda}^{1/2}\mathbf{U}^{\text{T}}\right\\}\\\
&\hskip
3.1298pt=\text{tr}\left\\{\boldsymbol{\Lambda}^{1/2}\mathbf{U}^{\text{T}}(\mathbf{M}_{r}-\mathbf{D}_{\mathbf{M}_{r}\mathbf{R}})\mathbf{U}\boldsymbol{\Lambda}^{1/2}\mathbf{W}\right\\},\end{split}$

as $n\to\infty$. The latter asymptotic expression is centered Gaussian with
asymptotic variance

$2\text{tr}\left[\left\\{\boldsymbol{\Lambda}^{1/2}\mathbf{U}^{\text{T}}\left(\mathbf{M}_{r}-\mathbf{D}_{\mathbf{M}_{r}\mathbf{R}}\right)\mathbf{U}\boldsymbol{\Lambda}^{1/2}\right\\}^{2}\right]=2\text{tr}\left[\left\\{\mathbf{R}\left(\mathbf{M}_{r}-\mathbf{D}_{\mathbf{M}_{r}\mathbf{R}}\right)\right\\}^{2}\right],$

using the trace cyclical property and finishing the proof. ∎

##  Appendix D Proof of Proposition 4

Consider the estimator given in (10). Notice first of all that the
deterministic correction

$\left[\frac{1}{n}\sum_{\ell=1}^{n}\left\\{\Phi^{-1}\left(\frac{\ell}{n+1}\right)\right\\}^{2}\right]^{-1}=1+\mathcal{O}\left(n^{-1}\ln(n)\right)$

is asymptotically insignificant. Suppose that
$\sup_{n}\lambda_{\max}(\mathbf{R}_{n})\leq\epsilon_{0}^{-1}$ for a certain
$\epsilon_{0}>0$. Fix now $i,m\in\\{1,\dots,k\\}$, $j\in\\{1,\dots,d_{i}\\}$
and $t\in\\{1,\dots,d_{m}\\}$. We desire to find a non-asymptotic deviation
inequality

$\mathbb{P}\left[\left|\frac{1}{n}\sum_{\ell=1}^{n}\left(\widehat{Z}_{ij}^{(\ell)}\widehat{Z}_{mt}^{(\ell)}-\rho_{ij,mt}\right)\right|\geq\epsilon\right]\leq
f(\epsilon,n)$ (60)

holding for $0<\epsilon\leq\delta$, where $\delta>0$ is a positive constant
only depending on $\epsilon_{0}$. Denote by
$\widehat{F}^{*}_{ij}=n/(n+1)\widehat{F}_{ij}$ the rescaled empirical cdf of
$X_{ij}$, and similarly for $X_{mt}$, en let $\widehat{H}$ be the joint
empirical cdf of
$(X_{ij}^{(1)},X_{mt}^{(1)}),\dots,(X_{ij}^{(n)},X_{mt}^{(n)})$, and $H$ be
the true cdf of $(X_{ij},X_{mt})$. Recall that $(X_{ij},X_{mt})$ has a
Gaussian copula, meaning that

$\left(\left(\Phi^{-1}\circ F_{ij}\right)(X_{ij}),\left(\Phi^{-1}\circ
F_{mt}\right)(X_{mt})\right)\sim\Phi_{\text{G}},$

where $\Phi_{\text{G}}$ stands for a bivariate normal distribution with means
zero, unit variances and correlation $\rho_{ij,mt}$. We also need the
Dvoretzky-Kiefer-Wolfowitz inequality (see, e.g., [34]):

$\mathbb{P}\left[\sup_{x\in\mathbb{R}}\left|\widehat{F}_{ij}(x)-F_{ij}(x)\right|\geq\epsilon\right]\leq
2\exp\left(-2n\epsilon^{2}\right)$ (61)

for any $\epsilon>0$. Now, consider the decomposition

$\begin{split}\frac{1}{n}\sum_{\ell=1}^{n}\left(\widehat{Z}_{ij}^{(\ell)}\widehat{Z}_{mt}^{(\ell)}-\rho_{ij,mt}\right)&=\int_{\mathbb{R}^{2}}\Phi^{-1}\left(\widehat{F}_{ij}^{*}(x)\right)\Phi^{-1}\left(\widehat{F}_{mt}^{*}(y)\right)d\widehat{H}(x,y)-\int_{\mathbb{R}^{2}}\Phi^{-1}\left(F_{ij}(x)\right)\Phi^{-1}\left(F_{mt}(y)\right)dH(x,y)\\\
&=A_{1n}+A_{2n}+A_{3n}+B_{1n}+B_{2n}+R_{n},\end{split}$

where (denoting $\phi$ for the standard normal density function)

$\begin{split}A_{1n}&=\int_{\mathbb{R}^{2}}\Phi^{-1}\left(F_{ij}(x)\right)\Phi^{-1}\left(F_{mt}(y)\right)d\left(\widehat{H}-H\right)(x,y)\\\
A_{2n}&=\int_{\mathbb{R}^{2}}\frac{\widehat{F}_{ij}^{*}(x)-F_{ij}(x)}{\phi\left(\Phi^{-1}\left(F_{ij}(x)\right)\right)}\Phi^{-1}\left(F_{mt}(y)\right)dH(x,y)\\\
A_{3n}&=\int_{\mathbb{R}^{2}}\frac{\widehat{F}_{mt}^{*}(y)-F_{mt}(y)}{\phi\left(\Phi^{-1}\left(F_{mt}(y)\right)\right)}\Phi^{-1}\left(F_{ij}(x)\right)dH(x,y)\\\
B_{1n}&=\int_{\mathbb{R}^{2}}\left\\{\Phi^{-1}\left(\widehat{F}_{ij}^{*}(x)\right)-\Phi^{-1}\left(F_{ij}(x)\right)\right\\}\Phi^{-1}\left(F_{mt}(y)\right)d\widehat{H}(x,y)-A_{2n}\\\
B_{2n}&=\int_{\mathbb{R}^{2}}\left\\{\Phi^{-1}\left(\widehat{F}_{mt}^{*}(y)\right)-\Phi^{-1}\left(F_{mt}(y)\right)\right\\}\Phi^{-1}\left(F_{ij}(x)\right)d\widehat{H}(x,y)-A_{3n}\\\
R_{n}&=\int_{\mathbb{R}^{2}}\left\\{\Phi^{-1}\left(\widehat{F}_{ij}^{*}(x)\right)-\Phi^{-1}\left(F_{ij}(x)\right)\right\\}\left\\{\Phi^{-1}\left(\widehat{F}_{mt}^{*}(y)\right)-\Phi^{-1}\left(F_{mt}(y)\right)\right\\}d\widehat{H}(x,y).\end{split}$

Lemma A.3. in [3] tells us that

$\mathbb{P}\left[\left|A_{1n}\right|\geq\epsilon\right]\leq
C_{1}\exp\left(-C_{2}n\epsilon^{2}\right)$

for $\epsilon\leq\delta_{1}$, where $C_{1},C_{2}>0$ and $\delta_{1}>0$ only
depend on $\epsilon_{0}$. Next, since

$dH(x,y)=\phi_{\text{G}}\left(\Phi^{-1}\left(F_{ij}(x)\right),\Phi^{-1}\left(F_{mt}(y)\right)\right)d\Phi^{-1}\left(F_{ij}(x)\right)d\Phi^{-1}\left(F_{mt}(y)\right),$

with $\phi_{\text{G}}$ the density of $\Phi_{\text{G}}$, we see that

$\begin{split}A_{2n}=\int_{\mathbb{R}^{2}}\frac{\widehat{F}_{ij}^{*}(x)-F_{ij}(x)}{\phi\left(\Phi^{-1}\left(F_{ij}(x)\right)\right)}\Phi^{-1}\left(F_{mt}(y)\right)\phi_{\text{G}}\left(\Phi^{-1}\left(F_{ij}(x)\right),\Phi^{-1}\left(F_{mt}(y)\right)\right)d\Phi^{-1}\left(F_{ij}(x)\right)d\Phi^{-1}\left(F_{mt}(y)\right),\end{split}$

where

$\begin{split}\int_{\mathbb{R}}\Phi^{-1}\left(F_{mt}(y)\right)\phi_{\text{G}}\left(\Phi^{-1}\left(F_{ij}(x)\right),\Phi^{-1}\left(F_{mt}(y)\right)\right)d\Phi^{-1}\left(F_{mt}(y)\right)&=\int_{\mathbb{R}}\phi\left(\Phi^{-1}\left(F_{ij}(x)\right)\right)\frac{\phi_{\text{G}}\left(\Phi^{-1}\left(F_{ij}(x)\right),s\right)}{\phi\left(\Phi^{-1}\left(F_{ij}(x)\right)\right)}\hskip
0.56917pts\hskip 0.56917ptds\\\ &=\rho_{ij,mt}\hskip
0.56917pt\phi\left(\Phi^{-1}\left(F_{ij}(x)\right)\right)\Phi^{-1}\left(F_{ij}(x)\right),\end{split}$

since $\mathbb{E}(X_{2}|X_{1}=x_{1})=\rho\hskip 0.56917ptx_{1}$ when
$(X_{1},X_{2})$ follows a bivariate standard normal distribution with
correlation $\rho$. Hence

$\begin{split}A_{2n}&=\rho_{ij,mt}\int_{\mathbb{R}}\Phi^{-1}\left(F_{ij}(x)\right)\left(\widehat{F}_{ij}^{*}(x)-F_{ij}(x)\right)d\Phi^{-1}\left(F_{ij}(x)\right)\\\
&=-\frac{\rho_{ij,mt}}{2}\int_{\mathbb{R}}\left\\{\Phi^{-1}\left(F_{ij}(x)\right)\right\\}^{2}d\left(\widehat{F}_{ij}^{*}-F_{ij}\right)(x)\\\
&=-\frac{\rho_{ij,mt}}{2}\frac{1}{n}\sum_{\ell=1}^{n}\left(\left\\{\Phi^{-1}\left(F_{ij}\left(X_{ij}^{(\ell)}\right)\right)\right\\}^{2}-1\right)\\\
&=-\frac{\rho_{ij,mt}}{2}\frac{1}{n}\sum_{\ell=1}^{n}\left(V_{\ell}^{2}-1\right),\end{split}$

where we did partial integration, and $V_{\ell}\sim\chi^{2}_{1}$ for
$\ell=1,\dots,n$. Hence, using Theorem 3.2 on page 45 of [41] (noting that
condition (P)(3.12) on page 45 holds for $\chi^{2}_{1}$, see also the proof of
Lemma A.3. in [3]), we have

$\mathbb{P}\left[\left|A_{2n}\right|\geq\epsilon\right]\leq
C_{3}\exp\left(-C_{4}n\epsilon^{2}\right)$

for $\epsilon\leq\delta_{2}$, where $C_{3},C_{4}$ and $\delta_{2}>0$ only
depend on $\epsilon_{0}$. A very similar argument holds for $A_{3n}$.

Next, we deal with the term $B_{1n}$. By symmetry, the term $B_{2n}$ can be
dealt with similarly. We use the mean value theorem, giving that

$\Phi^{-1}\left(\widehat{F}_{ij}^{*}(x)\right)-\Phi^{-1}\left(F_{ij}(x)\right)=\frac{\widehat{F}_{ij}^{*}(x)-F_{ij}(x)}{\phi\left(\Phi^{-1}\left(\widetilde{F}_{ij}^{*}(x)\right)\right)}$

for a certain $\widetilde{F}_{ij}^{*}(x)$ satisfying
$|\widetilde{F}_{ij}^{*}(x)-F_{ij}(x)|\leq|\widehat{F}_{ij}^{*}(x)-F_{ij}(x)|$.
Of course, $|\Phi^{-1}(t)|\to\infty$ when $t\to 0$ or $t\to 1$, and
$\phi(t)\to 0$ when $|t|\to\infty$, so we will need to further split up
$B_{1n}$ by defining the set $M_{\eta}=M_{\eta 1}\times M_{\eta 2}$, where

$\begin{split}M_{\eta
1}&=\left[F_{ij}^{-1}(\eta),F_{ij}^{-1}(1-\eta)\right]\\\ M_{\eta
2}&=\left[F_{mt}^{-1}(\eta),F_{mt}^{-1}(1-\eta)\right]\end{split}$

for a certain small $\eta>0$. Doing so, we can write

$B_{1n}=B_{\eta 11n}+B_{\eta 21n}+B_{\eta 31n}+B_{\eta 41n},$

where

$\begin{split}B_{\eta
11n}&=\int_{M_{\eta}^{\text{C}}}\left\\{\Phi^{-1}\left(\widehat{F}_{ij}^{*}(x)\right)-\Phi^{-1}\left(F_{ij}(x)\right)\right\\}\Phi^{-1}\left(F_{mt}(y)\right)d\widehat{H}(x,y)\\\
B_{\eta
21n}&=\int_{M_{\eta}}\left\\{\frac{\left(\widehat{F}_{ij}^{*}(x)-F_{ij}(x)\right)}{\phi\left(\Phi^{-1}\left(\widetilde{F}_{ij}^{*}(x)\right)\right)}-\frac{\left(\widehat{F}_{ij}^{*}(x)-F_{ij}(x)\right)}{\phi\left(\Phi^{-1}\left(F_{ij}(x)\right)\right)}\right\\}\Phi^{-1}\left(F_{mt}(y)\right)d\widehat{H}(x,y)\\\
B_{\eta
31n}&=\int_{M_{\eta}}\frac{\left(\widehat{F}_{ij}^{*}(x)-F_{ij}(x)\right)}{\phi\left(\Phi^{-1}\left(F_{ij}(x)\right)\right)}\Phi^{-1}\left(F_{mt}(y)\right)d\left(\widehat{H}-H\right)(x,y)\\\
B_{\eta
41n}&=\int_{M_{\eta}}\frac{\left(\widehat{F}_{ij}^{*}(x)-F_{ij}(x)\right)}{\phi\left(\Phi^{-1}\left(F_{ij}(x)\right)\right)}\Phi^{-1}\left(F_{mt}(y)\right)dH(x,y)-A_{2n}.\end{split}$

Notice that we did not apply the mean value theorem in the term $B_{\eta
11n}$, because we will decompose $R_{n}$ as

$R_{n}=R_{\eta 1n}+R_{\eta 2n}+R_{\eta 3n},$

where

$\begin{split}R_{\eta
1n}&=\int_{M_{\eta}^{\text{C}}}\left\\{\Phi^{-1}\left(\widehat{F}_{ij}^{*}(x)\right)-\Phi^{-1}\left(F_{ij}(x)\right)\right\\}\Phi^{-1}\left(\widehat{F}_{mt}^{*}(y)\right)d\widehat{H}(x,y)\\\
R_{\eta
2n}&=-\int_{M_{\eta}^{\text{C}}}\left\\{\Phi^{-1}\left(\widehat{F}_{ij}^{*}(x)\right)-\Phi^{-1}\left(F_{ij}(x)\right)\right\\}\Phi^{-1}\left(F_{mt}(y)\right)d\widehat{H}(x,y)\\\
R_{\eta
3n}&=\int_{M_{\eta}}\frac{\widehat{F}_{ij}^{*}(x)-F_{ij}(x)}{\phi\left(\Phi^{-1}\left(\widetilde{F}_{ij}^{*}(x)\right)\right)}\left\\{\Phi^{-1}\left(F_{mt}^{*}(y)\right)-\Phi^{-1}\left(F_{mt}(y)\right)\right\\}d\widehat{H}(x,y),\end{split}$

such that $B_{\eta 11n}$ and $R_{\eta 2n}$ cancel each other out. As for
$B_{\eta 21n}$, it holds almost surely that

$\left|B_{\eta
21n}\right|\leq\sup_{x\in\mathbb{R}}\left|\widehat{F}_{ij}^{*}(x)-F_{ij}(x)\right|\sup_{x\in
M_{\eta
1}}\left|\frac{1}{\phi\left(\Phi^{-1}\left(\widetilde{F}_{ij}^{*}(x)\right)\right)}-\frac{1}{\phi\left(\Phi^{-1}\left(F_{ij}(x)\right)\right)}\right|\sup_{y\in
M_{\eta 2}}\Phi^{-1}\left(F_{mt}(y)\right).$

The function $\Phi^{-1}\circ F_{mt}$ is uniformly bounded on $M_{\eta 2}$ and
since $\phi\circ\Phi^{-1}$ is continuous, it is uniformly continuous on
$M_{\eta 1}$. This, and the fact that
$|\widetilde{F}_{ij}^{*}-F_{ij}|\leq|\widehat{F}_{ij}^{*}-F_{ij}|$, results in

$\left|B_{\eta 21n}\right|\leq
K\sup_{x\in\mathbb{R}}\left|\widehat{F}_{ij}^{*}(x)-F_{ij}(x)\right|$

almost surely for a certain $K>0$. Hence, using (61), we obtain

$\begin{split}\mathbb{P}\left[\left|B_{\eta
21n}\right|\geq\epsilon\right]\leq\mathbb{P}\left[\sup_{x\in\mathbb{R}}\left|\widehat{F}_{ij}^{*}(x)-F_{ij}(x)\right|\geq\frac{\epsilon}{K}\right]&\leq\mathbb{P}\left[\sup_{x\in\mathbb{R}}\left|\widehat{F}_{ij}(x)-F_{ij}(x)\right|\geq\frac{\epsilon}{2K}\right]+\mathbb{P}\left[\sup_{x\in\mathbb{R}}\left|\widehat{F}_{ij}^{*}(x)-\widehat{F}_{ij}(x)\right|\geq\frac{\epsilon}{2K}\right]\\\
&\leq
2\exp\left(\frac{-n\epsilon^{2}}{2K^{2}}\right)+\mathbb{P}\left[\frac{1}{n+1}\geq\frac{\epsilon}{2K}\right].\end{split}$

Similar arguments hold for the terms $B_{\eta 31n},B_{\eta 41n}$ and $R_{\eta
3n}$.

The only term that is not discussed yet, is $R_{\eta 1n}$. We want this term
to converge to zero in probability when $\eta\to 0$ for every $n$. We can deal
with this in a similar way as in Corollary 5.6. of [40] (in fact, we are even
in the context of Section 6 on page $1133$ since Assumption 2.3(b) holds),
since the corresponding assumptions hold here (they are verified in the proof
of Theorem 3.1 of [28]). In our setting, the key ingredient is that there
exists $a=b=(1/2-\xi)/2$ for a certain $0<\xi<1/2$ such that

$\left|\Phi^{-1}(t)\right|\leq M_{1}r^{a}(t),\hskip
28.45274pt\text{with}\hskip 28.45274ptr(t)=\frac{1}{t(1-t)}$

for all $t\in(0,1)$ and a certain $M_{1}>0$, i.e., $|\Phi^{-1}(t)|\to\infty$
as $t\to 0$ or $t\to 1$, but not too fast. Applying the mean value theorem
again, we have

$\begin{split}\left|\Phi^{-1}\left(\widehat{F}_{ij}^{*}(x)\right)-\Phi^{-1}\left(F_{ij}(x)\right)\right|&=\left|\left(\widehat{F}_{ij}^{*}(x)-F_{ij}(x)\right)\left[\Phi^{-1}\left(\widetilde{F}_{ij}^{*}(x)\right)\right]^{\prime}\right|\\\
&\leq
M_{1}\left|\widehat{F}_{ij}^{*}(x)-F_{ij}(x)\right|r^{a+1}\left(\widetilde{F}_{ij}^{*}(x)\right).\end{split}$

Moreover, if we take $\widetilde{\epsilon}_{1}>0$ arbitrary, there exist
$M_{2},M_{3},M_{4}>0$ such that, uniformly in $n$, the events

$\begin{split}E_{1n}&=\left\\{\left|\widehat{F}_{ij}^{*}-F_{ij}\right|\leq
M_{2}r^{-1/2+\xi/4}(F_{ij})\right\\}\\\
E_{2n}&=\left\\{r^{a}\left(\widehat{F}_{mt}^{*}\right)\leq
M_{3}r^{a}(F_{mt})\right\\}\\\
E_{3n}&=\left\\{r^{a+1}\left(\widetilde{F}_{ij}^{*}\right)\leq
M_{4}r^{a+1}(F_{ij})\right\\}\end{split}$

satisfy $\mathbb{P}(E^{\text{C}})<\widetilde{\epsilon}_{1}/2$, for
$E=\cap_{i=1}^{3}E_{in}$. Combining the inequalities, we also see that
(similarly to expression (5.2) in [40])

$\mathbb{E}\left(\mathds{1}\left(E\right)\left|R_{\eta 1n}\right|\right)\leq
M_{1}^{2}M_{2}M_{3}M_{4}\int_{M_{\eta}^{\text{C}}}r^{a+1/2+\xi/4}\left(F_{ij}(x)\right)r^{a}\left(F_{mt}(y)\right)dH(x,y),$

where $\mathds{1}$ is the indicator function. By Hölder’s inequality, the
above integral is bounded by (noting that $M_{\eta}^{\text{C}}\subset(M_{\eta
1}^{\text{C}}\times\mathbb{R})\cup(\mathbb{R}\times M_{\eta 2}^{\text{C}})$)

$\begin{split}&\left(\int_{(0,\eta)\cup(1-\eta,1)}r^{p_{1}(a+1/2+\xi/4)}(t)dt\right)^{1/p_{1}}\left(\int_{0}^{1}r^{q_{1}a}(t)dt\right)^{1/q_{1}}\\\
&\hskip
113.81102pt+\left(\int_{0}^{1}r^{p_{1}(a+1/2+\xi/4)}(t)dt\right)^{1/p_{1}}\left(\int_{(0,\eta)\cup(1-\eta,1)}r^{q_{1}a}(t)dt\right)^{1/q_{1}}<\infty,\end{split}$

where $p_{1}$ and $q_{1}$ are chosen such that $p_{1}^{-1}+q_{1}^{-1}=1$,
$p_{1}(a+1/2+\xi/4)<1$ and $q_{1}a<1$ (see equation (3.5) in [40] and
corresponding explanation), making the integral bounded. Hence, by dominated
convergence, we have that $\mathds{1}(E)|R_{\eta 1n}|\xrightarrow{p}0$ as
$\eta\to 0$ for each $n$. Hence, for our arbitrarily chosen
$\widetilde{\epsilon}_{1}>0$, and another arbitrary
$\widetilde{\epsilon}_{2}>0$, we see that

$\begin{split}\mathbb{P}\left[\left|R_{\eta
1n}\right|>\widetilde{\epsilon}_{2}\right]&=\mathbb{P}\left[\left|R_{\eta
1n}\right|>\widetilde{\epsilon}_{2},\mathds{1}(E)=1\right]+\mathbb{P}\left[\left|R_{\eta
1n}\right|>\widetilde{\epsilon}_{2},\mathds{1}(E)=0\right]\\\
&=\mathbb{P}\left[\mathds{1}\left(E\right)\left|R_{\eta
1n}\right|>\widetilde{\epsilon}_{2}\right]+\mathbb{P}\left[\left|R_{\eta
1n}\right|>\widetilde{\epsilon}_{2},\mathds{1}(E)=0\right]\\\
&\leq\mathbb{P}\left[\mathds{1}\left(E\right)\left|R_{\eta
1n}\right|>\widetilde{\epsilon}_{2}\right]+\mathbb{P}\left[\mathds{1}(E)=0\right]\\\
&=\mathbb{P}\left[\mathds{1}\left(E\right)\left|R_{\eta
1n}\right|>\widetilde{\epsilon}_{2}\right]+\mathbb{P}\left(E^{\text{C}}\right)\\\
&<\frac{\widetilde{\epsilon}_{1}}{2}+\frac{\widetilde{\epsilon}_{1}}{2}=\widetilde{\epsilon}_{1}\end{split}$

for $\eta$ small enough, i.e., $|R_{\eta 1n}|\xrightarrow{p}0$ as $\eta\to 0$
for all $n$. This means that $\mathbb{P}[|R_{\eta 1n}|\geq\epsilon]$ can be
made zero for every $n$ by letting $\eta\to 0$, and, combining everything, we
have shown that a concentration inequality (60) holds. In simplified form, we
can say that (60) holds with

$f(\epsilon,n)=K_{1}\exp\left(-K_{2}n\epsilon^{2}\right)+K_{3}\mathbb{P}\left[\frac{1}{n+1}\geq\frac{\epsilon}{K_{4}}\right],$

for $0<\epsilon\leq\delta$, where $K_{1},K_{2},K_{3},K_{4},\delta>0$ possibly
only depend on $\epsilon_{0}$.

We now show that
$||\widehat{\mathbf{R}}_{n}-\mathbf{R}_{n}||_{\infty}=\mathcal{O}_{p}(\\{\ln(q_{n})/n\\}^{1/2})$.
Take an arbitrary $\varepsilon>0$. Let $M,N>0$ be such that $2-K_{2}M^{2}<0$,
$K_{1}q_{n}^{2-K_{2}M^{2}}<\varepsilon$, $Mn^{-1/2}\ln(q_{n})^{1/2}\leq\delta$
and $1/(n+1)<Mn^{-1/2}\ln(q_{n})^{1/2}/K_{4}$ for all $n>N$. Then, by the
union bound, it holds that

$\begin{split}\mathbb{P}\left[\frac{\left|\left|\widehat{\mathbf{R}}_{n}-\mathbf{R}_{n}\right|\right|_{\infty}}{n^{-1/2}\ln(q_{n})^{1/2}}>M\right]&=\mathbb{P}\left[\left|\left|\widehat{\mathbf{R}}_{n}-\mathbf{R}_{n}\right|\right|_{\infty}>Mn^{-1/2}\ln(q_{n})^{1/2}\right]\\\
&\leq
q_{n}^{2}K_{1}\exp\left\\{-K_{2}n\left(Mn^{-1/2}\ln(q_{n})^{1/2}\right)^{2}\right\\}+q_{n}^{2}K_{3}\mathbb{P}\left[\frac{1}{n+1}\geq\frac{Mn^{-1/2}\ln(q_{n})^{1/2}}{K_{4}}\right]\\\
&=K_{1}q_{n}^{2-K_{2}M^{2}}<\varepsilon,\end{split}$

for all $n>N$, proving the desired result. ∎

## References

  * Ansari and Fuchs [2023] J. Ansari, S. Fuchs, A simple extension of Azadkia & Chatterjee’s rank correlation to a vector of enogenous variables, preprint arxiv.2212.01621 (2023).
  * Azadkia and Chatterjee [2021] M. Azadkia, S. Chatterjee, A simple measure of conditional dependence, Ann. Stat. 49 (2021) 3070–3102.
  * Bickel and Levina [2008] P. J. Bickel, E. Levina, Regularized estimation of large covariance matrices, Ann. Stat. 36 (2008) 199–227.
  * Bien and Tibshirani [2011] J. Bien, R. J. Tibshirani, Sparse estimation of a covariance matrix, Biometrika 98 (2011) 807–820.
  * Bien and Tibshirani [2022] J. Bien, R. J. Tibshirani, Spcov: Sparse Estimation of a Covariance Matrix, 2022\. R package version 1.3.
  * Bigot et al. [2011] J. Bigot, R. J. Biscay, J.-M. Loubes, L. Muñiz-Alvarez, Group lasso estimation of high-dimensional covariance matrices, J. Mach. Learn. Res. 12 (2011) 3187–3225.
  * Cardoso [2011] J. R. Cardoso, Evaluating the Fréchet derivative of the matrix pth root, Electron. Trans. Numer. Anal. 38 (2011) 202–217.
  * Chatterjee [2021] S. Chatterjee, A new coefficient of correlation, J. Am. Stat. Assoc. 116 (2021) 2009–2022.
  * Chiquet et al. [2012] J. Chiquet, Y. Grandvalet, C. Charbonnier, Sparsity with sign-coherent groups of variables via the cooperative-lasso, Ann. Appl. Stat. 6 (2012) 795–860.
  * Clement and Desch [2008] P. Clement, W. Desch, An elementary proof of the triangle inequality for the Wasserstein metric, Proc. Am. Math. Soc. 136 (2008) 333–339.
  * De Keyser and Gijbels [2023a] S. De Keyser, I. Gijbels, Copula-based divergence measures for dependence between random vectors, in: L. A. García-Escudero, A. Gordaliza, A. Mayo, M. A. L. Gomez, M. A. Gil, P. Grzegorzewski, O. Hryniewicz (Eds.), Advances in Intelligent Systems and Computing, Vol. 1433, Building Bridges between Soft and Statistical Methodologies for Data Science, Springer, 2023a, pp. 104–111.
  * De Keyser and Gijbels [2023b] S. De Keyser, I. Gijbels, Parametric dependence between random vectors via copula-based divergence measures, preprint arxiv:2302.13611 (2023b).
  * Fan et al. [2009] J. Fan, Y. Feng, Y. Wu, Network exploration via the adaptive lasso and scad penalties, Ann. Appl. Stat. 3 (2009) 521–541.
  * Fan and Li [2001] J. Fan, R. Li, Variable selection via nonconcave penalized likelihood and its oracle properties, J. Am. Stat. Assoc. 96 (2001) 1348–1360.
  * Fermanian [2024] J.-D. Fermanian, Sparse M-estimators in semi-parametric copula models, to appear in Bernoulli, url: https://www.bernoullisociety.org/publications/bernoulli-journal/bernoulli-journal-papers (2024).
  * Fop [2021] M. Fop, Covglasso: sparse covariance matrix estimation, 2021. R package version 1.0.3.
  * Foygel and Drton [2010] R. Foygel, M. Drton, Extended Bayesian information criteria for Gaussian graphical models, in: J. Lafferty, C. Williams, J. Shawe-Taylor, R. Zemel, A. Culotta (Eds.), Advances in Neural Information Processing Systems 23 (NIPS 2010), 2010, pp. 604–612.
  * Fuchs [2023] S. Fuchs, Quantifying directed dependence via dimension reduction, J. Multivar. Anal., in press, available online, 105266 (2023).
  * Fuchs et al. [2021] S. Fuchs, F. M. L. Di Lascio, F. Durante, Dissimilarity functions for rank-invariant hierarchical clustering of continuous variables, Comput. Stat. Data Anal. 159 (2021) 107201.
  * Geenens and Lafaye de Micheaux [2022] G. Geenens, P. Lafaye de Micheaux, The Hellinger correlation, J. Am. Stat. Assoc. 117 (2022) 639–653.
  * Genest et al. [2009] C. Genest, B. Rémillard, D. Beaudoin, Goodness-of-fit tests for copulas: A review and a power study, Insur. Math. Econ. 44 (2009) 199–213.
  * Gijbels et al. [2021] I. Gijbels, V. Kika, M. Omelka, On the specification of multivariate association measures and their behaviour with increasing dimension, J. Multivar. Anal. 182 (2021) 104704.
  * Grothe et al. [2014] O. Grothe, J. Schnieders, J. Segers, Measuring association and dependence between random vectors, J. Multivar. Anal. 123 (2014) 96–110.
  * Hájek and Šidák [1967] J. Hájek, Z. Šidák, Theory of Rank Tests, Academia, Prague, 1967.
  * Higham [2008] N. J. Higham, Functions of Matrices, SIAM, 2008.
  * Hofert et al. [2019] M. Hofert, W. Oldford, A. Prasad, M. Zhu, A framework for measuring association of random vectors via collapsed random variables, J. Multivar. Anal. 172 (2019) 5–27.
  * Hotelling [1936] H. Hotelling, Relations between two sets of variates, Biometrika 28 (1936) 321–377.
  * Klaassen and Wellner [1997] C. A. J. Klaassen, J. A. Wellner, Efficient estimation in the bivariate normal copula model: normal margins are least favourable, Bernoulli 3 (1997) 55–77.
  * Lam and Fan [2009] C. Lam, J. Fan, Sparsistency and rates of convergence in large covariance matrix estimation, Ann. Stat. 37 (2009) 4254–4278.
  * Lê and Husson [2008] S. Lê, F. Husson, SensoMineR: A package for sensory data analysis, J. Sens. Stud. 23 (2008) 14–25.
  * Ledoit and Wolf [2004] O. Ledoit, M. Wolf, A well-conditioned estimator for large-dimensional covariance matrices, J. Multivar. Anal. 88 (2004) 365–411.
  * Llobell et al. [2020] F. Llobell, V. Cariou, E. Vigneau, A. Labenne, E. M. Qannari, Analysis and clustering of multiblock datasets by means of the STATIS and CLUSTATIS methods. Application to sensometrics, Food Qual. Prefer. 79 (2020) 103520.
  * Marshall et al. [2011] A. Marshall, I. Olkin, B. Arnold, Inequalities: theory of majorization and its applications, volume 2, Springer Series in Statistics, Springer, 2011\.
  * Massart [1990] P. Massart, The tight constant in the Dvoretzky-Kiefer-Wolfowitz inequality, Ann. Probab. 18 (1990) 1269–1283.
  * Medovikov and Prokhorov [2017] I. Medovikov, A. Prokhorov, A new measure of vector dependence, with applications to financial risk and contagion, J. Financ. Econom. 15 (2017) 474–503.
  * Mordant and Segers [2022] G. Mordant, J. Segers, Measuring dependence between random vectors via optimal transport, J. Multivar. Anal. 189 (2022) 104912.
  * Nelsen [1996] R. B. Nelsen, Nonparametric measures of multivariate association, in: L. Rüschendorf, B. Schweizer, M. D. Taylor (Eds.), IMS Lecture Notes - Monograph Series Vol. 28, Distributions with Fixed Marginals and Related Topics, 1996, pp. 223–232.
  * Nelsen [2006] R. B. Nelsen, An Introduction to Copulas, Springer Science and Business Media, New York, 2006.
  * Panaretos and Zemel [2019] V. M. Panaretos, Y. Zemel, Statistical aspects of Wasserstein distances, Annu. Rev. Stat. Appl. 6 (2019) 405–431.
  * Ruymgaart et al. [1972] F. H. Ruymgaart, G. R. Shorack, W. R. van Zwet, Asymptotic normality of nonparametric tests for independence, Ann. Math. Stat. 43 (1972) 1122–1135.
  * Saulis and Statulevičius [1991] L. Saulis, V. A. Statulevičius, Limit theorems for Large Deviations, Kluwer Academic Publishers, Dordrecht, 1991.
  * Schmid and Schmidt [2007] F. Schmid, R. Schmidt, Multivariate extensions of Spearman’s rho and related statistics, Stat. Probab. Lett. 77 (2007) 407–416.
  * Sklar [1959] A. Sklar, Fonctions de repartition à n dimensions et leurs marges, Publications de l’Institut Statistique de l’Université de Paris 8 (1959) 229–231.
  * Székely et al. [2007] G. J. Székely, M. L. Rizzo, N. K. Bakirov, Measuring and testing dependence by correlation of distances, Ann. Stat. 35 (2007) 2769–2794.
  * Takatsu [2011] A. Takatsu, Wasserstein geometry of Gaussian measures, Osaka J. Math. 48 (2011) 1005–1026.
  * Thompson and Therianos [1972] R. C. Thompson, S. Therianos, Inequalities connecting the eigenvalues of a hermitian matrix with the eigenvalues of complementary principal submatrices, Bull. Aust. Math. Soc. 6 (1972) 117–132.
  * Tsanas et al. [2014] A. Tsanas, M. A. Little, C. Fox, L. O. Ramig, Objective automatic assessment of rehabilitative speech treatment in Parkinson’s disease, IEEE Trans. Neural Syst. Rehabil. Eng. 22 (2014) 181–190.
  * Villani [2008] C. Villani, Optimal Transport: Old and New, volume 338, Springer Science & Business Media, 2008.
  * Wang [2014] H. Wang, Coordinate descent algorithm for covariance graphical lasso, Stat. Comput. 24 (2014) 521–529.
  * Warton [2008] D. I. Warton, Penalized normal likelihood and ridge regularization of correlation and covariance matrices, J. Am. Stat. Assoc. 103 (2008) 340–349.
  * Zhang et al. [2020] L. Zhang, D. Lu, X. Wang, The essential dependence for a group of random vectors, Commun. Stat. - Theory Methods 50 (2020) 1–37.
  * Zou and Li [2008] H. Zou, R. Li, One-step sparse estimates in nonconcave penalized likelihood models, Ann. Stat. 36 (2008) 1509–1533.

Supplementary Material
to
High-dimensional copula-based Wasserstein dependence

by

Steven De Keyser and Irène Gijbels

Fig. S1: Monte Carlo bias of the estimators
$\mathcal{D}_{\mathchoice{\mathbin{\vbox{\hbox{\scalebox{0.9}{$\displaystyle\bullet$}}}}}{\mathbin{\vbox{\hbox{\scalebox{0.9}{$\textstyle\bullet$}}}}}{\mathbin{\vbox{\hbox{\scalebox{0.9}{$\scriptstyle\bullet$}}}}}{\mathbin{\vbox{\hbox{\scalebox{0.9}{$\scriptscriptstyle\bullet$}}}}}}(\widehat{\mathbf{R}}_{\text{R},n})$
(penalty) and
$\mathcal{D}_{\mathchoice{\mathbin{\vbox{\hbox{\scalebox{0.9}{$\displaystyle\bullet$}}}}}{\mathbin{\vbox{\hbox{\scalebox{0.9}{$\textstyle\bullet$}}}}}{\mathbin{\vbox{\hbox{\scalebox{0.9}{$\scriptstyle\bullet$}}}}}{\mathbin{\vbox{\hbox{\scalebox{0.9}{$\scriptscriptstyle\bullet$}}}}}}(\widehat{\mathbf{R}}_{n})$
(no penalty) for
$\mathcal{D}_{t\ln(t)},\mathcal{D}_{(\sqrt{t}-1)^{2}},\mathcal{D}_{1}$ and
$\mathcal{D}_{2}$, based on $1000$ replications with sample sizes
$n=50,100,500$ as a function of $q$, in two different designs. Fig. S2: Monte
Carlo variance of the estimators
$\mathcal{D}_{\mathchoice{\mathbin{\vbox{\hbox{\scalebox{0.9}{$\displaystyle\bullet$}}}}}{\mathbin{\vbox{\hbox{\scalebox{0.9}{$\textstyle\bullet$}}}}}{\mathbin{\vbox{\hbox{\scalebox{0.9}{$\scriptstyle\bullet$}}}}}{\mathbin{\vbox{\hbox{\scalebox{0.9}{$\scriptscriptstyle\bullet$}}}}}}(\widehat{\mathbf{R}}_{\text{R},n})$
(penalty) and
$\mathcal{D}_{\mathchoice{\mathbin{\vbox{\hbox{\scalebox{0.9}{$\displaystyle\bullet$}}}}}{\mathbin{\vbox{\hbox{\scalebox{0.9}{$\textstyle\bullet$}}}}}{\mathbin{\vbox{\hbox{\scalebox{0.9}{$\scriptstyle\bullet$}}}}}{\mathbin{\vbox{\hbox{\scalebox{0.9}{$\scriptscriptstyle\bullet$}}}}}}(\widehat{\mathbf{R}}_{n})$
(no penalty) for
$\mathcal{D}_{t\ln(t)},\mathcal{D}_{(\sqrt{t}-1)^{2}},\mathcal{D}_{1}$ and
$\mathcal{D}_{2}$, based on $1000$ replications with sample sizes
$n=50,100,500$ as a function of $q$, in two different designs. Fig. S3:
Pairwise scatterplots of normal scores of $X$-$Y$ coordinates of $24$ people
rating smoothies (rating of the $i$’th person is described by the coordinates
$(X_{i},Y_{i})$). Fig. S4: Sample normal scores rank correlation matrix of the
$310$ dysphonia measures in the LSVT voice rehabilitation dataset.

Fig. S5: Sample normal scores rank correlation matrix of the $91$ dysphonia
measures before rearranging (left), after rearranging (middle) and after
rearranging, estimated with a ridge penalty (right).

Fig. S6: Sample normal scores rank correlation matrix of the $91$ dysphonia
measures after rearranging, estimated via lasso with $\omega_{n}=0.623$ (top
left), adaptive lasso with $\rho_{n}=0.287$ (top right), group lasso with
$\omega_{n}=0.445$ (bottom left), and group lasso with $\omega_{n}=0.670$
(bottom right).