continuous with respect to the Lebesgue measure, we have

$\|f^{*}_{n}-f_{0}\|^{2}_{L^{2}(\nu)}\leq
18^{2}B_{0}^{2}(s+1)^{4}d^{2s+\beta\vee 1}(NM)^{-4\beta/d}.$

By Lemma 3.2, finally we have

$\mathbb{E}\|f^{*}_{n}-f_{0}\|^{2}_{L^{2}(\nu)}\leq
C\mathcal{B}^{2}\frac{\mathcal{S}\mathcal{D}\log(\mathcal{S})(\log
n)^{3}}{n}+324B_{0}^{2}(s+1)^{4}d^{2s+\beta\vee 1}(NM)^{-4\beta/d},$

where $C$ does not depend on $n,d,N,M,s,\beta,B_{0},\mathcal{D},\mathcal{B}$
or $\mathcal{S}$, and $s=\lfloor\beta\rfloor$. This completes the proof of
Theorem 4.2. ∎

###  A.6 Proof of Corollary 5.1

We prove Corollary 5.1. Corollaries 5.2 and 5.3 can be proved similarly.

###### Proof.

Under the assumptions in Theorem 4.2, for any $N,M\in\mathbb{N}^{+}$, the
function class of ReLU multi-layer perceptrons
$\mathcal{F}_{n}=\mathcal{F}_{\mathcal{D},\mathcal{W},\mathcal{U},\mathcal{S},\mathcal{B}}$
with width $\mathcal{W}=38(s+1)^{2}d^{s+1}N\lceil\log_{2}(8N)\rceil$ and depth
$\mathcal{D}=21(s+1)^{2}M\lceil\log_{2}(8M)\rceil$, the prediction error of
the ERM $\hat{f}_{n}$ satisfies

$\mathbb{E}\|\hat{f}_{n}-f_{0}\|^{2}_{L^{2}(\nu)}\leq C\mathcal{B}^{2}(\log
n)^{3}\,\frac{1}{n}\mathcal{S}\mathcal{D}\log(\mathcal{S})+324B_{0}^{2}(s+1)^{4}d^{2s+\beta\vee
1}(NM)^{-4\beta/d},$

for $2n\geq\text{Pdim}(\mathcal{F}_{n})$, where $C>0$ is a constant not
depending on $n,d,\mathcal{B},\mathcal{S},\mathcal{D},B_{0}$,
$\beta,s,r,N$ or $M$.

For deep with fixed width networks, given any $N\in\mathbb{N}^{+}$, the
network width is fixed

$\mathcal{W}=38(s+1)^{2}d^{s+1}N\lceil\log_{2}(8N)\rceil.$

Recall that for any multilayer neural network in $\mathcal{F}_{n}$, its
parameters naturally satisfy

$\max\\{\mathcal{W},\mathcal{D}\\}\leq\mathcal{S}\leq\mathcal{W}(d+1)+(\mathcal{W}^{2}+\mathcal{W})(\mathcal{D}-1)+\mathcal{W}+1\leq
2\mathcal{W}^{2}\mathcal{D}.$

Then by plugging $\mathcal{S}\leq 2\mathcal{W}^{2}\mathcal{D}$ and
$\mathcal{D}=21(s+1)^{2}M\lceil\log_{2}(8M)\rceil$, we have

$\displaystyle\mathbb{E}\|\hat{f}_{n}-f_{0}\|^{2}_{L^{2}(\nu)}$
$\displaystyle\leq C\mathcal{B}^{2}(\log
n)^{3}\,\frac{1}{n}\mathcal{W}^{2}(M\lceil\log_{2}(8M)\rceil)^{2}\log(221(s+1)^{2}M\lceil\log_{2}(8M)\rceil\mathcal{W}^{2})$
$\displaystyle\qquad+324B_{0}^{2}(s+1)^{4}d^{2s+\beta\vee 1}(NM)^{-4\beta/d}.$

Note that the first term on the right hand side is increasing in $M$ while the
second term is decreasing in $M$. To achieve the optimal rate with respect to
$n$, we need a balanced choice of $M$ such that

$(\log n)^{3}M^{2}\log(M)^{2}/n\approx M^{-4\beta/d},$

in terms of their order. This leads to the choice of $M=\lfloor
n^{d/2(d+2\beta)}\rfloor$ and the network depth and size where

$\displaystyle\mathcal{W}$ $\displaystyle=$ $\displaystyle
38(s+1)^{2}d^{s+1}N\lceil\log_{2}(8N)\rceil,$ $\displaystyle\mathcal{S}$
$\displaystyle=$ $\displaystyle O(n^{d/2(d+2\beta)}(\log n)),$

the ERM $\hat{f}_{n}\in\arg\min_{f\in\mathcal{F}_{n}}L_{n}(f)$ satisfies

$\displaystyle\mathbb{E}\|\hat{f}_{n}-f_{0}\|^{2}_{L^{2}(\nu)}\leq$
$\displaystyle\Big{\\{}c_{1}\mathcal{B}^{2}(\log
n)^{5}+324B_{0}^{2}d^{2s+\beta\vee
1}N^{-4\beta/d}\Big{\\}}(s+1)^{4}n^{-2\beta/(d+2\beta)},$ $\displaystyle\leq$
$\displaystyle c_{2}\mathcal{B}^{2}N^{-4\beta/d}(s+1)^{4}d^{2s+\beta\vee
1}n^{-2\beta/(d+2\beta)}(\log n)^{5},$

for $2n\geq\text{Pdim}(\mathcal{F}_{n})$, where $c_{1},c_{2}>0$ are constants
which do not depend on $n,\mathcal{B},B_{0},\beta,s$ or $N$. This completes
the proof. ∎

###  A.7 Proof of Theorem 6.1

###### Proof.

We project the data to a low-dimensional space and then use DNN to do
approximation the low-dimensional function where the idea is similar to that
of Theorem 1.2 in Shen, Yang and Zhang (2020). Based on Theorem 3.1 in
Baraniuk and Wakin (2009), there exists a linear projector
$A\in\mathbb{R}^{d_{\delta}\times d}$ that maps a low-dimensional manifold in
a high-dimensional space to a low-dimensional space nearly preserving the
distance. Specifically, there exists a matrix
$A\in\mathbb{R}^{d_{\delta}\times d}$ such that
$AA^{T}=(d/d_{\delta})I_{d_{\delta}}$ where $I_{d_{\delta}}$ is an identity
matrix of size $d_{\delta}\times d_{\delta}$, and

$(1-\delta)\|x_{1}-x_{2}\|_{2}\leq\|Ax_{1}-Ax_{2}\|_{2}\leq(1+\delta)\|x_{1}-x_{2}\|_{2},$

for any $x_{1},x_{2}\in\mathcal{M}.$ And it is easy to check

$A(\mathcal{M}_{\rho})\subseteq
A([0,1]^{d})\subseteq[-\sqrt{\frac{d}{d_{\delta}}},\sqrt{\frac{d}{d_{\delta}}}]^{d_{\delta}}.$

Note that for any $z\in A(\mathcal{M})$, there exists a unique
$x\in\mathcal{M}$ such that $Ax=z$. To prove this, let
$x^{\prime}\in\mathcal{M}$ be another point on $\mathcal{M}$ satisfying
$Ax^{\prime}=z$, then $(1-\delta)\|x-x^{\prime}\|_{2}\leq\|Ax-
Ax^{\prime}\|_{2}\leq(1+\delta)\|x-x^{\prime}\|_{2}$ implies that
$\|x-x^{\prime}\|_{2}=0$. Then for any $z\in A(\mathcal{M})$, define
$x_{z}=\mathcal{SL}(\\{x\in\mathcal{M}:Ax=z\\})$ where $\mathcal{SL}(\cdot)$
is a set function which returns a unique element of a set. Note that if $Ax=z$
where $x\in\mathcal{M}$ and $z\in A(\mathcal{M})$, then $x=x_{z}$ by our
argument since $\\{x\in\mathcal{M}:Ax=z\\}$ is a set with only one element
when $z\in A(\mathcal{M})$. And we can see that
$\mathcal{SL}:A(\mathcal{M})\to\mathcal{M}$ is a differentiable function with
the norm of its derivative locates in $[1/(1+\delta),1/(1-\delta)]$, since

$\frac{1}{1+\delta}\|z_{1}-z_{2}\|_{2}\leq\|x_{z_{1}}-x_{z_{2}}\|_{2}\leq\frac{1}{1-\delta}\|z_{1}-z_{2}\|_{2},$

for any $z_{1},z_{2}\in A(\mathcal{M})\subseteq E$ where
$E:=[-\sqrt{{d}/{d_{\delta}}},\sqrt{{d}/{d_{\delta}}}]^{d_{\delta}}$. For the
high-dimensional function $f_{0}:[0,1]^{d}\to\mathbb{R}^{1}$, we define its
low-dimensional representation
$\tilde{f}_{0}:\mathbb{R}^{d_{\delta}}\to\mathbb{R}^{1}$ by

$\tilde{f}_{0}(z)=f_{0}(x_{z}),\quad{\rm for\ any}\ z\in
A(\mathcal{M})\subseteq\mathbb{R}^{d_{\delta}}.$

Recall that $f_{0}\in\mathcal{H}^{\beta}([0,1]^{d},B_{0})$, then
$\tilde{f}_{0}\in\mathcal{H}^{\beta}(A(\mathcal{M}),B_{0}/(1-\delta)^{\beta})$.
Note that $\mathcal{M}$ is compact and $A$ is a linear mapping, then by the
extended version of Whitney’ extension theorem in Fefferman (2006), there
exists a function
$\tilde{F}_{0}\in\mathcal{H}^{\beta}(E,B_{0}/(1-\delta)^{\beta})$ such that
$\tilde{F}_{0}(z)=\tilde{f}_{0}(z)$ for any $z\in A(\mathcal{M})$. With
$E=[-\sqrt{{d}/{d_{\delta}}},\sqrt{{d}/{d_{\delta}}}]^{d_{\delta}}$, by
Theorem 3.3, for any $N,M\in\mathbb{N}^{+}$, there exists a function
$\tilde{f}_{n}:\mathbb{R}^{d_{\delta}}\to\mathbb{R}^{1}$ implemented by a ReLU
FNN with width
$\mathcal{W}=38(s+1)^{2}d_{\delta}^{s+1}N\lceil\log_{2}(8N)\rceil$ and depth
$\mathcal{D}=21(s+1)^{2}M\lceil\log_{2}(8M)\rceil$ such that

$|\tilde{f}_{n}(z)-\tilde{F}_{0}(z)|\leq
36\frac{B_{0}}{(1-\delta)^{\beta}}(s+1)^{2}d^{1/2}d_{\delta}^{3s/2}(NM)^{-2\beta/d_{\delta}},$

for all $z\in E\backslash\Omega(E)$ where $\Omega(E)$ is a subset of $E$ with
an arbitrarily small Lebesgue measure as well as
$\Omega:=\\{x\in\mathcal{M}_{\rho}:Ax\in\Omega(E)\\}$ does.

If we define $f^{*}_{n}=\tilde{f}_{n}\circ A$ which is
$f^{*}_{n}(x)=\tilde{f}_{n}(Ax)$ for any $x\in[0,1]^{d}$, then
$f^{*}_{n}\in\mathcal{F}_{\mathcal{D},\mathcal{W},\mathcal{U},\mathcal{S},\mathcal{B}}$
is also a ReLU FNN with the same parameter as $\tilde{f}_{n}$. For any
$x\in\mathcal{M}_{\rho}\backslash\Omega$ and $z=Ax$, there exists a
$\tilde{x}\in\mathcal{M}$ such that $\|x-\tilde{x}\|_{2}\leq\rho$, then

$\displaystyle|f^{*}_{n}(x)-f_{0}(x)|=|\tilde{f}_{n}(Ax)-\tilde{F}_{0}(Ax)+\tilde{F}_{0}(Ax)-\tilde{F}_{0}(A\tilde{x})+\tilde{F}_{0}(A\tilde{x})-f_{0}(x)|$
$\displaystyle\leq|\tilde{f}_{n}(Ax)-\tilde{F}_{0}(Ax)|+|\tilde{F}_{0}(Ax)-\tilde{F}_{0}(A\tilde{x})|+|\tilde{F}_{0}(A\tilde{x})-f_{0}(x)|$
$\displaystyle\leq
36\frac{B_{0}}{(1-\delta)^{\beta}}(s+1)^{2}d^{1/2}d_{\delta}^{3s/2}(NM)^{-2\beta/d_{\delta}}+\frac{B_{0}}{1-\delta}\|Ax-A\tilde{x}\|_{2}+|f_{0}(\tilde{x})-f_{0}(x)|$
$\displaystyle\leq
36\frac{B_{0}}{(1-\delta)^{\beta}}(s+1)^{2}d^{1/2}d_{\delta}^{3s/2}(NM)^{-2\beta/d_{\delta}}+\frac{\rho
B_{0}}{1-\delta}\sqrt{\frac{d}{d_{\delta}}}+\rho B_{0}$
$\displaystyle=36\frac{B_{0}}{(1-\delta)^{\beta}}(s+1)^{2}d^{1/2}d_{\delta}^{3s/2}(NM)^{-2\beta/d_{\delta}}+\rho
B_{0}\\{(1-\delta)^{-1}\sqrt{{d}/{d_{\delta}}}+1\\}$
$\displaystyle\leq(36+C_{2})\frac{B_{0}}{(1-\delta)^{\beta}}(s+1)^{2}d^{1/2}d_{\delta}^{3s/2}(NM)^{-2\beta/d_{\delta}},$

where $C_{2}>0$ is a constant not depending on any parameter. The last
inequality follows from $\rho\leq
C_{2}(NM)^{-2\beta/d_{\delta}}(s+1)^{2}d^{1/2}d_{\delta}^{3s/2}\\{\sqrt{{d}/{d_{\delta}}}+1-\delta\\}^{-1}(1-\delta)^{1-\beta}$.
Since the probability measure $\nu$ of $X$ is absolutely continuous with
respect to the Lebesgue measure, we have

$\|f^{*}_{n}-f_{0}\|^{2}_{L^{2}(\nu)}\leq(36+C_{2})^{2}\frac{B_{0}^{2}}{(1-\delta)^{2\beta}}(s+1)^{4}dd_{\delta}^{3s}(NM)^{-4\beta/d_{\delta}},$
(C.14)

where $d_{\delta}=O(d_{\mathcal{M}}{\log(d/\delta)}/{\delta^{2}})$ is assumed
to satisfy $d_{\delta}\ll d$. By Lemma 3.2, we have

$\displaystyle\mathbb{E}\|\hat{f}_{n}-f_{0}\|^{2}_{L^{2}(\nu)}$
$\displaystyle\leq
C_{1}\mathcal{B}^{2}\frac{\mathcal{S}\mathcal{D}\log(\mathcal{S})(\log
n)^{3}}{n}+(36+C_{2})^{2}\frac{B_{0}^{2}}{(1-\delta)^{2\beta}}(\lfloor\beta\rfloor+1)^{4}dd_{\delta}^{3\lfloor\beta\rfloor}(NM)^{-4\beta/d_{\delta}},$

where $C_{1},C_{2}>0$ are constants that do not depend on
$n,\mathcal{B},\mathcal{S},\mathcal{D},B_{0},\beta,\delta,N$ or $M$,
$\lfloor\beta\rfloor=s$ is the biggest integer strictly smaller than $\beta$.
This completes the proof of Theorem 6.1.

∎

###  A.8 Proof of Theorem 6.2

To facilitate the proof, we first briefly review manifolds, partition of
unity, and function spaces defined on smooth manifolds. Details can be found
in Chen et al. (2019), Tu (2011), Lee (2006), Federer (1959) and Aamari et al.
(2019).

######  Definition A.1 (Chart).

Let $\mathcal{M}$ be a $d_{\mathcal{M}}$-dimensional Riemannian manifold
isometrically embedded in $\mathbb{R}^{d}$. A chart for $\mathcal{M}$ is a
pair $(U,\phi)$ such that $U\subset\mathcal{M}$ is open and
$\phi:U\mapsto\mathbb{R}^{d_{\mathcal{M}}}$, where $\phi$ is a homeomorphism,
i.e., bijective, $\phi$ and $\phi^{-1}$ are both continuous.

We say two charts $(U,\phi)$ and $(V,\psi)$ on $\mathcal{M}$ are $C^{k}$
compatible if and only if the transition functions,

$\phi\circ\psi^{-1}:\psi(U\cap V)\mapsto\phi(U\cap V)\quad{\rm
and}\quad\psi\circ\phi^{-1}:\phi(U\cap V)\mapsto\psi(U\cap V)$

are both $C^{k}$.

######  Definition A.2 ($C^{k}$ Atlas).

A $C^{k}$ atlas for $\mathcal{M}$ is a collection of pairwise $C^{k}$
compatible charts $\\{(U_{i},\phi_{i})\\}_{i\in\mathcal{A}}$ such that
$\bigcup_{i\in\mathcal{A}}U_{i}=\mathcal{M}$.

######  Definition A.3 (Smooth manifold).

A smooth manifold is a manifold together with a $C^{\infty}$ atlas.

######  Definition A.4 (Hölder functions on $\mathcal{M}$).

Let $\mathcal{M}$ be a $d_{\mathcal{M}}$-dimensional Riemannian manifold
isometrically embedded in $\mathbb{R}^{d}$. Let
$\\{(U_{i},P_{i})\\}_{i\in\mathcal{A}}$ be an atlas of $\mathcal{M}$ where the
$P_{i}^{\prime}s$ are orthogonal projections onto tangent space. For a
positive number $\beta>0$, a function $f:\mathcal{M}\mapsto\mathbb{R}$
belonging to Hölder class $\mathcal{H}^{\beta}(\mathcal{M},B_{0})$ is
$\beta$-Hölder smooth with constant $B_{0}$ if for each chart $(U_{i},P_{i})$
in the atlas, we have

  * 1.

$f\circ P_{i}^{-1}\in C^{s}$ with $\max_{\|\alpha\|_{1}\leq
s}|\partial^{\alpha}f(x)|\leq B_{0}$ for any $x\in U_{i}$.

  * 2.

For any $\|\alpha\|_{1}=s$ and $x,y\in U_{i}$,

$\sup_{x\not=y}\frac{|\partial^{\alpha}f(x)-\partial^{\alpha}f(y)|}{\|x-y\|_{2}^{r}}\leq
B_{0},$

where $s$ is the largest integer strictly smaller than $\beta$ and
$r=\beta-s$.

######  Definition A.5 (Partition of Unity, Definition 13.4 in Tu (2011)).

A $C^{\infty}$ partition of unity on a manifold $\mathcal{M}$ is a collection
of nonnegative $C^{\infty}$ functions
$\rho_{i}:\mathcal{M}\mapsto\mathbb{R}^{+}$ for $i\in\mathcal{A}$ such that

  * 1.

The collection of the supports, $\\{{\rm supp}(\rho_{i})\\}_{i\in\mathcal{A}}$
is locally finite, i.e., every point on $\mathcal{M}$ has a neighborhood that
meets only finitely many of ${\rm supp}(\rho_{i})$’s.

  * 2.

$\sum_{i\in\mathcal{A}}\rho_{i}=1$.

By Theorem 13.7 in Tu (2011), a $C^{\infty}$ partition of unity always exists
for a smooth manifold. This gives a decomposition
$f=\sum_{i\in\mathcal{A}}f_{i}$ with $f_{i}=f\rho_{i}$ and each $f_{i}$ has
the same regularity as $f$ since
$f_{i}\circ\phi_{i}^{-1}=(f\circ\phi_{i}^{-1})\times(\rho_{i}\circ\phi_{i}^{-1})$
for a chart $(U_{i},\phi_{i})$. And the decomposition means that we can
express $f$ as a sum of the $f_{i}$’s with each $f_{i}$ is only supported in a
single chart.

Our approach builds on the methods of Schmidt-Hieber (2019); Chen, Jiang and
Zhao (2019) and Chen et al. (2019) but there are some noteworthy new aspects:
(a) we apply linear coordinate maps instead of smooth coordinate maps, where
the linear coordinate maps can be exactly represented by shallow ReLU networks
without error; (b) we do not require the smoothness index of each coordinate
map and each function in the partition of unity to be no less than $\beta
d/d_{\mathcal{M}}$, which depends on the ambient dimension $d$ and can be
large; (c) we apply our new approximation result when approximating the low-
dimensional Hölder smooth functions on each projected chart, which leads to a
better prefactor of error compared to most existing results

###### Proof.

We prove Theorem 6.2 in three steps: (1) we first construct an finite atlas
that covers the manifold $\mathcal{M}$; (2) we project each chart linearly to
a $d_{\mathcal{M}}$-dimensional hypercube on which we approximate the low-
dimensional Hölder smooth functions respectively; (3) lastly, we combine the
approximation results on all charts to get a error bound of the approximation
on the whole manifold.

Step 1: Atlas Construction and Projection.
Let $B(x,r)$ denote the open Euclidean ball with radius $r>0$ and center
$x\in\mathbb{R}^{d}$. Given any $r>0$, we have an open cover
$\\{B(x,r)\\}_{x\in\mathcal{M}}$ of $\mathcal{M}$. By the compactness of
$\mathcal{M}$, there exists a finite cover
$\\{B(x_{i},r)\\}_{i=1,\ldots,C_{\mathcal{M}}}$ for some finite integer
$C_{\mathcal{M}}$ such that $\mathcal{M}\subset\bigcup_{i}B(x_{i},r)$. Let
$(1/\tau)$ denote the condition number of $\mathcal{M}$, then we can choose
proper radius $r<\tau/2$ such that $U_{i}=\mathcal{M}\cap B(x_{i},r)$ is
diffeomorphic to a ball in $\mathbb{R}^{d_{\mathcal{M}}}$ (Niyogi, Smale and
Weinberger, 2008). The definition and detailed introduction of condition
number (or its inverse called “reach”) can be found in Federer (1959) and
Aamari et al. (2019). Besides, the number of charts $C_{\mathcal{M}}$
satisfies

$C_{\mathcal{M}}\leq\lceil
S_{(\mathcal{M})}T_{d_{\mathcal{M}}}/r^{d_{\mathcal{M}}}\rceil,$

where $S_{\mathcal{M}}$ is the area of the surface of $\mathcal{M}$ and
$T_{d_{\mathcal{M}}}$ is the thickness of $U_{i}$’s, which is defined as the
average number of $U_{i}$’s that contain a point on $\mathcal{M}$. By equation
(19) in Chapter 2 of Conway and Sloane (2013), the thickness
$T_{d_{\mathcal{M}}}$ scales approximately linear in $d_{\mathcal{M}}$ and
there exist coverings such that $T_{d_{\mathcal{M}}}\leq
d_{\mathcal{M}}\log(d_{\mathcal{M}})+d_{\mathcal{M}}\log\log(d_{\mathcal{M}})+5d_{\mathcal{M}}\leq
7d_{\mathcal{M}}\log(d_{\mathcal{M}})$. Let the tangent space of $\mathcal{M}$
at $x_{i}$ be denoted by $T_{x_{i}}(\mathcal{M})$ and let
$V_{i}\in\mathbb{R}^{d\times d_{\mathcal{M}}}$ be the matrix concatenating the
orthonormal basis of the tangent space as column vectors. Then for any $x\in
U_{i}$ we can define the projection

$\phi_{i}(x)=a_{i}(V_{i}^{\top}(x-x_{i})+b_{i}),$

where $a_{i}\in(0,1]$ and $b_{i}$ are proper scalar and vector such that
$\phi_{i}(x)\in[0,1]^{d_{\mathcal{M}}}$ for any $x\in U_{i}$. Note that each
projection $\phi_{i}$ is a linear function, which can be computed by a one-
hidden layer ReLU network.

Step 2: Approximate low-dimensional functions.
For charts $\\{(U_{i},\phi_{i})\\}_{i=1}^{C_{\mathcal{M}}}$, we can
approximate the function on each chart by approximation the projected function
in the low-dimensional space. By Theorem 13.7 in Tu (2011), the target
function $f$ can be written as

$f=\sum_{i=1}^{C_{\mathcal{M}}}f\rho_{i}:=\sum_{i=1}^{C_{\mathcal{M}}}f_{i},$

where $\rho_{i}$’s are elements in $C^{\infty}$ partition of unity on
$\mathcal{M}$ being supported in $U_{i}$’s. Note that the manifold
$\mathcal{M}$ is compact and smooth and $\rho_{i}$’s are $C^{\infty}$, then
$f_{i}$’s have the same smoothness as $f$ itself for
$i=1,\ldots,C_{\mathcal{M}}$. Note that the collection of the supports,
$\\{{\rm supp}(\rho_{i})\\}_{i\in\mathcal{A}}$ is locally finite, and let
$C_{\rho}$ denote the maximum number of ${\rm supp}(\rho_{i})$’s that a point
on $\mathcal{M}$ can belong to. Besides, since each $\phi_{i}$ is linear
projection operator, it is not hard to show that each
$f_{i}\circ\phi_{i}^{-1}$ is a Hölder smooth function with order $\beta>0$ on
$\phi_{i}(U_{i})\subset[0,1]^{d_{\mathcal{M}}}$, i.e.,
$f_{i}\circ\phi_{i}^{-1}\in\mathcal{H}^{\beta}(\phi_{i}(U_{i}),\sqrt{d/d_{\mathcal{M}}}B_{0})$
for $i=1,\ldots,C_{\mathcal{M}}$. A detailed proof can be found in Lemma 2 of
Chen et al. (2019). By the extended version of Whitney’ extension theorem in
Fefferman (2006), we can approximate the smooth extension of
$f_{i}\circ\phi_{i}^{-1}$ on $[0,1]^{d_{\mathcal{M}}}$. By Corollary 3.1, for
any $M,N\in\mathbb{N}^{+}$, there exists a function $g_{i}$ implemented by a
ReLU network with width
$\mathcal{W}=38(\lfloor\beta\rfloor+1)^{2}3^{d_{\mathcal{M}}}(d_{\mathcal{M}})^{\lfloor\beta\rfloor+1}N\lceil\log_{2}(8N)\rceil$
and depth
$\mathcal{D}=21(\lfloor\beta\rfloor+1)^{2}M\lceil\log_{2}(8M)\rceil+2d_{\mathcal{M}}$
such that

$|f_{i}\circ\phi_{i}^{-1}(x)-g_{i}(x)|\leq
19\sqrt{d/d_{\mathcal{M}}}B_{0}(\lfloor\beta\rfloor+1)^{2}(d_{\mathcal{M}})^{\lfloor\beta\rfloor+(\beta\vee
1)/2}(NM)^{-2\beta/d_{\mathcal{M}}},$

for any $x\in\phi_{i}(U_{i})\subset[0,1]^{d_{\mathcal{M}}}$.

Step 3: Approximate the target function on the manifold.
By construction of subnetworks, the projected target functions
$f_{i}\circ\phi_{i}^{-1}$ on each region $\phi_{i}(U_{i})$ can be approximated
by ReLU networks $g_{i}$. Note that each projection $\phi_{i}$ is a linear
function can be computed by a one-hidden layer ReLU network. Then we stack two
more layer to $g_{i}$ and get $\tilde{g}_{i}=g_{i}\circ\phi_{i}$ such that for
any $x\in U_{i}$,

$|f_{i}(x)-\tilde{g}_{i}(x)|=|f_{i}(x)-g_{i}\circ\phi_{i}(x)|\leq
19B_{0}(\lfloor\beta\rfloor+1)^{2}d^{1/2}(d_{\mathcal{M}})^{\lfloor\beta\rfloor+\beta/2}(NM)^{-2\beta/d_{\mathcal{M}}},$

where $\tilde{g}_{i}$ is a ReLU activated network with width
$\mathcal{W}=38(\lfloor\beta\rfloor+1)^{2}3^{d_{\mathcal{M}}}(d_{\mathcal{M}})^{\lfloor\beta\rfloor+1}N\lceil\log_{2}(8N)\rceil$
and depth
$\mathcal{D}=21(\lfloor\beta\rfloor+1)^{2}M\lceil\log_{2}(8M)\rceil+2d_{\mathcal{M}}+2$.
Since there are $C_{\mathcal{M}}$ charts, we parallel these subnetworks
$\tilde{g}_{i}$ to get $\tilde{g}=\sum_{i=1}^{C_{\mathcal{M}}}\tilde{g}_{i}$
such that

$\displaystyle|f(x)-\tilde{g}(x)|$
$\displaystyle=\left|\sum_{i=1}^{C_{\mathcal{M}}}f_{i}(x)-\sum_{i=1}^{C_{\mathcal{M}}}\tilde{g}_{i}(x)\right|$
$\displaystyle\leq
C_{\rho}\max_{i=1,\ldots,C_{\mathcal{M}}}|f_{i}(x)-\tilde{g}_{i}(x)|$
$\displaystyle\leq
19C_{\rho}B_{0}(\lfloor\beta\rfloor+1)^{2}d^{1/2}(d_{\mathcal{M}})^{\lfloor\beta\rfloor+\beta/2}(NM)^{-2\beta/d_{\mathcal{M}}},$

for any $x\in\mathcal{M}$. Such a neural network $\tilde{g}$ has width
$\mathcal{W}=38C_{\mathcal{M}}(\lfloor\beta\rfloor+1)^{2}3^{d_{\mathcal{M}}}(d_{\mathcal{M}})^{\lfloor\beta\rfloor+1}N\lceil\log_{2}(8N)\rceil$
and depth
$\mathcal{D}=21(\lfloor\beta\rfloor+1)^{2}M\lceil\log_{2}(8M)\rceil+2d_{\mathcal{M}}+2$.
Recall that $C_{\mathcal{M}}\leq\lceil
S_{(\mathcal{M})}T_{d_{\mathcal{M}}}/r^{d_{\mathcal{M}}}\rceil\leq\lceil
7S_{(\mathcal{M})}d_{\mathcal{M}}\log(d_{\mathcal{M}})/r^{d_{\mathcal{M}}}\rceil\leq
C_{1}S_{(\mathcal{M})}(2/\tau)^{d_{\mathcal{M}}}d_{\mathcal{M}}\log(d_{\mathcal{M}})$
for some universal constant $C_{1}>0$, then width $\mathcal{W}\leq
266(\lfloor\beta\rfloor+1)^{2}\lceil
S_{\mathcal{M}}(6/\tau)^{d_{\mathcal{M}}}\rceil(d_{\mathcal{M}})^{\lfloor\beta\rfloor+2}N\lceil\log_{2}(8N)\rceil$.
Then we have

$\displaystyle|f(x)-\tilde{g}(x)|$ $\displaystyle\leq
C_{2}B_{0}(\lfloor\beta\rfloor+1)^{2}d^{1/2}(d_{\mathcal{M}})^{3\lfloor\beta\rfloor/2+1/2}(NM)^{-2\beta/d_{\mathcal{M}}},$

where $C_{2}>0$ is some constant not depending on
$n,d,d_{\mathcal{M}},N,M,\beta,B_{0}$ and $\tau$. And combining Lemma 3.2, we
have

$\mathbb{E}\|\hat{f}_{n}-f_{0}\|^{2}_{L^{2}(\nu)}\leq
C_{1}\mathcal{B}^{2}\frac{\mathcal{S}\mathcal{D}\log(\mathcal{S})(\log
n)^{3}}{n}+C_{3}{B_{0}^{2}}(\lfloor\beta\rfloor+1)^{4}d(d_{\mathcal{M}})^{3\lfloor\beta\rfloor+1}(NM)^{-4\beta/d_{\mathcal{M}}},$

where $C_{2}>0$ is some constant not depending on
$n,d,d_{\mathcal{M}},\mathcal{B},\mathcal{S},\mathcal{D},N,M,\beta,B_{0},\tau$
and $S_{\mathcal{M}}$. This completes the proof of Theorem 6.2. ∎

###  A.9 Proof of Theorem 6.3

###### Proof.

Let $E\subset\mathbb{R}^{d}$ be the support of $X$ with Minkowski dimension
$d^{*}\equiv{\rm dim}_{M}(E)$. Let
$T=\\{(x_{1}-x_{2})/\|x_{1}-x_{2}\|_{2}:x_{1},x_{2}\in\bar{E}\\}$ be the
standardized difference of set $\bar{E}$ where $\bar{E}$ is the closure of
$E$. By Lemma B.5, there exists an absolute constant $\kappa$, a realization
of random projection with entries i.i.d from Rademacher random variables
$A:\mathbb{R}^{d}\to\mathbb{R}^{d_{0}}$ such that for all
$\tau,\delta\in(0,1)$ if $d\geq
d_{0}\geq\kappa(\gamma^{2}(T)+\log(2/\tau))/\delta^{2}$,

$(1-\delta)\|x_{1}-x_{2}\|_{2}^{2}\leq\|Ax_{1}-Ax_{2}\|_{2}^{2}\leq(1+\delta)\|x_{1}-x_{2}\|_{2}^{2},$

for all $x_{1},x_{2}\in\bar{E}$, where $\gamma(T)$ is defined in Lemma B.5.
Note that every covering (by closed balls) of $E$ is also a covering of
$\bar{E}$, which implies ${\rm dim}_{M}(E)={\rm dim}_{M}(\bar{E})=d^{*}$. And
$\gamma(T)$ is also related to $d^{*}$ the intrinsic dimension of $E$. More
exactly, let $N_{0}=\mathcal{N}(\varepsilon,\|\cdot\|_{2},\bar{E})$ be the
covering number of $\bar{E}$ with radius $\varepsilon$ and
$C_{E}=\\{x_{i}\\}_{i=1}^{N_{0}}\subset\bar{E}$ be the set of anchor points.
Then for any $x\in\bar{E}$, there exists a $x_{i}$ such that
$\|x-x_{i}\|_{2}\leq\varepsilon.$ For the difference set
$\bar{E}-\bar{E}:=\\{x_{1}-x_{2}:x_{1},x_{2}\in\bar{E}\\}$, we can construct a
$2\varepsilon$-covering with $N_{0}^{2}$ anchor points
$\\{x_{1}-x_{2}:x_{1},x_{2}\in C_{E}\\}$. For any $y\in\bar{E}-\bar{E}$, there
exists $x,x^{\prime}\in\bar{E}$ such that $y=x-x^{\prime}$. And there exists
$x_{1},x_{2}\in C_{E}$ such that $\|x-x_{1}\|_{2}\leq\varepsilon$ and
$\|x^{\prime}-x_{2}\|_{2}\leq\varepsilon$. Then let $y^{\prime}=x_{1}-x_{2}$,
we have
$\|y-y^{\prime}\|_{2}=\|(x-x^{\prime})-(x_{1}-x_{2})\|_{2}\leq\|x-x_{1}\|_{2}+\|x^{\prime}-x_{2}\|_{2}\leq
2\varepsilon.$ This shows that
$\mathcal{N}(2\varepsilon,\|\cdot\|_{2},\bar{E}-\bar{E})\leq\mathcal{N}(\varepsilon,\|\cdot\|_{2},\bar{E})^{2}=N_{0}^{2}$,
and ${\rm dim}_{M}(\bar{E}-\bar{E})\leq 2d^{*}$. Let $\bar{T}$ denote the
bounded set $\bar{E}-\bar{E}$. Now we derive the relationship between the
covering number of $\bar{T}$ and that of $T$. Firstly, given any real number
$\delta>0$, we consider the subset
$\bar{T}_{\delta}:=\\{\bar{t}\in\bar{T}:\|\bar{t}\|_{2}\geq\delta\\}$ and
$T_{\delta}:=\\{\bar{t}/\|\bar{t}\|_{2}:\bar{t}\in\bar{T}_{\delta}\\}$. We
scale up the set
$\bar{T}_{\delta}:=\\{\bar{t}\in\bar{T}_{\delta}:\|\bar{t}\|_{2}\geq\delta\\}$
by $1/\delta$ times to get
$\frac{1}{\delta}\bar{T}_{\delta}:=\\{\bar{t}/\delta:\bar{t}\in\bar{T}_{\delta}\\}$.
By the definition of the Minkowski dimension (with respect the covering
number) and the property of scaling, it is easy to see that the
$\epsilon$-covering number of $\frac{1}{\delta}\bar{T}_{\delta}$ is no more
than $(1/\delta)^{2d^{*}}$ times larger than that of $\bar{T}_{\delta}$ since
${\rm dim}_{M}(\bar{T}_{\delta})\leq{\rm dim}_{M}(\bar{T})\leq 2d^{*}$, i.e.
for each $\delta>0$ we have,

$\displaystyle\mathcal{N}(\epsilon,\|\cdot\|_{2},\frac{1}{\delta}\bar{T}_{\delta})$
$\displaystyle\leq(1/\delta)^{2d^{*}}\mathcal{N}(\epsilon,\|\cdot\|_{2},\bar{T}_{\delta})$
$\displaystyle\leq(1/\delta)^{2d^{*}}\mathcal{N}(\epsilon,\|\cdot\|_{2},\bar{T})$
$\displaystyle\leq c_{0}(1/\delta)^{2d^{*}}(1/\epsilon)^{2d^{*}},$

where $c_{0}>0$ is a constant not depending on $d^{*},\epsilon$ and $\delta$.
This implies ${\rm dim}_{M}(\frac{1}{\delta}\bar{T}_{\delta})\leq 2d^{*}+{\rm
dim}_{M}(\bar{T}_{\delta})$ for $\delta>0$.

Now $\delta>0$, we link the Minkowski dimension of
$\frac{1}{\delta}\bar{T}_{\delta}$ to that $T_{\delta}$. Given any $\epsilon$,
suppose $\bar{t}_{1},\ldots,\bar{t}_{m}$ are the anchor points of a minimal
$\epsilon$-cover of $\frac{1}{\delta}\bar{T}_{\delta}$. By the definition of
covering, for any $\bar{t}\in\frac{1}{\delta}\bar{T}_{\delta}$, there exists
an anchor point $\bar{t}_{i}$ for some $i\in\\{1,\ldots,m\\}$ such that
$\|\bar{t}-\bar{t}_{i}\|_{2}\leq\epsilon$. Since
$\bar{t},\bar{t}_{i}\in\frac{1}{\delta}\bar{T}_{\delta}$, we have
$\|\bar{t}\|_{2}\geq 1,\|\bar{t}_{i}\|_{2}\geq 1$, and

$\displaystyle\Big{\|}\frac{\bar{t}}{\|\bar{t}\|_{2}}-\frac{\bar{t}_{i}}{\|\bar{t}_{i}\|_{2}}\Big{\|}_{2}$
$\displaystyle\leq\Big{\|}\frac{\bar{t}}{\|\bar{t}\|_{2}}-\frac{\bar{t}}{\|\bar{t}_{i}\|_{2}}+\frac{\bar{t}}{\|\bar{t}_{i}\|_{2}}-\frac{\bar{t}_{i}}{\|\bar{t}_{i}\|_{2}}\Big{\|}_{2}$
$\displaystyle\leq\Big{|}\frac{1}{\|\bar{t}\|_{2}}-\frac{1}{\|\bar{t}_{i}\|_{2}}\Big{|}\|\bar{t}\|_{2}+\frac{\|\bar{t}-\bar{t}_{i}\|_{2}}{\|\bar{t}_{i}\|_{2}}$
$\displaystyle\leq\Big{|}\frac{\epsilon}{\|\bar{t}\|_{2}(\|\bar{t}\|_{2}+\epsilon)}\Big{|}\|\bar{t}\|_{2}+\frac{\epsilon}{\|\bar{t}_{i}\|_{2}}$
$\displaystyle\leq 2\epsilon.$

Thus the ball around ${\bar{t}_{i}}/{\|\bar{t}_{i}\|_{2}}$ with radius
$2\epsilon$ covers ${\bar{t}}/{\|\bar{t}\|_{2}}$, which implies
$\mathcal{N}(2\epsilon,\|\cdot\|_{2},T_{\delta})\leq\mathcal{N}(\epsilon,\|\cdot\|_{2},\frac{1}{\delta}\bar{T}_{\delta})$.
Then ${\rm dim}_{M}(T_{\delta})\leq{\rm
dim}_{M}(\frac{1}{\delta}\bar{T}_{\delta})\leq 2d^{*}+{\rm
dim}_{M}(\bar{T}_{\delta})\leq 2d^{*}+{\rm dim}_{M}(\bar{T})\leq 4d^{*}$.
Since $\lim_{\delta\downarrow 0}\bar{T}_{\delta}=\bar{T}$ and
$\lim_{\delta\downarrow 0}{T}_{\delta}={T}$ are both bounded, then

$\displaystyle\sqrt{H(\varepsilon,\|\cdot\|_{2},T)}$
$\displaystyle=\sqrt{\log(\mathcal{N}(\varepsilon,\|\cdot\|_{2},T))}\leq
c_{1}\sqrt{d^{*}\log(1/\varepsilon)},$

for some constant $c_{1}>0$. Then by the definition of
$\gamma(T)=\int_{0}^{1}\sqrt{H(\varepsilon,\|\cdot\|_{2},T)}d\varepsilon$, we
know $\gamma^{2}(T)=cd^{*}$ for some constant $c>0$. And
$d_{0}\geq\kappa(\gamma^{2}(T)+\log(2/\tau))/\delta^{2}=\kappa(cd^{*}+\log(2/\tau))/\delta^{2}$.

Since each entry of $A$ is either $1$ or $-1$, then $A(E)\subseteq
A([0,1]^{d})\subseteq H:=[-\sqrt{dd_{0}},\sqrt{dd_{0}}]^{d_{0}}$. Note that
for any $z\in A(\bar{E})$, there exists a unique $x\in\bar{E}$ such that
$Ax=z$. To prove this, let $x^{\prime}\in\bar{E}$ be another point in
$\bar{E}$ satisfying $Ax^{\prime}=z$, then
$(1-\delta)\|x-x^{\prime}\|^{2}_{2}\leq\|Ax-
Ax^{\prime}\|_{2}^{2}\leq(1+\delta)\|x-x^{\prime}\|^{2}_{2}$ implies that
$\|x-x^{\prime}\|_{2}=0$. Then we can define a one-one map $\mathcal{SL}$ from
$A(\bar{E})$ to $\bar{E}$, i.e. $x_{z}=\mathcal{SL}(\\{x\in\bar{E}:Ax=z\\})$.
And we can see that $\mathcal{SL}:A(\bar{E})\to\bar{E}$ is a differentiable
function with the norm of its derivative locates in
$[\sqrt{1/(1+\delta)},\sqrt{1/(1-\delta)}]$, since

$\frac{1}{1+\delta}\|z_{1}-z_{2}\|_{2}^{2}\leq\|x_{z_{1}}-x_{z_{2}}\|_{2}^{2}\leq\frac{1}{1-\delta}\|z_{1}-z_{2}\|_{2}^{2},$

for any $z_{1},z_{2}\in A(\bar{E})$. For the high-dimensional function
$f_{0}:[0,1]^{d}\to\mathbb{R}^{1}$, we define its low-dimensional
representation $\tilde{f}_{0}:\mathbb{R}^{d_{0}}\to\mathbb{R}^{1}$ by

$\tilde{f}_{0}(z)=f_{0}(x_{z}),\quad{\rm for\ any}\ z\in
A(\bar{E})\subseteq\mathbb{R}^{d_{0}},$

with
$\tilde{f}_{0}\in\mathcal{H}^{\beta}(A(\bar{E}),B_{0}/(1-\delta)^{\beta/2})$
recalling that $f_{0}\in\mathcal{H}^{\beta}([0,1]^{d},B_{0})$. By the extended
version of Whitney’ extension theorem in Fefferman (2006), there exists a
function $\tilde{F}_{0}\in\mathcal{H}^{\beta}(H,B_{0}/(1-\delta)^{\beta/2})$
such that $\tilde{F}_{0}(z)=\tilde{f}_{0}(z)$ for any $z\in A(\bar{E})$. With
$H=[-\sqrt{dd_{0}},\sqrt{dd_{0}}]^{d_{0}}$, by Corollary 3.1, for any
$N,M\in\mathbb{N}^{+}$, there exists a function
$\tilde{f_{n}}:\mathbb{R}^{d_{0}}\to\mathbb{R}^{1}$ implemented by a ReLU FNN
with width
$\mathcal{W}=38(\lfloor\beta\rfloor+1)^{2}3^{d_{0}}d_{0}^{\lfloor\beta\rfloor+1}N\lceil\log_{2}(8N)\rceil$
and depth
$\mathcal{D}=21(\lfloor\beta\rfloor+1)^{2}M\lceil\log_{2}(8M)\rceil+2d_{0}$
such that

$|\tilde{f}_{n}(z)-\tilde{F}_{0}(z)|\leq
c_{2}\frac{B_{0}}{(1-\delta)^{\beta/2}}(\lfloor\beta\rfloor+1)^{2}d^{1/2}d_{0}^{\lfloor\beta\rfloor+(\beta\vee
1+1)/2}(NM)^{-2\beta/d_{0}},$

for any $z\in H$ where $c_{2}$ is a constant not depending on
$d,d_{0},\beta,N$ or $M$. If we define $f^{*}_{n}=\tilde{f}_{n}\circ A$ which
is $f^{*}_{n}(x)=\tilde{f}_{n}(Ax)$ for any $x\in[0,1]^{d}$, then
$f^{*}_{n}\in\mathcal{F}_{\mathcal{D},\mathcal{W},\mathcal{S},\mathcal{B}}$ is
also a ReLU FNN with the same parameter as $\tilde{f}_{n}$. For any $x\in{E}$,

$\displaystyle|f^{*}_{n}(x)-f_{0}(x)|$
$\displaystyle=|\tilde{f}_{n}(Ax)-\tilde{F}_{0}(Ax)|$ $\displaystyle\leq
c_{2}\frac{B_{0}}{(1-\delta)^{\beta/2}}(\lfloor\beta\rfloor+1)^{2}d^{1/2}d_{0}^{\lfloor\beta\rfloor+(\beta\vee
1+1)/2}(NM)^{-2\beta/d_{0}}.$

Combining with Lemma 3.2, since $X$ is supported on $E$, we have

$\mathbb{E}\|\hat{f}_{n}-f_{0}\|^{2}_{L^{2}(\nu)}\leq
C_{1}\mathcal{B}^{2}\frac{\mathcal{S}\mathcal{D}\log(\mathcal{S})(\log
n)^{3}}{n}+C_{2}\frac{B_{0}^{2}}{(1-\delta)^{\beta}}(\lfloor\beta\rfloor+1)^{4}dd_{0}^{2\lfloor\beta\rfloor+\beta\vee
1+1}(NM)^{-4\beta/d_{0}},$

where $d_{0}\geq\kappa d^{*}/\delta^{2}=O(d^{*}/\delta^{2})$ for some
constants $\kappa>0$ and $C_{1},C_{2}>0$ are constants that do not depend on
$n,d,d_{0}\mathcal{B},\mathcal{S},\mathcal{D},B_{0},\beta,\kappa,\delta,N$ or
$M$, $\lfloor\beta\rfloor=s$ is the biggest integer strictly smaller than
$\beta$. This completes the proof of Theorem 6.3. ∎

##  Appendix B Supporting Lemmas

For ease of reference, we collect several existing results that we used in our
proofs.

######  Lemma B.1 (Proposition 4.3. in Lu et al. (2021)).

For any $N,M,d\in\mathbb{N}^{+}$ and $\delta\in(0,3K]$ with $K=\lfloor
N^{1/d}\rfloor^{2}\lfloor M^{2/d}\rfloor$, there exists a one-dimensional
function $\phi$ implemented by a ReLU FNN with width $4\lfloor
N^{1/d}\rfloor+3$ and depth $4M+5$ such that

$\phi(x)=k,\quad{\rm if\ }x\in[\frac{k}{K},\frac{k+1}{K}-\delta\cdot
1_{k<K-1}],\ {\rm for\ }k=0,1,\ldots,K-1.$

######  Lemma B.2 (Proposition 4.4. in Lu et al. (2021)).

Given any $N,M,s\in\mathbb{N}^{+}$ and $\xi_{i}\in[0,1]$ for
$i=0,1,\ldots,N^{2}L^{2}-1$, there exists a function $\phi$ implemented by a
ReLU FNN with width $16s(N+1)\lceil\log_{2}(8N)\rceil$ and depth
$5(M+2)\lceil\log_{2}(4M)\rceil$ such that

$|\phi(i)-\xi_{i}|\leq N^{-2s}M^{-2s},\ {\rm for\ }i=0,1,\ldots,N^{2}M^{2}-1,$

and $0\leq\phi(x)\leq 1$ for any $x\in\mathbb{R}$.

The next lemma demonstrate that the production function and polynomials can be
approximated by ReLU neural networks. The basic idea is firstly to approximate
the square function using “sawtooth” functions then the production function,
which is firstly raised in Yarotsky (2017). A general polynomial can be
further approximated combining the approximated square function and production
function. The following two lemmas are more general results than those in
Yarotsky (2017).

######  Lemma B.3 (Lemma 4.2. in Lu et al. (2021)).

For any $N,M\in\mathbb{N}^{+}$, and $a,b\in\mathbb{R}$ with $a<b$, there
exists a function $\phi$ implemented by a ReLU FNN with width $9N+1$ and depth
$M$ such that

$|\phi(x,y)-xy|\leq 6(b-a)^{2}N^{-M}$

for any $x,y\in[a,b]$.

######  Lemma B.4 (Theorem 4.1 in Lu et al. (2021)).

Assume $P(x)=x^{\alpha}=x_{1}^{\alpha_{1}}x_{2}^{\alpha_{2}}\cdots
x_{d}^{\alpha_{d}}$ for $\alpha\in\mathbb{N}^{d}$ with $\|\alpha\|_{1}\leq
k\in\mathbb{N}^{+}$. For any $N,M\in\mathbb{N}^{+}$, there exists a function
$\phi$ implemented by a ReLU FNN with width $9(N+1)+k-1$ and depth $7k^{2}M$
such that

$|\phi(x)-P(x)|\leq 9k(N+1)^{-7kM},\quad{\rm for\ any\ }x\in[0,1]^{d}.$

The next lemma is a generalization of Johnson-Lindenstrauss theorem for
embedding a set with infinitely many elements, which is firstly proved in
Klartag and Mendelson (2005).

######  Lemma B.5 (Theorem 13.15 in Boucheron, Lugosi and Massart (2013)).

Let $A\subset\mathbb{R}^{d}$ and consider the random projection
$W:\mathbb{R}^{d}\to\mathbb{R}^{d_{0}}$ with its entries are independent
either standard Gaussian or Rademacher random variables. Let
$T=\\{(a_{1}-a_{2})/\|a_{1}-a_{2}\|_{2}:a_{1},a_{2}\in A\\}$ and define

$\gamma(T)=\int_{0}^{1}\sqrt{H(\varepsilon,\|\cdot\|_{2},T)}d\varepsilon,$

where $H(\varepsilon,\|\cdot\|_{2},T)$ is the $\varepsilon$-entropy of $T$
with respect to the norm $\|\cdot\|_{2}$. There exists an absolute constant
$\kappa^{\prime}$, such that for all $\tau,\delta\in(0,1)$ if
$d_{0}\geq\kappa^{\prime}(\gamma^{2}(T)+\log(2/\tau))/\delta^{2}$, then with
probability at least $1-\tau$,

$(1-\delta)\|a_{1}-a_{2}\|_{2}^{2}\leq\|Wa_{1}-Wa_{2}\|_{2}^{2}\leq(1+\delta)\|a_{1}-a_{2}\|_{2}^{2},$

for all $a_{1},a_{2}\in A$.

## References

  * Aamari et al. (2019) [author] Aamari, EddieE., Kim, JisuJ., Chazal, FrédéricF., Michel, BertrandB., Rinaldo, AlessandroA. and Wasserman, LarryL. (2019). Estimating the reach of a manifold. Electronic journal of statistics 13 1359–1399.
  * Allen-Zhu, Li and Song (2019) [author] Allen-Zhu, ZeyuanZ., Li, YuanzhiY. and Song, ZhaoZ. (2019). A convergence theory for deep learning via over-parameterization. In International Conference on Machine Learning 242–252. PMLR.
  * Anthony and Bartlett (1999) [author] Anthony, MartinM. and Bartlett, Peter L.P. L. (1999). Neural Network Learning: Theoretical Foundations. Cambridge University Press, Cambridge. 10.1017/CBO9780511624216
  * Aswani, Bickel and Tomlin (2011) [author] Aswani, AnilA., Bickel, PeterP. and Tomlin, ClaireC. (2011). Regression on manifolds: estimation of the exterior derivative. Ann. Statist. 39 48–81. 10.1214/10-AOS823
  * Baraniuk and Wakin (2009) [author] Baraniuk, Richard G.R. G. and Wakin, Michael B.M. B. (2009). Random projections of smooth manifolds. Found. Comput. Math. 9 51–77. 10.1007/s10208-007-9011-z
  * Bartlett et al. (2019) [author] Bartlett, Peter L.P. L., Harvey, NickN., Liaw, ChristopherC. and Mehrabian, AbbasA. (2019). Nearly-tight VC-dimension and pseudodimension bounds for piecewise linear neural networks. J. Mach. Learn. Res. 20 Paper No. 63, 17.
  * Bauer and Kohler (2019) [author] Bauer, BenediktB. and Kohler, MichaelM. (2019). On deep learning as a remedy for the curse of dimensionality in nonparametric regression. Ann. Statist. 47 2261–2285. 10.1214/18-AOS1747
  * Belkin and Niyogi (2003) [author] Belkin, MikhailM. and Niyogi, ParthaP. (2003). Laplacian eigenmaps for dimensionality reduction and data representation. Neural Comput. 15 1373–1396.
  * Belkin and Niyogi (2004) [author] Belkin, MikhailM. and Niyogi, ParthaP. (2004). Semi-supervised learning on Riemannian manifolds. Machine learning 56 209–239.
  * Bickel and Li (2007) [author] Bickel, Peter J.P. J. and Li, BoB. (2007). Local polynomial regression on unknown manifolds. In Complex datasets and inverse problems. IMS Lecture Notes Monogr. Ser. 54 177–186. Inst. Math. Statist., Beachwood, OH. 10.1214/074921707000000148
  * Birgé and Massart (1993) [author] Birgé, LucienL. and Massart, PascalP. (1993). Rates of convergence for minimum contrast estimators. Probab. Theory Related Fields 97 113–150. 10.1007/BF01199316
  * Birgé and Massart (1998) [author] Birgé, LucienL. and Massart, PascalP. (1998). Minimum contrast estimators on sieves: exponential bounds and rates of convergence. Bernoulli 4 329–375. 10.2307/3318720
  * Bishop and Peres (2016) [author] Bishop, Christopher J.C. J. and Peres, YuvalY. (2016). Fractals in Probability and Analysis. Cambridge Studies in Advanced Mathematics. Cambridge University Press. 10.1017/9781316460238
  * Boucheron, Lugosi and Massart (2013) [author] Boucheron, StéphaneS., Lugosi, GáborG. and Massart, PascalP. (2013). Concentration Inequalities: A Nonasymptotic Theory of Independence. Oxford university press.
  * Carlsson (2009) [author] Carlsson, GunnarG. (2009). Topology and data. Bulletin of the American Mathematical Society 46 255–308.
  * Chen, Jiang and Zhao (2019) [author] Chen, MinshuoM., Jiang, HaomingH. and Zhao, TuoT. (2019). Efficient approximation of deep relu networks for functions on low dimensional manifolds. Advances in Neural Information Processing Systems.
  * Chen et al. (2019) [author] Chen, MinshuoM., Jiang, HaomingH., Liao, WenjingW. and Zhao, TuoT. (2019). Nonparametric regression on low-dimensional manifolds using deep relu networks. arXiv preprint arXiv:1908.01842.
  * Cheng and Wu (2013) [author] Cheng, Ming-YenM.-Y. and Wu, Hau-TiengH.-T. (2013). Local linear regression on manifolds and its geometric interpretation. J. Amer. Statist. Assoc. 108 1421–1434. 10.1080/01621459.2013.827984
  * Chui, Li and Mhaskar (1996) [author] Chui, Charles KC. K., Li, XinX. and Mhaskar, Hrushikesh NarharH. N. (1996). Limitations of the approximation capabilities of neural networks with one hidden layer. Advances in Computational Mathematics 5 233–243.
  * Cloninger and Klock (2020) [author] Cloninger, AlexanderA. and Klock, TimoT. (2020). ReLU nets adapt to intrinsic dimensionality beyond the target domain. arXiv preprint arXiv:2008.02545.
  * Conway and Sloane (2013) [author] Conway, John HortonJ. H. and Sloane, Neil James AlexanderN. J. A. (2013). Sphere packings, lattices and groups 290\. Springer Science & Business Media.
  * Cox (1988) [author] Cox, Dennis D.D. D. (1988). Approximation of least squares regression on nested subspaces. Ann. Statist. 16 713–732. 10.1214/aos/1176350830
  * Devroye, Györfi and Lugosi (1996) [author] Devroye, LucL., Györfi, LászlóL. and Lugosi, GáborG. (1996). A Probabilistic Theory of Pattern Recognition. Applications of Mathematics (New York) 31\. Springer-Verlag, New York. 10.1007/978-1-4612-0711-5
  * Donoho and Grimes (2003) [author] Donoho, David L.D. L. and Grimes, CarrieC. (2003). Hessian eigenmaps: locally linear embedding techniques for high-dimensional data. Proc. Natl. Acad. Sci. USA 100 5591–5596. 10.1073/pnas.1031596100
  * Du et al. (2019) [author] Du, SimonS., Lee, JasonJ., Li, HaochuanH., Wang, LiweiL. and Zhai, XiyuX. (2019). Gradient descent finds global minima of deep neural networks. In International Conference on Machine Learning 1675–1685. PMLR.
  * Eldan and Shamir (2016) [author] Eldan, RonenR. and Shamir, OhadO. (2016). The power of depth for feedforward neural networks. In Conference on learning theory 907–940. PMLR.
  * Falconer (2004) [author] Falconer, KennethK. (2004). Fractal Geometry: Mathematical Foundations and Applications. John Wiley & Sons.
  * Farrell, Liang and Misra (2021) [author] Farrell, Max H.M. H., Liang, TengyuanT. and Misra, SanjogS. (2021). Deep neural networks for estimation and inference. Econometrica 89 181–213. 10.3982/ecta16901
  * Federer (1959) [author] Federer, HerbertH. (1959). Curvature measures. Transactions of the American Mathematical Society 93 418–491.
  * Fefferman (2006) [author] Fefferman, CharlesC. (2006). Whitney’s extension problem for $C^{m}$. Annals of Mathematics. 164 313–359.
  * Fefferman, Mitter and Narayanan (2016) [author] Fefferman, CharlesC., Mitter, SanjoyS. and Narayanan, HariharanH. (2016). Testing the manifold hypothesis. Journal of the American Mathematical Society 29 983–1049.
  * Friedman and Stuetzle (1981) [author] Friedman, Jerome H.J. H. and Stuetzle, WernerW. (1981). Projection pursuit regression. J. Amer. Statist. Assoc. 76 817–823.
  * Geman and Hwang (1982) [author] Geman, StuartS. and Hwang, Chii-RueyC.-R. (1982). Nonparametric maximum likelihood estimation by the method of sieves. Ann. Statist. 10 401–414.
  * Ghorbani et al. (2020) [author] Ghorbani, BehroozB., Mei, SongS., Misiakiewicz, TheodorT. and Montanari, AndreaA. (2020). Discussion of: “Nonparametric regression using deep neural networks with ReLU activation function”. Ann. Statist. 48 1898–1901. 10.1214/19-AOS1910
  * Györfi et al. (2002) [author] Györfi, LászlóL., Kohler, MichaelM., Krzy˙zak, AdamA. and Walk, HarroH. (2002). A Distribution-Free Theory of Nonparametric Regression. Springer-Verlag, New York. 10.1007/b97848
  * Härdle, Hall and Ichimura (1993) [author] Härdle, WolfgangW., Hall, PeterP. and Ichimura, HidehikoH. (1993). Optimal smoothing in single-index models. Ann. Statist. 21 157–178. 10.1214/aos/1176349020
  * Härdle and Stoker (1989) [author] Härdle, WolfgangW. and Stoker, Thomas M.T. M. (1989). Investigating smooth multiple regression by the method of average derivatives. J. Amer. Statist. Assoc. 84 986–995.
  * Hendriks (1990) [author] Hendriks, HarrieH. (1990). Nonparametric estimation of a probability density on a Riemannian manifold using Fourier expansions. Ann. Statist. 18 832–849. 10.1214/aos/1176347628
  * Hoffmann, Schaal and Vijayakumar (2009) [author] Hoffmann, HeikoH., Schaal, StefanS. and Vijayakumar, SethuS. (2009). Local dimensionality reduction for non-parametric regression. Neural Processing Letters 29 109\.
  * Horowitz and Härdle (1996) [author] Horowitz, Joel L.J. L. and Härdle, WolfgangW. (1996). Direct semiparametric estimation of single-index models with discrete covariates. Journal of the American Statistical Association 91 1632–1640.
  * Klartag and Mendelson (2005) [author] Klartag, BB. and Mendelson, ShaharS. (2005). Empirical processes and random projections. Journal of Functional Analysis 225 229–245.
  * Kohler, Krzyzak and Langer (2019) [author] Kohler, MichaelM., Krzyzak, AdamA. and Langer, SophieS. (2019). Estimation of a function of low local dimensionality by deep neural networks. arXiv preprint arXiv:1908.11140.
  * Kohler and Langer (2021) [author] Kohler, MichaelM. and Langer, SophieS. (2021). On the rate of convergence of fully connected deep neural network regression estimates. The Annals of Statistics 49 2231–2249.
  * Kong and Xia (2007) [author] Kong, EfangE. and Xia, YingcunY. (2007). Variable selection for the single-index model. Biometrika 94 217–229. 10.1093/biomet/asm008
  * Kpotufe (2011) [author] Kpotufe, SamoryS. (2011). k-NN regression adapts to local intrinsic dimension. arXiv preprint arXiv:1110.4300.
  * Kpotufe and Garg (2013) [author] Kpotufe, SamoryS. and Garg, Vikas KV. K. (2013). Adaptivity to Local Smoothness and Dimension in Kernel Regression. In NIPS 3075–3083.
  * Lee (2006) [author] Lee, John MJ. M. (2006). Riemannian Manifolds: An Introduction to Curvature, 176\. Springer Science & Business Media.
  * Lee, Bartlett and Williamson (1996) [author] Lee, Wee SunW. S., Bartlett, Peter L.P. L. and Williamson, Robert C.R. C. (1996). Efficient agnostic learning of neural networks with bounded fan-in. IEEE Trans. Inform. Theory 42 2118–2132.
  * Lee and Verleysen (2007) [author] Lee, John A.J. A. and Verleysen, MichelM. (2007). Nonlinear Dimensionality Reduction. Information Science and Statistics. Springer, New York. 10.1007/978-0-387-39351-3
  * Levina and Bickel (2005) [author] Levina, ElizavetaE. and Bickel, Peter JP. J. (2005). Maximum likelihood estimation of intrinsic dimension. In Advances in neural information processing systems 777–784.
  * Liang and Srikant (2016) [author] Liang, ShiyuS. and Srikant, RayadurgamR. (2016). Why deep neural networks for function approximation? arXiv preprint arXiv:1610.04161.
  * Lu et al. (2017) [author] Lu, ZhouZ., Pu, HongmingH., Wang, FeichengF., Hu, ZhiqiangZ. and Wang, LiweiL. (2017). The expressive power of neural networks: A view from the width. In Proceedings of the 31st International Conference on Neural Information Processing Systems 6232–6240.
  * Lu et al. (2021) [author] Lu, JianfengJ., Shen, ZuoweiZ., Yang, HaizhaoH. and Zhang, ShijunS. (2021). Deep network approximation for smooth functions. SIAM Journal on Mathematical Analysis 53 5465–5506.
  * Mohri, Rostamizadeh and Talwalkar (2018) [author] Mohri, MehryarM., Rostamizadeh, AfshinA. and Talwalkar, AmeetA. (2018). Foundations of Machine Learning. Adaptive Computation and Machine Learning. MIT Press, Cambridge, MA.
  * Nakada and Imaizumi (2020) [author] Nakada, RyumeiR. and Imaizumi, MasaakiM. (2020). Adaptive approximation and estimation of deep neural network with intrinsic dimensionality. Journal of Machine Learning Research 21 1–38.
  * Nemirovski, Polyak and Tsybakov (1983) [author] Nemirovski, A. S.A. S., Polyak, B. T.B. T. and Tsybakov, A. B.A. B. (1983). Estimates of the maximum likelihood type for a nonparametric regression. 273 1310–1314.
  * Nemirovski, Polyak and Tsybakov (1984) [author] Nemirovski, A. S.A. S., Polyak, B. T.B. T. and Tsybakov, A. B.A. B. (1984). Signal processing by the nonparametric maximum likelihood method. Problemy Peredachi Informatsii 20 29–46.
  * Nemirovski, Polyak and Tsybakov (1985) [author] Nemirovski, A. S.A. S., Polyak, B. T.B. T. and Tsybakov, A. B.A. B. (1985). The rate of convergence of nonparametric estimates of maximum likelihood type. Problemy Peredachi Informatsii 21 17–33.
  * Nguyen and Pham (2020) [author] Nguyen, Phan-MinhP.-M. and Pham, Huy TuanH. T. (2020). A rigorous framework for the mean field limit of multilayer neural networks. arXiv preprint arXiv:2001.11443.
  * Niyogi, Smale and Weinberger (2008) [author] Niyogi, ParthaP., Smale, StephenS. and Weinberger, ShmuelS. (2008). Finding the homology of submanifolds with high confidence from random samples. Discrete & Computational Geometry 39 419–441.
  * Pelletier (2005) [author] Pelletier, BrunoB. (2005). Kernel density estimation on Riemannian manifolds. Statist. Probab. Lett. 73 297–304. 10.1016/j.spl.2005.04.004
  * Petersen and Voigtlaender (2018) [author] Petersen, PhilippP. and Voigtlaender, FelixF. (2018). Optimal approximation of piecewise smooth functions using deep ReLU neural networks. Neural Networks 108 296–330.
  * Pollard (1984) [author] Pollard, DavidD. (1984). Convergence of Stochastic Processes. Springer-Verlag, New York. 10.1007/978-1-4612-5254-2
  * Rafaj (1987) [author] Rafaj, EwarystE. (1987). Nonparametric orthogonal series estimators of regression: a class attaining the optimal convergence rate in $L_{2}$. Statist. Probab. Lett. 5 219–224. 10.1016/0167-7152(87)90044-7
  * Schmidt-Hieber (2019) [author] Schmidt-Hieber, JohannesJ. (2019). Deep relu network approximation of functions on a manifold. arXiv preprint arXiv:1908.00695.
  * Schmidt-Hieber (2020) [author] Schmidt-Hieber, JohannesJ. (2020). Nonparametric regression using deep neural networks with ReLU activation function (with discussion). Ann. Statist. 48 1916–1921.
  * Shen and Wong (1994) [author] Shen, XiaotongX. and Wong, Wing HungW. H. (1994). Convergence rate of sieve estimates. Ann. Statist. 22 580–615. 10.1214/aos/1176325486
  * Shen, Yang and Zhang (2019) [author] Shen, ZuoweiZ., Yang, HaizhaoH. and Zhang, ShijunS. (2019). Nonlinear approximation via compositions. Neural Networks 119 74–84.
  * Shen, Yang and Zhang (2020) [author] Shen, ZuoweiZ., Yang, HaizhaoH. and Zhang, ShijunS. (2020). Deep network approximation characterized by number of neurons. Commun. Comput. Phys. 28 1768–1811. 10.4208/cicp.oa-2020-0149
  * Shen, Yang and Zhang (2022) [author] Shen, ZuoweiZ., Yang, HaizhaoH. and Zhang, ShijunS. (2022). Optimal approximation rate of ReLU networks in terms of width and depth. Journal de Mathématiques Pures et Appliquées 157 101-135.
  * Stone (1982) [author] Stone, Charles J.C. J. (1982). Optimal global rates of convergence for nonparametric regression. Ann. Statist. 10 1040–1053.
  * Stone (1986) [author] Stone, Charles J.C. J. (1986). The dimensionality reduction principle for generalized additive models. Ann. Statist. 14 590–606. 10.1214/aos/1176349940
  * Stone (1994) [author] Stone, Charles J.C. J. (1994). The use of polynomial splines and their tensor products in multivariate function estimation. Ann. Statist. 22 118–184. With discussion by Andreas Buja and Trevor Hastie and a rejoinder by the author. 10.1214/aos/1176325361
  * Suzuki (2018) [author] Suzuki, TaijiT. (2018). Adaptivity of deep ReLU network for learning in Besov and mixed smooth Besov spaces: optimal rate and curse of dimensionality. arXiv preprint arXiv:1810.08033.
  * Telgarsky (2016) [author] Telgarsky, MatusM. (2016). Benefits of depth in neural networks. In Conference on learning theory 1517–1539. PMLR.
  * Tenenbaum, De Silva and Langford (2000) [author] Tenenbaum, Joshua BJ. B., De Silva, VinV. and Langford, John CJ. C. (2000). A global geometric framework for nonlinear dimensionality reduction. Science 290 2319–2323.
  * Tu (2011) [author] Tu, Loring WL. W. (2011). Manifolds. In An Introduction to Manifolds 47–83. Springer.
  * Van de Geer (1987) [author] Van de Geer, SaraS. (1987). A new approach to least-squares estimation, with applications. Ann. Statist. 15 587–602. 10.1214/aos/1176350362
  * Van de Geer (1990) [author] Van de Geer, SaraS. (1990). Estimating a regression function. Ann. Statist. 18 907–924. 10.1214/aos/1176347632
  * Van de Geer (2000) [author] Van de Geer, Sara A.S. A. (2000). Applications of Empirical Process Theory. Cambridge University Press, Cambridge.
  * Van de Geer and Wegkamp (1996) [author] Van de Geer, SaraS. and Wegkamp, MartenM. (1996). Consistency for the least squares estimator in nonparametric regression. Ann. Statist. 24 2513–2523. 10.1214/aos/1032181165
  * Van der Vaart and Wellner (1996) [author] Van der Vaart, Aad W.A. W. and Wellner, Jon A.J. A. (1996). Weak Convergence and Pmpirical Processes: with Applications to Statistics. Springer-Verlag, New York. 10.1007/978-1-4757-2545-2
  * Vershynin (2018) [author] Vershynin, RomanR. (2018). High-Dimensional Probability: An Introduction with Applications in Data Science 47\. Cambridge university press.
  * Weinstein et al. (2013) [author] Weinstein, J. N.J. N., Collisson, E. A.E. A., Mills, G. B.G. B., Shaw, K. R.K. R., Ozenberger, B. A.B. A., Ellrott, K.K., Shmulevich, I.I., Sander, C.C. and Stuart, J. M.J. M. (2013). The cancer genome atlas pan-cancer analysis project. Nature genetics. 45 1113\.
  * Yang and Dunson (2016) [author] Yang, YunY. and Dunson, David B.D. B. (2016). Bayesian manifold regression. Ann. Statist. 44 876–905. 10.1214/15-AOS1390
  * Yang and Tokdar (2015) [author] Yang, YunY. and Tokdar, Surya T.S. T. (2015). Minimax-optimal nonparametric regression in high dimensions. Ann. Statist. 43 652–674. 10.1214/14-AOS1289
  * Yarotsky (2017) [author] Yarotsky, DmitryD. (2017). Error bounds for approximations with deep ReLU networks. Neural Networks 94 103–114.
  * Yarotsky (2018) [author] Yarotsky, DmitryD. (2018). Optimal approximation of continuous functions by very deep ReLU networks. In Conference on Learning Theory 639–649. PMLR.


# Tracing the magnetic field topology of the quiet corona using propagating
disturbances

Huw Morgan  Marianna Korsos  Department of Physics, Aberystwyth University,
Ceredigion, Cymru, SY23 3BZ, UK<EMAIL_ADDRESS>
###### Abstract

The motion of faint propagating disturbances (PD) in the solar corona reveals
an intricate structure which must be defined by the magnetic field. Applied to
quiet Sun observations by the Atmospheric Imaging Assembly (AIA)/Solar
Dynamics Observatory (SDO), a novel method reveals a cellular network, with
cells of typical diameters 50″ in the cool 304Å channel, and 100″ in the
coronal 193Å channel. The 193Å cells can overlie several 304Å cells, although
both channels share common source and sink regions. The sources are points, or
narrow corridors, of divergence that occupy the centres of cells. They are
significantly aligned with photospheric network features and enhanced magnetic
elements. This shows that the bright network is important to the production of
PDs, and confirms that the network is host to the source footpoint of quiet
coronal loops. The other footpoint, or the sinks of the PDs, form the
boundaries of the coronal cells. These are not significantly aligned with the
photospheric network - they are generally situated above the dark internetwork
photosphere. They form compact points or corridors, often without an obvious
signature in the underlying photosphere. We argue that these sink points can
either be concentrations of closed field footpoints associated with minor
magnetic elements in the internetwork, or concentrations of upward-aligned
open field. The link between the coronal velocity and magnetic fields is
strengthened by a comparison with a magnetic extrapolation, which shows
several general and specific similarities, thus the velocity maps offer a
valuable additional constraint on models.

Sun: corona—sun: CMEs—sun: solar wind

##  1 Introduction

Due to observational limitations, there are large uncertainties of the coronal
magnetic field topology above the quiet Sun. Reliable estimates of the
magnetic field strength from spectropolarimetrical observations currently
depend on exceptional observations of high signal (Kuridze et al., 2019), and
are particularly challenging for the quiet corona (e.g. Khomenko et al.,
2003). Within active regions (AR), systems of large and bright loops can be
observed in the extreme ultraviolet (EUV) or X-ray, providing a comparison
with magnetic models, including quantitative constraints in some cases
(Aschwanden, 2013). Above the quiet Sun, the structure of the low corona as
observed in EUV is bereft of distinct and bright coronal loops, leading to
uncertainty.

Insight into the quiet Sun coronal topology and connection to the lower
atmospheric layers is dependent largely on model extrapolation, constrained by
observations of the photospheric and chromospheric magnetic fields and their
time evolution (Bellot Rubio & Orozco Suárez, 2019). Convective flows of
photospheric granules are observed to form coherent supergranular cells of
motion with typical diameters of 32Mm (45″) (e.g. Tian et al., 2010; Noori et
al., 2019). A comprehensive review is given by Rincon & Rieutord (2018) .
Their boundaries are regions of convective downflow that form lanes of
magnetic concentrations that coincide with the bright chromospheric network
observed in the UV continuum. Schrijver & Title (2003) showed, using arguments
based on a potential field magnetic model, that around half of the field may
be rooted in the darker internetwork regions, with the other half arising from
the network and closing into the internetwork. Wiegelmann et al. (2010) found
that most quiet Sun magnetic loops that reach mid-chromospheric heights or
higher have one footpoint in the strong magnetic fields of the network, with
the other footpoint in the weaker internetwork regions. A schematic that helps
visualise these connections is figure 16 of Wedemeyer-Böhm et al. (2009),
which also gives a comprehensive overview of the field. We also refer the
reader to the detailed review of Wiegelmann et al. (2014), and section 5 in
particular.

This work uses the motion of faint propagating disturbances (PD) in time
series of EUV images to gain observational insight into the distribution of
the quiet Sun coronal magnetic field. The Time Normalized Optical Flow (TNOF)
method to enhance and characterise the PD motions was first described by
Morgan & Hutton (2018), with a more simple and efficient approach described by
Morgan & Korsos (2022). The amplitudes of the PD are very faint - at most
$\approx$4 DN $s^{-1}$, or less than 2% of the background signal. They are
ubiquitous and continuous, appearing quasi-periodically and showing a
preferential direction of motion over local regions of the corona. Past
studies of PD showing similar characteristics include Stenborg et al. (2011),
who state that the disturbances are a coronal phenomenon that exist
permanently everywhere, and Wang et al. (2009) who found PD in a fan-like loop
structure. The propagation velocities are on the order of tens of km$s^{-1}$,
and have quasi-periodicities, or repetition times, of a few minutes. The PD
are usually interpreted as slow magnetoacoustic waves, with some studies
emphasising propagation along open magnetic field (Stenborg et al., 2011). The
events presented by Morgan & Hutton (2018) were generally fainter compared to
these other studies, and PDs were found in closed field structures in an
active region and quiet Sun. Sheeley et al. (2014) used running difference
images and time-distance maps to show ‘flows’ in active regions and within
cellular plumes. They describe the global character of these apparent flows,
particularly within the context of widespread cellular plumes. We note that
the propagating disturbances that they describe included events considerably
brighter than those revealed by the TNOF method, and some of the examples had
speeds considerably higher ($\approx$90 km$s^{-1}$, compared to below 50
km$s^{-1}$ for TNOF) .

##  2 Coronal velocity field

The TNOF method ingests a time series of solar atmospheric images and produces
velocity vector maps (Morgan & Hutton, 2018; Morgan & Korsos, 2022). The
method is applied here to the 304 and 193Å EUV channels of the Atmospheric
Imaging Assembly (AIA, Lemen et al. (2011)) onboard the Solar Dynamics
Observatory (SDO, Pesnell et al. (2012)). The signature of faint PD are
enhanced relative to the background through the use of a time-normalizing
process applied to the calibrated time series signal. A Lucas-Kanade algorithm
is utilised as an optical flow method to trace the motion of the PDs. In this
work the period of study is between 2018 October 27 12:00UT and 14:00UT. There
are no substantial eruptions or flares during this period. The choice of a
two-hour period is a compromise between gaining the statistics necessary to
cleanly characterise the velocity field, and minimising the effects of large-
scale motions such as differential rotation, or other changes as described by
Morgan & Korsos (2022). The region of study is within $\pm$250″ from disk
center (relative to Earth) in both $x$ and $y$, shown as the boxed red regions
in figures 1a and b for 304 and 193Å respectively, with more detail shown in
figures 1c and d. The region is mostly quiet Sun, with a small, decaying,
equatorial coronal hole in the south-west which appears dark in the 193Å
image. This dark region was a clear equatorial coronal hole in previous
rotations, but by 2018 October 27 has shrunk in width.

Figure 1: Top row: The solar corona as observed by the AIA (a) 304Å and (b)
193Å channels at 2018 October 27 13:00UT. The red box surrounding disk center
bounds the region of interest. Bottom row: Detail of the region of interest in
(c) 304Å and (d) 193Å. These images have been processed by Multiscale Gaussian
Normalization (Morgan & Druckmüller, 2014).

The movie accompanying figure 2 shows the time series of 304 and 193Å time-
normalised and noise-reduced images. The movie shows complex patterns which
are difficult to decipher, although they give a clear sense of consistent and
repetitive motion. The Lucas-Kanade algorithm, applied to find the best least-
squares fit to the $x$ and $y$ velocity components at each pixel over the 600
time steps, reveals a coherent structure that must underlie these complex
patterns. These velocities are interpreted as the dominant direction and
magnitude of the motions over the two hours of observation. The magnitude of
the velocities range from 0 to 25 km$s^{-1}$ for 304Å with a most probable
speed of 5 km$s^{-1}$. For the 193Å channel, the speeds range from 0 to 70
km$s^{-1}$, with a most probable speed of 15 km$s^{-1}$.

Figure 2a and b show the velocity vectors for the 304 and 193Å channels
respectively, superimposed on background intensity images. In the 304Å
velocity field, there is a general pattern of flow lines starting (red) and
ending (blue) from points, or narrow corridors, of high field line density.
The flow lines between these sources and sinks form cells, or sectors, of
coherent flows of approximate diameters 50″. Flow lines do not necessarily
form and end at the same region - flow lines from a single source can lead to
two or more different sink regions. The 193Å channel velocity field shown in
figure 2b has a similar structure of coherent cells, but they are generally
larger, meaning coherent flows over larger areas. Some of the 193Å flow
systems are considerably larger than 100″. For example, the dark coronal hole
is a source of flows that reach to the neighbouring quiet Sun.

Figure 2: Top row: Velocity vector maps, overlaid on background MGN images,
for the (a) 304 and (b) 193Å channels. The velocity vectors are depicted using
a rainbow colour scheme, with a field line starting with colour red, and
advancing through yellow, green, and ending in blue (in the order of flow
direction). Bottom row: Greater detail of selected regions for the (c) 304Å
velocity field and (d) 193Å field. For 304Å in panel c, shown clockwise from
top left are $70\times 70$″ regions centered on (10,-75), (135,-165),
(-135,115), and (175, -135)″. For 193Å in panel d, shown clockwise from top
left are $90\times 90$″ regions centered on (135,105), (175,-55), (-135,-85),
and (45, 145)″.

The panels of figure 2c show greater detail of four selected regions of the
304Å velocity field, each of size $70\times 70$″. The first example (top left)
shows a clearly-formed cell with a dense sink region in the center. The flow
lines converge from all directions into this central sink. Some originate from
a point source region in the lower left of the area, others arise from a dense
corridor aligned north-south towards the left of the area. The top right panel
shows a similar cell structure, which has a central sink region, and also a
neighbouring point source region in the top left. The bottom left panel of
figure 2c shows an example of an extended, narrow sink corridor which meanders
from the bottom left to the top right corner. The bottom right panel shows a
central sink region where the incoming flow lines have a consistent pattern of
rotation around the central sink. In this case, the field lines describe one
complete rotation from their source to their center destination. There are
many examples of such rotations in the main velocity field of figure 2a, and
can be compared to the large distinct vortex described in section 7 and figure
20 of Morgan & Hutton (2018). A more detailed study may show if these
rotational structures are linked to certain features on the photosphere, for
example persistent rotational regions associated with supergranular vertices
(Requerey et al., 2018) .

The four panels of figure 2d show details of four selected regions from the
193Å velocity field. The two top panels show two typical cell regions. The
first, on the left, is a clear cell structure that has an extended sink
corridor in the center, with inwardly converging flow lines. The flow lines
emerge from two narrow source corridors that form the left and bottom boundary
of the cell. The top right panel shows another cell example, which has a
point-like sink region in the center, with three neighbouring source regions
that are more localised than the extended corridors in the first example. The
bottom left panel is an example of an extended corridor of convergence, and
the bottom right panel is an example of a rotated source region. These rotated
structures are rare in the 193Å channel compared to the 304Å. We speculate
that this is due to the corona being less closely connected to the photosphere
compared to the cooler layers observed by the 304Å channel, thus the smaller
cell structures in the 304Å channel may be closely associated with a rotating
region in the photosphere, causing a degree of rotation at that height. The
larger cellular structures in the 193Å channel may bridge above the smaller
cells of the 304Å channel, and be less prone to the rotation of the
photosphere at smaller spatial scales.

In both channels, the flow lines that feed into or out of dense corridors of
divergence/convergence tend to approach the corridors at a similar angle. That
is, on the approach to the corridor, a flow line tends to join the corridor at
an acute angle which is similar to other flow lines at that side of the
corridor. Flow lines that approach from the other side of the corridor follow
the same pattern, with the angle of approach reflecting the opposite side -
giving a pattern akin to a fern leaf. This suggests that the flow tends to one
direction along the corridor. The bottom and top left panels of figure 2d show
examples of fern-leaf shaped divergence corridors for the 193Å channel.

In the 193Å channel, the flow lines within the coronal hole area (the 193Å
dark region in the south-east) link to neighbouring brighter regions, and in
the 304Å channel there is no general difference in the pattern of flow lines
within this region compared to the quiet Sun. We would expect the center of
equatorial coronal holes to be dominated by open magnetic field and a
different flow pattern to quiet Sun regions, but the results seen here may
relate to the fact that the coronal hole has decayed. The coronal hole is
discussed further in section 5.

The top row of figure 3 shows a simplified representation of the complicated
flow fields. The blue and red lines represent certain regions where the
direction of the flow field changes direction abruptly. The direction of the
flow field at each pixel is given by the angle $\Omega=\arctan(v_{y}/v_{x})$.
The Sobel edge enhancement operator is applied to this angular image to give a
value at each pixel: very high values are found at regions where the angle in
velocity direction changes abruptly - both the red and blue lines in the
bottom row of figure 2 correspond to high values. The red (blue) colours
denote where these regions of abrupt angular change have positive (negative)
divergence. For convenience, we denote the regions of angular discontinuities
and positive or negative divergence $S^{+}$ and $S^{-}$ respectively. The blue
lines overlie the points and narrow corridors where flows converge from
surrounding areas (sinks), whilst the red lines overlie the points and
corridors where flows diverge to surrounding areas (sources).

Figure 3: Top row: Velocity vector maps shown without color or background
image for the (a) 304 and (b) 193Å channels. The red and blue lines show where
there is a large change in direction of the local velocity vector, with red
corresponding to positive divergence ($S^{+}$, or source of flows) and blue
corresponding to negative divergence ($S^{-}$, or where flows converge).
Middle row: (c) $S^{+}$ regions for the 304Å (red) and 193Å (green) velocity
fields. These are regions of large angular change and positive divergence in
the velocity vector fields, so trace out the center of flow source regions.
(d) $S^{-}$ regions for the 304Å (blue) and 193Å (green) velocity fields.
These trace regions of large angular change and negative divergence, or the
corridors and points where flows converge. Bottom row: Further detail of the
velocity vector field for the (e) 304 and (f) 193Å channels, for the small
region near disk center boxed in panels (c) and (d).

Figure 3c shows the $S^{+}$ distribution (flow sources) for the 304Å (red) and
193Å (green) velocity fields. There is a correlation between the two
distributions across most of the region, with a spatial alignment significance
(see Appendix A) of 2.7$\sigma$ and a percentage significance of 100% (i.e.
the alignment score is higher than all random cases). The 193Å $S^{+}$ forms
longer, more coherent, lines than the 304Å, thus showing the larger-scale
coherence of the 193Å flow fields, and the general trend for smaller-scale
structure in the cooler channel, as can be seen visually in figure 2a and b.
In figure 3b, which shows the $S^{-}$ distribution (flow sinks) for the 304Å
(blue) and 193Å (green) velocity fields, the fields are in broad agreement in
some regions, but the spatial match is overall weaker with a spatial alignment
significance of 1.7$\sigma$ and percentage 95%. Thus the cooler 304Å channel
and hotter 193Å channels are more likely to share common source regions to
their PDs, but less likely to share alignment between sink regions (or where
the PD flow lines end).

Figure 3e and f show greater details of the velocity vector fields for 304 and
193Å respectively in a small region near disk center. These figures emphasise
the fact that both channels can share the same source and sink regions, but
that the cooler 304Å channel tends to have more source and sink regions, and
has shorter flow paths with higher curvature between these regions. For
example, both channels share a dominant source near the region center. They
also have a sink point to the top right ($x,y=45,5$″). This sink point shows
rotation only in the 304Å channel.

##  3 Comparison with photospheric observations

Figure 4a shows an observation of the region of interest by the AIA 1700Å
channel, where the ultraviolet continuum is dominated by the temperature
minimum/photospheric intensity. This data shows the patchwork of dark
internetwork regions bounded by the bright network. Analysis of a time series
of such images taken between 2018 October 27 12:00 to 14:00UT show changes
only to small-scale features, whilst the brightest large-scale network
patterns remain fairly stable. The negative of the same data (black=bright) is
shown as the background image of figure 4b, with the detected brightest
network patterns highlighted in green. Figure 4c compares the bright
photospheric network (green) with the AIA 304Å velocity field characterisation
of $S^{+}$ (red, positive divergence or sources) and $S^{-}$(blue, negative
divergence or sinks). Visual inspection shows some areas of close agreement
between red and green, thus the sources of the AIA 304Å velocity field can
often closely align with the photospheric bright network. The spatial
alignment significance value between these is 2.8$\sigma$, with percentage
score 100%. In contrast, there is no significant agreement between the blue
and green, with a percentage score of 24%. This can be seen visually - the
blue lines tend not to align over the green regions. Thus the sinks of the AIA
304Å velocity field have no significant spatial correlation with the
photospheric bright network. Figure 4d shows the same comparison for the 193Å
channel. As with the 304Å channel, there is high significant alignment between
the red and green, with a score of 2.8$\sigma$, and 100%. There is no
significance between the blue and green, with percentage 47%.

Figure 4: (a) Observation of 2018 October 27 13:00UT of the region of study by
the AIA 1700Å channel, showing the photospheric UV continuum. (b) The negative
of the AIA 1700Å channel intensity (white = low intensity), with the brightest
network features highlighted in green. (c) Comparing the UV network (green)
with the AIA 304Å velocity field characterisation of $S^{+}$ (red, positive
divergence or sources) and $S^{-}$(blue, negative divergence or sinks). (d) As
panel (c), but for the AIA 193Å velocity field.

Figure 5 shows a comparison between the 304 and 193Å velocity fields and a
photospheric magnetogram calculated from Helioseismic and Magnetic Imager
(HMI/SDO, Scherrer et al. (2012)) observations. The top two panels show the
velocity vectors superimposed on the magnetogram images, whilst the bottom
four panels show the source and sink features of the velocity fields for two
smaller regions. Similar to the UV network presented in the previous figure,
there are several regions which show close correlation between the source
features (red) and features of both positive and negative enhanced
photospheric magnetic field. Figure 5d shows an extended 193Å source feature
that is closely aligned with the underlying, mostly negative, field in the HMI
data. The 193Å sink features are generally not closely aligned with
photospheric enhancements, but can sometimes coincide. For this same region in
the 304Å channel, shown in panel c, there is less obvious correlation,
although source features appear more likely to coincide with enhanced
photospheric field compared to sinks. Figure 5e shows close alignment between
a loop-shaped 304Å source feature and the photosphere. Most of the sink
features are not aligned with enhanced field, and indeed one prominent sink
feature sits within the looped source feature. A larger loop of sink points
surround the inner source loop. The same region in 193Å, shown in Figure 5f,
also shows close alignment between some source features and the photosphere,
but the pattern is not as clear as the 304Å case.

Figure 5: A comparison of photospheric magnetograms calculated from HMI
observations of 2018 October 27 13:00UT with the 304 (left column) and 193
(right column) velocity fields. The top row shows the velocity fields
superimposed on the HMI magnetogram for the whole region of study. The middle
row shows the source (red) and sink (blue) features of the velocity fields
superimposed on the magnetogram for a small region near disk center. The
bottom row shows another small region to the south of disk center.

##  4 Comparison with a potential magnetic field extrapolation

From the comparison with both the UV and magnetogram data, it is obvious that
there is a link between the distribution of the coronal velocity fields and
the underlying magnetic features of the photosphere. In this subsection, we
extend this comparison to a potential field source surface (PFSS)
extrapolation, based on the HMI magnetogram. Our code for extrapolation is
based on a numerical implementation of the Green’s function method (Sakurai,
1982; Boocock & Tsiklauri, 2019). We limit the comparison to a small region
extending from -65 to 50” in both $x$ and $y$, encompassing most of the region
shown in the middle row of Figure 5. For efficiency, we resize the magnetogram
to a quarter of the number of pixels, so that each new pixel is the local
average of four pixels of the original. To avoid edge effects, the
extrapolation area extends by 10” beyond the displayed area at all margins,
and the photospheric input to the extrapolation extends well beyond this. The
extrapolation volume extends to 0.2$R_{\odot}$ above the photosphere. We use
all the magnetogram pixels to calculate the potential field, not just enhanced
sources, and we do not fit the photospheric field to a smooth function. Thus
the potential at a given coronal point in the extrapolation volume is
calculated from all photospheric points within the visible horizon.

Figure 6b shows the result of the PFSS extrapolation. The field lines are
traced in the 3D volume, and their coordinates transformed to the observer’s
field of view. There are many similarities between the PFSS and velocity
fields, as well as some major differences. On a general level, the PFSS field
is cellular, with dense concentrations of footpoints aligned with stronger
photospheric magnetic elements. The field lines at greater heights (coloured
green/yellow/red) can bridge over closed underlying systems at low heights
(coloured black). Furthermore, whilst many field lines have one footpoint in a
dense source, and the other in a low-density internetwork region, lines can
also join in dense, narrow corridors, similar to the fern-leaf patterns seen
in the velocity fields. We can thus find most of the general features of the
velocity fields in the PFSS extrapolations.

Figure 6: Comparison of the velocity fields of the (a) 304 and (c) 193
channels, with (b) field lines traced from a potential field extrapolation,
for a small region near disk center. The extrapolated field lines are coloured
according to height, with black being very low-lying lines, followed by blue,
green, yellow and red (near the maximum extrapolated height of
0.2$R_{\odot}$). The background image is the HMI magnetogram observed at 2018
October 27 13:00UT.

Beyond these general similarities, there are also specific features that are
shared between the PFSS and the 304Å velocity field, including:

  * •

A very similar distribution of field lines within, and surrounding, the
central prominent feature at $x,y=[-10,-20]$”. This is also similar in the 193
velocity field.

  * •

The cell systems centered at $[-35,0]$”, and $[-55,-15]$”, and their
connection to the central prominent feature and each other.

  * •

A cross-shaped meeting of field lines at the boundary of distinct cell systems
at $[-10,-55]$”. Two other similar systems are seen at $[0,45]$” and
$[35,25]$”. The corresponding features in the PFSS extrapolation are at
slightly different locations.

We note that the 304Å velocity field contains several rotating features: two
of them are seen here centered at $[-50,-60]$” and $[45,5]$”. There is no
corresponding spiral pattern in the PFSS extrapolation. That there is no
spiral structure in the magnetic field of the PFSS model is expected, since
the model has no time-dependence, and is force-free.

##  5 The coronal hole

This section makes a more detailed analysis of the narrow coronal hole seen in
the south-east of the region of study. In particular, we expect coronal holes
to be dominated by open field, and further expect that field to be expanding
super-radially from the central regions of the coronal hole, which seems to be
in contradiction to the velocity field patterns seen in figure 2a and b, and
shown in greater detail in figure 7d for the 193Å channel.

We first define a boundary to the coronal hole using a differential emission
measure (DEM) analysis. The method we use is Solar Iterative Temperature
Emission Solver (SITES, Morgan & Pickering (2019); Pickering & Morgan (2019)),
and the input data is AIA/SDO data from the seven EUV channels. The resulting
DEM, at a cool temperature of 0.6MK, is shown in figure 7a. Figure 7b shows
the fractional emission measure (FEM), which is the emission at that
temperature divided by the total emission summed over all temperatures. The
map shows a clear region of enhanced FEM within, and surrounding, the coronal
hole. The hole boundary, shown as the red contour, is defined by any regions
that have an FEM over 3.5%, and with the number of connected pixels larger
than a few hundred. Figure 7c plots the coronal hole boundary over the HMI
magnetogram. Visually, there is no discernible difference between the
photosphere within the hole, and outside in the surrounding coronal hole.
Figure 8a shows that the histograms of photospheric magnetic field for inside
and outside the coronal hole are very similar - indeed, their mean and
standard deviations are close to identical. It is only when one histogram is
subtracted from the other that we see a small bias towards negative polarities
within the coronal hole, as shown in figure 8b.

Figure 7: A closer look at the small coronal hole. (a) A map of the
differential emission measure (DEM) at a temperature very near 0.6MK. (b) The
fractional emission measure (FEM, see text) at 0.6MK, or the percentage of
emission at this temperature compared to the total emission over all
temperatures. Values of FEM above 3.5% are used to estimate the coronal hole
boundary, shown in all panels as the red contour. (c) The HMI magnetogram for
this region. The horizontal blue line shows a slice of the magnetogram used
for the 2-dimensional potential field extrapolation shown in figure
LABEL:pfss2d_ch. (d) The 193 velocity field for this region. Figure 8: (a)
Histograms of the photospheric magnetogram values outside (blue) and inside
(red) the coronal hole, shown as a percentage per 0.76G bin of the total
number of pixels. (b) The difference between the two histograms (inside minus
outside).

We extract the photospheric field values for a line cutting east-west across
the coronal hole, shown as the horizontal blue line in figures 7c and d. This
field is shown in the lower panel of figure 9. We calculate a PFSS
extrapolation for this field, in two dimensions ($x$, and height), which is
displayed in the top panel of figure 9. Whilst a 2-dimensional extrapolation
cannot be used to study the coronal hole magnetic field properly, this example
helps to illustrate an argument that the velocity field may be giving a
correct approximation to the magnetic field. This coronal hole does not have a
large dominant polarity, and contains both positive and negative enhancements.
In particular, the positive enhancement seen here in the centre of the hole,
gives rise to a high density of field lines, from two system of closed
structures to both east and west. Open field bounding these closed systems
bridge across, and angle sharply upwards approximately above the positive
enhancement. We argue that the velocity field within the southern region of
the coronal hole, as seen in the 193Å channel (figure 7d), may arise from such
a configuration. We note that this coronal hole is decaying, and our results
cannot be generalised for all small equatorial coronal holes.

Figure 9: The bottom panel shows the values extracted from the HMI magnetogram
along the horizontal slice shown as a blue line in figure 7c. The top panel
shows the potential field extrapolation (two-dimensional: $x$, and height),
based on these values. The two vertical blue dashed lines show the boundaries
of the coronal hole - areas between these lines are within the coronal hole.

##  6 Discussion & Summary

The flow maps are tracing the dominant direction of motions of faint,
ubiquitous PDs in the solar atmosphere. The PDs must follow the direction of
the underlying magnetic field (e.g. Stenborg et al., 2011; Wang et al., 2009;
Sheeley et al., 2014). This is the simplest interpretation of the patterns we
see in the velocity flow maps, and of the differences and similarities between
the results for the 304 and 193Å channels. The PD velocity fields are tracing
the plane-of-sky projection of the magnetic field at the atmospheric height
corresponding to the dominant temperature of formation of that channel,
accepting that (i) both channels contain contributions from multiple emission
lines thus have a range of formation temperatures, and (ii) the height varies
in different atmospheric structures . We of course lack the three-dimensional
(or line of sight) information; we have only the projected (or plane-of-sky)
velocities. Nevertheless, the flow lines give a unique proxy to the quiet Sun
coronal magnetic field structure based directly on EUV coronal observations -
information that is currently impossible to gain without magnetic modelling.
The pattern seen in the quiet Sun region of figure 2, of cells, or sectors, of
flow line systems arising from and ending in points and narrow corridors,
leads to a hypothesis that the magnetic field topology has a similar sectorial
structure.

In the lower atmosphere, corresponding to the 304Å channel, we expect the
closed magnetic structures that form the quiet Sun corona to have generally
smaller structural scales than the overlying coronal field traced by the 193Å
channel, and this is what is seen in the PD velocity fields. In the cooler
304Å channel, the cell diameters are typically $\approx$$50$″, whilst the
hotter 193Å channel tends to have larger cell sizes, with some larger cells of
diameter 80 to larger than 100″. The coronal field traced by the 193Å channel
can share common sources and sinks with the 304Å channel, but can often bridge
over one or more cells in the cooler 304Å channel.

The cells tend to have central point-like, or corridor-like, source regions,
with neighbouring systems of sink points or corridors forming their
boundaries. Some cells show clear rotational motion. These are common in the
304Å channel, and rare in the 193Å channel. The apparent rotational flows
suggests that the magnetic field may be twisted due to the coherent large-
scale rotation of areas of the underlying photosphere. The projected fern-leaf
shape of narrow sink/source corridors suggests that there is a dominant
direction of flow along the narrow corridors, and a dominant orientation of
the magnetic field along the narrow corridors.

The comparison with the underlying photospheric UV intensity, and with
photospheric magnetograms, shows that the sources of the field lines tend to
overlie the bright network, whilst their sinks tend to lie in the darker
internetwork regions. This result is important. It reinforces the hypothesis
that the velocity fields are a tracer of the magnetic fields, and is
consistent with the accepted model of the bulk of the quiet Sun magnetic field
concentrated in network regions. Furthermore, if the PDs are slow
magnetoacoustic waves, then these brighter UV network regions, and enhanced
magnetic elements, are conducive to the production of waves. If the
internetwork regions were equally productive, then we would not see a clear
preferential direction of the PD flows from the network and towards
internetwork regions. Certainly there is much further work that can be done to
investigate this relationship, including a more detailed comparison with the
photospheric and lower-chromospheric magnetic field distribution, and a
comparison with the results of motion tracking in the photosphere (e.g. Chian
et al., 2019).

The most surprising result is the distribution of the flow sinks (convergence
regions). That the flow sources tend to emerge from dense, narrow points or
corridors, and that these tend to align with bright network regions, is
consistent with general models of the quiet Sun magnetic field. The source
field arising from the network boundaries is expected to expand with altitude,
filling the transition region and low corona, with the closed field forming
the basic building blocks of the quiet Sun corona. However, our results show
that the opposite footpoints (or sink footpoints) of these closed loop systems
are also distributed in dense narrow points or corridors, similar to their
source footpoints. That is, the field arising from the network does not spread
to sink footpoints distributed at low density through broad regions of the
larger internetwork regions. Whilst the field lines linking both footpoints
expand to fill intervening areas, the footpoints diverge from, and converge
to, distinct, small areas forming points or corridors. In contrast to the
source regions, the sink regions are not aligned with the photospheric network
boundaries, and there is not always a distinct signature of the sink regions
in the UV photospheric intensity or in photospheric magnetograms - they are
situated within the weak field of the dark internetwork regions.

Support for the presence of concentrated sink points/corridors in the quiet
Sun internetwork regions can be given by simple experiments with a two-
dimensional PFSS model. First, we place two strong magnetic elements, of
opposite sign, in otherwise zero field, resulting in the expected
configuration of coronal field as shown in figure 10a. We add to this a large
number of very weak small sources with random locations and random amplitudes
of standard deviation 1% of the large sources. Despite the weakness of this
photospheric random source, the field at low heights is perturbed, as shown in
figure 10b. We now add a weak negative element of amplitude 6% of the strong
elements, equidistant between them. The field is now quite strongly perturbed
at low to medium heights, and several field lines link the strong positive
element to the weak negative element, as shown in figure 10c - leading to a
concentration of field lines at the position of the small negative
enhancement. The experiment is repeated with two strong positive elements in
the right column of figure 10. In the case of zero field, or weak random
field, outside of the strong elements (figure 10d and e), the coronal field
has a concentration of upwardly-pointing open field at the centre of the
system. When a weak negative element is placed in the centre (figure 10f), we
have a strong concentration of both downwardly- pointing closed field as well
as the open field. We argue that such configurations may be common in the
quiet Sun, and can explain the dense sink regions for our velocity fields. In
the case of figure 10d, the central region between the two strong positive
sources has a dense concentration of upwardly-aligned open field, and this
would appear as a dense sink region in a coronal PD velocity field. Relatively
weak enhancements in the internetwork region can also lead to concentrations
of closed field lines in the internetwork, which would also appear as sink
regions in the velocity fields (figures 10c and f).

Figure 10: Experiments with a simple two-dimensional potential magnetic field.
The left column shows cases with two strong magnetic sources, one positive on
the left, and one negative on the right. The right column shows cases with two
strong positive sources. In each panel, the green line shows the photosphere.
Below this green line, the red line shows the strength of the source
(photospheric) field, with the location of the two strong sources shown by
vertical blue lines. The range of the source field is limited to small values
around zero in order to show small variations, so that the left and right
source peaks are heavily curtailed. The top row show the case where there are
two strong sources, and zero field elsewhere. The middle row adds small random
photospheric variations, with standard deviation amplitudes of 1% of the
amplitude of the two strong sources. The bottom row has the two strong
sources, random variations, and a small central negative source with amplitude
6% of the two strong sources. The position of this small negative source is
indicated by the vertical blue line, equally spaced between the two strong
sources.

We note that a recent detailed observational study of the magnetic
configuration of a supergranule by Robustini et al. (2019) suggested that the
chromospheric field lines tended to point inwards from the supergranule
boundary, and may form a canopy above the unipolar interior region.
Furthermore, they found a persistent internetwork small brightening coinciding
with a distinct blue-shifted (downflow) feature in the line-of-sight velocity
of the Ca II K line. Chian et al. (2019) showed repelling Lagrangian coherent
structures concentrated at the center of a supergranule. Thus central regions
in the internetwork are special - they are central source regions of local
supergranular-scale photospheric motions, and their properties may lead to a
higher concentration of overlying atmospheric magnetic field lines.

Our next efforts are aimed to better understand the connection between the
velocity fields and the atmospheric magnetic field. We will combine our PD
tracking method with spectroscopic observations of EUV lines, and will focus
on whether the source and sink regions coincide with regions of Doppler shifts
or other distinguishing properties. We plan to create disk-center velocity
fields for AIA/SDO during periods when other spacecraft hosting EUV imagers
(e.g. Solar Orbiter) are at 90∘ to Earth, enabling a comparison of the
equatorial off-limb structure in EUV with the velocity field distributions
from the SDO viewpoint. We are also interested in comparing our coronal flow
fields with photospheric and chromospheric flow fields. In the longer term, an
exciting possibility is to use our results as a proxy for the projected plane-
of-sky distribution of coronal magnetic fields, and use this as an empirical
constraint on magnetic extrapolation models. This can be particularly useful
for the quiet Sun and coronal holes, where the EUV emission does not trace
systems of clear loops as is seen in active regions.

We acknowledge (1) STFC grants ST/S000518/1 and ST/V00235X/1, (2) Leverhulme
grant RPG-2019-361, (3) the excellent facilities and support of SuperComputing
Wales. The SDO data used in this paper is courtesy of NASA/SDO and the AIA,
EVE, and HMI science teams.

## References

  * Aschwanden (2013) Aschwanden, M. J. 2013, ApJ, 763, 115, doi: 10.1088/0004-637X/763/2/115
  * Bellot Rubio & Orozco Suárez (2019) Bellot Rubio, L., & Orozco Suárez, D. 2019, Living Reviews in Solar Physics, 16, 1, doi: 10.1007/s41116-018-0017-1
  * Boocock & Tsiklauri (2019) Boocock, C. M., & Tsiklauri, D. 2019, A&A, 625, A47, doi: 10.1051/0004-6361/201834684
  * Chian et al. (2019) Chian, A. C. L., Silva, S. S. A., Rempel, E. L., et al. 2019, MNRAS, 488, 3076, doi: 10.1093/mnras/stz1909
  * Khomenko et al. (2003) Khomenko, E. V., Collados, M., Solanki, S. K., Lagg, A., & Trujillo Bueno, J. 2003, A&A, 408, 1115, doi: 10.1051/0004-6361:20030604
  * Kuridze et al. (2019) Kuridze, D., Mathioudakis, M., Morgan, H., et al. 2019, ApJ, 874, 126, doi: 10.3847/1538-4357/ab08e9
  * Lemen et al. (2011) Lemen, J. R., Title, A. M., Akin, D. J., et al. 2011, Sol. Phys., 172, doi: 10.1007/s11207-011-9776-8
  * Morgan & Druckmüller (2014) Morgan, H., & Druckmüller, M. 2014, Sol. Phys., 289, 2945, doi: 10.1007/s11207-014-0523-9
  * Morgan & Hutton (2018) Morgan, H., & Hutton, J. 2018, ApJ, 853, 145, doi: 10.3847/1538-4357/aaa4b9
  * Morgan & Korsos (2022) Morgan, H. & Korsos, M. 2022, arXiv:2206.09640
  * Morgan & Pickering (2019) Morgan, H., & Pickering, J. 2019, Sol. Phys., 294, 135, doi: 10.1007/s11207-019-1525-4
  * Noori et al. (2019) Noori, M., Javaherian, M., Safari, H., & Nadjari, H. 2019, Advances in Space Research, 64, 504, doi: https://doi.org/10.1016/j.asr.2019.04.027
  * Pesnell et al. (2012) Pesnell, W. D., Thompson, B. J., & Chamberlin, P. C. 2012, Sol. Phys., 275, 3, doi: 10.1007/s11207-011-9841-3
  * Pickering & Morgan (2019) Pickering, J., & Morgan, H. 2019, Sol. Phys., 294, 136, doi: 10.1007/s11207-019-1526-3
  * Requerey et al. (2018) Requerey, I. S., Cobo, B. R., Gošić, M., & Bellot Rubio, L. R. 2018, A&A, 610, A84, doi: 10.1051/0004-6361/201731842
  * Rincon & Rieutord (2018) Rincon, F., & Rieutord, M. 2018, Living Reviews in Solar Physics, 15, 6, doi: 10.1007/s41116-018-0013-5
  * Robustini et al. (2019) Robustini, C., Esteban Pozuelo, S., Leenaarts, J., & de la Cruz Rodríguez, J. 2019, A&A, 621, A1, doi: 10.1051/0004-6361/201833246
  * Sakurai (1982) Sakurai, T. 1982, Sol. Phys., 76, 301, doi: 10.1007/BF00170988
  * Scherrer et al. (2012) Scherrer, P. H., Schou, J., Bush, R. I., et al. 2012, Sol. Phys., 275, 207, doi: 10.1007/s11207-011-9834-2
  * Schrijver & Title (2003) Schrijver, C. J., & Title, A. M. 2003, ApJ, 597, L165, doi: 10.1086/379870
  * Sheeley et al. (2014) Sheeley, N. R., J., Warren, H. P., Lee, J., et al. 2014, ApJ, 797, 131, doi: 10.1088/0004-637X/797/2/131
  * Stenborg et al. (2011) Stenborg, G., Marsch, E., Vourlidas, A., Howard, R., & Baldwin, K. 2011, A&A, 526, A58, doi: 10.1051/0004-6361/201014369
  * Tian et al. (2010) Tian, H., Potts, H. E., Marsch, E., Attie, R., & He, J. S. 2010, A&A, 519, A58, doi: 10.1051/0004-6361/200913254
  * Wang et al. (2009) Wang, T. J., Ofman, L., & Davila, J. M. 2009, ApJ, 696, 1448, doi: 10.1088/0004-637X/696/2/1448
  * Wedemeyer-Böhm et al. (2009) Wedemeyer-Böhm, S., Lagg, A., & Nordlund, Å. 2009, Space Sci. Rev., 144, 317, doi: 10.1007/s11214-008-9447-8
  * Wiegelmann et al. (2014) Wiegelmann, T., Thalmann, J. K., & Solanki, S. K. 2014, A&A Rev., 22, 78, doi: 10.1007/s00159-014-0078-7
  * Wiegelmann et al. (2010) Wiegelmann, T., Solanki, S. K., Borrero, J. M., et al. 2010, ApJ, 723, L185, doi: 10.1088/2041-8205/723/2/L185

##  Appendix A Quantifying the agreement between two binary spatial
distributions

Two-dimensional masks $M_{0}$ and $M_{1}$, both with $n$ pixels and the same
shape, contain values of only zeros and ones, with the ones (zeros)
corresponding to some definition of true (false). For example, the blue and
green regions of figure 3a correspond to true values for $M_{0}$ and $M_{1}$
respectively. $M_{0}$ is convolved with a narrow two-dimensional Gaussian
kernel $k$ giving smoothed mask $S_{0}$:

$S_{0}=M_{0}\otimes k,$ (A1)

and the same for $S_{1}$. The purpose of the convolution is to spread narrow
structures in the mask, enabling some overlap between closely-aligned
structures that may not exactly overlap in both masks. In this work, we use a
sigma-width of 1 pixel for $k$. $P_{01}$ is the product of $S_{0}$ and
$S_{1}$: $P_{01}=S_{0}S_{1}$, and a mask $N_{01}$ is defined as
$N_{01}=P_{01}\geq t$, where $t$ is a small threshold. In this work, $t=0.05$.
Finally, a score $c_{01}$ for the agreement between the two masks $M_{0}$ and
$M_{1}$ is given by

$c_{01}=\frac{2n\sum N_{01}}{\sum N_{00}+\sum N_{11}},$ (A2)

where the summation is over all pixels. The denominator is a normalising
factor, where $N_{00}=P_{00}\geq t$, and $P_{00}=S_{0}S_{0}$ (and similar for
$N_{11}$). The demoniator ensures that $c_{01}=1$ for the case of
$M_{0}=M_{1}$, and $0\leq c<1$ for the general case.

A random test is used to quantify the significance of the score $c_{01}$.
$M_{0}$ is segmented into a set of coherent regions (groups of connected
pixels). Each group’s centroid is translated to a random location, with the
connected pixels rotated by a random angle around this new centroid. If
coordinates of the translated/rotated region lie outside of the bounds of
$M_{0}$, those coordinates are wrapped to the opposite boundary. Thus a new
random mask is created which has similar characteristics to the unmodified
mask, composed of the moved and rotated connected regions of $M_{0}$. A score
$c$ is calculated between this random mask and $M_{1}$. This process is
repeated many times (1000 for this work), with the score $c$ recorded for each
case. The resulting normal distribution of $c$ allows a test of the
significance of $c_{01}$ through evaluating the mean $m$ and standard
deviation $\sigma$ of $c$, and calculating $c_{01}$’s distance from $m$ in
units of $\sigma$. More simply, we can assign a percentage significance by
counting the number of cases of the random distribution with a lower score
than the actual distribution. For example, if 900 of the 1000 random cases
have a lower score than the actual, then the spatial agreement percentage is
90%.