# Energy Propagation in Scattering Convolution Networks Can Be Arbitrarily
Slow

Hartmut Führ, Max Getter<EMAIL_ADDRESS><EMAIL_ADDRESS>aachen.de Lehrstuhl für Geometrie und Analysis, RWTH Aachen University,
D-52056 Aachen, Germany

###### Abstract.

We analyze energy decay for deep convolutional neural networks employed as
feature extractors, such as Mallat’s wavelet scattering transform. For time-
frequency scattering transforms based on Gabor filters, it has been
established that energy decay is exponential, for arbitrary square-integrable
input signals. Our main results allow to prove that this is wrong for wavelet
scattering in arbitrary dimensions. In this setting, the energy decay of the
scattering transform acting on a generic square-integrable signal turns out to
be arbitrarily slow. The fact that this behavior holds for dense subsets of
$L^{2}(\mathbb{R}^{d})$ emphasizes that fast energy decay is generally not a
stable property of signals.

We complement these findings with positive results allowing to conclude fast
(up to exponential) energy decay for generalized Sobolev spaces that are
tailored to the frequency localization of the underlying filter bank.

Both negative and positive results highlight that energy decay in scattering
networks critically depends on the interplay of the respective frequency
localizations of the signal on the one hand, and of the employed filters on
the other.

Keywords: Deep convolutional neural networks, scattering transform, energy
propagation, filter banks, wavelets

AMS Subject Classification: Primary: 42C15, 68T07 Secondary: 42C40, 42B35

##  1\. Introduction

###  1.1. Motivation and related work

Up to this day, the tremendous success of convolutional neural networks (CNNs)
in computer vision tasks [22] like image classification is only partially
understood. This outstanding performance of CNNs is generally believed to be
due to their ability to capture information at multiple scales through the use
of convolutions and subsequently aggregating the information observed at these
scales by means of further operations such as pooling. Consequently, CNNs are
seemingly able to extract semantic content from images while discarding
irrelevant information. However, the available rigorous analytical
explanations for their success are still incomplete.

In his pioneering work [11], Stéphane Mallat introduced the scattering
transform, a nonlinear operator that cascades convolutions followed by the
modulus non-linearity. The scattering transform can be viewed as an infinitely
deep and infinitely wide CNN with the modulus as non-linearity, predetermined
filters, and no pooling [19]. Due to the use of hand-crafted filters based on
a wavelet construction, there is no need to train the network. The windowed
scattering transform was extended in [19] to allow for the use of other
(optionally learned) filters that are not wavelet-generated, with potentially
different filters for different layers, other non-linearities than the
modulus, and pooling operations between layers. The present work studies the
scattering transform in a more classical fashion, with, for simplicity of
exposition, identical (not necessarily wavelet-generated) filters across
network layers, the modulus as non-linearity, and no pooling.

Despite the absence of a training phase and pooling, Mallat-type scattering
networks are still performing comparably well to other state-of-the-art
models, even after more than a decade of progress in machine learning since
their invention (e.g., [15, 23]). They are used in various applications such
as vision and audio tasks, quantum chemistry, medicine, astronomy, and
manifold learning [1, 2, 5, 6, 8, 14, 16, 17]. In many of these, the
scattering transform (or a variant thereof) is employed in the preprocessing
phase of the input data in that it acts as a feature extractor, which is
supposed to improve the overall performance of the machine learning model.

Such scattering networks satisfy many of the nice properties that are
generally believed to be essential in a feature extractor [11, 12, 19, 13].
One of these properties, which is of particular importance both from a
practical as well as a theoretical point of view, is the question of fast
energy propagation across the network layers: Under mild assumptions on the
filters, the energy of any input signal decomposes, for every
$N\in\mathbb{N}$, into the aggregated energy that is contained in the first
$N$ layers of the scattering network and some energy remainder (see
Proposition 1.3 for details). Thus, the scattering transform preserves the
energy of an input signal if and only if the corresponding energy remainder
converges to zero as $N\to\infty$. In this sense, all of the original
information about the input signal is then still contained in the features
generated by the scattering network. In practice, however, only finitely many
features of the infinite scattering network can be computed. Therefore, a
lower bound for the loss of information when restricting the network to
finitely many features of the first $N$ layers is given by the corresponding
energy remainder. Hence, fast decay of this quantity is desirable.

Energy conservation in the above sense was first established in [11, Theorem
2.6] for wavelet-generated filter banks under a rather restrictive and
technical admissibility assumption. Under less restrictive assumptions on the
wavelets, but only in dimension $d=1$, the decay of the energy remainder was
shown [18, Theorem 3.1] to be at least of exponential order
$\mathcal{O}(\alpha^{N})$ for Sobolev functions with some unspecified, signal-
independent $\alpha\in(0,1)$. In recent years, quantitative results in a
similar fashion were also derived for other types of filter banks. Energy
decay was proven (in arbitrary dimension) to be at least of exponential order
$\mathcal{O}(\alpha^{N})$ for all finite-energy input signals, with some
unspecified, signal-independent $\alpha\in(0,1)$, if the filters of the
underlying scattering network form a so-called uniform covering frame [3,
Proposition 3.1]. In particular, this class of filter banks includes certain
Weyl-Heisenberg (Gabor) frames [3, Proposition 2.3]. However, the uniform
covering property requires the Fourier supports of the filters to be uniformly
bounded, so that a broad range of commonly used filter banks in signal
processing (e.g., wavelet-generated filter banks) is not adressed by this
result.

With refined estimates, but based on an idea similar to that of [18, Theorem
3.1], energy decay was proven to be at least of exponential order
$\mathcal{O}(\alpha^{N})$ for Sobolev functions if the filter bank is
generated by a bandlimited wavelet in dimension $d=1$, with a concretely
specified decay factor $\alpha$ that depends on the bandwidth of the generator
[20, Theorem 2], [21, Theorem 3.1]. If the filters do not necessarily have an
underlying structure as in the case of wavelet-generated filters, but instead
satisfy certain mild analyticity assumptions in arbitrary dimension
$d\in\mathbb{N}$, energy decay was shown to be at least of polynomial order
$\mathcal{O}(N^{-m_{d}})$ for Sobolev functions, where $m_{d}\in(0,1]$ and
$m_{d}\to 0$ as $d\to\infty$ [20, Theorem 1].

So far, an important question has remained unanswered:

Does exponential energy decay hold for the wavelet scattering transform,
globally on $L^{2}({\mathbb{R}}^{d})$?

###  1.2. Contributions

Our first main result, Theorem 2.2, provides a negative answer to the question
of global exponential energy decay for the wavelet scattering transform. In
fact, we show that energy decay can even become arbitrarily slow over
$L^{2}({\mathbb{R}}^{d})$ when emplyoing filter banks that have an underlying
structure similar to wavelet-generated filter banks (including the latter). It
turns out that, for given decay rate, the set of signals that do not obey this
rate in terms of scattering energy decay is dense in
$L^{2}({\mathbb{R}}^{d})$. Informally, energy decay can thus be considered
instable over $L^{2}({\mathbb{R}}^{d})$ for these types of scattering
networks.

We complement these negative findings with our second main result, Theorem
3.9, which is of positive nature. By exploiting the interplay between Fourier
decay of the input signal and frequency concentration of the underlying
filters of the scattering network, we provide explicit upper bounds on the
convergence rates of the energy remainder for large filter-dependent
subclasses of $L^{2}({\mathbb{R}}^{d})$, thereby generalizing, unifying, and
partially improving the findings of the previous works [3, 18, 20, 21].

In view of selecting wavelet-generated filters for the scattering network (as
is the case in many applications), these results have two main implications:

  * •

Fast energy propagation in wavelet scattering networks can only be guaranteed
if a priori knowledge about the global Fourier decay of the input data is
available, with sufficient conditions specified by Corollary 4.7.

  * •

The set of those signals in $L^{2}({\mathbb{R}}^{d})$, for which the mixed
$(\ell^{1},L^{2}({\mathbb{R}}^{d}))$ scattering norm considered in [11,
Section 2.5] is infinite, is dense in $L^{2}({\mathbb{R}}^{d})$. Results in
the spirit of [11, Theorem 2.12], which guarantee stability of the windowed
scattering transform to the action of small diffeomorphisms for signals with
finite mixed $(\ell^{1},L^{2}({\mathbb{R}}^{d}))$ scattering norm, therefore
do not apply to such (in this sense) ill-behaved input signals. On the other
hand, finiteness of this mixed scattering norm can be guaranteed by means of
our positive results, e.g., for all $f\in H^{s}_{\log}({\mathbb{R}}^{d})$,
$s>1$, which generalizes [13, Proposition 2.4] to arbitrary dimension
$d\in\mathbb{N}$ if the generating wavelet is bandlimited.

###  1.3. Structure of the paper

We begin our exposition in Section 1.5 by reviewing the windowed scattering
transform and clarifying our base assumptions on the underlying filters of the
scattering network.

Our results in Section 2 concern filter banks with an inherent structure
similar to those generated by wavelets. The starting point of our analysis is
Lemma 2.1, which states an approximate version of super-additivity of the
energy remainder for signals that are nearly separated in freqeuncy domain. We
build on this result by constructing an adversarial signal by means of an
infinite series of appropriately chosen, nearly pairwise separated (in
frequency domain) signals.

In Section 3, we derive upper bounds on the decay rate of the energy
remainder, which are based on the interplay between Fourier decay of the input
signal and the frequency concentration of the filters. Our results apply to a
broad range of filter banks and their corresponding signal classes. The
filters must satisfy a mild analyticity assumption, which can be seen as a
refinement of the setting considered in [20, Assumption 1].

Finally, we illustrate our results in Section 4 by outlining the consequences
for a certain class of filters with uniformly bounded bandwidth, and for
wavelet-generated filters. In doing so, we recover some existing results from
the literature, and we improve and generalize others. These results are of
considerable independent interest, which is further enhanced by the contrast
to the negative findings from Section 2.

###  1.4. Notation

Fix a dimension $d\in\mathbb{N}$. The sets
$\mathbb{N}^{d}\subseteq\mathbb{N}_{0}^{d}\subseteq\mathbb{Z}^{d}\subseteq{\mathbb{R}}^{d}\subseteq{\mathbb{C}}^{d}$
have their usual meanings. For $S\subseteq{\mathbb{R}}^{d}$ and
$x\in{\mathbb{R}}^{d}$, we define $S_{\geq x}:=\\{s\in S\leavevmode\nobreak\
|\leavevmode\nobreak\ s_{k}\geq x_{k}\leavevmode\nobreak\ \text{ for all
}k\in\\{1,\ldots,d\\}\\}$. We denote the Euclidean inner product of two
vectors $z,w\in{\mathbb{C}}^{d}$ by
$\left<\text{}\,z,w\,\text{}\right>:=\sum_{k=1}^{d}z_{k}\overline{w_{k}}$,
with associated norm $\left\lVert\text{}\cdot\text{}\right\rVert_{2}$, and the
respective open ball in ${\mathbb{R}}^{d}$ with radius $r>0$ and center
$x\in{\mathbb{R}}^{d}$ by $B_{r}(x)\subseteq{\mathbb{R}}^{d}$. Further, we
write $\mathcal{S}_{r,R}(x):=\overline{B_{R}(x)}\setminus B_{r}(x)$ for the
closed spherical shell with center $x$, inner radius $r>0$, and outer radius
$R>0$. For $x=0$, we set $\mathcal{S}_{r,R}:=\mathcal{S}_{r,R}(0)$. By
$\mathds{1}_{M}$, we refer to the indicator function of a set
$M\subseteq{\mathbb{R}}^{d}$. If $M$ is Lebesgue-measurable, its measure is
denoted by $\operatorname{vol}(M)\in[0,\infty]$. The sphere in
${\mathbb{R}}^{d}$ is the set
$\mathbb{S}^{d-1}:=\\{x\in{\mathbb{R}}^{d}\leavevmode\nobreak\
|\leavevmode\nobreak\ \left\lVert\text{}x\text{}\right\rVert_{2}=1\\}$.

As usual, $\operatorname{GL}_{d}({\mathbb{R}})$ is the set of $d\times d$
invertible matrices with entries from ${\mathbb{R}}$, and
$O_{d}({\mathbb{R}})$ is the subgroup of orthogonal matrices. It is well known
that any complex matrix $A$ has a singular value decomposition, where we
denote the value of a smallest (respectively largest) singular value of $A$ by
$\sigma_{\text{min}}(A)$ (respectively $\sigma_{\text{max}}(A)$). We denote
the identity matrix in dimension $d$ by $I_{d}$.

For sequences $E,W:\mathbb{N}\to{\mathbb{R}}$, we write
$W_{N}\in\mathcal{O}(E_{N})$ if there exist $C>0$ and $N_{0}\in\mathbb{N}$
such that $W_{N}\leq C\cdot E_{N}$ holds for all $N\in\mathbb{N}_{\geq
N_{0}}$. We denote the support of a function
$f:{\mathbb{R}}^{d}\to{\mathbb{C}}$ by
$\operatorname{supp}(f):=\overline{\left\\{\text{}x\in{\mathbb{R}}^{d}\leavevmode\nobreak\
|\leavevmode\nobreak\ f(x)\neq 0\text{}\right\\}}$, where the closure is taken
with respect to standard Euclidean topology on ${\mathbb{R}}^{d}$. For
$A\in\operatorname{GL}_{d}({\mathbb{R}})$, we also define
$D_{A}^{1}f:=|\det(A)|f(A\>\cdot)$, and
$D_{A}^{2}f:=|\det(A)|^{\nicefrac{{1}}{{2}}}f(A\>\cdot)$. Moreover, we set
$f^{*}:=\overline{f(-\>\cdot)}$.

For $f\in L^{1}({\mathbb{R}}^{d})$, we define the Fourier transform of $f$ by

$\mathcal{F}f(\cdot):=\widehat{f}(\cdot):=\int_{{\mathbb{R}}^{d}}f(x)\leavevmode\nobreak\
e^{-2\pi i\left<\text{}\,x,\cdot\,\text{}\right>}\>\mathrm{d}\,x\>.$

The Fourier transform extends uniquely to a unitary automorphism of
$L^{2}({\mathbb{R}}^{d})$ by this choice of normalization. Its inverse is the
unique operator extension of the inverse Fourier transform given by

$\mathcal{F}^{-1}f(\cdot):=\widehat{f}(-\>\cdot)=\int_{{\mathbb{R}}^{d}}f(x)\leavevmode\nobreak\
e^{2\pi i\left<\text{}\,x,\cdot\,\text{}\right>}\>\mathrm{d}\,x\>.$

If $\omega:(0,\infty)\to(0,\infty)$ is a nondecreasing function, we define

$\mathcal{F}L_{\omega}^{2}({\mathbb{R}}^{d}):=\left\\{\text{}f\in
L^{2}({\mathbb{R}}^{d})\leavevmode\nobreak\ \middle|\leavevmode\nobreak\
\omega(\left\lVert\text{}\cdot\text{}\right\rVert_{2})\widehat{f}\in
L^{2}({\mathbb{R}}^{d})\text{}\right\\}.$

For $s>0$, setting $\omega_{s}(t):=(1+t^{2})^{\frac{s}{2}}$, the fractional
Sobolev space of class regularity $s$ is
$H^{s}({\mathbb{R}}^{d}):=\mathcal{F}L^{2}_{\omega_{s}}({\mathbb{R}}^{d})$. We
define the $s$-fractional Log-Sobolev space by
$H^{s}_{\log}({\mathbb{R}}^{d}):=\mathcal{F}L^{2}_{\omega_{s,\log}}({\mathbb{R}}^{d})$,
where $\omega_{s,\log}(t):=\ln^{s}(e+t)$. Furthermore, let
$S({\mathbb{R}}^{d})$ denote the Schwartz space on ${\mathbb{R}}^{d}$. For a
Hilbert space $\mathcal{H}$ and a countable set $\Lambda$, we define

$\ell^{2}(\mathcal{H}):=\ell^{2}(\Lambda;\mathcal{H}):=\left\\{f=(f_{\lambda})_{\lambda\in\Lambda}\in\mathcal{H}^{\Lambda}\leavevmode\nobreak\
\middle|\leavevmode\nobreak\
\sum_{\lambda\in\Lambda}\left\lVert\text{}f_{\lambda}\text{}\right\rVert_{\mathcal{H}}^{2}<\infty\right\\},$

which itself is a Hilbert space, equipped with the inner product given by
$\left<\text{}\,f,g\,\text{}\right>:=\sum_{\lambda\in\Lambda}\left<\text{}\,f_{\lambda},g_{\lambda}\,\text{}\right>$
for $f,g\in\ell^{2}(\mathcal{H})$ (with unconditional convergence of the
series).

###  1.5. A brief review of the windowed scattering transform.

A scattering network computes cascades of convolutions followed by the
application of the modulus as non-linearity, with no pooling between the
layers. Given any $\psi\in L^{1}({\mathbb{R}}^{d})\cap
L^{2}({\mathbb{R}}^{d})$, we denote by

$U[\psi]:L^{2}({\mathbb{R}}^{d})\to L^{2}({\mathbb{R}}^{d}),\quad
f\mapsto|f*\psi|$

the single step scattering propagator associated to filtering with the
function $\psi$ (which in this context is thus commonly called a filter).
Moreover, we refer to an ordered sequence
$p=(\psi_{1},\ldots,\psi_{N})\in(L^{1}({\mathbb{R}}^{d})\cap
L^{2}({\mathbb{R}}^{d}))^{N}$ as a path of length $N\in\mathbb{N}$. Iterating
the single step scattering propagators along this path yields the scattering
propagator

$U[p]:L^{2}({\mathbb{R}}^{d})\to L^{2}({\mathbb{R}}^{d}),\quad f\mapsto
U[\psi_{N}]\ldots U[\psi_{1}]f=|\ldots|f*\psi_{1}|\ldots*\psi_{N}|.$

For notational convenience, we define the scattering propagator along the
empty path $e$ (the unique path of length zero) to be the identity operator
$U[e]:=\operatorname{id}_{L^{2}({\mathbb{R}}^{d})}$.

A designated function $\chi\in L^{1}({\mathbb{R}}^{d})\cap
L^{2}({\mathbb{R}}^{d})$ induces a windowed scattering propagator along a path
$p$ of length $N\in\mathbb{N}$ and window $\chi$ (referred to as output-
generating filter), given by

$S[p;\chi]:L^{2}({\mathbb{R}}^{d})\to L^{2}({\mathbb{R}}^{d}),\quad
f\mapsto(U[p]f)*\chi=|\ldots|f*\psi_{1}|\ldots*\psi_{N}|*\chi.$

By Young’s convolution inequality, the operators $U[p]$ and $S[p;\chi]$ are
well-defined.

Our main objective is to study the properties of the operators (see Figure 1)

$\displaystyle
U[P]:L^{2}({\mathbb{R}}^{d})\to\ell^{2}(P;L^{2}({\mathbb{R}}^{d})),\quad
f\mapsto\left(U[p]f\right)_{p\in P}$

and

$\displaystyle
S[P;\chi]:L^{2}({\mathbb{R}}^{d})\to\ell^{2}(P;L^{2}({\mathbb{R}}^{d})),\quad
f\mapsto\left(S[p;\chi]f\right)_{p\in P}$

for certain path sets $P$. In particular, this requires the operators to be
well-defined in the sense that, for any $f\in L^{2}({\mathbb{R}}^{d})$, the
series $\sum_{p\in P}\left\lVert\text{}U[p]f\text{}\right\rVert_{2}^{2}$
converges unconditionally. In this respect, let us now state our main
requirement on the filters, which are the basic building blocks of the
windowed scattering transform.

Figure 1. Illustration of the operators $U[P],S[P;\chi]$ for the path set
$P=\\{e,\psi_{1},\psi_{1}^{\prime},(\psi_{1},\psi_{2}),(\psi_{1},\psi_{2}^{\prime}),(\psi_{1}^{\prime},\psi_{2})\\}$
and output-generating filter $\chi$.

Throughout this paper, we assume that $\mathfrak{F}:=\\{\chi\\}\cup\Psi\subset
L^{1}({\mathbb{R}}^{d})\cap L^{2}({\mathbb{R}}^{d})$ is a countably infinite
collection of functions so that the corresponding filter transform

(1) $\displaystyle
L^{2}({\mathbb{R}}^{d})\to\ell^{2}(\mathfrak{F};L^{2}({\mathbb{R}}^{d})),\
f\mapsto(f*\chi)\cup\left(f*\psi\right)_{\psi\in\Psi}$

is an isometry. This is precisely the case if $\mathfrak{F}$ forms a semi-
discrete Parseval frame [10, Section 5.1.5], i.e., if for all $f\in
L^{2}({\mathbb{R}}^{d})$,

(2)
$\displaystyle\left\lVert\text{}f*\chi\text{}\right\rVert_{2}^{2}+\sum_{\psi\in\Psi}\left\lVert\text{}f*\psi\text{}\right\rVert_{2}^{2}=\left\lVert\text{}f\text{}\right\rVert_{2}^{2}.$

We will frequently use the fact (which follows directly from Parseval’s
theorem and the convolution theorem) that $\mathfrak{F}$ forms a semi-discrete
Parseval frame if and only if $\mathfrak{F}$ satisfies the Littlewood-Paley
condition

(3)
$\displaystyle|\widehat{\chi}(\xi)|^{2}+\sum_{\psi\in\Psi}|\widehat{\psi}(\xi)|^{2}=1\quad\text{a.e.
}\xi\in{\mathbb{R}}^{d}.$

If any of the latter equivalent conditions holds, then every $f\in
L^{2}({\mathbb{R}}^{d})$ admits a decomposition

(4) $\displaystyle f=f*\chi*\chi^{*}+\sum_{\psi\in\Psi}f*\psi*\psi^{*}.$

######  Remark 1.1.

To simplify the notation, we do not assign the filters $\Psi$ an explicit
(countably infinite) index set, unless we work in a more concrete setting.
This precludes the use of multiple occurrences of the same filter, which is
often allowed in the definition of the scattering transform. In principle,
this can be accommodated, primarily at notational cost, by extending the
subsequent discussion to multisets. To us the benefit of such an extension
seems quite limited, which is why we have refrained from making the necessary
adjustments.

We will mainly consider the operators $U[P]$ and $S[P;\chi]$ on nonempty
subsets $P$ of the set of all finite length scattering paths
$\mathcal{P}_{\Psi}:=\bigcup_{N=0}^{\infty}\Psi^{N}$, where we set
$\Psi^{0}:=\\{e\\}$. The windowed scattering transform associated with the
filters $\mathfrak{F}$ is the operator

(5)
$\displaystyle\mathcal{S}[\mathfrak{F}]:=S[\mathcal{P}_{\Psi};\chi]:L^{2}({\mathbb{R}}^{d})\to\ell^{2}(\mathcal{P}_{\Psi};L^{2}({\mathbb{R}}^{d})),\quad
f\mapsto\left(S[p;\chi]f\right)_{p\in\mathcal{P}_{\Psi}}.$

In the interest of generality, we do not make further assumptions on the
Parseval frame $\mathfrak{F}$ for now. Even in this very general setting, the
windowed scattering transform $\mathcal{S}[\mathfrak{F}]$ (see Figure 2) is
always a well-defined operator into the Hilbert space
$\ell^{2}(\mathcal{P}_{\Psi};L^{2}({\mathbb{R}}^{d}))$. This will be adressed
in more detail by Proposition 1.3.

Figure 2. Indication of the architecture of a scattering network as described
above, with filters $\psi_{j},\psi_{j}^{\prime}\in\Psi$ corresponding to the
$j$-th network layer, $j\in\\{1,2,3\\}$. The function $\chi$ is the output-
generating filter, which is (in our setup) the same across all layers.

Typically, the filters are concentrated in different areas in frequency
domain, with $\chi$ taking on the special role of a low-pass filter and the
remaining filters $\Psi$ being of high-pass nature. For the sake of
concreteness and later use, let us give one particular example of an
admissible collection $\mathfrak{F}$ in dimension $d=1$, where in this case
$\mathfrak{F}$ is generated from a bandlimited wavelet.

######  Example 1.2.

Let $\psi,\phi\in L^{1}({\mathbb{R}})\cap L^{2}({\mathbb{R}})$ satisfy
$\operatorname{supp}(\widehat{\psi})\subseteq[\frac{1}{2},2]$ and

$\displaystyle|\widehat{\phi}(\xi)|^{2}+\sum_{j=1}^{\infty}\left(|\widehat{\psi}(2^{-j}\cdot\xi)|^{2}+|\widehat{\psi}(-2^{-j}\cdot\xi)|^{2}\right)=1,\quad\text{a.e.
}\xi\in{\mathbb{R}}.$

Such a pair of functions arises, e.g., from the construction of an orthogonal
wavelet basis, with father wavelet (scaling function) $\phi$ and mother
wavelet $\psi$. Then,
$\mathfrak{F}:=\\{\chi\\}\cup\\{\psi_{j}\leavevmode\nobreak\
|\leavevmode\nobreak\ j\in\mathbb{Z}\setminus\\{0\\}\\}$ is a semi-discrete
Parseval frame, where $\chi:=\phi$ and

$\displaystyle\psi_{j}:=\begin{cases}D^{1}_{2^{j}}\psi=2^{j}\psi(2^{j}\>\cdot)&\text{for
}j\in\mathbb{N}\\\
D^{1}_{2^{|j|}}\psi(-\>\cdot)=2^{|j|}\psi(-2^{|j|}\>\cdot)&\text{for
}j\in\mathbb{Z}_{<0}\end{cases}.$

The filter transform is in this case the associated wavelet transform.

The following proposition, which was first stated in (the proof of) [11,
Theorem 2.6] and later generalized in [19, Proposition 1] and [20, Proposition
1], forms the basis for all further analysis in this paper.

######  Proposition 1.3.

Every $f\in L^{2}({\mathbb{R}}^{d})$ satisfies the energy decomposition, for
all $N\in\mathbb{N}_{0}$,

(6)
$\displaystyle\sum_{n=0}^{N-1}\left\lVert\text{}S[\Psi^{n};\chi]f\text{}\right\rVert_{\ell^{2}(\Psi^{n};L^{2}({\mathbb{R}}^{d}))}^{2}+\left\lVert\text{}U[\Psi^{N}]f\text{}\right\rVert_{\ell^{2}(\Psi^{N};L^{2}({\mathbb{R}}^{d}))}^{2}=\left\lVert\text{}f\text{}\right\rVert_{2}^{2}.$

Consequently,

$\left\lVert\text{}\mathcal{S}[\mathfrak{F}]f\text{}\right\rVert_{\ell^{2}(\mathcal{P}_{\Psi};L^{2}({\mathbb{R}}^{d}))}\leq\left\lVert\text{}f\text{}\right\rVert_{2},$

which ensures the well-definedness of the operator
$\mathcal{S}[\mathfrak{F}]:L^{2}({\mathbb{R}}^{d})\to\ell^{2}(\mathcal{P}_{\Psi};L^{2}({\mathbb{R}}^{d}))$.

Moreover,

(7)
$\displaystyle\lim_{N\to\infty}\left\lVert\text{}U[\Psi^{N}]f\text{}\right\rVert_{\ell^{2}(\Psi^{N};L^{2}({\mathbb{R}}^{d}))}^{2}=0$

holds if and only if

(8)
$\displaystyle\left\lVert\text{}\mathcal{S}[\mathfrak{F}]f\text{}\right\rVert_{\ell^{2}(\mathcal{P}_{\Psi};L^{2}({\mathbb{R}}^{d}))}^{2}=\left\lVert\text{}f\text{}\right\rVert_{2}^{2}.$

The energy decomposition identity (6) quantifies the loss of information about
the original input signal $f$ if the scattering network is restricted to its
first $N$ layers. Motivated by this important relationship, let us therefore
define the quantity, for any given $\emptyset\neq\Phi\subseteq\mathfrak{F}$
and $N\in\mathbb{N}_{0}$,

$W_{N}[\Phi](f):=\left\lVert\text{}U[\Phi^{N}]f\text{}\right\rVert_{\ell^{2}(\Phi^{N};L^{2}({\mathbb{R}}^{d}))}^{2}=\sum_{p\in\Phi^{N}}\left\lVert\text{}U[p](f)\text{}\right\rVert_{2}^{2}.$

Our main objective in this paper is to study the asymptotic behavior of the
quantity $W_{N}[\Psi](f)$ as $N\to\infty$, which, as we will show,
distinctively depends on the interplay between the choice of filters $\Psi$
and the input signal $f$. If there is no ambiguity about the underlying
filters $\Psi$, we will sometimes only write $W_{N}(f)$ to refer to
$W_{N}[\Psi](f)$.

As already pointed out in Section 1.1, there are several sufficient criteria
for $\mathfrak{F}$ available that guarantee the asymptotic energy decay (7).
More precisely, [18, Theorem 3.1] and [20, Theorem 1, Theorem 2] state an
upper bound on $W_{N}[\Psi](f)$, for all $f\in L^{2}({\mathbb{R}}^{d})$ and
$N\in\mathbb{N}_{\geq 2}$, which is of the type

(9) $\displaystyle
W_{N}[\Psi](f)\leq\int_{{\mathbb{R}}^{d}}|\widehat{f}(\xi)|^{2}\cdot
K_{N}(\xi)\>\mathrm{d}\,\xi\>$

for a family of integral kernels $(K_{N})_{N\in\mathbb{N}_{\geq 2}}$
(independent of $f$) that satisfy the following properties:

  * •

For all $N\in\mathbb{N}_{\geq 2}$, $K_{N}$ is continuous.

  * •

For all $N\in\mathbb{N}_{\geq 2}$, $0\leq K_{N+1}\leq K_{N}\leq 1$.

  * •

$\lim_{N\to\infty}K_{N}(\xi)=0$ holds pointwise for all
$\xi\in{\mathbb{R}}^{d}$.

By Dini’s theorem [4, Theorem 7.3], this upper integral bound entails, for all
$f\in L^{2}({\mathbb{R}}^{d})$, the convergence (7) (cf. [20, Proposition 2]).
In particular, the scattering transform $\mathcal{S}[\mathfrak{F}]$ is norm-
preserving in these settings by Proposition 1.3. Specifically the assumptions
of [20, Theorem 1] allow for some flexibility in the choice of the filters
while guaranteeing norm-preservation. Clearly, this implies that the single
signal that produces zero-features in every layer by means of the scattering
transform is the trivial null-signal $f=0$.

In order to produce many informative (in the sense of nontrivial) features by
means of the scattering transform, one typically chooses an output-generating
filter $\chi$ that is of low-pass type. By the Littlewood-Paley condition (3),
the remaining filters $\Psi$ are automatically of high-pass nature. Let us
briefly illustrate how informativeness of $\mathcal{S}[\mathfrak{F}]$ can be
achieved by imposing a low-pass condition on $\chi$: Let $0\neq g,\chi\in
L^{1}({\mathbb{R}}^{d})\cap L^{2}({\mathbb{R}}^{d})$ and suppose that
$\widehat{\chi}(0)\neq 0$. By continuity of $\widehat{\chi}$ and
$\mathcal{F}(|g|)$, and since
$\mathcal{F}(|g|)(0)=\left\lVert\text{}g\text{}\right\rVert_{1}>0$, there
exist $\varepsilon,\delta>0$ such that
$|\widehat{\chi}|\cdot|\mathcal{F}(|g|)|\geq\varepsilon$ on $B_{\delta}(0)$.
Thus, we have

$\left\lVert\text{}|g|*\chi\text{}\right\rVert_{2}^{2}=\left\lVert\text{}\mathcal{F}(|g|)\cdot\widehat{\chi}\text{}\right\rVert_{2}^{2}\geq\int_{B_{\delta}(0)}|\mathcal{F}(|g|)(\xi)|^{2}\cdot|\widehat{\chi}(\xi)|^{2}\>\mathrm{d}\,\xi\>\geq\varepsilon^{2}\cdot\operatorname{vol}(B_{\delta}(0))>0.$

Hence, any feature of this type is nontrivial. The assumption that
$\widehat{\chi}(0)\neq 0$ is of vital importance for the argument, as is shown
in [20, Appendix A] by the example of a scattering network (feature extractor)
with nontrivial kernel.

For the remainder of this section, we will collect some basic auxiliary
statements about the windowed scattering transform that will prove useful
later. We include their proofs for self-containment.

######  Lemma 1.4.

Let $N\in\mathbb{N}$. Then
$U[\mathfrak{F}^{N}]:L^{2}({\mathbb{R}}^{d})\to\ell^{2}(\mathfrak{F};L^{2}({\mathbb{R}}^{d}))$
is well-defined and norm-preserving, i.e., for all $f\in
L^{2}({\mathbb{R}}^{d})$,

$W_{N}[\mathfrak{F}](f)=\sum_{p\in\mathfrak{F}^{N}}\left\lVert\text{}U[p]f\text{}\right\rVert_{2}^{2}=\left\lVert\text{}f\text{}\right\rVert_{2}^{2}.$

###### Proof.

This is easily shown by induction: The base case $N=1$ is exactly one of the
characterizations of a semi-discrete Parseval frame, hence true for
$\mathfrak{F}$. The induction step is established as follows. Suppose that the
statement is true for some $N\in\mathbb{N}$. Then, for all $f\in
L^{2}({\mathbb{R}}^{d})$, we have

$\displaystyle
W_{N+1}[\mathfrak{F}](f)=\sum_{\psi\in\mathfrak{F}}W_{N}[\mathfrak{F}](|f*\psi|)=\sum_{\psi\in\mathfrak{F}}\left\lVert\text{}|f*\psi|\text{}\right\rVert_{2}^{2}=\sum_{\psi\in\mathfrak{F}}\left\lVert\text{}f*\psi\text{}\right\rVert_{2}^{2}=\left\lVert\text{}f\text{}\right\rVert_{2}^{2}.$

∎

Despite its simplicity and brevity, the argument illustrates how the
underlying inductive structure of the scattering network can be exploited in
proofs. The following Proposition (cf. [11, Proposition 2.5]) states the non-
expansiveness of the operators $U$ and $S$, which is also due to their
inductive definition. Considering the desirable properties of the scattering
transform as a feature extractor, this property can be considered as stability
against additive noise.

######  Proposition 1.5.

Let $\emptyset\neq P\subseteq\Psi^{N}$ for some $N\in\mathbb{N}_{0}$. Then,
for all $f,g\in L^{2}({\mathbb{R}}^{d})$,

$\displaystyle\left\lVert\text{}S[P;\chi]f-S[P;\chi]g\text{}\right\rVert_{\ell^{2}(P;L^{2}({\mathbb{R}}^{d}))}\leq\left\lVert\text{}U[P]f-U[P]g\text{}\right\rVert_{\ell^{2}(P;L^{2}({\mathbb{R}}^{d}))}\leq\left\lVert\text{}f-g\text{}\right\rVert_{2}.$

Moreover, if $\mathcal{S}[\mathfrak{F}]=S[\mathcal{P}_{\Psi};\chi]$ is norm-
preserving, then it is also nonexpansive.

###### Proof.

By the Littlewood-Paley condition (3), we have
$\left\lVert\text{}\widehat{\chi}\text{}\right\rVert_{\infty}\leq 1$. Thus,
for all $p\in P$, and all $f,g\in L^{2}({\mathbb{R}}^{d})$,

$\left\lVert\text{}S[p;\chi]f-S[p;\chi]g\text{}\right\rVert_{2}=\left\lVert\text{}(U[p]f-U[p]g)*\chi\text{}\right\rVert_{2}\leq\left\lVert\text{}U[p]f-U[p]g\text{}\right\rVert_{2},$

which proves the first inequality.

Observe that it suffices to prove the second inequality for the case
$P=\Psi^{N}$, which we do by induction on $N\in\mathbb{N}_{0}$. The base case
$N=0$ is trivial. For the purpose of the induction step, suppose that the
hypothesis is true for some $N\in\mathbb{N}_{0}$. By the reverse triangle
inequality, we have, for all $f,g\in L^{2}({\mathbb{R}}^{d})$,

$\displaystyle\lvert|U[\Psi^{N+1}]f-$ $\displaystyle
U[\Psi^{N+1}]g\rvert|_{\ell^{2}(\Psi^{N+1};L^{2}({\mathbb{R}}^{d}))}^{2}$
$\displaystyle=\sum_{p\in\Psi^{N}}\sum_{\psi\in\Psi}\left\lVert\text{}|(U[p]f)*\psi|-|(U[p]g)*\psi|\text{}\right\rVert_{2}^{2}$
$\displaystyle\leq\sum_{p\in\Psi^{N}}\sum_{\psi\in\Psi}\left\lVert\text{}|(U[p]f-U[p]g)*\psi|\text{}\right\rVert_{2}^{2}$
$\displaystyle=\sum_{p\in\Psi^{N}}\sum_{\psi\in\Psi}\left\lVert\text{}(U[p]f-U[p]g)*\psi\text{}\right\rVert_{2}^{2}$
$\displaystyle=\sum_{p\in\Psi^{N}}(\left\lVert\text{}U[p]f-U[p]g\text{}\right\rVert_{2}^{2}-\left\lVert\text{}S[p;\chi]f-S[p;\chi]g\text{}\right\rVert_{2}^{2})$
$\displaystyle=\left\lVert\text{}U[\Psi^{N}]f-U[\Psi^{N}]g\text{}\right\rVert_{\ell^{2}(\Psi^{N};L^{2}({\mathbb{R}}^{d}))}^{2}-\left\lVert\text{}S[\Psi^{N};\chi]f-S[\Psi^{N};\chi]g\text{}\right\rVert_{\ell^{2}(\Psi^{N};L^{2}({\mathbb{R}}^{d}))}^{2},$

where the second last step is due to $\mathfrak{F}$ being a frame. Applying
the induction hypothesis to the right-hand side of the latter inequality
concludes this part of the proof.

Finally, if $\mathcal{S}[\mathfrak{F}]$ is norm-preserving, the non-
expansiveness of $\mathcal{S}[\mathfrak{F}]$ follows directly from the above
estimates (which were derived independent of the induction hypothesis) using a
telescoping series argument. In fact, by Proposition 1.3, we have
$\lim_{N\to\infty}W_{N}[\Psi](h)=0$ for all $h\in L^{2}({\mathbb{R}}^{d})$,
which entails that, for all $f,g\in L^{2}({\mathbb{R}}^{d})$,

$\lim_{N\to\infty}\left\lVert\text{}U[\Psi^{N}]f-U[\Psi^{N}]g\text{}\right\rVert_{\ell^{2}(\Psi^{N};L^{2}({\mathbb{R}}^{d}))}^{2}=0.$

Consequently,

$\displaystyle\lvert|\mathcal{S}[\mathfrak{F}]f-$
$\displaystyle\mathcal{S}[\mathfrak{F}]g\rvert|_{\ell^{2}(\mathcal{P}_{\Psi};L^{2}({\mathbb{R}}^{d}))}^{2}$
$\displaystyle=\sum_{N=0}^{\infty}\left\lVert\text{}S[\Psi^{N};\chi]f-S[\Psi^{N};\chi]g\text{}\right\rVert_{\ell^{2}(\Psi^{N};L^{2}({\mathbb{R}}^{d}))}^{2}$
$\displaystyle\leq\sum_{N=0}^{\infty}\left(\left\lVert\text{}U[\Psi^{N}]f-U[\Psi^{N}]g\text{}\right\rVert_{\ell^{2}(\Psi^{N};L^{2}({\mathbb{R}}^{d}))}^{2}-\left\lVert\text{}U[\Psi^{N+1}]f-U[\Psi^{N+1}]g\text{}\right\rVert_{\ell^{2}(\Psi^{N+1};L^{2}({\mathbb{R}}^{d}))}^{2}\right)$
$\displaystyle=\left\lVert\text{}f-g\text{}\right\rVert_{2}^{2}.$

∎

We exploit the non-expansiveness of $U$ to derive some useful estimates
concerning the energy remainder.

######  Lemma 1.6.

Let $\emptyset\neq\Phi\subseteq\Psi$, and let $N\in\mathbb{N}$. Then, we have,
for all $f,g\in L^{2}({\mathbb{R}}^{d})$,

  1. a)

$|W_{N}[\Phi](f)-W_{N}[\Phi](g)|\leq\sqrt{2}\cdot\left\lVert\text{}f-g\text{}\right\rVert_{2}\cdot\sqrt{W_{N}[\Phi](f)+W_{N}[\Phi](g)}$,

  2. b)

$W_{N}[\Phi](f)\geq\frac{1}{2}\cdot
W_{N}[\Phi](g)-\left\lVert\text{}f-g\text{}\right\rVert_{2}^{2}.$

###### Proof.

Let $f,g\in L^{2}({\mathbb{R}}^{d})$. Applying the reverse triangle inequality
yields, for any $p\in\Phi^{N}$,

$\displaystyle|\left\lVert\text{}U[p]f\text{}\right\rVert_{2}^{2}-\left\lVert\text{}U[p]g\text{}\right\rVert_{2}^{2}|$
$\displaystyle=|\left\lVert\text{}U[p]f\text{}\right\rVert_{2}-\left\lVert\text{}U[p]g\text{}\right\rVert_{2}|\cdot(\left\lVert\text{}U[p]f\text{}\right\rVert_{2}+\left\lVert\text{}U[p]g\text{}\right\rVert_{2})$
$\displaystyle\leq\left\lVert\text{}U[p]f-U[p]g\text{}\right\rVert_{2}\cdot(\left\lVert\text{}U[p]f\text{}\right\rVert_{2}+\left\lVert\text{}U[p]g\text{}\right\rVert_{2}).$

Together with the Cauchy-Schwarz inequality and the non-expansiveness of
$U[\Phi^{N}]$, we obtain

$\displaystyle|W_{N}[\Phi](f)-W_{N}[\Phi](g)|^{2}$
$\displaystyle=\left|\sum_{p\in\Phi^{N}}\left\lVert\text{}U[p]f\text{}\right\rVert_{2}^{2}-\left\lVert\text{}U[p]g\text{}\right\rVert_{2}^{2}\right|^{2}$
$\displaystyle\leq\left(\sum_{p\in\Phi^{N}}\left\lVert\text{}U[p]f-U[p]g\text{}\right\rVert_{2}\cdot\left(\left\lVert\text{}U[p]f\text{}\right\rVert_{2}+\left\lVert\text{}U[p]g\text{}\right\rVert_{2}\right)\right)^{2}$
$\displaystyle\leq\left(\sum_{p\in\Phi^{N}}\left\lVert\text{}U[p]f-U[p]g\text{}\right\rVert_{2}^{2}\right)\cdot\left(\sum_{p\in\Phi^{N}}\left(\left\lVert\text{}U[p]f\text{}\right\rVert_{2}+\left\lVert\text{}U[p]g\text{}\right\rVert_{2}\right)^{2}\right)$
$\displaystyle\leq\left\lVert\text{}f-g\text{}\right\rVert_{2}^{2}\cdot\sum_{p\in\Phi^{N}}\left(2\left\lVert\text{}U[p]f\text{}\right\rVert_{2}^{2}+2\left\lVert\text{}U[p]g\text{}\right\rVert_{2}^{2}\right)$
$\displaystyle=2\left\lVert\text{}f-g\text{}\right\rVert_{2}^{2}\cdot(W_{N}[\Phi](f)+W_{N}[\Phi](g)).$

Taking square roots on both sides of the latter inequality proves a).

We again use the non-expansiveness of $U[\Phi]$ and find that

$\displaystyle W_{N}[\Phi](g)$
$\displaystyle=\left\lVert\text{}U[\Phi^{N}](g)-U[\Phi^{N}](f)+U[\Phi^{N}](f)\text{}\right\rVert_{\ell^{2}(\Phi^{N};L^{2}({\mathbb{R}}^{d}))}^{2}$
$\displaystyle\leq\left(\left\lVert\text{}U[\Phi^{N}](g)-U[\Phi^{N}](f)\text{}\right\rVert_{\ell^{2}(\Phi^{N};L^{2}({\mathbb{R}}^{d}))}+\left\lVert\text{}U[\Phi^{N}](f)\text{}\right\rVert_{\ell^{2}(\Phi^{N};L^{2}({\mathbb{R}}^{d}))}\right)^{2}$
$\displaystyle\leq\left(\left\lVert\text{}g-f\text{}\right\rVert_{2}+\left\lVert\text{}U[\Phi^{N}](f)\text{}\right\rVert_{\ell^{2}(\Phi^{N};L^{2}({\mathbb{R}}^{d}))}\right)^{2}$
$\displaystyle\leq
2\left\lVert\text{}f-g\text{}\right\rVert_{2}^{2}+2\left\lVert\text{}U[\Phi^{N}](f)\text{}\right\rVert_{\ell^{2}(\Phi^{N};L^{2}({\mathbb{R}}^{d}))}^{2}$
$\displaystyle=2\left\lVert\text{}f-g\text{}\right\rVert_{2}^{2}+2W_{N}[\Phi](f).$

By rearranging, we conclude the proof of b). ∎

It is now easy to derive the continuity of $W_{N}[\Phi]$, which we will use to
prove the density result, Proposition 2.3, in Section 2.

######  Corollary 1.7.

Let $\emptyset\neq\Phi\subseteq\Psi$, and let $N\in\mathbb{N}$. Then,
$W_{N}[\Phi]:L^{2}({\mathbb{R}}^{d})\to{\mathbb{R}}$ is continuous.

###### Proof.

Continuity at $f=0$ is obvious, since we have, for all $g\in
L^{2}({\mathbb{R}}^{d})$,
$W_{N}[\Phi](g)\leq\left\lVert\text{}g\text{}\right\rVert_{2}^{2}$. Now, fix
$\varepsilon>0$ and $f\in L^{2}({\mathbb{R}}^{d})\setminus\\{0\\}$. Define

$\delta=\delta(f,\varepsilon):=\min\left\\{\frac{\varepsilon}{\sqrt{10}\left\lVert\text{}f\text{}\right\rVert_{2}},\left\lVert\text{}f\text{}\right\rVert_{2}\right\\}>0.$

Applying Lemma 1.6a) to $g\in L^{2}({\mathbb{R}}^{d})$ with
$\left\lVert\text{}f-g\text{}\right\rVert_{2}<\delta$ gives

$\displaystyle|W_{N}[\Phi](f)-W_{N}[\Phi](g)|^{2}$ $\displaystyle\leq
2\left\lVert\text{}f-g\text{}\right\rVert_{2}^{2}\cdot(W_{N}[\Phi](f)+W_{N}[\Phi](g))$
$\displaystyle\leq
2\delta^{2}\cdot(\left\lVert\text{}f\text{}\right\rVert_{2}^{2}+\left\lVert\text{}g\text{}\right\rVert_{2}^{2})$
$\displaystyle\leq
2\delta^{2}\cdot(\left\lVert\text{}f\text{}\right\rVert_{2}^{2}+(\left\lVert\text{}f\text{}\right\rVert_{2}+\delta)^{2})$
$\displaystyle\leq
10\delta^{2}\cdot\left\lVert\text{}f\text{}\right\rVert_{2}^{2}\leq\varepsilon^{2}.$

∎

We conclude this section with a lemma describing the interplay of the
scattering propagator with dilations, which is similar to covariance (cf. [11,
Equation (20)]). Despite its elementary proof, this property forms the basis
for our main result in Section 2.

######  Lemma 1.8.

Let $N\in\mathbb{N}$, let $p\in(L^{1}({\mathbb{R}}^{d})\cap
L^{2}({\mathbb{R}}^{d}))^{N}$, and let
$A\in\operatorname{GL}_{d}({\mathbb{R}})$. Then, for all $f\in
L^{2}({\mathbb{R}}^{d})$,

$\displaystyle U[p]D_{A}^{1}f=D_{A}^{1}U[D_{A^{-1}}^{1}p]f,$

where $D_{A^{-1}}^{1}p:=(D_{A^{-1}}^{1}p_{1},\ldots,D_{A^{-1}}^{1}p_{N})$.
Moreover, if $\emptyset\neq\Phi\subseteq\mathfrak{F}$, then we have, for all
$f\in L^{2}({\mathbb{R}}^{d})$,

$W_{N}[\Phi](D_{A}^{2}f)=W_{N}[D_{A^{-1}}^{1}\Phi](f).$

###### Proof.

Let $f\in L^{2}({\mathbb{R}}^{d})$, and let $g\in L^{1}({\mathbb{R}}^{d})\cap
L^{2}({\mathbb{R}}^{d})$. A direct computation shows that

$(D_{A}^{1}f)*g=D_{A}^{1}(f*(D_{A^{-1}}^{1}g)).$

The modulus operator

$M:L^{2}({\mathbb{R}}^{d})\to L^{2}({\mathbb{R}}^{d}),\quad h\mapsto|h|$

commutes with $D_{A}^{1}$, which proves
$U[g]D_{A}^{1}f=D_{A}^{1}U[D_{A^{-1}}^{1}g]f$. Iterating this relation along
the path $p$ concludes the first identity.

Concerning the second, we note that $D_{A}^{1}=|\det(A)|^{\frac{1}{2}}\cdot
D_{A}^{2}$, which gives $U[p]D_{A}^{2}=D_{A}^{2}U[D_{A^{-1}}^{1}p]$. Since
$D_{A}^{2}$ is unitary on $L^{2}({\mathbb{R}}^{d})$, summing over all paths in
$\Phi^{N}$ entails the second identity. ∎

##  2\. Slow Scattering Propagation

In this section, we show that energy propagation can be arbitrarily slow in
scattering networks that employ filters $\mathfrak{F}=\\{\chi\\}\cup\Psi$ with
an underlying structure similar to those generated by wavelets.

As some of the details of this section are quite technical, we begin by
illustrating the conceptual idea behind them. In light of this, let us
consider the situation from Example 1.2 again, i.e., suppose that the high-
pass filters $\Psi=\\{\psi_{j}\leavevmode\nobreak\ |\leavevmode\nobreak\
j\in\mathbb{Z}\setminus\\{0\\}\\}$ are generated by dilations of $\psi\in
L^{1}({\mathbb{R}})\cap L^{2}({\mathbb{R}})$,
$\operatorname{supp}(\widehat{\psi})\subseteq[\frac{1}{2},2]$, according to

$\displaystyle\psi_{j}:=\begin{cases}D^{1}_{2^{j}}\psi=2^{j}\psi(2^{j}\>\cdot)&\text{for
}j\in\mathbb{N}\\\
D^{1}_{2^{|j|}}\psi(-\>\cdot)=2^{|j|}\psi(-2^{|j|}\>\cdot)&\text{for
}j\in\mathbb{Z}_{<0}\end{cases}.$

We want to show that for any nonincreasing null-sequence
$E\in{\mathbb{R}}_{>0}^{\mathbb{N}}$, there exists a function $f_{E}\in
L^{2}({\mathbb{R}})$ that satisfies the following conditions:

  1. a)

We have $\left\lVert\text{}f_{E}\text{}\right\rVert_{2}^{2}=E_{1}$.

  2. b)

For all $N\in\mathbb{N}$, we have that $W_{N}[\Psi](f)\geq\frac{1}{2}\cdot
E_{N}$.

Proof sketch. Define

$(a_{k})_{k\in\mathbb{N}}:=\left(\sqrt{E_{k}-E_{k+1}}\right)_{k\in\mathbb{N}},$

and note that a telescoping argument gives

$\lim_{N\to\infty}\sum_{k=1}^{N}a_{k}^{2}=\lim_{N\to\infty}E_{1}-E_{N+1}=E_{1}.$

Fix any $f_{0}\in L^{2}({\mathbb{R}})$ that satisfies
$\left\lVert\text{}f_{0}\text{}\right\rVert_{2}=1$ and
$\operatorname{supp}(\widehat{f_{0}})\subseteq[1,2]$. We will later (see
Corollary 2.8 and Corollary 4.7) show that, for all $N\in\mathbb{N}$,

$\lim_{m\to\infty}W_{N}\left(f_{m}\right)=1,$

where $f_{m}:=2^{m/2}f_{0}(2^{m}\,\cdot\,)$, $m\in\mathbb{N}$. This is a
consequence of Lemma 1.8.

Therefore, we can find a strictly increasing sequence
$(m_{k})_{k\in\mathbb{N}}\subseteq\mathbb{N}$ such that, for all
$k\in\mathbb{N}$, $m_{k+1}\geq 2+m_{k}$ and $W_{k}(f_{m_{k}})\geq\frac{1}{2}$.
We claim that

$f_{E}:=\sum_{k=1}^{\infty}a_{k}f_{m_{k}}\in L^{2}({\mathbb{R}})$

does the job. To this end, first note that, for all $m\in\mathbb{N}$,
$\operatorname{supp}(\widehat{f_{m}})\subseteq[2^{m},2^{m+1}]$. In particular,
the $(f_{m})_{m\in\mathbb{N}}$ are orthonormal, hence
$\left\lVert\text{}f_{E}\text{}\right\rVert_{2}^{2}=\sum_{k=1}^{\infty}|a_{k}|^{2}=E_{1}$.
Moreover, since

$\operatorname{supp}(\widehat{\psi_{j}})\subseteq[\operatorname{sgn}(j)\cdot
2^{|j|-1},\operatorname{sgn}(j)\cdot 2^{|j|+1}],$

we have

$\displaystyle
f_{E}*\psi_{j}=\begin{cases}a_{k}f_{m_{k}}*\psi_{m_{k}}&\text{if
}j=m_{k}\text{ for some }k\in\mathbb{N}\\\
a_{k}f_{m_{k}}*\psi_{m_{k}+1}&\text{if }j=m_{k}+1\text{ for some
}k\in\mathbb{N}\\\ 0&\text{else}\end{cases},\quad
j\in\mathbb{Z}\setminus\\{0\\}.$

Altogether, we obtain, for all $N\in\mathbb{N}$,

$\displaystyle W_{N}(f_{E})$
$\displaystyle=\sum_{j\in\mathbb{Z}\setminus\\{0\\}}W_{N-1}(|f_{E}*\psi_{j}|)$
$\displaystyle=\sum_{k=1}^{\infty}\left(W_{N-1}(|a_{k}f_{m_{k}}*\psi_{m_{k}}|)+W_{N-1}(|a_{k}f_{m_{k}}*\psi_{m_{k}+1}|)\right)$
$\displaystyle=\sum_{k=1}^{\infty}|a_{k}|^{2}\cdot\left(W_{N-1}(|f_{m_{k}}*\psi_{m_{k}}|)+W_{N-1}(|f_{m_{k}}*\psi_{m_{k}+1}|)\right)$
$\displaystyle=\sum_{k=1}^{\infty}|a_{k}|^{2}\cdot
W_{N}(f_{m_{k}})\geq\sum_{k=N}^{\infty}|a_{k}|^{2}\cdot
W_{k}(f_{m_{k}})\geq\frac{1}{2}\cdot\sum_{k=N}^{\infty}|a_{k}|^{2}=\frac{1}{2}\cdot
E_{N}.$

$\square$

To summarize, the central idea of the proof is as follows: Given $\delta>0$,
construct a sequence of signals $(f_{m_{k}})_{k\in\mathbb{N}}\subseteq
L^{2}({\mathbb{R}})$ such that (i) the energy remainder $W_{N}$ behaves
(super-)additively due to separation in the frequency domain and (ii) the
energy decays slowly for $f_{m_{k}}$ across the first $k$ layers of the
scattering network, according to $W_{k}(f_{m_{k}})\geq\delta$. Weighting the
signals with an $\ell^{2}$-sequence ensures square-integrability of $f_{E}$.
The energy that is contained in the scattering layers of $f_{E}$ of depth
greater than $N-1$ (i.e., $W_{N}(f_{E})$) is controlled by the weighted energy
remainders of the signals $(f_{m_{k}})_{k\geq N}$. This results in a slow
propagation of energy across all network layers. The speed of convergence is
bounded from below by the speed of convergence of the $\ell^{2}$-series of the
chosen weights.

In order to generalize this idea to larger classes of filter banks and higher
dimensions, we need to establish a much more general version of super-
additivity for the family of nonlinear operators
$(W_{N}:L^{2}({\mathbb{R}}^{d})\to{\mathbb{R}})_{N\in\mathbb{N}}$. In the
proof sketch above, we have implicitly used their additivity on separated
signals (cf. [13, Lemma 2.3]). However, if the generator $\psi$ fails to be
bandlimited, we cannot generally expect exact additivity on separated signals.
Moreover, we would like to allow for more flexibility in the choice of our
filter bank. The following lemma establishes an approximate version of super-
additivity for the operators $(W_{N})_{N\in\mathbb{N}}$ if the signals are
nearly separated in Fourier domain. We use the high-pass filters $\Psi$ to
measure the degree of separation.

######  Lemma 2.1.

Let $F=(f_{k})_{k\in\mathbb{N}_{0}}\subset L^{2}({\mathbb{R}}^{d})$, let
$(\eta_{k})_{k\in\mathbb{N}_{0}}\in{\mathbb{R}}_{>0}^{\mathbb{N}_{0}}$, and
let $a=(a_{k})_{k\in\mathbb{N}_{0}}\in\ell^{2}(\mathbb{N}_{0};{\mathbb{C}})$.
Assume that there exist finite sets $\Psi(F,k)\subseteq\Psi$,
$k\in\mathbb{N}_{0}$, such that the following hold:

  1. (i)

The series
$\sum_{k,j=0}^{\infty}|a_{k}a_{j}\left<\text{}\,f_{k},f_{j}\,\text{}\right>|$
converges.

  2. (ii)

The series $\sum_{k=0}^{\infty}\varepsilon_{k}$ converges, where
$\varepsilon_{k}:=(k+1)\cdot\eta_{k}+\sum_{j=k+1}^{\infty}\eta_{j}$.

  3. (iii)

The sets $\Psi(F,k)$, $k\in\mathbb{N}_{0}$, are pairwise disjoint.

  4. (iv)

We have, for every $k\in\mathbb{N}_{0}$,

$\displaystyle\sum_{\psi\in\Psi\setminus\Psi(F,k)}\left\lVert\text{}f_{k}*\psi\text{}\right\rVert_{2}^{2}\leq\eta_{k}.$

  5. (v)

We have, for every $k\in\mathbb{N}_{0}$ and $0\leq j<k$,

$\displaystyle\sum_{\psi\in\Psi(F,k)}\left\lVert\text{}f_{j}*\psi\text{}\right\rVert_{2}^{2}\leq\eta_{k}.$

Then, the series

$f:=\sum_{k=0}^{\infty}a_{k}f_{k}$

converges in $L^{2}({\mathbb{R}}^{d})$ with
$\left\lVert\text{}f\text{}\right\rVert_{2}^{2}\leq\sum_{k,j=0}^{\infty}|a_{k}a_{j}\left<\text{}\,f_{k},f_{j}\,\text{}\right>|$,
and we have, for every $N\in\mathbb{N}$, $n\in\mathbb{N}_{0}$,

(10) $\displaystyle
W_{N}\left(f\right)\geq\sum_{k=n}^{\infty}\left(\frac{|a_{k}|^{2}}{2}\cdot
W_{N}(f_{k})-2\left\lVert\text{}a\text{}\right\rVert_{\ell^{2}({\mathbb{C}})}^{2}\cdot\varepsilon_{k}\right).$

Moreover, suppose that in addition to our standing assumptions (i)-(v) there
exists $\delta>0$ such that

  1. (vi)

for all $k\in\mathbb{N}$,

$W_{k}(f_{k})\geq 4\delta.$

  2. (vii)

for all $k\in\mathbb{N}$,

$\varepsilon_{k}\leq\frac{\delta\cdot|a_{k}|^{2}}{2\left\lVert\text{}a\text{}\right\rVert_{\ell^{2}({\mathbb{C}})}^{2}}.$

Then we even have, for all $N\in\mathbb{N}$,

(11) $\displaystyle W_{N}(f)\geq\delta\cdot\sum_{k=N}^{\infty}|a_{k}|^{2}.$

###### Proof.

The series defining $f$ converges if

$\left\lVert\text{}\sum_{k=n}^{m}a_{k}f_{k}\text{}\right\rVert_{2}\xrightarrow{n,m\to\infty}0.$

Expanding the latter norm squared yields

(12)
$\displaystyle\left\lVert\text{}\sum_{k=n}^{m}a_{k}f_{k}\text{}\right\rVert_{2}^{2}=\sum_{k,j=n}^{m}a_{k}\overline{a_{j}}\cdot\left<\text{}\,f_{k},f_{j}\,\text{}\right>\leq\sum_{k,j=n}^{\infty}|a_{k}a_{j}\cdot\left<\text{}\,f_{k},f_{j}\,\text{}\right>|.$

Being the remainder of a convergent series by assumption (i), the right-hand
side of (12) converges to $0$, so that $f$ is in fact a well-defined element
of $L^{2}({\mathbb{R}}^{d})$. Setting $n=0$ in (12), we also obtain the
claimed bound for the (squared) norm of $f$ as $m\to\infty$.

Let us now turn to the proof of (10). Here, we first note that the recursive
nature of $W_{N}$ and Lemma 1.6 b) imply

$\displaystyle W_{N}(f)$ $\displaystyle=\sum_{\psi\in\Psi}W_{N-1}(|f*\psi|)$
$\displaystyle\geq\sum_{k=n}^{\infty}\sum_{\psi\in\Psi(F,k)}W_{N-1}(|f*\psi|)$
$\displaystyle\geq\sum_{k=n}^{\infty}\sum_{\psi\in\Psi(F,k)}\left(\frac{1}{2}\cdot
W_{N-1}(|(a_{k}f_{k})*\psi|)-\left\lVert\text{}|f*\psi|-|(a_{k}f_{k})*\psi|\text{}\right\rVert_{2}^{2}\right)$
$\displaystyle\geq\sum_{k=n}^{\infty}\sum_{\psi\in\Psi(F,k)}\left(\frac{|a_{k}|^{2}}{2}\cdot
W_{N-1}(|f_{k}*\psi|)-\left\lVert\text{}(f-a_{k}f_{k})*\psi\text{}\right\rVert_{2}^{2}\right)$
(13)
$\displaystyle=\sum_{k=n}^{\infty}\left(\frac{|a_{k}|^{2}}{2}\sum_{\psi\in\Psi(F,k)}W_{N-1}(|f_{k}*\psi|)-\text{Rest}_{k}\right).$

As a consequence of our assumptions (iii)-(v), the remainder is bounded by

$\displaystyle\text{Rest}_{k}$
$\displaystyle=\sum_{\psi\in\Psi(F,k)}\left\lVert\text{}\left(\sum_{j=0,j\neq
k}^{\infty}a_{j}f_{j}\right)*\psi\text{}\right\rVert_{2}^{2}$
$\displaystyle\leq\sum_{\psi\in\Psi(F,k)}\left(\sum_{j=0,j\neq
k}^{\infty}|a_{j}|\cdot\left\lVert\text{}f_{j}*\psi\text{}\right\rVert_{2}\right)^{2}$
$\displaystyle\leq
2\sum_{\psi\in\Psi(F,k)}\left[\left(\sum_{j=0}^{k-1}|a_{j}|\cdot\left\lVert\text{}f_{j}*\psi\text{}\right\rVert_{2}\right)^{2}+\left(\sum_{j=k+1}^{\infty}|a_{j}|\cdot\left\lVert\text{}f_{j}*\psi\text{}\right\rVert_{2}\right)^{2}\right]$
$\displaystyle\leq
2\left\lVert\text{}a\text{}\right\rVert_{\ell^{2}({\mathbb{C}})}^{2}\cdot\sum_{\psi\in\Psi(F,k)}\left(\sum_{j=0}^{k-1}\left\lVert\text{}f_{j}*\psi\text{}\right\rVert_{2}^{2}+\sum_{j=k+1}^{\infty}\left\lVert\text{}f_{j}*\psi\text{}\right\rVert_{2}^{2}\right)$
$\displaystyle\leq
2\left\lVert\text{}a\text{}\right\rVert_{\ell^{2}({\mathbb{C}})}^{2}\cdot\left(k\cdot\eta_{k}+\sum_{j=k+1}^{\infty}\eta_{j}\right).$

Moreover, for every $k\geq n$,

$\displaystyle\sum_{\psi\in\Psi(F,k)}W_{N-1}(|f_{k}*\psi|)$
$\displaystyle=\sum_{\psi\in\Psi}W_{N-1}(|f_{k}*\psi|)-\sum_{\psi\in\Psi\setminus\Psi(F,k)}W_{N-1}(|f_{k}*\psi|)$
$\displaystyle=W_{N}(f_{k})-\sum_{\psi\in\Psi\setminus\Psi(F,k)}W_{N-1}(|f_{k}*\psi|)$
$\displaystyle\geq W_{N}(f_{k})-\eta_{k}.$

Substituting this and the bound for $\text{Rest}_{k}$ into (2) yields

$\displaystyle W_{N}(f)$
$\displaystyle\geq\sum_{k=n}^{\infty}\left(\frac{|a_{k}|^{2}}{2}\sum_{\psi\in\Psi(F,k)}W_{N-1}(|f_{k}*\psi|)-\text{Rest}_{k}\right)$
$\displaystyle\geq\sum_{k=n}^{\infty}\left[\frac{|a_{k}|^{2}}{2}\cdot\left(W_{N}(f_{k})-\eta_{k}\right)-2\left\lVert\text{}a\text{}\right\rVert_{\ell^{2}({\mathbb{C}})}^{2}\cdot\left(k\cdot\eta_{k}+\sum_{j=k+1}^{\infty}\eta_{j}\right)\right]$
$\displaystyle\geq\sum_{k=n}^{\infty}\left[\frac{|a_{k}|^{2}}{2}\cdot
W_{N}(f_{k})-2\left\lVert\text{}a\text{}\right\rVert_{\ell^{2}({\mathbb{C}})}^{2}\cdot\left((k+1)\cdot\eta_{k}+\sum_{j=k+1}^{\infty}\eta_{j}\right)\right]$
$\displaystyle=\sum_{k=n}^{\infty}\left(\frac{|a_{k}|^{2}}{2}\cdot
W_{N}(f_{k})-2\left\lVert\text{}a\text{}\right\rVert_{\ell^{2}({\mathbb{C}})}^{2}\cdot\varepsilon_{k}\right).$

Finally, let us derive (11) from (10) under the additional assumptions (vi)
and (vii). Choosing $n=N$ in (10) and exploiting the upper bound for
$\varepsilon_{k}$, $k\in\mathbb{N}$, we find that

(14) $\displaystyle
W_{N}(f)\geq\sum_{k=N}^{\infty}\left(\frac{|a_{k}|^{2}}{2}\cdot
W_{N}(f_{k})-2\left\lVert\text{}a\text{}\right\rVert_{\ell^{2}({\mathbb{C}})}^{2}\cdot\varepsilon_{k}\right)\geq\sum_{k=N}^{\infty}\left(\frac{|a_{k}|^{2}}{2}\cdot
W_{N}(f_{k})-\delta\cdot|a_{k}|^{2}\right).$

Since $W_{N}(f)$ is nonincreasing in $N$ for fixed $f\in
L^{2}({\mathbb{R}}^{d})$, we have, for all $k\geq N$,

$W_{N}(f_{k})\geq W_{k}(f_{k})\geq 4\delta.$

Substituting this into (14) concludes the proof. ∎

We are now ready to state and prove a first theorem about arbitrarily slow
energy propagation in scattering networks. This is a generalization of the
idea outlined at the beginning of this section.

######  Theorem 2.2.

Let $F=(f_{m})_{m\in\mathbb{N}_{0}}\subset L^{2}({\mathbb{R}}^{d})$. Suppose
that for all tolerances
$(\eta_{k})_{k\in\mathbb{N}_{0}}\in{\mathbb{R}}_{>0}^{\mathbb{N}_{0}}$ there
exist finite sets $\Psi(F,k)\subseteq\Psi$, $k\in\mathbb{N}_{0}$, and a
subsequence $(f_{m_{k}})_{k\in\mathbb{N}_{0}}$ of $F$ with $m_{0}=0$ and such
that the following hold:

  1. (i)

The expression
$C_{F}:=\sup_{m\in\mathbb{N}_{0}}\left\lVert\text{}f_{m}\text{}\right\rVert_{2}^{2}$
is finite, i.e., $F$ is a bounded sequence.

  2. (ii)

We have, for all $j,k\in\mathbb{N}_{0}$ with $j<k$,

$\displaystyle|\left<\text{}\,f_{m_{k}},f_{m_{j}}\,\text{}\right>|\leq\eta_{k}.$

  3. (iii)

The sets $\Psi(F,k)$, $k\in\mathbb{N}_{0}$, are pairwise disjoint.

  4. (iv)

We have, for every $k\in\mathbb{N}_{0}$,

$\displaystyle\sum_{\psi\in\Psi\setminus\Psi(F,k)}\left\lVert\text{}f_{m_{k}}*\psi\text{}\right\rVert_{2}^{2}\leq\eta_{k}.$

  5. (v)

We have, for every $k\in\mathbb{N}_{0}$ and $0\leq j<k$,

$\displaystyle\sum_{\psi\in\Psi(F,k)}\left\lVert\text{}f_{m_{j}}*\psi\text{}\right\rVert_{2}^{2}\leq\eta_{k}.$

  6. (vi)

There exists $\delta>0$ such that, for all $k\in\mathbb{N}$,
$W_{k}(f_{m_{k}})\geq 4\delta$.

If $E=(E_{N})_{N\in\mathbb{N}}\in{\mathbb{R}}_{>0}^{\mathbb{N}}$ is a
nonincreasing null-sequence, then there exists $f_{E}\in
L^{2}({\mathbb{R}}^{d})$ that satisfies

  1. a)

$\left\lVert\text{}f_{E}\text{}\right\rVert_{2}^{2}\leq 2C_{F}\cdot(1+E_{1})$,

  2. b)

$\left\lVert\text{}f_{E}-f_{0}\text{}\right\rVert_{2}^{2}\leq 2C_{F}\cdot
E_{1}$, and

  3. c)

for all $N\in\mathbb{N}$, $W_{N}(f_{E})\geq\delta\cdot E_{N}$.

###### Proof.

Define

$\displaystyle(a_{k})_{k\in\mathbb{N}}:=\left(\sqrt{E_{k}-E_{k+1}}\right)_{k\in\mathbb{N}},$

and set $a_{0}:=1$. A telescoping argument gives, for every $N\in\mathbb{N}$,

$\displaystyle\sum_{k=N}^{\infty}|a_{k}|^{2}=\sum_{k=N}^{\infty}(E_{k}-E_{k+1})=E_{N}.$

In particular,
$a=(a_{k})_{k\in\mathbb{N}_{0}}\in\ell^{2}(\mathbb{N}_{0};{\mathbb{C}})$ with
$\left\lVert\text{}a\text{}\right\rVert_{\ell^{2}(\mathbb{N}_{0};{\mathbb{C}})}^{2}=1+E_{1}$.

We want to apply Lemma 2.1 to a subsequence $(f_{m_{k}})_{k\in\mathbb{N}_{0}}$
of $F$, which implies that $f_{E}$ can be chosen of the type

$f_{E}:=\sum_{k=0}^{\infty}a_{k}f_{m_{k}}.$

Here, $(f_{m_{k}})_{k\in\mathbb{N}_{0}}$ will be the subsequence of $F$ from
the prerequisites of this theorem that depends on tolerance levels
$(\eta_{k})_{k\in\mathbb{N}_{0}}$, which we will now specify.

Recall from Lemma 2.1 that $f_{E}$ converges in $L^{2}({\mathbb{R}}^{d})$ with
an upper bound for its squared norm given by
$\left\lVert\text{}f_{E}\text{}\right\rVert_{2}^{2}\leq\sum_{k,j=0}^{\infty}|a_{k}a_{j}\cdot\left<\text{}\,f_{m_{k}},f_{m_{j}}\,\text{}\right>|$.
Likewise,

$\left\lVert\text{}f_{E}-f_{0}\text{}\right\rVert_{2}^{2}=\left\lVert\text{}\sum_{k=1}^{\infty}a_{k}f_{m_{k}}\text{}\right\rVert_{2}^{2}\leq\sum_{k,j=1}^{\infty}|a_{k}a_{j}\cdot\left<\text{}\,f_{m_{k}},f_{m_{j}}\,\text{}\right>|.$

We will bound the latter series using the approximate orthogonality relation
(ii). Repeatedly applying the Cauchy-Schwarz inequality yields, for all
$n\in\mathbb{N}_{0}$,

$\displaystyle\sum_{k,j=n}^{\infty}$
$\displaystyle|a_{k}a_{j}\cdot\left<\text{}\,f_{m_{k}},f_{m_{j}}\,\text{}\right>|$
$\displaystyle=\sum_{k=n}^{\infty}|a_{k}|\cdot\left(\sum_{j=n}^{k-1}|a_{j}|\cdot\underbrace{|\left<\text{}\,f_{m_{k}},f_{m_{j}}\,\text{}\right>|}_{\leq\eta_{k}}+|a_{k}|\cdot\underbrace{|\left<\text{}\,f_{m_{k}},f_{m_{k}}\,\text{}\right>|}_{\leq
C_{F}}+\sum_{j=k+1}^{\infty}|a_{j}|\cdot\underbrace{|\left<\text{}\,f_{m_{k}},f_{m_{j}}\,\text{}\right>|}_{\leq\eta_{j}}\right)$
$\displaystyle\leq
C_{F}\cdot\sum_{k=n}^{\infty}|a_{k}|^{2}+\left(\sum_{j=n,j\neq
k}^{\infty}|a_{j}|^{2}\right)^{\frac{1}{2}}\cdot\left(\sum_{k=n}^{\infty}|a_{k}|\cdot\left[(k-n)\cdot\eta_{k}^{2}+\sum_{j=k+1}^{\infty}\eta_{j}^{2}\right]^{\frac{1}{2}}\right)$
$\displaystyle\leq
C_{F}\cdot\sum_{k=n}^{\infty}|a_{k}|^{2}+\left(\sum_{j=n}^{\infty}|a_{j}|^{2}\right)\cdot\left(\sum_{k=n}^{\infty}\left[(k-n)\cdot\eta_{k}^{2}+\sum_{j=k+1}^{\infty}\eta_{j}^{2}\right]\right)^{\frac{1}{2}}.$

Thus, if we choose the tolerances
$(\eta_{k})_{k\in\mathbb{N}_{0}}\in{\mathbb{R}}_{>0}^{\mathbb{N}_{0}}$ small
enough such that

$\sum_{k=0}^{\infty}\left[k\cdot\eta_{k}^{2}+\sum_{j=k+1}^{\infty}\eta_{j}^{2}\right]\leq
C_{F}^{2},$

then we obtain

$\displaystyle\sum_{k,j=n}^{\infty}|a_{k}a_{j}\cdot\left<\text{}\,f_{m_{k}},f_{m_{j}}\,\text{}\right>|\leq
2C_{F}\cdot\left(\sum_{j=n}^{\infty}|a_{j}|^{2}\right)=\begin{cases}2C_{F}\cdot(1+E_{1})&\text{
if }n=0\\\ 2C_{F}\cdot E_{1}&\text{ if }n=1\end{cases}.$

This proves the norm bounds in a) and b).

As in Lemma 2.1, define
$\varepsilon_{k}:=(k+1)\cdot\eta_{k}+\sum_{j=k+1}^{\infty}\eta_{j}$ for all
$k\in\mathbb{N}_{0}$. Since the tolerances
$(\eta_{k})_{k\in\mathbb{N}_{0}}\in{\mathbb{R}}_{>0}^{\mathbb{N}_{0}}$ are
free to choose, we can additionally assume that the series
$\sum_{k=0}^{\infty}\varepsilon_{k}$ converges and that it holds, for all
$k\in\mathbb{N}$,

$\varepsilon_{k}\leq\frac{\delta\cdot|a_{k}|^{2}}{2\left\lVert\text{}a\text{}\right\rVert_{\ell^{2}({\mathbb{C}})}^{2}}.$

By applying Lemma 2.1, we conclude that, for all $N\in\mathbb{N}$,

$W_{N}(f_{E})\geq\delta\cdot\sum_{k=N}^{\infty}|a_{k}|^{2}=\delta\cdot E_{N}.$

∎

Under certain assumptions on the high-pass filters $\Psi$, the latter theorem
guarantees that for any nonincreasing null-sequence
$E=(E_{N})_{N\in\mathbb{N}}\in{\mathbb{R}}_{>0}^{\mathbb{N}}$ there exists at
least one input signal $f_{E}\in L^{2}({\mathbb{R}}^{d})$ whose network energy
propagates slower across the network layers than $E$. Specifically, this
implies that the set

$\left\\{f\in L^{2}({\mathbb{R}}^{d})\leavevmode\nobreak\
\middle|\leavevmode\nobreak\ W_{N}(f)\notin\mathcal{O}(E_{N})\right\\}$

is nonempty. Our next result provides a sufficient criterion under which this
set is even dense in $L^{2}({\mathbb{R}}^{d})$.

######  Proposition 2.3.

Let $E=(E_{N})_{N\in\mathbb{N}}\in{\mathbb{R}}_{>0}^{\mathbb{N}}$ be a
nonincreasing null-sequence. Assume that for all $g\in
L^{2}({\mathbb{R}}^{d})$ there are $C_{g},\delta_{g}>0$ such that the
following holds: For every nonincreasing null-sequence
$E^{\prime}=(E_{N}^{\prime})_{N\in\mathbb{N}}\in{\mathbb{R}}_{>0}^{\mathbb{N}}$
with $\liminf_{N\to\infty}\nicefrac{{E_{N}}}{{E_{N}^{\prime}}}=0$ there exists
$f_{g,E^{\prime}}\in L^{2}({\mathbb{R}}^{d})$ such that

  1. (i)

$\left\lVert\text{}f_{g,E^{\prime}}-g\text{}\right\rVert_{2}^{2}\leq
C_{g}\cdot E_{1}^{\prime}$, and

  2. (ii)

$W_{N}(f_{g,E^{\prime}})\geq\delta_{g}\cdot E_{N}^{\prime}$ for all
$N\in\mathbb{N}$.

Then,

$Y_{E}:=\left\\{f\in L^{2}({\mathbb{R}}^{d})\leavevmode\nobreak\
\middle|\leavevmode\nobreak\ W_{N}(f)\in\mathcal{O}(E_{N})\right\\}$

is a countable union of nowhere dense sets in $L^{2}({\mathbb{R}}^{d})$. In
particular, $L^{2}({\mathbb{R}}^{d})\setminus Y_{E}$ is dense in
$L^{2}({\mathbb{R}}^{d})$.

###### Proof.

Define, for all $p\in\mathbb{N}$,

$M_{p}:=\left\\{f\in L^{2}({\mathbb{R}}^{d})\leavevmode\nobreak\
\middle|\leavevmode\nobreak\ \forall N\in\mathbb{N}:\leavevmode\nobreak\
W_{N}(f)\leq p\cdot E_{N}\right\\},$

and observe that $Y_{E}$ can be written as countable union of these sets,

$Y_{E}=\bigcup_{p\in\mathbb{N}}M_{p}.$

Our goal is to show that $M_{p}$ is nowhere dense in $L^{2}({\mathbb{R}}^{d})$
for every $p\in\mathbb{N}$. To this end, let us first recall from Corollary
1.7 that the nonlinear operator $W_{N}:L^{2}({\mathbb{R}}^{d})\to{\mathbb{R}}$
is continuous. Thus,

$M_{p}=\bigcap_{N\in\mathbb{N}}\left\\{f\in
L^{2}({\mathbb{R}}^{d})\leavevmode\nobreak\ \middle|\leavevmode\nobreak\
W_{N}(f)\leq p\cdot E_{N}\right\\}$

is closed and it only remains to show that the interior of $M_{p}$ is empty:
Fix $g\in M_{p}$, $\varepsilon>0$, and define

$\displaystyle
E_{N}^{\prime}:=\begin{cases}\frac{\varepsilon^{2}}{2C_{g}}&\text{if }N=1\\\
E_{1}^{\prime}\cdot\sqrt{\frac{E_{N}}{E_{1}}}&\text{if }N\geq
2\end{cases},\quad\text{for }N\in\mathbb{N}.$

Clearly,
$E^{\prime}=(E_{N}^{\prime})_{N\in\mathbb{N}}\in{\mathbb{R}}_{>0}^{\mathbb{N}}$
is a nonincreasing null-sequence that satisfies
$\liminf_{N\to\infty}\nicefrac{{E_{N}}}{{E_{N}^{\prime}}}=0$. Hence, there is
$f_{g,E^{\prime}}\in L^{2}({\mathbb{R}}^{d})$ that satisfies (i) and (ii) from
the assumptions of this lemma. Thus, there is $N_{0}\in\mathbb{N}$ such that

$W_{N_{0}}(f_{g,E^{\prime}})\geq\delta_{g}\cdot E_{N_{0}}^{\prime}>p\cdot
E_{N_{0}},$

which means

$f_{g,E^{\prime}}\in B_{\varepsilon}(g)\setminus M_{p},$

where $B_{\varepsilon}(g)$ denotes the open ball with radius $\varepsilon$ and
center $g$ in $L^{2}({\mathbb{R}}^{d})$ with respect to
$\left\lVert\text{}\cdot\text{}\right\rVert_{2}$. Since this entails that the
interior of $M_{p}$ is empty, we have shown in total that $Y_{E}$ is a
countable union of nowhere dense sets in $L^{2}({\mathbb{R}}^{d})$. Therefore,
its complement $L^{2}({\mathbb{R}}^{d})\setminus Y_{E}$ is dense in
$L^{2}({\mathbb{R}}^{d})$ by the Baire category theorem. ∎

For the remainder of this section, we will be examining the assumptions of
Theorem 2.2 and Proposition 2.3. While the number of conditions on the family
$F$ and the high-pass filters $\Psi$ that must be met simultaneously for our
results to apply is considerable, we will demonstrate that these conditions
can in fact be easily satisfied. To provide a concrete reference model, let us
next establish that $F$ can be obtained by $L^{2}$-normalized dilations of a
single function.

######  Lemma 2.4.

Let $f\in L^{2}({\mathbb{R}}^{d})$. If
$(A_{m})_{m\in\mathbb{N}}\in\operatorname{GL}_{d}({\mathbb{R}})^{\mathbb{N}}$
satisfies $\lim_{m\to\infty}\sigma_{\text{min}}(A_{m})=\infty$, then

$\lim_{m\to\infty}\left|\left<\text{}\,f,D_{A_{m}}^{2}f\,\text{}\right>\right|=0.$

###### Proof.

The statement is obvious if $f=0$. For the other case, fix $f\in
L^{2}({\mathbb{R}}^{d})\setminus\\{0\\}$ and $\varepsilon>0$. By Lebesgue’s
dominated convergence theorem there is $R>0$ so that
$f_{R}:=f\cdot\mathds{1}_{B_{R}(0)}$ satisfies

$\left\lVert\text{}f-f_{R}\text{}\right\rVert_{2}\leq\frac{\varepsilon}{2\left\lVert\text{}f\text{}\right\rVert_{2}}.$

For $m\in\mathbb{N}$, splitting the inner product gives

(15)
$\displaystyle\left|\left<\text{}\,f,D_{A_{m}}^{2}f\,\text{}\right>\right|\leq\left|\left<\text{}\,f,D_{A_{m}}^{2}f_{R}\,\text{}\right>\right|+\left|\left<\text{}\,f,D_{A_{m}}^{2}(f-f_{R})\,\text{}\right>\right|$

We obtain an upper bound for the second term by the Cauchy-Schwarz inequality,

(16)
$\displaystyle\left|\left<\text{}\,f,D_{A_{m}}^{2}(f-f_{R})\,\text{}\right>\right|\leq\left\lVert\text{}f\text{}\right\rVert_{2}\cdot\left\lVert\text{}D_{A_{m}}^{2}(f-f_{R})\text{}\right\rVert_{2}=\left\lVert\text{}f\text{}\right\rVert_{2}\cdot\left\lVert\text{}f-f_{R}\text{}\right\rVert_{2}\leq\frac{\varepsilon}{2}.$

We now turn our attention to the first term in (15). As can be seen from a
singular value decomposition of $A_{m}$, we have, for all
$x\in{\mathbb{R}}^{d}$,

$\displaystyle\left\lVert\text{}A_{m}x\text{}\right\rVert_{2}\geq\sigma_{\text{min}}(A_{m})\cdot\left\lVert\text{}x\text{}\right\rVert_{2}.$

Since by assumption $\lim_{m\to\infty}\sigma_{\text{min}}(A_{m})=\infty$, we
find that, for all $x\in{\mathbb{R}}^{d}\setminus\\{0\\}$,

$\displaystyle\mathds{1}_{B_{R}(0)}(A_{m}x)\leq\mathds{1}_{B_{\frac{R}{\sigma_{\text{min}}(A_{m})}}(0)}(x)\xrightarrow{m\to\infty}0.$

Hence, by Lebesgue’s dominated convergence theorem there is
$m_{0}\in\mathbb{N}$ so that, for all $m\geq m_{0}$,

$\displaystyle\left\lVert\text{}f\cdot\mathds{1}_{B_{\frac{R}{\sigma_{\text{min}}(A_{m})}}(0)}\text{}\right\rVert_{2}\leq\frac{\varepsilon}{2\left\lVert\text{}f\text{}\right\rVert_{2}}.$

Thus, for all $m\geq m_{0}$,

$\displaystyle\left|\left<\text{}\,f,D_{A_{m}}^{2}f_{R}\,\text{}\right>\right|$
$\displaystyle\leq\int_{{\mathbb{R}}^{d}}|f(x)|\cdot|D_{A_{m}}^{2}f(x)|\cdot\mathds{1}_{B_{R}(0)}(A_{m}x)\>\mathrm{d}\,x\>$
$\displaystyle\leq\int_{{\mathbb{R}}^{d}}|f(x)|\cdot|D_{A_{m}}^{2}f(x)|\cdot\mathds{1}_{B_{\frac{R}{\sigma_{\text{min}}(A_{m})}}(0)}(x)\>\mathrm{d}\,x\>$
(17)
$\displaystyle\leq\left\lVert\text{}f\cdot\mathds{1}_{B_{\frac{R}{\sigma_{\text{min}}(A_{m})}}(0)}\text{}\right\rVert_{2}\cdot\left\lVert\text{}D_{A_{m}}^{2}f\text{}\right\rVert_{2}\leq\frac{\varepsilon}{2}.$

Plugging (16) and (2) into (15) concludes the proof. ∎

The following lemma shows that $F$ satisfies assumptions (i)–(v) from Theorem
2.2 if it is generated by $L^{2}$-normalized matrix-dilations of a fixed
function for matrices that asymptotically become increasingly expansive.

######  Lemma 2.5.

Let
$(A_{m})_{m\in\mathbb{N}}\in\operatorname{GL}_{d}({\mathbb{R}})^{\mathbb{N}}$
be so that $\lim_{m\to\infty}\sigma_{\text{min}}(A_{m})=\infty$. Let $f\in
L^{2}({\mathbb{R}}^{d})$, and define $f_{m}:=D_{A_{m}}^{2}f$,
$m\in\mathbb{N}$. Finally, choose $f_{0}\in\\{0,f\\}$.

Then, $F=(f_{m})_{m\in\mathbb{N}_{0}}$ satisfies assumptions (i)–(v) from
Theorem 2.2.

###### Proof.

The statement is trivial if $f=0$. Hence, suppose $f\in
L^{2}({\mathbb{R}}^{d})\setminus\\{0\\}$ in the following. Furthermore,
assumptions (i)-(v) of Theorem 2.2 are clearly easier to meet if we are in the
case that $f_{0}=0$ (instead of $f_{0}=f$). Therefore, we will only spell out
the proof for $f_{0}=f$. For notational convenience, we set $A_{0}:=I_{d}$, so
we can write $f_{0}=D_{A_{0}}^{2}f$.

By the normalization of the dilations,
$C_{F}:=\sup_{m\in\mathbb{N}_{0}}\left\lVert\text{}f_{m}\text{}\right\rVert_{2}^{2}=\left\lVert\text{}f\text{}\right\rVert_{2}^{2}$
is finite, i.e., assumption (i) of Theorem 2.2 is satisfied. We now have to
show that for all tolerances
$(\eta_{k})_{k\in\mathbb{N}_{0}}\in{\mathbb{R}}_{>0}^{\mathbb{N}_{0}}$ there
exist finite sets $\Psi(F,k)\subseteq\Psi$, $k\in\mathbb{N}_{0}$, and a
subsequence $(f_{m_{k}})_{k\in\mathbb{N}_{0}}$ of $F$ with $m_{0}=0$ and such
that assumptions (ii)-(v) of Theorem 2.2 hold. We proceed by choosing the sets
$\Psi(F,k)\subseteq\Psi$, $k\in\mathbb{N}_{0}$, and
$({m_{k}})_{k\in\mathbb{N}}\in\mathbb{N}^{\mathbb{N}}$ inductively.

Starting with $k=0$ and $m_{0}=0$, we only need to show that condition (iv)
can be satisfied. Since

$\sum_{\psi\in\Psi}\left\lVert\text{}f_{m_{0}}*\psi\text{}\right\rVert_{2}^{2}\leq\left\lVert\text{}f\text{}\right\rVert_{2}^{2}<\infty,$

there is a finite subset $\Psi(F,0)\subset\Psi$ such that

$\displaystyle\sum_{\psi\in\Psi\setminus\Psi(F,0)}\left\lVert\text{}f_{m_{0}}*\psi\text{}\right\rVert_{2}^{2}\leq\eta_{0}.$

For the induction step, let $k\in\mathbb{N}$ and assume that
$\Psi(F,0),\ldots,\Psi(F,k-1)$ and $m_{0},\ldots,m_{k-1}$ are such that
(ii)-(v) hold. Our goal is now to obtain $\Psi(F,k)$ and $m_{k}$ with the
desired properties.

It is easy to meet condition (ii): For all $m\in\mathbb{N}_{0}$ and $0\leq
j<k$, we have

$\displaystyle|\left<\text{}\,f_{m},f_{m_{j}}\,\text{}\right>|=\left|\left<\text{}\,D_{A_{m_{j}}^{-1}}^{2}D_{A_{m}}^{2}f,f\,\text{}\right>\right|=\left|\left<\text{}\,D_{A_{m}A_{m_{j}}^{-1}}^{2}f,f\,\text{}\right>\right|.$

Moreover,

$\sigma_{\text{min}}\left(A_{m}A_{m_{j}}^{-1}\right)\geq\sigma_{\text{min}}\left(A_{m}\right)\cdot\sigma_{\text{min}}\left(A_{m_{j}}^{-1}\right)=\frac{\sigma_{\text{min}}\left(A_{m}\right)}{\sigma_{\text{max}}\left(A_{m_{j}}\right)}\geq\frac{\sigma_{\text{min}}\left(A_{m}\right)}{\max_{0\leq
l<k}\sigma_{\text{max}}\left(A_{m_{l}}\right)}.$

Since the latter bound is uniformly in $j$ for $0\leq j<k$, and
$\lim_{m\to\infty}\sigma_{\text{min}}\left(A_{m}\right)=\infty$ by assumption,
Lemma 2.4 implies that there exists $m_{k}^{\prime}\in\mathbb{N}_{0}$ such
that $|\left<\text{}\,f_{m_{k}},f_{m_{j}}\,\text{}\right>|<\eta_{k}$ holds for
all $j\in\\{0,\ldots,k-1\\}$ whenever we choose $m_{k}$ larger than
$m_{k}^{\prime}$.

The harder part is to satisfy the remaining conditions (iii)-(v)
simultaneously. We begin with a simple observation about the impact of matrix-
dilations on the frequency content of $f$. For $R>r>0$, let

$P_{r,R}f:=\mathcal{F}^{-1}(\mathcal{F}(f)\cdot\mathds{1}_{S_{r,R}})$

be the projection of $f$ on the subspace of $L^{2}({\mathbb{R}}^{d})$ whose
Fourier transform is supported in $S_{r,R}$. We introduce
$\varepsilon\in(0,\nicefrac{{1}}{{2}})$, which will be chosen later in the
proof (small enough). By Lebesgue’s dominated convergence theorem,
$\widehat{f}$ is concentrated in a spherical shell
$S_{r(\varepsilon),R(\varepsilon)}$ up to $\varepsilon$-error for some
$R(\varepsilon)>r(\varepsilon)>0$, meaning that

$\left\lVert\text{}P_{r(\varepsilon),R(\varepsilon)}f\text{}\right\rVert_{2}^{2}\geq(1-\varepsilon)\cdot\left\lVert\text{}f\text{}\right\rVert_{2}^{2}.$

Now, if $A\in\operatorname{GL}_{d}({\mathbb{R}})$, then $\widehat{D_{A}^{2}f}$
is concentrated in $S_{\sigma_{\text{min}}(A)\cdot
r(\varepsilon),\sigma_{\text{max}}(A)\cdot R(\varepsilon)}$ up to
$\varepsilon$-error, since

$\displaystyle\left\lVert\text{}P_{\sigma_{\text{min}}(A)\cdot
r(\varepsilon),\sigma_{\text{max}}(A)\cdot
R(\varepsilon)}D_{A}^{2}f\text{}\right\rVert_{2}^{2}$
$\displaystyle=\left\lVert\text{}\widehat{D_{A}^{2}f}\cdot\mathds{1}_{S_{\sigma_{\text{min}}(A)\cdot
r(\varepsilon),\sigma_{\text{max}}(A)\cdot
R(\varepsilon)}}\text{}\right\rVert_{2}^{2}$
$\displaystyle=\int_{{\mathbb{R}}^{d}}|\det(A)|^{-1}\cdot|\widehat{f}(A^{-T}\xi)|^{2}\cdot\mathds{1}_{S_{\sigma_{\text{min}}(A)\cdot
r(\varepsilon),\sigma_{\text{max}}(A)\cdot
R(\varepsilon)}}(\xi)\>\mathrm{d}\,\xi\>$
$\displaystyle=\int_{{\mathbb{R}}^{d}}|\widehat{f}(z)|^{2}\cdot\mathds{1}_{S_{\sigma_{\text{min}}(A)\cdot
r(\varepsilon),\sigma_{\text{max}}(A)\cdot
R(\varepsilon)}}(A^{T}z)\>\mathrm{d}\,z\>$
$\displaystyle\geq\int_{{\mathbb{R}}^{d}}|\widehat{f}(z)|^{2}\cdot\mathds{1}_{S_{r(\varepsilon),R(\varepsilon)}}(z)\>\mathrm{d}\,z\>$
$\displaystyle=\left\lVert\text{}\mathcal{F}P_{r(\varepsilon),R(\varepsilon)}f\text{}\right\rVert_{2}^{2}\geq(1-\varepsilon)\cdot\left\lVert\text{}f\text{}\right\rVert_{2}^{2}.$

Specifically, setting

$\displaystyle r_{<k}:=\min_{0\leq j<k}\sigma_{\text{min}}(A_{m_{j}})\cdot
r(\varepsilon)\quad\text{and}\quad R_{<k}:=\max_{0\leq
j<k}\sigma_{\text{max}}(A_{m_{j}})\cdot R(\varepsilon),$

the above frequency concentration estimate entails

$\displaystyle\left\lVert\text{}(\operatorname{id}_{L^{2}({\mathbb{R}}^{d})}-P_{r_{<k},R_{<k}})f_{m_{j}}\text{}\right\rVert_{2}^{2}\leq\varepsilon\cdot\left\lVert\text{}f\text{}\right\rVert_{2}^{2}\quad\text{for
}0\leq j<k.$

By the Littlewood-Paley condition (3) and the continuity of the Fourier
transform of the filters, there is a finite subset
$\widetilde{\Psi}\subseteq\Psi$ containing $\bigcup_{j=0}^{k-1}\Psi(F,j)$, and
such that

(18)
$\displaystyle|\widehat{\chi}(\xi)|^{2}+\sum_{\psi\in\widetilde{\Psi}}|\widehat{\psi}(\xi)|^{2}>1-\varepsilon\quad\text{
for all }\xi\in S_{r_{<k},R_{<k}}.$

By the Riemann-Lebesgue lemma, there exists $T\geq R_{<k}$ such that

(19)
$\displaystyle|\widehat{\chi}(\xi)|^{2}+\sum_{\psi\in\widetilde{\Psi}}|\widehat{\psi}(\xi)|^{2}<\varepsilon\quad\text{
whenever }|\xi|\geq T.$

Since $\lim_{m\to\infty}\sigma_{\text{min}}\left(A_{m}\right)=\infty$ by
assumption, there is $m_{k}^{\prime\prime}\in\mathbb{N}$ with
$\sigma_{\text{min}}(A_{m})\cdot r(\varepsilon)\geq T$ for all $m\geq
m_{k}^{\prime\prime}$. Define
$m_{k}:=\max\\{m_{k}^{\prime},m_{k}^{\prime\prime}\\}$,
$r_{k}:=\sigma_{\text{min}}(A_{m_{k}})\cdot r(\varepsilon)$, and
$R_{k}:=\sigma_{\text{max}}(A_{m_{k}})\cdot R(\varepsilon)$. Thus,

$\displaystyle\left\lVert\text{}(\operatorname{id}_{L^{2}({\mathbb{R}}^{d})}-P_{r_{k},R_{k}})f_{m_{k}}\text{}\right\rVert_{2}^{2}\leq\varepsilon\cdot\left\lVert\text{}f\text{}\right\rVert_{2}^{2}.$

Moreover, because of (19), the Littlewood-Paley condition (3), and the
continuity of the Fourier transform of the filters, there exists a finite
subset $\Psi(F,k,\varepsilon)\subseteq\Psi\setminus\widetilde{\Psi}$ so that

$\displaystyle\sum_{\psi\in\Psi(F,k,\varepsilon)}|\widehat{\psi}(\xi)|^{2}>1-\varepsilon\quad\text{
for all }\xi\in S_{r_{k},R_{k}},$

which itself entails

(20)
$\displaystyle\sum_{\psi\in\Psi\setminus\Psi(F,k,\varepsilon)}|\widehat{\psi}(\xi)|^{2}<\varepsilon\quad\text{
for all }\xi\in S_{r_{k},R_{k}}.$

At the same time, (18) and the fact that
$\Psi(F,k,\varepsilon)\subseteq\Psi\setminus\widetilde{\Psi}$, imply

(21)
$\displaystyle\sum_{\psi\in\Psi(F,k,\varepsilon)}|\widehat{\psi}(\xi)|^{2}<\varepsilon\quad\text{
for all }\xi\in S_{r_{<k},R_{<k}}.$

We claim that $\Psi(F,k):=\Psi(F,k,\varepsilon)$ with
$\varepsilon:=\frac{\eta_{k}}{2\left\lVert\text{}f\text{}\right\rVert_{2}^{2}}$
does the job.

In fact, first note that conditions (i)-(iii) are immediately satisfied by the
above construction. Concerning condition (iv), we have

$\displaystyle\sum_{\psi\in\Psi\setminus\Psi(F,k)}\left\lVert\text{}f_{m_{k}}*\psi\text{}\right\rVert_{2}^{2}$
$\displaystyle=\sum_{\psi\in\Psi\setminus\Psi(F,k)}\left\lVert\text{}(P_{r_{k},R_{k}}f_{m_{k}})*\psi\text{}\right\rVert_{2}^{2}+\sum_{\psi\in\Psi\setminus\Psi(F,k)}\left\lVert\text{}((\operatorname{id}_{L^{2}({\mathbb{R}}^{d})}-P_{r_{k},R_{k}})f_{m_{k}})*\psi\text{}\right\rVert_{2}^{2}$
$\displaystyle\leq\int_{{\mathbb{R}}^{d}}|\widehat{f_{m_{k}}}(\xi)|^{2}\cdot\sum_{\psi\in\Psi\setminus\Psi(F,k)}|\widehat{\psi}(\xi)|^{2}\cdot\mathds{1}_{S_{r_{k},R_{k}}}(\xi)\>\mathrm{d}\,\xi\>+\left\lVert\text{}(\operatorname{id}_{L^{2}({\mathbb{R}}^{d})}-P_{r_{k},R_{k}})f_{m_{k}}\text{}\right\rVert_{2}^{2}$
$\displaystyle\leq\varepsilon\cdot\int_{{\mathbb{R}}^{d}}|\widehat{f_{m_{k}}}(\xi)|^{2}\>\mathrm{d}\,\xi\>+\varepsilon\cdot\left\lVert\text{}f\text{}\right\rVert_{2}^{2}$
$\displaystyle=\varepsilon\cdot\left\lVert\text{}f_{m_{k}}\text{}\right\rVert_{2}^{2}+\varepsilon\cdot\left\lVert\text{}f\text{}\right\rVert_{2}^{2}=\eta_{k}.$

Finally, we obtain condition (v) by proceeding analogously to the previous
step. Indeed, for $0\leq j<k$, we have

$\displaystyle\sum_{\psi\in\Psi(F,k)}\left\lVert\text{}f_{m_{j}}*\psi\text{}\right\rVert_{2}^{2}$
$\displaystyle=\sum_{\psi\in\Psi(F,k)}\left\lVert\text{}(P_{r_{<k},R_{<k}}f_{m_{j}})*\psi\text{}\right\rVert_{2}^{2}+\sum_{\psi\in\Psi(F,k)}\left\lVert\text{}((\operatorname{id}_{L^{2}({\mathbb{R}}^{d})}-P_{r_{<k},R_{<k}})f_{m_{j}})*\psi\text{}\right\rVert_{2}^{2}$
$\displaystyle\leq\int_{{\mathbb{R}}^{d}}|\widehat{f_{m_{j}}}(\xi)|^{2}\cdot\sum_{\psi\in\Psi(F,k)}|\widehat{\psi}(\xi)|^{2}\cdot\mathds{1}_{S_{r_{<k},R_{<k}}}(\xi)\>\mathrm{d}\,\xi\>+\left\lVert\text{}(\operatorname{id}_{L^{2}({\mathbb{R}}^{d})}-P_{r_{<k},R_{<k}})f_{m_{j}}\text{}\right\rVert_{2}^{2}$
$\displaystyle\leq\varepsilon\cdot\int_{{\mathbb{R}}^{d}}|\widehat{f_{m_{j}}}(\xi)|^{2}\>\mathrm{d}\,\xi\>+\varepsilon\cdot\left\lVert\text{}f\text{}\right\rVert_{2}^{2}$
$\displaystyle=\varepsilon\cdot\left\lVert\text{}f_{m_{j}}\text{}\right\rVert_{2}^{2}+\varepsilon\cdot\left\lVert\text{}f\text{}\right\rVert_{2}^{2}=\eta_{k}.$

∎

We summarize our previous results in the case where $F$ is generated by
$L^{2}$-normalized matrix-dilations.

######  Corollary 2.6.

Let
$(A_{m})_{m\in\mathbb{N}}\in\operatorname{GL}_{d}({\mathbb{R}})^{\mathbb{N}}$
be so that $\lim_{m\to\infty}\sigma_{\text{min}}(A_{m})=\infty$. Suppose that

$\delta:=\inf_{\begin{subarray}{c}f\in L^{2}({\mathbb{R}}^{d}),\\\
\left\lVert\text{}f\text{}\right\rVert_{2}=1\end{subarray}}\inf_{k\in\mathbb{N}}\limsup_{m\to\infty}W_{k}(D_{A_{m}}^{2}f)>0.$

Then, for every $g\in L^{2}({\mathbb{R}}^{d})$ there exists a universal
constant $C_{g}>0$ with the following property:

For any nonincreasing null-sequence
$E=(E_{N})_{N\in\mathbb{N}}\in{\mathbb{R}}_{>0}^{\mathbb{N}}$ there is
$f_{g,E}\in L^{2}({\mathbb{R}}^{d})$ such that

  1. a)

$\left\lVert\text{}f_{g,E}\text{}\right\rVert_{2}^{2}\leq
C_{g}\cdot(1+E_{1})$,

  2. b)

$\left\lVert\text{}f_{g,E}-g\text{}\right\rVert_{2}^{2}\leq C_{g}\cdot E_{1}$,
and

  3. c)

$W_{N}(f_{g,E})\geq\frac{\left\lVert\text{}g\text{}\right\rVert_{2}^{2}}{8}\cdot\delta\cdot
E_{N}$ for all $N\in\mathbb{N}$.

Furthermore,

$Y_{E}:=\left\\{f\in L^{2}({\mathbb{R}}^{d})\leavevmode\nobreak\
\middle|\leavevmode\nobreak\ W_{N}(f)\in\mathcal{O}(E_{N})\right\\}$

is a countable union of nowhere dense sets in $L^{2}({\mathbb{R}}^{d})$. In
particular, $L^{2}({\mathbb{R}}^{d})\setminus Y_{E}$ is dense in
$L^{2}({\mathbb{R}}^{d})$.

###### Proof.

Let us first assume that $g\neq 0$. Define $g_{m}:=D_{A_{m}}^{2}g$ for
$m\in\mathbb{N}_{0}$, where $A_{0}:=I_{d}$. From the assumption that
$\delta>0$, we conclude that there exists a strictly increasing sequence
$(m_{k})_{k\in\mathbb{N}_{0}}\subseteq\mathbb{N}_{0}$ with $m_{0}=0$, and such
that
$W_{k}\left(\frac{g_{m_{k}}}{\left\lVert\text{}g\text{}\right\rVert_{2}}\right)\geq\frac{1}{2}\cdot\delta$,
or equivalently
$W_{k}\left(g_{m_{k}}\right)\geq\frac{\left\lVert\text{}g\text{}\right\rVert_{2}^{2}}{2}\cdot\delta$
for all $k\in\mathbb{N}$. The existence of $f_{g,E}$ that satisfies the
properties a)–c) then follows from Theorem 2.2 and Lemma 2.5.

For $g=0$, fix any $h\in L^{2}({\mathbb{R}}^{d})$ with
$\left\lVert\text{}h\text{}\right\rVert_{2}=1$. Set $h_{0}:=0\in
L^{2}({\mathbb{R}}^{d})$ and $h_{m}:=D_{A_{m}}^{2}h$ for $m\in\mathbb{N}$.
Again from the assumption that $\delta>0$, we conclude that there is a
strictly increasing sequence
$(m_{k})_{k\in\mathbb{N}_{0}}\subseteq\mathbb{N}_{0}$ with $m_{0}=0$, and such
that, for all $k\in\mathbb{N}$,
$W_{k}\left(h_{m_{k}}\right)\geq\frac{1}{2}\cdot\delta$. Applying Theorem 2.2
and Lemma 2.5 to the sequence $(h_{m_{k}})_{k\in\mathbb{N}}$ then yields
$f_{g,E}:=f_{h,E}\in L^{2}({\mathbb{R}}^{d})$ that satisfies
$\left\lVert\text{}f_{g,E}\text{}\right\rVert_{2}^{2}\leq 2E_{1}$ and, for all
$N\in\mathbb{N}$, $W_{N}(f_{g,E})\geq\frac{1}{8}\cdot\delta\cdot E_{N}$.

Overall, this also shows that we can apply Proposition 2.3, which completes
the proof. ∎

We are now ready to present the main theorem of this section. It states that
the prerequisites of Corollary 2.6 are satisfied if the high-pass filters
$\Psi$ admit an inclusive structure when $L^{1}$-dilated by the matrix
sequence $(A_{m}^{-1})_{m\in\mathbb{N}}$. Specifically, this includes filter
banks, where the high-pass filters $\Psi$ arise from the $L^{1}$-normalized
dilations by the matrix sequence $(A_{m})_{m\in\mathbb{N}}$ applied to a
finite number of generators, as is the case for wavelet-generated filter
banks. In these cases, both the result of arbitrarily slow energy propagation
and its consequence concerning the density of signals that fail to meet a
given decay rate apply.

######  Theorem 2.7.

Let
$(A_{m})_{m\in\mathbb{N}}\in\operatorname{GL}_{d}({\mathbb{R}})^{\mathbb{N}}$
be such that the following are true:

  1. (i)

We have $\lim_{m\to\infty}\sigma_{\text{min}}(A_{m})=\infty$.

  2. (ii)

The inclusion $\Psi_{m}\subseteq\Psi_{m+1}$ holds for all $m\in\mathbb{N}$,
where $\Psi_{m}:=D_{A_{m}^{-1}}^{1}\Psi$.

Then, $\Psi_{\infty}:=\bigcup_{m\in\mathbb{N}}\Psi_{m}$ is a semi-discrete
Parseval frame. Moreover, for all $f\in L^{2}({\mathbb{R}}^{d})$,
$k\in\mathbb{N}$,

$\displaystyle\lim_{m\to\infty}W_{k}[\Psi]\left(D_{A_{m}}^{2}f\right)=\left\lVert\text{}f\text{}\right\rVert_{2}^{2}.$

In particular, Corollary 2.6 applies with $\delta=1$.

###### Proof.

By the Littlewood-Paley condition (3), we have, for every $m\in\mathbb{N}$,

$\displaystyle|\hat{\chi}(A_{m}^{T}\xi)|^{2}+\sum_{\psi\in\Psi_{m}}|\hat{\psi}(\xi)|^{2}=|\hat{\chi}(A_{m}^{T}\xi)|^{2}+\sum_{\psi\in\Psi}|\hat{\psi}(A_{m}^{T}\xi)|^{2}=1\quad\text{a.e.
}\xi\in{\mathbb{R}}^{d}.$

In particular, for all $f\in L^{2}({\mathbb{R}}^{d})$, $m\in\mathbb{N}$,

$\sum_{\psi\in\Psi_{m}}\left\lVert\text{}f*\psi\text{}\right\rVert_{2}^{2}\leq\left\lVert\text{}f\text{}\right\rVert_{2}^{2},$

which implies

$\sum_{\psi\in\Psi_{\infty}}\left\lVert\text{}f*\psi\text{}\right\rVert_{2}^{2}\leq\left\lVert\text{}f\text{}\right\rVert_{2}^{2}.$

On the other hand, as a consequence of
$\lim_{m\to\infty}\sigma_{\text{min}}(A_{m}^{T})=\infty$, the Riemann-Lebesgue
lemma, and Lebesgue’s dominated convergence theorem, we find that

$\sum_{\psi\in\Psi_{\infty}}\left\lVert\text{}f*\psi\text{}\right\rVert_{2}^{2}\geq\sum_{\psi\in\Psi_{m}}\left\lVert\text{}f*\psi\text{}\right\rVert_{2}^{2}=\left\lVert\text{}f\text{}\right\rVert_{2}^{2}-\left\lVert\text{}f*D_{A_{m}^{-1}}^{1}\chi\text{}\right\rVert_{2}^{2}\xrightarrow{m\to\infty}\left\lVert\text{}f\text{}\right\rVert_{2}^{2}.$

Thus, $\Psi_{\infty}$ is a semi-discrete Parseval frame.

Fix $f\in L^{2}({\mathbb{R}}^{d})$. Recall from Lemma 1.8 that we have, for
all $k\in\mathbb{N}$, $m\in\mathbb{N}$,

$\displaystyle
W_{k}[\Psi](D_{A_{m}}^{2}f)=W_{k}[D_{A_{m}^{-1}}^{1}\Psi](f)=W_{k}[\Psi_{m}](f).$

Let $\varepsilon>0$. Since
$W_{k}[\Psi_{\infty}](f)=\left\lVert\text{}f\text{}\right\rVert_{2}^{2}$ by
Lemma 1.4, there is a finite subset
$\tilde{\Psi}=\tilde{\Psi}(\varepsilon)\subset\Psi_{\infty}$ such that

$W_{k}[\widetilde{\Psi}](f)\geq\left\lVert\text{}f\text{}\right\rVert_{2}^{2}-\varepsilon.$

Since $\widetilde{\Psi}$ is finite, there exists
$M=M(\varepsilon)\in\mathbb{N}$ with $\widetilde{\Psi}\subset\Psi_{M}$. Thus,
for all $m\geq M$,

$\left\lVert\text{}f\text{}\right\rVert_{2}^{2}\geq W_{k}[\Psi_{m}](f)\geq
W_{k}[\Psi_{M}](f)\geq
W_{k}[\widetilde{\Psi}](f)\geq\left\lVert\text{}f\text{}\right\rVert_{2}^{2}-\varepsilon,$

which concludes the second part of the theorem. ∎

In our main application of the latter theorem (concerning scattering with
wavelets, see Section 4.2), the matrices $(A_{m})_{m\in\mathbb{N}}$ are
generated by a single expansive matrix, which immediately implies that
condition (i) from Theorem 2.7 is fulfilled. Given its importance and
simplicity, we conclude this section by explicitly (re-)stating our main
negative findings for this particular instance of our assumptions, which is
much easier to verify than the assumptions of Theorem 2.2.

######  Corollary 2.8.

Let $A\in\operatorname{GL}_{d}({\mathbb{R}})$ be such that
$\sigma_{\min}(A)>1$. If $\Psi_{m}\subseteq\Psi_{m+1}$ holds for all
$m\in\mathbb{N}$, where $\Psi_{m}:=D_{A^{-m}}^{1}\Psi$, then
$\Psi_{\infty}:=\bigcup_{m\in\mathbb{N}}\Psi_{m}$ is a semi-discrete Parseval
frame. Moreover, for all $f\in L^{2}({\mathbb{R}}^{d})$, $k\in\mathbb{N}$,

$\displaystyle\lim_{m\to\infty}W_{k}[\Psi]\left(D_{A^{m}}^{2}f\right)=\left\lVert\text{}f\text{}\right\rVert_{2}^{2}.$

In particular, Corollary 2.6 applies with $\delta=1$.

##  3\. Convergence Rates for Scattering Propagation

In this section, we show that under mild analyticity assumptions on the
scattering filters $\Psi$, fast energy propagation can still be achieved for
large signal classes in $L^{2}({\mathbb{R}}^{d})$. These signal classes depend
on the required speed of energy propagation, i.e., as one would expect, the
faster the energy propagation, the smaller the signal class we can guarantee
this speed for. This is reflected by an interplay between the frequency
localization of the filters in terms of size and shape of their Fourier
supports, and the decay behavior of the Fourier transform of the input signal
relative to the filters.

Notably, our results in this section are very general,

  * •

since they hold in any dimension $d\in\mathbb{N}$;

  * •

because our assumptions can be easily satisfied;

  * •

as the setup provides several parameters that can be used for fine-tuning.

In particular, we will see in Section 4.2 that these results apply to filter
banks generated by bandlimited wavelets, thereby complementing the negative
results from the previous section about arbitrarily slow energy propagation in
such scattering networks.

Our analysis begins with a similar approach to that of [18, Theorem 3.1] and
[20, Theorem 1, Theorem 2], establishing an upper bound on $W_{N}[\Psi](f)$,
for all $f\in L^{2}({\mathbb{R}}^{d})$ and $N\in\mathbb{N}_{\geq 2}$, which is
of the type

(22) $\displaystyle
W_{N}[\Psi](f)\leq\int_{{\mathbb{R}}^{d}}|\widehat{f}(\xi)|^{2}\cdot
K_{N}(\xi)\>\mathrm{d}\,\xi\>$

for a family $(K_{N})_{N\in\mathbb{N}_{\geq 2}}$ of integral kernels
(independent of $f$) that satisfy additional properties. In fact, we will
partially combine the techniques from [18, 20] in the proof of Theorem 3.5,
which allows for more flexibility in our assumptions on the filters. On this
basis, we will then exploit the inductive structure of the scattering network
to involve the filters $\Psi$ in right-hand side of (22).

To clarify our assumptions for this section, we need to introduce some
additional notation. Let $\nu^{*}\in\mathbb{S}^{d-1}$ be the vector with
entries $\nu_{k}^{*}:=\nicefrac{{1}}{{\sqrt{d}}}$ for each
$k\in\\{1,\ldots,d\\}$. For $\rho\in[0,1)$, we denote by

$H^{\rho}:=\left\\{x\in{\mathbb{R}}_{\geq 0}^{d}\leavevmode\nobreak\
\middle|\leavevmode\nobreak\
\left<\text{}\,x,\nu^{*}\,\text{}\right>\geq(1-\rho)\cdot\left\lVert\text{}x\text{}\right\rVert_{2}\right\\}$

the closed cone in the first canonical orthant with its tip at the origin and
opening angle relative to $\nu^{*}$ parameterized by $\rho$. Note that
$H^{\rho}$ is precisely the first canonical orthant ${\mathbb{R}}_{\geq
0}^{d}$ if $\rho\geq 1-\frac{1}{\sqrt{d}}$. In particular, in dimension $d=1$,
we have, for all $\rho\in[0,1)$, $H^{\rho}=[0,\infty)$. The following
assumptions are motivated by [20, Assumption 1].

######  Assumption 3.1.

Let $\kappa\in\mathbb{N}_{\geq 2}$. Let
$(r_{j})_{j\in\mathbb{N}}\in{\mathbb{R}}_{>0}^{\mathbb{N}}$ be a strictly
increasing sequence of scales and suppose that
$\gamma:=\gamma(\kappa):=\inf_{j\in\mathbb{N}}\frac{r_{j}}{r_{j+\kappa}}>0$.
In addition to the Littlewood-Paley condition (3), we assume that there exists
a partition $\Psi=\bigcup_{j\in\mathbb{N}}\Psi_{j}$ such that for every
$\psi\in\Psi_{j}$, $j\in\mathbb{N}$, there is an orthogonal matrix
$A_{\psi}\in O_{d}({\mathbb{R}})$ such that

$\displaystyle\operatorname{supp}(\widehat{\psi})\subseteq
H_{A_{\psi}}^{\rho}\cap\mathcal{S}_{r_{j},r_{j+\kappa}},$

where $H_{A_{\psi}}^{\rho}:=\\{A_{\psi}x\leavevmode\nobreak\
|\leavevmode\nobreak\ x\in H^{\rho}\\}$.

Figure 3. Illustration of the allowed regions for the frequency supports of
two different filters from (possibly) two different families of high-pass
filters. The parameter $\kappa$ (left $\kappa=2$, right $\kappa=3$) reflects
the number of scales the filter may interfere with, and $\rho$ parameterizes
the opening angle of the colored segment.

The present setting may be regarded as a refinement of [20, Assumption 1], in
that it incorporates additional information pertaining to the frequency
localization of the filters, which ultimately affects the decay behavior of
the energy remainder. We think of this as follows: By the Littlewood-Paley
condition (3), the filters $\Psi$ induce a decomposition of the frequency
space ${\mathbb{R}}^{d}$ according to

${\mathbb{R}}^{d}=B_{r_{1}}(0)\cup\bigcup_{j\in\mathbb{N}}\bigcup_{\psi\in\Psi_{j}}(H_{A_{\psi}}^{\rho}\cap\mathcal{S}_{r_{j},r_{j+\kappa}}).$

The filters $\Psi$ are of high-pass nature in the sense that they leave a
frequency gap around zero,

(23) $\displaystyle\sum_{\psi\in\Psi}|\widehat{\psi}(\xi)|^{2}=0\quad\text{
for all }\xi\in B_{r_{1}}(0).$

By the Littlewood-Paley condition (3), this gap is filled by $\widehat{\chi}$.
Moreover, any high-pass filter is frequency-localized in a region that is the
intersection of a spherical shell, and a (rotated) cone with tip at the origin
and angular spread parameterized by $\rho$. The maximum number of scale-
interferences is specified by $\kappa$. The parameter $\gamma$ reflects the
(inverse of the) maximal relative spread between the inner and outer radius of
any such shell. Clearly, we have $\gamma\in(0,1)$, since the scales are
strictly increasing. The closer $\gamma$ is to $1$ and the closer $\rho$ is to
$0$, the more localized the high-pass filters are in frequency space.

These additional parameters will be reflected in the decay rates that we can
guarantee for the energy remainder of certain signals, offering new insight
into the interplay between the localization of the filters (in terms of size
and shape of their Fourier supports) and the frequency decay behavior of the
signals. We note however that our assumptions implicitly require the filters
to be bandlimited, which is slightly more restrictive than the setting
considered in [20, Assumption 1].

We begin our analyis with two auxiliary results about the geometry of the
allowed domain for the Fourier supports of the filters. The first one provides
an upper bound on the maximum distance from $\nu^{*}$ to any other point in
the set $H^{\rho}\cap\mathbb{S}^{d-1}$.

######  Lemma 3.2.

If $t>0$, then $\max_{x\in
H^{\rho}\cap\mathbb{S}^{d-1}}\left\lVert\text{}x-t\nu^{*}\text{}\right\rVert_{2}^{2}\leq
t^{2}-2t\cdot(1-\rho)+1.$

###### Proof.

A direct computation shows that, for all $x\in H^{\rho}\cap\mathbb{S}^{d-1}$,

$\displaystyle\left\lVert\text{}x-t\nu^{*}\text{}\right\rVert_{2}^{2}=\left\lVert\text{}x\text{}\right\rVert_{2}^{2}-2t\cdot\left<\text{}\,x,\nu^{*}\,\text{}\right>+t^{2}\cdot\left\lVert\text{}\nu^{*}\text{}\right\rVert_{2}^{2}\leq
t^{2}-2t\cdot(1-\rho)\cdot\left\lVert\text{}x\text{}\right\rVert_{2}+1=t^{2}-2t\cdot(1-\rho)+1.$

∎

The next lemma will be crucial for the induction step in the proof of Theorem
3.5.

######  Lemma 3.3.

There is a family $(\nu_{\psi})_{\psi\in\Psi}\subseteq{\mathbb{R}}^{d}$ so
that, for all $\psi\in\Psi$, and all
$\xi\in\operatorname{supp}(\widehat{\psi})$,

$\displaystyle\left\lVert\text{}\xi-\nu_{\psi}\text{}\right\rVert_{2}\leq\alpha\cdot\left\lVert\text{}\xi\text{}\right\rVert_{2},$

where
$\alpha=\alpha(\gamma,\rho):=\sqrt{1-\frac{4\gamma}{(1+\gamma)^{2}}\cdot(1-\rho)^{2}}\in(0,1)$.

###### Proof.

Since the function $(0,1)\to{\mathbb{R}},\
\gamma\mapsto\frac{4\gamma}{(1+\gamma)^{2}}$ is strictly increasing, we have

$0<\frac{4\gamma}{(1+\gamma)^{2}}\cdot(1-\rho)^{2}\leq\frac{4\gamma}{(1+\gamma)^{2}}<\lim_{\gamma\uparrow
1}\frac{4\gamma}{(1+\gamma)^{2}}=1.$

Thus, $\alpha\in(0,1)$. Now let, for every $\psi\in\Psi_{j}$,
$j\in\mathbb{N}$,

$\nu_{\psi}:=\frac{2r_{j}}{1+\gamma}\cdot(1-\rho)\cdot
A_{\psi}\nu^{*}\in{\mathbb{R}}^{d}.$

By Assumption 3.1, any $\xi\in\operatorname{supp}(\widehat{\psi})$ can be
written as $A_{\psi}\omega=\xi$ for some $\omega\in
H^{\rho}\cap\mathcal{S}_{r_{j},r_{j+\kappa}}$. Since $A_{\psi}$ is an
orthogonal matrix, it preserves the euclidean distances
$\left\lVert\text{}\xi\text{}\right\rVert_{2}=\left\lVert\text{}\omega\text{}\right\rVert_{2}$
and

$\|\xi-\nu_{\psi}\|_{2}=\left\lVert\text{}\omega-\frac{2r_{j}\cdot(1-\rho)}{1+\gamma}\cdot\nu^{*}\text{}\right\rVert_{2}.$

It therefore suffices to show that the latter term is bounded by
$\alpha\cdot\left\lVert\text{}\omega\text{}\right\rVert_{2}$. By Lemma 3.2,

$\displaystyle\frac{\left\lVert\text{}\omega-\frac{2r_{j}\cdot(1-\rho)}{(1+\gamma)}\cdot\nu^{*}\text{}\right\rVert_{2}}{\left\lVert\text{}\omega\text{}\right\rVert_{2}}$
$\displaystyle\leq\max_{s\in[r_{j},r_{j+\kappa}]}\left\lVert\text{}\frac{\omega}{\left\lVert\text{}\omega\text{}\right\rVert_{2}}-\frac{2r_{j}\cdot(1-\rho)}{s\cdot(1+\gamma)}\cdot\nu^{*}\text{}\right\rVert_{2}$
$\displaystyle\leq\max_{s\in[r_{j},r_{j+\kappa}]}\max_{x\in
H^{\rho}\cap\mathbb{S}^{d-1}}\Bigg{\|}x-\underbrace{\frac{2r_{j}\cdot(1-\rho)}{s\cdot(1+\gamma)}}_{=:t(s)}\cdot\nu^{*}\Bigg{\|}_{2}$
$\displaystyle\leq\max_{s\in[r_{j},r_{j+\kappa}]}\sqrt{t(s)^{2}-2t(s)\cdot(1-\rho)+1}$
$\displaystyle\leq\max_{s\in[r_{j},\gamma^{-1}r_{j}]}\sqrt{t(s)^{2}-2t(s)\cdot(1-\rho)+1}$
(24)
$\displaystyle=\max_{t\in\left[\frac{2\gamma\cdot(1-\rho)}{1+\gamma},\frac{2(1-\rho)}{1+\gamma}\right]}\sqrt{t^{2}-2t\cdot(1-\rho)+1}.$

Since the polynomial

$P:{\mathbb{R}}\to{\mathbb{R}},\quad t\mapsto t^{2}-2t\cdot(1-\rho)+1$

is convex, the maximum in (24) is attained at one of the boundary points of
the interval. A direct computation yields

$\displaystyle\max_{r\in\left[\frac{2\gamma\cdot(1-\rho)}{1+\gamma},\frac{2(1-\rho)}{1+\gamma}\right]}\sqrt{r^{2}-2r\cdot(1-\rho)+1}=\sqrt{1-\frac{4\gamma}{(1+\gamma)^{2}}\cdot(1-\rho)^{2}}.$

∎

The following lemma is due to Mallat (cf. [11, Lemma 2.7]) and will turn out
to be essential for proving the main result of this section. For the sake of
completeness, we reproduce its (elementary) proof.

######  Lemma 3.4.

Let $f\in L^{2}({\mathbb{R}}^{d})$, $g\in L^{1}({\mathbb{R}}^{d})$, and
$\nu\in{\mathbb{R}}^{d}$. If $g\geq 0$, then

$\int_{{\mathbb{R}}^{d}}\left|\text{}\mathcal{F}(|f|)(\xi)\text{}\right|^{2}\cdot\left(1-\left|\text{}\widehat{g}(\xi)\text{}\right|^{2}\right)\>\mathrm{d}\,\xi\>\leq\int_{{\mathbb{R}}^{d}}\left|\text{}\widehat{f}(\xi)\text{}\right|^{2}\cdot\left(1-\left|\text{}\widehat{g}(\xi-\nu)\text{}\right|^{2}\right)\>\mathrm{d}\,\xi\>.$

###### Proof.

Clearly, since $g\geq 0$, we have

$\displaystyle\left\lVert\text{}|f|*g\text{}\right\rVert_{2}^{2}=\left\lVert\text{}|f|*|M_{\nu}g|\text{}\right\rVert_{2}^{2}\geq\left\lVert\text{}f*M_{\nu}g\text{}\right\rVert_{2}^{2}.$

By the convolution theorem and Parseval’s theorem, we obtain

$\displaystyle\int_{{\mathbb{R}}^{d}}\left|\text{}\mathcal{F}(|f|)(\xi)\text{}\right|^{2}\cdot\left(1-\left|\text{}\widehat{g}(\xi)\text{}\right|^{2}\right)\>\mathrm{d}\,\xi\>$
$\displaystyle=\left\lVert\text{}|f|\text{}\right\rVert_{2}^{2}-\left\lVert\text{}|f|*g\text{}\right\rVert_{2}^{2}$
$\displaystyle\leq\left\lVert\text{}f\text{}\right\rVert_{2}^{2}-\left\lVert\text{}f*M_{\nu}g\text{}\right\rVert_{2}^{2}=\int_{{\mathbb{R}}^{d}}\left|\text{}\widehat{f}(\xi)\text{}\right|^{2}\cdot\left(1-\left|\text{}\widehat{g}(\xi-\nu)\text{}\right|^{2}\right)\>\mathrm{d}\,\xi\>.$

∎

We are now ready to establish an upper bound on the energy remainder that is
of the type (22). Previous work in this direction [18, 20, 21] showed that, in
dimension $d=1$, for certain wavelet-generated filter banks
$\Psi_{\text{wav}}$, there exist constants $C=C(\Psi_{\text{wav}})>0$ and
$\alpha=\alpha(\Psi_{\text{wav}})\in(0,1)$ such that, for all (real-valued,
cf. [18]) signals $f\in L^{2}({\mathbb{R}})$, and for all
$N\in\mathbb{N}_{\geq 2}$,

$W_{N}[\Psi_{\text{wav}}](f)\leq\int_{{\mathbb{R}}}|\widehat{f}(\xi)|^{2}\cdot\left(1-\left|\widehat{\vartheta}\left(C\cdot\alpha^{N-1}\cdot\xi\right)\right|^{2}\right)\>\mathrm{d}\,\xi\>.$

Here, $\widehat{\vartheta}$ denotes a concretely specified positive definite
and even function that ensures asymptotic decay of the right-hand side (a
Gaussian in [18], and a truncated power function in [20]). Our result is an
extension of this previous work, since it applies to the fairly large class of
filter banks satisfying Assumption 3.1, including filter banks that are not
necessarily structured as in the case of wavelet-generated filter banks. At
the same time, our result holds in arbitrary dimension $d\in\mathbb{N}$.
Moreover, for filters that satisfy our Assumption 3.1, which is slightly more
restrictive than [20, Assumption 1], we improve the bound given in [20,
Theorem 1] from polynomial decay of order $m_{d}$ (where $m_{d}\in(0,1]$ and
$m_{d}\to 0$ as $d\to\infty$) in the argument of $\widehat{\vartheta}$ to
exponential decay.

######  Theorem 3.5.

Let $\vartheta\in L^{1}({\mathbb{R}}^{d})$, $\vartheta\geq 0$, and suppose
that there exists a nonincreasing function $\eta:[0,\infty)\to[0,1]$ so that
$\eta(0)=1$ and
$|\widehat{\vartheta}|=\eta(\left\lVert\text{}\cdot\text{}\right\rVert_{2})$.
Then, there exists a universal constant $C_{\chi,\vartheta}>0$ such that, for
all $f\in L^{2}({\mathbb{R}}^{d})$, and all $N\in\mathbb{N}_{\geq 2}$,

(25) $\displaystyle
W_{N}(f)\leq\int_{{\mathbb{R}}^{d}}|\widehat{f}(\xi)|^{2}\cdot\left(1-\left|\widehat{\vartheta}\left(C_{\chi,\vartheta}\cdot\alpha^{N-1}\cdot\xi\right)\right|^{2}\right)\>\mathrm{d}\,\xi\>,$

where
$\alpha=\alpha(\gamma,\rho):=\sqrt{1-\frac{4\gamma}{(1+\gamma)^{2}}\cdot(1-\rho)^{2}}$.

Moreover, if there is $\widetilde{C}>0$ so that
$|\widehat{\vartheta}(\widetilde{C}\,\cdot)|\leq|\widehat{\chi}|$, then (25)
holds for all $N\in\mathbb{N}$ with $C_{\chi,\vartheta}=\widetilde{C}$.

###### Proof.

We perform induction on $N$.

First, let us assume that there is $\widetilde{C}>0$ so that
$|\widehat{\vartheta}(\widetilde{C}\;\cdot)|\leq|\widehat{\chi}|$. In this
scenario, it is easy to prove the base case, which works analogously to the
proof of the base case in [20, Theorem 1]. In fact, by the Littlewood-Paley
condition (3), for all $f\in L^{2}({\mathbb{R}}^{d})$,

$\displaystyle
W_{1}(f)=\sum_{\psi\in\Psi}\left\lVert\text{}f*\psi\text{}\right\rVert_{2}^{2}$
$\displaystyle=\int_{{\mathbb{R}}^{d}}\sum_{\psi\in\Psi}|\widehat{f}(\xi)|^{2}\cdot|\widehat{\psi}(\xi)|^{2}\>\mathrm{d}\,\xi\>$
$\displaystyle=\int_{{\mathbb{R}}^{d}}|\widehat{f}(\xi)|^{2}\cdot(1-|\widehat{\chi}(\xi)|^{2})\>\mathrm{d}\,\xi\>\leq\int_{{\mathbb{R}}^{d}}|\widehat{f}(\xi)|^{2}\cdot(1-|\widehat{\vartheta}(\widetilde{C}\cdot\xi)|^{2})\>\mathrm{d}\,\xi\>.$

The proof of the base case is harder if there is no $\widetilde{C}>0$ such
that $|\widehat{\vartheta}(\widetilde{C}\,\cdot)|\leq|\widehat{\chi}|$ holds
true. Here we proceed in three steps, motivated by [18, Section 6.3]. In the
first two steps we construct certain auxiliary functions, whose properties
will be exploited to eventually prove the base case in the last step.

Step 1: Let $r_{1}>0$ be as in Assumption 3.1, corresponding to the radius of
the frequency gap of the high-pass filters $\Psi$. Choose an even function
$h\in C^{\infty}({\mathbb{R}}^{d})$ that is supported in
$\overline{B_{\frac{r_{1}}{2}}(0)}$ and that satisfies $h(\xi)>0$ for all
$\xi\in B_{\frac{r_{1}}{2}}(0)$. Set $g:=\mathcal{F}(h*h)$. Then, $g\in
S({\mathbb{R}}^{d})$, $g=|\widehat{h}|^{2}\geq 0$,
$\operatorname{supp}(\widehat{g})\subseteq\overline{B_{r_{1}}(0)}$, and
$\widehat{g}(\xi)>0$ for all $\xi\in\overline{B_{\frac{r_{1}}{2}}(0)}$. In
particular,

$m_{g}:=\min_{\xi\in\overline{B_{\frac{r_{1}}{2}}(0)}}|\widehat{g}(\xi)|^{2}>0.$

Finally, we may assume that $\|\widehat{g}\|_{\infty}\leq 1$ by imposing
$\left\lVert\text{}h\text{}\right\rVert_{2}\leq 1$ if necessary.

Step 2: Let $\tilde{c}:[0,\infty)\to(0,\infty)$ be nonincreasing and such that

$\int_{{\mathbb{R}}^{d}}\tilde{c}(\left\lVert\text{}\nu\text{}\right\rVert_{2})\>\mathrm{d}\,\nu\>=1.$

Defining the functions $c:{\mathbb{R}}^{d}\to[0,\infty),\
\nu\mapsto\tilde{c}(\left\lVert\text{}\nu\text{}\right\rVert_{2})$ and
$F:=|\widehat{g}|^{2}*c$, we find that, for all $\xi\in{\mathbb{R}}^{d}$,

$\displaystyle F(\xi)=\int_{{\mathbb{R}}^{d}}|\widehat{g}(\nu)|^{2}\cdot
c(\xi-\nu)\>\mathrm{d}\,\nu\>\geq
m_{g}\int_{B_{\frac{r_{1}}{2}}(0)}c(\xi-\nu)\>\mathrm{d}\,\nu\>\geq
m_{g}\cdot\operatorname{vol}\left(B_{\frac{r_{1}}{2}}(0)\right)\cdot\tilde{c}\left(\left\lVert\text{}\xi\text{}\right\rVert_{2}+\frac{r_{1}}{2}\right).$

Thus, if we choose
$c=\tilde{c}(\left\lVert\text{}\cdot\text{}\right\rVert_{2})$ so that

$\lim_{s\to\infty}\frac{\eta^{2}(s)}{\tilde{c}\left(s+\frac{r_{1}}{2}\right)}=0,$

then there is $M>0$ such that, for all $\xi\in{\mathbb{R}}^{d}\setminus
B_{M}(0)$, it holds $F(\xi)\geq|\widehat{\vartheta}(\xi)|^{2}$. Furthermore,
$F$ is strictly positive and continuous. Hence,

$m_{F}:=\min_{\xi\in\overline{B_{M}(0)}}F(\xi)>0.$

Since $\lim_{s\to\infty}\eta(s)=0$, there exists
$C_{\chi,\vartheta}>\alpha^{-1}$ such that, for all
$\xi\in{\mathbb{R}}^{d}\setminus B_{r_{1}}(0)$,

$m_{F}\geq|\widehat{\vartheta}(C_{\chi,\vartheta}\cdot\alpha\cdot\xi)|^{2}.$

Finally, we deduce $\left\lVert\text{}F\text{}\right\rVert_{\infty}\leq 1$
from $\|\widehat{g}\|_{\infty}\leq 1$ and
$\left\lVert\text{}c\text{}\right\rVert_{1}=1$. Altogether, for all
$\xi\in{\mathbb{R}}^{d}\setminus B_{r_{1}}(0)$,

(26) $\displaystyle 0\leq 1-F(\xi)\leq
1-|\widehat{\vartheta}(C_{\chi,\vartheta}\cdot\alpha\cdot\xi)|^{2}.$

Step 3: Recall from (23) that $\sum_{\psi\in\Psi}|\widehat{\psi}|^{2}$
vanishes on $B_{r_{1}}(0)$. Since $\widehat{g}$ is supported in
$\overline{B_{r_{1}}(0)}$ and since $\|\widehat{g}\|_{\infty}\leq 1$, we have

$\sum_{\psi\in\Psi}|\widehat{\psi}(\xi)|^{2}\leq
1-|\widehat{g}(\xi)|^{2}\quad\text{ a.e. }\xi\in{\mathbb{R}}^{d}.$

Hence, for all $f\in L^{2}({\mathbb{R}}^{d})$,

$\displaystyle
W_{2}(f)=\sum_{\psi\in\Psi}\sum_{\psi^{\prime}\in\Psi}\left\lVert\text{}||f*\psi|*\psi^{\prime}|\text{}\right\rVert_{2}^{2}$
$\displaystyle=\sum_{\psi\in\Psi}\sum_{\psi^{\prime}\in\Psi}\left\lVert\text{}|f*\psi|*\psi^{\prime}\text{}\right\rVert_{2}^{2}$
$\displaystyle\leq\sum_{\psi\in\Psi}\int_{{\mathbb{R}}^{d}}\left|\text{}\mathcal{F}(|f*\psi|)(\xi)\text{}\right|^{2}\cdot\left(1-|\widehat{g}(\xi)|^{2}\right)\>\mathrm{d}\,\xi\>.$

Using (23) and Lemma 3.4, we conclude that, for all $\nu\in{\mathbb{R}}^{d}$,

$\displaystyle W_{2}(f)$
$\displaystyle\leq\sum_{\psi\in\Psi}\int_{{\mathbb{R}}^{d}}|\widehat{f}(\xi)|^{2}\cdot|\widehat{\psi}(\xi)|^{2}\cdot\left(1-|\widehat{g}(\xi-\nu)|^{2}\right)\>\mathrm{d}\,\xi\>$
$\displaystyle\leq\int_{{\mathbb{R}}^{d}\setminus
B_{r_{1}}(0)}|\widehat{f}(\xi)|^{2}\cdot\left(1-|\widehat{g}(\xi-\nu)|^{2}\right)\>\mathrm{d}\,\xi\>.$

Finally, we average this inequality with the function $c$ and insert the upper
bound from (26) to obtain

$\displaystyle W_{2}(f)$
$\displaystyle\leq\int_{{\mathbb{R}}^{d}}c(\nu)\cdot\int_{{\mathbb{R}}^{d}\setminus
B_{r_{1}}(0)}|\widehat{f}(\xi)|^{2}\cdot\left(1-|\widehat{g}(\xi-\nu)|^{2}\right)\>\mathrm{d}\,\xi\>\>\mathrm{d}\,\nu\>$
$\displaystyle=\int_{{\mathbb{R}}^{d}\setminus
B_{r_{1}}(0)}|\widehat{f}(\xi)|^{2}\cdot\int_{{\mathbb{R}}^{d}}c(\nu)-c(\nu)\cdot|\widehat{g}(\xi-\nu)|^{2}\>\mathrm{d}\,\nu\>\>\mathrm{d}\,\xi\>$
$\displaystyle=\int_{{\mathbb{R}}^{d}\setminus
B_{r_{1}}(0)}|\widehat{f}(\xi)|^{2}\cdot\left(1-F(\xi)\right)\>\mathrm{d}\,\xi\>$
$\displaystyle\leq\int_{{\mathbb{R}}^{d}}|\widehat{f}(\xi)|^{2}\cdot\left(1-|\vartheta(C_{\chi,\vartheta}\cdot\alpha\cdot\xi)|^{2}\right)\>\mathrm{d}\,\xi\>.$

This concludes the proof of the base case.

The induction step goes along the lines of [20, Theorem 1]: Suppose that (25)
holds for some $N\in\mathbb{N}$ for all $f\in L^{2}({\mathbb{R}}^{d})$. Our
goal is then to establish the analogous upper bound for $W_{N+1}(f)$. For
$\psi\in\Psi$, let $\nu_{\psi}\in{\mathbb{R}}^{d}$ be as in Lemma 3.3. Note
that the inverse Fourier transform of
$\widehat{\vartheta}\left(C_{\chi,\vartheta}\cdot\alpha^{N-1}\,\cdot\right)$
is nonnegative by the non-negativity of $\vartheta$. Thus, applying the
induction hypothesis to $|f*\psi|$, as well as employing Lemma 3.4, we find
that

$\displaystyle W_{N}(|f*\psi|)$
$\displaystyle\leq\int_{{\mathbb{R}}^{d}}|\mathcal{F}(|f*\psi|)(\xi)|^{2}\cdot\left(1-\left|\widehat{\vartheta}\left(C_{\chi,\vartheta}\cdot\alpha^{N-1}\cdot\xi\right)\right|^{2}\right)\>\mathrm{d}\,\xi\>$
$\displaystyle\leq\int_{{\mathbb{R}}^{d}}|\widehat{f}(\xi)|^{2}\cdot|\widehat{\psi}(\xi)|^{2}\cdot\left(1-\left|\widehat{\vartheta}\left(C_{\chi,\vartheta}\cdot\alpha^{N-1}\cdot(\xi-\nu_{\psi})\right)\right|^{2}\right)\>\mathrm{d}\,\xi\>.$

Summing over $\psi\in\Psi$ on both sides of the above inequality yields

(27) $\displaystyle
W_{N+1}(f)=\sum_{\psi\in\Psi}W_{N}(|f*\psi|)\leq\int_{{\mathbb{R}}^{d}}|\widehat{f}(\xi)|^{2}\cdot
h_{N-1}(\xi)\>\mathrm{d}\,\xi\>,$

where

$\displaystyle
h_{N-1}(\xi):=\sum_{\psi\in\Psi}|\widehat{\psi}(\xi)|^{2}\cdot\left(1-\left|\widehat{\vartheta}\left(C_{\chi,\vartheta}\cdot\alpha^{N-1}\cdot(\xi-\nu_{\psi})\right)\right|^{2}\right).$

Now, Lemma 3.3 comes into play to establish a pointwise upper bound for
$h_{N-1}$ that suffices to conclude the proof. Since
$\sum_{\psi\in\Psi}|\widehat{\psi}(\xi)|^{2}\leq 1$ holds for all
$\xi\in{\mathbb{R}}^{d}$, it is enough to show that we have, for all
$\psi\in\Psi$, and all $\xi\in\operatorname{supp}(\widehat{\psi})$,

$\displaystyle
1-\left|\widehat{\vartheta}\left(C_{\chi,\vartheta}\cdot\alpha^{N-1}\cdot(\xi-\nu_{\psi})\right)\right|^{2}\leq
1-\left|\widehat{\vartheta}\left(C_{\chi,\vartheta}\cdot\alpha^{N}\cdot\xi\right)\right|^{2}.$

In fact,
$1-|\widehat{\vartheta}|^{2}=1-\eta^{2}(\left\lVert\text{}\cdot\text{}\right\rVert_{2})$,
where $1-\eta^{2}$ is nondecreasing, and Lemma 3.3 implies that

$\left\lVert\text{}C_{\chi,\vartheta}\cdot\alpha^{N-1}\cdot(\xi-\nu_{\psi})\text{}\right\rVert_{2}\leq\left\lVert\text{}C_{\chi,\vartheta}\cdot\alpha^{N}\cdot\xi\text{}\right\rVert_{2}.$

Consequently, we have, for all $\xi\in{\mathbb{R}}^{d}$,

$h_{N-1}(\xi)\leq
1-\left|\widehat{\vartheta}\left(C_{\chi,\vartheta}\cdot\alpha^{N}\cdot\xi\right)\right|^{2}.$

Inserting this into (27) concludes the induction step and thus the proof of
the theorem. ∎

######  Remark 3.6.

Smaller values of $\alpha$ lead to faster convergence to $0$ of the integral
on the right side of (25) as $N\to\infty$. Hence, let us briefly comment on
the impact of the parameters on the value of $\alpha$. To this end, we note
that the function

$\alpha:(0,1)\times[0,1)\to(0,1),\quad(\gamma,\rho)\mapsto\sqrt{1-\frac{4\gamma}{(1+\gamma)^{2}}\cdot(1-\rho)^{2}}$

is strictly decreasing in $\gamma$, while it is strictly increasing in $\rho$.
Moreover, we have the asymptotic behavior

$\lim_{(\gamma,\rho)\to(1,0)}\alpha(\gamma,\rho)=0.$

We interpret this as follows: $\alpha$ is small if the size of the Fourier
supports of the filters is approximately constant across all scales and if the
filters are well localized in terms of their maximum angular frequency spread.
This result appears to be consistent with the earlier findings stated in [20,
Theorem 1], which indicate a faster decay in lower dimensions: A reduction of
the underlying dimension implicitly forces a higher concentration of angular
frequency, with a maximum concentration achieved in dimension $d=1$.

Figure 4. Plot of $\alpha$ as a function of the parameters $\gamma$ and
$\rho$. Smaller values of $\alpha$ guarantee faster decay.

Next, we show how Theorem 3.5 entails fast (up to exponential) energy decay
for signals belonging to generalized Sobolev spaces that are tailored to the
frequency localization of the high-pass filters $\Psi$. We introduce certain
weights to define these spaces.

######  Definition 3.7.

Let $k>0$. We say that a function $\omega:(0,\infty)\to(0,\infty)$ is a weakly
$t^{k}$-dominated weight if $\omega$ is nondecreasing and if there exists
$T>0$ so that the auxiliary function

$h_{k,\omega}:(0,\infty)\to(0,\infty),\ t\mapsto t^{k}\cdot\omega^{-2}(t)$

is bounded on $(0,T)$, nondecreasing on $[T,\infty)$, and satisfies
$\lim_{t\to\infty}h_{k,\omega}(t)=\infty$.

If, in addition to the above, $h_{k,\omega}$ is nondecreasing on the entire
interval $(0,\infty)$, then we say that $\omega$ is a strongly
$t^{k}$-dominated weight.

######  Remark 3.8.

If $\omega$ is differentiable, we can easily establish a sufficient criterion
for the (simultaneous) monotonicity of $\omega$ and $h_{k,\omega}$, relying on
the non-negativity of their derivatives. In fact, for all $t>0$,

$\displaystyle h_{k,\omega}^{\prime}(t)=k\cdot
t^{k-1}\cdot\omega^{-2}(t)-2t^{k}\cdot\omega^{-3}(t)\cdot\omega^{\prime}(t)=\omega^{-2}(t)\cdot
t^{k-1}\cdot\left(k-2t\cdot\omega^{-1}(t)\cdot\omega^{\prime}(t)\right).$

Thus, $h_{k,\omega}^{\prime}(t)\geq 0$ is equivalent to $k\cdot\omega(t)\geq
2t\cdot\omega^{\prime}(t)$. Altogether, if there is $T\geq 0$ so that for all
$t>T$,

$k\cdot\omega(t)\geq 2t\cdot\omega^{\prime}(t)\geq 0,$

then both $\omega$ and $h_{k,\omega}$ are nondecreasing on $(T,\infty)$.

We define, for all $\psi\in\Psi$,

$d_{\psi}:=\inf_{\xi^{\prime}\in{\mathbb{R}}^{d}}\sup_{\xi\in\operatorname{supp}(\widehat{\psi})}\left\lVert\text{}\xi-\xi^{\prime}\text{}\right\rVert_{2}=\min_{\xi^{\prime}\in{\mathbb{R}}^{d}}\max_{\xi\in\operatorname{supp}(\widehat{\psi})}\left\lVert\text{}\xi-\xi^{\prime}\text{}\right\rVert_{2}.$

For a weakly $t^{k}$-dominated weight, we consider the generalized Sobolev
space

$\mathcal{D}_{\omega}(\Psi;L^{2}({\mathbb{R}}^{d})):=\left\\{\text{}f\in
L^{2}({\mathbb{R}}^{d})\leavevmode\nobreak\ \middle|\leavevmode\nobreak\
\sum_{\psi\in\Psi}\omega^{2}(d_{\psi})\cdot\left\lVert\text{}f*\psi\text{}\right\rVert_{2}^{2}<\infty\text{}\right\\}.$

######  Theorem 3.9.

Let $\vartheta\in L^{1}({\mathbb{R}}^{d})$ be as in Theorem 3.5, and assume
that there are $k=k_{\vartheta}>0$, $C=C_{\vartheta}>0$ such that, for all
$\xi\in{\mathbb{R}}^{d}$,

(28) $\displaystyle 1-|\widehat{\vartheta}(\xi)|^{2}\leq
C\cdot\left\lVert\text{}\xi\text{}\right\rVert_{2}^{k}.$

If $\omega:(0,\infty)\to(0,\infty)$ is a weakly $t^{k}$-dominated weight, then
we have, for all $f\in\mathcal{D}_{\omega}(\Psi;L^{2}({\mathbb{R}}^{d}))$,

$W_{N}(f)\in\mathcal{O}\left(\omega^{-2}\left(\alpha^{-N}\right)\right),$

where
$\alpha=\alpha(\gamma,\rho):=\sqrt{1-\frac{4\gamma}{(1+\gamma)^{2}}\cdot(1-\rho)^{2}}\in(0,1)$.

Moreover, if there is $\widetilde{C}>0$ so that
$|\widehat{\vartheta}(\widetilde{C}\,\cdot)|\leq|\widehat{\chi}|$, and if
$\omega$ is a strongly $t^{k}$-dominated weight, then the following explicit
upper bound holds: For all $N\in\mathbb{N}_{\geq 2}$,

$\displaystyle
W_{N}(f)\leq\max\left\\{1,C\cdot\widetilde{C}^{k}\cdot\alpha^{-k}\right\\}\cdot\left(\sum_{\psi\in\Psi}\omega^{2}(d_{\psi})\cdot\left\lVert\text{}f*\psi\text{}\right\rVert_{2}^{2}\right)\cdot\omega^{-2}(\alpha^{-N}).$

######  Remark 3.10.

By the non-negativity of $\vartheta$, the condition (28) can only be satisfied
if $k=k_{\vartheta}\leq 2$. Let us briefly sketch the reason for this. By
definition of the $L^{1}$-Fourier transform, we have, for all
$\xi\in{\mathbb{R}}^{d}$,

$\displaystyle 1-|\widehat{\vartheta}(\xi)|^{2}$
$\displaystyle=\operatorname{Re}\left(|\widehat{\vartheta}(0)|^{2}-|\widehat{\vartheta}(\xi)|^{2}\right)$
$\displaystyle=\operatorname{Re}\left(\int_{{\mathbb{R}}^{2d}}\vartheta(x)\cdot\vartheta(y)\cdot\left(1-e^{-2\pi\cdot
i\cdot\left<\text{}\,\xi,x-y\,\text{}\right>}\right)\>\mathrm{d\lambda}\,(x,y)\>\right)$
$\displaystyle=\int_{{\mathbb{R}}^{2d}}\vartheta(x)\cdot\vartheta(y)\cdot\left(1-\cos(2\pi\cdot\left<\text{}\,\xi,x-y\,\text{}\right>)\right)\>\mathrm{d\lambda}\,(x,y)\>.$

For all $z\in[-1,1]$,

$1-\cos(z)\geq\frac{z^{2}}{3}.$

Hence, if $\xi=s\cdot e_{1}$, where $s>0$, and $e_{1}$ denotes the first
standard unit vector in ${\mathbb{R}}^{d}$, we obtain

$\displaystyle\frac{1-|\widehat{\vartheta}(\xi)|^{2}}{\left\lVert\text{}\xi\text{}\right\rVert_{2}^{2}}$
$\displaystyle\geq\int_{\left\\{\text{}(x,y)\in{\mathbb{R}}^{2d}\leavevmode\nobreak\
:\leavevmode\nobreak\ |2\pi\cdot s\cdot(x_{1}-y_{1})|\leq
1\text{}\right\\}}\vartheta(x)\cdot\vartheta(y)\cdot(x_{1}-y_{1})^{2}\>\mathrm{d\lambda}\,(x,y)\>$
$\displaystyle\xrightarrow{s\to
0}\int_{{\mathbb{R}}^{2d}}\vartheta(x)\cdot\vartheta(y)\cdot\left(x_{1}-y_{1}\right)^{2}\>\mathrm{d\lambda}\,(x,y)\>>0.$

We conclude that $k_{\vartheta}\leq 2$.

Now, observe that larger values for $k$ in (28) allow more flexibility in the
choice of $\omega$, as the assumptions regarding $h_{k,\omega}$ are easier to
fulfill. However, as justified above, the largest possible value for any
$\vartheta\in L^{1}({\mathbb{R}}^{d})$ is $k_{\vartheta}=2$. Finally, note
that, while we can show an analogous result to Theorem 3.5 for $\vartheta\in
L^{2}({\mathbb{R}}^{d})$, we would not gain a qualitative improvement to
Theorem 3.9, since the same threshold order of convergence $k_{\vartheta}=2$
in (28) would still hold.

###### Proof.

Fix $f\in\mathcal{D}_{\omega}(\Psi;L^{2}({\mathbb{R}}^{d}))$. Let, for every
$\psi\in\Psi$,

$\xi_{\psi}\in\operatorname*{argmin}_{\xi^{\prime}\in{\mathbb{R}}^{d}}\max_{\xi\in\operatorname{supp}(\widehat{\psi})}\left\lVert\text{}\xi-\xi^{\prime}\text{}\right\rVert_{2}.$

We start in a similar fashion to the induction step of the proof of Theorem
3.5. By the same theorem, by the path structure of the scattering network, and
by Lemma 3.4, we find that, for all sufficiently large $N\in\mathbb{N}$,

$\displaystyle W_{N}(f)$ $\displaystyle=\sum_{\psi\in\Psi}W_{N-1}(|f*\psi|)$
$\displaystyle\leq\sum_{\psi\in\Psi}\int_{{\mathbb{R}}^{d}}\left|\text{}\mathcal{F}(|f*\psi|)(\xi)\text{}\right|^{2}\cdot\left(1-\left|\text{}\widehat{\vartheta}\left(C_{\chi,\vartheta}\cdot\alpha^{N-1}\cdot\xi\right)\text{}\right|^{2}\right)\>\mathrm{d}\,\xi\>$
(29)
$\displaystyle\leq\sum_{\psi\in\Psi}\int_{{\mathbb{R}}^{d}}|\widehat{f}(\xi)|^{2}\cdot|\widehat{\psi}(\xi)|^{2}\cdot\left(1-\left|\text{}\widehat{\vartheta}\left(C_{\chi,\vartheta}\cdot\alpha^{N-1}\cdot(\xi-\xi_{\psi})\right)\text{}\right|^{2}\right)\>\mathrm{d}\,\xi\>.$

Our strategy is to split the integral into small scales and large scales
compared with $\alpha^{-N}$, and to establish upper bounds on those terms
separately. Note that our conditions on $h_{k,\omega}$ guarantee that, for all
sufficiently large $N\in\mathbb{N}$,

(30)
$\displaystyle\sup_{t\in(0,\alpha^{-N}]}h_{k,\omega}(t)=h_{k,\omega}(\alpha^{-N})=\alpha^{-kN}\cdot\omega^{-2}(\alpha^{-N}).$

For the small scales, we use (28) and (30) to derive

$\displaystyle\int_{B_{\alpha^{-N}}(\xi_{\psi})}$
$\displaystyle|\widehat{f}(\xi)|^{2}\cdot|\widehat{\psi}(\xi)|^{2}\cdot\left(1-\left|\text{}\widehat{\vartheta}\left(C_{\chi,\vartheta}\cdot\alpha^{N-1}\cdot(\xi-\xi_{\psi})\right)\text{}\right|^{2}\right)\>\mathrm{d}\,\xi\>$
$\displaystyle\leq\int_{B_{\alpha^{-N}}(\xi_{\psi})}|\widehat{f}(\xi)|^{2}\cdot|\widehat{\psi}(\xi)|^{2}\cdot
C\cdot\left\lVert\text{}C_{\chi,\vartheta}\cdot\alpha^{N-1}\cdot(\xi-\xi_{\psi})\text{}\right\rVert_{2}^{k}\>\mathrm{d}\,\xi\>$
$\displaystyle=C\cdot
C_{\chi,\vartheta}^{k}\cdot\alpha^{k(N-1)}\cdot\int_{B_{\alpha^{-N}}(\xi_{\psi})}|\widehat{f}(\xi)|^{2}\cdot|\widehat{\psi}(\xi)|^{2}\cdot\omega^{2}(\left\lVert\text{}\xi-\xi_{\psi}\text{}\right\rVert_{2})\cdot
h_{k,\omega}(\left\lVert\text{}\xi-\xi_{\psi}\text{}\right\rVert_{2})\>\mathrm{d}\,\xi\>$
$\displaystyle\leq C\cdot
C_{\chi,\vartheta}^{k}\cdot\alpha^{-k}\cdot\omega^{-2}(\alpha^{-N})\cdot\int_{B_{\alpha^{-N}}(\xi_{\psi})}|\widehat{f}(\xi)|^{2}\cdot|\widehat{\psi}(\xi)|^{2}\cdot\omega^{2}(d_{\psi})\>\mathrm{d}\,\xi\>.$

The monotonicity of $\omega$ suffices to control the large scales,

$\displaystyle\int_{{\mathbb{R}}^{d}\setminus B_{\alpha^{-N}}(\xi_{\psi})}$
$\displaystyle|\widehat{f}(\xi)|^{2}\cdot|\widehat{\psi}(\xi)|^{2}\cdot\left(1-\left|\text{}\widehat{\vartheta}\left(C_{\chi,\vartheta}\cdot\alpha^{N-1}\cdot(\xi-\xi_{\psi})\right)\text{}\right|^{2}\right)\>\mathrm{d}\,\xi\>$
$\displaystyle\leq\int_{{\mathbb{R}}^{d}\setminus
B_{\alpha^{-N}}(\xi_{\psi})}|\widehat{f}(\xi)|^{2}\cdot|\widehat{\psi}(\xi)|^{2}\cdot\omega^{2}(\left\lVert\text{}\xi-\xi_{\psi}\text{}\right\rVert_{2})\cdot\omega^{-2}(\left\lVert\text{}\xi-\xi_{\psi}\text{}\right\rVert_{2})\>\mathrm{d}\,\xi\>$
$\displaystyle\leq\omega^{-2}(\alpha^{-N})\cdot\int_{{\mathbb{R}}^{d}\setminus
B_{\alpha^{-N}}(\xi_{\psi})}|\widehat{f}(\xi)|^{2}\cdot|\widehat{\psi}(\xi)|^{2}\cdot\omega^{2}(d_{\psi})\>\mathrm{d}\,\xi\>.$

Inserting those estimates into (3) yields, for all sufficiently large
$N\in\mathbb{N}$,

$\displaystyle W_{N}(f)$ $\displaystyle\leq\max\left\\{1,C\cdot
C_{\chi,\vartheta}^{k}\cdot\alpha^{-k}\right\\}\cdot\omega^{-2}(\alpha^{-N})\cdot\sum_{\psi\in\Psi}\int_{{\mathbb{R}}^{d}}|\widehat{f}(\xi)|^{2}\cdot|\widehat{\psi}(\xi)|^{2}\cdot\omega^{2}(d_{\psi})\>\mathrm{d}\,\xi\>$
$\displaystyle=\max\left\\{1,C\cdot
C_{\chi,\vartheta}^{k}\cdot\alpha^{-k}\right\\}\cdot\left(\sum_{\psi\in\Psi}\omega^{2}(d_{\psi})\cdot\left\lVert\text{}f*\psi\text{}\right\rVert_{2}^{2}\right)\cdot\omega^{-2}(\alpha^{-N}).$

Now, if there is $\widetilde{C}>0$ so that
$|\widehat{\vartheta}(C_{\chi,\vartheta}\,\cdot)|\leq|\widehat{\chi}|$ and if
$h_{k,\omega}$ is nondecreasing on the entire interval $(0,\infty)$, then (3)
and (30), and thus also the latter bound for $W_{N}(f)$, are in fact valid for
all $N\in\mathbb{N}_{\geq 2}$. This concludes the proof of the theorem. ∎

The following lemma shows that fast decay (with respect to the weight
$\omega$) of the Fourier transform suffices, independent of $\Psi$, for a
signal to belong to the generalized Sobolev space
$\mathcal{D}_{\omega}(\Psi;L^{2}({\mathbb{R}}^{d}))$.

######  Lemma 3.11.

Let $k>0$. If $\omega:(0,\infty)\to(0,\infty)$ is a weakly $t^{k}$-dominated
weight, then

$\mathcal{F}L_{\omega}^{2}({\mathbb{R}}^{d})\subseteq\mathcal{D}_{\omega}(\Psi;L^{2}({\mathbb{R}}^{d})).$

###### Proof.

Recall from Assumption 3.1 that $\Psi$ can be decomposed into
$\Psi=\bigcup_{j\in\mathbb{N}}\Psi_{j}$ such that
$\operatorname{supp}(\widehat{\psi})\subseteq\mathcal{S}_{r_{j},r_{j+\kappa}}$
whenever $\psi\in\Psi_{j}$. In particular, $d_{\psi}\leq
r_{j+\kappa}\leq\gamma^{-1}\cdot r_{j}$. By assumption on $\omega$, there
exists $J\in\mathbb{N}$ such that $h_{k,\omega}$ is nondecreasing on
$(r_{J},\infty)$. Hence, for all $j\in\mathbb{N}_{>J}$,

(31) $\displaystyle\omega^{2}(d_{\psi})\leq\omega^{2}(\gamma^{-1}\cdot
r_{j})\leq\gamma^{-k}\cdot\omega^{2}(r_{j}).$

Let $f\in\mathcal{F}L_{\omega}^{2}({\mathbb{R}}^{d})$. We aim to show that the
sum
$\sum_{\psi\in\Psi}\omega^{2}(d_{\psi})\cdot\left\lVert\text{}f*\psi\text{}\right\rVert_{2}^{2}$
converges. To do so, we split the sum into small scales and large scales
compared with $J$,

$\displaystyle\sum_{j=1}^{J}\sum_{\psi\in\Psi_{j}}\int_{\mathcal{S}_{r_{j},r_{j+\kappa}}}|\widehat{f}(\xi)|^{2}\cdot|\widehat{\psi}(\xi)|^{2}\cdot\omega^{2}(d_{\psi})\>\mathrm{d}\,\xi\>+\sum_{j=J+1}^{\infty}\sum_{\psi\in\Psi_{j}}\int_{\mathcal{S}_{r_{j},r_{j+\kappa}}}|\widehat{f}(\xi)|^{2}\cdot|\widehat{\psi}(\xi)|^{2}\cdot\omega^{2}(d_{\psi})\>\mathrm{d}\,\xi\>,$

so that it suffices to bound these expressions individually. We can easily
control the small scales by the monotonicity of $\omega$,

$\displaystyle\sum_{j=1}^{J}\sum_{\psi\in\Psi_{j}}\int_{\mathcal{S}_{r_{j},r_{j+\kappa}}}$
$\displaystyle|\widehat{f}(\xi)|^{2}\cdot|\widehat{\psi}(\xi)|^{2}\cdot\omega^{2}(d_{\psi})\>\mathrm{d}\,\xi\>$
$\displaystyle\leq\omega^{2}(r_{J+\kappa})\cdot\sum_{j=1}^{J}\sum_{\psi\in\Psi_{j}}\int_{\mathcal{S}_{r_{j},r_{j+\kappa}}}|\widehat{f}(\xi)|^{2}\cdot|\widehat{\psi}(\xi)|^{2}\>\mathrm{d}\,\xi\>$
$\displaystyle\leq\frac{\omega^{2}(r_{J+\kappa})}{\omega^{2}(r_{1})}\cdot\sum_{j=1}^{J}\sum_{\psi\in\Psi_{j}}\int_{\mathcal{S}_{r_{j},r_{j+\kappa}}}|\widehat{f}(\xi)|^{2}\cdot|\widehat{\psi}(\xi)|^{2}\cdot\omega^{2}(\left\lVert\text{}\xi\text{}\right\rVert_{2})\>\mathrm{d}\,\xi\>.$

For the large scales, in addition to the monotonicity of $\omega$, we also use
(31) to find that

$\displaystyle\sum_{j=J+1}^{\infty}\sum_{\psi\in\Psi_{j}}\int_{\mathcal{S}_{r_{j},r_{j+\kappa}}}$
$\displaystyle|\widehat{f}(\xi)|^{2}\cdot|\widehat{\psi}(\xi)|^{2}\cdot\omega^{2}(d_{\psi})\>\mathrm{d}\,\xi\>$
$\displaystyle\leq\sum_{j=J+1}^{\infty}\sum_{\psi\in\Psi_{j}}\omega^{2}(\gamma^{-1}\cdot
r_{j})\cdot\int_{\mathcal{S}_{r_{j},r_{j+\kappa}}}|\widehat{f}(\xi)|^{2}\cdot|\widehat{\psi}(\xi)|^{2}\>\mathrm{d}\,\xi\>$
$\displaystyle\leq\sum_{j=J+1}^{\infty}\sum_{\psi\in\Psi_{j}}\gamma^{-k}\cdot\omega^{2}(r_{j})\cdot\int_{\mathcal{S}_{r_{j},r_{j+\kappa}}}|\widehat{f}(\xi)|^{2}\cdot|\widehat{\psi}(\xi)|^{2}\>\mathrm{d}\,\xi\>$
$\displaystyle\leq\sum_{j=J+1}^{\infty}\sum_{\psi\in\Psi_{j}}\gamma^{-k}\cdot\int_{\mathcal{S}_{r_{j},r_{j+\kappa}}}|\widehat{f}(\xi)|^{2}\cdot|\widehat{\psi}(\xi)|^{2}\cdot\omega^{2}(\left\lVert\text{}\xi\text{}\right\rVert_{2})\>\mathrm{d}\,\xi\>.$

Altogether, letting
$C=\max\left\\{\text{}\frac{\omega^{2}(r_{J+\kappa})}{\omega^{2}(r_{1})},\gamma^{-k}\text{}\right\\}$,
we obtain

$\displaystyle\sum_{\psi\in\Psi}\omega^{2}(d_{\psi})\cdot\left\lVert\text{}f*\psi\text{}\right\rVert_{2}^{2}$
$\displaystyle\leq
C\cdot\sum_{j=1}^{\infty}\sum_{\psi\in\Psi_{j}}\int_{\mathcal{S}_{r_{j},r_{j+\kappa}}}|\widehat{f}(\xi)|^{2}\cdot|\widehat{\psi}(\xi)|^{2}\cdot\omega^{2}(\left\lVert\text{}\xi\text{}\right\rVert_{2})\>\mathrm{d}\,\xi\>$
$\displaystyle\leq
C\cdot\kappa\cdot\sum_{\psi\in\Psi}\int_{{\mathbb{R}}^{d}}|\widehat{f}(\xi)|^{2}\cdot|\widehat{\psi}(\xi)|^{2}\cdot\omega^{2}(\left\lVert\text{}\xi\text{}\right\rVert_{2})\>\mathrm{d}\,\xi\>$
$\displaystyle\leq
C\cdot\kappa\cdot\int_{{\mathbb{R}}^{d}}|\widehat{f}(\xi)|^{2}\cdot\omega^{2}(\left\lVert\text{}\xi\text{}\right\rVert_{2})\>\mathrm{d}\,\xi\><\infty.$

∎

There are choices of the weight $\omega$ and the filter bank $\Psi$, for which
the inclusion
$\mathcal{F}L_{\omega}^{2}({\mathbb{R}}^{d})\subseteq\mathcal{D}_{\omega}(\Psi;L^{2}({\mathbb{R}}^{d}))$
is strict, i.e., the spaces $\mathcal{F}L_{\omega}^{2}({\mathbb{R}}^{d})$ and
$\mathcal{D}_{\omega}(\Psi;L^{2}({\mathbb{R}}^{d}))$ do not coincide, see
Remark 4.3.

As a consequence of Theorem 3.9 and Lemma 3.11, we obtain asymptotic
convergence rates for signals of (logarithmic) Sobolev regularity.

######  Corollary 3.12.

Suppose that Assumption 3.1 holds for $\Psi$ with parameters $\gamma$ and
$\rho$. As before, let

$\alpha:=\sqrt{1-\frac{4\gamma}{(1+\gamma)^{2}}\cdot(1-\rho)^{2}}\>\in(0,1).$

Then, the following hold:

  1. a)

If $f\in H^{s}({\mathbb{R}}^{d})$ for some $s>0$, then

$W_{N}(f)\in\mathcal{O}(\alpha^{2\min\left\\{\text{}s,1\text{}\right\\}\cdot
N}).$

  2. b)

If $f\in H^{s}_{\log}({\mathbb{R}}^{d})$ for some $s>0$, then

$W_{N}(f)\in\mathcal{O}(N^{-2s}).$

###### Proof.

Let

$\vartheta:{\mathbb{R}}^{d}\to{\mathbb{R}},\quad x\mapsto
e^{-\pi\cdot\left\lVert\text{}x\text{}\right\rVert_{2}^{2}}.$

Then, $\vartheta$ satisfies the prerequisites of Theorem 3.5. From the series
expansion of the one-dimensional Gaussian at $0$ we can see that, for all
$\xi\in{\mathbb{R}}^{d}$,

$1-|\widehat{\vartheta}(\xi)|^{2}\leq
2\pi\cdot\left\lVert\text{}\xi\text{}\right\rVert_{2}^{2}.$

Now, part a) follows immediately from Theorem 3.9 and Lemma 3.11 if we take

$\omega:(0,\infty)\to(0,\infty),\quad t\mapsto(1+t^{2})^{\frac{s}{2}}.$

In fact, using the criterion from Remark 3.8, it is straightforward to see
that $\omega$ is a strongly $t^{2}$-dominated weight if $s\in(0,1]$, since for
all $t>0$,

$2\omega(t)\geq
2s\cdot\frac{t^{2}}{1+t^{2}}\cdot(1+t^{2})^{\frac{s}{2}}=2t\cdot\omega^{\prime}(t)\geq
0.$

We draw the conclusion by noting that
$\mathcal{F}L_{\omega}^{2}({\mathbb{R}}^{d})=H^{s}({\mathbb{R}}^{d})$ and
that, for all $N\in\mathbb{N}$,

$\omega^{-2}(\alpha^{-N})\leq\alpha^{2s\cdot N}.$

For $s>1$, the statement simply follows from the inclusion
$H^{s}({\mathbb{R}}^{d})\subseteq H^{1}({\mathbb{R}}^{d})$.

In order to establish part b), let us consider

$\omega:(0,\infty)\to(0,\infty),\quad t\mapsto\ln^{s}(e+t).$

We verify the condition from Remark 3.8 again. In general, $\omega$ is not a
strongly $t^{2}$-dominated weight (if $s$ is too large). However, $\omega$ is
a weakly $t^{2}$-dominated weight. Indeed, for all $t\geq e^{s}-e$, we have
$\ln(e+t)\geq s$, which entails

$2\omega(t)=2\ln(e+t)\cdot\ln^{s-1}(e+t)\geq
2s\cdot\frac{t}{e+t}\cdot\ln^{s-1}(e+t)=2t\cdot\omega^{\prime}(t)\geq 0.$

Finally, by definition,
$\mathcal{F}L_{\omega}^{2}({\mathbb{R}}^{d})=H^{s}_{\log}({\mathbb{R}}^{d})$,
and we have, for all $N\in\mathbb{N}$,

$\omega^{-2}(\alpha^{-N})\leq\ln^{-2s}(\alpha^{-N})=\ln^{-2s}(\alpha^{-1})\cdot
N^{-2s}.$

This concludes the proof of part b) by Theorem 3.9. ∎

######  Remark 3.13.

Using arguments very similar to those in the proof of Theorem 3.9 together
with the compactly supported positive definite radial basis function

(32)
$\displaystyle\widehat{\vartheta}:{\mathbb{R}}^{d}\to{\mathbb{R}},\quad\xi\mapsto\left(1-\left\lVert\text{}\xi\text{}\right\rVert_{2}\right)_{+}^{\left\lfloor\frac{d}{2}\right\rfloor+1}$

in the above proof, one can derive the following explicit upper bounds (as
opposed to the asymptotic bounds in the above corollary):

  1. a)

If $f\in H^{s}({\mathbb{R}}^{d})$ for some $s>0$, then we have, for all
$N\in\mathbb{N}$,

$W_{N}(f)\leq\max\left\\{\text{}1,\frac{2\left(\left\lfloor\frac{d}{2}\right\rfloor+1\right)}{\alpha\cdot
r_{1}}\text{}\right\\}\cdot\left\lVert\text{}f\text{}\right\rVert_{H^{s}({\mathbb{R}}^{d})}^{2}\cdot\alpha^{\min\\{2s,1\\}\cdot
N}.$

  2. b)

If $f\in H^{s}_{\log}({\mathbb{R}}^{d})$ for some $s>0$, then we have, for all
$N\in\mathbb{N}$,

$W_{N}(f)\leq\max\left\\{\text{}1,\frac{2\left(\left\lfloor\frac{d}{2}\right\rfloor+1\right)}{\alpha\cdot
r_{1}}\text{}\right\\}\cdot\ln^{-\min\\{2s,1\\}}(\alpha^{-1})\cdot\left\lVert\text{}f\text{}\right\rVert_{H^{s}_{\log}({\mathbb{R}}^{d})}^{2}\cdot
N^{-\min\\{2s,1\\}}.$

The differences in the asymptotic rates of decay (compared with the above
corollary) are due to the fact that

  * •

the largest possible values $k_{\vartheta}$ in (28) differ if $\vartheta$ is a
Gaussian ($k_{\vartheta}=2$) or if $\vartheta$ is defined as in (32)
($k_{\vartheta}=1$).

  * •

the weight $\omega:(0,\infty)\to(0,\infty),t\mapsto\ln^{s}(e+t)$ is not
strongly $t$-dominated if $s$ is too large. However, it is easy to see that
$\omega$ is strongly $t$-dominated if $s\in(0,\frac{1}{2}]$.

##  4\. Applications

###  4.1. Scattering with UFC filters

In this section, we apply our results from Section 3 to a large class of
filter banks that is closely related to the class of uniform covering frames
introduced in [3].

######  Definition 4.1.

We say that the filters $\Psi$ have uniform frequency concentration (UFC) if
they fulfill Assumption 3.1, and if

$D_{\Psi}:=\sup_{\psi\in\Psi}d_{\psi}=\sup_{\psi\in\Psi}\inf_{\xi_{\psi}\in{\mathbb{R}}^{d}}\sup_{\xi\in\operatorname{supp}(\widehat{\psi})}\left\lVert\text{}\xi-\xi_{\psi}\text{}\right\rVert_{2}<\infty.$

It turns out that for such scattering networks, energy decay is exponential,
globally on $L^{2}({\mathbb{R}}^{d})$. Thereby, we complement the earlier
result [3, Proposition 3.3], which states global exponential energy decay for
scattering networks based on uniform covering frames. We have several comments
concerning the similarities and differences between our following result and
the result in [3]:

  * •

In [3], the output-generating low-pass filter is assumed to be bandlimited.
Our result also applies if that is not the case.

  * •

In [3], the frequency support of each filter $\psi\in\Psi$ is assumed to be
connected. Our result does not require connectedness of the frequency
supports. However, as is pointed out in [3, Remark 2.2], the motivation to
require connectedness of the frequency supports is to preclude certain
pathological behavior such as $\operatorname{supp}(\widehat{\psi})$ having two
connected components, where one component is near the origin and the other is
far from the origin. In our setting, we implicitly preclude such pathological
behavior by means of Assumption 3.1, which imposes frequency localization of
the filters.

  * •

If $D_{\Psi}<\infty$, then the frequency support of each filter $\psi\in\Psi$
is contained in a closed ball of radius $D_{\Psi}$. In particular, $\Psi$
satisfies the uniform covering property introduced in [3], i.e., for any
$R>0$, there exists $N\in\mathbb{N}$ such that for each $\psi\in\Psi$, the set
$\operatorname{supp}(\widehat{\psi})$ can be covered by $N$ cubes of side
length $2R$. In turn, if $\Psi$ satisfies the uniform covering property, and
if the frequency supports of the filters are connected, then
$D_{\Psi}<\infty$.

  * •

The findings in [3, Proposition 3.3] guarantee exponential decay of the energy
remainder: For all $f\in L^{2}({\mathbb{R}}^{d})$, and all $N\in\mathbb{N}$,

$W_{N}(f)\leq\alpha^{N}\cdot\left(\left\lVert\text{}f\text{}\right\rVert_{2}^{2}-\left\lVert\text{}f*\chi\text{}\right\rVert_{2}^{2}\right),$

for an unspecified constant $\alpha\in(0,1)$ depending only on the uniform
covering frame. The main advantage of our result is that we can explicitly
specify the values for all occurring quantities in the upper bound on the
energy remainder, including a precise value for $\alpha$.

######  Corollary 4.2.

Suppose that the filters $\Psi$ have uniform frequency concentration with
parameters $\gamma$, $\rho$, and $r_{1}$ from Assumption 3.1.

Then we have, for all $f\in L^{2}({\mathbb{R}}^{d})$, and for all
$N\in\mathbb{N}$,

$W_{N}(f)\leq\max\left\\{\text{}1,\frac{2\left(\left\lfloor\frac{d}{2}\right\rfloor+1\right)}{\alpha\cdot
r_{1}}\text{}\right\\}\cdot
D_{\Psi}\cdot\left(\left\lVert\text{}f\text{}\right\rVert_{2}^{2}-\left\lVert\text{}f*\chi\text{}\right\rVert_{2}^{2}\right)\cdot\alpha^{N},$