# From Kernel Methods to Neural Networks:
A Unifying Variational Formulation

Michael Unser Biomedical Imaging Group, École polytechnique fédérale de
Lausanne (EPFL), Station 17, CH-1015, Lausanne, Switzerland
(michael.unser@epfl.ch).

###### Abstract

The minimization of a data-fidelity term and an additive regularization
functional gives rise to a powerful framework for supervised learning. In this
paper, we present a unifying regularization functional that depends on an
operator ${\rm L}$ and on a generic Radon-domain norm. We establish the
existence of a minimizer and give the parametric form of the solution(s) under
very mild assumptions. When the norm is Hilbertian, the proposed formulation
yields a solution that involves radial-basis functions and is compatible with
the classical methods of machine learning. By contrast, for the total-
variation norm, the solution takes the form of a two-layer neural network with
an activation function that is determined by the regularization operator. In
particular, we retrieve the popular ReLU networks by letting the operator be
the Laplacian. We also characterize the solution for the intermediate
regularization norms $\|\cdot\|=\|\cdot\|_{L_{p}}$ with $p\in(1,2]$. Our
framework offers guarantees of universal approximation for a broad family of
regularization operators or, equivalently, for a wide variety of shallow
neural networks, including the cases (such as ReLU) where the activation
function is increasing polynomially. It also explains the favorable role of
bias and skip connections in neural architectures.

###### Contents

  1. 1 Introduction
  2. 2 Mathematical Preliminaries
    1. 2.1 Notations
    2. 2.2 Admissible Regularization Operators
    3. 2.3 Nontrivial Null Space and Related Projectors
  3. 3 Radon Transform
    1. 3.1 Classical Integral Formulation
    2. 3.2 Distributional Extension
    3. 3.3 Radon-Compatible Banach Spaces
    4. 3.4 Specific Radon Transforms
  4. 4 Unifying Variational Formulation
    1. 4.1 Representer Theorem for Radon-Domain Regularization
    2. 4.2 Connection with RKHS Methods
    3. 4.3 Universal Approximation Properties
    4. 4.4 Regularization Operators for Antisymmetric Activations
    5. 4.5 Specific Configurations
  5. 5 Supporting Mathematical Results
    1. 5.1 Kernel and Stability of Generalized Inverse Operators
    2. 5.2 Characterization of the Predual Space ${\mathcal{X}}_{{\mathrm{L}}_{\rm R}}$
  6. A Appendix: Duality Maps

##  1 Introduction

Regularization theory constitutes a powerful framework for the derivation of
algorithms for supervised learning [12, 39, 40]. Given a series of data points
$({\boldsymbol{x}}_{m},y_{m})\in\mathbb{R}^{d}\times\mathbb{R}$,
$m=1,\dots,M$, the basic problem (regression) is to find a mapping
$f:\mathbb{R}^{d}\to\mathbb{R}$ such that $f({\boldsymbol{x}}_{m})\approx
y_{m}$, without overfitting. The standard paradigm is to let $f$ be the
minimizer of a cost that consists of a data-fidelity term and an additive
regularization functional [8]. The minimization proceeds over a prescribed
class ${\mathcal{H}}$ of candidate functions. One usually distinguishes
between the parametric approaches (e.g., neural networks), where
${\mathcal{H}}={\mathcal{H}}_{\Theta}$ is a family of functions specified by a
finite set of parameters $\theta\in\Theta$ (e.g., the weights of the network),
and the nonparametric ones, where the properties of the solution are
controlled by the regularization functional. The focus of this paper is on the
nonparametric techniques. They rely on functional optimization, which means
that the minimization proceeds over a space of functions rather than over a
set of parameters. The regularization is usually chosen to be an increasing
function of the norm associated with a particular Banach space, which results
in a well-posed problem [16, 15, 51].

The functional-optimization point of view is often constructive, in that it
suggests or supports explicit learning architectures. For instance, the choice
of the Hilbertian regularization $R(f)=\|f\|^{2}_{{\mathcal{H}}}$ where
${\mathcal{H}}$ is a reproducing kernel Hilbert space (RKHS) results in a
closed-form solution that is a linear combination of kernels positioned on the
data [57, 7]. In fact, the RKHS setting yields a generic class of estimators
that is compatible with the classical kernel-based methods of machine
learning, including support vector machines [57, 39, 47, 46, 1, 51]. Likewise,
adaptive kernel methods are justifiable from the minimization of a generalized
total-variation norm, which favors sparse representions [9, 10, 3]. These
latter results actually take their root in spline theory [18, 27, 55].
Similarly, it has been demonstrated that shallow ReLU networks are solutions
of functional-optimization problems with an appropriate regularization. One
way to achieve this is to start from an explicit parameterization of an
infinite-width network [4] (the reverse engineering/synthesis approach).
Another way is to consider a regularization operator that is matched to the
neuronal activation with a $L_{1}$-type penalty111The precise formulation
involves the ${\mathcal{M}}$-norm (or total variation), which is the weak form
of $L_{1}$ associated with space of bounded Radon measures. In our account, we
take it as the default norm for the Lebesgue exponent $p=1$, with a slight
abuse of language.; for instance, a second-order derivative for $d=1$ [45, 35]
or, more generally, the Radon-domain counterpart of the Laplace operator whose
Green’s function is precisely a ReLU ridge [34, 36, 52]. Similar optimality
results can also be stated within the framework of reproducing-kernel Banach
spaces [6], which is a formal point of view that bridges the synthesis and
analysis approach of [4] and [36], respectively.

The second important benefit of the functional-optimization approach is that
it gives insight on the approximation capabilities (expressivity) of the
resulting learning architectures. This information is encapsulated in the
choice/definition of the native space ${\mathcal{H}}$ (typically, a Sobolev
space), which goes hand-in-hand with the regularization functional. Roughly
speaking, the native space ${\mathcal{H}}$ ought to be “large enough” to allow
for the approximation of any continuous function with an arbitrary degree of
precision. This universal approximation property is a central theme in the
theory of radial-basis functions (RBFs) [30, 58]. In machine learning, the
kernel estimators that meet this approximation requirement are called
universal [31]. When the basis functions are shifted replicates of a single
template $h:\mathbb{R}^{d}\to\mathbb{R}$, then the condition is equivalent to
$h$ being strictly positive definite, which means that its Fourier transform
is real-valued symmetric, and (strictly) positive [11]. Similar guarantees of
universal approximation exist for (shallow) neural networks under mild
conditions on the activation functions [14, 24, 29, 5, 37]. The main
difference with the RKHS framework, however, is that the universality results
for neural nets usually make the assumption that the input domain is a compact
subset of $\mathbb{R}^{d}$.

The purpose of this paper is to unify and extend these various approaches by
introducing a universal regularization functional. The latter has two
components: an admissible differential operator ${\rm L}$, and an $L_{p}$-type
Radon-domain norm. The resulting regularization operator is ${\rm L}_{\rm
R}={\mathrm{K}}_{\rm rad}{\mathrm{R}}{\rm L}$, where ${\mathrm{R}}$ is the
Radon transform and ${\mathrm{K}}_{\rm rad}$ the “filtering” operator of
computer tomography [32]. Our main result (Theorem 5) gives the parametric
form of the solution of the corresponding functional-optimization problems
under minimal hypotheses. Remarkably, the solution for $p=2$ is compatible
with the type of kernel expansions (RBFs) of classical machine learning while,
for $p=1$, it maps into a neural network with one hidden layer. The foundation
for this characterization is an abstract representer theorem for direct-sum
Banach spaces [53].

The primary effort in this paper consists in the development of a functional
framework that is adapted to the Radon transform and that fulfills the
hypotheses of the abstract theorem. The main contributions can be summarized
as follows.

  1. 1.

Construction and characterization of an extended family of native Banach
spaces ${\mathcal{X}}^{\prime}_{{\mathrm{L}}_{\rm R}}(\mathbb{R}^{d})$
associated with a generic Radon-domain norm
$\|\cdot\|_{{\mathcal{X}}^{\prime}}$ and a differential operator ${\rm L}$,
under the general admissibility conditions stated in Definition 2 (Theorem 6).

  2. 2.

Proof that (i) the sampling functionals
$\delta(\cdot-{\boldsymbol{x}}_{m}):{\mathcal{X}}^{\prime}_{{\mathrm{L}}_{\rm
R}}(\mathbb{R}^{d})\to\mathbb{R}$ are weak*-continuous; and (ii) the adjoint
of the regularization operator has a stable generalized inverse ${\rm
L}^{\ast\dagger}_{\rm R}$ (see Theorem 7 and accompanying explanations). These
technical points are essential to the argumentation (existence of solution).

  3. 3.

Extension and unification of a number of earlier optimality results for RBF
expansions and neural networks. While the present setup for $p=2$ and ${\rm
L}=(-\Delta)^{\gamma}$ is reminiscent of thin-plate splines [17, 28], the
resulting solution for a fixed $\gamma$ does not depend on the dimension $d$,
which makes it easier to deploy. Likewise, our variational formulation with
${\mathcal{X}}^{\prime}={\mathcal{M}}$ extends the results of Parhi and Nowak
[36] by: (i) proving that the neural network parameterization applies to all
the extreme points of the solution set, and (ii) by covering a much broader
class of activation functions, including those with polynomial growth (of
degree $n_{0}$).

  4. 4.

General guarantees of universality, subject to the admissibility condition in
Definition 2. While the result for $p=2$ is consistent with the known criteria
for kernel estimators [31], its counterpart for neural networks
$({\mathcal{X}}^{\prime}={\mathcal{M}})$ brings in a new twist: the addition
of a polynomial component. The latter, which is not present in the traditional
theory [5, 37], is necessary to lift the hypothesis of a compact input domain.
The two cases of greatest practical relevance are the sigmoid and the ReLU
activations which, in our formulation, require the addition of a bias
($n_{0}=0$) and an affine term $(n_{0}=1)$, respectively. Interestingly, the
ReLU case yields a neural architecture with a skip connection akin to ResNet
[22], which is highly popular in practice.

The paper is organised as follows: We start with mathematical preliminaries in
Section 2. In particular, we state our criteria of admissibility for ${\rm L}$
and show how to represent its polynomial null space. In Section 3, we review
the main properties of the Radon transform and specify the dual pair
$({\mathcal{X}}_{\rm Rad},{\mathcal{X}}^{\prime}_{\rm Rad})$ of hyper-
spherical Banach spaces that enter the definition of our native spaces. We
also provide formulas for the (filtered) Radon transform of RBFs and ridges
(the elementary constituents of neural networks). Section 4 is devoted to the
description and interpretation of our main result (Theorem 5). In particular,
we draw a connection with RKHS in Section 4.2. We discuss the issue of
universality in Section 4.3 and show in Section 4.4 how our framework can be
extended to handle antisymmetric activations, including sigmoids. We
complement our exposition in Section 4.5 with a listing of specific
configurations, many of which are intimately connected to splines. The
mathematical developments that support our formulation are presented in
Section 5. They include the characterization of the kernel of the inverse
operator ${\mathrm{L}}^{\ast\dagger}_{\rm R}$—the enabling ingredient of our
formulation—and the construction of the predual Banach space
${\mathcal{X}}_{{\mathrm{L}}_{\rm R}}(\mathbb{R}^{d})$.

##  2 Mathematical Preliminaries

###  2.1 Notations

We shall consider multidimensional functions $f$ on $\mathbb{R}^{d}$ that are
indexed by the variable ${\boldsymbol{x}}\in\mathbb{R}^{d}$. To describe their
partial derivatives, we use the multi-index
${\boldsymbol{k}}=(k_{1},\dots,k_{d})\in\mathbb{N}^{d}$ (where $\mathbb{N}$
includes $0$) with the notational conventions
${\boldsymbol{k}}!=\prod_{i=1}^{d}k_{i}!$,
$|{\boldsymbol{k}}|=k_{1}+\cdots+k_{d}$,
${\boldsymbol{x}}^{{\boldsymbol{k}}}=\prod_{i=1}^{d}x_{i}^{k_{i}}$ for any
${\boldsymbol{x}}\in\mathbb{R}^{d}$, and
$\partial^{\boldsymbol{k}}f({\boldsymbol{x}})=\frac{\partial^{|{\boldsymbol{k}}|}f(x_{1},\dots,x_{d})}{\partial^{k_{1}}_{x_{1}}\cdots\partial^{k_{d}}_{x_{d}}}$.
This allows us to write the multidimensional Taylor expansion around
${\boldsymbol{x}}={\boldsymbol{x}}_{0}$ of an analytical function
$f:\mathbb{R}^{d}\to\mathbb{R}$ explicitly as

$\displaystyle
f({\boldsymbol{x}})=\sum_{n=0}^{\infty}\sum_{|{\boldsymbol{k}}|=n}\frac{\partial^{\boldsymbol{k}}f({\boldsymbol{x}}_{0})({\boldsymbol{x}}-{\boldsymbol{x}}_{0})^{\boldsymbol{k}}}{{\boldsymbol{k}}!~}$
(1)

where the internal summation is over all multi-indices ${\boldsymbol{k}}$ such
that $k_{1}+\cdots+k_{d}=n$.

Schwartz’ space of smooth and rapidly decreasing test functions
$\varphi:\mathbb{R}^{d}\to\mathbb{R}$ equipped with the usual Fréchet-Schwartz
topology is denoted by ${\mathcal{S}}(\mathbb{R}^{d})$. Its continuous dual is
the space ${\mathcal{S}}^{\prime}(\mathbb{R}^{d})$ of tempered distributions.
In this setting, the Lebesgue spaces $L_{p}(\mathbb{R}^{d})$ for
$p\in[1,\infty)$ can be specified as the completion of
${\mathcal{S}}(\mathbb{R}^{d})$ equipped with the $L_{p}$-norm
$\|\cdot\|_{L_{p}}$, denoted as
$L_{p}(\mathbb{R}^{d})=\overline{({\mathcal{S}}(\mathbb{R}^{d}),\|\cdot\|_{L_{p}})}$.
For the endpoint $p=\infty$, we have
$\overline{({\mathcal{S}}(\mathbb{R}^{d}),\|\cdot\|_{L_{\infty}})}=C_{0}(\mathbb{R}^{d})$
with
$\|\varphi\|_{L_{\infty}}=\sup_{{\boldsymbol{x}}\in\mathbb{R}^{d}}|\varphi({\boldsymbol{x}})|$,
which is the space of continuous functions that vanish at infinity. The
continuous dual of $C_{0}(\mathbb{R}^{d})$ is the space
${\mathcal{M}}(\mathbb{R}^{d})=\\{f\in{\mathcal{S}}^{\prime}(\mathbb{R}^{d}):\|f\|_{{\mathcal{M}}}<\infty\\}$
of bounded Radon measures with

$\displaystyle\|f\|_{{\mathcal{M}}}=\sup_{\varphi\in{\mathcal{S}}(\mathbb{R}^{d}):\|\varphi\|_{L_{\infty}}\leq
1}\langle f,\varphi\rangle.$ (2)

The latter is a superset of $L_{1}(\mathbb{R}^{d})$, which is isometrically
embedded in it, in the sense that $\|f\|_{L_{1}}=\|f\|_{{\mathcal{M}}}$ for
all $f\in L_{1}(\mathbb{R}^{d})$.

The Fourier transform of a function $\varphi\in L_{1}(\mathbb{R}^{d})$ is
defined as

$\displaystyle\widehat{\varphi}({\boldsymbol{\omega}})\stackrel{{\scriptstyle\vartriangle}}{{=}}\mathcal{F}\\{\varphi\\}({\boldsymbol{\omega}})=\frac{1}{(2\pi)^{d}}\int_{\mathbb{R}^{d}}\varphi({\boldsymbol{x}})\mathrm{e}^{-\mathrm{i}\langle{\boldsymbol{\omega}},{\boldsymbol{x}}\rangle}{\rm
d}{\boldsymbol{x}}.$ (3)

Since the Fourier operator $\mathcal{F}$ continuously maps
${\mathcal{S}}(\mathbb{R}^{d})$ into itself, the transform can be extended by
duality to the whole space ${\mathcal{S}}^{\prime}(\mathbb{R}^{d})$ of
tempered distribution. Specifically,
$\widehat{f}=\mathcal{F}\\{f\\}\in{\mathcal{S}}^{\prime}(\mathbb{R}^{d})$ is
the (unique) generalized Fourier transform of
$f\in{\mathcal{S}}^{\prime}(\mathbb{R}^{d})$ if and only if
$\langle\widehat{f},\varphi\rangle=\langle f,\widehat{\varphi}\rangle$ for all
$\varphi\in{\mathcal{S}}(\mathbb{R}^{d})$, where
$\widehat{\varphi}=\mathcal{F}\\{\varphi\\}$ is the “classical” Fourier
transform of $\varphi$ defined by (3).

To control the minimal order $\alpha\geq 0$ of decay (resp., the maximal rate
of growth) of functions, we use the dual pair of spaces
$L_{1,\alpha}(\mathbb{R}^{d})$ and
$L_{\infty,-\alpha}(\mathbb{R}^{d})=\big{(}L_{1,\alpha}(\mathbb{R}^{d})\big{)}^{\prime}$.
These are the Banach spaces associated with the weighted norms

$\displaystyle\|f\|_{L_{1,\alpha}}\stackrel{{\scriptstyle\vartriangle}}{{=}}\int_{\mathbb{R}^{d}}(1+\|{\boldsymbol{x}}\|)^{\alpha}|f({\boldsymbol{x}})|{\rm
d}{\boldsymbol{x}}$ (4)
$\displaystyle\|f\|_{L_{\infty,-\alpha}}\stackrel{{\scriptstyle\vartriangle}}{{=}}\operatorname*{ess\,sup}_{{\boldsymbol{x}}\in\mathbb{R}}(1+\|{\boldsymbol{x}}\|)^{-\alpha}|f({\boldsymbol{x}})|,$
(5)

respectively. Specifically, the inclusion $f\in
L_{\infty,-n_{0}}(\mathbb{R}^{d})$ with $n_{0}\in\mathbb{N}$ indicates that
$f$ cannot grow faster than a polynomial of degree $n_{0}$, while the
condition $f\in L_{1,\alpha}(\mathbb{R}^{d})$ implies that
$f({\boldsymbol{x}})$ must be locally integrable and must decay (slightly)
faster than $1/\|{\boldsymbol{x}}\|^{\alpha+d}$ as
$\|{\boldsymbol{x}}\|\to\infty$.

###  2.2 Admissible Regularization Operators

The regularization operators ${\rm L}$ that are of interest to us are linear,
shift-invariant (LSI), and isotropic. Consequently, they are characterized by
a frequency response $\widehat{L}$ that is purely radial, with
$\widehat{L}({\boldsymbol{\omega}})=\widehat{L}_{\rm
rad}(\|{\boldsymbol{\omega}}\|)$, where the radial profile $\widehat{L}_{\rm
rad}:\mathbb{R}\to\mathbb{R}$ is a continuous, symmetric function. Our
condition for admissibility is that ${\rm L}$ be invertible in an appropriate
sense.

######  Definition 1 (Spline-admissible operators with trivial null space).

An isotropic LSI operator ${\rm L}$ has a trivial null space if its radial
frequency profile $\widehat{L}_{\rm rad}$ does not vanish over $\mathbb{R}$.
We then say that it is spline-admissible if $1/\widehat{L}_{\rm rad}\in
L_{1}(\mathbb{R})$ and $\rho_{\rm rad}=\mathcal{F}^{-1}\\{1/\widehat{L}_{\rm
rad}\\}\in L_{1}(\mathbb{R})$ where the operator
$\mathcal{F}^{-1}:L_{1}(\mathbb{R})\to C_{0}(\mathbb{R})$ is the classical
inverse Fourier transform.

The typical scenario is
$\widehat{L}({\boldsymbol{\omega}})=(1+\|{\boldsymbol{\omega}}\|^{2})^{\alpha/2}$
with $\alpha\geq 1$, which results in a stable inverse operator
${\mathrm{L}}^{-1}$ whose radially symmetric impulse response is the Bessel
potential of order $\alpha$. These operators play a central in the theory of
Sobolev spaces [21].

Distribution theory allows us to go further and to invert operators with
nontrivial null spaces, but only if the zeros of the frequency response are
located at isolated points. When the operator is isotropic, this reduces the
options to the cases where $\widehat{L}({\boldsymbol{\omega}})$ has a
(multiple) zero at ${\boldsymbol{\omega}}={\boldsymbol{0}}$. Specifically, we
shall say that ${\rm L}$ is of order $\gamma_{0}$ if
$|\widehat{L}({\boldsymbol{\omega}})|/\|{\boldsymbol{\omega}}\|^{\gamma_{0}}=C_{0}$
as $\|{\boldsymbol{\omega}}\|\to 0$. The second important parameter is the
asymptotic growth exponent of $\widehat{L}({\boldsymbol{\omega}})$. This is
the largest index $\gamma_{1}$ such that
$|\widehat{L}({\boldsymbol{\omega}})|\geq
C_{1}\|{\boldsymbol{\omega}}\|^{\gamma_{1}},$ for all
$\|{\boldsymbol{\omega}}\|>R$. It determines the smoothness of the Green’s
function of the operator.

######  Definition 2 (Spline-admissible operators with nontrivial null space).

An isotropic LSI operator ${\rm L}$ with radial frequency profile
$\widehat{L}_{\rm rad}$ is said to be spline-admissible with a polynomial null
space of degree $n_{0}$ if the following conditions are satisfied.

  1. 1.

The profile $\widehat{L}_{\rm rad}$ does not vanish over $\mathbb{R}$, except
for a zero of order $\gamma_{0}\in(n_{0},n_{0}+1]$ at the origin; that is,
$|\widehat{L}_{\rm rad}(\omega)|/|\omega|^{\gamma_{0}}=C_{0}$ as $\omega\to
0$.

  2. 2.

There exists an order $\gamma_{1}>1$, a constant $C_{1}>0$, and a radius
$R_{1}>0$ such that $|\widehat{L}_{\rm rad}(\omega)|\geq
C_{1}|\omega|^{\gamma_{1}}$ for all $|\omega|>R_{1}$ (ellipticity).

  3. 3.

For all $\varphi\in{\mathcal{S}}(\mathbb{R}^{d})$, ${\rm
L}^{\ast}\\{\varphi\\}\in L_{1,n_{0}}(\mathbb{R}^{d})$.

The connection between Condition 1 and the null space of ${\rm L}$ will be
explained in Section 2.3. Conditions 1 and 2 with $\gamma_{1}>1$ ensure that
$\rho_{\rm rad}=\mathcal{F}^{-1}\\{1/\widehat{L}_{\rm rad}\\}$, which is the
generalized inverse Fourier transform of the distribution $1/\widehat{L}_{\rm
rad}$, is identifiable as a continuous function $\mathbb{R}\to\mathbb{R}$. The
order $\gamma_{1}$ actually controls the degree of differentiability (Sobolev
smoothness) of $\rho_{\rm rad}$. Condition 3 is a mild technical constraint on
the decay of ${\rm L}^{\ast}\varphi$; this constraint has not appeared to be a
practical limitation so far. For instance, if ${\rm L}$ is an ordinary
differential operator (an arbitrary polynomial of the Laplace operator
$\Delta$) then ${\rm L}^{\ast}\\{\varphi\\}\in{\mathcal{S}}(\mathbb{R}^{d})$,
which is included in $L_{1,m}(\mathbb{R}^{d})$ for any $m\in\mathbb{Z}$. We
use this third condition for the handling of fractional operators whose
impulse response decays slowly.

An attractive class of admissible operators with
$\gamma_{0}=\gamma_{1}=\alpha$ and $n_{0}=\lceil\alpha-1\rceil$ are the
fractional Laplacians $(-\Delta)^{\tfrac{\alpha}{2}}$ with
$\alpha\in(1,\infty)$ whose frequency response is
$\|{\boldsymbol{\omega}}\|^{\alpha}$. The inverse of the fractional Laplacian
of order $\alpha$, which corresponds to a frequency-domain multiplication by
$\|{\boldsymbol{\omega}}\|^{-\alpha}$, is denoted by
$(-\Delta)^{-\tfrac{\alpha}{2}}$. Both operators are part of the same family
(isotropic LSI and scale-invariant), their distributional impulse response
being given by

$\displaystyle
k_{\alpha,d}({\boldsymbol{x}})=\mathcal{F}^{-1}\left\\{\frac{1}{\|{\boldsymbol{\omega}}\|^{\alpha}}\right\\}({\boldsymbol{x}})=\begin{cases}A_{\alpha,d}\;\|{\boldsymbol{x}}\|^{\alpha-d},&\alpha-d,-\alpha\notin
2\mathbb{N}\\\
B_{n,d}\;\|{\boldsymbol{x}}\|^{2n}\log(\|{\boldsymbol{x}}\|),&\alpha-d=2n\in
2\mathbb{N}\\\
(-\Delta)^{n}\\{\delta\\},&-\alpha/2=n\in\mathbb{N},\end{cases}$ (6)

with constants
$A_{d,\alpha}=\frac{\Gamma\big{(}\tfrac{d-\alpha}{2}\big{)}}{2^{\alpha}\pi^{d/2}\Gamma\big{(}\tfrac{\alpha}{2}\big{)}}$
and
$B_{d,n}=\frac{(-1)^{1+n}}{2^{2n+d-1}\pi^{d/2}\Gamma\big{(}n+\tfrac{d}{2}\big{)}n!}$
[19, 44]. The kernel $k_{\alpha,d}$ can also be interpreted as the Green’s
function of $(-\Delta)^{\tfrac{\alpha}{2}}$, with the corresponding radial
profile in Definition 2 being $\rho_{\rm rad}(t)=k_{\alpha,1}(t)$. In view of
(6), this means that $(-\Delta)^{\tfrac{\alpha}{2}}$ is admissible for
$\alpha>1$.

We note that the impulse response of the filtering operator ${\mathrm{K}}$ in
Theorem 1 is proportional to $k_{-d+1,d}({\boldsymbol{x}})$, which tells us
that it decays asymptotically like $1/\|{\boldsymbol{x}}\|^{2d-1}$ when $d$ is
even, or is a power of the Laplacian (local operator) otherwise. Functionally,
this means that
${\mathrm{K}}\big{(}{\mathcal{S}}(\mathbb{R}^{d})\big{)}={\mathcal{S}}(\mathbb{R}^{d})$
for even dimensions, and
${\mathrm{K}}\big{(}{\mathcal{S}}(\mathbb{R}^{d})\big{)}\subset
L_{1,d-1-\epsilon}(\mathbb{R}^{d})$ otherwise. Likewise, the impulse response
of the fractional Laplacians (of non-even order) decays asymptotically like
$1/\|{\boldsymbol{x}}\|^{\alpha+d}$, which implies that
$(-\Delta)^{\tfrac{\alpha}{2}}\big{(}{\mathcal{S}}(\mathbb{R}^{d})\big{)}\subset
L_{1,\alpha-\epsilon}(\mathbb{R}^{d})$ for arbitrarily small $\epsilon>0$, so
that the third condition in Definition 2 is met.

###  2.3 Nontrivial Null Space and Related Projectors

A sufficient condition for an LSI operator ${\rm L}$ to annihilate the
polynomials of degree $n_{0}$ is [56, p. 131]

$\displaystyle\partial^{{\boldsymbol{k}}}\widehat{L}({\boldsymbol{0}})=0,\mbox{
for all }{\boldsymbol{k}}\in\mathbb{N}^{d}\mbox{ with }|{\boldsymbol{k}}|\leq
n_{0}.$ (7)

We also recall that the directional derivative of a function along the
direction ${\boldsymbol{\xi}}\in\mathbb{S}^{d-1}$ (i.e.,
${\boldsymbol{\xi}}\in\mathbb{R}^{d}$ with $\|{\boldsymbol{\xi}}\|=1$) is
given by

$\displaystyle{\rm
D}_{{\boldsymbol{\xi}}}f={\boldsymbol{\xi}}^{\mathsf{T}}{\boldsymbol{\nabla}}f=\xi_{1}\partial^{{\boldsymbol{e}}_{1}}f+\cdots+\xi_{d}\partial^{{\boldsymbol{e}}_{d}}f.$
(8)

The operator ${\rm D}_{{\boldsymbol{\xi}}}$ is LSI with frequency response
$\widehat{{\rm
D}_{{\boldsymbol{\xi}}}}({\boldsymbol{\omega}})=(\mathrm{i}{\boldsymbol{\xi}}^{\mathsf{T}}{\boldsymbol{\omega}})$.
The $n$th iterate of ${\rm D}_{{\boldsymbol{\xi}}}$ yields the $n$th
derivative along ${\boldsymbol{\xi}}$ whose explicit expression in terms of
partial derivatives is

$\displaystyle{\rm
D}^{n}_{{\boldsymbol{\xi}}}f({\boldsymbol{x}})=\mathcal{F}^{-1}\\{(\mathrm{i}{\boldsymbol{\xi}}^{\mathsf{T}}{\boldsymbol{\omega}})^{n}\hat{f}({\boldsymbol{\omega}})\\}({\boldsymbol{x}})=\sum_{|{\boldsymbol{k}}|=n}\frac{n!}{{\boldsymbol{k}}!~}{\boldsymbol{\xi}}^{{\boldsymbol{k}}}\partial^{\boldsymbol{k}}f({\boldsymbol{x}}),$
(9)

where the right-hand side follows from the application of the multinomial
expansion to
$(\mathrm{i}{\boldsymbol{\xi}}^{\mathsf{T}}{\boldsymbol{\omega}})^{n}=(\xi_{1}\mathrm{i}\omega_{1}+\dots+\xi_{d}\mathrm{i}\omega_{d})^{n}$.

For isotropic operators, the directional derivatives
${\mathrm{D}}^{n}_{{\boldsymbol{\xi}}}\widehat{L}({\boldsymbol{0}})$ do not
dependent on the direction ${\boldsymbol{\xi}}$ and coincide with the radial
derivatives $\widehat{L}^{(n)}_{\rm rad}(0)$. In view of (9) and the
identification $\partial^{{\boldsymbol{k}}}\widehat{L}({\boldsymbol{0}})={\rm
D}^{k_{1}}_{{\boldsymbol{e}}_{1}}\cdots{\rm
D}^{k_{d}}_{{\boldsymbol{e}}_{d}}\widehat{L}({\boldsymbol{0}})$, (7) then has
the radial equivalent

$\displaystyle\widehat{L}^{(n)}_{\rm rad}(0)=\frac{{\rm d}^{n}\widehat{L}_{\rm
rad}(0)}{{\rm d}\omega^{n}}=0,\mbox{ for }n=0,1,\dots,n_{0},$ (10)

which is much simpler to test. It follows that an operator whose radial
frequency profile is such that $|\widehat{L}_{\rm
rad}(\omega)|/|\omega|^{\gamma_{0}}=C_{0}$ as $\omega\to 0$ will annihilate
all polynomials up to degree $n_{0}=\lceil\gamma_{0}-1\rceil$.

The null space of a spline-admissible operator ${\rm L}$ of order $\gamma_{0}$
therefore consists of the polynomials of degree $n_{0}=(\gamma_{0}-1)$ when
$\gamma_{0}$ is an integer and $n_{0}=\lfloor\gamma_{0}\rfloor$ otherwise when
$\gamma_{0}\notin\mathbb{N}$. We shall represent these polynomials by
expanding them in the monomial/Taylor basis

$\displaystyle
m_{\boldsymbol{k}}({\boldsymbol{x}})=\frac{{\boldsymbol{x}}^{{\boldsymbol{k}}}}{{\boldsymbol{k}}!}$
(11)

with $|{\boldsymbol{k}}|\leq n_{0}$. We also add a topological structure by
equipping the space with the $\ell_{2}$ norm of the Taylor coefficients, which
results in the description

$\displaystyle{\mathcal{P}}_{n_{0}}=\big{\\{}p_{0}=\sum_{|{\boldsymbol{k}}|\leq
n_{0}}b_{\boldsymbol{k}}m_{{\boldsymbol{k}}}:\|p_{0}\|_{{\mathcal{P}}}<\infty\big{\\}}\mbox{
with
}\|p_{0}\|_{{\mathcal{P}}}\stackrel{{\scriptstyle\vartriangle}}{{=}}\|(b_{\boldsymbol{k}})_{|{\boldsymbol{k}}|\leq
n_{0}}\|_{2}.$ (12)

To avoid a notational overload, we shall often denote this null space by
${\mathcal{P}}$, with the convention that
${\mathcal{P}}={\mathcal{P}}_{n_{0}}=\\{0\\}$ when
$n_{0}=\lceil\gamma_{0}-1\rceil<0$ (for the operators ${\rm L}$ whose null
space is trivial). The important point here is that (12) specifies a finite-
dimensional Banach subspace of ${\mathcal{S}}^{\prime}(\mathbb{R}^{d})$. Its
continuous dual ${\mathcal{P}}^{\prime}$ is finite-dimensional as well,
although it is composed of “abstract” elements
$p^{\ast}_{0}\in{\mathcal{P}}^{\prime}$ that are, in fact, equivalence classes
in ${\mathcal{S}}^{\prime}(\mathbb{R}^{d})$. Yet, it is possible to identify
every dual element $p^{\ast}_{0}\in{\mathcal{P}}^{\prime}$ as a true function
by selecting a particular dual basis
$\\{m_{{\boldsymbol{k}}}^{\ast}\\}_{|{\boldsymbol{k}}|\leq n_{0}}$ such that
$\langle
m^{\ast}_{{\boldsymbol{k}}},m_{{\boldsymbol{k}}^{\prime}}\rangle=\delta_{{\boldsymbol{k}}-{\boldsymbol{k}}^{\prime}}$
(Kroneker delta). Our specific choice is

$\displaystyle
m^{\ast}_{\boldsymbol{k}}=(-1)^{|{\boldsymbol{k}}|}\partial^{{\boldsymbol{k}}}\kappa_{\rm
iso}\in{\mathcal{S}}(\mathbb{R}^{d})$ (13)

with ${\boldsymbol{k}}\in\mathbb{N}^{d}$, where $\kappa_{\rm iso}$ is the
isotropic function described in Lemma 1.

######  Lemma 1 (adapted from [52]).

There exists an isotropic window $\kappa_{\rm
iso}\in{\mathcal{S}}(\mathbb{R}^{d})$ such that

$\displaystyle\langle
m_{{\boldsymbol{k}}},(-1)^{|{\boldsymbol{n}}|}\partial^{{\boldsymbol{n}}}\kappa_{\rm
iso}\rangle=\delta_{{\boldsymbol{k}}-{\boldsymbol{n}}}$ (14)

for all ${\boldsymbol{k}},{\boldsymbol{n}}\in\mathbb{N}^{d}$, subject to the
spectral constraints $\widehat{\kappa}_{\rm iso}({\boldsymbol{\omega}})=1$ for
$\|{\boldsymbol{\omega}}\|<\frac{1}{2}$, $1\geq\widehat{\kappa}_{\rm
iso}({\boldsymbol{\omega}})\geq 0$ for
$\frac{1}{2}<\|{\boldsymbol{\omega}}\|<1$, and $\widehat{\kappa}_{\rm
iso}({\boldsymbol{\omega}})=0$ for $\|{\boldsymbol{\omega}}\|\geq 1$.

This allows us to describe the dual space explicitly as

$\displaystyle{\mathcal{P}}^{\prime}={\mathcal{P}}^{\prime}_{n_{0}}=\big{\\{}p_{0}^{\ast}=\sum_{|{\boldsymbol{k}}|\leq
n_{0}}b^{\ast}_{\boldsymbol{k}}m^{\ast}_{{\boldsymbol{k}}}:\|p_{0}^{\ast}\|_{{\mathcal{P}}^{\prime}}<\infty\big{\\}}\mbox{
with
}\|p_{0}^{\ast}\|_{{\mathcal{P}}^{\prime}}\stackrel{{\scriptstyle\vartriangle}}{{=}}\|(b^{\ast}_{\boldsymbol{k}})\|_{2}$
(15)

where each elements $p_{0}^{\ast}$ has a unique representation in terms of its
coefficients $(b^{\ast}_{\boldsymbol{k}})_{|{\boldsymbol{k}}|\leq n_{0}}$. We
use the dual basis $\\{m_{\boldsymbol{k}}^{\ast}\\}$ to specify the projection
operator
$\mathrm{Proj}_{{\mathcal{P}}}:{\mathcal{S}}^{\prime}(\mathbb{R}^{d})\to{\mathcal{P}}_{n_{0}}$
as

$\displaystyle\mathrm{Proj}_{{\mathcal{P}}}\\{f\\}$
$\displaystyle=\sum_{|{\boldsymbol{k}}|\leq n_{0}}\langle
f,m_{\boldsymbol{k}}^{\ast}\rangle\;m_{\boldsymbol{k}},$ (16)

which is well-defined for any $f\in{\mathcal{S}}^{\prime}(\mathbb{R}^{d})$
since $m_{\boldsymbol{k}}^{\ast}\in{\mathcal{S}}(\mathbb{R}^{d})$. The
“transpose” of this operator is

$\displaystyle\mathrm{Proj}_{{\mathcal{P}}^{\prime}}\\{\nu\\}$
$\displaystyle=\sum_{|{\boldsymbol{k}}|\leq n_{0}}\langle
m_{\boldsymbol{k}},\nu\rangle\;m^{\ast}_{\boldsymbol{k}},$ (17)

which returns the projection of $\nu$ onto
${\mathcal{P}}^{\prime}_{n_{0}}\subseteq{\mathcal{S}}(\mathbb{R}^{d})$ under
the implicit assumption that $\nu$ has sufficient decay for $\nu\mapsto\langle
m_{\boldsymbol{k}},\nu\rangle$ to be well-defined—for instance, $\nu\in
L_{1,n_{0}}(\mathbb{R}^{d})$. Correspondingly, we also have that
$\mathrm{Proj}_{{\mathcal{P}}^{\prime}}\\{{\rm L}^{\ast}\varphi\\}=0$ for all
$\varphi$ such that ${\rm L}^{\ast}\varphi\in L_{1,n_{0}}(\mathbb{R}^{d})$
since $\langle m_{\boldsymbol{k}},{\rm L}^{\ast}\varphi\rangle=\langle{\rm
L}m_{\boldsymbol{k}},\varphi\rangle=0$ for $|{\boldsymbol{k}}|\leq n_{0}$. The
latter manipulation of the duality product is legitimate in reason of the
inclusion ${\mathcal{P}}_{n_{0}}\subset
L_{\infty,-n_{0}}(\mathbb{R}^{d})=\big{(}L_{1,n_{0}}(\mathbb{R}^{d})\big{)}^{\prime}$.

Even though the null space of an admissible operator ${\rm L}$ may be non-
trivial, its intersection with ${\mathcal{S}}(\mathbb{R}^{d})$ is always
$\\{0\\}$. This implies that ${\rm L}^{\ast}={\rm L}$ is injective on
${\mathcal{S}}(\mathbb{R}^{d})$ with ${\rm L}^{-1\ast}{\rm
L}^{\ast}\varphi=\varphi$ for all $\varphi\in{\mathcal{S}}(\mathbb{R}^{d})$
where ${\rm L}^{\ast-1}={\rm L}^{-1}$ is the LSI operator whose frequency
response is $1/|\widehat{L}|$.

##  3 Radon Transform

The Radon transform extracts the integrals of a function on $\mathbb{R}^{d}$
over all hyperplanes of dimension $(d-1)$. These hyperplanes are indexed over
$\mathbb{R}\times\mathbb{S}^{d-1}$, where
$\mathbb{S}^{d-1}=\\{{\boldsymbol{\xi}}\in\mathbb{R}^{d}:\|{\boldsymbol{\xi}}\|_{2}=1\\}$
is the unit sphere in $\mathbb{R}^{d}$. The coordinates of a hyperplane
associated with an offset $t\in\mathbb{R}$ and a normal vector
${\boldsymbol{\xi}}\in\mathbb{S}^{d-1}$ satisfy

${\boldsymbol{\xi}}^{\mathsf{T}}{\boldsymbol{x}}=\xi_{1}x_{1}+\dots+\xi_{d}x_{d}=t.$

Here, we first review the classical theory of the Radon transform [26],
starting with the case of test functions (Section 3.1), and extending it by
duality to tempered distributions (Section 3.2). Then, in Section 3.3, we
specify the Radon transform and its inverse on an appropriate class of
intermediate Banach spaces ${\mathcal{Y}}$ with
${\mathcal{S}}(\mathbb{R}^{d})\xhookrightarrow{\mbox{\tiny\rm
d.}}{\mathcal{Y}}\xhookrightarrow{\mbox{\tiny\rm
d.}}{\mathcal{S}}^{\prime}(\mathbb{R}^{d})$ (Theorem 3). Finally, in Section
3.4, we provide the (filtered) Radon transforms of the dictionary
elements—isotropic kernels and ridges—that are relevant to our investigation.

###  3.1 Classical Integral Formulation

The Radon transform of the function $f\in L_{1}(\mathbb{R}^{d})$ is defined as

$\displaystyle{\mathrm{R}}\\{f\\}(t,{\boldsymbol{\xi}})$
$\displaystyle=\int_{\mathbb{R}^{d}}\delta(t-{\boldsymbol{\xi}}^{\mathsf{T}}{\boldsymbol{x}})f({\boldsymbol{x}}){\rm
d}{\boldsymbol{x}},\quad(t,{\boldsymbol{\xi}})\in\mathbb{R}\times\mathbb{S}^{d-1}.$
(18)

The adjoint of ${\mathrm{R}}$ is the backprojection operator
${\mathrm{R}}^{\ast}$. Its action on
$g:\mathbb{R}\times\mathbb{S}^{d-1}\to\mathbb{R}$ yields the function

$\displaystyle{\mathrm{R}}^{\ast}\\{g\\}({\boldsymbol{x}})=\int_{\mathbb{S}^{d-1}}g(\underbrace{{\boldsymbol{\xi}}^{\mathsf{T}}{\boldsymbol{x}}}_{t},{\boldsymbol{\xi}}){\rm
d}{\boldsymbol{\xi}},\quad{\boldsymbol{x}}\in\mathbb{R}^{d}.$ (19)

Given the $d$-dimensional Fourier transform $\hat{f}$ of $f\in
L_{1}(\mathbb{R}^{d})$, one can calculate
${\mathrm{R}}\\{f\\}(\cdot,{\boldsymbol{\xi}}_{0})$ for any fixed
${\boldsymbol{\xi}}_{0}\in\mathbb{S}^{d-1}$ through the relation

$\displaystyle{\mathrm{R}}\\{f\\}(t,{\boldsymbol{\xi}}_{0})=\frac{1}{2\pi}\int_{\mathbb{R}}\hat{f}(\omega{\boldsymbol{\xi}}_{0})\mathrm{e}^{\mathrm{i}\omega
t}{\rm
d}\omega=\mathcal{F}^{-1}\\{\hat{f}(\cdot{\boldsymbol{\xi}}_{0})\\}\\{t\\},$
(20)

In other words, the restriction of $\hat{f}:\mathbb{R}^{d}\to\mathbb{C}$ along
the ray
$\\{{\boldsymbol{\omega}}=\omega{\boldsymbol{\xi}}_{0}:\omega\in\mathbb{R}\\}$
coincides with the 1D Fourier transform of
${\mathrm{R}}\\{f\\}(\cdot,{\boldsymbol{\xi}}_{0})$, a property that is
referred to as the Fourier-slice theorem.

To describe the functional properties of the Radon transform, one needs the
(hyper)spherical (or Radon-domain) counterparts of the spaces described in
Section 2.1. There, the Euclidean indexing with
${\boldsymbol{x}}\in\mathbb{R}^{d}$ must be replaced by
$(t,{\boldsymbol{\xi}})\in\mathbb{R}\times\mathbb{S}^{d-1}$. The spherical
counterpart of ${\mathcal{S}}(\mathbb{R}^{d})$ is
${\mathcal{S}}(\mathbb{R}\times\mathbb{S}^{d-1})$. Correspondingly, an element
$g\in{\mathcal{S}}^{\prime}(\mathbb{R}\times\mathbb{S}^{d-1})$ is a continuous
linear functional on ${\mathcal{S}}(\mathbb{R}\times\mathbb{S}^{d-1})$ whose
action on the test function $\phi$ is represented by the duality product
$g:\phi\mapsto\langle g,\phi\rangle_{\rm Rad}$. When $g$ can be identified as
an ordinary function $g:(t,{\boldsymbol{\xi}})\mapsto
g(t,{\boldsymbol{\xi}})\in\mathbb{R}$, one can write that

$\displaystyle\langle g,\phi\rangle_{\rm
Rad}=\int_{\mathbb{S}^{d-1}}\int_{\mathbb{R}}g(t,{\boldsymbol{\xi}})\phi(t,{\boldsymbol{\xi}}){\rm
d}t{\rm d}{\boldsymbol{\xi}}$ (21)

where ${\rm d}{\boldsymbol{\xi}}$ stands for the surface element on the unit
sphere $\mathbb{S}^{d-1}$.

The key property for analysis is that the Radon transform is continuous on
${\mathcal{S}}$ and invertible [26, 23, 41].

######  Theorem 1 (Continuity and invertibility of the Radon transform on
${\mathcal{S}}(\mathbb{R}^{d})$).

The Radon operator ${\mathrm{R}}$ continuously maps
${\mathcal{S}}(\mathbb{R}^{d})\to{\mathcal{S}}(\mathbb{R}\times\mathbb{S}^{d-1})$.
Moreover, ${\mathrm{R}}^{\ast}{\mathrm{K}}_{\rm
rad}{\mathrm{R}}={\mathrm{K}}{\mathrm{R}}^{\ast}{\mathrm{R}}={\mathrm{R}}^{\ast}{\mathrm{R}}{\mathrm{K}}=\mathrm{Id}\mbox{
on~}{\mathcal{S}}(\mathbb{R}^{d})$, where
${\mathrm{K}}=({\mathrm{R}}^{\ast}{\mathrm{R}})^{-1}=c_{d}(-\Delta)^{\tfrac{d-1}{2}}$
with $c_{d}=\frac{1}{2(2\pi)^{d-1}}$ is the so-called “filtering” operator,
and ${\mathrm{K}}_{\rm rad}$ its one-dimensional radial counterpart that acts
along the Radon-domain variable $t$. These operators are characterized by
their frequency response
$\widehat{K}({\boldsymbol{\omega}})=c_{d}\|{\boldsymbol{\omega}}\|^{d-1}$ and
$\widehat{K}_{\rm rad}(\omega)=c_{d}|\omega|^{d-1}$, respectively.

It should be noted, however, that the operator
${\mathrm{R}}:{\mathcal{S}}(\mathbb{R}^{d})\to{\mathcal{S}}(\mathbb{R}\times\mathbb{S}^{d-1})$
is not surjective, which means that not very hyper-spherical test function
$\phi\in{\mathcal{S}}(\mathbb{R}\times\mathbb{S}^{d-1})$ can be written as
$\phi={\mathrm{R}}\\{\varphi\\}$ with
$\varphi\in{\mathcal{S}}(\mathbb{R}^{d})$. A necessary condition is that
$\phi$ be even, but this is not sufficient [20, 26, 23]. The good news,
however, is that the range of ${\mathrm{R}}$ on
${\mathcal{S}}(\mathbb{R}^{d})$ is closed in ${\mathcal{S}}_{\rm
Rad}={\mathrm{R}}\big{(}{\mathcal{S}}(\mathbb{R}^{d})\big{)}$ equipped with
the nuclear topology of ${\mathcal{S}}(\mathbb{R}\times\mathbb{S}^{d-1})$ [23,
p. 60]. Since the underlying spaces are both Fréchet, this translates into the
transform $\varphi\mapsto{\mathrm{R}}\\{\varphi\\}$ being a homeomorphism of
${\mathcal{S}}(\mathbb{R}^{d})$ onto ${\mathcal{S}}_{\rm Rad}$.

######  Corollary 1.

The operator ${\mathrm{R}}:{\mathcal{S}}(\mathbb{R}^{d})\to{\mathcal{S}}_{\rm
Rad}$ is a continuous bijection, with a continuous inverse given by
${\mathrm{R}}^{-1}=({\mathrm{R}}^{\ast}{\mathrm{K}}_{\rm
rad}):{\mathcal{S}}_{\rm Rad}\to{\mathcal{S}}(\mathbb{R}^{d})$.

###  3.2 Distributional Extension

To extend the framework to distributions, one proceeds by duality. By invoking
the property that ${\mathrm{R}}^{\ast}{\mathrm{K}}_{\rm
rad}{\mathrm{R}}=\mathrm{Id}$ on ${\mathcal{S}}(\mathbb{R}^{d})$, we make the
manipulation

$\displaystyle\forall\varphi\in{\mathcal{S}}(\mathbb{R}^{d})\quad\langle
f,\varphi\rangle$ $\displaystyle=\langle
f,{\mathrm{R}}^{\ast}{\mathrm{K}}_{\rm rad}{\mathrm{R}}\\{\varphi\\}\rangle$
$\displaystyle=\langle{\mathrm{R}}\\{f\\},{\mathrm{K}}_{\rm
rad}{\mathrm{R}}\\{\varphi\\}\rangle_{\rm
Rad}=\langle{\mathrm{R}}\\{f\\},\phi\rangle_{\rm Rad},$ (22)

with $\phi={\mathrm{K}}_{\rm rad}{\mathrm{R}}\\{\varphi\\}\in{\mathrm{K}}_{\rm
rad}{\mathrm{R}}\big{(}{\mathcal{S}}(\mathbb{R}^{d})\big{)}\subset{\mathcal{S}}(\mathbb{R}\times\mathbb{S}^{d-1})$
and $\varphi={\mathrm{R}}^{\ast}\\{\phi\\}$. Relation (22), which is valid in
the classical sense for $f\in L_{1}(\mathbb{R}^{d})$, is then used as
definition to extend the scope of ${\mathrm{R}}$ for
$f\in{\mathcal{S}}^{\prime}(\mathbb{R}^{d})$.

######  Definition 3.

The distribution $g={\mathrm{R}}\\{f\\}\in\big{(}{\mathrm{K}}_{\rm
rad}{\mathrm{R}}\big{(}{\mathcal{S}})\big{)}^{\prime}$ is the (formal) Radon
transform of $f\in{\mathcal{S}}^{\prime}(\mathbb{R}^{d})$ if

$\displaystyle\forall\phi\in{\mathrm{K}}_{\rm
rad}{\mathrm{R}}\big{(}{\mathcal{S}}(\mathbb{R}^{d})\big{)}:\quad\langle
g,\phi\rangle_{\rm Rad}=\langle f,{\mathrm{R}}^{\ast}\\{\phi\\}\rangle.$ (23)

Likewise, $\tilde{g}={\mathrm{K}}_{\rm
rad}{\mathrm{R}}\\{f\\}\in{\mathcal{S}}_{\rm Rad}^{\prime}$ is the (formal)
filtered projection of $f\in{\mathcal{S}}^{\prime}(\mathbb{R}^{d})$ if

$\displaystyle\forall\phi\in{\mathcal{S}}_{\rm
Rad}:\quad\langle\tilde{g},\phi\rangle_{\rm Rad}=\langle
f,{\mathrm{R}}^{\ast}{\mathrm{K}}_{\rm rad}\\{\phi\\}\rangle.$ (24)

Finally,
$f={\mathrm{R}}^{\ast}\\{g\\}\in{\mathcal{S}}^{\prime}(\mathbb{R}^{d})$ is the
backprojection of
$g\in{\mathcal{S}}^{\prime}(\mathbb{R}\times\mathbb{S}^{d-1})$ if

$\displaystyle\forall\varphi\in{\mathcal{S}}(\mathbb{R}^{d}):\quad\langle{\mathrm{R}}^{\ast}\\{g\\},\varphi\rangle=\langle
g,{\mathrm{R}}\\{\varphi\\}\rangle_{\rm Rad}.$ (25)

While (25) identifies ${\mathrm{R}}^{\ast}\\{g\\}$ as a single, unique
distribution in ${\mathcal{S}}^{\prime}(\mathbb{R}^{d})$, this is not so for
(24) (resp., (23)), as the members of ${\mathcal{S}}^{\prime}_{\rm Rad}$
(resp., of $\big{(}{\mathrm{K}}_{\rm
rad}{\mathrm{R}}\big{(}{\mathcal{S}})\big{)}^{\prime}$) are equivalence
classes in ${\mathcal{S}}^{\prime}(\mathbb{R}\times\mathbb{S}^{d-1})$. To make
this explicit, we take advantage of the equivalence
${\mathrm{R}}^{\ast}\\{g\\}=0\Leftrightarrow\langle g,\phi\rangle_{\rm Rad}=0$
to identity the null space of the backprojection operator as being

$\displaystyle{\mathcal{N}}_{{\mathrm{R}}^{\ast}}=\\{g\in{\mathcal{S}}^{\prime}(\mathbb{R}\times\mathbb{S}^{d-1}):\langle
g,\phi\rangle_{\rm Rad}=0,\forall\phi\in{\mathcal{S}}_{\rm Rad}\\},$ (26)

which is a closed subspace of
${\mathcal{S}}^{\prime}(\mathbb{R}\times\mathbb{S}^{d-1})$. It is then
possible to describe ${\mathcal{S}}^{\prime}_{\rm Rad}$ as the abstract
quotient space
${\mathcal{S}}^{\prime}(\mathbb{R}\times\mathbb{S}^{d-1})/{\mathcal{N}}_{{\mathrm{R}}^{\ast}}$.
In other words, if we find a hyper-spherical distribution
$g_{0}\in{\mathcal{S}}^{\prime}(\mathbb{R}\times\mathbb{S}^{d-1})$ such that
(24) is met for a given $f\in{\mathcal{S}}^{\prime}(\mathbb{R}^{d})$, then,
strictly speaking, ${\mathrm{K}}_{\rm
rad}{\mathrm{R}}\\{f\\}\in{\mathcal{S}}^{\prime}_{\rm Rad}$ is the equivalence
class (or coset) given by

$\displaystyle{\mathrm{K}}_{\rm
rad}{\mathrm{R}}\\{f\\}=[g_{0}]=\\{g_{0}+h:h\in{\mathcal{N}}_{{\mathrm{R}}^{\ast}}\\}.$
(27)

Since $[g_{0}]=[g]$ for any $g\in{\mathrm{K}}_{\rm rad}{\mathrm{R}}\\{f\\}$,
we refer to the members of ${\mathrm{K}}_{\rm rad}{\mathrm{R}}\\{f\\}$ as
“formal” filtered projections of $f$ to remind us of this lack of unicity.

Based on those definitions, one obtains the classical result on the
invertibility of the (filtered) Radon transform on
${\mathcal{S}}^{\prime}(\mathbb{R}^{d})$ [26], which is the dual of Corollary
1.

######  Theorem 2 (Invertibility of the Radon transform on
${\mathcal{S}}^{\prime}(\mathbb{R}^{d})$).

It holds that ${\mathrm{R}}^{\ast}{\mathrm{K}}_{\rm
rad}{\mathrm{R}}=\mathrm{Id}$ on ${\mathcal{S}}^{\prime}(\mathbb{R}^{d})$.
More precisely, the filtered Radon transform ${\mathrm{K}}_{\rm
rad}{\mathrm{R}}:{\mathcal{S}}^{\prime}(\mathbb{R}^{d})\to{\mathcal{S}}^{\prime}_{\rm
Rad}$ is a continuous bijection, with a continuous inverse given by
$({\mathrm{K}}_{\rm
rad}{\mathrm{R}})^{-1}={\mathrm{R}}^{\ast}:{\mathcal{S}}^{\prime}_{\rm
Rad}\to{\mathcal{S}}^{\prime}(\mathbb{R}^{d})$.

To illustrate the fact that (24) does not identify a single distribution, we
consider the Dirac ridge
$\delta({\boldsymbol{\xi}}_{0}{\boldsymbol{x}}-t_{0})\in{\mathcal{S}}^{\prime}(\mathbb{R}^{d})$
and refer to the definition (18) of the Radon transform to deduce that, for
all $\phi={\mathrm{R}}\\{\varphi\\}\in{\mathcal{S}}_{\rm Rad}$ with
$\varphi\in{\mathcal{S}}(\mathbb{R}^{d})$,

$\displaystyle\langle\delta({\boldsymbol{\xi}}_{0}^{\mathsf{T}}\cdot-
t_{0}),{\mathrm{R}}^{\ast}{\mathrm{K}}_{\rm rad}\\{\phi\\}\rangle$
$\displaystyle=\langle\delta({\boldsymbol{\xi}}_{0}^{\mathsf{T}}\cdot-
t_{0}),\overbrace{{\mathrm{R}}^{\ast}{\mathrm{K}}_{\rm
rad}{\mathrm{R}}}^{\mathrm{Id}}\\{\varphi\\}\rangle$
$\displaystyle=\int_{\mathbb{R}^{d}}\delta({\boldsymbol{\xi}}_{0}^{\mathsf{T}}{\boldsymbol{x}}-t_{0})\varphi({\boldsymbol{x}}){\rm
d}{\boldsymbol{x}}={\mathrm{R}}\\{\varphi\\}(-t_{0},-{\boldsymbol{\xi}}_{0})$
$\displaystyle=\langle\delta\big{(}\cdot+(t_{0},{\boldsymbol{\xi}}_{0})\big{)},{\mathrm{R}}\\{\varphi\\}\rangle_{\rm
Rad}=\langle\delta\big{(}\cdot+(t_{0},{\boldsymbol{\xi}}_{0})\big{)},\phi\rangle_{\rm
Rad},$

which shows that $\delta\big{(}\cdot+{\boldsymbol{z}}_{0}\big{)}$ with
${\boldsymbol{z}}_{0}=(t_{0},{\boldsymbol{\xi}}_{0})$ is a formal filtered
projection of
$\delta({\boldsymbol{\xi}}_{0}^{\mathsf{T}}{\boldsymbol{x}}-t_{0})$. Moreover,
since
$\delta({\boldsymbol{\xi}}_{0}^{\mathsf{T}}{\boldsymbol{x}}-t_{0})=\delta(-{\boldsymbol{\xi}}_{0}^{\mathsf{T}}{\boldsymbol{x}}+t_{0})$,
the same holds true for $\delta(\cdot-{\boldsymbol{z}}_{0})$, as well as for
$\delta_{{\rm
Rad},{\boldsymbol{z}}_{0}}\stackrel{{\scriptstyle\vartriangle}}{{=}}\frac{1}{2}\big{(}\delta(\cdot-{\boldsymbol{z}}_{0})+\delta(\cdot+{\boldsymbol{z}}_{0})\big{)}$,
which has the advantage of being symmetric. While the general solution in
${\mathcal{S}}^{\prime}(\mathbb{R}\times\mathbb{S}^{d-1})$ is
${\mathrm{K}}_{\rm
rad}{\mathrm{R}}\\{\delta({\boldsymbol{\xi}}_{0}^{\mathsf{T}}\cdot-
t_{0})\\}=[\delta\big{(}\cdot\pm{\boldsymbol{z}}_{0}\big{)}]$, we shall see
that there also exists a way to specify a representer that is unique (i.e.,
$\delta_{{\rm Rad},{\boldsymbol{z}}_{0}})$ by restricting the range of
${\mathrm{K}}_{\rm rad}{\mathrm{R}}$ to a suitable space of measures.

The distributional extension of the Radon transform inherits most of the
properties of the “classical” operator defined in (18). Of special relevance
to us is the quasi-commutativity of ${\mathrm{R}}$ with convolution, also
known as the intertwining property. Specifically, let
$h,f\in{\mathcal{S}}^{\prime}(\mathbb{R}^{d})$ be two distributions whose
convolution $h\ast f$ is well-defined in
${\mathcal{S}}^{\prime}(\mathbb{R}^{d})$. Then,

$\displaystyle{\mathrm{R}}\\{h\ast
f\\}={\mathrm{R}}\\{h\\}\circledast{\mathrm{R}}\\{f\\}$ (28)

where the symbol “$\circledast$” denotes the 1D convolution along the radial
variable $t\in\mathbb{R}$ with $(u\circledast g)(t,{\boldsymbol{\xi}})=\langle
u(\cdot,{\boldsymbol{\xi}}),g(t-\cdot,{\boldsymbol{\xi}})\rangle$. In
particular, when $h={\rm L}\\{\delta\\}$ is the (isotropic) impulse response
of an LSI operator whose frequency response
$\widehat{L}({\boldsymbol{\omega}})=\widehat{L}_{\rm
rad}(\|{\boldsymbol{\omega}}\|)$ is purely radial, we get that

$\displaystyle{\mathrm{R}}\\{h\ast f\\}={\mathrm{R}}{\rm L}\\{f\\}={\rm
L}_{\rm rad}{\mathrm{R}}\\{f\\},$ (29)

where ${\rm L}_{\rm rad}$ is the radial convolution operator whose 1D
frequency response is $\widehat{L}_{\rm rad}$. Likewise, by duality, for
$g\in{\mathcal{S}}^{\prime}(\mathbb{R}\times\mathbb{S}^{d-1})$ we have that

$\displaystyle{\rm L}{\mathrm{R}}^{\ast}\\{g\\}={\mathrm{R}}^{\ast}{\rm
L}_{\rm rad}\\{g\\},$ (30)

under the implicit assumption that ${\rm L}\\{{\mathrm{R}}^{\ast}g\\}$ and
${\rm L}_{\rm rad}\\{g\\}$ are well-defined distributions. By taking
inspiration from Theorem 1, we can then use these relations for ${\rm
L}={\mathrm{K}}=({\mathrm{R}}^{\ast}{\mathrm{R}})^{-1}$ to show that
${\mathrm{R}}^{\ast}{\mathrm{K}}_{\rm
rad}{\mathrm{R}}\\{f\\}={\mathrm{R}}^{\ast}{\mathrm{R}}{\mathrm{K}}\\{f\\}={\mathrm{K}}{\mathrm{R}}^{\ast}{\mathrm{R}}\\{f\\}=f$
for a broad class of distributions. The first form is valid for all
$f\in{\mathcal{S}}^{\prime}(\mathbb{R}^{d})$ (Theorem 2), but there is a
slight restriction with the second form (resp., third form), which requires
that ${\mathrm{K}}\\{f\\}$ (resp., ${\mathrm{K}}\\{g\\}$ with
$g={\mathrm{R}}^{\ast}{\mathrm{R}}\\{f\\}\in{\mathcal{S}}^{\prime}(\mathbb{R}^{d})$)
be well-defined in ${\mathcal{S}}^{\prime}(\mathbb{R}^{d})$. While the latter
condition is always met when $d$ is odd, it may fail222For $d=2n$ even,
$\widehat{K}({\boldsymbol{\omega}})\propto\|{\boldsymbol{\omega}}\|^{2n-1}$
which is $C^{\infty}$ everywhere, except at the origin where it is only
$C^{2n-2}$. This means that ${\mathrm{K}}$ can properly handle (and
annihilate) polynomials only up to degree $2n-2$. in even dimensions with
distributions (e.g., polynomials) whose Fourier transform is singular at the
origin. The good news for our regularization framework is that these
problematic distributions are either excluded from the native space or
annihilated by ${\rm L}$, so that it is legitimate to write that ${\rm L}_{\rm
R}={\mathrm{K}}_{\rm rad}{\mathrm{R}}{\rm L}={\mathrm{R}}{\mathrm{K}}{\rm L}$,
where the second form has the advantage that ${\mathrm{K}}$ and ${\rm L}$ can
be pooled into a single augmented operator $({\mathrm{K}}{\mathrm{L}})$. An
alternative form is ${\rm L}_{\rm R}={\mathrm{Q}}_{\rm rad}{\mathrm{R}}$,
where ${\mathrm{Q}}_{\rm rad}={\mathrm{K}}_{\rm rad}{\mathrm{L}}_{\rm rad}$ is
the radial Radon-domain operator whose frequency response is $\widehat{Q}_{\rm
rad}(\omega)=c_{d}|\omega|^{d-1}\widehat{L}_{\rm rad}(\omega)$.

###  3.3 Radon-Compatible Banach Spaces

Our formulation requires the identification of Radon-domain Banach spaces over
which the backprojection operator ${\mathrm{R}}^{\ast}$ is invertible. This is
a non-trivial point because the extended operator
${\mathrm{R}}^{\ast}:{\mathcal{S}}^{\prime}(\mathbb{R}\times\mathbb{S}^{d-1})\to{\mathcal{S}}^{\prime}(\mathbb{R}^{d})$
in Definition 3 is not injective. In fact, it has the highly non-trivial null
space ${\rm ker}({\mathrm{R}}^{\ast})={\mathcal{S}}^{\perp}_{\rm Rad}$, which
is a superset of the odd Radon-domain distributions [20]. Yet,
${\mathrm{R}}^{\ast}$ is invertible on ${\mathcal{S}}^{\prime}_{\rm Rad}$ and
surjective on ${\mathcal{S}}^{\prime}(\mathbb{R}^{d})$ (Theorem 2).

To ensure invertibility, we therefore need to restrict ourselves to Banach
spaces that are embedded in ${\mathcal{S}}^{\prime}_{\rm Rad}$. To identify
such objects, we consider a generic Banach space
${\mathcal{X}}=({\mathcal{X}},\|\cdot\|_{{\mathcal{X}}})$ such that
${\mathcal{S}}(\mathbb{R}\times\mathbb{S}^{d-1})\xhookrightarrow{\mbox{\tiny\rm
d.}}{\mathcal{X}}\xhookrightarrow{\mbox{\tiny\rm
d.}}{\mathcal{S}}^{\prime}(\mathbb{R}\times\mathbb{S}^{d-1})$. This dense-
embedding hypothesis has several implications:

  1. 1.

The space ${\mathcal{X}}$ is the completion of
${\mathcal{S}}(\mathbb{R}\times\mathbb{S}^{d-1})$ in the
$\|\cdot\|_{{\mathcal{X}}}$ norm; i.e.,

$\displaystyle{\mathcal{X}}=\overline{\big{(}{\mathcal{S}}(\mathbb{R}\times\mathbb{S}^{d-1}),\|\cdot\|_{{\mathcal{X}}}\big{)}}.$
(31)

  2. 2.

The dual space
${\mathcal{X}}^{\prime}\xhookrightarrow{}{\mathcal{S}}^{\prime}(\mathbb{R}\times\mathbb{S}^{d-1})$
is equipped with the norm

$\displaystyle\|g\|_{{\mathcal{X}}^{\prime}}=\sup_{\phi\in{\mathcal{X}}:\;\|\phi\|_{{\mathcal{X}}}\leq
1}\langle
g,\phi\rangle=\sup_{\phi\in{\mathcal{S}}(\mathbb{R}\times\mathbb{S}^{d-1}):\;\|\phi\|_{{\mathcal{X}}}\leq
1}\langle g,\phi\rangle,$ (32)

where the restriction of
$\phi\in{\mathcal{S}}(\mathbb{R}\times\mathbb{S}^{d-1})$ on the right-hand
side of (32) is justified by the denseness of
${\mathcal{S}}(\mathbb{R}\times\mathbb{S}^{d-1})$ in ${\mathcal{X}}$.

  3. 3.

The definition of $\|g\|_{{\mathcal{X}}^{\prime}}$ given by the right-hand
side of (32) is valid for any distribution
$g\in{\mathcal{S}}^{\prime}(\mathbb{R}\times\mathbb{S}^{d-1})$ with
$\|g\|_{{\mathcal{X}}^{\prime}}=\infty$ for $g\notin{\mathcal{X}}^{\prime}$.
Accordingly, we can specify the topological dual of ${\mathcal{X}}^{\prime}$
as

$\displaystyle{\mathcal{X}}^{\prime}=\big{\\{}g\in{\mathcal{S}}^{\prime}(\mathbb{R}\times\mathbb{S}^{d-1}):\|g\|_{{\mathcal{X}}^{\prime}}<\infty\big{\\}}.$
(33)

Likewise, based on the pair $({\mathcal{S}}_{\rm
Rad},{\mathcal{S}}^{\prime}_{\rm Rad})$, we specify the Radon-compatible
Banach subspaces

$\displaystyle{\mathcal{X}}_{\rm Rad}$
$\displaystyle=\overline{({\mathcal{S}}_{\rm Rad},\|\cdot\|_{{\mathcal{X}}})}$
(34) $\displaystyle{\mathcal{X}}^{\prime}_{\rm Rad}$
$\displaystyle=\big{(}{\mathcal{X}}_{\rm
Rad}\big{)}^{\prime}=\big{\\{}g\in{\mathcal{S}}^{\prime}_{\rm
Rad}:\|g\|_{{\mathcal{X}}^{\prime}_{\rm Rad}}<\infty\big{\\}}$ (35)

where the underlying dual norms have a definition that is analogous to (32)
with ${\mathcal{S}}_{\rm Rad}$ and ${\mathcal{X}}_{\rm Rad}$ substituting for
${\mathcal{S}}(\mathbb{R}\times\mathbb{S}^{d-1})$ and ${\mathcal{X}}$.

######  Theorem 3 (adapted from [52]).

Let $({\mathcal{X}}_{\rm Rad},{\mathcal{X}}^{\prime}_{\rm Rad})$ be the dual
pair of spaces specified by (34) and (35). Then,

  1. 1.

the map $\ {\mathrm{R}}^{\ast}{\mathrm{K}}_{\rm rad}:{\mathcal{X}}_{\rm
Rad}\to{\mathcal{Y}}={\mathrm{R}}^{\ast}{\mathrm{K}}_{\rm
rad}\big{(}{\mathcal{X}}_{\rm Rad}\big{)}$ is an isometric bijection, with
${\mathrm{R}}{\mathrm{R}}^{\ast}{\mathrm{K}}_{\rm rad}=\mathrm{Id}$ on
${\mathcal{X}}_{\rm Rad}$;

  2. 2.

the map ${\mathrm{R}}^{\ast}:{\mathcal{X}}^{\prime}_{\rm
Rad}\to{\mathcal{Y}}^{\prime}={\mathrm{R}}^{\ast}\big{(}{\mathcal{X}}^{\prime}_{\rm
Rad}\big{)}$ is an isometric bijection, with ${\mathrm{K}}_{\rm
rad}{\mathrm{R}}{\mathrm{R}}^{\ast}=\mathrm{Id}$ on
${\mathcal{X}}^{\prime}_{\rm Rad}$.

Moreover, if there exists a complementary Banach space ${\mathcal{X}}^{\rm
c}_{\rm Rad}$ such that ${\mathcal{X}}={\mathcal{X}}_{\rm
Rad}\oplus{\mathcal{X}}^{\rm c}_{\rm Rad}$, then
${\mathcal{X}}^{\prime}={\mathcal{X}}^{\prime}_{\rm
Rad}\oplus({\mathcal{X}}^{\rm c}_{\rm Rad})^{\prime}$ where
$({\mathcal{X}}^{\rm c}_{\rm Rad})^{\prime}$ can be identified as the null
space of the backprojection operator
${\mathrm{R}}^{\ast}:{\mathcal{X}}^{\prime}\to{\mathcal{Y}}^{\prime}\xhookrightarrow{}{\mathcal{S}}(\mathbb{R}^{d})$.

The prototypical examples where those properties are met are
$({\mathcal{X}},{\mathcal{X}}^{\prime})=\big{(}L_{p}(\mathbb{R}\times\mathbb{S}^{d-1}),L_{q}(\mathbb{R}\times\mathbb{S}^{d-1})\big{)}$
with $p\in[1,\infty)$ and $q=p/(p-1)$ (conjugate exponent), as well as
$({\mathcal{X}},{\mathcal{X}}^{\prime})=\big{(}C_{0}(\mathbb{R}\times\mathbb{S}^{d-1}),{\mathcal{M}}(\mathbb{R}\times\mathbb{S}^{d-1})\big{)}$.
In fact, those hyper-spherical spaces have the convenient feature of admitting
a decomposition in their even and odd components.

######  Lemma 2.

Let ${\mathcal{Z}}=\mathbb{R}\times\mathbb{S}^{d-1}$. Then, for
${\mathcal{X}}=L_{p}({\mathcal{Z}})$ with $p\in[1,\infty)$ and
${\mathcal{X}}=C_{0}({\mathcal{Z}})$ for $p=\infty$, we have that
${\mathcal{X}}={\mathcal{X}}_{\rm Rad}\oplus{\mathcal{X}}^{\rm c}_{\rm Rad}$
where

$\displaystyle{\mathcal{X}}_{\rm Rad}$ $\displaystyle={\mathcal{X}}_{\rm
even}=\\{g\in{\mathcal{X}}:g({\boldsymbol{z}})=g(-{\boldsymbol{z}}),\forall{\boldsymbol{z}}\in{\mathcal{Z}}\\}={\mathcal{X}}(\mathbb{P}^{d})$
(36) $\displaystyle{\mathcal{X}}^{\rm c}_{\rm Rad}$
$\displaystyle={\mathcal{X}}_{\rm
odd}=\\{g\in{\mathcal{X}}:g({\boldsymbol{z}})=-g(-{\boldsymbol{z}}),\forall{\boldsymbol{z}}\in{\mathcal{Z}}\\}.$
(37)

###### Proof.

To establish this result directly is tricky because the characterization of
${\mathcal{S}}_{\rm Rad}$ involves some general moment conditions [20, 23,
26]. Instead, we consider the smaller space of even Radon-domain Lizorkin test
functions [25] described by

$\displaystyle{\mathcal{S}}_{\rm Liz,Rad}=\\{\phi\in{\mathcal{S}}_{\rm
even}({\mathcal{Z}}):\int_{\mathbb{R}}t^{k}\phi(t,{\boldsymbol{\xi}}){\rm
d}t=0,\forall{\boldsymbol{\xi}}\in\mathbb{S}^{d-1},k\in\mathbb{N}\\},$ (38)

which is such that ${\mathcal{S}}_{\rm Liz,Rad}\subset{\mathcal{S}}_{\rm
Rad}\subset{\mathcal{S}}_{\rm even}({\mathcal{Z}})$. We then invoke a general
result by Samko [43] that implies that $\overline{({\mathcal{S}}_{\rm
Liz,Rad},\|\cdot\|_{L_{p}})}=L_{p,{\rm
even}}({\mathcal{Z}})\supset{({\mathcal{S}}_{\rm Rad},\|\cdot\|_{L_{p}})}$ for
$p\in[1,\infty)$ and $\overline{({\mathcal{S}}_{\rm
Liz,Rad},\|\cdot\|_{L_{\infty}})}=C_{0,{\rm even}}({\mathcal{Z}})$ otherwise
[33]. The claim then follows from the observation that
$L_{p}({\mathcal{Z}})=L_{p,{\rm even}}({\mathcal{Z}})\oplus L_{p,{\rm
odd}}({\mathcal{Z}})$ with $L_{p,{\rm
even}}({\mathcal{Z}})=\overline{({\mathcal{S}}_{\rm Rad},\|\cdot\|_{L_{p}})}$
(because the completion is unique) and suitable adaptation for $p=\infty$. ∎

Correspondingly, we get that ${\mathcal{X}}^{\prime}_{\rm
Rad}={\mathrm{P}}_{\rm
even}({\mathcal{X}}^{\prime})={\mathcal{X}}^{\prime}_{\rm even}$ and
$({\mathcal{X}}^{\rm c}_{\rm Rad})^{\prime}=(\mathrm{Id}-{\mathrm{P}}_{\rm
even})({\mathcal{X}}^{\prime})={\mathcal{X}}^{\prime}_{\rm odd}$, with the
cases of greatest interest to us being ${\mathcal{M}}_{\rm
Rad}={\mathcal{M}}_{\rm
even}(\mathbb{R}\times\mathbb{S}^{d-1})={\mathcal{M}}(\mathbb{P}^{d})$ and
$L_{2,\rm Rad}=L_{2,{\rm
even}}(\mathbb{R}\times\mathbb{S}^{d-1})=L_{2}(\mathbb{P}^{d})$.

###  3.4 Specific Radon Transforms

The Fourier-slice theorem expressed by (20) remains valid for tempered
distributions [41] and therefore also yields a characterization of
${\mathrm{R}}\\{f\\}$ that is compatible with the Banach framework of Theorem
3. It is especially helpful when the underlying function $\rho_{\rm iso}$ is
isotropic with a known radial frequency profile $\widehat{\rho}_{\rm rad}$
such that $\mathcal{F}\\{\rho_{\rm
iso}\\}({\boldsymbol{\omega}})=\widehat{\rho}_{\rm
rad}(\|{\boldsymbol{\omega}}\|)$.

######  Proposition 1 (Radon transform of isotropic distributions).

Let $\rho_{\rm iso}$ be an isotropic distribution whose radial frequency
profile is $\widehat{\rho}_{\rm rad}:\mathbb{R}\to\mathbb{R}$. Then,

$\displaystyle{\mathrm{R}}\\{\rho_{\rm
iso}(\cdot-{\boldsymbol{x}}_{0})\\}(t,{\boldsymbol{\xi}}\\}$
$\displaystyle=\rho_{\rm
rad}(t-{\boldsymbol{\xi}}^{\mathsf{T}}{\boldsymbol{x}}_{0})$ (39)
$\displaystyle{\mathrm{K}}_{\rm rad}{\mathrm{R}}\\{\rho_{\rm
iso}(\cdot-{\boldsymbol{x}}_{0})\\}(t,{\boldsymbol{\xi}}\\}$
$\displaystyle=\nu_{\rm
rad}(t-{\boldsymbol{\xi}}^{\mathsf{T}}{\boldsymbol{x}}_{0})$ (40)

with $\rho_{\rm rad}(t)=\mathcal{F}^{-1}\\{\widehat{\rho}_{\rm
rad}(\omega)\\}(t)$ and $\nu_{\rm
rad}(t)=\tfrac{1}{2(2\pi)^{d-1}}\mathcal{F}^{-1}\\{|\omega|^{d-1}\widehat{\rho}_{\rm
rad}(\omega)\\}(t)$.

The other important building blocks for representing functions are ridges.
Specifically, a ridge is a multidimensional function

$\displaystyle
r_{{\boldsymbol{\xi}}_{0}}:\mathbb{R}^{d}\to\mathbb{R}:{\boldsymbol{x}}\mapsto
r({\boldsymbol{\xi}}_{0}^{\mathsf{T}}{\boldsymbol{x}})$ (41)

that is characterized by a profile $r:\mathbb{R}\to\mathbb{R}$ and a direction
${\boldsymbol{\xi}}_{0}\in\mathbb{S}^{d-1}$. In effect,
$r_{{\boldsymbol{\xi}}_{0}}$ varies along the axis specified by
${\boldsymbol{\xi}}_{0}$ and is constant within any hyperplane perpendicular
to ${\boldsymbol{\xi}}_{0}$. The connection between ridges and the Radon
transform is given by the ridge identity

$\displaystyle\forall\varphi\in{\mathcal{S}}(\mathbb{R}^{d}):\quad\langle
r_{{\boldsymbol{\xi}}_{0}},\varphi\rangle=\langle
r,{\mathrm{R}}\\{\varphi\\}(\cdot,{\boldsymbol{\xi}}_{0})\rangle,$ (42)

where the right-hand side duality product (1D) is well-defined for any
$r\in{\mathcal{S}}^{\prime}(\mathbb{R})$ because
${\mathrm{R}}\\{\varphi\\}(\cdot,{\boldsymbol{\xi}}_{0})\in{\mathcal{S}}(\mathbb{R})$
(by Theorem 1). When the profile $r:\mathbb{R}\to\mathbb{R}$ is locally
integrable, (42) is a simple consequence of Fubini’s theorem. For more general
distributional profiles $r\in{\mathcal{S}}^{\prime}(\mathbb{R})$, we use the
ridge identity as definition, which then leads to the following
characterization [52].

######  Theorem 4 (Filtered Radon transform of a ridge).

The filtered Radon transform of the (generalized) ridge
$r_{{\boldsymbol{\xi}}_{0}}$ with profile
$r\in{\mathcal{S}}^{\prime}(\mathbb{R}^{d})$ and direction
${\boldsymbol{\xi}}_{0}\in\mathbb{S}^{d-1}$ is given by

$\displaystyle{\mathrm{K}}_{\rm
rad}{\mathrm{R}}\\{r_{{\boldsymbol{\xi}}_{0}}\\}(t,{\boldsymbol{\xi}})=[r(t)\delta({\boldsymbol{\xi}}-{\boldsymbol{\xi}}_{0})],$
(43)

where
$[r(t)\delta({\boldsymbol{\xi}}-{\boldsymbol{\xi}}_{0})]\in{\mathcal{S}}_{\rm
Rad}^{\prime}$ is the equivalence class of distributions specified by (27). If
$r\in{\mathcal{M}}(\mathbb{R})$, then the latter has the unique, concrete
representer

$\displaystyle{\mathrm{K}}_{\rm
rad}{\mathrm{R}}\\{r_{{\boldsymbol{\xi}}_{0}}\\}(t,{\boldsymbol{\xi}})$
$\displaystyle=\frac{1}{2}\big{(}r(t)\delta({\boldsymbol{\xi}}-{\boldsymbol{\xi}}_{0})+r(-t)\delta({\boldsymbol{\xi}}+{\boldsymbol{\xi}}_{0})\big{)}$
(44)

in ${\mathcal{M}}_{\rm Rad}={\mathcal{M}}_{\rm
even}(\mathbb{R}\times\mathbb{S}^{d-1})$.

An important special case of (44) is the Radon transform of a Dirac ridge:
${\mathrm{K}}_{\rm
rad}{\mathrm{R}}\\{\delta({\boldsymbol{\xi}}_{0}^{\mathsf{T}}\cdot-
t_{0})\\}=\delta_{{\rm
Rad},(t_{0},{\boldsymbol{\xi}}_{0})}=\frac{1}{2}\big{(}\delta(\cdot-
t_{0})\delta(\cdot-{\boldsymbol{\xi}}_{0})+\delta(\cdot+t_{0})\delta(\cdot+{\boldsymbol{\xi}}_{0})\big{)}$,
which has already been mentioned in Section 3.2 (see also [34, Example 1]).

##  4 Unifying Variational Formulation

###  4.1 Representer Theorem for Radon-Domain Regularization

From now on, we shall use the generic symbol ${\mathcal{X}}$ to designate the
hyper-spherical Banach space $L_{q}(\mathbb{R}\times\mathbb{S}^{d-1})$ with
$q=(1,\infty)$ or $C_{0}(\mathbb{R}\times\mathbb{S}^{d-1})$ for $q=\infty$,
which fall into the category described by (31) with
$\|\cdot\|_{{\mathcal{X}}}=\|\cdot\|_{L_{q}}$.

The formulation of Theorem 5 below requires the specification of a native
space ${\mathcal{X}}_{{\mathrm{L}}_{\rm R}}^{\prime}(\mathbb{R}^{d})$ that is
tied to a Radon-domain norm $\|\cdot\|_{{\mathcal{X}}^{\prime}}$ and an
admissible regularization operator ${\rm L}_{\rm R}$. To properly identify
${\mathcal{X}}_{{\mathrm{L}}_{\rm R}}^{\prime}$, we first need to restrict the
dual pair $({\mathcal{X}},{\mathcal{X}}^{\prime})$ to the range of the
(filtered) Radon transform. This yields the Banach spaces $({\mathcal{X}}_{\rm
Rad},{\mathcal{X}}^{\prime}_{\rm Rad})$, as defined by (34) and (35), with the
pairs of interest being $(C_{0,{\rm Rad}},{\mathcal{M}}_{0,{\rm Rad}})$ and
$(L_{q,{\rm Rad}},L_{p,{\rm Rad}})$ with $\frac{1}{p}+\tfrac{1}{q}=1$ and
$p\in[1,\infty)$. Given some spline-admissible operator ${\rm L}$ (Definition
1 and 2), we then define our regularization operator and its adjoint as

$\displaystyle{\rm L}_{\rm R}$
$\displaystyle\stackrel{{\scriptstyle\vartriangle}}{{=}}{\mathrm{K}}_{\rm
rad}{\mathrm{R}}{\mathrm{L}}:{\mathcal{X}}^{\prime}_{{\mathrm{L}}_{\rm
R}}(\mathbb{R}^{d})\to{\mathcal{X}}^{\prime}_{\rm Rad},$ $\displaystyle{\rm
L}^{\ast}_{\rm R}$
$\displaystyle={\mathrm{L}}^{\ast}{\mathrm{R}}^{\ast}{\mathrm{K}}_{\rm
rad}:{\mathcal{X}}_{\rm Rad}\to{\mathcal{X}}_{{\mathrm{L}}_{\rm
R}}(\mathbb{R}^{d})$

where ${\mathrm{R}}$ denotes the Radon transform and ${\mathrm{K}}_{\rm rad}$
is the (self-adjoint) filtering operator such that ${\mathrm{K}}_{\rm
rad}{\mathrm{R}}{\mathrm{R}}^{\ast}=\mathrm{Id}$ on
${\mathcal{X}}^{\prime}_{\rm Rad}$ (Theorem 3). In order to establish
isometries, one needs to be able to invert ${\rm L}$ (resp., ${\rm L}_{\rm
R}$), which is feasible if one factors out the null space ${\mathcal{P}}$,
which is common to both. This motivates us to define the directed inverse
operators

$\displaystyle{\rm L}^{-1}_{\mathcal{P}}$
$\displaystyle\stackrel{{\scriptstyle\vartriangle}}{{=}}(\mathrm{Id}-\mathrm{Proj}_{{\mathcal{P}}}){\mathrm{L}}^{-1}:{\mathrm{R}}^{\ast}\big{(}{\mathcal{X}}^{\prime}_{\rm
Rad}\big{)}\to{\mathcal{X}}^{\prime}_{{\mathrm{L}}_{\rm R}}(\mathbb{R}^{d})$
$\displaystyle{\rm L}^{-1\ast}_{\mathcal{P}}$
$\displaystyle={\mathrm{L}}^{-1\ast}(\mathrm{Id}-\mathrm{Proj}_{{\mathcal{P}}^{\prime}}):{\mathcal{X}}_{{\mathrm{L}}_{\rm
R}}(\mathbb{R}^{d})\to{\mathrm{K}}_{\rm rad}{\mathrm{R}}({\mathcal{X}}_{\rm
Rad})$ $\displaystyle{\rm L}_{\rm R}^{\dagger}$
$\displaystyle\stackrel{{\scriptstyle\vartriangle}}{{=}}{\rm
L}^{-1}_{\mathcal{P}}{\mathrm{R}}^{\ast}:{\mathcal{X}}^{\prime}_{\rm
Rad}\to{\mathcal{X}}^{\prime}_{{\mathrm{L}}_{\rm R}}(\mathbb{R}^{d})$ (45)
$\displaystyle{\rm L}_{\rm R}^{\ast\dagger}$ $\displaystyle={\mathrm{R}}{\rm
L}^{-1\ast}_{\mathcal{P}}:{\mathcal{X}}_{{\mathrm{L}}_{\rm
R}}(\mathbb{R}^{d})\to{\mathcal{X}}_{\rm Rad}$ (46)

where the operators ${\rm L}_{\rm R}^{\dagger}$ and ${\rm L}_{\rm
R}^{\ast\dagger}$ are generalized inverses333The precise properties of these
inverse operators are stated in Theorem 6 and 9 . of ${\rm L}_{\rm R}$ and
${\rm L}_{\rm R}^{\ast}$, respectively. We now have all the ingredients to
specify our native space as

$\displaystyle{\mathcal{X}}^{\prime}_{{\mathrm{L}}_{\rm R}}(\mathbb{R}^{d})$
$\displaystyle={\mathrm{L}}^{\dagger}_{\rm
R}\big{(}{\mathcal{X}}^{\prime}_{\rm Rad}\big{)}\oplus{\mathcal{P}}$
$\displaystyle=\\{f\in L_{\infty,-n_{0}}(\mathbb{R}^{d}):\|{\mathrm{L}}_{\rm
R}\\{f\\}\|_{{\mathcal{X}}^{\prime}}+\|\mathrm{Proj}_{\mathcal{P}}\\{f\\}\|_{{\mathcal{P}}}<\infty\\}$
$\displaystyle=\\{{\mathrm{L}}^{\dagger}_{\rm
R}\\{w\\}+p_{0}:\quad(w,p_{0})\in{\mathcal{X}}_{\rm
Rad}^{\prime}\times{\mathcal{P}}\\},$ (47)

which is isometrically isomorphic to ${\mathcal{X}}_{\rm
Rad}^{\prime}\times{\mathcal{P}}$, as expressed by (47). The key property
there is that ${\rm L}_{\rm R}{\mathrm{L}}^{\dagger}_{\rm R}=\mathrm{Id}$ on
${\mathcal{X}}_{\rm Rad}^{\prime}$, while ${\rm L}_{\rm R}\\{p_{0}\\}=0$ for
all $p_{0}\in{\mathcal{P}}$ (Theorem 6). Moreover,
${\mathcal{X}}^{\prime}_{{\mathrm{L}}_{\rm R}}(\mathbb{R}^{d})$ is the
topological dual of the predual space

$\displaystyle{\mathcal{X}}_{{\mathrm{L}}_{\rm R}}(\mathbb{R}^{d})$
$\displaystyle={\rm L}_{\rm R}^{\ast}\big{(}{\mathcal{X}}_{\rm
Rad}\big{)}\oplus{\mathcal{P}}^{\prime}$
$\displaystyle=\\{\nu\in{\mathcal{S}}^{\prime}(\mathbb{R}^{d}):\|\nu\|_{{\mathcal{X}}_{{\rm
L}_{\rm R}}}=\max(\|{\rm L}_{\rm
R}^{\ast\dagger}\\{\nu\\}\|_{{\mathcal{X}}},\|\mathrm{Proj}_{{\mathcal{P}}^{\prime}}\\{\nu\\}\|_{{\mathcal{P}}^{\prime}})<\infty\\}$
$\displaystyle=\\{{\mathrm{L}}_{\rm
R}^{\ast}\\{v\\}+p_{0}^{\ast}:\quad(v,p_{0}^{\ast})\in{\mathcal{X}}_{\rm
Rad}\times{\mathcal{P}}^{\prime}\\},$ (48)

which is a bona fide Banach space, as shown in Theorem 9. The validity of this
dual pairing can be checked formally in the absence of null space components:
For any $(f,\nu)\in{\mathcal{X}}^{\prime}_{{\mathrm{L}}_{\rm
R}}(\mathbb{R}^{d})\times{\mathcal{X}}_{{\mathrm{L}}_{\rm R}}(\mathbb{R}^{d})$
with $\mathrm{Proj}_{\mathcal{P}}f=0$ and
$\mathrm{Proj}_{{\mathcal{P}}^{\prime}}\nu=0$, we have that

$\displaystyle\langle f,\nu\rangle$
$\displaystyle=\langle{\mathrm{L}}^{\dagger}_{\rm
R}\\{w\\},{\mathrm{L}}^{\ast}_{\rm R}\\{v\\}\rangle=\langle{\mathrm{L}}_{\rm
R}{\mathrm{L}}^{\dagger}_{\rm R}\\{w\\},v\rangle_{\rm Rad}=\langle
w,v\rangle_{\rm Rad}$

with $(w,v)\in{\mathcal{X}}^{\prime}_{\rm Rad}\times{\mathcal{X}}_{\rm Rad}$.
Finally, since $\mathrm{Proj}_{{\mathcal{P}}}$ continuously maps
${\mathcal{X}}^{\prime}_{{\mathrm{L}}_{\rm
R}}(\mathbb{R}^{d})\to{\mathcal{P}}\xhookrightarrow{\mbox{\tiny\rm
iso.}}{\mathcal{X}}^{\prime}_{{\mathrm{L}}_{\rm R}}(\mathbb{R}^{d})$, we can
identify $\mathrm{Proj}_{{\mathcal{P}}^{\prime}}$ as its adjoint
$(\mathrm{Proj}_{{\mathcal{P}}})^{\ast}=\mathrm{Proj}_{{\mathcal{P}}^{\prime}}:{\mathcal{X}}_{{\mathrm{L}}_{\rm
R}}(\mathbb{R}^{d})\to{\mathcal{P}}^{\prime}\xhookrightarrow{\mbox{\tiny\rm
iso.}}{\mathcal{X}}_{{\mathrm{L}}_{\rm R}}(\mathbb{R}^{d})$.

We now state our primary theorem.

######  Theorem 5.

Let $E:\mathbb{R}\times\mathbb{R}\to\mathbb{R}$ be a strictly convex loss
function, ${\rm L}$ an isotropic, spline-admissible operator with polynomial
null space ${\mathcal{P}}_{n_{0}}$ of degree $n_{0}$, possibly trivial with
the convention that ${\mathcal{P}}_{-1}=\\{0\\}$, and let
$\psi:\mathbb{R}^{+}\to\mathbb{R}^{+}$ be a convex, strictly increasing
function. Then, for any given
$({\boldsymbol{x}}_{m},y_{m})\in\mathbb{R}^{d}\times\mathbb{R},m=1,\dots,M$
with $M\geq{\rm dim}{\mathcal{P}}_{n_{0}}$, the generic functional-
optimization problem

$\displaystyle S=\arg\min_{f\in{\mathcal{X}}^{\prime}_{{\mathrm{L}}_{\rm
R}}(\mathbb{R}^{d})}\left(\sum_{m=1}^{M}E(y_{m},f({\boldsymbol{x}}_{m}))+\psi(\|{\mathrm{L}}_{\rm
R}f\|_{{\mathcal{X}}^{\prime}})\right),$ (49)

with ${\rm L}_{\rm R}={\mathrm{K}}_{\rm rad}{\mathrm{R}}{\mathrm{L}}$ and
${\mathcal{X}}^{\prime}$ as stated below, always has a solution.

  1. 1.

When
${\mathcal{X}}={\mathcal{X}}^{\prime}=L_{2}(\mathbb{R}\times\mathbb{S}^{d-1})$,
the solution of (49) is unique and representable by the linear kernel
expansion

$\displaystyle
f({\boldsymbol{x}})=p_{0}({\boldsymbol{x}})+\sum_{m=1}^{M}a_{m}\rho_{\rm
iso}({\boldsymbol{x}}-{\boldsymbol{x}}_{m}),$ (50)

where $\rho_{\rm
iso}=2(2\pi)^{d-1}\mathcal{F}^{-1}\\{1/(|\widehat{L}({\boldsymbol{\omega}})|^{2}\|{\boldsymbol{\omega}}\|^{d-1})\\}$
is a radial basis function, $(a_{m})\in\mathbb{R}^{M}$ an adequate set of
coefficients, and $p_{0}\in{\mathcal{P}}_{n_{0}}$ a polynomial that lies in
the null space of ${\mathrm{L}}_{\rm R}$.

  2. 2.

When ${\mathcal{X}}^{\prime}=L_{p}(\mathbb{R}\times\mathbb{S}^{d-1})$ with
$p\in(1,2]$, the solution is unique and admits the parametric representation

$\displaystyle f({\boldsymbol{x}})=p_{0}({\boldsymbol{x}})+{\rm L}_{\rm
R}^{\dagger}\circ{\mathrm{J}}_{q}\\{\sum_{m=1}^{M}a_{m}\nu_{{\boldsymbol{x}}_{m}}\\}({\boldsymbol{x}})$
(51)

with basis functions
$\nu_{{\boldsymbol{x}}_{1}},\ldots,\nu_{{\boldsymbol{x}}_{M}}\in
L_{q}(\mathbb{R}\times\mathbb{S}^{d-1})$ specified by (67) and parameters
$(a_{m})\in\mathbb{R}^{M}$, $p_{0}\in{\mathcal{P}}_{n_{0}}$, and where
${\mathrm{J}}_{q}$ is the pointwise nonlinearity given by (89) with
$q=p/(p-1)$. (The latter is the duality map
${\mathrm{J}}_{{\mathcal{X}}}:{\mathcal{X}}\to{\mathcal{X}}^{\prime}$ with
${\mathcal{X}}=L_{q}(\mathbb{R}\times\mathbb{S}^{d-1})$—see Appendix A for
explanations).

  3. 3.

When ${\mathcal{X}}^{\prime}={\mathcal{M}}(\mathbb{R}\times\mathbb{S}^{d-1})$,
the solution set is the weak$\ast$-closed convex hull of its extreme points,
which are all of the form

$\displaystyle f_{\rm
ext}({\boldsymbol{x}})=p_{0}({\boldsymbol{x}})+\sum_{k=1}^{K_{0}}a_{k}\rho_{\rm
rad}({\boldsymbol{\xi}}_{k}^{\mathsf{T}}{\boldsymbol{x}}-\tau_{k})$ (52)

with activation function $\rho_{\rm rad}=\mathcal{F}^{-1}\\{1/\widehat{L}_{\rm
rad}\\}$, for some $K_{0}\leq M-{\rm dim}{\mathcal{P}}_{n_{0}}$,
$(a_{k},{\boldsymbol{\xi}}_{k},\tau_{k})\in\mathbb{R}\times\mathbb{S}^{d-1}\times\mathbb{R}$
for $k=1,\dots,K_{0}$, and a null-space component
$p_{0}\in{\mathcal{P}}_{n_{0}}$. The optimal regularization cost associated
with (52) is $\|{\mathrm{L}}_{\rm R}f_{\rm
ext}\|_{\mathcal{M}}=\sum_{k=1}^{K_{0}}|a_{k}|$ and is shared by all
solutions.

###### Proof of Theorem 5.

Since ${\rm L}_{\rm R}^{\ast}$ is injective on ${\mathcal{S}}_{\rm Rad}$ and,
by extension, on the completed space ${\mathcal{X}}_{\rm Rad}$, the image
space ${\mathcal{U}}={\rm L}_{\rm R}^{\ast}\big{(}{\mathcal{X}}_{\rm
Rad}\big{)}$ is a bona fide Banach space (see proof of Theorem 9 for the
details of the construction of ${\mathcal{U}}$). Its continuous dual is given
by
${\mathcal{U}}^{\prime}={\mathrm{L}}^{-1}_{{\mathcal{P}}}{\mathrm{R}}^{\ast}\big{(}{\mathcal{X}}^{\prime}_{\rm
Rad}\big{)}$, in reason of the identities
${\mathrm{R}}{\mathrm{L}}^{-1\ast}_{{\mathcal{P}}}{\rm L}_{\rm
R}^{\ast}={\mathrm{R}}{\mathrm{L}}^{-1\ast}_{{\mathcal{P}}}{\mathrm{L}}^{\ast}{\mathrm{R}}^{\ast}{\mathrm{K}}_{\rm
rad}={\mathrm{R}}{\mathrm{R}}^{\ast}{\mathrm{K}}_{\rm rad}=\mathrm{Id}$ on
${\mathcal{X}}_{\rm Rad}$. Likewise, ${\mathcal{P}}^{\prime}$, as identified
by (15), is a finite-dimensional Banach space. Its dual is simply
$({\mathcal{P}}^{\prime})^{\prime}={\mathcal{P}}$ (the null space of both
${\rm L}$ and ${\rm L}_{\rm R}$), owing to the property that all finite-
dimensional spaces are reflexive. Using the notation for direct-sum topologies
of [53], we then observe that ${\mathcal{X}}_{{\mathrm{L}}_{\rm
R}}(\mathbb{R}^{d})=({\mathcal{U}}\oplus{\mathcal{P}}^{\prime})_{\ell_{\infty}}$,
whose formal dual
$({\mathcal{U}}\oplus{\mathcal{P}}^{\prime})^{\prime}_{\ell_{\infty}}=({\mathcal{U}}^{\prime}\oplus{\mathcal{P}})_{\ell_{1}}$
is precisely the native space ${\mathcal{X}}^{\prime}_{{\mathrm{L}}_{\rm
R}}(\mathbb{R}^{d})$ described by (47). By writing $f={\rm
L}_{{\mathcal{P}}}^{-1}{\mathrm{R}}^{\ast}\\{w\\}+p_{0}$ and recalling that
${\rm L}{\rm L}_{{\mathcal{P}}}^{-1}=\mathrm{Id}$ (right-inverse property), we
then identify the regularization functional as

$\displaystyle\|{\mathrm{L}}_{\rm R}f\|_{{\mathcal{X}}^{\prime}}$
$\displaystyle=\|{\mathrm{K}}_{\rm rad}{\mathrm{R}}\\{{\mathrm{L}}{\rm
L}^{-1}_{\mathcal{P}}{\mathrm{R}}^{\ast}w+{\mathrm{L}}p_{0}\\}\|_{{\mathcal{X}}^{\prime}}$
$\displaystyle=\|{\mathrm{K}}_{\rm
rad}{\mathrm{R}}{\mathrm{R}}^{\ast}\\{w\\}+{\mathrm{K}}_{\rm
rad}{\mathrm{R}}\\{0\\}\|_{{\mathcal{X}}^{\prime}}$
$\displaystyle=\|w\|_{{\mathcal{X}}^{\prime}_{\rm
Rad}}=\|\mathrm{Proj}_{{\mathcal{U}}^{\prime}}f\|_{{\mathcal{U}}^{\prime}}$

which, in effect, converts the seminorm over
${\mathcal{X}}^{\prime}_{{\mathrm{L}}_{\rm R}}(\mathbb{R}^{d})$ into a norm
over ${\mathcal{U}}^{\prime}$ by factoring out the null space of ${\rm L}_{\rm
R}$. The other mathematical ingredient for the optimization problem (49) to be
well-posed is the weak$\ast$ continuity of the sampling functionals
$\delta(\cdot-{\boldsymbol{x}}_{m}):f\mapsto f({\boldsymbol{x}}_{m})$ in the
underlying topology. This is equivalent to
$\delta(\cdot-{\boldsymbol{x}}_{m})\in{\mathcal{X}}_{{\mathrm{L}}_{\rm
R}}(\mathbb{R}^{d})$ for any ${\boldsymbol{x}}_{m}\in\mathbb{R}^{d}$. In the
present context, this condition can be reframed as
$\nu_{{\boldsymbol{x}}_{m}}={\rm L}^{\ast\dagger}_{\rm
R}\\{\delta(\cdot-{\boldsymbol{x}}_{m})\\}\in{\mathcal{X}}_{\rm
Rad}(\mathbb{R}\times\mathbb{S}^{d-1})$, which is a property that is
established in Theorem 7 for the cases ${\mathcal{X}}=C_{0}$ and
${\mathcal{X}}=L_{q}$ for any $q\geq 2$.

The existence of the solution and the parametric descriptions (50), (51), and
(52) then follows from the three cases of the abstract representer theorem for
direct sums [53, Theorem 3]. The duality map that is required for the first
and second scenarios is ${\mathrm{J}}_{\mathcal{U}}={\rm L}^{\dagger}_{\rm
R}\circ{\mathrm{J}}\circ{\rm L}^{\ast\dagger}_{\rm
R}:{\mathcal{U}}\to{\mathcal{U}}^{\prime}$ (see Appendix A, Proposition 4 with
${\mathrm{T}}={\rm L}^{\ast}_{\rm R}$, ${\mathrm{T}}^{-1}={\rm L}_{\rm
R}^{\ast\dagger}={\mathrm{R}}^{\ast}{\mathrm{L}}^{\ast-1}_{{\mathcal{P}}}$,
and $({\mathrm{T}}^{\ast})^{-1}={\rm L}^{\dagger}_{\rm
R}={\mathrm{L}}^{-1}_{{\mathcal{P}}}{\mathrm{R}}^{\ast}$).

To describe the solution set for the non-reflexive case
${\mathcal{X}}^{\prime}={\mathcal{M}}$, we invoke the third case of [53,
Theorem 3], which tells us that the extreme points of $S$ can all be expressed
as the sum of a null-space component plus a linear combination of $K_{0}\leq
M-{\rm dim}({\mathcal{P}})$ atoms $e_{k}$ that are selected adaptively within
a dictionary consisting of the extreme points of the unit ball in
${\mathcal{U}}^{\prime}$:
$B({\mathcal{U}}^{\prime})=\\{u\in{\mathcal{U}}^{\prime}:\|u\|_{{\mathcal{U}}^{\prime}}\leq
1\\}$. Because of the linear isomorphism
${\mathcal{U}}^{\prime}={\mathrm{L}}^{\dagger}_{\rm
R}\big{(}{\mathcal{M}}_{\rm Rad}\big{)}$, ${\rm
ext}B({\mathcal{U}})={\mathrm{L}}^{\dagger}_{\rm R}\big{(}{\rm
ext}B({\mathcal{M}}_{\rm Rad})\big{)}$. Next, we use the property that
${\mathcal{M}}_{\rm Rad}={\mathcal{M}}(\mathbb{P}^{d})$ whose extreme points
are $\\{\pm\delta_{{\rm
Rad},{\boldsymbol{z}}}\\}_{{\boldsymbol{z}}\in\mathbb{P}^{d}}$. Each
$\delta_{{\rm Rad},{\boldsymbol{z}}_{k}}\in{\rm ext}B({\mathcal{M}}_{\rm
Rad})$ then bijectively maps into an extreme point
$e_{k}={\mathrm{L}}^{\dagger}_{\rm R}\\{\delta_{{\rm
Rad},{\boldsymbol{z}}_{k}}\\}\in{\rm ext}B({\mathcal{U}})$, and vice versa.
Finally, by recalling that ${\mathrm{L}}^{\dagger}_{\rm
R}=(\mathrm{Id}-\mathrm{Proj}_{{\mathcal{P}}}){\mathrm{L}}^{-1}{\mathrm{R}}^{\ast}$
and by invoking Theorem 4 to show that
${\mathrm{L}}^{-1}{\mathrm{R}}^{\ast}\\{\delta_{{\rm
Rad},(t_{k},{\boldsymbol{\xi}}_{k})}\\}={\mathrm{L}}^{-1}{\mathrm{R}}^{\ast}{\mathrm{K}}_{\rm
rad}{\mathrm{R}}\\{\delta({\boldsymbol{\xi}}_{k}^{\mathsf{T}}{\boldsymbol{x}}-t_{k})\\}={\mathrm{L}}^{-1}\\{\delta({\boldsymbol{\xi}}_{k}^{\mathsf{T}}{\boldsymbol{x}}-t_{k})\\}=\rho_{\rm
rad}({\boldsymbol{\xi}}_{k}^{\mathsf{T}}{\boldsymbol{x}}-t_{k})$, we find that

$\displaystyle e_{k}=\pm\rho_{\rm
rad}({\boldsymbol{\xi}}_{k}^{\mathsf{T}}{\boldsymbol{x}}-t_{k})\mp p_{0,k},$
(53)

where $p_{0,k}=\mathrm{Proj}_{{\mathcal{P}}}\\{\rho_{\rm
rad}({\boldsymbol{\xi}}_{k}^{\mathsf{T}}{\boldsymbol{x}}-\tau_{k})\\}\in{\mathcal{P}}$.
Since every extreme point of $B({\mathcal{U}}^{\prime})$ is necessarily of the
form (53), we can substitute this expression in the generic expansion $f_{\rm
ext}=\tilde{p}_{0}+\sum_{k=1}^{K_{0}}a_{k}e_{k}$ which, upon collection of all
null-space components, yields (52). ∎

The two cases in Theorem 5 that are of direct practical relevance to machine
learning are Items 1 and 3. The first scenario yields a learning architecture
that is a linear expansion of RBFs, which also has a classical RKHS
interpretation, as explained in Section 4.2.

The form of the solution in Item 3 is equivalent to a shallow network with the
weights (${\boldsymbol{\xi}}_{k}$) of the hidden layer being normalized. It
actually turns out that this normalization is inconsequential when the
activation is a homogeneous function. This happens for instance when the
regularization operator ${\rm L}=(-\Delta)^{\gamma_{0}/2}$ is a fractional
Laplacian which maps into $\rho_{\rm Rad}(t)\propto|t|^{\gamma_{0}-1}$.
Indeed, for any
$({\boldsymbol{w}}_{k},b_{k})\in\mathbb{R}^{d}\times\mathbb{R}$, we have that

$\displaystyle|{\boldsymbol{w}}_{k}^{\mathsf{T}}{\boldsymbol{x}}-b_{k}|^{\gamma_{0}-1}=\|{\boldsymbol{w}}_{k}\|^{-\gamma_{0}+1}|{\boldsymbol{\xi}}_{k}^{\mathsf{T}}{\boldsymbol{x}}-t_{k}|^{\gamma_{0}-1}$
(54)

with ${\boldsymbol{\xi}}_{k}={\boldsymbol{w}}_{k}/\|{\boldsymbol{w}}_{k}\|$
and $t_{k}=b_{k}/\|{\boldsymbol{w}}_{k}\|$, which indicates that the
normalization (or lack thereof) can be absorbed in the weights $(a_{k})$ of
the output layer. The case $\gamma_{0}=1$ with ${\rm L}=\Delta$ (Laplacian) is
particularly attractive, as it nicely maps into a ReLU network with one hidden
layer and a skip connection to implement the affine component [36].

The form of the solution in Item 2 is more involved, but still useful to get
insight into the transition with $p$ from RBFs to neural nets. The equivalence
with Item 1 for $p=2$ is demonstrated in the Section 4.2. As for the behavior
as $p\to 1$, we observe that the effect of the duality map ${\mathrm{J}}_{q}$
as $q\to\infty$ is to pull a few maximal values to infinity, while attenuating
all other (non-supremum) values towards zero. In effect, this means that the
hyper-spherical function
${\mathrm{J}}_{q}\\{\sum_{m=1}^{M}a_{m}\nu_{{\boldsymbol{x}}_{m}}\\}$ will
exhibit peaks that become more and more pronounced, and eventually converge to
a sum of impulses as $p\to 1$, which is consistent with the limit form given
by (52).

###  4.2 Connection with RKHS Methods

The scenario ${\mathcal{X}}=L_{2}$ in Theorem 5 is compatible with the kernel
models of “classical” machine learning [38, 47]. This is because the
underlying native space is a reproducing-kernel Hilbert space whose
topological structure is now made explicit.

######  Proposition 2 (Characterization of RKHS for ${\mathcal{X}}=L_{2}$).

Let ${\rm L}$ be a spline admissible operator with a polynomial null space of
degree $n_{0}$ and consider the self-adjoint operator
${\mathrm{A}}=({\mathrm{L}}^{\ast}{\mathrm{K}}{\rm L})^{-1}$. Then, the native
space ${\mathcal{H}}=L^{\prime}_{2,{\rm L}_{\rm R}}(\mathbb{R}^{d})$, defined
by (47) with ${\mathcal{X}}^{\prime}={\mathcal{X}}=L_{2}$, is the reproducing-
kernel Hilbert space ${\mathcal{H}}={\mathcal{U}}^{\prime}\oplus{\mathcal{P}}$
associated with the composite inner product

$\displaystyle\ \langle f_{1},f_{2}\rangle_{{\mathcal{H}}}$
$\displaystyle=\langle{\rm L}_{\rm R}\\{f_{1}\\},{\rm L}_{\rm
R}\\{f_{2}\\}\rangle_{\rm Rad}+\sum_{|{\boldsymbol{k}}|\leq n_{0}}\langle
m_{\boldsymbol{k}}^{\ast},f_{1}\rangle\langle
m_{\boldsymbol{k}}^{\ast},f_{2}\rangle$ $\displaystyle=\langle({\rm
L}^{\ast}{\mathrm{K}}{\rm
L})\\{f_{1}\\},f_{2}\rangle+\sum_{|{\boldsymbol{k}}|\leq n_{0}}\langle
m_{\boldsymbol{k}}^{\ast},f_{1}\rangle\langle
m_{\boldsymbol{k}}^{\ast},f_{2}\rangle$ (55)

whose leading term can also be written as

$\displaystyle\langle({\rm L}^{\ast}{\mathrm{K}}{\rm
L})\\{f_{1}\\},f_{2}\rangle$ $\displaystyle=\langle({\rm
L}^{\ast}{\mathrm{K}}{\rm
L})\mathrm{Proj}_{{\mathcal{U}}^{\prime}}\\{f_{1}\\},\mathrm{Proj}_{{\mathcal{U}}^{\prime}}\\{f_{2}\\}\rangle$
$\displaystyle=\langle\mathrm{Proj}_{{\mathcal{U}}^{\prime}}\\{f_{1}\\},\mathrm{Proj}_{{\mathcal{U}}^{\prime}}\\{f_{2}\\}\rangle_{{\mathcal{U}}^{\prime}},$
(56)

where
$\mathrm{Proj}_{{\mathcal{U}}^{\prime}}=\mathrm{Id}-\mathrm{Proj}_{{\mathcal{P}}}:{\mathcal{H}}\to{\mathcal{U}}^{\prime}$.
The topological dual of ${\mathcal{H}}$ is the Hilbert space
${\mathcal{H}}^{\prime}=L_{2,{\rm L}_{\rm
R}}(\mathbb{R}^{d})={\mathcal{U}}\oplus{\mathcal{P}}^{\prime}$ equipped with
the inner product

$\displaystyle\ \langle g_{1},g_{2}\rangle_{{\mathcal{H}}^{\prime}}$
$\displaystyle=\langle{\rm L}_{\rm R}^{\ast\dagger}g_{1},{\rm L}_{\rm
R}^{\ast\dagger}g_{2}\rangle_{\rm Rad}+\sum_{|{\boldsymbol{k}}|\leq
n_{0}}\langle m_{\boldsymbol{k}},g_{1}\rangle\langle
m_{\boldsymbol{k}},g_{2}\rangle$ $\displaystyle=\langle{\rm L}_{\rm
R}^{\dagger}{\rm L}_{\rm
R}^{\ast\dagger}g_{1},g_{2}\rangle+\sum_{|{\boldsymbol{k}}|\leq n_{0}}\langle
m_{\boldsymbol{k}},g_{1}\rangle\langle m_{\boldsymbol{k}},g_{2}\rangle$
$\displaystyle=\langle{\mathrm{A}}\mathrm{Proj}_{{\mathcal{U}}}\\{g_{1}\\},\mathrm{Proj}_{{\mathcal{U}}}\\{g_{2}\\}\rangle+\sum_{|{\boldsymbol{k}}|\leq
n_{0}}\langle m_{\boldsymbol{k}},g_{1}\rangle\langle
m_{\boldsymbol{k}},g_{2}\rangle,$ (57)

where
$\mathrm{Proj}_{{\mathcal{U}}}=\mathrm{Id}-\mathrm{Proj}_{{\mathcal{P}}^{\prime}}:{\mathcal{H}}^{\prime}\to{\mathcal{U}}$.
The corresponding linear isometries (Riesz maps) that map
${\mathcal{U}}\to{\mathcal{U}}^{\prime}$,
${\mathcal{P}}^{\prime}\to{\mathcal{P}}$, and
${\mathcal{H}}\to{\mathcal{H}}^{\prime}$ are

$\displaystyle{\mathrm{J}}_{{\mathcal{U}}}$
$\displaystyle={\mathrm{A}}:{\mathcal{U}}\to{\mathcal{U}}^{\prime}$
$\displaystyle=(\mathrm{Id}-\mathrm{Proj}_{{\mathcal{P}}}){\mathrm{A}}(\mathrm{Id}-\mathrm{Proj}_{{\mathcal{P}}^{\prime}}):{\mathcal{U}}\oplus{\mathcal{P}}^{\prime}\to{\mathcal{U}}^{\prime},$
$\displaystyle{\mathrm{J}}_{{\mathcal{P}}^{\prime}}$
$\displaystyle=\mathrm{Proj}_{{\mathcal{P}}}:{\mathcal{P}}^{\prime}\to{\mathcal{P}},$
$\displaystyle=\mathrm{Proj}_{{\mathcal{P}}}\mathrm{Proj}_{{\mathcal{P}}^{\prime}}:{\mathcal{U}}\oplus{\mathcal{P}}^{\prime}\to{\mathcal{P}},$
$\displaystyle{\mathrm{J}}_{{\mathcal{H}}}$
$\displaystyle=\left((\mathrm{Id}-\mathrm{Proj}_{{\mathcal{P}}}){\mathrm{A}}(\mathrm{Id}-\mathrm{Proj}_{{\mathcal{P}}^{\prime}})+\mathrm{Proj}_{{\mathcal{P}}}\mathrm{Proj}_{{\mathcal{P}}^{\prime}}\right):{\mathcal{H}}\to{\mathcal{H}}^{\prime},$

where the second, alternative forms of ${\mathrm{J}}_{{\mathcal{U}}}$ and
${\mathrm{J}}_{{\mathcal{P}}^{\prime}}$ that include projectors are the proper
extension of those operators to the whole space
${\mathcal{H}}={\mathcal{U}}^{\prime}\oplus{\mathcal{P}}$.

Proposition 2 is obtained as a corollary of Theorem 9 with
${\mathcal{X}}=L_{2}$, with the help of the intertwining property
${\mathrm{L}}_{\rm R}={\mathrm{K}}_{\rm
rad}{\mathrm{R}}{\mathrm{L}}={\mathrm{R}}{\mathrm{K}}{\mathrm{L}}$ (see the
discussion at the end of Section 3.2). The technical part is to establish the
completeness of ${\mathcal{U}}$ (resp.,
${\mathcal{H}}^{\prime}={\mathcal{U}}\oplus{\mathcal{P}}^{\prime}$), which
then implies that of $\ {\mathcal{U}}^{\prime}$ (resp.,
${\mathcal{H}}={\mathcal{H}}^{\prime\prime}={\mathcal{U}}^{\prime}\oplus{\mathcal{P}}$)
by duality. It then suffices to verify that the semi-inner products associated
with each individual term in (55) and (57) induce the four component norms
$\|w\|_{{\mathcal{U}}^{\prime}}$, $\|p_{0}\|_{{\mathcal{P}}}$,
$\|v\|_{{\mathcal{U}}}$, $\|p_{0}^{\ast}\|_{{\mathcal{P}}^{\prime}}$ for
${\mathcal{X}}={\mathcal{X}}^{\prime}=L_{2}$ that appear in the definitions
(47) and (48) of the native space and of its predual. The denomination RKHS
applies to any Hilbert space ${\mathcal{H}}$ of functions on $\mathbb{R}^{d}$
such that $\delta(\cdot-{\boldsymbol{x}}_{0})\in{\mathcal{H}}^{\prime}$ for
any ${\boldsymbol{x}}_{0}\in\mathbb{R}^{d}$. In our case, this property is
equivalent to ${\rm L}^{\ast\dagger}_{\rm
R}\\{\delta(\cdot-{\boldsymbol{x}}_{0})\\}\in L_{2,{\rm Rad}}$, which follows
from Theorem 7.

To get further insight, we now show that the RBF solution (50) is a particular
case of the $L_{p}$ solution (51) with $p=2$. Since $L_{2}=(L_{2})^{\prime}$
is its own dual, ${\mathrm{J}}=\mathrm{Id}$, which allows us to manipulate
(51) as

$\displaystyle f$ $\displaystyle=p_{0}+{\rm
L}_{{\mathcal{P}}}^{-1}{\mathrm{R}}^{\ast}{\mathrm{R}}{\rm
L}_{{\mathcal{P}}}^{-1\ast}\\{\sum_{m=1}^{M}a_{m}\delta(\cdot-{\boldsymbol{x}}_{m})\\}$
$\displaystyle=p_{0}+{\rm L}_{{\mathcal{P}}}^{-1}{\mathrm{K}}^{-1}{\rm
L}_{{\mathcal{P}}}^{-1\ast}\\{\sum_{m=1}^{M}a_{m}\delta(\cdot-{\boldsymbol{x}}_{m})\\}$
$\displaystyle=p_{0}+(\mathrm{Id}-\mathrm{Proj}_{{\mathcal{P}}}){\mathrm{A}}(\mathrm{Id}-\mathrm{Proj}_{{\mathcal{P}}^{\prime}})\\{\sum_{m=1}^{M}a_{m}\delta(\cdot-{\boldsymbol{x}}_{m})\\}$
(58)
$\displaystyle=p_{0}+{\mathrm{A}}\\{\sum_{m=1}^{M}a_{m}\delta(\cdot-{\boldsymbol{x}}_{m})\\}=p_{0}+\sum_{m=1}^{M}a_{m}\rho_{\rm
iso}({\boldsymbol{x}}-{\boldsymbol{x}}_{m}),$ (59)

where $\rho_{\rm iso}=({\rm L}^{\ast}{\mathrm{K}}{\rm
L})^{-1}\\{\delta\\}={\mathrm{A}}\\{\delta\\}:\mathbb{R}^{d}\to\mathbb{R}$ is
the Green’s function of $({\rm L}^{\ast}{\mathrm{K}}{\rm L})$ and
$p_{0}\in{\mathcal{P}}$. The nonobvious simplification from (58) to (59)
results from two crucial observations: (1) the “orthogonality-to-the-null-
space” condition
$\sum_{m=1}^{M}a_{m}\delta(\cdot-{\boldsymbol{x}}_{m})\in{\mathcal{U}}$ is
necessary for optimality; and (2) the availability of the identity

$\displaystyle\forall
u\in{\mathcal{U}}:\quad(\mathrm{Id}-\mathrm{Proj}_{{\mathcal{P}}}){\mathrm{A}}(\mathrm{Id}-\mathrm{Proj}_{{\mathcal{P}}^{\prime}})\\{u\\}={\mathrm{A}}\\{u\\}=u^{\ast}\in{\mathcal{U}}^{\prime},$

which is tightly linked to the specification of the underlying spaces in
Proposition 2. Likewise, we find that the quadratic regularization cost
associated with the linear model (59) is
${\boldsymbol{a}}^{\mathsf{T}}{\bf{G}}{\boldsymbol{a}}$, where
${\bf{G}}\in\mathbb{R}^{M\times M}$ is a symmetric, conditionally positive-
definite matrix whose entries are calculated as follows:

$\displaystyle[{\bf{G}}]_{m,n}$ $\displaystyle=\langle{\rm L}_{\rm R}\rho_{\rm
iso}(\cdot-{\boldsymbol{x}}_{m}),{\rm L}_{\rm R}\rho_{\rm
iso}(\cdot-{\boldsymbol{x}}_{n})\rangle_{\rm Rad}$
$\displaystyle=\langle{\mathrm{R}}{\mathrm{K}}{\mathrm{L}}\\{\rho_{\rm
iso}(\cdot-{\boldsymbol{x}}_{m})\\},{\mathrm{R}}{\mathrm{K}}{\mathrm{L}}\\{\rho_{\rm
iso}(\cdot-{\boldsymbol{x}}_{n})\\}\rangle_{\rm Rad}$
$\displaystyle=\langle{\mathrm{K}}{\mathrm{L}}\\{\rho_{\rm
iso}(\cdot-{\boldsymbol{x}}_{m})\\},{\mathrm{R}}^{\ast}{\mathrm{R}}{\mathrm{K}}{\mathrm{L}}\\{\rho_{\rm
iso}(\cdot-{\boldsymbol{x}}_{n})\\}\rangle$ $\displaystyle=\langle\rho_{\rm
iso}(\cdot-{\boldsymbol{x}}_{m}),{\mathrm{L}}^{\ast}{\mathrm{K}}\underbrace{{\mathrm{R}}^{\ast}{\mathrm{R}}{\mathrm{K}}}_{\mathrm{Id}}{\mathrm{L}}\\{\rho_{\rm
iso}(\cdot-{\boldsymbol{x}}_{n})\\}\rangle$ $\displaystyle=\langle\rho_{\rm
iso}(\cdot-{\boldsymbol{x}}_{m}),\delta(\cdot-{\boldsymbol{x}}_{n})\rangle=\rho_{\rm
iso}({\boldsymbol{x}}_{n}-{\boldsymbol{x}}_{m})=\rho_{\rm
iso}({\boldsymbol{x}}_{m}-{\boldsymbol{x}}_{n}).$

As variant of the $L_{2}$ result in Theorem 5, we may also consider the
modified regularization operator $\tilde{\rm
L}={\mathrm{K}}^{-\frac{1}{2}}{\mathrm{L}}$ whose frequency response is
$\widehat{\tilde{L}}({\boldsymbol{\omega}})=\sqrt{2}(2\pi)^{(d-1)/2}\widehat{L}({\boldsymbol{\omega}})/\|{\boldsymbol{\omega}}\|^{(d-1)/2}$.
For this particular setting, $\tilde{{\mathrm{L}}}_{\rm
R}={\mathrm{R}}{\mathrm{K}}^{\frac{1}{2}}{\mathrm{L}}$, which translates into

$\|\tilde{{\mathrm{L}}}_{\rm
R}f\|^{2}_{L_{2}(\mathbb{R}\times\mathbb{S}^{d-1})}=\|{\mathrm{R}}{\mathrm{K}}^{\frac{1}{2}}{\mathrm{L}}f\|^{2}_{L_{2}(\mathbb{R}\times\mathbb{S}^{d-1})}=\|{\mathrm{L}}f\|^{2}_{L_{2}(\mathbb{R}^{d})},$

owing to the property that ${\mathrm{R}}{\mathrm{K}}^{\frac{1}{2}}$ is an
$L_{2}$ isometry [26]. The proposed Radon-domain regularization therefore
reduces to the standard energy functional associated with (semi-)reproducing-
kernel Hilbert spaces. The resulting basis function is $\tilde{\rho}_{\rm
iso}({\boldsymbol{x}})=\mathcal{F}^{-1}\\{1/|\widehat{L}|^{2}\\}({\boldsymbol{x}})$,
which is the same as the one encountered in the classical formulation that
does not involve the Radon transform. While this may suggest that the two
formulations are equivalent, there is an important difference that concerns
the dimension of the null space.

As a matter of illustration, we now compare two schemes that utilize the same
“linear” radial basis $\rho_{\rm
iso}({\boldsymbol{x}})\propto\|{\boldsymbol{x}}\|$. Based on (6), we deduce
that this corresponds to the choice
$|\widehat{L}({\boldsymbol{\omega}})|^{2}=\|{\boldsymbol{\omega}}\|^{d+1}$ in
the classical formulation, which induces the regularization norm
$\|(-\Delta)^{(d+1)/2}\\{f\\}\|_{L_{2}}$ with a polynomial null space of
degree $n_{0}=\lceil(d+1)/2\rceil>d/2$ [17, 58]. In our proposed Radon-domain
variant, the appropriate regularization norm is
$\|{\mathrm{R}}{\mathrm{K}}(-\Delta)^{1/2}\\{f\\}\|_{L_{2}}$ with a null space
of polynomial degree $n_{0}=0$. This solution is attractive because it does
not depend on the dimensionality of the data.

###  4.3 Universal Approximation Properties

The universal-approximation properties of the supervised-learning scheme
specified by (49) are supported by Theorem 6 below, which summarizes the
properties of the native space ${\mathcal{X}}^{\prime}_{{\rm L}_{\rm R}}$ in
relation to the regularization operator ${\rm L}_{\rm R}$ and its generalized
inverse ${\rm L}^{\dagger}_{\rm R}$. This result is a direct corollary (dual
counterpart) of Theorem 9, whose proof is given in Section 5.2.

######  Theorem 6 (Properties of the native space
${\mathcal{X}}^{\prime}_{{\rm L}_{\rm R}}$).

Let ${\rm L}$ be an isotropic LSI operator that is spline-admissible with a
polynomial null space ${\mathcal{P}}$ (possibly trivial) of degree $n_{0}$.
Then, the operators ${\rm L}_{\rm R}={\mathrm{K}}_{\rm
rad}{\mathrm{R}}{\mathrm{L}}:{\mathcal{X}}^{\prime}_{{\rm L}_{\rm
R}}\to{\mathcal{X}}^{\prime}_{\rm Rad}$ and ${\rm L}_{\rm
R}^{\dagger}=(\mathrm{Id}-\mathrm{Proj}_{{\mathcal{P}}}){\mathrm{L}}^{-1}{\mathrm{R}}^{\ast}:{\mathcal{X}}^{\prime}_{\rm
Rad}\to L_{\infty,-n_{0}}(\mathbb{R}^{d})$ (the adjoint of ${\rm L}_{\rm
R}^{\ast\dagger}$ in Theorem 9) are continuous and have the properties

$\displaystyle\forall w\in{\mathcal{X}}^{\prime}_{\rm Rad}:$
$\displaystyle\quad{\rm L}_{\rm R}{\rm L}_{\rm R}^{\dagger}\\{w\\}=w$ (60)
$\displaystyle\forall p_{0}\in{\mathcal{P}}:$ $\displaystyle\quad{\rm L}_{\rm
R}\\{p_{0}\\}=0$ (61) $\displaystyle\forall f\in{\mathcal{X}}^{\prime}_{{\rm
L}_{\rm R}}(\mathbb{R}^{d}):$ $\displaystyle\quad{\rm L}_{\rm R}^{\dagger}{\rm
L}_{\rm
R}\\{f\\}=(\mathrm{Id}-\mathrm{Proj}_{{\mathcal{P}}})\\{f\\}=\mathrm{Proj}_{{\mathcal{U}}^{\prime}}\\{f\\},$
(62)

where ${\mathcal{X}}^{\prime}_{{\rm L}_{\rm
R}}(\mathbb{R}^{d})\stackrel{{\scriptstyle\vartriangle}}{{=}}{\rm
L}^{\dagger}_{\rm R}({\mathcal{X}}^{\prime}_{\rm Rad})\oplus{\mathcal{P}}$ is
equipped with the composite norm $\|f\|_{{\mathcal{X}}^{\prime}_{{\rm L}_{\rm
R}}}=\|{\rm L}_{\rm R}\\{f\\}\|_{{\mathcal{X}}^{\prime}_{\rm
Rad}}+\|\mathrm{Proj}_{{\mathcal{P}}}\\{f\\}\|_{{\mathcal{P}}}$. The space
${\mathcal{X}}^{\prime}_{{\rm L}_{\rm R}}$ is complete and isomorphic to
${\mathcal{X}}^{\prime}_{\rm Rad}\times{\mathcal{P}}$ with $f={\rm
L}^{\dagger}_{\rm R}\\{u\\}+p_{0}\mapsto(w,p_{0})=({\rm L}_{\rm
R}\\{f\\},\mathrm{Proj}_{{\mathcal{P}}}\\{f\\})$. Moreover,
${\mathcal{S}}(\mathbb{R}^{d})\xhookrightarrow{}{\mathcal{X}}^{\prime}_{{\mathrm{L}}_{\rm
R}}(\mathbb{R}^{d})\xhookrightarrow{}L_{\infty,-n_{0}}(\mathbb{R}^{d})\xhookrightarrow{\mbox{\tiny\rm
d.}}{\mathcal{S}}^{\prime}(\mathbb{R}^{d})$.

To explain how Theorem 6 relates to universality, let us first consider the
case ${\mathcal{X}}^{\prime}=L_{2}$ for which we have just shown that
${\mathcal{X}}^{\prime}_{{\rm L}_{\rm R}}={\mathcal{H}}$ is a RKHS whose
(semi-)reproducing kernel is $\rho_{\rm iso}=({\rm L}^{\ast}{\mathrm{K}}{\rm
L})^{-1}\\{\delta\\}$. From the general properties of RKHS [2, 58], we know
that ${\mathcal{H}}$ (as a set) can be specified as
${\mathcal{H}}=\overline{{\rm span}\\{\rho_{\rm
iso}(\cdot-{\boldsymbol{y}})\\}}_{{\boldsymbol{y}}\in\mathbb{R}^{d}}+{\mathcal{P}}$,
which tells us that the class of RBF estimators of the form given by (50) is
dense in ${\mathcal{H}}$. This means that such estimators can yield an
approximation of any $f\in{\mathcal{H}}$ to an arbitrary degree of precision.
In particular, this applies to any $f\in{\mathcal{S}}(\mathbb{R}^{d})$, due to
the inclusion
${\mathcal{S}}(\mathbb{R}^{d})\subset{\mathcal{X}}^{\prime}_{{\mathrm{L}}_{\rm
R}}(\mathbb{R}^{d})$, as guaranteed by Theorem 6. Now, the key to universal
approximation is that ${\mathcal{S}}(\mathbb{R}^{d})$ is dense in most of the
classical function spaces [48], in particular, $C_{0}(\mathbb{R}^{d})$. We
then readily deduce that (50), for $M$ sufficiently large and a suitable
choice of the ${\boldsymbol{x}}_{m}$, has the ability to reproduce any
continuous function $f:\mathbb{R}^{d}\to\mathbb{R}$. This deduction, of
course, is consistent with the classical theory of kernel estimators: Our
admissibility conditions for $\widehat{L}_{\rm rad}$ in Definition 1 (resp.,
in Definition 2) ensure that the function $\rho_{\rm
iso}:\mathbb{R}^{d}\to\mathbb{R}$ is strictly positive-definite (resp.,
strictly conditionally positive-definite), which is the standard criterion for
universality [30, 31, 58].

The same kind of density argument can also be made for
${\mathcal{X}}^{\prime}={\mathcal{M}}$. The relevant basis elements there are
the atomic Radon-compatible Dirac measures $\delta_{{\rm
Rad},(t_{k},{\boldsymbol{\xi}}_{k})}\in{\mathcal{M}}_{\rm Rad}$ with
$(t_{k},{\boldsymbol{\xi}}_{k})\in\mathbb{R}\times\mathbb{S}^{d-1}$. These get
mapped into $e_{k}={\mathrm{L}}^{\dagger}_{\rm R}\\{\delta_{{\rm
Rad},(t_{k},{\boldsymbol{\xi}}_{k})}\\}\in{\mathcal{U}}^{\prime}$, which are
essentially ridges (up to some polynomial) characterized by $e_{k}=\rho_{\rm
rad}({\boldsymbol{\xi}}_{k}^{\mathsf{T}}\cdot-t_{k})-p_{0,k}$ with
$p_{0,k}=\mathrm{Proj}_{{\mathcal{P}}}\\{\rho_{\rm
rad}({\boldsymbol{\xi}}_{k}^{\mathsf{T}}\cdot-t_{k})\\}$. Thus, by setting
$w\approx\sum_{k}w_{k}\delta_{{\rm Rad},(t_{k},{\boldsymbol{\xi}}_{k})}$, we
can interpret the generative formula $f={\mathrm{L}}^{\dagger}_{\rm
R}\\{w\\}+p_{0}$ in Theorem 6 as a linear superposition of ridges plus a
global polynomial trend, which is compatible with the form of the estimator in
(52). We then invoke the property that
${\mathcal{S}}(\mathbb{R}^{d})\subset{\mathcal{M}}_{{\mathrm{L}}_{\rm
R}}(\mathbb{R}^{d})$, which implies that any continuous function can be
approximated as closely as desired by a member of
${\mathcal{M}}_{{\mathrm{L}}_{\rm R}}(\mathbb{R}^{d})$. As it turns out, the
latter is representable by a superposition of ridges plus a polynomial of
degree $n_{0}$. We emphasize that the presence of the polynomial term—the
guarantor of stability for Theorem 8—is essential to counterbalance the growth
of the individual atoms at infinity. This is an important aspect where our
analysis and conclusions deviate from those of [49].

###  4.4 Regularization Operators for Antisymmetric Activations

While the framework that has been discussed so far is very powerful, it has
one shortcoming: it yields canonical activation functions $\rho_{\rm rad}$
that are necessarily symmetric. In some cases such as ${\rm L}_{\rm
R}={\mathrm{K}}_{\rm rad}{\mathrm{R}}(-\Delta)$, these can be converted into
one-sided functions such as the ReLU by doctoring the polynomial term. That
said, the original scheme that is described by Theorem 5 does not allow for
sigmoids, which are frequently used for classification [8]. This is the reason
why we now introduce a variant that systematically produces anti-symmetric
activations, including sigmoids for $n_{0}=0$.

The idea is to substitute the (symmetric) filtering operator
${\mathrm{K}}_{\rm rad}$ by its anti-symmetric counterpart
$\tilde{{\mathrm{K}}}_{\rm rad}$, which includes an additional Hilbert
transform. Specifically, $\tilde{{\mathrm{K}}}_{\rm rad}$ is the hyper-
spherical radial filter whose frequency response is $\widehat{\tilde{K}}_{\rm
rad}(\omega)=-\mathrm{i}\,{\rm sign}(\omega)c_{p}|\omega|^{-d-1}$ and whose
adjoint is $\tilde{{\mathrm{K}}}^{\ast}_{\rm rad}=-\tilde{{\mathrm{K}}}_{\rm
rad}$. Since the action of the (radial) Hilbert transform
$\tilde{{\mathrm{H}}}_{\rm
rad}:\phi(\cdot,{\boldsymbol{\xi}})\mapsto\phi\circledast 1/(\pi t)$ is well-
defined on ${\mathcal{S}}(\mathbb{R}\times\mathbb{S}^{d-1})$ with
$\tilde{{\mathrm{H}}}^{\ast}_{\rm rad}=-\tilde{{\mathrm{H}}}_{\rm
rad}=\tilde{{\mathrm{H}}}^{-1}_{\rm rad}$, we have the identity
${\mathrm{R}}^{\ast}{\mathrm{K}}_{\rm
rad}{\mathrm{R}}={\mathrm{R}}^{\ast}{\mathrm{K}}_{\rm
rad}\tilde{{\mathrm{H}}}_{\rm rad}^{\ast}\tilde{{\mathrm{H}}}_{\rm
rad}{\mathrm{R}}={\mathrm{R}}^{\ast}\tilde{{\mathrm{K}}}_{\rm
rad}^{\ast}\tilde{{\mathrm{R}}}=\mathrm{Id}$ with
$\tilde{{\mathrm{R}}}\stackrel{{\scriptstyle\vartriangle}}{{=}}\tilde{{\mathrm{H}}}_{\rm
rad}{\mathrm{R}}$. Accordingly, we can essentially replicate the whole
mechanism of construction of spaces in Section 3.3 by subtituting
${\mathcal{S}}_{\rm Rad}$ by $\tilde{{\mathcal{S}}}_{\rm
Rad}=\tilde{{\mathrm{R}}}({\mathcal{S}}(\mathbb{R}\times\mathbb{S}^{d-1})=\tilde{{\mathrm{H}}}_{\rm
rad}({\mathcal{S}}_{\rm Rad})$, which is a space of odd functions that are
smooth ($C^{\infty}$) and included in
$L_{p}(\mathbb{R}\times\mathbb{S}^{d-1})$ for all $p\geq 1$. While the members
of $\tilde{{\mathcal{S}}}_{\rm Rad}$ do not necessarily decay rapidly, the
mapping ${\mathrm{R}}^{\ast}\tilde{{\mathrm{K}}}^{\ast}_{\rm
rad}:\tilde{{\mathcal{S}}}_{\rm Rad}\to{\mathcal{S}}(\mathbb{R}^{d})$ is still
guaranted to be an isomorphism (see [41, Theorem 3.3.1, p. 83] where similar
arguments are used). Under the assumption that the hyper-spherical norm
$\|\cdot\|_{{\mathcal{X}}}$ is continuous on $\tilde{{\mathcal{S}}}_{\rm
Rad}$, we can readily adapt the proof of Theorem 3 to establish the following.

######  Proposition 3 (Odd Radon-compatible Banach spaces).

Consider the Banach space $\tilde{{\mathcal{X}}}_{\rm
Rad}=\overline{(\tilde{{\mathcal{S}}}_{\rm Rad},\|\cdot\|_{{\mathcal{X}}})}$
of odd hyper-spherical functions. Then, the following holds.

  1. 1.

The map ${\mathrm{R}}^{\ast}\tilde{{\mathrm{K}}}^{\ast}_{\rm
rad}:\tilde{{\mathcal{X}}}_{\rm
Rad}\to\tilde{{\mathcal{Y}}}={\mathrm{R}}^{\ast}\tilde{{\mathrm{K}}}^{\ast}_{\rm
rad}\big{(}\tilde{{\mathcal{X}}}_{\rm Rad}\big{)}$ is an isometric bijection,
with $\tilde{{\mathrm{R}}}{\mathrm{R}}^{\ast}\tilde{{\mathrm{K}}}^{\ast}_{\rm
rad}=\mathrm{Id}$ on $\tilde{{\mathcal{X}}}_{\rm Rad}$.

  2. 2.

The map $\tilde{{\mathrm{R}}}^{\ast}:\tilde{{\mathcal{X}}}^{\prime}_{\rm
Rad}\to\tilde{{\mathcal{Y}}}^{\prime}=\tilde{{\mathrm{R}}}^{\ast}\big{(}\tilde{{\mathcal{X}}}^{\prime}_{\rm
Rad}\big{)}$ is an isometric bijection, with $\tilde{{\mathrm{K}}}_{\rm
rad}{\mathrm{R}}\tilde{{\mathrm{R}}}^{\ast}=\mathrm{Id}$ on
$\tilde{{\mathcal{X}}}^{\prime}_{\rm Rad}$.

The underlying definition of the “oddified” backprojection operator
$\tilde{{\mathrm{R}}}^{\ast}:\tilde{{\mathcal{X}}}^{\prime}_{\rm
Rad}\to\tilde{{\mathcal{Y}}}^{\prime}$ for $g\in\tilde{{\mathcal{X}}}_{\rm
Rad}^{\prime}$ is

$\displaystyle\langle\tilde{{\mathrm{R}}}^{\ast}\\{g\\},\varphi\rangle=\langle
g,\tilde{{\mathrm{R}}}\\{\varphi\\}\rangle_{\rm
Rad}=\langle\tilde{{\mathrm{H}}}^{\ast}_{\rm
rad}\\{g\\},{\mathrm{R}}\\{\varphi\\}\rangle_{\rm Rad}$ (63)

for all $\varphi\in\tilde{{\mathcal{Y}}}$ or, equivalently,
$\varphi\in{\mathcal{S}}(\mathbb{R}^{d})$ since
${\mathcal{S}}(\mathbb{R}^{d})$ is dense in ${\mathcal{Y}}$ by construction.
Likewise, by using the property that the Hilbert transform is a homeomorphism
from ${\mathcal{S}}_{\rm Liz,even}$ onto ${\mathcal{S}}_{\rm Liz,odd}$ [44]
with the underlying Lizorkin spaces being included in ${\mathcal{S}}_{\rm
Rad}$ and $\tilde{{\mathcal{S}}}_{\rm Rad}\subset L_{p,{\rm odd}}$,
respectively, we can adapt the proof of Lemma 2 to show that
$\tilde{L}_{q,{\rm Rad}}=L_{q,{\rm odd}}$ for $q\in(1,\infty)$,
$\tilde{C}_{0,{\rm Rad}}=C_{0,{\rm odd}}$ for $p=\infty$, and
$\tilde{{\mathcal{M}}}_{{\rm Rad}}=(\tilde{C}_{0,{\rm
Rad}})^{\prime}={\mathcal{M}}_{{\rm odd}}$.

The bottom line is that the whole argumentation, including the critical
Fourier-based proofs of Section 5, applies in this modified setting as well.
Accordingly, all theorems that mention the regularization operator ${\rm
L}_{\rm R}={\mathrm{K}}_{\rm rad}{\mathrm{R}}{\rm L}$ and the radial profile
$\rho_{\rm rad}=\mathcal{F}^{-1}\\{1/\widehat{L}_{\rm rad}\\}$ are also valid
for the odd setting where these quantifies are substituted by

$\displaystyle\tilde{{\mathrm{L}}}_{\rm R}$
$\displaystyle\stackrel{{\scriptstyle\vartriangle}}{{=}}\tilde{{\mathrm{K}}}_{\rm
rad}{\mathrm{R}}{\mathrm{L}}=\tilde{{\mathrm{K}}}_{\rm rad}{\mathrm{L}}_{\rm
rad}{\mathrm{R}},$ (64) $\displaystyle\tilde{\rho}_{\rm rad}(t)$
$\displaystyle\stackrel{{\scriptstyle\vartriangle}}{{=}}\mathcal{F}^{-1}\left\\{\frac{\mathrm{i}\,{\rm
sign}(\omega)}{\widehat{L}_{\rm rad}(\omega)}\right\\}(t),$ (65)

where (65) directly follows from (63) and Theorem 4. The conditions for
admissibility in Definitions 1 and 2, which are all Fourier-based, remain the
same, while the adjusted activation $\tilde{\rho}_{\rm rad}$ (the 1D Hilbert
transform of $\rho_{\rm rad}$) is real-valued and anti-symmetric because the
original Fourier profile $\widehat{L}_{\rm rad}:\mathbb{R}\to\mathbb{R}$ is
symmetric.

We note that our admissibility condition with $\gamma_{0}=1$ for odd variant
($n_{0}=0$) is compatible with the condition used by Barron to prove the
universality of neural networks with sigmoidal activations [5]. It is also
worth mentioning that the substitution of ${\mathrm{L}}_{\rm R}$ by
$\tilde{{\mathrm{L}}}_{\rm R}$ has no incidence on the form of the RBF in (50)
because of the unitary nature of the Hilbert transform.

###  4.5 Specific Configurations

The proposed framework encompasses a wide variety of kernels and activation
functions, with minimal restrictions. For instance, one can start with any
strictly positive-definite function $\rho_{{\rm rad},0}$ whose Fourier
transform is strictly positive, and construct some higher-order variants by
(fractional) integration. The variants are such that $\rho_{{\rm
rad},\gamma_{0}}(t)=\mathcal{F}^{-1}\\{\frac{\widehat{\rho}_{{\rm
rad},0}(\omega)}{|\omega|^{\gamma_{0}}}\\}(t)$ with suitable $\gamma_{0}>0$
and are guaranteed to meet the requirements in Definition 2. The simplest
scenario is to set $\rho_{{\rm rad},0}=\delta$, which maps into a Laplacian-
type regularization.

In Table 1, we provide examples of admissible operators together with their
corresponding symmetric and anti-symmetric activations. It is noteworthy that
the two most popular sigmoids (${\rm tanh}$ and ${\rm arctan}$) are part of
the framework. We can determine the explicit frequency response of their
regularization filter and show that it is first-order-admissible with a null
space that consists of the polynomials of degree $n_{0}=0$ (the constants).
The symmetric spline activations of odd degree $m-1$ and the anti-symmetric
ones of even degree are also known: they coincide with the ridge splines of
Parhi and Novak, which are tied to the Radon-domain operator ${\rm L}_{\rm
rad}=\frac{\partial^{m}}{\partial t^{m}}$ [36]. With the present formulation,
we can seamlessly extend this family to fractional orders, in direct analogy
with [54], by setting ${\rm L}=(-\Delta)^{\frac{\alpha+1}{2}}$.

$\rho_{\rm rad}(t)$ |  $\tilde{\rho}_{\rm rad}(t)$ |  $\widehat{L}_{\rm rad}$ or $\widehat{\tilde{L}}_{\rm rad}$ |  $n_{0}$ (null space)
---|---|---|---
Exponential |  |  |
$\mathrm{e}^{-|t|}$ |  N/A |  $1+|\omega|^{2}$ |  $-1$ (trivial)
|  Classical sigmoids |  |
|  $\frac{\tanh(\tfrac{t}{2})}{2}=\tfrac{1}{2}+\frac{1}{1+\mathrm{e}^{-t}}$ |  $\frac{\sinh(\pi\omega)}{\pi}$ |  $0$ (bias)
|  $\frac{\arctan(t)}{\pi}$ |  $\omega\mathrm{e}^{|\omega|}$ |  $0$
Ridge splines |  (of degree $n\in\mathbb{N}$) |  |
$\tfrac{1}{2}|t|$ (or ReLU) |  ${\color[rgb]{0,0,0}t\log|t|}$ |  $|\omega|^{2}$ |  $1$ (affine)
$\propto{\color[rgb]{0,0,0}t^{2n}\log|t|}$ |  $\frac{{\rm sign}(t)|t|^{2n}}{(2n)!}$ |  $|\omega|^{2n+1}$ |  $2n\geq 2$ (even)
$\tfrac{1}{2}\frac{|t|^{2n+1}}{(2n+1)!}$ |  $\propto t^{2n+1}\log|t|$ |  $|\omega|^{2n+2}$ |  $2n+1\geq 1$ (odd)
Fractional splines |  (degree $\alpha\in\mathbb{R}^{+}\backslash\mathbb{N}$) |  |
$\color[rgb]{0,0,0}\frac{|t|^{\alpha}\sin(\frac{\alpha\pi}{2})}{\pi\Gamma(\alpha)}$ |  $\color[rgb]{0,0,0}\frac{{\rm sign}(t)|t|^{\alpha}\cos(\frac{\alpha\pi}{2})}{\pi\Gamma(\alpha)}$ |  $|\omega|^{\alpha+1}$ |  $\lceil\alpha\rceil$
Table 1: Examples of admissible symmetric and anti-symmetric activation
functions with their corresponding regularization operator. The anti-symmetric
activation is given by (65) and requires the use of the modified filter
$\tilde{{\mathrm{K}}}_{\rm rad}$ in the statement of Theorem 5.

##  5 Supporting Mathematical Results

To answer the fundamental issue of the existence of a solution in Theorem 5,
we need to 1) prove that the “predual” space ${\mathcal{X}}_{{\rm
L}}(\mathbb{R}^{d})$ is a proper Banach space (Theorem 9); and 2) establish
the weak* continuity of the sampling functionals
$\delta(\cdot-{\boldsymbol{x}}_{k})$. As we shall see, both aspects largely
rest upon the functional characterization of the Schwartz kernel of the
pseudoinverse operator ${\mathrm{L}}_{\rm R}^{\ast\dagger}$ provided in
Theorem 7.

###  5.1 Kernel and Stability of Generalized Inverse Operators

Let $\nu:f\mapsto\langle\nu,f\rangle$ be a linear functional that is acting on
some Banach space ${\mathcal{X}}^{\prime}$. We recall that $\nu$ is
weak*-continuous if and only if $\nu\in{\mathcal{X}}$, where ${\mathcal{X}}$
is the predual of ${\mathcal{X}}^{\prime}$ [42]. The condition that
${\mathcal{X}}^{\prime}$ is reflexive (i.e.,
${\mathcal{X}}^{\prime\prime}={\mathcal{X}}$) is equivalent to the continuity
of $\nu$ on ${\mathcal{X}}$, which is the standard condition for analysis.
However, when ${\mathcal{X}}^{\prime}$ is not reflexive (e.g.,
${\mathcal{X}}^{\prime}={\mathcal{M}}=(C_{0})^{\prime}$), the constraint of
weak* continuity is stronger than continuity, contrary to what could be
suggested by the qualifier “weak.” In that case, the predual space
${\mathcal{X}}\subseteq{\mathcal{X}}^{\prime\prime}$, which is continuously
embedded in ${\mathcal{X}}^{\prime\prime}$, turns out to be smaller than
${\mathcal{X}}^{\prime\prime}$. Therefore, in order to establish the weak*
continuity of the Dirac functionals $\delta(\cdot-{\boldsymbol{x}}_{0})$ for
the scenarios in Theorem 5, we need to show that
$\delta(\cdot-{\boldsymbol{x}}_{0})\in{\mathcal{X}}_{{\mathrm{L}}_{\rm R}}$,
which can be reduced to proving that ${\rm L}_{\rm
R}^{\ast\dagger}\\{\delta(\cdot-{\boldsymbol{x}}_{0})\\}\in{\mathcal{X}}_{\rm
Rad}$.

######  Theorem 7 (Properties of the impulse response of ${\mathrm{L}}_{\rm
R}^{\ast\dagger}={\mathrm{R}}{\mathrm{L}}_{\mathcal{P}}^{-1\ast}$).

Let ${\rm L}$ be an isotropic operator such that
$\widehat{L}({\boldsymbol{\omega}})=\widehat{L}_{\rm
rad}(\|{\boldsymbol{\omega}}\|)$ where $\widehat{L}_{\rm
rad}:\mathbb{R}\to\mathbb{R}$ is a continuous function and $\rho_{\rm
rad}(t)=\mathcal{F}^{-1}\\{1/\widehat{L}_{\rm rad}\\}(t)$. We consider two
cases:

  1. 1.

Trivial null space: If ${\rm L}$ satisfies the admissibility conditions in
Definition 1, then ${\mathrm{L}}_{\rm
R}^{\ast\dagger}={\mathrm{R}}{\mathrm{L}}^{-1\ast}$ and

$\displaystyle\nu_{{\boldsymbol{x}}_{0}}(t,{\boldsymbol{\xi}})$
$\displaystyle={\mathrm{R}}{\mathrm{L}}^{-1\ast}\\{\delta(\cdot-{\boldsymbol{x}}_{0})\\}(t,{\boldsymbol{\xi}})=\rho_{\rm
rad}(t-{\boldsymbol{\xi}}^{\mathsf{T}}{\boldsymbol{x}}_{0})$ (66)

with ${\boldsymbol{x}}_{0}\in\mathbb{R}^{d}$ and
$(t,{\boldsymbol{\xi}})\in\mathbb{R}\times\mathbb{S}^{d-1}$. Moreover,
$\nu_{{\boldsymbol{x}}_{0}}\in{\mathcal{X}}_{\rm Rad}$ for
${\mathcal{X}}=C_{0}$ as well as ${\mathcal{X}}=L_{q}$ with $q\in[1,\infty]$.

  2. 2.

Nontrivial null space: If ${\rm L}$ satisfies the admissibility conditions in
Definition 2 with a polynomial null space of degree $n_{0}$, then

$\displaystyle\nu_{{\boldsymbol{x}}_{0}}(t,{\boldsymbol{\xi}})$
$\displaystyle={\rm L}_{\rm
R}^{\ast\dagger}\\{\delta(\cdot-{\boldsymbol{x}}_{0})\\}(t,{\boldsymbol{\xi}})$
$\displaystyle=\rho_{\rm
rad}(t-{\boldsymbol{\xi}}^{\mathsf{T}}{\boldsymbol{x}}_{0})-\sum_{n=0}^{n_{0}}\frac{(-{\boldsymbol{\xi}}^{\mathsf{T}}{\boldsymbol{x}}_{0})^{n}}{n!}\big{(}\kappa_{\rm
rad}\ast\partial^{n}\rho_{\rm rad}\big{)}(t)$ (67)

with $\nu_{{\boldsymbol{x}}_{0}}\in{\mathcal{X}}_{\rm Rad}$ for the same
spaces as in Item 1, but with $q\in[2,\infty]$. Moreover,

$\displaystyle\sup_{({\boldsymbol{x}}_{0},{\boldsymbol{\xi}})\in\mathbb{R}^{d}\times\mathbb{S}^{d-1}}(1+|{\boldsymbol{\xi}}^{\mathsf{T}}{\boldsymbol{x}}_{0}|)^{-n_{0}}\|\nu_{{\boldsymbol{x}}_{0}}(\cdot,{\boldsymbol{\xi}})\|_{L_{q}(\mathbb{R})}<\infty$
(68)

for any $q\in[2,\infty]$.

###### Proof.

To show that $\nu_{{\boldsymbol{x}}_{0}}\in{\mathcal{X}}_{\rm Rad}$, it is
sufficient to prove that
$\nu_{{\boldsymbol{x}}_{0}}\in{\mathcal{X}}(\mathbb{R}^{d}\times\mathbb{S}^{d-1})$
since $\nu_{{\boldsymbol{x}}_{0}}$ is in the range of the Radon transform by
construction.

When the null space of ${\rm L}$ is trivial, ${\rm L}$ has a stable
convolution inverse so that it suffices to show that
$\nu_{{\boldsymbol{x}}_{0}}\in L_{q}(\mathbb{R}\times\mathbb{S}^{d-1})\cap
C_{0}(\mathbb{R}\times\mathbb{S}^{d-1})$. To that end, we formally identify
the isotropic distribution $\rho_{\rm
iso}={\mathrm{L}}^{-1\ast}\\{\delta\\}={\mathrm{L}}^{-1}\\{\delta\\}=\mathcal{F}^{-1}\\{1/\widehat{L}_{\rm
rad}(\|{\boldsymbol{\omega}}\|)\\}$ and apply Proposition 1, which yields

$\nu_{{\boldsymbol{x}}_{0}}(t,{\boldsymbol{\xi}})={\mathrm{R}}\\{\rho_{\rm
iso}(\cdot-{\boldsymbol{x}}_{0})\\}(t,{\boldsymbol{\xi}})=\rho_{\rm
rad}(t-{\boldsymbol{\xi}}^{\mathsf{T}}{\boldsymbol{x}}_{0}).$

Due to our assumptions, this function is such that
$\|\nu_{{\boldsymbol{x}}_{0}}(\cdot,{\boldsymbol{\xi}})\|_{L_{1}}=\|\rho_{\rm
rad}\|_{L_{1}}<\infty$ for any fixed ${\boldsymbol{\xi}}\in\mathbb{S}^{d-1}$.
Moreover, since $1/\hat{L}_{\rm rad}\in L_{1}(\mathbb{R})$, $\rho_{\rm rad}$
is bounded, continuous, and vanishing at infinity (by the Riemann-Lebesgue
lemma), which gives $\rho_{\rm
rad}(\cdot-{\boldsymbol{\xi}}^{\mathsf{T}}{\boldsymbol{x}}_{0})\in
C_{0}(\mathbb{R})$ for any ${\boldsymbol{x}}_{0}\in\mathbb{R}^{d}$ and
${\boldsymbol{\xi}}\in\mathbb{S}^{d-1}$. Since $\rho_{\rm rad}\in
L_{\infty}(\mathbb{R})\cap L_{1}(\mathbb{R})$, we readily deduce that
$\rho_{\rm rad}(\cdot-{\boldsymbol{\xi}}^{\mathsf{T}}{\boldsymbol{x}}_{0})\in
L_{q}(\mathbb{R})$ for all intermediate $q\geq 1$, which then also yields
$\rho_{\rm rad}(t-{\boldsymbol{\xi}}^{\mathsf{T}}{\boldsymbol{x}}_{0})\in
L_{q}(\mathbb{R}\times\mathbb{S}^{d-1})$ because the surface of the unit
sphere $\mathbb{S}^{d-1}$ is bounded. Finally, since $\rho_{\rm
rad}:\mathbb{R}\to\mathbb{R}$ is continuous and vanishing at infinity,
$\nu_{{\boldsymbol{x}}_{0}}(t,{\boldsymbol{\xi}})$ is jointly continuous in
$(t,{\boldsymbol{\xi}})$ and vanishing at $t\to\pm\infty$ for all
${\boldsymbol{\xi}}\in\mathbb{S}^{d-1}$, which implies that
$\nu_{{\boldsymbol{x}}_{0}}\in C_{0}(\mathbb{R}\times\mathbb{S}^{d-1})$.

For the more difficult case of a nontrivial null space, we invoke the Fourier-
slice theorem to evaluate the 1D Fourier transform of
$\nu_{{\boldsymbol{x}}_{0}}(\cdot,{\boldsymbol{\xi}})$ with
${\boldsymbol{\xi}}$ fixed as

$\displaystyle\hat{\nu}_{{\boldsymbol{x}}_{0}}(\omega,{\boldsymbol{\xi}})$
$\displaystyle=\frac{\mathcal{F}\\{\delta(\cdot-{\boldsymbol{x}}_{0})-\sum_{|{\boldsymbol{k}}|\leq
n_{0}}\langle
m_{\boldsymbol{k}},\delta(\cdot-{\boldsymbol{x}}_{0})\rangle\;m_{\boldsymbol{k}}^{\ast}\\}(\omega{\boldsymbol{\xi}})}{\widehat{L}_{\rm
rad}(\omega)}$
$\displaystyle=\frac{\mathrm{e}^{-\mathrm{i}\omega{\boldsymbol{\xi}}^{\mathsf{T}}{\boldsymbol{x}}_{0}}-\sum_{|{\boldsymbol{k}}|\leq
n_{0}}\frac{{\boldsymbol{x}}_{0}^{\boldsymbol{k}}}{{\boldsymbol{k}}!}\;\widehat{m}_{\boldsymbol{k}}^{\ast}(\omega{\boldsymbol{\xi}})}{\widehat{L}_{\rm
rad}(\omega)}$
$\displaystyle=\frac{\mathrm{e}^{-\mathrm{i}\omega{\boldsymbol{\xi}}^{\mathsf{T}}{\boldsymbol{x}}_{0}}-\sum_{|{\boldsymbol{k}}|\leq
n_{0}}\frac{{\boldsymbol{x}}_{0}^{\boldsymbol{k}}}{{\boldsymbol{k}}!}\;(-\mathrm{i}\omega{\boldsymbol{\xi}})^{{\boldsymbol{k}}}\widehat{\kappa}_{\rm
rad}(\omega)}{\widehat{L}_{\rm rad}(\omega)}$
$\displaystyle=\frac{\mathrm{e}^{-\mathrm{i}\omega{\boldsymbol{\xi}}^{\mathsf{T}}{\boldsymbol{x}}_{0}}-\sum_{n=0}^{n_{0}}\frac{(-{\boldsymbol{\xi}}^{\mathsf{T}}{\boldsymbol{x}}_{0})^{n}}{n!}(\mathrm{i}\omega)^{n}\widehat{\kappa}_{\rm
rad}(\omega)}{\widehat{L}_{\rm rad}(\omega)},$ (69)

where the simplification of the summation over ${\boldsymbol{k}}$ results from
the multinomial expansion
$(-\mathrm{i}\omega{\boldsymbol{\xi}}^{\mathsf{T}}{\boldsymbol{x}}_{0})^{n}=(y_{1}+\cdots+y_{d})^{n}=\sum_{|{\boldsymbol{k}}|=n}\frac{n!}{{\boldsymbol{k}}!}{\boldsymbol{y}}^{{\boldsymbol{k}}}$
with
${\boldsymbol{y}}={\big{(}-\mathrm{i}\omega\xi_{i}x_{0,i}\big{)}_{i=1}^{d}}$.
The delicate point here is that (69) is potentially singular because it has a
pole of multiplicity $\gamma_{0}$ at $\omega=$0\. Fortunately, the condition
$\gamma_{0}\leq n_{0}+1$ ensures that there is a proper pole-zero
cancellation: By recalling that $\widehat{\kappa}_{\rm rad}(\omega)=1$ for
$\omega\in\Omega_{0}=[-R_{0},R_{0}]$ with $R_{0}=\frac{1}{2}$ and setting
$t_{0}={\boldsymbol{\xi}}^{\mathsf{T}}{\boldsymbol{x}}_{0}$, we identify the
numerator as the remainder of the Maclaurin series of
$\mathrm{e}^{-\mathrm{i}\omega t_{0}}$, which is bounded by

$\displaystyle\left|\mathrm{e}^{-\mathrm{i}\omega
t_{0}}-\sum_{n=0}^{N}\frac{(-\mathrm{i}t_{0})^{n}}{n!}\omega^{n}\right|$
$\displaystyle\leq\sup_{\omega\in\mathbb{R}}\left|(-\mathrm{i}t_{0})^{N+1}\mathrm{e}^{-\mathrm{i}\omega
t_{0}}\right|\;\frac{|\omega|^{N+1}}{(N+1)!}$
$\displaystyle\leq\frac{|t_{0}|^{N+1}}{(N+1)!}\;|\omega|^{N+1}.$ (70)

This then yields the estimate

$\displaystyle|\hat{\nu}_{{\boldsymbol{x}}_{0}}(\omega,{\boldsymbol{\xi}})|$
$\displaystyle\leq\frac{|{\boldsymbol{\xi}}^{\mathsf{T}}{\boldsymbol{x}}_{0}|^{n_{0}+1}}{C_{0}(n_{0}+1)!}|\omega|^{\epsilon}\quad\mbox{
as~}\omega\to 0$ $\displaystyle\mbox{ with }\epsilon=n_{0}+1-\gamma_{0}$
$\displaystyle=\begin{cases}0,&\gamma_{0}\in\mathbb{N}\\\
1-(\gamma_{0}-\lfloor\gamma_{0}\rfloor)\in(0,1),&\mbox{otherwise}.\end{cases}$
(71)

Since the denominator $\widehat{L}_{\rm rad}$ in (69) is continuous and non-
vanishing away from the origin,
$\hat{\nu}_{{\boldsymbol{x}}_{0}}(\cdot,{\boldsymbol{\xi}})$ is bounded on
$\Omega_{0}$, and, by extension, on any compact subset of $\mathbb{R}$.
Moreover, since
$|\mathrm{e}^{-\mathrm{i}\omega{\boldsymbol{\xi}}^{\mathsf{T}}{\boldsymbol{x}}_{0}}|=1$
and $\widehat{\kappa}_{\rm rad}$ is rapidly decreasing, there exists a
constant $C$ such that
$|\hat{\nu}_{{\boldsymbol{x}}_{0}}(\omega,{\boldsymbol{\xi}})|\leq
C|\omega|^{-\gamma_{1}}$ for $|\omega|>R_{1}$. This leads to several
conclusions. First, if $\gamma_{1}>1$, then
$\hat{\nu}_{{\boldsymbol{x}}_{0}}(\cdot,{\boldsymbol{\xi}})\in
L_{1}(\mathbb{R})$ so that $\nu_{{\boldsymbol{x}}_{0}}\in C_{0,\rm
Rad}(\mathbb{R}\times\mathbb{S}^{d-1})$ by the same argument as in the
nonsingular case. Second, if $\gamma_{1}>\alpha+\tfrac{1}{2}$, then
$\nu_{{\boldsymbol{x}}_{0}}(\cdot,{\boldsymbol{\xi}})\in
W_{2}^{\alpha}(\mathbb{R})$, where
$W_{2}^{\alpha}(\mathbb{R})=\\{f:\int_{\mathbb{R}}(1+|\omega|^{2})^{\alpha}|\hat{f}(\omega)|^{2}{\rm
d}\omega<\infty\\}$ is the Sobolev space of functions with finite-energy
derivatives up to order $\alpha$. Therefore, $\nu_{{\boldsymbol{x}}_{0}}\in
L_{q,\rm Rad}(\mathbb{R}\times\mathbb{S}^{d-1})$ for all $q\in[2,\infty]$
provided that $\gamma_{1}>1$. The explicit Radon-domain formula (67) with
$\kappa_{\rm rad}(t)=\mathcal{F}^{-1}\\{\widehat{\kappa}_{\rm
rad}(\omega)\\}(t)$ is obtained by taking the 1D inverse transform of (69).

Figure 1: Functions $\omega\mapsto|r_{N}(\omega)|$ for $N=1,\dots,10$. The
version on the right includes the upper bound specified by (74) as an overlay.

To refine our characterization of $\hat{\nu}_{{\boldsymbol{x}}_{0}}$, we
introduce the function

$\displaystyle
r_{N}(\omega)=\frac{\mathrm{e}^{-\mathrm{i}\omega}-\sum_{n=0}^{N}\frac{(-\mathrm{i}\omega)^{n}}{n!}}{\frac{(\mathrm{i}\omega)^{N}}{N!}}$
(72)

whose modulus is plotted in Figure 1. Some of the remarkable properties of
$r_{N}$ are

$\displaystyle r_{N}(\omega)=\frac{-\mathrm{i}\omega}{N+1}\ \mbox{ as }\
\omega\to 0$ (73)
$\displaystyle\forall\omega\in\mathbb{R}:\quad\left|r_{N}(\omega)\right|\leq\min(|\omega|/2,1.27)$
(74) $\displaystyle\lim_{\omega\to\infty}\left|r_{N}(\omega)\right|=1,$ (75)

with the global bound (74) being overlaid on the graph to demonstrate its
sharpness. This function will allows us to control the behavior of

$\displaystyle\hat{\nu}_{{\boldsymbol{x}}_{0}}(\omega,{\boldsymbol{\xi}})$
$\displaystyle=\frac{\mathrm{e}^{-\mathrm{i}t_{0}\omega}-\sum_{n\leq
n_{0}}\frac{(-\mathrm{i}t_{0}\omega)^{n}}{n!}\kappa_{\rm
rad}(\omega)}{\widehat{L}_{\rm rad}(\omega)}$
$\displaystyle=\frac{\widehat{\kappa}_{\rm
rad}(\omega)r_{n_{0}}(t_{0}\omega)\frac{(-\mathrm{i}t_{0}\omega)^{n_{0}}}{n_{0}!}+\big{(}1-\widehat{\kappa}_{\rm
rad}(\omega)\big{)}\mathrm{e}^{-\mathrm{i}t_{0}\omega}}{\widehat{L}_{\rm
rad}(\omega)}$

by splitting the frequency domain in three regions. First, for
$\omega\in\Omega_{0}=[0,R_{0}]$, where $\widehat{\kappa}_{\rm rad}(\omega)=1$,
we find that

$\displaystyle|\hat{\nu}_{{\boldsymbol{x}}_{0}}(\omega,{\boldsymbol{\xi}})|$
$\displaystyle=\left|\frac{r_{n_{0}}(t_{0}\omega)\frac{(-\mathrm{i}t_{0}\omega)^{n_{0}}}{n_{0}!}}{\widehat{L}_{\rm
rad}(\omega)}\right|\leq|t_{0}|^{n_{0}}\min{(|\omega|,2)}\frac{2\frac{|\omega|^{n_{0}}}{n_{0}!}}{|\widehat{L}_{\rm
rad}(\omega)|}.$ (76)

For the transition region $\omega\in\Omega_{01}=[R_{0},\tilde{R}_{1}]$ with
$\tilde{R}_{1}=\max(2R_{0},R_{1})$, where $0\leq\widehat{\kappa}_{\rm
rad}(\omega)\leq 1$, we bound the numerator by its maximum, which yields

$\displaystyle|\hat{\nu}_{{\boldsymbol{x}}_{0}}(\omega,{\boldsymbol{\xi}})|$
$\displaystyle\leq\frac{2\frac{|t_{0}\tilde{R}_{1}|^{n_{0}}}{n_{0}!}+1}{|\widehat{L}_{\rm
rad}(\omega)|}.$ (77)

Finally, for $\omega\in\Omega_{1}=[\tilde{R}_{1},\infty]$, where
$\widehat{\kappa}_{\rm rad}(\omega)=0$, we get the expected tail behavior

$\displaystyle|\hat{\nu}_{{\boldsymbol{x}}_{0}}(\omega,{\boldsymbol{\xi}})|$
$\displaystyle\leq\frac{1}{|\widehat{L}_{\rm
rad}(\omega)|}\leq\frac{1}{C_{1}|\omega|^{\gamma_{1}}}.$ (78)

Based on those bounds with $t_{0}=1$, we define the auxiliary function

$\displaystyle
u(\omega)=\begin{cases}\min{(|\omega|,2)}\frac{2\frac{|\omega|^{n_{0}}}{n_{0}!}}{|\widehat{L}_{\rm
rad}(\omega)|},&|\omega|<R_{0}\\\
\frac{2\frac{\tilde{R}_{1}^{n_{0}}}{n_{0}!}+1}{|\widehat{L}_{\rm
rad}(\omega)|},&R_{0}\leq|\omega|\leq\tilde{R}_{1}\\\
\frac{1}{|\widehat{L}_{\rm rad}(\omega)|},&\tilde{R}_{1}<|\omega|\end{cases}$
(79)

which, by construction, is such that $\|u\|_{L_{p}}<\infty$ for any $p\geq 1$.
We can now use (76), (77), and (78) to bound the norm of
$\hat{\nu}_{{\boldsymbol{x}}_{0}}(\cdot,{\boldsymbol{\xi}})$ by distinguishing
between two cases. For $t_{0}\leq 1$, we have
$\|\hat{\nu}_{{\boldsymbol{x}}_{0}}(\cdot,{\boldsymbol{\xi}})\|_{L_{p}}\leq\|u\|_{L_{p}}$,
while, for $t_{0}\geq 1$, we get
$\|\hat{\nu}_{{\boldsymbol{x}}_{0}}(\cdot,{\boldsymbol{\xi}})\|_{L_{p}}\leq|t_{0}|^{n_{0}}\|u\|_{L_{p}}$.
By combining these two inequalities, we obtain the universal norm estimate

$\displaystyle\|\hat{\nu}_{{\boldsymbol{x}}_{0}}(\cdot,{\boldsymbol{\xi}})\|_{L_{p}(\mathbb{R})}$
$\displaystyle\leq(1+|{\boldsymbol{\xi}}^{\mathsf{T}}{\boldsymbol{x}}_{0}|)^{n_{0}}\|u\|_{L_{p}}<+\infty,$
(80)

which holds for all
$({\boldsymbol{x}}_{0},{\boldsymbol{\xi}})\in\mathbb{R}^{d}\times\mathbb{S}^{d-1}$
and $p\geq 1$. Finally, we invoke the boundedness of the (inverse) Fourier
transform $\mathcal{F}^{-1}:L_{p}(\mathbb{R})\to L_{q}(\mathbb{R})$ for
$p\in[1,2]$ to obtain the generic bound

$\displaystyle\sup_{({\boldsymbol{x}}_{0},{\boldsymbol{\xi}})\in\mathbb{R}^{d}\times\mathbb{S}^{d-1}}(1+|{\boldsymbol{\xi}}^{\mathsf{T}}{\boldsymbol{x}}_{0}|)^{-n_{0}}\|\nu_{{\boldsymbol{x}}_{0}}(\cdot,{\boldsymbol{\xi}})\|_{L_{q}(\mathbb{R})}<\infty,$
(81)

which is the desired result.

∎

As complement to the proof of Theorem 7, we make the following remarks.

1\. The function
$\omega\mapsto\hat{\nu}_{{\boldsymbol{x}}_{0}}(\omega,{\boldsymbol{\xi}})$ in
(69) is $\epsilon$-Hölder continuous around the origin for $\gamma_{0}\notin
2\mathbb{N}$ and as smooth as $\widehat{L}$ when $\gamma_{0}=2n$. For
instance, the case where ${\rm L}=(-\Delta)^{n}$ is a non-fractional iterate
of the Laplacian corresponds to $\widehat{L}_{\rm rad}(\omega)=\omega^{2n}$
and $\hat{\nu}_{{\boldsymbol{x}}_{0}}(\cdot,{\boldsymbol{\xi}})\in
C^{\infty}(\mathbb{R})$, which then translates into
$t\mapsto\nu_{{\boldsymbol{x}}_{0}}(t,{\boldsymbol{\xi}})$ being rapidly
decreasing, but with a limited order of differentiability controlled by
$\gamma_{0}=\gamma_{1}=2n$.

2\. The proof can be readily adapted to characterize the partial derivatives
of $\nu_{{\boldsymbol{x}}_{0}}$ by replacing
$\delta(\cdot-{\boldsymbol{x}}_{0})$ by
$\partial^{{\boldsymbol{n}}}\delta(\cdot-{\boldsymbol{x}}_{0})$ with
$|{\boldsymbol{n}}|<n_{0}$. These distributions are such that
$\langle\partial^{{\boldsymbol{n}}}\delta(\cdot-{\boldsymbol{x}}_{0}),m_{\boldsymbol{k}}\rangle=(-1)^{|{\boldsymbol{n}}|}\partial^{{\boldsymbol{n}}}m_{\boldsymbol{k}}({\boldsymbol{x}}_{0})=(-1)^{|{\boldsymbol{n}}|}m_{{\boldsymbol{k}}-{\boldsymbol{n}}}({\boldsymbol{x}}_{0})$,
with the convention that $m_{{\boldsymbol{k}}-{\boldsymbol{n}}}=0$ if
$k_{m}<n_{m}$ for any $m\in\\{1,\dots,d\\}$.

3\. The leading term in (67) is $\rho_{\rm rad}(t-\tau_{0})$ with
$\tau_{0}={\boldsymbol{\xi}}^{\mathsf{T}}{\boldsymbol{x}}_{0}$, which, in the
non-trivial scenario, typically grows as $O(|t|^{\gamma_{0}-1})$. Our analysis
shows that the second correction term in (67), which is unbounded at infinity
as well, essentially neutralizes this growth. It is tempting to call this a
miraculous cancellation.

The bottom line is that, for any
$({\boldsymbol{x}}_{0},{\boldsymbol{\xi}}_{0})\in\mathbb{R}^{d}\times\mathbb{S}^{d-1}$,
the function $t\mapsto\nu_{{\boldsymbol{x}}_{0}}(t,{\boldsymbol{\xi}}_{0})$ is
continuous, bounded with a maximum that is proportional to
$|{\boldsymbol{\xi}}_{0}^{\mathsf{T}}{\boldsymbol{x}}_{0}|^{n_{0}}$ (see (81)
with $p=\infty$), and vanishing at infinity. This is a remarkable property
that also guarantees the boundedness of ${\rm L}_{\rm
R}^{\ast\dagger}:L_{1,n_{0}}(\mathbb{R}^{d})\to{\mathcal{X}}_{\rm Rad}$, which
is not obvious a priori. The enabling ingredient there is (68), which ensures
that the corresponding bounding constant in Theorem 8 is finite. Indeed, since
$\|{\boldsymbol{x}}\|\geq|{\boldsymbol{\xi}}^{\mathsf{T}}{\boldsymbol{x}}|$
with equality if and only if ${\boldsymbol{\xi}}$ and ${\boldsymbol{x}}$ are
colinear, we have that

$\displaystyle\|{\rm L}_{\rm R}^{\ast\dagger}\|$
$\displaystyle\leq\sup_{{\boldsymbol{x}}\in\mathbb{R}^{d},\;{\boldsymbol{\xi}}\in\mathbb{S}^{d-1}}(1+\|{\boldsymbol{x}}\|)^{-n_{0}}\|\nu_{{\boldsymbol{x}}}(\cdot,{\boldsymbol{\xi}})\|_{{\mathcal{X}}(\mathbb{R})}$
$\displaystyle\leq\sup_{{\boldsymbol{x}}\in\mathbb{R}^{d},\;{\boldsymbol{\xi}}\in\mathbb{S}^{d-1}}(1+|{\boldsymbol{\xi}}^{\mathsf{T}}{\boldsymbol{x}}|)^{-n_{0}}\|\nu_{{\boldsymbol{x}}}(\cdot,{\boldsymbol{\xi}})\|_{{\mathcal{X}}(\mathbb{R})}<\infty.$

######  Theorem 8 (Stability of Cartesian-to-Radon-domain mappings).

Let
$h_{{\boldsymbol{x}}}(t,{\boldsymbol{\xi}})={\mathrm{T}}\\{\delta(\cdot-{\boldsymbol{x}})\\}(t,{\boldsymbol{\xi}})\\}$
denote the generalized impulse response of the operator
${\mathrm{T}}:L_{1,\alpha}(\mathbb{R}^{d})\to{\mathcal{X}}_{\rm
Rad}(\mathbb{R}\times\mathbb{S}^{d-1})$ and let

$\displaystyle
C_{p,\alpha}=\sup_{{\boldsymbol{x}}\in\mathbb{R}^{d},\;{\boldsymbol{\xi}}\in\mathbb{S}^{d-1}}(1+\|{\boldsymbol{x}}\|)^{-\alpha}\|h_{{\boldsymbol{x}}}(\cdot,{\boldsymbol{\xi}})\|_{L_{p}(\mathbb{R})}.$
(82)

  1. 1.

${\mathcal{X}}=C_{0}$: If $C_{\infty,\alpha}<\infty$, then
${\mathrm{T}}:L_{1,\alpha}(\mathbb{R}^{d})\to C_{0,\rm Rad}$ is bounded with
$\|{\mathrm{T}}\|\leq C_{\infty,\alpha}$.

  2. 2.

${\mathcal{X}}=L_{p}$ with $p\in(1,\infty)$: If $C_{p,\alpha}<\infty$, then
${\mathrm{T}}:L_{1,\alpha}(\mathbb{R}^{d})\to L_{p,\rm Rad}$ is bounded with
$\|{\mathrm{T}}\|\leq\frac{2\pi^{d/2}}{\Gamma(d/2)}C_{p,\alpha}$.

The same holds true for the adjoint
${\mathrm{T}}^{\ast}:{\mathcal{X}}^{\prime}_{\rm Rad}\to
L_{\infty,-\alpha}(\mathbb{R}^{d})$ with
$\|{\mathrm{T}}^{\ast}\|=\|{\mathrm{T}}\|$.

###### Proof.

The function $\big{(}(t,{\boldsymbol{\xi}}),{\boldsymbol{x}}\big{)}\mapsto
h_{{\boldsymbol{x}}}(t,{\boldsymbol{\xi}})$ is the Schwartz kernel of the
operator ${\mathrm{T}}$, so that

$\displaystyle|g(t,{\boldsymbol{\xi}})|$
$\displaystyle=\big{|}{\mathrm{T}}\\{f\\}(t,{\boldsymbol{\xi}})\big{|}=\big{|}\int_{\mathbb{R}^{d}}h_{{\boldsymbol{x}}}(t,{\boldsymbol{\xi}})\;f({\boldsymbol{x}}){\rm
d}{\boldsymbol{x}}\big{|}$
$\displaystyle\leq\sup_{{\boldsymbol{x}}\in\mathbb{R}^{d},\;{\boldsymbol{\xi}}\in\mathbb{S}^{d-1}}\left((1+\|{\boldsymbol{x}}\|)^{-\alpha}\|h_{{\boldsymbol{x}}}(\cdot,{\boldsymbol{\xi}})\|_{L_{\infty}}\right)\;\int_{\mathbb{R}^{d}}(1+\|{\boldsymbol{x}}\|)^{\alpha}|f({\boldsymbol{x}})|{\rm
d}{\boldsymbol{x}}.$

Consequently,

$\displaystyle\|g\|_{L_{\infty}}=\sup_{(t,{\boldsymbol{\xi}})\in\mathbb{R}\times\mathbb{S}^{d-1}}|g(t,{\boldsymbol{\xi}})|$
$\displaystyle=C_{\infty,\alpha}\|f\|_{L_{1,\alpha}},$

which is the desired bound for ${\mathcal{X}}=C_{0}$.

To handle the reflexive case ${\mathcal{X}}=L_{p}$, we consider the adjoint
operator ${\mathrm{T}}^{\ast}$ whose Schwartz kernel
$\big{(}{\boldsymbol{x}},(t,{\boldsymbol{\xi}})\big{)}\mapsto
h_{{\boldsymbol{x}}}(t,{\boldsymbol{\xi}})$ is obtained by transposition. We
now show that ${\mathrm{T}}^{\ast}:L_{q}(\mathbb{R}\times\mathbb{S}^{d-1})\to
L_{\infty,-\alpha}(\mathbb{R}^{d})$ with $q=\tfrac{p}{p-1}$ is bounded which,
by duality, implies that the same holds true for
$({\mathrm{T}}^{\ast})^{\ast}={\mathrm{T}}:L_{1,\alpha}(\mathbb{R}^{d})\xrightarrow{\mbox{\tiny\
\rm c. }}L_{p}(\mathbb{R}\times\mathbb{S}^{d-1})$ since
$L_{1,\alpha}(\mathbb{R}^{d})$ is isometrically embedded in its bidual
$\big{(}L_{1,\alpha}(\mathbb{R}^{d})\big{)}^{\prime\prime}=\big{(}L_{\infty,-\alpha}(\mathbb{R}^{d})\big{)}^{\prime}$.
The action of the adjoint operator is described as

$\displaystyle f({\boldsymbol{x}})$