# Gelfand pairs on the Heisenberg group and Schwartz functions

Francesca Astengo, Bianca Di Blasio, Fulvio Ricci  Dipartimento di Matematica
Via Dodecaneso 35
16146 Genova
Italy<EMAIL_ADDRESS>Dipartimento di Matematica e Applicazioni
Via Cozzi 53
20125 Milano
Italy<EMAIL_ADDRESS>Scuola Normale Superiore
Piazza dei Cavalieri 7
M 56126 Pisa
Italy<EMAIL_ADDRESS>
###### Abstract.

Let $H_{n}$ be the $(2n+1)$–dimensional Heisenberg group and $K$ a compact
group of automorphisms of $H_{n}$ such that $(K\ltimes H_{n},K)$ is a Gelfand
pair. We prove that the Gelfand transform is a topological isomorphism between
the space of $K$–invariant Schwartz functions on $H_{n}$ and the space of
Schwartz function on a closed subset of $\mathbb{R}^{s}$ homeomorphic to the
Gelfand spectrum of the Banach algebra of $K$–invariant integrable functions
on $H_{n}$.

###### Key words and phrases:

Gelfand pair, Schwartz space, Heisenberg group

###### 2000 Mathematics Subject Classification:

Primary: 43A80 ; Secondary: 22E25

Work partially supported by MIUR and GNAMPA

##  1\. Introduction

A fundamental fact in harmonic analysis on $\mathbb{R}^{n}$ is that the
Fourier transform is a topological isomorphism of the Schwartz space
$\mathcal{S}(\mathbb{R}^{n})$ onto itself.

Various generalizations of this result for different classes of Lie groups
exist in the literature, in particular in the context of Gelfand pairs, where
the operator-valued Fourier transform can be replaced by the scalar-valued
spherical transform. Most notable is the case of a symmetric pair of the
noncompact type, with Harish-Chandra’s definition of a bi-$K$–invariant
Schwartz space on the isometry group (cf. [20, p. 489]).

The definition of a Schwartz space on a Lie group becomes quite natural on a
nilpotent group $N$ (say connected and simply connected). In that case one can
define the Schwartz space by identifying $N$ with its Lie algebra via the
exponential map.

The image of the Schwartz space on the Heisenberg group $H_{n}$ under the
group Fourier transform has been described by D. Geller [17]. Let $K$ be a
compact group of automorphisms of $H_{n}$ such that convolution of
$K$–invariant functions is commutative, in other words assume that $(K\ltimes
H_{n},K)$ is a Gelfand pair . Then a scalar-valued spherical transform
$\mathcal{G}_{K}$ (where $\mathcal{G}$ stands for “Gelfand transform”) of
$K$–invariant functions is available, and Geller’s result can be translated
into a characterization of the image under $\mathcal{G}_{K}$ of the space
$\mathcal{S}_{K}(H_{n})$ of $K$–invariant Schwartz functions. In the same
spirit, a characterization of
$\mathcal{G}_{K}\big{(}\mathcal{S}_{K}(H_{n})\big{)}$ is given in [7] for
closed subgroups $K$ of the unitary group ${\text{U}(n)}$.

In [3] we have proved that, for $K$ equal to ${\text{U}(n)}$ or
$\mathbb{T}^{n}$ (i.e. for radial - resp. polyradial - functions), an
analytically more significant description of
$\mathcal{G}_{K}\big{(}\mathcal{S}_{K}(H_{n})\big{)}$ can be obtained by
making use of natural homeomorphic embeddings of the Gelfand spectrum of
$L^{1}_{K}(H_{n})$ in Euclidean space. The result is that
$\mathcal{G}_{K}\big{(}\mathcal{S}_{K}(H_{n})\big{)}$ is the space of
restrictions to the Gelfand spectrum of the Schwartz functions on the ambient
space. This condition of “extendibility to a Schwartz function on the ambient
space” subsumes the rather technical condition on iterated differences in
discrete parameters that are present in the previous characterizations.

In this article we extend the result of [3] to general Gelfand pairs
$(K\ltimes H_{n},K)$, with $K$ a compact group of automorphisms of $H_{n}$.
Some preliminary notions and facts are required before we can give a precise
formulation of our main theorem.

Let $G$ be a connected Lie group and $K$ a compact subgroup thereof such that
$(G,K)$ is a Gelfand pair and denote by $L^{1}(G//K)$ the convolution algebra
of all bi-$K$–invariant integrable functions on $G$. The Gelfand spectrum of
the commutative Banach algebra $L^{1}(G//K)$ may be identified with the set of
bounded spherical functions with the compact-open topology. Spherical
functions are characterized as the joint eigenfunctions of all $G$–invariant
differential operators on $G/K$, normalized in the $L^{\infty}$-norm.
$G$–invariant differential operators on $G/K$ form a commutative algebra
$\mathbb{D}(G/K)$ which is finitely generated [19].

Given a finite set of generators $\left\\{V_{1},\ldots,V_{s}\right\\}$ of
$\mathbb{D}(G/K)$, we can assign to each bounded spherical function $\phi$ the
$s$-tuple
$\widehat{V}(\phi)=\left(\widehat{V_{1}}(\phi),\ldots,\widehat{V_{s}}(\phi)\right)$
of its eigenvalues with respect to these generators. In this way, the Gelfand
spectrum is identified with a closed subset $\Sigma_{K}^{V}$ of
$\mathbb{C}^{s}$. When all bounded spherical functions are of positive type
and the operators $V_{j}$ self-adjoint, $\Sigma_{K}^{V}\subset\mathbb{R}^{s}$.
As proved in [12], the Euclidean topology induced on $\Sigma_{K}^{V}$
coincides with the compact-open topology on the set of bounded spherical
functions (see also [8] for $G=K\ltimes H_{n}$ and $K\subset{\text{U}(n)}$).
When the Gelfand spectrum is identified with $\Sigma_{K}^{V}$, the spherical
transform will be denoted by ${\mathcal{G}}^{V}_{K}$.

Let $K$ be a compact group of automorphisms of $H_{n}$ such that $(K\ltimes
H_{n},K)$ is a Gelfand pair and denote by ${\mathbb{D}}_{K}$ the commutative
algebra of left-invariant and $K$–invariant differential operators on $H_{n}$.
Let $V=\left\\{V_{1},\ldots,V_{s}\right\\}$ be a set of formally self-adjoint
generators of ${\mathbb{D}}_{K}$. We denote by $\mathcal{S}(\Sigma_{K}^{V})$
the space of restrictions to $\Sigma_{K}^{V}$ of Schwartz functions on
$\mathbb{R}^{s}$, endowed with the quotient topology of
${\mathcal{S}}(\mathbb{R}^{s})/\\{f:f_{|_{\Sigma_{K}^{V}}}=0\\}$.

Our main result is the following:

######  Theorem 1.1.

The map ${\mathcal{G}}^{V}_{K}$ is a topological isomorphism between
${\mathcal{S}}_{K}(H_{n})$ and ${\mathcal{S}}(\Sigma_{K}^{V})$.

As customary, for us $H_{n}$ is understood as
$\mathbb{R}\times\mathbb{C}^{n}$, with canonical coordinates of the first
kind. It is well known that, under the action

$k\cdot(t,z)=(t,kz)\qquad\forall k\in{\text{U}(n)},\ (t,z)\in H_{n},$

${\text{U}(n)}$ is a maximal compact connected group of automorphisms of
$H_{n}$, and that every compact connected group of automorphisms of $H_{n}$ is
conjugated to a subgroup of ${\text{U}(n)}$. Therefore, if $K$ is a compact
group of automorphisms of $H_{n}$, then its identity component in $K$ is
conjugated to a subgroup of ${\text{U}(n)}$. For most of this article, we deal
with the case of $K$ connected and contained in ${\text{U}(n)}$, leaving the
discussion of the general case to the last section.

In Section 3 we show that it suffices to prove Theorem 1.1 for one particular
set of generators of ${\mathbb{D}}_{K}$, and in Section 4 we choose a
convenient set of generators. From a homogeneous Hilbert basis of
$K$–invariant polynomials on $\mathbb{R}^{2n}$ we derive by symmetrization $d$
differential operators $V_{1},\dots,V_{d}$, invariant under $K$; to these we
add the central operator $V_{0}=i^{-1}\partial_{t}$, obtaining in this way a
generating system of $d+1$ homogeneous operators. Here we benefit from the
deep study of the algebraic properties of the multiplicity-free actions of
subgroups of ${\text{U}(n)}$, developed in [5, 6, 7, 8, 9, 10]. In particular,
rationality of the “generalized binomial coefficients” is a crucial point in
our argument (see formula 7.5 below). It must be noticed that the proof of
rationality in [10] is based on the actual classification of multiplicity-free
actions. In this respect, our proof depends on the actual classification of
the groups $K$ giving rise to Gelfand pairs.

After these preliminaries, we split the proof of Theorem 1.1 into two parts.

In the first part we show that, if $m$ is a Schwartz function on
$\mathbb{R}^{d+1}$, its restriction to $\Sigma_{K}^{V}$ is the Gelfand
transform of a function $f$ in $\mathcal{S}_{K}(H_{n})$ (see Theorem 5.5
below). The argument is based on Hulanicki’s theorem [22], stating that
Schwartz functions on the real line operate on positive Rockland operators on
graded nilpotent Lie groups producing convolution operators with Schwartz
kernels. We adapt the argument in [29] to obtain a multivariate extension of
Hulanicki’s theorem (see Theorem 5.2 below).

In the second part we prove that the Gelfand transform
${\mathcal{G}}^{V}_{K}f$ of a function $f$ in $\mathcal{S}_{K}(H_{n})$ can be
extended to a Schwartz function on $\mathbb{R}^{d+1}$ (see Theorem 7.1 below).
The proof begins with an extension to the Schwartz space of the Schwarz–Mather
theorem [26, 28] for $C^{\infty}$ $K$–invariant functions (see Theorem 6.1
below). This allows us to extend to a Schwartz function on $\mathbb{R}^{d}$
the restriction of ${\mathcal{G}}^{V}_{K}f$ to the “degenerate part”
$\Sigma_{0}$ of the Gelfand spectrum (that corresponding to the one-
dimensional representations of $H_{n}$, or equivalently corresponding to the
eigenvalue $0$ for $V_{0}$). Then we associate to $f$ a Schwartz jet on
$\Sigma_{0}$. As in [3], the key tool here is the existence of “Taylor
coefficients” at points of $\Sigma_{0}$, proved by Geller [17] (see Theorem
7.2 below). The Whitney extension theorem (adapted to Schwartz jets in
Proposition 7.4) gives therefore a Schwartz extension to $\mathbb{R}^{d+1}$ of
the jet associated to $f$. To conclude the proof, it remains to prove that if
$f\in\mathcal{S}_{K}(H_{n})$ and the associated jet on $\Sigma_{0}$ is
trivial, then $\mathcal{G}_{K}^{V}f$ admits a Schwartz extension. This is done
by adapting an explicit interpolation formula already used in [3] (see
Proposition 7.5). For a nonconnected group $K$, we remark that, calling
$K_{0}$ the connected component of the identity, one can view the $K$-Gelfand
spectrum as the quotient of the $K_{0}$-Gelfand spectrum under the action of
the finite group $F=K/K_{0}$ (it is known that if $(K\ltimes H_{n},K)$ is a
Gelfand pair, so is $(K_{0}\ltimes H_{n},K_{0})$, cf.[4]). Starting from an
$F$–invariant generating system of $K_{0}$–invariant differential operators,
the $K$-Gelfand spectrum is then conveniently embedded in a Euclidean space by
means of the Hilbert map associated to the action of $F$ on the linear span of
these generators.

The paper is structured as follows. In Section 2 we recall some facts about
Gelfand pairs $(K\ltimes H_{n},K)$ and the associated Gelfand transform. In
Section 3 we show that it suffices to prove Theorem 1.1 for one particular set
of generators of ${\mathbb{D}}_{K}$. In Section 4 we choose a convenient set
of generators in the case where $K$ is a connected closed subgroup of
${\text{U}(n)}$. In Section 5 we show that every function in
$\mathcal{S}\bigl{(}\mathbb{R}^{d+1}\bigr{)}$ gives rise to the Gelfand
transform of a function in $\mathcal{S}_{K}(H_{n})$ via functional calculus.
In Section 6 we extend the Schwarz-Mather theorem [26, 28] to Schwartz spaces.
Section 7 is devoted to define a Schwartz extension on $\mathbb{R}^{d+1}$ of
the Gelfand transform of a function in $\mathcal{S}_{K}(H_{n})$. In Section 8
we show that our result holds for all compact groups of automorphisms of
$H_{n}$.

##  2\. Preliminaries

For the content of this section we refer to [5, 6, 8, 11, 12, 24].

###  2.1. The Heisenberg group and its representations

We denote by $H_{n}$ the Heisenberg group, i.e., the real manifold
$\mathbb{R}\times\mathbb{C}^{n}$ equipped with the group law

$(t,z)(u,w)=\bigl{(}t+u+\tfrac{1}{2}\text{\rm
Im}\,w\cdot\overline{z},z+w\bigr{)}\qquad t,u\in\mathbb{R},\quad\forall
z,w\in\mathbb{C}^{n},$

where $w\cdot\overline{z}$ is a short-hand writing for
$\sum_{j=1}^{n}w_{j}\,\overline{z_{j}}$. It is easy to check that Lebesgue
measure $dt\,dz$ is a Haar measure on $H_{n}$.

Denote by $Z_{j}$ and $\bar{Z}_{j}$ the complex left-invariant vector fields

$Z_{j}=\partial_{z_{j}}-{\textstyle{i\over
4}}\,\bar{z}_{j}\,\partial_{t}\qquad\bar{Z}_{j}=\partial_{\bar{z}_{j}}+{\textstyle{i\over
4}}\,z_{j}\,\partial_{t},$

and set $T=\partial_{t}$.

For $\lambda>0$, denote by ${\mathcal{F}}_{\lambda}$ the Fock space consisting
of the entire functions $F$ on $\mathbb{C}^{n}$ such that

$\|F\|_{\mathcal{F}_{\lambda}}^{2}=\left({\lambda\over
2\pi}\right)^{n}\,\int_{\mathbb{C}^{n}}|F(z)|^{2}\,\,e^{-\frac{\lambda}{2}|z|^{2}}\,dz<\infty\
,$

equipped with the norm $\|\cdot\|_{\mathcal{F}_{\lambda}}$. Then $H_{n}$ acts
on $\mathcal{F}_{\lambda}$ through the unitary representation $\pi_{\lambda}$
defined by

$[\pi_{\lambda}(t,z)F](w)=e^{i\lambda
t}\,e^{-\frac{\lambda}{2}w\cdot\bar{z}-\frac{\lambda}{4}|z|^{2}}\,F(w+z)\qquad\forall(z,t)\in
H_{n},\\!\\!\quad F\in{\mathcal{F}}_{\lambda},\\!\\!\quad w\in\mathbb{C}^{n}\
,$

and through its contragredient
$\pi_{-\lambda}(t,z)=\pi_{\lambda}(-t,\bar{z})$. These are the Bargmann
representations of $H_{n}$.

The space $\mathcal{P}(\mathbb{C}^{n})$ of polynomials on $\mathbb{C}^{n}$ is
dense in ${\mathcal{F}}_{\lambda}$ ($\lambda>0$) and an orthonormal basis of
$\mathcal{P}(\mathbb{C}^{n})$ seen as a subspace of ${\mathcal{F}}_{\lambda}$
is given by the monomials

$p_{\lambda,{\bf d}}(w)={w^{{\bf d}}\over((2/\lambda)^{|{{\bf d}}|}{{\bf
d}}!)^{1/2}}\qquad{{\bf d}}\in\mathbb{N}^{n}.$

Besides the Bargmann representations, $H_{n}$ has the one-dimensional
representations

$\tau_{w}(t,z)=e^{i\text{\rm Re}\,z\cdot\bar{w}}$

with $w\in\mathbb{C}^{n}$. The $\pi_{\lambda}$ ($\lambda\neq 0$) and the
$\tau_{w}$ fill up the unitary dual of $H_{n}$.

###  2.2. Gelfand pairs $(K\ltimes H_{n},K)$

Let $K$ be a compact groups of automorphisms of $H_{n}$ such that the
convolution algebra $L^{1}_{K}(H_{n})$ of integrable $K$–invariant functions
on $H_{n}$ is abelian, i.e., such that $(K\ltimes H_{n},K)$ is a Gelfand pair.
It is known [4] that this property holds for $K$ if and only if it holds for
its connected identity component $K_{0}$. On the other hand, every compact,
connected group of automorphisms of $H_{n}$ is conjugate, modulo an
automorphism, to a subgroup of ${\text{U}(n)}$, acting on $H_{n}$ via

$k\cdot(t,z)=(t,kz)\qquad\forall(t,z)\in H_{n},\quad k\in{\text{U}(n)}.$

For the remainder of this section, we assume that $K$ is connected, contained
in ${\text{U}(n)}$, and $(K\ltimes H_{n},K)$ is a Gelfand pair.

For the one-dimensional representation $\tau_{w}$, we have
$\tau_{w}(t,k^{-1}z)=\tau_{kw}(t,z)$. Therefore, $\tau_{w}(f)$ is a
$K$–invariant function of $w$ for every $f\in L^{1}_{K}(H_{n})$.

As to the Bargmann representations, if $k\in{\text{U}(n)}$,
$\pi_{\pm\lambda}^{k}(t,z)=\pi_{\pm\lambda}(t,kz)$ is equivalent to
$\pi_{\pm\lambda}$ for every $\lambda>0$ and every choice of the $\pm$ sign.
Precisely, we set

$\nu_{+}(k)F(z)=F(k^{-1}z)\ ,\qquad\nu_{-}(k)F(z)=F(\bar{k}^{-1}z)\ ,$

each of the two actions being the contragredient of the other. We then have

$\pi_{\lambda}(kz,t)=\nu_{+}(k)\pi_{\lambda}(z,t)\nu_{+}(k)^{-1}\
,\qquad\pi_{-\lambda}(kz,t)=\nu_{-}(k)\pi_{-\lambda}(z,t)\nu_{-}(k)^{-1}\ .$

By homogeneity, the decomposition of $\mathcal{F}_{\lambda}$ into irreducible
invariant subspaces under $\nu_{+}$ (resp. $\nu_{-}$) is independent of
$\lambda$ and can be reduced to the decomposition of the dense subspace
$\mathcal{P}(\mathbb{C}^{n})$ of polynomials.

It is known since [11, 5] that $(K\ltimes H_{n},K)$ is a Gelfand pair if and
only if $\nu_{+}$ (equivalently $\nu_{-}$) decomposes into irreducibles
without multiplicities (in other words, if and only if it is multiplicity
free). The subgroups of ${\text{U}(n)}$ giving multiplicity free actions on
$\mathcal{P}(\mathbb{C}^{n})$ have been classified by Kač [24] and the
resulting Gelfand pairs $(K\ltimes H_{n},K)$ are listed in [10, tables 1 and
2].

Under these assumptions, the space $\mathcal{P}(\mathbb{C}^{n})$ of
polynomials on $\mathbb{C}^{n}$ decomposes into $\nu_{+}$-irreducible
subspaces,

$\mathcal{P}(\mathbb{C}^{n})=\sum_{\alpha\in\Lambda}P_{\alpha}\ ,$

where $\Lambda$ is an infinite subset of the unitary dual $\widehat{K}$ of $K$
and $\alpha$ denotes the equivalence class of the action on $P_{\alpha}$. The
irreducible $\nu_{-}$–invariant subspaces of $\mathcal{P}(\mathbb{C}^{n})$ are

$P_{\alpha^{\prime}}=\\{\bar{p}(z)=\overline{p(\bar{z})}:p\in P_{\alpha}\\}\
,$

with the action of $K$ on $P_{\alpha^{\prime}}$ being equivalent to the
contragredient $\alpha^{\prime}$ of $\alpha$.

On the other hand,
$\mathcal{P}(\mathbb{C}^{n})=\sum_{m\in\mathbb{N}}\mathcal{P}_{m}(\mathbb{C}^{n})$,
where $\mathcal{P}_{m}(\mathbb{C}^{n})$ is the space of homogeneous
polynomials of degree $m$. Since $\nu_{\pm}$ preserves each
$\mathcal{P}_{m}(\mathbb{C}^{n})$, each $P_{\alpha}$ is contained in
$\mathcal{P}_{m}(\mathbb{C}^{n})$ for some $m$. We then say that $|\alpha|=m$,
so that

$\mathcal{P}_{m}(\mathbb{C})=\sum_{|\alpha|=m}P_{\alpha}=\sum_{|\alpha|=m}P_{\alpha^{\prime}}.$

As proved in [5], all the bounded spherical functions are of positive type.
Therefore there are two families of spherical functions. Those of the first
family are

$\eta_{Kw}(t,z)=\int_{K}e^{i{\rm Re}\langle z,kw\rangle}\,dk,\qquad
w\in\mathbb{C}^{n},$

parametrized by $K$–orbits in $\mathbb{C}^{n}$ and associated with the one-
dimensional representations of the Heisenberg group.

The elements of the second family are parametrized by pairs
$(\lambda,\alpha)\in\mathbb{R}^{*}\times\Lambda$. If $\lambda>0$ and
$\\{v_{1}^{\lambda},\ldots,v_{\text{dim}(P_{\alpha})}^{\lambda}\\}$ is an
orthonormal basis of $P_{\alpha}$ in the norm of ${\mathcal{F}}_{\lambda}$, we
have the spherical function

(2.1)
$\phi_{\lambda,\alpha}(t,z)=\frac{1}{\text{dim}(P_{\alpha})}\sum_{j=1}^{\text{dim}(P_{\alpha})}\langle\pi_{\lambda}(t,z)v_{j}^{\lambda}\,,v_{j}^{\lambda}\rangle_{\mathcal{F}_{\lambda}}\
.$

Taking, as we can, $v_{j}^{\lambda}$ as $\lambda^{|\alpha|/2}v_{j}^{1}$, we
find that

$\phi_{\lambda,\alpha}(z,t)=\phi_{1,\alpha}(\sqrt{\lambda}z,\lambda t)\ .$

For $\lambda<0$, the analogous matrix entries of the contragredient
representation give the spherical functions

$\phi_{\lambda,\alpha}(z,t)=\overline{\phi_{-\lambda,\alpha}(z,t)}\ .$

Setting for simplicity $\phi_{\alpha}=\phi_{1,\alpha}$,

$\phi_{\alpha}(z,t)=e^{it}\,q_{\alpha}(z,\bar{z})\,e^{-|z|^{2}/4},$

where
$q_{\alpha}\in\mathcal{P}(\mathbb{C}^{n})\otimes\overline{\mathcal{P}(\mathbb{C}^{n})}$
is a real $K$–invariant polynomial of degree $2|\alpha|$ in $z$ and $\bar{z}$
(cf. [6]).

Denote by ${\mathbb{D}}_{K}$ the algebra of left-invariant and $K$–invariant
differential operators on $H_{n}$. The symmetrization map establishes a linear
bijection from the space of $K$–invariant elements in the symmetric algebra
over $\mathfrak{h}_{n}$ to ${\mathbb{D}}_{K}$. Therefore every element
$D\in{\mathbb{D}}_{K}$ can be expressed as $\sum_{j=0}^{m}D_{j}T^{j}$, where
$D_{j}$ is the symmetrization of a $K$–invariant polynomial in $Z,\bar{Z}$.

Let $D$ be the symmetrization of the $K$–invariant polynomial
$P(Z,\bar{Z},T)$. With $\widehat{D}(\phi)$ denoting the eigenvalue of
$D\in{\mathbb{D}}_{K}$ on the spherical function $\phi$, we have

(2.2) $\widehat{D}(\eta_{Kw})=P(w,\bar{w},0)$

for the spherical functions associated to the one-dimensional representations

For $\lambda\neq 0$, $d\pi_{\lambda}(D)$ commutes with the action of $K$ and
therefore it preserves each $P_{\alpha}$. By Schur’s lemma,
$d\pi_{\lambda}(D)_{|_{P_{\alpha}}}$ is a scalar operator
$c_{\lambda,\alpha}(D)\,I_{{P_{\alpha}}}$. It follows from (2.1) that

$\widehat{D}(\phi_{\lambda,\alpha})=c_{\lambda,\alpha}(D)\ .$

In particular, for $D=T$, we have

$\widehat{T}(\eta_{Kw})=0\
,\qquad\widehat{T}\big{(}\phi_{\lambda,\alpha}\big{)}=i\lambda\ .$

##  3\. Embeddings of the Gelfand spectrum

Let $K$ be a compact group of automorphisms of $H_{n}$ such that $(K\ltimes
H_{n},K)$ is a Gelfand pair. The Gelfand spectrum of the commutative Banach
algebra $L^{1}_{K}(H_{n})$ is the set of bounded spherical functions endowed
with the compact-open topology. Given a set
$V=\\{V_{0},\,\,V_{1},\ldots,V_{d}\\}$ of formally self-adjoint generators of
${\mathbb{D}}_{K}$, we assign to each spherical function $\phi$ the
$(d+1)$-tuple
$\widehat{V}(\phi)=\left(\widehat{V_{0}}(\phi),\,\,\widehat{V_{1}}(\phi),\ldots,\widehat{V_{d}}(\phi)\right)$.
Since $d\pi(V_{j})$ is formally self-adjoint for every irreducible
representation $\pi$ , $\widehat{V}(\phi)$ is in $\mathbb{R}^{d+1}$. It has
been proved, in a more general context [12], that
$\Sigma_{K}^{V}=\\{\widehat{V}(\phi):\phi\text{ spherical}\\}$ is closed in
$\mathbb{R}^{d+1}$ and homeomorphic to the Gelfand spectrum via $\widehat{V}$
(see also [8]).

Once we have identified the Gelfand spectrum with $\Sigma_{K}^{V}$, the
Gelfand transform of a function $f$ in $L^{1}_{K}(H_{n})$ can be defined on
the closed subset $\Sigma_{K}^{V}$ of $\mathbb{R}^{d+1}$ as

$({\mathcal{G}}^{V}_{K}f)\,(\widehat{V}(\phi))=\int_{H_{n}}f\,\phi.$

In order to prove Theorem 1.1, we first show that different choices of the
generating system $V$ give rise to natural isomorphisms among the
corresponding restricted Schwartz spaces $\mathcal{S}(\Sigma_{K}^{V})$. It
will then suffice to prove Theorem 1.1 for one particular set of generators.

On the Schwartz space ${\mathcal{S}}(\mathbb{R}^{m})$ we consider the
following family of norms, parametrized by a nonnegative integer $p$:

$\|f\|_{(p,\mathbb{R}^{m})}=\sup_{y\in\mathbb{R}^{m},|\alpha|\leq
p}(1+|y|)^{p}|\partial^{\alpha}f(y)|.$

######  Lemma 3.1.

Let $E$ and $F$ be closed subsets of $\mathbb{R}^{n}$ and $\mathbb{R}^{m}$
respectively. Let $P:\mathbb{R}^{n}\to\mathbb{R}^{m}$ and
$Q:\mathbb{R}^{m}\to\mathbb{R}^{n}$ be polynomial maps such that $P(E)=F$ and
$Q\circ P$ is the identity on $E$. Given $f$ in $\mathcal{S}(F)$ we let
$P^{\flat}f=f\circ P|_{E}$. Then $P^{\flat}$ maps $\mathcal{S}(F)$ in
$\mathcal{S}(E)$ continuously.

###### Proof.

We show that if $f$ is in $\mathcal{S}(\mathbb{R}^{m})$, then $P^{\flat}f$ can
be extended to a function $\widetilde{P^{\flat}f}$ in
$\mathcal{S}(\mathbb{R}^{n})$ in a linear and continuous way. Let $\Psi$ be a
smooth function on $\mathbb{R}^{n}$ such that $\Psi(t)=1$ if $|t|\leq 1$ and
$\Psi(t)=0$ if $|t|>2$. Define

$\widetilde{P^{\flat}f}(x)=\Psi(x-Q\circ P(x))\,\,(f\circ P)(x)\qquad\forall
x\in\mathbb{R}^{n}.$

Clearly $\widetilde{P^{\flat}f}$ is smooth and
$\widetilde{P^{\flat}f}|_{E}=P^{\flat}f$. Moreover $\widetilde{P^{\flat}f}$ is
zero when $|x-Q\circ P(x)|>2$, so it suffices to prove rapid decay for
$|x-Q\circ P(x)|\leq 2$. Note that there exists $\ell$ in $\mathbb{N}$ such
that

$|x|\leq 2+|Q(P(x))|\leq C\,(1+|P(x)|)^{\ell}\qquad\forall
x\in\mathbb{R}^{n},\quad|x-Q\circ P(x)|\leq 2.$

Therefore given a positive integer $p$ there exists a positive integer $q$
such that $\|\widetilde{P^{\flat}f}\|_{(p,\mathbb{R}^{n})}\leq
C\,\|f\|_{(q,\mathbb{R}^{m})}.$ The thesis follows immediately from the
definition of the quotient topology on $\mathcal{S}(F)$ and $\mathcal{S}(E)$.
∎

######  Corollary 3.2.

Suppose that $\\{V_{0},\ldots,V_{d}\\}$ and $\\{W_{0},\ldots,W_{s}\\}$ are two
sets of formally self-adjoint generators of ${\mathbb{D}}_{K}$. Then the
spaces ${\mathcal{S}}(\Sigma_{K}^{V})$ and ${\mathcal{S}}(\Sigma_{K}^{W})$ are
topologically isomorphic.

###### Proof.

There exist real polynomials $p_{j}$, $j=0,1,\ldots,s$, and $q_{h}$,
$h=0,1,\ldots,d$, such that $W_{j}=p_{j}(V_{0},\ldots,V_{d})$ and
$V_{h}=q_{h}(W_{0},\ldots,W_{s}).$

Setting
$P=(p_{0},\,p_{1},\ldots,p_{s}):\mathbb{R}^{d+1}\rightarrow\mathbb{R}^{s+1}$
and
$Q=(q_{0},\,q_{1},\ldots,q_{d}):\mathbb{R}^{s+1}\rightarrow\mathbb{R}^{d+1}$,
we can apply Lemma 3.1 in both directions. ∎

##  4\. Choice of the generators

In this section $K$ shall be a closed connected subgroup of ${\text{U}(n)}$
such that $(K\ltimes H_{n},K)$ is a Gelfand pair. The subject of the following
lemma is the choice of a convenient set of formally self-adjoint generators of
${\mathbb{D}}_{K}$.

######  Lemma 4.1.

A generating system $\\{V_{0}=-iT,V_{1},\ldots,V_{d}\\}$ can be chosen such
that, for each $j=1,\ldots,d$,

  1. (1)

$V_{j}$ is homogeneous of even order $2\,m_{j}$;

  2. (2)

$V_{j}$ is formally self-adjoint and $\widehat{V}_{j}(\phi_{\alpha})$ is a
positive integer for every $\alpha$ in $\Lambda$;

  3. (3)

$\widehat{V}_{j}(\eta_{Kw})=\rho_{j}(w,\bar{w})$, for every $w$ in
$\mathbb{C}^{n}$, where $\rho_{j}$ is a nonnegative homogenous polynomial of
degree $2\,m_{j}$, strictly positive outside of the origin.

Notice that (1) and (2) imply that when $j=1,\ldots,d$

(4.1)
$\widehat{V}_{j}(\phi_{\lambda,\alpha})=|\lambda|^{m_{j}}\,\widehat{V}_{j}(\phi_{\alpha}),\qquad\forall\lambda\in\mathbb{R}\setminus\\{0\\},\quad\forall\alpha\in\Lambda.$

###### Proof.

Let $\mathbb{C}^{n}_{\mathbb{R}}$ denote $\mathbb{C}^{n}$ with the underlying
structure of a real vector space. We denote by
$\mathcal{P}(\mathbb{C}^{n}_{\mathbb{R}})\cong\mathcal{P}(\mathbb{C}^{n})\otimes\overline{\mathcal{P}(\mathbb{C}^{n})}$
the algebra of polynomials in $z$ and $\bar{z}$, and by
$\mathcal{P}^{K}(\mathbb{C}^{n}_{\mathbb{R}})$ the subalgebra of $K$–invariant
polynomials.

The fact that the representation of $K$ on $\mathcal{P}(\mathbb{C}^{n})$ is
multiplicity free implies that the trivial representation is contained in
$P_{\alpha}\otimes\overline{P_{\beta}}\subset\mathcal{P}(\mathbb{C}^{n}_{\mathbb{R}})$
if and only if $\alpha=\beta$, and with multiplicity one in each of them.
Therefore a linear basis of $\mathcal{P}^{K}(\mathbb{C}^{n}_{\mathbb{R}})$ is
given by the polynomials

(4.2) $p_{\alpha}(z,\overline{z})=\sum_{h=1}^{{\rm
dim}(P_{\alpha})}v_{h}(z)\,\overline{v_{h}(z)}=\sum_{h=1}^{{\rm
dim}(P_{\alpha})}|v_{h}(z)|^{2}\ ,$

where $\\{v_{1},\ldots,v_{{\rm dim}(P_{\alpha})}\\}$ is any orthonormal basis
of $P_{\alpha}$ in the ${\mathcal{F}}_{1}$-norm.

A result in [21] ensures that there exist $\delta_{1},\ldots,\delta_{d}$ in
$\Lambda$ such that the polynomials

$\gamma_{j}=p_{\delta_{j}}\qquad j=1,\ldots,d,$

freely generate $\mathcal{P}^{K}(\mathbb{C}_{\mathbb{R}}^{n})$. In [10] the
authors prove that $\gamma_{1},\ldots,\gamma_{d}$ have rational coefficients.
More precisely, setting $m_{j}=|\delta_{j}|$, each $\gamma_{j}$ can be written
in the form

$\gamma_{j}(z,\bar{z})=\sum_{|{\bf a}|=|{\bf b}|=m_{j}}\theta^{(j)}_{{\bf
a},{\bf b}}\,\,z^{\bf a}\bar{z}^{\bf b},$

where ${\bf a},{\bf b}$ are in $\mathbb{N}^{n}$ and $\theta^{(j)}_{{\bf
a},{\bf b}}$ are rational numbers.

The symmetrization $L_{\gamma_{j}}$ of $\gamma_{j}(Z,\bar{Z})$ is a homogenous
operator of degree $2m_{j}$ in ${\mathbb{D}}_{K}$ with rational coefficients,
and $\\{-iT,\,L_{\gamma_{1}},\ldots,L_{\gamma_{d}}\\}$ generate
${\mathbb{D}}_{K}$ [6, 8]. Moreover, the eigenvalues
$\widehat{L}_{\gamma_{j}}(\phi_{\alpha})$ are rational numbers [9, 10].

Fix any positive integer $m$ and denote by $M_{j,m}$ the matrix which
represents the restriction of $d\pi_{1}\left(L_{\gamma_{j}}\right)$ to
$\mathcal{P}_{m}(\mathbb{C}^{n})$ in the basis of monomials $w^{\alpha}$,
$|\alpha|=m$. For every $F$ in $\mathcal{F}_{1}(\mathbb{C}^{n})$,

$[d\pi_{1}(Z_{h})F](w)=[\partial_{w_{h}}F](w)\qquad[d\pi_{1}(\bar{Z}_{h})F](w)=-{1\over
2}\,w_{h}\,F(w)\qquad\forall w\in\mathbb{C}^{n},$

for $h=1,\ldots,n$. Therefore $M_{j,m}$ has rational entries, with
denominators varying in a finite set independent of $m$. We can then take $N$
such that the matrices $NM_{j,m}$ have integral entries, for all $m$ and
$j=1,\ldots,d$. Thus the characteristic polynomial of $NM_{j,m}$ is monic,
with integral coefficients and rational zeroes
$N\widehat{L_{\gamma_{j}}}(\phi_{\alpha})$; therefore these zeroes must be
integers. They all have the same sign, independently of $m$, equal to
$(-1)^{m_{j}}$ (cf. [8]).

We then define

$V_{j}=N\,(-1)^{m_{j}}\,L_{\gamma_{j}}+\mathcal{L}^{m_{j}},\qquad
j=1,\ldots,d,$

where
$\mathcal{L}=-2\sum_{j=1}^{n}\bigl{(}Z_{j}\overline{Z}_{j}+\overline{Z}_{j}Z_{j}\bigr{)}$
is the ${\text{U}(n)}$–invariant sublaplacian, satisfying
$\widehat{\mathcal{L}}(\phi_{\alpha})=2|\alpha|+n$. We show that
$\\{V_{0}=-iT,V_{1},\ldots,V_{d}\\}$ is a set of generators satisfying the
required conditions.

Since $\sum_{m_{j}=1}\gamma_{j}=\frac{1}{2}|z|^{2}$ (cf. [8]), we have

$\sum_{m_{j}=1}L_{\gamma_{j}}=\sum_{j=1}^{n}\bigl{(}Z_{j}\overline{Z}_{j}+\overline{Z}_{j}Z_{j}\bigr{)}=-\frac{1}{2}{\mathcal{L}}\
,$

and then

(4.3)
$\sum_{m_{j}=1}V_{j}=\sum_{m_{j}=1}\left(-N\,L_{\gamma_{j}}+\mathcal{L}\right)=\left(\frac{N}{2}+r\right)\,\mathcal{L}\
,$

where $r$ is the cardinality of the set
$\left\\{\delta_{j}\,:\,m_{j}=|\delta_{j}|=1\right\\}$. Therefore each
$L_{\gamma_{j}}$ is a polynomial in $V_{1},\ldots,V_{d}$. Since
$-iT,L_{\gamma_{1}},\ldots,L_{\gamma_{d}}$ generate ${\mathbb{D}}_{K}$, the
same holds for $-iT,V_{1},\ldots,V_{d}$.

Condition (1) follows from the homogeneity of $L_{\gamma_{j}}$ and
$\mathcal{L}^{m_{j}}$, and from (4.2). Since the polynomials in (4.2) are
real-valued, then the $V_{j}^{\prime}s$ are formally self-adjoint and
condition (2) is easily verified. Finally condition (3) follows from (2.2),
which gives

$\widehat{L_{\gamma_{j}}}(\eta_{Kw})=(-1)^{m_{j}}\,\gamma_{j}(w,\bar{w})\
,\qquad\widehat{{\mathcal{L}}^{m_{j}}}(\eta_{Kw})=|w|^{2m_{j}}$

for all $w$ in $\mathbb{C}^{n}$ and $j=1,\ldots,d$. ∎

Let $V=\\{V_{0},V_{1},\ldots,V_{d}\\}$ denote the privileged set of generators
chosen in Lemma 4.1. We set $\rho=(\rho_{1},\ldots,\rho_{d})$ the polynomial
map in (3) of Lemma 4.1.

Coordinates in $\mathbb{R}^{d+1}$ will be denoted by $(\lambda,\xi)$, with
$\lambda$ in $\mathbb{R}$ and $\xi$ in $\mathbb{R}^{d}$. So, if
$(\lambda,\xi)=\widehat{V}(\phi)$, then either
$\lambda=-i\widehat{T}(\phi)=0$, in which case $\phi=\eta_{Kw}$ and
$\xi=\rho(w,\bar{w})$, or $\lambda=-i\widehat{T}(\phi)\neq 0$, in which case
$\phi=\phi_{\lambda,\alpha}$ and
$\xi_{j}=\widehat{V_{j}}(\phi_{\lambda,\alpha})=|\lambda|^{m_{j}}\,\widehat{V_{j}}(\phi_{\alpha})$.

The spectrum $\Sigma_{K}^{V}$ consists therefore of two parts. The first part
is $\Sigma_{0}=\\{0\\}\times\rho(\mathbb{C}^{n})$, a semi-algebraic set. The
second part, $\Sigma^{\prime}$, is the countable union of the curves
$\Gamma_{\alpha}(\lambda)=\widehat{V}(\phi_{\lambda,\alpha})$, $\lambda\neq
0$. Each $\Gamma_{\alpha}$ and $\Sigma_{0}$ are homogeneous with respect to
the dilations

(4.4)
$(\lambda,\xi_{1}.\dots,\xi_{d})\mapsto(t\lambda,t^{m_{1}}\xi_{1}.\dots,t^{m_{d}}\xi_{d})\
,\qquad(t>0)$

and to the symmetry
$(\lambda,\xi_{1}.\dots,\xi_{d})\mapsto(-\lambda,\xi_{1}.\dots,\xi_{d})$. By
our choice of $V$, $\Sigma^{\prime}\cap\\{\lambda=1\\}$ is contained in the
positive integer lattice. Moreover $\Sigma^{\prime}$ is dense in
$\Sigma_{K}^{V}$ (cf. [8, 17])

For the sake of brevity, we denote by $\hat{f}$ the Gelfand transform
${\mathcal{G}}^{V}_{K}f$ of the integrable $K$–invariant function $f$ .

##  5\. Functional calculus

In this section we prove one of the two implications of Theorem 1.1 for a
closed connected subgroup $K$ of ${\text{U}(n)}$ such that $(K\ltimes
H_{n},K)$ is a Gelfand pair. More precisely we prove that if $m$ is a Schwartz
function on $\mathbb{R}^{d+1}$, its restriction to $\Sigma_{K}^{V}$ is the
Gelfand transform of a function $f$ in $\mathcal{S}_{K}(H_{n})$ (see Theorem
5.5 below).

The proof is based on a result of Hulanicki [22] (see Theorem 5.1 below) on
functional calculus for Rockland operators on graded groups and a multi-
variate extention of it.

A Rockland operator $D$ on a graded Lie group $N$ is a formally self-adjoint
left-invariant differential operator on $N$ which is homogeneous with respect
to the dilations and such that, for every nontrivial irreducible
representation $\pi$ of $N$, the operator $d\pi(D)$ is injective on the space
of $C^{\infty}$ vectors.

As noted in [23], it follows from [27] and [18] that a Rockland operator $D$
is essentially self-adjoint on the Schwartz space $\mathcal{S}(N)$, as well as
$d\pi(D)$ on the Gårding space for every unitary representation $\pi$. We keep
the same symbols for the self-adjoint extensions of such operators.

Let $N$ be a graded Lie group, and let $|\cdot|$ be a homogeneous gauge on it.
We say that a function on $N$ is Schwartz if and only if it is represented by
a Schwartz function on the Lie algebra $\mathfrak{n}$ in any given set of
canonical coordinates. The fact that changes of canonical coordinates are
expressed by polynomials makes this condition independent of the choice of the
coordinates. Given a homogeneous basis $\\{X_{1},\dots,X_{n}\\}$ of the Lie
algebra $\mathfrak{n}$ we keep the same notation $X_{j}$ for the associated
left-invariant vector fields on $N$. Following [14], we shall consider the
following family of norms on ${\mathcal{S}}(N)$, parametrized by a nonnegative
integer $p$:

(5.1) $\|f\|_{(p,N)}=\sup\\{(1+|x|)^{p}\,|X^{I}f(x)|\,:x\in N\,,\
\text{deg}\,X^{I}\leq p\\},$

where $X^{I}=X_{1}^{i_{1}}\cdots X_{n}^{i_{n}}$ and
$\text{deg}\,X^{I}=\sum\,i_{j}\text{deg}\,X_{j}$. Note that the Frechét space
structure induced on $\mathcal{S}(N)$ by this family of norms is independent
of the choice of the $X_{j}$ and is equivalent to that induced from
$\mathcal{S}(\mathfrak{n})$ via composition with the exponential map.

######  Theorem 5.1.

[22] Let $D$ a positive Rockland operator on a graded Lie group $N$ and let
$D=\int_{0}^{+\infty}\lambda\,dE(\lambda)$ be its spectral decomposition. If
$m$ is in $\mathcal{S}(\mathbb{R})$ and

$m(D)=\int_{0}^{+\infty}m(\lambda)\,dE(\lambda),$

then there exists $M$ in $\mathcal{S}(N)$ such that

$m(D)f=f\ast M\qquad\forall f\in\mathcal{S}(N).$

Moreover for every $p$ there exists $q$ such that

$\|M\|_{(p,N)}\leq C\,\|m\|_{(q,\mathbb{R})}.$

Suppose that $D_{1},\ldots,D_{s}$ form a commutative family of self-adjoint
operators on $N$ (in the sense that they have commuting spectral resolutions).
Then they admit a joint spectral resolution and one can define the bounded
operator $m(D_{1},\ldots,D_{s})$ for any bounded Borel function $m$ on their
joint spectrum in $\mathbb{R}^{s}$. The following theorem was proved in [29]
in a special situation.

######  Theorem 5.2.

Suppose that $N$ is a graded Lie group and $D_{1},\ldots,D_{s}$ form a
commutative family of positive Rockland operators on $N$. If $m$ is in
$\mathcal{S}(\mathbb{R}^{s})$, then there exists $M$ in $\mathcal{S}(N)$ such
that

$m(D_{1},\ldots,D_{s})f=f*M\ .$

Moreover, for every $p$ there exists $q$ such that

$\|M\|_{(p,N)}\leq C\,\|m\|_{(q,\mathbb{R}^{d})}$

###### Proof.

We prove the theorem by induction on $s$. By Theorem 5.1 the thesis holds when
$s=1$. Let $s\geq 2$ and suppose that the thesis holds for $s-1$. Let
$m(\lambda_{1},\dots,\lambda_{s})$ be in $\mathcal{S}(\mathbb{R}^{s})$. Then
there exist sequences $\\{\psi_{k}\\}$ in $\mathcal{S}(\mathbb{R}^{s-1})$ and
$\\{\varphi_{k}\\}$ in $\mathcal{S}(\mathbb{R})$ such that
$m(\lambda_{1},\dots,\lambda_{s-1},\lambda_{s})=\sum_{k}\psi_{k}(\lambda_{1},\dots,\lambda_{s-1})\varphi_{k}(\lambda_{s})$
and
$\sum_{k}\left\|\psi_{k}\otimes\varphi_{k}\right\|_{N,\mathbb{R}^{s}}<\infty$
for every $N$.

For a proof of this, one can first decompose $m$ as a sum of
$C^{\infty}$-functions supported in a sequence of increasing balls and with
rapidly decaying Schwartz norms, and then separate variables in each of them
by a Fourier series expansion (see also [29]).

By the inductive hypothesis, for every $k$ there exist $\Psi_{k}$ and
$\Phi_{k}$ in $\mathcal{S}(N)$ such that
$\psi_{k}(D_{1},\ldots,D_{s-1})f=f\ast\Psi_{k}$ and
$\varphi_{k}(D_{s})f=f\ast\Phi_{k}$ for every $f$ in $\mathcal{S}(N)$. Then

$\displaystyle\psi_{k}\otimes\varphi_{k}\left(D_{1},\ldots,D_{s-1},D_{s}\right)f$
$\displaystyle=\psi_{k}\left(D_{1},\ldots,D_{s-1}\right)\,\varphi_{k}\left(D_{s}\right)f$
$\displaystyle=f\ast\Phi_{k}\ast\Psi_{k}\ .$

By straightforward computations (cf. [14, Proposition 1.47]) and the inductive
hypothesis, we obtain that

$\displaystyle\|\Psi_{k}\ast\Phi_{k}\|_{(p,N)}$ $\displaystyle\leq
C_{p}\,\|\Psi_{k}\|_{(p^{\prime},N)}\,\|\Phi_{k}\|_{(p^{\prime\prime}+1,N)}$
$\displaystyle\leq
C_{p}\,\|\psi_{k}\|_{(q,\mathbb{R}^{s-1})}\|\varphi_{k}\|_{(q^{\prime},\mathbb{R})}$
$\displaystyle\leq
C_{p}\,\|\psi_{k}\otimes\varphi_{k}\|_{(q^{\prime\prime},\mathbb{R}^{s})}\ .$

Since the series $\sum_{k}\psi_{k}\otimes\varphi_{k}$ is totally convergent in
every Schwartz norm on $\mathbb{R}^{s}$, there exists a function $F$ in
$\mathcal{S}(N)$ such that

$\sum_{k}\Psi_{k}\ast\Phi_{k}=F$

and hence

$\sum_{k}f\ast\Psi_{k}\ast\Phi_{k}=f\ast F$

for every $f$ in $\mathcal{S}(N)$.

On the other hand, if $M\in\mathcal{S}^{\prime}(N)$ is the convolution kernel
of $m\left(D_{1},\ldots,D_{s-1},D_{s}\right)$, by the Spectral Theorem,

$\displaystyle m\left(D_{1},\ldots,D_{d-1},D_{d}\right)\,f$
$\displaystyle=\sum_{k}\psi_{k}\left(D_{1},\ldots,D_{d-1}\right)\otimes\varphi_{k}\left(D_{d}\right)\,f$
$\displaystyle=\sum_{k}f\ast\Psi_{k}\ast\Phi_{k},$

for every $f$ in $\mathcal{S}(N)$, with convergence in $L^{2}(N)$. Therefore
$\sum_{k}\Psi_{k}\ast\Phi_{k}$ converges to $M$ in $\mathcal{S}^{\prime}(N)$,
i.e. $F=M$ and $M$ is in $\mathcal{S}(N)$.

Finally, given $p$ there exists $q$ such that

$\|M\|_{(p,N)}\leq\|m\|_{(q,\mathbb{R}^{d})}\ .$

This follows from the Closed Graph Theorem. Indeed, we have shown that there
is a linear correspondence $m\mapsto M$ from $\mathcal{S}(\mathbb{R}^{d})$ to
$\mathcal{S}(N)$; moreover, reasoning as before, if $m_{h}\to m$ in
$\mathcal{S}(\mathbb{R}^{d})$ and $M_{h}\to\varphi$ in $\mathcal{S}(N)$ then
$M_{h}\to M$ by the Spectral Theorem in $\mathcal{S}^{\prime}(N)$. Therefore
$\varphi=M$. ∎

Going back to our case, we prove that the operators $V_{1},\dots,V_{d}$
satisfy the hypotheses of Theorem 5.2.

######  Lemma 5.3.

The differential operators $V_{1},\ldots,V_{d}$ defined in Lemma 4.1 form a
commutative family of positive Rockland operators on $H_{n}$.

###### Proof.

Suppose that $j=1,\ldots,d$. The formal self-adjointness of $V_{j}$ follows
directly from its definition and the identity
$\int_{H_{n}}Z_{k}f\,\overline{g}=-\int_{H_{n}}f\,Z_{k}\overline{g}$, for
every pair of Schwartz functions $f$ and $g$ on $H_{n}$.

We check now the injectivity condition on the image of $V_{j}$ in the
nontrivial irreducible unitary representations of $H_{n}$.

By (3) in Lemma 4.1, $d\tau_{w}(V_{j})=\rho_{j}(w)>0$ for $w\neq 0$.

As to the Bargmann representations $\pi_{\lambda}$, the Gårding space in
$\mathcal{F}_{|\lambda|}$ can be characterized as the space of those
$F=\sum_{\alpha\in\Lambda}F_{\alpha}$ (with $F_{\alpha}$ in $P_{\alpha}$ for
$\lambda>0$ and in $P_{\alpha^{\prime}}$ for $\lambda<0$) such that

$\sum_{\alpha\in\Lambda}(1+|\alpha|)^{N}\|F_{\alpha}\|_{\mathcal{F}_{|\lambda|}}^{2}<\infty$

for every integer $N$. For such an $F$,

$V_{j}F=\sum_{\alpha\in\Lambda}\widehat{V_{j}}(\phi_{\lambda,\alpha})F_{\alpha}$

where the series is convergent in norm, and therefore it is zero if and only
if $F_{\alpha}=0$ for every $\alpha$.

Positivity of $V_{j}$ follows from Plancherel’s formula: for
$f\in\mathcal{S}(H_{n})$,

$\int_{H_{n}}V_{j}f\,\bar{f}=\left(\frac{1}{2\pi}\right)^{n+1}\int_{0}^{+\infty}\sum_{\alpha\in\Lambda}\widehat{V_{j}}(\phi_{\lambda,\alpha})\big{(}\|\pi_{\lambda}(f)_{|_{P_{\alpha}}}\|_{HS}^{2}+\|\pi_{-\lambda}(f)_{|_{P_{\alpha^{\prime}}}}\|_{HS}^{2}\big{)}\,\lambda^{n}\,d\lambda\geq
0\ ,$

since the eigenvalues $\widehat{V_{j}}(\phi_{\lambda,\alpha})$ are positive.

Given a Borel subset $\omega$ of $\mathbb{R}^{+}$, define the operator
$E_{j}(\omega)$ on $L^{2}(H_{n})$ by

(5.2)
$\pi_{\lambda}\big{(}E_{j}(\omega)f\big{)}=\sum_{\alpha\in\Lambda}\chi_{\omega}\big{(}\widehat{V_{j}}(\phi_{\lambda,\alpha})\big{)})\pi_{\lambda}(f)\Pi_{\lambda,\alpha}\
,$

where $\chi_{\omega}$ is the characteristic function of $\omega$ and
$\Pi_{\lambda,\alpha}$ is the orthogonal projection of
$\mathcal{F}_{|\lambda|}$ onto $P_{\alpha}$ if $\lambda>0$, or onto
$P_{\alpha^{\prime}}$ if $\lambda<0$. Then $E_{j}=\\{E_{j}(\omega)\\}$
defines, for each $j$, a resolution of the identity, and, for
$f\in\mathcal{S}(H_{n})$,

$\int_{0}^{+\infty}\xi\,dE_{j}(\xi)f=V_{j}f\ .$

Therefore $E_{j}$ is the spectral resolution of the self-adjoint extension of
$V_{j}$. It is then clear that $E_{j}(\omega)$ and $E_{k}(\omega^{\prime})$
commute for every $\omega,\omega^{\prime}$ and $j,k$. ∎

######  Corollary 5.4.

Let $V_{0},\ldots,V_{d}$ be the differential operators defined in Lemma 4.1.
If $m$ is in $\mathcal{S}(\mathbb{R}^{d+1})$, then there exists $M$ in
$\mathcal{S}_{K}(H_{n})$ such that

$m(V_{0},\ldots,V_{d})f=f\ast M\qquad\forall f\in\mathcal{S}(H_{n}).$

Moreover, for every $p$ there exists $q$ such that

$\|M\|_{(p,H_{n})}\leq C\,\|m\|_{(q,\mathbb{R}^{d+1})}$

###### Proof.

We replace $V_{0}=-iT$ by $\tilde{V}_{0}=-iT+2\mathcal{L}$. By (4.3),
$\tilde{V}_{0}$ is a linear combination of the $V_{j}$. Therefore
$m(V_{0},\ldots,V_{d})=\tilde{m}(\tilde{V}_{0},\ldots,V_{d})$, where
$\tilde{m}$ is the composition of $m$ with a linear transformation of
$\mathbb{R}^{d+1}$.

Moreover, $\tilde{V}_{0}$ is a positive Rockland operator (cf. [15]), which
commutes with the other $V_{j}$ because so do $V_{0}$ and $\mathcal{L}$.
Applying Theorem 5.2,

$\tilde{m}(\tilde{V}_{0},\ldots,V_{d})f=f\ast M\ ,$

with $M\in\mathcal{S}_{K}(H_{n})$ and

$\|M\|_{(p,H_{n})}\leq C\,\|\tilde{m}\|_{(q,\mathbb{R}^{d+1})}\leq
C^{\prime}\,\|m\|_{(q,\mathbb{R}^{d+1})}\ .$

∎

######  Theorem 5.5.

Suppose that $m$ is a Schwartz function on $\mathbb{R}^{d+1}$. Then there
exists a function $M$ in $\mathcal{S}_{K}(H_{n})$ such that
$\widehat{M}=m_{|_{\Sigma_{K}^{V}}}$. Moreover the map $m\longmapsto M$ is a
continuous linear operator from $\mathcal{S}(\mathbb{R}^{d+1})$ to
$\mathcal{S}_{K}(H_{n})$.

###### Proof.

It follows from (5.2) that the joint spectrum of $V_{0},\dots,V_{d}$ is
$\Sigma_{K}^{V}$ (cf. [16, Theorem 1.7.10]). Therefore the continuous map
$m\mapsto M$ of Corollary 5.4 passes to the quotient modulo $\\{m:m=0\text{ on
}\Sigma_{K}^{V}\\}$.

On the other hand, by (5.2),

$\pi_{\lambda}\big{(}m(V_{0},\ldots,V_{d})f\big{)}=\sum_{\alpha\in\Lambda}m\big{(}\widehat{V}(\phi_{\lambda,\alpha})\big{)}\pi_{\lambda}(f)\text{proj}_{\lambda,\alpha}\
,$

and this must coincide with $\pi_{\lambda}(f)\pi_{\lambda}(M)$. It follows
that
$\widehat{M}\big{(}\widehat{V}(\phi_{\lambda,\alpha})\big{)}=m\big{(}\widehat{V}(\phi_{\lambda,\alpha})\big{)}$
for every $\lambda,\alpha$. By density, $\widehat{M}=m_{|_{\Sigma_{K}^{V}}}$.
∎

##  6\. Extension of Schwartz invariant functions on $\mathbb{R}^{m}$

Suppose that $K$ is a compact Lie group acting orthogonally on
$\mathbb{R}^{m}$. It follows from Hilbert’s Basis Theorem [30] that the
algebra of $K$–invariant polynomials on $\mathbb{R}^{m}$ is finitely
generated. Let $\rho_{1},\ldots,\rho_{d}$ be a set of generators and denote by
$\rho=(\rho_{1},\ldots,\rho_{d})$ the corresponding map from $\mathbb{R}^{m}$
to $\mathbb{R}^{d}$. The image $\Sigma=\rho(\mathbb{R}^{m})$ of $\rho$ is
closed in $\mathbb{R}^{d}$.

If $h$ is a smooth function on $\mathbb{R}^{d}$, then $h\circ\rho$ is in
$C^{\infty}_{K}(\mathbb{R}^{m})$, the space of $K$–invariant smooth functions
on $\mathbb{R}^{m}$. G. Schwarz [28] proved that the map $h\mapsto h\circ\rho$
is surjective from $C^{\infty}(\mathbb{R}^{d})$ to
$C^{\infty}_{K}(\mathbb{R}^{m})$, so that, passing to the quotient, it
establishes an isomorphism between $C^{\infty}(\Sigma)$ and
$C^{\infty}_{K}(\mathbb{R}^{m})$.

J. Mather [26] proved that the map $h\mapsto h\circ\rho$ is split-surjective,
i.e. there is a continuous linear operator
$\mathcal{E}:C^{\infty}_{K}(\mathbb{R}^{m})\rightarrow
C^{\infty}(\mathbb{R}^{d})$ such that $(\mathcal{E}f)\circ\rho=f$ for every
$f\in C^{\infty}_{K}(\mathbb{R}^{m})$.

From this one can derive the following analogue of the Schwarz–Mather theorem
for $\mathcal{S}_{K}(\mathbb{R}^{m})$.

######  Theorem 6.1.

There is a continuous linear operator
$\mathcal{E}^{\prime}:\mathcal{S}_{K}(\mathbb{R}^{m})\rightarrow\mathcal{S}(\mathbb{R}^{d})$
such that $(\mathcal{E}^{\prime}g)\circ\rho=g$ for every
$g\in\mathcal{S}_{K}(\mathbb{R}^{m})$. In particular, the map $g\mapsto
g\circ\rho$ is an isomorphism between $\mathcal{S}(\Sigma)$ and
$\mathcal{S}_{K}(\mathbb{R}^{m})$.

###### Proof.

It follows from Lemma 3.1 that the validity of the statement is independent of
the choice of the Hilbert basis $\rho$. We can then assume that the
polynomials $\rho_{j}$ are homogeneous of degree $\alpha_{j}$.

On $\mathbb{R}^{d}$ we define anisotropic dilations by the formula

$\delta_{r}(y_{1},\ldots,y_{k})=(r^{\alpha_{1}}\,y_{1},\ldots,r^{\alpha_{d}}\,y_{d})\qquad\forall
r>0\ ,$

and we shall denote by $|\cdot|_{\alpha}$ a corresponding homogeneous gauge,
e.g.

(6.1) $|y|_{\alpha}=c\sum_{j=1}^{d}|y_{j}|^{1/\alpha_{j}}\ ,$

satisfying $|\delta_{r}y|_{\alpha}=r|y|_{\alpha}$.

On $\mathbb{R}^{m}$ we keep isotropic dilations, given by scalar
multiplication. Clearly, $\rho$ is homogeneous of degree 1 with respect to
these dilations, i.e.,

$\rho(rx)=\delta_{r}\bigl{(}\rho(x)\bigr{)}\qquad\forall r>0,\quad
x\in\mathbb{R}^{m}\ ,$

and $\Sigma$ is $\delta_{r}$–invariant for all $r>0$.

Since $\rho$ is continuous, the image under $\rho$ of the unit sphere in
$\mathbb{R}^{m}$ is a compact set not containing 0 (in fact, since $|x|^{2}$
is a polynomial in the $\rho_{j}$, cf. [26], $\rho_{j}(x)=0$ for every $j$
implies that $x=0$).

Choosing the constant $c$ in (6.1) appropriately, we can assume that
$1\leq|\rho(x)|_{\alpha}\leq R$ for every $x$ in the unit sphere in
$\mathbb{R}^{m}$. It follows by homogeneity that for every $a,b$, $0\leq a<b$,

(6.2) $\rho\left(\\{x:a\leq|x|\leq
b\\}\right)\subset\\{y\,:\,a\leq|y|_{\alpha}\leq Rb\\}.$

Fix $\mathcal{E}:C^{\infty}(\mathbb{R}^{m})_{K}\rightarrow
C^{\infty}(\mathbb{R}^{d})$ a continuous linear operator satisfying the
condition $(\mathcal{E}f)\circ\rho=f$, whose existence is guaranteed by
Mather’s theorem.

Denote by $B_{s}$ the subset of $\mathbb{R}^{d}$ where $|y|_{\alpha}<s$. For
every $p\in\mathbb{N}$ there is $q\in N$ such that, for $f$ supported in the
unit ball,

$\|\mathcal{E}f\|_{C^{p}(B_{R^{2}})}<C_{p}\,\|f\|_{C^{q}}\ .$

Given $r>0$, set $f_{r}(x)=f(rx)$ and

$\mathcal{E}_{r}f=(\mathcal{E}f_{r})\circ\delta_{r^{-1}}\ .$

By the homogeneity of $\rho$, $(\mathcal{E}_{r}f)\circ\rho=f$. For $r>1$ and
$f$ supported on the ball of radius $r$, we have

(6.3) $\|\mathcal{E}_{r}f\|_{C^{p}(B_{rR^{2}})}<C_{p}\,r^{q}\,\|f\|_{C^{q}}\
.$

Let $\\{\varphi_{j}\\}_{j\geq 0}$ be a partition of unity on $\mathbb{R}^{m}$
consisting of radial smooth functions such that

  1. ($a$)

$\varphi_{0}$ is supported on $\\{x:|x|<1\\}$;

  2. ($b$)

for $j\geq 1$, $\varphi_{j}$ is supported on $\\{x:R^{j-2}<|x|<R^{j}\\}$;

  3. ($c$)

for $j\geq 1$, $\varphi_{j}(x)=\varphi_{1}(R^{-(j-1)}x)$.

Similarly, let $\\{\psi_{j}\\}_{j\geq 0}$ be a partition of unity on
$\mathbb{R}^{d}$ consisting of smooth functions such that

  1. ($a^{\prime}$)

$\psi_{0}$ is supported on $\\{y:|y|_{\alpha}<R\\}$;

  2. ($b^{\prime}$)

for $j\geq 1$, $\psi_{j}$ is supported on
$\\{y:R^{j-1}<|y|_{\alpha}<R^{j+1}\\}$;

  3. ($c^{\prime}$)

for $j\geq 1$, $\psi_{j}(y)=\psi_{1}(\delta_{R^{-(j-1)}}y)$.

For $f\in\mathcal{S}_{K}(\mathbb{R}^{m})$ define

$\mathcal{E}^{\prime}f(y)=\sum_{j=0}^{\infty}\sum_{\ell=-2}^{1}\psi_{j+\ell}(y)\mathcal{E}_{R^{j}}(\varphi_{j}f)=\sum_{j=0}^{\infty}\psi_{j}(y)\sum_{\ell=-2}^{1}\mathcal{E}_{R^{j-\ell}}(\varphi_{j-\ell}f)\
,$

with the convention that $\psi_{-1}=\psi_{-2}=\varphi_{-1}=0$. Then

$\mathcal{E}^{\prime}f\big{(}\rho(x)\big{)}=\sum_{j=0}^{\infty}\sum_{\ell=-2}^{1}\psi_{j+\ell}\big{(}\rho(x)\big{)}\varphi_{j}(x)f(x)\
.$

By (6.2), $\sum_{\ell=-2}^{1}\psi_{j+\ell}\big{(}\rho(x)\big{)}=1$ on the
support of $\varphi_{j}$, hence $\mathcal{E}^{\prime}f\circ\rho=f$.

We have the following estimate for the Schwartz norms in (5.1):

$\|f\|_{(p,\mathbb{R}^{m})}\sim\sum_{j=0}^{\infty}R^{jp}\|f\varphi_{j}\|_{C^{p}}\
.$

On $\mathbb{R}^{d}$ we adapt the Schwartz norms to the dilations $\delta_{r}$
by setting

$\|g\|^{\prime}_{(p,\mathbb{R}^{d})}=\sup_{y\in\mathbb{R}^{d},\sum
a_{j}\alpha_{j}\leq p}(1+|y|_{\alpha})^{p}\big{|}\partial^{\alpha}g(y)\big{|}\
.$

We then have

$\|g\|^{\prime}_{(p,\mathbb{R}^{d})}\sim\sum_{j=0}^{\infty}R^{jp}\sup_{\sum
a_{j}\alpha_{j}\leq
p}\|\partial^{\alpha}g\,\psi_{j}\|_{\infty}\lesssim\sum_{j=0}^{\infty}R^{jp}\|g\psi_{j}\|_{C^{p}}\
.$

Therefore

$\displaystyle\|\mathcal{E}^{\prime}f\|^{\prime}_{(p,\mathbb{R}^{m})}$
$\displaystyle\leq
C\sum_{j=0}^{\infty}R^{jp}\Big{\|}\psi_{j}\sum_{\ell=-2}^{1}\mathcal{E}_{R^{j-\ell}}(\varphi_{j-\ell}f)\Big{\|}_{C^{p}}$
$\displaystyle\leq
C\sum_{j=0}^{\infty}\sum_{\ell=-2}^{1}R^{jp}\|\mathcal{E}_{R^{j-\ell}}(\varphi_{j-\ell}f)\|_{C^{p}(B_{R^{j+1}})}$
$\displaystyle=C\sum_{j=0}^{\infty}\sum_{\ell=-2}^{1}R^{jp}\|\mathcal{E}_{R^{j}}(\varphi_{j}f)\|_{C^{p}(B_{R^{j+\ell+1}})}$
$\displaystyle=C\sum_{j=0}^{\infty}R^{jp}\|\mathcal{E}_{R^{j}}(\varphi_{j}f)\|_{C^{p}(B_{R^{j+2}})}\
.$

By (6.3), since $\varphi_{j}f$ is supported on the ball of radius $R^{j}$,

$\|\mathcal{E}^{\prime}f\|^{\prime}_{(p,\mathbb{R}^{m})}\leq
C_{p}\sum_{j=0}^{\infty}R^{j(p+q)}\|\varphi_{j}f\|_{C^{q}}\leq
C_{p}\|f\|_{(p+q,\mathbb{R}^{m})}\ .$

∎

##  7\. Schwartz extensions of the Gelfand transform of
$f\in\mathcal{S}_{K}(H_{n})$

In this section we suppose that $K$ is a closed connected subgroup of
${\text{U}(n)}$. The following theorem settles the proof of Theorem 1.1 in
this case.

######  Theorem 7.1.

Let $f$ be in $\mathcal{S}_{K}(H_{n})$. For every $p$ in $\mathbb{N}$ there
exist $F_{p}$ in $\mathcal{S}(\mathbb{R}^{d+1})$ and $q$ in $\mathbb{N}$, both
depending on $p$, such that ${F_{p}}_{|_{\Sigma_{K}^{V}}}=\widehat{f}$ and
$\left\|F_{p}\right\|_{(p,\mathbb{R}^{d+1})}\leq C_{p}\,\|f\|_{(q,H_{n})}.$

Notice that this statement implies the existence of a continuous map from
$\mathcal{S}(\Sigma_{K}^{V})$ to $\mathcal{S}_{K}(H_{n})$ that inverts the
Gelfand transform, even though its formulation is much weaker than that of
Theorem 6.1. We do not claim that for each $f$ a single $F$ can be found, all
of whose Schwartz norms are controlled by those of $f$. In addition, our proof
does not show if $F_{p}$ can be chosen to be linearly dependent on $f$.

The proof of Theorem 7.1 is modelled on that given in [3] for the cases
$K={\text{U}(n)},\mathbb{T}^{n}$, but with some relevant differences. On one
hand we present a simplification of the argument given there, disregarding the
partial results concerning extensions of $\widehat{f}$ with finite orders of
regularity; on the other hand extra arguments are required in the general
setting.

We need to show that the Gelfand transform $\widehat{f}$ of
$f\in\mathcal{S}_{K}(H_{n})$ extends from $\Sigma_{K}^{V}$ to a Schwartz
function on $\mathbb{R}^{d+1}$. Our starting point is the construction of a
Schwartz extension to all of $\\{0\\}\times\mathbb{R}^{d}$ of the restriction
of $\widehat{f}$ to

$\Sigma_{0}=\\{\widehat{V}(\eta_{Kw}):w\in\mathbb{C}^{n}\\}=\\{0\\}\times\rho(\mathbb{C}^{n})\
.$

If $\mathcal{F}f$ denotes the Fourier transform in
$\mathbb{C}^{n}\times\mathbb{R}$,

$\mathcal{F}f(\lambda,w)=\int_{\mathbb{C}^{n}\times\mathbb{R}}f(t,z)e^{-i(\lambda
t+\text{\rm Re}\,z\cdot\bar{w})}\,dw\,dt\ ,$

we denote

$\tilde{f}(w)=\mathcal{F}f(0,-w)=\hat{f}\big{(}0,\rho(w)\big{)}\ .$

To begin with, we set

(7.1)
${\widehat{f}\,}^{\sharp}=\mathcal{E}^{\prime}\tilde{f}\in\mathcal{S}(\mathbb{R}^{d})\
.$

Then ${\widehat{f}\,}^{\sharp}(\xi)=\widehat{f}(0,\xi)$ if
$(0,\xi)\in\Sigma_{0}$.

The next step consists in producing a Taylor development of $\widehat{f}$ at
$\lambda=0$. The following result is derived from [17]. In our setting the
formula must take into account the extended functions in (7.1).

######  Proposition 7.2.

Let $f$ be in $\mathcal{S}_{K}(H_{n})$. Then there exist functions $f_{j}$,
$j\geq 1$, in $\mathcal{S}_{K}(H_{n})$, depending linearly and continuously on
$f$, such that for any $p$ in $\mathbb{N}$,

$\widehat{f}(\lambda,\xi)=\sum_{j=0}^{p}\frac{\lambda^{j}}{j!}{\widehat{f_{j}}\,}^{\sharp}(\xi)+\frac{\lambda^{p+1}}{(p+1)!}\,{\widehat{f_{p+1}}}(\lambda,\xi),\qquad\forall(\lambda,\xi)\in\Sigma_{K},$

where $f_{0}=f$ and ${\widehat{f_{j}}\,}^{\sharp}$ is obtained from $f_{j}$
applying (7.1).

###### Proof.

For $f$ in $\mathcal{S}_{K}(H_{n})$, we claim that the restriction of
$u(\lambda,\xi)={\widehat{f}\,}^{\sharp}(\xi)$ to $\Sigma_{K}^{V}$ is in
$\mathcal{S}(\Sigma_{K}^{V})$. It is quite obvious that $u$ is smooth. Let
$\psi$ be a smooth function on the line, equal to 1 on $[-2,2]$ and supported
on $[-3,3]$. Define

$\displaystyle\Psi(\lambda,\xi)$
$\displaystyle=\psi(\lambda^{2}+\xi_{1}^{2/m_{1}}+\cdots+\xi_{d}^{2/m_{d}})$
$\displaystyle\qquad+\psi\bigg{(}\frac{\lambda^{2}}{\xi_{1}^{2/m_{1}}+\cdots+\xi_{d}^{2/m_{d}}}\bigg{)}\big{(}1-\psi(\lambda^{2}+\xi_{1}^{2/m_{1}}+\cdots+\xi_{d}^{2/m_{d}})\big{)}\
.$

By (4.1) and (2) in Lemma 4.1, $\Psi$ is equal to 1 on a neighborhood of
$\Sigma_{K}^{V}$. It is also homogeneous of degree 0 with respect to the
dilations (4.4) outside of a compact set. Then $\Psi u$ is in
$\mathcal{S}(\mathbb{R}^{d+1})$ and coincides with $u$ on $\Sigma_{K}^{V}$.

It follows from Corollary 5.4 that there exists $h$ in
$\mathcal{S}_{K}(H_{n})$ such that

$\widehat{f}(\lambda,\xi)-{\widehat{f}\,}^{\sharp}(\xi)=\widehat{h}(\lambda,\xi)\qquad\forall(\lambda,\xi)\in\Sigma^{V}_{K}.$

Since $\widehat{h}\big{(}0,\rho(w)\big{)}=0$ for every $w$,
$\int_{-\infty}^{+\infty}h(z,t)\,dt=0$ for every $z$. Therefore

$f_{1}(z,t)=\int_{-\infty}^{t}h(z,s)\,ds$

is in $\mathcal{S}_{K}(H_{n})$ and

$\widehat{h}(\lambda,\xi)=\lambda\,\widehat{f}_{1}(\lambda,\xi)\qquad\forall(\lambda,\xi)\in\Sigma^{V}_{K}.$

It easy to verify that the map $U:f\mapsto f_{1}$ is linear and continuous on
$\mathcal{S}_{K}(H_{n})$. We then define $f_{j}$, $j\geq 1$, by the recursion
formula $f_{j}=jUf_{j-1}$ and the thesis follows by induction. ∎

We use now the Whitney Extension Theorem [25] to extend the $C^{\infty}$-jet
$\\{\partial_{\xi}^{\alpha}{\widehat{f_{j}}\,}^{\sharp}\\}_{(j,\alpha)\in\mathbb{N}^{d+1}}$
to a Schwartz function on $\mathbb{R}^{d+1}$. In doing so, we must keep
accurate control of the Schwartz norms. For this purpose we use Lemma 4.1 in
[3], which reads as follows.

######  Lemma 7.3.

Let $k\geq 1$ and let $h(\lambda,\xi)$ be a $C^{k}$-function on
$\mathbb{R}^{m}\times\mathbb{R}^{n}$ such that

  1. (1)

$\partial_{\lambda}^{\alpha}h(0,\xi)=0$ for $|\alpha|\leq k$ and
$\xi\in\mathbb{R}^{n}$;

  2. (2)

for every $p\in\mathbb{N}$,

$\alpha_{p}(h)=\sup_{|\alpha|+|\beta|\leq
k}\big{\|}(1+|\cdot|)^{p}\partial_{\lambda}^{\alpha}\partial_{\xi}^{\beta}h\big{\|}_{\infty}<\infty\
.$

Then, for every $\varepsilon>0$ and $M\in\mathbb{N}$, there exists a function
$h_{\varepsilon,M}\in\mathcal{S}(\mathbb{R}^{m}\times\mathbb{R}^{n})$ such
that

  1. (1)

$\partial_{\lambda}^{\alpha}h_{\varepsilon,M}(0,\xi)=0$ for every
$\alpha\in\mathbb{N}^{m}$ and $\xi\in\mathbb{R}^{n}$;

  2. (2)

$\sup_{|\alpha|+|\beta|\leq
k-1}\big{\|}(1+|\cdot|)^{M}\partial_{\lambda}^{\alpha}\partial_{\xi}^{\beta}(h-h_{\varepsilon,M})\big{\|}_{\infty}<\varepsilon$;

  3. (3)

for every $p\in\mathbb{N}$ there is a constant $C_{k,p,M}$ such that

$\|h_{\varepsilon,M}\|_{(p,\mathbb{R}^{m+n})}\leq
C_{k,p,M}\big{(}1+\alpha_{M}(h)^{p}\varepsilon^{-p}\big{)}\big{\|}(1+|\cdot|)^{p}h\big{\|}_{\infty}\
.$

The following proposition is in [3, Proposition 4.2] for $K=\mathbb{T}^{n}$.
We give here a simplified proof.

######  Proposition 7.4.

Given $f\in\mathcal{S}_{K}(H_{n})$ and $p\in\mathbb{N}$, there are
$H\in\mathcal{S}(\mathbb{R}^{d+1})$ and $q\in\mathbb{N}$ such that
$\partial_{\lambda}^{j}H(0,\xi)={\widehat{f_{j}}\,}^{\sharp}(\xi)$ and

$\|H\|_{(p,\mathbb{R}^{d+1})}\leq C_{p}\|f\|_{(q,H_{n})}\ .$

###### Proof.

Let $\eta$ be a smooth function on $\mathbb{R}$ such that $\eta(t)=1$ if
$|t|\leq 1$ and $\eta(t)=0$ if $|t|\geq 2$. By Theorem 6.1 and Proposition
7.2, for every $k$ and $r$ there exists $q_{k,r}$ such that

(7.2) $\|\widehat{f_{k}}^{\sharp}\|_{(r,\mathbb{R}^{d})}\leq
C_{k,r}\|f\|_{(q_{k,r},H_{n})}\ .$

We fix $p\in\mathbb{N}$ and apply Lemma 7.3 to

$h_{k}(\lambda,\xi)=\eta(\lambda)\frac{\lambda^{k+1}}{(k+1)!}\widehat{f_{k+1}}^{\sharp}(\xi)\
.$

Hypothesis (1) is obviously satisfied and (2) also, because $h_{k}$ is a
Schwartz function. By (7.2),

(7.3) $\alpha_{r}(h_{k})\leq C_{k,r}\|f\|_{(q_{k+1,r},H_{n})}\ .$

Let $q$ be the maximum among the $q_{k,p}$ for $k\leq p+1$. Setting
$\varepsilon_{k}=2^{-k}\|f\|_{(q,H_{n})}$, $M=p$, for each $k$ there is a
function $H_{k}\in\mathcal{S}(\mathbb{R}^{d+1})$ such that

  1. (i)

$\partial_{\lambda}^{j}H_{k}(0,\xi)=0$ for all $j\in\mathbb{N}$ and
$\xi\in\mathbb{R}^{d}$;

  2. (ii)

$\sup_{|\alpha|+|\beta|\leq
k-1}\big{\|}(1+|\cdot|)^{p}\partial_{\lambda}^{\alpha}\partial_{\xi}^{\beta}(h_{k}-H_{k})\big{\|}_{\infty}<2^{-k}\|f\|_{(q,H_{n})}$;

  3. (iii)

for $k\leq p$, using (7.3),

$\|H_{k}\|_{(p,\mathbb{R}^{d+1})}\leq
C_{k,p}\big{(}1+\varepsilon_{k}^{-p}\|f\|_{(q,H_{n})}^{p}\big{)}\big{\|}(1+|\cdot|)^{p}h_{k}\big{\|}_{\infty}\leq
C_{p}\|f\|_{(q,H_{n})}\ .$

Define

$H=\sum_{k=0}^{p}h_{k}-\sum_{k=0}^{p}H_{k}+\sum_{k=p+1}^{\infty}(h_{k}-H_{k})\
.$

By (7.2), (ii) and (iii), the $p$-th Schwartz norm of $H$ is finite and
controlled by a constant times the $q$-th Schwartz norm of $f$.
Differentiating term by term, using (i) and the identity
$\partial_{\lambda}^{j}h_{k}(0,\xi)=\delta_{j,k+1}\widehat{f_{k+1}}^{\sharp}(\xi)$,
we obtain that $\partial_{\lambda}^{j}H(0,\xi)=\widehat{f_{j}}^{\sharp}(\xi)$
for every $j$. ∎

Let now $\varphi$ be a smooth function on $\mathbb{R}$ such that
$\varphi(t)=1$ if $|t|\leq 1/2$ and $\varphi(t)=0$ if $|t|\geq 3/4$. For $h$
defined on $\Sigma_{K}^{V}$, we define the function $Eh$ on $\mathbb{R}^{d+1}$
by

$Eh(\lambda,\xi)=\begin{cases}\displaystyle\sum_{\alpha\in\Lambda}h\big{(}\lambda,\xi_{(\lambda,\alpha)}\big{)}\prod_{\ell=1}^{d}\varphi\left(\frac{\xi_{\ell}}{|\lambda|^{m_{\ell}}}-\widehat{V_{\ell}}(\phi_{\alpha})\right)&\lambda\neq
0,\,\,\xi\in\mathbb{R}^{d}\\\ \quad
0&\lambda=0,\,\,\xi\in\mathbb{R}^{d},\end{cases}$

where
$\xi_{(\lambda,\alpha)}=\big{(}\widehat{V_{1}}(\phi_{\lambda,\alpha}),\ldots,\widehat{V_{d}}(\phi_{\lambda,\alpha})\big{)}=\big{(}|\lambda|^{m_{1}}\widehat{V_{1}}(\phi_{\alpha}),\ldots,|\lambda|^{m_{d}}\widehat{V_{d}}(\phi_{\alpha})\big{)}$.

Recall that, by Lemma 4.1, each $\widehat{V_{\ell}}(\phi_{\alpha})$ is a
positive integer. Therefore, if $\xi_{\ell}\leq 0$ for some $\ell$, every term
in the series vanishes, whereas, if $\xi$ is in $\mathbb{R}^{d}_{+}$, the
series reduces to at most one single term. Moreover, for every $g$ in
$\mathcal{S}(H_{n})$, $E\widehat{g}=\widehat{g}$ on $\Sigma^{\prime}$.

The proof of the following result goes as for [3, Lemma 3.1], using [7, p.
407] instead of [3, (2.2)]. In contrast with [3] we state it only for
vanishing of infinite order of the Taylor development of the Gelfand transform
on $\Sigma_{0}$.

######  Proposition 7.5.

Suppose that $g$ in $\mathcal{S}_{K}(H_{n})$ and
$\widehat{g_{j}}_{|_{\Sigma_{0}}}=0$ for every $j$. Then

  1. (1)

$\displaystyle E\widehat{g}(\lambda,\xi)=\widehat{g}(\lambda,\xi)$ for all
$(\lambda,\xi)\in\Sigma^{V}_{K}$;

  2. (2)

$\partial_{\lambda}^{s}(E\widehat{g})(0,\xi)=0$ for all $s$ and
$\xi\in\mathbb{R}^{d}$;

  3. (3)

for every $p\geq 0$ there exist a constant $C_{p}$ and an integer $q\geq 0$
such that

$\|E\widehat{g}\|_{(p,\mathbb{R}^{d+1})}\leq C_{p}\|g\|_{(q,H_{n})}\ .$

In particular, $E\widehat{g}\in\mathcal{S}(\mathbb{R}^{d+1})$.

To conclude the proof of Theorem 7.1, take $f$ in $\mathcal{S}_{K}(H_{n})$ and
$p$ in $\mathbb{N}$. Let $H$ be the function in
$\mathcal{S}(\mathbb{R}^{d+1})$, depending on $p$, defined as in Proposition
7.4. By Theorem 5.2, there exists $h$ in $\mathcal{S}(H_{n})$ such that

$H_{|_{\Sigma^{V}_{K}}}=\widehat{h}\ .$

Define

$F=E(\widehat{f}-\widehat{h})+H\ ,$

and the thesis follows easily.

##  8\. General compact groups of automorphisms of $H_{n}$

We have discussed in the previous sections the Gelfand pairs associated with
connected subgroups of ${\text{U}(n)}$. In this section we only assume that
$K$ is a compact group of automorphisms of $H_{n}$. Let $K_{0}$ be the
connected identity component of $K$. Then $K_{0}$ is a normal subgroup of $K$
and $F=K/K_{0}$ is a finite group. Conjugating $K$ with an automorphism if
necessary, we may suppose that $K_{0}$ is a subgroup of ${\text{U}(n)}$.

For $D$ in ${\mathbb{D}}_{K_{0}}$ and $w=kK_{0}$ in $F$, define $D^{w}$ by

$D^{w}f=D(f\circ k^{-1})\circ k\ \qquad\forall f\in C^{\infty}(H_{n}).$

Since $K_{0}$ is normal, $D^{w}$ is in ${\mathbb{D}}_{K_{0}}$. It is also
clear that ${\mathbb{D}}_{K_{0}}$ admits a generating set which is stable
under the action of the group $F$. Indeed, it suffices to add to any given
system of generators the $F$-images of its elements. Denoting by $\mathcal{V}$
the linear span of these generators, $F$ acts linearly on $\mathcal{V}$. Let
$V=\\{V_{1},\dots,V_{d}\\}$ be a basis of $\mathcal{V}$, orthonormal with
respect to an $F$–invariant scalar product. Clearly, $V$ is a generating set
for ${\mathbb{D}}_{K_{0}}$.

Applying Hilbert’s Basis Theorem as in Section 6, there exists a finite number
of (homogeneous) polynomials $\rho_{1},\ldots,\rho_{r}$ generating the
subalgebra ${P}_{F}(\mathbb{R}^{d})$ of $F$–invariant elements in
${P}(\mathbb{R}^{d})$. Let
$\rho=(\rho_{1},\ldots,\rho_{r}):\mathbb{R}^{d}\to\mathbb{R}^{r}$ be the
corresponding Hilbert map and let $W_{j}=\rho_{j}(V_{1},\ldots,V_{d})$ for
$j=1,\ldots,r$.

When $f$ is $K_{0}$–invariant and $w=kK_{0}$ in $F$, we set $f\circ w=f\circ
k$.

######  Lemma 8.1.

The set $W=\left\\{W_{1},,\ldots,W_{r}\right\\}$ generates
${{\mathbb{D}}_{K}}$. Moreover if $\psi$ is a $K$-spherical function, then

(8.1) $\psi=\frac{1}{|F|}\sum_{w\in F}\phi\circ w,$

for some $K_{0}$-spherical function $\phi$.

###### Proof.

Take $D$ in ${{\mathbb{D}}_{K}}$. As an element of ${{\mathbb{D}}_{K_{0}}}$,
$D$ is a polynomial in the $V_{j}$. Averaging over the action of $F$, we can
express $D$ as an $F$–invariant polynomial in the $V_{j}$. Hence $D$ is a
polynomial in
$W_{1}=\rho_{1}(V_{1},\ldots,V_{d}),\ldots,W_{r}=\rho_{r}(V_{1},\ldots,V_{d})$.

Recall that all $K_{0}$\- and $K$-spherical functions are of positive type
[5]. Let $\mathcal{P}_{K}$ (resp. $\mathcal{P}_{K_{0}}$) denote the convex set
of $K$–invariant (resp. $K_{0}$–invariant) functions of positive type equal to
$1$ at the identity element, and consider the linear map
$J:L^{\infty}_{K_{0}}\rightarrow L^{\infty}_{K}$ defined by
$J\varphi=\frac{1}{|F|}\sum_{w\in F}\varphi\circ w$. Since
$\mathcal{P}_{K_{0}}$ and $\mathcal{P}_{K}$ are weak*-compact and $J$ maps
$\mathcal{P}_{K_{0}}$ to $\mathcal{P}_{K}$, the extremal points of
$\mathcal{P}_{K}$ are images of extremal points of $\mathcal{P}_{K_{0}}$. This
proves that every $K$-spherical function has the form (8.1).

Conversely, if $\psi$ is given by (8.1) and $D\in{\mathbb{D}}_{K}$, then

$D\psi=\frac{1}{|F|}\sum_{w\in F}D(\varphi\circ w)=\frac{1}{|F|}\sum_{w\in
F}(D\varphi)\circ w=\hat{D}(\varphi)\,\psi\ ,$

showing that $\psi$ is $K$-spherical. ∎

From Lemma 8.1 we derive the following property of the Gelfand spectra:

$\rho(\Sigma_{K_{0}}^{V})=\Sigma_{K}^{W}\subset\mathbb{R}^{r}.$

If $V$ is as above, the linear action of $F$ on $\mathcal{V}$ leaves
$\Sigma^{V}_{K_{0}}$ invariant. For a $K_{0}$–invariant function $f$ and $w$
in $F$,

(8.2) $\mathcal{G}_{V}(f\circ w)=(\mathcal{G}_{V}f)\circ w$

Let $\mathcal{S}_{F}(\Sigma_{K_{0}}^{V})$ be the space of $F$–invariant
elements in $\mathcal{S}(\Sigma_{K_{0}}^{V})$.

######  Lemma 8.2.

The map $f\mapsto f\circ\rho$ is an isomorphism between
$\mathcal{S}(\Sigma_{K}^{W})$ and $\mathcal{S}_{F}(\Sigma_{K_{0}}^{V})$.

###### Proof.

If $f$ is in $\mathcal{S}(\Sigma_{K}^{W})$, let $\tilde{f}$ be any Schwartz
extension of $f$ to $\mathbb{R}^{r}$. Then $g=\tilde{f}\circ\rho$ is an
$F$–invariant Schwartz function on $\mathbb{R}^{d}$ and its restriction to
$\Sigma_{K_{0}}^{V}$ is $f\circ\rho$. This proves the continuity of the map.

Conversely, given $g$ in $\mathcal{S}_{F}(\Sigma_{K_{0}}^{V})$, let
$\tilde{g}$ be an $F$–invariant Schwartz extension of $g$ to $\mathbb{R}^{d}$.
Set $h=(\mathcal{E}^{\prime}\tilde{g})_{|_{\Sigma_{K}^{W}}}$, where
$\mathcal{E}^{\prime}$ is the operator of Theorem 6.1 for the group $F$. The
proof that the dependence of $h$ on $g$ is continuous is based on the simple
observation that, for any Schwartz norm $\|\ \|_{(N)}$ on $\mathbb{R}^{d}$,
the infimum of the norms of all extensions of $g$ is the same as the infimum
restricted to its $F$–invariant extensions. ∎

We can now prove Theorem 1.1 for general $K$. Assume that $K$ is a compact
group of automorphisms of $H_{n}$ and let $K_{0}$, $F$, $V_{1},\dots,V_{d}$,
$\rho$ be as above.

Take $f$ in $L^{1}_{K}(H_{n})$. Denote by $\mathcal{G}_{V}f$ (resp.
$\mathcal{G}_{W}f$) its Gelfand tranform as a $K_{0}$–invariant (resp.
$K$–invariant) function. Then $\mathcal{G}_{V}f=\mathcal{G}_{W}f\circ\rho$. In
particular, a $K_{0}$–invariant function is $K$–invariant if and only if
$\mathcal{G}_{V}f$ is $F$–invariant.

If $f$ is in $\mathcal{S}_{K}(H_{n})$, then $f$ is also $K_{0}$–invariant and
$\mathcal{G}_{V}f$ is in $\mathcal{S}_{F}(\Sigma_{K_{0}}^{V})$ by Theorem 7.1;
therefore $\mathcal{G}_{W}f$ is in $\mathcal{S}(\Sigma_{K}^{W})$ by Lemma 8.2.

Conversely, if $\mathcal{G}_{W}f$ is in $\mathcal{S}(\Sigma_{K}^{W})$, it
follows as before that $\mathcal{G}_{V}f$ is in
$\mathcal{S}_{F}(\Sigma_{K_{0}}^{V})$ and therefore $f$ is in
$\mathcal{S}_{K}(H_{n})$ by (8.2).

## References

  * [1]
  * [2]
  * [3] F. Astengo, B. Di Blasio, F. Ricci, Gelfand transforms of polyradial Schwartz functions on the Heisenberg group, J. Funct. Anal. 251 (2007), 772–791.
  * [4] C. Benson, J. Jenkins, R. L. Lipsman, G. Ratcliff, A geometric criterion for Gelfand pairs associated with the Heisenberg group, Pacific J. Math.  178  (1997), 1–36.
  * [5] C. Benson, J. Jenkins, G. Ratcliff, On Gelfand pairs associated with solvable Lie groups, Trans. Amer. Math. Soc. 321 (1990), 85–116.
  * [6] C. Benson, J. Jenkins, G. Ratcliff, Bounded spherical functions on Heisenberg groups, J. Funct. Anal. 105 (1992), 409–443.
  * [7] C. Benson, J. Jenkins, G. Ratcliff, The spherical transform of a Schwartz function on the Heisenberg group, J. Funct. Anal. 154 (1998), 379–423.
  * [8] C. Benson, J. Jenkins, G. Ratcliff, T. Worku Spectra for Gelfand pairs associated with the Heisenberg group, Colloq. Math. 71 (1996), 305-328.
  * [9] C. Benson, G. Ratcliff, Combinatorics and spherical functions on the Heisenberg group, Representation Theory 2 (1998), 79–105.
  * [10] C. Benson, G. Ratcliff, Rationality of the generalized binomial coefficients for a multiplicity free action, J. Austral. Math. Soc. Ser. A 68 (2000), 387–410.
  * [11] G. Carcano, A commutativity condition for algebras of invariant functions, Boll. Un. Mat. Ital. B 1 (1987), 1091–1105.
  * [12] F. Ferrari Ruffino, The topology of the spectrum for Gelfand pairs on Lie groups, Boll. Unione Mat. Ital. Sez. B 10 (2007), 569–579.
  * [13] G. B. Folland, Harmonic Analysis in Phase Space, Princeton University Press, Princeton, N.J., 1989.
  * [14] G. B. Folland, E. M. Stein, Hardy Spaces on Homogeneous Groups, Princeton University Press, Princeton, N.J., 1982.
  * [15] G. B. Folland, E. M. Stein, Estimates for the $\bar{\partial}_{b}$ complex and analysis on the Heisenberg group, Comm. Pure Appl. Math. 27 (1974), 429–522.
  * [16] R. Gangolli, V. S. Varadarajan, Harmonic Analysis of Spherical Function on Real Reductive Groups, Springer-Verlag, Berlin, 1988.
  * [17] D. Geller, Fourier analysis on the Heisenberg group. I. Schwartz space, J. Funct. Anal. 36 (1980), no. 2, 205–254.
  * [18] B. Helffler, J. Nourrigat Caracterisation des operateurs hypoelliptiques homogenes invariants a gauge sur un groupe de Lie gradué, Comm. Partial Differential Equations 4 (1979), 899–958.
  * [19] S. Helgason, Differential geometry and symmetric spaces, Academic Press, 1962\.
  * [20] S. Helgason, Groups and Geometric Analysis, Academic Press, 1984\.
  * [21] R. Howe, T. Umeda, The Capelli identity, the double commutant theorem and multiplicity-free actions, Math. Ann. 290 (1991), 565–619.
  * [22] A. Hulanicki, A functional calculus for Rockland operators on nilpotent Lie groups, Studia Math. 78 (1984), 253–266.
  * [23] A. Hulanicki, J. W. Jenkins, J. Ludwig, Minimum eigenvalues for positive, Rockland operators, Proc. Amer. Math. Soc. 94 (1985), 718–720.
  * [24] V. Kač, Some remarks on nilpotent orbits, J. Algebra 64 (1980), 190–213.
  * [25] B. Malgrange, Ideals of Differentiable Functions, Oxford University Press, Bombay, 1966\.
  * [26] J. N. Mather, Differentiable invariants, Topology 16 (1977), 145–155.
  * [27] E. Nelson, W. F. Stinespring, Representation of elliptic operators in an enveloping algebra, Amer. J. Math. 81 (1959), 547–560.
  * [28] G. W. Schwarz, Smooth functions invariant under the action of a compact Lie group, Topology 14 (1975), 63–68.
  * [29] A. Veneruso, Schwartz kernels on the Heisenberg group, Boll. Unione Mat. Ital. Sez. B 6 (2003), 657–666.
  * [30] H. Weyl, David Hilbert and his mathematical work, Bull. AMS 50 (1994), 612–654.