# Unbounded Weyl transform on the Euclidean motion group and Heisenberg motion
group

Somnath Ghosh and R.K. Srivastava  Department of Mathematics, Indian Institute
of Technology, Guwahati, India 781039<EMAIL_ADDRESS><EMAIL_ADDRESS>
###### Abstract.

In this article, we define Weyl transform on second countable type - $I$
locally compact group $G,$ and as an operator on $L^{2}(G),$ we prove that the
Weyl transform is compact when the symbol lies in $L^{p}(G\times\hat{G})$ with
$1\leq p\leq 2.$ Further, for the Euclidean motion group and Heisenberg motion
group, we prove that the Weyl transform can not be extended as a bounded
operator for the symbol belongs to $L^{p}(G\times\hat{G})$ with $2<p<\infty.$
To carry out this, we construct positive, square integrable and compactly
supported function, on the respective groups, such that $L^{p^{\prime}}$ norm
of its Fourier transform is infinite, where $p^{\prime}$ is the conjugate
index of $p.$

###### Key words and phrases:

Euclidean motion group, Fourier transform, Heisenberg motion group, Weyl
transform.

###### 2020 Mathematics Subject Classification:

Primary 43A15, 43A30; Secondary 33C05, 33C10

##  1\. Introduction

In [15], Hermann Weyl studied the quantization problem in quantum mechanics
and introduced a type of pseudo-differential operators. These operators are
useful in physics and mathematics, especially in PDE, harmonic analysis, time
frequency analysis. In [16], Wong called these operators as Weyl transform.
Further in [16], for the symbol in $L^{p}(\mathbb{R}^{2n})$ with $1\leq p\leq
2,$ compactness of the Weyl transform, as an operator on
$L^{2}(\mathbb{R}^{n}),$ is studied. Moreover in [12], Simon proved that the
operator is not even bounded when the symbol is in $L^{p}(\mathbb{R}^{2n})$
with $2<p<\infty.$

In general, for a locally compact group, Fourier transform is an operator
valued function. For the Heisenberg group, in [8], Weyl transform is defined
for operator valued symbol and proved its boundedness (even compactness) when
the symbol is in the corresponding $L^{p}$ spaces with $1\leq p\leq 2,$ while
unboundedness for $2<p<\infty.$ Further, in [9] Weyl transform on the upper
half plane, and in [3] Weyl transform on the quaternion Heisenberg group, are
studied.

In this article, we consider second countable type $I$ locally compact group
$G$ and define Weyl transform in view of its inversion formula. We prove that
the Weyl transform, as an operator on $L^{2}(G),$ is compact, when the symbol
is in $L^{p}(G\times\hat{G})$ with $1\leq p\leq 2,$ where $\hat{G}$ is the
dual of $G.$ Further, to prove that the Weyl transform can not be extended as
a bounded operator for the symbol is in $L^{p}(G\times\hat{G})$ with
$2<p<\infty,$ it is enough to construct a positive, square integrable and
compactly supported function on $G$ such that $L^{p^{\prime}}$ norm of its
Fourier transform is infinite, where $\frac{1}{p}+\frac{1}{p^{\prime}}=1.$ In
this perspective, we construct such type of functions for the Euclidean motion
group and Heisenberg motion group.

In addition, we want to mention that these examples will not follow directly
from the Euclidean space and Heisenberg group, respectively, for the following
reasons. The Fourier transform on the Euclidean motion group is operator
valued, whereas it is just a function for the Euclidean space. Secondly, the
case becomes more difficult for the Heisenberg motion group due to the
presence of metaplectic representation whose implicitness may not be visible
at the first instance.

##  2\. Weyl transform on locally compact groups

In this section, we recall some harmonic analysis results, namely Fourier
inversion, Plancherel formula, and Hausdorff-Young inequality on certain
locally compact groups. Then in terms of Wigner transform, we define Weyl
transform. After that, we prove the compactness of the Weyl transform for the
symbol in corresponding $L^{p}$ spaces with $1\leq p\leq 2.$ This section
concludes with a sufficient condition for the unboundedness of the Weyl
transform for $2<p<\infty.$

Let $G$ be a second countable locally compact group with type $I$ left regular
representation, and $A(G),$ $\hat{G}$ denote the Fourier algebra and dual of
$G,$ respectively. Then there is a standard measure $\mu$ on $\hat{G},$ called
the Plancherel measure, a $\mu$-measurable field $(\pi,\mathcal{H}_{\pi})$ of
representation and a measurable field $\mathcal{K}=(K_{\pi})$ of nonzero
positive self-adjoint operators such that $K_{\pi}$ is semi-invariant with
weight $\triangle^{-1},$ for almost all $\pi\in\hat{G},$ where $\triangle$ be
the modular function of $G$ and $A(G).$ If $G$ is unimodular, then
$K_{\pi}=I_{\mathcal{H}_{\pi}},$ the identity operator on $\mathcal{H}_{\pi}.$
For $f\in L^{1}(G)\cap A(G),$ the group Fourier transform

$\pi(f)=\int_{G}f(x)\pi(x)d\nu(x)$

is a bounded operator on $\mathcal{H}_{\pi},$ where $d\nu$ is a left Haar
measure on $G.$ Further, $f$ can be recovered by the inversion formula.

######  Theorem 2.1.

[7](Inversion theorem) Let $f\in L^{1}(G)\cap A(G).$ Then

$f(x)=\int_{\hat{G}}\text{tr}(\pi(x)^{-1}\pi(f)K_{\pi})d\mu(\pi).$

Next, we discuss the Plancherel formula for the Fourier transform, and for
this, we start with Schatten class operators. Let $S_{p}$ be the space of all
$p$-Schatten class operators, which is a Banach space with the norm
$\|T\|_{S_{p}}^{p}=\text{tr}(T^{*}T)^{p/2}$ for $1\leq p<\infty$ and for
$p=\infty,$ the norm is the usual operator norm.

######  Theorem 2.2.

[4](Plancherel formula) Let $f\in L^{1}\cap L^{2}(G).$ Then

$\int_{G}|f(x)|^{2}d\nu(x)=\int_{\hat{G}}\|\pi(f)K_{\pi}^{1/2}\|_{S_{2}}^{2}~{}d\mu(\pi).$

This map $L^{1}\cap L^{2}(G)\rightarrow L^{2}(\hat{G})$ can be extended
uniquely to a unitary map from $L^{2}(G)$ onto $L^{2}(\hat{G}).$

Let $f\in L^{1}\cap L^{p}(G),$ where $1\leq p\leq\infty.$ Define the $L^{p}$
Fourier transform of $f$ by
$\mathcal{F}_{p}(f)\pi=\pi(f)K_{\pi}^{1/p^{\prime}},$ where
$\frac{1}{p}+\frac{1}{p^{\prime}}=1.$ Then the following Hausdorff-Young
inequality holds.

######  Theorem 2.3.

[7] If $f\in L^{1}\cap L^{p}(G),$ where $1<p<2,$ then $\mathcal{F}_{p}(f)\in
L^{p^{\prime}}(\hat{G})$ and this map $f\mapsto\mathcal{F}_{p}(f)$ extends
uniquely to a bounded linear map from $L^{p}(G)$ into
$L^{p^{\prime}}(\hat{G})$ with norm less than or equal to $1.$

To study the Weyl transform, we need to define Wigner transform on $G$ and
investigate its boundedness properties. For this, we first describe the
following product spaces and the left translation operator.

Consider the measure $d\nu\otimes d\mu$ on $G\times\hat{G}$ and for $1\leq
p\leq\infty,$ let $L^{p}(G\times\hat{G},S_{p},d\nu\otimes d\mu)$ be the space
of $S_{p}$ valued functions satisfying

$\displaystyle\|f\|_{p,\nu\otimes\mu}^{p}=$
$\displaystyle\int_{G\times\hat{G}}\|f(x,\pi)K_{\pi}^{1/p}\|_{S_{p}}^{p}d\nu(x)d\mu(\pi)<\infty,\qquad
1\leq p<\infty,$ $\displaystyle\|f\|_{\infty,\nu\otimes\mu}=$
$\displaystyle\underset{(x,\pi)\in G\times\hat{G}}{\text{ess
sup}}\|f(x,\pi)\|_{S_{\infty}}<\infty.$

The left translation operator is defined by
$\tau_{x^{\prime}}f(x)=f(x^{\prime-1}x)$ for $f\in L^{p}(G).$ Further,
$C_{c}(G)$ denotes the space of all compactly supported continuous functions
on $G.$ Throughout this section, $p,p^{\prime}$ are the conjugate indices.

######  Definition 2.4.

Let $f,g\in C_{c}(G)$ and $(x,\pi)\in G\times\hat{G}.$ Then Wigner transform
associated with $f,g$ is defined by

$V(f,g)(x,\pi)=\int_{G}f(x^{\prime})\tau_{x^{\prime}}g(x)\pi(x^{\prime})d\nu(x^{\prime}).$

That is, $V(f,g)(x,\pi)=\pi{\left(f\cdot\tau g(x)\right)}.$

The following result proves that the Wigner transforms are in certain $L^{p}$
spaces.

######  Proposition 2.5.

Let $f,g\in C_{c}(G).$ Then $V(f,g)\in
L^{p^{\prime}}(G\times\hat{G},S_{p^{\prime}},d\nu\otimes d\mu)$ and

(2.1)
$\displaystyle\|V(f,g)\|_{p^{\prime},\nu\otimes\mu}\leq\|f\|_{2}\|g\|_{2}$

for $p^{\prime}\in[2,\infty].$ Thus $V:C_{c}(G)\times C_{c}(G)\rightarrow
L^{p^{\prime}}(G\times\hat{G},S_{p^{\prime}},d\nu\otimes d\mu)$ can be
extended uniquely to a bilinear operator $V:L^{2}(G)\times L^{2}(G)\rightarrow
L^{p^{\prime}}(G\times\hat{G},S_{p^{\prime}},d\nu\otimes d\mu)$ with

$\displaystyle\|V(f,g)\|_{p^{\prime},\nu\otimes\mu}\leq\|f\|_{2}\|g\|_{2}.$

###### Proof.

Let $p^{\prime}=\infty.$ Then

$\displaystyle\|V(f,g)\|_{\infty,\nu\otimes\mu}=$
$\displaystyle\underset{(x,\pi)\in G\times\hat{G}}{\text{ess
sup}}\|V(f,g)(x,\pi)\|_{S_{\infty}}$ $\displaystyle=$
$\displaystyle\underset{(x,\pi)\in G\times\hat{G}}{\text{ess
sup}}\|\int_{G}f(x^{\prime})\tau_{x^{\prime}}g(x)\pi(x^{\prime})d\nu(x^{\prime})\|_{S_{\infty}}$
$\displaystyle\leq$ $\displaystyle\,\underset{x\in G}{\text{ess
sup}}\int_{G}|f(x^{\prime})\tau_{x^{\prime}}g(x)|d\nu(x^{\prime})\leq\|f\|_{2}\|g\|_{2}.$

For $p^{\prime}=2,$ Plancherel formula, Theorem 2.2, gives

$\displaystyle\|V(f,g)\|_{2,\nu\otimes\mu}^{2}=$
$\displaystyle\int_{G}\int_{\hat{G}}\|\pi{\left(f\cdot\tau
g(x)\right)}K_{\pi}^{1/2}\|_{S_{2}}^{2}d\mu(\pi)d\nu(x)$ $\displaystyle=$
$\displaystyle\int_{G}\int_{G}|f(x^{\prime})\tau_{x^{\prime}}g(x)|^{2}d\nu(x^{\prime})d\nu(x)=\|f\|_{2}^{2}\|g\|_{2}^{2}.$

Then Riesz-Thorin interpolation theorem completes the proof. ∎

The following proposition is a way of writing the group Fourier transform in
terms of the Wigner transform.

######  Proposition 2.6.

Let $f,g\in L^{1}\cap L^{2}(G)$ and $\mathrm{C}=\int_{G}g(x)d\nu(x)\neq 0.$
Then $\pi(f)=\mathrm{C}^{-1}\int_{G}V(f,g)(x,\pi)d\nu(x)$ for $\pi\in\hat{G}.$

###### Proof.

$\displaystyle\int_{G}V(f,g)(x,\pi)d\nu(x)=$
$\displaystyle\int_{G}\int_{G}f(x^{\prime})g(x^{\prime-1}x)\pi(x^{\prime})d\nu(x)d\nu(x^{\prime})$
$\displaystyle=$
$\displaystyle\left(\int_{G}g(x)d\nu(x)\right)\left(\int_{G}f(x^{\prime})\pi(x^{\prime})d\nu(x^{\prime})\right)=\mathrm{C}~{}\pi(f).$

∎

In view of Proposition 2.6, the Inversion formula 2.1 can be reformulated.

######  Corollary 2.7.

Let $f\in L^{1}(G)\cap A(G)$ and $g\in L^{1}\cap L^{2}(G)$ with
$\mathrm{C}=\int_{G}g(x)d\nu(x)\neq 0.$ Then

$f(x)=\mathrm{C}^{-1}\int_{\hat{G}}\text{tr}\left(\pi(x)^{-1}\left(\int_{G}V(f,g)(x^{\prime},\pi)d\nu(x^{\prime})\right)K_{\pi}\right)d\mu(\pi).$

######  Definition 2.8.

Let $\varsigma\in L^{p}(G\times\hat{G},S_{p},d\nu\otimes d\mu),$ where $1\leq
p\leq 2.$ Then corresponding to $\varsigma,$ Weyl transform
$W_{\varsigma}:L^{2}(G)\rightarrow L^{2}(G)$ is defined by

$\displaystyle\langle W_{\varsigma}f,\bar{g}\rangle=\langle
V(f,g),\varsigma\rangle_{\nu\otimes\mu}=\int_{G}\int_{\hat{G}}\text{tr}\left(\varsigma^{*}(x,\pi)V(f,g)(x,\pi)K_{\pi}\right)d\mu(\pi)d\nu(x),$

where $f,g\in L^{2}(G).$ Here, by the abuse of notation
$\langle\cdot\,,\cdot\rangle_{\nu\otimes\mu}$ is used.

After a bit of calculation, we can conclude that

(2.2) $\displaystyle
W_{\varsigma}f(x)=\int_{\hat{G}}\int_{G}\text{tr}\left(\varsigma^{*}(x^{\prime}x,\pi)\pi(x^{\prime})K_{\pi}\right)f(x^{\prime})d\nu(x^{\prime})d\mu(\pi).$

Thus $W_{\varsigma}:L^{2}(G)\rightarrow L^{2}(G)$ is an integral operator with
kernel

$K(x,x^{\prime})=\int_{\hat{G}}\text{tr}\left(\varsigma^{*}(x^{\prime}x,\pi)\pi(x^{\prime})K_{\pi}\right)d\mu(\pi).$

In the following proposition, we investigate the boundedness of the Weyl
transform.

######  Proposition 2.9.

Let $\varsigma\in L^{p}(G\times\hat{G},S_{p},d\nu\otimes d\mu),$ where $1\leq
p\leq 2.$ Then $\|W_{\varsigma}\|\leq\|\varsigma\|_{p,\nu\otimes\mu}.$

###### Proof.

Since $|\text{tr}(A^{*}B)|\leq\|A\|_{S_{p}}\|B\|_{S_{p^{\prime}}},$ for
$f,g\in L^{2}(G)$ we have

$\displaystyle|\langle W_{\varsigma}f,\bar{g}\rangle|$
$\displaystyle\leq\int_{G}\int_{\hat{G}}|\text{tr}\left(\varsigma^{*}(x,\pi)V(f,g)(x,\pi)K_{\pi}\right)|d\mu(\pi)d\nu(x)$
$\displaystyle\leq\int_{G}\int_{\hat{G}}\|\varsigma(x,\pi)(K_{\pi}^{1/p})^{*}\|_{S_{p}}\|V(f,g)(x,\pi)K_{\pi}^{1/p^{\prime}}\|_{S_{p^{\prime}}}d\mu(\pi)d\nu(x).$

By Hölder’s inequality, the above integral is less than or equal to

$\displaystyle\left(\int_{G}\int_{\hat{G}}\|\varsigma(x,\pi)K_{\pi}^{1/p}\|_{S_{p}}^{p}d\mu(\pi)d\nu(x)\right)^{\frac{1}{p}}\left(\int_{G}\int_{\hat{G}}\|V(f,g)(x,\pi)K_{\pi}^{1/p^{\prime}}\|_{S_{p^{\prime}}}^{p^{\prime}}d\mu(\pi)d\nu(x)\right)^{\frac{1}{p^{\prime}}}$
$\displaystyle=\|\varsigma\|_{p,\nu\otimes\mu}\|V(f,g)\|_{p^{\prime},\nu\otimes\mu}\leq\|\varsigma\|_{p,\nu\otimes\mu}\|f\|_{2}\|g\|_{2}.$

∎

Further for $p=2,$ $W_{\varsigma}$ is a Hilbert-Schmidt operator, and for
$p=1,$ $W_{\varsigma}$ is a trace class operator.

######  Proposition 2.10.

If $\varsigma\in L^{2}(G\times\hat{G},S_{2},d\nu\otimes d\mu),$ then
$W_{\varsigma}$ is a Hilbert-Schmidt operator.

###### Proof.

Since $W_{\varsigma}$ is an integral operator, we have

$\displaystyle\|W_{\varsigma}\|_{S_{2}}^{2}$
$\displaystyle=\int_{G}\int_{G}|K(x,x^{\prime})|^{2}d\nu(x)d\nu(x^{\prime})$
$\displaystyle=\int_{G}\int_{G}\left|\int_{\hat{G}}\text{tr}\left(\varsigma^{*}(x^{\prime}x,\pi)\pi(x^{\prime})K_{\pi}\right)d\mu(\pi)\right|^{2}d\nu(x)d\nu(x^{\prime})$
$\displaystyle=\int_{G}\int_{G}\left|\int_{\hat{G}}\text{tr}\left(\varsigma^{*}(x,\pi)\pi(x^{\prime})K_{\pi}\right)d\mu(\pi)\right|^{2}d\nu(x)d\nu(x^{\prime}),$

where last equality is ensured by applying the change of variable
$x=x^{\prime}x,$ as $d\nu$ is a left invariant Haar measure. Since
$\text{tr}(T^{*})=\overline{\text{tr}(T)}$ and $K_{\pi}^{*}=K_{\pi},$ the
above integral becomes

$\displaystyle\int_{G}\int_{G}\left|\int_{\hat{G}}\text{tr}\left(K_{\pi}\pi(x^{\prime})^{-1}\varsigma(x,\pi)\right)d\mu(\pi)\right|^{2}d\nu(x)d\nu(x^{\prime}).$

Applying the Fourier inversion and Plancherel formula, we get

$\displaystyle\|W_{\varsigma}\|_{S_{2}}^{2}$
$\displaystyle=\int_{G}\int_{G}|\mathcal{F}^{-1}(\varsigma)(x,\cdot)(x^{\prime})|^{2}d\nu(x^{\prime})d\nu(x)$
$\displaystyle=\int_{G}\int_{\hat{G}}\|\varsigma(x,\pi)K_{\pi}^{1/2}\|_{S_{2}}^{2}d\mu(\pi)d\nu(x)=\|\varsigma\|_{2,\nu\otimes\mu}^{2}.$

∎

######  Proposition 2.11.

If $\varsigma\in L^{1}(G\times\hat{G},S_{1},d\nu\otimes d\mu),$ then
$W_{\varsigma}$ is a trace class operator.

###### Proof.

Since $W_{\varsigma}$ is an integral operator

$\displaystyle\|W_{\varsigma}\|_{S_{1}}$
$\displaystyle=\int_{G}|K(x,x)|d\nu(x)$
$\displaystyle=\int_{G}\left|\int_{\hat{G}}\text{tr}\left(\varsigma^{*}(xx,\pi)\pi(x)K_{\pi}\right)d\mu(\pi)\right|d\nu(x)$
$\displaystyle\leq\int_{G}\int_{\hat{G}}\left|\text{tr}\left(\pi(x)K_{\pi}\varsigma^{*}(xx,\pi)\right)\right|d\mu(\pi)d\nu(x)$
$\displaystyle\leq\int_{G}\int_{\hat{G}}\text{tr}\left(\left|\varsigma^{*}(xx,\pi)K_{\pi}\right|\right)d\mu(\pi)d\nu(x)=\|\varsigma\|_{1,\nu\otimes\mu},$

where the second last inequality true because of
$|\text{tr}(T)|\leq\text{tr}(T^{*}T)^{1/2}=\text{tr}(|T|).$ ∎

######  Theorem 2.12.

If $\varsigma\in L^{p}(G\times\hat{G},S_{p},d\nu\otimes d\mu),$ where $1\leq
p\leq 2,$ then $W_{\varsigma}$ is a compact operator.

###### Proof.

Let $\varsigma\in L^{p}(G\times\hat{G},S_{p},d\nu\otimes d\mu)$ and
$(\varsigma_{n})$ be a sequence in $C_{c}(G\times\hat{G})$ such that
$\|\varsigma_{n}-\varsigma\|_{p,\nu\otimes\mu}\rightarrow 0.$ From Proposition
2.9, it follows that $W_{\varsigma_{n}}$ converges to $W_{\varsigma}$ in
operator norm. But in view of Proposition 2.10, $W_{\varsigma_{n}}$ is a
compact operator for each $n.$ Hence $W_{\varsigma}$ is a compact operator. ∎

Now, we shall see whether the Weyl transform defined in (2.2) can be extended
as a bounded operator for $p>2.$ Next result gives a necessary and sufficient
condition for boundedness of the Weyl transform in terms of the Wigner
transform.

######  Proposition 2.13.

Let $2<p<\infty$ and $\frac{1}{p}+\frac{1}{p^{\prime}}=1.$ Then for all
$\varsigma\in L^{p}(G\times\hat{G},S_{p},d\nu\otimes d\mu),$ the Weyl
transform $W_{\varsigma}$ is a bounded operator on $L^{2}(G)$ if and only if
there exists a constant $C$ such that
$\|V(f,g)\|_{p^{\prime},\nu\otimes\mu}\leq C\|f\|_{2}\|g\|_{2}$ for all
$f,g\in L^{2}(G).$

###### Proof.

A similar argument as in Proposition 2.9 proves that $W_{\varsigma}$ is
bounded for all $\varsigma\in L^{p}(G\times\hat{G},S_{p},d\nu\otimes d\mu),$
whenever there exists a constant $C$ such that
$\|V(f,g)\|_{p^{\prime},\nu\otimes\mu}\leq C\|f\|_{2}\|g\|_{2}$ for all
$f,g\in L^{2}(G).$

Conversely, assume that $W_{\varsigma}$ is bounded for each $\varsigma\in
L^{p}(G\times\hat{G},S_{p},d\nu\otimes d\mu).$ Then there exists a constant
$C_{\varsigma}$ such that $\|W_{\varsigma}f\|_{2}\leq C_{\varsigma}\|f\|_{2}$
for all $f\in L^{2}(G).$ For $f,g\in C_{c}(G)$ with $\|f\|_{2}=\|g\|_{2}=1,$
define the bounded linear functional on
$L^{p}(G\times\hat{G},S_{p},d\nu\otimes d\mu)$ by
$\mathcal{Q}_{f,g}(\varsigma)=\langle W_{\varsigma}f,g\rangle.$ Then
$\sup|\mathcal{Q}_{f,g}(\varsigma)|\leq C_{\varsigma},$ where the supremum is
over all $f,g\in C_{c}(G)$ with $\|f\|_{2}=\|g\|_{2}=1.$ Therefore, by the
uniform boundedness principle, there exists a constant $C$ such that
$\|\mathcal{Q}_{f,g}\|\leq C$ for all $f,g\in C_{c}(G)$ with
$\|f\|_{2}=\|g\|_{2}=1.$ Hence $|\langle W_{\varsigma}f,g\rangle|\leq
C\|\varsigma\|_{p,\nu\otimes\mu}\|f\|_{2}\|g\|_{2},$ that is,
$\|W_{\varsigma}\|\leq C\|\varsigma\|_{p,\nu\otimes\mu}.$ Thus for $f,g\in
C_{c}(G),$

$\displaystyle\|V(f,g)\|_{p^{\prime},\nu\otimes\mu}=\sup\limits_{\|\varsigma\|_{p,\nu\otimes\mu}=1}|\langle
V(f,g),\varsigma\rangle_{\nu\otimes\mu}|=\sup\limits_{\|\varsigma\|_{p,\nu\otimes\mu}=1}|\langle
W_{\varsigma}f,\bar{g}\rangle|\leq C\|f\|_{2}\|g\|_{2}.$

The density argument completes the proof. ∎

A sufficient condition for unboundedness of the Weyl transform is obtained in
the following result.

######  Proposition 2.14.

Let $2<p<\infty$ and $f$ be a square integrable, compactly supported function
on $G$ with $\int_{G}f(x)d\nu(x)\neq 0.$ If $W_{\varsigma}$ is a bounded
operator on $L^{2}(G)$ for all $\varsigma\in
L^{p}(G\times\hat{G},S_{p},d\nu\otimes d\mu),$ then
$\int_{\hat{G}}\|\mathcal{F}_{p}(f)\pi\|_{S_{p^{\prime}}}^{p^{\prime}}~{}d\mu(\pi)<\infty.$

###### Proof.

Let $f$ be supported on a compact set $K.$ Then $\tilde{K}=KK=\\{xy:x,y\in
K\\}$ is compact and $V(f,f)$ is supported on $\tilde{K}\times\hat{G}.$ In
view of Proposition 2.6, instead of
$\int_{\hat{G}}\|\mathcal{F}_{p}(f)\pi\|_{S_{p^{\prime}}}^{p^{\prime}}~{}d\mu(\pi),$
it is enough to consider the following integral

$\displaystyle\int_{\hat{G}}\left\|\int_{G}V(f,f)(x,\pi)K_{\pi}^{1/p^{\prime}}d\nu(x)\right\|_{S_{p^{\prime}}}^{p^{\prime}}d\mu(\pi).$

Applying Minkowski’s integral inequality and Hölder’s inequality, the above
integral is less than or equal to

$\displaystyle\left(\int_{\tilde{K}}\left(\int_{\hat{G}}\|V(f,f)(x,\pi)K_{\pi}^{1/p^{\prime}}\|_{S_{p^{\prime}}}^{p^{\prime}}d\mu(\pi)\right)^{\frac{1}{p^{\prime}}}d\nu(x)\right)^{p^{\prime}}$
$\displaystyle\leq$
$\displaystyle\left(\int_{\tilde{K}}d\nu(x)\right)^{p^{\prime}/p}\int_{\tilde{K}}\int_{\hat{G}}\|V(f,f)(x,\pi)K_{\pi}^{1/p^{\prime}}\|_{S_{p^{\prime}}}^{p^{\prime}}d\mu(\pi)d\nu(x).$

Hence by Proposition 2.13, it completes the proof. ∎

To prove that the Weyl transform $W_{\varsigma}$ can not be extended as a
bounded operator for $\varsigma\in L^{p}(G\times\hat{G},S_{p},d\nu\otimes
d\mu),$ where $2<p<\infty,$ it is enough to consider the following problem.

Question : For $p\in(2,\infty),$ does there exists a square integrable,
compactly supported function $f$ on $G$ with $\int_{G}f(x)d\nu(x)\neq 0,$ such
that
$\int_{\hat{G}}\|\mathcal{F}_{p}(f)\pi\|_{S_{p^{\prime}}}^{p^{\prime}}~{}d\mu(\pi)=\infty$
?

In [12], Simon gave the example of such functions for $\mathbb{R}^{n},$ and
later on, it is considered for the Heisenberg group [8], quaternion Heisenberg
group [3] and upper half plane [9]. In this article, we define such functions
for the Euclidean motion group and Heisenberg motion group.

##  3\. Euclidean motion group

In this section, we briefly discuss the Fourier analysis on the Euclidean
motion group. Thereafter, we shall precisely write down the formula for the
Weyl transform in this setup and prove that it can not be extended as a
bounded operator for the symbol in the corresponding $L^{p}$ spaces with
$2<p<\infty.$

Let $n\in\mathbb{N}\setminus\\{1\\}$ and $SO(n)$ be the special orthogonal
group of order $n.$ Then Euclidean motion group $M(n)$ is the semidirect
product of $\mathbb{R}^{n}$ with $K=SO(n).$ The group law on $M(n)$ can be
expressed as

$(x_{1},k_{1})(x_{2},k_{2})=(x_{1}+k_{1}\cdot x_{2},k_{1}k_{2}),$

where $x_{1},x_{2}\in\mathbb{R}^{n}$ and $k_{1},k_{2}\in K.$ The Haar measure
on $M(n)$ can be written as $d\nu(x,k)=dxdk,$ where $dx$ and $dk$ are the Haar
measures on $\mathbb{R}^{n}$ and $K$ respectively. The Fourier analysis on
$M(n)$ can be discussed in the following two cases. For details, see [6, 10,
13].

For $n=2$ : An arbitrary element of $M(2)$ can be written as
$(z,e^{i\varphi}),$ where $z\in\mathbb{C},\varphi\in\mathbb{R}.$ All the, upto
unitarily equivalent, infinite dimensional irreducible unitary representations
of $M(2)$ are parametrized by $a>0.$ Moreover, for each $a>0,$ the
representation $\pi_{a},$ realized on $L^{2}(S^{1}),$ is given by

$\pi_{a}(z,e^{i\varphi})g(\theta)=e^{i\text{Re}(ae^{i\theta}z)}g(\theta-\varphi),$

where $g\in L^{2}(S^{1}).$ The Plancherel measure on $(0,\infty)$ is
$d\mu(a)=ada,$ where $da$ is the Lebesgue measures on $(0,\infty).$ Further,
for $f\in L^{1}(M(2))$ the group Fourier transform, defined by

$\displaystyle\pi_{a}(f)=\hat{f}(a)=\int_{M(2)}f(z,e^{i\varphi})\pi_{a}(z,e^{i\varphi})dzd\varphi,$

is a bounded operator on $L^{2}(S^{1}).$ For $\varsigma\in
L^{p}(M(2)\times(0,\infty),S_{p},d\nu\otimes d\mu)$ and $f\in L^{2}(M(2)),$
the Weyl transform $W_{\varsigma}$ takes the form

$\displaystyle
W_{\varsigma}f(z,e^{i\varphi})=\int_{0}^{\infty}\int_{M(2)}\text{tr}\left(\varsigma^{*}((w,e^{i\psi})(z,e^{i\varphi}))\pi_{a}(w,e^{i\psi})\right)f(w,e^{i\psi})adwd\psi
da.$

For $n\geq 3$ : Let $M=SO(n-1)$ be the subgroup of $K$ that fixes the point
$e_{1}=(1,0,\ldots,0).$ Let $\hat{M}$ be the unitary dual group of $M.$ Given
a unitary irreducible representation $\sigma\in\hat{M},$ realized on the
Hilbert space $\mathcal{H}_{\sigma}$ of dimension $d_{\sigma},$ consider the
space $L^{2}(K,\mathbb{C}^{d_{\sigma}\times d_{\sigma}})$ consisting of
$d_{\sigma}\times d_{\sigma}$ complex matrices valued functions $\varphi$ on
$K$ such that $\varphi(uk)=\sigma(u)\varphi(k),$ where $u\in M,~{}k\in K,$ and
satisfying

$\int_{K}\text{tr}(\varphi(k)^{\ast}\varphi(k))dk<\infty.$

For $(a,\sigma)\in(0,\infty)\times\hat{M},$ a unitary representation
$\pi_{a,\sigma}$ of $M(n)$ is defined by

$\pi_{a,\sigma}(x,k)(\varphi)(s)=e^{ia\left\langle s^{-1}\cdot
e_{1},\,x\right\rangle}\varphi(sk),$

where $\varphi\in L^{2}\left(K,\mathbb{C}^{d_{\sigma}\times
d_{\sigma}}\right).$ These are all, up to unitary equivalent, infinite
dimensional, unitary, irreducible representations that appear in the
Plancherel formula. The group Fourier transform of a function $f\in
L^{1}(M(n))$ is defined by

$\hat{f}(a,\sigma)=\int_{M(n)}f(x,k)\pi_{a,\,\sigma}(x,k)dxdk.$

Further, for $f\in L^{1}\cap L^{2}(M(n)),$ Plancheral formula holds

$\int_{M(n)}|f(x,k)|^{2}dxdk=\mathsf{c}_{n}\int_{0}^{\infty}\left(\sum_{\sigma\in\hat{M}}d_{\sigma}\|\hat{f}(a,\sigma)\|_{HS}^{2}\right)a^{n-1}da,$

for some constant $\mathsf{c}_{n}.$

Next, we construct the required square integrable and compactly supported
functions for $M(n).$ To do so, we need to discuss some properties of Bessel
functions.

For $\upsilon\in\mathbb{R},$ first kind Bessel function $J_{\upsilon}$ of
order $\upsilon$ is defined by

(3.1) $\displaystyle
J_{\upsilon}(t)=\sum_{l=0}^{\infty}\dfrac{(-1)^{l}(\frac{t}{2})^{\upsilon+2l}}{l!\,\Gamma(l+\upsilon+1)},$

where $t$ is a non-negative real number. Then for $n\geq 2,$ $a\in(0,\infty)$
and $x\in\mathbb{R}^{n},$ the following relation holds

(3.2)
$\displaystyle\int_{S^{n-1}}e^{iax\cdot\omega}d\omega=c_{n}\,(a|x|)^{1-\frac{n}{2}}J_{\frac{n}{2}-1}(a|x|),$

where $c_{n}>0$ is a constant depending only on $n$ and $d\omega$ is the
surface measure on $S^{n-1}.$

If $\xi,\zeta$ and $\eta$ are complex parameters with $\eta\neq 0,-1,\ldots,$
then the complex power series

(3.3)
$\displaystyle\sum_{l=0}^{\infty}\dfrac{(\xi)_{l}(\zeta)_{l}}{(\eta)_{l}}\,\frac{z^{l}}{l\,!},$

converges for $|z|<1,$ where $(\xi)_{0}=1$ and
$(\xi)_{l}=\xi(\xi+1)\cdots(\xi+l-1)$ for $l\geq 1.$ The sum of the above
series is denoted by ${}_{2}F_{1}(\xi,\zeta;\eta\,;z)$ and called it
hypergeometric function. Moreover, if Re$(\eta-\xi-\zeta)>0,$ then the series
(3.3) also converges for $|z|=1$ and

(3.4)
$\displaystyle{}_{2}F_{1}(\xi,\zeta;\eta\,;1)=\frac{\Gamma(\eta)\Gamma(\eta-\xi-\zeta)}{\Gamma(\eta-\xi)\Gamma(\eta-\zeta)}.$

For more details, see [1], page no. 62-66.

The following property of Bessel functions hold, see [14], page no. 385.

######  Lemma 3.1.

[14] Let $u,\alpha,\upsilon\in\mathbb{R}$ be such that $u,\alpha>0$ and
$\upsilon\geq 0.$ Then

$\displaystyle\int_{0}^{\infty}e^{-ut}t^{\alpha-1}J_{\upsilon}(t)dt$
$\displaystyle=\dfrac{(\frac{1}{2})^{\upsilon}\Gamma(\alpha+\upsilon)}{(1+u^{2})^{\frac{\alpha+\upsilon}{2}}\Gamma(\upsilon+1)}\,{}_{2}F_{1}\left(\frac{\alpha+\upsilon}{2},\frac{1-\alpha+\upsilon}{2};1+\upsilon\,;\frac{1}{1+u^{2}}\right),$

where ${}_{2}F_{1}$ is the hypergeometric function as defined by the series
(3.3).

Next lemma is needful to prove the main result in this section.

######  Lemma 3.2.

Let $n\in\mathbb{N}\setminus\\{1\\}$ and $\beta\in(\frac{n}{3},\frac{n}{2}).$
Then there exists $C>0$ and $a_{0}>0,$ both independent of $\beta,$ such that

$\displaystyle\int_{0}^{a}t^{\frac{n}{2}-\beta}J_{\frac{n}{2}-1}(t)dt\geq C$

for all $a\geq a_{0}.$

###### Proof.

Let $\frac{n}{3}<\beta<\frac{n}{2}$ and $\alpha=\frac{n}{2}-\beta+1,$ then
$1<\alpha<\frac{n}{6}+1.$ Again for $0<u<1,$ we have
$\frac{1}{2}<\frac{1}{1+u^{2}}<1.$ Now, we shall find an independent of $u$
and positive lower bound of the integral define in Lemma 3.1, when
$\alpha=\frac{n}{2}-\beta+1$ and $\upsilon=\frac{n}{2}-1.$ For that, consider
the following two cases.

For $n=2$ : Since $1<\alpha<\frac{4}{3},$ except for the first term, all other
terms in the series expansion of
${}_{2}F_{1}\left(\frac{\alpha}{2},\frac{1-\alpha}{2};1;\frac{1}{1+u^{2}}\right)$
are negative. Further, as $1-\frac{\alpha}{2}-\frac{1-\alpha}{2}>0,$ we get

$\displaystyle{}_{2}F_{1}\left(\frac{\alpha}{2},\frac{1-\alpha}{2};1;\frac{1}{1+u^{2}}\right)$
$\displaystyle=1+\sum_{l=1}^{\infty}\dfrac{(\frac{\alpha}{2})_{l}(\frac{1-\alpha}{2})_{l}}{(1)_{l}\,l!}\frac{1}{(1+u^{2})^{l}}$
(3.5) $\displaystyle\geq
1+\sum_{l=1}^{\infty}\dfrac{(\frac{\alpha}{2})_{l}(\frac{1-\alpha}{2})_{l}}{(1)_{l}\,l!}.$

From Lemma 3.1, using (3.4 \- 3), we have

$\displaystyle\int_{0}^{\infty}e^{-ut}t^{1-\beta}J_{0}(t)dt$
$\displaystyle=\int_{0}^{\infty}e^{-ut}t^{\alpha-1}J_{0}(t)dt$
$\displaystyle=\dfrac{\Gamma(\alpha)}{(1+u^{2})^{\alpha/2}}\,{}_{2}F_{1}\left(\frac{\alpha}{2},\frac{1-\alpha}{2};1;\frac{1}{1+u^{2}}\right)$
(3.6)
$\displaystyle\geq\frac{\Gamma(2)}{2}\frac{\Gamma(\frac{1}{2})}{\Gamma(1-\frac{\alpha}{2})\Gamma(\frac{\alpha+1}{2})}>0,$

where the last inequality true as $1\leq(1+u^{2})^{\alpha/2}\leq
2^{\alpha/2}\leq 2$ for $1<\alpha<\frac{4}{3}.$

For $n\geq 3$ : Since $\frac{n}{3}<\beta<\frac{n}{2},$ all the terms in the
series expansion of
${}_{2}F_{1}\left(\frac{n-\beta}{2},\frac{\beta-1}{2};\frac{n}{2};\frac{1}{1+u^{2}}\right)$
are positive. Hence
${}_{2}F_{1}\left(\frac{n-\beta}{2},\frac{\beta-1}{2};\frac{n}{2};\frac{1}{1+u^{2}}\right)\geq
1.$ Thus from Lemma 3.1, we have

$\displaystyle\int_{0}^{\infty}e^{-ut}t^{\frac{n}{2}-\beta}J_{\frac{n}{2}-1}(t)dt$
$\displaystyle=\dfrac{(\frac{1}{2})^{\frac{n}{2}-1}\Gamma(n-\beta)}{(1+u^{2})^{\frac{n-\beta}{2}}\Gamma(\frac{n}{2})}\,{}_{2}F_{1}\left(\frac{n-\beta}{2},\frac{\beta-1}{2};\frac{n}{2};\frac{1}{1+u^{2}}\right)$
(3.7)
$\displaystyle\geq\dfrac{(\frac{1}{2})^{\frac{n}{2}-1}\Gamma(n-\beta)}{2^{\frac{n-\beta}{2}}\Gamma(\frac{n}{2})}>0.$

Consider arbitrary $n\geq 2.$ Since for $x>0,$ $\Gamma(x)$ is continuous, the
lower bounds in (3) and (3) are independent of $\alpha,\beta.$ Hence for
$0<u<1$ and $\frac{n}{3}<\beta<\frac{n}{2},$ there exists $a_{0},C>0,$
independent of $u,\beta,$ such that for all $a\geq a_{0},$

$\displaystyle\int_{0}^{a}e^{-ut}t^{\frac{n}{2}-\beta}J_{\frac{n}{2}-1}(t)dt\geq
C.$

Therefore, $\lim_{u\rightarrow
0^{+}}\int_{0}^{a}e^{-ut}t^{\frac{n}{2}-\beta}J_{\frac{n}{2}-1}(t)dt\geq C$
for all $a\geq a_{0}.$ Thus

$\int_{0}^{a}t^{\frac{n}{2}-\beta}J_{\frac{n}{2}-1}(t)dt\geq C$

for all $a\geq a_{0}$ and $\frac{n}{3}<\beta<\frac{n}{2}.$ ∎

Now we prove the main results of the section.

######  Theorem 3.3.

Consider the square integrable and compactly supported function
$f_{\beta}(z,e^{i\varphi})=\chi_{B_{1}(0)}(z)\frac{1}{|z|^{\beta}},$ where
$\beta<1,$ on $M(2).$ Then for $1<q<2,$ there exist $\beta\in(\frac{2}{3},1)$
such that $\int_{0}^{\infty}\|\hat{f_{\beta}}(a)\|_{S_{q}}^{q}ada=\infty.$

###### Proof.

Let $e_{0}(\theta)=1,$ then $e_{0}\in L^{2}(S^{1}).$ For $a>0,$

$\displaystyle\langle\hat{f_{\beta}}(a)e_{0},e_{0}\rangle$
$\displaystyle=\frac{1}{2\pi}\int_{-\pi}^{\pi}\hat{f_{\beta}}(a)e_{0}(\theta)\overline{e_{0}(\theta)}d\theta$
$\displaystyle=\frac{1}{2\pi}\int_{-\pi}^{\pi}\int_{-\pi}^{\pi}\int_{\mathbb{C}}\chi_{B_{1}(0)}(z)\frac{1}{|z|^{\beta}}e^{i\text{Re}(ae^{i\theta}z)}e_{0}(\theta-\varphi)dzd\varphi
d\theta$
$\displaystyle=\int_{-\pi}^{\pi}\int_{0}^{1}\frac{1}{r^{\beta}}\left(\int_{-\pi}^{\pi}e^{iar\text{Re}(e^{i\theta}e^{i\psi})}d\theta\right)rdrd\psi.$

In view of (3.2), we can write

(3.8) $\displaystyle\langle\hat{f_{\beta}}(a)e_{0},e_{0}\rangle=2\pi
c_{2}\int_{0}^{1}\frac{1}{r^{\beta-1}}J_{0}(ar)dr=2\pi
c_{2}\,a^{\beta-2}\int_{0}^{a}\frac{1}{r^{\beta-1}}J_{0}(r)dr.$

Hence by Lemma 3.2, if $\beta\in(\frac{2}{3},1),$ then there exists $C>0$ and
$a_{0}>0$ such that $\langle\hat{f_{\beta}}(a)e_{0},e_{0}\rangle\geq 2\pi
c_{2}\,Ca^{\beta-2}$ for all $a\geq a_{0}.$ Thus for $1<q<2,$

$\displaystyle\int_{0}^{\infty}\|\hat{f_{\beta}}(a)\|_{S_{q}}^{q}ada$
$\displaystyle\geq\int_{0}^{\infty}\|\hat{f_{\beta}}(a)\|_{S_{\infty}}^{q}ada\geq\int_{0}^{\infty}|\langle\hat{f_{\beta}}(a)e_{0},e_{0}\rangle|^{q}ada$
$\displaystyle\geq\int_{a_{0}}^{\infty}|\langle\hat{f_{\beta}}(a)e_{0},e_{0}\rangle|^{q}ada\geq\tilde{C}\int_{a_{0}}^{\infty}a^{(\beta-2)q}ada.$

The integral $\int_{a_{0}}^{\infty}a^{(\beta-2)q+1}\,da$ is finite if and only
if $(\beta-2)q+1<-1,$ that is, $\beta<2-\frac{2}{q}.$ Since $q<2,$ we have
$2-\frac{2}{q}<1.$ If we choose
$\beta\in(\,\max\\{\frac{2}{3},2-\frac{2}{q}\\},1),$ the corresponding
function $f_{\beta}$ will be the required function. ∎

######  Theorem 3.4.

Let $n\geq 3.$ Consider the square integrable and compactly supported function
$f_{\beta}(x,k)=\chi_{B_{1}(0)}(x)\frac{1}{|x|^{\beta}},$ where
$\beta<\frac{n}{2},$ on $M(n).$ Then for $1<q<2,$ there exists
$\beta\in(\frac{n}{3},\frac{n}{2})$ such that
$\int_{0}^{\infty}\sum_{\sigma\in\hat{M}}d_{\sigma}\|\hat{f}(a,\sigma)\|_{S_{q}}^{q}a^{n-1}da=\infty.$

###### Proof.

Let $\sigma_{o}\in\hat{M},$ be the trivial representation. Consider the
function $\varphi_{0}$ on $K$ defined by $\varphi_{0}(k)=1$ for all $k\in K.$
Since $d_{\sigma_{o}}=1,$ we have $\varphi_{0}\in
L^{2}(K,\mathbb{C}^{d_{\sigma_{o}}\times d_{\sigma_{o}}}).$ Now for $a>0,$

$\displaystyle\langle\hat{f}_{\beta}(a,\sigma_{o})\varphi_{0},\varphi_{0}\rangle$
$\displaystyle=\int_{K}\int_{K}\int_{\mathbb{R}^{n}}f_{\beta}(x,k)e^{ia\left\langle
s^{-1}\cdot
e_{1},\,x\right\rangle}\varphi_{0}(sk)\overline{\varphi_{0}(s)}\,dxdkds$
$\displaystyle=\int_{K}\int_{\mathbb{R}^{n}}\chi_{B_{1}(0)}(x)\frac{1}{|x|^{\beta}}e^{ia\left\langle
s^{-1}\cdot e_{1},\,x\right\rangle}\,dxds.$

Again $S^{n-1}=\\{s^{-1}\cdot e_{1}:s\in K\\}.$ Therefore, using (3.2) we get

$\displaystyle\langle\hat{f}_{\beta}(a,\sigma_{o})\varphi_{0},\varphi_{0}\rangle$
$\displaystyle=\int_{S^{n-1}}\int_{\mathbb{R}^{n}}\chi_{B_{1}(0)}(x)\frac{1}{|x|^{\beta}}e^{ia\left\langle\omega,\,x\right\rangle}\,dxd\omega$
$\displaystyle=c_{n}\int_{\mathbb{R}^{n}}\chi_{B_{1}(0)}(x)\frac{1}{|x|^{\beta}}\,(a|x|)^{1-\frac{n}{2}}J_{\frac{n}{2}-1}(a|x|)dx$
$\displaystyle=c_{n}\int_{0}^{1}\int_{S^{n-1}}\frac{1}{r^{\beta}}\,(ar)^{1-\frac{n}{2}}J_{\frac{n}{2}-1}(ar)r^{n-1}drd\omega$
$\displaystyle=c_{n}\,a^{\beta-n}\int_{0}^{a}r^{\frac{n}{2}-\beta}J_{\frac{n}{2}-1}(r)dr.$

Hence by Lemma 3.2, if $\beta\in(\frac{n}{3},\frac{n}{2}),$ then there exists
$C>0$ and $a_{0}>0$ such that
$\langle\hat{f}_{\beta}(a,\sigma_{o})\varphi_{0},\varphi_{0}\rangle\geq
c_{n}\,Ca^{\beta-n}$ for all $a\geq a_{0}.$ Thus for $1<q<2,$

$\displaystyle\int_{0}^{\infty}\sum_{\sigma\in\hat{M}}d_{\sigma}\|\hat{f}(a,\sigma)\|_{S_{q}}^{q}a^{n-1}da$
$\displaystyle\geq\int_{0}^{\infty}\|\hat{f}(a,\sigma_{o})\|_{S_{q}}^{q}a^{n-1}da\geq\int_{0}^{\infty}\left|\langle\hat{f}_{\beta}(a,\sigma_{o})\varphi_{0},\varphi_{0}\rangle\right|^{q}a^{n-1}da$
$\displaystyle\geq\int_{a_{0}}^{\infty}\left|\langle\hat{f}_{\beta}(a,\sigma_{o})\varphi_{0},\varphi_{0}\rangle\right|^{q}a^{n-1}da\geq\tilde{C}\int_{a_{0}}^{\infty}a^{(\beta-n)q+n-1}da.$

The integral $\int_{a_{0}}^{\infty}a^{(\beta-n)q+n-1}da$ is infinite if
$(\beta-n)q+n-1\geq-1,$ that is, $\beta\geq n-\frac{n}{q}.$ Since $q<2,$ we
have $n-\frac{n}{q}<\frac{n}{2}.$ If we choose
$\beta\in(\,\max\\{\frac{n}{3},n-\frac{n}{q}\\},\frac{n}{2}),$ the
corresponding function $f_{\beta}$ will be the required function. ∎

##  4\. Heisenberg motion group

The Heisenberg group $\mathbb{H}^{n}=\mathbb{C}^{n}\times\mathbb{R}$ is a step
two nilpotent Lie group having center $\mathbb{R}$ that equipped with the
group law

$(z,t)\cdot(w,s)=\left(z+w,t+s+\frac{1}{2}\text{Im}(z\cdot\bar{w})\right).$

By the Stone-von Neumann theorem, the infinite dimensional irreducible unitary
representations of $\mathbb{H}^{n}$ can be parameterized by
$\mathbb{R}^{\ast}=\mathbb{R}\smallsetminus\\{0\\}.$ That is, each
$\lambda\in\mathbb{R}^{\ast}$ defines a Schrödinger representation
$\pi_{\lambda}$ of $\mathbb{H}^{n}$ via

$\pi_{\lambda}(z,t)\varphi(\xi)=e^{i\lambda
t}e^{i\lambda(x\cdot\xi+\frac{1}{2}x\cdot y)}\varphi(\xi+y),$

where $z=x+iy$ and $\varphi\in L^{2}(\mathbb{R}^{n}).$

Having chosen sublaplacian $\mathcal{L}$ of the Heisenberg group
$\mathbb{H}^{n}$ and its geometry, there is a larger group of isometries that
commute with $\mathcal{L}$, known as Heisenberg motion group. The Heisenberg
motion group $G$ is the semidirect product of $\mathbb{H}^{n}$ with the
unitary group $K=U(n).$ Since $K$ defines a group of automorphisms on
$\mathbb{H}^{n},$ via $k\cdot(z,t)=(kz,t),$ the group law on $G$ can be
expressed as

$(z,t,k_{1})\cdot(w,s,k_{2})=\left(z+k_{1}w,t+s-\frac{1}{2}\text{Im}(k_{1}w\cdot\bar{z}),k_{1}k_{2}\right).$

Since a right $K$-invariant function on $G$ can be thought as a function on
$\mathbb{H}^{n},$ the Haar measure on $G$ is given by $dg=dzdtdk,$ where
$dzdt$ and $dk$ are the normalized Haar measure on $\mathbb{H}^{n}$ and $K$
respectively.

For $k\in K$ define another set of representations of the Heisenberg group
$\mathbb{H}^{n}$ by $\pi_{\lambda,k}(z,t)=\pi_{\lambda}(kz,t).$ Since
$\pi_{\lambda,k}$ agrees with $\pi_{\lambda}$ on the center of
$\mathbb{H}^{n},$ it follows by the Stone-Von Neumann theorem for the
Schrödinger representation that $\pi_{\lambda,k}$ is equivalent to
$\pi_{\lambda}.$ Hence there exists an intertwining operator
$\mu_{\lambda}(k)$ satisfying

$\pi_{\lambda}(kz,t)=\mu_{\lambda}(k)\pi_{\lambda}(z,t)\mu_{\lambda}(k)^{\ast}.$

Then $\mu_{\lambda}$ can be thought of as a unitary representation of $K$ on
$L^{2}(\mathbb{R}^{n}),$ called metaplectic representation. Let
$(\sigma,\mathcal{H}_{\sigma})$ be an irreducible unitary representation of
$K$ and $\mathcal{H}_{\sigma}=\text{span}\\{e_{j}^{\sigma}:1\leq j\leq
d_{\sigma}\\}.$ For $k\in K,$ the matrix coefficients of the representation
$\sigma\in\hat{K}$ are given by

$\varphi_{ij}^{\sigma}(k)=\langle\sigma(k)e_{j}^{\sigma},e_{i}^{\sigma}\rangle,$

where $i,j=1,\ldots,d_{\sigma}.$

Let
$\phi_{\alpha}^{\lambda}(x)=|\lambda|^{\frac{n}{4}}\phi_{\alpha}(\sqrt{|\lambda|}x);~{}\alpha\in\mathbb{Z}_{+}^{n},$
where $\phi_{\alpha}$’s are the Hermite functions on $\mathbb{R}^{n}.$ Since
for each $\lambda\in\mathbb{R}^{\ast},$ the set
$\\{\phi_{\alpha}^{\lambda}:\alpha\in\mathbb{Z}_{+}^{n}\\}$ forms an
orthonormal basis for $L^{2}(\mathbb{R}^{n}),$ letting
$P_{m}^{\lambda}=\text{span}\\{\phi_{\alpha}^{\lambda}:~{}|\alpha|=m\\},$
$\mu_{\lambda}$ becomes an irreducible unitary representation of $K$ on
$P_{m}^{\lambda}.$ Hence, the action of $\mu_{\lambda}$ can be realized on
$P_{m}^{\lambda}$ by

(4.1)
$\mu_{\lambda}(k)\phi_{\gamma}^{\lambda}=\sum_{|\alpha|=|\gamma|}\eta_{\gamma\alpha}^{\lambda}(k)\phi_{\alpha}^{\lambda},$

where $\eta_{\alpha\gamma}^{\lambda}$’s are the matrix coefficients of
$\mu_{\lambda}(k).$ Define a bilinear form $\phi_{\alpha}^{\lambda}\otimes
e_{j}^{\sigma}$ on $L^{2}(\mathbb{R}^{n})\times\mathcal{H}_{\sigma}$ by
$\phi_{\alpha}^{\lambda}\otimes
e_{j}^{\sigma}=\phi_{\alpha}^{\lambda}e_{j}^{\sigma}.$ Then
$\\{\phi_{\alpha}^{\lambda}\otimes
e_{j}^{\sigma}:\alpha\in\mathbb{Z}_{+}^{n},1\leq j\leq d_{\sigma}\\}$ forms an
orthonormal basis for $L^{2}(\mathbb{R}^{n})\otimes\mathcal{H}_{\sigma}.$
Denote
$\mathcal{H}_{\sigma}^{2}=L^{2}(\mathbb{R}^{n})\otimes\mathcal{H}_{\sigma}.$

Define a representation $\rho_{\sigma}^{\lambda}$ of $G$ on the space
$\mathcal{H}_{\sigma}^{2}$ by

$\rho_{\sigma}^{\lambda}(z,t,k)=\pi_{\lambda}(z,t)\mu_{\lambda}(k)\otimes\sigma(k).$

In the article [11], it is shown that $\rho_{\sigma}^{\lambda}$ are all
possible irreducible unitary representations of $G$ that participate in the
Plancherel formula. Thus, in view of the above discussion, we shall denote the
partial dual of the group $G$ by
$G^{\prime}\cong\mathbb{R}^{\ast}\times\hat{K}.$ The Fourier transform of
$f\in L^{1}(G)$ defined by

$\hat{f}(\lambda,\sigma)=\int_{K}\int_{\mathbb{R}}\int_{\mathbb{C}^{n}}f(z,t,k)\rho_{\sigma}^{\lambda}(z,t,k)dzdtdk,$

is a bounded linear operator on $\mathcal{H}_{\sigma}^{2}.$ As the Plancherel
formula

$\int_{K}\int_{\mathbb{H}^{n}}|f(z,t,k)|^{2}dzdtdk=(2\pi)^{-n}\int_{\mathbb{R}\setminus\\{0\\}}\sum_{\sigma\in\hat{K}}d_{\sigma}\|\hat{f}(\lambda,\sigma)\|^{2}_{S_{2}}|\lambda|^{n}d\lambda$

holds for $f\in L^{2}(G),$ it follows that $\hat{f}(\lambda,\sigma)$ is a
Hilbert-Schmidt operator on $\mathcal{H}_{\sigma}^{2}.$ For detailed Fourier
analysis on the Heisenberg motion group, see [2, 5, 11].

First, we recall the example of the previously mentioned required function for
the Heisenberg group and show that the example can be extended to the
Heisenberg motion group.

Let $A=\\{(x,y,t):|x_{l}|\leq 1,|y_{l}|\leq 1,|t|\leq 1;l=1,\ldots,n\\}$ be a
compact subset of $\mathbb{R}^{2n}\times\mathbb{R}$ and

(4.2) $\displaystyle
g_{\xi}(z,t)=g_{\xi}(x,y,t)=|t|^{\xi}\prod\limits_{j=1}^{n}|x_{j}|^{\xi}\prod\limits_{j=1}^{n}|y_{j}|^{\xi}\chi_{A}(z,t),$

where $\xi>-\frac{1}{2}.$ Then the following result holds.

######  Theorem 4.1.

[8] For $1<q<2,$ there exists $\xi>-\frac{1}{2},$ such that

$\int_{\mathbb{R}\setminus\\{0\\}}|\langle\hat{g_{\xi}}(\lambda)\phi_{0}^{\lambda},\phi_{0}^{\lambda}\rangle|^{q}|\lambda|^{n}d\lambda$

is infinite.

The following proposition gives the required function for the Heisenberg
motion group.

######  Theorem 4.2.

Consider the function $f_{\xi}(z,t,k)=g_{\xi}(z,t)$ on $G,$ where $g_{\xi}$ is
defined in (4.2). Then $f_{\xi}$ is square integrable and compactly supported.
Further for $1<q<2,$ there exists $\xi>-\frac{1}{2},$ such that
$\int_{\mathbb{R}\setminus\\{0\\}}\sum_{\sigma\in\hat{K}}d_{\sigma}\|\hat{f}_{\xi}(\lambda,\sigma)\|^{q}_{S_{q}}|\lambda|^{n}d\lambda$
is infinite

###### Proof.

Let $1<q<2$ and $\lambda\in\mathbb{R}\setminus\\{0\\}.$ Then

$\displaystyle\sum_{\sigma\in\hat{K}}d_{\sigma}\|\hat{f_{\xi}}(\lambda,\sigma)\|^{q}_{S_{q}}$
$\displaystyle\geq\sum_{\sigma\in\hat{K}}\left(d_{\sigma}^{1/q}\|\hat{f_{\xi}}(\lambda,\sigma)\|_{S_{2}}\right)^{q}\geq\left(\sum_{\sigma\in\hat{K}}\left(d_{\sigma}^{1/q}\|\hat{f_{\xi}}(\lambda,\sigma)\|_{S_{2}}\right)^{2}\right)^{q/2}$
(4.3)
$\displaystyle\geq\left(\sum_{\sigma\in\hat{K}}d_{\sigma}^{2/q}\|\hat{f_{\xi}}(\lambda,\sigma)\|_{S_{2}}^{2}\right)^{q/2}\geq\left(\sum_{\sigma\in\hat{K}}d_{\sigma}\|\hat{f_{\xi}}(\lambda,\sigma)\|_{S_{2}}^{2}\right)^{q/2}.$

It is already discussed that $\\{\phi_{\alpha}^{\lambda}\otimes
e_{j}^{\sigma}:\alpha\in\mathbb{Z}_{+}^{n},1\leq j\leq d_{\sigma}\\}$ forms an
orthonormal basis for $L^{2}(\mathbb{R}^{n})\otimes\mathcal{H}_{\sigma}.$ Thus
by (4.1),

$\langle\hat{f_{\xi}}(\lambda,\sigma)(\phi_{\alpha}^{\lambda}\otimes
e_{j}^{\sigma}),(\phi_{\beta}^{\lambda}\otimes
e_{l}^{\sigma})\rangle=\int_{\mathbb{H}^{n}}\int_{K}f_{\xi}(z,t,k)\sum\limits_{|\gamma|=|\alpha|}\eta_{\alpha\gamma}(k)\Phi_{\gamma\beta}^{\lambda}(z,t)\phi_{lj}^{\sigma}(k)dzdtdk,$

where
$\Phi_{\gamma\beta}^{\lambda}(z,t)=\langle\pi_{\lambda}(z,t)\phi_{\gamma}^{\lambda},\phi_{\beta}^{\lambda}\rangle.$
Therefore, $\|\hat{f_{\xi}}(\lambda,\sigma)(\phi_{\alpha}^{\lambda}\otimes
e_{j}^{\sigma})\|_{2}^{2}$ is equal to

$\displaystyle\sum_{\beta\in\mathbb{Z}_{+}^{n}}\sum_{1\leq l\leq
d_{\sigma}}\left|\int_{\mathbb{H}^{n}}\int_{K}f_{\xi}(z,t,k)\sum\limits_{|\gamma|=|\alpha|}\eta_{\alpha\gamma}(k)\Phi_{\gamma\beta}^{\lambda}(z,t)\phi_{lj}^{\sigma}(k)dzdtdk\right|^{2}$
$\displaystyle=$ $\displaystyle\sum_{\beta\in\mathbb{Z}_{+}^{n}}\sum_{1\leq
l\leq
d_{\sigma}}\left|\sum\limits_{|\gamma|=|\alpha|}\langle\hat{g_{\xi}}(\lambda)\phi_{\gamma}^{\lambda},\phi_{\beta}^{\lambda}\rangle\int_{K}\eta_{\alpha\gamma}(k)\phi_{lj}^{\sigma}(k)dk\right|^{2}.$

That is,

$\displaystyle\|\hat{f_{\xi}}(\lambda,\sigma)\|_{S_{2}}^{2}=\sum_{\alpha,\beta\in\mathbb{Z}_{+}^{n}}\sum_{1\leq
j,l\leq
d_{\sigma}}\left|\sum\limits_{|\gamma|=|\alpha|}\langle\hat{g_{\xi}}(\lambda)\phi_{\gamma}^{\lambda},\phi_{\beta}^{\lambda}\rangle\int_{K}\eta_{\alpha\gamma}(k)\phi_{lj}^{\sigma}(k)dk\right|^{2}.$

Hence by the Peter-Weyl theorem (Plancherel) for compact groups, we get

$\displaystyle\sum_{\sigma\in\hat{K}}d_{\sigma}\|\hat{f_{\xi}}(\lambda,\sigma)\|_{S_{2}}^{2}$
$\displaystyle=\sum_{\alpha,\beta\in\mathbb{Z}_{+}^{n}}\int_{K}\left|\sum\limits_{|\gamma|=|\alpha|}\langle\hat{g_{\xi}}(\lambda)\phi_{\gamma}^{\lambda},\phi_{\beta}^{\lambda}\rangle\eta_{\alpha\gamma}(k)\right|^{2}dk$
$\displaystyle\geq\sum_{\alpha,\beta\in\mathbb{Z}_{+}^{n}\setminus\mathbb{N}^{n}}\int_{K}\left|\sum\limits_{|\gamma|=|\alpha|}\langle\hat{g_{\xi}}(\lambda)\phi_{\gamma}^{\lambda},\phi_{\beta}^{\lambda}\rangle\eta_{\alpha\gamma}(k)\right|^{2}dk$
$\displaystyle=\int_{K}\left|\langle\hat{g_{\xi}}(\lambda)\phi_{0}^{\lambda},\phi_{0}^{\lambda}\rangle\eta_{00}(k)\right|^{2}dk=|\langle\hat{g_{\xi}}(\lambda)\phi_{0}^{\lambda},\phi_{0}^{\lambda}\rangle|^{2}.$

Since $\mu_{\lambda}|_{P_{0}^{\lambda}}$ is irreducible, the last equality
follows from Schur’s orthogonality relation. Therefore, from (4), we can
conclude that

$\displaystyle\int_{\mathbb{R}\setminus\\{0\\}}\sum_{\sigma\in\hat{K}}d_{\sigma}\|\hat{f_{\xi}}(\lambda,\sigma)\|^{q}_{S_{q}}|\lambda|^{n}d\lambda$
$\displaystyle\geq\int_{\mathbb{R}\setminus\\{0\\}}\left(\sum_{\sigma\in\hat{K}}d_{\sigma}\|\hat{f_{\xi}}(\lambda,\sigma)\|_{S_{2}}^{2}\right)^{q/2}|\lambda|^{n}d\lambda$
$\displaystyle\geq\int_{\mathbb{R}\setminus\\{0\\}}|\langle\hat{g_{\xi}}(\lambda)\phi_{0}^{\lambda},\phi_{0}^{\lambda}\rangle|^{q}|\lambda|^{n}d\lambda.$

Thus Theorem 4.1 proves the result. ∎

######  Remark 4.3.

As compare to motion groups, the result in product space is straightforward.
Let $G$ be a second countable type $I$ locally compact unimodular group and
$(\pi,\mathcal{H}_{\pi})$ be its representation. Consider
$G_{P}=\mathbb{R}^{n}\times G.$ For $f\in L^{1}(G_{P}),$ Fourier transform
defined by $\hat{f}(x,\pi)=\int_{\mathbb{R}^{n}}\int_{G}f(y,u)e^{-2\pi ix\cdot
y}\pi(u)d\nu(u)dy$ is a bounded operator on $\mathcal{H}_{\pi}.$ If we take
$f(x,u)=f_{1}(x)f_{2}(u),$ then

(4.4)
$\displaystyle\int_{\mathbb{R}^{n}}\int_{\hat{G}}\|\hat{f}(x,\pi)\|_{S_{q}}^{q}d\mu(\pi)dy=\int_{\mathbb{R}^{n}}|\hat{f}_{1}(y)|^{q}dy\int_{\hat{G}}\|\hat{f}_{2}(\pi)\|_{S_{q}}^{q}d\mu(\pi).$

It is known that, for $q\in(1,2),$ there exists a positive, square integrable
and compactly supported function $f_{1}$ on $\mathbb{R}^{n}$ such that
$\int_{\mathbb{R}^{n}}|\hat{f}_{1}(y)|^{q}dy$ is infinite (see [12]). If we
choose a square integrable and compactly supported function $f_{2}$ on $G$
such that $\int_{\hat{G}}\|\hat{f}_{2}(\pi)\|_{S_{q}}^{q}d\mu(\pi)\neq 0,$
then by (4.4),
$\int_{\mathbb{R}^{n}}\int_{\hat{G}}\|\hat{f}(x,\pi)\|_{S_{q}}^{q}d\mu(\pi)dy$
is infinite. Hence in view of Proposition 2.14, the Weyl transform $W_{\zeta}$
on $G_{P}$ is not bounded for $\zeta\in L^{p}(G_{P}\times\widehat{G_{P}})$
with $2<p<\infty$ .

Acknowledgements: The first author gratefully acknowledges the support
provided by IIT Guwahati, Government of India.

## References

  * [1] G. E. Andrews, R. Askey and R. Roy, Special functions, Encyclopedia of Mathematics and its Applications, Cambridge University Press, Cambridge, (71) 1999.
  * [2] C. Benson, J. Jenkins and G. Ratcliff, Bounded K-spherical functions on Heisenberg groups, J. Funct. Anal. 105 (1992), no. 2, 409-443.
  * [3] L. Chen and J. Zhao, Weyl transform and generalized spectrogram associated with quaternion Heisenberg group, Bull. Sci. Math. 136 (2012), no. 2, 127-143.
  * [4] M. Duflo and C. C. Moore, On the regular representation of a nonunimodular locally compact group, J. Funct. Anal. 21 (1976), no. 2, 209-243.
  * [5] S. Ghosh and R. K. Srivastava, Benedicks-Amrein-Berthier theorem for the Heisenberg motion group and quaternion Heisenberg group, arXiv:1904.04023.
  * [6] K. Kumahara and K. Okamoto, An analogue of the Paley-Wiener theorem for the Euclidean motion group, Osaka Math. J. 10 (1973), 77-91.
  * [7] W. Nasserddine, $L^{p}$-Fourier inversion formula on certain locally compact groups, C. R. Math. Acad. Sci. Paris 357 (2019), no. 7, 583-588.
  * [8] L. Peng and J. Zhao, Weyl transforms associated with the Heisenberg group, Bull. Sci. Math. 132 (2008), no. 1, 78-86.
  * [9] L. Peng and J. Zhao, Weyl transforms on the upper half plane, Rev. Mat. Complut. 23 (2010), no. 1, 77-95.
  * [10] R. P. Sarkar and S. Thangavelu, On theorems of Beurling and Hardy for the Euclidean motion group, Tohoku Math. J. (2) 57 (2005), no. 3, 335-351.
  * [11] S. Sen, Segal-Bargmann transform and Paley-Wiener theorems on Heisenberg motion groups, Adv. Pure Appl. Math. 7 (2016), no. 1, 13-28.
  * [12] B. Simon, The Weyl transforms and $L^{p}$ functions on phase space, Proc. Amer. Math. Soc. 116 (1992), 1045-1047.
  * [13] M. Sugiura, Unitary representations and harmonic analysis, North-Holland Mathematical Library, North-Holland Publishing Co., Amsterdam; Kodansha, Ltd., Tokyo, (44) 1990.
  * [14] G. N. Watson, A treatise on the theory of Bessel functions, second edition, Cambridge University Press, Cambridge, 1944.
  * [15] H. Weyl, The Theory of Groups and Quantum Mechanics, Dover, 1950.
  * [16] M. W. Wong, Weyl Transform, Springer-Verlag, New York, 1998.