# Two wavelet multipliers and Landau-Pollak-Slepian operators on locally
compact abelian groups associated to right-$H$-translation-invariant functions

Aparajita Dasgupta  Aparajita Dasgupta Department of Mathematics Indian
Institute of Technology Delhi, Hauz Khas New Delhi - 110 016, India.
<EMAIL_ADDRESS>,  Swaraj Paul  Swaraj Paul Department of
Mathematics Indian Institute of Technology, Delhi, Hauz Khas New Delhi-110016,
India<EMAIL_ADDRESS>and  Santosh Kumar Nayak  Santosh Kumar Nayak:
Department of Mathematics Indian Institute of Technology, Delhi, Hauz Khas New
Delhi-110016 India<EMAIL_ADDRESS>
###### Abstract.

By using a coset of closed subgroup, we define a Fourier like transform for
locally compact abelian (LCA) topological groups. We studied two wavelet
multipliers and Landau-Pollak-Slepian operators on locally compact abelian
topological groups associated to the transform and show that the transforms
are $L^{p}$bounded linear operators, and are in Schatten p-class for $1\leq
p\leq\infty$. Finally, we determine their trace class and also obtain a
connection with the generalized Landau-Pollak-Slepian operators.

###### Key words and phrases:

Short time Fourier transform, Directional short time Fourier transform,
Generalized multiplier, Generalized two wavelet multipliers.

###### 1991 Mathematics Subject Classification:

Primary 47G10, 47G30, Secondary 42C40

The research of A. Dasgupta and S. Paul was supported by Core Research
Grant(RP03890G), Science and Engineering Research Board (SERB), DST, India. S.
Paul acknowledge Kinara Roy for her support during the preparation of the
work.

###### Contents

  1. 1 Introduction
  2. 2 Fourier like transform on locally compact abelian topological groups associated to the coset of closed subgroup
  3. 3 $L^{2}-$boundedness of generalized two wavelet multiplier
  4. 4 Schatten class boundednes of generalized two wavelet multiplier
  5. 5 $L^{p}-$boundedness of generalized two wavelet multiplier for $1\leq p\leq\infty$
  6. 6 Compactness of generalized two wavelet multipliers
  7. 7 Landau-Pollak-Slepian operator

##  1\. Introduction

The short time Fourier transform (STFT) has many applications specially in
signal and image processing for providing good time-frequency localization.
Singularities of discontinuity across a curve such an edges in an image in
multidimensional often hold the key information. To tackle the directional
singularity Candes [4] first introduced ridgelet transform, which is the
wavelet transform in Radon domain. Ridgelets are constant along the ridge or
hyperplane. Now a days Curvelets and shearlets play important role to
represent directional selectivity as they give optimal sparse approximations
for a class of bivariate functions exhibiting anisotropic features [5, 6, 8,
9]. The relation between wavelet transform and the above directional
representation is the above transform projects a hyperplane singularity into a
point singularity, then takes one-dimensional wavelet transform. We recall
that the short time Fourier transform (STFT) [10] of a function $f\in
L^{2}({\mathbb{R}}^{n})$ w.r.t. a window $g\in L^{2}({\mathbb{R}}^{n})$, is
the function
$V_{g}f(x,\omega)=\int_{{\mathbb{R}}^{n}}f(t)\overline{g(t-x)}e^{-2\pi i\omega
t}dt$, i.e.,

(1) $V_{g}f(x,\omega)=\widehat{f.T_{x}\bar{g}}(\omega).$

Since the STFT enjoy the orthogonality relation, it gives a full
reconstruction of a function/distribution $f$ for $f,g,h\in
L^{2}({\mathbb{R}}^{n})$ and $\widehat{f},\widehat{G}\in
L^{1}({\mathbb{R}}^{n})$ as

(2) $f(u)=\frac{1}{\langle
h,g\rangle}\int_{{\mathbb{R}}^{n}}\int_{{\mathbb{R}}^{n}}V_{g}f(x,\omega)M_{\omega}T_{x}h(u)d\omega
dx,\ \ \textrm{for\ all}\ u\in{\mathbb{R}}^{n},$

where $M_{\omega}$ and $T_{x}$ are the modulation and translation operator
respectively. But the STFT does not give the directional information about the
function or distribution. For directional sensitivity of time-frequency
decomposition Grafakos and Sansing [11] introduced a variant of STFT of
function $f\in L^{2}({\mathbb{R}})$ using Gabor ridge function $e^{2\pi
im(s-t)}.g(s-t)$ on $S^{n-1}\times{\mathbb{R}}\times{\mathbb{R}}$ as

(3)
$(\xi,x,\omega)\rightarrow\int_{{\mathbb{R}}^{n}}f(t)\overline{g(\xi.t-x)}e^{-2\pi
i\omega(\xi.t)}dt.$

But this transform does not provide a full reconstruction of a function $f$
and for this the authors in [11] use the derivative of Gabor ridge function,
i.e. “weighted Gabor ridge functions” for the analysis and synthesis of the
functions $f$. Moreover, this transform lost the Fourier transform
representation (1) of STFT, since $x\in{\mathbb{R}}^{n}$ and
$\omega\in{\mathbb{R}}$. Modifying the idea, Giv [3] introduced directional
short-time Fourier transform and studied some useful properties like
orthogonality, full reconstruction formula. Recently Mejjaoli and Omri [1]
introduced a generalized two-wavelet multiplier using directional STFT and
studied the $L^{p}$ boundedness and compactness of the two-wavelet multiplier.
Our aim in this paper is to give a generalization of two-wavelet multiplier on
locally compact abelian topological groups $G$ associated to right
$H$-translation invariant functions. For this as a generalization of
directional STFT, we will define a transform $\mathcal{D}_{H}^{g}f(\omega,zH)$
of $f\in L^{2}(G)$ using a character $\omega\in\widehat{G}$ and a window $g\in
L^{2}(G/H)$, where $H$ is a closed subgroup of $G$. In Section 2, we first
define the STFT, $\mathcal{D}_{H}^{g}f$, $f\in L^{1}(G)$ and$g\in
L^{\infty}(G/H)$ on $G/H$ which we have shown satisfies certain orthogonality
relations. In this section we have also defined the generalized multiplier and
generalized two-wavelet multiplier. In Section 3, 4 and 5 we have shown that
the generalized multiplier is bounded, Schatten class and $L^{p}-$ bounded,
$1\leq p\leq\infty$ for all symbol $\sigma$ in $L^{p}(\widehat{G}\times G/H),$
$1\leq p\leq\infty$. Finally, in Section 6 we have shown that the generalized
wavelet mulpliers are ompact operators for all symbol $\sigma$ in
$L^{1}(\widehat{G}\times G/H)$. In Section 7 we have defined the generalized
Landau-Pollak-Slepian operator and shown that its is actually unitarily
equivalent to scaler multiples of the generalized two-wavelet multiplier.

##  2\. Fourier like transform on locally compact abelian topological groups
associated to the coset of closed subgroup

$G$ denotes a locally compact abelian topological group with the Haar measure
$dm_{G}$ and $\widehat{G}$ is the dual group of $G$ with the Haar measure
$dm_{\widehat{G}}$ such that $dm_{\widehat{G}}$ is the dual measure of
$dm_{G}$. Let $H$ be a closed subgroup of an LCA group $G$ with the Haar
measure $dm_{H}$. The annihilator of $H$ is the set
$H^{\perp}\subset\widehat{G}$ given by
$H^{\perp}=\\{\eta\in\widehat{G}:\langle y,\eta\rangle=1\ \textrm{for\ all}\
y\in H\\}$. Moreover, $H^{\perp}$ is a closed subgroup of $\widehat{G}$, is
topologically isomorphic with the character group of $G/H$ and we have the
followings:

$(H^{\perp})^{\perp}=H\ \ \ \ \textrm{and}\ \ \ \
\widehat{H}=\widehat{G}/H^{\perp}.$

Let $f$ be any function in $L^{1}(G)$ then according to Theorem 28.54 in [2],
the function $x\rightarrow\int_{H}f(xy)dm_{H}(y)$ depends only on the [left]
coset of $H$ containing $x$, is a function on the quotient group $G/H$.
Moreover, the function $R_{H}$ is Haar measurable on $G/H$ and belongs to
$L^{1}(G/H)$, where $R_{H}$ is the function
$xH\rightarrow\int_{H}f(xy)dm_{H}(y)$. We normalize the Haar measure so that

(4)
$\int_{G/H}R_{H}(xH)dm_{G/H}(xH)=\int_{G/H}\int_{H}f(xy)dm_{H}(y)dm_{G/H}(xH)=\int_{G}f(x)dm_{G}(x).$

For $R_{H}\in L^{1}(G/H)$, the Fourier transform of $R_{H}$ is defined by

$\widehat{R_{H}}(\chi^{+})=\int_{G/H}R_{H}(xH)\overline{\chi^{+}(xH)}dm_{G/H}(xH).$

where the character $\chi^{+}(\in H^{\perp})$ of the group $G/H$ is defined by
$\chi^{+}(xH)=\chi(x)$ for $\chi\in\widehat{G}$. The relation between the
Fourier transform of the function on group $G/H$ and function on $G$ is the
following:

$\displaystyle\widehat{R_{H}}(\chi^{+})$
$\displaystyle=\int_{G/H}\int_{H}f(xy)dm_{H}(y)\overline{\chi^{+}(xH)}dm_{G/H}(xH)$
$\displaystyle=\int_{G/H}\int_{H}f(xy)\overline{\chi(xy)}dm_{H}(y)dm_{G/H}(xH)$
(5) $\displaystyle=\widehat{f}(\chi)$

Before describing the operator first we recall the short time Fourier
transform (STFT) for locally compact abelian group. Given an appropriate
window $g\in L^{2}(G)$, the STFT of $f\in L^{2}(G)$ w.r.t. $g$ at a point
$(x,\omega)\in G\times\widehat{G}$ is denoted as

$\mathcal{V}_{g}f(x,\omega)=\int_{G}f(y)\overline{g}(x^{-1}y)\overline{\omega(y)}dm_{G}(y)=\langle
f,M_{\omega}T_{x}g\rangle=(f.T_{x}\overline{g})^{\wedge}(\omega).$

The STFT enjoy the following properties for $f,g\in L^{2}(G)$,

(6)
$\displaystyle\int_{G}\int_{\widehat{G}}|\mathcal{V}_{g}f(x,\omega)|^{2}dxd\omega=\|g\|_{2}^{2}\|f\|_{2}^{2}.$

For $f,g\in L^{2}(G)$ and $\widehat{f},\widehat{G}\in L^{1}(\widehat{G})$, we
have the reconstruction formula for $f$ as

(7) $\displaystyle f(u)=\frac{1}{\langle
h,g\rangle}\int_{G}\int_{\widehat{G}}\mathcal{V}_{g}f(x,\omega)M_{\omega}T_{x}h(u)dm_{\widehat{G}}(\omega)dm_{G}(x)\
\ \textrm{for\ all}\ u\in G.$

Now for the closed subspace $H\subset G$, window $g\in L^{\infty}(G/H)$ and
$zH\in G/H$, we define the STFT like transform of $f\in L^{1}(G)$ using
right-$H$-translation-invariant functions on $G$ as a function on
$G/H\times\widehat{G}$ as

(8)
$\mathcal{D}_{H}^{g}f(\omega,zH)=\int_{G}f(x)\overline{w(x)g(z^{-1}xH)}dm_{G}(x)=\int_{G}M_{-w}f(x)g(z^{-1}xH)dm_{G}(x).$

Next we show how the operator $\mathcal{D}_{H}^{g}f(\omega,zH)$ can be written
with the inner product on $G/H$ in a natural way.

######  Theorem 2.1.

If $g\in L^{\infty}(G/H)$ and $f\in L^{1}(G)$, then for every
$(\omega,zH)\in\widehat{G}\times G/H$

(9)
$\mathcal{D}_{H}^{g}f(\omega,zH)=({f(x)\overline{g(z^{-1}xH)}})^{\wedge}(\omega)=\langle
R_{H}(M_{-\omega}f),T_{zH}g\rangle_{G/H}=\left(R_{H}(M_{-\omega}f)*g\right)(zH).$

###### Proof.

The result can be easily followed in view of Weils formula (4) and the
following:

$\displaystyle\mathcal{D}_{H}^{g}f(\omega,zH)$ $\displaystyle=$
$\displaystyle\int_{G}f(x)\overline{w(x)g(z^{-1}xH)}dx$ $\displaystyle=$
$\displaystyle\int_{G/H}\int_{H}f(xy)\overline{w(xy)g(z^{-1}xyH)}dm_{H}(y)dm_{G/H}(xH)$
$\displaystyle=$
$\displaystyle\int_{G/H}R_{H}(M_{-\omega}f)(xH)\overline{g(z^{-1}xH)}dm_{G/H}(xH)$
$\displaystyle=$ $\displaystyle\left(R_{H}(M_{-\omega}f)*g\right)(zH),$

where for the functions $f_{1},f_{2}\in L^{1}(G/H)$ the convolution in the
quotient space is defined by

$(f_{1}*f_{2})(xH)=\int_{G/H}f_{1}(yH)f_{2}(y^{-1}xH)dm_{G/H}(yH).$

∎

The transform $\mathcal{D}_{H}^{g}f(\omega,zH)$ can be regarded as a STFT in
the quotient group $G/H$ through the following theorem:

######  Theorem 2.2.

If $f\in L^{1}(G)$, $\widehat{f}\in L^{1}(\widehat{G})$ and $g\in
L^{1}(G/H)\cap L^{\infty}(G/H)$, then the transform
$\mathcal{D}_{H}^{g}f(\omega,zH)$ is the STFT of the function
$T_{-\omega}\widehat{f}$ with respect to the window $\widehat{G}$, evaluated
at $(0,-zH)$.

###### Proof.

From the previous theorem we have $\mathcal{D}_{H}^{g}f(\omega,zH)=\langle
R_{H}(M_{-\omega}f),T_{zH}g\rangle_{G/H}$. Since $f\in L^{1}(G)$ and
$\widehat{f}\in L^{1}(\widehat{G})$ then $R_{H}(M_{-\omega}f)\in L^{1}(G/H)$
and $\widehat{R_{H}(M_{-\omega}f)}\in L^{1}(G/H)$. So $R_{H}(M_{-\omega}f)\in
L^{2}(G/H)$. Also $g\in L^{2}(G/H)$. Hence using Plancherel’s theorem and (2)
we can write $\mathcal{D}_{H}^{g}f(\omega,zH)$ the followings for $\eta\in
H^{\perp}$:

$\displaystyle\mathcal{D}_{H}^{g}f(\omega,zH)$
$\displaystyle=\langle\widehat{R_{H}(M_{-\omega}f)}(\eta),\widehat{T_{zH}g}(\eta)\rangle_{G/H}$
$\displaystyle=\langle\widehat{M_{-\omega}f}(\eta),M_{-zH}\widehat{g}(\eta)\rangle_{H^{\perp}}$
$\displaystyle=\langle
T_{-\omega}\widehat{f}(\eta),M_{-zH}\widehat{g}(\eta)\rangle_{H^{\perp}}.$

Hence proved. ∎

The transform $\mathcal{D}_{H}^{g}f(\omega,zH)$ satisfy the following
orthogonality relations:

######  Theorem 2.3.

  * (i)

For every directional window $g\in L^{\infty}(G/H)$, the operator
$\mathcal{D}_{g}$ is bounded from $L^{1}(G)$ into
$L^{\infty}(\widehat{G}\times G/H)$ and the operator norm satisfies

(10) $\|D_{H}^{g}\|\leq\|g\|_{L^{\infty}(G/H)}.$

  * (ii)

Suppose $g_{1},g_{2}\in L^{\infty}(G/H)$ and $f_{1},f_{2}\in L^{1}(G)\cap
L^{2}(G)$. If at least one of the $g_{i}$’s is in $L^{1}(G/H)$, then
$\mathcal{D}_{H}^{g}f(\omega,zH)$ satisfies

(11) $\int_{\widehat{G}\times
G/H}\mathcal{D}_{H}^{g_{1}}f_{1}(\omega,zH)\overline{\mathcal{D}_{H}^{g_{2}}f_{2}(\omega,zH)}dm_{\widehat{G}}(w)dm_{G/H}(zH)=\langle
f_{1},f_{2}\rangle_{L^{2}(G)}\langle g_{2},g_{1}\rangle_{L^{2}(G/H)}.$

Moreover, if $g\in L^{1}(G/H)\cap L^{\infty}(G/H)$ and $f\in L^{1}(G)\cap
L^{2}(G)$, then $\mathcal{D}_{H}^{g}f(\omega,zH)\in
L^{2}(G/H\times\widehat{G})$ and

(12)
$\|\mathcal{D}_{H}^{g}f\|_{L^{2}(G/H\times\widehat{G})}=\|g\|_{L^{2}(G/H)}\|f\|_{L^{2}(G)}.$

###### Proof.

The proof of first part follows from Equation (8).
Since $g_{i}\in L^{\infty}(G/H)$ and $f_{i}\in L^{1}(G)\cap L^{2}(G)$,
$f_{i}(x)\overline{w(x)g_{i}(z^{-1}xH)}\in L^{1}(G)\cap L^{2}(G)$ for $i=1,2$.
Using Plancherel’s theorem we can write the followings:

$\displaystyle\int_{\widehat{G}\times
G/H}\mathcal{D}_{H}^{g_{1}}f_{1}(\omega,zH)\mathcal{D}_{H}^{g_{2}}f_{2}(\omega,zH)dm_{\widehat{G}}(w)dm_{G/H}(zH)$
$\displaystyle=\int_{\widehat{G}\times
G/H}({f_{1}(x)\overline{g_{1}(z^{-1}xH)}})^{\wedge}(\omega)\overline{({f_{2}(x)\overline{g_{2}(z^{-1}xH)}})^{\wedge}(\omega)}dm_{\widehat{G}}(w)dm_{G/H}(zH)$
$\displaystyle=\int_{{G}\times
G/H}f_{1}(x)\overline{g_{1}(z^{-1}xH)}\overline{f_{2}(x)}g_{2}(z^{-1}xH)dm_{G}(w)dm_{G/H}(zH)$
$\displaystyle=\int_{G}f_{1}(x)\overline{f_{2}(x)}\left(\int_{G/H}\overline{g_{1}(z^{-1}xH)}g_{2}(z^{-1}xH)(w)dm_{G/H}(zH)\right)dm_{G}(x).$

Hence the proof of (11) follows by noting

$\langle
g_{2},g_{1}\rangle_{G/H}=\int_{G/H}\overline{g_{1}(z^{-1}xH)}g_{2}(z^{-1}xH)(w)dm_{G/H}(zH),$

since $G/H$ is unimodular group. The equation (12) follows as a consequence of
(11). ∎

######  Corollary 2.4.

Suppose $g_{1},g_{2}\in L^{\infty}(G/H)$ and $f\in L^{1}(G)\cap L^{2}(G)$. If
at least one of the $g_{i}$’s is in $L^{1}(G/H)$ and $\langle
g_{2},g_{1}\rangle\neq 0$, then

$f(x)=\frac{1}{\langle g_{2},g_{1}\rangle}\int_{\widehat{G}\times
G/H}\mathcal{D}_{H}^{g_{1}}f(\omega,zH)\omega(x)g_{2}(z^{-1}xH)dm_{\widehat{G}}(w)dm_{G/H}(zH).$

Moreover, for non-zero function $g\in L^{1}(G/H)\cap L^{\infty}(G/H)$

$f(x)=\frac{1}{\|g\|_{L^{2}(G/H)}^{2}}\int_{\widehat{G}\times
G/H}\mathcal{D}_{H}^{g}f(\omega,zH)\omega(x)g(z^{-1}xH)dm_{\widehat{G}}(w)dm_{G/H}(zH)$

###### Proof.

The proof follows by using Theorem 2.3. ∎

######  Remark 1.

The weak version of the above Corollary 2.4 means that for every function $u$
in $L^{1}(G)\cap L^{2}(G)$ there exist unique $f\in L^{2}(G)$, s.t.

$\langle f,u\rangle=\frac{1}{\langle
g_{2},g_{1}\rangle}\int_{\widehat{G}\times
G/H}\mathcal{D}_{H}^{g_{1}}f_{1}(\omega,zH)\overline{\mathcal{D}_{H}^{g_{2}}u(\omega,zH)}dm_{\widehat{G}}(w)dm_{G/H}(zH)$

which is nothing but the equation (11).

######  Proposition 2.5.

We assume that $g\in L^{1}(G/H)\cap L^{\infty}(G/H)$, $f\in L^{1}(G)\cap
L^{2}(G)$ and $p$ belongs in $[2,\infty]$. We have

(13) $\|D_{g}(f)\|_{L^{p}(\widehat{G}\times
G/H)}\leq\|g\|_{L^{p}(G/H)}\|f\|_{L^{p^{\prime}}(G)}.$

######  Example 2.1.

Here we give one example where $G=\mathbb{R}^{n}$, $H=\\{x\in G:x.\theta=1,\
\theta\in S^{n-1}\\}$. For any element in $G/H$ we can write
$xH=(x.\theta)\theta H$. Then the quotient group $G/H=\\{t\theta
H,t\in\mathbb{R}\\}$. Hence the right-$H$-translation-invariant functions can
be written as

$\displaystyle R_{H}f(xH)=R_{H}f(t\theta H)=\int_{H}f(t\theta
h)dm_{H}(h)=\int_{z.\theta=t}f(z)dz=R_{\theta}f(t),$

where $R_{\theta}f(t)$ is the function on $S^{n-1}\times R$, which is the
Radon transform of $f$. In this case the transform
$\mathcal{D}_{H}^{g}f(\omega,zH)$ is represented using Theorem 2.1 as

$\displaystyle\mathcal{D}_{H}^{g}f(\omega,zH)=\int_{G/H}R_{H}(M_{-\omega}f)(xH)\overline{g(z^{-1}xH)}dm_{G/H}(xH).$

The quotient group is defined by $G/H=\\{t\theta H,t\in\mathbb{R}\\}$. Also,
the characterization of the quotient group is the subgroup $H$ of $G$ defines
an equivalence relation in $G$ by $g_{1}=g_{2}$ iff $g_{2}^{-1}g_{1}\in H$,
i.e., for $g_{1},g_{2}\in\mathbb{R}^{n}$ and $\theta\in S^{n-1}$,
$g_{1}.\theta=g_{2}.\theta$. So the norms are equal and hence the quotient
group $G/H$ is characterized by $\mathbb{R}$. Hence for $z\in\mathbb{R}$,
$\mathcal{D}_{H}^{g}f(\omega,zH)$ is defined on
$S^{n-1}\times\mathbb{R}\times\mathbb{R}^{n}$ as

$\displaystyle\mathcal{D}_{H}^{g}f(\omega,zH)=\int_{t\in\mathbb{R}}R_{\theta}(M_{-\omega}f)(t)\overline{g(t-z)}dt=$
$\displaystyle\int_{t\in\mathbb{R}}\int_{x.\theta=t}f(x)\overline{w(x)}dx\overline{g(t-z)}dt$
$\displaystyle=$
$\displaystyle\int_{t\in\mathbb{R}}\int_{x.\theta=t}f(x)\overline{w(x)}\overline{g(x.\theta-z)}dxdt$
$\displaystyle=$
$\displaystyle\int_{\mathbb{R}^{n}}f(x)\overline{w(x)}\overline{g(x.\theta-z)}dx,$

which is the directional short-time Fourier transform of the function $f$ with
respect to window $g$ [3, 1].

We define the generalized multiplier and generalized two-wavelet multiplier in
the following.

######  Definition 2.6.

Let $\sigma\in L^{\infty}(\widehat{G}\times G/H),$ we define the linear
operator $M_{\sigma,g}:L^{2}(G)\rightarrow L^{2}(G)$ by

$M_{\sigma,g}(f)=(D_{H}^{g})^{-1}(\sigma D_{H}^{g}f).$

This operator is called the generalized multiplier, where $0\neq g\in
L^{1}(G/H)\cap L^{\infty}(G/H)$.

######  Definition 2.7.

Let $u,v$ be a measurable functions on $G$ and $\sigma$ be measurable function
on $\widehat{G}\times G/H$, we define the generalized two wavelet multiplier
operator denoted by $P_{u,v,g}(\sigma)$ on $L^{p}(G),1\leq p\leq\infty$
defined by, for $g\neq 0,g\in L^{1}(G/H)\cap L^{\infty}(G/H)$,

$P_{u,v,g}(\sigma)(f)(t)=\int_{\widehat{G}\times
G/H}\sigma(w,zH)D_{H}^{g}(uf)(w,zH)g_{w,zH}(t)v(t)dm_{\widehat{G}}(\omega)dm_{G/H}(zH),$

where $g_{w,zH}(x)=g(z^{-1}xH)w(x)$.
$P_{u,v,g}(\sigma)$ in a weak sense, for $f\in L^{p}(G),1\leq p\leq\infty$ and
$h\in L^{p^{\prime}}(G)$,

$\langle
P_{u,v,g}(\sigma)(f),h\rangle=\int_{\widehat{G}}\int_{G/H}\sigma(w,zH)D_{H}^{g}(uf)(w,zH)\overline{D_{H}^{g}(vh)(w,zH)}dm_{\widehat{G}}(\omega)dm_{G/H}(zH).$

######  Proposition 2.8.

Let $f$ be in $L^{p}(G)$ and $h$ in $L^{p^{\prime}}$, where $1\leq p<\infty,\
\frac{1}{p}+\frac{1}{p^{\prime}}=1$. Then

$P^{\ast}_{u,v,g}(\sigma)=P_{v,u,g}(\overline{\sigma}).$

###### Proof.

For $f$ be in $L^{p}(G)$ and $h$ in $L^{p^{\prime}}$,

$\displaystyle\langle P_{u,v,g}(\sigma)(f),h\rangle_{L^{2}(G)}$
$\displaystyle=\int_{\widehat{G}}\int_{G/H}\sigma(w,zH)D_{H}^{g}(uf)(w,zH)\overline{D_{H}^{g}(vh)(w,zH)}dm_{\widehat{G}}(\omega)dm_{G/H}(zH)$
$\displaystyle=\overline{\int_{\widehat{G}}\int_{G/H}\overline{\sigma(w,zH)D_{H}^{g}(uf)(w,zH)}D_{H}^{g}(vh)(w,zH)dm_{\widehat{G}}(\omega)dm_{G/H}(zH)}$
$\displaystyle=\overline{\langle P_{v,u,g}(\overline{\sigma})(h),f\rangle}$
$\displaystyle=\langle f,P_{v,u,g}(\overline{\sigma})(h)\rangle.$

Hence we get,

$P^{\ast}_{u,v,g}(\sigma)(h)=P_{v,u,g}(\overline{\sigma})(h).$

This complete the proof. ∎

######  Proposition 2.9.

Let $\sigma\in L^{1}(\widehat{G}\times G/H)\cup L^{\infty}(\widehat{G}\times
G/H)$ and $u,v\in L^{2}(G)\cap L^{\infty}(G)$. Then

$\langle
P_{u,v,g}(\sigma)(f),h\rangle_{L^{2}(G)}=\|g\|_{L^{2}(G/H)}^{2}\langle\overline{v}M_{\sigma,g}(uf),h\rangle_{L^{2}(G)}.$

###### Proof.

In view of Definition 2.6 and Theorem 2.3 we conclude

(14) $\begin{split}\langle
P_{u,v,g}(\sigma)(f),h\rangle&=\int_{\widehat{G}}\int_{G/H}\sigma(w,zH)D_{H}^{g}(uf)(w,zH)\overline{D_{H}^{g}(vh)(w,zH)}dm_{\widehat{G}}(\omega)dm_{G/H}(zH)\\\
&=\int_{\widehat{G}}\int_{G/H}D_{H}^{g}(M_{\sigma,g}(uf))(w,zH)\overline{D_{H}^{g}(vh)(w,zH)}dm_{\widehat{G}}(\omega)dm_{G/H}(zH)\\\
&=\|g\|_{L^{2}(G/H)}^{2}\int_{G}M_{\sigma,g}(uf)(t)\overline{(vh)(t)}dt\\\
&=\|g\|_{L^{2}(G/H)}^{2}\langle\overline{v}M_{\sigma,g}(uf),h\rangle_{L^{2}(G)}.\end{split}$

∎

##  3\. $L^{2}-$boundedness of generalized two wavelet multiplier

In this section we will show the operators

$P_{u,v,g}(\sigma):L^{2}(G)\rightarrow L^{2}(G)$

are bounded linear operators for all symbol $\sigma$ in
$L^{p}(\widehat{G}\times G/H),~{}~{}~{}1\leq p\leq\infty$.
Let us assume $u,v$ be in $L^{2}(G)\cap L^{\infty}(G)$ such that

$\|u\|_{L^{2}(G)}=\|v\|_{L^{2}(G)}=1.$

######  Proposition 3.1.

Let $\sigma$ be in $L^{1}(\widehat{G}\times G/H)$, then the generalized two
wavelet multiplier $P_{u,v,g}(\sigma)$ is in $S_{\infty}$.

###### Proof.

For every functions $f$ and $h$ in $L^{2}(G)$, it follows from Definition 2.7,
Equation (8) and Cauchy-Schwarz inequality,

(15) $\begin{split}&|\langle P_{u,v,g}(\sigma)(f),h\rangle_{L^{2}(G)}|\\\
&\leq\int_{\widehat{G}\times
G/H}|\sigma(w,zH)D_{H}^{g}(uf)(w,zH)D_{H}^{g}(vh)(w,zH)|dm_{\widehat{G}}(\omega)dm_{G/H}(zH)\\\
&\leq\|D_{H}^{g}(uf)\|_{L^{\infty}(\widehat{G}\times
G/H)}\|D_{H}^{g}(vh)\|_{{L^{\infty}(\widehat{G}\times
G/H)}}\int_{\widehat{G}\times
G/H}|\sigma(w,zH)|dm_{\widehat{G}}(\omega)dm_{G/H}(zH)\\\
&\leq\|u\|_{L^{2}(G)}\|f\|_{L^{2}(G)}\|g\|_{L^{\infty}(G/H)}^{2}\|v\|_{L^{2}(G)}\|h\|_{L^{2}(G)}\|\sigma\|_{L^{1}(\widehat{G}\times
G/H)}.\end{split}$

Hence

$\|P_{u,v,g}(\sigma)\|_{S_{\infty}}\leq\|\sigma\|_{L^{1}(\widehat{G}\times
G/H)}\|g\|_{L^{\infty}(G/H)}^{2}.$

∎

######  Proposition 3.2.

Let $\sigma$ be in $L^{\infty}(\widehat{G}\times G/H)$, then the generalized
two wavelet multiplier operator $P_{u,v,g}(\sigma)$ is in $S_{\infty}$.

###### Proof.

Here $P_{u,v,g}(\sigma):L^{2}(G)\rightarrow L^{2}(G)$. For every functions $f$
and $h$ in $L^{2}(G)$ we have the followings from Definition 2.7, Cauchy-
Schwarz’s inequality and Plancherel formula 12 :

(16) $\begin{split}&|\langle P_{u,v,g}(\sigma)(f),h\rangle_{L^{2}(G)}|\\\
&\leq\int_{\widehat{G}\times
G/H}|\sigma(\omega,zH)D_{H}^{g}(uf)(\omega,zH)D_{H}^{g}(vh)(\omega,zH)|dm_{\widehat{G}}(\omega)dm_{G/H}(zH)\\\
&\leq\|D_{H}^{g}(uf)\|_{L^{2}(\widehat{G}\times
G/H)}\|D_{H}^{g}(vh)\|_{{L^{2}(\widehat{G}\times
G/H)}}\|\sigma\|_{L^{\infty}(\widehat{G}\times G/H)}\\\
&\leq\|u\|_{L^{\infty}(G)}\|f\|_{L^{2}(G)}\|g\|_{L^{2}(G/H)}^{2}\|v\|_{L^{\infty}(G)}\|h\|_{L^{2}(G)}\|\sigma\|_{L^{\infty}(\widehat{G}\times
G/H)}.\end{split}$

∎

Now we can proceed to interpret the generalized two wavelet multiplier

$P_{u,v,g}(\sigma):L^{2}(G)\rightarrow L^{2}(G)$

to all symbol $\sigma$ in $L^{p}(\widehat{G}\times G/H)~{}~{}1\leq
p\leq\infty$ being in $S_{\infty}$. It is given in the following theorem.

######  Theorem 3.3.

Let $\sigma\in L^{p}(\widehat{G}\times G/H),1\leq p\leq\infty$. Then there
exists a unique bounded linear operator $P_{u,v,g}(\sigma):L^{2}(G)\rightarrow
L^{2}(G)$ s.t.

$\|P_{u,v,g}(\sigma)\|_{S_{\infty}}\leq\|g\|_{L^{\infty}(G/H)}^{2/p}\|g\|_{L^{2}(G/H)}^{\frac{2(p-1)}{p}}(\|u\|_{L^{\infty}(G)}\|v\|_{L^{\infty}(G)})^{\frac{p-1}{p}}\|\sigma\|_{L^{p}(\widehat{G}\times
G/H)}.$

###### Proof.

Let $f\in L^{2}(G)$ and $T:L^{1}(\widehat{G}\times G/H)\cap
L^{\infty}(\widehat{G}\times G/H)\rightarrow L^{2}(G)$ given by

$T(\sigma):=P_{u,v,g}(\sigma)f.$

We have from Proposition 3.1

$\|P_{u,v,g}(\sigma)f\|_{L^{2}(G)}\leq\|f\|_{L^{2}(G)}\|g\|_{L^{\infty}(G/H)}^{2}\|\sigma\|_{L^{1}(\widehat{G}\times
G/H)}$

and from Proposition 3.2

$\|P_{u,v,g}(\sigma)f\|_{L^{2}(G)}\leq\|f\|_{L^{2}(G)}\|g\|_{L^{2}(G/H)}^{2}\|\sigma\|_{L^{\infty}(\widehat{G}\times
G/H)}\|u\|_{L^{\infty}(G)}\|v\|_{L^{\infty}(G)}.$

By Riesz-Thorin interpolation theorem, $T$ may uniquely extended to
$L^{p}(\widehat{G}\times G/H),1\leq p\leq\infty$ and

$\|T(\sigma)\|_{L^{2}(G)}=\|P_{u,v,g}(\sigma)f\|_{L^{2}(G)}.$

So

$\|P_{u,v,g}(\sigma)f\|_{L^{2}(G)}\leq\|g\|_{L^{\infty}(G/H)}^{2/p}\|g\|_{L^{2}(G/H)}^{\frac{2(p-1)}{p}}(\|u\|_{L^{\infty}(G)}\|v\|_{L^{\infty}(G)})^{\frac{p-1}{p}}\|f\|_{L^{2}(G)}\|\sigma\|_{L^{p}(\widehat{G}\times
G/H)}.$

∎

##  4\. Schatten class boundednes of generalized two wavelet multiplier

Our aim is to show the linear operators

$P_{u,v,g}:L^{2}(G)\rightarrow L^{2}(G)$

are in Schatten class $S_{p}$ for all symbol $\sigma$ in
$L^{p}(\widehat{G}\times G/H),~{}~{}~{}1\leq p\leq\infty$.

######  Proposition 4.1.

Let $\sigma$ be in $L^{1}(\widehat{G}\times G/H),$ then the generalized two
wavelet multiplier $P_{u,v,g}(\sigma)$ is in $S_{2}$ and we have

$\|P_{u,v,g}(\sigma)\|_{S_{2}}\leq\|g\|_{L^{\infty}(G/H)}\|\sigma\|_{L^{1}(\widehat{G}\times
G/H)}.$

###### Proof.

Let $\\{\phi_{j}:j=1,2,\cdots\\}$ be an orthonormal basis for $L^{2}(G)$.

$\sum_{j=1}^{\infty}\|P_{u,v,g}(\sigma)(\phi_{j})\|_{L^{2}(G)}^{2}=\sum_{j=1}^{\infty}\langle
P_{u,v,g}(\sigma)\phi_{j},P_{u,v,g}(\sigma)\phi_{j}\rangle.$

Now using Definition 2.7, Fubini’s theorem, Parseval’s identity and
Proposition 2.8, we have

$\begin{split}&\langle
P_{u,v,g}(\sigma)\phi_{j},P_{u,v,g}(\sigma)\phi_{j}\rangle\\\
&=\int_{\widehat{G}\times
G/H}\sigma(w,zH)D_{H}^{g}(u\phi_{j})(w,zH)\overline{D_{H}^{g}(vP_{u,v,g}\sigma\phi_{j})(w,zH)}dm_{\widehat{G}}(\omega)dm_{G/H}(zH)\\\
&=\int_{\widehat{G}\times G/H}\sigma(w,zH)\langle
u\phi_{j},\overline{g_{w,zH}}\rangle_{L^{2}(G)}\overline{\langle
vP_{u,v,g}(\sigma)(\phi_{j}),\overline{g_{w,zH}}\rangle_{L^{2}(G)}}dm_{\widehat{G}}(\omega)dm_{G/H}(zH)\\\
&=\int_{\widehat{G}\times
G/H}\sigma(w,zH)\langle\phi_{j},\overline{ug_{w,zH}}\rangle\overline{\langle
P_{u,v,g}(\sigma)(\phi_{j}),\overline{g_{w,zH}}\rangle}dm_{\widehat{G}}(\omega)dm_{G/H}(zH).\end{split}$

So

$\displaystyle\sum_{j=1}^{\infty}\|P_{u,v,g}(\sigma)(\phi_{j})\|_{L^{2}(G)}^{2}$
$\displaystyle=\sum_{j=1}^{\infty}\int_{\widehat{G}\times
G/H}\sigma(\omega,zH)\langle\phi_{j},\overline{ug_{\omega,zH}}\rangle\overline{\langle
P_{u,v,g}(\sigma)(\phi_{j}),\overline{vg_{\omega,zH}}\rangle}dm_{\widehat{G}}(\omega)dm_{G/H}(zH)$
$\displaystyle=\int_{\widehat{G}\times
G/H}\sigma(\omega,zH)\sum_{j=1}^{\infty}\langle\phi_{j},\overline{ug_{\omega,zH}}\rangle\langle
P_{v,u,g}^{\ast}(\sigma)(\overline{vg_{\omega,zH}}),\phi_{j}\rangle
dm_{\widehat{G}}(\omega)dm_{G/H}(zH)$ $\displaystyle=\int_{\widehat{G}\times
G/H}\sigma(\omega,zH)\langle
P_{v,u,g}^{\ast}(\sigma)(\overline{vg_{\omega,zH}}),\sum_{j=1}^{\infty}\phi_{j}\langle\phi_{j},\overline{ug_{\omega,zH}}\rangle\rangle
dm_{\widehat{G}}(\omega)dm_{G/H}(zH)$ $\displaystyle=\int_{\widehat{G}\times
G/H}\sigma(\omega,zH)\langle
P_{v,u,g}^{\ast}(\sigma)(\overline{vg_{\omega,zH}}),\overline{ug_{\omega,zH}}\rangle_{L^{2}(G)}dm_{\widehat{G}}(\omega)dm_{G/H}(zH).$

Thus

$\begin{split}&\sum_{j=1}^{\infty}\|P_{u,v,g}(\sigma)(\phi_{j})\|_{L^{2}(G)}^{2}\\\
&\leq\|g\|_{L^{2}(G/H)}^{2}\int_{\widehat{G}\times
G/H}|\sigma(\omega,zH)|\|P_{u,v,g}^{\ast}(\sigma)\|_{S_{\infty}}dm_{\widehat{G}}(\omega)dm_{G/H}(zH)\\\
&\leq\|g\|_{L^{\infty}(G/H)}^{4}\|\sigma\|_{L^{1}(\widehat{G}\times
G/H)}^{2}.\end{split}$

∎

######  Proposition 4.2.

Let $\sigma$ be a symbol in $L^{p}(\widehat{G}\times G/H),1\leq p<\infty.$
Then the generalized two-wavelet multiplier $P_{u,v,g}(\sigma)$ is compact.

###### Proof.

We know $L^{1}(\widehat{G}\times G/H)\cap L^{\infty}(\widehat{G}\times G/H)$
is dense in $L^{p}(\widehat{G}\times G/H),(1\leq p<\infty)$. Let $\sigma\in
L^{p}(\widehat{G}\times G/H)$, then there exist
$\\{\sigma_{n}\\}_{n\in\mathbb{N}}\subset L^{1}(\widehat{G}\times G/H)\cap
L^{\infty}(\widehat{G}\times G/H)$ s.t. $\sigma_{n}\rightarrow\sigma$ in
$L^{p}(\widehat{G}\times G/H)$ as $n\rightarrow\infty$.
Now by Theorem 3.3

$\displaystyle\|P_{u,v,g}(\sigma_{n})-P_{u,v,g}(\sigma)\|_{B(L^{2}(G))}=\|P_{u,v,g}(\sigma_{n}-\sigma)\|_{B(L^{2}(G))}$
$\displaystyle\leq\|g\|_{L^{\infty}(G/H)}^{2/p}\|g\|_{L^{2}(G/H)}^{\frac{2(p-1)}{p}}(\|u\|_{L^{\infty}(G)}\|v\|_{L^{\infty}(G)})^{\frac{p-1}{p}}\|\sigma_{n}-\sigma\|_{L^{p}(\widehat{G}\times
G/H)}.$

So $P_{u,v,g}(\sigma_{n})\rightarrow P_{u,v,g}(\sigma)$ in $B(L^{2}(G))$.
Hence $P_{u,v,g}(\sigma)$ is compact. ∎

######  Theorem 4.3.

Let $\sigma$ be in $L^{1}(\widehat{G}\times G/H)$. Then
$P_{u,v,g}(\sigma):L^{2}(G)\rightarrow L^{2}(G)$ is in $S_{1}$.

###### Proof.

Here $\sigma\in L^{1}(\widehat{G}\times G/H)$ then $P_{u,v,g}(\sigma)$ is in
$S_{2}$. So there exists an orthonormal basis $\\{\phi_{j}:j=1,2,3,\cdots\\}$
for the orthogonal compliment of the kernel $P_{u,v,g}(\sigma)$ consisting of
eigenvectors of $|P_{u,v,g}(\sigma)|$ and the subset
$\\{\psi_{j}:j=1,2,3,\cdots\\}$ of $L^{2}(G)$ such that

$P_{u,v,g}(\sigma)(f)=\sum_{j=1}^{\infty}s_{j}\langle
f,\phi_{j}\rangle_{L^{2}(G)}\psi_{j},$

where $s_{j},j=1,2,3,\cdots$ are positive singular values of
$P_{u,v,g}(\sigma)$ corresponding to $\phi_{j}$.
Then

$\|P_{u,v,g}(\sigma)\|_{S_{1}}=\sum_{j=1}^{\infty}s_{j}=\sum_{j=1}^{\infty}\langle
P_{u,v,g}(\sigma)(\phi_{j}),\psi_{j}\rangle_{L^{2}(G)}.$

Now by Bessel’s inequality, Cauchy-Schwarz’s inequality, Fubini’s theorem and
the fact $\|u\|_{L^{2}(G)}=\|u\|_{L^{2}(G)}=1,$ we get

$\displaystyle\sum_{j=1}^{\infty}\langle
P_{u,v,g}(\sigma)(\phi_{j}),\psi_{j}\rangle_{L^{2}(G)}$
$\displaystyle=\sum_{j=1}^{\infty}\int_{\widehat{G}\times
G/H}\sigma(\omega,zH)D_{H}^{g}(u\phi_{j})(\omega,zH)\overline{D_{H}^{g}(v\psi_{j})(\omega,zH)}dm_{\widehat{G}}(\omega)dm_{G/H}(zH)$
$\displaystyle=\int_{\widehat{G}\times
G/H}\sigma(\omega,zH)\sum_{j=1}^{\infty}\langle\phi_{j},\overline{ug_{\omega,zH}}\rangle_{L^{2}(G)}\langle\overline{vg_{\omega,zH}},\psi_{j}\rangle_{L^{2}(G)}dm_{\widehat{G}}(\omega)dm_{G/H}(zH)$
$\displaystyle\leq\int_{\widehat{G}\times
G/H}|\sigma(\omega,zH)|\Big{(}\sum_{j=1}^{\infty}|\langle\phi_{j},\overline{ug_{\omega,zH}}\rangle_{L^{2}(G)}|^{2}\Big{)}^{1/2}$
$\displaystyle\Big{(}\sum_{j=1}^{\infty}|\langle\overline{vg_{\omega,zH}},\psi_{j}\rangle_{L^{2}(G)}|^{2}\Big{)}^{1/2}dm_{\widehat{G}}(\omega)dm_{G/H}(zH)$
$\displaystyle\leq\int_{\widehat{G}\times
G/H}|\sigma(\omega,zH)|\|ug_{\omega,zH}\|_{L^{2}(G)}\|vg_{\omega,zH}\|_{L^{2}(G)}dm_{\widehat{G}}(\omega)dm_{G/H}(zH)$
$\displaystyle\leq\|\sigma\|_{L^{1}(\widehat{G}\times
G/H)}\|g\|_{L^{2}(G/H)}^{2}.$

∎

######  Corollary 4.4.

For $\sigma\in L^{1}(\widehat{G}\times G/H)$, we have the following trace
formula;

$\text{tr}(P_{u,v,g}(\sigma))=\int_{\widehat{G}\times
G/H}\sigma(\omega,zH)\langle
vg_{\omega,zH},ug_{\omega,zH}\rangle_{L^{2}(G)}dm_{\widehat{G}}(\omega)dm_{G/H}(zH).$

###### Proof.

Let $\\{\phi_{k}:k=1,2,\cdots\\}$ be an orthonormal basis for $L^{2}(G)$. By
Theorem 4.3, the generalized two wavelet multiplier $P_{u,v,g}(\sigma)$ is in
$S_{1}$.
Now by definition of trace formula , Fubini’s theorem and Parseval’s identity,
we have

$\displaystyle\text{tr}(P_{u,v,g}(\sigma))$
$\displaystyle=\sum_{k=1}^{\infty}\langle
P_{u,v,g}(\sigma)(\phi_{k}),\phi_{k}\rangle_{L^{2}(G)}$
$\displaystyle=\sum_{k=1}^{\infty}\int_{\widehat{G}\times
G/H}\sigma(\omega,zH)\langle\phi_{k},\overline{ug_{\omega,zH}}\rangle_{L^{2}(G)}\overline{\langle\phi_{k},\overline{vg_{\omega,zH}}\rangle_{L^{2}(G)}}dm_{\widehat{G}}(\omega)dm_{G/H}(zH)$
$\displaystyle=\int_{\widehat{G}\times
G/H}\sigma(\omega,zH)\sum_{k=1}^{\infty}\langle\phi_{k},\overline{ug_{\omega,zH}}\rangle_{L^{2}(G)}\overline{\langle\phi_{k},\overline{vg_{\omega,zH}}\rangle_{L^{2}(G)}}dm_{\widehat{G}}(\omega)dm_{G/H}(zH)$
$\displaystyle=\int_{\widehat{G}\times
G/H}\sigma(\omega,zH)\langle\overline{ug_{\omega,zH}},\overline{vg_{\omega,zH}}\rangle_{L^{2}(G)}dm_{\widehat{G}}(\omega)dm_{G/H}(zH).$

∎

We give a result

$P_{u,v,g}(\sigma):L^{2}(G)\rightarrow L^{2}(G)$

will be in $S_{p}$ for $\sigma\in L^{p}(G),~{}~{}1\leq p\leq\infty$.

######  Corollary 4.5.

Let $\sigma\in L^{p}(\widehat{G}\times G/H),1\leq p\leq\infty$. Then the
generalized two wavelet multiplier $P_{u,v,g}(\sigma):L^{2}(G)\rightarrow
L^{2}(G)$ is in $S_{p}$ and we have

$\|P_{u,v,g}\|_{S_{p}}\leq\|g\|_{L^{\infty}(G/H)}^{2/p}\|g\|_{L^{2}(G/H)}^{\frac{2(p-1)}{p}}(\|u\|_{L^{\infty}(G)}\|v\|_{L^{\infty}(G)})^{\frac{p-1}{p}}\|\sigma\|_{L^{p}(\widehat{G}\times
G/H)}.$

###### Proof.

The proof follows from above Theorem 4.3 and Proposition 3.2 and interpolation
theorem in [12].∎

##  5\. $L^{p}-$boundedness of generalized two wavelet multiplier for $1\leq
p\leq\infty$

Our aim is to show the linear operators

$P_{u,v,g}:L^{p}(G)\rightarrow L^{p}(G)$

are bounded operators for all symbol $\sigma$ in $L^{r}(\widehat{G}\times
G/H),~{}~{}~{}1<r\leq\infty$ for all $p\in[\frac{2r}{r+1},\frac{2r}{r-1}]$ and
for $r=1,p\in[1,\infty].$

Let us assume $0\neq g\in L^{\infty}(G/H)$ for this section.

######  Proposition 5.1.

Let $\sigma$ be in $L^{1}(\widehat{G}\times G/H),u\in L^{\infty}(G)$ and $v\in
L^{1}(G)$, then the generalized two wavelet multiplier

$P_{u,v,g}(\sigma):L^{1}(G)\rightarrow L^{1}(G)$

is a bounded operator and we have

$\|P_{u,v,g}(\sigma)\|_{B(L^{1}(G))}\leq\|u\|_{L^{\infty}(G)}\|v\|_{L^{1}(G)}\|g\|_{L^{\infty}(G/H)}^{2}\|\sigma\|_{L^{1}(\widehat{G}\times
G/H)}.$

###### Proof.

We know that

$P_{u,v,g}(\sigma)(f)(t)=\int_{\widehat{G}\times
G/H}\sigma(\omega,zH)D_{H}^{g}(uf)(\omega,zH)g_{\omega,zH}(t)\overline{v(t)}dm_{\widehat{G}}(\omega)dm_{G/H}(zH).$

Now by Equation 8 and Equation 10

$\displaystyle\|P_{u,v,g}(\sigma)(f)\|_{L^{1}(G)}$
$\displaystyle\leq\int_{\widehat{G}\times
G/H}\int_{G}|\sigma(\omega,zH)||D_{H}^{g}(uf)(\omega,zH)||g_{\omega,zH}(t)v(t)|dm_{\widehat{G}}(\omega)dm_{G/H}(zH)dt$
$\displaystyle\leq\int_{\widehat{G}\times
G/H}\int_{G}|\sigma(\omega,zH)||\langle
uf,g_{\omega,zH}\rangle_{L^{2}(G)}|\|g\|_{L^{\infty}(G/H)}|v(t)|dm_{\widehat{G}}(\omega)dm_{G/H}(zH)dt$
$\displaystyle\leq\|g\|_{L^{\infty}(G/H)}^{2}\|f\|_{L^{1}(G)}\|u\|_{L^{\infty}(G)}\int_{\widehat{G}\times
G/H}|\sigma(\omega,zH)|dm_{\widehat{G}}(\omega)dm_{G/H}(zH)\int_{G}|v(t)|dt$
$\displaystyle=\|g\|_{L^{\infty}(G/H)}^{2}\|f\|_{L^{1}(G)}\|u\|_{L^{\infty}(G)}\|\sigma\|_{L^{1}(\widehat{G}\times
G/H)}\|v\|_{L^{1}(G)}.$

∎

######  Proposition 5.2.

Let $\sigma$ be in $L^{1}(\widehat{G}\times G/H),u\in L^{1}(G)$ and $v\in
L^{\infty}(G)$, then the generalized two wavelet multiplier

$P_{u,v,g}(\sigma):L^{\infty}(G)\rightarrow L^{\infty}(G)$

is a bounded operator and we have

$\|P_{u,v,g}(\sigma)\|_{B(L^{\infty}(G))}\leq\|u\|_{L^{1}(G)}\|v\|_{L^{\infty}(G)}\|g\|_{L^{2}(G/H)}^{2}\|\sigma\|_{L^{1}(\widehat{G}\times
G/H)}.$

###### Proof.

Let $f$ be in $L^{\infty}(G)$, then by Equation 10 and Equation 8 we have

$\displaystyle|P_{u,v,g}(\sigma)(f)(t)|$
$\displaystyle\leq\int_{\widehat{G}\times
G/H}|\sigma(\omega,zH)||D_{H}^{g}(uf)(\omega,zH)||g_{\omega,zH}(t)||\overline{v(t)}|dm_{\widehat{G}}(\omega)dm_{G/H}(zH)$
$\displaystyle\leq\|u\|_{L^{1}(G)}\|v\|_{L^{\infty}(G)}\|g\|_{L^{2}(G/H)}^{2}\|\sigma\|_{L^{1}(\widehat{G}\times
G/H)}\|f\|_{L^{\infty}(G)}^{2}.$

Let $\|f\|_{L^{\infty}(G)}=1$, hence

$\|P_{u,v,g}(\sigma)\|_{B(L^{\infty}(G))}\leq\|u\|_{L^{1}(G)}\|v\|_{L^{\infty}(G)}\|g\|_{L^{2}(G/H)}^{2}\|\sigma\|_{L^{1}(\widehat{G}\times
G/H)}.$

∎

######  Theorem 5.3.

Let $u,v\in L^{1}(G)\cap L^{\infty}(G).$ Then for all $\sigma\in
L^{1}(\widehat{G}\times G/H),$ there exist a unique bounded linear operator

$P_{u,v,g}(\sigma):L^{p}(G)\rightarrow L^{p}(G),~{}~{}~{}~{}~{}1\leq
p\leq\infty,$

s.t.

$\|P_{u,v,g}(\sigma)\|_{B(L^{p}(G))}\leq\|u\|_{L^{1}(G)}^{1/p^{\prime}}\|v\|_{L^{1}(G)}^{1/p}\|u\|_{L^{\infty}(G)}^{1/p}\|v\|_{L^{\infty}(G)}^{1/p^{\prime}}\|g\|_{L^{\infty}(G/H)}\|\sigma\|_{L^{1}(\widehat{G}\times
G/H)}.$

###### Proof.

We know

$P_{u,v,g}(\sigma):L^{1}(G)\rightarrow L^{1}(G)$

is the adjoint of

$P_{v,u,g}(\overline{\sigma}):L^{\infty}(G)\rightarrow L^{\infty}(G).$

Now by Proposition 5.1 and Proposition 5.2 and interpolation theorem, for
$1\leq p\leq\infty$,

$\|P_{u,v,g}(\sigma)\|_{B(L^{p}(G))}\leq\|u\|_{L^{1}(G)}^{1/p^{\prime}}\|v\|_{L^{1}(G)}^{1/p}\|u\|_{L^{\infty}(G)}^{1/p}\|v\|_{L^{\infty}(G)}^{1/p^{\prime}}\|g\|_{L^{\infty}(G/H)}\|\sigma\|_{L^{1}(\widehat{G}\times
G/H)}.$

∎

We give another version of $L^{p}$\- boundedness of Theorem 5.3.

######  Theorem 5.4.

Let $\sigma$ be in $L^{1}(\widehat{G}\times G/H)$, $u$ and $v$ in
$L^{1}(G)\cap L^{\infty}(G)$. Then there exists a unique bounded linear
operator $P_{u,v,g}(\sigma):L^{p}(G)\rightarrow L^{p}(G)~{}~{}~{}1\leq
p\leq\infty$ s.t.

$\|P_{u,v,g}(\sigma)\|_{B(L^{p}(G))}\leq\text{max}\Big{(}\|u\|_{L^{1}(G)}\|v\|_{L^{\infty}(G)},\|u\|_{L^{\infty}(G)}\|v\|_{L^{1}(G)}\Big{)}\|g\|_{L^{\infty}(G)}^{2}\|\sigma\|_{L^{1}(\widehat{G}\times
G/H)}.$

###### Proof.

From Definition 2.7,

$\displaystyle P_{u,v,g}(\sigma)(f)(t)$ $\displaystyle=\int_{\widehat{G}\times
G/H}\sigma(\omega,zH)D_{H}^{g}(uf)(\omega,zH)g_{\omega,zH}(t)\overline{v(t)}dm_{\widehat{G}}(\omega)dm_{G/H}(zH)$
$\displaystyle=\int_{\widehat{G}\times
G/H}\sigma(\omega,zH)\int_{G}u(s)f(s)\overline{g_{\omega,zH}(s)}dsg_{\omega,zH}(t)\overline{v(t)}dm_{\widehat{G}}(\omega)dm_{G/H}(zH)$

So the integral operator is

$P_{u,v,g}(\sigma)(f)(t)=\int_{G}N(t;s)f(s)ds,$

with the kernel

$N(t;s)=\int_{\widehat{G}\times
G/H}\sigma(\omega,zH)u(s)\overline{g_{\omega,zH}(s)}g_{\omega,zH}(t)\overline{v(t)}dm_{\widehat{G}}(\omega)dm_{G/H}(zH).$

Now

(17) $\begin{split}\int_{G}|N(t;s)|dt&\leq\int_{G}\int_{\widehat{G}\times
G/H}|\sigma(\omega,zH)||u(s)||\overline{g_{\omega,zH}(s)}||g_{\omega,zH}(t)||\overline{v(t)}|dm_{\widehat{G}}(\omega)dm_{G/H}(zH)dt\\\
&\leq\|u\|_{L^{\infty}(G)}\|g\|_{L^{\infty}(G/H)}\|g\|_{L^{\infty}(G/H)}\|v\|_{L^{1}(G)}\|\sigma\|_{L^{1}(\widehat{G}\times
G/H)}.\end{split}$

Similarly

$\int_{G}|N(t;s)|ds\leq\|u\|_{L^{1}(G)}\|g\|_{L^{\infty}(G/H)}\|g\|_{L^{\infty}(G/H)}\|v\|_{L^{\infty}(G)}\|\sigma\|_{L^{1}(\widehat{G}\times
G/H)}.$

Thus by Schur’s test [7], we can conclude
$P_{u,v,g}(\sigma):L^{p}(G)\rightarrow L^{p}(G)$ bounded and

$\|P_{u,v,g}(\sigma)\|_{B(L^{p}(G))}\leq\text{max}\Big{(}\|u\|_{L^{1}(G)}\|v\|_{L^{\infty}(G)},\|u\|_{L^{\infty}(G)}\|v\|_{L^{1}(G)}\Big{)}\|g\|_{L^{\infty}(G)}^{2}\|\sigma\|_{L^{1}(\widehat{G}\times
G/H)}.$

∎

######  Proposition 5.5.

Let $\sigma$ be in $L^{1}(\widehat{G}\times G/H)$, $v$ in $L^{p}(G)$ and $u$
in $L^{p^{\prime}}(G)$ for $1<p\leq\infty$ then the generalized two-wavelet
multiplier $P_{u,v,g}(\sigma):L^{p}(G)\rightarrow L^{p}(G)$ is a bounded
linear operator and we have

$\|P_{u,v,g}(\sigma)\|_{B(L^{2}(G))}\leq\|u\|L^{p^{\prime}}(G)\|v\|_{L^{p}(G)}\|g\|_{L^{\infty}(G)}^{2}\|\sigma\|_{L^{1}(\widehat{G}\times
G/H)}.$

###### Proof.

For any $f\in L^{p}(G)$, consider the linear functional
$I_{f}:L^{p^{\prime}}(G)\rightarrow\mathbb{C}$ defined by

$I_{f}(h)=\langle h,P_{u,v,g}(\sigma)(f)\rangle_{L^{2}(G)}.$

Now from Definition 2.7, Equation 8 and Holder’s inequality, we have

$\displaystyle|\langle P_{u,v,g}(\sigma)(f),h\rangle_{L^{2}(G)}|$
$\displaystyle\leq\int_{\widehat{G}\times
G/H}|\sigma(\omega,zH)||D_{H}^{g}(uf)(\omega,zH)||D_{H}^{g}(vh)(\omega,zH)|dm_{\widehat{G}}(\omega)dm_{G/H}(zH)$
$\displaystyle=\int_{\widehat{G}\times G/H}|\sigma(\omega,zH)||\langle
uf,g_{\omega,zH}\rangle_{L^{2}(G)}||\langle
vh,g_{\omega,zH}\rangle_{L^{2}(G)}|dm_{\widehat{G}}(\omega)dm_{G/H}(zH)$
$\displaystyle\leq\int_{\widehat{G}\times G/H}|\sigma(\omega,zH)||\langle
u,\overline{f}g_{\omega,zH}\rangle_{L^{2}(G)}||\langle
h,\overline{v}g_{\omega,zH}\rangle_{L^{2}(G)}|dm_{\widehat{G}}(\omega)dm_{G/H}(zH)$
$\displaystyle\leq\int_{\widehat{G}\times
G/H}|\sigma(\omega,zH)|\|u\|_{L^{p^{\prime}}(G)}\|v\|_{L^{p}(G)}\|g\|_{L^{\infty}(G/H)}^{2}\|f\|_{L^{p}(G)}\|h\|_{L^{p^{\prime}}(G)}dm_{\widehat{G}}(\omega)dm_{G/H}(zH)$
$\displaystyle\leq\|\sigma\|_{L^{1}(\widehat{G}\times
G/H)}\|u\|_{L^{p^{\prime}}(G)}\|v\|_{L^{p}(G)}\|g\|_{L^{\infty}(G/H)}^{2}\|f\|_{L^{p}(G)}\|h\|_{L^{p^{\prime}}(G)}.$

By Riesz representation theorem

$\|P_{u,v,g}(\sigma)(f)\|=\|I_{f}\|_{B({L^{p^{\prime}}(G)})}.$

So

$|I_{f}(h)|\leq\|\sigma\|_{L^{1}(\widehat{G}\times
G/H)}\|u\|_{L^{p^{\prime}}(G)}\|v\|_{L^{p}(G)}\|g\|_{L^{\infty}(G/H)}^{2}\|f\|_{L^{p}(G)}\|h\|_{L^{p^{\prime}}(G)}.$

Then

$\|I_{f}\|_{B({L^{p^{\prime}}(G)})}\leq\|\sigma\|_{L^{1}(\widehat{G}\times
G/H)}\|u\|_{L^{p^{\prime}}(G)}\|v\|_{L^{p}(G)}\|g\|_{L^{\infty}(G/H)}^{2}\|f\|_{L^{p}(G)},$

and hence

$\|P_{u,v,g}(\sigma)\|_{B({L^{p}(G)})}\leq\|\sigma\|_{L^{1}(\widehat{G}\times
G/H)}\|u\|_{L^{p^{\prime}}(G)}\|v\|_{L^{p}(G)}\|g\|_{L^{\infty}(G/H)}^{2}.$

∎

We give the result for $p=1$ of Proposition 5.5.

######  Theorem 5.6.

Let $\sigma$ be in $L^{1}(\widehat{G}\times G/H)$, $v$ in $L^{p}(G)$ and $u$
in $L^{p^{\prime}}(G)$ for $1\leq p\leq\infty$ then the generalized two-
wavelet multiplier $P_{u,v,g}(\sigma):L^{p}(G)\rightarrow L^{p}(G)$ is a
bounded linear operator and we have

$\|P_{u,v,g}(\sigma)\|_{B(L^{2}(G))}\leq\|u\|_{L^{p^{\prime}}}(G)\|v\|_{L^{p}(G)}\|g\|_{L^{\infty}(G)}^{2}\|\sigma\|_{L^{1}(\widehat{G}\times
G/H)}.$

###### Proof.

We can show using Proposition 5.5 and Proposition 5.1. ∎

Let us consider $0\neq g\in L^{1}(G/H)\cap L^{\infty}(G/H)\subset
L^{q}(G/H),~{}~{}~{}(1<q<\infty)$ for rest of this section.

######  Theorem 5.7.

Let $\sigma$ be in $L^{r}(\widehat{G}\times G/H),~{}~{}~{}~{}r\in[1,2]$ and
$u,v\in L^{1}(G)\cap L^{\infty}(G).$ Then there exists a unique bounded linear
operator $P_{u,v,g}(\sigma):L^{p}(G)\rightarrow L^{p}(G)$ for all
$p\in[r,r^{\prime}]$ and we have

$\|P_{u,v,g}(\sigma)\|_{B(L^{p}(G))}\leq
c_{1}^{t}c^{1-t}_{2}\|\sigma\|_{L^{r}(\widehat{G}\times
G/H)},~{}~{}~{}~{}\frac{t}{r}+\frac{1-t}{r^{\prime}}=\frac{1}{p}.$

###### Proof.

Consider the linear functional

$I:(L^{1}(\widehat{G}\times G/H)\cap L^{2}(\widehat{G}\times
G/H))\times(L^{1}(G)\cap L^{2}(G))\rightarrow L^{1}(G)\cap L^{2}(G),$
$(\sigma,f)\mapsto P_{u,v,g}(\sigma)(f).$

By Proposition 5.1 for $\sigma\in L^{1}(\widehat{G}\times G/H),f\in L^{1}(G)$,

$\|I(\sigma,f)\|_{L^{1}(G)}=\|P_{u,v,g}(\sigma)(f)\|_{L^{1}(G)}\leq\|f\|_{L^{1}(G)}\|u\|_{L^{\infty}(G)}\|v\|_{L^{1}(G)}\|g\|_{L^{\infty}(G/H)}^{2}\|\sigma\|.$

By Theorem 3.2, for $\sigma\in L^{2}(\widehat{G}\times G/H),~{}~{}f\in
L^{2}(G)$,

$\begin{split}\|I(\sigma,f)\|_{L^{2}(G)}&=\|P_{u,v,g}(\sigma)(f)\|_{L^{2}(G)}\leq\|f\|_{L^{2}(G)}\Big{(}\|u\|_{L^{\infty}(G)}\|v\|_{L^{\infty}(G)}\Big{)}^{1/2}\\\
&\|\sigma\|_{L^{2}(\widehat{G}\times
G/H)}\|g\|_{L^{\infty}(G/H)}\|g\|_{L^{2}(G/H)}.\end{split}$

By multilinear interpolation theory, we get a unique bounded operator

$I:L^{r}(\widehat{G}\times G/H)\times L^{r}(\widehat{G}\times G/H)\rightarrow
L^{r}(G)$

such that

$\|I(\sigma,f)\|_{L^{r}(G)}\leq
c_{1}\|f\|_{L^{r}(G)}\|\sigma\|_{L^{r}(\widehat{G}\times G/H)},$

where

$c_{1}=\Big{(}\|g\|_{L^{\infty}(G/H)}^{2}\|u\|_{L^{\infty}(G)}\|v\|_{L^{1}(G)}\Big{)}^{\theta}\Big{(}\|g\|_{L^{2}(G/H)}^{2}\|g\|_{L^{\infty}(G/H)}\|u\|_{L^{\infty}(G)}\|v\|_{L^{\infty}(G)}\Big{)}^{\frac{1-\theta}{2}},$

and $\frac{\theta}{1}+\frac{1-\theta}{2}=\frac{1}{r}.$
By definition of $I$,

$\|P_{u,v,g}(\sigma)\|_{B(L^{r}(G))}\leq
c_{1}\|\sigma\|_{L^{r}(\widehat{G}\times G/H)}.$

Also $P_{v,u,g}(\overline{\sigma})$ is the adjoint of $P_{u,v,g}(\sigma)$, so
$P_{u,v,g}(\overline{\sigma})$ is a bounded linear operator on
$L^{r^{\prime}}(G)$ with the operator norm

$\|P_{v,u,g}(\overline{\sigma})\|_{B(L^{r^{\prime}}(G))}=\|P_{u,v,g}(\sigma)\|_{B(L^{r}(G))}\leq
c_{2}\|\sigma\|_{L^{r}(\widehat{G}\times G/H)},$

where

$c_{2}=\Big{(}\|g\|_{L^{\infty}(G/H)}^{2}\|u\|_{L^{1}(G)}\|v\|_{L^{\infty}(G)}\Big{)}^{\theta}\Big{(}\|g\|_{L^{2}(G/H)}^{2}\|g\|_{L^{\infty}(G/H)}\|u\|_{L^{\infty}(G)}\|v\|_{L^{\infty}(G)}\Big{)}^{\frac{1-\theta}{2}}.$

By interpolation theorem, for $p\in[r,r^{\prime}]$,

$\|P_{u,v,g}(\sigma)\|_{B(L^{p}(G))}\leq
c_{1}^{t}c_{2}^{1-t}\|\sigma\|_{L^{r}(\widehat{G}\times G/H)}.$

∎

######  Theorem 5.8.

Let $\sigma$ be in $L^{r}(\widehat{G}\times G/H),~{}~{}r\in[1,2)$ and $u,v\in
L^{r}(G)\cap L^{\infty}(G).$ Then there exists a linear bounded operator
$P_{u,v,g}(\sigma):L^{p}(G)\rightarrow L^{p}(G)$ for all $p\in[r,r^{\prime}]$
and we have

$\displaystyle\|P_{u,v,g}(\sigma)\|_{B(L^{p}(G))}$
$\displaystyle\leq\|g\|_{L^{\infty}(G/H)}\|g\|_{L^{r^{\prime}}}(G/H)\Big{(}\|u\|_{L^{r}(G)}\|v\|_{L^{\infty}(G)}\Big{)}^{t}\Big{(}\|v\|_{L^{r}(G)}\|u\|_{L^{\infty}(G)}\Big{)}^{1-t}\|\sigma\|_{L^{r}(\widehat{G}\times
G/H)},$

where $t=\frac{r-p}{p(r-2)}$.

###### Proof.

For any $f\in L^{r^{\prime}}(G)$ and $h\in L^{r}(G)$

$\displaystyle|\langle P_{u,v,g}(\sigma)(f),h\rangle_{L^{2}(G)}|$
$\displaystyle\leq\int_{\widehat{G}\times
G/H}|\Delta(\omega,zH)||D_{H}^{g}(uf)(\omega,zH)||D_{H}^{g}(vh)(\omega,zH)|dm_{\widehat{G}}(\omega)dm_{G/H}(zH)$
$\displaystyle\leq\|D_{H}^{g}(uf)\|_{L^{\infty}(\widehat{G}\times
G/H)}\|D_{H}^{g}(vh)\|_{L^{r^{\prime}}(\widehat{G}\times
G/H)}\|\sigma\|_{L^{r}(\widehat{G}\times G/H)}.$

By Equation (8)

$\|D_{H}^{g}(uf)\|_{L^{\infty}(\widehat{G}\times
G/H)}\leq\|g\|_{L^{\infty}(G/H)}\|u\|_{L^{r}(G)}\|f\|_{L^{r^{\prime}}(G)}.$

By Equation (13)

$\|D_{H}^{g}(vh)\|_{L^{r^{\prime}}(\widehat{G}\times
G/H)}\leq\|g\|_{L^{r^{\prime}}(G/H)}\|v\|_{L^{\infty}(G)}\|h\|_{L^{r}(G)}.$

So

$\displaystyle|\langle P_{u,v,g}(\sigma)(f),h\rangle_{L^{2}(G)}|$
$\displaystyle\leq\sigma\|_{L^{r}(\widehat{G}\times
G/H)}\|g\|_{L^{\infty}(G/H)}\|u\|_{L^{r}(G)}\|f\|_{L^{r^{\prime}}(G)}\|g\|_{L^{r^{\prime}}(G/H)}\|v\|_{L^{\infty}(G)}\|h\|_{L^{r}(G)}.$

Thus

$\|P_{u,v,g}(\sigma)\|_{B(L^{r^{\prime}}(G))}\leq\sigma\|_{L^{r}(\widehat{G}\times
G/H)}\|g\|_{L^{\infty}(G/H)}\|u\|_{L^{r}(G)}\|g\|_{L^{r^{\prime}}(G/H)}\|v\|_{L^{\infty}(G)}.$

∎

##  6\. Compactness of generalized two wavelet multipliers

Our aim is to show the linear operators

$P_{u,v,g}(\sigma):L^{p}(G)\rightarrow L^{p}(G)$

are compact operators for all symbol $\sigma$ in $L^{1}(\widehat{G}\times
G/H)$.

######  Proposition 6.1.

Under the same hypothesis of Theorem 5.3, the generalized two wavelet
multiplier $P_{u,v,g}(\sigma):L^{1}(G)\rightarrow L^{1}(G)$ is compact.

###### Proof.

Let $f_{n}\in L^{1}(G)$ s.t. $f_{n}\rightarrow 0$ weakly in $L^{1}(G)$. We
need to show, $P_{u,v,g}(\sigma)(f_{n})\rightarrow 0$ in $L^{1}(G)$.

$\|P_{u,v,g}(\sigma)(f_{n})\|_{L^{1}(G)}\leq\int_{\widehat{G}\times
G/H}\int_{G}|\sigma(\omega,zH)||\langle
f_{n},g_{\omega,zH}u\rangle_{L^{2}(G)}||g_{\omega,zH}(t)v(t)|dm_{\widehat{G}}(\omega)dm_{G/H}(zH).$

Now

$\sigma(\omega,zH)||\langle
f_{n},g_{\omega,zH}u\rangle_{L^{2}(G)}||g_{\omega,zH}(t)v(t)|\leq
C\|g\|_{L^{\infty}(G/H)}^{2}\sigma(\omega,zH)\|u\|_{L^{\infty}(G)}|v(t)|.$

By Dominated convergence theorem we conclude

$\lim_{n\rightarrow\infty}\|P_{u,v,g}(\sigma)f_{n}\|_{L^{1}(G)}=0$

∎

######  Theorem 6.2.

Under the hypothesis of Theorem 5.3, the bounded operator

$P_{u,v,g}(\sigma):L^{p}(G)\rightarrow L^{p}(G)$

is compact for $1\leq p\leq\infty$.

###### Proof.

We know $P_{u,v,g}(\sigma):L^{\infty}(G)\rightarrow L^{\infty}(G)$ is the
adjoint operator of $P_{u,v,g}(\sigma):L^{1}(G)\rightarrow L^{1}(G)$, which is
compact by Proposition 6.1. Hence by interpolation operator theorem for
compactness on $L^{1}(G)$ and on $L^{\infty}(G)$ can be extended to
compactness of $P_{u,v,g}(\sigma):L^{p}(G)\rightarrow L^{p}(G)$ for
$1<p<\infty$. ∎

##  7\. Landau-Pollak-Slepian operator

Suppose $C_{i},D$ and $\Omega$ are a compact neighbourhood of identity
elements of $G$, $G/H$ and $\widehat{G}$ respectively for $i=1,2$. Define the
operators $Q_{R}$ and $P_{R_{i}}$ as

$\displaystyle Q_{R}:L^{2}(G/H\times\widehat{G})\rightarrow
L^{2}(G/H\times\widehat{G})\ \ \textrm{and}\ \
P_{R_{i}}:L^{2}(G/H\times\widehat{G})\rightarrow L^{2}(G/H\times\widehat{G})$

by

(18) $\displaystyle
Q_{R}g(xH,\omega)=\chi_{D\times\Omega}(xH,\Omega)g(xH,\Omega)$

and

(19) $\displaystyle
P_{R_{i}}g(xH,\omega)=\mathcal{D}_{H}^{g}\left(\chi_{C_{i}}(x)(\mathcal{D}_{H}^{g})^{-1}g(xH,\omega)\right).$

Now we present some properties of the above operator in the next proposition.

######  Proposition 7.1.

The operators $Q_{R}$ and $P_{R_{i}}$ are self-adjoint projections.

###### Proof.

For $h,\phi\in G/H\times\widehat{G}$, we have

$\displaystyle\langle
P_{R_{i}}h,\phi\rangle=\langle\mathcal{D}_{H}^{g}\left(\chi_{C_{i}}(x)(\mathcal{D}_{H}^{g})^{-1}h\right),\phi\rangle$

Now we can write the following in view of Theorem 2.3

$\displaystyle\langle P_{R_{i}}h,\phi\rangle=$
$\displaystyle\langle\chi_{C_{i}}(x)(\mathcal{D}_{H}^{g})^{-1}h,(\mathcal{D}_{H}^{g})^{-1}\phi\rangle\
\ \|g\|^{2}_{L^{2}(G/H)}$ $\displaystyle=$
$\displaystyle\langle(\mathcal{D}_{H}^{g})^{-1}h,\chi_{C_{i}}(x)(\mathcal{D}_{H}^{g})^{-1}\phi\rangle\
\ \|g\|^{2}_{L^{2}(G/H)}$ $\displaystyle=$ $\displaystyle\langle
h,\mathcal{D}_{H}^{g}\left(\chi_{C_{i}}(x)(\mathcal{D}_{H}^{g})^{-1}\phi\right)\rangle$
$\displaystyle=$ $\displaystyle\langle h,P_{R_{i}}\phi\rangle.$

Hence the operator $P_{R_{i}}$ is the self-adjoint operator. Also $P_{R_{i}}$
is the projection operator by noting

$\displaystyle\langle P_{R_{i}}^{2}h,\phi\rangle$ $\displaystyle=\langle
P_{R_{i}}h,P_{R_{i}}\phi\rangle$
$\displaystyle=\langle\mathcal{D}_{H}^{g}\left(\chi_{C_{i}}(x)(\mathcal{D}_{H}^{g})^{-1}h\right),\mathcal{D}_{H}^{g}\left(\chi_{C_{i}}(x)(\mathcal{D}_{H}^{g})^{-1}\phi\right)\rangle$
$\displaystyle=\|g\|^{2}_{L^{2}(G/H)}\langle\chi_{C_{i}}(x)(\mathcal{D}_{H}^{g})^{-1}h,\chi_{C_{i}}(x)(\mathcal{D}_{H}^{g})^{-1}\phi\rangle$
$\displaystyle=\|g\|^{2}_{L^{2}(G/H)}\langle\chi_{C_{i}}(x)(\mathcal{D}_{H}^{g})^{-1}h,(\mathcal{D}_{H}^{g})^{-1}\phi\rangle$
$\displaystyle=\langle\mathcal{D}_{H}^{g}\left(\chi_{C_{i}}(x)(\mathcal{D}_{H}^{g})^{-1}h\right),\phi\rangle$
$\displaystyle=\langle P_{R_{i}}h,\phi\rangle.$

Similarly, $Q_{R}$ is also self-adjoint projection operator. ∎

The linear operator
$P_{R_{2}}Q_{R}P_{R_{1}}:L^{2}(G/H\times\widehat{G})\rightarrow
L^{2}(G/H\times\widehat{G})$ is called the generalized Landau-Pollak-Slepian
operator. Now we will obtain the relation between the generalized Landau-
Pollak-Slepian operator and the generalized two-wavelet multiplier.

######  Theorem 7.2.

Let $u$ and $v$ are the functions of $G$ defined by

$\displaystyle u=\frac{1}{\sqrt{|C_{1}|}}\chi_{C_{1}}(x)\ \ \ \textrm{and}\ \
\ v=\frac{1}{\sqrt{|C_{2}|}}\chi_{C_{2}}(x)$

then the generalized Landau-Pollak-Slepian operator

$P_{R_{2}}Q_{R}P_{R_{1}}:L^{2}(G/H\times\widehat{G})\rightarrow
L^{2}(G/h\times\widehat{G})$

is unitary equivalent to a scalar multiple of the generalized two-wavelet
multiplier

$P_{u,v,g}\left(\chi_{D\times\Omega}\right):L^{2}(G)\rightarrow L^{2}(G).$

In fact

$P_{R_{2}}Q_{R}P_{R_{1}}=\frac{\alpha(R_{1},R_{2})}{\|g\|_{L^{2}(G/H)}^{2}}\
\mathcal{D}_{H}^{g}\left(P_{u,v,g}\left(\chi_{D\times\Omega}\right)\right){\mathcal{D}_{H}^{g}}^{-1}$

where $\alpha(R_{1},R_{2})=\sqrt{|C_{1}||C_{2}|}$, where $|C_{i}|$ is the Haar
measure of the open set $C_{i}$.

###### Proof.

Clearly, $\|u\|_{L^{2}(G)}=1=\|v\|_{L^{2}(G)}$. We have

$\displaystyle\langle
P_{u,v,g}\left(\chi_{D\times\Omega}\right)f_{1},f_{2}\rangle_{L^{2}(G)}=\int_{\chi_{D\times\Omega}}D_{H}^{g}(uf_{1})(\omega,zH)\overline{D_{H}^{g}(vf_{2})(\omega,zH)}dm_{\widehat{G}}(\omega)dm_{G/H}(zH),$

where

$\displaystyle D_{H}^{g}(uf_{1})$
$\displaystyle=\int_{G}u(x)f_{1}(x)\overline{w(x)g(z^{-1}xH)}dm_{G}(x)$
$\displaystyle=\frac{1}{\sqrt{|C_{1}|}}\int_{C_{1}}f_{1}(x)\overline{w(x)g(z^{-1}xH)}dm_{G}(x)$
$\displaystyle=\frac{1}{\sqrt{|C_{1}|}}P_{R_{1}}\left(D_{H}^{g}(f_{1})\right).$

Hence we can write

$\displaystyle\langle
P_{u,v,g}\left(\chi_{D\times\Omega}\right)f_{1},f_{2}\rangle_{L^{2}(G)}$
$\displaystyle=\frac{1}{\sqrt{|C_{1}C_{2}|}}\int_{\chi_{D\times\Omega}}P_{R_{1}}\left(D_{H}^{g}(f_{1})\right)\overline{P_{R_{2}}\left(D_{H}^{g}(f_{2})\right)}dm_{\widehat{G}}(\omega)dm_{G/H}(zH)$
$\displaystyle=\frac{1}{\sqrt{|C_{1}C_{2}|}}\int_{G/H\times\widehat{G}}Q_{R}P_{R_{1}}\left(D_{H}^{g}(f_{1})\right)\overline{P_{R_{2}}\left(D_{H}^{g}(f_{2})\right)}dm_{\widehat{G}}(\omega)dm_{G/H}(zH)$
$\displaystyle=\frac{1}{\sqrt{|C_{1}C_{2}|}}\langle
Q_{R}P_{R_{1}}\left(D_{H}^{g}(f_{1})\right),P_{R_{2}}\left(D_{H}^{g}(f_{2})\right)\rangle_{L^{2}(G/H\times\widehat{G})}$
$\displaystyle=\frac{1}{\sqrt{|C_{1}C_{2}|}}\langle
P_{R_{2}}Q_{R}P_{R_{1}}\left(D_{H}^{g}(f_{1})\right),\left(D_{H}^{g}(f_{2})\right)\rangle_{L^{2}(G/H\times\widehat{G})}$
$\displaystyle=\frac{\|g\|_{L^{2}(G/H)}^{2}}{\sqrt{|C_{1}C_{2}|}}\langle\left(D_{H}^{g}\right)^{-1}P_{R_{2}}Q_{R}P_{R_{1}}\left(D_{H}^{g}(f_{1})\right),f_{2}\rangle_{L^{2}(G/H\times\widehat{G})}$

∎

## References

  * [1] Hatem Mejjaoli and Slim Omri, Spectral theorems associated with the directional short-time Fourier transform, _Journal of Pseudo-Differential Operators and Applications_ , 11.1 (2020), pp. 15–54.
  * [2] Edwin Hewitt and Kenneth A. Ross, Abstract Harmonic Analysis. Volume II: Structure and analysis for compact groups. Analysis on locally compact Abelian groups, 152 (1997).
  * [3] Hossein Hosseini Giv, Directional short-time Fourier transform, _Journal of Mathematical Analysis and Applications_ , 399.1 (2013), pp. 100–107.
  * [4] E. J. Candès, Ridgelets: theory and applications, _(Doctoral dissertation, Stanford University)_ , 1998.
  * [5] E. J. Candès and D. L. Donoho, New tight frames of curvelets and optimal representations of objects with piecewise $C^{2}$ singularities, Communications on Pure and Applied Mathematics, 57 (2004), pp. 219–266.
  * [6] E. J. Candes and D. L. Donoho, Continuous curvelet transform: I. Resolution of the wavefront set, Applied and Computational Harmonic Analysis, 19 (2005), pp. 162–197.
  * [7] G. B. Folland, Introduction to Partial Differential Equations, _2nd edn. Princeton University Press, Cambridge_(1995).
  * [8] G. Kutyniok and D. Labate, Resolution of the wavefront set using continuous shearlets, Transactions of the American Mathematical Society, 361 (2009), pp. 2719–2754.
  * [9] Bin Han, Swaraj Paul and Niraj K. Shukla, Microlocal analysis and characterization of Sobolev wavefront sets using shearlets, _Constructive Approximation_ (2021).
  * [10] Karlheinz Gröchenig, Foundations of time-frequency analysis, _Springer Science & Business Media_, 2001.
  * [11] Loukas Grafakos and Christopher Sansing, Gabor frames and directional time–frequency analysis, _Applied and Computational Harmonic Analysis_ , 25(1) (2008), pp. 47–67.
  * [12] M. W. Wong, Wavelet transforms and localization operators, vol.136, _Springer, New York_ ,(2002).