111The research of G. Ólafsson was partially supported by Simons grant 586106.

# Analytic continuation of Toeplitz operators and commuting families of
$C^{*}-$algebras

Khalid Bdarneh  Department of Mathematics, University of Jordan, Amman, Jordan
<EMAIL_ADDRESS>and  Gestur Ólafsson  Department of Mathematics, Louisiana
State University, Baton Rouge, LA 70803, USA<EMAIL_ADDRESS>
###### Abstract.

We consider the Toeplitz operators on the weighted Bergman spaces over the
unit ball $\mathbb{B}^{n}$ and their analytic continuation. We proved the
commutativity of the $C^{*}-$algebras generated by the analytic continuation
of Toeplitz operators with a special class of symbols that satisfy an
invariant property, and we showed that these commutative $C^{*}-$algebras with
symbols invariant under compact subgroups of $SU(n,1)$ are completely
characterized in terms of restriction to multiplicity free representations.
Moreover, we extended the restriction principal to the analytic continuation
case for suitable maximal abelian subgroups of $SU(n,1)$, we obtained the
generalized Segal-Bargmann transform and we showed that it acts as a
convolution operator. Furthermore, we proved that Toeplitz operators are
unitarly equivalent to a convolution operator and we provided integral
formulas for their spectra.

##  1\. introduction

Commutative $C^{*}-$algebras provide a framework for studying spectral
properties of operators. The Gelfand-Naimark theorem states that every
commutative $C^{*}-$algebra is $*-$ isomorphic to $C_{0}(X)$ for some locally
compact Hausdorff space $X$.

One interesting problem is to find families of commutative $C^{*}-$algebras
generated by Toeplitz operators. The algebra of Toeplitz operators is dense in
the algebra of bounded operators under the strong operator topology. However,
restricting the class of symbols ensures the existence of rich commutative
families of Toeplitz operators. For example, consider the Bergman space on the
unit disc $\mathbb{D}$, the $C^{*}-$algebra generated by Toeplitz operators
with essentially bounded symbols is commutative if and only if there is a
pencil of hyperbolic geodesics such that the symbols of the Toeplitz operators
are constant on the cycles of this pencil [11]. In other words, we need the
symbols to be invariant under the maximal abelian subgroups of
$\mathrm{Aut}(\mathbb{D})$. This result was extended for the unit ball
$\mathbb{B}^{n}$ for the case $n>1$ in [17]. In particular, the
$C^{*}-$algebra generated by Toeplitz operators with essentially bounded
symbols that are invariant under maximal abelian subgroups of
$\mathrm{Aut}(\mathbb{B}^{n})$ is commutative. Moreover, the authors in [4]
used tools from representation theory to extend this result to bounded
symmetric domains.

In this work we will construct commuting families of $C^{*}-$algebras
generated by the analytic continuation of the Toeplitz operators, and we will
develop the restriction principal for suitable maximal abelian subgroups of
$SU(n,1)$ which will allow us to find the spectral decomposition of the
analytic continuation of the Toeplitz operators.

Let $\mathbb{B}^{n}$ be the unit ball in $\mathbb{C}^{n}$, and consider the
weighted Bergman space

$A_{\lambda}^{2}(\mathbb{B}^{n})=\\{f\in
L^{2}(\mathbb{B}^{n},d\mu_{\lambda})\mid f\text{ is holomorphic on
}\mathbb{B}^{n}\\}$

with respect to the probability measure

$d\mu_{\lambda}(z)=\dfrac{\Gamma(\lambda)}{\pi^{n}\Gamma(\lambda-n)}(1-|z|^{2})^{\lambda-n-1}dz,\quad\lambda>n$

the Toeplitz operator is the operator

$\displaystyle T_{\varphi}^{(\lambda)}\mathrel{\mathop{\mathchar 58\relax}}$
$\displaystyle\,A_{\lambda}^{2}(\mathbb{B}^{n})\rightarrow
A_{\lambda}^{2}(\mathbb{B}^{n})$ $\displaystyle
T_{\varphi}^{(\lambda)}=P_{\lambda}(\varphi f)$

where $P_{\lambda}\mathrel{\mathop{\mathchar
58\relax}}L^{2}(\mathbb{B}^{n},d{\mu_{\lambda})\rightarrow
A_{\lambda}^{2}(\mathbb{B}^{n})}$ is the orthogonal projection

$P_{\lambda}f(x)=\int_{\mathbb{B}^{n}}f(y)K_{\lambda}(x,y)d\mu_{\lambda}(y)$

This definition of the Toeplitz operator works when $\lambda>n$. However, the
weighted Bergman spaces $A_{\lambda}^{2}(\mathbb{B}^{n})$ reduces to $\\{0\\}$
when we consider $\lambda\leq n$. The kernel of
$A_{\lambda}^{2}(\mathbb{B}^{n})$ is given by

$K_{\lambda}(z,w)=(1-z.\bar{w})^{-\lambda}$

So, a natural question would be, for which values of $\lambda$ is
$K_{\lambda}$ a reproducing kernel for a Hilbert space of holomorphic
functions on $\mathbb{B}^{n}$? We will denote this set by $W$. A half line
contained in $W$ was found by Vergne and Rossi [21]. However, using algebraic
methods Wallach determined the entire set $W$ by showing it also contains a
discrete set of points [22].

The authors in [3] constructed Hilbert spaces of holomorphic functions with
reproducing kernel

$K_{\lambda}(z,w)=(1-z.\bar{w})^{-\lambda}$

with weights $\lambda>0$, and they proved these spaces coincide with the
weighted Bergman spaces when we restrict the values of $\lambda$ to
$\lambda>n$.

In [7] the authors generalized the work of Chailuek and Hall that appeared in
[3], and proved the existence of the analytic continuation of weighted Bergman
spaces to include all weights $\lambda$ in $\mathbb{R}-V$ where $V$ a discrete
set. These spaces are Sobolev spaces of holomorphic functions of certain
order. Moreover, they proved the existence and uniqueness of the analytic
continuation of the Toeplitz operators for certain classes of symbols.

##  2\. Analytic continuation of Toeplitz operators

The definition of weighted Bergman spaces were extended to include the weights
$\lambda>0$, where these spaces are Sobolev spaces of holomorphic functions.
The construction of theses spaces was introduced in [3], we will present a
brief description of these spaces.

Consider the Euler operator

$N\mathrel{\mathop{\mathchar
58\relax}}=\sum_{j=1}^{n}z_{j}\dfrac{\partial}{\partial z_{j}}$

then $Nz^{r}=|r|z^{r}$ for all multi-indices $r$. For $\lambda>0$, choose a
non-negative integer $m$ such that $\lambda+2m>n$. Denote by
$\mathcal{O}(\mathbb{B}^{n})$ the space of holomorphic functions on the unit
ball. Define the space

$A_{\lambda,\\#}^{2}(\mathbb{B}^{n})\mathrel{\mathop{\mathchar
58\relax}}=\\{f\in\mathcal{O}(\mathbb{B}^{n})\mid N^{k}f\in
A_{\lambda}^{2}(\mathbb{B}^{n},d\mu_{\lambda+2m})\text{ for every }0\leq k\leq
m\,\\}$

This space with inner product defined by

$<f,g>_{\lambda}\mathrel{\mathop{\mathchar 58\relax}}=<Af,Bg>_{\lambda+2m}$

is a reproducing kernel Hilbert space, and the kernel is given by

$K_{\lambda}(x,y)=(1-<x,y>)^{-\lambda}$

The monomials $\\{z^{r}\\}$ forms an orthogonal basis for the space
$A_{\lambda,\\#}^{2}(\mathbb{B}^{n})$, and direct calculations show that

$<z^{r},z^{s}>_{\lambda,\\#}\,=\,\delta(r,s)\dfrac{r!\,\Gamma(\lambda)}{\Gamma(\lambda+|r|)}$

where $\delta(r,s)=1$ when $r=s$, and zero otherwise.

######  Definition 2.1.

[7] Assume for $f\in\mathcal{O}(\overline{\mathbb{B}^{n}})$ the function
$F_{\lambda,f}(z)\mathrel{\mathop{\mathchar
58\relax}}=T_{\varphi}^{(\lambda)}f(z)$ can be analytically continued from
$\lambda>n$ to some neighborhood of $\lambda_{0}>0$, and the operator
$T\mathrel{\mathop{\mathchar 58\relax}}f\mapsto F_{\lambda,f}$ extends to a
bounded operator on $A_{\lambda_{0},\\#}^{2}(\mathbb{B}^{n})$, then we say the
Toeplitz operator $T_{\varphi}^{(\lambda_{0})}$ exists and we set
$T=T_{\varphi}^{(\lambda_{0})}$.

Let $Z\mathrel{\mathop{\mathchar
58\relax}}=\\{\lambda\in\mathbb{C}-\\{n,n-1,n-2,n-3,...\\}\mid
C(\lambda)=0\\}$, where

$C(\lambda)\mathrel{\mathop{\mathchar
58\relax}}=\int_{\mathbb{B}^{n}}(1-|z|^{2})^{\lambda-n-1}dz$

For $k=0,1,2,...$ define the space

$BC^{k}(\mathbb{B}^{n})\mathrel{\mathop{\mathchar 58\relax}}=\\{f\in
C^{k}(\Omega)\mathrel{\mathop{\mathchar
58\relax}}\partial^{\nu}\overline{\partial^{\mu}}f\in L^{\infty}(\Omega)\text{
for every }|\nu|+|\mu|\leq k\\}$

The following theorem shows that the existence of the analytic continuation of
Toeplitz operators is guaranteed when the symbol $\varphi$ is in
$BC^{k}(\Omega)$.

######  Theorem 2.1 ([7]).

Let $k\geq 0$ be an integer. If $\varphi\in BC^{k}(\mathbb{B}^{n})$, then
$T_{\varphi}^{(\lambda)}$ exists for all
$\lambda\in(n-k,\infty)-(Z\cup\\{-2,-3,-4,...\\})$.

From now on, when we write $\lambda>0$, we mean
$\lambda\in\mathbb{R}^{+}-\\{n,n-1,n-2,...,0\\}$.

##  3\. $C^{*}$-algebra Generated by Toeplitz Operators

Representation theory can be used to construct commuting families of
$C^{*}-$algebras generated by Toeplitz operators. For example, it was shown in
[4] that certain multiplicity free representations corresponds to commuting
families of Toeplitz operators with symbols admitting an invariant property.
In particular, if $G$ is a Lie group of type I and $H\subset G$ is a compact
subgroup, then the restriction of the holomorphic discrete series
representation $\pi_{\lambda}|_{H}$ is multiplicity free if and only if the
family of Toeplitz operators over $A_{\lambda}^{2}(\mathbb{B}^{n})$ with
$H$-invariant symbols is commutative [4, Theorem 6.4]. In this section we will
construct commuting families of $C^{*}-$algebras, these families are generated
by the analytic continuation of Toeplitz operators. In particular, we will
extend [5, Theorem 2] to include the analytic continuation case. Moreover, we
will prove a density theorem for the $C^{*}-$algebra generated by the analytic
continuation of Toeplitz operators.

Let $G\mathrel{\mathop{\mathchar 58\relax}}=SU(n,1)$, and $H$ be a subgroup of
$G$. For $\lambda>0$, we consider the unitary representation of the group
$SU(n,1)$ on $A_{\lambda}^{2}(\mathbb{B}^{n})$, defined by

$\pi_{\lambda}(g)f(z)=j_{\lambda}(g^{-1},z)f(g^{-1}.z)$

where

$j_{\lambda}\mathrel{\mathop{\mathchar
58\relax}}SU(n,1)\times\mathbb{B}^{n}\rightarrow\mathbb{C}$
$j_{\lambda}\left(\begin{pmatrix}a&v\\\
w^{t}&d\end{pmatrix},z\right)=(w^{t}z+d)^{-\lambda}$

The map $j_{\lambda}$ is lifted to $\widetilde{SU(n,1)}\times\mathbb{B}^{n}$
to make sure it is well defined for all $\lambda>0$.

######  Proposition 3.1.

Let $\mathcal{L}(A_{\lambda}^{2}(\mathbb{B}^{n}))$ be the space of bounded
linear operators on the weighted Bergman space
$A_{\lambda}^{2}(\mathbb{B}^{n})$. For every $\lambda>0$, the map

$\displaystyle BC^{n}(\mathbb{B}^{n})$
$\displaystyle\rightarrow\mathcal{L}(A_{\lambda}^{2}(\mathbb{B}^{n}))$
$\displaystyle\varphi\longmapsto T_{\varphi}^{(\lambda)}$

is injective.

###### Proof.

Fix $\lambda>0$. Suppose $\varphi\in BC^{n}(\mathbb{B}^{n})$ and
$T_{\varphi}^{(\lambda)}=0$, then
$<T_{\varphi}^{(\lambda)}z^{n},z^{m}>_{\lambda}=0$ for every monomials $z^{n}$
and $z^{m}$. There exists $\varphi_{\epsilon}\in C^{\infty}(\mathbb{B}^{n})$
such that
$\mathinner{\\!\left\lVert\varphi_{\epsilon}\right\rVert}_{\infty}\leq\mathinner{\\!\left\lVert\varphi\right\rVert}_{\infty}$,
and $\varphi_{\epsilon}\rightarrow\varphi$ pointwise as $\epsilon\searrow 0$ .
Therefore $T_{\varphi_{\epsilon}}^{(\lambda)}\rightarrow
T_{\varphi}^{(\lambda)}$ in the weak operator topology. Since
$\varphi_{\epsilon}$ is holomorphic, then $T_{\varphi_{\epsilon}}^{(\lambda)}$
is the multiplication operator. So we have:

$\displaystyle 0=<T_{\varphi}^{(\lambda)}z^{n},z^{m}>_{\lambda}=$
$\displaystyle\lim_{\epsilon\rightarrow
0}<T_{\varphi_{\epsilon}}^{(\lambda)}z^{n},z^{m}>_{\lambda}$
$\displaystyle=\lim_{\epsilon\rightarrow
0}<\varphi_{\epsilon}z^{n},z^{m}>_{\lambda}$ $\displaystyle=<\varphi
z^{n},z^{m}>_{\lambda}$

which means $\varphi$ is orthogonal to the monomials on $\mathbb{B}^{n}$.
Therefore, $\varphi=0$ ∎

Let $\varphi\in BC^{n}(\mathbb{B}^{n})$ and $h\in\widetilde{SU(n,1)}$, define
$\varphi_{h}(z)\mathrel{\mathop{\mathchar 58\relax}}=\varphi(h^{-1}z)$. Let
$H$ be a subgroup of $\widetilde{SU(n,1)}$, we say $\varphi$ is $H$-invariant
if $\varphi_{h}(z)=\varphi(z)$ for every $h\in H$, and for a.e
$z\in\mathbb{B}^{n}$.

The following proposition is the cornerstone in finding commuting families of
Toeplitz operators. In particular, we need the Toeplitz operators to commute
with the representation $\pi_{\lambda}$, and this can be done by restricting
the representation $\pi_{\lambda}$ to suitable subgroup of $\widetilde{G}$.
The proof of the following proposition follows from [4] in the case where
$\lambda>n$, and the uniqueness of the analytic continuation for the case
$\lambda>0$.

######  Proposition 3.2.

Let $\lambda>0$. For every Toeplitz operator $T_{\varphi}^{(\lambda)}$ we have

$\pi_{\lambda}(g)\circ
T_{\varphi}^{(\lambda)}=T_{\varphi_{g}}^{(\lambda)}\circ\pi_{\lambda}(g)$

######  Corollary 3.1.

Let $\widetilde{H}$ be a subgroup of $\widetilde{SU(n,1)}$. The symbol
$\varphi$ is $\widetilde{H}$-invariant if and only if
$T_{\varphi}^{(\lambda)}$ intertwine with $\pi_{\lambda}|_{\widetilde{H}}$

Multiplicity free representations corresponds to commuting families of
Toeplitz operators, we will show this also remains true when we consider the
analytic continuation case. Let $H$ be a Lie group of type I, and
$\pi\mathrel{\mathop{\mathchar 58\relax}}H\rightarrow\mathcal{L(H)}$ be a
unitary representation of $H$ on a Hilbert space $\mathcal{H}$. The
representation $\pi$ admits a unique direct integral decomposition (up to
unitary equivalence)

$\pi\simeq\int_{\widehat{H}}^{\bigoplus}m_{\pi}(\tau)\tau d\mu(\tau)$

where $\widehat{H}$ is the set of equivalence classes of irreducible unitary
representation of $H$, $\mu$ is a Borel measure on $\widehat{H}$, and
$m_{\pi}\mathrel{\mathop{\mathchar
58\relax}}\widehat{H}\rightarrow\mathbb{N}\cup\\{\infty\\}$ is the
multiplicity function [13]. A unitary representation $\pi$ is called
multiplicity free if $m_{\pi}(\tau)\leq 1$ for almost all
$\tau\in\widehat{H}$.

Let $\mathrm{End}_{H}(\mathcal{H})$ be the set of operators that intertwine
with the representation $\pi$, the ring $\mathrm{End}_{H}(\mathcal{H})$ is
commutative if and only if the representation $\pi$ is multiplicity free [13].
Let ${H}$ be a subgroup of ${SU(n,1)}$, and $BC^{n}(\mathbb{B}^{n})^{{H}}$ be
the set of symbols in $BC^{n}(\mathbb{B})^{n}$ that are ${H}$-invariant. We
will use $\mathcal{T}^{(\lambda)}(BC^{n}(\mathbb{B}^{n})^{H})$ to denote the
$C^{*}$-algebra generated by Toeplitz operators with $H-$ invariant symbols on
the Hilbert space $A_{\lambda}^{2}(\mathbb{B}^{n})$.

######  Lemma 3.1 ([7]).

Let $\Omega$ be a domain in $\mathbb{C}^{n}$ and $F(x,y)$ be defined on
$\Omega\times\bar{\Omega}$ such that $F$ is holomorphic in first variable and
anti-holomorphic in the second variable. If $F(x,\bar{x})=0$ for every $x$ in
$\Omega$, then $F$ is identically zero on $\Omega\times\bar{\Omega}$.

The following lemma appeared in the proof of Theorem 2 in [7]. We will need it
to show that the space of Toeplitz operators with $H-$invariant symbols is
dense in $\mathrm{End}_{\widetilde{H}}(A_{\lambda}^{2}(\mathbb{B}^{n}))$.

######  Lemma 3.2.

Let $q$ and $r$ be integers, and $W$ be a finite dimensional subspace of
$A_{\lambda}^{2}(\mathbb{B}^{n})$ of dimension $qr$. Assume $W$ has basis
$\\{f_{1},...,f_{q},g_{1},...,g_{r}\\}$. If $u\in\mathbb{C}^{q\times r}$ is
such that

$\sum_{j}\sum_{i}<T_{\varphi}^{(\lambda)}f_{i},g_{j}>u_{ij}=0$

for every $\varphi\in BC^{n}(\mathbb{B}^{n})$, then $u=0$.

###### Proof.

Suppose we have $u\in\mathbb{C}^{q\times r}$ such that

$\sum_{j}\sum_{i}<T_{\varphi}^{(\lambda)}f_{i},g_{j}>u_{ij}=0$

for every $\varphi$. Then

$\int_{\Omega}\varphi(z)\sum_{j}\sum_{i}u_{ij}f_{i}(z)\overline{g_{j}(z)}.d\mu(z)=0$

for all $\varphi\in BC^{n}(\mathbb{B}^{n})$. Therefore, we have

(3.1) $\sum_{j}\sum_{i}u_{ij}f_{i}(z)\overline{g_{j}(z)}=0$

almost everywhere on $\Omega$. Since the left hand side of equation 3.1 is
continuous, then the equation holds everywhere on $\Omega$. Define the
function

$F(x,y)=\sum_{j}\sum_{i}u_{ij}f_{i}(x)\overline{g_{j}(\overline{y})}$

then $F(z,\bar{z})=0$ on $\Omega$. Therefore, the function $F$ is zero on
$\Omega\times\overline{\Omega}$. Since the functions $f_{i}^{\prime}s$ and
$g_{i}^{\prime}s$ are linearly independent, then $u_{ij}=0$ for every $i,j$. ∎

For $\lambda>n$, the $C^{*}-$algebra
$\mathcal{T}^{(\lambda)}(BC^{n}(\mathbb{B}^{n})^{{H}})$ generated by Toeplitz
operators with $H$\- invariant symbols, is dense in
$\mathrm{End}_{\widetilde{H}}(A_{\lambda}^{2}(\mathbb{B}^{n}))$ under the
strong operator topology, the following proposition shows that this density
result still holds for the analytic continuation of Toeplitz operators with
$H$\- invariant symbols, the proof is close to the proof of [7, Theorem 2]. If
$F(\lambda,x)$ is a function that depends on $\lambda$, then we will use the
notation $\mathrm{An}_{\lambda_{0}}(F)$ to denote the analytic continuation of
the function $F$ to a neighborhood of $\lambda_{0}$.

######  Proposition 3.3.

Let $H$ be a compact subgroup of $SU(n,1)$. For every $\lambda>0$, the space
$\mathcal{T}^{(\lambda)}(BC^{n}(\mathbb{B}^{n})^{H})$ is dense in
$\mathrm{End}_{\widetilde{H}}(A_{\lambda}^{2}(\mathbb{B}^{n}))$ the space of
$\widetilde{H}$-intertwining operators on $A_{\lambda}^{2}(\mathbb{B}^{n})$
under the strong operator topology.

###### Proof.

Since $H$ is compact, then $A_{\lambda}^{2}(\mathbb{B}^{n})$ decomposes into
an orthogonal finite dimensional $\widetilde{H}-$invariant subspaces. Let
$\lambda>0$ and $W$ be a $\widetilde{H}-$invariant finite dimensional subspace
of $A_{\lambda_{0}}^{2}(\mathbb{B}^{n})$ with basis
$f_{1},...,f_{q},g_{1},...,g_{r}$. Let $T$ be a bounded operator on
$A_{\lambda_{0}}^{2}(\mathbb{B}^{n})$, we will show that there exists
$\varphi\in BC^{n}(\mathbb{B}^{n})$ such that

$<Tf_{i},g_{j}>\,\,=\,\,<T_{\varphi}^{(\lambda_{0})}f_{i},g_{j}>$

Define the operator

$R\mathrel{\mathop{\mathchar
58\relax}}BC^{n}(\mathbb{B}^{n})\rightarrow\mathbb{C}^{q\times r}$
$(R\varphi)_{ij}\mathrel{\mathop{\mathchar
58\relax}}=\,\,<T_{\varphi}^{(\lambda_{0})}f_{i},g_{j}>$

which can be written as

$(R\varphi)_{ij}=\mathrm{An}_{\lambda_{0}}(<T_{\varphi}^{(\lambda)}f_{i},g_{j}>)$

Suppose $u\in\mathbb{C}^{q\times r}$ is orthogonal to the image of the
operator $R$, that is

$\sum_{j}\sum_{i}<T_{\varphi}^{(\lambda_{0})}f_{i},g_{j}>\overline{u_{ij}}=0$

for every $\varphi\in BC^{n}(\mathbb{B}^{n})$. So, we have

$\mathrm{An}_{\lambda_{0}}(\sum_{j}\sum_{i}<T_{\varphi}^{(\lambda)}f_{i},g_{j}>\overline{u_{ij}})=0$

which implies

$\sum_{j}\sum_{i}<T_{\varphi}^{(\lambda)}f_{i},g_{j}>\overline{u_{ij}}=0$

Therefore, by Lemma 3.2 we have $u_{ij}=0$. Which means that the image of $R$
is $\mathbb{C}^{q\times r}$.

Since $W$ was an arbitrary finite dimensional $\widetilde{H}-$invariant
subspace, then $\mathcal{T}^{(\lambda)}(BC^{n}(\mathbb{B}^{n})^{{H}})$ is
dense in $\mathrm{End}_{\widetilde{H}}(A_{\lambda}^{2}(\mathbb{B}^{n}))$ in
the weak operator topology. Since
$\mathcal{T}^{(\lambda)}(BC^{n}(\mathbb{B}^{n})^{H})$ is a convex subspace of
$\mathrm{End}_{\widetilde{H}}(A_{\lambda}^{2}(\mathbb{B}^{n}))$, then
$\mathcal{T}^{(\lambda)}(BC^{n}(\mathbb{B}^{n})^{H})$ is dense in
$\mathrm{End}_{\widetilde{H}}(A_{\lambda}^{2}(\mathbb{B}^{n}))$ in the strong
operator topology. ∎

One of our main goals is to establish a connection between the commutativity
of a subspace of analytic continuation of Toeplitz operators and multiplicity
free representations. First, we need the following Lemma. The proof can be
found in [4, Lemma 6.3].

######  Lemma 3.3.

Suppose that $V$ is a linear subspace of the space $\mathcal{L}(\mathcal{H})$
of bounded linear operators on a Hilbert space $\mathcal{H}$. If $V$ consists
of operators that commutes, then the closure of $V$ in the strong operator
topology also consists of operators that commutes.

######  Theorem 3.1.

Let $\lambda>0$.

  * 1)

If $\mathrm{End}_{\widetilde{H}}(A_{\lambda}^{2}(\mathbb{B}^{n}))$ is
commutative, then $\mathcal{T}^{(\lambda)}(BC^{n}(\mathbb{B}^{n})^{H})$ is a
commutative $C^{*}$-algebra.

  * 2)

If $\widetilde{H}$ is compact, then
$\mathcal{T}^{(\lambda)}(BC^{n}(\mathbb{B}^{n})^{H})$ is commutative if and
only if $\pi_{\lambda}|_{\widetilde{H}}$ is multiplicity free.

###### Proof.

(1) Using proposition 3.2 we have every Toeplitz operator with $H-$invariant
symbol commute with the representation $\pi$. Therefore
$\mathcal{T}^{(\lambda)}(BC^{n}(\mathbb{B}^{n})^{H})\subset\mathrm{End}_{\widetilde{H}}(A_{\lambda}^{2}(\mathbb{B}^{n}))$,
and the commutativity of
$\mathrm{End}_{\widetilde{H}}(A_{\lambda}^{2}(\mathbb{B}^{n}))$ implies
$\mathcal{T}^{(\lambda)}(BC^{n}(\mathbb{B}^{n})^{H})$ is also commutative.

(2) If the $C^{*}-$algebra
$\mathcal{T}^{(\lambda)}(BC^{n}(\mathbb{B}^{n})^{H})$ is commutative, then by
proposition 3.3 and lemma 3.3, we have the space
$\mathrm{End}_{\widetilde{H}}(A_{\lambda}^{2}(\mathbb{B}^{n}))$ is
commutative. Therefore, the representation $\pi_{\lambda}|_{\widetilde{H}}$ is
multiplicity free. Conversely, if the representation
$\pi_{\lambda}|_{\widetilde{H}}$ is multiplicity free, then
$\mathrm{End}_{\widetilde{H}}(A_{\lambda}^{2}(\mathbb{B}^{n}))$ is
commutative, and the commutativity of
$\mathcal{T}^{(\lambda)}(BC^{n}(\mathbb{B}^{n})^{H})$ follows from (1). ∎

##  4\. Spectral Representation of Toeplitz Operators

On the unit ball $\mathbb{B}^{n}$, the Segal-Bargman transform was used to
derive a formula for the spectrum of the Toeplitz operators, which was given
as an integral formula in terms of the symbols [17, 18]. We will extend the
results in [5] to the case when the weight $\lambda>0$. In particular, we will
consider the class of symbols that are invariant under maximal abelian
subgroups of $SU(n,1)$, and we will show that Toeplitz operators can be
realized as a convolution operator.

###  4.1.  The Restriction Principle

Let $M\subset\mathbb{B}^{n}$ be a submanifold. If $f\mapsto f|_{M}$ is
injective on the space of holomorphic functions, then $M$ is called
restriction injective [5]. Let $H$ be a closed subgroup of
$G\mathrel{\mathop{\mathchar 58\relax}}=SU(n,1)$, and $\widetilde{H}$ be the
inverse image of $H$ in the universal covering of $G$. Assume that the
submanifold $M\mathrel{\mathop{\mathchar
58\relax}}=H.\,z_{0}=\widetilde{H}.\,z_{0}$ is restriction injective, then
there exists a measure on $M$ that is invariant under the submanifolds $H$ and
$\widetilde{H}$ [5].

######  Proposition 4.1.

[5] For $\lambda>0$, the map $\chi_{\lambda}\mathrel{\mathop{\mathchar
58\relax}}{\widetilde{H}_{z_{0}}}\rightarrow\mathbb{C}$, defined by
$\chi_{\lambda}(h)=j_{\lambda}(h,z_{0})^{-1}$ is a unitary character
satisfying

$j_{\lambda}(hk,z_{0})=j_{\lambda}(h,z_{0})\chi_{\lambda}(k)^{-1}$

for all $h\in\widetilde{H},\,k\in{\widetilde{H}_{z_{0}}}$.

As $\chi_{\lambda}$ is a character of $\widetilde{H}_{z_{0}}$, we can consider
the induced representation $\tau_{\lambda}\mathrel{\mathop{\mathchar
58\relax}}=\mathrm{Ind}_{\widetilde{H}_{z_{0}}}^{\widetilde{H}}\chi_{\lambda}$.
The Hilbert space for the induced representation is denoted by
$L_{\chi_{\lambda}}^{2}(M,d\mu)$ which consists of the functions
$f\mathrel{\mathop{\mathchar 58\relax}}\tilde{H}\rightarrow\mathbb{C}$ that
are square integrable with respect to the invariant measure $\mu$ and satisfy

(4.1) $f(kh)=\chi_{\lambda}(h)^{-1}f(k)$

for all $k\in\widetilde{H}$ and $h\in\widetilde{H}_{z_{0}}$. Moreover,
$\widetilde{H}$ acts unitarily on $L_{\chi_{\lambda}}^{2}(M,d\mu)$ by

$h.f(k)=f(h^{-1}k)$

Notice that the space $M$ is identified with
$\widetilde{H}/\widetilde{H}_{z_{0}}\cong H/H_{z_{0}}$.

There are $n+2$ conjugacy classes of maximal abelian subgroups of
$\mathrm{Aut}(\mathbb{B}^{n})$ [19], we will list their representatives in the
next section. In this section, when we write a subgroup $H$ of $SU(n,1)$ we
mean a maximal abelian subgroup that is either quasi-elliptic, quasi-parabolic
or quasi-hyperbolic. We will associate a differential operator $N$ for each
one of them.

For the quasi-elliptic abelian subgroup $E(n)$, we define the operator
$N_{E(n)}\mathrel{\mathop{\mathchar 58\relax}}=0$ to be the zero operator. For
the Quasi-parabolic subgroup$P(n)$, we define the operator
$N_{P(n)}\mathrel{\mathop{\mathchar
58\relax}}=y\dfrac{\partial}{\partial\,y}$. Finally, for the quasi-hyperbolic
subgroup $H(n)$, we define the operator

$N_{H(n)}\mathrel{\mathop{\mathchar
58\relax}}=\dfrac{\sinh\,s}{\cosh\,s}\dfrac{\partial}{\partial\,s}$

Suppose $n\geq\lambda>0$, let $m$ be the smallest non-negative integer such
that $\lambda+2m>n$. Let $H$ ba a maximal abelian subgroup of $SU(n,1)$,
define the operators:

(4.2) $\displaystyle A_{\lambda}$ $\displaystyle\mathrel{\mathop{\mathchar
58\relax}}=(I+\dfrac{N_{H}}{\lambda+2m-1})...(I+\dfrac{N_{H}}{\lambda+m})$
$\displaystyle B_{\lambda}$ $\displaystyle\mathrel{\mathop{\mathchar
58\relax}}=(I+\dfrac{N_{H}}{\lambda+m-1})...(I+\dfrac{N_{H}}{\lambda})$

the operators $A_{\lambda}$ and $B_{\lambda}$ are injective. Define the space
$\mathcal{V}_{\lambda}$ to be the space of analytic functions
$f\mathrel{\mathop{\mathchar 58\relax}}\widetilde{H}\rightarrow\mathbb{C}$
such that $A_{\lambda}B_{\lambda}\,f\in L_{\chi_{\lambda+2m}}^{2}(M,d\mu)$ and
satisfy

$f(hh_{1})=\chi_{\lambda}(h_{1})^{-1}f(h)$

for all $h\in\tilde{H}$, and $h_{1}\in\tilde{H}_{z_{0}}$. Moreover, the space
$\mathcal{V}_{\lambda}$ is equipped with the inner product

$<f,g>_{\lambda}\mathrel{\mathop{\mathchar
58\relax}}=<A_{\lambda}B_{\lambda}\,f,A_{\lambda}B_{\lambda}\,g>_{L_{\chi_{\lambda+2m}}^{2}}$

If $\lambda>n$, then we take $A_{\lambda}=B_{\lambda}=I$.

Define the map $D_{\lambda}\mathrel{\mathop{\mathchar
58\relax}}\widetilde{H}\rightarrow\mathbb{C}$ by
$D_{\lambda}(h)=j_{\lambda}(h,z_{0})$, the cocycle relation of $j_{\lambda}$
implies

$D_{\lambda}(hh_{1})=\chi_{\lambda}(h_{1})^{-1}D_{\lambda}(h)$

If we apply the operators $A_{\lambda}B_{\lambda}$ on $D_{\lambda}$, then we
have

(4.3) $A_{\lambda}B_{\lambda}\,D_{\lambda}(h)=cD_{\lambda+2m}(h)$

in the quasi-elliptic and quasi-parabolic case, where $c$ is a constant that
depends on $h$. Therefore, $D_{\lambda}$ belongs to the space
$\mathcal{V}_{\lambda}$. In fact, equation 4.3 does not hold for the quasi-
hyperbolic case, but we still have $D_{\lambda}\in\mathcal{V}_{\lambda}$ even
in this case.

Now, define the restriction operator $R_{\lambda}\mathrel{\mathop{\mathchar
58\relax}}A_{\lambda}^{2}(\mathbb{B}^{n})\rightarrow\mathcal{V}_{\lambda}$ by

$R_{\lambda}(f)(h)=D_{\lambda}(h)f|_{M}(h.z_{0})$

and let $\mathcal{S}_{\lambda}$ be the closure of
$R_{\lambda}(A_{\lambda}^{2}(\mathbb{B}^{n}))$ in $\mathcal{V}_{\lambda}$. The
point evaluation maps are continuous in $A_{\lambda}^{2}(\mathbb{B}^{n})$, it
follows that $R_{\lambda}$ is closed. Therefore, the operator
$R_{\lambda}^{*}\mathrel{\mathop{\mathchar
58\relax}}\mathcal{S}_{\lambda}\rightarrow A_{\lambda}^{2}(\mathbb{B}^{n})$ is
well-defined and

$\displaystyle R_{\lambda}^{*}f(z)=$
$\displaystyle<R_{\lambda}^{*}f,K_{z}>_{\lambda}=<f,R_{\lambda}K_{z}>_{\mathcal{S}_{\lambda}}$
$\displaystyle=$
$\displaystyle\int_{\widetilde{H}/\widetilde{H}_{z_{0}}}A_{\lambda}B_{\lambda}f(h)\,\,\overline{A_{\lambda}B_{\lambda}\Big{(}D_{\lambda}(h)K(h.z_{0},z)\Big{)}}\,d\mu(h)$

Using integration by parts, the operator $R_{\lambda}^{*}$ can be written as

$R_{\lambda}^{*}f(z)=\int_{\widetilde{H}/\widetilde{H}_{z_{0}}}f(h)\,\,Q\,\overline{A_{\lambda}B_{\lambda}\Big{(}D_{\lambda}(h)K(h.z_{0},z)\Big{)}}\,d\mu(h)$

where $Q$ is a polynomial differential operator acting on $h$.

The following proposition shows that the restriction operator $R_{\lambda}$
intertwine the representations
$\big{(}\pi_{\lambda}|_{\widetilde{H}},A_{\lambda}^{2}(\mathbb{B}^{n})\big{)}$
and $\big{(}\tau_{\lambda},\mathcal{S}_{\lambda}\big{)}$ where
$\tau_{\lambda}$ is the left regular representation on
$\mathcal{S}_{\lambda}$.

######  Proposition 4.2.

If $f\in A_{\lambda}^{2}(\mathbb{B}^{n})$, and $h,h_{1}\in\widetilde{H}$, then
$R_{\lambda}(\pi_{\lambda}(h_{1})f)(h)=(\tau_{\lambda}(h_{1})R_{\lambda}f)(h)$.

###### Proof.

Let $f\in A_{\lambda}^{2}(\mathbb{B}^{n})$, $h$ and $h_{1}\in\widetilde{H}$,
then we have

$\displaystyle R_{\lambda}(\pi_{\lambda}(h_{1})f)(h)=$ $\displaystyle
D_{\lambda}(h)(\pi_{\lambda}(h_{1})f)|_{M}(h.z_{0})$ $\displaystyle=$
$\displaystyle
D_{\lambda}(h)j_{\lambda}(h_{1}^{-1},h.z_{0})f(h_{1}^{-1}h.z_{0})$
$\displaystyle=$ $\displaystyle
j_{\lambda}(h,z_{0})j_{\lambda}(h_{1}^{-1},h.z_{0})f(h_{1}^{-1}h.z_{0})$
$\displaystyle=$ $\displaystyle
j_{\lambda}(h_{1}^{-1}h,z_{0})f(h_{1}^{-1}h.z_{0})$ $\displaystyle=$
$\displaystyle\tau_{\lambda}(h_{1})\big{(}j_{\lambda}(h,z_{0})f(h.z_{0})\big{)}$
$\displaystyle=$ $\displaystyle(\tau_{\lambda}(h_{1})R_{\lambda}f)(h)\qed$

We will need the following lemma in the next section. Following [5], for
$h,k\in\widetilde{H}$ we define the operator

$R_{\lambda}(h,k)=D_{\lambda}(h)\,Q\overline{A_{\lambda}B_{\lambda}\big{(}D_{\lambda}(k)K_{\lambda}(k.z_{0},h.z_{0})\big{)}}$

######  Lemma 4.1.

Let $f\in\mathcal{S}_{\lambda}$, then

$R_{\lambda}R_{\lambda}^{*}(f)(h)=\int_{M}f(k)R_{\lambda}(h,k)\,d\mu(k)$

###### Proof.

Let $f\in\mathcal{S}_{\lambda}$, we have

$R_{\lambda}^{*}(f)(z)=\int_{\widetilde{H}/\widetilde{H}_{z_{0}}}f(h)\,Q\,\overline{A_{\lambda}B_{\lambda}\Big{(}D_{\lambda}(h)K(h.z_{0},z)\Big{)}}\,d\mu(h)$

Therefore, $R_{\lambda}R_{\lambda}^{*}(f)$ becomes

$\displaystyle R_{\lambda}R_{\lambda}^{*}(f)(h)=$ $\displaystyle
R_{\lambda}\big{(}\int_{\widetilde{H}/{\widetilde{H}_{z}{{}_{0}}}}f(k)\,Q\,\overline{A_{\lambda}B_{\lambda}\big{(}D_{\lambda}(k)K_{\lambda}(k.z_{0},z)\big{)}}.d\mu(k)\big{)}(h)$
$\displaystyle=$ $\displaystyle
D_{\lambda}(h)\int_{\widetilde{H}/{\widetilde{H}_{z}{{}_{0}}}}f(k)\,Q\,\overline{A_{\lambda}B_{\lambda}\big{(}D_{\lambda}(k)K_{\lambda}(k.z_{0},h.z_{0})\big{)}}.d\mu(k)$
$\displaystyle=$ $\displaystyle\int_{M}f(k)R_{\lambda}(h,k)\,d\mu(k)\qed$

The operator $R_{\lambda}R_{\lambda}^{*}$ is a positive and closed, hence the
square root $\sqrt{R_{\lambda}R_{\lambda}^{*}}$ is well defined and we the
have polar decomposition

$R_{\lambda}^{*}=U_{\lambda}\sqrt{R_{\lambda}R_{\lambda}^{*}}$

For a unique unitary map $U_{\lambda}\mathrel{\mathop{\mathchar
58\relax}}\mathcal{S}_{\lambda}\rightarrow A_{\lambda}^{2}(\mathbb{B}^{n})$.
The operator $U_{\lambda}$ is called the Segal-Bargmann transform see [15].
Since the operator $R_{\lambda}$ intertwine the representations
$\pi_{\lambda}|_{\widetilde{H}}$, and $\tau_{\lambda}$, then $U_{\lambda}$ is
a unitary $\widetilde{H}$-isomorphism. The above discussion can be summarized
in the following theorem, which shows that [5, Theorem 1] holds for
$\lambda>0$.

######  Theorem 4.1.

The Segal-Bargmann transform $U_{\lambda}\mathrel{\mathop{\mathchar
58\relax}}\Big{(}\mathcal{S}_{\lambda},\tau_{\lambda}\Big{)}\rightarrow\Big{(}A_{\lambda}^{2}(\mathbb{B}^{n}),\,\pi_{\lambda}|_{\widetilde{H}}\Big{)}$
is a unitary $\widetilde{H}-$isomorphism.

If $\widetilde{H}$ is Type I subgroup, then the representation
$\pi_{\lambda}|_{\widetilde{H}}$ admits a unique direct integral decomposition
( up to a unitary equivalence)

$\pi_{\lambda}|_{\widetilde{H}}\simeq\int_{\widehat{\widetilde{H}}}^{\bigoplus}m_{\pi_{\lambda}}(\sigma)\sigma
d\mu_{\lambda}(\sigma)$

where $\widehat{\widetilde{H}}$ is the set of equivalence classes of
irreducible unitary representation of $\widetilde{H}$, $\mu_{\lambda}$ is a
Borel measure on $\widehat{\widetilde{H}}$, and
$m_{\pi_{\lambda}}\mathrel{\mathop{\mathchar
58\relax}}\widehat{\widetilde{H}}\rightarrow\mathbb{N}\cup\\{\infty\\}$ is the
multiplicity function. Moreover, multiplicity free representations corresponds
to commutative $C^{*}-$algebras, in particular, Theorem 3.1 says that If $H$
is compact, then $\mathcal{T}^{(\lambda)}(BC^{n}(\mathbb{B}^{n})^{H})$ is
commutative if and only if $\pi_{\lambda}|_{\widetilde{H}}$ is multiplicity
free. Additionally, if an operator commutes with the representation
$\pi_{\lambda}$, then the operator also has a direct integral decomposition.
Furthermore, Corollary 3.1 says that Toeplitz operator
$T_{\varphi}^{(\lambda)}$ is an intertwining operator for $\pi_{\lambda}$ if
and only if the symbol $\varphi$ is $\widetilde{H}-$invariant. Since the
Segal-Bargmann transform $U_{\lambda}$ commute with $\pi_{\lambda}$ then the
diagonalization of the Toeplitz operator is

$U_{\lambda}^{*}T_{\varphi}^{(\lambda)}U_{\lambda}=\int_{\widehat{\widetilde{H}}}^{\bigoplus}m_{\varphi,{\lambda}}(\sigma)\mathrm{id}_{\mathcal{H}_{\sigma}}d\mu_{\lambda}(\sigma)$

where $m_{\varphi,{\lambda}}\mathrel{\mathop{\mathchar
58\relax}}\widehat{\widetilde{H}}\rightarrow\mathbb{C}$. The set
$(m_{\varphi,{\lambda}}(\sigma))_{\sigma}$ is the spectrum of the Toeplitz
operator $T_{\varphi}^{(\lambda)}$.

##  5\. Maximal Abelian Subgroups of $SU(n,1)$

Our main goal in this section is to describe the spectral decomposition of
Toeplitz operators with symbols that are invariant under the maximal abelian
subgroups of $SU(n,1)$. In particular, we will apply the restriction principle
to write the Toeplitz operators as a convolution operator on certain sections
of line bundles. In fact, we will show that the work in [5] can be generalized
to the analytic continuation case.

The group $SU(n,1)$ realizes the group of all automorphisms of the unit ball
$\mathbb{B}^{n}$. As mentioned before, there are n+2 conjugacy classes of
maximal abelian subgroups of $\mathrm{Aut}(\mathbb{B}^{n})$. It is easier to
describe their action on an unbounded realization of the unit ball
$\mathbb{B}^{n}$. Let $D_{n}$ be the unbounded domain defined by

$D_{n}=\\{(z^{{}^{\prime}},z_{n})\in\mathbb{C}^{n-1}\times\mathbb{C}\mid
Im(z_{n})-|z^{{}^{\prime}}|^{2}>0\\}$

The Cayley transform defines a biholomorphism $\mathbb{B}^{n}\rightarrow
D_{n}$, which is given by

$z\mapsto\dfrac{i}{1+z_{n}}(z^{{}^{\prime}},1-z_{n})$

with inverse

$z\mapsto\dfrac{1}{1-iz_{n}}(-2iz^{{}^{\prime}},1+iz_{n})$

Moreover, if we define

$C\mathrel{\mathop{\mathchar 58\relax}}=\begin{pmatrix}\begin{smallmatrix}i\\\
&&\ddots\\\ &&&i\\\ &&&&-i&&i\\\ &&&&1&&1\end{smallmatrix}\end{pmatrix}$

then the group $CSU(n,1)C^{-1}$ realizes the group of biholomorphisms of
$D_{n}$. Each maximal abelian subgroup is conjugate to one of the following
representative [19, 5] :

Quasi-elliptic: This maximal abelian subgroup corresponds to the maximal
compact torus in $SU(n,1)$, it is given by the $\mathbb{T}^{n}-$action on the
unit ball

$t.z=(t_{1}z_{1},...,t_{n}z_{n})$

where $t\in\mathbb{T}^{n}$ and $z\in\mathbb{B}^{n}$.

Quasi-parabolic: This group is isomorphic to
$\mathbb{T}^{n-1}\times\mathbb{R}$, and it acts on $D_{n}$ by

$(t,y).(z^{{}^{\prime}},z_{n})=(tz^{{}^{\prime}},z_{n}+y)$

Quasi-hyperbolic: This group is isomorphic to
$\mathbb{T}^{n-1}\times\mathbb{R}^{+}$ and acts on $D_{n}$ by

$(t,r).(z^{{}^{\prime}}.z_{n})=(rtz^{{}^{\prime}},r^{2}z_{n})$

Nilpotent: This group is isomorphic to $\mathbb{R}^{n-1}\times\mathbb{R}$ and
acts on $D_{n}$ by

$(b,s).(z^{{}^{\prime}},z_{n})=(z^{{}^{\prime}}+b,z_{n}+2i<z^{{}^{\prime}},b>+s+i|b|^{2})$

Quasi-nilpotent: This group is isomorphic to
$\mathbb{T}^{k}\times\mathbb{R}^{n-k-1}\times\mathbb{R}^{+}$ where $1\leq
k\leq n-2$, this group acts on $D_{n}$ by

$(t,b,s).(z^{{}^{\prime}},z^{{}^{\prime\prime}},z_{n})=(tz^{{}^{\prime}},z^{{}^{\prime\prime}}+b,z_{n}+2i<z^{{}^{\prime\prime}},b>+s+i|b|^{2})$

We will restrict our work to the first three classes of subgroups. In
particular, when we say a maximal abelian subgroup, we mean it is either
Quasi-elliptic, Quasi-parabolic, or Quasi-hyperbolic.

Let $H$ be a maximal abelian subgroup of $SU(n,1)$, and $\widetilde{H}$ be the
subgroup of $\widetilde{SU(n,1)}$ that covers $H$. The character
$\chi_{\lambda}\mathrel{\mathop{\mathchar
58\relax}}H_{z_{0}}\rightarrow\mathbb{C}$ can be lifted to $\widetilde{H}$.
This can be done by constructing a holomorphic embedding
$\widetilde{H}\hookrightarrow H/H_{z_{0}}\times\mathbb{R}$, and
$\chi_{\lambda}$ extends to $\widetilde{H}$ by defining

$\chi_{\lambda}\big{(}(h,x)\big{)}\mathrel{\mathop{\mathchar
58\relax}}=e^{2\pi i\lambda x}$

for all $h\in H/H_{z_{0}}$ and $x\in\mathbb{R}$. This construction will be
clear when we examine each case of the maximal abelian subgroups of $SU(n,1)$
in the following subsections.

Let $f$ be in $\mathcal{S}_{\lambda}$, and define
$\widetilde{f}\mathrel{\mathop{\mathchar 58\relax}}=f\chi_{\lambda}$ , we will
show that the operator $R_{\lambda}R_{\lambda}^{*}$ can be written as a
convolution operator. In particular, we will show that

$R_{\lambda}R_{\lambda}^{*}f=\chi_{-\lambda}.(\widetilde{f}*\phi_{H})$

where $\phi_{H}\in L^{1}(H/H_{z_{0}})$. Therefore, the space
$\mathcal{S}_{\lambda}$ can be realized as

$\mathcal{S}_{\lambda}=\\{f\in\mathcal{V}_{\lambda}\mathrel{\mathop{\mathchar
58\relax}}\big{(}\text{For }\psi\in\widehat{H/H_{z_{0}}}\text{ such that
}\widehat{\phi_{H}}(\psi)=0\big{)}\,\mathcal{F}_{H/H_{z_{0}}}\widetilde{f}(\psi)=0\\}$

where $\mathcal{F}_{H/H_{z_{0}}}$ is the Fourier transform on $H/H_{z_{0}}$.

###  5.1. Quasi-elliptic

Following [5], the quasi-elliptic abelian subgroup corresponds to the maximal
compact torus in $SU(n,1)$, and it is given by

$E(n)=\begin{Bmatrix}k_{t,a}=\begin{pmatrix}\begin{smallmatrix}at_{1}&\\\
&at_{2}\\\ &&\ddots\\\ &&&at_{n}\\\
&&&&a\end{smallmatrix}\par\end{pmatrix}\par\Bigg{|}\begin{smallmatrix}a,t_{1},...,t_{n}\in\mathbb{T}\\\
\mathrm{Det}(k_{t,a})=1\end{smallmatrix}\end{Bmatrix}$

The subgroup in the universal covering of $SU(n,1)$ that corresponds to $E(n)$
can be identified with

$\widetilde{E(n)}=\\{(t_{1},t_{2},...,t_{n},x)\in\mathbb{T}^{n}\times\mathbb{R}\mid
e^{2{\pi}i(n+1)x}t_{1}t_{2}...t_{n}=1\\}$

and the product on $\widetilde{E(n)}$ is then given by

$(t,x_{1}).(s,x_{2})=(ts,x_{1}+x_{2})$

Fix $z_{0}=\big{(}(2n)^{-1/2},...,(2n)^{-1/2}\big{)}\in\mathbb{B}^{n}$. The
action of $E(n)$ on $z_{0}$ is given by

$k_{t,a}.z_{0}=\dfrac{1}{\sqrt{2n}}(t_{1},...,t_{n})$

and the stabilizer of $z_{0}$ are given by

$E(n)_{z_{0}}=\\{k_{t,a}\mid t_{i}^{\prime}s=1\\}=\\{k_{t,a}\mid
a^{n+1}=1\\}\simeq\mathbb{Z}_{n}$
$\widetilde{E(n)_{z_{0}}}=\\{(1,1,...,1,\dfrac{m}{n+1})\mid
m\in\mathbb{Z}\\}\simeq\mathbb{Z}$

Let $q=(t,x)\in\widetilde{E(n)}$, then $D_{\lambda}(q)=e^{-2\pi i\lambda x}$.
The differential operators $A_{\lambda}$ and $B_{\lambda}$ (4.2) acts on
$D_{\lambda}$ as an identity map. In particular,
$A_{\lambda}B_{\lambda}(D_{\lambda}(q))=D_{\lambda}(q)$. Therefore,
$D_{\lambda}\in\mathcal{V}_{\lambda}$ for every $\lambda>0$.

Let $f$ be in $\mathcal{S}_{\lambda}$, by Lemma 4.1 we have that the operator
$R_{\lambda}R_{\lambda}^{*}$ is given by the integral

$R_{\lambda}R_{\lambda}^{*}(f)(h)=\int_{M}A_{\lambda}B_{\lambda}(f(k))R_{\lambda}(h,k)\,d\mu(k)$

where

$R_{\lambda}(h,k)=D_{\lambda}(h)\overline{A_{\lambda}B_{\lambda}\big{(}D_{\lambda}(k)K_{\lambda}(k.z_{0},h.z_{0})\big{)}}$

Let $h=(t,x)$ and $k=(s,y)$. Since the differential operators $A_{\lambda}$
and $B_{\lambda}$ acts as the identity map, then the operator
$R_{\lambda}(h,k)$ can be written as

$\displaystyle R_{\lambda}(h,k)$
$\displaystyle=D_{\lambda}(h)\overline{A_{\lambda}B_{\lambda}\big{(}D_{\lambda}(k)K_{\lambda}(k.z_{0},h.z_{0})\big{)}}$
$\displaystyle=D_{\lambda}(h)\overline{D_{\lambda}(k)K_{\lambda}(k.z_{0},h.z_{0})}$
$\displaystyle=D_{\lambda}(h)\overline{D_{\lambda}(k)}K_{\lambda}(h.z_{0},k.z_{0})$
$\displaystyle=e^{-2\pi i\lambda(x-y)}(1-<h.z_{0},k.z_{0}>)^{-\lambda}$
$\displaystyle=e^{-2\pi
i\lambda(x-y)}\big{(}1-\dfrac{1}{2n}\sum_{i=1}^{n}t_{i}(s_{i})^{-1}\big{)}^{-\lambda}$

Furthermore, if $f\in\mathcal{S}_{\lambda}$, then the operator
$R_{\lambda}R_{\lambda}^{*}$ becomes

$\displaystyle R_{\lambda}R_{\lambda}^{*}f(t,x)$
$\displaystyle=\int_{\mathbb{R}\times\mathbb{T}^{n}}f(s,y)e^{-2\pi
i(x-y)}\big{(}1-\dfrac{1}{2n}\sum_{i=1}^{n}t_{i}(s_{i})^{-1}\big{)}^{-\lambda}ds$
$\displaystyle=e^{-2\pi i(x-y)}\big{(}\widetilde{f}*\phi_{E(n)}\big{)}(t)$

where $\widetilde{f}(t)=e^{2\pi i\lambda x}f((t,x).z_{0})$ for every
$(t,x)\in\widetilde{E(n)}$, and the map $\phi_{E(n)}$ is given by

$\phi_{E(n)}(t)=\big{(}1-\dfrac{1}{2n}\sum_{i=1}^{n}t_{i}\big{)}^{-\lambda}$

where $t\in\mathbb{T}^{n}$.

###  5.2. Quasi-parabolic

The quasi-parabolic subgroup is isomorphic to
$\mathbb{T}^{n-1}\times\mathbb{R}$, and it acts on $D_{n}$, the unbounded
realization of the unit ball, by

$(t,y).(z^{{}^{\prime}},z_{n})=(tz^{{}^{\prime}},z_{n}+y)$

where $t\in\mathbb{T}^{n-1}$, $y\in\mathbb{R}$ and $(z^{{}^{\prime}},z_{n})\in
D_{n}$.

The Quasi-parabolic can be written as a subgroup of $SU(n,1)$ as

$P(n)=\begin{Bmatrix}p_{t,y,a}=\begin{pmatrix}\begin{smallmatrix}at_{1}&\\\
&at_{2}\\\ &&\ddots\\\ &&&at_{n-1}\\\
&&&&a(1+i\dfrac{y}{2})&&a(i\dfrac{y}{2})\\\
&&&&a(-i\dfrac{y}{2})&&a(1-i\dfrac{y}{2})\end{smallmatrix}\par\end{pmatrix}\par\Bigg{|}\begin{smallmatrix}a,t_{1},...,t_{n-1}\,\in\,\mathbb{T}\,,\,y\,\in\,\mathbb{R}\\\
a^{n+1}t_{1}...t_{n-1}=1\end{smallmatrix}\end{Bmatrix}$

The group $P(n)$ acts on the unit ball $\mathbb{B}^{n}$ by

$p_{t,y,a}.(z^{{}^{\prime}},z_{n})=\Big{(}\dfrac{2}{-iyz_{n}+2-iy}tz^{{}^{\prime}},\dfrac{(2+iy)z_{n}+iy}{-iyz_{n}+2-iy}\Big{)}$

Let $\widetilde{P(n)}$ be the subgroup of $\widetilde{SU(n,1)}$ that covers
$P(n)$. This group can be identified with

$\widetilde{P(n)}=\\{(t,y,x)\mid t\in\mathbb{T}^{n-1},x,y\in\mathbb{R}\quad
and\quad e^{2\pi i(n+1)x}t_{1}...t_{n-1}=1\\}$

with product

$(t,y,x).(r,w,m)=(tr,y+w,x+m)$

and the projection map is given by

$\displaystyle\widetilde{P(n)}$ $\displaystyle\rightarrow P(n)$
$\displaystyle(t,y,x)$ $\displaystyle\mapsto p_{t,y,e^{2\pi ix}}$

Let
$z_{0}=\Big{(}\dfrac{1}{\sqrt{2(n-1)}},...,\dfrac{1}{\sqrt{2(n-1)}},0\Big{)}$
be in $\mathbb{B}^{n}$. For each $q=(t,y,x)\in\widetilde{P(n)}$ we have

$D_{\lambda}(q)=2^{\lambda}e^{-2\pi i\lambda x}(2-iy)^{-\lambda}$

The operator $N=y\dfrac{\partial}{\partial\,y}$, and direct calculations shows
that

$A_{\lambda}B_{\lambda}\big{(}D_{\lambda}(q)\big{)}=2^{\lambda+2m}e^{-2\pi
i\lambda x}(2-iy)^{-(\lambda+2m)}$

Also, we have

$\displaystyle|A_{\lambda}B_{\lambda}\big{(}D_{\lambda}(q)\big{)}|^{2}$
$\displaystyle=2^{2(\lambda+2m)}|2-iy|^{-2(\lambda+2m)}$
$\displaystyle=\Big{(}1+\dfrac{|y|^{2}}{4}\Big{)}^{-(\lambda+2m)}$

Therefore, $D_{\lambda}\in\mathcal{S}_{\lambda}$ for all
$\lambda>\dfrac{1}{2}-2m$.

Let $h=(t,y,x)$ and $k=(t^{{}^{\prime}},y^{{}^{\prime}},x^{{}^{\prime}})$. The
operator $R_{\lambda}(h,k)$ can be written as

$\displaystyle R_{\lambda}(h,k)$
$\displaystyle=D_{\lambda}(h)Q\,\overline{A_{\lambda}B_{\lambda}\big{(}D_{\lambda}(k)K_{\lambda}(k.z_{0},h.z_{0})\big{)}}$
$\displaystyle=e^{-2\pi
i(x-x^{{}^{\prime}})}\,Q\overline{A_{\lambda}B_{\lambda}\Big{(}1-\dfrac{i}{2}(y^{{}^{\prime}}-y)-\dfrac{1}{2(n-1)}\sum_{i=1}^{n-1}t_{i}^{{}^{\prime}}t_{i}^{-1}\Big{)}^{-\lambda}}$
$\displaystyle=e^{-2\pi
i(x-x^{{}^{\prime}})}Q\,A_{\lambda}B_{\lambda}\Big{(}\big{(}1-\dfrac{i}{2}(y-y^{{}^{\prime}})-\dfrac{1}{2(n-1)}\sum_{i=1}^{n-1}t_{i}(t_{i}^{{}^{\prime}})^{-1}\big{)}^{-\lambda}\Big{)}$

where $Q$, $A_{\lambda}$ and $B_{\lambda}$ acts on $y^{{}^{\prime}}$.
Moreover, the operator $Q$ has the form

$Q=c_{1}+c_{2}y\dfrac{\partial}{\partial
y}+c_{3}y^{2}\dfrac{\partial^{2}}{\partial
y^{2}}+...+c_{2m}y^{2m}\dfrac{\partial^{2m}}{\partial y^{2m}}$

where $c_{i}^{\prime}s$ are constants .

If $f\in\mathcal{S}_{\lambda}$, then the operator $R_{\lambda}R_{\lambda}^{*}$
becomes

$\displaystyle R_{\lambda}R_{\lambda}^{*}f(t,y,x)$
$\displaystyle=\int\limits_{\mathbb{R}\times\mathbb{T}^{n-1}}f(t^{{}^{\prime}},y^{{}^{\prime}},z^{{}^{\prime}})e^{-2\pi
i(x-x^{{}^{\prime}})}\,\,Q\,A_{\lambda}B_{\lambda}\Big{(}\big{(}1-\dfrac{i}{2}(y-y^{{}^{\prime}})$
$\displaystyle\quad\quad\quad-\dfrac{1}{2(n-1)}\sum_{i=1}^{n-1}t_{i}(t_{i}^{{}^{\prime}})^{-1}\big{)}^{-\lambda}\Big{)}dt^{{}^{\prime}}dy^{{}^{\prime}}$
$\displaystyle=e^{-2\pi i\lambda x}(\widetilde{f}*\phi_{P(n)})(t,y)$

where $\widetilde{f}(t,y)=e^{2\pi i\lambda x}f(t,y,x)$ and the function
$\phi_{P(n)}$ is given by

$\phi_{P(n)}(t,y)=QA_{\lambda}B_{\lambda}\Big{(}\big{(}1-\dfrac{i}{2}y-\dfrac{1}{2(n-1)}\sum_{i=1}^{n-1}t_{i}\big{)}^{-\lambda}\Big{)}$

The function $\phi_{P(n)}$ is in $L^{1}(P(n))$ for all $\lambda>0$. To prove
this, first notice that

$A_{\lambda}B_{\lambda}\big{(}1-\dfrac{i}{2}y-\dfrac{1}{2(n-1)}\sum_{i=1}^{n-1}t_{i}\big{)}^{-\lambda}=\Big{(}1-\dfrac{1}{2(n-1)}\sum_{i=1}^{n-1}t_{i}\Big{)}^{2m}\Big{(}1-\dfrac{i}{2}y-\dfrac{1}{2(n-1)}\sum_{i=1}^{n-1}t_{i}\Big{)}^{-(\lambda+2m)}$

since $\big{|}\dfrac{1}{2(n-1)}\sum_{i=1}^{n-1}t_{i}\big{|}\leq 1$, then by
following an argument similar to [5, p. 212] we get

(5.1)
$\big{|}\Big{(}1-\dfrac{i}{2}y-\dfrac{1}{2(n-1)}\sum_{i=1}^{n-1}t_{i}\Big{)}^{-(\lambda+2m)}\big{|}\leq\big{|}1+(|y|-1)^{2}\big{|}^{-(\lambda+2m)/2}$

Since the operator $Q$ is a polynomial differential operator, then it is
enough to consider $y^{r}\dfrac{\partial^{r}}{\partial y^{r}}$. Equation 5.1
implies,
$y^{r}\Big{(}1-\dfrac{i}{2}y-\dfrac{1}{2(n-1)}\sum_{i=1}^{n-1}t_{i}\Big{)}^{-(\lambda+2m+r)}$
is in $L^{1}(P(n))$ for all $0\leq r\leq 2m$. Therefore, $\phi_{P(n)}$ is in
$L^{1}(P(n))$ for all $\lambda>1-2m$.

###  5.3. Quasi-hyperbolic

The quasi-hyperbolic subgroup is isomorphic to
$\mathbb{T}^{n-1}\times\mathbb{R}^{+}$ and it acts on the unbounded domain
$D_{n}$ by

$(t,r).(z^{{}^{\prime}},z_{n})=(rtz^{{}^{\prime}},r^{2}z_{n})$

As a subgroup of $SU(n,1)$ it can be written as [5]

$H(n)=\begin{Bmatrix}h_{t,s,a}=\begin{pmatrix}\begin{smallmatrix}at_{1}&\\\
&at_{2}\\\ &&\ddots\\\ &&&at_{n-1}\\\ &&&&a\,\cosh\,s&&a\,\sinh\,s\\\
&&&&a\,\sinh\,s&&a\,\cosh\,s\end{smallmatrix}\par\end{pmatrix}\par\Bigg{|}\begin{smallmatrix}a,t_{1},...,t_{n-1}\,\in\,\mathbb{T}\,,\,s\,\in\,\mathbb{R}\\\
a^{n+1}t_{1}...t_{n-1}=1\end{smallmatrix}\end{Bmatrix}$

and it acts on the unit ball $\mathbb{B}^{n}$ by

$h_{t,s,a}.(z^{{}^{\prime}},z_{n})=\Big{(}\dfrac{tz^{{}^{\prime}}}{z_{n}\,\sinh\,s+\cosh\,s},\dfrac{z_{n}\,\cosh\,s+\sinh\,s}{z_{n}\,\sinh\,s+\cosh\,s}\Big{)}$

The group $H(1)$ is simply connected. So we have
$H(1)\simeq\widetilde{H(1)}\simeq\mathbb{R}$. When $n>1$, then
$\widetilde{H(n)}$ can be identified with

$\widetilde{H(n)}=\\{(t,s,x)\mid t\in\mathbb{T}^{n-1},\,s,x\in\mathbb{R}\,\mid
e^{2{\pi}i(n+1)x}t_{1}t_{2}...t_{n-1}=1\\}$

with

$(t,s,x).(t^{{}^{\prime}},s^{{}^{\prime}},x^{{}^{\prime}})=(tt^{{}^{\prime}},s+s^{{}^{\prime}},x+x^{{}^{\prime}})$

Let
$z_{0}=\big{(}\dfrac{1}{\sqrt{2(n-1)}},...,\dfrac{1}{\sqrt{2(n-1)}},0\big{)}$,
then we have

$h_{t,s,a}.z_{0}=\big{(}\dfrac{t}{\cosh\,s}z_{0}^{{}^{\prime}},\tanh\,s\big{)}$

So, we have $H(n).z_{0}\simeq
H(n)/{H(n)_{z_{0}}}\simeq\mathbb{T}^{n-1}\times\mathbb{R}$.

Let $q=(t,s,x)\in\widetilde{H(n)}$, then we have $D_{\lambda}(q)=e^{-2\pi
i\lambda x}\big{(}\cosh\,s\big{)}^{-\lambda}$. The operator $N$ is defined by

$N=\dfrac{\sinh\,s}{\cosh\,s}\dfrac{\partial}{\partial\,s}$

Direct calculations show that

(5.2)
$\big{(}I+\dfrac{N}{r}\big{)}(\cosh\,s)^{-l}=(r-l)r^{-1}(\cosh\,s)^{-l}+lr^{-1}(\cosh\,s)^{-(l+2)}$

and for all $l>0$, we have

$\int_{\mathbb{R}}(\cosh\,s)^{-l}\,ds\,<\infty$

This with equation 5.2 implies

$\int_{\mathbb{R}}A_{\lambda}B_{\lambda}\big{(}(\cosh\,s)^{-\lambda}\big{)}^{2}\,ds\,<\infty$

Therefore, $D_{\lambda}$ is in $\mathcal{S}_{\lambda}$ for all $\lambda>0$.

Let $h=(t,s,x)$ and $k=(t^{{}^{\prime}},s^{{}^{\prime}},x^{{}^{\prime}})$. The
operator $R_{\lambda}(h,k)$ can be written as

$\displaystyle R_{\lambda}(h,k)$
$\displaystyle=D_{\lambda}(h)Q\,\overline{A_{\lambda}B_{\lambda}\big{(}D_{\lambda}(k)K_{\lambda}(k.z_{0},h.z_{0})\big{)}}$
$\displaystyle=e^{-2\pi
i(x-x^{{}^{\prime}})}\,Q\overline{A_{\lambda}B_{\lambda}\Big{(}\cosh(s^{{}^{\prime}}-s)-\dfrac{1}{2(n-1)}\sum_{i=1}^{n-1}t_{i}^{{}^{\prime}}t_{i}^{-1}\Big{)}^{-\lambda}}$
$\displaystyle=e^{-2\pi
i(x-x^{{}^{\prime}})}Q\,A_{\lambda}B_{\lambda}\Big{(}\big{(}\cosh(s-s^{{}^{\prime}})-\dfrac{1}{2(n-1)}\sum_{i=1}^{n-1}t_{i}(t_{i}^{{}^{\prime}})^{-1}\big{)}^{-\lambda}\Big{)}$

where $Q$, $A_{\lambda}$ and $B_{\lambda}$ acts on $s^{{}^{\prime}}$.
Moreover, the operator $Q$ has the form

$Q=a_{1}(s)+a_{2}(s)\tanh(s)\dfrac{\partial}{\partial
s}+a_{3}{\tanh^{2}(s)}\dfrac{\partial^{2}}{\partial
s^{2}}+...+a_{2m}(s){\tanh^{2m}(s)}\dfrac{\partial^{2m}}{\partial s^{2m}}$

and $a_{i}(s)$ are functions

$a_{i}(s)=\big{(}\sum_{j}c_{j}\dfrac{\partial^{j}}{\partial\,s^{j}}\big{)}\tanh\,s=\sum_{j}c_{j}\tanh^{(j)}(s)$

where $c_{i}^{\prime}s$ are constants.

If $f\in\mathcal{S}_{\lambda}$, then the operator $R_{\lambda}R_{\lambda}^{*}$
becomes

$\displaystyle R_{\lambda}R_{\lambda}^{*}f(t,y,x)$
$\displaystyle=\int\limits_{\mathbb{R}\times\mathbb{T}^{n-1}}f(t^{{}^{\prime}},y^{{}^{\prime}},z^{{}^{\prime}})e^{-2\pi
i(x-x^{{}^{\prime}})}\,Q\,A_{\lambda}B_{\lambda}\Big{(}\big{(}\cosh(s-s^{{}^{\prime}})$
$\displaystyle\quad\quad\quad\quad\quad\quad\quad\quad\quad\quad\quad\quad\quad\quad-\dfrac{1}{2(n-1)}\sum_{i=1}^{n-1}t_{i}(t_{i}^{{}^{\prime}})^{-1}\big{)}^{-\lambda}\Big{)}dt^{{}^{\prime}}ds^{{}^{\prime}}$
$\displaystyle=e^{-2\pi i\lambda x}(\widetilde{f}*\phi_{H(n)})(t,s)$

where $\widetilde{f}(t,s)=e^{2\pi i\lambda x}f(t,s,x)$ and the function
$\phi_{P(n)}$ is given by

$\phi_{H(n)}(t,s)=QA_{\lambda}B_{\lambda}\Big{(}\big{(}\cosh\,s-\dfrac{1}{2(n-1)}\sum_{i=1}^{n-1}t_{i}\big{)}^{-\lambda}\Big{)}$

######  Remark 5.1.

Direct calculations shows that

$A_{\lambda}B_{\lambda}\big{(}(\cosh
s)^{-\lambda}\big{)}=\sum_{j=1}^{2m}c_{j}(\cosh s)^{-(\lambda+2j)}$

where $c_{j}^{\prime}s$ are constants. Also, we have

$A_{\lambda}B_{\lambda}\Big{(}\big{(}\cosh
s\,-\dfrac{1}{2(n-1)}\sum_{i=1}^{n-1}t_{i}\big{)}^{-\lambda}\Big{)}=\sum_{j=1}^{2m}\gamma_{j}(s)\big{(}\cosh
s-\dfrac{1}{2(n-1)}\sum_{i=1}^{n-1}t_{i}\big{)}^{-(\lambda+2j)}$

where $\gamma_{j}(s)$ are functions that are constant multiples of
$\mathrm{sech}\,s$ and $\tanh s$. Now consider

$\displaystyle\dfrac{\partial}{\partial s}\big{(}\gamma_{j}(s)\big{(}\cosh
s-\dfrac{1}{2(n-1)}\sum_{i=1}^{n-1}$ $\displaystyle
t_{i}\big{)}^{-(\lambda+2j)}\big{)}=\gamma_{j}^{{}^{\prime}}(s)\big{(}\cosh
s-\dfrac{1}{2(n-1)}\sum_{i=1}^{n-1}t_{i}\big{)}^{-(\lambda+2j)}$
$\displaystyle+\gamma_{j}(s)(-\lambda-2j)\sinh s\,\,\big{(}\cosh
s-\dfrac{1}{2(n-1)}\sum_{i=1}^{n-1}t_{i}\big{)}^{-(\lambda+2j+1)}$

It can be readily seen that

$\dfrac{\partial^{k}}{\partial s^{k}}\Big{(}\gamma_{j}(s)\big{(}\cosh
s-\dfrac{1}{2(n-1)}\sum_{i=1}^{n-1}t_{i}\big{)}^{-(\lambda+2j)}\Big{)}=\sum_{r=0}^{k}a_{r}(s)\big{(}\cosh
s-\dfrac{1}{2(n-1)}\sum_{i=1}^{n-1}t_{i}\big{)}^{-(\lambda+2j+r)}$

where $a_{r}(s)$ is a linear combination of $\gamma_{j}(s)^{(i_{r})}$ and
powers of ${\sinh}^{q_{r}}$ and ${\cosh}^{w_{r}}$, where the powers
$q_{r},\,w_{r}$ are strictly less than $\lambda+2j+r$. Since

$\small{\dfrac{1}{2(n-1)}}\displaystyle\sum_{i=1}^{n-1}t_{i}\leq\dfrac{1}{2}$

it follows that

$\big{|}\cosh
s\,-\dfrac{1}{2(n-1)}\sum_{i=1}^{n-1}t_{i}\big{|}^{-\lambda}\leq\big{|}\cosh
s-\dfrac{1}{2}\big{|}^{-\lambda}$

Moreover, $|\tanh\,s|$ and $|\mathrm{sech}\,s|$ are bounded above by 1, and

$\int_{\mathbb{R}}(\sinh s)^{q}(\cosh s-\dfrac{1}{2})^{-v}\,ds<\infty$
$\int_{\mathbb{R}}(\cosh s)^{w}(\cosh s-\dfrac{1}{2})^{-v}\,ds<\infty$

for all $q,w<v$ where $v>1$. Therefore, we have $\phi_{H(n)}\in
L^{1}(H(n).z_{0})$ for all $\lambda>0$.

##  6\.  $T_{\varphi}^{(\lambda)}$ with Symbols Invariant under Maximal
Abelian Subgroups

In this section we will show that the analytic continuation of Toeplitz
operators can be written as a convolution operator on sections of line bundle.
It was shown in [5, Theorem 5.1] that for $\lambda>n$, the operator

$R_{\lambda}T_{\varphi}^{(\lambda)}R_{\lambda}^{*}\mathrel{\mathop{\mathchar
58\relax}}\mathcal{S}_{\lambda}\rightarrow\mathcal{S}_{\lambda}$

can be written as

$R_{\lambda}T_{\varphi}^{(\lambda)}R_{\lambda}^{*}f=f*\nu_{\varphi}$

We will show that we can extend it to cover the analytic continuation case.
First, we need the following Lemma, which shows how the representation
$\pi_{\lambda}$ acts on the kernel of $A_{\lambda}^{2}(\mathbb{B}^{n})$.

######  Lemma 6.1.

Let $g\in G$, and $z\in\mathbb{B}^{n}$, we have

$\pi_{\lambda}(g)K_{z}=\overline{j_{\lambda}(g,z)}K_{g.z}$

######  Theorem 6.1.

Let $\lambda_{0}>0$ and $H$ be a maximal abelian subgroup of $SU(n,1)$ such
that $H.z_{0}$ is restriction injective. Assume $\varphi\in
BC^{n}(\mathbb{B}^{n})^{H}$ is $H-$invariant, then

$R_{\lambda_{0}}T_{\varphi}^{(\lambda_{0})}R_{\lambda_{0}}^{*}f=A_{\lambda_{0}}B_{\lambda_{0}}(f)*\nu_{\varphi}^{\lambda_{0}}$

where $\nu_{\varphi}^{\lambda_{0}}\mathrel{\mathop{\mathchar
58\relax}}H/H_{z_{0}}\rightarrow\mathbb{C}$ is given by

$\nu_{\varphi}^{\lambda_{0}}(h.z_{0})\mathrel{\mathop{\mathchar
58\relax}}=\mathrm{An}_{\lambda_{0}}\Big{(}\overline{A_{\lambda_{0}}B_{\lambda_{0}}\big{(}<\overline{j_{\lambda}(h,z_{0})}K_{h.z_{0},\,\lambda},\,\,\,\varphi
K_{z_{0},\lambda}>_{\lambda}\big{)}}\Big{)}$

and $\mathrm{An}_{\lambda_{0}}$ denotes taking the analytic continuation in
$\lambda$ evaluated at $\lambda_{0}$

###### Proof.

Let $h\in\widetilde{H}$, we have

$\displaystyle
R_{\lambda_{0}}T_{\varphi}^{(\lambda_{0})}R_{\lambda_{0}}^{*}f($
$\displaystyle
h)=D_{\lambda_{0}}(h)T_{\varphi}^{(\lambda_{0})}R_{\lambda_{0}}^{*}f(h.z_{0})$
$\displaystyle=D_{\lambda_{0}}(h)<T_{\varphi}^{(\lambda_{0})}R_{\lambda_{0}}^{*}f,K_{h.z_{0},\,\lambda_{0}}>_{\lambda_{0}}$
$\displaystyle=D_{\lambda_{0}}(h)\mathrm{An}_{\lambda_{0}}\Big{(}<T_{\varphi}^{(\lambda)}R_{\lambda_{0}}^{*}f,K_{h.z_{0},\,\lambda_{0}}>_{A_{\lambda}^{2}}\Big{)}$
$\displaystyle=D_{\lambda_{0}}(h)\mathrm{An}_{\lambda_{0}}\Big{(}<\varphi
R_{\lambda_{0}}^{*}f,K_{h.z_{0},\,\lambda_{0}}>_{A_{\lambda}^{2}}\Big{)}$
$\displaystyle=D_{\lambda_{0}}(h)\mathrm{An}_{\lambda_{0}}\Big{(}\int_{\mathbb{B}^{n}}\varphi(z)R_{\lambda_{0}}^{*}f(z)\overline{K_{h.z_{0},\,\lambda_{0}}(z)}\,d\mu_{\lambda}(z)\Big{)}$
$\displaystyle=D_{\lambda_{0}}(h)\mathrm{An}_{\lambda_{0}}\Big{(}\int_{\mathbb{B}^{n}}\varphi(z)\int_{H/H_{z_{0}}}A_{\lambda_{0}}B_{\lambda_{0}}\big{(}f(k)\big{)}\overline{A_{\lambda_{0}}B_{\lambda_{0}}\big{(}D_{\lambda_{0}}(k)K_{z,\lambda_{0}}(k.z_{0})}\big{)}\,dk$
$\displaystyle\quad\quad\quad\quad\quad\quad\quad\quad\quad\quad\quad\quad\overline{K_{h.z_{0},\,\lambda_{0}}(z)}\,d\mu_{\lambda}(z)\Big{)}$

So, the operator
$R_{\lambda_{0}}T_{\varphi}^{(\lambda_{0})}R_{\lambda_{0}}^{*}f(h)$ can be
written as

$\mathrm{An}_{\lambda_{0}}\Big{(}\int\limits_{\mathbb{B}^{n}}\varphi(z)\Big{[}\int\limits_{H/H_{z_{0}}}A_{\lambda_{0}}B_{\lambda_{0}}\big{(}f(k)\big{)}\overline{A_{\lambda_{0}}B_{\lambda_{0}}\big{(}D_{\lambda_{0}}(k)K_{z,\lambda_{0}}(k.z_{0})\overline{D_{\lambda_{0}}(h)}K_{h.z_{0},\,\lambda_{0}}(z)}\big{)}dk\Big{]}\,dz\Big{)}$

and by interchanging the order of integration we get

$\mathrm{An}_{\lambda_{0}}\int\limits_{H/H_{z_{0}}}A_{\lambda_{0}}B_{\lambda_{0}}\big{(}f(k)\big{)}\overline{A_{\lambda_{0}}B_{\lambda_{0}}\big{(}<\overline{\,j_{\lambda}(h,z_{0})}K_{h.z_{0},\,\lambda},\,\,\,\varphi\,\overline{j_{\lambda}(k,z_{0})}K_{k.z_{0},\lambda}>_{\lambda}\big{)}}d\mu(k)$

For $k\in\widetilde{H}$, define $W_{k}\mathrel{\mathop{\mathchar
58\relax}}=<\overline{\,j_{\lambda}(h,z_{0})}K_{h.z_{0},\,\lambda},\,\,\varphi\overline{j_{\lambda}(k,z_{0})}K_{k.z_{0},\lambda}>_{\lambda}$.
Then $W_{k}$ can be written as

$\displaystyle W_{k}$
$\displaystyle=<\overline{\,j_{\lambda}(h,z_{0})}\pi(k^{-1})K_{h.z_{0},\,\lambda},\,\,\,\overline{j_{\lambda}(k,z_{0})}\pi(k^{-1})\big{(}\varphi
K_{k.z_{0},\lambda}\big{)}>_{\lambda}$
$\displaystyle=<\overline{\,j_{\lambda}(h,z_{0})}\overline{j_{\lambda}(k^{-1},h.z_{0})}K_{k^{-1}h.z_{0},\,\lambda}\,,\,\,\,\varphi\,\overline{j_{\lambda}(k,z_{0})}\overline{j_{\lambda}(k^{-1},k.z_{0})}K_{z_{0},\lambda}>_{\lambda}$
$\displaystyle=<\overline{j_{\lambda}(k^{-1}h,z_{0})}K_{k^{-1}h.z_{0},\,\lambda},\,\,\,\varphi
K_{z_{0},\lambda}>_{\lambda}$

Therefore,

$R_{\lambda_{0}}T_{\varphi}^{(\lambda_{0})}R_{\lambda_{0}}^{*}f(h)=\int\limits_{H/H_{z_{0}}}A_{\lambda_{0}}B_{\lambda_{0}}(f(k))\,\mathrm{An}_{\lambda_{0}}\Big{(}\overline{A_{\lambda_{0}}B_{\lambda_{0}}\big{(}<\overline{j_{\lambda}(k^{-1}h,z_{0})}K_{k^{-1}h.z_{0},\,\lambda},\varphi
K_{z_{0},\lambda}>_{\lambda}}\big{)}\,\Big{)}\qed$

Integration by parts implies the following corollary.

######  Corollary 6.1.

The operator $R_{\lambda_{0}}T_{\varphi}^{(\lambda_{0})}R_{\lambda_{0}}^{*}$
can be written as

$R_{\lambda_{0}}T_{\varphi}^{(\lambda_{0})}R_{\lambda_{0}}^{*}f(h)=f*Q\,\nu_{\varphi}^{\lambda_{0}}$

where $Q$ is a polynomial differential operator.

For $f\in\mathcal{S}_{\lambda}$, we can write

$\displaystyle U_{\lambda}^{*}T_{\varphi}^{(\lambda)}U_{\lambda}f$
$\displaystyle=(\sqrt{R_{\lambda}R_{\lambda}^{*}})^{-1}\,R_{\lambda}T_{\varphi}^{(\lambda)}R_{\lambda}^{*}(\sqrt{R_{\lambda}R_{\lambda}^{*}})^{-1}$
$\displaystyle=(R_{\lambda}R_{\lambda}^{*})^{-1}\,R_{\lambda}T_{\varphi}^{(\lambda)}R_{\lambda}^{*}$

This with Corollary 6.1 implies the following proposition, which gives the
spectral representation of the Toeplitz operators.

######  Theorem 6.2.

Let $\lambda>0$, $H$ be a maximal abelian subgroup of $SU(n,1)$, and let

$A=\\{\psi\in\widehat{H/H_{z_{0}}}\mid\widehat{\phi_{H}^{\lambda}}(\psi)\neq
0\\}.$

If $\varphi\in BC^{n}(\mathbb{B}^{n})^{H}$ is $H-$invariant. Then

$\mathscr{F}U_{\lambda}^{-1}T_{\varphi}^{(\lambda)}U_{\lambda}\mathscr{F}^{-1}\omega(\psi)=\dfrac{\widehat{Q\nu_{\varphi}^{\lambda}}(\psi)}{\widehat{\phi_{H}^{\lambda}}(\psi)}\omega(\psi)$

for all $\psi\in A$, and $\omega\in L^{2}(\widehat{H/H_{z_{0}}})$ such that
$\mathrm{supp\,}\omega\subseteq\mathrm{supp\,}\widehat{\phi_{H}^{\lambda}}$.
Here $\mathscr{F}f(\psi)=\mathscr{F}_{H/H_{z_{0}}}\widetilde{f}(\psi)$.

## References

  * [1] Sheldon Axler  “Toeplitz operators”  In _A glimpse at Hilbert space operators_ Springer, 2010, pp. 125–133
  * [2] Hélène Bommier-Hato, Miroslav Engliš and El Hassan Youssfi  “Analytic Continuation of Toeplitz Operators”  In _Journal of Geometric Analysis_ 25.4, 2015  DOI: 10.1007/s12220-014-9515-0
  * [3] Kamthorn Chailuek and Brian C. Hall  “Toeplitz operators on generalized Bergman spaces”  In _Integral Equations and Operator Theory_ 66.1, 2010  DOI: 10.1007/s00020-009-1734-6
  * [4] Matthew Dawson, Gestur Ólafsson and Raúl Quiroga-Barranco  “Commuting Toeplitz operators on bounded symmetric domains and multiplicity-free restrictions of holomorphic discrete series”  In _Journal of Functional Analysis_ 268, 2015  DOI: 10.1016/j.jfa.2014.12.002
  * [5] Matthew Dawson, Gestur Ólafsson and Raúl Quiroga-Barranco  “The restriction principle and commuting families of Toeplitz operators on the unit ball”  In _Sao Paulo Journal of Mathematical Sciences_ 12.2, 2018  DOI: 10.1007/s40863-018-0104-1
  * [6] Matthew Dawson, Gestur Ólafsson and Raul Quiroga-Barranco  “Toeplitz Operators on the Domain $\\{Z\in M_{2\times 2}(\mathbb{C})\mid ZZ^{*}<I\\}$ with $\mathrm{U}(2)\times\mathbb{T}^{2}$ -Invariant Symbols”  In _Operator Theory: Advances and Applications_ 279, 2020  DOI: 10.1007/978-3-030-44651-2˙9
  * [7] Miroslav Engliš  “Density of algebras generated by Toeplitz operators on Bergman spaces”  In _Arkiv för Matematik_ 30.1-2 Institut Mittag-Leffler, 1992, pp. 227–243  DOI: 10.1007/BF02384872
  * [8] Eskin  “Boundary value problems for ellipti pseudodifferential equations”  In _Math. Monog._ 52 Amer. Math. Soc., 1980
  * [9] Pavel I Etingof et al.  “Introduction to representation theory”, Student mathematical library  Providence, RI: American Mathematical Society, 2011
  * [10] Gerald B. Folland  “A course in abstract harmonic analysis, second edition”  In _A Course in Abstract Harmonic Analysis, Second Edition_ , 2016
  * [11] Sergei Grudsky, Raul Quiroga-Barranco and Nikolai Vasilevski  “Commutative C* -algebras of Toeplitz operators and quantization on the unit disk”  In _Journal of Functional Analysis_ 234.1 Elsevier, 2006, pp. 1–44
  * [12] Anthony Knapp  “Group Representations and Harmonic Analysis from Euler to Langlands, Part II”  In _Notices of the AMS_ , 1996, pp. 537–549
  * [13] Toshiyuki Kobayashi  “Multiplicity-free representations and visible actions on complex manifolds”  In _Publications of the Research Institute for Mathematical Sciences_ 41.3, 2005, pp. 497–549  DOI: 10.2977/prims/1145475221
  * [14] Andrew D. Lewis  “Notes on global analysis”, 2014
  * [15] Gestur Ólafsson and Bent Ørsted  “Generalizations of the Bargmann transform”  In _Lie theory and its applications in physics (Clausthal, 1995)_ World Sci. Publishing River Edge, NJ, 1996, pp. 3–14
  * [16] Raul Quiroga-Barranco  “Separately radial and radial Toeplitz operators on the unit ball and representation theory”  In _Boletin de la Sociedad Matematica Mexicana_ 22.2, 2016  DOI: 10.1007/s40590-016-0111-0
  * [17] Raul Quiroga-Barranco and Nikolai Vasilevski  “Commutative C*-algebras of Toeplitz operators on the unit ball, I. Bargmann-type transforms and spectral representations of Toeplitz operators”  In _Integral Equations and Operator Theory_ 59.3, 2007  DOI: 10.1007/s00020-007-1537-6
  * [18] Raul Quiroga-Barranco and Nikolai Vasilevski  “Commutative C*-algebras of toeplitz operators on the unit ball, II. Geometry of the level sets of symbols”  In _Integral Equations and Operator Theory_ 60.1, 2008  DOI: 10.1007/s00020-007-1540-y
  * [19] Raúl Quiroga-Barranco and Armando Sánchez-Nungaray  “Moment maps of Abelian groups and commuting Toeplitz operators acting on the unit ball”  In _Journal of Functional Analysis_ 281, 2021  DOI: 10.1016/j.jfa.2021.109039
  * [20] Harald Upmeier  “Toeplitz operators and index theory in several complex variables”  Springer Science & Business Media, 1995
  * [21] Michele Vergne and Hugo Rossi  “Analytic continuation of the holomorphic discrete series of a semi-simple Lie group”, 1976
  * [22] Nolan R Wallach  “The analytic continuation of the discrete series. I & II”  In _Transactions of the American Mathematical Society_ 251, 1979, pp. 1–17 and 19–37
  * [23] Kehe Zhu  “Operator theory in function spaces”  Boca Raton, FL: CRC Press, 1990