#  $G$-invariant Bergman kernel and geometric quantization on complex
manifolds with boundary

Chin-Yu Hsiao  Institute of Mathematics, Academia Sinica and National Center
for Theoretical Sciences, 6F, Astronomy-Mathematics Building, No.1, Sec.4,
Roosevelt Road, Taipei 10617, Taiwan<EMAIL_ADDRESS>or
<EMAIL_ADDRESS>,  Rung-Tzung Huang  Department of Mathematics,
National Central University, Chung-Li 320, Taiwan<EMAIL_ADDRESS>,
Xiaoshan Li  School of Mathematics and Statistics, Wuhan University, Wuhan,
430072, Hubei, China<EMAIL_ADDRESS>and  Guokuan Shao  School of
Mathematics (Zhuhai), Sun Yat-sen University, Zhuhai 519082, Guangdong, China
<EMAIL_ADDRESS>In memory of Professor Nessim Sibony

###### Abstract.

Let $M$ be a complex manifold with boundary $X$, which admits a holomorphic
Lie group $G$-action preserving $X$. We establish a full asymptotic expansion
for the $G$-invariant Bergman kernel under certain assumptions. As an
application, we get $G$-invariant version of Fefferman’s result about
regularity of biholomorphic maps on strongly pseudoconvex domains of
$\mathbb{C}^{n}$. Moreover, we show that the Guillemin-Sternberg map on a
complex manifold with boundary is Fredholm by developing reduction to boundary
technique, which establish “quantization commutes with reduction” in this
case.

###### Key words and phrases:

invariant Bergman kernel, geometric quantization, moment map, Fredholm
operator

###### 2020 Mathematics Subject Classification:

Primary: 32A25, 53D50, 58J40

Chin-Yu Hsiao was partially supported by Taiwan Ministry of Science and
Technology projects 108-2115-M-001-012-MY5, 109-2923-M-001-010-MY4 and
Academia Sinica Investigator Award.

Rung-Tzung Huang was supported by Taiwan Ministry of Science and Technology
project 109-2115-M-008-007-MY2 and 111-2115-M-008 -003 -MY2.

Xiaoshan Li was supported by National Natural Science Foundation of China
(Grant No. 12271411 and 11871380)

Guokuan Shao was supported by National Natural Science Foundation of China
(Grant No. 12001549) and the Fundamental Research Funds for the Central
Universities, Sun Yat-sen University (Grant No. 22qntd2901)

##  1\. Introduction

Let $M$ be a domain in a complex manifold $M^{\prime}$ with smooth boundary
$X$. Let $B$ be the orthogonal projection from $L^{2}(M)$ onto the space of
$L^{2}$ holomorphic functions on $M$ (Bergman projection). The study of
boundary behavior of $B$ is a classical subject in several complex variables.
When $X$ is strongly pseudoconvex, Feffermann [9] established an asymptotic
expansion for $B$ at the diagonal. A full asymptotic expansion of $B$ was
obtained by Boutete de Monvel and Sjöstrand [6]. The asymptotic of $B$ plays
an important role in some important probems in several complex variables. For
example, by using asymptotic expansion for $B$, Fefferman [9] established
boundary regularity of biholomorphic maps of domains in $\mathbb{C}^{n}$. When
$M$ is a weakly pseudoconvex domain, there are fewer results. In [20], the
first author and Savale obtained a pointwise expansion of $B$ at the diagonal
in the tangential direction when $M$ is a finite type weakly pseudoconvex
domain in $\mathbb{C}^{2}$. In general, it is very difficult to study the
boundary behavior of the Bergman projection on a weakly pseudoconvex domain
$M$. Let us see some simple examples and explain our motivation.

(1) Let $M:=\left\\{(z_{1},z_{2},z_{3})\in\mathbb{C}^{3};\,\lvert
z_{1}\rvert^{4}+\lvert z_{2}\rvert^{2}+\lvert z_{3}\rvert^{2}<1\right\\}$. $M$
admits an $S^{1}$-action:

$S^{1}\times M\to M,\ \
e^{i\theta}\cdot(z_{1},z_{2},z_{3})=(e^{-i\theta}z_{1},e^{i\theta}z_{2},e^{i\theta}z_{3}).$

(2) Let

$M:=\Big{\\{}(z_{1},z_{2},z_{3},z_{4},z_{5},z_{6})\in\mathbb{C}^{6};\,(\lvert
z_{5}\rvert^{4}+\lvert z_{6}\rvert^{2})(\sum^{4}_{j=1}\lvert
z_{j}\rvert^{2}+z_{1}z_{3}+z_{2}z_{4}+\overline{z}_{1}\overline{z}_{3}+\overline{z}_{2}\overline{z}_{4})<1\Big{\\}}.$

Then, $M$ admits a $G:=S^{1}\times SU(2)$ action:

$\begin{split}&(e^{i\theta},g)\cdot z=(w_{1},w_{2},\ldots,w_{6}),\\\
&(w_{1},w_{2})^{t}:=g(z_{1},z_{2})^{t},\ \
(w_{3},w_{4})^{t}:=\overline{g}(z_{3},z_{4})^{t},\ \
(w_{5},w_{6})=(e^{-i\theta}z_{5},e^{i\theta}z_{6}),\\\ &g\in SU(2),\ \
e^{i\theta}\in S^{1},\ \ z\in M,\end{split}$

where $z^{t}$ denotes the transpose of $z$.

In these examples, all the domains are weakly pseudoconvex but with group
action and the boundary reduced spaces of these examples are strongly
pseudoconvex CR manifolds (we refer the reader to [19, Section 2.5] for the
details and to Section 2.3 below for the meaning of reduced spaces). In [19],
the first author, Ma and Marinescu showed that the $G$-invariant Szegő
projection is a complex Fourier integral operator if the reduced space is a
strongly pseudoconex CR manifold (the whole CR manifold can be non strongly
pseudoconvex). Thus, it is quite natural and interesting to study
$G$-invariant Bergman projection on a non strongly pseudoconvex domain with
group action. This is the motivation of this work. In this work, we completely
study the $G$-invariant Bergman projection on a domain $M$ (can be non
strongly pseudoconvex) with group $G$ action under the assumption that the
boundary reduced space with respect to the group $G$ action is non-degenerate.
We show that the $G$-invariant Bergman projection on a such domain is a
complex Fourier integral operator. As an application, we get $G$-invariant
version of Fefferman’s result about regularity of biholomorphic maps on
strongly pseudoconvex domains of $\mathbb{C}^{n}$. Since the study of
$G$-invariant Bergman projection is closely related to geometric quantization,
we also study geometric quantization on complex manifolds with boundary.

We now formulate our main results. We refer to Section 2 for some notations
and terminology used here. Let $M$ be a relatively compact open subset with
smooth boundary $X$ of a complex manifold $M^{\prime}$ of dimension $n$,
$n\geq 3$. Let $\rho\in C^{\infty}(M^{\prime},\mathbb{R})$ be a defining
function of $X$, that is,

$X=\\{x\in M^{\prime};\,\rho(x)=0\\},\ \ M=\\{x\in M^{\prime};\,\rho(x)<0\\}$

and $d\rho(x)\neq 0$ at every point $x\in X$. Then the manifold $X$ is a CR
manifold with natural CR structure
$T^{1,0}X:=T^{1,0}M^{\prime}\cap\mathbb{C}TX$, where $T^{1,0}M^{\prime}$
denotes the holomorphic tangent bundle of $M^{\prime}$.

Suppose that $M^{\prime}$ admits a compact $d$-dimensional Lie group $G$
action. Let $\mathfrak{g}$ denote the Lie algebra of $G$. Let
$\mu:M^{\prime}\to\mathfrak{g}^{*}$, $\mu_{X}:=\mu|_{X}:X\to\mathfrak{g}^{*}$
be the associated moment maps (cf. Definition 2.2). We will work in the
following setting.

######  Assumption 1.1.

The $G$-action is holomorphic, preserves the boundary $X$, $0$ is a regular
value of $\mu_{X}$, $G$ acts freely on $\mu^{-1}(0)\cap X$, $\mu^{-1}(0)\cap
X\neq\emptyset$, the Levi form is positive or negative near $\mu^{-1}(0)\cap
X$.

Note that the $G$-action is holomorphic means that the $G$-action preserves
$J$, where $J$ is the complex structure map on $T^{1,0}M^{\prime}$. The
$G$-action preserves the boundary $X$ means that we can find a defining
function $\rho\in C^{\infty}(M^{\prime},\mathbb{R})$ of $X$ such that
$\rho(g\circ x)=\rho(x)$, for every $x\in M^{\prime}$ and every $g\in G$.

We take a $G$-invariant Hermitian metric $\langle\cdot\,|\,\cdot\,\rangle$ on
$\mathbb{C}TM^{\prime}$. The $G$-invariant Hermitian metric
$\langle\cdot\,|\,\cdot\,\rangle$ on $\mathbb{C}TM^{\prime}$ induces a
$G$-invariant Hermitian metric $\langle\cdot\,|\,\cdot\,\rangle$ on
$\oplus_{1\leq p+q\leq n,p,q\in\mathbb{N}_{0}}T^{*p,q}M^{\prime}$, where
$T^{*p,q}M^{\prime}$ denotes the bundle of $(p,q)$ forms on $M^{\prime}$. From
now on, we fix a defining function $\rho\in C^{\infty}(M^{\prime},\mathbb{R})$
of $X$ such that

$\begin{split}&\mbox{$\langle\,d\rho(x)\,|\,d\rho(x)\,\rangle=1$ on $X$},\\\
&\rho(g\circ x)=\rho(x),\ \ \forall x\in M^{\prime},\ \ \forall g\in
G.\end{split}$ (1.1)

Let $(\,\cdot\,|\,\cdot\,)_{M}$ be the $L^{2}$ inner product on
$\Omega^{0,q}_{c}(M)$ given by

$(\,u\,|\,v\,)_{M}:=\int_{M}\langle\,u\,|\,v\,\rangle dv_{M^{\prime}},\ \
u,v\in\Omega^{0,q}_{c}(M),$ (1.2)

where $dv_{M^{\prime}}$ is the volume form on $M^{\prime}$ induced by
$\langle\,\cdot\,|\,\cdot\,\rangle$. Let $L^{2}_{(0,q)}(M)$ be the $L^{2}$
completion of $\Omega^{0,q}_{c}(M)$ with respect to
$(\,\cdot\,|\,\cdot\,)_{M}$. We write $L^{2}(M):=L^{2}_{(0,0)}(M)$.

Let
$\overline{\partial}:C^{\infty}(\overline{M})\to\Omega^{0,1}(\overline{M}\,)$
be the Cauchy-Riemann operator on $\overline{M}$. We extend
$\overline{\partial}$ to $L^{2}(M)$:

$\overline{\partial}:{\rm Dom\,}\overline{\partial}\subset L^{2}(M)\rightarrow
L^{2}_{(0,1)}(M),$

where $u\in{\rm Dom\,}\overline{\partial}$ if we can find $u_{j}\in
C^{\infty}(\overline{M})$, $j=1,2,\ldots$, such that $u_{j}\rightarrow u$ in
$L^{2}(M)$ as $j\rightarrow+\infty$ and there is a $v\in L^{2}_{(0,1)}(M)$
such that $\overline{\partial}u_{j}\rightarrow v$ as $j\rightarrow+\infty$. We
set $\overline{\partial}u:=v$. Let

$H^{0}(\overline{M}):={\rm Ker\,}\overline{\partial}\subset L^{2}(M).$ (1.3)

Then $H^{0}(\overline{M})$ is a (possible infinite dimensional)
$G$-representation, its $G$-invariant part is the $G$-invariant $L^{2}$
holomorphic functions on $\overline{M}$. Let

$H^{0}(\overline{M})^{G}:=\left\\{u\in
H^{0}(\overline{M});\,h^{*}u=u,~{}\text{for any}~{}h\in G\right\\}.$ (1.4)

Let

$B_{G}:L^{2}(M)\rightarrow H^{0}(\overline{M})^{G}$ (1.5)

be the orthogonal projection with respect to $(\,\cdot\,|\,\cdot\,)_{M}$
($G$-invariant Bergman projection). The $G$-invariant Bergman kernel
$B_{G}(x,y)\in\mathscr{D}^{\prime}(M\times M)$ is the distribution kernel of
$B_{G}$.

We introduce some notations. For $x\in X$, let $\mathcal{L}_{x}$ denote the
Levi form of $X$ at $x$ (see (2.9)) and let
$\det\mathcal{L}_{x}:=\lambda_{1}(x)\cdots\lambda_{n-1}(x)$, where
$\lambda_{j}(x)$, $j=1,\ldots,n-1$, are the eigenvalues of $\mathcal{L}_{x}$
with respect to $\langle\,\cdot\,|\,\cdot\,\rangle$. For any
$\xi\in\mathfrak{g}$, we write $\xi_{M^{\prime}}$ to denote the vector field
on $M^{\prime}$ induced by $\xi$. That is,

$\mbox{$(\xi_{M^{\prime}}u)(x)=\frac{\partial}{\partial
t}\left(u(\exp(t\xi)\circ x)\right)|_{t=0}$, for any $u\in
C^{\infty}(M^{\prime})$}.$ (1.6)

For $x\in M^{\prime}$, set

$\underline{\mathfrak{g}}_{x}={\rm
Span\,}\left\\{\xi_{M^{\prime}}(x);\,\xi\in\mathfrak{g}\,\right\\}.$ (1.7)

Fix $x\in\mu^{-1}(0)\cap X$, consider the linear map

$\begin{array}[]{rll}R_{x}:\underline{\mathfrak{g}}_{x}&\rightarrow&\underline{\mathfrak{g}}_{x},\\\
u&\rightarrow&R_{x}u,\ \
\langle\,R_{x}u\,|\,v\,\rangle=\langle\,d\omega_{0}(x)\,,\,Ju\wedge
v\,\rangle,\end{array}$

where $\omega_{0}(x)=J^{t}(d\rho)(x)$, $J^{t}$ is the complex structure map on
$T^{*}M^{\prime}$. Let $\det R_{x}=\mu_{1}(x)\cdots\mu_{d}(x)$, where
$\mu_{j}(x)$, $j=1,2,\ldots,d$, are the eigenvalues of $R_{x}$. Fix
$x\in\mu^{-1}(0)\cap X$, put $Y_{x}=\left\\{g\circ x;\,g\in G\right\\}$.
$Y_{x}$ is a $d$-dimensional submanifold of $X$. The $G$-invariant Hermitian
metric $\langle\,\cdot\,|\,\cdot\,\rangle$ induces a volume form $dv_{Y_{x}}$
on $Y_{x}$. Put

$V_{{\rm eff\,}}(x):=\int_{Y_{x}}dv_{Y_{x}}.$

The first main result of this work is the following

######  Theorem 1.2.

With the notations and assumptions above and recall that we work with
Assumption 1.1. Let $\tau\in C^{\infty}(\overline{M})$ with ${\rm
supp\,}\tau\cap\mu^{-1}(0)\cap X=\emptyset$. Then, $\tau B_{G}\equiv 0\mod
C^{\infty}(\overline{M}\times\overline{M})$, $B_{G}\tau\equiv 0\mod
C^{\infty}(\overline{M}\times\overline{M})$.

Let $p\in\mu^{-1}(0)\cap X$. Let $U$ be an open local coordinate patch of $p$
in $M^{\prime}$, $D:=U\cap X$. If Levi form is negative on $D$, then

$B_{G}(z,w)\equiv 0\mod C^{\infty}((U\times
U)\cap(\overline{M}\times\overline{M})).$ (1.8)

Suppose that the Levi form is positive on $D$. Then,

$B_{G}(z,w)\equiv\int^{+\infty}_{0}e^{it\Psi(z,w)}b(z,w,t)dt\mod
C^{\infty}((U\times U)\cap(\overline{M}\times\overline{M})),$ (1.9)

where

$\begin{split}&b(z,w,t)\in S^{n-\frac{d}{2}}_{1,0}(((U\times
U)\cap(\overline{M}\times\overline{M}))\times\mathbb{R}_{+}),\\\
&b(z,w,t)\sim\sum^{+\infty}_{j=0}t^{n-\frac{d}{2}-j}b_{j}(z,w)\ \ \mbox{in
$S^{n-\frac{d}{2}}_{1,0}(((U\times
U)\cap(\overline{M}\times\overline{M}))\times\mathbb{R}_{+})$},\\\
&b_{j}(z,w)\in C^{\infty}((U\times U)\cap(\overline{M}\times\overline{M})),\ \
j=0,1,2,\ldots,\end{split}$ (1.10) $b_{0}(z,z)=2^{d}\frac{1}{V_{{\rm
eff\,}}(x)}\pi^{-n+\frac{d}{2}}\lvert\det
R_{x}\rvert^{-\frac{1}{2}}\lvert\det\mathcal{L}_{x}\rvert,\ \ \forall
x\in\mu^{-1}(0)\cap D,$ (1.11)

and

$\begin{split}&\Psi(z,w)\in C^{\infty}(((U\times
U)\cap(\overline{M}\times\overline{M}))),\ \ {\rm Im\,}\Psi\geq 0,\\\
&\Psi(z,z)=0,\ z\in\mu^{-1}(0)\cap D,\\\ &\mbox{${\rm Im\,}\Psi(z,w)>0$ if
$(z,w)\notin{\rm diag\,}((\mu^{-1}(0)\cap D)\times(\mu^{-1}(0)\cap D))$},\\\
&d_{x}\Psi(x,x)=-\omega_{0}(x)-id\rho(x),\ \
d_{y}\Psi(x,x)=\omega_{0}(x)-id\rho(x),\ \ x\in\mu^{-1}(0)\cap D,\\\
&\mbox{$\Psi|_{D\times D}=\Phi$, $\Phi$ is the phase as
in~{}\cite[cite]{[\@@bibref{}{HH}{}{}, Theorem 1.5]}}.\end{split}$ (1.12)

Moreover, let $z=(x_{1},\ldots,x_{2n-1},\rho)$ be local coordinates of
$M^{\prime}$ defined near $p$ in $M^{\prime}$ with $x(p)=0$ and
$x=(x_{1},\ldots,x_{2n-1})$ are local coordinates of $X$ defined near $p$ in
$X$. Then,

$\mbox{$\Psi(z,w)=\Phi(x,y)-i\rho(z)(1+f(z))-i\rho(w)(1+\overline{f}(w))+O(\lvert(z,w)\rvert^{3})$
near $(p,p)$},$ (1.13)

where $f\in C^{\infty}$, $f=O(\lvert z\rvert)$.

The above theorem lays a foundation to the study of Toeplitz quantization on
complex manifolds with boundary. We refer the reader to the discussion before
(2.10) for the meaning of $F\equiv G\mod C^{\infty}((U\times
U)\cap(\overline{M}\times\overline{M}))$.

Before we formulate our main result about geometric quantization on complex
manifolds with boundary, we give some historic remark about geometric
quantization theory. The famous geometric quantization conjecture of Guillemin
and Sternberg [11] states that for a compact pre-quantizable symplectic
manifold admitting a Hamiltonian action of a compact Lie group, the principle
of “quantization commutes with reduction” holds. This conjecture was first
proved independently by Meinrenken [24] and Vergne [32] for the case where the
Lie group is abelian, and by Meinrenken [25] in the general case, then Tian-
Zhang [30] gave a purely analytic proof in general case with various
generalizations, see [33] for a survey and complete references on this
subject. In the case of a non-compact symplectic manifold $M$ which has a
compact Lie group action $G$, this question was solved by Ma-Zhang [22, 23] as
a solution to a conjecture of Vergne in her ICM 2006 plenary lecture [34], see
[21] for a survey. Paradan [29] gave a new proof, cf. also the recent work
[13]. A natural choice for the quantum spaces of a compact symplectic manifold
is the kernel of the Dirac operator.

In [27], Ma-Zhang established the asymptotic expansion of the $G$-invariant
Bergman kernel for a positive line bundle $L$ over a compact symplectic
manifold $M$ and by using the asymptotic expansion of $G$-invariant Bergman
kernel, they could establish the “quantization commutes with reduction”
theorem when the power of the line bundle $L$ is high enough. In [16], the
first and second authors established the asymptotic expansion of the
$G$-invariant Szegő kernel for $(0,q)$ forms on a non-degenerate CR manifold
and they could establish the “quantization commutes with reduction” theorem
when the CR manifold admits a circle action.

The quantization of strongly pseudoconvex or more generally contact manifolds
via the Szegő projector or its generalizations was developed by Boutet de
Monvel and Guillemin [5] and can be applied to the Kähler quantization by
using the above construction (see e. g. [7, 28]). In [19], the first author,
Ma and Marinescu study the quantization of CR manifolds and the principle of
“quantization commutes with reduction”. An important difference between the CR
setting and the Kähler/symplectic setting is that the quantum spaces in the
case of compact Kähler/symplectic manifolds are finite dimensional, whereas
for the compact strongly pseudoconvex CR manifolds that they consider the
quantum spaces consisting of CR functions are infinite dimensional.

For manifolds with boundary, in [31], Tian-Zhang extended the results in [30]
to the case where the compact symplectic manifold with a non-empty boundary
under the assumption that the preimage of the moment map of the regular value
$0$ in the dual of the Lie algebra does not touch the boundary. The quantum
spaces considered in [31] are the kernel of the Dirac operator with Atiyah-
Patodi-Singer type boundary conditions [2] and hence finite dimensional.
Following the same line of [19], in this paper we study the quantization of
complex manifolds with boundary and the principle of “quantization commutes
with reduction”. The quantum spaces we consider are the spaces of $L^{2}$
holomorphic functions and could be infinite dimensional.

We now formulate our main results. By Assumption 1.1, $\mu^{-1}_{X}(0)$ is a
$d$-codimensional submanifold of $X$. We decompose $\mu^{-1}(0)\cap X$ into
two parts $\widehat{X}$ and $\widetilde{X}$ on which the Levi-form is strongly
pseudoconvex and pseudoconcave, respectively. From now on, we assume that
$\widehat{X}$ is non-emptey. Let

$\widehat{X}_{G}:=\widehat{X}/G,~{}\widetilde{X}_{G}=\widetilde{X}/G.$ (1.14)

It was proved in [19, Theorem 2.6] that $\widehat{X}_{G}$ is a compact CR
manifold. Let

$\overline{\partial}_{b}:{\rm Dom\,}\overline{\partial}_{b}\subset
L^{2}(\widehat{X}_{G})\rightarrow L^{2}_{(0,1)}(\widehat{X}_{G})$

be the tangential Cauchy-Riemann operator. For every $s\in\mathbb{R}$, let
$W^{s}(\overline{M})$ and $W^{s}(\widehat{X}_{G})$ denotes the Sobolev spaces
of $\overline{M}$ and $\widehat{X}_{G}$ of order $s$ (see the discussion after
Definition 2.1, for the precise meaning of $W^{s}(\overline{M})$). Let
$(\,\cdot\,|\,\cdot\,)_{\widehat{X}_{G}}$ be the $L^{2}$ inner product on
$L^{2}(\widehat{X}_{G})$ induced naturally by
$\langle\,\cdot\,|\,\cdot\,\rangle$. For every $s\in\mathbb{R}$, put

$\begin{split}&H^{0}(\overline{M})_{s}:=\left\\{u\in
W^{s}(\overline{M});\,\overline{\partial}u=0~{}\text{in the sense of
distributions}\right\\},\\\ &H^{0}_{b}(\widehat{X}_{G})_{s}:=\left\\{u\in
W^{s}(\widehat{X}_{G});\,\overline{\partial}_{b}u=0~{}\text{in the sense of
distributions}\right\\},\\\ &H^{0}(\overline{M})^{G}_{s}:=\left\\{u\in
H^{0}(\overline{M})_{s};\,h^{*}u=u~{}\text{in the sense of distributions for
all $h\in G$}\right\\}.\end{split}$ (1.15)

We write $H^{0}_{b}(\widehat{X}_{G}):=H^{0}_{b}(\widehat{X}_{G})_{0}$. Let
$\iota_{\widehat{X}}:\widehat{X}\hookrightarrow X$ be the natural inclusion
and let $\iota^{*}_{\widehat{X}}:C^{\infty}(X)\to C^{\infty}(\widehat{X})$ be
the pull-back by $\iota_{\widehat{X}}$. Let
$\iota_{G,\widehat{X}}:C^{\infty}(\widehat{X})^{G}\to
C^{\infty}(\widehat{X}_{G})$ be the natural identification. Let

$\tilde{\sigma}_{G}:H^{0}(\overline{M})^{G}\cap C^{\infty}(\overline{M})\to
H^{0}_{b}(\widehat{X}_{G}),\qquad\tilde{\sigma}_{G}=\iota_{G,\widehat{X}}\circ\iota^{*}_{\widehat{X}}\circ\gamma,$
(1.16)

where $\gamma$ denotes the operator of the restriction to the boundary $X$.
The map (1.16) is well defined. The map $\tilde{\sigma}_{G}$ does not extend
to a bounded operator on $L^{2}$, so it is necessary to consider its extension
to Sobolev spaces. We can check that $\tilde{\sigma}_{G}$ extends by density
to a bounded operator

$\tilde{\sigma}_{G}=\tilde{\sigma}_{G,s}:H^{0}(\overline{M})^{G}_{s}\to
H^{0}_{b}(\widehat{X}_{G})_{s-\frac{d}{4}-\frac{1}{2}},~{}\text{for every
$s\in\mathbb{R}$}$ (1.17)

(see Theorem 6.1 and Theorem 6.2 below). For every $s\in\mathbb{R}$, put

${\rm Coker\,}\tilde{\sigma}_{G,s}={\rm Coker\,}\tilde{\sigma}_{G}:=\\{u\in
H^{0}_{b}(\widehat{X}_{G})_{s-\frac{d}{4}-\frac{1}{2}};\,(\,u\,|\,\tilde{\sigma}_{G,s}v)_{\widehat{X}_{G}}=0,\forall
v\in H^{0}(\overline{M})^{G}_{s}\cap C^{\infty}(\overline{M})\\}.$ (1.18)

The following is our second main result

######  Theorem 1.3.

Let $M$ be a relatively compact open subset with smooth boundary $X$ of a
complex manifold $M^{\prime}$ of dimension $n$, $n\geq 3$. Let $G$ be a
compact Lie group acting on $M^{\prime}$ such that Assumption 1.1 holds. With
the notations used above, assume that ${\rm
dim\,}_{\mathbb{R}}\widehat{X}_{G}\geq 5$. Then, for every $s\in\mathbb{R}$,
the Guillemin-Sternberg map (1.17) is Fredholm. More precisely,
$\operatorname{Ker}\tilde{\sigma}_{G,s}$ and ${\rm
Coker\,}\tilde{\sigma}_{G,s}$ are finite dimensional subspaces of
$H^{0}(\overline{M})^{G}\cap C^{\infty}(\overline{M})^{G}$ and
$H^{0}_{b}(\widehat{X}_{G})\cap C^{\infty}(\widehat{X}_{G})$ respectively,
$\operatorname{Ker}\tilde{\sigma}_{G,s}$ and ${\rm
Coker\,}\tilde{\sigma}_{G,s}$ are independent of $s$.

The assumption ${\rm dim\,}_{\mathbb{R}}\widehat{X}_{G}\geq 5$ in Theorem 1.3
can be replaced by $\overline{\partial}_{b,\widehat{X}_{G}}$ has closed range,
where $\overline{\partial}_{b,\widehat{X}_{G}}$ denotes the tangential Cauchy-
Riemann operator on $\widehat{X}_{G}$.

Theorem 1.3 tells us that up to some finite dimensional spaces, the quantum
space $H^{0}(\overline{M})^{G}$ is isomorphic to the space of $L^{2}$ CR
functions on $\widehat{X}_{G}$.

Suppose that

$\begin{split}\mbox{$0$ is a regular value of $\mu$, $G$ acts freely on
$\mu^{-1}(0)$.}\end{split}$ (1.19)

Under (1.19), $\mu^{-1}(0)$ is a $d$-codimensional submanifold of
$M^{\prime}$. Let

$M_{G}^{\prime}:=\mu^{-1}(0)/G,\quad M_{G}:=(\mu^{-1}(0)\cap M)/G.$ (1.20)

In Theorem 2.6 below, we will show that $M_{G}$ is a complex manifold in
$M^{\prime}_{G}$ with smooth boundary $X_{G}$. In fact,
$X_{G}=\widehat{X}_{G}\cup\widetilde{X}_{G}$ and thus the the boundary $X_{G}$
is non-degenerate, hence $M_{G}$ is a domain in the complex manifold
$M^{\prime}_{G}$ with non-degenerate boundary.

Let $\iota:\mu^{-1}(0)\cap\overline{M}\hookrightarrow\overline{M}$ be the
natural inclusion and let $\iota^{*}:C^{\infty}(\overline{M})\to
C^{\infty}(\mu^{-1}(0)\cap\overline{M})$ be the pull-back by $\iota$. Let
$\iota_{G}:C^{\infty}(\mu^{-1}(0)\cap\overline{M})^{G}\to
C^{\infty}(\overline{M}_{G})$ be the natural identification. Let

$\sigma_{G}:H^{0}(\overline{M})^{G}\cap C^{\infty}(\overline{M})\to
H^{0}(\overline{M}_{G}),\qquad\sigma_{G}=\iota_{G}\circ\iota^{*}.$ (1.21)

The map (1.21) is well defined, see the construction of the complex reduction
in Section 2.3. The map $\sigma_{G}$ does not extend to a bounded operator on
$L^{2}$, so it is necessary to consider its extension to Sobolev spaces.
Actually, we have

$\sigma_{G}=P_{M_{G}}\sigma_{1}(P^{*}P)^{-1}P^{*}\ \ \mbox{on
$H^{0}(\overline{M})^{G}\cap C^{\infty}(\overline{M})$},$

where $P_{M_{G}}$ and $P$ are Poisson operators on $M$ and $M_{G}$
respectively and $\sigma_{1}$ is the CR Guillemin-Sternberg map introduced in
[19, (1.5)]. From [19, Theorem 5.3] and the regularity property for Poisson
operator (see Section 4), we can check that $\sigma_{G}$ extends by density to
a bounded operator

$\sigma_{G}=\sigma_{G,s}:H^{0}(\overline{M})^{G}_{s}\to
H^{0}(\overline{M}_{G})_{s-\frac{d}{4}},~{}\text{for every $s\in\mathbb{R}$}.$
(1.22)

This operator can be thought as a Guillemin-Sternberg map in the setting of
complex manifolds with boundary. It maps the “first quantize and then reduce”
space (the space of $G$-invariant Sobolev holomorphic functions on
$\overline{M}$) to the “first reduce and then quantize” space (the space of
Sobolev holomorphic functions on $\overline{M}_{G}$). Indeed, from the point
of view of quantum mechanics, the Hilbert space structures play an essential
role. It is natural, then, to investigate the extent to which the holomorphic
Guillemin-Sternberg map is Fredholm. Let $(\,\cdot\,|\,\cdot\,)_{M_{G}}$ be
the $L^{2}$ inner product on $L^{2}(M_{G})$ induced naturally by
$\langle\,\cdot\,|\,\cdot\,\rangle$. For every $s\in\mathbb{R}$, put

${\rm Coker\,}\sigma_{G,s}={\rm Coker\,}\sigma_{G}:=\\{u\in
H^{0}(\overline{M}_{G})_{s-\frac{d}{4}};\,(\,u\,|\,\sigma_{G}v)_{M_{G}}=0,\forall
v\in H^{0}(\overline{M})^{G}_{s}\cap C^{\infty}(\overline{M})\\}.$ (1.23)

The third main result of this work is the following.

######  Theorem 1.4.

Let $M$ be a relatively compact open subset with smooth boundary $X$ of a
complex manifold $M^{\prime}$ of dimension $n$, $n\geq 3$. Let $G$ be a
compact Lie group acting on $M^{\prime}$ such that Assumption 1.1 and (1.19)
hold. With the notations used above, assume that ${\rm
dim\,}_{\mathbb{C}}M_{G}\geq 3$. Then, for every $s\in\mathbb{R}$, the
holomorphic Guillemin-Sternberg map (1.22) is Fredholm. More precisely,
$\operatorname{Ker}\sigma_{G,s}$ and ${\rm Coker\,}\sigma_{G,s}$ are finite
dimensional subspaces of $H^{0}(\overline{M})^{G}\cap
C^{\infty}(\overline{M})^{G}$ and $H^{0}(\overline{M}_{G})\cap
C^{\infty}(M_{G})$ respectively, $\operatorname{Ker}\sigma_{G,s}$ and ${\rm
Coker\,}\sigma_{G,s}$ are independent of $s$.

In should be mentioned that the condition ${\rm dim\,}_{\mathbb{C}}M_{G}\geq
3$ in Theorem 1.4 can be replaced by $\overline{\partial}_{b,X_{G}}$ has
closed range.

Until further notice, we will not assume (1.19).

Suppose that $M^{\prime}$ admits another compact holomorphic Lie group action
$H$ such that $H$ commutes with $G$ and $H$ preserves the boundary $X$. Recall
that $\mu^{-1}(0)\cap X=\widehat{X}\cup\widetilde{X}$ on which the Levi form
is strongly pseudoconvex and pseudoconcave, respectively. Let

$\mathcal{R}=\left\\{\mathcal{R}_{m};\,m=1,2,\ldots\right\\}$

denote the set of all irreducible unitary representations of the group $H$,
including only one representation from each equivalence class. For each
$\mathcal{R}_{m}$, we write $\mathcal{R}_{m}$ as a matrix
$\left(\mathcal{R}_{m,j,k}\right)^{d_{m}}_{j,k=1}$, where $d_{m}$ is the
dimension of $\mathcal{R}_{m}$. Fix a Haar measure $d\nu(h)$ on $H$ so that
$\int_{H}d\nu(h)=1$. Take an irreducible unitary representation
$\mathcal{R}_{m}$, for every $h\in H$, put

$\chi_{m}(h):={\rm
Tr\,}\left(\mathcal{R}_{m,j,k}(h)\right)^{d_{m}}_{j,k=1}=\sum^{d_{m}}_{j=1}\mathcal{R}_{m,j,j}(h).$

Let $u\in C^{\infty}(M^{\prime})$ be a smooth function. The $m$-th Fourier
component of $u$ is given by

$u_{m}(x):=d_{m}\int_{H}(h^{*}u)(x)\overline{\chi_{m}(h)}d\nu(h)\in
C^{\infty}(M^{\prime}).$

For every $m\in\mathbb{N}$, put

$C^{\infty}_{m}(M^{\prime}):=\left\\{f\in C^{\infty}(M^{\prime});\,\mbox{there
is an $F\in C^{\infty}(M^{\prime})$ such that $f=F_{m}$ on
$M^{\prime}$}\right\\}.$

For every $m\in\mathbb{N}$, we define $C^{\infty}_{m}(\overline{M})$,
$C^{\infty}_{m}(\widehat{X}_{G})$ in the standard way. For every
$m\in\mathbb{N}$, let

$\begin{split}&H^{0}(\overline{M})_{(m)}:=H^{0}(\overline{M})\cap
C^{\infty}_{m}(\overline{M}),\\\
&H^{0}(\overline{M})^{G}_{(m)}:=H^{0}(\overline{M})^{G}\cap
C^{\infty}_{m}(\overline{M}),\\\
&H^{0}_{b}(\widehat{X}_{G})_{(m)}:=H^{0}_{b}(\widehat{X}_{G})\cap
C^{\infty}_{m}(\widehat{X}_{G}).\end{split}$ (1.24)

Let $\mathfrak{h}$ denote the Lie algebra of $H$. For any
$\xi\in\mathfrak{h}$, as (1.6), we write $\xi_{M^{\prime},H}$ to denote the
vector field on $M^{\prime}$ induced by $\xi$. For $x\in M^{\prime}$, set

$\underline{\mathfrak{h}}_{x}={\rm
Span\,}\left\\{\xi_{M^{\prime},H}(x);\,\xi\in\mathfrak{h}\,\right\\}.$ (1.25)

We assume that

$T^{1,0}_{x}\widehat{X}\oplus
T^{0,1}_{x}\widehat{X}\oplus\underline{\mathfrak{h}}_{x}=\mathbb{C}T_{x}\widehat{X},\
\ \mbox{for every $x\in\widehat{X}$},$ (1.26)

where
$T^{1,0}_{x}\widehat{X}:=T^{1,0}_{x}M^{\prime}\cap\mathbb{C}T_{x}\widehat{X}$,
$T^{0,1}_{x}\widehat{X}:=T^{0,1}_{x}M^{\prime}\cap\mathbb{C}T_{x}\widehat{X}$,
$T^{1,0}M^{\prime}$ and $T^{0,1}M^{\prime}$ denotes the holomorphic tangent
bundle of $M^{\prime}$ and the anti-holomorphic tangent bundle of $M^{\prime}$
respectively. We can repeat the proof of [15, Theorem 3.1, Appendix] with
minor change and deduce that

$\begin{split}&\mbox{${\rm dim\,}H^{0}(\overline{M})_{(m)}<+\infty$, ${\rm
dim\,}H^{0}(\overline{M})^{G}_{(m)}<+\infty$, ${\rm
dim\,}H^{0}_{b}(\widehat{X}_{G})_{(m)}<+\infty$, for every
$m\in\mathbb{N}$},\\\
&H^{0}(\overline{M})=\oplus_{m\in\mathbb{N}}H^{0}(\overline{M})_{(m)},\ \
H^{0}(\overline{M})^{G}=\oplus_{m\in\mathbb{N}}H^{0}(\overline{M})^{G}_{(m)},\
\
H^{0}_{b}(\widehat{X}_{G})=\oplus_{m\in\mathbb{N}}H^{0}_{b}(\widehat{X}_{G})_{(m)}.\end{split}$
(1.27)

From Theorem 1.3, Theorem 1.4 and (1.27), we deduce

######  Theorem 1.5.

With the same assumptions used in Theorem 1.3, suppose that $M^{\prime}$
admits another compact holomorphic Lie group action $H$ such that $H$ commutes
with $G$ and $H$ preserves the boundary $X$. Under the same notations above
and assume that (1.26) holds. Then, for $\lvert m\rvert\gg 1$, we have

${\rm dim\,}H^{0}(\overline{M})^{G}_{(m)}={\rm
dim\,}H^{0}_{b}(\widehat{X}_{G})_{(m)}.$

Assume further that (1.19) holds. Then, for $\lvert m\rvert\gg 1$, we have

${\rm dim\,}H^{0}(\overline{M})^{G}_{(m)}={\rm
dim\,}H^{0}(\overline{M}_{G})_{(m)}.$

As an application of Theorem 1.2, we establish $G$-invariant version of
Fefferman’s result about regularity of biholomorphic maps. Let $M_{1}$,
$M_{2}$ be bounded domains in $\mathbb{C}^{n}$. Assume that $M_{j},j=1,2$
admit a compact holomorphic Lie group action $G$. Let $F:M_{1}\rightarrow
M_{2}$ be a holomorphic map. $F$ is said to be $G$-invariant if $F(g\circ
z)=F(z)$, for all $z\in M_{1}$ and $g\in G$. Then from Theorem 1.2 and by
using the argument in [3], we have (see Section 6, for the details)

######  Theorem 1.6.

Let $M_{1}$, $M_{2}$ be bounded domains in $\mathbb{C}^{n}$ with smooth
boundary, $n\geq 3$. Assume that $M_{j}$ admits a compact holomorphic Lie
group action $G$ and Assumption 1.1 holds, for each $j=1,2$. Let
$F:M_{1}\rightarrow M_{2}$ be a $G$-invariant holomorphic map. Assume that the
induced map of $F$ on the quotient space still denoted by
$F:M_{1}/G\rightarrow M_{2}/G$ is onto, one-to-one and the differential of $F$
is invertible everywhere on the regular part of $M_{1}/G$. Then, $F$ extends
smoothly to the boundary.

In the end of this section, we give a simple example. Let

$M:=\left\\{(z_{1},z_{2},z_{3},z_{4})\in\mathbb{C}^{4};\,\lvert
z_{1}\rvert^{4}+\sum^{4}_{j=2}\lvert z_{j}\rvert^{2}<1\right\\}.$

$M$ admits a $S^{1}$-action:

$S^{1}\times M\to M,\ \
e^{i\theta}\cdot(z_{1},\ldots,z_{4})=(e^{-i\theta}z_{1},e^{i\theta}z_{2},\ldots,e^{i\theta}z_{4}).$

We can show that Assumption 1.1 holds in this example (see [19, Section 2.5]
for the details). Moreover, it is straightforward to check that
$0\in\mathbb{C}^{4}$ is a critical point of $\mu$ and hence (1.19) does not
hold. In this example, we have Theorem 1.2 and Theorem 1.3. Since $M_{G}$ has
singularities, we do not know if we have Theorem 1.5. It is quite interesting
to see if we have Theorem 1.5 for singular reduction.

Let us consider the shell domain

$M:=\left\\{(z_{1},z_{2},z_{3},z_{4})\in\mathbb{C}^{4};\,\frac{1}{2}<\lvert
z_{1}\rvert^{4}+\sum^{4}_{j=2}\lvert z_{j}\rvert^{2}<1\right\\}.$

Then, Assumption 1.1 and (1.19) hold in this example and we have Theorem 1.2,
Theorem 1.3 and Theorem 1.5 for this example.

##  2\. Preliminaries

###  2.1. Some standard notations

We use the following notations: $\mathbb{N}=\left\\{1,2,\ldots\right\\}$,
$\mathbb{N}_{0}=\mathbb{N}\cup\left\\{0\right\\}$, $\mathbb{R}$ is the set of
real numbers, $\overline{\mathbb{R}}_{+}:=\left\\{x\in\mathbb{R};\,x\geq
0\right\\}$. For a multiindex
$\alpha=(\alpha_{1},\ldots,\alpha_{m})\in\mathbb{N}_{0}^{m}$, we set
$\lvert\alpha\rvert=\alpha_{1}+\cdots+\alpha_{m}$. For
$x=(x_{1},\ldots,x_{m})\in\mathbb{R}^{m}$ we write

$\begin{split}&x^{\alpha}=x_{1}^{\alpha_{1}}\ldots
x^{\alpha_{m}}_{m},\quad\partial_{x_{j}}=\frac{\partial}{\partial
x_{j}}\,,\quad\partial^{\alpha}_{x}=\partial^{\alpha_{1}}_{x_{1}}\ldots\partial^{\alpha_{m}}_{x_{m}}=\frac{\partial^{\lvert\alpha\rvert}}{\partial
x^{\alpha}}\,,\\\ &D_{x_{j}}=\frac{1}{i}\partial_{x_{j}}\,,\quad
D^{\alpha}_{x}=D^{\alpha_{1}}_{x_{1}}\ldots D^{\alpha_{m}}_{x_{m}}\,,\quad
D_{x}=\frac{1}{i}\partial_{x}\,.\end{split}$

Let $z=(z_{1},\ldots,z_{m})$, $z_{j}=x_{2j-1}+ix_{2j}$, $j=1,\ldots,m$, be
coordinates of $\mathbb{C}^{m}$, where
$x=(x_{1},\ldots,x_{2m})\in\mathbb{R}^{2m}$ are coordinates in
$\mathbb{R}^{2m}$. We write

$\begin{split}&z^{\alpha}=z_{1}^{\alpha_{1}}\ldots
z^{\alpha_{m}}_{m}\,,\quad\overline{z}^{\alpha}=\overline{z}_{1}^{\alpha_{1}}\ldots\overline{z}^{\alpha_{m}}_{m}\,,\\\
&\partial_{z_{j}}=\frac{\partial}{\partial
z_{j}}=\frac{1}{2}\Big{(}\frac{\partial}{\partial
x_{2j-1}}-i\frac{\partial}{\partial
x_{2j}}\Big{)}\,,\quad\partial_{\overline{z}_{j}}=\frac{\partial}{\partial\overline{z}_{j}}=\frac{1}{2}\Big{(}\frac{\partial}{\partial
x_{2j-1}}+i\frac{\partial}{\partial x_{2j}}\Big{)},\\\
&\partial^{\alpha}_{z}=\partial^{\alpha_{1}}_{z_{1}}\ldots\partial^{\alpha_{m}}_{z_{m}}=\frac{\partial^{\lvert\alpha\rvert}}{\partial
z^{\alpha}}\,,\quad\partial^{\alpha}_{\overline{z}}=\partial^{\alpha_{1}}_{\overline{z}_{1}}\ldots\partial^{\alpha_{m}}_{\overline{z}_{m}}=\frac{\partial^{\lvert\alpha\rvert}}{\partial\overline{z}^{\alpha}}\,.\end{split}$

Let $\Omega$ be a $C^{\infty}$ orientable paracompact manifold. We let
$T\Omega$ and $T^{*}\Omega$ denotes the tangent bundle of $\Omega$ and the
cotangent bundle of $\Omega$ respectively. The complexified tangent bundle of
$\Omega$ and the complexified cotangent bundle of $\Omega$ will be denoted by
$\mathbb{C}T\Omega$ and $\mathbb{C}T^{*}\Omega$ respectively. We write
$\langle\,\cdot\,,\cdot\,\rangle$ to denote the pointwise duality between
$T\Omega$ and $T^{*}\Omega$. We extend $\langle\,\cdot\,,\cdot\,\rangle$
bilinearly to $\mathbb{C}T\Omega\times\mathbb{C}T^{*}\Omega$.

Let $E$ be a $C^{\infty}$ complex vector bundle over $\Omega$. The fiber of
$E$ at $x\in\Omega$ will be denoted by $E_{x}$. Let $F$ be another vector
bundle over $\Omega$. We write $F\boxtimes E^{*}$ to denote the vector bundle
over $\Omega\times\Omega$ with fiber over $(x,y)\in\Omega\times\Omega$
consisting of the linear maps from $E_{y}$ to $F_{x}$.

Let $Y\subset\Omega$ be an open set. The spaces of smooth sections of $E$ over
$Y$ and distribution sections of $E$ over $Y$ will be denoted by
$C^{\infty}(Y,E)$ and $\mathscr{D}^{\prime}(Y,E)$ respectively. Let
$\mathscr{E}^{\prime}(Y,E)$ be the subspace of $\mathscr{D}^{\prime}(Y,E)$
whose elements have compact support in $Y$ and set
$C^{\infty}_{c}(Y,E):=C^{\infty}(Y,E)\bigcap\mathscr{E}^{\prime}(Y,E)$. Fix a
volume form on $Y$ and a Hermitian metric on $E$, we get a natural $L^{2}$
inner product $(\,\cdot\,|\,\cdot\,)$ on $C^{\infty}_{c}(Y,E)$. Let
$L^{2}(Y,E)$ be the completion of $C^{\infty}_{c}(Y,E)$ with respect to
$(\,\cdot\,|\,\cdot\,)$ and the $L^{2}$ inner product $(\,\cdot\,|\,\cdot\,)$
can be extended to $L^{2}(Y,E)$ by density. Let $\left\|\cdot\right\|$ be the
$L^{2}$ norm corresponding to the $L^{2}$ inner product
$(\,\cdot\,|\,\cdot\,)$. For every $s\in\mathbb{R}$, let
$L_{s}:\mathscr{D}^{\prime}(Y,E)\rightarrow\mathscr{D}^{\prime}(Y,E)$ be a
properly supported classical elliptic pseudodifferential operator of order $s$
on $Y$ with values in $E$. Define

$W^{s}(Y,E):=\left\\{u\in\mathscr{D}^{\prime}(Y,E);\,L_{s}u\in
L^{2}(Y,E)\right\\}$

and for $u\in W^{s}(Y,E)$, let $\left\|u\right\|_{s}:=\left\|L_{s}u\right\|$.
We call $W^{s}(Y,E)$ the Sobolev space of order $s$ of sections of $E$ over
$Y$ (with respect to $L_{s}$) and for $u\in W^{s}(Y,E)$, we call the number
$\left\|u\right\|_{s}$ the Sobolev norm of $u$ of order $s$ (with respect to
$L_{s}$). Put

$\displaystyle W^{s}_{\rm
loc\,}(Y,E)=\big{\\{}u\in\mathscr{D}^{\prime}(Y,E);\,\varphi u\in
W^{s}(Y,E),\,\forall\varphi\in C^{\infty}_{c}(Y)\big{\\}}\,,$ $\displaystyle
W^{s}_{\rm comp\,}(Y,E)=W^{s}_{\rm loc}(Y,E)\cap\mathscr{E}^{\prime}(Y,E)\,.$

Let $U,V$ be open sets of $\Omega$. Let
$F:C^{\infty}_{c}(V)\to\mathscr{D}^{\prime}(U)$ be a continuous operator and
let $F(x,y)\in\mathscr{D}^{\prime}(U\times V)$ be the distribution kernel of
$F$. In this work, we will identify $F$ with $F(x,y)$. We say that $F$ is a
smoothing operator if $F(x,y)\in C^{\infty}(U\times V)$. Note that the
following conditions are equivalent.

$\begin{split}&F(x,y)\in C^{\infty}(U\times V).\\\ \
&F:\mathscr{E}^{\prime}(V)\to\mathscr{C}^{\infty}(U)~{}\text{is
continuous}.\\\ &F:W^{-s}_{\mathrm{comp}}(V)\to
W^{s}_{\mathrm{loc}}(U)~{}\text{is continuous for
all}~{}s\in\mathbb{N}_{0}.\end{split}$ (2.1)

For two continuous linear operators
$A,B:C^{\infty}_{c}(V)\to\mathscr{D}^{\prime}(U)$, we write $A\equiv B$ (on
$U\times V$) or $A(x,y)\equiv B(x,y)$ (on $U\times V$) if $A-B$ is a smoothing
operator, where $A(x,y),B(x,y)\in\mathscr{D}^{\prime}(U\times V)$ are the
distribution kernels of $A$ and $B$, respectively.

###  2.2. Complex manifolds with boundary

Let $M$ be a relatively compact open subset with smooth boundary $X$ of a
complex manifold $M^{\prime}$ of dimension $n$, $n\geq 3$. Let $\rho\in
C^{\infty}(M^{\prime},\mathbb{R})$ be a defining function of $X$, that is,

$X=\\{x\in M^{\prime};\,\rho(x)=0\\},\ \ M=\\{x\in M^{\prime};\,\rho(x)<0\\}$

and $d\rho(x)\neq 0$ at every point $x\in X$. Then the manifold $X$ is a CR
manifold with natural CR structure
$T^{1,0}X:=T^{1,0}M^{\prime}\cap\mathbb{C}TX$, where $T^{1,0}M^{\prime}$
denotes the holomorphic tangent bundle of $M^{\prime}$. Let
$T^{0,1}M^{\prime}:=\overline{T^{1,0}M^{\prime}}$,
$T^{0,1}X:=\overline{T^{1,0}X}$.

Assume that $M^{\prime}$ admits a holomorphic $d$-dimensional compact Lie
group $G$ action. From now on, we will use the same assumptions and notations
as in Section 1. Recall that we work with Assumption 1.1. We take a
$G$-invariant Hermitian metric $\langle\cdot\,|\,\cdot\,\rangle$ on
$\mathbb{C}TM^{\prime}$. The $G$-invariant Hermitian metric
$\langle\cdot\,|\,\cdot\,\rangle$ on $\mathbb{C}TM^{\prime}$ induces a
$G$-invariant Hermitian metric $\langle\cdot\,|\,\cdot\,\rangle$ on
$\mathbb{C}T^{*}M^{\prime}$.

From now on, we fix a defining function $\rho\in
C^{\infty}(M^{\prime},\mathbb{R})$ of $X$ such that

$\begin{split}&\mbox{$\langle\,d\rho(x)\,|\,d\rho(x)\,\rangle=1$ on $X$},\\\
&\rho(g\circ x)=\rho(x),\ \ \forall x\in M^{\prime},\ \ \forall g\in
G.\end{split}$ (2.2)

Let $\frac{\partial}{\partial\rho}\in C^{\infty}(X,TM^{\prime})$ be the global
real vector field on $X$ given by

$\begin{split}&\Big{\langle}\frac{\partial}{\partial\rho},d\rho\,\Big{\rangle}=1~{}\text{on}~{}X,\\\
&\Big{\langle}\frac{\partial}{\partial\rho}(p)\,\big{|}\,v\,\Big{\rangle}=0~{}\text{at
every}~{}p\in X,~{}\text{for every}~{}v\in T_{p}X.\end{split}$ (2.3)

Let $J:TM^{\prime}\to TM^{\prime}$ be the complex structure map and put

$T=J\left(\frac{\partial}{\partial\rho}\,\right)\in
C^{\infty}(X,TM^{\prime}\,).$ (2.4)

The $G$-invariant Hermitian metric $\langle\,\cdot\,|\,\cdot\,\rangle$ on
$\mathbb{C}TM^{\prime}$ induces by duality a Hermitian metric on
$\mathbb{C}T^{*}M^{\prime}$ and Hermitian metrics on $T^{*0,q}M^{\prime}$ the
bundle of $(0,q)$ forms on $M^{\prime}$, $q=1,\ldots,n$. We shall also denote
these Hermitian metrics by $\langle\,\cdot\,|\,\cdot\,\rangle$. Put

$T^{*1,0}X:=(T^{0,1}X\oplus\mathbb{C}T)^{\perp}\subset\mathbb{C}T^{*}X,\quad
T^{*0,1}X:=(T^{1,0}X\oplus\mathbb{C}T)^{\perp}\subset\mathbb{C}T^{*}X.$

Put

$\omega_{0}=J^{t}(d\rho),$ (2.5)

where $J^{t}$ is the complex structure map for the cotangent bundle
$T^{*}M^{\prime}$. Then, on $X$, $\omega_{0}\in C^{\infty}(X,T^{*}X)$ is the
global one form on $X$ satisfying

$\begin{split}&\mbox{ $\langle\omega_{0}(p),u\,\rangle=0$, for every $p\in X$
and every $u\in T_{p}^{1,0}X\oplus T_{p}^{0,1}X$},\\\ &\mbox{
$\langle\omega_{0},T\,\rangle=-1$ on $X$.}\end{split}$ (2.6)

It is easy to see that under Assumption 1.1, the $G$-action preserves
$\omega_{0}$. We have the pointwise orthogonal decompositions:

$\begin{split}&\mathbb{C}T^{*}X=T^{*1,0}X\oplus
T^{*0,1}X\oplus\\{\lambda\omega_{0};\lambda\in\mathbb{C}\,\\},\\\
&\mathbb{C}TX=T^{1,0}X\oplus T^{0,1}X\oplus\\{\lambda
T;\lambda\in\mathbb{C}\,\\}.\end{split}$ (2.7)

For $p\in X$, the Levi form of $X$ at $p$ is the Hermitian quadratic form on
$T^{1,0}_{p}X$ given by

$\mathcal{L}_{p}(U,\overline{V})=-\frac{1}{2i}\langle\,d\omega_{0}(p)\,,\,U\wedge\overline{V}\,\rangle,\
\ \forall U,V\in T^{1,0}_{p}X.$ (2.8)

We can check that the Levi form on $X$ defined in (2.8) is exactly

$\mathcal{L}_{p}(U,\overline{V})=\langle\partial\overline{\partial}\rho(p)\,,\,U\wedge\overline{V}\,\rangle,\
\ U,V\in T^{1,0}_{p}X.$ (2.9)

######  Definition 2.1.

$M$ is called weakly (strongly) pseudoconvex at $x\in X$ if $\mathcal{L}_{x}$
is semi-positive (positive) definite on $T_{x}^{1,0}X$. If $\mathcal{L}_{x}$
is semi-positive (positive) definite at every point of $X$, then $M$ is called
a weakly (strongly) pseudoconvex manifold.

Let $A$ be a $C^{\infty}$ vector bundle over $M^{\prime}$. Let $U$ be an open
set in $M^{\prime}$. Let

$\begin{split}&C^{\infty}(U\cap\overline{M},A),\ \
\mathscr{D}^{\prime}(U\cap\overline{M},A),\ \
C^{\infty}_{c}(U\cap\overline{M},A),\ \
\mathscr{E}^{\prime}(U\cap\overline{M},A),\\\ W^{s}&(U\cap\overline{M},A),\ \
W^{s}_{{\rm comp\,}}(U\cap\overline{M},A),\ \ W^{s}_{{\rm
loc\,}}(U\cap\overline{M},A),\end{split}$

(where $\ s\in\mathbb{R}$) denote the spaces of restrictions to
$U\cap\overline{M}$ of elements in

$\begin{split}C^{\infty}&(U\cap M^{\prime},A),\ \ \mathscr{D}^{\prime}(U\cap
M^{\prime},A),\ \ C^{\infty}(U\cap M^{\prime},A),\ \
\mathscr{E}^{\prime}(U\cap M^{\prime},A),\\\ &W^{s}(M^{\prime},A),\ \
W^{s}_{{\rm comp\,}}(M^{\prime},A),\ \ W^{s}_{{\rm
loc\,}}(M^{\prime},A),\end{split}$

respectively. Write

$\begin{split}&L^{2}(U\cap
M,A)=L^{2}(U\cap\overline{M},A):=W^{0}(U\cap\overline{M},A),\\\ &L^{2}_{{\rm
comp\,}}(U\cap\overline{M},A):=W^{0}_{{\rm comp\,}}(U\cap\overline{M},A),\ \
L^{2}_{{\rm loc\,}}(U\cap\overline{M},A):=W^{0}_{{\rm
loc\,}}(U\cap\overline{M},A).\end{split}$

For every $q=0,\ldots,n$, we denote

$\begin{split}\Omega^{0,q}&(U\cap\overline{M}):=C^{\infty}(U\cap\overline{M},T^{*0,q}M^{\prime}),\
\ \Omega^{0,q}(M^{\prime}):=C^{\infty}(M^{\prime},T^{*0,q}M^{\prime}),\\\
&\Omega^{0,q}_{c}(U\cap\overline{M}):=C^{\infty}_{c}(U\cap\overline{M},T^{*0,q}M^{\prime}),\\\
&\Omega^{0,q}_{c}(M^{\prime}):=C^{\infty}_{c}(M^{\prime},T^{*p,q}M^{\prime}),\
\ \Omega^{0,q}_{c}(M):=C^{\infty}_{c}(M,T^{*p,q}M^{\prime}).\end{split}$

Let $A$ and $B$ be $C^{\infty}$ vector bundles over $M^{\prime}$. Let $U$ be
an open set in $M^{\prime}$. Let

$F_{1},F_{2}:C^{\infty}_{c}(U\cap M,A)\rightarrow\mathscr{D}^{\prime}(U\cap
M,B)$

be continuous operators. Let
$F_{1}(x,y),F_{2}(x,y)\in\mathscr{D}^{\prime}((U\times U)\cap(M\times
M),A\boxtimes B^{*})$ be the distribution kernels of $F_{1}$ and $F_{2}$
respectively. We write

$F_{1}\equiv F_{2}\\!\\!\mod C^{\infty}((U\times
U)\cap(\overline{M}\times\overline{M}))$

or $F_{1}(x,y)\equiv F_{2}(x,y)\\!\\!\mod C^{\infty}((U\times
U)\cap(\overline{M}\times\overline{M}))$ if $F_{1}(x,y)=F_{2}(x,y)+r(x,y)$,
where $r(x,y)\in C^{\infty}((U\times
U)\cap(\overline{M}\times\overline{M}),A\boxtimes B^{*})$. Similarly, let
$\hat{F}_{1},\hat{F}_{2}:C^{\infty}_{c}(U\cap
M,A)\rightarrow\mathscr{D}^{\prime}(U\cap X,B)$ be continuous operators. Let
$\hat{F}_{1}(x,y),\hat{F}_{2}(x,y)\in\mathscr{D}^{\prime}((U\times
U)\cap(X\times M),A\boxtimes B^{*})$ be the distribution kernels of
$\hat{F}_{1}$ and $\hat{F}_{2}$ respectively. We write
$\hat{F}_{1}\equiv\hat{F}_{2}\\!\\!\mod C^{\infty}((U\times
U)\cap(X\times\overline{M}))$ or
$\hat{F}_{1}(x,y)\equiv\hat{F}_{2}(x,y)\\!\\!\mod C^{\infty}((U\times
U)\cap(X\times\overline{M}))$ if
$\hat{F}_{1}(x,y)=\hat{F}_{2}(x,y)+\hat{r}(x,y)$, where $\hat{r}(x,y)\in
C^{\infty}((U\times U)\cap(X\times\overline{M}),A\boxtimes B^{*})$. Similarly,
let $\tilde{F}_{1},\tilde{F}_{2}:C^{\infty}_{c}(U\cap
X,A)\rightarrow\mathscr{D}^{\prime}(U\cap M,B)$ be continuous operators. Let

$\tilde{F}_{1}(x,y),\tilde{F}_{2}(x,y)\in\mathscr{D}^{\prime}((U\times
U)\cap(M\times X),A\boxtimes B^{*})$

be the distribution kernels of $\tilde{F}_{1}$ and $\tilde{F}_{2}$
respectively. We write $\tilde{F}_{1}\equiv\tilde{F}_{2}\\!\\!\mod
C^{\infty}((U\times U)\cap(\overline{M}\times X))$ or
$\tilde{F}_{1}(x,y)\equiv\tilde{F}_{2}(x,y)\\!\\!\mod C^{\infty}((U\times
U)\cap(\overline{M}\times X))$ if
$\tilde{F}_{1}(x,y)=\tilde{F}_{2}(x,y)+\tilde{r}(x,y)$, where
$\tilde{r}(x,y)\in C^{\infty}((U\times U)\cap(\overline{M}\times X),A\boxtimes
B^{*})$.

Let $(\,\cdot\,|\,\cdot\,)_{M^{\prime}}$ and $(\,\cdot\,|\,\cdot\,)_{M}$ be
the $L^{2}$ inner products on $\Omega^{0,q}_{c}(M^{\prime})$ and
$\Omega^{0,q}_{c}(M)$ respectively given by

$\begin{split}&(\,u\,|\,v\,)_{M^{\prime}}:=\int_{M^{\prime}}\langle\,u\,|\,v\,\rangle
dv_{M^{\prime}},\ \ u,v\in\Omega^{0,q}_{c}(M^{\prime}),\\\
&(\,u\,|\,v\,)_{M}:=\int_{M}\langle\,u\,|\,v\,\rangle dv_{M^{\prime}},\ \
u,v\in\Omega^{0,q}_{c}(M),\end{split}$ (2.10)

where $dv_{M^{\prime}}$ is the volume form on $M^{\prime}$ induced by
$\langle\,\cdot\,|\,\cdot\,\rangle$. Let $L^{2}_{(0,q)}(M)$ and
$L^{2}_{(0,q)}(M^{\prime})$ be the $L^{2}$ completions of
$\Omega^{0,q}_{c}(M)$ and $\Omega^{0,q}_{c}(M^{\prime})$ with respect to
$(\,\cdot\,|\,\cdot\,)_{M}$ and $(\,\cdot\,|\,\cdot\,)_{M^{\prime}}$
respectively. It is clear that $\Omega^{0,q}(\overline{M})\subset
L^{2}_{(0,q)}(M)$. We write $L^{2}(M):=L^{2}_{(0,0)}(M)$. We extend
$(\,\cdot\,|\,\cdot\,)_{M}$ and $(\,\cdot\,|\,\cdot\,)_{M^{\prime}}$ to
$L^{2}_{(0,q)}(M)$ and $L^{2}_{(0,q)}(M^{\prime})$ in the standard way and let
$\left\|\cdot\right\|_{M}$ and $\left\|\cdot\right\|_{M^{\prime}}$ be the
corresponding $L^{2}$ norms. Let $T^{*0,q}X$ be the bundle of $(0,q)$ forms on
$X$. Recall that for every $x\in X$, we have

$T^{*0,q}_{x}X:=\left\\{u\in
T^{*0,q}_{x}M^{\prime};\,\langle\,u\,|\,\overline{\partial}\rho(x)\wedge
g\,\rangle=0,\ \ \forall g\in T^{*0,q-1}_{x}M^{\prime}\right\\}.$

Let $\Omega^{0,q}(X)$ be the space of smooth $(0,q)$ forms on $X$. Let
$(\,\cdot\,|\,\cdot\,)_{X}$ be the $L^{2}$ inner product on $\Omega^{0,q}(X)$
given by

$(\,u\,|\,v\,)_{X}:=\int_{X}\langle\,u\,|\,v\,\rangle dv_{X},$ (2.11)

where $dv_{X}$ is the volume form on $X$ induced by
$\langle\,\cdot\,|\,\cdot\,\rangle$. Let $L^{2}_{(0,q)}(X)$ be the $L^{2}$
completion of $\Omega^{0,q}(X)$ with respect to $(\,\cdot\,|\,\cdot\,)_{X}$.
We extend $(\,\cdot\,|\,\cdot\,)_{X}$ to $L^{2}_{(0,q)}(X)$ in the standard
way and let $\left\|\cdot\right\|_{X}$ be the corresponding $L^{2}$ norm. We
write $L^{2}(X):=L^{2}_{(0,0)}(X)$.

Fix $g\in G$. Let
$g^{*}:\Lambda^{r}_{x}(\mathbb{C}T^{*}M^{\prime})\rightarrow\Lambda^{r}_{g^{-1}\circ
x}(\mathbb{C}T^{*}M^{\prime})$ be the pull-back map. Since $G$ preserves $J$,
we have

$g^{*}:T^{*0,q}_{x}M^{\prime}\rightarrow T^{*0,q}_{g^{-1}\circ x}M^{\prime},\
\ \forall x\in M^{\prime}.$

Thus, for $u\in\Omega^{0,q}(M^{\prime})$, we have
$g^{*}u\in\Omega^{0,q}(M^{\prime})$. Put

$\Omega^{0,q}(M^{\prime})^{G}:=\left\\{u\in\Omega^{0,q}(M^{\prime});\,g^{*}u=u,\
\ \forall g\in G\right\\}.$

Let $u\in L^{2}_{(0,q)}(M^{\prime})$ and $g\in G$, we can also define $g^{*}u$
in the standard way. Put

$L^{2}_{(0,q)}(M^{\prime})^{G}:=\left\\{u\in
L^{2}_{(0,q)}(M^{\prime});\,g^{*}u=u,\ \ \forall g\in G\right\\}.$

Let $\Omega^{0,q}(\overline{M})^{G}$ denote the space of restrictions to $M$
of elements in $\Omega^{0,q}(M^{\prime})^{G}$. Let $L^{2}_{(0,q)}(M)^{G}$ be
the completion of $\Omega^{0,q}(\overline{M})^{G}$ with respect to
$(\,\cdot\,|\,\cdot\,)_{M}$. Similarly, let

$\Omega^{0,q}(X)^{G}:=\\{u\in\Omega^{0,q}(X);\,g^{*}u=u,\ \ \forall g\in
G\\}.$ (2.12)

Let $L^{2}_{(0,q)}(X)^{G}$ be the completion of $\Omega^{0,q}(X)^{G}$ with
respect to $(\,\cdot\,|\,\cdot\,)_{X}$. We write
$L^{2}(X)^{G}:=L^{2}_{(0,0)}(X)^{G}$, $L^{2}(M)^{G}:=L^{2}_{(0,0)}(M)^{G}$,
$L^{2}(M^{\prime})^{G}:=L^{2}_{(0,0)}(M^{\prime})^{G}$.

For $s\in\mathbb{R}$, we also use $\left\|\cdot\right\|_{s,X}$ to denote the
standard Sobolev norm on $X$ of order $s$. Let $A$ be a vector bundle over
$M^{\prime}$. Let $u\in W^{s}(\overline{M},A)$. We define

$\left\|u\right\|_{s,\overline{M}}:=\inf\left\\{\left\|\widetilde{u}\right\|_{s,M^{\prime}};\,u^{\prime}\in
W^{s}(M^{\prime},A),u^{\prime}|_{M}=u\right\\}.$

We call $\left\|u\right\|_{s,\overline{M}}$ the Sobolev norm of $u$ of order
$s$ on $\overline{M}$.

Let $s$ be a non-negative integer. We can also define Sobolev norm of order
$s$ on $\overline{M}$ as follows: Let $x_{0}\in X$ and let $U$ be an open
neighborhood of $x_{0}$ in $M^{\prime}$ with local coordinates
$x=(x_{1},\ldots,x_{2n})$. Let
$u\in\mathscr{E}^{\prime}(U\cap\overline{M})\bigcap W^{s}(\overline{M},A)$.
Let $\widetilde{u}\in\mathscr{E}^{\prime}(U)\bigcap W^{s}(M^{\prime},A)$ with
$\widetilde{u}|_{M}=u$. We define the Sobolev norm of order $s$ of $u$ on
$\overline{M}$ by

$\left\|u\right\|^{2}_{(s),\overline{M}}:=\sum_{\alpha\in\mathbb{N}_{0}^{2n},\lvert\alpha\rvert\leq
s}\int_{M}\lvert\partial^{\alpha}_{x}\widetilde{u}\rvert^{2}dv_{M^{\prime}}.$
(2.13)

By using partition of unity, for $u\in W^{s}(\overline{M},A)$, we define
$\left\|u\right\|^{2}_{(s),\overline{M}}$ in the standard way. As in function
case, we define $\left\|u\right\|^{2}_{(s),\overline{M}}$, for $u\in
W^{s}(\overline{M},A)$ in the similar way. It is well-known (see [12,
Corollary B.2.6]) that the two norms $\left\|\cdot\right\|_{s,\overline{M}}$
and $\left\|\cdot\right\|_{(s),\overline{M}}$ are equivalent for every non-
negative integer $s$.

###  2.3. The reduction of complex manifolds with boundary

As before, let $\mathfrak{g}$ denote the Lie algebra of $G$ and for any
$\xi\in\mathfrak{g}$, we write $\xi_{M^{\prime}}$ to denote the vector field
on $M^{\prime}$ induced by $\xi$ (see (1.6)). For $x\in M^{\prime}$, recall
that $\underline{\mathfrak{g}}_{x}$ is given by (1.7).

######  Definition 2.2.

The moment map associated to the form $\omega_{0}$ is the map
$\mu:M^{\prime}\to\mathfrak{g}^{*}$ defined by

$\langle\mu(x),\xi\rangle=\omega_{0}(\xi_{M^{\prime}}(x)),\qquad x\in
M^{\prime},\quad\xi\in\mathfrak{g}.$ (2.14)

The proof of the following lemma is standard, cf. for example, [1, Theorem 6].

######  Lemma 2.3.

The moment map $\mu:M^{\prime}\to\mathfrak{g}^{*}$ is $G$-equivariant, so $G$
acts on $Y^{\prime}:=\mu^{-1}(0)$, where $G$ acts on $\mathfrak{g}^{*}$
through co-adjoint representation.

###### Proof.

For all $g\in G$, $\xi\in\mathfrak{g}$ and $x\in M^{\prime}$, we have

$\displaystyle\xi_{M^{\prime}}(g\circ x)$ $\displaystyle=$
$\displaystyle\frac{d}{dt}\left(\exp(t\xi)\circ g\circ x\right)|_{t=0}$
$\displaystyle=$ $\displaystyle\frac{d}{dt}\left(g\circ
g^{-1}\circ\exp(t\xi)\circ g\circ x\right)|_{t=0}$ $\displaystyle=$
$\displaystyle
g_{*}\left(\operatorname{Ad}(g^{-1})\circ\xi\right)_{M^{\prime}}(x)$

and hence

$\displaystyle\langle\mu(g\circ x),\xi\rangle$ $\displaystyle=$
$\displaystyle\omega_{0}(\xi_{M^{\prime}}(g\circ x))\qquad\text{by}\
\eqref{E:cmpm}$ $\displaystyle=$
$\displaystyle\omega_{0}\left(g_{*}\left(\operatorname{Ad}(g^{-1})\circ\xi\right)_{M^{\prime}}(x)\right)\qquad\text{by}\
\eqref{e-que04062019}$ $\displaystyle=$
$\displaystyle\omega_{0}\left(\left(\operatorname{Ad}(g^{-1})\circ\xi\right)_{M^{\prime}}(x)\right)\qquad\text{by
$G$-invariance of $\omega$}$ $\displaystyle=$
$\displaystyle\langle\mu(x),\operatorname{Ad}(g^{-1})\circ\xi\rangle\qquad\text{by}\
\eqref{E:cmpm}$ $\displaystyle=$
$\displaystyle\langle\operatorname{Ad}(g)^{*}\mu(x),\xi\rangle.$

Thus, the moment map $\mu$ is $G$-equivariant. ∎

Note that $\mu_{X}=\mu|_{X}$ is the CR moment map associated to $\omega_{0}$
on $X$, cf. [16, 19]. Suppose that $\mu_{X}^{\,-1}(0)\not=\emptyset$, then it
is shown, in [19, Lemma 2.5], that if $G$ acts freely on $\mu_{X}^{\,-1}(0)$
and the Levi form is positive on $\mu_{X}^{\,-1}(0)$, then $0$ is a regular
value of $\mu_{X}$.

Set $Y^{\prime}_{G}:=\mu^{-1}(0)/G$. In this section, we assume that (1.19)
holds. $\mu^{-1}(0)$ is a smooth manifold. Since $G$ acts freely on
$Y^{\prime}$, $Y^{\prime}_{G}$ is a smooth manifold. Let

$g^{TM^{\prime}}=d\omega_{0}(\cdot,J\cdot).$

Then, $g^{TM^{\prime}}$ is a non-degenerate quadratic form on $TM^{\prime}$
near $\mu^{-1}(0)$. Let $T^{H}Y^{\prime}$ be the orthogonal complement of
$\underline{\mathfrak{g}}_{Y^{\prime}}$ in $TY^{\prime}$ with respect to
$g^{TM^{\prime}}$, where
$\underline{\mathfrak{g}}_{Y^{\prime}}:=\underline{\mathfrak{g}}|_{Y^{\prime}}$.
Then we have

$TY^{\prime}=T^{H}Y^{\prime}\oplus\underline{\mathfrak{g}}_{Y^{\prime}}.$
(2.16)

######  Lemma 2.4.

We have

$JT^{H}Y^{\prime}=T^{H}Y^{\prime}=JTY^{\prime}\cap TY^{\prime}.$

###### Proof.

Since $G$ acts freely on $Y^{\prime}$, then vector spaces
$\underline{\mathfrak{g}}_{x}$ defined in (1.7) form a vector bundle
$\underline{\mathfrak{g}}$ near $\mu^{-1}(0)$.

For $x\in Y^{\prime}$, by (1.19) and the fact that
$d\omega_{0}(\cdot,J\cdot\,)$ is non-degenerate on $T_{x}M^{\prime}$, we have
that $d\mu|_{TY^{\prime}}=0$ and
$d\mu|_{J\underline{\mathfrak{g}}_{x}}\to\mathfrak{g}^{*}$ is surjective.
Since $\dim Y^{\prime}+\dim\underline{\mathfrak{g}}=\dim TM^{\prime}$, we have

$J\underline{\mathfrak{g}}|_{Y^{\prime}}\oplus
TY^{\prime}=TM^{\prime}|_{Y^{\prime}}.$ (2.17)

By (2.16) and (2.17), we have the $G$-equivariant orthogonal decomposition on
$Y^{\prime}$,

$TM^{\prime}|_{Y^{\prime}}=\underline{\mathfrak{g}}|_{Y^{\prime}}\oplus
J\underline{\mathfrak{g}}|_{Y^{\prime}}\oplus T^{H}Y^{\prime}.$ (2.18)

Thus from (2.18) and $g^{TM^{\prime}}$ on $TM^{\prime}|_{Y^{\prime}}$ is
$J$-invariant, we get

$JT^{H}Y^{\prime}=T^{H}Y^{\prime}=JTY^{\prime}\cap TY^{\prime}.$ (2.19)

∎

Let $\pi:Y^{\prime}\to Y^{\prime}_{G}$ and $\iota:Y^{\prime}\hookrightarrow
M^{\prime}$ be the natural quotient and inclusion, respectively, then there is
a unique induced 1-form $\widetilde{\omega}_{0}$ on $Y^{\prime}_{G}$ such
$\pi^{*}\widetilde{\omega}_{0}=\iota^{*}\omega_{0}$.

Since $T^{H}Y^{\prime}$ is preserved by $J$, we can define the homomorphism
$J_{G}$ on $TY^{\prime}_{G}$ in the following way: For $V\in TY^{\prime}_{G}$,
we denote by $V^{H}$ its lift in $T^{H}Y^{\prime}$, and we define $J_{G}$ on
$Y^{\prime}_{G}$ by

$(J_{G}V)^{H}=J(V^{H}).$ (2.20)

Hence, we have $J_{G}:TY^{\prime}_{G}\to TY^{\prime}_{G}$ such that
$J_{G}^{2}=-\operatorname{id}$, where $\operatorname{id}$ denotes the identity
map $\operatorname{id}\,:\,TY^{\prime}_{G}\to TY^{\prime}_{G}.$ By complex
linear extension of $J_{G}$ to $\mathbb{C}TY^{\prime}_{G}$, the
$\sqrt{-1}$-eigenspace of $J_{G}$ is given by
$T^{1,0}Y^{\prime}_{G}\,=\,\left\\{V\in\mathbb{C}TY^{\prime}_{G}\,;\,J_{G}V\,=\,\sqrt{-1}V\right\\}.$

######  Lemma 2.5.

The almost complex structure $J_{G}$ is integrable, thus
$(Y^{\prime}_{G},J_{G})$ is a complex manifold.

###### Proof.

Let $u,v\in C^{\infty}(Y^{\prime}_{G},T^{1,0}Y^{\prime}_{G})$, then we can
find $U,V\in C^{\infty}(Y^{\prime}_{G},TY^{\prime}_{G})$ such that

$u=U-\sqrt{-1}J_{G}U,\qquad v=V-\sqrt{-1}J_{G}V.$

By (2.20), we have

$u^{H}=U^{H}-\sqrt{-1}JU^{H},\quad v=V^{H}-\sqrt{-1}JV^{H}\in
T^{1,0}X\cap\mathbb{C}TY^{\prime}.$

Since $T^{1,0}M^{\prime}$ is integrable and it is clearly that
$[u^{H},v^{H}]\in\mathbb{C}TY^{\prime},$ we have $[u^{H},v^{H}]\in
T^{1,0}M^{\prime}\cap\mathbb{C}TY^{\prime}.$ Hence, there is a $W\in
C^{\infty}(M^{\prime},TM^{\prime})$ such that

$[u^{H},v^{H}]=W-\sqrt{-1}JW.$

In particular, $W,JW\in TY^{\prime}$. Thus, $W\in TY^{\prime}\cap
JTY^{\prime}=T^{H}Y^{\prime}$. Let $X^{H}\in T^{H}Y^{\prime}$ be a lift of
$X\in TY^{\prime}_{G}$ such that $X^{H}=W$. Then we have

$[u,v]=\pi_{*}[u^{H},v^{H}]=\pi_{*}(X^{H}-\sqrt{-1}JX^{H})=X-\sqrt{-1}J_{G}X\in
T^{1,0}Y^{\prime}_{G},$

i.e. we have
$[C^{\infty}(Y^{\prime}_{G},T^{1,0}Y^{\prime}_{G}),C^{\infty}(Y^{\prime}_{G},T^{1,0}Y^{\prime}_{G})]\subset
C^{\infty}(Y^{\prime}_{G},T^{1,0}Y^{\prime}_{G}).$ Therefore, $J_{G}$ is
integrable. ∎

Let $M^{\prime}_{G}:=\mu^{-1}(0)/G$, $M_{G}:=(\mu^{-1}(0)\cap M)/G$,
$X_{G}:=(\mu^{-1}(0)\cap X)/G$. By combining Lemma 2.3 and Lemma 2.5, we have
the following

######  Theorem 2.6.

Under (1.19), $M^{\prime}_{G}$ is a complex manifold of dimension $n-d$ and
$M_{G}\subset M^{\prime}_{G}$ is a relatively compact open subset of
$M^{\prime}_{G}$ with smooth boundary $X_{G}$. In particular, the Levi form of
$X_{G}$ is negative or positive.

##  3\. $G$-invariant $\overline{\partial}$-Neumann problem

In this section, we will study $G$-invariant $\overline{\partial}$-Neumann
problem on $M$. Until further notice, we fix
$q\in\left\\{0,1,\ldots,n-1\right\\}$. Let
$\overline{\partial}:\Omega^{0,q}(\overline{M})\rightarrow\Omega^{0,q+1}(\overline{M})$
be the Cauchy-Riemann operator. We extend $\overline{\partial}$ to
$L^{2}_{(0,q)}(M)$:

$\overline{\partial}:{\rm Dom\,}\overline{\partial}\subset
L^{2}_{(0,q)}(M)\rightarrow L^{2}_{(0,q+1)}(M),$

where $u\in{\rm Dom\,}\overline{\partial}$ if we can find
$u_{j}\in\Omega^{0,q}(\overline{M})$, $j=1,2,\ldots$, such that
$u_{j}\rightarrow u$ in $L^{2}_{(0,q)}(M)$ as $j\rightarrow+\infty$ and there
is a $v\in L^{2}_{(0,q+1)}(M)$ such that $\overline{\partial}u_{j}\rightarrow
v$ as $j\rightarrow+\infty$. We set $\overline{\partial}u:=v$. Let

$\overline{\partial}^{\ast}:{\rm Dom\,}\overline{\partial}^{\ast}\subset
L^{2}_{(0,q+1)}(M)\rightarrow L^{2}_{(0,q)}(M)$

be the Hilbert adjoint of $\overline{\partial}$ with respect to
$(\,\cdot\,|\,\cdot\,)_{M}$. The Gaffney extension of the
$\overline{\partial}$-Neumann Laplacian is given by

$\Box^{(q)}:{\rm Dom\,}\Box^{(q)}\subset L^{2}_{(0,q)}(M)\rightarrow
L^{2}_{(0,q)}(M),$ (3.1)

where ${\rm Dom\,}\Box^{(q)}:=\\{u\in L^{2}_{(0,q)}(M);\,u\in{\rm
Dom\,}\overline{\partial}\cap{\rm
Dom\,}\overline{\partial}^{\ast},\overline{\partial}u\in{\rm
Dom\,}\overline{\partial}^{\ast},\overline{\partial}^{\ast}u\in{\rm
Dom\,}\overline{\partial}\\}$ and
$\Box^{(q)}u=(\overline{\partial}\,\overline{\partial}^{\ast}+\overline{\partial}^{\ast}\,\overline{\partial})u$,
$u\in{\rm Dom\,}\Box^{(q)}$. Put

${\rm Ker\,}\Box^{(q)}=\left\\{u\in{\rm
Dom\,}\Box^{(q)};\,\Box^{(q)}u=0\right\\}.$

It is easy to check that

${\rm Ker\,}\Box^{(q)}=\\{u\in{\rm
Dom\,}\Box^{(q)};\,\overline{\partial}u=0,\overline{\partial}^{\ast}u=0\\}.$
(3.2)

Since $G$ preserves $J$ and $(\,\cdot\,|\,\cdot\,)$ is $G$-invariant, it is
straightforward to see that

$\begin{split}&g^{*}\overline{\partial}=\overline{\partial}g^{*}\ \ \mbox{on
${\rm Dom\,}\overline{\partial}$},\\\
&g^{*}\overline{\partial}^{*}=\overline{\partial}^{*}g^{*}\ \ \mbox{on ${\rm
Dom\,}\overline{\partial}^{*}$},\\\ &g^{*}\Box^{(q)}=\Box^{(q)}g^{*}\ \
\mbox{on ${\rm Dom\,}\Box^{(q)}$}.\end{split}$ (3.3)

Put $({\rm Ker\,}\Box^{(q)})^{G}:={\rm Ker\,}\Box^{(q)}\bigcap
L^{2}_{(0,q)}(M)^{G}$.

Let $\overline{\partial}\rho^{\wedge}:T^{*0,q}M^{\prime}\rightarrow
T^{*0,q+1}M^{\prime}$ be the operator with wedge multiplication by
$\overline{\partial}\rho$ and let
$\overline{\partial}\rho^{\wedge,\ast}:T^{*0,q+1}M^{\prime}\rightarrow
T^{*0,q}M^{\prime}$ be its adjoint with respect to
$\langle\,\cdot\,|\,\cdot\,\rangle$, that is,

$\langle\,\overline{\partial}\rho\wedge
u\,|\,v\,\rangle=\langle\,u\,|\,\overline{\partial}\rho^{\wedge,\ast}v\,\rangle,\
\ u\in T^{*0,q}M^{\prime},\ \ v\in T^{*0,q+1}M^{\prime}.$ (3.4)

Denote by $\gamma$ the operator of restriction on $X$. By using the
calculation in page 13 of [10], we can check that

$\begin{split}&{\rm
Dom\,}\overline{\partial}^{\ast}\cap\Omega^{0,q+1}(\overline{M})=\\{u\in\Omega^{0,q+1}(\overline{M});\,\gamma\overline{\partial}\rho^{\wedge,\ast}u=0\\},\\\
&{\rm
Dom\,}\Box^{(q)}\cap\Omega^{0,q}(\overline{M})=\\{u\in\Omega^{0,q}(\overline{M});\,\gamma\overline{\partial}\rho^{\wedge,\ast}u=0,\gamma\overline{\partial}\rho^{\wedge,\ast}\overline{\partial}u=0\\}.\end{split}$
(3.5)

Let
$\overline{\partial}^{\ast}_{f}:\Omega^{0,q+1}(M^{\prime})\rightarrow\Omega^{0,q}(M^{\prime})$
be the formal adjoint of $\overline{\partial}$ with respect to
$(\,\cdot\,|\,\cdot\,)_{M^{\prime}}$, that is,

$(\,\overline{\partial}u\,|\,v\,)_{M^{\prime}}=(\,u\,|\,\overline{\partial}^{\ast}_{f}v\,)_{M^{\prime}},\
\ \forall u\in\Omega^{0,q}_{c}(M^{\prime}),\ \ \forall
v\in\Omega^{0,q+1}(M^{\prime}).$

It is easy to see that if $u\in{\rm
Dom\,}\overline{\partial}^{\ast}\cap\Omega^{0,q+1}(\overline{M})$, then
$\overline{\partial}^{\ast}u=\overline{\partial}^{\ast}_{f}u.$ Write
$\Box^{(q)}_{f}=\overline{\partial}\,\overline{\partial}_{f}^{\ast}+\overline{\partial}_{f}^{\ast}\overline{\partial}$.
Recall that we work with Assumption 1.1.

Let $\\{\omega^{j}\\}_{j=1}^{n}$ be an orthonormal basis of $T^{\star
1,0}M^{\prime}$ in a neighborhood of $X$ with
$\omega^{n}=\frac{\partial\rho}{\lvert\partial\rho\rvert}$. Let
$\\{L_{j}\\}_{j=1}^{n}$ be a dual frame of $\\{\omega^{j}\\}_{j=1}^{n}$ with
respect to $\langle\cdot|\cdot\rangle$. It is straightforward to check that

$T=\frac{i}{\sqrt{2}}(L_{n}-\bar{L}_{n}).$ (3.6)

Denote by $\bar{\partial}_{G}$ the operator $\bar{\partial}$ restricted on
$L^{2}_{(0,q)}(M^{\prime})^{G}$. As $\Box^{(q)}$, we can define the
$G$-invariant $\bar{\partial}$-Laplacian:

$\Box^{(q)}_{G}:=\bar{\partial}_{G}^{\ast}\bar{\partial}_{G}+\bar{\partial}_{G}\bar{\partial}_{G}^{\ast}:{\rm
Dom\,}\Box^{(q)}_{G}\subset L^{2}_{(0,q)}(M)^{G}\rightarrow
L^{2}_{(0,q)}(M)^{G}$ (3.7)

in the similar way, where $\bar{\partial}_{G}^{\ast}:{\rm
Dom\,}\bar{\partial}_{G}^{\ast}\subset L^{2}_{(0,q+1)}(M)^{G}\rightarrow
L^{2}_{(0,q)}(M)^{G}$ is the Hilbert space adjoint of $\bar{\partial}_{G}:{\rm
Dom\,}\bar{\partial}_{G}\subset L^{2}_{(0,q)}(M)^{G}\rightarrow
L^{2}_{(0,q+1)}(M)^{G}$.

######  Lemma 3.1.

Fix $q=1,\ldots,n-2$. We have

$\|u\|_{1,\overline{M}}\leq
C\Bigr{(}\|\Box_{G}^{(q)}u\|_{M}+\|u\|_{M}\Bigr{)},\forall u\in{\rm
Dom}\Box_{G}^{(q)}\cap\Omega_{G}^{0,q}(\overline{M}),$ (3.8)

where $C>0$ is a constant.

###### Proof.

Fix $p\in X$. Assume that $p\in\mu^{-1}(0)\cap X$. There exists a neighborhood
$V$ of $p$ in $X$ such that the Levi form is positive or negative on $V$. Let
$U$ be a neighborhood of $p$ in $M^{\prime}$ such that $U\cap X=V$. Let
$u\in{\rm Dom\,}\Box^{(q)}_{G}\cap\Omega^{0,q}(\overline{M})^{G}$. Let
$\chi\in C^{\infty}_{c}(U)$ and put $v:=\chi u$. Since the Levi form is
positive or negative on $V$ then one has

$\|v\|^{2}_{1,\overline{M}}\leq
C\Bigr{(}\|\overline{\partial}v\|^{2}_{M}+\|\overline{\partial}^{\ast}v\|^{2}_{M}+\|v\|^{2}_{M}\Bigr{)},$
(3.9)

where $C>0$ is a constant independent of $u$ ($C$ depends on $\chi$).

Now assume $p\notin\mu^{-1}(0)\cap X$. Then there exists a neighborhood $U$ of
$p$ in $M^{\prime}$ such that $\mu(\tilde{p})\neq 0,\forall\tilde{p}\in U$.
Moreover, we assume that $\overline{U}\cap\mu^{-1}(0)=\emptyset$. Let
$z=(z_{1},\ldots,z_{n})$ be holomorphic coordinates centered at $p$ defined on
an open neighborhood $U$ of $p$ in $M^{\prime}$ such that (3.6) holds. We will
use the same notations as in the discussion before (3.7). On $U$, we write
$u=\sideset{}{{}^{\prime}}{\sum}_{|J|=q}u_{J}\overline{\omega}^{J}$, where
$\sideset{}{{}^{\prime}}{\sum}$ means that the summation is performed only
over strictly increasing multiindices and for $J=(j_{1},\ldots,j_{q})$,
$\overline{\omega}^{J}=\overline{\omega}^{j_{1}}\wedge\cdots\wedge\overline{\omega}^{j_{q}}$.
Let $\chi\in C^{\infty}_{c}(U)$. For every strictly increasing multiindex $J$,
$\lvert J\rvert=q$, put $v_{J}:=\chi u_{J}$,
$v=\sideset{}{{}^{\prime}}{\sum}_{|J|=q}v_{J}\overline{\omega}^{J}$. We have

$\begin{split}\left\|v\right\|^{2}_{1,\overline{M}}\leq
C_{1}\Bigr{(}\sideset{}{{}^{\prime}}{\sum}_{|J|=q,j\in\left\\{1,\ldots,n\right\\}}\|\overline{L}_{j}v_{J}\|^{2}_{M}+\sideset{}{{}^{\prime}}{\sum}_{|J|=q,j\in\left\\{1,\ldots,n\right\\}}\|L_{j}v_{J}\|^{2}_{M}+\left\|v\right\|^{2}_{M}\Bigr{)},\end{split}$
(3.10)

where $C_{1}>0$ is a constant. Moreover, it is easy to see that

$\begin{split}&\|\overline{\partial}v\|^{2}_{M}+\|\overline{\partial}^{\ast}v\|^{2}_{M}\\\
&=\sideset{}{{}^{\prime}}{\sum}_{|J|=q,j\notin
J,j\in\left\\{1,\ldots,n\right\\}}\|\overline{L}_{j}v_{J}\|^{2}_{M}+\sideset{}{{}^{\prime}}{\sum}_{|J|=q,j\in
J,j\in\left\\{1,\ldots,n\right\\}}\|L_{j}v_{J}\|^{2}_{M}+O(\|v\|_{M}\cdot\|v\|_{1,\overline{M}}).\end{split}$
(3.11)

For $j=1,\ldots,n-1$ and every strictly increasing multiindex $J$, $\lvert
J\rvert=q$, we have

$\begin{split}&\left\|L_{j}v_{J}\right\|^{2}_{M}=\left\|\overline{L}_{j}v_{J}\right\|^{2}_{M}+O(\left\|v\right\|_{M}\left\|v\right\|_{1,\overline{M}}),\\\
&\left\|\overline{L}_{j}v_{J}\right\|^{2}_{M}=\left\|L_{j}v_{J}\right\|^{2}_{M}+O(\left\|v\right\|_{M}\left\|v\right\|_{1,\overline{M}}).\end{split}$
(3.12)

From (3.11) and (3.12), we deduce that

$\begin{split}&\|\overline{\partial}v\|^{2}_{M}+\|\overline{\partial}^{\ast}v\|^{2}_{M}\\\
&=\frac{1}{2}\sideset{}{{}^{\prime}}{\sum}_{|J|=q,j\in\left\\{1,\ldots,n-1\right\\}}\Bigr{(}\|\overline{L}_{j}v_{J}\|^{2}_{M}+\|L_{j}v_{J}\|^{2}_{M}\Bigr{)}\\\
&+\sideset{}{{}^{\prime}}{\sum}_{|J|=q,n\notin
J}\|\overline{L}_{n}v_{J}\|^{2}_{M}+\sideset{}{{}^{\prime}}{\sum}_{|J|=q,n\in
J}\|L_{n}v_{J}\|^{2}_{M}+O(\|v\|_{M}\cdot\|v\|_{1,\overline{M}}).\end{split}$
(3.13)

From (3.10), (3.13), we see that if $U$ is small enough, then

$\|v\|^{2}_{1,\overline{M}}\leq
C_{2}\Bigr{(}\|Tv\|^{2}_{M}+\|v\|^{2}_{M}+\|\overline{\partial}v\|^{2}_{M}+\left\|\overline{\partial}^{\ast}v\right\|^{2}_{M}\Bigr{)},$
(3.14)

where $C_{2}>0$ is a constant and
$\|Tv\|_{M}^{2}:=\sum_{|J|=q}^{\prime}\|Tv_{J}\|^{2}$.

Since $p\notin\mu^{-1}(0)\cap X$, there exists
$\xi_{M}\in\underline{\mathfrak{g}}$ such that
$\langle\omega_{0},\xi_{M}\rangle\neq 0$ on $\overline{U}$ when $\overline{U}$
is sufficiently small. Then

$\xi_{M}|_{X}+\langle\omega_{0},\xi_{M}\rangle T|_{X}\in T^{1,0}X\bigoplus
T^{0,1}X.$

Thus by Taylor’s expansion,

$\xi_{M}+\langle\omega_{0},\xi_{M}\rangle
T=\sum_{j=1}^{n-1}a_{j}L_{j}+\sum_{j=1}^{n-1}b_{j}\bar{L}_{j}+O(|z|)D,$

where $a_{j}$, $b_{j}$ are smooth functions, $j=1,\ldots,n-1$, $D$ is a first
order differential operator. Since $u\in\Omega_{G}^{0,q}(M)$, one has
$\xi_{M}v=O(\|u\|_{M})$. Then

$\langle\omega_{0},\xi_{M}\rangle
Tv_{J}=-\xi_{M}v_{J}+\sum_{j=1}^{n-1}a_{j}L_{j}v_{J}+\sum_{j=1}^{n-1}b_{j}\bar{L}_{j}v_{J}+O(|z|)Dv_{J}.$
(3.15)

Note that $\langle\omega_{0},\xi_{M}\rangle\neq 0$ on $\overline{U}$. Then we
can assume that $|\langle\omega_{0},\xi_{M}\rangle|\geq C>0$ on
$\overline{U}$, where $C$ is a constant. Hence

$\|Tv\|^{2}_{M}\leq\hat{C}\Bigr{(}\sum_{j=1}^{n-1}\sideset{}{{}^{\prime}}{\sum}_{|J|=q}\|\bar{L}_{j}v_{J}\|^{2}_{M}+\|u\|^{2}_{M}+\varepsilon_{p}\|v\|^{2}_{1,\overline{M}}\Big{)},$
(3.16)

where $\hat{C}>0$ is a constant and $\varepsilon_{p}>0$ is sufficiently small
when $U$ is chosen to be small.

From (3.12), (3.14) and (3.16), we deduce that if $U$ is small, then

$\left\|\chi g\right\|^{2}_{1,\overline{M}}\leq
C\Bigr{(}\left\|\Box^{(q)}_{G}g\right\|^{2}_{M}+\left\|g\right\|^{2}_{M}\Bigr{)},$
(3.17)

for all $g\in\Omega^{0,q}(\overline{M})^{G}\cap{\rm Dom\,}\Box^{(q)}_{G}$,
where $C>0$ is a constant independent of $g$ ($C$ depends on $\chi$).

As before, let $u\in\Omega^{0,q}(\overline{M})^{G}\cap{\rm
Dom\,}\Box^{(q)}_{G}$ and let $\hat{\chi}\in C^{\infty}_{c}(M^{\prime})$,
$\hat{\chi}\equiv 1$ near $X$, $\hat{\chi}\equiv 0$ outside some small
neighborhood of $X$ in $M^{\prime}$. From (3.9), (3.17) and by using partition
of unity, we have

$\left\|\hat{\chi}u\right\|^{2}_{1,\overline{M}}\leq\hat{C}\Bigr{(}\left\|\Box^{(q)}_{G}u\right\|^{2}_{M}+\left\|u\right\|^{2}_{M}\Bigr{)},$
(3.18)

where $\hat{C}>0$ is a constant independent of $u$. Since $\Box^{(q)}$ is
elliptic away the boundary $X$,

$\left\|(1-\hat{\chi})u\right\|^{2}_{1,\overline{M}}\leq\tilde{C}\Bigr{(}\left\|\Box^{(q)}_{G}u\right\|^{2}_{M}+\left\|u\right\|^{2}_{M}\Bigr{)},$
(3.19)

where $\tilde{C}>0$ is a constant independent of $u$. From (3.18) and (3.19),
the lemma follows. ∎

######  Lemma 3.2.

Fix $q=1,2,\ldots,n-2$. For all $k\in\mathbb{N}$, there exists $C_{k}>0$ such
that

$\|f\|_{k,\overline{M}}^{2}\leq
C_{k}(\|\Box_{G}^{(q)}f\|^{2}_{k-1,\overline{M}}+\|f\|^{2}_{M}),\forall
f\in\Omega^{0,q}(\overline{M})^{G}\cap{\rm Dom\,}\Box_{G}^{(q)}.$ (3.20)

###### Proof.

Fix $p\in X$. If $p\in\mu^{-1}(0)\cap X$, then by Assumption 1.1, $X$ is
strongly pseudoconvex or strongly pseudoconcave near $p$. Let $U$ be a
neighborhood of $p$ in $M^{\prime}$ such that $U\cap X$ is strongly
pseudoconvex or strongly pseudoconcave and choose a cut-off function $\eta\in
C_{c}^{\infty}(U)$. Then it is well-known that (see [8, Chapter 5]) for
$k\in\mathbb{N}$,

$\|\eta f\|_{k,\overline{M}}^{2}\leq
C_{k}\Bigr{(}\|\Box_{G}^{(q)}f\|_{k-1,\overline{M}}^{2}+\|f\|^{2}_{M}\Bigr{)},\forall
f\in\Omega^{0,q}(\overline{M})^{G}\cap{\rm Dom\,}\Box^{(q)}_{G},$ (3.21)

where $C_{k}>0$ is a constant independent of $f$.

Next we assume $p\not\in\mu^{-1}(0)\cap X$. We can assume that there exists a
neighborhood $U$ of $p$ in $M^{\prime}$ such that
$\overline{U}\cap\mu^{-1}(0)=\emptyset$ and special boundary coordinates
$(t_{1},t_{2},\cdots,t_{2n-1},\rho)$ centered at $p$ such that
$t_{1},\cdots,t_{2n-1}$ restricted to $X$ are coordinates for $X$. For $u\in
C_{c}^{\infty}(U\cap\overline{M})$, the partial Fourier transform of $u$ is
defined by

$\hat{u}(\tau,\rho):=\int_{\mathbb{R}^{2n-1}}e^{-i<t,\tau>}u(t,\rho)dt,$

where $t=(t_{1},\ldots,t_{2n-1})$. For $s\in\mathbb{R}$, the tangential
Sobolev norms $|||u|||_{s}$ of $u$ is defined by

$|||u|||_{s}^{2}=\int_{\mathbb{R}^{2n-1}}\int_{-\infty}^{0}(1+|\tau|^{2})^{s}|\hat{u}(\tau,\rho)|^{2}d\rho
d\tau.$

For $\delta>0$, $M_{\delta}$ is defined by
$M_{\delta}:=\\{z\in\overline{M}:\rho(z)>-\delta\\}$. Choose a $\delta$
sufficiently small such that the tangential Sobolev norm can be defined on
$M_{\delta}$ by the partition of unity and we will use
$|||\cdot|||_{s(M_{\delta})}$ to denote the tangential Sobolev norm on
$M_{\delta}$. By a similar argument in the proof of [8, Lemma 5.2.4], one has
for every $k\in\mathbb{N}$,

$\|f\|_{k,\overline{M}}\leq\hat{C}_{k}\Bigr{(}\|\Box^{(q)}_{G}f\|_{k-1,\overline{M}}+|||f|||_{k(M_{\delta})}+\|f\|_{M}\Bigr{)},\forall
f\in\Omega^{0,q}(\overline{M})^{G}\cap{\rm Dom\,}\Box^{(q)}_{G},$ (3.22)

where $\hat{C}_{k}>0$ is a constant. Here, we do not need the condition that
$M$ is pseudoconvex as in [8, Lemma 5.2.4] since we have one more term
$|||f|||_{k(M_{\delta})}$ in the above estimate (3.22) which can be controlled
by $\|\Box^{(q)}_{G}f\|_{k-1,\overline{M}}$ when $M$ is a bounded strongly
pseudoconvex domain.

Choose $\chi\in C_{c}^{\infty}(U\cap\overline{M})$. We have for every
$k\in\mathbb{N}$,

$\frac{1}{\tilde{C}_{k}}\sum_{|\alpha|\leq k-1}|||D_{t}^{\alpha}(\chi
f)|||_{1}\leq|||\chi f|||_{k}\leq\tilde{C}_{k}\sum_{|\alpha|\leq
k-1}|||D_{t}^{\alpha}(\chi f)|||_{1},$

for every $f\in\Omega^{0,q}(\overline{M})^{G}\cap{\rm Dom\,}\Box^{(q)}_{G}$,
where $\tilde{C}_{k}>1$ is a constant independent of $f$.

We prove the following

Claim: Let $k\in\mathbb{N}$. For any $\varepsilon>0$, $\varepsilon\ll 1$, we
can take $U$ small enough so that

$|||\chi f|||_{k}\leq
C\Bigr{(}\|\Box^{(q)}_{G}f\|_{k-1,\overline{M}}+\frac{1}{\varepsilon}\|f\|_{k-1,\overline{M}}+\varepsilon\|\chi
f\|_{k,\overline{M}}\Bigr{)},\forall
f\in\Omega^{0,q}(\overline{M})^{G}\cap{\rm Dom\,}\Box^{(q)}_{G},$ (3.23)

where $C>0$ is a constant independent of $\varepsilon$. Let $\mathcal{T}^{k}$
denote a $k$-th order tangential differential operator of the form
$D_{t}^{\alpha}$ where $|\alpha|=k$. Let
$f\in\Omega^{0,q}(\overline{M})^{G}\cap{\rm Dom\,}\Box^{(q)}_{G}$. Recall that
$D_{t}\chi f\in{\rm Dom\,}\overline{\partial}^{\ast}$, for any $D_{t}$. Then
from (3.14) one has

$\begin{split}|||\mathcal{T}^{k-1}(\chi f)|||_{1}&\leq
C\Bigr{(}\|T\mathcal{T}^{k-1}(\chi f)\|_{M}+\|\mathcal{T}^{k-1}(\chi
f)\|_{M}+\|\overline{\partial}\mathcal{T}^{k-1}(\chi
f)\|_{M}+\|\overline{\partial}^{\ast}\mathcal{T}^{k-1}(\chi
f)\|_{M}\Bigr{)}\\\ &\leq C_{1}\Bigr{(}\|T\mathcal{T}^{k-1}(\chi
f)\|_{M}+|||f|||_{k-1(M_{\delta})}+\|\overline{\partial}\mathcal{T}^{k-1}(\chi
f)\|_{M}+\|\overline{\partial}^{\ast}\mathcal{T}^{k-1}(\chi
f)\|_{M}\Bigr{)},\end{split}$ (3.24)

where $C,C_{1}>0$ are constants. It follows from [8, (5.2.14)] with some minor
modification that

$\begin{split}&\|\overline{\partial}\mathcal{T}^{k-1}(\chi
f)\|^{2}_{M}+\|\overline{\partial}^{\ast}\mathcal{T}^{k-1}(\chi
f)\|^{2}_{M}\leq
C_{2}\Bigr{(}|||f|||_{k-1(M_{\delta})}\cdot|||\Box^{(q)}_{G}f|||_{k-1(M_{\delta})}+\|f\|_{k-1,\overline{M}}^{2}\\\
&\quad+\left\|f\right\|_{k-1,\overline{M}}|||\overline{\partial}(\chi
f)|||_{k-1}+\left\|f\right\|_{k-1,\overline{M}}|||\overline{\partial}^{*}(\chi
f)|||_{k-1}\Bigr{)},\end{split}$ (3.25)

where $C_{2}>0$ is a constant. Next, we estimate $\|T\mathcal{T}^{k-1}(\chi
f)\|$. From (3.15), it follows that

$\langle\omega_{0},\xi_{M}\rangle T\mathcal{T}^{k-1}(\chi
f)=-\xi_{M}\mathcal{T}^{k-1}(\chi
f)+\sum_{j=1}^{n-1}a_{j}L_{j}\mathcal{T}^{k-1}(\chi
f)+\sum_{j=1}^{n-1}b_{j}\overline{L}_{j}\mathcal{T}^{k-1}(\chi
f)+O(|z|)D\mathcal{T}^{k-1}(\chi f).$ (3.26)

Note that $|\langle\omega_{0},\xi_{M}\rangle|\geq C>0$ on $\overline{U}$ with
constant $C>0$. Notice that

$\begin{split}\xi_{M}\mathcal{T}^{k-1}(\chi
f)&=[\xi_{M},\mathcal{T}^{k-1}](\chi f)+\mathcal{T}^{k-1}\xi_{M}(\chi f)\\\
&=[\xi_{M},\mathcal{T}^{k-1}](\chi
f)+\mathcal{T}^{k-1}[\xi_{M},\chi]f+\mathcal{T}^{k-1}\chi\xi_{M}f.\end{split}$
(3.27)

Since $f\in\Omega^{0,q}(\overline{M})^{G}$, then one has $\xi_{M}f=0$. From
this observation and (3.27), we have

$\left\|-\frac{1}{\langle\omega_{0},\xi_{M}\rangle}\xi_{M}\mathcal{T}^{k-1}(\chi
f)\right\|_{M}\leq C_{4}|||f|||_{k-1(M_{\delta})},$ (3.28)

where $C_{4}>0$ is a constant.

Fix $j=1,\ldots,n-1$. It is straightforward to check that for every
$\varepsilon>0$, we have

$\begin{split}&\left\|L_{j}\mathcal{T}^{k-1}\chi
f\right\|^{2}_{M}+\left\|\overline{L}_{j}\mathcal{T}^{k-1}\chi
f\right\|^{2}_{M}\\\ &\leq
C_{5}\Bigr{(}\left\|\overline{\partial}\mathcal{T}^{k-1}\chi
f\right\|^{2}_{M}+\left\|\overline{\partial}^{*}\mathcal{T}^{k-1}\chi
f\right\|^{2}_{M}+\frac{1}{\varepsilon}\left\|\mathcal{T}^{k-1}(\chi
f)\right\|^{2}_{M}+\varepsilon\left\|f\right\|^{2}_{k,\overline{M}}\Bigr{)},\end{split}$
(3.29)

where $C_{5}>0$ is a constant independent of $\varepsilon$. From (3.25) and
(3.29), we deduce that

$\begin{split}&\left\|L_{j}\mathcal{T}^{k-1}\chi
f\right\|^{2}_{M}+\left\|\overline{L}_{j}\mathcal{T}^{k-1}\chi
f\right\|^{2}_{M}\\\ &\leq
C_{6}\Bigr{(}|||f|||_{k-1(M_{\delta})}\cdot\||\Box^{(q)}_{G}f\||_{k-1(M_{\delta})}+\|f\|_{k-1,\overline{M}}^{2}\\\
&\quad+\left\|f\right\|_{k-1,\overline{M}}|||\overline{\partial}(\chi
f)|||_{k-1}+\left\|f\right\|_{k-1,\overline{M}}|||\overline{\partial}^{*}(\chi
f)|||_{k-1}\\\ &+\frac{1}{\varepsilon}\left\|\mathcal{T}^{k-1}(\chi
f)\right\|^{2}+\varepsilon\left\|f\right\|^{2}_{k,\overline{M}}\Bigr{)},\end{split}$
(3.30)

where $C_{6}>0$ is a constant independent of $\varepsilon$. From (3.26),
(3.28) and (3.30), we deduce that for every $\varepsilon>0$, $\varepsilon\ll
1$, we can take $U$ small enough so that

$\left\|T\mathcal{T}^{k-1}(\chi f)\right\|^{2}_{M}\leq
C_{7}\Bigr{(}|||\Box^{(q)}_{G}f|||^{2}_{k-1(M_{\delta})}+\frac{1}{\varepsilon}\left\|f\right\|^{2}_{k-1,\overline{M}}+\varepsilon\left\|\chi
f\right\|^{2}_{k,\overline{M}}\Bigr{)}.$ (3.31)

From (3.24), (3.25) and (3.31), we get the claim (3.23). From (3.21), (3.22),
(3.23) and by using partition of unity, the lemma follows. ∎

From the above lemma and the techniques of elliptic regularization, one has

######  Theorem 3.3.

Fix $q=1,\ldots,n-2$. Suppose $u\in{\rm Dom\,}\Box_{G}^{(q)}$ and
$\Box_{G}^{(q)}u=v$. If $v\in\Omega^{0,q}(\overline{M})^{G}$, then
$u\in\Omega^{0,q}(\overline{M})^{G}$. If $v\in
W^{s}(\overline{M},T^{*0,q}M^{\prime})\cap L^{2}_{(0,q)}(M)^{G}$, for some
$s\in\mathbb{N}_{0}$, then $u\in W^{s+1}(\overline{M},T^{*0,q}M^{\prime})\cap
L^{2}_{(0,q)}(M)^{G}$ and we have

$\|u\|_{s+1,\overline{M}}\leq C_{s}(\|v\|_{s,\overline{M}}+\|u\|_{M}),$

where $C_{s}>0$ is a constant independent of $u$.

######  Corollary 3.4.

For every $q=0,\ldots,n-2$, $\Box_{G}^{(q)}:{\rm Dom\,}\Box^{(q)}_{G}\subset
L^{2}_{(0,q)}(M)^{G}\rightarrow L^{2}_{(0,q)}(M)^{G}$ has closed range. In
particular, for $1\leq q\leq n-2$, ${\rm Ker\,}\Box_{G}^{(q)}$ is a finite
dimensional subspace of $\Omega^{0,q}(\overline{M})^{G}$.

Let $N_{G}^{(1)}:L^{2}_{(0,1)}(M)^{G}\rightarrow{\rm Dom\,}\Box_{G}^{(1)}$ be
the partial inverse of $\Box_{G}^{(1)}$. We have

$\begin{split}\Box_{G}^{(1)}N_{G}^{(1)}+B_{G}^{(1)}&=I\ {\rm on\,}\
L^{2}_{(0,1)}(M)^{G},\\\ N_{G}^{(1)}\Box_{G}^{(1)}+B_{G}^{(1)}&=I\ {\rm on\,}\
{\rm Dom\,}\Box_{G}^{(1)}.\end{split}$

By Theorem 3.3, we conclude that
$N_{G}^{(1)}:W^{s}(\overline{M},T^{*0,1}M^{\prime})\cap
L^{2}_{(0,1)}(M)^{G}\rightarrow W^{s+1}(\overline{M},T^{*0,1}M^{\prime})\cap
L^{2}_{(0,1)}(M)^{G}$ is continuous, for all $s\in\mathbb{N}_{0}$. In
particular,

$\mbox{$N^{(1)}_{G}:\Omega^{0,1}(\overline{M})^{G}\rightarrow\Omega^{0,1}(\overline{M})^{G}$
is continuous}.$ (3.32)

Let

$B_{G}:L^{2}(M)\rightarrow{\rm Ker\,}\Box^{(0)}_{G}$ (3.33)

be the orthogonal projection ($G$-invariant Bergman projection). Note that

$H^{0}(\overline{M})^{G}=({\rm Ker\,}\Box^{(q)})^{G}={\rm
Ker\,}\Box^{(0)}_{G}.$ (3.34)

Let

$Q_{G}:L^{2}(M)\rightarrow L^{2}(M)^{G}$

be the orthogonal projection. It is not difficult to see that

$\mbox{$Q_{G}:C^{\infty}(\overline{M})^{G}\rightarrow
C^{\infty}(\overline{M})^{G}$ is continuous}.$ (3.35)

We can deduce from [8, Theorem 4.4.5] that

$\mbox{$B_{G}=(I-\overline{\partial}^{*}_{G}N^{(1)}_{G}\overline{\partial}_{G})\circ
Q_{G}$ on $L^{2}(M)$}.$ (3.36)

From (3.32), (3.35) and (3.36), we get

######  Theorem 3.5.

We have $B_{G}:C^{\infty}(\overline{M})\rightarrow
C^{\infty}(\overline{M})^{G}$ is continuous.

##  4\. The Poisson operators and reduction to the boundary

Until further notice, we fix $q\in\left\\{0,1,\ldots,n-1\right\\}$. We first
introduce some notations. We remind the reader that for $s\in\mathbb{R}$, the
space $W^{s}(\overline{M},T^{*0,q}M^{\prime})$ was introduced in the
discussion after Definition 2.1. Let

$\overline{\partial}^{\ast}_{f}:\Omega^{0,q+1}(M^{\prime})\rightarrow\Omega^{0,q}(M^{\prime})$

be the formal adjoint of $\overline{\partial}$ with respect to
$(\,\cdot\,|\,\cdot\,)_{M^{\prime}}$. That is,

$(\,\overline{\partial}f\,|\,h\,)_{M^{\prime}}=(\,f\,|\,\overline{\partial}^{\ast}_{f}h\,)_{M^{\prime}},$

$f\in\Omega^{0,q}_{c}(M^{\prime})$, $h\in\Omega^{0,q+1}(M^{\prime})$. Let

$\Box^{(q)}_{f}=\overline{\partial}\,\overline{\partial}^{\ast}_{f}+\overline{\partial}^{\ast}_{f}\,\overline{\partial}:\Omega^{0,q}(M^{\prime})\rightarrow\Omega^{0,q}(M^{\prime})$

denote the complex Laplace-Beltrami operator on $(0,q)$ forms. As before, let
$\gamma$ denote the operator of restriction to the boundary $X$. Let us
consider the map

$\begin{split}F^{(q)}:W^{2}(\overline{M},T^{*0,q}M^{\prime})&\rightarrow
L^{2}_{(0,q)}(M)\oplus W^{\frac{3}{2}}(X,T^{*0,q}M^{\prime})\\\
u&\mapsto(\Box_{f}^{(q)}u,\gamma u).\end{split}$ (4.1)

It is well-known that (see [4]) $\dim{\rm Ker\,}F^{(q)}<\infty$ and ${\rm
Ker\,}F^{(q)}\subset\Omega^{0,q}(\overline{M})$. Let

$K^{(q)}:W^{2}(\overline{M},T^{*0,q}M^{\prime})\rightarrow{\rm Ker\,}F^{(q)}$
(4.2)

be the orthogonal projection with respect to $(\,\cdot\,|\,\cdot\,)_{M}$. Put
$\tilde{\Box}_{f}^{(q)}=\Box^{(q)}_{f}+K^{(q)}$ and consider the map

$\begin{split}\tilde{F}^{(q)}:W^{2}(\overline{M},T^{*0,q}M^{\prime})&\rightarrow
L^{2}_{(0,q)}(M)\oplus W^{\frac{3}{2}}(X,T^{*0,q}M^{\prime})\\\
u&\mapsto(\tilde{\Box}_{f}^{(q)}u,\gamma u).\end{split}$ (4.3)

Then $\tilde{F}^{(q)}$ is injective (see [14, Part II, Chapter 3]). Let

$\tilde{P}:C^{\infty}(X,T^{*0,q}M^{\prime})\rightarrow\Omega^{0,q}(\overline{M})$
(4.4)

be the Poisson operator for $\tilde{\Box}^{(q)}_{f}$ which is well-defined
since (4.3) is injective. The Poisson operator $\tilde{P}$ satisfies

$\begin{split}&\tilde{\Box}^{(q)}_{f}\tilde{P}u=0,\ \ \forall u\in
C^{\infty}(X,T^{*0,q}M^{\prime}),\\\ &\gamma\tilde{P}u=u,\ \ \forall u\in
C^{\infty}(X,T^{*0,q}M^{\prime}).\end{split}$ (4.5)

It is known that $\tilde{P}$ extends continuously

$\tilde{P}:W^{s}(X,T^{*0,q}M^{\prime})\rightarrow
W^{s+\frac{1}{2}}(\overline{M},T^{*0,q}M^{\prime}),\ \ \forall s\in\mathbb{R}$

(see [4, Page 29]). Let

$\tilde{P}^{\ast}:\hat{\mathscr{D}}^{\prime}(\overline{M},T^{*0,q}M^{\prime})\rightarrow\mathscr{D}^{\prime}(X,T^{*0,q}M^{\prime})$

be the operator defined by

$(\,\tilde{P}^{\ast}u\,|\,v\,)_{X}=(\,u\,|\,\tilde{P}v\,)_{M},\ \
u\in\hat{\mathscr{D}}^{\prime}(\overline{M},T^{*0,q}M^{\prime}),\ \ v\in
C^{\infty}(X,T^{*0,q}M^{\prime}),$

where $\hat{\mathscr{D}}^{\prime}(\overline{M},T^{*0,q}M^{\prime})$ denotes
the space of continuous linear map from $\Omega^{0,q}(\overline{M})$ to
$\mathbb{C}$ with respect to $(\,\cdot\,|\,\cdot\,)_{M}$. It is well-known
(see [4, page 30]) that $\tilde{P}^{\ast}$ is continuous:
$\tilde{P}^{\ast}:W^{s}(\overline{M},T^{*0,q}M^{\prime})\rightarrow
W^{s+\frac{1}{2}}(X,T^{*0,q}M^{\prime})$ and

$\tilde{P}^{\ast}:\Omega^{0,q}(\overline{M})\rightarrow
C^{\infty}(X,T^{*0,q}M^{\prime}),$

for every $s\in\mathbb{R}$.

It is well-known that the operator

$\tilde{P}^{\ast}\tilde{P}:C^{\infty}(X,T^{*0,q}M^{\prime})\rightarrow
C^{\infty}(X,T^{*0,q}M^{\prime})$

is a classical elliptic pseudodifferential operator of order $-1$ and
invertible since $\tilde{P}$ is injective (see [4]). Moreover, the operator

$(\tilde{P}^{\ast}\tilde{P})^{-1}:C^{\infty}(X,T^{*0,q}M^{\prime})\rightarrow
C^{\infty}(X,T^{*0,q}M^{\prime})$

is a classical elliptic pseudodifferential operator of order $1$.

When $q=0$, we simply write $P$, $P^{*}$ to denote $\tilde{P}$,
$\tilde{P}^{\ast}$ respectively.

We define a new inner product on $W^{-\frac{1}{2}}(X,T^{*0,q}M^{\prime})$ as
follows:

$[\,u\,|\,v\,]=(\,\tilde{P}u\,|\,\tilde{P}v)_{M},\ \ u,v\in
W^{-\frac{1}{2}}(X,T^{*0,q}M^{\prime}).$ (4.6)

Let

$Q:W^{-\frac{1}{2}}(X,T^{*0,q}M^{\prime})\rightarrow{\rm
Ker\,}\overline{\partial}\rho^{\wedge,\star}$ (4.7)

be the orthogonal projection onto ${\rm
Ker\,}\overline{\partial}\rho^{\wedge,\star}$ with respect to
$[\,\cdot\,|\,\cdot\,]$.

We consider the following operator

$\overline{\partial}_{\beta}=Q\gamma\overline{\partial}\tilde{P}:\Omega^{0,q}(X)\rightarrow\Omega^{0,q+1}(X).$
(4.8)

The operator $\overline{\partial}_{\beta}$ was introduced by the first author
in [14]. Let
$\overline{\partial}^{\dagger}_{\beta}:\Omega^{0,q+1}(X)\rightarrow\Omega^{0,q}(X)$
be the formal adjoint with respect to $[\,\cdot\,|\,\cdot\,]$. We recall the
following (see [14, Part II, Lemma 5.1, equation (5.3) and Lemma 5.2])

######  Proposition 4.1.

We have that $\overline{\partial}_{\beta}$ and
$\overline{\partial}^{\dagger}_{\beta}$ are classical pseudodifferential
operators of order $1$,

$\overline{\partial}_{\beta}\circ\overline{\partial}_{\beta}=0\ \ \mbox{on
$\Omega^{0,q}(X)$}$ (4.9)

and

$\begin{split}&\mbox{$\overline{\partial}_{\beta}=\overline{\partial}_{b}$+lower
order terms},\\\
&\mbox{$\overline{\partial}_{\beta}^{\dagger}=\gamma\overline{\partial}^{\star}_{f}\tilde{P}=\overline{\partial}^{\star}_{b}$+lower
order terms},\end{split}$ (4.10)

where $\overline{\partial}_{b}$ is the tangential Cauchy Riemann operator on
$X$ and $\overline{\partial}^{*}_{b}$ is the adjoint of
$\overline{\partial}_{b}$ with respect to $(\,\cdot\,|\,\cdot\,)_{X}$.

Let

$\Box^{(0)}_{\beta}:=\overline{\partial}^{\dagger}_{\beta}\,\overline{\partial}_{\beta}:\Omega^{0,q}(X)\rightarrow\Omega^{0,q}(X).$

Let $D\subset X$ be an open set. Assume that the Levi form is positive on $D$.
By the same arguments of [14, PartII, Chapter 7], we can show that there are
continuous operators $N,\hat{S}:C^{\infty}_{c}(D)\to C^{\infty}(D)$, which are
properly supported on $D$, such that

$\begin{split}\Box_{\beta}^{(0)}N+\hat{S}&\equiv I\ \text{on}\ D\times D,\\\
\Box_{\beta}^{(0)}\hat{S}&\equiv 0\ \text{on}\ D\times D.\end{split}$ (4.11)

Moreover, $N$ is a pseudodifferential operator of order $-1$ type
$(\frac{1}{2},\frac{1}{2})$ on $D$. $\hat{S}$ is a pseudodifferential operator
of order $0$ type $(\frac{1}{2},\frac{1}{2})$ on $D$ and for any local
coordinate patch $D_{0}\subset D$, we have

$\mbox{$\hat{S}(x,y)\equiv\int_{0}^{\infty}e^{it\varphi(x,y)}s(x,y,t)dt$ on
$D_{0}\times D_{0}$},$ (4.12)

where $\varphi\in C^{\infty}(D_{0}\times D_{0})$ is the phase function as in
[17], $s(x,y,t)\sim\sum_{j=0}^{+\infty}s_{j}(x,y)t^{n-1-j}$ in
$S^{n-1}_{1,0}(D_{0}\times D_{0}\times\mathbb{R}_{+})$, $s_{j}(x,y)\in
C^{\infty}(D_{0}\times D_{0})$, $j=0,1,\ldots$,
$s_{0}(x,x)=\frac{1}{2}\pi^{-n}|\det\mathcal{L}_{x}|$, for all $x\in D_{0}$,
where $\det\mathcal{L}_{x}:=\lambda_{1}(x)\cdots\lambda_{n}(x)$,
$\lambda_{j}(x)$, $j=1,\ldots,n$, are eigenvalues of $\mathcal{L}_{x}$ with
respect to $\langle\,\cdot\,|\,\cdot\,\rangle$.

Let $U$ be an open neighborhood of $\mu^{-1}(0)\cap X$ in $M^{\prime}$ such
that $U=GU=\\{gx|g\in G,x\in U\\}$. Set $D=U\cap X$, then $D=GD$. Let
$\hat{S}_{G}:C^{\infty}_{c}(D)\to C^{\infty}(D)$ be the continuous operator
with distribution kernel

$\hat{S}_{G}(x,y):=\int_{G}\hat{S}(x,gy)d\mu(g),$ (4.13)

where $d\mu$ is the Haar measure on $G$ with $\int_{G}d\mu=1$. Then by using
the method of [16], we conclude that for any local coordinate patch
$D_{0}\subset D$, if the Levi form is negative on $D_{0}$, then
$\hat{S}_{G}\equiv 0$ on $D_{0}\times D_{0}$. If the Levi form is positive on
$D_{0}$, then

$\mbox{$\hat{S}_{G}(x,y)\equiv\int_{0}^{\infty}e^{it\Phi(x,y)}a(x,y,t)dt$ on
$D_{0}\times D_{0}$},$ (4.14)

where $\Phi\in C^{\infty}(D_{0}\times D_{0})$ is the phase as in [16, Theorem
1.5] and

$\begin{split}&\mbox{$a(x,y,t)\sim\sum^{+\infty}_{j=0}a_{j}(x,y)t^{n-1-\frac{d}{2}-j}$
in $S^{n-1-\frac{d}{2}}_{1,0}(D_{0}\times D_{0}\times\mathbb{R}_{+})$},\\\
&\mbox{$a_{j}(x,y)\in C^{\infty}(D_{0}\times D_{0})$, $j=0,1,\ldots$},\\\
&\mbox{$a_{0}(x,x)$ is given by~{}\cite[cite]{[\@@bibref{}{HH}{}{}, Theorem
1.6]}}.\end{split}$ (4.15)

##  5\. $G$-invariant Bergman kernel asymptotics

We first introduce $G$-invariant Szegő projection. Let
$\overline{\partial}_{b,G}:\Omega^{0,q}(X)^{G}\rightarrow\Omega^{0,q+1}(X)^{G}$
be the tangential Cauchy-Riemann operator on $X$ acting on
$\Omega^{0,q}(X)^{G}$. We extend $\overline{\partial}_{b,G}$ to
$L^{2}_{(0,q)}(X)^{G}$:

$\begin{split}&{\rm Dom\,}\overline{\partial}_{b,G}=\left\\{u\in
L^{2}_{(0,q)}(X)^{G};\,\overline{\partial}_{b,G}u\in
L^{2}_{(0,q+1)}(X)^{G}\right\\}\,,\\\ &\qquad\overline{\partial}_{b,G}:{\rm
Dom\,}\overline{\partial}_{b,G}\ni u\longmapsto\overline{\partial}_{b,G}u\in
L^{2}_{(0,q+1)}(X)^{G},\end{split}$ (5.1)

where $\overline{\partial}_{b,G}u$ is defined in the sense of distributions.
Let

$\overline{\partial}^{*}_{b,G}:{\rm Dom\,}\overline{\partial}^{*}_{b,G}\subset
L^{2}_{(0,q+1)}(X)^{G}\rightarrow L^{2}_{(0,q)}(X)^{G}$

be the $L^{2}$ adjoint of $\overline{\partial}_{b,G}$. Let

$\Box^{(0)}_{b,G}=\overline{\partial}^{*}_{b,G}\,\overline{\partial}_{b,G}:{\rm
Dom\,}\Box^{(0)}_{b,G}\subset L^{2}(X)^{G}\rightarrow L^{2}(X)^{G},$ (5.2)

where ${\rm Dom\,}\Box^{(0)}_{b,G}=\left\\{u\in L^{2}(X)^{G};\,u\in{\rm
Dom\,}\overline{\partial}_{b,G},\overline{\partial}_{b,G}u\in{\rm
Dom\,}\overline{\partial}^{*}_{b,G}\right\\}$. Let

$S_{G}:L^{2}(X)\rightarrow{\rm Ker\,}\Box^{(0)}_{b,G}\subset L^{2}(X)^{G}$
(5.3)

be the orthogonal projection ($G$-invariant Szegő projection). It was proved
in [19, Theorem 3.17] that $\Box^{(0)}_{b,G}$ has closed range. Let

$N_{b}:L^{2}(X)^{G}\rightarrow{\rm Dom\,}\Box^{(0)}_{b,G}$

be the partial inverse of $\Box^{(0)}_{b,G}$. We have

$\begin{split}\mbox{$N_{b}\Box^{(0)}_{b,G}+S_{G}=I$ on ${\rm
Dom\,}\Box^{(0)}_{b,G}$},\\\ \mbox{$\Box^{(0)}_{b,G}N_{b}+S_{G}=I$ on
$L^{2}(X)^{G}$}.\end{split}$ (5.4)

Recall that $B_{G}$ is the $G$-invariant Bergman projection (see (3.33)). We
can now prove

######  Theorem 5.1.

Let $\tau\in C^{\infty}(\overline{M})$ with ${\rm
supp\,}\tau\cap\mu^{-1}(0)\cap X=\emptyset$. Then, $\tau B_{G}\equiv 0\mod
C^{\infty}(\overline{M}\times\overline{M})$, $B_{G}\tau\equiv 0\mod
C^{\infty}(\overline{M}\times\overline{M})$.

###### Proof.

Denote by $\gamma$ the restriction from $M^{\prime}$ to $X$. It follows from
the definition of $\overline{\partial}_{b}$ that
$\overline{\partial}_{b,G}\gamma B_{G}=0$ and $\Box^{(0)}_{b,G}\gamma
B_{G}=0$. From this observation and (5.4), we have

$\gamma B_{G}=(N_{b}\Box^{(0)}_{b,G}+S_{G})\gamma B_{G}=S_{G}\gamma B_{G},$
(5.5)

hence

$PS_{G}\gamma B_{G}=P\gamma B_{G}=B_{G}$ (5.6)

and

$P^{\ast}B_{G}=P^{\ast}P\gamma B_{G}.$

We dedeuce that

$\gamma B_{G}=(P^{\ast}P)^{-1}P^{\ast}B_{G}.$ (5.7)

By (5.6) and (5.7), we deduce that

$B_{G}=PS_{G}(P^{\ast}P)^{-1}P^{\ast}B_{G}.$ (5.8)

Assume that $\tau\in C^{\infty}(\overline{M}),{\rm
supp\,}\tau\cap\mu^{-1}(0)\cap X=\emptyset$. We have

$\tau B_{G}=\tau PS_{G}(P^{\ast}P)^{-1}P^{\ast}B_{G}.$ (5.9)

Let $\tilde{\tau}\in C^{\infty}(X)$, $\tilde{\tau}=1$ near ${\rm
supp\,}\tau\cap X$ and $\tilde{\tau}=0$ on $\mu^{-1}(0)\cap X$. Taking adjoint
in (5.9), we have

$\begin{split}B_{G}\tau&=B_{G}P(P^{\ast}P)^{-1}S_{G}P^{\ast}\tau\\\
&=B_{G}P(P^{\ast}P)^{-1}S_{G}\tilde{\tau}P^{\ast}\tau+B_{G}P(P^{\ast}P)^{-1}S_{G}(1-\tilde{\tau})P^{\ast}\tau.\end{split}$
(5.10)

By [19, Theorem 3.18], we know that

$S_{G}\tilde{\tau}\equiv 0.$ (5.11)

Moreover, from [18, Lemma 4.1], we have

$(1-\tilde{\tau})P^{\ast}\tau\equiv 0\mod C^{\infty}(X\times\overline{M}).$
(5.12)

From (5.10), (5.11), (5.12) and notice that
$B_{G}:C^{\infty}(\overline{M})\rightarrow C^{\infty}(\overline{M})^{G}$ is
continuous (see Theorem 3.5), we deduce that

$B_{G}\tau:W^{s}(\overline{M})\rightarrow C^{\infty}(\overline{M}),$

for all $s\in\mathbb{R}$. Hence $B_{G}\tau\equiv 0\mod
C^{\infty}(\overline{M}\times\overline{M})$. By taking adjoint, we deduce that
$\tau B_{G}\equiv 0\mod C^{\infty}(\overline{M}\times\overline{M})$. The
theorem follows. ∎

We now study the distribution kernel of $B_{G}$ near $\mu^{-1}(0)\cap X$. For
a Borel set $\Sigma\subset\mathbb{R}$, denote by $E(\Sigma)$ the spectral
projection of $\Box^{(0)}$ associated to $\Sigma$, where $E$ is the spectral
measure of $\Box^{(0)}$. Set
$H^{0}_{\leq\lambda}(\overline{M}):=RanE\big{(}(-\infty,\lambda]\big{)}$. Let
$B^{(0)}_{\leq\lambda}$ be the orthogonal projection onto
$H^{0}_{\leq\lambda}(\overline{M})$ and let
$B^{(0)}(x,y)\in\mathscr{D}^{\prime}(M\times M)$ be the distribution kernel of
$B^{(0)}_{\leq\lambda}$. First we recall the following theorem [18, Theorem
1.1].

######  Theorem 5.2.

Let $U$ be an open set of $M^{\prime}$ with $U\cap X\neq\emptyset$. Assume
that the Levi form is negative on $U\cap X$, then $B^{(0)}_{\leq\lambda}\equiv
0\mod C^{\infty}((U\times U)\cap(\overline{M}\times\overline{M}))$. Assume
that the Levi form is positive on $U\cap X$. We have

$B^{(0)}_{\leq\lambda}\equiv\int_{0}^{\infty}e^{it\phi(x,y)t}b(x,y,t)dt\
\text{mod}\ C^{\infty}\big{(}(U\times
U)\cap(\overline{M}\times\overline{M})\big{)},$ (5.13)

where

$\begin{split}&b(x,y,t)\sim\sum_{j=0}^{\infty}b_{j}(x,y)t^{n-j}\in
S^{n}_{1,0}\big{(}(U\times
U)\cap(\overline{M}\times\overline{M})\times]0,\infty[\big{)},\\\
&b_{j}(x,y)\in C^{\infty}((U\times U)\cap(\overline{M}\times\overline{M})),\ \
j=0,1,\ldots,\\\ &b_{0}(x,y)\neq 0,\\\ &\phi\in C^{\infty}((U\times
U)\cap(\overline{M}\times\overline{M})),\ \ {\rm Im\,}\phi\geq 0,\\\
&\mbox{$\phi(x,x)=0$, $x\in U\cap X$, $\phi(x,y)\neq 0$ if $(x,y)\notin{\rm
diag\,}((U\times U)\cap(X\times X))$},\\\ &\mbox{${\rm Im\,}\phi(x,y)>0$ if
$(x,y)\notin(U\times U)\cap(X\times X)$},\\\
&d_{x}\phi(x,x)=-\omega_{0}(x)-id\rho(x),\ \
d_{y}\phi(x,x)=\omega_{0}(x)-id\rho(x),\ \ \mbox{for every $x\in U\cap
X$}.\end{split}$ (5.14)

We refer the reader to [18, Theorem 5.26] for more properties of the phase
$\phi$ in (5.13). We also refer the reader to [18, (5.121)] for the explicit
formula for $b_{0}(x,x)$ in (5.14).

From Corollary 3.4, we know that $\Box^{(0)}_{G}$ has closed range. From this
observation, we can repeat the proof in [19, Theorem 3.17] and deduce that
there exists a constant $\lambda_{0}>0$ such that

$\begin{split}&B_{G}=B^{(0)}_{\leq\lambda_{0}}\circ Q_{G}\ \ \mbox{on
$L^{2}(M)$},\\\
&B_{G}(x,y)=\int_{G}B_{\leq\lambda_{0}}(x,gy)d\mu(g).\end{split}$ (5.15)

We recall some results in [18]. Fix $p\in\mu^{-1}(0)$ and let $U$ be an open
set of $p$ in $M^{\prime}$ with $U=GU$. Let $D:=U\cap X$. Assume the Levi form
is positive on $D$. Let

$L:C^{\infty}_{c}(U\cap\overline{M})\rightarrow C^{\infty}(D)$

be a continuous operator such that

$L-(P^{*}P)^{-1}P^{*}\equiv 0\mod C^{\infty}((U\times
U)\cap(X\times\overline{M})),$ (5.16)

$L$ is properly supported on $U\cap\overline{M}$, that is, for every $\chi\in
C^{\infty}_{c}(U\cap\overline{M})$, there is a $\tau\in C^{\infty}_{c}(D)$
such that $L\chi=\tau L$ on $C^{\infty}_{c}(U\cap\overline{M})$ and for every
$\tau_{1}\in C^{\infty}_{c}(D)$, there is a $\chi_{1}\in
C^{\infty}_{c}(U\cap\overline{M})$ such that $\tau_{1}L=L\chi_{1}$ on
$C^{\infty}_{c}(U\cap\overline{M})$ (see the discussion after [18, Theorem
5.18]). Since $U$ and $D$ are $G$-invariant, we can take $L$ so that

$\mbox{$LQ_{G}=Q_{G,X}L$ on $C^{\infty}_{c}(U\cap\overline{M})$},$ (5.17)

where $Q_{G,X}$ is the orthogonal projection from $L^{2}(X)$ onto
$L^{2}(X)^{G}$. It was shown in [18, (5.111), Theorem 6.11] that

$B_{\leq\lambda_{0}}-P\hat{S}L\equiv 0\mod C^{\infty}((U\times
U)\cap(\overline{M}\times\overline{M})),$ (5.18)

where $\hat{S}$ is as in (4.11). From (5.15) and (5.18), we deduce

$B_{G}-P\hat{S}LQ_{G}\equiv 0\mod C^{\infty}((U\times
U)\cap(\overline{M}\times\overline{M})).$ (5.19)

From (5.17) and (5.19), we get

######  Theorem 5.3.

With the notations and assumptions used above, we have

$B_{G}-P\hat{S}_{G}L\equiv 0\mod C^{\infty}((U\times
U)\cap(\overline{M}\times\overline{M})),$ (5.20)

where $\hat{S}_{G}$ is as in (4.13).

By (4.14) and applying the procedure in [14, Part II, Proposition 7.8, Theorem
7.9] we get

######  Theorem 5.4.

Let $p\in\mu^{-1}(0)\cap X$. Let $U$ be an open local coordinate patch of $p$
in $M^{\prime}$, $D:=U\cap X$. If the Levi form is negative on $D$, then
$B_{G}\equiv 0\mod C^{\infty}((U\times
U)\cap(\overline{M}\times\overline{M}))$. Assume that the Levi form is
positive on $D$. We have

$B_{G}(z,w)\equiv\int^{+\infty}_{0}e^{it\Psi(z,w)}b(z,w,t)dt\mod
C^{\infty}((U\times U)\cap(\overline{M}\times\overline{M})),$ (5.21)

where

$\begin{split}&b(z,w,t)\in S^{n-\frac{d}{2}}_{1,0}(((U\times
U)\cap(\overline{M}\times\overline{M}))\times\mathbb{R}_{+}),\\\
&b(z,w,t)\sim\sum^{+\infty}_{j=0}t^{n-\frac{d}{2}-j}b_{j}(z,w)\ \ \mbox{in
$S^{n-\frac{d}{2}}_{1,0}(((U\times
U)\cap(\overline{M}\times\overline{M}))\times\mathbb{R}_{+})$},\\\
&b_{j}(z,w)\in C^{\infty}((U\times U)\cap(\overline{M}\times\overline{M})),\ \
j=0,1,2,\ldots,\\\ &b_{0}(z,w)\neq 0,\end{split}$ (5.22)

and

$\begin{split}&\Psi(z,w)\in C^{\infty}(((U\times
U)\cap(\overline{M}\times\overline{M}))),\ \ {\rm Im\,}\Psi\geq 0,\\\
&\Psi(z,z)=0,\ z\in\mu^{-1}(0)\cap X,\\\ &\mbox{${\rm Im\,}\Psi(z,w)>0$ if
$(z,w)\notin{\rm diag\,}((\mu^{-1}(0)\cap D)\times(\mu^{-1}(0)\cap D))$},\\\
&d_{x}\Psi(x,x)=-\omega_{0}(x)-id\rho(x),\ \
d_{y}\Psi(x,x)=\omega_{0}(x)-id\rho(x),\ \ x\in\mu^{-1}(0)\cap D,\\\
&\mbox{$\Psi|_{D\times D}=\Phi$, $\Phi$ is as in
\eqref{e-gue230319ycdp}}.\end{split}$ (5.23)

Moreover, let $z=(x_{1},\ldots,x_{2n-1},\rho)$ be local coordinates of
$M^{\prime}$ defined near $p$ in $M^{\prime}$ with $x(p)=0$ and
$x=(x_{1},\ldots,x_{2n-1})$ are local coordinates of $X$ defined near $p$ in
$X$. Then,

$\mbox{$\Psi(z,w)=\Phi(x,y)-i\rho(z)(1+f(z))-i\rho(w)(1+\overline{f}(w))+O(\lvert(z,w)\rvert^{3})$
near $(p,p)$},$ (5.24)

where $f\in C^{\infty}$, $f=O(\lvert z\rvert)$.

From [14, Part II, Proposition 7.10] we have the following

######  Theorem 5.5.

With the notations and assumptions used above, for $b_{0}(z,w)$ in (5.22), we
have

$b_{0}(x,x)=2a_{0}(x,x),\ \ \mbox{for every $x\in\mu^{-1}(0)\cap D$},$

where $a_{0}$ is as in (4.15).

###### End of the proof of Theorem 1.2.

From Theorem 5.1, Theorem 5.4 and Theorem 5.5, we get Theorem 1.2. ∎

Let $D$ be an open set of ${\color[rgb]{1,0,0}\widehat{X}}$ in $X$ with
$D=GD$. Let $\chi\in C^{\infty}_{c}(D)^{G}$, $\chi\equiv 1$ near
$\mu^{-1}(0)\cap X$. Let $\chi_{1}\in C^{\infty}_{c}(D)^{G}$ with
$\chi_{1}\equiv 1$ on ${\rm supp\,}\chi$. Let
$\tilde{S}_{G}:=\chi_{1}\hat{S}_{G}\chi:C^{\infty}(X)\rightarrow
C^{\infty}(X)^{G}$, where $\hat{S}_{G}$ is as in (4.13). From Theorem 5.1 and
(5.19), we get

######  Theorem 5.6.

With the notations and assumptions above, we have

$B_{G}\equiv P\tilde{S}_{G}(P^{*}P)^{-1}P^{*}\mod
C^{\infty}(\overline{M}\times\overline{M}).$

##  6\. The proofs of Theorem 1.3, Theorem 1.4 and Theorem 1.6

For simplicity, until further notice, we assume that $\mu^{-1}(0)\cap X$ is
strongly pseudoconvex.

For every $s\in\mathbb{R}$, consider the map

$\begin{split}\hat{\sigma}_{s}=\hat{\sigma}:H^{0}(\overline{M})^{G}_{s}&\to
H^{0}_{b}(X)^{G}_{s-\frac{1}{2}}\\\ u&\mapsto(P^{\ast}P)^{-1}P^{\ast}u=\gamma
u.\end{split}$ (6.1)

Since $(P^{\ast}P)^{-1}P^{\ast}:H^{s}(\overline{M})^{G}_{s}\to
W^{s-\frac{1}{2}}(X)$ is continuous, for every $s\in\mathbb{R}$,
$\hat{\sigma}$ is well-defined. We define ${\rm Coker\,}\hat{\sigma}_{s}$ in
the following way:

${\rm Coker\,}\hat{\sigma}_{s}={\rm Coker\,}\hat{\sigma}:=\\{u\in
H^{0}_{b}(X)^{G}_{s-\frac{1}{2}};\,(\,u\,|\,\hat{\sigma}v)_{X}=0,\forall v\in
H^{0}(\overline{M})^{G}_{s}\cap C^{\infty}(\overline{M})\\}.$ (6.2)

######  Theorem 6.1.

We have that ${\rm Ker\,}\hat{\sigma}_{s}=\left\\{0\right\\}$ and ${\rm
Coker\,}\hat{\sigma}_{s}$ is a finite dimensional subspace of
$C^{\infty}(X)^{G}\cap H^{0}_{b}(X)^{G}$. Moreover, ${\rm
Coker\,}\hat{\sigma}_{s}$ is independent of $s$.

###### Proof.

It is obvious that ${\rm Ker\,}\hat{\sigma}=\\{0\\}$. We extend $\hat{\sigma}$
to
$\hat{\sigma}:\hat{\mathscr{D}}^{\prime}(\overline{M})\to\mathscr{D}^{\prime}(X)$
by putting

$\hat{\sigma}u=(P^{\ast}P)^{-1}P^{\ast}B_{G}u=S_{G}(P^{\ast}P)^{-1}P^{\ast}B_{G}u,\
u\in\hat{\mathscr{D}}^{\prime}(\overline{M}).$ (6.3)

Recall that $\hat{\mathscr{D}}^{\prime}(\overline{M})$ is the space of
continuous linear maps from $C^{\infty}(\overline{M})$ to $\mathbb{C}$. Since

$B_{G}P(P^{*}P)^{-1}S_{G}=(S_{G}(P^{*}P)^{-1}P^{*}B_{G})^{*}$

maps $C^{\infty}(X)$ to $C^{\infty}(\overline{M})$, the definition (6.3) is
well-defined, where

$(S_{G}(P^{*}P)^{-1}P^{*}B_{G})^{*}$

is the formal adjoint of $S_{G}(P^{*}P)^{-1}P^{*}B_{G}$ with respect to
$(\,\cdot\,|\,\cdot\,)_{X}$ and $(\,\cdot\,|\,\cdot\,)_{M}$.

Let
$\hat{\sigma}^{*}:\mathscr{D}^{\prime}(X)\to\hat{\mathscr{D}}^{\prime}(\overline{M})$
be the formal adjoint of $\hat{\sigma}$ with respect to the given $L^{2}$
inner products $(\,\cdot\,|\,\cdot\,)_{M}$ and $(\,\cdot\,|\,\cdot\,)_{X}$. We
have

$\hat{\sigma}\hat{\sigma}^{\ast}=S_{G}(P^{\ast}P)^{-1}P^{\ast}B_{G}P(P^{\ast}P)^{-1}S_{G}.$

From Theorem 5.6, we have

$\begin{split}\hat{\sigma}\hat{\sigma}^{\ast}&=S_{G}(P^{\ast}P)^{-1}P^{\ast}(P\tilde{S}_{G}(P^{\ast}P)^{-1}P^{\ast}+F)P(P^{\ast}P)^{-1}S_{G}\\\
&=S_{G}\tilde{S}_{G}(P^{\ast}P)^{-1}S_{G}+S_{G}\hat{F}S_{G},\end{split}$

where $F\equiv 0\mod C^{\infty}(\overline{M}\times\overline{M})$ and
$\hat{F}\equiv 0$. Thus

$S_{G}(P^{\ast}P)\hat{\sigma}\hat{\sigma}^{\ast}=S_{G}(P^{\ast}P)S_{G}\tilde{S}_{G}(P^{\ast}P)^{-1}S_{G}+S_{G}(P^{\ast}P)S_{G}\hat{F}S_{G}.$

From (4.14) and the complex stationary phase formula of Melin-Sjöstrand [26],
we deduce that

$S_{G}(P^{\ast}P)\hat{\sigma}\hat{\sigma}^{\ast}=S_{G}(I+R)S_{G},$ (6.4)

where $R$ is smoothing away $\mu^{-1}(0)\cap X$ and for any
$p\in\mu^{-1}(0)\cap X$, let $D$ be any small local coordinate patch of $X$,
$p\in D$, if the Levi form is negative on $D$, then $R\equiv 0$. If the Levi
form is positive on $D$, we have

$R\equiv\int^{+\infty}_{0}e^{it\Psi(x,y)}r(x,y,t)dt,$

$r(x,y,t)\in S^{n-1-\frac{d}{2}}_{1,0}(D\times D\times\mathbb{R}_{+})$,
$r(x,y,t)\sim\sum^{+\infty}_{j=0}t^{n-1-\frac{d}{2}-j}r_{j}(x,y)$ in
$S^{n-1-\frac{d}{2}}_{1,0}(D\times D\times\mathbb{R}_{+})$, $r_{j}(x,y)\in
C^{\infty}(D\times D)$, $j=0,1,\ldots$, $r_{0}(x,x)=0$, for every
$x\in\mu^{-1}(0)\cap D$. From [19, Theorem 4.9], we know that ${\rm
Ker\,}(I+R)$ is a finite dimensional subspace of $C^{\infty}(X)$. It follows
from

${\rm Coker\,}\hat{\sigma}_{s}\subset{\rm Ker\,}(I+R)\cap
H^{0}_{b}(X)^{G}_{s-\frac{1}{2}}$

that ${\rm Coker\,}\hat{\sigma}_{s}$ is a finite dimensional subspace of
$C^{\infty}(X)^{G}$ and ${\rm Coker\,}\hat{\sigma}_{s}$ is independent of $s$.
∎

As before, let $X_{G}:=(\mu^{-1}(0)\cap X)/G$. Let $\iota_{X}:\mu^{-1}(0)\cap
X\rightarrow X$ be the natural inclusion and let
$\iota^{*}_{X}:C^{\infty}(X)\rightarrow C^{\infty}(\mu^{-1}(0)\cap X)$ be the
pull-back by $\iota_{X}$. Let $\iota_{X,G}:C^{\infty}(\mu^{-1}(0)\cap
X)^{G}\rightarrow C^{\infty}(X_{G})$ be the natural identification. Put

$\begin{split}&H^{0}_{b}(X)^{G}:=\left\\{u\in
L^{2}(X)^{G};\,\overline{\partial}_{b}u=0\right\\},\\\
&H^{0}_{b}(X_{G}):=\left\\{u\in
L^{2}(X_{G});\,\overline{\partial}_{b,X_{G}}u=0\right\\},\end{split}$

where $\overline{\partial}_{b,X_{G}}$ denotes the tangential Cauchy-Riemann
operator on $X_{G}$. For every $s\in\mathbb{R}$, put

$\begin{split}&H^{0}_{b}(X)^{G}_{s}:=\left\\{u\in
W^{s}(X);\,\overline{\partial}_{b}u=0,\ \ g^{*}u=0,\ \ \forall g\in
G\right\\},\\\ &H^{0}_{b}(X_{G})_{s}:=\left\\{u\in
W^{s}(X_{G});\,\overline{\partial}_{b,X_{G}}u=0\right\\}.\end{split}$

Let

$\begin{split}\sigma_{1}:H^{0}_{b}(X)^{G}&\rightarrow H^{0}_{b}(X_{G}),\\\
u&\rightarrow\iota_{X,G}\circ\iota^{*}_{X}u.\end{split}$

From [19, Theorem 5.3], $\sigma_{1}$ extends by density to a bounded operator

$\sigma_{1}=\sigma_{1,s}:H^{0}_{b}(X)^{G}_{s}\rightarrow
H^{0}_{b}(X_{G})_{s-\frac{d}{4}},$

for every $s\in\mathbb{R}$. Let $\triangle^{X}$ and $\triangle^{X_{G}}$ be the
(positive) Laplacians on $X$ and $X_{G}$ respectively. For $s\in\mathbb{R}$,
let $\Lambda_{s}:=(I+\triangle^{X})^{\frac{s}{2}}$,
$\hat{\Lambda}_{s}:=(I+\triangle^{X_{G}})^{\frac{s}{2}}$. Fix
$s\in\mathbb{R}$. For $u,v\in W^{s}(X)$, $u^{\prime},v^{\prime}\in
W^{s}(X_{G})$, we define the inner products

$\begin{split}&(\,u\,|\,v\,)_{X,s}:=(\Lambda_{s}u\,|\,\Lambda_{s}v\,)_{X},\\\
&(\,u^{\prime}\,|\,v^{\prime}\,)_{X_{G},s}:=(\hat{\Lambda}_{s}u^{\prime}\,|\,\hat{\Lambda}_{s}v^{\prime}\,)_{X_{G}}\end{split}$

and let $\left\|\cdot\right\|_{X,s}$ and $\left\|\cdot\right\|_{X_{G},s}$ be
the corresponding norms. For every $s\in\mathbb{R}$, define

$({\rm Im\,}\sigma_{1,s})^{\perp}:=\left\\{u\in
H^{0}_{b}(X_{G})_{s-\frac{d}{4}};\,(\,\sigma_{1,s}v\,|\,u\,)_{X_{G},s-\frac{d}{4}}=0,\
\ \forall v\in H^{0}_{b}(X)^{G}_{s}\right\\}.$

The following theorem is the main result in [19] (see [19, Theorem 1.2])

######  Theorem 6.2.

With the notations and assumptions used above, assume that
$\overline{\partial}_{b,X_{G}}$ has closed range. Then, for every
$s\in\mathbb{R}$, ${\rm Ker\,}\sigma_{1,s}$ and $({\rm
Im\,}\sigma_{1,s})^{\perp}$ are finite dimensional subspaces of
$H^{0}_{b}(X)^{G}\cap C^{\infty}(X)^{G}$ and $H^{0}_{b}(X_{G})\cap
C^{\infty}(X_{G})$ respectively, ${\rm Ker\,}\sigma_{1,s}$ and the index ${\rm
dim\,}{\rm Ker\,}\sigma_{1,s}-{\rm dim\,}({\rm Im\,}\sigma_{1,s})^{\perp}$ are
independent of $s$.

For $s\in\mathbb{R}$, define

${\rm Coker\,}\sigma_{1}={\rm Coker\,}\sigma_{1,s}:=\left\\{u\in
H^{0}_{b}(X_{G})_{s-\frac{d}{4}};\,(\,u\,|\,\sigma_{1,s}v\,)_{X_{G}}=0,\ \
\forall v\in H^{0}_{b}(X)^{G}\cap C^{\infty}(X)^{G}\right\\},$ (6.5)

where $(\,\cdot\,|\,\cdot\,)_{X_{G}}$ is the $L^{2}$ inner product on $X_{G}$
induced by $\langle\,\cdot\,|\,\cdot\,\rangle$.

######  Theorem 6.3.

With the notations and assumptions used above, assume that
$\overline{\partial}_{b,X_{G}}$ has closed range. Then, ${\rm
Coker\,}\sigma_{1,s}=({\rm Im\,}\sigma_{1,\frac{d}{4}})^{\perp}$, for every
$s\in\mathbb{R}$. In particular, ${\rm Coker\,}\sigma_{1,s}$ is a finite
dimensional subspace of $H^{0}_{b}(X_{G})\cap C^{\infty}(X_{G})$ and ${\rm
Coker\,}\sigma_{1,s}$ is independent of $s$.

###### Proof.

Let $u\in{\rm Coker\,}\sigma_{1,s}$. By definition, we have

$(\,\sigma_{1,\frac{d}{4}}v\,|\,u\,)_{X_{G}}=0,\ \ \mbox{for every $v\in
H^{0}_{b}(X)^{G}\cap C^{\infty}(X)^{G}$}.$ (6.6)

From (6.6) and the proof of [19, Theorem 1.2], we can check that $u\in
C^{\infty}(X_{G})$. Thus, (6.6) holds for all $v\in
H^{0}_{b}(X)^{G}_{\frac{d}{4}}$. Hence, $u\in({\rm
Im\,}\sigma_{1,\frac{d}{4}})^{\perp}$. We have proved that ${\rm
Coker\,}\sigma_{1,s}\subset({\rm Im\,}\sigma_{1,\frac{d}{4}})^{\perp}$. It is
clear that $({\rm Im\,}\sigma_{1,\frac{d}{4}})^{\perp}\subset{\rm
Coker\,}\sigma_{1,s}$. The theorem follows. ∎

###### End of the proof of Theorem 1.3.

For every $s\in\mathbb{R}$, it is not difficult to see that
$\tilde{\sigma}_{G}=\tilde{\sigma}_{G,s}=\hat{\sigma}_{s}\circ\sigma_{1,s}$,
where $\tilde{\sigma}_{G,s}$ is given by (1.16), (1.17). From this
observation, Theorem 6.1, Theorem 6.2 and Theorem 6.3, Theorem 1.3 follows. ∎

We now prove Theorem 1.4. We assume that (1.19) holds. Recall $\mu^{-1}(0)\cap
X=\widehat{X}\cup\widetilde{X}$ on which the Levi form is strongly
pseudoconvex and pseudoconcave respectively. As before, let
$M_{G}=(\mu^{-1}(0)\cap M)/G$. For $u\in
H^{0}_{b}(\widehat{X}_{G})_{s},s\in\mathbb{R}$, we identify $u$ with an
element in $H^{0}_{b}(X_{G})_{s}$ by putting $u=0$ on $\widetilde{X}_{G}$. For
every $s\in\mathbb{R}$, let

$\begin{split}\sigma_{2,s}=\sigma_{2}:H^{0}_{b}(\widehat{X}_{G})_{s}&\rightarrow
H^{0}(\overline{M}_{G})_{s+\frac{1}{2}},\\\ u&\rightarrow
B_{M_{G}}P_{M_{G}}u,\end{split}$

where $B_{M_{G}}$ and $P_{M_{G}}$ are the Bergman projection on $M_{G}$ and
the Poisson operator on $M_{G}$ respectively. For every $s\in\mathbb{R}$, let

${\rm Coker\,}\sigma_{2}={\rm Coker\,}\sigma_{2,s}:=\left\\{u\in
H^{0}(\overline{M}_{G})_{s+\frac{1}{2}};\,(\,u\,|\,\sigma_{2,s}v\,)_{M_{G}}=0,\
\ \forall v\in H^{0}_{b}(\widehat{X}_{G})\cap
C^{\infty}(\widehat{X}_{G})\right\\}.$ (6.7)

Now we are in a position to prove a key result.

######  Theorem 6.4.

With the notations and assumptions used above, assume that
$\overline{\partial}_{b,X_{G}}$ has closed range. Let $s\in\mathbb{R}$. We
have that ${\rm Ker\,}\sigma_{2,s}$ and ${\rm Coker\,}\sigma_{2,s}$ are finite
dimensional subspaces of $H^{0}_{b}(\widehat{X}_{G})\cap
C^{\infty}(\widehat{X}_{G})$ and $H^{0}(\overline{M}_{G})\cap
C^{\infty}(\overline{M}_{G})$ respectively, ${\rm Ker\,}\hat{\sigma}_{2,s}$
and ${\rm Coker\,}\hat{\sigma}_{2,s}$ are independent of $s$.

###### Proof.

We extend $\sigma_{2}$ to
$\sigma_{2}:\mathscr{D}^{\prime}(\widehat{X}_{G})\rightarrow\hat{\mathscr{D}}^{\prime}(\overline{M}_{G})$
by putting

$\sigma_{2}u=B_{M_{G}}P_{M_{G}}S_{\widehat{X}_{G}}u,\ \
u\in\mathscr{D}^{\prime}(\widehat{X}_{G}),$

where $S_{\widehat{X}_{G}}$ denotes the Szegő projection on $\widehat{X}_{G}$.
Let
$\sigma^{*}_{2}:\hat{\mathscr{D}}^{\prime}(\overline{M}_{G})\rightarrow\mathscr{D}^{\prime}(\widehat{X}_{G})$
be the formal adjoint of $\sigma_{2}$ with respect to
$(\,\cdot\,|\,\cdot\,)_{\widehat{X}_{G}}$ and $(\,\cdot\,|\,\cdot\,)_{M_{G}}$.
We have

$\begin{split}&\sigma^{*}_{2}\sigma_{2}=S_{\widehat{X}_{G}}P^{*}_{M_{G}}B_{M_{G}}P_{M_{G}}S_{\widehat{X}_{G}}\\\
&=S_{\widehat{X}_{G}}P^{*}_{M_{G}}\Bigr{(}P_{M_{G}}\hat{S}_{\widehat{X}_{G}}(P^{*}_{M_{G}}P_{M_{G}})^{-1}P^{*}_{M_{G}}+F)P_{M_{G}}S_{\widehat{X}_{G}}\\\
&=S_{\widehat{X}_{G}}P^{*}_{M_{G}}P_{M_{G}}\hat{S}_{\widehat{X}_{G}}S_{\widehat{X}_{G}}+S_{\widehat{X}_{G}}\hat{F}S_{\widehat{X}_{G}},\end{split}$
(6.8)

where $\hat{S}_{X_{G}}$ is the operator as in (4.12), $F\equiv 0\mod
C^{\infty}(\overline{M}_{G}\times\overline{M}_{G})$, $\hat{F}\equiv 0$. From
(6.8), it is straightforward to check that

$S_{\widehat{X}_{G}}(P^{*}_{M_{G}}P_{M_{G}})^{-1}\sigma^{*}_{2}\sigma_{2}=S_{\widehat{X}_{G}}(I+\hat{R})S_{\widehat{X}_{G}},$

where $\hat{R}$ is a complex Fourier integral operator of the same type, the
same order, the same phase as $S_{\widehat{X}_{G}}$ but the leading term
vanishes at diagonal. From this observation, we can repeat the proof of [19,
Theorem 4.15] with minor change and deduce that ${\rm Ker\,}(I+\hat{R})$ is a
finite dimensional subspace of $C^{\infty}(\widehat{X}_{G})$. Thus, ${\rm
Ker\,}\hat{\sigma}_{2,s}$ is a finite dimensional subspace of
$H^{0}_{b}(\widehat{X}_{G})\cap C^{\infty}(\widehat{X}_{G})$.

Now

$\sigma_{2}(P^{\ast}_{M_{G}}P_{M_{G}})^{-1}\sigma_{2}^{\ast}=B_{M_{G}}P_{M_{G}}S_{\widehat{X}_{G}}(P^{\ast}_{M_{G}}P_{M_{G}})^{-1}S_{\widehat{X}_{G}}P^{\ast}_{M_{G}}B_{M_{G}}.$

From [14, Theorem 1.2], we have

$\sigma_{2}(P^{\ast}_{M_{G}}P_{M_{G}})^{-1}\sigma_{2}^{\ast}=B_{M_{G}}(I+R_{M_{G}})B_{M_{G}},$
(6.9)

where $R_{M_{G}}$ is a complex Fourier integral operator of the same type, the
same order, the same phase as $B_{M_{G}}$, but the leading term vanishes at
${\rm diag\,}(\widehat{X}_{G}\times\widehat{X}_{G})$. We can deduce from [19,
Theorem 4.15] that ${\rm Ker\,}(I+R_{M_{G}})$ is a finite dimensional subspace
of $C^{\infty}(\widehat{X}_{G})$. Hence, ${\rm Coker\,}\hat{\sigma}_{2,s}$ is
a finite dimensional subspace of $H^{0}(\overline{M}_{G})\cap
C^{\infty}(\overline{M}_{G})$. The proof follows. ∎

###### End of the proof of Theorem 1.4.

It is clear that for every $s\in\mathbb{R}$,

$\sigma_{G}=\sigma_{G,s}=\sigma_{2}\circ\sigma_{1}\circ\hat{\sigma}:H^{0}(\overline{M})_{s}^{G}\to
H^{0}(\overline{M}_{G})_{s-\frac{d}{4}},$

where $\sigma_{G}$ is given by (1.21), (1.22). From Theorem 6.1, Theorem 6.2,
Theorem 6.3 and Theorem 6.4, we deduce that $\sigma$ is Fredholm, ${\rm
Ker\,}\sigma$ and ${\rm Coker\,}\sigma$ are finite dimensional subspaces of
$H^{0}(\overline{M})^{G}\cap C^{\infty}(\overline{M})^{G}$ and
$H^{0}(\overline{M}_{G})\cap C^{\infty}(\overline{M}_{G})$ respectively.
Moreover, ${\rm Ker\,}\sigma$ and ${\rm Coker\,}\sigma$ are independent of the
choices of $s$. The proof is completed. ∎

###### Proof of Theorem 1.6.

In the end of this section, we will prove Theorem 1.6. We will not assume
(1.19). Let $M_{1}$, $M_{2}$ be bounded domains in $\mathbb{C}^{n}$ with
smooth boundary. Assume that $M_{j}$ admits a compact holomorphic Lie group
$G$ action and $M_{j}$ satisfies Assumption 1.1, for each $j=1,2$. Let

$\begin{split}&F:M_{1}\rightarrow M_{2},\\\
&z\rightarrow(F_{1}(z),\ldots,F_{n}(z)),\end{split}$

be the $G$-invariant map which satisfies the assumption of Theorem 1.6. We are
going to prove that $F$ extends smoothly to the boundary. Let
$\mu_{j}:M_{j}\rightarrow\mathfrak{g}^{*}$ be the corresponding moment map on
$M_{j}$ and let $X_{j}$ be the boundary of $M_{j}$, $j=1,2$.

######  Lemma 6.5.

Let $\tau\in C^{\infty}(\overline{M}_{1})$, $\tau\equiv 1$ near
$\mu^{-1}_{1}(0)\cap X_{1}$. Then, $(1-\tau)F$ extends smoothly to the
boundary.

###### Proof.

Fix $j\in\left\\{1,\ldots,n\right\\}$. Since $F_{j}(z)$ is bounded, $F_{j}(z)$
is a $G$-invariant $L^{2}$ holomorphic function on $M_{1}$. We have

$(\tau F_{j})(z)=\tau(z)(B_{G,M_{1}}F_{j})(z)=((\tau B_{G,M_{1}})F_{j})(z),$
(6.10)

where $B_{G,M_{1}}$ is the $G$-invariant Bergman projection on $M_{1}$. In
view of Theorem 1.2, we see that $\tau B_{G,M_{1}}\equiv 0\mod
C^{\infty}(\overline{M}_{1}\times\overline{M}_{1})$. From this observation and
(6.10), we deduce that $\tau F_{j}\in C^{\infty}(\overline{M}_{1})$. The lemma
follows. ∎

From Theorem 1.2, we see that

$B_{G,M_{1}}(\cdot,w)\in C^{\infty}(\overline{M}_{1}),\ \ \mbox{for every
$w\in M_{1}$}.$ (6.11)

Fix $p\in\mu^{-1}_{1}(0)\cap X_{1}$. Let $Z_{1},\ldots,Z_{n-d}\in
C^{\infty}(U,T^{1,0}\mathbb{C}^{n})$ such that

${\rm span\,}\left\\{\eta-
iJ\eta,Z_{1},\ldots,Z_{n-d};\,\eta\in\underline{\mathfrak{g}}_{x}\right\\}=T^{1,0}_{x}\mathbb{C}^{n},\
\ \mbox{for every $x\in U$},$ (6.12)

where $U$ is a small open set of $p$ in $\mathbb{C}^{n}$ and $J$ is the
complex structure map on $\mathbb{C}^{n}$. It follows from (1.9) and [17,
Theorem 1.10] that there are $f_{0},\ldots,f_{n-d}\in
H^{0}(\overline{M}_{1})^{G}\cap C^{\infty}(\overline{M}_{1})$ such that

$\begin{split}&{\rm det\,}\left((a_{j,\ell})^{n-d}_{j,\ell=0}\right)\neq 0,\\\
&a_{0,\ell}=f_{\ell}(p),\ \ \ell=0,\ldots,n-d,\\\
&a_{j,\ell}=(Z_{j}f_{\ell})(p),\ \
\ell=0,\ldots,n-d,j=1,\ldots,n-d.\end{split}$ (6.13)

From (6.11) and (6.13), combined with the proof of [3, Part 3 of the proof of
Lemma 1], we deduce that there are $n-d+1$ points $a_{0},a_{1},\ldots,a_{n-d}$
in $M_{1}$ such that

$\begin{split}&{\rm det\,}\left((b_{j,\ell})^{n-d}_{j,\ell=0}\right)\neq 0,\\\
&b_{0,\ell}=B_{G,M_{1}}(p,a_{\ell}),\ \ \ell=0,\ldots,n-d,\\\
&b_{j,\ell}=(Z_{j,x}B_{G,M})(p,a_{\ell}),\ \
\ell=0,\ldots,n-d,j=1,\ldots,n-d.\end{split}$ (6.14)

From Lemma 6.5, (6.14) and [3, Lemma 2], we get Theorem 1.6. ∎

## References

  * [1] C. Albert, _Le théorème de réduction de Marsden-Weinstein en géométrie cosymplectique et de contact_ , J. Geom. Phys., 6 (1989), 627-649.
  * [2] M. F. Atiyah, V. K. Patodi and I. M. Singer, _Spectral asymmetry and Riemannian geometry I_ , Proc. Camb. Philos. Soc. 77 (1975), 43-69.
  * [3] S. Bell and E. Ligocka, _A simplification and extension of Fefferman’s theorem on biholomorphic mappings_ , Invent. Math. 57 (1980), no. 3, 283–289.
  * [4] L. Boutet de Monvel, _Boundary problems for pseudo-differential operators_ , Acta Math. 126 (1971), 11-51.
  * [5] L. Boutet de Monvel and V. Guillemin, _The spectral theory of Toeplitz operators_ , Ann. of Math. Stud., vol 99, Princeton Univ. Press, Princeton, NJ, 1981. v+161 pp.
  * [6] L. Boutet de Monvel and J. Sjöstrand, _Sur la singularité des noyaux de Bergman et de Szegö_ , Astérisque, 34–35 (1976), 123–164.
  * [7] L. Charles, _Toeplitz operators and Hamiltonian torus actions_ , J. Funct. Anal. 236 (2006), no. 1, 299–350.
  * [8] S.-C. Chen and M.-C. Shaw, _Partial differential equations in several complex variables_ , AMS/IP Studies in Advanced Mathematics. 19. Providence, RI: American Mathematical Society (AMS). Somerville, MA: International Press, xii, 380 p., (2001).
  * [9] C. Fefferman, _The Bergman kernel and biholomorphic mappings of pseudoconvex domains_ , Invent. Math. 26 (1974), 1–65.
  * [10] G. B. Folland and J. J. Kohn, _The Neumann problem for the Cauchy-Riemann complex_ , Annals of Mathematics Studies, No. 75. Princeton University Press, Princeton, N.J.; University of Tokyo Press, Tokyo, 1972. viii+146 pp.
  * [11] V. Guillemin and S. Sternberg, _Geometric quantization and multiplicities of group representations_ , Invent. Math. 67 (1982), no. 3, 515-538.
  * [12] L. Hörmander, _The analysis of linear partial differential operators_. III. Pseudodifferential operators. Grundlehren der Mathematischen Wissenschaften [Fundamental Principles of Mathematical Sciences], 274. Springer-Verlag, Berlin, 1985. viii+525 pp.
  * [13] P. Hochs and Y. Song, _Equivariant indices of Spin-Dirac operators for proper moment maps_ , Duke Math. J. 166 (2017), 1125-1178.
  * [14] C.-Y. Hsiao, _Projections in several complex variables_ , Mém. Soc. Math. France, Nouv. Sér. 123 (2010).
  * [15] C.-Y. Hsiao, R.-T. Huang, X. Li and G. Shao, _$S^{1}$ -equivariant index theorems and Morse inequalities on complex manifolds with boundary_, J. Funct. Anal. 279 (2020), no. 3, 51 pp.
  * [16] C.-Y. Hsiao and R.-T. Huang, _$G$ -invariant Szegő kernel asymptotics and CR reduction_, Calculus of Variations and PDEs. 60 (2021), no 1, paper No. 47.
  * [17] C.-Y. Hsiao and G. Marinescu, _On the singularities of the Szegő projections on lower energy forms_ , J. Differential Geom. 107 (2017), no. 1, 83–155.
  * [18] C.-Y. Hsaio and G Marinescu, _On the singularities of the Bergman projections for lower energy forms on complex manifolds with boundary_ , arXiv: 1911.10928.
  * [19] C.-Y. Hsiao, X. Ma and G. Marinescu, _Geometric quantization on CR manifolds_ , arXiv: 1906.05627, to appear at Communications in Contemporary Mathematics. DOI: 10.1142/s0219199722500742
  * [20] C.-Y. Hsiao and N. Savale, _Bergman-Szegő kernel asymptotics in weakly pseudoconvex finite type cases_ , J. Reine Angew. Math. 791 (2022), 173–223.
  * [21] X. Ma, _Geometric quantization on Kähler and symplectic manifolds_ , International Congress of Mathematicians, vol. II, Hyderabad, India, August 19-27 (2010), 785–810.
  * [22] X. Ma and W. Zhang,_Geometric quantization for proper moment maps_ , C. R. Math. Acad. Sci. Paris, 347, (2009), 389-394.
  * [23] X. Ma and W. Zhang,_Geometric quantization for proper moment maps: the Vergne conjecture_ , Acta Math., 212, (2014), no. 1, 11–57.
  * [24] E. Meinrenken, _On Riemann-Roch formulas for multiplicities_ , J. Amer. Math. Soc. 9 (1996), 373-389.
  * [25] E. Meinrenken, _Symplectic surgery and the Spinc-Dirac operator_ , Adv. Math. 134 (1998), no. 2, 240-277.
  * [26] A. Melin and J. Sjöstrand, _Fourier integral operators with complex-valued phase functions_ , Springer Lecture Notes in Math., 459, (1975), 120–223.
  * [27] X. Ma and W. Zhang, _Bergman kernels and symplectic reduction_ , Astérisque 318 (2008), viii+154 pp.
  * [28] R. Paoletti, _Moment maps and equivariant Szegő kernels_ , J. Symplectic Geom. 2 (2003), 1, 133–175.
  * [29] P.-E. Paradan, _Formal geometric quantization II_ , Pacific J. Math. 253 (2011), 169-211.
  * [30] Y. Tian and W. Zhang, _An analytic proof of the geometric quantization conjecture of Guillemin-Sternberg_ , Invent. Math. 132, (1998), no. 2, 229-259.
  * [31] Y. Tian and W. Zhang, _Quantization formula for symplectic manifolds with boundary_ , Geom. Funct. Anal. 9, (1999), no. 3, 596-640.
  * [32] M. Vergne, _Multiplicities formula for geometric quantization I, II_ , Duke Math. J. 82 (1996), 143-179, 181-194.
  * [33] M. Vergne, _Quantification géométrique et réduction symplectique_ , Séminaire Bourbaki, Vol. 2000/2001. Asrérisque No. 282 (2002), Exp. No. 888, viii, 249-278.
  * [34] M. Vergne, _Applications of equivariant cohomology_ , International Congress of Mathematicians. Vol. I, Eur. Math. Soc., Zürich, 2007, pp. 635-664.