Let $S(V)$ be the Heisenberg group attached to $V$, and
$J(V)=S(V)\rtimes\operatorname{U}(V)$ be the Jacobi group (see Subsection
5.3). Put $\widetilde{\operatorname{U}_{V}}=\operatorname{U}(V)\times J(V)$.
If $g\in\operatorname{U}(V)$, an element in $J(V)$ whose image in
$\operatorname{U}(V)$ is $g$ is usually denoted by $\widetilde{g}$. An element
in $\widetilde{\operatorname{U}_{V}}$ is usually denoted by $\widetilde{x}$.
This means $\widetilde{x}=(x_{1},\widetilde{x_{2}})$,
$x=(x_{1},x_{2})\in\operatorname{U}_{V}$ and the image of $\widetilde{x}$ in
$\operatorname{U}_{V}$ is $x$.

  * •

The group $\operatorname{U}_{V}^{\prime}$ diagonally embeds in
$\widetilde{\operatorname{U}_{V}}$. Its image is again denoted by
$\operatorname{U}_{V}^{\prime}$. There is a natural map
$J(V)\to\widetilde{\operatorname{U}_{V}}$ and we let
$\widetilde{\operatorname{U}^{\prime}_{V}}$ its image.

  * •

We keep the notation from Subsection 5.3. In particular, we fix a minimal
parabolic subgroup $P_{0}$ of $\operatorname{U}(V)$ which fixes the maximal
isotropic flag (5.10). Standard parabolic subgroups of $\operatorname{U}(V)$
are those containing $P_{0}$. Let $\mathcal{F}_{V}$ be the subset of standard
D-parabolic subgroups of $J(V)$, and let $\mathcal{F}^{\prime}_{V}$ be the set
of standard parabolic subgroup of $\operatorname{U}^{\prime}_{V}$. We put
$P^{\prime}=P\cap\operatorname{U}(V)$ for $P\in\mathcal{F}_{V}$. Then by Lemma
5.7, the map $P\mapsto P^{\prime}$ is a bijection from $\mathcal{F}_{V}$ to
$\mathcal{F}_{V^{\prime}}$.

  * •

Let $P\in\mathcal{F}_{V}$. We denote by $P_{1}$ and $P_{2}$ respectively the
subgroup $P^{\prime}$ of the first and second factor of
$\operatorname{U}_{V}$. We put

(8.1) $\widetilde{P}=P_{1}\times P,\quad P_{\operatorname{U}}=P_{1}\times
P_{2},$

which are D-parabolic subgroups of $\widetilde{\operatorname{U}_{V}}$ and
$\operatorname{U}_{V}$ respectively. The notation $P^{\prime}$ usually
specifically means the parabolic subgroup of $\operatorname{U}_{V}^{\prime}$.

  * •

Let $\operatorname{Res}V$ be the symplectic space defined in Subsection 5.3.
We fix a polarization $\operatorname{Res}V=L\oplus L^{\vee}$ as in Subsection
5.4. We have the Weil representation $\omega=\omega_{\psi,\mu}$ of
$J(V)(\mathbb{A})$ realized on $\mathcal{S}(L^{\vee}(\mathbb{A}))$. For
$\phi\in\mathcal{S}(L^{\vee}(\mathbb{A}))$ and $P\in\mathcal{F}_{V}$ we have
theta theta function $\prescript{}{P}{\theta}(\cdot,\phi)$. We also have
$\omega^{\vee}=\omega_{\psi^{-1},\mu^{-1}}$ and
$\prescript{}{P}{\theta}^{\vee}(\cdot,\phi)$ defined in terms of
$\omega^{\vee}$.

  * •

Put $\operatorname{U}_{V,+}=\operatorname{U}_{V}\times L^{\vee}\times
L^{\vee}$. The group structure is given by the product group structure of
$\operatorname{U}(V)$ and the additive group $L^{\vee}\times L^{\vee}$. Let
$P\in\mathcal{F}_{V}$. We put

$M_{P,+}=M_{P_{\operatorname{U}}}\times M_{L^{\vee}}\times M_{L^{\vee}},\quad
N_{P,+}=N_{P_{\operatorname{U}}}\times N_{L^{\vee}}\times
N_{L^{\vee}}=N_{P_{\operatorname{U}}},\quad P_{+}=M_{P,+}N_{P,+},$

where we recall that $M_{L^{\vee}}$ and $N_{L^{\vee}}$ are defined in
Subsection 5.4. These are subgroups of $\operatorname{U}_{V,+}$. We often
write an element in $M_{P,+}$ as $(m,l_{1},l_{2})$ where $m\in
M_{P_{\operatorname{U}}}$ and $l_{1},l_{2}\in L^{\vee}$.

  * •

Put $\mathfrak{a}_{0}=\mathfrak{a}_{P_{0}}$. A truncation parameter is an
element in $\mathfrak{a}_{0}$.

####  8.2. Technical preparations

Let $P\in\mathcal{F}_{V}$ and $w$ be a weight on
$[\widetilde{\operatorname{U}_{V}}]_{\widetilde{P}}$. We can define various
function spaces as in Subsection 4.3. Of particular interest to us are
$\mathcal{S}_{w}([\widetilde{\operatorname{U}_{V}}]_{\widetilde{P}},\psi)$ and
$\mathcal{T}_{w}([\widetilde{\operatorname{U}_{V}}]_{\widetilde{P}},\psi)$.

The “approximation by the constant term” for the group
$\widetilde{\operatorname{U}_{V}}$ takes the following form.

######  Proposition 8.1.

Let $N>0,r\geq 0$, $X\in\mathcal{U}(\widetilde{\mathfrak{u}_{V}}_{\infty})$
and $P,Q\in\mathcal{F}_{V}$. Then there exists a continuous seminorm
$\|\cdot\|_{N,X,r}$ on
$\mathcal{T}_{N}([\widetilde{\operatorname{U}_{V}}]_{\widetilde{Q}},\psi)$,
such that

$\left|\mathrm{R}(X)\varphi(g)-\mathrm{R}(X)\varphi_{\widetilde{P}}(g)\right|\leq\|g\|^{N}_{P}d_{P_{\operatorname{U}}}^{Q_{\operatorname{U}}}(g)^{-r}\|\varphi\|_{N,X,r}$

holds for all
$\varphi\in\mathcal{T}_{N}([\widetilde{\operatorname{U}_{V}}]_{\widetilde{Q}},\psi)$
and $g\in\operatorname{U}_{V}(\mathbb{A})$.

###### Proof.

For $g\in\operatorname{U}_{V}(\mathbb{A})$, we have
$d_{\widetilde{P}}^{\widetilde{Q}}(g)=d_{P_{1}}^{Q_{1}}(g_{1})d_{P}^{Q}(g_{2})$
and
$d_{P_{\operatorname{U}}}^{Q_{\operatorname{U}}}(g)=d_{P_{1}}^{Q_{1}}(g_{1})d_{P_{2}}^{Q_{2}}(g_{2})$.
Therefore by Lemma 5.8, we have
$\min\\{d_{P_{\operatorname{U}}}^{Q_{\operatorname{U}}}(g),d_{P_{1}}^{Q_{1}}(g_{1})d_{P_{2}}^{Q_{2}}(g_{2})^{\frac{1}{2}}\\}\ll
d_{\widetilde{P}}^{\widetilde{Q}}(g)\ll
d_{P_{\operatorname{U}}}^{Q_{\operatorname{U}}}(g)$, and hence the proposition
follows from Theorem 4.25. ∎

Similarly to what we have done in the case of general linear groups, we
introduce various auxiliary spaces of functions.

Recall that we have the space of functions
$\mathcal{T}_{\mathcal{F}_{V}^{\prime}}(\operatorname{U}^{\prime}_{V})$
introduced in Subsection 4.9. Because there is a bijection between
$\mathcal{F}_{V}$ and $\mathcal{F}_{V}^{\prime}$, we denote it by
$\mathcal{T}_{\mathcal{F}_{V}}(\operatorname{U}^{\prime}_{V})$ and use
$\mathcal{F}_{V}$ as indices. By Theorem 4.25, the space
$\mathcal{T}_{\mathcal{F}_{V}}(\operatorname{U}^{\prime}_{V})$ consists of
tuples

$(\prescript{}{P}{\varphi})_{P\in\mathcal{F}_{V}}\in\prod_{P\in\mathcal{F}_{V}}\mathcal{T}([\operatorname{U}_{V}^{\prime}]_{P^{\prime}}),$

such that

(8.2)
$\prescript{}{P}{\varphi}-(\prescript{}{Q}{\varphi})_{P^{\prime}}\in\mathcal{S}_{d_{P^{\prime}}^{Q^{\prime}}}([\operatorname{U}_{V}^{\prime}]_{P^{\prime}})$

for any $P\subset Q\in\mathcal{F}_{V}$.

######  Lemma 8.2.

If
$({}_{P}\varphi)\in\mathcal{T}_{\mathcal{F}_{V}}(\operatorname{U}_{V}^{\prime})$
and $\phi\in\mathcal{S}(L^{\vee}(\mathbb{A}))$, then the family of products

$P\mapsto({}_{P}\varphi\prescript{}{P}{\theta(\cdot,\phi)})$

belongs to $\mathcal{T}_{\mathcal{F}_{V}}(\operatorname{U}_{V}^{\prime})$.

###### Proof.

By Proposition 5.9 and the above characterization (8.2), the family
$P\mapsto\prescript{}{P}{\theta(\cdot,\phi)}$ belongs to
$\mathcal{T}_{\mathcal{F}_{V}}(\operatorname{U}_{V}^{\prime})$. We thus prove
a stronger statement: if $(\prescript{}{P}{\varphi})$,
$(\prescript{}{P}{\varphi^{\prime}})\in\mathcal{T}_{\mathcal{F}_{V}}(\operatorname{U}_{V}^{\prime})$,
then the family

$P\mapsto(\prescript{}{P}{\varphi}\prescript{}{P}{\varphi^{\prime}})$

belongs to $\mathcal{T}_{\mathcal{F}_{V}}(\operatorname{U}_{V}^{\prime})$.
Using the Leibniz rule, we are reduced to that there exists $N_{0}>0$ such
that for any $X,Y\in\mathcal{U}((\mathfrak{u}^{\prime}_{V})_{\infty})$ and any
$r>0$, we have

$\left|\mathrm{R}(X){}_{P}\varphi(g)\mathrm{R}(Y){}_{P}\varphi^{\prime}(g)-\mathrm{R}(X){}_{Q}\varphi(g)\mathrm{R}(Y){}_{Q}\varphi^{\prime}(g)\right|\ll_{X,Y,r}\|g\|_{P^{\prime}}^{N}d_{P^{\prime}}^{Q^{\prime}}(g)^{-r}$

for any $g\in[\operatorname{U}_{V}^{\prime}]_{P^{\prime}}$. The left hand side
is bounded by

$\left|\mathrm{R}(X){}_{P}\varphi(g)\left(\mathrm{R}(Y){}_{P}\varphi^{\prime}(g)-\mathrm{R}(Y){}_{Q}\varphi^{\prime}(g)\right)\right|+\left|\mathrm{R}(Y){}_{Q}\varphi^{\prime}(g)\left(\mathrm{R}(X){}_{P}\varphi(g)-\mathrm{R}(X){}_{Q}\varphi(g)\right)\right|.$

The desired inequality then follows. ∎

For $P\in\mathcal{F}_{V}$, define a weight function $\Delta_{P}$ on
$[\operatorname{U}_{V}]_{P_{\operatorname{U}}}$ by

$\Delta_{P}(g)=\inf_{\gamma\in
M_{P^{\prime}}(F)N_{P^{\prime}}(\mathbb{A})}\lVert g_{1}^{-1}\gamma
g_{2}\rVert_{P^{\prime}}.$

For $P,Q\in\mathcal{F}_{V}$ with $P\subset Q$, define a weight
$d_{P}^{Q,\Delta}$ on $[\operatorname{U}_{V}]_{P_{\operatorname{U}}}$ by

$d_{P}^{Q,\Delta}(g)=\min\\{d_{P_{1}}^{Q_{1}}(g_{1}),d_{P_{2}}^{Q_{2}}(g_{2})\\}.$

Pulling back under the projection
$[\widetilde{\operatorname{U}_{V}}]_{\widetilde{P}}\to[\operatorname{U}_{V}]_{P_{\operatorname{U}}}$,
we get two weights on $[\widetilde{\operatorname{U}_{V}}]_{\widetilde{P}}$,
which are still denoted by $\Delta_{P}$ and $d_{P}^{Q,\Delta}$.

We define the space
$\mathcal{T}_{\mathcal{F}_{V}}^{\Delta}(\operatorname{U}_{V})$ (resp.
$\mathcal{T}_{\mathcal{F}_{V}}^{\Delta}(\widetilde{\operatorname{U}_{V}},\psi)$)
to be the space of functions

(8.3)
$(\prescript{}{P}{\varphi})\in\prod_{P\in\mathcal{F}_{V}}\mathcal{S}_{\Delta_{P}}([\operatorname{U}_{V}]_{P_{\operatorname{U}}})\quad\text{resp.}\quad\prod_{P\in\mathcal{F}_{V}}\mathcal{S}_{\Delta_{P}}([\widetilde{\operatorname{U}_{V}}]_{\widetilde{P}},\psi).$

such that for any $P\subset Q\in\mathcal{F}_{V}$, we have

${}_{P}\varphi-({}_{Q}\varphi)_{P}\in\mathcal{S}_{d_{P}^{Q,\Delta}}([\operatorname{U}_{V}]_{P_{\operatorname{U}}})\quad\text{resp.}\quad{}_{P}\varphi-({}_{Q}\varphi)_{\widetilde{P}}\in\mathcal{S}_{d_{P}^{Q,\Delta}}([\widetilde{\operatorname{U}_{V}}]_{\widetilde{P}},\psi).$

Since $d_{P}^{Q,\Delta}\ll d_{P}^{Q}$, by Theorem 4.25 and Proposition 8.1
respectively, that a family $({}_{P}\varphi)$ belongs to
$\mathcal{T}_{\mathcal{F}_{V}}^{\Delta}(\operatorname{U}_{V})$ (resp.
$\mathcal{T}_{\mathcal{F}_{V}}^{\Delta}(\widetilde{\operatorname{U}_{V}},\psi)$)
is equivalent to
$\prescript{}{P}{\varphi}\in\mathcal{S}_{\Delta_{P}}([\operatorname{U}_{V}]_{P_{\operatorname{U}}})$
(resp.
$\mathcal{S}_{\Delta_{P}}([\widetilde{\operatorname{U}_{V}}]_{\widetilde{P}},\psi)$
for each $P$ and that for all $P\subset Q\in\mathcal{F}_{V}$, there exists an
$N>0$ such that for all $X\in\mathcal{U}((\mathfrak{u}_{V})_{\infty})$ and all
$r>0$, we have

(8.4)
$\left|\mathrm{R}(X){}_{P}\varphi(g)-\mathrm{R}(X){}_{Q}\varphi(g)\right|\ll_{r,X}\|g\|_{P}^{N}d_{P}^{Q,\Delta}(g)^{-r}.$

holds for all $g\in[\operatorname{U}_{V}]_{P_{\operatorname{U}}}$ (resp.
$[\widetilde{\operatorname{U}_{V}}]_{\widetilde{P}}$).

######  Lemma 8.3.

We have the following assertions.

  1. (1)

For
$({}_{P}\varphi)\in\mathcal{T}^{\Delta}_{\mathcal{F}_{V}}(\widetilde{\operatorname{U}_{V}},\psi)$,
then the family of restrictions
$P\mapsto({}_{P}\varphi|_{[\operatorname{U}_{V}]_{P_{\operatorname{U}}}})$
belong to $\mathcal{T}^{\Delta}_{\mathcal{F}_{V}}(\operatorname{U}_{V})$.

  2. (2)

For
$({}_{P}\varphi)\in\mathcal{T}^{\Delta}_{\mathcal{F}_{V}}(\operatorname{U}_{V})$,
then the family of restrictions
$P\to({}_{P}\varphi|_{[\operatorname{U}_{V}^{\prime}]_{P^{\prime}}})$ belongs
to $\mathcal{T}_{\mathcal{F}_{V}}(\operatorname{U}_{V}^{\prime})$.

###### Proof.

The first follows from the characterizations (8.4) of elements in
$\mathcal{T}^{\Delta}_{\mathcal{F}_{V}}(\operatorname{U}_{V})$ and
$\mathcal{T}^{\Delta}_{\mathcal{F}_{V}}(\widetilde{\operatorname{U}_{V}},\psi)$.
The second follows from the additional fact that
$d_{P}^{Q,\Delta}|_{[\operatorname{U}_{V}^{\prime}]_{P^{\prime}}}=d_{P^{\prime}}^{Q^{\prime}}$.
∎

We fix a $\phi_{0}\in\mathcal{S}(L^{\vee}(\mathbb{A}))$ such that
$\lVert\phi_{0}\rVert_{L^{2}}=1$. For
$f\in\mathcal{S}(\operatorname{U}_{V}(\mathbb{A}))$ and
$\phi_{1}\in\mathcal{S}(L^{\vee}(\mathbb{A}))$, we define a function
$\widetilde{f}\in\mathcal{S}(\widetilde{\operatorname{U}_{V}}(\mathbb{A}),\psi)$
by

(8.5)
$\widetilde{f}(gs)=f(g^{-1})\langle\omega(s)\phi_{0},\overline{\phi_{1}}\rangle,\quad
s\in S(V)(\mathbb{A}),\ g\in\operatorname{U}_{V}(\mathbb{A}).$

For $\varphi\in\mathcal{T}^{0}([\operatorname{U}_{V}]_{P_{V}})$, as in the
case of general linear groups, we define a measure
$\varphi\cdot\overline{\prescript{}{P}{\theta(\cdot,\phi_{0})}}\in\mathcal{T}^{0}([\widetilde{\operatorname{U}_{V}}]_{\widetilde{P}},\psi)$
by

(8.6)
$\langle\varphi\cdot\overline{\prescript{}{P}{\theta(\cdot,\phi_{0})}},\beta\rangle=\int_{[\widetilde{\operatorname{U}_{V}}]_{P_{\operatorname{U}}}}\int_{[S(V)]_{P_{S}}}\beta(sg)\overline{\prescript{}{P}{\theta(sg,\phi_{0})}}\mathrm{d}s\varphi(g),$

where $\beta\in C_{c}([\widetilde{\operatorname{U}_{V}}]_{\widetilde{P}})$.

######  Lemma 8.4.

Let $\varphi\in\mathcal{T}^{0}([\operatorname{U}_{V}^{\prime}])$, then the
family

$P\in\mathcal{F}_{V}\mapsto\mathrm{R}(\widetilde{f})\left(\varphi_{P^{\prime}}\cdot\overline{\prescript{}{P}{\theta(\cdot,\phi_{0})}}\right)$

belongs to
$\mathcal{T}^{\Delta}_{\mathcal{F}_{V}}(\widetilde{\operatorname{U}_{V}},\psi^{-1})$.
Moreover if $y\in[\operatorname{U}_{V}]_{P_{\operatorname{U}}}$ we have

$\mathrm{R}(\widetilde{f})(\varphi\cdot\overline{\prescript{}{P}{\theta(\cdot,\phi_{0})}})(y)=\int_{[\operatorname{U}_{V}^{\prime}]_{P^{\prime}}}K_{f,P}(x,y)\theta^{\vee}(x,\phi_{1})\varphi(x).$

In particular the composition of this map followed by the restriction to
$[\operatorname{U}_{V}]_{P_{\operatorname{U}}}$ is independent from the choice
of $\phi_{0}$.

###### Proof.

This is proved in the same way as in the case of general linear groups, and in
particular Lemma 6.3, Lemma 6.5 and Lemma 6.7. ∎

####  8.3. A modified kernel

Let $\chi\in\mathfrak{X}(\operatorname{U}_{V})$ be a cuspidal datum. For
$f_{+}=f\otimes\phi_{1}\otimes\phi_{2}$ as above, we put

(8.7)
$K_{f\otimes\phi_{1}\otimes\phi_{2},P,\chi}(\widetilde{x},\widetilde{y})=K_{f,P_{\operatorname{U}},\chi}(x,y)\prescript{}{P}{\theta}^{\vee}(\widetilde{x}_{2},\phi_{1})\prescript{}{P}{\theta}(\widetilde{y}_{2},\phi_{2}),\quad\widetilde{x},\widetilde{y}\in[\widetilde{\operatorname{U}_{V}}]_{\widetilde{P}}.$

Where we denote $x,y$ to be the image of $\widetilde{x},\widetilde{y}$ in
$[\operatorname{U}_{V}]_{P_{\operatorname{U}}}$ respectively and
$\widetilde{x}_{2},\widetilde{y}_{2}\in[J(V)]$ are the second component of
$\widetilde{x}$ and $\widetilde{y}$. Let us now explain that the function
$K_{f\otimes\phi_{1}\otimes\phi_{2},P,\chi}$ extends continuously to a smooth
function $K_{f_{+},P,\chi}$ for all
$f_{+}\in\mathcal{S}(\operatorname{U}_{V,+}(\mathbb{A}))$. By Lemma 4.21 and
Proposition 5.9, there exists an $N_{0}>0$ such that for any $N>0$ and
$X\in\mathcal{U}(\widetilde{\mathfrak{u}}_{V,\infty})$, there is a continuous
semi-norm $\lVert\cdot\rVert_{\mathcal{S}}$ on
$\mathcal{S}(\operatorname{U}_{V,+}(\mathbb{A}))$ such that

(8.8)
$\sum_{\chi\in\mathfrak{X}(\operatorname{U})}\left\lvert\mathrm{R}_{\widetilde{y}}(X)K_{f_{+},P,\chi}(\widetilde{x},\widetilde{y})\right\rvert\leq\|f_{+}\|_{\mathcal{S}}\|\widetilde{x}\|_{\widetilde{P}}^{-N}\lVert\widetilde{y}\rVert_{\widetilde{P}}^{N+N_{0}}.$

holds for all
$f_{+}\in\mathcal{S}(\operatorname{U}_{V}(\mathbb{A}))\otimes\mathcal{S}(L^{\vee}(\mathbb{A}))\otimes\mathcal{S}(L^{\vee}(\mathbb{A}))$
(algebraic tensor product). Now let
$f_{+}\in\mathcal{S}(\operatorname{U}_{V,+}(\mathbb{A}))$ and
$f_{+,n}\in\mathcal{S}(\operatorname{U}_{V}(\mathbb{A}))\otimes\mathcal{S}(L^{\vee}(\mathbb{A}))\otimes\mathcal{S}(L^{\vee}(\mathbb{A}))$
a sequence of functions approaching $f_{+}$. Because of the estimate (8.8),
the sequence $K_{f_{+,n},P,\chi}$ is convergent to a function on
$[\widetilde{\operatorname{U}_{V}}]_{\widetilde{P}}\times[\widetilde{\operatorname{U}_{V}}]_{\widetilde{P}}$,
and this convergence is locally uniform for
$(\widetilde{x},\widetilde{y})\in[\widetilde{\operatorname{U}_{V}}]_{\widetilde{P}}\times[\widetilde{\operatorname{U}_{V}}]_{\widetilde{P}}$.
We denote this function by $K_{f_{+},P,\chi}$. It is clearly independent of
the choice of the sequence approximating $f_{+}$. Because the convergence is
locally uniform, $K_{f_{+},P,\chi}$ is a smooth function. Moreover the
estimate (8.8) continues to hold for $K_{f_{+},P,\chi}$. By the symmetry of
$\widetilde{x}$ and $\widetilde{y}$, the estimate (8.8) holds when
$\widetilde{x}$ and $\widetilde{y}$ on the right hand side are switched (and
with a possibly different $\|\cdot\|_{\mathcal{S}}$).

For $T\in\mathfrak{a}_{0}$, $\chi\in\mathfrak{X}(\operatorname{U}_{V})$, and
$x,y\in[\widetilde{\operatorname{U}_{V}}]$, we define modified kernels

$K_{f_{+},\chi}^{T}(x,y)=\sum_{P\in{\mathcal{F}_{V}}}\epsilon_{P}\sum_{\begin{subarray}{c}\gamma\in
P^{\prime}(F)\backslash\operatorname{U}_{V}^{\prime}(F)\\\ \delta\in
P^{\prime}(F)\backslash\operatorname{U}_{V}^{\prime}(F)\end{subarray}}\widehat{\tau}_{P^{\prime}}(H_{P^{\prime}}(\delta
y)-T_{P^{\prime}})K_{f_{+},P,\chi}(\gamma x,\delta y).$

and

$K_{f_{+}}^{T}(x,y)=\sum_{P\in{\mathcal{F}_{V}}}\epsilon_{P}\sum_{\begin{subarray}{c}\gamma\in
P^{\prime}(F)\backslash\operatorname{U}_{V}^{\prime}(F)\\\ \delta\in
P^{\prime}(F)\backslash\operatorname{U}_{V}^{\prime}(F)\end{subarray}}\widehat{\tau}_{P^{\prime}}(H_{P^{\prime}}(\delta
y)-T_{P^{\prime}})K_{f_{+},P}(\gamma x,\delta y).$

Here $\epsilon_{P}=(-1)^{\dim\mathfrak{a}_{P^{\prime}}}$ and
$\widehat{\tau}_{P^{\prime}}$ is the characteristic function of a certain cone
in $\mathfrak{a}_{P^{\prime}}$ defined in Subsection 4.5. In these
definitions, the convergence of the inner sum can be seen as follows. For
fixed $x$ and $y$, there are only finitely many $\delta\in
P^{\prime}(F)\backslash\operatorname{U}_{V}^{\prime}(F)$ (depending on $y$)
such that $\widehat{\tau}_{P^{\prime}}(H_{P^{\prime}}(\delta
y)-T_{P^{\prime}})\neq 0$, cf. [Arthur3]*Lemma 5.1. Hence the estimate (8.8)
implies that the summations defining $K_{f_{+}}^{T}$ and $K_{f_{+},\chi}^{T}$
are absolutely convergent.

######  Proposition 8.5.

For every $N>0$, there exists a continuous semi-norm
$\|\cdot\|_{\mathcal{S},N}$ on
$\mathcal{S}(\operatorname{U}_{V,+}(\mathbb{A}))$ such that for every
$f_{+}\in\mathcal{S}(\operatorname{U}_{V,+}(\mathbb{A}))$ and $T$ sufficiently
positive, we have

(8.9)
$\sum_{\chi\in\mathfrak{X}(\operatorname{U}_{V})}\left|K_{f_{+},\chi}^{T}(x,y)-K_{f_{+},\chi}(x,y)F^{\operatorname{U}_{V}^{\prime}}(x,T)\right|\leq
e^{-N\|T\|}\|x\|_{\operatorname{U}_{V}^{\prime}}^{-N}\|y\|_{\operatorname{U}_{V}^{\prime}}^{-N}\|f_{+}\|_{\mathcal{S},N}.$

In particular for $f_{+}\in\mathcal{S}(\operatorname{U}_{V,+}(\mathbb{A}))$
and $T$ sufficiently positive, the expression

$\sum_{\chi\in\mathfrak{X}(\operatorname{U}_{V})}\int_{[\operatorname{U}_{V}^{\prime}]\times[\operatorname{U}_{V}^{\prime}]}\left|K_{f_{+},\chi}^{T}(x,y)\right|\mathrm{d}x\mathrm{d}y$

is finite and defines a continuous semi-norm on
$\mathcal{S}(\operatorname{U}_{V,+}(\mathbb{A}))$.

###### Proof.

First, since the center of $\operatorname{U}^{\prime}_{V}$ is anisotropic, for
a fixed $T$ the function $x\mapsto F^{\operatorname{U}^{\prime}_{V}}(x,T)$ is
compactly supported. Therefore the second assertion on the absolute
convergence follows from the estimate (8.9).

Next we note that each summand in (8.9) is continuous in $f_{+}$. Thus by
continuity we only need to prove (8.9) when
$f_{+}=f\otimes\phi_{1}\otimes\phi_{2}$ where
$f\in\mathcal{S}(\operatorname{U}_{V}(\mathbb{A}))$,
$\phi_{1},\phi_{2}\in\mathcal{S}(L^{\vee}(\mathbb{A}))$. We will assume that
this is the case from now on.

We fix a $\phi_{0}\in\mathcal{S}(L^{\vee}(\mathbb{A}))$ with
$\lVert\phi_{0}\rVert_{L^{2}}=1$. Consider the following sequence of map

${{\mathcal{T}^{0}([\operatorname{U}_{V}^{\prime}])}}$${\mathcal{T}^{\Delta}_{\mathcal{F}_{V}}(\widetilde{\operatorname{U}_{V}},\psi^{-1})}$${\mathcal{T}^{\Delta}_{\mathcal{F}_{V}}(\operatorname{U}_{V})}$${\mathcal{T}_{\mathcal{F}_{V}}(\operatorname{U}^{\prime}_{V})}$${\mathcal{T}_{\mathcal{F}_{V}}(\operatorname{U}^{\prime}_{V})}$${{\mathcal{S}^{0}([\operatorname{U}^{\prime}_{V}])},}$$\scriptstyle{|_{\operatorname{U}_{V}}}$$\scriptstyle{|_{\operatorname{U}^{\prime}_{V}}}$$\scriptstyle{\cdot\theta}$$\scriptstyle{\Lambda^{T}}$$\scriptstyle{\Pi^{T}}$

where the first map sends
$\varphi\in\mathcal{T}^{0}([\operatorname{U}_{V}^{\prime}])$ to the family

$P\mapsto\mathrm{R}(\widetilde{f})\left(\varphi_{P^{\prime}}\cdot\overline{\prescript{}{P}{\theta(\cdot,\phi_{0})}}\right).$

We recall that $\widetilde{f}$ is defined in (8.5). The second and the third
maps are to restrict the family
$\left(\prescript{}{P}{\varphi}\right)_{P\in{\mathcal{F}_{V}}}$ to
$\left(\prescript{}{P}{\varphi}|_{[\operatorname{U}_{V}]_{P_{\operatorname{U}}}}\right)_{P\in{\mathcal{F}_{V}}}$
and then further to
$\left(\prescript{}{P}{\varphi}|_{[\operatorname{U}^{\prime}_{V}]_{P^{\prime}}}\right)_{P\in{\mathcal{F}_{V}}}$.
The fourth map sends a family
$(\prescript{}{P^{\prime}}{\varphi})_{P\in{\mathcal{F}_{V}}}$ to the family
$\left(\prescript{}{P}{\varphi}\prescript{}{P}{\theta(\cdot,\phi_{2})}\right)_{P\in{\mathcal{F}_{V}}}$.
By Lemma 8.4, Lemma 8.3 and Lemma 8.2, the targets of these maps are as
described. Using the closed graph theorem, one checks that all these maps are
continuous, cf. Remark 6.9. The last map is one of the truncation operators
defined in Proposition 4.27. We denote by $L_{f,\phi_{1},\phi_{2}}$ (resp.
$P_{f,\phi_{1},\phi}$) the composite of the sequence of the maps above, where
we use $\Lambda^{T}$ (resp. $\Pi^{T}$) in the last step.

Similarly, for $\chi\in\mathfrak{X}(\operatorname{U}_{V})$, we consider the
same chain of continuous linear maps as above, but we project to the
$\chi^{\vee}$-component before restricting to $\operatorname{U}_{V}^{\prime}$.
The resulting maps are denoted by $L_{f,\phi_{1},\phi_{2},\chi}$ and
$P_{f,\phi_{1},\phi_{2},\chi}$ respectively.

As in the proof of Theorem 6.8, the functions
$K^{T}_{f\otimes\phi_{1}\otimes\phi_{2}}(x,y)$ and
$K_{f\otimes\phi_{1}\otimes\phi_{2}}(x,y)F^{\operatorname{U}^{\prime}_{V}}(x,T)$
are the kernel functions of $L_{f,\phi_{1},\phi_{2}}$ (resp.
$P_{f,\phi_{1},\phi_{2}}$) respectively. The same is true with a
$\chi\in\mathfrak{X}(\operatorname{U}_{V})$ in the subscript. By Proposition
4.27, for any fixed $N$ and for all
$\varphi\in\mathcal{T}^{0}([\operatorname{U}^{\prime}_{V}])$, we have

$\sum_{\chi\in\mathfrak{X}(U)}\|L_{f,\phi_{1},\phi_{2},\chi}(\varphi)-P_{f,\phi_{1},\phi_{2},\chi}(\varphi)\|\ll
e^{-N\|T\|}\|\varphi\|_{1,-N}.$

The rest of the argument is to show that the implicit constant in this
estimate can be taken to be continuous seminorms of $f$, $\phi_{1}$ and
$\phi_{2}$. This is a consequence of the uniform boundedness principle, and
the argument is the same as the proof of Theorem 6.8. ∎

####  8.4. The coarse spectral expansion

For $\chi\in\mathfrak{X}(\operatorname{U}_{V})$,
$f_{+}\in\mathcal{S}(\operatorname{U}_{V,+}(\mathbb{A}))$ and
$T\in\mathfrak{a}_{0}$ a truncation parameter, we put

$J^{T}(f_{+})=\int_{[\operatorname{U}_{V}^{\prime}]\times[\operatorname{U}_{V}^{\prime}]}K_{f_{+}}^{T}(x,y)\mathrm{d}x\mathrm{d}y,\quad
J^{T}_{\chi}(f_{+})=\int_{[\operatorname{U}_{V}^{\prime}]\times[\operatorname{U}_{V}^{\prime}]}K_{f_{+},\chi}^{T}(x,y)\mathrm{d}x\mathrm{d}y.$

By Proposition 8.5, these integrals are absolutely convergent when $T$ is
sufficiently positive.

We define an action $\mathrm{L}_{+}$ of
$\operatorname{U}_{V}^{\prime}(\mathbb{A})$ on
$\mathcal{S}(\operatorname{U}_{V,+}(\mathbb{A}))$ by

$\mathrm{L}_{+}(h)f_{+}(m,l_{1},l_{2})=\left(\omega^{\vee}(h)f_{+}(h^{-1}m,\cdot,l_{2})\right)(l_{1}),\quad
m\in\operatorname{U}_{V}(\mathbb{A}),\ l_{1},l_{2}\in L^{\vee}(\mathbb{A}).$

The right hand side means that we first evaluate $f_{+}$ at $h^{-1}m$ and
$l_{2}$ to obtain a Schwartz function in the variable $l_{1}$. We apply the
Weil representation $\omega^{\vee}(h)$ to this Schwartz function and finally
evaluate at $l_{1}$. We similarly define an action $\mathrm{R}_{+}$ of
$\operatorname{U}_{V}^{\prime}(\mathbb{A})$ on
$\mathcal{S}(\operatorname{U}_{V,+}(\mathbb{A}))$ by

$\mathrm{R}_{+}(h)f_{+}(m,l_{1},l_{2})=\left(\omega(h)f_{+}(mh,l_{1},\cdot)\right)(l_{2}),\quad
m\in\operatorname{U}_{V}(\mathbb{A}),\ l_{1},l_{2}\in L^{\vee}(\mathbb{A}).$

The right hand side is interpreted similarly as in the case $\mathrm{L}_{+}$.

Using the action $L_{+}$ and $R_{+}$, for $P\in\mathcal{F}_{V}$ and
$f_{+}\in\mathcal{S}(\operatorname{U}_{V,+}(\mathbb{A}))$, the kernel function
$K_{f_{+},P}$ can also be written as

(8.10) $K_{f_{+},P}(x,y)=\sum_{m\in
M_{P_{+}}(F)}\int_{N_{P_{+}}(\mathbb{A})}\mathrm{L}_{+}(x)\mathrm{R}_{+}(y)f_{+}(mn)\mathrm{d}n$

where $x,y\in[\operatorname{U}^{\prime}_{V}]$.

######  Theorem 8.6.

As a function of $T$, the functions $J^{T}(f_{+})$ and $J^{T}_{\chi}(f_{+})$
are the restrictions of exponential polynomials whose purely polynomial term
are constants denoted by $J(f_{+})$ and $J_{\chi}(f_{+})$ respectively. The
linear forms $f_{+}\mapsto J(f_{+})$ and $f_{+}\mapsto J_{\chi}(f_{+})$ are
continuous and bi-$\operatorname{U}_{V}^{\prime}(\mathbb{A})$-invariant, i.e.

$J_{\chi}(\mathrm{L}_{+}(h_{1})\mathrm{R}_{+}(h_{2})f_{+})=J_{\chi}(f_{+}),\quad
J(\mathrm{L}_{+}(h_{1})\mathrm{R}_{+}(h_{2})f_{+})=J(f_{+}),\quad
h_{1},h_{2}\in\operatorname{U}_{V}^{\prime}(\mathbb{A}).$

Finally we have

$J(f_{+})=\sum_{\chi\in\mathfrak{X}(\operatorname{U}_{V})}J_{\chi}(f_{+})$

where the sum is absolutely convergent.

Before we delve into the proof of this theorem, let us first explain a variant
of the construction of the modified kernel for parabolic subgroups. Let us
take $Q=M_{Q}N_{Q}\in\mathcal{F}_{V}$. Then $Q_{\operatorname{U}}=Q_{1}\times
Q_{2}$ where $Q_{1}=Q_{2}$ are parabolic subgroup of $\operatorname{U}(V)$ and
$Q^{\prime}=Q_{1}=Q_{2}$ is a parabolic subgroup of
$\operatorname{U}_{V}^{\prime}$. Assume that $Q^{\prime}$ is the stabilizer of
the isotropic flag

(8.11) $0=X_{0}\subset X_{1}\subset\cdots\subset X_{r}$

in $V$. Recall that we constructed a decomposition $V=X\oplus V^{\prime}\oplus
X^{\vee}$ in Subsection 5.3, where $X=X_{r}$, $V^{\prime}=M_{Q_{V}}$, and
$V^{\prime}$ is perpendicular to $X\oplus X^{\vee}$ . We have a polarizations
$\operatorname{Res}V=L\oplus L^{\vee}$ and another polarization
$\operatorname{Res}V^{\prime}=L^{\prime}\oplus L^{\prime\vee}$ where
$L^{\prime}=V^{\prime}\cap L$ and $L^{\prime\vee}=V^{\prime}\cap L^{\vee}$. In
particular $L=L^{\prime}\oplus X$ and $L^{\vee}=L^{\prime\vee}+X^{\vee}$.

The Levi subgroup $M_{Q^{\prime}}$ is isomorphic to

$\prod_{i=1}^{r}\operatorname{GL}(X_{i}/X_{i-1})\times\operatorname{U}(V_{0}).$

We write $m\in M_{Q^{\prime}}$ as $(m_{\bullet},m_{*})$ where
$m_{\bullet}=(m_{1},\ldots,m_{r})$,
$m_{i}\in\operatorname{GL}(X_{i}/X_{i-1})$, and
$m_{*}\in\operatorname{U}(V_{0})$. Recall that in Subsection 8.1, we have
defined

$M_{Q_{2},+}=M_{Q_{2}}\times M_{Q_{L^{\vee}}}\times
M_{Q_{L^{\vee}}}=\left(\prod_{i=1}^{r}\operatorname{GL}(X_{i}/X_{i-1})\times\operatorname{U}(V^{\prime})\right)\times
L^{\prime\vee}\times L^{\prime\vee}.$

We have the D-Levi component $M_{\widetilde{Q}}=M_{Q_{1}}\times M_{Q}$ of
$\widetilde{Q}$, where

$M_{Q}=\prod_{i=1}^{r}\operatorname{GL}(X_{i}/X_{i-1})\times J(V^{\prime})$

is the D-Levi component of $Q$. If $\widetilde{m}\in M_{Q}$, we still denote
by $\widetilde{m}_{*}$ its component in $J(V_{0})$.

Take a $P\in\mathcal{F}_{V}$ with $P\subset Q$. Then $P\cap
M_{Q_{\operatorname{U}}}$ is a parabolic subgroup of
$M_{Q_{\operatorname{U}}}$. Denote temporarily by
$\iota_{Q_{\operatorname{U}}}:\mathfrak{X}(M_{Q_{\operatorname{U}}})\to\mathfrak{X}(\operatorname{U}_{V})$
the natural finite-to-one map. For $\chi\in\mathfrak{X}(\operatorname{U}_{V})$
and $f\in\mathcal{S}(M_{Q_{\operatorname{U}}}(\mathbb{A}))$, we put

$K_{f,P\cap
M_{Q_{\operatorname{U}}},\chi}=\sum_{\chi^{\prime}\in\iota_{Q_{\operatorname{U}}}^{-1}(\chi)}K_{f,P\cap
M_{Q_{\operatorname{U}}},\chi^{\prime}}$

where $K_{f,P\cap M_{Q_{\operatorname{U}}},\chi^{\prime}}$ is the kernel
function on $M_{Q_{\operatorname{U}}}$. Let
$\phi_{1},\phi_{2}\in\mathcal{S}(L^{\prime\vee}(\mathbb{A}))$ and
$f_{+}^{\prime}=f\otimes\phi_{1}\otimes\phi_{2}\in\mathcal{S}(M_{Q,+}(\mathbb{A}))$
be a test function. We define

$K_{f_{+}^{\prime},P\cap M_{Q},\chi}(x,y)=K_{f,P\cap
M_{Q_{\operatorname{U}}},\chi}(x,y)\cdot{}_{P\cap
J(V^{\prime})}\theta^{\vee}(x,\phi_{1})\cdot{}_{P\cap
J(V^{\prime})}\theta(y,\phi_{2})\mu^{-1}(\det x_{\bullet})\mu(\det
y_{\bullet}),$

where $(x,y)\in[M_{Q^{\prime}}]_{P^{\prime}\cap
M_{Q^{\prime}}}\times[M_{Q^{\prime}}]_{P^{\prime}\cap M_{Q^{\prime}}}$, and
the intersection $P\cap J(V^{\prime})$ is taken inside $J(V)$. As in
Subsection 8.3, we define $K_{f^{\prime}_{+},P\cap M_{Q},\chi}$ for all
$f^{\prime}_{+}\in\mathcal{S}(M_{Q_{+}}(\mathbb{A}))$ by continuity.

For $T\in\mathfrak{a}_{0}$, define

$K_{f^{\prime}_{+},\chi}^{M_{Q},T}(x,y)=\sum_{\begin{subarray}{c}P\subset Q\\\
P\in\mathcal{F}_{V}\end{subarray}}\epsilon_{P}^{Q}\sum_{\begin{subarray}{c}\gamma\in(M_{Q^{\prime}}\cap
P^{\prime})(F)\backslash M_{Q^{\prime}}(F)\\\ \delta\in(M_{Q^{\prime}}\cap
P^{\prime})(F)\backslash
M_{Q^{\prime}}(F)\end{subarray}}\widehat{\tau}_{P^{\prime}}^{Q^{\prime}}(H_{P^{\prime}}(\delta
y)-T_{P^{\prime}})K_{f_{+}^{\prime},P,\chi}(\gamma x,\delta y).$

where $(x,y)\in[M_{Q^{\prime}}]\times[M_{Q^{\prime}}]$. Using similar methods
as the proof of Proposition 8.5, we can show that when $T$ is sufficiently
positive, for any $f_{+}^{\prime}\in\mathcal{S}(M_{Q_{+}}(\mathbb{A}))$, the
integral

$\int_{A_{Q^{\prime}}^{\infty}\backslash[M_{Q^{\prime}}]\times[M_{Q^{\prime}}]}\left|K_{f^{\prime}_{+},\chi}^{M_{Q},T}(x,y)\right|\mathrm{d}x\mathrm{d}y$

is convergent and defines a continuous seminorm on
$\mathcal{S}(M_{Q_{+}}(\mathbb{A}))$. Here $A_{Q^{\prime}}^{\infty}$ embeds in
$M_{Q^{\prime}}\times M_{Q^{\prime}}$ diagonally. We thus define a
distribution

$J_{\chi}^{M_{Q},T}(f_{+}^{\prime})=\int_{A_{Q^{\prime}}^{\infty}\backslash[M_{Q^{\prime}}]\times[M_{Q^{\prime}}]}K_{f^{\prime}_{+},\chi}^{M_{Q},T}(x,y)\mathrm{d}x\mathrm{d}y.$

###### Proof of Theorem 8.6.

We define $\underline{\rho}_{Q}$ be the unique element in
$\mathfrak{a}_{Q^{\prime}}^{*}$ such that for any $m\in
M_{Q^{\prime}}(\mathbb{A})$ we have

$e^{\langle\underline{\rho}_{Q},H_{Q^{\prime}}(m)\rangle}=\lvert\det
m_{\bullet}\rvert.$

One check directly that $\underline{\rho}_{Q}$ concides with the definition of
[Zydor3]*Lemma 4.3.

By [Arthur1]*Section 2, there exist functions $\Gamma^{\prime}_{Q^{\prime}}$
on
$\mathfrak{a}_{Q^{\prime}}^{\operatorname{U}_{V}^{\prime}}\times\mathfrak{a}_{Q^{\prime}}^{\operatorname{U}_{V}^{\prime}}$,
for $Q\in\mathcal{F}_{V}$, that are compactly supported in the first variable
when the second variable stays in a compact and such that

(8.12) $\widehat{\tau}_{Q^{\prime}}(H-X)=\sum_{P\subset
Q\in\mathcal{F}_{V}}\epsilon_{Q^{\prime}}^{\operatorname{U}_{V}^{\prime}}\widehat{\tau}_{P^{\prime}}^{Q^{\prime}}(H)\Gamma^{\prime}_{Q^{\prime}}(H,X).$

Define a function $p_{Q}$ on $\mathfrak{a}_{Q^{\prime}}$ by

(8.13)
$p_{Q}(X)=\int_{\mathfrak{a}_{Q^{\prime}}}e^{\langle\underline{\rho}_{Q},H\rangle}\Gamma^{\prime}_{Q^{\prime}}(H,X)\mathrm{d}H.$

By [Zydor3]*Lemma 4.3 $p_{Q}$ is an exponential polynomial on
$\mathfrak{a}_{Q^{\prime}}$ with exponents contained in the set
$\\{\underline{\rho}_{R}\mid R\supset Q\\}$ and the pure polynomial term is
the constant
$\epsilon_{Q}\widehat{\theta}_{Q^{\prime}}(\underline{\rho}_{Q})^{-1}$ where
$\widehat{\theta}_{Q^{\prime}}$ is a homogeneous polynomial on
$\mathfrak{a}_{Q^{\prime}}$ defined in [Arthur1]*Section 2.

For
$Q\in\mathcal{F}_{V},f_{+}\in\mathcal{S}(\operatorname{U}_{V,+}(\mathbb{A}))$
and $T\in\mathfrak{a}_{0}$, define

$K_{f_{+},\chi}^{Q,T}(x,y)=\sum_{\begin{subarray}{c}P\subset Q\\\
P\in\mathcal{F}_{V}\end{subarray}}\epsilon_{P}^{Q}\sum_{\begin{subarray}{c}\gamma\in
P^{\prime}(F)\backslash Q^{\prime}(F)\\\ \delta\in P^{\prime}(F)\backslash
Q^{\prime}(F)\end{subarray}}\widehat{\tau}_{P^{\prime}}^{Q^{\prime}}(H_{P^{\prime}}(\delta
y)-T_{P^{\prime}})K_{f_{+},P,\chi}(\gamma x,\delta y),$

where
$(x,y)\in[\operatorname{U}_{V}^{\prime}]_{P^{\prime}}\times[\operatorname{U}_{V}^{\prime}]_{P^{\prime}}$.
Here $\epsilon_{P}^{Q}=(-1)^{\dim\mathfrak{a}_{P^{\prime}}^{Q^{\prime}}}$ and
$\widehat{\tau}_{P^{\prime}}^{Q^{\prime}}$ is the characteristic function
defined in Subsection 4.5. Using the inversion formula (8.12), for
$T,T^{\prime}\in\mathfrak{a}_{0}$ we have

$K_{f_{+},\chi}^{T}(x,y)=\sum_{Q\in\mathcal{F}_{V}}\sum_{\begin{subarray}{c}\gamma\in
Q^{\prime}(F)\backslash\operatorname{U}^{\prime}(F)\\\ \delta\in
Q^{\prime}(F)\backslash\operatorname{U}^{\prime}(F)\end{subarray}}\Gamma^{\prime}_{Q^{\prime}}(H_{Q^{\prime}}(\delta
y)-T^{\prime}_{Q^{\prime}},T_{Q^{\prime}}-T^{\prime}_{Q^{\prime}})K_{f_{+},\chi}^{Q,T^{\prime}}(\gamma
x,\delta y).$

We now relate $K_{f_{+},\chi}^{Q,T^{\prime}}$ to
$K_{f_{+},\chi}^{M_{Q},T^{\prime}}$ via parabolic descent. For
$f_{+}\in\mathcal{S}(\operatorname{U}_{V,+}(\mathbb{A}))$, we define its
parabolic descent as

$f_{+,Q}(m,l_{1},l_{2})=e^{\langle-2\rho_{Q^{\prime}}+\frac{1}{2}\underline{\rho}_{Q},H_{Q^{\prime}}(m_{1})\rangle}\int_{K^{\prime}}\int_{K^{\prime}}\int_{N_{Q_{\operatorname{U}}}(\mathbb{A})}\left(\mathrm{L}_{+}(k_{1})\mathrm{R}_{+}(k_{2})f_{+}\right)(mn_{Q},l_{1},l_{2})\mathrm{d}n_{Q}\mathrm{d}k_{1}\mathrm{d}k_{2},$

where $m\in M_{Q_{\operatorname{U}}}(\mathbb{A})$, $l_{1},l_{2}\in
L^{\prime}(\mathbb{A})$ . The element $m_{1}$ stands for the first component
of $m$ and we regard it as an element of $M_{Q^{\prime}}$ under the natural
identification $M_{Q_{1}}\cong M_{Q^{\prime}}$. We have
$f_{+,Q}\in\mathcal{S}(M_{Q_{+}}(\mathbb{A}))$, and for
$x,y\in[M_{Q^{\prime}}]$ and $P\subset Q\in\mathcal{F}_{V}$, we have

$\int_{K^{\prime}}\int_{K^{\prime}}K_{f_{+},P,\chi}(xk_{1},yk_{2})\mathrm{d}k_{1}\mathrm{d}k_{2}=e^{\langle
2\rho_{Q^{\prime}},H_{Q^{\prime}}(x)\rangle}e^{\langle
2\rho_{Q^{\prime}}+\underline{\rho}_{Q},H_{Q^{\prime}}(y)\rangle}K_{f_{+,Q},P\cap
M_{Q},\chi}(x,y).$

Indeed using (8.10) and the mixed model described in Subsection 5.4, direct
calculations give the identity without $\chi$. The argument in [Zydor3]*Lemma
1.3 shows that this implies the identity with the $\chi$. It follows that

$\int_{K^{\prime}}\int_{K^{\prime}}K^{T}_{f_{+},\chi}(xk_{1},yk_{2})\mathrm{d}k_{1}\mathrm{d}k_{2}=e^{\langle
2\rho_{Q^{\prime}},H_{Q^{\prime}}(x)\rangle}e^{\langle
2\rho_{Q^{\prime}}+\underline{\rho}_{Q},H_{Q^{\prime}}(y)\rangle}K_{f_{+,Q},\chi}^{M_{Q},T}(x,y).$

The rest of the calculation is the same as that in the proof of Theorem 6.10.
We omit the details and only record the final outcome. We have

$J_{\chi}^{T}(f_{+})=\sum_{Q\in\mathcal{F}_{V}}e^{\langle\underline{\rho}_{Q^{\prime}},T^{\prime}\rangle}p_{Q}(T_{Q^{\prime}}-T^{\prime}_{Q^{\prime}})J_{\chi}^{M_{Q},T^{\prime}}(f_{+,Q}).$

It is an exponential polynomial in $T$ whose purely polynomial term is a
constant that equals

$\sum_{Q\in\mathcal{F}_{V}}\epsilon_{Q}\widehat{\theta}_{Q^{\prime}}(\underline{\rho}_{Q})^{-1}e^{\langle\underline{\rho}_{Q^{\prime}},T^{\prime}\rangle}p_{Q}(T_{Q^{\prime}}-T^{\prime}_{Q^{\prime}})J_{\chi}^{M_{Q},T^{\prime}}(\widetilde{f_{+,Q}}).$

The invariance of the linear form $J_{\chi}$ is shown as that of Proposition
6.12. This concludes the proof. ∎

####  8.5. An alternative truncation operator

For later use we will need another truncation operator for Fourier–Jacobi
periods. It is an analogue of the regularized periods of Ichino and Yamana
[IY2].

Take $P\in\mathcal{F}_{V}$. Then $\operatorname{U}(V)\times P$ is a
D-parabolic subgroup of $\widetilde{\operatorname{U}_{V}}$. For
$\varphi\in\mathcal{T}([\widetilde{\operatorname{U}_{V}}],\psi)$, we can speak
of the constant term $\varphi_{\operatorname{U}(V)\times P}$. This amounts to
viewing the function $\varphi$ as a function in two variables
$(x_{1},\widetilde{x_{2}})\in[\operatorname{U}(V)]\times[J(V)]$, fixing
$x_{1}$, and taking the constant term along $P$ in the second variable
$\widetilde{x_{2}}$.

For $\varphi\in\mathcal{T}([\widetilde{\operatorname{U}_{V}}],\psi)$ and
$x\in[\operatorname{U}^{\prime}_{V}]$, we define

(8.14)
$\Lambda^{T}_{u}\varphi(x)=\sum_{P\in\mathcal{F}_{V}}\epsilon_{P}\sum_{\delta\in
P^{\prime}(F)\backslash\operatorname{U}^{\prime}_{V}(F)}\widehat{\tau}_{P^{\prime}}(H_{P^{\prime}}(\delta
x)-T_{P^{\prime}})\varphi_{\operatorname{U}(V)\times P}(\delta x).$

######  Proposition 8.7.

We have the following assertions.

  1. (1)

$\Lambda_{u}^{T}\varphi\in\mathcal{S}^{0}([\operatorname{U}^{\prime}_{V}])$.
And the map $\varphi\mapsto\Lambda_{u}^{T}\varphi$ induces a continuous map
$\mathcal{T}([\widetilde{\operatorname{U}_{V}}],\psi)\to\mathcal{S}^{0}([\operatorname{U}_{V}^{\prime}])$.

  2. (2)

For every $N>0$, there exists a continious seminorm $\lVert\cdot\rVert_{N}$ on
$\mathcal{T}([\widetilde{\operatorname{U}_{V}}],\psi)$ such that for all
$x\in[\operatorname{U}_{V}^{\prime}]$ and
$\varphi\in\mathcal{T}([\widetilde{\operatorname{U}_{V}}],\psi)$, we have

$\left|\Lambda^{T}_{u}\varphi(x)-F^{\operatorname{U}_{V}^{\prime}}(x,T)\varphi(x)\right|\leq
e^{-N\|T\|}\|x\|_{\operatorname{U}_{V}^{\prime}}^{-N}\lVert\varphi\rVert_{N}.$

###### Proof.

First note $d_{\operatorname{U}(V)\times P}^{\operatorname{U}(V)\times
Q}|_{\operatorname{U}^{\prime}_{V}}=d_{P}^{Q}$. Thus by Theorem 4.25 and Lemma
5.8, there exists an $N_{0}>0$ such that for any $P\subset
Q\in\mathcal{F}_{V}$ and
$X\in\mathcal{U}((\mathfrak{u}^{\prime}_{V})_{\infty})$, we have

(8.15) $\lvert\mathrm{R}(X)\varphi_{\operatorname{U}(V)\times
P}(x)-\mathrm{R}(X)\varphi_{\operatorname{U}(V)\times Q}(x)\rvert\ll
d_{P^{\prime}}^{Q^{\prime}}(x)^{-r}\|x\|_{P^{\prime}}^{N_{0}}.$

By (8.2), this is equivalent to the fact that the family
$P\mapsto\left(\varphi_{\operatorname{U}(V)\times
P}\right)|_{[\operatorname{U}_{V}^{\prime}]_{P^{\prime}}}$ belongs to
$\mathcal{T}_{\mathcal{F}_{V}^{\prime}}(\operatorname{U}_{V}^{\prime})$. The
second assertion follows from this by Proposition 4.27. The first assertion
follows from the second since the function $x\mapsto
F^{\operatorname{U}_{V}^{\prime}}(x,T)$ is compactly supported. ∎

Recall that for $f_{+}\in\mathcal{S}(\operatorname{U}_{V,+}(\mathbb{A}))$ and
$\chi\in\mathfrak{X}(\operatorname{U}_{V})$, we have a kernel function
$K_{f_{+},\chi}(\widetilde{x},\widetilde{y})$ for
$\widetilde{x},\widetilde{y}\in[\widetilde{\operatorname{U}_{V}}]$ (cf. (8.7)
for the definition for pure tensors). Applying $\Lambda_{u}^{T}$ to the second
variable, we get a function on
$[\widetilde{\operatorname{U}_{V}}]\times[\operatorname{U}_{V}^{\prime}]$
denoted by $K_{f_{+},\chi}\Lambda^{T}_{u}$.

######  Proposition 8.8.

We have the following assertions.

  1. (1)

For $f_{+}\in\mathcal{S}(\operatorname{U}_{V,+}(\mathbb{A}))$ and $T$
sufficiently positive, the expression

$\sum_{\chi\in\mathfrak{X}(\operatorname{U}_{V})}\int_{[\operatorname{U}_{V}^{\prime}]\times[\operatorname{U}_{V}^{\prime}]}\left|K_{f_{+},\chi}\Lambda^{T}_{u}(x,y)\right|\mathrm{d}x\mathrm{d}y$

is finite and defines a continuous seminorm on
$\mathcal{S}(\operatorname{U}_{V,+}(\mathbb{A}))$.

  2. (2)

For any $r>0$, there exists a continuous seminorm $\lVert\cdot\rVert$ on
$\mathcal{S}(\operatorname{U}_{V,+}(\mathbb{A}))$ such that

$\left|J_{\chi}^{T}(f_{+})-\int_{[\operatorname{U}_{V}^{\prime}]\times[\operatorname{U}_{V}^{\prime}]}K_{f_{+},\chi}\Lambda^{T}_{u}(x,y)\mathrm{d}x\mathrm{d}y\right|\leq
e^{-r\|T\|}\|f_{+}\|.$

for all $T$ sufficiently positive and
$f_{+}\in\mathcal{S}(\operatorname{U}_{V,+}(\mathbb{A}))$. In particular, the
absolutely convergent integral

$\int_{[\operatorname{U}_{V}^{\prime}]\times[\operatorname{U}_{V}^{\prime}]}K_{f_{+},\chi}\Lambda^{T}_{u}(x,y)\mathrm{d}x\mathrm{d}y$

is asymptotic to an exponential-polynomial in $T$, whose purely polynomial
term is a constant that equals $J_{\chi}(f_{+})$.

###### Proof.

By Proposition 8.7, for every $N>0$ there exists a continuous semi-norm
$\|\cdot\|_{\mathcal{T}}$ on
$\mathcal{T}_{N}([\widetilde{\operatorname{U}_{V}}],\psi)$ such for all
$x,y\in[\operatorname{U}_{V}^{\prime}]$, we have

$\sum_{\chi\in\mathfrak{X}(\operatorname{U}_{V})}\left|K_{f_{+},\chi}\Lambda_{u}^{T}(x,y)-F^{\operatorname{U}_{V}^{\prime}}(y,T)K_{f_{+},\chi}(x,y)\right|\leq
e^{-N\|T\|}\|y\|_{\operatorname{U}_{V}^{\prime}}^{-N}\sum_{\chi\in\mathfrak{X}(\operatorname{U}_{V})}\|K_{f_{+},\chi}(x,\cdot)\|_{\mathcal{T}},$

where $K_{f_{+},\chi}(x,\cdot)$ is regarded as an element in
$\mathcal{T}_{N}([\widetilde{\operatorname{U}_{V}}],\psi)$. By (8.8), there
exists $N_{0}>0$ and a continuous semi-norm $\|\cdot\|_{\mathcal{S}}$ on
$\mathcal{S}(\operatorname{U}_{V,+}(\mathbb{A}))$ such that

$\sum_{\chi\in\mathfrak{X}(\operatorname{U}_{V})}\|K_{f_{+},\chi}(\widetilde{x},\cdot)\|_{\mathcal{T}}\ll\|f_{+}\|_{\mathcal{S}}\|\widetilde{x}\|_{\widetilde{\operatorname{U}_{V}}}^{-N+N_{0}}.$

Combining the above two equations we obtain

$\sum_{\chi\in\mathfrak{X}(\operatorname{U}_{V})}\left|K_{f_{+},\chi}\Lambda_{u}^{T}(x,y)-F^{\operatorname{U}_{V}^{\prime}}(y,T)K_{f_{+},\chi}(x,y)\right|\leq
e^{-N\|T\|}\|f_{+}\|_{\mathcal{S}}\|x\|_{\operatorname{U}_{V}^{\prime}}^{-N+N_{0}}\|y\|_{\operatorname{U}_{V}^{\prime}}^{-N}.$

This proves the first assertion. The second assertion follows from the first
and Proposition 8.5. ∎

###  9\. The coarse geometric expansion: unitary groups

####  9.1. Geometric modified kernels

We keep the notation from the previous section. We also need the following
additional notation.

  * •

We put $\operatorname{U}_{V}^{+}=\operatorname{U}_{V}\times V$. The group
structure is given by the product of those of $\operatorname{U}_{V}$ and of
the additive group $V$. Note that this is not a subgroup of
$\widetilde{\operatorname{U}_{V}}$ but merely a closed subvariety. The group
$\operatorname{U}_{V}^{\prime}\times\operatorname{U}_{V}^{\prime}$ acts on
$\operatorname{U}_{V}^{+}$ from the right by
$(g,v)\cdot(x,y)=(x^{-1}gy,y^{-1}v)$.

  * •

Let $P\in\mathcal{F}_{V}$, we put

$P^{+}=P_{\operatorname{U}}\times P_{V},\quad
M_{P}^{+}=M_{P_{\operatorname{U}}}\times M_{P_{V}},\quad
N_{P}^{+}=N_{P_{\operatorname{U}}}\times N_{P_{V}}.$

These are subgroups of $\operatorname{U}_{V}^{+}$.

  * •

By [CZ]*Lemma 15.1.4.1 the categorical quotient
$\operatorname{U}_{V}^{+}//(\operatorname{U}^{\prime}_{V}\times\operatorname{U}^{\prime}_{V})$
is canonically identified with $\mathcal{A}=G^{+}//(H\times G^{\prime})$. The
canonical morphism $q_{V}:\operatorname{U}_{V}^{+}\to\mathcal{A}$ is given by

$((g_{1},g_{2}),v)\mapsto(a_{1},\ldots,a_{n};b_{1},\ldots,b_{n})$

where

$a_{i}=\operatorname{Trace}\wedge^{i}(g_{1}^{-1}g_{2}),\quad
b_{i}=2(-1)^{n-1}\tau^{-1}q_{V}(g_{1}^{-1}g_{2}v,v).$

Here $\mathcal{A}$ is viewed as a locally closed subscheme of
$\operatorname{Res}_{E/F}\mathbf{A}_{2n,E}$ as in Subsection 7.1.

  * •

For $\alpha\in\mathcal{A}(F)$, let $\operatorname{U}^{+}_{V,\alpha}$ be the
preimage of $\alpha$ in $\operatorname{U}^{+}_{V}$ as a closed subscheme. We
also put

$M^{+}_{P,\alpha}=\operatorname{U}^{+}_{V,\alpha}\cap M^{+}_{P}.$

For $f^{+}\in\mathcal{S}(\operatorname{U}_{V}^{+}(\mathbb{A}))$ and
$P\in\mathcal{F}_{V}$, we define

$k_{f^{+},P}(x,y)=\sum_{m^{+}\in
M_{P}^{+}(F)}\int_{N_{P}^{+}(\mathbb{A})}f^{+}(m^{+}n^{+}\cdot(x,y))\mathrm{d}n^{+},\quad
x,y\in[\operatorname{U}_{V}^{\prime}]_{P^{\prime}}.$

For $\alpha\in\mathcal{A}(F)$, we define similarly

$k_{f^{+},P,\alpha}(x,y)=\sum_{m^{+}\in
M_{P,\alpha}^{+}(F)}\int_{N_{P}^{+}(\mathbb{A})}f^{+}(m^{+}n^{+}\cdot(x,y))\mathrm{d}n^{+}.$

######  Lemma 9.1.

There are an integer $N$ and a semi-norm $\lVert\cdot\rVert_{\mathcal{S}}$ on
$\mathcal{S}(\operatorname{U}_{V}^{+}(\mathbb{A}))$ such that for all

$\sum_{m\in M_{P}^{+}(F)}\int_{N_{P}^{+}(\mathbb{A})}\left\lvert
f^{+}(mn\cdot(x,y))\right\rvert\mathrm{d}n\leq\lVert
f^{+}\rVert_{\mathcal{S}}\lVert
x\rVert_{\operatorname{U}_{V}^{\prime}}^{N}\lVert
y\rVert_{\operatorname{U}_{V}^{\prime}}^{N}.$

In particular the defining expressions of $k_{f^{+},P,\alpha}$ and
$k_{f^{+},P}$ are absolutely convergent and we have

$\sum_{\alpha\in\mathcal{A}(F)}k_{f^{+},P,\alpha}(x,y)=k_{f^{+},P}(x,y).$

###### Proof.

The proof is the same as Lemma 7.1. ∎

Similar to what we have done in Subsection 7.1, we identify
$\operatorname{Res}_{E/F}\mathbf{A}_{2n,E}$ with the affine space
$\mathbf{A}^{4n}$ over $F$, and denote again by $q_{V}$ the morphism
$\operatorname{U}_{V}^{+}\to\mathbf{A}^{4n}$. We extend the definition of
$k_{f^{+},P,\alpha}$ to all $\alpha\in F^{4n}$ by setting
$k_{f^{+},P,\alpha}=0$ if $\alpha\not\in\mathcal{A}(F)$. There is a $d\in
F^{\times}$ such that if $k_{f^{+},P,\alpha}$ is not identically zero for some
$\alpha\in F^{4n}$ and $P\in\mathcal{F}_{V}$, then
$\alpha\in(d\mathcal{O}_{F})^{4n}$. Put
$\Lambda=(d\mathcal{O}_{F})^{4n}\subset F^{4n}$. For each $\alpha\in\Lambda$,
take $u_{\alpha}\in C_{c}^{\infty}(F^{4n}_{\infty})$ and define the function
$f^{+}_{\alpha}=f^{+}u_{\alpha}$ as in Subsection 4.10 .

######  Lemma 9.2.

We have

$k_{f^{+}_{\alpha},P}(x,y)=k_{f^{+}_{\alpha},P,\alpha}(x,y)=k_{f^{+},P,\alpha}(x,y)$

for all $x,y\in[\operatorname{U}_{V}^{\prime}]_{P^{\prime}}$.

The proof is the same as Lemma 7.2.

We now relate the kernel function $k_{f^{+},P}$ and the kernel function
$K_{f_{+},P}$ defined in Subsection 8.3. We first define a partial Fourier
transform, cf. [Li92]*Section 2. For
$\phi_{1},\phi_{2}\in\mathcal{S}(L^{\vee}(\mathbb{A}))$ we define a partial
Fourier transform

(9.1)
$\mathcal{S}(L^{\vee}(\mathbb{A}))\otimes\mathcal{S}(L^{\vee}(\mathbb{A}))\to\mathcal{S}(V(\mathbb{A})),\quad\phi_{1}\otimes\phi_{2}\mapsto(\phi_{1}\otimes\phi_{2})^{\ddagger},$

by

(9.2)
$(\phi_{1}\otimes\phi_{2})^{{\ddagger}}(v)=\int_{L^{\vee}(\mathbb{A})}\phi_{1}(x+l^{\prime})\phi_{2}(x-l^{\prime})\psi(-2\operatorname{Tr}_{E/F}q_{V}(x,l))\mathrm{d}x,$

where we write $v=l+l^{\prime}$ where $l\in L(\mathbb{A})$ and
$L^{\vee}(\mathbb{A})$. In particular we have

$(\phi_{1}\otimes\phi_{2})^{{\ddagger}}(0)=\langle\phi_{1},\overline{\phi_{2}}\rangle_{L^{2}}$

where $\langle-,-\rangle_{L^{2}}$ stands for the $L^{2}$-inner product on
$L^{\vee}(\mathbb{A})$. We also have

$(\omega^{\vee}(g)\phi_{1}\otimes\omega(g)\phi_{2})^{\ddagger}(v)=(\phi_{1}\otimes\phi_{2})^{\ddagger}(g^{-1}v)$

for $g\in\operatorname{U}(V)(\mathbb{A})$ and $v\in V(\mathbb{A})$.

The partial Fourier transform (9.1) extends to a continuous isomorphism

$\mathcal{S}(\operatorname{U}_{V,+}(\mathbb{A}))\to\mathcal{S}(\operatorname{U}_{V}^{+}(\mathbb{A})),$

which we still denote by $-^{\ddagger}$. Let $f^{+}\mapsto f^{+}_{{\ddagger}}$
be its inverse. Concretely for
$f_{+}\in\mathcal{S}(\operatorname{U}_{V,+}(\mathbb{A}))$, we have

(9.3)
$f_{+}^{\ddagger}((g_{1},g_{2}),v)=\int_{L^{\vee}}\omega^{\vee}_{(1)}(g_{1})f_{+}((g_{1},g_{2}),x+l^{\prime},x-l^{\prime})\psi(-2\operatorname{Tr}_{E/F}q_{V}(x,l))\mathrm{d}x,$

where we write $v=l+l^{\prime}$ where $l\in L(\mathbb{A})$ and
$L^{\vee}(\mathbb{A})$. The notation is interpreted as follows. We first
evaluate $f_{+}$ at $(g_{1},g_{2})$ to obtain a Schwartz function on
$L^{\vee}(\mathbb{A})\times L^{\vee}(\mathbb{A})$. Then the Weil
representation $\omega^{\vee}(g_{1})$ acts on this Schwartz function on the
first variable (this is what the subscript $(1)$ indicates). Finally we take
the partial Fourier transform.

######  Lemma 9.3.

For all $x,y\in[\operatorname{U}_{V}^{\prime}]_{P^{\prime}}$, we have

$K_{f_{+},P}(x,y)=k_{f_{+}^{\dagger},P}(x,y).$

The proof is the same as Lemma 7.3.

Recall that we have defined in Subsection 8.4 the actions $\mathrm{L}_{+}$ and
$\mathrm{R}_{+}$ of $\operatorname{U}_{V}^{\prime}(\mathbb{A})$ on
$\mathcal{S}(\operatorname{U}_{V,+}(\mathbb{A}))$. We also have the left and
right translation of $\operatorname{U}_{V}^{\prime}(\mathbb{A})$ on
$\mathcal{S}(\operatorname{U}_{V}^{+}(\mathbb{A}))$ which we denote by
$\mathrm{L}^{+}$ and $\mathrm{R}^{+}$ respectively. More precisely we have

$\mathrm{L}^{+}(h)f^{+}(g,v)=f^{+}(h^{-1}g,v),\quad\mathrm{R}^{+}(h)f^{+}(g,v)=f^{+}(gh,h^{-1}v),$

for $h\in\operatorname{U}_{V}^{\prime}(\mathbb{A})$.

######  Lemma 9.4.

Let $f^{+}\in\mathcal{S}(G^{+}(\mathbb{A}))$ and
$x,y\in\operatorname{U}_{V}^{\prime}(\mathbb{A})$. We have

$(\mathrm{L}^{+}(x)\mathrm{R}^{+}(y)f^{+})_{{\dagger}}=\mathrm{L}_{+}(x)\mathrm{R}_{+}(y)(f^{+}_{\dagger}).$

###### Proof.

This is a direct computation. ∎

For $T\in\mathfrak{a}_{0}$ and $x,y\in[\operatorname{U}_{V}^{\prime}]$, we
define the modified kernel

$k_{f^{+}}^{T}(x,y)=\sum_{P\in\mathcal{F}_{V}}\epsilon_{P}\sum_{\begin{subarray}{c}\gamma\in
P^{\prime}(F)\backslash\operatorname{U}_{V}^{\prime}(F)\\\ \delta\in
P^{\prime}(F)\backslash\operatorname{U}_{V}^{\prime}(F)\end{subarray}}\widehat{\tau}_{P^{\prime}}(H_{P^{\prime}}(\delta
y)-T_{P^{\prime}})k_{f^{+},P}(\gamma x,\delta y),$

and for $\alpha\in\mathcal{A}(F)$ we put

$k_{f^{+},\alpha}^{T}(x,y)=\sum_{P\in\mathcal{F}_{V}}\epsilon_{P}\sum_{\begin{subarray}{c}\gamma\in
P^{\prime}(F)\backslash\operatorname{U}_{V}^{\prime}(F)\\\ \delta\in
P^{\prime}(F)\backslash\operatorname{U}_{V}^{\prime}(F)\end{subarray}}\widehat{\tau}_{P^{\prime}}(H_{P^{\prime}}(\delta
y)-T_{P^{\prime}})k_{f^{+},\alpha,P}(\gamma x,\delta y).$

######  Lemma 9.5.

For $T$ sufficiently positive, the integral

$\int_{[\operatorname{U}_{V}^{\prime}]\times[\operatorname{U}_{V}^{\prime}]}\left\lvert
k_{f^{+}}^{T}(x,y)\right\rvert\mathrm{d}x\mathrm{d}y$

is absolutely convergent and defines a continuous seminorm on
$\mathcal{S}(\operatorname{U}_{V}^{+}(\mathbb{A}))$. Put

$j^{T}(f^{+}):=\int_{[\operatorname{U}_{V}^{\prime}]\times[\operatorname{U}_{V}^{\prime}]}k_{f^{+}}^{T}(x,y)\mathrm{d}x\mathrm{d}y.$

Then $j^{T}$ is the restriction of an exponential-polynomial function of $T$
whose purely polynomial term is a constant that equals
$J((f^{+})_{\ddagger})$.

###### Proof.

The absolute convergence follows from Lemma 9.3 and Proposition 8.5, the
second follows from Lemma 9.3 and Theorem 8.6. ∎

####  9.2. The coarse geometric expansion

The following theorem gives the geometric expansions of relative trace
formulae on unitary group.

######  Theorem 9.6.

We have the following assertions.

  1. (1)

For $T$ sufficiently positive, the expression

$\sum_{\alpha\in\mathcal{A}(F)}\int_{[\operatorname{U}_{V}^{\prime}]\times[\operatorname{U}_{V}^{\prime}]}\left\lvert
k_{f^{+},\alpha}^{T}(x,y)\right\rvert\mathrm{d}x\mathrm{d}y$

is finite and defines a continuous seminorm on
$\mathcal{S}(\operatorname{U}_{V}^{+}(\mathbb{A}))$.

  2. (2)

For $\alpha\in\mathcal{A}(F)$ and $T$ sufficiently positive, put

$j^{T}_{\alpha}(f^{+})=\int_{[\operatorname{U}_{V}^{\prime}]\times[\operatorname{U}_{V}^{\prime}]}k_{f^{+},\alpha}^{T}(x,y)\mathrm{d}x\mathrm{d}y.$

Then $j^{T}_{\alpha}$ coincides with the restriction of an exponential-
polynomial function of $T$ whose purely polynomial term is a constant
$j_{\alpha}(f^{+})$.

  3. (3)

The linear form $f^{+}\mapsto j(f^{+})$ is continuous and satisfies the
invariant property that

$j(\mathrm{L}^{+}(x)\mathrm{R}^{+}(y)f^{+})=j(f^{+})$

for all $x,y\in\operatorname{U}^{\prime}_{V}(\mathbb{A})$.

The proof is the same as Theorem 7.6 and make use of Proposition 4.30.

####  9.3. Synthesis of the results: the coarse relative trace formula

We now summarize what we have done. For
$f_{+}\in\mathcal{S}(\operatorname{U}_{V,+}(\mathbb{A}))$ we put

$J_{\alpha}^{T}(f_{+})=j_{\alpha}^{T}(f_{+}^{\ddagger}),\quad
J_{\alpha}(f_{+})=j_{\alpha}(f_{+}^{\ddagger}).$

Then Theorem 9.6 tells us that if $T$ is sufficiently positive then
$J_{\alpha}^{T}(f_{+})$ is the restriction of a polynomial exponential and the
purely polynomial part is a constant that equals $J_{\alpha}(f_{+})$. By Lemma
9.4, the distribution $J_{\alpha}$ is
bi-$\operatorname{U}_{V}^{\prime}(\mathbb{A})$-invariant, i.e. for
$x,y\in\operatorname{U}_{V}^{\prime}(\mathbb{A})$ we have

$J_{\alpha}(\mathrm{L}_{+}(x)\mathrm{R}_{+}(y)f_{+})=J_{\alpha}(f_{+}).$

We summarize the coarse relative trace formula on the unitary groups as the
following theorem.

######  Theorem 9.7.

Let $f_{+}\in\mathcal{S}(\operatorname{U}_{V,+}(\mathbb{A}))$ be a test
function. Then we have

$\sum_{\chi\in\mathfrak{X}(\operatorname{U}_{V})}J_{\chi}(f_{+})=\sum_{\alpha\in\mathcal{A}(F)}J_{\alpha}(f_{+}).$

Each summand on both sides are
bi-$\operatorname{U}_{V}^{\prime}(\mathbb{A})$-invariant.

This is simply a combination of Theorem 8.6, Lemma 9.5 and Theorem 9.6.

##  Part III Spectral expansions of automorphic periods

###  10\. Periods and relative characters on general linear groups

####  10.1. Preliminaries

In addition to the groups we encountered in Part II, we will use in this part
the following notations.

  * •

Set $n\geq 1$. Define $G_{n}=\operatorname{Res}_{E/F}\operatorname{GL}_{n}$,
$G=G_{n}\times G_{n}$, $H=G_{n}$ embedded diagonally in $G$,
$G^{\prime}=\operatorname{GL}_{n,F}\times\operatorname{GL}_{n,F}$ seen as a
subgroup of $G$ and $G_{+}=G\times E_{n}$.

  * •

Let $e_{n}=(0,\cdots,0,1)\in E_{n}$ and let $P_{n}$ be the mirabolic subgroup
of $G_{n}$ which consists of matrices whose last row is $e_{n}$.

  * •

Define $B=B_{n}\times B_{n}$, $T=T_{n}\times T_{n}$, $N=N_{n}\times N_{n}$,
and $P_{n,n}=P_{n}\times P_{n}$ seen as subgroups of $G$.

  * •

If $A$ is a subgroup of $G$, set $A_{H}:=A\cap H$.

We defined various spaces of functions in Part II. We will use only use the
following spaces, whose definitions are given in Subsection 4.3.

  * •

The Schwartz spaces $\mathcal{S}(G(\mathbb{A}))$, $\mathcal{S}([G])$, and
$\mathcal{S}(\mathbb{A}_{E})$.

  * •

The space $\mathcal{T}([G])=\bigcup_{N\geq 0}\mathcal{T}_{N}([G])$ of
functions of uniformly moderate growth.

  * •

For $\chi\in\mathfrak{X}(G)$ a cuspidal datum, the spaces
$\mathcal{S}_{\chi}([G])$ and $\mathcal{T}_{\chi}([G])$ (see Subsection 4.7).

In addition to these spaces, at one point in the proof of Theorem 10.4, we are
going to make use of the Harish-Chandra Schwartz space $\mathcal{C}([G])$
defined in [BPCZ]*Section 2.5.8. The precise definition is not needed in our
argument and we will only use the two following facts.

  1. (1)

There is a continuous inclusion $\mathcal{C}([G])\subset\mathcal{T}([G])$
([BPCZ]*Equation (2.4.5.25)).

  2. (2)

For all $f\in\mathcal{C}([G])$, we have
$\int_{[G]}\left|f(g)\right|^{2}\mathrm{d}g<\infty$, and moreover the map
$f\in\mathcal{C}([G\times G])\mapsto\int_{[G]}f(g,g)\mathrm{d}g$ is continuous
([BPCZ]*Equation (2.4.5.24)).

The inclusion $\mathcal{C}([G])\subset\mathcal{T}([G])$ induces by restriction
of (4.6) a pairing $\mathcal{C}([G])\times\mathcal{S}([G])\to\mathbb{C}$. For
any subset $\mathfrak{X}\subset\mathfrak{X}(G)$, define
$\mathcal{C}_{\mathfrak{X}}([G])$ to be the orthogonal of
$\mathcal{S}_{\mathfrak{X}^{c}}([G])$ in $\mathcal{C}([G])$, as in Subsection
4.7.

Denote characters of $[N_{n}]$ by

(10.1)
$\psi_{n,\pm}=\psi^{E}\left(\pm(-1)^{n}\sum_{i=1}^{n-1}u_{i,i+1}\right),\quad
u\in[N_{n}],$

and define a character of $[N]$ by $\psi_{N}=\psi_{n,+}\boxtimes\psi_{n,-}$.
This is a generic character trivial on $[N_{H}]$. The $(-1)^{n}$ factor is for
compatibility reasons in the local comparison of the relative trace formula.

####  10.2. Regularity conditions on cuspidal data for $\operatorname{GL}_{n}$

Let $\Pi$ be an irreducible automorphic representation of $G_{n}(\mathbb{A})$.
Set
$\Pi^{\mathsf{c}}=\Pi\circ\mathsf{c}=\\{\varphi\circ\mathsf{c}\mid\varphi\in\Pi\\}$
and $\Pi^{*}=(\Pi^{\mathsf{c}})^{\vee}$.

Let
$\chi=(\chi_{n},\chi_{n}^{\prime})\in\mathfrak{X}(G)=\mathfrak{X}(\operatorname{GL}_{n})\times\mathfrak{X}(\operatorname{GL}_{n})$
be a cuspidal datum. Take $(M_{n},\pi)$ and $(M^{\prime}_{n},\pi^{\prime})$ to
be representatives of $\chi_{n}$ and $\chi_{n}^{\prime}$ respectively. Write
$M_{n}=\prod\operatorname{GL}_{n_{i}}$,
$M_{n}^{\prime}=\prod\operatorname{GL}_{n_{j}^{\prime}}$ and
$\pi=\boxtimes\pi_{i}$, $\pi^{\prime}=\boxtimes\pi_{j}^{\prime}$ accordingly.
We shall say that

  * •

$\chi_{n}$ (resp. $\chi_{n}^{\prime}$) is regular if all the $\pi_{i}$ are
mutually non-isomorphic (resp. all the $\pi_{j}^{\prime}$ are mutually non-
isomorphic),

  * •

$\chi$ is $G$-regular if $\chi_{n}$ and $\chi_{n}^{\prime}$ are regular,

  * •

$\chi$ is $(H,\mu^{-1})$-regular if $\mu^{-1}\pi_{i}$ is never isomorphic to
the contragredient $(\pi_{j}^{\prime})^{\vee}$ for any $i$ and $j$,

  * •

$\chi$ is $(G,H,\mu^{-1})$-regular if it is $G$ and $(H,\mu^{-1})$-regular.

Let $\chi_{n}\in\mathfrak{X}(G_{n})$ be a cuspidal datum. Assume that it is
represented by $(M_{n},\pi)$ which admits the following decomposition

(10.2) $M_{n}=\prod_{i\in I}G_{n_{i}}^{d_{i}}\times\prod_{j\in
J}G_{n_{j}}^{d_{j}}\times\prod_{k\in
K}G_{n_{k}}^{d_{k}},\quad\pi=\bigotimes_{i\in I}\pi_{i}^{\boxtimes
d_{i}}\boxtimes\bigotimes_{j\in J}\pi_{j}^{\boxtimes
d_{j}}\boxtimes\bigotimes_{k\in K}\pi_{k}^{\boxtimes d_{k}},$

where the $\pi_{l}$’s are distinct irreducible cuspidal automorphic
representations of the $G_{n_{l}}(\mathbb{A})$ such that we have the following
conditions.

  * •

For all $i\in I$, $\pi_{i}\not\cong\pi_{i}^{*}$.

  * •

For all $j\in J$, $\pi_{j}\cong\pi_{j}^{*}$ and
$L(s,\pi_{j},\mathrm{As}^{(-1)^{n+1}})$ has no pole at $s=1$.

  * •

For all $k\in K$, $\pi_{k}\cong\pi_{k}^{*}$ and
$L(s,\pi_{k},\mathrm{As}^{(-1)^{n+1}})$ has a pole at $s=1$.

We say that $\chi_{n}$ is Hermitian if for all $i\in I$ there exists $i^{*}\in
I$ such that $\pi_{i^{*}}\cong(\pi_{i})^{*}$, and if for all $j\in J$, $d_{j}$
is even. Therefore $\chi_{n}$ is Hermitian and regular if in the above
decomposition we have $J=\emptyset$, for all $l\in I\cup K$ the exponent
$d_{l}$ is $1$, and if for all $i\in I$ the index $i^{*}$ is uniquely defined
and the map $i\mapsto i^{*}$ is an involution of $I$ with no fixed point. In
this case, let $I^{\prime}\subset I$ be a subset such that
$I=I^{\prime}\sqcup(I^{\prime})^{*}$ and define

(10.3) $L_{\pi}=\prod_{i\in I^{\prime}}G_{2n_{i}}\times\prod_{k\in
K}G_{n_{k}}.$

This is a Levi subgroup of $G_{n}$ containing $M_{n}$.

For $\chi\in\mathfrak{X}(G)$ with decomposition
$\chi=(\chi_{n},\chi_{n}^{\prime})$, we shall say that $\chi$ is Hermitian if
$\chi_{n}$ and $\chi_{n}^{\prime}$ are Hermitian. In this case, we take
$(M_{n},\pi)$ and $(M^{\prime}_{n},\pi^{\prime})$ to be representatives of
$\chi_{n}$ and $\chi_{n}^{\prime}$ respectively, and define
$L_{\pi\boxtimes\pi^{\prime}}=L_{\pi}\times L_{\pi^{\prime}}$ which is a Levi
subgroup of $G$.

######  Remark 10.1.

We have defined in Section 1.1 the notion of regular Hermitian Arthur
parameter, which is an irreducible automorphic representation
$\Pi=\Pi_{n}\boxtimes\Pi_{n}^{\prime}$ of $G(\mathbb{A})$. If
$\Pi=I_{P}^{G}\pi$ is a $(G,H,\mu^{-1})$-regular Hermitian Arthur parameter,
then $(M_{P},\pi)$ represents a $(G,H,\mu^{-1})$-regular Hermitian cuspidal
datum $\chi$ of $G$. The requirement that $\Pi_{n}$ is discrete corresponds to
$I=\emptyset$ in (10.2). Therefore, if $\Pi$ is a semi-discrete Hermitian
Arthur parameter, then $\Pi$ and $\chi$ are automatically
$(G,H,\mu^{-1})$-regular, as otherwise some
$L(s,\pi_{k},\operatorname{As}^{(-1)^{l}})$ would have a pole at $s=1$ for
$l=n,n+1$ which is not possible.

####  10.3. Rankin–Selberg periods

For $f\in\mathcal{T}([G])$ set

(10.4) $W_{f}(g):=\int_{[N]}f(ng)\psi_{N}^{-1}(n)\mathrm{d}n,\;g\in
G(\mathbb{A}).$

Recall that we have defined in Subsection 5.2 a representation
$\mathrm{R}_{\mu^{-1}}$ of $G_{n}(\mathbb{A})$ realized on
$\mathcal{S}(\mathbb{A}_{E,n})$ and given by

$(\mathrm{R}_{\mu^{-1}}(h)\Phi)(x)=\left|\det
h\right|_{E}^{\frac{1}{2}}\mu^{-1}(\det h)\Phi(xh),\quad h\in G_{n},\quad x\in
E_{n}.$

Then for $f\in\mathcal{T}([G])$ and $\Phi\in\mathcal{S}(\mathbb{A}_{E,n})$, we
set

(10.5) $Z^{\mathrm{RS}}_{\psi}(f,\Phi,\mu,s)=\int_{N_{H}(\mathbb{A})\backslash
H(\mathbb{A})}W_{f}(h)\lvert\det
h\rvert_{E}^{s}(\mathrm{R}_{\mu^{-1}}(h)\Phi)(e_{n})\mathrm{d}h,$

for every $s\in\mathbb{C}$ such that this expression converges absolutely.
Beware that $\det h$ is the determinant of $h$ seen as an element of
$G_{n}(\mathbb{A})$ and not $G(\mathbb{A})$.

If $c$ is a real number, set
$\mathcal{H}_{>c}:=\\{s\in\mathbb{C}\;|\;\Re(s)>c\\}$, and if $c<C$, set
$\mathcal{H}_{]c,C[}:=\\{s\in\mathbb{C}\;|\;c<\Re(s)<C\\}$.

######  Lemma 10.2.

Let $N\geq 0$. There exists $c_{N}>0$ such that the following holds.

  * •

For every $f\in\mathcal{T}_{N}([G])$, $\Phi\in\mathcal{S}(\mathbb{A}_{E,n})$
and $s\in\mathcal{H}_{>c_{N}}$, the defining integral of
$Z^{\mathrm{RS}}_{\psi}(f,\Phi,\mu,s)$ converges absolutely.

  * •

For every $s\in\mathcal{H}_{>c_{N}}$, the bilinear form
$(f,\Phi)\in\mathcal{T}_{N}([G])\times\mathcal{S}(\mathbb{A}_{E,n})\mapsto
Z^{\mathrm{RS}}_{\psi}(f,\Phi,\mu,s)$ is separately continuous.

  * •

For every $f\in\mathcal{T}_{N}([G])$ and
$\Phi\in\mathcal{S}(\mathbb{A}_{E,n})$, the function
$s\in\mathcal{H}_{>c_{N}}\mapsto Z^{\mathrm{RS}}_{\psi}(f,\Phi,\mu,s)$ is
holomorphic and bounded in vertical strips.

###### Proof.

The proof follows the same line as that of [BPCZ]*Lemma 7.2.0.1. Denote by
$V_{B}$ the $F$-vector space of additive characters $N\to\mathbb{G}_{a}$, and
fix a height function $\lVert\cdot\rVert_{V_{B}}$ on $V_{B}(\mathbb{A})$ as in
(4.3). The adjoint action of $T$ on $N$ induces a dual action of $T$ on
$V_{B}$, which we denote by $\operatorname{Ad}^{*}$. We also fix a height
function $\lVert\cdot\rVert_{\mathbb{A}_{E}}$ on $\mathbb{A}_{E}$.

Define an additive character $l\in V_{B}$ by

$l(u,u^{\prime})=\operatorname{Tr}_{E/F}\left((-1)^{n}\left(\tau\sum_{i=1}^{n-1}u_{i,i+1}-\tau\sum_{i=1}^{n-1}u^{\prime}_{i,i+1}\right)\right),\quad(u,u^{\prime})\in
N.$

Then $\psi_{N}=\psi\circ l$. It is readily checked that there exists $N_{0}>0$
such that

(10.6) $\prod_{i=1}^{n-1}\lVert
t_{i}t_{i+1}^{-1}\rVert_{\mathbb{A}_{E}}\ll\lVert\mathrm{Ad}^{*}(t^{-1})l\rVert^{N_{0}}_{V_{B}}\quad
t\in T_{n}(\mathbb{A}).$

By [BPCZ]*Lemma 2.6.1.1.2, for any $N_{1}>0$, there is a continuous seminorm
$\lVert\cdot\rVert_{N}$ on $\mathcal{T}_{N}([G])$ such that

$\lvert
W_{f}(t)\rvert\leq\lVert\operatorname{Ad}^{*}(t^{-1})l\rVert_{V_{P}}^{-N_{1}}\lVert
t\rVert_{T}^{N}\lVert f\rVert_{N},$

for all $t\in T(\mathbb{A})$ and $f\in\mathcal{T}_{N}([G])$. As $\Phi$ is
Schwartz, for all $N_{2}>0$ there is a seminorm $\lVert\cdot\rVert$ such that

$\lvert\Phi(e_{n}t)\rvert\leq\lVert
t_{n}\rVert^{-N_{2}}_{\mathbb{A}_{E}}\lVert\Phi\rVert,$

for all $t\in T_{n}(\mathbb{A})$.

By the Iwasawa decomposition for $H$ and the estimates above, we are reduced
to showing the existence of $c_{N}>0$ such that for every $C>c_{N}$ there
exists $N^{\prime}>0$ such that

$\int_{T_{n}(\mathbb{A})}\prod_{i=1}^{n}\lVert
t_{i}\rVert^{-N^{\prime}}_{\mathbb{A}_{E}}\lVert
t\rVert_{T_{n}}^{N}\delta_{B_{n}}(t)^{-1}\lvert\det
t\rvert^{s}_{E}\mathrm{d}t$

converges uniformly on vertical strips for $s\in\mathcal{H}_{]c_{n},C[}$. This
is an elementary calculation. ∎

For every $f\in\mathcal{S}([G])$ and $\Phi\in\mathcal{S}(\mathbb{A}_{E,n})$,
we define

$\Theta^{\prime}(h,\Phi)=\sum_{x\in
E_{n}\backslash\\{0\\}}(\mathrm{R}_{\mu^{-1}}(h)\Phi)(x),\quad h\in[G_{n}],$

and

(10.7)
$Z^{\mathrm{RS}}_{n}(f,\Phi,\mu,s)=\int_{[H]}f(h)\Theta^{\prime}(h,\Phi)\lvert\det
h\rvert^{s}\mathrm{d}h.$

Since $f\in\mathcal{S}([G])$ and
$\left|\Theta^{\prime}(h,\Phi)\right|\ll\lVert h\rVert_{H}^{D}$ for some $D$
(cf. Lemma 5.5), we conclude by [BP]*Proposition A.1.1 (vi) that the integral
converges absolutely for all $s\in\mathbb{C}$ and yields an entire function
bounded on vertical strips.

######  Remark 10.3.

Our definition of $\Theta^{\prime}(.,\Phi)$ mirrors the Epstein–Eisenstein
series of [JS]*Section 4. It differs from the theta function $\Theta(.,\Phi)$
defined in Subsection 5.2 by the term $\mu(\det g)^{-1}\lvert\det
g\rvert^{\frac{1}{2}}\Phi(0)$. When $f\in\mathcal{S}_{\chi}([G])$ with $\chi$
a $(H,\mu^{-1})$-regular cuspidal data, we have for all $s\in\mathbb{C}$

$\int_{[H]}f(h)\mu(\det h)^{-1}\lvert\det
h\rvert^{s+\frac{1}{2}}\mathrm{d}h=0$

It follows that for such an $f$ we have

$Z_{n}^{\mathrm{RS}}(f,\Phi,\mu,s)=\int_{[H]}f(h)\Theta(h,\Phi)|\det
h|^{s}dh.$

######  Theorem 10.4.

Let $\chi\in\mathfrak{X}(G)$ be a $(G,H,\mu^{-1})$-regular cuspidal datum.

  1. (1)

For every $f\in\mathcal{T}_{\chi}([G])$ and
$\Phi\in\mathcal{S}(\mathbb{A}_{E,n})$, the function $s\mapsto
Z^{\mathrm{RS}}_{\psi}(f,\Phi,\mu,s)$, a priori defined on some right half-
plane, extends to an entire function on $\mathbb{C}$ of finite order in
vertical strips.

  2. (2)

For all $s\in\mathbb{C}$ and $\Phi\in\mathcal{S}(\mathbb{A}_{E,n})$, the
restriction to $\mathcal{S}_{\chi}([G])$ of

$f\in\mathcal{S}([G])\mapsto Z^{\mathrm{RS}}_{n}(f,\Phi,\mu,s),$

extends continuously to a linear form on $\mathcal{T}_{\chi}([G])$.

Moreover, if we keep the same notations for these continuations, then for all
$f\in\mathcal{T}_{\chi}([G])$, $\Phi\in\mathcal{S}(\mathbb{A}_{E,n})$ and
$s\in\mathbb{C}$ we have

(10.8)
$Z_{\psi}^{\mathrm{RS}}(f,\Phi,\mu,s)=Z_{n}^{\mathrm{RS}}(f,\Phi,\mu,s).$

###### Proof.

We first claim that if $f\in\mathcal{S}_{\chi}([G])$,
$\Phi\in\mathcal{S}(\mathbb{A}_{E,n})$ and $\Re s\gg 0$, then the identity
(10.8) holds. Assuming this for the moment, we fix a $\Phi$ and consider
$Z_{n}^{\mathrm{RS}}$ and $Z_{\psi}^{\mathrm{RS}}$ as (families of) linear
forms in $f$.

We temporarily set $w=(w_{n},w_{n})\in G(F)$, where $w_{n}$ the longest Weyl
group element in $G_{n}(F)$. For $f\in\mathcal{T}([G])$, set
$\widetilde{f}(g)=f(\prescript{t}{}{g}^{-1})=f(w\prescript{t}{}{g}^{-1})\in\mathcal{T}([G])$.

For $s\in\mathbb{C}$, we consider the linear form on $\mathcal{T}([G])$ given
by

(10.9)
$f\mapsto\widetilde{Z}^{\mathrm{RS}}_{\psi}(f,\Phi,\mu,s):=Z_{\psi}^{\mathrm{RS}}(\widetilde{f},\widehat{\Phi},\mu^{-1},s),$

where we denote by $\widehat{\Phi}$ the Fourier transform of $\Phi$ defined as

$\widehat{\Phi}(y)=\int_{\mathbb{A}_{E,n}}\Phi(x)\psi_{E}(x\prescript{t}{}{y})\mathrm{d}x.$

Note that if $f\in\mathcal{T}_{\chi}([G])$ then so does $\widetilde{f}$. By
Lemma 10.2 we see that $Z^{\mathrm{RS}}_{\psi}$ and
$\widetilde{Z}^{\mathrm{RS}}_{\psi}$ enjoy the following properties.

  1. (1)

For all $N>0$ there exists $C_{N}>0$ such that for all
$s\in\mathcal{H}_{>C_{N}}$, $Z^{\mathrm{RS}}_{\psi}(\cdot,\Phi,\mu,s)$ and
$\widetilde{Z}^{\mathrm{RS}}_{\psi}(\cdot,\Phi,\mu,s)$ are continuous linear
functionals on $\mathcal{T}_{N,\chi}([G])$.

  2. (2)

For all $N>0$ and $f\in\mathcal{T}_{N,\chi}([G])$, $s\mapsto
Z^{\mathrm{RS}}_{\psi}(f,\Phi,\mu,s)$ and
$s\mapsto\widetilde{Z}^{\mathrm{RS}}_{\psi}(f,\Phi,\mu,s)$ are holomorphic
functions on $\mathcal{H}_{>C_{N}}$, bounded on vertical strips.

  3. (3)

By the claim that (10.8) holds for $f\in S_{\chi}([G])$ and $\Re s\gg 0$, we
conclude that for $f\in S_{\chi}([G])$ the functions $s\mapsto
Z^{\mathrm{RS}}_{\psi}(f,\Phi,\mu,s)$ and
$s\mapsto\widetilde{Z}^{\mathrm{RS}}_{\psi}(f,\Phi,\mu,s)$ have analytic
continuations to all $s\in\mathbb{C}$.

  4. (4)

By remark 10.3 we may replace $\Theta^{\prime}$ by $\Theta$ and use the
Poisson summation formula to obtain for any $f\in\mathcal{S}_{\chi}([G])$ the
functional equation

$Z_{n}^{\mathrm{RS}}(f,\Phi,\mu,s)=Z_{n}^{\mathrm{RS}}(\widetilde{f},\widehat{\Phi},\mu^{-1},-s).$

Therefore it follows from (10.8) that

(10.10)
$Z^{\mathrm{RS}}_{\psi}(f,\Phi,\mu,s)=\widetilde{Z}^{\mathrm{RS}}_{\psi}(f,\Phi,\mu,-s).$

Let $N^{\prime}>0$ and choose $N>0$ such that we have a continuous inclusion
$\mathrm{L}^{2}_{-N^{\prime}}([G])^{\infty}\subset\mathcal{T}_{N}([G])$ (see
Subsection 4.3.6). By Proposition 4.20, the space $\mathcal{S}_{\chi}([G])$ is
dense in $\mathrm{L}^{2}_{-N^{\prime},\chi}([G])^{\infty}$. It follows that
the two functionals $f\mapsto Z_{\psi}^{\mathrm{RS}}(f,\Phi,\mu,s)$ and
$f\mapsto\widetilde{Z}_{\psi}^{\mathrm{RS}}(f,\Phi,\mu,s)$ defined on the LF-
space $S_{\chi}([G])$ satisfy the hypotheses of [BPCZ]*Corollary A.0.11.2. As
$\mathcal{T}_{\chi}([G])=\bigcup_{N^{\prime}>0}\mathrm{L}^{2}_{-N^{\prime},\chi}([G])^{\infty}$,
this corollary shows the analytic continuation of $Z_{\psi}^{\mathrm{RS}}$ to
$\mathbb{C}$ for all $f\in\mathcal{T}_{\chi}([G])$, and the continuous
extension of $Z_{n}^{\mathrm{RS}}$ to $\mathcal{T}_{\chi}([G])$. Since the
equality (10.8) is true for $f\in\mathcal{S}_{\chi}([G])$ and $\Re s\gg 0$, it
is true for $f\in\mathcal{T}_{\chi}([G])$ by density, and finally for all $s$
by analytic continuation. This concludes the proof.

It remains to prove our claim that the identity (10.8) holds for all
$f\in\mathcal{S}_{\chi}([G])$, $\Phi\in\mathcal{S}(\mathbb{A}_{E,n})$ and $\Re
s\gg 0$. The proof is a variant of the unfolding calculation in the classical
Rankin–Selberg convolution.

Let us introduce the following notation. Let $1\leq r\leq n$ and denote by
$N_{r,n}$ the unipotent radical of the standard parabolic subgroup of $G_{n}$
with Levi component $(G_{r})\times(G_{1})^{n-r}$. Set
$N_{r}^{G}:=N_{r,n}\times N_{r,n}$ and $N_{r}^{H}:=N_{r}^{G}\cap H$. Let
$P_{r}$ be the mirabolic subgroup of $G_{r}$ whose last row is $e_{r}$.

For $f\in\mathcal{S}([G])$ we define

$f_{N_{r}^{G},\psi}(g)=\int_{[N_{r}^{G}]}f(ug)\psi_{N}(u)^{-1}du,\quad g\in
G(\mathbb{A}).$

For $s\in\mathbb{C}$ and $\Phi\in\mathcal{S}(\mathbb{A}_{E,n})$ we set

$Z^{\mathrm{RS}}_{r}(f,\Phi,\mu,s)=\int_{P_{r}(F)N_{r}^{H}(\mathbb{A})\backslash
H(\mathbb{A})}f_{N_{r}^{G},\psi}(h)(\mathrm{R}_{\mu^{-1}}(h)\Phi)(e_{n})|\det
h|_{E}^{s}\mathrm{d}h.$

For $r=1$ we get
$Z^{\mathrm{RS}}_{1}(f,\Phi,\mu,s)=Z_{\psi}^{\mathrm{RS}}(f,\Phi,\mu,s)$. The
same argument as in the proof of Lemma 10.2 shows that for every $1\leq r\leq
n$ there exists $c_{r}>0$ such that the following assertions hold.

  * •

For all $f\in\mathcal{S}([G])$ and $\Phi\in\mathcal{S}(\mathbb{A}_{E,n})$, the
expression defining $Z^{\mathrm{RS}}_{r}(f,\Phi,\mu,s)$ converges absolutely
for $s\in\mathcal{H}_{>c_{r}}$.

  * •

For all $s\in\mathcal{H}_{>c_{r}}$, $(f,\Phi)\mapsto
Z^{\mathrm{RS}}_{r}(f,\Phi,\mu,s)$ is separately continuous.

  * •

For every $f\in\mathcal{S}([G])$ and $\Phi\in\mathcal{S}(\mathbb{A}_{E,n})$,
the function $s\in\mathcal{H}_{>c_{r}}\mapsto
Z^{\mathrm{RS}}_{r}(f,\Phi,\mu,s)$ is holomorphic and bounded in vertical
strips.

The desired identity (10.8) then follows from the following claim: for all
$1\leq r\leq n-1$, for all $f\in\mathcal{S}_{\chi}([G])$ and
$\Phi\in\mathcal{S}(\mathbb{A}_{E,n})$ there exists $c>0$ such that for all
$s\in\mathcal{H}_{>c}$ we have

(10.11)
$Z^{\mathrm{RS}}_{r+1}(f,\Phi,\mu,s)=Z^{\mathrm{RS}}_{r}(f,\Phi,\mu,s).$

We now prove identity (10.11). First for $r$, $f$ and $\Phi$, we pick $c>0$
such that $Z^{\mathrm{RS}}_{r+1}(f,\Phi,\mu,s)$ and
$Z^{\mathrm{RS}}_{r}(f,\Phi,\mu,s)$ are well-defined for
$s\in\mathcal{H}_{>c}$. Let $U_{r+1}$ be the unipotent radical of $P_{r}$.
Then $N_{r,n}=U_{r+1}N_{r+1,n}$. Set $U_{r+1}^{G}=U_{r+1}\times U_{r+1}\leq G$
and $U_{r+1}^{H}=U_{r+1}\leq H$. We then compute

$\displaystyle Z^{\mathrm{RS}}_{r+1}(f,\Phi,\mu,s)$
$\displaystyle=\int_{P_{r+1}(F)N_{r+1}^{H}(\mathbb{A})\backslash
H(\mathbb{A})}f_{N_{r+1}^{G},\psi}(h)(\mathrm{R}_{\mu^{-1}}(h)\Phi)(e_{n})|\det
h|_{E}^{s}\mathrm{d}h$
$\displaystyle=\int_{G_{r}(F)N_{r}^{H}(\mathbb{A})\backslash
H(\mathbb{A})}\int_{[U_{r+1}^{H}]}f_{N_{r+1}^{G},\psi}(uh)(\mathrm{R}_{\mu^{-1}}(uh)\Phi)(e_{n})|\det
uh|_{E}^{s}\mathrm{d}u\mathrm{d}h$
$\displaystyle=\int_{G_{r}(F)N_{r}^{H}(\mathbb{A})\backslash
H(\mathbb{A})}(\mathrm{R}_{\mu^{-1}}(h)\Phi)(e_{n})|\det
h|_{E}^{s}\int_{[U_{r+1}^{H}]}f_{N_{r+1}^{G},\psi}(uh)\mathrm{d}u\mathrm{d}h.$

By Fourier inversion on the compact abelian group
$U_{r+1}^{G}(F)U^{H}_{r+1}(\mathbb{A})\backslash U^{G}_{r+1}(\mathbb{A})$ we
get

$\displaystyle\int_{[U_{r+1}^{H}]}f_{N_{r+1}^{G},\psi}(uh)\mathrm{d}u$
$\displaystyle=\sum_{\gamma\in P_{r}(F)\backslash
G_{r}(F)}\int_{[U_{r+1}^{G}]}f_{N_{r+1}^{G},\psi}(u\gamma
h)\psi_{N}(u)^{-1}\mathrm{d}u+\int_{[U_{r+1}^{G}]}f_{N_{r+1}^{G},\psi}(uh)\mathrm{d}u$
$\displaystyle=\sum_{\gamma\in P_{r}(F)\backslash
G_{r}(F)}f_{N_{r}^{G},\psi}(\gamma
h)+\int_{[U_{r+1}^{G}]}f_{N_{r+1}^{G},\psi}(uh)\mathrm{d}u.$

Therefore we have

$Z^{\mathrm{RS}}_{r+1}(f,\Phi,\mu,s)=Z^{\mathrm{RS}}_{r}(f,\Phi,\mu,s)+F_{r}(s),$

where

$F_{r}(s)=\int_{G_{r}(F)N_{r}^{H}(\mathbb{A})\backslash
H(\mathbb{A})}(\mathrm{R}_{\mu^{-1}}(h)\Phi)(e_{n})|\det
h|_{E}^{s}\int_{[U_{r+1}^{G}]}f_{N^{G}_{r+1},\psi}(uh)\mathrm{d}u\mathrm{d}h.$

As our functions are holomorphic for $\Re(s)\gg 0$, it remains to show that
$F_{r}$ vanishes on some right half-plane of $\mathbb{C}$. Let $Q_{r}$ be the
standard parabolic subgroup of $G$ with Levi component $L_{r}=(G_{r}\times
G_{n-r})\times(G_{r}\times G_{n-r})$, and set $Q_{r}^{H}=Q_{r}\cap H$. Let
$K_{n}$ be a good maximal compact subgroup of $G_{n}(\mathbb{A})$ as in
Subsection 4.5. Let $\chi^{L}$ be the inverse image of $\chi$ in
$\mathfrak{X}(L_{r})$. We write any element in $\mathfrak{X}(L_{r})$ as
$(\chi_{1},\chi_{2})$ where $\chi_{1}\in\mathfrak{X}(G_{r}^{2})$ and
$\chi_{2}\in\mathfrak{X}(G_{n-r}^{2})$. Then by [BPCZ]*Section 2.9.6.10 we
have the decomposition

(10.12)
$\mathcal{C}_{\chi^{L}}([L_{r}])=\bigoplus_{(\chi_{1},\chi_{2})\in\chi^{L}}\mathcal{C}_{\chi_{1}}([G_{r}^{2}])\operatorname{\widehat{\otimes}}\mathcal{C}_{\chi_{2}}([G_{n-r}^{2}]).$

Define for all $s\in\mathbb{C}$ and $k\in K_{n}$

$f_{Q_{r},k,s}=\delta_{Q_{r}}^{-\frac{1}{2}+\frac{2s+1}{4n-4r}}\mathrm{R}(k)f_{Q_{r}}\Big{|}_{[L_{r}]},$

where $f_{Q_{r}}$ is the constant term of $f$ with respect to $Q_{r}$. As
$\chi$ is $G$-regular, [BPCZ]*Corollary 2.9.7.2 applies and for $\Re s>0$ and
$k\in K_{n}$ we have
$(f_{Q_{r},k,s})\Big{|}_{[L_{r}]}\in\mathcal{C}_{\chi^{L}}([L_{r}])$. Finally,
for $k\in K_{n}$, set
$\Phi_{k,n-r}=(\mathrm{R}(k)\Phi)\Big{|}_{\\{0\\}\times\mathbb{A}_{E,{n-r}}}\in\mathcal{S}(\mathbb{A}_{E,n-r})$.
Then using the Iwasawa decomposition
$H(\mathbb{A})=Q_{r}^{H}(\mathbb{A})K_{n}$ we get just as in the proof of
[BPCZ]*Proposition 7.2.0.2:

$F_{r}(s)=\int_{K_{n}}\left(P_{G_{r}^{\Delta}}^{\mu^{-1}}\operatorname{\widehat{\otimes}}\mathcal{Z}_{n-r}^{\mathrm{RS}}\left(\cdot,\Phi_{k,n-r},\mu,s+2r\frac{2s+1}{4n-4r}\right)\right)(f_{Q_{r},k,s})dk,$

where

  * •

$P_{G_{r}^{\Delta}}^{\mu^{-1}}$ stands for the period integral over the
diagonal subgroup $G_{r}$ of $G_{r}^{2}$, against the character $\mu^{-1}$,
which defines a continuous linear form on $\mathcal{C}([G_{r}^{2}])$
([BPCZ]*Equation 2.4.5.24),

  * •

$\mathcal{Z}_{n-r}^{\mathrm{RS}}$ is $Z_{\psi^{r}}^{\mathrm{RS}}$ for
$G_{n-r}$, with $\psi^{r}=\psi((-1)^{r}.)$. By Lemma 10.2, for every
$\Phi^{\prime}\in\mathcal{S}(\mathbb{A}_{E,n-r})$ it is a continuous linear
form on $\mathcal{T}([G_{n-r}^{2}])$ provided that $s$ lies in some right
half-plane of $\mathbb{C}$ (that can be chosen independently of
$\Phi^{\prime}$),

  * •

the completed tensor product is taken relatively to the decomposition (10.12),
recalling the continuous inclusion
$\mathcal{C}([G^{2}_{n-r}])\subset\mathcal{T}([G^{2}_{n-r}])$.

Since $\chi$ is $(H,\mu^{-1})$-regular, for every preimage
$(\chi_{1},\chi_{2})\in\mathfrak{X}(G_{r}^{2})\times\mathfrak{X}(G_{n-r}^{2})$
with $\chi_{1}=(\chi_{1}^{\prime},\chi_{1}^{\prime\prime})$ we have
$\mu^{-1}\chi_{1}^{\prime}\neq\chi_{1}^{\prime\prime\vee}$. Thus,
$P_{G_{r}^{\Delta}}^{\mu^{-1}}$ vanishes identically on
$\mathcal{C}_{\chi_{1}}([G_{r}^{2}])$, which implies that $F_{r}(s)=0$
whenever $\Re(s)\gg 0$. This proves the identity (10.11) and hence finishes
the proof of the theorem. ∎

The linear form $Z_{\psi}^{\mathrm{RS}}$ given by Theorem 10.4 depends on a
fixed Schwartz function $\Phi$, but it is possible to obtain uniform bounds
when varying $\Phi$ as asserted by the following lemma.

######  Lemma 10.5.

Let $N>0$. There exist continuous semi-norms $\lVert.\rVert_{N}$ and
$\lVert.\rVert$ on $\mathcal{T}_{N,\chi}([G])$ and
$\mathcal{S}(\mathbb{A}_{E,n})$ respectively such that

$\left|Z^{\mathrm{RS}}_{\psi}(f,\Phi,\mu,0)\right|\ll\lVert
f\rVert_{N}\lVert\Phi\rVert,\;f\in\mathcal{T}_{N,\chi}([G]),\;\Phi\in\mathcal{S}(\mathbb{A}_{E,n}).$

###### Proof.

Take $c_{N}$ given by Lemma 10.2, and let $D>c_{N}$. Recall that
$\widetilde{Z}^{\mathrm{RS}}_{\psi}$ is defined in (10.9). By Theorem 10.4,
the functions $s\mapsto e^{s^{2}}Z^{\mathrm{RS}}_{\psi}(f,\Phi,\mu,s)$ and
$s\mapsto e^{s^{2}}\widetilde{Z}^{\mathrm{RS}}_{\psi}(f,\Phi,\mu,s)$ are
entire and of rapid decay in vertical strips, and moreover by the proof of
Lemma 10.2 there exist continuous semi-norms $\lVert.\rVert_{N}$ and
$\lVert.\rVert$ on $\mathcal{T}_{N,\chi}([G])$ and
$\mathcal{S}(\mathbb{A}_{E,n})$ respectively such that

(10.13)
$\left\\{\begin{array}[]{l}|Z^{\mathrm{RS}}_{\psi}(f,\Phi,\mu,s)|\leq\lVert
f\rVert_{N}\lVert\Phi\rVert,\\\
|\widetilde{Z}^{\mathrm{RS}}_{\psi}(f,\Phi,\mu,s)|\leq\lVert
f\rVert_{N}\lVert\Phi\rVert,\end{array}\right.\;s=D+it,\;t\in\mathbb{R},\;f\in\mathcal{T}_{N,\chi}([G]),\;\Phi\in\mathcal{S}(\mathbb{A}_{E,n}).$

For every $f$ and $\Phi$, consider the function

$\mathcal{Z}(f,\Phi,\mu,s)=\frac{e^{-s^{2}}}{2\pi}\left(\int_{-\infty}^{\infty}\frac{e^{(D+it)^{2}}Z^{\mathrm{RS}}_{\psi}(f,\Phi,\mu,D+it)}{D+it-s}\mathrm{d}t+\int_{-\infty}^{\infty}\frac{e^{(D+it)^{2}}\widetilde{Z}^{\mathrm{RS}}_{\psi}(f,\Phi,\mu,D+it)}{D+it+s}\mathrm{d}t\right).$

$\mathcal{Z}$ is well defined and holomorphic on $\mathcal{H}_{]-D,D[}$.
Moreover, by (10.13) we have

(10.14) $\left|\mathcal{Z}(f,\Phi,\mu,0)\right|\ll\lVert
f\rVert_{N}\lVert\Phi\rVert,\;f\in\mathcal{T}_{N,\chi}([G]),\;\Phi\in\mathcal{S}(\mathbb{A}_{E,n}).$

By the functional equation (10.10) and Cauchy’s integration formula, we see
that $\mathcal{Z}(f,\Phi,\mu,s)$ and $Z^{\mathrm{RS}}_{\psi}(f,\Phi,\mu,s)$
coincide on $\mathcal{H}_{]-D,D[}$. Therefore, (10.14) concludes the proof of
the lemma. ∎

####  10.4. Flicker–Rallis periods

Let $\chi\in\mathfrak{X}(G)$ be a $(G,H,\mu^{-1})$-regular cuspidal datum,
represented by $(M_{P},\pi)$. Put $\Pi=I_{P(\mathbb{A})}^{G(\mathbb{A})}\pi$
and for all $\lambda\in i\mathfrak{a}_{P}^{G,*}$ set
$\Pi_{\lambda}=I_{P(\mathbb{A})}^{G(\mathbb{A})}\pi_{\lambda}$. Assume that
all the $\Pi_{\lambda}$ are generic and let
$\mathcal{W}(\Pi_{\lambda},\psi_{N})$ be the corresponding Whittaker models
for $\psi_{N}$, cf. (10.1). For $\varphi\in\Pi$, $g\in G(\mathbb{A})$ and
$\lambda\in\mathfrak{a}_{P,\mathbb{C}}^{*}$ the Eisenstein series
$E(g,\varphi,\lambda)$ is absolutely convergent for $\Re\lambda$ in a suitable
cone, and admits a meromorphic continuation to
$\mathfrak{a}_{P,\mathbb{C}}^{*}$ which is regular on $i\mathfrak{a}_{P}^{*}$.
By [Lap]*Theorem 2.2, for every $\lambda\in i\mathfrak{a}_{P}^{*}$ the map
$\varphi\mapsto E(\cdot,\varphi,\lambda)$ induces a continuous map
$\Pi\to\mathcal{T}_{N}([G])$ for some $N>0$. If $\varphi\in\Pi$ and
$\lambda\in i\mathfrak{a}_{P}^{*}$, set

$W(g,\varphi,\lambda)=\int_{[N]}E(ng,\varphi,\lambda)\psi_{N}^{-1}(n)\mathrm{d}n,\quad
g\in G(\mathbb{A}).$

Then $W(\cdot,\varphi,\lambda)\in\mathcal{W}(\Pi_{\lambda},\psi_{N})$.

If $f\in\mathcal{S}([G])$ and $\lambda\in i\mathfrak{a}_{P}^{*}$ we define as
in (4.13)

(10.15) $f_{\Pi_{\lambda}}=\sum_{\varphi\in\mathcal{B}_{P,\pi}}\langle
f,E(\cdot,\varphi,\lambda)\rangle E(\cdot,\varphi,\lambda).$

Here $\mathcal{B}_{P,\pi}$ is a $K$-basis of
$\mathcal{A}_{P,\pi,\mathrm{cusp}}(G)$, cf. Subsection 4.6, and
$\langle\cdot,\cdot\rangle$ is the natural pairing between $\mathcal{S}([G])$
and $\mathcal{T}([G])$ defined in (4.6). By Proposition 4.18 (3), there exists
an $N>0$ such that this series converges absolutely in $\mathcal{T}_{N}([G])$.
We define a function

(10.16)
$W_{f,\Pi_{\lambda}}(g)=W_{f_{\Pi_{\lambda}}}(g)=\int_{[N]}f_{\Pi_{\lambda}}(ng)\psi_{N}(n)\mathrm{d}n,\quad
g\in G(\mathbb{A}),$

which is the same expression as in (10.4) but with $\psi_{N}$ for
compatibility reasons. Note that
$W_{f_{\Pi_{\lambda}}}\in\mathcal{W}(\Pi_{\lambda},\psi_{N}^{-1})$.

If $f\in\mathcal{S}(G(\mathbb{A}))$, we denote by $I_{P}(\lambda,f)$ the
action of $f$ on $\Pi$ we get by transport from the action of
$\mathcal{S}(G(\mathbb{A}))$ on $\Pi_{\lambda}$ via $\varphi\in\Pi\mapsto
e^{\langle\lambda,H_{P}(\cdot)\rangle}\varphi\in\Pi_{\lambda}$.

If $f\in\mathcal{S}([G])$, set

(10.17)
$P_{G^{\prime},\eta}(f)=\int_{[G^{\prime}]}f(g^{\prime})\eta_{n+1}(g^{\prime})\mathrm{d}g^{\prime},$

where $\eta_{n+1}(g_{1},g_{2})=\eta(\det g_{1}g_{2})^{n+1}$. The integral is
absolutely convergent and defines a continuous functional on
$\mathcal{S}([G])$ by [BPCZ]*Theorem 6.2.6.1. Let $P_{n,n}=P_{n}\times P_{n}$
be the product of mirabolic groups, $P_{n,n}^{\prime}=P_{n,n}\cap G^{\prime}$,
and for $\mathtt{S}$ a sufficiently large finite set of places of $F$ and
$\lambda\in i\mathfrak{a}_{P}^{*}$, put

(10.18)
$\beta_{\eta}(W)=(\Delta_{G^{\prime}}^{\mathtt{S},*})^{-1}L^{\mathtt{S},*}(1,\Pi,\mathrm{As}_{G})\int_{N^{\prime}(F_{\mathtt{S}})\backslash
P_{n,n}^{\prime}(F_{\mathtt{S}})}W(p_{\mathtt{S}})\eta_{n+1}(p_{\mathtt{S}})\mathrm{d}p_{\mathtt{S}},\quad
W\in\mathcal{W}(\Pi_{\lambda},\psi_{N})$

where we have set

$L(s,\Pi,\mathrm{As}_{G})=L(s,\Pi_{1},\mathrm{As}^{(-1)^{n+1}})L(s,\Pi_{2},\mathrm{As}^{(-1)^{n+1}})$

with $\Pi=\Pi_{1}\boxtimes\Pi_{2}$, and where
$\Delta_{G^{\prime}}^{\mathtt{S},*}$ is defined in Subsection 3.4. This
expression is absolutely convergent and independent of $\mathtt{S}$ as soon as
it is sufficiently large by [Fli] and [JS]. The same definition also applies
to any $W\in\mathcal{W}(\Pi_{\lambda},\psi_{N}^{-1})$

Recall that a Levi subgroup $L_{\pi}$ attached to $(M_{P},\pi)$ was defined in
(10.3). We equip $i\mathfrak{a}_{M_{P}}^{L_{\pi},*}$ with the measure
described in [BPC22]*Section 6.2.8, which is just a normalization of the usual
Lebesgue measure on the induced $\mathbb{R}$-vector space. The following is
[BPC22]*Corollary 6.2.7.1.

######  Theorem 10.6.

Let $\chi\in\mathfrak{X}(G)$ be a $G$-regular cuspidal datum represented by
$(M_{P},\pi)$, and let $f\in\mathcal{S}_{\chi}([G])$. If $\chi$ is Hermitian,
then the function $\lambda\in
i\mathfrak{a}_{M_{P}}^{L_{\pi},*}\mapsto\beta_{\eta}(W_{f,\Pi_{\lambda}})$ is
Schwartz, and the resulting map

$\mathcal{S}_{\chi}([G])\to\mathcal{S}(i\mathfrak{a}_{M_{P}}^{L_{\pi},*}),\quad
f\mapsto\left(\lambda\mapsto\beta_{\eta}(W_{f,\Pi_{\lambda}})\right)$

is continuous. Moreover, we have

$P_{G^{\prime},\eta}(f)=\left\\{\begin{aligned}
&2^{-\dim\mathfrak{a}_{L_{\pi}}}\int_{i\mathfrak{a}_{M_{P}}^{L_{\pi},*}}\beta_{\eta}(W_{f,\Pi_{\lambda}})\mathrm{d}\lambda,&&\text{if
$\chi$ is Hermitian};\\\ &0,&&\text{otherwise.}\end{aligned}\right.$

####  10.5. The relative character $I_{\Pi_{\lambda}}$

Assume that $\chi\in\mathfrak{X}(G)$ is a $(G,H,\mu^{-1})$-regular and
Hermitian cuspidal datum represented by $(M_{P},\pi)$. For
$f\in\mathcal{S}([G])$ and $\Phi\in\mathcal{S}(\mathbb{A}_{E,n})$ we put

(10.19) $P_{H}(f,\Phi)=\int_{[H]}f(h)\Theta^{\prime}(g,\Phi)\mathrm{d}h.$

It is absolutely convergent and for a fixed $\Phi$, it defines a continuous
linear form on $\mathcal{S}([G])$. As $\chi$ is $(G,H,\mu^{-1})$-regular,
Theorem 10.4 says that the restriction of $P_{H}(\cdot,\Phi)$ to
$\mathcal{S}_{\chi}([G])$ extends continuously to $\mathcal{T}_{\chi}([G])$
and we denote this extension by $P_{H}^{*}(\cdot,\Phi)$. By Theorem 10.4 we
have

(10.20)
$P_{H}^{*}(E(\cdot,\varphi,\lambda),\Phi)=Z_{\psi}^{\mathrm{RS}}(W(\varphi,\lambda),\mu,\Phi,0).$

Let $\lVert\cdot\rVert$ be a norm on the $\mathbb{R}$-vector space
$i\mathfrak{a}_{M_{P}}^{L_{\pi},*}$.

######  Lemma 10.7.

Let $f\in\mathcal{S}(G(\mathbb{A}))$. There exists $N>0$ such that for every
$\lambda\in i\mathfrak{a}_{M_{P}}^{L_{\pi},*}$, the function

(10.21) $g\mapsto\sum_{\varphi\in
B_{P,\pi}}E(g,I_{P}(\lambda,f)\varphi,\lambda)\overline{\beta_{\eta}(W(\varphi,\lambda))}$

is defined by an absolutely convergent sum in $\mathcal{T}_{N,\chi}([G])$.
Moreover, for every continuous semi-norm $\nu_{N}$ on $\mathcal{T}_{N}([G])$
and every $d>0$ there exists a continuous semi-norm $\lVert\cdot\rVert_{d}$ on
$\mathcal{S}(G(\mathbb{A}))$ such that

(10.22) $\nu_{N}\left(\sum_{\varphi\in
B_{P,\pi}}E(\cdot,I_{P}(\lambda,f)\varphi,\lambda)\overline{\beta_{\eta}(W(\varphi,\lambda))}\right)\leq\lVert
f\rVert_{d}(1+\lVert\lambda\rVert)^{-d},\;f\in\mathcal{S}(G(\mathbb{A})),\;\lambda\in
i\mathfrak{a}_{M_{P}}^{L_{\pi},*}.$

###### Proof.

Let us write $\Pi^{\vee}_{-\lambda}$ for
$(\Pi_{\lambda})^{\vee}=(\Pi^{\vee})_{-\lambda}$, and
$\chi^{\vee}\in\mathfrak{X}(G)$ for the cuspidal datum represented by
$(M_{P},\pi^{\vee})$. It is $(G,H,\mu^{-1})$-regular and Hermitian.

By (10.15) and (10.16) we have

(10.23) $W_{K_{f,\chi}(g,\cdot),\Pi^{\vee}_{-\lambda}}=\sum_{\varphi\in
B_{P,\pi}}\langle
K_{f,\chi}(g,\cdot),E(\cdot,\overline{\varphi},-\lambda)\rangle\overline{W(\varphi,\lambda)},$

where the right hand is absolutely convergent in
$\mathcal{W}(\Pi^{\vee},\psi_{N}^{-1})$ by Proposition 4.18 (1). Note that

$\langle
K_{f,\chi}(g,\cdot),E(\cdot,\overline{\varphi},-\lambda)\rangle=\langle
K_{f}(g,\cdot),E(\cdot,\overline{\varphi},-\lambda)\rangle=E(g,I_{P}(\lambda,f)\varphi,\lambda).$

Therefore by continuity of $\beta_{\eta}$ applied to (10.23) we see that for
all $g\in[G]$

(10.24)
$\beta_{\eta}\left(W_{K_{f,\chi}(g,\cdot),\Pi^{\vee}_{-\lambda}}\right)=\sum_{\varphi\in
B_{P,\pi}}E(g,I_{P}(\lambda,f)\varphi,\lambda)\overline{\beta_{\eta}(W(\varphi,\lambda))}.$

It follows from Proposition 4.18 (1) and the fact that there exists $N$ such
that the map $\varphi\in\Pi\mapsto E(\varphi,\lambda)\in\mathcal{T}_{N}([G])$
is continuous ( [Lap]*Theorem 2.2) that, as a function of $g$, the RHS sum of
(10.24) is absolutely convergent in $\mathcal{T}_{N,\chi}([G])$.

Let $d>0$ and $X\in\mathcal{U}(\mathfrak{g}_{\infty})$. By Lemma 4.21, for
every continuous semi-norm $\nu$ on $\mathcal{S}([G])$, there is an
$N_{\nu}>0$ and a continuous semi-norm $\lVert.\rVert_{\nu}$ on
$\mathcal{S}(G(\mathbb{A}))$ such that

$\nu(K_{f,\chi}(g,\cdot))\ll\lVert g\rVert_{G}^{N_{\nu}}\lVert
f\rVert_{\nu},\quad g\in[G],\;f\in\mathcal{S}(G(\mathbb{A})).$

Note that
$\mathrm{R}_{g}(X)\beta_{\eta}\left(W_{K_{f,\chi}(g,\cdot),\Pi^{\vee}_{-\lambda}}\right)=\beta_{\eta}\left(W_{K_{\mathrm{L}(X)f,\chi}(g,\cdot),\Pi^{\vee}_{-\lambda}}\right)$,
where we use $\mathrm{R}_{g}(X)$ for $\mathrm{R}(X)$ applied to
$g\mapsto\beta_{\eta}\left(W_{K_{f,\chi}(g,\cdot),\Pi^{\vee}_{-\lambda}}\right)$.
The first part of Theorem 10.6 then implies the existence of $N_{d}>0$ and
$\lVert.\rVert_{d}$ such that

$\left|\beta_{\eta}\left(W_{K_{\mathrm{L}(X)f,\chi}(g,\cdot),\Pi^{\vee}_{-\lambda}}\right)\right|\ll\lVert
f\rVert_{d}\lVert
g\rVert_{G}^{N_{d}}(1+\lVert\lambda\rVert)^{-d},\;g\in[G],\;f\in\mathcal{S}(G(\mathbb{A})).$

Combining this with (10.24) this proves the desired estimate (10.22). ∎

It follows from Lemma 10.7, the continuity of $P_{H}^{*}(\cdot,\Phi)$ on
$\mathcal{T}_{\chi}([G])$ and (10.20) that we can define for every $\lambda\in
i\mathfrak{a}_{M_{P}}^{L_{\pi},*}$, $f\in\mathcal{S}(G(\mathbb{A}))$ and
$\Phi\in\mathcal{S}(\mathbb{A}_{E,n})$ the relative character

(10.25)
$I_{\Pi_{\lambda}}(f\otimes\Phi)=\sum_{\varphi\in\mathcal{B}_{P,\pi}}Z_{\psi}^{\mathrm{RS}}(W(I_{P}(f,\lambda)\varphi,\lambda),\Phi,\mu,0)\overline{\beta_{\eta}(W(\varphi,\lambda))},$

where this sum is absolutely convergent and does not depend on the choice of
$\mathcal{B}_{P,\pi}$.

######  Lemma 10.8.

For every $\lambda\in i\mathfrak{a}_{M_{P}}^{L_{\pi},*}$,
$f\in\mathcal{S}(G(\mathbb{A}))$ and $\Phi\in\mathcal{S}(\mathbb{A}_{E,n})$ we
have

(10.26)
$I_{\Pi_{\lambda}}(f\otimes\Phi)=P_{H}^{*}\left(g\mapsto\beta_{\eta}\left(W_{K_{f,\chi}(g,\cdot),\Pi^{\vee}_{-\lambda}}\right),\Phi\right).$

Moreover, the distribution $I_{\Pi_{\lambda}}$ extends by continuity to
$\mathcal{S}(G_{+}(\mathbb{A}))$ and the linear functional
$f_{+}\mapsto\int_{i\mathfrak{a}_{M_{P}}^{L_{\pi},*}}I_{\Pi_{\lambda}}(f_{+})\mathrm{d}\lambda$
is well defined and continuous.

###### Proof.

Equation (10.26) follows from applying $P_{H}^{*}(\cdot,\Phi)$ to (10.21), and
the second part of the statement is a consequence of Lemma 10.5 and the
estimate (10.22) of Lemma 10.7. ∎

####  10.6. Spectral expansion of $I_{\chi}$

Let $f\in\mathcal{S}(G(\mathbb{A}))$ and $\Phi\in\mathcal{S}(\mathbb{A}_{E})$.
Let $\chi\in\mathfrak{X}(G)$ be a cuspidal datum. Recall that we have defined
in (6.14) the kernel $K_{f\otimes\Phi,\chi}$. Let us put

$K^{H}_{f\otimes\Phi,\chi}(g)=\int_{[H]}K_{f\otimes\Phi,\chi}(h,g)\mathrm{d}h,\quad
K^{G^{\prime}}_{f,\chi}(g)=\int_{[G^{\prime}]}K_{f,\chi}(g,g^{\prime})\eta_{n+1}(g^{\prime})\mathrm{d}g^{\prime},\quad
g\in[G].$

We now write a spectral expansion of $I_{\chi}$ the relative character defined
in Theorem 6.10.

######  Proposition 10.9.

Assume that $\chi$ is $(G,H,\mu^{-1})$-regular. We have the following
assertions.

  1. (1)

If $\chi$ is not Hermitian, then $K^{G^{\prime}}_{f,\chi}(g)=0$ for every
$g\in[G]$. Moreover $I_{\chi}(f\otimes\Phi)=0$.

  2. (2)

Assume that $\chi$ is Hermitian. Then

(10.27)
$I_{\chi}(f\otimes\Phi)=\int_{[G^{\prime}]}K^{H}_{f\otimes\Phi,\chi}(g^{\prime})\eta_{n+1}(g^{\prime})\mathrm{d}g^{\prime}=P_{H}^{*}\left(K^{G^{\prime}}_{f,\chi},\Phi\right),$

where the middle integral is absolutely convergent.

###### Proof.

Let us assume first that $\chi$ is not Hermitian. By definition we have

$K_{f,\chi}^{G^{\prime}}(g)=P_{G^{\prime},\eta}\left(K_{f,\chi}(g,\cdot)\right).$

Recall that $\chi^{\vee}\in\mathfrak{X}(G)$ is the $(G,H,\mu^{-1})$-regular
cuspidal datum represented by $(M_{P},\pi^{\vee})$. Since $\chi^{\vee}$ is not
Hermitian, the linear form $P_{G^{\prime},\eta}$ vanishes identically on
$\mathcal{S}_{\chi^{\vee}}([G])$ by Theorem 10.6. It follows that
$K_{f,\chi}^{G^{\prime}}(g)=0$. By Proposition 6.13 we conclude that

$\left|i_{\chi}^{T}(f\otimes\Phi)\right|\ll e^{-N\lVert T\rVert}$

for some $N>0$ and all $T$ sufficiently positive. By Proposition 6.14, the
function $T\mapsto i_{\chi}^{T}(f\otimes\Phi)$ is exponential polynomial with
constant term $I_{\chi}(f\otimes\Phi)$. It follows that
$I_{\chi}(f\otimes\Phi)=0$.

We now assume that $\chi$ is Hermitian. By definition we have

$K_{f\otimes\Phi,\chi}(\cdot,g^{\prime})=K_{f,\chi}(\cdot,g^{\prime})\Theta(.,\Phi).$

By [BPCZ]*Lemma 2.10.1.3, the family of functions $g^{\prime}\mapsto
K_{f,\chi}(\cdot,g^{\prime})$ is absolutely integrable over $[G^{\prime}]$ in
$\mathcal{T}_{N}([G])$ for some $N$. By Theorem 10.4, the linear form
$P_{H}(\cdot,\Phi)$ extends continuously from $\mathcal{S}_{\chi}([G])$ to
$\mathcal{T}_{\chi}([G])$. Since

$K_{f\otimes\Phi,\chi}^{H}(g^{\prime})=P_{H}\left(K_{f,\chi}(\cdot,g^{\prime}),\Phi\right)=P_{H}^{*}\left(K_{f,\chi}(\cdot,g^{\prime}),\Phi\right),$

we conclude that the integral

(10.28)
$\int_{[G^{\prime}]}K_{f\otimes\Phi,\chi}^{H}(g^{\prime})\eta_{n+1}(g^{\prime})\mathrm{d}g^{\prime}$

is absolutely convergent, and equals

$P_{H}^{*}\left(\int_{[G^{\prime}]}K_{f\otimes\Phi,\chi}(\cdot,g^{\prime})\eta_{n+1}(g^{\prime})\mathrm{d}g^{\prime},\Phi\right)=P_{H}^{*}\left(K_{f,\chi}^{G^{\prime}},\Phi\right).$

The absolute convergence of (10.28), together with Theorem 6.8 and Theorem
6.10 implies that this integral equals $I_{\chi}(f\otimes\Phi)$. ∎

######  Theorem 10.10.

Assume that $\chi\in\mathfrak{X}(G)$ is an Hermitian and
$(G,H,\mu^{-1})$-regular cuspidal datum, represented by $(M_{P},\pi)$. Then
for all $f_{+}\in\mathcal{S}(G_{+}(\mathbb{A}))$ we have

(10.29)
$I_{\chi}(f_{+})=2^{-\dim\mathfrak{a}_{L_{\pi}}}\int_{i\mathfrak{a}_{M_{P}}^{L_{\pi},*}}I_{\Pi_{\lambda}}(f_{+})\mathrm{d}\lambda,$

where the integral on the right is absolutely convergent.

###### Proof.

By Theorem 6.10 and Lemma 10.8, the two sides of (10.29) are continuous in
$f_{+}$, and the RHS is absolutely convergent. We may therefore assume that
$f_{+}=f\otimes\Phi$. Let us continue with the notations of Proposition 10.9.
As $\chi^{\vee}$ is Hermitian and $(G,H,\mu^{-1})$-regular, Theorem 10.6
applies and yields for all $g\in[G]$

(10.30)
$K_{f,\chi}^{G^{\prime}}(g)=P_{G^{\prime},\eta}(K_{f,\chi}(g,\cdot))=2^{-\dim\mathfrak{a}_{L_{\pi}}}\int_{i\mathfrak{a}_{M_{P}}^{L_{\pi},*}}\beta_{\eta}\left(W_{K_{f,\chi}(g,\cdot),\Pi^{\vee}_{-\lambda}}\right)\mathrm{d}\lambda.$

By Lemmas 10.7 and 10.8 we have

$P_{H}^{*}\left(g\mapsto\int_{i\mathfrak{a}_{M_{P}}^{L_{\pi},*}}\beta_{\eta}\left(W_{K_{f,\chi}(g,\cdot),\Pi^{\vee}_{-\lambda}}\right)\mathrm{d}\lambda,\Phi\right)=\int_{i\mathfrak{a}_{M_{P}}^{L_{\pi},*}}I_{\Pi_{\lambda}}(f_{+})\mathrm{d}\lambda.$

It now follows from Proposition 10.9 and (10.30) that

$I_{\chi}(f\otimes\Phi)=P_{H}^{*}\left(K^{G^{\prime}}_{f,\chi},\Phi\right)=2^{-\dim\mathfrak{a}_{L_{\pi}}}\int_{i\mathfrak{a}_{M_{P}}^{L_{\pi},*}}I_{\Pi_{\lambda}}(f_{+})\mathrm{d}\lambda.$

This proves the theorem. ∎

###  11\. Periods and relative characters on the unitary groups

####  11.1. Notations

We keep the notations from Section 8. We fix a nondegenerate skew
$\mathsf{c}$-Hermitian space $V$ of dimension $n$, and a polarization
$\operatorname{Res}V=L+L^{\vee}$. To shorten notation, we will drop the
subscript $V$ from all the groups, so that in particular
$\operatorname{U}=\operatorname{U}_{V}=\operatorname{U}(V)\times\operatorname{U}(V)$,
$\operatorname{U}^{\prime}=\operatorname{U}(V)$ seen as a subgroup of
$\operatorname{U}$ by the diagonal embedding,
$\operatorname{U}_{+}=\operatorname{U}\times L^{\vee}\times L^{\vee}$. We will
denote by $S(V)$ the Heisenberg group, $J(V)=S(V)\rtimes U(V)$ the Jacobi
group (see Subsection 5.3) and write
$\widetilde{\operatorname{U}}=\operatorname{U}(V)\times J(V)$.

Recall that we have defined in Subsection 5.4 the Weil representation
$\omega_{\psi,\mu,V}$ realized on $\mathcal{S}(L^{\vee}(\mathbb{A}))$, and for
any $\phi\in\mathcal{S}(L^{\vee}(\mathbb{A}))$ the theta function
$\theta_{\psi,\mu,V}(\cdot,\phi)\in\mathcal{T}([\operatorname{U}(V)])$. The
characters $\psi$ and $\mu$ are fixed throughout this section, hence we simply
write $\theta(\cdot,\phi)$ for $\theta_{\psi,\mu,V}(\cdot,\phi)$, and
$\theta^{\vee}(\cdot,\phi)$ for $\theta_{\psi^{-1},\mu^{-1},V}(\cdot,\phi)$.

####  11.2. Regularity conditions on cuspidal data for unitary groups

Let $\chi_{=}(\chi_{V},\chi_{V}^{\prime})\in\mathfrak{X}(\operatorname{U})$ be
a cuspidal datum represented by $(M_{Q},\sigma)$ where $M_{Q}$ is a Levi
subgroup of a standard parabolic subgroup $Q$ of $\operatorname{U}$ and
$\sigma$ is an irreducible cuspidal automorphic representation of
$M_{Q}(\mathbb{A})$. We have the following decompositions.

  * •

$M_{Q}=M_{V}\times M^{\prime}_{V}$, $M_{V}=G_{n_{1}}\times\cdots\times
G_{n_{r}}\times\operatorname{U}(V_{0})$,
$M^{\prime}_{V}=G_{n_{1}^{\prime}}\times\cdots\times
G_{n_{r^{\prime}}^{\prime}}\times\operatorname{U}(V_{0}^{\prime})$.

  * •

$\sigma=\sigma_{V}\boxtimes\sigma^{\prime}_{V}$,
$\sigma_{V}=\pi_{1}\boxtimes\cdots\boxtimes\pi_{r}\boxtimes\sigma_{0}$,
$\sigma^{\prime}_{V}=\pi_{1}^{\prime}\boxtimes\cdots\boxtimes\pi_{r^{\prime}}^{\prime}\boxtimes\sigma_{0}^{\prime}$.

where

  * •

$V_{0}$ and $V_{0}^{\prime}$ are nondegenerate skew $\mathsf{c}$-Hermitian
vector spaces of dimension $n_{0}$ and $n_{0}^{\prime}$ respectively,
$2(n_{1}+\cdots+n_{r})+n_{0}=2(n_{1}^{\prime}+\cdots+n_{r^{\prime}}^{\prime})+n_{0}^{\prime}=n$,

  * •

$\pi_{i}$ and $\pi_{i}^{\prime}$ are cuspidal automorphic representations of
the $G_{n_{i}}(\mathbb{A})$ and $G_{n_{i}^{\prime}}(\mathbb{A})$ respectively,

  * •

$\sigma_{0}$ and $\sigma_{0}^{\prime}$ are cuspidal automorphic
representations of $\operatorname{U}(V_{0})(\mathbb{A})$ and
$\operatorname{U}(V_{0}^{\prime})(\mathbb{A})$ respectively.

We shall say that

  * •

$\chi_{V}$ is regular if the representations
$\pi_{1},\ldots,\pi_{r},\pi_{1}^{*},\ldots,\pi_{r}^{*}$ are pairwise distinct,
and the same applies to $\chi_{V}^{\prime}$,

  * •

$\chi$ is $\operatorname{U}$-regular if $\chi_{V}$ and $\chi^{\prime}_{V}$ are
both regular,

  * •

$\chi$ is $(\operatorname{U}^{\prime},\mu^{-1})$-regular if for any $i,j$, the
representation $\mu^{-1}\pi_{i}$ is neither isomorphic to
$(\pi_{j}^{\prime})^{\vee}$ nor $(\pi_{j}^{\prime})^{\mathsf{c}}$,

  * •

$\chi$ is $(\operatorname{U},\operatorname{U}^{\prime},\mu^{-1})$-regular if
it is both $\operatorname{U}$-regular and $\operatorname{U}^{\prime}$-regular.

####  11.3. Truncated Fourier–Jacobi periods

Let $\chi\in\mathfrak{X}(\operatorname{U})$ be a cuspidal datum represented by
$(M_{Q},\sigma)$. For
$\varphi\in\mathcal{A}_{Q,\sigma,\mathrm{cusp}}(\operatorname{U})$ and
$\lambda\in\mathfrak{a}_{Q}^{*}$ the Eisenstein series
$E(\cdot,\varphi,\lambda)$ is meromorphic in $\lambda$ and is holomorphic when
$\lambda\in i\mathfrak{a}_{Q}^{*}$. Recall that we have defined in Subsection
8.5 the truncation operator $\Lambda_{u}^{T}$, where $T$ is a parameter in
$\mathfrak{a}_{0}$.

######  Proposition 11.1.

Let $T\in\mathfrak{a}_{0}$ be sufficiently positive in the sense of Subsection
4.5. Then for any
$\varphi\in\mathcal{A}_{Q,\sigma,\mathrm{cusp}}(\operatorname{U})$,
$\lambda\in i\mathfrak{a}_{Q}^{*}$ and
$\phi\in\mathcal{S}(L^{\vee}(\mathbb{A}))$ we have the following assertions.

  1. (1)

The integral

(11.1)
$\int_{[\operatorname{U}^{\prime}]}\Lambda_{u}^{T}\left(E(\cdot,\varphi,\lambda)\cdot\theta^{\vee}(\cdot,\phi)\right)(x)\mathrm{d}x$

is absolutely convergent.

  2. (2)

If $\chi$ is $(\operatorname{U}^{\prime},\mu^{-1})$-regular, the integral
(11.1) is independent of $T$.

###### Proof.

The first assertion follows directly from Proposition 8.7. We now show the
second assertion. We make use of the notations from Section 8. In particular,
for any subgroup $H\subset\operatorname{U}$ we set
$H^{\prime}:=H\cap\operatorname{U}^{\prime}$.

Recall that $\mathcal{F}$ is the set of $D$-parabolic subgroups of $J(V)$ (see
Subsection 5.3). For each
$\widetilde{\varphi}\in\mathcal{T}([\widetilde{\operatorname{U}}],\psi)$,
$R\in\mathcal{F}$ and $x\in
R^{\prime}(F)\backslash\operatorname{U}^{\prime}(\mathbb{A})$, we define

$\Lambda_{u}^{R,T}\widetilde{\varphi}(x)=\sum_{\begin{subarray}{c}S\subset
R\\\ S\in\mathcal{F}\end{subarray}}\epsilon_{S}^{R}\sum_{\delta\in
S^{\prime}(F)\backslash
R^{\prime}(F)}\widehat{\tau}_{S^{\prime}}^{R^{\prime}}(H_{S^{\prime}}(\delta
x)-T_{S^{\prime}})\widetilde{\varphi}_{\operatorname{U}(V)\times S}(\delta
x),$

where we recall that $\widehat{\tau}_{S^{\prime}}^{R^{\prime}}$ is defined in
Subsection 4.5. This is the relative version of the truncation operator
$\Lambda_{u}^{T}$ defined in (8.14). One show similarly as in Proposition 8.7
that $\Lambda^{R,T}_{u}\widetilde{\varphi}$ is rapidly decreasing as a
function on $R^{\prime}(F)\backslash\operatorname{U}^{\prime}(\mathbb{A})$. It
follows that the integral

$\int_{R^{\prime}(F)\backslash\operatorname{U}^{\prime}(\mathbb{A})}\Lambda_{u}^{R,T}\widetilde{\varphi}(x)\mathrm{d}x$

converges absolutely. By (8.12) we have

$\widehat{\tau}_{S^{\prime}}(H_{S^{\prime}}(x)-T-T^{\prime})=\sum_{R\supset
S}\epsilon_{R}\widehat{\tau}_{S^{\prime}}^{R^{\prime}}(H_{R^{\prime}}(x)-T_{R^{\prime}})\Gamma^{\prime}_{R^{\prime}}(H_{R^{\prime}}(x)-T_{R^{\prime}},T^{\prime}_{R^{\prime}}).$

When both $T$ and $T+T^{\prime}$ are sufficiently positive we therefore have

$\Lambda_{u}^{T+T^{\prime}}\widetilde{\varphi}(x)=\sum_{R\in\mathcal{F}}\sum_{\delta\in
R^{\prime}(F)\backslash\operatorname{U}^{\prime}(F)}\Gamma^{\prime}_{R^{\prime}}(H_{R^{\prime}}(\delta
x)-T_{R^{\prime}},T^{\prime}_{R^{\prime}})\Lambda_{u}^{R,T}\widetilde{\varphi}(\delta
x).$

Note that as functions on $[\operatorname{U}^{\prime}]$ we have
$(\Lambda_{u}^{R,T}\widetilde{\varphi})_{R^{\prime}}=\Lambda_{u}^{R,T}\widetilde{\varphi}_{\widetilde{R}}$,
where $\widetilde{R}=R^{\prime}\times R$ is a $D$-parabolic subgroup of
$\widetilde{\operatorname{U}}$ (see (8.1)). It follows that

$\displaystyle\int_{[\operatorname{U}^{\prime}]}\Lambda_{u}^{T+T^{\prime}}\widetilde{\varphi}(x)\mathrm{d}x$
$\displaystyle=\sum_{R\in\mathcal{F}}\int_{R^{\prime}(F)\backslash\operatorname{U}^{\prime}(\mathbb{A})}\Gamma^{\prime}_{R^{\prime}}(H_{R^{\prime}}(x)-T_{R^{\prime}},T^{\prime}_{R^{\prime}})\Lambda_{u}^{R,T}\widetilde{\varphi}(x)\mathrm{d}x$
$\displaystyle=\sum_{R\in\mathcal{F}}\int_{[\operatorname{U}^{\prime}]_{R^{\prime}}}\Gamma^{\prime}_{R^{\prime}}(H_{R^{\prime}}(x)-T_{R^{\prime}},T^{\prime}_{R^{\prime}})(\Lambda_{u}^{R,T}\widetilde{\varphi}_{\widetilde{R}})(x)\mathrm{d}x$
$\displaystyle=\sum_{R\in\mathcal{F}}\int_{[M_{R^{\prime}}]}\int_{K^{\prime}}e^{\langle-2\rho_{R^{\prime}},H_{R^{\prime}}(m)\rangle}\Gamma^{\prime}_{R^{\prime}}(H_{R^{\prime}}(m)-T_{R^{\prime}},T^{\prime}_{R^{\prime}})\Lambda^{R,T}_{u}\widetilde{\varphi}_{\widetilde{R}}(mk)\mathrm{d}m\mathrm{d}k.$

We now apply the previous calculation to
$\widetilde{\varphi}=E(\cdot,\varphi,\lambda)\theta^{\vee}(\cdot,\phi)$ where
$\varphi\in\mathcal{A}_{Q,\sigma}(\operatorname{U})$, $\lambda\in
i\mathfrak{a}_{Q}^{*}$ and $\phi\in\mathcal{S}(L^{\vee}(\mathbb{A}))$. Set
$R_{\operatorname{U}}=R^{\prime}\times R^{\prime}$. It is a parabolic subgroup
of $\operatorname{U}$. By Proposition 5.9, we have

$\left(E(\cdot,\varphi,\lambda)\theta^{\vee}(\cdot,\phi)\right)_{\widetilde{R}}(x)=E_{R_{\operatorname{U}}}(x,\varphi,\lambda)\cdot{}_{R}\theta^{\vee}(x,\phi),$

where ${}_{R}\theta^{\vee}$ is defined in (5.20). We will therefore show for
that $R\neq J(V)$ and $k\in K^{\prime}$, the integral

$\int_{[M_{R^{\prime}}]}e^{\langle-2\rho_{R^{\prime}},H_{R^{\prime}}(m)\rangle}\Gamma^{\prime}_{R^{\prime}}(H_{R^{\prime}}(m)-T_{R^{\prime}},T^{\prime}_{R^{\prime}})\Lambda^{R,T}_{u}\left(E_{R_{\operatorname{U}}}(\cdot,\varphi,\lambda)\cdot{}_{R}\theta^{\vee}(\cdot,\phi)\right)(mk)\mathrm{d}m$

vanishes, which concludes the proof of the proposition. By Lemma 5.7, we have
$R\neq J(V)$ if and only if $R_{\operatorname{U}}\neq\operatorname{U}$. By the
description of the mixed model (5.15), we see that for every $a\in
A_{R^{\prime}}^{\infty}$ we have

$\Lambda^{R,T}_{u}\left(E_{R_{\operatorname{U}}}(\varphi,\lambda)\cdot{}_{R}\theta^{\vee}(\phi)\right)(amk)=e^{\langle\lambda+\rho_{R_{\operatorname{U}}},H_{R_{\operatorname{U}}}(a)\rangle}\mu^{-1}(a)|a|^{\frac{1}{2}}\Lambda^{R,T}_{u}\left(E_{R_{\operatorname{U}}}(\varphi,\lambda)\cdot{}_{R}\theta^{\vee}(\phi)\right)(mk).$

It is therefore enough to show that if
$R_{\operatorname{U}}\neq\operatorname{U}$

$\int_{[M_{R^{\prime}}]^{1}}\Lambda^{R,T}_{u}\left(E_{R_{\operatorname{U}}}(\cdot,\varphi,\lambda)\cdot{}_{R}\theta^{\vee}(\cdot,\phi)\right)(mk)\mathrm{d}m=0.$

The constant term of the cuspidal Eisenstein series is given by

$E_{R_{\operatorname{U}}}(mk,\varphi,\lambda)=\sum_{w\in
W(Q;R_{\operatorname{U}})}E^{R_{\operatorname{U}}}(mk,M(w,\lambda)\varphi,w\lambda),$

where $W(Q;R_{\operatorname{U}})$ is some subset of the Weyl group of
$\operatorname{U}$ (see [MW95]*II.1.7). Note that the representation $w\pi$ is
also $(\operatorname{U}^{\prime},\mu^{-1})$-regular, so that we may assume
that $w=1$ (which implies $Q\subset R_{\operatorname{U}})$. We are then left
to show that the integral

(11.2)
$\int_{[M_{R^{\prime}}]^{1}}\Lambda^{R,T}_{u}\left(E^{R_{\operatorname{U}}}(\cdot,\varphi,\lambda)\cdot{}_{R}\theta^{\vee}(\cdot,\phi)\right)(mk)\mathrm{d}m$

vanishes when $R_{\operatorname{U}}\supset Q$ and
$R_{\operatorname{U}}\neq\operatorname{U}$.

Form the decomposition $M_{R_{\operatorname{U}}}=M_{R,\bullet}\times M_{R,*}$,
where $M_{R,\bullet}$ is isomorphic to a product of
$\operatorname{GL}_{n_{i}}$ and $M_{R,*}$ is isomorphic to a product of two
identical unitary group. We write similarly
$M_{R^{\prime}}=M_{R^{\prime},\bullet}\times M_{R^{\prime},*}$. Then
$M_{R,\bullet}\cong M_{R^{\prime},\bullet}^{2}$ and the embedding
$M_{R^{\prime},\bullet}\subset M_{R,\bullet}$ is the diagonal inclusion. Write
$Q_{1}\times Q_{2}=M_{R,\bullet}\cap Q$ and the restriction of the
representation $\sigma$ to $M_{Q_{1}}(\mathbb{A})$ and $M_{Q_{2}}(\mathbb{A})$
as $\sigma=\sigma_{1}\boxtimes\sigma_{2}$. We can assume that
$\mathrm{R}(k)(\varphi)|_{[M_{R_{\operatorname{U}}}]}=(\varphi_{1}\otimes\varphi_{2})\otimes\varphi_{*}$,
where $\varphi_{1}\otimes\varphi_{2}\in\mathcal{A}_{{Q_{1}\times
Q_{2}},\sigma_{1}\otimes\sigma_{2}}(M_{R,\bullet})$ and
$\varphi_{*}\in\mathcal{A}_{{Q\cap M_{R,*}}}(M_{R,*})$. Moreover, it follows
from the description of the parabolic subgroups of $J(V)$ in Subsection 5.3
that the operator $\Lambda_{u}^{R,T}$ is a product of

  * •

the usual Arthur operator $\Lambda^{T}$ attached to the product of general
linear groups $M_{R^{\prime},\bullet}$, seen as an operator of the space
$\mathcal{T}([M_{R,\bullet}])$ acting on the second component (see [Arthur4]),

  * •

the truncation operator $\Lambda_{u}^{T}$ built in Section 8.5 attached to the
unitary group $M_{R^{\prime},*}$, seen as an operator of the space
$\mathcal{T}([M_{R,*}])$.

For $m\in[M_{R^{\prime}}]^{1}$ write $m=m_{\bullet}m_{*}$ according to this
decomposition. Using the description of mixed model (5.15), we see that
${}_{R}\theta^{\vee}(mk,\phi)=\mu^{-1}(m_{\bullet})\cdot{}_{R}\theta^{\vee}(m_{*}k,\phi)$.
Thus the integral in (11.2) has

(11.3)
$\int_{[M_{R^{\prime},\bullet}]^{1}}E(m_{\bullet},\varphi_{1,},\lambda)\Lambda^{T}E(m_{\bullet},\varphi_{2},\lambda)\mu^{-1}(m_{\bullet})\mathrm{d}m_{\bullet}$

as a factor. The hypothesis $R_{\operatorname{U}}\neq\operatorname{U}$ implies
that $M_{R^{\prime},\bullet}$ is not trivial. By Langlands’ formula for the
inner product of truncated Eisenstein series applied to each general linear
group ([Arthur4]), (11.3) is zero as $\chi$ is
$(\operatorname{U}^{\prime},\mu^{-1})$-regular. This concludes the proof. ∎

####  11.4. The relative character $J_{Q,\sigma}$

Assume now that $\chi\in\mathfrak{X}(\operatorname{U})$ is a
$(\operatorname{U}^{\prime},\mu^{-1})$-regular cuspidal datum represented by
$(M_{Q},\sigma)$. For
$\varphi\in\mathcal{A}_{Q,\sigma,\mathrm{cusp}}(\operatorname{U})$,
$\phi\in\mathcal{S}(L^{\vee}(\mathbb{A}))$ and $T$ sufficiently positive, we
put

$\mathcal{P}(\varphi,\phi,\lambda)=\int_{[\operatorname{U}^{\prime}]}\Lambda_{u}^{T}\left(E(\cdot,\varphi,\lambda)\cdot\theta^{\vee}(\cdot,\phi)\right)(h)\mathrm{d}h,$

which is the truncated Fourier–Jacobi period. By Proposition 11.1 this does
not depend on $T$, justifying the notation. It is a meromorphic function in
$\lambda$ and is holomorphic when $\lambda\in i\mathfrak{a}_{Q}^{*}$.
Moreover, it follows from the continuity of $\Lambda_{u}^{T}$ shown in
Proposition 8.7 that there are continuous semi-norms $\lVert\cdot\rVert$ and
$\lVert\cdot\rVert_{L^{\vee}}$ on
$\mathcal{A}_{Q,\sigma,\mathrm{cusp}}(\operatorname{U})$ and
$\mathcal{S}(L^{\vee}(\mathbb{A}))$ respectively such that

(11.4)
$\left|\mathcal{P}(\varphi,\phi,\lambda)\right|\ll\lVert\varphi\rVert\lVert\phi\rVert_{L^{\vee}},\;\varphi\in\mathcal{A}_{Q,\sigma,\mathrm{cusp}}(\operatorname{U}),\;\phi\in\mathcal{S}(L^{\vee}(\mathbb{A})).$

Recall that we have fixed in Subsection 4.5 a maximal compact subgroup $K$ of
$\operatorname{U}(\mathbb{A})$. Let $\mathcal{B}_{Q,\sigma}$ be a basis of
$\mathcal{A}_{Q,\sigma,\mathrm{cusp}}(\operatorname{U})$ as in Subsection 4.6.
By Proposition 4.18 (1), for all
$f\in\mathcal{S}(\operatorname{U}(\mathbb{A}))$ the series

$\sum_{\varphi\in\mathcal{B}_{Q,\sigma}}(I_{Q}(\lambda,f)\varphi)\otimes\overline{\varphi}$

is absolutely convergent in
$\mathcal{A}_{Q,\sigma,\mathrm{cusp}}(\operatorname{U})\operatorname{\widehat{\otimes}}\overline{\mathcal{A}_{Q,\sigma,\mathrm{cusp}}(\operatorname{U})}$.
We can define for all $\lambda\in i\mathfrak{a}_{Q}^{*}$ and
$\phi_{1},\phi_{2}\in\mathcal{S}(L^{\vee}(\mathbb{A}))$ the relative character

(11.5)
$J_{Q,\sigma}(\lambda,f\otimes\phi_{1}\otimes\phi_{2})=\sum_{\varphi\in\mathcal{B}_{Q,\sigma}}\mathcal{P}(I_{Q}(\lambda,f)\varphi,\phi_{1},\lambda)\overline{\mathcal{P}(\varphi,\overline{\phi_{2}},\lambda)}.$

This is independent of the choice of basis and for fixed $f$, $\phi_{1}$ and
$\phi_{2}$ it is an holomorphic expression in $\lambda\in
i\mathfrak{a}_{Q}^{*}$. It follows from (11.4) and Proposition 4.18 (1) that
for fixed $\lambda$, the map $f\otimes\phi_{1}\otimes\phi_{2}\mapsto
J_{Q,\sigma}(\lambda,f\otimes\phi_{1}\otimes\phi_{2})$ extends by continuity
to $\mathcal{S}(\operatorname{U}_{+}(\mathbb{A}))$. Moreover we have the
following functional equation.

######  Lemma 11.2.

Let $(M_{Q_{1}},\sigma_{1})$ and $(M_{Q_{2}},\sigma_{2})$ be two pairs
representing the same $(\operatorname{U}^{\prime},\mu^{-1})$-regular cuspidal
datum $\chi$. Then for all $w\in W(Q_{1},Q_{2})$ (see Subsection 4.5) such
that $\sigma_{2}=w\sigma_{1}$, and for every $\lambda\in
i\mathfrak{a}_{Q_{1}}^{*}$, $f\in\mathcal{S}(\operatorname{U}(\mathbb{A}))$,
$\phi_{1},\phi_{2}\in\mathcal{S}(L^{\vee}(\mathbb{A}))$, we have

$J_{Q_{1},\sigma_{1}}(\lambda,f\otimes\phi_{1}\otimes\phi_{2})=J_{Q_{2},\sigma_{2}}(w\lambda,f\otimes\phi_{1}\otimes\phi_{2}).$

###### Proof.

This follows from the functional equation of Eisenstein series and the fact
that the intertwining operator sends $\mathcal{B}_{Q_{1},\sigma_{1}}$ to a
$K$-basis $\mathcal{B}_{Q_{2},\sigma_{2}}$. ∎

######  Proposition 11.3.

There exist continuous semi-norms $\lVert\cdot\rVert$ on
$\mathcal{S}(\operatorname{U}(\mathbb{A}))$ and $\lVert\cdot\rVert_{L^{\vee}}$
on $\mathcal{S}(L^{\vee}(\mathbb{A}))$ such that

(11.6)
$\sum_{(M_{Q},\sigma)}\int_{i\mathfrak{a}_{Q}^{*}}\left|J_{Q,\sigma}(\lambda,f\otimes\phi_{1}\otimes\phi_{2})\right|\mathrm{d}\lambda\leq\lVert
f\rVert\lVert\phi_{1}\rVert_{L^{\vee}}\lVert\phi_{2}\rVert_{L^{\vee}},\;f\in\mathcal{S}(\operatorname{U}(\mathbb{A})),\;\phi_{1},\phi_{2}\in\mathcal{S}(L^{\vee}(\mathbb{A})),$

where $(M_{Q},\sigma)$ ranges over a set of representatives of
$(\operatorname{U},\operatorname{U}^{\prime},\mu^{-1})$-regular cuspidal data
of $\operatorname{U}$.

###### Proof.

Fix $\phi_{1}$ and $\phi_{2}$. We begin by showing that the functional
$f\mapsto\left(((Q,\sigma),\lambda)\mapsto
J_{Q,\sigma}(\lambda,f,\phi_{1},\phi_{2})\right)$ valued in the space of
$L^{1}$ functions with variables $((Q,\sigma),\lambda)$ (given the product of
the counting measure and the Lebesgue measure) is continuous. To prove this,
it is enough to show that for every
$f\in\mathcal{S}(\operatorname{U}(\mathbb{A}))$ the sum

(11.7)
$\sum_{(M_{Q},\sigma)}\int_{i\mathfrak{a}_{Q}^{*}}\left|J_{Q,\sigma}(\lambda,f\otimes\phi_{1}\otimes\phi_{2})\right|\mathrm{d}\lambda$

is finite. Indeed, if it is, then our functional is the pointwise limit of a
sequence of continuous forms on $\mathcal{S}(\operatorname{U}(\mathbb{A}))$,
hence it is continuous by the closed graph theorem.

Let $f\in\mathcal{S}(\operatorname{U}(\mathbb{A}))$. By Proposition 11.1, for
$T$ positive enough (11.7) is

$\sum_{(M_{Q},\sigma)}\int_{i\mathfrak{a}_{Q}^{*}}\left|\sum_{\varphi\in\mathcal{B}_{Q,\sigma}}\int_{[\operatorname{U}^{\prime}]\times[\operatorname{U}^{\prime}]}\Lambda_{u}^{T}\left(E(I_{Q}(\lambda,f)\varphi,\lambda)\cdot\theta^{\vee}(\phi_{1})\right)(h_{1})\overline{\Lambda_{u}^{T}\left(E(\varphi,\lambda)\cdot\theta^{\vee}(\overline{\phi_{2}})\right)}(h_{2})\mathrm{d}h_{1}\mathrm{d}h_{2}\right|\mathrm{d}\lambda.$

By Proposition 4.18 (1), we can switch the last sum and integral, so that this
expression is bounded above by

(11.8)
$\sum_{(M_{Q},\sigma)}\int_{i\mathfrak{a}_{Q}^{*}}\int_{[\operatorname{U}^{\prime}]\times[\operatorname{U}^{\prime}]}\left|\sum_{\varphi\in\mathcal{B}_{Q,\sigma}}\Lambda_{u}^{T}\left(E(I_{Q}(\lambda,f)\varphi,\lambda)\cdot\theta^{\vee}(\phi_{1})\right)(h_{1})\overline{\Lambda_{u}^{T}\left(E(\varphi,\lambda)\cdot\theta^{\vee}(\overline{\phi_{2}})\right)}(h_{2})\right|\mathrm{d}h_{1}\mathrm{d}h_{2}\mathrm{d}\lambda.$

By the Dixmier–Malliavin theorem, we may assume that $f=f_{1}*f_{2}^{*}$ where
$f_{2}^{*}(g)=\overline{f_{2}}(g^{-1})$. By change of basis, the inner term in
(11.8) is

$\sum_{\varphi\in\mathcal{B}_{Q,\sigma}}\Lambda_{u}^{T}\left(E(I_{Q}(\lambda,f_{1})\varphi,\lambda)\cdot\theta^{\vee}(\phi_{1})\right)(h_{1})\overline{\Lambda_{u}^{T}\left(E(I_{Q}(\lambda,f_{2})\varphi,\lambda)\cdot\theta^{\vee}(\overline{\phi_{2}})\right)}(h_{2}).$

It follows from the Cauchy–Schwarz inequality that, up to replacing
$\overline{\phi_{2}}$ by $\phi_{2}$, (11.8) is bounded above by the square
root of the product over $i=1,2$ of

(11.9)
$\sum_{(M_{Q},\sigma)}\int_{i\mathfrak{a}_{Q}^{*}}\int_{[\operatorname{U}^{\prime}]\times[\operatorname{U}^{\prime}]}\sum_{\varphi\in\mathcal{B}_{Q,\sigma}}\left\lvert\Lambda_{u}^{T}\left(E(I_{Q}(\lambda,f_{i})\varphi,\lambda)\cdot\theta^{\vee}(\phi_{i})\right)(h_{i})\right\rvert^{2}\mathrm{d}h_{1}\mathrm{d}h_{2}\mathrm{d}\lambda.$

Set $g_{i}=f_{i}*f_{i}^{*}$. By Proposition 4.18 (2), the series

$\sum_{\varphi\in\mathcal{B}_{Q,\sigma}}E(I_{Q}(\lambda,g_{i})\varphi,\lambda)\cdot\theta^{\vee}(\phi_{i})\overline{E(\varphi,\lambda)\cdot\theta^{\vee}(\phi_{i})}$

is absolutely convergent in
$\mathcal{T}([\operatorname{U}\times\operatorname{U}])$. Using again a change
of variable and Proposition 8.7 we have

$\displaystyle\sum_{\varphi\in\mathcal{B}_{Q,\sigma}}\left\lvert\Lambda_{u}^{T}\left(E(I_{Q}(\lambda,f_{i})\varphi,\lambda)\cdot\theta^{\vee}(\phi_{i})\right)\right\rvert^{2}$
$\displaystyle=\sum_{\varphi\in\mathcal{B}_{Q,\sigma}}\Lambda_{u}^{T}\left(E(I_{Q}(\lambda,g_{i})\varphi,\lambda)\cdot\theta^{\vee}(\phi_{i})\right)\overline{\Lambda_{u}^{T}\left(E(\varphi,\lambda)\cdot\theta^{\vee}(\phi_{i})\right)}$
(11.10)
$\displaystyle=\Lambda_{u}^{T}\left(\sum_{\varphi\in\mathcal{B}_{Q,\sigma}}E(I_{Q}(\lambda,g_{i})\varphi,\lambda)\cdot\theta^{\vee}(\phi_{i})\otimes\overline{E(\varphi,\lambda)\cdot\theta^{\vee}(\phi_{i})}\right)\Lambda_{u}^{T},$

where we use $\Lambda_{u}^{T}F\Lambda_{u}^{T}$ to denote truncation in the
left and right variables of a function $F$. Moreover, we can switch the
integrals by positivity and (11.9) becomes

(11.11)
$\operatorname{vol}([\operatorname{U}^{\prime}])\int_{[\operatorname{U}^{\prime}]}\sum_{(M_{Q},\sigma)}\int_{i\mathfrak{a}_{Q}^{*}}\Lambda_{u}^{T}\left(\sum_{\varphi\in\mathcal{B}_{Q,\sigma}}E(I_{Q}(\lambda,g_{i})\varphi,\lambda)\cdot\theta^{\vee}(\phi_{i})\otimes\overline{E(\varphi,\lambda)\cdot\theta^{\vee}(\phi_{i})}\right)\Lambda_{u}^{T}(h,h)\mathrm{d}\lambda\mathrm{d}h.$

Set

$K_{g_{i},\mathrm{reg}}=\sum_{\chi\;(\operatorname{U},\operatorname{U}^{\prime},\mu^{-1})-\mathrm{reg}}K_{g_{i},\chi}$

where the sum ranges over
$(\operatorname{U},\operatorname{U}^{\prime},\mu^{-1})$-regular cuspidal data
of $\operatorname{U}$. By Lemma 4.22 $K_{g_{i},\mathrm{reg}}$ is well-defined,
belongs to $\mathcal{T}([\operatorname{U}\times\operatorname{U}])$, and we
have the spectral expansion

$\displaystyle
K_{g_{i},\mathrm{reg}}(x,y)\theta^{\vee}(x,\phi_{i})\theta(y,\phi_{i})=\sum_{(M_{Q},\sigma)}\int_{i\mathfrak{a}_{Q}^{*}}\sum_{\varphi\in\mathcal{B}_{Q,\sigma}}E(x,I_{Q}(\lambda,g_{i})\varphi,\lambda)\theta^{\vee}(x,\phi_{i})\overline{E(y,\varphi,\lambda)\theta^{\vee}(y,\phi_{i})}\mathrm{d}\lambda.$

That this decomposition coincides with the one given in Lemma 4.22 follows
from the functional equation of Eisenstein series and the
$\operatorname{U}$-regularity of the considered cuspidal data (see Remark
4.23). Moreover, the double integral
$\sum_{(M_{Q},\sigma)}\int_{i\mathfrak{a}_{Q}^{*}}$ is absolutely convergent
for fixed $x$ and $y$ by Lemma 4.22. Therefore, for fixed
$h\in[\operatorname{U}^{\prime}]$ we have

$\displaystyle\sum_{(M_{Q},\sigma)}\int_{i\mathfrak{a}_{Q}^{*}}\Lambda_{u}^{T}\left(\sum_{\varphi\in\mathcal{B}_{Q,\sigma}}E(I_{Q}(\lambda,g_{i})\varphi,\lambda)\cdot\theta^{\vee}(\phi_{i})\otimes\overline{E(\varphi,\lambda)\cdot\theta^{\vee}(\phi_{i})}\right)\Lambda_{u}^{T}(h,h)\mathrm{d}\lambda$
$\displaystyle=\Lambda_{u}^{T}\left(\sum_{(M_{Q},\sigma)}\int_{i\mathfrak{a}_{Q}^{*}}\sum_{\varphi\in\mathcal{B}_{Q,\sigma}}E(I_{Q}(\lambda,g_{i})\varphi,\lambda)\cdot\theta^{\vee}(\phi_{i})\otimes\overline{E(\varphi,\lambda)\cdot\theta^{\vee}(\phi_{i})}\mathrm{d}\lambda\right)\Lambda_{u}^{T}(h,h)$
$\displaystyle=\Lambda_{u}^{T}\left(K_{g_{i},\mathrm{reg}}\theta^{\vee}_{x}(\phi_{i})\theta_{y}(\phi_{i})\right)\Lambda_{u}^{T}(h,h),$

where we write $\theta_{x}^{\vee}(\phi_{i})$ (resp. $\theta_{y}(\phi_{i})$)
for $\theta^{\vee}(\phi_{i})$ seen as a function of the first variable (resp.
$\theta(\phi_{i})$ as a function of the second variable). Indeed, it follows
from the definition given in (8.14) and [Arthur3]*Lemma 5.1 that for any
$F\in\mathcal{T}([\operatorname{U}\times\operatorname{U}])$,
$\Lambda_{u}^{T}F\Lambda_{u}^{T}F(h,h)$ is a finite sum of left-translates of
constant terms of $F$ (independent of $F$) evaluated at $h$. These constant
terms are integrals over compact subsets and therefore commute with the
absolutely convergent double integral
$\sum_{(M_{Q},\sigma)}\int_{i\mathfrak{a}_{Q}^{*}}$. This implies that

$\eqref{eq:Selberg_before_continuity}\leq\operatorname{vol}([\operatorname{U}^{\prime}])\int_{[\operatorname{U}^{\prime}]}\Lambda_{u}^{T}\left(K_{g_{i},\mathrm{reg}}\theta^{\vee}_{x}(\phi_{i})\theta_{y}(\phi_{i})\right)\Lambda_{u}^{T}(h,h)\mathrm{d}h,$

and this expression is finite by Proposition 8.7. Therefore, (11.7) is also
finite and this concludes the proof of the claim.

It remains to obtain uniform bounds on the $\phi_{i}$. As the $g_{i}$ depend
only on $f$, we see that our bound is uniform in $\phi_{1}$ and $\phi_{2}$, in
the sense that there exists a continuous semi-norm $\lVert.\rVert_{L^{\vee}}$
on $\mathcal{S}(L^{\vee}(\mathbb{A}))$ such that for every $f$

$\sup_{\begin{subarray}{c}\phi_{i}\in\mathcal{S}(L^{\vee}(\mathbb{A}))\\\
\lVert\phi_{i}\rVert_{L^{\vee}}\leq
1\end{subarray}}\sum_{(M_{Q},\sigma)}\int_{i\mathfrak{a}_{Q}^{*}}\left|J_{Q,\sigma}(\lambda,f\otimes\phi_{1}\otimes\phi_{2})\right|\mathrm{d}\lambda<\infty.$

The bound (11.6) now follows from the uniform boundedness principle. ∎

As a byproduct of the proof, we obtain the following lemma.

######  Lemma 11.4.

Let $\chi\in\mathfrak{X}(\operatorname{U})$ be a
$(\operatorname{U},\operatorname{U}^{\prime},\mu^{-1})$-regular cuspidal datum
represented by $(M_{Q},\sigma)$. Let
$f_{+}\in\mathcal{S}(\operatorname{U}_{+}(\mathbb{A}))$. Then for $T$
sufficiently positive we have

(11.12)
$\int_{i\mathfrak{a}_{Q}^{*}}J_{Q,\sigma}(\lambda,f_{+})\mathrm{d}\lambda=\int_{[\operatorname{U}^{\prime}]\times[\operatorname{U}^{\prime}]}\Lambda_{u}^{T}\left(K_{f_{+},\chi}\right)\Lambda_{u}^{T}(x,y)\mathrm{d}x\mathrm{d}y.$

###### Proof.

As both sides are continuous in $f_{+}$ by Proposition 11.3 and Proposition
8.7, we may assume that $f_{+}=f\otimes\phi_{1}\otimes\phi_{2}$. By definition
(see (8.7))

$K_{f_{+},\chi}(x,y)=K_{f,\chi}(x,y)\theta^{\vee}(x,\phi_{1})\theta(y,\phi_{2}).$

By Proposition 11.1, the finiteness of (11.8), the dominated convergence
theorem and Proposition 8.7, we have

$\displaystyle\int_{i\mathfrak{a}_{Q}^{*}}J_{Q,\sigma}(\lambda,f_{+})\mathrm{d}\lambda$
$\displaystyle=\int_{i\mathfrak{a}_{Q}^{*}}\int_{[\operatorname{U}^{\prime}]\times[\operatorname{U}^{\prime}]}\sum_{\varphi\in\mathcal{B}_{Q,\sigma}}\Lambda_{u}^{T}\left(E(I_{Q}(\lambda,f)\varphi,\lambda)\theta^{\vee}(\phi_{1})\right)(x)\overline{\Lambda_{u}^{T}\left(E(\varphi,\lambda)\theta^{\vee}(\overline{\phi_{2}})\right)}(y)\mathrm{d}x\mathrm{d}y\mathrm{d}\lambda$
(11.13)
$\displaystyle=\int_{[\operatorname{U}^{\prime}]\times[\operatorname{U}^{\prime}]}\Lambda_{u}^{T}\left(\int_{i\mathfrak{a}_{Q}^{*}}\sum_{\varphi\in\mathcal{B}_{Q,\sigma}}E(I_{Q}(\lambda,f)\varphi,\lambda)\theta^{\vee}(\phi_{1})\overline{E(\varphi,\lambda)\theta^{\vee}(\overline{\phi_{2}})}\mathrm{d}\lambda\right)\Lambda_{u}^{T}(x,y)\mathrm{d}x\mathrm{d}y$

By Lemma 4.22 and the $\operatorname{U}$-regularity of $\chi$ (see Remark
4.23), we have

$\eqref{eq:intermediate_limit}=\int_{[\operatorname{U}^{\prime}]\times[\operatorname{U}^{\prime}]}\Lambda_{u}^{T}\left(K_{f_{+},\chi}\right)\Lambda_{u}^{T}(x,y)\mathrm{d}x\mathrm{d}y.$

This concludes. ∎

####  11.5. Spectral expansion of $J_{\chi}$

We now compute the spectral term $J_{\chi}$ from Section 8 in term of the
relative character $J_{Q,\sigma}$.

Let $\chi\in\mathfrak{X}(\operatorname{U})$ be a cuspidal datum. Recall that
we defined a modified kernel function $K_{f_{+},\chi}^{T}$ in Subsection 8.3,
and that in Theorem 8.6 (2) we proved that the purely polynomial term of the
exponential-polynomial

$J^{T}_{\chi}(f_{+})=\int_{[\operatorname{U}^{\prime}]\times[\operatorname{U}^{\prime}]}K_{f_{+},\chi}^{T}(x,y)\mathrm{d}x\mathrm{d}y,$

is a constant which we denoted by $J_{\chi}(f_{+})$. The map $f_{+}\mapsto
J_{\chi}(f_{+})$ is a continuous linear form on
$\mathcal{S}(\operatorname{U}_{+}(\mathbb{A}))$.

######  Theorem 11.5.

Let $\chi\in\mathfrak{X}(\operatorname{U})$ be a
$(\operatorname{U},\operatorname{U}^{\prime},\mu^{-1})$-regular cuspidal
datum, represented by $(M_{Q},\sigma)$. Then for all
$f\in\mathcal{S}(\operatorname{U}(\mathbb{A}))$,
$\phi_{1},\phi_{2}\in\mathcal{S}(L^{\vee}(\mathbb{A}))$, we have

(11.14)
$J_{\chi}(f\otimes\phi_{1}\otimes\phi_{2})=\int_{i\mathfrak{a}^{*}_{Q}}J_{Q,\sigma}(\lambda,f\otimes\phi_{1}\otimes\phi_{2})\mathrm{d}\lambda.$

###### Proof.

Write $f_{+}=f\otimes\phi_{1}\otimes\phi_{2}$. By Lemma 11.4, for $T_{1}$ and
$T_{2}$ positive enough, we have

$\int_{[\operatorname{U}^{\prime}]\times[\operatorname{U}^{\prime}]}(\Lambda^{T_{1}}_{u}K_{f_{+},\chi}\Lambda^{T_{2}}_{u})(x,y)\mathrm{d}x\mathrm{d}y=\int_{i\mathfrak{a}_{Q}^{*}}J_{Q,\sigma}(\lambda,f_{+})\mathrm{d}\lambda.$

As the RHS is independent of $T_{1}$ and $T_{2}$, we conclude that so is the
LHS. By Proposition 8.7 (2), we have

$\lim_{d(T_{1})\to\infty}\int_{[\operatorname{U}^{\prime}]\times[\operatorname{U}^{\prime}]}(\Lambda^{T_{1}}_{u}K_{f_{+},\chi}\Lambda^{T_{2}}_{u})(x,y)\mathrm{d}x\mathrm{d}y=\int_{[\operatorname{U}^{\prime}]\times[\operatorname{U}^{\prime}]}K_{f_{+},\chi}\Lambda^{T_{2}}_{u}(x,y)\mathrm{d}x\mathrm{d}y$

The theorem now follows from the second assertion of Proposition 8.8. ∎

##  Part IV Comparison of the relative trace formulae

###  12\. Infinitesimal geometric distributions

####  12.1. General linear groups: simplification

Let us retain the notation from Section 7. Recall that we have geometric
distributions as follows. Take $f^{+}\in\mathcal{S}(G^{+}(\mathbb{A}))$ and
$\alpha\in\mathcal{A}(F)$. For $T\in\mathfrak{a}_{n+1}$ sufficiently positive,
we have defined

$i_{\alpha}^{T}(f^{+})=\int_{[H]}\int_{[G^{\prime}]}k_{f^{+},\alpha}^{T}(h,g^{\prime})\eta_{G^{\prime}}(g^{\prime})\mathrm{d}g^{\prime}\mathrm{d}h,$

where $\eta_{G^{\prime}}(g_{1},g_{2})=\eta(\det g_{1})^{n}\eta(\det
g_{2})^{n+1}$ (cf. (6.1)). The goal of this subsection is to simplify it.

Let us introduce the following additional spaces and group actions.

  * •

We put

$S_{n}=\\{g\in G_{n}\mid gg^{\mathsf{c},-1}=1\\},$

which is a closed subvariety of $G_{n}$ over $F$. The group $G_{n}^{\prime}$
acts on $S_{n}$ from the right by the usual conjugation, and we have an
isomorphism of $G_{n}^{\prime}$-varieties $S_{n}\simeq G_{n}/G_{n}^{\prime}$.
Let

$\nu:G_{n}\to S_{n},\quad\nu(g)=gg^{\mathsf{c},-1}$

be the natural projection.

  * •

Put $X=S_{n}\times L^{\vee,-}\times L^{-}$, on which $G_{n}^{\prime}$ acts
from the right by

$(\gamma,w,v)\cdot g^{\prime}=(g^{\prime-1}\gamma
g^{\prime},wg^{\prime},g^{\prime-1}v).$

Put $G_{n}^{+}=G_{n}\times L^{\vee,-}\times L^{-}$, on which $G^{\prime}$ acts
on the right by

$(g,w,v)\cdot(g_{1}^{\prime},g_{2}^{\prime})=(g_{1}^{\prime-1}gg_{2}^{\prime},wg_{1}^{\prime},g_{1}^{\prime-1}v).$

We extend the map $\nu$ to a map $G_{n}^{+}\to X$ by setting
$\nu(g,u,v)=(\nu(g),u,v)$. By composing this with the map $G_{n}\times
G_{n}\to G_{n}$, $(g_{1},g_{2})\mapsto g_{1}^{-1}g_{2}$, we further obtain to
a map $G^{+}\to X$. Note that $G^{+}(\mathbb{A})\to X(\mathbb{A})$ is
surjective.

  * •

Note that the quotient morphism $G/H\to G_{n}$, $(g_{1},g_{2})\mapsto
g_{1}^{-1}g_{2}$ and $\nu$ are both principal homogeneous spaces, the
categorical quotients $G_{n}^{+}//G^{\prime}$, $X//G_{n}^{\prime}$ and
$G^{+}//(H\times G^{\prime})$ are canonically identified, and are all denoted
by $\mathcal{A}$. The categorical quotient $q:G^{+}\to\mathcal{A}$ factors
through the map $\nu:G^{+}\to X$, and we denote again by $q:X\to\mathcal{A}$
the induced map. If $\alpha\in\mathcal{A}(F)$, we let $X_{\alpha}$ be the
inverse image of $\alpha$ in $X$ (as a closed subscheme).

  * •

Recall that $\mathcal{F}$ denotes the set of standard D-parabolic subgroups of
$J_{n}$, c.f. Subsection 5.1. Let $P=MN\in\mathcal{F}$, write
$P_{n}=M_{n}N_{n}$. We put

$M_{n}^{+}=M_{n}\times M_{L^{\vee}}^{-}\times M_{L}^{-},\quad
N_{n}^{+}=N_{n}\times N_{L^{\vee}}^{\vee,-}\times N_{L}^{-}.$

The right action of $G_{n}^{\prime}$ on $G_{n}^{+}$ restricts to an action of
$M_{n}^{\prime}$ on $M_{n}^{+}$.

  * •

Composing the embedding $M_{n}^{+}\subset G_{n}^{+}$ with the map
$G_{n}^{+}\to\mathcal{A}$ we obtain a map $M_{n}^{+}\to\mathcal{A}$, and for
any $\alpha\in\mathcal{A}(F)$ we denote by $M^{+}_{n,\alpha}$ the inverse
image of $\alpha$ in $M^{+}_{n}$ (as a closed subscheme).

For $\varphi\in\mathcal{S}(X(\mathbb{A}))$, define a function
$k_{\varphi,P,\alpha}$ on $[G_{n}^{\prime}]_{P_{n}^{\prime}}$ by

$k_{\varphi,P,\alpha}(h)=\sum_{m\in M_{n}^{+}(F)\cap
X_{\alpha}(F)}\int_{N_{n}^{+}(\mathbb{A})/N^{\prime}_{n}(\mathbb{A})}\varphi(h^{-1}\nu(xn)h)\mathrm{d}n,$

where $x$ is any element in $M_{n}^{+}$ such that $\nu(x)=m$, and
$N_{n}^{+}/N_{n}^{\prime}$ stands for $N_{n}/N_{n}^{\prime}\times
N_{P,L^{\vee}}^{\vee,-}\times N_{P,L}^{-}$. The same argument as Lemma 7.1
gives that

$\sum_{\alpha\in\mathcal{A}(F)}\sum_{m\in
M_{n,\alpha}^{+}(F)}\int_{N_{n}^{+}(\mathbb{A})/N_{n}^{\prime}(\mathbb{A})}\lvert\varphi(h^{-1}\nu(xn)h)\rvert\mathrm{d}n$

is convergent.

Let us define an integral transform

$\mathcal{S}(G^{+}(\mathbb{A}))\to\mathcal{S}(X(\mathbb{A})),\quad
f^{+}\mapsto\varphi_{f^{+}}$

as follows. For any $(\gamma,w,v)\in X(\mathbb{A})$ we put

(12.1) $\varphi_{f^{+}}(\gamma,w,v)=\left\\{\begin{aligned}
&\int_{G_{n}(\mathbb{A})}\int_{G_{n}^{\prime}(\mathbb{A})}f^{+}((h^{-1},h^{-1}xg_{n}^{\prime}),w,v)\mathrm{d}h\mathrm{d}g_{n}^{\prime},&&\text{$n$
odd};\\\
&\int_{G_{n}(\mathbb{A})}\int_{G_{n}^{\prime}(\mathbb{A})}f^{+}((h^{-1},h^{-1}xg_{n}^{\prime}),w,v)\mu(\det
xg_{n}^{\prime})\mathrm{d}h\mathrm{d}g_{n}^{\prime},&&\text{$n$
even}.\end{aligned}\right.$

where $x\in G_{n}(\mathbb{A})$ is any element such that $\nu(x)=\gamma$. We
note that because the map $G^{+}\to X$ is a smooth morphism, the integral
transform $f^{+}\mapsto\varphi_{f^{+}}$ is surjective, cf. [AG]*Theorem B.2.4.
If $\mathtt{S}$ is a finite set of places, and
$f^{+}\in\mathcal{S}(G^{+}(F_{\mathtt{S}}))$, then we define
$\varphi_{f^{+}}\in\mathcal{S}(X(F_{\mathtt{S}}))$ in the same way,
integrating over $F_{\mathtt{S}}$-points of the groups.

######  Lemma 12.1.

For all $\alpha\in\mathcal{A}(F)$ and
$f^{+}\in\mathcal{S}(G^{+}(\mathbb{A}))$, the expression

$\int_{[H]}\int_{[G_{n}^{\prime}]}\sum_{\begin{subarray}{c}\gamma\in
P_{H}(F)\backslash H(F)\\\ \delta_{2}\in P_{2}^{\prime}(F)\backslash
G_{n}^{\prime}(F)\end{subarray}}\sum_{m\in
M_{\alpha}^{+}(F)}\int_{N^{+}(\mathbb{A})}\left|f^{+}(mn\cdot(\gamma
h,\delta_{2}g^{\prime}))\right|\mathrm{d}n\mathrm{d}g_{2}^{\prime}\mathrm{d}h$

is convergent and we have

(12.2) $\int_{[H]}\int_{[G_{n}^{\prime}]}\sum_{\begin{subarray}{c}\gamma\in
P_{H}(F)\backslash H(F)\\\ \delta_{2}\in P_{2}^{\prime}(F)\backslash
G_{n}^{\prime}(F)\end{subarray}}k_{f^{+},P,\alpha}(\gamma
h,\delta_{2}g^{\prime})\eta_{G^{\prime}}(g^{\prime})\mathrm{d}g_{2}^{\prime}\mathrm{d}h=k_{\varphi_{f^{+}},P,\alpha}(g_{1}^{\prime})\eta(g_{1}^{\prime}).$

Here we recall that $g^{\prime}=(g_{1}^{\prime},g_{2}^{\prime})\in
G^{\prime}(\mathbb{A})$, $g_{1}^{\prime},g_{2}^{\prime}\in
G_{n}^{\prime}(\mathbb{A})$, and $\delta_{2}g^{\prime}$ stands for the element
$(g_{1}^{\prime},\delta_{2}g_{2}^{\prime})\in G^{\prime}(\mathbb{A})$.

###### Proof.

We will prove (12.2). The absolute convergence can be obtained by the same
computation, with only obvious minor modifications.

We first unfold the sum over $\gamma$ and $\delta$ to conclude that (12.2)
equals

$\int_{[H]_{P_{H}}}\int_{[G_{n}^{\prime}]_{P_{n}^{\prime}}}k_{f^{+},P,\alpha}(h,g^{\prime})\eta_{G^{\prime}}(g^{\prime})\mathrm{d}g_{2}^{\prime}\mathrm{d}h.$

Recall that we have fixed maximal compact subgroups $K_{H}$ and