# Weighted Bergman kernels for nearly holomorphic functions on bounded
symmetric domains

Miroslav Engliš  Mathematics Institute, Silesian University in Opava, Na
Rybníčku 1, 74601 Opava, Czech Republic and Mathematics Institute, Žitná 25,
11567 Prague 1, Czech Republic<EMAIL_ADDRESS>,  El-Hassan Youssfi  Aix-
Marseille Université, I2M UMR CNRS 7373, 39 Rue F-Juliot-Curie, 13453
Marseille Cedex 13, France<EMAIL_ADDRESS>and  Genkai Zhang
Mathematical Sciences, Chalmers University of Technology, and Mathematical
Sciences, Göteborg University, SE-412 96 Göteborg, Sweden<EMAIL_ADDRESS>
###### Abstract.

We identify the standard weighted Bergman kernels of spaces of nearly
holomorphic functions, in the sense of Shimura, on bounded symmetric domains.
This also yields a description of the analogous kernels for spaces of
“invariantly-polyanalytic” functions — a generalization of the ordinary
polyanalytic functions on the ball which seems to be the most appropriate one
from the point of view of holomorphic invariance. In both cases, the kernels
turn out to be given by certain spherical functions, or equivalently Heckman-
Opdam hypergeometric functions, and a conjecture relating some of these to a
Faraut-Koranyi hypergeometric function is formulated based on the study of low
rank situations. Finally, analogous results are established also for compact
Hermitian symmetric spaces, where explicit formulas in terms of multivariable
Jacobi polynomials are given.

###### Key words and phrases:

Nearly holomorphic functions, polyanalytic functions, Bergman kernel, bounded
symmetric domain

###### 1991 Mathematics Subject Classification:

Primary 32M15; Secondary 46E22, 53C35

Research of M. Engliš supported by GAČR grant no. 21-27941S and RVO funding
for IČO 67985840. Research of G. Zhang supported by Swedish Research Council
(VR) grants no. 2018-03402 and 2022-02861.

##  1\. Introduction

Let $\Omega$ be an irreducible bounded symmetric domain in ${\mathbf{C}}^{d}$,
$d\geq 1$, in its Harish-Chandra realization, and denote by $p$ its genus and
by $h(z,w)$ the associated Jordan triple determinant, which is a holomorphic
polynomial in $z$ and $\overline{w}$ on ${\mathbf{C}}^{d}$. The standard
weighted Bergman spaces on $\Omega$ are the spaces

(1) $A_{\nu}(\Omega)\equiv
A_{\nu}:=L^{2}(\Omega,d\mu_{\nu})\cap\mathcal{O}(\Omega)$

of all holomorphic functions on $\Omega$ square-integrable with respect to the
measure

(2) $d\mu_{\nu}(z):=h(z,z)^{\nu-p}\,dz,$

where $dz$ stands for the Lebesgue measure. It is well known that $A_{\nu}$ is
nontrivial if and only if $\nu>p-1$, and in that case $A_{\nu}$ possesses a
reproducing kernel — the _weighted Bergman kernel_ — given by

(3) $K_{\nu}(z,w)=c_{\nu}h(z,w)^{-\nu},$

where $c_{\nu}=1/\mu_{\nu}(\Omega)$ is a constant which can be evaluated
explicitly.

For any $\phi\in\operatorname{Aut}(\Omega)$, the group of all biholomorphic
self-maps of $\Omega$, the Jordan triple determinant satisfies the
transformation rule

(4) $h(\phi z,\phi w)=\frac{h(a,a)h(z,w)}{h(z,a)h(a,w)},\qquad a=\phi^{-1}0.$

(We will mostly write just $\phi z$ instead of $\phi(z)$.) It follows that the
Riemannian metric

(5) $ds^{2}=-\sum_{j,k=1}^{d}\frac{\partial^{2}\log h(z,z)}{\partial
z_{j}\partial\overline{z}_{k}}$

is invariant under $\operatorname{Aut}(\Omega)$. Recall now that, quite
generally, for an arbitrary Kähler manifold $\Omega$ with Kähler metric
$ds^{2}=\sum_{j,k}g_{j\overline{k}}\,dz_{j}\,d\overline{z}_{k}$, the invariant
Cauchy-Riemann operator $\overline{D}$, introduced by Peetre [PZ], is the map
from functions into holomorphic vector fields defined by

$\overline{D}f=(\overline{D}f)^{j}\frac{\partial}{\partial
z_{j}},\qquad(\overline{D}f)^{j}=g^{\overline{l}j}\overline{\partial}_{l}f,$

where we have started to employ the Einstein summation convention, and also to
write for brevity
$\overline{\partial}_{l}:=\partial/\partial\overline{z}_{l}$; namely, it is
the $\overline{\partial}$ operator combined with the Riesz lemma identifying
(0,1)-forms with holomorphic vector fields. Here $g^{\overline{k}j}$ is the
inverse matrix to $g_{j\overline{k}}$. One can iterate this construction and
set, for $m=1,2,\dots$,

$(\overline{D}{}^{m}f)^{k_{m}\dots
k_{1}}=g^{\overline{l}_{m}k_{m}}\overline{\partial}_{l_{m}}\dots
g^{\overline{l}_{2}k_{2}}\overline{\partial}_{l_{2}}g^{\overline{l}_{1}k_{1}}\overline{\partial}_{l_{1}}f.$

It turns out that the tensor field $(\overline{D}{}^{m}f)^{k_{m}\dots k_{1}}$
is symmetric in the indices $k_{1},\dots,k_{m}$ [PZ], and in fact coincides
with the contravariant derivative $f^{/k_{1}\dots k_{m}}$ with respect to the
Hermitian connection [En]. The $m$-th Cauchy-Riemann space $\mathcal{N}^{m}$
[EZ], or the space of _nearly holomorphic functions of order $m$_, is, by
definition, the kernel of $\overline{D}{}^{m}$:

$\mathcal{N}^{m}(\Omega)\equiv\mathcal{N}^{m}:=\\{f\in
C^{\infty}(\Omega):\overline{D}{}^{m}f=0\text{ on }\Omega\\}.$

An alternative definition is due to Shimura [Shi]: $\mathcal{N}^{m}$ is the
vector space of all functions on $\Omega$ that can (locally) be written as
polynomials of degree $<m$ in the derivatives $\partial_{j}\Psi$, with
holomorphic coefficients, where $\Psi$ is a (local) potential for the Kähler
metric, i.e. $g_{j\overline{k}}=\overline{\partial}_{k}\partial_{j}\Psi$.
(This space does not depend on the choice of the local potential $\Psi$.) See
e.g. Proposition 7 in [EZ] for a proof of the equivalence of these two
definitions.

The above construction applies, in particular, to our bounded symmetric domain
$\Omega$ with the invariant metric (5), possessing a global Kähler potential
$\Psi(z)=-\log h(z,z)$. In analogy with (1), we can consider the _weighted
Bergman spaces of nearly holomorphic functions_

(6) $\mathcal{N}^{m}_{\nu}:=L^{2}(\Omega,d\mu_{\nu})\cap\mathcal{N}^{m}.$

Of course, if $m=1$ then $\mathcal{N}^{1}=\mathcal{O}(\Omega)$ and
$\mathcal{N}^{1}_{\nu}=A_{\nu}$ for any $\nu$.

For the simplest bounded symmetric domain $\Omega={\mathbf{B}}^{d}$, the unit
ball of ${\mathbf{C}}^{d}$, $d\geq 1$, the Jordan triple determinant is given
simply by $h(z,w)=1-\langle z,w\rangle$, so that
$\Psi_{j}=\frac{\overline{z}_{j}}{1-|z|^{2}}$. Nearly holomorphic functions of
order $m$ on ${\mathbf{B}}^{d}$ are thus precisely the polynomials of degree
$\leq m-1$ in $(1-|z|^{2})^{-1}\overline{z}$, with holomorphic coefficients.
In other words,

(7)
$\mathcal{N}^{m}({\mathbf{B}}^{d})=(1-|z|^{2})^{1-m}\mathcal{P}^{m}({\mathbf{B}}^{d}),$

where $\mathcal{P}^{m}({\mathbf{B}}^{d})$ consists, by definition, of all
linear combinations, with holomorphic coefficients, of
$(1-|z|^{2})^{m-1-|\alpha|}\overline{z}^{\alpha}$, where
$\alpha=(\alpha_{1},\dots,\alpha_{d})$ is a multiindex of length
$|\alpha|:=\alpha_{1}+\dots+\alpha_{d}<m$; that is, by a simple check,
$\mathcal{P}^{m}({\mathbf{B}}^{d})$ consists of all polynomials of degree
$\leq m-1$ in $\overline{z}$, with holomorphic coefficients. (Indeed, in one
direction, $(1-|z|^{2})^{m-1-|\alpha|}\overline{z}^{\alpha}$ is clearly a
polynomial in $\overline{z}$ of degree $m-1$ with holomorphic coefficients;
while in the other direction,

$\displaystyle\overline{z}^{\alpha}$
$\displaystyle=(|z|^{2}+(1-|z|^{2}))^{m-1-|\alpha|}\overline{z}^{\alpha}$
$\displaystyle=\sum_{j=0}^{m-1-|\alpha|}\binom{m-1-|\alpha|}{j}|z|^{2j}(1-|z|^{2})^{m-1-|\alpha|-j}\overline{z}^{\alpha}$
$\displaystyle=\sum_{|\beta|<m-1-|\alpha|}\binom{m-1-|\alpha|}{\beta}z^{\beta}(1-|z|^{2})^{m-1-|\alpha+\beta|}\overline{z}^{\alpha+\beta}$

is a linear combination of $(1-|z|^{2})^{m-1-|\gamma|}\overline{z}^{\gamma}$
with holomorphic coefficients.) The space $\mathcal{P}^{m}({\mathbf{B}}^{d})$
is thus nothing else than the well-known space of _$m$ -analytic_ functions on
the ball, as studied by many authors. The reproducing kernel of the space

$L^{2}({\mathbf{B}}^{d},(1-|z|^{2})^{s}\,dz)\cap\mathcal{P}^{q}({\mathbf{B}}^{d}),\qquad
s>-1,$

was recently found by the second author [You] to be

(8)
$P^{q}_{s+d+1}(z,w):=\frac{\Gamma(q+s+d)}{\pi^{d}\Gamma(q+s)}\frac{(1-\langle
w,z\rangle)^{q-1}}{(1-\langle
z,w\rangle)^{q+s+d}}P^{(d,s)}_{q-1}(1-2|\phi_{z}w|^{2}),$

where $P^{(d,s)}_{n}$ denotes the Jacobi polynomial of degree $n$ with
parameters $d,s$, and $\phi_{z}\in\operatorname{Aut}({\mathbf{B}}^{d})$ is the
biholomorphic self-map of ${\mathbf{B}}^{d}$ interchanging $z$ and the origin.
Returning to our general bounded symmetric domain $\Omega$, we are thus led to
define, by analogy with (7), the space of _invariantly polyanalytic functions
of order $m$_ on $\Omega$ as

(9)
$\mathcal{P}^{m}(\Omega)\equiv\mathcal{P}^{m}:=h(z,z)^{m-1}\mathcal{N}^{m},$

and consider the corresponding weighted Bergman spaces

(10) $\mathcal{P}^{m}_{\nu}:=L^{2}(\Omega,d\mu_{\nu})\cap\mathcal{P}^{m}.$

Our aim in this paper is to find the reproducing kernels $N^{m}_{\nu}$ and
$P^{m}_{\nu}$ of the spaces $\mathcal{N}^{m}_{\nu}$ and
$\mathcal{P}^{m}_{\nu}$, respectively, thus generalizing the formulas (3)
(which corresponds to $m=1$) and (8) (which corresponds to
$\Omega={\mathbf{B}}^{d}$).

On an abstract level, the answer is given by the group representation theory,
more specifically, by the Plancherel formula for certain representations of
the identity connected component $G$ of the automorphism group
$\operatorname{Aut}(\Omega)$ of $\Omega$. Namely, from the fact that
$\overline{D}^{m}f$ is a tensor, it follows that the action of $G$ by
composition preserves the space $\mathcal{N}^{m}$; in other words,

$f\longmapsto f\circ\phi^{-1},\qquad f\in\mathcal{N}^{m},\;\phi\in G,$

is a representation of $G$ on $\mathcal{N}^{m}$. In combination with the
transformation rule for $\mu_{\nu}$,

(11) $d\mu_{\nu}(\phi
z)=\Big{|}\frac{h(a,a)^{\nu/2}}{h(z,a)^{\nu}}\Big{|}^{2}\,d\mu_{\nu}(z),\qquad
a=\phi^{-1}0,\;\phi\in G,$

which follows from (4), this implies that

(12) $f\longmapsto\frac{h(a,a)^{\nu/2}}{h(z,a)^{\nu}}\,f\circ\phi^{-1},\qquad
a=\phi 0,\;\phi\in G,$

is a projective unitary representation of $G$ on $\mathcal{N}^{m}_{\nu}$. It
is now a result of the third author [Zh3] that for each $m=1,2,\dots$,
$\mathcal{N}^{m}_{\nu}$ comes as an orthogonal direct sum of irreducible
components which can be identified with certain so-called relative discrete
series representations of $G$. Finally, a general Plancherel formula of
Shimeno [Shm], applied to these representations, implies that the reproducing
kernel at the origin of each of these irreducible components must be a
constant multiple of $\phi_{\lambda,\ell}$, the spherical function of $G$ with
parameter $\ell$ (describing the representation, actually $\ell=\nu$) and
weight $\lambda$ (uniquely associated to each of the irreducible components).
In this way, the reproducing kernel $N^{m}_{\nu}$ is thus expressed as a
finite sum of terms involving spherical functions. (For the particular case of
$\Omega={\mathbf{B}}^{d}$, this expression was obtained in [Zh1].)

Recoursing to the available theory of multivariable special functions (see
e.g. Anker [Ank]), the spherical functions $\phi_{\lambda,\ell}$ can also be
expressed as Heckman-Opdam hypergeometric functions, or, if one wishes, as
multivariable Jacobi polynomials of Debiard [De1] (and many other authors).
For instance, the result for ${\mathbf{B}}^{d}$ from [Zh1] just mentioned
reads

$N^{m}_{s+d+1}(z,w)=k^{m}_{s+d+1}(|\phi_{w}z|^{2})$

with

(13)
$k^{m}_{s+d+1}(t):=\sum_{l=0}^{m-1}c_{l}(s){}_{2}\\!F_{1}\Big{(}\begin{matrix}-l,l-s-1\\\
d\end{matrix}\Big{|}\frac{t}{t-1}\Big{)},$

where

$c_{l}(s)=\frac{(s-2l+1)\Gamma(s+d+1-l)d}{\pi^{d}l!\Gamma(s-l+2)},$

and ${}_{2}\\!F_{1}$ is the ordinary (Gauss) hypergeometric function. On the
other hand, from (8) one can express $P^{m}_{s+d+1}$ and $N^{m}_{s+d+1}$ in
terms of a single Jacobi polynomial $P^{(d,s)}_{m-1}$. Comparing both
expressions leads (after working out the details) to the equality

(14)
$(1-t)^{q-1}\sum_{l=0}^{q-1}c_{l}(s+2q-2){}_{2}\\!F_{1}\Big{(}\begin{matrix}-l,l-s-2q+1\\\
d\end{matrix}\Big{|}\frac{t}{t-1}\Big{)}=\frac{\Gamma(q+s+d)}{\pi^{d}\Gamma(q+s)}P^{(d,s)}_{q-1}(1-2t).$

It is amusing to prove this (valid) formula directly (cf. Lemma 33 below);
note that

(15)
$P^{(d,s)}_{n}(1-2t)=\binom{n+d}{d}{}_{2}\\!F_{1}\Big{(}\begin{matrix}-n,n+1+s+d\\\
d+1\end{matrix}\Big{|}t\Big{)}.$

Performing explicit computer calculations for rank 2 and rank 3 bounded
symmetric domains indicates that, analogously to the rank 1 situation just
described, even for general bounded symmetric domains the kernels
$N^{m}_{\nu}$ and $P^{m}_{\nu}$ can in some cases be expressed not only as a
finite sum of terms involving Heckman-Opdam hypergeometric functions, but
actually as a constant multiple of a single special function, namely a
hypergeometric function of Faraut and Koranyi [FK] with certain parameters. We
offer a conjecture to this effect, together with some consequences that would
follow; the latter include relations among the two kinds of hypergeometric
functions, as well as a generalization of a theorem of Helgason [He, Theorem
V.4.5] describing, in effect, the reproducing kernel for a certain space of
radial functions on the complex projective space ${\mathbf{C}}P^{d}$.

The paper is organized as follows. Section 2 contains the necessary background
material on bounded symmetric domains. Section 3 lists some elementary facts
about the kernels $N^{m}_{\nu}$ and $P^{m}_{\nu}$ and discusses radial nearly-
holomorphic functions, which are relevant for the sequel. The expressions for
$N^{m}_{\nu}$ in terms of spherical functions and Heckman-Opdam hypergeometric
functions are presented in Section 4. Section 5 describes the computations for
particular bounded symmetric domains and the resulting conjectures mentioned
above. The final section, Section 6, briefly treats also the dual case of
compact Hermitian symmetric spaces.

##  2\. Prerequisites on bounded symmetric domains

Throughout the rest of this paper, $\Omega$ will be an irreducible bounded
symmetric domain in ${\mathbf{C}}^{d}$ in its Harish-Chandra realization (i.e.
a Cartan domain). We denote by $G$ the identity connected component of the
group $\operatorname{Aut}(\Omega)$ of all biholomorphic self-maps of $\Omega$,
and by $K$ the stabilizer in $G$ of the origin $0\in\Omega$. Then $K$ consists
precisely of the unitary maps on ${\mathbf{C}}^{d}$ that preserve $\Omega$,
and $\Omega$ is isomorphic to the coset space $G/K$. We further denote by
$r,a,b$ and $p$ the rank, the characteristic multiplicities and the genus of
$\Omega$, respectively, so that

(16) $p=(r-1)a+b+2,\qquad d=\frac{r(r-1)}{2}a+rb+r.$

If $b=0$, $\Omega$ is said to be of _tube type_.

Irreducible bounded symmetric domains were completely classified by E. Cartan.
There are four infinite series of such domains plus two exceptional domains in
${\mathbf{C}}^{16}$ and ${\mathbf{C}}^{27}$. For future reference, we include
a table with brief descriptions of these domains and with the corresponding
values of $r,a,b,p$ and $d$. The symbol $\mathbf{O}$ stands for the division
algebra of octonions.

Domain | Description |
---|---|---
$I_{mn}$ |  $Z\in{\mathbf{C}}^{m\times n}$: $\|Z\|_{{\mathbf{C}}^{n}\to{\mathbf{C}}^{m}}<1$ | $n\geq m\geq 1$
|  $r=m$, $a=2$, $b=n-m$, $p=n+m$, $d=mn$ |
$II_{n}$ |  $Z\in I_{nn}$, $Z=Z^{t}$ | $n\geq 2$
|  $r=n$, $a=1$, $b=0$, $p=n+1$, $d=\frac{1}{2}n(n+1)$ |
$III_{m}$ |  $Z\in I_{mm}$, $Z=-Z^{t}$ | $m\geq 5$
|  $r=[\frac{m}{2}]$, $a=4$, $b=2(m-2r)$, $p=2m-2$, $d=\frac{1}{2}m(m-1)$ |
$IV_{n}$ |  $Z\in{\mathbf{C}}^{n\times 1}$, $|Z^{t}Z|<1$, $1+|Z^{t}Z|^{2}-2Z^{*}Z>0$ | $n\geq 5$
|  $r=2$, $a=n-2$, $b=0$, $p=d=n$ |
$V$ |  $Z\in\mathbf{O}^{1\times 2}$, $\|Z\|<1$ |
|  $r=2$, $a=6$, $b=4$, $p=12$, $d=16$ |
$V\\!I$ |  $Z\in\mathbf{O}^{3\times 3}$, $Z=Z^{*}$, $\|Z\|<1$ |
|  $r=3$, $a=8$, $b=0$, $p=18$, $d=27$ |

The unit balls ${\mathbf{B}}^{d}=I_{1d}$ are the only bounded symmetric
domains of rank 1, and the only bounded symmetric domain with smooth boundary.

For $x\in\Omega$, $\phi_{x}$ will denote the (unique) geodesic symmetry which
interchanges $x$ and the origin, i.e.

(17) $\phi_{x}\circ\phi_{x}=\text{id},\;\phi_{x}(0)=x,\;\phi_{x}(x)=0,$

and $\phi_{x}$ has only an isolated fixed-point. (In fact, $\phi_{x}$ has only
one fixed point, namely the geodesic mid-point between $0$ and $x$.) Note that
from the definition of $K$ it is immediate that any $\phi\in G$ is of the form
$\phi=\phi_{x}k$, where $k\in K$ and $x\in\Omega$. (In fact $x=\phi(0).$)

It is known that the ambient space ${\mathbf{C}}^{d}=:Z$ possesses a structure
of _Jordan-Banach $*$-triple system_ (or  _JB*-triple_ for short) for which
$\Omega$ is the open unit ball. That is, there exists a Jordan triple product

$\\{\cdot,\cdot,\cdot\\}:Z\times Z\times Z\to Z,\qquad
x,y,z\mapsto\\{x,y,z\\},$

(linear and symmetric in $x,z$ and anti-linear in $y$) such that

$\Omega=\\{z\in Z:\|\\{z,z,\cdot\\}\|<1\\}.$

Moreover, if one uses the notation, for $x,y\in Z$,

$\displaystyle D(x,y)$ $\displaystyle=\\{x,y,\cdot\\}:\;Z\to Z,$
$\displaystyle Q(x)$ $\displaystyle=\\{x,\cdot,x\\}:\;Z\to Z,$

then for every $x\in\Omega$, $D(x,x)$ is Hermitian and has nonnegative
spectrum, and $iD(x,x)$ is a triple derivation. The linear operator

(18) $B(x,y)=I-2D(x,y)+Q(x)Q(y)$

on $Z$ is called the _Bergman operator_.

Two vectors $x,y\in Z$ are said to be _orthogonal_ (in the Jordan-theoretic
sense) if $D(x,y)=0$, and a vector $v\in Z$ is called a  _tripotent_ if
$\\{v,v,v\\}=v$. For any tripotent $v$, the ambient space admits the _Peirce
decomposition_

(19) $Z=Z_{0}(v)\oplus Z_{1/2}(v)\oplus Z_{1}(v)$

into the orthogonal components

$Z_{j/2}(v):=\\{z\in Z:D(v,v)z=\frac{j}{2}z\\}.$

(The orthogonality is only with respect to the inner product in
${\mathbf{C}}^{d}$, not in the triple-product (Jordan-theoretic) sense.) Each
$Z_{j/2}(v)$ is a subtriple of $Z$, and $Z_{1}(v)$ is a JB*-algebra under the
product $x\circ y=\\{xvy\\}$, with unit $v$ and involution $z^{*}=\\{vzv\\}$.
A tripotent $v$ is called _minimal_ if $\dim Z_{1}(v)=1$. Any maximal set
$e_{1},\dots,e_{r}$ of pairwise orthogonal minimal tripotents is called a
_Jordan frame_ ; its cardinality $r$ is independent of the frame and equal to
the rank $r$ of $\Omega$. For any Jordan frame $e_{1},\dots,e_{r}$, we
similarly as above have the _joint Peirce decomposition_

(20) $Z=\bigoplus_{0\leq i\leq j\leq r}Z_{ij}$

with

(21) $Z_{ij}=\\{z\in
Z:D(e_{k},e_{k})z=\frac{\delta_{ik}+\delta_{jk}}{2}\;\forall k=1,\dots,r\\}.$

Given any Jordan frame $e_{1},\dots,e_{r}$ — which we choose and fix once and
for all from now on — any $z\in Z$ has a _polar decomposition_

(22) $z=k(t_{1}e_{1}+\dots+t_{r}e_{r})$

with $k\in K$ and $t_{1}\geq t_{2}\geq\dots\geq t_{r}\geq 0$; the numbers
$t_{1},\dots,t_{r}$, called the _singular numbers_ of $z$, are determined
uniquely, but $k$ need not be (it is if all the $t_{j}$ are distinct).
Further, $z\in\Omega$ if and only if $t_{1}<1$, $z\in\partial\Omega$ if and
only if $t_{1}=1$, and $z$ belongs to the Shilov boundary $\partial_{e}\Omega$
of $\Omega$ if and only if $t_{1}=\dots=t_{r}=1$; that is, if and only if
$z=ke$, where $e=e_{1}+\dots+e_{r}$ is a  _maximal tripotent_.

Since the Jordan triple product is invariant under $K$ (i.e.
$\\{kx,ky,kx\\}=k\\{x,y,z\\}$ $\forall k\in K$), it is immediate from (21)
that under the decomposition (20), the Bergman operator $B(z,z)$ with $z$ as
in (22) is given by

(23) $B(z,z)|_{Z_{ij}}=(1-t_{i}^{2})(1-t_{j}^{2})I|_{Z_{ij}}$

(where $t_{0}:=0$).

There exists a unique polynomial $h(x,y)$ on
${\mathbf{C}}^{d}\times{\mathbf{C}}^{d}$, holomorphic in $x$ and anti-
holomorphic in $y$, which is $K$-invariant, in the sense that

$h(kx,ky)=h(x,y)\qquad\forall k\in K,$

and satisfies

$h(z,z)=\prod_{j=1}^{r}(1-t_{j}^{2})\quad\text{ for $z$ as in (\ref{tQA})}.$

It is known that $h(x,y)$ is irreducible, of degree $r$ in $x$ as well as in
$\overline{y}$, and $h(x,0)=h(0,x)=1$ $\forall x\in{\mathbf{C}}^{d}$; also,
$h(x,y)^{p}=\det B(x,y)$. Further, the measure

(24) $h(z,z)^{\nu-p}\,dz$

is finite if and only if $\nu>p-1$, and the corresponding weighted Bergman
kernel — i.e. the reproducing kernel of the space of all holomorphic functions
on $\Omega$ square-integrable with respect to (24) — is equal to

(25) $K_{\nu}(x,y)=c_{\nu}h(x,y)^{-\nu}$

where

(26)
$c_{\nu}=\frac{\Gamma_{\Omega}(\nu)}{\pi^{d}\Gamma_{\Omega}(\nu-\frac{d}{r})}.$

Here $\Gamma_{\Omega}$ is the _Gindikin-Koecher Gamma function_

$\Gamma_{\Omega}(\nu):=\prod_{j=1}^{r}\Gamma\Big{(}\nu-\frac{j-1}{2}a\Big{)}.$

In the polar coordinates (22), the measures (24) assume the form

(27) $\int_{\Omega}f(z)\,h(z,z)^{\nu}\,dz=\quad
c_{\Omega}\int_{[0,1]^{r}}\int_{K}f(k\sum_{j=1}^{r}\sqrt{t_{j}}e_{j})\,dk\,d\mu_{b,\nu,a}(t),$

where $d\mu_{b,\nu,a}$ is the _Selberg measure_

(28)
$d\mu_{b,\nu,a}(t):=\prod_{j=1}^{r}(1-t_{j})^{\nu-p}\prod_{j=1}^{r}t_{j}^{b}\prod_{1\leq
i<j\leq r}|t_{i}-t_{j}|^{a}\,dt,$

where $dt\equiv dt_{1}\,\dots\,dt_{r}$. Here $dk$ is the normalized Haar
measure on the (compact) group $K$, and

(29)
$c_{\Omega}=\frac{\pi^{d}\Gamma(\frac{a}{2}+1)^{r}}{\Gamma_{\Omega}(\frac{ra}{2}+1)\Gamma_{\Omega}(\frac{d}{r})}.$

Let ${\mathbf{P}}$ denote the vector space of all (holomorphic) polynomials on
${\mathbf{C}}^{d}$. We endow ${\mathbf{P}}$ with the _Fock_ (or  _Fischer_)
inner product

(30) $\displaystyle\langle f,g\rangle_{F}:$
$\displaystyle=\pi^{-d}\int_{{\mathbf{C}}^{d}}f(z)\,\overline{g(z)}\,e^{-|z|^{2}}\,dz$
$\displaystyle=(f(\partial)g^{*})(0)=(g^{*}(\partial)f)(0),\vphantom{\int}$

where

$g^{*}(z):=\overline{g(\overline{z})}.$

This makes ${\mathbf{P}}$ into a pre-Hilbert space, and the action

$f\mapsto f\circ k^{-1},\qquad k\in K,$

is a unitary representation of $K$ on ${\mathbf{P}}$. It is a deep result of
W. Schmid [Sch] that this representation has a multiplicity-free decomposition
into irreducibles

${\mathbf{P}}=\bigoplus_{\mathbf{m}}\;{\mathbf{P}}_{\mathbf{m}}$

where ${\mathbf{m}}$ ranges over all signatures, i.e. $r$-tuples
${\mathbf{m}}=(m_{1},m_{2},\dots,m_{r})\in{\mathbf{Z}}^{r}$ satisfying
$m_{1}\geq m_{2}\geq\dots\geq m_{r}\geq 0$. Polynomials in
${\mathbf{P}}_{\mathbf{m}}$ are homogeneous of degree
$|{\mathbf{m}}|:=m_{1}+m_{2}+\dots+m_{r}$; in particular, ${\mathbf{P}}_{(0)}$
are the constants and ${\mathbf{P}}_{(1)}$ the linear polynomials. Any
holomorphic function on $\Omega$ thus has a decomposition
$f=\sum_{\mathbf{m}}f_{\mathbf{m}}$,
$f_{\mathbf{m}}\in{\mathbf{P}}_{\mathbf{m}}$, which refines the usual
homogeneous expansion.

Since the spaces ${\mathbf{P}}_{\mathbf{m}}$ are finite dimensional, they
automatically possess a reproducing kernel: there exist polynomials
$K_{\mathbf{m}}(x,y)$ on ${\mathbf{C}}^{d}\times{\mathbf{C}}^{d}$, holomorphic
in $x$ and $\overline{y}$, such that for each $f\in{\mathbf{P}}_{\mathbf{m}}$
and $y\in{\mathbf{C}}^{d}$,

(31) $f(y)=\langle f,K_{\mathbf{m}}(\cdot,y)\rangle_{F}.$

From the definition of the spaces ${\mathbf{P}}_{\mathbf{m}}$ it also follows
that the kernels $K_{\mathbf{m}}(x,y)$ are $K$-invariant.

It is a consequence of Schur’s lemma from representation theory that for any
$K$-invariant inner product $\langle\cdot,\cdot\rangle$ on ${\mathbf{P}}$,
${\mathbf{P}}_{\mathbf{m}}$ and ${\mathbf{P}}_{\mathbf{n}}$ are orthogonal if
${\mathbf{m}}\neq{\mathbf{n}}$, while on each ${\mathbf{P}}_{\mathbf{m}}$,
$\langle\cdot,\cdot\rangle$ is proportional to
$\langle\cdot,\cdot\rangle_{F}$. In particular, for the inner product

$\langle
f,g\rangle_{\nu}:=c_{\nu}\int_{\Omega}f(z)\,\overline{g(z)}\,d\mu_{\nu}(z)\qquad(\nu>p-1),$

(with $c_{\nu}$ as in (25)) we have, for any
$f_{\mathbf{m}}\in{\mathbf{P}}_{\mathbf{m}}$ and
$g_{\mathbf{n}}\in{\mathbf{P}}_{\mathbf{n}}$,

(32) $\langle f_{\mathbf{m}},g_{\mathbf{n}}\rangle_{\nu}=\frac{\langle
f_{\mathbf{m}},g_{\mathbf{n}}\rangle_{F}}{(\nu)_{\mathbf{m}}}$

(cf. [FK]), where $(\nu)_{\mathbf{m}}$ is the _generalized Pochhammer symbol_

$(\nu)_{\mathbf{m}}:=(\nu)_{m_{1}}(\nu-\tfrac{a}{2})_{m_{2}}\dots(\nu-\tfrac{r-1}{2}a)_{m_{r}};$

here

$(\nu)_{k}:=\nu(\nu+1)\dots(\nu+k-1)\qquad\Big{(}=\frac{\Gamma(\nu+k)}{\Gamma(\nu)}\text{
if }\nu\neq 0,-1,-2,\dots,\Big{)}$

is the ordinary Pochhammer symbol.

A consequence of the relation (32) is the _Faraut-Koranyi formula_

(33) $h(x,y)^{-\nu}=\sum_{\mathbf{m}}\;(\nu)_{\mathbf{m}}K_{\mathbf{m}}(x,y)$

relating the reproducing kernels $K_{\nu}$ from (25) and $K_{\mathbf{m}}$ from
(31).

As already mentioned, the point $e=e_{1}+\dots+e_{r}$ belongs to the Shilov
boundary $\partial_{e}\Omega$ of $\Omega$. The group $K$ acts transitively on
$\partial_{e}\Omega$, so that $\partial_{e}\Omega=\\{ke,k\in K\\}\simeq K/L$,
where $L$ is the stabilizer of $e$ in $K$. Each Peter-Weyl space
${\mathbf{P}}_{\mathbf{m}}$ contains a unique $L$-invariant polynomial
$\phi_{\mathbf{m}}$ satisfying the normalization condition
$\phi_{\mathbf{m}}(e)=1$. We will sometimes write just
$\phi_{\mathbf{m}}(t_{1},\dots,t_{r})$ instead of
$\phi_{\mathbf{m}}(t_{1}e_{1}+\dots+t_{r}e_{r})$. These _spherical
polynomials_ $\phi_{\mathbf{m}}$ satisfy $\phi_{(0)}\equiv 1$,

(34) $\phi_{(m_{1}+1,m_{2}+1,\dots,m_{r}+1)}(t_{1},\dots,t_{r})=t_{1}\cdots
t_{r}\,\phi_{\mathbf{m}}(t_{1},\dots,t_{r}),$

and are related to the reproducing kernels $K_{\mathbf{m}}$ by the formula

(35)
$K_{\mathbf{m}}(x,e)=\frac{d_{\mathbf{m}}}{(d/r)_{\mathbf{m}}}\phi_{\mathbf{m}}(x),$

where $d_{\mathbf{m}}:=\dim{\mathbf{P}}_{\mathbf{m}}$. It is known that the
last dimension is given by the formula ([Up2], Lemmas 2.5 and 2.6)

$d_{\mathbf{m}}=\frac{(d/r)_{\mathbf{m}}}{(q_{\Omega})_{\mathbf{m}}}\,\pi_{\mathbf{m}}$

where

(36) $q_{\Omega}:=\frac{r-1}{2}a+1$

and

(37) $\pi_{\mathbf{m}}:=\prod_{1\leq i<j\leq
r}\frac{m_{i}-m_{j}+\frac{j-i}{2}a}{\frac{j-i}{2}a}\,\frac{(\frac{j-i+1}{2}a)_{m_{i}-m_{j}}}{(\frac{j-i-1}{2}a+1)_{m_{i}-m_{j}}}.$

Thus we may rewrite (35) as

(38)
$K_{\mathbf{m}}(x,e)=\frac{\pi_{\mathbf{m}}}{(q_{\Omega})_{\mathbf{m}}}\,\phi_{\mathbf{m}}(x).$

Combining the last formula with the fact that [FK, Lemma 3.2]

$K_{\mathbf{m}}(\hbox{$\sum_{j}$}t_{j}e_{j},\hbox{$\sum_{j}$}s_{j}e_{j})=K_{\mathbf{m}}(\hbox{$\sum_{j}$}t_{j}s_{j}e_{j},e),$

we thus get

(39)
$K_{\mathbf{m}}(k\hbox{$\sum_{j}$}t_{j}e_{j},k\hbox{$\sum_{j}$}t_{j}e_{j})=\frac{\pi_{\mathbf{m}}}{(q_{\Omega})_{\mathbf{m}}}\,\phi_{\mathbf{m}}(t_{1}^{2},\dots,t_{r}^{2}).$

The polynomials $\phi_{\mathbf{m}}$ have also a combinatorial interpretation
in terms of _Jack symmetric polynomials_ $J^{(\lambda)}_{\mathbf{m}}$ with
parameter $\lambda$ (cf. [MD], Section 10 of Chapter VI): namely,

(40)
$\phi_{\mathbf{m}}(t_{1},\dots,t_{r})=j_{\mathbf{m}}^{-1}\,J^{(2/a)}_{\mathbf{m}}(t_{1},\dots,t_{r}),$

where

(41)
$j_{\mathbf{m}}:=J^{(2/a)}_{\mathbf{m}}(\underbrace{1,\dots,1}_{r})=\Big{(}\frac{2}{a}\Big{)}^{|{\mathbf{m}}|}\Big{(}\frac{ra}{2}\Big{)}_{\mathbf{m}}.$

We will usually suppress the superscripts $(2/a)$ in the sequel.

Recall that in any Jordan algebra $J$ with unit $v$ and product $x\circ y$ an
element $x$ is called _invertible_ if it has a (necessarily unique) inverse
$y=:x^{-1}$ satisfying $x\circ y=v$ and $x^{2}\circ y=x$. In the special case
that the Jordan algebra arises as $J=Z_{1}(v)$ for a tripotent $v$ of the
JB*-triple $Z$ then invertibility of $z\in J$ is equivalent to the
invertibility of the operator $Q(z)$ on $J$ and $z^{-1}=Q(z)^{-1}Q(v)z$. In
particular, taking the inverse is a rational map on $J$ that can be written
(see e.g. [Up2, Chapter 4]) in exact (i.e. reduced) form as
$z^{-1}=p(z)/N(z)$, where $p:J\to J$ is a polynomial which generalizes the
matrix adjoint and $N:J\to{\mathbf{C}}$ is a polynomial called the
_determinant polynomial_ , or  _Koecher norm_ , of the Jordan algebra. In
particular, fixing a Jordan frame $e_{1},\dots,e_{r}$ of $Z$ the above applies
to the Jordan algebras $Z_{1}(e_{1}+\dots+e_{j})$, $1\leq j\leq r$; we denote
the corresponding determinant polynomials by $N_{j}$ and extend them to all of
$Z$ by defining $N_{j}(z):=N_{j}(P^{(j)}_{1}(z))$, where $P^{(j)}_{1}$ is the
canonical projection of $Z$ onto $Z_{1}(e_{1}+\dots+e_{j})$ given by the
Peirce decomposition (19). For a signature ${\mathbf{m}}$, the  _conical
polynomial_ $N^{\mathbf{m}}$ associated with ${\mathbf{m}}$ is

(42) $N^{\mathbf{m}}:=N_{1}^{m_{1}-m_{2}}N_{2}^{m_{2}-m_{3}}\cdots
N_{r}^{m_{r}}.$

In particular,

$N^{\mathbf{m}}\Big{(}{\sum_{j=1}^{r}t_{j}e_{j}}\Big{)}=\prod_{j=1}^{r}t_{j}^{m_{j}}.$

Each polynomial space ${\mathbf{P}}_{\mathbf{m}}$ is then spanned by
$N^{\mathbf{m}}\circ k$, $k\in K$. In particular, the conical polynomials are
related to the spherical polynomials by

$\phi_{\mathbf{m}}(z)=\int_{L}N^{\mathbf{m}}(lz)\,dl,$

where $dl$ stands for the normalized Haar measure on $L$.

Standard references for the material in this section are [Ar], [Lo], [FK], or
[Up2].

##  3\. Radial nearly holomorphic functions

The following relation between the nearly-holomorphic reproducing kernels
$N^{m}_{\nu}$ and the invariantly-polyanalytic reproducing kernels
$P^{m}_{\nu}$ is elementary.

######  Proposition 1.

$P^{m}_{\nu}(z,w)=h(z,z)^{m-1}h(w,w)^{m-1}N^{m}_{\nu+2(m-1)}(z,w)$.

###### Proof.

By their very definition (9), the mapping

$T:f(z)\longmapsto h(z,z)^{m-1}f(z)$

is a bijection of $\mathcal{N}^{m}$ onto $\mathcal{P}^{m}$. By (2), $T$
clearly acts isometrically from $L^{2}(\Omega,d\mu_{\nu})$ into
$L^{2}(\Omega,d\mu_{\nu-2(m-1)})$, for any $\nu\in{\mathbf{R}}$. Thus if
$\\{e_{j}(z)\\}_{j}$ is an orthonormal basis for
$\mathcal{N}^{m}_{\nu+2(m-1)}$, then $\\{h(z,z)^{m-1}e_{j}(z)\\}_{j}$ will be
an orthonormal basis for $\mathcal{P}^{m}_{\nu}$. Recalling the familiar
formula for a reproducing kernel in terms of an orthonormal basis

(43) $K(z,w)=\sum_{j}e_{j}(z)\overline{e_{j}(w)},$

the assertion follows. ∎

We also readily get a transformation formula for $N^{m}_{\nu}(z,w)$.

######  Proposition 2.

For any $\phi\in\operatorname{Aut}(\Omega)$,

(44)
$N^{m}_{\nu}(z,w)=\frac{h(a,a)^{\nu}}{h(z,a)^{\nu}h(a,w)^{\nu}}N^{m}_{\nu}(\phi
z,\phi w),\qquad a:=\phi^{-1}0.$

In particular,

(45) $N^{m}_{\nu}(z,w)=h(z,w)^{-\nu}N^{m}_{\nu}(\phi_{w}z,0).$

###### Proof.

Since $\overline{D}{}^{m}f$ is a tensor and $\phi$ is just a coordinate
change, the kernel $\mathcal{N}^{m}$ of $\overline{D}{}^{m}$ is automatically
invariant under the composition $f\mapsto f\circ\phi$ with $\phi$. As already
observed in the Introduction, it therefore follows from the transformation
formula (11) for the measure $d\mu_{\nu}$ (which formula is in turn a
consequence of the transformation rule (4) for the Jordan triple determinant,
in combination with the fact that the measure $d\mu_{0}$ is
$\operatorname{Aut}(\Omega)$-invariant) that the operator (12) acts unitarily
on $\mathcal{N}^{m}_{\nu}$. Employing again the formula (43), we thus obtain

$N^{m}_{\nu}(z,w)=\frac{h(a,a)^{\nu}}{h(z,a)^{\nu}h(a,w)^{\nu}}N^{m}_{\nu}(\phi^{-1}z,\phi^{-1}w),\qquad
a:=\phi 0.$

Replacing $\phi$ by $\phi^{-1}$ yields (44), and taking $\phi=\phi_{w}$ in
(44) yields (45). ∎

######  Corollary 3.

$P^{m}_{\nu}(z,w)=h(w,z)^{m-1}h(z,w)^{1-m-\nu}P^{m}_{\nu}(\phi_{w}z,0)$.

###### Proof.

Combine the last two propositions. ∎

We have thus reduced the identification of both $N^{m}_{\nu}$ and
$P^{m}_{\nu}$ to finding the reproducing kernel $N^{m}_{\nu}(z,0)$ at zero.
Note that by (44),

$N^{m}_{\nu}(kz,0)=N^{m}_{\nu}(z,0)\qquad\forall k\in K,$

where as before $K$ is the stabilizer of the origin $0\in\Omega$ in
$G=\operatorname{Aut}(\Omega)_{0}$; in other words, $N^{m}_{\nu}(\cdot,0)$ is
a _radial_ function. We now proceed to identify the radial nearly holomorphic
functions.

Recall that an element $z\in{\mathbf{C}}^{d}$ is called _quasi-invertible_
with respect to another element $w\in{\mathbf{C}}^{d}$ if, by definition, the
Bergman operator (18) is invertible on ${\mathbf{C}}^{d}$, and the _quasi-
inverse_ $z^{w}$ is then defined as

$z^{w}:=B(z,w)^{-1}(z-Q(z)w).$

Note that $z^{w}$ is holomorphic in $z$ and anti-holomorphic in $w$. Since
$\det B(z,w)=h(z,w)^{p}$ does not vanish on $\Omega\times\Omega$, the quasi-
inverse $z^{w}$ is, in particular, defined for all $z,w\in\Omega$. It is now a
result of [Zh3, formula (3.2) and Proposition 3.1], that, first of all,
$\overline{D}=B(z,z)\overline{\partial}$, and furthermore

(46)
$\partial\Psi=\overline{z}^{\overline{z}}=B(\overline{z},\overline{z})^{-1}(\overline{z}-Q(\overline{z})\overline{z}),$

where as before $\partial\Psi$ stands for the vector of derivatives
$\partial_{j}\Psi$ of the Kähler potential $\Psi(z)=-\log h(z,z)$.

######  Proposition 4.

Radial functions in $\mathcal{N}^{m}$ consist precisely of functions of the
form

(47) $p(z,z^{z}),$

where $p(z,w)$ is a polynomial in $z,\overline{w}\in{\mathbf{C}}^{d}$ of
degree $<m$ in each argument which is $K$-invariant in the sense that

(48) $p(kz,kw)=p(z,w)\qquad\forall k\in K.$

Consequently, the radial functions in $\mathcal{N}^{m}$ are the linear span of
$K_{\mathbf{m}}(z,z^{z})$, with $|{\mathbf{m}}|<m$.

###### Proof.

By (46) and the very definition of nearly-holomorphic functions, any
$f\in\mathcal{N}^{m}$ is of the form

$f(z)=p(z,z^{z}),$

with $p(z,w)$ holomorphic in $z\in\Omega$ and a polynomial of degree $<m$ in
$\overline{w}$. Since elements of $K$ are Jordan triple automorphisms, we have
$kz^{kw}=k(z^{w})$ for any $k\in K$, hence

$f(kz)=p(kz,k(z^{z}))$

with the same $p$. Thus $f$ is radial if and only if

$p(z,z^{z})=p(kz,kz^{kz})\qquad\forall z\in\Omega,\forall k\in K.$

The last equality means that, for any fixed $k\in K$, the two holomorphic
functions $p(z,\overline{w}^{z})$ and $p(kz,k\overline{w}^{kz})$ of
$z,w\in\Omega$ coincide on the anti-diagonal $z=\overline{w}$. By the well-
known uniqueness principle [BM, Proposition II.4.7], they must coincide for
all $z,w$. Since, for each fixed $z\in\Omega$, the image of $\Omega$ under the
(non-constant anti-holomorphic) map $w\mapsto\overline{w}^{z}$ is a (nonempty)
open set and $p$ is a polynomial in the second argument, actually
$p(z,y)=p(kz,ky)$ for all $z\in\Omega$ and $y\in{\mathbf{C}}^{d}$, proving
(48). Now it is well known basically from Schur’s lemma [AE, Proposition 2]
that the functions $p$ satisfying (48) are spanned by $K_{\mathbf{m}}(z,w)$,
as ${\mathbf{m}}$ ranges over all signatures. As $K_{\mathbf{m}}(z,w)$ is
homogeneous of degree $|{\mathbf{m}}|$ in both $z$ and $\overline{w}$, the
proposition follows. ∎

Thanks to the last proposition, we can reduce the identification of
$N^{m}_{\nu}(\cdot,0)$ to that of the reproducing kernel at 0 of a certain
space of symmetric polynomials on ${\mathbf{R}}^{r}$ (with $r$, as before,
denoting the rank of $\Omega$). First of all, denote by
$\mathcal{R}^{m}_{\nu}$ the subspace of all radial functions in
$\mathcal{N}^{m}_{\nu}$, and let $R^{m}_{\nu}(z,w)$ be its reproducing kernel.
Then

(49) $N^{m}_{\nu}(\cdot,0)=R^{m}_{\nu}(\cdot,0).$

Indeed, by the very definition of a reproducing kernel, $R^{m}_{\nu}(\cdot,0)$
is the (unique) element of $\mathcal{R}^{m}_{\nu}$ which reproduces the value
at 0 for all elements of $\mathcal{R}^{m}_{\nu}$. Now $N^{m}_{\nu}(\cdot,0)$
reproduces the value at 0 even for all elements of $\mathcal{N}^{m}_{\nu}$,
and belongs to $\mathcal{R}^{m}_{\nu}$ (being radial). So by uniqueness, (49)
follows.

Secondly, the space $\mathcal{R}^{m}_{\nu}$ can be described explicitly as
follows. For ease of notation, let us write for an $r$-tuple
$t=(t_{1},\dots,t_{r})\in{\mathbf{R}}_{+}^{r}$,

$\displaystyle
t^{b}:=\prod_{j=1}^{r}t_{j}^{b},\qquad(1-t)^{\nu}:=\prod_{j=1}^{r}(1-t_{j})^{\nu},\qquad\sqrt{t}=t^{1/2},$
$\displaystyle\frac{t}{1-t}:=\Big{(}\frac{t_{1}}{1-t_{1}},\dots,\frac{t_{r}}{1-t_{r}}\Big{)},\qquad
dt:=dt_{1}\dots dt_{r},$

and so forth, and let $d\rho_{b,\nu,a}$ be the modified Selberg measure

(50) $d\rho_{b,\nu,a}(t):=c_{\Omega}\;t^{b}(1+t)^{-\nu}\prod_{1\leq i<j\leq
r}|t_{i}-t_{j}|^{a}\,dt.$

Finally, if $e_{1},\dots,e_{r}$ is a fixed Jordan frame, we will write just
$te$ for $t_{1}e_{1}+\dots+t_{r}e_{r}$. Let $\mathcal{S}^{m}$ be the vector
space of all symmetric polynomials of degree $<m$ in $r$ variables, denote

$\mathcal{S}^{m}_{\nu}:=\mathcal{S}^{m}\cap
L^{2}({\mathbf{R}}_{+}^{r},d\rho_{b,\nu,a}),$

and let $S^{m}_{\nu}(x,y)$ be the reproducing kernel of
$\mathcal{S}^{m}_{\nu}$.

######  Proposition 5.

The mapping $V$ from $\mathcal{S}^{m}$ into functions on $\Omega$ given by

(51) $Vf(k\sqrt{t}e):=f\Big{(}\frac{t}{1-t}\Big{)},\qquad k\in
K,\;t\in[0,1]^{r},$

is a bijection from $\mathcal{S}^{m}$ onto radial functions in
$\mathcal{N}^{m}$. Furthermore, $V$ sends $\mathcal{S}^{m}_{\nu}$ unitarily
onto $\mathcal{R}^{m}_{\nu}$, and

(52) $R^{m}_{\nu}(\cdot,0)=VS^{m}_{\nu}(\cdot,0).$

###### Proof.

Let $z=k\sqrt{t}e$ be the polar decomposition of $z\in\Omega$. From the
formula (23) for the action of $B(z,z)$ on the Peirce subspaces (and the
similar formula for the action of $Q(z)$), one gets

$z^{z}=k\frac{\sqrt{t}}{1-t}e.$

Hence

$K_{\mathbf{m}}(z,z^{z})=K_{\mathbf{m}}(k\sqrt{t}e,k\tfrac{\sqrt{t}}{1-t}e)=K_{\mathbf{m}}(\tfrac{t}{1-t}e,e).$

This is, as we have seen in Section 2, up to a constant factor just the Jack
symmetric polynomial $J_{\mathbf{m}}(\frac{t}{1-t})$ in $r$ variables
evaluated at $\frac{t}{1-t}$. Since $J_{\mathbf{m}}$, $|{\mathbf{m}}|<m$, span
all symmetric polynomials of degree $<m$, by the preceding proposition the
radial functions in $\mathcal{N}^{m}$ are precisely those of the form $Vf$,
with $V$ as in (51) and $f$ a symmetric polynomial of degree $<m$. This proves
the first assertion.

As for the second, we have by (27)

(53)
$\|Vf\|^{2}_{L^{2}(d\mu_{\nu})}=c_{\Omega}\int_{[0,1]^{r}}\Big{|}f\Big{(}\frac{t}{1-t}\Big{)}\Big{|}^{2}\,d\mu_{b,\nu,a}(t).$

Making the change of variable $t_{j}=\frac{x_{j}}{1+x_{j}}$,
$x\in{\mathbf{R}}_{+}^{r}$, we have

$\displaystyle\frac{t}{1-t}=x,\qquad dt=(1+x)^{-2}\,dx,\qquad
t^{b}=x^{b}(1+x)^{-b},$ $\displaystyle(1-t)^{\nu-p}=(1+x)^{p-\nu},\qquad
t_{i}-t_{j}=\frac{x_{i}-x_{j}}{(1+x_{i})(1+x_{j})},$

implying, by a direct computation using (16), that

(54) $c_{\Omega}\,d\mu_{b,\nu,a}(t)=d\rho_{b,\nu,a}(x).$

By (53), the second claim follows.

Finally, (52) follows from the general formula (43) (applied to
$\mathcal{S}^{m}_{\nu}$ and $\mathcal{R}^{m}_{\nu}$), together with the fact
that under the above change of variables $x=\frac{t}{1-t}$, the point $t=0$
corresponds to $x=0$. ∎

We summarize our findings so far as the main result of this section.

######  Theorem 6.

The reproducing kernels $N^{m}_{\nu}$ and $P^{m}_{\nu}$ of the spaces
$\mathcal{N}^{m}_{\nu}$ and $\mathcal{P}^{m}_{\nu}$, respectively, are given
by

$\displaystyle N^{m}_{\nu}(z,w)$
$\displaystyle=h(z,w)^{-\nu}N^{m}_{\nu}(\phi_{z}w,0),$ $\displaystyle
P^{m}_{\nu}(z,w)$
$\displaystyle=\frac{h(z,z)^{m-1}h(w,w)^{m-1}}{h(z,w)^{\nu+2m-2}}N^{m}_{\nu+2m-2}(\phi_{w}z,0),$

where

$N^{m}_{\nu}(k\sqrt{t}e,0)=S^{m}_{\nu}(\tfrac{t}{1-t},0),$

where $S^{m}_{\nu}$ is the reproducing kernel of the $L^{2}$ space of
symmetric polynomials of degree $<m$ on ${\mathbf{R}}_{+}^{r}$ with respect to
the measure (50).

###### Proof.

Combine Propositions 1, 2, 4 and 5, and the formula (49). ∎

We conclude this section by a simple observation concerning the nontriviality
of the spaces $\mathcal{N}^{m}_{\nu}$ and $\mathcal{P}^{m}_{\nu}$.

######  Lemma 7.

A polynomial $P$ belongs to $L^{2}({\mathbf{R}}_{+}^{r},d\rho_{b,\nu,a})$ if
and only if its degree in each variable is less than $(\nu-p+1)/2$.

###### Proof.

Let $n_{1}$ be the degree of $P(x)$ in the variable $x_{1}$; thus the leading
term in the $x_{1}$ variable is $p_{1}(x^{\prime})x_{1}^{n_{1}}$, where the
polynomial $p_{1}$ in the $r-1$ variables $x^{\prime}=(x_{2},\dots,x_{r})$ is
not identically zero. The zero-set of $p_{1}$ is therefore a variety in
${\mathbf{R}}^{r-1}$ of codimension at least 1; we can therefore choose a
closed ball $Q$ (of positive finite radius) lying wholly in
$\\{y\in{\mathbf{R}}^{r-1}:y_{j}\neq y_{k}\text{ for all }j\neq k\\}$ such
that $|p_{1}|>0$ on $Q$. Set $R:=1+\sup\\{\|y\|:y\in Q\\}$. Then if $P\in
L^{2}({\mathbf{R}}_{+}^{r},d\rho_{b,\nu,a})$, the integral

$\int_{R}^{\infty}\int_{Q}|P(x_{1},x^{\prime})|^{2}\,d\rho_{b,\nu,a}(x_{1},x^{\prime})$

has to be finite. However, due to our choice of $Q$ and $R$, the integrand is
$\asymp x_{1}^{2n_{1}}$ (uniformly in $x^{\prime}$), while the measure is
$\asymp x_{1}^{b-\nu+(r-1)a}\,dx$ (uniformly in $x^{\prime}$). Consequently,
$x_{1}^{2n_{1}+(r-1)a+b-\nu}$ must be integrable at infinity, implying that
$2n_{1}+(r-1)a+b-\nu=2n_{1}+p-2-\nu<-1$, or $n_{1}<(\nu-p+1)/2$. Similarly,
$n_{j}<(\nu-p+1)/2$ for the degree $n_{j}$ of $P(x)$ in the variable $x_{j}$,
$j=1,\dots,r$.

Conversely, let $P(x)=x_{1}^{n_{1}}\dots x_{r}^{n_{r}}$ with
$n_{j}<(\nu-p+1)/2$ for all $j$. Making again the change of variable
$x=t/(1-t)$ shows that the $L^{2}$-norm of $P$ with respect to
$c_{\Omega}^{-1}d\rho_{b,\nu,a}$ equals

$\int_{[0,1]^{r}}\prod_{j=1}^{r}\Big{(}t_{j}^{2n_{j}+b}(1-t_{j})^{\nu-p-2n_{j}}\Big{)}\prod_{1\leq
i<j\leq r}|t_{i}-t_{j}|^{a}\,dt.$

The second term in the integrand is bounded (by 1), while the first term
yields just the product of single-variable integrals

$\int_{0}^{1}t_{j}^{2n_{j}+b}(1-t_{j})^{\nu-p-2n_{j}}\,dt_{j},$

which are finite since $2n_{j}+b>-1$ and $\nu-p-2n_{j}>-1$. ∎

######  Proposition 8.

  * (a)

$\mathcal{N}^{m}_{\nu}\neq\\{0\\}$ if and only if $\nu>p-1$, and
$\mathcal{P}^{m}_{\nu}\neq\\{0\\}$ if and only if $\nu>p+1-2m$.

  * (b)

$\mathcal{N}^{m}_{\nu}\setminus\mathcal{N}^{m-1}_{\nu}\neq\\{0\\}$ if and only
if there exists a signature ${\mathbf{m}}$ with $|{\mathbf{m}}|=m-1$ and
$m_{1}<\frac{\nu-p+1}{2}.$

  * (c)

In fact, $K_{\mathbf{m}}(z,z^{z})\in\mathcal{N}^{m}_{\nu}$ if and only if
$|{\mathbf{m}}|<m$ and $m_{1}<\frac{\nu-p+1}{2}.$

###### Proof.

(a) If $\mathcal{N}^{m}_{\nu}\neq\\{0\\}$ then its reproducing kernel is not
identically zero; by the last theorem, this is equivalent, in turn, to
$N^{m}_{\nu}(\cdot,0)\not\equiv 0$ and $S^{m}_{\nu}(\cdot,0)\not\equiv 0$.
Thus $\mathcal{S}^{m}_{\nu}$ contains a nonzero polynomial $p(x)$ (even one
that does not vanish at the origin). By the last lemma, necessarily
$\nu-p+1>0$, or $\nu>p-1$.

Conversely, if $\nu>p-1$, then $\mathcal{N}^{1}_{\nu}=A_{\nu}$ is nontrivial
(it contains all bounded holomorphic functions on $\Omega$), hence so is
$\mathcal{N}^{m}_{\nu}\supset\mathcal{N}^{1}_{\nu}$.

This settles the assertion for $\mathcal{N}^{m}_{\nu}$; the one for
$\mathcal{P}^{m}_{\nu}$ then follows from Proposition 1.

(b) By the same argument as in the proof of part (a),
$\mathcal{N}^{m}_{\nu}\setminus\mathcal{N}^{m-1}_{\nu}\neq\\{0\\}$ if and only
if $\mathcal{S}^{m}_{\nu}$ contains a polynomial $P$ whose total degree is
$m-1$ and whose degree in each variable is $<\frac{\nu-p+1}{2}$. If
$x^{\alpha}$, with $\alpha$ a multiindex, is any monomial in the top degree
homogeneous component of $P$, then the nonincreasing rearrangement of $\alpha$
yields the desired signature ${\mathbf{m}}$.

(c) This follows in the same way as for part (b) from the fact that $V$ maps
$K_{\mathbf{m}}(z,z^{z})$ into a (nonzero) constant multiple of the Jack
polynomial $J_{\mathbf{m}}(x)$, and $J_{\mathbf{m}}(x)$ is equal to the
symmetrization of (writing $x^{\mathbf{m}}:=x_{1}^{m_{1}}\dots x_{r}^{m_{r}}$)

(55)
$x^{\mathbf{m}}+\sum_{{\mathbf{n}}<{\mathbf{m}}}c_{{\mathbf{m}}{\mathbf{n}}}x^{\mathbf{n}}$

where the sum is over (some) signatures ${\mathbf{n}}$ smaller than
${\mathbf{m}}$ with respect to the lexicographic order; cf. Mcdonald [MD,
formula (10.13)]. ∎

######  Corollary 9.

$\mathcal{R}^{m}_{\nu}=\operatorname{span}\\{K_{\mathbf{m}}(z,z^{z}):|{\mathbf{m}}|<m,\;m_{1}<\tfrac{\nu-p+1}{2}\\}$.

In particular, if $q$ denotes the nonnegative integer such that

$q<\frac{\nu-p+1}{2}\leq q+1,$

then

(56) $\displaystyle\mathcal{R}^{m}_{\nu}$
$\displaystyle=\operatorname{span}\\{K_{\mathbf{m}}(z,z^{z}):|{\mathbf{m}}|<m\\}\qquad\text{if
}m\leq q+1,$ (57) $\displaystyle\mathcal{R}^{m}_{\nu}$
$\displaystyle=\operatorname{span}\\{K_{\mathbf{m}}(z,z^{z}):m_{1}\leq
q\\}\qquad\text{if }m\geq rq+1.$

This means that for $m\geq rq+1$, $\mathcal{R}^{m}_{\nu}$ and, hence,
$\mathcal{N}^{m}_{\nu}$ equals $\mathcal{N}^{rq+1}_{\nu}$ — i.e. the spaces
$\mathcal{N}^{m}_{\nu}$ “stabilize” and stop growing with $m$ (for fixed
$\nu$). Likewise, $N^{m}_{\nu}(z,w)=N^{rq+1}_{\nu}(z,w)$ for all $m\geq rq+1$
if $2q-1<\nu-p\leq 2q+1$.

######  Remark 10.

We pause to note that while, clearly,

$\mathcal{N}^{1}\subset\mathcal{N}^{2}\subset\mathcal{N}^{2}\subset\dots,$

no such inclusions hold for $\mathcal{P}^{m}$, except when the rank $r=1$.
More specifically, for $r>1$, the function $\mathbf{1}$ (constant one) belongs
to $\mathcal{P}^{1}$, but not to any $\mathcal{P}^{m}$, $m\geq 2$. Indeed,
$\mathbf{1}\in\mathcal{P}^{m}\iff h(z,z)^{1-m}\in\mathcal{N}^{m}$, by (9); and
by Proposition 4, the latter is equivalent to

$(1-t)^{1-m}=\sum_{|{\mathbf{m}}|<m}c_{\mathbf{m}}\phi_{\mathbf{m}}(\tfrac{t}{1-t})$

with some coefficients $c_{\mathbf{m}}$. Passing again to $x=\frac{t}{1-t}$,
this translates into

(58) $(1+x)^{m-1}=\sum_{|{\mathbf{m}}|<m}c_{\mathbf{m}}\phi_{\mathbf{m}}(x).$

But by the Faraut-Koranyi formula (33), the left-hand side equals

$\sum_{\mathbf{n}}(1-m)_{\mathbf{n}}\frac{\pi_{\mathbf{n}}(-1)^{|{\mathbf{n}}|}}{(q_{\Omega})_{\mathbf{n}}}\phi_{\mathbf{n}}(x).$

Since the $\phi_{\mathbf{m}}$ are linearly independent, (58) can hold only if

$(1-m)_{\mathbf{n}}=0\qquad\text{whenever }|{\mathbf{n}}|\geq m.$

However, for ${\mathbf{n}}=(m-1,1)$ one has
$(1-m)_{\mathbf{n}}=(-1)^{m}(m-1)!(m-1+\frac{a}{2})$ which is nonzero. So
$\mathbf{1}\notin\mathcal{P}^{m}$ for $m\geq 2$ if $r>1$. ∎

##  4\. Spherical functions

Any $g\in G$ can be uniquely written in the form $g=k\phi_{w}$, with
$w=g^{-1}0\in\Omega$, $k\in K$ and $\phi_{w}$ the geodesic reflection (17)
interchanging $0$ and $w$. This yields the formula for the complex Jacobian

(59) $J_{g}(z)=\det k\cdot(-1)^{d}\frac{h(w,w)^{p/2}}{h(z,w)^{p}},$

which shows that the projective representation (12) is actually nothing else
than

$f\longmapsto J_{\phi^{-1}}^{\nu/p}\cdot f\circ\phi^{-1},\qquad\phi\in G.$

In order to make this not only projective but genuine representation if
$\nu/p$ is not an integer, one needs to pass from $G$ to its universal cover
$\widetilde{G}$. The elements of $\widetilde{G}$ can be thought of as elements
$g$ of $G$ together with a consistent choice of $\log J_{g}$. The operators

(60) $U^{(\nu)}_{g}:f\longmapsto J_{g^{-1}}^{\nu/p}\cdot f\circ g^{-1},\qquad
g\in\widetilde{G},$

then define a (honest, not only projective) unitary representation of
$\widetilde{G}$ on $L^{2}(\Omega,d\mu_{\nu})$; and one has
$\Omega=\widetilde{G}/\widetilde{K}$, where $\widetilde{K}$, the preimage of
$K$ under the covering map $\widetilde{G}\to G$, is the universal cover of $K$
and the stabilizer of $0\in\Omega$ in $\widetilde{G}$. (Actually one has
$\widetilde{K}\cong K\times{\mathbf{R}}$, but we will not need this fact.)

Using (60), one can identify $L^{2}(\Omega,d\mu_{\nu})$ with a subspace of
$L^{2}(\widetilde{G}/Z(\widetilde{G}))$, the $L^{2}$ space on the quotient of
$\widetilde{G}$ modulo its center $Z(\widetilde{G})$ with respect to a
suitably normalized Haar measure on $\widetilde{G}$. Namely, for $f\in
L^{2}(\Omega,d\mu_{\nu})$, the function $f^{\\#}$ on $\widetilde{G}$ defined
by

(61) $f^{\\#}(g):=f(g0)J_{g}(0)^{-\nu/p},\qquad g\in\widetilde{G},$

satisfies

(62) $(U^{(\nu)}_{g}f)^{\\#}=f^{\\#}\circ g^{-1}$

(i.e. the map $f\mapsto f^{\\#}$ intertwines the representation (60) with the
left regular representation of $\widetilde{G}$ on $L^{2}(\widetilde{G})$) and

(63) $f^{\\#}(gk)=f^{\\#}(g)J_{k}^{-\nu/p},\qquad
g\in\widetilde{G},\;k\in\widetilde{K}.$

(Note that $J_{k}\equiv J_{k}(0)$ is a constant function, so we will write
just $J_{k}$ instead of $J_{k}(0)$ or $J_{k}(z)$.) Furthermore, $f\mapsto
f^{\\#}$ is a unitary isomorphism of $L^{2}(\Omega,d\mu_{\nu})$ onto the
subspace $L^{2}(\widetilde{G},\nu)$ of all functions in
$L^{2}(\widetilde{G}/Z(\widetilde{G}))$ satisfying the transformation rule
(63); the inverse of the map $f\mapsto f^{\\#}$ is given by $F\mapsto
F^{\flat}$, where

(64) $F^{\flat}(g0):=F(g)J_{g}(0)^{\nu/p}$

(the right-hand side depends only on $g0$, thanks to (63)). See Proposition
2.1 in [DOZ] for the proof of all these facts. (Note: there is a misprint in
the first formula of Section 2 in [DOZ], the $\tau_{\nu}$ there should be
$\tau_{-\nu}$.)

Using the above identification, one can view also $\mathcal{N}^{m}_{\nu}$ as a
subspace of $L^{2}(\widetilde{G},\nu)$ invariant under the left regular
representation (62). Note that radial functions on $\Omega$, i.e. those
satisfying $f(kz)=f(z)$ for all $z\in\Omega$ and $k\in K$, correspond to
functions $f^{\\#}$ on $\widetilde{G}$ satisfying

(65)
$f^{\\#}(k^{\prime}gk)=J_{k^{\prime}}^{-\nu/p}f^{\\#}(g)J_{k}^{-\nu/p},\qquad
g\in\widetilde{G},\;k,k^{\prime}\in\widetilde{K}.$

Such functions on $\widetilde{G}$ are called _$\nu$ -spherical_.

The representation theory for the space $L^{2}(\widetilde{G},\nu)$ has been
developed by Shimeno [Shm] (his notation $\tau_{-\ell}(k)$ corresponds to our
$J_{k}^{-\nu/p}$). (Note that there is a $\widetilde{G}$-equivariant
isomorphism between $L^{2}(\widetilde{G},\nu)$ and
$L^{2}(\widetilde{G},-\nu)$, so we can always assume that $\nu>0$ as we have
started with.) Namely, let $\widetilde{G}=\widetilde{K}AN$ be the Iwasawa
decomposition of $\widetilde{G}$ and let $\mathfrak{g}$, $\mathfrak{k}$,
$\mathfrak{p}$ and $\mathfrak{a}$ be the Lie algebras of $G$ (and
$\widetilde{G}$), $K$ (and $\widetilde{K}$), $AN$ and $A$, respectively, so
that $\mathfrak{g}=\mathfrak{k}+\mathfrak{p}$ is the Cartan decomposition of
$\mathfrak{g}$ and $\mathfrak{a}\subset\mathfrak{p}$ is a maximal Abelian
subspace of $\mathfrak{p}$. For $g\in\widetilde{G}$ let $H(g)$ be the element
in the Lie algebra $\mathfrak{a}$ of $A$ uniquely determined by
$g\in\widetilde{K}\exp H(g)N$. Similarly, let $\kappa(g)\in\widetilde{K}$ be
uniquely determined by $g\in\kappa(g)AN$. For
$\lambda\in\mathfrak{a}^{*{\mathbf{C}}}$, the complexification of the dual
$\mathfrak{a}^{*}$ of $\mathfrak{a}$, one defines the _spherical function_
$\phi_{\lambda,\nu}$ of type $\nu$ by

(66)
$\phi_{\lambda,\nu}(g):=\int_{\widetilde{K}/Z(\widetilde{G})}e^{-(\lambda+\rho)H(g^{-1}k)}J^{\nu/p}_{k^{-1}\kappa(g^{-1}k)}\,dk,$

where $Z(\widetilde{G})$ denotes the center of $\widetilde{G}$, $dk$ is the
invariant measure on the quotient $\widetilde{K}/Z(\widetilde{G})$ with total
mass 1, and $\rho\in\mathfrak{a}^{*}$ is the half-sum of positive roots (see
(72) below). Then $\phi_{\lambda,\nu}$ is a $\nu$-spherical function on
$\widetilde{G}$, and one defines the spherical Fourier transform $\hat{f}$ of
a $\nu$-spherical function $f$ on $\widetilde{G}$ by

(67)
$\hat{f}(\lambda):=\int_{\widetilde{G}/Z(\widetilde{G})}f(g)\phi_{-\lambda,-\nu}(g)\,dg,\qquad\lambda\in\mathfrak{a}^{*{\mathbf{C}}}.$

This definition makes sense e.g. whenever $f$ is compactly supported modulo
$Z(\widetilde{G})$. The main result of [Shm] then states that there is an
inversion formula

$f(g)=\int_{\bigcup_{j=0}^{r}D_{\nu,j}+i\mathfrak{a}^{*}_{\Theta_{j}}}\hat{f}(\lambda)\phi_{\lambda,\nu}(g)\,d\gamma(\lambda)$

where $D_{\nu,j}$ and $\mathfrak{a}^{*}_{\Theta_{j}}$ are certain systems of
hyperplanes in $\mathfrak{a}^{*}$ and $d\gamma$ is a certain measure on them;
and there is also a corresponding Plancherel theorem. See Theorems 6.7 and 6.8
in [Shm] for the details. Both $D_{\nu,j}$ and $\mathfrak{a}^{*}_{\Theta_{j}}$
have codimension $j$ in $\mathfrak{a}^{*}$; in particular, for $j=r$,
$\mathfrak{a}^{*}_{\Theta_{r}}=\\{0\\}$ and

(68) $D_{\nu,r}=\\{\lambda_{\mathbf{m}}:{\mathbf{m}}\text{ is a signature with
}m_{1}<\tfrac{\nu-p+1}{2}\\}$

where

(69)
$\lambda_{\mathbf{m}}:=\frac{1}{2}\sum_{j=1}^{r}\lambda_{j}\beta_{j},\qquad\lambda_{r+1-j}=p-1-\nu-(j-1)a+2m_{j}.$

(Here $\beta_{1},\dots,\beta_{r}\in\mathfrak{a}^{*}$ are the long roots of the
root system of $\Omega$; see below.) Thus $D_{\nu,r}$ is a finite discrete set
in $\mathfrak{a}^{*}$. Furthermore, the Plancherel measure $d\gamma$ on
$D_{\nu,r}+i\mathfrak{a}^{*}_{\Theta_{r}}=D_{\nu,r}$ reduces just to a
multiple $d_{r}(\lambda,\nu)\delta_{\lambda}$ of the Dirac mass at each
$\lambda=\lambda_{\mathbf{m}}$. Here $d_{r}(\lambda,\nu)$ is given by an
explicit expression involving $\Gamma$-functions; see (4.18), (4.19), (4.21)
and (6.18) in [Shm]. Altogether, it thus follows that the space
$\mathcal{A}^{\mathbf{m}}_{\nu}(\widetilde{G})$ spanned in
$L^{2}(\widetilde{G})$ by $\widetilde{G}$-translates of
$\phi_{\lambda_{\mathbf{m}},\nu}$, for each signature ${\mathbf{m}}$, is an
irreducible direct summand of $L^{2}(\widetilde{G})$, and as ${\mathbf{m}}$
varies these summands are mutually orthogonal. Such summands are called
_relative discrete series_ representations of $\widetilde{G}$. Note that by
(68), only signatures ${\mathbf{m}}$ with $m_{1}<\frac{\nu-p+1}{2}$ occur (in
particular, there are no relative discrete series representations if $\nu\leq
p-1$).

Let

$\mathcal{A}^{\mathbf{m}}_{\nu}(\Omega):=\\{F^{\flat}:F\in\mathcal{A}^{\mathbf{m}}_{\nu}(\widetilde{G})\\}$

be the space of functions on $\Omega$ corresponding to
$\mathcal{A}^{\mathbf{m}}_{\nu}(\widetilde{G})$ via (64). It is then, next,
the central result of [Zh3] that $\mathcal{A}^{\mathbf{m}}_{\nu}(\Omega)$ is
actually a space of nearly holomorphic functions of order $|{\mathbf{m}}|$,
and that, as ${\mathbf{m}}$ varies, these spaces exhaust all nearly
holomorphic functions. Namely, let $N^{\mathbf{m}}$ be the conical polynomial
on $\Omega$ from (42); one can then form the composition
$N^{\mathbf{m}}(\overline{z}^{\overline{z}})$ with the quasi-inverse (46),
which gives a function on $\Omega$. By Theorem 4.7 in [Zh3], the space spanned
by $U^{(\nu)}_{g}N^{\mathbf{m}}(\overline{z}^{\overline{z}})$,
$g\in\widetilde{G}$, coincides with $\mathcal{A}^{\mathbf{m}}_{\nu}(\Omega)$.

Finally, arguing as in the beginning of Section 3 in [Zh1], it follows from
Shimeno’s Plancherel formula that the reproducing kernel
$A^{\mathbf{m}}_{\nu}(z,w)$ of the space
$\mathcal{A}^{\mathbf{m}}_{\nu}(\Omega)$ is given for $w=0$ (i.e. at the
origin) simply by the appropriate multiple of the spherical function:

(70)
$A^{\mathbf{m}}_{\nu}(z,0)=d_{r}(\lambda_{\mathbf{m}},\nu)\phi_{\lambda_{\mathbf{m}},\nu}^{\flat}(z).$

Summarizing the discussion so far, we have thus arrived at the following
result.

######  Theorem 11.

The nearly-holomorphic reproducing kernel $N^{m}_{\nu}$ at the origin is given
by

(71) $N^{\mathbf{m}}_{\nu}(z,0)=\sum_{\begin{subarray}{c}|{\mathbf{m}}|<m\\\
m_{1}<\frac{\nu-p+1}{2}\end{subarray}}d_{r}(\lambda_{\mathbf{m}},\nu)\phi_{\lambda_{\mathbf{m}},\nu}^{\flat}(z),$

with $d_{r}(\lambda,\nu)$, $\phi_{\lambda,\nu}$ and $\lambda_{\mathbf{m}}$ as
above.

We pause to note that actually
$d_{r}(\lambda,\nu)=\|\phi_{\lambda,\nu}\|^{-2}_{L^{2}(\widetilde{G})}$, cf.
Remark 6.9 in [Shm].

We conclude by recalling the relation between the spherical functions
$\phi_{\lambda,\nu}$ and the hypergeometric functions of Heckman and Opdam
[HS, Part I]. With our notation $\mathfrak{g}$ for the Lie algebra of $G$, let
$\Sigma$ be the restricted root system of the pair
$(\mathfrak{g},\mathfrak{a})$. Then $\Sigma$ is a root system of type BC, i.e.
has the form

$\Sigma=\\{\pm\tfrac{1}{2}\beta_{j},\pm\beta_{j},\pm\tfrac{1}{2}(\beta_{j}\pm\beta_{k}),\;1\leq
j,k\leq r,\;j\neq k\\},$

where $\\{\beta_{j}\\}_{j}$ is a certain basis of $\mathfrak{a}^{*}$. Here the
_short roots_ $\pm\frac{1}{2}\beta_{j}$ have multiplicity $m_{S}=2b$, the
_long roots_ $\pm\beta_{j}$ have multiplicity $m_{L}=1$, and the _middle
roots_ $\frac{1}{2}(\pm\beta_{j}\pm\beta_{k})$ have multiplicity $m_{M}=a$; if
$b=0$, then the short roots are actually absent (and if $r=1$, then the middle
roots are actually absent). The positive roots are $\frac{1}{2}\beta_{j}$,
$\beta_{j}$, $j=1,\dots,r$, and $\frac{1}{2}(\beta_{j}\pm\beta_{k})$, $1\leq
k<j\leq r$; the _half-sum of positive roots_ is thus given by

(72) $\rho=\sum_{j=1}^{r}\frac{b+1+(j-1)a}{2}\,\beta_{j}.$

Let now $F(\lambda,\mathbf{k}_{\nu},\cdot)$ be the Heckman-Opdam
hypergeometric function with parameter
$\lambda\in\mathfrak{a}^{*{\mathbf{C}}}$ corresponding to the root system
$2\Sigma$ with multiplicities $\mathbf{k}_{\nu}$ given by

(73)
$\mathbf{k}_{\nu,S}=\frac{m_{S}}{2}-\nu=b-\nu,\qquad\mathbf{k}_{\nu,L}=\frac{m_{L}}{2}+\nu=\frac{1}{2}+\nu,\qquad\mathbf{k}_{\nu,M}=\frac{m_{M}}{2}=\frac{a}{2},$

for the “doubles” of the short, long and middle roots of $\Sigma$,
respectively. Then for any $g\in A\subset\widetilde{G}$ we have

(74) $\phi_{\lambda,\nu}(g)=h(g0,g0)^{-\nu/2}F(\lambda,\mathbf{k}_{\nu},g).$

See [HS, Theorem 5.2.2] (cf. also Remark 3.8 in [Shm]). Note further that by
[HS, (4.4.10)] (cf. also Remark 5.12 in [Shm]), if $\lambda$ is a dominant
weight (i.e.
$\frac{\langle\lambda,\alpha\rangle}{\langle\alpha,\alpha\rangle}\in{\mathbf{N}}$
for all positive roots $\alpha$ of $\Sigma$), then

(75)
$F(\lambda,\rho_{\nu},\mathbf{k}_{\nu},\cdot)=c(\lambda+\rho_{\nu},\mathbf{k}_{\nu})P(\lambda,\mathbf{k}_{\nu},\cdot),$

where $\rho_{\nu}$ is given by (72) with $b$ replaced by $b+\nu$, and
$P(\lambda,\mathbf{k}_{\nu},\cdot)$ are the multivariable Jacobi polynomials
(cf. Debiard [De1], [De2]). Here $c(\lambda,\mathbf{k})$ is the generalized
$c$-function of Harish-Chandra with line bundle parameter $\nu$ (cf. (3.15) in
[Shm] or (3.4.3) in [HS]).

Combining (71), (74) and (72) with Theorem 6, we can express also the
invariantly-polyanalytic reproducing kernels $P^{m}_{\nu}$ and the reproducing
kernels $S^{m}_{\nu}$ of the spaces $\mathcal{S}^{m}_{\nu}$ of symmetric
polynomials on ${\mathbf{R}}_{+}^{r}$ in terms of spherical functions, or
Heckman-Opdam hypergeometric functions, or multivariable Jacobi polynomials.
We omit the details.

######  Example 12.

For $\Omega={\mathbf{B}}^{d}$, the unit ball of ${\mathbf{C}}^{d}$, we have
$r=1$, $b=d-1$, $a$ is not defined, $p=d+1$ and $h(z,w)=1-\langle z,w\rangle$.
The elements of the group $G=SU(1,n)$ can be identified with
$(n+1)\times(n+1)$ complex matrices $\left(\begin{smallmatrix}A&B\\\
C^{t}&D\end{smallmatrix}\right)$, with $A\in{\mathbf{C}}$,
$B,C\in{\mathbf{C}}^{1\times n}$ and $D\in{\mathbf{C}}^{n\times n}$, acting by
$z\mapsto(Az+B)(Cz+D)^{-1}$, with $z\in{\mathbf{B}}^{d}$ written as row
vector. The Lie algebra $\mathfrak{a}$ equals ${\mathbf{R}}H$, where

$H=\begin{pmatrix}1&0^{1\times(n-1)}&0\\\ 0^{(n-1)\times
1}&0^{(n-1)\times(n-1)}&0^{(n-1)\times 1}\\\
0&0^{1\times(n-1)}&1\end{pmatrix},$

and defining $\beta\in\mathfrak{a}^{*}$ by $\beta(H)=2$ the root system is
given by $\Sigma=\\{\pm\frac{1}{2}\beta,\pm\beta\\}$. We have

$\exp(tH)=\begin{pmatrix}\cosh t&0^{1\times(n-1)}&\sinh t\\\ 0^{(n-1)\times
1}&I^{(n-1)\times(n-1)}&0^{(n-1)\times 1}\\\ \sinh t&0^{1\times(n-1)}&\cosh
t\end{pmatrix},$

hence $|\exp(tH)0|=|\tanh t|$ and

(76) $\cosh t=h(\exp(tH)0,\exp(tH)0)^{-1/2}.$

The spherical functions $\phi_{\lambda,\nu}$ are given by ([Shm, (8.2) and
(8.3)])

$\displaystyle\phi_{\lambda,\nu}(\exp tH)$ $\displaystyle=(\cosh
t)^{-\nu}{}_{2}\\!F_{1}\Big{(}\begin{matrix}\frac{d-\nu+\lambda}{2},\frac{d-\nu-\lambda}{2}\\\
d\end{matrix}\Big{|}-\sinh^{2}t\Big{)}$ (77) $\displaystyle=(\cosh
t)^{\nu}{}_{2}\\!F_{1}\Big{(}\begin{matrix}\frac{d+\nu+\lambda}{2},\frac{d+\nu-\lambda}{2}\\\
d\end{matrix}\Big{|}-\sinh^{2}t\Big{)},$

that is, by (64) and (76), and since $J_{\exp tH}(0)=\cosh^{-p}t$,

$\displaystyle\phi_{\lambda,\nu}^{\flat}(z)$
$\displaystyle=(1-|z|^{2})^{\nu}{}_{2}\\!F_{1}\Big{(}\begin{matrix}\frac{d-\nu+\lambda}{2},\frac{d-\nu-\lambda}{2}\\\
d\end{matrix}\Big{|}\frac{|z|^{2}}{|z|^{2}-1}\Big{)}$
$\displaystyle={}_{2}\\!F_{1}\Big{(}\begin{matrix}\frac{d+\nu+\lambda}{2},\frac{d+\nu-\lambda}{2}\\\
d\end{matrix}\Big{|}\frac{|z|^{2}}{|z|^{2}-1}\Big{)},$

where ${}_{2}\\!F_{1}$ is the Gauss hypergeometric function and on the right-
hand sides, we write just $\lambda$ for $\lambda(H)$. Using the standard
transformation formula for ${}_{2}\\!F_{1}$ [BE, §2.1 (22)]

(78) ${}_{2}\\!F_{1}\Big{(}\begin{matrix}a,b\\\
c\end{matrix}\Big{|}z\Big{)}=(1-z)^{-a}{}_{2}\\!F_{1}\Big{(}\begin{matrix}a,c-b\\\
c\end{matrix}\Big{|}\frac{z}{z-1}\Big{)}=(1-z)^{-b}{}_{2}\\!F_{1}\Big{(}\begin{matrix}c-a,b\\\
c\end{matrix}\Big{|}\frac{z}{z-1}\Big{)},$

this can also be written as

$\displaystyle\phi_{\lambda,\nu}^{\flat}(z)$
$\displaystyle=(1-|z|^{2})^{\frac{d+\lambda+\nu}{2}}{}_{2}\\!F_{1}\Big{(}\begin{matrix}\frac{d+\lambda+\nu}{2},\frac{d+\lambda-\nu}{2}\\\
d\end{matrix}\Big{|}|z|^{2}\Big{)}$
$\displaystyle=(1-|z|^{2})^{\frac{d-\lambda+\nu}{2}}{}_{2}\\!F_{1}\Big{(}\begin{matrix}\frac{d-\lambda+\nu}{2},\frac{d-\lambda-\nu}{2}\\\
d\end{matrix}\Big{|}|z|^{2}\Big{)}.$

The elements $\lambda_{\mathbf{m}}\in\mathfrak{a}^{*}$,
${\mathbf{m}}=(m_{1})$, are given by
$\lambda_{\mathbf{m}}=\lambda_{1}\frac{\beta}{2}$ with
$\lambda_{1}=2m_{1}+d-\nu$, $m_{1}\in{\mathbf{Z}}$, $0\leq
m_{1}<\frac{\nu-d}{2}$. The corresponding space
$\mathcal{A}^{\mathbf{m}}_{\nu}({\mathbf{B}}^{d})$ is spanned by
$G$-translates of the function $\overline{z}_{1}^{m_{1}}(1-|z|^{2})^{-m_{1}}$
under the action (12) [Zh3, Section 5] and coincides with the orthogonal
complement $\mathcal{N}^{m_{1}+1}_{\nu}\ominus\mathcal{N}^{m_{1}}_{\nu}$ [Zh1,
pp. 116-117]. The reproducing kernel of
$\mathcal{A}^{\mathbf{m}}_{\nu}({\mathbf{B}}^{d})$ at the origin is given by
(71) with

$d_{1}(\lambda_{\mathbf{m}},\nu)=\frac{(\nu-d-2m_{1})\pi^{-d}\Gamma(d+m_{1})\Gamma(\nu-
m_{1})}{m_{1}!\Gamma(d)\Gamma(\nu-d+1-m_{1})},$

in complete agreement with [Zh1, Section 3, bottom of p. 116]. (Note that
there is a misprint in the formula (1.5) in [Zh1]: the $\Gamma(\alpha+1+d)$ in
the numerator should be $\Gamma(\alpha+1+d-l)$. Also the labeling of spherical
functions is different there: our $\phi_{\lambda,\nu}$ corresponds to
$\phi_{i\lambda,\nu}$ in [Zh1].)

By [Op, p. 90], the Heckman-Opdam hypergeometric function for root system BC1
is given by

$F(\lambda,\mathbf{k},\exp
tH)={}_{2}\\!F_{1}\Big{(}\begin{matrix}\tfrac{d+\nu+\lambda}{2},\tfrac{d+\nu-\lambda}{2}\\\
d\end{matrix}\Big{|}-\sinh^{2}t\Big{)},$

in full accordance with (74) and (77) in view of (76).

Finally, for rank one (75) reduces just to (15), and thus for
$m-1<\frac{\nu-p+1}{2}$

$\displaystyle N^{m}_{\nu}(z,0)$
$\displaystyle=\sum_{m_{1}=0}^{m-1}d_{1}(\lambda_{\mathbf{m}},\nu){}_{2}\\!F_{1}\Big{(}\begin{matrix}-m_{1},d-\nu+m_{1}\\\
d\end{matrix}\Big{|}\frac{|z|^{2}}{|z|^{2}-1}\Big{)}$
$\displaystyle=\sum_{m_{1}=0}^{m-1}d_{1}(\lambda_{\mathbf{m}},\nu)(1-|z|^{2})^{-m_{1}}{}_{2}\\!F_{1}\Big{(}\begin{matrix}-m_{1},\nu-
m_{1}\\\ d\end{matrix}\Big{|}|z|^{2}\Big{)}\qquad\text{by \eqref{WR}}$
$\displaystyle=\sum_{m_{1}=0}^{m-1}d_{1}(\lambda_{\mathbf{m}},\nu)\frac{(1-|z|^{2})^{-m_{1}}}{\binom{m_{1}+d-1}{d-1}}P^{(d-1,\nu-d-2m_{1})}_{m_{1}}(1-2|z|^{2}),$

recovering (13) and its reformulation in terms of Jacobi polynomials.

From Theorem 6 we see that the reproducing kernel at the origin for the
subspace of all polynomials of degree $<m$ in
$L^{2}({\mathbf{R}}_{+},c_{{\mathbf{B}}^{d}}t^{d-1}(1+t)^{-\nu}\,dt)$ equals

(79) $S^{m}_{\nu}(x,0)=\sum_{0\leq
m_{1}<\min(m,\frac{\nu-d}{2})}d_{1}(\lambda_{(m_{1})},\nu)\;{}_{2}\\!F_{1}\Big{(}\begin{matrix}-m_{1},d-\nu+m_{1}\\\
d\end{matrix}\Big{|}-x\Big{)},$

the summands being actually mutually orthogonal. ∎

##  5\. Faraut-Koranyi hypergeometric functions

With Theorem 6 in mind, let us return to the reproducing kernels
$S^{m}_{\nu}(x,y)$ of the subspaces $\mathcal{S}^{m}_{\nu}$ of symmetric
polynomials of degree $<m$ in $L^{2}({\mathbf{R}}_{+}^{r},d\rho_{b,\nu,a})$.
By Propositions 4 and 5, the functions

(80) $K_{\mathbf{m}}(xe,e),\qquad|{\mathbf{m}}|<m,\;m_{1}<\tfrac{\nu-p+1}{2},$

span $\mathcal{S}^{m}_{\nu}$. The following easy fact — capturing, in effect,
the standard Gram-Schmidt orthogonalization process — describes how to extract
the reproducing kernel from an arbitrary basis.

######  Proposition 13.

Let $\mathcal{H}$ be a finite-dimensional Hilbert space of functions with (not
necesarily orthogonal) basis $\\{f_{j}\\}$. Denote by $\mathbf{U}(x)$ the
column vector $(f_{j}(x))_{j}$. Then the reproducing kernel of $\mathcal{H}$
is given by

(81) $K_{\mathcal{H}}(x,y)=\mathbf{U}(y)^{*}\mathbf{G}^{-1}\mathbf{U}(x),$

where $\mathbf{G}=(\langle
f_{j},f_{k}\rangle_{\mathcal{H}})_{j,k=1}^{\dim\mathcal{H}}$ is the Grammian
matrix of the basis $\\{f_{j}\\}_{j}$.

###### Proof.

Let $\\{e_{l}\\}_{l}$ be an orthonormal basis of $\mathcal{H}$ and let
$e_{l}=\sum_{j}c_{lj}f_{j}$ be the expressions of $e_{l}$ as linear
combinations of the $f_{j}$. Let $C$ denote the matrix
$(c_{lj})_{l,j=1}^{\dim\mathcal{H}}$. From

$\delta_{lm}=\langle
e_{l},e_{m}\rangle_{\mathcal{H}}=\sum_{j,k}c_{lj}\overline{c_{mk}}\langle
f_{j},f_{k}\rangle_{\mathcal{H}}=(C\mathbf{G}C^{*})_{lm}$

we see that $C\mathbf{G}C^{*}$ is the identity matrix; that is,
$\mathbf{G}=(C^{*}C)^{-1}$. Hence by (43)

$\displaystyle K_{\mathcal{H}}(x,y)$
$\displaystyle=\sum_{l}e_{l}(x)\overline{e_{l}(y)}=\sum_{l,j,k}c_{lj}f_{j}(x)\overline{c_{lk}f_{k}(y)}$
$\displaystyle=\mathbf{U}(y)C^{*}C\mathbf{U}(x)=\mathbf{U}(y)\mathbf{G}^{-1}\mathbf{U}(x),$

proving the claim. ∎

For the basis (80), the Grammian matrix $\mathbf{G}$ is in principle easy to
compute explicitly for low values of $m$ and $r$. For instance, for $r=2$,
starting from

$(1-t)^{n}=h(te,e)^{n}=\sum_{{\mathbf{m}}:m_{1}\leq
n}(-n)_{\mathbf{m}}K_{\mathbf{m}}(te,e),\qquad n=0,1,2,\dots,$

one recursively reads off $(-n)_{(n,m_{2})}K_{(n,m_{2})}(te,e)$ as the
homogeneous component of degree $n+m_{2}$ in $(1-t)^{n}$:

$\displaystyle K_{(0)}=\mathbf{1},\qquad K_{(1)}=t_{1}+t_{2},\qquad
K_{(1,1)}=\frac{2t_{1}t_{2}}{a+2},$ $\displaystyle
K_{(2,0)}=\frac{t_{1}^{2}+t_{2}^{2}}{2}+\frac{at_{1}t_{2}}{a+2},\qquad
K_{(2,1)}=\frac{2t_{1}t_{2}(t_{1}+t_{2})}{a+4},$ $\displaystyle
K_{(2,2)}=\frac{2t_{1}^{2}t_{2}^{2}}{(a+2)(a+4)},\qquad\dots\;.$

(For brevity, we have omitted the arguments $(te,e)$.) This reduces the
computation of $\mathbf{G}$ to evaluation of the integrals

(82)
$\int_{0}^{\infty}\int_{0}^{\infty}x_{1}^{q_{1}}x_{2}^{q_{2}}(1+x_{1})^{-\nu}(1+x_{2})^{-\nu}|x_{1}-x_{2}|^{a}\,dx_{1}\,dx_{2}.$

For $a$ an even nonnegative integer, the last integral can be evaluated by
expanding $(x_{1}-x_{2})^{a}$ via the binomial theorem and integrating term by
term using the standard formula

$\int_{0}^{\infty}\frac{x^{q}}{(1+x)^{\nu}}\,dx=\frac{\Gamma(q+1)\Gamma(\nu-q-1)}{\Gamma(\nu)}\equiv
B(q+1,\nu-q-1),\qquad-1<q<\nu-1,$

for the Beta integral. The outcome is that (82) equals

$\sum_{j=0}^{a}\frac{(-a)_{j}}{j!}B(q_{1}+j+1,\nu-q_{1}-a-1)B(q_{2}+a-j+1,\nu-
q_{2}-a-1),\qquad a\in 2{\mathbf{N}}.$

Taking for $\mathbf{U}(x)$ the column vector
$(K_{\mathbf{m}}(xe,e))_{|{\mathbf{m}}|<m,m_{1}<(\nu-p+1)/2}$, one can then
use (81) to obtain a formula for $S^{m}_{\nu}(x,0)$.

For $a\notin 2{\mathbf{N}}$, a possible way of evaluating (82) is first making
the change of variable $x=\frac{t}{1-t}$, which transforms (82) into

(83)
$\displaystyle\int_{0}^{1}\int_{0}^{1}t_{1}^{q_{1}}t_{2}^{q_{2}}(1-t_{1})^{\nu-a-2-q_{1}}(1-t_{2})^{\nu-a-2-q_{2}}|t_{1}-t_{2}|^{a}\,dt_{1}\,dt_{2}$
$\displaystyle\hskip
40.00006pt=\int_{0}^{1}\int_{0}^{1}(1-t_{1})^{q_{1}}(1-t_{2})^{q_{2}}t_{1}^{\nu-a-2-q_{1}}t_{2}^{\nu-a-2-q_{2}}|t_{1}-t_{2}|^{a}\,dt_{1}\,dt_{2}.$

Introducing temporarily the notation

$I(\alpha,\beta,\gamma,\delta):=\int_{0}^{1}\int_{0}^{t_{1}}(1-t_{1})^{\alpha}(1-t_{2})^{\beta}t_{1}^{\gamma}t_{2}^{\delta}|t_{1}-t_{2}|^{a}\,dt_{2}\,dt_{1},$

(83) thus equals

$I(q_{1},q_{2},\nu-a-2-q_{1},\nu-a-2-q_{2})+I(q_{2},q_{1},\nu-a-2-q_{2},\nu-a-2-q_{1}).$

Now making the change of variable $t_{2}=yt_{1}$ yields

$\displaystyle I(\alpha,\beta,\gamma,\delta)$
$\displaystyle=\int_{0}^{1}\int_{0}^{1}(1-t_{1})^{\alpha}(1-yt_{1})^{\beta}t_{1}^{\gamma}(yt_{1})^{\delta}t_{1}^{a}(1-y)^{a}\,t_{1}\,dy\,dt_{1}$
$\displaystyle=\sum_{j=0}^{\infty}\frac{(-\beta)_{j}}{j!}\int_{0}^{1}\int_{0}^{1}(1-t_{1})^{\alpha}(yt_{1})^{j}t_{1}^{\gamma}(yt_{1})^{\delta}t_{1}^{a}(1-y)^{a}\,t_{1}\,dy\,dt_{1}$
(84)
$\displaystyle=\sum_{j=0}^{\infty}\frac{(-\beta)_{j}}{j!}B(\alpha+1,j+\gamma+\delta+a+2)B(a+1,j+\delta+1).$

For $\beta\in{\mathbf{N}}$ — which is our case in (83) — the series
terminates, and one thus has an expression for (82), albeit the formula is a
bit more unwieldy than the one from the previous paragraph.

Similarly, for rank 3, recall that

$K_{\mathbf{m}}(te,e)=\frac{\pi_{\mathbf{m}}}{(q_{\Omega})_{\mathbf{m}}}\phi_{\mathbf{m}}(te),$

with $\pi_{\mathbf{m}}$ and $q_{\Omega}$ given by (37) and (36), respectively,
and $\phi_{\mathbf{m}}$ the spherical polynomial corresponding to the
signature ${\mathbf{m}}$. Using the formula (34), one can again successively
read off $K_{\mathbf{m}}(te,e)$ as the term of the appropriate homogeneity
degree in

$(1-t)^{n}=h(te,e)^{n}=\sum_{{\mathbf{m}}:m_{1}\leq
n}(-n)_{\mathbf{m}}\frac{\pi_{\mathbf{m}}}{(q_{\Omega})_{\mathbf{m}}}\phi_{\mathbf{m}}(te),\qquad
n=0,1,2,\dots.$

This yields (omitting again the argument $te$)

$\displaystyle\phi_{(0)}=\mathbf{1},\qquad\phi_{(1)}=\frac{t_{1}+t_{2}+t_{3}}{3},\qquad\phi_{(1,1)}=\frac{t_{1}t_{2}+t_{1}t_{3}+t_{2}t_{3}}{3},$
$\displaystyle\phi_{(1,1,1)}=t_{1}t_{2}t_{3},\qquad\phi_{(2)}=\frac{(a+2)(t_{1}^{2}+t_{2}^{2}+t_{3}^{2})}{3(3a+2)}+\frac{2a(t_{1}t_{2}+t_{1}t_{3}+t_{2}t_{3})}{3(3a+2)},$
$\displaystyle\phi_{(2,1)}=\frac{(a+1)(t_{1}^{2}t_{2}+t_{1}^{2}t_{3}+t_{2}^{2}t_{1}+t_{2}^{2}t_{3}+t_{3}^{2}t_{1}+t_{3}^{2}t_{2})}{3(3a+2)}+\frac{3at_{1}t_{2}t_{3}}{3(3a+2)},$
$\displaystyle\phi_{(2,1,1)}=\frac{(t_{1}+t_{2}+t_{3})t_{1}t_{2}t_{3}}{3},$
$\displaystyle\phi_{(2,2)}=\frac{(a+2)(t_{1}^{2}t_{2}^{2}+t_{1}^{2}t_{3}^{2}+t_{2}^{2}t_{3}^{2})}{3(3a+2)}+\frac{2a(t_{1}+t_{2}+t_{3})t_{1}t_{2}t_{3}}{3(3a+2)},$
$\displaystyle\phi_{(2,2,1)}=\frac{(t_{1}t_{2}+t_{1}t_{3}+t_{2}t_{3})t_{1}t_{2}t_{3}}{3},\qquad\phi_{(2,2,2)}=t_{1}^{2}t_{2}^{2}t_{3}^{2},\qquad\dots\;.$

This once more reduces the computation of $\mathbf{G}$ to the evaluation of
the three-variable analogue of (82), which for $a\in 2{\mathbf{N}}$ is again
by the same “bare hands” method seen to be equal to

$\sum_{j,k,l=0}^{a}\frac{(-a)_{j}(-a)_{k}(-a)_{l}}{j!k!l!}B(q_{1}+1+j+k,\nu-2a-1-q_{1})\\\
\quad\times B(q_{2}+1+a-j+l,\nu-2a-1-q_{2})\\\ \quad\times
B(q_{3}+1+2a-k-l,\nu-2a-1-q_{3}),\qquad a\in 2{\mathbf{N}}.$

For $a\notin 2{\mathbf{N}}$, one can again proceed as for (84), but the
outcome is quite cumbersome.

Carrying out all these calculations leads to the following conjecture.

Recall that for $\alpha,\beta,\gamma\in{\mathbf{C}}$, the _Faraut-Koranyi
hypergeometric function_ on $\Omega$ with parameters $\alpha,\beta,\gamma$ is
defined by [FK]

(85) ${}_{2}{\mathcal{F}}^{\Omega}_{1}\Big{(}\begin{matrix}\alpha,\beta\\\
\gamma\end{matrix}\Big{|}z\Big{)}:=\sum_{\mathbf{m}}\frac{(\alpha)_{\mathbf{m}}(\beta)_{\mathbf{m}}}{(\gamma)_{\mathbf{m}}}K_{\mathbf{m}}(z,z).$

Here $\gamma$ is assumed to be such that $(\gamma)_{\mathbf{m}}\neq 0$
$\forall{\mathbf{m}}$. Alternatively, one sometimes views these just as
symmetric functions on ${\mathbf{R}}_{+}^{r}$ [Yan]:

(86) ${}_{2}{\mathcal{F}}_{1}\Big{(}\begin{matrix}\alpha,\beta\\\
\gamma\end{matrix}\Big{|}t\Big{)}:=\sum_{\mathbf{m}}\frac{(\alpha)_{\mathbf{m}}(\beta)_{\mathbf{m}}}{(\gamma)_{\mathbf{m}}}K_{\mathbf{m}}(te,e),$

the two variants being related simply by
${}_{2}\\!{\mathcal{F}}^{\Omega}_{1}(z)={}_{2}\\!{\mathcal{F}}_{1}(t)$ for
$z=k\sqrt{t}e$.

######  Conjecture 14.

Assume that $m\geq rq+1$ where $q$ is the nonnegative integer such that
$q<\frac{\nu-p+1}{2}\leq q+1$. Then the reproducing kernel $S^{m}_{\nu}$ on
${\mathbf{R}}_{+}^{r}$ at the origin is given by

(87)
$S^{m}_{\nu}(x,0)=c^{q}_{\nu}{}_{2}{\mathcal{F}}_{1}\Big{(}\begin{matrix}-q,-b-\nu+2p+q-2r\\\
p\end{matrix}\Big{|}-x\Big{)},$

where

(88) $c^{q}_{\nu}=\frac{\Gamma_{\Omega}(\nu-
p-q+2r+b)\Gamma_{\Omega}(q_{\Omega})\Gamma_{\Omega}(p+q)}{\pi^{d}\Gamma_{\Omega}(\nu-
p-q+2r-q_{\Omega})\Gamma_{\Omega}(q_{\Omega}+q)\Gamma_{\Omega}(p)}.$

The last conjecture holds for $r=1$, by (8), (15), Theorem 6 and (78). It has
also been verified by computer for

$\displaystyle
r=2,\;q\in\\{0,1,2\\},\;a\in\\{1,2,3,4,5,6,7,8\\},\;b\in\\{0,1,2,3\\},\text{
$\nu$ arbitrary,}$ $\displaystyle
r=2,\;q=3,\;a\in\\{1,2,3,4\\},\;b\in\\{0,1,2,3\\},\text{ $\nu$ arbitrary,}$
$\displaystyle
r=3,\;q\in\\{0,1,2\\},\;a\in\\{2,4\\},\;b\in\\{0,1,2,3\\},\text{ $\nu$
arbitrary,}$ $\displaystyle r=3,\;q\in\\{0,1,2\\},\;a=8,\;b=0,\text{ $\nu$
arbitrary,}$

and a couple more values of $a$, $b$ and $\nu$ for $r\in\\{2,3\\}$ and
$q\in\\{0,1,2\\}$. (Note that the above values of $r,a,b$ include, in
particular, both exceptional bounded symmetric domains of dimensions 16 and
27.)

Note that the hypothesis of the conjecture, that is,

(89) $q-1<\frac{\nu-p-1}{2}\leq q\leq\frac{m-1}{r},$

corresponds precisely to the case (57) of the “stabilized” kernels from
Corollary 9. Without this hypothesis, the conjecture fails, as the following
example shows.

######  Example 15.

Let $r=2$, $m=2$ and $\nu-p>1$ (note that this corresponds to the case (56) in
Corollary 9). The space $\mathcal{S}^{m}_{\nu}$ is thus spanned by
$K_{(0)}(xe,e)=\mathbf{1}$ and $K_{(1)}(xe,e)=x_{1}+x_{2}$. Performing the
calculations outlined above yields

$\tfrac{1}{C}S^{m}_{\nu}(x,0)=K_{(0)}+\frac{(b-\nu+a+3)(2b-2\nu+a+4)}{a^{2}+(7+4b-2\nu)a+(4b^{2}-4b\nu+16b-6\nu+14)}K_{(1)}$

with some constant $C$. (We have omitted the arguments $(xe,e)$ at $K_{(0)}$
and $K_{(1)}$.) Plainly, the right-hand side is not of the form
${}_{2}{\mathcal{F}}_{1}$. ∎

In the remaining case from Corollary 9 (i.e. $q+1<m<rq+1$), the kernels can be
expected to be even more “ugly” than in the last example.

By the results of the preceding sections, the validity of the conjecture would
have the following consequences.

######  Corollary 16.

(Subject to Conjecture 14) Assume that $m\geq rq+1$ where $q$ is the
nonnegative integer such that $q<\frac{\nu-p+1}{2}\leq q+1$. Then the nearly-
holomorphic reproducing kernel $N^{m}_{\nu}$ at the origin is given by

$N^{m}_{\nu}(k\sqrt{t}e,0)=c^{q}_{\nu}{}_{2}{\mathcal{F}}_{1}\Big{(}\begin{matrix}-q,-b-\nu+2p+q-2r\\\
p\end{matrix}\Big{|}\frac{t}{t-1}\Big{)},$

or

$N^{m}_{\nu}(z,0)=c^{q}_{\nu}h(z,z)^{-q}{}_{2}{\mathcal{F}}^{\Omega}_{1}\Big{(}\begin{matrix}-q,b+\nu-
p-q+2r\\\ p\end{matrix}\Big{|}z\Big{)}.$

###### Proof.

By Theorem 6,

$N^{m}_{\nu}(k\sqrt{t}e,0)=S^{m}_{\nu}(\tfrac{t}{1-t},0),$

and (87) gives the first formula. The second formula then follows from the
Kummer relation (a counterpart of (78) for the ordinary ${}_{2}\\!F_{1}$)

(90) ${}_{2}{\mathcal{F}}_{1}\Big{(}\begin{matrix}\alpha,\beta\\\
\gamma\end{matrix}\Big{|}t\Big{)}=(1-t)^{-\alpha}{}_{2}{\mathcal{F}}_{1}\Big{(}\begin{matrix}\alpha,\gamma-\beta\\\
\gamma\end{matrix}\Big{|}\frac{t}{t-1}\Big{)},$

see [Yan, formula (35)]. ∎

######  Theorem 17.

(Subject to Conjecture 14) Assume that $m\geq rq+1$ where $q$ is the
nonnegative integer such that $q<\frac{\nu-p+1}{2}\leq q+1$. Then with the
notation (73), (69) and (85),

(91) $\displaystyle\sum_{|{\mathbf{m}}|:\;m_{1}\leq
q}d_{r}(\lambda_{\mathbf{m}},\nu)F(\lambda_{\mathbf{m}},\mathbf{k}_{\nu},g)=$
$\displaystyle\hskip
40.00006pth(z,z)^{-q}c^{q}_{\nu}{}_{2}{\mathcal{F}}^{\Omega}_{1}\Big{(}\begin{matrix}-q,b+\nu-
p-q+2r\\\ p\end{matrix}\Big{|}z\Big{)}$

for $z=g0$ with $g\in A$.

###### Proof.

Since $J_{g}(0)=g(g0,g0)^{p/2}$ for $g\in A$, (64) and (74) yield

$\phi_{\lambda,\nu}^{\flat}(g0)=F(\lambda,\mathbf{k}_{\nu},g0).$

Thus by (71), the left-hand side of (91) equals $N^{m}_{\nu}(g0,0)$. By
Corollary 16, the latter is precisely the right-hand side of (91). ∎

Note that for $r=1$, (91) recovers the formula (14) from the Introduction.

######  Remark 18.

By Theorem 4.2 of Beerends and Opdam [BO], $F(\lambda,\mathbf{k}_{\nu},\cdot)$
for the special value

$\lambda=-\alpha\sum_{j}\beta_{j}+\rho_{\nu},\qquad\alpha\in{\mathbf{C}},$

can be expressed in terms of

${}_{2}{\mathcal{F}}_{1}\Big{(}\begin{matrix}\alpha,d/r+\nu-\alpha\\\
d/r\end{matrix}\Big{|}\cdot\Big{)};$

however the ${}_{2}\\!{\mathcal{F}}_{1}$ in (91) does not seem reducible to
this form. ∎

Using again Theorem 6, the conjecture also implies a formula for the
invariantly-polyanalytic kernel $S^{m}_{\nu}(x,0)$.

######  Corollary 19.

(Subject to Conjecture 14) Assume that $m\geq rq+1$ where $q$ is the
nonnegative integer such that $q<\frac{\nu-p-1}{2}+m\leq q+1$. Then the
reproducing kernel $P^{m}_{\nu}$ at the origin is given by

$P^{m}_{\nu}(z,0)=c^{m}_{\nu+2m-2}h(z,z)^{m-1-q}{}_{2}{\mathcal{F}}_{1}\Big{(}\begin{matrix}-q,b+\nu+2m-2-p-q+2r\\\
p\end{matrix}\Big{|}z\Big{)}.$

###### Proof.

By Theorem 6, $P^{m}_{\nu}(z,0)=h(z,z)^{m-1}N^{m}_{\nu+2m-2}(z,0)$, and the
claim follows by Corollary 16. ∎

######  Example 20.

Continuing our example of $\Omega={\mathbf{B}}^{d}$ from the previous section,
for rank 1 the Faraut-Koranyi hypergeometric function coincides with the
ordinary Gauss hypergeometric function

${}_{2}\\!{\mathcal{F}}^{{\mathbf{B}}^{d}}_{1}\Big{(}\begin{matrix}\alpha,\beta\\\
\gamma\end{matrix}\Big{|}z\Big{)}={}_{2}\\!F_{1}\Big{(}\begin{matrix}\alpha,\beta\\\
\gamma\end{matrix}\Big{|}|z|^{2}\Big{)}.$

By (79) and (87) we therefore get, for $0\leq q<\frac{\nu-d}{2}\leq q+1\leq
m$,

$\sum_{j=0}^{q}d_{1}(\lambda_{(j)},\nu){}_{2}\\!F_{1}\Big{(}\begin{matrix}-j,d-\nu+j\\\
d\end{matrix}\Big{|}-x\Big{)}=c^{q}_{\nu}{}_{2}\\!F_{1}\Big{(}\begin{matrix}-q,d+q+1-\nu\\\
d+1\end{matrix}\Big{|}-x\Big{)}.$

This is, of course, just (14) in disguise. ∎

######  Remark 21.

The formula (79) actually shows that
$d_{1}(\lambda_{(j)},\nu){}_{2}\\!F_{1}\Big{(}\begin{matrix}-j,d-\nu+j\\\
d\end{matrix}\Big{|}-x\Big{)}$ is the reproducing kernel of the orthogonal
complement $\mathcal{S}^{j}_{\nu}\ominus\mathcal{S}^{j-1}_{\nu}$, $0\leq
j<\frac{\nu-d}{2}$ (with $\mathcal{S}^{-1}_{\nu}:=\\{0\\}$). Theorem V.4.5 in
Helgason [He] identifies the last ${}_{2}\\!F_{1}$ as the spherical function
for the compact dual $SU(d+1)/SU(d)={\mathbf{C}}P^{d}$ of ${\mathbf{B}}^{d}$.
∎

By the reproducing property, Conjecture 14 is equivalent to

(92)
$\displaystyle\int_{{\mathbf{R}}_{+}^{r}}K_{\mathbf{m}}(xe,e)\;{}_{2}{\mathcal{F}}_{1}\Big{(}\begin{matrix}-q,-b-\nu+2p+q-2r\\\
p\end{matrix}\Big{|}-x\Big{)}\,d\rho_{b,\nu,a}(x)$ $\displaystyle\hskip
80.00012pt=\frac{1}{c^{q}_{\nu}}\delta_{{\mathbf{m}},(0)},\qquad\forall{\mathbf{m}}\text{
with }m_{1}\leq q,$

where $q$ is the nonnegative integer such that $q<\frac{\nu-p+1}{2}\leq q+1$
and $m\geq rq+1$. Taking in particular $m=(0)$ yields

$\frac{1}{c^{q}_{\nu}}=\int_{{\mathbf{R}}_{+}^{r}}{}_{2}{\mathcal{F}}_{1}\Big{(}\begin{matrix}-q,-b-\nu+2p+q-2r\\\
p\end{matrix}\Big{|}-x\Big{)}\,d\rho_{b,\nu,a}(x)$

(subject to the validity of Conjecture 14). The last integral can be evaluated
explicitly.

######  Proposition 22.

$\int_{{\mathbf{R}}_{+}^{r}}{}_{2}{\mathcal{F}}_{1}\Big{(}\begin{matrix}-q,-b-\nu+2p+q-2r\\\
p\end{matrix}\Big{|}-x\Big{)}\,d\rho_{b,\nu,a}(x)\\\
=\frac{1}{c_{\nu-q}}\;{}_{3}{\mathcal{F}}^{\Omega}_{2}\Big{(}\begin{matrix}-q,b+\nu-
p-q+2r,d/r\\\ p,\nu-q\end{matrix}\Big{|}e\Big{)},$

where the Faraut-Koranyi function ${}_{3}{\mathcal{F}}^{\Omega}_{2}$ is
defined analogously as in (85).

###### Proof.

Making again the change of variable $x=\frac{t}{1-t}$, we get from (54) and
(86)

$\displaystyle\int_{{\mathbf{R}}_{+}^{r}}{}_{2}{\mathcal{F}}_{1}\Big{(}\begin{matrix}-q,-b-\nu+2p+q-2r\\\
p\end{matrix}\Big{|}-x\Big{)}\,d\rho_{b,\nu,a}(x)$ $\displaystyle\hskip
20.00003pt=c_{\Omega}\int_{[0,1]^{r}}{}_{2}{\mathcal{F}}_{1}\Big{(}\begin{matrix}-q,-b-\nu+2p+q-2r\\\
p\end{matrix}\Big{|}\frac{t}{t-1}\Big{)}\,d\mu_{b,\nu,a}(t)$
$\displaystyle\hskip
20.00003pt=c_{\Omega}\int_{[0,1]^{r}}(1-t)^{-q}{}_{2}{\mathcal{F}}_{1}\Big{(}\begin{matrix}-q,b+\nu-
p-q+2r\\\ p\end{matrix}\Big{|}t\Big{)}\,d\mu_{b,\nu,a}(t)$
$\displaystyle\hskip
20.00003pt=c_{\Omega}\int_{[0,1]^{r}}{}_{2}{\mathcal{F}}_{1}\Big{(}\begin{matrix}-q,b+\nu-
p-q+2r\\\ p\end{matrix}\Big{|}t\Big{)}\,d\mu_{b,\nu-q,a}(t)$
$\displaystyle\hskip
20.00003pt=c_{\Omega}\sum_{|{\mathbf{m}}|<m}\frac{(-q)_{\mathbf{m}}(b+\nu-
p-q+2r)_{\mathbf{m}}}{(p)_{\mathbf{m}}}\int_{[0,1]^{r}}K_{\mathbf{m}}(te,e)\,d\mu_{b,\nu-q,a}(t).$

If $\\{\psi_{j}\\}_{j=1}^{d_{\mathbf{m}}}$ is an orthonormal basis of
${\mathbf{P}}_{\mathbf{m}}$ with respect to the Fock norm, the last integral
equals, by (43),

$\displaystyle\int_{[0,1]^{r}}K_{\mathbf{m}}(\sqrt{t}e,\sqrt{t}e)\,d\mu_{b,\nu-q,a}(t)$
$\displaystyle=\int_{K}\int_{[0,1]^{r}}K_{\mathbf{m}}(k\sqrt{t}e,k\sqrt{t}e)\,d\mu_{b,\nu-q,a}(t)\,dk$
$\displaystyle=\frac{1}{c_{\Omega}}\int_{\Omega}K_{\mathbf{m}}(z,z)\,d\mu_{\nu-q}(z)\qquad\text{by
\eqref{tME}}$
$\displaystyle=\frac{1}{c_{\Omega}}\int_{\Omega}\sum_{j}|\psi_{j}(z)|^{2}\,d\mu_{\nu-q}(z)$
$\displaystyle=\frac{1}{c_{\Omega}}\sum_{j}\|\psi_{j}\|^{2}_{\nu-q}$
$\displaystyle=\frac{1}{c_{\Omega}}\sum_{j}\frac{\|\psi_{j}\|^{2}_{F}}{(\nu-q)_{\mathbf{m}}c_{\nu-q}}\qquad\text{by
\eqref{tAU}}$
$\displaystyle=\frac{d_{\mathbf{m}}}{c_{\Omega}(\nu-q)_{\mathbf{m}}c_{\nu-q}}.$

Consequently,

$\displaystyle\int_{{\mathbf{R}}_{+}^{r}}{}_{2}{\mathcal{F}}_{1}\Big{(}\begin{matrix}-q,-b-\nu+2p+q-2r\\\
p\end{matrix}\Big{|}-x\Big{)}\,d\rho_{b,\nu,a}(x)$ $\displaystyle\hskip
10.00002pt=\sum_{|{\mathbf{m}}|<m}\frac{(-q)_{\mathbf{m}}(b+\nu-
p-q+2r)_{\mathbf{m}}}{(p)_{\mathbf{m}}}\frac{d_{\mathbf{m}}}{(\nu-q)_{\mathbf{m}}c_{\nu-q}}$
$\displaystyle\hskip
10.00002pt=\sum_{|{\mathbf{m}}|<m}\frac{(-q)_{\mathbf{m}}(b+\nu-
p-q+2r)_{\mathbf{m}}}{(p)_{\mathbf{m}}}\frac{(d/r)_{\mathbf{m}}}{(\nu-q)_{\mathbf{m}}c_{\nu-q}}K_{\mathbf{m}}(e,e)$
$\displaystyle\hskip
10.00002pt=\frac{1}{c_{\nu-q}}\;{}_{3}{\mathcal{F}}^{\Omega}_{2}\Big{(}\begin{matrix}-q,b+\nu-
p-q+2r,d/r\\\ p,\nu-q\end{matrix}\Big{|}e\Big{)},$

as claimed. Here the second equality is due to (35). ∎

The formula (88) thus gives a conjectured value for this
${}_{3}{\mathcal{F}}_{2}$ function.

For rank 1, we have $b+\nu-p-q+2r=\nu-q$, so the ${}_{3}\\!F_{2}$ becomes
${}_{2}\\!F_{1}$ and (88) follows by the standard formula for
${}_{2}\\!F_{1}\Big{(}\begin{matrix}a,b\\\ c\end{matrix}\Big{|}1\Big{)}$.

##  6\. Compact Hermitian symmetric spaces

We now consider also the compact duals of Hermitian symmetric spaces
$\hat{\Omega}$, the simplest examples of these being the complex projective
space ${\mathbf{C}}P^{d}$ as the compact dual of the unit ball
${\mathbf{B}}^{d}$ (including, in particular, the Riemann sphere
${\mathbf{C}}P^{1}$ as the compact dual of the unit disc). Most results are
obtained by formally replacing $\nu$ by $-\nu$, $h(z,z)$ by $h(z,-z)$, and
$\Omega\subset{\mathbf{C}}^{d}$ by the open chart
${\mathbf{C}}^{d}\subset\hat{\Omega}$. We shall be rather brief.

The symmetric space $\Omega=G/K$ has its compact dual $\hat{\Omega}=\hat{G}/K$
where $\hat{G}$ is a simply connected compact Lie group with Lie algebra
$\hat{\mathfrak{g}}=\mathfrak{k}+i\mathfrak{p}$. There is a dense open subset
of $\hat{\Omega}$ that is biholomorphic to ${\mathbf{C}}^{d}$, and we shall
simply identify this local chart with ${\mathbf{C}}^{d}$ throughout. The
stabilizer subgroup $K$ of the origin in $\hat{G}$ is the same as in the
bounded case. For $x\in\hat{\Omega}$, there is again a unique geodesic
symmetry $\hat{\phi}_{x}\in\hat{G}$ which interchanges $x$ and the origin,
i.e. $\hat{\phi}_{x}\circ\hat{\phi}_{x}=\text{id}$, $\hat{\phi}_{x}(0)=x$,
$\hat{\phi}_{x}(x)=0$, and $\hat{\phi}_{x}$ has only isolated fixed points.
Any $g\in\hat{G}$ can be uniquely written in the form $g=\hat{\phi}_{x}k$ with
$k\in K$ and $x=g0\in\hat{\Omega}$. The measure

$d\hat{\mu}_{\nu}(z):=h(z,-z)^{-\nu-p}\,dz$

on ${\mathbf{C}}^{d}\subset\hat{\Omega}$ is finite if and only of $\nu>-1$,
and one can again consider the spaces

$\hat{A}_{\nu}:=L^{2}(\hat{\Omega},d\hat{\mu}_{\nu})\cap\mathcal{O}({\mathbf{C}}^{d}).$

The elements of $\hat{A}_{\nu}$ extend to holomorphic sections on all of
$\hat{\Omega}$ if and only if $\nu$ is an _integer_ , which we will assume
from now on throughout the rest of this section. In that case,

$\hat{A}_{\nu}=\bigoplus_{{\mathbf{m}}:m_{1}\leq\nu}{\mathbf{P}}_{\mathbf{m}},$

and $\hat{A}_{\nu}$ possesses a reproducing kernel, given by

$\hat{K}_{\nu}(z,w)=\hat{c}_{\nu}\;h(z,-w)^{\nu},\qquad
z,w\in{\mathbf{C}}^{d}\subset\hat{\Omega},\;\nu\in{\mathbf{N}},$

where

$\hat{c}_{\nu}=\frac{\Gamma_{\Omega}(\nu+p)}{\pi^{d}\Gamma_{\Omega}(\nu+p-\frac{d}{r})}.$

(Here, as before, $p$, $r$, $a$ and $b$ denote the genus, the rank, and the
characteristic multiplicities of $\hat{\Omega}$, which are all the same as for
$\Omega$.) From the transformation rule

$h(\hat{\phi}z,-\hat{\phi}w)=\frac{h(a,-a)h(z,-w)}{h(z,-a)h(a,-w)},\qquad
a=\hat{\phi}^{-1}0,\qquad z,w\in{\mathbf{C}}^{d},\;\hat{\phi}\in\hat{G},$

it again follows that the measure $d\hat{\mu}_{0}$ is $\hat{G}$-invariant and
that $\hat{\Psi}(z):=-\log h(z,-z)$ is the Kähler potential for a
$\hat{G}$-invariant Riemannian metric on $\hat{\Omega}$. We thus again have
the associated Cauchy-Riemann operator $\overline{D}$, and the corresponding
spaces $\hat{\mathcal{N}}^{m}:=\operatorname{Ker}\overline{D}{}^{m}$ of nearly
holomorphic functions on $\hat{\Omega}$ of order $m$, as well as their
Bergman-type subspaces

$\hat{\mathcal{N}}^{m}_{\nu}:=L^{2}(\hat{\Omega},d\hat{\mu}_{\nu})\cap\operatorname{Ker}\overline{D}{}^{m}.$

One can also proceed to define the invariantly polyanalytic functions
$\hat{\mathcal{P}}^{m}$ and their Bergman-type subspaces
$\hat{\mathcal{P}}^{m}_{\nu}$ as in the bounded case.

In the polar coordinates (22), the measures $d\hat{\mu}_{\nu}$ assume the form

(93) $\int_{\hat{\Omega}}f(z)\,d\hat{\mu}_{\nu}(z)=\quad
c_{\Omega}\int_{{\mathbf{R}}_{+}^{r}}\int_{K}f(k\sqrt{t}e)\,dk\,d\hat{\mu}_{b,\nu,a}(t),$

where

(94) $d\hat{\mu}_{b,\nu,a}(t):=t^{b}(1+t)^{-\nu-p}\prod_{1\leq i<j\leq
r}|t_{i}-t_{j}|^{a}\,dt$

and $c_{\Omega}$ is given by (29).

By the above transformation rule for $h(z,-w)$, it again also follows that

$f\longmapsto\frac{h(a,-a)^{-\nu/2}}{h(z,-a)^{-\nu}}f\circ\hat{\phi}^{-1},\qquad
a=\hat{\phi}0,\;\hat{\phi}\in\hat{G},\;\nu\in{\mathbf{N}},$

is a projective unitary representation of $\hat{G}$ on
$\hat{\mathcal{N}}^{m}_{\nu}$. Let $\hat{\mathcal{S}}^{m}$ be the vector space
of all symmetric polynomials of degree $<m$ in $r$ variables, denote

$\hat{\mathcal{S}}^{m}_{\nu}:=\hat{\mathcal{S}}^{m}\cap
L^{2}([0,1]^{r},d\hat{\rho}_{b,\nu,a}),$

where

$d\hat{\rho}_{b,\nu,a}(t):=c_{\Omega}\;t^{b}(1-t)^{\nu}\prod_{1\leq i<j\leq
r}|t_{i}-t_{j}|^{a}\,dt,$

and let $\hat{S}^{m}_{\nu}(x,y)$ be the reproducing kernel of
$\hat{\mathcal{S}}^{m}_{\nu}$. Proceeding as in Section 3 above, we then
obtain the following analogue of Theorem 6.

######  Theorem 23.

  * (a)

For any $\hat{\phi}\in\hat{G}$, the reproducing kernel $\hat{N}^{m}_{\nu}$ of
$\hat{\mathcal{N}}^{m}_{\nu}$ satisfies

$\hat{N}^{m}_{\nu}(z,w)=\frac{h(z,-a)^{\nu}h(a,-w)^{\nu}}{h(a,-a)^{\nu}}\hat{N}^{m}_{\nu}(\hat{\phi}z,\hat{\phi}w),\qquad
a:=\hat{\phi}^{-1}0;$

in particular,

$\hat{N}^{m}_{\nu}(z,w)=h(z,-w)^{\nu}\hat{N}^{m}_{\nu}(\hat{\phi}_{w}z,0).$

  * (b)

Radial functions in $\hat{\mathcal{N}}^{m}$ consist precisely of functions of
the form

$p(z,(-z)^{z}),$

where $p(z,w)$ is a polynomial in $z,\overline{w}\in{\mathbf{C}}^{d}$ of
degree $<m$ in each argument which is $K$-invariant in the sense of (48).

  * Consequently, the radial functions in $\hat{\mathcal{N}}^{m}$ coincide with the linear span of $K_{\mathbf{m}}(z,(-z)^{z})$, $|{\mathbf{m}}|<m$.

  * (c)

The mapping $\hat{V}$ from $\hat{\mathcal{S}}^{m}$ into functions on
$\hat{\Omega}$ given by

$\hat{V}f(k\sqrt{x}e):=f\Big{(}\frac{x}{1+x}\Big{)},\qquad k\in
K,\;x\in{\mathbf{R}}_{+}^{r},$

is a bijection from $\hat{\mathcal{S}}^{m}$ onto radial functions in
$\hat{\mathcal{N}}^{m}$. Furthermore, $\hat{V}$ sends
$\hat{\mathcal{S}}^{m}_{\nu}$ unitarily onto the subspace
$\hat{\mathcal{R}}^{m}_{\nu}$ of radial function in
$\hat{\mathcal{N}}^{m}_{\nu}$, and

$\hat{N}^{m}_{\nu}(\cdot,0)=V\hat{S}^{m}_{\nu}(\cdot,0).$

###### Proof.

The proof is the same as for Propositions 2, 4 and 5, hence omitted. ∎

Unlike the bounded case, for the compact dual we can give an explicit formula
for the kernel $\hat{N}^{m}_{\nu}$ in terms of multivariable Jacobi
polynomials $P^{(\alpha,\beta,a/2)}_{\mathbf{m}}$ (also called Heckman-Opdam
polynomials; see [HS, Section 1.3]). Recall from [De3, Section 4.b] that
$P^{(\alpha,\beta,a/2)}_{\mathbf{m}}(t)$ are symmetric polynomials on
${\mathbf{R}}^{r}$ such that

  * (i)

$P^{(\alpha,\beta,a/2)}_{\mathbf{m}}(t)$ is the symmetrization of

(95)
$t^{\mathbf{m}}+\sum_{{\mathbf{n}}<{\mathbf{m}}}c_{{\mathbf{m}}{\mathbf{n}}}t^{\mathbf{n}}$

where the sum is over (some) signatures ${\mathbf{n}}$ smaller than
${\mathbf{m}}$ with respect to the lexicographic order; and

  * (ii)

$P^{(\alpha,\beta,a/2)}_{\mathbf{m}}(t)$, $|{\mathbf{m}}|\geq 0$, are
orthogonal on $[-1,+1]^{r}$ with respect to the measure

$(1-t)^{\alpha}(1+t)^{\beta}\prod_{1\leq i<j\leq r}|t_{i}-t_{j}|^{a}\,dt.$

By change of variable, it follows that
$P^{(\alpha,\beta,a/2)}_{\mathbf{m}}(1-2t)$ are orthogonal on $[0,1]^{r}$ with
respect to the measure $t^{\alpha}(1-t)^{\beta}\prod_{1\leq i<j\leq
r}|t_{i}-t_{j}|^{a}\,dt$, with the norm-square on $[-1,+1]^{r}$ given by
$2^{r(r-1)a/2+r\alpha+r\beta+r}$ times the norm-square on $[0,1]^{r}$. Setting
in particular $\alpha=b$, $\beta=\nu$ we get an orthogonal basis for symmetric
polynomials with respect to $d\hat{\rho}_{b,\nu,a}$ on $[0,1]^{r}$. By (43),
we thus arrive at the following theorem.

######  Theorem 24.

For $\nu\in{\mathbf{N}}$, the reproducing kernel $\hat{S}^{m}_{\nu}$ at the
origin is given by

(96)
$\hat{S}^{m}_{\nu}(t,0)=\sum_{|{\mathbf{m}}|<m}\frac{2^{d+r\nu}P^{(b,\nu,a/2)}_{\mathbf{m}}(1)}{\|P^{(b,\nu,a/2)}_{\mathbf{m}}\|^{2}}\;P^{(b,\nu,a/2)}_{\mathbf{m}}(1-2t).$

Here the norm-square is understood on $[-1,+1]^{r}$.

We remark that explicit formulas both for $P^{(b,\nu,a/2)}_{\mathbf{m}}(1)$
and for $\|P^{(b,\nu,a/2)}_{\mathbf{m}}\|^{2}$ are available, see Theorems
3.5.5 and 3.6.6 in [HS].

######  Example 25.

For rank $r=1$, $P^{(b,\nu,a/2)}_{\mathbf{m}}$ are up to a constant factor
just the ordinary Jacobi polynomials $P^{(b,\nu)}_{n}$ of degree $n$ on
$[-1,+1]$:

$P^{(b,\nu,a/2)}_{(n)}(t)=\frac{2^{n}}{\binom{2n+b+\nu}{n}}P^{(b,\nu)}_{n}(t).$

From the formulas [BE2, Section 10.8]

$P^{(b,\nu)}_{n}(1)=\binom{n+b}{n},\qquad\|P^{(b,\nu)}_{n}\|^{2}=\frac{2^{b+\nu+1}\Gamma(n+b+1)\Gamma(n+\nu+1)}{n!(2n+b+\nu+1)\Gamma(n+b+\nu+1)},$

we therefore get

$\hat{S}^{m}_{\nu}(t,0)=\sum_{j=0}^{m-1}\frac{\Gamma(j+\nu+1)}{(d-1)!j!^{2}(d+2j+\nu)\Gamma(d+j+\nu)}\;P^{(d-1,\nu)}_{j}(1-2t).\qed$

Using Theorem 23, we can also obtain from (96) a formula for the nearly-
holomorphic reproducing kernel $\hat{N}^{m}_{\nu}(z,w)$ on $\hat{\Omega}$ in
terms of multivariable Jacobi polynomials.

######  Corollary 26.

For $\nu\in{\mathbf{N}}$, the nearly-holomorphic reproducing kernel
$\hat{N}^{m}_{\nu}$ is given by
$\hat{N}^{m}_{\nu}(z,w)=h(z,-w)^{\nu}\hat{N}^{m}_{\nu}(\hat{\phi}_{w}z,0)$,
where

$\hat{N}^{m}_{\nu}(k\sqrt{x}e,0)=\sum_{|{\mathbf{m}}|<m}\frac{2^{d+r\nu}P^{(b,\nu,a/2)}_{\mathbf{m}}(1)}{\|P^{(b,\nu,a/2)}_{\mathbf{m}}\|^{2}}\;P^{(b,\nu,a/2)}_{\mathbf{m}}\Big{(}\frac{1-x}{1+x}\Big{)}.$

###### Proof.

Straightforward from Theorem 23 and (96). ∎

One can also get the invariantly-polyanalytic kernels $\hat{P}^{m}_{\nu}$. We
leave the details (which are utterly routine) to the interested reader.

######  Remark 27.

Note that in (96) there is no restriction on $m_{1}$ in the sum, in contrast
to Corollary 9 or (71); the reason being, of course, that
$d\hat{\rho}_{b,\nu,a}$ is a finite measure on $[0,1]^{r}$ for
$\nu\in{\mathbf{N}}$, so that the corresponding $L^{2}$ space contains all
polynomials. Still, proceeding as in Section 5, one can get the following
analogue of Conjecture 14 for the compact case. For $q,\nu\in{\mathbf{N}}$,
let $\mathcal{Q}^{q}_{\nu}$ be the subspace of
$L^{2}([0,1]^{r},d\hat{\rho}_{b,\nu,a})$ spanned by
$\\{K_{\mathbf{m}}(te,e):\;m_{1}\leq q\\}$, and let $Q^{q}_{\nu}$ be its
reproducing kernel. Then it seems that

(97)
$Q^{q}_{\nu}(t,0)=\hat{c}^{q}_{\nu}\;{}_{2}{\mathcal{F}}_{1}\Big{(}\begin{matrix}-q,\nu+p+q\\\
p\end{matrix}\Big{|}t\Big{)},$

where

$\hat{c}^{q}_{\nu}=\frac{\Gamma_{\Omega}(p+q)\Gamma_{\Omega}(p+q+\nu)\Gamma_{\Omega}(q_{\Omega})}{\pi^{d}\Gamma_{\Omega}(p)\Gamma_{\Omega}(q+q_{\Omega})\Gamma_{\Omega}(q+q_{\Omega}+\nu)}.$

This has been checked for the same set of values of $r,q,a,b$ as for
Conjecture 14.

Note that in view of (55) and (95), the Jacobi polynomials
$P^{(b,\nu,a/2)}_{\mathbf{m}}(1-2t)$ with $m_{1}\leq q$ form an orthogonal
basis for $\mathcal{Q}^{q}_{\nu}$, thus again by (43)

$Q^{q}_{\nu}(t,0)=\sum_{|{\mathbf{m}}|:\,m_{1}\leq
q}\frac{2^{d+r\nu}P^{(b,\nu,a/2)}_{\mathbf{m}}(1)}{\|P^{(b,\nu,a/2)}_{\mathbf{m}}\|^{2}}\;P^{(b,\nu,a/2)}_{\mathbf{m}}(1-2t).$

Hence (97) gives a conjectured value for this sum. ∎

We conclude this section by deriving the counterpart of Section 4, i.e. the
representation theory of for the $L^{2}$ spaces of sections of line bundles —
especially the results of [Zh3] — for the compact case. We give a
representation theoretic proof of Corollary 26.

We follow the presentation as in [Lo]. We consider the holomorphic line bundle
$\mathcal{L}$ over $\hat{G}/K$,

(98) $\hat{G}\times_{K,\tau}{\mathbf{C}}\to\hat{\Omega}=\hat{G}/K,$

where $\tau(k)=(\det\text{Ad}(k)|_{\mathfrak{p}^{+}})^{1/p}$, $k\in K$. This
is the holomorphic line bundle such that $\mathcal{L}^{p}=\mathcal{K}^{-1}$
and it generates the Picard group of $\hat{\Omega}$; see [Lo, 7.1-7.11]. Here
$\mathcal{K}^{-1}$ is the dual of the canonical line bundle. Let
$\tau_{\nu}=\tau^{\nu}$ for any fixed integer $\nu$, where as before we assume
that $\nu\geq 0$.

Let $L^{2}(\hat{\Omega};\nu)$ be the $L^{2}$-space of sections of the line
bundle $\mathcal{L}^{\nu}$. We normalize the measure so that the realization
of sections $f\in L^{2}(\hat{\Omega};\nu)$ as functions on $L^{2}(\hat{G})$ is
an isometry. More precisely $L^{2}(\hat{\Omega};\nu)$ consists of $f\in
L^{2}(\hat{G})$ such that

$\tau_{\nu}(k)f(gk)=f(g),\qquad k\in K,$

and

$\|f\|_{\nu}^{2}=\int_{\hat{G}}|f(g)|^{2}\,dg<\infty,$

where $dg$ is the Haar measure on $\hat{G}$ normalized so that
$\int_{\hat{G}}dg=1$.

The space $V:={\mathbf{C}}^{d}$ can be realized as an open subset in
$\hat{\Omega}$ and we shall realize the space $L^{2}(\hat{\Omega};\nu)$ as
point-wise functions on $V$. Under our assumption $\nu\geq 0$ the space of
holomorphic sections of the line bundle (98) is non-zero, and there exists a
global frame $e_{\nu}(z)$ with point-wise norm

$\|e_{\nu}(z)\|_{z}^{2}=h(z,-z)^{-\nu}.$

Then a section $f\in L^{2}(\hat{\Omega};\nu)$ will be written as
$f=f(z)e_{\nu}(z)$ for a point-wise function on $V$ such that

$\|f\|^{2}_{\nu}=\hat{c}_{\nu}\int_{V}|f(z)|^{2}h(z,-z)^{-\nu}d\mu_{0}(z),\qquad
f=f(z)e_{\nu}(z),$

where

$d\mu_{0}(z)=\frac{dz}{h(z,-z)^{p}}$

is the $\hat{G}$-invariant (Kähler) measure on $\hat{\Omega}$. To avoid
confusion we write $L^{2}(V,\nu)$ for the space of $L^{2}$-functions $f(z)$
with the above norm. As an $L^{2}$-space and unitary representation of
$\hat{G}$, $L^{2}(\hat{\Omega};\nu)=L^{2}(V,\nu)$ via this identification.

Let
$\mathfrak{g}^{{\mathbf{C}}}=\mathfrak{p}^{+}+\mathfrak{k}^{{\mathbf{C}}}+\mathfrak{p}^{-}$
be the Harish-Chandra decomposition of $\mathfrak{g}^{{\mathbf{C}}}$. We use
the same complex structure for $\Omega$ as for $\hat{\Omega}$, so that
$\mathfrak{p}^{+}=T_{0}^{(1,0)}(\hat{\Omega})\equiv V$ is the holomorphic
tangent space at $0\in V\subset\hat{\Omega}$. Let
$\mathfrak{t}\subset\mathfrak{k}^{{\mathbf{C}}}$ be a Cartan subalgebra, and
let $\gamma_{1}>\cdots>\gamma_{r}$ be the Harish-Chandra strongly ortogonal
roots so that $\gamma_{1}$ is the highest root for $\mathfrak{p}^{+}$ as
representation of $\mathfrak{k}^{{\mathbf{C}}}$. In particular $\gamma_{1}$ is
the highest root of $\mathfrak{g}^{{\mathbf{C}}}$ as representation of
$\mathfrak{g}^{{\mathbf{C}}}$. Let $\mathfrak{t}^{-}$ be the span of the co-
roots of $\gamma_{1},\cdots,\gamma_{r}$ and let
$\mathfrak{t}=\mathfrak{t}^{-}+\mathfrak{t}^{+}$ with
$\gamma_{1},\cdots,\gamma_{r}$ vanishing on $\mathfrak{t}^{+}$. The root space
decomposition of
$\mathfrak{g}^{{\mathbf{C}}}=\mathfrak{p}^{+}+\mathfrak{k}^{{\mathbf{C}}}+\mathfrak{p}^{-}$
is refined as
$\mathfrak{g}^{{\mathbf{C}}}=(\mathfrak{p}^{+}+\mathfrak{k}^{+})+\mathfrak{t}+(\mathfrak{k}^{-}+\mathfrak{p}^{-})$
with $\mathfrak{k}^{-}+\mathfrak{p}^{-}$ the space of negative root vectors,
and
${\mathfrak{k}^{+}+\mathfrak{k}^{-}}\subset[\mathfrak{k}^{{\mathbf{C}}},\mathfrak{k}^{{\mathbf{C}}}]$.

The $L^{2}$-space $L^{2}(\hat{\Omega},\nu)$ is decomposed as

(99) $L^{2}(\hat{\Omega},\nu)=\sum_{{\mathbf{m}}}V_{\nu,{\mathbf{m}}}$

where each $V_{\nu,{\mathbf{m}}}$ is of highest weight whose restriction on
$\mathfrak{t}^{-}$ is

$\frac{\nu}{2}+m_{1}\gamma_{1}+\cdots+m_{r}\gamma_{r},\qquad\frac{\nu}{2}:=\frac{1}{2}\nu(\gamma_{1}+\cdots+\gamma_{r}),$

where $m_{j}$ are nonnegative integers subject to the condition

(100) $m_{1}\geq\cdots\geq m_{r}\geq 0.$

When $\Omega=G/K$ is not of tube type this does not define completely the
highest weights and it requires some extra specifications; however the highest
weights of these representations that appear in $L^{2}(\hat{\Omega},\nu)$ are
uniquely determined by the condition above, see [Sci], [Shm], [Zh2].

Recall the $\tau_{\nu}$-spherical functions on $\hat{G}$

$\tau(k_{1})^{\nu}\tau(k_{2})^{\nu}f(k_{1}gk_{2})=f(g),\qquad
g\in\hat{G},\;k_{1},k_{2}\in K.$

As functions on $\hat{G}$ each space $V_{\nu,{\mathbf{m}}}$ contains a unique
$\tau_{\nu}$-spherical function $\Psi_{\nu,{\mathbf{m}}}$ normalized by
$\Psi_{\nu,{\mathbf{m}}}(e)=1$. We set

$\phi_{\nu,{\mathbf{m}}}(z)=J_{g}(0)^{-\frac{\nu}{p}}\Psi_{\nu,{\mathbf{m}}}(g),\qquad
g\cdot 0=z,$

as a trivialization of the $\tau_{\nu}$-spherical function
$\Psi_{\nu,{\mathbf{m}}}$. In particular $\phi_{\nu,{\mathbf{m}}}(z)$ is now
both left and $K$-invariant, and thus can be realized as a left $K$-invariant
function on $V\subset\hat{\Omega}$,
$\phi_{\nu,{\mathbf{m}}}(kz)=\phi_{\nu,{\mathbf{m}}}(z)$,
$\phi_{\nu,{\mathbf{m}}}(0)=1$, and

$\phi_{\nu,{\mathbf{m}}}(z)=h(z,-z)^{-\frac{\nu}{2}}\Psi_{\nu,{\mathbf{m}}}(\hat{\phi}_{z}).$

In the notation above $\phi_{{\mathbf{m}}}(z)$ is the coefficient of the
section $\Psi_{\nu,{\mathbf{m}}}$ with respect to the frame $e_{\nu}(z)$. The
orthogonality relations for $\phi_{\nu,{\mathbf{m}}}$ read now

$\hat{c}_{\nu}\int_{V}\phi_{\nu,{\mathbf{m}}}(z)\overline{\phi_{\nu,{\mathbf{m}}^{\prime}}(z)}h(z,-z)^{-\nu}d\mu_{0}(z)\\\
=\hat{c}_{\nu}c_{\Omega}2^{r}\int_{{\mathbf{R}}_{+}^{r}}\phi_{\nu,{\mathbf{m}}}(x)\overline{\phi_{\nu,{\mathbf{m}}^{\prime}}(x)}\prod_{j=1}^{r}(1+x_{j}^{2})^{\nu-p}\prod_{1\leq
j<k\leq r}(x_{j}^{2}-x_{k}^{2})^{a}\prod_{j=1}^{r}x_{j}^{2b+1}\,dx_{j}\\\
=\frac{1}{d_{\nu,{\mathbf{m}}}}\delta_{{\mathbf{m}},{\mathbf{m}}^{\prime}},$

where $d_{\nu,{\mathbf{m}}}=\dim V_{\nu,{\mathbf{m}}}$ is the dimension of
$V_{\nu,{\mathbf{m}}}$ (which can be computed using the Weyl dimension
formula). These are the Jacobi polynomials of Heckman and Opdam. (The
functions $\Psi_{\nu,{\mathbf{m}}}$ are the spherical functions
$\phi_{\lambda,\nu}$ studied by Shimeno for specific discrete values of the
parameter $\lambda$; see [Shm, Remark 5.12].)

In particular, for ${\mathbf{m}}=(0)$, $V_{\nu,(0)}$ is the Bergman space of
holomorphic sections of the line bundle defined by $\nu$ in
$L^{2}(\hat{\Omega},\nu)$. It can be realized as the space of holomorphic
polynomials of degree $\leq\nu$ and has reproducing kernel
$\hat{c}_{\nu}h(z,-w)^{\nu}$. The corresponding Heckman-Opdam polynomial is
the constant function $\phi_{\nu,(0)}(z)=1$.

We equip $\hat{\Omega}$ with the $\hat{G}$-invariant (Kähler) metric and let
$\overline{D}$ be the associated invariant Cauchy-Riemann operator. We
describe the decomposition (99) using the kernels of $\overline{D}^{m}$. We
shall need some results on the vanishing properties of Shimura operators on
the spaces $V_{\nu,{\mathbf{m}}}$ obtained in [SaZ]. First we recall the
Shimura operators using our present formulation. Recall from Section 2 the
Hua-Schmid decomposition

$\otimes^{m}V=\sum_{|{\mathbf{m}}|=m}S^{{\mathbf{m}}}V$

of the symmetric tensor product $\otimes^{m}V$ under $K$. Let
$P_{{\mathbf{m}}}$ be the corresponding projection. It is a general fact that
$\bar{D}^{m}:C^{\infty}(G,K;\tau_{\nu})\to
C^{\infty}(G,K;\tau_{\nu}\otimes\otimes^{m}V)$, where as before $V$ is
identified as the holomorphic tangent space $T_{0}^{(1,0)}(\hat{\Omega})$ of
$\hat{\Omega}$ at $0$, and $C^{\infty}(G,K;\tau_{\nu}\otimes\otimes^{m}V)$ is
the space of smooth sections of the line bundle
$\mathcal{L}^{\nu}\otimes\otimes^{m}T^{(1,0)}$ realized as functions on
$\hat{G}$ transforming under $K$ as

$\tau_{\nu}(k)\otimes^{m}\text{Ad}(k)f(gk)=f(g),\qquad g\in\hat{G}.$

The Shimura operators are defined by

$L_{{\mathbf{m}}}=(\bar{D}^{|{\mathbf{m}}|})^{\ast}P_{{\mathbf{m}}}\bar{D}^{|{\mathbf{m}}|}.$

We have then

$(\bar{D}^{m+1})^{\ast}\bar{D}^{m+1}=\sum_{|{\mathbf{m}}|=m+1}L_{{\mathbf{m}}}.$

######  Theorem 28.

The kernel $\operatorname{Ker}\bar{D}^{m+1}$ in $L^{2}(\hat{\Omega},\nu)$ is
precisely the direct sum

$\operatorname{Ker}\bar{D}^{m+1}=\sum_{|{\mathbf{m}}|\leq
m}^{\oplus}V_{\nu,{\mathbf{m}}}.$

In particular the reproducing kernel at the origin for the space of of nearly
holomorphic sections of order $m+1$ in $L^{2}(\hat{\Omega},\nu)$ is given by

$\hat{N}_{\nu}^{m+1}(z,0)=\sum_{|{\mathbf{m}}|\leq
m}d_{{\mathbf{m}}}\phi_{\nu,{\mathbf{m}}}(z).$

###### Proof.

The operator $L_{{\mathbf{m}}}$ acts on each irreducible component
$V_{\nu,{\mathbf{n}}}$ in (99) as a non-negative scalar multiple of the
identity, by Schur’s lemma, and their eigenvalues are shown in [SaZ] to be
given by Okounkov polynomials. More precisely, the eigenvalue of
$L_{{\mathbf{m}}}$ on $V_{\nu,{\mathbf{n}}}$ is a symmetric polynomial
$\widetilde{L}_{{\mathbf{m}}}(\frac{\nu}{2}+{\mathbf{n}}+\rho)$ of
$\frac{\nu}{2}+{\mathbf{n}}+\rho$, where $\rho$ is the half-sum of positive
roots of $\mathfrak{t}$ in $\mathfrak{g}^{{\mathbf{C}}}$. (One may also take
$\frac{\nu}{2}$ into the definition of $\rho$ as above.) It follows from [SaZ,
Theorem 5.1] that
$\widetilde{L}_{{\mathbf{m}}}(\frac{\nu}{2}+{\mathbf{n}}+\rho)=0$ unless
${\mathbf{m}}\subseteq{\mathbf{n}}$ (i.e. $m_{j}\leq n_{j}$ for all
$j=1,\dots,r$). This implies that

(101) $\sum_{|{\mathbf{n}}|\leq
m}^{\oplus}V_{\nu,{\mathbf{n}}}\subseteq\operatorname{Ker}(\bar{D}^{m+1})^{\ast}\bar{D}^{m+1}=\operatorname{Ker}\bar{D}^{m+1}.$

Now we prove the reverse inclusion, namely that if $|{\mathbf{n}}|>m$ then
$\bar{D}^{m+1}$ on $V_{\nu,{\mathbf{n}}}$ is non-zero. Suppose to the contrary
that $\bar{D}^{m+1}:V_{\nu,{\mathbf{n}}}\to 0$. We use the formulation as in
[SaZ, Section 3.4] for the realization of $V_{\nu,{\mathbf{n}}}$ to compute
the action of $\bar{D}^{m+1}$. As a unitary representation
$(V_{\nu,{\mathbf{n}}},\hat{G},\pi_{{\mathbf{n}}})$ of $\hat{G}$, the space
$V_{\nu,{\mathbf{n}}}$ contains a unique non-zero vector $v_{\nu}$ such that

$\pi_{{\mathbf{n}}}(k)v_{\nu}=\tau_{\nu}(k)v_{\nu}$

where $\tau_{\nu}$ is the one-dimensional representation defined as above.
Moreover both representations $\tau_{-\nu}$ and $\tau_{\nu}$ appear in
$V_{\nu,{\mathbf{n}}}$. As functions on $G$, the space
$V_{\nu,{\mathbf{n}}}\subset L^{2}(\hat{\Omega},\nu)\subset L^{2}(\hat{G})$ is
obtained as

$v\in V_{\nu,{\mathbf{n}}}\mapsto
f_{v}(g)=\langle\pi_{{\mathbf{n}}}(g^{-1})v,v_{-\nu}\rangle,\qquad f_{v}\in
V_{\nu,{\mathbf{n}}}\subset L^{2}(\hat{G}),$

where with some abuse of notation we have used the same notation
$V_{\nu,{\mathbf{n}}}$ both as $\hat{G}$-representation and as a space of
functions. The assumption $\bar{D}^{m+1}:V_{\nu,{\mathbf{n}}}\to 0$ implies in
particular that $\bar{D}^{m+1}f_{v_{-\nu}}=0$, and its evaluation at $g=e$
implies further that

$\pi_{{\mathbf{n}}}(X)v_{-\nu}=0$

for all $X\in S^{m+1}(\mathfrak{p}^{-})$. Let $X=X_{1}Y$ where
$X_{1}\in\mathfrak{p}^{-}$ is an arbitrary negative root vector and $Y\in
S^{{\mathbf{m}}}(\mathfrak{p}^{-})$ is a $\mathfrak{k}^{+}$-lowest weight
vector in $S^{{\mathbf{m}}}(\mathfrak{p}^{-})$ with lowest weight
$-(m_{1}\gamma_{1}+\cdots+m_{r}\gamma_{r})$ with $m_{1}\geq\cdots\geq
m_{r}\geq 0$. (A construction of all lowest weight vectors is found in [Up1]
but we shall not need the explicit form.) We have then
$\pi_{{\mathbf{n}}}(X_{1})\pi_{{\mathbf{n}}}(Y)v_{-\nu}=0$. Since $v_{-\nu}$
defines a one-dimensional representation of $\mathfrak{k}^{\mathbb{C}}$ we
have always $\pi_{{\mathbf{n}}}(X)\pi_{{\mathbf{n}}}(Y)v_{-\nu}=0$, for
$X\in\mathfrak{k}^{-}\subset[\mathfrak{k}^{{\mathbf{C}}},\mathfrak{k}^{{\mathbf{C}}}]$.
In other words, $\pi_{{\mathbf{n}}}(Y)v_{-\nu}$ is a lowest weight vector for
the $\mathfrak{g}^{\mathbb{C}}$-representation unless it vanishes. However by
the Hua-Schmid decomposition the element $\pi_{{\mathbf{n}}}(Y)v_{-\nu}$ has
lowest weight $-\frac{\nu}{2}-(m_{1}\gamma_{1}+\cdots+m_{r}\gamma_{r})$,
$m_{1}+\cdots+m_{r}=m<|{\mathbf{n}}|$. But the space $V_{\nu,{\mathbf{n}}}$
has lowest weight $-\frac{\nu}{2}-(n_{1}\gamma_{1}+\cdots+n_{r}\gamma_{r})$
and thus $\pi_{{\mathbf{n}}}(Y)v_{-\nu}=0$. Acting by $k\in K$ we find
$\pi_{\mathbf{n}}({\text{Ad}(k)Y})\pi_{\mathbf{n}}(k)v_{-\nu}=0$. Again
$v_{-\nu}$ defines a one-dimensional representation of $K$,
$\pi_{\mathbf{n}}(k)v_{-\nu}=\tau_{-\nu}(k)v_{-\nu}$ with scalar character
$\tau_{-\nu}(k)$. Thus $\pi_{\mathbf{n}}({\text{Ad}(k)Y})v_{-\nu}=0$.
Furthermore $\\{\text{Ad}(k)Y,k\in K\\}$ generates the irreducible the
representation $S^{{\mathbf{m}}}(\mathfrak{p}^{-})$ so we get
$\pi_{\mathbf{n}}(X)v_{-\nu}=0$ for all $X\in
S^{{\mathbf{m}}}(\mathfrak{p}^{-})$, and further
$\pi_{\mathbf{n}}(X)v_{-\nu}=0$ for all $X\in S^{m}(\mathfrak{p}^{-})$.
Continuing this procedure we get that $v_{-\nu}=0$, a contradiction. This
proves our claim on $\operatorname{Ker}\bar{D}^{m+1}$ and then on the
reproducing kernel. ∎

######  Remark 29.

As noted in [Shm, Remark 5.12] the spherical functions
$\phi_{\nu,{\mathbf{m}}}(z)$ here are precisely the Heckman-Opdam polynomials
in Corollary 26 under proper coordinate change. Thus Theorem 28 is just an
abstract restatement and a different proof of the expansion in Corollary 26
(with $m+1$ replacing $m$ for notational convenience) with interpretation of
the coefficients using the dimension $d_{\nu,m}$. ∎

######  Remark 30.

The subspace $V_{\nu,{\mathbf{m}}}$ can also be described using, as in Section
3, the quasi-inverse $\partial\log h(z,-z)$. In the local coordinates $z\in
V\subset\hat{\Omega}$ the space $V_{\nu,{\mathbf{m}}}$ consists of functions

$f(z)=\otimes^{m}(\partial\log h(z,-z))(F(z))$

where $F$ is a holomorphic section of the bundle
$\mathcal{L}^{\nu}\otimes\otimes^{m}T^{(1,0)}$ in the highest weight
representation above. ∎

######  Remark 31.

It follows from the proof above that for any ${\mathbf{n}}$ there exists an
${\mathbf{n}}^{\prime}$, $|{\mathbf{n}}^{\prime}|\leq|{\mathbf{n}}|$ such that
the eigenvalue
$\widetilde{L}_{{\mathbf{n}}^{\prime}}(\frac{\nu}{2}+{\mathbf{n}}+\rho)$ of
the Shimura operator on $V_{\nu,{\mathbf{n}}}$ is nonvanishing,
$\widetilde{L}_{{\mathbf{n}}^{\prime}}(\frac{\nu}{2}+{\mathbf{n}}+\rho)\neq
0$. This might be a known fact or can be proved by using Koornwinder’s formula
(see [Ko], [Ok] and [SaZ, Theorem 5.5]) for
$\widetilde{L}_{{\mathbf{n}}^{\prime}}$, which in turn can give a different
proof of the reverse inclusion of (101). ∎

######  Example 32.

Let us again make everything more specific for the rank one case, i.e. when
$\hat{G}/K={\mathbf{C}}P^{d}$ is the complex projective space. In this case it
is more convenient to use the realization of ${\mathbf{C}}P^{d}$ as
${\mathbf{C}}P^{d}=U(d+1)/U(d)\times U(1)$. We choose the Cartan subalgebra of
$U(d+1)$ as diagonal matrices identified as ${\mathbf{R}}^{d+1}$, with the
Harish-Chandra root $\beta=(1,0,\cdots,0,-1)$. The highest weights above are
now $(\nu+m,0,\cdots,-m)$. The sections of the line bundle with parameter
$\nu$ on ${\mathbf{C}}P^{d}=U(d+1)/U(d)\times U(1)$ can be realized as
functions on the sphere $S^{2d+1}=U(d+1)/U(d)$ and the representation space
with the highest weight $(\nu+m,0,\cdots,-m)$ is the space of
$(p,q)=(\nu+m,m)$-spherical harmonic polynomials. We write
$\phi_{\nu,(m)}=\phi_{\nu,m}$.

When $\nu=0$, i.e. the spherical case, the highest weight is of the form
${\mathbf{m}}=m\beta$ with spherical polynomial

$\phi_{0,m}(\exp(H))={}_{2}\\!F_{1}\Big{(}\begin{matrix}-m,d+m\\\
d\end{matrix}\Big{|}\sin^{2}\frac{\beta(H)}{2}\Big{)};$

see [He, Theorem V.4.5] and Remark 21 above. For general $\nu\geq 0$,

$\phi_{\nu,m}(\exp(tH))={}_{2}\\!F_{1}\Big{(}\begin{matrix}d+m+\nu,-m\\\
d\end{matrix}\Big{|}\sin^{2}t\Big{)}.$

See [SaZ], [JW].

By the Schur orthogonality we have

$\langle\phi_{\nu,m},\phi_{\nu,m^{\prime}}\rangle=\frac{1}{d_{\nu,m}}\delta_{m,m^{\prime}},$

where $d_{\nu,m}$ is the dimension of the representation space $V_{\nu,m}$.
Here the inner product is given by

$\langle\phi,\psi\rangle=\hat{c}_{\nu}\int_{0}^{\frac{\pi}{2}}\phi(\sin^{2}t)\overline{\psi(\sin^{2}t)}\sin^{2\nu+1}(2t)\sin^{2(d-1)-2\nu}(t)dt\\\
=\hat{c}_{\nu}\int_{0}^{\frac{\pi}{2}}\phi(\sin^{2}t)\overline{\psi(\sin^{2}t)}\sin^{2\nu}(2t)\sin^{2(d-1)-2\nu}(t)d\sin^{2}t\\\
=\hat{c}_{\nu}\int_{0}^{1}\phi(x)\overline{\psi(x)}(1-x)^{\nu}x^{d-1}dx.$

The $\tau_{\nu}$-spherical function above is

$\phi_{\nu,m}(x)={}_{2}\\!F_{1}\Big{(}\begin{matrix}-m,m+d+\nu\\\
d\end{matrix}\Big{|}x\Big{)}.$

The dimension of the representation space $V_{\nu,m}$ can be easily found
using the Weyl dimension formula and equals

$d_{\nu,m}=\frac{(2m+\nu+d)(m+\nu+1)_{d-1}(m+1)_{d-1}}{d!(d-1)!}.$

In particular,

$d_{\nu,0}=\frac{(\nu+d)(\nu+1)_{d-1}}{d!}=\frac{(\nu+1)_{d}}{d!}=\binom{\nu+d}{d}$

which is precisely the dimension of the space of polynomials
$\mathcal{P}_{\leq\nu}({\mathbf{C}}^{d})$ on ${\mathbf{C}}^{d}$ of degree
$\leq\nu$ realized as the holomorphic sections in $L^{2}(\hat{\Omega},\nu)$.

So we are computing the sum

$\sum_{m\leq n}d_{\nu,m}\phi_{\nu,m}(x)=\\\ \sum_{m\leq
n}\frac{(2m+\nu+d)(m+\nu+1)_{d-1}(m+1)_{d-1}}{d!(d-1)!}\;{}_{2}\\!F_{1}\Big{(}\begin{matrix}-m,m+d+\nu\\\
d\end{matrix}\Big{|}x\Big{)}.$

To carry out the summation we use the following elementary observation.

######  Lemma 33.

Let $d\mu(x)$ be a finite Borel measure on ${\mathbf{R}}_{+}$ such that all
polynomials are dense in $L^{2}({\mathbf{R}}_{+},d\mu)$. Let
$\\{p_{m}\\}_{m=0}^{\infty}$ be the orthonormal basis obtained from the Gram-
Schmidt orthogonalization of the polynomials $\\{x^{m}\\}_{m=0}^{\infty}$.
Then the reproducing kernel $\sum_{m=0}^{n}p_{m}(x)p_{m}(0)$ evaluated at $0$
is

$\sum_{m=0}^{n}p_{m}(x)p_{m}(0)=A_{n}q_{n}(x)$

for some constant $A_{n}$, where $\\{q_{n}(x)\\}_{n=0}^{\infty}$ is the
orthonormal basis obtained from $\\{x^{n}\\}_{n=0}^{\infty}$ for the space
$L^{2}({\mathbf{R}}_{+},d\tilde{\mu})$, where $d\tilde{\mu}=x\,d\mu(x)$.

###### Proof.

Write $P_{n}(x)=\sum_{m=0}^{n}p_{m}(x)p_{m}(0)$. We prove that $P_{n}(x)$ is
orthogonal to all polynomials $x^{m}$, $0\leq m\leq n-1$, in the space
$L^{2}({\mathbf{R}}_{+},d\tilde{\mu})$. Indeed the inner product of $x^{n}$
and $P_{m}$ in $L^{2}({\mathbf{R}}_{+},d\tilde{\mu})$ is

$\int_{0}^{\infty}x^{m}P_{n}(x)x\,d\mu(x)=\int_{0}^{\infty}x^{m+1}P_{n}(x)\,d\mu(x)=x^{m+1}|_{x=0}=0,$

since $P_{n}(x)$ is the reproducing kernel at 0 in
$L^{2}({\mathbf{R}}_{+},d\mu)$ for the polynomials of degree $\leq n$ and
$0<m+1\leq n$. Thus $P_{n}$ is proportional to $q_{n}$. This proves the lemma.
∎

######  Theorem 34.

The reproducing kernel $\hat{N}^{n}_{\nu}(z,0)$ at the origin for the space
$\hat{N}^{n}_{\nu}({\mathbf{C}}P^{d})$, under the local trivialization above
using the local frame $e_{\nu}$ on ${\mathbf{C}}^{d}\subset{\mathbf{C}}P^{d}$,
is

$\hat{N}^{n}_{\nu}(z,0)=\sum_{m\leq
n}d_{\nu,m}\phi_{\nu,m}(x)=A_{n}\;{}_{2}\\!F_{1}\Big{(}\begin{matrix}-n,n+d+\nu+2\\\
d+1\end{matrix}\Big{|}x\Big{)},\quad x=\frac{|z|^{2}}{{1+|z|^{2}}},$

where the positive constant $A_{n}$ is given by (103) below.

###### Proof.

We use Lemma 33. The polynomials $\\{\phi_{\nu,m}(x)\\}$ form an orthogonal
basis for the space $L^{2}((0,1),d\mu(x))$, $d\mu(x)=(1-x)^{\nu}x^{d-1}\,dx$,
and they are the same orthogonal basis as obtained from the Gram-Schmidt
process from the measure $d\mu(x)$. The orthogonal basis for the measure
$d\tilde{\mu}(x)=x\,d\mu(x)=(1-x)^{\nu}x^{d+1}$ is
${}_{2}\\!F_{1}\Big{(}\begin{matrix}-m,m+d+\nu+2\\\
d+1\end{matrix}\Big{|}x\Big{)}$. Thus

(102) $\begin{split}\sum_{m\leq n}d_{\nu,m}\phi_{\nu,m}(x)&=\sum_{m\leq
n}d_{\nu,m}\phi_{\nu,m}(x)\\\
&=A_{n}\;{}_{2}\\!F_{1}\Big{(}\begin{matrix}-n,n+d+\nu+2\\\
d+1\end{matrix}\Big{|}x\Big{)}\end{split}$

for some constant $A_{n}$. To find $A_{n}$, we view (102) as an identity of
two polynomials of $x\in{\mathbf{R}}$. The leading coefficients of $x^{n}$ in
(102) are

$d_{n,\nu}\frac{(-n)_{n}(n+d+\nu)_{n}}{(d)_{n}n!}=A_{n}\frac{(-n)_{n}(n+d+\nu+2)_{n}}{(d+1)_{n}n!}.$

Thus

(103)
$\begin{split}A_{n}&=d_{n,\nu}\frac{(n+d+\nu)_{n}(d+1)_{n}}{(n+d+\nu+2)_{n}(d)_{n}}=d_{n,\nu}\frac{(n+d+\nu)_{n}(d+n)}{(n+d+\nu+2)_{n}d}\\\
&=\frac{(2n+\nu+d)(n+\nu+1)_{d-1}(n+1)_{d-1}}{d!(d-1)!}\frac{(n+d+\nu)_{n}(d+n)}{(n+d+\nu+2)_{n}d}\\\
&=\frac{(n+\nu+1)_{d+1}(n+1)_{n+d-1}}{(2n+d+\nu+1)d!^{2}}.\end{split}$

∎

## References

  * [Ank] J.-P. Anker: An introduction to Dunkl theory and its analytic aspects, Analytic, algebraic and geometric aspects of differential equations (G. Filipuk, Y. Haraoka, S. Michalik, eds.), Trends in Math., Birkhäuser, 2017, pp. 3–58.
  * [Ar] J. Arazy: A survey of invariant Hilbert spaces of analytic functions on bounded symmetric domains, Multivariable operator theory (R.E. Curto, R.G. Douglas, J.D. Pincus, N. Salinas, eds.), Contemp. Math. 185, AMS, Providence, 1995, pp. 7–65.
  * [AE] J. Arazy, M. Engliš: $Q_{p}$-spaces on bounded symmetric domains, J. Funct. Spaces Appl. 6 (2008), 205–240.
  * [BE] H. Bateman, A. Erdélyi, Higher transcendental functions, vol. I, McGraw-Hill, New York - Toronto - London 1953.
  * [BE2] H. Bateman, A. Erdélyi, Higher transcendental functions, vol. II, McGraw-Hill, New York - Toronto - London 1953.
  * [BO] R.J. Beerends, E.M. Opdam: Certain hypergeometric series related to the root system BC, Trans. Amer. Math. Soc. 339 (1993), 581–609.
  * [BM] S. Bochner, W.T. Martin, Several complex variables, Princeton University Press, Princeton, 1948.
  * [De1] A. Debiard: Système différentiel hypergéométrique et parties radiales des opérateurs invariants des espaces symétriques de type $BC_{p}$, Lecture Notes Math. 1296, Springer, New York-Berlin, 1988, pp. 42–124.
  * [De2] A. Debiard: Le système différentiel hypergéométrique de type BC et son spectre, C. R. Acad. Sci. Paris Ser. I Math. 314 (1992), 793–796.
  * [De3] A. Debiard: Opérateurs différentiels d’Euler-Poisson-Darboux gńéralisés et fonctions hypergéométriques de plusieurs variables, Complex analysis, harmonic analysis and applications, Pitman Res. Notes Math. Ser. 347, Longman, Harlow, 1996, pp. 56–73.
  * [DOZ] A.H. Dooley, B. Ørsted, G. Zhang: Relative discrete series of line bundles over bounded symmetric domains, Ann. Inst. Fourier 46 (1996), 1011–1026.
  * [En] M. Engliš: The asymptotics of a Laplace integral on a Kähler manifold, J. reine angew. Math. 528 (2000), 1-39.
  * [EZ] M. Engliš, G. Zhang: Toeplitz operators on higher Cauchy-Riemann spaces, Documenta Math. 22 (2017), 1081–1116.
  * [FK] J. Faraut, A. Korányi: Function spaces and reproducing kernels on bounded symmetric domains, J. Funct. Anal. 88 (1990), 64–89.
  * [HS] G. Heckman, H. Schlichtkrull, Harmonic analysis and special functions on symmetric spaces, Academic Press, 1994.
  * [He] S. Helgason, Groups and geometric analysis, Academic Press, Orlando, 1984.
  * [JW] K. D. Johnson, N. R. Wallach: Composition series and intertwining operators for the spherical principal series. I, Trans. Amer. Math. Soc. 229 (1977), 137–173.
  * [Ka] J. Kaneko: Selberg integrals and hypergeometric functions associated with Jack polynomials, SIAM J. Math. Anal. 24 (1993), 1086–1110.
  * [Ko] T. Koornwinder: Okounkov’s BC-type interpolation Macdonald polynomials and their $q=1$ limit, Sém. Lothar. Combin. B72a (2015), 27 pp.
  * [Lo] O. Loos, Bounded symmetric domains and Jordan pairs, University of California, Irvine, 1977.
  * [MD] I.G. MacDonald, Symmetric functions and Hall polynomials (2nd edition), Clarendon Press, Oxford, 1995.
  * [Ok] A. Okounkov: $BC$-type interpolation Macdonald polynomials and binomial formula for Koornwinder polynomials, Transform. Groups 3 (1998), 181–207.
  * [Op] E.M. Opdam: Harmonic analysis for certain representations of graded Hecke algebras, Acta Math. 175 (1995), 75–121.
  * [PZ] J. Peetre, G. Zhang: Invariant Cauchy-Riemann operators and relative discrete series of line bundles over the unit ball of ${\mathbf{C}}^{d}$, Michigan Math. J. 45 (1998), 387–397.
  * [SaZ] S. Sahi, G. Zhang: Positivity of Shimura operators, Math. Res. Lett. 26 (2019), 587–626.
  * [Sci] H. Schlichtkrull: One-dimensional K-types in finite dimensional representations of semisimple Lie groups: A generalization of Helgason’s theorem, Math. Scand. 54 (1984), 279–294.
  * [Sch] W. Schmid: Die Randwerte holomorpher Funktionen auf hermitischen Räumen, Invent. Math. 9 (1969), 61–80.
  * [Shm] N. Shimeno: The Plancherel formula for spherical functions with a one-dimensional $K$-type on a simply connected simple Lie group of Hermitian type, J. Funct. Anal. 121 (1994), 330–388.
  * [Shi] G. Shimura, On a class of nearly holomorphic automorphic forms, Ann. Math. 123 (1986), 347-406.
  * [Up1] H. Upmeier, Jordan algebras and harmonic analysis on symmetric spaces, Amer. J. Math. 108 (1986), 1–25.
  * [Up2] H. Upmeier, Toeplitz operators and index theory in several complex variables, Operator Theory: Advances and Applications 81, Birkhäuser Verlag, Basel, 1996.
  * [Yan] Z. Yan: A class of generalized hypergeometric functions in several variables, Canad. J. Math. 44 (1992), 1317–1338.
  * [You] E.-H. Youssfi: Polyanalytic reproducing kernels in ${\mathbf{C}}^{n}$, Complex Anal. Synergies 7 (2021), art. no. 28.
  * [Zh1] G. Zhang: A weighted Plancherel formula II. The case of the ball, Studia Math. 102 (1992), 103–120.
  * [Zh2] G. Zhang: Berezin transform on compact Hermitian symmetric spaces, Manuscripta Math. 97 (1998), 371–388.
  * [Zh3] G. Zhang: Nearly holomorphic functions and relative discrete series of weighted $L^{2}$-spaces on bounded symmetric domains, J. Math. Kyoto Univ. 42 (2002), 207–221.