# Hankel transform, $\mathcal{K}$-Bessel functions and zeta distributions in
the Dunkl setting

Dominik Brennecken  Institut für Mathematik, Universität Paderborn, Warburger
Str. 100, D-33098 Paderborn, Germany<EMAIL_ADDRESS>
###### Abstract.

We study analytic aspects of the Dunkl-type Hankel transform, which goes back
to Baker and Forrester and, in an earlier symmetrized version, to Macdonald.
Moreover, we introduce a Dunkl analogue of the Bessel function and
$\mathcal{K}$-Bessel function generalizing those of a symmetric cone. Further,
we take a look at zeta integrals and their distributional extensions in the
Dunkl setting. These distributions are closely related to Dunkl-type Riesz
distributions. Further, we study regularity properties of the zeta
distributions and prove a functional equation relating zeta distributions and
their Dunkl transform.

###### Key words and phrases:

Dunkl theory, Zeta distributions, Hankel transform, Laplace transform, Jack
polynomials, hypergeometric functions associated with root systems

###### 2000 Mathematics Subject Classification:

Primary 33C67; Secondary 33C52

##  1\. Introduction

Zeta integrals are functionals which integrate a function $f$ against a
complex power $Q^{\nu}$ of a quadratic form on some underlying space. In
particular, they occur in the analysis on symmetric cones, where they depend
on a representation of the underlying Jordan algebra, see [FK94, Chapter XVI]
and [Cl02]. An important special case are matrix cones, where zeta
distributions are related to Wishart distributions, see [Mu82, FK94, Ru06].
All these zeta distributions have common properties: they can be analytically
extended in the parameter $\nu$ and they satisfy a functional equation
relating the zeta distributions of a function and the zeta distributions of
its Fourier transform. Moreover, zeta distributions on symmetric cones are
closely related to Riesz distributions. Important tools in their study are the
Hankel transform and the Laplace transform. Since radial analysis on symmetric
cones is related to type $A$ Dunkl theory, one expects a generalization to
this setting. In fact, the Laplace transform and Riesz distributions in the
Dunkl setting have already been studied in [Rö20, BR22]) and the Hankel
transform was considered in [BF98], but its analytic aspects have not been
studied so far. It turns out that the theory of zeta distributions and the
Hankel transform on symmetric cones is also closely related to type $B$ Dunkl
theory and that the type $A$ Dunkl theory related to Laplace transforms and
Riesz distributions will be an essential tool. We will use Laplace transform
identities for Jack polynomials, hypergeometric series of Jack polynomials and
hypergeometric functions as studied in [BR22]. We study different types of
Bessel functions which are analogues of Bessel functions on symmetric cones.
The usual Bessel function and $\mathcal{K}$-Bessel function was already
considered in [He55] for matrix cones, see [FK94] for general symmetric cones.
Of central interest in this paper is a two-variable hypergeometric series of
Jack polynomials $C_{\lambda}$ (of $n$ variables and arbitrary index
$\alpha$), which we call a Bessel kernel (see Section 3)

$\mathscr{J}_{\nu}(w,z)\coloneqq{}_{0}F_{1}(\nu;w,-z)=\sum\limits_{\lambda\in\Lambda_{+}^{n}}\frac{(-1)^{\left|\lambda\right|}}{[\nu]_{\lambda}}\frac{C_{\lambda}(w)C_{\lambda}(z)}{\left|\lambda\right|!C_{\lambda}(\underline{1})},\quad\nu\in\mathbb{C}.$

It generalizes the kernel of the Hankel transform on a symmetric cone, given
explicitly by the Bessel function of the cone, which can be identified as a
type $B$ Bessel function (cf. [Rö07]). We define a non-symmetric counterpart
of the Bessel kernel, named $\mathscr{E}_{\nu}$, which is given by a
hypergeometric series of non-symmetric Jack polynomials. The kernel
$\mathscr{E}_{\nu}$ is closely related to the type $B$ Dunkl kernel. The
Dunkl-type Hankel transform of a Schwartz function $f_{0}$ is defined by the
kernel $\mathscr{E}_{\nu}$ (see Section 5)

$\mathcal{H}_{\nu}f_{0}(y)\coloneqq\frac{1}{\Gamma_{n}(\nu)}\int_{\mathbb{R}^{n}_{+}}\mathscr{E}_{\nu}(-x,y)f_{0}(x)\Delta(x)^{\nu-\mu_{0}-1}\omega^{A}(x)\;\mathrm{d}x,$

where $\Delta(x)=x_{1}\cdots x_{n}$ and

$\omega^{A}(x)=\prod\limits_{i<j}\left|x_{i}-x_{j}\right|^{2k}.$

It turns out that the transform $\mathcal{H}_{\nu}$ is given by a Dunkl
transform of type $B$ (where the multiplicity depends on the parameter $\nu$)

$\mathcal{F}^{B}f(y)=2^{-n\nu}\mathcal{H}_{\nu}f_{0}(\tfrac{y^{2}}{4}),$

and $f$ is defined by $f(x)=f_{0}(x^{2})$, here $x^{2}$ is understood
componentwise. That is a known connection for spaces of rectangular matrices,
or more generally for symmetric cones, cf. Herz [He55, FK94, Ru06]. This
implies a type $B$ Bernstein identity which was also calculated in [Li16]

$\Delta(T^{B})^{2}\Delta(x^{2})^{\mu}=\mathcal{B}(\mu)\Delta(x^{2})^{\mu-1},\quad\mu\in\mathbb{C},$

where $\mathcal{B}$ is a polynomial and $\Delta(T^{B})^{2}$ is the type $B$
Dunkl operator associated to the polynomial $\Delta^{2}$. Later on in our
paper, this Bernstein identity will be important for the analytic continuation
of zeta distributions, a family of tempered distributions defined by (see
Section 6)

$\braket{\zeta_{\alpha},f}\coloneqq\frac{1}{\Gamma_{n}(\alpha)}\int_{\mathbb{R}^{n}}f(x)\Delta(x^{2})^{\alpha-\nu}\omega^{B}(x)\;\mathrm{d}x,$

for $\alpha\in\mathbb{C}$ with large real part, so that the analytic extension
in the parameter $\alpha$ to the whole complex plane $\mathbb{C}$ is obtained
in terms of the functional equation

$\Delta(T^{B})^{2}\zeta_{\alpha}=4^{n}b(\alpha-\nu)\zeta_{\alpha-1},$

here $b$ is a certain polynomial. This zeta distribution satisfies a
characteristic property relating it to its Dunkl transform

$\zeta_{\alpha}=2^{n(2\alpha-\nu)}\mathcal{F}^{B}\zeta_{\nu-\alpha}.$

This functional equation is wellknown for spaces of rectangular matrices as
well as for symmetric cones, where the Dunkl transform is replaced by a usual
Fourier transform, cf. [Ru06, FK94]. The results about Riesz distributions in
[Rö20] make it possible to explicitly determine those zeta distributions,
which are positive measures , so that the corresponding parameter
$\alpha\in\mathbb{C}$ lies in a generalized Wallach set. A key tool to verify
the functional equation of zeta distributions is the following generalization
of the $\mathcal{K}$-Bessel function of a symmetric cone (see Section 4)

$\mathcal{K}_{\nu}(w,z)\coloneqq\int_{\mathbb{R}_{+}^{n}}E^{A}(-x,w)E^{A}(-\tfrac{1}{x},z)\Delta(x)^{\nu-\mu_{0}-1}\omega^{A}(x)\;\mathrm{d}x,$

with $E^{A}$ the type $A$ Dunkl kernel. In fact, after symmetrization over the
action of the symmetric group and a particular choice of a type $A$
multiplicity parameter, $\mathcal{K}_{\nu}$ coincides with a symmetrized
version of the $\mathcal{K}$-Bessel function of a symmetric cone, as studied
in [Cl88, Di90, FK94]. The Bessel function $\mathcal{K}_{\nu}$ is defined for
all $\nu\in\mathbb{C}$, $w,z\in\mathbb{C}^{n}$ with
$\mathrm{Re}\,w,\,\mathrm{Re}\,z>0$ and satisfies

$\mathcal{K}_{\nu}(w,z)=\mathcal{K}_{-\nu}(z,w)\textrm{ and
}\left|\mathcal{K}_{\nu}(w,z)\right|\leq\mathcal{K}_{\mathrm{Re}\,\nu}(\mathrm{Re}\,w,\mathrm{Re}\,z).$

We shall also introduce a generalized $\mathcal{K}$-Bessel function with a
multivariate index $\nu\in\mathbb{C}^{n}$. The Dunkl operator $\Delta(T^{A})$
acts on the $\mathcal{K}$-Bessel function by shifting the parameter $\nu$ and
we prove growth condition on $\mathcal{K}_{\nu}$. Finally, on symmetric cones,
the $\mathcal{K}$-Bessel function defines an eigenfunction of the system of
Bessel operators. Similarly, we obtain in the present setthat the Dunkl-type
$\mathcal{K}$-Bessel function is (up to squared variables) an eigenfunction of
$\mathbb{Z}_{2}^{n}$-invariant type $B$ Dunkl operators, which play the role
of the Bessel operators on symmetric cones.

##  2\. The rational Dunkl setting

For a general background on rational Dunkl theory, the reader is referred to
[Du89, dJ93, Rö03, DX14]. We equip the $n$-dimensional Euclidean space
$\mathbb{R}^{n}$ with the usual inner product
$\braket{x,y}=\sum_{i=1}^{n}x_{i}y_{i}$ and extend this naturally to a
$\mathbb{C}$-bilinear form on $\mathbb{C}^{n}$. In $\mathbb{R}^{n}$ we
consider the root systems $R\in\left\\{A,B\right\\}$ with

$\displaystyle A$ $\displaystyle\coloneqq A_{n-1}=\left\\{\pm(e_{i}-e_{j})\mid
1\leq i<j\leq n\right\\},$ $\displaystyle B$ $\displaystyle\coloneqq
B_{n}=\left\\{\pm e_{i}\mid 1\leq i\leq n\right\\}\cup\left\\{\pm(e_{i}\pm
e_{j})\mid 1\leq i<j\leq n\right\\},$

where $(e_{i})_{1\leq i\leq n}$ is the canonical basis of $\mathbb{R}^{n}$.
The corresponding Weyl groups are

$W_{\\!A}\coloneqq\mathcal{S}_{n},\quad
W_{\\!B}\coloneqq\mathcal{S}_{n}\ltimes\mathbb{Z}_{2}^{n},$

where the symmetric group $\mathcal{S}_{n}$ acts by permutation of coordinates
and $\mathbb{Z}_{2}^{n}$ acts by sign changes of coordinates. A multiplicity
function on $R$ is a $W_{\\!R}$-invariant function
$\kappa_{R}:R\to\mathbb{C}$. Moreover, $W_{R}$ acts on functions
$f:\mathbb{R}^{n}\to\mathbb{C}$ by the assignment $w.f=x\mapsto f(w^{-1}x)$.
We consider multiplicity functions of the specific form

$\kappa_{A}=k,\quad\kappa_{B}=(k,k^{\prime}),$

where $k$ is the value on $\pm(e_{i}\pm e_{j})$ and $k^{\prime}$ is the value
on $e_{i}$. The (rational) Dunkl operator associated to $(R,\kappa_{R})$ into
direction $\xi\in\mathbb{R}^{n}$ is

$T_{\xi}^{R}=T_{\xi}^{R}(\kappa_{R})\coloneqq\partial_{\xi}+\frac{1}{2}\sum\limits_{\alpha\in
R}\kappa_{R}(\alpha)\braket{\alpha,\xi}\frac{1-s_{\alpha}}{\braket{\alpha,\cdot}},$

where
$s_{\alpha}(x)=x-2\tfrac{\braket{\alpha,x}}{\braket{\alpha,\alpha}}\alpha$ is
the reflection in the hyperplane perpendicular to $\alpha$. The Dunkl
operators commute for fixed $(R,\kappa_{R})$, i.e.
$T_{\xi}^{R}T_{\eta}^{R}=T_{\eta}^{R}T_{\xi}^{R}$ on $C^{2}(\Omega)$ for all
$\xi,\eta\in\mathbb{R}^{n}$ and $W_{\\!R}$-invariant open sets
$\Omega\subseteq\mathbb{R}^{n}$, so that $p(T^{R})$ is defined for any
polynomial $p$ on $\mathbb{R}^{n}$. For abbreviation, we write
$T_{i}^{R}\coloneqq T_{e_{i}}^{R}$. If $\mathrm{Re}\,\kappa_{R}\geq 0$, the
Dunkl operators define an $W_{\\!R}$-invariant symmetric non-degenerated
bilinear form on the space of polynomials
$\mathcal{P}\coloneqq\mathbb{C}[\mathbb{R}^{n}]$ by $[p,q]^{R}\coloneqq
p(T^{R})q(0)$, which is positive definite on $\mathbb{R}[\mathbb{R}^{n}]$ for
$\kappa_{R}\geq 0$. Moreover, polynomials of different homogeneous degree are
orthogonal with respect to this pairing.
If $\mathrm{Re}\,\kappa_{R}\geq 0$, then for each $\lambda\in\mathbb{C}^{n}$,
the joint eigenvalue problem

$\begin{cases}T_{\xi}^{R}f=\braket{\lambda,\xi}f,&\textrm{ for all
}\xi\in\mathbb{R}^{n}\\\ f(0)=1\end{cases}$

has an unique holomorphic solution
$E^{R}(\lambda,\cdot)=E_{\kappa_{R}}^{R}(\lambda,\cdot):\mathbb{C}^{n}\to\mathbb{C}$,
called the Dunkl kernel associated to $(R,\kappa_{R})$. The Dunkl kernel is
positive on $\mathbb{R}^{n}\times\mathbb{R}^{n}$ (if in addition
$\kappa_{R}\geq 0$) and satisfies

$E^{R}(w\lambda,wz)=E^{R}(\lambda,z),\quad
E^{R}(s\lambda,z)=E^{R}(\lambda,sz),\quad E^{R}(\lambda,z)=E^{R}(z,\lambda),$

for all $\lambda,z\in\mathbb{C}^{n},\,w\in W_{\\!R}$ and $s\in\mathbb{C}$.
Moreover, the Bessel function associated to $(R,\kappa_{R})$ is defined as

$J^{R}(\lambda,z)=J^{R}_{\kappa_{R}}(\lambda,z)\coloneqq\frac{1}{\\#W_{\\!R}}\sum\limits_{w\in
W_{\\!R}}E^{R}(\lambda,wz).$

An important weight function in Dunkl theory is

$\omega^{R}(x)=\omega_{\kappa_{R}}^{R}(x)=\prod\limits_{\alpha\in
R}\left|\braket{x,\alpha}\right|^{\kappa_{R}(\alpha)}.$

The weight functions associated to the root systems $A$ and $B$ are related to
each other by the equation

$\omega^{B}(x)=\Delta(x^{2})^{k^{\prime}}\omega^{A}(x^{2}),$ (2.1)

where $x^{2}$ is understood componentwise and

$\Delta(x)\coloneqq x_{1}\cdots x_{n}.$ (2.2)

This weight occurs in the Dunkl transform, defined for $f\in
L^{1}(\mathbb{R}^{n},|\omega^{R}(x)|\mathrm{d}x)$ by

$\mathcal{F}^{R}f(\xi)\coloneqq\mathcal{F}_{\kappa_{R}}^{R}f(\xi)\coloneqq\frac{1}{c_{R}}\int_{\mathbb{R}^{n}}E^{R}(-i\xi,x)f(x)\omega^{R}(x)\;\mathrm{d}x,$

with the constant

$c_{R}=\int_{\mathbb{R}^{n}}e^{-\left|x\right|^{2}/2}\omega^{R}(x)\;\mathrm{d}x,$

which defines an automorphism of the Schwartz space
$\mathscr{S}(\mathbb{R}^{n})$. In particular, for $\kappa_{R}=0$, the Dunkl
transform coincides with the usual Euclidean Fourier transform. For
$f\in\mathscr{S}(\mathbb{R}^{n})$ we have

$T_{\xi}^{R}\mathcal{F}^{R}=-\mathcal{F}^{R}m_{i\xi}\quad\textrm{ and
}\quad\mathcal{F}^{R}T_{\xi}^{R}=m_{i\xi}\mathcal{F}^{R},$ (2.3)

with the multiplication operator $m_{z}\coloneqq f\mapsto\braket{z,\cdot}f$
for $z\in\mathbb{C}^{n}$.
To improve readability, we will omit the subscript $\kappa_{R}$ throughout the
paper.

##  3\. The type A Dunkl-Laplace transform

First, we summarize some facts about the Dunkl-Laplace transform from [Rö20,
BR22]. Throughout this section, we consider a non-negative multiplicity
$\kappa_{A}=k\geq 0$.
The (type A) Dunkl-Laplace transform of a locally integrable function $f\in
L^{1}_{\textrm{loc}}(\mathbb{R}^{n}_{+})$ is defined on the cone
$\mathbb{R}_{+}^{n}=]0,\infty[^{n}$ by

$\mathcal{L}f(z)\coloneqq\int_{\mathbb{R}_{+}^{n}}E^{A}(-x,z)f(x)\omega^{A}(x)\;\mathrm{d}x$

for $z\in\mathbb{C}^{n}$, provided the integral exists. Most of the analytic
aspects of this transform are based on the following properties of the type
$A$ Dunkl kernel:

$\displaystyle E^{A}(\underline{s}+w,z)$
$\displaystyle=e^{\braket{\underline{s},w}}E^{A}(w,z),$ $\displaystyle
w,z\in\mathbb{C}^{n},\,s\in\mathbb{C},$ (3.1)
$\displaystyle\left|E^{A}(-x,z)\right|$ $\displaystyle\leq
e^{-\min_{i}\mathrm{Re}\,z_{i}\cdot\left\lVert x\right\rVert_{1}},$
$\displaystyle x\in\mathbb{R}^{n}_{+},\mathrm{Re}\,z>0,$
$\displaystyle\left|E^{A}(-x,z)\right|$ $\displaystyle\leq
E^{A}(-x,\mathrm{Re}\,z),$ $\displaystyle
x\in\mathbb{R}^{n},\mathrm{Re}\,z>0,$ $\displaystyle 0$
$\displaystyle<E^{A}(-x,y)\leq 1,$ $\displaystyle x,y\in\mathbb{R}_{+}^{n},$

where $\left\lVert x\right\rVert_{1}\coloneqq\sum_{i}\left|x_{i}\right|$ and
$\underline{s}\coloneqq(s,\ldots,s)$. For us, the following properties of
$\mathcal{K}$ will be relevant:

######  Theorem 3.1 ([Rö20]).

Consider $f\in L^{1}_{\textrm{loc}}(\mathbb{R}_{+}^{n})$. Then:

  1. (i)

If $\mathcal{L}f(a)$ exists for $a\in\mathbb{R}^{n}$, then $\mathcal{L}f$
exists and defines a holmorphic function on
$H_{n}(a)\coloneqq\left\\{z\in\mathbb{C}^{n}\mid\mathrm{Re}\,z>a\right\\}$,
where $\mathrm{Re}\,z>a$ is defined componentwise. Moreover, for any
polynomial $p\in\mathbb{C}[\mathbb{R}^{n}]$,

$p(-T^{A})\mathcal{L}f(z)=\mathcal{L}(pf)(z),\quad z\in H_{n}(a).$

  2. (ii)

If $\left|f(x)\right|\leq e^{-s\left\lVert x\right\rVert_{1}}$ for some
$s\in\mathbb{R}$, then $\mathcal{L}f$ exists on $H_{n}(\underline{s})$.

  3. (iii)

If $\mathcal{L}f(\underline{s})$ exists for some $s\in\mathbb{R}$ and
$y\mapsto\mathcal{L}f(\underline{s}+iy)\in
L^{1}(\mathbb{R}^{n},\omega^{A}(x)\mathrm{d}x),$ then $f$ has a continuous
representative $f_{0}$ such that

$\frac{(-i)^{n}}{c_{A}^{2}}\int_{\mathrm{Re}\,z=\underline{s}}\mathcal{L}f(z)E^{A}(x,z)\omega^{A}(z)\;\mathrm{d}z=\begin{cases}f_{0}(x)&\textrm{if
}x>0\\\ 0&\textrm{otherwise.}\end{cases}$

The integral is understood as an $n$-fold line integral.

We consider non-symmetric Jack polynomials
$(E_{\eta})_{\eta\in\mathbb{N}_{0}^{n}}$ and symmetric Jack polynomials
$(P_{\lambda})_{\lambda\in\Lambda_{+}^{n}}$ (of index $\alpha=\tfrac{1}{k}$)
with
$\Lambda_{+}^{n}=\left\\{\lambda\in\mathbb{N}_{0}^{n}\mid\lambda_{1}\geq\ldots\lambda_{n}\geq
0\right\\}$, the partitions of length at most $n$. We assume monic
normalization, i.e. $E_{\eta}(x)=x^{\eta}+...$ and
$P_{\lambda}(x)=x^{\lambda}+...$, for $\eta\in\mathbb{N}_{0}^{n}$ and
$\lambda\in\Lambda_{+}^{n}$. For the precise definition of these polynomials
the reader is referred to [Sta89, Sa98, BF98, SZ07, Fo10].

######  Theorem 3.2 ([BR22]).

Consider $\mu\in\mathbb{C}$ with $\mathrm{Re}\,\mu>\mu_{0}\coloneqq k(n-1)$.
The Jack polynomials satisfy the following Laplace transform identities for
$\mathrm{Re}\,z>0$:

$\displaystyle\int_{\mathbb{R}_{+}^{n}}E^{A}(-x,z)E_{\eta}(x)\Delta(x)^{\mu-\mu_{0}-1}\omega^{A}(x)\;\mathrm{d}x$
$\displaystyle=\Gamma_{n}(\underline{\mu}+\eta_{+})E_{\eta}(\tfrac{1}{z})\Delta(z)^{-\mu},$
$\displaystyle\int_{\mathbb{R}_{+}^{n}}E^{A}(-x,z)P_{\lambda}(x)\Delta(x)^{\mu-\mu_{0}-1}\omega^{A}(x)\;\mathrm{d}x$
$\displaystyle=\Gamma_{n}(\underline{\mu}+\lambda)P_{\lambda}(\tfrac{1}{z})\Delta(z)^{-\mu},$

where $\eta_{+}$ is the unique partition contained in the orbit
$\mathcal{S}_{n}\eta$, and $\Gamma_{n}$ is the generalized gamma function

$\Gamma_{n}(z)\coloneqq
c_{A}(2\pi)^{-\tfrac{n}{2}}\prod\limits_{j=1}^{n}\Gamma(z_{j}-k(j-1)).$

In particular,

$\int_{\mathbb{R}_{+}^{n}}E^{A}(-x,z)\Delta(x)^{\mu-\mu_{0}-1}\omega^{A}(x)\;\mathrm{d}x=\Gamma_{n}(\underline{\mu})\Delta(z)^{-\mu}.$

There is a well-known renormalization of the symmetric Jack polynomials,
called the $C$-normalization and denoted by
$(C_{\lambda})_{\lambda\in\Lambda_{+}^{n}}$. As well as a similar
renormalization $(L_{\eta})_{\eta\in\mathbb{N}_{0}^{n}}$ of the non-symmetric
Jack polynomials, such that

$\sum\limits_{\begin{subarray}{c}\eta\in\mathbb{N}_{0}^{n}\\\
\left|\eta\right|=p\end{subarray}}L_{\eta}(x)=\sum\limits_{\begin{subarray}{c}\lambda\in\Lambda_{+}^{n}\\\
\left|\lambda\right|=p\end{subarray}}C_{\lambda}(x)=(x_{1}+\ldots+x_{n})^{p};$

see for instance [Sta89, Fo10, BR22]. For parameters $\mu\in\mathbb{C}^{p}$
and $\nu\in\mathbb{C}^{q}$ with
$\nu_{i}\notin\left\\{0,k,\ldots,k(n-1)\right\\}-\mathbb{N}_{0}$, for all
$1\leq i\leq q$, we define the Jack-hypergeometric series (cf. [BF98, Ma13,
BR22]):

$\displaystyle{}_{p}K_{q}(\mu;\nu;w,z)$
$\displaystyle\coloneqq\sum\limits_{\eta\in\mathbb{N}_{0}^{n}}\frac{[\mu_{1}]_{\eta_{+}}\cdots[\mu_{p}]_{\eta_{+}}}{[\nu_{1}]_{\eta_{+}}\cdots[\nu_{q}]_{\eta_{+}}}\frac{L_{\eta}(w)L_{\eta}(z)}{\left|\eta\right|!\,L_{\eta}(\underline{1})},$
$\displaystyle{}_{p}F_{q}(\mu;\nu;w,z)$
$\displaystyle\coloneqq\sum\limits_{\lambda\in\Lambda_{+}^{n}}\frac{[\mu_{1}]_{\lambda}\cdots[\mu_{p}]_{\lambda}}{[\nu_{1}]_{\lambda}\cdots[\nu_{q}]_{\lambda}}\frac{C_{\lambda}(w)C_{\lambda}(z)}{\left|\lambda\right|!\,C_{\lambda}(\underline{1})},$

with the generalized Pochhammer symbol

$[\alpha]_{\lambda}\coloneqq\frac{\Gamma_{n}(\underline{\alpha}+\lambda)}{\Gamma_{n}(\underline{\alpha})},\textrm{
for }\alpha\in\mathbb{C},\,\lambda\in\Lambda_{+}^{n}.$

For abbreviation we shall write

$\Gamma_{n}(\alpha)=\Gamma_{n}(\underline{\alpha})\textrm{ for
}\alpha\in\mathbb{C}\;\textrm{ and
}\;[\mu]_{\lambda}=[\mu_{1}]_{\lambda}\cdots[\mu_{p}]_{\lambda}\textrm{ for
}\mu\in\mathbb{C}^{p}.$

Considering $\mathcal{S}_{n}$-means, the two types of hypergeometric series
are related by

$\frac{1}{n!}\sum\limits_{\sigma\in\mathcal{S}_{n}}{}_{p}K_{q}(\mu;\nu;w,\sigma
z)={}_{p}F_{q}(w,z).$

######  Theorem 3.3 ([BR22]).

For $\mu\in\mathbb{C}^{p}$, $\nu\in\mathbb{C}^{q}$ with
$\nu_{i}\notin\left\\{0,k,\ldots,k(n-1)\right\\}-\mathbb{N}_{0}$ for $1\leq
i\leq q$, the following hold.

  1. (i)

If $p\leq q$, then ${}_{p}K_{q}(\mu;\nu;\cdot,\cdot)$ and
${}_{p}F_{q}(\mu;\nu;\cdot,\cdot)$ are entire. Moreover, they are holomorphic
in $(\mu,\nu)$.

  2. (ii)

If $p=q+1$, then ${}_{p}K_{q}(\mu;\nu;\cdot,\cdot)$ and
${}_{p}F_{q}(\mu;\nu;\cdot,\cdot)$ are holomorphic on the domain
$\left\\{(w,z)\in\mathbb{C}^{n}\times\mathbb{C}^{n}\mid\left\lVert
w\right\rVert_{\infty}\cdot\left\lVert z\right\rVert_{\infty}<1\right\\}$.
Again, they are holomorphic in $(\mu,\nu)$.

  3. (iii)

For $p\leq q$ and $\mathrm{Re}\,\mu^{\prime}>\mu_{0}$,

$\displaystyle\quad\quad\int_{\mathbb{R}_{+}^{n}}E^{A}(-x,z)\,{}_{p}K_{q}(\mu;\nu;w,x)$
$\displaystyle\Delta(x)^{\mu^{\prime}-\mu_{0}-1}\omega^{A}(x)dx$
$\displaystyle=\Gamma_{n}(\mu^{\prime})\Delta(z)^{-\mu^{\prime}}\,{}_{p+1}K_{q}((\mu^{\prime},\mu);\nu;w,\tfrac{1}{z}).$

for all $w,z\in\mathbb{C}^{n}$ with $\mathrm{Re}\,z>0$ if $p<q$ and
additionally $\left\lVert
w\right\rVert_{\infty}\cdot\left\lVert\tfrac{1}{z}\right\rVert_{\infty}<\tfrac{1}{n}$
if $p=q$. The same is true for ${}_{p}F_{q}$ instead of ${}_{p}K_{q}$.

  4. (iv)

For $p\leq q+1$ and $w\in\mathbb{C}^{n}$

$\quad\quad\Delta(T){}_{p}K_{q}(\mu;\nu;w,\cdot)=\frac{[\mu]_{\underline{1}}}{[\nu]_{\underline{1}}}\Delta(w)\,{}_{p}K_{q}(\mu+\underline{1};\nu+\underline{1};w,\cdot).$

The same is true for ${}_{p}F_{q}$ instead of ${}_{p}K_{q}$.

  5. (v)

${}_{0}K_{0}(w,z)=E^{A}(w,z)$ and ${}_{0}F_{0}(w,z)=J^{A}(w,z)$, as already
observed in [BF98].

##  4\. Bessel kernel and $\mathcal{K}$-Bessel function

In this section, we define a $\mathcal{K}$-Bessel function and a Bessel kernel
for the root system $A_{n-1}$. All this will be done in line with radial
analysis on symmetric cones.
To become more precise, consider an irreducible symmetric cone $\Omega=G/K$
with associated Euclidean Jordan algebra $V$ of dimension $m$, rank $n$ and
Peirce dimension constant $d$. For $k=\tfrac{d}{2}$, the Bessel function of
index
$\nu\in\mathbb{C}\setminus(\left\\{k,\ldots,k(n-1)\right\\}-\mathbb{N}_{0})$
associated to $\Omega$ is defined as the following entire function on
$V_{\mathbb{C}}$, the complexification of $V$

$\mathcal{J}_{\nu}^{\Omega}(z)={}_{0}F_{1}^{\Omega}(\nu,-z),$

where ${}_{0}F_{1}^{\Omega}$ is a hypergeometric series of spherical
polynomials (cf. [FK94]). Since $\mathcal{J}_{\nu}^{\Omega}$ is $K$-invariant,
it can be considered as function of the spectrum of $z$, so that
$\mathcal{J}_{\nu}^{\Omega}$ becomes a two-variable ${}_{0}F_{1}$ series of
Jack polynomials as introduced in Section 3, with one variable set to
$\underline{1}$. This connection is verified in the subsequent remark. It is
an important observation, that this Bessel function, more precisely the two-
variable ${}_{0}F_{1}$ series, is essentially a type $B$ Bessel function. This
is one reasons why the type $B$ Dunkl theory shows up in this setting.

######  Remark 4.1.

Let $(Z_{\lambda})_{\lambda\in\Lambda_{+}^{n}}$ be the spherical polynomials
of $\Omega$, normalized such that

$(\mathrm{tr}\,x)^{p}=\sum\limits_{\begin{subarray}{c}\lambda\in\Lambda_{+}^{n}\\\
\left|\lambda\right|=p\end{subarray}}Z_{\lambda}(x),$

where $\mathrm{tr}$ is the Jordan trace. Then
$Z_{\lambda}(x)=C_{\lambda}(\mathrm{spec}\,x)$, where
$\mathrm{spec}\,x\in\mathbb{R}^{n}$ denotes the spectrum of $x$ in decreasing
order. A hypergeometric series associated to $\Omega$ is of the form

${}_{p}F_{q}^{\Omega}(\mu;\nu;x)=\sum\limits_{\lambda\in\Lambda_{+}^{n}}\frac{[\mu]_{\lambda}}{[\nu]_{\lambda}\left|\lambda\right|!}Z_{\lambda}(x)={}_{p}F_{q}(\mu;\nu,\mathrm{spec}\,x).$

There is no canonical terminology of a two-variable hypergeometric series of a
symmetric cone, but consider for a moment the function

${}_{p}F_{q}^{\Omega}(\mu;\nu;x,y)\coloneqq{}_{p}F_{q}^{\Omega}(\mu;\nu;P(\sqrt{x})y),$

where $P$ is the quadratic representation of the Jordan algebra $V$. We note
that ${}_{0}F_{1}^{\Omega}$ occurs as kernel of the Hankel transform of the
symmetric cone, see [FK94]. By [FK94, Corollary XI.3.2], the $K$-mean of
${}_{p}F_{q}^{\Omega}$ is computed as

$\displaystyle\int_{K}{}_{p}F_{q}^{\Omega}(\mu;\nu;x,ky)\;\mathrm{d}k$
$\displaystyle=\sum\limits_{\lambda\in\Lambda_{+}^{n}}\frac{[\mu]_{\lambda}}{[\nu]_{\lambda}\left|\lambda\right|!}\int_{K}Z_{\lambda}(P(\sqrt{x})ky)\;\mathrm{d}k$
$\displaystyle=\sum\limits_{\lambda\in\Lambda_{+}^{n}}\frac{[\mu]_{\lambda}}{[\nu]_{\lambda}}\frac{Z_{\lambda}(x)Z_{\lambda}(y)}{\left|\lambda\right|!Z_{\lambda}(e)}={}_{p}F_{q}(\mu;\nu;\mathrm{spec}\,x,\mathrm{spec}\,y).$

for arbitrary $x\in\Omega$.

######  Definition 4.2.

For $\nu\in\mathbb{C}$ with
$\nu\notin\left\\{0,k,\ldots,k(n-1)\right\\}-\mathbb{N}_{0}$, we define the
type $A$ (non-)symmetric Bessel kernels as

$\displaystyle\mathscr{E}_{\nu}(w,z)$
$\displaystyle\coloneqq{}_{0}K_{1}(\nu;w,-z)=\sum\limits_{\eta\in\mathbb{N}_{0}^{n}}\frac{(-1)^{\left|\eta\right|}}{[\nu]_{\eta_{+}}}\frac{L_{\eta}(w)L_{\eta}(z)}{\left|\eta\right|!\,L_{\eta}(\underline{1})},$
$\displaystyle\mathscr{J}_{\nu}(w,z)$
$\displaystyle\coloneqq{}_{0}F_{1}(\nu;w,-z)=\sum\limits_{\lambda\in\Lambda_{+}^{n}}\frac{(-1)^{\left|\lambda\right|}}{[\nu]_{\lambda}}\frac{C_{\lambda}(w)C_{\lambda}(z)}{\left|\lambda\right|!\,C_{\lambda}(\underline{1})},$

which are entire functions in the variables $w,z$ by Theorem 3.3 and
holomorphic in the parameter $\nu\in\mathbb{C}$ satisfying the condition
above. In particular, the Bessel kernels are related via

$\frac{1}{n!}\sum\limits_{\sigma\in\mathcal{S}_{n}}\mathscr{E}_{\nu}(w,\sigma
z)=\mathscr{J}_{\nu}(w,z).$

From now on, we consider a fixed parameter $\nu\in\mathbb{C}$. The type $A$
multiplicity is

$\kappa_{A}=k\geq 0,$

and the type $B$ multiplicity is

$\kappa_{B}=(k,k^{\prime})\quad\textrm{with}\quad
k^{\prime}=\nu-\mu_{0}-\tfrac{1}{2},\quad\textrm{where}\quad\mu_{0}\coloneqq
k(n-1).$

Hence, we have a one-to-one correspondence between the pairs
$(\kappa_{A},\nu)$ and the type $B$ multiplicities $\kappa_{B}$. The following
proposition justifies the term ”Bessel-kernel” for the functions
$\mathscr{J}_{\nu}$ and $\mathscr{E}_{\nu}$.

######  Proposition 4.3.

The type $B$ Dunkl kernel and Bessel function satisfy

$\displaystyle{}_{0}K_{1}(\nu;\tfrac{w^{2}}{2},\tfrac{z^{2}}{2})$
$\displaystyle=\frac{1}{2^{n}}\sum\limits_{\tau\in\mathbb{Z}_{2}^{n}}E^{B}(w,\tau
z),$ $\displaystyle{}_{0}F_{1}(\nu;\tfrac{w^{2}}{2},\tfrac{z^{2}}{2})$
$\displaystyle=J^{B}(w,z),$

In particular,

$\displaystyle\mathscr{E}_{\nu}(\tfrac{w^{2}}{2},\tfrac{z^{2}}{2})$
$\displaystyle=\frac{1}{2^{n}}\sum\limits_{\tau\in\mathbb{Z}_{2}^{n}}E^{B}(iw,\tau
z),$ $\displaystyle\mathscr{J}_{\nu}(\tfrac{w^{2}}{2},\tfrac{z^{2}}{2})$
$\displaystyle=J^{B}(iw,z).$

We remark, that the statement on the Bessel kernel was already observed in
[Rö07, Proposition 4.5].

###### Proof.

The proof is similar to [Rö07] for the Bessel function. Using [BF98,
Proposition 4.18], we obtain the following formula for the Dunkl pairing
$[\cdot,\cdot]^{B}$ of non-symmetric Jack polynomials $E_{\eta}$

$E_{\eta}((T_{x}^{B})^{2})E_{\mu}(x^{2})\Big{|}_{x=0}=[E_{\eta}(x^{2}),E_{\mu}(x^{2})]^{B}=\begin{cases}4^{\left|\eta\right|}[\nu]_{\eta_{+}}k^{\left|\eta\right|}\tfrac{d_{\eta}^{\prime}e_{\eta}}{d_{\eta}},&\textrm{
if }\eta=\mu\\\ 0,&\textrm{ otherwise}\end{cases}$

with certain constants $d_{\eta},d_{\eta}^{\prime},e_{\eta}$, satisfying
$\tfrac{e_{\eta}}{d_{\eta}}=E_{\eta}(\underline{1})$, cf. [Fo10, Formula
(12.3.3)]. Therefore, by definition of the renormalization $L_{\eta}$ of the
non-symmetric Jack polynomials (cf. [BR22]),

$[L_{\eta}(x^{2}),L_{\mu}(x^{2})]^{B}=4^{\left|\eta\right|}\left|\eta\right|!\,[\nu]_{\eta_{+}}L_{\eta}(\underline{1})\cdot\delta_{\eta\mu},$

where $\delta_{\mu\eta}$ is the Kronecker delta. Since
$(L_{\eta})_{\eta\in\mathbb{N}_{0}^{n}}$ is a homogeneous basis for
$\mathbb{C}[\mathbb{R}^{n}]$, we have an expression

$\frac{1}{2^{n}}\sum\limits_{\tau\in\mathbb{Z}_{2}^{n}}E^{B}(w,\tau
z)=\sum\limits_{\mu\in\mathbb{N}_{0}^{n}}a_{\mu}(w)L_{\mu}(z^{2}),$

with certain coefficients $a_{\mu}(w)\in\mathbb{C}$. Together with the
$\mathbb{Z}_{2}^{n}$-invariance of $L_{\mu}(x^{2})$, the eigenvalue equation
for the Dunkl kernel leads to

$\displaystyle L_{\eta}(w^{2})$
$\displaystyle=\frac{1}{2^{n}}\sum\limits_{\tau\in\mathbb{Z}_{2}^{n}}L_{\eta}((\tau
w)^{2})E^{B}(\tau
w,z)\Big{|}_{z=0}=\frac{1}{2^{n}}\sum\limits_{\tau\in\mathbb{Z}_{2}^{n}}L_{\eta}((T_{z}^{B})^{2})E^{B}(\tau
w,z)\Big{|}_{z=0}$
$\displaystyle=\sum\limits_{\mu\in\mathbb{N}_{0}^{n}}a_{\mu}(w)[L_{\eta}(z^{2}),L_{\mu}(z^{2})]^{B}=a_{\eta}(w)4^{\left|\eta\right|}\left|\eta\right|!\,[\nu]_{\eta_{+}}L_{\eta}(\underline{1}).$

Finally, equating coefficients leads to the stated formula. ∎

######  Lemma 4.4.

Let $w,z\in\mathbb{C}^{n}$ with $\mathrm{Re}\,z>0$ and
$\mathrm{Re}\,\nu>\mu_{0}$. Then

$\displaystyle\int_{\mathbb{R}_{+}^{n}}E^{A}(-x,z)\mathscr{E}_{\nu}(w,x)\Delta(x)^{\nu-\mu_{0}-1}\omega^{A}(x)\;\mathrm{d}x$
$\displaystyle=\Gamma_{n}(\nu)E^{A}(w,-\tfrac{1}{z})\Delta(z)^{-\nu},$
$\displaystyle\int_{\mathbb{R}_{+}^{n}}J^{A}(-x,z)\mathscr{J}_{\nu}(w,x)\Delta(x)^{\nu-\mu_{0}-1}\omega^{A}(x)\;\mathrm{d}x$
$\displaystyle=\Gamma_{n}(\nu)J^{A}(w,-\tfrac{1}{z})\Delta(z)^{-\nu}.$

###### Proof.

This is immediate from part (iii) of Theorem 3.3 and the observation

${}_{1}K_{1}(\nu;\nu;w,z)={}_{0}K_{0}(w,z)=E^{A}(w,z).$

∎

######  Theorem 4.5 (Integral representation).

Let $\mathrm{Re}\,\nu>2\mu_{0}+1,w\in\mathbb{C}^{n}$ and
$x\in\mathbb{R}_{+}^{n}$. Then for all $s\in\mathbb{R}_{+}$, we have

$\displaystyle\mathscr{E}_{\nu}(w,x)\Delta(x)^{\nu-\mu_{0}-1}$
$\displaystyle=\frac{\Gamma_{n}(\nu)}{c_{A}^{2}\,i^{n}}\int_{\mathrm{Re}(\zeta)=\underline{s}}E^{A}(x,\zeta)E^{A}(w,-\tfrac{1}{\zeta})\Delta(\zeta)^{-\nu}\omega^{A}(\zeta)\,\mathrm{d}\zeta,$
$\displaystyle\mathscr{J}_{\nu}(w,x)\Delta(x)^{\nu-\mu_{0}-1}$
$\displaystyle=\frac{\Gamma_{n}(\nu)}{c_{A}^{2}\,i^{n}}\int_{\mathrm{Re}(\zeta)=\underline{s}}J^{A}(x,\zeta)J^{A}(w,-\tfrac{1}{\zeta})\Delta(\zeta)^{-\nu}\omega^{A}(\zeta)\,\mathrm{d}\zeta.$

###### Proof.

Due to the Cauchy-type inversion formula of the Dunkl-Laplace transform
(Theorem 3.1 (iii)), it suffices to prove that the right hand side of Lemma
4.4 is integrable as a function of $z$ over $\underline{s}+i\mathbb{R}^{n}$
for $s>0$ with respect to the measure $\omega^{A}(x)\mathrm{d}x$. The map
$z\mapsto\tfrac{1}{z}$ is bounded on $\underline{s}+i\mathbb{R}^{n}$ and so is
$z\mapsto E^{A}(w,-\tfrac{1}{z})$. Thus, we only have to verify the
integrability condition for $\Delta(z)^{-\nu}$. Since
$\omega^{A}(x)=\left|D(x)\right|^{2k}$ with $D(x)=\prod_{i<j}(x_{i}-x_{j})$ of
degree $(n-1)$ in $x_{i}$, the function

$y\mapsto\Delta(\underline{s}+y^{2})^{-\tfrac{1}{2}\mathrm{Re}\,\nu}\omega^{A}(y)$

is integrable over $\mathbb{R}^{n}$ iff $\mathrm{Re}\,\nu>2\mu_{0}+1$, since
$\sqrt{\underline{s}+y_{i}^{2}}\sim\left|y_{i}\right|$ for large
$\left|y_{i}\right|$. Hence, the claim holds for
$\mathrm{Re}\,\nu>2\mu_{0}+1$. ∎

######  Corollary 4.6 (Recurrence formulas).

For $w,z\in\mathbb{C}^{n}$ and $\nu\in\mathbb{C}$ with
$\mathrm{Re}\,\nu>\mu_{0}$,

  1. (i)

$\displaystyle\Delta(T_{z}^{A})\mathscr{E}_{\nu}(w,z)=\frac{(-1)^{n}}{[\nu]_{\underline{1}}}\Delta(w)\mathscr{E}_{\nu+1}(w,z)$.

  2. (ii)

In addition $\mathrm{Re}\,z>0$, then

$\Delta(T_{z}^{A})\left(\mathscr{E}_{\nu}(w,z)\Delta(z)^{\nu-\mu_{0}-1}\right)=[\nu-1]_{\underline{1}}\,\mathscr{E}_{\nu-1}(w,z)\Delta(z)^{\nu-\mu_{0}-2}.$

The same formulas is true if $\mathscr{E}_{\nu}$ is replaced by
$\mathscr{J}_{\nu}$. In particular, due to the second recurrence property,
$\mathscr{E}_{\nu}$ and $\mathscr{J}_{\nu}$ can be analytically extended to
$\nu\in\mathbb{C}$ as functions on
$\mathbb{C}^{n}\times\left\\{\mathrm{Re}\,z>0\right\\}$, so that (i) and (ii)
are both valid for arbitrary $\nu\in\mathbb{C}$.

###### Proof.

  1. (i)

Denoting $f^{-}(x)=f(-x)$, we have $T_{\xi}^{A}f^{-}=-(T_{\xi}^{A}f)^{-}$.
Hence, part (i) is a consequence of Theorem 3.3 (iv).

  2. (ii)

By analyticity in $\nu$ and $z$, we can assume that
$\mathrm{Re}\,\nu>2\mu_{0}+2$ and $z=x\in\mathbb{R}_{+}^{n}$. The claim can be
deduced from the integral representation of the Bessel kernels in Theorem 4.5.
By changing the order of differentation and integration, the stated formula
follows from $\Delta(T_{x}^{A})E^{A}(x,\zeta)=\Delta(\zeta)E^{A}(x,\zeta)$ and
$[\nu-1]_{\underline{1}}=\tfrac{\Gamma_{n}(\nu)}{\Gamma_{n}(\nu-1)}$. By
[Rö03, Proposition 2.6] we have

$\left|\partial_{\xi}^{x}E^{A}(x,\underline{s}+iy)\right|\leq(s+\left|y\right|)e^{\braket{x,\underline{s}}},$

which is locally bounded in $x$. By similar computations as in Theorem 4.5,
the condition $\mathrm{Re}\,\nu>2\mu_{0}+2$ shows that the function

$y\mapsto\partial_{\xi}^{x}E^{A}(x,\underline{s}+iy)\Delta(\underline{s}+iy)^{\nu-\mu_{0}-1}\omega^{A}(y)$

is dominated on $\mathbb{R}^{n}$ by

$y\mapsto
e^{\braket{x,\underline{s}}}(s+\left|y\right|)\Delta(\underline{s}+iy)^{\nu-\mu_{0}-1}\omega^{A}(y),$

which is locally bounded in $x$ and integrable over $\mathbb{R}^{n}$. This
justifies the change of order of differentation and integration.

∎

We now come to the definition of the Dunkl-type $\mathcal{K}$-Bessel function
which shares important properties with the $\mathcal{K}$-Bessel functions on
symmetric cones. Recall, that on a symmetric cone $\Omega$ the
$\mathcal{K}$-Bessel function is defined for $\mathbf{s}\in\mathbb{C}^{n}$ as

$\mathcal{K}_{\mathbf{s}}^{\Omega}(x,y)=\int_{\Omega}e^{-\braket{x,u^{-1}}-\braket{y,u}}\Delta_{\mathbf{s}}(u)\det(u)^{-\tfrac{m}{n}}\;\mathrm{d}u,$
(4.1)

where $\Delta_{\mathbf{s}}$ is the generalized power function of $\Omega$
(which coincides with $\det^{s}$ if $\mathbf{s}=(s,\ldots,s)$), see [FK94,
Chapter XVI, Section 3]. The $\mathcal{K}$-Bessel function for arbitrary
symmetric cones was first considered in [Cl88] and further studied in [Di90].

######  Definition 4.7.

The Dunkl-type $\mathcal{K}$-Bessel of index $\nu\in\mathbb{C}$ is defined by

$\mathcal{K}_{\nu}(w,z)\coloneqq\int_{\mathbb{R}_{+}^{n}}E^{A}(-x,w)E^{A}(-\tfrac{1}{x},z)\Delta(x)^{\nu-\mu_{0}-1}\omega^{A}(x)\;\mathrm{d}x.$

The convergence and further properties of the integral will be investigated in
the subsequent theorem.

By a change of variables, the following Lemma is immediate.

######  Lemma 4.8.

On $\mathbb{R}_{+}^{n}$, the $\mathcal{S}_{n}$-invariant measure

$\Delta(x)^{-\mu_{0}-1}\omega^{A}(x)\mathrm{d}x$ (4.2)

is invariant under the transformation $x\mapsto\tfrac{1}{x}$ as well as under
$x\mapsto sx$ for $s>0$.

This invariance will be highly relevant in the subsequent results. The measure
(4.2) has to be understood as the Dunkl analogue of the invariant measure of a
symmetric cone. In fact, if $k$ is related to a symmetric cone $\Omega=G/K$,
then the measure (4.2) is the radial part of the $G$-invariant measure on
$\Omega$, cf. [FK94, Theorem VI.2.3].

######  Remark 4.9.

Consider the situation of an irreducible symmetric cone $\Omega=G/K$ inside a
simple Euclidean Jordan algebra $V$ of dimension $m$, rank $n$ and with Peirce
dimension constant $d$. Fix a Jordan frame $(c_{1},\ldots,c_{n})$, write
$k=\tfrac{d}{2}$ and denote by

$\mathrm{spec}\,x\in
C_{+}\coloneqq\left\\{\xi\in\mathbb{R}_{+}^{n}\mid\xi_{1}\geq\ldots\geq\xi_{n}\right\\}$

the spectrum of $x\in\Omega$ or, more generally
$\mathrm{spec}\,x\in\mathbb{R}^{n}$ for $x\in V$. By [Rö20, Remark 3.2], the
$K$-mean of the $\mathcal{K}$-Bessel function (4.1) with the particular index
$\mathbf{s}=\underline{\nu}=(\nu,\ldots,\nu)$ becomes

$\displaystyle(\mathcal{K}_{\underline{\nu}}^{\Omega})^{\\#}(x,y)$
$\displaystyle\coloneqq\int_{\Omega}\left(\int_{K}e^{-(kx,u^{-1})}\;\mathrm{d}k\right)e^{-(y,u)}\det(u)^{-\nu-\tfrac{m}{n}}\;\mathrm{d}u$
$\displaystyle=\int_{\Omega}J^{A}(-\mathrm{spec}\,x,\mathrm{spec}\,u^{-1})e^{-(y,u)}\det(u)^{-\nu-\frac{m}{n}}\;\mathrm{d}u$
$\displaystyle=c_{0}\int_{C_{++}}J^{A}(-\mathrm{spec}\,x,\tfrac{1}{\xi})\left(\int_{K}e^{-(y,k\xi)}\;dk\right)\Delta(\xi)^{-\nu-\mu_{0}-1}\omega^{A}(\xi)\;d\xi$
$\displaystyle=c_{0}\int_{C_{++}}J^{A}(-\mathrm{spec}\,x,\tfrac{1}{\xi})J^{A}(-\mathrm{spec}\,y,\xi)\Delta(\xi)^{-\nu-\mu_{0}-1}\omega^{A}(\xi)\;d\xi$
$\displaystyle=\frac{c_{0}}{n!}\sum_{\sigma\in\mathcal{S}_{n}}\mathcal{K}_{-\nu}(\mathrm{spec}\,y,\sigma\mathrm{spec}\,x)=\frac{c_{0}}{n!}\sum_{\sigma\in\mathcal{S}_{n}}\mathcal{K}_{\nu}(\sigma\mathrm{spec}\,x,\mathrm{spec}y),$

where $C_{++}=\left\\{\xi\in\mathbb{R}^{n}\mid\xi_{1}\geq\ldots\xi_{n}\geq
0\right\\}$ and the last equality is verified in the next theorem. Hence, this
definition of the $\mathcal{K}$-Bessel function in the Dunkl setting is
consistent with that on symmetric cones up to a constant factor and
symmetrization. In particular, note that the one variable versions are related
by

$\mathcal{K}_{\underline{\nu}}^{\Omega}(x,e)=\mathcal{K}_{\nu}(\mathrm{spec}\,x,\underline{1}).$

######  Theorem 4.10.

The Dunkl-type $\mathcal{K}$-Bessel function $\mathcal{K}_{\nu}$ exists for
all $\nu\in\mathbb{C}$ and $w,z\in\mathbb{C}^{n}$ with
$\mathrm{Re}\,w,\mathrm{Re}\,z>0$. Moreover, $\mathcal{K}$ is holomorphic in
$\nu,w,z$ and has the following properties:

  1. (i)

$\mathcal{K}_{\nu}(\sigma w,\sigma z)=\mathcal{K}_{\nu}(w,z)$ for all
$\sigma\in\mathcal{S}_{n}$.

  2. (ii)

$\mathcal{K}_{\nu}(w,z)=\mathcal{K}_{-\nu}(z,w)$ and
$\left|\mathcal{K}_{\nu}(w,z)\right|\leq\mathcal{K}_{\mathrm{Re}\,\nu}(\mathrm{Re}\,w,\mathrm{Re}\,z)$.

  3. (iii)

If $\nu\in\mathbb{R}$ and $x,y\in\mathbb{R}_{+}^{n}$, then

$0<\mathcal{K}_{\nu}(x,y)\leq\begin{cases}\Gamma_{n}(\nu)\Delta(x)^{-\nu}&\textrm{
if }\nu>\mu_{0}\\\ \Gamma_{n}(-\nu)\Delta(y)^{\nu}&\textrm{ if
}\nu<-\mu_{0}\end{cases}$

and $\nu\mapsto\mathcal{K}_{\nu}(x,y)$ is convex.

  4. (iv)

If $\nu\in\mathbb{R}$, $\left|\nu\right|\leq\mu_{0}$ and
$x,y\in\mathbb{R}_{+}^{n}$, then for all $\epsilon>0$,

$0<\mathcal{K}_{\nu}(x,y)\leq\Gamma_{n}(\mu_{0}+1)\Delta(x)^{-\mu_{0}-1}+\Gamma_{n}(\mu_{0}+\epsilon)\Delta(y)^{-\mu_{0}-\epsilon}.$

  5. (v)

Recurrence formulas:

$\displaystyle\Delta(T^{A})\mathcal{K}_{\nu}(w,\cdot)$
$\displaystyle=(-1)^{n}\mathcal{K}_{\nu-1}(w,\cdot),$
$\displaystyle\Delta(T^{A})\mathcal{K}_{\nu}(\cdot,z)$
$\displaystyle=(-1)^{n}\mathcal{K}_{\nu+1}(\cdot,z).$

  6. (vi)

If $\mathrm{Re}\,\nu<-\mu_{0}$, then

$\lim\limits_{\epsilon\to 0}\epsilon^{-n\nu}\mathcal{K}_{\nu}(w,\epsilon
z)=\Gamma_{n}(-\nu)\Delta(z)^{\nu}.$

If $\mathrm{Re}\,\nu>\mu_{0}$, then

$\lim\limits_{\epsilon\to 0}\epsilon^{n\nu}\mathcal{K}_{\nu}(\epsilon
w,z)=\Gamma_{n}(\nu)\Delta(w)^{-\nu}.$

These properties generalize those of $\mathcal{K}$-Bessel functions associated
to symmetric cones as stated in [FK94, Chapter XVI, Section 3].

The proof is similar to the case of symmetric cones as in [FK94, Chapter XVI,
Section 3], see also [Ru06] for real positive definite symmetric matrices.

###### Proof.

Let $\mathrm{Re}\,\nu>\mu_{0}$ and $w,z\in\mathbb{C}^{n}$ with
$\mathrm{Re}\,w,\mathrm{Re}\,z>0$. According to the estimates (3.1),

$\left|E^{A}(-x,w)E^{A}(-\tfrac{1}{x},z)\right|\leq E^{A}(-x,\mathrm{Re}\,w).$

Hence, the Laplace transform identities in Theorem 3.2 lead to

$\displaystyle\int_{\mathbb{R}_{+}^{n}}$
$\displaystyle\left|E^{A}(-x,w)E^{A}(-\tfrac{1}{x},z)\Delta(x)^{\nu-\mu_{0}-1}\right|\omega^{A}(x)\;\mathrm{d}x$
$\displaystyle\leq\int_{\mathbb{R}_{+}^{n}}E^{A}(-x,\mathrm{Re}\,w)\Delta(x)^{\mathrm{Re}\,\nu-\mu_{0}-1}\omega^{A}(x)\;\mathrm{d}x=\Gamma_{n}(\mathrm{Re}\,\nu)\Delta(\mathrm{Re}\,w)^{-\mathrm{Re}\,\nu}.$

In particular, $\mathcal{K}_{\nu}(w,z)$ exists and is holomorphic in
$(w,z,\nu)\in\left\\{\mathrm{Re}\,w>0\right\\}\times\left\\{\mathrm{Re}\,z>0\right\\}\times\left\\{\mathrm{Re}\,\nu>\mu_{0}\right\\}$
by usual theorems on holomorphic parameter integrals. Moreover, the stated
estimate in (iii) is true in the case $\mathrm{Re}\,\nu>\mu_{0}$.
Consider the case $\mathrm{Re}\,\nu<-\mu_{0}$. By the change of variables
$\xi\mapsto\tfrac{1}{\xi}$ in the defining integral for
$\mathcal{K}_{\nu}(w,z)$ and Lemma 4.8 we have
$\mathcal{K}_{\nu}(w,z)=\mathcal{K}_{-\nu}(z,w)$. Thus,
$\mathcal{K}_{\nu}(w,z)$ exists and is holomorphic in
$(w,z,\nu)\in\left\\{\mathrm{Re}\,w>0\right\\}\times\left\\{\mathrm{Re}\,z>0\right\\}\times\left\\{\mathrm{Re}\,\nu<-\mu_{0}\right\\}$.
Thereby, the estimate in (iii) for $\mathrm{Re}\,\nu<-\mu_{0}$ is true as
well. Finally, for $x\in\mathbb{R}_{+}^{n}$,
$\nu\mapsto\Delta(x)^{\mathrm{Re}\,\nu-\mu_{0}-1}$ is convex on $\mathbb{R}$,
so that $\mathcal{K}_{\nu}(w,z)$ exists and is holomorphic in
$(w,z,\nu)\in\left\\{\mathrm{Re}\,w>0\right\\}\times\left\\{\mathrm{Re}\,z>0\right\\}\times\mathbb{C}$,
where $\nu\mapsto\mathcal{K}_{\nu}(x,y)$ is convex on $\mathbb{R}$ for fixed
$x,y\in\mathbb{R}_{+}^{n}$. It remains to prove the properties (i), (ii) and
(iv)-(vi).

  1. (i)

By properties of the Dunkl kernel, this follows by a change of variables.

  2. (ii)

By Lemma 4.8, this is done by the change of variables $x\mapsto\tfrac{1}{x}$.
The estimate is an immediate consequence of (3.1).

  3. (iv)

To prove the stated estimate, consider an arbitrary $\epsilon>0$ and split the
defining integral for $\mathcal{K}_{\nu}$ according to

$\mathbb{R}_{+}^{n}=\left\\{\Delta(\xi)<1\right\\}\sqcup\left\\{\Delta(\xi)>1\right\\}\sqcup\left\\{\Delta(\xi)=1\right\\},$

where the last set has measure zero and need not to be discussed.

     * (a)

Since $E^{A}(-\tfrac{1}{\xi},y)\leq 1$ for $\xi,y\in\mathbb{R}_{+}^{n}$ and
$\Delta(\xi)^{\nu-\mu_{0}-1}<1$ for $\Delta(\xi)>1$, we obtain

$\displaystyle\int_{\Delta(\xi)>1}E^{A}(-\xi,x)E^{A}(-\tfrac{1}{\xi},y)\Delta(\xi)^{\nu-\mu_{0}-1}\omega^{A}(\xi)\;\mathrm{d}\xi$
$\displaystyle\leq\int_{\mathbb{R}_{+}^{n}}E^{A}(-\xi,x)\omega^{A}(\xi)\;\mathrm{d}\xi=\Gamma_{n}(\mu_{0}+1)\Delta(x)^{-\mu_{0}-1},$

where again Theorem 3.2 was used.

     * (b)

As $\Delta(\xi)^{-\nu-\mu_{0}-1}<\Delta(\xi)^{\epsilon-1}$ for
$\Delta(\xi)>1$, we obtain

$\displaystyle\int_{\Delta(\xi)<1}E^{A}(-\xi,x)E^{A}(-\tfrac{1}{\xi},y)\Delta(\xi)^{\nu-\mu_{0}-1}\omega^{A}(\xi)\;\mathrm{d}\xi$
$\displaystyle=\int_{\Delta(\xi)>1}E^{A}(-\tfrac{1}{\xi},x)E^{A}(-\xi,y)\Delta(\xi)^{-\nu-\mu_{0}-1}\omega^{A}(\xi)\;\mathrm{d}\xi$
$\displaystyle\leq\int_{\mathbb{R}_{+}^{n}}E^{A}(-\xi,x)\Delta(\xi)^{\epsilon-1}\omega^{A}(\xi)\;\mathrm{d}\xi=\Gamma_{n}(\mu_{0}+\epsilon)\Delta(x)^{-\mu_{0}-\epsilon}.$

  4. (v)

This is a consequence of the eigenvalue equation for $E^{A}$.

  5. (vi)

It suffices to check the second limit, the first one can then be deduced from
(ii). Recall from (3.1), that $\left|E^{A}(-\tfrac{\epsilon}{x},z)\right|\leq
1$ for $\mathrm{Re}\,z>0$. By the change of variables
$x\mapsto\tfrac{x}{\epsilon}$, dominated convergence and Theorem 3.2, we just
get

$\displaystyle\quad\quad\quad\epsilon^{n\nu}\mathcal{K}_{\nu}(\epsilon w,z)$
$\displaystyle=\epsilon^{n\nu}\int_{\mathbb{R}_{+}^{n}}E^{A}(-x,\epsilon
w)E^{A}(-\tfrac{1}{x},z)\Delta(x)^{\nu-\mu_{0}-1}\omega^{A}(x)\;\mathrm{d}x$
$\displaystyle=\int_{\mathbb{R}_{+}^{n}}E^{A}(-x,w)E^{A}(-\tfrac{\epsilon}{x},z)\Delta(x)^{\nu-\mu_{0}-1}\omega^{A}(x)\;\mathrm{d}x,$

and as $\epsilon$ tends to $0$, the last integral converges to

$\int_{\mathbb{R}_{+}^{n}}E^{A}(-x,w)\Delta(x)^{\nu-\mu_{0}-1}\omega^{A}(x)\;\mathrm{d}x=\Gamma_{n}(\nu)\Delta(w)^{-\nu}.$

∎

######  Proposition 4.11.

For $w\in\mathbb{C}^{n}$ with $\mathrm{Re}\,w>0$ put
$f_{w}(x)=\mathcal{K}_{\nu-\mu_{0}-\tfrac{1}{2}}(x^{2},w)$. Then, $f_{w}$
satisfies the eigenvalue equation

$\frac{1}{4}(T_{i}^{B})^{2}f_{w}=w_{i}f_{w}\quad\textrm{ for all
}i=1,\ldots,n,$

where the type $B$ multiplicity of the Dunkl operator on the left side is
given as before by the value $k$ on $\pm e_{i}\pm e_{j}$ and
$k^{\prime}=\nu-\mu_{0}-\tfrac{1}{2}$ on $\pm e_{i}$.

This is a generalization of the Bessel system on a symmetric cone, which is
solved by the $\mathcal{K}$-Bessel function, see for instance [FK94, Page 358]
for the one-variable case, i.e. $w=\underline{1}$, or [Mö13] for some further
eigenvalue equations of the $\mathcal{K}$-Bessel function.

###### Proof.

To improve readability we put $\nu^{\prime}=\nu+\tfrac{1}{2}$. The proof will
be done by direct computation and is divided into several steps:

  1. (i)

First, consider a $C^{2}$-function $f(x)=f_{0}(x^{2})$. Then, as noticed in
[BF98],

$\hskip
25.0pt\frac{1}{4}(T_{i}^{B})^{2}f(x)\\!=\\!x_{i}^{2}((T_{i}^{A})^{2}f_{0})(x^{2})+(\nu^{\prime}-\mu_{0})(T_{i}^{A}f_{0})(x^{2})+k\sum\limits_{j\neq
i}(T_{i}^{A}f_{0})(s_{ij}x^{2}).$

Hence,

$\hskip
25.0pt\frac{1}{4}(T_{i,x}^{B})^{2}E^{A}(-x^{2},\xi)\\!=\\!\left(x_{i}^{2}\xi_{i}^{2}-(\nu^{\prime}-\mu_{0})\xi_{i}\right)\\!E^{A}(-x^{2},\xi)-k\sum\limits_{j\neq
i}\xi_{i}E^{A}(-x^{2},s_{ij}\xi).$

  2. (ii)

By a change of variables and $E^{A}(\sigma x,\sigma y)=E^{A}(x,y)$ for
$\sigma\in\mathcal{S}_{n}$, we have

$\displaystyle\hskip 25.0pt\int_{\mathbb{R}_{+}^{n}}\sum\limits_{j\neq
i}\xi_{i}$ $\displaystyle
E^{A}(-x^{2},s_{ij}\xi)E^{A}(-\tfrac{1}{\xi},w)\Delta(\xi)^{\nu^{\prime}-2(\mu_{0}+1)}\omega_{k}(\xi)\;\mathrm{d}\xi$
$\displaystyle=\int_{\mathbb{R}_{+}^{n}}E^{A}(-x^{2},\xi)\left(\sum\limits_{j\neq
i}\xi_{j}E^{A}(-\tfrac{1}{s_{ij}\xi},w)\right)\Delta(\xi)^{\nu^{\prime}-2(\mu_{0}+1)}\omega_{k}(\xi)\;\mathrm{d}\xi.$

  3. (iii)

For abbreviation put $g(\xi)=E^{A}(\xi,w)$, so that

$\displaystyle\hskip 35.0ptT_{i,\xi}^{A}$
$\displaystyle(\xi_{i}^{2}g(-\tfrac{1}{\xi}))$ $\displaystyle=\frac{\partial
g}{\partial\xi_{i}}(-\tfrac{1}{\xi})+2\xi_{i}g(-\tfrac{1}{\xi})+k\sum\limits_{j\neq
i}\frac{\xi_{i}^{2}g(-\tfrac{1}{\xi})-\xi_{j}^{2}g(-\tfrac{1}{s_{ij}\xi})}{\xi_{i}-\xi_{j}}$
$\displaystyle=(T_{i}^{A}g)(-\tfrac{1}{\xi})+2\xi_{i}g(-\tfrac{1}{\xi_{i}})+k\sum\limits_{j\neq
i}\Bigg{(}\frac{\xi_{i}^{2}g(-\tfrac{1}{\xi})-\xi_{j}^{2}g(-\tfrac{1}{s_{ij}\xi})}{\xi_{i}-\xi_{j}}$
$\displaystyle\quad-\frac{g(-\tfrac{1}{\xi})-g(-s_{ij}\tfrac{1}{\xi})}{\tfrac{1}{\xi_{j}}-\tfrac{1}{\xi_{i}}}\Bigg{)}$
$\displaystyle=(w_{i}+2\xi_{i})g(-\tfrac{1}{\xi})+k\sum\limits_{j\neq
i}\xi_{i}g(-\tfrac{1}{\xi})+\xi_{j}g(-\tfrac{1}{s_{ij}\xi})$
$\displaystyle=(w_{i}+(2+\mu_{0})\xi_{i})g(-\tfrac{1}{\xi})+k\sum\limits_{j\neq
i}\xi_{j}g(-\tfrac{1}{s_{ij}\xi}).$

Moreover, since $T_{i}^{A}$ acts on $\mathcal{S}_{n}$-invariant functions as a
partial derivative,

$T_{i,\xi}^{A}\Delta(\xi)^{\nu^{\prime}-2(\mu_{0}+1)}=(\nu^{\prime}-2(\mu_{0}+1))\frac{\Delta(\xi)^{\nu^{\prime}-2(\mu_{0}+1)}}{\xi_{i}}.$

Therefore,

$\displaystyle
T_{i,\xi}^{A}\left(\xi_{i}^{2}E^{A}(-\tfrac{1}{\xi},w)\Delta(\xi)^{\nu^{\prime}-2(\mu_{0}+1)}\right)$
$\displaystyle=\Big{[}\left(w_{i}+(\nu^{\prime}-\mu_{0})\xi_{i}\right)E^{A}(-\tfrac{1}{\xi},w)+k\sum\limits_{j\neq
i}\xi_{j}E^{A}(-\tfrac{1}{s_{ij}\xi},w)\Big{]}\Delta(\xi)^{\nu^{\prime}-2(\mu_{0}+1)}.$

Thus, we conclude that

$\displaystyle\int_{\mathbb{R}_{+}^{n}}x_{i}^{2}\xi_{i}^{2}E^{A}(-x^{2},\xi)E^{A}(-\tfrac{1}{\xi},w)\Delta(\xi)^{\nu^{\prime}-2(\mu_{0}+1)}\omega^{A}(\xi)\;\mathrm{d}\xi$
$\displaystyle=-\int_{\mathbb{R}_{+}^{n}}\xi_{i}^{2}T_{i,\xi}^{A}E^{A}(-x^{2},\xi)E^{A}(-\tfrac{1}{\xi},w)\Delta(\xi)^{\nu^{\prime}-2(\mu_{0}+1)}\omega^{A}(\xi)\;\mathrm{d}\xi$
$\displaystyle=\int_{\mathbb{R}_{+}^{n}}E^{A}(-x^{2},\xi)T_{i,\xi}^{A}\left(\xi_{i}^{2}E^{A}(-\tfrac{1}{\xi},w)\Delta(\xi)^{\nu^{\prime}-2(\mu_{0}+1)}\right)\omega^{A}(\xi)\;\mathrm{d}\xi$
$\displaystyle=\int_{\mathbb{R}_{+}^{n}}\left(w_{i}+(\nu^{\prime}-\mu_{0})\xi_{i}\right)E^{A}(-x^{2},\xi)E^{A}(-\tfrac{1}{\xi},w)\Delta(\xi)^{\nu^{\prime}-2(\mu_{0}+1)}\omega^{A}(\xi)\;\mathrm{d}\xi$
$\displaystyle\quad+\int_{\mathbb{R}_{+}^{n}}E^{A}(-x^{2},\xi)\cdot
k\sum\limits_{j\neq
i}\xi_{j}E^{A}(-\tfrac{1}{s_{ij}\xi},w)\Delta(\xi)^{\nu^{\prime}-2(\mu_{0}+1)}\omega^{A}(\xi)\;\mathrm{d}\xi.$

Here, the second equation is justified by the skew symmetry of the Dunkl
operators on $L^{2}(\mathbb{R}^{n},\omega^{A}(x)\mathrm{d}x)$, which is based
on integration by parts. So it suffices to show that there occur no boundary
terms if we integrate by parts, which can be seen as follows: As
$E^{A}(-\tfrac{1}{\xi},w)\longrightarrow 0$ for
$\xi\to\partial\mathbb{R}_{+}^{n}$ and as $E^{A}(-x^{2},\xi)\longrightarrow 0$
exponetially for $\xi\to\infty$ (estimate 3.1), the boundary terms vanish on
$\partial\mathbb{R}_{+}^{n}$ and in $\infty$.

  4. (iv)

Putting the things together, we obtain

$\displaystyle\frac{1}{4}$
$\displaystyle(T_{i,x}^{B})^{2}\mathcal{K}_{\nu^{\prime}-\mu_{0}-1}(x^{2},w)$
$\displaystyle=\int_{\mathbb{R}_{+}^{n}}\frac{1}{4}(T_{i,x}^{B})^{2}E^{A}(-x^{2},\xi)E^{A}(-\tfrac{1}{\xi},w)\Delta(\xi)^{\nu^{\prime}-2(\mu_{0}+1)}\omega(\xi)\;\mathrm{d}\xi$
$\displaystyle=w_{i}\mathcal{K}_{\nu^{\prime}-\mu_{0}-1}(x^{2},w).$

∎

We may also study the $\mathcal{K}$-Bessel functions with a multivariate
index, similar to those on symmetric cones. For this, we consider for
$\lambda\in\mathbb{C}^{n}$ the unique analytic solution
$\mathcal{G}(\lambda,\cdot)\coloneqq\mathcal{G}_{k}(\lambda,\cdot)$ in
$\left\\{\mathrm{Re}\,z>0\right\\}$ of the following system

$\begin{cases}\mathscr{D}_{j}(k)f=\lambda_{j}f,\quad\textrm{for all
}j=1,\ldots,n\\\ f(\underline{1})=1\end{cases}$

with the so called Cherednik operators

$\mathscr{D}_{j}(k)f(x)=\Big{(}x_{j}T_{j}^{A}(k)+k(1-n)+k\sum\limits_{i>j}s_{ij}\Big{)}f(x).$

$\mathcal{G}$ is called the Cherednik kernel of type $A_{n-1}$. The reader is
referred to [BR22] where this kernel and its Dunkl-Laplace transform were
studied. We summarize the following properties:

######  Theorem 4.12 ([BR22]).

For arbitrary $\lambda,z\in\mathbb{C}^{n}$ with $\mathrm{Re}\,z>0$ and
$\nu\in\mathbb{C}$, the Cherednik kernel $\mathcal{G}$ of type $A_{n-1}$
satisfies:

  1. (i)

If $\lambda\in\Lambda_{+}^{n}$, then
$\mathcal{G}(\lambda-\rho,z)=\frac{E_{\lambda}(z)}{E_{\lambda}(\underline{1})}$
is a normalized non-symmetric Jack polynomials of index $\alpha=\tfrac{1}{k}$
and $\rho=\rho(k)=-\frac{k}{2}(n-1,n-3,\ldots,-n+3,-n+1)$.

  2. (ii)

$\mathcal{G}$ is positive on $\mathbb{R}^{n}\times\mathbb{R}_{+}^{n}$.

  3. (iii)

$\mathcal{G}(\lambda+\underline{\nu},z)=\Delta(z)^{\nu}\mathcal{G}(\lambda,z)$
and $\mathcal{G}(-\rho,z)=1$.

  4. (iv)

$\left|\mathcal{G}(\lambda,x)\right|\leq\mathcal{G}(\mathrm{Re}\,\lambda,x)$
and
$\left|\mathcal{G}(\lambda-\rho,x)\right|\leq\max\limits_{\sigma\in\mathcal{S}_{n}}x^{\sigma\mathrm{Re}\,\lambda}$
for all $x\in\mathbb{R}_{+}^{n}$.

  5. (v)

For $\lambda\in\mathbb{C}^{n}$ with $\mathrm{Re}\,\lambda>\mu_{0}$,

$\int_{\mathbb{R}_{+}^{n}}E^{A}(-z,x)\mathcal{G}(\lambda-\rho,x)\Delta(x)^{-\mu_{0}-1}\omega^{A}(x)\;\mathrm{d}x=\Gamma_{n}(\lambda)\mathcal{G}(\lambda,\tfrac{1}{x}).$

  6. (vi)

$\mathcal{G}(-\lambda,z)=\mathcal{G}(\lambda^{R},\frac{1}{z^{R}})$ with
$(y_{1},\ldots,y_{n})^{R}=(y_{n},\ldots,y_{1})$.

The hypergeometric function of type $A_{n-1}$ is defined as

$\mathcal{F}(\lambda,z)\coloneqq\mathcal{F}_{k}(\lambda,z)\coloneqq\frac{1}{n!}\sum_{\sigma\in\mathcal{S}_{n}}\mathcal{G}_{k}(\lambda,\sigma
z).$

In particular, for $\lambda\in\Lambda_{+}^{n}$,
$\mathcal{F}(\lambda-\rho,z)=\frac{P_{\lambda}(z)}{P_{\lambda}(\underline{1})}$
is a normalized symmetric Jack polynomial of index $\alpha=\tfrac{1}{k}$. In
the case $k=\tfrac{d}{2}$, where $d$ is the Peirce dimension constant of an
irreducible symmetric cone, the $\mathcal{F}(\lambda,\cdot)$ are precisely the
spherical functions of the cone.

######  Definition 4.13.

For arbitrary $\lambda\in\mathbb{C}^{n}$ we define the $\mathcal{K}$-Bessel
function

$\mathcal{K}_{\lambda}(w,z)=\int_{\mathbb{R}_{+}^{n}}E^{A}(-x,w)E^{A}(-\tfrac{1}{x},z)\mathcal{G}(\lambda-\rho,x)\Delta(x)^{-\mu_{0}-1}\omega^{A}(x)\;\mathrm{d}x.$

The convergence of the integral will be discussed in the next theorem and is
in accordance with the previous case $\lambda=\underline{\nu}$ with
$\nu\in\mathbb{C}$. This more general $\mathcal{K}$-Bessel function is also
related to its counterpart on a symmetric cone $\Omega$. Similar computations
as in Remark 4.9 show that

$\int_{K\times
K}\mathcal{K}_{\mathbf{s}}^{\Omega}(kx,k^{\prime}y)\;\mathrm{d}(k,k^{\prime})=\frac{c_{0}}{n!^{2}}\sum\limits_{\tau,\sigma\in\mathcal{S}_{n}}\mathcal{K}_{\mathbf{s}}(\sigma\,\mathrm{spec}\,x,\tau\,\mathrm{spec}\,y).$

One only has to use that

$\int_{K}\Delta_{\mathbf{s}}(kx)\;\mathrm{d}k=\mathcal{F}(\mathbf{s}-\rho,x)=\frac{1}{n!}\sum\limits_{\sigma\in\mathcal{S}_{n}}\mathcal{G}(\mathbf{s}-\rho,\sigma
x),$

which is, initially true for partitions $\mathbf{s}\in\Lambda_{+}^{n}$, since
in this case, we have the spherical polynomials on the left hand side, which
coincide with the Jack polynomials. By Carlson’s theorem as in [BR22, Lemma
5.2], the stated equality is valid for all $\mathbf{s}\in\mathbb{C}^{n}$.
Similar continuation arguments are done in [BR22].

######  Theorem 4.14.

The map $(\lambda,w,z)\mapsto\mathcal{K}_{\lambda}(w,z)$ is holomorphic on the
domain
$\mathbb{C}^{n}\times\left\\{\mathrm{Re}\,w>0\right\\}\times\left\\{\mathrm{Re}\,z>0\right\\}$
and satisfies:

  1. (i)

$\mathcal{K}_{\lambda}(w,z)=\mathcal{K}_{-\lambda^{R}}(z^{R},w^{R})$ and
$\left|\mathcal{K}_{\lambda}(w,z)\right|\leq\mathcal{K}_{\mathrm{Re}\,\lambda}(\mathrm{Re}\,w,\mathrm{Re}\,z)$.

  2. (ii)

If $\lambda\in\mathbb{R}^{n}$ and $x,y\in\mathbb{R}_{+}^{n}$, then

$\displaystyle 0<\mathcal{K}_{\lambda}(x,y)$
$\displaystyle\leq\begin{cases}\Gamma_{n}(\lambda)\mathcal{G}(\lambda-\rho,\tfrac{1}{x}),&\textrm{if
}\lambda>\mu_{0}\\\
\Gamma_{n}(-\lambda^{R})\mathcal{G}(-\lambda^{R}-\rho,\tfrac{1}{x^{R}}),&\textrm{if
}\lambda<\mu_{0}\end{cases}$
$\displaystyle=\begin{cases}\Gamma_{n}(\lambda)\mathcal{G}(-\lambda^{R}-\rho,x^{R}),&\textrm{if
}\lambda>\mu_{0}\\\
\Gamma_{n}(-\lambda^{R})\mathcal{G}(\lambda-\rho,x),&\textrm{if
}\lambda<\mu_{0}\end{cases}.$

Moreover $\nu\mapsto\mathcal{K}_{\lambda+\underline{\nu}}$ is a convex
function $\mathbb{R}\to\mathbb{R}_{+}$.

  3. (iii)

Recurrence formulas:

$\displaystyle\Delta(T^{A})\mathcal{K}_{\lambda}(w,\cdot)$
$\displaystyle=(-1)^{n}\mathcal{K}_{\lambda-\underline{1}}(w,\cdot),\quad\mathrm{Re}\,w>0,$
$\displaystyle\Delta(T^{A})\mathcal{K}_{\lambda}(\cdot,z)$
$\displaystyle=(-1)^{n}\mathcal{K}_{\lambda+\underline{1}}(\cdot,z),\quad\mathrm{Re}\,z>0.$

###### Proof.

The existence and analyticity of $\mathcal{K}_{\nu}(w,z)$ is checked for
$\mathrm{Re}\,\lambda>\mu_{0}$ and $\mathrm{Re}\,\lambda<-\mu_{0}$ exactly as
in Theorem 4.10, using the stated properties of the Cherednik kernel in
Theorem 4.12. The existence and analyticity in the case
$-\mu_{0}\leq\mathrm{Re}\,\lambda\leq\mu_{0}$ can be deduced by a convexity
property, as in Theorem 4.10, which will be proven below.

  1. (i)

This is a immediate consequence of Theorem 4.12 (vi).

  2. (ii)

Suppose that $\lambda>\mu_{0}$. Similar to the proof of Theorem 4.10 we obtain
by Theorem 4.12 (v),(vi) that

$\displaystyle\hskip 35.0pt0<\mathcal{K}_{\lambda}(x,y)$
$\displaystyle\leq\int_{\mathbb{R}_{+}^{n}}E^{A}(-\xi,x)\mathcal{G}(\lambda-\rho,x)\Delta(x)^{-\mu_{0}-1}\Delta(x)\omega^{A}(x)\;\mathrm{d}x$
$\displaystyle=\Gamma_{n}(\lambda)\mathcal{G}(\lambda-\rho,\tfrac{1}{x})=\Gamma_{n}(\lambda)\mathcal{G}(-\lambda^{R}-\rho,x^{R}).$

By part (i), the case $\lambda<-\mu_{0}$ reduces to the case
$\lambda>\mu_{0}$. Moreover, in view of Remark 4.12 (iii) we have

$\mathcal{G}(\lambda+\underline{\nu}-\rho(k),x)=\mathcal{G}(\lambda-\rho(k),x)\Delta(x)^{\nu},$

so that the convexity in $\nu$ is verified just as in Theorem 4.10. Together
with part (i), this shows that the integral exists for all
$\lambda\in\mathbb{C}^{n}$ and has the stated analyticity property.

  3. (iii)

This is the same argument as in the proof of Theorem 4.10.

∎

##  5\. The Hankel transform and its connection to type $B$ Dunkl theory

The Hankel transform for the root system $A_{n-1}$ was already introduced in
[BF98] and before in [Ma13] in a symmetrized version, both at a rather formal
level. In this section, we will discuss its analytic aspects and its
connection to the type $B$ Dunkl transform. The Bessel function
$\mathcal{J}_{\nu}^{\Omega}$ of a symmetric cone $\Omega$ in a Euclidean
Jordan algebra $V$ defines the Hankel transform on $\Omega$ by

$\mathcal{H}^{\Omega}_{\nu}f(u)\coloneqq\frac{1}{\Gamma_{\Omega}(\nu)}\int_{\Omega}\mathcal{J}_{\nu}^{\Omega}(P(\sqrt{u})v)f(v)\det(v)^{\nu-\tfrac{m}{n}}\;\mathrm{d}v,$

cf. the notions from Section 4. By Remark 4.1 and Definition 4.2, the kernel
$\mathcal{J}_{\nu}^{\Omega}(P(\sqrt{u})v)$ differs from
$\mathscr{J}_{\nu}(\mathrm{spec}\,u,\mathrm{spec}\,v)$ just by taking
$K$-means and a arbitrary choice of the multiplicity $k$ in
$\mathscr{J}_{\nu}$. In view of [FK94, Theorem VI.2.3], the Hankel transform
of a $K$-invariant function $f:V\to\mathbb{C}$, $f(u)=f_{0}(\mathrm{spec}\,u)$
becomes

$\mathcal{H}_{\nu}^{\Omega}f(u)=\mathrm{const}\cdot\int_{\mathbb{R}^{n}}f_{0}(x)\mathscr{J}_{\nu}(x,\mathrm{spec}\,u)\Delta(x)^{\nu-\mu_{0}-1}\omega^{A}(x)\;\mathrm{d}x.$

This will serves as our starting point for the definition of a Dunkl-type
Hankel transform.
Consider a self-adjoint representation $\phi:V\to\mathrm{End}\,E$ of $V$ on
some real Euclidean space $E$, as well as the associated quadratic form
$Q:E\to V$, whose image is contained in the closure $\overline{\Omega}$, see
[FK94, Chapter XVI]. In [He55, Mu82, Ru06] the following case was studied: the
space $V=\mathrm{Sym}_{n}(\mathbb{R})$ of real symmetric matrices, the cone
$\Omega=\mathrm{Pos}_{n}(\mathbb{R})$ of positive definite matrices, the
Euclidean space $E=\mathbb{R}^{n\times m}$ equipped with the Hilbert-Schmidt
inner product, the representation $\phi(x)\xi=x\xi$ and associated quadratic
form $Q(\xi)=\xi\xi^{T}$. It is important to note, that the Fourier transform
of an integrable radial function $f(\xi)=F(Q(\xi))$ is given by a Hankel
transform of $F$, namely

$\widehat{f}(\eta)=\int_{E}e^{-i\braket{\xi,\eta}_{E}}f(\xi)d\xi=\textrm{const}\cdot\mathcal{H}^{\Omega}_{\nu}F(\tfrac{Q(\eta)}{4})$

with $\nu=\tfrac{m}{2n}$. In the Dunkl setting, the Fourier transform is
replaced by a Dunkl transform of type $B$, $Q$ is replaced by $x\mapsto
x^{2}=(x_{1}^{2},\ldots,x_{n}^{2})$ and the parameter $\nu$ will be given by
the associated multiplicity.
We proceed similar as in [Ru06], where the case
$\Omega=\mathrm{Pos}_{n}(\mathbb{R})$ is treated, for arbitrary symmetric
cones see also [FK94, Chapter XVI, Section 2]. Some of our results are
inspired by computations in [Ma13], which where done in the case of symmetric
functions.

######  Definition 5.1.

For $\nu\in\mathbb{C}$ with $\mathrm{Re}\,\nu>\mu_{0}$, the Hankel transform
is defined as

$\mathcal{H}_{\nu}f(w)\coloneqq\frac{1}{\Gamma_{n}(\nu)}\int_{\mathbb{R}_{+}^{n}}f(x)\mathscr{E}_{\nu}(x,w)\Delta(x)^{\nu-\mu_{0}-1}\omega^{A}(x)\;\mathrm{d}x,\quad
w\in\mathbb{C}^{n},$

whenever the integral exists for measurable
$f:\mathbb{R}_{+}^{n}\to\mathbb{C}$.

######  Lemma 5.2.

Consider $\nu\in\mathbb{C}$ with $\mathrm{Re}\,\nu>\mu_{0}$ and put
$e_{z}(x)\coloneqq E^{A}(x,-z)$ for $z\in\mathbb{C}^{n}$ with
$\mathrm{Re}\,z>0$. Then:

  1. (i)

$e_{z}\in L_{\nu}^{2}(\mathbb{R}^{n}_{+})\coloneqq
L^{2}(\mathbb{R}_{+}^{n},\Delta(x)^{\nu-\mu_{0}-1}\omega^{A}(x)\mathrm{d}x)$
for $\nu\in\mathbb{R}$ with $\nu>\mu_{0}$.

  2. (ii)

For $z\in\mathbb{C}^{n}$ with $\mathrm{Re}\,z>0$,

$\mathcal{H}_{\nu}(E^{A}(\cdot,-z))=\Delta(z)^{-\nu}E^{A}(\cdot,-\tfrac{1}{z}).$

In particular, $\mathcal{H}_{\nu}$ is involutive on
$\mathcal{U}\coloneqq\mathrm{span}_{\mathbb{C}}\left\\{E^{A}(\cdot,-z)\mid\mathrm{Re}\,z>0\right\\}$.

  3. (iii)

For $\nu\in\mathbb{R}$, $\nu>\mu_{0}$ we may equip
$L_{\nu}^{2}(\mathbb{R}^{n}_{+})$ with the canonical inner product

$\braket{f,g}_{L_{\nu}^{2}(\mathbb{R}^{n}_{+})}\coloneqq\int_{\mathbb{R}_{+}^{n}}f(x)\overline{g(x)}\Delta(x)^{\nu-\mu_{0}-1}\omega^{A}(x)\;\mathrm{d}x.$

Then $\mathcal{U}\subseteq L_{\nu}^{2}(\mathbb{R}^{n}_{+})$ is dense. More
precisely, the span of each set of the form $\left\\{e_{z}\mid
z=\underline{s}+iy,\,y\in\mathbb{R}^{n}\right\\}$ with fixed $s>0$ is already
dense.

###### Proof.

Part (i) is a consequence of estimate (3.1) and part (ii) is a reformulation
of Lemma 4.4. Thus, it remains to prove part (iii). Fix some $s>0$ and assume
that $f\in L_{\nu}^{2}(\mathbb{R}^{n}_{+})$ satisfies
$\braket{e_{z},f}_{L_{\nu}^{2}(\mathbb{R}^{n}_{+})}=0$ for all
$z=\underline{s}+iy,\,y\in\mathbb{R}^{n}$. Then

$0=\braket{e_{\underline{s}+iy},f}_{L_{\nu}^{2}(\mathbb{R}^{n}_{+})}=\mathcal{L}(\overline{f}\Delta^{\nu-\mu_{0}-1})(\underline{s}+iy),$

so that we conclude that $f\Delta^{\nu-\mu_{0}-1}=0$ a.e. by injectivity of
the Dunkl-Laplace transform from Theorem 3.1 (iii). Hence $f=0$. ∎

######  Theorem 5.3.

For $\nu\in\mathbb{R}$ with $\nu>\mu_{0}$, the Hankel transform
$\mathcal{H}_{\nu}$ extends uniquely to an involutive isometric isomorphism of
$L_{\nu}^{2}(\mathbb{R}^{n}_{+})$.

###### Proof.

By Lemma 5.2 it suffices to show that $\mathcal{H}_{\nu}$ is unitary on the
space generated by the functions $e_{z}(x)=E^{A}(x,-z)$ with
$\mathrm{Re}\,z>0$. The Laplace transform of hypergeometric series shows that
${}_{1}K_{0}$ has an analytic extension (cf. [BR22, Corollary 6.9]) satisfying

$\Gamma_{n}(\nu)\Delta(z)^{-\nu}{}_{1}K_{0}(\nu;w,-\tfrac{1}{z})=\int_{\mathbb{R}_{+}^{n}}E^{A}(-x,z)E^{A}(-x,w)\Delta(x)^{\nu-\mu_{0}-1}\omega^{A}(x)\;\mathrm{d}x$

for arbitrary $w\in\mathbb{C}^{n}$ and $z\in\mathbb{C}^{n}$ with
$\mathrm{Re}\,w,\,\mathrm{Re}\,z>0$. As $\overline{e_{w}}=e_{\overline{w}}$ on
$\mathbb{R}^{n}$

$\braket{e_{z},e_{w}}_{L_{\nu}^{2}(\mathbb{R}^{n}_{+})}=\Gamma_{n}(\nu)\Delta(z)^{-\nu}{}_{1}K_{0}(\nu;\overline{w},-\tfrac{1}{z}).$

Thus, by part (i) of Lemma 5.2 we conclude that

$\displaystyle\braket{\mathcal{H}_{\nu}e_{z},\mathcal{H}_{\nu}e_{w}}_{L_{\nu}^{2}(\mathbb{R}^{n}_{+})}$
$\displaystyle=\Delta(z)^{-\nu}\Delta(\overline{w})^{-\nu}\braket{e_{1/z},e_{1/w}}_{L_{\nu}^{2}(\mathbb{R}^{n}_{+})}$
$\displaystyle=\Gamma_{n}(\nu)\Delta(\overline{w})^{-\nu}{}_{1}K_{0}(\nu;\tfrac{1}{\overline{w}},-z)$
$\displaystyle=\overline{\Gamma_{n}(\nu)\Delta(w)^{-\nu}{}_{1}K_{0}(\nu;\tfrac{1}{w},-\overline{z})}$
$\displaystyle=\overline{\braket{e_{w},e_{z}}_{L_{\nu}^{2}(\mathbb{R}^{n}_{+})}}=\braket{e_{z},e_{w}}_{L_{\nu}^{2}(\mathbb{R}^{n}_{+})}.$

∎

The Hankel transform $\mathcal{H}_{\nu}f$ of a function $f\in
L_{\nu}^{2}(\mathbb{R}^{n}_{+})$ can also be uniquely characterized in terms
of the Dunkl-Laplace transform with the following lemma.

######  Lemma 5.4.

Consider $f\in L_{\nu}^{2}(\mathbb{R}^{n}_{+})$, so in particular
$\mathcal{L}_{k}(f\Delta^{\nu-\mu_{0}-1})(z)$ exists for all
$z\in\mathbb{C}^{n}$ with $\mathrm{Re}\,z>0$. Moreover, assume that
$g:\mathbb{R}_{+}^{n}\to\mathbb{C}$ is measurable and $s>0$, such that
$\mathcal{L}(g\Delta^{\nu-\mu_{0}-1})(\underline{s})$ exists. Then:

  1. (i)

For all $z\in\mathbb{C}^{n}$ with $\mathrm{Re}\,z>0$

$\mathcal{L}((\mathcal{H}_{\nu}f)\Delta^{\nu-\mu_{0}-1})(z)=\Delta(z)^{-\nu}\mathcal{L}(f\Delta^{\nu-\mu_{0}-1})(\tfrac{1}{z}).$

  2. (ii)

If
$\mathcal{L}(g\Delta^{\nu-\mu_{0}-1})(z)=\Delta(z)^{-\nu}\mathcal{L}(f\Delta^{\nu-\mu_{0}-1})(\tfrac{1}{z})$
for all $z\in\mathbb{C}$ with $\mathrm{Re}\,z>s$, then $g\in
L_{\nu}^{2}(\mathbb{R}^{n}_{+})$, $\mathcal{L}(g\Delta^{\nu-\mu_{0}-1})(z)$
exists for all $\mathrm{Re}\,z>0$ and $g=\mathcal{H}_{\nu}f$.

###### Proof.

For $g=\mathcal{H}_{\nu}f$ we have by Lemma 5.2 and Theorem 5.3

$\displaystyle\mathcal{L}(g\Delta^{\nu-\mu_{0}-1})(z)$
$\displaystyle=\braket{e_{z},\overline{g}}_{L_{\nu}^{2}(\mathbb{R}^{n}_{+})}=\braket{\mathcal{H}_{\nu}e_{z},\overline{f}}_{L_{\nu}^{2}(\mathbb{R}^{n}_{+})}$
$\displaystyle=\Delta(z)^{-\nu}\braket{e_{1/z},\overline{f}}_{L_{\nu}^{2}(\mathbb{R}^{n}_{+})}=\Delta(z)^{-\nu}\mathcal{L}(f\Delta^{\nu-\mu_{0}-1})(\tfrac{1}{z}).$

This proves (i). The second property follows immediately from injectivity of
the Dunkl-Laplace transform and part (i), since from the assumption we
conclude as before
$\mathcal{L}(g\Delta^{\nu-\mu_{0}-1})(z)=\mathcal{L}((\mathcal{H}_{\nu}f)\Delta^{\nu-\mu_{0}-1})(z)$
for all $z\in\mathbb{C}^{n}$ with $\mathrm{Re}\,z>s$. ∎

To facilitate readability, we will write
$L^{p}(\Omega,h(x)\mathrm{d}x)\coloneqq
L^{p}(\Omega,\left|h(x)\right|\mathrm{d}x)$ for measurable
$h:\Omega\to\mathbb{C}$, where $\Omega\subseteq\mathbb{R}^{n}$ is a Borel set.
Recall the correspondence between the pairs of type $A$ multiplicities
$\kappa_{A}=k$ together with a parameter $\nu\in\mathbb{C}$, and type $B$
multiplicities $\kappa_{B}$, namely

$(\kappa_{A},\nu)=(k,\nu)\longleftrightarrow\kappa_{B}=(k,k^{\prime})\textrm{
with }k^{\prime}=\nu-\mu_{0}-\tfrac{1}{2},\,\mu_{0}=k(n-1).$

Further, assume that $\mathrm{Re}\,\nu>\mu_{0}+\tfrac{1}{2}$, i.e.
$\mathrm{Re}\,\kappa_{B}\geq 0$, so that $\mathcal{F}^{B}$ is an automorphism
of $\mathscr{S}(\mathbb{R}^{n})$, injective on
$L^{1}(\mathbb{R}^{n},\omega^{B}(x)\mathrm{d}x)$, and extends to an unitary
map of $L^{2}(\mathbb{R}^{n},\omega^{B}(x)\mathrm{d}x)$ in the case
$\kappa_{B}\geq 0$.
The following integral decomposition has to be seen as a Dunkl analogue of the
formula [FK94, Proposition XVI.2.1], where we replaced the integration over
the Stiefel manifold by summation over the $\mathbb{Z}_{2}^{n}$-action. This
is quite simple, but the stated formula has the same important role as on
symmetric cones.

######  Proposition 5.5 (Integral decomposition).

Consider $f\in L^{1}(\mathbb{R}^{n},\omega^{B}(x)\mathrm{d}x)$. Then

$\int_{\mathbb{R}^{n}}f(x)\omega^{B}(x)\;\mathrm{d}x=\frac{1}{2^{n}}\sum\limits_{\tau\in\mathbb{Z}_{2}^{n}}\int_{\mathbb{R}_{+}^{n}}f(\tau
x^{\tfrac{1}{2}})\Delta(x)^{\nu-\mu_{0}-1}\omega^{A}(x)\;\mathrm{d}x,$

where $x^{\tfrac{1}{2}}$ has to be understood componentwise. In particular, if
$f$ is $\mathbb{Z}_{2}^{n}$-invariant

$\int_{\mathbb{R}^{n}}f(x)\omega^{B}(x)\;\mathrm{d}x=\int_{\mathbb{R}_{+}^{n}}f(x^{\tfrac{1}{2}})\Delta(x)^{\nu-\mu_{0}-1}\omega^{A}(x)\;\mathrm{d}x,$
(5.1)

###### Proof.

Since $\omega^{B}$ is $\mathbb{Z}_{2}^{n}$-invariant, we have with (2.1) that

$\int_{\mathbb{R}^{n}}f(x)\omega^{B}(x)\;\mathrm{d}x=\sum\limits_{\tau\in\mathbb{Z}_{2}^{n}}\int_{\mathbb{R}_{+}^{n}}f(\tau
x)\Delta(x^{2})^{k^{\prime}}\omega^{A}(x^{2})\;\mathrm{d}x.$

Hence, the change of variables $x\leftrightarrow x^{\tfrac{1}{2}}$ and
$\nu-\mu_{0}-1=k^{\prime}-\tfrac{1}{2}$ gives the stated formula. ∎

The Hankel transform and the type $B$ Dunkl transform are closely related. The
connection is given in the subsequent theorem.

######  Theorem 5.6.

Recall the Dunkl transform $\mathcal{F}^{B}$. Then
$c_{B}=2^{n\nu}\Gamma_{n}(\nu)$ and ff $f\in
L^{1}(\mathbb{R}^{n},\omega^{B}(x)\mathrm{d}x)$ is
$\mathbb{Z}_{2}^{n}$-invariant, the measurable function
$f_{0}:\mathbb{R}_{+}^{n}\to\mathbb{C}$ defined by $f(x)=f_{0}(x^{2})$
satisfies

$\mathcal{F}^{B}f(\xi)=2^{-n\nu}\mathcal{H}_{\nu}f_{0}(\tfrac{\xi^{2}}{4}),\quad\xi\in\mathbb{R}^{n}.$
(5.2)

###### Proof.

The computation of $c_{B}$ can be deduced from Theorem 3.2 and Proposition 5.5

$\displaystyle c_{B}$
$\displaystyle=\int_{\mathbb{R}^{n}}e^{-\tfrac{\left|x\right|^{2}}{2}}\omega^{B}(x)\;\mathrm{d}x=\int_{\mathbb{R}_{+}^{n}}e^{-\tfrac{1}{2}\braket{x,\underline{1}}}\Delta(x)^{\nu-\mu_{0}-1}\omega^{A}(x)\;\mathrm{d}x$
$\displaystyle=2^{n\nu}\int_{\mathbb{R}_{+}^{n}}E^{A}(-x,\underline{1})\Delta(x)^{\nu-\mu_{0}-1}\omega^{A}(x)\;\mathrm{d}x=2^{n\nu}\Gamma_{n}(\nu).$

Further, the integral formula (5.1) of Proposition 5.5 and the relation
between the Bessel kernel $\mathscr{E}_{\nu}$ with squared argument and the
type $B$ Dunkl kernel from Proposition 4.3, we deduce that

$\mathcal{F}^{B}f(\xi)=\frac{1}{c_{B}}\int_{\mathbb{R}_{+}^{n}}f_{0}(x)\mathscr{E}_{\nu}(x,\tfrac{\xi^{2}}{4})\Delta(x)^{\nu-\mu_{0}-1}\omega^{A}(x)\;\mathrm{d}x,$

so (5.2) is proven. ∎

The following lemma is a Dunkl analogue of [Ru06, Theorem 3.1]. On the space
$\mathbb{R}^{n\times m}$ of rectangular matrices the author describes the
action of the so-called Cayley-Laplacian $\det(\partial^{T}\partial)$ on
radial functions, i.e. functions of the form $f(x)=f_{0}(xx^{T})$. We replace
the Cayley-Laplacian the type $B$ Dunkl operator $\Delta(T^{B})^{2}$ and
describe its action on $\mathbb{Z}_{2}^{n}$-invariant functions $f$.

######  Theorem 5.7.

Let $f\in C^{\infty}(\mathbb{R}^{n})$ be a $\mathbb{Z}_{2}^{n}$-invariant
function.

  1. (i)

There exists a smooth function $f_{0}\in C^{\infty}(\mathbb{R}^{n})$ with
$f(x)=f_{0}(x^{2})$.

  2. (ii)

The Dunkl operator $\Delta(T^{B})^{2}$ acts according to

$\Delta(T^{B})^{2}f(x)=\mathscr{L}_{\nu}f_{0}(x^{2}),$ (5.3)

where the operator on the right hand side is defined as

$\mathscr{L}_{\nu}\coloneqq
4^{n}\Delta(x)^{1+\mu_{0}-\nu}\Delta(T^{A})\Delta(x)^{\nu-\mu_{0}}\Delta(T^{A}).$

Here the powers of $\Delta(x)$ are understood as multiplication operator.

###### Proof.

  1. (i)

This was done by Whitney [Wh43] for univariate functions, and the multivariate
case is easily reduced to this case.

  2. (ii)

By continuity it suffices to check (5.3) on
$\mathbb{R}^{n}\setminus\left\\{\Delta(x)=0\right\\}$. Since

$\mathrm{supp}(T_{\xi}^{R}f)\subseteq W_{\\!R}.\mathrm{supp}\,f\quad\textrm{
for }R\in\left\\{A,B\right\\}$

we can assume without loss of generality that $f\in
C_{c}^{\infty}(\mathbb{R}^{n})$ and that
$\mathrm{supp}\,f\cap\left\\{\Delta(x)=0\right\\}=\emptyset$, i.e. that
$f_{0}\in C_{c}^{\infty}(\mathbb{R}_{+}^{n})$. Since the Dunkl transform
$\mathcal{F}^{B}$ is injective, we can prove identity (5.3) under the action
of $\mathcal{F}^{B}$. Identities (2.3) and Theorem 5.6 show that

$\displaystyle\quad\quad\quad\quad\mathcal{F}^{B}(\Delta(T^{B})^{2}$
$\displaystyle f)(\xi)=\Delta(i\xi)^{2}\mathcal{F}^{B}f(\xi)$
$\displaystyle=\frac{(-1)^{n}2^{-n\nu}}{\Gamma_{n}(\nu)}\Delta(\xi)^{2}\int_{\mathbb{R}_{+}^{n}}\mathscr{E}_{\nu}(\tfrac{\xi^{2}}{4},x)f_{0}(x)\Delta(x)^{\nu-\mu_{0}-1}\omega^{A}(x)\;\mathrm{d}x.$

The recurrence formulas of Corollary 4.6 lead to

$\displaystyle\quad\quad\quad\quad(-1)^{n}\Delta(\xi)^{2}\mathscr{E}_{\nu}(\tfrac{\xi^{2}}{4},x)\Delta(x)^{\nu-\mu_{0}-1}$
$\displaystyle=(-4)^{n}\Delta(\tfrac{\xi^{2}}{4})\mathscr{E}_{\nu}(\tfrac{\xi^{2}}{4},x)\Delta(x)^{\nu-\mu_{0}-1}$
$\displaystyle=4^{n}\Delta(T^{A}_{x})\\!\left(\\!\Delta(x)^{\nu-\mu_{0}}\Delta(T^{A}_{x})\mathscr{E}_{\nu}(\tfrac{\xi^{2}}{4},x)\\!\right).$

The Dunkl operators $T_{\xi}^{A}$ are skew symmetric on
$C_{c}^{\infty}(\mathbb{R}^{n})\subseteq
L^{2}(\mathbb{R}^{n},\omega^{A}(x)\mathrm{d}x)$ (cf. [dJ93]). Hence we obtain
for $g(x)=\mathscr{L}_{\nu}f_{0}(x^{2})$ that

$\displaystyle\quad\quad\quad\mathcal{F}^{B}$
$\displaystyle(\Delta(T^{B})^{2}f)(\xi)$
$\displaystyle=\frac{4^{n}2^{-n\nu}}{\Gamma_{n}(\nu)}\int_{\mathbb{R}_{+}^{n}}\big{(}\Delta(T^{A}_{x})\big{(}\Delta(x)^{\nu-\mu_{0}-1}\Delta(T^{A}_{x})\mathscr{E}_{\nu}(\tfrac{\xi^{2}}{4},x)\big{)}\big{)}f_{0}(x)\omega^{A}(x)\;\mathrm{d}x$
$\displaystyle=\frac{4^{n}2^{-n\nu}}{\Gamma_{n}(\nu)}\int_{\mathbb{R}_{+}^{n}}\mathscr{E}_{\nu}(\tfrac{\xi^{2}}{4},x)(\Delta(T^{A}_{x})(\Delta(x)^{\nu-\mu_{0}-1}\Delta(T^{A}_{x})f_{0}))(x)\omega^{A}(x)\;\mathrm{d}x$
$\displaystyle=\frac{2^{-n\nu}}{\Gamma_{n}(\nu)}\int_{\mathbb{R}_{+}^{n}}\mathscr{E}_{\nu}(\tfrac{\xi^{2}}{4},x)(\mathscr{L}_{\nu}f_{0})(x)\Delta(x)^{\nu-\mu_{0}-1}\omega^{A}(x)\;\mathrm{d}x$
$\displaystyle=2^{-n\nu}\mathcal{H}_{\nu}(\mathscr{L}_{\nu}f_{0})(\tfrac{\xi^{2}}{4})=\mathcal{F}^{B}g(\xi),$

where in the last line again Theorem 5.6 was used.

∎

######  Example 5.8 (Type $B$ Bernstein identity).

Due to [Rö20, Lemma 5.4] or equally by Theorem 3.2 for $\eta=\underline{1}$
one has the following type $A$ Bernstein identity:

$\Delta(T^{A})\Delta(x)^{\mu}=b(\mu)\Delta(x)^{\mu-1},\quad\mu\in\mathbb{C},$

where $b(\mu)=\prod_{j=1}^{n}(\mu+k(j-1))$. If we choose
$f(x)=\Delta(x^{2})^{\mu}$ in Theorem 5.7, we obtain from this type $A$
Bernstein identity that

$\displaystyle\Delta(T^{B})^{2}\Delta(x^{2})^{\mu}$
$\displaystyle=4^{n}[\Delta(\xi)^{1+\mu_{0}-\nu}\Delta(T^{A})\Delta(\xi)^{\nu-\mu_{0}}\Delta(T^{A})\Delta(\xi)^{\mu}]_{\xi=x^{2}}$
$\displaystyle=4^{n}b(\mu)[\Delta(\xi)^{1+\mu_{0}-\nu}\Delta(T^{A})\Delta(\xi)^{\nu+\mu-\mu_{0}-1}]_{\xi=x^{2}}$
$\displaystyle=4^{n}b(\mu)b(\nu+\mu-\mu_{0}-1)\Delta(x)^{\mu-1}$
$\displaystyle\eqqcolon\mathcal{B}(\mu)\Delta(x^{2})^{\mu-1}.$

Thus, we have

$\Delta(T^{B})^{2}\Delta(x^{2})^{\mu}=\mathcal{B}(\mu)\Delta(x^{2})^{\mu-1},$
(5.4)

where $\mathcal{B}$ is the polynomial
$\mathcal{B}(\mu)=4^{n}\prod_{j=1}^{n}(\mu+k(j-1))(\mu-\tfrac{1}{2}+k^{\prime}+k(j-1))$.
This Bernstein identity was independently proven in [Li16, Proposition 3.1.2]
by direct computation.

##  6\. Zeta integrals and zeta distributions in the type B Dunkl setting

Recall the symmetric cone setting from the previous section. For
$\alpha\in\mathbb{C}$ with
$\mathrm{Re}\,\alpha>\tfrac{d}{2}(n-1)-\tfrac{m}{2n}$, the zeta integral of
index $\alpha$ is defined a Schwartz function $f\in\mathscr{S}(E)$,
$E=\mathbb{R}^{n\times m}$ by

$Z(f;\alpha)=\int_{E}f(\xi)\det(Q(\xi))^{\alpha}\;\mathrm{d}\xi,$

where $\det$ is the Jordan determinant, see [FK94, Chapter XVI]. The zeta
integral can be meromorphically extended in the parameter $\alpha$ to the
complex plane $\mathbb{C}$ such that it satisfies the following characteristic
functional equation

$\frac{Z(\widehat{f};\alpha-\tfrac{m}{2n})}{\Gamma_{\Omega}(\alpha)}=\pi^{\tfrac{n}{2}}4^{n\alpha}\frac{Z(f;-\alpha)}{\Gamma_{\Omega}(\tfrac{m}{2n}-\alpha)},$

where $\Gamma_{\Omega}$ is again the Gindikin gamma function of $\Omega$. We
shall derive a similar functional equation, where the Fourier transform on $E$
is replaced by the type $B$ Dunkl transform on $\mathbb{R}^{n}$ and the
dimension constant $\frac{m}{2n}$ is replaced by a continuous parameter
depending on the type $B$ multiplicity.

To become more precise, consider $\kappa_{A}=k\geq 0$, $\nu\in\mathbb{C}$ with
$\mathrm{Re}\,\nu>\mu_{0}+\tfrac{1}{2}$ and $\kappa_{B}=(k,k^{\prime})$ with
$k^{\prime}=\nu-\mu_{0}-\tfrac{1}{2}$.

######  Definition 6.1.

We define the zeta integral of index $\alpha$, $\mathrm{Re}\,\alpha>\mu_{0}$
of a Schwartz function $f\in\mathscr{S}(\mathbb{R}^{n})$ by

$\mathcal{Z}(f;\alpha)\coloneqq\int_{\mathbb{R}^{n}}f(x)\Delta(x^{2})^{\alpha-\nu}\omega^{B}(x)\;\mathrm{d}x=\int_{\mathbb{R}^{n}}f(x)\Delta(x^{2})^{\alpha-\mu_{0}-1}\omega^{A}(x^{2})\;\mathrm{d}x.$

Obviously, the integral converges absolutely and depends holomorphically on
$\alpha$. Moreover, $\mathcal{Z}(\cdot,\alpha)$ is $W_{\\!B}$-invariant, i.e.
$\mathcal{Z}(\tau f,\alpha)=\mathcal{Z}(f,\alpha)$ for all $\tau\in W_{\\!B}$.

The following example will show which kind of functional equation we can
expect for this zeta integral in the Dunkl setting and how it can be extended
meromorphically in the parameter $\alpha\in\mathbb{C}$.

######  Example 6.2.

Consider the Gauss function

$g(x)\coloneqq e^{-\left|x\right|^{2}}=E^{A}(-\underline{1},x^{2}).$

If $\alpha\in\mathbb{C}$ with $\mathrm{Re}\,\alpha>\mu_{0}$, then Theorem 3.2
and Proposition 5.5 show that

$\mathcal{Z}(g;\alpha)=\int_{\mathbb{R}_{+}^{n}}E^{A}(-\underline{1},x)\Delta(x)^{\alpha-\mu_{0}-1}\omega^{A}(x)\;\mathrm{d}x=\Gamma_{n}(\alpha).$

Therefore, the zeta integral of $g$ can be meromorphically extended in the
parameter $\alpha\in\mathbb{C}$. Moreover, the type $B$ Dunkl transform of $g$
can be computed by Lemma 4.4 and Theorem 5.6 as

$\mathcal{F}^{B}g(x)=2^{-n\nu}g(\tfrac{x}{2})=2^{-n\nu}E^{A}(-\underline{1},\tfrac{x^{2}}{4}).$

Thus, for $\mathrm{Re}\,\alpha<\mathrm{Re}\,\nu-\mu_{0}$, we get

$\displaystyle\mathcal{Z}(\mathcal{F}^{B}g;\nu-\alpha)$
$\displaystyle=2^{-n\nu}\int_{\mathbb{R}_{+}^{n}}E^{A}(-\underline{1},\tfrac{x}{4})\Delta(x)^{\nu-\alpha-\mu_{0}-1}\omega^{A}(x)\;\mathrm{d}x$
$\displaystyle=2^{n(\nu-2\alpha)}\int_{\mathbb{R}_{+}^{n}}e^{-\braket{x,\underline{1}}}\Delta(x)^{\nu-\alpha-\mu_{0}-1}\omega^{A}(x)\;\mathrm{d}x$
$\displaystyle=2^{n(\nu-2\alpha)}\Gamma_{n}(\nu-\alpha),$

which can be extended meromorphically in $\alpha\in\mathbb{C}$. We therefore
obtain

$\frac{\mathcal{Z}(g,\alpha)}{\Gamma_{n}(\alpha)}=2^{n(2\alpha-\nu)}\frac{\mathcal{Z}(\mathcal{F}^{B}g,\nu-\alpha)}{\Gamma_{n}(\nu-\alpha)}=1.$
(6.1)

We see that both sides of this equation are entire functions in $\alpha$ and
we have found a functional equation between the zeta integrals of $g$ and
$\mathcal{F}^{B}g$. We shall see that both the functional equation and the
analytic extension will be valid for arbitrary Schwartz functions.

######  Proposition 6.3.

Consider a $\mathbb{Z}_{2}^{n}$-invariant Schwartz function
$f\in\mathscr{S}(\mathbb{R}^{n})$. Then there exists a Schwartz function
$f_{0}\in\mathscr{S}(\mathbb{R}^{n})$ such that $f(x)=f_{0}(x^{2})$.

###### Proof.

Since $f|_{\mathbb{R}_{+}^{n}}$ extends to a Schwartz function, we can use
[Ste19, Theorem 1.1, $\eta=0$] to obtain for arbitrary
$\alpha,\beta\in\mathbb{N}_{0}^{n}$ that

$\sup\limits_{x\in\mathbb{R}^{n}_{+}}\left|x^{\alpha}\left(\tfrac{1}{x}\tfrac{\partial}{\partial
x}\right)^{\beta}f(x)\right|<\infty,$

where $\left(\tfrac{1}{x}\tfrac{\partial}{\partial
x}\right)^{\beta}=\left(\tfrac{1}{x_{1}}\tfrac{\partial}{\partial
x_{1}}\right)^{\beta_{1}}\cdots\left(\tfrac{1}{x_{n}}\tfrac{\partial}{\partial
x_{n}}\right)^{\beta_{n}}.$ As stated in Theorem 5.7 (i), we can find a smooth
function $g\in C^{\infty}(\mathbb{R}^{n})$ with $f(x)=g(x^{2})$. Consider the
change of variables $t_{i}=x_{i}^{2}$, i.e.
$\tfrac{1}{x_{i}}\tfrac{\partial}{\partial x_{i}}=2\tfrac{\partial}{\partial
t_{i}}$. The function $g$ satisfies

$\displaystyle\sup\limits_{t\in\mathbb{R}_{+}^{n}}\left|t^{\alpha}\left(\tfrac{\partial}{\partial
t}\right)^{\beta}\\!\\!g(t)\right|$
$\displaystyle=\frac{1}{2^{\left|\beta\right|}}\sup\limits_{x\in\mathbb{R}_{+}^{n}}\left|x^{2\alpha}\left[\left(\tfrac{1}{x}\tfrac{\partial}{\partial
x}\right)^{\beta}\\!\\!f\right](x)\right|<\infty.$

Application of [JP16, Theorem 4.2] now shows that the function
$g|_{\mathbb{R}_{+}^{n}}$ is the restriction of a Schwartz function
$f_{0}\in\mathscr{S}(\mathbb{R}^{n})$. In particular, $f(x)=f_{0}(x^{2})$. ∎

Our zeta integrals have a close connection to the Riesz distributions in the
Dunkl setting of type $A$, which are defined for
$\alpha\in\mathbb{C},\,\mathrm{Re}\,\alpha>\mu_{0}$ by

$\braket{R_{\alpha},f}\coloneqq\frac{1}{\Gamma_{n}(\alpha)}\int_{\mathbb{R}_{+}^{n}}f(x)\Delta(x)^{\alpha-\mu_{0}-1}\omega^{A}(x)\;\mathrm{d}x,\quad
f\in\mathscr{S}(\mathbb{R}^{n}).$

These Riesz distributions were studied in [Rö20]. They extend to a (weakly)
holomorphic map
$\mathbb{C}\to\mathscr{S}^{\prime}(\mathbb{R}^{n}),\,\alpha\mapsto
R_{\alpha}$.

######  Lemma 6.4.

Let $f\in\mathscr{S}(\mathbb{R}^{n})$. Then the function

$\alpha\mapsto\braket{\zeta_{\alpha},f}\coloneqq\frac{\mathcal{Z}(f;\alpha)}{\Gamma_{n}(\alpha)}$

extends to an entire function on $\mathbb{C}$. If
$f\in\mathscr{S}(\mathbb{R}^{n})$ is $\mathbb{Z}_{2}^{n}$-invariant with
$f(x)=f_{0}(x^{2})$ and $f_{0}\in\mathscr{S}(\mathbb{R}^{n})$, then this
extension is given in terms of the Riesz distributions $R_{\alpha}$ via

$\braket{\zeta_{\alpha},f}=\braket{R_{\alpha},f_{0}}.$

We call $\zeta_{\alpha}$ a Dunkl-type zeta distribution of index
$\alpha\in\mathbb{C}$. That $\zeta_{\alpha}$ is in fact a tempered
distribution will be proven in the subsequent theorem.

###### Proof.

By $\mathbb{Z}_{2}^{n}$-invariance of the zeta integrals, we may asssume that
$f$ is a $\mathbb{Z}_{2}^{n}$-invariant function, otherwise we can consider
$\mathbb{Z}_{2}^{n}$-mean. By Theorem 6.3 we can find
$f_{0}\in\mathscr{S}(\mathbb{R}^{n})$ such that $f(x)=f_{0}(x^{2})$. For
$\alpha\in\mathbb{C}$ with $\mathrm{Re}\,\alpha>\mu_{0}$, Proposition 5.5
shows that

$\braket{\zeta_{\alpha},f}=\braket{R_{\alpha},f_{0}}.$

Hence, that statement follows from the analytic extension property for the
Dunkl-type Riesz distributions $R_{\alpha},\alpha\in\mathbb{C}$. ∎

Recall the Dunkl operators $T_{\xi}^{R}$ and the Dunkl transform
$\mathcal{F}^{R}$ act continuously on the space of Schwartz functions
$\mathscr{S}(\mathbb{R}^{n})$, equipped with the usual locally convex
topology. Hence, by duality we consider the usual actions on the space of
tempered distributions $\mathscr{S}^{\prime}(\mathbb{R}^{n})$ via

$\displaystyle\braket{T_{\xi}^{R}u,\cdot}$
$\displaystyle=\braket{u,-T_{\xi}^{R}\cdot},$
$\displaystyle\braket{\mathcal{F}^{R}u,\cdot}$
$\displaystyle=\braket{u,\mathcal{F}^{R}\cdot},$

for $u\in\mathscr{S}^{\prime}(\mathbb{R}^{n})$.

######  Theorem 6.5.

The functionals $\zeta_{\alpha}$ with $\alpha\in\mathbb{C}$ have the following
properties:

  1. (i)

$\displaystyle\zeta_{\alpha}\in\mathscr{S}^{\prime}(\mathbb{R}^{n})$ and
$\zeta_{\alpha}$ is $W_{\\!B}$-invariant.

  2. (ii)

If $\mathrm{Re}\,\alpha>\mu_{0}$, then $\zeta_{\alpha}$ is a positive measure
with support $\mathbb{R}^{n}$.

  3. (iii)

$\displaystyle\Delta(T^{B})^{2}\zeta_{\alpha}=4^{n}b(\alpha-\nu)\zeta_{\alpha-1}$
with $b(\mu)=\prod\limits_{j=1}^{n}(z+k(j-1))$.

  4. (iv)

$\displaystyle\Delta(x^{2})\zeta_{\alpha}=b(\alpha-\mu_{0})\zeta_{\alpha+1}$.

###### Proof.

Part (ii) is obvious. If $\mathrm{Re}\,\alpha>\mu_{0}+1$, then the skew
symmetry of Dunkl operators and the Bernstein identity (5.4) of Example 5.8
show that

$\mathcal{Z}(\Delta(T^{B})^{2}f;\alpha)=\mathcal{B}(\alpha-\nu)\mathcal{Z}(f;\alpha-1).$

In particular,

$\braket{\zeta_{\alpha},\Delta(T^{B})^{2}f}=\frac{\Gamma_{n}(\alpha-1)}{\Gamma_{n}(\alpha)}\mathcal{B}(\alpha-\nu)\braket{\zeta_{\alpha-1},f}.$

By definition, $\mathcal{B}(\alpha-\nu)=4^{n}b(\alpha-\nu)b(\alpha-\mu_{0}-1)$
and moreover

$b(\alpha-\mu_{0}-1)=\prod\limits_{j=1}^{n}(\alpha-1-k(j-1))=\frac{\Gamma_{n}(\alpha)}{\Gamma_{n}(\alpha-1)}.$

Hence,

$\braket{\zeta_{\alpha},\Delta(T^{B})^{2}f}=4^{n}b(\alpha-\nu)\braket{\zeta_{\alpha-1},f},$

so that analytic extension according to Lemma 6.4 shows that
$\zeta_{\alpha-1}$ is a tempered distribution provided $\zeta_{\alpha}$ is so
and $b(\alpha-\nu)\neq 0$. The set

$M=\left\\{\alpha\in\mathbb{C}\mid b(\alpha+k-\nu)\neq 0\textrm{ for all
}k\in\mathbb{Z}\right\\}$

is dense in $\mathbb{C}$ and for all $\alpha\in M$ we have
$\zeta_{\alpha}\in\mathscr{S}^{\prime}(\mathbb{R}^{n})$. But, the map
$\alpha\mapsto\braket{\zeta_{\alpha},f}$ is holomorphic for all
$f\in\mathscr{S}(\mathbb{R}^{n})$ and as a dual of a Fréchet space, it is
closed under pointwise limits. Hence all $\zeta_{\alpha}$ are tempered.
Thus, we have proven parts (i) and (iii). Part (iv) is immediate for
$\mathrm{Re}\,\alpha>\mu_{0}$, since

$b(\alpha-\mu_{0})=\frac{\Gamma_{n}(\alpha+1)}{\Gamma_{n}(\alpha)},$

and follows by analytic extension in general. ∎

In line with Example 6.2 we next obtain a general functional equation for our
zeta distributions. The idea of the proof is the same as for the analogous
results in [FK94, Ru06]. The $\mathcal{K}$-Bessel function and its asymptotic
properties will play an essential role.

######  Theorem 6.6.

For all $\alpha\in\mathbb{C}$, the zeta distributions $\zeta_{\alpha}$ satisfy
the functional equation

$\zeta_{\alpha}=2^{n(2\alpha-\nu)}\mathcal{F}^{B}\zeta_{\nu-\alpha}.$ (6.2)

Moreover, in view of by Theorem 5.6,

$\braket{R_{\alpha},f}=4^{n(\alpha-\nu)}\braket{R_{\nu-\alpha},\mathcal{H}_{\nu}g},\quad
f\in\mathscr{S}(\mathbb{R}^{n}),g(x)=f(\tfrac{x}{4}).$

For the proof of identity (6.2), we start with two lemmata, which are needed
to outsource some technicalities.

######  Lemma 6.7.

Consider $g\in\mathscr{S}(\mathbb{R}^{n})$ and $\alpha\in\mathbb{C}$.

  1. (i)

Assume that $\mathrm{Re}\,\alpha>\mu_{0}$. Then for arbitrary $\epsilon>0$,
the following integrals exist and coincide

$\displaystyle\int_{\mathbb{R}_{+}^{n}}\int_{\mathbb{R}^{n}}g(x)E^{A}(-\tfrac{1}{s},x^{2}+\underline{\epsilon})\Delta(s)^{-\alpha-\mu_{0}-1}\omega^{B}(x)\omega^{A}(s)\;\mathrm{d}x\;\mathrm{d}s$
$\displaystyle=\Gamma_{n}(\alpha)\int_{\mathbb{R}^{n}}g(x)\Delta(x^{2}+\underline{\epsilon})^{-\alpha}\omega^{B}(x)\;\mathrm{d}x.$

  2. (ii)

For $\mathrm{Re}\,\alpha<\mathrm{Re}\,\nu-\mu_{0}$ and $\epsilon>0$, the
following integral and limit exists

$\lim\limits_{\epsilon\to
0}\int_{\mathbb{R}^{n}}g(x)\Delta(x^{2}+\underline{\epsilon})^{-\alpha}\omega^{B}(x)\;\mathrm{d}x=\mathcal{Z}(g,\nu-\alpha).$

Furthermore, the integral on the left hand side exists for arbitrary
$\alpha\in\mathbb{C}$ and defines an entire function in $\alpha$.

###### Proof.

  1. (i)

By Theorem 3.2 and the condition $\mathrm{Re}\,\alpha>\mu_{0}$, we obtain

$\displaystyle\int_{\mathbb{R}^{n}_{+}}\int_{\mathbb{R}^{n}}|g(x)|E^{A}(-\tfrac{1}{s},x^{2}+\underline{\epsilon})\Delta(s)^{-\mathrm{Re}\,\alpha-\mu_{0}-1}|\omega^{B}(x)|\omega^{A}(s)\;\mathrm{d}x\;\mathrm{d}s$
$\displaystyle=\int_{\mathbb{R}^{n}}|g(x)|\int_{\mathbb{R}_{+}^{n}}E^{A}(-s,x^{2}+\underline{\epsilon})\Delta(s)^{\mathrm{Re}\,\alpha-\mu_{0}-1}\omega^{A}(s)\;\mathrm{d}s\,|\omega^{B}(x)|\;\mathrm{d}x$
$\displaystyle=\Gamma_{n}(\mathrm{Re}\,\alpha)\int_{\mathbb{R}^{n}}|g(x)|\Delta(x^{2}+\underline{\epsilon})^{-\mathrm{Re}\,\alpha}|\omega^{B}(x)|\;\mathrm{d}x<\infty,$

since $g$ is a Schwartz function, and
$x\mapsto\Delta(x^{2}+\epsilon)^{-\alpha}\omega^{B}(x)$ is continuous on
$\mathbb{R}^{n}$, and of polynomial growth. Hence, it is justified to change
the order of integration to obtain with Theorem 3.2 the stated equality.

  2. (ii)

Since $x\mapsto\Delta(x^{2}+\epsilon)^{-\alpha}\omega^{B}(x)$ is of polynomial
growth, the integral exists for all $\alpha\in\mathbb{C}$. For
$\mathrm{Re}\,\alpha<\mathrm{Re}\,\nu-\mu_{0}$ and $0<\epsilon<1$

$\left|\Delta(x^{2}+\underline{\epsilon})^{-\alpha}\right|\leq\begin{cases}\Delta(x^{2})^{-\mathrm{Re}\,\alpha}&\textrm{
if }\mathrm{Re}\,\alpha>0\\\ \Delta(x^{2}+1)^{-\mathrm{Re}\,\alpha}&\textrm{
if }\mathrm{Re}\,\alpha\leq 0\end{cases}.$

The dominated convergence theorem now proves the stated limit. Finally,
analyticity follows by usual theorems on holomorphic parameter integrals.

∎

######  Lemma 6.8.

Choose $m\in\mathbb{N}_{0}$ such that $\mu_{0}-m<\mathrm{Re}\,\nu-\mu_{0}$.
Consider $g\in\mathscr{S}(\mathbb{R}^{n})$,
$\widetilde{g}(x)=g(x)\Delta(x)^{m}$ and let $\alpha\in\mathbb{C}$ with
$\mathrm{Re}\,\alpha>\mu_{0}-m$.

  1. (i)

For arbitrary $\epsilon>0$ the following integrals exist and coincide

$\displaystyle\int_{\mathbb{R}_{+}^{n}}\int_{\mathbb{R}^{n}}g(x)E^{A}(-s,\tfrac{x^{2}}{4})E^{A}(-\tfrac{1}{s},\underline{\epsilon})\Delta(s)^{\nu-\alpha-\mu_{0}-1}\omega^{B}(x)\omega^{A}(s)\Delta(x)^{m}\;\mathrm{d}x\;\mathrm{d}s$
$\displaystyle=\epsilon^{n(\nu-\alpha)}\int_{\mathbb{R}^{n}}g(x)\mathcal{K}_{\alpha-\nu}(\underline{1},\epsilon\tfrac{x^{2}}{4})\Delta(x)^{m}\omega^{B}(x)\;\mathrm{d}x,$

where $\mathcal{K}_{\alpha-\nu}$ is the $\mathcal{K}$-Bessel function
according to Definition 4.7.

  2. (ii)

The following limit exists

$\quad\quad\quad\lim\limits_{\epsilon\to
0}\epsilon^{n(\nu-\alpha)}\int_{\mathbb{R}^{n}}\widetilde{g}(x)\mathcal{K}_{\alpha-\nu}(\underline{1},\epsilon\tfrac{x^{2}}{4})\omega^{B}(x)\;\mathrm{d}x=\frac{\Gamma_{n}(\nu-\alpha)}{4^{n\alpha}}\mathcal{Z}(g,\alpha-\nu+m).$

Moreover, the integral on the left hand side is holomorphic in $\alpha$ on the
domain $\left\\{\mathrm{Re}\alpha>\mu_{0}-m\right\\}$.

###### Proof.

  1. (i)

It suffices to justify, that we can change the order of integration.
Everything else follows from the definition of the $\mathcal{K}$-Bessel
function and its properties (Theorem 4.10). The change of variables
$s\mapsto\epsilon s$ and
$\mathcal{K}_{\nu-\alpha}(w,z)=\mathcal{K}_{\alpha-\nu}(z,w)$ lead to

$\displaystyle\int_{\mathbb{R}_{+}^{n}}\int_{\mathbb{R}^{n}}|g(x)|E^{A}(-s,\tfrac{x^{2}}{4})E^{A}(-\tfrac{1}{s},\underline{\epsilon})\Delta(s)^{\mathrm{Re}(\nu-\alpha)-\mu_{0}-1}\Delta(x)^{m}|\omega^{B}(x)|\omega^{A}(s)\;\mathrm{d}x\;\mathrm{d}s$
$\displaystyle=\epsilon^{n\mathrm{Re}(\nu-\alpha)}\int_{\mathbb{R}^{n}}\left|g(x)\right|\mathcal{K}_{\mathrm{Re}(\nu-\alpha)}(\epsilon\tfrac{x^{2}}{4},\underline{1})|\omega^{B}(x)|\;\mathrm{d}x.$

To see that this integral is finite, we observe that by Theorem 4.10 (iii)

$\displaystyle\Delta(x^{2})^{m}\mathcal{K}_{\mathrm{Re}(\alpha-\nu)}(\underline{1},\epsilon\tfrac{x^{2}}{4})$
$\displaystyle\leq C(\alpha)\begin{cases}1&\textrm{ if
}\mathrm{Re}\,\alpha>\mathrm{Re}\,\nu+\mu_{0}\\\
1+\Delta(x^{2})^{-\mu_{0}-\epsilon^{\prime}}&\textrm{ if
}\nu-\mu_{0}\leq\mathrm{Re}\,\alpha\leq\mathrm{Re}\,\nu+\mu_{0}\\\ &\textrm{
and arbitrary }0<\epsilon^{\prime}<1\\\
\epsilon^{n(\mathrm{Re}\,\alpha-\mathrm{Re}\,\nu)}\Delta(x^{2})^{\mathrm{Re}\,\alpha-\nu}&\textrm{
if }\mathrm{Re}\,\alpha<\mathrm{Re},\nu-\mu_{0}\end{cases},$

with some constant $C(\alpha)$. Hence, under the conditions
$\mathrm{Re}\,\alpha>\mu_{0}-m$ and $\mu_{0}-m<\mathrm{Re}\,\nu-\mu_{0}$, the
function

$x\mapsto\Delta(x^{2})^{m}\mathcal{K}_{\mathrm{Re}(\alpha-\nu)}(\underline{1},\epsilon\tfrac{x^{2}}{4})$

is of polynomial growth (choose
$0<\epsilon^{\prime}<m-2\mu_{0}-\mathrm{Re}\,\nu$ in the second case), and the
integral is finite.

  2. (ii)

The same estimates as in part (i), dominated converges, and the asymptotics of
Theorem 4.10 (vi) lead to the stated limit.

∎

###### Proof of Theorem 6.6.

The left hand side of the functional equation is given by a positive measure
if $\mathrm{Re}\,\alpha>\mu_{0}$, while the right hand side is a positive
measure if $\mathrm{Re}\,\alpha<\mathrm{Re}\,\nu-\mu_{0}$. So there are
possibly no indices $\alpha$ for which both sides are positive measures. To
bypass this problem, we replace the argument $f\in\mathscr{S}(\mathbb{R}^{n})$
by
$\widetilde{f}\coloneqq\Delta^{2m}(\Delta(T^{B})^{2m}f)\in\mathscr{S}(\mathbb{R}^{n})$
with large $m\in\mathbb{N}_{0}$, so large that
$\mu_{0}-m<\mathrm{Re}\,\nu-\mu_{0}$.

  1. (i)

Pick $\alpha\in\mathbb{C}$ with $\mathrm{Re}\,\alpha>\mu_{0}$. For
$s\in\mathbb{R}_{+}^{n}$ put $e_{s}(x)=E^{A}(-s,x^{2})$. The type $B$ Dunkl
transform of $e_{1/s}$ is computed by Lemma 5.2 and Theorem 5.6 (ii) as

$\mathcal{F}^{B}e_{1/s}(x)=2^{-n\nu}e_{s}(\tfrac{x}{2})\Delta(s)^{\nu}.$

The Plancherel theorem (cf. [dJ93]) for the Dunkl transform leads to

$\quad\quad\int_{\mathbb{R}^{n}}(\mathcal{F}^{B}\widetilde{f})(x)e_{1/s}(x)\omega^{B}(x)\;\mathrm{d}x=2^{-n\nu}\Delta(s)^{\nu}\int_{\mathbb{R}^{n}}\widetilde{f}(x)\cdot
e_{s}(\tfrac{x}{2})\omega^{B}(x)\;\mathrm{d}x.$

Multiplying this equation with
$\Delta(s)^{-\alpha-\mu_{0}-1}E^{A}(-\tfrac{1}{s},\underline{\epsilon})$,
$\epsilon>0$ and integrating over $\mathbb{R}_{+}^{n}$ gives

$\begin{split}\int_{\mathbb{R}^{n}_{+}}\int_{\mathbb{R}^{n}}(\mathcal{F}^{B}\widetilde{f})(x)E^{A}(-\tfrac{1}{s},x^{2}+\underline{\epsilon})\Delta(s)^{-\alpha-\mu_{0}-1}\omega^{B}(x)\omega^{A}(s)\;\mathrm{d}x\;\mathrm{d}s\\\
\quad\quad\quad=2^{-n\nu}\\!\int_{\mathbb{R}_{+}^{n}}\int_{\mathbb{R}^{n}}\widetilde{f}(x)E^{A}(-s,\tfrac{x^{2}}{4})E^{A}(-\tfrac{1}{s},\underline{\epsilon})\Delta(s)^{\nu-\alpha-\mu_{0}-1}\omega^{B}(x)\omega^{A}(s)\;\mathrm{d}x\;\mathrm{d}s.\end{split}$

By parts (i) of Lemmata 6.7 and 6.8, this reduces to

$\displaystyle\begin{split}\int_{\mathbb{R}^{n}}\mathcal{F}^{B}\widetilde{f}(x)&\Delta(x^{2}+\underline{\epsilon})^{-\alpha}\omega^{B}(x)\;\mathrm{d}x\\\
&=\frac{2^{-n\nu}\epsilon^{n(\nu-\alpha)}}{\Gamma_{n}(\alpha)}\int_{\mathbb{R}^{n}}\widetilde{f}(x)\mathcal{K}_{\alpha-\nu}(\underline{1},\epsilon\tfrac{x^{2}}{4})\omega^{B}(x)\;\mathrm{d}x.\end{split}$
(6.3)

In view of Lemma 6.7 (ii), the left hand side of (6.3) exists for all
$\alpha\in\mathbb{C}$ and is holomorphic in $\alpha$. For
$\mathrm{Re}\,\alpha>\mu_{0}-m$ the right hand side of (6.3) exists and
depends holomorphically on $\alpha$ by Lemma 6.8 (ii), because

$\widetilde{f}(x)\mathcal{K}_{\alpha-\nu}(\underline{1},\epsilon\tfrac{x^{2}}{4})=\Delta(T^{B})^{2m}f(x)\cdot\Delta(x^{2})^{m}\mathcal{K}_{\alpha-\nu}(\underline{1},\epsilon\tfrac{x^{2}}{4})$

and $\Delta(T^{B})^{2m}f\in\mathscr{S}(\mathbb{R}^{n})$. Hence, equality (6.3)
is true for all $\mathrm{Re}\,\alpha>\mu_{0}-m$.

  2. (ii)

Assume that $\alpha\in\mathbb{C}$ is contained in the strip
$\mu_{0}-m<\mathrm{Re}\,\alpha<\mathrm{Re}\,\nu-\mu_{0}$, which is non-empty
by our choice of $m$. Then, on the one hand, by Lemma 6.7

$\displaystyle\int_{\mathbb{R}^{n}}\mathcal{F}^{B}\widetilde{f}(x)\Delta(x^{2}+\underline{\epsilon})^{-\alpha}\omega^{B}(x)\;\mathrm{d}x$
$\displaystyle\underset{\epsilon\to
0}{\longrightarrow}\int_{\mathbb{R}^{n}}\mathcal{F}^{B}\widetilde{f}(x)\Delta(x^{2})^{-\alpha}\omega^{B}(x)\;\mathrm{d}x$
$\displaystyle=\mathcal{Z}(\mathcal{F}^{B}\widetilde{f};\nu-\alpha)$
$\displaystyle=\Gamma_{n}(\nu-\alpha)\braket{\zeta_{\alpha},\mathcal{F}^{B}\widetilde{f}}$
$\displaystyle=\Gamma_{n}(\nu-\alpha)\braket{\zeta_{\alpha},\Delta(T^{B})^{2m}(\Delta^{2m}f)}$
$\displaystyle=\Gamma_{n}(\nu-\alpha)\;p(\alpha)\braket{\zeta_{\alpha},f},$

with some polynomial $p$ (independent of $f$), where Theorem 6.5 was used.
On the other hand, Lemma 6.8 shows that

$\displaystyle\quad\quad\frac{2^{-n\nu}\epsilon^{n(\nu-\alpha)}}{\Gamma_{n}(\alpha)}$
$\displaystyle\int_{\mathbb{R}^{n}}\widetilde{f}(x)\mathcal{K}_{\alpha-\nu}(\underline{1},\epsilon\tfrac{x^{2}}{4})\omega^{B}(x)\;\mathrm{d}x$
$\displaystyle\underset{\epsilon\to
0}{\longrightarrow}\frac{\Gamma_{n}(\nu-\alpha)}{\Gamma(\alpha)2^{n(\nu+2\alpha}}\int_{\mathbb{R}^{n}}(\Delta(T^{B})^{2m}f)(x)\cdot\Delta(x^{2})^{\alpha-\nu+m}\omega^{B}(x)\;\mathrm{d}x$
$\displaystyle=\frac{\Gamma_{n}(\nu-\alpha)}{\Gamma_{n}(\alpha)2^{n(\nu+2\alpha)}}\mathcal{Z}(\Delta(T^{B})^{2m}f;\alpha-\nu+m)$
$\displaystyle=\frac{\Gamma_{n}(\nu-\alpha)}{\Gamma_{n}(\alpha)\Gamma_{n}(\alpha-\nu+m)2^{n(\nu+2\alpha)}}\braket{\zeta_{\alpha-\nu+m},\Delta(T^{B})^{2m}f}$
$\displaystyle=\frac{\Gamma_{n}(\nu-\alpha)}{\Gamma_{n}(\alpha)\Gamma_{n}(\alpha-\nu+m)}\;q(\alpha)\;\braket{\zeta_{\alpha},f},$

with some polynomial $q$ (independent of $f$), where Theorem 6.5 was used.
Therefore, we have found a meromorphic function $\rho$ on $\mathbb{C}$ such
that

$\zeta_{\alpha}=\rho(\alpha)\mathcal{F}^{B}\zeta_{\nu-\alpha}.$

Finally, by equation (6.1) of Example 6.2, we see that
$\rho(\alpha)=2^{n(2\alpha-\nu)}$.

∎

##  7\. Regularity of the zeta distributions

As for Riesz distributions in [Rö20], one may ask which of the zeta
distributions are regular and which are positive measures. The case of zeta
distributions reduces to the case of Riesz distributions, where the question
was answered in [Rö20]. It turns out that the set of indices
$\alpha\in\mathbb{C}$ for which $\zeta_{\alpha}$ is a positive measure only
depends on the multiplicity parameter $k$ of $\kappa_{B}=(k,k^{\prime})$.

######  Lemma 7.1.

Consider $\alpha\in\mathbb{C}$.

  1. (i)

$\zeta_{\alpha}$ is a positive or complex measure iff $R_{\alpha}$ is a
positive or complex measure, respectively.

  2. (ii)

If $\zeta_{\alpha}$ is a complex measure, then $\mathrm{Re}\,\alpha>\mu_{0}$
or $\alpha$ is contained in the finite set

$[0,\infty[\;\cap\;(\left\\{0,k,\ldots,k(n-1)\right\\}-\mathbb{N}_{0}).$

  3. (iii)

$\zeta_{\alpha}$ is a positive measure iff $\alpha$ is contained in the
generalized Wallach set

$W_{k}=\left\\{0,k,\ldots,k(n-1)\right\\}\cup\;]k(n-1),\infty[.$

###### Proof.

If we can show part (i), then the other parts are immediate from [Rö20,
Theorem 5.15]. First assume that $R_{\alpha}$ is a measure $\mu$. Then
$\mathrm{supp}\,\mu\subseteq\overline{\mathbb{R}_{+}^{n}}$ (cf. [Rö20]). Let
$f\in\mathscr{S}(\mathbb{R}^{n})$ and choose
$f_{0}\in\mathscr{S}(\mathbb{R}^{n})$ so that

$\frac{1}{2^{n}}\sum\limits_{\tau\in\mathbb{Z}_{2}^{n}}f(\tau
x)=f_{0}(x^{2}).$

Thus, the $W_{\\!B}$-invariance of $\zeta_{\alpha}$ gives

$\displaystyle\braket{\zeta_{\alpha},f}$
$\displaystyle=\braket{R_{\alpha},f_{0}}=\int_{\overline{\mathbb{R}_{+}^{n}}}f_{0}(x)\;\mathrm{d}\mu(x)=\frac{1}{2^{n}}\sum\limits_{\tau\in\mathbb{Z}_{2}^{n}}\int_{\overline{\mathbb{R}_{+}^{n}}}f(\tau\sqrt{x}\;\mathrm{d}\mu(x,$

i.e. $\zeta_{\alpha}$ is a measure. Conversely, if $\zeta_{\alpha}$ is a
measure $\mu$, then for $f\in\mathscr{S}(\mathbb{R}^{n})$

$\braket{R_{\alpha},f}=\braket{\zeta_{\alpha},f(x^{2})}=\int_{\mathbb{R}^{n}}f(x^{2})\;\;\mathrm{d}\mu(x),$

i.e. $R_{\alpha}$ is a measure. ∎

To conclude the study of zeta distributions, we shall explicitly compute
$\zeta_{\alpha}$ for $\alpha\in\left\\{0,k,\ldots,k(n-1)\right\\}$, the
discrete part of the generalized Wallach set. On the continuous part of the
generalized Wallach set, $\zeta_{\alpha}$ is given by the measure

$\tfrac{1}{\Gamma_{n}(\alpha)}\Delta(x^{2})^{\alpha-\nu}\omega^{B}(x)\;\mathrm{d}x=\tfrac{1}{\Gamma_{n}(\alpha)}\Delta(x^{2})^{\alpha}\omega^{A}(x^{2})\;\mathrm{d}x.$

In the following, $W_{\\!B}$ acts on $\mathscr{S}^{\prime}(\mathbb{R}^{n})$ by
$\braket{wu,f}=\braket{u,w^{-1}f}$ for
$u\in\mathscr{S}^{\prime}(\mathbb{R}^{n}),\,f\in\mathscr{S}(\mathbb{R}^{n})$
and $w\in W_{\\!B}$.

######  Theorem 7.2.

The zeta distribution on the discrete part of the Wallach set are given as

  1. (i)

$\zeta_{0}=\delta_{0}$.

  2. (ii)

For $r=1,\ldots,n-1$, $\zeta_{kr}$ is a positive measure, namely

$\zeta_{kr}=\frac{1}{n!}\sum\limits_{\sigma\in\mathcal{S}_{n}}(\zeta_{kn}^{(r)}\otimes\delta_{0}^{(n-r)})^{\sigma},$

where $\zeta_{kn}^{(r)}$ is the zeta distribution of index $kn$ on
$\mathbb{R}^{r}$ and $\delta_{0}^{(n-r)}$ is the Dirac measure in
$0\in\mathbb{R}^{n-r}$. Moreover, the support of $\zeta_{kr}$ is the stratum

$\quad\partial_{r}\mathbb{R}^{n}\coloneqq\left\\{x\in\mathbb{R}^{n}\mid
x_{i_{j}}=0\textrm{ for a sequence }1\leq i_{1}<\ldots<i_{r}\leq n\right\\}.$

###### Proof.

  1. (i)

By Theorem 6.6 and Theorem 5.6 (i) we obtain

$\displaystyle\braket{\zeta_{0},f}$
$\displaystyle=2^{-n\nu}\braket{\zeta_{\nu},\mathcal{F}^{B}f}=\frac{1}{2^{n\nu}\Gamma_{n}(\nu)}\int_{\mathbb{R}^{n}}\mathcal{F}^{B}f(x)\omega^{B}(x)\;\mathrm{d}x$
$\displaystyle=\mathcal{F}^{B}\mathcal{F}^{B}f(0)=f(0)=\braket{\delta_{0},f}.$

  2. (ii)

It suffices to verify the stated formula for $W_{\\!B}$-invariant
$f\in\mathscr{S}(\mathbb{R}^{n})$. Hence there is an
$\mathcal{S}_{n}$-invariant $f_{0}\in\mathscr{S}(\mathbb{R}^{n})$ with
$f(x)=f_{0}(x^{2})$. By [Rö20, Theorem 5.11] we have

$R_{kr}=\frac{1}{n!}\sum\limits_{\sigma\in\mathcal{S}_{n}}(R_{kn}^{(r)}\otimes\delta_{0}^{(n-r)})^{\sigma},$

where $R_{kn}^{(r)}$ is the Dunkl-type Riesz distribution on $\mathbb{R}^{r}$.
Therefore

$\displaystyle\braket{\zeta_{kr},f}$
$\displaystyle=\braket{R_{kr},f_{0}}=\frac{1}{n!}\sum\limits_{\sigma\in\mathcal{S}_{n}}\braket{(R_{kr}^{(r)}\otimes\delta_{0}^{(n-r)})^{\sigma},f_{0}}$
$\displaystyle=\braket{R_{kr}^{(r)},f_{0}(\cdot,0^{n-r})}=\braket{\zeta_{kr}^{(r)},f(\cdot,0^{n-r})}$
$\displaystyle=\frac{1}{n!}\sum\limits_{\sigma\in\mathcal{S}_{n}}\braket{(\zeta_{kr}^{(r)}\otimes\delta_{0}^{(n-r)})^{\sigma},f}.$

Since $r<n$, the index $kn$ is contained in the continuous part of the Wallach
set associated to the zeta distributions $\zeta_{\alpha}^{(r)}$ on
$\mathbb{R}^{r}$. Thus, $\zeta_{kn}^{(r)}$ has support $\mathbb{R}^{r}$, by
Theorem 6.5 (ii). In particular, the support of $\zeta_{kr}$ is

$\bigcup\limits_{\sigma\in\mathcal{S}_{n}}\sigma(\mathbb{R}^{r}\times\left\\{0\right\\}^{n-r})=\partial_{r}\mathbb{R}^{n}.$

∎

## References

  * [BF98] T.H. Baker, P.J. Forrester, Non-symmetric Jack polynomials and integral kernels. _Duke Math. J._ 95 (1998), 1–50.
  * [BR22] D. Brennecken, M. Rösler, Dunkl-Laplace transform of Jack polynomials and of hypergeometric functions. _Trans. Amer. Math. Soc._ 376 (2023), 2419–2446, doi.org/10.1090/tran/8860.
  * [Cl88] J.L. Clerc, Fonctions K de Bessel pour les algèbres de Jordan. _In: Harmonic Analysis, P. Eymard, J.P. Pier (Eds.), Lecture Notes in Math._ , vol. 1359 (1988), 122–134.
  * [Cl02] J.L. Clerc, Zeta distributions associated to a representation of a Jordan algebra. _Math. Z._ 239 (2002), no. 2, 263-276.
  * [Di90] H. Dib, Fonctions de Bessel sur une algèbre de Jordan. _J. Math. Pures Appl._ 69 (1990), 403–448.
  * [Du89] C.F. Dunkl, Differential-difference operators associated to reflection groups. _Trans. Amer. Math. Soc._ 311 (1989), 167–183.
  * [DX14] C. Dunkl, Y. Xu, Orthogonal polynomials of Several Variables, 2nd ed., _Encyclopediaof Mathematics and its Applications_ , vol. 155, Cambridge University Press, Cambridge, 2014.
  * [FK94] J. Faraut, A. Korányi, Analysis on Symmetric Cones. _Oxford Mathematical Monographs_. Oxford Science Publications. Oxford University Press, the Clarendon Press, New York, 1994.
  * [Fo10] P.J. Forrester, Log-Gases and Random Matrices. _London Math. Soc. Monogr. Ser._ , 34. Princeton University Press, Princeton, NJ, 2010.
  * [JP16] S. Jakšić, B. Prangoski, Extension theorem of Whitney type for $\mathcal{S}(\mathbb{R}_{+}^{d})$ by use of the Kernel theorem. _Publ. Inst. Math. (Beograd)(N.S.)_ vol. 99 (2016), 59–65.
  * [dJ93] M. de Jeu, The Dunkl transform. _Invent. Math._ 113 (1993), 147–162.
  * [He55] C. Herz, Bessel functions of matrix argument. _Ann. of Math._ 61 (1955), 474–523.
  * [Ma13] I.G. Macdonald, Hypergeometric functions I. _arXiv_ : 1309.4568v1 (math.CA).
  * [Mö13] J. Möllers, A geometric quantization of the Kostant-Sekiguchi correspondence for scalar type unitary highest weight representations. _Doc. Math._ 18 (2013), 785-855.
  * [Mu82] R.J. Muirhead, Aspects of multivariate statistical theory. _Wiley Ser. Probab. Stat._ , New York, 1982.
  * [Li16] Y. Liu, Riesz distributions associated to Dunkl operators, Dissertation. _Cornell University_ , 2016
  * [Rö98] M. Rösler, Generalized Hermite polynomials and the heat equation for Dunkl operators. _Commun. Math. Phys._ 192 (1998), 519–542.
  * [Rö99] M. Rösler, Positivity of Dunkl’s intertwining operator. _Duke. Math. J._ 98 (1999), 445-463.
  * [Rö03] M. Rösler, Dunkl operators: theory and applications. _In: E. Koelink, W. van Assche (Eds.), Lecture Notes in Math._ vol. 1817 (2003), 93–136.
  * [RV04] M. Rösler, M. Voit, Positivity of Dunkl’s intertwining operator via the trigonometric setting. _Internat. Math. Res. Not. IMRN_ 63 (2004), 3379–3389.
  * [Rö07] M. Rösler, Bessel convolutions on matrix cones. _Compos. Math._ 143 (2007), 749–779.
  * [RKV13] M. Rösler, T. Koornwinder, M. Voit, Limit transition between hypergeometric functions of type BC and type A. _Compos. Math._ 149 (2013), 1381–1400.
  * [Rö20] M. Rösler, Riesz distributions and the Laplace transform in the Dunkl setting of type A. _J. Funct. Anal._ 278 (2020), no 12, 108506, 29 pp.
  * [Ru06] B. Rubin, Riesz potentials and integral geometry in the space of rectangular matrices. _Adv. Math._ 205 (2006), 549–598.
  * [Sa98] S. Sahi, The binomial formula for nonsymmetric Macdonald polynomials. _Duke Math. J._ 94 (1998), 465–477.
  * [SZ07] S. Sahi, G. Zhang, Biorthogonal expansion of non-symmetric Jack functions. _SIGMA Symmetry Integrability Geom. Methods Appl._ 3 (2007), paper 106, 9 pp.
  * [Ste19] K. Stempak, An extension problem for $C^{\infty}$ functions with symmetries. _J. Approx. Theory_ 240 (2019), 114–125.
  * [Sta89] R.P. Stanley, Some combinatorial properties of Jack symmetric functions. _Adv. Math._ 77 (1989), 76–115.
  * [Wh43] H. Whitney. Differentiable even functions. _Duke Math. J._ 10 (1943), 159–160.