# Orthogonality of the big $-1$ Jacobi polynomials for non-standard parameters

Howard S. Cohl  Applied and Computational Mathematics Division, National
Institute of Standards and Technology, Gaithersburg, MD 20899-8910, USA
<EMAIL_ADDRESS>and  Roberto S. Costas-Santos  Department of Quantitative
Methods, Universidad Loyola Andalucía, Sevilla, Spain<EMAIL_ADDRESS>
###### Abstract.

The big $-1$ Jacobi polynomials $(Q_{n}^{(0)}(x;\alpha,\beta,c))_{n}$ have
been classically defined for $\alpha,\beta\in(-1,\infty)$, $c\in(-1,1)$. We
extend this family so that wider sets of parameters are allowed, i.e., they
are non-standard. Assuming initial conditions $Q^{(0)}_{0}(x)=1$,
$Q^{(0)}_{-1}(x)=0$, we consider the big $-1$ Jacobi polynomials as monic
orthogonal polynomials which therefore satisfy the following three-term
recurrence relation

$xQ^{(0)}_{n}(x)=Q^{(0)}_{n+1}(x)+b_{n}Q^{(0)}_{n}(x)+u_{n}Q^{(0)}_{n-1}(x),\quad
n=0,1,2,\ldots.$

For standard parameters, the coefficients $u_{n}>0$ for all $n$. We discuss
the situation where Favard’s theorem cannot be directly applied for some
positive integer $n$ such that $u_{n}=0$. We express the big $-1$ Jacobi
polynomials for non-standard parameters as a product of two polynomials. Using
this factorization, we obtain a bilinear form with respect to which these
polynomials are orthogonal.

###### 2020 Mathematics Subject Classification:

Primary 44A20; Secondary 42C05; 33C05;

††copyright: ©2009: American Mathematical Society

##  1\. Introduction

In 2011, Vinet and Zhedanov in [19] obtained a new family of “classical”
orthogonal polynomials which can be obtained from the little $q$-Jacobi
polynomials in the limit $q\to-1$. By “classical” it is meant that these
polynomials satisfy (apart from a three-term recurrence relation) a nontrivial
eigenvalue equation of the form

${\mathcal{L}}(p_{n}(x))=\lambda_{n}p_{n}(x),$

where $\mathcal{L}$ is a linear differential-difference operator which is of
first order in the derivative $\partial_{x}$ and contains a reflection
operator $R$ which acts as $Rf(x)=f(-x)$. They referred to these polynomials
as being in the continuous $-1$ hypergeometric orthogonal polynomial scheme.
This was illustrated by using a limiting process from the Askey–Wilson
polynomials to the Bannai–Ito polynomials. In [20] the same authors applied
this limit by starting with the big $q$-Jacobi polynomials in order to obtain
hypergeometric representations for the big $-1$ Jacobi polynomials.

The polynomials the authors have considered in this $-1$ hypergeometric
orthogonal polynomial scheme are $q$-polynomials for standard parameters,
i.e., those values for the parameters whose weight function related to these
families is positive definite (see [15]). The main aim of this work is to
investigate the property of orthogonality for the big $-1$ polynomials for
those values of the parameters, $\alpha$, and $\beta$, for which the
coefficient $u_{n}$ in the recurrence relation (1.2) relative to these
polynomials can be equal to zero for some $n\in\mathbb{N}_{0}$.

In the last three decades, some of the classical orthogonal polynomials with
non-classical parameters have been provided with certain non-standard
orthogonality properties. In the pioneering work by Kwon and Littlejohn in
1995 [9], Sobolev orthogonality for the Laguerre polynomials $L_{n}^{(-N)}(x)$
with $N\in\mathbb{N}$, was obtained using the following inner product:

$\left\langle
f,g\right\rangle=\sum_{m=0}^{N-1}\sum_{j=0}^{m}B_{m,j}(N)\left(f^{(m)}(0)g^{(j)}(0)+f^{(j)}(0)g^{(m)}(0)\right)+\int_{0}^{\infty}f^{(N)}(x)g^{(N)}(x)\,{\mathrm{e}}^{-x}\,{\mathrm{d}}x,$

where the superscripts represent differentiation and

$B_{m,j}(N):=\left\\{\begin{array}[]{ll}\displaystyle\sum_{p=0}^{j}(-1)^{m+j}\binom{N-1-p}{m-p}\binom{N-1-p}{j-p},&0\leq
j\leq m\leq N-1,\\\\[19.91692pt]
\displaystyle\frac{1}{2}\sum_{p=0}^{m}\binom{N-1-p}{m-p}^{2},&0\leq j=m\leq
N-1.\end{array}\right.$

In 1998, Álvarez de Morales and co-authors [3] found, using a different
technique, the orthogonality for ultraspherical polynomials
$C_{n}^{-N+\frac{1}{2}}$, where $N\in{\mathbb{N}}$. Later in [1], M. Alfaro et
al. considered the cases for the Jacobi polynomials in which both parameters,
$\alpha$, and $\beta$, were negative integers, proving that in such a case the
Jacobi polynomials satisfy a Sobolev orthogonality. In [2], M. Alfaro et al.
considered the situation where the inner product can be written in the form

(1.1) $\langle f,g\rangle=F^{t}AG+\int
f^{(N)}(x)g^{(N)}(x)\,\mathrm{d}\mu(x)\,,$

where $F$ and $G$ are vectors obtained by evaluating $f$ and $g$ and maybe
their derivatives at some points, $A$ is a symmetric real matrix, and
$\mathrm{d}\mu$ is the orthogonality measure associated with the $N$th
derivative of either the Laguerre, ultraspherical or Jacobi polynomials.

Note that for the parameters considered in this situation there exists a
$n=N\in{\mathbb{N}}_{0}$ so that the coefficient $u_{n}$ in the recurrence
relation

(1.2) $xp_{n}=p_{n+1}+b_{n}p_{n}+u_{n}p_{n-1}\,,\qquad n=0,1,2,....$

vanishes, i.e., $u_{N}=0$. We say a non-standard parameter (or set of
parameters) is degenerate if there exists at least one positive integer $N$
for which $u_{N}=0$.

The first term in the inner product (1.1) plays the role of the orthogonality
for the set of polynomials of degree less than $N$, since if $f,g$ are two
polynomials, with $\deg f$, $\deg g<N$, then $f^{(N)}(x)=g^{(N)}(x)=0$, and
one has

$\langle f,g\rangle=F^{t}AG.$

In this term, the points for the evaluation are the roots of $p_{N}$; this
ensures that the inner product vanishes when one entry is $p_{n}$ with $n\geq
N$, since $p_{N}$ is a factor of any $p_{n}$ with $n\geq N$. The second term
is relevant for polynomials with degree greater than $N$, and it is needed in
order to have an orthogonality characterizing the sequence
$(p_{n})_{n=0}^{\infty}$. The technique used in [3] is applicable for all
classical orthogonal polynomials where $u_{n}$ vanishes at certain
$n=N\in\mathbb{N}_{0}$, and in fact it has been used, among others, in [11,
12, 13, 14].

In 2009 [5], Costas-Santos and Sanchez-Lara studied this problem for classical
discrete polynomials, i.e., discrete polynomials in the Askey Scheme of
generalized hypergeometric orthogonal polynomials. We consider classical
discrete polynomials for which there exists $N$ for which $u_{N}=0$, i.e., for
non-standard degenerate parameters. Note that this can only happen for the
Racah, Hahn, dual Hahn, and Krawtchouk polynomials which are the Wilson,
continuous Hahn, continuous dual Hahn and Meixner polynomials for which some
of its parameters are equal to a negative integer. The corresponding three-
term recurrence relation for these polynomials also presents one vanishing
coefficient i.e., $u_{N}=0$, and the inner product found can be written as

(1.3) $\langle f,g\rangle=\int f(x)g(x)\,\mathrm{d}\mu_{d}(x)+\int
f^{(N)}(x)g^{(N)}(x)\,\mathrm{d}\mu_{c}(x)\,,$

where $\mathrm{d}\mu_{d}$ is a discrete measure with a finite number of masses
and $\mathrm{d}\mu_{c}$ is an absolutely continuous measure. For more details
and further reading see [5, 6, 8, 16, 18] and references therein. The
organization of the paper is as follows. In Section 2 we provide some
preliminary material and in Section 3 we obtain the property of orthogonality
for the big $-1$ Jacobi polynomials for non-standard degenerate parameters.

##  2\. Preliminary material and notations

We adopt the following notations:
$\mathbb{N}_{0}:=\\{0\\}\cup\mathbb{N}=\\{0,1,2,...\\}$, and we use the sets
$\mathbb{P}$, $\mathbb{P}^{\prime}$, to represent the linear space of
polynomials with complex coefficients and its algebraic dual space. We denote
by $\langle{\bf u},p\rangle$ the duality bracket for ${\bf
u}\in\mathbb{P}^{\prime}$ and $p\in\mathbb{P}$. The big $-1$ Jacobi
polynomials have representations given in terms of the Gauss hypergeometric
function defined as [17, (15.2.1)]

(2.1)
${}_{2}F_{1}\\!\left(\genfrac{}{}{0.0pt}{}{a,b}{c};z\right):=\sum_{n=0}^{\infty}\frac{(a)_{n}(b)_{n}}{(c)_{n}}\frac{z^{n}}{n!},$

where $|z|<1$ and the shifted factorial is defined as [17, (5.2.4-5)]
$(a)_{n}:=(a)(a+1)\cdots(a+n-1)$, $a\in\mathbb{C}$, $n\in{\mathbb{N}}_{0}$.
The following ratio of two gamma functions [17, Chapter 5] are related to the
shifted factorial, namely for $a\in\mathbb{C}\setminus-{{\mathbb{N}}}_{0}$,
one has

(2.2) $(a)_{n}=\frac{\Gamma(a+n)}{\Gamma(a)},$

which allows one to extend the definition to non-positive integer values of
$n$. For a description of the properties of the gamma function, see [17,
hrefhttp://dlmf.nist.gov/5Chapter 5].

##  3\. Big $-1$ Jacobi polynomials

The monic big $-1$ Jacobi polynomials can be defined in terms of the Gauss
hypergeometric function as follows:

(3.1) $\hskip
8.5359ptQ_{n}^{(0)}(x):=\\!\kappa_{n}\left\\{\begin{array}[]{ll}\displaystyle\\!\\!{}_{2}F_{1}\\!\left(\genfrac{}{}{0.0pt}{}{-\frac{n}{2},\frac{n+\alpha+\beta+2}{2}}{\frac{\alpha+1}{2}};\dfrac{1\\!-\\!x^{2}}{1\\!-\\!c^{2}}\right)\\!+\\!\dfrac{n(1-x)}{(c+1)(\alpha+1)}{}_{2}F_{1}\\!\left(\genfrac{}{}{0.0pt}{}{1-\frac{n}{2},\frac{n+\alpha+\beta+2}{2}}{\frac{\alpha+3}{2}};\dfrac{1\\!-\\!x^{2}}{1\\!-\\!c^{2}}\right),\
\quad\mathrm{if}\ n\ \mathrm{even},\\\\[18.0pt]
\displaystyle\\!{}_{2}F_{1}\\!\left(\genfrac{}{}{0.0pt}{}{-\frac{n-1}{2},\frac{n+\alpha+\beta+1}{2}}{\frac{\alpha+1}{2}};\dfrac{1\\!-\\!x^{2}}{1\\!-\\!c^{2}}\right)\\!-\\!\dfrac{(\alpha\\!+\\!\beta\\!+\\!n\\!+\\!1)(1\\!-\\!x)}{(1\\!+\\!c)(\alpha\\!+\\!1)}{}_{2}F_{1}\\!\left(\genfrac{}{}{0.0pt}{}{-\frac{n-1}{2},\frac{n+\alpha+\beta+3}{2}}{\frac{\alpha+3}{2}};\dfrac{1\\!-\\!x^{2}}{1\\!-\\!c^{2}}\right),\
\mathrm{if}\ n\ \mathrm{odd},\end{array}\right.$

where $\kappa_{n}$ is defined as

(3.2) $\hskip
8.5359pt\kappa_{n}:=\\!\left\\{\begin{array}[]{ll}\displaystyle\\!\dfrac{(1-c^{2})^{\frac{1}{2}n}(\frac{1}{2}(\alpha+1))_{\frac{1}{2}n}}{(\frac{1}{2}(n+\alpha+\beta+2))_{\frac{1}{2}n}},\quad\mathrm{if}\
n\ \mathrm{even},\\\\[18.0pt]
\displaystyle\\!(c+1)\dfrac{(1-c^{2})^{\frac{1}{2}(n-1)}(\frac{1}{2}(\alpha+1))_{\frac{1}{2}(n-1)}}{(\frac{1}{2}(n+\alpha+\beta+1))_{\frac{1}{2}(n-1)}},\quad\mathrm{if}\
n\ \mathrm{odd}.\end{array}\right.$

The big $-1$ Jacobi polynomials $Q_{n}^{(0)}(x;\alpha,\beta,c)$, satisfy the
three-term recurrence relation [20, (2.20), (2.21)]:

(3.3)
$xQ_{n}^{(0)}(x)=Q_{n+1}^{(0)}(x)+b_{n}Q_{n}^{(0)}(x)+u_{n}Q_{n-1}^{(0)}(x),$

where

(3.4) $\hskip
8.5359ptb_{n}:=\\!\left\\{\begin{array}[]{ll}\displaystyle\\!-c+\dfrac{(c-1)n}{\alpha+\beta+2n}+\dfrac{(c+1)(\beta+n+1)}{\alpha+\beta+2n+2},\quad\mathrm{if}\
n\ \mathrm{even},\\\\[14.0pt]
\displaystyle\\!c-\dfrac{(c-1)(n+1)}{\alpha+\beta+2n+2}-\dfrac{(c+1)(\beta+n)}{\alpha+\beta+2n},\quad\mathrm{if}\
n\ \mathrm{odd},\end{array}\right.$

and

(3.5) $\hskip
8.5359ptu_{n}:=\\!\left\\{\begin{array}[]{ll}\displaystyle\\!\dfrac{(c-1)^{2}n(\alpha+\beta+n)}{(\alpha+\beta+2n)^{2}},&\quad\mathrm{if}\
n\ \mathrm{even},\\\\[14.0pt]
\displaystyle\\!\dfrac{(c+1)^{2}(\alpha+n)(\beta+n)}{(\alpha+\beta+2n)^{2}},&\quad\mathrm{if}\
n\ \mathrm{odd},\end{array}\right.$

with initial conditions $Q^{(0)}_{-1}(x)=0$, $Q^{(0)}_{0}(x)=1$. For any real
$c\neq 1$ and real $\alpha$, $\beta$ satisfying the restriction
$\alpha,\beta>-1$, the recurrence coefficients $b_{n}$, $u_{n}$ are real and
positive. Hence the big $-1$ Jacobi polynomials are positive definite
orthogonal polynomials.

######  Remark 3.1.

Observe that if $\alpha$ (resp. $\beta$) is a negative odd integer, namely
$\alpha=-2N+1$ (resp. $\beta=-2N+1$) then $u_{2N-1}=0$, i.e., $\alpha$ ( resp.
$\beta$) is non-standard degenerate parameter. So we can apply the degenerate
version of Favard’s theorem (see [6, Theorem 2.2]), i.e., there exist moment
functionals, $\mathscr{L}_{0}$ and $\mathscr{L}_{N}$, such that the polynomial
sequence is orthogonal with respect to the bilinear form

$\langle
p,r\rangle=\mathscr{L}_{0}(pr)+\mathscr{L}_{N}(\mathscr{T}^{(2N-1)}p\mathscr{T}^{(2N-1)}r),\quad
p,r\in\mathbb{P},$

where $\mathscr{T}$ is a certain lowering operator.

Taking the former remark into account, the Gauss hypergeometric function with
a suitable normalization factorizes as follows.

######  Lemma 3.2 (Factorization).

Let $n,N\in{\mathbb{N}}$, $a\in\mathbb{C}$. Then

(3.6)
$\begin{split}(-N+1)_{n+N}\,{}_{2}F_{1}\\!\left(\genfrac{}{}{0.0pt}{}{-n-N,a}{-N+1};x\right)=&(N+1)_{n}(-N+1)_{N}\,{}_{2}F_{1}\\!\left(\genfrac{}{}{0.0pt}{}{-N,a}{-N+1};x\right){}_{2}F_{1}\\!\left(\genfrac{}{}{0.0pt}{}{-n,a+N}{N+1};x\right).\end{split}$

###### Proof.

By definition one has

$\begin{split}(-N+1)_{n+N}\,{}_{2}F_{1}\\!\left(\genfrac{}{}{0.0pt}{}{-n-N,a}{-N+1};x\right)=&(-N+1)_{n+N}\sum_{k=0}^{n+N}\dfrac{(-n-N,a)_{k}}{(-N+1,1)_{k}x^{k}}\\\
=&\sum_{k=0}^{n+N}\dfrac{(-n-N,a)_{k}}{(1)_{k}}(-N+k+1)_{n+N-k}x^{k}.\end{split}$

Then setting $k=k+N$ we have

(3.7)
$\displaystyle(-N+1)_{n+N}\,{}_{2}F_{1}\\!\left(\genfrac{}{}{0.0pt}{}{-n-N,a}{-N+1};x\right)$
$\displaystyle=$
$\displaystyle\sum_{k=0}^{n}\dfrac{(-n-N,a)_{k+N}}{(1)_{k+N}}(k+1)_{n-k}x^{k+N}$
$\displaystyle=$
$\displaystyle\dfrac{(-n-N,a)_{N}n!}{N!}x^{N}\sum_{k=0}^{n}\dfrac{(-n,a+N)_{k}}{(N+1,1)_{k}}x^{k}$
$\displaystyle=$
$\displaystyle(N+1)_{n}(a)_{N}(-x)^{N}\,{}_{2}F_{1}\\!\left(\genfrac{}{}{0.0pt}{}{-n,a+N}{N+1};x\right).$

Setting $n=0$ in (3.7), one obtains

(3.8)
$(-N+1)_{N}\,{}_{2}F_{1}\\!\left(\genfrac{}{}{0.0pt}{}{-N,a}{-N+1};x\right)=(a)_{N}(-x)^{N}.$

Combining (3.7) and (3.8) produces the result. ∎

Now that we have obtained factorization for the Gauss hypergeometric function,
we will use it to obtain a factorization for the big $-1$ Jacobi polynomials
when the parameter $\alpha$ (resp. $\beta$) is equal to $-2N-1$ for some
positive integer $N$.

######  Lemma 3.3 (Factorization of big $-1$ Jacobi polynomials).

Let $m,M,N\in\mathbb{N}_{0}$, $\alpha,\beta,c\in\mathbb{C}$, $c\neq\pm 1$,
with $\alpha=-2N-1$ or $\beta=-2N-1$. Then

(3.9) $\displaystyle Q^{(0)}_{2N+1}(x;-2N-1,\beta,c)=(x^{2}-1)^{N}(x-1),$
(3.10) $\displaystyle
Q^{(0)}_{2N+1}(x;\alpha,-2N-1,c)=(x^{2}-c^{2})^{N}(x+c),$ (3.11)
$\displaystyle
Q^{(0)}_{2N+1+m}(x;-2N-1,\beta,c)=(-1)^{m}(x^{2}-1)^{N}(x-1)Q^{(0)}_{m}(-x;2N+1,\beta,-c),$
(3.12) $\displaystyle
Q^{(0)}_{2N+1+m}(x;\alpha,-2N-1,c)=(x^{2}-c^{2})^{N}(x+c)Q^{(0)}_{m}(x;\alpha,2N+1,-c).$

###### Proof.

Identities (3.9) and (3.10) follow from (3.12) and (3.11) by setting $m=0$ in
these. We will first prove (3.11) and then (3.12). Consider $m=2M$,
$n=2N+1+m$. Then one has

$\displaystyle
Q^{(0)}_{2N+1+m}(x;\alpha,-2N-1,c)=\kappa_{n}\biggl{(}{}_{2}F_{1}\\!\left(\genfrac{}{}{0.0pt}{}{-N-M,M+\widehat{\beta}}{-N};\dfrac{1-x^{2}}{1-c^{2}}\right)$
(3.13) $\displaystyle\hskip
113.81102pt+\frac{(M+\widehat{\beta})(1-x)}{(c+1)N}{}_{2}F_{1}\\!\left(\genfrac{}{}{0.0pt}{}{-N-M,M+1+\widehat{\beta}}{-N+1};\frac{1-x^{2}}{1-c^{2}}\right)\biggr{)},$

where $\widehat{\beta}=(\beta+1)/2$. Next using Lemma 3.2 and after a tedious
calculation comparing the final expression with the one for
$Q_{m}(-x;2N+1,\beta,-c)$ and its corresponding normalization coefficient, the
identity for the even $m$ values holds. For $n=2N+1+2M+1$, i.e., $m=2M+1$, for
$\alpha=-2N-1$, the proof is similar and will be omitted. Hence (3.11)
follows. The identity (3.12) holds by applying the same procedure, which
completes the proof. ∎

A different way to guess and then obtain the factorization for the big $-1$
Jacobi polynomials for non-standard parameters throughout is to use the
following two step procedure. The first step is to use Lemma 3.2 in order to
prove the identities (3.9), (3.10). Once we have that, it is straightforward
to find, e.g., for $m\geq 0$, one has

$Q_{2N+1+m}^{(0)}(x;-2N-1,\beta,c)=Q_{2N+1}^{(0)}(x;-2N-1,\beta,c)P_{m}(x),$

where $P_{m}\in\mathbb{P}$ of degree $m$. In the second step, we obtain a new
recurrence relation for these new polynomials and comparing the recurrence
coefficients, we prove this new polynomial sequence is related to the original
one.

######  Lemma 3.4.

For any $n,N\in{\mathbb{N}}$, $\alpha,\beta,c\in\mathbb{C}$, $c\neq\pm 1$,
with $\alpha=-2N-1$ (resp. $\beta=-2N-1$), the following recurrence relation
holds for $n\geq 0$:

$xQ^{(0)}_{n}(x;2N+1,\beta,-c)=-Q^{(0)}_{n+1}(x;2N+1,\beta,-c)+\widehat{b}_{n}Q^{(0)}_{n}(x;2N+1,\beta,-c)+\widehat{u}_{n}Q^{(0)}_{n-1}(x;2N+1,\beta,-c),$

where

$\widehat{b}_{n}=-b_{n+2N+1}(-2N-1,\beta,c)=b_{n}(2N+1,\beta,-c),\qquad\widehat{u}_{n}=u_{n+2N+1}(-2N-1,\beta,c)=u_{n}(2N+1,\beta,-c).$

Respectively, the following recurrence relation holds for $n\geq 0$:

$xQ^{(0)}_{n}(x;\alpha,2N+1,-c)=Q^{(0)}_{n+1}(x;\alpha,2N+1,-c)+\widetilde{b}_{n}Q^{(0)}_{n}(x;\alpha,2N+1,-c)+\widetilde{u}_{n}Q^{(0)}_{n-1}(x;\alpha,2N+1,-c),$

where

$\widetilde{b}_{n}=b_{n+2N+1}(\alpha,-2N-1,c)=b_{n}(\alpha,2N+1,-c),\qquad\widetilde{u}_{n}=u_{n+2N+1}(\alpha,-2N-1,c)=u_{n}(\alpha,2N+1,-c).$

###### Proof.

We start by considering $\alpha=-2N-1$, then one has $u_{2N+1}=0$. In fact,

$xQ^{(0)}_{2N+1}(x;2N+1,\beta,c)=(x^{2}-1)^{N}(x-1).$

If we consider $n\geq m+2N+1$, then

$Q^{(0)}_{2N+1+m}(x;-2N-1,\beta,c)=Q^{(0)}_{2N+1}(x;-2N-1,\beta,c)(-1)^{m}Q^{(0)}_{m}(-x;2N+1,\beta,-c),$

and therefore the recurrence relation for these polynomials can be written as

$\begin{split}x(-1)^{m}Q^{(0)}_{m}(-x;2N+1,\beta,-c)=&(-1)^{m+1}Q^{(0)}_{m+1}(-x;2N+1,\beta,-c)+b_{m+2N+1}(-1)^{m}Q^{(0)}_{m}(-x;2N+1,\beta,-c)\\\
&+u_{m+2N+1}(-1)^{m-1}Q^{(0)}_{m-1}(-x;2N+1,\beta,-c).\end{split}$

After some straightforward calculations the result follows. For $\beta=-2N-1$,
the derivation is analogous which completes the proof. ∎

Note that if $u_{2N+1}=0$ for some $N\in\mathbb{N}$ then the orthogonality
property for the big $-1$ Jacobi polynomials holds for polynomials of degree
less or equal to $2N+1$. In the next result, we construct, thanks to the
factorization of the big $-1$ polynomials, a new property of orthogonality
that is valid for the big $-1$ Jacobi polynomials even when $n>2N+1$, i.e., we
can extend the property of orthogonality for the big $-1$ polynomials for non-
standard degenerate parameters.

######  Theorem 3.5 (Orthogonality of big $-1$ Jacobi polynomials for non-
standard parameters).

Let $N\in{\mathbb{N}}_{0}$, $c\in\mathbb{C}$, $c\neq\pm 1$. Define the linear
operator $\bf u$ [20, Section 4] as

$\langle{\bf
u},pq\rangle:=\int_{[-c,-1]\cup[1,c]}p(x)q(x)\,\dfrac{x}{|x|}(x+1)(c-x)(x^{2}-1)^{\frac{1}{2}(\alpha-1)}(c^{2}-x^{2})^{\frac{1}{2}(\beta-1)}\,\mathrm{d}x.$

The norm with respect to the linear functional ${\bf u}\in\mathbb{P}^{\prime}$
is given by $h_{n}\\!:=\\!h_{n}(\alpha,\beta,c)$,

(3.14) $\hskip 8.5359pth_{n}(\alpha,\beta,c)\\!:=\\!\langle{\bf
u},Q_{n}^{(0)}Q_{n}^{(0)}\rangle\\!=\\!\langle{\bf
u},1\rangle\\!\left\\{\begin{array}[]{ll}\displaystyle\\!\dfrac{2(c^{2}-1)^{n}(\frac{1}{2}n)!(\frac{1}{2}(\alpha+1),\frac{1}{2}(\beta+1))_{\frac{1}{2}n}}{(\frac{1}{2}(\alpha+\beta)+1)_{n}(\frac{1}{2}(\alpha+\beta+n)+1)_{\frac{1}{2}n}},\
\mathrm{if}\ n\ \mathrm{even},\\\\[18.0pt]
\displaystyle\\!\dfrac{2(c\\!-\\!1)^{n-1}(c\\!+\\!1)^{n+1}(\frac{1}{2}(n\\!-\\!1))!(\frac{1}{2}(\alpha\\!+\\!1),\frac{1}{2}(\beta\\!+\\!1))_{\frac{1}{2}(n+1)}}{(\frac{1}{2}(\alpha\\!+\\!\beta)+1)_{n}(\frac{1}{2}(\alpha\\!+\\!\beta\\!+\\!n\\!+\\!1))_{\frac{1}{2}(n+1)}},\
\mathrm{if}\ n\ \mathrm{odd}.\end{array}\right.$

Then, the polynomial sequences

(3.15)
$\displaystyle(Q_{n}^{(0)}(x;-2N-1,\beta,c)),\quad(Q_{n}^{(0)}(x;\alpha,-2N-1,c)),$

are orthogonal with respect to the bilinear forms

(3.16) $\displaystyle\langle
p,q\rangle_{1}={\mathscr{L}}_{0}(p,q)+\lambda_{N}(\beta)\langle{\bf
u}_{1},(\tau_{\alpha}^{2N+1}p)(\tau_{\alpha}^{2N+1}q)\rangle,$ (3.17)
$\displaystyle\langle
p,q\rangle_{2}={\mathscr{L}}_{1}(p,q)+\lambda_{N}(\alpha)\langle{\bf
u}_{2},(\tau_{\beta}^{2N+1}p)(\tau_{\beta}^{2N+1}q)\rangle,$

respectively. Here $\tau_{\alpha}$, $\tau_{\beta}$ are linear operators
defined such that

$\tau_{\alpha}[Q^{(0)}_{n}(x;\alpha,\beta,c)]=Q_{n-1}(-x;\alpha+2,\beta,-c),$
$\tau_{\beta}[Q^{(0)}_{n}(x;\alpha,\beta,c)]=Q_{n-1}(x;\alpha,\beta+2,-c),$

and

$\langle{\bf u}_{1},f(x,\alpha,\beta,c)\rangle:=\langle{\bf
u},f(-x,-\alpha,\beta,-c)\rangle,\quad\langle{\bf
u}_{2},f(x,\alpha,\beta,c)\rangle:=\langle{\bf
u},f(x,\alpha,-\beta,-c)\rangle,$

$\mathscr{L}_{0}$ and $\mathscr{L}_{1}$ are two moment linear functionals, and

$\lambda_{N}(\alpha)=\frac{1}{2}h_{2N+1}(-2N-1,\alpha,c),\quad\lambda_{N}(\beta)=\frac{1}{2}h_{2N+1}(-2N-1,\beta,c).$

###### Proof.

Let us consider the situation corresponding to (3.16), i.e., $\alpha=-2N-1$,
with $N\in\mathbb{N}$, $\beta\in\mathbb{C}$. We need to prove the the
properties of orthogonality $\langle Q_{n}^{(0)},Q_{m}^{(0)}\rangle_{1}=0$ for
all $m<n$, $\langle Q_{n}^{(0)},Q_{n}^{(0)}\rangle_{1}\neq 0$ for all $n$ in
order to prove that $(Q_{n}^{(0)})$ is a monic orthogonal polynomial sequence
with respect to $\langle,\rangle_{1}$. Let us prove this in three steps: (i)
$m\leq n<2N+1$; (ii) $m<2N+1\leq n$; (iii) $2N+1\leq n\leq m$. (i): If $m\leq
n<2N+1$ the coefficient $u_{k}$ of the recurrence relation is non-zero for
$k=0,1,\dots,2N$, so by the Favard result (see [4]), there exists a moment
linear functional, namely $\mathscr{L}_{0}$ such that the big $-1$ Jacobi
polynomials $(Q_{j}^{(0)})_{j=0}^{2N}$ are orthogonal with respect to it. Due
to the fact that $\tau_{\alpha}^{2N+1}(Q_{j})=0$ for $j=0,1,....,2N$,
therefore the property of orthogonality holds in this case, i.e.,

$\langle Q_{n}^{(0)},Q_{m}^{(0)}\rangle_{1}=\langle{\bf
u},Q_{n}^{(0)}Q_{m}^{(0)}\rangle=h_{n}(-2N-1,\beta,c)\delta_{n,m},$

where $\delta_{n,m}$ is the Kronecker (delta) symbol. (ii): If $m<2N+1\leq n$,
then $\mathscr{L}_{0}(Q^{(0)}_{n},Q^{(0)}_{m})=0$; at the same time
$\tau_{\alpha}^{2N+1}(Q_{j})=0$ for $j=0,1,....,2N$, therefore the property of
orthogonality holds in this case. (iii): If $2N+1\leq n\leq m$, then
$\mathscr{L}_{0}(Q^{(0)}_{n},Q^{(0)}_{m})=0$; and since

$\tau_{\alpha}^{2N+1}Q^{(0)}_{n}(x;-2N-1,\beta,c)=Q^{(0)}_{n-(2N+1)}(-x;2N+1,\beta,-c),$

is orthogonal with respect to the linear functional $\bf u$, one has

$\begin{split}\langle
Q_{n}^{(0)},Q_{m}^{(0)}\rangle_{1}=&\lambda_{N}(\beta)\langle{\bf
u},\tau_{\alpha}^{2N+1}Q^{(0)}_{n}(x;-2N-1,\beta,c)\tau_{\alpha}^{2N+1}Q^{(0)}_{m}(x;-2N-1,\beta,c)\rangle\\\\[-1.42262pt]
=&\lambda_{N}(\beta)\langle{\bf
u},Q^{(0)}_{n-2N-1}(-x;2N+1,\beta,-c)Q^{(0)}_{m-2N-1}(-x;2N+1,\beta,-c)=h_{n}(-2N-1,\beta,c)\delta_{n,m},\end{split}$

and the property of orthogonality holds in this situation. Moreover, since the
polynomials are monic, it is straightforward to check using the recurrence
relation (3.3) that

(3.18) $h_{n}=\langle{\bf
u},xQ^{(0)}_{n}(x)Q^{(0)}_{n-1}(x)\rangle=u_{n}\langle{\bf
u},xQ^{(0)}_{n-1}(x)Q^{(0)}_{n-1}(x)\rangle=u_{n}h_{n-1}.$

Furthermore, the norm squared (3.14) and the recurrence coefficient $u_{n}$
given in (3.3) satisfies (3.18). The situation which corresponds to (3.17),
i.e., $\beta=-2N-1$, with $N\in\mathbb{N}$, $\alpha\in\mathbb{C}$ is similar
so we will omit its proof. Hence the result holds. ∎

######  Remark 3.6.

Note that if $\alpha=-2N-1$ and $\beta\in\mathbb{C}$ and due to (3.9) one must
consider the moment linear form [10]

${\mathscr{L}_{0}}(p,q)=\sum_{j=0}^{N}\lambda_{j}\,p^{(j)}(1)\,q^{(j)}(1)+\sum_{j=0}^{N-1}\mu_{j}\,p^{(j)}(-1)\,q^{(j)}(-1),$

where $\lambda_{j},\mu_{j}$ are positive. So, taking into account (3.11), we
have

${\mathscr{L}}_{0}(Q^{(0)}_{j}(x),\pi(x))=0,\quad j\geq 2N+1,$

for any $\pi\in\mathbb{P}$. Moreover, by construction, if $0\leq j<2N+1$ one
has

$\langle
Q^{(0)}_{j}(x),Q^{(0)}_{j}(x)\rangle_{1}={\mathscr{L}}_{0}\left(Q^{(0)}_{j}(x)p,Q^{(0)}_{j}(x)\right)=\langle{\bf
u},Q_{j}^{(0)}Q_{j}^{(0)}\rangle=h_{j}\neq 0.$

If $j\geq 2N+1$ and taking into account Lemma 3.4 one has

$\begin{split}\langle
Q^{(0)}_{j}(x),Q^{(0)}_{j}(x)\rangle_{1}=&\lambda_{N}(\beta)\langle{\bf
u},Q_{j-2N-1}^{(0)}(-x;2N+1,\beta,-c)Q_{j-2N-1}^{(0)}(-x;2N+1,\beta,-c)\rangle\\\
=&\lambda_{N}(\beta)h_{j-2N-1}(2N+1,\beta,-c)=h_{j}(-2N-1,\beta,c).\end{split}$

Observe the last identity must be considered as the formal limit:

$\tfrac{1}{2}\lim_{\epsilon\to
0}h_{2N+1}(-2N-1+\epsilon,\beta,c)h_{j-2N-1}(2N+1+\epsilon,\beta,-c)=\lim_{\epsilon\to
0}h_{j}(-2N-1+\epsilon,\beta,c).$

For $\beta=-2N-1$, it is analogous since in (3.10) one must consider the
moment linear form [10]

${\mathscr{L}_{1}}(p,q)=\sum_{j=0}^{N}\kappa_{j}p^{(j)}(1)q^{(j)}(-c)+\sum_{j=0}^{N-1}\ell_{j}p^{(j)}(-1)q^{(j)}(c),$

where $\kappa_{j},\ell_{j}$ are positive. Moreover, in a similar way

$\dfrac{1}{2}\lim_{\epsilon\to
0}h_{2N+1}(\alpha,-2N-1+\epsilon,c)h_{j-2N-1}(\alpha,2N+1+\epsilon,-c)=\lim_{\epsilon\to
0}h_{j}(\alpha,-2N-1+\epsilon,c).$

##  4\. Conclusion and future work

We have studied the big $-1$ Jacobi polynomials, which are a $q\to-1$ limit of
the Bannai–Ito polynomials, when at least one of the parameters $\alpha$,
$\beta$, are non-standard and the coefficient $u_{n}$ of the three-term
recurrence relation (3.3) is equal to zero for a certain index, namely $N$,
i.e., $\alpha=-2N-1$ or $\beta=-2N-1$ where $N$ is a positive integer. We have
also obtained the Gauss hypergeometric representation, its factorization, and
the property of orthogonality using similar techniques that were used in [5].

In [6, Section 4], a similar procedure for the big $q$-Jacobi polynomials was
applied for non-standard parameters for which the $u_{n}$ coefficient of its
recurrence relation is equal to zero for some $n\in\mathbb{N}_{0}$. Along
these lines we are investigating the method for obtaining the orthogonality
property of the big $-1$ Jacobi polynomials for non-standard parameters
following a procedure analogous to the one followed by T. E. Pérez and M. A.
Piñar in [16] for the generalized Laguerre polynomials. In [6] the authors
also studied the $q$-Racah polynomials for non-standard parameters for which
the $u_{n}$ coefficient vanishes for some $n\in\mathbb{N}_{0}$ and since the
Bannai–Ito polynomials can be obtained from the $q$-Racah polynomials ([7,
§14.2]) by taking an analogous limit $q\to-1$ (see [20]), it makes sense to
perform a similar study for the Bannai–Ito polynomials. We are also working on
obtaining explicit hypergeometric representations, the factorization, and the
property of orthogonality for the Bannai–Ito polynomials and other families of
the continuous $-1$ hypergeometric orthogonal polynomial scheme for non-
standard parameters.

## References

  * [1] M. Alfaro, M. Á. de Morales, and M. L. Rezola, _Orthogonality of the Jacobi polynomials with negative integer parameters_ , J. Comput. Appl. Math. 145 (2002), no. 2, 379–386 (English). MR 1917278
  * [2] M. Alfaro, T. E. Pérez, M. A. Piñar, and M. L. Rezola, _Sobolev orthogonal polynomials: the discrete-continuous case_ , Methods Appl. Anal. 6 (1999), no. 4, 593–616 (English). MR 1795525
  * [3] M. Álvarez de Morales, T. E. Pérez, and M. A. Piñar, _Sobolev orthogonality for the Gegenbauer polynomials $\\{C_{n}^{(-N+1/2)}\\}_{n\geq 0}$_, J. Comput. Appl. Math. 100 (1998), no. 1, 111–120 (English). MR 1658734
  * [4] T. S. Chihara, _An Introduction to Orthogonal Polynomials_ , Gordon and Breach Science Publishers, New York-London-Paris, 1978, Mathematics and its Applications, Vol. 13, Reprinted by Dover Publications.
  * [5] R. S. Costas-Santos and J. F. Sánchez-Lara, _Extensions of discrete classical orthogonal polynomials beyond the orthogonality_ , Journal of Computational and Applied Mathematics 225 (2009), no. 2, 440–451. MR 2494714
  * [6] by same author, _Orthogonality of $q$-polynomials for non-standard parameters_, Journal of Approximation Theory 163 (2011), no. 9, 1246–1268. MR 2832754
  * [7] R. Koekoek, P. A. Lesky, and R. F. Swarttouw, _Hypergeometric orthogonal polynomials and their $q$-analogues_, Springer Monographs in Mathematics, Springer-Verlag, Berlin, 2010, With a foreword by Tom H. Koornwinder.
  * [8] A. B. J. Kuijlaars, A. Martinez-Finkelshtein, and R. Orive, _Orthogonality of Jacobi polynomials with general parameters_ , Electron. Trans. Numer. Anal. 19 (2005), 1–17. MR 2149265
  * [9] K. H. Kwon and L. L. Littlejohn, _The orthogonality of the Laguerre polynomials $\\{L_{n}^{-k}(x)\\}$ for positive integers $k$_, vol. 2, 1995, Special functions (Torino, 1993), pp. 289–303. MR 1343537
  * [10] S. G. Moreno and E. M. García-Caballero, _Linear interpolation and Sobolev orthogonality_ , J. Approx. Theory 161 (2009), no. 1, 35–48. MR 2558142
  * [11] by same author, _Non-standard orthogonality for the little $q$-Laguerre polynomials_, Appl. Math. Lett. 22 (2009), no. 11, 1745–1749 (English). MR 2560989
  * [12] by same author, _New orthogonality relations for the continuous and the discrete $q$-ultraspherical polynomials_, J. Math. Anal. Appl. 369 (2010), no. 1, 386–399 (English). MR 2643877
  * [13] by same author, _Non-classical orthogonality relations for big and little $q$-Jacobi polynomials_, J. Approx. Theory 162 (2010), no. 2, 303–322 (English). MR 2581384
  * [14] by same author, _Non-classical orthogonally relations for continuous $q$-Jacobi polynomials_, Taiwanese J. Math. 15 (2011), no. 4, 1677–1690 (English). MR 2848982
  * [15] J. Pelletier, L. Vinet, and A. Zhedanov, _Continuous $-1$ hypergeometric orthogonal polynomials_, arXiv:2209.10727 (2022), 47 pages.
  * [16] T. E. Pérez and M. A. Piñar, _On Sobolev orthogonality for the generalized Laguerre polynomials_ , J. Approx. Theory 86 (1996), no. 3, 278–285 (English). MR 1405981
  * [17] _NIST Digital Library of Mathematical Functions_ , https://dlmf.nist.gov/, Release 1.1.10 of 2023-06-15, F. W. J. Olver, A. B. Olde Daalhuis, D. W. Lozier, B. I. Schneider, R. F. Boisvert, C. W. Clark, B. R. Miller, B. V. Saunders, H. S. Cohl, and M. A. McClain, eds.
  * [18] J. F. Sánchez-Lara, _On the Sobolev orthogonality of classical orthogonal polynomials for non standard parameters_ , Rocky Mountain J. Math. 47 (2017), no. 1, 267–288 (English). MR 3619764
  * [19] L. Vinet and A. Zhedanov, _A ‘missing’ family of classical orthogonal polynomials_ , J. Phys. A 44 (2011), no. 8, 085201, 16. MR 2770369
  * [20] by same author, _A limit $q=-1$ for the big $q$-Jacobi polynomials_, Trans. Amer. Math. Soc. 364 (2012), no. 10, 5491–5507. MR 2931336