# Empirical and Full Bayes estimation of the type of a Pitman-Yor process

S.E.M.P. Franssen and A.W. van der Vaart

###### Abstract

The Pitman-Yor process is a random discrete probability distribution of which
the atoms can be used to model the relative abundance of species. The process
is indexed by a type parameter $\sigma$, which controls the number of
different species in a finite sample from a realization of the distribution. A
random sample of size $n$ from the Pitman-Yor process of type $\sigma>0$ will
contain of the order $n^{\sigma}$ distinct values (“species”). In this paper
we consider the estimation of the type parameter by both empirical Bayes and
full Bayes methods. We derive the asymptotic normality of the empirical Bayes
estimator and a Bernstein-von Mises theorem for the full Bayes posterior, in
the frequentist setup that the observations are a random sample from a given
true distribution. We also consider the estimation of the second parameter of
the Pitman-Yor process, the prior precision. We apply our results to derive
the limit behaviour of the likelihood ratio in a setting of forensic
statistics.

##  1 Introduction

The Pitman-Yor process [21, 17] is a random discrete probability distribution,
which can be used as a model for the relative abundance of species. It is
characterised by a _type_ parameter $\sigma$. Our main aim is statistical
inference on this type parameter.

The Pitman-Yor process of type $\sigma=0$ is the Dirichlet process [9], which
is well understood, while negative types correspond to finitely discrete
distributions and were considered in [5]. In this paper we concentrate on
Pitman-Yor processes of positive type. The Pitman-Yor process is also known as
the two-parameter Poisson-Dirichlet Process, is an example of a Poisson-
Kingman process [20], and a species sampling process of Gibbs type [6].

The easiest definition is through _stick-breaking_ ([17, 13]), as follows. The
family of nonnegative Pitman-Yor processes is given by three parameters: a
number $\sigma\in[0,1)$, a number $M>-\sigma$ and an atomless probability
distribution $G$ on some measurable space $({\cal X},{\cal A})$. We say that a
random probability measure $P$ on $({\cal X},{\cal A})$ is a Pitman-Yor
process (of nonnegative type), denoted $P\sim\mathop{\rm
PY}\nolimits\left(\sigma,M,G\right)$, if $P$ can be represented as

$P=\sum_{i=1}^{\infty}W_{i}\delta_{\theta_{i}},$

where $W_{i}=V_{i}\prod_{j=1}^{i-1}(1-V_{j})$ for $V_{i}$ independent
variables with $V_{i}\sim\mathop{\rm B}\nolimits(1-\sigma,M+i\sigma)$,
independent of
$\theta_{i}\,\raise-2.0pt\hbox{$\stackrel{{\scriptstyle\mbox{\scriptsize
iid}}}{{\sim}}$}\,G$.

It is clear from this definition that the realisations of $P$ are discrete
probability measures, with countably many atoms at random locations, with
random weights. If one first draws $P\sim\mathop{\rm
PY}\nolimits\left(\sigma,M,G\right)$, and next given $P$ a random sample
$X_{1},\ldots,X_{n}$ from $P$, then ties among the latter observations are
likely. Each tie represents an observed species. Each species is identified by
a label $\theta_{i}$, but this does not play an important role in the present
paper, except that labels are unique by the assumption that $G$ is atomless.
If the weights $(W_{i})$ correctly model the relative abundance of distinct
species, then the (conditional) probability that a next observation $X_{n+1}$
will be distinct from $X_{1},\ldots,X_{n}$ is indicative of the likelihood of
finding a new species given the past observations.

In a forensic setup one may consider two instances not present in a “database”
$X_{1},\ldots,X_{n}$ and compare the hypotheses that these two values are two
independent draws $X_{n+1},X_{n+2}$ from the population or copies of a single
draw $X_{n+1}$. If one of the new instances is a characteristic (e.g. DNA
profile) found at a crime scene and the other a characteristic of a suspect,
one can so derive the ratio of the probabilities that the characteristic at
the crime scene originates from the perpetrator (two copies of a single draw
$X_{n+1}$) or that the match is by chance (two draws $X_{n+1},X_{n+2}$ that
happen to be the same). See [3, 4] (and Section 2.3) for further discussion.

As shown in [4] such probabilities are readily calculable under the Pitman-Yor
model, but depend strongly on the type parameter $\sigma$. Thus it is
desirable to estimate this parameter from the observed data. In this paper we
consider both the empirical Bayes estimator (which is the maximum likelihood
estimator in the Bayesian setup with the Pitman-Yor process viewed as a prior)
and the full Bayes posterior of $\sigma$. We prove that the empirical Bayes
estimator is asymptotically normal, and show that the posterior distribution
satisfies a corresponding Bernstein-von Mises theorem. We apply these results
to the forensic problem.

The Pitman-Yor process is characterised by a second parameter $M$, called the
prior precision. As its name suggests, this is a prior modelling parameter,
and perhaps is better not estimated. We show that the asymptotics of the type
parameter are almost independent of the prior precision, and that the problem
of estimating the prior precision degenerates as $n\rightarrow\infty$.

Other applications of the Pitman-Yor process, in genetics or topic modelling,
can be found in [23, 22, 12, 1, 8]. The limiting case of the Pitman-Yor
process with $M\downarrow 0$ is a (particular) Poisson-Kingman process (see
[21], or Example 14.47 in [11]). Estimation of the type parameter for this
process was considered in [8], who give conditions so that the maximum
likelihood estimator converges to a limit at a certain rate. The present paper
generalizes and refines these results in the case of a general $M$ (by
providing a sharp rate and a limit distribution), and adds a full Bayes
analysis.

The posterior distribution when the Pitman-Yor process is used as a
nonparametric prior on the distribution of the observations was studied in
[14] for known type parameter and continuous observations, and for unknown
type and general observations in [10]. In the latter paper it is shown that
the posterior distribution is not very sensitive to the type parameter, and
its asymptotics could be established knowing just consistency for this
parameter, and hence without interception of the precise results of the
present paper.

##  2 Main results

We consider statistical procedures derived from the Bayesian model in which a
probability distribution $P$ is drawn from the Pitman-Yor process, viewed as a
prior over the set of probability measures, and next given $P$ the
observations $X_{1},\ldots,X_{n}$ are an i.i.d. sample from $P$. To estimate
the parameters $\sigma$ (or $(\sigma,M)$) of the Pitman-Yor process, we
consider the maximum likelihood estimator, based on the marginal distribution
of $(X_{1},\ldots,X_{n})$ in this setup, and a full Bayes approach. Because
the maximum likelihood estimator estimates a parameter of the prior, in this
setup this estimator is also called an empirical Bayes estimator. The full
Bayes approach adds a prior distribution over $\sigma$ (or $(\sigma,M)$) to
the Bayesian hierarchy and then uses the conditional distribution of $\sigma$
given $X_{1},\ldots,X_{n}$, the posterior distribution, for further inference.

While these two procedures are defined by the Bayesian setup, our theoretical
results obtained below are frequentist Bayes: we study these two procedures,
both functions of the observations $X_{1},\ldots,X_{n}$, under the assumption
that these observations are a random sample from a given distribution $P_{0}$.

Both statistical procedures are based on the Bayesian likelihood for
$X_{1},\ldots,X_{n}$. This can be conveniently obtained by considering the
exchangeable random partition of the set $\\{1,2,\ldots,n\\}$ generated by the
sample through the equivalence relation

$i\equiv j\qquad\qquad\text{ if and only if }\qquad\qquad X_{i}=X_{j}.$

An alternative way to generate $X_{1},\ldots,X_{n}$ is to generate first the
partition and next attach to each set in the partition a value generated
independently from the center measure $G$ (see e.g. [11], Lemma 14.11 for a
precise statement), duplicating this as many times as there are indices in the
set, in order to form the observations $X_{1},\ldots,X_{n}$. Because the
parameter $\sigma$ enters only in creating the partition, the partition is a
sufficient statistic for $\sigma$. Because of exchangeability, the vector
$(K_{n},N_{n,1},\ldots,N_{n,K_{n}})$ of the number $K_{n}$ of sets in the
partition and the cardinalities $N_{n,i}$ of the partitioning sets (i.e. the
multiplicity of $X_{i}$ in $X_{1},\ldots,X_{n}$) is already sufficient for
$(M,\sigma)$ and hence the empirical Bayes estimator and posterior
distribution of $(M,\sigma)$ are the same, whether based on observations
$(X_{1},\ldots,X_{n})$ or on observations
$(K_{n},N_{n,1},\ldots,N_{n,K_{n}})$.

The likelihood function for $(M,\sigma)$ is therefore equal to the probability
of a particular partition, called the _exchangeable partition probability
functio_ n (EPPF). For the Pitman-Yor process this is given by (see [19], or
[11, page 465])

$p_{\sigma}(N_{n,1},\ldots,N_{n,K_{n}})=\frac{\prod_{i=1}^{K_{n}-1}(M+i\sigma)}{(M+1)^{[n-1]}}\prod_{j=1}^{K_{n}}(1-\sigma)^{[N_{n,j}-1]}.$
(1)

Here $a^{[n]}=a(a+1)\cdots(a+n-1)$ is the ascending factorial, with
$a^{[0]}=1$ by convention.

Although we adopt the Pitman-Yor process as a prior for the distribution $P$
of the observations, and then arrive at the likelihood (1), we shall
investigate the maximum likelihood estimator and posterior distribution under
the frequentist-Bayes assumption that in reality the observations
$X_{1},\ldots,X_{n}$ are an i.i.d. sample from a given distribution $P_{0}$.
It turns out that the asymptotic properties of the maximizer of (1) are then
determined by the function $\alpha_{0}:(1,\infty)\to\mathbb{N}$ given by

$\alpha_{0}(u):=\\#\Bigl{\\{}x:\frac{1}{P_{0}\\{x\\}}\leq u\Bigr{\\}}.$ (2)

The function $\alpha_{0}$ is nondecreasing and increases by jumps of size 1.
Following [15], we shall assume that this function is regularly varying. The
paper [15] derived (distributional) limits of characteristics such as $K_{n}$.
In the present paper we use similar methods to analyse the likelihood function
(1).

Recall that a measurable function $\alpha:(0,\infty)\to\mathbb{R}_{+}$ is said
to be _regularly varying_ (at $\infty$) of order $\gamma$ if, for all $u>0$,
as $n\rightarrow\infty$,

$\frac{\alpha(nu)}{\alpha(n)}\rightarrow u^{\gamma}.$

It is known (see e.g. [2] or the appendix to [7]) that if the limit of the
sequence of quotients on the left exists for every $u$, then it necessarily
has the form $u^{\gamma}$, for some $\gamma$. If we write
$\alpha(u)=u^{\gamma}L(u)$, then $L$ will be _slowly varying_ : a function
that is regularly varying of order 0. Then $\alpha(n)=n^{\gamma}L(n)$, and it
can be shown that $n^{\gamma-\delta}\ll\alpha(n)\ll n^{\gamma+\delta}$, for
every $\delta>0$, so that the rate of growth of $\alpha$ is $n^{\gamma}$ to
“first order”.

Since the function $\alpha_{0}$ in (2) increases by steps, it is discontinuous
and so is the associated slowly varying function. We assume that there exists
$\sigma_{0}\in(0,1)$ and a continuously differentiable slowly varying function
$L_{0}$ such that, for every $u>1$,

$\displaystyle\bigl{|}\alpha_{0}(u)-u^{\sigma_{0}}L_{0}(u)\bigr{|}$
$\displaystyle\leq Cu^{\beta_{0}},$ $\displaystyle\text{for some
}\beta_{0}<\sigma_{0}\text{ and }C>0,$ (3)
$\displaystyle\bigl{|}L_{0}^{\prime}(u)\bigr{|}$ $\displaystyle\leq
C_{\delta}\,u^{-1+\delta},$ $\displaystyle\text{for any }\delta>0,\text{ for
some }C_{\delta}.$ (4)

If (3) holds for a slowly varying function $L_{0}$, then $\alpha_{0}$ is
regularly varying of order $\sigma_{0}$ (see Lemma 14). The rationale for (4)
is that a slowly varying function $u\mapsto L(u)$ will always grow slower than
any power $u^{\delta}$. It may not be differentiable, but if it is, then it is
reasonable that the derivative grows slower than any power of $u^{-1+\delta}$.
Condition (4) is satisfied, for instance, by powers $L_{0}(u)=(\log u)^{r}$ of
the logarithmic function, for $r\in\mathbb{R}$, and the functions
$L_{0}(u)=e^{(\log u)^{r}}$, for $r<1$. Although we assume (3)-(4) for $u>1$,
the bounds are asymptotic in the sense that if they hold for $u>U$ and some
$U$, then they are valid for $u>1$ and possibly larger constants $C$ and
$C_{\delta}$ (and some extension of $L_{0}$), since the right sides are
bounded away from zero on intervals $(1,U]$.

######  Example 1.

For the probability distribution $(p_{j})_{j\in\mathbb{N}}$ with
$p_{j}=c/j^{\alpha}$, for some $\alpha>1$, we have
$\alpha_{0}(u):=\\#(j:1/p_{j}\leq u)=\lfloor(cu)^{1/\alpha}\rfloor$. Then
$|\alpha_{0}(u)-(cu^{1/\alpha})|\leq 1$ and hence (3)-(4) are satisfied with
$\sigma_{0}=1/\alpha$, $L_{0}(u)=c^{1/\alpha}$, $C=1$, $\beta_{0}=0$, and
$C_{\delta}=0$.

######  Example 2.

In [8] it is assumed that $|\alpha_{0}(u)-Lu^{\sigma_{0}}|\leq
Cu^{\sigma_{0}/2}\sqrt{\log(eu)}$, for every $u>1$ and some constant $C$. This
implies (3)-(4) with $L_{0}$ constant and any $\beta_{0}$ slightly bigger than
$\sigma_{0}/2$ (and hence easily satisfying the restriction in (3)).

###  2.1 Estimating the type parameter

The following theorem shows that the empirical likelihood estimator is
asymptotically normal, after scaling by the rate $\sqrt{\alpha_{0}(n)}$ and
centering at the zero $\sigma_{0,n}$ of the function

$E_{0,n}(\sigma)=\int_{0}^{n}\alpha_{0}\Bigl{(}\frac{n}{s}\Bigr{)}e^{-s}\Bigl{(}\frac{1}{\sigma}-\sum_{m=1}^{\infty}\frac{s^{m}}{m!(m-\sigma)}\Bigr{)}\,ds.$
(5)

In Lemma 8 these zeros are shown to converge to the exponent $\sigma_{0}$ of
regular variation of $\alpha_{0}$, at a rate depending on the function
$\alpha_{0}$.

For the moment we keep $M$ fixed and let $\hat{\sigma}_{n}$ be the maximizer
of the likelihood (1) with respect to $\sigma$, for given $M$. The following
theorem shows that the asymptotic behaviour of $\hat{\sigma}_{n}$ is the same
for every $M$.

######  Theorem 1.

Assume that $P_{0}$ is discrete with atoms such that
$\alpha_{0}(u):=\\#\\{x:1/P_{0}\\{x\\}\leq u\\}$ satisfies (3)-(4) with
exponent $\sigma_{0}\in(0,1)$. Then the empirical Bayes estimator
$\hat{\sigma}_{n}$, the point of maximum of (1), satisfies
$\sqrt{\alpha_{0}(n)}(\hat{\sigma}_{n}-\sigma_{0,n})\rightsquigarrow
N(0,\tau_{1}^{2}/\tau_{2}^{4})$, where $\sigma_{0,n}$ are the roots of the
functions $E_{0,n}$ in (5) and $\tau_{1}$ is given in (12) and
$\tau_{2}^{2}=-E_{0}^{\prime}(\sigma_{0})$, for $E_{0}$ given in (9).

The proof of the theorem is deferred to Section 3.1.

The theorem centers the estimators at the zeros $\sigma_{0,n}$ of the
functions $E_{0,n}$ in (5). It is shown in Lemma 8 that these zeros tend to
the coefficient of regular variation $\sigma_{0}$ of $\alpha_{0}$, and hence
$\hat{\sigma}_{n}\rightarrow\sigma_{0}$, in probability. However, the rate of
this convergence may be too slow to replace $\sigma_{0,n}$ by $\sigma_{0}$ in
the centering of $\hat{\sigma}_{n}$. For the case that the function $L_{0}$ in
(3) can be taken constant, Lemma 8 shows that
$\sigma_{0,n}-\sigma_{0}=O(n^{-(\sigma_{0}-\beta)})$, for $\beta_{0}$ as in
(3), and hence if $\beta_{0}<\sigma_{0}/2$, then
$\sqrt{\alpha_{0}(n)}(\sigma_{0,n}-\sigma_{0})\rightarrow 0$, and hence also
$\sqrt{\alpha_{0}(n)}(\hat{\sigma}_{n}-\sigma_{0})\rightsquigarrow
N(0,\tau_{1}^{2}/\tau_{2}^{4})$. If $\alpha_{0}$ contains nontrivial slowly
varying terms, then the rate of convergence
$\sigma_{0,n}\rightarrow\sigma_{0}$ will typically be much slower than
$\alpha_{0}(n)^{-1/2}$ and the latter result will fail. (Lemma 8 gives the
rate $L_{0}^{\prime}(n)n/L_{0}(n)$, and its proof shows that this is sharp,
for instance: $1/\log n$ if $L_{0}(s)=\log s$.)

In general, we could say that the estimators $\hat{\sigma}_{n}$ roughly, but
possibly not quite, estimate the degree of regular variation of $\alpha_{0}$.
If one believes in the likelihood, then this is an indication that the type
parameter has a more subtle interpretation than the degree of regular
variation, rendering it extra worth while to use principled methods for its
estimation. (If interest were in the coefficient of regular variation
$\sigma_{0}$, then direct approaches may be preferable.)

In the special case considered in Example 2, the rate of the estimator is
$\sqrt{\alpha_{n}(n)}=n^{\sigma_{0}/2}$. This is a faster rate than obtained
(in the case that $M=0$) in [8], who showed that
$\hat{\sigma}_{n}=\sigma_{0}+O_{P}(n^{-{\sigma_{0}}/2}\sqrt{\log n})$ under
the condition in Example 2. (Our improved rate centers at $\sigma_{0,n}$; for
centering at $\sigma_{0}$, the condition of Example 2 must be slightly
strengthened to have an exponent strictly smaller than $\sigma_{0}/2$ rather
than $\sigma_{0}/2$.)

The asymptotic variance of the sequence
$\sqrt{\alpha_{0}(n)}(\hat{\sigma}_{n}-\sigma_{0,n})$ is a one-dimensional
form of the sandwich formula, which is clear if it is written as
$\tau_{2}^{-2}\tau_{1}^{2}\tau_{2}^{-2}$. It appears that in general
$\tau_{1}\not=\tau_{2}$, which is explainable by the fact that the likelihood
used to define $\hat{\sigma}_{n}$ is the Bayesian marginal likelihood, which
is misspecified relative to the frequentist distribution of
$X_{1},\ldots,X_{n}$: the likelihood behaves like a general contrast function
rather than a likelihood.

Next consider the posterior distribution of $\sigma$ given
$X_{1},\ldots,X_{n}$ in the model $\sigma\sim\Pi_{\sigma}$, $P\mathchar
25194\relax\sigma\sim\mathop{\rm PY}\nolimits\left(\sigma,M,G\right)$ and
$X_{1},\ldots,X_{n}\mathchar 25194\relax P,\sigma\sim P$, for a given prior
distribution $\Pi_{\sigma}$ on $(0,1)$. Since the likelihood for observing
$X_{1},\ldots,X_{n}$ is proportional to (1), by Bayes rule the posterior
distribution has density relative to $\Pi_{\sigma}$ proportional to (1). We
study the posterior distribution under the assumption that
$X_{1},\ldots,X_{n}$ are an i.i.d. sample from a distribution $P_{0}$.

######  Theorem 2.

Assume that $P_{0}$ is discrete with atoms such that
$\alpha_{0}(u):=\\#\\{x:1/P_{0}\\{x\\}\leq u\\}$ satisfies (3)-(4) with
exponent $\sigma_{0}\in(0,1)$. For a prior distribution $\Pi_{\sigma}$ on
$\sigma\in(0,1)$ with a bounded density that is positive and continuous at
$\sigma_{0}$, the posterior distribution of $\sigma$ satisfies

$\sup_{B}\Bigl{|}\Pi_{n}(\sigma\in B\mathchar 25194\relax
X_{1},\ldots,X_{n})-N\Bigl{(}\hat{\sigma}_{n},\frac{1}{\alpha_{0}(n)\tau_{2}^{2}}\Bigr{)}(B)\Bigr{|}\rightarrow
0,$

where the supremum is taken over all Borel sets $B$ in $(0,1)$, and
$\tau_{2}^{2}=-E_{0}^{\prime}(\sigma_{0})$, for $E_{0}$ given in (9). In
particular, the posterior distribution for $\sigma$ contracts to $\sigma_{0}$.
Furthermore, the posterior mean $\tilde{\sigma}_{n}=\mathbb{E}(\sigma\mathchar
25194\relax X_{1},\ldots,X_{n})$ satisfies
$\sqrt{\alpha_{0}(n)}(\tilde{\sigma}_{n}-\hat{\sigma}_{n})\rightarrow 0$, in
probability.

The proof Theorem 2 is deferred to Section 3.2.

Apart from the unusual scaling rate $\alpha_{0}(n)$, Theorem 2 is of the
Bernstein-von Mises type, for a misspecified model (see [16]).
Misspecification arises, because the likelihood corresponds to the Bayesian
model, but the observations are sampled from $P_{0}$.

###  2.2 Estimating the precision

The parameter $M$ is commonly referred to as the _prior precision_. This name
suggests that this is truly a prior modelling parameter and estimating it from
the data may be illogical. Theorem 1 shows that the maximum likelihood
estimator $\hat{\sigma}_{n,M}$ of $\sigma$, for a given $M$, satisfies that
the sequence $\sqrt{\alpha_{0}(n)}(\hat{\sigma}_{n,M}-\sigma_{n,0})$ tends to
a centered normal distribution, for every $M$, where the limit is independent
of $M$. Inspection of the proof shows that $\hat{\sigma}_{n,M_{n}}$, for a
sequence $M_{n}$, has the same behaviour, as long as
$M_{n}\ll\sqrt{\alpha_{0}(n)}/\log n$. Furthermore, if $M$ is equipped with a
prior over a compact (or slowly increasing) interval, then the posterior
distribution of $\sigma$ still satisfies the assertion of Theorem 2, where the
limit does not involve $M$. Thus for a very wide range of prior precisions,
the estimators for the type parameter are asymptotically equivalent. This may
again suggest that the parameter $M$ plays a different role than the type
parameter.

Nevertheless, we might use the likelihood function to obtain also a maximum
likelihood or Bayes estimator for $M$. In the following theorem we consider
the maximum likelihood estimator, where the parameter $M$ is restricted to a
compact set $[0,\bar{M}]$. (The proof extends to $\bar{M}=\bar{M}_{n}$ that
increase not too fast to infinity.)

The limiting value $M_{0}$ of the maximum likelihood estimator $\hat{M}_{n}$
depends on the fine details of the regular variation of the function
$\alpha_{0}$ in (3)-(4), through the limit (assumed to exist)

$K_{0}:=\lim_{n\rightarrow\infty}\Bigl{[}\frac{c_{0}L_{0}^{\prime}(n)n\log
n}{L_{0}(n)}+\log L_{0}(n)\Bigr{]},$

where $c_{0}=\Gamma(1-\sigma_{0})(1+\sigma_{0})/(\sigma_{0}\tau_{2}^{2})$.
Define $M_{0}=0$ or $M_{0}=\bar{M}$ if the limit is $K_{0}=\infty$ or
$K_{0}=-\infty$, respectively, and otherwise, set it equal to the maximizer in
$[0,\bar{M}]$ of the function

$M\mapsto\frac{M}{\sigma_{0}}\bigl{(}K_{0}+\log\Gamma(1-\sigma_{0})\bigr{)}+\log\Gamma(1+M)-\log\Gamma\Bigl{(}1+\frac{M}{\sigma_{0}}\Bigr{)}.$

######  Theorem 3.

Assume that $P_{0}$ is discrete with atoms such that
$\alpha_{0}(u):=\\#\\{x:1/P_{0}\\{x\\}\leq u\\}$ satisfies (3)-(4) with
exponent $\sigma_{0}\in(0,1)$ and a function $L_{0}$ such that $u\mapsto
L_{0}^{\prime}(u)u$ is slowly varying. Then the joint maximum likelihood
estimator $(\hat{M}_{n},\hat{\sigma}_{n})$ satisfies $\hat{M}_{n}\rightarrow
M_{0}$ in probability and $\hat{\sigma}_{n}$ has identical behaviour as in
Theorem 1.

The proof of the theorem is deferred to Section 3.3.

######  Example 3.

If $L_{0}$ is constant, then $K_{0}=\log L_{0}$ is finite, and it can be
arbitrary large or small, depending on the nonasymptotic properties of the
sequence $P_{0}\\{x_{j}\\}$. For instance, the choices $p_{j}=c/j^{\alpha}$
for every $j>J$, and $p_{j}=\eta$, for $j\leq J$, are possible for every
constants $c>0$, $\alpha>1$, $J\in\mathbb{N}$, and $\eta\in(0,1)$ such that
$J\eta+\sum_{j>J}c/j^{\alpha}=1$. Then
$\alpha_{0}(u)=J+\lfloor(cu)^{1/\alpha}-J\rfloor=\lfloor(cu)^{1/\alpha}\rfloor$,
for $u>1/\eta$, and hence $L_{0}=c^{1/\alpha}$, as (3) is determined by
$u\rightarrow\infty$. Depending on the constants $c$ and $\alpha$, the
constant $M_{0}$ can be any value in $[0,\bar{M}]$.

######  Example 4.

If $L_{0}(u)=\log u$, then $K_{0}=\infty$, and hence $M_{0}=\bar{M}$. If
$L_{0}(u)=1/\log u$, then $K_{0}=-\infty$, and hence $M_{0}=0$.

###  2.3 Forensic application

Consider again the Bayesian model in which $X_{1},\ldots,X_{n}$ are drawn
independently from a distribution $P$ generated from the Pitman-Yor process. A
next observation $X_{n+1}$ will either be equal to one of the current
observations $X_{1},\ldots,X_{n}$ or constitute a new type. If
$\tilde{X}_{1},\ldots,\tilde{X}_{K_{N}}$ are the distinct values in
$X_{1},\ldots,X_{n}$ and $N_{n,1},\ldots,N_{n,K_{n}}$ are the multiplicities
of these values in the sample, then it is known that (see [18, 20], or [11],
page 465)

$\Pr(X_{n+1}=\tilde{X}_{i}\mathchar 25194\relax
X_{1},\ldots,X_{n},\sigma,M)=\frac{N_{n,i}-\sigma}{M+n},\qquad
i=1,\ldots,K_{n}.$ (6)

The remaining mass $(M+K_{n}\sigma)/(M+n)$ is the probability of obtaining a
new species, distinct from $\tilde{X}_{1},\ldots,\tilde{X}_{K_{N}}$. The
probability distribution defined by these numbers is known as the _prediction
probability function_.

In the forensic setup discussed in [3, 4], the sample $X_{1},\ldots,X_{n}$
represents a database of characteristics of individuals (say DNA profiles),
and given is a new profile, not present in the database, that has been found
both at the crime scene and on a defendant who has been charged with the
crime. The prosecution argues that the defendant is the perpetrator who left
her profile on the crime scene and hence only a single new observation
$X_{n+1}$ is involved. The defence argues that the perpetrator and the
defendant are two different individuals, who happen to have the same profile,
and hence two independent observations $X_{n+1},X_{n+2}$ are involved, which
were observed to take the same value. The two hypotheses can be made precise
in a Bayesian hierarchy describing a generative model for the database
$X_{1},\ldots,X_{n}$, the profile $X_{n+1}$ found at the crime scene and the
profile $Y$ found on the suspect. According to the prosecution the generative
model is:

  * (i)

$(\sigma,M)\sim\Pi_{\sigma,M}$.

  * (ii)

$P\mathchar 25194\relax(\sigma,M)\sim\text{Pitman-Yor }(\sigma,M)$.

  * (iii)

$X_{1},\ldots,X_{n+1}\mathchar 25194\relax
P,\sigma,M\,\raise-2.0pt\hbox{$\stackrel{{\scriptstyle\mbox{\scriptsize
iid}}}{{\sim}}$}\,P$.

  * (p)

$Y\mathchar 25194\relax X_{1},\ldots,X_{n+1},P,\sigma,M\sim\delta_{X_{n+1}}$.

The fourth step (p) expresses that the profile $Y$ found on the defendant is
identical to the profile found on the crime scene, because it results from the
same individual $X_{n+1}$ chosen from the population. The defence agrees with
steps (i)-(iii) of the hierarchy, but replaces (p) by:

  * (d)

$Y\mathchar 25194\relax X_{1},\ldots,X_{n+1},P,\sigma,M\sim P$.

This expresses that the defendant’s profile is just another draw $X_{n+2}$
from the population. In the observed data the value of this draw happens to be
the same as the profile $X_{n+1}$ at the crime scene.

To decide on the case we might evaluate the ratio of the likelihoods of the
full evidence $X_{1},\ldots,X_{n+1},Y$ under the two hypotheses. Since the
(marginal) likelihood of $X_{1},\ldots,X_{n+1}$ is determined by (i)–(iii), it
is the same under both hypotheses. Hence the relative likelihood is the ratio
of the conditional likelihoods of $Y$ given $X_{1},\ldots,X_{n+1}$. For the
prosecution $Y$ depends deterministically on $X_{1},\ldots,X_{n+1}$ (it must
be equal to $X_{n+1}$) and hence the conditional probability of the observed
value is equal to 1. For the defence the conditional likelihood of $Y$ is the
probability that an $(n+2)$th observation $X_{n+2}$ happens to be of the same
species as $X_{n+1}$. Since $X_{n+1}$ has multiplicity $N_{n+1,K_{n+1}}=1$
among $X_{1},\ldots,X_{n+1}$ (by our assumption on the observed data), given
$(\sigma,M)$ this probability is $(1-\sigma)/(M+n+1)$, as determined by the
prediction probability function (6). The unconditional probability is the
integral of this relative to the posterior distribution of $(\sigma,M)$, i.e.
$\mathbb{E}\bigl{(}(1-\sigma)/(M+n+1)\mathchar 25194\relax
X_{1},\ldots,X_{n+1}\bigr{)}$. The likelihood ratio of prosecution versus
defence is therefore

$1\ \Big{/}\ \ \mathbb{E}\Bigl{(}\frac{1-\sigma}{n+1+M}\mathchar 25194\relax
X_{1},\ldots,X_{n+1}\Bigr{)}.$

######  Theorem 4.

Under the assumptions of Theorem 2, the posterior distribution of
$\phi=(1-\sigma)/(M+n+1)$ satisfies

$\sup_{B}\Bigl{|}\Pi_{n}(\phi\in B\mathchar 25194\relax
X_{1},\ldots,X_{n})-N\Bigl{(}\frac{1-\hat{\sigma}_{n}}{n+M+1},\frac{1}{(n+M+1)^{2}\alpha_{0}(n)\tau_{2}^{2}}\Bigr{)}(B)\Bigr{|}\rightarrow
0.$

Moreover, the posterior mean $\tilde{\phi}_{n}=\mathbb{E}\bigl{(}\phi\mathchar
25194\relax X_{1},\ldots,X_{n+1}\bigr{)}$ satisfies

$\sqrt{\alpha_{n}(n)}\Bigl{(}\frac{1}{n\tilde{\phi}_{n}}-\frac{1}{1-\sigma_{n,0}}\Bigr{)}\rightsquigarrow
N\Bigl{(}0,\frac{\tau_{1}^{2}}{(1-\sigma_{0})^{4}\tau_{2}^{4}}\Bigr{)}.$

These assertions remain true if $M$ is equipped with a prior supported on a
compact interval in $[0,\infty)$.

###### Proof.

The first assertion is immediate from Theorem 2 and the definition of $\phi$.
For the second assertion we note that

$\frac{1}{n\tilde{\phi}_{n}}-\frac{1}{1-\sigma_{0,n}}==\frac{\tilde{\sigma}_{n}-\sigma_{0,n}+(1-\tilde{\sigma}_{n})(M+1)/(M+n+1)}{(1-\tilde{\sigma}_{n})/(M+n+1)}.$

By Theorem 2 the sequence
$\sqrt{\alpha_{0}(n)}(\tilde{\sigma}_{n}-\sigma_{n,0})$ is asymptotically
equivalent to the sequence
$\sqrt{\alpha_{0}(n)}(\hat{\sigma}_{n}-\sigma_{0,n})$, which tends to the
$N(0,\tau_{1}^{2}/\tau_{2}^{4})$-distribution, by Theorem 1. The second
assertion follows by Slutzky’s lemma. ∎

##  3 Proofs

The logarithm of the likelihood (1) can be written

$\displaystyle\Lambda_{n}(\sigma,M)$
$\displaystyle=\sum_{l=1}^{K_{n}-1}\log(M+l\sigma)+\\!\\!\sum_{j=1:N_{n,j}\geq
2}^{K_{n}}\\!\\!\sum_{l=1}^{N_{n,j}-1}\log(l-\sigma)-\sum_{i=1}^{n-1}\log(M+i)$
$\displaystyle=\sum_{l=1}^{K_{n}-1}\log(M+l\sigma)+\sum_{l=1}^{n-1}\log(l-\sigma)Z_{n,l+1}-\sum_{i=1}^{n-1}\log(M+i),$
(7)

where $Z_{n,l}=\\#(1\leq j\leq K_{n}:N_{n,j}\geq l)$ is the number of distinct
values of multiplicity at least $l$ in the sample $X_{1},\ldots,X_{n}$. (In
the case that all observations are distinct and hence $N_{n,j}=1$ for every
$j$, the second term of the likelihood is equal to 0.)

For the proofs of Theorems 1 and 2, we fix the argument $M$ and drop it from
the notation. The concavity of the logarithm shows that the log likelihood is
a strictly concave function of $\sigma$. For $\sigma\downarrow 0$, it tends to
a finite value if $M>0$ and to $-\infty$ if $M=0$, while for $\sigma\uparrow
1$ it tends to $-\infty$ if the term with $l=1$ is present in the second sum,
i.e. if there is at least one tied observation. This happens with probability
tending to 1 as $n\rightarrow\infty$. The derivative of the log likelihood
with respect to $\sigma$ is equal to

$\Lambda_{n}^{\prime}(\sigma)=\sum_{l=1}^{K_{n}-1}\frac{l}{M+l\sigma}-\sum_{l=1}^{n-1}\frac{Z_{n,l+1}}{l-\sigma}.$
(8)

The left limit at $\sigma=0$ is
$\Lambda_{n}^{\prime}(0)=\frac{1}{2}K_{n}(K_{n}-1)/M-\sum_{l=1}^{n-1}l^{-1}Z_{n,l+1}$.
Since $Z_{n,l}\leq Z_{n,1}=K_{n}$, a crude bound on the sum is $K_{n}\log n$,
which shows that the derivative at $\sigma=0$ tends to infinity if
$K_{n}\gg\log n$. In that case the unique maximum of the log likelihood in
$[0,1]$ is taken in the interior of the interval, and hence $\hat{\sigma}_{n}$
satisfies $\Lambda_{n}^{\prime}(\hat{\sigma}_{n})=0$.

Set $\alpha_{n}=\alpha_{0}(n)$. Under the condition that $\alpha_{0}$ is
regularly varying of exponent $\sigma_{0}\in(0,1)$, the sequence $\alpha_{n}$
is of the order $n^{\sigma_{0}}$ up to slowly varying terms. By Theorems 9 and
1‘ of [15], the sequence $K_{n}/\alpha_{n}$ tends almost surely to
$\Gamma(1-\sigma_{0})$ and hence in particular $K_{n}\gg\log n$, and the
conclusion of the preceding paragraph that
$\Lambda_{n}^{\prime}(\hat{\sigma}_{n})=0$ pertains.

By Lemma 8, the functions $E_{0,n}/\alpha_{n}$, for $E_{0,n}$ defined in (5),
converge to the function $E_{0}$ defined by

$E_{0}(\sigma)=\frac{\Gamma(1-\sigma_{0})}{\sigma}-\sum_{m=1}^{\infty}\frac{\Gamma(m+1-\sigma_{0})}{m!(m-\sigma)},$
(9)

and this function vanishes at $\sigma=\sigma_{0}$. By monotonicity of these
functions, the zeros $\sigma_{0,n}$ of the functions $E_{0,n}$ tend to the
zero $\sigma_{0}\in(0,1)$ of the limit function.

###  3.1 Proof of Theorem 1

The monotonicity of $\Lambda_{n}^{\prime}$, the definition of
$\hat{\sigma}_{n}$ and the fact that $-E_{0,n}(\sigma_{0,n})=0$ give that

$\displaystyle\Pr\bigl{(}\sqrt{\alpha_{n}}(\hat{\sigma}_{n}-\sigma_{0,n})\leq
x\bigr{)}=\Pr\Bigl{(}\Lambda_{n}^{\prime}\Bigl{(}\sigma_{0,n}+\frac{x}{\sqrt{\alpha_{n}}}\Bigr{)}\leq
0\Bigr{)}$
$\displaystyle\qquad=\Pr\biggl{(}\frac{1}{\sqrt{\alpha_{n}}}\Bigl{[}\Lambda_{n}^{\prime}\Bigl{(}\sigma_{0,n}+\frac{x}{\sqrt{\alpha_{n}}}\Bigr{)}-E_{0,n}\Bigl{(}\sigma_{0,n}+\frac{x}{\sqrt{\alpha_{n}}}\Bigr{)}\Bigr{]}$
$\displaystyle\qquad\qquad\qquad\qquad\qquad\qquad\leq-\frac{1}{\sqrt{\alpha_{n}}}\Bigl{[}E_{0,n}\Bigl{(}\sigma_{0,n}+\frac{x}{\sqrt{\alpha_{n}}}\Bigr{)}-E_{0,n}(\sigma_{0,n})\Bigr{]}\biggr{)}.$

The variables in the left side of the last probability are asymptotically
normal by Lemma 5, while by the mean value theorem the numbers on the right
side of the inequality are equal to
$-xE_{0,n}^{\prime}(\sigma_{n})/\alpha_{n}$ for numbers $\sigma_{n}$ between
$\sigma_{0,n}$ and $\sigma_{0,n}+x/\sqrt{\alpha_{n}}$ and hence
$\sigma_{n}\rightarrow\sigma_{0}$. Thus
$-xE_{0,n}^{\prime}(\sigma_{n})/\alpha_{n}\rightarrow x\tau_{2}^{2}$, by Lemma
8. The asymptotic normality of $\hat{\sigma}_{n}$ follows.

###  3.2 Proof of Theorem 2

We first prove that the posterior distribution is
$\sqrt{\alpha_{n}}$-consistent:
$\Pi_{n}\bigl{(}\sqrt{\alpha_{n}}|\sigma-\sigma_{0,n}|>m_{n}\mathchar
25194\relax X_{1},\ldots,X_{n}\bigr{)}\rightarrow 0$ in probability for any
$m_{n}\rightarrow\infty$. By the monotonicity of $\Lambda_{n}^{\prime}$ and
the fact that $\Lambda_{n}^{\prime}(\hat{\sigma}_{n})=0$, for given
$\sigma_{n}>\hat{\sigma}_{n}$,

$\displaystyle\ \Lambda_{n}(\sigma)$
$\displaystyle\geq\Lambda_{n}(\sigma_{n}),$ $\displaystyle\text{ if
}\hat{\sigma}_{n}<\sigma<\sigma_{n},$ $\displaystyle\Lambda_{n}(\sigma)$
$\displaystyle\leq\Lambda_{n}(\sigma_{n})+\Lambda_{n}^{\prime}(\sigma_{n})(\sigma-\sigma_{n}),$
$\displaystyle\text{ if }\sigma>\sigma_{n}.$

It follows that

$\displaystyle\Pi_{n}\bigl{(}\sigma>\sigma_{n}\mathchar 25194\relax
X_{1},\ldots,X_{n}\bigr{)}$
$\displaystyle=\frac{\int_{\sigma_{n}}^{1}e^{\Lambda_{n}(\sigma)}\,d\Pi_{\sigma}(\sigma)}{\int_{0}^{1}e^{\Lambda_{n}(\sigma)}\,d\Pi_{\sigma}(\sigma)}$
$\displaystyle\leq\frac{\int_{\sigma_{n}}^{1}e^{\Lambda_{n}(\sigma_{n})+\Lambda_{n}^{\prime}(\sigma_{n})(\sigma-\sigma_{n})}\,d\Pi_{\sigma}(\sigma)}{\int_{\hat{\sigma}_{n}}^{\sigma_{n}}e^{\Lambda_{n}(\sigma_{n})}\,d\Pi_{\sigma}(\sigma)}$
$\displaystyle\lesssim\frac{\int_{0}^{\infty}e^{\Lambda_{n}^{\prime}(\sigma_{n})u}\,du}{\sigma_{n}-\hat{\sigma}_{n}}=\frac{1}{-\Lambda_{n}^{\prime}(\sigma_{n})(\sigma_{n}-\hat{\sigma}_{n})},$

where the proportionality constant depends on the density of $\Pi_{\sigma}$
only. If we choose $\sigma_{n}=\sigma_{0,n}+x/\sqrt{\alpha_{n}}$, then, by
Lemmas 5 and 7, since $E_{0,n}(\sigma_{0,n})=0$,

$\frac{\Lambda_{n}^{\prime}(\sigma_{n})}{\sqrt{\alpha_{n}}}=O_{P}(1)+\frac{E_{0,n}(\sigma_{n})}{\sqrt{\alpha_{n}}}=O_{P}(1)+\frac{E_{0,n}^{\prime}(\tilde{\sigma}_{n})}{\alpha_{n}}x=O_{P}(1)-\tau_{2}^{2}x.$

Theorem 1 gives that
$\sqrt{\alpha_{n}}(\sigma_{n}-\hat{\sigma}_{n})=x+O_{p}(1)$, and hence the
probability that
$-\Lambda_{n}^{\prime}(\sigma_{n})(\sigma_{n}-\hat{\sigma}_{n})$ is bigger
than some fixed constant can be made arbitrarily large by choosing large
enough $x$. This shows that the preceding display tends to zero in probability
for $\sigma_{n}=\sigma_{0,n}+m_{n}/\sqrt{\alpha_{n}}$ and any
$m_{n}\rightarrow\infty$. The probability of the events
$\sigma_{n}>\hat{\sigma}_{n}$, on which the preceding displays are valid, then
also tends to one. Combined with a similar argument on the left tail of the
posterior distribution, this shows that the posterior contracts to
$\sigma_{0,n}$ at rate $1/\sqrt{\alpha_{n}}$.

Since $\sqrt{\alpha_{n}}(\hat{\sigma}_{n}-\sigma_{0,n})=O_{p}(1)$, by Theorem
1, there exists $m_{n}\rightarrow\infty$ so that the sets
$C_{n}:=\\{\sigma:\sqrt{\alpha_{n}}|\sigma-\hat{\sigma}_{n}|\leq m_{n}\\}$
have posterior probability tending to one. The total variation measure between
the posterior measure $\Pi_{n}(\sigma\in\cdot\mathchar 25194\relax
X_{1},\ldots,X_{n})$ and the conditioned posterior measure
$\Pi_{n}(\sigma\in\cdot\mathchar 25194\relax X_{1},\ldots,X_{n},\sigma\in
C_{n})$ is bounded above by $2\Pi_{n}(\sigma\notin C_{n}\mathchar 25194\relax
X_{1},\ldots,X_{n})$ and hence tends to zero in probability. Thus it suffices
to prove the Gaussian approximation to the conditioned posterior measure.

By Lemma 6 and the fact that $\Lambda_{n}^{\prime}(\hat{\sigma}_{n})=0$, a
second order Taylor expansion gives

$\sup_{\sigma\in
C_{n}}\Bigl{|}\frac{\Lambda_{n}(\sigma)-\Lambda_{n}(\hat{\sigma}_{n})}{\alpha_{n}(\sigma-\hat{\sigma}_{n})^{2}}+\frac{1}{2}\tau_{2}^{2}\Bigr{|}\,\hbox{$\stackrel{{\scriptstyle\mbox{\scriptsize
P}}}{{\rightarrow}}$}\,0.$

We conclude that there exist random variables $\epsilon_{n}$ that tend to zero
in probability with, for every $\sigma\in C_{n}$,

$-\frac{1}{2}(\sigma-\hat{\sigma}_{n})^{2}\alpha_{n}(\tau_{2}^{2}+\epsilon_{n})\leq\Lambda_{n}(\sigma)-\Lambda_{n}(\hat{\sigma}_{n})\leq-\frac{1}{2}(\sigma-\hat{\sigma}_{n})^{2}\alpha_{n}(\tau_{2}^{2}-\epsilon_{n}).$

Then, for $\pi_{\sigma}$ a density of the prior measure $\Pi_{\sigma}$,

$\displaystyle\Pi_{n}\bigl{(}\sigma\in B\mathchar 25194\relax
X_{1},\ldots,X_{n},\sigma\in C_{n}\bigr{)}$
$\displaystyle\leq\frac{\int_{B\cap
C_{n}}e^{-(\sigma-\hat{\sigma}_{n})^{2}\alpha_{n}(\tau_{2}^{2}-\epsilon_{n})/2}\,d\sigma}{\int_{C_{n}}e^{-(\sigma-\hat{\sigma}_{n})^{2}\alpha_{n}(\tau_{2}^{2}+\epsilon_{n})/2}\,d\sigma}\frac{\displaystyle{\sup_{\sigma\in
C_{n}}}\pi_{\sigma}(\sigma)}{\displaystyle{\inf_{\sigma\in
C_{n}}}\pi_{\sigma}(\sigma)}$

By changing variables we see that
$\Pi_{n}\bigl{(}\sqrt{\alpha_{n}}(\sigma-\hat{\sigma}_{n})\in B\mathchar
25194\relax X_{1},\ldots,X_{n},\sigma\in C_{n})$ can be bounded above by

$\displaystyle\frac{\int_{-m_{n}}^{m_{n}}1_{B}(s)e^{-s^{2}(\tau_{2}^{2}-\epsilon_{n})/2}\,ds}{\int_{-m_{n}}^{m_{n}}e^{-s^{2}(\tau_{2}^{2}+\epsilon_{n})/2}\,ds}(1+o_{P}(1))$
$\displaystyle\qquad\qquad=\frac{\Pr\bigl{(}Z/\sqrt{\tau_{2}^{2}-\epsilon_{n}}\in
B\cap(-m_{n},m_{n})\bigr{)}}{\Pr\bigl{(}Z/\sqrt{\tau_{2}^{2}+\epsilon_{n}}\in(-m_{n},m_{n})\bigr{)}}(1+o_{P}(1)),$

for $Z$ a standard normal variable and the probabilities understood to refer
to $Z$ only. This tends in probability to $\Pr(Z/\tau_{2}\in B)$, uniformly in
$B$.

By the same method, switching $+$ and $-$ signs and $\sup$ and $\inf$, we can
derive the same expression as an asymptotic lower bound. This concludes the
proof of the first assertion of Theorem 2.

Because convergence in total variation norm implies convergence of the
expectations of bounded, measurable functions, to prove the convergence of the
posterior mean, it suffices to show that

$\int_{\sqrt{\alpha_{n}}(\sigma-\hat{\sigma}_{n})>m_{n}}\sqrt{\alpha_{n}}|\sigma-\hat{\sigma}_{n}|\,d\Pi_{n}(\sigma\mathchar
25194\relax X_{1},\ldots,X_{n})\rightarrow 0,$

in probability, for every $m_{n}\rightarrow\infty$, combined with a similar
estimate on the left tail. By the argument at the beginning of the proof this
expectation is bounded above by, for
$\sigma_{n}=\hat{\sigma}_{n}+x/\sqrt{\alpha_{n}}$ with $x>0$,

$\displaystyle\frac{\int_{\sqrt{\alpha_{n}}(\sigma-\hat{\sigma}_{n})>m_{n}}\sqrt{\alpha_{n}}(\sigma-\hat{\sigma}_{n})e^{\Lambda_{n}^{\prime}(\sigma_{n})(\sigma-\sigma_{n})}\,d\Pi_{\sigma}(\sigma)}{\sigma_{n}-\hat{\sigma}_{n}}$
$\displaystyle\qquad\qquad\qquad\qquad=\frac{\int_{m_{n}-x}^{\infty}(u+x)e^{\Lambda_{n}^{\prime}(\sigma_{n})u/\sqrt{\alpha_{n}}}\,du}{x}$

Because
$\Lambda_{n}^{\prime}(\sigma_{n})/\sqrt{\alpha_{n}}=\sqrt{\alpha_{n}}(\sigma_{n}-\sigma_{0,n})\Lambda_{n}^{\prime\prime}(\tilde{\sigma}_{n})/\alpha_{n}=(x+O_{P}(1))(-\tau_{2}^{2}+o_{P}(1))$,
the exponential in the integrand is with probability arbitrarily close to 1
bounded above by $e^{-cu}$, for some $c>0$, if $x$ is chosen large enough. On
the event where this is the case the integral is bounded above by a multiple
of $\int_{m_{n}-x}^{\infty}(u+x)e^{-cu}\,du\rightarrow 0$, as
$n\rightarrow\infty$. It follows that the right side tends to zero in
probability.

###  3.3 Proof of Theorem 3

The maximum likelihood estimator of $M$ can be obtained in two steps: first we
maximize the log likelihood (7) over $\sigma$ for fixed $M$, yielding
$\hat{\sigma}_{n,M}$, and next we maximize the “profile log likelihood”
$M\mapsto\Lambda_{n}(\hat{\sigma}_{n,M},M)$. From Theorem 1 we know that
$\hat{\sigma}_{n,M}$ will be contained in a neighbourhood of $\sigma_{0}$,
with probability tending to 1. To proceed further, we first obtain a
stochastic expansion of $\hat{\sigma}_{n,M}$ that refines this result.

The estimator $\hat{\sigma}_{n,M}$ solves $\Lambda_{n}^{\prime}(\sigma,M)=0$,
where the prime means the partial derivative with respect to $\sigma$. The
derivative can be written

$\displaystyle\Lambda_{n}^{\prime}(\sigma,M)$
$\displaystyle=\sum_{i=1}^{K_{n}-1}\frac{i}{M+i\sigma}-\sum_{l=1}^{n-1}\frac{1}{l-\sigma}Z_{n,l+1}$
$\displaystyle=\frac{K_{n}}{\sigma}-G_{n}(\sigma)-\frac{h_{\sigma,M}(K_{n})}{\sigma},$

for $G_{n}(\sigma)=\sum_{l=1}^{n-1}\frac{1}{l-\sigma}Z_{n,l+1}$, and

$h_{\sigma,M}(k)=1+\sum_{l=1}^{k-1}\frac{M}{M+l\sigma}\leq
1+\frac{M}{\sigma}\log\Bigl{(}1+\frac{k\sigma}{M}\Bigr{)}.$ (10)

Expansion of the equation $\Lambda_{n}^{\prime}(\hat{\sigma}_{n,M},M)=0$
around the zero $\sigma_{0,n}$ of the function $E_{0,n}$ in (5) gives

$\hat{\sigma}_{n,M}-\sigma_{0,n}=-\frac{\Lambda^{\prime}_{n}(\sigma_{0,n},M)}{\Lambda^{\prime\prime}_{n}(\sigma_{0,n},M)+\Lambda^{\prime\prime\prime}_{n}(\tilde{\sigma}_{n,M},M)(\hat{\sigma}_{n,M}-\sigma_{0,n})/2}.$

It is shown in Lemma 5 that
$V_{n}:=\bigl{(}K_{n}/\sigma_{n,0}-G_{n}(\sigma_{n,0})\bigr{)}/\sqrt{\alpha_{n}}$,
which is free of $M$, tends to a centered normal distribution, and it is shown
in Lemma 6 that the sequence
$\bigl{(}-K_{n}/\sigma_{0,n}^{2}-G_{n}^{\prime}(\sigma_{0,n})\bigr{)}/\alpha_{n}$
tends in probability to $-\tau_{2}^{2}:=E_{0}^{\prime}(\sigma_{0})$. It can
similarly be seen that
$Z_{n}:=\bigl{(}2K_{n}/\sigma_{0,n}^{3}-G_{n}^{\prime\prime}(\sigma_{0,n})\bigr{)}/\alpha_{n}$
tends in probability to a constant $z$. Furthermore, it follows from the bound
in (10) and the fact that $K_{n}/\alpha_{n}$ converges almost surely, that
$h_{\sigma_{0,n},M}(K_{n})=O_{P}(\log n)$, uniformly in $M$ belonging to
compacta, and from the definition of $h_{\sigma,M}(K_{n})$ that its first and
second partial derivatives relative to $\sigma$ are of the same order.
Therefore, uniformly in $M$,

$\displaystyle\Lambda^{\prime}_{n}(\sigma_{0,n},M)$
$\displaystyle=\frac{K_{n}}{\sigma_{0,n}}-G_{n}(\sigma_{0,n})-\frac{h_{\sigma_{0,n},M}(K_{n})}{\sigma_{0,n}}=\alpha_{n}^{1/2}V_{n}-\frac{h_{\sigma_{0,n},M}(K_{n})}{\sigma_{0,n}},$
$\displaystyle\Lambda^{\prime\prime}_{n}(\sigma_{0,n},M)$
$\displaystyle=\\!\frac{-K_{n}}{\sigma_{0,n}^{2}}-\\!G_{n}^{\prime}(\sigma_{0,n})-\\!\frac{d}{d\sigma}\Bigl{[}\frac{h_{\sigma,M}(K_{n})}{\sigma}\Bigr{]}_{\sigma=\sigma_{0,n}}\\!\\!\\!\\!\\!=\\!-\alpha_{n}\tau_{2}^{2}+\\!O_{P}(\log
n),$ $\displaystyle\Lambda^{\prime\prime\prime}_{n}(\tilde{\sigma}_{n,M},M)$
$\displaystyle=\\!\frac{2K_{n}}{\sigma_{0,n}^{3}}-\\!G_{n}^{\prime\prime}(\sigma_{0,n})-\\!\frac{d^{2}}{d\sigma^{2}}\Bigl{[}\frac{h_{\sigma,M}(K_{n})}{\sigma}\Bigr{]}_{\sigma=\sigma_{0,n}}\\!\\!\\!\\!\\!=\\!\alpha_{n}Z_{n}+O_{P}(\log
n).$

Substituting these expansions in the preceding display gives that

$\displaystyle\hat{\sigma}_{n,M}-\sigma_{0,n}$
$\displaystyle=\frac{\alpha_{n}^{-1/2}V_{n}-\alpha_{n}^{-1}h_{\sigma_{0,n},M}(K_{n})/\sigma_{0,n}}{\tau_{2}^{2}-Z_{n}(\hat{\sigma}_{n,M}-\sigma_{0,n})+O_{P}(\log
n/\alpha_{n})}$
$\displaystyle=\\!\Bigl{(}\frac{V_{n}}{\alpha_{n}^{1/2}\tau_{2}^{2}}-\frac{h_{\sigma_{0,n},M}(K_{n})}{\alpha_{n}\sigma_{0,n}\tau_{2}^{2}}\Bigr{)}\Bigl{(}1+\frac{Z_{n}(\hat{\sigma}_{n,M}-\sigma_{0,n})}{\tau_{2}^{2}}+O_{P}\Bigl{(}\frac{\log
n}{\alpha_{n}}\Bigr{)}\Bigr{)}.$

We can solve $\hat{\sigma}_{n,M}-\sigma_{0,n}$ from this as

$(\hat{\sigma}_{n,M}-\sigma_{0,n})\Bigl{(}1-\frac{V_{n}Z_{n}}{\alpha_{n}^{1/2}\tau_{2}^{4}}\Bigr{)}=\frac{V_{n}}{\alpha_{n}^{1/2}\tau_{2}^{2}}-\frac{h_{\sigma_{0,n},M}(K_{n})}{\alpha_{n}\sigma_{0,n}\tau_{2}^{2}}+O_{P}\Bigl{(}\frac{\log
n}{\alpha_{n}^{3/2}}\Bigr{)}.$

Hence, for $W_{n,M}=h_{\sigma_{0,n},M}(K_{n})/(\sigma_{0,n}\tau_{2}^{2}\log
n)$,

$\displaystyle\hat{\sigma}_{n,M}$
$\displaystyle=\sigma_{0,n}+\frac{V_{n}}{\alpha_{n}^{1/2}\tau_{2}^{2}}+\frac{V_{n}^{2}Z_{n}}{\alpha_{n}\tau_{2}^{6}}-\frac{W_{n,M}\log
n}{\alpha_{n}}+O_{P}\Bigl{(}\frac{\log n}{\alpha_{n}^{3/2}}\Bigr{)}$
$\displaystyle=:\tilde{\sigma}_{n}-W_{n,M}\log n/\alpha_{n}+O_{P}(\log
n/\alpha_{n}^{3/2}).$

The variables $V_{n}$, $Z_{n}$ and $W_{n,M}$ are all bounded in probability.
The quantity $\tilde{\sigma}_{n}$ is defined as the sum of the first three
terms on the right in the preceding line, and does not depend on $M$.

We are now ready to expand the profile likelihood
$\Lambda_{n}(\hat{\sigma}_{n,M},M)$. The first and third terms on the far
right side of (7) can be expanded with the help of Lemma 16 as, uniformly in
$M$ in compacta,

$\displaystyle\sum_{l=1}^{K_{n}-1}\log(M+l\hat{\sigma}_{n,M})$
$\displaystyle=K_{n}\log
K_{n}+K_{n}\log(\hat{\sigma}_{n,M}/e)+\Bigl{(}\frac{M}{\hat{\sigma}_{n,M}}-\frac{1}{2}\Bigr{)}\log
K_{n}$
$\displaystyle\qquad+\log\frac{\sqrt{2\pi}}{\hat{\sigma}_{n,M}}-\log\Gamma\Bigl{(}1+\frac{M}{\hat{\sigma}_{n,M}}\Bigr{)}+O_{P}\Bigl{(}\frac{1}{K_{n}}\Bigr{)},$
$\displaystyle\sum_{i=1}^{n-1}\log(M+i)$ $\displaystyle=n\log
n-n+\Bigl{(}M-\frac{1}{2}\Bigr{)}\log n+$
$\displaystyle\qquad+\log\sqrt{2\pi}-\log\Gamma(1+M)+O\Bigl{(}\frac{1}{n}\Bigr{)}.$

To find the point of maximum $M$, we can drop terms that depend on $K_{n}$ and
$n$ only, and add terms that do not depend on $M$. Thus finding the maximizer
of the profile likelihood is equivalent to finding the maximizer of

$\displaystyle(K_{n}-1)\log\frac{\hat{\sigma}_{n,M}}{\tilde{\sigma}_{n}}+\sum_{l=1}^{n-1}\log\frac{l-\hat{\sigma}_{n,M}}{l-\tilde{\sigma}_{n}}Z_{n,l+1}+\frac{M\log
K_{n}}{\hat{\sigma}_{n,M}}-M\log n$
$\displaystyle\qquad\qquad\qquad+\log\Gamma(1+M)-\log\Gamma\Bigl{(}1+\frac{M}{\hat{\sigma}_{n,M}}\Bigr{)}+O_{P}\Bigl{(}\frac{1}{K_{n}}\Bigr{)}.$
(11)

The sum of the first two terms can be expanded as

$\displaystyle(K_{n}-1)\log\Bigl{(}1-\frac{W_{n,M}\log
n}{\alpha_{n}\tilde{\sigma}_{n}}+O_{P}\Bigl{(}\frac{\log
n}{\alpha_{n}^{3/2}}\Bigr{)}\Bigr{)}-\sum_{l=1}^{n-1}\log\Bigl{(}1-\frac{\hat{\sigma}_{n,M}-\tilde{\sigma}_{n}}{l-\tilde{\sigma}_{n}}\Bigr{)}Z_{n,l+1}$
$\displaystyle\qquad=-\frac{K_{n}W_{n,M}\log
n}{\alpha_{n}\tilde{\sigma}_{n}}-\sum_{l=1}^{n-1}\Bigl{(}\frac{\hat{\sigma}_{n,M}-\tilde{\sigma}_{n}}{l-\tilde{\sigma}_{n}}\Bigr{)}Z_{n,l+1}+O_{P}\Bigl{(}\frac{\log
n}{\sqrt{\alpha}_{n}}\Bigr{)}$ $\displaystyle\qquad=-\frac{W_{n,M}\log
n}{\alpha_{n}}\Bigl{(}\frac{K_{n}}{\tilde{\sigma}_{n}}-\sum_{l=1}^{n-1}\frac{Z_{n,l+1}}{l-\tilde{\sigma}_{n}}\Bigr{)}+O_{P}\Bigl{(}\frac{\log
n}{\sqrt{\alpha}_{n}}\Bigr{)}$ $\displaystyle\qquad=-\frac{W_{n,M}\log
n}{\alpha_{n}}\bigl{(}\sqrt{\alpha_{n}}\tilde{V}_{n}+E_{0,n}(\tilde{\sigma}_{n})\bigr{)}+O_{P}\Bigl{(}\frac{\log
n}{\sqrt{\alpha_{n}}}\Bigr{)},$

where $\tilde{V}_{n}$ tends to a centered normal distribution by Lemma 5. The
right side is of the order $O_{P}(\log n/\sqrt{\alpha_{n}})$.

Since
$L_{n}=\sqrt{\alpha_{n}}\bigl{(}K_{n}/\alpha_{n}-\Gamma(1-\sigma_{0})\bigr{)}$
tends to a centered normal distribution, we have
$K_{n}=\alpha_{n}\Gamma(1-\sigma_{0})+O_{P}(\sqrt{\alpha_{n}})$, so that $\log
K_{n}=\log\alpha_{n}+\log\Gamma(1-\sigma_{0})+O_{P}(1/\sqrt{\alpha_{n}})$.
Therefore, if $\alpha_{n}=n^{\sigma_{0}}\bar{L}_{0}(n)$, then the sum of the
third and fourth terms on the right of (11) are

$\displaystyle M\\!\Bigl{[}\frac{\sigma_{0}\log
n\\!+\\!\log\bar{L}_{0}(n)\\!+\\!\log\Gamma(1-\sigma_{0})\\!+\\!O_{P}(1/\\!\sqrt{\alpha_{n}})}{\sigma_{0,n}}\Bigr{]}\\!\Bigl{[}\\!1\\!+\\!O_{P}\Bigl{(}\\!\frac{1}{\sqrt{\alpha_{n}}}\\!\Bigr{)}\\!\Bigr{]}\\!-\\!M\log
n$ $\displaystyle\qquad=M\Bigl{(}\frac{\sigma_{0}}{\sigma_{0,n}}-1\Bigr{)}\log
n+\frac{M}{\sigma_{0,n}}\bigl{(}\log\bar{L}_{0}(n)+\log\Gamma(1-\sigma_{0})\bigr{)}+O_{P}\Bigl{(}\frac{\log
n}{\sqrt{\alpha_{n}}}\Bigr{)}.$

By Lemma 8,
$\sigma_{0,n}-\sigma_{0}=O(n^{-\delta})+O\bigl{(}L_{0}^{\prime}(n)n/L_{0}(n)\bigr{)}$,
whence the right side is of the order $(\log
n)L_{0}^{\prime}(n)n/L_{0}(n)+\log L_{0}(n)+\log\Gamma(1-\sigma_{0})$.

We conclude that up to terms that do not depend on $M$, the profile log
likelihood (11) is equal to, for a constant $c>0$,

$\displaystyle\frac{M}{\sigma_{0}}\Bigl{[}\frac{L_{0}^{\prime}(n)nc\log
n}{L_{0}(n)}+\log L_{0}(n)+\log\Gamma(1-\sigma_{0})\Bigr{]}$
$\displaystyle\qquad\qquad\qquad\qquad\qquad+\log\Gamma(1+M)-\log\Gamma\Bigl{(}1+\frac{M}{\sigma_{0}}\Bigr{)}+o_{P}(1).$

If the term within square brackets tends to infinity, then asymptotically this
term dominates and the maximizing value $\hat{M}_{n}$ will tend to the end
point of the interval. If this term tends to minus infinity, then it also
dominates and the maximum value will tend to 0. If the term converges to a
limit, then the whole expression tends to a function of the form $M\mapsto
aM+\log\Gamma(1+M)-\log\Gamma(1+{M}/{\sigma_{0}})$. This function is concave
and tends to $-\infty$ as $M\rightarrow\infty$. Its derivative at zero may be
both positive or negative, depending on $a$ and $\sigma_{0}$, and the point of
maximum of the function may be at zero or at some positive location, possibly
the upper limit $\bar{M}$ of the parameter. The sequence $\hat{M}_{n}$ will
tend to this point of maximum.

###  3.4 Lemmas

For $g_{\sigma}(m)=\sum_{l=1}^{m-1}\frac{1}{l-\sigma}$, set

$\displaystyle\tau_{1}^{2}$
$\displaystyle=\frac{(2^{\sigma_{0}}-1)\Gamma(1-\sigma_{0})}{\sigma_{0}^{2}}+\sum_{m=1}^{\infty}\frac{\Gamma(m-\sigma_{0})(g_{\sigma_{0}}(m+1)+g_{\sigma_{0}}(m))}{m!}$
(12)
$\displaystyle\qquad-\sum_{k=2}^{\infty}\sum_{m=1}^{\infty}\frac{g_{\sigma_{0}}(k)\Gamma(k+m+1-\sigma_{0})}{k!m!2^{k+m-\sigma_{0}}(m-\sigma_{0})}-\sum_{m=2}^{\infty}\frac{g_{\sigma_{0}}(m)\Gamma(m-\sigma_{0})}{m!\,2^{m-\sigma_{0}-1}}.$

######  Lemma 5.

For any $\sigma_{n}\,\hbox{$\stackrel{{\scriptstyle\mbox{\scriptsize
P}}}{{\rightarrow}}$}\,\sigma_{0}\in(0,1)$ and
$M_{n}=o\bigl{(}\sqrt{\alpha_{0}(n)}/\log n\bigr{)}$, we have
$\alpha_{0}(n)^{-1/2}\bigl{(}\Lambda_{n}^{\prime}(\sigma_{n},M_{n})-E_{0,n}(\sigma_{n})\bigr{)}\rightsquigarrow
N(0,\tau_{1}^{2})$.

###### Proof.

We denote the true distribution by
$P_{0}=\sum_{i=1}^{\infty}p_{i}\delta_{x_{i}}$. The variables $Z_{n,l}$ can be
written as $Z_{n,l}=\sum_{j=1}^{\infty}1_{M_{n,j}\geq l}$, for $M_{n,j}$ the
number of observations equal to $x_{j}$. As $K_{n}=Z_{n,1}$, the function
$\Lambda_{n}^{\prime}$ can be written in the form

$\Lambda_{n}^{\prime}(\sigma,M)=\sum_{j=1}^{\infty}\Bigl{[}\frac{1_{M_{n,j}\geq
1}}{\sigma}-g_{\sigma}(M_{n,j})\Bigr{]}-\frac{h_{\sigma,M}(K_{n})}{\sigma},$

where $g_{\sigma}(0)=g_{\sigma}(1)=0$ and
$g_{\sigma}(m)=\sum_{l=1}^{m-1}\frac{1}{l-\sigma}$, for $m\geq 2$. It is shown
in [15] (and repeated below) that
$\mathbb{E}K_{n}/\alpha_{0}(n)\rightarrow\Gamma(1-\sigma_{0})$ and hence
Jensen’s inequality and (10) give $\mathbb{E}h_{\sigma_{n},M}(K_{n})\leq
1+(M/\sigma_{n})\log(1+\mathbb{E}K_{n}\sigma_{n}/M)=O(M\log
n)=o(\alpha_{0}(n)^{1/2})$, so that the term on the far right is
asymptotically negligible.

To prove the asymptotic normality of the infinite sum, after centering and
scaling, we first consider the case that the sample size $n$ is taken to be a
random variable $N_{n}$ with the Poisson distribution with mean $n$,
independent of the original variables. The resulting variables $M_{N_{n},j}$
are then Poisson distributed with means $np_{j}$ and independent across $j$,
and the asymptotic normality can be proved using the Lindeberg central limit
theorem. (For reference, an appropriate infinite series version is formulated
in Lemma 11.) As shown in the proof of (i) and (iii) of Lemma 7, the function
$E_{0,n}(\sigma)$ defined in (5) is equal to the expectation of
$V_{k,n}=\sum_{j=1}^{\infty}\bigl{[}1_{M_{n,j}\geq 1}/\sigma-
g_{\sigma}(M_{n,j})\bigr{]}$. Because $(1_{M_{n,j}=0})g_{\sigma}(M_{n,j})=0$,
by Lemma 7 (ii), (v), (iv) and (viii), the variance of $V_{k,n}$ is given by

$\displaystyle\sum_{j=1}^{\infty}\mathop{\rm
var}\nolimits\Bigl{(}\frac{1_{M_{N_{n},j}\geq
1}}{\sigma}-g_{\sigma}(M_{N_{n},j})\Bigr{)}=\sum_{j=1}^{\infty}\Bigl{[}\frac{e^{-np_{j}}(1-e^{-np_{j}})}{\sigma^{2}}$
$\displaystyle\qquad\qquad+\mathop{\rm var}\nolimits
g_{\sigma}(M_{N_{n},j})-2\frac{e^{-np_{j}}}{\sigma}\mathbb{E}g_{\sigma}(M_{N_{n},j})\Bigr{]}\sim\alpha_{0}(n)\tau_{1}^{2},$
(13)

where $\tau_{1}^{2}$ is equal to $(ii)/\sigma^{2}+(v)-(vi)-(2/\sigma)(vii)$,
for $(ii),(v),(vi)$ and $(vii)$ shorthand for the expressions on the right
sides in Lemma 7 (ii), (v), (vi) and (vii). Furthermore, the variables
$1_{M_{N_{n},j}\geq 1}$ are bounded and hence trivially satisfy the Lindeberg
condition, while the Lindeberg condition on the variables
$g_{\sigma}(M_{N_{n},j})$ follows from the boundedness of
$\alpha_{0}(n)^{-1}\sum_{j}\mathbb{E}g_{\sigma}(M_{N_{n},j})^{3}$, by Lemma 7
(viii). Thus the Poissonized sums are asymptotically normal, by Lemma 11.

The proof can be completed by applying Lemma 10 to the variables
$V_{k,n}=\sum_{j=1}^{\infty}\bigl{(}1_{M_{k,j}\geq
1}/\sigma_{n}-g_{\sigma_{n}}(M_{k,j})\bigr{)}$, with
$a_{n}=\alpha_{0}(n)^{-1/2}$ and $E_{0,n}$ as given. To show that
$a_{n}(V_{k_{n},n}-V_{n,n})$ tends to zero in probability, we split $V_{k,n}$
in $\sum_{j}1_{M_{k,j}\geq 1}/\sigma_{n}$ and
$\sum_{j}g_{\sigma_{n}}(M_{k,j})$ and handle the two parts separately. Because
$a_{k_{n}}/a_{n}\rightarrow 1$, it is not a loss of generality to assume that
$k_{n}\geq n$. Because the binomial distributions $\text{binomial}(n,p_{j})$
are stochastically ordered in $n$ and the functions $m\mapsto 1_{m\geq 1}$ and
$m\mapsto g_{\sigma}(m)$ are increasing, the variables
$a_{n}\sum_{j}(1_{M_{k,j}\geq 1}-1_{M_{n,j}\geq 1})/\sigma$ and
$a_{n}\sum_{j}(g_{\sigma_{n}}(M_{k,j})-g_{\sigma_{n}}(M_{n,j}))$ are
nonnegative, and hence it suffices to show that their expectations tend to
zero. This is shown in Lemma 9. ∎

######  Lemma 6.

For any $\tilde{\sigma}_{n}\,\hbox{$\stackrel{{\scriptstyle\mbox{\scriptsize
P}}}{{\rightarrow}}$}\,\sigma_{0}\in(0,1)$ and
$M_{n}=o\bigl{(}\alpha_{0}(n)/\log n\bigr{)}$, we have
$\alpha_{0}(n)^{-1}\Lambda_{n}^{\prime\prime}(\tilde{\sigma}_{n},M_{n})\rightarrow
E_{0}^{\prime}(\sigma_{0})$, in probability.

###### Proof.

The second derivative is given by

$\displaystyle\Lambda_{n}^{\prime\prime}(\sigma)$
$\displaystyle=-\sum_{l=1}^{K_{n}-1}\frac{l^{2}}{(M+l\sigma)^{2}}-\sum_{l=1}^{n-1}\frac{1}{(l-\sigma)^{2}}Z_{n,l+1}$
$\displaystyle=-\frac{K_{n}-1}{\sigma^{2}}+\frac{1}{\sigma^{2}}\sum_{l=1}^{K_{n}-1}\Bigl{[}\frac{2M}{M+\sigma
l}-\frac{M^{2}}{(M+\sigma
l)^{2}}\Bigr{]}-\sum_{l=1}^{n-1}\frac{1}{(l-\sigma)^{2}}Z_{n,l+1}.$

It is shown in [15] (see his formula (66), or see the proof of Lemma 5), that
$K_{n}/\alpha_{0}(n)\rightarrow\Gamma(1-\sigma_{0})$ and
$Z_{n,l}/\alpha_{0}(n)\rightarrow\Gamma(l-\sigma_{0})/(l-1)!$, for every
$l\geq 1$, in probability and in mean. We use this to infer the convergence of
the first and last terms on the right divided by $\alpha_{0}(n)$. That the
limit is equal to $E_{0}^{\prime}(\sigma_{0})$ follows by inspection of its
form and Lemma 8. The second term is bounded above in absolute value by a
multiple of $M\log K_{n}$ and divided by $\alpha_{0}(n)$ tends to zero. ∎

###  3.5 Technical lemmas

In the next lemmas $(p_{j})_{j=1}^{\infty}$ is a given infinite probability
vector and $\alpha$ is the cumulative distribution function of the counting
measure on the points $1/p_{j}$, for $j\in\mathbb{N}_{+}$. Furthermore, the
function $g_{\sigma}:\mathbb{N}\to\mathbb{N}$ is given by
$g_{\sigma}(1)=g_{\sigma}(2)=0$ and

$g_{\sigma}(m)=\sum_{l=1}^{m-1}\frac{1}{l-\sigma},\qquad m\geq 2.$

######  Lemma 7.

Suppose that $\alpha(u):=\\#\\{j:1/p_{j}\leq u\\}$ is regularly varying at
$\infty$ of order $\gamma\in(0,1)$. Then, for any
$\sigma_{n}\rightarrow\sigma\in(0,1)$, and independent
$M_{n,j}\sim\text{Poisson}(np_{j})$,

  * (i)

$\frac{1}{\alpha(n)}\sum_{j=1}^{\infty}\mathbb{E}1_{M_{n,j}\geq
1}\rightarrow\Gamma(1-\gamma)$,

  * (ii)

$\frac{1}{\alpha(n)}\sum_{j=1}^{\infty}\mathop{\rm var}\nolimits
1_{M_{n,j}\geq 1}\rightarrow(2^{\gamma}-1)\Gamma(1-\gamma)$,

  * (iii)

$\frac{1}{\alpha(n)}\sum_{j=1}^{\infty}\mathbb{E}g_{\sigma_{n}}(M_{n,j})\rightarrow\sum_{m=1}^{\infty}\frac{\Gamma(m+1-\gamma)}{m!(m-\sigma)}$,

  * (iv)

$\frac{1}{\alpha(n)}\sum_{j=1}^{\infty}\mathbb{E}\frac{\partial
g_{\sigma_{n}}}{\partial\sigma}(M_{n,j})\rightarrow\sum_{m=1}^{\infty}\frac{\Gamma(m+1-\gamma)}{m!(m-\sigma)^{2}}$,

  * (v)

$\frac{1}{\alpha(n)}\sum_{j=1}^{\infty}\mathbb{E}g_{\sigma_{n}}^{2}(M_{n,j})\rightarrow\sum_{m=1}^{\infty}\frac{\Gamma(m+1-\gamma)(g_{\sigma}(m+1)+g_{\sigma}(m))}{m!(m-\sigma)}$,

  * (vi)

$\frac{1}{\alpha(n)}\sum_{j=1}^{\infty}\bigl{(}\mathbb{E}g_{\sigma_{n}}(M_{n,j})\bigr{)}^{2}\rightarrow\sum_{k=2}^{\infty}\sum_{m=1}^{\infty}\frac{g_{\sigma}(k)\Gamma(k+m+1-\gamma)}{k!m!2^{k+m-\gamma}(m-\sigma)}$.

  * (vii)

$\frac{1}{\alpha(n)}\sum_{j=1}^{\infty}e^{-np_{j}}\mathbb{E}g_{\sigma_{n}}(M_{n,j})\rightarrow\sum_{m=2}^{\infty}\frac{g_{\sigma}(m)\gamma\Gamma(m-\gamma)}{m!\,2^{m-\gamma}}$.

  * (viii)

$\frac{1}{\alpha(n)}\sum_{j=1}^{\infty}\mathbb{E}g_{\sigma_{n}}^{3}(M_{n,j})\rightarrow\sum_{m=1}^{\infty}\frac{\Gamma(m+1-\gamma)(g_{\sigma}^{2}(m+1)+g_{\sigma}(m+1)g_{\sigma}(m)+g_{\sigma}^{2}(m))}{m!(m-\sigma)}$,

All limits on the right sides are finite.

###### Proof.

Assertions (i) and (ii) were stated in [15]; we include proofs for
completeness.

The series in the left side of (i) is

$\sum_{j=1}^{\infty}\Pr(M_{n,j}\geq
1)=\sum_{j=1}^{\infty}(1-e^{-np_{j}})=\int_{1}^{\infty}\\!(1-e^{-n/u})\,d\alpha(u)=\int_{0}^{n}\\!\alpha\Bigl{(}\frac{n}{s}\Bigr{)}e^{-s}\,ds,$

by Fubini’s theorem (or partial integration), since
$1-e^{-n/u}=\int_{0}^{n/u}e^{-s}\,ds$. By the definition of regular variation
$\alpha(n/s)/\alpha(n)\rightarrow s^{-\gamma}$, for every $s$, as
$n\rightarrow\infty$. By Potter’s theorem ([2], Theorem 1.5.6), for every
$\delta>0$ there exists $M>1$ such that $\alpha(n/s)/\alpha(n)\leq
s^{-\gamma-\delta}\vee s^{-\gamma+\delta}$, for every $s<n/M$. We can choose
$\delta$ so that $\gamma+\delta<1$, and then
$\int_{0}^{\infty}(s^{-\gamma-\delta}\vee
s^{-\gamma+\delta})e^{-s}\,ds<\infty$. For the corresponding $M$, we then have
$\int_{0}^{n/M}\alpha(n/s)/\alpha(n)\,e^{-s}\,ds\rightarrow\int_{0}^{\infty}s^{-\gamma}e^{-s}\,ds=\Gamma(1-\gamma)$,
by the dominated convergence theorem. For $s\geq n/M$, we have
$\alpha(n/s)\leq\alpha(M)$ and hence
$\int_{n/M}^{n}\alpha(n/s)e^{-s}\,ds\leq\alpha(M)e^{-n/M}=o(\alpha(n))$, as
$n\rightarrow\infty$.

The series in (ii) is
$\sum_{j=1}^{\infty}e^{-np_{j}}(1-e^{-np_{j}})=\sum_{j=1}^{\infty}(1-e^{-2np_{j}})-\sum_{j=1}^{\infty}(1-e^{-np_{j}})$.
By the first paragraph this is asymptotic to
$\bigl{(}\alpha(2n)-\alpha(n)\bigr{)}\Gamma(1-\gamma)\sim(2^{\gamma}-1)\alpha(n)\Gamma(1-\gamma)$,
by regular variation of $\alpha$.

For (iii) we write

$\sum_{j=1}^{\infty}\mathbb{E}g_{\sigma}(M_{n,j})=\\!\sum_{j=1}^{\infty}\sum_{m=2}^{\infty}g_{\sigma}(m)\frac{e^{-np_{j}}(np_{j})^{m}}{m!}\\!=\sum_{m=2}^{\infty}\frac{g_{\sigma}(m)}{m!}\\!\int_{1}^{\infty}\\!\\!e^{-n/u}\Bigl{(}\frac{n}{u}\Bigr{)}^{m}\,d\alpha(u).$

Substituting $e^{-n/u}(n/u)^{m}=\int_{0}^{n/u}e^{-s}s^{m-1}(m-s)\,ds$ (valid
for $m>0$) and using Fubini’s theorem, we can rewrite the right side as

$\displaystyle\sum_{m=2}^{\infty}\frac{g_{\sigma}(m)}{m!}\int_{0}^{n}\alpha\Bigl{(}\frac{n}{s}\Bigr{)}e^{-s}s^{m-1}(m-s)\,ds$
$\displaystyle=g_{\sigma}(2)\int_{0}^{n}\alpha\Bigl{(}\frac{n}{s}\Bigr{)}e^{-s}s\,ds+\sum_{m=2}^{\infty}\frac{g_{\sigma}(m+1)-g_{\sigma}(m)}{m!}\int_{0}^{n}\alpha\Bigl{(}\frac{n}{s}\Bigr{)}e^{-s}s^{m}\,ds$
$\displaystyle=\int_{0}^{n}\alpha\Bigl{(}\frac{n}{s}\Bigr{)}e^{-s}\Bigl{(}\sum_{m=1}^{\infty}\frac{s^{m}}{m!(m-\sigma)}\Bigr{)}\,ds.$

As before, regular variation and Potter’s theorem give for $s\leq n/M$ the
bound $\alpha(n/s)/\alpha(n)\lesssim s^{-\gamma-\delta}\vee
s^{-\gamma+\delta}$, and then

$\frac{\alpha(n/s)}{\alpha(n)}\Bigl{(}\sum_{m=1}^{\infty}\frac{s^{m}}{m!(m-\sigma)}\Bigr{)}\lesssim(s^{-\gamma-\delta}\vee
1)(e^{s}-1-s)\frac{1}{s},\qquad s\leq n/M.$

Furthermore, the left side tends pointwise to
$s^{-\gamma}\sum_{m=1}^{\infty}s^{m}/(m!(m-\sigma))$. By the dominated
convergence theorem,

$\displaystyle\int_{0}^{n/M}\frac{\alpha(n/s)}{\alpha(n)}e^{-s}\Bigl{(}\sum_{m=1}^{\infty}\frac{s^{m}}{m!(m-\sigma)}\Bigr{)}\,ds\rightarrow\int_{0}^{\infty}s^{-\gamma}e^{-s}\sum_{m=1}^{\infty}\frac{s^{m}}{m!(m-\sigma)}\,ds.$

The right side is the limit as given. Since $\sum_{j}p_{j}=1$, we have that
$\alpha(u)=\\#(p_{j}\geq 1/u)\leq u$. Therefore

$\int_{n/M}^{n}\alpha\Bigl{(}\frac{n}{s}\Bigr{)}e^{-s}\Bigl{(}\sum_{m=1}^{\infty}\frac{s^{m}}{m!(m-\sigma)}\Bigr{)}\,ds\leq\int_{n/M}^{\infty}\frac{n}{s}\frac{1}{s}\,ds\leq
M.$

This is of lower order than $\alpha(n)$ and hence the proof of the third
assertion is complete.

For (iv) we follow the same approach as in (iii), replacing $g_{\sigma}$ by
$\dot{g}_{\sigma}=\partial/\partial\sigma g_{s}$, and then at the end
substitute $\dot{g}_{s}(m+1)-\dot{g}_{\sigma}(m)=1/(m-\sigma)^{2}$.

For (v) again we follow the same approach as under (iii), now replacing
$g_{\sigma}$ by $g_{\sigma}^{2}$. At the end we write the difference
$g_{\sigma}^{2}(m+1)-g_{\sigma}^{2}(m)$ as
$(m-\sigma)^{-1}\bigl{(}g_{\sigma}(m+1)+g_{\sigma}(m)\bigr{)}$ and complete
the argument as before, where we can bound $g_{\sigma}(m+1)+g_{\sigma}(m)$ by
a multiple of $\log m$, for large $m$, and use that $\sum_{m}s^{m}\log
m/m!\lesssim e^{s}(s^{\delta}\vee 1)$, for every $s$, by Lemma 15, with a
sufficiently small $\delta>0$.

The series in (vi) is equal to

$\displaystyle\sum_{j=1}^{\infty}\sum_{k=2}^{\infty}\sum_{m=2}^{\infty}g_{\sigma}(k)g_{\sigma}(m)\frac{e^{-2np_{j}}(np_{j})^{k+m}}{k!m!}$
$\displaystyle\qquad\quad=\sum_{k=2}^{\infty}\sum_{m=2}^{\infty}\frac{g_{\sigma}(k)g_{\sigma}(m)}{k!\,m!}\\!\int_{1}^{\infty}\\!\\!e^{-2n/u}\Bigl{(}\frac{n}{u}\Bigr{)}^{k+m}\,d\alpha(u).$

Substituting $e^{-2n/u}(2n/u)^{k+m}=\int_{0}^{2n/u}e^{-s}s^{k+m-1}(k+m-s)\,ds$
and using Fubini’s theorem, we can rewrite the right side as

$\displaystyle\sum_{k=2}^{\infty}\sum_{m=2}^{\infty}\frac{g_{\sigma}(k)g_{\sigma}(m)}{k!\,m!\,2^{k+m}}\int_{0}^{2n}\alpha\Bigl{(}\frac{2n}{s}\Bigr{)}e^{-s}s^{k+m-1}(k+m-s)\,ds$
$\displaystyle\quad=\int_{0}^{2n}\alpha\Bigl{(}\frac{2n}{s}\Bigr{)}\,\frac{d}{ds}\Bigl{[}\Bigl{(}\sum_{k=2}^{\infty}\frac{g_{\sigma}(k)s^{k}}{k!\,2^{k}}\Bigr{)}^{2}e^{-s}\Bigr{]}\,ds$
$\displaystyle\quad=\int_{0}^{2n}\alpha\Bigl{(}\frac{2n}{s}\Bigr{)}\,\Bigl{(}\sum_{k=2}^{\infty}\frac{g_{\sigma}(k)s^{k}}{k!\,2^{k}}\Bigr{)}e^{-s}\Bigl{[}\sum_{k=1}^{\infty}\frac{\bigl{(}g_{\sigma}(k+1)-g_{\sigma}(k)\bigr{)}s^{k}}{k!\,2^{k}}\Bigr{]}\,ds.$

In view of Lemma 15 and because $g_{\sigma}(k+1)-g_{\sigma}(k)=1/(k-\sigma)$,
the integrand is bounded above by a multiple of
$\alpha(2n/s)(e^{s/2}-1)e^{-s}s^{-1}(e^{s/2}-1)$. Using the dominated
convergence theorem and arguments as before, we see that the right side
divided by $\alpha(n)$ is asymptotic to the right side of (vi).

The extra factor $e^{-np_{j}}$ in (vii) relative to (iii) leads to the same
expression as in (iii), except that $e^{-n/u}$ must be replaced by
$e^{-2n/u}$. Following the same argument, we find that the series in (vii) is
equal to

$\sum_{m=2}^{\infty}\frac{g_{\sigma}(m)}{m!\,2^{m}}\int_{0}^{2n}\alpha\Bigl{(}\frac{2n}{s}\Bigr{)}e^{-s}s^{m-1}(m-s)\,ds.$

The functions
$\sum_{m=2}^{\infty}\frac{g_{\sigma}(m)}{m!2^{m}}e^{-s}s^{m-1}|m-s|$ are
uniformly integrable (thanks to the factors $2^{m}$ that are extra relative to
(iii)) . Therefore, by arguments as before the display is asymptotically
equivalent to the expression obtained by replacing $\alpha(2n/s)$ by
$\alpha(n)(2/s)^{\gamma}$. Finally we can use that
$m\Gamma(m-\gamma)-\Gamma(m+1-\gamma)=\gamma\Gamma(m-\gamma)$.

For (viii) we follow the same approach as under (iii), replacing $g_{\sigma}$
by $g_{\sigma}^{3}$, where at the end we write the difference
$g_{\sigma}^{3}(m+1)-g_{\sigma}^{3}(m)$ as
$(m-\sigma)^{-1}(g_{\sigma}^{2}(m+1)+g_{\sigma}(m+1)g_{\sigma}(m)+g_{\sigma}^{2}(m))$.

The finiteness of the limitss can be proved with the help of Lemma 13 by
comparison with standard series. ∎

For $\alpha(u):=\\#\\{j:1/p_{j}\leq u\\}$ as in the preceding, define a
function $E_{n}$ by

$E_{n}(\sigma)=\int_{0}^{n}\alpha\Bigl{(}\frac{n}{s}\Bigr{)}e^{-s}\Bigl{(}\frac{1}{\sigma}-\sum_{m=1}^{\infty}\frac{s^{m}}{m!(m-\sigma)}\Bigr{)}\,ds.$
(14)

######  Lemma 8.

If the function $\alpha$ is regularly varying at $\infty$ of order
$\gamma\in(0,1)$, then the functions $E_{n}$ in (14) satisfy, as
$n\rightarrow\infty$, for $\sigma_{n}\rightarrow\sigma\in(0,1)$,

$\displaystyle\frac{E_{n}(\sigma_{n})}{\alpha(n)}$ $\displaystyle\rightarrow
E(\sigma):=\frac{\Gamma(1-\gamma)}{\sigma}-\sum_{m=1}^{\infty}\frac{\Gamma(m+1-\gamma)}{m!(m-\sigma)},$
(15) $\displaystyle\frac{E_{n}^{\prime}(\sigma_{n})}{\alpha(n)}$
$\displaystyle\rightarrow E^{\prime}(\sigma).$

The limit function $E$ vanishes at $\sigma=\gamma$ and the zeros
$\sigma_{0,n}$ of $E_{n}$ satisfy $\sigma_{0,n}\rightarrow\gamma$.
Furthermore, if there exists a continuously differentiable function
$L:[1,\infty)\to\mathbb{R}$ such that $|\alpha(u)-u^{\gamma}L(u)|\leq
Cu^{\beta}$, for every $u>1$, and some $C>0$ and $\beta<\gamma$, and such that
$s\mapsto L^{\prime}(s)s$ is slowly varying at $\infty$, then, as
$n\rightarrow\infty$,

$\sigma_{0,n}-\gamma=O\Bigl{(}\frac{n^{\beta-\gamma}}{L(n)}\Bigr{)}-\frac{\Gamma(1-\gamma)(1+\gamma)}{\gamma^{2}E^{\prime}(\gamma)}\Bigl{(}\frac{L^{\prime}(n)n}{L(n)}\Bigr{)}\bigl{(}1+o(1)\bigr{)}.$

In particular, if $L$ can be taken constant, then
$\sigma_{0,n}-\gamma=O(n^{\beta-\gamma})$.

###### Proof.

As shown in the proof of Lemma 7 (i) and (iii), the function $E_{n}(\sigma)$
is equal to $\sum_{j=1}^{\infty}\bigl{(}\mathbb{E}1_{M_{n,j}\geq
1}/\sigma-\mathbb{E}g_{\sigma}(M_{n,j})\bigr{)}$, with the notation as in the
lemma. Therefore, by the lemma the limit function $E$ is equal to the limit in
(i) divided by $\sigma$ minus the limit in (iii), i.e. the right side of (15).
The limit in (iii) at $\sigma=\gamma$ can be written

$\displaystyle\sum_{m=1}^{\infty}\frac{\Gamma(m-\gamma)}{m!}=\sum_{m=1}^{\infty}\int_{0}^{\infty}\frac{s^{m-\gamma-1}}{m!}e^{-s}\,ds=\int_{0}^{\infty}(1-e^{-s})s^{-\gamma-1}\,ds.$

By partial integration, this can be further rewritten as
$\int_{0}^{\infty}x^{-\gamma}/\gamma\,e^{-x}\,dx=\Gamma(1-\gamma)/\gamma$.
Thus $E(\gamma)=\Gamma(1-\gamma)/\gamma-\Gamma(1-\gamma)/\gamma=0$.

The limit of $E_{n}^{\prime}(\sigma_{n})/\alpha(n)$ is obtained similarly from
(i) and (iv) of Lemma 7, and it is seen to be equal to the derivative
$E^{\prime}(\sigma)$.

The functions $E_{n}$ and $E$ are monotonely decreasing and
$E_{n}(\gamma-\epsilon)/\alpha(n)\rightarrow E(\gamma-\epsilon)>0$ and
$E_{n}(\gamma+\epsilon)/\alpha(n)\rightarrow E(\gamma+\epsilon)<0$, for every
$\epsilon>0$, by (15). This shows that
$\sigma_{0,n}\in(\gamma-\epsilon,\gamma+\epsilon)$ eventually and hence
$\sigma_{0,n}\rightarrow\gamma$.

By the mean value theorem,
$E_{n}(\sigma_{0,n})-E_{n}(\gamma)=E_{n}^{\prime}(\tilde{\sigma}_{n})(\sigma_{0,n}-\gamma)$,
for some $\tilde{\sigma}_{n}\rightarrow\gamma$. Since $E_{n}(\sigma_{0,n})=0$
and $E_{n}^{\prime}(\tilde{\sigma}_{n})/\alpha(n)\rightarrow
E^{\prime}(\gamma)<0$, it follows that
$\sigma_{0,n}-\gamma=-\bigl{(}E_{n}(\gamma)/\alpha(n)\bigr{)}/\bigl{(}E^{\prime}(\gamma)+o(1)\bigr{)}$.
This shows that $\sigma_{0,n}\rightarrow\gamma$ at the same rate of
convergence as $E_{n}(\gamma)/\alpha(n)\rightarrow E(\gamma)=0$.

To investigate the latter rate, define functions $H_{n}$ by

$H_{n}(\sigma)=\int_{0}^{n}L\Bigl{(}\frac{n}{s}\Bigr{)}s^{-\gamma}e^{-s}\Bigl{(}\frac{1}{\sigma}-\sum_{m=1}^{\infty}\frac{s^{m}}{m!(m-\sigma)}\Bigr{)}\,ds.$

The functions $H_{n}$ are monotonely decreasing, with, by the assumption on
$\alpha$,

$\displaystyle\bigl{|}E_{n}(\sigma)-n^{\gamma}H_{n}(\sigma)\bigr{|}$
$\displaystyle\leq\int_{0}^{n}\Bigl{(}\frac{n}{s}\Bigr{)}^{\beta}e^{-s}\Bigl{|}\frac{1}{\sigma}-\sum_{m=1}^{\infty}\frac{s^{m}}{m!(m-\sigma)}\Bigr{|}\,ds$
$\displaystyle\lesssim
n^{\beta}\Bigl{(}\frac{1}{\sigma}+\frac{1}{1-\sigma}\int_{0}^{n}\frac{1}{s^{1+\beta}}e^{-s}(e^{s}-1-s)\,ds\Bigr{)}.$

The right side is $O(n^{\beta})$, if $\sigma$ is bounded away from 0 and 1.

Let $\bar{H}_{n}$ be defined as $H_{n}$, but with the integral extended from
$(0,n)$ to the full half line $(0,\infty)$. The assumption on $\alpha$ does
not specify $L(u)$ for $u\leq 1$, and hence does not specify $L(n/s)$ for
$s\geq n$, but we can extend the function $L$ to a continuously differentiable
function on $(0,\infty)$ in such a way that it vanishes on a neighbourhood of
$0$ and hence is uniformly bounded on $[0,1]$. Then
$|H_{n}(\sigma)-\bar{H}_{n}(\sigma)|\lesssim\int_{n}^{\infty}s^{-\gamma}e^{-s}(1+s^{-1}(e^{s}-1))\,ds=O(n^{-\gamma})$,
if $\sigma$ is bounded away from 0 and 1.

Splitting the two parts of the integrand in $\bar{H}_{n}$ and performing
partial integration on the second part we find

$\displaystyle\bar{H}_{n}(\gamma)$
$\displaystyle=\int_{0}^{\infty}L\Bigl{(}\frac{n}{s}\Bigr{)}\frac{s^{-\gamma}}{\gamma}e^{-s}\,ds-\int_{0}^{\infty}L\Bigl{(}\frac{n}{s}\Bigr{)}\sum_{m=1}^{\infty}\frac{s^{m-\gamma-1}}{m!}\,e^{-s}\,ds$
$\displaystyle\qquad\qquad\qquad\qquad\qquad\qquad+\int_{0}^{\infty}L^{\prime}\Bigl{(}\frac{n}{s}\Bigr{)}\frac{n}{s^{2}}\sum_{m=1}^{\infty}\frac{s^{m-\gamma}}{m!(m-\gamma)}\,e^{-s}\,ds$
$\displaystyle=\int_{0}^{\infty}L\Bigl{(}\frac{n}{s}\Bigr{)}\frac{s^{-\gamma}}{\gamma}e^{-s}\,ds+\int_{0}^{\infty}L\Bigl{(}\frac{n}{s}\Bigr{)}(1-e^{-s})\,d\Bigl{(}\frac{s^{-\gamma}}{\gamma}\Bigr{)}$
$\displaystyle\qquad\qquad\qquad\qquad\qquad\qquad\qquad+\int_{0}^{\infty}L^{\prime}\Bigl{(}\frac{n}{s}\Bigr{)}\frac{n}{s^{2}}\sum_{m=1}^{\infty}\frac{s^{m-\gamma}}{m!(m-\gamma)}\,e^{-s}\,ds$
$\displaystyle=0+\int_{0}^{\infty}L^{\prime}\Bigl{(}\frac{n}{s}\Bigr{)}\frac{n}{s^{2}}\Bigl{[}(1-e^{-s})\frac{s^{-\gamma}}{\gamma}+\sum_{m=1}^{\infty}\frac{s^{m-\gamma}e^{-s}}{m!(m-\gamma)}\Bigr{]}\,ds$
$\displaystyle\sim
L^{\prime}(n)n\int_{0}^{\infty}\frac{1}{s}\Bigl{[}(1-e^{-s})\frac{s^{-\gamma}}{\gamma}+\sum_{m=1}^{\infty}\frac{s^{m-\gamma}e^{-s}}{m!(m-\gamma)}\Bigr{]}\,ds,$

since the function $s\mapsto L^{\prime}(s)s$ is slowly varying at infinity, so
that $L^{\prime}(n/s)n/s\sim L^{\prime}(n)n$ as $n\rightarrow\infty$, for
every $s>0$. To justify the last step we can use Potter’s theorem ([2],
Theorem 1.5.6) as before, to infer for every $\delta>0$ the existence of a
constant $M>1$ such that $L^{\prime}(n/s)n/s/(L^{\prime}(n)n)\lesssim
s^{\delta}\vee s^{-\delta}$, for $s<n/M$ and $n\geq M$, so that on this
interval the integrand is dominated by a multiple of $(s^{\delta}\vee
s^{-\delta})\bigl{[}s^{-1}(1-e^{-s})s^{-\gamma}+\sum_{m=1}s^{m-\gamma}e^{-s}/(m+1)!\bigr{]}\lesssim
s^{-\gamma-\delta}\wedge s^{-1-\gamma+\delta}$, which is integrable for
sufficiently small $\delta>0$. Furthermore, for $s>n/M$, the function
$|L^{\prime}(n/s)n/s|$ is uniformly bounded, whence the integral over the
interval $[n/M,\infty)$ is bounded above by a multiple of
$\int_{n/M}^{\infty}s^{-1-\gamma}\,ds\lesssim n^{-\gamma}\ll L^{\prime}(n)n$.

By the identities obtained in the beginning of the proof, the integral in the
right of the preceding display is identical to
$\Gamma(1-\gamma)/\gamma^{2}+\Gamma(1-\gamma)/\gamma=\Gamma(1-\gamma)(1+\gamma)/\gamma^{2}$.

Combining the preceding, we find that
$E_{n}(\gamma)=n^{\gamma}\bigl{(}\bar{H}_{n}(\gamma)+O(n^{-\gamma})\bigr{)}+O(n^{\beta})=n^{\gamma}L^{\prime}(n)n+O(n^{\beta})$
and hence
$E_{n}(\gamma)/\alpha(n)=O(n^{\gamma}L^{\prime}(n)n/\alpha(n))+O(n^{\beta}/\alpha(n))$.
∎

######  Lemma 9.

Suppose that $\alpha(u):=\\#\\{j:1/p_{j}\leq u\\}$ is regularly varying at
$\infty$ of order $\gamma\in(0,1)$. Then for any
$\sigma_{n}\rightarrow\sigma\in(0,1)$, and independent
$M_{n,j}\sim\text{Binomial}(n,p_{j})$, and $k_{n}\geq n$ with
$k_{n}-n=O(\sqrt{n})$,

$\displaystyle\sum_{j=1}^{\infty}\mathbb{E}(1_{M_{k_{n},j}\geq
1}-1_{M_{n,j}\geq 1})$ $\displaystyle=o\bigl{(}\alpha(n)^{1/2}\bigr{)},$

Furthermore, if there exists a continuously differentiable function
$L:[1,\infty)\to\mathbb{R}$ such that $|\alpha(u)-u^{\gamma}L(u)|\leq
Cu^{\beta}$, for every $u>1$, and some $C>0$ and $\beta<\gamma$, and
$|L^{\prime}(u)|\leq C_{\delta}u^{-1+\delta}$, for every $u>1$ and $\delta>0$
and some $C_{\delta}>0$, then

$\displaystyle\sum_{j=1}^{\infty}\mathbb{E}\bigl{(}g_{\sigma_{n}}(M_{k_{n},j})-g_{\sigma_{n}}(M_{n,j})\bigr{)}$
$\displaystyle=o\bigl{(}\alpha(n)^{1/2}\bigr{)}.$

###### Proof.

Because $\Pr(M_{n,j}=0)=(1-p_{j})^{n}$, the left side of the first assertion
is equal to

$\sum_{j=1}^{\infty}\bigl{(}(1-p_{j})^{n}-(1-p_{j})^{k_{n}}\bigr{)}=\int_{1}^{\infty}\Bigl{(}1-\frac{1}{u}\Bigr{)}^{n}\Bigl{(}1-\Bigl{(}1-\frac{1}{u}\Bigr{)}^{k_{n}-n}\Bigr{)}\,d\alpha(u).$

By the inequalities $1-x\leq e^{-x}$, for $x\in\mathbb{R}$, and
$1-(1-x)^{r}\leq rx$, for $x\in[0,1]$ and $r\in\mathbb{N}$, this is bounded
above by

$\int_{1}^{\infty}e^{-n/u}(k_{n}-n)\frac{1}{u}\,d\alpha(u)=\frac{k_{n}-n}{n}\int_{0}^{n}\alpha\Bigl{(}\frac{n}{s}\Bigr{)}e^{-s}(1-s)\,ds,$

by Fubini’s theorem, since $e^{-n/u}(n/u)=\int_{0}^{n/u}e^{-s}(1-s)\,ds$. As
in the proof of Lemma 7, the integral is
$\alpha(n)\bigl{(}\Gamma(1-\gamma)-\Gamma(2-\gamma)\bigr{)}(1+o(1))$.
Therefore, the preceding display divided by $\alpha(n)^{1/2}$ is of the order
$\alpha(n)^{1/2}(k_{n}-n)/n\sim\alpha(n)^{1/2}n^{-1/2}$. This tends to zero,
as for every $\delta>0$ we have that $\alpha(n)\leq n^{\gamma+\delta}$
eventually, by Potter’s theorem, where $\gamma<1$ by assumption.

To prove the second assertion we first write

$\displaystyle\sum_{j=1}^{\infty}\mathbb{E}g_{\sigma_{n}}(M_{n,j})$
$\displaystyle=\sum_{j=1}^{\infty}\sum_{m=2}^{n}g_{\sigma}(m)\binom{n}{m}p_{j}^{m}(1-p_{j})^{n-m}$
$\displaystyle=\sum_{m=2}^{n}g_{\sigma}(m)\binom{n}{m}\int_{1}^{\infty}\Bigl{(}\frac{1}{u}\Bigr{)}^{m}\Bigl{(}1-\frac{1}{u}\Bigr{)}^{n-m}\,d\alpha(u).$

Writing $(1/u)^{m}(1-1/u)^{n-m}=\int_{0}^{1/u}s^{m-1}(1-s)^{n-m-1}(m-ns)\,ds$
(for $m>0$) and applying Fubini’s theorem, we can rewrite this as

$\displaystyle\sum_{m=2}^{n}g_{\sigma}(m)\binom{n}{m}\int_{0}^{1}\alpha\Bigl{(}\frac{1}{s}\Bigr{)}s^{m-1}(1-s)^{n-m-1}(m-ns)\,ds$
$\displaystyle\qquad=\int_{0}^{1}\sum_{l=1}^{n-1}\frac{1}{l-\sigma}\sum_{m=l+1}^{n}\binom{n}{m}s^{m-1}(1-s)^{n-m-1}(m-ns)\,\alpha\Bigl{(}\frac{1}{s}\Bigr{)}\,ds$
$\displaystyle\qquad=\int_{0}^{1}\sum_{l=1}^{n-1}\frac{n-l}{l-\sigma}\binom{n}{l}s^{l}(1-s)^{n-l-1}\,\alpha\Bigl{(}\frac{1}{s}\Bigr{)}\,ds,$

by Lemma 12. Thus the left side of the second assertion can be written in the
form, with $k=k_{n}$,

$\displaystyle\int_{0}^{1}\sum_{l=1}^{n-1}\frac{s^{l}(1-s)^{n-l-1}}{l-\sigma}\biggl{[}(k-l)\binom{k}{l}(1-s)^{k-n}-(n-l)\binom{n}{l}\biggr{]}\,\alpha\Bigl{(}\frac{1}{s}\Bigr{)}\,ds$
$\displaystyle\qquad\qquad+\int_{0}^{1}\sum_{l=n+1}^{k}\frac{k-l}{l-\sigma}\binom{k}{l}s^{l}(1-s)^{k-l-1}\,\alpha\Bigl{(}\frac{1}{s}\Bigr{)}\,ds.$
(16)

Because $\alpha(1/s)\leq 1/s$, the second term is bounded above by
$\sum_{l>n}(k-l)/(l-\sigma)\binom{k}{l}B(l,k-l)$, for $B$ the beta function.
This is further bounded by $k\sum_{l>n}1/((l-\sigma)l)\lesssim k/n\lesssim 1$.

By the assumption that $|\alpha(u)-u^{\gamma}L(u)|\leq Cu^{\beta}$, if in the
first term, we replace $\alpha(1/s)$ by $s^{-\gamma}L(1/s)$, the error is
bounded above by a multiple of

$\int_{0}^{1}\sum_{l=1}^{n-1}\frac{s^{l}(1-s)^{n-l-1}}{l-\sigma}\Bigl{|}(k-l)\binom{k}{l}(1-s)^{k-n}-(n-l)\binom{n}{l}\Bigr{|}\,s^{-\beta}\,ds.$

The sum of the terms with $l>\sqrt{n}$ is bounded above by $a_{k,n}+a_{n,n}$,
for

$a_{k,n}=\sum_{l>\sqrt{n}}\frac{k-l}{l-\sigma}\binom{k}{l}B(l-\beta+1,k-l)\lesssim\sum_{l>\sqrt{n}}\frac{\Gamma(l-\beta+1)}{(l-\sigma)l!}\frac{k!}{\Gamma(k-\beta+1)}.$

In view of Lemma 13, $a_{k,n}$ is bounded above by a multiple of
$\sqrt{n}^{-\beta}k^{\beta}=O(n^{\beta/2})=o\bigl{(}\alpha(n)^{1/2}\bigr{)}$.
In the sum of the terms with $l\leq\sqrt{n}$, we decompose $k-l=(k-n)+(n-l)$
and $\binom{k}{l}=\sum_{i}\binom{n}{l-i}\binom{k-n}{i}$, and bound

$\displaystyle\Bigl{|}(k-l)\binom{k}{l}(1-s)^{k-n}-(n-l)\binom{n}{l}\Bigr{|}\leq(k-n)\binom{k}{l}(1-s)^{k-n}$
$\displaystyle\qquad\qquad\qquad+(n-l)\Bigl{[}\binom{n}{l}\bigl{(}1-(1-s)^{k-n}\bigr{)}+\sum_{i\geq
1}\binom{n}{l-i}\binom{k-n}{i}\Bigr{]}.$

The middle term in the right is bounded above by $(n-l)\binom{n}{l}(k-n)s$.
Thus the sum of the terms with $l\leq\sqrt{n}$ is bounded above by the sum of
the three integrals

$\displaystyle\int_{0}^{1}\sum_{l\leq\sqrt{n}}\frac{s^{l}(1-s)^{n-l-1}}{l-\sigma}(k-n)\binom{k}{l}(1-s)^{k-n}s^{-\beta}ds$
$\displaystyle=\sum_{l\leq\sqrt{n}}\frac{B(l+1-\beta,k-l)}{l-\sigma}(k-n)\binom{k}{l}=\sum_{l\leq\sqrt{n}}\frac{\Gamma(l+1-\beta)}{(l-\sigma)l!}\frac{\Gamma(k-l)}{(k-l)!}\frac{(k-n)k!}{\Gamma(k+1-\beta)}$
$\displaystyle\qquad\lesssim\sum_{l\leq\sqrt{n}}\frac{1}{l^{1+\beta}}\frac{k-n}{k-l}k^{\beta}\lesssim
n^{\beta-1/2}\leq n^{\beta/2},$
$\displaystyle\int_{0}^{s}\sum_{l\leq\sqrt{n}}\frac{s^{l}(1-s)^{n-l-1}}{l-\sigma}(n-l)\binom{n}{l}(k-n)ss^{-\beta}ds$
$\displaystyle=\sum_{l\leq\sqrt{n}}\frac{B(l+2-\beta,n-l)}{l-\sigma}(k-n)(n-l)\binom{n}{l}=\sum_{l\leq\sqrt{n}}\frac{\Gamma(l+2-\beta)}{(l-\sigma)l!}\frac{(k-n)n!}{\Gamma(n+2-\beta)}$
$\displaystyle\qquad\lesssim\sqrt{n}^{1-\beta}(k-n)n^{\beta-1}\lesssim
n^{\beta/2},$
$\displaystyle\int_{0}^{1}\sum_{l\leq\sqrt{n}}\frac{s^{l}(1-s)^{n-l-1}}{l-\sigma}(n-l)\sum_{i}\binom{n}{l-i}\binom{k-n}{i}s^{-\beta}ds$
$\displaystyle=\sum_{l\leq\sqrt{n}}\sum_{i\geq
1}\frac{B(l+1-\beta,n-l)}{l-\sigma}(n-l)\binom{n}{l-i}\binom{k-n}{i}$
$\displaystyle\qquad=\sum_{l\leq\sqrt{n}}\sum_{i\geq
1}\frac{\Gamma(l+1-\beta)}{(l-\sigma)(l-i)!}\frac{(n-l)!}{(n-l+i)!}\frac{n!}{\Gamma(n+1-\beta)}\binom{k-n}{i}$
$\displaystyle\qquad\lesssim\sum_{l\leq\sqrt{n}}\sum_{i\geq
1}l^{i-1-\beta}\frac{1}{(n/2)^{i}}n^{\beta}\binom{k-n}{i}\leq\sum_{i\geq
1}\frac{\sqrt{n}^{i-\beta}}{(n/2)^{i}}n^{\beta}\binom{k-n}{i}$
$\displaystyle\qquad\leq\Bigl{(}1+\frac{2}{\sqrt{n}}\Bigr{)}^{k-n}n^{\beta/2}\lesssim
n^{\beta/2}.$

We conclude that replacing $\alpha(1/s)$ by $s^{-\gamma}L(1/s)$ in the first
part of (16) makes a difference of at most of the order
$n^{\beta/2}=o\bigl{(}\alpha(n)^{1/2}\bigr{)}$. Finally, we consider the
expression

$\displaystyle\int_{0}^{1}\sum_{l=1}^{n-1}\frac{s^{l}(1-s)^{n-l-1}}{l-\sigma}\biggl{[}(k-l)\binom{k}{l}(1-s)^{k-n}-(n-l)\binom{n}{l}\biggr{]}\,s^{-\gamma}L\Bigl{(}\frac{1}{s}\Bigr{)}\,ds$
$\displaystyle\qquad=\sum_{l=1}^{n-1}\frac{\Gamma(l+1-\gamma)}{(l-\sigma)l!}\Big{[}\frac{k!}{\Gamma(k+1-\gamma)}\mathbb{E}L(1/S_{l,k})-\frac{n!}{\Gamma(n+1-\gamma)}\mathbb{E}L(1/S_{l,n})\Bigr{]},$

where $S_{l,k}$ is a random variable with the beta distribution with
parameters $l+1-\gamma$ and $k-l$. The bound $|L^{\prime}(u)|\leq
C_{\delta}u^{-1+\delta}$ gives that $L(u)\lesssim u^{\delta}$, and hence
$|\mathbb{E}L(1/S_{l,k})|\lesssim\mathbb{E}S_{l,k}^{-\delta}=B(l+1-\gamma-\delta,k-l)/B(l+1-\gamma,k-l)\lesssim
k^{\delta}$. Therefore, after bounding the difference with the help of the
triangle inequality, the sum of the terms with $l>m$, can be bounded by
$b_{k,m}+b_{n,m}$, for

$b_{k,m}=\sum_{l>m}\frac{\Gamma(l+1-\gamma)}{(l-\sigma)l!}\frac{k!}{\Gamma(k+1-\gamma)}k^{\delta}\lesssim\Bigl{(}\frac{k}{m}\Bigr{)}^{\gamma}k^{\delta}.$

For $m=n^{1/2+\delta/\gamma+\eta}$, the right side is of the order
$n^{\gamma/2-\eta\gamma}=o\bigl{(}\alpha(n)^{1/2}\bigr{)}$, for any $\eta>0$.
The sum of the terms with $l\leq m$ is bounded above by

$\displaystyle\sum_{l=1}^{m}\frac{\Gamma(l+1-\gamma)}{(l-\sigma)l!}\Big{[}\frac{k!}{\Gamma(k+1-\gamma)}-\frac{n!}{\Gamma(n+1-\gamma)}\Bigr{]}\mathbb{E}L(1/S_{l,k})$
$\displaystyle\qquad\qquad\qquad\qquad\qquad\qquad\qquad+\frac{n!}{\Gamma(n+1-\gamma)}\Bigl{[}\mathbb{E}L(1/S_{l,k})-\mathbb{E}L(1/S_{l,n})\Bigr{]}$
$\displaystyle\qquad\lesssim\sum_{l=1}^{m}\frac{1}{l^{1+\gamma}}\Bigl{[}k^{\gamma}-n^{\gamma}+O\Bigl{(}\frac{1}{n}\Bigr{)}\Bigr{]}k^{\delta}+\sum_{l=1}^{m}\frac{n^{\gamma}}{l^{1+\gamma}}\Bigl{|}\mathbb{E}L(1/S_{l,k})-\mathbb{E}L(1/S_{l,n})\Bigr{|}.$

Here
$k^{\gamma}-n^{\gamma}=n^{\gamma}\bigl{(}(1+(k-n)/n)^{\gamma}-1\bigr{)}\lesssim
n^{\gamma-1/2}$, so that the first term is of the order
$n^{\gamma-1/2}k^{\delta}=o\bigl{(}\alpha(n)^{1/2}\bigr{)}$, for sufficiently
small $\delta>0$, and hence is asymptotically negligible. For the second term
we represent $S_{l,k}$ and $S_{l,n}$ using independent, gamma variables
$\bar{\Gamma}_{l}$, $\Gamma_{n-l}$ and $\Gamma_{k-n}$ with shape parameters
$l+1-\gamma$, $n-l$ and $k-n$, and write
$|\mathbb{E}L(1/S_{l,k})-\mathbb{E}L(1/S_{l,n})|$ as

$\displaystyle\Bigl{|}\mathbb{E}L\Bigl{(}\frac{\bar{\Gamma}_{l}+\Gamma_{n-l}+\Gamma_{k-n}}{\bar{\Gamma}_{l}}\Bigr{)}-\mathbb{E}L\Bigl{(}\frac{\bar{\Gamma}_{l}+\Gamma_{n-l}}{\bar{\Gamma}_{l}}\Bigr{)}\Bigr{|}$
$\displaystyle\quad=\Bigl{|}\mathbb{E}\int_{0}^{\Gamma_{k-n}}L^{\prime}\Bigl{(}\frac{\bar{\Gamma}_{l}+\Gamma_{n-l}+u}{\bar{\Gamma}_{l}}\Bigr{)}\frac{1}{\bar{\Gamma}_{l}}\,du\Bigr{|}$
$\displaystyle\quad\lesssim\int_{0}^{\infty}\Pr(\Gamma_{k-n}>u)\mathbb{E}\frac{1}{(\bar{\Gamma}_{l}+\Gamma_{n-l}+u)^{1-\delta}}\frac{1}{\bar{\Gamma}_{l}^{\delta}}\,du\leq\mathbb{E}\Gamma_{k-n}\mathbb{E}\frac{1}{\Gamma_{n-l}^{1-\delta}}\mathbb{E}\frac{1}{\bar{\Gamma}_{l}^{\delta}}.$

The three expecations can be computed explicitly in terms of the gamma
function. Substituting the resulting expressions in the second sum of the
second last display, we see that this is bounded above by

$\sum_{l=1}^{m}\frac{n^{\gamma}(k-n)}{l^{1+\gamma}}\frac{\Gamma(n-l-1+\delta)}{\Gamma(n-l)}\frac{\Gamma(l+1-\gamma-\delta)}{\Gamma(l+1-\gamma)}\lesssim
n^{\gamma}(k-n)\sum_{l=1}^{m}\frac{1}{l^{1+\gamma+\delta}(n-l)^{1-\delta}}.$

Since the summation indices satisfy $l\leq m\ll n$, so that $n-l\gtrsim n/2$,
this is of the order
$n^{-1/2+\gamma+\delta}=o\bigl{(}\alpha(n)^{1/2}\bigr{)}$, for sufficiently
small $\delta$. ∎

###  3.6 Supporting lemmas

######  Lemma 10.

Suppose that $V_{k,n}$, for $k,n\in\mathbb{N}$, are random variables
independently of random variables $N_{n}\sim\text{Poisson}(n)$ so that
$a_{n}(V_{N_{n},n}-E_{n})\rightsquigarrow N(0,\tau^{2})$ and
$a_{n}(V_{k_{n},n}-V_{n,n})\rightarrow 0$ in probability for every $k_{n}$
with $|k_{n}-n|=O(\sqrt{n})$, for $n\rightarrow\infty$ and given numbers
$a_{n}$ and $E_{n}$. Then $a_{n}(V_{n,n}-E_{n})\rightsquigarrow
N(0,\tau^{2})$.

###### Proof.

For any Lipschitz function $h:\mathbb{R}\to[0,1]$ and $k_{n}$ as given, as
$n\rightarrow\infty$,

$\Bigl{|}\mathbb{E}h\bigl{(}a_{n}(V_{k_{n},n}-E_{n})\bigr{)}-\mathbb{E}h\bigl{(}a_{n}(V_{n,n}-E_{n})\bigr{)}\Bigr{|}\leq\mathbb{E}a_{n}|V_{k_{n},n}-V_{n,n}|\wedge
1\rightarrow 0.$

By the central limit theorem the probability
$\Pr\bigl{(}|N_{n}-n|>\sqrt{n}M\bigr{)}$ can be made arbitrarily small
uniformly in $n$ by choosing sufficiently large $M$. Then
$\mathbb{E}h\bigl{(}a_{n}(V_{n,n}-E_{n})\bigr{)}$ is arbitrarily close to

$\displaystyle\mathbb{E}h\bigl{(}a_{n}(V_{n,n}-E_{n})\bigr{)}\sum_{k:|k-n|\leq\sqrt{n}M}\Pr(N_{n}=k)$
$\displaystyle\qquad\qquad=\sum_{k:|k-n|\leq\sqrt{n}M}\mathbb{E}h\bigl{(}a_{n}(V_{k,n}-E_{n})\bigr{)}\Pr(N_{n}=k)+o(1),$

by the preceding display, as $n\rightarrow\infty$, for every fixed $M$. The
sum in the right side is arbitrarily close to
$\mathbb{E}h\bigl{(}a_{n}(V_{N_{n},n}-E_{n})\bigr{)}$ uniformly in $n$, if $M$
is chosen sufficiently large, which tends to $\mathbb{E}h(\tau Z)$, for $Z\sim
N(0,1)$ as $n\rightarrow\infty$, by assumption. We conclude that
$\mathbb{E}h\bigl{(}a_{n}(V_{n,n}-E_{n})\bigr{)}$ is arbitrarily close to
$\mathbb{E}h(\tau Z)$, as $n\rightarrow\infty$. ∎

######  Lemma 11.

If $X_{n,1},X_{n,2},\ldots$ are independent random variables with
$s_{n}^{2}:=\sum_{j=1}^{\infty}\mathop{\rm var}\nolimits X_{n,j}<\infty$ and
$s_{n}^{-2}\sum_{j=1}^{\infty}\mathbb{E}X_{n,j}^{2}1_{|X_{n,j}|>\epsilon
s_{n}}\rightarrow 0$, for every $\epsilon>0$, then
$\sum_{j=1}^{\infty}(X_{n,j}-\mathbb{E}X_{n,j})/s_{n}\rightsquigarrow N(0,1)$.

###### Proof.

The variables $Y_{n,j}=(X_{n,j}-\mathbb{E}X_{n,j})/s_{n}$ have mean zero and
$\sum_{j}\mathbb{E}Y_{n,j}^{2}=1$. Choose integers $k_{n}\uparrow\infty$ such
that $\sum_{j\leq k_{n}}\mathbb{E}Y_{n,j}^{2}\uparrow 1$. Then the sequence
$\sum_{j>k_{n}}Y_{n,j}$ tends to zero in second mean and hence it suffices to
show that $\sum_{j\leq k_{n}}Y_{n,j}\rightsquigarrow N(0,1)$. The latter
follows from the Lindeberg central limit theorem provided that $\sum_{j\leq
k_{n}}\mathbb{E}Y_{n,j}^{2}1_{|Y_{n,j}|>\epsilon}\rightarrow 0$, for every
$\epsilon>0$. To see that this is satisfied, first note that the Lindeberg
condition implies that $\max_{j}\mathbb{E}X_{n,j}^{2}/s_{n}^{2}\rightarrow 0$
and hence both
$s_{n}^{-2}\mathbb{E}\sum_{j}|\mathbb{E}X_{n,j}|^{2}1_{|X_{n,j}|>\epsilon
s_{n}}\leq o(1)\sum_{j}\Pr(|X_{n,j}|>\epsilon s_{n})\rightarrow 0$ and
$|\mathbb{E}X_{n,j}|\leq\epsilon s_{n}$, for every $j$ eventually, for every
fixed $\epsilon>0$. This shows that the variables $X_{n,j}$ also satisfy the
centered “infinite Lindeberg condition”
$\sum_{j}\mathbb{E}Y_{n,j}^{2}1_{|Y_{n,j}|>\epsilon}\rightarrow 0$, for every
$\epsilon>0$, which implies the Lindeberg condition for the finite array
$Y_{n,1},\ldots,Y_{n,k_{n}}$. ∎

######  Lemma 12.

For every $p\in[0,1]$ and $l\in\mathbb{N}\cup\\{0\\}$ and $n\in\mathbb{N}$,

$\sum_{m=l+1}^{n}\binom{n}{m}p^{m-1}(1-p)^{n-m-1}(m-np)=(n-l)\binom{n}{l}p^{l}(1-p)^{n-l-1}.$

###### Proof.

For $X_{n-1}$ and $X_{n}$ the numbers of successes in the first $n-1$ and $n$
independent Bernoulli trials with success probability $p$, we have
$\\{X_{n}\geq l+1\\}\subset\\{X_{n-1}\geq l\\}$ and $\\{X_{n-1}\geq
l\\}-\\{X_{n}\geq l+1\\}=\\{X_{n-1}=l,B_{n}=0\\}$, for $B_{n}$ the outcome of
the $n$th trial. This gives the identity $\Pr(X_{n-1}\geq l)-\Pr(X_{n}\geq
l+1)=\Pr(X_{n-1}=l)(1-p)$. We multiply this by $n/(1-p)$ to obtain the
identity given by the lemma, which we first rewrite using that
$m\binom{n}{m}=n\binom{n-1}{m-1}$ and $(n-l)\binom{n}{l}=n\binom{n-1}{l}$. ∎

######  Lemma 13.

For every $\gamma\in(0,1)$, as $n\rightarrow\infty$,

$\frac{\Gamma(n-\gamma)n^{\gamma}}{\Gamma(n)}=1+O\Bigl{(}\frac{1}{n}\Bigr{)}.$

###### Proof.

By Stirling’s approximation, the quotient is

$\displaystyle\frac{\sqrt{{2\pi}/{(n-\gamma)}}\left(\frac{n-\gamma}{e}\right)^{n-\gamma}\bigl{(}1+O(1/n)\bigr{)}n^{\gamma}}{\sqrt{{2\pi}/{n}}\left(\frac{n}{e}\right)^{n}\bigl{(}1+O(1/n)\bigr{)}}$
$\displaystyle=\Bigl{(}\frac{n-\gamma}{n}\Bigr{)}^{n}e^{\gamma}\Bigl{(}1+O\Bigl{(}\frac{1}{n}\Bigr{)}\Bigr{)}$
$\displaystyle=\Bigl{(}e^{-\gamma}+O\Bigl{(}\frac{1}{n}\Bigr{)}\Bigr{)}e^{\gamma}\Bigl{(}1+O\Bigl{(}\frac{1}{n}\Bigr{)}\Bigr{)}.$

∎

######  Lemma 14.

If $\alpha:(1,\infty)\to\mathbb{R}$ satisfies $|\alpha(u)-u^{\gamma}L(u)|\leq
Cu^{\beta}$, for $u>1$ and $\beta<\gamma$ and a slowly varying function $L$,
then $\alpha$ is regularly varying of order $\gamma$.

###### Proof.

Since $L$ is slowly varying, we have $L(n)\gtrsim n^{-\delta}$, for every
$\delta>0$, and hence $n^{\gamma}L(n)\gtrsim n^{\gamma-\delta}\gg n^{\beta}$,
for sufficiently small $\delta>0$. Since $|\alpha(n)-n^{\gamma}L(n)|\lesssim
n^{\beta}$ by assumption, it follows that $\alpha(n)\gg n^{\beta}$ and
$n^{\gamma}L(n)/\alpha(n)\rightarrow 1$, as $n\rightarrow\infty$. By the
assumption we have
$\alpha(nu)=(nu)^{\gamma}L(nu)+O(n^{\beta})=(nu)^{\gamma}L(n)(1+o(1))+O(n^{\beta})$,
for every $u$ as $n\rightarrow\infty$, since $L$ is slowly varying, and hence
$\alpha(nu)/\alpha(n)=u^{\gamma}\bigl{(}n^{\gamma}L(n)/\alpha(n)\bigr{)}(1+o(1))+O(n^{\beta}/\alpha(n))\rightarrow
u^{\gamma}$. ∎

######  Lemma 15.

For every $s\geq 0$ and $\delta\in(0,1]$:

  * (i)

$\sum_{m=1}^{\infty}s^{m}m^{\delta}/m!\leq s^{\delta}e^{s}$,

  * (ii)

$\sum_{m=2}^{\infty}s^{m}m^{\delta}/m!\leq s^{\delta}(e^{s}-1)$.

###### Proof.

Since $s^{m}m^{\delta}/m!=(s^{m}m/m!)^{\delta}(s^{m}/m!)^{1-\delta}$, Hölder’s
inequality with $p=1/\delta$ and $q=1/(1-\delta)$ gives that the sums on the
left are bounded above by
$\bigl{(}\sum_{m}(s^{m}m/m!)\bigr{)}^{\delta}\bigl{(}\sum_{m}(s^{m}/m!)\bigr{)}^{1-\delta}$,
where the summation starts at $m=1$ for (i) and at $m=2$ for (ii). In the case
of (i) the first series is bounded above by $se^{s}$ and the second by
$e^{s}-1$, while in the case of (ii) the bounds $s(e^{s}-1)$ and $e^{s}-1-s$
pertain. ∎

######  Lemma 16.

For $K\rightarrow\infty$, we have $\sum_{i=1}^{K-1}\log(M+i\sigma)=K\log
K+K\log(\sigma/e)+(M/\sigma-1/2)\log
K+\log(\sqrt{2\pi}/\sigma)-\log\Gamma(1+M/\sigma)+O(1/K)$, where the remainder
is bounded above by a universal multiple of $(M/\sigma+1)^{2}/K$, for all
$M\geq 0,\sigma>0$.

###### Proof.

The sum is equal to
$(K-1)\log\sigma+\log\Gamma(K+M/\sigma)-\log\Gamma(1+M/\sigma)$. By the
expansion for the log Gamma function, the middle term can be expanded as

$\log\Gamma\Bigl{(}K+\frac{M}{\sigma}\Bigr{)}=\Bigl{(}K+\frac{M}{\sigma}-\frac{1}{2}\Bigr{)}\log\Bigl{(}K+\frac{M}{\sigma}\Bigr{)}-\Bigl{(}K+\frac{M}{\sigma}\Bigr{)}+\log\sqrt{2\pi}+O\Bigl{(}\frac{1}{K}\Bigr{)},$

where the remainder term is uniform in $M$ and $\sigma$. Next expand
$\log(K+M/\sigma)$ as $\log K+M/(\sigma K)+O((M/\sigma)^{2}/K^{2})$. Finally
we collect terms. ∎

## References

  * [1] Arbel, J., De Blasi, P., and Prünster, I. Stochastic approximations to the Pitman-Yor process.  Bayesian Analysis (2018).  Advance publication.
  * [2] Bingham, N. H., Goldie, C. M., and Teugels, J. L. Regular variation, vol. 27 of Encyclopedia of Mathematics and its Applications.  Cambridge University Press, Cambridge, 1989.
  * [3] Cereda, G. Current challenges in statistical DNA evidence evaluation.  PhD thesis, Leiden University, 2017.
  * [4] Cereda, G., and Gill, R. D. A nonparametric bayesian approach to the rare type match problem, 2019\.
  * [5] De Blasi, P., Lijoi, A., and Prünster, I. An asymptotic analysis of a class of discrete nonparametric priors.  Statistica Sinica 23, 3 (2013), 1299–1321.
  * [6] de Blasi et al. Are Gibbs-type priors the most natural generalization of the Dirichlet process?  IEEE Transactions on Pattern Analysis and Machine Intelligence 37, 2 (2015), 212–229.
  * [7] de Haan, L., and Ferreira, A. Extreme value theory.  Springer Series in Operations Research and Financial Engineering. Springer, New York, 2006.  An introduction.
  * [8] Favaro, S., and Naulet, Z. Near-optimal estimation of the unseen under regularly varying tail populations, 2021.
  * [9] Ferguson, T. Prior distributions on spaces of probability measures.  Ann. Statist. 2 (1974), 615–629.
  * [10] Franssen, S. E. M. P., and van der Vaart, A. W. The bernstein-von mises theorem for the pitman-yor process of nonnegative type, 2020.
  * [11] Ghosal, S., and van der Vaart, A. Fundamentals of Nonparametric Bayesian Inference.  Cambridge University Press, 2017.
  * [12] Goldwater, S., Griffiths, T. L., and Johnson, M. Interpolating between types and tokens by estimating power-law generators.  In Advances in neural information processing systems (2005).
  * [13] Ishwaran, H., and James, L. F. Gibbs sampling methods for stick-breaking priors.  Journal of the American Statistical Association 96, 453 (2001), 161–173.
  * [14] James, L. Large sample asymptotics for the two-parameter Poisson-Dirichlet process.  Pushing the limits of contemporary Statistics: Contributions in Honor of Jayanta k. Ghosh 3 (2008).
  * [15] Karlin, S. Central limit theorems for certain infinite urn schemes.  Journal of Mathematics and Mechanics 17, 4 (1967).
  * [16] Kleijn, B. J. K., and van der Vaart, A. W. The Bernstein-Von-Mises theorem under misspecification.  Electron. J. Stat. 6 (2012), 354–381.
  * [17] Perman, M., Pitman, J., and Yor, M. Size-biased sampling of Poisson point processes and excursions.  Probab. Theory Related Fields 92, 1 (1992), 21–39.
  * [18] Pitman, J. Exchangeable and partially exchangeable random partitions.  Probab. Theory Related Fields 102, 2 (1995), 145–158.
  * [19] Pitman, J. Some developments of the Blackwell-MacQueen urn scheme.  Institute of Mathematical Statistics Lecture Notes - Monograph Series 30 (1996), 245–267.
  * [20] Pitman, J. Poisson-Kingman partitions.  In Statistics and Science: a Festschrift for Terry Speed, vol. 40 of IMS Lecture Notes Monogr. Ser. Inst. Math. Statist., Beachwood, OH, 2003, pp. 1–34.
  * [21] Pitman, J., and Yor, M. The two-parameter Poisson-Dirichlet distribution derived from a stable subordinator.  Ann. Probab. 25, 2 (1997), 855–900.
  * [22] Teh, Y. W. A hierarchical Bayesian language model based on Pitman-Yor processes.  In ACL-44 Proceedings of the 21st International Conference on Computational Linguistics and the 44th annual meeting of the Association for Computational Linguistics (2006), pp. 985–992.
  * [23] Wood, F., Archambeau, C., Gasthaus, J., James, L., and Teh, Y. W. A stochastic memoizer for sequence data.  In Proceedings of the 26th Annual International Conference on Machine Learning (New York, NY, USA, 2009), ICML ’09, Association for Computing Machinery, p. 1129–1136.