# Trigonometrically approximated maximum likelihood estimation for stable law

Muneya Matsui  Department of Business Administration, Nanzan University, 18
Yamazato-cho Showa-ku Nagoya, 466-8673, Japan<EMAIL_ADDRESS>and
Naoya Sueishi  Graduate School of Economics, Kobe University, 2-1 Rokkodai-
cho, Nada-ku, Kobe, Hyogo, Japan<EMAIL_ADDRESS>
###### Abstract.

A trigonometrically approximated maximum likelihood estimation for
$\alpha$-stable laws is proposed. The estimator solves the approximated
likelihood equation, which is obtained by projecting a true score function on
the space spanned by trigonometric functions. The projected score is expressed
only by real and imaginary parts of the characteristic function and their
derivatives, so that we can explicitly construct the targeting estimating
equation. We study the asymptotic properties of the proposed estimator and
show consistency and asymptotic normality. Furthermore, as the number of
trigonometric functions increases, the estimator converges to the exact
maximum likelihood estimator, in the sense that they have the same asymptotic
law. Simulation studies show that our estimator outperforms other moment-type
estimators, and its standard deviation almost achieves the Cramér–Rao lower
bound. We apply our method to the estimation problem for $\alpha$-stable
Ornstein–Uhlenbeck processes in a high-frequency setting. The obtained result
demonstrates the theory of asymptotic mixed normality.
Key words.  characteristic function, stable distributions, Fisher information
matrix, maximum likelihood estimator, $\alpha$-stable Ornstein–Uhlenbeck
processes.

###### 2010 Mathematics Subject Classification:

Primary 60E07, 62E17; Secondary 62F12, 62E20

Matsui’s research is partly supported by the JSPS Grant-in-Aid for Scientific
Research C (19K11868). Sueishi thanks the support by the JSPS Grant-in-Aid for
Scientific Research B (19H01473)

##  1\. Introduction

This paper proposes a novel characteristic function (ch.f.)-based estimation
method called the trigonometrically approximated maximum likelihood estimation
(TMLE). Our approach relies on the approximation method of a score function in
Brant [1], where the approximation is done by projecting a true score on the
functional space spanned by trigonometric functions. By solving the
approximated score equation, we conduct TMLE. Because the projected score is
expressed by combinations of real and imaginary parts of the ch.f. and its
derivatives, we can easily calculate the approximated score solely from the
original ch.f. This method is effective for the class of distributions whose
ch.f.’s are available in closed form but whose densities are not. An example
is the class of infinitely divisible (ID) distributions, which provides finite
dimensional distributions for fundamental stochastic processes such as Lévy
processes and additive processes (see, e.g. Sato [44]).

Specifically we apply the proposed method to the estimation of stable laws,
which hold a prominent position among ID distributions. For details and
notable properties of stable laws, see Samorodnitsky and Taqqu [43], Nolan
[38] and references therein. We show that the trigonometrically approximated
maximum likelihood (TML) estimator converges to the true maximal likelihood
(ML) estimator as the number of trigonometric functions increases. Therefore,
TMLE takes over nice properties of the maximum likelihood estimation (MLE),
such as asymptotic normality and efficiency. Furthermore, we extend TMLE to an
estimation method suitable for the parameter estimation of $\alpha$-stable
Ornstein–Uhlenbeck processes. In what follows, we briefly introduce the
literature and explain features of our estimator by comparing it with previous
estimation methods.

The parameter estimation based on the ch.f. was originally proposed by Press
[41] and has been developed into various directions. A regression-type
estimator was proposed by Koutrouvelis [24, 25] and further investigated by
Kogan and Williams [22]. Paulson et al. [40] considered an estimator that
minimizes the squared integrated distance between the true ch.f. and the
empirical one. Heathcote [17] implemented the method and discussed its
efficiency and robustness. Feuerverger and McDunnough [13, 14] showed that
their moment-type estimator can attain arbitrary high efficiency. Notice that
some of the estimation methods are applicable not only for stable laws but
also for general distributions with explicit ch.f’s.

The estimator by [13, 14] can be viewed as the generalized method of moments
(GMM) estimator [15]. In the econometric literature, the GMM estimator and its
variants [16, 19, 21, 35, 39] are commonly used to achieve semiparametrically
efficient estimation. However, they are suboptimal for the estimation of the
stable law if they use only a fixed number of moment conditions, constructed
from the ch.f. Indeed, to attain high efficiency in the GMM estimator of [13]
the number of moment conditions should be increased. Kunitomo and Owada [26]
showed that the empirical likelihood (EL) estimator is asymptotically normal
and efficient if the number of moment conditions increases at a certain rate
with the sample size.

Although the GMM-type estimators satisfy desirable properties, they have two
deficiencies. First, the number of moment conditions can increase only at a
slow rate. For instance, in the EL estimator it must increase at the rate
$n^{1/3-\eta}$ for $\eta>0$, where $n$ is the sample size. Thus, the EL
estimator cannot achieve high efficiency in a finite sample. Second, previous
studies [4, 45, 48] point out that the weight matrix of the optimal GMM
estimator tends to be singular as the number of moment conditions increases.
Therefore, the GMM-type estimators are infeasible when the number of moment
conditions is relatively large compared to the sample size.

To overcome these problems, Carrasco and Florens [3] proposed a version of the
GMM estimator that utilizes a continuum of moment conditions (hereafter, the
CGMM estimator). See also [2, 4, 5]. The CGMM uses an uncountable number of
moment conditions from the beginning, which may cause the degeneracy of a
certain covariance operator. However, the degeneracy problem is avoided by
introducing regularization. Thus, the number of moment conditions is not
restricted by the sample size.

TMLE addresses the same issues with CGMM. As we will see in the next section,
the TML estimator has a close relation with the optimally weighted GMM
estimator. The number of trigonometric functions in TMLE plays the same role
as the number of moment conditions in GMM estimation. Moreover, to obtain the
approximated score function, we calculate a covariance matrix that is similar
to the weight matrix of the optimal GMM. The novelty of TMLE is that the
number of trigonometric functions can be increased regardless of the sample
size to satisfy asymptotic normality and efficiency. Moreover, the covariance
matrix of TMLE is non-singular even if its dimension is large. Furthermore,
our simulation shows that the TML estimator outperforms the CGMM estimator in
terms of finite sample efficiency.

As for the exact MLE, the asymptotic theory has been established earlier by
DuMouchel [11] and recently elaborated by Matsui [28] under continuous
parametrizations. Since the ch.f. is the Fourier transform of the density, the
likelihood function can be obtained by numerically evaluating the inversion
formula. However, one encounters several difficulties in practical
implementation. So far the methods of implementation have been studied only
intermittently. Started from an early work by DuMouchel [12], several attempts
have been made, especially in the symmetric case (see references in Nolan [38,
Section 4]). However, in the general non-symmetric case, there are only a few
studies [33, 37].

There are two issues in implementing MLE: computational complexity and
accuracy. There exists a clear trade-off relation between these two aspects.
For example, to ease the computational burden of the inversion formula for the
densities, Mittnik et al. [33] have examined MLE solely by the FFT-based
approximation of density functions. However, that limits the accuracy,
especially in the tail. Nolan [37] uses a precomputed spline approximation of
stable densities in doing MLE, which is equipped as the program “STABLE”. The
program is fast, but the values reported by it have limited accuracy in
several regions due to technical difficulties in approximation (see also
[42]). On the other hand, regarding the pursuit of accuracy, few papers have
rigorously validated the accuracy of their implemented ML estimators. Assuming
symmetry, Matsui [31] improved the accuracy of ML estimators and certified the
accuracy by comparing the true Fisher information matrix with the empirical
one calculated from simulated ML estimators, and have observed that the two
are almost identical. To obtain high accuracy, a finite integral
representation of Zolotarev [49] is used for the central part of the density
and an infinite series expansion of the density is applied for the tail part.
Although the procedure is highly accurate, its computational burden is quite
large. Besides one has to determine the regions where the finite integral
representation or the series expansion is appropriate.

Turning now to TMLE, since it uses the ch.f. and its derivatives only, we do
not need numerical integrations nor series expansions of densities, so that it
is computationally simple and light. Furthermore, it gives high accuracy. Our
simulation result shows that the variance of the TML estimator almost achieves
the Cramér–Rao lower bound. Thus, TMLE shares nice properties with MLE such as
asymptotic efficiency, while it is computationally reasonably tractable.
Finally and most importantly, the procedure given in TMLE is easy to extend to
other contexts. Here we apply it to approximate the conditional ML estimator
of $\alpha$-stable Ornstein–Uhlenbeck (OU) processes. Applications to ID
distributions other than stable laws are also possible (cf. [46]). Since the
class of ID distributions constitute marginal laws of Lévy processes, we may
consider, for instance OU processes driven by Lévy processes, which have been
studied intensively in recent years. In summary, the method of TMLE has
potential application in various active areas.

The paper is organized as follows. After introducing several preliminaries in
Section 2.1, we rigorously define TMLE in Section 2.2. The non-degeneracy of
the covariance matrix for TMLE is also proved in this section. TMLE is
compared with other related estimation methods in Section 2.3. Asymptotics
such as consistency and asymptotic normality are obtained in Section 3.
Subsection 3.1 treats the case where the number of trigonometric functions is
fixed. As the number of trigonometric functions increases, the TML estimator
is shown to converge to the ML estimator regardless of sample size (see
Subjection 3.2), so that TMLE is proved to have asymptotic efficiency. Most
proofs and auxiliary results for Section 3 are given in the Appendix. We
briefly discuss efficiency in finite samples at the end of Section 3. In
Section 4 we examine finite sample performance of TMLE by Monte Carlo
simulation, which shows that the TML estimator outperforms other
asymptotically efficient estimators. Indeed, we see that the standard
deviation of TMLE almost achieves the Cramér–Rao bound. Extending the TMLE
method, we consider the parameter estimation for $\alpha$-stable
Ornstein–Uhlenbeck processes in Section 5, where we numerically observe that
the asymptotic mixed normality holds for the mean reversion parameter and
asymptotic normality holds for other parameters in a high-frequency setting.

##  2\. Trigonometrically approximated maximum likelihood estimation

###  2.1. Preliminary

We work on a continuous parametrization of the stable law (cf. [49, 36]) whose
ch.f. is given by

(2.1)
$\displaystyle\begin{split}\varphi(u;\theta)=\left\\{\begin{array}[]{ll}\exp\Big{(}-|\sigma
u|^{\alpha}\big{\\{}1+i\beta\,{\rm sign}u\tan\frac{\pi\alpha}{2}(|\sigma
u|^{1-\alpha}-1)\big{\\}}+i\mu u\Big{)}&\text{if}\ \alpha\neq 1\\\
\exp\Big{(}-|\sigma u|-i\sigma u\,(2\beta/\pi)\log|\sigma u|+i\mu
u\Big{)}&\text{if}\ \alpha=1,\end{array}\right.\end{split}$

where $\mu\in{\mathbb{R}},\,\sigma\in(0,\infty),\ \alpha\in(0,2]$, and
$\beta\in[-1,1]$ are parameters. The parameter vector is denoted by
$\theta=(\theta_{1},\theta_{2},\theta_{3},\theta_{4})^{\top}=(\mu,\sigma,\alpha,\beta)^{\top}$.
Moreover, we denote the parameter space and its interior by $\Theta$ and
$\mathring{\Theta}$, respectively. Because the expression (2.1) is continuous
in $\alpha$, the case $\alpha=1$ may not be necessary. By the inversion
formula, we see that the density $f(x;\theta)$ satisfies the condition for a
location-scale family.

We use the following notations throughout. Denote real and imaginary parts of
the ch.f. $\varphi(u;\theta)$ respectively by $\varphi^{R}(u;\theta)$ and
$\varphi^{I}(u;\theta)$, that is,
${\mathbb{E}}[e^{iuX}]={\mathbb{E}}[\cos(uX)]+i{\mathbb{E}}[\sin(uX)]=\varphi^{R}(u;\theta)+i\varphi^{I}(u;\theta)$.
As usual $f^{\prime}$ and $f^{\prime\prime}$ denote the first and the second
derivatives of $f$ with respect to (w.r.t.) $x$, respectively. Moreover,
$f_{\theta}=(f_{\theta_{1}},f_{\theta_{2}},f_{\theta_{3}},f_{\theta_{4}})^{\top}$
denotes the vector of partial derivatives of $f$ w.r.t. $\theta$. The second-
order partial derivatives w.r.t. $x$ and $\theta$ are denoted by

$f_{\theta_{i}}^{\prime}=\frac{\partial^{2}f}{\partial
x\partial\theta_{i}}=\frac{\partial^{2}f}{\partial\theta_{i}\partial x},\quad
f_{\theta_{i}\theta_{j}}=\frac{\partial^{2}f}{\partial\theta_{i}\partial\theta_{j}}=\frac{\partial^{2}f}{\partial\theta_{j}\partial\theta_{i}},\quad
i,j=1,\ldots,4,$

that is, all derivatives treated are interchangeable in our case (cf. [28,
Section 2]).

###  2.2. Trigonometrically approximated likelihood estimation

As described in the introduction, we utilize the approximated score function
proposed by Brant [1], which we call the trigonometrically approximated score
function (TSF for short). By solving the approximated score equation by TSF,
we define TMLE. The key idea is that TSF is the projection of the true score
function onto a subspace spanned by trigonometric functions. The projected
score has a nice expression given solely by the ch.f. and its derivatives.

In what follows, we precisely explain the procedure. Let $L^{2}(f)$ be a
Hilbert space with inner product

(2.2) $\displaystyle\langle
h_{1},h_{2}\rangle=\int_{-\infty}^{\infty}h_{1}(x)\bar{h}_{2}(x)f(x;\theta)dx={\mathbb{E}}_{\theta}[h_{1}(X)\bar{h}_{2}(X)],\quad
h_{1},h_{2}\in L^{2}(f).$

Notice that the expectation is taken w.r.t. $f(x;\theta)$, the density of $X$,
and both $\langle\cdot,\cdot\rangle$ and $L^{2}(f)$ depend on $\theta$; for
convenience we abbreviate $\theta$ in these notations. For a given set of
points $(u_{1},\dots,u_{k})$ such that $|u_{i}|\neq|u_{j}|$ for $i\neq j$, we
prepare a vector of trigonometric functions

$g(x)=(\cos(u_{1}x),\cdots,\cos(u_{k}x),\sin(u_{1}x),\cdots,\sin(u_{k}x))^{\top}$

and its expectation

(2.3)
$\displaystyle\gamma(\theta):={\mathbb{E}}_{\theta}[g(X)]=(\varphi^{R}(u_{1};\theta),\cdots,\varphi^{R}(u_{k};\theta),\varphi^{I}(u_{1};\theta),\cdots,\varphi^{I}(u_{k};\theta))^{\top}.$

The elements of vector $g$ are linearly independent and span a subspace of
$L^{2}(f)$. Generally, $g$ constitutes a non-orthogonal basis.

Next we project the score $S(x;\theta)=f_{\theta}(x;\theta)/f(x;\theta)$ onto
the subspace spanned by centered trigonometric functions $g-\gamma(\theta)$.
The projected score $\widetilde{S}(x;\theta)$ satisfies

$\displaystyle S(x;\theta)$
$\displaystyle=\underbrace{A\,(g(x)-\gamma(\theta))}_{:=\widetilde{S}(x;\theta)}+(g(x)-\gamma(\theta))^{\perp}$

with some coefficient $A\,(4\times 2k\ \mathrm{matrix})$, where
$(g(x)-\gamma(\theta))^{\perp}$ denotes the orthogonal complement of the space
spanned by $\\{1,g\\}$. By taking the inner product of both sides with
$g(x)-\gamma(\theta)$, we obtain

$\displaystyle A$
$\displaystyle=\bm{\big{\langle}}S,g-\gamma(\theta)\bm{\big{\rangle}}\,\bm{\big{\langle}}g-\gamma(\theta),g-\gamma(\theta)\bm{\big{\rangle}}^{-1}$

with

$\displaystyle\bm{\big{\langle}}g-\gamma(\theta),g-\gamma(\theta)\bm{\big{\rangle}}={\mathbb{E}}_{\theta}[(g(X)-\gamma(\theta))\,(g(X)-\gamma(\theta))^{\top}]=\Sigma(\theta)$
$\displaystyle\text{and}\quad\bm{\big{\langle}}S,g-\gamma(\theta)\bm{\big{\rangle}}={\mathbb{E}}_{\theta}[S(X;\theta)\,(g(X)-\gamma(\theta))^{\top}]=\gamma_{\theta}(\theta),$

where the inner product of vectors is taken elementwise. Here under
exchangability of expectation and derivative, $\gamma_{\theta}(\theta)$ is
expressed by a $4\times 2k$ derivative matrix 111This is denominator-layout
notation (the Hessian formulation, the gradient) where a vector-by-vector
derivative is given by the transpose of Jacobian.:

(2.4)
$\displaystyle\gamma_{\theta}(\theta):=\frac{\partial\gamma(\theta)}{\partial\theta}=(\varphi^{R}_{\theta}(u_{1};\theta),\ldots,\varphi^{R}_{\theta}(u_{k};\theta),\varphi^{I}_{\theta}(u_{1};\theta),\dots,\varphi^{I}_{\theta}(u_{k};\theta)).$

The elements of $\Sigma(\theta)$ are given by

$\displaystyle\text{Cov}_{\theta}(\cos(u_{i}X),\cos(u_{j}X))$
$\displaystyle=\frac{1}{2}\big{(}\varphi_{\theta}^{R}(u_{i}+u_{j};\theta)+\varphi^{R}_{\theta}(u_{i}-u_{j};\theta)\big{)}-\varphi_{\theta}^{R}(u_{i};\theta)\varphi^{R}_{\theta}(u_{j};\theta),$
(2.5) $\displaystyle\text{Cov}_{\theta}(\cos(u_{i}X),\sin(u_{j}X))$
$\displaystyle=\frac{1}{2}\big{(}\varphi_{\theta}^{I}(u_{i}+u_{j};\theta)-\varphi^{I}_{\theta}(u_{i}-u_{j};\theta)\big{)}-\varphi_{\theta}^{R}(u_{i};\theta)\varphi^{I}_{\theta}(u_{j};\theta),$
$\displaystyle\text{Cov}_{\theta}(\sin(u_{i}X),\sin(u_{j}X))$
$\displaystyle=\frac{1}{2}\big{(}-\varphi_{\theta}^{R}(u_{i}+u_{j};\theta)+\varphi^{R}_{\theta}(u_{i}-u_{j};\theta)\big{)}-\varphi_{\theta}^{I}(u_{i};\theta)\varphi^{I}_{\theta}(u_{j};\theta).$

Thus TSF is constructed by

(2.6)
$\displaystyle\widetilde{S}(x;\theta)=\gamma_{\theta}(\theta)\,\Sigma(\theta)^{-1}(g(x)-\gamma(\theta)).$

It is easy to see that

(2.7) $\displaystyle\widetilde{I}(\theta)={\rm
var}_{\theta}(\widetilde{S}(X;\theta))=\gamma_{\theta}(\theta)\Sigma(\theta)^{-1}\gamma_{\theta}(\theta)^{\top},$

which will be shown to converge to the true Fisher information matrix as
$k\to\infty$ in the proof of Theorem 3.4.

Now we write the empirical TSF by

$\displaystyle\widetilde{S}_{n}(\theta)=\frac{1}{n}\sum_{j=1}^{n}\widetilde{S}(X_{j};\theta)$

and define the TML estimator $\hat{\theta}_{n}$ as an element of the set

(2.8) $\displaystyle\\{\theta\in C\,\mid\,\widetilde{S}_{n}(\theta)=0\\}$

with some compact set $C\subset\mathring{\Theta}$. Since the existence of
exact zeros of the empirical TSF $\widetilde{S}_{n}(\theta)$ is not always
assured in a small sample, if there are no roots, we choose one of local
minimum values of $|\widetilde{S}_{n}(\theta)|$ as the estimator. Although
there might exist multiple local minima, we can appropriately choose a
consistent sequence, as we will discuss shortly.

######  Remark 2.1.

$(\mathrm{i})$ Interestingly the expectation of the derivative of
$\widetilde{S}(X;\theta)$ w.r.t. $\theta$ satisfies

${\mathbb{E}}_{\theta}\left[-\frac{\partial\widetilde{S}(X;\theta)}{\partial\theta}\right]=\gamma_{\theta}(\theta)\Sigma(\theta)^{-1}\gamma_{\theta}(\theta)^{\top}=\widetilde{I}(\theta)$

under certain regularity conditions. The relation corresponds to the second
definition of the Fisher information matrix.
$(\mathrm{ii})$ As one can see, the procedure of TMLE is applicable to any
laws with explicit ch.f.’s. Thus TMLE could be a powerful statistical tool for
the class of ID distributions, whose ch.f is given by the Lévy–Khintchine
representation $($e.g. [44, Section 8]$)$.

Finally, we discuss the non-degeneracy of $\widetilde{I}(\theta)$. One may
wonder if an arbitrary choice of evaluation points of $g$ results in the
singularity of $\widetilde{I}$. In fact, we avoid this by the condition
$|u_{i}|\neq|u_{j}|$ for $i\neq j$. Recall that $\widetilde{S}(x;\theta)$ is
constructed by the projection on a space spanned by trigonometric functions.
The idea behind this is to choose $(u_{i})_{i\leq k}$ such that trigonometric
functions keep linear independence. In the following lemma we show that the
rows of $\gamma_{\theta}(\theta)$ are linearly independent for $k\geq 2$ and
the inverse of the Grammian matrix $\Sigma(\theta)^{-1}$ is positive definite.

######  Lemma 2.2.

Suppose that $u_{i}\neq 0$ for all $i=1,\dots,k$. For any
$\theta\in\mathring{\Theta}$ and for any choice of
$(u_{1},\ldots,u_{k})\in{\mathbb{R}}^{k}$ such that $|u_{i}|\neq|u_{j}|$ for
$i\neq j$,
$(\mathrm{i})$ the rows of $\gamma_{\theta}(\theta)$ are linearly independent
for $k\geq 2$, namely $\rm{rank}(\gamma_{\theta}(\theta))=4$.
$(\mathrm{ii})$ $\Sigma(\theta)$ is positive definite.

The proof is give in Appendix A.1. The result is crucial for the asymptotic
theory (see the next section) and for the optimization procedure to obtain the
TML estimator (Section 4).

###  2.3. Comparison with GMM and other related estimators

The TML estimator has a close similarity to the GMM estimator. Based on moment
conditions ${\mathbb{E}}_{\theta_{0}}[g(X)-\gamma(\theta_{0})]=0$, the
optimally weighted two-step GMM estimator is defined by

(2.9) $\displaystyle\hat{\theta}_{gmm}=\arg\min_{\theta\in
C}\left(\frac{1}{n}\sum_{i=1}^{n}(g(X_{i})-\gamma(\theta))\right)^{\top}\hat{\Sigma}^{-1}\left(\frac{1}{n}\sum_{i=1}^{n}(g(X_{i})-\gamma(\theta))\right),$

where $\hat{\Sigma}$ is a consistent estimator for $\Sigma(\theta_{0})$.
Typically, $\hat{\Sigma}$ is obtained by

(2.10)
$\displaystyle\hat{\Sigma}(\tilde{\theta})=\frac{1}{n}\sum_{i=1}^{n}(g(X_{i})-\gamma(\tilde{\theta}))(g(X_{i})-\gamma(\tilde{\theta}))^{\top},$

where $\tilde{\theta}$ is a preliminary consistent estimator of $\theta_{0}$.
The first-order condition of the minimization problem yields

(2.11)
$\displaystyle\frac{1}{n}\sum_{j=1}^{n}\gamma_{\theta}(\hat{\theta}_{gmm})\hat{\Sigma}^{-1}(g(X_{i})-\gamma(\hat{\theta}_{gmm}))=0.$

Comparing with (2.6), we see that the two estimators are closely related,
although their derivations are quite different: the TML estimator is obtained
by the projection of the true score on trigonometric functions, while the GMM
estimator is constructed based on the moment conditions. One clear difference
is that we treat $\Sigma(\theta)$ as a function of $\theta$ in the estimator
of TMLE (2.6), and do not use a predetermined matrix.

Some studies pointed out that the GMM estimator is infeasible when the grid of
$(u_{1},\dots,u_{k})$ is too fine, because the weight matrix becomes singular
[2, 4, 45, 48]. There might be some confusion regarding this argument. Even
though the GMM estimator is infeasible if (2.10) is used, $\Sigma(\theta_{0})$
can be estimated by $\Sigma(\tilde{\theta})$ because the explicit form of
$\Sigma(\theta)$ is available for the stable law, cf. (2.2). We do not need to
estimate the variance by its sample analog. We call the GMM estimator using
$\Sigma(\tilde{\theta})$ as the weight matrix the explicit GMM estimator.
Lemma 2.2 shows that $\Sigma(\tilde{\theta})$ is non-singular as long as
evaluation points are properly chosen. Therefore, the explicit GMM estimator
is indeed feasible no matter how fine the grid is. Notice, however, that the
non-degeneracy of the weight matrix does not imply that explicit GMM can
utilize an arbitrary large number of moment conditions to satisfy desirable
asymptotic properties. We will discuss this issue in the next section (Remark
3.8).

We explain other related estimators, which are compared with TML estimator in
the simulation. The asymptotics of these estimators are discussed in the next
section.
Other related estimators
$(\mathrm{i})$ The CGMM estimator is based on a continuum (continuous
function) of moment conditions
${\mathbb{E}}_{\theta_{0}}[h(u,X;\theta_{0})]=0$, where
$h(u,X;\theta)=e^{iuX}-\varphi(u;\theta)$ (see [3, 4] for a detailed
definition). More precisely, for a suitably chosen covariance operator $K$, it
minimizes $\|K^{-1/2}h_{n}(u;\theta)\|$ w.r.t. $\theta$, where $\|\cdot\|$ is
a norm and $h_{n}(u;\theta)=\sum_{j=1}^{n}h(u,X_{j};\theta)$. Thus the CGMM
estimator can be viewed as a continuous version of (2.9) (notice that $K$
plays a role of $\Sigma$). The idea behind is to pursuit efficiency by
capturing more information ([3, Sec. 5.2]). A problem of CGMM is that an
estimate of optimal $K$ is not invertible in general. This is dealt with a
regularization parameter introduced in the estimation of $K^{-1/2}$.
$(\mathrm{ii})$ The EL estimator [26] also relies on the moment conditions
${\mathbb{E}}_{\theta_{0}}[g(X)-\gamma(\theta_{0})]=0$ and maximizes the
empirical likelihood subject to the moment conditions. Similar to the GMM
estimator it suffers from a severe problem of degeneracy if moment conditions
are too many, because it is a one-step estimator that implicitly estimates the
optimal weight matrix. We cannot use $\Sigma(\tilde{\theta})$ as the weight
matrix for the EL estimator.

######  Remark 2.3.

The models considered by [45] and [2] are different from ours. They treat more
complicated financial models such as affine price models. It is not certain if
$\Sigma(\tilde{\theta})$ is non-singular for distributions other than the
stable law when the grid of evaluation points is fine. Moreover, the condition
of our lemma is violated if symmetric grid points $(-u_{k},u_{k})_{k\geq 1}$
are selected as in [45].

##  3\. Asymptotics of trigonometric approximated maximum likelihood
estimation

This section investigates asymptotic properties of TMLE. The asymptotics of
TMLE with arbitrary fixed evaluation points are studied in Subsection 3.1. In
Subsection 3.2 we consider the case where the number of points increases. We
show that the TML estimator converges to the ML estimator in the sense that
they have the same asymptotic law. The asymptotics of other related estimators
are discussed at the end of this section.

###  3.1. Asymptotics of TMLE when the number of points is fixed

The TML estimator with fixed evaluation points could be regarded as a
$Z$-estimator, and we exploit theorems from van der Vaart [47, Theorems 5.41
and Theorem 5.42]. The following is the first main result.

######  Theorem 3.1.

Let $C$ be any compact subset of $\mathring{\Theta}$ and let
$\theta_{0}\in\mathring{C}$ be a true parameter. Suppose that evaluation
points $(u_{1},\ldots,u_{k})\in{\mathbb{R}}^{k}$ of $g$ satisfy
$|u_{i}|\neq|u_{j}|$ for $i\neq j$ and $u_{i}\neq 0$ for all $i$. Then the
probability that $\widetilde{S}_{n}(\theta)=0$ has at least one root tends to
$1$ as $n\to\infty$, and there exists a sequence of roots $\hat{\theta}_{n}$
such that $\hat{\theta}_{n}\to\theta_{0}$ in probability. Moreover, every
consistent sequence of roots $\hat{\theta}_{n}$ satisfies

(3.1)
$\displaystyle\sqrt{n}\,(\hat{\theta}_{n}-\theta_{0})=\widetilde{I}(\theta_{0})^{-1}\frac{1}{\sqrt{n}}\sum_{j=1}^{n}\widetilde{S}(X_{j};\theta_{0})+o_{p}(1).$

In particular, the sequence $\sqrt{n}\,(\hat{\theta}_{n}-\theta_{0})$ is
asymptotically normal with mean zero and covariance matrix
$\widetilde{I}(\theta_{0})^{-1}$.

Theorem 3.1 does not rule out the existence of multiple roots. However, non-
uniqueness of solutions is not a serious problem because we can utilize
existing consistent estimators for $\theta_{0}$ such as the quantile-based
estimator of McCulloch[32]. We can find the consistent root by choosing the
root closest to a preliminary consistent estimator (see p.70 of [47]).

The proof of Theorem 3.1 is given in Appendix A.2. Here we make a remark about
the idea behind the proof.

######  Remark 3.2.

The TML estimator could be regarded as a $Z$-type estimator whose estimating
equation approximates the first-order condition of MLE. If the estimator
converges to a local minimum/maximum of the likelihood function, then
consistency fails. We can avoid this at least locally through the condition
$[\,|u_{i}|\neq|u_{j}|\,\&\,u_{i}\neq 0\,]$ $($see Lemma 2.2$)$, that is, the
condition that for sufficiently large samples, the Hessian of TMLE
$($derivative of $-\widetilde{S}(\theta)$ w.r.t. $\theta)$ is positive
definite in a neighborhood of true parameter $\theta_{0}$ $($cf. Remark 2.1
$(\mathrm{i})$$)$. Thus asymptotically and locally we can choose a unique zero
which maximizes the likelihood function.

###  3.2. Asymptotics of TMLE when the number of points goes to infinity

We show that the TML estimator converges to the ML estimator in the sense that
they have the same asymptotic law (Theorem 3.6). Throughout this subsection,
we take equally spaced evaluation points
$(u_{1},u_{2},\ldots,u_{k})=(\tau,2\tau,\ldots,k\tau)$ and study the limit
behavior of TMLE when the number $k$ increases to infinity, while the interval
length $\tau>0$ goes to zero.

Referring to Brant [1], we first describe that TSF can be regarded as a two-
step approximation of the true score function. From this one can grasp how the
limit operation $k\to\infty$ and $\tau\to 0$ works, which is also useful to
follow the proofs. The first step is based on a wrapping of $f(x;\theta)$,

(3.2) $\displaystyle
f_{\tau}(x;\theta)=\sum_{j=-\infty}^{\infty}f\,\Big{(}x+\frac{2\pi
j}{\tau};\theta\Big{)},\quad\tau>0$

from which we construct a wrapped version of the score function ($4\times 1$
vector)

(3.3) $\displaystyle
S_{\tau}(x;\theta)=\frac{\frac{\partial}{\partial\theta}f_{\tau}(x;\theta)}{f_{\tau}(x;\theta)}\left(=:\frac{f_{\tau,\theta}(x;\theta)}{f_{\tau}(x;\theta)}\right).$

Notice that both $f_{\tau}$ and $S_{\tau}$ are periodic functions with period
$2\pi/\tau$. By letting $\tau\to 0$ we obtain convergences $f_{\tau}\to f$ and
$S_{\tau}\to S$ (Lemmas B.1 and B.2).

The second approximation step is the projection of $S_{\tau}(x;\theta)$ into
the subspace of $L^{2}(f_{\tau})$ that is spanned by trigonometric functions,
as done in Section 2. Here, $L^{2}(f_{\tau})$ is the $L^{2}$ space on
$[-\pi/\tau,\,\pi/\tau]:=I_{\tau}$, whose inner product is given by

$\langle
h_{1},h_{2}\rangle_{\tau,\theta}={\mathbb{E}}_{\tau,\theta}[h_{1}\bar{h}_{2}]=\int_{I_{\tau}}h_{1}(x)\bar{h}_{2}(x)f_{\tau}(x;\theta)dx,\quad
h_{1},\,h_{2}\in L^{2}(f_{\tau}).$

Now setting $(u_{1},\ldots,u_{k})=(\tau,2\tau,\ldots,k\tau)$ in the previous
definitions (2.3) and (2.4), we obtain

${\mathbb{E}}_{\tau,\theta}[(g(X)-\gamma(\theta))\,(g(X)-\gamma(\theta))^{\top}]=\Sigma(\theta)\quad\text{and}\quad{\mathbb{E}}_{\tau,\theta}[S_{\tau}(X;\theta)\,(g(X)-\gamma(\theta))^{\top}]=\gamma_{\theta}(\theta).$

Thus the projection coincides with TSF $\widetilde{S}$ with equidistant
evaluation points. Since $S_{\tau}$ is periodic, its projection on
trigonometric functions will be close to $S_{\tau}$ itself for a sufficiently
large $k$. We can rigorously prove the convergence $\widetilde{S}\to S_{\tau}$
as $k\to\infty$ (Proposition B.5). Therefore, we obtain $\widetilde{S}\to S$
as $k\to\infty$ and $\tau\to 0$.

######  Remark 3.3.

Notice that TSF is originally obtained by the projection of the true score
function $f_{\theta}(x;\theta)/f(x;\theta)$ into the subspace of the
$L^{2}(f)$ space spanned by $\\{1,g(x)\\}$. The equivalence of the two
procedures follows from the same logic as that the Fourier coefficients of a
wrapped function are equivalent to the Fourier transform of
$f_{\theta}(x;\theta)$ at the corresponding points $k\tau,\,k\in{\mathbb{N}}$
$($see also [1, p.993]$)$, that is, for $t=k\tau$,

${\mathbb{E}}_{\tau,\theta}[S_{\tau}(X;\theta)e^{itX}]=\int_{I_{\tau}}e^{itx}f_{\tau,\theta}(x;\theta)dx=\int_{-\infty}^{\infty}e^{itx}f_{\theta}(x;\theta)dx={\mathbb{E}}_{\theta}[S(X;\theta)e^{itX}]=\varphi_{\theta}(t;\theta).$

In the proofs of this subsection, we implicitly use properties of Fourier
series expansion of the wrapped version $S_{\tau}(x;\theta)$ $($e.g. Theorems
B.3 and B.4$)$. Thus we take equidistant points on $g$ and consider
$L^{2}(f_{\tau})$.

It is not straight forward to extend Theorem 3.1 to the case of $k\to\infty$
and $\tau\to 0$. The difficulty stems from the fact that the proof of the
theorem evaluates the second-order derivatives of $\widetilde{S}(x;\theta)$
with respect to $\theta$, which is quite complicated when $k\to\infty$ and
$\tau\to 0$. Instead we apply [47, Theorem 5.7], the assertion of which is
weaker than that of Theorem 3.1 but assures that sequences $\hat{\theta}_{n}$
of (approximate) zeros of $\widetilde{S}_{n}(\theta)$ include a consistency
sequence.

Let $\bar{B}_{\delta}(\theta_{0})=\\{\theta:|\theta-\theta_{0}|\leq\delta\\}$
be a closed ball with center $\theta_{0}$ and radius $\delta$. We have the
following theorem.

######  Theorem 3.4.

Let $C$ be any compact subset of $\mathring{\Theta}$ and
$\theta_{0}\in\mathring{C}$ be a true parameter. Then, for any $k$ and $\tau$,
there exists a sufficiently small $\delta>0$ such that any sequence of
estimators $\tilde{\theta}_{n}$ satisfying
$\tilde{\theta}_{n}\in\bar{B}_{\delta}(\theta_{0})\cap C$ for all sufficiently
large $n$ and $|\widetilde{S}_{n}(\tilde{\theta}_{n})|=o_{p}(1)$ converges in
probability to $\theta_{0}$. The result holds even when $\tau\to 0$ if
$k^{-1}=o(\tau^{2+\alpha_{0}+\delta})$.

The problem of TMLE is that $|\widetilde{S}_{n}(\theta)|$ potentially has
multiple local minima. However, $|\widetilde{S}_{n}(\theta)|$ asymptotically
has a unique minimum in a neighborhood of $\theta_{0}$. The theorem states
that if we restrict the parameter space in a neighborhood of $\theta_{0}$,
then any sequence of estimators $(\tilde{\theta}_{n})$ that satisfies
$|\widetilde{S}_{n}(\tilde{\theta}_{n})|=o_{p}(1)$ is consistent for
$\theta_{0}$.

Theorem 3.4 immediately implies the existence of a consistent sequence of
TMLE. Indeed, let
$\tilde{\theta}_{n}=\arg\min_{\theta\in\bar{B}_{\delta}(\theta_{0})\cap
C}|\widetilde{S}_{n}(\theta)|$ for a sufficiently small $\delta>0$. Then, from
the proof of Theorem 3.4, we see that $\tilde{\theta}_{n}$ satisfies
$|\widetilde{S}_{n}(\tilde{\theta}_{n})|=o_{p}(1)$, and thus that
$\tilde{\theta}_{n}$ is consistent. We emphasize that the introduction of
$\bar{B}_{\delta}(\theta_{0})$ is only for theoretical exposition. By the
existence of a preliminary consistent estimator, we can obtain a consistent
sequence of TMLE without knowing $\bar{B}_{\delta}(\theta_{0})$.

######  Remark 3.5.

$(\mathrm{i})$ Similarly as in the fixed-points case, we can choose a sequence
of consistent roots $\hat{\theta}_{n}$ of $\widetilde{S}_{n}(\theta)=0$.
Practically, we use a variant of the Newton–Raphson algorithm with a
consistent initial value. See Section 4 for the detail.
$(\mathrm{ii})$ We observe in the proof that under the condition of $\tau$ and
$k$ in Theorem 3.4, $\widetilde{S}(x;\theta)$ converges to the true score
$S(x;\theta)$ as $\tau\to 0$. By this fact, consistency of $\hat{\theta}_{n}$
is kept even in the limit $\tau\to 0$.
$(\mathrm{iii})$ The condition $k^{-1}=o(\tau^{2+\alpha_{0}+\delta})$ of
Theorem 3.4 depends on an unknown true parameter $\alpha_{0}$ and $\delta$.
Because $\delta$ can be arbitrarily small, there exists $\delta$ such that
$\alpha_{0}+\delta\leq 2$ for any $\theta_{0}\in\mathring{C}$. Then, the
condition is satisfied if $k^{-1}=o(\tau^{4})$ although using too many
evaluation points is computationally demanding. In actual implementation the
dependence on $\alpha_{0}$ is not a major issue, see Remark 3.7. Notice that
the condition of $k$ and $\tau$ for consistency is included in that for
asymptotic normality, see Theorem 3.6.

Next we study the asymptotic normality and efficiency of the TML estimator,
taking the consistent sequence $(\hat{\theta}_{n})$ of TMLE. For this purpose,
first, we evaluate the distance between the score of TML
$\widetilde{S}(x;\theta)$ and that of MLE $S(x;\theta)$, which can be
$o(n^{-1/2})$ depending on $k,\,\tau$ (Lemma B.6). Then, we evaluate the
distance between $\sqrt{n}(\hat{\theta}_{n}-\theta_{0})$ and
$\sqrt{n}(\tilde{\theta}_{n}-\theta_{0})$, whose expressions are given by
scores $\widetilde{S}$ and $S$ respectively, and show that it converges to $0$
in probability.

######  Theorem 3.6.

Let $C$ be any compact set on $\mathring{\Theta}$ and
$\theta_{0}\in\mathring{C}$ be a true parameter. Denote the ML estimator and
the consistent TML estimator respectively by $\tilde{\theta}_{n}$ and
$\hat{\theta}_{n}$. If $k\to\infty$ and $\tau\to 0$ such that
$n^{1/2}\big{\\{}\tau^{\alpha_{0}}\log
1/\tau+(\tau^{2+\alpha_{0}+\delta}k)^{-1/2}\big{\\}}\to 0$ holds for
$\delta>0$ of Theorem 3.4, then
$\sqrt{n}\,(\hat{\theta}_{n}-\tilde{\theta}_{n})=o_{p}(1)$; thus,
$\hat{\theta}_{n}$ has the same asymptotic distribution as
$\tilde{\theta}_{n}$.

Theorem 3.6 states that the TML estimator is asymptotically normal and
efficient because it inherits the asymptotic properties of the ML estimator
[28]. As far as we know, our result is the first to rigorously establish the
asymptotic normality and efficiency of the estimator based on the countable
points of the ch.f. Here we see the difference between our result and previous
results.

Previous studies show that the asymptotic variance of the GMM estimator, which
is obtained under fixed $k$ and $\tau$, can be arbitrarily close to the
Crámer–Rao lower bound when $k$ is large and $\tau$ is small ([13, 48]). In
other words, they employ a sequential asymptotic framework, where the limit
w.r.t. $n$ is taken first and then the limit w.r.t. $k$ and $\tau$ is taken.
However, the sequential asymptotic theory does not provide a good
approximation to the finite sample distribution in general. If the number of
moment conditions is too large, the GMM estimator does not satisfy asymptotic
normality (see, for instance, [10] and references therein).

Kunitomo and Owada [26] showed that the EL estimator satisfies asymptotic
normality if the number of moment conditions grows slowly depending on the
sample size. However, their asymptotic framework is also different from ours.
Their proof consists of two steps. First, they choose $\tau=K/k$ for fixed
$K>0$, and prove that the EL estimator converges in distribution to a normal
distribution if $k$ satisfies $k=O(n^{\frac{1}{3}-\eta})$ with $\eta>0$, so
that $k$ is not independent of the sample size $n$. Then, they show that the
asymptotic variance converges to the Crámer–Rao lower bound as $K\to\infty$.
Therefore, they do not specify the condition for the relation between $k$ and
$\tau$ under which the EL estimator is asymptotically normal and efficient.

We close this section with a brief discussion of finite sample behaviors. As
stated above the number of moment conditions is limited by sample size for
both GMM and EL estimators. Since the number should be increased for
efficiency, those estimators could not be optimal in finite samples. The CGMM
estimator avoids this deficiency and can exploit a full continuum of moment
conditions. However, similar to the GMM estimator, it has the estimation step
of the optimal covariance operator. In view of simulations, this step causes
some efficiency loss. Notice that the EL estimator skips this step since it
estimates both the parameter vector and the optimal weight all at once. Taking
above facts into consideration, we consider TMLE. In the TML estimator, $\tau$
and $k$ can be arbitrarily small and large respectively, regardless of the
sample size. Moreover even if $k$ is sufficiently large, no singularity is
observed, that is, it just converges to the ML estimator as $k\to\infty$ and
$\tau\to 0$. Notice that in view of the form (2.6), the explicit weight matrix
$\gamma_{\theta}(\theta)\Sigma(\theta)^{-1}$ includes $\theta$, and thus TMLE
implicitly estimates the weight matrix simultaneously with $\theta_{0}$.

######  Remark 3.7.

The condition on $\tau$ and $k$ in Theorem 3.6 depends on the unknown
parameter $\alpha_{0}$. In actual implementation, the dependence on
$\alpha_{0}$ is not a major issue. Our simulation shows that a use of $100$
evaluation points is sufficient for efficient estimation for any value of
$\alpha_{0}$. Using many more evaluation points does not improve the
efficiency of the estimator when $n=1000$. This is probably because the
estimation error of MLE dominates the approximation error of TSF when $n$ is
not so large.

######  Remark 3.8.

The explicit GMM estimator is well defined even when we increase the
evaluation number $k$ of ch.f. depending on $\tau$. In particular we could
take the equally spaced points as done in the TML estimator. However, it
remains to be seen whether the asymptotic efficiency beyond a sequential
framework holds in the limit of $k$ and $\tau$. Indeed for the continuous
version of GMM $($see [13, Equation (5.1)]$)$, which could be regarded as the
limit of $k$ and $\tau$, it is shown that the optimal weight is not always
integrable $($[13, Equation (5.9)]$)$, and indeed we could not obtain it.
Although it is not explicitly mentioned in [13], by ignoring the integrability
and proceeding with some calculations, the first-order condition of explicit
GMM with the optimal weight leads to the likelihood equation. We stay with
only two certain facts: $(a)$ The score of explicit GMM is expressed by a
combination of trigonometric functions and thus it has a more distance to the
score of MLE than that of TMLE does, $(b)$ If we estimate
$\hat{\theta}^{k+1}_{n}$ by the explicit GMM with
$\Sigma(\hat{\theta}_{n}^{k})$ iteratively it becomes close to the TML
estimator. Indeed, in a small simulation study, which is not reported here, we
could not find a clear difference in the mean square errors between the TML
estimator and the explicit GMM using $\Sigma(\tilde{\theta}_{n})$ with
$\tilde{\theta}_{n}$ a consistent estimator.

##  4\. Simulation results

The performance of ch.f.-based estimators is examined in this section. We
compare the TML estimator with the EL estimator and the CGMM estimator. See
Subsection 2.3 for definitions and relations of these estimators. All of the
estimators are asymptotically efficient under certain conditions.

To find a root of $\widetilde{S}_{n}(\theta)=0$ for TMLE, we utilize the
method of scoring algorithm, which is a variant of the Newton–Raphson
algorithm. The method replaces the derivative of the objective function with
its expected value in the iteration process; that is, given a preliminary
estimator $\hat{\theta}_{n}^{0}$, the sequence $(\hat{\theta}_{n}^{l})_{l\geq
1}$ is renewed by

$\hat{\theta}_{n}^{l+1}=\hat{\theta}_{n}^{l}+\delta\widetilde{I}(\hat{\theta}_{n}^{l})^{-1}\widetilde{S}_{n}(\hat{\theta}_{n}^{l}),$

where $\delta$ ($0<\delta\leq 1$) is a scalar that adjusts the step length.
The procedure is computationally stable because the positive definiteness of
$\widetilde{I}(\theta)$ is guaranteed by Lemma 2.2. If the initial value of
the iteration is $\sqrt{n}$-consistent and $\delta=1$, then the one-step
estimator $\hat{\theta}_{n}^{1}$ is asymptotically equivalent to the TML
estimator ([47, Theorem 5.45]). However, we iterate the procedure until the
sequence converges.

To obtain the TML and EL estimators, we need to determine a vector of grid
points $u=(u_{1},\dots,u_{k})$ for $g$. The TML estimator uses $101$
equidistant points $u=(0.01,0.06,\dots,5.01)$. The EL estimator uses 11 points
$u=(0.1,0.6,\dots,5.1)$ when $\alpha\neq 1.9$ and 6 points
$u=(0.1,1.1,\dots,5.1)$ when $\alpha=1.9$. The selection of the grid points
for the EL estimator is similar to that of [26]. We use only 6 points when
$\alpha=1.9$ because our simulation showed that the EL estimator with 11 grid
points performs poorly when $\alpha\geq 1.7$.

For the minimization procedure for CGMM, because solving the original
minimization problem of [3] is computationally involved, we utilize
Proposition 3.4 of [2], which gives a simpler yet equivalent minimization
problem. To evaluate the integrals used in the objective function, we adopt
the Gauss–Hermite formula. Recall that the problem of CGMM is that an estimate
of $K$ is not invertible, and we need the regularization to estimate
$K^{-1/2}$. The regularization parameter is set to be $0.01$ in our
simulation. Although we repeated the estimation using different values of the
regularization parameter, the result was insensitive to this choice in this
finite sample simulation.

The simulation is conducted using R software with package libstableR. The data
generating process takes six different values of
$\alpha\in\\{0.5,0.7,1.0,1.3,1.6,1.9\\}$ and two different values of
$\beta\in\\{0,0.5\\}$. The values of $\sigma$ and $\mu$ are fixed to $1$ and
$0$, respectively. As the initial value of all estimators, we use the
quantile-based estimator of McCulloch [32], which is consistent and easy to
calculate but not necessarily efficient.

Table 1 reports the result of 1000 repetitions with 1000 observations. The
mean, standard deviation, skewness, and kurtosis are reported. We see that all
estimators are nearly unbiased. Moreover, except for a few cases, we do not
see a clear deviation from a normal distribution in term of the skewness and
kurtosis. However, the standard deviation is quite different among estimators.
The TML estimator dominates other two estimators for all cases even though its
computational burden is quite small. The CGMM estimator performs poorly
especially when the true value of $\alpha$ is small. In contrast, the EL
estimator performs rather poorly when the true value of $\alpha$ is large.

Table 1. Simulation result |  |  | QMLE | ELE | CGMM
---|---|---|---|---|---
$\alpha$ | $\beta$ |  | $\hat{\alpha}$ | $\hat{\beta}$ | $\hat{\sigma}$ | $\hat{\mu}$ | $\hat{\alpha}$ | $\hat{\beta}$ | $\hat{\sigma}$ | $\hat{\mu}$ | $\hat{\alpha}$ | $\hat{\beta}$ | $\hat{\sigma}$ | $\hat{\mu}$
0.5 | 0 | Mean | 0.5001 | -0.0023 | 1.0000 | -0.0009 | 0.5000 | -0.0029 | 0.9980 | 0.0008 | 0.5006 | -0.0027 | 0.9996 | -0.0013
|  | Sd | 0.0191 | 0.0455 | 0.0773 | 0.0266 | 0.0245 | 0.0582 | 0.0827 | 0.0305 | 0.0584 | 0.1286 | 0.1092 | 0.0444
|  | Skew | 0.1126 | -0.0615 | 0.4498 | -0.1202 | 0.1309 | 0.0033 | 0.3717 | 0.0900 | 0.0106 | 0.0222 | 0.0715 | 0.0665
|  | Kur | 2.9610 | 3.0564 | 3.5118 | 2.8604 | 3.0749 | 3.1392 | 3.2826 | 3.1535 | 3.1382 | 3.1803 | 2.9802 | 3.0663
| 0.5 | Mean | 0.4995 | 0.4997 | 1.0038 | 0.0009 | 0.4980 | 0.5019 | 1.0042 | 0.0011 | 0.5002 | 0.4996 | 1.0010 | 0.0021
|  | Sd | 0.0181 | 0.0387 | 0.0752 | 0.0366 | 0.0232 | 0.0543 | 0.0778 | 0.0400 | 0.0584 | 0.1282 | 0.1094 | 0.0546
|  | Skew | 0.0128 | -0.0277 | 0.1693 | 0.2090 | 0.1800 | 0.0950 | 0.2312 | 0.2377 | 0.1184 | 0.1164 | -0.0054 | 0.0166
|  | Kur | 2.8393 | 2.8909 | 3.0163 | 3.2009 | 2.7689 | 2.9204 | 3.0465 | 3.2012 | 3.0269 | 2.9172 | 2.8122 | 3.0560
0.7 | 0 | Mean | 0.7000 | -0.0016 | 1.0032 | 0.0002 | 0.7008 | -0.0039 | 0.9958 | 0.0018 | 0.7001 | -0.0071 | 0.9928 | 0.0020
|  | Sd | 0.0240 | 0.0436 | 0.0575 | 0.0337 | 0.0301 | 0.0552 | 0.0609 | 0.0365 | 0.0598 | 0.1051 | 0.0785 | 0.0490
|  | Skew | 0.0809 | 0.0572 | 0.0975 | -0.0026 | 0.2372 | 0.0630 | 0.2889 | -0.0138 | 0.1167 | 0.0224 | 0.1172 | -0.0165
|  | Kur | 2.8056 | 2.8497 | 3.0761 | 2.9722 | 3.0582 | 3.0206 | 3.0453 | 3.0562 | 2.9967 | 2.7643 | 2.88841 | 2.9925
| 0.5 | Mean | 0.7023 | 0.4997 | 0.9987 | 0.0012 | 0.7017 | 0.4988 | 0.9993 | 0.0004 | 0.7035 | 0.5041 | 0.9969 | 0.0011
|  | Sd | 0.0243 | 0.0399 | 0.0549 | 0.0409 | 0.0303 | 0.0500 | 0.0580 | 0.0442 | 0.0607 | 0.1042 | 0.0723 | 0.0553
|  | Skew | 0.2565 | -0.0634 | 0.3667 | 0.3090 | 0.1111 | -0.1013 | 0.1919 | 0.0458 | -0.0167 | 0.1035 | 0.1691 | 0.2004
|  | Kur | 2.9837 | 2.8447 | 3.0964 | 3.2816 | 2.7998 | 2.8292 | 2.8247 | 2.8587 | 2.8587 | 3.2082 | 3.2045 | 3.3522
1.0 | 0 | Mean | 1.0018 | -0.0004 | 1.0038 | 0.0006 | 1.0025 | -0.0004 | 1.0036 | 0.0002 | 1.0005 | 0.0009 | 1.0030 | 0.0007
|  | Sd | 0.0353 | 0.0568 | 0.0453 | 0.0492 | 0.0396 | 0.0639 | 0.0466 | 0.0492 | 0.0611 | 0.0936 | 0.0524 | 0.0547
|  | Skew | 0.1197 | -0.0608 | 0.1635 | 0.1755 | 0.0158 | -0.0283 | 0.1766 | 0.1077 | -0.0337 | -0.0345 | 0.1477 | 0.1223
|  | Kur | 3.0653 | 3.3013 | 2.8804 | 2.9999 | 2.8650 | 2.9374 | 2.8817 | 3.0077 | 2.6764 | 3.1695 | 2.8678 | 2.8860
| 0.5 | Mean | 1.0003 | 0.4995 | 1.0002 | 0.0002 | 1.0025 | 0.5003 | 0.9997 | 0.0002 | 1.0009 | 0.5024 | 0.9991 | -0.0006
|  | Sd | 0.0345 | 0.0478 | 0.0434 | 0.0477 | 0.0384 | 0.0583 | 0.0446 | 0.0493 | 0.0603 | 0.0930 | 0.0502 | 0.0562
|  | Skew | 0.0915 | -0.1710 | 0.0779 | 0.0943 | 0.0296 | -0.0994 | 0.1300 | 0.1252 | 0.0966 | 0.0847 | 0.1006 | 0.1239
|  | Kur | 2.8549 | 2.8708 | 2.8244 | 3.0870 | 2.9937 | 3.0937 | 2.8579 | 3.1045 | 2.9499 | 3.1468 | 2.8947 | 2.9264
1.3 | 0 | Mean | 1.3011 | -0.0027 | 1.0000 | 0.0001 | 1.3015 | -0.0040 | 1.0002 | 0.0009 | 1.3006 | -0.0031 | 0.9986 | -0.0003
|  | Sd | 0.0449 | 0.0744 | 0.0372 | 0.0523 | 0.0508 | 0.0821 | 0.0380 | 0.0533 | 0.0619 | 0.1011 | 0.0399 | 0.0560
|  | Skew | -0.0304 | 0.0089 | 0.0887 | -0.0614 | -0.0008 | 0.1513 | 0.0864 | -0.0081 | -0.1612 | 0.0679 | 0.0605 | 0.0072
|  | Kur | 2.9152 | 2.9449 | 2.9022 | 3.0298 | 2.8310 | 3.2701 | 2.8996 | 2.9636 | 3.1164 | 2.9232 | 3.0110 | 2.9021
| 0.5 | Mean | 1.3014 | 0.5002 | 0.9996 | 0.0003 | 1.3025 | 0.4983 | 1.0003 | 0.0015 | 1.3034 | 0.5012 | 0.9995 | 0.0015
|  | Sd | 0.0439 | 0.0652 | 0.0366 | 0.0527 | 0.0504 | 0.0754 | 0.0377 | 0.0542 | 0.0593 | 0.1087 | 0.0403 | 0.0587
|  | Skew | 0.0393 | -0.0369 | 0.1030 | 0.0184 | -0.0080 | -0.0293 | 0.1208 | 0.0148 | 0.0620 | -0.0295 | 0.1192 | 0.0673
|  | Kur | 2.9720 | 2.7660 | 2.8661 | 3.1141 | 3.1213 | 2.9781 | 2.9181 | 3.0627 | 2.9224 | 2.8581 | 2.8740 | 3.0668
1.6 | 0 | Mean | 1.6004 | 0.0004 | 0.9990 | -0.0016 | 1.6052 | 0.0008 | 1.0001 | -0.0019 | 1.5995 | -0.0039 | 0.9987 | -0.0002
|  | Sd | 0.0491 | 0.1137 | 0.0318 | 0.0560 | 0.0605 | 0.1345 | 0.0321 | 0.0586 | 0.0552 | 0.1479 | 0.0332 | 0.0613
|  | Skew | -0.0663 | 0.0843 | 0.1365 | -0.0923 | 0.5137 | -0.0980 | 0.0637 | -0.0865 | -0.1163 | 0.0493 | 0.1434 | -0.1285
|  | Kur | 2.8879 | 3.2288 | 2.8530 | 3.0718 | 4.9116 | 2.9819 | 3.2748 | 3.0262 | 2.9319 | 3.1401 | 2.8545 | 3.1150
| 0.5 | Mean | 1.6006 | 0.5033 | 0.9986 | -0.0009 | 1.6023 | 0.5111 | 0.9983 | 0.0000 | 1.6009 | 0.5033 | 0.9986 | -0.0003
|  | Sd | 0.0485 | 0.1008 | 0.0315 | 0.0553 | 0.0608 | 0.1303 | 0.0334 | 0.0610 | 0.0565 | 0.1483 | 0.0333 | 0.0607
|  | Skew | 0.0094 | 0.0804 | 0.0905 | -0.0294 | 0.0124 | 0.4189 | 0.2196 | 0.0206 | -0.0258 | 0.0723 | 0.1685 | -0.0662
|  | Kur | 2.9767 | 2.9599 | 2.9079 | 3.0665 | 2.8251 | 3.3305 | 2.9167 | 3.3463 | 3.1096 | 3.3187 | 2.8926 | 3.2160
1.9 | 0 | Mean | 1.9025 | 0.0160 | 0.9998 | -0.0018 | 1.9022 | 0.0041 | 0.9986 | -0.0012 | 1.8957 | 0.0052 | 0.9985 | -0.0020
|  | Sd | 0.0390 | 0.3478 | 0.0267 | 0.0531 | 0.0452 | 0.4375 | 0.0286 | 0.0611 | 0.0425 | 0.5299 | 0.0297 | 0.0650
|  | Skew | -0.0739 | 0.0623 | -0.0295 | -0.0352 | -0.5660 | -0.0011 | 0.1676 | -0.0335 | -0.2156 | -0.0312 | 0.1440 | -0.1065
|  | Kur | 2.9590 | 3.9513 | 2.9083 | 3.0549 | 2.9949 | 3.1632 | 2.9147 | 3.0477 | 2.8248 | 2.4583 | 2.8669 | 2.9937
| 0.5 | Mean | 1.9012 | 0.5317 | 0.9994 | -0.0009 | 1.8997 | 0.5259 | 0.9980 | 0.0026 | 1.8917 | 0.4707 | 0.9981 | -0.0012
|  | Sd | 0.0376 | 0.3027 | 0.0266 | 0.0532 | 0.0451 | 0.3659 | 0.0282 | 0.0596 | 0.0423 | 0.4651 | 0.0296 | 0.0642
|  | Skew | -0.1684 | -0.3778 | 0.0187 | -0.0131 | -0.6621 | -0.6294 | 0.1596 | 0.0285 | -0.2116 | -0.8586 | 0.1379 | -0.0667
|  | Kur | 2.8764 | 3.4267 | 3.0886 | 3.1041 | 3.3209 | 3.6659 | 2.9539 | 2.9691 | 2.8167 | 3.4851 | 2.8465 | 2.9645
Table 2. Efficiency $\alpha$ | $\beta$ | $\hat{\alpha}$ | $\hat{\beta}$ | $\hat{\delta}$ | $\hat{\mu}$
---|---|---|---|---|---
0.5 | 0 | 0.800 | 0.772 | 0.978 | 0.913
| 0.5 | 0.821 | 0.805 | 0.969 | 0.974
0.7 | 0 | 0.956 | 0.952 | 1.018 | 1.006
| 0.5 | 0.924 | 0.883 | 1.001 | 1.006
1.0 | 0 | 0.988 | 0.965 | 1.009 | 0.955
| 0.5 | 0.997 | 0.992 | 1.005 | 1.006
1.3 | 0 | 0.999 | 1.003 | 1.026 | 1.009
| 0.5 | 0.999 | 1.006 | 1.006 | 1.015
1.6 | 0 | 0.995 | 0.988 | 1.017 | 0.996
| 0.5 | 0.986 | 0.997 | 1.003 | 1.008
1.9 | 0 | 0.913 | 0.824 | 0.987 | 1.002
| 0.5 | 0.922 | 0.886 | 0.983 | 0.999

Table 2 reports the ratio of the standard deviation of the ML estimator to
that of the TML estimator under the same setting as that for Table 1. Thus,
larger values indicate higher efficiency of TMLE. Because MLE is
computationally demanding, we use the theoretical standard deviation
calculated by Nolan [38]. The simulation generally supports our theory. The
TML estimator is on par with the ML estimator for $\alpha$ between 1.0 and
1.6. However, for a smaller $\alpha$ we observe a slight degradation in the
performance.

######  Remark 4.1.

$(\mathrm{i})$ The performance of the CGMM estimator may be improved if the
regularization parameter is determined in a date-driven way $($cf. [5]$)$.
$(\mathrm{ii})$ Although the result is not reported here, we also investigated
the performance of the TML estimator by using 11 grid points. The TML and the
EL estimators have similar means and standard deviations in that case. This is
not surprising, because two estimators are first-order equivalent if the same
grid points are used. However, in a finite sample, the EL estimator performs
poorly if a large number of grid points is used.
$(\mathrm{iii})$ If the parameter regions are close to the boundaries, such as
$\alpha=2$ or $\beta=\pm 1$, some elements of the Fisher information matrix
are known to diverge to $\infty$ $($see [11, 34, 29]$)$. It will be
interesting to see what happens for these estimators in such situations, and
we will pursue this in future work.
$(\mathrm{vi})$ In finite samples, the optimal choice of the grid points in
the EL and TML estimators will depend on the true value of $\alpha$. If
$\alpha$ is small, the estimators with more grid points around the origin
often perform better. Further study of this topic will also be part of future
work.

##  5\. Extensions to estimation of stable OU-processes

As an extension, we estimate the parameters of $\alpha$-stable
Ornstein–Uhlenbeck processes $(X_{t})_{t\geq 0}$ based on $n$ discrete
observations $(X_{h},X_{2h},\ldots,X_{nh})$ with an interval $h>0$. By letting
$n\to\infty$ with/without $h\to 0$, we can consider both high and low
frequently observed processes. Let $(Z^{\alpha}_{t})_{t\in{\mathbb{R}}}$ be a
symmetric $\alpha$-stable Lévy process with
${\mathbb{E}}[e^{iuZ_{1}^{\alpha}}]=e^{-|\sigma
u|^{\alpha}},\,u\in{\mathbb{R}}$, where $\sigma>0$ is the scale parameter.
Then a stationary version $(X_{t})_{t\geq 0}$ has the form

(5.1) $\displaystyle
X_{t}=\int_{-\infty}^{t}e^{-\lambda(t-v)}dZ^{\alpha}_{v}=\int_{s}^{t}e^{-\lambda(t-v)}dZ^{\alpha}_{v}+e^{-\lambda(t-s)}X_{s},\quad
t\geq 0.$

Notice that (5.1) satisfies the stochastic differential equation

$dX_{t}=-\lambda X_{t}dt+dZ^{\alpha}_{t}$

and that due to the independent increments of $(Z^{\alpha}_{t})$, the process
is a Markov process (see, e.g. [43, Sec. 3.6] for the definition and
properties of $\alpha$-stable Ornstein–Uhlenbeck processes). For this type of
Markov processes, several estimation methods based on the ch.f. have been
established. One finds a detailed review in [48, Sec. 2.2.2] (see also [45, 6,
2]). All of them are variants of the GMM estimator based on the conditional
ch.f. Our estimator also relies on the conditional ch.f., but it appears as
the result of projection of the conditional score function on trigonometric
functions.

We see our estimator more closely. Write the conditional likelihood equation

$\frac{1}{n}\sum_{t=1}^{n}S(X_{h(t+1)};\theta\mid
X_{ht})=\frac{1}{n}\sum_{t=1}^{n}\frac{f_{\theta}(X_{h(t+1)};\theta\mid
X_{ht})}{f(X_{h(t+1)};\theta\mid X_{ht})}=0,$

where $\theta=(\alpha,\sigma,\lambda)^{\top}$ is our targeting parameter
vector. Here $f(x_{h(t+1)};\theta\mid x_{ht})$ and
$f_{\theta}(x_{h(t+1)};\theta\mid x_{ht})$ are respectively the conditional
density of $X_{h(t+1)}$ given $X_{ht}$ and its derivative vector. We
approximate the likelihood equation by projecting the conditional score
$f_{\theta}/f$ onto the subspace spanned by $\\{1,g\\}$. The estimator is
obtained as a zero of this approximated equation. We call this estimation
trigonometrically approximated conditional MLE (TCMLE for short).

We present the exact form of TCMLE. In the definition (5.1) since the last two
terms are independent, the conditional ch.f. is written as

$\displaystyle\varphi_{t\mid s}(u;\theta):={\mathbb{E}}[e^{iuX_{t}}\mid
X_{s}]=e^{iue^{-\lambda(t-s)}X_{s}}e^{-|\sigma
u|^{\alpha}(\lambda\alpha)^{-1}(1-e^{-\alpha\lambda(t-s)})}.$

A conditional version of $\gamma(\theta)$ at $t$ given $s$, $t>s$ is provided
by

$\gamma_{t\mid s}(\theta)=\big{(}\varphi_{t\mid
s}^{R}(u_{1};\theta),\ldots,\varphi_{t\mid s}^{R}(u_{k};\theta),\varphi_{t\mid
s}^{I}(u_{1};\theta),\ldots,\varphi_{t\mid
s}^{I}(u_{k};\theta)\big{)}^{\top},$

where $(u_{1},\ldots,u_{k})$ is a vector of evaluation points of the ch.f.
Accordingly we have

$\gamma_{t\mid s,\theta}(\theta)=\frac{\partial\gamma_{t\mid
s}(\theta)}{\partial\theta}=\big{(}\varphi^{R}_{t\mid
s,\theta}(u_{1};\theta),\ldots,\varphi^{R}_{t\mid
s,\theta}(u_{k};\theta),\varphi^{I}_{t\mid
s,\theta}(u_{1};\theta),\ldots,\varphi^{I}_{t\mid
s,\theta}(u_{k};\theta)\big{)},$

where

$\displaystyle\varphi_{t\mid
s;\theta}^{R}(u;\theta)=\frac{\partial\varphi_{t\mid
s}^{R}(u;\theta)}{\partial\theta},\qquad\varphi_{t\mid
s;\theta}^{I}(u;\theta)=\frac{\partial\varphi_{t\mid
s}^{I}(u;\theta)}{\partial\theta}.$

The quantity corresponding to $\Sigma(\theta)$ is defined by

$\Sigma_{t\mid s}(\theta)={\mathbb{E}}_{t\mid s}[(g(X)-\gamma_{t\mid
s}(\theta))(g(X)-\gamma_{t\mid s}(\theta))^{\top}],$

where ${\mathbb{E}}_{t\mid s}[\cdot]$ denotes the expectation operator deduced
from the conditional density $f(x_{t};\theta\mid x_{s})$. Now we obtain the
trigonometrically approximated conditional score function as

$\widetilde{S}(x_{ht};\theta\mid x_{hs}):=\gamma_{ht\mid
hs,\theta}(\theta)\Sigma_{ht\mid
hs}(\theta)^{-1}\big{(}g(x_{ht})-\gamma_{ht\mid hs}(\theta)\big{)},$

so that the corresponding empirical score is given by

$\widetilde{S}_{n}(\theta)=\frac{1}{n}\sum_{t=1}^{n}\widetilde{S}(X_{h(t+1)};\theta\mid
X_{ht}).$

The TCML estimator is the solution $\hat{\theta}_{n}$ of
$\widetilde{S}_{n}(\theta)=0$.

######  Remark 5.1.

Notice that similarly to the i.i.d. case, GMM-type estimators again encounter
the singularity problem in the covariance matrix (2.10) as the number of the
grid points increases, see [3] $($cf. [48, Sec. 2.2.2]$)$. Thus one needs to
increase the grid number depending on sample sizes or needs a regularization
parameter for the optimal covariance operator. Although we do not have a
rigorous theoretical result for the non-i.i.d. case, in view of numerical
simulations, we do not observe such a singularity in TCMLE.

Hereafter, we will examine the finite sample performance of our estimator in
high frequency settings. In the literature asymptotic theories for quasi-MLE
have been established for OU processes driven by more general locally stable
Lévy processes, especially for $\lambda$ [27, 8, 9]. The main tool is locally
asymptotic stable property of the processes. The joint asymptotic distribution
of all parameters $\theta$ has been derived by [9, Theorem 3.2]. Since TMLE
converges to MLE as $k\to\infty,\,\tau\to 0$, for reasonably large $k$ and
small $\tau$ TCMLE should approximate this asymptotic distribution also. We
briefly explain this asymptotic in order to compare it with our simulation
results.

Assume that we observe $(X_{t})_{t\geq 0}$ on $[0,T]$ with frequency $h=T/n$,
that is, we observe a discretized process $(X_{t_{i}})_{0\leq i\leq n}$ with
$t_{i}=i\cdot T/n$. Let $a_{n}={\rm diag}(n^{1/2}\log
n,n^{1/2},n^{1/\alpha_{0}-1/2})$ and then as $n\to\infty$

$a_{n}(\hat{\theta}_{n}-\theta_{0})\stackrel{{\scriptstyle
d}}{{\to}}V_{0}^{-1/2}N,$

where $N$ is the standard normal r.v. independent of $V_{0}$ and

$\displaystyle V_{0}=\left(\begin{array}[]{ccc}c_{11}&c_{12}&0\\\
c_{21}&c_{22}&0\\\ 0&0&c_{33}\int_{0}^{T}X^{2}_{u}du\end{array}\right).$

Here $c_{ij},\,i,j=1,2,3$ are some constants depending on $T$. In other words,
$\hat{\theta}_{n}$ has asymptotic mixed normality. If numerical simulations
replicate this asymptotic mixed normality, then the validity of TCMLE would be
assured.

Table 3. TCMLE for $\alpha$-stable OU processes with frequency $h=0.1,0.01$

$h=10^{-1}$ | $\alpha=1.3$ | $\alpha=1.5$ | $\alpha=1.8$
---|---|---|---
$\hat{\theta}$ | $\hat{\alpha}$ | $\hat{\sigma}$ | $\hat{\lambda}$ | $\lambda^{\ast}$ | $\hat{\alpha}$ | $\hat{\sigma}$ | $\hat{\lambda}$ | $\lambda^{\ast}$ | $\hat{\alpha}$ | $\hat{\sigma}$ | $\hat{\lambda}$ | $\lambda^{\ast}$
Mean | $1.299$ | $1.004$ | $1.001$ | $-.0748$ | $1.500$ | $1.003$ | $1.004$ | $-.102$ | $1.801$ | $1.001$ | $1.013$ | $-.110$
Sd | $0.050$ | $0.064$ | $0.036$ | $0.914$ | $0.050$ | $0.052$ | $0.061$ | $1.128$ | $0.044$ | $0.036$ | $0.112$ | $1.384$
Skew | $0.073$ | $0.221$ | $0.284$ | $0.169$ | $0.044$ | $0.147$ | $0.336$ | $0.074$ | $-0.126$ | $0.220$ | $0.383$ | $0.000$
Kur | $2.626$ | $3.061$ | $5.407$ | $3.296$ | $2.682$ | $3.083$ | $3.875$ | $3.429$ | $2.575$ | $3.086$ | $3.259$ | $2.971$
Mean | $1.302$ | $1.001$ | $0.999$ | $0.008$ | $1.503$ | $1.000$ | $1.001$ | $-.043$ | $1.803$ | $1.000$ | $1.007$ | $-.078$
Sd | $0.042$ | $0.053$ | $0.025$ | $0.922$ | $0.042$ | $0.042$ | $0.047$ | $1.139$ | $0.037$ | $0.028$ | $0.090$ | $1.405$
Skew | $0.135$ | $0.117$ | $0.183$ | $-.008$ | $0.003$ | $0.060$ | $0.336$ | $0.077$ | $-.125$ | $0.211$ | $0.338$ | $0.025$
Kur | $2.753$ | $2.764$ | $4.504$ | $3.230$ | $2.764$ | $2.764$ | $4.592$ | $3.136$ | $2.830$ | $3.023$ | $3.606$ | $3.232$
$h=10^{-2}$ | $\alpha=1.3$ | $\alpha=1.5$ | $\alpha=1.8$
$\hat{\theta}$ | $\hat{\alpha}$ | $\hat{\sigma}$ | $\hat{\lambda}$ | $\lambda^{\ast}$ | $\hat{\alpha}$ | $\hat{\sigma}$ | $\hat{\lambda}$ | $\lambda^{\ast}$ | $\hat{\alpha}$ | $\hat{\sigma}$ | $\hat{\lambda}$ | $\lambda^{\ast}$
Mean | $1.307$ | $1.020$ | $1.033$ | $-.206$ | $1.505$ | $1.012$ | $1.061$ | $-.022$ | $1.800$ | $1.005$ | $1.123$ | $-.284$
Sd | $0.129$ | $0.194$ | $0.182$ | $0.789$ | $0.110$ | $0.143$ | $0.260$ | $0.094$ | $0.070$ | $0.078$ | $0.389$ | $1.275$
Skew | $0.089$ | $0.737$ | $1.703$ | $-.382$ | $-.064$ | $0.646$ | $1.203$ | $-.021$ | $-.360$ | $0.602$ | $0.901$ | $-.357$
Kur | $2.703$ | $3.832$ | $10.67$ | $4.415$ | $2.792$ | $3.544$ | $5.925$ | $2.974$ | $2.795$ | $3.298$ | $4.218$ | $3.427$
Mean | $1.309$ | $1.006$ | $1.010$ | $-.089$ | $1.506$ | $1.004$ | $1.025$ | $-.133$ | $1.802$ | $1.002$ | $1.074$ | $-.214$
Sd | $0.107$ | $0.156$ | $0.109$ | $0.0683$ | $0.093$ | $0.118$ | $0.170$ | $0.851$ | $0.057$ | $0.063$ | $0.295$ | $1.195$
Skew | $0.166$ | $0.428$ | $0.453$ | $-.101$ | $-.035$ | $0.469$ | $0.814$ | $-.017$ | $-.376$ | $0.537$ | $1.057$ | $-.072$
Kur | $2.844$ | $2.910$ | $4.889$ | $3.355$ | $2.689$ | $3.133$ | $5.172$ | $3.219$ | $2.871$ | $3.442$ | $5.089$ | $3.150$

Monte Carlo simulations of TCMLE for parameters $(\alpha,\sigma,\lambda)$ of
$\alpha$-stable OU processes with frequency $h=0.1$ (top two rows) and
$h=0.01$ (bottom two rows). For the first and third rows we take $n=1000$ and
for the second and fourth rows $n=1500$. The estimation procedures are
repeated $500$ times. Here $\lambda^{\ast}$ is a modified version of
$\hat{\lambda}$, see (5.3) for definition. In each cell $4$ statistical
characteristics of estimators are presented.

In our simulation, we set $h=0.1,0.01$ and $n=1000,1500$ so that a path of
process $(X_{t})_{t\geq 0}$ on $t\in[0,nh]$ is observed in $n$ different
times. We repeated the estimation procedure $500$ times in three different
cases $\alpha=1.3,1.5$ and $1.8$ with $\sigma=\lambda=1.0$. We uses $101$
equidistant points $u=(0.05,\dots,5.05)$ for evaluation of empirical
conditional ch.f.

Figure 1.  QQ plots for estimators $\hat{\lambda}_{n}$ and its modification
$\lambda^{\ast}_{n}$ with sample sizes $n=1000$ (left two columns) and
$n=1500$ (right tow columns), where outliers for $\lambda^{\ast}_{n}$ are
removed. From top to bottom we change parameter $\alpha=1.3,1.5,1.8$
successively.

The results are summarized in Table 3, where $4$ statistics are presented
similarly as before. Judging from the mean values, all estimators are
unbiased, while obviously the standard deviations of $n=1500$ are smaller than
$n=1000$. The standard deviations for $h=0.1$ are smaller than those for
$h=0.01$. In view of skewness and kurtosis, asymptotic normality follows for
$\hat{\alpha}_{n}$ and $\hat{\sigma}_{n}$ (We check QQ-plots also, though we
do not present here), though these values for $\hat{\lambda}_{n}$ show quite
different values. Values of $\hat{\lambda}_{n}$ are studied in detail bellow.
We find difficulty in confirming orders $a_{n}$ directly, since accuracy of
the asymptotic scale matrix $V_{0}$ depends on both $h$ and $T=nh$.
Alternatively by observing the asymptotic mixed normality for
$\hat{\lambda}_{n}$ we see that TCMLE properly functions.

Thus, we consider skewness, kurtosis, and QQ-plots for a version

(5.3)
$\displaystyle\lambda^{\ast}_{n}:=\sqrt{W}(\hat{\lambda}_{n}-\bar{\lambda}),\quad
W=\int_{0}^{nh}X^{2}_{v}dv,$

where $\bar{\lambda}$ is the mean of $500$ samples of $\hat{\lambda}_{n}$. The
values of $\lambda_{n}^{\ast}$ are presented in Table 3. Since the original
$\lambda^{\ast}_{n}$ sometimes takes huge outliers (mostly negative), we
present further modified values of $\lambda^{\ast}_{n}$, that is, we remove
several very small values. In cases $\alpha=1.3,n=1000,1500,h=0.01$ and
$\alpha=1.5,n=1500,h=0.01$ we remove $1$ outlier, and in cases
$\alpha=1.3,1.5,h=0.01,n=1000$, we remove $5$ outliers. Notice that skewness
and kurtosis are sensitive to outliers, and the two characteristics of
$\lambda^{\ast}_{n}$ often show worse values than those of
$\hat{\lambda}_{n}$; for instance, in $\alpha=1.3,1.5,h=0.01,n=1000$, and then
$($skewness, kurtosis$)$ are $(-1,43,11.44)$ and $(-3.50,31.37)$ respectively.

In Figure 1, we present QQ-plots of $\hat{\lambda}_{n}$ and
$\lambda^{\ast}_{n}$ for $\alpha=1.3,1.5,1.8$ and sample sizes $n=1000,1500$.
Similarly to Table 3, outliers of $\lambda^{\ast}_{n}$ are removed. In view of
these graphs, $\hat{\lambda}_{n}$ is well-rescaled, and plots for
$\lambda^{\ast}_{n}$ almost show normality. Judging from these facts we
conclude that the theory of asymptotic mixed normality for $\hat{\lambda}_{n}$
is depicted correctly with TCMLE.

##  Appendix A Supplementary results and proofs for Section 3.1

###  A.1. Proof of Lemma 2.2

$(\mathrm{i})$ It suffices to take $k=2$ with non-zero $|u_{1}|\neq|u_{2}|$ in
(2.4) and prove that rows of $\gamma_{\theta}(\theta)$ are linearly
independent. We take a similar approach as in the proof of [28, Theorem 3.1],
where the non-singularity of the Fisher information matrix is shown for the
stable law. Let ${\bf a}=(a_{1},\ldots,a_{4})^{\top}\in{\mathbb{R}}^{4}$ and
we show that

(A.1) $\displaystyle\gamma_{\theta}(\theta)^{\top}{\bf a}=\bf 0$

implies ${\bf a}=\bf 0$ by indirect proof. Assume (A.1) for ${\bf a}\neq\bf
0$, then

$\big{(}\varphi^{R}_{\theta}(u_{1};\theta)+i\varphi^{I}_{\theta}(u_{1};\theta),\varphi^{R}_{\theta}(u_{2};\theta)+i\varphi^{I}_{\theta}(u_{2};\theta)\big{)}^{\top}{\bf
a}=\big{(}\varphi_{\theta}(u_{1};\theta),\varphi_{\theta}(u_{2};\theta)\big{)}^{\top}{\bf
a}=\bf 0$

holds. Therefore, the complex vectors

$\left(\begin{array}[]{c}\varphi_{\mu}(u_{1};\theta)\\\
\varphi_{\mu}(u_{2};\theta)\end{array}\right),\left(\begin{array}[]{c}\varphi_{\sigma}(u_{1};\theta)\\\
\varphi_{\sigma}(u_{2};\theta)\end{array}\right),\left(\begin{array}[]{c}\varphi_{\alpha}(u_{1};\theta)\\\
\varphi_{\alpha}(u_{2};\theta)\end{array}\right),\left(\begin{array}[]{c}\varphi_{\beta}(u_{1};\theta)\\\
\varphi_{\beta}(u_{2};\theta)\end{array}\right)$

are linearly dependent with real coefficients ${\bf a}\neq\bf 0$. For
convenience we omit the parameter $\theta$ in $\varphi(u;\theta)$ and
$\varphi_{\theta_{i}}(u;\theta)$ and functions of them.

For $\psi(u)=\log\varphi(u)$, we define
$\psi_{\theta_{i}}(u)=\partial\psi(u)/\partial\theta_{i},\,i=1,\ldots,4$, so
that we have $\varphi_{\theta_{i}}(u)=\psi_{\theta_{i}}(u)\varphi(u)$, where

$\displaystyle\psi_{\mu}(u)$ $\displaystyle=iu,$
$\displaystyle\psi_{\sigma}(u)$
$\displaystyle=-\alpha|u|^{\alpha}+iu(\alpha|u|^{\alpha-1}-1)\beta\tan(\pi\alpha/2),$
$\displaystyle\psi_{\alpha}(u)$
$\displaystyle=-|u|^{\alpha}\log|u|+iu|u|^{\alpha-1}\log|u|\beta\tan(\pi\alpha/2)+iu(|u|^{\alpha-1}-1)(\pi\beta/2)\cos^{-2}(\pi\alpha/2),$
$\displaystyle\psi_{\beta}(u)$
$\displaystyle=iu(|u|^{\alpha-1}-1)\tan(\pi\alpha/2).$

Now we look
$\sum_{i=1}^{4}a_{i}\varphi_{\theta_{i}}(u_{k})=\sum_{i=1}^{4}a_{i}\psi_{\theta_{i}}(u_{k})\varphi(u_{k})$
for $k=1,2$. If these sums are zero, then

$\sum_{i=1}^{4}a_{i}\psi_{\theta_{i}}(u_{k})=0.$

In view of the real parts of $\psi_{\sigma}$ and $\psi_{\alpha}$,

$a_{2}\alpha+a_{3}\log|u_{k}|=0,\quad k=1,2,$

should hold for $|u_{1}|\neq|u_{2}|$. This is impossible unless
$a_{2}=a_{3}=0$. Moreover, from $\psi_{\mu}$ and $\psi_{\beta}$, we have

$a_{1}-a_{4}\tan(\pi\alpha/2)+a_{4}\tan(\pi\alpha/2)|u_{k}|^{\alpha-1}=0,\quad
k=1,2,$

which is impossible unless $a_{1}=a_{4}=0$. Notice that at $\alpha=1$, the
above equation reduces to $a_{1}=a_{4}(2/\pi)\log|u_{k}|$, and thereby the
result is kept. Thus, we obtain ${\bf a}=\bf 0$, which is a contradiction.
$(\mathrm{ii})$ Let ${\bf a}=(a_{1},\ldots,a_{2k})^{\top}\neq\bf 0$. Since
trigonometric functions in $g(x)$ constitute a basis on $L^{2}(-\pi,\pi)$ (see
e.g. [7, Theorem 1]), they are linearly independent. Thus, ${\bf
a}^{\top}g(x)=0$ for any $x\in(-\pi,\pi)$ if and only if ${\bf a}=\bf 0$, cf.
[23, p.9]. This implies that ${\bf a}^{\top}(g(x)-\gamma(\theta))$ is non-
deterministic as a function of $x$ for ${\bf a}\neq\bf 0$. Now for any ${\bf
a}\neq\bf 0$ it follows from Fubini that

${\bf a}^{\top}\Sigma(\theta){\bf a}=\int_{-\infty}^{\infty}\\{{\bf
a}^{\top}(g(x)-\gamma(\theta))\\}^{2}f(x;\theta)dx>0,$

since the support of $f(x;\theta)$ is the whole real line and ${\bf
a}^{\top}g(x)$ is continuous in $x$. Thus, $\Sigma(\theta)$ is positive
definite. $\square$

######  Remark A.1.

$(\mathrm{i})$ We could not remove the condition $|u_{i}|\neq|u_{j}|$. Indeed,
we take $(u_{1},u_{2},\ldots,u_{k}),\,k\geq 3$ such that $u_{1}=-u_{2}$ and
$u_{i}=0,\,i\geq 3$, and ${\bf a}=(1,-1,0,\ldots,0)$, and then ${\bf
a}^{\top}g(x)=0$ holds for all $x\in{\mathbb{R}}$.
$(\mathrm{ii})$ In view of [7, Theorem 1], we do not necessarily use the same
set of points $(u_{1},u_{2},\ldots,u_{k})$ for both $\varphi^{R}$ and
$\varphi^{I}$. We might obtain a better estimator with two different sets of
evaluation points.

###  A.2. Proof of Theorem 3.1

We will check the conditions of [47, Theorem 5.41] where $\psi_{\theta}(x)$
there corresponds to $\widetilde{S}(x;\theta)$ here. Since the conditions of
[47, Theorems 5.41 and 5.42] are the same, the existence of a consistent root
follows from Theorem 5.42, while asymptotic normality and (3.1) directly
follow from Theorem 5.41.

In view of (2.6), $\widetilde{S}(x;\theta)$ is twice continuously
differentiable for every $x$ because the ch.f. $\varphi(x;\theta)$ is three
times continuously differentiable w.r.t. $\theta$ on
$C\subset\mathring{\Theta}$. Note that in [28], the derivatives of
$\varphi(u;\theta)$ w.r.t. $\theta$ are obtained up to the second order. In a
similar manner we could obtain the third-order derivatives even in the
complicated case of $\alpha=1$.

From (2.6) and (2.7),
${\mathbb{E}}_{\theta_{0}}[\widetilde{S}(X;\theta_{0})]=0$ and
${\mathbb{E}}_{\theta_{0}}[|\widetilde{S}(X;\theta_{0})|^{2}]<\infty$ follow.
Write the first-order derivative matrix ($4\times 4$ denominator layout) as

$\frac{\partial\widetilde{S}(x;\theta)}{\partial\theta}=\Big{(}\frac{\partial\widetilde{S}}{\partial\mu}^{\top},\,\frac{\partial\widetilde{S}}{\partial\sigma}^{\top},\,\frac{\partial\widetilde{S}}{\partial\alpha}^{\top},\,\frac{\partial\widetilde{S}}{\partial\beta}^{\top}\Big{)}^{\top},$

where each row $\partial\widetilde{S}/\partial\theta_{j},\,j=1,\ldots,4$ is a
$1\times 4$ vector and the $i,j$ element of
$\partial\widetilde{S}/\partial\theta$ is

(A.2)
$\displaystyle\Big{[}\frac{\partial\widetilde{S}(x;\theta)}{\partial\theta}\Big{]}_{ij}$
$\displaystyle=\Big{[}\frac{\partial^{2}\gamma(\theta)}{\partial\theta_{j}\partial\theta_{i}}+\frac{\partial\gamma(\theta)}{\partial\theta_{i}}\Sigma(\theta)^{-1}\frac{\partial\Sigma(\theta)}{\partial\theta_{j}}\Big{]}\Sigma(\theta)^{-1}(g(x)-\gamma(\theta))$
$\displaystyle\qquad+\frac{\partial\gamma(\theta)}{\partial\theta_{j}}\Sigma(\theta)^{-1}\frac{\partial\gamma(\theta)}{\partial\theta_{i}}^{\top}.$

Thus
${\mathbb{E}}[\partial\widetilde{S}(X;\theta_{0})/\partial\theta]=\gamma_{\theta}(\theta_{0})\Sigma(\theta_{0})^{-1}\gamma_{\theta}(\theta_{0})^{\top}$
exists and is non-singular according to Lemma 2.2 (cf. Remark 2.1).

From the form of the first-order partial derivative (A.2), it is not difficult
to observe

(A.3)
$\displaystyle\frac{\partial^{2}\widetilde{S}(x;\theta)}{\partial\theta\partial\theta_{k}}=A(\theta)(g(x)-\gamma(\theta))+B(\theta),\quad
k=1,2,3,4,$

where $A(\theta)$ and $B(\theta)$ are continuous matrix functions in
$\theta\in C$. Since $C$ is compact and $g$ is integrable, we could find a
dominant integrable function $\ddot{\psi}(x)$ for each element of (A.3). Now
due to [47, Theorem 5.42], there exists a consistent sequence of roots
$\hat{\theta}_{n}$, and by [47, Theorem 5.41] this sequence satisfies
asymptotic normality together with (3.1). $\square$

##  Appendix B Supplementary results and proofs for Section 3.2

For the proofs of Theorem 3.4 (Subsection B.1) and Theorem 3.6 (Subsection
B.2), we need supplementary results. We use two theorems from [1] (Theorems
B.3 and B.4), and the following lemmas and a proposition. Throughout this
section, let $c,\,c^{\prime},\,c^{\prime\prime}$, etc. be generic positive
constants whose values are not of interest. For a given function $g(x)$, we
denote by $g_{\tau}(x)$ the $2\pi/\tau$ periodic function:

$g_{\tau}(x)=\sum_{j=-\infty}^{\infty}g\,\big{(}x+2\pi
j/\tau\big{)},\quad\tau>0.$

The first lemma is an easy application of Lemma 6.2 in [1].

######  Lemma B.1.

Let $f_{\tau},\,f_{\tau}^{\prime},\,f_{\tau,\theta_{i}}$ and
$f_{\tau,\theta_{i}}^{\prime},\,i=1,\ldots,4$ be periodic functions
respectively based on $f,\,f^{\prime},\,f_{\theta_{i}}$ and
$f_{\theta_{i}}^{\prime}$. Then, for any $\theta\in C$, we have

$\displaystyle\sup_{x\in I_{\tau}}|f_{\tau}(x;\theta)-f(x;\theta)|$
$\displaystyle\leq c_{1}\tau^{1+\alpha},\qquad\sup_{x\in
I_{\tau}}|f_{\tau}^{\prime}(x;\theta)-f^{\prime}(x;\theta)|\leq
c_{2}\tau^{2+\alpha},$ $\displaystyle\sup_{x\in
I_{\tau}}|f_{\tau,\theta_{i}}(x;\theta)-f_{\theta_{i}}(x;\theta)|$
$\displaystyle\leq c_{3}\tau^{1+\alpha}\log 1/\tau,\qquad\sup_{x\in
I_{\tau}}|f_{\tau,\theta_{i}}^{\prime}(x;\theta)-f_{\theta_{i}}^{\prime}(x;\theta)|\leq
c_{4}\tau^{2+\alpha}\log 1/\tau,$

where $c_{i},\,i=1,\ldots,4$ are positive constants that depend on neither
$\theta$ nor $x$.

###### Proof.

Since the proof is a slight modification of that for [1], we only state the
difference in the condition. For sufficiently large $|x|$, we clearly have
$f=O(|x|^{-(1+\alpha)})$. In view of Lemma 2.1 in [28],
$f,\,f^{\prime},\,f_{\theta_{i}}$ and $f_{\theta_{i}}^{\prime}$ are continuous
in $x\in{\mathbb{R}}$ and satisfy

$\displaystyle|f^{\prime}|=O(|x|^{-(2+\alpha)}),\quad|f_{\theta_{i}}|=O(|x|^{-(1+\alpha)}\log|x|),\quad|f_{\theta_{i}}^{\prime}|=O(|x|^{-(2+\alpha)}\log|x|),$

for sufficiently large $|x|$, where the orders of left are uniform in $i$. We
replace the tail condition (6.1) of [1, Lemma 6.2] with the above bounds, and
then the results follow from the reproduction of the proof. ∎

As stated before Theorem 3.6, we evaluate the distance between $\widetilde{S}$
and $S$ through the two-step approximation; we will evaluate the distance
between $S$ and $S_{\tau}$ in Lemma B.2 and that between $S_{\tau}$ and
$\widetilde{S}$ in Proposition B.5. In this regard we define several
notations. Denote the $i$-th element of $S(x;\theta)$ by $S_{i}(x;\theta)$ for
$i=1,\ldots,4$, namely,

$S_{i}(x;\theta)=\frac{f_{\theta_{i}}(x;\theta)}{f(x;\theta)}.$

Similarly, denote the $i$-th component of $S_{\tau}(x;\theta)$ and
$\widetilde{S}(x;\theta)$ respectively by $S_{\tau,i}(x;\theta)$ and
$\widetilde{S}_{i}(x;\theta)$ for $i=1,\ldots,4$. Notice that in the proofs,
we often omit $i$ when the corresponding proof holds uniformly in $i$. In such
a case we make a caution in the beginning.

######  Lemma B.2.

For every $x$, $S_{\tau,i}(x;\theta)$ converges to
$S_{i}(x;\theta),\,i=1,\ldots,4$ uniformly in $\theta\in C$ as $\tau\to 0$.

###### Proof.

Since the proof is valid for all $i$, we omit subscript $i$ of $\theta_{i}$
and write $S(x;\theta),\,S_{\tau}(x;\theta),\,f_{\theta},\,f_{\tau,\theta}$ to
indicate elements of them for convenience. Since

(B.1) $\displaystyle
S_{\tau}(x;\theta)-S(x;\theta)=\frac{f_{\tau,\theta}(x;\theta)-f_{\theta}(x;\theta)}{f_{\tau}(x;\theta)}+S(x;\theta)\frac{f(x;\theta)-f_{\tau}(x;\theta)}{f_{\tau}(x;\theta)},$

we evaluate

$\displaystyle|S_{\tau}(x;\theta)-S(x;\theta)|\leq\Big{|}\frac{f_{\tau,\theta}-f_{\theta}}{f}\Big{|}+|S|\Big{|}\frac{f-f_{\tau}}{f}\Big{|}.$

Since $f$ is unimodal and the support is ${\mathbb{R}}$, $f>0$ holds on
${\mathbb{R}}$. Indeed $|f(x;\theta)|\sim c|x|^{-(1+\alpha)}$ for sufficiently
large $|x|$. Due to Proposition 2.3 of [28], $S(x;\theta)=O(\log|x|)$. Now in
view of Lemma B.1 the left hand-side converges uniformly in $\theta\in C$ as
$\tau\to 0$. ∎

The next proposition relies on [1, Theorems 6.5 and 6.6], and for convenience
we state them.

######  Theorem B.3 (Theorem 6.5 in [1]).

Let $h(x)$ be a differentiable function of period $2\pi/\tau$, such that
$|h^{\prime}(x)|<A$ for all $x$. Then there exists a trigonometric polynomial
of degree $k$, $q(x)=a_{0}+\sum_{j=1}^{k}(a_{j}\cos j\tau x+b_{j}\sin j\tau
x)$, satisfying

$\sup_{x\in{\mathbb{R}}}|q(x)-h(x)|\leq KA/(k\tau),$

where $K$ is an absolute constant.

Notice that Theorem B.3 is based on [20, Theorem 1, p.2], where one could see
more precisely that $K$ does not depend on any specialization of the
corresponding functions.

######  Theorem B.4 (Theorem 6.6 in [1]).

Let $h(x)$ be continuous, and let $r(x)$ and $s(x)$ be $k$th degree
trigonometric polynomials in $x$ such that

(B.2) $\displaystyle\sup_{x\in I_{\tau}}|h(x)-r(x)|<\delta$

and

(B.3) $\displaystyle\int_{I_{\tau}}|h(x)-s(x)|^{2}dx\leq d.$

Then

$\sup_{x\in I_{\tau}}|h(x)-s(x)|\leq 4(k\tau d)^{1/2}+5\delta.$

######  Proposition B.5.

There exists $\tau_{0}>0$ such that when $\tau\leq\tau_{0}$ we could take an
upper bound $A>0$ with which
$|S_{\tau,i}^{\prime}(x;\theta)|<A,\,i=1,\ldots,4$ hold uniformly in
$\tau\leq\tau_{0}$, $x\in{\mathbb{R}}$ and $\theta\in C$. Moreover, for any
$i$,

(B.4) $\displaystyle\sup_{x\in
I_{\tau}}|S_{\tau,i}(x;\theta)-\widetilde{S}_{i}(x;\theta)|\leq
KA\\{4(\tau^{2+\alpha}k)^{-1/2}+5(\tau k)^{-1}\\},$

where $K$ is a positive constant which does not depend on $\tau,\,k$ and
$\theta$.

###### Proof.

By the periodicity, it is enough to show that there exists $\tau_{0}>0$ such
that $\sup_{\tau\leq\tau_{0}}\sup_{x\in I_{\tau},\theta\in
C}|S_{\tau,i}^{\prime}(x;\theta)|<A$. For convenience we omit subscript $i$ of
$\theta_{i}$ and write, for instance,
$f_{\theta},\,f_{\tau,\theta},\,f_{\theta}^{\prime}$ and
$f_{\tau,\theta}^{\prime}$ since the proof is valid for all $i$. Accordingly
$i$ in $S_{\tau,i}$ and $\widetilde{S}_{i}$ and their derivatives is also
abbreviated. Firstly we consider $|S_{\tau,i}^{\prime}(x;\theta)|$. We
evaluate

$S_{\tau}^{\prime}(x)=\frac{f_{\tau,\theta}^{\prime}(x)f_{\tau}(x)}{f_{\tau}(x)^{2}}-\frac{f_{\tau}^{\prime}(x)f_{\tau,\theta}(x)}{f_{\tau}(x)^{2}}:=J_{1}-J_{2}.$

The numerator of $J_{1}$ is dominated by

$\displaystyle|f_{\tau,\theta}^{\prime}f_{\tau}|$
$\displaystyle\leq|f_{\tau,\theta}^{\prime}-f_{\theta}^{\prime}||f_{\tau}-f|+|f_{\tau,\theta}^{\prime}-f_{\theta}^{\prime}|f+|f_{\theta}^{\prime}||f_{\tau}-f|+|f_{\theta}^{\prime}f|$
$\displaystyle\leq c\big{\\{}\tau^{3+2\alpha}\log 1/\tau+\tau^{2+\alpha}\log
1/\tau\cdot
f+\tau^{1+\alpha}|f_{\theta}^{\prime}|+|f_{\theta}^{\prime}f|\big{\\}},$

where in the last step we use Lemma B.1. Similarly the numerator of $J_{2}$ is
bounded by

$\displaystyle|f_{\tau}^{\prime}f_{\tau,\theta}|$
$\displaystyle\leq|f_{\tau}^{\prime}-f^{\prime}||f_{\tau,\theta}-f_{\theta}|+|f_{\tau}^{\prime}-f^{\prime}||f_{\theta}|+|f^{\prime}||f_{\tau,\theta}-f_{\theta}|+|f^{\prime}f_{\theta}|$
$\displaystyle\leq c\big{\\{}\tau^{3+2\alpha}\log
1/\tau+\tau^{2+\alpha}|f_{\theta}|+\tau^{1+\alpha}\log
1/\tau\cdot|f^{\prime}|+|f^{\prime}f_{\theta}|\big{\\}}.$

Next the denominator is evaluated. It follows by the definition that

$f_{\tau}(x)-f(x)=\sum_{j\neq 0}f(x+2\pi j/\tau).$

We recall that for sufficiently large $|x|$, there exists a uniform constant
$\underline{c}>0$ in $\theta\in C$ such that
$f(x;\theta)\geq\underline{c}|x|^{-(1+\alpha)}$. Thus, if $x\in I_{\tau}$, so
that $\pi/\tau(2j-1)\leq x+2\pi j/\tau\leq\pi/\tau(2j+1)$, we have

$\sum_{j\neq 0}f(x+2\pi j/\tau)\geq\underline{c}\sum_{j\neq 0}|3\pi
j/\tau|^{-(1+\alpha)}\geq c^{\prime}\tau^{1+\alpha},$

for sufficiently small $\tau$. Hence $f_{\tau}(x)-f(x)\geq c\tau^{1+\alpha}$
on $x\in I_{\tau}$, which yields

(B.5) $\displaystyle f_{\tau}(x;\theta)^{2}\geq
c(f(x;\theta)^{2}+\tau^{2(1+\alpha)})\geq
c^{\prime}f(x;\theta)\tau^{1+\alpha}.$

Now, correcting the above results, we obtain

$J_{1}\leq c\big{(}\tau\log
1/\tau+|f_{\theta}^{\prime}/f|\big{)}\quad\text{and}\quad J_{2}\leq
c\big{(}\tau\log
1/\tau+\tau|f_{\theta}/f|+|f^{\prime}/f^{2}|\tau^{1+\alpha}\log
1/\tau+|f^{\prime}f_{\theta}/f^{2}|\big{)}.$

Then due to the tail behaviors of $f,f^{\prime},f_{\theta}$ and
$f_{\theta}^{\prime}$ in Lemma 2.1 in [28], for some small $\tau_{0}>0$ it is
easy to find a uniform bound which does not depend on $\tau\leq\tau_{0}$,
$x\in I_{\tau}$, and even $\theta\in C$. We specially notice that
$|f^{\prime}/f^{2}|=O(|x|^{\alpha})$ and
$|f^{\prime}f_{\theta}/f^{2}|=O(|x|^{-1}\log|x|)$ for sufficiently large
$|x|$. Thus, we prove the existence of $A$ such that
$|S_{\tau,i}^{\prime}|\leq A$ for all $i$.

For the inequality (B.4), we rely on Theorem B.4 and will check its conditions
(B.2) and (B.3). In our case, $h(x)=S_{\tau}(x;\theta)$ and
$s(x)=\widetilde{S}(x;\theta)$. Due to Theorem B.3 and the result just proved,
(B.2) holds with $\delta=KA/(k\tau)$, where $r(x)$ is an unspecified
trigonometric polynomial. For (B.3) let $m_{\tau}=\min_{x\in
I_{\tau}}f_{\tau}(x;\theta)\geq c\tau^{1+\alpha}$ and we obtain

$\displaystyle\int_{I_{\tau}}(S_{\tau}(x;\theta)-\widetilde{S}(x;\theta))^{2}dx$
$\displaystyle\leq\frac{1}{m_{\tau}}\int_{I_{\tau}}|\widetilde{S}(x;\theta)-S_{\tau}(x;\theta)|^{2}f_{\tau}(x;\theta)dx$
$\displaystyle\leq\frac{1}{m_{\tau}}\int_{I_{\tau}}|r(x)-S_{\tau}(x;\theta)|^{2}f_{\tau}(x;\theta)dx$
$\displaystyle\leq\frac{1}{m_{\tau}}\Big{|}\frac{KA}{k\tau}\Big{|}^{2}\leq
c\Bigg{(}\frac{KA}{k\tau^{(3+\alpha)/2}}\Bigg{)}^{2},$

where the second inequality follows from the fact that
$\widetilde{S}(x;\theta)$ is the projection of $S_{\tau}(x;\theta)$ on
$L^{2}_{\tau}(f)$. Now we get (B.4) through Theorem B.4. ∎

###  B.1. Proof of Theorem 3.4

###### Proof of Theorem 3.4.

Because $\delta$ can be arbitrarily small without loss of generality, we
assume that $\bar{B}_{\delta}(\theta_{0})\subset C$. Thus, we confine our
parameter space to $\bar{B}_{\delta}(\theta_{0})$ and show that any sequence
$\tilde{\theta}_{n}$ such that
$\tilde{\theta}_{n}\in\bar{B}_{\delta}(\theta_{0})$ and
$|\widetilde{S}_{n}(\tilde{\theta}_{n})|=o_{p}(1)$ also satisfies
$\tilde{\theta}_{n}\stackrel{{\scriptstyle p}}{{\to}}\theta_{0}$.

Let
$\widetilde{S}(\theta)={\mathbb{E}}_{\theta_{0}}[\widetilde{S}(X;\theta)]$.
Then, by [47, Theorem 5.7], it is enough to show that there exist $\delta>0$
such that for any $0<\varepsilon\leq\delta/2$

(B.6)
$\displaystyle\inf_{\theta\in\bar{B}_{\delta}(\theta_{0}),\,|\theta-\theta_{0}|\geq\varepsilon}|\widetilde{S}(\theta)|>|\widetilde{S}(\theta_{0})|=0$

and that

(B.7)
$\displaystyle\sup_{\theta\in\bar{B}_{\delta}(\theta_{0})}|\widetilde{S}_{n}(\theta)-\widetilde{S}(\theta)|\stackrel{{\scriptstyle
p}}{{\to}}0.$

Notice that the TML estimator is included in the class of estimators we
consider. Indeed, let
$\hat{\theta}_{n}=\arg\min_{\theta\in\bar{B}_{\delta}(\theta_{0})}\,|\widetilde{S}_{n}(\theta)|$.
Then we see easily from (B.7) that
$|\widetilde{S}_{n}(\hat{\theta}_{n})|\leq|\widetilde{S}_{n}(\theta_{0})|\stackrel{{\scriptstyle
p}}{{\to}}|\widetilde{S}(\theta_{0})|=0$.

For any given $k$ and $\tau>0$, the condition (B.6) follows from Lemma 2.2 and
continuity of $\tilde{I}(\theta)$. In case $\tau\to 0$, we have
$\widetilde{I}(\theta)\to I(\theta)$ for all
$\theta\in\bar{B}_{\delta}(\theta_{0})$ if
$k^{-1}=o(\tau^{2+\alpha_{0}+\delta})$, which is established at the end of the
proof . Because $I(\theta)$ is continuous and positive definite at
$\theta_{0}$ (see the proof of [28, Theorem 3.1]), we establish the first
condition even when $\tau\to 0$.

The condition (B.7) is satisfied if each element of $\widetilde{S}(x;\theta)$
is continuous in $\theta$ for every $x$ and is bounded by an integrable
function (the envelope condition) [47, p.46]. As the proof is similar for any
elements, we only establish the result for $\widetilde{S}_{3}(x;\theta)$, the
third element of $\widetilde{S}(x;\theta)$. It is clear that
$\widetilde{S}_{3}(x;\theta)$ is continuous in $\theta$ for any fixed $k$ and
$\tau$. Moreover, it follows from Proposition B.5 and Lemma B.2 that
$\widetilde{S}_{3}(x;\theta)\to S_{3}(x;\theta)$ uniformly over
$\theta\in\bar{B}_{\delta}(\theta_{0})$ if $\tau\to 0$ and
$k^{-1}=o(\tau^{2+\alpha_{0}+\delta})$. Since $S_{3}(x;\theta)$ is continuous
in $\theta$, so is $\widetilde{S}_{3}(x;\theta)$ even when $\tau\to 0$. For
the envelope condition, we use the following inequality

(B.8)
$\displaystyle|\widetilde{S}_{3}(x;\theta)|\leq|S_{\tau,3}(x;\theta)|+|\widetilde{S}_{3}(x;\theta)-S_{\tau,3}(x;\theta)|.$

The second term, which is periodic with period $2\pi/\tau$, has the same bound
as that of Proposition B.5. Thus, if $k^{-1}=o(\tau^{2+\alpha_{0}+\delta})$,
it is bounded for any $\theta\in\bar{B}_{\delta}(\theta_{0})$. For the first
term, we have

(B.9)
$\displaystyle|S_{\tau,3}(x;\theta)|\leq\frac{|f_{\alpha}(x;\theta)|}{f(x;\theta)}+\frac{\sum_{j\neq
0}|f_{\alpha}(x+2\pi
j/\tau;\theta)|}{f_{\tau}(x;\theta)}=:|S_{3}(x;\theta)|+R_{3}(x;\theta).$

The first part satisfies $O(\log|x|)$ for sufficiently large $|x|$ (see [28,
Proposition 2.3]). Since $S_{\tau,3}(x;\theta)$ is periodic, we consider the
bound of $R_{3}(x;\theta)$ on $I_{\tau}$.

The interval $I_{\tau}$ is further divided into two regions: $|x|\leq M$ and
$M<|x|\leq\pi/\tau$, where the constant $M$ is chosen such that
$\underline{c}x^{-(1+\alpha)}\leq f(x;\theta)\leq\overline{c}x^{-(1+\alpha)}$
holds for all $|x|>M$ and all $\theta\in C$. This is possible due to the tail
property of stable laws (see, e.g. [43, Proposition 1.2.15]). Here,
$\underline{c}\leq\overline{c}$ are constants that do not depend on $\theta$.
Since the tail of $|f_{\alpha}(x;\theta)|$ is bounded by
$c|x|^{-(1+\alpha)}\log|x|$, which is decreasing on $|x|>M\,(\geq e)$, the
numerator of $R_{3}$ is bounded on $|x|\leq\pi/\tau$ by

$\displaystyle c\sum_{j\neq 0}|x+2\pi j/\tau|^{-(1+\alpha)}\log|x+2\pi
j/\tau|$ $\displaystyle\leq c\sum_{j\neq 0}|xj|^{-(1+\alpha)}\log|xj|$
$\displaystyle\leq c|x|^{-(1+\alpha)}\log|x|,$

while the denominator has the lower bound $f(x;\theta)\geq c|x|^{-(1+\alpha)}$
on $|x|>M$. Thus,

(B.10) $\displaystyle\sup_{\theta\in C}R_{3}(x;\theta)\leq c{\bf
1}_{\\{|x|\leq M\\}}+c^{\prime}\log|x|{\bf 1}_{\\{|x|>M\\}}.$

This together with the bound for $|S_{3}(x;\theta)|$ yields

(B.11) $\displaystyle\sup_{\theta\in C}|S_{\tau,3}(x;\theta)|\leq
c+c^{\prime}\log|x|$

on $I_{\tau}$. Since $S_{\tau,3}$ is periodic, the bound holds for all
$x\in{\mathbb{R}}$. Thus we obtain

$\int_{-\infty}^{\infty}\sup_{\theta\in
C}|S_{\tau,3}(x;\theta)|f(x;\theta_{0})dx\leq
c+c^{\prime}\int_{|x|>M}|x|^{-(1+\alpha_{0})}\log|x|dx<\infty.$

Finally we prove $\widetilde{I}(\theta)\to I(\theta)$. Since the proofs are
similar, we only show the convergence for the information of $\alpha$, that
is, $\widetilde{I}_{\alpha}(\theta)\to I_{\alpha}(\theta)$. Since
$\widetilde{S}_{3}(x;\theta)\to S_{3}(x;\theta)$ pointwise in $x$, in view of

$\widetilde{I}_{\alpha}(\theta)=\int\widetilde{S}_{3}(x;\theta)^{2}f(x;\theta)dx,$

it suffices to show that $\widetilde{S}_{3}(x;\theta)^{2}$ has an integrable
dominant function w.r.t. the measure $f(x;\theta)dx$. Observe that

$\displaystyle|\widetilde{S}_{3}(x;\theta)|^{2}$ $\displaystyle\leq
2\big{(}S_{\tau,3}(x;\theta)^{2}+|\widetilde{S}_{3}(x;\theta)-S_{\tau,3}(x;\theta)|^{2}\big{)},$

where the second term has the bound (B.4). For the first term, we notice that
(B.11) holds for all sufficiently small $\tau>0$ and the bound does not depend
on $\tau>0$ (see the derivation process also). Thus, there exists a
$\tau_{0}>0$ such that for all $\tau\leq\tau_{0}$, if
$k^{-1}=o(\tau^{2+\alpha_{0}+\delta})$,
$\sup_{\theta\in\bar{B}_{\delta}(\theta_{0})}|\widetilde{S}_{3}(x;\theta)|^{2}\leq(c+c^{\prime}\log|x|)^{2}$
holds. Hence $\widetilde{S}_{3}(x;\theta)^{2}$ has a dominant integrable
function. ∎

###  B.2. Proof of Theorem 3.6

We first show $\sqrt{n}$-consistency of $\hat{\theta}_{n}$ by using a similar
argument with the proof of [47, Theorem 5.21]. Here we use the fact that
$\widetilde{S}(x;\theta)$ can be arbitrarily close to the score function
$S(x;\theta)$ of the ML estimator, and establish the result by using the
properties of $S(x;\theta)$. The condition on $k$ and $\tau$ assures us that
$\widetilde{S}$ and $S$ are close enough for our purpose. To obtain the
asymptotic distribution of $\hat{\theta}_{n}$, we show that
$n^{1/2}(\hat{\theta}_{n}-\tilde{\theta}_{n})\stackrel{{\scriptstyle
p}}{{\to}}0$, where $\tilde{\theta}_{n}$ is the ML estimator. This result
implies that the TML and ML estimators are first-order equivalent.

Before the proof we need the following lemma which gives a moment bound for
the stochastic distance of $\widetilde{S}_{n}$ and $S_{n}$. The bound is
proved to be a function of $k$ and $\tau$. The proof is rather technical, and
we fully exploit the two-step approximation scheme of Section 3.2.

######  Lemma B.6.

Let
$B_{\varepsilon}(\theta_{0})=\\{\theta:|\theta-\theta_{0}|<\varepsilon\\}$.
Then for arbitrary small $\varepsilon>0$,

$\displaystyle{\mathbb{E}}_{\theta_{0}}\big{[}\sup_{\theta\in
B_{\varepsilon}(\theta_{0})}|\widetilde{S}(X;\theta)-S(X;\theta)|\big{]}\leq
c\tau^{\alpha_{0}}\log 1/\tau+c(\tau^{2+\alpha_{0}+\varepsilon}k)^{-1/2}.$

###### Proof.

The left hand-side is bounded by

$\displaystyle\sum_{i=1}^{4}{\mathbb{E}}_{\theta_{0}}\big{[}\sup_{\theta\in
B_{\varepsilon}(\theta_{0})}|\widetilde{S}_{i}(X;\theta)-S_{i}(X;\theta)|\big{]}.$

We evaluate each element in the last sum and decompose them into the following
three integrals,

$\displaystyle\int_{-\infty}^{\infty}\sup_{\theta\in
B_{\varepsilon}(\theta_{0})}|\widetilde{S}_{i}(x;\theta)-S_{i}(x;\theta)|f(x;\theta_{0})dx$
$\displaystyle\leq\int_{-\infty}^{\infty}\sup_{\theta\in
B_{\varepsilon}(\theta_{0})}|\widetilde{S}_{i}(x;\theta)-S_{\tau,i}(x;\theta)|f(x;\theta_{0})dx$
$\displaystyle\quad+\Big{(}\int_{I_{\tau}}dx+\int_{I_{\tau}^{c}}dx\Big{)}\sup_{\theta\in
B_{\varepsilon}(\theta_{0})}|S_{\tau,i}(x;\theta)-S_{i}(x;\theta)|f(x;\theta_{0})$
$\displaystyle=:J_{1}+(J_{2}+J_{3}).$

For $J_{1}$ we observe that
$|\widetilde{S}_{i}(x;\theta)-S_{\tau,i}(x;\theta)|$ is periodic and has the
bound in (B.4). Thus taking $\sup_{\theta\in B_{\varepsilon}(\theta_{0})}$ in
(B.4), we obtain $J_{1}\leq c(\tau^{2+\alpha_{0}+\varepsilon}k)^{-1/2}$.

Turning to $J_{2}$, in view of (B.1) it suffices to evaluate

(B.12) $\displaystyle|S_{\tau,i}(x;\theta)-S_{i}(x;\theta)|$
$\displaystyle\leq\frac{\sum_{j\neq 0}|f_{\theta_{i}}(x+2\pi
j/\tau;\theta)|}{f_{\tau}(x;\theta)}+|S_{i}(x;\theta)|\cdot\frac{\sum_{j\neq
0}f(x+2\pi j/\tau;\theta)}{f_{\tau}(x;\theta)}$
$\displaystyle=:R_{i}(x;\theta)+R^{\prime}_{i}(x;\theta).$

Although $R_{i}(x;\theta)$ is treated in the proof of Theorem 3.4, we use
another bound here. Recall that $x^{-(1+\alpha)}\log x,\,(x>e)$ is decreasing.
Since $\pi(2|j|-1)/\tau\leq|x+2\pi j/\tau|\leq\pi(2|j|+1)/\tau,\,j\neq 0$, the
numerator has a bound

$\displaystyle c\sum_{j\neq 0}|x+2\pi j/\tau|^{-(1+\alpha)}\log|x+2\pi
j/\tau|$ $\displaystyle\leq c\tau^{1+\alpha}\sum_{j\neq
0}|\pi(2|j|-1)|^{-(1+\alpha)}\log|\pi(2|j|-1)/\tau|$ $\displaystyle\leq
c\tau^{1+\alpha}\log 1/\tau,$

for a sufficiently small $\tau>0$. In the meanwhile, the denominator has a
uniform lower bound on $|x|\leq M$, and on $M<|x|\leq\pi/\tau$ it satisfies

$f_{\tau}(x;\theta)\geq f(x;\theta)\geq c|x|^{-(1+\alpha)},$

where the constant $M$ is the one in the proof of Theorem 3.4. Thus, we have

$\displaystyle
R_{i}(x;\theta)\leq\left\\{\begin{array}[]{ll}c\tau^{1+\alpha}\log
1/\tau&\text{on}\ |x|\leq M\\\ c^{\prime}|\tau x/\pi|^{1+\alpha}\log
1/\tau&\text{on}\ M<|x|\leq\pi/\tau,\end{array}\right.$

and since $|\tau x/\pi|\leq 1$, we further obtain

(B.15) $\displaystyle\sup_{\theta\in
B_{\varepsilon}(\theta_{0})}R_{i}(x;\theta)\leq\left\\{\begin{array}[]{ll}c\tau^{1+\alpha_{0}-\varepsilon}\log
1/\tau&\text{on}\ |x|\leq M\\\ c^{\prime}|\tau
x/\pi|^{1+\alpha_{0}-\varepsilon}\log 1/\tau&\text{on}\
M<|x|\leq\pi/\tau.\end{array}\right.$

Now,

$\displaystyle\int_{I_{\tau}}\sup_{\theta\in
B_{\varepsilon}(\theta_{0})}R_{i}(x;\theta)f(x;\theta_{0})dx$
$\displaystyle\leq c\tau^{1+\alpha_{0}-\varepsilon}\log 1/\tau\int_{|x|\leq
M}f(x;\theta_{0})dx$
$\displaystyle\quad+c^{\prime}\tau^{1+\alpha_{0}-\varepsilon}\log
1/\tau\int_{M<|x|\leq\pi/\tau}|x/\pi|^{1+\alpha_{0}-\varepsilon}f(x;\theta_{0})dx$
$\displaystyle\leq c\tau^{1+\alpha_{0}-\varepsilon}\log
1/\tau\Big{(}1+\int_{M<|x|\leq\pi/\tau}|x/\pi|^{-\varepsilon}dx\Big{)}$
$\displaystyle\leq c\tau^{\alpha_{0}}\log 1/\tau.$

Concerning $R^{\prime}_{i}(x;\theta)$, we observe that on $I_{\tau}$,

(B.16) $\displaystyle\sup_{\theta\in
B_{\varepsilon}(\theta_{0})}|S_{i}(x;\theta)|\leq c\log|x|\leq c\log 1/\tau,$

while by similar calculations to that of $R_{i}$,

$\displaystyle\left|\frac{\sum_{j\neq 0}f(x+2\pi
j/\tau;\theta)}{f_{\tau}(x;\theta)}\right|\leq\left\\{\begin{array}[]{ll}c\tau^{1+\alpha}&\text{on}\
|x|\leq M\\\ c^{\prime}|\tau x/\pi|^{1+\alpha}&\text{on}\
M<|x|\leq\pi/\tau.\end{array}\right.$

Thus, except for constants, $R^{\prime}_{i}$ has the same uniform bound as
(B.15). Now collecting above results, we reach

$J_{2}\leq c\tau^{\alpha_{0}}\log 1/\tau.$

We again use the bound (B.12) for $J_{3}$. For $R_{i}$ of $J_{3}$ we recall
the argument in the proof of Theorem 3.4. Define

$|S_{\tau,i}|(x;\theta)=\sum_{j=-\infty}^{\infty}\frac{|f_{\theta_{i}}(x+2\pi
j/\tau;\theta)|}{f_{\tau}(x;\theta)}=\frac{|f_{\theta_{i}}(x;\theta)|}{f_{\tau}(x;\theta)}+R_{i}(x;\theta)$

and then

$R_{i}(x;\theta)\leq|S_{\tau,i}|(x;\theta)+|S_{i}(x;\theta)|.$

Notice that $|S_{\tau,i}|(x;\theta)$ is periodic and by exactly the same way
as that for $S_{\tau,\alpha}(x;\theta)$, $|S_{\tau,i}|(x;\theta)$ has the
bound $c\log|x|$ on $I_{\tau}^{c}$ (cf. (B.11)). Since
$S_{i}(x;\theta)=O(\log|x|)$ for large $|x|$, $\sup_{\theta\in
B_{\varepsilon}(\theta_{0})}R_{i}(x;\theta)\leq c\log|x|$. Due to (B.16)
together with $R^{\prime}_{i}\leq|S|$, we further obtain $\sup_{\theta\in
B_{\varepsilon}(\theta_{0})}R^{\prime}_{i}(x;\theta)\leq c^{\prime}\log|x|$.
Thus

$\displaystyle J_{3}$ $\displaystyle\leq\int_{I_{\tau}^{c}}\sup_{\theta\in
B_{\varepsilon}(\theta_{0})}\big{(}R_{i}(x;\theta)+R^{\prime}_{i}(x;\theta)\big{)}f(x;\theta_{0})dx$
$\displaystyle\leq c\int_{I_{\tau}^{c}}|x|^{-(1+\alpha_{0})}\log|x|dx$
$\displaystyle\leq c\tau^{\alpha_{0}}\log 1/\tau.$

Therefore, we obtain the result. ∎

Now we are ready to prove Theorem 3.6.

###### Proof of Theorem 3.6.

We start to see $\widetilde{S}_{n}(\hat{\theta}_{n})=o_{p}(n^{1/2})$. Let
$\tilde{\theta}_{n}$ be the ML estimator and let
$D_{n}:=\\{\omega:S_{n}(\tilde{\theta}_{n})=0\\}$. Then,
${\mathbb{P}}(D_{n})\to 1$ as $n\to\infty$. By the definition of
$\hat{\theta}_{n}$ and the consistency of both $\hat{\theta}_{n}$ and
$\tilde{\theta}_{n}$, we observe that

$\displaystyle{\mathbb{P}}\,\big{(}|n^{1/2}\widetilde{S}_{n}(\hat{\theta}_{n})|>\delta\big{)}\leq{\mathbb{P}}\big{(}|n^{1/2}\widetilde{S}_{n}(\tilde{\theta}_{n})|>\delta\big{)}$
$\displaystyle\leq{\mathbb{P}}\,\Big{(}\big{\\{}n^{1/2}|\widetilde{S}_{n}(\tilde{\theta}_{n})-S_{n}(\tilde{\theta}_{n})|>\delta\big{\\}}\cap
D_{n}\Big{)}+{\mathbb{P}}\,\big{(}D_{n}^{c}\big{)}$
$\displaystyle\leq{\mathbb{P}}\,\Big{(}\big{\\{}n^{1/2}|\widetilde{S}_{n}(\tilde{\theta}_{n})-S_{n}(\tilde{\theta}_{n})|>\delta\big{\\}}\cap\\{\tilde{\theta}_{n}\in
B_{\varepsilon}(\theta_{0})\\}\cap
D_{n}\Big{)}+{\mathbb{P}}\,\big{(}\tilde{\theta}_{n}\notin
B_{\varepsilon}(\theta_{0})\big{)}+{\mathbb{P}}\,\big{(}D_{n}^{c}\big{)}$
$\displaystyle\leq\delta^{-1}{\mathbb{E}}_{\theta_{0}}\,\big{[}n^{1/2}|\widetilde{S}_{n}(\tilde{\theta}_{n})-S_{n}(\tilde{\theta}_{n})|{\bf
1}_{\\{\tilde{\theta}_{n}\in B_{\varepsilon}(\theta_{0})\\}}\big{]}+o(1)$
$\displaystyle\leq\delta^{-1}n^{1/2}{\mathbb{E}}_{\theta_{0}}\big{[}\sup_{\theta\in
B_{\varepsilon}(\theta_{0})}|\widetilde{S}(X;\theta)-S(X;\theta)|\big{]}+o(1).$

Due to Lemma B.6 and the condition on $\tau,\,k$, the right-hand side
converges to $0$ as $n\to\infty$.

Next we prove

(B.17) $\displaystyle
n^{1/2}\big{(}\widetilde{S}_{n}(\hat{\theta}_{n})-\widetilde{S}(\hat{\theta}_{n})\big{)}-n^{1/2}\big{(}\widetilde{S}_{n}(\theta_{0})-\widetilde{S}(\theta_{0})\big{)}\stackrel{{\scriptstyle
p}}{{\to}}0,$

where $\widetilde{S}(\theta)$ is the function
$\theta\mapsto{\mathbb{E}}_{\theta_{0}}[\widetilde{S}(X;\theta)]$ (This
corresponds to [47, (5.22)]). Write the left-hand side as

$\displaystyle\underbrace{n^{1/2}\big{\\{}\widetilde{S}_{n}(\hat{\theta}_{n})-S_{n}(\hat{\theta}_{n})-\big{(}\widetilde{S}(\hat{\theta}_{n})-S(\hat{\theta}_{n})\big{)}\big{\\}}}_{I_{1}}+\underbrace{n^{1/2}\big{\\{}S_{n}(\hat{\theta}_{n})-S(\hat{\theta}_{n})-\big{(}S_{n}(\theta_{0})-S(\theta_{0})\big{)}\big{\\}}}_{I_{2}}$
$\displaystyle-\underbrace{n^{1/2}\big{\\{}\widetilde{S}_{n}(\theta_{0})-S_{n}(\theta_{0})-\big{(}\widetilde{S}(\theta_{0})-S(\theta_{0})\big{)}\big{\\}}}_{I_{3}}=:I_{1}+I_{2}-I_{3},$

where $S(\theta)$ is a function
$\theta\mapsto{\mathbb{E}}_{\theta_{0}}[S(X;\theta)]$. We study
$n^{1/2}\big{(}\widetilde{S}_{n}(\hat{\theta}_{n})-S_{n}(\hat{\theta}_{n})\big{)}$
in $I_{1}$ first. Observe that for any $\delta>0$,

$\displaystyle{\mathbb{P}}\,\big{(}n^{1/2}|\widetilde{S}_{n}(\hat{\theta}_{n})-S_{n}(\hat{\theta}_{n})|>\delta\big{)}$
$\displaystyle={\mathbb{P}}\,\big{(}\\{n^{1/2}|\widetilde{S}_{n}(\hat{\theta}_{n})-S_{n}(\hat{\theta}_{n})|>\delta\\}\cap\\{\hat{\theta}_{n}\in
B_{\varepsilon}(\theta_{0})\\}\big{)}+{\mathbb{P}}\,\big{(}\hat{\theta}_{n}\notin
B_{\varepsilon}(\theta_{0})\big{)}$
$\displaystyle\leq\delta^{-1}{\mathbb{E}}_{\theta_{0}}\big{[}n^{1/2}|\widetilde{S}_{n}(\hat{\theta}_{n})-S_{n}(\hat{\theta}_{n})|{\bf
1}_{\\{\hat{\theta}_{n}\in B_{\varepsilon}(\theta_{0})\\}}\big{]}+o(1)\to 0$

as $n\to\infty$, where we borrow the logic of the previous proof in the last
step. In a similar manner, other quantities in $I_{1}$ and $I_{3}$ converge to
$0$ in probability. For the quantity $I_{2}$, we use the continuous
differentiability of the score of MLE: $\theta\mapsto S(x;\theta)$ for every
$x$. In view of Lemma 2.1, in [28] each element of the matrix
$S_{\theta}(x;\theta)=\partial S(x;\theta)/\partial\theta$ is continuous in
$x$ and $\|S_{\theta}(x;\theta)\|=O((\log|x|)^{2})$ in the tail for all
$\theta\in C$, where $\|\cdot\|$ a matrix norm. Hence
${\mathbb{E}}\big{[}\sup_{\theta\in
C}\|S_{\theta}(X;\theta)\|^{2}\big{]}<\infty$, and the Lipschitz condition for
$S(x;\theta)$ given in [47, Theorem 5.21] is satisfied, where $\dot{\psi}(x)$
there is replaced with $\sup_{\theta\in C}\|S_{\theta}(x;\theta)\|$ here. Then
due to [47, Example 19.7], the functions $S(x;\theta)$ form a Donsker class
(see also the proof of [47, Theorem 5.21]) and by [47, Lemma 19.24],
$I_{2}\stackrel{{\scriptstyle p}}{{\to}}0$.

Recall that $\widetilde{S}_{n}(\hat{\theta}_{n})=o_{p}(n^{1/2})$ and
$\widetilde{S}(\theta_{0})=0$, and we may write

$\displaystyle
n^{1/2}\big{(}\widetilde{S}_{n}(\hat{\theta}_{n})-\widetilde{S}(\hat{\theta}_{n})\big{)}$
$\displaystyle=n^{1/2}\big{(}\widetilde{S}(\theta_{0})-\widetilde{S}(\hat{\theta}_{n})\big{)}+o_{p}(1)$
$\displaystyle=n^{1/2}\big{(}\widetilde{S}(\theta_{0})-S(\theta_{0})\big{)}+n^{1/2}\big{(}S(\theta_{0})-S(\hat{\theta}_{n})\big{)}-n^{1/2}\big{(}\widetilde{S}(\hat{\theta}_{n})-S(\hat{\theta}_{n})\big{)}+o_{p}(1)$
$\displaystyle=n^{1/2}\big{(}S(\theta_{0})-S(\hat{\theta}_{n})\big{)}+o_{p}(1),$

where we again use Lemma B.6 in the last step.

Now applying the delta method to the last quantity, we find from (B.17) that

$n^{1/2}I(\theta_{0})(\hat{\theta}_{n}-\theta_{0})+n^{1/2}o_{p}(|\hat{\theta}_{n}-\theta_{0}|)=n^{1/2}\big{(}\widetilde{S}_{n}(\theta_{0})-\widetilde{S}(\theta_{0})\big{)}+o_{p}(1),$

where $I(\theta_{0})$ is the Fisher information, that is, the derivative of
$\theta\mapsto-{\mathbb{E}}_{\theta_{0}}[S(X;\theta)]$ at $\theta_{0}$ (Here
$I(\theta_{0})$ corresponds to $-V_{\theta_{0}}$ in [47, Theorem 5.21]). Since
$I(\theta_{0})$ is invertible (see the beginning of the proof for Theorem
3.4), we have

$n^{1/2}|\hat{\theta}_{n}-\theta_{0}|\leq\|I(\theta_{0})^{-1}\|n^{1/2}|I(\theta_{0})(\hat{\theta}_{n}-\theta_{0})|=O_{p}(1)+o_{p}(n^{1/2}|\hat{\theta}_{n}-\theta_{0}|).$

This implies $\hat{\theta}_{n}$ is $n^{1/2}$-consistent, so that
$n^{1/2}o_{p}(|\hat{\theta}_{n}-\theta_{0}|)=o_{p}(1)$. Thus we obtain

$n^{1/2}I(\theta_{0})(\hat{\theta}_{n}-\theta_{0})=n^{1/2}\widetilde{S}_{n}(\theta_{0})+o_{p}(1).$

Finally, by the asymptotic linear representation of the ML estimator [28,
Theorem 3.1], we have

$\displaystyle n^{1/2}(\hat{\theta}_{n}-\tilde{\theta}_{n})$
$\displaystyle=n^{1/2}(\hat{\theta}_{n}-\theta_{0})-n^{1/2}(\tilde{\theta}_{n}-\theta_{0})+o_{p}(1)$
$\displaystyle=I(\theta_{0})^{-1}n^{1/2}\big{(}\widetilde{S}_{n}(\theta_{0})-S_{n}(\theta_{0})\big{)}+o_{p}(1).$

Again by Lemma B.6 the left is $o_{p}(1)$. ∎

##  Appendix C Supplement for Section 5

Exact forms of derivative vectors of the conditional ch.f. $\varphi_{t\mid
s,\theta}(u;\theta)$ with $\theta=(\lambda,\alpha,\sigma)$ are given. For
convenience we define $\psi_{t\mid s}(u;\theta)=\log\varphi_{t\mid
s}(u;\theta)$ and present expressions for $\psi_{t\mid
s,\theta_{i}}(u;\theta)=\partial\psi_{t\mid
s}(u;\theta)/\partial\theta_{i},i=1,2,3$, which are

$\displaystyle\psi_{t\mid s,\lambda}(u;\theta)$
$\displaystyle=-iu(t-s)X_{s}e^{-\lambda(t-s)}+\frac{|\sigma
u|^{\alpha}}{\lambda\alpha}\big{(}\lambda^{-1}(1-e^{-\alpha\lambda(t-s)})-\alpha(t-s)e^{-\alpha\lambda(t-s)}\big{)},$
$\displaystyle\psi_{t\mid s,\sigma}(u;\theta)$
$\displaystyle=-\sigma^{\alpha-1}|u|^{\alpha}\lambda^{-1}(1-e^{-\alpha\lambda(t-s)}),$
$\displaystyle\psi_{t\mid s,\alpha}(u;\theta)$ $\displaystyle=\frac{|\sigma
u|^{\alpha}}{\lambda\alpha}\big{\\{}(-\log|\sigma
u|+\alpha^{-1})(1-e^{-\lambda\alpha(t-s)})-\lambda(t-s)e^{-\lambda\alpha(t-s)}\big{\\}}.$

## References

  * [1] Brant, R. (1984) Approximate likelihood and probability calculations based on transforms. The Annals of Statistics 12, 989–1005.
  * [2] Carrasco, M., Chernov, M., Florens, J.P. and Ghysels, E. (2007). Efficient estimation of general dynamic models with a continuum of moment conditions. Journal of Econometrics 140, 529–573.
  * [3] Carrasco, M. and Florens, J.P. (2000) Generalization of GMM to a continuum of moment conditions. Econometric Theory 16, 797–834.
  * [4] Carrasco, M. and Florens, J.P. (2002) Efficient GMM estimation using the empirical characteristic function. Working paper; Department of Econometrics: University of Rochester.
  * [5] Carrasco, M. and Kotchoni, R. (2017) Efficient estimation using the characteristic function. Econometric Theory 33, 479–526.
  * [6] Chacko, G. and Viceira, L.M. (2003) Spectral GMM estimation of continuous-time processes. Journal of Econometrics 116, 259–292.
  * [7] Christensen, O. and Christensen, K.L. (2006) Linear independence and series expansions in function spaces. The American Mathematical Monthly 113, 611–627.
  * [8] Clément, E. and Gloter, A. (2019) Estimating functions for SDE driven by stable Lévy processes. Annales de l’Institut Henri Poincaré, Probabilités et Statistiques 55, (2019), 1316–1348.
  * [9] Clément, E. and Gloter, A. (2020) Joint estimation for SDE driven by locally stable Lévy processes. Electronic Journal of Statistics 14, 2922–2956.
  * [10] Donald S.G., Imbens, G.W. and Newey, W.K. (2003) Empirical likelihood estimation and consistent tests with conditional moment restrictions, Journal of Econometrics 117, 55–93.
  * [11] DuMouchel, W.H. (1973) On the asymptotic normality of the maximum-likelihood estimate when sampling from a stable distribution. The Annals of Statistics 1, 948–957.
  * [12] DuMouchel, W.H. (1975) Stable distributions in statistical inference: 2. Information from stably distributed samples. Journal of the American Statistical Association 70, 386–393.
  * [13] Feuerverger, A. and McDunnough, P. (1981) On the efficiency of empirical characteristic function procedures. Journal of the Royal Statistical Society. Series B (Methodological) 43, 20–27.
  * [14] Feuerverger, A. and McDunnough, P. (1981) On some Fourier methods for inference. Journal of the American Statistical Association 76, 379–387.
  * [15] Hansen, L.P. (1982) Large sample properties of generalized method of moments estimators. Econometrica 50, 1029–1054.
  * [16] Hansen, L.P., Heaton, J. and Yaron, A. (1996) Finite-sample properties of some alternative GMM estimators. Journal of Business & Eonomic Statistics 14, 262–280.
  * [17] Heathcote, C.R. (1977) The integrated squared error estimation of parameters. Biometrika 64, 255–264.
  * [18] Heyde, C. C. (1997) Quasi-Likelihood And Its Application: A General Approach to Optimal Parameter Estimation, Springer, New York.
  * [19] Imbens, G. W., Spady, R. H. and Johnson, P. (1997) Information theoretic approaches to inference in moment condition models Econometrica 66. 333–357.
  * [20] Jackson, D. (1930) Theory of Approximation. A.M.S. Colloq. Pub. 11.
  * [21] Kitamura, Y. and Stuzer, M. An information-theoretic alternative to generalized method of moments estimation Econometrica 65, 861–874.
  * [22] Kogon, S.M. and Williams, D.B. (1998). Characteristic function based estimation of stable distribution parameters. In: A Practical Guide to Heavy Tails: Statistical Techniques and Applications, Birkhäuser, Boston, 311–338.
  * [23] Kolmogorov A.N. and Fomin S.V. (2012) Elements of the Theory of Functions and Functional Analysis [Two Volumes in One], Martino Publishing, Eastford.
  * [24] Koutrouvelis, I.A. (1980) Regression-type estimation of the parameters of stable laws. Journal of the American Statistical Association 75, 918–928.
  * [25] Koutrouvelis, I.A. (1981) An iterative procedure for the estimation of the parameters of stable laws. Communications in Statistics-Simulation and Computation 10, 17–28.
  * [26] Kunitomo, N. and Owada, T. (2006). Empirical likelihood estimation of Lévy processes. Graduate School of Economics, University of Tokyo Discussion Paper.
  * [27] Masuda, H. (2019). Non-Gaussian quasi-likelihood estimation of SDE driven by locally stable Lévy process. Stochastic Processes and their Applications 129, 1013–1059.
  * [28] Matsui, M. (2020) Asymptotics of maximum likelihood estimation for stable law with continuous parameterization. Communications in Statistics - Theory and Methods (online)
  * [29] Matsui, M. (2005) Fisher information matrix of general stable distributions close to the normal distribution. Mathematical Methods of Statistics 14, 224–251.
  * [30] Matsui, M. and Takemura, A. (2006) Some improvements in numerical evaluation of symmetric stable density and its derivatives. Communications in Statistics - Theory and Methods 35, 149–172.
  * [31] Matsui, M. and Takemura, A. (2008) Goodness-of-fit tests for symmetric stable distributions – empirical characteristic function approach. Test 17, 546–566.
  * [32] McCulloch, J.H. (1986) Simple consistent estimators of stable distribution parameters. Communications in Statistics - Simulation andComputation 15, 1109–1136.
  * [33] Mittnik, S., Doganoglu, T., and Chenyao, D. (1999). Maximum likelihood estimation of stable Paretian models. Mathematical and Computer Modelling 29, 275–293.
  * [34] Nagaev, A.V. and Shkol’nik, S.M. (1988) Some properties of symmetric stable distributions close to the normal distribution. Theory of Probability & Its Applications 33, 139–144.
  * [35] Newey, W. K. and Smith, R. J. Higher order properties of GMM and generalized empirical likelihood estimators. Econometrica 72, 219–255.
  * [36] Nolan, J.P. (1998) Parameterizations and modes of stable distributions. Statistics & Probability Letters 38, 187–195.
  * [37] Nolan, J.P. (2001) Maximum likelihood estimation and diagnostics for stable distributions. In: Lévy Processes: Theory and Applications (O. E. Barndorff-Nielsen et al. eds.), Birkhäuser, Boston, 379–400.
  * [38] Nolan, J. P. (2020). Univariate Stable Distributions: Models for Heavy Tailed Data. Springer, Cham.
  * [39] Qin, J. and Lawless, J. (1994) Empirical Likelihood and General Estimating Equations. The Annals of Statistics 22, 300–325.
  * [40] Paulson, A.S., Holcomb, E.W. and Leitch, R.A. (1975) The estimation of the parameters of the stable laws. Biometrika 62, 163–170.
  * [41] Press, S.J. (1972) Estimation in univariate and multivariate stable distributions. Journal of the American Statistical Association 67, 842–846.
  * [42] Royuela-del-Val, J., Simmross-Wattenberg, F., and Alberola-López, C. (2017). libstable: Fast, Parallel, and High-Precision Computation of $\alpha$-Stable Distributions in R, C/C++, and MATLAB. Journal of Statistical Software 78, 1–23
  * [43] Samorodnitsky, G. and Taqqu, M.S. (1994) Stable Non-Gaussian Random Processes. Stochastic Models with Infinite Variance. Chapman and Hall, London.
  * [44] Sato, K. (1999) Lévy Processes and Infinitely Divisible Distributions. Cambridge University Press, Cambridge.
  * [45] Singleton, K.J. (2001) Estimation of affine asset pricing models using the empirical characteristic function. Journal of Econometrics 102, 111–141.
  * [46] Sueishi, N. and Nishiyama, Y. (2005) Estimation of Levy Processes in Mathematical Finance: A Comparative Study. In: MODSIM 2005 International Congress on Modelling and Simulation. (Zerger, A. and Argent, R.M. eds.) Modelling and Simulation Society of Australia and New Zealand, December 2005, 953–959.
  * [47] van der Vaart, A.W. (2000) Asymptotic Statistics (Cambridge Series in Statistical and Probabilistic Mathematics). Cambridge University Press, Cambridge.
  * [48] Yu, J. (2004) Empirical characteristic function estimation and its applications. Econometric Reviews 23, 93–123.
  * [49] Zolotarev, V.M.  (1986) One-Dimensional Stable Distributions. AMS Translation of Mathematics Monographs, 65, American Mathematics Society, Providence. (Transl. of the original 1983 Russian)