# A Ridge-Regularised Jackknifed Anderson-Rubin Test††thanks: This research is
funded by the European Research Council via Consolidator grant number 647152
and the German National Merit Foundation. We thank Frank Kleibergen, Damian
Kozbur, Anna Mikusheva, and Bent Nielsen for useful comments and discussions.

Max-Sebastian Dovì  Department of Economics, University of Oxford, Manor Road,
OX1 3UQ, Oxford, UK  Anders Bredahl Kock  Department of Economics, University
of Oxford, Manor Road, OX1 3UQ, Oxford, UK  Department of Economics and
CREATES, Aarhus University, Denmark  Sophocles Mavroeidis  Department of
Economics, University of Oxford, Manor Road, OX1 3UQ, Oxford, UK

###### Abstract

We consider hypothesis testing in instrumental variable regression models with
few included exogenous covariates but many instruments – possibly more than
the number of observations. We show that a ridge-regularised version of the
jackknifed [4] test controls asymptotic size in the presence of
heteroskedasticity, and when the instruments may be arbitrarily weak.
Asymptotic size control is established under weaker assumptions than those
imposed for recently proposed jackknifed AR tests in the literature.
Furthermore, ridge-regularisation extends the scope of jackknifed AR tests to
situations in which there are more instruments than observations. Monte-Carlo
simulations indicate that our method has favourable finite-sample size and
power properties compared to recently proposed alternative approaches in the
literature. An empirical application on the elasticity of substitution between
immigrants and natives in the US illustrates the usefulness of the proposed
method for practitioners.

Keywords: instrumental variables, weak identification, high dimensional
models, ridge regression.

JEL codes: C12, C36, C55.

##  1 Introduction

Instrumental variables (IVs) are commonly employed in economics and related
fields to estimate causal effects from observational data. Interest in
inference with (very) many IVs that are potentially weak has recently received
increased attention for at least two reasons. First, when identification is
weak, researchers may attempt to obtain more precise inference by using a
large number of IVs to capture the limited exogenous variation in the
endogenous covariates. Second, recent econometric approaches and theoretical
results make a number of IVs that approaches or even exceeds the number of
observations more common. For instance, the so-called granular IV approach
proposed by [15] leads to a number of IVs equal to the size of the cross-
section of the panel considered. This number can approach, and potentially
outstrip, the number of available observations. Another example is the so-
called saturation approach to the identification of (weighted) local average
treatment effects as in [9], which involves considering as IVs a sufficient
number of interactions between the original IVs and the covariates. The number
of IVs generated in this way can easily outstrip the number of observations
available. However, there is no guarantee that a vast number of IVs will be
jointly informative about the causal effects of interest.111Pretesting the
strength of IVs, though practical, could be rather inefficient as a way of
controlling the size of subsequent inference on causal effects. Moreover,
weak-IV pretests are not available when the number of IVs exceeds the number
of observations. Hence, it is important to have methods of inference that
remain reliable when using (very) many weak IVs. The present paper contributes
to the literature on the development of such methods.

We propose a ridge-regularised jackknifed Anderson-Rubin (RJAR) test to
construct confidence sets for the coefficients of endogenous variables in
weakly-identified and heteroskedastic IV models when the number of IVs is
large. Jackknife-based methods have recently been used in this context because
they are applicable in an asymptotic framework where the number of IVs
diverges with the number of observations. However, by relying on existing
central limit theorems developed for standard projection matrices, these
methods require that the number of IVs be less than the number of
observations, and often perform poorly when the number of IVs is close to (but
still less than) the number of observations. Recently-proposed regularisation
approaches for inference under many IVs require strong identification or a
sparse relationship between the endogenous variables and the IVs to work well.
By combining jackknifing with ridge regularisation, we provide a test that
remains valid under heteroskedasticity, arbitrarily weak identification, and
more IVs than observations, all while achieving good power both when the
relationship between the endogenous variables and the IVs is sparse and when
it is dense.

In the context of fully weak-identification robust inference with an
increasing number of IVs, it is helpful to distinguish three different
asymptotic regimes. The first ‘moderately many’ IVs regime allows the number
of IVs, $k_{n}$, to grow with the sample size, $n$, but still requires it to
be asymptotically negligible with respect to the sample size. Examples include
[5], who require $k_{n}/n^{1/3}\to 0$, and [23], who require $k_{n}/n\to 0$.

The second ‘many’ IVs regime allows the number of IVs to be of the same order
of magnitude as the number of observations. [2] provide an [4, henceforth AR ]
test that remains valid for $k_{n}/n\to\lambda$, $0\leq\lambda<1$,
$n\to\infty$, provided that the error terms are homoskedastic and a strong
balanced-design assumption of the IVs is satisfied.222See [14, 3, 1] for a
discussion of (the restrictiveness of) the strong balanced-design
assumption.,333See [19] for a bootstrapped version of the [2] AR test. [10]
provide analogous results for the GMM version of the AR statistic under the
assumption of independently and identically distributed (i.i.d.) data. These
results were recently extended in the form of a jackknifed AR statistic by
[14, henceforth CMS ] and [22, henceforth MS ] to the case where errors are
allowed to display arbitrary heteroskedasticity, and the only assumption on
the IVs is that the diagonal entries of the projection matrix of the IVs are
bounded away from unity from above.

The third ‘very many’ IVs regime allows the number of IVs to grow with $n$,
and further allows the number of IVs to be greater than the number of
observations. [8, henceforth BCCH ] propose a Sup Score test that remains
valid under mild conditions, and allows the number of IVs to increase
exponentially with the sample size. [12] propose a ridge-regularised AR
statistic that allows for more IVs than observations under the assumption of
i.i.d. data. [20] extend [6] by proposing weak-identification robust factor-
based statistics that in principle allow for the number of IVs to be larger
than the number of observations, provided that the factor structure of the IVs
is sufficiently strong.

Using the notation of the model introduced in the next section, Table 1
provides a schematic overview of the main assumptions and results in the
literature on inference that is robust to many weak IVs.

Our test provides a twofold extension of the existing literature. First, it
allows for valid inference under many IVs while further weakening the
assumptions of similar tests proposed by CMS and MS. This is made possible by
deriving the limiting behaviour of the RJAR statistic from the bottom up,
without relying on the existing results in [13] or [18]. Second, this test
allows for more IVs than observations. The only other approach currently
available in the literature that is robust to heteroskedastic error terms and
more IVs than observations is the Sup Score test of BCCH. Simulations show
that the RJAR test has power comparable to the Sup Score test of BCCH whenever
the signal in the first stage is sparse (i.e., only a few of the IVs are
informative), and substantially more power when the signal in the first stage
is dense (i.e., not sparse). Finally, we provide a comparison of the most
recently proposed approaches to conducting inference in possibly
heteroskedastic linear IV models when the number of IVs is not negligible with
respect to the sample size. Indeed, using extensive simulation evidence and an
empirical application based on [11], we provide a comparison between these
existing ‘state-of-the-art’ methods in a controlled and comparable setting.

Notation. $I_{p}$ denotes the $p\times p$ identity matrix. The entry $(i,j)$
for an $m\times p$ matrix $A$ is denoted as $A_{ij}$ for $1\leq i\leq m$,
$1\leq j\leq p$. $||A||_{F}\coloneqq\sqrt{\text{tr}(A^{\prime}A)}$ denotes the
Frobenius norm of $A$, and $\text{tr}(B)\mathrel{\mathop{\mathchar
58\relax}}=\sum_{i=1}^{m}B_{ii}$ for any $m\times m$ matrix $B$ with entries
given by $B_{ij}$ for $1\leq i,j\leq m$. The remaining notation follows
standard conventions.

Organisation of the paper. Section 2 specifies the linear IV model considered
throughout. Section 3 introduces the RJAR test, and provides the main
asymptotic results. Section 4 provides simulation evidence on the size and
power of our RJAR test and compares it with the tests proposed by BCCH, CMS
and MS. Section 5 considers an empirical application based on [11]. Section 6
concludes. All proofs are given in the Appendix.

Table 1: Weak-identification robust inference with many IVs: schematic comparison of main assumptions and results in the literature. |  $k_{n}$ |  $Z$ |  $\varepsilon$
---|---|---|---
[19, 2] |  $k_{n}/n\to\lambda$, $0\leq\lambda<1$, $n\to\infty$ |  $Z$ is fixed, obeys strong balanced design assumption |  i.i.d., $\mathbb{E}[\varepsilon_{i}]=0$, $\mathbb{E}[\varepsilon_{i}^{4}]<\infty$
|  |  |
[8] |  $\log(\text{max}(k_{n},n))=o(n^{1/3})$, $n\to\infty$ |  $Z$ is fixed |  independent, $\mathbb{E}[\varepsilon_{i}]=0$, $\mathbb{E}[\varepsilon_{i}^{3}]<\infty$
|  |  |
[20] |  Factor-dependent |  Potentially dependent data, $\mathbb{E}[Z_{il}^{4}]<\infty$, $l=1,\dots,k_{n}$, $\mathbb{E}[\varepsilon_{i}^{4}]<\infty$
|  |  |
[12] |  $n\to\infty$ |  $Z_{i}\varepsilon_{i}$ is i.i.d., $\mathbb{E}[Z_{i}\varepsilon_{i}]=0$, $\mathbb{E}[X_{ig}^{2}]<\infty$
|  |  |
[10] |  $k_{n}/n\to\lambda$, $0\leq\lambda<1$, $n\to\infty$ |  $Z_{i}\varepsilon_{i}$ is i.i.d., $Z$ obeys strong balanced design assumption, $\mathbb{E}[Z_{i}\varepsilon_{i}]=0$, $\mathbb{E}[Z_{i}^{8}\varepsilon_{i}^{8}]<\infty$
|  |  |
[14] |  $k_{n}/n\to\lambda$, $0\leq\lambda<1$, $k_{n}\to\infty$ as $n\to\infty$ |  $Z$ is fixed, $P_{ii}\leq 1-\delta$, $0<\delta<1$ |  independent, $\mathbb{E}[\varepsilon_{i}]=0$, $\mathbb{E}[\varepsilon_{i}^{4}]<\infty$
|  |  |
[22] |  $k_{n}/n\to\lambda$, $0\leq\lambda<1$, $k_{n}\to\infty$ as $n\to\infty$ |  $Z$ is fixed, $P_{ii}\leq 1-\delta$, $0<\delta<1$ |  independent, $\mathbb{E}[\varepsilon_{i}]=0$, $\mathbb{E}[\varepsilon_{i}^{6}]<\infty$
RJAR |  $r_{n}\to\infty$ as $n\to\infty$ |  $Z$ is fixed, $\sum_{i=1}^{n}\sum_{j\neq i}(P^{\gamma_{n}}_{ij})^{2}\geq cr_{n}^{\omega}$, $c>0$, $\omega\geq 1$ |  independent, $\mathbb{E}[\varepsilon_{i}]=0$, $\mathbb{E}[\varepsilon_{i}^{4}]<\infty$
_Notes_ :
$n$ is the number of observations, $k_{n}$ is the number of IVs, and $g$ is
the number of endogenous variables. $Z$ is the $n\times k_{n}$ matrix of IVs.
$X$ is the $n\times g$ matrix of endogenous variables. $\varepsilon$ is the
$n\times 1$ vector of structural error terms. $r_{n}\mathrel{\mathop{\mathchar
58\relax}}=\text{rank}(Z)$. See also Equation (1) below.
$P\mathrel{\mathop{\mathchar 58\relax}}=Z(Z^{\prime}Z)^{-1}Z^{\prime}$,
$P^{\gamma_{n}}\mathrel{\mathop{\mathchar
58\relax}}=Z(Z^{\prime}Z+\gamma_{n}I_{k_{n}})^{-1}Z^{\prime}$ for
$\gamma_{n}\geq 0$ if $r_{n}=k_{n}$ and $\gamma_{n}\geq\gamma_{-}>0$ if
$r_{n}<k_{n}$.

##  2 Model

We consider the heteroskedastic linear IV model given by

$\displaystyle y$ $\displaystyle=X\beta+\varepsilon$ (1a) $\displaystyle X$
$\displaystyle=Z\Pi+V,$ (1b)

where $y$ is an $n\times 1$ vector containing the dependent variable, $X$ is
an $n\times g$ matrix containing the endogenous variables, $\beta$ is a
$g\times 1$ coefficient vector, $\varepsilon$ is an $n\times 1$ vector
containing the structural error terms, $Z$ is an $n\times k_{n}$ matrix
containing the IVs, $\Pi$ is a $k_{n}\times g$ coefficient matrix, and $V$ is
an $n\times g$ matrix containing the first-stage errors. $k_{n}$ can diverge
with $n$, but $g$ is fixed. Also, let $y_{i}$, $X_{i}$, $\varepsilon_{i}$,
$Z_{i}$, and $V_{i}$ denote the $i^{\text{th}}$ row of $y,X,\varepsilon,Z$,
and $V$, respectively. As in BCCH, CMS and MS, we treat $Z$ as fixed (non-
stochastic), and assume that any exogenous covariates have been partialled out
(see the discussion of Assumption 3 below). We _exclusively_ consider methods
that allow for arbitrarily weak identification, i.e., methods that control
asymptotic size irrespective of the value of $\Pi$.

Inference is conducted on the coefficient vector $\beta$ by testing hypotheses
of the following type for a prespecified $\beta_{0}\in\Re^{g}$:

$H_{0}\mathrel{\mathop{\mathchar 58\relax}}\beta=\beta_{0}\text{ vs.
}H_{1}\mathrel{\mathop{\mathchar 58\relax}}\beta\neq\beta_{0}.$ (2)

For a given non-randomised test of asymptotic size $\alpha\in(0,1)$, a
confidence set of asymptotic coverage $1-\alpha$ can be constructed as the
collection of those $\beta_{0}$ for which $H_{0}$ in Equation (2) is not
rejected. For convenience, define

$e(\beta_{0})\coloneqq\left[e_{1}(\beta_{0}),\ldots,e_{n}(\beta_{0})\right]^{\prime},\qquad
e_{i}(\beta_{0})\coloneqq y_{i}-X_{i}\beta_{0},\qquad i=1,\dots,n,$ (3)

which we refer to as the structural error under the null hypothesis.

It should be noted that so long as the error term remains additive, the
linearity in the structural equation (1a) and the first-stage equation (1b)
can be relaxed to allow for any (known) real-valued function, without
affecting the asymptotic properties of the RJAR test.

##  3 The RJAR test

This section introduces our proposed RJAR test, derives its large sample
properties under the null hypothesis in Equation (2), and discusses its
relationship to the most closely related tests in the literature: the
jackknifed AR tests of CMS and MS, and the Sup Score test of BCCH.

###  3.1 Definition of the RJAR test

The original AR test of the null hypothesis in Equation (2) can be thought of
as a test that the IVs are exogenous using the implied structural errors in
Equation (3). More specifically, the AR test is a Wald test of the
significance of the IVs in the auxiliary regression of the structural errors
under the null, $e_{i}(\beta_{0})$, on the IVs, $Z_{i}$.444The AR statistic is
defined as $e(\beta_{0})^{\prime}Z\widehat{\Sigma}Z^{\prime}e(\beta_{0})/n$,
where $\widehat{\Sigma}$ is a consistent estimator of
$\text{Var}[Z^{\prime}e(\beta_{0})/\sqrt{n}]$. The weak-IV-robust AR test of
the null hypothesis in Equation (2) with asymptotic size $\alpha$ rejects the
null if and only if the AR statistic exceeds the $1-\alpha$ quantile of a
$\chi^{2}$ distribution with $k_{n}$ degrees of freedom. When the number of
IVs $k_{n}$ grows with $n$, i.e., when there are many (potentially weak) IVs,
the $\chi^{2}$ approximation of the original AR statistic breaks down. The
recent papers by CMS and MS, which we shall briefly review in the next
subsection, propose jackknifed versions of the AR test that remain valid when
$k_{n}$ grows with $n$, but $k_{n}<n$. Our proposed method combines
jackknifing with ridge regularisation of the auxiliary regression of
$e_{i}(\beta_{0})$ on $Z_{i}$.

We first standardise the IVs in-sample (after partialling out any covariates)
as in BCCH (p. 2393) so that

$\frac{1}{n}\sum_{i=1}^{n}Z_{ij}^{2}=1\quad\text{for}\quad j=1,\dots,k_{n}.$
(4)

The RJAR test for the testing problem in Equation (2) is then based on the
following statistic:

$\displaystyle RJAR_{\gamma_{n}}(\beta_{0})$
$\displaystyle\coloneqq\frac{1}{\sqrt{r_{n}}\sqrt{\hat{\Phi}_{\gamma_{n}}(\beta_{0})}}\sum_{i=1}^{n}\sum_{j\neq
i}P_{ij}^{\gamma_{n}}e_{i}(\beta_{0})e_{j}(\beta_{0}),$ (5)

where $r_{n}\coloneqq\text{rank}(Z)$ (assumed to be positive without loss of
generality),

$\hat{\Phi}_{\gamma_{n}}(\beta_{0})\coloneqq\frac{2}{r_{n}}\sum_{i=1}^{n}\sum_{j\neq
i}(P_{ij}^{\gamma_{n}})^{2}e_{i}^{2}(\beta_{0})e_{j}^{2}(\beta_{0}),$ (6)

and $P^{\gamma_{n}}\mathrel{\mathop{\mathchar
58\relax}}=Z(Z^{\prime}Z+\gamma_{n}I_{k_{n}})^{-1}Z^{\prime}$ is the ridge-
regularised projection matrix for $\gamma_{n}\geq 0$ if $r_{n}=k_{n}$ and
$\gamma_{n}>0$ if $r_{n}<k_{n}$. $\gamma_{n}$ is a (sequence of)
regularisation parameter(s) whose choice we discuss next.

We set the penalty parameter $\gamma_{n}$ to

$\gamma^{*}_{n}\coloneqq\max\underset{\gamma_{n}\in\Gamma_{n}}{\arg\max}\sum_{i=1}^{n}\sum_{j\neq
i}(P^{\gamma_{n}}_{ij})^{2},$ (7)

where $\Gamma_{n}\coloneqq\\{\gamma_{n}\in\Re\mathrel{\mathop{\mathchar
58\relax}}\gamma_{n}\geq 0\text{ if }r_{n}=k_{n},\text{ and
}\gamma_{n}\geq\gamma_{-}>0\text{ if }r_{n}<k_{n}\\}$ for some constant
$\gamma_{-}$ not depending on $n$. The existence of $\gamma^{*}_{n}$ is shown
in Lemma 3 in the Appendix. We let $\gamma_{n}^{*}$ be an element of
$\underset{\gamma_{n}\in\Gamma_{n}}{\arg\max}\sum_{i=1}^{n}\sum_{j\neq
i}(P^{\gamma_{n}}_{ij})^{2}$ out of conservativeness, i.e., to make Assumption
3 below as plausible as possible given the IVs. Furthermore, we let
$\gamma^{*}_{n}$ be the maiximal element of this set because the maximiser of
the first argument is not necessarily unique without imposing additional
assumptions on the singular values and left-singular vectors of the IVs
(although in practice we only found unique maximisers). We choose to take the
maximum of the maximisers to make the smallest eigenvalues of the ridge-
regularised Gram matrix, $Z^{\prime}Z+\gamma_{n}I_{k_{n}}$, as far away from
zero as possible when $r_{n}<k_{n}$.

We now have all the ingredients to define our new test.

######  Definition 1.

The RJAR test rejects $H_{0}\mathrel{\mathop{\mathchar
58\relax}}\beta=\beta_{0}$ in Equation (2) at significance level
$\alpha\in(0,1)$ if and only if

$RJAR_{\gamma^{*}_{n}}(\beta_{0})>\mathcal{Q}(1-\alpha),$ (8)

where $RJAR_{\gamma_{n}}(\beta_{0})$ is defined in Equation (5),
$\gamma^{*}_{n}$ is defined in Equation (7), and $\mathcal{Q}(1-\alpha)$ is
the $(1-\alpha)$ quantile of the Standard Normal distribution.

###  3.2 Asymptotic properties of the RJAR test

We make the following assumptions to derive the limiting distribution of
$RJAR_{\gamma^{*}_{n}}(\beta_{0})$ under the null hypothesis in Equation (2).

######  Assumption 1.

$\\{\varepsilon_{i}\\}_{i\in\mathbb{N}}$ is a sequence of independent random
variables satisfying $\mathbb{E}[\varepsilon_{i}]=0$,
$\inf_{i\in\mathbb{N}}\text{Var}[\varepsilon_{i}]>0$, and
$\sup_{i\in\mathbb{N}}\mathbb{E}[\varepsilon_{i}^{4}]<\infty$.

######  Assumption 2.

$r_{n}=\text{rank}(Z)\to\infty$ as $n\to\infty$.

######  Assumption 3.

There exist $\gamma_{-}>0$, $c>0$, $\omega\geq 1$, $N\geq 2$ such that for all
$n\geq N$

  1. 1.

If $r_{n}=k_{n}$, then there exists a $\gamma_{n}\in[0,\infty)$ such that
$\sum_{i=1}^{n}\sum_{j\neq i}(P^{\gamma_{n}}_{ij})^{2}\geq cr_{n}^{\omega}$

  2. 2.

If $r_{n}<k_{n}$, then there exists a $\gamma_{n}\in[\gamma_{-},\infty)$ such
that $\sum_{i=1}^{n}\sum_{j\neq i}(P^{\gamma_{n}}_{ij})^{2}\geq
cr_{n}^{\omega}$

Assumption 1 is a mild condition on the structural error terms, and allows for
conditional heteroskedasticity. It is the same as in CMS. It is slightly less
restrictive than the one in MS (who require finite sixth moments on the
structural error terms), and slightly more restrictive than the one in BCCH
(who require finite third moments).

Assumption 2 is a weak technical assumption. It implies that both $k_{n}$ and
$n$ diverge. It also allows for the sum of the number of IVs and the number of
exogenous covariates to be larger than the number of observations, provided
that the number of exogenous covariates that have been partialled out be
sufficiently small (so that the rank of the matrix of partialled IVs continues
to diverge). This assumption is weaker than the restriction on the
dimensionality in MS and CMS, who require $r_{n}=k_{n}$, $k_{n}<n$ for each
$n\in\mathbb{N}$, and $k_{n}\to\infty$ as $n\to\infty$. BCCH prove asymptotic
size control of their Sup Score test under the assumption that $\log
k_{n}=o(n^{1/3})$.

Assumption 3 is a high-level assumption on the number of IVs and their
correlation structure. Assumption 3 implies that $\gamma^{*}_{n}$ satisfies
$\sum_{i=1}^{n}\sum_{j\neq i}(P^{\gamma_{n}^{*}}_{ij})^{2}\geq
cr_{n}^{\omega}$. When $k_{n}<n$ (and there are no exogenous covariates, as in
CMS and MS), Assumption 3 is weaker than the _balanced-design_ assumption in
CMS and MS which requires for $P\mathrel{\mathop{\mathchar
58\relax}}=Z(Z^{\prime}Z)^{-1}Z^{\prime}$

$\underset{1\leq i\leq n}{\text{ max }}P_{ii}\leq 1-\delta,$ (9)

for some $0<\delta<1$. The balanced-design assumption implies Assumption 3.

######  Proposition 1.

Suppose (as in CMS and MS) that $\text{rank}(P)=k_{n}$ and that there exists a
$0<\delta<1$ such that $\underset{1\leq i\leq n}{\text{ max }}P_{ii}\leq
1-\delta$. Then Assumption 3 is satisfied.

It is difficult to give more primitive conditions for Assumption 3 that remain
both easily-verifiable and sufficiently general when $k_{n}>n$. Figure 1 shows
that for the Gaussian data-generating process (DGP) design used in the
simulations in Section 4, the ratio of $\sum_{i=1}^{n}\sum_{j\neq
i}(P^{\gamma_{n}}_{ij})^{2}$ to $r_{n}$ is constant, lending plausibility to
Assumption 3 in this context. Figure 1 also shows that $\gamma^{*}_{n}$ grows
with the number of observations, which implies that our penalty parameter may
grow with the sample size (in contrast to the one proposed in [12]). In
practice, and similarly to the heuristics recommended in CMS, MS, and [18], we
recommend that in a given application, practitioners check the implied minimum
value of $c$ in Assumption 3. If very small values are observed, then
Assumption 3 may be questionable.

0300060009000025005000750010000$n$$\gamma^{*}_{n}$ (a)

0.000.010.020.030.040.05025005000750010000$n$max
${}_{\gamma_{n}>0}r_{n}^{-1}\sum_{i=1}^{n}\sum_{j\neq
i}(P^{\gamma_{n}}_{ij})^{2}$ (b)

Figure 1: Numerical Evidence on Assumption 3 for $k_{n}=1.9n$. Gaussian IVs
drawn with mean 0 and $\text{Var}\left[Z_{il}\right]=0.3$,
$\text{Corr}\left[Z_{il},Z_{im}\right]=0.5^{|l-m|}$, and independently across
$i$.

We are now in a position to derive the asymptotic distribution of the RJAR
test under the null hypothesis.

######  Theorem 1.

Suppose Assumptions 1–3 and the null hypothesis in Equation (2) hold. Consider
any sequence $\gamma_{n}$ such that, for all $n$ sufficiently large,
$\sum_{i=1}^{n}\sum_{j\neq i}(P^{\gamma_{n}}_{ij})^{2}\geq cr_{n}^{\omega}$
for some $c>0$ and $\omega\geq 1$. Then, the statistic
$RJAR_{\gamma_{n}}(\beta_{0})$ defined in Equation (5) satisfies

$RJAR_{\gamma_{n}}(\beta_{0})\overset{d}{\to}\mathcal{N}[0,1].$

######  Corollary 1.

Under Assumptions 1–3 and the null hypothesis in Equation (2), the RJAR test
given in Definition 1 has asymptotic size $\alpha$.

Notice that we did not impose any assumption on the coefficients of the
instruments $\Pi$ in the first-stage regression in Equation (1). Therefore,
the RJAR test is robust to arbitrarily weak identification.

###  3.3 Closest alternatives in the literature

The RJAR test is similar to the jackknifed AR tests proposed in CMS and MS.
The distinguishing feature is the use of a ridge-regularised projection matrix
$P^{\gamma_{n}}$, which makes the RJAR test applicable also when
$r_{n}<k_{n}$, unlike the aforementioned two tests that use the normal
projection matrix $P$, and cannot be computed when $k_{n}>n$.

CMS assume $r_{n}=k_{n}<n$, and propose the jackknifed AR statistic given by

$AR_{CMS}(\beta_{0})\mathrel{\mathop{\mathchar
58\relax}}=\frac{1}{\sqrt{k_{n}}\sqrt{\hat{\Phi}_{CMS}(\beta_{0})}}\sum_{i=1}^{n}\sum_{j\neq
i}C_{ij}e_{i}(\beta_{0})e_{j}(\beta_{0}),$

where $C\mathrel{\mathop{\mathchar 58\relax}}=A-B$,
$A\mathrel{\mathop{\mathchar 58\relax}}=P+\Delta$,
$B\mathrel{\mathop{\mathchar 58\relax}}=(I_{n}-P)D(I_{n}-D)^{-1}(I_{n}-P)$,
$\Delta\mathrel{\mathop{\mathchar
58\relax}}=PD(I_{n}-D)^{-1}P-\frac{1}{2}PD(I_{n}-D)^{-1}-\frac{1}{2}D(I_{n}-D)^{-1}P$,
and $D$ is the diagonal matrix containing the diagonal elements of $P$.
$\hat{\Phi}_{CMS}(\beta_{0})\mathrel{\mathop{\mathchar
58\relax}}=\frac{2}{k_{n}}\sum_{i=1}^{n}\\*\sum_{j\neq
i}C_{ij}^{2}\\*e_{i}^{2}(\beta_{0})\\*e_{j}^{2}(\beta_{0})$. Under the null
hypothesis in Equation (2), CMS show that $AR_{CMS}(\beta_{0})$ converges to a
Standard Normal distribution. The CLT underlying this result is a modified
version of Lemma A2 of [13] proposed in [7, Appendix A.4 ].

MS also assume that $r_{n}=k_{n}<n$, and propose a different jackknifed AR
statistic that can be obtained from $RJAR_{\gamma_{n}}(\beta_{0})$ in Equation
(5) by setting $\gamma_{n}=0$, and replacing
$\hat{\Phi}_{\gamma_{n}}(\beta_{0})$ with

$\hat{\Phi}_{MS}(\beta_{0})\mathrel{\mathop{\mathchar
58\relax}}=\frac{2}{k_{n}}\sum_{i=1}^{n}\sum_{j\neq
i}\frac{P_{ij}^{2}}{M_{ii}M_{jj}+M_{ij}^{2}}\left[e_{i}(\beta_{0})M_{i}e(\beta_{0})\right]\left[e_{j}(\beta_{0})M_{j}e(\beta_{0})\right],$
(10)

where $M=I_{n}-P$, and $M_{i}$ is the $i^{\text{th}}$ row of $M$. The reason
why the unregularised jackknifed AR test in MS uses
$\hat{\Phi}_{MS}(\beta_{0})$ instead of the variance estimator given in
Equation (6) evaluated at $\gamma_{n}=0$, is because, according to their
Theorem 4, the former yields higher power than the latter. It follows that the
unregularised version of our RJAR test, which arises when $\gamma^{*}_{n}=0$
in Equation (7), will be dominated by the jackknifed AR test of MS in terms of
power, and practitioners may prefer the latter in those cases. We note,
however, that the asymptotic size control and superior power of the jackknifed
AR test of MS are proven under the balanced design assumption given in
Equation (9) which is strictly stronger than our Assumption 3. This can have
implications for the finite-sample performance of the jackknifed AR test of MS
compared to the RJAR, which we investigate in Section 4.

When $k_{n}$ is larger than $n$, the AR tests in CMS and MS are not
applicable, since $Z^{\prime}Z$ does not have full rank. In this case, the Sup
Score test of BCCH remains valid. BCCH first standardise the IVs as in
Equation (4), and then propose the Sup Score statistic given by

${S}(\beta_{0})=\underset{1\leq j\leq k_{n}}{\text{ max
}}\frac{\left|\frac{1}{\sqrt{n}}\sum_{i=1}^{n}e_{i}(\beta_{0})Z_{ij}\right|}{\sqrt{\frac{1}{n}\sum_{i=1}^{n}\left(e_{i}(\beta_{0})\right)^{2}Z_{ij}^{2}}}.$

BCCH propose to use the critical value
$\nu_{\alpha}=c_{BCCH}\sqrt{n}\\*\mathcal{Q}\left(1-\alpha/(2k_{n})\right)$,
$c_{BCCH}>1$ and $\alpha\in(0,1)$. They show that comparing their Sup Score
statistic to this critical value yields a test of the null hypothesis in
Equation (2) that has asymptotic size less than or equal to $\alpha$. Being a
supremum-norm test suggests that the BCCH Sup Score test will work well with a
sparse first stage (i.e. where only a few elements of $\Pi$ are zero), but may
have lower power than the RJAR test when the first stage is dense. This is
verified in the simulations in Section 4.

##  4 Simulations

We now investigate the size and power properties of the RJAR test and compare
them to those of the tests proposed in BCCH, CMS and MS. We take our
simulation setup from [18] who in turn take theirs from BCCH. The DGP is given
by

$\displaystyle y_{i}$ $\displaystyle=X_{i}\\*\beta+\varepsilon_{i}$ (11a)
$\displaystyle X_{i}$ $\displaystyle=Z_{i}^{\prime}\\*\pi+v_{i},$ (11b)
$\displaystyle\begin{bmatrix}\varepsilon_{i}\\\ v_{i}\end{bmatrix}$
$\displaystyle\sim\mathcal{N}\left[0,\left[\begin{smallmatrix}\sigma_{\varepsilon}^{2}&\sigma_{\varepsilon\\*v}\\\
\sigma_{\varepsilon\\*v}&\sigma_{v}^{2}\end{smallmatrix}\right]\right],$

for $i=1,\dots,n=100$, with $\sigma_{\varepsilon}^{2}=2$, $\sigma_{v}^{2}=1$
and $\sigma_{\varepsilon
v}=0.6\sigma_{\varepsilon}\sigma_{v}$.555Heteroskedasticity could be
incorporated in standard ways, e.g., by scaling the structural error by the
value of a randomly selected IV for every observation. As is to be expected
given that all the approaches considered here are robust to heteroskedasticity
of arbitrary form, such modifications do not qualitatively affect the
simulations. The IVs are independent Gaussian with mean 0 and
$\text{Var}\left[Z_{il}\right]=0.3$ and
$\text{Corr}\left[Z_{il},Z_{im}\right]=0.5^{|l-m|}$.666Unreported simulations
show that the results remain qualitatively unchanged if uncorrelated Gaussian
IVs are considered instead. $\pi=\varrho\\*\kappa$, where $\kappa$ is a vector
of zeros and ones that varies with the type of DGP considered (sparse or
dense, as modelled below), and $\varrho$ is some scalar that ensures that for
a given concentration parameter, $\mu^{2}$, the following relationship is
satisfied:

$\mu^{2}=\frac{n\\*\pi^{\prime}\mathbb{E}\left[Z_{i}\\*Z_{i}^{\prime}\right]\\*\pi}{\sigma_{v}^{2}}.$

This implies that

$\varrho=\sqrt{\frac{\sigma_{v}^{2}\\*\mu^{2}}{n\kappa^{\prime}\mathbb{E}\left[Z_{i}\\*Z_{i}^{\prime}\right]\kappa}}.$

To illustrate how the sparsity structure of the first stage in Equation (11b)
can affect the size and power of the studied tests, we consider both a sparse
first stage and a dense first stage. Sparsity in the first stage is modelled
by setting $\kappa=[\iota_{5}^{\prime},0_{k_{n}-5}^{\prime}]^{\prime}$, where
$\iota_{q}$ is a $q\times 1$ vector of ones, and $0_{q}$ is a $q\times 1$
vector of zeros. Density in the first stage is modelled by setting
$\kappa=[\iota_{0.4k_{n}}^{\prime},0_{0.6k_{n}}^{\prime}]^{\prime}$.

We consider $k_{n}=30,90,190$. Throughout, $\gamma_{-}$ in Assumption 3 is set
equal to 1. For the case of 30 IVs, the RJAR test does not impose any
regularisation ($\gamma^{*}_{n}=0$). For the case of 90 IVs,
$\gamma^{*}_{n}=12.048$. We note that $\sum_{i=1}^{n}\sum_{j\neq
i}(P^{\gamma^{*}_{n}}_{ij})^{2}=11.123>8.845=\sum_{i=1}^{n}\sum_{j\neq
i}(P_{ij})^{2}$. This shows that even in the case where $r_{n}<n$, ridge
regularisation can make Assumption 3 strictly more plausible. For the case of
190 IVs, $\gamma^{*}_{n}=109.187$. The variance estimator of MS occasionally
yields a negative value. These cases are conservatively interpreted as a
failure to reject the null hypothesis. As recommended by BCCH, $c_{BCCH}=1.1$.
The number of Monte Carlo replications is 10,000.

###  4.1 Size

Figure 2 shows the simulation results for the sparse first stage for tests of
size 0.01 to 0.99, that is the rejection frequency under
$H_{0}\mathrel{\mathop{\mathchar 58\relax}}\beta_{0}=1$. As far as the
illustration of the tests’ size properties is concerned, the dense first stage
yields virtually the same results, and is hence omitted. Since all tests are
robust to weak IVs, the rejection frequencies of the tests are not affected by
the value of the concentration parameter. For the case of 30 IVs, the AR tests
of CMS and MS and the RJAR test have correct size, while the BCCH Sup Score
test is undersized.

For the case of 90 IVs, the AR test of CMS appears to control size for common
small nominal test sizes (e.g., 0.05 or 0.1). The AR test of MS appears to be
generally oversized. For example, at nominal level 0.05, the rejection
frequency of the test is 0.189.777Unreported simulations confirm that if $n$
and $k_{n}$ are increased to about 1,000 and 900, respectively, there is no
longer any size distortion. The BCCH Sup Score test continues to be
undersized, while the RJAR test continues to have correct size irrespective of
the value of $\mu^{2}$.

For the case of 190 IVs, only the BCCH Sup Score test and the RJAR test are
feasible. As before, the BCCH Sup Score test is undersized, while the RJAR
test has correct size.

0.000.250.500.751.000.000.250.500.751.00Nominal Test SizeRejection Frequency
(a) $k_{n}=30$

0.000.250.500.751.000.000.250.500.751.00Nominal Test SizeRejection Frequency
(b) $k_{n}=90$

0.000.250.500.751.000.000.250.500.751.00Nominal Test SizeRejection Frequency
(c) $k_{n}=190$

Figure 2: PP Plots for Sparse IVs, $\mu^{2}=0$,
$H_{0}\mathrel{\mathop{\mathchar 58\relax}}\beta_{0}=1$.

###  4.2 Power

Figures 3–8 show the power of the tests when the number of IVs and the
sparsity pattern of the first stage is varied. It is still the case that
$H_{0}\mathrel{\mathop{\mathchar 58\relax}}\beta_{0}=1$. For the case of 30
IVs (Figure 3 and Figure 4), the AR test of CMS has similar power to the RJAR
test, while MS is slightly more powerful than the RJAR test. The BCCH Sup
Score test is less powerful than all other tests. For the case of 90 sparse
IVs (Figure 5), the RJAR test is slightly more powerful than the BCCH Sup
Score test. The AR test of MS fails to control the size, while the AR test of
CMS exhibits power properties substantially worse than those of the BCCH Sup
Score test and the RJAR test. For the case of 90 dense IVs (Figure 6), the AR
tests of CMS and MS exhibit the same lower power and failure to control the
test size as in the case of 90 sparse IVs. In this dense setting, the RJAR
test is substantially more powerful than the BCCH Sup Score test. For the case
of 190 sparse and dense IVs (Figure 7 and Figure 8), the RJAR test is more
powerful than the BCCH Sup Score test. Thus, for all the DGPs that are
considered here, the RJAR test is as powerful as existing methods whenever
these are applicable, and sometimes much more powerful.

0.000.250.500.751.00-10123$\beta$Rejection Frequency (a) $\mu^{2}=0$

0.000.250.500.751.00-10123$\beta$Rejection Frequency (b) $\mu^{2}=30$

0.000.250.500.751.00-10123$\beta$Rejection Frequency (c) $\mu^{2}=60$

0.000.250.500.751.00-10123$\beta$Rejection Frequency (d) $\mu^{2}=180$

Figure 3: Power Curves for 30 Sparse IVs. Nominal test size of 5% indicated by
the grey horizontal line.

0.000.250.500.751.00-10123$\beta$Rejection Frequency (a) $\mu^{2}=0$

0.000.250.500.751.00-10123$\beta$Rejection Frequency (b) $\mu^{2}=30$

0.000.250.500.751.00-10123$\beta$Rejection Frequency (c) $\mu^{2}=60$

0.000.250.500.751.00-10123$\beta$Rejection Frequency (d) $\mu^{2}=180$

Figure 4: Power Curves for 30 Dense IVs. Nominal test size of 5% indicated by
the grey horizontal line.

0.000.250.500.751.00-10123$\beta$Rejection Frequency (a) $\mu^{2}=0$

0.000.250.500.751.00-10123$\beta$Rejection Frequency (b) $\mu^{2}=30$

0.000.250.500.751.00-10123$\beta$Rejection Frequency (c) $\mu^{2}=60$

0.000.250.500.751.00-10123$\beta$Rejection Frequency (d) $\mu^{2}=180$

Figure 5: Power Curves for 90 Sparse IVs. Nominal test size of 5% indicated by
the grey horizontal line.

0.000.250.500.751.00-10123$\beta$Rejection Frequency (a) $\mu^{2}=0$

0.000.250.500.751.00-10123$\beta$Rejection Frequency (b) $\mu^{2}=30$

0.000.250.500.751.00-10123$\beta$Rejection Frequency (c) $\mu^{2}=60$

0.000.250.500.751.00-10123$\beta$Rejection Frequency (d) $\mu^{2}=180$

Figure 6: Power Curves for 90 Dense IVs. Nominal test size of 5% indicated by
the grey horizontal line.

0.000.250.500.751.00-10123$\beta$Rejection Frequency (a) $\mu^{2}=0$

0.000.250.500.751.00-10123$\beta$Rejection Frequency (b) $\mu^{2}=30$

0.000.250.500.751.00-10123$\beta_{0}$Rejection Frequency (c) $\mu^{2}=60$

0.000.250.500.751.00-10123$\beta$Rejection Frequency (d) $\mu^{2}=180$

Figure 7: Power Curves for 190 Sparse IVs. Nominal test size of 5% indicated
by the grey horizontal line.

0.000.250.500.751.00-10123$\beta$Rejection Frequency (a) $\mu^{2}=0$

0.000.250.500.751.00-10123$\beta$Rejection Frequency (b) $\mu^{2}=30$

0.000.250.500.751.00-10123$\beta$Rejection Frequency (c) $\mu^{2}=60$

0.000.250.500.751.00-10123$\beta$Rejection Frequency (d) $\mu^{2}=180$

Figure 8: Power Curves for 190 Dense IVs. Nominal test size of 5% indicated by
the grey horizontal line.

##  5 Empirical Application

We consider an empirical application based on [11]. The coefficient of
interest is given by $\beta_{s}$ in the following model:

$\displaystyle y_{is}$
$\displaystyle=\beta_{s}X_{is}+\delta_{s}^{\prime}W_{i}+\varepsilon_{is},$
(12)

where $y_{is}$ is the difference between residual log wages for immigrant and
native men in skill group $s$ in city $i$,888As discussed in [11, p. 11,
footnote 17 ], residual wages are wages once observed characteristics of the
entire US workforce are controlled for. $X_{is}$ is the log ratio of immigrant
to native hours worked in skill group $s$ of both men and women in city $i$,
and $W_{i}$ is a vector of city-level controls with coefficient vector
$\delta_{s}$, and $\varepsilon_{is}$ is the structural error. In the context
of the production function specified in [11, Section I ], $\beta_{s}$ can be
interpreted as the (negative) inverse elasticity of substitution between
immigrants and natives in the US in their respective skill group. As in [11],
we consider two skill groups $s=h,c$ (high school or college equivalent)
separately.

[11] raises the concern that unobserved factors in a city may lead to both
higher wages and higher employment levels of immigrants relative to natives,
causing $X_{is}$ to be endogenous. [11] proposes to use the ratio of the
number of immigrants from country $l$ in city $i$ to the total number of
immigrants from foreign country $l$ in the US as an IV. The rationale for
these IVs is that existing immigrant enclaves are likely to attract additional
immigrant labour through social and cultural channels unrelated to labour
market outcomes. We consider two sets of IVs. First, we consider the original
setup of [11], using as IVs the $k_{n}=38$ different countries of origin of
the immigrants. Second, motivated by the saturation approach of [9], we
consider the setup where these 38 original IVs are interacted with the $q=9$
available controls (including a constant). This yields $k_{n}=342$ IVs. In
both cases, the number of observations (i.e., the number of cities) is
$n=124$.999Note that the controls and the IVs are at the city level, and hence
the same for the applications to high-school workers and college workers.
Therefore, the projection matrix and the ridge-regularised projection matrix
will also be the same across these two skill groups.

We construct (weak-identification robust) confidence sets for $\beta_{s}$ by
inverting the AR tests of CMS and MS, the Sup Score test, and the RJAR test.
Thus, the 95% confidence set for any test is obtained as the collection of
$\beta_{s,0}$ for which that test does not reject the null at 5% level of
significance. As in the simulation exercise in Section 4, $\gamma_{-}=1$, and
$c_{BCCH}=1.1$. A grid of 100 values for $\beta_{s,0}$ is used for $s=h,c$.
Data is taken from a single cross section, as made available by [16].

Figure 9 shows the confidence sets when $k_{n}=38$ for high-school workers and
college workers. We find that $\gamma^{*}_{n}=0$, implying that no
regularisation is needed. The 95% confidence sets for each test are given by
all the points below the grey horizontal line. This is in line with our
simulations in Section 4, where we found that regularisation was not needed to
maximise the sum in Assumption 3 when $k_{n}/n=0.3$. The confidence sets for
both skill groups broadly confirm the results in [11]. We find that the
confidence sets for high-school workers is smallest for the jackknifed AR
statistic of CMS, whereas the BCCH Sup Score test yields the smallest
confidence interval for the application to college workers. Based on the power
results in Section 4, this suggests a very sparse first stage for college
workers, i.e., a few nationalities being highly predictive of inflows of
immigrant labour. In line with the simulation evidence on power for
$k_{n}/n=0.3$ in Section 4, the differences in confidence sets across the four
different tests considered appear to be reasonably small.

(a) High-school workers

(b) College workers

Figure 9: 95 % confidence sets for $\beta_{s}$ for the application in Equation
(12) with $k_{n}=38$ IVs. $\text{max}_{i}P_{ii}=0.944$. $\gamma_{n}^{*}=0$,
$r^{-1}_{n}\sum_{i=1}^{n}\sum_{j\neq i}(P^{\gamma^{*}_{n}}_{ij})^{2}=0.513$.

Figure 10 shows the confidence sets when $k_{n}=342$ for high-school workers
and college workers, respectively. Since $r_{n}=n-q=124-9=115<342=k_{n}$, the
jackknifed AR statistics of CMS and MS are not applicable. We find that
$\gamma^{*}_{n}=5.299$ and $r^{-1}_{n}\sum_{i=1}^{n}\sum_{j\neq
i}(P^{\gamma^{*}_{n}}_{ij})^{2}=0.106$. In line with the simulation results on
power reported in Section 4, the RJAR test yields smaller confidence intervals
than the BCCH sup score test. The qualitative conclusions with respect to the
case of 38 IVs remain unchanged.

(a) High-school workers

(b) College workers

Figure 10: 95 % confidence sets for $\beta_{s}$ for the application in
Equation (12) with $k_{n}=342$ IVs. $\gamma_{n}^{*}=5.299$,
$r^{-1}_{n}\sum_{i=1}^{n}\sum_{j\neq i}(P^{\gamma^{*}_{n}}_{ij})^{2}=0.106$.

##  6 Conclusion

We contributed to the literature on (very) many IVs in the cross-sectional
linear IV model by proposing a new, ridge-regularised jackknifed AR test. Our
test compares favourably with existing methods in the literature both
theoretically, by allowing for high-dimensional IVs and weakening a common
assumption on the IVs’ projection matrix, and practically, by having correct
asymptotic size and displaying favourable power properties even when the
number of IVs approaches or exceeds the number of observations.

##  Appendix A Proofs

Throughout this appendix, $C>0$ denotes a universal constant that can change
across lines. CSHNW refers to [13] from where we also borrow our summation
conventions. Since all the proofs in this appendix are under the null
hypothesis, we write $\varepsilon_{i}$ and $\hat{\Phi}_{\gamma_{n}}$ for
$e_{i}(\beta_{0})$ and $\hat{\Phi}_{\gamma_{n}}$, respectively. We also define
$\Phi_{\gamma_{n}}\mathrel{\mathop{\mathchar
58\relax}}=\frac{2}{r_{n}}\sum_{i=1}^{n}\sum_{j\neq
i}(P_{ij}^{\gamma_{n}})^{2}\mathbb{E}[\varepsilon_{i}^{2}]\mathbb{E}[\varepsilon_{j}^{2}]$,

The following singular value decomposition of the $n\times k_{n}$ matrix of
instruments $Z$ of rank $r_{n}$ will be used frequently (see, e.g., [21, p. 60
]):

$Z=USQ^{\prime},$

where $U$ is an $n\times n$ matrix such that $U^{\prime}U=UU^{\prime}=I_{n}$,
$Q$ is a $k_{n}\times k_{n}$ matrix such that
$Q^{\prime}Q=QQ^{\prime}=I_{k_{n}}$, and $S$ is the $n\times k_{n}$ matrix
given by

$S=\begin{bmatrix}D&0_{r_{n}\times(k_{n}-r_{n})}\\\ 0_{(n-r_{n})\times
r_{n}}&0_{(n-r_{n})\times(k-r_{n})}\\\ \end{bmatrix},$

where $D$ is the diagonal $r_{n}\times r_{n}$ matrix containing the singular
values of $Z$. Hence, one can write

$\displaystyle
P^{\gamma_{n}}=USQ^{\prime}(QS^{\prime}SQ^{\prime}+Q\gamma_{n}I_{k_{n}}Q^{\prime})^{-1}QS^{\prime}U^{\prime}=US(S^{\prime}S+\gamma_{n}I_{k_{n}})^{-1}S^{\prime}U^{\prime}=U\tilde{D}U^{\prime},$
(A.1)

where $\tilde{D}=S(S^{\prime}S+\gamma_{n}I_{k_{n}})^{-1}S^{\prime}$ is the
diagonal $n\times n$ matrix with diagonal entries given by given by
$\tilde{D}_{ll}=\frac{D_{ll}^{2}}{D_{ll}^{2}+\gamma_{n}}\leq 1$ for
$l=1,\dots,r_{n}$, and zero otherwise. Note that the diagonal entries of
$\tilde{D}$ are also the the eigenvalues of $P^{\gamma_{n}}$.

###  A.1 Lemmas

The following lemma collects some properties of $P^{\gamma_{n}}$. Recall that
$Z$ has rank $r_{n}$.

######  Lemma 1.

Fix $n\geq 3$. For all $h,m=1,\dots,n$ and $\gamma_{n}\geq 0$ if $r_{n}=k_{n}$
and $\gamma_{n}>0$ if $r_{n}<k_{n}$ one has

  1. (i)

$0\leq(P^{\gamma_{n}})^{j}_{hh}\leq P^{\gamma_{n}}_{hh}\leq 1$ for all
positive integers $j$,

  2. (ii)

$\sum_{i=1}^{n}(P^{\gamma_{n}}_{hi})^{2}=(P^{\gamma_{n}})^{2}_{hh}\leq
P^{\gamma_{n}}_{hh}$,

  3. (iii)

$\sum_{i=1}^{n}P^{\gamma_{n}}_{ii}=\sum_{l=1}^{r_{n}}\frac{D_{ll}^{2}}{D_{ll}^{2}+\gamma_{n}}\leq
r_{n}$,

  4. (iv)

$\sum_{i=1}^{n}(P^{\gamma_{n}})^{2}_{ii}=\sum_{l=1}^{r_{n}}\frac{D_{ll}^{4}}{\left(D_{ll}^{2}+\gamma_{n}\right)^{2}}\leq
r_{n}$,

  5. (v)

$|P^{\gamma_{n}}_{hm}|\leq 1$,

  6. (vi)

For any $\mathcal{I}_{2}\subseteq\\{1,\ldots,n\\}^{2}$ and any
$\mathcal{I}_{3}\subseteq\\{1,\ldots,n\\}^{3}$,

    1. (a)

$\sum_{\mathcal{I}_{2}}(P^{\gamma_{n}}_{ij})^{4}\leq r_{n}$,

    2. (b)

$\sum_{\mathcal{I}_{3}}(P^{\gamma_{n}}_{ij})^{2}(P^{\gamma_{n}}_{jl})^{2}\leq
r_{n}$.

###### Proof.

By the arguments prior to this lemma, the non-zero eigenvalues of
$P^{\gamma_{n}}$ are $\tilde{D}_{ll}=\frac{D_{ll}^{2}}{D_{ll}^{2}+\gamma_{n}}$
for $l=1,\dots,r_{n}$. (iii) now follows from the trace of $P^{\gamma_{n}}$
equaling the sum of its non-zero eigenvalues. Furthermore, the eigenvalues of
$(P^{\gamma_{n}})^{2}$ are given by
$\tilde{D}_{ll}^{2}=\frac{D_{ll}^{4}}{(D_{ll}^{2}+\gamma_{n})^{2}}$ for
$l=1,\dots,r_{n}$, and zero otherwise. (iv) now follows in the same way as
(iii).

By Equation (A.1) one has that

$(P^{\gamma_{n}})^{j}_{hh}=\sum_{l=1}^{n}\tilde{D}_{ll}^{j}U_{hl}^{2}\leq\sum_{l=1}^{n}\tilde{D}_{ll}U_{hl}^{2}=P^{\gamma_{n}}_{hh}\leq
1,$ (A.2)

which verifies (i) since $0\leq\tilde{D}_{ll}\leq 1$; the last inequality
following from the largest diagonal entry of $P^{\gamma_{n}}$ being bounded
from above by its largest eigenvalue, which is no greater than one. Next, by
(i),

$\displaystyle\sum_{i=1}^{n}(P^{\gamma_{n}}_{hi})^{2}=(P^{\gamma_{n}})^{2}_{hh}\leq
P^{\gamma_{n}}_{hh}\leq 1,$

such that (ii) and (v) follow. Furthermore, by (v) and (iv),

$\sum_{\mathcal{I}_{2}}(P^{\gamma_{n}}_{ij})^{4}\leq\sum_{i=1}^{n}\sum_{j=1}^{n}(P^{\gamma_{n}}_{ij})^{2}=\sum_{i=1}^{n}(P^{\gamma_{n}})^{2}_{ii}\leq
r_{n},$

and (vi) follows since

$\sum_{\mathcal{I}_{3}}(P^{\gamma_{n}}_{ij})^{2}(P^{\gamma_{n}}_{jl})^{2}\leq\sum_{j=1}^{n}\sum_{i=1}^{n}(P^{\gamma_{n}}_{ij})^{2}\sum_{l=1}^{n}(P^{\gamma_{n}}_{jl})^{2}=\sum_{j=1}^{n}\left((P^{\gamma_{n}})^{2}_{jj}\right)^{2}\leq\sum_{j=1}^{n}(P^{\gamma_{n}})^{2}_{jj}\leq
r_{n},$

the penultimate inequality being a consequence of (i) and the last of (iv).

∎

######  Lemma 2.

Fix $n\geq 4$. For all $\gamma_{n}\geq 0$ if $r_{n}=k_{n}$ and $\gamma_{n}>0$
if $r_{n}<k_{n}$ one has

$\left|\sum_{i<j<l<m}P^{\gamma_{n}}_{il}P^{\gamma_{n}}_{jl}P^{\gamma_{n}}_{im}P^{\gamma_{n}}_{jm}\right|\leq
Cr_{n}.$

###### Proof.

The proof follows Lemma B2 in CSHNW closely. For $G\mathrel{\mathop{\mathchar
58\relax}}=\text{diag}(P^{\gamma_{n}}_{11},\dots,P^{\gamma_{n}}_{nn})$ observe
that $(P^{\gamma_{n}}-G)^{4}$ equals

$\displaystyle(P^{\gamma_{n}})^{4}-(P^{\gamma_{n}})^{3}G-(P^{\gamma_{n}})^{2}GP^{\gamma_{n}}+(P^{\gamma_{n}})^{2}G^{2}-P^{\gamma_{n}}G(P^{\gamma_{n}})^{2}+P^{\gamma_{n}}GP^{\gamma_{n}}G$
$\displaystyle+$ $\displaystyle
P^{\gamma_{n}}G^{2}P^{\gamma_{n}}-P^{\gamma_{n}}G^{3}-G(P^{\gamma_{n}})^{3}+G(P^{\gamma_{n}})^{2}G+GP^{\gamma_{n}}GP^{\gamma_{n}}$
$\displaystyle-$ $\displaystyle
GP^{\gamma_{n}}G^{2}+G^{2}(P^{\gamma_{n}})^{2}-G^{2}P^{\gamma_{n}}G-G^{3}P^{\gamma_{n}}+G^{4}.$

Since $\text{tr}(A^{\prime})=\text{tr}(A)$ and $\text{tr}(AB)=\text{tr}(BA)$
for square matrices $A$ and $B$,

$\displaystyle\text{tr}((P^{\gamma_{n}}-G)^{4})=$
$\displaystyle\text{tr}((P^{\gamma_{n}})^{4})-4\text{tr}((P^{\gamma_{n}})^{3}G)+4\text{tr}((P^{\gamma_{n}})^{2}G^{2})+2\text{tr}(P^{\gamma_{n}}GP^{\gamma_{n}}G)$
$\displaystyle-4\text{tr}(P^{\gamma_{n}}G^{3})+\text{tr}(G^{4})$
$\displaystyle\leq$
$\displaystyle\text{tr}((P^{\gamma_{n}})^{4})+4|\text{tr}((P^{\gamma_{n}})^{3}G)|+4\text{tr}((P^{\gamma_{n}})^{2}G^{2})+2\text{tr}(P^{\gamma_{n}}GP^{\gamma_{n}}G)$
$\displaystyle+4|\text{tr}(P^{\gamma_{n}}G^{3})|+\text{tr}(G^{4}).$

Next, because $0\leq(P^{\gamma_{n}})^{j}_{ii}\leq P^{\gamma_{n}}_{ii}$ for all
positive integers $j$ by Lemma 1 (i), and $G_{ii}\geq 0$, one gets

$\displaystyle\text{tr}((P^{\gamma_{n}}-G)^{4})\leq$
$\displaystyle\text{tr}(P^{\gamma_{n}})+4\text{tr}(P^{\gamma_{n}}G)+4\text{tr}(P^{\gamma_{n}}G^{2})+2\text{tr}(P^{\gamma_{n}}GP^{\gamma_{n}}G)$
$\displaystyle+4\text{tr}(P^{\gamma_{n}}G^{3})+\text{tr}(G^{4}).$

Since $0\leq P^{\gamma_{n}}_{ii}\leq 1$ and $G^{j}\leq I_{n}$ (elementwise)
for all positive integers $j$,
$\text{tr}(P^{\gamma_{n}}G^{j})\leq\text{tr}(P^{\gamma_{n}})\leq r_{n}$,
$\text{tr}(P^{\gamma_{n}}GP^{\gamma_{n}}G)\leq\text{tr}((P^{\gamma_{n}})^{2})\leq
r_{n}$. Furthermore,
$\text{tr}(G^{4})\leq\text{tr}(G)=\text{tr}(P^{\gamma_{n}})\leq r_{n}$. Hence,

$\text{tr}((P^{\gamma_{n}}-G)^{4})\leq 16r_{n}.$ (A.3)

As in the proof of Lemma B2 in CSHNW, define the lower-triangular matrix $L$
with entries $L_{ij}=P^{\gamma_{n}}_{ij}\mathbf{1}_{i>j}$, so that
$P^{\gamma_{n}}=L+L^{\prime}+G$. Then

$\displaystyle(P^{\gamma_{n}}-G)^{4}=$ $\displaystyle(L+L^{\prime})^{4}$
$\displaystyle=$ $\displaystyle
L^{4}+L^{2}LL^{\prime}+L^{2}L^{\prime}L+L^{2}L^{\prime
2}+LL^{\prime}L^{2}+LL^{\prime}LL^{\prime}$
$\displaystyle+LL^{\prime}L^{\prime}L+LL^{\prime
3}+L^{\prime}LL^{2}+L^{\prime}LLL^{\prime}+L^{\prime}LL^{\prime}L+L^{\prime}LL^{\prime
2}$ $\displaystyle+L^{\prime 2}L^{2}+L^{\prime 2}LL^{\prime}+L^{\prime
2}L^{\prime}L+L^{\prime 4}.$

Note for all positive integers $j$, $[(L^{\prime})^{j}]^{\prime}=L^{j}$. Since
$\text{tr}(A^{\prime})=\text{tr}(A)$ and $\text{tr}(AB)=\text{tr}(BA)$ for any
square matrices $A$ and $B$,

$\displaystyle\text{tr}((P^{\gamma_{n}}-G)^{4})=2\text{tr}(L^{4})+8\text{tr}(L^{3}L^{\prime})+4\text{tr}(L^{2}L^{\prime
2})+2\text{tr}(L^{\prime}LL^{\prime}L).$ (A.4)

We consider each of the terms on the right-hand side of Equation (A.4)
separately. Before proceeding, note that:

$\displaystyle\begin{split}(L)^{2}_{ab}&=\sum_{l=1}^{n}L_{al}L_{lb},\\\
(L^{\prime})^{2}_{ab}&=\sum_{l=1}^{n}L_{la}L_{bl}=\sum_{l=1}^{n}L_{bl}L_{la}=(L)^{2}_{ba},\\\
(L^{\prime}L)_{ab}&=\sum_{l=1}^{n}L_{la}L_{lb},\\\
(L)^{3}_{ab}&=\sum_{l=1}^{n}(L)^{2}_{al}L_{lb}=\sum_{l=1}^{n}\sum_{m=1}^{n}L_{am}L_{ml}L_{lb},\\\
(L)^{4}_{ab}&=\sum_{l=1}^{n}(L)^{2}_{al}(L)^{2}_{lb}=\sum_{l=1}^{n}\sum_{m=1}^{n}\sum_{j=1}^{n}L_{am}L_{ml}L_{lj}L_{jb},\\\
(L^{3}L^{\prime})_{ab}&=\sum_{l=1}^{n}(L)^{3}_{al}(L^{\prime})_{lb}=\sum_{l=1}^{n}\sum_{m=1}^{n}\sum_{j=1}^{n}L_{am}L_{mj}L_{jl}L_{bl},\\\
(L^{2}L^{\prime
2})_{ab}&=\sum_{l=1}^{n}(L)^{2}_{al}(L^{\prime})^{2}_{lb}=\sum_{l=1}^{n}(L)^{2}_{al}(L)^{2}_{bl}=\sum_{l=1}^{n}\sum_{m=1}^{n}\sum_{j=1}^{n}L_{am}L_{ml}L_{bj}L_{jl},\\\
(L^{\prime}LL^{\prime}L)_{ab}&=\sum_{l=1}^{n}(L^{\prime}L)_{al}(L^{\prime}L)_{lb}=\sum_{l=1}^{n}\sum_{m=1}^{n}\sum_{j=1}^{n}L_{ma}L_{ml}L_{jl}L_{jb}.\end{split}$
(A.5)

Using the expression for $(L)^{4}_{ab}$ in Equation (A.5),

$\displaystyle\text{tr}(L^{4})$
$\displaystyle=\sum_{i=1}^{n}\sum_{l=1}^{n}\sum_{m=1}^{n}\sum_{j=1}^{n}L_{im}L_{ml}L_{lj}L_{ji}=\sum_{i,j,l,m}P^{\gamma_{n}}_{ji}\mathbf{1}_{j>i}P^{\gamma_{n}}_{im}\mathbf{1}_{i>m}P^{\gamma_{n}}_{ml}\mathbf{1}_{m>l}P^{\gamma_{n}}_{lj}\mathbf{1}_{l>j}$
(A.6) $\displaystyle=0,$

since there is no combination of indices $i,j,l,m$ that jointly satisfy each
of the indicator functions, as this would require $m<i<j<l<m$.

Using the expression for $(L^{3}L^{\prime})_{ab}$ in Equation (A.5),

$\displaystyle\text{tr}(L^{3}L^{\prime})$
$\displaystyle=\sum_{i=1}^{n}\sum_{l=1}^{n}\sum_{m=1}^{n}\sum_{j=1}^{n}L_{im}L_{mj}L_{jl}L_{il}=\sum_{i,j,l,m}P^{\gamma_{n}}_{im}\mathbf{1}_{i>m}P^{\gamma_{n}}_{mj}\mathbf{1}_{m>j}P^{\gamma_{n}}_{jl}\mathbf{1}_{j>l}P^{\gamma_{n}}_{il}\mathbf{1}_{i>l}$
(A.7)
$\displaystyle=\sum_{l<j<m<i}P^{\gamma_{n}}_{im}P^{\gamma_{n}}_{mj}P^{\gamma_{n}}_{jl}P^{\gamma_{n}}_{il}=\sum_{i<j<l<m}P^{\gamma_{n}}_{ml}P^{\gamma_{n}}_{lj}P^{\gamma_{n}}_{ji}P^{\gamma_{n}}_{mi}$
$\displaystyle=\sum_{i<j<l<m}P^{\gamma_{n}}_{ij}P^{\gamma_{n}}_{jl}P^{\gamma_{n}}_{im}P^{\gamma_{n}}_{lm}.$

The third equality follows since the product within the summations is non-zero
only for $l<j<m<i$. The fourth equality follows by replacing the indices
according to the dictionary $\\{i\mathrel{\mathop{\mathchar
58\relax}}m,j\mathrel{\mathop{\mathchar
58\relax}}j,l\mathrel{\mathop{\mathchar
58\relax}}i,m\mathrel{\mathop{\mathchar 58\relax}}l\\}$. The last equality
follows from the symmetry of $P^{\gamma_{n}}$.

Using the expression for $(L^{2}L^{\prime 2})_{ab}$ in Equation (A.5),

$\displaystyle\text{tr}(L^{2}L^{\prime 2})$
$\displaystyle=\sum_{i=1}^{n}\sum_{l=1}^{n}\sum_{m=1}^{n}\sum_{j=1}^{n}L_{im}L_{ml}L_{ij}L_{jl}$
(A.8)
$\displaystyle=\sum_{i,j,l,m}P^{\gamma_{n}}_{im}\mathbf{1}_{i>m}P^{\gamma_{n}}_{ml}\mathbf{1}_{m>l}P^{\gamma_{n}}_{ij}\mathbf{1}_{i>j}P^{\gamma_{n}}_{jl}\mathbf{1}_{j>l}$
$\displaystyle=$
$\displaystyle\sum_{i>j>l,i>m>l}P^{\gamma_{n}}_{im}P^{\gamma_{n}}_{ml}P^{\gamma_{n}}_{ij}P^{\gamma_{n}}_{jl}=\sum_{i>j>l,i>m>l}P^{\gamma_{n}}_{ij}P^{\gamma_{n}}_{jl}P^{\gamma_{n}}_{lm}P^{\gamma_{n}}_{mi}$
$\displaystyle=$
$\displaystyle\sum_{i>j=m>l}P^{\gamma_{n}}_{ij}P^{\gamma_{n}}_{jl}P^{\gamma_{n}}_{lm}P^{\gamma_{n}}_{mi}+\sum_{i>j>m>l}P^{\gamma_{n}}_{ij}P^{\gamma_{n}}_{jl}P^{\gamma_{n}}_{lm}P^{\gamma_{n}}_{mi}$
$\displaystyle+\sum_{i>m>j>l}P^{\gamma_{n}}_{ij}P^{\gamma_{n}}_{jl}P^{\gamma_{n}}_{lm}P^{\gamma_{n}}_{mi}$
$\displaystyle=$
$\displaystyle\sum_{i>j>l}P^{\gamma_{n}}_{ij}P^{\gamma_{n}}_{jl}P^{\gamma_{n}}_{lj}P^{\gamma_{n}}_{ji}+\sum_{i<j<l<m}P^{\gamma_{n}}_{ml}P^{\gamma_{n}}_{li}P^{\gamma_{n}}_{ij}P^{\gamma_{n}}_{jm}$
$\displaystyle+\sum_{i>m>j>l}P^{\gamma_{n}}_{ij}P^{\gamma_{n}}_{jl}P^{\gamma_{n}}_{lm}P^{\gamma_{n}}_{mi}$
$\displaystyle=$
$\displaystyle\sum_{i>j>l}P^{\gamma_{n}}_{ij}P^{\gamma_{n}}_{jl}P^{\gamma_{n}}_{lj}P^{\gamma_{n}}_{ji}+\sum_{i<j<l<m}P^{\gamma_{n}}_{ml}P^{\gamma_{n}}_{li}P^{\gamma_{n}}_{ij}P^{\gamma_{n}}_{jm}$
$\displaystyle+\sum_{i<j<l<m}P^{\gamma_{n}}_{mj}P^{\gamma_{n}}_{ji}P^{\gamma_{n}}_{il}P^{\gamma_{n}}_{lm}$
$\displaystyle=$
$\displaystyle\sum_{i<j<l}(P^{\gamma_{n}}_{ij})^{2}(P^{\gamma_{n}}_{jl})^{2}+2\sum_{i<j<l<m}P^{\gamma_{n}}_{ij}P^{\gamma_{n}}_{il}P^{\gamma_{n}}_{jm}P^{\gamma_{n}}_{lm}.$

The third equality follows since the product within the summations is non-zero
only when both $i>j>l$ and $i>m>l$. The fourth equality follows from the
symmetry of $P^{\gamma_{n}}$. The sixth equality follows by replacing the
indices in the second summation according to the dictionary
$\\{i\mathrel{\mathop{\mathchar 58\relax}}m,j\mathrel{\mathop{\mathchar
58\relax}}l,l\mathrel{\mathop{\mathchar
58\relax}}i,m\mathrel{\mathop{\mathchar 58\relax}}j\\}$. The seventh equality
follows by replacing the indices in the third summation according to the
dictionary $\\{i\mathrel{\mathop{\mathchar
58\relax}}m,j\mathrel{\mathop{\mathchar
58\relax}}j,l\mathrel{\mathop{\mathchar
58\relax}}i,m\mathrel{\mathop{\mathchar 58\relax}}l\\}$. The last equality
follows from the symmetry of $P^{\gamma_{n}}$.

Using the expression for $(L^{\prime}LL^{\prime}L)_{ab}$ in Equation (A.5),

$\displaystyle\begin{split}\text{tr}(L^{\prime}LL^{\prime}L)=&\sum_{i=1}^{n}\sum_{l=1}^{n}\sum_{m=1}^{n}\sum_{j=1}^{n}L_{mi}L_{ml}L_{jl}L_{ji}\\\
=&\sum_{i,j,l,m}P^{\gamma_{n}}_{mi}\mathbf{1}_{m>i}P^{\gamma_{n}}_{ml}\mathbf{1}_{m>l}P^{\gamma_{n}}_{jl}\mathbf{1}_{j>l}P^{\gamma_{n}}_{ji}\mathbf{1}_{j>i}\\\
=&\sum_{i,j,l,m}P^{\gamma_{n}}_{ij}\mathbf{1}_{i>j}P^{\gamma_{n}}_{l>j}\mathbf{1}_{l>j}P^{\gamma_{n}}_{lm}\mathbf{1}_{l>m}P^{\gamma_{n}}_{im}\mathbf{1}_{i>m}\\\
=&\sum_{m=j<i=l}P^{\gamma_{n}}_{ij}P^{\gamma_{n}}_{lj}P^{\gamma_{n}}_{lm}P^{\gamma_{n}}_{im}+\sum_{m=j<l<i}P^{\gamma_{n}}_{ij}P^{\gamma_{n}}_{lj}P^{\gamma_{n}}_{lm}P^{\gamma_{n}}_{im}\\\
&+\sum_{m=j<i<l}P^{\gamma_{n}}_{ij}P^{\gamma_{n}}_{lj}P^{\gamma_{n}}_{lm}P^{\gamma_{n}}_{im}+\sum_{j<m<i=l}P^{\gamma_{n}}_{ij}P^{\gamma_{n}}_{lj}P^{\gamma_{n}}_{lm}P^{\gamma_{n}}_{im}\\\
&+\sum_{m<j<i=l}P^{\gamma_{n}}_{ij}P^{\gamma_{n}}_{lj}P^{\gamma_{n}}_{lm}P^{\gamma_{n}}_{im}+\sum_{m<j<l<i}P^{\gamma_{n}}_{ij}P^{\gamma_{n}}_{lj}P^{\gamma_{n}}_{lm}P^{\gamma_{n}}_{im}\\\
&+\sum_{m<j<i<l}P^{\gamma_{n}}_{ij}P^{\gamma_{n}}_{lj}P^{\gamma_{n}}_{lm}P^{\gamma_{n}}_{im}+\sum_{j<m<l<i}P^{\gamma_{n}}_{ij}P^{\gamma_{n}}_{lj}P^{\gamma_{n}}_{lm}P^{\gamma_{n}}_{im}\\\
&+\sum_{j<m<i<l}P^{\gamma_{n}}_{ij}P^{\gamma_{n}}_{lj}P^{\gamma_{n}}_{lm}P^{\gamma_{n}}_{im}\\\
=&\sum_{j<i}(P^{\gamma_{n}}_{ij})^{4}+\sum_{j<l<i}(P^{\gamma_{n}}_{ij})^{2}(P^{\gamma_{n}}_{lj})^{2}+\sum_{j<i<l}(P^{\gamma_{n}}_{ij})^{2}(P^{\gamma_{n}}_{lj})^{2}\\\
&+\sum_{j<m<i}(P^{\gamma_{n}}_{ij})^{2}(P^{\gamma_{n}}_{im})^{2}+\sum_{m<j<i}(P^{\gamma_{n}}_{ij})^{2}(P^{\gamma_{n}}_{im})^{2}\\\
&+4\sum_{i<j<l<m}P^{\gamma_{n}}_{li}P^{\gamma_{n}}_{mi}P^{\gamma_{n}}_{lj}P^{\gamma_{n}}_{mj}\\\
=&\sum_{j<i}(P^{\gamma_{n}}_{ij})^{4}+2\sum_{i<j<l}(P^{\gamma_{n}}_{li})^{2}(P^{\gamma_{n}}_{ji})^{2}\\\
&+\sum_{i<j<l}(P^{\gamma_{n}}_{li})^{2}(P^{\gamma_{n}}_{lj})^{2}+4\sum_{i<j<l<m}P^{\gamma_{n}}_{li}P^{\gamma_{n}}_{mi}P^{\gamma_{n}}_{lj}P^{\gamma_{n}}_{mj}\\\
=&\sum_{i<j}(P^{\gamma_{n}}_{ij})^{4}+2\sum_{i<j<l}\left((P^{\gamma_{n}}_{ij})^{2}(P^{\gamma_{n}}_{il})^{2}+(P^{\gamma_{n}}_{il})^{2}(P^{\gamma_{n}}_{jl})^{2}\right)\\\
&+4\sum_{i<j<l<m}P^{\gamma_{n}}_{il}P^{\gamma_{n}}_{jl}P^{\gamma_{n}}_{im}P^{\gamma_{n}}_{jm}.\end{split}$
(A.9)

The third equality follows by replacing the indices in the summation according
to the dictionary $\\{i\mathrel{\mathop{\mathchar
58\relax}}j,j\mathrel{\mathop{\mathchar
58\relax}}i,l\mathrel{\mathop{\mathchar
58\relax}}m,m\mathrel{\mathop{\mathchar 58\relax}}l\\}$. The fourth equality
follows since the product within the summation is non-zero only when both
$i>j,m$ and $l>j,m$. The fifth equality follows from replacing the indices in
the last four summations in the previous line according to the dictionaries
$\\{i\mathrel{\mathop{\mathchar 58\relax}}m,j\mathrel{\mathop{\mathchar
58\relax}}j,l\mathrel{\mathop{\mathchar
58\relax}}l,m\mathrel{\mathop{\mathchar 58\relax}}i\\}$,
$\\{i\mathrel{\mathop{\mathchar 58\relax}}l,j\mathrel{\mathop{\mathchar
58\relax}}j,l\mathrel{\mathop{\mathchar
58\relax}}m,m\mathrel{\mathop{\mathchar 58\relax}}i\\}$,
$\\{i\mathrel{\mathop{\mathchar 58\relax}}m,j\mathrel{\mathop{\mathchar
58\relax}}i,l\mathrel{\mathop{\mathchar
58\relax}}l,m\mathrel{\mathop{\mathchar 58\relax}}j\\}$,
$\\{i\mathrel{\mathop{\mathchar 58\relax}}l,j\mathrel{\mathop{\mathchar
58\relax}}i,l\mathrel{\mathop{\mathchar
58\relax}}m,m\mathrel{\mathop{\mathchar 58\relax}}j\\}$, respectively. The
sixth equality follows from replacing the indices in the second, third, fourth
and fifth summations in the previous line according to the dictionaries
$\\{i\mathrel{\mathop{\mathchar 58\relax}}l,j\mathrel{\mathop{\mathchar
58\relax}}i,l\mathrel{\mathop{\mathchar 58\relax}}j\\}$,
$\\{i\mathrel{\mathop{\mathchar 58\relax}}j,j\mathrel{\mathop{\mathchar
58\relax}}i,l\mathrel{\mathop{\mathchar 58\relax}}l\\}$,
$\\{i\mathrel{\mathop{\mathchar 58\relax}}l,j\mathrel{\mathop{\mathchar
58\relax}}i,m\mathrel{\mathop{\mathchar 58\relax}}j\\}$,
$\\{i\mathrel{\mathop{\mathchar 58\relax}}l,j\mathrel{\mathop{\mathchar
58\relax}}j,m\mathrel{\mathop{\mathchar 58\relax}}i\\}$, respectively. The
last equality follows by the symmetry of $P^{\gamma_{n}}$.

Let
$\mathfrak{S}\coloneqq\sum_{i<j<l<m}P^{\gamma_{n}}_{il}P^{\gamma_{n}}_{jl}P^{\gamma_{n}}_{im}P^{\gamma_{n}}_{jm}+P^{\gamma_{n}}_{ij}P^{\gamma_{n}}_{jl}P^{\gamma_{n}}_{im}P^{\gamma_{n}}_{lm}+P^{\gamma_{n}}_{ij}P^{\gamma_{n}}_{il}P^{\gamma_{n}}_{jm}P^{\gamma_{n}}_{lm}$.
Substituting the expressions in Equations (A.6)–(A.9) into Equation (A.4)
yields

$\displaystyle\text{tr}((P^{\gamma_{n}}-G)^{4})=$ $\displaystyle
8\sum_{i<j<l<m}P^{\gamma_{n}}_{ij}P^{\gamma_{n}}_{jl}P^{\gamma_{n}}_{im}P^{\gamma_{n}}_{lm}$
$\displaystyle+4\left(\sum_{i<j<l}(P^{\gamma_{n}}_{ij})^{2}(P^{\gamma_{n}}_{jl})^{2}+2\sum_{i<j<l<m}P^{\gamma_{n}}_{ij}P^{\gamma_{n}}_{il}P^{\gamma_{n}}_{jm}P^{\gamma_{n}}_{lm}\right)$
$\displaystyle+2\left(\sum_{i<j}(P^{\gamma_{n}}_{ij})^{4}+2\sum_{i<j<l}\left((P^{\gamma_{n}}_{ij})^{2}(P^{\gamma_{n}}_{il})^{2}+(P^{\gamma_{n}}_{il})^{2}(P^{\gamma_{n}}_{jl})^{2}\right)\right.$
$\displaystyle\text{\phantom{ ddddd
}}+\left.4\sum_{i<j<l<m}P^{\gamma_{n}}_{il}P^{\gamma_{n}}_{jl}P^{\gamma_{n}}_{im}P^{\gamma_{n}}_{jm}\right)$
$\displaystyle=$ $\displaystyle 2\sum_{i<j}(P^{\gamma_{n}}_{ij})^{4}$
$\displaystyle+4\sum_{i<j<l}\left((P^{\gamma_{n}}_{ij})^{2}(P^{\gamma_{n}}_{jl})^{2}+(P^{\gamma_{n}}_{il})^{2}(P^{\gamma_{n}}_{jl})^{2}+(P^{\gamma_{n}}_{ij})^{2}(P^{\gamma_{n}}_{il})^{2}\right)$
$\displaystyle+8\mathfrak{S}.$

Next, by the triangle inequality and Lemma 1 (vi),

$\displaystyle|\mathfrak{S}|\leq$
$\displaystyle\frac{1}{4}\sum_{i<j}(P^{\gamma_{n}}_{ij})^{4}$ (A.10)
$\displaystyle+\frac{1}{2}\sum_{i<j<l}\left((P^{\gamma_{n}}_{ij})^{2}(P^{\gamma_{n}}_{jl})^{2}+(P^{\gamma_{n}}_{il})^{2}(P^{\gamma_{n}}_{jl})^{2}+(P^{\gamma_{n}}_{ij})^{2}(P^{\gamma_{n}}_{il})^{2}\right)$
$\displaystyle+\frac{1}{8}\text{tr}((P^{\gamma_{n}}-G)^{4})$
$\displaystyle\leq$ $\displaystyle Cr_{n}.$

Take $\\{u_{i}\\}$ to be a sequence of i.i.d. mean-zero and unit variance
random variables. Define

$\displaystyle\Delta_{1}\coloneqq\sum_{i<j<l}\left(P^{\gamma_{n}}_{ij}P^{\gamma_{n}}_{il}u_{j}u_{l}+P^{\gamma_{n}}_{ij}P^{\gamma_{n}}_{jl}u_{i}u_{l}+P^{\gamma_{n}}_{il}P^{\gamma_{n}}_{jl}u_{i}u_{j}\right),$
$\displaystyle\Delta_{2}\coloneqq\sum_{i<j<l}\left(P^{\gamma_{n}}_{ij}P^{\gamma_{n}}_{il}u_{j}u_{l}+P^{\gamma_{n}}_{ij}P^{\gamma_{n}}_{jl}u_{i}u_{l}\right),\Delta_{3}\coloneqq\sum_{i<j<l}\left(P^{\gamma_{n}}_{il}P^{\gamma_{n}}_{jl}u_{i}u_{j}\right).$

Then by Lemma 1 (vi),

$\displaystyle\mathbb{E}[\Delta_{3}^{2}]$
$\displaystyle=\sum_{i<j<l}(P^{\gamma_{n}}_{il})^{2}(P^{\gamma_{n}}_{jl})^{2}+2\sum_{i<j<l<m}P^{\gamma_{n}}_{il}P^{\gamma_{n}}_{jl}P^{\gamma_{n}}_{im}P^{\gamma_{n}}_{jm}$
$\displaystyle\leq
r_{n}+2\sum_{i<j<l<m}P^{\gamma_{n}}_{il}P^{\gamma_{n}}_{jl}P^{\gamma_{n}}_{im}P^{\gamma_{n}}_{jm}.$

Furthermore,

$\mathbb{E}[\Delta_{2}\Delta_{3}]=\sum_{i<j<l<m}P^{\gamma_{n}}_{ij}P^{\gamma_{n}}_{il}P^{\gamma_{n}}_{jm}P^{\gamma_{n}}_{lm}+\sum_{i<j<l<m}P^{\gamma_{n}}_{ij}P^{\gamma_{n}}_{jl}P^{\gamma_{n}}_{im}P^{\gamma_{n}}_{lm},$

and

$\displaystyle\mathbb{E}[\Delta_{2}^{2}]=$
$\displaystyle\sum_{\\{i,l\\}<j<k}P^{\gamma_{n}}_{ij}P^{\gamma_{n}}_{il}P^{\gamma_{n}}_{mj}P^{\gamma_{n}}_{ml}+\sum_{i<\\{j,l\\}<m}P^{\gamma_{n}}_{ij}P^{\gamma_{n}}_{jm}P^{\gamma_{n}}_{il}P^{\gamma_{n}}_{lm}$
$\displaystyle+\sum_{i<j<l<m}P^{\gamma_{n}}_{ij}P^{\gamma_{n}}_{im}P^{\gamma_{n}}_{jl}P^{\gamma_{n}}_{lm}+\sum_{l<i<j<m}P^{\gamma_{n}}_{ij}P^{\gamma_{n}}_{jm}P^{\gamma_{n}}_{li}P^{\gamma_{n}}_{lm}$
$\displaystyle=$
$\displaystyle\sum_{i<j<l}(P^{\gamma_{n}}_{ij})^{2}(P^{\gamma_{n}}_{il})^{2}+\sum_{i<j<l}(P^{\gamma_{n}}_{ij})^{2}(P^{\gamma_{n}}_{jl})^{2}+2\sum_{i<m<j<l}P^{\gamma_{n}}_{ij}P^{\gamma_{n}}_{il}P^{\gamma_{n}}_{mj}P^{\gamma_{n}}_{ml}$
$\displaystyle+2\sum_{i<j<l<m}P^{\gamma_{n}}_{ij}P^{\gamma_{n}}_{jm}P^{\gamma_{n}}_{il}P^{\gamma_{n}}_{lm}$
$\displaystyle+\sum_{i<j<l<m}P^{\gamma_{n}}_{ij}P^{\gamma_{n}}_{im}P^{\gamma_{n}}_{jl}P^{\gamma_{n}}_{lm}+\sum_{i<j<l<m}P^{\gamma_{n}}_{jl}P^{\gamma_{n}}_{lm}P^{\gamma_{n}}_{ij}P^{\gamma_{n}}_{im}$
$\displaystyle=$
$\displaystyle\sum_{i<j<l}(P^{\gamma_{n}}_{ij})^{2}(P^{\gamma_{n}}_{il})^{2}+\sum_{i<j<l}(P^{\gamma_{n}}_{ij})^{2}(P^{\gamma_{n}}_{jl})^{2}+2\mathfrak{S}$
$\displaystyle\leq Cr_{n},$

where the last inequality follows from Lemma 1 (vi) and Equation (A.10). Since
$\Delta_{1}=\Delta_{2}+\Delta_{3}$,
$\mathbb{E}[\Delta_{1}^{2}]=\mathbb{E}[\Delta_{2}^{2}]+\mathbb{E}[\Delta_{3}^{2}]+2\mathbb{E}[\Delta_{2}\Delta_{3}]\leq
Cr_{n}+2\mathfrak{S}\leq Cr_{n}$. Hence by the triangle inequality and the
expression for $\mathbb{E}[\Delta_{3}^{2}]$,

$\displaystyle\left|\sum_{i<j<l<m}P^{\gamma_{n}}_{il}P^{\gamma_{n}}_{jl}P^{\gamma_{n}}_{im}P^{\gamma_{n}}_{jm}\right|$
$\displaystyle\leq C\left(\mathbb{E}[\Delta_{3}^{2}]+r_{n}\right)\leq
C\left(\mathbb{E}[(\Delta_{1}-\Delta_{2})^{2}]+r_{n}\right)$
$\displaystyle\leq
C(\mathbb{E}[\Delta_{1}^{2}]+\mathbb{E}[\Delta_{2}^{2}]+r_{n})$
$\displaystyle\leq Cr_{n}.$

∎

######  Lemma 3.

Let $\gamma_{-}>0$ and
$\Gamma_{n}=\Gamma(\gamma_{-})\coloneqq\\{\gamma_{n}\in\Re\mathrel{\mathop{\mathchar
58\relax}}\gamma_{n}\geq 0\text{ if }r_{n}=k_{n},\text{ and
}\gamma_{n}\geq\gamma_{-}>0\text{ if }r_{n}<k_{n}\\}$. Then if
$P^{\gamma_{n}}$ is a non-diagonal matrix,101010We note that Assumption 3
excludes the case where $P^{\gamma_{n}}$ is diagonal, so that $\gamma^{*}_{n}$
exists under the assumptions made in this paper.

$\gamma^{*}_{n}\coloneqq\max\underset{\gamma_{n}\in\Gamma_{n}}{\arg\max}\sum_{i=1}^{n}\sum_{j\neq
i}(P^{\gamma_{n}}_{ij})^{2}\in[0,\infty)$

exists.

###### Proof.

Notice that

$\displaystyle\sum_{i=1}^{n}\sum_{j\neq i}(P^{\gamma_{n}}_{ij})^{2}$
$\displaystyle=\sum_{i=1}^{n}\sum_{j=1}^{n}(P^{\gamma_{n}}_{ij})^{2}-\sum_{i=1}^{n}(P^{\gamma_{n}}_{ii})^{2}$
$\displaystyle=\sum_{i=1}^{n}(P^{\gamma_{n}})^{2}_{ii}-\sum_{i=1}^{n}(P^{\gamma_{n}}_{ii})^{2}$
$\displaystyle=\sum_{l=1}^{r_{n}}\left(\frac{D_{ll}^{2}}{D_{ll}^{2}+\gamma_{n}}\right)^{2}-\sum_{i=1}^{n}\left(\sum_{l=1}^{r_{n}}\frac{D_{ll}^{2}}{D_{ll}^{2}+\gamma_{n}}U^{2}_{il}\right)^{2},$

where the third equality follows from Lemma 1 (iv) and Equation (A.2). Since

$\underset{\gamma_{n}\to\infty}{\text{ lim
}}\frac{D^{2}_{ll}}{D^{2}_{ll}+\gamma_{n}}=0,$

and $U^{2}_{il}\leq 1$ for $l=1,\ldots,r_{n}$ and $i=1,\ldots,n$, it follows
that

$\displaystyle\underset{\gamma_{n}\to\infty}{\text{ lim
}}\sum_{i=1}^{n}\sum_{j\neq i}(P^{\gamma_{n}}_{ij})^{2}$ $\displaystyle=0.$

Hence, the maximum is not attained for arbitrarily large $\gamma_{n}$. Since
$P^{\gamma_{n}}$ is a non-diagonal matrix by assumption,
$\sum_{i=1}^{n}\sum_{j\neq i}(P^{\gamma_{n}}_{ij})^{2}$ is strictly positive.
This leaves a compact set over which the non-zero $\sum_{i=1}^{n}\sum_{j\neq
i}(P^{\gamma_{n}}_{ij})^{2}$ is maximised, such that a maximiser exists.

∎

######  Lemma 4.

Under Assumptions 1, 2 and the null hypothesis in Equation (2),
$|\hat{\Phi}_{\gamma_{n}}-\Phi_{\gamma_{n}}|\overset{p}{\to}0$.

###### Proof.

Defining $\eta_{i}\mathrel{\mathop{\mathchar
58\relax}}=\varepsilon_{i}^{2}-\mathbb{E}[\varepsilon^{2}_{i}]$, one can write

$\hat{\Phi}_{\gamma_{n}}-\Phi_{\gamma_{n}}=\frac{2}{r_{n}}\sum_{i=1}^{n}\sum_{j\neq
i}(P_{ij}^{\gamma_{n}})^{2}\left(\eta_{i}\eta_{j}+\mathbb{E}[\varepsilon^{2}_{j}]\eta_{i}+\mathbb{E}[\varepsilon^{2}_{i}]\eta_{j}\right),$

and it follows that

$\displaystyle\left|\hat{\Phi}_{\gamma_{n}}-\Phi_{\gamma_{n}}\right|\leq$
$\displaystyle\frac{2}{r_{n}}\left|\sum_{i=1}^{n}\sum_{j\neq
i}(P_{ij}^{\gamma_{n}})^{2}\eta_{i}\eta_{j}\right|+\frac{2}{r_{n}}\left|\sum_{i=1}^{n}\sum_{j\neq
i}(P_{ij}^{\gamma_{n}})^{2}\mathbb{E}[\varepsilon^{2}_{j}]\eta_{i}\right|$
(A.11) $\displaystyle+\frac{2}{r_{n}}\left|\sum_{i=1}^{n}\sum_{j\neq
i}^{n}(P_{ij}^{\gamma_{n}})^{2}\mathbb{E}[\varepsilon^{2}_{i}]\eta_{j}\right|$
$\displaystyle\equiv$ $\displaystyle A_{1}+A_{2}+A_{3}.$

Consider each of $\mathbb{E}[A_{1}^{2}]$, $\mathbb{E}[A_{2}^{2}]$, and
$\mathbb{E}[A_{3}^{2}]$ in turn.

$\displaystyle\mathbb{E}[A_{1}^{2}]=$
$\displaystyle\frac{4}{r_{n}^{2}}\sum_{i=1}^{n}\sum_{j\neq
i}\sum_{h=1}^{n}\sum_{g\neq
h}(P_{ij}^{\gamma_{n}})^{2}(P_{hg}^{\gamma_{n}})^{2}\mathbb{E}[\eta_{i}\eta_{j}\eta_{h}\eta_{g}]$
$\displaystyle=$ $\displaystyle\frac{4}{r_{n}^{2}}\sum_{i=1}^{n}\sum_{j\neq
i}\sum_{h\neq i}\sum_{g\neq
h}(P_{ij}^{\gamma_{n}})^{2}(P_{hg}^{\gamma_{n}})^{2}\mathbb{E}[\eta_{i}\eta_{j}\eta_{h}\eta_{g}]$
$\displaystyle+\frac{4}{r_{n}^{2}}\sum_{i=1}^{n}\sum_{j\neq i}\sum_{g\neq
i}(P_{ij}^{\gamma_{n}})^{2}(P_{ig}^{\gamma_{n}})^{2}\mathbb{E}[\eta_{i}^{2}\eta_{j}\eta_{g}]$
$\displaystyle=$ $\displaystyle\frac{4}{r_{n}^{2}}\sum_{i=1}^{n}\sum_{j\neq
i}\sum_{h\neq
i}\sum_{g\notin\\{h,i\\}}(P_{ij}^{\gamma_{n}})^{2}(P_{hg}^{\gamma_{n}})^{2}\mathbb{E}[\eta_{i}]\mathbb{E}[\eta_{j}\eta_{h}\eta_{g}]$
$\displaystyle+$ $\displaystyle\frac{4}{r_{n}^{2}}\sum_{i=1}^{n}\sum_{j\neq
i}\sum_{h\neq
i}(P_{ij}^{\gamma_{n}})^{2}(P_{ih}^{\gamma_{n}})^{2}\mathbb{E}[\eta_{i}^{2}\eta_{j}\eta_{h}]$
$\displaystyle+\frac{4}{r_{n}^{2}}\sum_{i=1}^{n}\sum_{j\neq i}\sum_{g\neq
i}(P_{ij}^{\gamma_{n}})^{2}(P_{ig}^{\gamma_{n}})^{2}\mathbb{E}[\eta_{i}^{2}\eta_{j}\eta_{g}]$
$\displaystyle=$ $\displaystyle\frac{8}{r_{n}^{2}}\sum_{i=1}^{n}\sum_{j\neq
i}\sum_{h\neq
i}(P_{ij}^{\gamma_{n}})^{2}(P_{ih}^{\gamma_{n}})^{2}\mathbb{E}[\eta_{i}^{2}\eta_{j}\eta_{h}]$
$\displaystyle=$ $\displaystyle\frac{8}{r_{n}^{2}}\sum_{i=1}^{n}\sum_{j\neq
i}\sum_{h\notin\\{i,j\\}}(P_{ij}^{\gamma_{n}})^{2}(P_{ih}^{\gamma_{n}})^{2}\mathbb{E}[\eta_{i}^{2}]\mathbb{E}[\eta_{j}]\mathbb{E}[\eta_{h}]$
$\displaystyle+\frac{8}{r_{n}^{2}}\sum_{i=1}^{n}\sum_{j\neq
i}(P_{ij}^{\gamma_{n}})^{4}\mathbb{E}[\eta_{i}^{2}]\mathbb{E}[\eta_{j}^{2}]$
$\displaystyle=$ $\displaystyle\frac{8}{r_{n}^{2}}\sum_{i=1}^{n}\sum_{j\neq
i}(P_{ij}^{\gamma_{n}})^{4}\mathbb{E}[\eta_{i}^{2}]\mathbb{E}[\eta_{j}^{2}]$
$\displaystyle\leq$ $\displaystyle\frac{C}{r_{n}^{2}}\sum_{i=1}^{n}\sum_{j\neq
i}(P_{ij}^{\gamma_{n}})^{4}$ $\displaystyle\leq$
$\displaystyle\frac{C}{r_{n}},$

The fourth equality follows from $\mathbb{E}[\eta_{i}]=0$ and the symmetry of
$P^{\gamma_{n}}$. The sixth equality follows from $\mathbb{E}[\eta_{j}]=0$.
The first inequality follows from Assumption 1, which implies
$\text{sup}_{i\in\mathbb{N}}\mathbb{E}[\eta_{i}^{2}]<\infty$. The second
inequality follows from Lemma 1 (vi). Next, concerning $\mathbb{E}[A_{2}^{2}]$
one has

$\displaystyle\mathbb{E}[A_{2}^{2}]=$
$\displaystyle\frac{4}{r_{n}^{2}}\sum_{i=1}^{n}\sum_{j\neq
i}\sum_{h=1}^{n}\sum_{g\neq
h}(P_{ij}^{\gamma_{n}})^{2}(P_{hg}^{\gamma_{n}})^{2}\mathbb{E}[\varepsilon^{2}_{j}]\mathbb{E}[\varepsilon^{2}_{g}]\mathbb{E}[\eta_{i}\eta_{h}]$
$\displaystyle=$ $\displaystyle\frac{4}{r_{n}^{2}}\sum_{i=1}^{n}\sum_{j\neq
i}\sum_{h\neq i}\sum_{g\neq
h}(P_{ij}^{\gamma_{n}})^{2}(P_{hg}^{\gamma_{n}})^{2}\mathbb{E}[\varepsilon^{2}_{j}]\mathbb{E}[\varepsilon^{2}_{g}]\mathbb{E}[\eta_{i}]\mathbb{E}[\eta_{h}]$
$\displaystyle+$ $\displaystyle\frac{4}{r_{n}^{2}}\sum_{i=1}^{n}\sum_{j\neq
i}\sum_{g\neq
i}(P_{ij}^{\gamma_{n}})^{2}(P_{ig}^{\gamma_{n}})^{2}\mathbb{E}[\varepsilon^{2}_{j}]\mathbb{E}[\varepsilon^{2}_{g}]\mathbb{E}[\eta_{i}^{2}]$
$\displaystyle=$ $\displaystyle\frac{4}{r_{n}^{2}}\sum_{i=1}^{n}\sum_{j\neq
i}\sum_{g\neq
i}(P_{ij}^{\gamma_{n}})^{2}(P_{ig}^{\gamma_{n}})^{2}\mathbb{E}[\varepsilon^{2}_{j}]\mathbb{E}[\varepsilon^{2}_{g}]\mathbb{E}[\eta_{i}^{2}]$
$\displaystyle\leq$ $\displaystyle\frac{C}{r_{n}^{2}}\sum_{i=1}^{n}\sum_{j\neq
i}\sum_{g\neq i}(P_{ij}^{\gamma_{n}})^{2}(P_{ig}^{\gamma_{n}})^{2}$
$\displaystyle\leq$ $\displaystyle\frac{C}{r_{n}},$

by the same reasoning that led to the penultimate display. Finally, similar
arguments imply that $\mathbb{E}[A_{3}^{2}]\leq\frac{C}{r_{n}}$ such that by
Markov’s inequality and Equation (A.11) it follows that
$\hat{\Phi}_{\gamma_{n}}-\Phi_{\gamma_{n}}=O_{p}({r^{-1/2}_{n}})$. Thus, by
Assumption 2, $|\hat{\Phi}_{\gamma_{n}}-\Phi_{\gamma_{n}}|\overset{p}{\to}0$.
∎

###  A.2 Proof of Proposition 1

By the assumptions in the statement, $P$ has rank $k_{n}$ and $P_{ii}\leq
1-\delta$, which implies that $k_{n}<n$. Notice that

$\displaystyle\frac{1}{k_{n}}\sum_{i=1}^{n}\sum_{j\neq i}(P_{ij})^{2}$
$\displaystyle=\frac{1}{k_{n}}\left(\sum_{i=1}^{n}\sum_{j=1}^{n}(P_{ij})^{2}-\sum_{i=1}^{n}(P_{ii})^{2}\right)=\frac{1}{k_{n}}\left(\sum_{i=1}^{n}(P^{2})_{ii}-\sum_{i=1}^{n}(P_{ii})^{2}\right)$
$\displaystyle\scriptsize=\frac{1}{k_{n}}\left(\sum_{i=1}^{n}P_{ii}-\sum_{i=1}^{n}(P_{ii})^{2}\right)\geq\frac{1}{k_{n}}\left(\sum_{i=1}^{n}P_{ii}-(1-\delta)\sum_{i=1}^{n}P_{ii}\right)=\delta,$

where the third equality follows from the idempotency of $P$, the first
inequality follows since $\text{max}_{i}P_{ii}\leq 1-\delta$ by assumption of
the statement, and the last equality follows since $\text{tr}(P)=k_{n}$.

Next, observe that

$\displaystyle\sum_{i=1}^{n}\sum_{j\neq i}(P_{ij}^{\gamma_{n}})^{2}=$
$\displaystyle\sum_{i=1}^{n}\sum_{j\neq
i}(P_{ij})^{2}+\sum_{i=1}^{n}\sum_{j\neq
i}(P_{ij}^{\gamma_{n}})^{2}-\sum_{i=1}^{n}\sum_{j\neq i}(P_{ij})^{2}$
$\displaystyle=$ $\displaystyle\sum_{i=1}^{n}\sum_{j\neq
i}(P_{ij})^{2}+\sum_{i=1}^{n}\sum_{j=1}^{n}(P_{ij}^{\gamma_{n}})^{2}-\sum_{i=1}^{n}\sum_{j=1}^{n}(P_{ij})^{2}$
$\displaystyle+\sum_{i=1}^{n}(P_{ii})^{2}-\sum_{i=1}^{n}(P_{ii}^{\gamma_{n}})^{2}.$

Furthermore, by Equation (A.1) $P_{ii}\geq P^{\gamma_{n}}_{ii}\geq 0$ for all
$\gamma_{n}\geq 0$ (see also the last equality in Equation (A.2)) such that
$\sum_{i=1}^{n}(P_{ii})^{2}-\sum_{i=1}^{n}(P_{ii}^{\gamma_{n}})^{2}\geq 0$. It
follows that

$\displaystyle\sum_{i=1}^{n}\sum_{j\neq
i}(P_{ij}^{\gamma_{n}})^{2}\geq\sum_{i=1}^{n}\sum_{j\neq
i}(P_{ij})^{2}-(||P||_{F}^{2}-||P^{\gamma_{n}}||_{F}^{2}).$ (A.12)

In addition,

$\displaystyle||P||_{F}^{2}-||P^{\gamma_{n}}||_{F}^{2}$
$\displaystyle=(||P||_{F}+||P^{\gamma_{n}}||_{F})(||P||_{F}-||P^{\gamma_{n}}||_{F})$
(A.13) $\displaystyle\leq 2||P||_{F}(||P||_{F}-||P^{\gamma_{n}}||_{F})$
$\displaystyle\leq 2\sqrt{k_{n}}||P-P^{\gamma_{n}}||_{F},$

the inequalities following from
$||P^{\gamma_{n}}||_{F}\leq||P||_{F}=\sqrt{k_{n}}$ (using Equation (A.1)
compare $||P^{\gamma_{n}}||_{F}$ and $||P||_{F}$). Using Equation (A.1) once
more yields that

$\displaystyle P-P^{\gamma_{n}}=U(\check{D}-\tilde{D})U^{\prime},$

where $\check{D}$ is the $n\times n$ diagonal matrix with upper left block
being the $k_{n}\times k_{n}$ identity matrix. Therefore,

$\displaystyle||P-P^{\gamma_{n}}||_{F}^{2}$
$\displaystyle=\text{tr}([P-P^{\gamma_{n}}]^{\prime}[P-P^{\gamma_{n}}])=\text{tr}(U(\check{D}-\tilde{D})^{2}U^{\prime})$
(A.14)
$\displaystyle=\sum_{i=1}^{k_{n}}\mathinner{\biggl{(}1-\frac{D_{ii}^{2}}{D_{ii}^{2}+\gamma_{n}}\biggr{)}}^{2}.$

Combining Equations (A.12)–(A.14) implies that

$\displaystyle\sum_{i=1}^{n}\sum_{j\neq
i}(P_{ij}^{\gamma_{n}})^{2}\geq\sum_{i=1}^{n}\sum_{j\neq
i}(P_{ij})^{2}-2\sqrt{k_{n}}\sqrt{\sum_{i=1}^{k_{n}}\mathinner{\biggl{(}1-\frac{D_{ii}^{2}}{D_{ii}^{2}+\gamma_{n}}\biggr{)}}^{2}},$

such that

$\displaystyle\frac{1}{k_{n}}\sum_{i=1}^{n}\sum_{j\neq
i}(P_{ij}^{\gamma_{n}})^{2}\geq\frac{1}{k_{n}}\sum_{i=1}^{n}\sum_{j\neq
i}(P_{ij})^{2}-2\sqrt{\frac{1}{k_{n}}\sum_{i=1}^{k_{n}}\mathinner{\biggl{(}1-\frac{D_{ii}^{2}}{D_{ii}^{2}+\gamma_{n}}\biggr{)}}^{2}}.$

Since

$\displaystyle\frac{1}{k_{n}}\sum_{i=1}^{n}\sum_{j\neq
i}(P_{ij})^{2}\leq\frac{1}{k_{n}}\sum_{i=1}^{n}\sum_{j=1}^{n}(P_{ij})^{2}=\frac{1}{k_{n}}\text{tr}(P^{2})=\frac{1}{k_{n}}\text{tr}(P)=1,$

and
$\gamma_{n}\mapsto\frac{1}{k_{n}}\sum_{i=1}^{k_{n}}\mathinner{\bigl{(}1-\frac{D_{ii}^{2}}{D_{ii}^{2}+\gamma_{n}}\bigr{)}}^{2}$
is continuous, 0 at 0, and tends to $1$ as $\gamma_{n}\to\infty$, there exists
a $\gamma_{n}^{0}>0$ such that

$\displaystyle
2\sqrt{\frac{1}{k_{n}}\sum_{i=1}^{k_{n}}\mathinner{\biggl{(}1-\frac{D_{ii}^{2}}{D_{ii}^{2}+\gamma_{n}^{0}}\biggr{)}}^{2}}=\frac{1}{2k_{n}}\sum_{i=1}^{n}\sum_{j\neq
i}(P_{ij})^{2}$

ensuring that

$\displaystyle\frac{1}{k_{n}}\sum_{i=1}^{n}\sum_{j\neq
i}(P_{ij}^{\gamma_{n}^{0}})^{2}\geq\frac{1}{2k_{n}}\sum_{i=1}^{n}\sum_{j\neq
i}(P_{ij})^{2}\geq\delta/2.$ (A.15)

Finally, since $\gamma^{*}_{n}$ is the penalty parameter that maximises
$\gamma_{n}\mapsto\sum_{i=1}^{n}\sum_{j\neq i}(P_{ij}^{\gamma_{n}})^{2}$,

$\displaystyle\sum_{i=1}^{n}\sum_{j\neq
i}(P_{ij}^{\gamma^{*}_{n}})^{2}\geq\sum_{i=1}^{n}\sum_{j\neq
i}(P_{ij}^{\gamma_{n}^{0}})^{2}\geq\frac{k\delta}{2},$

so that Assumption 3 is satisfied for $\omega=1$. ∎

###  A.3 Proof of Theorem 1

We begin by showing that

$\displaystyle{RJAR}^{*}_{\gamma_{n}}$
$\displaystyle\coloneqq\frac{1}{\sqrt{r_{n}}\sqrt{{\Phi}_{\gamma_{n}}}}\sum_{i=1}^{n}\sum_{j\neq
i}P_{ij}^{\gamma_{n}}\varepsilon_{i}\varepsilon_{j}$ (A.16)

converges in distribution to a standard normal for all sequences of
$\gamma_{n}$ that satisfy Assumption 3.111111We do not directly invoke Lemma 2
in [18] because it would require strengthening the assumptions used in this
paper. In particular, we would have to replace the condition in Assumption 3
that $\sum_{i=1}^{n}\sum_{j\neq i}(P^{\gamma_{n}}_{ij})^{2}\geq
cr_{n}^{\omega}$ with the stronger condition that $\sum_{i=1}^{n}\sum_{j\neq
i}(P^{\gamma_{n}}_{ij})^{2}\geq ck_{n}$. To this end, define
$\mathcal{U}_{n}\mathrel{\mathop{\mathchar
58\relax}}=2\sum_{i=2}^{n}\sum_{j=1}^{i-1}P^{\gamma_{n}}_{ij}\varepsilon_{i}\varepsilon_{j}$,
$s_{n}^{2}\mathrel{\mathop{\mathchar
58\relax}}=\mathbb{E}[\mathcal{U}_{n}^{2}]$ and note that by the symmetry of
$P^{\gamma_{n}}$, $RJAR^{*}_{\gamma_{n}}$ can be written as

$RJAR^{*}_{\gamma_{n}}=s_{n}^{-1}\mathcal{U}_{n}.$

We proceed by establishing that
$s_{n}^{-1}\mathcal{U}_{n}\overset{d}{\to}\mathcal{N}[0,1]$ as $n\to\infty$.
This, in turn, follows from [17, Corollary 3.1 ]) upon verifying that i) for
all $\epsilon>0$

$s_{n}^{-2}\sum_{i=2}^{n}\mathbb{E}[Y^{2}_{ni}I(|Y_{ni}|>\epsilon s_{n})]\to
0\qquad\text{as }n\to\infty,$ (A.17)

where
$Y_{ni}=2\sum_{j=1}^{i-1}P^{\gamma_{n}}_{ij}\varepsilon_{i}\varepsilon_{j}$,
and ii)

$s_{n}^{-2}\mathcal{V}_{n}^{2}\overset{p}{\to}1\qquad\text{as }n\to\infty,$
(A.18)

where
$\mathcal{V}_{n}^{2}=\sum_{i=2}^{n}\mathbb{E}[Y^{2}_{ni}|\varepsilon_{1},\dots,\varepsilon_{i-1}]$.
Consider first the condition in Equation (A.17) and write

$\mathbb{E}[Y^{2}_{ni}]=4\sum_{j=1}^{i-1}\sum_{h=1}^{i-1}P_{ij}^{\gamma_{n}}P_{ih}^{\gamma_{n}}\mathbb{E}[\varepsilon_{i}^{2}\varepsilon_{j}\varepsilon_{h}]=4\sum_{j=1}^{i-1}(P_{ij}^{\gamma_{n}})^{2}\mathbb{E}[\varepsilon_{i}^{2}]\mathbb{E}[\varepsilon_{j}^{2}],$

so that, upon using that $\mathcal{U}_{n}=\sum_{i=2}^{n}Y_{ni}$ and
$\mathbb{E}[Y_{ni}Y_{nj}]=0$ for $i\neq j$, one gets

$s_{n}^{2}=\sum_{i=2}^{n}\mathbb{E}[Y^{2}_{ni}]=4\sum_{i=2}^{n}\sum_{j=1}^{i-1}(P_{ij}^{\gamma_{n}})^{2}\mathbb{E}[\varepsilon_{i}^{2}]\mathbb{E}[\varepsilon_{j}^{2}].$
(A.19)

Furthermore,

$\displaystyle\mathbb{E}[Y_{ni}^{4}]=$ $\displaystyle
16\sum_{j=1}^{i-1}\sum_{h=1}^{i-1}\sum_{m=1}^{i-1}\sum_{l=1}^{i-1}P_{ij}^{\gamma_{n}}P_{ih}^{\gamma_{n}}P_{im}^{\gamma_{n}}P_{il}^{\gamma_{n}}\mathbb{E}[\varepsilon_{i}^{4}\varepsilon_{j}\varepsilon_{h}\varepsilon_{m}\varepsilon_{l}]$
$\displaystyle=$ $\displaystyle
16\sum_{j=1}^{i-1}(P_{ij}^{\gamma_{n}})^{4}\mathbb{E}[\varepsilon_{i}^{4}]\mathbb{E}[\varepsilon_{j}^{4}]+48\sum_{j=1}^{i-1}\sum_{h\neq
j}(P_{ij}^{\gamma_{n}})^{2}(P_{ih}^{\gamma_{n}})^{2}\mathbb{E}[\varepsilon_{i}^{4}]\mathbb{E}[\varepsilon_{j}^{2}]\mathbb{E}[\varepsilon_{h}^{2}],$

so that

$\displaystyle\sum_{i=2}^{n}\mathbb{E}[Y_{ni}^{4}]=$ $\displaystyle
16\sum_{i=2}^{n}\sum_{j=1}^{i-1}(P_{ij}^{\gamma_{n}})^{4}\mathbb{E}[\varepsilon_{i}^{4}]\mathbb{E}[\varepsilon_{j}^{4}]$
$\displaystyle+48\sum_{i=2}^{n}\sum_{j=1}^{i-1}\sum_{h\neq
j}(P_{ij}^{\gamma_{n}})^{2}(P_{ih}^{\gamma_{n}})^{2}\mathbb{E}[\varepsilon_{i}^{4}]\mathbb{E}[\varepsilon_{j}^{2}]\mathbb{E}[\varepsilon_{h}^{2}].$

It follows that

$\displaystyle\frac{\sum_{i=2}^{n}\mathbb{E}[Y_{ni}^{4}]}{s^{4}_{n}}$
$\displaystyle\leq
C\frac{\sum_{i=2}^{n}\sum_{j=1}^{i-1}(P_{ij}^{\gamma_{n}})^{4}+\sum_{i=2}^{n}\sum_{j=1}^{i-1}\sum_{h\neq
j}(P_{ij}^{\gamma_{n}})^{2}(P_{ih}^{\gamma_{n}})^{2}}{\left(\sum_{i=1}^{n}\sum_{j\neq
i}(P^{\gamma_{n}}_{ij})^{2}\right)^{2}}$ (A.20) $\displaystyle\leq
C\frac{\sum_{i=1}^{n}\sum_{j=1}^{n}(P_{ij}^{\gamma_{n}})^{2}+\sum_{i=1}^{n}\sum_{j=1}^{n}(P_{ij}^{\gamma_{n}})^{2}\sum_{h=1}^{n}(P_{ih}^{\gamma_{n}})^{2}}{\left(\sum_{i=1}^{n}\sum_{j\neq
i}(P^{\gamma_{n}}_{ij})^{2}\right)^{2}}$
$\displaystyle=C\frac{\sum_{i=1}^{n}(P^{\gamma_{n}})^{2}_{ii}+\sum_{i=1}^{n}\left((P^{\gamma_{n}})^{2}_{ii}\right)^{2}}{\left(\sum_{i=1}^{n}\sum_{j\neq
i}(P^{\gamma_{n}}_{ij})^{2}\right)^{2}}$ $\displaystyle\leq
C\frac{\sum_{i=1}^{n}(P^{\gamma_{n}})^{2}_{ii}+\sum_{i=1}^{n}(P^{\gamma_{n}})^{2}_{ii}}{\left(\sum_{i=1}^{n}\sum_{j\neq
i}(P^{\gamma_{n}}_{ij})^{2}\right)^{2}}$ $\displaystyle\leq
C\frac{\sum_{i=1}^{n}(P^{\gamma_{n}})^{2}_{ii}}{r^{2\omega}_{n}}$
$\displaystyle\leq Cr^{1-2\omega}_{n}$ $\displaystyle\to 0,$

where the first inequality follows from Assumption 1. The second inequality
follows from Lemma 1 (v). The third inequality follows from Lemma 1 (i). The
fourth inequality follows from Assumption 3. The fifth inequality follows from
Lemma 1 (iii). The limit holds by Assumption 2 and Assumption 3. The condition
in Equation (A.17) follows from Equation (A.20).

In order to verify the convergence in Equation (A.18) it suffices to show that

$s_{n}^{-4}\mathbb{E}\left[(\mathcal{V}_{n}^{2}-s_{n}^{2})^{2}\right]=\frac{\mathbb{E}[\mathcal{V}_{n}^{4}]+s_{n}^{4}-2s_{n}^{2}\mathbb{E}[\mathcal{V}_{n}^{2}]}{s_{n}^{4}}=\frac{\mathbb{E}[\mathcal{V}_{n}^{4}]}{s_{n}^{4}}+1-2\frac{\mathbb{E}[\mathcal{V}_{n}^{2}]}{s_{n}^{2}}\to
0.$

Since
$\mathcal{V}_{n}^{2}=\sum_{i=2}^{n}\mathbb{E}[Y^{2}_{ni}|\varepsilon_{1},\dots,\varepsilon_{i-1}]$
it follows that

$\displaystyle\mathbb{E}[\mathcal{V}_{n}^{2}]=\sum_{i=2}^{n}\mathbb{E}[Y_{ni}^{2}]=s_{n}^{2},$

the last equality following from Equation (A.19). It remains to be verified
that

$\frac{\mathbb{E}[\mathcal{V}_{n}^{4}]}{s_{n}^{4}}\to 1.$ (A.21)

To this end, observe that

$\mathcal{V}_{n}^{2}=\sum_{i=2}^{n}\mathbb{E}[Y^{2}_{ni}|\varepsilon_{1},\dots,\varepsilon_{i-1}]=4\sum_{i=2}^{n}\sum_{j=1}^{i-1}\sum_{h=1}^{i-1}P_{ij}^{\gamma_{n}}P_{ih}^{\gamma_{n}}\mathbb{E}[\varepsilon_{i}^{2}]\varepsilon_{j}\varepsilon_{h},$

such that

$\mathcal{V}_{n}^{4}=16\sum_{i=2}^{n}\sum_{j=2}^{n}\sum_{h=1}^{i-1}\sum_{l=1}^{i-1}\sum_{m=1}^{j-1}\sum_{w=1}^{j-1}P^{\gamma_{n}}_{ih}P^{\gamma_{n}}_{il}P^{\gamma_{n}}_{jm}P^{\gamma_{n}}_{jw}\mathbb{E}[\varepsilon_{i}^{2}]\mathbb{E}[\varepsilon_{j}^{2}]\varepsilon_{h}\varepsilon_{l}\varepsilon_{m}\varepsilon_{w}.$

For $i\leq j$, tedious but straightforward calculations included for
completeness yield

$\displaystyle\mathbb{E}\left[\sum_{h=1}^{i-1}\sum_{l=1}^{i-1}\sum_{m=1}^{j-1}\sum_{w=1}^{j-1}P^{\gamma_{n}}_{ih}P^{\gamma_{n}}_{il}P^{\gamma_{n}}_{jm}P^{\gamma_{n}}_{jw}\varepsilon_{h}\varepsilon_{l}\varepsilon_{m}\varepsilon_{w}\right]$
$\displaystyle=$
$\displaystyle\sum_{l=1}^{i-1}\sum_{h=1}^{i-1}\sum_{m=1}^{j-1}\sum_{w\neq
m}^{j-1}P^{\gamma_{n}}_{il}P^{\gamma_{n}}_{ih}P^{\gamma_{n}}_{jm}P^{\gamma_{n}}_{jw}\mathbb{E}[\varepsilon_{l}\varepsilon_{h}\varepsilon_{m}\varepsilon_{w}]$
$\displaystyle+\sum_{l=1}^{i-1}\sum_{h\neq l}^{i-1}\sum_{m\neq
l}^{j-1}P^{\gamma_{n}}_{il}P^{\gamma_{n}}_{ih}(P^{\gamma_{n}}_{jm})^{2}\mathbb{E}[\varepsilon_{l}]\mathbb{E}[\varepsilon_{h}\varepsilon_{m}^{2}]$
$\displaystyle+\sum_{l=1}^{i-1}\sum_{h\neq
l}^{i-1}P^{\gamma_{n}}_{il}P^{\gamma_{n}}_{ih}(P^{\gamma_{n}}_{jl})^{2}\mathbb{E}[\varepsilon_{l}^{3}]\mathbb{E}[\varepsilon_{h}]+\sum_{l=1}^{i-1}\sum_{m=1}^{j-1}(P^{\gamma_{n}}_{il})^{2}(P^{\gamma_{n}}_{jm})^{2}\mathbb{E}[\varepsilon_{l}^{2}\varepsilon_{m}^{2}]$
$\displaystyle=$
$\displaystyle\sum_{l=1}^{i-1}\sum_{h=1}^{i-1}\sum_{m=1}^{j-1}\sum_{w\neq
m}^{j-1}P^{\gamma_{n}}_{il}P^{\gamma_{n}}_{ih}P^{\gamma_{n}}_{jm}P^{\gamma_{n}}_{jw}\mathbb{E}[\varepsilon_{l}\varepsilon_{h}\varepsilon_{m}\varepsilon_{w}]+\sum_{l=1}^{i-1}\sum_{m=1}^{j-1}(P^{\gamma_{n}}_{il})^{2}(P^{\gamma_{n}}_{jm})^{2}\mathbb{E}[\varepsilon_{l}^{2}\varepsilon_{m}^{2}]$
$\displaystyle=$ $\displaystyle\sum_{l=1}^{i-1}\sum_{h\neq
l}^{i-1}\sum_{m=1}^{j-1}\sum_{w\neq
m}^{j-1}P^{\gamma_{n}}_{il}P^{\gamma_{n}}_{ih}P^{\gamma_{n}}_{jm}P^{\gamma_{n}}_{jw}\mathbb{E}[\varepsilon_{l}\varepsilon_{h}\varepsilon_{m}\varepsilon_{w}]$
$\displaystyle+\sum_{l=1}^{i-1}\sum_{m\neq l}^{j-1}\sum_{w\neq
m}^{j-1}(P^{\gamma_{n}}_{il})^{2}P^{\gamma_{n}}_{jm}P^{\gamma_{n}}_{jw}\mathbb{E}[\varepsilon_{l}^{2}\varepsilon_{w}]\mathbb{E}[\varepsilon_{m}]$
$\displaystyle+\sum_{l=1}^{i-1}\sum_{w\neq
l}^{j-1}(P^{\gamma_{n}}_{il})^{2}P^{\gamma_{n}}_{jl}P^{\gamma_{n}}_{jw}\mathbb{E}[\varepsilon_{l}^{3}]\mathbb{E}[\varepsilon_{w}]+\sum_{l=1}^{i-1}\sum_{m=1}^{j-1}(P^{\gamma_{n}}_{il})^{2}(P^{\gamma_{n}}_{jm})^{2}\mathbb{E}[\varepsilon_{l}^{2}\varepsilon_{m}^{2}]$
$\displaystyle=$ $\displaystyle\sum_{l=1}^{i-1}\sum_{h\neq
l}^{i-1}\sum_{m=1}^{j-1}\sum_{w\neq
m}^{j-1}P^{\gamma_{n}}_{il}P^{\gamma_{n}}_{ih}P^{\gamma_{n}}_{jm}P^{\gamma_{n}}_{jw}\mathbb{E}[\varepsilon_{l}\varepsilon_{h}\varepsilon_{m}\varepsilon_{w}]+\sum_{l=1}^{i-1}\sum_{m=1}^{j-1}(P^{\gamma_{n}}_{il})^{2}(P^{\gamma_{n}}_{jm})^{2}\mathbb{E}[\varepsilon_{l}^{2}\varepsilon_{m}^{2}]$
$\displaystyle=$ $\displaystyle\sum_{l=1}^{i-1}\sum_{h\neq l}^{i-1}\sum_{m\neq
h}^{j-1}\sum_{w\neq
m}^{j-1}P^{\gamma_{n}}_{il}P^{\gamma_{n}}_{ih}P^{\gamma_{n}}_{jm}P^{\gamma_{n}}_{jw}\mathbb{E}[\varepsilon_{l}\varepsilon_{h}\varepsilon_{m}\varepsilon_{w}]$
$\displaystyle+\sum_{l=1}^{i-1}\sum_{h\neq
l}^{i-1}\sum_{w\notin\\{h,l\\}}^{j-1}P^{\gamma_{n}}_{il}P^{\gamma_{n}}_{ih}P^{\gamma_{n}}_{jh}P^{\gamma_{n}}_{jw}\mathbb{E}[\varepsilon_{l}]\mathbb{E}[\varepsilon_{w}]\mathbb{E}[\varepsilon_{h}^{2}]$
$\displaystyle+\sum_{l=1}^{i-1}\sum_{h\neq
l}^{i-1}P^{\gamma_{n}}_{il}P^{\gamma_{n}}_{ih}P^{\gamma_{n}}_{jh}P^{\gamma_{n}}_{jl}\mathbb{E}[\varepsilon_{l}^{2}]\mathbb{E}[\varepsilon_{h}^{2}]+\sum_{l=1}^{i-1}\sum_{m=1}^{j-1}(P^{\gamma_{n}}_{il})^{2}(P^{\gamma_{n}}_{jm})^{2}\mathbb{E}[\varepsilon_{l}^{2}\varepsilon_{m}^{2}]$
$\displaystyle=$ $\displaystyle\sum_{l=1}^{i-1}\sum_{h\neq l}^{i-1}\sum_{m\neq
h}^{j-1}\sum_{w\neq
m}^{j-1}P^{\gamma_{n}}_{il}P^{\gamma_{n}}_{ih}P^{\gamma_{n}}_{jm}P^{\gamma_{n}}_{jw}\mathbb{E}[\varepsilon_{l}\varepsilon_{h}\varepsilon_{m}\varepsilon_{w}]+\sum_{l=1}^{i-1}\sum_{h\neq
l}^{i-1}P^{\gamma_{n}}_{il}P^{\gamma_{n}}_{ih}P^{\gamma_{n}}_{jh}P^{\gamma_{n}}_{jl}\mathbb{E}[\varepsilon_{l}^{2}]\mathbb{E}[\varepsilon_{h}^{2}]$
$\displaystyle+\sum_{l=1}^{i-1}\sum_{m=1}^{j-1}(P^{\gamma_{n}}_{il})^{2}(P^{\gamma_{n}}_{jm})^{2}\mathbb{E}[\varepsilon_{l}^{2}\varepsilon_{m}^{2}]$
$\displaystyle=$ $\displaystyle\sum_{l=1}^{i-1}\sum_{h\neq l}^{i-1}\sum_{m\neq
h}^{j-1}\sum_{w\notin\\{m,h\\}}^{j-1}P^{\gamma_{n}}_{il}P^{\gamma_{n}}_{ih}P^{\gamma_{n}}_{jm}P^{\gamma_{n}}_{jw}\mathbb{E}[\varepsilon_{l}\varepsilon_{h}\varepsilon_{m}\varepsilon_{w}]$
$\displaystyle+\sum_{l=1}^{i-1}\sum_{h\neq
l}^{i-1}\sum_{m\notin\\{h,l\\}}^{j-1}P^{\gamma_{n}}_{il}P^{\gamma_{n}}_{ih}P^{\gamma_{n}}_{jm}P^{\gamma_{n}}_{jh}\mathbb{E}[\varepsilon_{l}]\mathbb{E}[\varepsilon_{m}]\mathbb{E}[\varepsilon_{h}^{2}]$
$\displaystyle+2\sum_{l=1}^{i-1}\sum_{h\neq
l}^{i-1}P^{\gamma_{n}}_{il}P^{\gamma_{n}}_{ih}P^{\gamma_{n}}_{jl}P^{\gamma_{n}}_{jh}\mathbb{E}[\varepsilon_{l}^{2}]\mathbb{E}[\varepsilon_{h}^{2}]+\sum_{l=1}^{i-1}\sum_{m=1}^{j-1}(P^{\gamma_{n}}_{il})^{2}(P^{\gamma_{n}}_{jm})^{2}\mathbb{E}[\varepsilon_{l}^{2}\varepsilon_{m}^{2}]$
$\displaystyle=$ $\displaystyle\sum_{l=1}^{i-1}\sum_{h\neq l}^{i-1}\sum_{m\neq
h}^{j-1}\sum_{w\notin\\{m,h\\}}^{j-1}P^{\gamma_{n}}_{il}P^{\gamma_{n}}_{ih}P^{\gamma_{n}}_{jm}P^{\gamma_{n}}_{jw}\mathbb{E}[\varepsilon_{l}\varepsilon_{h}\varepsilon_{m}\varepsilon_{w}]$
$\displaystyle+2\sum_{l=1}^{i-1}\sum_{h\neq
l}^{i-1}P^{\gamma_{n}}_{il}P^{\gamma_{n}}_{ih}P^{\gamma_{n}}_{jl}P^{\gamma_{n}}_{jh}\mathbb{E}[\varepsilon_{l}^{2}]\mathbb{E}[\varepsilon_{h}^{2}]+\sum_{l=1}^{i-1}\sum_{m=1}^{j-1}(P^{\gamma_{n}}_{il})^{2}(P^{\gamma_{n}}_{jm})^{2}\mathbb{E}[\varepsilon_{l}^{2}\varepsilon_{m}^{2}]$
$\displaystyle=$ $\displaystyle\sum_{l=1}^{i-1}\sum_{h\neq l}^{i-1}\sum_{m\neq
h}^{j-1}\sum_{w\notin\\{m,h,l\\}}^{j-1}P^{\gamma_{n}}_{il}P^{\gamma_{n}}_{ih}P^{\gamma_{n}}_{jm}P^{\gamma_{n}}_{jw}\mathbb{E}[\varepsilon_{l}\varepsilon_{h}\varepsilon_{m}]\mathbb{E}[\varepsilon_{w}]$
$\displaystyle+\sum_{l=1}^{i-1}\sum_{h\neq
l}^{i-1}\sum_{m\notin\\{h,l\\}}^{j-1}P^{\gamma_{n}}_{il}P^{\gamma_{n}}_{ih}P^{\gamma_{n}}_{jm}P^{\gamma_{n}}_{jl}\mathbb{E}[\varepsilon_{l}^{2}]\mathbb{E}[\varepsilon_{h}]\mathbb{E}[\varepsilon_{m}]$
$\displaystyle+\sum_{l=1}^{i-1}\sum_{h\neq
l}^{i-1}P^{\gamma_{n}}_{il}P^{\gamma_{n}}_{ih}(P^{\gamma_{n}}_{jl})^{2}\mathbb{E}[\varepsilon_{l}^{3}]\mathbb{E}[\varepsilon_{h}]+2\sum_{l=1}^{i-1}\sum_{h\neq
l}^{i-1}P^{\gamma_{n}}_{il}P^{\gamma_{n}}_{ih}P^{\gamma_{n}}_{jl}P^{\gamma_{n}}_{jh}\mathbb{E}[\varepsilon_{l}^{2}]\mathbb{E}[\varepsilon_{h}^{2}]$
$\displaystyle+\sum_{l=1}^{i-1}\sum_{m=1}^{j-1}(P^{\gamma_{n}}_{il})^{2}(P^{\gamma_{n}}_{jm})^{2}\mathbb{E}[\varepsilon_{l}^{2}\varepsilon_{m}^{2}]$
$\displaystyle=$ $\displaystyle 2\sum_{l=1}^{i-1}\sum_{h\neq
l}^{i-1}P^{\gamma_{n}}_{il}P^{\gamma_{n}}_{ih}P^{\gamma_{n}}_{jl}P^{\gamma_{n}}_{jh}\mathbb{E}[\varepsilon_{l}^{2}]\mathbb{E}[\varepsilon_{h}^{2}]+\sum_{l=1}^{i-1}\sum_{m=1}^{j-1}(P^{\gamma_{n}}_{il})^{2}(P^{\gamma_{n}}_{jm})^{2}\mathbb{E}[\varepsilon_{l}^{2}\varepsilon_{m}^{2}]$
$\displaystyle=$ $\displaystyle 2\sum_{l=1}^{i-1}\sum_{h\neq
l}^{i-1}P^{\gamma_{n}}_{il}P^{\gamma_{n}}_{ih}P^{\gamma_{n}}_{jl}P^{\gamma_{n}}_{jh}\mathbb{E}[\varepsilon_{l}^{2}]\mathbb{E}[\varepsilon_{h}^{2}]+\sum_{l=1}^{i-1}\sum_{m\neq
l}^{j-1}(P^{\gamma_{n}}_{il})^{2}(P^{\gamma_{n}}_{jm})^{2}\mathbb{E}[\varepsilon_{l}^{2}]\mathbb{E}[\varepsilon_{m}^{2}]$
$\displaystyle+\sum_{l=1}^{i-1}(P^{\gamma_{n}}_{il})^{2}(P^{\gamma_{n}}_{jl})^{2}\mathbb{E}[\varepsilon_{l}^{4}]$
$\displaystyle=$ $\displaystyle 2\sum_{h=1}^{i-1}\sum_{l\neq
h}P^{\gamma_{n}}_{ih}P^{\gamma_{n}}_{il}P^{\gamma_{n}}_{jh}P^{\gamma_{n}}_{jl}\mathbb{E}[\varepsilon_{h}^{2}]\mathbb{E}[\varepsilon_{l}^{2}]$
$\displaystyle+\sum_{h=1}^{i-1}(P^{\gamma_{n}}_{ih})^{2}(P^{\gamma_{n}}_{jh})^{2}\text{Var}[\varepsilon_{h}^{2}]+\sum_{h=1}^{i-1}\sum_{l=1}^{j-1}(P^{\gamma_{n}}_{ih})^{2}(P^{\gamma_{n}}_{jl})^{2}\mathbb{E}[\varepsilon_{h}^{2}]\mathbb{E}[\varepsilon_{l}^{2}],$

where
$\text{Var}[\varepsilon_{h}^{2}]\coloneqq\mathbb{E}[\varepsilon_{h}^{4}]-\mathbb{E}[\varepsilon_{h}^{2}]\mathbb{E}[\varepsilon_{h}^{2}]$.

Defining $q\coloneqq\text{min}\\{i,j\\}$, the above display in turn implies

$\displaystyle\mathbb{E}[\mathcal{V}_{n}^{4}]=$ $\displaystyle
32\sum_{i=2}^{n}\mathbb{E}[\varepsilon_{i}^{2}]\sum_{j=2}^{n}\mathbb{E}[\varepsilon_{j}^{2}]\sum_{h=1}^{q-1}\sum_{l\neq
h}P^{\gamma_{n}}_{ih}P^{\gamma_{n}}_{il}P^{\gamma_{n}}_{jh}P^{\gamma_{n}}_{jl}\mathbb{E}[\varepsilon_{h}^{2}]\mathbb{E}[\varepsilon_{l}^{2}]$
$\displaystyle+16\sum_{i=2}^{n}\mathbb{E}[\varepsilon_{i}^{2}]\sum_{j=2}^{n}\mathbb{E}[\varepsilon_{j}^{2}]\sum_{h=1}^{q-1}(P^{\gamma_{n}}_{ih})^{2}(P^{\gamma_{n}}_{jh})^{2}\text{Var}[\varepsilon_{h}^{2}]$
$\displaystyle+16\sum_{i=2}^{n}\mathbb{E}[\varepsilon_{i}^{2}]\sum_{j=2}^{n}\mathbb{E}[\varepsilon_{j}^{2}]\sum_{h=1}^{i-1}\sum_{l=1}^{j-1}(P^{\gamma_{n}}_{ih})^{2}(P^{\gamma_{n}}_{jl})^{2}\mathbb{E}[\varepsilon_{h}^{2}]\mathbb{E}[\varepsilon_{l}^{2}]$
$\displaystyle\equiv$
$\displaystyle\mathfrak{A}_{n}+\mathfrak{B}_{n}+s_{n}^{4}.$

In order to establish the convergence in Equation (A.21), it is sufficient to
show that $\frac{\mathfrak{A}_{n}+\mathfrak{B}_{n}}{s_{n}^{4}}\to 0$. To this
end, note that

$\displaystyle\mathfrak{A}_{n}$ $\displaystyle\leq
C\sum_{i=1}^{n}\sum_{j=1}^{n}\sum_{h=1}^{q-1}\sum_{l\neq
h}P^{\gamma_{n}}_{ih}P^{\gamma_{n}}_{il}P^{\gamma_{n}}_{jh}P^{\gamma_{n}}_{jl}$
(A.22) $\displaystyle=C\left(\sum_{i=1}^{n}\sum_{j\neq
i}\sum_{h=1}^{q-1}\sum_{l\neq
h}P^{\gamma_{n}}_{ih}P^{\gamma_{n}}_{il}P^{\gamma_{n}}_{jh}P^{\gamma_{n}}_{jl}+\sum_{i=1}^{n}\sum_{h=1}^{q-1}\sum_{l\neq
h}(P^{\gamma_{n}}_{ih})^{2}(P^{\gamma_{n}}_{il})^{2}\right)$
$\displaystyle\leq C\left(\sum_{i=1}^{n}\sum_{j<i}\sum_{h<j}\sum_{l\neq
h}P^{\gamma_{n}}_{ih}P^{\gamma_{n}}_{il}P^{\gamma_{n}}_{jh}P^{\gamma_{n}}_{jl}+r_{n}\right)$
$\displaystyle\leq
C\left(\sum_{i=1}^{n}\sum_{j<i}\sum_{h<j}\sum_{l<h}P^{\gamma_{n}}_{ih}P^{\gamma_{n}}_{il}P^{\gamma_{n}}_{jh}P^{\gamma_{n}}_{jl}+r_{n}\right)$
$\displaystyle\leq
C\left(\left|\sum_{i=1}^{n}\sum_{j<i}\sum_{h<j}\sum_{l<h}P^{\gamma_{n}}_{ih}P^{\gamma_{n}}_{il}P^{\gamma_{n}}_{jh}P^{\gamma_{n}}_{jl}\right|+r_{n}\right)$
$\displaystyle\leq Cr_{n}$

The first inequality follows from Assumption 1. The second inequality follows
from Lemma 1 (vi). The third inequality follows from the symmetry of
$P^{\gamma_{n}}$. The fifth inequality follows from Lemma 2. Next,

$\displaystyle\mathfrak{B}_{n}$ $\displaystyle\leq
C\sum_{i=1}^{n}\sum_{j=1}^{n}\sum_{h=1}^{q-1}(P^{\gamma_{n}}_{ih})^{2}(P^{\gamma_{n}}_{jh})^{2}\leq
Cr_{n},$ (A.23)

the first inequality following from Assumption 1 and the second from Lemma 1
(vi). Thus,

$\displaystyle\frac{\mathfrak{A}_{n}+\mathfrak{B}_{n}}{s^{4}_{n}}$
$\displaystyle\leq C\frac{r_{n}}{\left(\sum_{i=1}^{n}\sum_{j\neq
i}(P^{\gamma_{n}}_{ij})^{2}\right)^{2}}\leq Cr^{1-2\omega}_{n}\to 0.$ (A.24)

where the second inequality follows from Assumption 3 and the convergence from
Assumption 2 and Assumption 3. Equation (A.24) verifies the condition in
Equation (A.21), which in turn implies that the condition condition in
Equation (A.18) holds.

Having verified the conditions in Equation (A.17) and Equation (A.18) we
conclude that $RJAR^{*}_{\gamma_{n}}\overset{d}{\to}\mathcal{N}[0,1]$. By
Lemma 4 $|\hat{\Phi}_{\gamma_{n}}-\Phi_{\gamma_{n}}|\overset{p}{\to}0$ and
hence also $\hat{\Phi}_{\gamma_{n}}/\Phi_{\gamma_{n}}\overset{p}{\to}1$ under
Assumptions 1 and 3. Hence, under the null hypothesis, the continuous mapping
theorem implies that
$RJAR_{\gamma_{n}}(\beta_{0})\overset{d}{\to}\mathcal{N}[0,1]$.

∎

## References

  * [1] Stanislav Anatolyev  “Inference in regression models with many regressors”  In _Journal of Econometrics_ 170.2, 2012, pp. 368–382
  * [2] Stanislav Anatolyev and Nikolay Gospodinov  “Specification Testing in Models with Many Instruments”  In _Econometric Theory_ 27.2, 2011, pp. 427–441
  * [3] Stanislav Anatolyev and Pavel Yaskov  “Asymptotics of Diagonal Elements of Projection Matrices Under Many Instruments/Regressors”  In _Econometric Theory_ 33, 2017, pp. 717–738
  * [4] Theodore Anderson and Herman Rubin  “Estimation of the Parameters of a Single Equation in a Complete System of Stochastic Equations”  In _The Annals of Mathematical Statistics_ 20, 1949, pp. 46–63
  * [5] Donald Andrews and James Stock  “Testing with many weak instruments”  In _Journal of Econometrics_ 138.1, 2007, pp. 24–46
  * [6] Jushan Bai and Serena Ng  “Instrumental Variable Estimation in a Data Rich Environment”  In _Econometric Theory_ 26.6, 2010, pp. 1577–1606
  * [7] Paul Bekker and Federico Crudu  “Jackknife instrumental variable estimation with heteroskedasticity”  In _Journal of Econometrics_ 185.2, 2015, pp. 332–342
  * [8] Alexandre Belloni, Daniel Chen, Victor Chernozhukov and Christian Hansen  “Sparse Models and Methods for Optimal Instruments With an Application to Eminent Domain”  In _Econometrica_ 80.6, 2012, pp. 2369–2429
  * [9] Christine Blandhol, John Bonney, Magne Mogstad and Alexander Torgovitsky  “When is TSLS _Actually_ LATE?”  In _Becker Friedman Institute Working Papers_ , 2022, pp. 1–68
  * [10] Maurice Bun, Helmut Farbmacher and Rutger Poldermans  “Finite sample properties of the GMM Anderson-Rubin test”  In _Econometric Reviews_ 39.10, 2020, pp. 1042–1056
  * [11] David Card  “Immigration and Inequality”  In _American Economic Review Papers and Proceedings_ 99.2, 2009, pp. 1–21
  * [12] Marine Carrasco and Guy Tchuente  “Regularization Based Anderson Rubin Tests for Many Instruments”  In _University of Kent, School of Economics Discussion Papers_ , 2016, pp. 1–34
  * [13] John Chao, Norman Swanson, Jerry Hausmann, Whitney Newey and Tiemen Woutersen  “Asymptotic Distribution of JIVE in a Heteroskedastic IV Regression with Many Instruments”  In _Econometric Theory_ 28, 2012, pp. 42–86
  * [14] Federico Crudu, Giovanni Mellace and Zsolt Sándor  “Inference in Instrumental Variable Models with Heteroskedasticity and Many Instruments”  In _Econometric Theory_ 77, 2020, pp. 1–30
  * [15] Xavier Gabaix and Ralph Koijen  “Granular Instrumental Variables”  In _NBER Working Paper Series_ , 2020, pp. 1–98
  * [16] Paul Goldsmith-Pinkham, Isaac Sorkin and Henry Swift  “Bartik Instruments: What, When, Why, and How”  In _American Economic Review_ 110.8, 2020, pp. 2586–2624
  * [17] Peter Hall and Chris Heyde  “Martingale Limit Theory and Its Application”  Academic Press, 1980
  * [18] Christian Hansen and Damian Kozbur  “Instrumental variables estimation with many weak instruments using regularized JIVE”  In _Journal of Econometrics_ 182.2, 2014, pp. 290–308
  * [19] Maximilien Kaffo and Wenjie Wang  “On bootstrap validity for specification testing with many weak instruments”  In _Economics Letters_ 157, 2017, pp. 107–111
  * [20] Georges Kapetanios, Lynda Khalaf and Massimiliano Marcellino  “Factor-Based Identification-Robust Interference in IV Regressions”  In _Journal of Applied Econometrics_ 31.5, 2015, pp. 821–842
  * [21] Helmut Lütkepohl  “Handbook of Matrices”  Wiley, 1996
  * [22] Anna Mikusheva and Liyang Sun  “Inference with Many Weak Instruments”  In _Review of Economic Studies (Forthcoming)_ , 2022, pp. 1–37
  * [23] Peter C B Phillips and Wayne Yuan Gao  “Structural inference from reduced forms with many instruments”  In _Journal of Econometrics_ 199.2, 2017, pp. 96–116