thresholding test appears to have good power properties over all alternatives
considered. However, tests based on the jackknife AR or jackknife LM statistic
may still provide better power than the threshholding test for very positive
values of $(\beta_{0}-\beta)$.In order to consider the effect of correlated
instruments on the power properties of the test in a setting with plausibly
many instruments, I additionally examine local power under a fourth instrument
regime. This setup adds the ten cubic terms $z_{\ell i}^{3},\ell=1,\dots,10$
to the interactions and quadratic terms of the third instrument regime for a
total of 75 instruments, $d_{z}=75$. Appendix I provides the simulated sizes
of tests under this fourth regime. Figure 7.2 plots calibrated local power
curves under this fourth instrument regime. While all tests have lower power
in this regime than in the regime considered in Figure 7.1, the many
instrument jackknife AR and jackknife LM tests appear to face a steeper power
decline than tests based on the jackknife K-statistic or the thresholding
statistic.These results should not be interpreted as critiques of the
benchmark testing procedures of Anderson and Rubin (1949); Staiger and Stock
(1997); Belloni et al. (2012); Crudu et al. (2021); Mikusheva and Sun (2021),
and Matsushita and Otsu (2022), whose work I rely on and was inspired by.

##  8 Conclusion

I propose a new test for the structural parameter in a linear instrumental
variables model. This test is based on a jackknife version of the K-statistic
and the limiting behavior of the test is analyzed via a novel direct Gaussian
approximation argument. I show that, as long as an auxiliary parameter can be
consistently estimated, the test is robust to both the strength of
identification and the number of instruments; the limiting distribution of the
test statistic does not depend on either of these factors. Consistency of the
auxiliary parameter can be achieved under approximate sparsity using simple-
to-implement $\ell_{1}$-penalized methods.I characterize the behavior of the
jackknife K-statistic in local neighborhoods of the null. To address a power
deficiency that tests based on jackknife K-statistic inherit from their non-
jackknife namesakes, I propose a testing procedure that decides whether the
researcher should run a test via the jackknife K-statistic or one via the sup-
score statistic based on the value of a conditioning statistic. While this
combination does not fully address the power decline, I show that it works
well in a simulation study and leave further refinements to future work.

## References

  * Anderson and Rubin (1949) Anderson, T. W. and H. Rubin (1949).  Estimation of the Parameters of a Single Equation in a Complete System of Stochastic Equations.  The Annals of Mathematical Statistics 20(1), 46 – 63.
  * Andrews et al. (2004) Andrews, D. W., M. Moreira, and J. H. Stock (2004, August).  Optimal invariant similar tests for instrumental variables regression.  (299).
  * Andrews and Stock (2007) Andrews, D. W. and J. H. Stock (2007).  Testing with many weak instruments.  Journal of Econometrics 138(1), 24–46.  50th Anniversary Econometric Institute.
  * Andrews et al. (2006) Andrews, D. W. K., M. J. Moreira, and J. H. Stock (2006).  Optimal two-sided invariant similar tests for instrumental variables regression.  Econometrica 74(3), 715–752.
  * Andrews (2016) Andrews, I. (2016).  Conditional linear combination tests for weakly identified models.  Econometrica 84(6), 2155–2182.
  * Angrist et al. (1999) Angrist, J. D., G. W. Imbens, and A. B. Krueger (1999).  Jackknife instrumental variables estimation.  Journal of Applied Econometrics 14(1), 57–67.
  * Angrist and Krueger (1991) Angrist, J. D. and A. B. Krueger (1991).  Does compulsory school attendance affect schooling and earnings?  The Quarterly Journal of Economics 106(4), 979–1014.
  * Bekker (1994) Bekker, P. A. (1994).  Alternative approximations to the distributions of instrumental variable estimators.  Econometrica 62(3), 657–681.
  * Belloni et al. (2012) Belloni, A., D. Chen, V. Chernozhukov, and C. Hansen (2012).  Sparse models and methods for optimal instruments with an application to eminent domain.  Econometrica 80(6), 2369–2429.
  * Belloni et al. (2018) Belloni, A., V. Chernozhukov, D. Chetverikov, C. Hansen, and K. Kato (2018).  High-dimensional econometrics and regularized gmm.
  * Bound et al. (1995) Bound, J., D. A. Jaeger, and R. M. Baker (1995).  Problems with instrumental variables estimation when the correlation between the instruments and the endogeneous explanatory variable is weak.  Journal of the American Statistical Association 90(430), 443–450.
  * Celentano et al. (2020) Celentano, M., A. Montanari, and Y. Wu (2020, 09–12 Jul).  The estimation error of general first order methods.  In J. Abernethy and S. Agarwal (Eds.), Proceedings of Thirty Third Conference on Learning Theory, Volume 125 of Proceedings of Machine Learning Research, pp. 1078–1141. PMLR.
  * Chamberlain (1987) Chamberlain, G. (1987).  Asymptotic efficiency in estimation with conditional moment restrictions.  Journal of Econometrics 34(3), 305–334.
  * Chao et al. (2012) Chao, J. C., N. R. Swanson, J. A. Hausman, W. K. Newey, and T. Woutersen (2012).  Asymptotic distribution of jive in a heteroskedastic iv regression with many instruments.  Econometric Theory 28(1), 42–86.
  * Chatterjee (2006) Chatterjee, S. (2006).  A generalization of the Lindeberg principle.  The Annals of Probability 34(6), 2061 – 2076.
  * Chatterjee (2010) Chatterjee, S. (2010).  A new approach to strong embeddings.
  * Chernozhukov et al. (2013) Chernozhukov, V., D. Chetverikov, and K. Kato (2013).  Gaussian approximations and multiplier bootstrap for maxima of sums of high-dimensional random vectors.  The Annals of Statistics 41(6), 2786 – 2819.
  * Chernozhukov et al. (2017) Chernozhukov, V., D. Chetverikov, and K. Kato (2017).  Central limit theorems and bootstrap in high dimensions.  The Annals of Probability 45(4), 2309–2352.
  * Chernozhukov et al. (2022) Chernozhukov, V., W. K. Newey, and R. Singh (2022).  Automatic debiased machine learning of causal and structural effects.  Econometrica 90(3), 967–1027.
  * Chetverikov and Sørensen (2021) Chetverikov, D. and J. R.-V. Sørensen (2021).  Analytic and bootstrap-after-cross-validation methods for selecting penalty parameters of high-dimensional m-estimators.  ArXiv NA, 1–50.
  * Crudu et al. (2021) Crudu, F., G. Mellace, and Z. Sándor (2021).  Inference in instrumental variable models with heteroskedasticity and many instruments.  Econometric Theory 37(2), 281–310.
  * Derenoncourt (2022) Derenoncourt, E. (2022, February).  Can you move to opportunity? evidence from the great migration.  American Economic Review 112(2), 369–408.
  * Dobbie et al. (2018) Dobbie, W., J. Goldin, and C. S. Yang (2018, February).  The effects of pretrial detention on conviction, future crime, and employment: Evidence from randomly assigned judges.  American Economic Review 108(2), 201–40.
  * Friedman et al. (2010) Friedman, J., R. Tibshirani, and T. Hastie (2010).  Regularization paths for generalized linear models via coordinate descent.  Journal of Statistical Software 33(1), 1–22.
  * Gilchrist and Sands (2016) Gilchrist, D. S. and E. G. Sands (2016).  Something to talk about: Social spillovers in movie consumption.  Journal of Political Economy 124(5), 1339–1382.
  * Götze et al. (2019) Götze, F., A. Naumov, V. Spokoiny, and V. Ulyanov (2019).  Large ball probabilities, Gaussian comparison and anti-concentration.  Bernoulli 25(4A), 2538 – 2563.
  * Gotze et al. (2021) Gotze, F., H. Sambale, and A. Sinulis (2021).  Concentration inequalities for polynomials in alpha-sub-exponential random variables.  Electronic Journal of Probability 26(none), 1 – 22.
  * Han and Phillips (2006) Han, C. and P. C. B. Phillips (2006).  Gmm with many moment conditions.  Econometrica 74(1), 147–192.
  * Harrell (2015) Harrell, F. E. (2015).  Regression Modeling Strategies.  Spring Series in Statistics. Springer Cham.
  * Horn and Johnson (2012) Horn, R. and C. Johnson (2012).  Matrix Analysis.  Cambridge University Press.
  * Jou and Morgan (2023) Jou, A. and T. Morgan (2023).  Do relief programs compensate affected populations? evidence from the great depression and the new deal.  Working Paper.
  * Kleibergen (2002) Kleibergen, F. (2002, 02).  Pivotal statistics for testing structural parameters in instrumental variables regression.  Econometrica 70, 1781–1803.
  * Kleibergen (2005) Kleibergen, F. (2005).  Testing parameters in gmm without assuming that they are identified.  Econometrica 73(4), 1103–1123.
  * Kline et al. (2020) Kline, P., R. Saggio, and M. Sølvsten (2020).  Leave-out estimation of variance components.  Econometrica 88(5), 1859–1898.
  * Lee et al. (2022) Lee, D. S., J. McCrary, M. J. Moreira, and J. Porter (2022, October).  Valid t-ratio inference for iv.  American Economic Review 112(10), 3260–90.
  * Lim et al. (2022) Lim, D., W. Wang, and Y. Zhang (2022).  A conditional linear combination test with many weak instruments.
  * Lindeberg (1922) Lindeberg, J. W. (1922).  Eine neue herleitung des exponentialgesetzes in der wahrscheinlichkeitsrechnung.  Mathematische Zeitschrift 15, 211–225.
  * Maestas et al. (2013) Maestas, N., K. J. Mullen, and A. Strand (2013, August).  Does disability insurance receipt discourage work? using examiner assignment to estimate causal effects of ssdi receipt.  American Economic Review 103(5), 1797–1829.
  * Matsushita and Otsu (2022) Matsushita, Y. and T. Otsu (2022).  A jackknife lagrange multiplier test with many weak instruments.  Econometric Theory, 1–24.
  * Mikusheva (2023) Mikusheva, A. (2023).  Many weak instruments in time series econometrics.  Working Paper.
  * Mikusheva and Sun (2021) Mikusheva, A. and L. Sun (2021, 12).  Inference with many weak instruments.  The Review of Economic Studies 89(5), 2663–2686.
  * Moreira (2009) Moreira, M. (2009, 10).  Tests with correct size when instruments can be arbitrarily weak.  Journal of Econometrics 152, 131–140.
  * Moreira (2001) Moreira, M. J. (2001).  Tests with correct size when instruments can be arbitrarily weak.  Citeseer.
  * Moreira (2003) Moreira, M. J. (2003).  A conditional likelihood ratio test for structural models.  Econometrica 71(4), 1027–1048.
  * Nazarov (2003) Nazarov, F. (2003).  On the Maximal Perimeter of a Convex Set in $R^{n}$ with Respect to a Gaussian Measure.  Berlin, Heidelberg: Springer Berlin Heidelberg.
  * Nelson and Startz (1990) Nelson, C. R. and R. Startz (1990).  Some further results on the exact small sample properties of the instrumental variable estimator.  Econometrica 58(4), 967–976.
  * Newey and Windmeijer (2009) Newey, W. K. and F. Windmeijer (2009).  Generalized method of moments with many weak moment conditions.  Econometrica 77(3), 687–719.
  * Paravisini et al. (2014) Paravisini, D., V. Rappoport, P. Schnabl, and D. Wolfenzon (2014, 09).  Dissecting the Effect of Credit Supply on Trade: Evidence from Matched Credit-Export Data.  The Review of Economic Studies 82(1), 333–359.
  * Petersen and Pedersen (2012) Petersen, K. B. and M. S. Pedersen (2012, nov).  The matrix cookbook.  Version 20121115.
  * Pouzo (2015) Pouzo, D. (2015).  Bootstrap consistency for quadratic forms of sample averages with increasing dimension.  Electronic Journal of Statistics 9(2), 3046 – 3097.
  * R Core Team (2021) R Core Team (2021).  R: A Language and Environment for Statistical Computing.  Vienna, Austria: R Foundation for Statistical Computing.
  * Sambale (2022) Sambale, H. (2022).  Some notes on concentration for $\alpha$-subexponential random variables.
  * Sampat and Williams (2019) Sampat, B. and H. L. Williams (2019, January).  How do patents affect follow-on innovation? evidence from the human genome.  American Economic Review 109(1), 203–36.
  * Staiger and Stock (1997) Staiger, D. and J. H. Stock (1997).  Instrumental variables regression with weak instruments.  Econometrica 65(3), 557–586.
  * Stock and Yogo (2005) Stock, J. and M. Yogo (2005).  Testing for Weak Instruments in Linear IV Regression.  New York: Cambridge University Press.
  * Tan (2017) Tan, Z. (2017).  Regularized calibrated estimation of propensity scores with model misspecification and high-dimensional data.  ArXiv NA, 1–60.
  * van der Greer (2016) van der Greer, S. (2016).  Estimation and Testing under Sparsity.  Lecture Notes in Mathematics. Springer, New York, NY.
  * van Wieringen (2023) van Wieringen, W. N. (2023).  Lecture notes on ridge regression.

##  Appendix A Proofs of Results in Section 3

###  A.1 Proof of Lemma 3.1

The statement $\sup_{a<0}\big{|}\mathrm{Pr}(\text{\it JK}_{I}(\beta_{0})\leq
a)-\mathrm{Pr}(\text{\it JK}_{G}(\beta_{0})\leq a)\big{|}=0$ is immediate
since both $\text{\it JK}_{I}(\beta_{0})$ and $\text{\it JK}_{G}(\beta_{0})$
are always weakly positive.It thus suffices to show

$\sup_{a\geq 0}\big{|}\mathrm{Pr}(\text{\it JK}_{I}(\beta_{0})\leq
a)-\mathrm{Pr}(\text{\it JK}_{G}(\beta_{0})\leq a)\big{|}\to 0$

Before proceeding, we will introduce some notation. Let $\tilde{H}=s_{n}H$ and
$\tilde{h}_{ij}=s_{n}h_{ij}$, where $s_{n}$ is as in Assumption 3.2. Recall
that $\tilde{h}_{ii}=0$ and define

$\displaystyle N$
$\displaystyle\mathrel{\mathop{\ordinarycolon}}=\frac{1}{\sqrt{n}}\sumop\slimits@_{i=1}^{n}\epsilon_{i}(\beta_{0})\sumop\slimits@_{j=1}^{n}\tilde{h}_{ij}r_{j}$
$\displaystyle\mathrel{\mathop{\ordinarycolon}}=\frac{1}{\sqrt{n}}\sumop\slimits@_{i=1}^{n}\tilde{\epsilon}_{i}(\beta_{0})\sumop\slimits@_{j=1}^{n}\tilde{h}_{ij}\tilde{r}_{j}$
$\displaystyle D$
$\displaystyle\mathrel{\mathop{\ordinarycolon}}=\frac{1}{n}\sumop\slimits@_{i=1}^{n}\epsilon_{i}^{2}(\beta_{0})\big{(}\sumop\slimits@_{j=1}^{n}\tilde{h}_{ij}r_{j}\big{)}^{2}$
$\displaystyle\mathrel{\mathop{\ordinarycolon}}=\frac{1}{n}\sumop\slimits@_{i=1}^{n}\kappa_{i}^{2}(\beta_{0})\big{(}\sumop\slimits@_{j=1}^{n}\tilde{h}_{ij}\tilde{r}_{j}\big{)}^{2}$

where $(\tilde{\epsilon}_{i}(\beta_{0}),\tilde{r}_{i})$ are jointly Gaussian
with the same mean and covariance matrix as $(\epsilon_{i}(\beta_{0}),r_{i})$
and $\kappa_{i}^{2}(\beta_{0})=\mathbb{E}[\epsilon_{i}^{2}(\beta_{0})]$.Under
this notation we can write $\text{\it
JK}_{I}(\beta_{0})=\frac{N^{2}}{D}\bm{1}_{\\{D>0\\}}$ and $\text{\it
JK}_{G}(\beta_{0})=\frac{\tilde{N}^{2}}{\tilde{D}}$. Dealing with these forms
of the statistics is difficult for the interpolation argument, since the
denominator is random. Instead, we will notice that since $D=0\implies N=0$
and $\mathrm{Pr}(\tilde{D}>0)=1$, for any $a\geq 0$ we can rewrite the events

$\begin{split}\\{\text{\it JK}_{I}(\beta_{0})\leq a\\}=\\{N^{2}-aD\leq
0\\}\hbox{ }\text{ and }\hbox{ }\\{\text{\it JK}_{G}(\beta_{0})\leq
a\\}\overset{\text{a.s}}{=}\\{\tilde{N}^{2}-a\tilde{D}\leq 0\\}\end{split}$
(A.1)

With this in mind define

$\displaystyle\text{\it
JK}^{a}\mathrel{\mathop{\ordinarycolon}}=N^{2}-aD\hbox{ }\text{ and }\hbox{
}\tilde{\text{\it
JK}}^{a}\mathrel{\mathop{\ordinarycolon}}=\tilde{N}^{2}-a\tilde{D}$

Showing Lemma 3.1 is then equivalent to showing that
$\sup_{a}|\mathrm{Pr}(\text{\it JK}^{a}\leq 0)-\mathrm{Pr}(\tilde{\text{\it
JK}}^{a}\leq 0)|\to 0$. We do so in a few lemmas, the final result being shown
in Lemma A.6 at the bottom of this subsection.

######  Lemma A.1 (Lindeberg Interpolation).

Suppose that Assumptions 3.1, 3.2 and 3.3 hold. Let
$\varphi(\cdot)\mathrel{\mathop{\ordinarycolon}}\mathbb{R}\to\mathbb{R}$ be
such that $\varphi(\cdot)\in C_{b}^{3}(\mathbb{R})$ with
$L_{2}(\varphi)=\sup_{x}|\varphi^{\prime\prime}(x)|$ and
$L_{3}(\varphi)=\sup_{x}|\varphi^{\prime\prime\prime}(x)|$. Then, there is a
constant $M$ that depends only on the constant $c$ such that:

$|\mathbb{E}[\varphi(\text{\it JK}^{a})-\varphi(\tilde{\text{\it
JK}}^{a})]|\leq\frac{M(a^{3}\vee 1)}{\sqrt{n}}(L_{2}(\varphi)+L_{3}(\varphi))$

###### Proof of Lemma A.1.

Begin by defining the leave-one-out numerator, denominator, and decomposed
statistics

$\displaystyle N_{-i}$
$\displaystyle\mathrel{\mathop{\ordinarycolon}}=\frac{1}{\sqrt{n}}\sumop\slimits@_{j\neq
i}\dot{\epsilon}_{j}(\beta_{0})\sumop\slimits@_{\ell\neq
i}\tilde{h}_{j\ell}\dot{r}_{\ell}$ $\displaystyle D_{-i}$
$\displaystyle\mathrel{\mathop{\ordinarycolon}}=\frac{1}{n}\sumop\slimits@_{j\neq
i}\ddot{\epsilon}_{j}^{2}(\beta_{0})\big{(}\sumop\slimits@_{\ell\neq
i}\tilde{h}_{j\ell}\dot{r}_{\ell}\big{)}^{2}$ $\displaystyle\text{\it
JK}_{-i}$ $\displaystyle\mathrel{\mathop{\ordinarycolon}}=N_{-i}^{2}-aD_{-i}$

where for each $\ell\in[n]$, $\dot{\epsilon}_{\ell}(\beta_{0})$ is equal to
$\epsilon_{\ell}(\beta_{0})$ if $\ell>i$ and
$\tilde{\epsilon}_{\ell}(\beta_{0})$ if $\ell<i$, $\dot{r}_{\ell}$ is equal to
$r_{\ell}$ if $\ell>i$ and $\tilde{r}_{\ell}$ if $\ell<i$, and
$\ddot{\epsilon}_{\ell}^{2}(\beta_{0})$ is equal to
$\kappa_{\ell}^{2}(\beta_{0})$ if $\ell<i$ and
$\epsilon_{\ell}^{2}(\beta_{0})$ if $\ell>i$. While the definitions of
$\dot{\epsilon}_{\ell},\dot{r}_{\ell},$ and $\ddot{\epsilon}_{\ell}$ depend on
$i$ because we will be considering only one deviation at a time, we will
supress the dependence of these variables on $i$ to simplify notation.Next,
define the one-step deviations

$\begin{split}{}_{1i}&\mathrel{\mathop{\ordinarycolon}}=\epsilon_{i}(\beta_{0})\sumop\slimits@_{j=1}^{n}\tilde{h}_{ij}\dot{r}_{j}+r_{i}\sumop\slimits@_{j=1}^{n}\tilde{h}_{ji}\dot{\epsilon}_{j}(\beta_{0})\\\
\tilde{\Delta}_{1i}&\mathrel{\mathop{\ordinarycolon}}=\tilde{\epsilon}_{i}(\beta_{0})\sumop\slimits@_{j=1}^{n}\tilde{h}_{ij}\dot{r}_{j}+\tilde{r}_{i}\sumop\slimits@_{j=1}^{n}\tilde{h}_{ji}\dot{\epsilon}_{j}(\beta_{0})\\\
{}_{2i}&\mathrel{\mathop{\ordinarycolon}}=\underbrace{a\epsilon_{i}^{2}(\beta_{0})(\sumop\slimits@_{j=1}^{n}\tilde{h}_{ij}\dot{r}_{j})^{2}+ar_{i}^{2}\sumop\slimits@_{j=1}^{n}\tilde{h}_{ji}^{2}\ddot{\epsilon}_{j}^{2}(\beta_{0})}_{{}_{2i}^{a}}+\underbrace{2ar_{i}\sumop\slimits@_{j=1}^{n}\ddot{\epsilon}_{j}^{2}(\beta_{0})\sumop\slimits@_{\ell\neq
i}\tilde{h}_{j\ell}\tilde{h}_{ji}\dot{r}_{\ell}}_{{}_{2i}^{b}}\\\
\tilde{\Delta}_{2i}&\mathrel{\mathop{\ordinarycolon}}=\underbrace{a\kappa_{i}^{2}(\beta_{0})(\sumop\slimits@_{j=1}^{n}\tilde{h}_{ij}\dot{r}_{j})^{2}+a\tilde{r}_{i}^{2}\sumop\slimits@_{j=1}^{n}\tilde{h}_{ji}^{2}\ddot{\epsilon}_{j}^{2}(\beta_{0})}_{{}_{2i}^{a}}+\underbrace{2a\tilde{r}_{i}\sumop\slimits@_{j=1}^{n}\ddot{\epsilon}_{j}^{2}(\beta_{0})\sumop\slimits@_{\ell\neq
i}\tilde{h}_{j\ell}\tilde{h}_{ji}\dot{r}_{\ell}}_{\tilde{\Delta}_{2i}^{b}}\end{split}$
(A.2)

These one-step deviations contain all the terms associated with observation
$i$ in the expression of the numerator and denominator of the test statistics.
To demonstrate, note that these one-step deviations satisfy
$N_{-1}+n^{-1/2}{}_{11}=N$ and $aD_{-1}+n^{-1}{}_{21}=aD$ as

$\displaystyle N$
$\displaystyle=\frac{1}{\sqrt{n}}\sumop\slimits@_{i=1}^{n}\epsilon_{i}(\beta_{0})\sumop\slimits@_{j=1}^{n}\tilde{h}_{ij}r_{j}$
$\displaystyle=\frac{1}{\sqrt{n}}\sumop\slimits@_{j>1}\epsilon_{j}(\beta_{0})\sumop\slimits@_{\ell=1}^{n}\tilde{h}_{j\ell}r_{j}+\epsilon_{1}(\beta_{0})\frac{1}{\sqrt{n}}\sumop\slimits@_{j>1}\tilde{h}_{1j}r_{j}$
$\displaystyle=\frac{1}{\sqrt{n}}\sumop\slimits@_{j>1}\epsilon_{j}(\beta_{0})\Big{\\{}\tilde{h}_{j1}r_{1}+\sumop\slimits@_{\ell>1}h_{j\ell}r_{\ell}\Big{\\}}+\epsilon_{1}(\beta_{0})\frac{1}{\sqrt{n}}\sumop\slimits@_{j>1}\tilde{h}_{1j}r_{j}$
$\displaystyle=\underbrace{\frac{1}{\sqrt{n}}\sumop\slimits@_{j>1}\epsilon_{j}(\beta_{0})\sumop\slimits@_{\ell>1}h_{j\ell}r_{\ell}}_{N_{-1}}+\underbrace{\epsilon_{1}(\beta_{0})\frac{1}{\sqrt{n}}\sumop\slimits@_{j>1}\tilde{h}_{1j}r_{j}+r_{1}\frac{1}{\sqrt{n}}\sumop\slimits@_{j>1}\tilde{h}_{j1}\epsilon_{j}(\beta_{0})}_{n^{-1/2}{}_{11}}$
and $\displaystyle D$
$\displaystyle=\frac{1}{n}\sumop\slimits@_{i=1}^{n}\epsilon_{i}^{2}(\beta_{0})\big{(}\sumop\slimits@_{j=1}^{n}\tilde{h}_{ij}r_{j})^{2}$
$\displaystyle=\frac{1}{n}\sumop\slimits@_{j>1}\epsilon_{j}^{2}(\beta_{0})\big{(}\sumop\slimits@_{\ell=1}^{n}\tilde{h}_{j\ell}r_{\ell}\big{)}^{2}+\epsilon_{1}^{2}(\beta_{0})\frac{1}{n}\big{(}\sumop\slimits@_{j>1}\tilde{h}_{1j}r_{j}\big{)}^{2}$
$\displaystyle=\frac{1}{n}\sumop\slimits@_{j>1}\epsilon_{j}^{2}(\beta_{0})\big{(}\tilde{h}_{j1}r_{1}+\sumop\slimits@_{\ell\neq
1}\tilde{h}_{\ell
j}r_{\ell}\big{)}^{2}+\epsilon_{1}^{2}(\beta_{0})\frac{1}{n}\big{(}\sumop\slimits@_{j>1}\tilde{h}_{1j}r_{j}\big{)}^{2}$
$\displaystyle=\underbrace{\frac{1}{n}\sumop\slimits@_{j>1}\epsilon_{j}^{2}(\beta_{0})\big{(}\sumop\slimits@_{\ell>1}\tilde{h}_{\ell,j}r_{\ell}\big{)}^{2}}_{D_{-1}}$
$\displaystyle\;\;\;\;\;\;+\underbrace{\epsilon_{1}^{2}(\beta_{0})\frac{1}{n}\big{(}\sumop\slimits@_{j>1}\tilde{h}_{1j}r_{j}\big{)}^{2}+r_{1}^{2}\frac{1}{n}\sumop\slimits@_{j>1}\tilde{h}_{j1}^{2}\epsilon_{j}^{2}(\beta_{0})+2r_{1}\frac{1}{n}\sumop\slimits@_{j>1}\epsilon_{j}^{2}(\beta_{0})\sumop\slimits@_{\ell>1}\tilde{h}_{\ell
j}r_{\ell}}_{(an)^{-1}{}_{21}}$

Using the one-step deviations, write the difference
$\mathbb{E}[\varphi(K^{a})-\varphi(\tilde{K}^{a})]$ as a telescoping sum, one
by one replacing $({}_{1i},{}_{2i})$ with
$(\tilde{\Delta}_{1i},\tilde{\Delta}_{2i})$ in the expressions of $\text{\it
JK}^{a}=N^{2}-aD$ until we arrive at $\tilde{\text{\it
JK}}^{a}=\tilde{N}^{2}-a\tilde{D}$.

$\begin{split}\mathbb{E}[\varphi(\text{\it JK}^{a})-\varphi(\tilde{\text{\it
JK}}^{a})]&=\sumop\slimits@_{i=1}^{n}\mathbb{E}[\varphi(\text{\it
JK}_{-i}+n^{-1/2}N_{-i}{}_{1i}+n^{-1}{}_{1i}^{2}-n^{-1}{}_{2i})]\\\
&\hphantom{=\sumop\slimits@_{i=1}^{n}\;\;}-\mathbb{E}[\varphi(\text{\it
JK}_{-i}+n^{-1/2}N_{-i}\tilde{\Delta}_{1i}+n^{-1}\tilde{\Delta}_{1i}^{2}-n^{-1}\tilde{\Delta}_{2i})]\end{split}$
(A.3)

Via a second-order Taylor expansion, we can write each term inside the summand

$\displaystyle\mathbb{E}[\text{Term}_{i}]$
$\displaystyle=\mathbb{E}[\varphi^{\prime}(\text{\it
JK}_{-i})\\{2n^{-1/2}N_{-i}({}_{1i}-\tilde{\Delta}_{1i})+n^{-1}({}_{1i}^{2}-{}_{1i}^{2})-n^{-1}({}_{2i}-\tilde{\Delta}_{2i})\\}]$
$\displaystyle+\mathbb{E}[\varphi^{\prime\prime}(\text{\it
JK}_{-i})\\{4n^{-1}N_{-i}^{2}({}_{1i}^{2}-\tilde{\Delta}_{1i}^{2})+n^{-2}({}_{1i}^{4}-\tilde{\Delta}_{1i}^{4})-n^{-2}({}_{2i}^{2}-{}_{2i}^{2})\\}]$
$\displaystyle+\mathbb{E}[\varphi^{\prime\prime}(\text{\it
JK}_{-i})\\{4n^{-3/2}N_{-i}({}_{1i}^{3}-\tilde{\Delta}_{1i}^{3})+4n^{-3/2}N_{-i}({}_{1i}{}_{2i}-\tilde{\Delta}_{1i}\tilde{\Delta}_{2i})\\}]$
$\displaystyle+\mathbb{E}[\varphi^{\prime\prime}(\text{\it
JK}_{-i})\\{2n^{-2}({}_{1i}^{2}{}_{2i}-\tilde{\Delta}_{1i}^{2}\tilde{\Delta}_{2i})\\}]+R_{i}+\tilde{R}_{i}$

where $R_{i}$ and $\tilde{R}_{i}$ are remainder terms to be examined later.
Let $\mathcal{F}_{-i}$ denote the sigma algebra generated by all random
variables whose index is not equal to $i$. Since (a) for each $i\in[n]$ the
mean and covariance matrix of $(\epsilon_{i}(\beta_{0}),r_{i})$ is the same as
the mean and covariance matrix of
$(\tilde{\epsilon}_{i}(\beta_{0}),\tilde{r}_{i})$, (b)
$\mathbb{E}[\epsilon_{i}^{2}(\beta_{0})]=\kappa_{i}^{2}(\beta_{0})$, and (c)
random variables are independent across indices, we have that

$\begin{split}\mathbb{E}[{}_{1i}-\tilde{\Delta}_{1i}|\mathcal{F}_{-i}]&=\mathbb{E}[{}_{1i}^{2}-\tilde{\Delta}_{1i}^{2}|\mathcal{F}_{-i}]=\mathbb{E}[{}_{2i}-\tilde{\Delta}_{2i}|\mathcal{F}_{-i}]\\\
&=\mathbb{E}[{}_{2i}^{b}-\tilde{\Delta}_{2i}^{b}|\mathcal{F}_{-i}]=\mathbb{E}[{}_{1i}{}_{2i}^{b}-\tilde{\Delta}_{1i}\tilde{\Delta}_{2i}^{b}|\mathcal{F}_{-i}]=0\end{split}$
(A.4)

Using this we can simplify the prior display

$\displaystyle\mathbb{E}[\text{Term}_{i}]$
$\displaystyle=\underbrace{n^{-2}\mathbb{E}[\varphi^{\prime\prime}(\text{\it
JK}_{-i})({}_{1i}^{4}-{}_{1i}^{4})]}_{\mathbf{A}_{i}}-\underbrace{n^{-2}\mathbb{E}[\varphi^{\prime\prime}(\text{\it
JK}_{-i})(({}_{2i}^{a})^{2}-(\tilde{\Delta}_{2i}^{a})^{2})]}_{\mathbf{B}_{i}}$
$\displaystyle-\underbrace{2n^{-2}\mathbb{E}[\varphi^{\prime\prime}(\text{\it
JK}_{-i})({}_{2i}^{a}{}_{2i}^{b}-\tilde{\Delta}_{2i}^{a}\tilde{\Delta}_{2i}^{b})]}_{\mathbf{C}_{i}}+\underbrace{4n^{-3/2}\mathbb{E}[\varphi^{\prime\prime}(\text{\it
JK}_{-i})N_{-i}({}_{1i}^{3}-\tilde{\Delta}_{1i}^{3})]}_{\mathbf{D}_{i}}$
$\displaystyle+\underbrace{4n^{-3/2}\mathbb{E}[\varphi^{\prime\prime}(\text{\it
JK}_{-i})N_{-i}({}_{1i}{}_{2i}^{a}-\tilde{\Delta}_{1i}\tilde{\Delta}_{2i}^{a})]}_{\mathbf{E}_{i}}+\underbrace{2n^{-2}\mathbb{E}[\varphi^{\prime\prime}(\text{\it
JK}_{-i})({}_{1i}^{2}{}_{2i}-\tilde{\Delta}_{1i}^{2}\tilde{\Delta}_{2i})}_{\mathbf{F}_{i}}$
$\displaystyle+R_{i}+\tilde{R}_{i}$

where for some $\bar{\text{\it JK}}_{1i}$ and $\bar{\text{\it JK}}_{2i}$ we
can write

$\displaystyle R_{i}$
$\displaystyle=\mathbb{E}[\varphi^{\prime\prime\prime}(\bar{\text{\it
JK}}_{1i})\\{n^{-1/2}N_{-i}{}_{1i}+n^{-1}{}_{1i}^{2}+n^{-1}{}_{2i}\\}^{3}]$
$\displaystyle\tilde{R}_{i}$
$\displaystyle=\mathbb{E}[\varphi^{\prime\prime\prime}(\bar{\text{\it
JK}}_{2i})\\{n^{-1/2}N_{-i}\tilde{\Delta}_{1i}+n^{-1}\tilde{\Delta}_{1i}^{2}+n^{-1}\tilde{\Delta}_{2i}\\}^{3}]$

Applications of Lemmas F.1 and F.2, Cauchy-Schwarz, and the generalized Hölder
inequality,111$\mathbb{E}[|fgk|]^{3}\leq\mathbb{E}[|f|^{3}]\mathbb{E}[|g|^{3}]\mathbb{E}[|k|^{3}]$
will allow us to bound for a fixed constant $M$ that depends only on $c$,

$\displaystyle|\mathbf{A}_{i}|$
$\displaystyle\leq\frac{M}{n^{2}}L_{2}(\varphi)$
$\displaystyle|\mathbf{B}_{i}|$
$\displaystyle\leq\frac{Ma^{2}}{n^{2}}L_{2}(\varphi)$
$\displaystyle|\mathbf{C}_{i}|$
$\displaystyle\leq\frac{Ma^{2}}{n^{3/2}}L_{2}(\varphi)$
$\displaystyle|\mathbf{D}_{i}|$
$\displaystyle\leq\frac{M}{n^{3/2}}L_{2}(\varphi)$
$\displaystyle|\mathbf{E}_{i}|$ $\displaystyle\leq\frac{M(a\vee
1)}{n^{3/2}}L_{2}(\varphi)$ $\displaystyle|\mathbf{F}_{i}|$
$\displaystyle\leq\frac{Ma^{3}}{n^{3/2}}L_{2}(\varphi)$

and

$\displaystyle|R_{i}|+|\tilde{R}_{i}|$
$\displaystyle\leq\frac{M}{n^{3/2}}L_{3}(\varphi)+\frac{Ma^{3}}{n^{3}}L_{3}(\varphi)$

Combining these bounds and summing over $n$ gives the result.∎

######  Lemma A.2 (Gaussian Denominator Anti-Concentration).

Suppose that Assumptions 3.1 and 3.2 hold. Then for any sequence
$\delta_{n}\searrow 0$,

$\mathrm{Pr}(\tilde{D}\leq\delta_{n})\to 0$

###### Proof of Lemma A.2.

By Assumption 3.1, we know that $\kappa_{i}^{2}(\beta_{0})\in[c^{-1},c]$ for
all $i=1,\dots,n$ so that
$\tilde{D}\geq\frac{c^{-1}}{n}\sumop\slimits@_{i=1}^{n}(\sumop\slimits@_{j=1}^{n}\tilde{h}_{ij}r_{j})^{2}$.
Then

$\displaystyle\mathrm{Pr}(\tilde{D}\leq\delta_{n})$
$\displaystyle\leq\mathrm{Pr}\big{(}\frac{1}{cn}\sumop\slimits@_{i=1}^{n}\big{(}\sumop\slimits@_{j=1}^{n}\tilde{h}_{ij}\tilde{r}_{j}\big{)}^{2}\leq\tilde{\delta}_{n}\big{)}$
$\displaystyle=\mathrm{Pr}\big{(}\|\tilde{r}^{\prime}\bar{H}^{1/2}\|^{2}\leq\delta_{n}\big{)}$
(A.5)

where
$\tilde{r}\mathrel{\mathop{\ordinarycolon}}=(\tilde{r}_{1},\dots,\tilde{r}_{n})^{\prime}\in\mathbb{R}^{n}$
and
$\bar{H}\mathrel{\mathop{\ordinarycolon}}=\frac{1}{cn}\tilde{H}\tilde{H}^{\prime}\in\mathbb{R}^{n\times
n}$. $\bar{H}$ is symmetric and positive semidefinite so we can take
$\bar{H}^{1/2}$ to be its symmetric square root, which will also be symmetric
and positive semidefinite (and thus not necessarily equal to
$\sqrt{\frac{c}{n}}\tilde{H}$). I provide two bounds on (A.5), the first of
which corresponds to the strong identification setting while the second
corresponds to weak identification._First Bound._ Since $\delta_{n}\searrow 0$
we will eventually have that $\delta_{n}<c^{-1}/2$. When this happens we can
bound using Chebyshev’s inequality and
$c^{-1}<\mathbb{E}[r^{\prime}\bar{H}r]<c$:

$\displaystyle\mathrm{Pr}(\tilde{r}^{\prime}\bar{H}\tilde{r}\leq\delta_{n})$
$\displaystyle=\mathrm{Pr}(\tilde{r}^{\prime}\bar{H}\tilde{r}-\mathbb{E}[\tilde{r}^{\prime}\bar{H}\tilde{r}]\leq\delta_{n}-\mathbb{E}[\tilde{r}^{\prime}\bar{H}\tilde{r}])$
$\displaystyle\leq\mathrm{Pr}(\tilde{r}^{\prime}\bar{H}\tilde{r}-\mathbb{E}[r^{\prime}\bar{H}r]\geq\mathbb{E}[\tilde{r}^{\prime}\bar{H}\tilde{r}]-\delta_{n})$
$\displaystyle\leq\mathrm{Pr}(|\tilde{r}^{\prime}\bar{H}\tilde{r}-\mathbb{E}[r^{\prime}\bar{H}r]|\geq\frac{1}{2c})$
$\displaystyle\leq 2c\operatorname{Var}(r^{\prime}\bar{H}r)$ (A.6)

Under strong identification we will expect
$\operatorname{Var}(r^{\prime}\bar{H}r)\to 0$._Second Bound._ For the second
bound, we will directly use bounds on the density of Gaussian quadratic forms
from Götze et al. (2019). The vector $r^{\prime}\bar{H}^{1/2}$ is Gaussian
with covariance matrix ${}_{r}=\bar{H}^{1/2}\bm{R}\bar{H}^{1/2}$ where
$\bm{R}=\operatorname{diag}(\operatorname{Var}(r_{1}),\dots,\operatorname{Var}(r_{n}))$.
Let ${}_{1}=\sumop\slimits@_{k=1}^{n}\lambda_{k}^{2}({}_{r})$ and
${}_{2}=\sumop\slimits@_{k=2}^{n}\lambda_{k}^{2}({}_{r})$. By Assumption 3.2
and Lemma G.5, ${}_{2}/{}_{1}$ is bounded away from zero. Using Theorem H.4 we
can then bound for some constant $C>0$

$\begin{split}\mathrm{Pr}(\|r^{\prime}H\|^{1/2}\leq\delta_{n})\leq
C\delta_{n}{}_{1}^{-1}\end{split}$ (A.7)

_Combining Bounds._ To combine the bounds in (A.6) and (A.7), first write

$\operatorname{Var}(\tilde{r}^{\prime}\bar{H}\tilde{r})=2\text{trace}(\bm{R}\bar{H}\bm{R}\bar{H})+4\mu_{r}\bar{H}\bm{R}\bar{H}\mu_{r}$

for $\mu_{r}=\mathbb{E}[r]$. Using the fact that
$\bar{H}^{1/2}\bm{R}\bar{H}^{1/2}$ is symmetric positive definite we can
bound:

$\displaystyle\mu_{r}^{\prime}\bar{H}\bm{R}\bar{H}\mu_{r}$
$\displaystyle=(\mu_{r}^{\prime}\bar{H}^{1/2})^{\prime}(\bar{H}^{1/2}\bm{R}\bar{H}^{1/2})(\bar{H}^{1/2}\mu_{r})$
$\displaystyle\leq\lambda_{1}(\bar{H}^{1/2}\bm{R}\bar{H}^{1/2})\|\mu_{r}^{\prime}\bar{H}^{1/2}\|^{2}$
$\displaystyle=\sqrt{\lambda_{1}^{2}(\bar{H}^{1/2}\bm{R}\bar{H}^{1/2})}\|\mu_{r}^{\prime}\bar{H}^{1/2}\|^{2}$
$\displaystyle=\sqrt{\lambda_{1}(\bar{H}^{1/2}\bm{R}\bar{H}\bm{R}\bar{H}^{1/2})}\|\mu_{r}^{\prime}\bar{H}^{1/2}\|^{2}$
$\displaystyle\leq\sqrt{\text{trace}(\bar{H}^{1/2}\bm{R}\bar{H}\bm{R}\bar{H}^{1/2})}\|\mu_{r}^{\prime}\bar{H}^{1/2}\|^{2}$
$\displaystyle=\sqrt{\text{trace}(\bm{R}\bar{H}\bm{R}\bar{H})}\|\mu_{r}^{\prime}\bar{H}\|^{2}\leq
c^{2}{}_{1}^{1/2}$ (A.8)

where the first equality uses the symmetric square root of $\bar{H}$, the
first inequality comes from Courant-Fischer minmax principle and the third
equality uses the fact that the eigenvalues of $A^{2}$ are the squares of the
eigenvalues of $A$, for any generic symmetric matrix $A$. The second
inequality comes from the fact that a matrix times its transpose is always
positive semidefinite and that for $M$ psd,
$\lambda_{1}(M)\leq\sqrt{\text{trace}(M^{2})}$ since the trace is the sum of
the (weakly positive) eigenvalues. The final inequality uses
$\mu_{r}^{\prime}\bar{H}\mu_{r}=\frac{c}{n}\sumop\slimits@_{i=1}^{n}(\mathbb{E}[\tilde{\Pi}_{i}])^{2}\leq\frac{c}{n}\sumop\slimits@_{i=1}^{n}\mathbb{E}[(\tilde{\Pi}_{i})^{2}]\leq
c^{2}$.Combining (A.6), (A.7), and (A.8) gives us

$\mathrm{Pr}(\tilde{D}\leq\delta_{n})\leq
C\min\left\\{{}_{1}+{}_{1}^{1/2},\delta_{n}{}_{1}^{-1}\right\\}$ (A.9)

Regardless of the behavior of 1, this tends to zero as $\delta_{n}\to 0$.∎

######  Remark A.1 (Final Anticoncentration Bound).

To give an explicit bound on (A.9) in terms of $\delta_{n}$ we note that, if
$x^{\star}$ solves

$x^{\star}+\sqrt{x^{\star}}=\frac{c}{x^{\star}}$

then for any $x\geq 0$, $\min\\{x+\sqrt{x},c/x\\}\leq
x^{\star}+\sqrt{x^{\star}}$. Using this, notice that
$(x^{\star})^{2}+(x^{\star})^{3/2}=c$ so that $x^{\star}\leq\sqrt{c}$. This
allows us to bound (A.9)

$\displaystyle\mathrm{Pr}(\tilde{D}\leq\delta_{n})\leq
C\min\\{{}_{1}+{}_{1}^{1/2},\delta_{n}{}_{1}^{-1}\\}\leq
C(\delta_{n}^{1/2}+\delta_{n}^{1/4})$

######  Lemma A.3.

Let $X_{n}$ and $Y_{n}$ be two sequences of random variables and let
$W_{n}=X_{n}/Y_{n}$. Then for any $c\in\mathbb{R}$ and any $\delta>0$:

$\displaystyle\mathrm{Pr}(0\leq X_{n}-cY_{n}\leq\delta)$
$\displaystyle\leq\mathrm{Pr}(c\leq
W_{n}\leq\delta^{1/2}+c)+\mathrm{Pr}(Y_{n}\leq\delta^{1/2})$ and
$\displaystyle\mathrm{Pr}(-\delta\leq X_{n}-cY_{n}\leq 0)$
$\displaystyle\leq\mathrm{Pr}(c-\delta^{1/2}\leq W_{n}\leq
c)+\mathrm{Pr}(Y_{n}\leq\delta^{1/2})$

###### Proof.

Define the event $\Omega=\\{Y_{n}\geq\delta^{1/2}\\}$. We can bound

$\displaystyle\mathrm{Pr}(0\leq X_{n}-cY_{n}\leq\delta)$
$\displaystyle=\mathrm{Pr}(cY_{n}\leq X_{n}\leq\delta+cY_{n})$
$\displaystyle\leq\mathrm{Pr}(\\{cY_{n}\leq
X_{n}\leq\delta+cY_{n}\\}\cap\Omega)+\mathrm{Pr}({}^{c})$
$\displaystyle=\mathrm{Pr}(\\{c\leq
W_{n}\leq\delta/Y_{n}+c\\}\cap\Omega)+\mathrm{Pr}({}^{c})$
$\displaystyle\leq\mathrm{Pr}(c\leq
W_{n}\leq\delta^{1/2}+c)+\mathrm{Pr}({}^{c})$

The second statement of the lemma follows symmetrically.∎

######  Lemma A.4.

Suppose that $X_{n}$ and $Y_{n}$ are sequences of (real-valued) random
variables such that $Y_{n}=O_{p}(1)$ and for any $x\in\mathbb{R}$

$|\mathrm{Pr}(X_{n}\leq x)-\mathrm{Pr}(Y_{n}\leq x)|\to 0$

Then $X_{n}=O_{p}(1)$.

###### Proof.

Pick any $\epsilon>0$, and let $M_{\epsilon/2}$ be such that
$\mathrm{Pr}(Y_{n}>M_{\epsilon/2})\leq\epsilon/2$ for all $n\geq
N_{\epsilon}$. In addition, let $\tilde{N}_{\epsilon}$ be such that
$|\mathrm{Pr}(X_{n}\leq M_{\epsilon/2})-\mathrm{Pr}(Y_{n}\leq
M_{\epsilon/2})|\leq\epsilon/2$ for all $n\geq\tilde{N}_{\epsilon}$. Then for
all $n\geq N_{\epsilon}\vee\tilde{N}_{\epsilon/2}$,

$\displaystyle\mathrm{Pr}(X_{n}>M_{\epsilon/2})$
$\displaystyle\leq\mathrm{Pr}(Y_{n}>M_{\epsilon/2})+|\mathrm{Pr}(X_{n}>M_{\epsilon/2})-\mathrm{Pr}(Y_{n}>M_{\epsilon/2})|$
$\displaystyle\leq\epsilon/2+|\mathrm{Pr}(Y_{n}\leq
M_{\epsilon/2})-\mathrm{Pr}(X_{n}\leq M_{\epsilon/2})|$
$\displaystyle\leq\epsilon/2+\epsilon/2=\epsilon$

∎

######  Lemma A.5.

Suppose that $X_{n}$ and $Y_{n}$ are sequences of (real-valued) random
variables such that $Y_{n}=O_{p}(1)$ and for any $\Delta\in\mathbb{R}$

$\sup_{x\leq\Delta}|\mathrm{Pr}(X_{n}\leq x)-\mathrm{Pr}(Y_{n}\leq x)|\to 0$

Then $\sup_{x\in\mathbb{R}}|\mathrm{Pr}(X_{n}\leq x)-\mathrm{Pr}(Y_{n}\leq
x)|\to 0$.

###### Proof.

Pick an $\epsilon>0$. By Lemma A.4, $X_{n}=O_{p}(1)$. Pick a constant
$M_{\epsilon/3}$ such that $\mathrm{Pr}(X_{n}>M_{\epsilon/3})\leq\epsilon/3$
and $\mathrm{Pr}(Y_{n}>M_{\epsilon/3})\leq\epsilon/3$. Then for any
$x\in\mathbb{R}$ we can bound $|\mathrm{Pr}(X_{n}\leq x)-\mathrm{Pr}(Y_{n}\leq
x)|$ by considering two cases:Case 1. If $x\leq M_{\epsilon/3}$, then,

$|\mathrm{Pr}(X_{n}\leq x)-\mathrm{Pr}(Y_{n}\leq x)|\leq\sup_{x\leq
M_{\epsilon/3}}|\mathrm{Pr}(X_{n}\leq x)-\mathrm{Pr}(Y_{n}\leq x)|$ (A.10)

by hypothesis, there is an $N_{\epsilon}$ such that for $n\geq N_{\epsilon}$
the RHS of (A.10) is less than $\epsilon$.Case 2. If $x>M_{\epsilon/3}$ we can
bound

$\displaystyle|\mathrm{Pr}(X_{n}\leq x)-\mathrm{Pr}(Y_{n}\leq x)|$
$\displaystyle\leq|\mathrm{Pr}(X_{n}\leq M_{\epsilon/3})-\mathrm{Pr}(Y_{n}\leq
M_{\epsilon/3})|$ $\displaystyle\;\;\;+|\mathrm{Pr}(M_{\epsilon/3}<X_{n}\leq
x)-\mathrm{Pr}(M_{\epsilon/3}<Y_{n}\leq x)|$
$\displaystyle\leq|\mathrm{Pr}(X_{n}\leq M_{\epsilon/3})-\mathrm{Pr}(Y_{n}\leq
M_{\epsilon/3})|+\epsilon/3+\epsilon/3$ (A.11)

By hypothesis, there is an $N_{\epsilon/3}$ such that $|\mathrm{Pr}(X_{n}\leq
M_{\epsilon/3})-\mathrm{Pr}(Y_{n}\leq N_{\epsilon/3})|\leq\epsilon/3$.WLOG
$N_{\epsilon/3}\geq N_{\epsilon}$. Combining the bounds in (A.10) and (A.11),
for any $n\geq N_{\epsilon/3}$ and any $x\in\mathbb{R}$,

$|\mathrm{Pr}(X_{n}\leq x)-\mathrm{Pr}(Y_{n}\leq x)|\leq\epsilon$

Since this holds for all $x$, this gives the result.∎

######  Lemma A.6 (Approximate Distribution).

Under Assumptions 3.1, 3.2 and 3.3

$\sup_{a\in\mathbb{R}}|\mathrm{Pr}(\text{\it JK}_{I}(\beta_{0})\leq
a)-\mathrm{Pr}(\text{\it JK}_{G}(\beta_{0})\leq a)|\to 0$

###### Proof of Lemma A.6.

First, fix a $\Delta\geq 0$ and consider any $a\leq\Delta$. As in Lemma A.2,
let
$\tilde{\varphi}(\cdot)\mathrel{\mathop{\ordinarycolon}}\mathbb{R}\to\mathbb{R}$
be three times continuously differentiable with bounded derivatives up to the
third order such that $\tilde{\varphi}(x)$ is 1 if $x\leq 0$,
$\tilde{\varphi}(x)$ is decreasing if $x\in(0,1)$, and $\tilde{\varphi}(x)$ is
zero if $x\geq 1$. Consider a sequence $\gamma_{n}\searrow 0$ slowly enough
such that $(\gamma_{n}^{-2}+\gamma_{n}^{-3})/\sqrt{n}\to 0$ and define
$\varphi_{n}(x)=\tilde{\varphi}(\frac{x}{\gamma_{n}})$.By Lemma A.1 we can
write for some constant $M$ that depends only on :

$\displaystyle\mathrm{Pr}(\text{\it JK}_{I}(\beta_{0})\leq
a)=\mathrm{Pr}(\text{\it JK}^{a}\leq 0)$
$\displaystyle\leq\mathbb{E}[\varphi_{n}(\text{\it JK}^{a})]$
$\displaystyle\leq\mathbb{E}[\varphi_{n}(\tilde{\text{\it
JK}}^{a})]+\frac{M}{\sqrt{n}}(\gamma_{n}^{2}+\gamma_{n}^{-3})$
$\displaystyle\leq\mathrm{Pr}(\tilde{\text{\it JK}}^{a}\leq
0)+\mathrm{Pr}(0\leq\tilde{N}^{2}-a\tilde{D}\leq\gamma_{n})+\frac{M}{\sqrt{n}}(\gamma_{n}^{2}+\gamma_{n}^{-3})$
Applying Lemma A.3 and $\\{\tilde{\text{\it JK}}^{a}\leq 0\\}=\\{\text{\it
JK}_{G}(\beta_{0})\leq a\\}$ gives: $\displaystyle\leq\mathrm{Pr}(\text{\it
JK}_{G}(\beta_{0})\leq
a)+\underbrace{\mathrm{Pr}(a\leq\tilde{N}^{2}/\tilde{D}\leq
a+\gamma_{n}^{1/2})}_{\mathbf{A}}$
$\displaystyle\;\;\;+\underbrace{\mathrm{Pr}(\tilde{D}\leq\gamma_{n}^{1/2})}_{\mathbf{B}}+\frac{M}{\sqrt{n}}(\gamma_{n}^{-2}+\gamma_{n}^{-3})$

By Lemma F.3, we can bound $\mathbf{A}\leq M\gamma_{n}^{1/2}$ while by Lemma
A.2 and Remark A.1, $\mathbf{B}\leq M\gamma_{n}^{1/4}$. Since $\gamma_{n}$ is
chosen such that $\frac{M}{\sqrt{n}}(\gamma_{n}^{-2}+\gamma_{n}^{-3})\to 0$ we
can conclude that $\mathrm{Pr}(\text{\it JK}_{I}(\beta_{0})\leq
a)\leq\mathrm{Pr}(\text{\it JK}_{G}(\beta_{0})\leq a)+o(1)$. A symmetric
argument with $\varphi_{n}(x)=\tilde{\varphi}(1-\frac{x}{\gamma_{n}})$ gives a
lower bound so that, in total

$\mathrm{Pr}(\text{\it JK}_{G}(\beta_{0})\leq
a)-\bm{e}\leq\mathrm{Pr}(\text{\it JK}_{I}(\beta_{0})\leq
a)\leq\mathrm{Pr}(\text{\it JK}_{G}(\beta_{0})\leq a)+\bm{e}$

where

$\bm{e}=M\big{(}\frac{\gamma_{n}^{-2}+\gamma_{n}^{-3}}{\sqrt{n}}+\gamma_{n}^{1/2}+\gamma_{n}^{1/4}\big{)}=o(1)$

Since the constant M depends only on , this gives us that for any fixed
$\Delta>0$

$\sup_{a\leq\Delta}\big{|}\mathrm{Pr}(\text{\it JK}_{I}(\beta_{0})\leq
a)-\mathrm{Pr}(\text{\it JK}_{G}(\beta_{0})\leq a)\big{|}\leq
C\big{(}\frac{\gamma_{n}^{-2}+\gamma_{n}^{-3}}{\sqrt{n}}+\gamma_{n}^{1/2}+\gamma_{n}^{1/4}\big{)}=o(1)$
(A.12)

where $C$ is a constant that depends only on .Noting that the numerator
$\text{\it JK}_{G}(\beta_{0})$ is $O_{p}(1)$ under Assumption 3.3 while the
inverse of the denominator of $\text{\it JK}_{G}(\beta_{0})$ is $O_{p}(1)$ by
Lemma A.2, we can apply Lemma A.5. This step shows that the result in (A.12)
implies that the approximation error tends to zero uniformly over the real
line, which is the desired result. Optimizing over $\gamma_{n}$ in the
expression of (A.12) yields the rate of decay in Remark 3.3.∎

###  A.2 Proof of Proposition 3.1

###### Proof of Proposition 3.1.

As at the top of Section A.1, recall that $\tilde{h}_{ii}=0$, and define

$\displaystyle N$
$\displaystyle=\frac{1}{\sqrt{n}}\sumop\slimits@_{i=1}^{n}\epsilon_{i}(\beta_{0})\sumop\slimits@_{j=1}^{n}\tilde{h}_{ij}r_{j}$
$\displaystyle D$
$\displaystyle=\frac{1}{n}\sumop\slimits@_{i=1}^{n}\epsilon_{i}^{2}(\beta_{0})(\sumop\slimits@_{j=1}^{n}\tilde{h}_{ij}r_{j})^{2}$

where $\tilde{h}_{ij}=s_{n}h_{ij}$. The goal is to show that
$\mathrm{Pr}(\text{\it JK}_{I}(\beta_{0})\leq a)\to 0$ for any fixed
$a\in\mathbb{R}_{+}$. The event $\\{\text{\it JK}_{I}(\beta_{0})\leq a\\}$ is
equivalently expressed $\\{N^{2}-aD\leq 0\\}$ so that $\mathrm{Pr}(\text{\it
JK}(\beta_{0})\leq a)=\mathrm{Pr}(N^{2}-aD\leq 0)$.Under Assumptions 3.1 and
3.2, $aD=O_{p}(1)$ so by Lemma A.8 it suffices to show that
$\mathrm{Pr}(|N|\leq M)\to 0$ for any fixed $M\geq 0$. By assumption
$P=\mathbb{E}[N^{2}]\to\infty$ so we move to show that
$\operatorname{Var}(N)=O(1)$ and then apply Lemma A.7 to conclude. To this
end, recall the definition of
$\eta_{i}=\epsilon_{i}(\beta_{0})-\mathbb{E}[\epsilon_{i}(\beta_{0})]$, define
$\mu_{i}=\mathbb{E}[\epsilon_{i}(\beta_{0})]={}_{i}(\beta-\beta_{0})$, and let

$\displaystyle N_{1}$
$\displaystyle\coloneqq\frac{1}{\sqrt{n}}\sumop\slimits@_{i=1}^{n}\eta_{i}\sumop\slimits@_{j=1}^{n}\tilde{h}_{ij}r_{j}$
$\displaystyle N_{2}$
$\displaystyle\coloneqq\frac{1}{\sqrt{n}}\sumop\slimits@_{i=1}^{n}\mu_{i}\sumop\slimits@_{j=1}^{n}\tilde{h}_{ij}r_{j}$

Notice that $N=N_{1}+N_{2}$. To show that $\operatorname{Var}(N_{1})=O(1)$,
define $\bm{a}_{i}=\eta_{i}\sumop\slimits@_{j=1}^{n}\tilde{h}_{ij}r_{j}$.
Since $\mathbb{E}[\eta_{i}r_{i}]=0$, we have that
$\operatorname{Cov}(\bm{a}_{i},\bm{a}_{j})=0$ for $i\neq j$. Thus,

$\operatorname{Var}(N_{1})=\operatorname{Var}(\sumop\slimits@_{i=1}^{n}\bm{a}_{i}/\sqrt{n})=n^{-1}\sumop\slimits@_{i=1}^{n}\operatorname{Var}(\bm{a}_{i})=n^{-1}\sumop\slimits@_{i=1}^{n}\operatorname{Var}(\eta_{i})\mathbb{E}[(\sumop\slimits@_{j=1}^{n}\tilde{h}_{ij}r_{j})^{2}]\leq
c^{2}$

where the final inequality follows from an upper bound on
$\operatorname{Var}(\eta_{i})$ from Assumption 3.1 and by definition of
$\tilde{h}_{ij}=s_{n}h_{ij}$ from Assumption 3.2.To show that
$\operatorname{Var}(N_{2})=O(1)$ let
$\bm{b}_{i}=\sumop\slimits@_{j=1}^{n}\tilde{h}_{ji}\tilde{\Pi}_{j}(\beta-\beta_{0})$
and rewrite
$N_{2}=\frac{1}{\sqrt{n}}\sumop\slimits@_{i=1}^{n}r_{i}\bm{b}_{i}$. Under
Assumption 3.3(ii),
$|\bm{b}_{i}|=|\mathbb{E}[\sumop\slimits@_{j=1}^{n}\tilde{h}_{ji}\epsilon_{j}(\beta_{0})]|\leq
c^{1/2}$, so we can bound

$\operatorname{Var}(N_{2})=\operatorname{Var}(\sumop\slimits@_{i=1}^{n}r_{i}\bm{b}_{i}/\sqrt{n})=n^{-1}\sumop\slimits@_{i=1}^{n}\bm{b}_{i}^{2}\operatorname{Var}(r_{i})\leq
c^{2}$

Since $\operatorname{Var}(N)\leq
2\operatorname{Var}(N_{1})+2\operatorname{Var}(N_{2})$, we can conclude.∎

######  Lemma A.7.

Suppose that $X_{n}$ is a sequence of random variables such that
$\mathbb{E}[X_{n}^{2}]\to\infty$ while $\operatorname{Var}(X_{n})=O(1)$. Then,
for any $M\geq 0$, $\mathrm{Pr}(|X_{n}|\leq M)\to 0$.

###### Proof.

First, note that $\operatorname{Var}(|X_{n}|)\leq\operatorname{Var}(X_{n})$ so
$\operatorname{Var}(|X_{n}|)=O(1)$. Moreover
$\operatorname{Var}(|X_{n}|)=\mathbb{E}[X_{n}^{2}]-(\mathbb{E}[|X_{n}|])^{2}$,
so $\mathbb{E}[X_{n}^{2}]\to\infty$ and $\operatorname{Var}(|X_{n}|)=O(1)$
implies that $\mathbb{E}[|X_{n}|]\to\infty$. Then,

$\displaystyle\mathrm{Pr}(|X_{n}|\leq M)$
$\displaystyle=\mathrm{Pr}(|X_{n}|-\mathbb{E}[|X_{n}|]\leq
M-\mathbb{E}[|X_{n}|])$
$\displaystyle=\mathrm{Pr}(\mathbb{E}[|X_{n}|]-|X_{n}|\geq\mathbb{E}[|X_{n}|-M)$
$\displaystyle\leq\mathrm{Pr}(|\mathbb{E}[|X_{n}|]-|X_{n}||\geq\mathbb{E}[|X_{n}|]-M)$
$\displaystyle\leq\frac{\operatorname{Var}(|X_{n}|)}{\mathbb{E}[|X_{n}|]-M}$

Since $\operatorname{Var}(|X_{n}|)=O(1)$ but $\mathbb{E}[|X_{n}|]\to\infty$,
this tends to zero.∎

######  Lemma A.8.

Suppose that $X_{n}$ and $Y_{n}$ are random variables such that
$Y_{n}=O_{p}(1)$ and, for any $M\geq 0$, $\mathrm{Pr}(|X_{n}|\leq M)\to 0$.
Then, for any $M_{1}\geq 0$, $\mathrm{Pr}(X_{n}^{2}-Y_{n}\leq M_{1})\to 0$.

###### Proof.

Pick any $\epsilon>0$. We want to show that, eventually,
$\mathrm{Pr}(X_{n}^{2}-Y_{n}>M_{1})\geq 1-\epsilon$. Since $Y_{n}=O_{p}(1)$,
there is a fixed constant $M_{Y}$ such that $\mathrm{Pr}(|Y_{n}|\leq
M_{Y})\geq 1-\epsilon/2$. Since $\mathrm{Pr}(|X_{n}|\leq M)\to 0$ for any
$M\geq 0$, there exists an $N_{X}$ such that, for $n\geq N_{X}$,
$\mathrm{Pr}(X_{n}^{2}\leq M_{1}+M_{Y})\leq\epsilon/2$. A union bound
completes the argument (on the eventuality $n\geq N_{X}$):

$\displaystyle\mathrm{Pr}(X_{n}^{2}-Y_{n}>M)$
$\displaystyle\geq\mathrm{Pr}(X_{n}^{2}>M_{1}+M_{Y},|Y_{n}|\leq M_{Y})$
$\displaystyle=1-\mathrm{Pr}(\\{X_{n}^{2}<M_{1}+M_{Y}\\}\cup\\{|Y_{n}|>M_{Y}\\})$
$\displaystyle\geq 1-\epsilon/2-\epsilon/2=1-\epsilon$

∎

###  A.3 Proof of Lemma 3.2

###### Proof of Lemma 3.2.

For $N$ and $D$ defined at the top of Section A.1 define $\mathaccent
0{\symlargesymbols}062{N}=N+{}_{N}$ and $\mathaccent
0{\symlargesymbols}062{D}=D+{}_{D}$. We can then write $\text{\it
JK}(\beta_{0})=\mathaccent 0{\symlargesymbols}062{N}^{2}/\mathaccent
0{\symlargesymbols}062{D}$ and rewrite

$\text{\it JK}(\beta_{0})-\text{\it
JK}_{I}(\beta_{0})=\frac{2ND{}_{N}+D{}_{N}-N^{2}{}_{D}}{D^{2}+D{}_{D}}$

Apply Lemma F.2 to see that $N^{2}=O_{p}(1)$ while under Assumption 3.2,
$D=O_{p}(1)$. Thus, $2ND{}_{n}+D{}_{n}-N^{2}{}_{D}=o_{p}(1)$. Meanwhile, by
Lemma A.11, $\mathrm{Pr}(D^{2}\leq\delta_{n})\to 0$ for any sequence
$\delta_{n}\to 0$. Apply Lemma A.9 to conclude.∎

######  Lemma A.9.

Let $A_{n},B_{n}$ and $Y_{n}$ be sequences of random variables such that
$A_{n}=o_{p}(1)$ and $B_{n}=o_{p}(1)$. If $Y_{n}$ is such that for any
sequence $\delta_{n}\to 0$, $\mathrm{Pr}(|Y_{n}|\leq\delta_{n})\to 0$, then,

$\bigg{|}\frac{A_{n}}{Y_{n}+B_{n}}\bigg{|}=o_{p}(1)$

###### Proof.

Fix any $\epsilon>0$. We show that

$\bigg{|}\frac{A_{n}}{Y_{n}+B_{n}}\bigg{|}\leq\epsilon$

on an intersection of events whose probability tends to one. By Lemma G.1
there is a sequence $\epsilon_{n}\searrow 0$ such that

$\mathrm{Pr}(|A_{n}|\leq\epsilon_{n})\to 1\hbox{ }\text{ and }\hbox{
}\mathrm{Pr}(\epsilon|B_{n}|\leq\epsilon_{n})\to 1$

Consider the intersection of events ${}_{1}\cap{}_{2}\cap{}_{3}$ where

${}_{1}\mathrel{\mathop{\ordinarycolon}}=\\{\epsilon|Y_{n}|\geq
2\epsilon_{n}\\},\;\;{}_{2}\mathrel{\mathop{\ordinarycolon}}=\\{\epsilon|B_{n}|\leq\epsilon_{n}\\},\;\;{}_{3}\mathrel{\mathop{\ordinarycolon}}=\\{|A_{n}|\leq\epsilon_{n}\\}$

By assumption, $\mathrm{Pr}({}_{1}\cap{}_{2}\cap{}_{3})\to 1$. On this event
$|Y_{n}+B_{n}|\geq\epsilon_{n}/\epsilon>0$ and $|A_{n}|\leq\epsilon_{n}$ so
that
$|A_{n}/(Y_{n}+B_{n})|\leq|\epsilon_{n}/(\epsilon_{n}/\epsilon)|\leq\epsilon$.∎

######  Lemma A.10 (Denominator Interpolation).

Suppose that Assumptions 3.1 and 3.2 hold. Let
$\varphi(\cdot)\mathrel{\mathop{\ordinarycolon}}\mathbb{R}\to\mathbb{R}$ be
such that $\varphi(\cdot)\in C_{b}^{3}(\mathbb{R})$ with
$L_{2}(\varphi)=\sup_{x}|\varphi^{\prime\prime}(x)|$ and
$L_{3}(\varphi)=\sup_{x}|\varphi^{\prime\prime\prime}(x)|$. Then there is a
constant $M$ that depends only on the constant $c$ such that:

$|\mathbb{E}[\varphi(D)-\varphi(\tilde{D})]|\leq\frac{M}{\sqrt{n}}(L_{2}(\varphi)+L_{3}(\varphi))$

###### Proof of Lemma A.10.

We inherit the definitions of $D_{-i}$, ${}_{2i}^{a}$, ${}_{2i}^{b}$,
$\tilde{\Delta}_{2i}^{a}$, and $\tilde{\Delta}_{2i}^{b}$ from the proof of
Lemma A.1 with $a=1$. Then, as before we can write

$\displaystyle\mathbb{E}[\varphi(D)-\varphi(\tilde{D})]$
$\displaystyle=\sumop\slimits@_{i=1}^{n}\mathbb{E}[\varphi(D_{-i}+n^{-1}{}_{2i}^{a}+n^{-1}{}_{2i}^{b})]$
$\displaystyle\hphantom{=\sumop\slimits@_{i=1}^{n}}-\mathbb{E}[\varphi(D_{-i}+n^{-1}\tilde{\Delta}_{2i}^{a}+n^{-1}\tilde{\Delta}_{2i}^{b})]$

We examine each term via a second-order Taylor expansion around $D_{-i}$

$\displaystyle\mathbb{E}[\text{Term}_{i}]$
$\displaystyle=\frac{1}{n}\mathbb{E}[\varphi^{\prime}(D_{-i})\\{({}_{2i}^{a}-\tilde{\Delta}_{2i}^{a})+({}_{2i}^{b}-\tilde{\Delta}_{2i}^{b})\\}]$
$\displaystyle+\frac{1}{2n^{2}}\mathbb{E}[\varphi^{\prime\prime}(D_{-i})\\{(({}_{2i}^{a})^{2}-(\tilde{\Delta}_{2i}^{a})^{2})+2({}_{2i}^{a}{}_{2i}^{b}-\tilde{\Delta}_{2i}^{a}\tilde{\Delta}_{2i}^{b})+(({}_{2i}^{b})^{2}-({}_{2i}^{b})^{2})\\}]$
$\displaystyle+R_{i}+\tilde{R}_{i}$

where $R_{i}$ and $\tilde{R}_{i}$ are remainder terms to be analyzed later.
Using the restrictions in (A.4) we can simplify the above display:

$\displaystyle\mathbb{E}[\text{Term}_{i}]$
$\displaystyle=\underbrace{0.5n^{-2}\mathbb{E}[\varphi^{\prime\prime}(D_{-i})(({}_{2i}^{a})^{2}-(\tilde{\Delta}_{2i}^{a})^{2})]}_{\dot{\mathbf{A}}_{i}}+\underbrace{n^{-2}\mathbb{E}[\varphi^{\prime\prime}(K_{-i})({}_{2i}^{a}{}_{2i}^{b}-\tilde{\Delta}_{2i}^{a}\tilde{\Delta}_{2i}^{b})}_{\dot{\mathbf{B}}_{i}}$
$\displaystyle+R_{i}+\tilde{R}_{i}$

Using Lemma F.1 we can bound

$\displaystyle|\mathbf{A}_{i}|$
$\displaystyle\leq\frac{M}{n^{2}}L_{2}(\varphi)$
$\displaystyle|\mathbf{B}_{i}|$
$\displaystyle\leq\frac{M}{n^{3/2}}L_{2}(\varphi)$

For some $\bar{D}_{1i}$ and $\bar{D}_{2i}$ we can express

$\displaystyle R_{i}$
$\displaystyle=\mathbb{E}[\varphi^{\prime\prime\prime}(\bar{D}_{1i})\\{n^{-1}{}_{2i}^{a}+{}_{2i}^{b}\\}^{3}]\leq\frac{M}{n^{3/2}}L_{3}(\varphi)+\frac{M}{n^{3}}L_{3}(\varphi)$
$\displaystyle R_{i}$
$\displaystyle=\mathbb{E}[\varphi^{\prime\prime\prime}(\bar{D}_{2i})\\{n^{-1}\tilde{\Delta}_{2i}^{a}+\tilde{\Delta}_{2i}^{b}\\}^{3}]\leq\frac{M}{n^{3/2}}L_{3}(\varphi)+\frac{M}{n^{3}}L_{3}(\varphi)$

where the inequalities again come from applications of Lemma F.1. Combining
these bounds and summing over the $n$ terms gives the result.∎

######  Lemma A.11 (Denominator anti-concentration).

Suppose that Assumptions 3.1 and 3.2 hold. Then, for any sequence
$\delta_{n}\searrow 0$,

$\mathrm{Pr}(D\leq\delta_{n})\to 0$

###### Proof of Lemma A.11.

Let
$\tilde{\varphi}(\cdot)\mathrel{\mathop{\ordinarycolon}}\mathbb{R}\to\mathbb{R}$
be three times continuously differentiable with bounded derivatives up to the
third order such that $\tilde{\varphi}(x)$ is 1 if $x\leq 0$,
$\tilde{\varphi}(x)$ is decreasing if $x\in(0,1)$, and $\tilde{\varphi}(x)$ is
zero if $x\geq 1$.Consider a second sequence $\gamma_{n}\searrow 0$ slowly
enough such that $(\gamma_{n}^{-2}+\gamma_{n}^{-3})/\sqrt{n}\to 0$. Take
$\varphi_{n}(x)=\tilde{\varphi}(\frac{x-\delta_{n}}{\gamma_{n}})$.By Lemma
A.10 and since $\tilde{\varphi}(\cdot)$ has bounded derivatives up to the
third order, there is a fixed constant $M_{1}>0$ that depends only on $c$ such
that

$\displaystyle\mathrm{Pr}(D\leq\delta_{n})$
$\displaystyle\leq\mathrm{Pr}(\tilde{D}\leq\delta_{n}+\gamma_{n})+\frac{M_{1}}{\sqrt{n}}(\gamma_{n}^{-2}+\gamma_{n}^{-3})$

Let $\gamma_{n}$ be a sequence tending to zero such that
$(\gamma_{n}^{-2}+\gamma_{n}^{-3})/\sqrt{n}\to 0$ and conclude by applying
Lemma A.2.∎

###  A.4 Proof of Lemma 3.3

For any $j=1,\dots,d_{b}$ define the matrix
$B_{j}=\operatorname{diag}(b_{j}(z_{1}),\dots,b_{j}(z_{n}))$ and collect
observations
$\epsilon(\beta_{0})=(\epsilon_{1}(\beta_{0}),\dots,\epsilon_{n}(\beta_{0}))^{\prime}\in\mathbb{R}^{n}$,
$r=(r_{1},\dots,r_{n})^{\prime}\in\mathbb{R}^{n}$,
$\hat{r}=(\hat{r}_{1},\dots,\hat{r}_{n})^{\prime}\in\mathbb{R}^{n}$, and
$\xi=(\xi_{1},\dots,\xi_{n})^{\prime}\in\mathbb{R}^{n}$. In addition, collect
$b_{\epsilon}=(b_{\epsilon 1},\dots,b_{\epsilon n})\in\mathbb{R}^{d_{b}\times
n}$ where $b_{\epsilon
i}=\epsilon_{i}(\beta_{0})b(z_{i})\in\mathbb{R}^{d_{b}}$. Finally, let
$\bm{H}=\frac{s_{n}}{\sqrt{n}}H$, $\tilde{H}=s_{n}H$ and
$\tilde{h}_{ij}=s_{n}h_{ij}$.Step 1: ${}_{N}\to_{p}0$. To show that
${}_{N}\to_{p}0$ write

N $\displaystyle=|\epsilon(\beta_{0})^{\prime}\bm{H}(\hat{r}-r)|$
$\displaystyle=|\epsilon(\beta_{0})^{\prime}\bm{H}(b_{\epsilon}^{\prime}\hat{\gamma}-b_{\epsilon}^{\prime}\gamma)-\epsilon(\beta_{0})^{\prime}\bm{H}\xi|$
$\displaystyle\leq\underbrace{\max_{1\leq j\leq
d_{b}}|\epsilon(\beta_{0})^{\prime}\bm{H}B_{j}\epsilon(\beta_{0})|\|\hat{\gamma}-\gamma\|_{1}}_{\mathbf{A}}+\underbrace{\|\epsilon(\beta_{0})^{\prime}\bm{H}\|_{2}\|\xi_{2}\|_{2}}_{\mathbf{B}}$

To bound $\mathbf{A}$ we move to apply Theorem H.1 to the quadratic form
$\epsilon(\beta_{0})^{\prime}(\bm{H}B_{j})\epsilon(\beta_{0})$. First notice
that, under Assumption 3.4(v), we have

$\|\mathbb{E}[\bm{H}b_{j}\epsilon(\beta_{0})]\|_{2}=\frac{1}{n}\sumop\slimits@_{i=1}^{n}(\mathbb{E}[s_{n}\sumop\slimits@_{j\neq
i}h_{ij}b(z_{j})\epsilon_{j}(\beta_{0})])^{2}\leq c^{2}$

In the notation of Theorem H.1 this give us an upper bound on
$\|\mathbb{E}f^{(1)}(X)\|_{\text{HS}}$. Next, Assumption 3.2 gives us that the
Frobenius norm of $\bm{H}=\frac{s_{n}}{\sqrt{n}}\bm{H}$ is bounded, since the
rows of $s_{n}H$ are square summable, $\sumop\slimits@_{j\neq
i}(s_{n}h_{ij})^{2}\leq c$ for all $i=1,\dots,n$. In the notation of Theorem
H.1 this gives us an upper bound on $\|\mathbb{E}f^{(2)}(X)\|_{\text{HS}}$.
Applying Theorem H.1 and a union bound then gives us that

$\max_{1\leq j\leq
d_{b}}|\epsilon(\beta_{0})^{\prime}\bm{H}B_{j}\epsilon(\beta_{0})-\mathbb{E}[\epsilon(\beta_{0})^{\prime}\bm{H}B_{j}\epsilon(\beta_{0})]|=O_{p}(\log^{2/a}(d_{b}))$
(A.13)

Since $\max_{1\leq j\leq
d_{b}}|\mathbb{E}[\epsilon(\beta_{0})^{\prime}\bm{H}B_{j}\epsilon(\beta_{0})]|\leq
c$ under Assumption 3.4(v), (A.13) gives that

$\max_{1\leq j\leq
d_{b}}|\epsilon(\beta_{0})^{\prime}\bm{H}B_{j}\epsilon(\beta_{0})|=O_{p}(\log^{2/a}(d_{b}))$

Since $\log^{2/a}(d_{b})\|\mathaccent
0{\symlargesymbols}062{\gamma}-\gamma\|_{1}\to_{p}0$ by assumption, this
yields that $\mathbf{A}\to_{p}0$.To bound $\mathbf{B}$ see that
$\|\epsilon(\beta_{0})^{\prime}\bm{H}\|_{2}=\frac{s_{n}^{2}}{n}\sumop\slimits@_{i=1}^{n}(\sumop\slimits@_{j\neq
i}h_{ij}\epsilon_{i}(\beta_{0}))^{2}=O_{p}(1)$ under Assumption 3.3(ii) while
under Assumption 3.4 $\|\xi\|_{2}=o(1)$.Step 2: ${}_{D}\to_{p}0$. Notice that
$a^{2}-b^{2}=2b(a-b)+(a-b)^{2}$ and bound:

$\displaystyle|{}_{D}|$
$\displaystyle\leq\underbrace{\frac{1}{n}\sumop\slimits@_{i=1}^{n}\epsilon_{i}^{2}(\beta_{0})\big{|}\sumop\slimits@_{j\neq
i}\tilde{h}_{ij}r_{j}\big{|}}_{\mathbf{E}}\times\max_{i}|\sumop\slimits@_{j\neq
i}\tilde{h}_{ij}(\hat{r}_{j}-r_{j})|$
$\displaystyle+\underbrace{\frac{1}{n}\sumop\slimits@_{i=1}^{n}\epsilon_{i}^{2}(\beta_{0})}_{\mathbf{F}}\times\max_{i}|\sumop\slimits@_{j\neq
i}\tilde{h}_{ij}(\hat{r}_{j}-r_{j})|^{2}$

Since both $\mathbf{E}=O_{p}(1)$ and $\mathbf{F}=O_{p}(1)$ under Assumptions
3.1 and 3.2, it suffices to show that

$\max_{i}|\sumop\slimits@_{j\neq i}\tilde{h}_{ij}(\mathaccent
0{\symlargesymbols}062{r}_{j}-r_{j})|\to_{p}0$

To do so write

$\displaystyle\max_{i}\big{|}\sumop\slimits@_{j\neq
i}\tilde{h}_{ij}\\{\hat{r}_{j}-r_{j}\\}\big{|}\leq\underbrace{\max_{\begin{subarray}{c}1\leq
i\leq n\\\ 1\leq j\leq d_{b}\end{subarray}}\big{|}\sumop\slimits@_{j\neq
i}\tilde{h}_{ij}b(z_{j})\epsilon_{j}(\beta_{0})\big{|}\|\hat{\gamma}-\gamma\|_{1}}_{\mathbf{A}}+\underbrace{\max_{\begin{subarray}{c}1\leq
i\leq n\\\ 1\leq j\leq d_{b}\end{subarray}}\big{|}\sumop\slimits@_{j\neq
i}\tilde{h}_{ij}b(z_{j})\xi_{j}\big{|}}_{\mathbf{B}}$

To bound $\mathbf{A}$, note that by Assumption 3.4(v)
$\max_{i,j}|\mathbb{E}[\sumop\slimits@_{j\neq
i}\tilde{h}_{ij}b(z_{j})\epsilon_{j}(\beta_{0})|\leq c$. Under Assumptions 3.2
and 3.4(ii), $\max_{i,j}\sumop\slimits@_{j\neq
i}\tilde{h}_{ij}^{2}b^{2}(z_{j})\leq c^{2}$ so we can apply Theorem H.1 and a
union bound to obtain that

$\max_{\begin{subarray}{c}1\leq i\leq n\\\ 1\leq j\leq
d_{b}\end{subarray}}\big{|}\sumop\slimits@_{j\neq
i}\tilde{h}_{ij}b(z_{j})\epsilon_{j}(\beta_{0})\big{|}=O_{p}(\log^{1/a}(d_{b}n))$

Along with the implied rate on $\|\hat{\gamma}-\gamma\|_{1}$ from Assumption
3.4(iv) this shows that $\mathbf{A}\to_{p}0$.To show that $\mathbf{B}\to 0$,
use Cauchy-Schwarz, $\sumop\slimits@_{j\neq
i}\tilde{h}_{ij}^{2}b^{2}(z_{j})\leq c$ for any $i,j$ by Assumptions 3.2 and
3.4(ii), and $\sumop\slimits@_{i=1}^{n}\xi_{i}^{2}=o(1)$ by Assumption
3.4(iii).

###  A.5 Proof of Theorem 3.1

Apply Lemma A.12 with $X_{n}=\text{\it JK}(\beta_{0})$, $Y_{n}=\text{\it
JK}_{I}(\beta_{0})$ and $Z_{n}=\text{\it JK}_{G}(\beta_{0})$. The density of
$Z_{n}$ is uniformly bounded by Lemma F.3.

######  Lemma A.12.

Let $X_{n}$, $Y_{n}$, and $Z_{n}$ be sequences of random variables such that
$|X_{n}-Y_{n}|\to_{p}0$, the distribution of $Z_{n}$ is absolutely continuous
with respect to Lebesgue measure and the density functions of $Z_{n}$ are
uniformly bounded and $\sup_{a\in\mathbb{R}}|\mathrm{Pr}(Y_{n}\leq
a)-\mathrm{Pr}(Z_{n}\leq a)|\to 0.$ Then
$\sup_{a\in\mathbb{R}}|\mathrm{Pr}(X_{n}\leq a)-\mathrm{Pr}(Z_{n}\leq a)|\to
0.$

###### Proof.

For any $a\in\mathbb{R}$ and $\epsilon>0$ we have that $\\{X_{n}\leq
a\\}\subseteq\\{Y_{n}\leq a+\epsilon\\}\cup\\{|X_{n}-Y_{n}|>\epsilon\\}$;
thus, by applying union bound and rearranging we obtain:

$\displaystyle\mathrm{Pr}(X_{n}\leq a)$
$\displaystyle\leq\mathrm{Pr}(Y_{n}\leq
a+\epsilon)+\mathrm{Pr}(|Y_{n}-X_{n}|>\epsilon)$
$\displaystyle\leq\mathrm{Pr}(Z_{n}\leq a+\epsilon)+|\mathrm{Pr}(Y_{n}\leq
a+\epsilon)-\mathrm{Pr}(Z_{n}\leq a+\epsilon)|$
$\displaystyle\;\;\;\;\;\;\;\;\;+\mathrm{Pr}(|Y_{n}-X_{n}|>\epsilon)$ so that
$\displaystyle\mathrm{Pr}(X_{n}\leq a)-\mathrm{Pr}(Z_{n}\leq a)$
$\displaystyle\leq\mathrm{Pr}(a<Z_{n}\leq a+\epsilon)+|\mathrm{Pr}(Y_{n}\leq
a+\epsilon)-\mathrm{Pr}(Z_{n}\leq a+\epsilon)|$
$\displaystyle\;\;\;\;\;\;\;\;\;+\mathrm{Pr}(|Y_{n}-X_{n}|>\epsilon)$

Let $\epsilon_{n}\to 0$ be a sequence tending to zero such that
$\mathrm{Pr}(|X_{n}-Y_{n}|>\epsilon_{n})\to 0$ (Lemma G.1). Applying a
supremum to the above display yields

$\displaystyle\sup_{a\in\mathbb{R}}\mathrm{Pr}(X_{n}\leq
a)-\mathrm{Pr}(Z_{n}\leq a)$
$\displaystyle\leq\sup_{a\in\mathbb{R}}\mathrm{Pr}(a<Z_{n}\leq
a+\epsilon_{n})$ $\displaystyle\;+\sup_{a\in\mathbb{R}}|\mathrm{Pr}(Y_{n}\leq
a+\epsilon_{n})-\mathrm{Pr}(Z_{n}\leq a+\epsilon_{n})|$
$\displaystyle\;\;+\mathrm{Pr}(|Y_{n}-X_{n}|>\epsilon_{n})$

The first term goes to zero as $\epsilon_{n}\to 0$ since $Z_{n}$ has a
uniformly bounded density; the second term goes to zero by
$\sup_{a\in\mathbb{R}}|\mathrm{Pr}(Y_{n}\leq a)-\mathrm{Pr}(Z_{n}\leq a)|\to
0$ and the third term goes to zero by definition of $\epsilon_{n}$ and
$|Y_{n}-X_{n}|\to_{p}0$.We can apply a symmetric argument to show that
$\sup_{a\in\mathbb{R}}\mathrm{Pr}(Z_{n}\leq a)-\mathrm{Pr}(X_{n}\leq a)\leq
o(1)$ which completes the claim of the lemma.∎

##  Appendix B Proofs of Results in Section 4

The statement of Theorem 4.1 relies on showing

$\displaystyle\sup_{(a_{1},a_{2})\in\mathbb{R}^{2}}\big{|}\mathrm{Pr}(\text{\it
JK}(\beta_{0})\leq a_{1},C\leq a_{2})-\mathrm{Pr}(\text{\it
JK}_{G}(\beta_{0})\leq a_{1},C_{G}\leq a_{2})\big{|}\to 0$ and
$\displaystyle\sup_{(a_{1},a_{2})\in\mathbb{R}^{2}}\big{|}\mathrm{Pr}(\text{\it
S}(\beta_{0})\leq a_{1},C\leq a_{2})-\mathrm{Pr}(\text{\it
S}_{G}(\beta_{0})\leq a_{1},C_{G}\leq a_{2})\big{|}\to 0$

In particular, since $(\text{\it JK}_{G}(\beta_{0})\perp C_{G})$ and
$(\text{\it S}_{G}(\beta_{0})\perp C_{G})$ under $H_{0}$, showing the above
will imply the test based on $T(\beta_{0};\tau)$ has asymptotic size $\alpha$
for any choice of cutoff $\tau$.The second line in the above display follows
imediately from Theorem H.5 after verifying Assumption H.2, below.The first
line in the top display relies on a joint interpolation of the infeasible
$\text{\it JK}_{I}(\beta_{0})$ test statistic and the infeasible conditioning
statistic $C_{I}$, which could be constructed if $\rho(z_{i})$ was known to
the researcher.

$C_{I}\coloneqq\max_{1\leq i\leq
n}\big{|}\frac{1}{\sqrt{n}}\sumop\slimits@_{i=1}^{n}h_{ij}r_{j}\big{/}(n^{-1}\sumop\slimits@_{i=1}^{n}h_{ij}^{2})^{1/2}\big{|}$
(B.1)

This joint interpolation argument is rather involved however, and deferred to
Appendix D. The interpolation argument for the conditioning statistic very
closely follows the results in Chernozhukov et al. (2013). The results of
Section 4 rely on showing that the difference between $C$ and $C_{I}$ can be
treated as negligible. This in turn reduces to verifying Assumption H.2, which
is done in Lemma B.1, below.

######  Lemma B.1.

Suppose that Assumption 3.4 holds. Then there are sequences
$\delta_{n}\searrow 0$, $\beta_{n}\searrow 0$ such that

$\mathrm{Pr}\big{(}\max_{i\in[n]}n^{-1}\sumop\slimits@_{j=1}^{n}\dot{h}_{ij}^{2}(\mathaccent
0{\symlargesymbols}062{r}_{j}-r_{j})^{2}>\delta_{n}^{2}/\log^{2}(n)\big{)}\leq\beta_{n}$

where $\dot{h}_{ij}=h_{ij}/(n^{-1}\sumop\slimits@_{j=1}^{n}h_{ij}^{2})^{1/2}$.

###### Proof.

In view of Lemma G.1 it suffices to show

$\max_{1\leq i\leq
n}\frac{1}{n}\sumop\slimits@_{j=1}^{n}\dot{h}_{ij}^{2}(\hat{r}_{i}-r_{i})^{2}=o_{p}(1/\log^{2}(n))$
(B.2)

Notice that we can bound

$\displaystyle\max_{1\leq i\leq
n}\frac{1}{n}\sumop\slimits@_{j=1}^{n}(\hat{r}_{i}-r_{i})^{2}$
$\displaystyle=\max_{1\leq i\leq n}\big{|}(\mathaccent
0{\symlargesymbols}062{\gamma}-\gamma)^{\prime}n^{-1}\sumop\slimits@_{j=1}^{n}\epsilon_{j}^{2}(\beta_{0})b(z_{i})b(z_{j})^{\prime}(\mathaccent
0{\symlargesymbols}062{\gamma}-\gamma)\big{|}$
$\displaystyle\;\;\;+\max_{1\leq i\leq
n}|n^{-1}\sumop\slimits@_{j=1}^{n}\dot{h}_{ij}^{2}\xi_{j}^{2}|$
$\displaystyle\leq\max_{\begin{subarray}{c}1\leq i\leq n\\\ 1\leq j,k\leq
d_{b}\end{subarray}}\underbrace{\big{|}n^{-1}\sumop\slimits@_{j=1}^{n}\epsilon_{j}^{2}(\beta_{0})b_{j}(z_{j})b_{k}(z_{j})\big{|}}_{\mathbf{A}_{ijk}}\|\mathaccent
0{\symlargesymbols}062{\gamma}-\gamma\|_{1}^{2}$
$\displaystyle\;\;+n^{-1/2}\max_{1\leq i\leq
n}(n^{-1}\sumop\slimits@_{j=1}^{n}\dot{h}_{ij}^{4})^{1/2}(\sumop\slimits@_{j=1}^{n}\xi_{j}^{4})^{1/2}$

Under Assumption 3.4(i,ii) each $\mathbf{A}_{ijk}$ is $\upsilon$-sub-
exponential by Theorem H.1 (that is $\|\mathbf{A}_{ijk}\|_{\psi_{\upsilon}}$
is bounded). An application of Lemma G.2 then yields that
$\max_{i,j,k}|\mathbf{A}_{ijk}|=O_{p}(\log^{1/\nu}(d_{b}n))$. Along with
Assumption 3.4(iv) this gives that
$\max_{i,j,k}|\mathbf{A}_{ijk}|\|\mathaccent
0{\symlargesymbols}062{\gamma}-\gamma\|_{1}=O_{p}(\log^{-3/(v\wedge
1)}(d_{b}n))=o_{p}(\log^{-2}(n))$. Meanwhile by definition of $\dot{h}_{ij}$,
$\max_{i}(n^{-1}\sumop\slimits@_{j=1}^{n}\dot{h}_{ij}^{4})^{1/2}=O(1)$ while
by Assumption 3.4(iii) $(\sumop\slimits@_{j=1}^{n}\xi_{j}^{4})^{1/2}=o(1)$.
Since $\log^{2}(n)/\sqrt{n}\to 0$ this shows (B.2).∎

###  B.1 Proof of Theorem 4.1

The first result in Theorem 4.1 with $\text{\it JK}(\beta_{0})$ and $C$
replaced with their infeasible analogs $\text{\it JK}_{I}(\beta_{0})$ and
$C_{I}$ follows from the argument in Appendix D. After verifying that
$|\text{\it JK}(\beta_{0})-\text{\it JK}(\beta_{0})|\to_{p}0$ via Lemma 3.3
and that Assumption H.2 is satisfied via Lemma B.1 follow the same steps as in
the proof of Belloni et al. (2018), Theorem 2.1 to see that approximation
result holds for the feasible $\text{\it JK}(\beta_{0})$ and $C$.For the
second statement, I show that the conditions of Theorem H.6 are satisfied. To
see that Assumption H.1(i,ii) is satisfied under Assumption 3.1 use (i) the
definition of
$\dot{h}_{ij}=h_{ij}\big{/}(n^{-1}\sumop\slimits@_{j=1}^{n}h_{ij}^{2})^{1/2}$;
(ii) that the variance of each $r_{j}$ is bounded away from zero and (iii)
that the fourth moments of $r_{j}$ are bounded from above. Assumption H.1(iii)
is satisfied with $B_{n}=\log^{1/\upsilon}(n)$ by Assumption 4.1(i,iii) and
Lemma G.2. Finally Assumption H.2 is satisfied by applying Lemma B.1. Apply
Theorem H.6 to conclude.

##  Appendix C Proofs of Results in Section 5

Throughout this section, define the scaled elements of the infeasible and
gaussian numerators and denominators

$\displaystyle N_{\ell}$
$\displaystyle=\frac{s_{n,\ell}}{\sqrt{n}}\sumop\slimits@_{i=1}^{n}\epsilon_{i}(\beta_{0})\sumop\slimits@_{j=1}^{n}h_{ij}r_{j}$
$\displaystyle\tilde{N}_{\ell}$
$\displaystyle=\frac{s_{n,\ell}}{\sqrt{n}}\sumop\slimits@_{i=1}^{n}\tilde{\epsilon}_{i}(\beta_{0})\sumop\slimits@_{j=1}^{n}h_{ij}\tilde{r}_{j}$
$\displaystyle D_{\ell k}$
$\displaystyle=\frac{s_{\ell,n}s_{m,k}}{n}\sumop\slimits@_{i=1}^{n}\epsilon_{i}^{2}(\beta_{0})(\sumop\slimits@_{j=1}^{n}h_{ij}r_{\ell
j})(\sumop\slimits@_{j=1}^{n}h_{ij}r_{kj})$ $\displaystyle\tilde{D}_{\ell k}$
$\displaystyle=\frac{s_{\ell,n}s_{m,k}}{n}\sumop\slimits@_{i=1}^{n}\epsilon_{i}^{2}(\beta_{0})(\sumop\slimits@_{j=1}^{n}h_{ij}\tilde{r}_{\ell
j})(\sumop\slimits@_{j=1}^{n}h_{ij}\tilde{r}_{kj})$

Collect these in $N=(N_{1},\dots N_{d_{x}})^{\prime}\in\mathbb{R}^{d_{x}}$,
$\tilde{N}=(\tilde{N}_{1},\dots,\tilde{N}_{d_{x}})^{\prime}\in\mathbb{R}^{d_{x}}$,
$D=[D_{\ell k}]_{\ell,k\in[d_{x}]}\in\mathbb{R}^{d_{x}\times d_{x}}$, and
$\tilde{D}=[\tilde{D}_{\ell k}]_{\ell,k\in[d_{x}]}\in\mathbb{R}^{d_{x}\times
d_{x}}$. After multiplying by scaling matrix
$\operatorname{diag}(s_{1,n},\dots,s_{d_{x},n})$ and the inverse of the
scaling matrix we rewrite the infeasible and gaussian test statistics

$\displaystyle\text{\it JK}_{I}(\beta_{0})$
$\displaystyle=N^{\prime}D^{-1}N\bm{1}_{\\{\lambda_{\min}(D)>0\\}}$
$\displaystyle\text{\it JK}_{G}(\beta_{0})$
$\displaystyle=\tilde{N}^{\prime}\tilde{D}^{-1}\tilde{N}$

These are the representations of the test statsitics we will largely work
through in this section.

###  C.1 Proof of Lemma 5.1

Lemma 5.1 follows immediately from the joint gaussian approximation argument
established in Appendix D.

###  C.2 Proof of Lemma 5.2

Define the matrix ${}_{D}=[({}_{D})_{\ell k}]_{\ell,k\in[d_{x}]}$ and the
vector ${}_{N}=[({}_{N})_{\ell}]_{\ell\in[d_{x}]}$ where

$\displaystyle({}_{D})_{\ell k}$
$\displaystyle\coloneqq\frac{s_{\ell,n}s_{k,n}}{n}\sumop\slimits@_{i=1}^{n}\epsilon_{i}^{2}(\beta_{0})\big{(}\mathaccent
0{\symlargesymbols}062{}_{\ell,i}\mathaccent
0{\symlargesymbols}062{}_{k,i}-\mathaccent
0{\symlargesymbols}062{}_{\ell,i}^{I}\mathaccent
0{\symlargesymbols}062{}_{k,i}^{I}\big{)}$ $\displaystyle({}_{N})_{\ell}$
$\displaystyle\coloneqq\frac{s_{\ell,n}}{\sqrt{n}}\sumop\slimits@_{i=1}^{n}\epsilon_{i}(\beta_{0})(\mathaccent
0{\symlargesymbols}062{}_{\ell,i}-\mathaccent
0{\symlargesymbols}062{}^{I}_{\ell,i})$

Under the conditions of Lemma 5.2 we have that $\|{}_{D}\|\to_{p}0$ and
$\|{}_{N}\|\to_{p}0$.Using this notation, we can write the infeasible version
of the test statistic as $\text{\it JK}^{I}(\beta_{0})=N^{\prime}D^{-1}N$
while the feasible version is written $\text{\it
JK}(\beta_{0})=(N+{}_{N})^{\prime}(D+{}_{D})^{-1}(N+{}_{N})$. Add and subtract
$D^{-1}$ to get

$\displaystyle\text{\it JK}(\beta_{0})$
$\displaystyle=\big{(}N+{}_{N}\big{)}^{\prime}\big{(}(D+{}_{D})^{-1}\pm
D^{-1}\big{)}\big{(}N+{}_{N}\big{)}$ $\displaystyle=\text{\it
JK}^{I}(\beta_{0})+N^{\prime}\big{(}(D+{}_{D})^{-1}-D^{-1}\big{)}N+{}_{N}\big{(}(D+{}_{D})^{-1}-D^{-1})N$
$\displaystyle\;\;+{}_{N}^{\prime}\big{(}(D+{}_{D})^{-1}-D^{-1}\big{)}{}_{N}+N^{\prime}D^{-1}{}_{N}+{}_{N}D^{-1}N+{}_{N}D^{-1}{}_{N}$

Via Lemma D.2 we have that $\|D^{-1}\|=(\lambda_{\min}(D))^{-1}=O_{p}(1)$ and
by assumption we have that ${}_{N}\to_{p}0$. It therefore suffices to show
that

$\|(D+{}_{D})^{-1}-D^{-1}\|\to_{p}0$ (C.1)

To do so, we can use the following equality from Horn and Johnson (2012), p.
381.

$\|(D+{}_{D})^{-1}-D^{-1}\|\leq\frac{\|D^{-1}\|^{2}\|{}_{D}\|}{1-\|D^{-1}{}_{D}\|}$

Since $\|D^{-1}\|=O_{p}(1)$ and ${}_{D}\to_{p}0$, this gives (C.1).

###  C.3 Proof of Theorem 5.1

Under Assumption 5.4, the conditions of Lemma 5.2 can be verified following
the same steps as the proof of Lemma 3.3. Combine Lemma 5.2 and Lemma 5.1 as
in the proof of Theorem 3.1 to conclude.

###  C.4 Proof of Theorem 5.2

Follows from the same argument as the proof of Theorem 5.1 using the joint
interpolation of $\text{\it JK}(\beta_{0})$ and $C$ established in Appendix D.

##  Appendix D Joint Gaussian Approximation of $\text{\it JK}(\beta_{0})$ and
$C$

The main results of Sections 4 and 5 rely on a joint interpolation of the
conditioning and testing statistics as well as a joint interpolation of the
conditioning and testing statistics. The joint interpolation of $\text{\it
JK}(\beta_{0})$ and the conditioning statistic $C$ is given in Section D.2
after introducing some notation in Section D.1. The joint gaussian
approximation of $S(\beta_{0})$ and $C$ follows immediately from results in
Belloni et al. (2018); Chernozhukov et al. (2017). The result is presented
below for the general form of the $\text{\it JK}(\beta_{0})$ statistic under
$H_{0}$ however the proof strategy is very similar when using the decomposed
form of $\text{\it JK}(\beta_{0})$ when $d_{x}=1$. This proof is available on
request.

###  D.1 Notation

#### Jackknife Statistic Definitions.

Define $\tilde{h}_{\ell,ij}=s_{n,\ell}h_{ij}$ for each $\ell=1,\dots,d_{x}$
and the scaled leave-one-out quasi-numerator and denominators

$\displaystyle U_{-i}$
$\displaystyle=\bigg{[}\frac{1}{\sqrt{n}}\sumop\slimits@_{j=1}^{n}\dot{\epsilon}_{j}(\beta_{0})\sumop\slimits@_{k\neq
i}\tilde{h}_{\ell,jk}\dot{r}_{\ell k}\bigg{]}_{1\leq\ell\leq
d_{x}}\in\mathbb{R}^{d_{x}}$ $\displaystyle D_{-i}$
$\displaystyle=\bigg{[}\frac{1}{n}\sumop\slimits@_{j=1}^{n}\ddot{\epsilon}_{i}^{2}(\beta_{0})\big{(}\sumop\slimits@_{k\neq
i}\tilde{h}_{\ell,ij}\dot{r}_{\ell j}\big{)}\big{(}\sumop\slimits@_{k\neq
i}\tilde{h}_{\ell,ij}\dot{r}_{mj}\big{)}\bigg{]}_{\begin{subarray}{c}1\leq\ell\leq
d\\\ 1\leq m\leq d_{x}\end{subarray}}\in\mathbb{R}^{d_{x}\times d_{x}}$

where $\dot{\epsilon}_{j}(\beta_{0})$ is equal to
$\tilde{\epsilon}_{j}(\beta_{0})$ if $j<i$ and equal to
$\epsilon_{j}(\beta_{0})$ if $j>i$, $\dot{r}_{\ell j}$ is equal to
$\tilde{r}_{\ell j}$ if $j<i$ and equal to $r_{j}$ if $j>i$, and
$\ddot{\epsilon}_{j}(\beta_{0})$ is equal to
$\mathbb{E}[\epsilon_{j}^{2}(\beta_{0})]$ if $j<i$ and equal to
$\epsilon_{j}(\beta_{0})$ if $j>i$. As in the proof of Lemma 3.1 while the
definitions of $\dot{\epsilon}_{j}(\beta_{0}),\dot{r}_{\ell j}$, and
$\ddot{\epsilon}_{j}(\beta_{0})$ depend on $i$ this dependence is suppressed
to conslidate notation and since we only consider one step deviations at a
time.Also define the one step deviations

Ui
$\displaystyle=\big{[}\epsilon_{i}(\beta_{0})\sumop\slimits@_{j=1}^{n}\tilde{h}_{\ell,ij}\dot{r}_{\ell
j}+r_{\ell
i}\sumop\slimits@_{j=1}^{n}\tilde{h}_{\ell,ji}\dot{\epsilon}_{j}(\beta_{0})\big{]}_{1\leq\ell\leq
d}\in\mathbb{R}^{d}$ $\displaystyle\tilde{\Delta}_{Ui}$
$\displaystyle=\big{[}\tilde{\epsilon}_{i}(\beta_{0})\sumop\slimits@_{j=1}^{n}\tilde{h}_{\ell,ij}\dot{r}_{\ell
j}+\tilde{r}_{\ell
i}\sumop\slimits@_{j=1}^{n}\tilde{h}_{\ell,ji}\dot{\epsilon}_{j}(\beta_{0})\big{]}_{1\leq\ell\leq
d}\in\mathbb{R}^{d}$ Di $\displaystyle=\underbrace{\big{[}({}_{Di}^{a})_{\ell
m}\big{]}_{\begin{subarray}{c}1\leq\ell\leq d\\\ 1\leq m\leq
d\end{subarray}}}_{{}_{Di}^{a}}+\underbrace{\big{[}({}_{Di}^{b})_{\ell
m}\big{]}_{\begin{subarray}{c}1\leq\ell\leq d\\\ 1\leq m\leq
d\end{subarray}}}_{{}_{Di}^{b}}$ $\displaystyle\tilde{\Delta}_{Di}$
$\displaystyle=\underbrace{\big{[}(\tilde{\Delta}_{Di}^{a})_{\ell
m}\big{]}_{\begin{subarray}{c}1\leq\ell\leq d\\\ 1\leq m\leq
d\end{subarray}}}_{\tilde{\Delta}_{Di}^{a}}+\underbrace{\big{[}(\tilde{\Delta}_{Di}^{b})_{\ell
m}\big{]}_{\begin{subarray}{c}1\leq\ell\leq d\\\ 1\leq m\leq
d\end{subarray}}}_{\tilde{\Delta}_{Di}^{b}}$

where

$\displaystyle({}_{Di}^{a})_{\ell m}$
$\displaystyle=\epsilon_{i}^{2}(\beta_{0})\big{(}\sumop\slimits@_{j=1}^{n}\tilde{h}_{\ell,ij}r_{\ell
j}\big{)}\big{(}\sumop\slimits@_{j=1}^{n}\tilde{h}_{\ell,ij}\dot{r}_{\ell
j}\big{)}\big{(}\sumop\slimits@_{j=1}^{n}h_{m,ij}r_{m,ij}\big{)}^{2}+r_{\ell
i}r_{ki}\sumop\slimits@_{j=1}^{n}\tilde{h}_{\ell,ij}\tilde{h}_{m,ij}\ddot{\epsilon}_{j}^{2}(\beta_{0})$
$\displaystyle(\tilde{\Delta}_{Di}^{a})_{\ell m}$
$\displaystyle=\tilde{\epsilon}_{i}^{2}(\beta_{0})\big{(}\sumop\slimits@_{j=1}^{n}\tilde{h}_{\ell,ij}r_{\ell
j}\big{)}\big{(}\sumop\slimits@_{j=1}^{n}\tilde{h}_{\ell,ij}\dot{r}_{\ell
j}\big{)}\big{(}\sumop\slimits@_{j=1}^{n}h_{m,ij}r_{m,ij}\big{)}^{2}+\tilde{r}_{\ell
i}\tilde{r}_{ki}\sumop\slimits@_{j=1}^{n}\tilde{h}_{\ell,ij}\tilde{h}_{m,ij}\ddot{\epsilon}_{j}^{2}(\beta_{0})$
$\displaystyle({}_{Di}^{b})_{\ell m}$ $\displaystyle=r_{\ell
i}\sumop\slimits@_{j=1}^{n}\ddot{\epsilon}_{j}^{2}(\beta_{0})\sumop\slimits@_{k\neq
i}\tilde{h}_{\ell,ji}\tilde{h}_{m,jk}\dot{r}_{mk}+r_{ki}\sumop\slimits@_{j=1}^{n}\ddot{\epsilon}_{j}^{2}(\beta_{0})\sumop\slimits@_{k\neq
i}\tilde{h}_{\ell,ji}\tilde{h}_{m,jk}\dot{r}_{\ell k}$
$\displaystyle(\tilde{\Delta}_{Di}^{b})_{\ell m}$
$\displaystyle=\tilde{r}_{\ell
i}\sumop\slimits@_{j=1}^{n}\ddot{\epsilon}_{j}^{2}(\beta_{0})\sumop\slimits@_{k\neq
i}\tilde{h}_{\ell,ji}\tilde{h}_{m,jk}\dot{r}_{mk}+\tilde{r}_{ki}\sumop\slimits@_{j=1}^{n}\ddot{\epsilon}_{j}^{2}(\beta_{0})\sumop\slimits@_{k\neq
i}\tilde{h}_{\ell,ji}\tilde{h}_{m,jk}\dot{r}_{\ell k}$

Notice that in this notation we can write the test statistic and gaussian test
statistics, after scaling by $\operatorname{diag}(s_{n,1},\dots,s_{n,d_{x}})$,
as

$\displaystyle C(\beta_{0})$
$\displaystyle=(U_{-1}+{}_{U1}/\sqrt{n})^{\prime}(D_{-1}+{}_{D1}/n)^{-1}(U_{-1}+{}_{U1}/\sqrt{n})\bm{1}\\{\lambda_{\text{min}}(D_{-1}+{}_{D1})^{-1})>0\\}$
$\displaystyle\tilde{C}(\beta_{0})$
$\displaystyle=(U_{-n}+\tilde{\Delta}_{Un}/\sqrt{n})^{\prime}(\tilde{D}_{-1}+\tilde{\Delta}_{D1}/n)^{-1}(U_{-n}+\tilde{\Delta}_{U1}/\sqrt{n})$

In this proof we will use these representations for the test statistics.
Finally define

$\displaystyle U$ $\displaystyle=U_{-1}+{}_{U1}/\sqrt{n}$
$\displaystyle=U_{-n}+\tilde{\Delta}_{Un}/\sqrt{n}$ $\displaystyle D$
$\displaystyle=D_{-1}+{}_{D1}/n$ $\displaystyle=D_{-n}+{}_{Dn}/n$

#### Conditioning Statistic Definitions.

Let $h_{\ell,ii}=0$ for any $\ell=1,\dots,d_{x}$ and $i=1,\dots,n$. Define
$\tilde{h}_{\ell,ij}=h_{\ell,ij}/\omega_{\ell i}$ for $\omega_{\ell
i}=n^{-1}\sumop\slimits@_{j=1}^{n}|h_{\ell,ij}|$. Also define the one-step
deviations:

Ci
$\displaystyle\coloneqq(\tilde{h}_{1,ji}r_{1i},-\tilde{h}_{1,ji}r_{1i},\dots,\tilde{h}_{d_{x},ji}r_{d_{x}i},-\tilde{h}_{d_{x},ji}r_{d_{x}i})_{1\leq
j\leq n}^{\prime}\in\mathbb{R}^{2nd_{x}}$ Ci
$\displaystyle\coloneqq(\tilde{h}_{1,ji}\tilde{r}_{1i},-\tilde{h}_{1,ji}\tilde{r}_{1i},\dots,\tilde{h}_{d_{x},ji}\tilde{r}_{d_{x}i},-\tilde{h}_{d_{x},ji}\tilde{r}_{d_{x}i})_{1\leq
j\leq n}^{\prime}\in\mathbb{R}^{2nd_{x}}$ And the leave-one-out vector
$\displaystyle C_{-i}$
$\displaystyle\coloneqq\frac{1}{\sqrt{n}}\sumop\slimits@_{j<i}\tilde{\Delta}_{Cj}+\frac{1}{\sqrt{n}}\sumop\slimits@_{j>i}{}_{Cj}\in\mathbb{R}^{2nd_{x}}$

Notice that $C=\max_{1\leq\iota\leq
2nd_{x}}(C_{-1}+\frac{1}{\sqrt{n}}{}_{C1})_{\iota}$ while
$\tilde{C}=\max_{1\leq\iota\leq 2nd_{x}}(C_{-n}+{}_{Cn})_{\iota}$.

#### Function Definitions.

As in Chernozhukov et al. (2013) consider the “smooth max” function,
$F_{\beta}\mathrel{\mathop{\ordinarycolon}}\mathbb{R}^{p}\to\mathbb{R}$
defined

$F_{\beta}(z)=\beta^{-1}\log\bigg{(}\sumop\slimits@_{i=1}^{n}\exp(\beta
z_{i})\bigg{)}$

which satisfies

$0\leq F_{\beta}(z)-\max_{1\leq i\leq n}z_{i}\leq\beta^{-1}\log p.$

Section F.2 notes some useful properties of the smooth max function which we
will use in the joint interpolation argument. In addition let
$\varphi(\cdot)\in C_{b}^{3}(\mathbb{R})$ be such that $\varphi(x)=1$ if
$x\leq 0$, $\varphi^{\prime}(x)<0$ for $x\in(0,1)$, and $\varphi(x)=0$ for
$x\geq 1$. For any $\gamma>0$ and $a=(a_{1},a_{2})^{\prime}\in\mathbb{R}^{2}$
define the function
$\tilde{\varphi}(\cdot,\cdot,\cdot)\mathrel{\mathop{\ordinarycolon}}\mathbb{R}^{d_{x}}\times\text{vec}(\mathbb{R}^{d_{x}\times
d_{x}})\times\mathbb{R}^{2nd_{x}}\to\mathbb{R}$ via

$\tilde{\varphi}_{\gamma,a}(u,\text{vec}(d),c)\coloneqq\phi_{\gamma,a_{1}}(u,\text{vec}(d))\tau_{\gamma,a_{2}}(c)$
(D.1)

where

$\displaystyle\phi_{\gamma,a_{1}}(u,\text{vec}(d))$
$\displaystyle\coloneqq\varphi\left(\frac{u^{\prime}d^{-1}u-a_{1}}{\gamma\lambda_{\min}^{5}(d)}\right)$
$\displaystyle\tau_{\gamma,a}(c)$
$\displaystyle\coloneqq\varphi\left(\frac{F_{1/\gamma}(c)-a_{2}}{\gamma}\right)$

The function $\tilde{\varphi}_{\gamma,a}(\cdot,\cdot,\cdot)$ is meant to
approximate the indicator function $\bm{1}\\{K(\beta_{0})\leq
a_{1}\\}\bm{1}\\{C\leq a_{2}\\}$ with $\gamma$ governing the quality of
approximation. Where it is obvious, we will supress the subscripts $\gamma,a$
from our notation.

###  D.2 Main Argument

######  Lemma D.1 (Joint Lindeberg Interpolation).

Suppose that Assumptions 5.1, 5.2 and 5.3 hold. Then there is a fixed constant
$M$

$\left|\mathbb{E}[\tilde{\varphi}_{\gamma,a}(U,\text{vec}(D),C)-\tilde{\varphi}_{\gamma,a}(\tilde{U},\text{vec}(\tilde{D}),\tilde{C})]\right|\leq\frac{M_{1}\log^{M_{2}}(n)}{\sqrt{n}}(\gamma^{-1}+\gamma^{-2}+\gamma^{-3})$
(D.2)

###### Proof of Lemma D.1.

We can bound the difference on the left hand side of (D.2) using the
telescoping sum

$\begin{split}\sumop\slimits@_{i=1}^{n}\big{|}\mathbb{E}[\tilde{\varphi}_{\gamma,a}(U_{-i}&+{}_{Ui}/\sqrt{n},\text{vec}(D_{-i}+{}_{Di}/n),C_{-i}+{}_{Ci}/\sqrt{n})]\\\
&-\mathbb{E}[\tilde{\varphi}_{\gamma,a}(U_{-i}+{}_{Ui}/\sqrt{n},\text{vec}(D_{-i}+{}_{Di}/n),C_{-i}+{}_{Ci}/\sqrt{n})]\big{|}\end{split}$
(D.3)

By second degree Taylor expansion, we break each of the summands in (D.3) into
first order, second order, and remainder terms; each of which are bounded
below. We make use of the following moment conditions implied by (i)
indpendence of observations across $i=1,\dots,n$ and (ii) the mean and
covariance matrix of $(\epsilon_{i}(\beta_{0}),r_{i})$ being equal to the mean
and covariance matrix of $(\tilde{\epsilon}_{i}(\beta_{0}),r_{i})$

$\begin{split}0&=\mathbb{E}[{}_{Ui}-\tilde{\Delta}_{Ui}|\mathcal{F}_{-i}]=\mathbb{E}[{}_{Ui}{}_{Ui}^{\prime}-\tilde{\Delta}_{Ui}\tilde{\Delta}_{Ui}^{\prime}|\mathcal{F}_{-i}]=\mathbb{E}[\text{vec}({}_{Di})-\text{vec}(\tilde{\Delta}_{Di})|\mathcal{F}_{-i}]\\\
&=\mathbb{E}[{}_{Ci}-\tilde{\Delta}_{Ci}|\mathcal{F}_{-i}]=\mathbb{E}[{}_{Ui}\otimes\text{vec}({}_{Di}^{b})^{\prime}-\tilde{\Delta}_{Ui}\otimes\text{vec}(\tilde{\Delta}_{Di}^{b})^{\prime}|\mathcal{F}_{-i}]\\\
&=\mathbb{E}[{}_{Ci}\otimes{}_{Ui}-\tilde{\Delta}_{Ci}\otimes\tilde{\Delta}_{Ui}|\mathcal{F}_{-i}]=\mathbb{E}[{}_{Ci}\otimes\text{vec}(\tilde{\Delta}_{Di}^{b})-\tilde{\Delta}_{Ci}\otimes\text{vec}(\tilde{\Delta}_{Di}^{b})|\mathcal{F}_{-i}]\\\
&=\mathbb{E}[\text{vec}({}_{Di}^{b})\text{vec}({}_{Di}^{b})^{\prime}-\text{vec}(\tilde{\Delta}_{Di}^{b})\text{vec}(\tilde{\Delta}_{Di}^{b})^{\prime}|\mathcal{F}_{-i}]\end{split}$
(D.4)

where $\mathcal{F}_{-i}$ denotes the sub-sigma algebra generated by all
observations not equal to $i$, $\otimes$ denotes the Kronecker product, and I
apologize for the abuse of the equal sign in the above display.First Order
Terms.First order terms can be expressed

$\displaystyle\text{First Order}_{i}$
$\displaystyle=\sumop\slimits@_{\ell=1}^{d_{x}}\mathbb{E}\left[\frac{\partial}{\partial
U_{\ell}}\tilde{\varphi}(U_{-i},\text{vec}(D_{-i}),C_{-i})(({}_{Ui})_{\ell}-(\tilde{\Delta}_{Ui})_{\ell})\right]/\sqrt{n}$
$\displaystyle+\sumop\slimits@_{\ell=1}^{d_{x}}\sumop\slimits@_{m=1}^{d_{x}}\mathbb{E}\left[\frac{\partial}{\partial
D_{\ell m}}\tilde{\varphi}(U_{-i},\text{vec}(D_{-i}),C_{-i})(({}_{Di})_{\ell
m}-(\tilde{\Delta}_{Di})_{\ell m})\right]/n$
$\displaystyle+\sumop\slimits@_{\ell=1}^{2nd_{x}}\mathbb{E}\left[\frac{\partial}{\partial
C_{\ell}}\tilde{\varphi}(U_{-i},\text{vec}(D_{-i}),C_{-i})(({}_{Ci})_{\ell}-(\tilde{\Delta}_{Ci})_{\ell})\right]/\sqrt{n}$

These terms are all equal to zero after applying the matched moments in
(D.4).Second Order Terms. After canceling out terms using the matched moments
in (D.4) the second order terms that remain can be expressed

$\displaystyle\text{2nd Order}_{i}$
$\displaystyle=\frac{1}{n^{3/2}}\sumop\slimits@_{\ell=1}^{d_{x}}\sumop\slimits@_{m=1}^{d_{x}}\sumop\slimits@_{n=1}^{d_{x}}\underbrace{\mathbb{E}\bigg{[}\frac{\partial^{2}}{\partial
U_{\ell}\partial
D_{mn}}\tilde{\varphi}(U_{-i},\text{vec}(D_{-i}),C_{-i})(({}_{Ui})_{\ell}({}_{Di}^{a})_{mn}-(\tilde{\Delta}_{Ui})_{\ell}(\tilde{\Delta}_{Di}^{a})_{mn})\bigg{]}}_{\mathbf{A}_{\ell
mn}}$
$\displaystyle=\frac{1}{n^{2}}\sumop\slimits@_{\ell=1}^{d_{x}}\sumop\slimits@_{m=1}^{d_{x}}\sumop\slimits@_{n=1}^{d_{x}}\sumop\slimits@_{o=1}^{d_{x}}\underbrace{\mathbb{E}\bigg{[}\frac{\partial^{2}}{\partial
U_{\ell}\partial
D_{mn}}\tilde{\varphi}(U_{-i},\text{vec}(D_{-i}),C_{-i})(({}_{Di}^{a})_{\ell
m}({}_{Di}^{a})_{no}-(\tilde{\Delta}_{Di}^{a})_{\ell
m}(\tilde{\Delta}_{Di}^{a})_{no})\bigg{]}}_{\mathbf{B}_{\ell mno}}$
$\displaystyle=\frac{2}{n^{2}}\sumop\slimits@_{\ell=1}^{d_{x}}\sumop\slimits@_{m=1}^{d_{x}}\sumop\slimits@_{n=1}^{d_{x}}\sumop\slimits@_{o=1}^{d_{x}}\underbrace{\mathbb{E}\bigg{[}\frac{\partial^{2}}{\partial
U_{\ell}\partial
D_{mn}}\tilde{\varphi}(U_{-i},\text{vec}(D_{-i}),C_{-i})(({}_{Di}^{b})_{\ell
m}({}_{Di}^{a})_{no}-(\tilde{\Delta}_{Di}^{a})_{\ell
m}(\tilde{\Delta}_{Di}^{b})_{no})\bigg{]}}_{\mathbf{C}_{\ell mno}}$
$\displaystyle=\frac{1}{n^{3/2}}\sumop\slimits@_{\ell=1}^{2nd_{x}}\sumop\slimits@_{m=1}^{d_{x}}\sumop\slimits@_{n=1}^{d_{x}}\underbrace{\mathbb{E}\bigg{[}\frac{\partial^{2}}{\partial
C_{\ell}\partial
D_{mn}}\tilde{\varphi}(U_{-i},\text{vec}(D_{-i}),C_{-i})(({}_{Ci})_{\ell}({}_{Di}^{a})_{mn}-(\tilde{\Delta}_{Ci})_{\ell}(\tilde{\Delta}_{Di}^{a})_{mn})\bigg{]}}_{\mathbf{D}_{\ell
mn}}$

To bound each $\mathbf{A}_{\ell mn}$, $\mathbf{B}_{\ell mno}$, and
$\mathbf{C}_{\ell mno}$ we use the fact that the second order derivatives of
$\tilde{\varphi}$ are bounded up to a log power of $n$ via repeated
application of Lemmas F.12 and F.15. Under Assumption 5.1 the absolute value
of terms $({}_{Ui})_{\ell}$,$|{}_{Di}^{a}|_{mn}$, and
$({}_{Di}^{b}/\sqrt{n})_{no}$ can also be shown to have bounded third moments
via the exact same steps as in the proof of Lemma F.1. Putting these together
with generalized Holder’s inequality will yield a finite constants $M_{1}$ and
$M_{2}$ such that $|\mathbf{A}_{lmn}|\leq
M_{1}\log^{M_{2}}(n)(\gamma^{-1}+\gamma^{-2})$,$\mathbf{B}_{\ell mno}\leq
M_{1}\log^{M_{2}}(n)(\gamma^{-1}+\gamma^{-2})$, and $|\mathbf{C}_{\ell
mno}|\leq M_{1}\log^{M_{2}}(n)n^{1/2}(\gamma^{-1}+\gamma^{-2})$. To bound
$\mathbf{D}_{\ell mn}$ terms notice that

$\displaystyle\sumop\slimits@_{\ell=1}^{2nd_{x}}\mathbf{D}_{\ell mn}$
$\displaystyle=\sumop\slimits@_{\ell=1}^{2nd_{x}}\mathbb{E}\bigg{[}\frac{\partial}{\partial
D_{mn}}\phi(U_{-i},\text{vec}(D_{-i}))\frac{\partial}{\partial
C_{\ell}}\tau(C_{-i})(({}_{C-i})_{\ell}({}_{Di}^{a})_{mn}-(\tilde{\Delta}_{Ci})_{\ell}(\tilde{\Delta}_{Di}^{a})_{mn}))\bigg{]}$
Apply Lemma F.1 to bound ${}_{Di}^{a}$, and Lemmas F.12 and F.15 to bound the
derivative of $\phi(\cdot)$ and Cauchy-Schwarz to split up the Ci and Di terms
$\displaystyle\leq\sqrt{M_{1}\log^{M_{2}}(n)\gamma^{-2}}\mathbb{E}\bigg{[}\sumop\slimits@_{\ell=1}^{2nd_{x}}(\partial_{\ell}\tau(C_{-i}))^{2}(({}_{Ci})_{\ell}+(\tilde{\Delta}_{Ci})_{\ell})^{2}\bigg{]}^{1/2}$
$\displaystyle\leq\sqrt{M_{1}\log^{M_{2}}(n)\gamma^{-2}}\mathbb{E}\bigg{[}\max_{1\leq\ell\leq
n}(({}_{Ci})_{2\ell}+(\tilde{\Delta}_{Ci})_{2\ell})^{2}\sumop\slimits@_{\ell=1}^{2nd_{x}}(\partial_{\ell}\tau(C_{-i}))^{2}\bigg{]}^{1/2}$
By Lemma F.8 and chain rule we have that
$\sumop\slimits@_{\ell=1}^{2nd_{x}}(\partial_{\ell}\tau(C_{-i}))^{2}\leq\gamma^{-2}$.
Moreover $({}_{Ci})^{a/2}_{\ell}$ is sub-exponential so via Lemma G.2 the
second moment of the maximum is bounded by a power of $\log(n)$. After
updating the constant $M_{1}$ and $M_{2}$ this yields $\displaystyle\leq
M_{1}\log^{M_{2}}(n)\gamma^{-2}$

Putting these all together and summing over the remaining indices gives

$|\text{Second
Order}_{i}|\leq\frac{M_{1}\log^{M_{2}}(n)}{n^{3/2}}(\gamma^{-1}+\gamma^{-2})$
(D.5)

Remainder Terms. The first remainder term can be expressed

$\displaystyle\text{Remainder}_{i}$
$\displaystyle=\frac{1}{n^{3/2}}\sumop\slimits@_{\ell=1}^{d_{x}}\sumop\slimits@_{m=1}^{d_{x}}\sumop\slimits@_{n=1}^{d_{x}}\mathbb{E}\bigg{[}\frac{\partial^{3}}{\partial
U_{\ell}\partial U_{m}\partial
U_{n}}\tilde{\varphi}(\bar{U},\text{vec}(\bar{D}),\bar{C})({}_{Ui})_{\ell}({}_{Ui})_{m}({}_{Ui})_{n}\bigg{]}$
$\displaystyle+\frac{1}{n^{3}}\sumop\slimits@_{(\ell,m)}\sumop\slimits@_{(n,o)}\sumop\slimits@_{(q,p)}\mathbb{E}\bigg{[}\frac{\partial^{3}}{\partial
D_{\ell m}\partial D_{no}\partial
D_{pq}}\tilde{\varphi}(\bar{U},\text{vec}(\bar{D}),\bar{C})({}_{Di})_{\ell
m}({}_{Di})_{no}({}_{Di})_{qp}\bigg{]}$
$\displaystyle+\frac{1}{n^{3/2}}\sumop\slimits@_{\ell=1}^{2nd_{x}}\sumop\slimits@_{m=1}^{2nd_{x}}\sumop\slimits@_{n=1}^{2nd_{x}}\mathbb{E}\bigg{[}\frac{\partial^{3}}{\partial
C_{\ell}\partial C_{m}\partial
C_{n}}\tilde{\varphi}(\bar{U},\text{vec}(\bar{D}),\bar{C})({}_{Ci})_{\ell}({}_{Ci})_{m}({}_{Ci})_{n}\bigg{]}$
$\displaystyle+\frac{1}{n^{2}}\sumop\slimits@_{\ell=1}^{d_{x}}\sumop\slimits@_{m=1}^{d_{x}}\sumop\slimits@_{(n,o)}\mathbb{E}\bigg{[}\frac{\partial^{3}}{\partial
U_{\ell}\partial U_{m}\partial
D_{no}}\tilde{\varphi}(\bar{U},\text{vec}(\bar{D}),\bar{C})({}_{Ui})_{\ell}({}_{Ui})_{m}({}_{Di})_{no}\bigg{]}$
$\displaystyle+\frac{1}{n^{5/2}}\sumop\slimits@_{\ell=1}^{d_{x}}\sumop\slimits@_{(m,n)}\sumop\slimits@_{(o,p)}\mathbb{E}\bigg{[}\frac{\partial^{3}}{\partial
U_{\ell}\partial D_{mn}\partial
D_{op}}\tilde{\varphi}(\bar{U},\text{vec}(\bar{D}),\bar{C})({}_{Ui})_{\ell}({}_{Di})_{mn}({}_{Di})_{op}\bigg{]}$
$\displaystyle+\frac{1}{n^{5/2}}\sumop\slimits@_{\ell=1}^{2nd_{x}}\sumop\slimits@_{(m,n)}\sumop\slimits@_{(o,p)}\mathbb{E}\bigg{[}\frac{\partial^{3}}{\partial
C_{\ell}\partial D_{mn}\partial
D_{op}}\tilde{\varphi}(\bar{U},\text{vec}(\bar{D}),\bar{C})({}_{Ci})_{\ell}({}_{Di})_{mn}({}_{Di})_{op}\bigg{]}$
$\displaystyle+\frac{1}{n^{2}}\sumop\slimits@_{\ell=1}^{2nd_{x}}\sumop\slimits@_{m=1}^{2nd_{x}}\sumop\slimits@_{(n,o)}\mathbb{E}\bigg{[}\frac{\partial^{3}}{\partial
C_{\ell}\partial C_{m}\partial
D_{no}}\tilde{\varphi}(\bar{U},\text{vec}(\bar{D}),\bar{C})({}_{Ci})_{\ell}({}_{Ci})_{m}({}_{Di})_{no}\bigg{]}$
$\displaystyle+\frac{1}{n^{3/2}}\sumop\slimits@_{\ell=1}^{2nd_{x}}\sumop\slimits@_{m=1}^{2nd_{x}}\sumop\slimits@_{n=1}^{d_{x}}\mathbb{E}\bigg{[}\frac{\partial^{3}}{\partial
C_{\ell}\partial C_{m}\partial
U_{n}}\tilde{\varphi}(\bar{U},\text{vec}(\bar{D}),\bar{C})({}_{Ci})_{\ell}({}_{Ci})_{m}({}_{Ui})_{n}\bigg{]}$
$\displaystyle+\frac{1}{n^{2}}\sumop\slimits@_{\ell=1}^{2nd_{x}}\sumop\slimits@_{m=1}^{2nd_{x}}\sumop\slimits@_{n=1}^{d_{x}}\mathbb{E}\bigg{[}\frac{\partial^{3}}{\partial
C_{\ell}\partial C_{m}\partial
U_{n}}\tilde{\varphi}(\bar{U},\text{vec}(\bar{D}),\bar{C})({}_{Ci})_{\ell}({}_{Ci})_{m}({}_{Ui})_{n}\bigg{]}$

where $\bar{U},\text{vec}(\bar{D})$, and $\bar{C}$ vary term by term but are
always in the hyper-rectangles $[U_{-i},U+{}_{Ui}]$,
$[\text{vec}(D_{-i}),\text{vec}(D_{-i}+{}_{Di})]$, and
$[C_{-i},C_{-i}+{}_{Ci}]$, respectively. As such, any moment conditions that
apply to $U,D,C$ also apply to $(\bar{U},\bar{D},\bar{C})$.Repeated
application of generalized Hölder inequality, Lemma F.1 to bound moments of Ui
and $({}_{Di}/\sqrt{n})$, Lemma F.15 to bound moments of the second and third
derivatives of $\phi(\tilde{U},\text{vec}(\tilde{D}))$, Lemma F.11 to bound
the sums of derivatives of $\tau(\tilde{C})$, and Lemma G.2 to bound moments
of $\max_{1\leq\ell\leq n}({}_{Ci})_{\ell}$ will yield that

$|\text{Remainder}_{i}|\leq\frac{M_{1}\log^{M_{2}}(n)}{n^{3/2}}(\gamma^{-1}+\gamma^{-2}+\gamma^{-3})$
(D.6)

Symmetric logic will bound the other remainder term. Summing (D.5) and (D.6)
over indices gives the result.∎

######  Lemma D.2 (Denominator Anticoncentration).

Suppose that Assumptions 5.1, 5.2 and 5.3 hold. Then for any sequence
$\delta_{n}\to 0$ we have that
$\mathrm{Pr}(\lambda_{\min}(\tilde{D})\leq\tilde{\delta}_{n})\to 0$.

###### Proof.

By Lemma D.4 it suffices to show that for any fixed
$a\in\mathcal{S}^{d_{x}-1}$ and any $\delta_{n}\to 0$,
$\mathrm{Pr}(a^{\prime}Da\leq\delta_{n})\to 0$. For any such $a$ write:

$\displaystyle a^{\prime}\tilde{D}a$
$\displaystyle=\frac{1}{n}\sumop\slimits@_{i=1}^{n}\mathbb{E}[\epsilon_{i}^{2}(\beta_{0})]\big{(}\sumop\slimits@_{\ell=1}^{d_{x}}\sumop\slimits@_{j=1}^{n}a_{\ell}\tilde{h}_{\ell,ij}r_{\ell,j}\big{)}^{2}$
$\displaystyle\geq\frac{1}{cn}\sumop\slimits@_{i=1}^{n}\big{(}\sumop\slimits@_{\ell=1}^{d_{x}}\sumop\slimits@_{j=1}^{n}a_{\ell}\tilde{h}_{\ell,ij}r_{\ell,j}\big{)}^{2}$
Define
$\dot{s}_{n,j}=\max_{\\{\ell\mathrel{\mathop{\ordinarycolon}}a_{\ell}\neq
0\\}}s_{n,\ell}$ and $\dot{h}_{ij}=s_{n}h_{ij}$
$\displaystyle=\frac{1}{cn}\sumop\slimits@_{i=1}^{n}\big{(}\sumop\slimits@_{j=1}^{n}\dot{h}_{ij}\sumop\slimits@_{\ell=1}^{d_{x}}\frac{a_{\ell}s_{n,\ell}}{s_{n}}r_{\ell,j}\big{)}^{2}$

By Assumption 5.1 we have that
$\lambda_{\min}(\mathbb{E}[D])\geq\underline{c}$ so that
$\mathbb{E}[\frac{1}{n}\sumop\slimits@_{i=1}^{n}\big{(}\sumop\slimits@_{\ell=1}^{d_{x}}\sumop\slimits@_{j=1}^{n}a_{\ell}\tilde{h}_{\ell,ij}r_{\ell,j}\big{)}^{2}]\geq
c^{-1}$. Moreover, by Assumption 5.1,
$\operatorname{Var}(\sumop\slimits@_{\ell=1}^{d_{x}}\frac{a_{\ell}s_{n,\ell}}{s_{n}})$
is bounded from above and below. Define the matrix
$\tilde{H}=[\dot{h}_{ij}]_{ij}$ and follow the same steps as Lemma D.2 to
conclude.∎

######  Lemma D.3 (Gaussian Approximation).

Suppose that Assumptions 5.1, 5.2 and 5.3 hold. Then

$\sup_{a\in\mathbb{R}}\big{|}\mathrm{Pr}(\text{\it JK}_{I}(\beta_{0})\leq
a)-\mathrm{Pr}(\text{\it JK}_{G}(\beta_{0})\leq a)\big{|}\to 0$

###### Proof.

Let $a=(a_{1},a_{2})$ and $\tilde{\phi}_{\gamma,a}$ be as in (D.1):

$\displaystyle\mathrm{Pr}(N^{\prime}D^{-1}N\leq a_{1},C\leq a_{2})$
$\displaystyle\leq\mathbb{E}[\tilde{\phi}_{\gamma,a}(U,\text{vec}(D),C)]$
$\displaystyle\leq\mathbb{E}[\tilde{\phi}_{\gamma,a}(\tilde{U},\text{vec}(\tilde{D}),\tilde{C})]+\frac{M_{1}\log^{M}_{2}(n)}{\sqrt{n}}(\gamma^{-1}+\gamma^{-2})$
$\displaystyle\leq\mathrm{Pr}(\tilde{N}^{\prime}\tilde{D}^{-1}\tilde{N}\leq
a_{1},\tilde{C}\leq
a_{2})+\mathrm{Pr}(a_{1}\leq\tilde{N}^{\prime}\tilde{D}^{-1}N\leq
a_{1}+\gamma\lambda_{\min}^{5}(D))$ $\displaystyle\;\;\;+\mathrm{Pr}(a_{2}\leq
C\leq
a_{2}+\gamma)+\frac{M_{1}\log_{2}^{M_{2}}(n)}{\sqrt{n}}(\gamma^{-1}+\gamma^{-2}+\gamma^{-3})$
$\displaystyle\leq\mathrm{Pr}(\tilde{N}^{\prime}\tilde{D}^{-1}\tilde{N}\leq
a_{1},\tilde{C}\leq
a_{2})+\mathrm{Pr}(a_{1}\leq\tilde{N}^{\prime}\tilde{D}^{-1}N\leq
a_{1}+\gamma\lambda_{\min}^{5}(D))$ $\displaystyle\;\;\;+\mathrm{Pr}(a_{2}\leq
C\leq
a_{2}+\gamma)+\frac{M_{1}\log_{2}^{M_{2}}(n)}{\sqrt{n}}(\gamma^{-1}+\gamma^{-2}+\gamma^{-3})$

Let $\gamma\to 0$ at a rate such that
$\frac{\log^{M_{2}}(n)}{\sqrt{n}}\gamma^{-3}\to 0$ and apply Lemmas D.1 and
D.2 to conclude as in the proof of Lemma A.6. A symmetric argument shows that
the lower bound tends to zero.∎

######  Lemma D.4.

Let ${}_{n}\in\mathbb{R}^{d\times d}$ be a sequence of random positive-
semidefinite matrices. Suppose that for any fixed $a\in\mathcal{S}^{d-1}$ and
any $\delta_{n}\to 0$ we have that
$\mathrm{Pr}(a^{\prime}{}_{n}a\leq\delta_{n})\to 0$ and
$\mathrm{Pr}(\lambda_{\max}^{2}({}_{n})\geq\delta_{n}^{-1})\to 0$. Then for
any $\delta_{n}\to 0$,
$\mathrm{Pr}(\lambda_{\min}^{2}({}_{n})\leq\delta_{n})\to 0$.

###### Proof.

Take any preliminary sequence $\delta_{n}\to 0$. It suffices to show that
there is another sequence $\tilde{\delta}_{n}$ weakly larger than
$\delta_{n}/2$ such that
$\mathrm{Pr}(\lambda_{\min}^{2}({}_{n})\leq\tilde{\delta}_{n})\to 0$. For any
$m\in\mathbb{N}$ let $\mathcal{A}_{m}$ be a set of points in
$\mathcal{S}^{d-1}$ such that

$\max_{a\in\mathcal{S}^{d-1}}\min_{\tilde{a}\in\mathcal{A}_{m}}\|a-\tilde{a}\|\leq\delta_{m}^{2}$

From here let $\tilde{n}_{j}$ be defined

$\tilde{n}_{j}=\inf\\{n\geq
j\mathrel{\mathop{\ordinarycolon}}\min_{\tilde{a}\in\mathcal{A}_{n,j}}\mathrm{Pr}(\tilde{a}^{\prime}{}_{n}a\leq
2\delta_{n_{j}})<\delta_{n_{j}}\\}$

Define a new sequence $\tilde{\delta}_{n}\to 0$, weakly larger than
$\delta_{n}$, via

$\tilde{\delta}_{n}=\begin{cases}1&\text{if }0\leq 0\leq n<\tilde{n}_{1}\\\
\delta_{i}&\text{if }\tilde{n}_{i}\leq n<\tilde{n}_{i+1}\end{cases}$

and notice that, by definition
$\mathrm{Pr}(\min_{a\in\mathcal{A}_{\tilde{n}_{j}}}a^{\prime}{}_{n}a\leq
2\tilde{\delta}_{n})<\delta_{\tilde{n}_{j}}$.We wish to show that
$\lambda_{\min}^{2}({}_{n})>\tilde{\delta}_{n}$ on an intersection of events
whose probability tends to one. Since n is positive semi-definite,
$\|x\|^{2}_{{}_{n}}=x^{\prime}{}_{n}x$ defines a seminorm. By triangle
inequality

$\lambda_{\min}^{2}({}_{n_{j}})\geq\min_{\mathcal{A}_{n_{j}}}a^{\prime}{}_{n_{j}}a-\lambda_{\max}^{2}({}_{n})\tilde{\delta}_{n_{j}}^{2}$

Define the events

${}_{1}=\\{\min_{\mathcal{A}_{\tilde{n}_{j}}}a^{\prime}{}_{n}a\geq
2\tilde{\delta}_{n}\\}\hbox{ }\text{ and }\hbox{
}{}_{2}=\\{\lambda_{\max}({}_{n})\leq\tilde{\delta}_{n}^{-1/2}\\}$

On the intersection of these events, whose probabilities tend to one, we have
$\lambda_{\min}^{2}({}_{n})\geq\tilde{\delta}_{n}$.∎

##  Appendix E Incorporating Exogenous Controls

In this section, I analyze the model with exogeneous controls. To this end,
define the vector
$z_{2}=(z_{21}^{\prime},\dots,z_{2n}^{\prime})^{\prime}\in\mathbb{R}^{n\times
d_{c}}$. Let
$P_{2}=z_{2}(z_{2}^{\prime}z_{2})^{-1}z_{2}^{\prime}\in\mathbb{R}^{n\times n}$
denote the projection onto the column space of $z_{2}$ and $M_{2}=I_{n}-P_{2}$
denote the projection onto to orthocomplement of the column space. Focus will
be on the case where $d_{x}=1$ to simplify notation, but the basic concepts
apply generally to $d_{x}>1$.For
$y\coloneqq(y_{1},\dots,y_{n})^{\prime}\in\mathbb{R}^{n}$ and
$x\coloneqq(x_{1}^{\prime},\dots,x_{n}^{\prime})^{\prime}\in\mathbb{R}^{n\times}$
define $y^{\perp}\coloneqq M_{2}y$ and $x^{\perp}\coloneqq M_{2}x$ as the
“partialled out” versions of $y$ and $x$, respectively. Let $y^{\perp}_{i}$ be
the $i$th element of $y^{\perp}$ and $x^{\perp}_{i}$ be the $i$th element of
$x^{\perp}$. From here we can define $\epsilon(\beta_{0})\coloneqq
y-x\beta_{0}$, $\epsilon^{\perp}(\beta_{0})=M_{2}\epsilon(\beta_{0})$ and
$r^{\perp}\coloneqq M_{2}r$ where as in the main text
$r=(r_{1},\dots,r_{n})^{\prime}$ is constructed
$r_{i}=x_{i}-\rho(z_{i})\epsilon_{i}(\beta_{0})$. The definition of
$\rho(z_{i})$ does not change after partialling out $z_{2}$ since all
expectations are understood to be conditional on the instruments $z$. Notice
that $\epsilon^{\perp}(\beta_{0})$ is mean zero. Finally I assume that the
controls have been partialled out of hat matrix so that the effective hat
matrix is $M_{2}H$ and the vector $\mathaccent
0{\symlargesymbols}062{\Pi}\in\mathbb{R}^{n}$ is defined $\mathaccent
0{\symlargesymbols}062{\Pi}=(M_{2}H)(M_{2}r)$. This does not make a difference
for the numerator of the $\text{\it JK}(\beta_{0})$ statistic but does affect
the denominator slightly. When this is not done, inference may be
conservative.Using matrix notation in the numerator to make things clear, we
can write the version of the $\text{\it JK}(\beta_{0})$ statistic with the
partialled out vectors, $\epsilon^{\perp}(\beta_{0})$ and $r^{\perp}$, in
terms of the original vectors, $\epsilon(\beta_{0})$ and $r$,

$\displaystyle\text{\it JK}_{I}(\beta_{0})$
$\displaystyle=\frac{\big{(}\frac{1}{\sqrt{n}}\epsilon(\beta_{0})^{\prime}M_{2}\tilde{H}M_{2}r\big{)}^{2}}{\frac{1}{n}\sumop\slimits@_{i=1}^{n}(\epsilon_{i}^{\perp}(\beta_{0}))^{2}\big{(}\sumop\slimits@_{j=1}^{n}\bm{h}_{ij}r_{j}\big{)}^{2}}$
$\displaystyle=\frac{\big{(}\frac{1}{\sqrt{n}}\sumop\slimits@_{i=1}^{n}\epsilon_{i}(\beta_{0})\sumop\slimits@_{j=1}^{n}\bm{h}_{ij}r_{j}\big{)}^{2}}{\frac{1}{n}\sumop\slimits@_{i=1}^{n}(\epsilon_{i}^{\perp}(\beta_{0}))^{2}\big{(}\sumop\slimits@_{j=1}^{n}\bm{h}_{ij}r_{j}\big{)}^{2}}$

where $\bm{h}_{ij}=[M_{2}\tilde{H}M_{2}]_{ij}$, $\tilde{H}=s_{n}H$, and
$m_{ij}=[M_{2}]_{ij}$. I seek to characterize the limiting distribution of
$\text{\it JK}(\beta_{0})$ under $H_{0}$. To do so, we show that quantiles
$\text{\it JK}(\beta_{0})$ can be approximated by quantiles of the gaussian
analog statistic

$\displaystyle\text{\it
JK}_{G}(\beta_{0})=\frac{\big{(}\frac{1}{\sqrt{n}}\tilde{\epsilon}(\beta_{0})^{\prime}M_{2}\tilde{H}M_{2}\tilde{r}\big{)}^{2}}{\frac{1}{n}\sumop\slimits@_{i=1}^{n}\operatorname{Var}(\epsilon_{i})\big{(}\sumop\slimits@_{j=1}^{n}\bm{h}_{ij}\tilde{r}_{j}\big{)}^{2}}$

where $(\tilde{\epsilon}_{i},\tilde{\epsilon}_{i}(\beta_{0}),\tilde{r}_{i})$
are generated gaussian independent of the data and with the same mean and
covariance as $(\epsilon_{i},\epsilon_{i}(\beta_{0}),r_{i})$. Since
$\operatorname{Var}(\tilde{\epsilon}(\beta_{0}))=\operatorname{Var}(\epsilon_{i})$
under $H_{0}$, $\mathbb{E}[\tilde{\epsilon}(\beta_{0})^{\prime}M_{2}]=0$, and
$\tilde{r}\perp\tilde{\epsilon}(\beta_{0})$, this gaussian analog statistic
has a $\chi^{2}_{1}$ distribution conditional on any realization of
$\tilde{r}$ and thus its unconditional distribution is also
$\chi^{2}_{1}$.Showing that quantiles of $\text{\it JK}(\beta_{0})$ can be
approximated by quantiles of $\tilde{\text{\it JK}}(\beta_{0})$ proceeds in
two steps. In the first step, we show that $\text{\it JK}(\beta_{0})$
converges in probability to an intermediate statistic.

$\displaystyle\text{\it
JK}^{\text{int}}(\beta_{0})=\frac{\big{(}\frac{1}{\sqrt{n}}\sumop\slimits@_{i=1}^{n}\epsilon_{i}(\beta_{0})\sumop\slimits@_{j=1}^{n}\bm{h}_{ij}r_{j}\big{)}^{2}}{\frac{1}{n}\sumop\slimits@_{i=1}^{n}\epsilon_{i}^{2}(\sumop\slimits@_{j\neq
i}\bm{h}_{ij}r_{j})^{2}}$

We will then show that quantiles of this intermediate statistic can be
approximated by quantiles of $\tilde{\text{\it JK}}(\beta_{0})$. In view of
Lemma 3.2, it suffices to show for the first step that ${}_{D}\to_{p}0$, where

$\displaystyle{}_{D}=\frac{1}{n}\sumop\slimits@_{i=1}^{n}((\epsilon_{i}^{\perp}(\beta_{0}))^{2}-\epsilon_{i}^{2})\mathaccent
0{\symlargesymbols}062{}_{i}^{2}$

To do this, notice that under $H_{0}$ we can write
$\epsilon_{i}^{\perp}(\beta_{0})=\epsilon_{i}+z_{2i}^{\prime}(\mathaccent
0{\symlargesymbols}062{\Gamma}-\Gamma)$ where $\mathaccent
0{\symlargesymbols}062{\Gamma}=(z_{2}^{\prime}z_{2})^{-1}z_{2}\epsilon(\beta_{0})$
is a $\sqrt{n}$-consistent estimate of . Exploiting this fact we get

$\displaystyle{}_{D}=(\mathaccent
0{\symlargesymbols}062{\Gamma}-\Gamma)^{\prime}\frac{1}{n}\sumop\slimits@_{i=1}^{n}(\mathaccent
0{\symlargesymbols}062{}_{i})^{2}z_{2i}z_{2i}^{\prime}(\mathaccent
0{\symlargesymbols}062{\Gamma}-\Gamma)+2(\mathaccent
0{\symlargesymbols}062{\Gamma}-\Gamma)^{\prime}\frac{1}{n}\sumop\slimits@_{i=1}^{n}\epsilon_{i}z_{2i}\mathaccent
0{\symlargesymbols}062{}_{i}$

Both of these terms will tend to zero by the consistency $\mathaccent
0{\symlargesymbols}062{\Gamma}$ to , giving that ${}_{D}\to_{p}0$.In our
second step, we argue that quantiles of $\text{\it
JK}^{\text{int}}(\beta_{0})$ can be approximated by quantiles of $\text{\it
JK}_{G}(\beta_{0})$. To make this comparasion, we can follow almost exactly
the same steps as in Appendix A. The only difference between analysis in this
case and analysis in the original case is that the partialling out of controls
leads the test statistic to not strictly have a jackknife form; the effective
hat matrix $M_{2}HM_{2}$ no longer has a deleted diagonal. However, as I will
argue below, this will not make a difference in the interpolation argument
since the diagonal terms of $[P_{2}]_{ii}$ are small in the sense that they
sum to $d_{c}$.The (A.2) analog one step deviations for the numerator are
given

1i $\displaystyle=\epsilon_{i}(\beta_{0})\sumop\slimits@_{j\neq
i}\bm{h}_{ij}\dot{r}_{j}+r_{i}\sumop\slimits@_{j\neq
i}\bm{h}_{ji}\dot{\epsilon}_{j}(\beta_{0})+\bm{h}_{ii}\epsilon_{i}(\beta_{0})r_{i}$
$\displaystyle\tilde{\Delta}_{1i}$
$\displaystyle=\tilde{\epsilon}_{i}(\beta_{0})\sumop\slimits@_{j\neq
i}\bm{h}_{ij}\dot{r}_{j}+\tilde{r}_{j}\sumop\slimits@_{j\neq
i}\bm{h}_{ji}\dot{\epsilon}_{j}(\beta_{0})+\bm{h}_{ii}\tilde{\epsilon}_{i}(\beta_{0})\tilde{r}_{i}$

where as Appendix A, a dotted variable is equal to the gaussian analog if
$j>i$ but equal to the standard version otherwise.The first and second moments
of the first two terms in 1i can be matched with their gaussian analog terms
as in the proof of Lemma A.1. While we cannot match seconds moments of the
third term in the one step deviation, this sum of all these third terms can be
treated as negligible after scaling by $1/\sqrt{n}$ as
$\sumop\slimits@_{i=1}^{n}|\bm{h}_{ii}|\lesssim d_{c}$. This is because
$M_{2}\tilde{H}M_{2}=\tilde{H}-P_{2}\tilde{H}-\tilde{H}P_{2}-P_{2}\tilde{H}P_{2}$.
The matrix $\tilde{H}$ has zeros on it’s diagonal. Meanwhile

$\displaystyle|[P_{2}\tilde{H}]_{ii}|^{2}=\Big{|}\sumop\slimits@_{j=1}^{n}[P_{2}]_{ij}\tilde{H}_{ji}\Big{|}^{2}\leq\Big{(}\sumop\slimits@_{j=1}^{n}[P_{2}]_{ij}^{2}\Big{)}\Big{(}\sumop\slimits@_{j\neq
i}H_{ji}^{2}\Big{)}\lesssim[P_{2}]_{ii}$

where the final inequality comes because the matrix $P_{2}$ is symmetric and
idempotent and since $\Big{(}\sumop\slimits@_{j\neq
i}H_{ji}^{2}\Big{)}\lesssim 1$ by Assumption 3.2(ii). A similar argument can
be used to show that $[P_{2}\tilde{H}P_{2}]_{ii}^{2}\lesssim[P_{2}]_{ii}$.
Since $P_{2}$ is a projection matrix we must have that
$\|P_{2}He_{j}\|\leq\|He_{j}\|$ for any basis vector $e_{j}\in\mathbb{R}^{n}$.
Thus
$\sumop\slimits@_{j=1}^{n}[P_{2}H]_{ji}^{2}\leq\sumop\slimits@_{j=1}^{n}[H]_{ji}^{2}$.
Finally, we can use the fact that the trace of $P_{2}$ is equal to its rank to
show that $\sumop\slimits@_{i=1}^{n}|\bm{h}_{ii}|\lesssim d_{c}$The one step
deviations in the denominator can be bounded using the same logic. These one
step deviations are given

2i $\displaystyle=\epsilon_{i}^{2}(\sumop\slimits@_{j\neq
i}\bm{h}_{ij}\dot{r}_{j})^{2}+r_{i}^{2}\sumop\slimits@_{j\neq
i}\bm{h}_{ji}^{2}\ddot{\epsilon}_{j}^{2}+r_{i}\sumop\slimits@_{j\neq
i}\ddot{\epsilon}_{j}\big{(}\sumop\slimits@_{k\neq
j,i}\bm{h}_{ji}\bm{h}_{jk}r_{k}\big{)}$
$\displaystyle\;\;\;+\epsilon_{i}^{2}\big{(}\bm{h}_{ii}^{2}r_{i}^{2}+2\bm{h}_{ii}r_{j}\sumop\slimits@_{j\neq
i}\bm{h}_{ij}r_{j})^{2}$ $\displaystyle\tilde{\Delta}_{2i}$
$\displaystyle=\tilde{\epsilon}_{i}^{2}(\sumop\slimits@_{j\neq
i}\bm{h}_{ij}\dot{r}_{j})^{2}+\tilde{r}_{i}^{2}\sumop\slimits@_{j\neq
i}\bm{h}_{ji}^{2}\ddot{\epsilon}_{j}^{2}+\tilde{r}_{i}\sumop\slimits@_{j\neq
i}\ddot{\epsilon}_{j}\big{(}\sumop\slimits@_{k\neq
j,i}\bm{h}_{ji}\bm{h}_{jk}r_{k}\big{)}$
$\displaystyle\;\;\;+\epsilon_{i}^{2}\big{(}\bm{h}_{ii}^{2}r_{i}^{2}+2\bm{h}_{ii}r_{j}\sumop\slimits@_{j\neq
i}\bm{h}_{ij}r_{j})^{2}$

where $\ddot{\epsilon}_{j}$ is equal to $\operatorname{Var}(\epsilon_{j})$ if
$j<i$ and equal to $\epsilon_{j}$ if $j>i$. The first three terms in this
expansion are can be dealt with exactly as in the proof of Lemma A.1. The
fourth term is new, however summing over the fourth terms and scaling by $1/n$
will be negligible as $\sumop\slimits@_{i=1}^{n}|\bm{h}_{ii}|\lesssim d_{c}$.
After showing the lindeberg interpolation step, the rest of the proof follows
exactly as in Appendix A.

##  Appendix F Relevant Moment Bounds

###  F.1 Moment Bounds for Section 3

Here I provide some lemmas that are useful in the proof of Lemmas A.1–A.6

######  Lemma F.1.

Let
${}_{1i},\tilde{\Delta}_{1i},{}_{2i}^{a},\tilde{\Delta}_{2i}^{a},{}_{2i}^{b}$,$\tilde{\Delta}_{2i}^{b}$
be as in (A.2). Then under Assumptions 3.1 and 3.2 there is a constant $M>0$
such that for any $k=1,\dots,6$:

$\displaystyle\mathbb{E}[|{}_{1i}|^{k}]$ $\displaystyle\leq M$
$\displaystyle\mathbb{E}[|\tilde{\Delta}_{1i}|^{k}]$ $\displaystyle\leq M$ and
for any $k=1,\dots,3$: $\displaystyle\mathbb{E}[|{}_{2i}^{a}|^{k}]$
$\displaystyle\leq M\alpha^{k}$
$\displaystyle\mathbb{E}[|\tilde{\Delta}_{2i}^{k}|]$ $\displaystyle\leq
M\alpha^{k}$ $\displaystyle\mathbb{E}[|{}_{2i}^{b}/\sqrt{n}|^{k}]$
$\displaystyle\leq M\alpha^{k}$
$\displaystyle\mathbb{E}[|\tilde{\Delta}_{2i}^{b}/\sqrt{n}|^{k}]$
$\displaystyle\leq M\alpha^{k}$

###### Proof.

First, since

$\sumop\slimits@_{j=1}^{n}h_{ij}^{2}\mathbb{E}[(r_{j}-\mathbb{E}[r_{j}])^{2}\leq\mathbb{E}[(\sumop\slimits@_{i=1}^{n}\tilde{h}_{ij}r_{j})^{2}]\leq
1$

the constants are bounded, $\sumop\slimits@_{i=1}^{n}\tilde{h}_{ij}^{2}\leq
c$. Applying Lemma F.4 with $X_{i}=h_{ij}r_{j}$ and
$X_{i}=h_{ij}\epsilon_{j}(\beta_{0})$ we see that there is a constant $A$ such
that for any $k=1,\dots,6$

$\mathbb{E}\big{[}\big{|}\sumop\slimits@_{i=1}^{n}\tilde{h}_{ij}r_{j}\big{|}^{k}\big{]}\leq
A\hbox{ }\text{ and }\hbox{
}\mathbb{E}\big{[}\big{|}\sumop\slimits@_{i=1}^{n}\tilde{h}_{ij}\epsilon_{j}(\beta_{0})\big{|}^{k}\big{]}\leq
A$ (F.1)

The bounds on $\mathbb{E}[|{}_{1i}^{k}|]$ and
$\mathbb{E}[|\tilde{\Delta}_{1i}^{k}|]$ immediately follow from this result
and the bounds on moments of $r_{i}$ and $\epsilon_{i}(\beta_{0})$ in
Assumption 3.1.The bounds on $\mathbb{E}[|{}_{2i}^{a}|^{k}]$ and
$\mathbb{E}[|\tilde{\Delta}_{2i}^{a}|^{k}]$ also follow from (F.1) after
noting that there is a finite constant $B$ such that:

$\mathbb{E}[(\sumop\slimits@_{i=1}^{n}\tilde{h}_{ij}^{2}\epsilon_{i}^{2}(\beta_{0}))^{k}]\leq
B$

Finally to bound $\mathbb{E}[|{}_{2i}^{b}/\sqrt{n}|^{k}]$ and
$\mathbb{E}[|\tilde{\Delta}_{2i}^{b}/\sqrt{n}|^{k}]$ apply Lemma F.6 with
$v_{j}=\epsilon_{j}^{2}(\beta_{0})\sumop\slimits@_{k\neq
i,j}\tilde{h}_{jk}r_{k}$, noting that $\mathbb{E}[|v_{j}|^{3}]$ is bounded by
(F.1).∎

######  Lemma F.2.

Let $N$ and $N_{-i}$ be defined as in Section A.1. Under Assumptions 3.1, 3.2
and 3.3 there is a fixed constant $M$ such that for all $i=1,\dots,n$ and any
$k=1,\dots,6$,

$\mathbb{E}[|N|^{k}]+\mathbb{E}[|N_{-i}|^{k}]\leq M$

###### Proof.

We show the bound for $\mathbb{E}[|N|^{k}]$ and note that the bound for
$N_{-i}$ follows from symmetric logic. Write
$\epsilon_{i}(\beta_{0})=\eta_{i}+\gamma_{i}$ where
$\gamma_{i}={}_{i}(\beta-\beta_{0})$ and $\eta_{i}$ is mean zero. Decompose
$N=N_{1}+N_{2}+N_{3}$:

$N_{1}=\frac{1}{\sqrt{n}}\sumop\slimits@_{i=1}^{n}\eta_{i}\sumop\slimits@_{j=1}^{n}\tilde{h}_{ij}\dot{r}_{j},\;N_{2}=\frac{1}{\sqrt{n}}\sumop\slimits@_{i=1}^{n}r_{i}\sumop\slimits@_{j=1}^{n}\tilde{h}_{ji}\gamma_{j},\hbox{
}\text{ and }\hbox{
}N_{3}=\frac{1}{\sqrt{n}}\sumop\slimits@_{i=1}^{n}\eta_{i}\sumop\slimits@_{j=1}^{n}\tilde{h}_{ij}\mathbb{E}[r_{j}]$

where $\dot{r}_{j}=r_{j}-\mathbb{E}[r_{j}]$.Since via Assumption 3.2,
$\sumop\slimits@_{i=1}^{n}h_{ji}^{2}\leq c$ and via Assumption 3.1,
$|\gamma_{j}|\leq c$, we can bound,

$(\sumop\slimits@_{j=1}^{n}h_{ji}\gamma_{j}/\sqrt{n})^{4}\leq(\frac{c}{\sqrt{n}}\sumop\slimits@_{i=1}^{n}|h_{ji}|)^{4}\leq
c^{8}\implies(\sumop\slimits@_{j=1}^{n}h_{ji}\gamma_{j}/\sqrt{n})^{6}\leq
c^{8}(\sumop\slimits@_{j=1}^{n}h_{ji}\gamma_{j}/\sqrt{n})^{2}$

Under Assumption 3.3, $\mathbb{E}[N_{2}^{2}]\leq c$ while Assumption 3.2
implies that $(\sumop\slimits@_{i=1}^{n}h_{ij}\mathbb{E}[r_{j}])^{2}\leq c$ so
that $\mathbb{E}[N_{3}^{2}]\leq c^{2}$.An absolute bound on the higher moments
of $N_{2}$ then follows from an application of Lemma F.4 with
$X_{i}=r_{i}\sumop\slimits@_{j=1}^{n}h_{ji}\gamma_{j}/\sqrt{n}$. An absolute
bound on the higher moments of $N_{3}$ follows from symmetric logic.To bound
higher moments of $N_{1}$ define
$v_{i}=\sumop\slimits@_{j<i}\\{\eta_{i}h_{ij}r_{j}+\dot{r}_{i}h_{ji}\eta_{j}\\}$
and write $N_{1}=\frac{1}{\sqrt{n}}\sumop\slimits@_{i=2}^{n}v_{i}$. The
sequence $v_{2},\dots,v_{n}$ is a martingale difference array. Via the same
procedure as the bounds on $\mathbb{E}[|{}_{1i}|^{k}]$ as in Lemma F.1 one can
verify that there is a fixed constant $M$ such that
$\mathbb{E}[|v_{i}|^{k}]\leq M$ for all $k=1,\dots,6$. The bound on the higher
moments of $N$ then follows from Lemma F.7.The bounds for moments of $N_{-i}$
follow symmetric logic.∎

######  Lemma F.3.

Let $\tilde{N}$ and $\tilde{D}$ be defined as in Section A.1. Let
$f(\cdot,\tilde{r})$ be the density function of
$\frac{\tilde{N}}{\tilde{D}^{1/2}}|\tilde{r}$. Under Assumptions 3.1 and 3.3
there is a constant $M>0$ such that $\sup_{x}|f(x,\tilde{r})|\leq M$ for
almost all $\tilde{r}$.

###### Proof.

Recall that

$\tilde{N}=\frac{1}{\sqrt{n}}\sumop\slimits@_{i=1}^{n}\tilde{\epsilon}_{i}(\beta_{0})\sumop\slimits@_{j=1}^{n}\tilde{h}_{ij}\tilde{r}_{j}\hbox{
}\text{ and }\hbox{
}\tilde{D}^{1/2}=\sqrt{\frac{1}{n}\sumop\slimits@_{i=1}^{n}\kappa_{i}^{2}(\beta_{0})(\sumop\slimits@_{j=1}^{n}\tilde{h}_{ij}\tilde{r}_{j})^{2}}$

The distribution of $\tilde{\epsilon}_{i}(\beta_{0})|\tilde{r}_{i}$ is

$\tilde{\epsilon}_{i}(\beta_{0})|\tilde{r}\sim
N\left(\mu_{i}(r_{i}),(1-\rho_{i}^{2})\operatorname{Var}(\epsilon_{i}(\beta_{0}))\right)$

where
$\mu_{i}(r_{i})={}_{i}(\beta-\beta_{0})+\frac{\operatorname{Cov}(\epsilon_{i}(\beta_{0}),r_{i})}{\operatorname{Var}(r_{i})}(r_{i}-\mathbb{E}[r_{i}])$
and $\rho_{i}=\operatorname{corr}(\epsilon_{i}(\beta_{0}),r_{i})$. Define
$\bar{\Pi}_{i}\mathrel{\mathop{\ordinarycolon}}=\sumop\slimits@_{j=1}^{n}\tilde{h}_{ij}\tilde{r}_{j}$.
Then, conditional on $\tilde{r}$,

$\frac{\tilde{N}}{\tilde{D}^{1/2}}\sim
N\bigg{(}\frac{\frac{1}{\sqrt{n}}\sumop\slimits@_{i=1}^{n}\mu_{i}(r_{i})\bar{\Pi}_{i}}{\sqrt{\frac{1}{n}\sumop\slimits@_{i=1}^{n}\kappa_{i}^{2}(\beta_{0})\bar{\Pi}_{i}^{2}}},\frac{\frac{1}{n}\sumop\slimits@_{i=1}^{n}(1-\rho_{i}^{2})\operatorname{Var}(\epsilon_{i}(\beta_{0}))\bar{\Pi}_{i}^{2}}{\frac{1}{n}\sumop\slimits@_{i=1}^{n}\kappa_{i}^{2}(\beta_{0})\bar{\Pi}_{i}^{2}}\bigg{)}$
(F.2)

The maximum of the normal density is proportional to the inverse of the
standard deviation so it suffices to show that the variance in (F.2) is
bounded away from zero. To this end, notice that under Assumptions 3.1 and 3.3

$(1-\delta^{2})c^{-2}\leq(1-\rho_{i}^{2})\frac{\operatorname{Var}(\epsilon_{i}(\beta_{0}))}{\kappa_{i}^{2}(\beta_{0})}\leq
c^{2}$

By Lemma G.8 to this gives that the conditional variance is also larger than
$(1-\delta^{2})c^{-2}>0$.∎

######  Lemma F.4.

Let $X_{1},\dots,X_{n}$ be random variables such that
$\mathbb{E}[X_{i}]=\mu_{i}$ and
$\mathbb{E}[(\sumop\slimits@_{i=1}^{n}X_{i})^{2}]\leq C$. Suppose that for any
$i=1,\dots,n$ there is a constant $U$ such that

$\mathbb{E}[(X_{i}-\mu_{i})^{3}]\leq U\mathbb{E}[(X_{i}-\mu_{i})^{2}]\hbox{
}\text{ and }\hbox{ }\mathbb{E}[(X_{i}-\mu_{i})^{6}]^{1/3}\leq
U\mathbb{E}[(X_{i}-\mu_{i})^{2}]$

Then $\mathbb{E}[(\sumop\slimits@_{i=1}^{n}X_{i})^{6}]\leq
64U^{3}C^{3}+32C^{3}$.

###### Proof.

First write

$\displaystyle\mathbb{E}[(\sumop\slimits@_{i=1}^{n}X_{i})^{2}]=\sumop\slimits@_{i=1}^{n}\mathbb{E}(X_{i}-\mu_{i})^{2}+(\sumop\slimits@_{i=1}^{n}\mu_{i})^{2}\leq
C$

To bound $\mathbb{E}[(\sumop\slimits@_{i=1}^{n}X_{i})^{6}]$ expand out

$\displaystyle\mathbb{E}[(\sumop\slimits@_{i=1}^{n}X_{i})^{6}]$
$\displaystyle=\mathbb{E}[(\sumop\slimits@_{i=1}^{n}(X_{i}-\mu_{i})+\sumop\slimits@_{i=1}^{n}\mu_{i})^{6}]$
$\displaystyle\lesssim\mathbb{E}[(\sumop\slimits@_{i=1}^{n}(X_{i}-\mu_{i}))^{6}]+(\sumop\slimits@_{i=1}^{n}\mu_{i})^{6}$
$\displaystyle=\sumop\slimits@_{i=1}^{n}\mathbb{E}[(X_{i}-\mu_{i})^{6}]+\sumop\slimits@_{i=1}^{n}\sumop\slimits@_{j=1}^{n}\mathbb{E}[(X_{i}-\mu_{i})^{3}(X_{j}-\mu_{j})^{3}]$
$\displaystyle\;\;\;\;\;\;+\sumop\slimits@_{i=1}^{n}\sumop\slimits@_{j=1}^{n}\mathbb{E}[(X_{i}-\mu_{i})^{4}(X_{j}-\mu_{j})^{2}]$
$\displaystyle+\sumop\slimits@_{i=1}^{n}\sumop\slimits@_{j=1}^{n}\sumop\slimits@_{k\neq
i,j}\mathbb{E}[(X_{i}-\mu_{i})^{2}(X_{j}-\mu_{i})^{2}(X_{k}-\mu_{k})^{2}]+(\sumop\slimits@_{i=1}^{n}\mu_{i})^{6}$
$\displaystyle\leq\sumop\slimits@_{i=1}^{n}\mathbb{E}[(X_{i}-\mu_{i})^{6}]+\sumop\slimits@_{i=1}^{n}\sumop\slimits@_{j=1}^{n}\mathbb{E}[(X_{i}-\mu_{i})^{3}]\mathbb{E}[(X_{j}-\mu_{j})^{3}]$
$\displaystyle+\sumop\slimits@_{i=1}^{n}\sumop\slimits@_{j=1}^{n}\mathbb{E}[(X_{i}-\mu_{i})^{6}]^{4/6}\mathbb{E}[(X_{j}-\mu_{j})^{6}]^{2/6}$
$\displaystyle+\sumop\slimits@_{i=1}^{n}\sumop\slimits@_{j=1}^{n}\sumop\slimits@_{k\neq
i,j}\mathbb{E}[(X_{i}-\mu_{i})^{6}]^{1/3}\mathbb{E}[(X_{j}-\mu_{i})^{6}]^{1/3}\mathbb{E}[(X_{k}-\mu_{k})^{6}]^{1/3}$
$\displaystyle+C^{3}$
$\displaystyle=\bigg{(}\sumop\slimits@_{i=1}^{n}(\mathbb{E}[(X_{i}-\mu_{i})^{6}])^{1/3}\bigg{)}^{3}+\sumop\slimits@_{i=1}^{n}\sumop\slimits@_{j=1}^{n}\mathbb{E}[(X_{i}-\mu_{i})^{3}]\mathbb{E}[(X_{j}-\mu_{j})^{3}]+C^{3}$
$\displaystyle\leq\bigg{(}\sumop\slimits@_{i=1}^{n}(\mathbb{E}[(X_{i}-\mu_{i})^{6}])^{1/3}\bigg{)}^{3}+\bigg{(}\sumop\slimits@_{i=1}^{n}\mathbb{E}[(X_{i}-\mu_{i})^{3}]\bigg{)}^{2}+C^{3}$
$\displaystyle\leq
2U^{3}\bigg{(}\sumop\slimits@_{i=1}^{n}\mathbb{E}[(X_{i}-\mu_{i})^{2}]\bigg{)}^{3}+C^{3}$
$\displaystyle\leq 2U^{3}C^{3}+C^{3}$

where the implied constant in the second line is 32 by an application of Lemma
G.8, the third line comes from expanding out the power, the first inequality
by application of Hölder’s inequality, and the penultimate inequality comes
from applying bounds on the third and sixth central moments in terms of the
second moments.∎

######  Lemma F.5.

Let $h=(h_{1},\dots,h_{n})\in\mathbb{R}^{n}$ be such that
$\sumop\slimits@_{i=1}^{n}h_{i}^{2}\leq b$. Suppose that $X_{1},\dots,X_{n}$
are such that $\mathbb{E}[|X_{i}|^{k}]\leq M$ for all $k=1,2,3$. Then

$\mathbb{E}\big{[}\big{|}\sumop\slimits@_{i=1}^{n}h_{i}^{2}X_{i}\big{|}^{3}\big{]}\leq
b^{3}M^{3}$

###### Proof.

We can expand out

$\displaystyle\mathbb{E}\big{[}\big{|}\sumop\slimits@_{i=1}^{n}h_{i}^{2}X_{i}\big{|}^{3}]$
$\displaystyle\leq\sumop\slimits@_{i=1}^{n}\sumop\slimits@_{j=1}^{n}\sumop\slimits@_{k=1}^{n}h_{i}^{2}h_{j}^{2}h_{k}^{2}\mathbb{E}[|X_{i}||X_{j}||X_{k}|]$
$\displaystyle\leq
M^{3}\sumop\slimits@_{i=1}^{n}h_{i}^{2}\sumop\slimits@_{j=1}^{n}h_{j}^{2}\sumop\slimits@_{k=1}^{n}h_{k}^{2}$
$\displaystyle\leq M^{3}\big{(}\sumop\slimits@_{i=1}^{n}h_{i}^{2})^{3}\leq
c^{3}M^{3}$

∎

######  Lemma F.6.

Let $v_{1},\dots,v_{n}$ be random variables such that
$\mathbb{E}[|v_{i}|^{3}]\leq M$ for all $i=1,\dots,n$. Let
$h=(h_{1},\dots,h_{n})\in\mathbb{R}^{n}$ be a vector of weights such that
$\|h\|_{2}\leq c$. Then

$\mathbb{E}\big{[}\big{|}\frac{1}{\sqrt{n}}\sumop\slimits@_{i=1}^{n}h_{i}v_{i}\big{|}^{3}\big{]}\leq
c^{3}M$

###### Proof.

We can expand out

$\displaystyle\mathbb{E}\big{[}\big{|}\frac{1}{\sqrt{n}}\sumop\slimits@_{i=1}^{n}h_{i}v_{i}\big{|}^{3}\big{]}$
$\displaystyle\leq\frac{1}{n^{3/2}}\sumop\slimits@_{i=1}^{n}\sumop\slimits@_{j=1}^{n}\sumop\slimits@_{k=1}^{n}|h_{i}||h_{j}||h_{k}|\mathbb{E}[|v_{i}||v_{j}||v_{k}|]$
$\displaystyle\leq\frac{M}{n^{3/2}}\sumop\slimits@_{i=1}^{n}|h_{i}|\sumop\slimits@_{j=1}^{n}|h_{j}|\sumop\slimits@_{k=1}^{n}|h_{k}|\leq\frac{M}{n^{3/2}}\|h\|_{1}^{3}\leq
Mc^{3}$

where the second inequality follows from generalized Hölder’s inequality,

$|\mathbb{E}[fgh]|\leq(\mathbb{E}[|f|^{3}]\mathbb{E}[|g|^{3}]\mathbb{E}[|h|^{3}])^{1/3}$

and the fourth inequality from $\|h\|_{1}\leq\sqrt{n}\|h\|_{2}$.∎

######  Lemma F.7.

Let $v_{1},\dots,v_{n}$ be a martingale difference array such that
$\mathbb{E}[|v_{i}|^{l}]\leq M$ for all $l=1,\dots,k$. Then there is a fixed
constant $C_{k}$ that only depends on $k$ such that

$\mathbb{E}[(\frac{1}{\sqrt{n}}\sumop\slimits@_{i=1}^{n}v_{i})^{k}]\leq
C_{k}M$

###### Proof.

We move to apply Theorem H.3 with
$X_{t}=\sumop\slimits@_{i=1}^{t}v_{i}/\sqrt{n}$.

$\displaystyle\mathbb{E}[(\frac{1}{\sqrt{n}}\sumop\slimits@_{i=1}^{n}v_{i})^{k}]$
$\displaystyle\leq\mathbb{E}[(\max_{s\leq
n}\sumop\slimits@_{t=1}^{s}X_{s})^{k}]$ $\displaystyle\leq
C_{k}\mathbb{E}\big{[}\big{(}\sumop\slimits@_{i=1}^{n}v_{i}^{2}/n\big{)}^{k/2}\big{]}\leq
C_{k}\mathbb{E}\big{[}\frac{1}{n}\sumop\slimits@_{i=1}^{n}v_{i}^{k}\big{]}\leq
C_{k}M$

where the second inequality comes from Theorem H.3 and the third comes from an
application of Jensen’s inequality to the sample mean.∎

###  F.2 Useful Properties of Smooth Max

######  Lemma F.8 (Chernozhukov et al. (2013), Lemma A.2).

For every $1\leq j,k,l\leq p$,

$\displaystyle\partial_{j}F_{\beta}(z)$ $\displaystyle=\pi_{j}(z),$
$\displaystyle\partial_{j}\partial_{k}F_{\beta}(z)$ $\displaystyle=\beta
w_{jk}(z),$ $\displaystyle\partial_{j}\partial_{k}\partial_{l}F_{\beta}(z)$
$\displaystyle=\beta^{2}q_{jkl}(z)$

where for $\delta_{jk}\mathrel{\mathop{\ordinarycolon}}=\bm{1}\\{j=k\\}$,

$\displaystyle\pi_{j}(z)$
$\displaystyle\mathrel{\mathop{\ordinarycolon}}=e^{\beta
z_{j}}\bigg{/}\sumop\slimits@_{i=1}^{n}e^{\beta z_{i}},\hbox{
}\;\;\;\;w_{jk}\mathrel{\mathop{\ordinarycolon}}=(\pi_{j}\delta_{jk}-\pi_{j}\pi_{k})(z)$
$\displaystyle q_{jkl}(z)$
$\displaystyle\mathrel{\mathop{\ordinarycolon}}=(\pi_{j}\delta_{jl}\delta_{jk}-\pi_{j}\pi_{l}\delta_{jk}-\pi_{j}\pi_{k}(\delta_{jl}+\delta_{kl})+2\pi_{j}\pi_{k}\pi_{l})(z)$

Moreover,

$\displaystyle\pi_{j}(z)$ $\displaystyle\geq 0,$
$\displaystyle\sumop\slimits@_{j=1}^{p}\pi_{i}(z)$ $\displaystyle=1,$
$\displaystyle\sumop\slimits@_{j,k=1}^{p}|w_{jk}(z)|$ $\displaystyle\leq 2,$
$\displaystyle\sumop\slimits@_{j,k,l=1}^{p}|q_{jkl}|$ $\displaystyle\leq 6$

######  Lemma F.9 (Chernozhukov et al. (2013), Lemma A.3).

For every $x,z\in\mathbb{R}^{p}$,

$|F_{\beta}(x)-F_{\beta}(z)|\leq\max_{1\leq j\leq p}|x_{j}-z_{j}|.$

######  Lemma F.10 (Chernozhukov et al. (2013), Lemma A.4).

Let $\varphi(\cdot)\mathrel{\mathop{\ordinarycolon}}\mathbb{R}\to\mathbb{R}$
be such that $\varphi\in C_{b}^{3}(\mathbb{R})$ and define
$m\mathrel{\mathop{\ordinarycolon}}\mathbb{R}^{p}\to\mathbb{R}$,
$z\mapsto\varphi(F_{\beta}(z))$. The derivatives (up to the third order) of
$m$ are given

$\displaystyle\partial_{j}m(z)$ $\displaystyle=(\partial
g(F(\beta))\pi_{j})(z)$ $\displaystyle\partial_{j}\partial_{k}m(z)$
$\displaystyle=(\partial^{2}g(F_{\beta})\pi_{j}\pi_{k}+\partial
g(F_{\beta})\beta w_{jk})(z)$
$\displaystyle\partial_{j}\partial_{k}\partial_{l}m(z)$
$\displaystyle=(\partial^{3}g(F_{\beta})\pi_{j}\pi_{k}\pi_{l}+\partial^{2}g(F_{\beta})\beta(w_{jk}\pi_{l}+w_{jl}\pi_{k}+w_{kl}\pi_{j})+\partial
g(F_{\beta})\beta^{2}q_{jkl})(z)$

where $\pi_{j},w_{jk},q_{jkl}$ are as described in Lemma F.8.

######  Lemma F.11 (Chernozhukov et al. (2013), Lemma A.5).

Define
$L_{1}(\varphi)=\sup_{x}|\varphi^{\prime}(x)|,L_{2}(\varphi)=\sup_{x}|\varphi^{\prime\prime}(x)|$,
and $L_{3}(\varphi)=\sup_{x}|\varphi^{\prime\prime\prime}(x)|$.For every
$1\leq j,k,l\leq p$,

$\displaystyle|\partial_{j}\partial_{k}m(z)|\leq U_{jk}(z)\hbox{ }\text{ and
}\hbox{ }|\partial_{j}\partial_{k}\partial_{l}m(z)|\leq U_{jkl}(z)$

where for
$W_{jk}(z)\mathrel{\mathop{\ordinarycolon}}=(\pi_{j}\delta_{jk}+\pi_{j}\pi_{k})(z)$,

$\displaystyle U_{jk}(z)$
$\displaystyle\mathrel{\mathop{\ordinarycolon}}=(L_{2}\pi_{j}\pi_{k}+L_{1}\beta
W_{jk}(z)$ $\displaystyle U_{jkl}(z)$
$\displaystyle\mathrel{\mathop{\ordinarycolon}}=(L_{3}\pi_{j}\pi_{k}\pi_{l}+L_{2}\beta(W_{jk}\pi_{l}+W_{jl}\pi_{k}+W_{kl}\pi_{j})+L_{1}\beta^{2}Q_{jkl})(z)$
$\displaystyle Q_{jkl}(z)$
$\displaystyle\mathrel{\mathop{\ordinarycolon}}=(\pi_{j}\delta_{jl}\delta_{jk}+\pi_{j}\pi_{k}\delta_{jk}+\pi_{j}\pi_{k}(\delta_{jl}+\delta_{kl})+2\pi_{j}\pi_{k}\pi_{l})(z).$

Moreover,

$\displaystyle\sumop\slimits@_{j,k=1}^{p}U_{jk}(z)\leq(L_{2}+2L_{1}\beta)\hbox{
}\text{ and }\hbox{
}\sumop\slimits@_{j,k,l=1}^{p}U_{jkl}(z)\leq(L_{3}+6L_{2}\beta+6L_{1}\beta^{2}).$

###  F.3 Moment Bounds for Sections 4 and 5

######  Lemma F.12.

Suppose that Assumption 5.1 holds and let $N$ and $D$ be as defined at the top
of Section D.2 Then under $H_{0}$, for any $k$ there is a fixed constant
$C_{k}$ such that for any $\ell=1,\dots,d_{x}$

$\mathbb{E}[|N_{\ell}|^{k}]\leq C_{k}\hbox{ }\text{ and }\hbox{
}\mathbb{E}[|D_{\ell\ell}|^{k}]\leq C_{k}\log^{2k/a}(n)$

###### Proof.

Let $\eta_{\ell i}=r_{i}-\mathbb{E}[r_{i}]$ and write

$N_{\ell}=\underbrace{\frac{1}{\sqrt{n}}\sumop\slimits@_{i=1}^{n}\epsilon_{i}(\beta_{0})\sumop\slimits@_{j=1}^{n}\tilde{h}_{ij}\eta_{\ell
j}}_{N_{\ell}^{1}}+\frac{1}{\sqrt{n}}\underbrace{\sumop\slimits@_{i=1}^{n}\epsilon_{i}(\beta_{0})\sumop\slimits@_{j=1}^{n}\tilde{h}_{ij}\mathbb{E}[r_{\ell
j}]}_{N_{\ell}^{2}}$

To bound moments of $N_{\ell}^{1}$ use the fact that $N_{\ell}^{1}$ is a
quadratic form in mean-zero $a$-sub-exponential variables. By Theorem H.1,
$N_{\ell}^{1}$ is therefore also $a$-sub-exponential with parameter $a/2$;
thus $(N_{\ell}^{1})^{a/2}$ is sub-exponential and Lemma G.2 provides the
moment bound for arbitrary moments. To bound moments of $N_{\ell}^{2}$ we use
the fact that
$\max_{i}\big{|}\sumop\slimits@_{j=1}^{n}\tilde{h}_{ij}\mathbb{E}[r_{\ell
j}]\big{|}$ is bounded by assumption and apply Burkholder-Davis-Gundy (Theorem
H.3) after adding and subtracting $\mathbb{E}[\epsilon_{i}(\beta_{0})]$.To
bound moments of $D_{\ell\ell}$ we decompose

$|D|\leq\frac{1}{n}\sumop\slimits@_{i=1}^{n}\epsilon_{i}^{2}(\beta_{0})\max_{1\leq
i\leq n}\big{|}\sumop\slimits@_{j=1}^{n}h_{ij}r_{j}\big{|}^{2}$

Apply Theorem H.1 to see that $\sumop\slimits@_{j=1}^{n}h_{ij}r_{j}$ is
$\alpha$-sub-exponential and Lemma G.2 to bound the RHS by a log-power of
$n$.∎

###  F.4 Matrix Derivative Lemmas

The purpose of this section is largely to establish some matrix derivative
expressions that will be useful for the Lindeberg interpolation in

######  Lemma F.13.

Let $D\in\mathbb{R}^{d\times d}$ be a symmetric, real matrix such that
$\det(D)\neq 0$. Let $N\in\mathbb{R}^{d}$ be a vector. The derivatives up to
the derivatives of quadratic form $N^{\prime}D^{-1}N$ are given.First Order:

$\displaystyle\frac{\partial}{\partial N_{l}}$
$\displaystyle=2\sumop\slimits@_{j=1}^{d}(D^{-1})_{jl}N_{j},\hbox{ }\hbox{
}\hbox{ }\hbox{ }\frac{\partial}{\partial
D_{lm}}=-2\sumop\slimits@_{j=1}^{d}\sumop\slimits@_{k=1}^{d}(D^{-1})_{jl}(D^{-1})_{km}N_{j}N_{k},$
Second Order: $\displaystyle\frac{\partial^{2}}{\partial N_{l}N_{m}}$
$\displaystyle=2(D^{-1})_{lm},\hbox{ }\hbox{ }\hbox{ }\hbox{
}\frac{\partial^{2}}{\partial N_{l}\partial
D_{pq}}=-2\sumop\slimits@_{j=1}^{d}(D^{-1})_{jp}(D^{-1})_{ql}N_{j},$
$\displaystyle\frac{\partial^{2}}{\partial D_{lm}\partial D_{qj}}$
$\displaystyle=\sumop\slimits@_{j=1}^{d}\sumop\slimits@_{k=1}^{d}\left\\{(D^{-1})_{lp}(D^{-1})_{qj})(D^{-1})_{km}+(D^{-1})_{kp}(D^{-1})_{mq}(D^{-1})_{lj}\right\\}N_{j}N_{k}$
Third Order: $\displaystyle\frac{\partial^{3}}{\partial N_{l}\partial
N_{m}\partial N_{p}}$ $\displaystyle=0,\hbox{ }\hbox{ }\hbox{ }\hbox{
}\frac{\partial^{3}}{\partial N_{l}\partial N_{m}\partial
D_{pq}}=-2(D^{-1})_{lp}(D^{-1})_{qm}$
$\displaystyle\frac{\partial^{3}}{\partial D_{lm}\partial D_{pq}\partial
N_{r}}$
$\displaystyle=2\sumop\slimits@_{j=1}^{d}\bigg{\\{}(D^{-1})_{lp}(D^{-1})_{qj}(D^{-1})_{rm}+(D^{-1})_{rp}(D^{-1})_{mq}(D^{-1}){lj}\bigg{\\}}N_{j}$
$\displaystyle\frac{\partial^{3}}{\partial D_{lm}D_{pq}D_{rs}}$
$\displaystyle=2\sumop\slimits@_{j=1}^{d}\sumop\slimits@_{j=1}^{d}\bigg{\\{}(D^{-1})_{lr}(D^{-1})_{ps}(D^{-1})_{qj}(D^{-1})_{km}+(D^{-1})_{lp}(D^{-1})_{qr}(D^{-1})_{js}(D^{-1})_{km}$
$\displaystyle\hphantom{2\sumop\slimits@\sumop\slimits@\sumop\slimits@}+(D^{-1})_{lp}(D^{-1})_{qj}(D^{-1})_{kr}(D^{-1})_{ms}+(D^{-1})_{kr}(D^{-1})_{ps}(D^{-1})_{mq}(D^{-1})_{lj}$
$\displaystyle\hphantom{2\sumop\slimits@\sumop\slimits@\sumop\slimits@}+(D^{-1})_{kp}(D^{-1})_{mr}(D^{-1})_{qs}(D^{-1})_{lj}+(D^{-1})_{rp}(D^{-1})_{mq}(D^{-1})_{lr}(D^{-1}){js}\bigg{\\}}N_{j}N_{k}$

###### Proof.

The derivative of an element of the the inverse of a matrix $\bm{X}$ can be
expressed (Petersen and Pedersen, 2012)

$\frac{\partial(\bm{X}^{-1})_{kl}}{\partial\bm{X}_{ij}}=-(\bm{X}^{-1})_{ki}(\bm{X}^{-1})_{jl}$
(F.3)

repeated application of this identity as well as the expression of the
quadratic form

$N^{\prime}D^{-1}N=\sumop\slimits@_{j=1}^{d}\sumop\slimits@_{k=1}^{d}(D^{-1})_{jk}N_{j}N_{k}$

leads to the result, bearing in mind that the inverse of a symmetric matrix is
symmetric.∎

######  Lemma F.14.

Let D be a symmetric positive definite matrix. Then, for any $p>3$, the
derivatives of $(\det(D))^{p}$ are given up to the third order by

$\displaystyle\frac{\partial\,(\det(D))^{p}}{\partial D_{lm}}$
$\displaystyle=p(\det(D))^{p-1}(D^{-1})_{lm}$
$\displaystyle\frac{\partial^{2}\,(\det(D))^{p}}{\partial D_{lm}\partial
D_{pq}}$
$\displaystyle=\frac{p!}{(p-2)!}(\det(D))^{p-2}(D^{-1})_{pq}(D^{-1})_{lm}$
$\displaystyle\;\;+p(\det(D))^{p-1}(D^{-1})_{lp}(D^{-1})_{mq}$
$\displaystyle\frac{\partial^{3}\,(\det(D))^{p}}{\partial D_{lm}\partial
D_{pq}\partial D_{rs}}$
$\displaystyle=\frac{p!}{(p-3)!}(\det(D))^{p-3}(D^{-1})_{rs}(D^{-1})_{pq}(D^{-1})_{lm}$
$\displaystyle\;\;+\frac{p!}{(p-2)!}(\det(D))^{p-2}\bigg{\\{}(D^{-1})_{pq}(D^{-1})_{lr}(D^{-1})_{ps}+(D^{-1})_{pr}(D^{-1})_{qs}(D^{-1})_{lm}$
$\displaystyle\hphantom{\;\;+\frac{p!}{(p-2)!}(\det(D))^{p-2}\bigg{\\{}(D^{-1})_{pq}(D^{-1})_{lr}(D^{-1})_{ps}}\;\,+(D^{-1})_{rs}(D^{-1})_{lp}(D^{-1})_{mq}\bigg{\\}}$
$\displaystyle\;\;+p(\det(D))^{p-1}\bigg{\\{}(D^{-1})_{lr}(D^{-1})_{qs}(D^{-1})_{mq}+(D^{-1})_{lp}(D^{-1})_{mr}(D^{-1})_{qs}\bigg{\\}}$

###### Proof.

We can express the derivative of the detrminant (Petersen and Pedersen, 2012),

$\begin{split}\frac{\partial,\det(\bm{X})}{\partial\,\bm{X}_{ij}}=\det(\bm{X})(\bm{X}^{-1})_{ij}\end{split}$
(F.4)

Repeated application of this and (F.3) yields the result.∎

######  Lemma F.15.

For any $p>4$ define the function
$\gamma(N,\text{vec}(D))\mathrel{\mathop{\ordinarycolon}}\mathbb{R}^{d}\times\mathbb{R}^{d^{2}}$
by

$\gamma(N,\text{vec}(D))\mathrel{\mathop{\ordinarycolon}}=\begin{cases}(\det(D))^{p}(N^{\prime}D^{-1}N-c)&\text{if
}\det(D)\neq 0\\\ 0&\text{if }\det(D)=0\end{cases}$

This function is thrice continously differentiable. Futher the $k^{\text{th}}$
moments of all partial derivatives of this function up to the third order are
bounded

$\mathbb{E}[(\partial^{\alpha}\gamma(N,\text{vec}(D))^{k}]\leq
C_{k}(\max_{\iota\leq d}\mathbb{E}[|D_{\iota\iota}|^{2pdk}]\vee\max_{\iota\leq
d}\mathbb{E}[|N_{\iota\iota}|^{6k})$

where $C_{k}$ is a positive constant that only depends on $k$ and $d$.

###### Proof.

The first statement is clear by examination of the derivatives in Lemmas F.13
and F.14 as well as the inequality (F.5) below. For the moment bounds, we may
extensive use of following bounds on elements of $D^{-1}$ for a positive-
definite $D^{-1}$:

$\begin{split}|\det(D)(D^{-1})_{jk}|\leq\det(D)\text{trace}(D^{-1})&\leq
d\lambda_{\max}(D^{-1})\big{(}\prodop\slimits@_{m=1}^{d}\lambda_{m}(D)\big{)}\\\
&=d\prodop\slimits@_{m=2}^{d}\lambda_{m}(D)\\\ &\leq
d\big{(}\sumop\slimits@_{m=2}^{d}\lambda_{m}(D)\big{)}^{d-1}\\\ &\leq
d(\text{trace}(D))^{d-1}\end{split}$ (F.5)

where the first inequality uses the fact that the largest element of a
positive semidefinite matrix is on the diagonal and the fact that the diagonal
elements of a positive semidefinite matrix are weakly positive, the second
inequality uses the fact that the trace is the sum of the eigenvalues and the
determinant is the product of the eigenvalues, the equality comes from
$\frac{1}{\lambda_{\min}(D)}=\lambda_{\max}(D^{-1})$, the third inequality
uses the AM-GM inequality and the fourth again uses that the trace is the sum
of the (weakly positive) eigenvalues.The moment bounds follow from (F.5) and
the expressions in Lemmas F.13 and F.14. We give an example of how this is
done for the first order derivatives, higher order derivatives follow from
similar logic. For the following let $A$ be an arbitrary random variable.First
Order.

$\displaystyle\mathbb{E}\bigg{|}A\frac{\partial\gamma}{\partial
N_{l}}\bigg{|}^{k}$
$\displaystyle\lesssim\sumop\slimits@_{j=1}^{d}\mathbb{E}|(\text{trace}(D))^{kdp}N_{j}^{k}A^{k}|$
$\displaystyle\lesssim\sumop\slimits@_{j=1}^{d}\sumop\slimits@_{\iota=1}^{d}\mathbb{E}[D_{\iota\iota}^{kdp}N_{j}^{k}A^{k}]$
$\displaystyle\leq\sumop\slimits@_{j=1}^{d}\sumop\slimits@_{\iota=1}^{d}\gamma^{2kdp}\mathbb{E}[N_{j}^{2k}A^{2k}]$
$\displaystyle\mathbb{E}\bigg{|}A\frac{\partial\gamma}{\partial
D_{lm}}\bigg{|}^{k}$
$\displaystyle=p\mathbb{E}\bigg{|}A\det(D)^{p-1}\sumop\slimits@_{j=1}^{d}\sumop\slimits@_{j^{\prime}=1}^{d}(D^{-1})_{lm}(D^{-1})_{jj^{\prime}}N_{j}N_{j^{\prime}}\bigg{|}^{k}$
$\displaystyle\lesssim
p\sumop\slimits@_{j=1}^{d}\sumop\slimits@_{j^{\prime}=1}^{d}\mathbb{E}[|(\text{trace}(D))^{2k(d-1)+(p-3)kd}A^{k}N_{j}^{k}N_{j^{\prime}}^{k}|$
$\displaystyle\leq\sumop\slimits@_{j=1}^{d}\sumop\slimits@_{j^{\prime}=1}^{d}\gamma^{2kd(p-1)}\mathbb{E}[A^{2k}N_{j}^{2k}N_{j^{\prime}}^{2k}]$

∎

##  Appendix G Technical Lemmas

###  G.1 Probability Lemmas

######  Lemma G.1.

Let $X_{n}$ be a sequence of random variables such that $X_{n}=o_{p}(1)$, that
is for any $\delta>0$, $\mathrm{Pr}(|X_{n}|\geq\delta)\to 0$. Then, there is a
sequence $\delta_{n}\to 0$ such that $\mathrm{Pr}(|X_{n}|\geq\delta_{n})\to
0$.

###### Proof.

Take a preliminary sequence $\tilde{\delta}_{n}\to 0$ and define

$\tilde{n}_{j}=\inf\\{n\mathrel{\mathop{\ordinarycolon}}\mathrm{Pr}(|X_{n}|>\tilde{\delta}_{j})<\tilde{\delta}_{j}\\}$

Because $\mathrm{Pr}(|X_{n}|>\delta)\to 0$ for any fixed $\delta$, we know
that $n_{j}$ is finite. Define a new sequence $\delta_{n}\to 0$ as below:

$\begin{split}\delta_{n}=\begin{cases}1&\text{if }0\leq n<\tilde{n}_{1}\\\
\tilde{\delta}_{i}&\text{if }\tilde{n}_{i}\leq
n<\tilde{n}_{i+1}\end{cases}\end{split}$ (G.1)

By construction, this sequence satisfies
$\mathrm{Pr}(X_{n}\geq\delta_{n})\leq\delta_{n}$ whenever $n\geq n_{1}$.∎

######  Lemma G.2.

Suppose that $X_{1},\dots,X_{n}$ are $\alpha$-subexponential such that
$\mathrm{Pr}(|X_{i}|\geq t)\leq 2\exp(-t^{\alpha}/K)$ for all $t\geq 0$ and