# On the Existence and Information of Orthogonal Moments For
Inference††thanks: Research funded by Ministerio de Ciencia e Innovación grant
PID2021-127794NB-I00 and Comunidad de Madrid, grants EPUC3M11 (VPRICIT) and
H2019/HUM-589.

Facundo Argañaraz
Universidad Carlos III de Madrid Department of Economics. E-mail:
<EMAIL_ADDRESS>Juan Carlos Escanciano
Universidad Carlos III de Madrid Department of Economics. E-mail:
<EMAIL_ADDRESS>
(March 20th, 2023)

###### Abstract

Locally Robust (LR)/Orthogonal/Debiased moments have been proved useful with
machine learning or high dimensional first steps, but their existence has not
been investigated for general models and parameters. In this paper, we provide
a necessary and sufficient condition, referred to as Restricted Local Non-
surjectivity (RLN), for the existence of such orthogonal moments to conduct
robust inference on parameters of interest in regular semiparametric models.
Importantly, RLN does not require identification of the parameters of interest
or identification of the nuisance parameters. Thus, orthogonal moments exist
under rather general conditions. However, for orthogonal moments to be
informative for inference, the efficient Fisher Information matrix for the
parameter must be non-zero (though possibly singular). We use these results to
characterize the existence of orthogonal moments in a class of models with
Unobserved Heterogeneity (UH), and to clarify the important role played by the
support of UH for such characterization. Our results deliver functional
differencing moments as a special case, and they also extend them to general
functionals of UH. We also investigate existence of orthogonal moments and
their relevance for models defined by moment restrictions with possibly
different conditioning variables, and we characterize orthogonal moments for
heterogenous parameters in treatment effects, for sample selection models, and
for popular models of demand of differentiated products.

Keywords:

Debiased Inference; Machine Learning; Unobserved Heterogeneity.

_JEL classification:_

C14; C31; C33; C35

##  1 Introduction

A recent and growing literature in economics and machine learning recommends
Locally Robust (LR)/Orthogonal/Debiased moments for inference on parameters of
interest in the presence of high dimensional first steps, see, e.g., Athey and
Wager (2021), Belloni et al. (2012, 2017), Bravo, Escanciano and Van Keilegom
(2020), Chernozhukov et al. (2018), Chernozhukov et al. (2016, 2022,
henceforth CEINR), Farrell (2015), Nekipelov, Semenova and Syrgkanis (2022)
and Sasaki and Ura (2021), among many others. The literature on debiased
machine learning has shown that LR moments have several advantages over plug-
in approaches. Most notably, LR moments are useful to reduce model selection
and regularization biases when machine learning estimators are used as first
steps. Furthermore, orthogonal moments have a fast rate of convergence and
provide valid confidence intervals for structural parameters under more
general conditions than plug-in methods (CEINR). However, a fundamental
question that has not been addressed in the literature is the existence of
such orthogonal moments for inference. If one equates reliable inference to
orthogonal moments, we are asking:

Under what minimal conditions is reliable inference on a parameter of interest
possible?

In this paper, we give a necessary and sufficient condition for existence of
LR moments and, when they exist, we characterize when they are informative for
inference on the parameter of interest.

To introduce the main result, consider a semiparametric model where the
parameter of interest is fixed at a specified known value under the null
hypothesis, referred to as the restricted model. If this restricted model is
(locally) nonparametric, in the sense that the set of scores of the restricted
model can approximate arbitrarily well any zero-mean square-integrable
function, then we say the model satisfies Restricted Local Surjectivity (RLS).
Otherwise, we say the model satisfies Restricted Local Non-Surjectivity (RLN).
The first result of this paper is that, modulo regularity conditions, RLN is
necessary and sufficient for the existence of a LR moment function for
inference on the parameter of interest in any regular semiparametric model.

We argue that RLN follows from mild conditions, which are weaker than the ones
needed for relevance of the orthogonal moments, and which in turn are weaker
than the ones required for identification of the parameter of interest.
Indeed, we show that RLN, and hence the existence of orthogonal moments, does
not require the parameter of interest to be identified or the nuisance
parameters to be identified. Of course, lack of identification of the
parameter of interest will affect the power properties of testing procedures.
A non-zero efficient Fisher Information matrix for the parameter of interest
is sufficient for RLN, and necessary for orthogonal moments to be informative
(i.e. for the corresponding tests to have non-trivial local power). Regular
local identification further requires full rank of the Fisher information
(Rothenberg (1971) and Escanciano (2022)). This emphasizes that the conditions
for existence of orthogonal moments are separated from those of relevance. We
study and characterize both set of conditions in this paper.

We apply our results to models with Unobserved Heterogeneity (UH), where UH is
understood as all those factors that are unobserved by the econometrician but
whose presence cannot be disregarded. The existence of LR moments is
particularly important in models with UH, and in the vast majority of models
where nuisance parameters are either not identified (as in, e.g., discrete
choice models) or identified by ill-posed problems (see, e.g., Carrasco,
Florens and Renault 2007). This is so because regularization is often
necessary for estimating first steps under these circumstances, and orthogonal
moments have the potential to greatly reduce regularization biases for
reliable inference in these cases.

The panel data literature in economics has been particularly interested in
carrying out flexible estimation and robust inference in the presence of UH.
The usual and simplest approach to inference consists in the construction of
moments that do not depend on the nuisance parameter (the density of UH in
some of our applications), referred to as Nuisance-Free (NF) moments in this
paper. In linear fixed effects settings, this can be simply accomplished by
differencing. In non-linear settings, the situation is more challenging. In an
important paper, Bonhomme (2012) developed a systematic method to construct NF
moments by functional differencing. He found a condition, closely related to
RLN, that is necessary but not sufficient for the estimation of fixed
parameters by functional differencing. We characterize LR in this setting, and
we show that the support of UH plays an important role in this
characterization. We spell out additional smoothness and support conditions
under which the sufficient condition for NF moments in Bonhomme (2012) becomes
necessary for LR, thereby showing that all orthogonal moments are NF under the
derived conditions. Thus, the proposed characterization of LR when applied to
nonlinear panel data and under some additional smoothness and support
conditions delivers functional differencing as a special case. Our paper
generalizes functional differencing by also considering other parameters of
interests, including nonlinear parameters such as average partial effects and
parameters involving the distribution of UH. We show that for these other
parameters, orthogonal moments, though locally insensitive to nuisance
parameters, are in general not NF.

A main implication of our paper is that orthogonality is useful in more
general settings than previously recognized. In particular, we show that
exclusion restrictions are not necessary for the existence of orthogonal
moments of structural parameters in semiparametric sample selection models,
but they are necessary for the commonly used partly linear model with an
endogenous variable. These results are obtained as special cases of a general
characterization of orthogonal moments in models defined by conditional moment
restrictions with possibly different conditioning variables.
Overidentification and estimation under misspecification in these models have
been studied in Chen and Santos (2018) and Ai and Chen (2007), respectively,
but the existence and information of orthogonal moments have not been derived
under this generality, to the best of our knowledge. Our results extend those
in Chernozhukov et al. (2018) on the characterization of orthogonal moments in
conditional moment models with the same conditioning variables. More broadly,
our results generalize existing characterizations of orthogonal moments to
essentially any regular semiparametric model and any smooth functional. As
novel applications, we characterize orthogonal moments for heterogeneous
parameters in treatment effect models, for sample selection models, and for
the fixed parameters and moments of the nonparametric UH in the popular demand
model for differentiated products in Berry, Levinsohn and Pakes (1995,
henceforth BLP).

When RLN holds at all parameter values in the parameter space, we can use our
results to construct robust asymptotic confidence regions for the parameter of
interest in the usual way. These confidence regions will inherit the
robustness and efficiency properties of the orthogonal moments based tests.
Additionally, if orthogonal moments exist and they identify the structural
parameter of interest, then one can proceed as in, e.g., CEINR, and propose LR
estimators by a Generalized Methods of Moments (GMM) approach applied to our
orthogonal moments. The derived conditions for existence and relevance of
orthogonal moments are fundamental for these subsequent constructions. In
particular, our characterization of orthogonal moments can be use to construct
them and to choose among them based on their information about the parameter
of interest. An automatic construction of orthogonal moments for inference and
estimation in models with UH will be carried out in a companion paper.

Restricted local non-surjectivity may seem like a mild and natural condition
to hold. Yet, this condition is not satisfied for a number of important
models/parameters. Ishwaran (1999) showed that several semiparametric mixtures
of exponential families satisfy RLS, including the Weibull semiparametric
mixture used in the seminal work by Heckman and Singer (1984). Chamberlain
(2010) showed that RLS is generally satisfied in nonlinear binary choice panel
data with fixed effects and a finite number of periods $T$, the Logit fixed
effects being an exception for $T=2$. Bajari, Hahn, Hong and Ridder (2011) has
shown that RLS holds in games with multiple equilibria, if the number of
equilibria is sufficiently large. As models become more flexible, for example
by making UH more flexible, we expect new examples of RLS models to easily
arise. An important implication of this paper is that restricting UH is
necessary in these RLS models if the goal is to obtain reliable inference on
the parameters of interest.

The rest of the paper is organized as follows. Section 2 introduces the
setting, concepts and the main result. Section 3 applies the general theory to
the class of models with UH. Section 4 shows how the results are extended to
general models and other smooth functionals of interest. Section 5 discusses
the application to the class of conditional moment restrictions, including the
applications to heterogenous treatment effect parameters, sample selection and
the BLP model. Finally, Section 6 concludes. An Appendix contains the proofs
of the main results, and a Supplementary Appendix gathers further discussion
and extensions.

##  2 Restricted Local Surjectivity and Local Robustness

###  2.1 Setting

The data is an independent and identically distributed (iid) sample
$Z_{1},...,Z_{n}$ from a distribution $\mathbb{P}_{0}$ that belongs to a
semiparametric model
$\mathcal{P}=\\{\mathbb{P}_{\lambda}:\lambda=(\theta,\eta)\in\Lambda\equiv\Theta\times\Xi\\},$
where $\Theta\subset\mathbb{R}^{p}$ and $\Xi$ is an arbitrary subset of a
typically infinite dimensional space.111The results on existence of orthogonal
moments also hold for infinite dimensional parameter of interest. We focus on
finite dimensional parameters of interest for ease of exposition. The
parameters that generated the data are denoted by
$\left(\theta_{0},\eta_{0}\right)\in\Theta\times\Xi,$ i.e.
$\mathbb{P}_{0}=\mathbb{P}_{\theta_{0},\eta_{0}}.$ The parameter of interest
is a vector of functionals $\psi(\lambda_{0})\in\mathbb{R}^{d_{\psi}},$
although for the sake of exposition we focus first on
$\psi(\lambda_{0})=\theta_{0},$ denoted as the structural parameter, with
$d_{\psi}=p$ and where we treat $\eta_{0}$ as an unknown nuisance parameter.
For example, $\eta_{0}$ could be the unknown density of UH. Consider the
restricted model
$\mathcal{P}_{0}=\\{\mathbb{P}_{\theta_{0},\eta}:\eta\in\Xi\\}$ that assumes
$\theta_{0}$ is known. Surjectivity refers to the situation where the set of
scores of the model $\mathcal{P}_{0},$ defined formally below, can approximate
arbitrary well any zero-mean square-integrable function of the data $Z_{i}$.
This spanning condition, which can be considered an appealing one from a
fitting point of view, will have important implications for inference on the
structural parameter $\theta_{0}.$

The problem we address in this paper is inference about $\theta_{0}$ (more
generally, $\psi(\lambda_{0}),$ see Section 4), in the presence of an unknown,
possibly high dimensional, nuisance parameter $\eta_{0}$ (respectively,
$\lambda_{0}$). That is, we aim at testing the hypotheses

$H_{0}:\theta_{0}=\bar{\theta}\qquad vs\qquad
H_{1}:\theta_{0}\neq\bar{\theta},$

for a known $\bar{\theta},$ e.g. $\bar{\theta}=0,$ when $\eta_{0}$ is unknown
under both $H_{0}$ and $H_{1}.$

Inference is based on a $k$-dimensional vector of moments, $k\in\mathbb{N},$
constructed such that under the null hypothesis:

$\mathbb{E}\left[g\left(Z,\bar{\theta},\eta_{0}\right)\right]=0,$ (2.1)

where $Z$ is an independent copy of $Z_{i},$ $\mathbb{E}$ denotes expectation
under $\mathbb{P}_{0},$ and $g(\cdot)$ is a measurable square-integrable
function. A natural example of $g$ is the score of the model $\mathcal{P}$
with respect to $\theta$ at $\bar{\theta},$ often computed as

$g\left(Z,\bar{\theta},\eta_{0}\right)\equiv
s_{\theta}\left(Z,\bar{\theta},\eta_{0}\right)=\frac{\partial\log
f_{\theta,\eta_{0}}\left(Z,\bar{\theta},\eta_{0}\right)}{\partial\theta},$
(2.2)

where $f_{\theta,\eta}$ is the density of $\mathbb{P}_{\theta,\eta}$ with
respect to a $\sigma$-finite measure $\mu$. The score satisfies (2.1) with
$k=p,$ but it will not be in general an orthogonal moment, as we define it
below.

For expositional purposes, we motivate first the problem in the parametric
case studied in the landmark contribution by Neyman (1959), where
$\Xi\subset\mathbb{R}^{q}$ was endowed with the Euclidean inner product
$\langle u,v\rangle=u^{\prime}v$ $(A^{\prime}$ denotes the transpose of the
matrix $A)$. Neyman (1959) investigated conditions under which, for a given
$\sqrt{n}-$consistent estimator $\hat{\eta}$ of $\eta_{0},$ the asymptotic
distribution of the sample moment
$\mathbb{E}_{n}\left[g\left(Z,\hat{\eta},\bar{\theta}\right)\right]$ will not
depend on that of the first step, $\sqrt{n}(\hat{\eta}-\eta_{0}),$ where
henceforth
$\mathbb{E}_{n}\left[h\left(Z\right)\right]=n^{-1}\sum_{i=1}^{n}h\left(Z_{i}\right)\
$denotes the sample mean operator. A simple Taylor expansion argument yields,
under standard regularity conditions,

$\sqrt{n}\mathbb{E}_{n}\left[g\left(Z,\hat{\eta},\bar{\theta}\right)\right]=\sqrt{n}\mathbb{E}_{n}\left[g\left(Z,\bar{\theta},\eta_{0}\right)\right]+\frac{\partial\mathbb{E}\left[g\left(Z,\bar{\theta},\eta_{0}\right)\right]}{\partial\eta^{\prime}}\sqrt{n}(\hat{\eta}-\eta_{0})+o_{P}(1).$

Neyman’s (1959) sufficient condition for asymptotic invariance to the
distribution of $\sqrt{n}(\hat{\eta}-\eta_{0})$ was local robustness, in the
sense of

$\frac{\partial\mathbb{E}\left[g\left(Z,\bar{\theta},\eta_{0}\right)\right]}{\partial\eta}=0.$

To achieve this goal, Neyman (1959) used a moment function $g$ sufficiently
regular (termed a Cramer function) so that the following generalized
information equality holds for $j=1,...,q,$

$\frac{\partial\mathbb{E}\left[g\left(Z,\bar{\theta},\eta_{0}\right)\right]}{\partial\eta_{j}}=-\mathbb{E}\left[g\left(Z,\bar{\theta},\eta_{0}\right)s_{\eta,j}\left(Z,\bar{\theta},\eta_{0}\right)\right],$
(2.3)

where $s_{\eta,j}\left(Z,\bar{\theta},\eta_{0}\right)=\partial\log
f_{\theta,\eta_{0}}\left(Z,\bar{\theta},\eta_{0}\right)/\partial\eta_{j}$ and
$\eta_{j}$ is the $j-th$ component of $\eta.$ By (2.3), locally robust moments
are moments orthogonal to the scores $s_{\eta,j}$, and they can be constructed
starting from any original moment function $m(\cdot)$ by performing a least
squares projection to obtain the “residual”

$g\left(Z,\bar{\theta},\eta_{0}\right)=m\left(Z,\bar{\theta},\eta_{0}\right)-\beta_{m}^{\prime}s_{\eta}\left(Z,\bar{\theta},\eta_{0}\right),$
(2.4)

where
$\beta_{m}=\mathbb{E}\left[s_{\eta}s_{\eta}^{\prime}\right]^{-1}\mathbb{E}\left[s_{\eta}m^{\prime}\right]$
and $s_{\eta}\equiv(s_{\eta,1},...,s_{\eta,q})^{\prime}.$ The term
$\beta_{m}^{\prime}s_{\eta}\left(Z,\bar{\theta},\eta_{0}\right)$ is the
orthogonal projection of $m$ onto the linear space generated by the scores
$s_{\eta,j},$ $j=1,...,q$—the so-called tangent space of nuisance parameters.
Existence of orthogonal moments was not an issue in the parametric case of
Neyman (1959), because the tangent space of nuisance parameters is finite
dimensional (of dimension at most $q$), while the space of zero-mean square-
integrable $k-$valued functions has infinite dimension for a continuous $Z,$
creating a surplus of orthogonal moments.222For a discrete $Z$ taking $r$
distinct values, the space of zero-mean $k-$valued square-integrable moments
has dimension $k(r-1),$ see Tsiatis (2006, pg. 12). Thus, orthogonal moments
exist as soon as $k(r-1)>q.$ An insight of our paper is that this may not be
the case for nonparametric models.333Semiparametric extensions of the
C($\alpha$) test have been considered in Choi, Hall and Schick (1996), and
more recently, in Lee (2022), for strongly and weakly identified nuisance
parameters, respectively. However, these papers focus on likelihood settings
and do not study the existence of orthogonal moments. Escanciano (2012)
proposed a semiparametric C($\alpha$) test for general moment restriction
models, but he did not investigate existence.

Whether orthogonal moments exist or not in semiparametric models will depend
on a condition that we introduce. Let again $f_{\theta,\eta}$ denote the
density of $\mathbb{P}_{\theta,\eta}$ with respect to a $\sigma$-finite
measure $\mu$, and let $f_{0}\equiv f_{\theta_{0},\eta_{0}}$ be the true
density. Let $L_{2}\equiv L_{2}(f_{0})$ denote the Hilbert space of
$\mathbb{P}_{0}-$square integrable measurable functions with inner product
$\langle h,f\rangle=\int
hfd\mathbb{P}_{0}=\mathbb{E}\left[h\left(Z\right)f\left(Z\right)\right]$ and
norm $\left\|h\right\|^{2}=\langle
h,h\rangle=\mathbb{E}\left[h^{2}\left(Z\right)\right]$. The set
$L_{2}^{0}\equiv L_{2}^{0}(f_{0})$ is the subspace of zero-mean functions in
$L_{2},$ i.e. $h\in L_{2}$ with $\mathbb{E}\left[h\left(Z\right)\right]=0.$
More generally, $L_{2}(f)$ and $L_{2}^{0}(f)$ are defined analogously for any
density $f.$

A parametric submodel or a path
$\tau\in[0,\varepsilon)\mapsto\mathbb{P}_{\tau}\equiv\mathbb{P}_{\theta_{0},\eta_{\tau}}\in\mathcal{P}_{0},$
$\varepsilon>0,$ satisfies the so-called Differentiability in Quadratic Mean
(DQM)

$\int\left[\frac{f_{\tau}^{1/2}-f_{0}^{1/2}}{\tau}-\frac{1}{2}s_{\eta}f_{0}^{1/2}\right]^{2}d\mu\rightarrow
0,\text{ as }\tau\downarrow 0,$ (2.5)

where $f_{\tau}$ is density of $\mathbb{P}_{\tau}$. We drop the dependence of
the path on $s_{\eta}$ for simplicity of notation. The function $s_{\eta}$ is
the score of the path $\mathbb{P}_{\tau}$, in most cases $s_{\eta}=d\log
f_{\theta_{0},\eta_{\tau}}/d\tau,$ where henceforth $d/d\tau$ is the
derivative from the right (i.e. for nonnegative values of $\tau$) at $\tau=0.$
An implication of (2.5) is that $s_{\eta}\in L_{2}^{0}.$

Define the tangent space of nuisance parameters as

$T_{0}=\\{s_{\eta}\in L_{2}^{0}:(\ref{msd})\text{ holds}\\}.$

Let $\overline{T_{0}}$ denote the closure of $T_{0}$ in the mean-square norm
topology. Our semiparametric models are regular in the sense that $T_{0}$ is a
linear subspace. Regularity and DQM are standard assumptions in the
semiparametric efficiency theory (see, e.g., Newey 1994). Also, for future
reference, we introduce the orthocomplement of $\overline{T_{0}}$ as
follows444Note that if
$\mathbb{E}\left[g\left(Z\right)s_{\eta}\left(Z\right)\right]=0$ for all
$s_{\eta}\in T_{0},$ then
$\mathbb{E}\left[g\left(Z\right)s_{\eta}\left(Z\right)\right]=0$ for all
$s_{\eta}\in\overline{T_{0}},$ by continuity of the inner product.

$\overline{T_{0}}^{\perp}=\\{g\in
L_{2}:\mathbb{E}\left[g\left(Z\right)s_{\eta}\left(Z\right)\right]=0\text{ for
all }s_{\eta}\in T_{0}\\}.$

As in Neyman (1959), we require regularity conditions on the moments to be
able to interchange derivatives and moments. Let $\mathbb{E}_{\tau}$ denote
the expectation under the path $\mathbb{P}_{\tau}.$

###### Definition 2.1

We denote the class of Cramer moments $\mathcal{G}_{0}$ as the set of $g\in
L_{2}^{0}$ for which, for all paths satisfying (2.5), the derivatives
$d\mathbb{E}[g(Z,\bar{\theta},\eta_{\tau})]/d\tau$ and
$d\mathbb{E}_{\tau}[g(Z,\bar{\theta},\eta_{0})]/d\tau$ are well defined, and
moreover

$\frac{d}{d\tau}\mathbb{E}_{\tau}[g(Z,\bar{\theta},\eta_{0})]=\mathbb{E}[g(Z,\bar{\theta},\eta_{0})s_{\eta}(Z)].$
(2.6)

These regularity conditions are standard in the semiparametric literature,
see, e.g., Newey (1990, 1994). Condition (2.6) holds, for example, when $g$ is
bounded. More generally, suppose

$g(Z,\theta,\eta)=m(Z)-{\textstyle\int}m(z)f_{\theta,\eta}(z)d\mu(z),$

for a measurable moment $m.$ Then, $g\in\mathcal{G}_{0}$ provided for all
paths (2.5)

$\lim\sup_{\tau\downarrow
0}{\textstyle\int}m^{2}(z)f_{\tau}(z)d\mu(z)<\infty.$

The following condition is a maintained regularity assumption throughout the
paper when referring to moment functions.

###### Assumption 1

$0\neq g\in\mathcal{G}_{0}.$

A formal definition of Restricted Local Surjectivity (RLS) and Restricted
Local Non-surjectivity (RLN) is given as follows. For the moment, we work
under the null hypothesis, so all our definitions apply to
$\theta_{0}=\bar{\theta}.$ We will investigate power in Section 2.2.2. We
introduce the definition of RLS and RLN.

###### Definition 2.2

The model $\mathcal{P}$ satisfies RLS at $\theta_{0}=\bar{\theta}$ when the
tangent space $T_{0}$ is dense in $L_{2}^{0},$ i.e.
$\overline{T_{0}}=L_{2}^{0}$, and RLN when there is $0\neq
g\in\overline{T_{0}}^{\perp}\cap\mathcal{G}_{0}.$

The intersection with $\mathcal{G}_{0}$ in the definition of RLN refers to the
aforementioned maintained regularity condition. We aim to relate the concept
of non-surjectivity with that of LR/orthogonal moments, so we introduce
formally the latter.555For a related definition of LR in CEINR and the
relation to the present definition see Section 2.2.1.

###### Definition 2.3

The moment function $g(Z,\bar{\theta},\eta_{0})\neq 0$ is LR with respect to
$T_{0}\ $if $g\in\mathcal{G}_{0}$ and

$\frac{d}{d\tau}\mathbb{E}[g(Z,\bar{\theta},\eta_{\tau})]=0,$ (2.7)

for all paths satisfying (2.5).

###  2.2 Main Result: Existence of Orthogonal Moments

The implications of RLN on robustness can be seen as follows. For any score
function $g$ and parametric submodel under the null hypothesis it holds

$\mathbb{E}_{\tau}[g(Z,\bar{\theta},\eta_{\tau})]=0,\text{ all
}\tau\in[0,\varepsilon).$ (2.8)

Differentiating this equation, we obtain by the chain rule and (2.6)

$\displaystyle 0$
$\displaystyle=\frac{d}{d\tau}\mathbb{E}_{\tau}[g(Z,\bar{\theta},\eta_{0})]+\frac{d}{d\tau}\mathbb{E}[g(Z,\bar{\theta},\eta_{\tau})]$
$\displaystyle=\mathbb{E}[g(Z,\bar{\theta},\eta_{0})s_{\eta}(Z)]+\frac{d}{d\tau}\mathbb{E}[g(Z,\bar{\theta},\eta_{\tau})].$
(2.9)

Local robustness further implies that the second term is zero, so the first
term must be zero for all scores $s_{\eta}$ in $\overline{T_{0}}.$ From this
equation it follows:

###### Theorem 2.4

RLN is necessary and sufficient for existence of LR moments.

Equation (2.9) explains the link between orthogonality and LR. A LR moment $g$
necessarily satisfies the orthogonality condition

$\mathbb{E}[g(Z,\bar{\theta},\eta_{0})s_{\eta}(Z)]=0\text{ for all
}s_{\eta}\in T_{0}.$

Given the importance of RLN for debiased inference, we establish some
sufficient and necessary conditions for it in the next subsections.

####  2.2.1 Sufficient conditions for RLN based on influence functions

Our definition of LR/orthogonality is model-based, while the construction of
orthogonal moments in CEINR is estimation-based. There is, of course, a close
relation between the two, as we now show. Suppose $\hat{\eta}$ is a consistent
estimator for $\eta_{0}$ in the model $\mathcal{P}_{0}.$ Let
$\eta(\mathbb{P})$ denote the probabilistic limit of $\hat{\eta}$ under
general misspecification, i.e. when the data distribution is given by
$\mathbb{P},$ consistent with $\eta_{0}=\eta(\mathbb{P}_{0})$. The First Step
Influence Function (FSIF) $\phi^{LR}$ pertaining to an original moment
function $m$ and $\eta(\mathbb{P})$ is defined in CEINR as the unique function
$\phi^{LR}$ with zero-mean and finite variance satisfying

$\frac{d}{d\tau}\mathbb{E}[m(Z,\bar{\theta},\eta_{\tau})]=\mathbb{E}[\phi^{LR}(Z,\bar{\theta},\eta_{0})s_{\eta}(Z)],$
(2.10)

for all paths $\mathbb{P}_{\tau},$ $\eta_{\tau}=\eta(\mathbb{P}_{\tau}),$ with
corresponding scores $s_{\eta}(Z)$ such that their linear span is dense in
$L_{2}^{0}.$ Having scores with this property is the precise meaning of
“misspecification” considered first in Newey (1994). Under these conditions,
CEINR showed in great generality that $g^{LR}=m+\phi^{LR}$ is a LR moment.
Their definition is estimation based because it is relative to the mapping
$\eta(\mathbb{P})$ defined by the estimator $\hat{\eta},$ while ours is model-
based because it is relative to the model $\mathcal{P}.$

A key observation of our paper is that as long as $\hat{\eta}$ is a consistent
estimator for $\eta_{0}$ in a model satisfying RLS, the precise meaning of
misspecification in Newey (1994) holds. Thus, replacing $g$ by $m$ in (2.9)
and using the definition of $\phi^{LR}$ in (2.10), it follows that
$\phi^{LR}=-m,$ thereby leading to a zero LR moment in the construction
proposed by CEINR for all identifying moment functions. We state the result in
the next Proposition.

###### Proposition 2.1

RLS implies that the LR moment of CEINR is zero for all identifying moment
functions.

We illustrate Proposition 2.1 with a running example. This example also serves
to illustrate how our results are also applicable to conditional moment
restrictions.

Example 1: Partly Linear Model with Endogeneity. Consider a data observation
$Z=(Y,W)$ where $Y=(Y_{1},Y_{2})$ are endogenous variables and $W=(X,Z_{2})$
are exogenous variables, with $X\ $controls and $Z_{2}$ Instrumental Variables
(IV), following the model

$Y_{1}=\theta_{0}Y_{2}+\eta_{0}\left(X\right)+\varepsilon,\text{
}\mathbb{E}\left[\left.\varepsilon\right|W\right]=0\text{ a.s.}$

This is the standard partly linear model (Robinson, 1988) commonly used in
applied work, possibly with a flexible high dimensional specification for
$\eta_{0}$. Suppose the researcher applies CEINR with an original identifying
moment function given by the IV moment
$m(Z,\theta_{0},\eta_{0})=(Y_{1}-\theta_{0}Y_{2}-\eta_{0}\left(X\right))Z_{2}$
and a consistent nonparametric first step estimator $\hat{\eta}$ for
$\eta(\mathbb{P})=\mathbb{E}_{\mathbb{P}}\left[\left.Y_{1}-\theta_{0}Y_{2}\right|X\right]$
to do inference on the structural parameter $\theta_{0}$. CEINR considered
paths $\mathbb{P}_{\tau}$ with CDF $F_{\tau}=(1-\tau)F_{0}+\tau G$ for
$\tau\in[0,1],$ where $F_{0}$ denotes the CDF of $\mathbb{P}_{0},$ and $G$ is
some alternative distribution. We assume that $G$ is chosen so that
$\eta_{\tau}\left(X\right)=\mathbb{E}_{\tau}\left[\left.Y_{1}-\theta_{0}Y_{2}\right|X\right]$
exists and possibly other regularity conditions are satisfied (such as
existence and square-integrability of scores). The corresponding FSIF follows
from taking derivatives in the orthogonality condition

$\mathbb{E}_{\tau}[(Y_{1}-\theta_{0}Y_{2}-\eta_{\tau}\left(X\right))\mathbb{E}\left[\left.Z_{2}\right|X\right]]=0,$

so that

$\displaystyle\frac{d}{d\tau}\mathbb{E}[m(Z,\theta_{0},\eta_{\tau})]$
$\displaystyle=-\frac{d}{d\tau}\mathbb{E}_{\tau}[(Y_{1}-\theta_{0}Y_{2}-\eta_{0}\left(X\right))\mathbb{E}\left[\left.Z_{2}\right|X\right]]$
$\displaystyle=-\mathbb{E}[(Y_{1}-\theta_{0}Y_{2}-\eta_{0}\left(X\right))\mathbb{E}\left[\left.Z_{2}\right|X\right]s_{\eta}(Z)].$

The FSIF of CEINR is thus
$\phi^{LR}=-(Y_{1}-\theta_{0}Y_{2}-\eta_{0}\left(X\right))\mathbb{E}\left[\left.Z_{2}\right|X\right]$
and the LR moment of CEINR boils down to the Double-Machine-Learning (DML)
approach of Chernozhukov et al. (2018), $g^{LR}=m+\phi^{LR},$ i.e.

$g^{LR}(Z,\theta_{0},\eta_{0})=(Y_{1}-\theta_{0}Y_{2}-\eta_{0}\left(X\right))\left(Z_{2}-\mathbb{E}\left[\left.Z_{2}\right|X\right]\right).$

If there are no exclusion restrictions, in the sense that $Z_{2}\subset X,$
then $\mathbb{E}\left[\left.Z_{2}\right|X\right]=Z_{2},$ $\phi^{LR}=-m$ and
$g^{LR}\equiv 0.$ Indeed, one can show (see Chernozhukov et al. (2018) and
Section 5 for a more general result)

$\overline{T_{0}}^{\perp}=\\{g\in
L_{2}^{0}:g\left(Z\right)=(Y_{1}-\theta_{0}Y_{2}-\eta_{0}\left(X\right))\left(\zeta(W)-\mathbb{E}\left[\left.\zeta(W)\right|X\right]\right),\text{
}\zeta\in L_{2}\\}.$

It is then clear that $\overline{T_{0}}^{\perp}=\\{0\\}$ iff $W=X.$ Exclusion
restrictions, in the sense of existence of an exogenous variable distinct from
$X,$ are sufficient and necessary for the existence of LR moments. Exclusion
restrictions are a minimal requirement for inference on $\theta_{0}$ in this
example. $\square$

####  2.2.2 When are orthogonal moments informative?

The concepts of RLS and RLN involve only the null hypothesis. This corresponds
to our desire to have tests with controlled size under a general set of
circumstances, e.g. with high dimensional nuisance parameters. We now turn
into local power considerations. To that end, we also need to incorporate
paths that change the parameter of interest. We consider paths
$\tau\in[0,\varepsilon)\mapsto\mathbb{P}_{\tau}\equiv\mathbb{P}_{\lambda_{\tau}}\in\mathcal{P},$
$\lambda_{\tau}=(\theta_{\tau},\eta_{\tau}),$ satisfying DQM

$\int\left[\frac{f_{\tau}^{1/2}-f_{0}^{1/2}}{\tau}-\frac{1}{2}sf_{0}^{1/2}\right]^{2}d\mu\rightarrow
0,\text{ as }\tau\downarrow 0,$ (2.11)

where again $f_{\tau}$ is density of $\mathbb{P}_{\tau}$ and $s\in L_{2}^{0}$
is the score of the path. By the chain rule, the score of the model $s$ has
the representation $s=\delta^{\prime}s_{\theta}+s_{\eta},$ for a score
$s_{\eta}$ for the nuisance parameter $\eta_{0},$ a vector of constants
$\delta\in\mathbb{R}^{p}$ and score for the parameter of interest
$s_{\theta},$ often obtained as in (2.2). Similarly to the restricted model,
define the full tangent space of the model as

$T=\\{s\in L_{2}^{0}:(\ref{QMD})\text{ holds}\\}.$

Let $\overline{T}$ denote the closure of $T$ in the mean-square norm topology.

We define Cramer moments with respect to the full parameter.

###### Definition 2.5

We denote the class of Cramer moments $\mathcal{G}$ as the moments $g\in
L_{2}^{0}$ for which for all paths satisfying (2.11) the derivatives
$d\mathbb{E}[g(Z,\theta_{\tau},\eta_{\tau})]/d\tau$ and
$d\mathbb{E}_{\tau}[g(Z,\theta_{0},\eta_{0})]/d\tau$ are well defined, and
moreover

$\frac{d}{d\tau}\mathbb{E}_{\tau}[g(Z,\theta_{0},\eta_{0})]=\mathbb{E}[g(Z,\theta_{0},\eta_{0})s(Z)].$

The power of score-based tests comes from the non-zero mean property at values
$\theta_{0}\neq\bar{\theta}.$ For any score function $g$ and parametric
submodel within the model it holds

$\mathbb{E}_{\tau}[g(Z,\theta_{\tau},\eta_{\tau})]=0,\text{ all
}\tau\in[0,\varepsilon).$ (2.12)

If, in addition, $g$ is a Cramer orthogonal moment, then by
$s=\delta^{\prime}s_{\theta}+s_{\eta}$ and orthogonality

$\displaystyle\frac{d}{d\tau}\mathbb{E}[g(Z,\theta_{\tau},\eta_{0})]$
$\displaystyle=-\frac{d}{d\tau}\mathbb{E}_{\tau}[g(Z,\theta_{0},\eta_{0})]$
$\displaystyle=-\mathbb{E}[g(Z,\theta_{0},\eta_{0})s(Z)]$
$\displaystyle=-\mathbb{E}[g(Z,\theta_{0},\eta_{0})s_{\theta}^{\prime}(Z)]\delta$
$\displaystyle=-\mathbb{E}[g(Z,\theta_{0},\eta_{0})\tilde{s}_{\theta}^{\prime}(Z)]\delta,$
(2.13)

where $\tilde{s}_{\theta}(Z)$ is the so-called efficient score, the projection
of $s_{\theta}$ onto $\overline{T_{0}}^{\perp}$, i.e.

$\tilde{s}_{\theta}(Z)=s_{\theta}(Z)-\Pi_{\overline{T_{0}}}s_{\theta}(Z),$

where $\Pi_{V}$ denotes the orthogonal projection operator onto the closed
subspace $V.$ Several important implications follow from this result. First,
the slope in (2.13) is maximized in absolute value (and subject to a
normalization) by $g=\delta^{\prime}\tilde{s}_{\theta}.$ Under RLS,
$\overline{T_{0}}^{\perp}=\\{0\\}$ and hence $\tilde{s}_{\theta}=0.$
Therefore, RLN is necessary for the local power to be non-trivial. Second,
$\tilde{s}_{\theta}$ may be zero even under RLN. Thus, RLN, and hence
existence of orthogonal moments, may not be sufficient for non-trivial local
power, i.e. for orthogonal moments to be informative about the parameter of
interest, as our running example illustrates.

Example 1: Partly Linear Model with Endogeneity, cont. An orthogonal moment in
this example is

$g\left(Z,\theta_{0},\eta_{0}\right)=(Y_{1}-\theta_{0}Y_{2}-\eta_{0}\left(X\right))\left(\zeta(W)-\mathbb{E}\left[\left.\zeta(W)\right|X\right]\right),\text{
}\zeta\in L_{2}.$

For a path such that $\theta_{\tau}=\bar{\theta}+\tau\delta+o(1),$ the power
of the orthogonal moment-based test is determined by

$\displaystyle\frac{d}{d\tau}\mathbb{E}[g(Z,\theta_{\tau},\eta_{0})]$
$\displaystyle=-\mathbb{E}[Y_{2}\left(\zeta(W)-\mathbb{E}\left[\left.\zeta(W)\right|X\right]\right)]\delta$
$\displaystyle=-\mathbb{E}[\left(\mathbb{E}\left[\left.Y_{2}\right|W\right]-\mathbb{E}\left[\left.Y_{2}\right|X\right]\right)\zeta(W)]\delta.$

Note how RLN is necessary for the local power to be non-trivial: the slope is
zero if there are no exclusion restrictions and $W=X.$ However, RLN is not
sufficient for non-trivial local power. The slope will be also zero if,
conditional on $X,$ $Y_{2}$ is mean independent of $Z_{2}.$ This is the
classical IV relevance condition, which implies that orthogonal moments are
informative (e.g. by taking
$\zeta(W)=\zeta^{\ast}(W)=\mathbb{E}\left[\left.Y_{2}\right|W\right]-\mathbb{E}\left[\left.Y_{2}\right|X\right]$).
It is important to note that we do not need to compute $\tilde{s}_{\theta}$ in
this example to claim informativeness of orthogonal moments. We also note that
$\zeta^{\ast}(W)=\mathbb{E}\left[\left.Y_{2}\right|W\right]-\mathbb{E}\left[\left.Y_{2}\right|X\right]$
guarantees non-trivial power when $\zeta^{\ast}(W)\neq 0,$ but the common
choice $\zeta(W)=Z_{2}$ in DML may not lead to informative orthogonal moments.
As an illustration, when $Z_{2}$ is a binary IV, then $\zeta^{\ast}(W)=0$ a.s.
is equivalent to $Cov\left[\left.Y_{2},Z\right|X\right]=0$ a.s. (see
Proposition 3.1. in Caetano and Escanciano, 2021). Clearly,
$Cov\left[\left.Y_{2},Z\right|X\right]=\mathbb{E}\left[\left.\zeta^{\ast}(W)Z_{2}\right|X\right]=0$
implies $\mathbb{E}\left[\zeta^{\ast}(W)Z_{2}\right]=0,$ but the reciprocal is
generally not true. Our results on relevance can be used to select informative
orthogonal moments. $\square$

An important point of this paper is that the conditions for existence of
orthogonal moments (i.e. RLN) are separate from the conditions needed for
orthogonal moments to be informative, i.e. corresponding moment based tests to
have non-trivial asymptotic local power. If $\tilde{s}_{\theta}=0,$ orthogonal
moments may exist but they will have trivial local power. For orthogonal
moments to be informative it is additionally required that
$\tilde{s}_{\theta}\neq 0.$ When $\theta$ is univariate, as in Example 1,
relevance is equivalent to (regular and local) identification of $\theta_{0}$.

Example 1: Partly Linear Model with Endogeneity, cont. In this model
$\theta_{0}$ is identified iff $\zeta^{\ast}(W)\neq 0.$ Indeed,

$\theta_{0}=\frac{\mathbb{E}\left[\zeta^{\ast}(W)\tilde{Y}_{1}\right]}{\mathbb{E}\left[\left(\zeta^{\ast}(W)\right)^{2}\right]},$

where for a generic variable $V,$
$\tilde{V}=V-\mathbb{E}\left[\left.V\right|X\right].$ An orthogonal moment
which achieves identification under the minimal condition is

$g\left(Z\right)=(\tilde{Y}_{1}-\theta_{0}\tilde{Y}_{2})\zeta^{\ast}(W),$

corresponding to an IV moment. Inference based on this moment can be
implemented by machine learning methods, e.g. Lasso, Random Forest, Boosting,
estimating the conditional means $\mathbb{E}\left[\left.Y_{1}\right|X\right],$
$\mathbb{E}\left[\left.Y_{2}\right|X\right]$ and the long regression
$\mathbb{E}\left[\left.Y_{2}\right|W\right]$. $\square$

Sections 4 and 5 generalize the insights of Example 1 by providing a necessary
and sufficient condition for orthogonal moments to be informative for general
functionals and models without the need to compute the efficient score of the
functional.

To give further insights into the power of tests based on orthogonal moments,
the following representation is useful

$\overline{T_{0}}^{\perp}=\tilde{S}_{\theta}\oplus\overline{T}^{\perp},$

where $\tilde{S}_{\theta}$ is the linear span of $\tilde{s}_{\theta}.$ This
decomposition and our characterization of orthogonal moments allow us to write
any orthogonal moment $g$ as
$g=c_{g}^{\prime}\tilde{s}_{\theta}+\Pi_{\overline{T}^{\perp}}g,$ where
$c_{g}^{\prime}\tilde{s}_{\theta}$ is the orthogonal projection of $g$ onto
$\tilde{S}_{\theta}.$ The local power of the corresponding score test is
determined by the coefficients $c_{g},$ the efficient Fisher Information
matrix
$\tilde{I}_{\theta}=\mathbb{E}[\tilde{s}_{\theta}(Z)\tilde{s}_{\theta}^{\prime}(Z)]$
and the direction of departure $\delta,$ through the slope (2.13):

$\frac{d}{d\tau}\mathbb{E}[g(Z,\theta_{\tau},\eta_{0})]=-c_{g}^{\prime}\tilde{I}_{\theta}\delta\equiv-\langle
c_{g},\delta\rangle_{\tilde{I}_{\theta}},$

where $\langle\cdot,\cdot\rangle_{\tilde{I}_{\theta}}$ is the efficient Fisher
inner product.

Indeed, by Lecam’s Third Lemma and (2.13), under local alternatives
$H_{1n}:\theta_{0}=\bar{\theta}+\delta n^{-1/2}$ and for a scalar LR moment

$\sqrt{n}\mathbb{E}_{n}\left[g(Z,\bar{\theta},\eta_{0})\right]\longrightarrow_{d}N\left(c_{g}^{\prime}\tilde{I}_{\theta}\delta,\mathbb{E}\left[g^{2}(Z,\bar{\theta},\eta_{0})\right]\right).$

LR moments that are orthogonal to the full model are not informative about
$\theta_{0},$ because if $g\in\overline{T}^{\perp}$ then $c_{g}=0,$ and hence
the drift $c_{g}^{\prime}\tilde{I}_{\theta}\delta=0,$ regardless of the
direction $\delta$ and the rank of $\tilde{I}_{\theta}.$ The part
$\Pi_{\overline{T}^{\perp}}g$ of the orthogonal moment $g$ does not contribute
to the relevance (the drift or signal), while increasing the asymptotic
variance (the noise) of
$\sqrt{n}\mathbb{E}_{n}\left[g(Z,\bar{\theta},\eta_{0})\right]$. The drift is
also zero if $\tilde{I}_{\theta}=0,$ i.e. $\tilde{s}_{\theta}=0$ a.s. On the
contrary, if $\tilde{s}_{\theta}\neq 0$ then we can select a $\delta\neq 0$
and $g=\delta^{\prime}\tilde{s}_{\theta}$ such that
$\delta^{\prime}\tilde{I}_{\theta}\delta>0$ (select $\delta$ as the
eigenvector associated to a positive eigenvalue of $\tilde{I}_{\theta}).$ For
a detailed analysis of local power in the general multivariate case, see
Section S.1 of the Supplementary Appendix.

This discussion also explains why identification of the structural parameter
may not be necessary for the existence and relevance of orthogonal moments.
Relevance requires $\tilde{I}_{\theta}\neq 0,$ while local regular
identification requires full rank of $\tilde{I}_{\theta},$ see Rothenberg
(1971) and Escanciano (2022) for parametric and semiparametric models,
respectively. Existence of orthogonal moments and their relevance allows for a
singular Fisher information matrix $\tilde{I}_{\theta}$, though this matrix
has to be a non-zero matrix. In the scalar case, singular and zero matrix are
the same, but in the multivariate case they are not. We illustrate this
situation in Section S.2 in the Supplementary Appendix by showing that
exclusion restrictions are not necessary in sample selection models for
orthogonal moments to exist and being informative about the structural
parameter. This example is representative of large class of partially
identified models for which this situation arises. Lack of identification of
the structural parameter will do have an impact on power, though, as
directions of identification failure, i.e. $\delta^{\prime}s$ such that
$\delta^{\prime}\tilde{s}_{\theta}(Z)=0,$ will also have zero slope by (2.13).
These directions correspond to zeros of the efficient information matrix
$\tilde{I}_{\theta}=\mathbb{E}[\tilde{s}_{\theta}(Z)\tilde{s}_{\theta}^{\prime}(Z)].$

We also note that identification of nuisance parameters is not required for
the existence and relevance of orthogonal moments. Indeed, orthogonal moments
are defined to be robust, at least locally, to such identification failures.
In independent work, Lee (2022) has also documented the robustness to
identification failures of efficient score tests in likelihood settings with
weakly identified nuisance parameters. We generalize these results to non-
likelihood settings, general parameters of interest, and possibly non-
identified nuisance parameters. Section S.2 further discusses the relation
between identification failures and existence of orthogonal moments,
illustrating these insights with examples in sample selection models and
measurement error models.

##  3 Models with Unobserved Heterogeneity

###  3.1 Basic Model and Results

In this section we specialize the previous findings to an important class of
models in econometrics: models with Unobserved Heterogeneity (UH). We focus on
models with a particular structure that commonly arise in settings with UH. We
assume an iid sample $\\{(Z_{i},\alpha_{i})\\},$ where $\alpha_{i}$ denotes
UH, a random vector of arbitrary dimension $d_{\alpha}$. In this framework,
$\eta_{0}$ is the density of UH with respect to a $\sigma$-finite measure
$\nu.$ We assume regularity of the model, in the sense that there is a linear
tangent space $T(\eta_{0})\subset L_{2}^{0}(\eta_{0})$ for UH, such that the
nuisance score has the representation

$s_{\eta}(Z)=\mathbb{E}\left[\left.b(\alpha)\right|Z\right]\text{ for some
}b\in T(\eta_{0})\subset L_{2}^{0}(\eta_{0}).$ (3.1)

Regularity of this model has been shown under great generality, see, e.g., Le
Cam and Yang (1988) and Lemma 25.34 in Van der Vaart (1998).

The conditional mean structure of $s_{\eta}$ will have important implications
for orthogonality, as we now show. By iterated expectations

$\displaystyle\mathbb{E}[g(Z,\bar{\theta},\eta_{0})s_{\eta}(Z)]$
$\displaystyle=\mathbb{E}[g(Z,\bar{\theta},\eta_{0})b(\alpha)]$
$\displaystyle=\mathbb{E}[\mathbb{E}\left[\left.g(Z,\bar{\theta},\eta_{0})\right|\alpha\right]b(\alpha)]$
$\displaystyle=\mathbb{E}[\Pi_{\overline{T(\eta_{0})}}\mathbb{E}\left[\left.g(Z,\bar{\theta},\eta_{0})\right|\alpha\right]b(\alpha)].$
(3.2)

If all $s_{\eta}$ are obtained as (3.1) for different $b^{\prime}s,$ $b\in
T(\eta_{0}),$ then, orthogonality of $g$ is equivalent to orthogonality of
$\mathbb{E}\left[\left.g(Z,\bar{\theta},\eta_{0})\right|\alpha\right]$ to
$T(\eta_{0}).$ In particular, a sufficient condition for orthogonality is

$\mathbb{E}\left[\left.g(Z,\bar{\theta},\eta_{0})\right|\alpha\right]=0\text{
}\eta_{0}-a.s.$ (3.3)

Such condition will be also necessary if
$\mathbb{E}\left[\left.g(Z,\bar{\theta},\eta_{0})\right|\alpha\right]$ belongs
to the mean square closure of $T(\eta_{0}),$ as stated in the following
results. Henceforth, the symbol $\perp$ denotes orthogonality in the
corresponding Hilbert space inner product.

###### Theorem 3.1

If (3.1) holds, then a necessary and sufficient condition for LR is the
existence of $g\neq 0$ such that

$\mathbb{E}\left[\left.g(Z,\bar{\theta},\eta_{0})\right|\alpha\right]\perp
T(\eta_{0}).$

The following Corollary applies when UH is nonparametric, i.e. when
$T(\eta_{0})$ is dense in $L_{2}^{0}(\eta_{0})$.

###### Corollary 3.1

If (3.1) holds and UH is nonparametric, then (3.3) with $g\neq 0$ is necessary
and sufficient for existence of a LR moment.

These results apply to the following general class of semiparametric mixture
models

$f_{\theta_{0},\eta_{0}}\left(z\right)=\int
f_{z|\alpha}\left(z|\alpha;\theta_{0}\right)\eta_{0}\left(\alpha\right)d\nu\left(\alpha\right),$
(3.4)

where the conditional density $f_{z|\alpha}\left(z|\alpha;\theta_{0}\right)$
is known up to the parameter $\theta_{0}\in\Theta\subset\mathbb{R}^{p},$ and
again $\eta_{0}$ is the unknown density of the UH (with some abuse of notation
we use the same $\alpha$ for denoting the random vector and its realization).
Under mild smoothness conditions, scores of this model satisfy (3.1) with
$b(\alpha)=d\log\eta_{\tau}/d\tau$. If UH is nonparametric, then by Corollary
3.1 a necessary and sufficient condition for LR is existence of $0\neq g\in
L_{2}$ such that

$\int
g(z,\theta_{0},\eta_{0})f_{z|\alpha}\left(z|\alpha;\theta_{0}\right)d\mu(z)=0,\text{
}\eta_{0}-a.s.$ (3.5)

If such a function $g$ exists, it may depend on $\eta_{0}$ through its support
only, since $f_{z|\alpha}\left(z|\alpha;\theta_{0}\right)$ does not depend on
$\eta_{0}$. The following example illustrates this point.

Example 2: Normal means model with general UH. Consider a data observation
$Z_{i}=\alpha_{i}+\sqrt{\theta_{0}}u_{i},$ $u_{i}\sim N(0,1),$ $\alpha_{i}$
independent of $u_{i},$ with a density given by $\eta_{0}.$ This model has
been extensively studied in the literature, see e.g. Laird (1978), Fan (1991),
Hall and Meister (2007) and Butucea and Comte (2009), often assuming
$\theta_{0}=1\ $and that $\eta_{0}$ is an absolutely continuous density with
respect to the Lebesgue measure on $\mathbb{R}$. The necessary and sufficient
condition for existence of a LR moment in a general version of this example
where UH is not necessarily absolutely continuous is

$\int
g(z,\bar{\theta},\eta_{0})\phi(\left(z-\alpha\right)/\sqrt{\bar{\theta}})dz=0\text{
}\eta_{0}-a.s.$

where $\phi$ is the standard Gaussian pdf. If the true distribution of UH is
discrete with, say, $J$ points of support $\alpha_{1},...,\alpha_{J}$,
$J<\infty,$ then there exists a LR moment, but it will depend on $\eta_{0}$
through its support points. In fact, we can construct such LR moment as
follows: define the functions
$\phi_{j}(z):=\phi(\left(z-\alpha_{j}\right)/\sqrt{\bar{\theta}}),$ which
depend on $\eta_{0}$ through its support, and for an original moment function
$m(z)$ define

$g\left(Z,\bar{\theta},\eta_{0}\right)=m\left(Z\right)-\beta_{m}^{\prime}\bar{\phi}\left(Z,\bar{\theta},\eta_{0}\right),$

where
$\beta_{m}=\langle\bar{\phi},\bar{\phi}^{\prime}\rangle^{-1}\langle\bar{\phi},m\rangle,$
$\bar{\phi}\equiv(\phi_{1},...,\phi_{J})^{\prime}$, and $\langle
f,g\rangle=\int f(z)g(z)dz.$ In this example, there is a fundamental
difference between the necessary and sufficient condition (3.5) and a
sufficient condition that checks for (3.5) to hold for all
$\alpha\in\mathbb{R}$. As we show in Section S.2 in the Supplementary Appendix
there is no non-zero solution to (3.5) when $\eta_{0}-a.s$ is replaced by all
$\alpha\in\mathbb{R}$. $\square$

In many different settings, researchers might not be interested in $\eta_{0},$
as their interest might rely exclusively on $\theta_{0}$. Because of this, we
now give assumptions under which a LR moment $g$ will be free of the support
of $\eta_{0},$ that is, assumptions under which the stronger condition holds

$\mathbb{E}\left[\left.g(Z,\bar{\theta})\right|\alpha\right]=0\text{ all
}\alpha\in\mathcal{A}\subset\mathbb{R}^{d_{\alpha}},$ (3.6)

where $\mathcal{A}$ is a known connected open set of $\mathbb{R}^{d_{\alpha}}$
which contains the support of $\eta_{0}$, denoted by $\mathcal{S}.$

If there is a moment $g$ satisfying (3.6), it will be necessarily NF since
$f_{z|\alpha}\left(z|\alpha;\theta_{0}\right)$ does not depend on $\eta_{0}\
$and $\mathcal{A}$ is known. A real analytic function is an infinitely
differentiable function such that its power expansion coincides with its
Taylor series.

###### Proposition 3.1

If $\mathcal{S}$ contains an open set $U$ and the left hand side of (3.6) is
real analytic on $\mathcal{A}$, then a necessary and sufficient condition for
LR is (3.6) and hence $g(Z,\theta_{0})$ is NF.

In the Supplementary Appendix (Section S.4) we give primitive conditions for
the left hand side of (3.6) to be real analytic in exponential families. Many
popularly used models satisfy the analytic property, including various forms
of logit models, duration models, etc. Fox et al. (2012) also used
analyticity, but for the different problem of identifying the distribution of
UH in the mixed logit model. We note that the assumption that $\mathcal{A}$
contains the support of $\eta_{0}$, $\mathcal{S},$ which is used here and in
the functional differencing literature, can be relaxed to just
$U\subset\mathcal{A}.$ Also, in the univariate UH case the condition
$U\subset\mathcal{S}$ can be relaxed to $\mathcal{S}$ containing a limit point
rather than a whole open set (cf. Krantz and Parks (2002)).

###  3.2 Introducing covariates and functional differencing

The results of the previous section can be generalized to the inclusion of
covariates. Now the data observation is $Z=(Y,X),$ where $Y$ is a vector of
dependent variables and $X$ a vector of covariates and possibly other
variables such as initial conditions, $\eta_{0}\left(\alpha|x\right)$ denotes
the conditional density of UH given covariates $X=x,$ and the density of the
observed data is

$f_{\theta_{0},\eta_{0}}\left(z\right)=\int
f_{y|\alpha,x}\left(y|\alpha,x;\theta_{0}\right)\eta_{0}\left(\alpha|x\right)d\nu\left(\alpha\right).$
(3.7)

In this specification $f_{\theta_{0},\eta_{0}}$ is the density of
$\mathbb{P}_{0}$ with respect to $\mu=\mu_{Y}\times v_{X},$ where $\mu_{Y}$ is
a $\sigma$-finite measure and $v_{X}$ is the probability measure of $X.$ The
conditional density of heterogeneity $\eta_{0}\left(\alpha|x\right)$ depends
on $x$ in an unrestricted way, as in a fixed effects approach.666This setting
includes panel data applications with $Y_{i}=(Y_{i1},...,Y_{iT}),$ for a
finite number of periods $T<\infty.$ The vector $X_{i}$ includes predetermined
covariates $W_{i}=(W_{i1},...,W_{iT}),$ as well as possibly initial values
$Y_{i0}.$ Under mild smoothness conditions, scores of the nuisance parameter
of this model satisfy

$s_{\eta}(Z)=\mathbb{E}\left[\left.b(\alpha,X)\right|Z\right]\text{ for some
}b\in T(\eta_{0})\subset\mathcal{H}\equiv L_{2}(\eta_{0}\times f_{X}),$ (3.8)

where $b(\alpha,x)=d\log\eta_{\tau}\left(\alpha|x\right)/d\tau$ is the
conditional score and $f_{X}$ is the density of $v_{X}$ with respect to some
$\sigma$-finite measure $\mu_{X}.$

As in the previous subsection, by iterated expectations

$\mathbb{E}[g(Z,\theta_{0},\eta_{0})s_{\eta}(Z)]=\mathbb{E}[\Pi_{\overline{T(\eta_{0})}}\mathbb{E}\left[\left.g(Z,\theta_{0},\eta_{0})\right|\alpha,X\right]b(\alpha,X)].$

From this, an analogous result to Theorem 3.1 with covariates holds. We only
give the case with nonparametric UH, i.e. fixed effects setting, as the
semiparametric UH, i.e. correlated effects case, follows mutatis mutandis as
in the model without covariates.

###### Proposition 3.2

A necessary and sufficient condition for existence of a LR moment in (3.7)
with nonparametric UH is existence of $0\neq g\in L_{2}$ such that

$\mathbb{E}\left[\left.g(Z,\bar{\theta},\eta_{0})\right|X,\alpha\right]=0\text{,
a.s}.$ (3.9)

We relate this result with the functional differencing approach of Bonhomme
(2012). He starts with the apparently different goal of finding a NF moment
$g_{FD}(z,\theta_{0})$ such that

$\int
g_{FD}(z,\theta_{0})L_{\theta_{0},x}f\left(y\right)d\mu_{Y}\left(y\right)=0,\text{
}v_{X}-\text{a.s.},\text{ all }f\in L_{2}(\pi_{\alpha}),$ (3.10)

where $L_{\theta_{0},x}$ is the linear mapping

$L_{\theta_{0},x}f\left(y\right)=\int_{\mathcal{A}}f_{y|\alpha,x}\left(y|\alpha,x;\theta_{0}\right)f\left(\alpha\right)d\nu\left(\alpha\right),$

which is defined on $L_{2}(\pi_{\alpha}),$ for a user-specific weight function
$\pi_{\alpha},$ and where $\mathcal{A}$ is a known set that contains the
support of $\eta_{0}\left(\alpha|x\right),$ for all $x.$ Bonhomme (2012,
Theorem 2) then shows that if $g_{FD}$ satisfies (3.10), then it necessarily
satisfies

$\mathbb{E}\left[\left.\pi_{Y}(Y)g_{FD}(X,\theta_{0})\right|X\right]=0,\text{
}v_{X}-a.s.,$

for some user-specific weights $\pi_{Y}$. We provide conditions in the
Supplementary Appendix (Section S.3) under which $g=\pi_{Y}g_{FD}$ satisfies
the stronger moment restriction

$\mathbb{E}\left[\left.\pi_{Y}(Y)g_{FD}(X,\theta_{0})\right|X,\alpha\right]=0,\text{
}v_{X}\times\pi_{\alpha}-a.s.$

An even stronger version is

$\mathbb{E}\left[\left.g(Z,\theta_{0})\right|X,\alpha\right]=0,\text{
}v_{X}-a.s.,\text{ {all} }\alpha\in\mathcal{A},$ (3.11)

for some $g\neq 0.$ Whether this equation holds for all $\alpha,$
$\pi_{\alpha}-$a.s., or only $\eta_{0}-$a.s. in $\alpha$ could be critical for
the existence of orthogonal moments, as the previous Example 2 illustrates. If
(3.11) holds, the LR moment function $g(y,x,\theta_{0})$ will not depend on
$\eta_{0},$ i.e. it will be NF.777For interesting applications of (3.11) to
dynamic logit discrete choice see Kitazawa (2013, 2016), and more prominently
Honoré and Weidner (2021). If the necessary condition (3.9) holds, without
(3.11) being true, then $g(y,x,\theta_{0},\eta_{0})$ may still exist but it
may depend on $\eta_{0}$ through its support.

In parallel with Proposition 3.1, we now give conditions under which (3.9) and
(3.11) are equivalent, and thus both are necessary and sufficient conditions
for LR.

###### Proposition 3.3

In the semiparametric mixture model (3.7), if
$\mathbb{E}\left[\left.g(Z,\theta_{0},\eta_{0})\right|\alpha,X\right]$ is a
real-analytic function of $\alpha,$ $v_{X}-$a.s., and the support of
$\eta_{0}\left(\cdot|x\right)$ contains an open set contained in
$\mathcal{A}$, $v_{X}-$a.s., then if (3.9) holds, (3.11) will also hold and
$g(Z,\theta_{0})$ will be NF.

This result shows that commonly used sufficient conditions for NF moments in
functional differencing, such as (3.11), are also necessary for existence of
LR moments under support conditions on UH and smoothness of the model. Thus,
functional differencing is a special case of our characterization. Without
these additional assumptions, the conditions of functional differencing are
sufficient when the support of $\pi_{\alpha}$ includes that of UH, but they
may not be necessary. Furthermore, if the support of $\pi_{\alpha}$ does not
include that of UH, then the moments from functional differencing may not be
orthogonal. A practical take away of this discussion is that although
orthogonal moments might depend on the UH (e.g. trough its support), they are
LR, which make them less sensitive to large regularization biases if UH is
estimated. This observation might be useful for proposing efficient functional
differencing methods through feasible versions of the efficient scores (cf.
Bonhomme, 2012).

Another important remark is that for other parameters of interest, such as
average marginal effects, finding NF moments seems much more difficult, if not
impossible. We generalize the characterization of LR moments to general
functionals in Section 4.

###  3.3 Random Coefficient Models

If UH is independent of the covariates, the density of the data is

$f_{\theta_{0},\eta_{0}}\left(z\right)=\int
f_{y|\alpha,x}\left(y|\alpha,x;\theta_{0}\right)\eta_{0}\left(\alpha\right)d\nu\left(\alpha\right).$
(3.12)

Under nonparametric UH, the necessary and sufficient condition for LR
simplifies to conditioning only on the UH, as shown in the next result.

###### Proposition 3.4

A necessary and sufficient condition for existence of a LR moment in (3.12)
with nonparametric UH is existence of $0\neq g\in L_{2}$ such that

$\mathbb{E}\left[\left.g(Z,\bar{\theta},\eta_{0})\right|\alpha\right]=0\text{,
}\eta_{0}-\text{a.s}.$ (3.13)

Propositions 3.2 and 3.4 imply that having the independence assumption between
UH and covariates substantially increases the possibility of existence of LR
moments. It is clear that
$\mathbb{E}\left[\left.g(Z,\bar{\theta},\eta_{0})\right|\alpha\right]=0$ a.s.
could hold with
$\mathbb{E}\left[\left.g(Z,\bar{\theta},\eta_{0})\right|X,\alpha\right]\neq
0,$ corresponding to the case where $g$ is only orthogonal in the random
coefficient model and not in a fixed effects setting. We illustrate this point
with an example.

Example 3: Linear Random Coefficient Model. Consider the linear random
coefficient model with $Y=\theta_{0}X_{1}+X_{2}^{\prime}\alpha$, where $X_{2}$
usually contains an intercept, so this model generalizes the classical linear
regression model to some coefficients being random. Take the moment function
$g(Z,\theta_{0},\eta_{0})=(\tilde{Y}-\theta_{0}\tilde{X}_{1})\tilde{X}_{1},$
where for a generic variable $V,$
$\tilde{V}=V-\mathbb{E}\left[\left.V\right|X_{2}\right].$ To see that this
moment is an orthogonal moment, note
$\tilde{Y}=\theta_{0}\tilde{X}_{1}+X_{2}^{\prime}(\alpha-\mathbb{E}\left[\alpha\right])$,
and check that our necessary and sufficient condition (3.13) holds

$\displaystyle\mathbb{E}\left[\left.g(Z,\bar{\theta},\eta_{0})\right|\alpha\right]$
$\displaystyle=\mathbb{E}\left[\left.\tilde{X}_{1}X_{2}^{\prime}(\alpha-\mathbb{E}\left[\alpha\right])\right|\alpha\right]$
$\displaystyle=\mathbb{E}\left[\tilde{X}_{1}X_{2}^{\prime}\right](\alpha-\mathbb{E}\left[\alpha\right])$
$\displaystyle=0\text{, }\eta_{0}-\text{a.s}.$

In contrast,
$\mathbb{E}\left[\left.g(Z,\bar{\theta},\eta_{0})\right|X,\alpha\right]=\tilde{X}_{1}X_{2}^{\prime}(\alpha-\mathbb{E}\left[\alpha\right])\neq
0,$ so this moment is not orthogonal in a fully fixed effects setting (though
it will be orthogonal in a model where $\alpha$ is only assumed to be
independent of $X_{1},$ conditional on $X_{2}).$ $\square$

##  4 General Models and Functionals of Interest

We generalize our previous results to general models and parameters of
interest. Consider again a generic semiparametric model
$\mathcal{P}=\\{\mathbb{P}_{\theta,\eta}:\theta\in\Theta,\eta\in\Xi\\},$ with
parameter space
$\Lambda=\\{\lambda=(\theta,\eta):\theta\in\Theta,\eta\in\Xi\\}.$ We assume,
by the chain rule, that the scores of the model have the representation
$s=S_{\lambda_{0}}h$, $h\in\mathbf{H},$ for a score operator $S_{\lambda_{0}}$
given by

$S_{\lambda_{0}}h=s_{\theta}^{\prime}\delta+S_{\eta}b,\text{ }h=(\delta,b)\in
T(\lambda_{0})\subseteq\mathbf{H},$ (4.1)

for a Hilbert space $\mathbf{H}=\mathbb{R}^{p}\times\mathcal{H},$ with inner
product
$\langle(\delta_{1},b_{1}),(\delta_{2},b_{2})\rangle_{\mathbf{H}}:=\delta_{1}^{\prime}\delta_{2}+\langle
b_{1},b_{2}\rangle_{\mathcal{H}}$. Here $s_{\theta}$ is the ordinary score
function of $\theta,$ and $S_{\eta}$ is the nuisance score operator from
$T(\eta_{0})\subset\mathcal{H}$ to $L_{2}$ (see, e.g., Begun, Hall, Huang and
Wellner (1983) for an introduction to score operators). In this section we
consider a parameter of interest given by a smooth functional of $\lambda$,
$\psi(\lambda)\in\mathbb{R}^{d_{\psi}},$ where the precise sense of smoothness
is defined below. Our previous results correspond to the leading example of
$\psi(\lambda)=\theta,$ but this current setting also covers other parameters
of interest such as average marginal effects or moments of the UH, as, for
example, in

$\psi(\lambda)=\mathbb{E}_{\eta_{0}}\left[r(\alpha)\right],\text{ }r\in
L_{2}(\eta_{0}).$

The current setting is also general by allowing other nuisance score operators
different from the conditional mean
$S_{\eta}b=\mathbb{E}\left[\left.b(\alpha)\right|Z\right]$ in models with UH.

As in the previous section, consider paths $\lambda_{\tau}$ with tangents
$h=(\delta,b)\in\mathbf{H}$ such that
$d\lambda_{\tau}/d\tau=(\delta,b\eta_{0}),$ and functionals $\psi(\cdot)$ such
that its derivative $\dot{\psi}(h)=d\psi(\lambda_{\tau})/d\tau\ $is linear and
continuous in $T(\lambda_{0})\subseteq\mathbf{H.}$888This parametrization of
paths in terms of $b\eta_{0}$ rather than $b$ is convenient for cases where
$\eta_{0}$ is a density, and fits our previous discussion. For other cases, we
often use paths with tangents $(\delta,b)$. Linearity and continuity of
$\dot{\psi}(h)\in\mathbb{R}^{d_{\psi}}$ guarantees, by the Riesz
representation theorem, the existence of a vector
$r_{\psi}=(r_{\psi,j})_{j=1}^{d_{\psi}},$ with $r_{\psi,j}\in T(\lambda_{0})$
for all $j=1,...,d_{\psi},$ such that

$\dot{\psi}(h)=\langle r_{\psi},h\rangle_{\mathbf{H}}.$

For multivariate functionals this representation is understood componentwise.
For example, for $\psi(\lambda)=\theta\in\mathbb{R}^{p},$ the corresponding
Riesz presenter $r_{\psi}$ is such that
$r_{\psi,j}=(e_{j},0)\in\mathbb{R}^{p}\times\mathcal{H},$ with $e_{j}$
denoting the $j-$th canonical vector with a one in the $j-th$ coordinate and
zero everywhere else. That is, if
$\theta_{\tau,j}=\theta_{0,j}+\tau\delta_{j}+o(1),$ then
$d\theta_{\tau,j}/d\tau=\delta_{j}=e_{j}^{\prime}\delta,$ where
$\theta_{0}=(\theta_{0,1},...,\theta_{0,p})^{\prime}$ and
$\delta=(\delta_{1},...,\delta_{p})^{\prime}.$

We show how all the previous theory of the paper can be extended to this more
general setting. The null and alternative hypotheses are now

$H_{0}:\psi(\lambda_{0})=\bar{\psi}\qquad vs\qquad
H_{1}:\psi(\lambda_{0})\neq\bar{\psi},$

for a known $\bar{\psi}\in\mathbb{R}^{d_{\psi}},$ and with other aspects of
$\lambda_{0}$ unknown under both $H_{0}$ and $H_{1}.$ The concepts of RLS and
RLN are the same, but now the restricted tangent space is the space of scores
from paths satisfying (2.11) such that $\psi(\lambda_{\tau})=\bar{\psi}$ for
all $\tau$ sufficiently small. A key object is the nuisance score operator
pertaining to $\psi(\lambda_{0}),$ which we define as

$S_{\psi_{0}^{\perp}}=S_{\lambda_{0}}\circ\Pi_{r_{\psi}^{\perp}},$

where $\circ$ denotes composition of operators, $\Pi_{r_{\psi}^{\perp}}$ is
the projection onto the orthocomplement of the linear span of
$(r_{\psi,j})_{j=1}^{d_{\psi}},$ and $S_{\lambda_{0}}$ is defined in (4.1).
The corresponding adjoint operator to $S_{\psi_{0}^{\perp}}$ satisfies

$\langle S_{\psi_{0}^{\perp}}h,s\rangle=\langle
h,S_{\psi_{0}^{\perp}}^{\ast}s\rangle_{\mathbf{H}},\text{ for all
}h\in\mathbf{H},\text{ }s\in L_{2}^{0},$

and it is given by

$S_{\psi_{0}^{\perp}}^{\ast}=\Pi_{r_{\psi}^{\perp}}\circ
S_{\lambda_{0}}^{\ast},$

where $S_{\lambda_{0}}^{\ast}$ is the adjoint operator of $S_{\lambda_{0}},$
given by

$S_{\lambda_{0}}^{\ast}g=\left(\mathbb{E}\left[s_{\theta}(Z)g(Z)\right],S_{\eta}^{\ast}g\right).$
(4.2)

As expected, for $\psi(\lambda)=\theta,$ $S_{\psi_{0}^{\perp}}=S_{\eta_{0}}$
and $S_{\psi_{0}^{\perp}}^{\ast}=S_{\eta_{0}}^{\ast}.$

We characterize orthogonal moments for inference on $\psi(\lambda_{0}).$
Henceforth, for a linear operator $K$ between the Hilbert Spaces
$\mathcal{H}_{1}$ and $\mathcal{H}_{2},$ define the range of $K$ as
$\mathcal{R}(K):=\\{f\in\mathcal{H}_{2}:$ $f=Kb$ some $b\in\mathcal{H}_{1}\\}$
and the kernel of $K$ as $\mathcal{N}(K):=\\{f\in\mathcal{H}_{1}:$ $Kf=0\\}.$
For multivariate functionals and $C\in\mathbb{R}^{d_{\psi}},$ we define
$C^{\prime}r_{\psi}=\sum_{j=1}^{d_{\psi}}C_{j}r_{\psi,j}\in\mathbf{H}.$

###### Proposition 4.1

Orthogonal moments for $\psi(\lambda_{0})$ in this setting are given by the
non-zero elements of $\mathcal{N}(S_{\psi_{0}^{\perp}}^{\ast}).$ Thus,
orthogonal moments exist iff
$\mathcal{N}(S_{\psi_{0}^{\perp}}^{\ast})\neq\left\\{0\right\\}.$ Furthermore,
$g\in\mathcal{N}(S_{\psi_{0}^{\perp}}^{\ast})\ $iff
$S_{\lambda_{0}}^{\ast}g=C^{\prime}r_{\psi},$ for $C\in\mathbb{R}^{d_{\psi}}.$

This result relies on a novel characterization of the tangent space of the
restricted model based on the Riesz representer and duality. It shows that RLS
boils down to the injectivity of the linear operator
$S_{\psi_{0}^{\perp}}^{\ast}$, similar to a completeness assumption. These
types of injectivity conditions are familiar in econometrics and have been
used in the nonparametric instrumental variable and measurement error
literatures; see e.g. Newey and Powell (2003) and Hu and Schennack (2008).

For the structural parameter $\psi(\lambda)=\theta,$
$S_{\lambda_{0}}^{\ast}g=C^{\prime}r_{\psi}$ implies, by (4.2) and
$r_{\psi,j}=(e_{j},0)),$ that $S_{\eta_{0}}^{\ast}g=0$, which corresponds to
the functional differencing moments (3.3) and (3.9) for models with UH
(without and with covariates, respectively). In a setting where the
conditional likelihood given UH is parametric, the nuisance score operator
$S_{\eta_{0}}^{\ast}$ is known under the null hypothesis (up to support
conditions), which gives rise to the possibility of functional differencing
(NF moments).

As another illustration, consider now the case
$\psi(\lambda)=\mathbb{E}_{\eta}\left[r(\alpha)\right]$ in the model (3.4)
with a known function $r\in L_{2}(\eta_{0})\equiv\mathcal{H}$ and
nonparametric UH. Then,

$\displaystyle\dot{\psi}(h)$ $\displaystyle=\int
r(\alpha)b\left(\alpha\right)\eta_{0}\left(\alpha\right)d\nu\left(\alpha\right)$
$\displaystyle=\langle r,b\rangle_{\mathcal{H}}.$

Therefore, the corresponding Riesz representer is
$r_{\psi}=(0,r-\psi(\lambda_{0}))\in\mathbf{H}.$999The centering in $r_{\psi}$
is necessary for $r_{\psi}\in T(\lambda_{0})=\mathbb{R}^{p}\times
L_{2}^{0}(\eta_{0}).$ Then, an orthogonal moment exists if we can find $0\neq
g\in L_{2}^{0}$ such that

$\mathbb{E}\left[s_{\theta}(Z,\lambda_{0})g(Z,\lambda_{0})\right]=0,$ (4.3)

and for a constant $C$

$\mathbb{E}\left[\left.g(Z,\lambda_{0})\right|\alpha\right]=C(r(\alpha)-\psi(\lambda_{0})),\text{
}\eta_{0}-\text{a.s}.$ (4.4)

Even when (4.4) can be solved for a moment $g(Z,\lambda_{0})$ only depending
on $\theta_{0}$ (since the conditional distribution given $\alpha$ depends
only on $\theta_{0}),$ the orthogonality condition in (4.3) will in general
re-introduce the dependence on $\lambda_{0}$ through the dependence on the
score $s_{\theta}(Z,\lambda_{0}).$

The distinct cases $C=0$ and $C\neq 0$ represent two fundamentally different
situations. The case $C=0$ allows for the possibility that the parameter of
interest, here $\psi(\lambda_{0}),$ does not have a finite efficiency bound.
The case $C\neq 0$ imposes a finite efficiency bound for $\psi(\lambda_{0})$,
as shown in Escanciano (2022). Also, Escanciano (2023) has shown that (4.4)
with $C\neq 0$ imposes smoothness conditions on $r$ for conditional
likelihoods that are smooth in the UH, so we expect that (4.4) with $C\neq 0$
will not hold for functionals such as CDFs or quantiles of the UH (which
correspond to discontinuous $r).$

As in the functional differencing literature, equation (4.4) is easier to
verify for discrete choice models, where it implies a system of equations with
a finite number of unknowns, and a number of equations equals the support of
$\eta_{0}.$ Automatic constructions for orthogonal moments satisfying (4.3)
and (4.4) will be carried out in a companion paper.

When are orthogonal moments informative in this general setting? The insights
from the local power investigation, our characterization of orthogonal moments
and duality reveal that orthogonal moments are not informative when the score
of the parameter of interest belongs to the closure of the restricted tangent
space. To see this, when the parameter of interest is the structural parameter
$\psi(\lambda)=\theta,$ from (2.13) and $S_{\eta_{0}}^{\ast}g=0,$ if
$s_{\theta}\in\mathcal{N}(S_{\eta_{0}}^{\ast})^{\perp}=\overline{\mathcal{R}(S_{\eta_{0}})}$

$\displaystyle\frac{d}{d\tau}\mathbb{E}[g(Z,\theta_{\tau},\eta_{0})]$
$\displaystyle=-\mathbb{E}[g(Z,\theta_{0},\eta_{0})s_{\theta}^{\prime}(Z)]\delta$
$\displaystyle=0.$

We generalize this insight to the general case. Define
$s_{\psi}:=S_{\lambda_{0}}r_{\psi},$ which plays the role of the score of the
parameter of interest in this more general setting. Here, the application of
$S_{\lambda_{0}}$ is componentwise when $\psi(\cdot)$ is multivariate.

###### Proposition 4.2

Orthogonal moments are informative for $\psi(\lambda_{0})$ iff
$s_{\psi}\notin\mathcal{N}(S_{\psi_{0}^{\perp}}^{\ast})^{\perp}.$

We can interpret the condition
$s_{\psi}\notin\mathcal{N}(S_{\psi_{0}^{\perp}}^{\ast})^{\perp}$ as a general
relevance condition. It generalizes the conditional relevance condition of IV
to any smooth parameter and any regular semiparametric model. For the
structural parameter $\psi(\lambda)=\theta,$ $s_{\psi}=s_{\theta},$
$S_{\psi_{0}^{\perp}}^{\ast}=S_{\eta_{0}}^{\ast},$ and
$s_{\theta}\notin\mathcal{N}(S_{\eta_{0}}^{\ast})^{\perp}$ means
$\tilde{s}_{\theta}\neq 0$ (the efficient Fisher information for the parameter
is not a matrix of zeros). This is the general interpretation of relevance.

As an illustration, we can answer a question that remained open in the
literature of functional differencing: when is a functional differencing
moment informative? Proposition 4.2 gives the answer: a LR moment
$g(Z,\theta_{0},\eta_{0})$ for $\psi(\lambda)=\theta$ is informative if
$\mathbb{E}[g(Z,\theta_{0},\eta_{0})s_{\theta}(Z)]\neq 0.$ This condition is
generally weaker than the local identification results for $\theta_{0}$
derived in Bonhomme (2012). Proposition 4.2 generalizes this to other
functionals. Next section considers a practically important class of models in
econometrics, conditional moment restriction models, generalizing Example 1.
In these models, the previous analysis carries over, but with score operators
replaced by simpler conditional moment derivatives.

##  5 Applications to Conditional Moment Restrictions

The partly linear model with endogeneity is a special case of a general class
of models defined by Conditional Moment Restrictions (CMR), for which the
results of this paper are applicable. Consider the semiparametric model such
that

$\mathcal{P}=\\{P:\mathbb{E}_{P}\left[\left.\rho_{j}(Y,X,\theta_{0},\eta_{0})\right|W_{j}\right]=0\text{
a.s. for all }j=1,...,J,\text{ }\theta_{0}\in\Theta,\text{
}\eta_{0}\in\Xi\\},$ (5.1)

where $\Theta\subset\mathbb{R}^{p}$ and $\Xi$ is possibly infinite
dimensional. These models have been studied in Ai and Chen (2007) and Chen and
Santos (2018). The data observation is $Z=(Y,X,W),$ where $W$ denotes the
union of distinct random elements of the conditioning variables $W_{j}.$

Define

$m_{j}(W_{j},\theta,\eta)=\mathbb{E}_{\mathbb{P}}\left[\left.\rho_{j}(Y,X,\theta,\eta)\right|W_{j}\right]$

and the derivatives

$\nabla m(W,\theta,\eta)[h]=(\nabla m_{1}(W_{1},\theta,\eta)[h],...,\nabla
m_{J}(W_{J},\theta,\eta)[h]),$

where

$\nabla
m_{j}(W_{j},\theta,\eta)[h]=\frac{d}{d\tau}m_{j}(W_{j},\theta+\tau\delta,\eta+\tau
b),\text{ }h=(\delta,b)\in T(\lambda_{0})\subseteq\mathbf{H}$

and $\mathbf{H}=\mathbb{R}^{p}\times\mathcal{H},$ for a Hilbert Space
$(\mathcal{H},\langle\cdot,\cdot\rangle_{\mathcal{H}}).$ As in previous
sections, when $\eta$ is a density we replace $\eta+\tau b$ by $\eta(1+\tau
b)$ for better interpretation. Let $L_{2}(W_{j})$ denote the Hilbert space
$L_{2}(f)$ when $f$ is the density of $W_{j},$ and define the linear mapping

$\nabla m:h\in\mathbf{H}\rightarrow\nabla
m(W_{a},\theta,\eta)[h]\in{\textstyle\bigotimes\nolimits_{j=1}^{J}}L_{2}(W_{j}).$

Define also

$M_{\psi_{0}^{\perp}}=\left(\nabla
m\right)\circ\Pi_{r_{\psi}^{\perp}}\qquad\text{and}\qquad
M_{\psi_{0}^{\perp}}^{\ast}=\Pi_{r_{\psi}^{\perp}}\circ\left(\nabla
m\right)^{\ast},$

where $\left(\nabla m\right)^{\ast}$ is the adjoint operator of $\nabla m.$
The following result combines Theorem 4.1 in Chen and Santos (2018) with our
results. Let $\psi(\lambda_{0})$ be a smooth functional, with Riesz
representer $r_{\psi}\in\overline{T(\lambda_{0})}$ such that, for all
$h\in\mathbf{H}$

$\dot{\psi}(h)=\langle r_{\psi},h\rangle_{\mathbf{H}}.$

###### Theorem 5.1

Let Assumptions 4.1 and 4.2 in Chen and Santos (2018) hold. Then, the set of
orthogonal moments for $\psi(\lambda_{0})$ is given by

$\overline{T_{0}}^{\perp}=\\{g\in
L_{2}:g(Z)={\textstyle\sum\nolimits_{j=1}^{J}}\rho_{j}(Z,\theta_{0},\eta_{0})\varphi_{j}(W_{j})\text{
for
}\varphi=(\varphi_{j})_{j=1}^{J}\in\mathcal{N}(M_{\psi_{0}^{\perp}}^{\ast})\\}.$

In particular, RLS holds for $\psi(\lambda_{0})$ in the model (5.1) if
$\mathcal{R}(\nabla m\circ\Pi_{r_{\psi}^{\perp}})$ is dense in
${\textstyle\bigotimes\nolimits_{j=1}^{J}}L_{2}(W_{j}).$ Equivalently,
orthogonal moments for $\psi(\lambda_{0})$ exist iff
$\mathcal{N}(M_{\psi_{0}^{\perp}}^{\ast})\neq\left\\{0\right\\}.$ Furthermore,
if $g$ is an orthogonal moment it has the representation in
$\overline{T_{0}}^{\perp}$ with $0\neq\varphi$ such that $(\nabla
m)^{\ast}\varphi=C^{\prime}r_{\psi}$ for a vector $C\in\mathbb{R}^{d_{\psi}}.$

Assumptions 4.1 and 4.2 in Chen and Santos (2018) are sufficient conditions
for regularity of the model (5.1). As a special case consider the structural
parameter $\psi(\lambda)=\theta,$ corresponding to $r_{\psi,j}=(e_{j},0).$ In
this case the deviation $h$ has $\delta=0,$ so it suffices to consider
$\nabla_{\eta}m_{j}(W_{j},\theta_{0},\eta)[b]=\frac{d}{d\tau}m_{j}(W_{j},\theta_{0},\eta+\tau
b),$ and RLS holds if

$\mathcal{R}(\nabla_{\eta}m)=\left\\{\nabla_{\eta}m(W_{j},\theta,\eta)[b]:b\in
T(\eta_{0})\right\\}$

is dense in ${\textstyle\bigotimes\nolimits_{j=1}^{J}}L_{2}(W_{j}),$ where
$\nabla_{\eta}m$ is the vector of conditional means derivatives with respect
to $\eta,$ keeping $\theta_{0}$ fixed. The condition
$\mathcal{N}(M_{\psi_{0}^{\perp}}^{\ast})\neq\\{0\\}$ can be interpreted as
the extension of exclusion restrictions in Example 1. We refer to the set
$\mathcal{N}(M_{\psi_{0}^{\perp}}^{\ast})$ as the set of “orthogonal
instruments”, as they are transformations of the conditioning variables
leading to orthogonal moments. This setting generalizes Example 1.

Example 1: Partly Linear Model with Endogeneity, cont. This model corresponds
to (5.1) with $J=1,$ and

$\rho_{1}(Z,\eta,\theta_{0})=Y_{1}-\theta_{0}Y_{2}-\eta_{0}\left(X\right).$

Here, $\nabla_{\eta}m(W,\eta,\theta_{0})[b]=-b(x)$ and
$\overline{\mathcal{R}}(\nabla_{\eta}m)=L_{2}(X).$ Therefore,

$\mathcal{N}(M_{\psi_{0}^{\perp}}^{\ast})=\left(L_{2}(X)\right)^{\perp}=\\{\varphi(W)=\zeta(W)-\mathbb{E}\left[\left.\zeta(W)\right|X\right],\text{
}\zeta\in L_{2}\\}.$

In particular, RLS holds iff $L_{2}(X)=L_{2}(W),$ i.e. iff $Z_{2}\subset X.$
$\square$

To see when orthogonal scores are informative on the functional of interest in
(5.1), we specialize Proposition 4.2 to this setting. Define $\nabla
m_{\psi}:=\left(\nabla m\right)[r_{\psi}],\ $where again the application of
$\nabla m$ is componentwise when $\psi(\cdot)$ is multivariate.

###### Proposition 5.1

Orthogonal moments in (5.1) are informative for $\psi(\lambda_{0})$ iff
$\nabla m_{\psi}\notin\mathcal{N}(M_{\psi_{0}^{\perp}}^{\ast})^{\perp}.$

An important insight of this paper is that it is not necessary to compute
$\tilde{s}_{\theta}$ to check for the relevance condition. This is
particularly relevant for cases where computing $\tilde{s}_{\theta}$ is
complicated (as in most of the examples considered in the paper).

Example 1: Partly Linear Model with Endogeneity, cont. For
$\psi(\lambda)=\theta,$ corresponding to $r_{\psi}=(1,0),$ we have

$\displaystyle\nabla m_{\psi}$ $\displaystyle=\left(\nabla m\right)[r_{\psi}]$
$\displaystyle=\frac{d}{d\tau}m(W,\theta_{0}+\tau,\eta_{0})$
$\displaystyle=\mathbb{E}\left[\left.-Y_{2}\right|W\right].$

Therefore, for orthogonal moments to be informative $\nabla
m_{\psi}\notin\mathcal{N}(M_{\psi_{0}^{\perp}}^{\ast})^{\perp}=L_{2}(X)$, or
equivalently

$\mathbb{E}\left[\left.-Y_{2}\right|W\right]\notin L_{2}(X),$

which is precisely the conditional relevance condition. The complicated
expression for $\tilde{s}_{\theta}$ is given in, e.g., Chernozhukov et al.
(2018), and involves the inverse of conditional variances. $\square$

###### Remark 5.1

The results of this section are extended to general orthogonality restrictions
as follows. Suppose the model is defined by

$\mathcal{P}=\\{P:\mathbb{E}_{P}\left[\rho_{j}(Y,X,\theta_{0},\eta_{0})\varphi_{j}\left(W_{j}\right)\right]=0\text{
for all }\varphi_{j}\in\Gamma_{j}\subseteq L_{2}(W),\text{ all }j=1,...,J\\},$

where $\theta_{0}\in\Theta,$ $\eta_{0}\in\Xi,$ and $\Gamma_{j}$ is a closed
linear subspace of $L_{2}(W_{j}).$ Then, all the previous results on existence
and relevance hold with $\nabla m_{j}$ replaced by
$\Pi_{\Gamma_{j}}\circ\nabla m_{j}$ and $\left(\nabla m_{j}\right)^{\ast}$
replaced by $\left(\nabla m_{j}\right)^{\ast}\circ\Pi_{\Gamma_{j}}.$ Likewise,
the definition of $\nabla m_{\psi}$ is now $\nabla
m_{\psi}=\Pi_{\Gamma}\circ\left(\nabla m\right)[r_{\psi}],$ where
$\Pi_{\Gamma}$ is applied coordinatewise, i.e. $\Pi_{\Gamma}\circ\left(\nabla
m\right)=(\Pi_{\Gamma_{j}}\circ\left(\nabla m_{j}\right))_{j=1}^{J}.$

As applications of these results, we consider heterogenous parameters in
models of treatment effects, sample selection models, and the popular demand
model for differentiated products in Berry, Levinsohn and Pakes (1995).

###  5.1 Application to Heterogenous Parameters in Treatment Effects

Consider the following generalization of Example 1 to a model with
heterogenous parameters (i.e. interactions),

$\mathbb{E}\left[\left.Y_{1}-\theta_{0}Y_{2}-\eta_{0}\left(Y_{2},X\right)\right|W\right]=0\text{
a.s.,}$ (5.2)

where $W=(X,Z_{2})$, with $X\ $possibly high dimensional, i.e. the dimension
$d_{X}$ of $X$ can be large, much larger than the sample size, and the
function $\eta_{0}(\cdot)$ has the representation

$\eta_{0}\left(Y_{2},X\right)=\eta_{01}+\eta_{02}^{\prime}\left(X-\eta_{03}\right)+{\displaystyle\sum\limits_{l=1}^{d_{X}}}\eta_{04,l}Y_{2}\left(X_{l}-\eta_{03,l}\right).$

The parameters $\eta_{03}$ are the means of $X$, i.e,

$\mathbb{E}\left[X-\eta_{03}\right]=0.$ (5.3)

Models with interactions are commonly used in applied work, particularly when
the interest is in understanding heterogenous (in observables) treatment
effects. This example falls under the setting of the previous section, with
$\rho_{1}(Z,\theta_{0},\eta_{0})=Y_{1}-\theta_{0}Y_{2}-\eta_{0}\left(Y_{2},X\right),$
$W_{1}=W,$ $\rho_{1+l}(Z,\theta_{0},\eta_{0})=X_{l}-\eta_{03,l},$ $W_{1+l}\
$empty, $l=1,...,d_{X},$ and $J=1+d_{X}$. As parameter of interest, we take
$\psi(\lambda_{0})=\eta_{04,l}$, for some $l.$ For instance, we may be
interested in testing if the causal response function depends on the covariate
$X_{l},$ i.e. testing $H_{0}:\eta_{04,l}=0$ vs $H_{1}:\eta_{04,l}\neq 0$.

Our next result characterizes orthogonal moments for
$\psi(\lambda_{0})=\eta_{04,l}$ in this example. Define the random vector

$Q_{l}=(Y_{2},1,X-\eta_{03},Y_{2}\left(X_{-l}-\eta_{03,-l}\right)),$

and its projection onto the exogenous variables

$\xi_{l}\equiv\xi_{l}(W)=\mathbb{E}\left[\left.Q_{l}\right|W\right]=(p_{0}(W),1,X-\eta_{03},p_{0}(W)\left(X_{-l}-\eta_{03,-l}\right)),$

where $p_{0}(W)=\mathbb{E}\left[\left.Y_{2}\right|W\right],$ and $X_{-l}$ and
$\eta_{03,-l}$ denote all coordinates of $X$ and $\eta_{03}$ but the $l-th.$

###### Proposition 5.2

For the model (5.2) and (5.3), orthogonal moments for
$\psi(\lambda_{0})=\eta_{04,l}$ are given by
$g(Z)=\left(Y_{1}-\theta_{0}Y_{2}-\eta_{0}\left(Y_{2},X\right)\right)\varphi(W),$
where orthogonal instruments are given by

$\varphi(W)=\zeta(W)-\Pi_{\xi_{l}}\zeta(W),$ (5.4)

for some $\zeta\in L_{2}(W).$ The orthogonal moment will be informative about
$\psi(\lambda_{0})=\eta_{04,l}$ if

$\mathbb{E}\left[Y_{2}\left(X_{l}-\eta_{03,l}\right)\varphi(W)\right]\neq 0.$
(5.5)

This proposition implies that for the heterogenous parameter
$\psi(\lambda_{0})=\eta_{04,l},$ relevance can be achieved with just one
instrument, provided (5.5) holds. This relevance condition means that the
partial correlation between $Y_{2}\left(X_{l}-\eta_{03,l}\right)$ and $\zeta$,
after removing the effect of $\xi_{l},$ must be non-zero. When this condition
holds, we can identify $\eta_{04,l}$ from the orthogonal moment as

$\displaystyle 0$
$\displaystyle=\mathbb{E}\left[\left(Y_{1}-\theta_{0}Y_{2}-\eta_{0}\left(Y_{2},X\right)\right)\varphi(W)\right]$
$\displaystyle=\mathbb{E}\left[\left(Y_{1}-\gamma_{l}^{\prime}Q_{l}-\eta_{04,l}Y_{2}\left(X_{l}-\eta_{03,l}\right)\right)\varphi(W)\right],$

where
$\gamma_{l}=(\theta_{0},\eta_{01},\eta_{02}^{\prime},\eta_{04,-l})^{\prime}.$
To implement a LR moment satisfying the minimal relevance condition under
(5.5), we recommend $\varphi(W)=\varphi^{\ast}(W),$ where
$\zeta_{l}^{\ast}(W)=p_{0}(W)\left(X_{l}-\eta_{03,l}\right)$ and

$\varphi^{\ast}(W)=\zeta^{\ast}(W)-\Pi_{\xi_{l}}\zeta^{\ast}(W).$

For this choice, under the minimal relevance condition

$\eta_{04,l}=\frac{\mathbb{E}\left[\left(Y_{1}-\gamma_{l}^{\prime}Q_{l}\right)\varphi^{\ast}(W)\right]}{\mathbb{E}\left[Y_{2}\left(X_{l}-\eta_{03,l}\right)\varphi^{\ast}(W)\right]},$

To implement inference in this example (testing
$H_{0}:\eta_{04,l}=\bar{\eta}_{4,l}$ vs
$H_{1}:\eta_{04,l}\neq\bar{\eta}_{4,l})$ based on orthogonal moments, we can
use the following algorithm (we assume that prior to apply this algorithm we
have centered $X$):

  * •

Step 1: Run Lasso, Logit Lasso, Random Forest, or any other machine learning
method for prediction of $p_{0}(W),$ denoted as $\hat{p}.$ Compute
$\hat{\xi}_{l}(W)=(1,X,\hat{p},\hat{p}X_{-l})$ and
$\hat{\zeta}_{l}^{\ast}(W)=\hat{p}X_{l}.$ In the exogenous case where
$Y_{2}=Z_{2},$ this step is not needed and $\hat{\xi}_{l}(W)=Q_{l}$ and
$\hat{\zeta}_{l}^{\ast}(W)=Y_{2}X_{l}.$

  * •

Step 2: Run, e.g., Lasso for estimating $\gamma_{l}$ as the coefficient of
$\hat{\xi}_{l},$ say $\hat{\gamma}_{l},$ in the projection of
$Y_{1}-\bar{\eta}_{4,l}Y_{2}X_{l}$ on $\hat{\xi}_{l}.$ Compute
$\hat{Y}_{1}^{\ast}=Y_{1}-\hat{\gamma}_{l}^{\prime}Q_{l}.$

  * •

Step 3: Run, e.g., Lasso for estimating $\varphi^{\ast}(W)\ $in (5.4) based on
$\hat{\xi}_{l}(W)$ and $\hat{\zeta}_{l}^{\ast}(W)$, to obtain
$\hat{\varphi}^{\ast}\equiv\hat{\varphi}^{\ast}(W).$

  * •

Step 4: Base inference on the sample analog of the orthogonal moment

$\mathbb{E}\left[\left(Y_{1}^{\ast}-\bar{\eta}_{4,l}Y_{2}\left(X_{l}-\eta_{03,l}\right)\right)\varphi^{\ast}(W)\right]=0,$

where $Y_{1}^{\ast}=Y_{1}-\gamma_{l}^{\prime}Q_{l}$ is estimated by
$\hat{Y}_{1}^{\ast}$ and $\varphi^{\ast}$ by $\hat{\varphi}^{\ast}.$

If the goal is estimation of $\eta_{04,l}$, we can use the following variation
of the previous algorithm, where Steps 1 and 3 remain the same, but Steps 2
and 4 change to:

  * •

Step 2: Run, e.g., Lasso for estimating $\gamma_{l}$ as the coefficient of
$\hat{\xi}_{l},$ say $\hat{\gamma}_{l},$ in the projection of $Y_{1}$ on
$\hat{\xi}_{l}$ and $\hat{\zeta}_{l}^{\ast}.$ Compute
$\hat{Y}_{1}^{\ast}=Y_{1}-\hat{\gamma}_{l}^{\prime}Q_{l}.$

  * •

Step 4: Run an IV regression $\hat{Y}_{1}^{\ast}$ on $Y_{2}X_{l}$ with IV
$\hat{\varphi}^{\ast}$ to estimate $\eta_{04,l}$.

For high dimensional settings we also recommend using cross-fitting in both
algorithms, as in Chernozhukov et al. (2018). Asymptotic distribution theory
for these procedures can be obtained by a routine application of CEINR.

A generalization of this setting is the conditional moment model

$\mathbb{E}\left[\left.Y_{1}-\lambda_{01}(X)-\lambda_{02}\left(X\right)Y_{2}\right|W\right]=0\text{
a.s.,}$

where $\lambda_{0}=(\lambda_{01},\lambda_{02})$ have tangents
$d\lambda_{\tau}/d\tau=h\in\mathbf{H}=\Gamma_{1}\times\Gamma_{2},$
$\Gamma_{j}$ is a closed subspace of $L_{2}(X),$ $j=1,2.$ Assume a general
parameter of interest $\psi(\lambda_{0})$ such that for $r_{2}\in\Gamma_{2},$

$\dot{\psi}(h)=\mathbb{E}\left[h_{2}\left(X\right)r_{2}\left(X\right)\right],\text{
all }h_{2}\in\Gamma_{2}.$

Applying our results, we find that sufficient conditions for orthogonal
instruments $\varphi$ are

$\Pi_{\Gamma_{1}}\varphi=0\text{ and }\Pi_{\Gamma_{2}}\varphi p_{0}=Cr_{2}.$
(5.6)

The previous application with interactions corresponds to
$\Gamma_{1}=\Gamma_{2}=\\{\gamma_{0}+\gamma_{1}^{\prime}X:\gamma_{0}\in\mathbb{R},\gamma_{1}\in\mathbb{R}^{d_{x}}\\},$
and $r_{2}(X)=\tilde{X}_{l},$ where $\tilde{X}_{l}$ is the orthogonal
projection of $X_{l}$ onto the orthocomplement of $(1,X_{-l}).$ Our previous
choice $\varphi^{\ast}$ satisfies (5.6) for this case. As another
illustration, if $\Gamma_{1}=\Gamma_{2}=L_{2}(X),$ then (5.6) boils down to
finding $\zeta\in L_{2}(W)$ such that

$r_{2}(X)=\mathbb{E}\left[\left.\Delta(W)\zeta(W)\right|X\right],$ (5.7)

and then construct
$\varphi(W)=\zeta(W)-\mathbb{E}\left[\left.\zeta(W)\right|X\right],$ where
$\Delta(W)=p_{0}(W)-q_{0}(X)$ and
$q_{0}(X)=\mathbb{E}\left[\left.Y_{2}\right|X\right].$ An example of such
$\zeta(W)$ is

$\zeta(W)=\Delta(W)\left(\mathbb{E}\left[\left.\Delta^{2}(W)\right|X\right]\right)^{-1}r_{2}(X),$

though this requires the identification condition
$\mathbb{E}\left[\left.\Delta^{2}(W)\right|X=x\right]\neq 0$ for a.s. $x,$ and
$\zeta\in L_{2}(W).$ The corresponding orthogonal instrument is

$\varphi(W)=\left(\Delta(W)\left(\mathbb{E}\left[\left.\Delta^{2}(W)\right|X\right]\right)^{-1}-1\right)r_{2}(X).$

Other choices of $\zeta$ satisfying (5.7) may require weaker identifying
assumptions. This method can applied to many functionals that summarize
treatment effect heterogeneity, including average derivatives, policy effects,
group effects, etc.

Our approach contributes to the existing literature on inference in
heterogenous treatment effects. A similar specification to the simplest
version of our model, when we specifically model how exogenous regressors and
their interaction with the treatment variable affect the outcome, can be found
in Tian et al. (2014), although they only consider pure randomized
experiments. Wager and Athey (2018) and Athey et al. (2019) propose Random
Forest and generalized versions to estimate function-valued heterogenous
parameters, such as $\lambda_{02}\left(x\right)$ above, for both selection on
observables and IV settings. See also Künzel et al. (2019) for further
discussion on the fully heterogenous case.

There are also papers proposing inference on functionals of
$\lambda_{02}\left(x\right).$ Nekipelov, Semenova and Syrgkanis (2022)
considered a binary choice model with interactions and instrumental variables.
Qiu et al. (2021) works without assuming selection on observables, and in
particular, they consider a setting where the treatment $Y_{2}$ and the
instrument $Z_{2}$ are binary. We do not impose any restriction on the support
of the treatment variable or the instrument (i.e. these can take on a finite
number of values or even be continuous) and also we allow for multiple
instruments. Given that they model the treatment effect on the subpopulation
of compliers (allowing for many regressors), the program they need to solve,
which is a Lasso-type problem, is non-convex. This makes the inference
procedure a more difficult task than in our setting, where the estimation and
inference can be accomplished through an IV procedure with generated variables
from machine learning fits. Syrgkanis et al. (2019) is the closest work to
ours. Their setting is similar to our generalization, and they aim at
estimating what they call the Conditional Average Treatment Effect (CATE) (our
$\lambda_{02}\left(X\right)),$ although they also estimate projections of this
quantity. The procedure of Syrgkanis et al. (2019) requires dealing with two
different estimators of the CATE in order to obtain fully orthogonal moments.
In our case, this is accomplished directly by using the orthogonal moments
that we characterize.101010We base inference on the orthogonal moment
$\mathbb{E}\left[\left(Y_{1}-\gamma_{l}^{\prime}Q_{l}-\eta_{04,l}Y_{2}\left(X_{l}-\eta_{03,l}\right)\right)\varphi(W)\right]=0$
rather than the non-orthogonal moment
$\mathbb{E}\left[\left(Y_{1}-\gamma_{l}^{\prime}\xi_{l}-\eta_{04,l}Y_{2}\left(X_{l}-\eta_{03,l}\right)\right)\varphi(W)\right]=0$.
The latter fails to be orthogonal to estimates of $\xi_{l},$ which is the
problem that Syrgkanis et al. (2019) faces. Importantly, we derive conditions
under which the orthogonal moments that we obtain are informative with respect
to heterogenous parameter of interest, which also seems to be a novelty in the
literature of heterogeneous treatment.

###  5.2 Application to Sample Selection Models

Consider the following high dimensional specification for a sample selection
model, where a data observation $Z=(Y_{1},Y_{2},X)$ satisfies
$Y_{1}=Y^{\ast}Y_{2}$ with

$Y^{\ast}=X^{\prime}\theta_{02}+U,\text{ }Y_{2}=1(X^{\prime}\theta_{01}-V\geq
0),$

such that the following CMR hold

$\mathbb{E}\left[\left.Y_{2}-\eta_{01}(X^{\prime}\theta_{01})\right|X\right]=0$

and

$\mathbb{E}\left[\left.Y_{1}-\left(X^{\prime}\theta_{02}+\eta_{02}(X^{\prime}\theta_{01})\right)Y_{2}\right|Y_{2},X\right]=0.$

This setting also falls under the general CMR model, with
$\rho_{1}(Z,\theta_{0},\eta_{0})=Y_{2}-\eta_{01}(X^{\prime}\theta_{01}),$
$W_{1}=X,$
$\rho_{2}(Z,\theta_{0},\eta_{0})=Y_{1}-\left(X^{\prime}\theta_{02}+\eta_{02}(X^{\prime}\theta_{01})\right)Y_{2}$,
and $W_{2}=(Y_{2},X).$ Here, $\theta_{0}=(\theta_{01},\theta_{02})$ and
$\eta_{0}=(\eta_{01},\eta_{02}).$ This is the well known sample selection
model, see, e.g. Heckman (1979). The information in semiparametric versions of
this model was studied in detail by Chamberlain (1986), though orthogonal
moments for $\theta_{0}$ have not been characterized, to the best of our
knowledge. This is done in the following result.

###### Proposition 5.3

For the Sample Selection Model above, orthogonal moments for
$\psi(\lambda_{0})=\theta_{0}$ are given by

$g(Z)=\left(Y_{2}-\eta_{01}(X^{\prime}\theta_{01})\right)\varphi_{1}(X)+\left(Y_{1}-\left(X^{\prime}\theta_{02}+\eta_{02}(X^{\prime}\theta_{01})\right)\right)Y_{2}\varphi_{2}(X),$

where

$\varphi_{j}(X)=\zeta_{j}(X)-\mathbb{E}\left[\left.\zeta_{j}(X)\right|X^{\prime}\theta_{01}\right],$
(5.8)

and $\zeta_{j}\in L_{2}(X).$ The orthogonal moment will be informative about
$\psi(\lambda_{0})=\theta_{02}$ if

$\mathbb{E}\left[XY_{2}\varphi_{2}(X)\right]\neq 0.$

For relevance, the main factor is the rank of the matrix

$\Sigma=\mathbb{E}\left[\tilde{X}\tilde{X}^{\prime}\eta_{01}(X^{\prime}\theta_{01})\right],$

where
$\tilde{X}=X-\mathbb{E}\left[\left.X\right|X^{\prime}\theta_{01}\right].$ The
matrix $\Sigma$ is clearly of reduced rank, as $\Sigma\theta_{01}=0$ (since
$\tilde{X}^{\prime}\theta_{01}=0$ a.s.). If $\Sigma\neq 0$, then we can
construct orthogonal moments for $\theta_{0}$ that are relevant for inference
for $\theta_{02}.$ These are based on
$\zeta_{2}(X)=X\eta_{01}(X^{\prime}\theta_{01}).$ Implementations of
estimators and inference for sample selection models based on the orthogonal
moments that we derive here will be developed in a companion paper. In Section
S.2 of the Supplementary Appendix we discuss further this example, when
inference is on $\theta_{02}.$ An insight here is that existence of LR moments
does not require exclusion restrictions (i.e. that some component of
$\theta_{02}$ is zero).

###  5.3 Application to BLP

To describe the model, let $t\in\mathcal{T}$ be a market. Denote the set of
available products in market $t$ by
$\mathcal{J}_{t}=\left\\{1,2,\cdots,J_{t}\right\\}$.111111We will use the
terms products and goods interchangeably. Following the usual convention in
the literature, $j=0$ represents the outside option, for all
$t\in\mathcal{T}$. Let $X_{jt}\in\mathbb{R}^{d_{X}}$ and
$Y_{jt}\in\mathbb{R}^{d_{Y}}$ be observed characteristics of product $j$ in
market $t$, e.g. price. The indirect utility of individual $i$ from purchasing
product $j$ in market $t$ is as follows:

$\begin{split}U_{ijt}&=X_{jt}^{\prime}\theta_{0}+Y_{jt}^{\prime}\alpha_{i}+\xi_{jt}+\varepsilon_{ijt}\\\
&=\mu_{jt}+Y_{jt}^{\prime}\alpha_{i}+\varepsilon_{ijt},\end{split}$

where $\mu_{jt}=X_{jt}^{\prime}\theta_{0}+\xi_{jt}$ is the mean utility
associated with product $j$ in market $t$. The indirect utility of the outside
option is normalized to be $U_{i0t}=\varepsilon_{i0t}$. Another common
assumption is that $\varepsilon_{i0t}$ and
$\left(\varepsilon_{i0t}\right)_{j\in\mathcal{J}_{t}}$ are i.i.d. with type-I
extreme value distribution. Moreover, $\alpha$ is distributed with a density
$\eta_{0}$ (w.r.t. a $\sigma-$finite measure $\nu$), and independent of
$\left(X_{jt},Y_{jt},\xi_{jt}\right)_{j\in\mathcal{J}_{t}}$ and
$\left(\varepsilon_{ijt}\right)_{j\in\mathcal{J}_{t}\cup\left\\{0\right\\}}$.
In addition, without loss of generality, assume that
$\mathcal{J}=\mathcal{J}_{t}$, for all $t\in\mathcal{T}$.121212Our derivation
is made given a market $t$. The previous assumptions imply that the market
share for good $j$ in market $t$ is a mixed logit of the form

$\begin{split}s_{jt}&=\sigma_{j}\left(\mu_{t},Y_{t},\eta_{0}\right)\\\
&=\int\frac{\exp\left(\mu_{jt}+Y_{jt}^{\prime}\alpha\right)}{1+\sum_{k=1}^{J}\exp\left(\mu_{kt}+Y_{kt}^{\prime}\alpha\right)}\eta_{0}(\alpha)d\nu(\alpha),\end{split}$

where $\mu_{t}=\left(\mu_{jt}\right)_{j\in\mathcal{J}}$,
$Y_{t}=\left(Y_{jt}\right)_{j\in\mathcal{J}}\in\mathbb{R}^{d_{Y}\times J}$,
$X_{t}=\left(X_{jt}\right)_{j\in\mathcal{J}}\in\mathbb{R}^{d_{X}\times J}$,
$s_{t}=\left(s_{jt}\right)_{j\in\mathcal{J}}$. The function $\sigma_{j}$ is
the market share for product $j$, a fundamental object in these models. This
mixed logit model is invertible (see Berry and Haile, 2014). This implies that
we can use $s_{jt}=\sigma_{j}\left(\mu_{t},Y_{t},\eta_{0}\right)$, to obtain

$\mu_{jt}=\sigma_{j}^{-1}\left(s_{t},Y_{t},\eta_{0}\right)=X_{jt}^{\prime}\theta_{0}+\xi_{jt}.$
(5.9)

####  5.3.1 Moment Restrictions

Often, it is assumed that the econometrician observes a vector of instruments
$W_{jt}\in\mathbb{R}^{d_{W_{j}}}$, $j=1,2,\cdots,J$ and
$t\in\mathcal{T}$.131313This is essential for identification, see Berry and
Haile (2014) and the discussion in Berry and Haile (2016). Assume further that
the following exogeneity condition holds a.s. for each $j$ and market $t$

$\mathbb{E}\left[\xi_{jt}|W_{jt}\right]=0.$ (5.10)

Combine (5.9) and (5.10) to get the following moment restrictions:

$\mathbb{E}\left[\sigma_{j}^{-1}\left(s_{t},Y_{t},\eta_{0}\right)-X_{jt}^{\prime}\theta_{0}|W_{jt}\right]=0.$

This example fits our previous setting with
$\rho_{j}(Z_{t},\theta_{0},\eta_{0})=\sigma_{j}^{-1}\left(s_{t},Y_{t},\eta_{0}\right)-X_{jt}^{\prime}\theta_{0},$
where $Z_{t}=\left(s_{t},Y_{t},X_{t},W_{t}\right)$. For simplicity, we will
suppress the dependence of the variables on the market $t$.

To simplify notation, denote, for $j=1,2,\cdots,J,$

$\displaystyle p_{j}(\alpha,\theta)$
$\displaystyle:=\frac{\exp\left(\mu_{j}(\theta)+Y_{j}^{\prime}\alpha\right)}{1+\sum_{k=1}^{J}\exp\left(\mu_{k}(\theta)+Y_{k}^{\prime}\alpha\right)},$
$\displaystyle\sigma_{j}(\theta,\eta)$
$\displaystyle:=\sigma_{j}(\mu(\theta),Y,\eta),$
$\displaystyle\mu_{j}(\theta)$ $\displaystyle:=X_{j}^{\prime}\theta+\xi_{j},$
$\displaystyle\sigma_{j}^{-1}(\theta,\eta)$
$\displaystyle:=\sigma_{j}^{-1}(s_{j}(\theta,\eta),Y,\eta),$ $\displaystyle
s_{j}(\theta,\eta)$ $\displaystyle:=\sigma_{j}(\theta,\eta).$

####  5.3.2 Orthogonal moments

As an illustration, we characterize orthogonal moments for two parameters: the
fixed parameter $\psi_{1}(\lambda)=\theta\ $and a moment of the random
coefficients of UH,
$\psi_{2}(\lambda)=\mathbb{E}_{\eta_{0}}\left[r(\alpha)\right].$ Other
applications would include average derivatives such as

$\psi_{3}(\lambda)=\mathbb{E}_{\lambda}\left[\frac{\partial\sigma_{j}^{-1}(\theta,\eta)}{\partial
Y_{k}}\right],\text{ }j,k\in\\{1,...,J\\}.$

The automatic estimation of orthogonal moments for these and other
applications will be carried out in a companion paper, based on the
characterizations derived here.

All orthogonal moments have the representation

$\displaystyle g(Z,\theta_{0},\eta_{0})$
$\displaystyle={\textstyle\sum\nolimits_{j=1}^{J}}\rho_{j}(Z,\theta_{0},\eta_{0})\varphi_{j}(W_{j})$
$\displaystyle\equiv\rho^{\prime}(Z,\theta_{0},\eta_{0})\varphi(W)$

for a suitable “orthogonal instrument” $\varphi=(\varphi_{j})_{j=1}^{J}$
satisfying $(\nabla m)^{\ast}\varphi=C^{\prime}r_{\psi}.$ Then, for each
parameter we characterize what this last equation means. We start
characterizing the score operators and their adjoints in the next result.
Define

$Q_{j}(W_{j},\alpha,\theta,\eta)=\mathbb{E}\left[{\textstyle\sum\nolimits_{l=1}^{J}}\frac{\partial\sigma_{j}^{-1}(\theta,\eta)}{\partial
s_{l}}\frac{\partial
p_{l}(\alpha,\theta)}{\partial\theta}-X_{j}|W_{j},\alpha\right]$

and

$M_{j}(W_{j},\alpha,\theta,\eta)=\mathbb{E}\left[{\textstyle\sum\nolimits_{l=1}^{J}}\frac{\partial\sigma_{j}^{-1}(\theta,\eta)}{\partial
s_{l}}p_{l}(\alpha,\theta)|W_{j},\alpha\right].$

###### Proposition 5.4

For the BLP model $\nabla
m(W,\theta,\eta)[h]=\nabla_{\theta}m(W,\theta,\eta)[\delta]+\nabla_{\eta}m(W,\theta,\eta)[b],$
where

$\nabla_{\theta}m_{j}(W_{j},\theta,\eta)[\delta]=\int\delta^{\prime}Q_{j}(W_{j},\alpha,\theta,\eta)\eta_{0}(\alpha)d\nu(\alpha),\text{
}\delta\in\mathbb{R}^{p},$ $\nabla_{\eta}m_{j}(W_{j},\theta,\eta)[b]=\int
M_{j}(W_{j},\alpha,\theta,\eta)b(\alpha)\eta_{0}(\alpha)d\nu(\alpha),\text{
}b\in L_{2}(\eta_{0}).$

The adjoint $(\nabla
m)^{\ast}[\varphi]=\left(\left(\nabla_{\eta}m\right)^{\ast}[\varphi],\left(\nabla_{\theta}m\right)^{\ast}[\varphi]\right)$
is given by

$\displaystyle\left(\nabla_{\theta}m\right)^{\ast}[\varphi]$
$\displaystyle=\mathbb{E}\left[Q^{\prime}(W,\alpha,\theta,\eta)\varphi(W)\right],$
$\displaystyle\left(\nabla_{\eta}m\right)^{\ast}[\varphi]$
$\displaystyle=\mathbb{E}\left[M^{\prime}(W,\alpha,\theta,\eta)\varphi(W)|\alpha\right],$

where $Q=(Q_{1},...,Q_{J})^{\prime}$ and $M=(M_{1},...,M_{J})^{\prime}.$

##### Orthogonal moments for fixed parameters

For $\psi_{1}(\lambda)=\theta,$ from Proposition 5.4, $(\nabla
m)^{\ast}\varphi=C^{\prime}r_{\psi}$ boils down to

$\left(\nabla_{\eta}m\right)^{\ast}[\varphi]=\mathbb{E}\left[M^{\prime}(W,\alpha,\theta_{0},\eta_{0})\varphi(W)|\alpha\right]=0,\text{
}\eta_{0}-\text{a.s}.$ (5.11)

Therefore, orthogonal moments are
$g(Z,\theta_{0},\eta_{0})\equiv\rho^{\prime}(Z,\theta_{0},\eta_{0})\varphi(W),$
where $\varphi(W)$ satisfies (5.11).

Also, in this example

$\nabla m_{\psi}=\left(\nabla m\right)[r_{\psi}]=\int
Q(W,\alpha,\theta,\eta)\eta_{0}(\alpha)d\nu(\alpha).$

Then, orthogonal moments are informative for $\psi_{1}(\lambda)=\theta$ when
$\nabla m_{\psi}$ is not orthogonal to such $\varphi(W)$ solving (5.11).

##### Orthogonal moments for moments of UH

For $\psi_{2}(\lambda)=\mathbb{E}_{\eta_{0}}\left[r(\alpha)\right],$ the Riesz
presenter $r_{\psi}$ is $(0,r(\alpha)-\psi_{2}(\lambda_{0})),$ and from
Proposition 5.4, $(\nabla m)^{\ast}\varphi=C^{\prime}r_{\psi}$ boils down to

$\mathbb{E}\left[Q(W,\alpha,\theta_{0},\eta_{0})^{\prime}\varphi(W)\right]=0$
(5.12)

and

$\mathbb{E}\left[M^{\prime}(W,\alpha,\theta_{0},\eta_{0})\varphi(W)|\alpha\right]=C(r(\alpha)-\psi_{2}(\lambda_{0})),\text{
}\eta_{0}-\text{a.s}.$ (5.13)

Thus, for $\psi_{2}(\lambda)=\mathbb{E}_{\eta_{0}}\left[r(\alpha)\right],$
orthogonal moments are
$g(Z,\theta_{0},\eta_{0})\equiv\rho^{\prime}(Z,\theta_{0},\eta_{0})\varphi(W),$
where $\varphi(W)$ satisfies (5.12) and (5.13).

In this example

$\nabla m_{\psi}=\left(\nabla m\right)[r_{\psi}]=\int
M(W,\alpha,\theta,\eta)\left(r(\alpha)-\psi_{2}(\lambda_{0})\right)\eta_{0}(\alpha)d\nu(\alpha).$

Then, orthogonal moments are informative for
$\psi_{2}(\lambda)=\mathbb{E}_{\eta_{0}}\left[r(\alpha)\right]$ when $\nabla
m_{\psi}$ is not orthogonal to such $\varphi(W)$ solving (5.12) and (5.13).

##  6 Conclusions

In the recent decade, there has been an increasing interest in LR moments, due
to their convenient properties in contexts with machine learning first step
estimators. Yet, the existence problem of such orthogonal moments has not been
studied in detail. This paper contributes by providing a necessary and
sufficient condition for existence of such moments, denoted as Restricted
Local Non-Surjectivity, to conduct inference on structural parameters in
general regular semiparametric models. Importantly, RLN does not require
identification of the parameters of interest or identification of the nuisance
parameters. Our work delivers a positive result: existence of LR moments
follows under quite weak general conditions. For instance, in models with
conditional moment restrictions, these requirements are extensions of commonly
used exclusion restrictions. In models with UH, under support conditions,
orthogonal moments can be constructed through functional differencing. In BLP-
like models, these are combinations of both.

The wide applicability of our results is illustrated with various applications
to models with endogenous variables, treatment effect heterogeneity, sample
selection models, nonlinear panel data models, random coefficient models,
models for demand with differentiated products, and measurement error models.
Nevertheless, we stress that our characterization on the existence of
orthogonal moments holds essentially for any regular
semiparametric/nonparametric model and any smooth parameter.

In our analysis, we have considered the existence and the power of orthogonal
moments as two different issues, and we have characterized both. In a
companion paper, we will propose automatic methods to construct (efficient)
orthogonal moments to conduct reliable inference for multiple functionals of
interests in relevant applications.

##  7 Appendix: Proofs of Main Results

Proof of Theorem 2.4: Suppose RLN holds, so there exists $0\neq
g\in\overline{T_{0}}^{\perp}\cap\mathcal{G}_{0}.$ By (2.9),

$\frac{d}{d\tau}\mathbb{E}[g(Z,\theta_{0},\eta_{\tau})]=0,$ (7.1)

and hence $g$ is a LR moment. Suppose now that there exists a LR moment $g,$
then again by (2.9),

$\mathbb{E}[g(Z,\theta_{0},\eta_{0})s_{\eta}(Z)]=0,$

and therefore $0\neq g\in\overline{T_{0}}^{\perp}\cap\mathcal{G}_{0}$.
$\square$

Proof of Proposition 2.1: By RLS and (2.9), $\phi=-m,$ leading to a zero LR
moment in the construction proposed by CEINR for all identifying moment
functions $m$. $\square$

Proof of Theorem 3.1: From (3.2), orthogonality holds iff

$\mathbb{E}[\Pi_{\overline{T(\eta_{0})}}\mathbb{E}\left[\left.g(Z,\bar{\theta},\eta_{0})\right|\alpha\right]b(\alpha)]=0$

for all $b\in\overline{T(\eta_{0})},$ or equivalently

$\Pi_{\overline{T(\eta_{0})}}\mathbb{E}\left[\left.g(Z,\bar{\theta},\eta_{0})\right|\alpha\right]=0\text{
a.s.}$

$\square$

Proof of Proposition 3.1: The result follows from the Identity Theorem for
real-analytic functions, see Corollary 1.2.6 in Krantz and Parks (2002), and
the fact that $f_{z|\alpha}\left(z|\alpha;\theta_{0}\right)$ does not depend
on $\eta_{0}.$ $\square$

Proof of Proposition 3.2: If $T(\eta_{0})$ is dense, orthogonality holds iff

$\mathbb{E}[\mathbb{E}\left[\left.g(Z,\bar{\theta},\eta_{0})\right|\alpha,X\right]b(\alpha,X)]=0$

for all $b,$ or equivalently

$\mathbb{E}\left[\left.g(Z,\bar{\theta},\eta_{0})\right|\alpha,X\right]=0\text{
a.s.}$

$\square$

Proof of Proposition 3.3: Following the same arguments as Proposition 3.1, but
applied on the set of $v_{X}-$probability one where the conditions hold.
$\square$

Proof of Proposition 3.4: If $T(\eta_{0})$ is dense and because scores are
only functions of $\alpha$ by independence, orthogonality holds iff

$\mathbb{E}[\mathbb{E}\left[\left.g(Z,\bar{\theta},\eta_{0})\right|\alpha\right]b(\alpha)]=0$

for all $b,$ or equivalently

$\mathbb{E}\left[\left.g(Z,\bar{\theta},\eta_{0})\right|\alpha\right]=0\text{
a.s.}$

$\square$

Proof of Proposition 4.1: The restricted model corresponds to paths
$\lambda_{\tau}$ for which
$\psi(\lambda_{\tau})=\psi(\lambda_{0})=\bar{\psi},$ and hence $\langle
r_{\psi},h\rangle_{\mathbf{H}}=0.$ For this set of $h^{\prime}s,$
$h=\Pi_{r_{\phi}^{\perp}}h,$ and hence
$S_{\lambda_{0}}h=S_{\lambda_{0}}\circ\Pi_{r_{\phi}^{\perp}}h=S_{\psi_{0}^{\perp}}h.$
The corresponding adjoint operator to $S_{\psi_{0}^{\perp}}$ is

$S_{\psi_{0}^{\perp}}^{\ast}=\Pi_{r_{\phi}^{\perp}}\circ
S_{\lambda_{0}}^{\ast},$

where
$S_{\lambda_{0}}^{\ast}g=\left(\mathbb{E}\left[s_{\theta}(Z)g(Z)\right],S_{\eta}^{\ast}b\right).$
By duality,
$\overline{\mathcal{R}(S_{\psi_{0}^{\perp}})}^{\perp}=\mathcal{N}(S_{\psi_{0}^{\perp}}^{\ast}),$
see Luenberger (1997, p.157).
$\overline{T_{0}}^{\perp}=\mathcal{N}(S_{\psi_{0}^{\perp}}^{\ast})\neq 0$ iff
there exists a $g\neq 0$ such that $\Pi_{r_{\phi}^{\perp}}\circ
S_{\lambda_{0}}^{\ast}g=0.$ This is equivalent to
$S_{\lambda_{0}}^{\ast}g=\sum_{j=1}^{d_{\psi}}C_{j}r_{\psi,j}$ for constants
$C_{j}.$ $\square$

Proof of Proposition 4.2: Orthogonal moments are
$\mathcal{N}(S_{\psi_{0}^{\perp}}^{\ast})$, while the score of the parameter
$\psi(\lambda_{0})$ is $s_{\psi}=S_{\lambda_{0}}r_{\psi}$. To see the latter,
note $h=\Pi_{r_{\phi}}h+\Pi_{r_{\phi}^{\perp}}h,$ where $\Pi_{r_{\phi}}$
denotes the orthogonal projection onto $r_{\phi}.$ A path where only the
parameter of interest changes is one where $\Pi_{r_{\phi}^{\perp}}h=0,$ and
therefore the score $S_{\lambda_{0}}\Pi_{r_{\phi}}h$ is in the linear span of
$s_{\psi}.$ Thus, from the local power calculations orthogonal moments are
informative when $s_{\psi}$ is not orthogonal to the orthogonal moments, i.e.
$s_{\psi}\notin\mathcal{N}(S_{\psi_{0}^{\perp}}^{\ast})^{\perp}.$ $\square$

Proof of Theorem 5.1: Theorem 4.1 in Chen and Santos (2018) yields that the
orthocomplement of the tangent space of the full model is

$\overline{T}^{\perp}=\\{g\in
L_{2}^{0}:g(Z)={\textstyle\sum\nolimits_{j=1}^{J}}\rho_{j}(Z,\lambda_{0})r_{j}(W_{j})=0\text{
for }r\in\overline{\mathcal{R}}^{\perp}(\nabla m)\\}.$

We are not concerned with the full model, but rather the restricted model
where deviations satisfy $\langle r_{\psi},h\rangle_{\mathbf{H}}=0.$ For these
deviations $h=\Pi_{r_{\psi}^{\perp}}h.$ Thus, the result follows by
restricting the domain of $\nabla m$ to directions for which
$h=\Pi_{r_{\psi}^{\perp}}h.$ Next, use duality, see Luenberger (1997, p.157),
and the arguments in the proof of Proposition 4.1. $\square$

Proof of Proposition 5.1: Follows from the same arguments as Proposition 4.2,
but specialized to the current setting. As before, we decompose
$h=\Pi_{r_{\phi}}h+\Pi_{r_{\phi}^{\perp}}h,$ and for a path where only the
parameter of interest changes it holds $\Pi_{r_{\phi}^{\perp}}h=0,$ and
therefore the instrument $\left(\nabla m\right)[\Pi_{r_{\phi}}h]$ is in the
linear span of $\nabla m_{\psi}=\left(\nabla m\right)[r_{\psi}].$ Thus, we
compute for an orthogonal moment
$g(Z,\theta_{\tau},\eta_{0})=\rho^{\prime}(Y,X,\theta_{0},\eta_{0})\varphi(W),$
by iterated expectations

$\displaystyle\frac{d}{d\tau}\mathbb{E}[g(Z,\theta_{\tau},\eta_{\tau})]$
$\displaystyle=-\mathbb{E}[\left(\nabla
m\right)^{\prime}[\Pi_{r_{\phi}}h]\varphi(W)]$ $\displaystyle=0$

if $\nabla m_{\psi}$ is orthogonal to
$\varphi=(\varphi_{j})_{j=1}^{J}\in\mathcal{N}(M_{\psi_{0}^{\perp}}^{\ast}),$
and not zero otherwise. $\square$

Proof of Proposition 5.2: We compute the derivative operators in this example.
Note $\nabla
m(W,\theta,\eta)[h]=\nabla_{\theta}m(W,\theta,\eta)[\delta]+\nabla_{\eta}m(W,\theta,\eta)[b],$
where

$\displaystyle\nabla_{\theta}m_{1}(W,\theta,\eta)[\delta]$
$\displaystyle=-\mathbb{E}\left[Y_{2}|W\right]\delta,\text{
}\delta\in\mathbb{R},$
$\displaystyle\nabla_{\theta}m_{j}(W,\theta,\eta)[\delta]$
$\displaystyle=0,\text{ }j=2,...,J\equiv 1+d_{X},$
$\displaystyle\nabla_{\eta}m_{1}(W_{j},\theta,\eta)[b]$
$\displaystyle=-\left(1,X-\eta_{03},-\eta_{02}-\eta_{04}\mathbb{E}\left[\left.Y_{2}\right|W\right],\mathbb{E}\left[\left.Y_{2}\right|W\right]\left(X-\eta_{03}\right)\right)^{\prime}b,\text{
}b\in\mathbb{R}^{1+3d_{X}},$
$\displaystyle\nabla_{\eta}m_{1+j}(W_{j},\theta,\eta)[b]$
$\displaystyle=\left(0,0,-1,0\right)^{\prime}b,\text{ where the -1 is in the
}1+d_{X}+j\text{ positition, }$

and $j=1,...,d_{X},$ $b\in\mathbb{R}^{1+3d_{X}}.$ Since $W_{j}$ is empty for
$j>1,$ the other moments do not contribute to the orthogonal moments, which
have the expression

$g(Z)=\left(Y_{1}-\theta_{0}Y_{2}-\eta_{0}\left(Y_{2},X\right)\right)\varphi(W).$

Orthogonality means, with $\eta_{\tau}$ such that
$\eta_{4,l,\tau}=\bar{\eta}_{4,l},$

$\displaystyle\frac{d}{d\tau}\mathbb{E}[g(Z,\theta_{\tau},\eta_{\tau})]$
$\displaystyle=\mathbb{E}[\left(\nabla
m\right)^{\prime}[\Pi_{r_{\psi}^{\perp}}h]\varphi(W)]$ $\displaystyle=0,$

where

$\left(\nabla
m\right)[\Pi_{r_{\psi}^{\perp}}h]=-\left(\mathbb{E}\left[Y_{2}|W\right],1,X-\eta_{03},-\eta_{01}-\eta_{02}\mathbb{E}\left[\left.Y_{2}\right|W\right],\mathbb{E}\left[\left.Y_{2}\right|W\right]\left(X_{-l}-\eta_{03,-l}\right)\right)^{\prime}h.$

Note that $\left(\nabla m\right)[\Pi_{r_{\psi}^{\perp}}h]$ is in the linear
span of

$\xi_{l}\equiv\xi_{l}(W)=\mathbb{E}\left[\left.Q_{l}\right|W\right]=(\mathbb{E}\left[\left.Y_{2}\right|W\right],1,X-\eta_{03},\mathbb{E}\left[\left.Y_{2}\right|W\right]\left(X_{-l}-\eta_{03,-l}\right)),$

where

$Q_{l}=(Y_{2},1,X-\eta_{03},Y_{2}\left(X_{-l}-\eta_{03,-l}\right)).$

Thus, orthogonality will hold if $\varphi(W)=\zeta(W)-\Pi_{\xi_{l}}\zeta(W),$
for $\zeta\in L_{2}(W).$ The orthogonal moment will be informative about
$\psi(\lambda_{0})=\eta_{04,l}$ if

$\displaystyle\frac{d}{d\tau}\mathbb{E}[g(Z,\theta_{0},\eta_{\tau})]$
$\displaystyle=\mathbb{E}[\mathbb{E}\left[\left.Y_{2}\right|W\right]\left(X_{l}-\eta_{03,l}\right)\varphi(W)],$
$\displaystyle=\mathbb{E}[Y_{2}\left(X_{l}-\eta_{03,l}\right)\varphi(W)]$
$\displaystyle\neq 0,$

where now $\eta_{\tau}$ is such that only $\eta_{4,l,\tau}\ $varies. $\square$

Proof of Proposition 5.3: We compute the derivative operators in this example.
Since $\psi(\lambda_{0})=\theta_{0},$ $\nabla
m(W,\theta,\eta)[h]=\nabla_{\eta}m(W,\theta,\eta)[b],$ where
$b=(b_{1},b_{2}),$

$\displaystyle\nabla_{\eta}m_{1}(X,\theta,\eta)[b]$
$\displaystyle=-b_{1}(X^{\prime}\theta_{01}),\text{ }b_{1}\in L_{2},$
$\displaystyle\nabla_{\eta}m_{2}(W,\theta,\eta)[b]$
$\displaystyle=-b_{2}(X^{\prime}\theta_{01})Y_{2},\text{ }b_{2}\in L_{2}.$

Orthogonal moments have the expression

$g(Z)=\left(Y_{2}-\eta_{01}(X^{\prime}\theta_{01})\right)\varphi_{1}(X)+\left(Y_{1}-\left(X^{\prime}\theta_{02}+\eta_{02}(X^{\prime}\theta_{01})\right)\right)Y_{2}\varphi_{2}(W),$

where we have used that $Y_{1}=Y_{1}Y_{2}.$ By the expression of
$\nabla_{\eta}m_{1},$
$\varphi_{1}(X)=\zeta_{1}(X)-\mathbb{E}\left[\left.\zeta_{1}(X)\right|X^{\prime}\theta_{01}\right],$
for some $\zeta_{1}\in L_{2}(X).$ Next, we use that
$Y_{2}\varphi_{2}(W)=Y_{2}\varphi_{2}(1,X),$

$\displaystyle\frac{d}{d\tau}\mathbb{E}[g(Z,\theta_{0},\eta_{\tau})]$
$\displaystyle=\mathbb{E}[b_{2}(X^{\prime}\theta_{01})Y_{2}\varphi_{2}(W)]$
$\displaystyle=\mathbb{E}[b_{2}(X^{\prime}\theta_{01})Y_{2}\varphi_{2}(1,X)]$
$\displaystyle=\mathbb{E}[b_{2}(X^{\prime}\theta_{01})\eta_{01}(X^{\prime}\theta_{01})\varphi_{2}(1,X)]$

and hence, it suffices that
$\varphi_{2}(X,1)=\zeta_{2}(X)-\mathbb{E}\left[\left.\zeta_{2}(X)\right|X^{\prime}\theta_{01}\right]$
for some $\zeta_{2}\in L_{2}(X).$ With some abuse of notation we denote
$\varphi_{2}(X,1)$ by $\varphi_{2}(X).$ The orthogonal moment will be
informative about $\psi(\lambda_{0})=\theta_{02}$ if

$\frac{d}{d\tau}\mathbb{E}[g(Z,\theta_{01},\theta_{02,\tau},\eta_{0})]=\mathbb{E}[XY_{2}\varphi_{2}(X)]\neq
0.$

$\square$

Proof of Proposition 5.4: By the chain rule $\nabla
m(W,\theta,\eta)[h]=\nabla_{\theta}m(W,\theta,\eta)[\delta]+\nabla_{\eta}m(W,\theta,\eta)[b].$
Also by the change rule

$\displaystyle\frac{\partial}{\partial\theta}\mathbb{E}\left[\sigma_{j}^{-1}(\theta,\eta)-\theta^{\prime}X_{j}|W_{j}\right]$
$\displaystyle=\mathbb{E}\left[{\textstyle\sum\nolimits_{l=1}^{J}}\frac{\partial\sigma_{j}^{-1}(\theta,\eta)}{\partial
s_{l}}\int\frac{\partial
p_{l}(\alpha,\theta)}{\partial\theta}\eta_{0}(\alpha)d\nu(\alpha)-X_{j}|W_{j}\right]$
$\displaystyle=\int
Q_{j}(W_{j},\alpha,\theta,\eta)\eta_{0}(\alpha)d\nu(\alpha)$

and similarly,

$\displaystyle\frac{\partial}{\partial\tau}\mathbb{E}\left[\sigma_{j}^{-1}(\theta,\eta(1+\tau
b))-\theta^{\prime}X_{j}|W_{j}\right]$
$\displaystyle=\mathbb{E}\left[{\textstyle\sum\nolimits_{l=1}^{J}}\frac{\partial\sigma_{j}^{-1}(\theta,\eta)}{\partial
s_{l}}\int
p_{l}(\alpha,\theta)b(\alpha)\eta_{0}(\alpha)d\nu(\alpha)|W_{j}\right]$
$\displaystyle=\int
M_{j}(W_{j},\alpha,\theta,\eta)b(\alpha)\eta_{0}(\alpha)d\nu(\alpha).$

This yields

$\nabla_{\theta}m_{j}(W_{j},\theta,\eta)[\delta]=\int\delta^{\prime}Q_{j}(W_{j},\alpha,\theta,\eta)\eta_{0}(\alpha)d\nu(\alpha),\text{
}\delta\in\mathbb{R}^{p}$

and

$\nabla_{\eta}m_{j}(W_{j},\theta,\eta)[b]=\int
M_{j}(W_{j},\alpha,\theta,\eta)b(\alpha)\eta_{0}(\alpha)d\nu(\alpha),\text{
}b\in L_{2}(v).$

The adjoint $(\nabla
m)^{\ast}[\varphi]=\left(\left(\nabla_{\eta}m\right)^{\ast}[\varphi],\left(\nabla_{\theta}m\right)^{\ast}[\varphi]\right)$
is computed from

$\displaystyle\mathbb{E}\left[\nabla^{\prime}m(W,\theta,\eta)[h]\varphi(W)\right]$
$\displaystyle={\textstyle\sum\nolimits_{l=1}^{J}}\mathbb{E}\left[\nabla^{\prime}m_{j}(W_{j},\theta,\eta)[h]\varphi_{j}(W_{j})\right]$
$\displaystyle={\textstyle\sum\nolimits_{l=1}^{J}}\mathbb{E}\left[\nabla_{\theta}m_{j}(W_{j},\theta,\eta)[\delta]\varphi_{j}(W_{j})\right]$
$\displaystyle+{\textstyle\sum\nolimits_{l=1}^{J}}\mathbb{E}\left[\nabla_{\eta}m_{j}(W_{j},\theta,\eta)[b]\varphi_{j}(W_{j})\right]$
$\displaystyle=\delta^{\prime}\mathbb{E}\left[Q(W,\alpha,\theta,\eta)\varphi(W)\right]$
$\displaystyle+\int\mathbb{E}\left[M^{\prime}(W,\alpha,\theta,\eta)\varphi(W)|\alpha\right]b(\alpha)\eta_{0}(\alpha)d\nu(\alpha).$

From this equality, it follows that

$\displaystyle\left(\nabla_{\theta}m\right)^{\ast}[\varphi]$
$\displaystyle=\mathbb{E}\left[Q^{\prime}(W,\alpha,\theta,\eta)\varphi(W)\right]$
$\displaystyle\left(\nabla_{\eta}m\right)^{\ast}[\varphi]$
$\displaystyle=\mathbb{E}\left[M^{\prime}(W,\alpha,\theta,\eta)\varphi(W)|\alpha\right].$

$\square$

## References

  * (1)
  * AI07 (2007) Ai, C., and Chen, X. (2007): “Estimation of Possibly Misspecified Semiparametric Conditional Moment Restriction Models with Different Conditioning Variables”, _Journal of Econometrics_ , 141(1), 5-43.
  * AW (2021) Athey, S. and Wager, S. (2021): “Policy Learning with Observational Data”, _Econometrica_ , 89(1), 133-161.
  * ATW (2019) Athey, S., Tibshirani, J., Wager, S., Athey, S., Tibshirani, J. and Wager, S. (2019) : ”Generalized Random Forests”, _The Annals of Statistics_ , 47(2), 1148-1178.
  * Bajari, Hahn, Hong, Ridder (2011) Bajari, P., Hahn, J., Hong, H., and Ridder, G. (2011): “A Note on Semiparametric Estimation of Finite Mixtures of Discrete Choice Models with Application to Game Theoretic Models”, _International Economic Review_ , 52(3), 807-824.
  * Begun, Hall, Huang, and Wellner (1983) Begun, J. M., Hall, W. J., Huang, W. M., and Wellner, J. A. (1983): “Information and Asymptotic Efficiency in Parametric-Nonparametric Models”, _Annals of Statistics_ , 11, 432-452.
  * Belloni, Chen, Chernozhukov, Hansen (2012) Belloni, A., Chen, D., Chernozhukov, V., and Hansen, C. (2012): “Sparse Models and Methods for Optimal Instruments with an Application to Eminent Domain”, _Econometrica_ , 80(6), 2369-2429
  * Belloni, Chernozhukov, Fernandez Val, Hansen (2017) Belloni, A., Chernozhukov, V., Fernández-Val, I., and Hansen, C. (2017): “Program Evaluation and Causal Inference with High-Dimensional Data”, _Econometrica_ , 85(1), 233-298.
  * BH (2014) Berry, S. T., and Haile, P. A. (2014): “Identification in Differentiated Products Markets Using Market Level Data”, _Econometrica_ , 82(5), 1749-1797.
  * BH16 (2016) Berry, S. , and Haile, P. (2016): “Identification in Differentiated Products Markets”, _Annual Review of Economics_ , 8, 27-52.
  * BLP (1995) Berry, S., Levinsohn, J., and Pakes, A. (1995): “Automobile Prices in Market Equilibrium”, _Econometrica_ , 63, 841-890.
  * Bonhomme (2012) Bonhomme, S. (2012): “Functional Differencing”, _Econometrica_ , 80(4), 1337-1385.
  * Bravo, Escanciano, Van Keilegom (2020) Bravo, F., Escanciano, J. C., and Van Keilegom, I. (2020): “Two-Step Semiparametric Empirical Likelihood Inference”, _The Annals of Statistics_ , 48(1), 1-26.
  * B09 (2009) Butucea, C., and Comte, F. (2009): “Adaptive Estimation of Linear Functionals in the Convolution Model and Applications”, arXiv:0902.1443.
  * CE (2021) Caetano, C., and J.C. Escanciano (2021): “Identifying Multiple Marginal Effects with a Single Instrument,”  _Econometric Theory_ , 37(3), 464-494.
  * Carrasco, Florens and Renault (2007) Carrasco, M., J. P. Florens, and E. Renault (2007): “Linear Inverse Problem in Structural Econometrics Estimation Based on Spectral Decomposition and Regularization” , _Handbook of Econometrics_ , 6, 5633-5751.
  * Chamberalin (1986) Chamberlain, G. (1986): “Asymptotic Efficiency in Semi-Parametric Models with Censoring”, _Journal of Econometrics_ , 34, 305-334.
  * Chamberlain (2010) Chamberlain, G. (2010): “Binary Response Models for Panel Data: Identification and Information”, _Econometrica_ , 78(1), 159-168.
  * Chen, Santos (2018) Chen, X. and Santos, A. (2018): “Overidentification in Regular Models”, _Econometrica_ , 86(5), 1771-1817.
  * Chernozhukov, Chetverikov, Demirer, Duflo,Hansen, Newey, Robins (2018a) Chernozhukov, V., Chetverikov, D., Demirer, M., Duflo, E., Hansen, C., Newey, W., and Robins, J. (2018): “Double/Debiased Machine Learning for Treatment and Structural Parameters”, _The Econometrics Journal_ , 21(1), C1-C68.
  * Chernozhukov, Escanciano, Ichimura, Newey (2016) Chernozhukov, V., Escanciano, J. C., Ichimura, H., and Newey, W. K. (2016): “Locally Robust Semiparametric Estimation”., arXiv:1608.00033v1.
  * Chernozhukov, Escanciano, Ichimura, Newey and Robins (2022) Chernozhukov, V., Escanciano, J.C., Ichimura, H., Newey, W.K. and J.M. Robins (2022): “Locally Robust Semiparametric Estimation”, _Econometrica,_ 90, 1501-1535.
  * Choi, Hall, Schick (1996) Choi, S., Hall, W. J., and Schick, A.(1996): “Asymptotically uniformly most powerful tests in parametric and semiparametric models”, _The Annals of Statistics_ , 24(2):841–861.
  * Escanciano (2022) Escanciano, J.C. (2012): “Semiparametric Efficient Tests”, presented at the Mini-Workshop: Frontiers in Quantile Regression, Mathematisches Forschungsinstitut Oberwolfach.
  * Escanciano (2022) Escanciano, J.C. (2022): “Semiparametric Identification and Fisher Information”, _Econometric Theory_ , 38, 301-338.
  * Escanciano (2023) Escanciano, J.C. (2023): “Irregular identification of structural models with nonparametric unobserved heterogeneity”, forthcoming _Journal of Econometrics_.
  * F91 (1991) Fan, J. (1991): “On the Optimal Rates of Convergence for Nonparametric Deconvolution Problems”, _The Annals of Statistics_ , 1257-1272.
  * Farrell (2015) Farrell, M. H. (2015): “Robust Inference on Average Treatment Effects with Possibly More Covariates than Observations”, _Journal of Econometrics_ , 189(1), 1-23.
  * Fox, Kim, Ryan, and Bajari (2012) Fox, J. T., Kim, K. I., Ryan, S. P., and Bajari, P. (2012): “The Random Coefficients Logit Model is Identified”, _Journal of Econometrics_ , 166 (2), 204-212.
  * HM (2007) Hall, P., and Meister, A. (2007): “A Ridge-parameter Approach to Deconvolution”, arXiv:0710.3491.
  * H79 (1979) Heckman, J. J. (1979): “Sample Selection Bias As a Specification Error,”  _Econometrica,_ 47, 153–161.
  * Heckman and Singer (1984a) Heckman, J.J. and Singer, B. (1984): “The Identifiability of the Proportional Hazard Model”, _Review of Economic Studies_ , 51, 231-241.
  * Honore, Weidner (2021) Honoré, B. E. and Weidner, M. (2021): “Moment Conditions for Dynamic Panel Logit Models with Fixed Effects”, _arXiv:2005.05942_.
  * Hu, Schennach (2008) Hu, Y. and SM Schennach (2008): “Instrumental Variable Treatment of Nonclassical Measurement Error Models,”  _Econometrica,_ 76, 195-216.
  * Ishwaran (1999) Ishwaran, H. (1999): “Information in Semiparametric Mixtures of Exponential Families,”  _Annals of Statistics_ , 27, 159-177.
  * Johnson (2004) Johnson, E. G. (2004): “Identification in Discrete Choice Models with Fixed Effects”, _Bureau of Labor Statistics_ , working paper, Citeseer.
  * Kitazawa (2013) Kitazawa, Y. (2013): “Exploration of Dynamic Fixed Effects Logit Models From a Traditional Angle”, _Kyushu Sangyo University, Faculty of Economics_ , discussion papers 60.
  * Kitazawa2 (2016) Kitazawa, Y. (2016): “Root-N Consistent Estimations of Time Dummies For The Dynamic Fixed Effects Logit Models: Monte Carlo Illustrations”, _Kyushu Sangyo University, Faculty of Economics_ , discussion papers 72.
  * KP (2002) Krantz, S.G. and Parks, H.R. (2002): _A primer of real analytic functions_. Springer Science & Business Media.
  * KSBY (2019) KÜnzel, S.R., Sekhon, J.S., Bickel, P.J. and Yu, B., (2019): “Metalearners for Estimating Heterogeneous Treatment Effects using Machine Learning”, _Proceedings of the National Academy of Sciences_ , 116(10), 4156-4165.
  * Laird78 (1978) Laird, N. (1978):“Nonparametric Maximum Likelihood Estimation of a Mixing Distribution”, Journal of the American Statistical Association, 73(364), 805-811.
  * LC (1988) Le Cam, L. and Yang, G.L. (1988): “On the Preservation of Local Asymptotic Normality Under Information Loss”, The Annals of Statistics, 483-520.
  * Lee (2022) Lee, A. (2022): “Robust and efficient inference for non-regular semiparametric models”, working paper.
  * Luenberger (1997) Luenberger, D. G. (1997): _Optimization by Vector Space Methods_. New York: John Wiley & Sons. 1969 edition.
  * Manski (1987) Manski, C. F. (1987): “Semiparametric Analysis of Random Effects Linear Models from Binary Panel Data”, _Econometrica_ , 55(2), 357-362.
  * Nekipelov, Semenova, Syrgkanis (2022) Nekipelov, D., Semenova, V., and Syrgkanis, V. (2022): “Regularized Orthogonal Machine Learning for Nonlinear Semiparametric Models”, _The Econometrics Journal_ , 25(1), 233-255.
  * Newey (1990) Newey, W. K. (1990): “Semiparametric Efficiency Bounds”,  _Journal of Applied Econometrics_ , 5, 99-135.
  * Newey (1994) Newey, W. K. (1994): “The Asymptotic Variance of Semiparametric Estimators”, _Econometrica_ , 62, 1349-1382.
  * Newey and Powell (2003) Newey, W. K., and J. Powell (2003): “Instrumental Variables Estimation for Nonparametric Models”,  _Econometrica_ , 71, 1565-1578.
  * Neyman (1959) Neyman, J. (1959): “Optimal Asymptotic Tests of Composite Statistical Hypotheses”, _Probability and Statistics: The Harald Cramer Volume_ , 213-34.
  * Qiu (2021) Qiu, Y., Tao, J. and Zhou, X.H. (2021): “Inference of Heterogeneous Treatment Effects using Observational Data with High-dimensional Covariates”, _Journal of the Royal Statistical Society Series B: Statistical Methodology_ , 83(5), 1016-1043.
  * R88 (1988) Robinson, P. M. (1988): “Root-N-consistent Semiparametric Regression”, Econometrica: Journal of the Econometric Society, 931-954.
  * Rothenberg (1971) Rothenberg, R. J. (1971): “Identification in Parametric Models, ”  _Econometrica_ , 39(3), 577-591.
  * Sasaki, Ura (2021) Sasaki, Y. and Ura, T. (2021): “Estimation and Inference for Policy Relevant Treatment Effects”, forthcoming _Journal of Econometrics_.
  * Syrgkanis (2019) Syrgkanis, V., Lei, V., Oprescu, M., Hei, M., Battocchi, K. and Lewis, G., (2019): “Machine Learning Estimation of Heterogeneous Treatment Effects with Instruments”, _Advances in Neural Information Processing Systems_ , 32.
  * Tian (2014) Tian, L., Alizadeh, A.A., Gentles, A.J. and Tibshirani, R., (2014): “A Simple Method for Estimating Interactions between a Treatment and a Large Number of Covariates”, _Journal of the American Statistical Association_ , 109(508), 1517-1532.
  * T06 (2006) Tsiatis, A. A. (2006). _Semiparametric Theory and Missing Data_ , Springer New York, NY.
  * van der Vaart (1998) Van der Vaart, A. W. (1998). _Asymptotic Statistics_ , 3, Cambridge Series in Statistical and Probabilistic Mathematics. Cambridge University Press, Cambridge.
  * Wager (2018) Wager, S. and Athey, S., (2018): “Estimation and Inference of Heterogeneous Treatment Effects using Random Forests”, _Journal of the American Statistical Association_ , 113(523), 1228-1242.

Supplementary Appendix

##  S.1 Further results on local power

Suppose that we want to test:

$H_{0}:\theta_{0}=\bar{\theta}\qquad vs\qquad
H_{1n}:\theta_{0}=\bar{\theta}+\frac{\delta}{\sqrt{n}},$ (S.1.1)

where $\delta\neq 0$ is a $p-$dimensional vector of known constants. Notice
that we are restricting attention to local alternatives, and that
$P\in\mathcal{P}$ is a maintained hypothesis.

Let $\varphi_{n}$ be an arbitrary test of (S.1.1). In particular,
$\varphi_{n}:\left\\{Z_{i}\right\\}_{i=1}^{n}\mapsto\left[0,1\right]$, i.e.
$\varphi_{n}$ is a measurable function of the sample giving the probability of
rejecting the null hypothesis. The null and alternative marginal models are,
respectively,

$\mathcal{P}_{0}=\\{\mathbb{P}_{\bar{\theta},\eta}:\eta\in H\\}\ \text{and
}\mathcal{P}_{1}=\\{\mathbb{P}_{\theta_{0},\eta}:\eta\in
H,\theta_{0}\neq\bar{\theta}\\}.$

The full model is $\mathcal{P}\equiv\mathcal{P}_{0}\cup\mathcal{P}_{1}.$ We
introduce some important definitions. Let
$\mathbb{P}_{1/\sqrt{n},s_{\eta}}^{n}$ denote the product measure of $n$
copies of $\mathbb{P}_{1/\sqrt{n},s_{\eta}},$
$\mathbb{P}_{1/\sqrt{n},s_{\eta}}^{n}={\textstyle\prod\nolimits_{i=1}^{n}}\mathbb{P}_{1/\sqrt{n},s_{\eta}},$
where $\mathbb{P}_{\tau,s_{\eta}}\equiv\mathbb{P}_{\tau}$ is the path defined
in (2.5), and we emphasize the dependence on the score $s_{\eta}.$

###### Definition S.1.1

A test $\varphi_{n}:\left\\{Z_{i}\right\\}_{i=1}^{n}\mapsto\left[0,1\right]$
has asymptotic level $\alpha$, if for any path
$\tau\mapsto\mathbb{P}_{\tau,s_{\eta}}\in\mathcal{P}_{0}$, it follows that:

$\limsup_{n\rightarrow\infty}\int\varphi_{n}\;d\mathbb{P}_{1/\sqrt{n},s_{\eta}}^{n}\leq\alpha.$

###### Definition S.1.2

A test $\varphi_{n}:\left\\{Z_{i}\right\\}_{i=1}^{n}\mapsto\left[0,1\right]$
has an asymptotic local power function $\pi:L_{2}^{0}\mapsto\left[0,1\right]$,
if for any path $\tau\mapsto\mathbb{P}_{\tau,s_{\eta}}\in\mathcal{P}$, it
follows that

$\lim_{n\rightarrow\infty}\int\varphi_{n}\;d\mathbb{P}_{1/\sqrt{n},s_{\eta}}^{n}=\pi(s_{\eta}).$

The test is a locally unbiased asymptotic level $\alpha$ test if
$\pi(s_{\eta})\geq\alpha$ for all
$\tau\mapsto\mathbb{P}_{\tau,s_{\eta}}\in\mathcal{P}_{1}.$

We now investigate in detail the asymptotic local power function of orthogonal
score based tests, generalizing the discussion in the main text to a
$k-$dimensional orthogonal moment. Define the test statistic