# Efficient semi-supervised inference for logistic regression under case-
control studies

Zhuojun Quan
Department of Statistics and Data Science
School of Management
Fudan University
Shanghai, P.R.China
Yuanyuan Lin
Department of Statistics
The Chinese University of Hong Kong
Shatin, New Territories, Hong Kong
Kani Chen
Department of Mathematics
Hong Kong University of Science and Technology
Clear Water Bay, Kowloon, Hong Kong
Wen Yu
Department of Statistics and Data Science
School of Management
Fudan University
Shanghai, P.R.China

###### Abstract

Semi-supervised learning has received increasingly attention in statistics and
machine learning. In semi-supervised learning settings, a labeled data set
with both outcomes and covariates and an unlabeled data set with covariates
only are collected. We consider an inference problem in semi-supervised
settings where the outcome in the labeled data is binary and the labeled data
is collected by case-control sampling. Case-control sampling is an effective
sampling scheme for alleviating imbalance structure in binary data. Under the
logistic model assumption, case-control data can still provide consistent
estimator for the slope parameter of the regression model. However, the
intercept parameter is not identifiable. Consequently, the marginal case
proportion cannot be estimated from case-control data. We find out that with
the availability of the unlabeled data, the intercept parameter can be
identified in semi-supervised learning setting. We construct the likelihood
function of the observed labeled and unlabeled data and obtain the maximum
likelihood estimator via an iterative algorithm. The proposed estimator is
shown to be consistent, asymptotically normal, and semiparametrically
efficient. Extensive simulation studies are conducted to show the finite
sample performance of the proposed method. The results imply that the
unlabeled data not only helps to identify the intercept but also improves the
estimation efficiency of the slope parameter. Meanwhile, the marginal case
proportion can be estimated accurately by the proposed method.

MSC:
Some key words: Biased sampling; Case-control study; Classification; Maximum
likelihood estimator; Semiparametric efficiency; Semi-supervised inference.

##  1 Introduction

Logistic regression (Cox, 1958) is probably the most fundamental statistical
model to fit binary or multi-category outcome. Let $Y$ be the binary response
variable taking values $0,1$ and let $X$ be an observable $p$-dimensional
vector of covariates. A binary logistic regression assumes that

$\mathsf{P}(Y=1\mid X=x)=1-\mathsf{P}(Y=0\mid
X=x)=\frac{\exp(\alpha+\beta^{\top}x)}{1+\exp(\alpha+\beta^{\top}x)},$ (1)

where $\beta\in{\mathbb{R}}^{p}$ is the slope parameter and
$\alpha\in{\mathbb{R}}$ is the intercept. The logistic regression is often
regarded as a special case of the generalized linear models with logit link
function. Model (1) is particularly popular in the analysis of medical and
epidemiological data for the study of the effect of certain exposure to
possible disease or hazards. Usually if disease is present $Y=1$ (cases), else
$Y=0$ (controls). Prospective studies follow a sample of subjects or
individuals and the record corresponding $Y$. For logistic regression model
under prospective studies, the samples are random samples from the underlying
population.

Nonetheless, many diseases are rare in medical studies, and large prospective
study might produce very few disease cases and thus very little information
about the disease of interest. Due to the rareness of some diseases, it is
sometimes impractical to study them by following up of an initially healthy
population, for example the study of smoking and lung cancer. Retrospective
study is an effective tool for the study of the effect of certain exposure to
rare disease incidence, which enjoys important advantages over prospective
study. Case-control study is a special retrospective study, by taking separate
random samples from the case population and the control population, when the
case population and the control population can be clearly separated (Chen and
Lo, 1999). In general, case-control sample is a special case of the so-called
response biased sampling, where the sampling schemes rely on the value of
outcome; see Chen(2001), Manski and Lerman (1977), Cosslett (1981), Scott and
Wild (1986,1997), Fithian and Hastie (2014), and Yao et al. (2017). For
logistic regression under case-control study, a remarkable finding is that,
the prospective estimating equation is valid for a consistent estimate of the
slope parameter $\beta,$ except the intercept term $\alpha$ (Prentice and
Pyke, 1979). In classical logistic regression under case-control study, the
intercept term and the density function of $X$ are not identifiable/estimable,
as the score function of the intercept $\alpha$ lies in the linear space
spanned by the score function of the density function of $X$. The
identifiability/estimability of logistic regression under (multiple) case-
control studies has been carefully studied by Tang et al. (2021).

In recent years, semi-supervised learning has gained significant attentions in
statistics and machine learning. Under semi-supervised setting, there are
typically two sources of data: (i) A relatively small labeled data set
collected according to certain sampling scheme, denoted by $\mathcal{L}$,
containing observations of the outcome variable $Y\in\mathbb{R}$ and the
covariates $X\in\mathbb{R}^{p}$. (ii) A relatively large unlabeled data set,
denoted by $\mathcal{U}$, containing only observations of covariate $X$. Semi-
supervised setting is rather common in practical applications, as the outcome
variable is often more costly or difficult to collect than the predictors
(Chapelle et al., 2009). For example, in analyzing the electronic health
records (EHR) data, labeling of the outcome variable is often very laborious
whereas the covariates are relatively easy to obtain (Gronsbell and Cai, 2018;
Chakrabortty and Cai, 2018). Earlier works on semi-supervised learning focus
on classification problems (Nigam et al., 2000; Ando and Zhang, 2005, 2007;
Belkin et al., 2006; Wang and Shen, 2007; Wang et al., 2008; etc). Semi-
supervised learning in regression framework were reported by Belkin et al.
(2006), Wasserman and Lafferty (2007), Johnson and Zhang (2008), among many
others. A comprehensive review on related works can be found in Zhu and
Goldberg (2009).

How to integrate the labeled and unlabeled data for a joint analysis, that can
produce more accurate and efficient parameter estimation, has become one of
central concerns in semi-supervised learning. This is known as semi-supervised
inference in statistics. Let $\mathbb{P}$ be the joint distribution of
$(Y,X^{\top})^{\top}$ and let $\mathbb{P}_{X}$ be the marginal distribution of
$X$. And let $\theta^{\star}\equiv\theta^{\star}(\mathbb{P})$ be a target
parameter of interest. Generally speaking, whether the unlabeled data can be
used to improve the estimation efficiency of $\theta^{\star}$ rests on whether
$\theta^{\star}$ is explicitly or implicitly related to $\mathbb{P}_{X}$, as
acquiring a large unlabeled data set is approximately equivalent to knowing
$\mathbb{P}_{X}$ (Zhang and Oles, 2000). Recently, important works on semi-
supervised inference without any parametric model assumptions relating $X$ and
$Y$ were reported. Azriel et al. (2016) considered the construction of the
best linear predictor in semi-supervised framework. Semi-supervised inference
of the population mean of a continuous response variable was studied by Zhang
et al. (2019). Chakrabortty and Cai (2018) proposed a class of efficient and
adaptive estimators to improve estimation efficiency in semi-supervised
settings. Efficient and robust semi-supervised estimation of the averaged
treatment effects was studied by Cheng et al. (2018). Efficient evaluation of
model predictive performance with semi-supervised methods was considered by
Gronsbell and Cai (2018). For high dimensional linear models, novel semi-
supervised calibrated estimator for the explained variance was developed by
Cai and Guo (2020).

In this paper, we study an efficient semi-supervised inference procedure for
the logistic regression, where the labeled data is collected by case-control
sampling. With the availability of the unlabeled data, the intercept parameter
becomes identifiable under mild conditions, contrary to classical case-control
logistic regression; more importantly, the resulting estimate of the slope
parameters are proved to be semiparametric efficient. By maximizing the
nonparametric likelihood function of the combined data, the resulting
estimators for the intercept and slope parameters are shown to be consistent,
asymptotically normal and asymptotically efficient. An iterative algorithm is
proposed to compute the maximum likelihood estimators numerically.

The rest of the paper is organized as follows. In Section 2, we introduce some
necessary notation and the structure of the observed data. In Section 3, we
first discuss the identifiability issue of the model parameters. Then the
likelihood function is derived to estimate the parameters and the iterative
algorithm is given out. We also show the large sample properties of the
resulting estimators. The numerical results are presented in Section 4,
including extensive simulation studies and the analysis of the Pima Indians
diabetes data. Section 5 concludes. All the technical details are summarized
in the Appendix.

##  2 Notation and data structure

Using the notation defined in Section 1, we call the class with $Y=1$ case
population and the class with $Y=0$ control population. Define the function

$\displaystyle\phi(x,\theta)=\frac{\exp(\alpha+\beta^{\top}x)}{1+\exp(\alpha+\beta^{\top}x)},$

where $\theta=(\alpha,\beta^{\top})^{\top}$. Then the binary logistic
regression model (1) can be written as $\mathsf{P}(Y=1|X=x)=\phi(x,\theta)$.

The structure of the observed data is as follows. We have a random sample of
size $n_{1}$ from the case population and a random sample of size $n_{0}$ from
the control population. The two samples compose the labeled data of size
$n=n_{1}+n_{0}$. Besides the labeled data, we collect a random sample from
$\mathbb{P}_{X}$, i.e., the marginal distribution of $X$, with size $N-n$,
which is treated as the unlabeled data. We denote the labeled data by ${\cal
D}_{\cal L}=\\{(y_{i},x_{i}),i=1,\ldots,n\\}$ and the unlabeled data by ${\cal
D}_{\cal U}=\\{x_{i},i=n+1,\ldots,N\\}$.

Assume that $\mathbb{P}_{X}$ has a density. Denote the density function and
the cumulative distribution function of $X$ by $f(x)$ and $F(x)$,
respectively. Let $f_{1}(x)$ be the conditional density function of $X$ given
$Y=1$ and $f_{0}(x)$ be the one given $Y=0$. Under the logistic model (1),
using the Bayes formula, it is not difficult to show that

$\displaystyle f_{1}(x)=\frac{\phi(x,\theta)f(x)}{c(\theta,F)},\
f_{0}(x)=\frac{(1-\phi(x,\theta))f(x)}{1-c(\theta,F)},$ (2)

where $c(\theta,F)=\int_{\cal X}\phi(x,\theta)dF(x)$ with ${\cal X}$ being the
sample space of the covariates $X$.

##  3 Main results

###  3.1 Identifiability issue

As we have mentioned, the case-control studies are quite common for analyzing
binary response data to alleviate obvious imbalance between cases and
controls. Essentially, the case-control sampling is a biased sampling scheme.
However, Farewell (1979) argued that if the logistic model (1) holds for $Y$
and $X$, then for the case-control sampling data, the observed conditional
probability of $Y=1$ given $X$ still satisfies the form of (1) with the same
$\beta$ but a different $\alpha$. That is, the slope parameter $\beta$ in (1)
is still identifiable for a single case-control sample. As we have mentioned
in Section 1, Prentice and Pyke (1979) showed that by fitting a usual
prospective logistic model for the case-control data, one can still get a
consistent estimator for $\beta$ and the corresponding inference procedure is
also valid. Contrary to $\beta$, the intercept parameter $\alpha$ is not
identifiable under a single case-control sample. Although $\beta$ is more
important to understand the relationship between $Y$ and $X$, $\alpha$ is also
useful for accurate classification and prediction. To identify $\alpha$, some
extra information outside the case-control sample is needed.

For the semi-supervised setting, we shall always assume the sizes of cases and
controls, i.e., $n_{1}$ and $n_{0}$, and the size of the unlabeled data, i.e.,
$N-n$, tend to infinity. Meanwhile, we assume that the support of $X$ is not
limited to any hyperplane whose dimension is smaller than $p$. Under these
assumptions, $f(x)$ is identifiable as $N-n$ goes to infinity. Knowing from
the classical result of case-control studies, $\beta$ is identifiable as long
as $n_{1}$ and $n_{0}$ tend to infinity. Let $x_{1}$ and $x_{2}$ be two
different values in the support of $X$ such that
$\beta^{\top}x_{1}\not=\beta^{\top}x_{2}$. From (2), we have that
$f_{0}(x_{1})/f(x_{1})=(1-\phi(x_{1},\theta))/(1-c(\theta,F))$ and
$f_{0}(x_{2})/f(x_{2})=(1-\phi(x_{2},\theta))/(1-c(\theta,F))$. From these two
quantities, we can solve out $\alpha$ as long as
$\beta^{\top}x_{1}\not=\beta^{\top}x_{2}$. Thus, $\alpha$ is identifiable when
$\beta\not=0$. If $\beta=0$, it is clear that different $\alpha$ values give
the same case and control densities which are the same as $f(x)$. That means
$\alpha$ is not identifiable if $\beta=0$.

###  3.2 Maximum likelihood estimation

To estimate the unknown parameters, we use the maximum likelihood estimation
approach. The likelihood function of the observed data is given by

$\prod_{i=1}^{n}f_{1}(x_{i})^{y_{i}}f_{0}(x_{i})^{1-y_{i}}\prod_{i=n+1}^{N}f(x_{i}).$

Using the relationship in (2) and taking the logarithm of the likelihood, we
obtain the following log-likelihood function of $\theta$ and $f$

$\displaystyle l(\theta,f)$ $\displaystyle=$
$\displaystyle\sum_{i=1}^{n}\left[y_{i}\log\phi(x_{i},\theta)+(1-y_{i})\log(1-\phi(x_{i},\theta))\right]$
$\displaystyle-\left[n_{1}\log
c(\theta,F)+n_{0}\log(1-c(\theta,F))\right]+\sum_{i=1}^{N}\log f(x_{i}).$

To deal with the non-parametric component $f$, we adopt the discretization
approach in Zeng and Lin (2007) and many others. Specifically, assume that
there is no tie in the observed values of $X$. We restrict $F$ to be a step
function with non-negative jumps only at the observed $x_{i}$’s. Let $p_{i}$
be the jump size of $F$ at $x_{i}$, $i=1,\ldots,N$. Accordingly, $p_{i}$’s
satisfy that $p_{i}\geqslant 0$ and $\sum_{i=1}^{N}p_{i}=1$. Meanwhile, by
this restriction, $c(\theta,F)=\sum_{i=1}^{N}\phi(x_{i},\theta)p_{i}$. To
distinguish from its original definition, we use the notation
$\tilde{c}(\theta,{\bf p})$ to represent this discretization version instead,
where ${\bf p}=(p_{1},\ldots,p_{N})$.

After the discretization, the log-likelihood function becomes

$\displaystyle l(\theta,{\bf p})$ $\displaystyle=$
$\displaystyle\sum_{i=1}^{n}\left[y_{i}\log\phi(x_{i},\theta)+(1-y_{i})\log(1-\phi(x_{i},\theta))\right]$
$\displaystyle-\left[n_{1}\log\tilde{c}(\theta,{\bf
p})+n_{0}\log(1-\tilde{c}(\theta,{\bf p}))\right]+\sum_{i=1}^{N}\log p_{i},$

where $\tilde{c}(\theta,{\bf p})=\sum_{i=1}^{N}\phi(x_{i},\theta)p_{i}$, as
defined before. We maximize the log-likelihood $l(\theta,{\bf p})$ with
respect to $\theta$ and $\bf{p}$ under the constraints $\sum_{i=1}^{N}p_{i}=1$
and $p_{i}\geqslant 0,i=1,\ldots,N$. Denote the maximizer by $\hat{\theta}$
and $\hat{{\bf p}}=(\hat{p}_{1},\ldots,\hat{p}_{N})$. Define the estimator
$\hat{F}(x)=\sum_{x_{i}\in\mathbf{r}(x)}\hat{p}_{i}$, where $\mathbf{r}(x)$ is
the rectangular set decided by $x$. Then $(\hat{\theta},\hat{F}(x))$ is
treated as the maximum likelihood estimator (MLE) of $(\theta,F(x))$.

Based on the logistic model (1) and the MLEs of the model parameters, a
natural estimator for the case proportion, i.e., $\mathsf{P}(Y=1)$, is given
by $\hat{P}=\sum_{i=1}^{N}\phi(x_{i};\hat{\theta})\hat{p}_{i}$.

###  3.3 The algorithm

The optimization problem is

$\max_{\theta,{\bf p}}l(\theta,{\bf p})\ \ \ \mbox{subject to}\ \ \
\sum_{i=1}^{N}p_{i}=1,p_{i}\geqslant 0,i=1,\ldots,N.$ (3)

Motivated by the idea of coordinate descent, we develop an iterative algorithm
to solve this constrained optimization problem (3).

Specifically, let ${\bf p}^{(0)}=(p_{1}^{(0)},\ldots,p_{N}^{(0)})$ be an
initial value of $\bf p$. For the $j$-th ($j\geqslant 0$) step of the
iteration, given the value of ${\bf p}^{(j)}$, we update the value of $\theta$
by maximizing

$\displaystyle\sum_{i=1}^{n}\left[y_{i}\log\phi(x_{i},\theta)+(1-y_{i})\log(1-\phi(x_{i},\theta))\right]-\left[n_{1}\log\tilde{c}^{(j)}(\theta)+n_{0}\log(1-\tilde{c}^{(j)}(\theta))\right]$
(4)

with respect to $\theta$, where
$\tilde{c}^{(j)}(\theta)=\sum_{i=1}^{N}\phi(x_{i},\theta)p_{i}^{(j)}$. Denote
the maximizer by $\theta^{(j)}$. The optimization can be solved by the
gradient descent algorithm.

Given the value of $\theta^{(j)}$ and ${\bf p}^{(j)}$, we update the value of
$\bf p$ by maximizing $l(\theta^{(j)},{\bf p})$ with respect to $\bf p$,
subject to the constraints $\sum_{i=1}^{N}p_{i}=1$ and
$\sum_{i=1}^{N}\phi(x_{i},\theta^{(j)})p_{i}=\tilde{c}^{(j)}(\theta^{(j)})$.
This is a convex optimization problem. The unique solution is given by

$p_{i}=\left(\frac{n_{1}}{\tilde{c}^{(j)}(\theta^{(j)})}\phi(x_{i},\theta^{(j)})+\frac{n_{0}}{1-\tilde{c}^{(j)}(\theta^{(j)})}\left(1-\phi(x_{i},\theta^{(j)})\right)+N-n_{1}-n_{0}\right)^{-1},$

$i=1,\ldots,N$. Thus, the value of $p^{(j)}$’s is updated by

$p_{i}^{(j+1)}=\left(\frac{n_{1}}{\tilde{c}^{(j)}(\theta^{(j)})}\phi(x_{i},\theta^{(j)})+\frac{n_{0}}{1-\tilde{c}^{(j)}(\theta^{(j)})}\left(1-\phi(x_{i},\theta^{(j)})\right)+N-n_{1}-n_{0}\right)^{-1},$
(5)

$i=1,\ldots,N$. We iteratively update the estimate values by maximizing (4)
and calculating (5) until convergence. The resulting values are treated as the
MLEs.

The iterative algorithm is summarized in Algorithm 1.

Algorithm 1 Iterated algorithm for computing the MLEs

Set an initial value $\textbf{p}^{(0)}.$

repeat

Step 1. Given $\textbf{p}^{(j)}$, compute $\theta^{(j)}$ by maximizing (4).

Step 2. Given $(\theta^{(j)},\textbf{p}^{(j)})$, obtain $\textbf{p}^{(j+1)}$
according to (5).

until $\|\theta^{(j+1)}-\theta^{(j)}\|\leqslant\epsilon$ and
$\|\textbf{p}^{(j+1)}-\textbf{p}^{(j)}\|\leqslant\epsilon$, where $\|\cdot\|$
is the Euclidean norm and $\epsilon$ is a given constant.

At each iteration, the value of the objective function $l(\theta,{\bf p})$
increases. Since $l(\theta,{\bf p})$ is continuous in $\theta$ and $\bf p$ and
is bound by $l(\hat{\theta},\hat{\bf p})$, the proposed iterative algorithm
will at least converge to a local maximizer. To obtain the global maximizer
with higher probability, one may try different initial values.

###  3.4 Large-sample properties

We discuss the asymptotic properties of the proposed MLE $\hat{\theta}$ and
$\hat{F}(x)$. Let $\theta_{0}=(\alpha_{0},\beta_{0}^{\top})^{\top}$ and
$F_{0}(x)$ be the true value of $\theta$ and $F(x)$, respectively. Let
$\mathcal{G}=\\{g\in\mbox{BV}[\mathbb{R}^{d}]:|g|\leqslant 1\\}$, where
$\mbox{BV}[D]$ is the set of functions on domain $D$ with bounded total
variation. The following regularity conditions are imposed.

C1. The density $f(x)$ is continuous with bounded support.

C2. If $\mathsf{P}(u^{\top}X=0)=1$ for some constant vector $u$, then $u=0$.

C3. The true parameter value $\theta_{0}\in{\mathcal{B}}$, where
${\mathcal{B}}$ is a compact set.

The condition $\it C1$ implies that the covariates are bounded. The condition
$\it C2$ means that the covariates vector does not lie in a degenerated linear
space. The condition $\it C3$ assumes that $\theta_{0}$ is an interior point
of a compact parameter space. They are all common regularity conditions.

The main results includes the consistency and the asymptotic normality of the
MLEs, as given in the following two theorems.

######  Theorem 1.

(Consistency) Suppose that conditions C1-C3 hold. If $n/N\to c_{1}$ for some
constant $c_{1}\in(0,1)$ and $n_{1}/n_{0}\to c_{2}$ for some constant
$c_{2}\in(0,\infty)$ as $N\to\infty$, then $|\hat{\theta}-\theta_{0}|\to 0$
and $\sup_{x\in\mathbb{R}^{p}}|\hat{F}(x)-F_{0}(x)|\to 0$ almost surely.

######  Theorem 2.

(Asymptotic normality) Suppose that conditions C1-C3 hold. If $n/N\to c_{1}$
for some constant $c_{1}\in(0,1)$ and $n_{1}/n_{0}\to c_{2}$ for some constant
$c_{2}\in(0,\infty)$ as $N\to\infty$, then
$\sqrt{N}(\hat{\theta}-\theta_{0},\hat{F}-F_{0})$ converges weakly to a zero-
mean Gaussian process in the metric space $\mathbb{R}^{p+1}\times
L^{\infty}(\mathcal{G})$. The limiting covariance matrix of
$\sqrt{N}(\hat{\theta}-\theta_{0})$ attains the semi-parametric efficiency
bound.

We prove the two theorems in the Appendix. For any
$(v,g_{1})\in\mathcal{V}\times\mathcal{G}$, the asymptotic covariance matrix
for

$\displaystyle\sqrt{N}v^{\top}\left(\hat{\theta}-\theta_{0}\right)+\sqrt{N}\int_{\cal
X}\vec{g}_{1}(x)d\left\\{\hat{F}(x)-F_{0}(x)\right\\}$

can be estimated by
$(v^{\top},\vec{g_{1}}^{\top})I_{N}^{-1}(v^{\top},\vec{g_{1}}^{\top})^{\top}$,
where $NI_{N}$ is the negative Hessian matrix of the log-likelihood function
$l(\theta,{\bf p})$ with respect to $(\theta,{\bf p})$ and
$\vec{g_{1}}=(g_{1}\left(x_{1}\right),\ldots,g_{1}\left(x_{N}\right))$. By
taking $\vec{g}_{1}=0$, the asymptotic covariance matrix of
$\sqrt{N}(\hat{\theta}-\theta_{0})$ can be estimated by the upper left
$(p+1)\times(p+1)$ matrix of $I_{N}^{-1}$.

Note that in the two theorems we assume that the sample size of the labeled
data and the unlabeled data are comparable in the limiting sense. As we have
mentioned in Section 1, in many machine learning applications, the size of
labeled data is small or moderate while the size of unlabeled data is much
larger. When $n/N\to 0$ as $n\to\infty$, it can be shown that the convergence
rate of $\hat{\theta}$ is at the speed of $\sqrt{n}$ and
$\sqrt{n}(\hat{\theta}-\theta_{0})$ converges in distribution to a normal
distribution with mean zero. The corresponding covariance matrix of the
limiting distribution equals to the one obtained from the case-control sample
with a known covariate distribution $F$. That is, if $n/N\to 0$, we can make
inference about $\theta_{0}$ based on the semi-supervised data as if the
marginal distribution of covariates were known to us.

##  4 Numerical results

###  4.1 Simulation studies

We conduct some simulation studies to show the finite sample performance of
the proposed method. We consider a 2-dimensional covariates vector
$X=(X_{1},X_{2})$, where $X_{1}$ are $X_{2}$ are independently generated from
standard normal distribution. The response $Y$ is generated from the logistic
model (1) with the parameters $\alpha$, $\beta_{1}$, and $\beta_{2}$. The
labeled data are drawn by the case-control sampling from the case population
and the control population, respectively. The unlabeled data are drawn from
the marginal distribution of $X$. We use $\mathsf{p}$ to represent the
population percentage of the cases, i.e., $\mathsf{p}=\mathsf{P}(Y=1)$, and
use $\mathsf{q}$ to represent the proportion of the cases in the labeled data,
i.e., $\mathsf{q}=n_{1}/n$.

In the first sets of the study, we concentrate on performance of the
estimation procedure. We try different values of the regression parameters
which lead to different values of $\mathsf{p}$. We also consider different
choices for $n$, $\mathsf{q}$, and $N$. For the generated labeled data and
unlabeled data, we use the proposed approach to obtain the MLEs of the
regression parameters and the estimated standard errors. At the same time, we
use the labeled data only to estimate the parameters, that is, to fit the
model (1) with a single case-control sample. One thousand replications are
carried out. For each regression parameter, we record the average bias and
empirical standard errors of the MLEs, the average of the estimated standard
errors, and the empirical coverage percentage of the 95% Wald confidence
interval. We also report the average bias of the estimator of
$\mathsf{P}(Y=1)$, which is described in Section 3.2. In Table 1 we list the
cases with different regression parameters and the values of $\mathsf{p}$,
$n$, and $\mathsf{q}$. The simulation results are listed in Table 2.

Table 1: Parameter values for data generating and sampling in estimation
Case | $\alpha$ | $\beta_{1}$ | $\beta_{2}$ | $n_{0}$ | $n_{1}$ | $\mathsf{p}$ | $\mathsf{q}$
---|---|---|---|---|---|---|---
$A_{1}$ | $-6$ | $-2$ | 2 | 400 | 80 | 0.037 | 0.167
$A_{2}$ | $-5$ | $-2$ | 2 | 400 | 80 | 0.067 | 0.167
$A_{3}$ | $-4$ | 2 | 1 | 400 | 80 | 0.081 | 0.167
$A_{4}$ | $-4$ | 2 | 2 | 400 | 80 | 0.116 | 0.167
$B_{1}$ | $-6$ | $-2$ | 2 | 300 | 150 | 0.037 | 0.333
$B_{2}$ | $-5$ | $-2$ | 2 | 300 | 150 | 0.067 | 0.333
$B_{3}$ | $-4$ | 2 | 1 | 300 | 150 | 0.081 | 0.333
$B_{4}$ | $-4$ | 2 | 2 | 300 | 150 | 0.116 | 0.333
$C_{1}$ | $-6$ | $-2$ | 2 | 200 | 200 | 0.037 | 0.5
$C_{2}$ | $-5$ | $-2$ | 2 | 200 | 200 | 0.067 | 0.5
$C_{3}$ | $-4$ | 2 | 1 | 200 | 200 | 0.081 | 0.5
$C_{4}$ | $-4$ | 2 | 2 | 200 | 200 | 0.116 | 0.5
Table 2: Summary results for estimation and inference

Case | Type | N | Bias | SE | ESE | CP | P Bias
---|---|---|---|---|---|---|---
$\alpha$ | $\beta_{1}$ | $\beta_{2}$ | $\alpha$ | $\beta_{1}$ | $\beta_{2}$ | $\alpha$ | $\beta_{1}$ | $\beta_{2}$ | $\alpha$ | $\beta_{1}$ | $\beta_{2}$
$A_{1}$ | single | 480 | 1.553 | $-0.048$ | 0.050 | 0.470 | 0.282 | 0.280 | 0.457 | 0.263 | 0.264 | 0.132 | 0.944 | 0.955 | 0.130
| 1 | 960 | $-0.227$ | $-0.071$ | 0.061 | 1.108 | 0.265 | 0.269 | 1.983 | 0.260 | 0.259 | 0.967 | 0.957 | 0.953 | 0.003
| 4 | 2400 | $-0.137$ | $-0.059$ | 0.053 | 0.538 | 0.258 | 0.258 | 0.564 | 0.254 | 0.252 | 0.960 | 0.962 | 0.956 | 0.001
| 10 | 5280 | $-0.095$ | $-0.021$ | 0.038 | 0.491 | 0.254 | 0.258 | 0.490 | 0.247 | 0.247 | 0.956 | 0.947 | 0.951 | 0.000
$A_{2}$ | single | 480 | 0.957 | $-0.036$ | 0.031 | 0.395 | 0.269 | 0.267 | 0.406 | 0.260 | 0.259 | 0.352 | 0.952 | 0.952 | 0.100
| 1 | 960 | $-0.100$ | $-0.041$ | 0.056 | 0.557 | 0.266 | 0.269 | 0.548 | 0.258 | 0.259 | 0.951 | 0.951 | 0.946 | 0.003
| 4 | 2400 | $-0.093$ | $-0.034$ | 0.039 | 0.464 | 0.254 | 0.255 | 0.464 | 0.254 | 0.255 | 0.968 | 0.951 | 0.947 | 0.001
| 10 | 5280 | $-0.078$ | $-0.034$ | 0.044 | 0.446 | 0.260 | 0.259 | 0.446 | 0.254 | 0.254 | 0.951 | 0.948 | 0.955 | 0.002
$A_{3}$ | single | 480 | 0.779 | 0.026 | 0.021 | 0.262 | 0.232 | 0.178 | 0.298 | 0.235 | 0.183 | 0.261 | 0.959 | 0.963 | 0.086
| 1 | 960 | $-0.180$ | 0.038 | 0.028 | 0.847 | 0.247 | 0.185 | 1.440 | 0.235 | 0.181 | 0.954 | 0.943 | 0.951 | 0.000
| 4 | 2400 | $-0.078$ | 0.036 | 0.010 | 0.421 | 0.245 | 0.183 | 0.413 | 0.234 | 0.178 | 0.948 | 0.953 | 0.949 | 0.000
| 10 | 5280 | $-0.083$ | 0.035 | 0.020 | 0.402 | 0.232 | 0.179 | 0.390 | 0.233 | 0.178 | 0.949 | 0.959 | 0.948 | 0.000
$A_{4}$ | single | 480 | 0.362 | 0.039 | 0.034 | 0.340 | 0.258 | 0.262 | 0.358 | 0.258 | 0.258 | 0.774 | 0.953 | 0.954 | 0.051
| 1 | 960 | $-0.130$ | 0.045 | 0.056 | 0.471 | 0.273 | 0.270 | 0.466 | 0.258 | 0.259 | 0.949 | 0.945 | 0.954 | $-0.002$
| 4 | 2400 | $-0.089$ | 0.042 | 0.048 | 0.421 | 0.262 | 0.259 | 0.417 | 0.258 | 0.258 | 0.952 | 0.957 | 0.950 | $-0.001$
| 10 | 5280 | $-0.068$ | 0.043 | 0.056 | 0.410 | 0.255 | 0.255 | 0.406 | 0.258 | 0.259 | 0.954 | 0.957 | 0.957 | 0.001
$B_{1}$ | single | 450 | 2.489 | -0.047 | 0.049 | 0.361 | 0.238 | 0.239 | 0.366 | 0.233 | 0.233 | 0.000 | 0.946 | 0.953 | 0.296
| 1 | 900 | $-0.336$ | $-0.043$ | 0.041 | 1.385 | 0.230 | 0.222 | 2.469 | 0.219 | 0.219 | 0.971 | 0.940 | 0.959 | 0.000
| 4 | 2250 | $-0.127$ | $-0.024$ | 0.021 | 0.565 | 0.206 | 0.209 | 0.646 | 0.205 | 0.205 | 0.964 | 0.950 | 0.954 | 0.000
| 10 | 4950 | $-0.095$ | $-0.016$ | 0.021 | 0.371 | 0.202 | 0.211 | 0.381 | 0.197 | 0.198 | 0.965 | 0.958 | 0.935 | $-0.001$
$B_{2}$ | single | 450 | 1.864 | -0.047 | 0.041 | 0.311 | 0.229 | 0.228 | 0.327 | 0.230 | 0.230 | 0.005 | 0.957 | 0.952 | 0.267
| 1 | 900 | $-0.154$ | $-0.035$ | 0.035 | 0.646 | 0.224 | 0.226 | 0.884 | 0.220 | 0.220 | 0.962 | 0.953 | 0.946 | 0.000
| 4 | 2250 | $-0.081$ | $-0.036$ | 0.039 | 0.382 | 0.213 | 0.218 | 0.365 | 0.213 | 0.213 | 0.950 | 0.960 | 0.954 | 0.001
| 10 | 4950 | $-0.086$ | $-0.026$ | 0.031 | 0.341 | 0.206 | 0.210 | 0.338 | 0.208 | 0.208 | 0.955 | 0.963 | 0.966 | 0.000
$B_{3}$ | single | 450 | 1.708 | 0.029 | 0.018 | 0.217 | 0.210 | 0.159 | 0.234 | 0.206 | 0.159 | 0.000 | 0.941 | 0.951 | 0.253
| 1 | 900 | $-0.109$ | 0.035 | 0.013 | 0.476 | 0.208 | 0.153 | 0.490 | 0.202 | 0.151 | 0.968 | 0.945 | 0.945 | 0.000
| 4 | 2250 | $-0.061$ | 0.026 | 0.019 | 0.350 | 0.197 | 0.146 | 0.343 | 0.197 | 0.145 | 0.956 | 0.955 | 0.950 | 0.000
| 10 | 4950 | $-0.060$ | 0.025 | 0.013 | 0.321 | 0.198 | 0.146 | 0.312 | 0.195 | 0.142 | 0.941 | 0.959 | 0.947 | 0.000
$B_{4}$ | single | 450 | 1.304 | 0.039 | 0.033 | 0.274 | 0.237 | 0.229 | 0.283 | 0.227 | 0.226 | 0.021 | 0.946 | 0.943 | 0.217
| 1 | 900 | $-0.091$ | 0.041 | 0.033 | 0.396 | 0.229 | 0.229 | 0.390 | 0.223 | 0.222 | 0.948 | 0.951 | 0.950 | $-0.001$
| 4 | 2250 | $-0.047$ | 0.025 | 0.030 | 0.314 | 0.215 | 0.223 | 0.322 | 0.217 | 0.218 | 0.949 | 0.957 | 0.955 | 0.000
| 10 | 4950 | $-0.080$ | 0.033 | 0.029 | 0.318 | 0.219 | 0.221 | 0.308 | 0.216 | 0.216 | 0.946 | 0.951 | 0.944 | $-0.002$
$C_{1}$ | single | 400 | 3.190 | $-0.048$ | 0.046 | 0.346 | 0.255 | 0.247 | 0.344 | 0.238 | 0.238 | 0.000 | 0.950 | 0.948 | 0.463
| 1 | 800 | $-0.511$ | $-0.051$ | 0.045 | 1.798 | 0.229 | 0.230 | 2.301 | 0.215 | 0.215 | 0.970 | 0.946 | 0.936 | $-0.001$
| 4 | 2000 | $-0.166$ | $-0.018$ | 0.025 | 0.492 | 0.193 | 0.185 | 0.476 | 0.193 | 0.193 | 0.962 | 0.951 | 0.961 | $-0.002$
| 10 | 4400 | $-0.121$ | $-0.021$ | 0.014 | 0.384 | 0.184 | 0.176 | 0.394 | 0.182 | 0.182 | 0.940 | 0.955 | 0.959 | $-0.002$
$C_{2}$ | single | 400 | 2.577 | $-0.031$ | 0.041 | 0.290 | 0.239 | 0.236 | 0.305 | 0.233 | 0.233 | 0.000 | 0.951 | 0.962 | 0.433
| 1 | 800 | $-0.178$ | $-0.034$ | 0.030 | 0.682 | 0.220 | 0.233 | 0.878 | 0.216 | 0.216 | 0.968 | 0.951 | 0.938 | $-0.001$
| 4 | 2000 | $-0.090$ | $-0.029$ | 0.031 | 0.343 | 0.206 | 0.202 | 0.340 | 0.202 | 0.202 | 0.952 | 0.946 | 0.955 | 0.000
| 10 | 4400 | $-0.087$ | $-0.026$ | 0.035 | 0.304 | 0.208 | 0.211 | 0.305 | 0.195 | 0.196 | 0.953 | 0.942 | 0.939 | 0.000
$C_{3}$ | single | 400 | 2.406 | 0.039 | 0.023 | 0.196 | 0.216 | 0.174 | 0.219 | 0.210 | 0.163 | 0.000 | 0.956 | 0.944 | 0.419
| 1 | 800 | $-0.210$ | 0.032 | 0.007 | 0.978 | 0.213 | 0.148 | 2.263 | 0.199 | 0.146 | 0.972 | 0.938 | 0.954 | $-0.002$
| 4 | 2000 | $-0.073$ | 0.026 | 0.024 | 0.360 | 0.196 | 0.135 | 0.347 | 0.191 | 0.136 | 0.962 | 0.949 | 0.948 | 0.000
| 10 | 4400 | $-0.070$ | 0.028 | 0.014 | 0.296 | 0.192 | 0.130 | 0.291 | 0.186 | 0.131 | 0.948 | 0.947 | 0.956 | $-0.001$
$C_{4}$ | single | 400 | 2.002 | 0.024 | 0.039 | 0.254 | 0.230 | 0.233 | 0.265 | 0.230 | 0.231 | 0.000 | 0.951 | 0.956 | 0.384
| 1 | 800 | $-0.089$ | 0.022 | 0.033 | 0.401 | 0.217 | 0.213 | 0.383 | 0.218 | 0.219 | 0.950 | 0.955 | 0.958 | $-0.002$
| 4 | 2000 | $-0.085$ | 0.038 | 0.035 | 0.288 | 0.220 | 0.223 | 0.296 | 0.211 | 0.211 | 0.957 | 0.948 | 0.944 | $-0.002$
| 10 | 4400 | $-0.052$ | 0.026 | 0.033 | 0.277 | 0.211 | 0.209 | 0.274 | 0.206 | 0.206 | 0.952 | 0.950 | 0.961 | $-0.001$

  * 1

Bias: average bias of the estimates; SE: empirical standard error of the
estimates; SEE: average of the estimated standard errors; CP: empirical
coverage probabilities of Wald-type confidence intervals with $95\%$
confidence level; P Bias: average bias of the estimator of $\mathsf{P}(Y=1)$.

Note that in Table 2, $N=480$ corresponds to the scenario where a single case-
control sample is used to fit the logistic model (“single” in the column
“Type”). The numbers in the column “Type” stands for the ratio between the
size of the unlabeled data and that of the labeled data (for example, in case
$A_{1}$, $N=960$ means that the ratio between size of the unlabeled data and
that of the labeled data is 1). It has been mentioned that the single case-
control sample cannot identify the intercept parameter $\alpha$. Thus, the
estimate of $\alpha$ given by the single case-control sample is obviously
biased. Meanwhile, the single case-control sample cannot consistently estimate
$\mathsf{P}(Y=1)$, either. When the unlabeled data is introduced, the proposed
MLE of $\alpha$ is ultimately unbiasd. Moreover, incorporating the unlabeled
data also helps in increasing the estimation accuracy of $\beta$. The standard
errors of the proposed MLEs of $\beta_{1}$ and $\beta_{2}$ are smaller than
those obtained from the single case-control sample. Moreover, the standard
errors of the proposed MLEs decreases in almost all the scenarios as $N$
increases, especially when $|\mathsf{p}-\mathsf{q}|$ is relatively larger. The
proposed estimator of $\mathsf{P}(Y=1)$ also shows reasonable finite sample
performance.

In the second sets of the study, we focus on the prediction performance. For
the regression parameters, we choose the values of Case A1 to A4 listed in
Table 1. For the training data, we set $n_{0}=400$ and $n_{1}=80$. The size of
the unlabeled data is $4800$. Another labeled data set with size $4800$ is
generated to serve as the testing data. We consider four methods for
prediction. The first two are two existing semi-supervised learning approaches
called self-training proposed by Yarowsky (1995) and co-training proposed by
Blum and Mitchell (1998), respectively. The Logistic model is used to be the
base learner in the two approaches. The third one is to use the labeled data
only to train the logistic model. The fourth one is the proposed approach
using the MLE for model training. One thousand replications are carried out.
We record the average bias of the model parameter estimates and the average
AUC (i.e., area under the curve) of the four methods. Moreover, for each
method, we use the fitted probability that maximizes the difference between
the training true positive rate and false positive rate to be the cut-off
point and obtain the corresponding accuracy score (i.e., the percentage of the
corrected prediction), recall, precision, and F1 score. Finally, we also
record the average absolute bias of the estimation of $\mathsf{P}(Y=1)$ on the
testing data. The results are summarized in Table 3.

Table 3: Summary results for prediction

Case | Method | Bias | AUC | accuracy | recall | precision | F1 score | P Bias
---|---|---|---|---|---|---|---|---
$\alpha$ | $\beta_{1}$ | $\beta_{2}$
$A_{1}$ | self-training | -5.2324 | -2.8577 | 2.8598 | 0.9461 | 0.8658 | 0.8737 | 0.2085 | 0.3339 | 0.0201
| co-training | 3.7452 | 0.6760 | -1.2364 | 0.9382 | 0.8562 | 0.8729 | 0.1961 | 0.3179 | 0.2891
| case-control | 1.5553 | -0.0508 | 0.0542 | 0.9461 | 0.8653 | 0.8742 | 0.2081 | 0.3333 | 0.0600
| proposed | -0.1100 | -0.0407 | 0.0447 | 0.9462 | 0.8657 | 0.8742 | 0.2085 | 0.3339 | 0.0079
$A_{2}$ | self-training | -5.8382 | -3.1816 | 3.1887 | 0.9377 | 0.8539 | 0.8622 | 0.3088 | 0.4510 | 0.0103
| co-training | 2.8309 | 0.6959 | -1.2123 | 0.9298 | 0.8467 | 0.8574 | 0.2971 | 0.4376 | 0.2628
| case-control | 0.9470 | -0.0423 | 0.0449 | 0.9378 | 0.8535 | 0.8627 | 0.3082 | 0.4504 | 0.0500
| proposed | -0.0711 | -0.0377 | 0.0403 | 0.9378 | 0.8540 | 0.8621 | 0.3091 | 0.4513 | 0.0106
$A_{3}$ | self-training | -4.7757 | 3.1405 | 1.5812 | 0.9033 | 0.8159 | 0.8169 | 0.2901 | 0.4239 | 0.0117
| co-training | 1.7238 | -0.5740 | -0.3010 | 0.8883 | 0.8171 | 0.7866 | 0.2882 | 0.4172 | 0.2477
| case-control | 0.7730 | 0.0411 | 0.0211 | 0.9033 | 0.8151 | 0.8178 | 0.2895 | 0.4233 | 0.0514
| proposed | -0.0680 | 0.0385 | 0.0203 | 0.9034 | 0.8154 | 0.8177 | 0.2897 | 0.4235 | 0.0137
$A_{4}$ | self-training | -6.3554 | 3.6116 | 3.6215 | 0.9295 | 0.8415 | 0.8529 | 0.4203 | 0.5588 | 0.0153
| co-training | 1.9428 | -0.7585 | -1.1693 | 0.9211 | 0.8350 | 0.8443 | 0.4088 | 0.5465 | 0.2177
| case-control | 0.3498 | 0.0509 | 0.0529 | 0.9295 | 0.8422 | 0.8519 | 0.4215 | 0.5597 | 0.0264
| proposed | -0.0818 | 0.0503 | 0.0515 | 0.9295 | 0.8419 | 0.8525 | 0.4209 | 0.5594 | 0.0128

  * Bias: average bias of the estimates; P Bias: average absolute bias of the estimation of $\mathsf{P}(Y=1)$ on the testing data.

Note that the self-training and co-training approaches are designed for the
prospective sampling for the labeled data. For the case-control sampling, the
two semi-supervised learning approaches give out similar performances on the
AUC, accuracy score, recall, precision, and F1 score compared with the
proposed method. However, the two approaches show poor performance on the
estimation of the regression parameters in the Logistic model, while the
proposed MLE gives out consistent estimates. Consequently, the proposed method
has the best prediction accuracy on the marginal percentage $\mathsf{P}(Y=1)$.
This is because our proposed method accounts for the biasd sampling scheme in
the labeled data. From our results, it seems that ignoring the case-control
sampling scheme does not obviously affect the overall accuracy of the
individual prediction conditioning on its covariates.

###  4.2 Pima Indians diabetes data

We apply the proposed method to the Pima Indians diabetes data, which is
originally from the National Institute of Diabetes and Digestive and Kidney
disease. The data set consists of a target variable called “Outcome” standing
for the presence or absence of the diabetes (coded as 1 and 0, respectively)
and several medical predictors. Here we use the variables “Glucose”,
“Pregnancies”, and “BMI” as the covariates. There are 268 cases (i.e.,
“Outcome” coded as 1) and 500 controls (i.e., “Outcome” coded as 0) in the
data. We randomly split the data into the training set (80% of the full set)
and the testing set (20% of the full set). In the training set, we randomly
draw 50 samples from the cases and 50 samples from the controls to form a
case-control sample. Moreover, we randomly draw 200 samples from the training
set and use the covariates information to form an unlabeled set. The 300
samples are treated as a semi-supervised data set and the proposed approach is
used to get the parameter estimates. Then the fitted logistic model is adopted
to do prediction on the testing set. For the $i$th data point in the testing
set, we calculate the estimated case probability $\phi(x_{i};\hat{\theta})$
and use the mean absolute deviation (MAD)
$n_{t}^{-1}\sum_{i=1}^{n_{t}}|y_{i}-\phi(x_{i};\hat{\theta})|$, where $n_{t}$
is the size of the testing set, to measure the prediction accuracy. We also
calculate the MAD based on the case-control sample only (i.e., the 50 cases
and 50 controls) for comparison. We repeat the sampling procedure for 100
times and get the average of the regression parameter estimates, the average
of the estimated standard errors, and the MAD. We treat the parameter
estimates obtained from the full training data set and the corresponding MAD
for the testing data set as the benchmark. The analysis results are summarized
in Table 4.

Table 4: Estimation results for Pima India diabetes data

|  | Parameter | MAD | Case proportion
---|---|---|---|---
|  | $\alpha$ | $\beta_{1}$ | $\beta_{2}$ | $\beta_{3}$
Case-control | AEST | $-0.1925$ | 1.1380 | 0.4084 | 0.7109 | 0.3381 |
| AESE | 0.2513 | 0.3020 | 0.2549 | 0.3063 |  |
Proposed MLE | AEST | $-0.8971$ | 1.1320 | 0.3942 | 0.7157 | 0.3101 | 0.3403
| AESE | 0.5548 | 0.2995 | 0.2491 | 0.3032 |  |
Full training set | EST | $-0.7811$ | 1.0518 | 0.3963 | 0.6522 | 0.3126 |
| ESE | 0.0107 | 0.0137 | 0.0098 | 0.0143 |  |

  * 1

EST.A: average of estimated parameter values; ESE.A: average of estimated
standard errors; EST: estimated parameter value; ESE: estimated standard
error.

It is quite obvious to see that the average value of the proposed MLE of
$\alpha$ is much closer to the MLE based on the full training set than that
obtained from the case-control sample. The introduction of the unlabeled data
helps in identifying the intercept parameter. Meanwhile, compared with the
results based on the single case-control sample, the estimation efficiency of
$\beta$ and the prediction accuracy also increase when the unlabeled data are
incorporated.

##  5 Concluding remarks

We consider the semi-supervised inference with binary response data under the
case-control sampling. Assuming the data following logistic regression model,
we find that the unlabeled data helps to identify the intercept parameter
which cannot be identified by single case-control data. The likelihood
function of the observed data is derived and the MLE of the model parameters
are obtained by an iterative algorithm. The MLE is shown to reach the
semiparametric efficient bound asymptotically. Numerical studies show that the
unlabeled data not only helps in identifying the intercept parameter but also
increases the estimation efficiency of the slope parameter. Compared with some
existing prediction methods for semi-supervised binary data, the proposed
approach give out competitive prediction performances. When the labeled data
is case-control data, the proposed approach gives out the most accurate
prediction on the marginal case percentage.

In real applications, the models are easily misspecified. For the binary
response data, when the logistic model is misspecified, the model parameter
becomes a certain population risk minimizer defined through the log-likelihood
function of the logistic model. Under this circumstance, the case-control
sampling cannot provide a consistent estimator of this risk minimizer. The
limit of the MLE may depend on the ratio between the number of cases and that
of controls. Although there exist unlabeled data, it seems that the proposed
approach does not provide consistent estimator directly for misspecified
logistic model. Either it is not clear how the model misspecifiation would
affect the performance of prediction. This would be an interesting direction
for further investigation in the future.

## Appendix

Here we give out the proof of the two theorems in Section 3.4.

Proof of Theorem 1: For simplicity of notation, we just consider $p=1$ in this
proof and it can be easily extended to a more general space. By definition
$\phi(x,\theta)\in(0,1)$ for all $x$. According to Theorem 1 in Vardi (1985),
maximizing $l(\theta,\textbf{p})$ yields unique solution for $F(x)$ when
$\theta$ is fixed. Denote the jump size $p_{i}$ by $F\\{x_{i}\\}$ and
$\hat{p}_{i}$ by $\hat{F}\\{x_{i}\\}$, $i=1,\ldots,N$. Using the method of
Lagrange multipliers and some calculus, we can get

$\hat{F}\\{x_{i}\\}=\left[\frac{n_{1}}{\tilde{c}(\hat{\theta},\hat{F})}\phi(x_{i},\hat{\theta})+\frac{n_{0}}{1-\tilde{c}(\hat{\theta},\hat{F})}\left(1-\phi(x_{i},\hat{\theta})\right)+N-n_{1}-n_{0}\right]^{-1},$

where $\tilde{c}(\hat{\theta},\hat{F})\equiv\tilde{c}(\hat{\theta},\hat{\bf
p})=\sum_{i=1}^{N}\phi(x_{i},\hat{\theta})\hat{F}\\{x_{i}\\}$. Since $\hat{F}$
is uniformly bounded and monotone, for any sub-sequence of $\\{\hat{F}\\}$,
there is a further sub-sequence that converges to some monotone function
$F^{\star}$ point-wise. Without loss of generality, assume that $\hat{\theta}$
converges to $\theta^{\star}$ for the same sub-sequence.

Denote

$F_{1}(x;\theta,F)=\int_{-\infty}^{x}\frac{\phi(u,\theta)}{\tilde{c}(\theta,F)}dF(u),\
F_{2}(x;\theta,F)=\int_{-\infty}^{x}\frac{1-\phi(u,\theta)}{\\{1-\tilde{c}(\theta,F)\\}}dF(u)$

and $F_{3}(x;\theta,F)=F_{3}(x;F)=F(x)$, where $\theta\in\mathcal{B}$ and $F$
is a distribution function. Let

$\bar{G}_{N}(x;\theta,F)=\frac{n_{1}}{N}F_{1}(x;\theta,F)+\frac{n_{0}}{N}F_{2}(x;\theta,F)+\frac{N-n_{1}-n_{0}}{N}F_{3}(x;F).$

Then

$\bar{G}_{N}(x;\theta,F)=\lambda_{N1}\int_{-\infty}^{x}\frac{\phi(u,\theta)}{\tilde{c}(\theta,F)}dF(u)+\lambda_{N0}\int_{-\infty}^{x}\frac{1-\phi(u,\theta)}{1-\tilde{c}(\theta,F)}dF(u)+(1-\lambda_{N1}-\lambda_{N0})\int_{-\infty}^{x}dF(u),$
(6)

where $\lambda_{N1}=n_{1}/N$ and $\lambda_{N0}=n_{0}/N$. Obviously,
$\bar{G}_{N}(x;\theta,F)\to\bar{G}(x;\theta,F)$ as $N\to\infty$, where

$\bar{G}(x;\theta,F)=\rho_{1}\int_{-\infty}^{x}\frac{\phi(u,\theta)}{\tilde{c}(\theta,F)}dF(u)+\rho_{0}\int_{-\infty}^{x}\frac{1-\phi(u,\theta)}{1-\tilde{c}(\theta,F)}dF(u)+(1-\rho_{1}-\rho_{0})\int_{-\infty}^{x}dF(u),$

$\rho_{1}=\lim\limits_{N\to\infty}\lambda_{N1}$ and
$\rho_{0}=\lim\limits_{N\to\infty}\lambda_{N0}$. It follows immediately from
(6) and $\phi(x,\theta)\in(0,1)$ that

$F_{0}(x)=\int_{-\infty}^{x}\left[\lambda_{N1}\frac{\phi(u,\theta_{0})}{\tilde{c}(\theta_{0},F_{0})}+\lambda_{N0}\frac{1-\phi(u,\theta_{0})}{1-\tilde{c}(\theta_{0},F_{0})}+(1-\lambda_{N0}-\lambda_{N1})\right]^{-1}d\bar{G}_{N}(u;\theta_{0},F_{0}).$
(7)

Denote

$\mathbb{F}_{N_{i}}(x)=\left\\{\begin{array}[]{llr}n_{1}^{-1}\sum_{i=1}^{n_{1}}I\\{x_{i}\leqslant
x\\}&i=1,&\\\ \\\ n_{0}^{-1}\sum_{i=n_{1}+1}^{n_{0}+n_{1}}I\\{x_{i}\leqslant
x\\}&i=2,&\\\ \\\
(N-n_{1}-n_{0})^{-1}\sum_{i=n_{0}+n_{1}+1}^{N}I\\{x_{i}\leqslant
x\\}&i=3.&\end{array}\right.$

Then we can write

$\mathbb{F}_{N}(x)\equiv N^{-1}\sum_{i=1}^{N}I\\{x_{i}\leqslant
x\\}=\lambda_{N1}\mathbb{F}_{N1}(x)+\lambda_{N0}\mathbb{F}_{N2}(x)+(1-\lambda_{N1}-\lambda_{N0})\mathbb{F}_{N_{3}}(x)$

as the empirical distribution. Write
$\mathbb{X}_{N}=\sqrt{N}(\mathbb{F}_{N}-\lambda_{N1}F_{1}-\lambda_{N0}F_{2}-(1-\lambda_{N1}-\lambda_{N0})F_{3})=\sqrt{N}(\mathbb{F}_{N}-\bar{G}_{N})$.
When $\theta=\theta_{0}$, $F=F_{0}$ and $\mathcal{F}$ is a Donsker class,
$\left\\{\mathbb{X}_{N}(f):f\in\mathcal{F}\right\\}$ converges in distribution
in $l^{\infty}(\mathcal{F})$ to the mean zero Gaussian
process$\\{\mathbb{X}(f):f\in\mathcal{F}\\}$.

Then, we consider a class

$\displaystyle\mathcal{A}=\bigg{\\{}$
$\displaystyle\frac{I_{(-\infty,x]}(X)}{\frac{1}{N}\left[\frac{n_{1}}{\tilde{c}(\theta,F)}\phi\left(X,\theta\right)+\frac{n_{0}}{1-\tilde{c}(\theta,F)}\left\\{1-\phi\left(X,\theta\right)\right\\}+N-n_{1}-n_{0}\right]}:\theta\in\mathcal{B},x\in
R,$ $\displaystyle F\text{ is a distribution
function},\tilde{c}(\theta,F)=\sum_{i=1}^{N}\phi\left(x_{i},\theta\right)F\left\\{x_{i}\right\\}\bigg{\\}}.$

Similar to Zeng and Yin (2006), It can be shown that $\mathcal{A}$ is a
Donsker class. Combining the fact that $\mathbb{X}_{N}$ converges to a mean
zero Gaussian process and the bounded convergence theorem, We conclude that
uniformly in $x$, $\hat{F}(x)\to F^{\star}(x)$, where

$F^{\star}(x)=E\left\\{\frac{I(X\leq
x)}{\rho_{1}\frac{\phi(X,\theta^{\star})}{\tilde{c}(\theta^{\star},F^{\star})}+\rho_{0}\frac{\\{1-\phi(X,\theta^{\star})\\}}{\\{1-\tilde{c}(\theta^{\star},F^{\star})\\}}+1-\rho_{0}-\rho_{1}}\right\\}$

and $\mathrm{E}(\cdot)$ refers to the expectation taken with respect to
$\bar{G}(x;\theta_{0},F_{0})$.

We next construct another function

$\tilde{F}\\{x_{i}\\}=\left[\frac{n_{1}}{{\tilde{c}}(\theta_{0},F_{0})}\phi(x_{i},\theta_{0})+\frac{n_{0}}{1-{\tilde{c}}(\theta_{0},F_{0})}\\{1-\phi(x_{i},\theta_{0})\\}+N-n_{1}-n_{0}\right]^{-1}.$

By (7) and the fact that $\mathbb{X}_{N}$ converges to a mean zero Gaussian
process, we conclude that $\lim\limits_{N\to\infty}\tilde{F}(x)=F_{0}(x)$
uniformly in $x$, where

$\tilde{F}(x)=\frac{1}{N}\sum_{i=1}^{N}\frac{I\\{x_{i}\leqslant
x\\}}{\left[\frac{\lambda_{N1}}{{\tilde{c}}(\theta_{0},F_{0})}\phi(x_{i},\theta_{0})+\frac{\lambda_{N0}}{1-{\tilde{c}}(\theta_{0},F_{0})}\\{1-\phi(x_{i},\theta_{0})\\}+(1-\lambda_{N1}-\lambda_{N0})\right]}.$

Note that $\hat{F}(x)$ is absolutely continuous with respect to $\tilde{F}(x)$
with $\hat{F}(x)$ can be written as

$\hat{F}(x)=\int_{-\infty}^{x}\frac{\frac{n_{1}}{{\tilde{c}}(\theta_{0},F_{0})}\phi(u,\theta_{0})+\frac{n_{2}}{1-{\tilde{c}}(\theta_{0},F_{0})}\\{1-\phi(u,\theta_{0})\\}+N-n_{1}-n_{0}}{\frac{n_{1}}{\tilde{c}(\hat{\theta},\hat{F})}\phi(u,\hat{\theta})+\frac{n_{0}}{1-\tilde{c}(\hat{\theta},\hat{F})}\\{1-\phi(u,\hat{\theta})\\}+N-n_{1}-n_{0}}d\tilde{F}(u).$

Then

$F^{\star}(x)=\int_{-\infty}^{x}\frac{\frac{\rho_{1}}{{\tilde{c}}(\theta_{0},F_{0})}\phi(u,\theta_{0})+\frac{\rho_{2}}{1-{\tilde{c}}(\theta_{0},F_{0})}\\{1-\phi(u,\theta_{0})\\}+1-\rho_{1}-\rho_{0}}{\frac{\rho_{1}}{\tilde{c}(\theta^{\star},F^{\star})}\phi(u,\theta^{\star})+\frac{\rho_{0}}{1-\tilde{c}(\theta^{\star},F^{\star})}\\{1-\phi(u,\theta^{\star})\\}+1-\rho_{1}-\rho_{0}}dF_{0}(u).$

This implies that $F^{\star}(x)$ is absolutely continuous with respect to
$F_{0}(x)$ and

$\lim\limits_{n\rightarrow\infty}\frac{\hat{F}\\{x\\}}{\tilde{F}\\{x\\}}=\frac{f^{*}(x)}{f^{\star}_{0}(x)},$

where $f^{\star}$ and $f^{\star}_{0}$ are the density functions of $F^{\star}$
and $F_{0}$ respectively.

Finally, We show that $\theta^{\star}=\theta_{0}$ and $F^{\star}=F_{0}.$ Using
the fact that $l_{n}(\hat{\theta},\hat{F})\geq l_{n}(\theta_{0},\tilde{F})$,
we have that

$\displaystyle\frac{1}{N}\sum_{i=1}^{n_{1}}\log{\frac{\phi(x_{i},\hat{\theta})}{\phi(x_{i},\theta_{0})}}+\frac{1}{N}\sum_{i=n_{1}+1}^{n_{0}+n_{1}}\log{\frac{1-\phi(x_{i},\hat{\theta})}{1-\phi(x_{i},\theta_{0})}}-\frac{n_{1}}{N}\log\frac{c(\hat{\theta},\hat{F})}{\tilde{c}(\theta_{0},\tilde{F})}-$
$\displaystyle\frac{n_{0}}{N}\log\frac{1-c(\hat{\theta},\hat{F})}{1-\tilde{c}(\theta_{0},\tilde{F})}+\frac{1}{N}\sum_{i=1}^{N}\log\frac{\hat{F}\\{x_{ik}\\}}{\tilde{F}\\{x_{ik}\\}}\geq
0.$

Taking limits on both sides and defining

$A_{1}(\theta,F)=\log\frac{\phi(x,\theta)}{\tilde{c}(\theta,F)}f(x),\
A_{2}(\theta,F)=\log\frac{1-\phi(x,\theta)}{1-\tilde{c}(\theta,F)}f(x),$

and $A_{3}(F)=\log f(x)$, where $f(x)$ is the density function of $F(x)$, we
obtain that

$\rho_{1}E_{1}\left[\frac{A_{1}(\theta^{\star},F^{\star})}{A_{1}(\theta_{0},F_{0})}\right]+\rho_{0}E_{0}\left[\frac{A_{2}(\theta^{\star},F^{\star})}{A_{2}(\theta_{0},F_{0})}\right]+(1-\rho_{1}-\rho_{0})E_{X}\left[\frac{A_{3}(F^{\star})}{A_{3}(F_{0})}\right]\geq
0,$

where $\rho_{1}=\lim\limits_{N\to\infty}\lambda_{N1}$ ,
$\rho_{0}=\lim\limits_{N\to\infty}\lambda_{N0}$, $\mathrm{E}_{1}(\cdot)$
refers to the expectation taken with respect to $f_{1}(x)$,
$\mathrm{E}_{0}(\cdot)$ refers to the expectation taken with respect to
$f_{0}(x)$ and $\mathrm{E}_{X}(\cdot)$ refers to the expectation taken with
respect to the marginal density of $X$. Thus the left-hand side is the
weighted sum of three negative Kullback-Leibler distance. Therefore, the
identifiability condition C2 requires that $\theta^{\star}=\theta_{0}$ and
$F^{\star}=F_{0}$ with probability one. Furthermore, the continuity of $F_{0}$
ensures that the convergence is uniform in $\cal X$.∎

Proof of Theorem 2: The proof is based on the argument on MLE of Van der Vaart
(2000). We first introduce the set

$\mathcal{H}=\\{(v,g):v\in\mathbb{R}^{d+1},||v||\leqslant 1,g\in
BV[d],|g|\leqslant 1\\}.$

Let $\boldsymbol{P}_{n}$ be the empirical measure of $n$ i.i.d observations,
that is,
$\boldsymbol{P}_{n}\\{g(X)\\}=n^{-1}\sum_{i=1}^{n}g\left(X_{i}\right)$. Denote
$\boldsymbol{P}\\{g(X)\\}=\mathsf{E}[g(X)],$ where $g(\cdot)$ is an arbitrary
measurable function. Separate samples of fixed size are drawn from three sub-
populations in our setting. Similar to Breslow et al. (2000), we can modify
this setting so that it involves a simple random sample of size $N$ from a
biased sampling model. Specifically, consider the semiparametric biased
sampling model
$\mathbf{P}=\left\\{P_{\theta,F};\theta\in\Theta,F\in\mathbf{F}\right\\}$,
defiened on $\mathcal{Z}=\\{0,1,2\\}\times\mathcal{X}$ with density given by

$p(z,\theta,F)=p(i,x,\theta,F)=\left\\{\begin{array}[]{llr}\frac{n_{0}}{N}\frac{\\{1-\phi(x,\theta)\\}}{1-c(\theta,F)}f(x)&i=0,&\\\
\\\ \frac{n_{1}}{N}\frac{\phi(x,\theta)}{c(\theta,F)}f(x)&i=1,&\\\ \\\
\frac{N-n}{N}f(x)&i=2.&\end{array}\right.$

Furthermore, we abbreviate $l(\theta,F)$ as $\log p(z,\theta,F)$. Let
$l_{\theta}(\theta,F)$ be the derivative of $l(\theta,F)$ with respect to
$\theta$ along the direction of $\theta+\varepsilon v,$ and let
$l_{F}(\theta,F)\left[g\right]$ be the derivative of $l(\theta,F)$ with
respect to $F$ along the direction of $F_{\varepsilon}=F+\varepsilon\int gdu$.
Moreover, we denote the derivative of $l_{\theta}(\theta,F)$ with respect to
$\theta$ as $l_{\theta\theta}(\theta,F)$. Similarly, we denote the derivative
of $l_{\theta}(\theta,F)$ with respect to $F$ as $l_{\theta
F}(\theta,F)\left[g\right]$ and denote the derivative of
$l_{F}(\theta,F)\left[g\right]$ with respect to $F$ along the path
$F+\varepsilon\left(\hat{F}-F\right)$ as
$l_{FF}(\theta,F)\left[g,\hat{F}-F\right]$.

By the definition of $(\hat{\theta},\hat{F})$, we have that for any
$\left(v,g\right)\in\mathcal{H}$,

$P_{N}\left\\{l_{\theta}\left(\hat{\theta},\hat{F}\right)v^{\top}+l_{F}\left(\hat{\theta},\hat{F}\right)\left[g\right]\right\\}=0.$

Note that
$P\left\\{l_{\theta}\left(\theta_{0},F_{0}\right)v^{\top}+l_{F}\left(\theta_{0},F_{0}\right)\left[g\right]\right\\}=0$.
Thus, we obtain that

$\displaystyle\sqrt{N}\left(\boldsymbol{P}_{N}-\boldsymbol{P}\right)\left\\{l_{\theta}\left(\hat{\theta},\hat{F}\right)v^{\top}+l_{F}\left(\hat{\theta},\hat{F}\right)\left[g\right]\right\\}$
(8) $\displaystyle=$
$\displaystyle-\sqrt{N}\boldsymbol{P}\left\\{l_{\theta}\left(\hat{\theta},\hat{F}\right)v^{\top}+l_{F}\left(\hat{\theta},\hat{F}\right)\left[g\right]\right\\}+\sqrt{N}\boldsymbol{P}\left\\{l_{\theta}\left(\theta_{0},F_{0}\right)v^{\top}+l_{F}\left(\theta_{0},F_{0}\right)\left[g\right]\right\\}.$

Define

$\mathcal{A}_{1}=\left\\{l_{\theta}(\theta,F)v^{\top}+l_{F}(\theta,F)\left[g\right]:\|v\|\leq
1,\int gdx=0,F+\int\varepsilon gdu\geq 0\right\\}.$

According to the explicit expression of $l_{\theta}(\theta,F),l_{F}(\theta,F)$
and the preservation of the Donsker class under algebraic operations, we have
that $\mathcal{A}_{1}$ is a Donsker class for any positive $\epsilon$. Using
the Donsker theorem, we have that

$\sqrt{N}\left(\boldsymbol{P}_{N}-\boldsymbol{P}\right)\left\\{l_{\theta}\left(\theta_{0},F_{0}\right)v^{\top}+l_{F}\left(\theta_{0},F_{0}\right)\left[g\right]\right\\}+o_{p}(1).$
(9)

By using Talyor expansion, the right hand side of (8) can be formulated as

$\displaystyle
l_{\theta}\left(\theta_{0},F_{0}\right)v^{\top}+l_{F}\left(\theta_{0},F_{0}\right)\left[g\right]-l_{\theta}\left(\hat{\theta},\hat{F}\right)v^{\top}-l_{F}\left(\hat{\theta},\hat{F}\right)\left[g\right]$
(10) $\displaystyle=$
$\displaystyle\left(\hat{\theta}-\theta_{0}\right)l_{\theta\theta}\left(\theta_{0},F_{0}\right)v^{\top}+l_{FF}\left(\theta_{0},F_{0}\right)\left[g,\hat{F}-F_{0}\right]+\left(\hat{\theta}-\theta_{0}\right)l_{F\theta}\left(\theta_{0},F_{0}\right)\left[g\right]v^{\top}+v^{\top}l_{F\theta}\left[\hat{F}-F_{0}\right]$
$\displaystyle+o\left(\left\|\hat{\theta}-\theta_{0}\right\|+\left|\hat{F}-F_{0}\right|\right).$

Then, the right side of (8) equals to

$\sqrt{N}\left\\{\left(\hat{\theta}-\theta_{0}\right)\Omega_{\theta}\left(v,g\right)+\int\Omega_{F}\left(v,g\right)d\left(\hat{F}-F_{0}\right)\right\\}+o\left\\{\sqrt{N}\left(\left\|\hat{\theta}-\theta_{0}\right\|+\left|\hat{F}-F_{0}\right|\right)\right\\},$

where $\Omega_{\theta}\left(v,g\right)$ is the operator

$\Omega_{\theta}\left(v,g\right)=P\left\\{l_{\theta\theta}\left(\theta_{0},F_{0}\right)v^{\top}+l_{F\theta}\left(\theta_{0},F_{0}\right)\left[g\right]\right\\}$

and

$\Omega_{F}\left(v,g\right)=P\left\\{v^{\top}l_{\theta
F}\left(\theta_{0},F_{0}\right)+l_{FF}\left(\theta_{0},F_{0}\right)\left[g,\hat{F}-F_{0}\right]\right\\}.$

According to Theorem 4.7 in Rudin (1973), the information operator is
invertible if and only if it is one-to-one. To show that
$\left(\Omega_{\theta},\Omega_{F}\right)$ is injective, suppose that
$\Omega_{\theta}\left(v,g\right)=0$ and $\Omega_{F}\left(v,g\right)=0$. Then

$\Omega_{\theta}\left(v,g\right)v^{\top}+\int\Omega_{F}\left(v,g\right)gdF_{0}=0.$

The derivation of $\Omega^{\prime}s$ tell us that

$\Omega_{\theta}\left(v,g\right)v^{\top}+\int\Omega_{F}\left(v,g\right)gdF_{0}=-E\left\\{l_{\theta}\left(\theta_{0},F_{0}\right)^{T}v+l_{F}\left(\theta_{0},F_{0}\right)\left[g\right]\right\\}^{2}.$

Hence,
$l_{\theta}\left(\theta_{0},F_{0}\right)v^{\top}+l_{F}\left(\theta_{0},F_{0}\right)\left[g\right]=0$
almost surely. By condition C2, $\left(v^{\top},g\right)=0$ and
$\operatorname{ker}\left(\Omega_{\theta},\Omega_{F}\right)=0,$ then the
operator is surjective. As a result, $\left(\Omega_{\theta},\Omega_{F}\right)$
is one-to-one and thus invertible. Now with
$\left(\tilde{\Omega}_{\theta}\left(v,g\right),\tilde{\Omega}_{F}\left(v,g\right)\right)=\left(\Omega_{\theta},\Omega_{F}\right)^{-1}\left(v,g\right).$
Consequently, expression (8) can be written as

$\displaystyle\sqrt{N}\left\\{\left(\hat{\theta}-\theta_{0}\right)v^{\top}+\int
gd\left(\hat{F}-F_{0}\right)\right\\}+o\left\\{\sqrt{N}\left(\left\|\hat{\theta}-\theta_{0}\right\|+\left|\hat{F}-F_{0}\right|\right)\right\\}$
(11) $\displaystyle=$
$\displaystyle\sqrt{N}\left(\boldsymbol{P}_{N}-\boldsymbol{P}\right)\left\\{l_{\theta}\left(\theta_{0},F_{0}\right)\tilde{\Omega}_{\theta}\left(v,g\right)^{\top}+l_{F}\left(\theta_{0},F_{0}\right)\int\tilde{\Omega}_{F}\left(v,g\right)dF_{0}\right\\}.$

It follows immediately from (9) that the right hand side of (11) converges in
distribution to a zero-mean Gaussian process. By the Slutsky’s theorem, we now
only need to show that

$\sqrt{N}\left(\left\|\hat{\theta}-\theta_{0}\right\|+\left|\hat{F}-F_{0}\right|\right)=O_{p}(1)$
(12)

By definition,

$\left|\left|\hat{\theta}-\theta_{0}\right|\right|+\left|\hat{F}-F_{0}\right|=\sup\limits_{(v,g)\in\mathcal{H}}\left|v^{T}\left(\hat{\theta}-\theta_{0}\right)+\int_{X}gd\left(\hat{F}-F_{0}\right)\right|$

. Thus, (11) gives that

$\sqrt{N}\left|\left|\hat{\theta}-\theta_{0}\right|\right|+\sqrt{N}\left|\hat{F}-F_{0}\right|=O_{p}(1)+o\left(\sqrt{N}\left\|\hat{\theta}-\theta_{0}\right\|+\sqrt{N}\left|\hat{F}-F_{0}\right|\right).$

Therefore, (12) holds immediately. Now we have that

$\displaystyle\sqrt{N}\left\\{\left(\hat{\theta}-\theta_{0}\right)v^{\top}+\int
gd\left(\hat{F}-F_{0}\right)\right\\}$ (13) $\displaystyle=$
$\displaystyle\sqrt{N}\left(\boldsymbol{P}_{N}-\boldsymbol{P}\right)\left\\{l_{\theta}\left(\theta_{0},F_{0}\right)\tilde{\Omega}_{\theta}\left(v,g\right)^{\top}+l_{F}\left(\theta_{0},F_{0}\right)\int\tilde{\Omega}_{F}\left(v,g\right)dF_{0}\right\\}+o_{p}(1).$

Then $\sqrt{N}\left(\hat{\theta}-\theta_{0}\right)v^{\top}+\int
gd\left(\hat{F}_{N}-F_{0}\right)$ converges weakly to a Gaussian process.
Finally, when $g=0$ in (11), the estimator $\hat{\theta}$ is asymptotically
linear with influence function
$l_{\theta}\left(\theta_{0},F_{0}\right)\tilde{\Omega}_{\theta}^{\top}+l_{F}\left(\theta_{0},F_{0}\right)\int\tilde{\Omega}_{F}dF_{0}$,
which lies in the linear space spanned by the score functions

$\Big{\\{}l_{\theta}\left(\theta_{0},F_{0}\right)\tilde{\Omega}_{\theta}\left(v,0\right)^{\top}+l_{F}\left(\theta_{0},F_{0}\right)\int\tilde{\Omega}_{F}\left(v,0\right)dF_{0}:\left(\tilde{\Omega}_{\theta}\left(v,g\right),\tilde{\Omega}_{F}\left(v,g\right)\right)\in\mathcal{H}\Big{\\}}.$

It follows from Proposition 1 in Bickel et al. (1993) that $\hat{\theta}$ is
semiparametric efficient. This completes the proof of Theorem 2.∎

Acknowledgment

Yuanyuan Lin’s research is partially supported by the Hong Kong Research
Grants Council (Grant No.14306219 and 14306620), Direct Grants for Research,
The Chinese University of Hong Kong. Kani Chen’s research is supported by Hong
Kong General Research Fund 16212117. Wen Yu’s research is supported by the
National Natural Science Foundation of China Grants (12071088).

## References

  * [1] Ando, R. K. & Zhang, T. (2005). A framework for learning predictive structures from multiple tasks and unlabelled data. _J. Mach. Learn. Res._ 6, 1817-1853.
  * [2] Ando, R. K. & Zhang, T. (2007). Two-view feature generation model for semi-supervised learning. In _Proc. 24th Int. Conf. Machine Learning_ , Ed. Z. Ghahramani, pp. 25-32. New York: Association for Computing Machinery.
  * [3] Azriel, D., Brown, L. D., Sklar, M., Berk, R., Buja, A. & Zhao, L. (2016). Semi-supervised linear regression. _arXiv:_ 1612.02391v2.
  * [4] Belkin, M., Niyogi, P. & Sindhwani, V. (2006). Manifold regularization: A geometric framework for learning from labelled and unlabelled examples. _J. Mach. Learn. Res._ 7, 2399-2434.
  * [5] Bickel, P. J., Klaassen, C. A., Bickel, P. J., Ritov, Y., Klaassen, J., Wellner, J. A., & Ritov, Y. (1993). Effcient and adaptive estimation for semiparametric models, volume 4. Johns Hopkins University Press Baltimore.
  * [6] Blum, A. & Mitchell, T. (1995). Combining labeled and unlabeled data with co-training. _Proceedings of the eleventh annual conference on computational learning theory_ , 92-100.
  * [7] Breslow, N. E., Robins, J. M., Wellner, J. A., et al. (2000). On the semi-parametric efficiency of logistic regression under case-control sampling. Bernoulli, 6, 447-455.
  * [8] Cai, T. T. & Guo, Z. (2020). Semisupervised inference for explained variance in high dimensional linear regression and its applications. _J. R. Statist. Soc._ B 82, 391-419.
  * [9] Chakrabortty, A. & Cai, T. (2018). Efficient and adaptive linear regression in semi-supervised settings. _Ann. Statist._ 46, 1541-1572.
  * [10] Chapelle, O., Scholkopf, B., & Zien, A. (2009). Semi-supervised learning. _IEEE Transactions on Neural Networks_ 20, 542.
  * [11] Chen, K. (2001). Generalized case-cohort sampling. _J. R. Statist. Soc._ B 63, 791-809.
  * [12] Chen, K. & Lo, S-H. (1999). Case-cohort and case-control analysis with Cox’s model. _Biometrika_ 86, 755-764.
  * [13] Cheng, D., Ananthakrishnan, A. & Cai, T. (2018). Efficient and robust semi-supervised estimation of average treatment effects in electronic medical records data. _arXiv:_ 1804.00195v2.
  * [14] Chapelle, O., Schölkopf, B. & Zien, A. (2009). _Semi-Supervised Learning_. Cambridge, MA: MIT Press.
  * [15] Cosslett, S. R. (1981). Maximum likelihood estimator for choice-based samples. _Econometrica_ 49, 1289-1316.
  * [16] Cox, D. (1958). The regression analysis of binary sequences (with discussion). J. R. Statist. Soc. B. 20, 215-242.
  * [17] Farewell, V. T. (1979). Some results on the estimation of logistic models based on retrospective data. _Biometrika_ 66, 27-32.
  * [18] Fithian, W. and Hastie, T. (2014). Local case-control sampling: efficient subsampling in imbalanced data sets. _Ann. Statist._ 42, 1693-1724.
  * [19] Gronsbell, J. L. & Cai, T. (2018). Semi-supervised approaches to efficient evaluation of model prediction performance. _J. R. Statist. Soc._ B 80, 579-594.
  * [20] Johnson, R. & Zhang, T. (2008). Graph-based semi-supervised learning and spectral kernel design. _IEEE Trans. Info. Theory_ 54, 275-288.
  * [21] Manski, C. F. & Lerman, S. R. (1977). The estimation of choice probabilities from choice based samples. Econometrica 45, 1977-1988.
  * [22] Nigam, K., McCallum, A., Thrun, S. & Mitchell, T. (2000). Text classification from labelled and unlabelled documents using EM. _Mach. Learn._ 39, 103-134.
  * [23] Prentice, R. L. & Pyke, R. (1979). Logistic disease incidence models and case-control studies. _Biometrika_ 66, 403-411.
  * [24] Rudin, W. (1973). Functional Analysis. New York: McGraw-Hill.
  * [25] Scott, A. J. & Wild, C. J. (1986). Fitting logistic models under case-control or choice based sampling. _J. R. Statist. Soc._ B 48, 170-182.
  * [26] Tang, W., Lin, Y., Dai, L. & Chen, K. (2021). Combining case-control studies for identifiability and efficiency improvement in logistic regression. arXiv:2106.00939.
  * [27] van der Vaart, A. W. (2000). _Asymptotic statistics (Vol. 3)_. Cambridge University Press, Cambridge.
  * [28] Vardi, Y. (1985). Empirical distributions in selection bias models. _Ann. Statist._ , 13, 178-203.
  * [29] Wang, J. & Shen, X. (2007). Large margin semi-supervised learning. _J. Mach. Learn. Res._ 8, 1867-91.
  * [30] Wang, J., Shen, X. & Liu, Y. (2008). Probability estimation for large-margin classifiers. _Biometrika_ 95, 149-67.
  * [31] Wasserman, L. & Lafferty, J. (2007). Statistical analysis of semi-supervised regression. In _Adv. Neural Info. Proces. Syst._ , vol. 20, Ed. J. C. Platt, D. Koller, Y. Singer and S. T. Roweis, pp. 801-8. Cambridge, MA: MIT Press.
  * [32] Yao, Y., Yu, W., & Chen, K. (2017). End-point sampling. _Statist. Sinica_ 27, 415-435.
  * [33] Yarowsky, D. (1995). Unsupervised word sense disambiguation rivaling supervised methods. _33rd annual meeting of the association for computational linguistics_ , 189-196.
  * [34] Zhang, A., Brown, L. D. & Cai, T. T. (2019). Semi-supervised inference: General theory and estimation of means. _Ann. Statist._ 47, 2538-66.
  * [35] Zhang, T. & Oles, F. (2000). The value of unlabelled data for classification problems. In _Proc. 17th Int. Conf. Machine Learning_ , Ed. P. Langley, pp. 1191-8. San Francisco, CA: Morgan Kaufmann.
  * [36] Zeng, D., Yin, G., & Ibrahim, J. G. (2006). Semiparametric transformation models for survival data with a cure fraction. _Journal of the American Statistical Association_ , 101, 670-684.
  * [37] Zeng, D. & Lin, D. Y. (2007). Semiparametric transformation models with random effects for recurrent events. _J. Am. Statist. Assoc._ 102, 167-179.
  * [38] Zhu, X. & Goldberg, A. B. (2009). Introduction to semi-supervised learning. _Synth. Lect. Artif. Intell. Mach. Learn._ 3, 1-130.