* Efron, (2011) Efron, B. (2011).  Tweedie’s formula and selection bias.  Journal of the American Statistical Association, 106(496):1602–1614.
  * Eggermont and LaRiccia, (2000) Eggermont, P. P. B. and LaRiccia, V. N. (2000).  Maximum likelihood estimation of smooth monotone and unimodal densities.  The Annals of Statistics, 28(3):922–947.
  * El Karoui et al., (2013) El Karoui, N., Bean, D., Bickel, P. J., Lim, C., and Yu, B. (2013).  On robust regression with high-dimensional predictors.  Proceedings of the National Academy of Sciences, 110(36):14557–14562.
  * Faraway, (1992) Faraway, J. J. (1992).  Smoothing in adaptive estimation.  The Annals of Statistics, 20(1):414–427.
  * Folland, (1999) Folland, G. B. (1999).  Real Analysis: Modern Techniques and their Applications, volume 40.  John Wiley & Sons.
  * Groeneboom and Jongbloed, (2014) Groeneboom, P. and Jongbloed, G. (2014).  Nonparametric Estimation under Shape Constraints.  Cambridge University Press.
  * Gupta et al., (2023) Gupta, S., Lee, J. C. H., and Price, E. (2023).  Finite-sample symmetric mean estimation with Fisher information rate.  In The Thirty Sixth Annual Conference on Learning Theory, pages 4777–4830. PMLR.
  * Hampel, (1974) Hampel, F. R. (1974).  The influence curve and its role in robust estimation.  Journal of the American Statistical Association, 69(346):383–393.
  * Hampel et al., (2011) Hampel, F. R., Ronchetti, E. M., Rousseeuw, P. J., and Stahel, W. A. (2011).  Robust Statistics: The approach based on influence functions.  John Wiley & Sons.
  * Hansen, (2022) Hansen, B. E. (2022).  A modern Gauss–Markov theorem.  Econometrica, 90(3):1283–1294.
  * He and Shao, (2000) He, X. and Shao, Q.-M. (2000).  On parameters of increasing dimensions.  Journal of Multivariate Analysis, 73(1):120–135.
  * Hoerl and Kennard, (1970) Hoerl, A. E. and Kennard, R. W. (1970).  Ridge regression: biased estimation for nonorthogonal problems.  Technometrics, 12(1):55–67.
  * Huber, (1964) Huber, P. J. (1964).  Robust estimation of a location parameter.  The Annals of Mathematical Statistics, 35(1):73–101.
  * Huber, (1967) Huber, P. J. (1967).  The behavior of maximum likelihood estimates under nonstandard conditions.  In Proceedings of the fifth Berkeley Symposium on Mathematical Statistics and Probability, volume 1, pages 221–233. Berkeley, CA: University of California Press.
  * Huber and Ronchetti, (2009) Huber, P. J. and Ronchetti, E. M. (2009).  Robust Statistics.  Wiley, 2nd edition.
  * Hyvärinen, (2005) Hyvärinen, A. (2005).  Estimation of non-normalized statistical models by score matching.  Journal of Machine Learning Research, 6:695–709.
  * Hyvärinen, (2007) Hyvärinen, A. (2007).  Some extensions of score matching.  Computational Statistics & Data Analysis, 51(5):2499–2512.
  * Janková et al., (2020) Janková, J., Shah, R. D., Bühlmann, P., and Samworth, R. J. (2020).  Goodness-of-fit testing in high dimensional generalized linear models.  Journal of the Royal Statistical Society Series B: Statistical Methodology, 82(3):773–795.
  * Jin, (1990) Jin, K. (1990).  Empirical Smoothing Parameter Selection in Adaptive Estimation.  University of California, Berkeley.
  * Johnson, (2004) Johnson, O. (2004).  Information Theory and the Central Limit Theorem.  World Scientific.
  * Johnson and Barron, (2004) Johnson, O. and Barron, A. (2004).  Fisher information inequalities and the central limit theorem.  Probability Theory and Related Fields, 129:391–409.
  * Jolicoeur-Martineau et al., (2020) Jolicoeur-Martineau, A., Piché-Taillefer, R., Combes, R. T. d., and Mitliagkas, I. (2020).  Adversarial score matching and improved sampling for image generation.  arXiv preprint arXiv:2009.05475.
  * Jones, (1992) Jones, M. C. (1992).  Estimating densities, quantiles, quantile densities and density quantiles.  Annals of the Institute of Statistical Mathematics, 44:721–727.
  * Kao et al., (2023) Kao, Y.-C., Xu, M., and Zhang, C.-H. (2023).  Rate adaptive estimation of the center of a symmetric distribution.  arXiv preprint arXiv:2303.01992.
  * Koehler et al., (2022) Koehler, F., Heckett, A., and Risteski, A. (2022).  Statistical efficiency of score matching: The view from isoperimetry.  arXiv preprint arXiv:2210.00726.
  * Laha, (2021) Laha, N. (2021).  Adaptive estimation in symmetric location model under log-concavity constraint.  Electronic Journal of Statistics, 15(1):2939–3014.
  * Lederer and Oesting, (2023) Lederer, J. and Oesting, M. (2023).  Extremes in high dimensions: methods and scalable algorithms.  arXiv preprint arXiv:2303.04258.
  * Lei and Wooldridge, (2022) Lei, L. and Wooldridge, J. (2022).  What estimators are unbiased for linear models?  arXiv preprint arXiv:2212.14185.
  * Lerasle, (2019) Lerasle, M. (2019).  Selected topics on robust statistical learning theory.  arXiv preprint arxiv:1908.10761.
  * Ley and Swan, (2013) Ley, C. and Swan, Y. (2013).  Stein’s density approach and information inequalities.  Electronic Communications in Probability, 18:1–14.
  * Li et al., (2023) Li, G., Wei, Y., Chen, Y., and Chi, Y. (2023).  Towards faster non-asymptotic convergence for diffusion-based generative models.  arXiv preprint arXiv:2306.09251.
  * Loh, (2021) Loh, P.-L. (2021).  Scale calibration for high-dimensional robust regression.  Electronic Journal of Statistics, 15(2):5933–5994.
  * Lyu, (2012) Lyu, S. (2012).  Interpretation and generalization of score matching.  arXiv preprint arXiv:1205.2629.
  * Mammen, (1989) Mammen, E. (1989).  Asymptotics with increasing dimension for robust regression with applications to the bootstrap.  The Annals of Statistics, pages 382–400.
  * Mammen and Park, (1997) Mammen, E. and Park, B. U. (1997).  Optimal smoothing in adaptive location estimation.  Journal of Statistical Planning and Inference, 58(2):333–348.
  * Mardia et al., (2016) Mardia, K. V., Kent, J. T., and Laha, A. K. (2016).  Score matching estimators for directional distributions.  arXiv preprint arXiv:1604.08470.
  * Maronna and Yohai, (1981) Maronna, R. A. and Yohai, V. J. (1981).  Asymptotic behavior of general $M$-estimates for regression and scale with random carriers.  Zeitschrift für Wahrscheinlichkeitstheorie und verwandte Gebiete, 58:7–20.
  * Parisi, (1981) Parisi, G. (1981).  Correlation functions and computer simulations.  Nuclear Physics B, 180(3):378–384.
  * Parzen, (1979) Parzen, E. (1979).  Nonparametric statistical data modeling.  Journal of the American Statistical Association, 74(365):105–121.
  * Portnoy, (1985) Portnoy, S. (1985).  Asymptotic behavior of $M$ estimators of $p$ regression parameters when $p^{2}/n$ is large; II. normal approximation.  The Annals of Statistics, 13(4):1403–1417.
  * Pötscher and Preinerstorfer, (2022) Pötscher, B. M. and Preinerstorfer, D. (2022).  A modern Gauss–Markov theorem? Really?  arXiv preprint arXiv:2203.01425.
  * Roberts and Tweedie, (1996) Roberts, G. O. and Tweedie, R. L. (1996).  Exponential convergence of Langevin distributions and their discrete approximations.  Bernoulli, pages 341–363.
  * Rockafellar, (1997) Rockafellar, R. T. (1997).  Convex Analysis.  Princeton University Press.
  * Samworth and Johnson, (2004) Samworth, R. and Johnson, O. (2004).  Convergence of the empirical process in Mallows distance, with an application to bootstrap performance.  arXiv preprint math/0406603.
  * Samworth and Shah, (2024) Samworth, R. J. and Shah, R. D. (2024).  Modern Statistical Methods and Theory.  Cambridge University Press.
  * Schick, (1986) Schick, A. (1986).  On asymptotically efficient estimation in semiparametric models.  The Annals of Statistics, 14(3):1139–1151.
  * Serrin and Varberg, (1969) Serrin, J. and Varberg, D. E. (1969).  A general chain rule for derivatives and the change of variables formula for the Lebesgue integral.  The American Mathematical Monthly, 76(5):514–520.
  * Silverman, (1986) Silverman, B. W. (1986).  Density Estimation.  Chapman & Hall.
  * Song and Ermon, (2019) Song, Y. and Ermon, S. (2019).  Generative modeling by estimating gradients of the data distribution.  Advances in Neural Information Processing Systems, 32:11895–11907.
  * Song et al., (2020) Song, Y., Garg, S., Shi, J., and Ermon, S. (2020).  Sliced score matching: A scalable approach to density and score estimation.  In Uncertainty in Artificial Intelligence, pages 574–584.
  * Song and Kingma, (2021) Song, Y. and Kingma, D. P. (2021).  How to train your energy-based models.  arXiv preprint arXiv:2101.03288.
  * Song et al., (2021) Song, Y., Sohl-Dickstein, J., Kingma, D. P., Kumar, A., Ermon, S., and Poole, B. (2021).  Score-based generative modeling through stochastic differential equations.  arXiv preprint arXiv:2011.13456.
  * Sriperumbudur et al., (2017) Sriperumbudur, B., Fukumizu, K., Gretton, A., Hyvärinen, A., and Kumar, R. (2017).  Density estimation in infinite dimensional exponential families.  Journal of Machine Learning Research, 18:1–59.
  * (83) Stein, C. (1956a).  Efficient nonparametric testing and estimation.  In Proceedings of the Third Berkeley Symposium on Mathematical Statistics and Probability, volume 1, pages 187–195.
  * (84) Stein, C. (1956b).  Inadmissibility of the usual estimator for the mean of a multivariate normal distribution.  In Proceedings of the Third Berkeley Symposium on Mathematical Statistics and Probability, Volume 1: Contributions to the Theory of Statistics, volume 3, pages 197–207. University of California Press.
  * Stone, (1975) Stone, C. J. (1975).  Adaptive maximum likelihood estimators of a location parameter.  The Annals of Statistics, 3(2):267–284.
  * van der Vaart, (1998) van der Vaart, A. W. (1998).  Asymptotic Statistics.  Cambridge University Press.
  * van Eeden, (1970) van Eeden, C. (1970).  Efficiency-robust estimation of location.  The Annals of Mathematical Statistics, 41(1):172–181.
  * Vincent, (2011) Vincent, P. (2011).  A connection between score matching and denoising autoencoders.  Neural Computation, 23(7):1661–1674.
  * Wainwright, (2019) Wainwright, M. J. (2019).  High-Dimensional Statistics: A Non-Asymptotic Viewpoint, volume 48.  Cambridge University Press.
  * Yang et al., (2019) Yang, Y., Martin, R., and Bondell, H. (2019).  Variational approximations using Fisher divergence.  arXiv preprint arXiv:1905.05284.
  * Yohai and Maronna, (1979) Yohai, V. J. and Maronna, R. A. (1979).  Asymptotic behavior of $M$-estimators for the linear model.  The Annals of Statistics, pages 258–268.
  * Young and Shah, (2023) Young, E. H. and Shah, R. D. (2023).  Sandwich boosting for accurate estimation in partially linear models for grouped data.  arXiv preprint arXiv:2307.11401.
  * Yu et al., (2020) Yu, M., Gupta, V., and Kolar, M. (2020).  Simultaneous inference for pairwise graphical models with generalized score matching.  Journal of Machine Learning Research, 21(91):1–51.
  * Yu et al., (2022) Yu, S., Drton, M., and Shojaie, A. (2022).  Generalized score matching for general domains.  Information and Inference: A Journal of the IMA, 11(2):739–780.
  * Zou and Yuan, (2008) Zou, H. and Yuan, M. (2008).  Composite quantile regression and the oracle model selection theory.  The Annals of Statistics, 36(3):1108–1126.

##  6 Appendix

###  6.1 Proofs for Section 2

Throughout this subsection, we work in the setting of Lemmas 1 and 5.
Moreover, define $Q_{0}(u):=\sup\\{z\in[-\infty,\infty]:F_{0}(z)\leq u\\}$ for
$u\in[0,1]$. Then $z_{\min}:=\inf(\operatorname{supp}p_{0})=Q_{0}(0)$ and
$z_{\max}:=\sup(\operatorname{supp}p_{0})=F_{0}^{-1}(1)$, and
$\mathcal{S}_{0}=\\{z\in\mathbb{R}:F_{0}(z)\in(0,1)\\}$.

###### Proof of Lemma 1.

If $F_{0}^{-1}(v)<Q_{0}(v)$ for some $v\in[0,1]$, then because
$p_{0}(\pm\infty)=0$ and $p_{0},F_{0}$ are both continuous, we have
$p_{0}(z)=0$ for all
$z\in[F_{0}^{-1}(v),Q_{0}(v)]=\\{z\in[-\infty,\infty]:F_{0}(z)=v\\}$.
Therefore, $p_{0}(z)=J_{0}\bigl{(}F_{0}(z)\bigr{)}$ for all $z\in\mathbb{R}$
and
$J_{0}(v)=p_{0}\bigl{(}F_{0}^{-1}(v)\bigr{)}=p_{0}\bigl{(}Q_{0}(v)\bigr{)}$
for all $v\in[0,1]$, with $J_{0}(0)=J_{0}(1)=0$. Since $\lim_{u\nearrow
v}F_{0}^{-1}(u)=F_{0}^{-1}(v)$ and $\lim_{u\searrow v}F_{0}^{-1}(u)=Q_{0}(v)$
for all $v\in[0,1]$, it follows from the continuity of $p_{0}$ on
$[-\infty,\infty]$ that $J_{0}$ is continuous on $[0,1]$. Moreover, the least
concave majorant $\hat{J}_{0}$ satisfies
$\sup_{u\in[0,1]}\hat{J}_{0}(u)=\sup_{u\in[0,1]}J_{0}(u)<\infty$ and

$\hat{J}_{0}(0)=J_{0}(0)=0=J_{0}(1)=\hat{J}_{0}(1).$ (42)

In particular, $\hat{J}_{0}$ is concave and bounded, so it is also continuous
on $[0,1]$. By Rockafellar, (1997, Theorem 24.1), the right derivative
$\hat{J}_{0}^{(\mathrm{R})}$ is decreasing and right-continuous on $[0,1)$,
with $\hat{J}_{0}^{(\mathrm{R})}(u)\in\mathbb{R}$ for all $u\in(0,1)$. Thus,
$\psi_{0}^{*}$ is finite-valued on $\mathcal{S}_{0}$. If $z_{n}\searrow z$ for
some $z\in\mathbb{R}$, then either $u_{n}:=F_{0}(z_{n})\searrow F_{0}(z)=:u<1$
or $u_{n}=u=1$ for all $n$. In both cases,
$\psi_{0}^{*}(z_{n})=\hat{J}_{0}^{(\mathrm{R})}(u_{n})\nearrow\hat{J}_{0}^{(\mathrm{R})}(u)=\psi_{0}^{*}(z)$,
so $\psi_{0}^{*}$ is right-continuous as a function from $\mathbb{R}$ to
$[-\infty,\infty]$. Since $F_{0}$ is increasing, $\psi_{0}^{*}$ is decreasing.

We have $z_{\min}=\inf\\{z\in\mathbb{R}:F_{0}(z)>0\\}$ and $\log
F_{0}(z)\searrow-\infty$ as $z\searrow z_{\min}$. Thus, if $z_{\min}>-\infty$,
then

$\limsup_{u\searrow 0}\frac{J_{0}(u)}{u}=\limsup_{z\searrow
z_{\min}}\frac{J_{0}\bigl{(}F_{0}(z)\bigr{)}}{F_{0}(z)}=\limsup_{z\searrow
z_{\min}}\frac{p_{0}(z)}{F_{0}(z)}=\limsup_{z\searrow z_{\min}}\,(\log
F_{0})^{\prime}(z)=\infty,$ (43)

where the final equality follows from the mean value theorem. Together with
Lemma 46, this implies that
$\psi_{0}^{*}(z)=\hat{J}_{0}^{(\mathrm{R})}(0)=\infty$ for all $z\leq
z_{\min}$. Similarly, if $z_{\max}<\infty$, then
$\psi_{0}^{*}(z)=\hat{J}_{0}^{(\mathrm{R})}(1)=-\infty$ for all $z\geq
z_{\max}$. Therefore, in all cases, $\psi_{0}^{*}(z)\in\mathbb{R}$ if and only
if $z\in\mathcal{S}_{0}$. ∎

The next lemma characterises precisely the class of density quantile functions
of uniformly continuous densities on $\mathbb{R}$, and shows explicitly how to
recover a density from a continuous density quantile function that is strictly
positive on $(0,1)$.

######  Lemma 19.

A function $J\colon[0,1]\to[0,\infty)$ is the density quantile function of a
uniformly continuous density on $\mathbb{R}$ if and only if

$\text{ $J$ is continuous on $[0,1]$ with $J(0)=J(1)=0$ and
}Q_{J}(u):=\int_{1/2}^{u}\frac{1}{J}<\infty$ (44)

for all $u\in(0,1)$. In this case, $Q_{J}$ is a strictly increasing,
continuously differentiable bijection from $(0,1)$ to
$\bigl{(}Q_{J}(0),Q_{J}(1)\bigr{)}$, and the function
$p_{J}\colon\mathbb{R}\to\mathbb{R}$ given by

$p_{J}(z):=\begin{cases}(Q_{J}^{-1})^{\prime}(z)\;&\text{for
}z\in\bigl{(}Q_{J}(0),Q_{J}(1)\bigr{)}\\\ 0\;&\text{otherwise}\end{cases}$

is a uniformly continuous density with corresponding density quantile function
$J$. If in addition $J>0$ on $(0,1)$, then a density $p_{0}$ has density
quantile function $J$ if and only if $p_{0}(\cdot)=p_{J}(\cdot-\mu)$ for some
$\mu\in\mathbb{R}$.

###### Proof.

If $p_{0}$ is a uniformly continuous density with density quantile function
$J_{0}=p_{0}\circ F_{0}^{-1}$, then Lemma 1 shows that $J_{0}$ is continuous
with $J_{0}(0)=J_{0}(1)=0$. Introducing $U\sim U(0,1)$ and
$Z:=F_{0}^{-1}(U)\sim P_{0}$, we have $p_{0}(Z)=J_{0}(U)$ and
$\mathbb{P}\bigl{(}J_{0}(U)=0\bigr{)}=\mathbb{P}\bigl{(}p_{0}(Z)=0\bigr{)}=0$.
Moreover, since $F_{0}$ is continuous, $(F_{0}\circ F_{0}^{-1})(u)=u$ for all
$u\in(0,1)$, so $U=F_{0}(Z)$. Therefore,

$\int_{1/2}^{u}\frac{1}{J_{0}}=\mathbb{E}\biggl{(}\frac{1}{J_{0}(U)}\mathbbm{1}_{\\{U\in[1/2,u)\\}}\biggr{)}=\mathbb{E}\biggl{(}\frac{1}{p_{0}(Z)}\mathbbm{1}_{\\{Z\in[F_{0}^{-1}(1/2),F_{0}^{-1}(u))\\}}\biggr{)}\leq
F_{0}^{-1}(u)-F_{0}^{-1}\Bigl{(}\frac{1}{2}\Bigr{)}<\infty$ (45)

for $u\in[1/2,1)$, with equality if and only if $p_{0}(z)>0$ Lebesgue almost
everywhere on $\bigl{[}F_{0}^{-1}(1/2),F_{0}^{-1}(u)\bigr{)}$. Similarly, for
$u\in(0,1/2]$, (45) remains true, with an analogous equality condition.

Conversely, if $J\colon[0,1]\to\mathbb{R}$ satisfies (44), then $Q_{J}$ is
strictly increasing with $Q_{J}^{\prime}(u)=1/J(u)$ for all $u\in(0,1)$. Thus,
$Q_{J}$ is a continuously differentiable bijection from $(0,1)$ to
$\bigl{(}Q_{J}(0),Q_{J}(1)\bigr{)}$, so by the inverse function theorem,
$F_{J}:=Q_{J}^{-1}\colon\bigl{(}Q_{J}(0),Q_{J}(1)\bigr{)}\to(0,1)$ is a well-
defined, continuously differentiable bijection with derivative
$p_{J}:=F_{J}^{\prime}=1/(Q_{J}^{\prime}\circ F_{J})=J\circ F_{J}$ satisfying
$J=p_{J}\circ F_{J}^{-1}$ on $(0,1)$. We have $\lim_{z\searrow
Q_{J}(0)}F_{J}(z)=0$ and $\lim_{z\nearrow Q_{J}(1)}F_{J}(z)=1$, while
$\lim_{z\searrow Q_{J}(0)}p_{J}(z)=\lim_{z\nearrow Q_{J}(1)}p_{J}(z)=0$ since
$J$ is continuous at $0$ and $1$. Therefore, setting $p_{J}(z)=0$ for
$z\in\mathbb{R}\setminus\bigl{(}Q_{J}(0),Q_{J}(1)\bigr{)}$, we conclude that
$p_{J}$ is a uniformly continuous density on $\mathbb{R}$ with corresponding
distribution function $F_{J}$, quantile function $Q_{J}$ and density quantile
function $J$. Moreover, for every $\mu\in\mathbb{R}$, the density
$p_{J}(\cdot-\mu)$ also has density quantile function $J$.

Finally, if $p_{0}$ is a uniformly continuous density with density quantile
function $J=p_{0}\circ F_{0}^{-1}>0$ on $(0,1)$, then for every $u\in(0,1)$,
equality holds in (44) and hence

$Q_{J}(u)=\int_{1/2}^{u}\frac{1}{J}=F_{0}^{-1}(u)-F_{0}^{-1}\Bigl{(}\frac{1}{2}\Bigr{)}.$

Therefore, letting $\mu:=F_{0}^{-1}(1/2)$, we deduce that
$p_{0}(\cdot)=p_{J}(\cdot-\mu)$, which completes the proof. ∎

######  Lemma 20.

Let $\mathcal{T}$ be as in Proposition 9(a). If $t\in\mathcal{T}$, then
$p_{0}(t)=\hat{J}_{0}\bigl{(}F_{0}(t)\bigr{)}>0$.

###### Proof.

Since $\hat{J}_{0}$ is a non-negative concave function, we must have
$\hat{J}_{0}>0$ on $(0,1)$, as otherwise $J_{0}=\hat{J}_{0}=0$ and hence
$p_{0}=J_{0}\circ F_{0}=0$ on $\mathbb{R}$ by Lemma 1, which is a
contradiction. Moreover, $\psi_{0}^{*}(t)\in\mathbb{R}$, so again by Lemma 1,
$t\in\mathcal{S}_{0}$. Thus, $v:=F_{0}(t)\in(0,1)$ and $p_{0}(t)=(J_{0}\circ
F_{0})(t)=J_{0}(v)$. Suppose for a contradiction that
$J_{0}(v)<\hat{J}_{0}(v)$. Then by Lemma 45 and the fact that $J_{0}$ and
$\hat{J}_{0}$ are continuous at $v$, there exists $\delta>0$ such that
$\hat{J}_{0}$ is affine on $[v-\delta,v+\delta]$. Thus,
$\hat{J}_{0}^{(\mathrm{R})}$ is constant on $[v-\delta,v+\delta)$, so
$\psi_{0}^{*}=\hat{J}_{0}^{(\mathrm{R})}\circ F_{0}$ is constant on the open
interval $\bigl{(}Q_{0}(v-\delta),F_{0}^{-1}(v+\delta)\bigr{)}$, which
contains $t$. This contradicts the definition of $\mathcal{T}$, so
$p_{0}(t)=J_{0}(v)=\hat{J}_{0}(v)>0$, as required. ∎

######  Lemma 21.

If $\psi\in\Psi_{\downarrow}(p_{0})$ satisfies $\psi=\psi_{0}^{*}$
$P_{0}$-almost everywhere, then in fact $\psi=\psi_{0}^{*}$ on $\mathbb{R}$.

###### Proof.

First consider $z\in\mathcal{S}_{0}$, let $u:=F_{0}(z)\in(0,1)$ and let
$u_{1}:=\inf\bigl{\\{}v\in[0,1]:\hat{J}_{0}^{(\mathrm{R})}(v)=\hat{J}_{0}^{(\mathrm{R})}(u)\bigr{\\}}$.
Then $\hat{J}_{0}^{(\mathrm{R})}(u_{1})=\hat{J}_{0}^{(\mathrm{R})}(u)$ by the
right-continuity of $\hat{J}_{0}^{(\mathrm{R})}$. Moreover, define
$z_{1}:=F_{0}^{-1}(u_{1})$ and $z_{2}:=Q_{0}(u)$, so that $z_{1}\leq z\leq
z_{2}$ and

$F_{0}(z_{1}-\delta)<u_{1}=F_{0}(z_{1})\leq
F_{0}(z_{2})=u<F_{0}(z_{2}+\delta)$

for all $\delta>0$. Then
$\psi_{0}^{*}(z_{1}-\delta)=\hat{J}_{0}^{(\mathrm{R})}\bigl{(}F_{0}(z_{1}-\delta)\bigr{)}>\hat{J}_{0}^{(\mathrm{R})}(u_{1})=\psi_{0}^{*}(z_{1})$
for all such $\delta$, so $z_{1}\in\mathcal{T}$ and hence $p_{0}(z_{1})>0$ by
Lemma 20. It follows that $P_{0}\bigl{(}[z_{j},z_{j}+\delta)\bigr{)}>0$ for
$j\in\\{1,2\\}$, so because $\psi$ and $\psi_{0}^{*}$ are decreasing, right-
continuous functions that agree $P_{0}$-almost everywhere,

$\psi(z)\leq\psi(z_{1})=\psi_{0}^{*}(z_{1})=\hat{J}_{0}^{(\mathrm{R})}(u_{1})=\hat{J}_{0}^{(\mathrm{R})}(u)=\psi_{0}^{*}(z_{2})=\psi(z_{2})\leq\psi(z).$

Thus, $\psi(z)=\psi_{0}^{*}(z)$ for all $z\in\mathcal{S}_{0}$. Together with
Lemma 1, this implies that $\lim_{z\searrow z_{\min}}\psi(z)=\lim_{z\searrow
z_{\min}}\psi_{0}^{*}(z)=\infty$ and $\lim_{z\nearrow
z_{\max}}\psi(z)=\lim_{z\nearrow z_{\max}}\psi_{0}^{*}(z)=-\infty$, so
$\psi=\psi_{0}^{*}=\infty$ on $(-\infty,z_{\min}]$ and
$\psi=\psi_{0}^{*}=-\infty$ on $[z_{\max},\infty)$. Thus, $\psi=\psi_{0}^{*}$
on $\mathbb{R}$, as required. ∎

###### Proof of Theorem 2.

Let $\chi_{t}:=\mathbbm{1}_{(-\infty,t)}\in\Psi_{\downarrow}(p_{0})$ for
$t\in(\infty,\infty]$. Then, letting $U\sim U(0,1)$, we have $(F_{0}\circ
F_{0}^{-1})(U)=U$, so

$\displaystyle\int_{\mathbb{R}}\psi_{0}^{*}\chi_{t}\,dP_{0}=\int_{(-\infty,t)}(\hat{J}_{0}^{(\mathrm{R})}\circ
F_{0})\,dP_{0}$
$\displaystyle=\mathbb{E}\bigl{\\{}\bigl{(}\hat{J}_{0}^{(\mathrm{R})}\circ
F_{0}\circ F_{0}^{-1}\bigr{)}(U)\,\mathbbm{1}_{\\{F_{0}^{-1}(U)\leq
t\\}}\bigr{\\}}$
$\displaystyle=\mathbb{E}\bigl{\\{}\hat{J}_{0}^{(\mathrm{R})}(U)\,\mathbbm{1}_{\\{U\leq
F_{0}(t)\\}}\bigr{\\}}=\int_{0}^{F_{0}(t)}\hat{J}_{0}^{(\mathrm{R})}$
$\displaystyle=\hat{J}_{0}\bigl{(}F_{0}(t)\bigr{)}-\hat{J}_{0}(0)\geq
J_{0}\bigl{(}F_{0}(t)\bigr{)}=p_{0}(t),$

where the last two equalities are due to Rockafellar, (1997, Corollary 24.2.1)
and Lemma 1 respectively. By (42) and Lemma 20, equality holds throughout in
the final line if $F_{0}(t)\in\\{0,1\\}$ or $t\in\mathcal{T}$, so

$\int_{\mathbb{R}}\psi_{0}^{*}\chi_{t}\,dP_{0}\,\begin{cases}\geq
p_{0}(t)&\;\text{for all }t\in(\infty,\infty]\\\ =p_{0}(t)&\;\text{for all
}t\in(-\infty,z_{\min}]\cup[z_{\max},\infty]\cup\mathcal{T}.\end{cases}$ (46)

(a) In particular, taking $t=\infty$, we have
$\int_{\mathbb{R}}\psi_{0}^{*}\,dP_{0}=0$.

(b, c) Case 1: Suppose that
$i^{*}(p_{0})=\int_{\mathbb{R}}(\psi_{0}^{*})^{2}\,dP_{0}<\infty$. Then
$\psi_{0}^{*}\in\Psi_{\downarrow}(p_{0})$ since it is decreasing and right-
continuous. For measurable functions $\psi$ on $\mathbb{R}$, we write
$\|{\psi}\|\equiv\|{\psi}\|_{L^{2}(P_{0})}:=(\int_{\mathbb{R}}\psi^{2}\,dP_{0})^{1/2}$,
and for $\psi_{1},\psi_{2}\in L^{2}(P_{0})$, denote by
$\langle\psi_{1},\psi_{2}\rangle\equiv\langle\psi_{1},\psi_{2}\rangle_{L^{2}(P_{0})}:=\int_{\mathbb{R}}\psi_{1}\psi_{2}\,dP_{0}$
their $L^{2}(P_{0})$ inner product. Recalling that
$D_{p_{0}}(\cdot\,,\lambda)$ is a convex (quadratic) function on the convex
cone $\Psi_{\downarrow}(p_{0})$, we will deduce from (46) that $\psi_{0}^{*}$
satisfies the first-order stationarity condition

$\displaystyle-\int_{\mathcal{S}_{0}}p_{0}\,d\psi\leq\int_{\mathbb{R}}\psi_{0}^{*}\psi\,dP_{0}=\langle\psi_{0}^{*},\psi\rangle\in\mathbb{R}$
(47)

for all $\psi\in\Psi_{\downarrow}(p_{0})$, with equality when
$\psi=\psi_{0}^{*}$. To see this, fix $t_{0}\in\mathcal{S}_{0}$ and define
$g(z,t):=\mathbbm{1}_{\\{z<t\leq t_{0}\\}}-\mathbbm{1}_{\\{t_{0}<t\leq z\\}}$
for $z,t\in\mathbb{R}$. Then
$g(\cdot,t)=\chi_{t}(\cdot)-\mathbbm{1}_{\\{t>t_{0}\\}}$ for all
$t\in\mathbb{R}$ and
$\psi(t_{0})-\psi(z)=\int_{\mathcal{S}_{0}}g(z,t)\,d\psi(t)$ for all
$z\in\mathcal{S}_{0}$. Since
$\int_{\mathbb{R}}|\psi_{0}^{*}||\psi(t_{0})-\psi|\,dP_{0}\leq\|{\psi_{0}^{*}}\|\bigl{(}|\psi(t_{0})|+\|{\psi}\|\bigr{)}<\infty$
and $\int_{\mathbb{R}}\psi_{0}^{*}\,dP_{0}=0$ by (46), we can apply Fubini’s
theorem to obtain

$\displaystyle\int_{\mathbb{R}}\psi_{0}^{*}\psi\,dP_{0}$
$\displaystyle=\psi(t_{0})\int_{\mathbb{R}}\psi_{0}^{*}\,dP_{0}-\int_{\mathbb{R}}\psi_{0}^{*}(z)\int_{\mathcal{S}_{0}}g(z,t)\,d\psi(t)\,dP_{0}(z)=-\int_{\mathcal{S}_{0}}\int_{\mathbb{R}}\psi_{0}^{*}(z)\,g(z,t)\,dP_{0}(z)\,d\psi(t).$
(48)

On the right-hand side of (48), the outer Lebesgue–Stieltjes integral is with
respect to a negative measure, so it follows from (46) that

$\displaystyle\int_{\mathcal{S}_{0}}\int_{\mathbb{R}}\psi_{0}^{*}(z)\,g(z,t)\,dP_{0}(z)\,d\psi(t)$
$\displaystyle=\int_{(z_{\min},t_{0}]}\biggl{(}\int_{\mathbb{R}}\psi_{0}^{*}\chi_{t}\,dP_{0}\biggr{)}\,d\psi(t)+\int_{(t_{0},z_{\max})}\biggl{(}\int_{\mathbb{R}}\psi_{0}^{*}(\chi_{t}-1)\,dP_{0}\biggr{)}\,d\psi(t)$
$\displaystyle\leq\int_{(z_{\min},t_{0}]}p_{0}(t)\,d\psi(t)+\int_{(t_{0},z_{\max})}p_{0}(t)\,d\psi(t)=\int_{\mathcal{S}_{0}}p_{0}\,d\psi.$
(49)

Together with (48), this proves the inequality (47) for all
$\psi\in\Psi_{\downarrow}(p_{0})$. It remains to show that (47) holds with
equality when $\psi=\psi_{0}^{*}$. Let $\nu^{*}$ for the Lebesgue–Stieltjes
measure induced by $\psi_{0}^{*}$, so that the set $\mathcal{T}$ in Lemma 20
is the support of $\nu^{*}$. Then $\mathcal{T}^{c}$ is an open set with
$\nu^{*}(\mathcal{T}^{c})=0$, so
$\psi_{0}^{*}(t_{0})-\psi_{0}^{*}(z)=\int_{\mathcal{T}}g(z,t)\,d\psi_{0}^{*}(t)$
for all $z\in\mathbb{R}$. Thus, (48) holds for $\psi=\psi_{0}^{*}$ when the
Lebesgue–Stieltjes integrals (with respect to $\nu^{*}$) are restricted to
$\mathcal{S}_{0}\cap\mathcal{T}$. Moreover, (49) is still valid if we
intersect the domains of all outer integrals with $\mathcal{T}$, again because
$\nu^{*}(\mathcal{T}^{c})=0$. We established that (46) is an equality for all
$t\in\mathcal{T}$, so (49) and hence (47) hold with equality when
$\psi=\psi_{0}^{*}$.

For $\psi\in\Psi_{\downarrow}(p_{0})$, it follows from (47) that
$\int_{\mathcal{S}_{0}}p_{0}\,d\psi>-\infty$ and hence
$D_{p_{0}}(\psi,\lambda)\in\mathbb{R}$ for every $\lambda>0$, so

$\displaystyle D_{p_{0}}(\psi,\lambda)$
$\displaystyle=\int_{\mathbb{R}}(\psi-\lambda\psi_{0}^{*})^{2}\,dP_{0}+\int_{\mathbb{R}}(\lambda\psi_{0}^{*})^{2}\,dP_{0}+2\lambda\,\biggl{\\{}\int_{\mathbb{R}}\psi_{0}^{*}(\psi-\lambda\psi_{0}^{*})\,dP_{0}+\int_{\mathcal{S}_{0}}p_{0}\,d\psi\biggr{\\}}$
$\displaystyle\geq\int_{\mathbb{R}}(\psi-\lambda\psi_{0}^{*})^{2}\,dP_{0}-\lambda^{2}\int_{\mathbb{R}}(\psi_{0}^{*})^{2}\,dP_{0}$
(50)
$\displaystyle=\int_{\mathbb{R}}(\psi-\lambda\psi_{0}^{*})^{2}\,dP_{0}+\int_{\mathbb{R}}(\lambda\psi_{0}^{*})^{2}\,dP_{0}+2\lambda^{2}\int_{\mathcal{S}_{0}}p_{0}\,d\psi_{0}^{*}$
$\displaystyle=\int_{\mathbb{R}}(\psi-\lambda\psi_{0}^{*})^{2}\,dP_{0}+D_{p_{0}}(\lambda\psi_{0}^{*},\lambda)\geq
D_{p_{0}}(\lambda\psi_{0}^{*},\lambda).$

Thus,
$\psi\in\operatorname*{argmin}_{\psi\in\Psi_{\downarrow}(p_{0})}D_{p_{0}}(\psi,\lambda)$
if and only if $\psi=\lambda\psi_{0}^{*}$ $P_{0}$-almost everywhere. By Lemma
21, this holds if and only if $\psi=\lambda\psi_{0}^{*}$ on $\mathbb{R}$,
which proves the uniqueness assertion in (b).

Moreover, since $\hat{J}_{0}^{(\mathrm{R})}(0)=\lim_{u\searrow
0}\hat{J}_{0}^{(\mathrm{R})}(u)>0$ and $F_{0}(z)>0$ for all
$z>Q_{0}(0)=z_{\min}$, we have
$\int_{(z_{\min},z)}(\psi_{0}^{*})^{2}\,dP_{0}=\int_{(z_{\min},z)}(\hat{J}_{0}^{(\mathrm{R})}\circ
F_{0})^{2}\,dP_{0}>0$ for all $z>z_{\min}$, so
$i^{*}(p_{0})=\|{\psi_{0}^{*}}\|^{2}>0$. Now for
$\psi\in\Psi_{\downarrow}(p_{0})$ such that
$\int_{\mathbb{R}}\psi^{2}\,dP_{0}>0$, we have
$-\int_{\mathcal{S}_{0}}p_{0}\,d\psi\geq 0$ since $\psi$ is decreasing. It
follows from (47) and the Cauchy–Schwarz inequality that

$\displaystyle
V_{p_{0}}(\psi)=\frac{\int_{\mathbb{R}}\psi^{2}\,dP_{0}}{\bigl{(}\int_{\mathcal{S}_{0}}p_{0}\,d\psi\bigr{)}^{2}}\geq\frac{\|{\psi}\|^{2}}{\langle\psi_{0}^{*},\psi\rangle^{2}}\geq\frac{1}{\|{\psi_{0}^{*}}\|^{2}}=\frac{1}{i^{*}(p_{0})}\in(0,\infty),$

with equality if and only if $\psi=\lambda\psi_{0}^{*}$ $P_{0}$-almost
everywhere for some $\lambda>0$. By Lemma 21, this holds if and only if
$\psi=\lambda\psi_{0}^{*}$ on $\mathbb{R}$, so the proof of _(c)_ is complete.

Case 2: Suppose instead that
$i^{*}(p_{0})=\int_{\mathbb{R}}(\psi_{0}^{*})^{2}\,dP_{0}=\infty$. For each
$n\in\mathbb{N}$, define $\psi_{n}^{*}:=(\psi_{0}^{*}\wedge n)\vee(-n)$. This
is bounded, decreasing and right-continuous, so
$\psi_{n}^{*}\in\Psi_{\downarrow}(p_{0})$, and we claim that

$\displaystyle-\int_{\mathcal{S}_{0}}p_{0}\,d\psi_{n}^{*}=\int_{\mathbb{R}}\psi_{0}^{*}\psi_{n}^{*}\,dP_{0}\in\mathbb{R}.$
(51)

Indeed, by the quantile transform used to establish (46),
$|\int_{\mathbb{R}}\psi_{0}^{*}\psi_{n}^{*}\,dP_{0}|\leq
n\int_{\mathbb{R}}|\psi_{0}^{*}|\,dP_{0}=n\int_{0}^{1}|\hat{J}_{0}^{(\mathrm{R})}|\leq
2n\sup_{u\in[0,1]}\hat{J}_{0}(u)<\infty$ since $\hat{J}_{0}$ is concave and
bounded. Therefore, we may apply Fubini’s theorem as above to see that (48)
above holds when $\psi=\psi_{n}^{*}$. Furthermore, writing $\nu_{n}^{*}$ for
the Lebesgue–Stieltjes measure associated with $\psi_{n}^{*}$, we have
$\nu_{n}^{*}(\mathcal{T}^{c})=\nu^{*}(\mathcal{T}^{c}\cap I_{n})=0$, where
$I_{n}:=\\{z\in\mathbb{R}:\psi_{0}^{*}(z)\in(-n,n]\\}$. Consequently, by
intersecting the domains of all outer integrals in (49) with $\mathcal{T}$ and
arguing as above, we deduce that (49) holds with equality for
$\psi=\psi_{n}^{*}$. This yields (51) for every $n\in\mathbb{N}$. For
$\lambda>0$, it then follows by the monotone convergence theorem that

$\displaystyle
D_{p_{0}}(\lambda\psi_{n}^{*},\lambda)=\lambda^{2}\,\biggl{(}\int_{\mathbb{R}}(\psi_{n}^{*})^{2}\,dP_{0}-2\int_{\mathbb{R}}\psi_{0}^{*}\psi_{n}^{*}\,dP_{0}\biggr{)}\leq-\lambda^{2}\int_{\mathbb{R}}(\psi_{n}^{*})^{2}\,dP_{0}$
$\displaystyle=-\lambda^{2}\int_{\mathbb{R}}\bigl{(}(\psi_{0}^{*})^{2}\wedge
n^{2}\bigr{)}\,dP_{0}$
$\displaystyle\searrow-\lambda^{2}\int_{\mathbb{R}}(\psi_{0}^{*})^{2}\,dP_{0}=-\infty$

as $n\to\infty$. Thus,
$\inf_{\psi\in\Psi_{\downarrow}(p_{0})}D_{p_{0}}(\psi,\lambda)=-\infty$.

Together with (50), this shows that
$\inf_{\psi\in\Psi_{\downarrow}(p_{0})}D_{p_{0}}(\psi,\lambda)=-\lambda^{2}i^{*}(p_{0})$
in both cases, which completes the proof of _(b)_.

(d) Suppose that $p_{0}$ is absolutely continuous on $\mathbb{R}$ with score
function $\psi_{0}=p_{0}^{\prime}/p_{0}$. Recall that $(F_{0}\circ
F_{0}^{-1})(u)=u$ for all $u\in(0,1)$, so that letting $U\sim U(0,1)$ and
$Z=F_{0}^{-1}(U)\sim P_{0}$, we have

$\displaystyle\int_{0}^{u}\psi_{0}\circ F_{0}^{-1}$
$\displaystyle=\mathbb{E}\bigl{(}(\psi_{0}\circ
F_{0}^{-1})(U)\mathbbm{1}_{\\{U\leq(F_{0}\circ F_{0}^{-1})(u)\\}}\bigr{)}$
(52) $\displaystyle=\mathbb{E}\bigl{(}\psi_{0}(Z)\mathbbm{1}_{\\{Z\leq
F_{0}^{-1}(u)\\}}\bigr{)}=\int_{(-\infty,F_{0}^{-1}(u)]}\psi_{0}\,dP_{0}=\int_{-\infty}^{F_{0}^{-1}(u)}p_{0}^{\prime}=(p_{0}\circ
F_{0}^{-1})(u)=J_{0}(u)$

for all $u\in(0,1)$. This yields the representation (17) of $\psi_{0}^{*}$.

We assume henceforth that
$i(p_{0})=\int_{\mathbb{R}}\psi_{0}^{2}\,dP_{0}=\|{\psi_{0}}\|^{2}<\infty$,
since otherwise the inequality $i^{*}(p_{0})\leq i(p_{0})$ holds trivially.
Then
$\int_{\mathbb{R}}|\psi\psi_{0}|\,dP_{0}=\langle|\psi|,|\psi_{0}|\rangle\leq\|{\psi}\|\,\|{\psi_{0}}\|<\infty$
for all $\psi\in\Psi_{\downarrow}(p_{0})$. In particular,
$\int_{\mathbb{R}}|p_{0}^{\prime}|=\int_{\mathbb{R}}|\psi_{0}|\,dP_{0}<\infty$,
so by the dominated convergence theorem,

$\int_{\mathbb{R}}\psi_{0}\,dP_{0}=\int_{\mathbb{R}}p_{0}^{\prime}=\lim_{t\to\infty}\int_{-t}^{t}p_{0}^{\prime}=\lim_{t\to\infty}\bigl{(}p_{0}(t)-p_{0}(-t)\bigr{)}=p_{0}(\infty)-p_{0}(-\infty)=0.$

Recalling that $g(z,t)=\mathbbm{1}_{\\{z<t\leq
t_{0}\\}}-\mathbbm{1}_{\\{t_{0}<t\leq z\\}}$ for $z,t\in\mathbb{R}$ and
$\psi(t_{0})-\psi(z)=\int_{\mathcal{S}_{0}}g(z,t)\,d\psi(t)$ for all
$z\in\mathcal{S}_{0}$, we similarly have
$\int_{\mathbb{R}}p_{0}^{\prime}(z)g(z,t)\,dz=-\bigl{(}p_{0}(\infty)-p_{0}(t)\bigr{)}=p_{0}(t)$
for all $t>t_{0}$ and
$\int_{\mathbb{R}}p_{0}^{\prime}(z)g(z,t)\,dz=p_{0}(t)-p_{0}(-\infty)=p_{0}(t)$
for all $t\leq t_{0}$. For $\psi\in\Psi_{\downarrow}(p_{0})$, it then follows
from Fubini’s theorem that

$\displaystyle\int_{\mathbb{R}}\psi\psi_{0}\,dP_{0}=\psi(t_{0})\int_{\mathbb{R}}\psi_{0}\,dP_{0}-\int_{\mathbb{R}}p_{0}^{\prime}(z)\int_{\mathcal{S}_{0}}g(z,t)\,d\psi(t)\,dz=-\int_{\mathcal{S}_{0}}p_{0}(t)\,d\psi(t)$

and hence that

$\displaystyle
D_{p_{0}}(\psi,\lambda)=\int_{\mathbb{R}}\psi^{2}\,dP_{0}+2\lambda\int_{\mathcal{S}_{0}}p_{0}\,d\psi=\|{\psi}\|^{2}-2\lambda\langle\psi,\psi_{0}\rangle$
$\displaystyle=\|{\psi-\lambda\psi_{0}}\|^{2}-\|{\lambda\psi_{0}}\|^{2}$ (53)
$\displaystyle\geq-\lambda^{2}\|{\psi_{0}}\|^{2}=-\lambda^{2}i(p_{0})>-\infty.$

Thus, by _(b)_ ,
$-\lambda^{2}i^{*}(p_{0})=\inf_{\psi\in\Psi_{\downarrow}(p_{0})}D_{p_{0}}(\psi,\lambda)\geq-\lambda^{2}i(p_{0})>-\infty$,
so $i^{*}(p_{0})\leq i(p_{0})<\infty$. Equality holds if and only if
$\inf_{\psi\in\Psi_{\downarrow}(p_{0})}\|{\psi-\lambda\psi_{0}}\|=0$, i.e.
$\psi_{0}\in\Psi_{\downarrow}(p_{0})$, which is equivalent to $p_{0}$ being
log-concave. ∎

###### Proof of Lemma 4.

(a) By Lemma 46 and (42),

$\lim_{z\to-\infty}\psi_{0}^{*}(z)=\lim_{z\to-\infty}\hat{J}_{0}^{(\mathrm{R})}\bigl{(}F_{0}(z)\bigr{)}=\hat{J}_{0}^{(\mathrm{R})}(0)=\sup_{u\in(0,1)}\frac{J_{0}(u)}{u}>0.$

On the other hand, recalling from Lemma 1 that $p_{0}=J_{0}\circ F_{0}$, we
have

$\limsup_{z\searrow z_{\min}}\,h_{0}(z)=\limsup_{z\searrow
z_{\min}}\frac{p_{0}(z)}{F_{0}(z)}=\limsup_{u\searrow 0}\frac{J_{0}(u)}{u},$

which by (43) can only be finite if $z_{\min}=-\infty$. Since $J_{0}$ is
continuous, $\sup_{u\in(0,1)}J_{0}(u)/u<\infty$ if and only if
$\limsup_{u\searrow 0}J_{0}(u)/u<\infty$, which implies the desired
conclusion.

(b) Likewise,

$\lim_{z\to\infty}\psi_{0}^{*}(z)=\hat{J}_{0}^{(\mathrm{L})}(1)=-\sup_{u\in(0,1)}\frac{J_{0}(u)}{1-u}\in[-\infty,0),$

and is finite if and only if

$\limsup_{z\nearrow z_{\max}}\,h_{0}(z)=\limsup_{z\nearrow
z_{\max}}\frac{p_{0}(z)}{1-F_{0}(z)}=\limsup_{u\nearrow
1}\frac{J_{0}(u)}{1-u}$

is finite, in which case $z_{\max}=\infty$ by arguing similarly to (43). ∎

######  Lemma 22.

Let $s_{0}:=\inf\\{z\in\mathbb{R}:\psi_{0}^{*}(z)\leq 0\\}$ and
$t_{0}:=\sup\\{z\in\mathbb{R}:\psi_{0}^{*}(z)<0\\}$. For
$s,t\in\mathcal{S}_{0}$ such that $s\leq t$, we have

$\int_{s}^{t}\psi_{0}^{*}\,\begin{cases}\geq\log\hat{J}_{0}\bigl{(}F_{0}(t)\bigr{)}-\log\hat{J}_{0}\bigl{(}F_{0}(s)\bigr{)}\;\;&\text{if
}t\leq t_{0}\\\
\leq\log\hat{J}_{0}\bigl{(}F_{0}(t)\bigr{)}-\log\hat{J}_{0}\bigl{(}F_{0}(s)\bigr{)}\;\;&\text{if
}s\geq s_{0},\end{cases}$ (54)

and hence

$\begin{cases}p_{0}(t)\leq
p_{0}(s)\exp\bigl{(}\int_{s}^{t}\psi_{0}^{*}\bigr{)}\;&\text{if }t\leq
t_{0}\text{ and }s\in\mathcal{T}\\\ p_{0}(t)\geq
p_{0}(s)\exp\bigl{(}\int_{s}^{t}\psi_{0}^{*}\bigr{)}\;&\text{if }s\geq
s_{0}\text{ and }t\in\mathcal{T}.\end{cases}$ (55)

###### Proof.

For $s,t\in\mathcal{S}_{0}$ with $s\leq t$, we have $0<F_{0}(s)\leq
F_{0}(t)<1$. By Lemma 1 and the fact that $\psi_{0}^{*}$ is decreasing,

$-\infty<\psi_{0}^{*}(t)(t-s)\leq\int_{s}^{t}\psi_{0}^{*}\leq\psi_{0}^{*}(s)(t-s)<\infty.$

Introducing $U\sim U(0,1)$ and $Z:=F_{0}^{-1}(U)\sim P_{0}$, we have
$F_{0}(Z)=U$ by the continuity of $F_{0}$. Therefore,
$\psi_{0}^{*}(Z)=\hat{J}_{0}^{(\mathrm{R})}(U)$ and
$\mathbb{P}\bigl{(}J_{0}(U)=0\bigr{)}=\mathbb{P}\bigl{(}p_{0}(Z)=0\bigr{)}=0$.
If $t\leq t_{0}$, then $\hat{J}_{0}^{(\mathrm{R})}\circ F_{0}=\psi_{0}^{*}\geq
0$ on $(s,t]$ and hence $\hat{J}_{0}^{(\mathrm{R})}\geq 0$ on
$(F_{0}(s),F_{0}(t)]$. Therefore,

$\displaystyle\int_{s}^{t}\psi_{0}^{*}$
$\displaystyle\geq\mathbb{E}\biggl{(}\frac{\psi_{0}^{*}(Z)}{p_{0}(Z)}\mathbbm{1}_{\\{Z\in(s,t]\\}}\biggr{)}=\mathbb{E}\biggl{(}\frac{\hat{J}_{0}^{(\mathrm{R})}(U)}{J_{0}(U)}\mathbbm{1}_{\\{U\in(F_{0}(s),F_{0}(t)]\\}}\biggr{)}$
$\displaystyle=\int_{F_{0}(s)}^{F_{0}(t)}\frac{\hat{J}_{0}^{(\mathrm{R})}}{J_{0}}\geq\int_{F_{0}(s)}^{F_{0}(t)}\frac{\hat{J}_{0}^{(\mathrm{R})}}{\hat{J}_{0}}=\log\hat{J}_{0}\bigl{(}F_{0}(t)\bigr{)}-\log\hat{J}_{0}\bigl{(}F_{0}(s)\bigr{)},$

where the final equality holds because $\log\hat{J}_{0}$ is finite-valued,
Lipschitz and hence absolutely continuous on $[F_{0}(s),F_{0}(t)]$ with
derivative equal to $\hat{J}_{0}^{(\mathrm{R})}/\hat{J}_{0}$, Lebesgue almost
everywhere. This proves the first line of (54). If $s\in\mathcal{T}$, then by
Lemma 20,

$\log\hat{J}_{0}\bigl{(}F_{0}(t)\bigr{)}-\log\hat{J}_{0}\bigl{(}F_{0}(s)\bigr{)}=\log\frac{\hat{J}_{0}\bigl{(}F_{0}(t)\bigr{)}}{p_{0}(s)}\geq\log\frac{J_{0}\bigl{(}F_{0}(t)\bigr{)}}{p_{0}(s)}=\log\frac{p_{0}(t)}{p_{0}(s)},$

which yields the first line of (55). The proofs in the case $s_{0}\leq s\leq
t$ are analogous, except that now $\hat{J}_{0}^{(\mathrm{R})}\circ
F_{0}=\psi_{0}^{*}\leq 0$ on $(s,t]$ and hence $\hat{J}_{0}^{(\mathrm{R})}\leq
0$ on $(F_{0}(s),F_{0}(t)]$, so the directions of all inequalities above are
reversed. ∎

###### Proof of Lemma 5.

Since
$\sup_{u\in(0,1)}\hat{J}_{0}(u)=\sup_{u\in(0,1)}J_{0}(u)=\sup_{z\in\mathcal{S}_{0}}p_{0}(z)>0$
and $\hat{J}_{0}$ is concave, we have

$\hat{J}_{0}^{(\mathrm{R})}(0)=\lim_{u\searrow
0}\hat{J}_{0}^{(\mathrm{R})}(u)>0>\lim_{u\nearrow
1}\hat{J}_{0}^{(\mathrm{R})}(u)=\hat{J}_{0}^{(\mathrm{R})}(1).$

We have $(F_{0}\circ F_{0}^{-1})(u)=u$ for all $u\in(0,1)$, so if $\delta>0$
is sufficiently small, then $z_{1}:=F_{0}^{-1}(\delta)$ and
$z_{2}:=F_{0}^{-1}(1-\delta)$ are elements of $\mathcal{S}_{0}$ satisfying
$\psi_{0}^{*}(z_{1})=\hat{J}_{0}^{(\mathrm{R})}(\delta)>0>\hat{J}_{0}^{(\mathrm{R})}(1-\delta)=\psi_{0}^{*}(z_{2})$.
By Lemma 1, $\psi_{0}^{*}$ is decreasing and
$\\{z\in\mathbb{R}:\psi_{0}^{*}(z)\in\mathbb{R}\\}=\mathcal{S}_{0}$, so

$\phi_{0}^{*}(z):=\int_{z_{1}}^{z}\psi_{0}^{*}$

is well-defined in $[-\infty,\infty)$ for all $z\in\mathbb{R}$, with
$\phi_{0}^{*}(z)\in\mathbb{R}$ for all $z\in\mathcal{S}_{0}$. By Lemma 22,

$\phi_{0}^{*}(z)=\begin{cases}-\int_{z}^{z_{1}}\psi_{0}^{*}\leq\log\hat{J}_{0}\bigl{(}F_{0}(z)\bigr{)}-\log\hat{J}_{0}(\delta)\;\;&\text{for
}z\leq z_{1}\\\
\phi_{0}^{*}(z_{2})+\int_{z_{2}}^{z}\psi_{0}^{*}\leq\phi_{0}^{*}(z_{2})+\log\hat{J}_{0}\bigl{(}F_{0}(z)\bigr{)}-\log\hat{J}_{0}(1-\delta)\;\;&\text{for
}z\geq z_{2}.\end{cases}$

Together with (42), this shows that
$\phi_{0}^{*}(z)=\phi_{0}^{*}(z_{\min})=\lim_{z^{\prime}\searrow
z_{\min}}\phi_{0}^{*}(z^{\prime})=-\infty$ for all $z\leq z_{\min}$, and
$\phi_{0}^{*}(z)=\phi_{0}^{*}(z_{\max})=\lim_{z^{\prime}\nearrow
z_{\max}}\phi_{0}^{*}(z^{\prime})=-\infty$ for all $z\geq z_{\max}$. We deduce
that $\phi_{0}^{*}$ is concave and continuous as a function from $\mathbb{R}$
to $[-\infty,\infty)$, with
$\\{z\in\mathbb{R}:\phi_{0}^{*}(z)>-\infty\\}=\mathcal{S}_{0}$. By
Rockafellar, (1997, Theorem 24.2), $\phi_{0}^{*}$ has right derivative
$\psi_{0}^{*}$ on $\mathcal{S}_{0}$, with
$(\phi_{0}^{*})^{\prime}=\psi_{0}^{*}$ Lebesgue almost everywhere on
$\mathcal{S}_{0}$. Since $\psi_{0}^{*}$ is decreasing,

$\phi_{0}^{*}(z)\leq\begin{cases}-\psi_{0}^{*}(z_{1})(z_{1}-z)&\;\;\text{for
}z\leq z_{1}\\\
\phi_{0}^{*}(z_{2})+\int_{z_{2}}^{z}\psi_{0}^{*}\leq\phi_{0}^{*}(z_{2})-|\psi_{0}^{*}(z_{2})|(z-z_{2})&\;\;\text{for
}z\geq z_{2},\end{cases}$

so

$\int_{\mathbb{R}}e^{\phi_{0}^{*}}\leq\frac{e^{\phi_{0}^{*}(z_{1})}}{\psi_{0}^{*}(z_{1})}+\int_{z_{1}}^{z_{2}}e^{\phi_{0}^{*}}+\frac{e^{\phi_{0}^{*}(z_{2})}}{|\psi_{0}^{*}(z_{2})|}<\infty.$

Therefore, $p_{0}^{*}:=e^{\phi_{0}^{*}}/\int_{\mathbb{R}}e^{\phi_{0}^{*}}$ is
a continuous log-concave density such that
$\operatorname{supp}p_{0}^{*}=\mathcal{S}_{0}$ and $\log
p_{0}^{*}=\phi_{0}^{*}-\log\bigl{(}\int_{\mathbb{R}}e^{\phi_{0}^{*}}\bigr{)}$
has right derivative $\psi_{0}^{*}$ on $\mathcal{S}_{0}$.

If $\tilde{p}$ is another continuous log-concave density such that
$\log\tilde{p}$ has right derivative $\psi_{0}^{*}$ on $\mathcal{S}_{0}$, then
by Rockafellar, (1997, Corollary 24.2.1),
$\log\tilde{p}(z)=\log\tilde{p}(z_{1})+\int_{z_{1}}^{z}\psi_{0}^{*}$ for all
$z\in\mathbb{R}$. This implies that $\tilde{p}=p_{0}^{*}$, so $p_{0}^{*}$ is
the unique density with the required properties.

Finally, suppose that $p_{0}$ is a continuous log-concave density on
$\mathbb{R}$. Then $\phi_{0}:=\log p_{0}$ is concave, and on
$\mathcal{S}_{0}=\operatorname{supp}p_{0}=\\{z\in\mathbb{R}:\phi_{0}(z)>-\infty\\}$,
its right derivative $\psi_{0}:=\phi_{0}^{(\mathrm{R})}$ is decreasing and
right-continuous. Thus, $p_{0}$ has right derivative
$p_{0}^{(\mathrm{R})}=\psi_{0}p_{0}$ on $\mathcal{S}_{0}$. Moreover, $F_{0}$
is strictly increasing and differentiable on
$\mathcal{S}_{0}=\\{z\in\mathbb{R}:F_{0}(z)\in(0,1)\\}$, so $F_{0}^{-1}$ is
differentiable on $(0,1)$ with derivative
$(F_{0}^{-1})^{\prime}(u)=1/(p_{0}\circ F_{0}^{-1})(u)$ for $u\in(0,1)$, and
$(F_{0}^{-1}\circ F_{0})(z)=z$ for all $z\in\mathcal{S}_{0}$. Consequently,
$J_{0}=p_{0}\circ F_{0}^{-1}$ has right derivative
$J_{0}^{(\mathrm{R})}(u)=p_{0}^{(\mathrm{R})}\bigl{(}F_{0}^{-1}(u)\bigr{)}/p_{0}\bigl{(}F_{0}^{-1}(u)\bigr{)}=(\psi_{0}\circ
F_{0}^{-1})(u)$ for $u\in(0,1)$. Since $\psi_{0}$ is decreasing, $J_{0}$ is
therefore concave on $(0,1)$ by Rockafellar, (1997, Theorem 24.2); see also
Bobkov, (1996, Proposition A.1(c)). Therefore, $\hat{J}_{0}=J_{0}$ and
$\hat{J}_{0}^{(\mathrm{R})}=J_{0}^{(\mathrm{R})}=\psi_{0}\circ F_{0}^{-1}$ on
$(0,1)$, so $\psi_{0}^{*}=\psi_{0}\circ F_{0}^{-1}\circ F_{0}=\psi_{0}$ on
$\mathcal{S}_{0}$. Together with the arguments in the previous paragraph, this
implies that $\phi_{0}^{*}=\phi_{0}$ and hence $p_{0}^{*}=p_{0}$, as claimed.
∎

###### Proof of Proposition 6.

Given $\epsilon\in(0,1)$, let
$a\geq(1-\epsilon^{2})/\epsilon^{2}>(1-\epsilon)/\epsilon$ and define
$p_{0}\colon\mathbb{R}\to\mathbb{R}$ by

$p_{0}(z)=\begin{cases}\epsilon ae^{-a(|z|-1)}/2\;\;&\text{for }|z|\geq 1\\\
\epsilon ae^{b(|z|-1)}/2\;\;&\text{for }|z|\leq 1,\end{cases}$

where $b>0$ uniquely satisfies $1-\epsilon=\int_{-1}^{1}\epsilon
ae^{b(|z|-1)}/2\,dz=\epsilon a(1-e^{-b})/b$. Then $\int_{\mathbb{R}}p_{0}=1$,
so $p_{0}$ is a symmetric, absolutely continuous and piecewise log-affine
density, and $b=(1-e^{-b})a\epsilon/(1-\epsilon)\leq a\epsilon/(1-\epsilon)$.
Moreover,

$\psi_{0}(z):=\frac{p_{0}^{\prime}(z)}{p_{0}(z)}=\begin{cases}-a\;\;&\text{for
}z>1\\\ b\;\;&\text{for }z\in(0,1)\\\ -\psi_{0}(-z)\;\;&\text{for
}z<0,\end{cases}$

so
$i(p_{0})=\int_{\mathbb{R}}\psi_{0}^{2}\,p_{0}=b^{2}(1-\epsilon)+a^{2}\epsilon$
and $\psi_{0}$ is increasing on $(-1,0)\cup(0,1)$. Thus, $p_{0}$ is log-convex
on $[-1,1]$, and log-affine on both $(-\infty,1]$ and $[1,\infty)$. Since
$\int_{\mathbb{R}}zp_{0}(z)\,dz=0$, the proof of Dümbgen et al., (2011, Remark
2.11(ii)) ensures that the log-concave maximum likelihood projection
$q_{0}\equiv p_{0}^{\mathrm{ML}}$ of $p_{0}$ satisfies

$p_{0}^{\mathrm{ML}}(z)=\begin{cases}p_{0}(z)\;\;&\text{for }|z|\geq
1+\delta\\\ p_{0}(1+\delta)=\dfrac{\epsilon ae^{-a\delta}}{2}\;\;&\text{for
}|z|\leq 1+\delta,\end{cases}$

where $\delta>0$ uniquely solves
$1=\int_{\mathbb{R}}p_{0}^{\mathrm{ML}}=\epsilon\bigl{(}a(1+\delta)+1\bigr{)}e^{-a\delta}$.
Then the associated score function
$\psi_{0}^{\mathrm{ML}}:=q_{0}^{(\mathrm{R})}/q_{0}$ is given by

$\psi_{0}^{\mathrm{ML}}(z)=\begin{cases}a\;\;&\text{for }z<-(1+\delta)\\\
0\;\;&\text{for }z\in[-(1+\delta),1+\delta)\\\ -a\;\;&\text{for }z\geq
1+\delta.\end{cases}$

Therefore, $\psi_{0}^{\mathrm{ML}}\in\Psi_{\downarrow}(p_{0})$ with

$\displaystyle\int_{\mathbb{R}}(\psi_{0}^{\mathrm{ML}})^{2}\,dP_{0}=2a^{2}\int_{1+\delta}^{\infty}p_{0}=a^{2}\epsilon
e^{-a\delta}=a\bigl{(}p_{0}(1+\delta)+p_{0}(-1-\delta)\bigr{)}=-\int_{\mathbb{R}}p_{0}\,d\psi_{0}^{\mathrm{ML}},$

so

$\frac{1}{V_{p_{0}}(\psi_{0}^{\mathrm{ML}})}=\frac{\bigl{(}\int_{\mathbb{R}}p_{0}\,d\psi_{0}^{\mathrm{ML}}\bigr{)}^{2}}{\int_{\mathbb{R}}(\psi_{0}^{\mathrm{ML}})^{2}\,dP_{0}}=-\int_{\mathbb{R}}p_{0}\,d\psi_{0}^{\mathrm{ML}}=a^{2}\epsilon
e^{-a\delta}.$

Furthermore, $J_{0}^{\prime}(u)=(\psi_{0}\circ F_{0}^{-1})(u)$ whenever
$u=F_{0}(z)$ for $z\neq\\{-1,0,1\\}$, so $J_{0}$ is symmetric about $1/2$,
convex on $[F_{0}(-1),F_{0}(1)]$, and linear on both $[0,F_{0}(-1)]$ and
$[F_{0}(1),1]$. Hence, $\hat{J}_{0}=J_{0}$ on $[0,F_{0}(-1)]\cup[F_{0}(1),1]$
and $\hat{J}_{0}=p_{0}(1)$ on $[F_{0}(-1),F_{0}(1)]$, so

$\psi_{0}^{*}(z)=\hat{J}_{0}^{(\mathrm{R})}\bigl{(}F_{0}(z)\bigr{)}=\begin{cases}a\;\;&\text{for
}z<-1\\\ 0\;\;&\text{for }z\in[-1,1)\\\ -a\;\;&\text{for }z\geq 1.\end{cases}$

Thus, by Theorem 2(c),
$1/V_{p_{0}}(\psi_{0}^{*})=i^{*}(p_{0})=\int_{\mathbb{R}}(\psi_{0}^{*})^{2}\,p_{0}=a^{2}\epsilon$,
so

$\frac{V_{p_{0}}(\psi_{0}^{*})}{V_{p_{0}}(\psi_{0}^{\mathrm{ML}})}=e^{-a\delta}=\frac{1}{\epsilon\bigl{(}a(1+\delta)+1\bigr{)}}<\frac{1}{\epsilon(a+1)}\leq\epsilon$

and

$\mathrm{ARE}^{*}(p_{0})=\frac{i^{*}(p_{0})}{i(p_{0})}=\frac{a^{2}\epsilon}{b^{2}(1-\epsilon)+a^{2}\epsilon}\geq\frac{a^{2}\epsilon}{(a\epsilon)^{2}/(1-\epsilon)+a^{2}\epsilon}=1-\epsilon,$

as required. ∎

###### Proof of Lemma 7.

By Lemma 1, $\hat{J}_{0}$ is continuous on $[0,1]$ with
$\hat{J}_{0}(0)=\hat{J}_{0}(1)=0$, and we can find $u^{*}\in(0,1)$ such that
$\hat{J}_{0}(u^{*})=J_{0}(u^{*})=\|{J_{0}}\|_{\infty}=\|{p_{0}}\|_{\infty}<\infty$.
Then by the Cauchy–Schwarz inequality and Rockafellar, (1997, Corollary
24.2.1),

$\displaystyle\int_{0}^{u^{*}}\bigl{(}\hat{J}_{0}^{(\mathrm{R})}\bigr{)}^{2}$
$\displaystyle\geq\frac{\bigl{(}\int_{0}^{u^{*}}\hat{J}_{0}^{(\mathrm{R})}\bigr{)}^{2}}{u^{*}}=\frac{\hat{J}_{0}(u^{*})^{2}}{u^{*}}=\frac{\|{p_{0}}\|_{\infty}^{2}}{u^{*}},$
$\displaystyle\int_{u^{*}}^{1}\bigl{(}\hat{J}_{0}^{(\mathrm{R})}\bigr{)}^{2}$
$\displaystyle\geq\frac{\bigl{(}\int_{u^{*}}^{1}\hat{J}_{0}^{(\mathrm{R})}\bigr{)}^{2}}{1-u^{*}}=\frac{\hat{J}_{0}(u^{*})^{2}}{1-u^{*}}=\frac{\|{p_{0}}\|_{\infty}^{2}}{1-u^{*}},$

with equality if and only if $\hat{J}_{0}$ is linear on both $[0,u^{*}]$ and
$[u^{*},1]$. Thus, by Remark 3,

$i^{*}(p_{0})=\int_{0}^{1}\bigl{(}\hat{J}_{0}^{(\mathrm{R})}\bigr{)}^{2}\geq\|{p_{0}}\|_{\infty}^{2}\Bigl{(}\frac{1}{u^{*}}+\frac{1}{1-u^{*}}\Bigr{)}\geq
4\|{p_{0}}\|_{\infty}^{2}.$

Equality holds if and only if $u^{*}=1/2$ and
$\hat{J}_{0}(u)=\min(u,1-u)/\sigma$ for some $\sigma>0$. In this case, letting
$\mu:=F_{0}^{-1}(1/2)$, we deduce that
$\psi_{0}^{*}(z)=(\hat{J}_{0}^{(\mathrm{R})}\circ
F_{0})(z)=-\operatorname{sgn}(z-\mu)/\sigma$ for all $z\in\mathbb{R}$, so
$p_{0}^{*}$ is a Laplace density of the stated form. ∎

To prove Proposition 9, we require some further definitions and lemmas. Let
$\mathcal{T}$ be as in the proposition. Then $\mathcal{T}^{c}$ is an open
subset of $\mathbb{R}$ and hence has a unique representation as a countable
disjoint union $\bigcup_{k=1}^{K}(s_{k},t_{k})$ of open intervals, where
$K\in\mathbb{N}_{0}\cup\\{\infty\\}$ and $-\infty\leq s_{k}<t_{k}\leq\infty$
for every $k$. Recall from Lemma 22 that
$s_{0}=\inf\\{z\in\mathbb{R}:\psi_{0}^{*}(z)\leq 0\\}$ and
$t_{0}=\sup\\{z\in\mathbb{R}:\psi_{0}^{*}(z)<0\\}$. For $t\in\mathbb{R}$, let
$\psi_{0}(t-):=\lim_{z\nearrow t}\psi_{0}^{*}(z)$. For $k\in\mathbb{N}$ such
that $k\leq K$, inductively define $p_{k}\colon[-\infty,\infty]\to\mathbb{R}$
by

$p_{k}(z):=\begin{cases}p_{k-1}(z)\;\;&\text{for }z\leq s_{k}\\\
p_{k-1}(s_{k})e^{\psi_{0}^{*}(s_{k})(z-s_{k})}\;\;&\text{for
}z\in[s_{k},t_{k}]\\\
p_{k-1}(z)\cdot\dfrac{p_{k-1}(s_{k})e^{\psi_{0}^{*}(s_{k})(t_{k}-s_{k})}}{p_{k-1}(t_{k})}\;\;&\text{for
}z>t_{k}\end{cases}$

if $s_{k}>s_{0}$, and otherwise let

$p_{k}(z):=\begin{cases}p_{k-1}(z)\cdot\dfrac{p_{k-1}(t_{k})e^{\psi_{0}^{*}(t_{k}-)(s_{k}-t_{k})}}{p_{k-1}(s_{k})}\;\;&\text{for
}z<s_{k}\\\\[6.0pt]
p_{k-1}(t_{k})e^{\psi_{0}^{*}(t_{k}-)(z-t_{k})}\;\;&\text{for
}z\in[s_{k},t_{k}]\\\ p_{k-1}(z)\;\;&\text{for }z\geq t_{k}.\end{cases}$

if $s_{k}\leq s_{0}$. When $K<\infty$, define $p_{k}:=p_{K}$ for all
$k\in\mathbb{N}$ with $k>K$. By Lemmas 1 and 20,
$\psi_{0}^{*}(z)\in\mathbb{R}$ and $p_{0}(z)>0$ for all $z\in\mathcal{T}$.
Since $s_{k},t_{k}\in\mathcal{T}\cup\\{-\infty,\infty\\}$ and
$p_{k}(s_{0})=p_{k-1}(s_{0})$ for $k\in\mathbb{N}$ such that $k\leq K$, it
follows by induction that $p_{k}>0$ on $\mathcal{T}$ for all $k\in\mathbb{N}$,
with $p_{k}(s_{0})=p_{0}(s_{0})$. In particular, $p_{k-1}(s_{k})>0$ whenever
$s_{k}>-\infty$ and $p_{k-1}(t_{k})>0$ whenever $t_{k}<\infty$.

######  Lemma 23.

For every $k\in\mathbb{N}_{0}$, the following statements hold.

  1. (a)

$p_{k}\leq p_{0}$ on $\mathcal{T}$, and
$\|{p_{k}}\|_{\infty}=\|{p_{0}}\|_{\infty}$;

  2. (b)

$\displaystyle p_{0}(z)\int_{s}^{t}p_{k}\geq p_{k}(z)\int_{s}^{t}p_{0}$ if
$z\in\\{s,t\\}$ for some $s,t\in\mathcal{T}\cup\\{-\infty,\infty\\}$ such that
$s\leq t$;

  3. (c)

$\dfrac{p_{k}(z)}{p_{k}(s_{j})}=\begin{cases}p_{0}(z)/p_{0}(s_{j})&\text{if
}z\in[s_{j},t_{j}]\text{ for some }j\in\mathbb{N}\text{ with }j>k\\\
p_{j}(z)/p_{j}(s_{j})&\text{if }z\in[s_{j},t_{j}]\text{ for some
}j\in\mathbb{N}\text{ with }j<k;\end{cases}$

  4. (d)

$\displaystyle\psi_{0}^{*}(t-)\int_{s}^{t}p_{k}\leq
p_{k}(t)-p_{k}(s)\leq\psi_{0}^{*}(s)\int_{s}^{t}p_{k}$ for all
$s,t\in\mathcal{T}\cup\\{-\infty,\infty\\}$ such that $s\leq t$.

###### Proof.

We will proceed by induction on $k$. When $k=0$, (a), (b) and (c) hold
trivially, so it remains to prove (d). This holds with equality when $s=t$, so
now let $s,t\in\mathcal{T}\cup\\{-\infty,\infty\\}$ be such that $s<t$. By the
concavity of $\hat{J}_{0}$ together with Lemmas 1 and 20, $v:=F_{0}(s)$ and
$w:=F_{0}(t)$ satisfy $0\leq v<w\leq 1$ and

$p_{0}(t)-p_{0}(s)=\hat{J}_{0}(w)-\hat{J}_{0}(v)\begin{cases}\leq\hat{J}_{0}^{(\mathrm{R})}(v)(w-v)=\psi_{0}^{*}(s)\int_{s}^{t}p_{0}\\\\[3.0pt]
\geq\hat{J}_{0}^{(\mathrm{L})}(w)(w-v)=\displaystyle\lim_{u\nearrow
w}\hat{J}_{0}^{(\mathrm{R})}(u)(w-v)=\textstyle\psi_{0}^{*}(t-)\int_{s}^{t}p_{0},\end{cases}$
(56)

where $\hat{J}_{0}^{(\mathrm{L})}(w)=\lim_{u\nearrow
w}\hat{J}_{0}^{(\mathrm{R})}(u)$ by Rockafellar, (1997, Theorem 24.1).

Next, consider a general $k\in\mathbb{N}$. If $k>K$, then $p_{k}=p_{k-1}$ and
hence (a)–(d) hold by induction. Supposing now that $k\leq K$, let
$a_{k}:=\psi_{0}^{*}(s_{k})$ if $s_{k}>-\infty$ and otherwise let
$a_{k}:=\psi_{0}^{*}(t_{k}-)$, so that $\psi_{0}^{*}=a_{k}$ on
$(s_{k},t_{k})$.

(a) By part (c) of the inductive hypothesis and (55),

$r_{k}:=\frac{p_{k-1}(t_{k})e^{a_{k}(s_{k}-t_{k})}}{p_{k-1}(s_{k})}=\frac{p_{0}(t_{k})e^{a_{k}(s_{k}-t_{k})}}{p_{0}(s_{k})}\begin{cases}\leq
1\;\;&\text{if }s_{k}\leq s_{0}\\\ \geq 1\;\;&\text{if }s_{k}\geq
s_{0}.\end{cases}$ (57)

Therefore,

$\begin{cases}p_{k}=r_{k}p_{k-1}\leq p_{k-1}\text{ on
}(-\infty,s_{k}]\quad\text{and}\quad p_{k}=p_{k-1}\text{ on
}[t_{k},\infty)\;\;&\text{if }s_{k}\leq s_{0}\\\ p_{k}=p_{k-1}\text{ on
}(-\infty,s_{k}]\quad\text{and}\quad p_{k}=r_{k}^{-1}p_{k-1}\leq p_{k-1}\text{
on }[t_{k},\infty)\;\;&\text{if }s_{k}\geq s_{0},\end{cases}$ (58)

while for $z\in[s_{k},t_{k}]$, we have

$p_{k}(z)=\begin{cases}p_{k-1}(t_{k})e^{a_{k}(z-t_{k})}\leq
p_{k-1}(t_{k})\;\;&\text{if }s_{k}\leq s_{0}\\\
p_{k-1}(s_{k})e^{a_{k}(z-s_{k})}\leq p_{k-1}(s_{k})\;\;&\text{if }s_{k}\geq
s_{0}.\end{cases}$

This shows that $p_{k}\leq p_{k-1}$ on
$\mathbb{R}\setminus(s_{k},t_{k})\supseteq\mathcal{T}$ and
$\|{p_{k}}\|_{\infty}=\|{p_{k-1}}\|_{\infty}$, so (a) holds by induction.

(b) Taking $s=s_{k}$ and $t=t_{k}$ in (56), we deduce from part (c) of the
inductive hypothesis that

$\frac{p_{k-1}(t_{k})-p_{k-1}(s_{k})}{\int_{s_{k}}^{t_{k}}p_{k-1}}=\frac{p_{0}(t_{k})-p_{0}(s_{k})}{\int_{s_{k}}^{t_{k}}p_{0}}=a_{k}\begin{cases}\geq
0\;\;&\text{if }s_{k}\leq s_{0}\\\ \leq 0\;\;&\text{if }s_{k}\geq
s_{0}.\end{cases}$ (59)

There is nothing to prove when $s=t$, so let
$s,t\in\mathcal{T}\cup\\{-\infty,\infty\\}$ be such that $s<t$. If either
$(s,t)\subseteq(-\infty,s_{k})$ or $(s,t)\subseteq(t_{k},\infty)$, then (58)
implies that $p_{k}=rp_{k-1}$ on $[s,t]$ for some
$r\in\\{1,r_{k},r_{k}^{-1}\\}$, so by part (c) of the inductive hypothesis,

$p_{0}(z)\int_{s}^{t}p_{k}=rp_{0}(z)\int_{s}^{t}p_{k-1}\geq
rp_{k-1}(z)\int_{s}^{t}p_{0}=p_{k}(z)\int_{s}^{t}p_{0}$

for $z\in\\{s,t\\}$. It remains to consider the case
$(s_{k},t_{k})\subseteq(s,t)$. Assume first that $s_{k}\leq s_{0}$, so that
$a_{k}\in[0,\infty)$. If $a_{k}=0$, then by part (c) of the inductive
hypothesis and (55),

$p_{k-1}(z)=\frac{p_{0}(z)}{p_{0}(s_{k})}\cdot p_{k-1}(s_{k})\leq
p_{k-1}(s_{k})\leq p_{k-1}(s_{k})=p_{k-1}(t_{k})$

for all $z\in[s_{k},t_{k}]$, so

$\int_{s}^{t}p_{k}=\frac{p_{k-1}(t_{k})}{p_{k-1}(s_{k})}\int_{-\infty}^{s_{k}}p_{k-1}+\int_{s_{k}}^{t_{k}}p_{k-1}(t_{k})\,dz+\int_{t_{k}}^{t}p_{k-1}\geq\int_{s}^{s_{k}}p_{k-1}+\int_{s_{k}}^{t_{k}}p_{k-1}+\int_{t_{k}}^{t}p_{k-1}=\int_{s}^{t}p_{k-1}.$

Suppose instead that $a_{k}>0$. Taking $s=-\infty$ and $t=s_{k}$ in part (d)
of the inductive hypothesis, we obtain
$p_{k-1}(s_{k})\geq\psi_{0}^{*}(s_{k}-)\int_{-\infty}^{s_{k}}p_{k-1}\geq
a_{k}\int_{-\infty}^{s_{k}}p_{k-1}$, so

$\frac{\int_{-\infty}^{s_{k}}p_{k-1}}{p_{k-1}(s_{k})}\leq\frac{1}{a_{k}}.$

Together with (59), this implies that

$\displaystyle\int_{-\infty}^{t_{k}}p_{k}=\int_{-\infty}^{s_{k}}r_{k}p_{k-1}+p_{k-1}(t_{k})\cdot\frac{1-e^{a_{k}(s_{k}-t_{k})}}{a_{k}}$
$\displaystyle=p_{k-1}(t_{k})e^{a_{k}(s_{k}-t_{k})}\biggl{(}\frac{\int_{-\infty}^{s_{k}}p_{k-1}}{p_{k-1}(s_{k})}-\frac{1}{a_{k}}\biggr{)}+\frac{p_{k-1}(t_{k})}{a_{k}}$
$\displaystyle\geq
p_{k-1}(s_{k})\biggl{(}\frac{\int_{-\infty}^{s_{k}}p_{k-1}}{p_{k-1}(s_{k})}-\frac{1}{a_{k}}\biggr{)}+\frac{p_{k-1}(t_{k})}{a_{k}}$
$\displaystyle=\int_{-\infty}^{s_{k}}p_{k-1}+\frac{p_{k-1}(t_{k})-p_{k-1}(s_{k})}{a_{k}}=\int_{-\infty}^{t_{k}}p_{k-1}.$

Combining this with (57) and (58) yields

$\int_{s}^{t}p_{k}=\int_{-\infty}^{t_{k}}p_{k}-\int_{-\infty}^{s}r_{k}p_{k-1}+\int_{t_{k}}^{t}p_{k-1}\geq\int_{-\infty}^{t_{k}}p_{k-1}-\int_{-\infty}^{s}p_{k-1}+\int_{t_{k}}^{t}p_{k-1}=\int_{s}^{t}p_{k-1}.$

In view of (58) and part (b) of the inductive hypothesis, we conclude that if
$(s_{k},t_{k})\subseteq(s,t)$ and $s_{k}\leq s_{0}$, then

$p_{0}(z)\int_{s}^{t}p_{k}=p_{0}(z)\int_{s}^{t}p_{k-1}\geq
p_{k-1}(z)\int_{s}^{t}p_{0}\geq p_{k}(z)\int_{s}^{t}p_{0}$

for $z\in\\{s,t\\}$, which proves (b) in this case. On the other hand, if
$(s_{k},t_{k})\subseteq(s,t)$ and $s_{k}>s_{0}$, then instead
$a_{k}\in(-\infty,0]$, but the arguments are similar and hence omitted.

(c) For $j\in[K]$ such that $j>k$, we have
$(s_{j},t_{j})\cap(s_{k},t_{k})=\emptyset$, so by (58) and part (c) of the
inductive hypothesis,

$\frac{p_{k}(z)}{p_{k}(s_{j})}=\frac{p_{k-1}(z)}{p_{k-1}(s_{j})}=\begin{cases}p_{0}(z)/p_{0}(s_{j})&\text{if
}z\in[s_{j},t_{j}]\text{ for }j\in\mathbb{N}\text{ with }j>k\\\
p_{j}(z)/p_{j}(s_{j})&\text{if }z\in[s_{j},t_{j}]\text{ for
}j\in\mathbb{N}\text{ with }j<k.\end{cases}$

(d) There is nothing to prove when $s=t$, so let
$s,t\in\mathcal{T}\cup\\{-\infty,\infty\\}$ be such that $s<t$. If either
$(s,t)\subseteq(-\infty,s_{k})$ or $(s,t)\subseteq(t_{k},\infty)$, then by
(58), $p_{k}=rp_{k-1}$ on $[s,t]$ for some $r\in\\{1,r_{k},r_{k}^{-1}\\}$.
Thus, by part (d) of the inductive hypothesis,

$p_{k}(t)-p_{k}(s)=r\bigl{(}p_{k-1}(t)-p_{k-1}(s)\bigl{)}\>\begin{cases}\leq
r\psi_{0}^{*}(s)\int_{s}^{t}p_{k-1}=\psi_{0}^{*}(s)\int_{s}^{t}p_{k}\\\\[3.0pt]
\geq
r\psi_{0}^{*}(t-)\int_{s}^{t}p_{k-1}=\psi_{0}^{*}(t-)\int_{s}^{t}p_{k},\end{cases}$

as required. In the remaining case, $(s_{k},t_{k})\subseteq(s,t)$, and
similarly by (58) and part (d) of the inductive hypothesis,

$\displaystyle\psi_{0}^{*}(s_{k}-)\int_{s}^{s_{k}}p_{k}$ $\displaystyle\leq
p_{k}(s_{k})-p_{k}(s)\leq\psi_{0}^{*}(s)\int_{s}^{s_{k}}p_{k},$
$\displaystyle\psi_{0}^{*}(t-)\int_{t_{k}}^{t}p_{k}$ $\displaystyle\leq
p_{k}(t)-p_{k}(t_{k})\leq\psi_{0}^{*}(t_{k})\int_{t_{k}}^{t}p_{k}.$

Moreover,
$a_{k}\int_{s_{k}}^{t_{k}}p_{k}=a_{k}p_{k}(t_{k})\int_{s_{k}}^{t_{k}}e^{a(z-t_{k})}\,dz=p_{k}(t_{k})-p_{k}(s_{k})$
and $a_{k}=\psi_{0}^{*}(s_{k})=\psi_{0}^{*}(t_{k}-)$, so

$\psi_{0}^{*}(t_{k}-)\int_{s_{k}}^{t_{k}}p_{k}=p_{k}(t_{k})-p_{k}(s_{k})=\psi_{0}^{*}(s_{k})\int_{s_{k}}^{t_{k}}p_{k}.$

Since $\psi_{0}^{*}$ is decreasing, combining this with the two inequalities
above yields (d). ∎

######  Lemma 24.

For $z\in\mathbb{R}$, we have

$\lim_{k\to\infty}p_{k}(z)=\frac{p_{0}(s_{0})}{p_{0}^{*}(s_{0})}\cdot
p_{0}^{*}(z).$ (60)

Moreover, let $t_{\min}:=\inf\mathcal{T}$ and $t_{\max}:=\sup\mathcal{T}$. If
$s,t\in\mathcal{T}\cup\\{-\infty,\infty\\}$ are such that $-\infty<s\vee
t_{\min}\leq t\wedge t_{\max}<\infty$, then

$\lim_{k\to\infty}\int_{s}^{t}p_{k}=\frac{p_{0}^{*}(s_{0})}{p_{0}(s_{0})}\int_{s}^{t}p_{0}^{*}.$
(61)

###### Proof.

For $k\in\mathbb{N}$, let $\phi_{k}:=\log p_{k}$ and define $r_{k}$ as in
(57). Then as noted above, $\phi_{k}>-\infty$ on $\mathcal{T}$, and Lemma
23(c) ensures that

$\log
r_{k}=\psi_{0}^{*}(s_{k})(t_{k}-s_{k})-\bigl{(}\phi_{k-1}(t_{k})-\phi_{k-1}(s_{k})\bigr{)}=\int_{s_{k}}^{t_{k}}\psi_{0}^{*}(z)\,dz-\bigl{(}\phi_{0}(t_{k})-\phi_{0}(s_{k})\bigr{)}.$
(62)

Fix $t\in\mathcal{T}$ such that $t\geq s_{0}$, and for convenience, define
$[K]=\emptyset$ when $K=0$ and $[K]=\mathbb{N}$ when $K=\infty$. Let
$\mathcal{K}$ be the set of $\ell\in[K]$ such that
$(s_{\ell},t_{\ell})\subseteq(s_{0},t)$, so that
$(s_{\ell},t_{\ell})\cap(s_{0},t)=\emptyset$ for
$\ell\in[K]\setminus\mathcal{K}$ and hence
$\mathcal{T}^{c}\cap(s_{0},t)=\bigcup_{\ell\in\mathcal{K}}(s_{\ell},t_{\ell})$.
Similarly to the proof of Lemma 22, let $U\sim U(0,1)$, so that
$Z:=F_{0}^{-1}(U)\sim P_{0}$, $F_{0}(Z)=U$ and
$\psi_{0}^{*}(Z)=\hat{J}_{0}^{(\mathrm{R})}(U)$. We have $\psi_{0}^{*}\leq 0$
and $p_{0}>0$ on $\mathcal{T}\cap(s_{0},t)$, and moreover $\hat{J}_{0}$ is
Lipschitz on $(F_{0}(s_{0}),F_{0}(t)]$ with $\hat{J}_{0}^{(\mathrm{R})}\leq
0$, so

$\displaystyle\int_{s_{0}}^{t}\psi_{0}^{*}\mathbbm{1}_{\mathcal{T}}$
$\displaystyle=\mathbb{E}\biggl{(}\frac{\psi_{0}^{*}(Z)}{p_{0}(Z)}\mathbbm{1}_{\\{Z\in\mathcal{T}\cap(s_{0},t)\\}}\biggr{)}=\mathbb{E}\biggl{(}\frac{\hat{J}_{0}^{(\mathrm{R})}(U)}{\hat{J}_{0}(U)}\mathbbm{1}_{\\{F_{0}^{-1}(U)\in\mathcal{T}\cap(s_{0},t)\\}}\biggr{)}$
$\displaystyle=\mathbb{E}\biggl{(}\frac{\hat{J}_{0}^{(\mathrm{R})}(U)}{\hat{J}_{0}(U)}\mathbbm{1}_{\\{F_{0}^{-1}(U)\in(s_{0},t]\\}}\biggr{)}-\sum_{\ell\in\mathcal{K}}\mathbb{E}\biggl{(}\frac{\hat{J}_{0}^{(\mathrm{R})}(U)}{\hat{J}_{0}(U)}\mathbbm{1}_{\\{F_{0}^{-1}(U)\in(s_{\ell},t_{\ell}]\\}}\biggr{)}$
$\displaystyle=\int_{F_{0}(s_{0})}^{F_{0}(t)}\frac{\hat{J}_{0}^{(\mathrm{R})}}{\hat{J}_{0}}-\sum_{\ell\in\mathcal{K}}\int_{F_{0}(s_{\ell})}^{F_{0}(t_{\ell})}\frac{\hat{J}_{0}^{(\mathrm{R})}}{\hat{J}_{0}}$
$\displaystyle=\log\hat{J}_{0}\bigl{(}F_{0}(t)\bigr{)}-\log\hat{J}_{0}\bigl{(}F_{0}(s_{0})\bigr{)}-\sum_{\ell\in\mathcal{K}}\bigl{\\{}\log\hat{J}_{0}\bigl{(}F_{0}(t_{\ell})\bigr{)}-\log\hat{J}_{0}\bigl{(}F_{0}(s_{\ell})\bigr{)}\bigr{\\}}$
$\displaystyle=\phi_{0}(t)-\phi_{0}(s_{0})-\sum_{\ell\in\mathcal{K}}\bigl{(}\phi_{0}(t_{\ell})-\phi_{0}(s_{\ell})\bigr{)},$
(63)

where the second and final equalities above follow from Lemma 20 and the fact
that $s_{0},t,s_{\ell},t_{\ell}\in\mathcal{T}\cup\\{-\infty,\infty\\}$ for
$\ell\in[K]$. In addition, for all such $\ell$, we have
$\phi_{\ell}(t)=\phi_{\ell-1}(t)+(\log
r_{\ell})\mathbbm{1}_{\\{\ell\in\mathcal{K}\\}}$ by the definition of
$p_{\ell}$, and $\phi_{0}^{*}:=\log p_{0}^{*}$ satisfies
$\phi_{0}^{*}(t)-\phi_{0}^{*}(s_{0})=\int_{s_{0}}^{t}\psi_{0}^{*}$. Thus, by
induction together with (62) and (63),

$\displaystyle\phi_{k}(t)-\phi_{0}(s_{0})$
$\displaystyle=\phi_{0}(t)-\phi_{0}(s_{0})+\sum_{\ell\in\mathcal{K}\cap[k]}\log
r_{\ell}=\phi_{0}(t)-\phi_{0}(s_{0})-\sum_{\ell\in\mathcal{K}\cap[k]}\Bigl{(}\phi_{0}(t_{\ell})-\phi_{0}(s_{\ell})-\int_{s_{\ell}}^{t_{\ell}}\psi_{0}^{*}\Bigr{)}$
$\displaystyle=\biggl{(}\int_{s_{0}}^{t}\psi_{0}^{*}\mathbbm{1}_{\mathcal{T}}+\sum_{\ell\in\mathcal{K}}\int_{s_{\ell}}^{t_{\ell}}\psi_{0}^{*}\biggr{)}+\sum_{\ell\in\mathcal{K}:\ell>k}\Bigl{(}\phi_{0}(t_{\ell})-\phi_{0}(s_{\ell})-\int_{s_{\ell}}^{t_{\ell}}\psi_{0}^{*}\Bigr{)}$
$\displaystyle=\int_{s_{0}}^{t}\psi_{0}^{*}+\sum_{\ell\in\mathcal{K}:\ell>k}\Bigl{(}\phi_{0}(t_{\ell})-\phi_{0}(s_{\ell})-\int_{s_{\ell}}^{t_{\ell}}\psi_{0}^{*}\Bigr{)}\to\phi_{0}^{*}(t)-\phi_{0}^{*}(s_{0})$

as $k\to\infty$, which yields (60) for $z=t$. On the other hand, if
$t\in\mathcal{T}$ and $t<s_{0}$, then we can instead let $\mathcal{K}$ be the
set of $\ell\in[K]$ such that $(s_{\ell},t_{\ell})\subseteq(t,s_{0})$, and
deduce by similar reasoning that

$\displaystyle\phi_{k}(t)-\phi_{0}(s_{0})$
$\displaystyle=\phi_{0}(t)-\phi_{0}(s_{0})+\sum_{\ell\in\mathcal{K}\cap[k]}\Bigl{(}\phi_{0}(t_{\ell})-\phi_{0}(s_{\ell})-\int_{s_{\ell}}^{t_{\ell}}\psi_{0}^{*}\Bigr{)}$
$\displaystyle=-\int_{t}^{s_{0}}\psi_{0}^{*}-\sum_{\ell\in\mathcal{K}:\ell>k}\Bigl{(}\phi_{0}(t_{\ell})-\phi_{0}(s_{\ell})-\int_{s_{\ell}}^{t_{\ell}}\psi_{0}^{*}\Bigr{)}\to\int_{s_{0}}^{t}\psi_{0}^{*}=\phi_{0}^{*}(t)-\phi_{0}^{*}(s_{0})$

as $k\to\infty$. Having proved that (60) holds for all $z\in\mathcal{T}$, we
now consider $z\in\mathcal{T}^{c}$, for which there exists a unique
$\ell\in[K]$ such that $z\in(s_{\ell},t_{\ell})$. If $s_{\ell}>s_{0}$, then
$p_{\ell}(z)=p_{\ell}(s_{\ell})e^{\psi_{0}^{*}(s_{\ell})(z-s_{\ell})}$, while
if $s_{\ell}\leq s_{0}$, then
$p_{\ell}(z)=p_{\ell}(t_{\ell})e^{\psi_{0}^{*}(s_{\ell})(z-t_{\ell})}$. For
$k>\ell$, it follows from Lemma 23(c) that

$p_{k}(z)=\begin{cases}p_{k}(t_{\ell})e^{\psi_{0}^{*}(s_{\ell})(z-t_{\ell})}&\text{if
}s_{\ell}\leq s_{0}\\\
p_{k}(s_{\ell})e^{\psi_{0}^{*}(s_{\ell})(z-s_{\ell})}&\text{if
}s_{\ell}>s_{0}.\end{cases}$

Moreover, $s_{\ell},t_{\ell}\in\mathcal{T}\cup\\{-\infty,\infty\\}$, so if
$s_{\ell}\leq s_{0}$, then

$p_{k}(z)=p_{k}(t_{\ell})e^{\psi_{0}^{*}(s_{\ell})(z-t_{\ell})}\to\frac{p_{0}(s_{0})}{p_{0}^{*}(s_{0})}\cdot
p_{0}^{*}(t_{\ell})e^{\psi_{0}^{*}(s_{\ell})(z-t_{\ell})}=\frac{p_{0}(s_{0})}{p_{0}^{*}(s_{0})}\cdot
p_{0}^{*}(z)$

as $K\to\infty$, and (60) holds similarly when $s_{\ell}>s_{0}$.

Finally, let $s,t\in\mathcal{T}\cup\\{-\infty,\infty\\}$ be such that
$-\infty<s\vee t_{\min}\leq t\wedge t_{\max}<\infty$. If $t>t_{\max}$, then
there exists a unique $j\in[K]$ such that $t_{\max}=s_{j}<t_{j}=t=\infty$, and
by Lemma 5 and its proof,
$\psi_{0}^{*}(t_{\max})=\lim_{z\to\infty}\psi_{0}^{*}(z)\in(-\infty,0)$.
Moreover, by Lemma 23(c),

$p_{k}(z)=p_{k}(t_{\max})e^{\psi_{0}^{*}(t_{\max})(z-t_{\max})}$

for all $z>t_{\max}$ and $k\geq j$. Similarly, if $s<t_{\min}$, then there
exists a unique $\ell\in[K]$ such that $-\infty=s=s_{\ell}<t_{\ell}=t_{\min}$,
and $\psi_{0}^{*}(t_{\min}-)=\lim_{z\to-\infty}\psi_{0}^{*}(z)\in(0,\infty)$.
In addition, by Lemma 23(c),

$p_{k}(z)=p_{k}(t_{\min})e^{\psi_{0}^{*}(t_{\min}-)(z-t_{\min})}$

for all $z<t_{\min}$ and $k\geq\ell$. Thus, for $k\geq j\vee\ell$, it follows
from Lemma 23(a) that in all cases,

$p_{k}(z)\leq\begin{cases}p_{0}(t_{\min})e^{\psi_{0}^{*}(t_{\min}-)(z-t_{\min})}\;\;&\quad\text{for
}z\in(s,s\vee t_{\min})\\\ \|{p_{0}}\|_{\infty}\;\;&\quad\text{for }z\in[s\vee
t_{\min},t\wedge t_{\max}]\\\
p_{0}(t_{\max})e^{\psi_{0}^{*}(t_{\max})(z-t_{\max})}\;\;&\quad\text{for
}z\in(t\wedge t_{\max},t),\end{cases}$

so the pointwise supremum $\sup_{k\geq j\vee\ell}p_{k}$ is integrable on
$(s,t)$, and hence (61) follows from (60) and the dominated convergence
theorem. ∎

###### Proof of Proposition 9.

(a) We will prove the following stronger statement: for
$s,t\in\mathcal{T}\cup\\{-\infty,\infty\\}$ such that $s\leq t$, and
$z\in\\{s,t\\}$, we have

$\frac{p_{0}^{*}(z)}{F_{0}^{*}(t)-F_{0}^{*}(s)}\leq\frac{p_{0}(z)}{F_{0}(t)-F_{0}(s)}.$
(64)

Assume that at least one of $s,t$ is finite, since otherwise (64) holds
trivially. Let $t_{\min}:=\inf\mathcal{T}$ and $t_{\max}:=\sup\mathcal{T}$. If
$-\infty<s\vee t_{\min}\leq t\wedge t_{\max}<\infty$, then by Lemma 24,

$p_{0}(z)\int_{s}^{t}p_{0}^{*}=\frac{p_{0}^{*}(s_{0})}{p_{0}(s_{0})}\cdot
p_{0}(z)\lim_{k\to\infty}\int_{s}^{t}p_{k}\geq\lim_{k\to\infty}\frac{p_{0}^{*}(s_{0})}{p_{0}(s_{0})}\cdot
p_{k}(z)\int_{s}^{t}p_{0}=p_{0}^{*}(z)\int_{s}^{t}p_{0}$ (65)

for $z\in\\{s,t\\}$. Otherwise, it follows from (65) that

$\displaystyle\displaystyle
p_{0}(s)\int_{s}^{\infty}p_{0}^{*}=p_{0}(s)\cdot\sup_{t^{\prime}\in\mathcal{T}}\int_{s}^{t^{\prime}}p_{0}^{*}\geq
p_{0}^{*}(s)\cdot\sup_{t^{\prime}\in\mathcal{T}}\int_{s}^{t^{\prime}}p_{0}=p_{0}^{*}(s)\int_{s}^{\infty}p_{0}\quad$
$\displaystyle\text{if }-\infty<s<t_{\max}=t=\infty$
$\displaystyle\displaystyle
p_{0}(t)\int_{-\infty}^{t}p_{0}^{*}=p_{0}(t)\cdot\sup_{s^{\prime}\in\mathcal{T}}\int_{s^{\prime}}^{t}p_{0}^{*}\geq
p_{0}^{*}(t)\cdot\sup_{s^{\prime}\in\mathcal{T}}\int_{s^{\prime}}^{t}p_{0}=p_{0}^{*}(t)\int_{-\infty}^{t}p_{0}\quad$
$\displaystyle\text{if }-\infty=s=t_{\min}<t<\infty,$

which completes the proof of (64). In particular, for $z\in\mathcal{T}$,
taking $(s,t)=(-\infty,z)$ and $(s,t)=(z,\infty)$ in (64), we obtain

$p_{0}(z)\int_{z}^{\infty}p_{0}^{*}\geq
p_{0}^{*}(z)\int_{z}^{\infty}p_{0}\qquad\text{and}\qquad
p_{0}(z)\int_{-\infty}^{z}p_{0}^{*}\geq p_{0}^{*}(z)\int_{-\infty}^{z}p_{0},$

which proves (23). Summing these inequalities yields $p_{0}(z)\geq
p_{0}^{*}(z)$, as required.

(b) Since $s_{0}\in\mathcal{T}$ and
$p_{0}^{*}(z)=p_{0}^{*}(s_{0})\exp\bigl{(}\int_{s_{0}}^{z}\psi_{0}^{*}\bigr{)}\leq
p_{0}^{*}(s_{0})$ for $z\in\mathbb{R}$, we deduce that
$\|{p_{0}^{*}}\|_{\infty}=p_{0}^{*}(s_{0})\leq
p_{0}(s_{0})\leq\|{p_{0}}\|_{\infty}$. Finally, by Theorem 2(c),
$i^{*}(p_{0})=\bigl{(}\int_{\mathbb{R}}(\psi_{0}^{*})^{2}\,p_{0}\big{/}V_{p_{0}}(\psi_{0}^{*})\bigr{)}^{1/2}=-\int_{\mathbb{R}}p_{0}\,d\psi_{0}^{*}$;
see the equality case of (47). Similarly,
$i(p_{0}^{*})=-\int_{\mathbb{R}}p_{0}^{*}\,d\psi_{0}^{*}$, so because
$p_{0}^{*}\leq p_{0}$ on the support $\mathcal{T}$ of the Lebesgue–Stieltjes
measure induced by $\psi_{0}^{*}$, we have

$i^{*}(p_{0})=-\int_{\mathbb{R}}p_{0}\,d\psi_{0}^{*}\geq-\int_{\mathbb{R}}p_{0}^{*}\,d\psi_{0}^{*}=i(p_{0}^{*}),$

as required. ∎

###  6.2 Additional examples for Section 2

######  Example 25.

For densities $p_{0}$ that are uniformly continuous and locally absolutely
continuous on $\mathbb{R}$, it is possible to have
$i^{*}(p_{0})<\infty=i(p_{0})$. This is unfavourable from the perspective of
statistical efficiency because $\mathrm{ARE}^{*}(p_{0})=0$ in this case. To
construct an example of such a density $p_{0}$, let

$J_{0}(u):=\begin{cases}u\biggl{(}2+\sin\Bigl{(}\dfrac{\pi}{4u}\Bigr{)}\biggr{)}\;&\text{for
}u\in(0,1/2]\\\ 0&\text{for }u=0\end{cases}\qquad\text{and}\qquad
J_{0}(u):=J_{0}(1-u)\;\;\text{for }u\in(1/2,1],$

so that $J_{0}$ is continuous on $[0,1]$ and infinitely differentiable on
$(0,1)$, with $u\leq J_{0}(u)\leq 3u$ for all $u\in[0,1/2]$. The least concave
majorant $\hat{J}_{0}$ of $J_{0}$ is given by $\hat{J}_{0}(u)=3\min(u,1-u)$
for $u\in[0,1]$. In addition, by the Cauchy–Schwarz inequality,

$\biggl{(}\int_{0}^{1}(J_{0}^{\prime})^{2}\biggr{)}^{1/2}\geq\int_{0}^{1}|J_{0}^{\prime}|=2\int_{0}^{1/2}\,\Bigl{|}2+\sin\Bigl{(}\frac{u}{4\pi}\Bigr{)}-\frac{\pi}{4u}\cos\Bigl{(}\frac{u}{4\pi}\Bigr{)}\Bigr{|}=\infty,$

so $J_{0}$ is not of bounded variation on $(0,1)$. Next, defining
$Q_{0}\colon(0,1)\to\mathbb{R}$ by $Q_{0}(u):=\int_{1/2}^{u}1/J_{0}$, we have
$\lim_{u\nearrow 1}Q_{0}(u)=-\lim_{u\searrow
0}Q_{0}(u)=\int_{0}^{1/2}1/J_{0}\geq\int_{0}^{1/2}1/(3u)\,du=\infty$. Then by
Lemma 19, $Q_{0}$ is a strictly increasing, continuously differentiable
bijection from $(0,1)$ to $\mathbb{R}$, and $p_{0}:=(Q_{0}^{-1})^{\prime}$ is
a strictly positive, uniformly continuous density on $\mathbb{R}$ with density
quantile function $J_{0}$. Moreover, by Lemma 1,
$p_{0}^{\prime}=(J_{0}^{\prime}\circ F_{0})\,p_{0}$ is continuous, so $p_{0}$
is locally absolutely continuous on $\mathbb{R}$. By Remark 3,

$i^{*}(p_{0})=\int_{0}^{1}\bigl{(}\hat{J}_{0}^{(\mathrm{R})}\bigr{)}^{2}=\int_{0}^{1}3^{2}=9<\infty=\int_{0}^{1}(J_{0}^{\prime})^{2}=i(p_{0}),$

so indeed $\mathrm{ARE}^{*}(p_{0})=0$. By modifying this construction
slightly, one can also exhibit a unimodal density $p_{0}$ with
$\mathrm{ARE}^{*}(p_{0})=0$.

######  Example 26.

For $z\in\mathbb{R}$, let

$p_{0}(z):=\frac{1}{2}(1+z^{2})^{-3/2},$

so that $p_{0}$ is the density of $W/\sqrt{2}$ when $W\sim t_{2}$. Then by
Dümbgen et al., (2011, Example 2.9), the log-concave maximum likelihood
projection $p_{0}^{\mathrm{ML}}$ is a standard Laplace density given by

$p_{0}^{\mathrm{ML}}(z)=\frac{1}{2}e^{-|z|}$

for $z\in\mathbb{R}$, with score function
$\psi_{0}^{\mathrm{ML}}(\cdot)=-\operatorname{sgn}(\cdot)$. Therefore, the
corresponding regression $M$-estimator
$\hat{\beta}_{\psi_{0}^{\mathrm{ML}}}\in\operatorname*{argmax}_{\beta\in\mathbb{R}^{d}}\sum_{i=1}^{n}\log
p_{0}^{\mathrm{ML}}(Y_{i}-X_{i}^{\top}\beta)=\operatorname*{argmin}_{\beta\in\mathbb{R}^{d}}\sum_{i=1}^{n}|Y_{i}-X_{i}^{\top}\beta|$
is the LAD estimator. On the other hand, by straightforward computations,

$\displaystyle\psi_{0}(z)$ $\displaystyle=(\log
p_{0})^{\prime}(z)=-\frac{3z}{1+z^{2}},$ $\displaystyle\psi_{0}^{*}(z)$
$\displaystyle=(\hat{J}_{0}^{(\mathrm{R})}\circ
F_{0})(z)=\psi_{0}\bigl{(}(z\wedge
3^{-1/2})\vee(-3^{-1/2})\bigr{)}=\Bigl{(}-\frac{3\sqrt{3}}{4}\Bigr{)}\vee\psi_{0}(z)\wedge\frac{3\sqrt{3}}{4}$

for $z\in\mathbb{R}$, so similarly to Example 11, the optimal convex loss is a
Huber-like function given by

$\ell_{0}^{*}(z)=-\log p_{0}^{*}(z)=-\int_{0}^{z}\psi_{0}^{*}+\log
2=\begin{cases}\,\dfrac{3\log\bigl{(}(1+z^{2})/2\bigr{)}}{2}&\text{for
}z\in[-3^{-1/2},3^{-1/2}]\\\\[6.0pt]
\,\dfrac{3\bigl{(}\sqrt{3}|z|-1+\log(4/9)\bigr{)}}{4}&\text{for
}z\in\mathbb{R}\setminus[-3^{-1/2},3^{-1/2}].\end{cases}$

This means that $p_{0}^{*}\neq p_{0}^{\mathrm{ML}}$, and
$\hat{\beta}_{\psi_{0}^{*}}\in\operatorname*{argmin}_{\beta\in\mathbb{R}^{d}}\sum_{i=1}^{n}\ell_{0}^{*}(Y_{i}-X_{i}^{\top}\beta)$
is usually different from $\hat{\beta}_{\psi_{0}^{\mathrm{ML}}}$, with the
ratio of their asymptotic covariances being equal to

$\frac{V_{p_{0}}(\psi_{0}^{*})}{V_{p_{0}}(\psi_{0}^{\mathrm{ML}})}=\frac{1/i^{*}(p_{0})}{1/\bigl{(}4p_{0}(0)^{2}\bigr{)}}=\frac{1}{i^{*}(p_{0})}=-\biggl{(}\int_{-1/\sqrt{3}}^{1/\sqrt{3}}\;p_{0}\,d\psi_{0}\biggr{)}^{-1}=\frac{80}{93}\approx
0.860.$

######  Example 27.

For $\alpha,\sigma>0$, consider the symmetrised Pareto density
$p_{0}\colon\mathbb{R}\to\mathbb{R}$ given by

$p_{0}(z):=\frac{\alpha\sigma^{\alpha}}{2(|z|+\sigma)^{\alpha+1}}.$

The corresponding distribution has a finite mean if and only if $\alpha>1$, in
which case its log-concave maximum likelihood projection (Dümbgen et al.,,
2011) is a Laplace density given by

$p_{0}^{\mathrm{ML}}(z):=\frac{\alpha-1}{2\sigma}\exp\biggl{(}-\frac{(\alpha-1)|z|}{\sigma}\biggr{)}$
(66)

for $z\in\mathbb{R}$; see Chen and Samworth, (2013, p. 1382) or Samworth and
Shah, (2024, Exercise 10.14). On the other hand, for general
$\alpha,\sigma>0$, routine calculations yield

$J_{0}(u)=\frac{\alpha\bigl{(}2\min(u,1-u)\bigr{)}^{1+1/\alpha}}{2\sigma}\quad\text{and}\quad\hat{J}_{0}(u)=\frac{\alpha\min(u,1-u)}{\sigma}$

for $u\in[0,1]$. Since $p_{0}$ is symmetric about 0, it follows that
$\psi_{0}^{*}(z)=(1-2\mathbbm{1}_{\\{z\geq 0\\}})\alpha/\sigma$ and hence

$\ell_{0}^{*}(z)=-\log
p_{0}^{*}(z)=\log\Bigl{(}\frac{\alpha}{2\sigma}\Bigr{)}-\frac{\alpha|z|}{\sigma},\qquad
p_{0}^{*}(z)=\frac{\alpha}{2\sigma}\exp\Bigl{(}-\frac{\alpha|z|}{\sigma}\Bigr{)}$

for $z\in\mathbb{R}$. Therefore, $p_{0}^{*}$ is also a Laplace density, but
while $p_{0}^{\mathrm{ML}}\neq p_{0}^{*}$ when $\alpha>1$, both corresponding
regression $M$-estimators $\hat{\beta}_{\psi_{0}^{\mathrm{ML}}}$ and
$\hat{\beta}_{\psi_{0}^{*}}$ in (21) minimise
$\beta\mapsto\sum_{i=1}^{n}|Y_{i}-X_{i}^{\top}\beta|$ over $\mathbb{R}^{d}$,
and hence coincide when there is a unique minimiser (i.e. least absolute
deviation estimator).

######  Proposition 28.

For $\rho\in(0,1)$, $\mu_{1},\mu_{2}\in\mathbb{R}$ and $\sigma>0$, let
$P_{0}:=(1-\rho)N(\mu_{1},\sigma^{2})+\rho N(\mu_{2},\sigma^{2})$ and denote
by $p_{0}^{\mathrm{ML}}$ and $p_{0}^{*}$ its log-concave maximum likelihood
and Fisher divergence projections respectively.

  1. (a)

If $|\mu_{1}-\mu_{2}|\leq 2\sigma$, then $P_{0}$ has density
$p_{0}^{\mathrm{ML}}=p_{0}^{*}$.

  2. (b)

If $|\mu_{1}-\mu_{2}|>2\sigma$, then $p_{0}^{\mathrm{ML}}\neq p_{0}^{*}$.
Moreover, $\psi_{0}^{*}$ is continuous on $\mathbb{R}$, and there exist
$z_{1}\in(-\infty,a]$ and $z_{2}\in[b,\infty)$ such that $\psi_{0}^{*}$ is
decreasing on $(-\infty,z_{1}]\cup[z_{2},\infty)$ and constant on
$[z_{1},z_{2}]$.

###### Proof.

By Cule et al., (2010) and Samworth and Shah, (2024, Exercise 10.11), $p_{0}$
is log-concave if and only if $|\mu_{1}-\mu_{2}|\leq 2\sigma$. We will
slightly refine this result by first noting that

$\displaystyle p_{0}(z)$
$\displaystyle=\frac{1-\rho}{\sqrt{2\pi}\sigma}\exp\biggl{(}-\frac{(z-\mu_{1})^{2}}{2\sigma^{2}}\biggr{)}+\frac{\rho}{\sqrt{2\pi}\sigma}\exp\biggl{(}-\frac{(z-\mu_{2})^{2}}{2\sigma^{2}}\biggr{)}$
$\displaystyle\psi_{0}(z)$ $\displaystyle=(\log
p_{0})^{\prime}(z)=\frac{1}{p_{0}(z)}\biggl{\\{}\frac{(1-\rho)(\mu_{1}-z)}{\sqrt{2\pi}\sigma^{3}}\exp\biggl{(}-\frac{(z-\mu_{1})^{2}}{2\sigma^{2}}\biggr{)}+\frac{\rho(\mu_{2}-z)}{\sqrt{2\pi}\sigma^{3}}\exp\biggl{(}-\frac{(z-\mu_{2})^{2}}{2\sigma^{2}}\biggr{)}\biggr{\\}}$
$\displaystyle\psi_{0}^{\prime}(z)$ $\displaystyle=(\log
p_{0})^{\prime\prime}(z)=-\frac{g(z)}{2\pi\sigma^{4}p_{0}(z)^{2}}\exp\biggl{(}-\frac{(z-\mu_{1})^{2}+(z-\mu_{2})^{2}}{2\sigma^{2}}\biggr{)}$

for $z\in\mathbb{R}$, where

$\displaystyle
g(z):=(1-\rho)^{2}\exp\Bigl{(}-\frac{\mu_{2}-\mu_{1}}{2\sigma^{2}}(2z-\mu_{1}-\mu_{2})\Bigr{)}$
$\displaystyle+\rho^{2}\exp\Bigl{(}\frac{\mu_{2}-\mu_{1}}{2\sigma^{2}}(2z-\mu_{1}-\mu_{2})\Bigr{)}$
$\displaystyle+\rho(1-\rho)\Bigl{\\{}2-\Bigl{(}\frac{\mu_{1}-\mu_{2}}{\sigma}\Bigr{)}^{2}\Bigr{\\}}$

is a convex function of $z$ with

$\min_{z\in\mathbb{R}}g(z)=\rho(1-\rho)\Bigl{\\{}4-\Bigl{(}\frac{\mu_{1}-\mu_{2}}{\sigma}\Bigr{)}^{2}\Bigr{\\}}.$

(a) If $|\mu_{1}-\mu_{2}|\leq 2\sigma$, then $g\geq 0$ on $\mathbb{R}$ and
hence $P_{0}$ has a log-concave density $p_{0}=p_{0}^{\mathrm{ML}}=p_{0}^{*}$.

(b) Otherwise, if $|\mu_{1}-\mu_{2}|>2\sigma$, then there exist $a<b$ such
that $g\geq 0$ and hence $\psi_{0}^{\prime}\leq 0$ on
$(-\infty,a]\cup[b,\infty)$, while $g\leq 0$ and $\psi_{0}^{\prime}\geq 0$ on
$[a,b]$. Therefore, by Dümbgen et al., (2011, Examples 2.11(ii) and 2.12),
there exist $a^{\prime}\in(-\infty,a]$ and $b^{\prime}\in[b,\infty)$ such that
$\phi:=\log p_{0}^{\mathrm{ML}}$ agrees with $\phi_{0}:=\log p_{0}$ on
$(-\infty,a^{\prime}]\cup[b^{\prime},\infty)$, and $\phi$ is affine on
$[a^{\prime},b^{\prime}]$.

Suppose for a contradiction that $p_{0}^{\mathrm{ML}}$ is differentiable on
$\mathbb{R}$. Then
$\psi_{0}(a^{\prime})=\phi_{0}^{\prime}(a^{\prime})=\phi^{\prime}(a^{\prime})=\phi^{\prime}(b^{\prime})=\psi_{0}(b^{\prime})$,
so defining $\ell\colon\mathbb{R}\to\mathbb{R}$ by
$\ell(z):=\phi_{0}(a^{\prime})+\psi_{0}(a^{\prime})(z-a^{\prime})$, we have
$\phi=\ell$ on $[a^{\prime},b^{\prime}]$ and
$\ell(z)=\phi_{0}(b^{\prime})+\psi_{0}(b^{\prime})(z-b^{\prime})$ for
$z\in\mathbb{R}$. Since $\phi_{0}$ is concave on both $(-\infty,a]$ and
$[b,\infty)$ while being convex on $[a,b]$, it follows that $\phi_{0}\leq\ell$
on $(-\infty,a]\cup[b,\infty)$ and

$\phi_{0}(z)\leq\frac{b-z}{b-a}\,\phi_{0}(a)+\frac{z-a}{b-a}\,\phi_{0}(b)\leq\frac{b-z}{b-a}\,\ell(a)+\frac{z-a}{b-a}\,\ell(b)=\ell(z)=\phi(z)$

for all $z\in[a,b]$. Therefore, $\log p_{0}=\phi_{0}\leq\phi=\log
p_{0}^{\mathrm{ML}}$ on $\mathbb{R}$, but since
$\int_{\mathbb{R}}p_{0}=\int_{\mathbb{R}}p_{0}^{\mathrm{ML}}=1$, we must have
$p_{0}=p_{0}^{\mathrm{ML}}$ because both functions are continuous. However,
this contradicts the fact that $p_{0}$ is not log-concave, so
$p_{0}^{\mathrm{ML}}$ is not differentiable (at either $a^{\prime}$ or
$b^{\prime}$).

On the other hand, $p_{0}$ and its corresponding quantile function
$F_{0}^{-1}$ are differentiable on $\mathbb{R}$, so $J_{0}=p_{0}\circ
F_{0}^{-1}$ is differentiable on $(0,1)$. By Lemma 47, $\hat{J}_{0}$ is
continuously differentiable on $(0,1)$, so $(\log
p_{0}^{*})^{(\mathrm{R})}=\psi_{0}^{*}=\hat{J}_{0}^{(\mathrm{R})}\circ F_{0}$
is continuous on $\mathbb{R}$. Therefore, $\log p_{0}^{*}$ is differentiable
on $\mathbb{R}$ and hence $p_{0}^{\mathrm{ML}}\neq p_{0}^{*}$.

Moreover, by the first line of the proof of (b), $J_{0}^{\prime}=\psi_{0}\circ
F_{0}^{-1}$ is increasing on $[F_{0}(a),F_{0}(b)]$, and decreasing on both
$(0,F_{0}(a)]$ and $[F_{0}(b),1)$. We deduce from Lemma 48 that there exist
$z_{1}\in(-\infty,a]$ and $z_{2}\in[b,\infty)$ such that $\psi_{0}^{*}$ is
decreasing on $(-\infty,z_{1}]\cup[z_{2},\infty)$ and constant on
$[z_{1},z_{2}]$. ∎

###  6.3 Proofs for Section 3

###### Proof of Proposition 13.

In both (a) and (b), $X_{1}$ and $\varepsilon_{1}$ are independent, so by the
final assertion of Lemma 1, we have
$\mathbb{E}\bigl{\\{}X_{1}\psi_{0}^{*}(Y_{1}-X_{1}^{\top}\beta_{0})\bigr{\\}}=\mathbb{E}(X_{1})\,\mathbb{E}\psi_{0}^{*}(\varepsilon_{1})=0$.
Therefore, $(\beta_{0},\psi_{0}^{*})$ satisfies the constraint (29). Moreover,
$Y_{1}-X_{1}^{\top}\beta_{0}=\varepsilon_{1}$ has density
$q_{\beta_{0}}=p_{0}$, so by Theorem 2(_b_ ,  _c_),

$Q(\beta_{0},\psi_{0}^{*})=D_{p_{0}}(\psi_{0}^{*})=-\frac{1}{i^{*}(p_{0})}=\inf_{\psi\in\Psi_{\downarrow}(p_{0})}D_{p_{0}}(\psi)=\inf_{\psi\in\Psi_{\downarrow}(p_{0})}Q(\beta_{0},\psi).$

_(a)_ Since $p_{0}$ is symmetric and has a continuous distribution function
$F_{0}$, we have $F_{0}(z)=1-F_{0}(-z)$ for every $z\in\mathbb{R}$, so
$J_{0}(u):=(p_{0}\circ
F_{0}^{-1})(u)=p_{0}\bigl{(}-F_{0}^{-1}(1-u)\bigr{)}=J_{0}(1-u)$ and hence
$\hat{J}_{0}(u)=\hat{J}_{0}(1-u)$ for all $u\in[0,1]$. For every
$z\in\mathbb{R}$, we have $F_{0}(z)\in(0,1)$ because
$\mathcal{S}(p_{0})=\mathbb{R}$, so

$\psi_{0}^{*}(-z)=\hat{J}_{0}^{(\mathrm{R})}\bigl{(}F_{0}(-z)\bigr{)}=\hat{J}_{0}^{(\mathrm{R})}\bigl{(}1-F_{0}(z)\bigr{)}=-\hat{J}_{0}^{(\mathrm{L})}\bigl{(}F_{0}(z)\bigr{)}=-\lim_{u\nearrow
F_{0}(z)}\hat{J}_{0}^{(\mathrm{R})}(u)=-\lim_{z^{\prime}\nearrow
z}\psi_{0}^{*}(z^{\prime}),$ (67)

where the penultimate equality follows from Rockafellar, (1997, Theorem 24.1).
Thus,
$\psi_{0}^{*}\in\Psi_{\downarrow}^{\mathrm{anti}}(p_{0})=\Psi_{\downarrow}^{\mathrm{anti}}(q_{\beta_{0}})$.

Next, we claim that if $(\beta,\psi)\in\Gamma^{\mathrm{anti}}$ is such that
$\beta\neq\beta_{0}$, then necessarily $Q(\beta,\psi)\geq
0>Q(\beta_{0},\psi_{0}^{*})$, which proves _(a)_. Indeed,
$\mathbb{E}(X_{1}X_{1}^{\top})$ is positive definite by assumption, so
$T:=X_{1}^{\top}(\beta-\beta_{0})$ satisfies
$\mathbb{E}(T^{2})=(\beta-\beta_{0})^{\top}\mathbb{E}(X_{1}X_{1}^{\top})(\beta-\beta_{0})>0$
and hence $\mathbb{P}(T\neq 0)>0$. By (29),
$\mathbb{E}\bigl{(}T\psi(\varepsilon_{1}-T)\bigr{)}=0$, so because $T$ and
$\varepsilon_{1}$ are independent, there exists $t\neq 0$ such that
$\mathbb{E}|\psi(\varepsilon_{1}-t)|<\infty$. Since
$\varepsilon_{1}\overset{d}{=}-\varepsilon_{1}$ and $\psi$ agrees Lebesgue
almost everywhere on $\mathbb{R}$ with an antisymmetric function, we have
$\mathbb{E}\psi(\varepsilon_{1}+t)=\mathbb{E}\psi(-\varepsilon_{1}+t)=-\mathbb{E}\psi(\varepsilon_{1}-t)\in\mathbb{R}$.
Finally, since
$|\psi(\varepsilon_{1})|\leq|\psi(\varepsilon_{1}-t)|\vee|\psi(\varepsilon_{1}+t)|$,
this shows that $\mathbb{E}|\psi(\varepsilon_{1})|<\infty$. Thus,
$\mathbb{E}\psi(\varepsilon_{1})=\mathbb{E}\psi(-\varepsilon_{1})=-\mathbb{E}\psi(\varepsilon_{1})$,
so again because $T$ and $\varepsilon_{1}$ are independent,

$\mathbb{E}\bigl{(}T\psi(\varepsilon_{1})\bigr{)}=\mathbb{E}(T)\,\mathbb{E}\psi(\varepsilon_{1})=0=\mathbb{E}\bigl{(}T\psi(\varepsilon_{1}-T)\bigr{)}.$
(68)

For $z\in\mathbb{R}$, we have

$T^{2}\int_{0}^{1}p_{0}(z+sT)\,ds=T\int_{z}^{z+T}p_{0}(w)\,dw=\int_{-\infty}^{\infty}Tp_{0}(w)\bigl{(}\mathbbm{1}_{\\{w-T<z\leq
w\\}}-\mathbbm{1}_{\\{w<z\leq w-T\\}}\bigr{)}\,dw.$

Hence, by Fubini’s theorem and the independence of $T$ and $\varepsilon_{1}$,
we have

$\displaystyle\mathbb{E}\int_{0}^{1}\int_{\mathbb{R}}T^{2}p_{0}(z+sT)\,d\psi(z)\,ds$
$\displaystyle=\mathbb{E}\int_{-\infty}^{\infty}p_{0}(w)\int_{\mathbb{R}}T\bigl{(}\mathbbm{1}_{\\{w-T<z\leq
w\\}}-\mathbbm{1}_{\\{w<z\leq w-T\\}}\bigr{)}\,d\psi(z)\,dw$
$\displaystyle=\mathbb{E}\int_{-\infty}^{\infty}T\bigl{(}\psi(w)-\psi(w-T)\bigr{)}\,p_{0}(w)\,dw$
$\displaystyle=\mathbb{E}\bigl{(}T\psi(\varepsilon_{1})\bigr{)}-\mathbb{E}\bigl{(}T\psi(\varepsilon_{1}-T)\bigr{)}=0.$

Defining $g\colon[0,1]\to\mathbb{R}$ by
$g(s):=-\mathbb{E}\int_{\mathbb{R}}T^{2}p_{0}(z+sT)\,d\psi(z)\geq 0$, we
therefore have $\int_{0}^{1}g=0$, so $g=0$ Lebesgue almost everywhere. Now
$p_{0}$ is continuous on $\mathbb{R}$, so by Fatou’s lemma,

$\displaystyle 0$
$\displaystyle\leq-\mathbb{E}\int_{\mathbb{R}}T^{2}p_{0}(z+T)\,d\psi(z)=g(1)\leq\liminf_{s\nearrow
1}g(s)=0,$ (69) $\displaystyle 0$
$\displaystyle\leq-\mathbb{E}(T^{2})\int_{\mathbb{R}}p_{0}(z)\,d\psi(z)=-\mathbb{E}\int_{\mathbb{R}}T^{2}p_{0}(z)\,d\psi(z)=g(0)\leq\liminf_{s\searrow
0}g(s)=0.$ (70)

By (69), $\int_{\mathbb{R}}p_{0}(z+T)\mathbbm{1}_{\\{T\neq 0\\}}\,d\psi(z)=0$
almost surely. Moreover, since $\mathbb{E}(T^{2})>0$, (70) implies that
$\int_{\mathbb{R}}p_{0}(z+T)\mathbbm{1}_{\\{T=0\\}}\,d\psi(z)=\mathbbm{1}_{\\{T=0\\}}\int_{\mathbb{R}}p_{0}(z)\,d\psi(z)=0$
almost surely, so

$\mathbb{E}\int_{\mathbb{R}}p_{0}\bigl{(}z+X_{1}^{\top}(\beta-\beta_{0})\bigr{)}\,d\psi(z)=\mathbb{E}\int_{\mathbb{R}}p_{0}(z+T)\,d\psi(z)=0.$
(71)

Hence, by the definition of $Q(\beta,\psi)$ in (28),

$Q(\beta,\psi)=\mathbb{E}\int_{-\infty}^{\infty}\psi^{2}\bigl{(}z-X_{1}^{\top}(\beta-\beta_{0})\bigr{)}\,p_{0}(z)\,dz\geq
0>Q(\beta_{0},\psi_{0}^{*}),$

as claimed.

_(b)_ For $c\in\mathbb{R}$, $\beta\in\mathbb{R}^{d}$,
$\psi\in\Psi_{\downarrow}(q_{\beta})$ and
$x=(\tilde{x},1)\in\mathbb{R}^{d-1}\times\mathbb{R}$, we have
$x^{\top}(\beta_{0}-\beta)=x^{\top}(\beta_{0}-\beta-c\mathsf{e}_{d})+c$, so
$\psi_{c}(\cdot):=\psi(\cdot+c)$ satisfies

$\displaystyle\int_{-\infty}^{\infty}\psi_{c}^{2}\bigl{(}z+x^{\top}(\beta_{0}-\beta-c\mathsf{e}_{d})\bigr{)}\,p_{0}(z)\,dz$
$\displaystyle=\int_{-\infty}^{\infty}\psi^{2}\bigl{(}z+x^{\top}(\beta_{0}-\beta)\bigr{)}\,p_{0}(z)\,dz,$
$\displaystyle\text{and}\quad\int_{\mathbb{R}}p_{0}\bigl{(}z-x^{\top}(\beta_{0}-\beta-c\mathsf{e}_{d})\bigr{)}\,d\psi_{c}(z)$
$\displaystyle=\int_{\mathbb{R}}p_{0}\bigl{(}z-x^{\top}(\beta_{0}-\beta-c\mathsf{e}_{d})-c\bigr{)}\,d\psi(z)$
$\displaystyle=\int_{\mathbb{R}}p_{0}\bigl{(}z-x^{\top}(\beta_{0}-\beta)\bigr{)}\,d\psi(z).$

It follows that if $X_{1d}=1$ almost surely, then
$Q(\beta+c\mathsf{e}_{d},\psi_{c})=Q(\beta,\psi)$.

For $\beta_{c}:=\beta_{0}+c\mathsf{e}_{d}$, we have
$q_{\beta_{c}}(\cdot)=p_{0}(\cdot+c)$, and
$\psi_{c}\in\Psi_{\downarrow}(q_{\beta_{c}})$ if and only if
$\psi\in\Psi_{\downarrow}(q_{\beta_{0}})=\Psi_{\downarrow}(p_{0})$. Thus,

$0>-\frac{1}{i^{*}(p_{0})}=Q(\beta_{0},\psi_{0}^{*})=\inf_{\psi\in\Psi_{\downarrow}(p_{0})}Q(\beta_{0},\psi)=\inf_{\psi_{c}\in\Psi_{\downarrow}(q_{\beta_{c}})}Q(\beta_{c},\psi_{c})$
(72)

for all $c\in\mathbb{R}$.

Next, for any $(\beta,\psi)\in\Gamma$, we claim that there exists
$c\in\mathbb{R}$ such that $\mathbb{E}\psi(\varepsilon_{1}-c)=0$. Indeed,
$X_{1d}=1$ almost surely, so $T=X_{1}^{\top}(\beta-\beta_{0})$ satisfies
$\mathbb{E}\psi(\varepsilon_{1}-T)=\mathbb{E}\psi(Y_{1}-X_{1}^{\top}\beta)=0$.
By the independence of $T$ and $\varepsilon$, it follows that
$\varPsi(t):=\int_{-\infty}^{\infty}\psi(z-t)\,p_{0}(z)\,dz=\mathbb{E}\psi(\varepsilon_{1}-t)$
is well-defined and finite for $P_{T}$-almost every $t\in\mathbb{R}$, where
$P_{T}$ denotes the distribution of $T$. Since $\psi$ is decreasing, $\varPsi$
is increasing, so there exist $t_{1}\leq t_{2}$ in the support of $P_{T}$ such
that $-\infty<\varPsi(t_{1})\leq 0\leq\varPsi(t_{2})<\infty$, and
$|\psi(\varepsilon_{1}-t)|\leq|\psi(\varepsilon_{1}-t_{1})|\vee|\psi(\varepsilon_{1}-t_{2})|$
for all $t\in[t_{1},t_{2}]$. It follows from the dominated convergence theorem
that $\varPsi(t)$ is continuous in $t\in[t_{1},t_{2}]$, so by the intermediate
value theorem, there exists $c\in[t_{1},t_{2}]$ such that
$\mathbb{E}\psi(\varepsilon_{1}-c)=\varPsi(c)=0$, as claimed.

By (29) together with the independence of
$\tilde{T}:=T-c=X_{1}^{\top}(\beta-\beta_{c})$ and $\varepsilon_{1}$, we
deduce that

$\mathbb{E}\bigl{(}\tilde{T}\psi(\varepsilon_{1}-c)\bigr{)}=0=\mathbb{E}\bigl{(}\tilde{T}\psi(\varepsilon_{1}-c-\tilde{T})\bigr{)}.$

This is similar to (68) in the proof of _(a)_ , but with $T$ and
$\varepsilon_{1}$ replaced by $\tilde{T}$ and $\varepsilon_{1}-c$
respectively.

Suppose now that $\beta\neq\beta_{c^{\prime}}$ for any
$c^{\prime}\in\mathbb{R}$. Since $\mathbb{E}(X_{1}X_{1}^{\top})$ is positive
definite by assumption,
$\mathbb{E}(\tilde{T}^{2})=(\beta-\beta_{c})^{\top}\mathbb{E}(X_{1}X_{1}^{\top})(\beta-\beta_{c})>0$.
By arguing similarly to _(a)_ and noting that $p_{c}(\cdot):=p_{0}(\cdot+c)$
is the density of $\varepsilon_{1}-c$, we conclude as in (71) that

$\mathbb{E}\int_{\mathbb{R}}p_{0}\bigl{(}z+X_{1}^{\top}(\beta-\beta_{0})\bigr{)}\,d\psi(z)=\mathbb{E}\int_{\mathbb{R}}p_{0}(z+T)\,d\psi(z)=\mathbb{E}\int_{\mathbb{R}}p_{c}(z+\tilde{T})\,d\psi(z)=0.$

Thus, if $(\beta,\psi)\in\Gamma$ is such that $\beta\neq\beta_{c^{\prime}}$
for any $c^{\prime}\in\mathbb{R}$, then

$Q(\beta,\psi)=\mathbb{E}\int_{-\infty}^{\infty}\psi^{2}\bigl{(}z-X_{1}^{\top}(\beta-\beta_{0})\bigr{)}\,p_{0}(z)\,dz\geq
0>Q(\beta_{0},\psi_{0}^{*}).$

Together with (72), this completes the proof. ∎

In the rest of this section, we establish the antitonic efficiency results in
Sections 3.1 and 3.2. We first present a unified analysis of the score
estimators in both settings, before completing the proofs of Theorems 14 and
16 in Sections 6.3.1 and 6.3.2 respectively.

Lemma 29 below demonstrates the $L^{2}(P_{0})$ consistency of the initial
score estimates $\tilde{\psi}_{n,j}$. It extends the proof of van der Vaart,
(1998, Lemma 25.64) to kernel density estimators based on out-of-sample
residuals $\\{\hat{\varepsilon}_{i}:i\in I_{j+1}\\}$ instead of the unobserved
regression errors $\varepsilon_{i}$. The three folds of the data will be
denoted by $\mathcal{D}_{j}\equiv\mathcal{D}_{n,j}:=\\{(X_{i},Y_{i}):i\in
I_{j}\\}$ for $j=1,2,3$.

######  Lemma 29.

Under  _(A2)–(A4)_, suppose that for $j\in\\{1,2,3\\}$, the pilot estimators
$\bar{\beta}_{n}^{(j)}$ in (30) are defined in terms of a differentiable,
antisymmetric and strictly decreasing function $\psi$ satisfying
$\mathbb{E}\psi(\varepsilon_{1})=0$. Then for each $j$, we have

$\int_{\mathbb{R}}(\tilde{\psi}_{n,j}-\psi_{0})^{2}\,p_{0}\overset{p}{\to}0$

as $n\to\infty$.

###### Proof.

It suffices to prove the result for
$\tilde{\psi}_{n}\equiv\tilde{\psi}_{n,1}$, since
$\tilde{\psi}_{n,2},\tilde{\psi}_{n,3}$ are obtained by permuting the three
folds of the data (which have roughly the same size). Since
$\mathbb{E}\\{X_{1}\psi(Y_{1}-X_{1}^{\top}\beta_{0})\\}=\mathbb{E}(X_{1})\,\mathbb{E}\psi(\varepsilon_{1})=0$,
it follows from e.g. Yohai and Maronna, (1979) or He and Shao, (2000) that
$\bar{\beta}_{n}\equiv\bar{\beta}_{n}^{(1)}$ satisfies
$\sqrt{n}(\bar{\beta}_{n}-\beta_{0})=O_{p}(1)$. We have
$nh_{n}^{2}=(nh_{n}^{3}\gamma_{n}^{2})(h_{n}\gamma_{n}^{2})^{-1}\to\infty$ by
(A2), so $\|{\bar{\beta}_{n}-\beta_{0}}\|=o_{p}(h_{n})$ as $n\to\infty$.

By (A4) and the Cauchy–Schwarz inequality,
$\|{p_{0}}\|_{\infty}\leq\int_{\mathbb{R}}|p_{0}^{\prime}|\leq\bigl{(}\int_{\\{p_{0}>0\\}}(p_{0}^{\prime})^{2}/p_{0}\bigr{)}^{1/2}=i(p_{0})^{1/2}<\infty$.
Since $K$ is twice continuously differentiable and supported on $[-1,1]$ by
(A3), we have
$\int_{\mathbb{R}}K^{2}\vee\int_{\mathbb{R}}(K^{\prime})^{2}<\infty$. Denote
by $P_{X}$ the distribution of $X_{1}$ on $\mathbb{R}^{d}$. For
$y\in\mathbb{R}$, $n\in\mathbb{N}$ and $h\equiv h_{n}$, let
$\tilde{p}_{n}(y)\equiv\tilde{p}_{n,1}(y)=|I_{2}|^{-1}\sum_{i\in
I_{2}}K_{h}(y-\hat{\varepsilon}_{i})$ and

$\displaystyle
p_{n}(y):=\mathbb{E}\bigl{(}\tilde{p}_{n}(y)\\!\bigm{|}\\!\mathcal{D}_{1}\bigr{)}$
$\displaystyle=\int_{\mathbb{R}^{d}}\int_{\mathbb{R}}K_{h}\bigl{(}y-z-x^{\top}(\beta_{0}-\bar{\beta}_{n})\bigr{)}p_{0}(z)\,dz\,dP_{X}(x)$
$\displaystyle=\int_{\mathbb{R}^{d}}\int_{\mathbb{R}}p_{0}\bigl{(}y-uh-x^{\top}(\beta_{0}-\bar{\beta}_{n})\bigr{)}K(u)\,du\,dP_{X}(x).$

Since $\|\bar{\beta}_{n}-\beta_{0}\|=o_{p}(h)$ and $p_{0}$ is uniformly
continuous on $\mathbb{R}$, it follows from the bounded convergence theorem
that $p_{n}(y)\overset{p}{\to}p_{0}(y)$ for every $y\in\mathbb{R}$. Moreover,
by Fubini’s theorem,

$\displaystyle\int_{\mathbb{R}}|p_{n}-p_{0}|$ $\displaystyle\hskip
14.22636pt=\int_{\mathbb{R}}\,\biggl{|}\int_{\mathbb{R}^{d}}\int_{\mathbb{R}}\bigl{\\{}p_{0}\bigl{(}y-uh-x^{\top}(\beta_{0}-\bar{\beta}_{n})\bigr{)}-p_{0}(y)\bigr{\\}}K(u)\,du\,dP_{X}(x)\biggr{|}\,dy$
$\displaystyle\hskip
14.22636pt\leq\int_{\mathbb{R}}\int_{\mathbb{R}^{d}}\int_{\mathbb{R}}\int_{\mathbb{R}}|p_{0}^{\prime}(z)|(\mathbbm{1}_{\\{y<z\leq
y-uh-x^{\top}(\beta_{0}-\bar{\beta}_{n})\\}}+\mathbbm{1}_{\\{y-uh-x^{\top}(\beta_{0}-\bar{\beta}_{n})<z\leq
y\\}})\,dz\,K(u)\,du\,dP_{X}(x)\,dy$ $\displaystyle\hskip
14.22636pt=\biggl{(}\int_{\mathbb{R}}|p_{0}^{\prime}(z)|\,dz\biggr{)}\int_{\mathbb{R}^{d}}\int_{\mathbb{R}}|uh+x^{\top}(\bar{\beta}_{n}-\beta_{0})|K(u)\,du\,dP_{X}(x)$
$\displaystyle\hskip
14.22636pt\leq\biggl{(}\int_{\mathbb{R}}|p_{0}^{\prime}|\biggr{)}\biggl{(}h\int_{\mathbb{R}}|u|K(u)\,du+\|{\bar{\beta}_{n}-\beta_{0}}\|\cdot\mathbb{E}\|{X_{1}}\|\biggr{)}=O_{p}(h).$
(73)

By (A2) and the dominated convergence theorem, we may differentiate under the
integral sign to obtain

$\displaystyle
p_{n}^{\prime}(y)=\mathbb{E}\bigl{(}\tilde{p}_{n}^{\prime}(y)\\!\bigm{|}\\!\mathcal{D}_{1}\bigr{)}$
$\displaystyle=\int_{\mathbb{R}^{d}}\int_{\mathbb{R}}K_{h}^{\prime}\bigl{(}y-z-x^{\top}(\beta_{0}-\bar{\beta}_{n})\bigr{)}p_{0}(z)\,dz\,dP_{X}(x).$

For $y\in\mathbb{R}$, we have

$\displaystyle\operatorname{Var}\bigl{(}\tilde{p}_{n}(y)\\!\bigm{|}\\!\mathcal{D}_{1}\bigr{)}$
$\displaystyle\leq\frac{1}{|I_{2}|}\int_{\mathbb{R}^{d}}\int_{\mathbb{R}}K_{h}\bigl{(}y-z-x^{\top}(\beta_{0}-\bar{\beta}_{n})\bigr{)}^{2}p_{0}(z)\,dz\,dP_{X}(x)\leq\frac{\|{p_{0}}\|_{\infty}\int_{\mathbb{R}}K^{2}}{|I_{2}|h},$
(74)
$\displaystyle\operatorname{Var}\bigl{(}\tilde{p}_{n}^{\prime}(y)\\!\bigm{|}\\!\mathcal{D}_{1}\bigr{)}$
$\displaystyle\leq\frac{1}{|I_{2}|}\int_{\mathbb{R}^{d}}\int_{\mathbb{R}}K_{h}^{\prime}\bigl{(}y-z-x^{\top}(\beta_{0}-\bar{\beta}_{n})\bigr{)}^{2}p_{0}(z)\,dz\,dP_{X}(x)\leq\frac{\|{p_{0}}\|_{\infty}\int_{\mathbb{R}}(K^{\prime})^{2}}{|I_{2}|h^{3}}.$
(75)

Since $|I_{2}|h_{n}^{3}\to\infty$ by (A2), it follows from Lemma 41 that
$\tilde{p}_{n}(y)-p_{n}(y)\overset{p}{\to}0$ and
$\tilde{p}_{n}^{\prime}(y)-p_{n}^{\prime}(y)\overset{p}{\to}0$. Defining
$\tau(z):=\int_{\mathbb{R}}|p_{0}^{\prime}(y-z)-p_{0}^{\prime}(y)|\,dy\leq
2\int_{\mathbb{R}}|p_{0}^{\prime}|$ for $z\in\mathbb{R}$, we have $\lim_{z\to
0}\tau(z)=0$ by continuity of translation (Folland,, 1999, Proposition 8.5).
Thus, since $\bar{\beta}_{n}-\beta_{0}\overset{p}{\to}0$, it follows from the
bounded convergence theorem that

$\displaystyle\int_{\mathbb{R}}|p_{n}^{\prime}-p_{0}^{\prime}|$
$\displaystyle=\int_{\mathbb{R}}\,\biggl{|}\int_{\mathbb{R}^{d}}\int_{\mathbb{R}}\bigl{\\{}p_{0}^{\prime}\bigl{(}y-uh-x^{\top}(\beta_{0}-\bar{\beta}_{n})\bigr{)}-p_{0}^{\prime}(y)\bigr{\\}}K(u)\,du\,dP_{X}(x)\biggr{|}\,dy$
$\displaystyle\leq\int_{\mathbb{R}^{d}}\int_{\mathbb{R}}\tau\bigl{(}uh+x^{\top}(\beta_{0}-\bar{\beta}_{n})\bigr{)}K(u)\,du\,dP_{X}(x)\overset{p}{\to}0.$

This means that every subsequence of $(p_{n})$ has a further subsequence
$(p_{n_{k}})$ such that with probability 1, we have $p_{n_{k}}^{\prime}\to
p_{0}^{\prime}$ and hence
$(p_{n_{k}}^{\prime})^{2}/p_{n_{k}}\to(p_{0}^{\prime})^{2}/p_{0}$ Lebesgue
almost everywhere. By the Cauchy–Schwarz inequality,

$\displaystyle\frac{p_{n}^{\prime}(y)^{2}}{p_{n}(y)}$
$\displaystyle=\frac{\bigl{(}\int_{\mathbb{R}^{d}}\int_{\mathbb{R}}p_{0}^{\prime}\bigl{(}y-uh-x^{\top}(\beta_{0}-\bar{\beta}_{n})\bigr{)}K(u)\,du\,dP_{X}(x)\bigr{)}^{2}}{\int_{\mathbb{R}^{d}}\int_{\mathbb{R}}p_{0}\bigl{(}y-uh-x^{\top}(\beta_{0}-\bar{\beta}_{n})\bigr{)}K(u)\,du\,dP_{X}(x)}$
$\displaystyle\leq\int_{\mathbb{R}^{d}}\int_{\mathbb{R}}\frac{(p_{0}^{\prime})^{2}}{p_{0}}\bigl{(}y-uh-x^{\top}(\beta_{0}-\bar{\beta}_{n})\bigr{)}K(u)\,du\,dP_{X}(x)$

for $y\in\mathbb{R}$ and $n\in\mathbb{N}$, so by Fubini’s theorem,

$\int_{\mathbb{R}}\frac{(p_{n}^{\prime})^{2}}{p_{n}}\leq\int_{\mathbb{R}}\frac{(p_{0}^{\prime})^{2}}{p_{0}}$
(76)

for all $n$. Thus, by applying a slight generalisation of Scheffé’s lemma (van
der Vaart,, 1998, Lemma 2.29) to subsequences of $(p_{n})$, we obtain

$\displaystyle\int_{\mathbb{R}}\Bigl{(}\frac{p_{n}^{\prime}}{\sqrt{p_{n}}}-\frac{p_{0}^{\prime}}{\sqrt{p_{0}}}\Bigr{)}^{2}\overset{p}{\to}0$
(77)

as $n\to\infty$. Since $\int_{\mathbb{R}}\psi_{0}^{2}\,p_{0}=i(p_{0})<\infty$
and $\alpha_{n}\to\infty$ while $\gamma_{n}\to 0$, we can argue similarly to
the proof of van der Vaart, (1998, Lemma 25.64) and deduce by the dominated
convergence theorem that

$\mathbb{E}\biggl{(}\int_{\tilde{S}_{n,1}^{c}}(\tilde{\psi}_{n}-\psi_{0})^{2}\,p_{0}\biggm{|}\mathcal{D}_{1}\biggr{)}=\int_{\mathbb{R}}\psi_{0}(y)^{2}\,p_{0}(y)\cdot\bigl{\\{}\mathbb{P}\bigl{(}|\tilde{p}_{n}^{\prime}(y)|>\alpha_{n}\\!\bigm{|}\\!\mathcal{D}_{1}\bigr{)}+\mathbb{P}\bigl{(}\tilde{p}_{n}(y)<\gamma_{n}\\!\bigm{|}\\!\mathcal{D}_{1}\bigr{)}\bigr{\\}}\,dy\overset{p}{\to}0.$

Moreover, on $\tilde{S}_{n,1}$, we have

$\displaystyle|\tilde{\psi}_{n}-\psi_{0}|\sqrt{p_{0}}$
$\displaystyle\leq\frac{|\tilde{p}_{n}^{\prime}|}{\tilde{p}_{n}}|\sqrt{p_{0}}-\sqrt{p_{n}}|+\frac{\sqrt{p_{n}}}{\tilde{p}_{n}}|\tilde{p}_{n}^{\prime}-p_{n}^{\prime}|+\frac{|p_{n}^{\prime}|}{\sqrt{p_{n}}}\frac{|p_{n}-\tilde{p}_{n}|}{|\tilde{p}_{n}|}+\Bigl{|}\frac{p_{n}^{\prime}}{\sqrt{p_{n}}}-\frac{p_{0}^{\prime}}{\sqrt{p_{0}}}\Bigr{|}$
$\displaystyle\leq\frac{\alpha_{n}}{\gamma_{n}}|p_{0}-p_{n}|^{1/2}+\frac{\sqrt{p_{n}}}{\gamma_{n}}|\tilde{p}_{n}^{\prime}-p_{n}^{\prime}|+\frac{|p_{n}^{\prime}|}{\sqrt{p_{n}}}\frac{|p_{n}-\tilde{p}_{n}|}{\gamma_{n}}+\Bigl{|}\frac{p_{n}^{\prime}}{\sqrt{p_{n}}}-\frac{p_{0}^{\prime}}{\sqrt{p_{0}}}\Bigr{|},$

so by (74)–(77) and (A2),

$\displaystyle\frac{1}{4}\,\mathbb{E}\biggl{(}\int_{\tilde{S}_{n,1}}(\tilde{\psi}_{n}-\psi_{0})^{2}\,p_{0}\biggm{|}\mathcal{D}_{1}\biggr{)}$
$\displaystyle\hskip
14.22636pt\leq\frac{\alpha_{n}^{2}}{\gamma_{n}^{2}}\int_{\mathbb{R}}|p_{n}-p_{0}|+\int_{\mathbb{R}}\frac{p_{n}(y)}{\gamma_{n}^{2}}\operatorname{Var}\bigl{(}\tilde{p}_{n}^{\prime}(y)\,|\,\mathcal{D}_{1}\bigr{)}\,dy+\int_{\mathbb{R}}\frac{(p_{n}^{\prime})^{2}}{\gamma_{n}^{2}\,p_{n}}(y)\operatorname{Var}\bigl{(}\tilde{p}_{n}(y)\,|\,\mathcal{D}_{1}\bigr{)}\,dy$
$\displaystyle\hskip
312.9803pt+\int_{\mathbb{R}}\Bigl{(}\frac{p_{n}^{\prime}}{\sqrt{p_{n}}}-\frac{p_{0}^{\prime}}{\sqrt{p_{0}}}\Bigr{)}^{2}$
$\displaystyle\hskip
14.22636pt\leq\frac{\alpha_{n}^{2}}{\gamma_{n}^{2}}\int_{\mathbb{R}}|p_{n}-p_{0}|+\frac{\|{p_{0}}\|_{\infty}\int_{\mathbb{R}}(K^{\prime})^{2}}{|I_{2}|h_{n}^{3}\gamma_{n}^{2}}+\frac{\|{p_{0}}\|_{\infty}\int_{\mathbb{R}}K^{2}}{|I_{2}|h_{n}\gamma_{n}^{2}}\int_{\mathbb{R}}\frac{(p_{0}^{\prime})^{2}}{p_{0}}+\int_{\mathbb{R}}\Bigl{(}\frac{p_{n}^{\prime}}{\sqrt{p_{n}}}-\frac{p_{0}^{\prime}}{\sqrt{p_{0}}}\Bigr{)}^{2}$
$\displaystyle\hskip
14.22636pt=O_{p}\Bigl{(}\frac{\alpha_{n}^{2}}{\gamma_{n}^{2}}h_{n}\Bigr{)}+O_{p}\Bigl{(}\frac{1}{|I_{2}|h_{n}^{3}\gamma_{n}^{2}}\Bigr{)}+o_{p}(1)=o_{p}(1).$

Therefore, by Lemma 41,
$\int_{\mathbb{R}}(\tilde{\psi}_{n}-\psi_{0})^{2}\,p_{0}\overset{p}{\to}0$. ∎

Next, we use Lemma 29 and properties of antitonic projections (Section 6.5) to
show that the projected score estimates $\hat{\psi}_{n,j}$ are consistent in
$L^{2}(P_{0})$ for the population-level antitonic projection $\psi_{0}^{*}$ of
$\psi_{0}$.

######  Lemma 30.

Suppose that  _(A1)–(A5)_ hold and that $\mathbb{E}\psi(\varepsilon_{1})=0$.
Then for $j\in\\{1,2,3\\}$, we have

$\int_{\mathbb{R}}(\hat{\psi}_{n,j}-\psi_{0}^{*})^{2}\,p_{0}\overset{p}{\to}0.$
(78)

Moreover,
$\hat{\Psi}_{n,j}(t):=\int_{\mathbb{R}}\hat{\psi}_{n,j}(z-t)\,p_{0}(z)\,dz\in\mathbb{R}$
for all $t\in\mathbb{R}$, and for any sequence $(u_{n})$ satisfying
$u_{n}(\alpha_{n}/\gamma_{n})\to 0$, we have

$\displaystyle\sup_{s,t\in[-u_{n},u_{n}]}\int_{\mathbb{R}}\bigl{(}\hat{\psi}_{n,j}(z-t)-\hat{\psi}_{n,j}(z-s)\bigr{)}^{2}\,p_{0}(z)\,dz\overset{p}{\to}0,$
(79) $\displaystyle\Delta_{n}(u_{n}):=$
$\displaystyle\sup_{\begin{subarray}{c}s,t\in[-u_{n},u_{n}]:\\\ s\neq
t\end{subarray}}\,\biggl{|}\frac{\hat{\Psi}_{n,j}(t)-\hat{\Psi}_{n,j}(s)}{t-s}-i^{*}(p_{0})\biggr{|}\overset{p}{\to}0$
(80)

as $n\to\infty$.

###### Proof.

Similarly to Lemma 29, it suffices to prove this result when $j=1$, so we will
drop the index $j$ in all of our notation below. In the terminology of Section
6.5,
$\hat{\psi}_{n}\equiv\hat{\psi}_{n,1}=\Pi_{\downarrow}(\tilde{\psi}_{n},\tilde{P}_{n})$
is defined as an $L^{2}(\tilde{P}_{n})$-antitonic projection of the initial
score estimator $\tilde{\psi}_{n}\equiv\tilde{\psi}_{n,1}$, where
$\tilde{P}_{n}\equiv\tilde{P}_{n,1}$ denotes the distribution with density
$\tilde{p}_{n}\equiv\tilde{p}_{n.1}$. For each $n$, let

$\tilde{\psi}_{n}^{*}:=\Pi_{\downarrow}(\tilde{\psi}_{n},P_{0})$

denote the $L^{2}(P_{0})$-antitonic projection of $\tilde{\psi}_{n}$. Then by
(116) and the definition (31) of $\tilde{\psi}_{n}$, we have
$\|{\hat{\psi}_{n}}\|_{\infty}\vee\|{\tilde{\psi}_{n}^{*}}\|_{\infty}\leq\|{\tilde{\psi}_{n}}\|_{\infty}\leq\alpha_{n}/\gamma_{n}<\infty$.
We have $\tilde{\psi}_{n}\in L^{2}(P_{0})\cap L^{2}(\tilde{P}_{n})$ for all
$n$, so by (114) in Lemma 44 together with the triangle inequality,

$\displaystyle\|{\hat{\psi}_{n}-\tilde{\psi}_{n}^{*}}\|_{L^{2}(P_{0})}^{2}$
$\displaystyle\leq\int_{\mathbb{R}}(\tilde{\psi}_{n}^{*}-\hat{\psi}_{n})(\hat{\psi}_{n}-\tilde{\psi}_{n})\,d(\tilde{P}_{n}-P_{0})$
$\displaystyle\leq\|{(\tilde{\psi}_{n}^{*}-\hat{\psi}_{n})(\hat{\psi}_{n}-\tilde{\psi}_{n})}\|_{\infty}\int_{\mathbb{R}}|\tilde{p}_{n}-p_{0}|\leq
4\Bigl{(}\frac{\alpha_{n}}{\gamma_{n}}\Bigr{)}^{2}\int_{\mathbb{R}}(|\tilde{p}_{n}-p_{n}|+|p_{n}-p_{0}|),$
(81)

where
$p_{n}(y)=\mathbb{E}\bigl{(}\tilde{p}_{n}(y)\\!\bigm{|}\\!\mathcal{D}_{1}\bigr{)}$
for $y\in\mathbb{R}$. By (A4),
$\mathbb{E}(|\varepsilon_{1}|^{\delta})=\int_{\mathbb{R}}|z|^{\delta}\,p_{0}(z)\,dz<\infty$
for some $\delta\in(0,1]$. Fix $\rho\in\bigl{(}0,\delta/(\delta+1)\bigr{)}$.
Then by Lemma 39, (A1), (A3) and the fact that
$\bar{\beta}_{n}\overset{p}{\to}\beta_{0}$ when
$\mathbb{E}\psi(\varepsilon_{1})=0$, we have

$\displaystyle\mathbb{E}\biggl{(}\int_{\mathbb{R}}|\tilde{p}_{n}-p_{n}|\Bigm{|}\mathcal{D}_{1}\biggr{)}$
$\displaystyle\leq\frac{2^{1-2\rho}\|{K}\|_{\infty}C_{\delta,\rho}^{\rho}}{(|I_{2}|h)^{\rho}}(1+h)^{\delta(1-\rho)}\Bigl{(}1+\int_{\mathbb{R}^{d}}\int_{\mathbb{R}}|z|^{\delta}\,p_{0}\bigl{(}z-x^{\top}(\bar{\beta}_{n}-\beta_{0})\bigr{)}\,dz\,dP_{X}(x)\Bigr{)}$
$\displaystyle=\frac{2^{1-2\rho}\|{K}\|_{\infty}C_{\delta,\rho}^{\rho}}{(|I_{2}|h)^{\rho}}(1+h)^{\delta(1-\rho)}\bigl{(}1+\mathbb{E}\bigl{\\{}|\varepsilon_{1}+X_{1}^{\top}(\bar{\beta}_{n}-\beta_{0})|^{\delta}\\!\bigm{|}\\!\mathcal{D}_{1}\bigr{\\}}\bigr{)}$
$\displaystyle\leq\frac{2^{1-2\rho}\|{K}\|_{\infty}C_{\delta,\rho}^{\rho}}{(|I_{2}|h)^{\rho}}(1+h)^{\delta(1-\rho)}\bigl{\\{}1+\mathbb{E}(|\varepsilon_{1}|^{\delta})+\mathbb{E}(\|{X_{1}}\|^{\delta})\cdot\|{\bar{\beta}_{n}-\beta_{0}}\|^{\delta}\bigr{\\}}$
$\displaystyle=O_{p}\bigl{(}(nh)^{-\rho}\bigr{)}$

as $n\to\infty$ and $h\to 0$ with $nh\to\infty$, where
$C_{\delta,\rho}=\int_{\mathbb{R}}(1+|z|)^{-\delta(1-\rho)/\rho}\,dz<\infty$.
Thus, by Lemma 41 and (A2),
$\int_{\mathbb{R}}|\tilde{p}_{n}-p_{n}|=O_{p}\bigl{(}(nh)^{-\rho}\bigr{)}=O_{p}(n^{-2\rho/3})$.
Combining this with (6.3), (81) and (A2) yields
$\|{\hat{\psi}_{n}-\tilde{\psi}_{n}^{*}}\|_{L^{2}(P_{0})}^{2}=O_{p}\bigl{(}(h_{n}\vee
n^{-2\rho/3})(\alpha_{n}/\gamma_{n})^{2}\bigr{)}=o_{p}(1)$ as $n\to\infty$.
Hence, by applying the triangle inequality for $\|{{\cdot}}\|_{L^{2}(P_{0})}$
together with (115) and Lemma 29, we obtain

$\displaystyle\biggl{(}\int_{\mathbb{R}}(\hat{\psi}_{n}-\psi_{0}^{*})^{2}\,p_{0}\biggr{)}^{1/2}=\|{\hat{\psi}_{n}-\psi_{0}^{*}}\|_{L^{2}(P_{0})}$
$\displaystyle\leq\|{\hat{\psi}_{n}-\tilde{\psi}_{n}^{*}}\|_{L^{2}(P_{0})}+\|{\tilde{\psi}_{n}^{*}-\psi_{0}^{*}}\|_{L^{2}(P_{0})}$
$\displaystyle\leq\|{\hat{\psi}_{n}-\tilde{\psi}_{n}^{*}}\|_{L^{2}(P_{0})}+\|{\tilde{\psi}_{n}-\psi_{0}}\|_{L^{2}(P_{0})}\overset{p}{\to}0,$

which proves (78).

For $t\in\mathbb{R}$, let $p_{t}(\cdot):=p_{0}(\cdot+t)$. We have
$\|{\hat{\psi}_{n}}\|_{\infty}\leq\|{\tilde{\psi}_{n}}\|_{\infty}\leq\alpha_{n}/\gamma_{n}$
for each $n$, so
$\hat{\Psi}_{n}(t)\equiv\hat{\Psi}_{n,1}(t)=\int_{\mathbb{R}}\hat{\psi}_{n}\,p_{t}$
is finite for every $t$. Moreover,

$\displaystyle\int_{\mathbb{R}}(\hat{\psi}_{n}-\psi_{0}^{*})^{2}\,p_{t}$
$\displaystyle\leq
3\biggl{(}\int_{\mathbb{R}}\hat{\psi}_{n}^{2}\bigl{(}\sqrt{p_{t}}-\sqrt{p_{0}}\bigr{)}^{2}+\int_{\mathbb{R}}\bigl{(}\hat{\psi}_{n}\sqrt{p_{0}}-\psi_{0}^{*}\sqrt{p_{0}}\bigr{)}^{2}+\int_{\mathbb{R}}(\psi_{0}^{*})^{2}\bigl{(}\sqrt{p_{0}}-\sqrt{p_{t}}\bigr{)}^{2}\biggr{)}$
$\displaystyle\leq
3\biggl{(}\Bigl{(}\frac{\alpha_{n}}{\gamma_{n}}\Bigr{)}^{2}\int_{\mathbb{R}}\bigl{(}\sqrt{p_{t}}-\sqrt{p_{0}}\bigr{)}^{2}+\int_{\mathbb{R}}(\hat{\psi}_{n}-\psi_{0}^{*})^{2}\,p_{0}+\int_{\mathbb{R}}(\psi_{0}^{*})^{2}\bigl{(}\sqrt{p_{0}}-\sqrt{p_{t}}\bigr{)}^{2}\biggr{)}$
(82)

and

$\displaystyle\int_{\mathbb{R}}\bigl{(}\hat{\psi}_{n}(z-t)-\hat{\psi}_{n}(z)\bigr{)}^{2}\,p_{0}(z)\,dz$
$\displaystyle\hskip 28.45274pt\leq
3\int_{\mathbb{R}}\bigl{\\{}\bigl{(}\hat{\psi}_{n}(z-t)-\psi_{0}^{*}(z-t)\bigr{)}^{2}+\bigl{(}\psi_{0}^{*}(z-t)-\psi_{0}^{*}(z)\bigr{)}^{2}+\bigl{(}\psi_{0}^{*}(z)-\hat{\psi}_{n}(z)\bigr{)}^{2}\bigr{\\}}\,p_{0}(z)\,dz$
$\displaystyle\hskip
28.45274pt=3\biggl{(}\int_{\mathbb{R}}(\hat{\psi}_{n}-\psi_{0}^{*})^{2}\,p_{t}+\int_{\mathbb{R}}\bigl{(}\psi_{0}^{*}(z-t)-\psi_{0}^{*}(z)\bigr{)}^{2}\,p_{0}(z)\,dz+\int_{\mathbb{R}}(\hat{\psi}_{n}-\psi_{0}^{*})^{2}\,p_{0}\biggr{)}.$
(83)

Since $\psi_{0}^{*}$ is decreasing, it is continuous Lebesgue almost
everywhere and
$(\psi_{0}^{*})^{2}(z-t)\leq(\psi_{0}^{*})^{2}(z-t_{0})\vee(\psi_{0}^{*})^{2}(z+t_{0})$
for all $z\in\mathbb{R}$ and $t\in[-t_{0},t_{0}]$. Thus, by (A5) and the
dominated convergence theorem,
$\int_{\mathbb{R}}(\psi_{0}^{*})^{2}\,p_{t}=\int_{\mathbb{R}}(\psi_{0}^{*})^{2}(z-t)\,p_{0}(z)\,dz\to\int_{\mathbb{R}}(\psi_{0}^{*})^{2}\,p_{0}$
as $t\to 0$. Hence by the continuity of $p_{0}$ and a slight generalisation of
Scheffé’s lemma (van der Vaart,, 1998, Lemma 2.29),

$\int_{\mathbb{R}}(\psi_{0}^{*})^{2}\bigl{(}\sqrt{p_{0}}-\sqrt{p_{t}}\bigr{)}^{2}\to
0\quad\text{and}\quad\int_{\mathbb{R}}\bigl{(}\psi_{0}^{*}(z-t)-\psi_{0}^{*}(z)\bigr{)}^{2}p_{0}(z)\,dz\to
0$ (84)

as $t\to 0$. Furthermore, $i(p_{0})<\infty$ by (A4), so Lemma 37 implies that

$\int_{\mathbb{R}}\,(\sqrt{p_{t}}-\sqrt{p_{0}})^{2}=O(t^{2})$ (85)

as $t\to 0$. Therefore, letting $(u_{n})$ be such that
$u_{n}(\alpha_{n}/\gamma_{n})\to 0$, we deduce from (78), (82) and (84) that

$\sup_{t\in[-u_{n},u_{n}]}\int_{\mathbb{R}}(\hat{\psi}_{n}-\psi_{0}^{*})^{2}\,p_{t}\overset{p}{\to}0.$
(86)

Hence by (83) and (84),

$\sup_{s,t\in[-u_{n},u_{n}]}\int_{\mathbb{R}}\bigl{(}\hat{\psi}_{n}(z-t)-\hat{\psi}_{n}(z-s)\bigr{)}^{2}\,p_{0}(z)\,dz\leq
4\sup_{t\in[-u_{n},u_{n}]}\int_{\mathbb{R}}\bigl{(}\hat{\psi}_{n}(z-t)-\hat{\psi}_{n}(z)\bigr{)}^{2}\,p_{0}(z)\,dz\overset{p}{\to}0.$

This yields (79). Next, define
$\Psi_{0}^{*}(t):=\int_{\mathbb{R}}\psi_{0}^{*}(z-t)\,p_{0}(z)\,dz=\int_{\mathbb{R}}\psi_{0}^{*}\,p_{t}$
for $t\in[-t_{0},t_{0}]$, and observe that $\Psi_{0}^{*}(t)\in\mathbb{R}$ for
all $t\in[-t_{0},t_{0}]$ by (A5) and the Cauchy–Schwarz inequality. Then by
Cauchy–Schwarz again,

$\displaystyle\bigl{|}\hat{\Psi}_{n}(t)-\hat{\Psi}_{n}(s)-\bigl{(}\Psi_{0}^{*}(t)-\Psi_{0}^{*}(s)\bigr{)}\bigr{|}^{2}$
$\displaystyle=\biggl{|}\int_{\mathbb{R}}(\hat{\psi}_{n}-\psi_{0}^{*})(p_{t}-p_{s})\biggr{|}^{2}$
$\displaystyle\leq\biggl{\\{}\int_{\mathbb{R}}(\hat{\psi}_{n}-\psi_{0}^{*})^{2}\bigl{(}\sqrt{p_{t}}+\sqrt{p_{s}}\bigr{)}^{2}\biggr{\\}}\biggl{\\{}\int_{\mathbb{R}}\bigl{(}\sqrt{p_{t}}-\sqrt{p_{s}}\bigr{)}^{2}\biggr{\\}}$
$\displaystyle\leq\biggl{\\{}2\int_{\mathbb{R}}(\hat{\psi}_{n}-\psi_{0}^{*})^{2}(p_{t}+p_{s})\biggr{\\}}\biggl{\\{}\int_{\mathbb{R}}\bigl{(}\sqrt{p_{t}}-\sqrt{p_{s}}\bigr{)}^{2}\biggr{\\}}$

for $s,t\in[-t_{0},t_{0}]$. By (85),
$\int_{\mathbb{R}}\bigl{(}\sqrt{p_{t}}-\sqrt{p_{s}}\bigr{)}^{2}=\int_{\mathbb{R}}\bigl{(}\sqrt{p_{t-s}}-\sqrt{p_{0}}\bigr{)}^{2}=O\bigl{(}(t-s)^{2}\bigr{)}$
as $s,t\to 0$, so it follows from (86) that

$\sup_{\begin{subarray}{c}s,t\in[-u_{n},u_{n}]:\\\ s\neq
t\end{subarray}}\,\biggl{|}\frac{\hat{\Psi}_{n}(t)-\hat{\Psi}_{n}(s)-\bigl{(}\Psi_{0}^{*}(t)-\Psi_{0}^{*}(s)\bigr{)}}{t-s}\biggr{|}=o_{p}(1)$
(87)

as $n\to\infty$. Similarly, by the Cauchy–Schwarz inequality,

$\displaystyle\bigl{|}\Psi_{0}^{*}(t)-\Psi_{0}^{*}(s)-\bigl{(}\Psi_{0}^{*}(t-s)-\Psi_{0}^{*}(0)\bigr{)}\bigr{|}^{2}$
$\displaystyle=\biggl{|}\int_{\mathbb{R}}\bigl{(}\psi_{0}^{*}(z-s)-\psi_{0}^{*}(z)\bigr{)}\bigl{(}p_{0}(z+t-s)-p_{0}(z)\bigr{)}\,dz\biggr{|}^{2}$
$\displaystyle\leq
2\\!\int_{\mathbb{R}}\bigl{(}\psi_{0}^{*}(z-s)-\psi_{0}^{*}(z)\bigr{)}^{2}(p_{t-s}+p_{0})(z)\,dz\\!\int_{\mathbb{R}}\bigl{(}\sqrt{p_{t-s}}-\sqrt{p_{0}}\bigr{)}^{2}.$

Since
$\int_{\mathbb{R}}\bigl{(}\psi_{0}^{*}(z-s)-\psi_{0}^{*}(z)\bigr{)}^{2}p_{t-s}(z)\,dz=\int_{\mathbb{R}}\bigl{(}\psi_{0}^{*}(z-t)-\psi_{0}^{*}(z+s-t)\bigr{)}^{2}p_{0}(z)\,dz$,
we deduce from (84) and (85) that

$\sup_{\begin{subarray}{c}s,t\in[-u_{n},u_{n}]:\\\ s\neq
t\end{subarray}}\,\biggl{|}\frac{\Psi_{0}^{*}(t)-\Psi_{0}^{*}(s)-\bigl{(}\Psi_{0}^{*}(t-s)-\Psi_{0}^{*}(0)\bigr{)}}{t-s}\biggr{|}\to
0$ (88)

as $n\to\infty$. Finally, by (A4), $\int_{\mathbb{R}}|p_{0}^{\prime}|\leq
i(p_{0})^{1/2}<\infty$, so $p_{0}$ is of bounded variation on $\mathbb{R}$.
Moreover, by (A4)–(A5),

$\displaystyle 0\leq-\int_{\mathbb{R}}p_{0}(z+t)\,d\psi_{0}^{*}(z)$
$\displaystyle=\int_{\mathbb{R}}\psi_{0}^{*}(z-t)p_{0}^{\prime}(z)\,dz=\int_{\mathbb{R}}\psi_{0}^{*}(z-t)\,\psi_{0}(z)\,p_{0}(z)\,dz$
$\displaystyle\leq\biggl{(}\int_{\mathbb{R}}\psi_{0}^{*}(z-t)^{2}\,p_{0}(z)\,dz\biggr{)}^{1/2}\biggl{(}\int_{\mathbb{R}}\psi_{0}^{2}(z)\,p_{0}(z)\,dz\biggr{)}^{1/2}<\infty$

for $t\in\\{-t_{0},t_{0}\\}$, where the first equality holds by Fubini’s
theorem, similarly to (111). Therefore, by Lemma 38(i) and Theorem 2 _(c)_ ,
$(\Psi_{0}^{*})^{\prime}(0)=-\int_{\mathbb{R}}p_{0}\,d\psi_{0}^{*}=\bigl{(}\int_{\mathbb{R}}(\psi_{0}^{*})^{2}\,p_{0}\big{/}V_{p_{0}}(\psi_{0}^{*})\bigr{)}^{1/2}=i^{*}(p_{0})\in(0,\infty)$,
so

$\sup_{\begin{subarray}{c}s,t\in[-u_{n},u_{n}]:\\\ s\neq
t\end{subarray}}\,\biggl{|}\frac{\Psi_{0}^{*}(t-s)-\Psi_{0}^{*}(0)}{t-s}-i^{*}(p_{0})\biggr{|}\to
0$ (89)

as $n\to\infty$. Combining (87)–(89) yields (80). ∎

####  6.3.1 Proofs for Section 3.1

Since the function $\psi$ in (30) is chosen to be antisymmetric and bounded,
the condition $\mathbb{E}\psi(\varepsilon_{1})=\int_{\mathbb{R}}\psi p_{0}=0$
in Lemma 29 is satisfied whenever $p_{0}$ is symmetric.

######  Corollary 31.

Suppose that $p_{0}$ is symmetric and that  _(A1)–(A5)_ hold. Then for
$j\in\\{1,2,3\\}$ and $t\in\mathbb{R}$, we have

$\hat{\Psi}_{n,j}^{\mathrm{anti}}(t):=\int_{\mathbb{R}}\hat{\psi}_{n,j}^{\mathrm{anti}}(z-t)\,p_{0}(z)\,dz=\frac{\hat{\Psi}_{n,j}(t)-\hat{\Psi}_{n,j}(-t)}{2},$

and the conclusions of Lemma 30 hold for $(\hat{\psi}_{n,j}^{\mathrm{anti}})$
and $(\hat{\Psi}_{n,j}^{\mathrm{anti}})$ in place of $(\hat{\psi}_{n,j})$ and
$(\hat{\Psi}_{n,j})$ respectively.

###### Proof.

It suffices to consider $j=1$. We write
$\hat{\psi}_{n}^{\mathrm{anti}}\equiv\hat{\psi}_{n,1}^{\mathrm{anti}}$ and
$\hat{\Psi}_{n}^{\mathrm{anti}}\equiv\hat{\Psi}_{n,1}^{\mathrm{anti}}$. Since
$p_{0}$ is symmetric, $\psi_{0}(z)=p_{0}^{\prime}(z)/p_{0}(z)=-\psi_{0}(-z)$
for all $z\in\mathbb{R}$. By (67), $\psi_{0}^{*}(z)=-\psi_{0}^{*}(-z)$ for
Lebesgue almost every $z\in\mathbb{R}$, so

$\displaystyle\int_{\mathbb{R}}(\hat{\psi}_{n}^{\mathrm{anti}}-\psi_{0}^{*})^{2}\,p_{0}$
$\displaystyle=\int_{\mathbb{R}}\biggl{(}\frac{\hat{\psi}_{n}(z)-\hat{\psi}_{n}(-z)}{2}-\frac{\psi_{0}^{*}(z)-\psi_{0}^{*}(-z)}{2}\biggr{)}^{2}\,p_{0}(z)\,dz$
$\displaystyle\leq\int_{\mathbb{R}}\frac{\bigl{(}\hat{\psi}_{n}(z)-\psi_{0}^{*}(z)\bigr{)}^{2}+\bigl{(}\hat{\psi}_{n}(-z)-\psi_{0}^{*}(-z)\bigr{)}^{2}}{2}\,p_{0}(z)\,dz=\int_{\mathbb{R}}(\hat{\psi}_{n}-\psi_{0}^{*})^{2}\,p_{0}$

for each $n$. Similarly, for $s,t\in\mathbb{R}$, we have

$\displaystyle\int_{\mathbb{R}}\bigl{(}\hat{\psi}_{n}^{\mathrm{anti}}(z-t)$
$\displaystyle-\hat{\psi}_{n}^{\mathrm{anti}}(z-s)\bigr{)}^{2}\,p_{0}(z)\,dz$
$\displaystyle\leq\int_{\mathbb{R}}\frac{\bigl{(}\hat{\psi}_{n}(z-t)-\hat{\psi}_{n}(z-s)\bigr{)}^{2}+\bigl{(}\hat{\psi}_{n}(z+t)-\hat{\psi}_{n}(z+s)\bigr{)}^{2}}{2}\,p_{0}(z)\,dz$

and

$\hat{\Psi}_{n}^{\mathrm{anti}}(t)=\int_{\mathbb{R}}\frac{\hat{\psi}_{n}(z-t)-\hat{\psi}_{n}(t-z)}{2}\,p_{0}(z)\,dz=\int_{\mathbb{R}}\frac{\hat{\psi}_{n}(z-t)-\hat{\psi}_{n}(t+z)}{2}\,p_{0}(z)\,dz=\frac{\hat{\Psi}_{n}(t)-\hat{\Psi}_{n}(-t)}{2}.$

Since $\mathbb{E}\psi(\varepsilon_{1})=0$, the desired conclusions for
$(\hat{\psi}_{n,j}^{\mathrm{anti}})$ and $(\hat{\Psi}_{n,j}^{\mathrm{anti}})$
therefore follow from Lemma 30. ∎

The proof of Theorem 14 relies on the following ‘asymptotic equicontinuity’
result, which is similar to Bickel, (1975, Lemma 4.1).

######  Lemma 32.

Suppose that  _(A1)–(A5)_ hold for a linear model (27) in which $p_{0}$ is
symmetric. For $\beta\in\mathbb{R}^{d}$, $n\in\mathbb{N}$ and
$j\in\\{1,2,3\\}$, define

$R_{n,j}(\beta):=\frac{1}{\sqrt{n}}\sum_{i\in
I_{j+2}}X_{i}\bigl{\\{}\hat{\psi}_{n,j}^{\mathrm{anti}}(\varepsilon_{i}-X_{i}^{\top}\beta)-\hat{\Psi}_{n,j}^{\mathrm{anti}}(X_{i}^{\top}\beta)\bigr{\\}}.$

Then for every $j\in\\{1,2,3\\}$ and $M>0$, writing
$\Theta_{n}:=\\{\beta\in\mathbb{R}^{d}:\|{\beta}\|\leq Mn^{-1/2}\\}$, we have

$\sup_{\beta\in\Theta_{n}}\|{R_{n,j}(\beta)-R_{n,j}(0)}\|=o_{p}(1)$

as $n\to\infty$.

###### Proof.

It suffices to consider $j=1$, so we drop the $j$ subscript from
$\hat{\psi}_{n,j}^{\mathrm{anti}}$, $\hat{\Psi}_{n,j}^{\mathrm{anti}}$ and
$R_{n,j}$; we also write
$\mathcal{D}^{\prime}\equiv\mathcal{D}_{n}^{\prime}:=\mathcal{D}_{1}\cup\mathcal{D}_{2}\cup\\{X_{i}:i\in
I_{3}\\}$. For each $\beta\in\Theta_{n}$ and $i\in I_{3}$, we have
$\mathbb{E}\bigl{\\{}\hat{\psi}_{n}^{\mathrm{anti}}(\varepsilon_{i}-X_{i}^{\top}\beta)\,|\,\mathcal{D}^{\prime}\bigr{\\}}=\hat{\Psi}_{n}^{\mathrm{anti}}(X_{i}^{\top}\beta)$,
so
$\mathbb{E}\bigl{(}R_{n}(\beta)\\!\bigm{|}\\!\mathcal{D}^{\prime}\bigr{)}=0$.
Since $\max_{i\in[n]}|X_{i}^{\top}\beta|\,\alpha_{n}/\gamma_{n}=o_{p}(1)$ by
Cauchy–Schwarz and (A1), it follows from the weak law of large numbers and
Corollary 31 that

$\displaystyle\bigl{\|}\operatorname{Cov}\bigl{(}R_{n}(\beta)-R_{n}(0)\bigm{|}\mathcal{D}^{\prime}\bigr{)}\bigr{\|}_{\mathrm{op}}$
$\displaystyle\leq\frac{1}{n}\sum_{i\in
I_{3}}\bigl{\|}\operatorname{Cov}\bigl{(}X_{i}\\{\hat{\psi}_{n}^{\mathrm{anti}}(\varepsilon_{i}-X_{i}^{\top}\beta)-\hat{\psi}_{n}^{\mathrm{anti}}(\varepsilon_{i})\\}\bigm{|}\mathcal{D}^{\prime}\bigr{)}\bigr{\|}_{\mathrm{op}}$
$\displaystyle=\frac{1}{n}\sum_{i\in
I_{3}}\|{X_{i}}\|^{2}\operatorname{Var}\bigl{(}\hat{\psi}_{n}^{\mathrm{anti}}(\varepsilon_{i}-X_{i}^{\top}\beta)-\hat{\psi}_{n}^{\mathrm{anti}}(\varepsilon_{i})\bigm{|}\mathcal{D}^{\prime}\bigr{)}$
$\displaystyle\leq\biggl{(}\frac{1}{n}\sum_{i\in
I_{3}}\|{X_{i}}\|^{2}\biggr{)}\max_{i\in
I_{3}}\int_{\mathbb{R}}\bigl{(}\hat{\psi}_{n}^{\mathrm{anti}}(z-X_{i}^{\top}\beta)-\hat{\psi}_{n}^{\mathrm{anti}}(z)\bigr{)}^{2}\,p_{0}(z)\,dz$
$\displaystyle=O_{p}(1)\,o_{p}(1)=o_{p}(1).$

Thus, for every $\epsilon>0$, we have

$\displaystyle\mathbb{P}\bigl{(}\|{R_{n}(\beta)-R_{n}(0)}\|>\epsilon\\!\bigm{|}\\!\mathcal{D}^{\prime}\bigr{)}$
$\displaystyle\leq\epsilon^{-2}\,\mathbb{E}\bigl{(}\|{R_{n}(\beta)-R_{n}(0)}\|^{2}\\!\bigm{|}\\!\mathcal{D}^{\prime}\bigr{)}$
$\displaystyle\leq
d\epsilon^{-2}\,\bigl{\|}\operatorname{Cov}\bigl{(}R_{n}(\beta)-R_{n}(0)\\!\bigm{|}\\!\mathcal{D}^{\prime}\bigr{)}\bigr{\|}_{\mathrm{op}}\overset{p}{\to}0$

as $n\to\infty$, so $\|{R_{n}(\beta)-R_{n}(0)}\|\overset{p}{\to}0$ for every
$\beta\in\Theta_{n}$ by Lemma 41.

For each $\delta\in(0,1)$ and $n\in\mathbb{N}$, we can find a $(\delta
Mn^{-1/2})$-covering set $\Theta_{n,\delta}$ of $\Theta_{n}$ such that
$|\Theta_{n,\delta}|\leq 3\delta^{-d}$ (e.g. Wainwright,, 2019, Example 5.8),
so since $d$ is fixed as $n\to\infty$, it follows by a union bound that

$\max_{\beta\in\Theta_{n,\delta}}\|{R_{n}(\beta)-R_{n}(0)}\|\overset{p}{\to}0.$

In addition, for every $\beta\in\Theta_{n}$, there exists
$\vartheta_{\beta}\in\Theta_{n,\delta}$ such that
$\|{\beta-\vartheta}\|\leq\delta Mn^{-1/2}$. Since
$\hat{\psi}_{n}^{\mathrm{anti}}$ is decreasing and
$\hat{\Psi}_{n}^{\mathrm{anti}}$ is increasing for each $n$, we have

$\displaystyle\hat{\psi}_{n}^{\mathrm{anti}}\bigl{(}\varepsilon_{i}-X_{i}^{\top}\vartheta_{\beta}+\delta
Mn^{-1/2}\|{X_{i}}\|\bigr{)}$
$\displaystyle\leq\hat{\psi}_{n}^{\mathrm{anti}}(\varepsilon_{i}-X_{i}^{\top}\beta^{\prime})\leq\hat{\psi}_{n}^{\mathrm{anti}}\bigl{(}\varepsilon_{i}-X_{i}^{\top}\vartheta_{\beta}-\delta
Mn^{-1/2}\|{X_{i}}\|\bigr{)},$
$\displaystyle\hat{\Psi}_{n}^{\mathrm{anti}}\bigl{(}X_{i}^{\top}\vartheta_{\beta}-\delta
Mn^{-1/2}\|{X_{i}}\|\bigr{)}$
$\displaystyle\leq\hat{\Psi}_{n}^{\mathrm{anti}}(X_{i}^{\top}\beta^{\prime})\leq\hat{\Psi}_{n}^{\mathrm{anti}}\bigl{(}X_{i}^{\top}\vartheta_{\beta}+\delta
Mn^{-1/2}\|{X_{i}}\|\bigr{)}$

for $\beta^{\prime}\in\\{\beta,\vartheta_{\beta}\\}$. For each
$\vartheta\in\Theta_{n,\delta}$, let

$r_{n}(\vartheta):=\sum_{i\in
I_{3}}\frac{\|{X_{i}}\|}{\sqrt{n}}\biggl{\\{}\hat{\psi}_{n}^{\mathrm{anti}}\biggl{(}\varepsilon_{i}-X_{i}^{\top}\vartheta-\frac{\delta
M\|{X_{i}}\|}{\sqrt{n}}\biggr{)}-\hat{\psi}_{n}^{\mathrm{anti}}\biggl{(}\varepsilon_{i}-X_{i}^{\top}\vartheta+\frac{\delta
M\|{X_{i}}\|}{\sqrt{n}}\biggr{)}\biggr{\\}}.$

Then

$\displaystyle\sup_{\beta\in\Theta_{n}}\|R_{n}(\beta)-R_{n}(\vartheta_{\beta})\|\leq\max_{\vartheta\in\Theta_{n,\delta}}\bigl{\\{}r_{n}(\vartheta)+\mathbb{E}\bigl{(}r_{n}(\vartheta)\bigm{|}\mathcal{D}^{\prime}\bigr{)}\bigr{\\}}.$

By (A1), $V_{n}:=\max_{i\in
I_{3}}M(1+\delta)n^{-1/2}\|{X_{i}}\|=o_{p}(\gamma_{n}/\alpha_{n})$, so it
follows from Corollary 31, the independence of
$\mathcal{D}_{1},\mathcal{D}_{2},\mathcal{D}_{3}$ and Lemma 42 that

$\displaystyle\operatorname{Var}\bigl{(}r_{n}(\vartheta)\bigm{|}\mathcal{D}^{\prime}\bigr{)}$
$\displaystyle\leq\sum_{k\in I_{3}}\frac{\|{X_{k}}\|^{2}}{n}\max_{i\in
I_{3}}\int_{\mathbb{R}}\biggl{\\{}\hat{\psi}_{n}^{\mathrm{anti}}\biggl{(}z-X_{i}^{\top}\vartheta-\frac{\delta
M\|{X_{i}}\|}{\sqrt{n}}\biggr{)}-\hat{\psi}_{n}^{\mathrm{anti}}\biggl{(}z-X_{i}^{\top}\vartheta+\frac{\delta
M\|{X_{i}}\|}{\sqrt{n}}\biggr{)}\biggr{\\}}^{2}p_{0}(z)\,dz=o_{p}(1).$

Together with a union bound over $\vartheta\in\Theta_{n,\delta}$, this shows
that
$\max_{\vartheta\in\Theta_{n,\delta}}\bigl{\\{}r_{n}(\vartheta)-\mathbb{E}\bigl{(}r_{n}(\vartheta)\bigm{|}\mathcal{D}^{\prime}\bigr{)}\bigr{\\}}=o_{p}(1)$.
Moreover, by (A1) and Corollary 31,

$\displaystyle\max_{\vartheta\in\Theta_{n,\delta}}\mathbb{E}\bigl{(}r_{n}(\vartheta)\bigm{|}\mathcal{D}^{\prime}\bigr{)}$
$\displaystyle=\max_{\vartheta\in\Theta_{n,\delta}}\sum_{i\in
I_{3}}\frac{\|{X_{i}}\|}{\sqrt{n}}\biggl{\\{}\hat{\Psi}_{n}^{\mathrm{anti}}\biggl{(}X_{i}^{\top}\vartheta+\frac{\delta
M\|{X_{i}}\|}{\sqrt{n}}\biggr{)}-\hat{\Psi}_{n}^{\mathrm{anti}}\biggl{(}X_{i}^{\top}\vartheta-\frac{\delta
M\|{X_{i}}\|}{\sqrt{n}}\biggr{)}\biggr{\\}}$ $\displaystyle\leq\sum_{i\in
I_{3}}\frac{2\delta
Mi^{*}(p_{0})\|{X_{i}}\|^{2}}{n}(1+\eta_{n,i})=\frac{2\delta
Mi^{*}(p_{0})}{3}\mathbb{E}(\|{X_{1}}\|^{2})+o_{p}(1),$

where by (80) and Lemma 42, $(\eta_{n,i}:i\in I_{3})$ are random variables
satisfying $\max_{i\in I_{3}}|\eta_{n,i}|\leq\Delta_{n}(V_{n})=o_{p}(1)$.
Thus,

$\displaystyle\sup_{\beta\in\Theta_{n}}\|{R_{n}(\beta)-R_{n}(0)}\|\leq\sup_{\beta\in\Theta_{n}}\|R_{n}(\beta)-R_{n}(\vartheta_{\beta})\|+\max_{\vartheta\in\Theta_{n,\delta}}\|{R_{n}(\vartheta)-R_{n}(0)}\|$
$\displaystyle\hskip
28.45274pt\leq\max_{\vartheta\in\Theta_{n,\delta}}\bigl{\\{}r_{n}(\vartheta)-\mathbb{E}\bigl{(}r_{n}(\vartheta)\bigm{|}\mathcal{D}^{\prime}\bigr{)}\bigr{\\}}+2\max_{\vartheta\in\Theta_{n,\delta}}\mathbb{E}\bigl{(}r_{n}(\vartheta)\bigm{|}\mathcal{D}^{\prime}\bigr{)}+\max_{\vartheta\in\Theta_{n,\delta}}\|{R_{n}(\vartheta)-R_{n}(0)}\|$
$\displaystyle\hskip 28.45274pt\leq\frac{4\delta
Mi^{*}(p_{0})}{3}\,\mathbb{E}(\|{X_{1}}\|^{2})+o_{p}(1)$

as $n\to\infty$. Since this holds for all $\delta\in(0,1)$, the desired
conclusion follows. ∎

As a first step towards proving the asymptotic normality of
$\hat{\beta}_{n}^{\dagger}$ and $\hat{\beta}_{n}^{\ddagger}$ in Theorem 14, we
show in Lemma 33 below that they are $\sqrt{n}$-consistent estimators of
$\beta_{0}$. To this end, we exploit the convexity of the induced loss
functions
$z\mapsto\hat{\ell}_{n,j}^{\mathrm{sym}}(z)=-\int_{0}^{z}\hat{\psi}_{n,j}^{\mathrm{anti}}$,
similarly to He and Shao, (2000, Theorem 2.1). We denote the Euclidean unit
sphere in $\mathbb{R}^{d}$ by
$\mathcal{S}_{d-1}:=\\{u\in\mathbb{R}^{d}:\|u\|=1\\}$.

######  Lemma 33.

For $n\in\mathbb{N}$, $j\in\\{1,2,3\\}$ and $t>0$, we have

$\displaystyle\bigl{\\{}\|{\hat{\beta}_{n}^{(j)}-\beta_{0}}\|>t\bigr{\\}}$
$\displaystyle\subseteq\biggl{\\{}\inf_{u\in\mathcal{S}_{d-1}}u^{\top}\sum_{i\in
I_{j+2}}X_{i}\hat{\psi}_{n,j}^{\mathrm{anti}}\bigl{(}\varepsilon_{i}-tX_{i}^{\top}u\bigr{)}\leq
0\biggr{\\}},$ (90)
$\displaystyle\bigl{\\{}\|{\hat{\beta}_{n}^{\ddagger}-\beta_{0}}\|>t\bigr{\\}}$
$\displaystyle\subseteq\biggl{\\{}\inf_{u\in\mathcal{S}_{d-1}}u^{\top}\sum_{j=1}^{3}\sum_{i\in
I_{j+2}}X_{i}\hat{\psi}_{n,j}^{\mathrm{anti}}\bigl{(}\varepsilon_{i}-tX_{i}^{\top}u\bigr{)}\leq
0\biggr{\\}}.$ (91)

As a consequence, if $p_{0}$ is symmetric and  _(A1)–(A5)_ are satisfied, then

  1. (a)

$\sqrt{n}(\hat{\beta}_{n}^{(j)}-\beta_{0})=O_{p}(1)$ and
$\sqrt{n}(\hat{\beta}_{n}^{\ddagger}-\beta_{0})=O_{p}(1)$;

  2. (b)

$\displaystyle\frac{1}{\sqrt{n}}\sum_{i\in
I_{j+2}}X_{i}\hat{\psi}_{n,j}^{\mathrm{anti}}\bigl{(}Y_{i}-X_{i}^{\top}\hat{\beta}_{n}^{(j)}\bigr{)}=o_{p}(1)$
and $\displaystyle\frac{1}{\sqrt{n}}\sum_{j=1}^{3}\sum_{i\in
I_{j+2}}X_{i}\hat{\psi}_{n,j}^{\mathrm{anti}}\bigl{(}Y_{i}-X_{i}^{\top}\hat{\beta}_{n}^{\ddagger}\bigr{)}=o_{p}(1)$.

###### Proof.

It suffices to consider $j=1$. We write
$\hat{\psi}_{n}^{\mathrm{anti}}\equiv\hat{\psi}_{n,1}^{\mathrm{anti}}$ and
$\hat{\ell}_{n}^{\mathrm{sym}}\equiv\hat{\ell}_{n,1}^{\mathrm{sym}}$. The
function $\beta\mapsto\sum_{i\in
I_{3}}\hat{\ell}_{n}^{\mathrm{sym}}(Y_{i}-X_{i}^{\top}\beta)=:\hat{L}_{n}^{\mathrm{sym}}(\beta)$
is convex on $\mathbb{R}^{d}$, with
$\hat{\beta}_{n}\equiv\hat{\beta}_{n}^{(1)}\in\operatorname*{argmin}_{\beta\in\mathbb{R}^{d}}\hat{L}_{n}^{\mathrm{sym}}(\beta)$,
so
$h\mapsto\hat{L}_{n}^{\mathrm{sym}}\bigl{(}(1-h)\hat{\beta}_{n}+h\beta_{0}\bigr{)}=:G(h)$
is convex and increasing on $[0,\infty)$. Since

$g(h):=-(\hat{\beta}_{n}-\beta_{0})^{\top}\sum_{i\in
I_{3}}X_{i}\hat{\psi}_{n}^{\mathrm{anti}}\bigl{(}\varepsilon_{i}-(1-h)X_{i}^{\top}(\hat{\beta}_{n}-\beta_{0})\bigr{)}$

is a subgradient of $G$ at $h$, it follows that $g(h)\geq 0$ for every $h>0$.
Hence, if $\|{\hat{\beta}_{n}-\beta_{0}}\|>t>0$, then

$\displaystyle 0\leq
g\biggl{(}1-\frac{t}{\|{\hat{\beta}_{n}-\beta_{0}}\|}\biggr{)}$
$\displaystyle=-(\hat{\beta}_{n}-\beta_{0})^{\top}\sum_{i\in
I_{3}}X_{i}\hat{\psi}_{n}^{\mathrm{anti}}\biggl{(}\varepsilon_{i}-tX_{i}^{\top}\frac{\hat{\beta}_{n}-\beta_{0}}{\|{\hat{\beta}_{n}-\beta_{0}}\|}\biggr{)}$
$\displaystyle\leq-\|{\hat{\beta}_{n}-\beta_{0}}\|\inf_{u\in\mathcal{S}_{d-1}}u^{\top}\sum_{i\in
I_{3}}X_{i}\hat{\psi}_{n}^{\mathrm{anti}}(\varepsilon_{i}-tX_{i}^{\top}u),$
(92)

so (90) holds. By analogous reasoning based on
$\beta\mapsto\hat{L}_{n}^{\ddagger}(\beta):=\sum_{j=1}^{3}\sum_{i\in
I_{j+2}}\hat{\ell}_{n,j}^{\mathrm{sym}}(Y_{i}-X_{i}^{\top}\beta)$, we obtain
(91).

(a) Fix $M>0$ and write
$\hat{\Psi}_{n}^{\mathrm{anti}}\equiv\hat{\Psi}_{n,1}^{\mathrm{anti}}$. By
(A1), $V_{n}:=\max_{i\in
I_{3}}Mn^{-1/2}\|{X_{i}}\|=o_{p}(\gamma_{n}/\alpha_{n})$. We have
$|I_{3}|/n\to 1/3$, so for each $u\in\mathcal{S}_{d-1}$, Corollary 31 yields

$\displaystyle\sum_{i\in
I_{3}}\frac{X_{i}}{\sqrt{n}}\hat{\Psi}_{n}^{\mathrm{anti}}\Bigl{(}\frac{MX_{i}^{\top}u}{\sqrt{n}}\Bigr{)}$
$\displaystyle=\sum_{i\in
I_{3}}\frac{X_{i}}{\sqrt{n}}\biggl{\\{}\hat{\Psi}_{n}^{\mathrm{anti}}\Bigl{(}\frac{MX_{i}^{\top}u}{\sqrt{n}}\Bigr{)}-\hat{\Psi}_{n}^{\mathrm{anti}}(0)\biggr{\\}}=\frac{Mi^{*}(p_{0})}{n}\sum_{i\in
I_{3}}X_{i}X_{i}^{\top}u(1+\eta_{n,i})$

where by (80) and Lemma 42, $(\eta_{n,i}:i\in I_{3})$ are random variables
satisfying $\max_{i\in I_{3}}|\eta_{n,i}|\leq\Delta_{n}(V_{n})=o_{p}(1)$.
Therefore,

$\displaystyle\sup_{u\in\mathcal{S}_{d-1}}\Biggl{\|}\,\sum_{i\in
I_{3}}\frac{X_{i}}{\sqrt{n}}\hat{\Psi}_{n}^{\mathrm{anti}}\Bigl{(}\frac{MX_{i}^{\top}u}{\sqrt{n}}\Bigr{)}-\frac{Mi^{*}(p_{0})}{3}\,\mathbb{E}(X_{1}X_{1}^{\top})u\,\Biggr{\|}$
$\displaystyle\hskip 28.45274pt\leq
Mi^{*}(p_{0})\,\Biggl{(}\Biggl{\|}\,\sum_{i\in
I_{3}}\frac{X_{i}X_{i}^{\top}}{n}-\frac{\mathbb{E}(X_{1}X_{1}^{\top})}{3}\Biggr{\|}_{\mathrm{op}}+\frac{\sum_{i\in
I_{3}}\|{X_{i}}\|^{2}}{n}\cdot\Delta_{n}(V_{n})\Biggr{)}=o_{p}(1).$ (93)

Since $\mathbb{E}(X_{1}X_{1}^{\top})$ is positive definite by (A1), its
minimum eigenvalue $\lambda_{\min}$ is strictly positive. Then by (93) and
Lemma 32,

$\displaystyle\inf_{u\in\mathcal{S}_{d-1}}u^{\top}\sum_{i\in
I_{3}}\frac{X_{i}}{\sqrt{n}}\hat{\psi}_{n}^{\mathrm{anti}}\Bigl{(}\varepsilon_{i}-\frac{MX_{i}^{\top}u}{\sqrt{n}}\Bigr{)}$
$\displaystyle=\inf_{u\in\mathcal{S}_{d-1}}u^{\top}\biggl{\\{}\sum_{i\in
I_{3}}\frac{X_{i}}{\sqrt{n}}\hat{\Psi}_{n}^{\mathrm{anti}}\Bigl{(}\frac{MX_{i}^{\top}u}{\sqrt{n}}\Bigr{)}+R_{n}\Bigl{(}\frac{Mu}{\sqrt{n}}\Bigr{)}\biggr{\\}}$
$\displaystyle=\inf_{u\in\mathcal{S}_{d-1}}u^{\top}\biggl{\\{}\frac{Mi^{*}(p_{0})}{3}\,\mathbb{E}(X_{1}X_{1}^{\top})u+R_{n}(0)\biggr{\\}}+o_{p}(1)$
$\displaystyle\geq\frac{M\lambda_{\min}i^{*}(p_{0})}{3}-\|{R_{n}(0)}\|+o_{p}(1).$
(94)

Writing
$\mathcal{D}^{\prime}:=\mathcal{D}_{1}\cup\mathcal{D}_{2}\cup\\{X_{i}:i\in
I_{3}\\}$, we have

$\mathbb{E}\biggl{(}\frac{1}{\sqrt{n}}\sum_{i\in
I_{3}}X_{i}\bigl{(}\hat{\psi}_{n}^{\mathrm{anti}}(\varepsilon_{i})-\psi_{0}^{*}(\varepsilon_{i})\bigr{)}\biggm{|}\mathcal{D}^{\prime}\biggr{)}=\frac{1}{\sqrt{n}}\sum_{i\in
I_{3}}X_{i}\int_{\mathbb{R}}(\hat{\psi}_{n}^{\mathrm{anti}}-\psi_{0}^{*})\,p_{0}=0$

because $p_{0}$ is symmetric and $\hat{\psi}_{n}^{\mathrm{anti}},\psi_{0}^{*}$
are antisymmetric. Together with Corollary 31, this implies that

$\displaystyle\mathbb{E}\biggl{(}\biggl{\|}\frac{1}{\sqrt{n}}\sum_{i\in
I_{3}}X_{i}\bigl{(}\hat{\psi}_{n}^{\mathrm{anti}}(\varepsilon_{i})-\psi_{0}^{*}(\varepsilon_{i})\bigr{)}\biggr{\|}^{2}\biggm{|}\mathcal{D}^{\prime}\biggr{)}$
$\displaystyle=\frac{1}{n}\sum_{i\in
I_{3}}\mathbb{E}\bigl{\\{}\|{X_{i}}\|^{2}(\hat{\psi}_{n}^{\mathrm{anti}}-\psi_{0}^{*})^{2}(\varepsilon_{i})\\!\bigm{|}\\!\mathcal{D}^{\prime}\bigr{\\}}$
$\displaystyle=\frac{1}{n}\sum_{i\in
I_{3}}\|{X_{i}}\|^{2}\int_{\mathbb{R}}(\hat{\psi}_{n}^{\mathrm{anti}}-\psi_{0}^{*})^{2}\,p_{0}\overset{p}{\to}0.$

We have
$\mathbb{E}\bigl{(}X_{i}\psi_{0}^{*}(\varepsilon_{i})\bigr{)}=\mathbb{E}(X_{i})\int_{\mathbb{R}}\psi_{0}^{*}\,p_{0}=0$
for each $i$, so by the central limit theorem,

$R_{n}(0)=\frac{1}{\sqrt{n}}\sum_{i\in
I_{3}}X_{i}\hat{\psi}_{n}^{\mathrm{anti}}(\varepsilon_{i})=\frac{1}{\sqrt{n}}\sum_{i\in
I_{3}}X_{i}\psi_{0}^{*}(\varepsilon_{i})+o_{p}(1)=O_{p}(1)$ (95)

as $n\to\infty$. Since $M\lambda_{\min}i^{*}(p_{0})>0$, we deduce from (90)
and (94) that
$\limsup_{n\to\infty}\mathbb{P}(\|{\hat{\beta}_{n}-\beta_{0}}\|\geq
M/\sqrt{n})\to 0$ as $M\to\infty$, so
$\sqrt{n}(\hat{\beta}_{n}-\beta_{0})=O_{p}(1)$ as $n\to\infty$. The
$\sqrt{n}$-consistency of $\hat{\beta}_{n}^{\ddagger}$ follows similarly from
(91).

(b) Since the errors $\varepsilon_{i}$ have an absolutely continuous density
$p_{0}$, the conditional distribution of $(Y_{i}:i\in I_{3})$ given
$\mathcal{D}^{\prime}=\mathcal{D}_{1}\cup\mathcal{D}_{2}\cup\\{X_{i}:i\in
I_{3}\\}$ is absolutely continuous with respect to Lebesgue measure on
$\mathbb{R}$. We used $\mathcal{D}_{1}\cup\mathcal{D}_{2}$ to obtain the
convex function $\hat{\ell}_{n}^{\mathrm{sym}}$, whose subdifferential at
$z\in\mathbb{R}$ is

$\partial\hat{\ell}_{n}^{\mathrm{sym}}(x)=\bigl{[}\hat{\psi}_{n}^{\mathrm{anti}}(z),\hat{\psi}_{n}^{\mathrm{anti}}(z-)\bigr{]},$

where $\hat{\psi}_{n}^{\mathrm{anti}}(z-):=\lim_{z^{\prime}\nearrow
z}\hat{\psi}_{n}^{\mathrm{anti}}(z^{\prime})$. Since
$\hat{\psi}_{n}^{\mathrm{anti}}$ is decreasing,
$A:=\bigl{\\{}z\in\mathbb{R}:\hat{\psi}_{n}^{\mathrm{anti}}(z)<\hat{\psi}_{n}^{\mathrm{anti}}(z-)\bigr{\\}}$
is countable. Thus, applying Lemma 40 to the linear subspace
$W:=\bigl{\\{}(X_{i}^{\top}\beta)_{i\in
I_{3}}:\beta\in\mathbb{R}^{d}\bigr{\\}}$ of dimension at most $d$, we have

$\mathbb{P}\biggl{(}\max_{\beta\in\mathbb{R}^{d}}\,\sum_{i\in
I_{3}}\mathbbm{1}_{\\{Y_{i}-X_{i}^{\top}\beta\in A\\}}\leq
d\Bigm{|}\mathcal{D}^{\prime}\biggr{)}=1.$ (96)

Moreover,
$\hat{\beta}_{n}\in\operatorname*{argmin}_{\beta\in\mathbb{R}^{d}}\hat{L}_{n}^{\mathrm{sym}}(\beta)$,
so writing $\partial\hat{L}_{n}^{\mathrm{sym}}(\beta)$ for the subdifferential
of the convex function $\hat{L}_{n}^{\mathrm{sym}}$ at
$\beta\in\mathbb{R}^{d}$ Rockafellar, (1997, p. 315), we have

$0\in\partial\hat{L}_{n}^{\mathrm{sym}}(\hat{\beta}_{n})=\sum_{i\in
I_{3}}X_{i}\partial\hat{\ell}_{n}^{\mathrm{sym}}(Y_{i}-X_{i}^{\top}\hat{\beta}_{n}),$

so there exist
$a_{i}\in\partial\hat{L}_{n}^{\mathrm{sym}}(Y_{i}-X_{i}^{\top}\hat{\beta}_{n})$
for $i\in I_{3}$ such that $\sum_{i\in I_{3}}X_{i}a_{i}=0$. For each $i\in
I_{3}$, we have
$\bigl{|}a_{i}-\hat{\psi}_{n}^{\mathrm{anti}}(Y_{i}-X_{i}^{\top}\hat{\beta}_{n})\bigr{|}\leq
2\|{\hat{\psi}_{n}^{\mathrm{anti}}}\|_{\infty}\mathbbm{1}_{\\{Y_{i}-X_{i}^{\top}\hat{\beta}_{n}\in
A\\}}\leq
2(\alpha_{n}/\gamma_{n})\,\mathbbm{1}_{\\{Y_{i}-X_{i}^{\top}\hat{\beta}_{n}\in
A\\}}$, so by (96),

$\Biggl{|}\,\sum_{i\in
I_{3}}X_{i}\hat{\psi}_{n}^{\mathrm{anti}}(Y_{i}-X_{i}^{\top}\hat{\beta}_{n})\,\Biggr{|}\leq\sum_{i\in
I_{3}}\|{X_{i}}\|\cdot\bigl{|}a_{i}-\hat{\psi}_{n}^{\mathrm{anti}}(Y_{i}-X_{i}^{\top}\hat{\beta}_{n})\bigr{|}\leq
2d\max_{i\in I_{3}}\|{X_{i}}\|\,\frac{\alpha_{n}}{\gamma_{n}}$

almost surely. Assumption (A1) ensures that the right-hand side is
$o_{p}(n^{1/2})$ as $n\to\infty$, so (b) holds for
$\hat{\beta}_{n}=\hat{\beta}_{n}^{(1)}$, and the result for
$\hat{\beta}_{n}^{\ddagger}$ follows similarly by considering
$\hat{L}_{n}^{\ddagger}$. ∎

###### Proof of Theorem 14.

For $j\in\\{1,2,3\\}$, Lemma 33 _(b)_ implies that

$\displaystyle\frac{1}{\sqrt{n}}\sum_{i\in
I_{j+2}}X_{i}\hat{\Psi}_{n,j}^{\mathrm{anti}}\bigl{(}X_{i}^{\top}(\hat{\beta}_{n}^{(j)}-\beta_{0})\bigr{)}+R_{n,j}(\hat{\beta}_{n}^{(j)}-\beta_{0})$
$\displaystyle=\frac{1}{\sqrt{n}}\sum_{i\in
I_{j+2}}X_{i}\hat{\psi}_{n,j}^{\mathrm{anti}}\bigl{(}\varepsilon_{i}-X_{i}^{\top}(\hat{\beta}_{n}^{(j)}-\beta_{0})\bigr{)}$
$\displaystyle=\frac{1}{\sqrt{n}}\sum_{i\in
I_{j+2}}X_{i}\hat{\psi}_{n,j}^{\mathrm{anti}}(Y_{i}-X_{i}^{\top}\hat{\beta}_{n}^{(j)})=o_{p}(1)$
(97)

and, by Lemma 33 _(a)_ ,

$\biggl{\|}\frac{1}{\sqrt{n}}\sum_{i\in
I_{j+2}}X_{i}X_{i}^{\top}(\hat{\beta}_{n}^{(j)}-\beta_{0})\biggr{\|}\leq\frac{1}{n}\sum_{i\in
I_{j+2}}\|{X_{i}}\|^{2}\|{\sqrt{n}(\hat{\beta}_{n}^{(j)}-\beta_{0})}\|=O_{p}(1).$

Moreover, by (A1) and Lemma 33(a), $\max_{i\in
I_{j+2}}|X_{i}^{\top}(\hat{\beta}_{n}^{(j)}-\beta_{0})|\,\alpha_{n}/\gamma_{n}=o_{p}(1)$
as $n\to\infty$, so arguing similarly to (93), we have

$\displaystyle\frac{1}{\sqrt{n}}\sum_{i\in
I_{j+2}}X_{i}\hat{\Psi}_{n,j}^{\mathrm{anti}}\bigl{(}X_{i}^{\top}(\hat{\beta}_{n}^{(j)}-\beta_{0})\bigr{)}=\frac{i^{*}(p_{0})}{3}\,\mathbb{E}(X_{1}X_{1}^{\top})\sqrt{n}(\hat{\beta}_{n}^{(j)}-\beta_{0})+o_{p}(1).$
(98)

From (98) and (97), followed by Lemma 32 and Lemma 33(a), and then (95), we
deduce that

$\displaystyle\frac{i^{*}(p_{0})}{3}\mathbb{E}(X_{1}X_{1}^{\top})\sqrt{n}(\hat{\beta}_{n}^{(j)}-\beta_{0})$