# Statistical Learning for Individualized Asset Allocation

Yi Ding, Yingying Li, and Rui Song Assistant Professor, Faculty of Business
Administration, University of Macau, Macau (e-mail<EMAIL_ADDRESS>Corresponding author. Professor, Department of ISOM and Department of Finance,
Hong Kong University of Science and Technology, Clear Water Bay, Kowloon, Hong
Kong (e-mail:yyli@ust.hk)Corresponding author. Professor, Department of
Statistics, North Carolina State University, Raleigh, North Carolina, USA
(e-mail: rsong@ncsu.edu).

(October 12, 2022)

###### Abstract

We establish a high-dimensional statistical learning framework for
individualized asset allocation. Our proposed methodology addresses
continuous-action decision-making with a large number of characteristics. We
develop a discretization approach to model the effect of continuous actions
and allow the discretization frequency to be large and diverge with the number
of observations. We estimate the value function of continuous-action using
penalized regression with our proposed generalized penalties that are imposed
on linear transformations of the model coefficients. We show that our proposed
Discretization and Regression with generalized fOlded concaVe penalty on
Effect discontinuity (DROVE) approach enjoys desirable theoretical properties
and allows for statistical inference of the optimal value associated with
optimal decision-making. Empirically, the proposed framework is exercised with
the Health and Retirement Study data in finding individualized optimal asset
allocation. The results show that our individualized optimal strategy improves
the financial well-being of the population.

Keywords: Individualization; High-dimensional statistical learning;
Continuous-action decision-making; Penalized regression

##  1 Introduction

With the rapid development of artificial intelligence, particularly machine
learning, a revolution is underway in individualization. In precision
medicine, thousands or millions of genetic characteristics are taken into
consideration to determine the optimal treatment for an individual patient.
Large retail corporations analyze massive datasets of the behavior and
personal characteristics of customers to tailor their offerings to individual
customers.

Individualization is not new in asset allocation. For example, before
providing financial advice, investment companies ask their customers to answer
questionnaires in order to determine their customers’ risk profiles. An asset
allocation strategy would then be recommended in the form of, for instance, a
given proportion of stocks versus bonds that the customer should hold in their
portfolio. However, often in these questionnaires, many of the questions are
subjective and/or hypothetical. Incorporating objective data of the investors
can ameliorate the weakness of the reliance of such personalization on
subjective opinion.

In this study, we develop a statistical learning framework for individualized
asset allocation. We focus on finding an individualized optimal proportion of
wealth that should be invested in stocks for a consumption-based utility
optimization problem. The essence of the problem is decision-making, where the
action can take value on a continuous set. More broadly, our study provides a
theoretical basis and practical showcase for continuous-action decision-making
research, such as dose decision in precision medicine, movement angle in
robotic control, and campaign duration in personalized marketing.

### Related Literature

Methodologically, our study is closely related to the studies seeking the
optimal individualized treatment rule (ITR) in fields such as personalized
medicine. Q-learning and A-learning are the most popular methods for finding
the optimal ITR. Q-learning (Watkins, 1989) models the treatment responses,
and A-learning (Murphy, 2003; Robins, 2004) models the contrast (regret)
function; see, for example, Qian and Murphy, (2011); Zhao et al., (2012); Shi
et al., (2018); Athey and Wager, (2019); Zhu et al., (2019). These studies
focus primarily on the problem in which the treatment comes from a fixed
number of discrete levels, typically binary treatment.

There is a growing interest in the study of continuous-action decision-making
where the treatment comes from a continuous set. Laber and Zhao, (2015)
propose a direct tree-based optimal rule searching method. Chen et al., (2016)
propose a direct nonparametric method by extending the outcome-weighted
learning method of Zhao et al., (2012). More recently, Cai et al., (2020)
propose an off-policy evaluation method by adaptively discretizing the action
space using a deep jump Q-learning; Zhu et al., (2020) study a kernel assisted
optimal dose rule method; while Zhou et al., (2021) propose a dimension
reduced kernel approximation method. These studies focus on the estimation of
optimal policy or the effect of continuous policy with low-dimensional
covariates.

Theoretically, our study relates to the literature on penalized regression,
especially in high-dimensional settings. The well-known approaches of
penalized regression include the least absolute shrinkage and selection
operator (LASSO, Tibshirani, 1996), smoothly clipped absolute deviation (SCAD,
Fan and Li, 2001), and minimax concave penalty (MCP, Zhang et al., 2010). The
theoretical properties and implementation of the penalized regression
estimator under high-dimensional settings are investigated by Meinshausen and
Bühlmann, (2006), Zhao and Yu, (2006), Zhang and Huang, (2008), Lv and Fan,
(2009), Fan and Lv, (2011), Wang et al., (2013), among others. These studies
investigate the problem with penalties imposed directly on model coefficients.
For the generalized penalties that are imposed on a linear transformation of
coefficients, Tibshirani and Taylor, (2011) and Arnold and Tibshirani, (2016)
investigate the generalized penalty problems using lasso penalties
(generalized lasso) and their studies focus mainly on the computational
aspect. She, (2010) discusses the sparsity recovery property of the
generalized lasso estimator when the number of variables is fixed.

In economics and finance, our study relates to the literature on household
finance for the elderly and studies that use Health and Retirement Study (HRS)
and Consumption and Activities Mail Survey (CAMS) data. Using HRS and CAMS
data, the “retirement savings puzzle” (Banks et al., 1998; Palumbo, 1999) is
studied by Hurd and Rohwedder, (2003) and Haider and Stephens Jr, (2007).
Rosen and Wu, (2004), Hong et al., (2004) and De Nardi and Yang, (2014) study
the problem of asset allocation modeling. By analyzing HRS and CAMS data,
Engen et al., (1999) and Munnell et al., (2012) underscore a universal
inadequacy of post-retirement savings.

### Main Challenges and Our Contributions

Developing methodologies for continuous-action decision making is challenging.
Today, it is common to collect a large number of characteristics and
incorporate them into the decision-making process. When there are many action
levels, the dimensionality of the problem becomes even larger, posing
challenges in identifying informative characteristics and estimating the
model. The policy value, especially the optimal value, is an important target
of inference for policy evaluation. For continuous-action decision-making,
there are infinite possible options in the search for the optimal policy,
posing substantial challenges to the statistical inference of the optimal
value.

Empirically, substantial challenges come from the data. The HRS and CAMS
survey data contain noisy and incomplete observations. The observed stock
ratio data are skewed and concentrated around zero. In addition, the important
utility variable is unobserved. Data preparation needs to be carefully done in
order to obtain sufficient high-quality data for the statistical learning.

The main contributions of our paper are as follows.

First, in the field of individualized decision-making, we develop a high-
dimensional statistical learning framework to study continuous-action
decision-making. Specifically, we develop a discretization approach to model
the effect of continuous actions and allow the discretization level to be
large and diverge with the number of observations. The size of the
discretization level is carefully analyzed in order to balance the
approximation accuracy of the discretization with the compatibility in the
penalized regression framework.

Second, we propose a Discretization and Regression with generalized fOlded
concaVe penalty on Effect discontinuity (DROVE) method to estimate the model
for continuous-action decision-making, whose design accommodates the
continuous nature of the actions. Different from the standard penalized
regression that penalizes the coefficients directly, DROVE penalizes the
effect differences between adjacent action levels. Our estimator enjoys model
coefficient estimation consistency.

Third, our novel approach provides valid statistical inference for the optimal
value of continuous action when there are a large number of characteristics.
We obtain the central limit theorem for the proposed DROVE estimator of the
model coefficients as well as the value that is associated with optimal
decision-making. To the best of our knowledge, our study is the first to
achieve optimal value inference for continuous action, especially under a
setting with high-dimensional characteristics.

Last but not least, in the fields of economics and finance, as a pioneer work,
our study provides an individualized asset allocation using a high-dimensional
statistical learning method that processes personal characteristic
information. To address the data challenges, we use trajectory path models,
which allow us to generate pseudo consumption and income for randomly assigned
stock ratios. We then obtain the utility from the pseudo consumption paths and
individual risk aversion models. Our empirical exercise with HRS and CAMS data
shows that our individualized optimal asset allocation strategy substantially
improves the financial well-being of the population and surpasses benchmark
strategies that assign fixed stock ratios to all households. The superior
performance of our method demonstrates the importance of individualization in
asset allocation.

The paper proceeds as follows. We present the statistical learning framework
in Section 2 and develop the theoretical properties in Section 3. Sections 4
and 5 are devoted to simulation and empirical studies, respectively. Section 6
contains concluding remarks.

##  2 Statistical Learning Framework for Continuous-Action Decision-Making

###  2.1 Model Setup

Suppose that we have $n$ observations
$(\mathbf{X}_{i},A_{i},Y_{i})_{i=1,...,n}$, where
$\mathbf{X}_{i}\in\bm{\mathcal{X}}$ is a length-$d$ vector of covariates,
$A_{i}$ is the action that comes from continuous support $\mathcal{A}$, and
$Y_{i}$ is a random outcome. Under the potential outcome framework (Rubin,
1974), we denote $Y^{a}$ as the potential outcome that would have been
observed under action level $a\in\mathcal{A}$. Following the literature of
causal inference (e.g., Robins, 2004), we make the stable unit treatment value
assumption (SUTVA) that $Y=Y^{a}$ if the action $A=a$ (consistency). In
addition, we consider the randomization assumption that $A$ is independent of
$\mathbf{X}$ and the potential outcomes (e.g., Murphy, 2005), and the
positivity assumption on the density function of $A\in\mathcal{A}$ (e.g., Chen
et al., 2016). A deterministic policy $\pi$ maps the features space to the
action space, $\pi:\bm{\mathcal{X}}\to\mathcal{A}$. Under the SUTVA
assumption, the optimal policy $\pi^{*}$ determines the optimal action that
maximizes the expected reward given the characteristics $\mathbf{X}$, that is,
$\pi^{*}(x)=\mathop{\arg\max}_{a}E(Y^{a}|\mathbf{X}=x,A=a)=\mathop{\arg\max}_{a}E(Y|\mathbf{X}=x,A=a).$

Under the above framework, a particular example of interest is finding the
optimal individualized asset allocation. For individualized asset allocation,
$\mathbf{X}$ is individual characteristics, such as the financial and
demographic status; $Y$ is the economic reward or utility; $A$ is the
proportion of total wealth invested in stocks, that is, stock ratio, which can
be continuous between 0 and 1; and $\pi^{*}(\cdot):\mathbb{R}^{d}\to[0,1]$ is
the investment decision rule that yields the optimal stock ratio given the
individual characteristics.

We study the optimal continuous-action decision-making by modeling the
conditional expected reward as a value function
$Q(x,a)=E(Y|\mathbf{X}=x,A=a)$. We consider the following model

$Y=Q_{n}(\mathbf{X},A)+\varepsilon,\quad\quad
Q_{n}(\mathbf{X},A)=\bm{\psi}_{0}^{T}\mathbf{X}+\sum_{k=2}^{L_{n}}\bm{\psi}^{T}_{k}\mathbf{X}\cdot\mathbf{1}_{\\{A\in[A_{(k)},A_{(k+1)})\\}},$
(2.1)

where $Q_{n}(\mathbf{X},A)$ is the value function and $\varepsilon$ is the
noise term that is independent of $\mathbf{X}$,
$\\{0=A_{(1)}<...<A_{(L_{n})}=1\\}$ is a series of grids, $L_{n}\leq n$ is the
number of grids that can be large and increase with sample size $n$, and
$\bm{\psi}_{0},\bm{\psi}_{2},...,\bm{\psi}_{L_{n}}$, are $d$-dimensional
vectors, $\bm{\psi}_{1}=\mathbf{0}$ and $A_{(L_{n}+1)}>1$. The two components,
$\bm{\psi}_{0}^{T}\mathbf{X}$ and
$\sum_{k=2}^{L_{n}}\bm{\psi}^{T}_{k}\mathbf{X}\cdot\mathbf{1}_{\\{A\in[A_{(k)},A_{(k+1)})\\}},$
represent the main effect of characteristics and the treatment effect of the
action on individual characteristics, respectively.

We write
$\check{\mathbf{X}}_{i}=\big{(}(\mathbf{X}_{i})^{T},(\mathbf{X}_{i})^{T}\cdot\mathbf{1}_{\\{A_{i}\in[A_{(2)},A_{(3)})\\}},...,(\mathbf{X}_{i})^{T}\cdot\mathbf{1}_{\\{A_{i}\in[A_{(L_{n})},A_{(L_{n}+1)})\\}}\big{)}^{T},$
and
$\bm{\beta}_{n}=(\beta_{1},...,\beta_{p})^{T}=(\bm{\psi}_{0}^{T},\bm{\psi}^{T}_{2},...,\bm{\psi}^{T}_{L_{n}})^{T}$
as a length-$p$ vector of the coefficients, where $p=d\times L_{n}$. Hence,
the value function can be represented as
$Q_{n}(\mathbf{X}_{i},A_{i})=\bm{\beta}_{n}^{T}\check{\mathbf{X}}_{i}.$ We
allow $p$ to be large, such that $\log p=o(n)$. In particular, we include the
case where characteristics space can be large, so that $\log d=o(n)$. As to
the discretization level $L_{n}$, it can grow slowly with $n$ so that the
approximation error in the working model diminishes to zero as $n\to\infty$.
We discuss the growth rate of $L_{n}$ in more detail in Section 2.2. Under the
working model (2.1), we denote the true population parameter as
$\bm{\beta}_{n}^{\star}=(\bm{\psi}_{0}^{\star T},\bm{\psi}^{\star
T}_{2},...,\bm{\psi}^{\star T}_{L_{n}})^{T}$.

###  2.2 Penalized Regression with Generalized Penalties

In order to estimate the coefficients $\bm{\beta}_{n}^{\star}$ in the high-
dimensional Q-function (2.1), we develop a novel penalized regression
formulation, which accommodates the fact that decisions are made from a
continuum. Intuitively, when the discretization level $L_{n}$ is large, the
distance between adjacent action levels $|A_{(k)}-A_{(k+1)}|$ becomes small
and so is the difference in their effects,
$(|\bm{\psi}_{k,j}-\bm{\psi}_{k+1,j}|)_{1\leq j\leq d}$. We impose penalties
on $|\bm{\psi}_{k,j}-\bm{\psi}_{k+1,j}|$ to shrink the difference in effect
between two adjacent decisions, $A_{(k)}$ and $A_{(k+1)}$, on each covariate
$j=1,...,d$.

Formally, we write the following penalized regression:

$\min_{\bm{\beta}_{n}=(\bm{\psi}_{0}^{T},...,\bm{\psi}^{T}_{L_{n}})^{T}}\sum_{i=1}^{n}(Y_{i}-\bm{\beta}_{n}^{T}\check{\mathbf{X}}_{i})^{2}+n\sum_{k=0}^{L_{n}}p_{\lambda_{n}}(\bm{\psi}_{k})+n\sum_{k=2}^{L_{n}-1}\sum_{j=1}^{d}p_{\lambda_{n}}(\bm{\psi}_{k,j}-\bm{\psi}_{k+1,j}),$
(2.2)

where the penalty function is
$p_{\lambda_{n}}(\bm{\xi}):=\sum_{i=1}^{h}p_{\lambda_{n}}(\xi_{i})$ for any
vector $\bm{\xi}=(\xi_{1},\xi_{2},...,\xi_{h})^{T}$, and
$p_{\lambda_{n}}(\cdot)$ is a penalty function with tuning parameter
$\lambda_{n}$. The first penalties
$\sum_{k=0}^{L_{n}}p_{\lambda_{n}}(\bm{\psi}_{k})$ penalize the main effect
and the treatment effect, similar to the binary case; see, for example, Zhu et
al., (2019). The second penalties
$\sum_{k=2}^{L_{n}-1}\sum_{j=1}^{d}p_{\lambda_{n}}(\bm{\psi}_{k,j}-\bm{\psi}_{k+1,j})$
penalize the discontinuity.

###### Remark 2.1

The penalties,
$\big{(}p_{\lambda_{n}}(\bm{\psi}_{k,j}-\bm{\psi}_{k+1,j})\big{)}_{2\leq k\leq
L_{n}-1,1\leq j\leq d}$, generalize the idea of fused lasso (Tibshirani et
al., 2005), which imposes penalties on the difference in adjacent coordinates
$|\beta_{i}-\beta_{i+1}|$. She, (2010), Tibshirani and Taylor, (2011) and
Arnold and Tibshirani, (2016) study the generalized lasso problem, which is a
penalized regression as formulated in (2.3) with the lasso penalty function
for $p_{\lambda}(\cdot)$. In contrast to these works, we propose a generalized
folded concave penalty and investigate the statistical properties of the
estimator under the high-dimensional setting. In the literature on
individualized decision making, to the best of our knowledge, it is the first
time that regression with a generalized folded concave penalized is formulated
in studying the effect of continuous actions.

The proposed penalized regression (2.2) can be categorized as one with
_generalized_ penalties, where penalization is imposed on linear
transformations of the coefficients, $\mathbf{D}\bm{\beta}_{n}$, for some
$K\times p$ matrix $\mathbf{D}=(\mathbf{d}_{1},...,\mathbf{d}_{K})^{T}$:

$\min_{\bm{\beta}_{n}}\frac{1}{n}\sum_{i=1}^{n}(Y_{i}-\bm{\beta}_{n}^{T}\check{\mathbf{X}}_{i})^{2}+p_{\lambda_{n}}(\mathbf{D}\bm{\beta}_{n}).$
(2.3)

More generally, we consider the following generalized linear model (Fan and
Lv, 2011) where the density function
$f(\mathbf{Y};\check{\mathbf{X}},\bm{\beta}_{n})$ satisfies

$f(\mathbf{Y};\check{\mathbf{X}},\bm{\beta}_{n})=\prod_{i=1}^{n}f_{0}(Y_{i};\bm{\beta}_{n}^{T}\check{\mathbf{X}}_{i})=\prod_{i=1}^{n}\exp\bigg{\\{}\frac{Y_{i}\bm{\beta}_{n}^{T}\check{\mathbf{X}}_{i}-b(\bm{\beta}_{n}^{T}\check{\mathbf{X}}_{i})}{\phi}\bigg{\\}}c(Y_{i},\phi),$
(2.4)

where $\bm{\beta}_{n}=(\beta_{1},...,\beta_{p})^{T}$ is a $p\times 1$ vector
of regression coefficients, $\phi\in(0,\infty)$ is the nuisance parameter of
dispersion,
$\check{\mathbf{X}}=(\check{\mathbf{X}}_{1},\check{\mathbf{X}}_{2},...,\check{\mathbf{X}}_{n})^{T}$,
$b(\cdot)$ is twice continuously differentiable with
$b^{\prime\prime}(\cdot)>0$, and $c(Y_{i},\phi)$ is the base measure that
represents the density function for $Y_{i}$ when $\bm{\beta}_{n}=\mathbf{0}$,
for example, $c(Y_{i},\phi)=(\pi\phi)^{-1/2}\exp(-Y_{i}^{2}/\phi)$ for normal
density. For a given matrix $\mathbf{D}$, the penalized likelihood function
with generalized penalties is

$\displaystyle\mathcal{L}_{n}(\bm{\beta}_{n},\lambda_{n},\mathbf{D})=$
$\displaystyle
l_{n}(\bm{\beta}_{n})-\sum_{k=1}^{K}p_{\lambda_{n}}(\mathbf{d}_{k}^{T}\bm{\beta}_{n})=\frac{1}{n}\big{(}\mathbf{Y}^{T}\check{\mathbf{X}}\bm{\beta}_{n}-\mathbf{1}^{T}\mathbf{b}(\check{\mathbf{X}}\bm{\beta}_{n})\big{)}-\sum_{k=1}^{K}p_{\lambda_{n}}(\mathbf{d}_{k}^{T}\bm{\beta}_{n}),$
(2.5)

where
$l_{n}(\bm{\beta}_{n})=\big{(}\mathbf{Y}^{T}\check{\mathbf{X}}\bm{\beta}_{n}-\mathbf{1}^{T}\mathbf{b}(\check{\mathbf{X}}\bm{\beta}_{n})\big{)}/n$
and
$\mathbf{b}(\check{\mathbf{X}}\bm{\beta}_{n})=\big{(}b(\bm{\beta}_{n}^{T}\check{\mathbf{X}}_{1}),...,b(\bm{\beta}_{n}^{T}\check{\mathbf{X}}_{n})\big{)}^{T}$.

We next write
$\mathbf{D}=(\mathbf{D}_{signal}^{T},\mathbf{D}_{null}^{T})^{T}$, where
$\mathbf{D}_{signal}$ is a $K_{1}\times p$ matrix, $\mathbf{D}_{null}$ is a
$K_{0}\times p$ matrix, and $K=K_{0}+K_{1}$. Suppose that the true
coefficients,
$\bm{\beta}_{n}^{\star}=(\beta^{\star}_{1},...,\beta^{\star}_{p})^{T}$,
satisfy $\mathbf{D}_{null}\bm{\beta}_{n}^{\star}=\mathbf{0}$, that is,
$\mathbf{d}_{k}^{T}\bm{\beta}_{n}^{\star}=0$, for $k=K_{1}+1,K_{1}+2,...,K$.
In terms of $\mathbf{D}_{signal}$, we impose no constraint on its shape or
rank.

For any positive semi-definite matrix $\mathbf{A}=(a_{ij})$, we define
$\|\mathbf{A}\|_{2}=\max_{\|\mathbf{x}\|_{2}\leq
1}\|\mathbf{A}\mathbf{x}\|_{2}$ and $\|\mathbf{x}\|_{2}=\sqrt{\sum x_{i}^{2}}$
for any vector $\mathbf{x}=(x_{i})$.

We impose the following assumptions on $\mathbf{D}$ and
$\rho(t,\lambda)=p_{\lambda}(t)/\lambda$.

###### Assumption 2.1

$\zeta_{\min}^{+}(\mathbf{D}_{null}\mathbf{D}_{null}^{T})\geq c$ and
$\max_{1\leq k\leq K}\|\mathbf{d}_{k}\|_{2}\leq C$ for some constants $c,C>0$,
where $\zeta_{\min}^{+}(\mathbf{D}_{null}\mathbf{D}_{null}^{T})$ denotes the
smallest nonzero eigenvalue of $\mathbf{D}_{null}\mathbf{D}_{null}^{T}$.

###### Remark 2.2

Assumption 2.1 is met by our design of $\mathbf{D}$ in (2.2) as
$\zeta_{\min}^{+}(\mathbf{D}_{null}\mathbf{D}_{null}^{T})\geq 1$ and
$\max_{1\leq k\leq K}\|\mathbf{d}_{k}\|_{2}\leq\sqrt{2}$.

###### Assumption 2.2

(1) $\rho(t,\lambda)$ is increasing and concave in $t\in[0,\infty)$; (2)
$\rho(t,\lambda)$ is differentiable in $t\in(0,\infty)$ with
$\rho^{\prime}(0+,\lambda)>0$; and (3) if $\rho^{\prime}(t,\lambda)$ is
dependent on $\lambda$, $\rho^{\prime}(t,\lambda)$ is increasing in
$\lambda\in(0,\infty)$ and $\rho^{\prime}(0+)$ is independent of $\lambda$.

###### Remark 2.3

Assumption 2.2 describes the characteristics of a folded concave function
class; see, for example, Lv and Fan, (2009); Fan and Lv, (2011). Popular
examples of folded concave functions include SCAD (Fan and Li, 2001) and MCP
(Zhang et al., 2010). Our numerical examples uses SCAD as representative of
the folded concave penalty function family.

We define $s_{n}=p-\text{rank}(\mathbf{D}_{null})$. We regard $s_{n}$ as the
nonsparsity coefficient, which is a natural extension of the nonsparsity
coefficient for standard penalized regression when $\mathbf{D}$ is an identity
matrix. The definition of $s_{n}$ is in line with that of the degrees of
freedom for generalized lasso (Tibshirani and Taylor, 2011, 2012). The true
model is considered to be sparse in the sense that $s_{n}\ll n$. The sparsity
of the model comes from two sources. One source of sparsity is the large
covariate space in which there can be many irrelevant variables. In real
applications, for example, investment agencies collect a large number of
covariates, although only a small proportion of the variables are useful. The
sparsity that comes from high-dimensional covariates is the usual notion of
model sparsity discussed in the literature. In addition, we assume sparsity on
the treatment effect difference between adjacent action levels. This is
essentially to assume that the effect of continuous action exhibits
smoothness; hence when $L_{n}$ is large, many adjacent action levels have
roughly the same treatment effect. This is sensible in real applications. For
example, in asset allocation, when there are small changes in stock ratio,
utility would not vary drastically.

Let
$g_{n}=2^{-1}\min\\{|\mathbf{d}_{j}^{T}\bm{\beta}_{n}^{\star}|,\mathbf{d}_{j}^{T}\bm{\beta}_{n}^{\star}\neq
0\\}$ be half of the minimum signal. We impose the following assumptions.

###### Assumption 2.3

$g_{n}\gg\lambda_{n}\gg\max(\sqrt{s_{n}/n},\sqrt{(\log p)/n})$,
$\max(s_{n},\log p)=o(n)$, and
$p_{\lambda_{n}}^{\prime}(g_{n})=o\Big{(}\min(n^{-1/2}s_{n}^{-1/2},n^{-1/2}K_{1}^{-1}s_{n}^{1/2})\Big{)}$.

Assumption 2.3 states that the minimal signal should be sufficiently large to
be distinguishable from the noise. If $g_{n}$ decreases as $L_{n}$ grows, the
minimal signal condition would constrain the number of discretization levels
that we are able to handle. Asymptotically, $L_{n}$ can be $O(n^{\varsigma})$
for some $\varsigma<1/3$. In Appendix A of the supplementary materials, we
discuss the rate of $L_{n}$ under a varying coefficient example. We illustrate
the choice of $L_{n}$ with our practical example in Section 4.1. Additional
regularity conditions (Assumptions C.1–C.4) are in Appendix C of the
supplementary materials.

We summarize the proposed approach for continuous-action decision-making:
Discretization and Regression with generalized fOlded concaVe penalty on
Effect discontinuity (DROVE) as follows.

DROVE

Discretize the action support into a series of grids $A_{(1)}<...<A_{(L_{n})}$
with $L_{n}$ growing with $n$, and $L_{n}=O(n^{\varsigma})$ for some
$\varsigma<1/3$.

Perform the penalized regression with the generalized folded concave penalty
on effect discontinuity, (2.2), and obtain the estimated value function
$\widehat{Q}_{n}(\mathbf{{X}},A)$. The GLLN algorithm to be introduced in
Section 3.1 can be used to solve (2.2).

The estimated optimal decision making is
$\widehat{\pi}^{*}_{n}(\mathbf{\mathcal{X}}):=\min\\{A:\widehat{Q}_{n}(\mathbf{\mathcal{X}},A)=\max_{A_{(1)}\leq
A\leq A_{(L_{n})}}\widehat{Q}_{n}(\mathbf{\mathcal{X}},A)\\}$ for an
individual with characteristics $\mathbf{\mathcal{X}}$, and the optimal value
function is
$\widehat{Q}^{*}_{n}(\mathbf{\mathcal{X}}):=\widehat{\bm{\psi}}_{0}^{T}\mathbf{\mathcal{X}}+\max\Big{(}0,(\widehat{\bm{\psi}}^{T}_{k}\mathbf{\mathcal{X}})_{k=2,...,L_{n}}\Big{)}$.

##  3 Statistical Properties

In this section, we present the statistical properties of the proposed DROVE
methodology for continuous-action decision-making. The proofs are in Appendix
E of the supplementary materials.

###  3.1 Theoretical Properties of Coefficient Estimation

The following theorems give the statistical properties of our coefficient
estimator.

###### Theorem 3.1

Under Assumptions 2.1–2.3, C.1 and C.2, there exists a strict local maximizer
$\widehat{\bm{\beta}}_{n}=(\widehat{\bm{\psi}}_{0}^{T},\widehat{\bm{\psi}}_{2}^{T},...,\widehat{\bm{\psi}}_{L_{n}}^{T})^{T}$
of the penalized likelihood function $\mathcal{L}_{n}$, which satisfies

$P(\widehat{\bm{\psi}}_{Z_{0}}=\mathbf{0})\to 1\,\text{as
}n\to\infty,\quad\text{ and
}\quad\|\widehat{\bm{\beta}}_{n}-\bm{\beta}_{n}^{\star}\|_{2}=O_{p}(\sqrt{s_{n}/n}),$

where $\widehat{\bm{\psi}}_{Z_{0}}=(\widehat{\bm{\psi}}_{i,j})_{(i,j)\in
Z_{0}}$, $Z_{0}=\\{(i,j):\bm{\psi}^{\star}_{i,j}=0,i=0,2,...,L_{n},1\leq j\leq
d\\}$. In addition,

$P\big{(}\mathbf{D}_{null}\widehat{\bm{\beta}}_{n}=\mathbf{0}\big{)}\to
1\,\text{as }n\to\infty,\quad\text{ and
}\quad\|\widehat{\bm{\theta}}_{n}-\bm{\theta}_{n}^{\star}\|_{2}=O_{p}(\sqrt{s_{n}/n}),$
(3.1)

where $\widehat{\bm{\theta}}_{n}=\mathbf{M}^{-1}\widehat{\bm{\beta}}_{n}$ and
$\bm{\theta}_{n}^{\star}=\mathbf{M}^{-1}\bm{\beta}^{\star}_{n}$ for the
transformation matrix $\mathbf{M}$ defined in (B.2) in Appendix B.

###### Theorem 3.2

Under the assumptions of Theorem 3.1 and Assumption C.3, with probability
tending to 1 as $n\to\infty$, the local maximizer in Theorem 3.1 satisfies

$\sqrt{n}\bm{\Omega}_{n}(\widehat{\bm{\beta}}_{n}-\bm{\beta}_{n}^{\star})\xrightarrow[]{\mathcal{D}}\mathcal{N}(0,\phi\mathbf{G}),$

where $\bm{\Omega}_{n}$ is a $q\times p$ matrix, $q\leq s_{n}$ and fixed,
$n\bm{\Omega}_{n}\mathbf{U}_{0}\mathbf{B}_{n}^{-1}\mathbf{U}^{T}_{0}\bm{\Omega}_{n}^{T}\to\mathbf{G}$,
$\mathbf{G}$ is a $q\times q$ positive definite matrix,
$\|\mathbf{G}\|_{2}=O(1)$, $\mathbf{U}_{0}$ and $\mathbf{B}_{n}$ are defined
in (B.1) in Appendix B and Assumption C.3, respectively.

In practice, the covariance matrix $\phi\mathbf{G}$ needs to be estimated.
Following the conventional technique (e.g., that of Fan and Li, 2001), we
estimate the variance using the following sandwich formula:
$n\bm{\Omega}_{n}\mathbf{\widehat{U}}_{0}\widehat{\mathbf{B}}_{n}^{-1}\Big{(}\sum_{i=1}^{n}(\tilde{\mathbf{z}}_{i}\tilde{\mathbf{z}}_{i}^{T}\widehat{\varepsilon}_{i}^{2})\Big{)}\widehat{\mathbf{B}}_{n}^{-1}\mathbf{\widehat{U}}_{0}^{T}\bm{\Omega}_{n}^{T},$
where $\mathbf{\widehat{U}}_{0}$ is the orthogonal matrix that spans the null
space of $\widehat{\mathbf{D}}_{null}$, $\widehat{\mathbf{D}}_{null}$ is the
sub-matrix of $\mathbf{D}$, which satisfies
$\widehat{\mathbf{D}}_{null}\widehat{\bm{\beta}}_{n}=\mathbf{0}$,
$\widehat{\mathbf{B}}_{n}=\widetilde{\mathbf{X}}_{signal}^{T}\mathbf{\Sigma}(\check{\mathbf{X}}\widehat{\bm{\beta}}_{n})\widetilde{\mathbf{X}}_{signal}$,
$\widetilde{\mathbf{X}}_{signal}=(\tilde{\mathbf{z}}_{1},...,\tilde{\mathbf{z}}_{n})^{T}=\check{\mathbf{X}}\mathbf{\widehat{U}_{0}}$,
and
$\mathbf{\Sigma}(\bm{\delta})=\operatorname{diag}\big{(}b^{\prime\prime}(\delta_{1}),...b^{\prime\prime}(\delta_{n})\big{)}$
for any $\bm{\delta}\in\mathbb{R}^{n}$.

According to Theorems 3.1 and 3.2, our coefficient estimator achieves the
oracle property in that it identifies the true model with probability tending
to one and enjoys strong consistency property.

Next, with regard to implementation, we introduce the following generalized
local linear approximation (GLLA) algorithm, which is a generalization of the
LLA algorithm (Zou and Li, 2008; Fan et al., 2014).

Generalized local linear approximation (GLLA)

Initialize $\widehat{\bm{\beta}}_{n}^{(0)}=\widehat{\bm{\beta}}_{n}^{init}$,
and
$\mathbf{\widehat{\Theta}}^{(0)}=\big{(}\widehat{\bm{\vartheta}}_{1}^{(0)},\widehat{\bm{\vartheta}}_{2}^{(0)},...,\widehat{\bm{\vartheta}}_{K}^{(0)}\big{)}^{T}$,
where
$\widehat{\bm{\vartheta}}^{(0)}_{k}=\widehat{w}^{(0)}_{k}\mathbf{d}_{k}$, and
$\widehat{w}_{k}^{(0)}=\rho_{\lambda_{n}}^{\prime}(|\mathbf{d}_{k}^{T}\widehat{\bm{\beta}}_{n}^{(0)}|)$
for $1\leq k\leq K$.

For $m$=1, 2, …, repeat the following till convergence.

  1. a.

Solve
$\bm{\widehat{\beta}}_{n}^{(m)}=\mathop{\arg\max}_{\bm{\beta}_{n}}l_{n}(\bm{\widehat{\beta}}_{n})-\lambda_{n}\|\mathbf{\widehat{\Theta}}^{(m-1)}\bm{\widehat{\beta}}_{n}\|_{1}$,
where $\|\cdot\|_{1}$ denotes the $\ell_{1}$ vector norm, such that
$\|\bm{w}\|_{1}=\sum_{i=1}^{n}|w_{i}|$ for any $\bm{w}=(w_{1},...,w_{n})^{T}$.

  2. b.

Update
$\mathbf{\widehat{\Theta}}^{(m)}=\big{(}\widehat{\bm{\vartheta}}_{1}^{(m)},\widehat{\bm{\vartheta}}_{2}^{(m)},...,\widehat{\bm{\vartheta}}_{K}^{(m)}\big{)}^{T}$,
where
$\widehat{\bm{\vartheta}}^{(m)}_{k}=\widehat{w}^{(m)}_{k}\mathbf{d}_{k}$, and
$\widehat{w}_{k}^{(m)}=\rho_{\lambda_{n}}^{\prime}(|\mathbf{d}_{k}^{T}\bm{\widehat{\beta}}_{n}^{(m)}|)$
for $1\leq k\leq K$.

###### Assumption 2.2’

$\rho^{\prime}(0+,\lambda)\geq a_{1}$; $\rho^{\prime}(t,\lambda)\geq a_{1}$
for $t\in(0,a_{2}\lambda)$; $\rho^{\prime}(t,\lambda)=0$ for $t>a\lambda$ with
constants $a>a_{2}>0$, and $a_{1}>0$.

###### Remark 3.1

Assumption 2.2’ holds for the folded concave penalty, such as SCAD and MCP.

Proposition 3.1 gives the property of the GLLA algorithm.

###### Proposition 3.1

Under the assumptions of Theorem 3.1 and Assumption 2.2’, assume that the
oracle estimator $\bm{\widehat{\beta}}_{n}$ in Theorem 3.1 is unique. In
addition, assume that

$\max_{s_{n}+1\leq j\leq p}|\widehat{\theta}_{nj}^{init}|\leq
a_{2}\lambda_{n},\quad and\quad\min_{1\leq k\leq
K_{1}}|\mathbf{d}_{k}^{T}\widehat{\bm{\beta}}_{n}^{init}|\geq a\lambda_{n},$
(3.2)

for $a_{2}$ and $a$ defined in Assumption 2.2’ and
$\bm{\widehat{\theta}}_{n}^{init}=\mathbf{M}^{-1}\bm{\widehat{\beta}}_{n}^{init}$.
Then with probability tending to 1, the GLLA algorithm initialized by
$\widehat{\bm{\beta}}_{n}^{init}$ finds the oracle estimator
$\widehat{\bm{\beta}}_{n}$ after one iteration.

###### Remark 3.2

One potential choice of the initial value is the generalized lasso estimator,
which uses $\ell_{1}$ penalty. The algorithm for solving generalized lasso
problem is discussed in Tibshirani and Taylor, (2011) and Arnold and
Tibshirani, (2016). When $\mathbf{D}=\mathbf{I}$, the error bound exists for
standard lasso estimator under proper designs; see (C1) and Corollary 3 in Fan
et al., (2014).

About the computation cost of the algorithm, in particular the computation
complexity with respect to $L_{n}$, heuristically, the generalized lasso needs
$O(\max(K^{2}n,Kn^{2}))$ operations; see, for example, Tibshirani and Taylor,
(2011). The total computation costs of Steps I and II, therefore, are
$O(\max(dL_{n},n)dL_{n}n)$ and $O(\max(dL_{n},n)dL_{n}n)$, respectively. In
practice, the maximum iterations can be set as $O(\log n)$ and the algorithm
converges fast within a few iterations. The total computation cost of the
algorithm is $O\big{(}(\log n)\max(dL_{n},n)dL_{n}n\big{)}$, which is
polynomial in $L_{n}$, implying that the algorithm is scalable to high
dimensions.

###  3.2 Optimal Value Estimation and Inference

One important advantage of our DROVE approach is that it allows for proper
inference of the value associated with a decision rule, in particular, the
optimal decision that achieves the maximum value.

Let $\mathbf{\mathcal{X}}$ be the personal characteristic vector that belongs
to the testing population. We use a different notation,
$\mathbf{\mathcal{X}}$, to distinguish the testing population from the
estimation sample $\mathbf{X}$ and stress that $\mathbf{\mathcal{X}}$ is
independent of $\mathbf{Y}$. Suppose that the value function (2.1) holds for
$\mathbf{\mathcal{X}}$; thus it follows that
$E(R|\mathbf{\mathcal{X}},A)=Q_{n}(\mathbf{\mathcal{X}},A)$. Let
$\pi^{*}_{n}(\cdot)$ denote the optimal decision, and let $Q^{*}_{n}(\cdot)$
denote the optimal Q-function associated with $\pi^{*}_{n}(\cdot)$. More
formally,

$\displaystyle\pi^{*}_{n}(\mathbf{\mathcal{X}}):=\min\\{A:Q_{n}(\mathbf{\mathcal{X}},A)=\max_{A_{(1)}\leq
A\leq A_{(L_{n})}}Q_{n}(\mathbf{\mathcal{X}},A)\\},$ (3.3) $\displaystyle
Q^{*}_{n}(\mathbf{\mathcal{X}}):={\bm{\psi}^{\star}_{0}}^{T}\mathbf{\mathcal{X}}+\max\Big{(}0,({\bm{\psi}^{\star}_{k}}^{T}\mathbf{\mathcal{X}})_{k=2,...,L_{n}}\Big{)}.$
(3.4)

Suppose that $\mathbf{\mathcal{X}}\sim F$, and the optimal value is
$E\big{(}Q^{*}_{n}(\mathbf{\mathcal{X}})\big{)}=\int
Q^{*}_{n}(\mathbf{\mathcal{X}})dF(\mathbf{\mathcal{X}})$. The expectation is
taken over the distribution of the testing population $\mathbf{\mathcal{X}}$
given the working model $Q_{n}$. More generally, for any given decision rule
$\pi(\cdot):\mathbb{R}^{d}\to[0,1]$ and for
$\mathbf{\mathcal{X}}\in\mathbb{R}^{d}$, we use
$\check{\mathbf{\mathcal{X}}}_{\pi}$ to denote a length-$p$ vector

$\check{\mathbf{\mathcal{X}}}_{\pi}=\Big{(}(\mathbf{\mathcal{X}})^{T},(\mathbf{\mathcal{X}})^{T}\cdot\mathbf{1}_{\\{\pi(\mathbf{\mathcal{X}})\in[A_{(2)},A_{(3)})\\}},...,(\mathbf{\mathcal{X}})^{T}\cdot\mathbf{1}_{\\{\pi(\mathbf{\mathcal{X}})\in[A_{(L_{n})},A_{(L_{n}+1)})\\}}\Big{)}^{T}.$
(3.5)

Using the notation in (3.5), the optimal Q-function is
$Q^{*}_{n}(\mathbf{\mathcal{X}})={\bm{\beta}^{\star}_{n}}^{T}\check{\mathbf{\mathcal{X}}}_{\pi^{*}_{n}}$,
and the optimal value is: $E\big{(}Q^{*}_{n}(\mathbf{\mathcal{X}})\big{)}=\int
Q^{*}_{n}(\mathbf{\mathcal{X}})dF(\mathbf{\mathcal{X}})={\bm{\beta}^{\star}_{n}}^{T}\int\check{\mathbf{\mathcal{X}}}_{\pi^{*}_{n}}dF(\mathbf{\mathcal{X}})={\bm{\beta}^{\star}_{n}}^{T}E(\check{\mathbf{\mathcal{X}}}_{\pi^{*}_{n}}).$

Let $\mathbb{P}_{N}(\cdot)$ denote the empirical mean measure for a sample of
size $N$: $\mathbb{P}_{N}(\bm{\omega})=\sum_{i=1}^{N}\omega_{i}/N$ for any
$\bm{\omega}=(\omega_{i})_{1\leq i\leq N}$. Given a testing sample of size
$N$, $(\mathbf{\mathcal{X}}_{i})_{1\leq i\leq N}\sim F$, we estimate the
optimal value by

$\mathbb{P}_{N}\widehat{Q}^{*}_{n}(\mathbf{\mathcal{X}})=\frac{1}{N}\sum_{i=1}^{N}\widehat{Q}^{*}_{n}(\mathbf{\mathcal{X}}_{i})=\widehat{\bm{\beta}}_{n}^{T}\Big{(}\frac{1}{N}\sum_{i=1}^{N}\check{\mathbf{\mathcal{X}}}_{i,\widehat{\pi}^{*}_{n}}\Big{)}=\widehat{\bm{\beta}}_{n}^{T}\mathbb{P}_{N}(\check{\mathbf{\mathcal{X}}}_{\widehat{\pi}^{*}_{n}}),$
(3.6)

where

$\displaystyle\widehat{\pi}^{*}_{n}(\mathbf{\mathcal{X}}):=\min\\{A:\widehat{Q}_{n}(\mathbf{\mathcal{X}},A)=\max_{A_{(1)}\leq
A\leq A_{(L_{n})}}\widehat{Q}_{n}(\mathbf{\mathcal{X}},A)\\},$ (3.7)
$\displaystyle\widehat{Q}^{*}_{n}(\mathbf{\mathcal{X}}):=\widehat{\bm{\psi}}_{0}^{T}\mathbf{\mathcal{X}}+\max\Big{(}0,(\widehat{\bm{\psi}}^{T}_{k}\mathbf{\mathcal{X}})_{k=2,...,L_{n}}\Big{)}.$
(3.8)

###### Remark 3.3

To estimate and make inference about the optimal value, we use the testing
sample that is distinct from the Q-function estimation sample. This approach
is similar in spirit to the sample-splitting method in machine learning
literature and is advocated by many recent studies on treatment effect
evaluation, such as Chernozhukov et al., (2017), Wager and Athey, (2018), and
Athey and Wager, (2019). The sample-splitting reduces bias and facilitates the
valid inference of the optimal value. A discussion about the results of
optimal value estimation using the estimation sample is in Appendix D of the
supplementary materials. On the other hand, because we are mostly interested
in predicting the decision-making effect on a broad population that extends
beyond the estimation sample for which the decision-making effects are
observable, it is also practically reasonable to use the testing sample to
evaluate the effect of optimal decision-making.

###### Assumption 3.1

Assume

(1) $s_{n}=o(n^{\delta_{1}})$ and $L_{n}=o(n^{\delta_{2}})$ for some
$\delta_{1},\delta_{2}>0$, and $2\delta_{1}+\delta_{2}<1$.

(2) $(\mathbf{\mathcal{X}}_{j})_{1\leq j\leq N}$ are i.i.d. and independent of
$\mathbf{Y}$, $E\big{(}Q^{*}_{n}(\mathbf{\mathcal{X}})^{2}\big{)}<\infty$,
$n=O(N)$, and $N=O(n^{M})$ for some $M>1$. There exist some constants
$r_{0},r_{1}$ and $r_{2}>0$, for any $t>0$, $P(|\chi_{j}|>t)\leq
r_{0}\exp(-r_{1}t^{r_{2}})$ for all $1\leq j\leq d$, where
$\mathbf{\mathcal{X}}=:(\chi_{1},...,\chi_{d})^{T}$.

For the statistical inference of the optimal value, we further impose
regularity condition; see Assumption C.4 in Appendix C of the supplmentary
materials.

Theorem 3.3 gives the asymptotic distribution of the estimated optimal value.

###### Theorem 3.3

Under the assumptions of Theorem 3.1 and Assumption 3.1, with probability
tending to 1 as $n,N\to\infty$, the
$\mathbb{P}_{N}\widehat{Q}^{*}_{n}(\mathbf{\mathcal{X}})$ defined in (3.6)
satisfies

(i)

$\mathbb{P}_{N}\widehat{Q}^{*}_{n}(\mathbf{\mathcal{X}})-E\big{(}Q^{*}_{n}(\mathbf{\mathcal{X}})\big{)}=o_{p}(1).$
(3.9)

(ii) If, in addition, $2\delta_{1}+\delta_{2}<1/2$, Assumptions C.3 and C.4
hold,
$\lim_{n\to\infty}nE{\check{\mathbf{\mathcal{X}}}^{T}_{\pi^{*}_{n}}}\mathbf{U}_{0}\mathbf{B}_{n}^{-1}\mathbf{U}^{T}_{0}E\check{\mathbf{\mathcal{X}}}_{\pi^{*}_{n}}\phi=\sigma_{1}^{2}$
and
$\lim_{n,N\to\infty}\operatorname{var}\big{(}Q^{*}_{n}(\mathbf{\mathcal{X}})\big{)}n/N=\sigma_{2}^{2}$
for some constants $\sigma_{1}>0$, $\sigma_{2}\geq 0$. Let
$\sigma^{2}_{*}=\sigma_{1}^{2}+\sigma_{2}^{2}$, then

$\sqrt{n}\Big{(}\mathbb{P}_{N}\widehat{Q}^{*}_{n}(\mathbf{\mathcal{X}})-E\big{(}Q^{*}_{n}(\mathbf{\mathcal{X}})\big{)}\Big{)}\xrightarrow[]{\mathcal{D}}\mathcal{N}(0,\sigma^{2}_{*}).$
(3.10)

In practice, in order to apply Theorem 3.3 to perform feasible statistical
inference, we need to estimate the variance. We estimate the variance using
the following sandwich formula:

$\widehat{\sigma}^{2}_{*}=n\mathbb{P}_{N}\check{\mathbf{\mathcal{X}}}_{\widehat{\pi}^{*}_{n}}^{T}\mathbf{\widehat{U}}_{0}\widehat{\mathbf{B}}_{n}^{-1}\Big{(}\sum_{i=1}^{n}(\tilde{\mathbf{z}}_{i}\tilde{\mathbf{z}}_{i}^{T}\widehat{\varepsilon}_{i}^{2})\Big{)}\widehat{\mathbf{B}}_{n}^{-1}\mathbf{\widehat{U}}_{0}^{T}\mathbb{P}_{N}\check{\mathbf{\mathcal{X}}}_{\widehat{\pi}^{*}_{n}}+\widehat{\operatorname{var}}\big{(}\widehat{Q}^{*}_{n}(\mathbf{\mathcal{X}})\big{)}n/N,$
(3.11)

where $\mathbb{P}_{N}{\check{\mathbf{\mathcal{X}}}}_{\widehat{\pi}^{*}_{n}}$
and
$\widehat{\operatorname{var}}\big{(}\widehat{Q}^{*}_{n}(\mathbf{\mathcal{X}})\big{)}$
are the sample mean of $\check{\mathbf{\mathcal{X}}}_{\widehat{\pi}^{*}_{n}}$
and sample variance of $\widehat{Q}^{*}_{n}(\mathbf{\mathcal{X}})$,
respectively, based on the testing sample $(\mathbf{\mathcal{X}}_{j})_{1\leq
j\leq N}$.

###### Remark 3.4

Our Theorems 3.1 and 3.3 are established for the continuous-action decision
making in which the discretization level $L_{n}\to\infty$ and hence require
intrinsically different methodological design and novel mathematical treatment
than existing studies on the binary case (e.g., Shi et al., 2016). We device a
innovative DROVE approach, and obtain a new result (3.1) in Theorem 3.1, which
ensures that the adjacent treatment levels can be identified. Unlike Shi et
al., (2016), we work under the assumptions for transformed design matrices and
transformed parameters (Assumptions C.1–C.3). More essentially, we consider a
more relaxed constraint on the general penalty matrix $\mathbf{D}$. We use a
partial reparametrization technique to show the statistical properties of
DROVE; see Appendix B of the supplmentary materials for more details.

###### Remark 3.5

For continuous action, there exists no nonparametric/semiparametric approach
that yields a $\sqrt{n}$-consistent estimator of effect curve without imposing
parametric assumptions; see, e.g., the discussion in Kennedy et al., (2017).
In this study, on the other hand, we focus on the high-dimensional parametric
model (2.1), in which the parameter space is sparse in the general sense,
hence our approach recovers the true sparse model consistently. In addition,
we allow the discretization level $L_{n}$ to diverge slowly with $n$, such
that $L_{n}s_{n}^{2}=o(\sqrt{n})$, under which setting, inference of the
optimal value is obtainable.

In addition to the optimal value, our approach also allows for inference of
the value difference between the optimal decision and a given decision rule.
For a given decision rule, say, $\pi(\cdot):\mathbb{R}^{d}\to[0,1]$, the
associated Q-function and the value of $\pi(\cdot)$ are
$Q_{n}(\mathbf{\mathcal{X}},\pi)={\bm{\psi}^{\star}_{0}}^{T}\mathbf{\mathcal{X}}+\sum_{k=2}^{L_{n}}{\bm{\psi}^{\star}_{k}}^{T}\mathbf{\mathcal{X}}\cdot\mathbf{1}_{\\{\pi(\mathbf{\mathcal{X}})\in[A_{(k)},A_{(k+1)})\\}}$,
and

$\displaystyle E\big{(}Q_{n}(\mathbf{\mathcal{X}},\pi)\big{)}$
$\displaystyle=\int
Q_{n}(\mathbf{\mathcal{X}},\pi)dF(\mathbf{\mathcal{X}})={\bm{\beta}^{\star}_{n}}^{T}E(\check{\mathbf{\mathcal{X}}}_{\pi}).$
(3.12)

The value difference between $\pi^{*}_{n}$ and $\pi$ is
$E\big{(}Q^{*}_{n}(\mathbf{\mathcal{X}})\big{)}-E\big{(}Q_{n}(\mathbf{\mathcal{X}},\pi)\big{)}$,
which is also called the regret of the policy $\pi$; see, for example, Athey
and Wager, (2019).

We estimate the value associated with $\pi(\cdot)$ using
$\mathbb{P}_{N}\widehat{Q}_{n}(\mathbf{\mathcal{X}},\pi)$, that is,

$\mathbb{P}_{N}\widehat{Q}_{n}(\mathbf{\mathcal{X}},\pi)=\frac{1}{N}\sum_{i=1}^{N}\widehat{Q}_{n}\Big{(}\mathbf{\mathcal{X}}_{i},\pi(\mathbf{\mathcal{X}}_{i})\Big{)}=\widehat{\bm{\beta}}_{n}^{T}\mathbb{P}_{N}(\check{\mathbf{\mathcal{X}}}_{\pi}),$
(3.13)

and we estimate the value difference between $\pi^{*}_{n}$ and $\pi$ using
$\mathbb{P}_{N}\widehat{Q}^{*}_{n}(\mathbf{\mathcal{X}})-\mathbb{P}_{N}\widehat{Q}_{n}(\mathbf{\mathcal{X}},\pi)$.

Proposition 3.2 gives the asymptotic distribution of the estimated value
difference
$\mathbb{P}_{N}\widehat{Q}^{*}_{n}(\mathbf{\mathcal{X}})-\mathbb{P}_{N}\widehat{Q}_{n}(\mathbf{\mathcal{X}},\pi)$.

###### Proposition 3.2

Under the assumptions of Theorem 3.1 and Assumption 3.1, given a decision rule
$\pi(\cdot)$: $\mathbb{R}^{d}\rightarrow[0,1]$, with probability tending to 1
as $n,N\to\infty$, the $\mathbb{P}_{N}\widehat{Q}^{*}_{n}$ and
$\mathbb{P}_{N}\widehat{Q}_{n}(\mathcal{X},\pi)$ defined in (3.6) and (3.13)
satisfy

(i)
$\quad\mathbb{P}_{N}\widehat{Q}^{*}_{n}(\mathbf{\mathcal{X}})-\mathbb{P}_{N}\widehat{Q}_{n}(\mathbf{\mathcal{X}},\pi)-\Big{(}EQ^{*}_{n}(\mathbf{\mathcal{X}})-EQ_{n}(\mathbf{\mathcal{X}},\pi)\Big{)}=o_{p}(1).$

(ii) If, in addition, $2\delta_{1}+\delta_{2}<1/2$, Assumptions C.3 and C.4
hold,
$\lim_{n\to\infty}nE({\check{\mathbf{\mathcal{X}}}_{\pi^{*}_{n}}}-{\check{\mathbf{\mathcal{X}}}_{\pi}})^{T}\mathbf{U}_{0}\mathbf{B}_{n}^{-1}\mathbf{U}^{T}_{0}E({\check{\mathbf{\mathcal{X}}}_{\pi^{*}_{n}}}-{\check{\mathbf{\mathcal{X}}}_{\pi}})\phi=\sigma^{2}_{1}$,
and
$\lim_{n,N\to\infty}\operatorname{var}\Big{(}Q^{*}_{n}(\mathbf{\mathcal{X}})-Q_{n}(\mathbf{\mathcal{X}},\pi)\Big{)}n/N=\sigma^{2}_{2}$
for some constants $\sigma_{1}>0$, and $\sigma_{2}\geq 0$. Let
$\sigma^{2}_{*,\pi}=\sigma^{2}_{1}+\sigma^{2}_{2}$, then

$\sqrt{n}\Big{(}\mathbb{P}_{N}\widehat{Q}^{*}_{n}(\mathbf{\mathcal{X}})-\mathbb{P}_{N}\widehat{Q}_{n}(\mathbf{\mathcal{X}},\pi)-\big{(}EQ^{*}_{n}(\mathbf{\mathcal{X}})-EQ_{n}(\mathbf{\mathcal{X}},\pi)\big{)}\Big{)}\xrightarrow[]{\mathcal{D}}\mathcal{N}(0,\sigma^{2}_{*,\pi}).$

Similar to (3.11), we can esimate $\sigma^{2}_{*,\pi}$ using the following
sandwich formula

$\widehat{\sigma}^{2}_{*,\pi}=n\mathbb{P}_{N}({\check{\mathbf{\mathcal{X}}}}_{\widehat{\pi}^{*}_{n}}-\check{\mathbf{\mathcal{X}}}_{\pi})^{T}\mathbf{\widehat{U}}_{0}\widehat{\mathbf{B}}_{n}^{-1}\Big{(}\sum_{i=1}^{n}(\tilde{\mathbf{z}}_{i}\tilde{{\mathbf{z}}}_{i}^{T}\widehat{\varepsilon}_{i}^{2})\Big{)}\widehat{\mathbf{B}}_{n}^{-1}\mathbf{\widehat{U}}_{0}^{T}\big{(}\mathbb{P}_{N}({\check{\mathbf{\mathcal{X}}}}_{\widehat{\pi}^{*}_{n}}-\check{\mathbf{\mathcal{X}}}_{\pi})\big{)}\\\
+\widehat{\operatorname{var}}\big{(}\widehat{Q}^{*}_{n}(\mathbf{\mathcal{X}})-\widehat{Q}_{n}(\mathbf{\mathcal{X}},\pi)\big{)}n/N,$
(3.14)

where $\mathbb{P}_{N}\check{\mathbf{\mathcal{X}}}_{\pi}$ and
$\widehat{\operatorname{var}}\big{(}\widehat{Q}^{*}_{n}(\mathbf{\mathcal{X}})-\widehat{Q}_{n}(\mathbf{\mathcal{X}},\pi)\big{)}$
are the sample mean of $\check{\mathbf{\mathcal{X}}}_{\pi}$ and sample
variance of
$\widehat{Q}^{*}_{n}(\mathbf{\mathcal{X}})-\widehat{Q}_{n}(\mathbf{\mathcal{X}},\pi)$,
respectively.

##  4 Simulation Study

###  4.1 Simulation Setup

We generate data from the model
$Y=\bm{\psi}_{0}^{T}\mathbf{X}+\sum_{k=2}^{L_{n}}\bm{\psi}_{k}^{T}\mathbf{X}\cdot\mathbf{1}_{\\{A=A_{(k)}\\}}+\varepsilon$
and calibrate the parameters based on the empirical data as described in
Section 5. Specifically, the covariates $\mathbf{X}$ is a vector of length 9
and randomly drawn from the personal characteristics that are used in the
Q-learning estimation in the empirical study, which includes three binary
variables, two categorical variables, three continuous variables and one
intercept term.

We divide the continuous action interval $[0,1]$ into 11 discrete levels,
i.e., $L_{n}=11$. The stock ratio $A_{i}$ for each $\mathbf{X}_{i}$ is
randomly assigned from $\\{0,0.1,...,1\\}$ with equal probability. The choice
of $L_{n}$ is the same as that in the empirical studies. Empirically, in order
to balance the applicability of the methodology with the approximation
accuracy of the working model, the choice of $L_{n}$ takes both the
theoretical rate and the common practice into consideration. For example, in
our personalized asset allocation study using the HRS data, the training
sample size is $n=$ 2,000, $n^{1/3}\approx 13$, and we choose $L_{n}=11$. On
the other hand, it is a common practice for mutual funds to use a 10%
incrementation and make 0%, 10%, 20%, …, 100% recommendations.111See:
https://institutional.vanguard.com/assets/pdf/vrpa/InvestorQuestionnaireAssetAllocationInsert.pdf.
We illustrate the case when $L_{n}=20$ in Appendix G of the supplementary
materials. When a larger $n$ is available, nice numerical properties can be
expected for a larger range of $L_{n}$.

The Q-function parameters are learned from the empirical study and with hard
thresholding (see Appendix F of the supplementary materials for a full
description). The number of coefficients is therefore $p=11\times 9=99$. The
noise, $\varepsilon$, is generated from $\mathcal{N}(0,0.5^{2})$. We design
the generalized penalty matrix
$\mathbf{D}=(\mathbf{D}_{signal}^{T},\mathbf{D}^{T}_{null})^{T}$ according to
(2.2). The
$\bm{\beta}_{n}=(\bm{\psi}_{0}^{T},\bm{\psi}_{2}^{T},...,\bm{\psi}_{L_{n}}^{T})^{T}$
has 55 zeros, and $\text{rank}(\mathbf{D}_{null})=76$, thus the nonsparsity
coefficient is $s_{n}=99-76=23$. The minimal signal is $g_{n}=0.4$.

###  4.2 Simulation Results

We estimate the model using the DROVE method. For comparison, we also evaluate
the results using the standard lasso (std-lasso) and standard scad222Existing
approaches for the decision-making problem under the high-dimensional setting
focus on binary problems, among which one of the most comparable to ours is
Shi et al., (2016), who use the SCAD estimator to fit Q-function. The std-scad
estimator we evaluate, therefore, can be considered as an ad-hoc extension of
Shi et al., (2016) by discretization to the continuous-action setting.  (std-
scad). We also present the results of the infeasible oracle estimator. The
oracle solution is obtained by performing a least squares regression of the
response $\mathbf{Y}$ over $\mathbf{\widetilde{X}}_{signal}$, as defined in
(C.2) in Appendix C. The regression yields the oracle transformed estimator
$\widehat{\bm{\theta}}_{n}$, and the oracle estimator
$\widehat{\bm{\beta}}_{n}$ is obtained by the transformation
$\widehat{\bm{\beta}}_{n}=\mathbf{M}\widehat{\bm{\theta}}_{n}$. The tuning
parameters are chosen by minimizing the validation error.

We conduct 500 independent replications with the sample sizes $n=$ 2000 and
$n=$ 3000. We evaluate the coefficient estimation accuracy of various methods
by the $\ell_{2}$ and $\ell_{1}$ errors, and measure the parameter selection
accuracy by the false positive rate (FP/N, the number of all false positives
divided by that of all total negatives in $\widehat{\bm{\beta}}_{n}$), and
false negative rate (FN/P, the number of all false negatives divided by that
of all total positives in $\widehat{\bm{\beta}}_{n}$). The total negatives (N)
and the total positives (P) in $\bm{\beta}_{n}$ are $55$ and $44$,
respectively. For counting the false positives and false negatives, we set the
threshold level to be $10^{-4}$. The results are in Table 1. The results show
that our DROVE estimator has the lowest estimation error among all compared
methods.

Table 1: Coefficient estimation and parameter selection accuracy of various
methods

Error | oracle | DROVE | std-scad | std-lasso
---|---|---|---|---
$n=$ 2000
$\|\bm{\widehat{\beta}}_{n}-\bm{\beta}_{n}\|_{2}$ | 1.683(0.322) | 4.370(0.986) | 4.844(0.788) | 5.348(0.904)
$\|\bm{\widehat{\beta}}_{n}-\bm{\beta}_{n}\|_{1}$ | 8.158(1.628) | 24.054(7.083) | 29.113(6.089) | 37.195(6.034)
FP/N | - | 0.148(0.083) | 0.242(0.109) | 0.740(0.082)
FN/P | - | 0.135(0.070) | 0.177(0.069) | 0.073(0.046)
$n=$ 3000
$\|\bm{\widehat{\beta}}_{n}-\bm{\beta}_{n}\|_{2}$ | 1.348(0.239) | 3.221(0.862) | 3.909(0.723) | 4.344(0.699)
$\|\bm{\widehat{\beta}}_{n}-\bm{\beta}_{n}\|_{1}$ | 6.558(1.225) | 16.486(5.648) | 23.033(5.555) | 30.466(4.964)
FP/N | - | 0.095(0.076) | 0.237(0.118) | 0.772(0.074)
FN/P | - | 0.094(0.068) | 0.124(0.062) | 0.042(0.035)

NOTE: Coefficient estimation accuracy is measured by the $\ell_{2}$ and
$\ell_{1}$ errors, and parameter selection accuracy is measured by the false
positive rate (FP/N) and false negative rate (FN/P). We report the mean and
standard deviation (in parentheses) from 500 replications.

Next, we estimate the optimal value and construct its confidence interval
(CI). According to Theorem 3.3, the confidence interval of the optimal value
at a significance level $\alpha>0$
is
$\big{(}\mathbb{P}_{N}\widehat{Q}^{*}_{n}(\mathbf{\mathcal{X}})\big{)}\pm\widehat{\sigma}(\mathbb{P}_{N}\widehat{Q}^{*}_{n})\times\mathcal{N}^{-1}(1-0.5\alpha)$,
where $\mathcal{N}(\cdot)$ is the cumulative distribution function of the
standard normal distribution,
$\widehat{\sigma}(\mathbb{P}_{N}\widehat{Q}^{*}_{n})=\widehat{\sigma}^{*}_{n}/\sqrt{n}$.
The feasible variance estimator in (3.11) is used. The estimation sample sizes
are $n=$ 2000 and 3000, and the testing sample sizes are $N=$ 5000 and 15,000.
The testing sample is randomly generated independently based on empirical data
of the personal characteristics. Table 2 reports the estimated optimal and
empirical coverages of 90%, 95% and 99% CI from 500 replications. The results
suggest that the estimated optimal value is close to the actual one and the
estimated CIs show good accuracy. When the estimation sample size increases
from 2000 to 3000, the empirical coverage goes closer to the nominal one.

Table 2: Estimated optimal value and empirical coverage of confidence
intervals for the optimal value

($n,N$) | (2000,5000) | (2000, 15,000) | (3000,5000) | (3000, 15,000)
---|---|---|---|---
Estimates |  $E(Q^{*}_{n})$=0.667
$\mathbb{P}_{N}\widehat{Q}^{*}_{n}$ | 0.665 | 0.663 | 0.664 | 0.667
$\widehat{\sigma}(\mathbb{P}_{N}\widehat{Q}^{*}_{n})$ | 0.024 | 0.023 | 0.020 | 0.019
Coverage |
90% CI | 0.838 | 0.828 | 0.874 | 0.864
95% CI | 0.898 | 0.896 | 0.932 | 0.928
99% CI | 0.958 | 0.952 | 0.988 | 0.990

NOTE: The estimation sample sizes are $n=$ 2000 and 3000, and the testing
sample sizes are $N=$ 5000 and 15,000. The results are the empirical coverage
of 90%, 95% and 99% CI from $500$ replications, together with the mean of the
estimated optimal values, $\mathbb{P}_{N}(\widehat{Q}^{*}_{n})$, and the mean
of the estimated standard deviations,
$\widehat{\sigma}(\mathbb{P}_{N}\widehat{Q}^{*}_{n})$.

Finally, we construct confidence intervals for the difference between the
optimal value and the value of a pre-specified decision rule $\pi(\cdot)$. We
build confidence intervals according to Proposition 3.2 and use the feasible
standard deviation estimator in (3.14). We calculate the empirical coverage
from 500 replications. The results are in Table 3. The results show that the
estimated value difference is close to the true one for different estimation
sample sizes and testing sample sizes. Moreover, the estimated CIs exhibit
good accuracy. When the estimation sample size increases from 2000 to 3000,
the empirical coverage of the confidence interval goes closer to the nominal
level.

Table 3: Estimated value differences and the empirical coverage of confidence
intervals for the value difference between the optimal decision and pre-
specified decision rules.

($n,N$) | (2000,5000) | (2000, 15,000) | (3000,5000) | (3000, 15,000)
---|---|---|---|---
Estimates | $E(Q^{*}_{n})-E(Q_{n,\pi_{1}})=1.037$
$\mathbb{P}_{N}\widehat{Q}^{*}_{n}-\mathbb{P}_{N}\widehat{Q}_{n,\pi_{1}}$ | 1.034 | 1.030 | 1.039 | 1.040
$\widehat{\sigma}(\mathbb{P}_{N}\widehat{Q}^{*}_{n}-\mathbb{P}_{N}\widehat{Q}_{n,\pi_{1}})$ | 0.032 | 0.030 | 0.026 | 0.025
Coverage |
90% CI | 0.830 | 0.816 | 0.856 | 0.844
95% CI | 0.896 | 0.888 | 0.914 | 0.902
99% CI | 0.958 | 0.954 | 0.978 | 0.978
Estimates | $E(Q^{*}_{n})-E(Q_{n,\pi_{2}})=1.734$
$\mathbb{P}_{N}\widehat{Q}^{*}_{n}-\mathbb{P}_{N}\widehat{Q}_{n,\pi_{2}}$ | 1.736 | 1.732 | 1.730 | 1.733
$\widehat{\sigma}(\mathbb{P}_{N}\widehat{Q}^{*}_{n}-\mathbb{P}_{N}\widehat{Q}_{n,\pi_{2}})$ | 0.036 | 0.036 | 0.030 | 0.029
Coverage |
90% CI | 0.846 | 0.846 | 0.866 | 0.862
95% CI | 0.902 | 0.908 | 0.928 | 0.920
99% CI | 0.962 | 0.958 | 0.988 | 0.988

NOTE: The estimation sample sizes are $n=$ 2000 and 3000, and the testing
sample sizes are $N=$ 5000 and 15,000. The results are the empirical coverage
of 90%, 95% and 99% CI from 500 replications, together with the mean of the
estimated value difference
$\mathbb{P}_{N}\widehat{Q}^{*}_{n}-\mathbb{P}_{N}\widehat{Q}_{n,\pi}$ and the
mean of the estimated standard deviation
$\widehat{\sigma}(\mathbb{P}_{N}\widehat{Q}^{*}_{n}-\mathbb{P}_{N}\widehat{Q}_{n,\pi})$.

More simulation examples under different simulation settings with various
nonsparsity degree $s_{n}$ and various choices of $L_{n}$ are in Appendix G of
the supplementary materials. The results show robustness in the performance of
our estimator under various settings.

In summary, the simulation results show that our proposed methodology gives
rise to better model estimation accuracy than the standard penalized
regression when sparsity occurs to the linear projection of the coefficients.
Our approach also allows for valid inferences of the optimal value and value
differences using the feasible variance estimator, which performs well in
terms of empirical coverage.

##  5 Empirical Study

###  5.1 Individualized Asset Allocation Optimization

We study the asset allocation optimization problem under a consumption-based
utility framework. Specifically, we adopt the additive utility with constant
relative risk aversion (CRRA, e.g., Hall, 1978). The utility has the form
$U(C_{1},C_{2},...,C_{T})=\sum_{t=1}^{T}\gamma^{t}u(C_{t})$, where $C_{t}$ is
the consumption at time $t$, $u(C)=C^{1-\rho}/(1-\rho)$, $\rho$ is the risk
aversion parameter that reflects the sensitivity of utility to the randomness
in the income, and $\gamma$ is the discount rate (set as $0.96$; see, e.g.,
Gourinchas and Parker, 2002). The consumption evolves according to the net
income $I_{t}$ and wealth $W_{t}$ constraint $I_{t}-C_{t}=W_{t+1}-W_{t}$. The
objective is to maximize the expected utility by choosing the optimal
proportion of the total financial asset that is invested in stocks, i.e.,
stock ratio, $A$. The utility optimization problem spans the preceding five
years (i.e., $T=5$).

Formally, the optimal stock ratio solves the following optimization
problem333In this exploratory study, we consider a simplified situation where
utilities are only determined by the consumption within the next five years.
This does not mean, however, that individuals do not care what happens after
five years. In fact, in our model, consumption is considered optimal with
respect to wealth, in particular, $C_{5}$ depends on $W_{5}$. An individual’s
preference for a high $C_{5}$ is typically consistent with a high $W_{5}$,
which means that one does care about having enough savings for five years
later. In future works where multi-stage decision-making is studied, this
would become even less an issue. :
$\pi^{*}(\mathbf{X})=\mathop{\text{argmax}}_{A\in[0,1]}E\big{(}U(C_{1},C_{2},...C_{5}|\mathbf{X},A)\big{)},$
where $\mathbf{X}$ is the personal characteristics. The asset allocation
decision $A$ determines the distribution of income and consequently influences
the dynamics of wealth and consumption. We consider randomness in income from
not only the financial returns but also from other sources such as wages and
medical expenses, the distributions of which vary from individual to
individual. Furthermore, the risk aversion parameter $\gamma$ also differs
between individuals and relates to their personal characteristics.

###  5.2 Data

We use the Health and Retirement Study (HRS) data from 1992 to 2014. HRS is a
national-level longitudinal survey of more than 22,000 U.S. residents over the
age of 50. We include key variables on households finance (e.g., total
financial wealth, value of stocks held and income) and individual
characteristics (e.g., age, marital status, education level, health condition
and working status) in our analysis. We also incorporate the Consumption and
Activities Mail Survey (CAMS) data that contain consumption information, which
began in 2002, complementing the HRS. The CAMS data cover approximately 6,800
people, which is a subset of respondents covered in the HRS data. The relevant
variable in CAMS that we incorporate in our analysis is the nondurable
consumption. The raw HRS and CAMS data are noisy and contain many missing
observations. In addition, the observed stock ratios are concentrated around
zero. Instead of using the raw data, we use pseudo observations that are
randomly generated from models of trajectory paths for consumption and income.
We also learn risk aversion parameters by matching observed investment
profiles with individual characteristics. Details of the model-based random
experiment generation are in Appendix H of the supplementary materials.

###  5.3 Statistical Learning Implementation and Results

We conduct the analysis based on three samples: _estimation sample_ ,
_evaluation sample_ and _testing sample_. For the observations that are
covered by both CAMS and HRS, we split them into an estimation sample and an
evaluation sample. The estimation of the Q-function is performed on the
_estimation sample_ , which contains 2,000 observations. The model fitting is
evaluated on the _evaluation sample_ , which includes 1,000 observations. We
put all the 15,000+ observations that are not used for the model estimation in
the _testing sample_ , which includes those that are not covered by CAMS. The
_testing sample_ is used in evaluating the value of strategies.

####  5.3.1 Q-function estimation

We include the following covariates in the Q-function (2.1): gender,
education, unhealthiness, working status, marital status, age, household
wealth, income, and an intercept term, thus, $d=9$. We follow the common
practice in asset allocation applications and adopt the 0.1 incrementation in
stock ratios. The stock ratios are divided into 11 levels $0,0.1,...,1$ (i.e.,
$L_{n}=11$). The total number of coefficients is therefore $p=9\times 11=99$.
All covariates are standardized to have a mean of 0 and standard deviation of
0.1. For each individual $i$ in the estimation sample, we randomly assign a
stock ratio $A_{i}$ (0, 0.1, …, 1) and perform the model-based random
experiments. We set the utility reward variable $Y_{i}$ to be a Gaussianized
score based on the rank of the average utility achieved by $A_{i}$ within all
stock ratio levels; see Appendix H.3. We estimate the value function (2.1)
using these 2000 observations in the estimation sample. The details of the
personal characteristics included in the model and the estimated model
coefficients are in Appendix I of the supplementary materials.

We then check the goodness of fit of the model on the evaluation sample of
size 1000. The evaluation sample is different from the estimation sample but
within the subset covered by the CAMS that has consumption information, and
hence the utility scores are obtainable. The out-of-sample $R^{2}$ is 31.4%
(in-sample $R^{2}$: 45.0%), suggesting a good fit.

####  5.3.2 Strategy performance

We evaluate the performance of our estimated individualized optimal asset
allocation on the testing sample ($N\geq$15,000). The testing sample contains
the individuals in HRS who are not included in the estimation sample. Because
most observations of the testing sample are not covered by CAMS, consumption
information is scarce. Nevertheless, because our final Q-function does not
include the consumption variable, using the estimated Q-function, we can
evaluate the performance of various strategies on the testing sample.

We compare the performance of our individualized optimal asset allocation
strategy444The proposed strategy is based on the DROVE estimator we develope.
We also evaluated the strategy based on standard SCAD estimator (std-scad)
used in Shi et al., (2016) by direct discretization to the continuous-action
setting. Compared to the std-scad strategy, our DROVE approach achieves a
higher average value (0.619 vs. 0.597), and the difference between the two is
statistically significant.  $\widehat{\pi}^{*}_{n}$ with the following
benchmark strategies: (a) we assign the same fixed (0, 0.1, …, 1) stock ratio
to all individuals, a strategy denoted by $\pi_{j}=j,j=0,0.1,0.2,...,1$; (b)
we assign the originally observed stock ratio, which is denoted by
$\pi_{obs}$.

For each strategy, we compute the estimated policy value
$\mathbb{P}_{N}\widehat{Q}_{n,\pi}=\sum_{i=1}^{N}\widehat{Q}_{n}(\mathcal{X}_{i},\pi(\mathcal{X}_{i}))/N$.
We construct CIs for $E(Q^{*}_{n})-E(Q_{n}(\mathbf{\mathcal{X}},\pi))$
according to Proposition 3.2. The results are in Table 4.

The results in Table 4 suggest that our individualized optimal asset
allocation decisions substantially improve over the benchmark strategies in
terms of policy values. Compared to the observed stock ratios, the
individualized strategy improves the average utility reward from 0.216 to
0.619. It also yields an average utility reward that is $0.316$–$1.708$
greater, relative to the fixed strategies. The 95% CIs of the value
differences are away from zero, suggesting that the individualized optimal
strategy achieves a significantly higher value compared with the values of the
observed and fixed strategies. This result demonstrates the importance of
individualization in asset allocation.

Table 4: Performance of various strategies.

Strategy | $\mathbb{P}_{N}\widehat{Q}_{n,\pi}$ | $\mathbb{P}_{N}\widehat{Q}^{*}_{n}-\mathbb{P}_{N}\widehat{Q}_{n,\pi}$ |  95% CI of ($EQ^{*}_{n}-EQ_{n,\pi}$)
---|---|---|---
$\pi_{obs}$ | 0.216 | 0.403 | 0.328 | 0.479
$\pi_{0}$ | 0.303 | 0.316 | 0.227 | 0.406
$\pi_{0.1}$ | 0.238 | 0.381 | 0.268 | 0.493
$\pi_{0.2}$ | 0.244 | 0.375 | 0.273 | 0.477
$\pi_{0.3}$ | 0.272 | 0.347 | 0.255 | 0.439
$\pi_{0.4}$ | 0.267 | 0.352 | 0.250 | 0.454
$\pi_{0.5}$ | -0.382 | 1.001 | 0.857 | 1.145
$\pi_{0.6}$ | -0.334 | 0.953 | 0.816 | 1.091
$\pi_{0.7}$ | -0.362 | 0.981 | 0.841 | 1.121
$\pi_{0.8}$ | -0.374 | 0.993 | 0.850 | 1.137
$\pi_{0.9}$ | -1.089 | 1.708 | 1.546 | 1.870
$\pi_{1.0}$ | -1.079 | 1.698 | 1.541 | 1.855
$\hat{\pi}^{*}_{n}$ | 0.619 | – | – | –

NOTE: This table gives the results of the value comparison among the estimated
optimal strategy, ($\widehat{\pi}^{*}_{n}$), observed strategy, ($\pi_{obs}$),
and fixed strategies, ($\pi_{j}=j$, $j=0,0.1,...,1$). The values are the
estimated value (i.e., the average estimated utility reward) for various
strategies, $\mathbb{P}_{N}\widehat{Q}_{n,\pi}$, the estimated value
improvement when using the optimal strategy over the benchmark strategies,
$\mathbb{P}_{N}\widehat{Q}^{*}_{n}-\mathbb{P}_{N}\widehat{Q}_{n,\pi}$, and the
95% CIs of the value difference $EQ^{*}_{n}-EQ_{n,\pi}$. The model estimation
uses a random estimation sample of 2000. The testing sample comprises the
remaining 15,000+ observations in the HRS from distinct individuals.

A low proportion of households participate in the equity market; few of them
hold stocks (e.g., Hong et al., 2004; Campbell, 2006). We also find a low
participation in equities. Notably, when adopting the optimal strategy, stock
market participation increases from 38% to 63%; see Appendix J of the
supplementary materials for more details. These results show that our optimal
individualized strategy improves the financial well-being of the population
and it entails a higher stock market participation on average, which indicates
a healthier and more active stock market.

##  6 Conclusion and Discussion

We develop a high-dimensional statistical learning methodology for continuous-
action decision-making with an important application in individualized asset
allocation. We show that our DROVE approach enjoys consistency in the model
coefficients estimation. Moreover, our approach achieves valid statistical
inference on the optimal value. Empirically, we apply the proposed methodology
to study the individualized asset allocation problem using HRS and CAMS data.
Under a consumption-based utility framework, our individualized optimal asset
allocation strategy substantially improves the financial well-being of the
population. The outperformance of our individualized optimal strategy over the
fixed stock ratio strategies highlights the benefit of individualization.

The statistical learning framework developed in this paper has broad
implications. Methodologically, our approach can be extended to the study of
stochastic policies, which can have advantages for problems with partially
observed states (e.g., Singh et al., 1994) and be of interest for applications
such as mobile health under infinite horizon settings (e.g., Luckett et al.,
2019; Liao et al., 2021). Another important direction is to extend our
framework from the single period to multiple-period decision-making. For
multiple-stage problems, one potential approach would be to model the
Q-function for each stage using the method developed in this paper and then
perform a backward recursive procedure to obtain the optimal dynamic
decisions. About estimation and inference of the optimal value, challenges
arise in the multi-stage problem given that the estimation errors in the
optimal rule and the associated optimal value in the latter stage carry over
to the former stages. In terms of application, for example, not limited to the
setting exercised in this paper, the framework can be readily applied to
achieving other wealth management objectives, such as post-retirement saving
adequacy. The framework can also be extended to the study of multi-class asset
allocation, and to incorporate dynamic prediction models of mean and
volatility of financial returns.

## Supplementary Materials

The online supplementary materials contain the additional assumptions, proofs
of the main results in the article, and additional discussions on the
theoretical and numerical results.

## Acknowledgement

We thank the editor, the associate editor, and two anonymous referees for very
constructive comments.

## Funding

Research is partially supported by startup fund of Hong Kong Polytechnic
University, RGC GRF 15302321, RGC GRF16502118 and T31-604/18N of the HKSAR,
and NSF-DMS-1555244.

## Disclosure statement

The authors report there are no competing interests to declare.

## References

  * Arnold and Tibshirani, (2016) Arnold, T. B. and Tibshirani, R. J. (2016).  Efficient implementations of the generalized lasso dual path algorithm.  Journal of Computational and Graphical Statistics, 25(1):1–27.
  * Athey and Wager, (2019) Athey, S. and Wager, S. (2019).  Efficient policy learning.  arXiv preprint arXiv:1702.02896.
  * Banks et al., (1998) Banks, J., Blundell, R., and Tanner, S. (1998).  Is there a retirement-savings puzzle?  American Economic Review, pages 769–788.
  * Cai et al., (2020) Cai, H., Shi, C., Song, R., and Lu, W. (2020).  Deep jump q-evaluation for offline policy evaluation in continuous action space.  arXiv preprint arXiv:2010.15963.
  * Campbell, (2006) Campbell, J. Y. (2006).  Household finance.  The Journal of Finance, 61(4):1553–1604.
  * Chen et al., (2016) Chen, G., Zeng, D., and Kosorok, M. R. (2016).  Personalized dose finding using outcome weighted learning.  Journal of the American Statistical Association, 111(516):1509–1521.
  * Chernozhukov et al., (2017) Chernozhukov, V., Chetverikov, D., Demirer, M., Duflo, E., Hansen, C., and Newey, W. (2017).  Double/debiased/neyman machine learning of treatment effects.  American Economic Review, 107(5):261–65.
  * De Nardi and Yang, (2014) De Nardi, M. and Yang, F. (2014).  Bequests and heterogeneity in retirement wealth.  European Economic Review, 72:182–196.
  * Engen et al., (1999) Engen, E. M., Gale, W. G., Uccello, C. E., Carroll, C. D., and Laibson, D. I. (1999).  The adequacy of household saving.  Brookings Papers on Economic Activity, 1999(2):65–187.
  * Fan and Li, (2001) Fan, J. and Li, R. (2001).  Variable selection via nonconcave penalized likelihood and its oracle properties.  Journal of the American Statistical Association, 96(456):1348–1360.
  * Fan and Lv, (2011) Fan, J. and Lv, J. (2011).  Nonconcave penalized likelihood with np-dimensionality.  IEEE Transactions on Information Theory, 57(8):5467–5484.
  * Fan et al., (2014) Fan, J., Xue, L., and Zou, H. (2014).  Strong oracle optimality of folded concave penalized estimation.  The Annals of Statistics, 42(3):819.
  * Gourinchas and Parker, (2002) Gourinchas, P.-O. and Parker, J. A. (2002).  Consumption over the life cycle.  Econometrica, 70(1):47–89.
  * Haider and Stephens Jr, (2007) Haider, S. J. and Stephens Jr, M. (2007).  Is there a retirement-consumption puzzle? evidence using subjective retirement expectations.  The Review of Economics and Statistics, 89(2):247–264.
  * Hall, (1978) Hall, R. E. (1978).  Stochastic implications of the life cycle-permanent income hypothesis: theory and evidence.  Journal of Political Economy, 86(6):971–987.
  * Hong et al., (2004) Hong, H., Kubik, J. D., and Stein, J. C. (2004).  Social interaction and stock-market participation.  The Journal of Finance, 59(1):137–163.
  * Hurd and Rohwedder, (2003) Hurd, M. and Rohwedder, S. (2003).  The retirement-consumption puzzle: Anticipated and actual declines in spending at retirement.  Technical report, National Bureau of Economic Research.
  * Kennedy et al., (2017) Kennedy, E. H., Ma, Z., McHugh, M. D., and Small, D. S. (2017).  Nonparametric methods for doubly robust estimation of continuous treatment effects.  Journal of the Royal Statistical Society. Series B, Statistical Methodology, 79(4):1229.
  * Laber and Zhao, (2015) Laber, E. and Zhao, Y. (2015).  Tree-based methods for individualized treatment regimes.  Biometrika, 102(3):501–514.
  * Liao et al., (2021) Liao, P., Klasnja, P., and Murphy, S. (2021).  Off-policy estimation of long-term average outcomes with applications to mobile health.  Journal of the American Statistical Association, 116(533):382–391.
  * Luckett et al., (2019) Luckett, D. J., Laber, E. B., Kahkoska, A. R., Maahs, D. M., Mayer-Davis, E., and Kosorok, M. R. (2019).  Estimating dynamic treatment regimes in mobile health using v-learning.  Journal of the American Statistical Association.
  * Lv and Fan, (2009) Lv, J. and Fan, Y. (2009).  A unified approach to model selection and sparse recovery using regularized least squares.  The Annals of Statistics, pages 3498–3528.
  * Meinshausen and Bühlmann, (2006) Meinshausen, N. and Bühlmann, P. (2006).  High-dimensional graphs and variable selection with the lasso.  The Annals of Statistics, pages 1436–1462.
  * Munnell et al., (2012) Munnell, A. H., Webb, A., Golub-Sass, F., et al. (2012).  The national retirement risk index: An update.  Center for Retirement Research at Boston College, 1:719–744.
  * Murphy, (2003) Murphy, S. A. (2003).  Optimal dynamic treatment regimes.  Journal of the Royal Statistical Society: Series B (Statistical Methodology), 65(2):331–355.
  * Murphy, (2005) Murphy, S. A. (2005).  An experimental design for the development of adaptive treatment strategies.  Statistics in Medicine, 24(10):1455–1481.
  * Palumbo, (1999) Palumbo, M. G. (1999).  Uncertain medical expenses and precautionary saving near the end of the life cycle.  The Review of Economic Studies, 66(2):395–421.
  * Qian and Murphy, (2011) Qian, M. and Murphy, S. A. (2011).  Performance guarantees for individualized treatment rules.  The Annals of Statistics, 39(2):1180.
  * Robins, (2004) Robins, J. M. (2004).  Optimal structural nested models for optimal sequential decisions.  In Proceedings of the Second Seattle Symposium in Biostatistics, pages 189–326. Springer.
  * Rosen and Wu, (2004) Rosen, H. S. and Wu, S. (2004).  Portfolio choice and health status.  Journal of Financial Economics, 72(3):457–484.
  * Rubin, (1974) Rubin, D. B. (1974).  Estimating causal effects of treatments in randomized and nonrandomized studies.  Journal of Educational Psychology, 66(5):688.
  * She, (2010) She, Y. (2010).  Sparse regression with exact clustering.  Electronic Journal of Statistics, 4:1055–1096.
  * Shi et al., (2018) Shi, C., Fan, A., Song, R., and Lu, W. (2018).  High-dimensional a-learning for optimal dynamic treatment regimes.  The Annals of Statistics, 46(3):925.
  * Shi et al., (2016) Shi, C., Song, R., and Lu, W. (2016).  Robust learning for optimal treatment decision with np-dimensionality.  Electronic Journal of Statistics, 10:2894.
  * Singh et al., (1994) Singh, S. P., Jaakkola, T., and Jordan, M. I. (1994).  Learning without state-estimation in partially observable markovian decision processes.  In Machine Learning Proceedings 1994, pages 284–292. Elsevier.
  * Tibshirani, (1996) Tibshirani, R. (1996).  Regression shrinkage and selection via the lasso.  Journal of the Royal Statistical Society. Series B (Methodological), pages 267–288.
  * Tibshirani et al., (2005) Tibshirani, R., Saunders, M., Rosset, S., Zhu, J., and Knight, K. (2005).  Sparsity and smoothness via the fused lasso.  Journal of the Royal Statistical Society: Series B (Statistical Methodology), 67(1):91–108.
  * Tibshirani and Taylor, (2011) Tibshirani, R. J. and Taylor, J. (2011).  The solution path of the generalized lasso.  The Annals of Statistics, 39(3):1335–1371.
  * Tibshirani and Taylor, (2012) Tibshirani, R. J. and Taylor, J. (2012).  Degrees of freedom in lasso problems.  The Annals of Statistics, 40(2):1198–1232.
  * Wager and Athey, (2018) Wager, S. and Athey, S. (2018).  Estimation and inference of heterogeneous treatment effects using random forests.  Journal of the American Statistical Association, 113(523):1228–1242.
  * Wang et al., (2013) Wang, L., Kim, Y., and Li, R. (2013).  Calibrating non-convex penalized regression in ultra-high dimension.  The Annals of Statistics, 41(5):2505.
  * Watkins, (1989) Watkins, C. (1989).  Learning from delayed rewards.  PhD thesis, King’s College, Cambridge.
  * Zhang et al., (2010) Zhang, C.-H. et al. (2010).  Nearly unbiased variable selection under minimax concave penalty.  The Annals of Statistics, 38(2):894–942.
  * Zhang and Huang, (2008) Zhang, C.-H. and Huang, J. (2008).  The sparsity and bias of the lasso selection in high-dimensional linear regression.  The Annals of Statistics, pages 1567–1594.
  * Zhao and Yu, (2006) Zhao, P. and Yu, B. (2006).  On model selection consistency of lasso.  Journal of Machine Learning Research, 7(Nov):2541–2563.
  * Zhao et al., (2012) Zhao, Y., Zeng, D., Rush, A. J., and Kosorok, M. R. (2012).  Estimating individualized treatment rules using outcome weighted learning.  Journal of the American Statistical Association, 107(499):1106–1118.
  * Zhou et al., (2021) Zhou, W., Zhu, R., and Zeng, D. (2021).  A parsimonious personalized dose-finding model via dimension reduction.  Biometrika.
  * Zhu et al., (2020) Zhu, L., Lu, W., Kosorok, M. R., and Song, R. (2020).  Kernel assisted learning for personalized dose finding.  In Proceedings of the 26th ACM SIGKDD International Conference on Knowledge Discovery & Data Mining, pages 56–65.
  * Zhu et al., (2019) Zhu, W., Zeng, D., and Song, R. (2019).  Proper inference for value function in high-dimensional q-learning for dynamic treatment regimes.  Journal of the American Statistical Association, 114(527):1404–1417.
  * Zou and Li, (2008) Zou, H. and Li, R. (2008).  One-step sparse estimates in nonconcave penalized likelihood models.  The Annals of Statistics, 36(4):1509.