# Can Strategic Data Collection Improve
the Performance of Poverty Prediction Models?

Satej Soman
School of Information & Global Policy Lab
University of California, Berkeley
Berkeley, CA 94704
<EMAIL_ADDRESS>
Emily Aiken
School of Information & Global Policy Lab
University of California, Berkeley
Berkeley, CA 94704
<EMAIL_ADDRESS>
Esther Rolf
Harvard Data Science Initiative &
Center for Research on Computation and Society
Harvard University
Cambridge, MA
<EMAIL_ADDRESS>
Joshua Blumenstock
School of Information, Goldman School of Public Policy
& Global Policy Lab
University of California, Berkeley
Berkeley, CA 94704
<EMAIL_ADDRESS>

###### Abstract

Machine learning-based estimates of poverty and wealth are increasingly being
used to guide the targeting of humanitarian aid and the allocation of social
assistance. However, the ground truth labels used to train these models are
typically borrowed from existing surveys that were designed to produce
national statistics – not to train machine learning models. Here, we test
whether adaptive sampling strategies for ground truth data collection can
improve the performance of poverty prediction models. Through simulations, we
compare the status quo sampling strategies (uniform at random and stratified
random sampling) to alternatives that prioritize acquiring training data based
on model uncertainty or model performance on sub-populations. Perhaps
surprisingly, we find that none of these active learning methods improve over
uniform-at-random sampling. We discuss how these results can help shape future
efforts to refine machine learning-based estimates of poverty.

##  1 Introduction

Over the past decade, several influential studies have demonstrated how
machine learning (ML) – when applied to digital data sources – can provide
accurate, timely, and granular estimates of wealth and poverty [1, 2, 3, 4, 5,
6, 7]. These ML-based estimates, which can be produced at a fraction of the
cost of traditional survey-based methods of data collection, can fill in data
gaps in poor, fragile, and conflict-affected regions where recent surveys do
not exist [8]. And increasingly, the ML-based estimates are being used to
inform critical policy decisions, including the targeting of humanitarian aid
[9] and the allocation of social assistance [10, 11].

Despite the important role ML-based poverty estimates now play in development
policy and disaster response, very little consideration has been given to the
sampling strategy used to collect the ground truth labels used to train the ML
models. Instead, the vast majority of poverty prediction models are trained on
publicly-available household surveys that were collected for a different
purpose (typically, to provide nationally representative demographic and
health statistics). These standardized surveys typically use (stratified)
random sampling to reduce measurement error [12]. However, sampling training
data according to these strategies may not optimize predictive power for a
machine learning algorithm subject to a budget constraint for data collection.
In an adaptive sampling strategy, each incremental round of data acquisition
is informed by an aspect of the performance of the model trained on data seen
in previous rounds of data collection. In this paper, we investigate whether
adaptive sampling strategies for label data collection improve the performance
of machine learning models trained to predict poverty from digital data
sources, focusing on model accuracy and fairness.

##  2 Application Context

This work builds on an emergency cash-transfer program run by the government
of Togo that used poverty estimates from digital data sources to prioritize
the distribution of emergency mobile money payments in response to the
COVID-19 pandemic (described in more detail in Appendix A and Aiken et al.
[9]). In designing the program, the government sought to distribute cash to
60,000 of the country’s poorest individuals. At the time, however, the
government did not have any traditional poverty data with which to determine
eligibility. Instead, the Togolese government turned to satellite imagery
analysis to identify the country’s poorest districts, and then used machine
learning models using features derived from subscribers’ mobile phone records
to identify the poorest subscribers in eligible districts for humanitarian
aid. The ground truth labels for the wealth prediction models came from
surveys conducted in Togo prior to the program’s launch. An evaluation of the
program’s targeting found that relative to alternative poverty targeting
methods (such as geographic or occupation-based targeting), the phone-based
approach was most effective at targeting cash transfers to the poorest people
in the country [9].

In this paper, we study whether using an adaptive strategy for sampling the
poverty labels obtained from the phone survey in Togo could have improved the
accuracy of the trained machine learning model. Our results speak most
directly to recent efforts to estimate wealth and poverty from mobile phone
data. More broadly, these results can inform the larger community of
researchers and policymakers interested in estimating social and demographic
characteristics from digital data in contexts where traditional data are
unavailable or out-of-date.

##  3 Methods

We conduct experiments to simulate various data acquisition strategies using
an in-person household survey conducted in Togo in 2018 ($N$ = 4,595) with a
stratified random sampling strategy 111While the survey was stratified by
region and urban/rural designation, our experiment treats this survey as if it
were sampled uniform-at-random.. The survey collected information on daily
per-capita consumption for each household interviewed (measured in USD/day),
which we use as our ground truth measure of poverty. We match the household
survey to anonymized features derived from call data records provided by the
nation’s two cellular network operators. After partitioning the survey-
labelled features into a label pool ($N_{\small{\textrm{LP}}}$ = 0.75$N$ =
3,446) and a holdout validation set ($N_{\small{\textrm{V}}}$ = 0.25$N$ =
1,149), we train random forest regressor models [13] on subsets of the label
pool of incrementally increasing size to simulate how the resulting model
performs throughout the data acquisition process. The model’s performance is
evaluated against the holdout set across all dataset sizes.

The baseline of acquiring data by uniform random sampling is compared against
two adaptive data acquisition approaches, in which each data point still
remaining in the label pool is assigned a heuristic sampling weight. In the
query-by-committee strategy, this weight is proportional to the variance of
the outputs of the individual decision trees within the random forest
regressor; unlabeled points for which the ensemble’s constituent models
disagree the most are upweighted in the next round of data acquisition. An
alternative is margin-based uncertainty sampling, in which an independent
logistic regression classifier is trained on the same features to predict
whether a given individual falls below a relevant poverty threshold; points
whose probability of classification is closer to 50% are upweighted in order
to gain more information on points whose classification is close to random.
Query-by-committee and uncertainty sampling are two standard methods for
active learning for model accuracy [14]. They have both been shown to
substantially improve model performance subject to a budget constraint for
data collection in small and relatively low-dimensional datasets unrelated to
the poverty prediction task we consider in this paper [14, 15, 16, 17].

Inspired by literature on active learning for fairness outcomes, [18, 19], we
also consider adaptively constructing the training dataset by increasing the
sampling weights for the points from the demographic categories with the
lowest accuracy (accuracy-weighted), highest mean-squared-error (MSE-
weighted), or highest exclusion/inclusion error rates (disparity-weighted). In
these group-based strategies, each member of the demographic category is
assigned the same weight, but each group’s weight differs by group-specific
model performance in each round of simulated data collection.

Focusing on these demographic category weights also allows us to assess
aspects of fairness of the model during the data collection process, as
measured by minimum group accuracy, maximum group loss, or total inclusion-
exclusion errors. These goals are independent of increasing the model’s
predictive power: even if deviation from uniform-at-random sampling provides
no benefits in increasing accuracy or reducing MSE, there may be benefits in
model fairness. In this investigation, the demographic category is geographic
residence in each of the major admin-1 regions of Togo (depicted in more
detail in Figure 1(a)), inspired by previous analysis indicating there may
exist slight geographic bias to the phone-based models [9].

We run 50 simulations for each strategy, with incremental dataset sizes
$S_{t}$ logarithmically spaced between 0 and the full label pool size to
better illustrate the effects of different strategies at smaller sample sizes.
Within each simulation, the newly sampled points are appended to the
previously-seen dataset ($S_{t}-S_{t-1}$ new data points are seen at time $t$
rather than $S_{t}$ points sampled memorylessly). Our code for the experiments
is available on GitHub.

##  4 Results

We present the results of each of our sampling strategies, comparing
individual level active learning methods (query-by-committee and uncertainty
sampling) and group-level active learning methods (accuracy, MSE, and
disparity weighting) to a baseline of uniform-at-random sampling. In Figure 1,
we report the average performance of each sampling method across 50
simulations, with shaded regions representing boostrapped 95% confidence
intervals. Evaluating across a number of standard machine learning accuracy
metrics (Spearman $\rho$, mean-square-error, AUROC, accuracy, precision, and
recall).222Poverty predictions are continuous and produced with regression
models; we binarize predictions using the international poverty line of
US$1.90/day to obtain binary classification metrics., it is clear that neither
model-based nor group-weighting strategies provide substantial gains to
overall model performance above uniform at random sampling. With some
strategies, such as the query by committee approach, we do see some gains to
model precision, but these are on the order of 1% and are offset by reduction
in recall (which is a more important metric for aid prioritization, since a
model with lower recall will mistakenly exclude more recipients who should
receive aid).

Figure 1: Comparison of model-based data acquisition strategies on key machine
learning performance metrics as dataset size incrementally grows.

Our group-based sampling strategies rely on upweighting groups (regions of
Togo) for which our model performs worse. We evaluate these strategies based
on the same metrics as above, and three additional fairness-related metrics
calculated for the major geographic regions in Togo. In Figure 2, we plot
minimum group accuracy, maximum group loss, and a summary statistic for
targeting inclusion/exclusion errors (absolute demographic deviation; ADD,
explained in more detail in Appendix B). Mean values are plotted, with shaded
regions representing bootstrapped 95% confidence intervals. Again, we observe
negligible differences in fairness between our active learning approaches and
the uniform-at-random baseline sampling strategy.

Additional experiments in Appendix C detail similar results when using a
feature set of reduced dimensionality and a hyperparameter tuning procedure
across dataset sizes.

Figure 2: Comparison of group weighting strategies on standard machine
learning metrics, as well as fairness-oriented metrics such as minimum group-
level accuracy, maximum group-level mean-square-error, and total
inclusion/exclusion errors (ADD).

##  5 Discussion

Our results indicate that a well-implemented, uniformly-sampled survey may be
close to optimal as a source of training data for machine learning models that
aim to predict poverty levels from digital trace data. In particular, neither
the model-based data acquisition strategies nor the group characteristic
weighting approaches we considered showed appreciable gains on accuracy or
fairness metrics over simple uniform random sampling.

This result has important implications for the growing research literature –
and increasing number of policy applications – that rely on estimates of
poverty predicted from digital trace data. Nearly all such estimates have been
generated from “found” data, i.e., survey data that were collected for another
purpose, and which were subsequently re-used as labels for training a poverty
prediction algorithm [e.g., 1, 2, 3, 4, 5, 6, 7]. Our hypothesis in initiating
this analysis was that such re-use was sub-optimal; if the poverty prediction
use case was known ex ante, we expected there to be performance gains from a
more strategic approach to obtaining training data. However, our results
indicate that — at least for the adaptive approaches that we tested – no such
gains exist.

Whether this is a general finding, or one that is specific to our particular
data context (i.e., mobile phone data from Togo), is an area of future work
that we are continuing to explore. We also see several new directions in which
to extend this analysis. For instance, all of the simulations described here
assume an identical labeling cost for all data points. This uniform-cost model
is appropriate in settings where the cost of interviewing an individual is the
same regardless of their physical location in the country – as is the case
with phone-based surveys. By contrast, many settings involve highly
heterogeneous sampling costs – such as household surveys where some households
are easily accessed and others are remote. In such settings, factoring in the
costs of traveling to survey locations and conducting surveys may provide more
realistic strategies for collecting training labels in household surveys. In
the current investigation, in which groups are defined geographically, group-
specific sampling is especially cost-infeasible because where the group to be
sampled may change in every round of incremental data acquisition. In
continued work, we plan to investigate cost-sensitive data acquisition
strategies, and generate theoretical explanations for the lack of gains in any
of the non-uniform sampling strategies.

## Acknowledgments and Disclosure of Funding

This material is based upon work supported by the National Science Foundation
under Grant No. DGE- 2125913.

## References

  * Blumenstock et al. [2015] Joshua E Blumenstock, Gabriel Cadamuro, and Robert On.  Predicting poverty and wealth from mobile phone metadata.  _Science_ , 350(6264):1073–1076, 2015.
  * Jean et al. [2016] Neal Jean, Marshall Burke, Michael Xie, W Matthew Davis, David B Lobell, and Stefano Ermon.  Combining satellite imagery and machine learning to predict poverty.  _Science_ , 353(6301):790–794, 2016.
  * Steele et al. [2017] Jessica E. Steele, Pål Roe Sundsøy, Carla Pezzulo, Victor A. Alegana, Tomas J. Bird, Joshua Evan Blumenstock, Johannes Bjelland, Kenth Engø-Monsen, Yves-Alexandre de Montjoye, Asif M. Iqbal, Khandakar N. Hadiuzzaman, Xin Lu, Erik Wetter, Andrew J. Tatem, and Linus Bengtsson.  Mapping poverty using mobile phone and satellite data.  _Journal of The Royal Society Interface_ , 14(127):20160690, February 2017.  ISSN 1742-5689, 1742-5662.  doi: 10.1098/rsif.2016.0690.  URL http://rsif.royalsocietypublishing.org/content/14/127/20160690.
  * Blumenstock [2018] Joshua E Blumenstock.  Estimating economic characteristics with phone data.  In _AEA papers and proceedings_ , volume 108, pages 72–76, 2018.
  * Burke et al. [2020] Marshall Burke, Anne Driscoll, David Lobell, and Stefano Ermon.  Using Satellite Imagery to Understand and Promote Sustainable Development.  Technical Report w27879, National Bureau of Economic Research, October 2020.  URL https://www.nber.org/papers/w27879.
  * Yeh et al. [2020] Christopher Yeh, Anthony Perez, Anne Driscoll, George Azzari, Zhongyi Tang, David Lobell, Stefano Ermon, and Marshall Burke.  Using publicly available satellite imagery and deep learning to understand economic well-being in Africa.  _Nature communications_ , 11(1):1–11, 2020.
  * Chi et al. [2022] Guanghua Chi, Han Fang, Sourav Chatterjee, and Joshua E. Blumenstock.  Microestimates of wealth for all low- and middle-income countries.  _Proceedings of the National Academy of Sciences_ , 119(3), January 2022.  ISSN 0027-8424, 1091-6490.  doi: 10.1073/pnas.2113658119.  URL https://www.pnas.org/content/119/3/e2113658119.  ISBN: 9782113658118 Publisher: National Academy of Sciences Section: Social Sciences.
  * Jerven [2013] Morten Jerven.  Poor numbers.  In _Poor Numbers_. Cornell University Press, 2013.
  * Aiken et al. [2022] Emily Aiken, Suzanne Bellue, Dean Karlan, Chris Udry, and Joshua E Blumenstock.  Machine learning and phone data can improve targeting of humanitarian aid.  _Nature_ , 603(7903):864–870, 2022.
  * Smythe and Blumenstock [2022] Isabella S. Smythe and Joshua E. Blumenstock.  Geographic microtargeting of social assistance with high-resolution poverty maps.  _Proceedings of the National Academy of Sciences_ , 119(32):e2120025119, August 2022.  doi: 10.1073/pnas.2120025119.  URL https://www.pnas.org/doi/10.1073/pnas.2120025119.  Publisher: Proceedings of the National Academy of Sciences.
  * Gentilini et al. [2021] Ugo Gentilini, Saksham Khosla, and Mohamed Almenfi.  Cash in the city.  Technical report, World Bank, 2021.
  * Howes and Lanjouw [1998] Stephen Howes and Jean Olson Lanjouw.  Does sample design matter for poverty rate comparisons?  _Review of Income and Wealth_ , 44(1):99–109, 1998.
  * Pedregosa et al. [2011] F. Pedregosa, G. Varoquaux, A. Gramfort, V. Michel, B. Thirion, O. Grisel, M. Blondel, P. Prettenhofer, R. Weiss, V. Dubourg, J. Vanderplas, A. Passos, D. Cournapeau, M. Brucher, M. Perrot, and E. Duchesnay.  Scikit-learn: Machine learning in Python.  _Journal of Machine Learning Research_ , 12:2825–2830, 2011\.
  * Settles [2009] Burr Settles.  Active learning literature survey.  Technical report, University of Wisconsin-Madison, 2009.
  * Settles and Craven [2008] Burr Settles and Mark Craven.  An analysis of active learning strategies for sequence labeling tasks.  In _proceedings of the 2008 conference on empirical methods in natural language processing_ , pages 1070–1079, 2008.
  * McCallumzy and Nigamy [1998] Andrew Kachites McCallumzy and Kamal Nigamy.  Employing em and pool-based active learning for text classification.  In _Proc. International Conference on Machine Learning (ICML)_ , pages 359–367. Citeseer, 1998.
  * Körner and Wrobel [2006] Christine Körner and Stefan Wrobel.  Multi-class ensemble-based active learning.  In _European conference on machine learning_ , pages 687–694. Springer, 2006.
  * Abernethy et al. [2020] Jacob Abernethy, Pranjal Awasthi, Matthäus Kleindessner, Jamie Morgenstern, Chris Russell, and Jie Zhang.  Active sampling for min-max fairness.  _arXiv preprint arXiv:2006.06879_ , 2020.
  * Cai et al. [2022] William Cai, Ro Encarnacion, Bobbie Chern, Sam Corbett-Davies, Miranda Bogen, Stevie Bergman, and Sharad Goel.  Adaptive sampling strategies to construct equitable training datasets.  _arXiv preprint arXiv:2202.01327_ , 2022.
  * Tifrea et al. [2022] Alexandru Tifrea, Jacob Clarysse, and Fanny Yang.  Uniform versus uncertainty sampling: When being active is less efficient than staying passive.  In _Workshop on Adaptive Experimental Design and Active Learning in the Real World_ , 2022.

## Supplementary Materials

##  Appendix A Additional Background

Togo is a West African country with a population of 8 million, over half of
whom live below with international poverty line. In March 2020, the government
imposed a lockdown order within the country in response to the first confirmed
COVID-19 cases in the country. These lockdowns proved economically disruptive,
with risks of food insecurity rising as work became scarce for Togolese
citizens employed informally [9].

After an initial pilot of an emergency cash transfer program, named "Novissi",
implemented through mobile money payments to residents in the greater
metropolitan area surrounding the capital city of Lomé, the Togolese
government sought to expand the program to 100 of Togo’s poorest 397 cantons.
Registrants were required to use a SIM card to register for the expanded
program. Historical call records and cell phone usage metadata tied to those
SIM cards were used to predict consumption, as illustrated in Figure 1(b).

(a) Location of Togo (green) in Africa and major regions of the country
(inset).

(b) True versus predicted log daily consumption for the holdout validation
set, with a random forest trained on phone usage features.

Figure A1: Geographic context and poverty prediction model output for Togo.

Given the geographic distribution of socioeconomic characteristics and
potential differences in cell phone penetration and usage patterns across the
country, it is not surprising that models trained using these data show
differential performance across populations living in different regions (Fig.
A2) (or other demographic breakdowns). Aiken et al. [9] found that aid
targeting errors using this approach, despite the variance in performance,
were lower than other, non-machine-learning-based approaches. Nevertheless,
the difference in performances across regions was the starting point for our
investigations into improving fairness characteristics, described below.

(a) Accuracy by region under sampling strategies.

(b) MSE by region under sampling strategies.

Figure A2: Per-group accuracy and MSE for each region of Togo. Note the
differential performance of the model across regions, and the slight
differences in the crossover points at which the model’s performance for one
subgroup improves compared to other subgroups, such as the MSE in Maritime vs
Centrale.

##  Appendix B Fairness Metrics Explained

###  B.1 Accuracy

We follow the standard definition of accuracy, but calculate it for every
group $g$ in the demographic category under consideration, $G$:

$A_{g}=\frac{\textrm{TP}_{g}+\textrm{TN}_{g}}{\textrm{TP}_{g}+\textrm{TN}_{g}+\textrm{FP}_{g}+\textrm{FN}_{g}}~{}.$

where TPg, TNg, FPg, and FNg are the true positive, true negative, false
positive, and false negative rates, respectively, calculated for group $g$.

The summary statistic, worst-case group accuracy, is the minimum over all
groups: $\displaystyle\min_{g\in G}A_{g}$. Because we want to sample from the
groups with the lowest accuracy, the weighting for each unseen data point $i$
in the label pool belonging to group $g$ is proportional to the group accuracy
subtracted from one: $w_{i}\propto 1-A_{g}$.

###  B.2 Mean squared error

As with accuracy, we calculate mean squared error for each demographic group:

$\textrm{MSE}_{g}=\frac{1}{|g|}\sum_{i\in
g}\left(y_{i}-\hat{y}_{i}\right)^{2}~{}.$

The summary statistic across groups, worst-case group MSE, is the maximum, and
the sampling weight for each point $i$ is directly proportional to the group
MSE.

###  B.3 Demographic disparity and absolute demographic deviance

The final fairness metric we present is based on a measure of targeting
exclusion developed in Aiken et al. [9], namely group demographic parity
(DP)333This definition of demographic parity differs from other definitions in
the relevant literature.: defined as the proportion of each group predicted to
live below the US$ 1.90 threshold, minus the true proportion of the group
living below the same threshold:

$\textrm{DP}_{g}=\frac{\left(\textrm{TP}_{g}+\textrm{FP}_{g}\right)-\left(\textrm{TP}_{g}+\textrm{FN}_{g}\right)}{N_{g}}~{}.$

A positive value for DP indicates a given group is over-represented in the
model-based targeting, while a negative value means the group is
insufficiently targeted for aid receipt. For the disparity weighting strategy,
the sampling weight is, like accuracy, proportional to one minus the group
demographic parity value: $w_{i}\propto 1-\textrm{DP}_{g}$.

For a summary statistic, we define absolute demographic deviance (ADD) as the
sum of the absolute values of the demographic parity for each group, with the
intent of capturing both errors of inclusion as well as errors of exclusion in
a single number:

$\textrm{ADD}=\sum_{g\in G}\left|\textrm{DP}_{g}\right|~{}.$

##  Appendix C Dimensionality Reduction and Cross-Validation

###  C.1 Dimensionality reduction

One potential issue with active learning approaches is that the dimension of
the feature space makes it difficult to isolate the important dimensions along
which new point selection would be beneficial [20]. To test this hypothesis in
our context, we used a principal components analysis to reduce the dimension
of our feature space from approximately 850 features to 50. As seen in Figure
A3, no appreciable gains were seen to using strategies other than uniform-at-
random sampling in the PCA dataset (lighter colors), either.

Figure A3: Comparison of point-based strategies on the original data (darker
colors) versus a PCA-compressed representation of the data (lighter colors).

###  C.2 Cross-validation

To maintain a simpler experimental setup, we skipped the usual cross-
validation steps on models trained at each incremental dataset size $S_{t}$.
To ensure that the lack of cross-validation did not artificially constrain the
model performance, we present the results of cross-validating the model using
three-fold cross-validation on the training set to find optimal tree depth at
each incremental dataset size. As with dimensionality reduction, cross-
validation did not show appreciable gains to model characteristics.

Figure A4: Comparison of point-based strategies without cross-validation
(darker colors, circular markers) versus the same strategies, with three-fold
cross-validation run at each incremental dataset size (lighter colors, square
markers).


* Veras et al. (2016a) Veras, D., Mustill A. J., Gänsicke B. T., Redfield S., Georgakarakos N., Bowler A. B., Lloyd M. J. S. (2016a). Full-lifetime simulations of multiple unequal-mass planets across all phases of stellar evolution. Monthly Notices of the Royal Astronomical Society, 458, 3942–3967.
  * Veras et al. (2016b) Veras, D., Marsh, T. R., & Gänsicke, B. T. (2016b). Dynamical mass and multiplicity constraints on co-orbital bodies around stars. Monthly Notices of the Royal Astronomical Society, 461, 1413–1420.
  * Veras et al. (2017a) Veras, D., Carter, P. J., Leinhardt, Z. M., Gänsicke, B. T. (2017a). Explaining the variability of WD 1145+017 with simulations of asteroid tidal disruption. Monthly Notices of the Royal Astronomical Society, 465, 1008–1022.
  * Veras et al. (2017b) Veras, D., Georgakarakos, N., Dobbs-Dixon, I., Gänsicke, B. T. (2017b). Binary star influence on post-main-sequence multi-planet stability. Monthly Notices of the Royal Astronomical Society, 465, 2053–2059.
  * Veras et al. (2017c) Veras, D., Mustill, A. J., & Gänsicke, B. T. (2017c). The unstable fate of the planet orbiting the A star in the HD 131399 triple stellar system. Monthly Notices of the Royal Astronomical Society, 465, 1499–1504.
  * Veras et al. (2018a) Veras, D., Georgakarakos, N., Gänsicke, B. T., Dobbs-Dixon, I. (2018a). Effects of non-Kozai mutual inclinations on two-planet system stability through all phases of stellar evolution. Monthly Notices of the Royal Astronomical Society, 481, 2180–2188.
  * Veras, Xu & Rebassa-Mansergas (2018b) Veras, D., Xu, S., Rebassa-Mansergas, A. (2018b). The critical binary star separation for a planetary system origin of white dwarf pollution. Monthly Notices of the Royal Astronomical Society, 473, 2871–2880.
  * Veras et al. (2019a) Veras, D., Efroimsky, M., Makarov, V. V., Boué, G., Wolthoff, V., Reffert, S…. Gänsicke, B. T. (2019a). Orbital relaxation and excitation of planets tidally interacting with white dwarfs. Monthly Notices of the Royal Astronomical Society, 486, 3831–3848.
  * Veras et al. (2019b) Veras, D., Higuchi, A., & Ida, S. (2019b). Speeding past planets? Asteroids radiatively propelled by giant branch Yarkovsky effects. Monthly Notices of the Royal Astronomical Society, 485, 708–724.
  * Veras & Fuller (2019) Veras, D., Fuller, J. (2019). Tidal circularization of gaseous planets orbiting white dwarfs. Monthly Notices of the Royal Astronomical Society, 489, 2941–2953.
  * Veras & Wolszczan (2019) Veras, D., Wolszczan, A. (2019). Survivability of radio-loud planetary cores orbiting white dwarfs. Monthly Notices of the Royal Astronomical Society, 488, 153–163.
  * Veras (2020) Veras, D. (2020). The white dwarf planet WD J0914+1914 b: barricading potential rocky pollutants? Monthly Notices of the Royal Astronomical Society, 493, 4692–4699.
  * Veras & Fuller (2020) Veras, D., Fuller, J. (2020). The dynamical history of the evaporating or disrupted ice giant planet around white dwarf WD J0914+1914. Monthly Notices of the Royal Astronomical Society, 492, 6059–6066.
  * Veras & Heng (2020) Veras, D., Heng, K. (2020). The lifetimes of planetary debris discs around white dwarfs. Monthly Notices of the Royal Astronomical Society, 496, 2292–2308.
  * Veras & Kurosawa (2020) Veras, D., Kurosawa, K. (2020). Generating metal-polluting debris in white dwarf planetary systems from small-impact crater ejecta. Monthly Notices of the Royal Astronomical Society, 494, 442–457.
  * Veras & Scheeres (2020) Veras, D. & Scheeres, D. J. (2020). Post-main-sequence debris from rotation-induced YORP break-up of small bodies - II. Multiple fissions, internal strengths, and binary production. Monthly Notices of the Royal Astronomical Society, 492, 2437-2445.
  * Veras, McDonald & Makarov (2020a) Veras, D., McDonald, C. H., Makarov, V. V. (2020a). Constraining the origin of the planetary debris surrounding ZTF J0139+5245 through rotational fission of a triaxial asteroid. Monthly Notices of the Royal Astronomical Society, 492, 5291–5296.
  * Veras et al. (2020b) Veras, D., Reichert, K., Flammini Dotti, F., Cai, M. X., Mustill, A. J., Shannon, A…. Spurzem, R. (2020b). Linking the formation and fate of exo-Kuiper belts within Solar system analogues. Monthly Notices of the Royal Astronomical Society, 493, 5062–5078.
  * Veras (2021) Veras, D. (2021). Planetary Systems Around White Dwarfs. Oxford Research Encyclopedia of Planetary Science, id. 1. doi:10.1093/acrefore/9780190647926.013.238
  * Veras & Hinkley (2021) Veras, D. & Hinkley, S. (2021). The post-main-sequence fate of the HR 8799 planetary system. Monthly Notices of the Royal Astronomical Society, 505, 1557–1566.
  * Veras et al. (2021) Veras, D., Georgakarakos, N., Mustill, A. J., et al. (2021). The entry geometry and velocity of planetary debris into the Roche sphere of a white dwarf. Monthly Notices of the Royal Astronomical Society, 506, 1148–1164.
  * Veras & Vidotto (2021) Veras, D. & Vidotto, A. A. (2021). Planetary magnetosphere evolution around post-main-sequence stars. Monthly Notices of the Royal Astronomical Society, 506, 1697–1703.
  * Veras et al. (2022) Veras, D., Birader, Y., & Zaman, U. (2022). Orbit decay of 2-100 au planetary remnants around white dwarfs with no gravitational assistance from planets. Monthly Notices of the Royal Astronomical Society, 510, 3379–3388.
  * Veras & Rosengren (2023) Veras, D., & Rosengren, A.J. (2023). The smallest planetary drivers of white dwarf pollution. Monthly Notices of the Royal Astronomical Society, 519, 6257-6266.
  * Veras et al. (2023a) Veras, D., Georgakarakos, N., & Dobbs-Dixon, I. (2023a). High-resolution resonant portraits of a single-planet white dwarf system. Monthly Notices of the Royal Astronomical Society, 518, 4537-4550.
  * Veras et al. (2023b) Veras, D., Ida, S., Grishin, E., et al. (2023b). Planetesimals drifting through dusty and gaseous white dwarf debris discs: Types I, II and III-like migration. Monthly Notices of the Royal Astronomical Society, 524, 1-17.
  * Villaver & Livio (2007) Villaver, E. & Livio, M. (2007). Can Planets Survive Stellar Evolution? The Astrophysical Journal, 661, 1192-1201.
  * Villaver & Livio (2009) Villaver, E. & Livio, M. (2009). The Orbital Evolution of Gas Giant Planets Around Giant Stars. The Astrophysical Journal Letters, 705, L81-L85.
  * Villaver et al. (2014) Villaver, E., Livio, M., Mustill, A. J., et al. (2014). Hot Jupiters and Cool Stars. The Astrophysical Journal, 794, 3, 15pp.
  * Völschow, Banerjee & Hessman (2014) Völschow, M., Banerjee, R., Hessman, F. V. (2014). Second generation planet formation in NN Serpentis? Astronomy and Astrophysics 562, A19, 4pp.
  * Voyatzis et al. (2013) Voyatzis, G., Hadjidemetriou, J. D., Veras, D., Varvoglis, H. (2013). Multiplanet destabilization and escape in post-main-sequence systems. Monthly Notices of the Royal Astronomical Society, 430, 3383–3396.
  * Walters et al. (2021) Walters, N., Farihi, J., Marsh, T. R., et al. (2021). A test of the planet-star unipolar inductor for magnetic white dwarfs. Monthly Notices of the Royal Astronomical Society, 503, 3743-3758.
  * Wang et al. (2019) Wang, T-gui., Jiang, N., Ge, J., Cutri, R. M., Jiang, P., Sheng, Z…. Wright, E. L. (2019). An Ongoing Mid-infrared Outburst in the White Dwarf 0145+234: Catching in Action the Tidal Disruption of an Exoasteroid? The Astrophysical Journal, 886, L5, 8pp. DOI:10.3847/2041-8213/ab53ed
  * Wang et al. (2023) Wang, L., Zhang, X., Wang, J., et al. (2023). White Dwarfs with Infrared Excess from LAMOST Data Release 5. Astrophysical Journal, 944, 23, 11pp.
  * Wilson et al. (2014) Wilson, D. J., Gänsicke B. T., Koester D., Raddi R., Breedt E., Southworth J., Parsons S. G. (2014). Variable emission from a gaseous disc around a metal-polluted white dwarf. Monthly Notices of the Royal Astronomical Society, 445, 1878–1884.
  * Wilson et al. (2019b) Wilson, T. G., Farihi, J., Gänsicke, B. T., Swan, A. (2019b). The unbiased frequency of planetary signatures around single and binary white dwarfs using Spitzer and Hubble. Monthly Notices of the Royal Astronomical Society, 487, 133–146.
  * Wyatt et al. (2014) Wyatt, M. C., Farihi, J., Pringle, J. E., Bonsor, A. (2014). Stochastic accretion of planetesimals on to white dwarfs: constraints on the mass distribution of accreted material from atmospheric pollution. Monthly Notices of the Royal Astronomical Society, 439, 3371–3391.
  * Xu & Jura (2014) Xu, S., Jura, M. (2014). The Drop during Less than 300 Days of a Dusty White Dwarf’s Infrared Luminosity. The Astrophysical Journal, 792, L39, 6pp.
  * Xu et al. (2015) Xu, S., Ertel, S., Wahhaj, Z., et al. (2015). An extreme-AO search for giant planets around a white dwarf. VLT/SPHERE performance on a faint target GD 50. Astronomy and Astrophysics, 579, L8, 5pp.
  * Xu et al. (2016) Xu, S., Jura, M., Dufour, P., Zuckerman, B. (2016). Evidence for Gas from a Disintegrating Extrasolar Asteroid. The Astrophysical Journal, 816, L22, 6pp.
  * Xu et al. (2017) Xu, S., Zuckerman, B., Dufour, P., Young, E. D., Klein, B., Jura, M. (2017). The Chemical Composition of an Extrasolar Kuiper-Belt-Object. The Astrophysical Journal, 836, 1, L7.
  * Xu et al. (2018a) Xu, S., Su, Kate Y. L., Rogers, L. K., Bonsor, A., Olofsson, J., Veras, D…. Gänsicke, B. T. (2018a). Infrared Variability of Two Dusty White Dwarfs. The Astrophysical Journal, 866, 108, 12pp.
  * Xu et al. (2018b) Xu, S., Rappaport, S., van Lieshout, R., et al. (2018b). A dearth of small particles in the transiting material around the white dwarf WD 1145+017. Monthly Notices of the Royal Astronomical Society, 474, 4795–4809.
  * Xu et al. (2019a) Xu, S., Hallakoun, N., Gary, B., Dalba, P. A., Debes, J., Dufour, P…. Zuckerman, B. (2019a). Shallow Ultraviolet Transits of WD 1145+017. The Astronomical Journal, 157, 255, 12pp.
  * Xu, Lai & Dennihy (2020) Xu, S., Lai, S., Dennihy, E. (2020). Infrared Excesses around Bright White Dwarfs from Gaia and unWISE. I. The Astrophysical Journal, 902, 127, 10pp.
  * Xu & Bonsor (2021) Xu, S. & Bonsor, A. (2021). Exogeology from Polluted White Dwarfs. Elements, 17, 241. doi:10.48550/arXiv.2108.08384
  * Xu et al. (2021) Xu, S., Diamond-Lowe, H., MacDonald, R. J., et al. (2021). Gemini/GMOS Transmission Spectroscopy of the Grazing Planet Candidate WD 1856+534 b. The Astronomical Journal, 162, 296, 15pp.
  * Yarza et al. (2023) Yarza, R., Razo Lopez, N., Murguia-Berthier, A., et al. (2023). Hydrodynamics and survivability during post-main-sequence planetary engulfment. The Astrophysical Journal, 954, 176, 16pp.
  * Zeebe (2015) Zeebe, R. E. (2015). Dynamic Stability of the Solar System: Statistically Inconclusive Results from Ensemble Integrations. The Astrophysical Journal, 798, 8, 13pp.
  * Zhang et al. (2021) Zhang, Y., Liu, S.-F., & Lin, D. N. C. (2021). Orbital Migration and Circularization of Tidal Debris by Alfvén-wave Drag: Circumstellar Debris and Pollution around White Dwarfs. The Astrophysical Journal, 915, 91, 11pp.
  * Zhou et al. (2016) Zhou, G., Kedziora-Chudczer, L., Bailey, J., et al. (2016). Simultaneous infrared and optical observations of the transiting debris cloud around WD 1145+017. Monthly Notices of the Royal Astronomical Society, 463, 4422-4432.
  * Zink, Batygin & Adams (2020) Zink, J. K., Batygin, K., Adams, F. C. (2020). The Great Inequality and the Dynamical Disintegration of the Outer Solar System. The Astronomical Journal, 160, 232, 9pp.
  * Zotos & Veras (2020) Zotos, E. E. & Veras, D. (2020). The grain size survival threshold in one-planet post-main-sequence exoplanetary systems. Astronomy and Astrophysics, 637, A14, 17pp.
  * Zotos et al. (2020) Zotos, E. E., Veras, D., Saeed, T., Darriba, L. A. (2020). Short-term stability of particles in the WD J0914+1914 white dwarf planetary system. Monthly Notices of the Royal Astronomical Society, 497, 5171–5181.
  * Zuckerman & Becklin (1987) Zuckerman, B., Becklin, E. E. (1987). Excess infrared radiation from a white dwarf – an orbiting brown dwarf? Nature, 330, 138–140.
  * Zuckerman et al. (2003) Zuckerman, B., Koester, D., Reid, I. N., Hünsch, M. (2003). Metal Lines in DA White Dwarfs. The Astrophysical Journal, 596, 477–495.
  * Zuckerman et al. (2010) Zuckerman, B., Melis, C., Klein, B., Koester, D., Jura, M. (2010). Ancient Planetary Systems are Orbiting a Large Fraction of White Dwarf Stars. The Astrophysical Journal, 722, 725–736.
  * Zuckerman & Young (2018) Zuckerman, B. & Young, E. D. (2018). Characterizing the Chemistry of Planetary Materials Around White Dwarf Stars. Handbook of Exoplanets, ISBN 978-3-319-55332-0. Springer International Publishing AG, part of Springer Nature, 2018, id.14