# Poverty rate prediction using multi-modal survey and earth observation data

Simone Fobi  AI for Good Lab, MicrosoftRedmondUSA<EMAIL_ADDRESS>,
Manuel Cardona  Innovations for Poverty ActionMexico CityMexico
<EMAIL_ADDRESS>,  Elliott Collins  Innovations for Poverty
ActionWashington DCUSA<EMAIL_ADDRESS>,  Caleb Robinson  AI for
Good Lab, MicrosoftRedmondUSA<EMAIL_ADDRESS>,  Anthony Ortiz
AI for Good Lab, MicrosoftRedmondUSA<EMAIL_ADDRESS>,  Tina
Sederholm  AI for Good Lab, MicrosoftRedmondUSA<EMAIL_ADDRESS>,
Rahul Dodhia  AI for Good Lab, MicrosoftRedmondUSA<EMAIL_ADDRESS>and  Juan Lavista Ferres  AI for Good Lab, MicrosoftRedmondUSA
<EMAIL_ADDRESS>
(2023)

###### Abstract.

This work presents an approach for combining household demographic and living
standards survey questions with features derived from satellite imagery to
predict the poverty rate of a region. Our approach utilizes visual features
obtained from a single-step featurization method applied to freely available
10m/px Sentinel-2 surface reflectance satellite imagery. These visual features
are combined with ten survey questions in a proxy means test (PMT) to estimate
whether a household is below the poverty line. We show that the inclusion of
visual features reduces the mean error in poverty rate estimates from 4.09% to
3.88% over a nationally representative out-of-sample test set. In addition to
including satellite imagery features in proxy means tests, we propose an
approach for selecting a subset of survey questions that are complementary to
the visual features extracted from satellite imagery. Specifically, we design
a survey variable selection approach guided by the full survey and image
features and use the approach to determine the most relevant set of small
survey questions to include in a PMT. We validate the choice of small survey
questions in a downstream task of predicting the poverty rate using the small
set of questions. This approach results in the best performance – errors in
poverty rate decrease from 4.09% to 3.71%. We show that extracted visual
features encode geographic and urbanization differences between regions.

poverty rate, proxy-means test, satellite imagery, machine learning

††journalyear: 2023††copyright: acmlicensed††conference: ACM SIGCAS/SIGCHI
Conference on Computing and Sustainable Societies; August 16–19, 2023; Cape
Town, South Africa††booktitle: ACM SIGCAS/SIGCHI Conference on Computing and
Sustainable Societies (COMPASS ’23), August 16–19, 2023, Cape Town, South
Africa††price: 15.00††doi: 10.1145/3588001.3609359††isbn:
979-8-4007-0149-8/23/08††ccs: Computing methodologies Classification and
regression trees††ccs: Computing methodologies Image representations††ccs:
Computing methodologies Supervised learning††ccs: Computing methodologies
Feature selection

##  1\. Introduction

Monitoring poverty is a crucial component for achieving Sustainable
Development Goal 1: No Poverty by 2030. Global poverty rates had decreased to
a low of 8.6% in 2019, however estimates suggests that the COVID-19 pandemic
contributed to a 1% increase by the end of 2020, rewinding progress in poverty
reduction (United Nations, 2023). Thus, accurate measurements of poverty rates
are needed to track progress towards poverty elimination goals and to measure
the impact of poverty reduction strategies and interventions.

The poverty rate is defined at a regional level as the proportion of people
that fall below the poverty line (Organisation of Economic Co-operation and
Development, 2023) – a threshold that represents the minimum amount of money
required to cover a household’s basic needs. If a households’ income or
consumption expenditures are below this threshold, then the household is
considered to be poor. Household consumption expenditure data are needed to
estimate poverty rates and are calculated from representative and extensive
household demographic and living standard surveys such as the Demographic and
Health Surveys (DHS) and the Living Standards Measurement Study (LSMS).

While conducting extensive surveys represents an optimal approach for
estimating household consumption expenditure and, by extension, the poverty
rate, a single round of such surveys over an entire country is laborious and
entails substantial costs (Hall et al., 2022). The significant obstacles of
high survey costs pose a major challenge, particularly for lower-income
economies, in accurately assessing poverty rates over different locations and
periods. Consequently, this hinders the effective implementation of poverty
alleviation programs. Two sets of methodological improvements have been made
to estimate poverty in an attempt to mitigate the high costs of extensive
surveys: i) Proxy Means Tests (PMT) and ii) satellite imagery and machine
learning based methods. PMTs utilize a small survey (10 to 30 questions) to
obtain a set of measurable household variables which are in turn used for
poverty estimation (Kshirsagar et al., 2017). Here, the regression
coefficients for the PMT are calibrated to the extensive survey data, and the
calibrated PMT is used to make out-of-sample predictions given a much smaller
survey (Brown et al., 2018).

More recently, machine learning methods have been used to estimate asset
wealth and poverty across multiple countries using non-survey datasets such as
satellite imagery (Jean et al., 2016; Yeh et al., 2020; Head et al., 2017;
Tingzon et al., 2019; Tang et al., 2022; Ni et al., 2020; Jarry et al., 2021),
call data records (Pokhriyal and Jacques, 2017; Steele et al., 2017), social
media activity (Ledesma et al., 2020) or a combination of these (Chi et al.,
2022). Notably, these works show that greater than 50% of the variation in
survey-measured poverty can be explained with satellite image-based poverty
predictions (Hall et al., 2022).

Both PMT and satellite-image based methods have strengths and weaknesses. PMTs
utilize a reduced set of survey questions, thereby decreasing survey cost and
time, but necessarily discard additional relevant information for improved
poverty estimates. Satellite-image based methods utilize readily available
data with large geographic and temporal coverage to estimate poverty but can
only incorporate information that is readily available in the imagery.

Figure 1. Illustration of our methodology that combines satellite imagery in a
proxy means test to estimate poverty rate. 1.) Image features are extracted
from 10m/pixel Sentinel 2A data; 2.) A subset of survey questions are selected
considering their information overlap with the satellite image features; 3.)
Image features are combined with the subset of most important survey questions
to predict the poverty rate.

### Contribution

In this work we seek to combine the benefits of both PMT based and satellite
imagery based approaches and thus ask a central question: “What additional
information can be obtained from earth observation data to better complement
PMTs when predicting poverty rates?”. To this effect, we develop an approach
to integrate image features and survey data to better estimate the poverty
rate. Our contributions are three-fold:

  * •

This work shows that satellite image features are useful supplements to proxy
means tests in estimating poverty rates. With ten survey questions, we show
that including image features reduces the mean error in poverty rate estimates
from 4.09% to 3.88%. At a larger number of survey questions (20 - 50),
combining image features with survey questions always outperforms models that
use only survey questions to estimate the poverty rate.

  * •

We propose an approach for choosing survey questions that better complement
image features. This approach achieves the best performance, reducing the
poverty rate estimation error from 4.09% to 3.71%.

  * •

We perform interpretability analysis to understand the features extracted from
satellite imagery. We show that image features capture geographic differences
in urbanization levels, which better complement survey questions that do not
highlight geographic differences. By exploiting image features to select a
small set of survey questions, our approach maximizes the combined information
captured in the small set of survey questions plus image features.

##  2\. Problem Formulation

For a geographic area, $a$, the poverty rate, $pr_{a}$, is defined as the
proportion of households living below some poverty line111Poverty lines are
typically set at a national level., pl. Given the consumption expenditures of
a household, $hce_{i}$, obtained from a survey, a binary household poverty
indicator, $y_{i}$, is assigned using the criteria below:

$y_{i}=\begin{cases}\text{poor}&hce_{i}\leq pl\\\ \text{non-
poor}&hce_{i}>pl\end{cases}$

Using this, $pr_{a}$ is set as the weighted average of $y_{i}$ for all
households in the region:

(1) $pr_{a}=\frac{\sum\limits_{i\in a}w_{i}y_{i}}{\sum\limits_{i\in a}w_{i}}$

Here the weight given to each household, $w_{i}$, are given by the survey used
to calculate the household consumption expenditures.

We would like to develop a model, $\mathcal{F}$, that estimates a household
poverty indicator, $\hat{y}_{i}$, as a function of a household’s survey
responses, $x_{i}$, and the satellite imagery features, $m_{c}$, collected
from the same cluster222A cluster is an anonymized location assigned to a
group of households., $c$, that the household belongs to:

(2) $\hat{y}_{i}=\mathcal{F}(x_{i},m_{c})$

The predicted poverty rate, $\hat{pr}_{a}$, can then be calculated as a
weighted average of $\hat{y}_{i}$ over all households in a region.

Further, we would like to identify a subset of questions in a household survey
that can be combined with cluster level satellite imagery features to maximize
the performance of $\mathcal{F}$.

##  3\. Data

We use two datasets in this study, the Ethiopia Socioeconomic Survey (ESS),
that consists of household level surveys used to determine poverty status, and
Sentinel 2 imagery, that is used to derive household cluster level features.

### Poverty rate

The Ethiopia Socioeconomic Survey (ESS) dataset is a nationally representative
household Living Standards Measurement Study (LSMS) carried out by the World
Bank between 2015 to 2016 and is used to calculate the true poverty rate over
different regions. This survey captures responses from 4954 households
distributed over 528 clusters in Ethiopia (Central Statistical Agency of
Ethiopia, 2015).

Specifically, LSMS computed household consumption expenditures obtained from
the survey responses are used to obtain the binary indicator of poor / non-
poor households. Household consumption expenditure represents the value of
goods and services purchased by a household during a given period. Consumption
expenditures (adjusted to the household size) are compared to the national
poverty line threshold of $\sim$ 3 USD/day (in 2011 dollars) where households
with expenditures greater than the threshold are classified as non-poor.

Figure 2. Spatial distribution of the poverty rate over Ethiopia calculated
from the 2015/2016 Living Standards Measurement Study.

Figure 2 shows the spatial distribution of the poverty rate for geographic
clusters within the dataset. Here, 66% of the geographic clusters within the
dataset have a poverty rate less than 20% while 4.4% have a poverty rate of
over 80%.

### Survey questions

The LSMS household survey responses from the above survey are used to develop
the PMT. Categorical survey responses from the survey are converted to binary
variables. Seasonal survey variables (e.g. consumption of seasonal foods) are
excluded from the analysis together with variables with incomplete responses.
A comprehensive list of survey questions can be accessed here.

### Sentinel-2 Imagery

Sentinel-2 Level-2A data, is an open source 10-60 m/px spatial resolution
satellite imagery product provided by the European Space Agency (European
Space Agency, 2022). Sentinel-2 provides surface reflectance products in cloud
optimized GeoTIFF format, accessible through Microsoft’s Planetary Computer
(Microsoft Open Source and Matt McFarland and Rob Emanuele and Dan Morris and
Tom Augspurger, 2022). The LSMS survey period is used as the date query range
to find cloud-free images for each of the geolocated clusters. For each
location we select the lowest cloud cover image tile and download a 10 x 10 km
crop of RGB imagery centered on the geolocated cluster.

##  4\. Multimodal Poverty Rate Prediction

Our proposed method involves extracting relevant features from imagery and
combining the image features with survey variables to estimate the poverty
rate. In addition, we use the image features and survey variables to determine
the right set of small survey questions to utilize in a proxy means test for
estimating the poverty rate. Here we provide in depth descriptions of the
methods used.

### Image feature extraction.

Image feature extraction is performed using the MOSAIKS featurization method
proposed by (Rolf et al., 2021). This featurization method is attractive as it
does not require complex vision-based deep learning models, large compute
resources, or hyper-parameter tuning to extract relevant image features. The
empirical MOSAIKS approach is used, where a random set of 20 images are first
selected from the training set. Of the randomly selected images, a random set
of k, 3 X 3 image subpatches are selected and these become the filters for
feature extraction. Zero Component Analysis (ZCA) whitening are applied to the
k filters as a image processing normalization step. The processed filters are
then convolved with all images in the dataset and a RELU non-linear activation
is applied. Following this, an average pooling function is applied to the
outputs to produce a scalar value for each filter k. The resulting vector is
the image feature vector which is used in subsequent analysis. This
featurization step can be liken to a two layer non-trainable convolutional
neural network with 27 parameters per filter. In this work, 128 filters are
extracted from images and used for subsequent analysis. By randomly selecting
a small set of training images to build filters, this method provides a metric
of similarity between images in the dataset and those selected as filters.
This image feature extraction step is chosen over training an image based
model because of the data-hungry nature of image models compared to the small
number of images (528) in our dataset.

### Selecting survey questions for a PMT

A small set of relevant survey questions are identified using a two-step
process: First, an Explainable Boosting Machine (EBM) (Lou et al., 2012) is
trained to predict the poverty indicator ($y$) using all survey variables. EBM
is a tree-based gradient boosting algorithm that learns from only one feature
at a time, building smaller models (known as feature functions) for each
feature. To obtain a poverty prediction given a sample’s input features,
outputs from all feature functions are added, thus highlighting the
contribution of each feature. EBM models combine the interpretability of
Generalized Additive Models (GAMs) (Hastie and Tibshirani, 1990) with the
performance of gradient boosting models. After training the EBM model, the
contribution of each survey variable to the prediction is measured and used to
rank the survey variables. Given the survey feature importance scores, the n
most important survey variables are selected. A final step of retrieving other
variables belonging to the same questions as the most important variables is
performed. This process yields a reduced set of survey questions for the PMT
and is called the survey guided approach.

The reduced set of questions are evaluated in a downstream task. Here, the
small set of survey questions are used to train a model to predict the poverty
indicator, $y_{i}$, of a household. The model performance is measured with a
metric called poverty rate error – the difference between the true poverty
rate,pr, and the predicted poverty rate,$\hat{pr}$.

To obtain survey questions that better complement image features when used in
a PMT, we include image features in the survey question selection task. Survey
variables and image features are first used to train a model to predict the
poverty indicator. Feature importance scores are used to select the small set
of survey questions to be used in the PMT. This approach is called the survey
+ image guided approach. We compare the performance of the survey question
selection methods when only survey variables are used (survey guided) compared
to when survey variables are combined with image features (survey + image
guided).

### Data split

The dataset of 4954 households is split into train and test sets to ensure
out-of-sample performance is measured. Sixty-eight percent of the households
are randomly placed in the train set while thirty-two percent are placed in
the test set. Household allocation to train and test sets is done such that
the poverty rate in each set is equal to the national poverty rate. This data
split strategy is chosen to evaluate our model when a new set of random
households are surveyed. The train dataset is used to determine the most
salient survey questions. The test set is not used for variable selection but
only used after the downstream task to evaluate the performance of the reduced
set of survey variables in estimating the poverty rate.

### Training and optimization.

Both the survey variable selection model and the downstream poverty prediction
model are trained with the households from the train set using k-fold cross-
validation with $k=10$. The models are trained with a negative log-likelihood
loss. The LSMS survey sample weights are used during training to re-weight the
model such that the model learns from underrepresented samples. A grid search
hyper-parameter optimization over the learning rate (0.005, 0.0075, 0.01),
number of leaves (10, 31, 50) and interactions (5, 10, 20) is performed using
households in the training set during the survey variable selection step. The
obtained hyper-parameters are used to evaluate the choice of variables in the
downstream model. The results are reported on households from the out-of-
sample test set. Uncertainty is measured by bootstrapping 100 samples from the
test set 1000 times and computing the mean and standard deviations.

### Metrics

The main metric for evaluation is the poverty rate error (pre). This error is
the absolute difference between the true poverty rate and the predicted
poverty rate.

$\texttt{pre}=|pr-\hat{pr}|$

The mean and standard deviations of pre across the number of bootstrapping
iterations (1000) is obtained and reported.

##  5\. Results and discussion

### Evaluating the impact of image feature inclusion

A key contribution of this work is the inclusion of image features with
household survey variables to enchance proxy means testing. In this section,
we evaluate the impact of image feature inclusion by measuring the poverty
rate error when survey questions with and without image features are used in a
PMT to estimate the poverty rate.

Figure 3. Mean poverty rate (%) error as a function of number of survey questions. Errors are lower when image features are included compared to when only survey questions are used. Table 1. Mean and standard deviation of the poverty rate error (PRE). Results are reported by variable selection method and by PMT model input. The variable selection method shows the set of features used to determine the top 10 survey questions input into the PMT. PMT model inputs shows whether or not image features are combined with the top 10 questions to train the PMT model. Means of the PRE across 1000 iterations are reported with standard deviations shown in brackets. Variable selection method | PMT model inputs | Mean PRE (Std PRE)
---|---|---
All | Rural | Urban
Standard | Survey | 4.09 (3.03) | 0.41 (0.30) | 1.07 ($\sim$ 0)
Survey + Image | 3.88 (2.94) | 0.32 (0.23) | 0.66 ($\sim$ 0)
Survey Guided | Survey | 4.00 (3.10) | 0.65 (0.34) | 0.36 ($\sim$ 0)
Survey + Image | 3.81 (2.92) | 0.63 (0.31) | 0.86 ($\sim$ 0)
Survey + Image Guided | Survey | 3.95 (2.93) | 1.04 (0.36) | 0.70 ($\sim$ 0)
Survey + Image | 3.71 (2.84) | 0.30 (0.23) | 1.11 ($\sim$ 0)

Figure 4. Agreement between the true poverty rate and the predicted poverty
rate at the cluster level when standard PMT questions are used compared to
standard PMT questions + image features.

Figure 3 shows the mean poverty rate error over an increasing number of survey
questions with and without the inclusion of image features. At zero survey
questions (where only image features are used), the poverty rate error is
highest though not much higher than when only ten survey questions are used.
With the addition of a small number of survey questions (10) while retaining
images, the poverty rate error decreases from 4.1 % to 3.81 %. Increasing the
number of survey question to 20, reduced the poverty error rate with and
without images, though larger gains are observed when images are included.
Beyond 20 questions, adding more questions increases the poverty rate error
when images are included in the PMT. However, the increased error observed
with the inclusion of image features remains lower than the errors observed
when only survey questions are used in the PMT. In general, image features
consistently improve the poverty rate estimation over using only survey
variables. As more survey questions ($>$20) are acquired, a slight decreases
in the effectiveness of image features in estimating the poverty rate is
observed.

Figure 5. Pearson correlation between the regional survey indicators and
survey indicators that highly correlate with image features. Survey features
that highly correlate with image features also correlate with households being
in the urban region of Addis Ababa. This suggests that image features capture
the geographic differences between urban and non-urban areas.

### Evaluating the performance of the survey variable selection task

The survey variable selection process is evaluated by measuring the poverty
rate error of a PMT when the reduced set of survey questions are used. Here,
the error in the poverty rate obtained with the set of 10 survey questions
from our variable selection approach is compared to the error in the poverty
rate when a standard set of PMT questions are used. The Poverty Probability
Index (PPI) is a poverty measurement tool that uses 10 questions about
household characteristics and asset ownership in a PMT to compute the
likelihood of being poor(Innovations for Poverty Action, 2022). In Ethiopia,
10 standard questions are used to compute the PPI.

Table 1 shows the mean poverty rate error and in brackets the standard
deviations when the 10 standard questions are used. When images are combined
with the 10 standard questions for Ethiopia, improvements in poverty rate
estimation are observed as the error means and standard deviations in the
poverty rate decrease from 4.09 % to 3.88 % and 3.03 % and 2.94%,
respectively. This observation is consistent across the full test set and by
urbanization level.333Note that the standard deviations in the poverty error
rate are very small for urban regions and as a result are reported as
approximately zero.

The variable selection method is performed in two ways: using only survey
variables (survey guided) and using survey variables with image features
(survey + image guided).

The selected questions obtained through the survey guided variable selection
mechanism provides improved estimates of the poverty rate across the whole
dataset when only survey questions are used in the PMT. The gains are obtained
with improved performance in the urban set. Inclusion of images features to
the survey questions obtained through the survey guided mechanism further
reduces the overall poverty rate errors from 4.09 % to 3.81%, compared to 3.88
% when the standard questions are used. Inclusion of image features to the
survey questions obtained from the survey guided variable selection process
compared to inclusion of image features to the standard questions shows that
images are helpful in reducing the errors in the overall poverty rate estimate
but may not be tuned to performing better within a specific urbanization
region. This suggests that image features capture the differences between
urbanization levels and geographies but do not extract sufficient differences
within the same urbanization region. This behavior may be a limitation of the
images used as they are at a medium resolution of 10 m/pixel, and thus may not
capture adequate differences within geographies (e.g. differences in rooftop
materials within the same region or image scene) that are more evident at
higher resolutions.

The survey + image guided variable selection mechanism performs the best in
estimating the overall poverty rate when image features are combined with the
small set of obtained survey variables. The overall poverty rate errors
decrease from 4.09 % to 3.71 % while the rural poverty rate errors decrease
from 0.32 % to 0.30 %. Under the survey + image guided, the poverty rate
errors increase from 0.66 % to 1.11 %. The improved performance for the
overall and rural set suggests that by guiding the variable selection process
with both image features and survey variables, complementary survey variables
are selected to maximize information for improved performance. In addition to
decreased means in the poverty rate error, the survey + image guided mechanism
also shows the lowest standard deviations in the poverty rate errors.

### Cluster level performance

Using the ten standard questions, poverty rate estimates are also evaluated at
the cluster level for the test set. Figure 4 shows the agreement between the
true poverty rates and the predicted poverty rates with and without image
features. Here each point represents a geographic cluster. $r^{2}$ values are
reported at the top left corner of each sub-figure. The line of best fit is
also shown along side the 95 % confidence interval of a linear regression
model. Adding image features to the standard questions increases the $r^{2}$
to 0.42 from 0.30 when image features are not used. When survey + image guided
variables are used, the $r^{2}$ increases to 0.45. These findings suggest that
inclusion of image features from the single-step featurization method supports
better estimates of the poverty rate at varying spatial resolutions from
higher resolution geographic clusters to lower resolution nationwide
estimates.

### Inspection of image features

Image feature inspection is important to understanding the visual
characteristics that are retrieved during image feature extraction and how
these characteristics relate to survey questions. The non-trainable nature of
the featurization approach does not support direct extraction of spatial
characteristics that may correlate with poverty rates, as result we explore an
alternative approach described in this section. Principal component analysis
(PCA) is first applied to the image features to reduced the image feature
vector from 128 to its top three most relevant components. This dimensionality
reduction step supports the interpretability of image features while
preserving the relevant extracted information. Pearson correlations between
the principal components and the survey variables are computed as an
approximation of the information captured by images. Image features that
highly correlate with survey variables provide some interpretation of the
image features. The following ten survey questions had the highest Pearson
correlation with the top three components of the reduced image features: urban
area (urban/rural), whether the household is in Addis Ababa, whether the
household is in Gambelia, the use of Mud/Dung as the predominant floor
material, the cooking fuel, the use of piped water into the yard as the main
source of drinking water, whether the household purchases charcoal, the use of
an electric oven, the number of televisions owned by the house and the used of
non-specified solid waste disposal. Of the set of most correlated survey
variables, survey variables such as regional and urbanization differences are
more observable from imagery than others. To further understand these
correlations, these ten survey variables are correlated with geographic survey
variables to better understand their spatial characteristics.

Figure 5 shows the Pearson correlations between the ten survey variables that
agree with images (shown on the y-axis) and the regional survey variables
(x-axis). Addis Ababa and Gambelia are omitted from the y-axis as correlations
to themselves are 1. Amongst the remaining eight survey variables with the
highest image feature correlations, high correlations with the Addis Ababa
regional indicator are also observed.

This observation highlights two findings. First, while image features may be
correlated with less observable variables such as water source and cooking
material, these correlations emerge because image features capture the
underlying geographic differences that the less observable variables encode.
Secondly, image features from the single-step featurization strongly encode
the differences between urban (Addis Ababa) and non-urban regions. This second
observation further explains why the inclusion of image features improves
overall estimates of the poverty rate across rural and urban regions but does
not perform as well within an urbanization region.

### Complementary question discovery

The ten standard and survey guided questions consist of a regional indicator
which encodes the regions within the survey. In the survey + image guided
approach, the regional indicator is excluded (as the image features capture
regional and geographic differences across clusters) and the survey variable
selection model is allowed to chose the ten most relevant non-geographic
survey questions. Through this process the survey + image guided approach
retains five out of the ten questions initially chosen by the survey guided
approach. Five new complementary survey questions are discovered when image
features and survey question are used to determine the most important survey
questions, thereby maximizing the available information to estimate a poverty
rate.

##  6\. Conclusion

In this paper, we propose an approach to predicting the poverty rate of a
region through the combination of survey variables and satellite image
features. We show that the inclusion of image features consistently improves
the estimation of the poverty rate over using survey questions alone.

We also proposed a method for selecting the most relevant survey questions for
a PMT when image features are available. Our survey + image guided approach
outperforms other survey question section methods for poverty rate estimation.
Additionally, we conducted an interpretability analysis around the visual
features extracted by the single-step featurization method. We found that the
satellite image features capture underlying geographic differences in
urbanization levels complementing the survey questions for improved overall
estimation of the poverty rate.

Overall we demonstrate that satellite image features provide additional
information for poverty rate prediction. This better supports the possibility
of using a cost effective smaller set of survey variables as is the case with
proxy-means tests while keeping reasonable poverty predictions.

## References

  * (1)
  * Brown et al. (2018) Caitlin Brown, Martin Ravallion, and Dominique Van de Walle. 2018\.  A poor means test? Econometric targeting in Africa.  _Journal of Development Economics_ 134 (2018), 109–124.
  * Central Statistical Agency of Ethiopia (2015) Central Statistical Agency of Ethiopia. 2015\.  Ethiopia Socioeconomic Survey,Wave 3 (ESS3) 2015-2016.  https://microdata.worldbank.org/index.php/catalog/2783.
  * Chi et al. (2022) Guanghua Chi, Han Fang, Sourav Chatterjee, and Joshua E Blumenstock. 2022.  Microestimates of wealth for all low-and middle-income countries.  _Proceedings of the National Academy of Sciences_ 119, 3 (2022), e2113658119.
  * European Space Agency (2022) European Space Agency. 2022\.  Sentinel-2 Level-2A.  https://planetarycomputer.microsoft.com/dataset/sentinel-2-l2a.
  * Hall et al. (2022) Ola Hall, Mattias Ohlsson, and Thorsteinn Rögnvaldsson. 2022\.  A review of explainable AI in the satellite data, deep machine learning, and human poverty domain.  _Patterns_ 3, 10 (2022), 100600.
  * Hastie and Tibshirani (1990) T Hastie and R Tibshirani. 1990.  Generalized Additive Models. Chapman Hall & CRC.  _Monographs on Statistics & Applied Probability. Chapman and Hall/CRC_ 1 (1990).
  * Head et al. (2017) Andrew Head, Mélanie Manguin, Nhat Tran, and Joshua E Blumenstock. 2017.  Can human development be measured with satellite imagery?  _Ictd_ 17 (2017), 16–19.
  * Innovations for Poverty Action (2022) Innovations for Poverty Action. 2022\.  About the PPI: A Poverty Measurement Tool.  https://www.povertyindex.org/about-ppi#Why%20measure%20poverty.
  * Jarry et al. (2021) Robin Jarry, Marc Chaumont, Laure Berti-Équille, and Gérard Subsol. 2021.  Assessment of CNN-based methods for poverty estimation from satellite images. In _Pattern Recognition. ICPR International Workshops and Challenges: Virtual Event, January 10-15, 2021, Proceedings, Part VII_. Springer, 550–565.
  * Jean et al. (2016) Neal Jean, Marshall Burke, Michael Xie, W Matthew Davis, David B Lobell, and Stefano Ermon. 2016\.  Combining satellite imagery and machine learning to predict poverty.  _Science_ 353, 6301 (2016), 790–794.
  * Kshirsagar et al. (2017) Varun Kshirsagar, Jerzy Wieczorek, Sharada Ramanathan, and Rachel Wells. 2017.  Household poverty classification in data-scarce environments: a machine learning approach.  _arXiv preprint arXiv:1711.06813_ (2017).
  * Ledesma et al. (2020) Chiara Ledesma, Oshean Lee Garonita, Lorenzo Jaime Flores, Isabelle Tingzon, and Danielle Dalisay. 2020\.  Interpretable poverty mapping using social media data, satellite images, and geospatial information.  _arXiv preprint arXiv:2011.13563_ (2020).
  * Lou et al. (2012) Yin Lou, Rich Caruana, and Johannes Gehrke. 2012.  Intelligible models for classification and regression. In _Proceedings of the 18th ACM SIGKDD international conference on Knowledge discovery and data mining_. 150–158.
  * Microsoft Open Source and Matt McFarland and Rob Emanuele and Dan Morris and Tom Augspurger (2022) Microsoft Open Source and Matt McFarland and Rob Emanuele and Dan Morris and Tom Augspurger. 2022\.  _microsoft/PlanetaryComputer: October 2022_.  https://doi.org/10.5281/zenodo.7261897
  * Ni et al. (2020) Ye Ni, Xutao Li, Yunming Ye, Yan Li, Chunshan Li, and Dianhui Chu. 2020\.  An investigation on deep learning approaches to combining nighttime and daytime satellite imagery for poverty prediction.  _IEEE Geoscience and Remote Sensing Letters_ 18, 9 (2020), 1545–1549.
  * Organisation of Economic Co-operation and Development (2023) Organisation of Economic Co-operation and Development. 2023.  Poverty rate.  https://data.oecd.org/inequality/poverty-rate.htm.
  * Pokhriyal and Jacques (2017) Neeti Pokhriyal and Damien Christophe Jacques. 2017\.  Combining disparate data sources for improved poverty prediction and mapping.  _Proceedings of the National Academy of Sciences_ 114, 46 (2017), E9783–E9792.
  * Rolf et al. (2021) Esther Rolf, Jonathan Proctor, Tamma Carleton, Ian Bolliger, Vaishaal Shankar, Miyabi Ishihara, Benjamin Recht, and Solomon Hsiang. 2021\.  A generalizable and accessible approach to machine learning with global satellite imagery.  _Nature communications_ 12, 1 (2021), 1–11.
  * Steele et al. (2017) Jessica E Steele, Pål Roe Sundsøy, Carla Pezzulo, Victor A Alegana, Tomas J Bird, Joshua Blumenstock, Johannes Bjelland, Kenth Engø-Monsen, Yves-Alexandre De Montjoye, Asif M Iqbal, et al. 2017\.  Mapping poverty using mobile phone and satellite data.  _Journal of The Royal Society Interface_ 14, 127 (2017), 20160690\.
  * Tang et al. (2022) Binh Tang, Yanyan Liu, and David S Matteson. 2022.  Predicting poverty with vegetation index.  _Applied Economic Perspectives and Policy_ 44, 2 (2022), 930–945.
  * Tingzon et al. (2019) Isabelle Tingzon, Ardie Orden, KT Go, S Sy, V Sekara, Ingmar Weber, M Fatehkia, M García-Herranz, and D Kim. 2019\.  MAPPING POVERTY IN THE PHILIPPINES USING MACHINE LEARNING, SATELLITE IMAGERY, AND CROWD-SOURCED GEOSPATIAL INFORMATION.  _International Archives of the Photogrammetry, Remote Sensing & Spatial Information Sciences_ (2019).
  * United Nations (2023) United Nations. 2023\.  Goal 1: End poverty in all its forms everywhere.  https://www.un.org/sustainabledevelopment/poverty/.
  * Yeh et al. (2020) Christopher Yeh, Anthony Perez, Anne Driscoll, George Azzari, Zhongyi Tang, David Lobell, Stefano Ermon, and Marshall Burke. 2020\.  Using publicly available satellite imagery and deep learning to understand economic well-being in Africa.  _Nature communications_ 11, 1 (2020), 1–11.


# Integral morphisms and log blow-ups

Fumiharu Kato

###### Abstract.

This paper is a revision of the author’s old preprint “Exactness, integrality,
and log modifications”. We will prove that any quasi-compact morphism of fs
log schemes can be modified locally on the base to an integral morphism by
base change by fs log blow-ups.

###### 2010 Mathematics Subject Classification:

Primary: 14A99, Secondary: 14E05

##  1\. Introduction

The aim of this paper is to prove the following theorem:

######  Theorem 1.1.

Let $f\colon X\rightarrow Y$ be a quasi-compact morphism of fs $(=$ fine and
saturated$)$ log schemes. Then for any $y\in Y$ there exists an étale
neighborhood $V\rightarrow Y$ of $\overline{y}$ $(=$ a separable closure of
$y)$ and an fs log blow-up
$V^{\prime}:=\operatorname{Bl}_{\mathcal{K}}(V)\rightarrow V$ along a coherent
log ideal $\mathcal{K}\subset\mathcal{M}_{V}$, by which the fs base change
$f_{V^{\prime}}\colon X_{V^{\prime}}\rightarrow V^{\prime}$ is integral.

Here, by fs log blow-up (resp. fs base change) we mean a log blow-up (resp.
base change) in the category of fs log schemes; cf. Remark 3.6.

This theorem has been announced and proved in somewhat incomplete and
inaccurate form in the author’s old preprint [5], a first draft of which has
actually been written in 1997, and afterwards put in the arXiv in 1999. Since
then, mainly because the author has been away from log geometry, the paper has
been kept unpublished; the author apologizes for all inconvenience caused
thereby. While there have been much progress and many new results in log
geometry last two decades, the paper has sometimes been referred to. Moreover,
it seems, to the best of the author’s knowledge, that the theorem itself has
not yet been written anywhere, even in the foundational book [11] by Ogus, and
became folklore among experts.

In fact, the theorem is nowadays a consequence of combination of known
results. For example, Luc Illusie, Kazuya Kato, and Chikara Nakayama proved in
[4] (see also [11, III.2.6.7]) a weak version of the theorem, where “integral”
is replaced by “$\mathbb{Q}$-integral”. Then by a further fs log blow-up of
the base to make the log structure free (i.e., to make each stalk of
$\overline{\mathcal{M}}=\mathcal{M}/\mathcal{O}^{\times}$ a free monoid), the
resulting map becomes integral (cf. [11, I.4.7.5]). Since, due to Nizioł [10,
4.11], the composition of fs log blow-ups is again an fs log blow-up, this
actually suffices to prove the theorem.

In the mean time, in August 2020, Michael Temkin asked the author some
questions on the preprint, and suggested the final form of the theorem
presented as above. Based upon the fact that the theorem has to be referred to
in a recent work [2] of him and his coauthors, and that the theorem has not
yet been presented in published form, he encouraged the author to revise the
old preprint for publishing. This is the situation from whence the present
paper comes out, where we keep the original proof based on the technique of
toric flattening, the original idea of which is attributed to Takeshi
Kajiwara.

Let us mention some consequences of Theorem 1.1. Tsuji [13, II.3.4] proved
that any integral and quasi-compact morphism between fs log schemes can be
made saturated by fs base change by “multiplication-by-$n$” map. Combined with
our result, this yields the following:

######  Corollary 1.2.

Let $f\colon X\rightarrow Y$ be a quasi-compact morphism between fs log
schemes. Then for any $y\in Y$ there exists a diagram

$\lx@xy@svg{\hbox{\raise 0.0pt\hbox{\kern
14.38945pt\hbox{\ignorespaces\ignorespaces\ignorespaces\hbox{\vtop{\kern
0.0pt\offinterlineskip\halign{\entry@#!@&&\entry<EMAIL_ADDRESS>0.0pt\hbox{\hbox{\kern 0.0pt\raise 0.0pt\hbox{\hbox{\kern 3.0pt\raise
0.0pt\hbox{$\textstyle{V^{\prime\prime}\ignorespaces\ignorespaces\ignorespaces\ignorespaces\ignorespaces\ignorespaces\ignorespaces\ignorespaces}$}}}}}}}\ignorespaces\ignorespaces\ignorespaces\ignorespaces{}{\hbox{\lx@xy@droprule}}\ignorespaces{\hbox{\kern
44.98112pt\raise 0.0pt\hbox{\hbox{\kern 0.0pt\raise
0.0pt\hbox{\lx@xy@tip{1}\lx@xy@tip{-1}}}}}}{\hbox{\lx@xy@droprule}}{\hbox{\lx@xy@droprule}}\ignorespaces\ignorespaces\ignorespaces\ignorespaces{}{\hbox{\lx@xy@droprule}}\ignorespaces{\hbox{\kern
0.0pt\raise-29.49998pt\hbox{\hbox{\kern 0.0pt\raise
0.0pt\hbox{\lx@xy@tip{1}\lx@xy@tip{-1}}}}}}{\hbox{\lx@xy@droprule}}{\hbox{\lx@xy@droprule}}{\hbox{\kern
44.98112pt\raise 0.0pt\hbox{\hbox{\kern 0.0pt\raise 0.0pt\hbox{\hbox{\kern
3.0pt\raise
0.0pt\hbox{$\textstyle{V^{\prime}\ignorespaces\ignorespaces\ignorespaces\ignorespaces\ignorespaces\ignorespaces\ignorespaces\ignorespaces}$}}}}}}}\ignorespaces\ignorespaces\ignorespaces\ignorespaces{\hbox{\kern
60.57668pt\raise 0.0pt\hbox{\hbox{\kern 0.0pt\raise
0.0pt\hbox{\lx@xy@hook{1}}}}}}{\hbox{\lx@xy@droprule}}\ignorespaces{\hbox{\kern
91.16835pt\raise 0.0pt\hbox{\hbox{\kern 0.0pt\raise
0.0pt\hbox{\lx@xy@tip{1}\lx@xy@tip{-1}}}}}}{\hbox{\lx@xy@droprule}}{\hbox{\lx@xy@droprule}}\ignorespaces\ignorespaces\ignorespaces\ignorespaces{}{\hbox{\lx@xy@droprule}}\ignorespaces{\hbox{\kern
52.7789pt\raise-29.49998pt\hbox{\hbox{\kern 0.0pt\raise
0.0pt\hbox{\lx@xy@tip{1}\lx@xy@tip{-1}}}}}}{\hbox{\lx@xy@droprule}}{\hbox{\lx@xy@droprule}}{\hbox{\kern
91.16835pt\raise 0.0pt\hbox{\hbox{\kern 0.0pt\raise 0.0pt\hbox{\hbox{\kern
3.0pt\raise
0.0pt\hbox{$\textstyle{\operatorname{Bl}_{K}(V)\ignorespaces\ignorespaces\ignorespaces\ignorespaces\ignorespaces\ignorespaces\ignorespaces\ignorespaces}$}}}}}}}\ignorespaces\ignorespaces\ignorespaces\ignorespaces{}{\hbox{\lx@xy@droprule}}\ignorespaces{\hbox{\kern
109.59393pt\raise-30.38885pt\hbox{\hbox{\kern 0.0pt\raise
0.0pt\hbox{\lx@xy@tip{1}\lx@xy@tip{-1}}}}}}{\hbox{\lx@xy@droprule}}{\hbox{\lx@xy@droprule}}\ignorespaces\ignorespaces\ignorespaces\ignorespaces{}{\hbox{\lx@xy@droprule}}\ignorespaces{\hbox{\kern
158.6112pt\raise 0.0pt\hbox{\hbox{\kern 0.0pt\raise
0.0pt\hbox{\lx@xy@tip{1}\lx@xy@tip{-1}}}}}}{\hbox{\lx@xy@droprule}}{\hbox{\lx@xy@droprule}}{\hbox{\kern
158.6112pt\raise 0.0pt\hbox{\hbox{\kern 0.0pt\raise 0.0pt\hbox{\hbox{\kern
3.0pt\raise
0.0pt\hbox{$\textstyle{V\ignorespaces\ignorespaces\ignorespaces\ignorespaces\ignorespaces\ignorespaces\ignorespaces\ignorespaces}$}}}}}}}\ignorespaces\ignorespaces\ignorespaces\ignorespaces{}{\hbox{\lx@xy@droprule}}\ignorespaces{\hbox{\kern
165.63898pt\raise-30.38885pt\hbox{\hbox{\kern 0.0pt\raise
0.0pt\hbox{\lx@xy@tip{1}\lx@xy@tip{-1}}}}}}{\hbox{\lx@xy@droprule}}{\hbox{\lx@xy@droprule}}\ignorespaces\ignorespaces\ignorespaces\ignorespaces{}{\hbox{\lx@xy@droprule}}\ignorespaces{\hbox{\kern
203.25844pt\raise 0.0pt\hbox{\hbox{\kern 0.0pt\raise
0.0pt\hbox{\lx@xy@tip{1}\lx@xy@tip{-1}}}}}}{\hbox{\lx@xy@droprule}}{\hbox{\lx@xy@droprule}}{\hbox{\kern
203.25844pt\raise 0.0pt\hbox{\hbox{\kern 0.0pt\raise 0.0pt\hbox{\hbox{\kern
3.0pt\raise
0.0pt\hbox{$\textstyle{Y}$}}}}}}}{\hbox{\kern-14.38945pt\raise-40.88885pt\hbox{\hbox{\kern
0.0pt\raise 0.0pt\hbox{\hbox{\kern 3.0pt\raise
0.0pt\hbox{$\textstyle{\operatorname{S}(Q^{\prime})\ignorespaces\ignorespaces\ignorespaces\ignorespaces}$}}}}}}}\ignorespaces\ignorespaces\ignorespaces\ignorespaces{}{\hbox{\lx@xy@droprule}}\ignorespaces\ignorespaces\ignorespaces{\hbox{\kern
21.28053pt\raise-46.07634pt\hbox{{}\hbox{\kern 0.0pt\raise
0.0pt\hbox{\hbox{\kern 3.0pt\hbox{\hbox{\kern
0.0pt\raise-0.8264pt\hbox{$\scriptstyle{\mu}$}}}\kern
3.0pt}}}}}}\ignorespaces{\hbox{\kern
38.38945pt\raise-40.88885pt\hbox{\hbox{\kern 0.0pt\raise
0.0pt\hbox{\lx@xy@tip{1}\lx@xy@tip{-1}}}}}}{\hbox{\lx@xy@droprule}}{\hbox{\lx@xy@droprule}}{\hbox{\kern
38.38945pt\raise-40.88885pt\hbox{\hbox{\kern 0.0pt\raise
0.0pt\hbox{\hbox{\kern 3.0pt\raise
0.0pt\hbox{$\textstyle{\operatorname{S}(Q^{\prime})\ignorespaces\ignorespaces\ignorespaces\ignorespaces}$}}}}}}}\ignorespaces\ignorespaces\ignorespaces\ignorespaces{\hbox{\kern
67.16837pt\raise-40.88885pt\hbox{\hbox{\kern 0.0pt\raise
0.0pt\hbox{\lx@xy@hook{1}}}}}}{\hbox{\lx@xy@droprule}}\ignorespaces{\hbox{\kern
91.24336pt\raise-40.88885pt\hbox{\hbox{\kern 0.0pt\raise
0.0pt\hbox{\lx@xy@tip{1}\lx@xy@tip{-1}}}}}}{\hbox{\lx@xy@droprule}}{\hbox{\lx@xy@droprule}}{\hbox{\kern
91.24336pt\raise-40.88885pt\hbox{\hbox{\kern 0.0pt\raise
0.0pt\hbox{\hbox{\kern 3.0pt\raise
0.0pt\hbox{$\textstyle{\operatorname{Bl}_{K}(Q)\ignorespaces\ignorespaces\ignorespaces\ignorespaces}$}}}}}}}\ignorespaces\ignorespaces\ignorespaces\ignorespaces{}{\hbox{\lx@xy@droprule}}\ignorespaces\ignorespaces\ignorespaces{\hbox{\kern
132.62137pt\raise-45.3958pt\hbox{{}\hbox{\kern 0.0pt\raise
0.0pt\hbox{\hbox{\kern 3.0pt\hbox{\hbox{\kern
0.0pt\raise-1.50694pt\hbox{$\scriptstyle{\pi}$}}}\kern
3.0pt}}}}}}\ignorespaces{\hbox{\kern
152.01952pt\raise-40.88885pt\hbox{\hbox{\kern 0.0pt\raise
0.0pt\hbox{\lx@xy@tip{1}\lx@xy@tip{-1}}}}}}{\hbox{\lx@xy@droprule}}{\hbox{\lx@xy@droprule}}{\hbox{\kern
152.01952pt\raise-40.88885pt\hbox{\hbox{\kern 0.0pt\raise
0.0pt\hbox{\hbox{\kern 3.0pt\raise
0.0pt\hbox{$\textstyle{\operatorname{S}(Q)}$}}}}}}}\ignorespaces}}}}\ignorespaces,$

with all vertical arrows strict and all squares cartesian in the category of
fs log schemes, such that

  * •

$V\rightarrow Y$ is an étale neighborhood of $\overline{y}$;

  * •

$\pi$ is an fs log blow-up of
$\operatorname{S}(Q)=\operatorname{Spec}\mathbb{Z}[Q]$ $($with the log
structure by $Q\rightarrow\mathbb{Z}[Q]$$)$ along an ideal $K\subset Q$;

  * •

$\operatorname{S}(Q^{\prime})\hookrightarrow\operatorname{Bl}_{K}(Q)$ is an
arbitrary affine patch of $\operatorname{Bl}_{K}(Q)$;

  * •

$\mu$ is the morphism induced from the multiplication-by-$n$ homomorphism
$Q^{\prime}\rightarrow Q^{\prime}$ for some positive integrer $n$,

and that the fs base change $f_{V^{\prime\prime}}\colon
X\times_{Y}V^{\prime\prime}\rightarrow V^{\prime\prime}$ is saturated.

Our second application is to log flatness.

######  Corollary 1.3.

Let $f\colon X\rightarrow Y$ be a log flat and finitely presented morphism
between fs log schemes. Then for any $y\in Y$ there exists an étale
neoghborhood $V\rightarrow Y$ of $\overline{y}$ and an fs log blow-up
$V^{\prime}\rightarrow V$ such that the underlying morphism of the fs base
change $f_{V^{\prime}}\colon X\times_{Y}V^{\prime}\rightarrow V^{\prime}$ is
flat.

Indeed, we may assume that $f$ is integral, log flat, and of finite
presentation. We may further assume that $f$ has a global chart by $h\colon
Q\rightarrow P$, which is “neat” at a point $x\in X$ (as in Lemma 2.4 below)
such that $Q\cong\overline{\mathcal{M}}_{Y,\overline{y}}$ ($y=f(x)$) and
$\overline{P}\cong\overline{\mathcal{M}}_{X,\overline{x}}$. Then, since
$\overline{h}\colon\overline{Q}=Q\rightarrow\overline{P}$ is integral, so is
$h$ (cf. Lemma 2.2 (5)). Since $Q$ is sharp (i.e., $Q^{\times}=\\{1\\}$) and
$h$ is local (i.e., $h^{-1}(P^{\times})=Q^{\times}$), the ring homomorphism
$\mathbb{Z}[P]\rightarrow\mathbb{Z}[Q]$ is flat (cf. Lemma 2.2 (6)). Since the
log flatness implies that $X$ is flat over
$Y\otimes_{\mathbb{Z}[Q]}\mathbb{Z}[P]$ ([12, 4.15]), we deduce that $f$
itself is flat.

As for the interaction between log flatness and usual flatness, much has been
studied recently by some authors. Among them, we refer to a preprint by Gillam
[3]. It seems that, combining our result with many of the results therein, we
can deduce several useful consequences.

Finally, let us remark that, if $Y$ in Theorem 1.1 is log regular, then $V$
can be equal to $Y$ itself, i.e., one can find an fs log blow-up of $Y$ itself
that makes the morphism $f$ integral by fs base change. This version of the
theorem has been proven independently in [2, 3.6.11], which we will include,
with a few comments, at the end of this paper (Proposition 4.2).

######  Remark 1.4.

The original paper [5] included an “exactness version” of the theorem, where
“integral” is replaced by “exact”, which we do not include in the present
paper, since it follows immediately from the result in [4] mentioned above.

The composition of this paper is as follows. In the next section, we will
collect some basics of integral morphisms and neat charts. In Section 3, we
will overview log blow-ups. We will then prove the main theorem in Section 4.

The original version of the paper owes much to Richard Pink, Takeshi Kajiwara,
and Max Planck Institute für Mathematik in Bonn, Germany. In addition, the
preparation of the present version owes much to Michael Temkin for his
encouragement, and to Chikara Nakayama for valuable discussions.

###  1.5. Notation and conventions

All rings and monoids are assumed to be commutative. For a monoid $M$, we
denote by $M^{\times}$ and $M^{\mathrm{gp}}$ the subgroup of invertible
elements and the associated group, respectively, and write
$\overline{M}=M/M^{\times}$.

All sheaves on schemes are considered with respect to the étale topology. For
a point $x$ of a scheme, we denote by $\overline{x}$ a separable closure of
$x$. For a log scheme $X$, we denote by
$\alpha_{X}\colon\mathcal{M}_{X}\rightarrow\mathcal{O}_{X}$ the log structure
of $X$, and write
$\overline{\mathcal{M}}_{X}=\mathcal{M}_{X}/\mathcal{O}^{\times}_{X}$. We
denote by $\underline{X}$ the underlying scheme of $X$, which is also
considered as a log scheme with the trivial log structure.

For a monoid $P$, we denote by $\operatorname{S}(P)$ the log scheme whose
underlying scheme is the affine scheme $\operatorname{Spec}\mathbb{Z}[P]$ with
the log structure induced from $P\rightarrow\mathbb{Z}[P]$.
($\operatorname{S}(P)$ is denoted by $\mathsf{A}_{P}$ in [11].)

##  2\. Integral homomorphisms

Let us first recall the definition of integral homomorphisms.

######  Definition 2.1.

A homomorphism $h\colon Q\rightarrow P$ of integral monoids is said to be
integral if, for any integral monoid $Q^{\prime}$ and any homomorphism
$Q\rightarrow Q^{\prime}$, the push-out $P\oplus_{Q}Q^{\prime}$ in the
category of monoids is an integral monoid.

It can be shown ([6, (4.1)][11, I.4.6.2]) that $h\colon Q\rightarrow P$ is
integral if and only if it has the following property: if
$h(a_{1})b_{1}=h(a_{2})b_{2}$ for $a_{1},a_{2}\in Q$ and $b_{1},b_{2}\in P$,
there exists $a_{3},a_{4}\in Q$ and $b\in P$ such that $b_{1}=h(a_{3})b$,
$b_{2}=h(a_{4})b$ and $a_{1}a_{3}=a_{2}a_{4}$.

######  Lemma 2.2.

$(1)$ The composition of integral homomorphisms is integral. For a diagram
$Q\stackrel{{\scriptstyle h}}{{\rightarrow}}P\stackrel{{\scriptstyle
k}}{{\rightarrow}}R$ of integral monoids, if $k\circ h$ is integral and $k$ is
exact, then $h$ is integral; if $k\circ h$ is integral and $h$ is surjective,
then $k$ is integral.

$(2)$ The pushout of an integral homomorphism in the category of monoids is
integral.

$(3)$ If $P$ is an integral monoid, and $N\subset P$ is a submonoid, then the
canonical map $P\rightarrow P/N$ is integral.

$(4)$ An integral and local homomorphism of integral monoids is exact.

$(5)$ A homomorphism $h\colon Q\rightarrow P$ of integral monoids is integral
if and only if $\overline{h}\colon\overline{Q}\rightarrow\overline{P}$ is
integral.

$(6)$ A homomorphism $h\colon Q\rightarrow P$ of integral monoids is integral
if the homomorphism of monoid algebras $\mathbb{Z}[Q]\rightarrow\mathbb{Z}[P]$
is flat. The converse is true, if $h$ is local and $Q$ is sharp.

Recall that a homomorphism $h\colon Q\rightarrow P$ of integral monoids said
to be exact if $(h^{\mathrm{gp}})^{-1}(P)=Q$, where $h^{\mathrm{gp}}\colon
Q^{\mathrm{gp}}\rightarrow P^{\mathrm{gp}}$ is the associated group
homomorphism. Recall also that, for a monoid $P$ and a submonoid $N\subset P$,
the quotient monoid $P/N$ is given by $P/\sim$ (endowed with the natural
monoid structure), where $a\sim b$ if and only if $ac=bd$ for some $c,d\in N$.

###### Proof.

For (1), (4), and (6), see [11, I.4.6.3 & I.4.6.7]. (2) is immediate from the
definition of integral homomorphisms. (5) follows from (1), (3), and the fact
that a homomorphism of integral monoids of the form $Q\rightarrow\overline{Q}$
is always exact. Hence it suffices to show (3). To show that $\pi\colon
P\rightarrow P/N$ ($a\mapsto\overline{a}$) is integral, take
$a_{1},a_{2},b_{1},b_{2}\in P$ such that
$\overline{a_{1}}\overline{b_{1}}=\overline{a_{2}}\overline{b_{2}}$; the last
equality means $a_{1}b_{1}c_{1}=a_{2}b_{2}c_{2}$ for $c_{1},c_{2}\in N$, and
if we set $a_{3}=b_{1}c_{1}$, $a_{4}=b_{2}c_{2}$ and $b=\overline{1}$, then we
have $a_{1}a_{3}=a_{2}a_{4}$, $\overline{b_{1}}=\overline{a_{3}}b$ and
$\overline{b_{2}}=\overline{a_{4}}b$, which shows that $\pi$ is integral. ∎

######  Definition 2.3.

A morphism $f\colon X\rightarrow Y$ of integral log schemes is said to be
integral at $x\in X$ if the monoid homomorphism
$\overline{\mathcal{M}}_{Y,\overline{y}}\rightarrow\overline{\mathcal{M}}_{X,\overline{x}}$,
where $y=f(x)$, is integral, or equivalently,
$\mathcal{M}_{Y,\overline{y}}\rightarrow\mathcal{M}_{X,\overline{x}}$ is
integral (cf. Lemma 2.2 (6)). We say $f$ is integral if it is integral at all
points of $X$.

By Lemma 2.2 (2), integral morphisms are stable under base change in the
category of fine log schemes. It is, however, not true that integral morphisms
are stable under base change in the category of fs log schemes, cf. [11,
I.4.6.5]. So it is often convenient to refer to a base change (or a fiber
product) in the category of fs log schemes as fs base change for emphasis.

Let us finally mention some technical facts on charts.

######  Lemma 2.4.

Let $f\colon X\rightarrow Y$ be a morphism of fs log schemes. Then, for any
$x\in X$ and $y=f(x)$, there exists commutative diagram

$\textstyle{X\ignorespaces\ignorespaces\ignorespaces\ignorespaces}$$\scriptstyle{f}$$\textstyle{U\ignorespaces\ignorespaces\ignorespaces\ignorespaces\ignorespaces\ignorespaces\ignorespaces\ignorespaces\ignorespaces\ignorespaces\ignorespaces\ignorespaces}$$\scriptstyle{g}$$\textstyle{\operatorname{S}(P)\ignorespaces\ignorespaces\ignorespaces\ignorespaces}$$\textstyle{Y}$$\textstyle{V\ignorespaces\ignorespaces\ignorespaces\ignorespaces\ignorespaces\ignorespaces\ignorespaces\ignorespaces}$$\textstyle{\operatorname{S}(Q)}$

comprised of an étale neighborhood $V\rightarrow Y$ of $\overline{y}$, an fppf
neighborhood $U\rightarrow X$ of $\overline{x}$, and a chart $Q\rightarrow P$
of $g\colon U\rightarrow V$ such that the following conditions are satisfied:

  * (a)

$P$ and $Q$ are fs monoids;

  * (b)

$Q\cong\overline{\mathcal{M}}_{Y,\overline{y}}$, and
$\overline{P}\cong\overline{\mathcal{M}}_{X,\overline{x}}$;

  * (c)

$Q^{\mathrm{gp}}\rightarrow P^{\mathrm{gp}}$ is injective and
$P^{\mathrm{gp}}/Q^{\mathrm{gp}}\cong\mathcal{M}^{\mathrm{gp}}_{X/Y,\overline{x}}$.

$($Note that, in this situation, $Q$ is sharp, and $Q\rightarrow P$ is
local.$)$

###### Proof.

Since $Y$ is fs, one can take on an étale neighborhood of $\overline{y}$ a
chart subordinate to the fs monoid
$Q=\overline{\mathcal{M}}_{Y,\overline{y}}$. Then one can construct a local
chart of $f$ as above according to the recipe as in [11, III.1.2.7], where one
can take $P$ to be an fs monoid by the construction as in [11, II.2.4.4] ∎

######  Lemma 2.5.

Let $f\colon X\rightarrow Y$ be a morphism of fine log schemes.

$(1)$ Suppose $f$ has a $($global$)$ chart by a homomorphism $h\colon
Q\rightarrow P$ of fine monoids, such that the conditions (b) and (c) in Lemma
2.4 is satisfied. Then $f$ is integral if and only if the homomorphism $h$ is
integral.

$(2)$ If $f$ is integral at $x$, then it is integral at all points in an open
neighborhood of $x$.

###### Proof.

See [11, III.2.5.2]. ∎

##  3\. Log blow-ups

In this section, we briefly recall the notion of log blow-ups and their basic
properties.

Recall first that an ideal of a monoid $M$ is a subset $K\subset M$ such that
$x\in K$ and $a\in M$ imply $ax\in K$. Trivial ideals are $\emptyset$ and $M$
itself. It follows from Dickson’s lemma that any ideal of a finitely generated
monoid is finitely generated. If $\pi\colon M\rightarrow\overline{M}$ is the
canonical map, the map $K\mapsto\pi(K)$ gives a bijection from the set of
ideals of $M$ to the set of ideals of $\overline{M}$.

Let $X$ be a log scheme. A log ideal of $X$ is a sheaf of ideals $\mathcal{K}$
of $\mathcal{M}_{X}$. We denote by $\overline{\mathcal{K}}$ the corresponding
ideal of $\overline{\mathcal{M}}_{X}$. For a morphism $f\colon X\rightarrow Y$
of fine log schemes and a log ideal $\mathcal{K}$ of $Y$, one has the
extension of the log ideal
$\mathcal{K}\mathcal{M}_{X}=(f^{-1}\mathcal{K})\mathcal{M}_{X}$, which is a
log ideal of $X$.

######  Example 3.1.

Let $P$ be a monoid and $K\subset P$ an ideal. One has the log ideal
$\widetilde{K}$ associated to $K$ on $X=\operatorname{S}(P)$, constructed as
follows. For any open subset $U\subset X$, we have a monoid homomorphism
$P\rightarrow\mathcal{M}_{X}(U)$, and hence the extension ideal
$K\mathcal{M}_{X}(U)$ of $\mathcal{M}_{X}(U)$. Then $\widetilde{K}$ is the
sheafification of the subpresheaf of ideals of $\mathcal{M}_{X}$ given by
$U\mapsto K\mathcal{M}_{X}(U)$. Note that, for any $x\in X$, we have
$\widetilde{K}_{\overline{x}}=K\mathcal{M}_{X,\overline{x}}$.

######  Definition 3.2.

A log ideal $\mathcal{K}$ of $X$ is called coherent at $x\in X$ if there
exists a local chart $U\rightarrow\operatorname{S}(P)$, where $U$ is an étale
neighborhood around $\overline{x}$, and an ideal $K\subset P$ such that
$\mathcal{K}|_{U}=\widetilde{K}\mathcal{M}_{U}$ (let us say, in this
situation, that $K$ is a chart of $\mathcal{K}$ over $U$). A log ideal is
called coherent if it is coherent at all points.

######  Remark 3.3 (cf. [11, II.2.6.2]).

If a log ideal $\mathcal{K}$ of $X$ is coherent at $x\in X$, then, for any
local chart $U\rightarrow\operatorname{S}(P)$ around $\overline{x}$, the
pullback ideal $K\subset P$ of $\overline{\mathcal{K}}_{\overline{x}}$ by
$P\rightarrow\mathcal{M}_{X,\overline{x}}\rightarrow\overline{\mathcal{M}}_{X,\overline{x}}$
generates $\mathcal{K}$ around $\overline{x}$; i.e.,
$\widetilde{K}\mathcal{M}_{U^{\prime}}=\mathcal{K}|_{U^{\prime}}$ over an
étale neighborhood $U^{\prime}$ of $\overline{x}$ contained in $U$.

A coherent log ideal $\mathcal{K}$ of a log scheme $X$ is said to be
invertible if, for any $x\in X$, $\overline{\mathcal{K}}_{\overline{x}}$ is a
principal (i.e. generated by a single element) ideal of
$\overline{\mathcal{M}}_{X,\overline{x}}$, or equivalently, there exist étale
locally a chart $U\rightarrow\operatorname{S}(P)$ and an ideal $K\subset P$ as
in Definition 3.2 with $K$ being principal.

######  Definition 3.4.

A morphism $f\colon X^{\prime}\rightarrow X$ of fine (resp. fs) log schemes is
called a log blow-up along a coherent log ideal $\mathcal{K}$ if it has the
following universal mapping property:

  * (a)

$\mathcal{K}\mathcal{M}_{X^{\prime}}$ is an invertible log ideal of
$\mathcal{M}_{X^{\prime}}$;

  * (b)

If $g\colon T\rightarrow X$ is a morphism of fine (resp. fs) log schemes such
that $\mathcal{K}\mathcal{M}_{T}$ is invertible, then there exists a uniquely
morphism $T\rightarrow X^{\prime}$ such that the diagram

$\textstyle{T\ignorespaces\ignorespaces\ignorespaces\ignorespaces\ignorespaces\ignorespaces\ignorespaces\ignorespaces}$$\scriptstyle{g}$$\textstyle{X^{\prime}\ignorespaces\ignorespaces\ignorespaces\ignorespaces}$$\scriptstyle{f}$$\textstyle{X}$

commutes.

The log blow-up along a coherent log ideal is unique up to isomorphisms. Since
every extension of invertible log ideal is again invertible, we have:

######  Lemma 3.5.

The family of log blow-ups is stable under base change. More precisely, if
$X_{\mathcal{K}}\rightarrow X$ is a log blow-up of a fine log scheme $X$ along
a coherent log ideal $\mathcal{K}$, and if $Y\rightarrow X$ is a morphism of
fine log schemes, then $X_{\mathcal{K}}\times_{X}Y\rightarrow Y$ is a log
blow-up of $Y$ along $\mathcal{K}\mathcal{M}_{Y}$.

If $X$ is an fs log scheme, and $f\colon X^{\prime}\rightarrow X$ is a log
blow-up in the category of fine log schemes, then the saturation
$f^{\mathrm{sat}}\colon X^{\prime\mathrm{sat}}\rightarrow X$ gives a log blow-
up of $X$ along the same coherent log ideal in the category of fs log schemes.
Hence, to show the existence of log blow-ups, it suffices to deal with the
case of fine log schemes.

######  Remark 3.6.

As indicated above, the log blow-ups in the category of fs log schemes are
rather similar to normalized blow-ups, i.e., blow-up followed by
normalization. To make clear the distinction, we will call log blow-ups taken
in the category of fs log schemes fs log blow-ups.

The following construction of log blow-ups is due to Kazuya Kato [7, (1.3.3)]
(cf. [11, III.2.6]): We first construct the log blow-up

$\operatorname{Bl}_{K}(P)\longrightarrow\operatorname{S}(P)$

of $\operatorname{S}(P)$, where $P$ is a fine monoid, along the coherent log
ideal $\widetilde{K}$ constructed from an ideal $K\subset P$. Let $I(K)$ be
the ideal of $\mathbb{Z}[P]$ generated by $K$, and consider the natural
morphism
$\operatorname{Proj}\bigoplus_{n}I(K)^{n}\rightarrow\operatorname{Spec}\mathbb{Z}[P]$.
$\operatorname{Proj}\bigoplus_{n}I(K)^{n}$ has the affine open covering

$\operatorname{Proj}\bigoplus_{n}I(K)^{n}=\bigcup_{t\in
K}\operatorname{Spec}\mathbb{Z}[P\langle t^{-1}K\rangle].$

Here, $P\langle E\rangle$ for a subset $E$ of $P^{\mathrm{gp}}$ denotes the
smallest fine submonoid in $P^{\mathrm{gp}}$ that contains $P$ and $E$. The
canonical log structures given by $P\langle
t^{-1}K\rangle\rightarrow\mathbb{Z}[P\langle t^{-1}K\rangle]$ glue to a fine
log structure on $\operatorname{Proj}\bigoplus_{n}I(K)^{n}$. Then it follows
that
$\operatorname{Bl}_{K}(P):=\operatorname{Proj}\bigoplus_{n}I(K)^{n}\rightarrow\operatorname{S}(P)$
gives a log blow-up of $\operatorname{S}(P)$ along $\widetilde{K}$.

To give a more explicit local description, take generators
$t_{0},\ldots,t_{r}$ of $K$. Then $\operatorname{Bl}_{K}(P)$ is the union of
the affine log schemes

$\operatorname{Spec}\mathbb{Z}\big{[}P\big{\langle}\textrm{{\footnotesize$\frac{t_{0}}{t_{i}},\ldots,\frac{t_{r}}{t_{i}}$}}\big{\rangle}\big{]},$

with the log structure induced from $P\langle
t_{0}/t_{i},\ldots,t_{r}/t_{i}\rangle\rightarrow\mathbb{Z}[P\langle
t_{0}/t_{i},\ldots,t_{r}/t_{i}\rangle]$, i.e., the affine log schemes
$\operatorname{S}(P\langle t_{0}/t_{i},\ldots,t_{r}/t_{i}\rangle)$, for
$i=0,\ldots,r$.

Let $X$ be a fine log scheme, and $\mathcal{K}$ a coherent log ideal of $X$.
Suppose there exist a chart $\lambda\colon X\rightarrow\operatorname{S}(P)$
modeled on a fine monoid $P$ and an ideal $K\subset P$ such that
$\mathcal{K}=\widetilde{K}\mathcal{M}_{X}$. Then, by Lemma 3.5,

$\operatorname{Bl}_{\mathcal{K}}(X)=X\times_{\operatorname{S}(P)}\operatorname{Bl}_{K}(P)\longrightarrow
X$

gives a log blow-up of $X$ along $\mathcal{K}$.

In general, we take an étale covering $\\{U_{\alpha}\\}_{\alpha\in L}$ of $X$
such that each $U_{\alpha}$ allow a chart
$U_{\alpha}\rightarrow\operatorname{S}(P_{\alpha})$ with an ideal
$K_{\alpha}\subset P_{\alpha}$ satisfying
$\mathcal{K}|_{U_{\alpha}}=\widetilde{K}_{\alpha}\mathcal{M}_{U_{\alpha}}$.
Then, by the universality of log blow-ups, the local log blow-ups
$\operatorname{Bl}_{\mathcal{K}_{\alpha}}(U_{\alpha})\rightarrow U_{\alpha}$
constructed as above glue to a log blow-up of $X$ along $\mathcal{K}$.

######  Example 3.7.

Let $P$ be a sharp fs monoid, and set $X=\operatorname{S}(P)$. Set
$M=P^{\mathrm{gp}}$ and $N=\operatorname{Hom}_{\mathbb{Z}}(M,\mathbb{Z})$. The
scheme $X$ is an affine toric variety corresponding to the corn $\sigma$ in
$N_{\mathbb{R}}$ such that $\sigma^{\vee}\cap M=P$; i.e., $\sigma$ is the dual
corn of the corn in $M_{\mathbb{R}}$ generated by $P$. Let
$\phi\colon\sigma\rightarrow\mathbb{R}_{\geq 0}$ be a continuous convex
piecewise linear function satisfying the following conditions (cf. [9, p.27]):

  * (a)

$\phi(\lambda x)=\lambda\phi(x)$ for $x\in\sigma$ and
$\lambda\in\mathbb{R}_{\geq 0}$;

  * (b)

$\phi(N\cap\sigma)\subset\mathbb{Z}$.

The function $\phi$ induces an ideal $K_{\phi}$ of $P$ given by

$K_{\phi}=\\{m\in M\mid\langle x,m\rangle\geq\phi(x)\ \textrm{for all}\
x\in\sigma\\}.$

Then the fs log blow-up of $X$ along the log ideal $\widetilde{K}_{\phi}$ is
the normalization of the blow-up of the toric variety $X=X_{\sigma}$ obtained
from the coarsest subdividing fan $\Sigma_{\phi}$ of the cone $\sigma$ such
that $\phi$ is linear on each cone in $\Sigma_{\phi}$; cf. [9, p.31, Theorem
10].

##  4\. Proof of the theorem

######  Lemma 4.1.

Let $P,Q$ be sharp fs monoids, and $h\colon Q\hookrightarrow P$ an injective
homomorphism. Consider the induced morphism $f\colon
X=\operatorname{S}(P)\rightarrow Y=\operatorname{S}(Q)$ of fs log schemes.
Then there exists an ideal $K\subset Q$ such that the following conditions are
satisfied: if

$\textstyle{X^{\prime}\ignorespaces\ignorespaces\ignorespaces\ignorespaces\ignorespaces\ignorespaces\ignorespaces\ignorespaces}$$\scriptstyle{f^{\prime}}$$\textstyle{X\ignorespaces\ignorespaces\ignorespaces\ignorespaces}$$\scriptstyle{f}$$\textstyle{\operatorname{Bl}_{K}(Y)\ignorespaces\ignorespaces\ignorespaces\ignorespaces}$$\textstyle{Y}$

is the fs base change of $f$ by the fs log blow-up
$\operatorname{Bl}_{K}(Y)\rightarrow Y$, then the underlying scheme-theoretic
morphism of $f^{\prime}$ is equidimensional.

Note that $X^{\prime}\rightarrow X$ is isomorphic to the fs log blow-up along
$\widetilde{KP}=\widetilde{K}\mathcal{M}_{X}$, i.e.,
$X^{\prime}\cong\operatorname{Bl}_{KP}(X)$.

###### Proof.

In this proof, we follow the original argument in [5, 3.16] based on the idea
of T. Kajiwara, which we note is similar to the argument in [1, Lemma 4.3].

We use the following notation:

  * •

$M_{Q}=Q^{\mathrm{gp}}$, $M_{P}=P^{\mathrm{gp}}$;

  * •

$N_{Q}=\operatorname{Hom}_{\mathbb{Z}}(M_{Q},\mathbb{Z})$,
$N_{P}=\operatorname{Hom}_{\mathbb{Z}}(M_{P},\mathbb{Z})$;

  * •

$\sigma_{Q}$ $($resp. $\sigma_{P})$ $=$ the cone in $N_{Q}$ $($resp. $N_{P})$
such that $Q=\sigma^{\vee}_{Q}\cap M_{Q}$ $($resp. $P=\sigma^{\vee}_{P}\cap
M_{P})$.

  * •

$\Sigma_{Q}$ (resp. $\Sigma_{P}$) $=$ the fan made up from the faces of the
cone $\sigma_{Q}$ (resp. $\sigma_{P}$).

Note that we have a map $\Phi\colon\Sigma_{P}\rightarrow\Sigma_{Q}$ of fans
that induces the morphism of affine toric schemes
$\operatorname{Spec}\mathbb{Z}[P]\rightarrow\operatorname{Spec}\mathbb{Z}[Q]$.

Consider the subset $\Sigma_{P,1}\subset\Sigma_{P}$ (resp.
$\Sigma_{Q,1}\subset\Sigma_{Q}$) of rays, i.e., cones of dimension $1$. Each
$\rho\in\Sigma_{P,1}$ is mapped by $\Phi$ onto a ray in $\Sigma_{Q}$ or to a
point (i.e. the origin of $N_{Q}$). If $\rho$ is mapped onto a ray, then take
the primitive base $n_{1}\in N_{Q}$ of $\Phi(\rho)$, and extend it to a
$\mathbb{Z}$-base $n_{1},\ldots,n_{r}$ of $N_{Q}$. The $r+1$-rays spanned by
$n_{1},\ldots,n_{r},-(n_{1}+\cdots+n_{r})$ defines the projective $r$-space
$\mathbb{P}^{r}_{\mathbb{Z}}$, and hence the ideal $\mathcal{O}(-1)$ gives
rise to a support function, denoted by $\phi_{\rho}$, i.e., a continuous
convex piecewise linear function $N_{Q,\mathbb{R}}\rightarrow\mathbb{R}_{\geq
0}$ satisfying the conditions (a) and (b) in Example 3.7; we denote the
restriction of $\phi_{\rho}$ onto $\sigma_{Q}$ by the same symbol. If
$\Phi(\rho)$ is a point, then set $\phi_{\rho}=0$. Set

$\phi=\sum_{\rho\in\Sigma_{P,1}}\phi_{\rho},$

and let $\Sigma^{\prime}_{Q}$ be the coarsest fan that subdivides $\Sigma_{Q}$
such that $\phi$ is linear on each cone in $\Sigma^{\prime}_{Q}$. (The author
learned this way of constructing $\phi$ from T. Kajiwara.)

Now, let $K\subset Q$ be an ideal constructed from $\phi$ as in Example 3.7.
Consider the fs log blow-up $\operatorname{Bl}_{K}(Y)\rightarrow Y$, and the
fs base change $f^{\prime}\colon
X^{\prime}:=X\times_{Y}\operatorname{Bl}_{K}(Y)\rightarrow\operatorname{Bl}_{K}(Y)$
of $f$. Then, $X^{\prime}\rightarrow X$ is the log blow-up of $X$ along
$\widetilde{K}\mathcal{M}_{X}$, which is the normalized toric blow-up induced
from the piecewise linear function on $N_{P,\mathbb{R}}$ given by the pull-
back of $\phi$. If we denote the corresponding fan of
$X\times_{Y}\operatorname{Bl}_{K}(Y)$ by $\Sigma^{\prime}_{P}$, then the
induced map
$\Phi^{\prime}\colon\Sigma^{\prime}_{P}\rightarrow\Sigma^{\prime}_{Q}$ maps
each ray onto either a ray or a point (the origin), and hence mapping each
cone onto a cone. Therefore, the morphism $f^{\prime}\colon
X^{\prime}\rightarrow\operatorname{Bl}_{K}(Y)$ is equidimensional by [1, Lemma
4.1]. ∎

###### Proof of Theorem 1.1.

Let $Q=\overline{\mathcal{M}}_{Y,\overline{y}}$, and take an étale local chart
$Y\leftarrow V\rightarrow\operatorname{S}(Q)$ around $\overline{y}$. For any
$x\in X_{V}=X\times_{Y}V$, take an fppf local chart $X_{V}\leftarrow
U_{x}\rightarrow\operatorname{S}(P_{x})$ around $\overline{x}$, where $P_{x}$
is an fs monoid, which extends to a local chart of $f$ as in Lemma 2.4. Since
$f$ is quasi-compact, one can take finitely many $x_{1},\ldots,x_{n}\in X_{V}$
such that $X_{V}$ is covered by the union of $U_{i}:=U_{x_{i}}$ for
$i=1,\ldots,n$. We set $P_{i}=P_{x_{i}}$ for $i=1,\ldots,n$.

For $i=1,\ldots,n$, there exists by Lemma 4.1 an ideal $K_{i}\subset Q$ such
that the fs base change
$\operatorname{Bl}_{K_{i}P_{i}}(P_{i})=\operatorname{S}(P_{i})\times_{\operatorname{S}(Q)}\operatorname{Bl}_{K_{i}}(Q)\rightarrow\operatorname{Bl}_{K_{i}}(Q)$
by the corresponding log blow-up is equidimensional. Set $K=K_{1}\cdots
K_{n}$. Then
$\operatorname{Bl}_{KP_{i}}(P_{i})=\operatorname{S}(P_{i})\times_{\operatorname{S}(Q)}\operatorname{Bl}_{K}(Q)\rightarrow\operatorname{Bl}_{K}(Q)$
is equidimensional for any $i=1,\ldots,n$.

One can further perform a toric blow-up of the toric scheme
$\operatorname{Bl}_{K}(Q)$ so that the resulting toric scheme is smooth over
$\mathbb{Z}$ (cf. [9, p.32, Theorem 11]). Since the composition of fs log
blow-ups is again an fs log blow-up ([10, 4.11]), we may assume that there
exists an ideal $K\subset Q$ such that

  * (a)

the induced morphism

$None$
$\operatorname{Bl}_{KP_{i}}(P_{i})=\operatorname{S}(P_{i})\times_{\operatorname{S}(Q)}\operatorname{Bl}_{K}(Q)\longrightarrow\operatorname{Bl}_{K}(Q)$

is equidimensional for each $i=1,\ldots,n$;

  * (b)

$\operatorname{Bl}_{K}(Q)$ is smooth over $\mathbb{Z}$.

We claim that $(\ast)_{i}$ is integral for each $i=1,\ldots,n$. Since toric
schemes are Cohen-Macaulay, the properties (a) and (b) imply that the
underlying scheme-theoretic morphism of $(\ast)_{i}$ is flat. Thus, for any
cones $\sigma$ from the fan of $\operatorname{Bl}_{K}(Q)$ and $\tau$ from the
fan of $\operatorname{Bl}_{KP_{i}}(P_{i})$ such that $\tau$ is mapped to
$\sigma$, the affine portion of $(\ast)_{i}$

$\operatorname{S}(P^{\prime}_{i})\longrightarrow\operatorname{S}(Q^{\prime})$

where $Q^{\prime}=\sigma^{\prime}\cap M_{Q}$ and
$P^{\prime}_{i}=\tau^{\prime}\cap M_{P}$, is flat, and hence is integral by
Lemma 2.2 (6) and Lemma 2.5 (2). This means $(\ast)_{i}$ is integral.

Now, by Lemma 2.5 (2), we deduce that $U_{i}\rightarrow V$ is integral for any
$i=1,\ldots,n$, and hence $X_{V}\rightarrow V$ is integral. ∎

Let us finally remark that, the argument of the above proofs shows that, if we
start from a toroidal morphism (in the sense as in [1, §1]) $f\colon
X\rightarrow Y$, then, since $f$ is described globally by a morphism of
polyhedral complexes $f_{\Delta}\colon\Delta_{X}\rightarrow\Delta_{Y}$ of K.
Kato’s fans (cf. [8]), one can actually do the above argument globally on $Y$;
cf. [1, 4.4]. Since the only question here lies as to whether one can take a
global log blow-up of $Y$, one can slightly generalize the situation to $Y$
being log regular but without assuming $f$ to be toroidal. This situation has
been treated in [2], which we include here for the reader’s convenience:

######  Proposition 4.2 ([2, 3.6.11]).

Let $f\colon X\rightarrow Y$ be a quasi-compact morphism of fs log schemes,
where $Y$ is log regular. Then there exists an fs log blow-up
$Y^{\prime}:=\operatorname{Bl}_{\mathcal{K}}(Y)\rightarrow Y$ along a coherent
log ideal $\mathcal{K}\subset\mathcal{M}_{Y}$ such that the fs base change
$f_{Y^{\prime}}\colon X_{Y^{\prime}}\rightarrow Y^{\prime}$ is integral.

## References

  * [1] Abramovich, D.; Karu, K.: Weak semistable reduction in characteristic $0$, Invent. Math. 139 (2000), no. 2, 241–273.
  * [2] Abramovich, D.; Temkin, M.; Wlodarczyk, J.: Relative Desingularization and principalization of ideals, preprint, arXiv:2003.03659.
  * [3] Gillam, W.D.: Logarithmic flatness, preprint, arXiv:1601.02422.
  * [4] Illusie, L.; Kato, K.; Nakayama, C.: Quasi-unipotent Logarithmic Riemann-Hilbert Correspondences, J. Math. Sci. Univ. Tokyo 12 (2005), 1–66.
  * [5] Kato, F.: Exactness, integrality, and log modifications, preprint, arXiv:math/9907124.
  * [6] Kato, K.: Logarithmic structures of Fontaine–Illusie, in Algebraic Analysis, Geometry and Number Theory (J.-I. Igusa, ed.). Johns Hopkins Univ., 1988, 191–224.
  * [7] Kato, K.: Logarithmic degeneration and Dieudonne theory, preprint.
  * [8] Kato, K.: Toric singularities, Amer. J. Math. 116 (1994), no. 5, 1073–1099.
  * [9] Kempf, G., Knudsen, F., Mumford, D., Saint-Donat, B.: Toroidal Embeddings I, Lecture Note in Math. 339, Springer-Verlag, Berlin, Heidelberg, New York (1973).
  * [10] Nizioł, W.: Toric singularities: log-blow-ups and global resolutions, Journal of Algebraic Geometry 15 (2006), 1–29.
  * [11] Ogus, A.: Lectures on Logarithmic Algebraic Geometry, Cambridge University Press, Nov. 2018.
  * [12] Olsson, M.C.: Logarithmic geometry and algebraic stacks, Ann. Sci. École Norm. Sup. (4) 36 (2003), no. 5, 747–791.
  * [13] Tsuji, T.: Saturated morphisms of logarithmic schemes, Tunis. J. Math. 1 (2019), no. 2, 185–220.

Department of Mathematics, Tokyo Institute of Technology, 2-12-1 Ookayama,
Meguro, Tokyo 152-8551, Japan (e-mail<EMAIL_ADDRESS>


#  Ftklipse – Design and Implementation of an Extendable Computer Forensics
Environment
Software Requirements Specification Document

Marc-André Laverdière  Serguei A. Mokhov  Suhasini Tsapa  Djamel Benredjem

(April 2006)

##  Chapter 1 Introduction

###  1.1 Purpose

The purpose behind this document is to describe the features of ftklipse, an
extendable platform for computer forensics. This document will explain the
product for the customer, as well as provide a detailed specification for the
developer.

###  1.2 Scope

Ftklipse is a thick-client solution for forensics investigation. It allows to
collect and preserve evidence, to analyze it and to report on it.

It supports chain of custody management, access control policies and batch
operation of its included tools in order to facilitate and accelerate the
investigation. The environment itself and its tools are configurable as well.

###  1.3 Definitions and Acronyms

Cryptographic Hash Function

Function mapping input data of an arbitrary size to a fixed-sized output that
is highly collision resistant.

JVM

The Java Virtual Machine. Program and framework allowing the execution of
program developed using the Java programming language.

GUI

Graphical User Interface.

###  1.4 Compliance

This document was written based on [So98].

##  Chapter 2 Overall Description

###  2.1 Product Perspective

  * •

Ftklipse is meant to be a stand-alone product, depending on a variety of
standard tools organized as plug-ins.

  * •

Ftklipse is meant to be extendable using plug-ins that will add evidence
gathering and analysis properties

  * •

The product has only one interface, a graphical user interface residing on the
client computer

####  2.1.1 System interfaces

The only interface to the system will be its GUI.

####  2.1.2 User Interfaces

Ftlipse implements a user interfaces that is evidence-centric. It offers
wizards for each of its features for ease of use. It allows investigators to
record notes for each piece of evidence as well as to record additional
reporting information. Please refer to Figure 2.1 and Figure 2.2 for an
example of the look and feel of the application.

Figure 2.1: User Interface Showing the Case Introduction Figure 2.2: User
Interface Showing the Evidence Information and Notes

####  2.1.3 Software Interfaces

The product must expose a software interface for plug-in developers to use.
The interfaces provided must allow to:

  * •

Register the plug-in

  * •

Extend the Graphical User Interface’s tool menus (window, pop-up, etc.)

  * •

Offer an interface for the plug-in to implement to allow callbacks enabling
execution

###  2.2 Product Functions

The system will implement the following functionalities:

  * •

Creation of cases

  * •

Evidence Gathering using integrated and plug-in tools

  * •

Evidence Integrity validation using a hash function

  * •

Evidence Import from any media to an existing case

  * •

Logging of all operations performed on the evidence

  * •

Validation of integrity of evidence after each operation over it

  * •

Display of evidence in read-only mode either in ASCII, Unicode or Hex formats

  * •

Recording of investigative notes for each piece of evidence

  * •

Capability to extract a part of the evidence into another file

  * •

Capability to copy and rename the copy of the evidence

  * •

Generation of reports in PDF and LaTeX2e formats that includes listing of the
evidence in the case, a printout of selected parts of the evidence, the
investigative notes related to selected parts of the evidence and a customized
executive summary, introduction, and conclusion. It also integrates the chain
of custody information for each part of the evidence displaying the principal,
time stamp and operation performed on the evidence.

  * •

An extendable set of tools through a plug-in architecture

  * •

Tool-specific defaults and configuration screens

###  2.3 User Characteristics

Users are cyber forensics investigators. They are experienced using existing
sets of tools, and will be trained in the use of ftklipse before its
deployment.

Indirect users are investigators, prosecutors, judges and laypersons, which
will consult the reports generated. They expect reports of high quality which
demonstrate objectivity and methodology.

###  2.4 Constraints

####  2.4.1 Hardware Constraints

Any computer able to operate the Eclipse platform can be used to operate
Ftklipse.

####  2.4.2 Software Constraints

It is assumed that the investigator’s computer supports and includes the
following programs:

  * •

JVM, version 5 or higher

  * •

LaTeX2e, preferably pdflatex

Other tools are not assumed to be present, as they are integrated in each
plug-in.

In the case of using Ftklipse for evidence collection only, only the JVM is
required.

###  2.5 Assumptions and Dependencies

The software assumes a non-hostile environment (i.e. not aiming at disturbing
its operation).

###  2.6 Apportioning of requirements

Some features are to be implemented in later versions of Ftklipse, notably:

  * •

Integration of the Access Control framework with administrator screens

  * •

LaTeXoutput of reports

  * •

Object-specific logging

  * •

Hexadecimal and image display

  * •

Evidence Extraction

##  Chapter 3 Specific Requirements

###  3.1 External Interfaces

The product must expose a software interface for plug-in developers to use.
The interfaces provided must allow to:

  * •

Register the plug-in

  * •

Extend the Graphical User Interface’s tool menus (window, pop-up, etc.)

  * •

Offer an interface for the plug-in to implement to allow callbacks enabling
execution

###  3.2 Functional Requirements

####  3.2.1 Domain Model

Our domain model is a traditional police investigation one, augmented with
some information specific to cyber forensics and our requirements[Deb]. It is
summarized in Figure 3.1.

Figure 3.1: Domain Model for Ftklipse

####  3.2.2 Use Case Model

The use case model for Ftklipse is illustrated in Figure 3.2.

Figure 3.2: Use Case Diagram for Ftklipse

###  3.3 Requirements Description

####  3.3.1 Creation of cases

##### Description

Ftklipse allows the creation of cases with their associated metadata, as
specified in section 3.5.

##### Criticality

This feature is critical to the software

##### Technical Issues

None

##### Dependencies with Other Requirements

None

####  3.3.2 Evidence Gathering

##### Description

Ftklipse allows to run different tools in order to perform evidence collection
on a live system.

##### Criticality

This feature is critical to the software.

##### Technical Issues

The collection of the output of the gathering tool can be problematic,
considering the variety of tools and their working. The redirection of the
tool’s standard input and output in a manner useful to the investigator should
be considered.

##### Dependencies with Other Requirements

None

####  3.3.3 Evidence Analysis

##### Description

Ftklipse allows to run different tools on one or more selected evidences, as
well as to operate a batch analysis. In the latter case, the system must offer
a GUI to the user that allows the selection of the evidence and operations to
perform on it.

##### Criticality

The ability to analyze the evidence is critical. However, the automated
analysis of multiple pieces of evidence is not critical.

##### Technical Issues

The development of a generic programming interface for the variety of analysis
tools is likely to be complex.

##### Dependencies with Other Requirements

None

####  3.3.4 Evidence Integrity Validation

##### Description

Ftklipse records the SHA-1 signature of every piece of evidence and ensures
that the evidence is kept correct during the investigation. In the case of a
corruption of the evidence, Ftklipse detects it and records which operation
caused this corruption.

##### Criticality

This feature is important to the operation of the software, although not
critical.

##### Technical Issues

##### Dependencies with Other Requirements

####  3.3.5 Evidence Import

##### Description

Ftklipse allows to import evidence that was collected outside of itself. The
evidence must be accompanied by a SHA-1 digest that is correct in order to
import the evidence in the system.

##### Criticality

This feature is important, although not critical.

##### Technical Issues

The encoding and format of the SHA-1 signature can vary from one tool to
another.

##### Dependencies with Other Requirements

####  3.3.6 Logging

##### Description

All operations are logged globally by Ftklipse. Furthermore, all operations
related to a given piece of evidence are logged for that evidence
specifically.

##### Criticality

The global logging is critical to Ftklipse. The specific logging is important,
but not essential.

##### Technical Issues

##### Dependencies with Other Requirements

####  3.3.7 Evidence Display

##### Description

The evidence can be visualized, if authorized, in read-only mode either in
ASCII, Unicode or Hex formats. Furthermore, images can be viewed within
Ftklipse and can be opened in an external viewer program.

##### Criticality

This function is critical to the operation of the software in ASCII.

##### Technical Issues

##### Dependencies with Other Requirements

####  3.3.8 Recording of Investigative Notes

##### Description

The investigator must be able to record information regarding each piece of
evidence, as well as report-specific information.

##### Criticality

This function is critical to the operation of Ftklipse.

##### Technical Issues

##### Dependencies with Other Requirements

####  3.3.9 Evidence Extraction

##### Description

The investigator must be able to select a subset of the viewed evidence and
extract it into another file, which will then be treated as evidence itself.
Ftklipse must record this operation and keep relationship information in the
database of evidence.

##### Criticality

This feature is of moderate importance.

##### Technical Issues

##### Dependencies with Other Requirements

####  3.3.10 Evidence Cloning

##### Description

The investigator must be able to copy a piece of evidence in full and
optionally to rename the copy.

##### Criticality

This feature is nice to have.

##### Technical Issues

##### Dependencies with Other Requirements

####  3.3.11 Report Generation

##### Description

The investigator must be able to generate a report for a selected case that
includes all evidence, their notes, as well as other report-specific data. The
output formats can be PDF or LaTeX2e.

##### Criticality

This feature is critical.

##### Technical Issues

##### Dependencies with Other Requirements

####  3.3.12 Plug-in Architecture

##### Description

Ftklipse allows third-party developers to create plug-ins that can be added at
configuration time by system administrators.

##### Criticality

This feature is critical.

##### Technical Issues

##### Dependencies with Other Requirements

####  3.3.13 Access Control Management

##### Description

Ftklipse operates with an access control list for each case, piece of
evidence, and report information. Each user must be authenticated and each
operation must be authorized in the view of the user’s access rights.

Notably, the rights that must be implemented are:

  * •

View rights over a case or piece of evidence. This defines if the user is
authorized to be aware of the existence of a given case or piece of evidence.

  * •

Read rights over a case or piece of evidence. This defines if the user, being
previously granted view rights over the object, is able to read the case’s
information or visualize or operate on a piece of evidence.

  * •

Write rights over a case or piece of evidence. This defines if the user is
authorized to add to the general case notes or the evidence notes. This also
defines if the user is allowed to add evidence to a given case.

By default, Ftklipse must offer default access rights based on the user’s
role, as well as default access rights for different categories of objects.

Ftklipse must provide GUI tools to manage the both user and object rights.

##### Criticality

This feature is important, not critical.

##### Technical Issues

The implementation of the access control algorithm can be complex.
Furthermore, some administration functions (such as the impact of a
redefinition of default rights) require some thought to ensure that no
previously confidential information becomes publicly available.

##### Dependencies with Other Requirements

####  3.3.14 Tool-specific defaults and configuration screens

##### Description

Each tool is responsible to maintain its state, notably regarding its default
settings which must be modifiable by the user and preserved from one run of
ftklipse to another.

Each tool must supply a screen that allows to set the proper parameters before
the operation of the tool.

Default options are to be used on direct invocation of the tool.

##### Criticality

This feature is important

##### Technical Issues

##### Dependencies with Other Requirements

###  3.4 Performance Requirements

Ftklipse does not have any particular performance requirements

###  3.5 Logical Database Requirements

A database is required in order to store the case management and chain of
custody information.

The database must be able to store:

  * •

The relationship between parts of the evidence

  * •

The operations done on the evidence, including its time stamp, its description
and the investigator that performed it.

The information that must be tracked by the database is the following:

  * •

The case’s meta-information (ID, details, description, timestamps,
investigators)

  * •

The case’s evidence.

  * •

The user credentials.

  * •

The object access control lists.

  * •

The chain of custody over every piece of evidence. This includes the
cryptographic hash sums, the operations performed on the evidence and the
principal who performed it.

###  3.6 Design Constraints

The design must take in consideration that the base implementation language is
Java. It also must take in consideration the different options of the tools
that can be plugged into it.

###  3.7 Software System Attributes

In this section, we describe the non-functional attributes of Ftklipse.

####  3.7.1 Security

####  3.7.2 Reliability

The software must behave correctly during 20 continuous hours of operation.

####  3.7.3 Availability

There are no availability constraints.

####  3.7.4 Maintainability

The software must allow for tool plug-ins to be integrated automatically. The
software must also be self-updatable.

####  3.7.5 Portability

The software must operate on POSIX and Windows systems. Tools integrated in
the software must be adjusted accordingly.

## Bibliography

  * [Deb] M. Debbabi.  Course notes from inse 6150.
  * [So98] S. Standards and C. of.  the ieee. ieee recommended practice for software requirements specifications, 1998.

##  Chapter 4 Supporting Information

###### Contents

  1. 1 Introduction
    1. 1.1 Purpose
    2. 1.2 Scope
    3. 1.3 Definitions and Acronyms
    4. 1.4 Compliance
  2. 2 Overall Description
    1. 2.1 Product Perspective
      1. 2.1.1 System interfaces
      2. 2.1.2 User Interfaces
      3. 2.1.3 Software Interfaces
    2. 2.2 Product Functions
    3. 2.3 User Characteristics
    4. 2.4 Constraints
      1. 2.4.1 Hardware Constraints
      2. 2.4.2 Software Constraints
    5. 2.5 Assumptions and Dependencies
    6. 2.6 Apportioning of requirements
  3. 3 Specific Requirements
    1. 3.1 External Interfaces
    2. 3.2 Functional Requirements
      1. 3.2.1 Domain Model
      2. 3.2.2 Use Case Model
    3. 3.3 Requirements Description
      1. 3.3.1 Creation of cases
      2. 3.3.2 Evidence Gathering
      3. 3.3.3 Evidence Analysis
      4. 3.3.4 Evidence Integrity Validation
      5. 3.3.5 Evidence Import
      6. 3.3.6 Logging
      7. 3.3.7 Evidence Display
      8. 3.3.8 Recording of Investigative Notes
      9. 3.3.9 Evidence Extraction
      10. 3.3.10 Evidence Cloning
      11. 3.3.11 Report Generation
      12. 3.3.12 Plug-in Architecture
      13. 3.3.13 Access Control Management
      14. 3.3.14 Tool-specific defaults and configuration screens
    4. 3.4 Performance Requirements
    5. 3.5 Logical Database Requirements
    6. 3.6 Design Constraints
    7. 3.7 Software System Attributes
      1. 3.7.1 Security
      2. 3.7.2 Reliability
      3. 3.7.3 Availability
      4. 3.7.4 Maintainability
      5. 3.7.5 Portability
  4. 4 Supporting Information

###### List of Figures

  1. 2.1 User Interface Showing the Case Introduction
  2. 2.2 User Interface Showing the Evidence Information and Notes
  3. 3.1 Domain Model for Ftklipse
  4. 3.2 Use Case Diagram for Ftklipse