# The age of secrecy and unfairness in recidivism prediction

Cynthia Rudin∗, Caroline Wang† Beau Coker†

Departments of Computer Science, Electrical and Computer Engineering, and
Statistical Science,
Duke University,
LSRC Building, Durham, NC 27708, USA

> 111∗To whom correspondence should be addressed; E-mail<EMAIL_ADDRESS>> Symbol † denotes equal contribution
>
> In our current society, secret algorithms make important decisions about
> individuals. There has been substantial discussion about whether these
> algorithms are unfair to groups of individuals. While noble, this pursuit is
> complex and ultimately stagnating because there is no clear definition of
> fairness and competing definitions are largely incompatible. We argue that
> the focus on the question of fairness is misplaced, as these algorithms fail
> to meet a more important and yet readily obtainable goal: transparency. As a
> result, creators of secret algorithms can provide incomplete or misleading
> descriptions about how their models work, and various other kinds of errors
> can easily go unnoticed. By trying to partially reconstruct the COMPAS model
> — a recidivism-risk scoring model used throughout the criminal justice
> system — we show that it does not seem to depend linearly on the defendant’s
> age, despite statements to the contrary by the model’s creator. This
> observation has not been made before despite many recently published papers
> on this algorithm. Furthermore, by subtracting from COMPAS its
> (hypothesized) nonlinear age component, we show that COMPAS does not
> necessarily depend on race, contradicting ProPublica’s analysis, which
> assumed linearity in age. In other words, faulty assumptions about a
> proprietary algorithm lead to faulty conclusions that go unchecked. Were the
> algorithm transparent in the first place, this likely would not have
> occurred. We demonstrate other issues with definitions of fairness and lack
> of transparency in the context of COMPAS, including that a simple model
> based entirely on a defendant’s age is as ‘unfair’ as COMPAS by ProPublica’s
> chosen definition. We find that there are many defendants with low risk
> score but long criminal histories, suggesting that data inconsistencies
> occur frequently in criminal justice databases. We argue that transparency
> satisfies a different notion of procedural fairness by providing both the
> defendants and the public with the opportunity to scrutinize the methodology
> and calculations behind risk scores for recidivism.

##  1 Introduction

Secret algorithms control important decisions about individuals, such as
judicial bail, parole, sentencing, lending decisions, credit scoring,
marketing, and access to social services. These algorithms may not do what we
think they do, and they may not do what we want.

There have been numerous debates about fairness in the literature, mainly
stemming from a flawed analysis by the ProPublica group (?, ?) of data from
Broward County FL, claiming that the proprietary prediction model COMPAS
(Correctional Offender Management Profiling for Alternative Sanctions) (?) is
racially biased. COMPAS is used throughout the criminal justice system in the
U.S., and its predictions have serious consequences in the lives of many
people (?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?). The bottom line from these debates
is that there is not a single correct definition of fairness and the fact that
multiple types of fairness are incompatible. We put aside typical fairness
considerations for a moment to focus on more pressing issues.

One issue with COMPAS is that it is complicated. It is based on 137 variables
(?) that are collected from a questionnaire. This is a serious problem because
typographical or data entry errors, data integration errors, missing data, and
other types of errors abound when relying on manually-entered data.
Individuals with long criminal histories are sometimes given low COMPAS scores
(which labels them as low risk), and vice versa. In the past, there have been
documented cases where individuals have received incorrect COMPAS scores based
on incorrect criminal history data (?, ?) and have had no mechanism to correct
it after a decision was made based on that incorrect score. This problem has
inadvertently occurred with other (even transparent) scoring systems, at least
in one case leading to the release of a dangerous individual who committed a
murder while on bail (?, ?). An error in a complicated model is much harder to
find than an error in a simple model, and it not clear how many times
typographical errors in complicated models have led to inappropriate releases
that resulted in crimes, after decades of widespread use of these models. The
question of whether calculation errors occur often in these models is of
central importance to the present work.

A separate issue with COMPAS is that it is proprietary, which means its
calculations cannot be double-checked for individual cases, and its
methodology cannot be verified. Furthermore, it is unclear how the data COMPAS
collects contribute to its automated assessments. For instance, while some of
the questions on the COMPAS questionnaire are the same as those in almost
every risk score — age, and number of past crimes committed — other questions
seem to be direct proxies for socioeconomic status, such as “How hard is it
for you to find a job ABOVE minimum wage compared to others?” It is not clear
that such data should be collected or permitted for the purposes in which
these risk scores are used.

Though creators of proprietary algorithms often provide descriptions of how
their models work, by nature, it is difficult for third-parties to verify
these descriptions. This may allow errors in documentation to propagate
unchecked for years. By partially reconstructing COMPAS in Broward County, we
show in Section 2.2 that COMPAS may depend nonlinearly on age, contradicting
its stated methodology. As a result, ProPublica’s conclusion that being
African-American leads to a higher COMPAS score, even controlling for criminal
history and sex, based on a logistic regression would be invalid because the
linearity assumption is violated. While adding a nonlinear age term to the
regression could mitigate the impact of this particular issue, it misses the
larger point. Without transparency, incorrect high-level statements about a
model (e.g., its dependence on age is linear) go unchecked, and this can have
downstream consequences for independent analyses of a model.

While COMPAS depends heavily on age, we show in Sections 2.3 through 2.6 that
it does not seem to depend in such a strong way on either criminal history or
proxies for race. We discuss several possible reasons, but it is possible that
COMPAS depends less on criminal history than we might expect. This leads to
the possibility that COMPAS depends heavily on variables that we may not want
it to depend on.

In Section 3 we pinpoint many individuals whose COMPAS scores seem unusually
low given their criminal histories. Since COMPAS is proprietary, we cannot
fully determine whether these low scores are due to errors in calculation,
data entry errors or errors from some other source (or even if they are errors
at all).

COMPAS’ creator Northpointe disagreed with each of ProPublica’s claims on
racial bias (?) based on their definition of fairness. Their rebuttal did not
include arguments as to the type of fairness we consider here, and in
particular, why it benefits the justice system to use a model that is
complicated or proprietary.

Work in machine learning has shown that complicated, black-box, proprietary
models are not necessary for recidivism risk assessment. Researchers have
shown (on several datasets, including the data from Broward County) that
interpretable models are just as accurate as black box machine learning models
for predicting recidivism (?, ?, ?, ?, ?, ?). These simple models involve age
and counts of past crimes, and indicate that younger people, and those with
longer criminal histories, are more likely to reoffend. A judge could easily
memorize the models within these works, and compute the risk assessments
without even a calculator (?, ?). Despite this knowledge, complicated models
are still being used.

Given that we do not need proprietary models, why we should allow proprietary
models at all? The answer is the same as it is in any other application: by
protecting intellectual property, we incentivize companies to perform research
and development. Since COMPAS has been at the forefront of the fairness debate
about modern machine learning methods, it is easy to forget that COMPAS is not
one of these methods. It is a product of years of painstaking theoretical and
empirical sociological study. For a company like Northpointe to invest the
time and effort into creating such a model, it seems reasonable to afford the
company intellectual property protections. However, as we discussed, machine
learning methods — either standard black-box or, better yet, recently-
developed interpretable ones — can predict equally well or better than bespoke
models like COMPAS (?, ?, ?, ?). For important applications like criminal
justice, academics have always been willing to devote their time and energy.
High-performing predictive models can therefore be created with no cost to the
criminal justice system. Allowing proprietary models to incentivize model
development is not necessary in the first place.

Neglecting to use transparent models has consequences. We provide two
arguments for why transparency should be prioritized over other forms of
fairness. First, no matter which technical definition of fairness one chooses,
it is easier to debate the fairness of a transparent model than a proprietary
model. Transparent algorithms provide defendants and the public with
imperative information about tools used for safety and justice, allowing a
wider audience to participate in the discussion of fairness. Second,
transparency constitutes its own type of procedural fairness that should be
seriously considered (see (?) for a discussion). We argue that it is not fair
that life-changing decisions are made with an error-prone system, without
entitlement to a clear, verifiable, explanation.

In Section 2, we try to partially reconstruct COMPAS for Broward County and
show how it is likely to be inconsistent with its official documentation; in
Section 3, we identify a number of individuals with long criminal histories
but low risk scores; and in Section 4, we describe transparency as a form of
fairness. We consider the most transparent non-trivial predictive model we
could find: age. Younger people tend to be at higher risk of recidivism. Our
goal in this section is to modify the discussion of fairness to be through the
lens of transparency.

##  2 Reconstructing COMPAS

Even with our limited data, we may have been able to partially reconstruct
parts of the COMPAS model as it is implemented in Broward County. We will next
describe these attempts. Note that other groups besides ProPublica have
attempted to explain COMPAS (?, ?). One attempt (?) uses only linear models
(and our evidence suggests that COMPAS’ dependence on age is nonlinear).
Another attempt (?) modeled COMPAS with a generalized additive model, and we
hypothesize that they had the correct model form, based on COMPAS’
documentation. However, their analysis has similar issues to that of
ProPublica or (?): all of these groups attempted to explain COMPAS scores
using ProPublica’s limited set of features, but this type of analysis is not
valid because there are many unmeasured features. COMPAS’s actual dependence
on the observable features may be totally different than what they report. For
instance, if age is correlated with unmeasured survey data, their model would
exploit this correlation to compensate for the missing survey data, leading to
an incorrect estimate of how COMPAS depends on age. Our approach instead
attempts to isolate and subtract off the parts of COMPAS that we think can be
explained from our data, using COMPAS’ documentation to guide us. If our
stated assumptions are correct, our analysis is valid even in the presence of
many unmeasured features. The analysis still holds even if the measured and
unmeasured features are correlated. Of course, ground truth cannot be made
available (except to the designers of COMPAS), so our hypotheses cannot be
verified.

###  2.1 COMPAS as described by its creator

There are two COMPAS recidivism risk assessments of interest: the general
score and the violence score. These scores assess the risk that a defendant
will commit a general or violent crime, and we use them to predict crime
within the next two years. Each score is given as an integer between 1 and 10
but is based on a raw score that can take any value. Higher scores indicate
higher risk. The raw score is computed by a formula and the final integer
score is normalized based on a local population. We will therefore attempt to
reconstruct the raw scores.

To compute the COMPAS raw scores, Northpointe collects 137 variables from a
questionnaire, computes a variety of subscales, and finally linearly combines
the subscales and two age variables — the defendant’s age at the time of the
current offense and age at the time of the first offense — to compute the raw
risk scores. For example, using the equation exactly as written in the COMPAS
documentation, the violent recidivism raw score is given by:

Violent Recidivism Risk Score $\displaystyle=$ (age$*-$w)+(age-at-first-
arrest$*-$w)+(history of violence $*$ w) $\displaystyle+\textrm{(vocation
education $*$ w) + (history of noncompliance $*$ w),}$

where the variables not related to age are subscales and the weights “w” may
be different for each variable. The notation “age$*-$w” would commonly
indicate “age times the negative of w.” We have little knowledge of how the
subscales depend on the questionnaire; the documentation states only which
questionnaire items are used for which subscales. Table 1 shows for each
subscale the recidivism score(s) to which it relates and the number of
underlying questionnaire items we can compute using our data. We use the data
made available by ProPublica222 https://github.com/propublica/compas-analysis;
the Propublica dataset is missing features needed to compute the COMPAS score,
and so we supplement this dataset with probation data from the Broward Clerk’s
Office. However, there remain missing items often related to subjective survey
questions that cannot be computed without access to Northpointe’s data, which
are not publicly available. Notes on our data processing can be found in the
supplement.

Table 1: COMPAS subcales are inputs to the recidivism scores. We have some but not all of the questionnaire features that determine each subscale. For one of the History of Noncompliance features, “Was this person on probation or parole at the time of the current offense?”, we can compute only whether the person was on probation. Subscale | # Features we have | Relevant
---|---|---
| / Total | Recidivism Score
Criminal Involvement | 4/4 | General
History of Noncompliance | 3/5 | Violent
History of Violence | 8/9 | Violent
Vocational/Educational | 0/12 | Both
Substance Abuse | 0/10 | General

According to our understanding of the COMPAS documentation, the general and
violent COMPAS scores are both linear models, where age and age-at-first-
arrest are the only factors that can have negative contributions to the COMPAS
scores.

###  2.2 COMPAS seems to depend nonlinearly on age, contradicting its
documentation

Let us consider if the linear model given by COMPAS is supported by our data.
We make the following assumption:

> Data Assumption: For most values of age, there is at least one person in our
> dataset with age-at-first-arrest equal to their age and the lowest possible
> risk score for each subscale.

First, note that age and age-at-first-arrest are the only COMPAS score inputs
that have negative contribution to the COMPAS scores. Next, note that the
least-risky value for age-at-first-arrest occurs when it is equal to current
age (this implies that the individual has committed only one arrestable
offense). Thus, individuals satisfying the Data Assumption should have the
lowest COMPAS raw score for their age, which is key for the rest of our age
analysis. In what follows, we describe attempts to confirm that these
individuals are present in the data.

First, and most importantly, we used our data to check directly that for most
values of age, there is at least one person in our dataset who has age-at-
first-arrest equal to their age and who does not have criminal history. For
the COMPAS general score, we can check if the Data Assumption holds for the
Criminal Involvement subscale because we have all the inputs to it. We can
only approximately check this assumption for the History of Violence and
History of Noncompliance subscales because we do not have all the inputs.
However, the Data Assumption seems plausible given that (1) the inputs to the
subscales can take only a few integer values333In all but one case, the
subscale inputs are binary or count variables that take up to only 6 values.
The exception is the Criminal Involvement Subscale, which takes the total
number of prior arrests, and for which we have data to directly validate. and
(2) their typical content (e.g., “Do you have a job?” or “How many times has
this person been sentenced to jail for 30 days or more?”) suggests the least-
risky input values are fairly likely. We cannot check the Data Assumption for
the subscales for which we do not have data. However, the Data Assumption
seems to hold for the subscales for which we do have data (see Figure 12 in
the appendix), leading us to believe it might hold generally.

If the COMPAS documentation were correct in that COMPAS is a linear function
of age, then, as long as the Data Assumption holds, if we plot the COMPAS raw
score against age for all people, the lower bound should be a line with slope
equal to the sum of the coefficients on age and age-at-first-arrest, since age
equals age-at-first-arrest. Figure 1 shows this is not the case. Also, the
people near the lower bound of Figure 1 often have no criminal history, and
have age-at-first-arrest equal to age (see the appendix for more analysis).
Thus, our evidence indicates the COMPAS model is not the linear model given
above within the documentation. Recall that except for some typos, there
should be no noise in the COMPAS scores. The COMPAS scores we observe should
be the direct output of a computer program.

Despite the lack of agreement of the data with the documentation, we would
still like to reconstruct as much of the COMPAS black box as possible, so we
make several weaker hypotheses about its form, none of which contradict the
COMPAS documentation:

> Model Assumption 1: The COMPAS raw score is an additive model with respect
> to each input (age, age-at-first-arrest, and the subscales).
>
> Model Assumption 2: The additive terms corresponding to each input except
> age and age-at-first-arrest are never negative.
>
> Model Assumption 3: For the lowest risk score, (a) the additive term
> corresponding to age-at-first-arrest is lowest when age-at-first-arrest is
> as large as possible (i.e., equal to age) and (b) the additive terms
> corresponding to the subscales are zero.

We believe Model Assumption 2 should hold because all of the inputs except age
and age-at-first-arrest (e.g. number of violent felonies, number of times on
probation) should lead to a higher risk of violence or recidivism, and
therefore a higher COMPAS score. Should Model Assumption 3 not hold, people
with nonzero criminal history would have lower COMPAS scores than those with
none, which is not intuitive. With these Model Assumptions and under the Data
Assumption, the lower bound observed in Figure 1 is exactly the additive term
corresponding to age.

Reconstructing COMPAS’s dependence on age is important because we know that
COMPAS, if it is like other scoring systems, should depend heavily on age in
order to predict well. If we could isolate and subtract off its dependence on
age, we could more easily determine its dependence on protected attributes
such as criminal history and race. Based on the lower bound of the COMPAS
score with respect to age, we present a conjecture of approximately how the
COMPAS score may depend on age, at least in Broward County, and we have a
similar conjecture for the violence recidivism counterpart:

Conjecture: The COMPAS general recidivism model is a nonlinear additive model.
Its dependence on age in Broward County is approximately a linear spline,
defined as follows:

$\displaystyle\textrm{for ages }\leq
33.26,f_{\textrm{age}}(\textrm{age})=-0.056\times\textrm{age}-0.179$
$\displaystyle\textrm{for ages between 33.26 and
50.02},f_{\textrm{age}}(\textrm{age})=-0.032\times\textrm{age}-0.963$
$\displaystyle\textrm{for ages $\geq$
50.02},f_{\textrm{age}}(\textrm{age})=-0.021\times\textrm{age}-1.541.$

Similarly, the COMPAS violence recidivism model is a nonlinear additive model,
with a dependence on age that is approximately a linear spline, defined by:

$\displaystyle\textrm{for ages }\leq 21.77,f_{\textrm{viol
age}}(\textrm{age})=-0.205\times\textrm{age}+1.815$ $\displaystyle\textrm{for
ages between 21.77 and 34.58},f_{\textrm{viol
age}}(\textrm{age})=-0.070\times\textrm{age}-1.113$ $\displaystyle\textrm{for
ages between 34.58 and 48.36},f_{\textrm{viol
age}}(\textrm{age})=-0.040\times\textrm{age}-2.166$ $\displaystyle\textrm{ for
ages $\geq$ 48.36},f_{\textrm{viol
age}}(\textrm{age})=-0.025\times\textrm{age}-2.882.$

(a) $f_{\textrm{age}}$

(b) $f_{\textrm{viol age}}$

Figure 1:  Scatter plot of COMPAS general recidivism score versus age and
scatter plot of COMPAS violence recidivism score versus age. Age splines used
to approximate the lower bound of the scatter plots are shown in red; age
outliers that were removed from the analysis are also in red.

We used a two-stage procedure to fit the functions $f_{\textrm{age}}$ and
$f_{\textrm{viol age}}$, with the goal of obtaining the closest approximation
to the true functional relationship between age and the COMPAS raw scores as
possible. First, we fit quadratic and quartic polynomials respectively to the
lower bounds of the scatter plots of individuals’ COMPAS general recidivism
scores and the COMPAS violent recidivism scores. Points more than $c=.05$
below these age polynomials were deemed “age outliers” (a handful of
individuals whose age seems to be incorrect) and removed from the analysis.
Less than ten individuals (for each score) were removed due to this reason.
The age splines were fit using the lower bound of the subsets of individuals
whom we hypothesize to satisfy the data assumption (individuals with age equal
to age-at-first-arrest and no known contribution to any subscale), shown in
Figure 2. In the left figure, several of the age outliers appear as if they
were in a line, but if we add in the age outliers from the full population
(not just those with no criminal history) the apparent line is no longer a
lower bound.

(a) $f_{\textrm{age}}$

(b) $f_{\textrm{viol age}}$

Figure 2: Fitting $f_{\textrm{age}}$ and $f_{\textrm{viol age}}$ to the data.
The data are represented by the black dots and the fitting is shown as the
multicolored line. Points which we deemed as age outliers are shown in red. 92
individuals were used to fit $f_{\textrm{viol age}}$ and 100 individuals were
used to fit $f_{\textrm{age}}$.

As discussed above, COMPAS’ documentation claims a linear dependence on age.
Even if COMPAS constructed the model to only take age as a linear argument,
its predictions, as applied to a sample with representative covariate
information, apparently induce nonlinear dependence on it. For the COMPAS
documentation to claim such linear dependence can thus be misleading.

It is possible that age’s only contribution to the COMPAS general recidivism
score is $f_{\textrm{age}}$ (similarly, $f_{\textrm{viol sage}}$ for the
violence score). Let us describe why this seems to be true. The remainders of
general COMPAS minus $f_{\textrm{age}}$ and violent COMPAS minus
$f_{\textrm{viol age}}$ do not seem to depend on age. After subtracting out
the age polynomials, we employed machine learning methods along with linear
models (Table 2) to model the remainder (COMPAS score minus the age
polynomial). We ran each of these algorithms on the data, once using criminal
history and age features only (with-age models), and once using just criminal
history (without-age models). Machine learning methods are powerful,
nonparametric models that are capable of modeling nonlinear functions very
closely, given the right features. Thus, if the remainder depends on age
coupled with criminal history, it is likely the accuracy will vary between the
with-age and without-age models. However, instead, Tables 2 and 3 (for the
general and violence scores, respectively) show the accuracy of the machine
learning models was almost identical between the with-age and without-age
models.

Importantly, if the dependence on age is additive, COMPAS does not use
features that couple age and criminal history, such as the rate of crimes
committed over time, despite the potential usefulness of this type of feature
(see, e.g., (?)).

| Linear | Random | Boosted | SVM
---|---|---|---|---
| Model | Forest | Dec. Trees |
Without Age | 0.565 | 0.527 | 0.512 | 0.521
With Age | 0.562 | 0.524 | 0.506 | 0.514
Table 2: Root mean square error for several machine learning algorithms for predicting COMPAS score minus age polynomial ($f_{\textrm{age}}$), with age included as a feature (bottom row), and without age (top row). We are trying to determine whether the COMPAS remainder (general COMPAS after subtracting the main age terms) still depends on age. The numbers for “with age” look very similar to the numbers “without age.” Thus, age does not seem to participate in the remainder term because accuracy does not change between the two rows. Race and age at first arrest are included as predictors for both “with age” and “without age” predictions.  | Linear | Random | Boosted | SVM
---|---|---|---|---
| Model | Forest | Dec. Trees |
Without Age | 0.471 | 0.460 | 0.453 | 0.462
With Age | 0.463 | 0.447 | 0.439 | 0.447
Table 3: Analogous to Table 2 but for the COMPAS violence recidivism score,
predicting COMPAS violence score minus $f_{\textrm{viol age}}$. Again, age
does not seem to participate in this remainder.

The fact that the lower bounds $f_{\textrm{age}}$ and $f_{\textrm{viol age}}$
seem to vary smoothly and uniformly with age, with only few outliers,
indicates that the data entering into the COMPAS scores is high quality with
respect to age. This has implications for our later analysis.

Now that we have hypothesized the dependence of COMPAS on age, we wish to
explain its dependence on criminal history variables. We do this separately
for the general and violence scores in Sections 2.3 and 2.4, respectively.

###  2.3 Criminal history and the COMPAS general recidivism score

Unlike the dependence on age, the COMPAS general score does not seem to
display a clear dependence on criminal history. Figure 3 shows a scatter plot
of COMPAS general score remainder (which we define as the COMPAS score minus
the age spline $f_{\textrm{age}}$) against the total number of prior charges,
which is one of the variables determining the Criminal Involvement Subscale
(left panel), and the unweighted sum of the variables in the Criminal
Involvement Subscale (right panel). Note that we would ideally plot the
remainder against the Criminal Involvement Subscale itself, but we do not know
how the inputs are combined to form the subscale. Even excluding the age
outliers (highlighted in green), there is no smooth lower bound as seen in
Figure 1. Therefore we transition from searching for simple dependence of the
COMPAS general score on its subscale items to searching for more complex
dependencies.

(a) COMPAS $-f_{\textrm{age}}$ vs. number of priors. The green points are age
outliers.

(b) COMPAS $-f_{\textrm{age}}$ vs. unweighted sum of Criminal Involvement
Subscale components. The green points are age outliers.

Figure 3: We do not find a clear relationship between the COMPAS general score
after subtracting the age spline and criminal history. Note that in each plot
there are a few observations with a large number of prior charges that are
outside of the plot range. Left: COMPAS $-f_{\textrm{age}}$ vs. number of
priors. The green points are age outliers. Right: COMPAS $-f_{\textrm{age}}$
vs. unweighted sum of Criminal Involvement Subscale components.

To then investigate whether the COMPAS general score depends in a more complex
way on the Criminal Involvement Subscale items listed in the Appendix in Table
12, we ran several machine learning algorithms (random forests (?), boosted
decision trees (?), and support vector machines with a radial basis kernel
function) on the subscale items our data has, to see if the COMPAS general
recidivism score could be explained (either linearly or nonlinearly) by the
subscale components. We tried predicting both the general score itself and the
general score after subtracting $f_{\textrm{age}}$. Figure 4 shows a scatter
plot of predictions versus the actual values for the two prediction problems.
We make two observations from this figure. By comparing the two panels, we can
see that the COMPAS general score seems to depend heavily on age, as the
predictions of the COMPAS score remainder (right panel) are much worse than
the predictions of the COMPAS score itself (left panel); this is because
criminal history is correlated with age. After subtracting our reconstructed
dependence on age (right panel), we see the ability of the criminal history
variables to predict the COMPAS score remainder is surprisingly unsuccessful.
Thus the dependence of the COMPAS general score on criminal history, as
captured by the components of the Criminal Involvement Subscale, seems to be
weak.

(a) Predictions of COMPAS vs. actual values.

(b) Predictions of COMPAS $-f_{\textrm{age}}$ vs. actual values.

Figure 4: Predicting general remainder. Left: Predictions of COMPAS vs. actual
values. Right: Predictions of COMPAS $-f_{\textrm{age}}$ vs. actual values.

###  2.4 Criminal history and the COMPAS violent recidivism score

We gained more traction reconstructing the COMPAS violent recidivism score
than the general score. Figure 5 shows the COMPAS violence score after
subtracting the age spline $f_{\textrm{viol age}}$ against the unweighted sum
of the Violence History Subscale components. Excluding the age outliers, this
subtraction produced a crisp lower bound on the remainder, unlike the bounds
we obtained trying various individual components and weighted sums of the
components. We estimate the dependency on the Violence History Subscale as a
piecewise linear function, which we call $g_{\textrm{viol hist}}$. Next, in
Figure 6, we plot the remainder after also subtracting this dependency on
Violence History (that is, the remainder of the COMPAS violence score after
subtracting both $f_{\textrm{viol age}}$ and $g_{\textrm{viol hist}}$) against
the unweighted sum of the components of the History of Noncompliance Subscale,
on which the violence score should also depend. There is not a sharp lower
bound that is consistent across the horizontal axis, which means this sum, by
itself, is not likely to be an additive term within COMPAS. Therefore, we do
not estimate a dependency on the unweighted sum of items in the History of
Noncompliance Subscale.

Figure 5: COMPAS $-f_{\textrm{age}}$ vs. sum of History of Violence
components. Green points are age outliers. Figure 6: COMPAS
$-f_{\textrm{age}}-g_{\textrm{viol hist}}$ vs. sum of History of Noncompliance
components. Green points are age outliers.

As with the COMPAS general score, we investigate if the COMPAS violence score
depends on its subscale components in a more complex way. Figure 7 shows the
results of three separate prediction problems using all of the components in
the History of Violence and History of Noncompliance subscales. From left to
right, we use gradient boosted trees to predict the COMPAS violence score, the
COMPAS violence score after subtracting $f_{\textrm{viol age}}$, and the
COMPAS violence score after subtracting $f_{\textrm{viol age}}$ and
$g_{\textrm{viol hist}}$. Comparing the panels in Figure 7 from left to right,
we see that the predictions degrade, emphasizing the importance of
$f_{\textrm{viol age}}$ and $g_{\textrm{viol hist}}$, respectively, to the
COMPAS violence score. That is, after subtracting these components, the input
variables are much less able to predict the remaining COMPAS contribution.
Thus, the dependence of the COMPAS violence score on criminal history, as
captured by the Violence History and History of Noncompliance subscales, seems
to be weak.

(a) Predictions of COMPAS vs. actual values.

(b) Predictions of $\text{COMPAS}-f_{\textrm{viol age}}$ vs. actual values.

(c) Predictions of $\text{COMPAS}-f_{\textrm{viol age}}-g_{\textrm{viol
hist}}$ vs. actual values.

Figure 7: Predicting violent remainder. Left: Predictions of COMPAS vs. actual
values. Center: Predictions of $\text{COMPAS}-f_{\textrm{viol age}}$ vs.
actual values. Right: Predictions of $\text{COMPAS}-f_{\textrm{viol
age}}-g_{\textrm{viol hist}}$ vs. actual values.

###  2.5 Caveats

For both the general and violent COMPAS scores, we were unable to capture the
remainder of the COMPAS score (i.e., after subtracting the reconstructed
dependency on age) using the various criminal history variables that
constitute the subscales. There could be several reasons for this, including
the following, among other things:

  * •

It is possible that our data are flawed. We obtained these data from a
combination of ProPublica, the Broward County Sheriff, as well as the Broward
County Clerk’s office. We believe most of these data should be approximately
the same as the data entered into the general COMPAS score. Furthermore, based
on the analysis above, our age data on individuals seems to be high quality,
so there is no a priori reason that the criminal history data would be
substantially lower quality.

It is also possible that the way we calculated the criminal history subscale
items for COMPAS differs from the way the Broward County Sheriff’s Office
calculates them. Our data processing is discussed in the appendix.

  * •

It is possible that we did not hypothesize the correct model form used in
COMPAS; that is, our machine learning models may not have been able to express
the nonlinearities present in COMPAS. While this could be true, we used very
flexible models that should be able to fit (or even overfit) the COMPAS
scores. Thus, we believe this is not a likely explanation.

  * •

It is possible that our data are incomplete. We know this is true, as COMPAS
depends on factors other than criminal history. However, this leads to
questions of what COMPAS can reasonably depend heavily on. Criminal history
data are less noisy and less susceptible to manipulation than other survey
questions; criminal history features do not depend on the survey-taker telling
the truth about the answers. If COMPAS depended more heavily on survey
questions than on criminal history, it could lead precisely to a kind of bias
that we might want to avoid. For instance, if COMPAS did not depend heavily on
the number of prior crimes, it might depend more heavily on socioeconomic
proxies (e.g., “How hard is it for you to find a job ABOVE minimum wage
compared to others?” which is one of several questions on the COMPAS
questionnaire that directly relates to socioeconomic status).

  * •

There is something in the procedure of calculating the COMPAS score that
causes it to be calculated inconsistently. Since we do not know COMPAS, we
cannot check this possibility. In the past, there have been documented cases
where individuals have received incorrect COMPAS scores based on incorrect
criminal history data (?, ?), and no mechanism to correct it after a decision
was made based on that incorrect score. We do not know whether this happens
often enough to influence the scatter plot in a visible way. However, this
type of miscalculation is one of the biggest dangers in the use of proprietary
models. As we know from the calculations of other scoring systems besides
COMPAS, if the number of crimes is not taken into account properly, or if the
scoring system is calculated improperly in other ways, it could lead (and has
led in the past) to unfair denial of parole, and dangerous criminals being
released pre-trial.

Data quality issues, either for us or for the COMPAS score itself, are not
just a problem for our analysis, but a problem that is likely to plague almost
every jurisdiction that uses COMPAS or any other secret algorithm. This is
discussed more in the next section.

###  2.6 Propublica seems to be incorrect in its claims of how COMPAS depends
on race

If age and the components we have of the Criminal Involvement, History of
Noncompliance, and History of Violence subscales can only explain COMPAS
scores to a limited extent, then either the few components of these subscales
that we are missing, or the remaining subscales, Substance Abuse for the
violence score and Vocational/Educational for both scores, must be a large
component of the scores. Reasoning these two subscales are highly correlated
with race, we then tried to model the COMPAS remainders (i.e., after
subtracting the age splines) with race as a feature, in addition to the
available subscale components. Tables 4 and 5, respectively, show the results
of several machine learning methods for predicting the general and violence
score remainders. We see that these features cannot explain the COMPAS
violence score remainders very well. Thus, to conclude, we hypothesize that
COMPAS has at most weak dependence on race after conditioning on age and
criminal history.

| Linear Model | Random Forest | Boosting | SVM
---|---|---|---|---
Without Race | 0.572 | 0.533 | 0.520 | 0.522
With Race | 0.562 | 0.524 | 0.506 | 0.514
Table 4: RMSE of machine learning methods for predicting COMPAS general recidivism raw score after subtracting $f_{\textrm{age}}$ with and without race as a feature. There is little difference with and without race. The differences between algorithms are due to differences in model form. Age at COMPAS screening date and age-at-first-arrest are included as features. | Linear Model | Random Forest | Boosting | SVM
---|---|---|---|---
Without Race | 0.472 | 0.460 | 0.450 | 0.461
With Race | 0.463 | 0.447 | 0.439 | 0.447
Table 5: RMSE of machine learning methods for predicting COMPAS violence
recidivism raw score after subtracting $f_{\textrm{viol age}}$ with and
without race as a feature. Age at COMPAS screening date and age-at-first-
arrest are included as features.

We replicated ProPublica’s finding that a model with race predicts COMPAS
well, but disagree with their conclusions. We repeated ProPublica’s linear
logistic regression on our slightly modified dataset, leading to a model,
provided in the supplementary materials in Table 9, whose race coefficient is
large and significantly different from zero. Coefficients both for age and
race are both large.

There are several flaws in this analysis. First, the linearity assumption is
likely to be wrong, as we know from considering the age analysis above.
Second, the existence of an accurate model that depends on race is not
sufficient to prove that COMPAS depends on race. Race is correlated with both
criminal history and with age in this dataset. Because the linearity
assumption is probably wrong, it is easily possible that race would appear to
be significant, regardless of whether COMPAS is actually using race or its
proxies (aside from criminal history and age) as important variables. As shown
in Tables 4 and 5, including race as a variable to predict COMPAS does not
improve prediction accuracy. That is, for each model we created that uses
race, we found another almost equally accurate model that does not use race.
Thus, it is not clear that race or its proxies (aside from criminal history
and age) are necessarily important factors in COMPAS.

In a separate analysis, Fisher et al. (?) consider all models that approximate
COMPAS with low loss, and among these, find models that depend the most and
the least on race.

##  3 COMPAS sometimes labels individuals with long or serious criminal
histories as low-risk

We examine whether COMPAS scores can be low for individuals who pose a serious
threat. Recently in California (?, ?), a defendant with a long criminal
history was released pre-trial after a criminal history variable was
inadvertently mistyped into a scoring system as being much lower than its true
value. The defendant murdered a bystander before his trial.

Typographical (data-entry) errors are extremely common (?), which means risk-
score errors occur regularly. For instance, if each person’s COMPAS
questionnaire contains 100+ questions, even a 1% error rate could cause
multiple wrong entries on almost every person’s questionnaire. Data entry
errors are a serious problem for medical records (?), and in numerous other
applications. The threat of typographical errors magnifies as the complexity
of the scoring system increases; California had a very simple scoring system,
and still typographical errors have caused serious events discussed above.

In what follows, we use real names and public records found easily on the
Internet, following ProPublica, which is a news organization that compiled
this public database and published real names and pictures of individuals
within their report. It has been long debated whether this kind of information
should be public, with citizen protection being a main argument (e.g.,
background checks for potential employees in sensitive jobs). There is an
irony of this information being public, in contrast to the information
actually used to make high-stakes decisions about these individuals being
secret. However, this public information may allow us to determine that the
secret information is potentially sometimes incorrect.

Consider the case of Desmond Rolle, whose criminal history included
trafficking cocaine and aggravated battery of a pregnant woman, (felony
battery — domestic battery by strangulation). He was given a COMPAS score of 1
(the lowest possible risk). A similar problem occurs for several individuals
in the database. Table LABEL:table:people shows several such individuals who
have COMPAS scores that appear to have been calculated with incorrect inputs,
or whose criminal history information somehow has not been considered within
the COMPAS formula. None of these individuals has scores below the age spline
(none are age outliers).

While it is possible that COMPAS includes mitigating factors (employment,
education, drug treatment) that reduce its score, it seems unlikely that they
would reduce the score all the way to the lowest possible value, but since the
model is not published, we cannot actually determine this. According to (?),
the only negatively weighted factors in COMPAS are age and age at first
arrest, but according to our analysis above, these variables remain
essentially constant with age for older individuals. This indicates there is
no way to reduce a high score that might arise from a lengthy criminal
history. Thus, what we are observing (long criminal histories with a low
COMPAS violence score) should be impossible unless inputs have been entered
incorrectly or omitted from the COMPAS score altogether.

COMPAS general or violence scores do not include the current charges. Thus, in
the case of Martin Owens in the ProPublica database, charged with a serious
crime (kidnapping) but no prior crimes, he still receives the lowest-risk
COMPAS score of 1.

There are many individuals in the database whose COMPAS scores appear to be
unreasonably high; however, it is possible that for those individuals, there
are extra risk factors that cause them to be labeled high risk that are not in
our database (e.g., incomplete criminal history information). Missing
information would be able to explain COMPAS scores that seem too high, but it
cannot explain COMPAS scores that are too low, such as the ones we presented
above in Table LABEL:table:people. Figure 8 shows the predictions of a machine
learning model versus COMPAS score. There are a huge number of individuals
whose COMPAS score is much larger than the machine learning predictions, and
also, there are many individuals for whom the machine learning model (a
boosted decision tree) indicates high risk of recidivism, but the COMPAS score
indicates a lower risk.

Figure 8: Predicted probability of violent recidivism vs. COMPAS violence
decile score. Individuals listed in Table LABEL:table:people are highlighted.

In cases like that of prisoner Glenn Rodrïguez whose parole was denied because
of a miscalculated COMPAS score (?, ?), he did not notice the error on his
COMPAS form until after his parole was denied. Complicated forms, even if one
is asked to check them over, lead to human error. We are certain, for
instance, that there are still errors in this paper, no matter how many times
we have checked it over – and the longer the paper, the more errors it
probably contains.

Glenn Rodrïguez is a proven example of a data input error, but his case
further demonstrates that data input transparency alone is insufficient to
help wronged defendants, the scoring mechanism must also be transparent. Mr.
Rodrïguez was unable to change his parole sentence because he was unable to
demonstrate that the data error impacted his COMPAS score.

##  4 Is age unfair? Fairness through the lens of transparent models

Differences in true/false positive rates do not consider other factors like
age and criminal history. In ProPublica’s regression analysis of the COMPAS
score, ProPublica conditioned on age and criminal history among other factors,
indicating they thought COMPAS would hold to some notion of fairness had it
depended only on age and criminal history. (Otherwise, why condition on those
two factors explicitly?). They used the significance of the coefficient on
race to support their conclusion of bias against blacks. However, they used a
linear term for age and did not handle unmeasured confounding, so this
analysis was faulty. The faulty regression analysis leaves the differences in
true/positive rates as their only remaining evidence of bias. However, the
true/false positive rates are not conditioned on age and criminal history;
that was why they performed the regression analysis, suggesting the regression
could mitigate bias. In other words, Propublica’s second analysis (the
regression model) was invalid because it assumed a linear dependence on age.
Their first analysis (the true/false positive rate analysis) would also then
have been invalid for exactly the reasons why they conducted the second
analysis (which is that they would consider the model fair if COMPAS did not
depend on race when conditioned on age and criminal history).

Age is a well-known determining risk factor for recidivism. Many recidivism
scoring systems depend on age (?, ?, ?, ?, ?, ?, ?, ?, ?) since it has no
direct causal relationship with race (race does not cause age, age does not
cause race), and it is a good predictor of future crime. For adults, the risk
of recidivism decreases with age.444Figure 10 in the appendix plots the
probability of 2-year recidivism (defined by arrest within 2 years) as a
function of age for individuals in Broward County, Florida, showing how it
decreases as a function of age.

On the other hand, in the Broward County data, African-American people tend to
be disproportionately represented at younger ages than Caucasians; the median
age of a COMPAS assessment on an African-American person is 27 years whereas
the median age of a Caucasian is 33 years.555see the supplementary materials
for full distributions This means that more African-Americans will be labeled
as high risk than Caucasians. This also means that more African-Americans will
be mistakenly labeled as high risk than caucasians. It also means that more
Caucasians will be mistakenly labeled as low risk than African-Americans.
Because of the difference in ages between blacks and whites in the dataset,
even models that consider only age and criminal history can be as “unfair” as
COMPAS by ProPublica’s true/false positive rate analysis.

Figure 9 shows the true positive rate (TPR), false positive rate (FPR), true
negative rate (TNR) and false negative rates (FNR) for the model age, which is
defined to be “If age $\leq$ 24, then predict arrest within 2 years, otherwise
predict no arrest.”

Figure 9: Rates for the simple age model and for the COMPAS score. Age appears
also to be unfair.

The figure also shows the rates for the COMPAS general recidivism score. The
data were divided into 10 randomly chosen folds, and the rates are plotted for
all folds, showing a consistent pattern across folds. Indeed, we observe
higher false positive rates for African-Americans, and higher false negative
rates for Caucasians. There is an elevated $\approx$10% higher FPR for
African-Americans than for Caucasians for age, and a $\approx$10% higher FNR
for Caucasians than African-Americans for age. These differing levels
constitute one of the accusations of unfairness by ProPublica, which means
that age is an unfair risk prediction model by this definition. COMPAS seems
to be more “unfair” than age, but as we have seen, it may be possible to
explain this unfairness by a combination of age and other features that differ
between the distributions of African-Americans and Caucasians and have little
to do with the COMPAS score itself. In fact, we also know from (?) that a very
simple model involving age and the number of priors is just as unfair as
COMPAS by this definition.

The many definitions of fairness often contradict with each other. Had
ProPublica considered just the transparent model age (instead of analyzing
COMPAS), it would have been easy to see that the difference in age populations
between African-Americans and Caucasians caused their two notions of fairness
to disagree. In that case, would they still have written an article claiming
that the difference in true and false positive rates meant that COMPAS was
unfair? Or would they have claimed that the use of age was unfair rather than
conditioning on it? Or would they have claimed the data collection process was
unfair? Using a transparent model can sometimes enlighten an understanding of
the debate on fairness.

The consequences of using age in criminal risk assessments are explained
nicely by (?), however, the use of criminal history to assess fairness is
confusing for additional reasons. If we do use criminal history in risk
prediction, since African-Americans tend to have longer criminal histories,
their scores will be worse. On the other hand, if we do not use criminal
history, our risk predictions would be worse. In that case, we could be
releasing dangerous criminals based on poor pre-trial risk assessments, which
leads to poor decisions for the public.

Of course, if we eliminate the most important predictors of recidivism (age
and criminal history) on grounds of leading to unfair models, it is not clear
that any useful predictors of criminal recidivism remain. In that case, we
could be stuck in the situation where a human decision-maker provides non-
transparent, potentially biased decisions.

The point of this exercise is not to determine whether the age model is fair
by any given definition – the age model is transparent, which makes it much
easier to debate, and useful for explaining different possible definitions of
fairness and how they may never intersect. ProPublica’s regression analysis
seems to assume that using age in a risk prediction model is reasonable. But
is age unfair? If we cannot decide on whether the age model is fair, we
certainly cannot decide on whether COMPAS is unfair. However, it is certainly
much easier to debate about the transparent and simple age model than about a
black-box scoring system. While a review of the numerous definitions of
fairness (?, ?) is outside the scope of this work, a potentially easy way to
alter the definition of fairness is to control for non-protected covariates
such as age. In that case, as long as predictions for African-Americans and
Caucasians have equal true/false positive rates for each age group, then the
model would be considered fair. Of course, a problem with this definition is
that any policy that targets young people disproportionately affects African-
Americans.

##  5 Discussion

After attempting to isolate COMPAS’ dependence on age, we were able to
investigate how much COMPAS can depend on criminal history and proxies for
race. We found that it is unlikely that COMPAS depends heavily on either of
them. Machine learning methods for predicting COMPAS scores performed equally
well with or without direct knowledge of race. This seems to contradict
ProPublica’s claims, but ProPublica’s methodological assumptions (at least
about COMPAS depending linearly with age) were wrong, which caused their
conclusions to be faulty.

Northpointe claims the current charge is not helpful for prediction of future
violent offenses (?). (Oddly, they have a separate “Current Violence” scale
that includes the current charges, but which is not claimed to be predictive.)
How much should one weigh the current charges with the COMPAS scores? This is
not clear. Because COMPAS is a black box, it is difficult for practitioners to
combine the current charge (or any other outside information) with the COMPAS
scores. Because the current charges are separate, COMPAS scores are not single
numbers that represents risk. Instead their interpretation has a large degree
of freedom. Could decision-makers fail to realize that the COMPAS score does
not include the current charge? Perhaps this alone could lead to faulty
decision-making.

We showed examples where COMPAS scores can label individuals with long
criminal histories as low-risk. This could easily stem from a lack of
transparency in COMPAS and could lead to dangerous situations for the public.
Even if COMPAS were completely fair, by some reasonable definition of
fairness, this would not stop it from being miscalculated, leading to a form
of procedural unfairness. Since it is known that COMPAS is no more useful for
predicting recidivism than simple, interpretable models, there is no good
reason to continue using complicated, expensive, error-prone proprietary
models for this purpose. There is a mystique behind the use of black box
models for prediction. However, just because a model is a proprietary does not
mean it is any better than a publicly available model (?).

Interestingly, a system that relies only on judges – and does not use machine
learning at all – has similar disadvantages to COMPAS; the thought processes
of judges is (like COMPAS) a black box that provides inconsistent error-prone
decisions. Removing COMPAS from the criminal justice system, without a
transparent alternative, would still leave us with a black box.

Privacy of data should be more of a concern than it presently is. If COMPAS
does not depend heavily on most of the 137 variables, including the proxies
for socioeconomic status, it is not clear if Northpointe is justified in
collecting such private information. COMPAS is a risk and needs assessment,
but is that private information necessary to assess an individual’s needs? All
evidence suggests it does not seem to be necessary for estimating risk.
Determination of needs seems to be a complicated causal question about who
benefits from what types of treatments. This issue is beyond the scope of this
article, but is important. Northpointe’s control over criminal risk scores is
analogous to Equifax’s control over credit scores, and leads to inherent
privacy risks.

Thus, our findings indicate that some form of unfairness caused by COMPAS can
affect almost everyone involved in the justice system: 1) lack of transparency
makes it difficult to assess any of the myriad forms of fairness, leading to
faulty arguments like those of ProPublica. Lack of transparency can hide bias
towards underrepresented groups, or conversely, it can make fair models seem
biased. 2) The unnecessary complexity of COMPAS could cause injustice to those
who had typos in the COMPAS computation, and as a result, were given extra
long sentences or denial of parole. (This is procedural unfairness.) 3) It is
possibly unfair to the taxpayers and judicial system to pay for the collection
of long COMPAS surveys and COMPAS predictions when simpler, transparent,
options are available. (Poor designation of public resources is unfair to
everyone.) 4) The subgroup of people who provided very private personal
information (e.g., about their family history of crime or poverty) to
Northpointe has potentially been wronged. (There is a form of privacy
unfairness in being forced to provide personal information to an entity when
it is unnecessary to do so.)

The problems with COMPAS pertain to many industries. Without community
standards or policy requirements for transparency, business considerations
disincentivize creators of models to disclose their formulas. However, this
lack of transparency is precisely what allows error to propagate and results
in damage to society (?). Merely being able to explain black box models is not
sufficient to resolve this – the models need to be fully transparent, and in
criminal justice, there is no loss in predictive accuracy for using a
transparent model.

### Code

Our code is here: https://github.com/beauCoker/age_of_unfairness

## References

  * 1. J. Larson, S. Mattu, L. Kirchner, J. Angwin, How we analyzed the COMPAS recidivism algorithm, Tech. rep., ProPublica (2016).
  * 2. J. Angwin, J. Larson, S. Mattu, L. Kirchner, Machine bias, Tech. rep., ProPublica (2016).
  * 3. T. Brennan, W. Dieterich, B. Ehret, Evaluating the predictive validity of the COMPAS risk and needs assessment system, Criminal Justice and Behavior 36, 21 (2009).
  * 4. D. Citron, (Un)fairness of risk scores in criminal sentencing, Forbes, Tech section (2016).
  * 5. S. Corbett-Davies, E. Pierson, A. Feller, S. Goel, A computer program used for bail and sentencing decisions was labeled biased against blacks. it’s actually not that clear., Washington Post (op-ed) (2016).
  * 6. unknown, Risk Assessment Project Phase III: Racial Impact Analysis of the Proposed Risk Assessment Scales, Tech. rep., Pennsylvania Commission on Sentencing (2018).
  * 7. B. Netter, Using group statistics to sentence individual criminals: an ethical and statistical critique of the virginia risk assessment program, The Journal of Criminal Law and Criminology pp. 699–729 (2007).
  * 8. C. T. Lowenkamp, E. J. Latessa, Understanding the risk principle: How and why correctional interventions can harm low-risk offenders, Topics in community corrections 2004, 3 (2004).
  * 9. S. D. Gottfredson, G. R. Jarjoura, Race, gender, and guidelines-based decision making, Journal of Research in Crime and Delinquency 33, 49 (1996).
  * 10. R. E. Redding, Evidence-based sentencing: the science of sentencing policy and practice, Chap. J. Crim. Just. 1, 1 (2009).
  * 11. S. Baradaran, Race, prediction, and discretion, Geo. Wash. L. Rev. 81, 157 (2013).
  * 12. J. Petersilia, S. Turner, Guideline-based justice: Prediction and racial minorities, Crime & Justice 9, 151 (1987).
  * 13. M. S. Crow, The complexities of prior record, race, ethnicity, and policy: Interactive effects in sentencing, Criminal Justice Review (2008).
  * 14. A. W. Flores, C. T. Lowenkamp, K. Bechtel, False positives, false negatives, and false analyses: A rejoinder to “Machine bias: There’s software used across the country to predict future criminals”, Federal probation 80 (2016).
  * 15. Northpointe Inc., Measurement & treatment implications of COMPAS core scales, Tech. rep. (2009).
  * 16. R. Wexler, When a computer program keeps you in jail: How computers are harming criminal justice, New York Times (2017).
  * 17. R. Wexler, Code of silence: How private companies hide flaws in the software that governments use to decide who goes to prison and who gets out, Washington Monthly (2017).
  * 18. E. Westervelt, Did a bail reform algorithm contribute to this San Francisco man’s murder?, NPR, August 8 (2017).
  * 19. V. Ho, Miscalculated score said to be behind release of alleged twin peaks killer, SFGate (San Francisco Chronicle) (2017).
  * 20. W. Dieterich, C. Mendoza, T. Brennan, COMPAS risk scales: Demonstrating accuracy equity and predictive parity: Performance of the COMPAS risk scales in broward county (2016).
  * 21. J. Zeng, B. Ustun, C. Rudin, Interpretable classification models for recidivism prediction, Journal of the Royal Statistical Society: Series A (Statistics in Society) 180, 689 (2017).
  * 22. N. Tollenaar, P. van der Heijden, Which method predicts recidivism best?: a comparison of statistical, machine learning and data mining predictive models, Journal of the Royal Statistical Society: Series A (Statistics in Society) 176, 565 (2013).
  * 23. E. Angelino, N. Larus-Stone, D. Alabi, M. Seltzer, C. Rudin, Learning certifiably optimal rule lists for categorical data, ACM SIGKDD International Conference on Knowledge Discovery and Data Mining (KDD) (2017).
  * 24. E. Angelino, N. Larus-Stone, D. Alabi, M. Seltzer, C. Rudin, Certifiably optimal rule lists for categorical data, Journal of Machine Learning Research 19, 1 (2018).
  * 25. C. Rudin, B. Ustun, Optimized scoring systems: Toward trust in machine learning for healthcare and criminal justice, Interfaces 48, 399 (2018). Special Issue: 2017 Daniel H. Wagner Prize for Excellence in Operations Research Practice September-October 2018.
  * 26. B. Ustun, C. Rudin, Learning optimized risk scores from large-scale datasets, Journal of Machine Learning Research (2019). Accepted.
  * 27. C. Coglianese, D. Lehr, Transparency and algorithmic governance, Tech. rep., Working Paper. Penn Program on Regulation, University of Pennsylvania Law School (2018).
  * 28. S. Tan, R. Caruana, G. Hooker, Y. Lou, Distill-and-compare: Auditing black-box models using transparent model distillation, AAAI/ACM Conference on AI, Ethics, and Society (2018).
  * 29. M. T. Stevenson, C. Slobogin, Algorithmic risk assessments and the double-edged sword of youth, Tech. Rep. Vanderbilt Law Research Paper No. 18-36 (2018). Available at SSRN: https://ssrn.com/abstract=3225350 or http://dx.doi.org/10.2139/ssrn.3225350.
  * 30. S. D. Bushway, A. M. Piehl, The inextricable link between age and criminal history in sentencing, Crime & Delinquency 53, 156 (2007).
  * 31. A. Liaw, M. Wiener, Classification and regression by randomforest, R News 2, 18 (2002).
  * 32. T. Chen, C. Guestrin, Xgboost: A scalable tree boosting system, 22nd SIGKDD Conference on Knowledge Discovery and Data Mining (2016).
  * 33. A. Fisher, C. Rudin, F. Dominici, Assessing a variable’s importance by studying many prediction models simultaneously, arXiv:1801.01489 [stat.ME] (2018).
  * 34. S. D. Bushway, E. G. Owens, A. M. Piehl, Sentencing guidelines and judicial discretion: Quasi-experimental evidence from human calculation errors, Journal of Empirical Legal Studies 9 (2012).
  * 35. M. M. Wahi, D. V. Parks, R. C. Skeate, S. B. Goldin, Reducing errors from the electronic transcription of data collected on paper forms: A research data case study, J Am Med Inform Assoc. 15, 386 (2008).
  * 36. Northpointe, Practitioner’s Guide to COMPAS Core, Tech. rep., Northpointe (2013).
  * 37. Pennsylvania Commission on Sentencing, Risk Assessment Project Phase III: The Development and Validation of the Proposed Risk Assessment Scales (2012).
  * 38. M. Nafekh, L. L. Motiuk, The Statistical Information on Recidivism, Revised 1 (SIR-R1) Scale: A Psychometric Examination (Correctional Service of Canada. Research Branch, 2002).
  * 39. P. Howard, B. Francis, K. Soothill, L. Humphreys, OGRS 3: The revised offender group reconviction scale, Tech. rep., Ministry of Justice (2009).
  * 40. L. Helmus, D. Thornton, R. K. Hanson, K. M. Babchishin, Improving the predictive accuracy of static- 99 and static-2002 with older sex offenders: Revised age weights, Sexual Abuse: A Journal Of Research & Treatment 24, 64 (2012).
  * 41. C. M. Langton, et al., Actuarial assessment of risk for reoffense among adult sex offenders evaluating the predictive accuracy of the static-2002 and five other instruments, Criminal Justice and Behavior 34, 37 (2007).
  * 42. G. C. Barnes, J. M. Hyatt, Classifying adult probationers by forecasting future offending, Tech. rep., National Institute of Justice, U.S. Department of Justice (2012).
  * 43. P. B. Hoffman, S. Adelberg, The salient factor score: A nontechnical overview, Fed. Probation 44, 44 (1980).
  * 44. S. Turner, J. Hess, J. Jannetta, Development of the California Static Risk Assessment Instrument (CSRA), University of California, Irvine, Center for Evidence-Based Corrections (2009).
  * 45. K. Patrick, Should an algorithm determine whether a criminal gets bail?, Inside Sources (2018).
  * 46. A. Fairness, T. in Machine Learning (FATML), https://www.fatml.org (2018). [Online; accessed 10-June-2018].
  * 47. R. Berk, H. Heidari, S. Jabbari, M. Kearns, A. Roth, Fairness in criminal justice risk assessments: The state of the art, Sociological Methods & Research (2017).
  * 48. C. Rudin, Stop explaining black box machine learning models for high stakes decisions and use interpretable models instead, Nature Machine Intelligence 1, 206 (2019).

Table 6: Individuals whose COMPAS violence decile score is low (low-risk), but who have significant criminal histories. # Priors counts the prior charges up to but not including the current offense (i.e., the offense we believe triggered the COMPAS score calculation), since this is how COMPAS counts prior offenses. However, the current offense may be included in Selected Prior Charges. Note that any Subsequent Crimes beyond the 2 year mark of the COMPAS score calculation or outside of Broward County may not be contained in our database. The notation “(F/M,N)” next to each charge gives the charge degree (F=Felony or M=Misdemeanor) and the number of instances of this charge (N).  Name |  | COMPAS
---
Violence Decile
| #
---
Priors
| Selected
---
Prior Charges
| Selected
---
Subsequent Charges
| Vilma
---
Dieppa
1 | 4 |  | Aggravated Battery (F,1),
---
Child Abuse (F,1),
Resist Officer w/Violence (F,1)
| David
---
Selzer
1 | 14 |  | Battery on Law Enforc Officer (F,3),
---
Aggravated Assault W/Dead Weap (F,1),
Aggravated Battery (F,1),
Resist/obstruct Officer W/viol (F,1)
| Berry
---
Sanders
1 | 15 |  | Attempted Murder 1st Degree (F,1),
---
Resist/obstruct Officer W/viol (F,1),
Agg Battery Grt/Bod/Harm (F,1),
Carrying Concealed Firearm (F,1)
| Armed Sex Batt/vict
---
12 Yrs + (F,2), Aggravated
Assault W/dead Weap (F,3),
Kidnapping (F,1)
| Fernando
---
Walker
1 | 22 |  | Aggrav Battery w/Deadly Weapon (F,1),
---
Driving Under The Influence (M,2),
Carrying Concealed Firearm (F,1)
| Steven
---
Glover
1 | 28 |  | Robbery / Deadly Weapon (F,11),
---
Poss Firearm Commission Felony (F,7)
| Rufus
---
Jackson
1 | 40 |  | Resist/obstruct Officer W/viol (F,3),
---
Battery on Law Enforc Officer (F,2),
Attempted Robbery Deadly Weapo (F,1),
Robbery 1 / Deadly Weapon (F,1)
| Miguel
---
Gonzalez
2 | 6 |  | Murder in the First Degree (F,1),
---
Aggrav Battery w/Deadly Weapon (F,1),
Carrying Concealed Firearm (F,1)
| William
---
Kelly
2 | 17 |  | Aggravated Assault (F,5),
---
Aggravated Assault W/dead Weap (F,2),
Shoot/throw Into Vehicle (F,2),
Battery Upon Detainee (F,1)
| Richard
---
Campbell
2 | 21 |  | Armed Trafficking In Cocaine (F,1),
---
Poss Weapon Commission Felony (F,1),
Carrying Concealed Firearm (F,1)
| John
---
Coleman
2 | 25 |  | Attempt Murder in the First Degree (F,1),
---
Carrying Concealed Firearm (F,1),
Felon in Pos of Firearm or Amm (F,1)
| Oscar
---
Pope
2 | 38 |  | Aggravated Battery (F,3),
---
Robbery / Deadly Weapon (F,3),
Kidnapping (F,1),
Carrying Concealed Firearm (F,2)
| Grand Theft in the
---
3rd Degree (F,3)
| Travis
---
Spencer
3 | 16 |  | Aggravated Assault W/dead Weap (F,1),
---
Burglary Damage Property>$1000 (F,1),
Burglary Unoccupied Dwelling (F,1)
| Michael
---
Avila
3 | 17 |  | Aggravated Assault W/dead Weap (F,2),
---
Aggravated Assault w/Firearm (F,2),
Discharge Firearm From Vehicle (F,1),
Home Invasion Robbery (F,1)
| Fail Register
---
Vehicle (M,2)
| Terrance
---
Murphy
3 | 20 |  | Solicit to Commit Armed Robbery (F,1),
---
Armed False Imprisonment (F,1),
Home Invasion Robbery (F,1)
| Driving While
---
License Revoked (F,3)
| Anthony
---
Hawthorne
3 | 25 |  | Attempt Sexual Batt / Vict 12+ (F,1),
---
Resist/obstruct Officer W/viol (F,1),
Poss Firearm W/alter/remov Id# (F,1)
| Stephen
---
Brown
3 | 36 |  | Carrying Concealed Firearm (F,2),
---
Battery On Law Enforce Officer (F,1),
Kidnapping (F,1),
Aggravated Battery (F,1)
| Driving While
---
License Revoked (F,3)
| Samuel
---
Walker
3 | 36 |  | Murder in the First Degree (F,1),
---
Poss Firearm Commission Felony (F,1),
Solicit to Commit Armed Robbery (F,1)
|  Petit Theft $100-$300
---
(M,1)
| Jesse
---
Bernstein
4 | 10 |  | Aggravated Battery / Pregnant (F,1),
---
Sex Battery Vict Mental Defect (F,1),
Shoot/throw In Occupied Dwell (F,1)
| Tresspass in Struct/Convey
---
Occupy (M,1)
| Shandedra
---
Hardy
4 | 16 |  | Aggrav Battery w/Deadly Weapon (F,1),
---
Felon in Pos of Firearm or Amm (F,4)
| Resist/Obstruct W/O
---
Violence (M,1), Possess
Drug Paraphernalia (M,1)

##  6 Acknowledgments

We acknowledge partial funding from the Laura and John Arnold Foundation.
Thank you to the Broward County Sheriff’s office.

## Supplementary materials

### Supporting figures and tables

#### Probability of recidivism as a function of age

Figure 10 shows the probability of a new charge within 2 years as a function
of age. The probability is a decreasing function of age.

Figure 10: Probability of charge within 2 years as a function of age (red).
The blue scatter plot is useful for understanding the distribution of ages of
individuals; each individual who was arrested has a dot at their age on the
horizontal axis, and a “1” on the vertical axis.

#### Age histograms

Figure 11 shows the normalized histograms of African-Americans and Caucasians
within Broward County who were evaluated with COMPAS between the beginning of
2013 and the end of 2014. These histograms do not involve COMPAS scores
themselves, only information about the set of individuals who received COMPAS
scores. The histogram for African-Americans is skewed to the left, which means
African-Americans tend to be younger on average when their COMPAS score is
calculated in the Broward County dataset.

Figure 11: Normalized histograms of age for African-Americans and Caucasians
within the Broward County dataset.

#### Predictions of recidivism with and without race

Tables 7 and 8 show predictions of recidivism and violent recidivism,
respectively, with and without race as a feature. The results are very similar
with and without race.

| Logistic Regression | Random Forest | Boosting | SVM
---|---|---|---|---
Without Race | 0.330 | 0.321 | 0.312 | 0.298
With Race | 0.328 | 0.318 | 0.309 | 0.313
Table 7: Misclassification error of machine learning methods for predicting general recidivism with and without race as a feature. Age at COMPAS screening date and age-at-first-arrest are included as features. Unlike when predicting the COMPAS raw score remainder, we include the current offense in criminal history features. | Logistic Regression | Random Forest | Boosting | SVM
---|---|---|---|---
Without Race | 0.159 | 0.165 | 0.158 | 0.160
With Race | 0.159 | 0.161 | 0.160 | 0.160
Table 8: Misclassification error of machine learning methods for predicting
violent recidivism with and without race as a feature. Age at COMPAS screening
date and age-at-first-arrest are included as features. Unlike when predicting
the COMPAS raw score remainder, we include the current offense in criminal
history features.

### Fitting $f_{\textrm{age}}$ and $f_{\textrm{viol age}}$

#### Checking Data Assumptions

As explained in Section 2.2, we make the assumption that there exist
individuals in our data with the lowest possible COMPAS score for each age. We
do not have all the inputs to the COMPAS score, but if the assumption holds
for the inputs we have access to, it lends evidence that the Data Assumption
holds generally. In Figure 12, we show the counts of individuals who have
current age equal to age-at-first-arrest and have all zero values for the
COMPAS subscale inputs within our data.

(a)

(b)

Figure 12:  Age vs the count of individuals who satisfy the following
conditions: (1) Age equals age-at-first-arrest, (2) All zeros for the subscale
inputs in our data, and that correspond to the particular COMPAS score we
consider (i.e. Criminal Involvement Subscale for the general score; History of
Violence Subscale and History of Noncompliance Subscale for the violence
score).

#### age-at-first-arrest

The COMPAS lower bounds $f_{\textrm{age}}$ and $f_{\textrm{viol age}}$ are
defined by the current age at which the COMPAS score is computed, not the age-
at-first-arrest. We chose to begin the analysis with current age because (1)
the relationship between current age and the COMPAS score is the clearest in
the data, and (2) according to the COMPAS documentation, the age variables are
the only variables that have a linear relationship with the score. Figure 13
shows that there is not a smooth lower bound for COMPAS vs. age-at-first-
arrest as there was with the current age.

(a)

(b)

Figure 13:  COMPAS raw score versus the age variables. age-at-first-arrest is
depicted in red, while current age is depicted in blue and jittered to the
right. There is no clear pattern for the lower bound with age-at-first-arrest,
as there is for current age. Thus, age-at-first-arrest is not used to define
an initial lower bound.

#### Individuals on the lower bound having the lowest possible COMPAS scores
for their age

The functions $f_{\textrm{age}}$ and $f_{\textrm{viol age}}$ are defined by
individuals who have zero values for the subscale components we can compute,
and lie close to the age spline (i.e. were not deemed age outliers)—in other
words, the individuals who could satisfy the Data Assumption. While there is
no way for us to know if these individuals truly satisfy the Data Assumption,
in this section we explain why we believe these individuals actually do
satisfy the assumption.

Figure 14 shows raw COMPAS scores for these individuals who define the age
splines. For many ages, there are several individuals whose COMPAS raw scores
have identical values. The number of such individuals for each age is shown by
the size of the ball in the figure. Their current age is usually equal to
their age-at-first-arrest. Figure 15 shows this, where individuals with
current age equal to age-at-first-arrest are plotted in blue, and others are
in red.

(a) Lowest violent COMPAS raw score for each age, for individuals with no
criminal history and no history of non-compliance. The smallest balls
represent only one individual, the larger balls represent 7 individuals with
identical COMPAS raw scores. 83 individuals are represented in this plot.

(b) Lowest general COMPAS raw score for each age, for individuals with no
criminal involvement. The smallest balls represent only one individual, the
larger balls represent 6 individuals with identical COMPAS raw scores. 103
individuals are represented in this plot.

Figure 14:

(a)

(b)

Figure 15:  COMPAS raw score versus current age. Individuals such that current
age is equal to age-at-first-arrest are shown in blue, whereas others for
which current age is not equal to age-at-first-arrest are shown in red. The
balls are semi-transparent, so purple balls consist of overlapping blue and
red balls. We define the lower boundary for the Violence Score using
individuals that have the additional constraint that they cannot have violence
history or non-compliance history; for the General Score, we use the
constraint that individuals cannot have criminal involvement history. Note
that individuals on the lower bound are mostly blue—that is, individuals have
age equal to age at first arrest. Others tend to have higher COMPAS scores.

It is possible that these individuals have nonzero values for the unobserved
subscale components, which would imply that the true age functions could lie
below $f_{\textrm{age}}$ and $f_{\textrm{viol age}}$. However, we believe that
Figure 14 – which shows that that for many age values there are multiple
individuals with exactly the same COMPAS raw score on the lower bound of our
data – combined with the smoothness of the lower bound, provides compelling
evidence that all these individuals actually have zero values for the
unobserved subscale components. Some alternative hypotheses and the reasons
why we find them unlikely, are outlined below:

  * •

Some individuals on the lower bound have zeros for unobserved subscale
components, others have nonzero values for the unobserved subscale components.
We find this unlikely for two reasons. Since age and age-at-first-arrest are
the only inputs with negative contribution to the COMPAS score, if we fix age,
COMPAS scores for the individuals with positive values for unobserved subscale
components should be higher than the lowest possible COMPAS score. Thus, for
these individual to lie on the lower bound we observe, there cannot be any
individuals in the data who have the lowest possible COMPAS score for that
age. Second, this would have to occur in a systematic way, to create the
smooth, nonlinear lower bound we observe.

  * •

All individuals on the lower bound have nonzero values for unobserved inputs
because some of these inputs contribute negatively to the COMPAS score,
thereby negating the positive contribution from other inputs, allowing the
individual to have the lowest possible COMPAS score. We find this unlikely
because according to the COMPAS documentation, age and age-at-first-arrest are
the only variables that contribute negatively to the COMPAS score. As
discussed above, age-at-first-arrest equals age for most of the individuals on
the lower bound, so there is no contribution from age-at-first-arrest. Though
we claim the COMPAS documentation is not completely accurate, we believe it is
reliable in this respect because it would be unintuitive for the other inputs
to contribute negatively to the COMPAS scores.

  * •

Some individuals on the lower bound have nonzero values for unobserved inputs,
but manual overrides occurred for all of these individuals so that the only
contribution to their score was age. This is possible, but implies that the
only factor influencing their COMPAS scores was age, which then agrees with
our hypothesis.

For these reasons, we believe that these individuals have the lowest possible
COMPAS scores for their age and lie on the true age functions.

### An alternative explanation of the nonlinearity in age

In this section, we briefly consider an alternative explanation for the
nonlinearity in age. As observed in Section 4, there are a greater proportion
of individuals at lower ages than at higher ages in our data. It has been
proposed that if age follows a Gaussian distribution with respect to the
COMPAS score, it is possible that the nonlinear behavior we observe in the
lower bound of individuals’ COMPAS raw scores vs current age is induced by
more extreme values appearing when we have more samples. However, exploratory
analysis mitigates this concern.

Figure 16 shows the number of individuals per age group. As Figure 17 shows,
sampling $\min(150,\text{\text{number of individuals per age}})$ for each age
group does not alter the nonlinearity of the lower bound. Moreover, age
splines fit using only the sampled data are very close to the original age
splines.

(a)

(b)

Figure 16:

(a)

(b)

Figure 17:  COMPAS raw score versus the age variables. The initial age
polynomials fitted using the complete data are in red. The age polynomials
fitted using a sample of $n=\min(150,\text{{number individuals per age}})$ are
in green. The two bounds are extremely close, mitigating the concern that the
nonlinearity is due to the number of observations per age group.

### Logistic Regression

We attempt to replicate ProPublica’s logistic regression of the COMPAS score
category (Medium or High Risk vs. Low Risk) on various features, including
race. Coefficient estimates and standard errors are shown in Table 9. Since
recidivism (the outcome for our recidivism prediction models) is used as a
covariate in ProPublica’s analysis, we exclude any observation for which there
is less than two years of data beyond the screening date. Note that if 2-year
recidivism is used in ProPublica’s model, it is using information that by
definition is not available at the time that the COMPAS score is calculated.

| Our Results |  | ProPublica’s Results
---|---|---|---
| Estimate | Standard Error |  | Estimate | Standard Error
Female | 0.123 | 0.085 |  | 0.221*** | 0.080
Age: Greater than 45 | -1.489*** | 0.129 |  | -1.356*** | 0.099
Age: Less than 25 | 1.445*** | 0.071 |  | 1.308*** | 0.076
Black | 0.521*** | 0.072 |  | 0.477*** | 0.069
Asian | -0.271 | 0.503 |  | -0.254*** | 0.478
Hispanic | -0.301* | 0.130 |  | -0.428*** | 0.128
Native American | 0.390 | 0.678 |  | 1.394* | 0.766
Other | -0.713*** | 0.159 |  | -0.826*** | 0.162
Number of Priors | 0.155*** | 0.006 |  | 0.269*** | 0.011
Misdemeanor | -0.464*** | 0.069 |  | -0.311*** | 0.067
Two year Recidivism | 0.491*** | 0.068 |  | 0.686*** | 0.064
Constant | -1.593*** | 0.082 |  | -1.526*** | 0.079
Table 9: Logistic regression coefficient estimates and standard errors
computed in a similar way to ProPublica. The significance levels are not
valid, since the model assumptions of linearity are broken. *$p<0.1$;
**$p<0.05$; ***$p<0.01$. Our results are based on 5759 observations while
ProPublica’s results are based on 6,172 observations.

### Data processing

Our data includes the same raw data collected by ProPublica, which includes
COMPAS scores for all individuals who were scored in 2013 and 2014, obtained
from the Broward County Sheriff’s Office. There are 18,610 individuals, but we
follow ProPublica in examining only the 11,757 of these records which were
assessed at the pretrial stage. We also used public criminal records from the
Broward County Clerk’s Office to obtain the events/documents and disposition
for each case, which we used in our analysis to infer probation events.

In their analysis (?, ?), ProPublica processed the raw data, which includes
charge, arrest, prison, and jail information, into features aggregated by
person, like the number of priors or whether or not a new charge occurred
within two years. We too process the raw data into features, partly to ensure
the quality of the features and partly to create new features as defined by
the components of the COMPAS subscales (see Tables 10-14). Note that while
ProPublica publishes the code for their analysis and the raw data, they do not
publish the code for processing the raw data. Thus we did that from scratch.

The screening date is the date on which the COMPAS score was calculated.

  * •

Our features correspond to an individual on a particular screening date. If a
person has multiple screening dates, we compute the features for each
screening date, such that the set of events for calculating features for
earlier screening dates is included in the set of events for later screening
dates.

  * •

On occasion, an individual will have multiple COMPAS scores calculated on the
same date. There appears to be no information distinguishing these scores
other than their identification number. We take the scores with the larger
identification number.

  * •

Any charge with degree “(0)” seems to be a very minor offense, so we exclude
these charges. All other charge degrees are included, meaning charge degrees
other than felonies and misdemeanors are included.

  * •

Some components of the Violence Subscale require classifying the type of each
offense (e.g., whether or not it is a weapons offense). We infer this from the
statute number, most of which correspond to statute numbers from the Florida
state crime code.

  * •

The raw data includes arrest data as well as charge data. Because the arrest
data does not include the statute, which is necessary for the Violence
Subscale, we use the charge data and not the arrest data throughout the
analysis. While the COMPAS subscales appear to be based on arrest data, we
believe the charge data should provide similar results.

  * •

For each person on each COMPAS screening date, we identify the offense — which
we call the current offense — that we believe triggered the COMPAS screening.
The current offense date is the date of the most recent charge that occurred
on or before the COMPAS screening date. Any charge that occurred on the
current offense date is part of the current offense. In some cases, there is
no prior charge that occurred near the COMPAS screening date, suggesting
charges may be missing from the dataset. For this reason we consider charges
that occurred within 30 days of the screening date for computing the current
offense. If there are no charges in this range, we say the current offense is
missing. For any part of our analysis that requires criminal history, we
exclude observations with missing current offenses. All components of the
COMPAS subscales that we compute are based on data that occurred prior to (not
including) the current offense date, which is consistent with how the COMPAS
score is calculated according to (?).

  * •

The events/documents data includes a number of events (e.g., “File Affidavit
Of Defense” or “File Order Dismissing Appeal”) related to each case, and thus
to each person. To determine how many prior offenses occurred while on
probation, or if the current offense occurred while on probation, we define a
list of event descriptions indicating that an individual was taken on or off
probation. Unfortunately, there appear to be missing events, as individuals
often have consecutive “On” or consecutive “Off” events (e.g., two “On” events
in a row, without an “Off” in between). In these cases, or if the first event
is an “Off” event or the last event is an “On” event, we define two
thresholds, $t_{\textrm{on}}$ and $t_{\textrm{off}}$. If an offense occurred
within $t_{\textrm{on}}$ days after an “On” event or $t_{\textrm{off}}$ days
before an “Off” event, we count the offense as occurring while on probation.
We set $t_{\textrm{on}}$ to 365 and $t_{\textrm{off}}$ to 30. On the other
hand, the “number of times on probation” feature is just the count of “On”
events and the “number of times the probation was revoked” feature is just the
count of “File order of Revocation of Probation” event descriptions (i.e.,
there is no logic for inferring missing probation events for these two
features).

  * •

Age is defined as the age in years, rounded down to the nearest integer, on
the COMPAS screening date.

  * •

Recidivism is defined as any charge that occurred within two years of the
COMPAS screening date. For any part of our analysis that requires recidivism,
we use only observations for which we have two years of subsequent data.

  * •

A juvenile charge is defined as an offense that occurred prior to the
defendant’s 18th birthday.

### Machine learning implementation

Here we discuss the implementation of the various machine learning methods
used in this paper. To predict the COMPAS general and violence raw score
remainders (Tables 2, 4, and 5), we use a linear regression (base R), random
forests (randomForest package), Extreme Gradient Boosting (xgboost package),
and SVM (e1071 package). To clarify, we predict the COMPAS raw scores (not the
decile scores, since these are computed by comparing the raw scores to a
normalization group) after subtracting the age polynomials ($f_{\textrm{age}}$
for the general raw score and $f_{\textrm{viol age}}$ for the violence raw
score). For XGBoost and SVM we select hyperparameters by performing 5-fold
cross validation on a grid of hyperparameters and then re-train the method on
the set of hyperparameters with the smallest cross validation error. For
random forest we use the default selection of hyperparameters. For the COMPAS
general raw score remainder, we use the available Criminal Involvement
Subscale features (Table 12), while for the COMPAS violence raw score
remainder, we use the available History of Violence Subscale and History of
Noncompliance Subscale features listed in tables Tables 10 and 11,
respectively. For both types of COMPAS raw scores, we also use the age-at-
first-arrest. Race and age at screening date may or may not be included as
features, as indicated when the results are discussed. To predict general and
violent two-year recidivism (Tables 7 and 8), we use the same methods,
features, and cross validation technique as used to predict the raw COMPAS
score remainders, except we adapt each method for classification instead of
regression (for linear regression, we substitute logistic regression) and we
include the current offense in the features. All code is written in R and is
available on GitHub666https://github.com/beauCoker/age_of_unfairness.

### Subscale tables

The features that compose the subscales used by COMPAS and that we use for
prediction are listed in Tables 10-14. The Criminal History, Substance Abuse,
and Vocation/Education Subscales (Tables 12, 14, and 13, respectively) are
inputs to the COMPAS general recidivism score, while the History of Violence,
History of Noncompliance, and Vocation/Education Subscales (Tables 10, 11, and
13, respectively) are inputs to the COMPAS violent recidivism score.

Table 10: History of Violence Subscale. We compute the components in bold font. We do not have the data to compute the other components. The feature for family violent arrests is always 0 so it is not useful for prediction. We classify a charge as family violence if the statute is 741.28, which corresponds to the definition of domestic violence in the Florida crime code. In our dataset there were no instances of this statute.  Subscale Items | Values
---|---
Prior juvenile felony offense arrests | 0,1,2+
| Prior violent felony property offense arrests
---
0,1,2,3,4,5+
| Prior murder/voluntary manslaughter arrests
---
0,1,2,3+
| Prior felony assault offense arrests
---
(excluding murder, sex, or domestic violence)
0,1,2,3+
| Prior misdemeanor assault offense arrests
---
(excluding murder, sex, domestic violence)
0,1,2,3+
Prior family violence arrests | 0,1,2,3+
Prior sex offense arrests | 0,1,2,3+
Prior weapons offense arrest | 0,1,2,3+
| Disciplinary infractions for
---
fighting/threatening other inmates/staff
Yes/No
Table 11: History of Noncompliance Subscale. We compute the components in bold font. We do not have the data to compute the other components.  Subscale Items | Values
---|---
| On probation or parole at time of current offense*
---
Probation/Parole/Both/Neither*
Number of parole violations | 0,1,2,3,4,5+
| Number of times person has been returned
---
to prison while on parole
0,1,2,3,4,5+
| Number of new charge/arrests while on
---
probation
0,1,2,3,4,5+
| Number of probation violations/revocations
---
0,1,2,3,4,5+
* = Only “On Probation” and “Not On Probation” computed.
Table 12: Criminal Involvement Subscale. We computed all the components. To compute the number of arrests component, we interpreted a charge as an arrest.  Subscale Items | Values
---|---
| Number times offender has been arrested as adult/juvenile
---
for criminal offense
Any value accepted
|  Number times offender sentenced to jail for $\geq$30 days
---
0,1,2,3,4,5+
| Number of new commitments to state/federal prison
---
(include current)
0,1,2,3,4,5+
| Number of times person sentenced to probation as adult
---
0,1,2,3,4,5+
Table 13: Vocation/Education Subscale. We do not have the data to compute any of these components.  Subscale Items | Values
---|---
Completed high school diploma/GED | Y/N
Final grade completed in school | ___
Usual grades in high school | A,B,C,D,E/F, did not attend
Suspended/expelled from school | Y/N
Failed/repeated a grade level | Y/N
Currently have a job | Y/N
| Have a skill/trade/profession in which
---
you can find work
Y/N
Can verify employer/school (if attending) | Y/N
| Amount of time worked or in school over
---
past 12 months
| 12 months full time,
---
12 months part time,
6$+$ months FT,
0-6 months PT/FT
| Feel that you need more training in new
---
job or career skill
Y/N
| If you were to get (or have) a good job,
---
how would you rate your chance of being successful?
Good, Fair, Poor
| How hard is it for you to find a job
---
above minimum wage compared to others?
| Easier, Same,
---
Harder, Much Harder
Table 14: Substance Abuse Subscale. We do not have the data to compute any of these components.  Subscale Items | Values
---|---
| Do you think your current/past legal
---
problems are partly because of alcohol or drugs?
Y/N
| Were you using alcohol when arrested for
---
your current offense?
Y/N
| Were you using drugs when arrested for
---
your current offense?
Y/N
| Are you currently in formal treatment for
---
alcohol/drugs?
Y/N
| Have you ever been in formal treatment
---
for alcohol/drugs?
Y/N
| Do you think you would benefit from
---
getting treatment for alcohol?
Y/N
| Do you think you would benefit from
---
getting treatment for drugs?
Y/N
| Did you use heroin, cocaine, crack, or
---
methamphetamines as a juvenile?
Y/N