# Using Property Elicitation to Understand the Impacts of Fairness Constraints

Jessie Finocchiaro  Center for Research on Computation and Society, Harvard
University<EMAIL_ADDRESS>
###### Abstract

Predictive algorithms are often trained by optimizing some loss function, to
which regularization functions are added to impose a penalty for violating
constraints. As expected, the addition of such regularization functions can
change the minimizer of the objective. It is not well-understood which
regularizers change the minimizer of the loss, and, when the minimizer does
change, _how_ it changes. We use _property elicitation_ to take first steps
towards understanding the joint relationship between the loss and
regularization functions and the optimal decision for a given problem
instance. In particular, we give a necessary and sufficient condition on loss
and regularizer pairs for when a property changes with the addition of the
regularizer, and examine some regularizers satisfying this condition standard
in the fair machine learning literature. We empirically demonstrate how
algorithmic decision-making changes as a function of both data distribution
changes and hardness of the constraints.

##  1 Introduction

Machine learning is increasingly being used for prediction and resource
allocation tasks pertaining to human livelihood; algorithms often make
predictions based on patterns in historical data to make or supplement
decisions about future events. For example, algorithms are commonly used to
determine a whether or not a loan applicant should receive a loan [30, 2, 31],
estimate a patient’s risk of heart disease [28, 25, 14], and estimate need for
public assistance [21], among other settings. Typically, an algorithm tries to
predict something like the probability of an applicant repaying the loan if
granted one, and then uses this prediction to assign a treatment to the
applicant, such as granting or not granting a loan. Implicit in this model is
the use of an underlying distribution to assign a treatment by computing some
underlying summary statistic, or _property_ , of the distribution over
outcomes. Property elicitation studies the relationship between the choice of
objective function, treatment assignments, and various statistics. For
example, minimizing squared loss corresponds to predicting the _expected
value_ of the outcome (the probability of repayment) and deciding whether or
not to give a loan based on the expected value being above a given threshold.
This contrasts with minimizing the 0-1 loss, which corresponds to learning the
_mode_ , of whether the person is more likely than not to repay a loan, and
the assigned treatment is simply the decision to grant a loan.

In most practical optimization tasks, however, one faces constraints on the
treatment space, especially when the treatments impact human livelihood and
when resources are scarce. In particular, fairness constraints are often
employed to enforce the (approximately) equal algorithmic treatment of
different predefined groups. Instead of minimizing the original loss function,
these algorithms often instead minimize loss + weight * regularizer, where the
regularization term adds a penalty for violating certain desiderata about
community-level outcomes.

However, to date, there is little understanding of how adding regularization
functions into the optimization problem changes the property of the data
distribution learned. We give a necessary and sufficient condition for
regularizers to preserve an elicited property: the property elicited by the
fairness regularizer must be equivalent to the property elicited by the
original loss. However, with additional knowledge about the possible outcome
distribution, we can ask when generally non-equivalent optimization problems
become equivalent with this additional knowledge. We demonstrate our results
on group fairness regularizers, though other regularization functions can be
used as well (e.g., [26]).

To this end, we introduce the notion of _regularized property elicitation_ ,
and what it means for two properties to be equivalent. In Theorem 1 we show
that, under mild conditions on the regularizer, a regularized property is
equivalent to the original property if and only if the property elicited by
the regularizer is equivalent to the original property. We apply Theorem 1 to
a handful of popular fairness regularizers– the absolute difference of
demographic parity, expected equality of opportunity, and equalized false
positive rates– and demonstrate they are not equivalent to cost-sensitive
classifications. However, it is not necessarily the case that a regularizer
changes the elicited property: many additive regularizers yield regularized
properties equivalent to the original, namely (multi)calibration and bounded
group loss.111We are not assigning a value judgment to whether or not a
regularizer changes a property. In these cases, while the property does not
change, using the regularizer is still effective because of practical
limitations on the experessivity of the hypothesis class $\mathcal{H}$, among
other optimization challenges. It does suggest in some sense that these
equivalent regularizers value “accuracy as fairness,” in line with sentiment
from the original works.

In § 3, we present Theorem 1, which gives the necessary and sufficient
condition for the equivalence of properties, and in § 4 demonstrate these
conditions on common fairness regularizers for binary classification. For
those regularizers that do change an elicited property, we additionally
provide examples and geometric intuition about _for which data distributions_
the regularizers change (or do not change) the optimal decision, enforcing the
imposed constraints. Finally, in § 5, we demonstrate our results with
empirical evaluation on synthetic data, a heart attack risk analysis dataset
[28], and the German lending dataset [18].

###  1.1 Literature review

In machine learning, a variety of pre-, in-, and post-processing techniques
have emerged in recent years to make algorithmic decision-making more fair or
equitable. We focus on one algorithmic aspect of in-processing wherein one
modifies the learning algorithm itself by adding a soft constraint to the
objective function, which is some weighted metric of the fairness violation.
The addition of fairness regularizers is one common approach to try to improve
algorithmic decision-making in practice, though their effects are generally
not well-understood (cf. [19, 33, 7, 3, 13, 4, 8, 16, 17]). While many
proposed fairness metrics are situated in binary classification settings,
extensions beyond the binary setting have been proposed more recently [33, 7,
20, 9, 34]. Our framework is general enough to handle a variety of prediction
tasks and regularizers beyond the fair machine learning literature.

We study the impact of regularization functions on the “right” decision an
algorithm should make as a function of the underlying data distribution
through the lens of property elicitation. Property elicitation is well
understood on an individual basis for a variety of discrete prediction tasks
[24, 23, 10, 22] and continuous estimation problems [32, 29, 6, 12, 11] on an
individual level. However, to the best of our knowledge, no work has been done
on property elicitation at the community level, which is necessary when
regularizers are not additive in treatments $\mathbf{t}$. Regularizers
considering community-level outcomes and group membership requires we extend
traditional notions of property elicitation.

##  2 Background

We are primarily concerned with evaluating the optimal treatment for various
prediction tasks. Consider an agent $i\in\\{1,2,\ldots,m\\}=[m]$ who will
achieve some outcome $y^{(i)}\in\mathcal{Y}$ with probability
$p^{(i)}\in\Delta_{\mathcal{Y}}$, where $\Delta_{\mathcal{Y}}$ is the simplex
over a finite set of outcomes $\mathcal{Y}$. A central decision-maker (often a
principal or algorithm) assigns a treatment $t^{(i)}\in\mathcal{T}$ to the
agent, and their error is scored according to a loss function
$L:\mathcal{T}\times\mathcal{Y}\to\mathbb{R}_{+}$. As shorthand, denote
$L(t^{(i)};p^{(i)}):=\mathbb{E}_{Y\sim p^{(i)}}L(t^{(i)},Y)$ as the expected
loss over $p^{(i)}$. Moreover, we assume each agent $i$ is a member of a group
$s^{(i)}\in\mathcal{S}$, and want to ensure agents of different groups are
treated fairly by the centralized decision-maker. Let
$n_{g}:=|\\{i\in[m]:s^{(i)}=g\\}|$ be the number of agents belonging to group
$g$, which we assume is positive. Often, we are concerned with possibly set-
valued functions, $\Gamma:\Delta_{\mathcal{Y}}\to
2^{\mathcal{T}}\setminus\\{\emptyset\\}$; for shorthand, we denote this
$\Gamma:\Delta_{\mathcal{Y}}\rightrightarrows\mathcal{T}$.

In supervised machine learning, predictions are made by learning a hypothesis
function $h:\mathcal{X}\to\mathcal{T}$ mapping features $x\in\mathcal{X}$ to
treatments $t\in\mathcal{T}$. We assume $\mathcal{T}$ is a finite set unless
otherwise stated. If the class of hypotheses $\mathcal{H}$ is sufficiently
expressive, then $t$ encapsulates how the optimal hypothesis _should assign
treatment_ , given an input $x$. Equivalently, we are concerned with optimal
decisions under $p^{(i)}=\Pr[Y\mid X=x^{(i)}]$. For simplicity, we abstract
away $\mathcal{X}$ and proceed with $p^{(i)}\in\Delta_{\mathcal{Y}}$ and
$t^{(i)}\in\mathcal{T}$ in the sequel.

###  2.1 Regularization functions

Often, “fair” algorithms constrain optimization to ensure certain desiderata
are satisfied. However, some standard optimization algorithms such as
stochastic gradient descent often softens these constraints, adding an
additional penalty to the loss function for violating the constraints. We
study how the addition of regularization functions
$\mathcal{R}:\mathcal{T}^{m}\times\mathcal{S}^{m}\times\Delta_{\mathcal{Y}}^{m}\to\mathbb{R}_{+}$
(henceforth: regularizers) change the optimal treatment assigned by minimizing
the expected loss.

For example, imposing group fairness constraints, one might aim to ensure
treatments are independent of the sensitive statistic (as in demographic
parity) or treatments are calibrated to line up with the true probabilities of
positive classification (as in calibration). In this setting, given a
collection of individuals $\\{(s^{(i)},p^{(i)})\\}$, we aim to optimize

$\displaystyle\min_{\mathbf{t}\in\mathcal{T}^{m}}L^{\mathcal{R},\lambda}(\mathbf{t};\mathbf{s};\mathbf{p})$
$\displaystyle:=(1-\lambda)\underbrace{\left[\frac{1}{m}\sum_{i=1}^{m}L(t^{(i)};p^{(i)})\right]}_{\text{expected
loss over $m$
agents}}+\lambda\mathcal{R}(\mathbf{t};\mathbf{s};\mathbf{p})~{}.$ (1)

Because the regularizer might not be additive in $\mathbf{t}$, the treatment
of an individual is not necessarily independent of the treatment of others.
This necessitates the optimization of $\mathbf{t}\in\mathcal{T}^{m}$ rather
than considering each data point individually, as is standard in unregularized
property elicitation.

###  2.2 Property elicitation

When making predictions, a decision-maker often aims to learn a _property_
$\Gamma:\Delta_{\mathcal{Y}}\rightrightarrows\mathcal{T}$, which is simply a
function mapping probability distributions to treatments. Examples of commonly
sought properties include the expected value $EV(p)=\\{\mathbb{E}_{Y\sim
p}[Y]\\}$, the mode $\mathrm{mode}(p)=\arg\,\max_{y}p_{y}$,
$\alpha$-quantiles, and rankings.

######  Definition 1 (Property, elicits).

A _property_ is a function
$\Gamma:\Delta_{\mathcal{Y}}\rightrightarrows\mathcal{T}$ mapping probability
distributions to reports. If $|\mathcal{T}|$ is finite, we call $\Gamma$ a
_finite property_. Moreover, a minimizable222One that attains the infimum in
its first argument for all $y\in\mathcal{Y}$ loss
$L:\mathcal{T}\times\mathcal{Y}\to\mathbb{R}_{+}$ _elicits_ a property
$\Gamma$ if, for all $p\in\Delta_{\mathcal{Y}}$,

$\displaystyle\Gamma(p)$
$\displaystyle=\arg\,\min_{t\in\mathcal{T}}L(t;p)~{}.$

Conversely, we denote the _level set_ of a property
$\Gamma_{t}=\\{p\in\Delta_{\mathcal{Y}}\mid t\in\Gamma(p)\\}$ as the set of
distributions yielding the same optimal treatment.

Throughout, we assume that properties are _nonredundant_ , meaning that the
level set $\Gamma_{t}$ is full-dimensional333The affine dimension of the set
equals the affine dimension of the simplex for all $\mathbf{t}\in\mathcal{T}$
and for each $p\in\mathrm{relint}({\hat{\Gamma}}_{\mathbf{t}})$, we have
$|{\hat{\Gamma}}(\mathbf{p})|=1$. This precludes the consideration of
treatments that are rarely optimal, or only optimal if and only if another
treatment is optimal as well.

Every minimizable loss elicits some property; we denote $\mathrm{prop}[L]$ as
the (unique) property elicited by the loss $L$. For example, the squared loss
elicits the expected value [6, 29], and the level set
$\Gamma_{0}=\\{p\in\Delta_{\mathcal{Y}}:\mathbb{E}_{p}[Y]=\\{0\\}\\}$ of the
expected value is the set of distributions with zero mean. We will later study
the geometry of the level sets of various properties to characterize the how
the minimizers of unregularized losses differ from those of their regularized
counterparts. In order to do so, we consider the property $\Gamma$ evaluated
on a population. Given $\mathbf{p}\in\Delta_{\mathcal{Y}}^{m}$, we consider
the extension ${\hat{\Gamma}}(\mathbf{p}):=[\Gamma(p^{(i)})]_{i}$ with level
sets
${\hat{\Gamma}}_{\mathbf{t}}:=\bigcap_{i}\\{\mathbf{p}\in\Delta_{\mathcal{Y}}^{m}\mid
t^{(i)}\in\Gamma(p^{(i)})\\}$.

We now extend Definition 1 to include population-level reports for loss
functions to encapsulate the case where the regularizer is not additive in
$\mathbf{t}$ or dependent on $\mathbf{s}$.

######  Definition 2 (Regularized property elicitation).

A _regularized property_ is a function
$\Theta^{\mathcal{R},\lambda}:\mathcal{S}^{m}\times\Delta_{\mathcal{Y}}^{m}\rightrightarrows\mathcal{T}^{m}$
mapping beliefs over outcomes to population-level treatments. Similarly, an
objective function $L$ regularized by $\mathcal{R}$ (weighted by $\lambda$),
denoted $L^{\mathcal{R},\lambda}$, elicits a regularized property if, for all
$\mathbf{s}\in\mathcal{S}^{m}$ and $\mathbf{p}\in\Delta_{\mathcal{Y}}^{m}$,

$\displaystyle\Theta^{\mathcal{R},\lambda}(\mathbf{s};\mathbf{p})$
$\displaystyle=\arg\,\min_{\mathbf{t}\in\mathcal{T}^{m}}L^{\mathcal{R},\lambda}(\mathbf{t};\mathbf{s};\mathbf{p}).$

We let $\mathrm{prop}[L^{\mathcal{R},\lambda}]$ denote the regularized
property elicited by $L^{\mathcal{R},\lambda}$.

Denoting the level set of a regularized property requires some nuance because
we are concerned with the change in optimal treatments as a function outcome
distributions $\mathbf{p}$, but the regularized property is a function of
$\mathbf{s}$ as well as $\mathbf{p}$. Therefore, we denote the level set
$\Theta^{\mathcal{R},\lambda}_{\mathbf{t};\mathbf{s}}=\\{\mathbf{p}\in\Delta_{\mathcal{Y}}^{m}\mid\mathbf{t}\in\Theta^{\mathcal{R},\lambda}(\mathbf{s};\mathbf{p})\\}$
denote the level set of the regularized property
$\Theta^{\mathcal{R},\lambda}$. If $\mathbf{s}$ is clear from context, we
sometimes omit it and write $\Theta^{\mathcal{R},\lambda}_{\mathbf{t}}$.

##  3 Equivalence of (regularized) properties

With an understanding of regularized property elicitation, we are now equipped
to ask when a property “changes” with the addition of a regularizer to a loss;
this requires us to consider what it means for properties to be unchanged, or
equivalent.

######  Definition 3 (Equivalence of properties).

A property $\Gamma:\Delta_{\mathcal{Y}}\rightrightarrows\mathcal{T}$ _is
equivalent to_ a regularized property
$\Theta:\mathcal{S}^{m}\times\Delta_{\mathcal{Y}}^{m}\rightrightarrows\mathcal{T}^{m}$
on $\mathbf{s}$ (denoted $\Gamma\equiv_{\mathbf{s}}\Theta$ or
${\hat{\Gamma}}\equiv_{\mathbf{s}}\Theta$) if, for all
$p\in\Delta_{\mathcal{Y}}^{m}$, we have
$\mathbf{t}\in\Gamma(\mathbf{p})\iff\mathbf{t}\in\Theta(\mathbf{s};\mathbf{p})$.

In general, but particularly for large sets of agents, equivalence of a
regularized property to its unregularized counterpart is a rather strong
condition: when there is a “universally fair” report, equivalence holds if
(and only if) the regularizer elicits essentially the same property as the
original loss.

######  Theorem 1.

Fix $\lambda\in(0,1)$ and $\mathbf{s}\in\mathcal{S}^{m}$. Let a loss $L$
elicit the nonredundant finite property $\Gamma$. Moreover, let
$L^{\mathcal{R},\lambda}$ elicit the finite property $\Theta$, and
$\mathcal{R}$ elicit finite property $H$. Then (1)
${\hat{\Gamma}}\equiv_{\mathbf{s}}H\implies{\hat{\Gamma}}\equiv_{\mathbf{s}}\Theta$.
If, additionally, there exists a $\mathbf{t}^{\prime}$ such that
$\\{\mathbf{t}^{\prime}\\}\subseteq H(\mathbf{p})$ for all
$\mathbf{p}\in\Delta_{\mathcal{Y}}^{m}$, then (2)
${\hat{\Gamma}}\equiv_{\mathbf{s}}\Theta\implies{\hat{\Gamma}}\equiv_{\mathbf{s}}H$.

###### Proof.

(1) The first statement is immediate as $H\equiv_{\mathbf{s}}{\hat{\Gamma}}$
implies

$\displaystyle\mathbf{t}\in\arg\,\min_{\mathbf{t}^{\prime}}\mathcal{R}(\mathbf{t}^{\prime};\mathbf{p})$
$\displaystyle\iff\mathbf{t}\in\arg\,\min_{\mathbf{t}^{\prime}}L(\mathbf{t}^{\prime};\mathbf{p})$
$\displaystyle\iff\mathbf{t}\in\arg\,\min_{\mathbf{t}^{\prime}}\lambda\mathcal{R}(\mathbf{t}^{\prime};\mathbf{p})$
$\displaystyle\iff\mathbf{t}\in\arg\,\min_{\mathbf{t}^{\prime}}(1-\lambda)L(\mathbf{t}^{\prime};\mathbf{p})$
$\displaystyle\implies\mathbf{t}$
$\displaystyle\in\arg\,\min_{\mathbf{t}^{\prime}}\lambda\mathcal{R}(\mathbf{t}^{\prime};\mathbf{p})+(1-\lambda)L(\mathbf{t}^{\prime};\mathbf{p})$

Now
$\mathbf{t}\in{\hat{\Gamma}}(\mathbf{p})\implies\mathbf{t}\in\Theta(\mathbf{p})$.
If $\mathbf{t}\in\Theta(\mathbf{p})$, then consider two cases: if
$\mathbf{t}\in H(\mathbf{p})$, we are done by assumption. If
$\mathbf{t}\not\in H(\mathbf{p})$, then there exists a $\mathbf{t}^{\prime}\in
H(\mathbf{p})\cap{\hat{\Gamma}}(\mathbf{p})$, which implies
$\mathbf{t}^{\prime}\in\Theta(\mathbf{p})$. If
$\mathbf{t}\in\Theta(\mathbf{p})$ as well, it must be the case that
$\mathbf{t}\in H(\mathbf{p})\cap{\hat{\Gamma}}(\mathbf{p})$.

(2) We show the contrapositive. Suppose $\Gamma\not\equiv_{\mathbf{s}}H$. Then
there exists a $\mathbf{t}$ such that
$H_{\mathbf{t}}\setminus{\hat{\Gamma}}_{\mathbf{t}}\neq\emptyset$. In
particular,
$H_{\mathbf{t}^{\prime}}\setminus{\hat{\Gamma}}_{\mathbf{t}^{\prime}}\neq\emptyset$
by nonredundancy (and nontriviality) of $\Gamma$ and
$H_{\mathbf{t}^{\prime}}\cap{\hat{\Gamma}}_{\mathbf{t}^{\prime}}={\hat{\Gamma}}_{\mathbf{t}^{\prime}}\neq\emptyset$
since $H_{\mathbf{t}^{\prime}}=\Delta_{\mathcal{Y}}^{m}$. Moreover,
$H_{\mathbf{t}^{\prime}}\cap{\hat{\Gamma}}_{\mathbf{t}^{\prime}}\neq\emptyset$
implies $\Theta_{\mathbf{t}^{\prime}}\cap
H_{\mathbf{t}^{\prime}}\neq\emptyset$. We claim
$(\Theta_{\mathbf{t}^{\prime}}\cap
H_{\mathbf{t}^{\prime}})\setminus{\hat{\Gamma}}_{\mathbf{t}^{\prime}}\neq\emptyset$,
which implies
$\Theta_{\mathbf{t}^{\prime}}\setminus{\hat{\Gamma}}_{\mathbf{t}^{\prime}}\neq\emptyset$,
yielding $\Theta\not\equiv_{\mathbf{s}}\Gamma$.

For contradiction, suppose $(\Theta_{\mathbf{t}^{\prime}}\cap
H_{\mathbf{t}^{\prime}})\setminus{\hat{\Gamma}}_{\mathbf{t}^{\prime}}=\emptyset$.
Observe $\mathbf{t}^{\prime}\in
H(\mathbf{p})\cap\Gamma(\mathbf{p})\implies\mathbf{t}^{\prime}\not\in\Theta(\mathbf{p})$,
which cannot be true by the construction of $\Theta$. Moreover, if
$\mathbf{t}^{\prime}\in
H(\mathbf{p})\cap(\Delta_{\mathcal{Y}}^{m}\setminus{\hat{\Gamma}}(\mathbf{p}))$,
then we must have $\mathbf{t}^{\prime}\not\in\Theta(\mathbf{p})$ by
assumption. Either way, $\mathbf{t}^{\prime}\in
H(\mathbf{p})\implies\mathbf{t}^{\prime}\not\in\Theta(\mathbf{p})$. Therefore,
we must have
$H_{\mathbf{t}^{\prime}}\cap\Theta_{\mathbf{t}^{\prime}}=\emptyset$, which
yields a contradiction.

∎

Intuitively, Theorem 1 says that the property elicited by a regularized loss
function is the same as the unregularized loss if and only if the regularizer
elicits the same property as the loss itself. Since loss functions are
measurements of accuracy, then equivalence of properties implies an algorithm
values accuracy as fairness. The assumption that one report is always optimal
implies the existence of an “unconditionally fair” treatment, which is
satisfied by group fairness regularizers such as demographic parity (assigning
every agent the same treatment), equalized false positives (assigning everyone
the negative treatment), equalized false negatives (assigning everyone the
positive treatment), and expected equality of opportunity (assigning the
negative treatment).

##  4 (Non)equivalence of common fairness metrics for binary classification

We now evaluate five common fairness regularizers, and apply Theorem 1 to show
nonequivalence between binary classification tasks and their regularized
counterparts. For each regularizer, we give restrictions on
$\Delta_{\mathcal{Y}}^{m}$ such that the regularized property is equivalent to
the original under these restrictions.

To build intuition, we will examine simple cases of how regularizers change
elicited properties with populations of $m=2$ agents belonging to different
groups $\mathbf{s}=(a,b)$.

Figure 1 provides some additional intuition for the proof of Theorem 1. Each
subfigure gives the level sets of the property elicited by the mode
regularized by the demographic parity violation (DP), where each point in
$[0,1]^{2}$ represents $\mathbf{p}\in\Delta_{\mathcal{Y}}^{2}$ by
$(\Pr_{p^{(1)}}[Y=1],\Pr_{p^{(2)}}[Y=1])$. Each colored cell depicts a
different level set of a regularized property $\Theta^{DP,\lambda}$. This
regularized property is overlaid on the (unregularized) mode, so that, upon
visual inspection, one observes the regions where the two properties differ.
As $\lambda\to 0$, the regularized property becomes increasingly similar to
the unregularized, and as $\lambda\to 1$, the regularized property
increasingly resembles the property elicited by $\mathcal{R}$.

###  4.1 Demographic parity

In the context of binary classification, one might be interested in
regularizing their loss with the demographic parity violation, measured by the
absolute difference of the rates at which agents are assigned the positive
treatment from each of two groups. Any treatment that assigns the positive
treatment at the same rate optimizes the demographic parity regularizer, which
is not equivalent to the mode. That is,
$H(\mathbf{s};\mathbf{p})\supseteq\\{\mathbf{0},\mathbb{1}\\}$ for all
$\mathbf{p}\in\Delta_{\mathcal{Y}}^{m}$ and $\mathbf{s}\in\mathcal{S}^{m}$.
Thus, if $\mathcal{S}=\\{a,b\\}$444This is simply for ease of exposition, and
can be relaxed., we can apply Theorem 1 to conclude the DP-regularized mode is
not equivalent to the unregularized mode.

$\displaystyle L^{DP,\lambda}(\mathbf{t};\mathbf{s};\mathbf{p})$
$\displaystyle=\frac{1-\lambda}{m}\sum_{i=1}^{m}L(t^{(i)};p^{(i)})+\lambda\left|\frac{1}{n_{a}}\sum_{i:s^{(i)}=a}t^{(i)}-\frac{1}{n_{b}}\sum_{i:s^{(i)}=b}t^{(i)}\right|$
(DP)

Now, with $\mathcal{T}=\\{0,1\\}$, if $L$ is the 0-1 loss555These derivations
also hold if $L$ is squared loss, hinge loss, and many other losses for binary
classification., we can evaluate $L^{DP,\lambda}$ for each treatment in
$\mathcal{T}^{2}=\\{(1,1),(0,1),(1,0),(0,0)\\}$.

$\displaystyle L^{DP,\lambda}((1,1);(p^{(1)},p^{(2)}))$
$\displaystyle=\frac{1-\lambda}{2}\left[(1-p^{(1)})+(1-p^{(2)})\right]$
$\displaystyle L^{DP,\lambda}((0,1);(p^{(1)},p^{(2)}))$
$\displaystyle=\frac{1-\lambda}{2}\left[p^{(1)}+(1-p^{(2)})\right]+\lambda$
$\displaystyle L^{DP,\lambda}((1,0);(p^{(1)},p^{(2)}))$
$\displaystyle=\frac{1-\lambda}{2}\left[(1-p^{(1)})+p^{(2)}\right]+\lambda$
$\displaystyle L^{DP,\lambda}((0,0);(p^{(1)},p^{(2)}))$
$\displaystyle=\frac{1-\lambda}{2}\left[p^{(1)}+p^{(2)}\right].$

These expected losses now enable us to study the level sets
$\Theta^{DP,\lambda}_{\mathbf{t};\mathbf{s}}=\\{\mathbf{p}\in\Delta_{\mathcal{Y}}^{m}\mid\mathbf{t}\in\Theta^{DP,\lambda}(\mathbf{s};\mathbf{p})\\}$.
Abusing notation, we use $\Theta^{DP,\lambda}_{\mathbf{t}}$ understanding
$\mathbf{s}=(a,b)$.

We have have $(0,0)\in\arg\,\min_{\mathbf{t}\in\mathcal{T}^{2}}$ if

$\displaystyle\frac{1-\lambda}{2}\left[(1-p^{(1)})+(1-p^{(2)})\right]$
$\displaystyle\leq\frac{1-\lambda}{2}\left[p^{(1)}+(1-p^{(2)})\right]+\lambda$
$\displaystyle\iff\frac{1-3\lambda}{2(1-\lambda)}$ $\displaystyle\leq p^{(1)}$
$\displaystyle\frac{1-\lambda}{2}\left[(1-p^{(1)})+(1-p^{(2)})\right]$
$\displaystyle\leq\frac{1-\lambda}{2}\left[p^{(2)}+(1-p^{(1)})\right]+\lambda$
$\displaystyle\iff\frac{1-3\lambda}{2(1-\lambda)}$ $\displaystyle\leq p^{(2)}$
$\displaystyle\frac{1-\lambda}{2}\left[(1-p^{(1)})+(1-p^{(2)})\right]$
$\displaystyle\leq\frac{1-\lambda}{2}\left[p^{(1)}+p^{(2)}\right]$
$\displaystyle\iff p^{(1)}+p^{(2)}$ $\displaystyle\leq 1~{}.$

Therefore, the level set $\Theta^{DP,\lambda}_{(0,0)}$ can be described by the
polyhedron

$\displaystyle\Theta^{DP,\lambda}_{(0,0)}$
$\displaystyle=\left\\{p\in[0,1]^{2}\mid\begin{bmatrix}0&-1&\frac{1-3\lambda}{2(1-\lambda)}\\\
-1&0&\frac{1-3\lambda}{2(1-\lambda)}\\\
1&-1&1\end{bmatrix}\begin{bmatrix}p^{(1)}\\\ p^{(2)}\\\
1\end{bmatrix}\geq\mathbf{0}\right\\}~{}.$

Observe that the final constraint is actually one on the marginal $P[Y]$: the
expected outcome over the whole population should be less likely to be $1$
than $0$. We can evaluate the rest of the level sets in a similar manner.

Now let us gain some geometric intuition for how these level sets change by
referencing Figure 1. For two agents belonging to different groups, each point
in the figure represents a pair $\mathbf{p}:=(p^{(1)},p^{(2)})$ of true
probabilities for the two agents. The pair $\mathbf{p}\in[0,1]^{2}$, and the
region $[0,1]^{2}$ can be divided into up to $|\mathcal{T}^{m}|$ regions for
which each $\mathbf{t}\in\mathcal{T}^{m}$ is contained in
$\Theta^{DP,\lambda}(\mathbf{p})$. The sequence of figures in Figure 1 denotes
the level sets of $\Theta^{DP,\lambda}$ as one varies $\lambda\in[0,1]$. For
intuition, one can observe that the regions where the players receive the same
treatment (blue and red) grows as $\lambda$ increases, starting with $1/2$ of
the $[0,1]^{2}$ space, and increasing to all of $[0,1]^{2}$ as $\lambda\to 1$.

Figure 1: Visualizing the level sets of the $DP$-regularized property
$\Theta^{DP,\lambda}$ for different values of $\lambda\in[0,1]$, where $m=2$
and $\mathbf{s}=(a,B)$. Each point $(p^{(1)},p^{(2)})$ in a square represents
$(\Pr_{p^{(1)}}[Y=1],\Pr_{p^{(2)}}[Y=1])$, and each colored cell represents
sets of $(p^{(1)},p^{(2)})$ pairs such that the optimal treatment is the same
for all points in the cell. For example, the magenta cell (lower right) is the
set of distributions where the decision-maker prefers to attribute the
positive treatment ($t^{(i)}=1$) to the first agent, and the negative
treatment ($t^{(i)}=0$) to the second agent.

We now turn our attention towards the regions of $\Delta_{\mathcal{Y}}^{m}$
where the regularized and unregularized properties are equivalent with a
demographic parity regularizer. First, we observe that if uniform treatment of
a population is optimal on the unregularized property, it is also optimal on
the regularized property, which is actually a corollary of Theorem 1.

######  Proposition 1.

Fix $\lambda\in(0,1)$. Let $L$ elicit $\Gamma$, $L^{\mathcal{R},\lambda}$
elicit $\Theta$, and $\mathcal{R}$ elicit $H$. For all $t\in\mathcal{T}^{m}$
and $\mathbf{s}\in\mathcal{S}^{m}$, ${\hat{\Gamma}}_{\mathbf{t}}\cap
H_{\mathbf{t};\mathbf{s}}\subseteq\Theta_{t}$.

###### Proof.

$\mathbf{p}\in{\hat{\Gamma}}_{\mathbf{t}}\cap
H_{\mathbf{t};\mathbf{s}}\implies L(\mathbf{t};\mathbf{p})\leq
L(\mathbf{t}^{\prime};\mathbf{p})$ and
$\mathcal{R}(\mathbf{t};\mathbf{s};\mathbf{p})\leq\mathcal{R}(\mathbf{t}^{\prime};\mathbf{s};\mathbf{p})$
for all $t^{\prime}\in\mathcal{T}^{m}$, which in turn implies
$L(\mathbf{t};\mathbf{p})+\mathcal{R}(\mathbf{t};\mathbf{s};\mathbf{p})\leq
L(\mathbf{t}^{\prime};\mathbf{p})+\mathcal{R}(\mathbf{t}^{\prime};\mathbf{s};\mathbf{p})\implies(1-\lambda)L(\mathbf{t};\mathbf{p})+\lambda\mathcal{R}(\mathbf{t};\mathbf{s};\mathbf{p})\leq(1-\lambda)L(\mathbf{t}^{\prime};\mathbf{p})+\lambda\mathcal{R}(\mathbf{t}^{\prime};\mathbf{s};\mathbf{p})$
for all $t^{\prime}\in\mathcal{T}^{m}$. ∎

We apply this result to the “universally fair” reports via demographic parity
$\mathbf{0}$ and $\mathbb{1}$.

######  Corollary 1.

Fix $\mathbf{s}\in\mathcal{S}^{m}$ and $\lambda\in[0,1]$. Let $L$ elicit
$\Gamma$ and $\mathbf{L}^{DP,\lambda}$ elicit $\Theta$.
${\hat{\Gamma}}_{\mathbf{0}}\subseteq\Theta_{\mathbf{0};\mathbf{s}}$.
Moreover,
${\hat{\Gamma}}_{\mathbb{1}}\subseteq\Theta_{\mathbf{s};\mathbb{1}}$.

###### Proof.

Let $H:=\mathrm{prop}[L^{\lambda,DP}]$. For all
$\mathbf{p}\in\Delta_{\mathcal{Y}}^{m}$, we have
$\\{\mathbf{0},\mathbf{1}\\}\subseteq H(\mathbf{p})$. Therefore,
${\hat{\Gamma}}_{\mathbf{0}}\cap H_{\mathbf{0}}={\hat{\Gamma}}_{\mathbf{0}}$
(and similarly with ${\hat{\Gamma}}_{\mathbb{1}}\cap H_{\mathbb{1}}$).
Therefore, ${\hat{\Gamma}}_{\mathbf{0}}={\hat{\Gamma}}_{\mathbf{0}}\cap
H_{\mathbf{0}}\subseteq\Theta_{\mathbf{0}}$ and
${\hat{\Gamma}}_{\mathbb{1}}={\hat{\Gamma}}_{\mathbb{1}}\cap
H_{\mathbb{1}}\subseteq\Theta_{\mathbb{1}}$. ∎

We now turn our attention to the opposite case: if, while regularized,
treating different groups differently (and uniformly within the groups) is
optimal, then it is also optimal in the unregularized setting. In particular,
this holds for treatments maximizing $\mathcal{R}$.

######  Proposition 2.

Fix $\mathbf{s}\in\\{a,b\\}^{m}$ and $\lambda\in[0,1]$. Fix
$\mathbf{t}=\mathbb{1}_{a}$ (or $\mathbb{1}_{b}$ without loss of generality).
Let $L$ elicit $\Gamma$ over outcomes $\mathcal{Y}=\\{0,1\\}$.
$\Theta^{DP,\lambda}_{\mathbf{t};\mathbf{s}}\subseteq{\hat{\Gamma}}_{\mathbf{t}}$.

###### Proof.

With $\mathbf{s}$ fixed,
$t\in\arg\,\max_{\mathbf{t}^{\prime}}DP(\mathbf{t}^{\prime};\mathbf{p})$ for
all $\mathbf{p}\in\Delta_{\mathcal{Y}}^{m}$. Therefore,

$\displaystyle(1-\lambda)L(\mathbf{t};\mathbf{p})+\lambda
DP(\mathbf{t};\mathbf{p})$
$\displaystyle\leq(1-\lambda)L(\mathbf{t}^{\prime};\mathbf{p})+\lambda
DP(\mathbf{t}^{\prime};\mathbf{p})\forall\mathbf{t}^{\prime}$
$\displaystyle\implies(1-\lambda)L(\mathbf{t};\mathbf{p})$
$\displaystyle\leq(1-\lambda)L(\mathbf{t}^{\prime};\mathbf{p})\qquad\forall\mathbf{t}^{\prime}~{},$

which implies the result. ∎

With that, we partially characterize the relationship between the
unregularized and DP-regularized level sets for standard binary
classification. In the simple case with $m=2$ agents, this characterization is
complete: if the optimal treatment is uniform, it stays uniform. Moreover, if
the most “unfair” treatment wherein all the members of one group receive the
treatment, and none of the second group is optimal in the regularized setting,
it is also optimal in the unregularized setting. In any other setting, the
optimal treatment changes with the addition of a DP regularizer.

###  4.2 Equalized FPR

Following a similar process to § 4.1, we now consider the regularizer that
measures the absolute difference of false positive rates across groups, where
the false positive rate is given by
$FPR_{g}(\mathbf{t};\mathbf{s};\mathbf{p})=\Pr[Y^{(i)}=0\mid
t^{(i)}=1,s^{(i)}=g]=\frac{1}{|\\{i:t^{(i)}=1,s^{(i)}=g\\}|}\sum_{i:s^{(i)}=g,t^{(i)}=1}(1-p^{(i)})$.
The optimization problem then becomes

$\displaystyle L^{FPR,\lambda}(\mathbf{t};\mathbf{s};\mathbf{p})$
$\displaystyle=\frac{1-\lambda}{m}\sum_{i}L(t^{(i)};p^{(i)})+\lambda\left|FPR_{a}(\mathbf{t};\mathbf{s};\mathbf{p})-FPR_{b}(\mathbf{t};\mathbf{s};\mathbf{p})\right|$
(FPR)

The FPR regularizer computes the difference of false positive rates between
groups, so one can observe that the false positive rate of a group is is
reduced by assigning more negative treatments $t^{(i)}=0$. We can see in
Figure 2 that the FPR regularizer then makes it worse for an algorithm to
assign the positive treatment to an agent $i$ even if $p^{(i)}$ is slightly
greater than $1/2$, as marked by the $\star$ in Figure 2(R).

As in § 4.1, we can apply Proposition 1 to show that if assigning everyone the
negative treatment is optimal in the unregularized setting, it is also the
optimal treatment with the FPR regularizer.

######  Corollary 2.

Fix $\mathbf{s}\in\mathcal{S}^{m}$. Let $L$ elicit $\Gamma$ and
$L^{FPR,\lambda}$ elicit $\Theta^{FPR,\lambda}$.
${\hat{\Gamma}}_{\mathbf{0}}\subseteq\Theta^{FPR,\lambda}_{\mathbf{0};\mathbf{s}}$.

###### Proof.

For all $\mathbf{p}\in\Delta_{\mathcal{Y}}^{m}$, we have $\mathbf{0}\in H(p)$.
Therefore ${\hat{\Gamma}}_{0}={\hat{\Gamma}}_{0}\cap
H_{\mathbf{0};\mathbf{s}}\subseteq\Theta_{\mathbf{0}}$ by Proposition 1. ∎

Figure 2: Visualizing the level sets of the $FPR$-regularized property
$\Theta^{FPR,\lambda}$ for different values of $\lambda\in[0,1]$, where $m=2$
and $s=(a,b)$. Each point $(p^{(1)},p^{(2)})$ in a square represents
$(\Pr_{p^{(1)}}[Y=1],\Pr_{p^{(2)}}[Y=1])$, and each colored cell represents
sets of $(p^{(1)},p^{(2)})$ pairs such that the optimal treatment is the same
for all points in the cell. For example, the magenta cell (lower right) is the
set of distributions where the decision-maker prefers to attribute the
positive treatment ($t^{(1)}=1$) to the first, and the negative treatment
($t^{(2)}=0$) to the second agent.

###  4.3 Expected equality of opportunity

While standard equality of opportunity (cf. [15]) requires access to observed
labels, we are interested in equality of opportunity in expectation, and
consider a variant that does not require access to labels proposed by Blandin
and Kash [5]. Consider the treatment space $\mathcal{T}=\\{0,1\\}^{m}$ and
regularizer
$\mathcal{R}(\mathbf{t};\mathbf{s};\mathbf{p})=|EEO_{a}(\mathbf{t};\mathbf{s};\mathbf{p})-EEO_{b}(\mathbf{t};\mathbf{s};\mathbf{p})|$,
where

$\displaystyle EEO_{g}(\mathbf{t};\mathbf{s};\mathbf{p};g)$
$\displaystyle=\Pr_{i\sim[m]}[t^{(i)}=1\mid y^{(i)}=1,s^{(i)}=g]$
$\displaystyle=\frac{\Pr[Y^{(i)}=1\mid
t^{(i)}=1,s^{(i)}=g]\Pr[t^{(i)}=1]}{\Pr[Y^{(i)}=1]}$
$\displaystyle=\frac{\left(\frac{1}{|\\{i:s^{(i)}=g,t^{(i)}=1\\}|}\sum_{i:t^{(i)}=1,s^{(i)}=g}(p^{(i)})\right)(\sum_{i}t^{(i)})}{\sum_{i}p^{(i)}}~{}.$

In order to apply Theorem 1, we can observe that $\mathbf{0}$ is a
“universally fair” treatment, as $EEO_{a}=EEO_{b}=0$, regardless of
$\mathbf{p}$ and $\mathbf{s}$, and we observe nonequivalence. Moreover, we can
apply Proposition 1 to show that uniform treatment being optimal in the
unregularized case implies it is also optimal with the EEO regularizer as
well.

######  Corollary 3.

Fix $\mathbf{s}\in\mathcal{S}^{m}$, and let $L$ elicit $\Gamma$ over outcomes
$\mathcal{Y}=\\{0,1\\}$ and $L^{EEO,\lambda}$ elicit $\Theta^{EEO,\lambda}$.
${\hat{\Gamma}}_{\mathbf{0}}\subseteq\Theta^{EEO,\lambda}_{\mathbf{0}}$.

Figure 3: Level sets of the EEO-regularized mode on $\mathcal{Y}=\\{0,1\\}$.

###  4.4 Equivalent regularizers

In the previous section, we use Theorem 1 to show the nonequivalence of
reguliarized properties, and examine a few common regularizers to show some
restrictions that recover equivalence under certain distributional assumptions
on the outcomes. Turning our attention to the converse of Theorem 1, we
examine two regularizers that elicit the mode, and thus the regularized
property is equivalent to the unregularized on all of
$\Delta_{\mathcal{Y}}^{m}$: calibration [27] and bounded group loss [1]. In
some sense, this suggests that these regularizers value accuracy as fairness.
If models are as accurate as they could possibly be, the most “fair”
treatments to assign are also the most accurate. In practice, the regularizers
mitigate unfairness arising from limited expressivity of the model: if the
model was perfectly expressive and could predict the mode perfectly, it would
assign the same treatments even with heavy penalties for “unfairness.”

#### Calibration

Calibration constraints ensure that the predicted value $t^{(i)}$ most closely
lines up with the true probability $p^{(i)}$, regularizing the loss by the
sums of the absolute differences $|t^{(i)}-p^{(i)}|$. The absolute difference
elicits the $1/2$-quantile, which is also the mode on $\mathcal{Y}=\\{0,1\\}$,
so the regularizer
$\mathcal{R}(\mathbf{t};\mathbf{s};\mathbf{p})=\sum_{g}\frac{1}{n_{g}}\sum_{i:s^{(i)}=g}|t^{(i)}-p^{(i)}|$
elicits the mode in binary classification problems.

Formally, consider the objective

$\displaystyle L^{Cal,\lambda}(\mathbf{t};\mathbf{s};\mathbf{p})$
$\displaystyle=\frac{1-\lambda}{m}\sum_{i}L(t^{(i)};p^{(i)})+\lambda\sum_{g}\frac{1}{n_{g}}\sum_{i:s^{(i)}=g}|t^{(i)}-p^{(i)}|$
(Cal)

This constraint does not include any comparisons across group averages, so the
optimal report is obtained by giving individual predictions. In binary
classification, the $1/2$-quantile is the same as the mode, so the property is
given $\Theta^{Cal,\lambda}(\mathbf{s};\mathbf{p})=\mathrm{mode}(\mathbf{p})$.

#### Bounded group loss

We now consider the constraint on bounded group loss: $\mathbb{E}_{Y\mid
S=s}L(r,Y)<\epsilon$ for all $s\in\mathcal{S}$, introduced by Agarwal et al.
[1]. To model bounded group loss as a soft constraint, we simply weigh the
expected loss conditioned on the group size as a regularizer, so accuracy is
more incentivized on small groups.

$\displaystyle L^{BGL,\lambda}(\mathbf{t};\mathbf{s};\mathbf{p})$
$\displaystyle=\frac{1-\lambda}{m}\sum_{i}L(t^{(i)};p^{(i)})+\sum_{g}\frac{\lambda}{n_{g}}\sum_{i:s^{(i)}=g}L(t^{(i)};p^{(i)})$

Adding this constraint as a fairness regularizer does not change the property
elicited (e.g.,
$\Theta^{\lambda}(\mathbf{s};\mathbf{p})={\hat{\Gamma}}(\mathbf{p})$ for all
$p\in\Delta_{\mathcal{Y}}^{m}$). In part this is because it still encourages
the hypothesis to learn what is best for each individual in the population,
where other constraints add a regularizer that compares the deviation between
two groups.

######  Corollary 4.

Let $L$ elicit $\Gamma$.
${\hat{\Gamma}}\equiv_{\mathbf{s}}\Theta^{BGL,\lambda}$ for all
$\mathbf{s}\in\mathcal{S}^{m}$ and $\lambda\in[0,1]$.

###### Proof.

The regularizer
$\mathcal{R}(\mathbf{t};\mathbf{s};\mathbf{p}):=\sum_{g}\frac{1}{n_{g}}\sum_{i:s^{(i)}=g}L(t^{(i)};p^{(i)})$
is additive in $\mathbf{t}$, and elicits the same property as $L$ since it is
simply a reweighing of $L$. ∎

##  5 Experiments

While property elicitation allows us to reason about what a treatment an
algorithm _should_ assign, we examine whether or not these decisions are
consistent with the treatments assigned by algorithms in practice with simple
models. We first generate a set of synthetic datasets to understand how a
classifier’s decisions change as one navigates the space of data
distributions. Moving through this space demonstrates the relationship between
loss and regularizer in the synthetic setting as the data distribution over
changes in $\Delta_{\mathcal{Y}}^{m}$. We then evaluate the effect of the
regularizer weight $\lambda$ on treatment assignment in cardiovascular disease
risk prediction [28] and lending [18] datasets, where the data distribution is
fixed. In both settings, we train a linear classifier over $30$ trials with
binary cross entropy loss with (a) no regularizer, (b) demographic parity
difference (c) false positive rate difference (d) false negative rate
difference, (e) equality of opportunity difference, and compute the fairness
violations of the classifier trained on each of the four losses, where
elicited property values are shown in Figure 9.

###  5.1 Effect of the data distribution

Recall that we applied Theorem 1 and its intuition in Figures 1, 2, and 3 to
conclude the mode is not equivalent to $\Theta^{\mathcal{R},\lambda}$ for
various regularizers including (DP), (FPR), False Negative Rates (in § A), and
Expected Equality of Opportunity. However, the equivalence of regularized
properties and their unregularized counterparts is a rather strong condition,
as pointwise equivalence must hold for _every_ set of data distributions. In
practice, the true data distribution may be somewhere in the space of
distributions where the property value does not change for the chosen value of
$\lambda$. With the knowledge that equivalent distributions have no
endogeneous differences in hand, we generate a set of synthetic distributions
to understand tradeoffs to regularizers as we move though the space of data
distributions.

We generate generate synthetic datasets for binary classification as follows:
there are two groups, $\mathcal{S}=\\{a,b\\}$ with $\Pr[a]=\Pr[b]=1/2$, a
member of each group has $\Pr[Y=1\mid S=g]=p_{g}\in[0,1]$. Each set of agents
is represented by $x=\\{p_{a},p_{b},r_{1},\ldots,r_{k}\\}$, where
$r_{1},\ldots,r_{k}$ are uniformly random values in $[-1,1]$. We then train a
linear classifier via stochastic gradient descent (30 trials with learning
rate = 0.001, 1500 epochs, 10000 $(p_{a},p_{b})$ pairs, $k=3$), that minimizes
the binary cross entropy loss regularized by either demographic parity, false
positive rate, false negative rate, or difference in equality of opportunity
with $\lambda=0.15$. The simplicity of features is intentional: the “perfect”
decision should be fully realizable in the unregularized setting, so the
benchmark accuracy should be relatively high. Fixing the probability for a
positive outcome $p_{a}=0.3$ for a member of group $a$, we vary the
probability of a positive outcome $p_{b}$ for a member of group $b$ to observe
how fairness violations change as the underlying data distribution changes.
For intuition, by design of the datasets, we reason about the “average member”
of the population and reference the level sets drawn in Figure 6. Fixing
$p_{a}$ and varying $p_{b}$ can be thought of as understanding what happens in
decision making as one moves vertically up the line $\\{(0.3,p_{b})\mid
p_{b}\in[0,1]\\}$, denoted by the black dashed lines in Figure 6. For the
level sets of $\Theta^{DP,0.15}$, $\Theta^{FPR,0.15}$, and
$\Theta^{EEO,0.15}$, this approximately suggests that the algorithm should
predict negatively for members of both groups until $p_{b}\geq 2/3$, at which
point it should predict positively for the second agent in group $b$. However,
for $\Theta^{FNR,0.15}$, one should always give give a member of group $a$ the
negative treatment in expectation, while giving a member of group $b$ the
positive treatment as long as $p_{b}\geq 1/2+\epsilon$, marking its similarity
to the unregularized property on this line.

In Figure 4(TL), we observe approximately equal accuracy of all models
regardless of the choice of regularizer. Moreover, most fairness-regularized
models tended to yield lower DP violations than the unregularized models, and
in Figure 5(L) one observes this gap is larger as $p_{b}$ increases over
$1/2$, in line with the intuition provided by Figure 6. Similar trends follow
with FPR and EEO regularized models, but the difference is least salient with
FNR regularized models. This is unsurprising given the similarity between the
mode and $\Theta^{FNR,0.15}$ along the line $p_{a}=0.3$ mentioned above and
demonstrated in Figure 6.

Figure 4: Regularizer values with synthetic data generated via $\Pr[Y=1\mid
g=a]=0.3$ and $\Pr[Y=1\mid g=b]$ on the horizontal axis.

Figure 5: Regularizer values with synthetic data generated via $\Pr[Y=1\mid
g=a]=0.3$ and $\Pr[Y=1\mid g=b]$ on the horizontal axis. Figure 6: Fixing
$p_{a}=0.3$, examining how the property value changes as a function of $p_{b}$
for different regularizers. Demographic parity results in different decisions
only if $p_{b}\in[1/2,3/4]$, FPR if $p_{b}\in[1/2,2/3]$, FNR has essentially
the same property values on the line $p_{a}=0.3$, and EEO leads to a small
region where optimal decisions change for $p_{b}\in[1/2,2/3]$.

###  5.2 The effect of choice of $\lambda$

Conversely to the interpretation of the experiments in § 5.1, to gain
intuition for why decisions might change as a function of $\lambda$, we now
consider each dataset representing a $(p_{a},p_{b})$ point in one of Figures
1–3, and consider how the level set it belongs to changes as one changes
$\lambda$. We examine two datasets, German lending [18] and heart disease risk
prediction [28]. For both datasets, we train 30 linear models with 1500
epochs, learning rate of $0.001$.

#### German lending

In the German lending dataset, we treat age as the sensitive attribute, using
an indicator thresholded at 25 years old. On the entire dataset, we have
$\Pr[Y=1\mid S\geq 25]=0.728$ and $\Pr[Y=1\mid S<25]=0.578$, and an unbalanced
group representation with $\Pr[S<25]=0.191$.

Perhaps surprisingly, we observe little impact of the choice of $\lambda$: in
fact, for large values of $\lambda$, the regularized model seems to yield
_less fair_ treatments on average, demonstrated in Figure 7. Upon closer
inspection, this can be explained partly by the observation that
$(p_{a},p_{b})=(0.728,0.578)$: a distribution that warrants treating the
“average member” of each subpopulation the same, which aligns with most
fairness regularizers. This is demonstrated in Figure 9, where the
$(p_{a},p_{b})$ coordinate is denoted by a _g_ , for German.

Figure 7: Effect of $\lambda$ on regularizer values on the German lending
dataset [18]. Because the $(p_{a},p_{b})$ point summarizing group differences
in the dataset are at a point where regularized decisions are the same as
unregluarized decisions, it is unsurprising that regularizers do not
significantly reduce unfairness, regardless of $\lambda$.

Figure 8: Distributions of accuracy and fairness violations in lending data.
In general, it seems the models are tending to make similar predictions, which
often nearly equal medians.

Figure 9: The level sets of different regularized properties as $\lambda$
changes. (Top to bottom: DP, FPR, FNR, EEO). The $g$ represents the “average”
members of each group in the German lending dataset, and $h$ the heart disease
risk dataset.

#### Heart disease risk

In the heart disease risk prediction dataset, we treat sex as the sensitive
attribute, and observe $\Pr[Y=1\mid S=0]=0.75$ and $\Pr[Y=1\mid S=1]=0.449$
yields a $p_{a},p_{b}$ pair warranting different treatments for the “average”
member of each group, and $\Pr[S=1]=0.63$ for a more attribute-balanced
dataset. The relationship between the optimal treatment of the “average”
member of both groups as $\lambda$ changes can be seen in Figure 9, denoted by
$h$.

Figure 10 demonstrates little significant difference between the unregularized
and regularized models. On average, the models perform about equally in terms
of fairness violations (Figure 11), but the regularized models have a higher
variance in their fairness violations, probably because of limited
expressivity of the hypothesis class.

Figure 10: Effect of $\lambda$ on regularizer values on the heart disease risk
dataset [28].

Figure 11: Distributions of accuracy and fairness violations in heart disease
data. In general, it seems the models are tending to make similar predictions,
which often nearly equal medians.

##  6 Discussion and Conclusion

In this work, we extend the notion of property elicitation to consider
regularized loss functions, and give a necessary and sufficient condition on a
regularizer to be equivalent to the original property. We apply this condition
to demonstrate the (non-)equivalence of properties with a handful of
regularizers common in the fair machine learning literature. Finally, we show
how the choice and weight of regularization function can change decision-
making on synthetic data as well as the German lending and heart disease risk
datasets.

#### Limitations and considerations

The main intent of this work is to provide conceptual insight about how
fairness regularizers change algorithmic decision-making and predictions. The
insights provided rely on the hypothesis class being sufficiently expressive,
and should not be solely used to justify the use of a regularizer. The
addition of a regularizer and insights given are agnostic to the data itself
and therefore agnostic to pre-processing and post-processing of data.
Additional pre- or post-processing of the data may change the elicited
property, though we leave this to future work.

#### Future work

There are many directions for future work. This work serves as a proof of
concept for the extension of property elicitation to accommodate
regularization functions, demonstrated on a handful of regularizers, but
applying the necessary and sufficient condition on the equivalence of
properties under different regularizers and more general prediction tasks
remains an open direction of work. Moreover, it is important to understand how
model complexity as well as pre- and post-processing of data can affect
results. Finally, the addition of a regularizer seems to increase the
complexity of the optimization problem linearly in $m$. Understanding if there
are more efficient ways to frame the optimization problem for certain
regularization functions or data distributions also remains an open line of
work.

### Acknowledgements

This material is based upon work supported by the National Science Foundation
under Award No. 2202898. Thanks to Yiling Chen, Francisco Marmolejo Cossio,
Esther Rolf, and Arpita Biswas for feedback and comments, as well as
participants in the EC Gender Inclusion Workshop.

## References

  * Agarwal et al. [2019] A. Agarwal, M. Dudik, and Z. S. Wu.  Fair regression: Quantitative definitions and reduction-based algorithms.  In _International Conference on Machine Learning_ , pages 120–129. PMLR, 2019.
  * Arutjothi and Senthamarai [2017] G. Arutjothi and C. Senthamarai.  Prediction of loan status in commercial bank using machine learning classifier.  In _2017 International Conference on Intelligent Sustainable Systems (ICISS)_ , pages 416–419. IEEE, 2017.
  * Bechavod and Ligett [2017] Y. Bechavod and K. Ligett.  Penalizing unfairness in binary classification.  _arXiv preprint arXiv:1707.00044_ , 2017.
  * Berk et al. [2017] R. Berk, H. Heidari, S. Jabbari, M. Joseph, M. Kearns, J. Morgenstern, S. Neel, and A. Roth.  A convex framework for fair regression.  _arXiv preprint arXiv:1706.02409_ , 2017.
  * Blandin and Kash [2022] J. Blandin and I. Kash.  Fairness over utilities via multi-objective rewards.  2022\.
  * Brier et al. [1950] G. W. Brier et al.  Verification of forecasts expressed in terms of probability.  _Monthly weather review_ , 78(1):1–3, 1950.
  * Denis et al. [2021] C. Denis, R. Elie, M. Hebiri, and F. Hu.  Fairness guarantee in multi-class classification.  _arXiv preprint arXiv:2109.13642_ , 2021.
  * Do et al. [2022] H. Do, P. Putzel, A. S. Martin, P. Smyth, and J. Zhong.  Fair generalized linear models with a convex penalty.  In _International Conference on Machine Learning_ , pages 5286–5308. PMLR, 2022.
  * Donini et al. [2018] M. Donini, L. Oneto, S. Ben-David, J. S. Shawe-Taylor, and M. Pontil.  Empirical risk minimization under fairness constraints.  _Advances in Neural Information Processing Systems_ , 31, 2018.
  * Finocchiaro et al. [2019] J. Finocchiaro, R. Frongillo, and B. Waggoner.  An embedding framework for consistent polyhedral surrogates, 2019.  URL https://arxiv.org/abs/1907.07330.
  * Fissler [2017] T. Fissler.  _On higher order elicitability and some limit theorems on the Poisson and Wiener space_.  PhD thesis, 2017.
  * Frongillo and Kash [2014] R. Frongillo and I. Kash.  General truthfulness characterizations via convex analysis.  In _Web and Internet Economics_ , pages 354–370. Springer, 2014\.
  * Goel et al. [2018] N. Goel, M. Yaghini, and B. Faltings.  Non-discriminatory machine learning through convex fairness criteria.  In _Proceedings of the AAAI Conference on Artificial Intelligence_ , volume 32, 2018.
  * Goldstein et al. [2017] B. A. Goldstein, A. M. Navar, and R. E. Carter.  Moving beyond regression techniques in cardiovascular risk prediction: applying machine learning to address analytic challenges.  _European heart journal_ , 38(23):1805–1814, 2017\.
  * Hardt et al. [2016] M. Hardt, E. Price, and N. Srebro.  Equality of opportunity in supervised learning.  _Advances in neural information processing systems_ , 29, 2016.
  * Huang and Vishnoi [2019] L. Huang and N. Vishnoi.  Stable and fair classification.  In K. Chaudhuri and R. Salakhutdinov, editors, _Proceedings of the 36th International Conference on Machine Learning_ , volume 97 of _Proceedings of Machine Learning Research_ , pages 2879–2890. PMLR, 09–15 Jun 2019.  URL https://proceedings.mlr.press/v97/huang19e.html.
  * Jung et al. [2020] C. Jung, S. Kannan, C. Lee, M. Pai, A. Roth, and R. Vohra.  Fair prediction with endogenous behavior.  In _Proceedings of the 21st ACM Conference on Economics and Computation_ , pages 677–678, 2020.
  * Kamiran and Calders [2009] F. Kamiran and T. Calders.  Classifying without discriminating.  In _2009 2nd international conference on computer, control and communication_ , pages 1–6. IEEE, 2009.
  * Kamishima et al. [2012] T. Kamishima, S. Akaho, H. Asoh, and J. Sakuma.  Fairness-aware classifier with prejudice remover regularizer.  In _Joint European conference on machine learning and knowledge discovery in databases_ , pages 35–50. Springer, 2012.
  * Konstantinov and Lampert [2021] N. Konstantinov and C. H. Lampert.  Fairness through regularization for learning to rank.  _CoRR_ , abs/2102.05996, 2021.  URL https://arxiv.org/abs/2102.05996.
  * Kube et al. [2023] A. R. Kube, S. Das, and P. J. Fowler.  Community- and data-driven homelessness prevention and service delivery: Optimizing for equity.  _Journal of the American Medical Informatics Association_ , 30(6):1032–1041, 04 2023.  ISSN 1527-974X.  doi: 10.1093/jamia/ocad052.
  * Lambert [2018] N. S. Lambert.  Elicitation and evaluation of statistical forecasts.  2018\.  URL https://web.stanford.edu/ñlambert/papers/elicitability.pdf.
  * Lambert and Shoham [2009] N. S. Lambert and Y. Shoham.  Eliciting truthful answers to multiple-choice questions.  In _Proceedings of the 10th ACM conference on Electronic commerce_ , pages 109–118, 2009.
  * Lambert et al. [2008] N. S. Lambert, D. M. Pennock, and Y. Shoham.  Eliciting properties of probability distributions.  In _Proceedings of the 9th ACM Conference on Electronic Commerce_ , pages 129–138, 2008.
  * Lloyd-Jones [2010] D. M. Lloyd-Jones.  Cardiovascular risk prediction: basic concepts, current status, and future directions.  _Circulation_ , 121(15):1768–1777, 2010.
  * Mireshghallah et al. [2021] F. Mireshghallah, H. A. Inan, M. Hasegawa, V. Rühle, T. Berg-Kirkpatrick, and R. Sim.  Privacy regularization: Joint privacy-utility optimization in language models, 2021.
  * Pleiss et al. [2017] G. Pleiss, M. Raghavan, F. Wu, J. Kleinberg, and K. Q. Weinberger.  On fairness and calibration.  _Advances in neural information processing systems_ , 30, 2017.
  * Rahman [2021] R. Rahman.  Heart attack analysis and prediction dataset.  https://www.kaggle.com/datasets/rashikrahmanpritom/heart-attack-analysis-prediction-dataset, 2021\.  URL https://www.kaggle.com/datasets/rashikrahmanpritom/heart-attack-analysis-prediction-dataset.
  * Savage [1971] L. J. Savage.  Elicitation of personal probabilities and expectations.  _Journal of the American Statistical Association_ , 66(336):783–801, 1971.
  * Sheikh et al. [2020] M. A. Sheikh, A. K. Goel, and T. Kumar.  An approach for prediction of loan approval using machine learning algorithm.  In _2020 International Conference on Electronics and Sustainable Communication Systems (ICESC)_ , pages 490–494. IEEE, 2020.
  * Singh et al. [2021] V. Singh, A. Yadav, R. Awasthi, and G. N. Partheeban.  Prediction of modernized loan approval system based on machine learning approach.  In _2021 International Conference on Intelligent Technologies (CONIT)_ , pages 1–4. IEEE, 2021.
  * Steinwart et al. [2014] I. Steinwart, C. Pasin, R. Williamson, and S. Zhang.  Elicitation and Identification of Properties.  In _Proceedings of The 27th Conference on Learning Theory_ , pages 482–526, 2014.
  * Williamson and Menon [2019] R. Williamson and A. Menon.  Fairness risk measures.  In _International Conference on Machine Learning_ , pages 6786–6797. PMLR, 2019.
  * Zafar et al. [2017] M. B. Zafar, I. Valera, M. G. Rogriguez, and K. P. Gummadi.  Fairness constraints: Mechanisms for fair classification.  In _Artificial intelligence and statistics_ , pages 962–970. PMLR, 2017.

##  Appendix A Additional example of non-equivalent properties

###  A.1 Equalized FNR

Similarly, we consider false negative rates. Our objective is

$\displaystyle L^{FNR,\lambda}(\mathbf{t};\mathbf{s};\mathbf{p})$
$\displaystyle=\frac{1}{m}\sum_{i}L(t^{(i)},p^{(i)})+\lambda\left|\frac{1}{n_{a}}\sum_{i:s^{(i)}=a,t^{(i)}=0}p^{(i)}-\frac{1}{n_{b}}\sum_{i:s^{(i)}=b,t^{(i)}=0}p^{(i)}\right|$

Like the FPR regularizer, since the FNR regularizer computes the difference of
false negative rates between groups, one can observe that a way to reduce the
false negative rate of a group is to assign more positive treatments
$t^{(i)}=1$. Again, we see in figure 12 that the FNR regularizer then makes it
worse for an algorithm to assign the negative treatment to an agent $i$ even
if $p^{(i)}$ slightly less than $1/2$.

######  Corollary 5.

Let $L:[0,1]\times\\{0,1\\}\to[0,1]$ and $\psi:r\mapsto\mathbf{1}\\{r\geq
1/2\\}$ indirectly elicit the mode over $\mathcal{Y}=\\{0,1\\}$ such that
$L(y,y)=0$, and let
$\Theta^{FPR,\lambda}:=\psi\circ\mathrm{prop}[L^{FPR,\lambda}]$. Then
$\Theta^{FPR,\lambda}$ is not equivalent to the mode for $\lambda>0$.

Figure 12: Visualizing the level sets of the $FNR$-regularized property
$\Theta^{FNR,\lambda}$ for different values of $\lambda\in[0,1]$, where $m=2$
and $s=(a,b)$. Each point $(p^{(1)},p^{(2)})$ in a square represents
$(\Pr_{p^{(1)}}[Y=1],\Pr_{p^{(2)}}[Y=1])$, and each colored cell represents
sets of $(p^{(1)},p^{(2)})$ pairs such that the optimal treatment is the same
for all points in the cell. For example, the magenta cell is the set of
distributions where the decision-maker prefers to attribute the positive
treatment ($t^{(i)}=1$) to the agent in group $a$, and the negative treatment
($t^{(i)}=0$) to the agent in group $b$.