# Total energy calculation for the metallic hcp phase of Zn in the bulk,
layered, and quantum dot limits

D. Olguín111 Permanent address: Departamento de Física, Centro de
Investigación y de Estudios Avanzados del Instituto Politécnico Nacional, C.P.
07300, Ciudad de México, México

(Received November 12, 2020, in final form February 12, 2020)

###### Abstract

The structural and electronic properties of the metallic hcp phase of Zn in
the bulk, monolayer, bilayer, and quantum dot limits have been studied by
using total energy calculations. From our calculated density of states and
electronic band structure, in agreement with previous work, bulk hybridization
of the Zn–$4s$, $3p$, and $3d$ orbitals is obtained. Furthermore, we found
that this orbital hybridization is also obtained for the monolayer, bilayer,
and quantum dot systems. At the same time, we found that the Zn monolayer and
bilayer systems show electronic properties characteristic of lamellar systems,
while the quantum dot system shows the behavior predicted for a 0D system.

Key words: ab initio calculations, metallic Zn, quantum dots, 2D systems

###### Abstract

Äîñëäæåíî ñòðóêòóðí òà åëåêòðîíí âëàñòèâîñò ìåòàëíî¿ ãåêñàãîíàëüíî¿
ùëüíîóêëàäåíî¿ ôàçè Zn â îá’ì, áàãàòîøàðîâèõ ãðàíèöÿõ â ãðàíèö êâàíòîâî¿ òîêè,
âèêîðèñòîâóþè îáèñëåííÿ ïîâíî¿ åíåðã¿. Ç îáèñëåíü ãóñòèíè ñòàíâ åëåêòðîííî¿
çîííî¿ ñòðóêòóðè, ùî óçãîäæóþòüñÿ ç íàøèìè ïîïåðåäíìè ðåçóëüòàòàìè, îòðèìàíî
îá’ìíó ãáðèäèçàöþ $4s$, $3p$ $3d$ îðáòàëåé Zn. Êðì òîãî, ìè çíàéøëè, ùî
ãáðèäèçàöÿ îðáòàëåé òàêîæ îòðèìóòüñÿ äëÿ ìîíîøàðîâèõ, äâîøàðîâèõ ñèñòåì ñèñòåì
ç êâàíòîâîþ òîêîþ. Îäíîàñíî âñòàíîâëåíî, ùî ìîíîøàðîâ äâîøàðîâ ñèñòåìè Zn
âèÿâëÿþòü åëåêòðîíí âëàñòèâîñò, õàðàêòåðí äëÿ ëàìåëàðíèõ ñèñòåì, â òîé àñ, ÿê
ñèñòåìà êâàíòîâèõ òîîê äåìîíñòðó ïîâåäíêó, ïåðåäáàåíó äëÿ 0D ñèñòåìè.

Ключов слова: îáèñëåííÿ ç ïåðøèõ ïðèíöèïâ, ìåòàëíèé Zn, êâàíòîâ òîêè, 2D
ñèñòåìè

##  1 Introduction

The study of the electronic properties of quantum dots (QDs) is very
interesting for both basic and applied physics. Small enough quantum dots can
contain a few atoms, and their energy spectra are expected to behave like
those of “big artificial atoms”, showing discrete energy levels [1]. To the
best of our knowledge, the electronic and optical properties of semiconductor
quantum dots have been summarized in early and recent interesting reviews
found in the literature [2, 3, 4, 5]. By contrast, for metallic quantum dots,
a lack of work is evident. However, important reviews, mainly on metallic
clusters, can be found in [6, 7, 8, 9, 10, 11].

Metallic quantum dots, in their nanocomposite form, have been proposed for use
in memory devices [12, 13] and in coloring glasses. By using the multicolor
detection properties of semiconductors, optical devices applicable in
nanoscience and nanotechnology, as well as in chemistry, biology, and
medicine, can be developed [14, 16, 15, 17, 18, 19, 20, 21, 22, 23, 24]. As
far as we know, metallic Zn quantum dots have not been studied or reported in
the recent scientific literature. However, it is found that Zn clusters were
studied recently, where the reports show stable Zn clusters having different
stacking arrangement of the atoms as well as interesting electronic properties
[25, 26, 27, 28, 29, 30]. On the other hand, there is an increasing interest
in different Zn-based nanoparticles, the same for biomaterials, as well as in
solid state physics, and other technological applications [31, 32, 33, 34, 35,
36]. Therein, it has been showed that by using controlled conditions several
Zn-based nanocrystals, passivated or dopped, can be grown in the symmetric
zinc-blende phase [37, 38, 39, 40, 41]. It is also interesting to note the
report stating that at low temperature the system ZrZn2 shows ferromagnetic
and superconductor properties [42, 43].

Although, no reports on Zn metallic nanocrystals are known, recent work on Zn
nanodisks [44, 47, 46, 45] showed that Zn metallic nanodisks, with sizes of
520 nm wide and 144 nm thick, can exhibit a mixture of metallic and
semiconductor properties [44]. Thus, with the aim of contributing to the study
of the electronic properties of metallic quantum dots, in this work, we
present the study of the electronic band structure of metallic Zn quantum
dots. For completeness, calculations for the bulk and layered systems
(monolayer and bilayer cases) are also presented. To show the stability of the
systems studied, we have calculated the cohesive energy, and our found values
will be presented below.

Much of the theoretical work on modelling the quantum dots has been done under
the assumption that the finite thickness of a quantum dot, usually much
smaller than the lateral extension of the confinement, can be neglected.
However, the typical size of a quantum dot ranges from a few Angstroms to
micrometers [1, 5]. A complete analysis of the electronic properties of QDs
requires the study of their behavior as a function of the QD size. However,
these additional calculations, which require considerable computational
resources, are outside the scope of the present work.

The remainder of this paper is organized as follows. Section 2 describes the
computational strategy used in our calculations. In section 3, a discussion of
our results on the calculated cohesive energies and electronic properties for
the systems studied is presented. Finally, section 4 presents our conclusions.

##  2 Calculation method

In our calculations, we used the full-potential linearized augmented plane-
wave method (FP-LAPW) as implemented in the WIEN2K code [48]. In this method,
wave functions, charge density, and potential are expanded in spherical
harmonics within non-overlapping muffin-tin spheres, while plane waves are
used in the remaining interstitial region of the unit cell. In the code, the
core states are treated with a multiconfiguration relativistic Dirac-Fock
approach, whereas valence states are treated in a scalar relativistic
approach. The exchange-correlation energy is calculated using the GGA
correction of Perdew et al. [49, 50]. The atomic electronic configuration used
was Zn [Ar] $4s$ $3d$; where the Zn $3p$ states were treated as valence-band
states using the local orbital extension of the LAPW method [48].

To find the minimum of the total energy for each system as a function of the
variational parameters, the muffin-tin radius and the cutoff energy, a very
careful step analysis was performed, where for the bulk case a value of 11.0
for $R_{\rm MT}$$K_{\rm max}$ (where $R_{\rm MT}$ is the muffin-tin radius and
$K_{\rm max}$ is the plane wave cutoff) and a value of 13.0 Ry for the cutoff
energy parameter ($G_{\rm max}$) were used, while for the monolayer case the
values used were 10.5 for $R_{\rm MT}$$K_{\rm max}$ and 13.0 Ry for $G_{\rm
max}$, and for the bilayer case the values used were $R_{\rm MT}$$K_{\rm
max}$=10.5 and $G_{\rm max}$=14.0 Ry.

At the same time, in our calculations, an appropriate set of k-points in the
irreducible sector of the Brillouin zone was used. Here, we used a value of
1600 k-points for the bulk case, equivalent to a $(15\times 15\times 7)$
Monkhorst-Pack grid of the unit cell, and 2000 k-points for the layer systems,
equivalent to $(20\times 20\times 4)$ Monkhorst-Pack grid [51]. It should be
noted that for the quantum dots, the variational parameters used for the total
energy and electronic calculations were $R_{\rm MT}$$K_{\rm max}$=7.0, $G_{\rm
max}$=12.0 Ry, and only the Gamma point of the first Brillioun zone was used
[52, 53].

To study the Zn low dimensional systems, we used the slab model. Therein, for
the monolayer case, one Zn-atomic layer in the hcp lattice was constructed,
while for the bilayer case, we used two Zn-atomic layers in the hcp lattice
keeping the bulk hcp distance between the Zn layers, and for the QD we used a
finite slab containing thirteen atomic sites having obtained a pyramidal QD
type, whose dimensions were 5.32 Å for the basis and 2.65 Å for the height.
Here, we passivated the Zn-QD with Hydrogen atoms, since it had been showed
that the passivation stabilizes the bare QD [53]. In the inset of figures 3
and 5, the slab models used are shown schematically, where we can distinguish
the different atomic sites for metallic hcp Zn (red symbols).

To properly isolate each slab from their periodic images and to avoid large
vacuum zones, which, in plane-wave methods, considerably increase the demand
of computational resources, i.e., to keep the calculations accessible, based
on the fact that in bulk hcp zinc, the nearest neighbor distances range
between 2.66 Å and 2.91 Å for the in-plane and interlayer neighbors,
respectively, a vacuum of 10 Å with the upper slab was considered for the
layered systems, whereas a vacuum of 13 Å for the upper side and 8 Å for the
lateral side was used for the QD system.

##  3 Results

Before a detailed discussion of the electronic properties of Zn systems, we
first show our calculated structural properties of the bulk crystal. The Zn
hexagonal crystal structure is showed in the inset of figure 1. Our calculated
lattice parameters are $a=2.6712$ Å and $c=4.9586$ Å. In excelent agreement
with the experimental values: $a=2.6649$ Å and $c=4.9468$ Å [54].

Then, for the layered systems we relaxed the atomic positions in the slab
using the Hellmann-Feynman forces until the force component was less than 0.5
mRy/au. Our results show that, for the monolayer case, the geometry is rigid
and does not allow displacements for the in-plane nearest neighbor distances,
and for the bilayer case we found that the interlayer distances slightly
increase from 2.93 Å (the bulk value) to 2.98 Å. Meanwhile, for the QD case,
the system relaxes from 2.93 Å to 2.90 Å, and the in-plane nearest neighbor
distances remain almost constant, where the initial and final values are 2.66
Å and 2.67 Å, respectively. In order to show the trend in the stability of the
layered systems, we calculated the cohesive energy[55]

$E_{\rm coh}=\frac{N_{\rm Zn}E^{\rm free}_{i}-E^{\rm supercell}_{\rm
Tot}}{N_{\rm Zn}}\,,$

where, $E_{i}^{\rm free}$ is the total energy of the free $i$-th atom, $N_{\rm
Zn}$ the number of Zn atoms in the layer system, and $E_{\rm Tot}^{\rm
supercell}$ the total energy of the supercell. We found that the cohesive
energy for the bilayer case is 0.73 eV/atom, while for the monolayer case the
cohesive energy is 0.56 eV/atom.

To calculate the cohesive energy for the hydrogen passivated quantum dot, we
need to compute the binding energy

$E_{\rm binding}=N_{\rm Zn}E^{\rm free}_{\rm Zn}+N_{\rm H}E^{\rm free}_{\rm
H}-E^{\rm QD}_{\rm Tot},$

and then, by removing the Zn-H energy energy from the binding energy, we will
have the cohesive energy for the QD [56, 57], i.e.,

$E^{\text{QD}}_{\rm coh}=\frac{E_{\rm
binding}+\mu_{\text{H}}N_{\text{H}}}{N_{\rm Zn}}\,,$

where $\mu_{\text{H}}$ is the hydrogen chemical potential. From this
expression, we find that the cohesive energy for the QD is 0.29 eV/atom.
Considering the relatively low values for the cohesive energies, we found that
our calculated systems could be considered stable with weak chemical bonds.
Thus, from the experimental point of view, special attention will be necessary
to study these systems.

###  3.1 Electronic properties: Zn bulk

Figure 1: (Color online) The calculated partial density of states (PDOS) for
metallic hcp Zn. The left-hand side panel shows the $4s$ (red lines),
$3(p_{x}+p_{y})\ ({\rm blue\ lines})$ and $3p_{z}\ ({\rm green\ lines})$
orbitals; for comparison, the $3p$ obitals (black dashed lines), and the $3d$
orbitals (black lines) are also shown. The right-hand side panel shows the
$3d_{z^{2}}$ (green lines), $3(d_{x^{2}+y^{2}}+d_{xy})$ (red lines), and
$3(d_{xz}+d_{yz})$ (blue lines) orbitals; the inset shows the hcp unit cell.
The zero of the energies represents the Fermi level.

Figure 1 shows the calculated partial density of states (PDOS) for the bulk
hcp Zn. For the sake of clarity, the left-hand panel shows the Zn–$4s$ and
$3p$ orbitals, while the right-hand panel shows the Zn–$3d$ orbitals. The plot
shows that there is a non-zero population of states at the Fermi level, that
is, we have obtained a system with metallic characteristics, as expected.

From the plot, we can distinguish three energy regions where the calculated
states are located: below $E_{\text{F}}$, the first region with energies from
$-10.5$ to $-6.7$ eV, where the main contribution to the PDOS arises from the
$3d$ orbitals; states that are mainly located in the energy range from $-8.5$
eV to $-6.7$ eV; a second range of energies from $-2.9$ eV to 0.0 eV; and a
third energy region located above $E_{\text{F}}$, with energies ranging from
1.6 eV to 3.6 eV.

From the left-hand side panel and from the comparison of the different
calculated states shown, we can observe the already known hybridization of the
Zn–$4s$, $3p$, and $3d$ orbitals, in good agreement with earlier work [58, 59,
60, 61].

A detailed analysis of the calculated PDOS shows that, in addition to the
highly localized 3$d_{z^{2}}$ (green lines), the 3($d_{x^{2}+y^{2}}$+$d_{yx}$)
(red lines), and the 3($d_{xz}$+$d_{yz}$) orbitals (blue lines), the bands
whose peaks are located at $-8.25$ eV have important contributions from the
$4s$ orbitals. The main peaks for the $3(p_{x}+p_{y})$ orbitals (blue lines)
are located in the energy ranges from $-8.13$ eV to $-7.45$ eV and from $-2.9$
eV to $0.0$ eV, as well as at energies above $E_{\text{F}}$, from $1.6$ eV to
$3.6$ eV. The contributions of the $3p_{z}$ orbitals (green lines) are
obtained at $-7.8$ eV and $3.5$ eV.

Figure 2: (Color online) The calculated atomic orbital contributions to the
electronic band structure of bulk hcp Zn, where the orbital character of the
electronic bands is represented by circles (colors are a guide to the eye).
The left-hand panel shows the $4s$ orbitals, the middle panel shows the $3p$
orbitals, and the right-hand panel shows the $3d$ orbitals.

Further information on the calculated electronic states can be obtained from
the analysis of the atomic orbital contributions to the electronic band
structure, as shown in figure 2, where the atomic orbital contributions to the
electronic band structure are depicted by circles. A larger circle size
corresponds to a higher charge within the atomic sphere, qualitatively showing
those orbitals that constitute each electronic band [48].

Figure 2 (a) shows that the lower band located in the energy range from
$-10.5$ eV to $-8.0$ eV, showing high dispersion, originates from the $4s$
states (yellow circles) and hybridizes with the $3d$ orbitals. Figure 2 (a)
and figure 2 (b) show that in the energy range from –7.0 eV to 3.0 eV, the
electronic bands are a hybridization of the $4s$ and $3p$ states (magenta and
gray circles). Figure 2 (c) shows that the $3d$ orbitals are mainly localized
in the energy range from $-8.5$ eV to $-6.7$ eV, as we have commented above.

###  3.2 Zn-monolayer case and Zn-bilayer case

Figure 3 shows our calculated PDOS for the layered systems, where the upper
panel depicts the results for the Zn-monolayer case and the lower panel
depicts those for the Zn-bilayer case. As in the bulk case, we see that both
systems have metallic characteristics.

At the same time, our results show three energy regions where the calculated
states are located. Two of these energy regions are below $E_{\text{F}}$, the
lower one in the energy range from approximately $-9.2$ eV to $-6.8$ eV, where
the $3d$ states are located, and the second one in the energy range from
$-6.4$ eV to $0.9$ eV, where the $4s$ and $3p$ states are located. Above
$E_{\text{F}}$, a third energy range from 0.9 eV to 3.4 eV is obtained, where
a second set of $4s$ and $3p$ states is located.

Figure 3: (Color online) Same as figure 1 but for the Zn-monolayer case, upper
panel, and the Zn-bilayer case, lower panel. The inset in the right-hand panel
shows a schematic of the slab model used in our calculations.

Using the bulk values as a reference, a detailed comparison of the calculated
energies for the layered systems shows that the $3d$ states are located in the
energy range from $-8.5$ ($-9.2$) eV to $-6.2$ ($-6.8$) eV (where the first
values refer to the monolayer case and the values in parentheses correspond to
the bilayer case), almost at the same energy values already found for the bulk
case.

By contrast, the calculated values for the $4s$ and $3p$ states move up
slightly in energy. For example, the main peak of the lower $4s$ bands is now
located at $-7.7$ ($-7.8$) eV. Furthermore, a second peak for the $4s$ bands
is obtained at $-6.8$ ($-6.8$) eV; although this band is also present in the
bulk case, it is more noticeable in the layered cases. Additionally, in the
energy range from $-6.4$ eV to 0.6 eV, a new shape of the $4s$ orbitals is
obtained; here, these states change from a constant step-like form to a
sawtooth form (compare with figure 1). At energies above $E_{\text{F}}$ we
obtain three peaks arising from the $4s$ states, one peak located at 1.3 (1.3)
eV, another located at 1.8 (1.8) eV, and a third located at 2.5 eV. For
comparison, note that in the bulk case, only one band is obtained, namely, the
one located at 1.6 eV.

Figure 4: (Color online) Same as figure 2 but for the Zn monolayer (upper
panel) and bilayer (lower panel) cases.

The $3p$ orbitals also changed in shape compared with the bulk case; here, the
contribution of the $3(p_{x}+p_{y})$ orbitals (blue lines) to the PDOS in the
energy range from $-7.7$ ($-7.6$) eV to $-6.8$ ($-7.2$) eV seems to be
remarkable. There are other noticeable peaks at $-0.8$ ($-1.6$) eV and $-0.3$
($-0.8$) eV along with the other peaks in the conduction band at 0.7 (0.8) eV,
1.8 (1.8) eV, and 2.5 (1.8) eV. The contributions of the $3p_{z}$ orbitals
(green lines) are obtained at the energy range from $-1.5$ eV to 3.3 eV,
showing a step-like shape at these energies.

The obtained changes in the PDOS can also be observed from the calculated
atomic orbital contributions to the electronic band structure, as shown in
figure 4, where, due to the different coordination in the apical direction
between the layered and bulk cases, the number of bands obtained for the
monolayer case is half that for the bilayer and bulk cases.

The calculated energy differences for the $4s$ and $3p$ states can be
summarized as follows: in the layered cases, the $4s$ states located in the
energy range from $-8.5$ ($-9.3$) eV to $-7.3$ ($-7.3$) eV show less
dispersion than those in the bulk case. Additionally, in the energy range from
$-6.4$ ($-7.0$) eV to $0.0$ eV, the calculated energy values at the high
symmetry points move up in energy for the layered systems (compare figure 2
and figure 4).

Finally, we conclude that, except for the differences in the calculated energy
values, in general, the calculated electronic properties of the layered
systems show a pattern similar to that found for the bulk case, revealing that
the in-plane coordination produces lamellar systems, an interesting property
of several 2D systems that are the object of recent and intensive work [62,
63, 65, 64, 66, 67, 68].

###  3.3 Zn quantum dot

Figure 5 shows the calculated PDOS for the hydrogen-passivated Zn quantum dot
case. For the different peaks associated with the energies found, both below
and above $E_{\text{F}}$, the PDOS shows the structure predicted for a box-
like quantum dot [69, 70, 71, 72], namely, a delta function behavior; i.e.,
our results show the effects of quantum confinement [73].

Figure 5: (Color online) Same as figure 1 but for Zn QDs. The inset in the
right-hand side panel depicts a schematic representation of the slab used in
our calculations, arranged in the Zn hcp phase, which consist of 13 Zn atoms
and 48 H atoms (red and green symbols, respectively).

Furthermore, as a consequence of the atomic electronic configuration used in
our calculations and due to the strong coupling between the atoms that form
the QD, the obtained PDOS clearly shows the hybridization of the Zn–$4s$,
$3p$, and $3d$ atomic orbitals.

A detailed analysis of the calculated PDOS shows that although the Zn–$3d$
orbitals are mainly localized in the energy range from $-8.2$ eV to $-6.2$ eV,
non-negligible contributions of these orbitals to the PDOS over the entire
energy range shown in the plot are also found.

Figure 6: (Color online) The calculated electronic bands for hcp Zn QDs.
Colors are a guide to the eye.

At the same time, for the different energies shown, important contributions
from the $4s$ states (red lines) and $3p$ states (blue and green lines) are
also obtained, showing the above-mentioned hybridization of the Zn–$4s$, $3p$,
and $3d$ orbitals.

As in the previous cases, we can use the electronic band structure to
complement the analysis of the calculated PDOS. The calculated electronic band
structure for the QD case, depicted in figure 6, shows discrete energy levels,
which correspond to the atom-like PDOS shown in figure 5. From the plot, it is
easy to see how the energy levels are grouped. The lower energy states,
located in the energy range from $-7.8$ to $-6.2$ eV, correspond to the $3d$
orbitals that hybridize with the $4s$ and $3p$ orbitals. In the energy range
from $-5.5$ to $-4.2$ eV, we obtain an isolated set of energy levels that
correspond mainly to the $4s$ and $3p$ states found in the PDOS. In the energy
range from $-3.4$ to 2.2 eV, we obtain a group of $4s$ and $3p$ states forming
a dense set of energy levels, and the final energy range, from 3.0 eV to 4.0
eV, shows the last set of energy levels, whose main contribution arises from
the $3p$ orbitals.

Although it is found that low-dimensional zinc oxides, zinc chalcogenides, and
other zinc-based systems have been more studied than the low dimensional Zn
systems [74, 75, 76, 77, 78, 79], considering the lack of information found
for the layered systems and metallic quantum dots, we hope that our study will
be useful and motivate future work.

At the same time, to avoid speculations and compare our results with new and
future data, the same for the layered systems as well as for the quantum dot,
more work will be necessary in order to study the convergence of the total
energy as a function of the quantum dot size, the hydrogen passivation,
surface effects, and other model parameters of the supercell method used.

##  4 Conclusions

We have studied the electronic properties of the metallic hcp phase of Zn in
the bulk and low-dimensional monolayer, bilayer, and quantum dot systems from
first-principles calculations. For the bulk case, our results reproduce the
previously published data on the structural and electronic properties. For the
monolayer and bilayer cases, our results show that the calculated systems
produce lamellar systems, an interesting property of novel 2D systems that
attracts attention of the scientific community. By contrast, for the quantum
dot system, we found that our calculated DOS reproduces the predicted atom-
like behavior of 0D systems. Our calculated cohesive energy for the studied
systems, namely 0.73 eV/atom for the bilayer case, 0.56 eV/atom for the
monolayer case, and 0.29 eV/atom for the QD, show that although our calculated
values are smaller the systems are stable. Considering the interest in and
prospects for application of the different layered systems as well QDs, we
hope that our work motivates future research on these and other non-
traditional metallic low-dimensional systems.

## Acknowledgements

Computational resources from ABACUS-Cinvestav, Conacyt grant
EDOMEX-2011-COI-165873, and Xiuhcoatl-Cinvestav are gratefully acknowledged.

## References

  * [1] Kastner M.A., Phys. Today, 1993, 46, 24–31, doi:10.1063/1.881393.
  * [2] Lehtonen O., Sundholm D., Vänskä T., Phys. Chem. Chem. Phys., 2008, 10, 4535–4550,
doi:10.1039/b804212h.

  * [3] Boxberg F., Tulkki J., Rep. Prog. Phys., 2007, 70, 1425–1471, doi:10.1088/0034-4885/70/8/R04.
  * [4] Zwanenburg F.A., Dzurak A.S., Morello A., Simmons M.Y., Hollenberg Ll.C.L., Klimeck G., Rogge S., Coppersmith S.N., Eriksson M.A., Rev. Mod. Phys., 2013, 85, 961–1019, doi:10.1103/RevModPhys.85.961.
  * [5] Reimann S.M., Manninen M., Rev. Mod. Phys., 2002, 74, 1283–1342, doi:10.1103/RevModPhys.74.1283.
  * [6] Brack M., Rev. Mod. Phys., 1993, 65, 677–732, doi:10.1103/RevModPhys.65.677.
  * [7] Cheshnovsky O., Taylor K.J., Conceicao J., Smalley R.E., Phys. Rev. Lett., 1990, 64, 1785–1788,
doi:10.1103/PhysRevLett.64.1785.

  * [8] De Heer W.A., Rev. Mod. Phys., 1993, 65, 611–676, doi:10.1103/RevModPhys.65.611.
  * [9] Furche F., Ahlrichs R., Weis P., Jacob C., Gilb S., Bierweiler T., Kappes M.M., J. Chem. Phys., 2002, 117, 6982–6990, doi:10.1063/1.1507582.
  * [10] Häberlen O.D., Chung S.-Ch., Stener M., Rösch N., J. Chem. Phys., 1997, 106, 5189–5201, doi:10.1063/1.473518.
  * [11] Taylor K.J., Pettiette-Hall C.L., Cheshnovsky O., Smalley R.E., J. Chem. Phys., 1992, 96 3319–3329,
doi:10.1063/1.461927.

  * [12] Lin C.W., Pan T.S., Chen M.C., Yang Y.J., Tai Y., Chen Y.F., App. Phys. Lett., 2011, 99, 023303,
doi:10.1063/1.3605596.

  * [13] Jin Z., Liu G., Wang J., AIP Adv., 2013, 3, 052113, doi:10.1063/1.4804948.
  * [14] Zhang J., Sheng J., Int. J. Hydrogen Energy, 2009, 34, 3531–3534, doi:10.1016/j.ijhydene.2009.02.032.
  * [15] Hatef A., Sadeghi S.M., Boulais E., Meunier M., Nanotechnol., 2013, 24, 015502,
doi:10.1088/0957-4484/24/1/015502.

  * [16] Kang M., Lee H., Kang T., Kim B., J. Mater. Sci. Technol., 2015, 31, 573–580,
doi:10.1016/j.jmst.2015.01.007.

  * [17] Zhang J., Sheng J., J. Nano Res., 2015, 32, 66–70, doi:10.4028/www.scientific.net/JNanoR.32.66.
  * [18] Olejnik M., Krajnik B., Kowalska D., Lin G., Mackowski S., J. Phys.: Condens. Matter, 2013, 25,
194103 (11 pages), doi:10.1088/0953-8984/25/19/194103.

  * [19] Sheng J., Chen S., Zhang J., Li J., Yu J., Int. J. Hydrogen Energy, 2009, 34, 1119–1122,
doi:10.1016/j.ijhydene.2008.10.063.

  * [20] Dražić M.S., Cerovski V.Z., Žikić R., Phys. Status Solidi B, 2014, 251, 1438–1450, doi:10.1002/pssb.201350243.
  * [21] Shahid R., Muhammad N., Gonfa G., Toprak M.S., Muhammed M., J. Phys. Chem. Solids, 2015, 85, 34–38, doi:10.1016/j.jpcs.2015.04.020.
  * [22] Dai W., Dong H., Fugetsu B., Cao Y., Lu H., Ma X., Zhang X., 2015, 11, 4158–4164,
doi:10.1002/smll.201500208.

  * [23] Haughn C.R., Steenbergen E.H., Bissell L.J., Chen E.Y., Eyink K.G., Zide J.M.O., Doty M.F., Appl. Phys. Lett., 2014, 105, 103108, doi:10.1063/1.4895519.
  * [24] Frade T., Melo M.E., Gomes A., Surf. Coat. Technol., 2012, 206, 3459–3466,
doi:10.1016/j.surfcoat.2012.02.012.

  * [25] Piotrowski M.J., Piquini P., Da Silva J.L.F., Phys. Rev. B, 2010, 81, 155446, doi:10.1103/PhysRevB.81.155446.
  * [26] Wang J., Wang G., Zhao J., Phys. Rev. A, 2003, 68, 013201, doi:10.1103/PhysRevA.68.013201.
  * [27] Iokibe K., Tachikawa H., Azumi K., J. Phys. B: At. Mol. Opt. Phys., 2007, 40, 427–436,
doi:10.1088/0953-4075/40/2/015.

  * [28] Gutsev G.L., Weatherford C.W., Belay K.G., Ramachandran B.R., Jena P., J. Chem. Phys., 2013, 138, 164303, doi:10.1063/1.4799917.
  * [29] Aguado A., Vega A., Lebon A., Issendorff B., Nanoscale, 2018, 10, 19162–19181, doi:10.1039/c8nr05517c.
  * [30] Aguado A., Vega A., Lebon A., Issendorff B., Angew. Chem. Int. Ed., 2015, 54, 2111–2115,
doi:10.1002/anie.201409835.

  * [31] Ali A., Phull A.R., Zia M., Nanotechnol Rev., 2018, 7, 413–441, doi:10.1515/ntrev-2018-0067.
  * [32] Su Y., Cockerill I., Wang Y., Qin Y.X., Chang L., Zheng Y., Zhu D., Trends Biotechnol., 2018, 37, 428–441,
doi:10.1016/j.tibtech.2018.10.009.

  * [33] Su Y., Wang K., Gao J., Yang Y., Qin Y.X., Zheng Y., Zhu D., Acta Biomater., 2019, 98, 174–185,
doi:10.1016/j.actbio.2019.03.055.

  * [34] Darroudi M., Nasab N.K., Salimizand H., Dehnad A., Nanomed. J., 2019, 6, 258–262,
doi:10.22038/nmj.2019.06.000003.

  * [35] Seyedmajidi S.A., Seyedmajidi M., Moghadamnia A., Haghanifar S., Ziaei R., Zahedpasha S., Halalkhor S., Dent. Res. J., 2014, 11, 475–480.
  * [36] Cerovic A., Miletic I., Sobajic S., Blagojevic U., Radusinovic M., El-Sohemy A., Biol. Trace Elem. Res., 2007, 116, 61–71, doi:10.1007/BF02685919.
  * [37] Pal S., Goswami B., Sarkar P., J. Phys. Chem., 2001, 111, 16072–16075, doi:10.1021/jp074950j.
  * [38] Karanikolos G.N., Alexandridis P., Mountziaris T.J., Mater. Sci. Eng. B, 2008, 152, 66–71,
doi:10.1016/j.mseb.2008.06.028.

  * [39] Goswami B., Pal S., Sarkar P., Seifert G., Springborg M., Phys. Rev. B, 2006, 73, 205312,
doi:10.1103/PhysRevB.73.205312.

  * [40] Goswami B., Pal S., Sarkar P., Phys. Rev. B, 2007, 76, 045323, doi:10.1103/PhysRevB.76.045323.
  * [41] Christian P., Liu E., Polyhedron, 2010, 29, 691–696, doi:10.1016/j.poly.2009.10.003.
  * [42] Day C., Phys. Today, 2001, 54, 16–19, doi:10.1063/1.1420499.
  * [43] Pfleiderer C., Uhlarz M., Hayden S.M., Vollmer R., Löhneysen H.v., Bernhoeft N.R., Lonzarichk G.G., Nature, 2001, 412, 59–61, doi:10.1038/35083531.
  * [44] Lin J.H., Huang Y.J., Su Y.P., Liu C.A., Devan R.S., Ho C.H., Wang Y.P., Lee H.W., Chang C.M., Liou Y., Ma Y.R., RSC Adv., 2012, 2, 2123–2127, doi:10.1039/c2ra00972b.
  * [45] López R., Vigueras-Santiago E., Hernández-López S., Acuña P., Argueta-Vega A., Colín-Becerril N., Villa-Sánchez G., Rosales-Dávalos J., Rev. Mex. Fis., 2017, 63, 308–313.
  * [46] Lin J.H., Patil R.A., Devan R.S., Liu Z.A., Wang Y.P., Ho C.H., Liou Y., Ma Y.R., Sci. Rep., 2014, 4, 6967, doi:10.1038/srep06967.
  * [47] Devan R.S., Lin J.H., Huang Y.J., Yang Ch.Ch., Wu S.Y., Liou Y., Ma Y.R., Nanoscale, 2011, 3, 4339–4345,
doi:10.1039/c1nr10694e.

  * [48] Blaha P., Schwarz K., Luitz J., WIEN97, Vienna University of Technology, 1997.
  * [49] Perdew J.P., Wang Y., Phys. Rev. B, 1992, 45 13244–13249, doi:10.1103/PhysRevB.45.13244
  * [50] Perdew J.P., Burke S., Ernwerhof M., Phys. Rev. Lett., 1996, 77 3865–3868,
doi:10.1103/PhysRevLett.77.3865.

  * [51] Monkhorst H.J., Pack J.D., Phys. Rev. B, 1976, 13, 5188–5192, doi:10.1103/PhysRevB.13.5188.
  * [52] Xi Y., Zhao X., Wang A., Wang X., Bu H., Zhao M., Physica E, 2013, 49, 52–60,
doi:10.1016/j.physe.2013.01.003.

  * [53] Yamijala S.S.R.K.C., Bandyopadhyay A., Pati S.K., Chem. Phys. Lett., 2014, 603, 28–32,
doi:10.1016/j.cplett.2014.04.025.

  * [54] Jette E., J. Chem. Phys., 1935, 3, 605–616, doi:10.1063/1.1749562.
  * [55] Wang D., Sun Z., Han D., Liu L., Niu L., RSC Adv., 2017, 7, 11834–11839, doi:10.1039/c7ra00483d.
  * [56] Zdetsis A.D., J. Phys. Chem. C, 2011, 115, 14507–14516, doi:10.1021/jp2023007.
  * [57] Zdetsis A.D., Koukaras E.N., Garoufalis C.S., Appl. Phys. Lett., 2007, 91, 203112, doi:10.1063/1.2813019.
  * [58] Daniukt S., Jarlborgj T., Kontrym-Sznajdt G., Majsnerowskit J., Stachowiakt H., J. Phys.: Condens. Matter, 1989, 1, 8397–8406, doi:10.1088/0953-8984/1/44/011.
  * [59] Singh P.P., Phys. Rev. Lett., 1994, 72, 2446–2449, doi:10.1103/PhysRevLett.72.2446.
  * [60] Blaha P., Schwarz K., Dederichs P.H., Phys. Rev. B, 1988, 38, 9368–9364, doi:10.1103/PhysRevB.38.9368.
  * [61] Sha X.W., Papaconstantopoulos D.A., Mehl M.J., Bernstein N., Phys. Rev. B, 2011, 84, 184109,
doi:10.1103/PhysRevB.84.184109.

  * [62] Ullah N., Zhang R.Q., Murtza G., Yar A., Mahmood A., Solid State Comm., 2016, 246, 54–58,
doi:10.1016/j.ssc.2016.07.005.

  * [63] Singh N., Kaloni T.P., Schwingenschögl U., App. Phys. Lett., 2013, 102, 023101, doi:10.1063/1.4781382.
  * [64] Rudenko A.N., Katsnelson M.I., Phys. Rev. B, 2014, 89, 201408(R), doi:10.1103/PhysRevB.89.201408.
  * [65] Kaloni T.P., Gangopadhyay S., Singh N., Jones B., Schwingenschögl U., Phys. Rev. B, 2013, 88,
235418, doi:10.1103/PhysRevB.88.235418.

  * [66] Zhang S., Yan Z., Li Y., Chen Z., Zeng H., Angew. Chem. Int. Ed., 2015, 54, 3112–3115,
doi:10.1002/anie.201411246.

  * [67] Xian L., Pérez Paz Bianco E., Ajayan P.M., Rubio A., 2D Mater., 2017, 4, 041003 (7 pages),
doi:10.1088/2053-1583/aa8418.

  * [68] Zhou J., Sumpter B.G., Kent P.R.C., Huang J., ACS App. Mater. Interfaces, 2015, 7, 1458–1464,
doi:10.1021/am505655m.

  * [69] Arakawa Y., Sakaki H., Appl. Phys. Lett., 1982, 40, 939–941, doi:10.1063/1.92959.
  * [70] Batabyal R., Dev B.N., Physica E, 2014, 64, 224–233, doi:10.1016/j.physe.2014.07.017.
  * [71] Lee S.J., Shin N.H., Ko J.J., Parkt M.J., Kiimmel R., Semicond. Sci. Technol., 1992, 7,
1072–1079, doi:10.1088/0268-1242/7/8/008.

  * [72] Nilius N., Wallis T.M., Ho W., Science, 2002, 297, 1853–1856, doi:10.1126/science.1075242.
  * [73] Harman T.C., Taylor P.J., Walsh M.P., LaForge B.E., Science, 2002, 297, 2229–2232,
doi:10.1126/science.1072886.

  * [74] Guo C.F., Wang Y., Jiang P., Cao S., Miao J., Zhang Z., Liu Q., Nanotechnology, 2008, 19,
445710 (8 pages), doi:10.1088/0957-4484/19/44/445710.

  * [75] Azpiroz J.M., Mosconi E., Ugalde J.M., de Angelis F., J. Phys. Chem. C, 2014, 118,
3274–3284, doi:10.1021/jp409182r.

  * [76] Kumar H., Barman P.B., Singh R.R., Int. J. Sci. Eng. Res., 2014, 5, No. 5, 40–53.
  * [77] Son D.I., Kwon B.W., Park D.H., Seo W.S., Yi Y., Angadi B., Lee Ch.-L., Choi W.K., Nat. Nanotechnol., 2012, 7, 465–471, doi:10.1038/NNANO.2012.71.
  * [78] De Queiroz A.A.A., Martins M., Soares D.A.W., Franca E.J., J. Mol. Struct., 2008, 873, 121–129,
doi:10.1016/j.molstruc.2007.03.013.

  * [79] Fang X., Zhai T., Gautam U.K. , Li L., Wu L., Bando Y., Golberg D., Prog. Mater Sci., 2011, 56, 175–287,
doi:10.1016/j.pmatsci.2010.10.001.

Ukrainian 0

Îáèñëåííÿ ïîâíî¿ åíåðã¿ äëÿ ìåòàëíî¿ ãåêñàãîíàëüíî¿ ùëüíîóêëàäåíî¿ ôàçè Zn â
îá’ì, áàãàòîøàðîâèõ ãðàíèöÿõ â ãðàíèö êâàíòîâî¿ òîêè Ä. Îëãóí

Öåíòð äîñëäæåíü òà ïäâèùåííÿ êâàëôêàö¿ Íàöîíàëüíîãî ïîëòåõííîãî íñòèòóòó —
ïäðîçäë Êåðåòàðî, ì. Ñàíòüÿãî äå Êåðåòàðî, Ìåêñèêà


# Robust Fairness-aware Learning Under Sample Selection Bias

Wei Du &Xintao Wu University of Arkansas
{wd005<EMAIL_ADDRESS>
###### Abstract

The underlying assumption of many machine learning algorithms is that the
training data and test data are drawn from the same distributions. However,
the assumption is often violated in real world due to the sample selection
bias between the training and test data. Previous research works focus on
reweighing biased training data to match the test data and then building
classification models on the reweighed training data. However, how to achieve
fairness in the built classification models is under-explored. In this paper,
we propose a framework for robust and fair learning under sample selection
bias. Our framework adopts the reweighing estimation approach for bias
correction and the minimax robust estimation approach for achieving robustness
on prediction accuracy. Moreover, during the minimax optimization, the
fairness is achieved under the worst case, which guarantees the model’s
fairness on test data. We further develop two algorithms to handle sample
selection bias when test data is both available and unavailable. We conduct
experiments on two real-world datasets and the experimental results
demonstrate its effectiveness in terms of both utility and fairness metrics.

##  1 Introduction

Traditional supervised machine learning assumes that training data and test
data are independently and identically distributed (iid), i.e., each example
$t$ with pairs of feature input $x$ and label $y$ drawn from the same
distribution $\mathcal{Q}=P(x,y)$. The conditional label distribution,
$P(y|x)$, is estimated as $\hat{P}(y|x)$ (aka, a classifier $f(x)$) from the
given training dataset $\mathcal{D}_{s}$. Similarly, in the fair machine
learning, we aim to learn a fair classifier $f(x,a)$ from the training dataset
drawn from $\mathcal{Q}=P(x,a,y)$ where $a$ is a protected attribute such as
gender or race. However, when the distributions on training and test data sets
do not match, we are facing sample selection bias or covariate shift. Sample
selection bias occurs frequently in reality — the available training data have
been collected in a biased manner whereas the test is instead performed over a
more general population. The classifier $f$ simply learned from the training
dataset is vulnerable to sample selection bias and will incur more accuracy
loss over test data. Moreover, the fair classifier trained with the biased
data cannot guarantee fairness over test data. This is a serious concern when
it is critical and imperative to achieve fairness in many applications.

In this paper, we develop a framework for robust and fair learning under
sample selection bias. We embrace the uncertainty incurred by sample selection
bias by producing predictions that are both fair and robust in test data. Our
framework adopts the reweighing estimation approach for bias correction and
the minimax robust estimation approach for achieving robustness on prediction
accuracy. Moreover, during the minimax optimization, the fairness is achieved
under the worst case, which guarantees the model’s fairness on test data. To
address the intractable issue, we approximate the fairness constraint using
the boundary fairness and combine into the classifier’s loss function as a
penalty. The modified loss function is minimized in view of the most adverse
distribution within a Wasserterin ball centered at the empirical distribution
of the training data.

We present two algorithms, RFLearn1 for the scenario where the unlabeled test
dataset $\mathcal{D}$ is available, and RFLearn2 for the scenario where
$\mathcal{D}$ is unavailable. In RFLearn1, we estimate the sample selection
probability via its density ratio of training data and test data and then
correct the bias in loss function.

In RFLearn2, we introduce some natural assumptions, i.e., the samples in the
same cluster have the same selection probability which is within a range from
the uniform selection probability. The algorithm first clusters the training
data and robustifies the sample selection probability estimation of each
cluster within a Wasserterin ball. We test our algorithms on two real-world
datasets and experimental results demonstrate that our algorithms can achieve
both good performance on prediction and fairness.

##  2 Problem Formulation

We first define notations throughout this paper. Let $X$ denote the feature
space, $A$ the protected attribute, and $Y$ the label set. Let $\mathcal{Q}$
denote the true distribution over $X\times A\times Y$ according to which test
samples $t=(x,a,y)$ are drawn.

For simplicity, we assume both $y$ and $a$ are binary where $y=1$ ($0$)
denotes the favorable (unfavorable) decision and $a=1$ ($0$) denotes the
majority (minority) group. Under the sample selection bias setting, the
learning algorithm receives a training dataset $\mathcal{D}_{s}$ of
$N_{\mathcal{D}_{s}}$ labeled points $t_{1},\cdots,t_{N_{\mathcal{D}_{s}}}$
drawn according to a biased distribution $\mathcal{Q}_{s}$ over $X\times
A\times Y$. This sample bias can be represented by a random binary variable
$s$ that controls the selection of points, i.e., $s=1$ for selected and $s=0$
otherwise. We consider the problem of building over $\mathcal{D}_{s}$ a fair
classifier $f$ assigning labels $\hat{Y}\in\\{0,1\\}$ that depends only on
$X,A$ (so $\hat{Y}\perp(Y,S)|X,A$) under certain fairness constraints. Many
fairness notions were proposed in the literature, such as demographic parity
and mistreatment parity. In this paper, we choose the classic demographic
parity and use risk difference ($RD$) as the fairness quantity. In short, $RD$
measures the difference of the positive predictions between the majority group
and minority group.

Problem Formulation 1 (Fair Classifier Under Sample Selection Bias) With the
observed $\mathcal{D}_{s}$, how to construct a fair classifier $f$ that
minimizes the expected loss $\mathbb{E}_{(x,a,y)\in\mathcal{Q}}[l(f(x,a),y)]$
subject to $|RD(\mathcal{Q})|\leq\tau$ where $l$ is the loss function,
$RD(\mathcal{Q})$ is the risk difference over distribution $\mathcal{Q}$,
i.e.,
$RD(\mathcal{Q})=|P_{\mathcal{Q}}(\hat{y}=1|a=1)-P_{\mathcal{Q}}(\hat{y}=1|a=0)|$,
and $\tau\in[0,1]$ is a threshold for the fairness constraint.

##  3 Fair Classifier under Sample selection bias

The probability of drawing $t=(x,a,y)$ according to the true but unobserved
distribution $\mathcal{Q}$ is straightforwardly related to the observed
distribution $\mathcal{Q}_{s}$. By definition of the random selection variable
$s$, the observed biased distribution $\mathcal{Q}_{s}$ can be expressed by
$P_{\mathcal{Q}_{s}}(t)=P_{\mathcal{Q}}(t|s=1)$ or
$P_{\mathcal{Q}_{s}}(x,a,y)=P_{\mathcal{Q}}(x,a,y|s=1)$. Assuming
$P(s=1|x,a)\neq 0$ for all $t\in X\times A\times Y$, by the Bayes formula, we
have

$P_{\mathcal{Q}}(t)=\frac{P(t|s=1)P(s=1)}{P(s=1|x,a)}=\frac{P(s=1)}{P(s=1|x,a)}P_{\mathcal{Q}_{s}}(t)$
(1)

Hence, if we define and construct the new distribution $\hat{\mathcal{Q}}_{s}$
as $\frac{P(s=1)}{P(s=1|t)}{\mathcal{Q}_{s}}$, i.e.,
$P_{\hat{\mathcal{Q}}_{s}}(x,a,y)=\frac{P(s=1)}{P(s=1|x,a,y)}P_{\mathcal{Q}_{s}}(x,a,y)$,
we have the following result.

$\displaystyle\mathbb{E}_{(x,a,y)\in\hat{\mathcal{Q}}_{s}}[l(f(x,a),y)]=\sum_{x,a,y}l(f(x,a),y))P_{\hat{\mathcal{Q}}_{s}}(x,a,y)$
$\displaystyle=\sum_{x,a,y}{{l(f(x,a),y))}\frac{P(s=1)}{P(s=1|x,a,y)}P_{{\mathcal{Q}_{s}}}(x,a,y)}$
$\displaystyle=\sum_{x,a,y}{{l(f(x,a),y))}\frac{P(s=1)}{P(s=1|x,a,y)}P_{{\mathcal{Q}}}(x,a,y|s=1)}$
$\displaystyle=\sum_{x,a,y}{{l(f(x,a),y))}\frac{P(s=1)}{P(s=1|x,a,y)}\frac{P_{\mathcal{Q}(s=1|x,a,y)}P_{\mathcal{Q}}(x,a,y)}{P_{\mathcal{Q}}(s=1)}}$
$\displaystyle=\sum_{x,a,y}l(f(x,a),y))P_{\mathcal{Q}}(x,a,y)$
$\displaystyle=\mathbb{E}_{(x,a,y)\in\mathcal{Q}}[l(f(x,a),y)]$

Similarly we have $RD(\hat{\mathcal{Q}}_{s})=RD(\mathcal{Q})$. Equivalently,
if we define and construct a modified training dataset $\hat{\mathcal{D}}_{s}$
by introducing a weight $\frac{P(s=1)}{P(s=1|x,a,y)}$ to each record
$t\in\mathcal{D}_{s}$, we can approximate
$\mathbb{E}_{(x,a,y)\in\hat{\mathcal{Q}}_{s}}[l(f(x,a),y)]$ using
$\mathbb{E}_{(x,a,y)\in\hat{\mathcal{D}}_{s}}[l(f(x,a),y)]$, which can be
expressed as
$\frac{1}{N_{\mathcal{D}_{s}}}\sum_{i=1}^{N_{\mathcal{D}_{s}}}\frac{P(s=1)}{P(s=1|t)}l(f(x_{i},a_{i}),y_{i})$.

If we can derive the probabilities $P(s=1)$ and $P(s=1|x,a)$, we could derive
the true probability $P_{\mathcal{Q}}$ from the biased one
$P_{\mathcal{Q}_{s}}$ exactly and correct the sample selection bias in fair
classification.

######  Theorem 1.

Under sample selection bias, the classifier $f$ that minimizes
$\mathbb{E}_{(x,a,y,s)\in\hat{\mathcal{D}}_{s}}[l(f(x,a),y)]$ subject to
$RD(\hat{\mathcal{D}}_{s})\leq\tau$ is a fair classifier.

Note that the classifier $f$ which tries to minimize
$\frac{1}{N_{\mathcal{D}_{s}}}\sum_{i=1}^{N_{\mathcal{D}_{s}}}l(f(x_{i},a_{i}),y_{i})$
under $RD(\mathcal{D}_{s})\leq\tau$ would incur a large generalization error
and cannot achieve fairness with respect to the true distribution
$\mathcal{Q}$.

The sample selection bias causes training data to be selected non-uniformly
from the population to be modeled. Generally there are four types of sample
selection bias, missing completely at random when $P(s=1|x,a,y)=p(s=1)$,
missing at random when $P(s=1|x,a,y)=p(s=1|x,a)$, missing at random-class when
$P(s=1|x,a,y)=p(s=1|y)$, and missing not at random when there is no
independence assumption between $x$, $a$, $y$ and $s$. In the following, we
focus on missing at random, which indicates biasedness of the selected sample
depends on the feature vector $x$ and protected attribute $a$.

The fairness constraint $RD(\hat{\mathcal{D}}_{s})\leq\tau$ can be derived as

$\leavevmode\resizebox{469.75499pt}{}{
$|\frac{\sum\mathbbm{1}_{(x_{i},a_{i})\in\mathcal{D}_{s}^{11}}\frac{P(s=1)}{P(s=1|x_{i},a_{i})}}{\sum\mathbbm{1}_{(x_{i},a_{i})\in\mathcal{D}_{s}^{\cdot
1}}\frac{P(s=1)}{P(s=1|x_{i},a_{i})}}-\frac{\sum\mathbbm{1}_{(x_{i},a_{i})\in\mathcal{D}_{s}^{10}}\frac{P(s=1)}{P(s=1|x_{i},a_{i})}}{\sum\mathbbm{1}_{(x_{i},a_{i})\in\mathcal{D}_{s}^{\cdot
0}}\frac{P(s=1)}{P(s=1|x_{i},a_{i})}}|\leq\tau$},$ (2)

where $\mathbbm{1}_{[.]}$ is an indicator function,
$\mathcal{D}_{s}^{ij}=\\{(x_{i},a_{i})|\hat{Y}=i,A=j\\}$ where
$i,j\in\\{0,1\\}$, $\mathcal{D}_{s}^{\cdot j}=\\{(x_{i},a_{i})|A=j\\}$ where
$j\in\\{0,1\\}$ and $\cdot$ represents $\\{0,1\\}$.

Then the minimization of the loss on $\hat{\mathcal{D}}_{s}$ subject to the
fairness constraint is as:

$\displaystyle\min_{\textbf{w}\in\mathcal{W}}L(\textbf{w})=\frac{1}{N_{\mathcal{D}_{s}}}\sum_{i=1}^{N_{\mathcal{D}_{s}}}\frac{P(s=1)}{P(s=1|x_{i},a_{i})}l(f(x_{i},a_{i}),y_{i})$
(3) $\displaystyle\text{subject to}\hskip 28.45274ptRD(\hat{D}_{s})\leq\tau$

where w are the parameters of the classifier $f$ and $R(\hat{D}_{s})$ is shown
in Eq. 2. We then have our following corollary.

######  Corollary 1.

With the assumption that selection variable $s$ and label $y$ are independent
given $x$, i.e., $P(s|x,a,y)=P(s|x,a))$, the classifier $f$ that minimizes
$\mathbb{E}_{(x,a,y,s)\in\hat{\mathcal{D}}_{s}}[l(f(x,a),y)]$ subject to
$RD(\hat{\mathcal{D}}_{s})\leq\tau$ is a fair classifier under sample
selection bias where $\hat{\mathcal{D}}_{s}$ is constructed by weighting each
sample $t\in\mathcal{D}_{s}$ with $\frac{P(s=1)}{P(s=1|x,a)}$.

##  4 Robust Fairness-aware Learning

To obtain the optimal solution of Eq. 3, we need to derive the sample
selection probability $P(s=1|t)$. However, it is rather challenging to get the
true $P(s=1|t)$ practically because the selection mechanism is usually
unknown. Instead, we need to estimate the sample selection probability and use
the estimated probability $\hat{P}(s=1|t)$ as the true $P(s=1|t)$.

To take the estimation error between $P(s=1|t)$ and $\hat{P}(s=1|t)$ into
consideration, we adopt the approach of minimax robust minimization Hu et al.
(2018); Liu and Ziebart (2014); Wen et al. (2014) which advocates for the
worst case of any unknown true sample selection probability. We make an
assumption here that the true $P(s=1|x,a)$ is with the $\epsilon$ range of the
estimated $\hat{P}(s=1|x,a)$. Therefore, any value of $P(s=1|x,a)$ in this
$\epsilon$ range represents the possible real unknown distribution
$\mathcal{Q}$. Following the standard robust optimization approaches and
taking the estimation error into consideration, we reformulate Eq. 3:

$\begin{aligned} &\min_{\textbf{w}\in\mathcal{W}}\hskip
2.84544pt\max_{{P}(s=1|x_{i},a_{i})}\hskip
2.84544ptL(\textbf{w},{P})=\frac{1}{N_{\mathcal{D}_{s}}}\sum_{i=1}^{N_{\mathcal{D}_{s}}}\frac{P(s=1)}{{P}(s=1|x_{i},a_{i})}l(f(x_{i},a_{i}),y_{i})\\\
&\text{subject to}\hskip
2.84544pt|P(s=1|x_{i},a_{i})-\hat{P}(s=1|x_{i},a_{i})|\leq\epsilon\\\ &\hskip
42.67912ptRD(\hat{D}_{s})\leq\tau\end{aligned}$

(4)

In fact, $P(s=1)$ is a constant and does not affect the problem formulation
and optimization. The robust minimax optimization can be treated as an
adversarial game by two players. One player selects ${P}(s=1|x_{i},a_{i})$
within the $\epsilon$ range of the estimated $\hat{P}(s=1|x_{i},a_{i})$ to
maximize the loss of the objective, which can be seen as the worst case of
$\mathcal{Q}$. As aforementioned, the selection probability is determined by
the corresponding distribution $\mathcal{Q}$ and then different selection
probability can represent different $\mathcal{Q}$. Another player minimizes
the worst case loss to find the optimal $\mathbf{w}$. There are two advantages
of robust minimax optimization of Eq. 4. First, it takes the worst case
induced by the estimation error into consideration, thus the obtained
classifier $f$ is robust to any possible $\mathcal{Q}$ within the error range
of the estimation. Second, during the minimax optimization, the fairness is
achieved under the worst case. Therefore, we can guarantee the fairness for
any possible $\mathcal{Q}$ within the error range of the estimation.

The computation of $RD(\hat{D}_{s})$ involves the indicator function, as shown
in Eq. 2, which makes it computationally intractable to reach the optimal
solution of Eq. 4. To address the intractable issue, we approximate the
fairness constraint using the boundary fairness Zafar et al. (2017) and it has
been proved that decision boundary fairness is a concept of risk difference.
We define $C(t,\textbf{w})$ be the covariance between the sensitive attribute
and the signed distance from the non-sensitive attribute vector to the
decision boundary. Then we can write the boundary fairness on
$\mathcal{D}_{s}$ as:

$C_{\mathcal{D}_{s}}(t,\textbf{w})=\frac{1}{N_{\mathcal{D}_{s}}}\sum_{i=1}^{N_{\mathcal{D}_{s}}}(a_{i}-\bar{a})d_{\textbf{w}(\textbf{x}_{i})},$
(5)

where $a_{i}$ is the sensitive attribute value of $t_{i}$,
$\bar{a}=\frac{1}{N_{\mathcal{D}_{s}}}{\sum_{i=1}^{N_{\mathcal{D}_{s}}}a_{i}}$
is the mean value of the sensitive attribute and
$d_{\textbf{w}(\textbf{x}_{i})}$ is the distance to the decision boundary of
the classifier $f$ and is formally defined as
$d_{\textbf{w}(\textbf{x}_{i})}=\textbf{w}^{T}\textbf{x}_{i}$. Similarly we
will have the boundary fairness on $\hat{\mathcal{D}}_{s}$ as:

$C_{\hat{\mathcal{D}}_{s}}(t,\textbf{w})=\frac{1}{N_{\mathcal{D}_{s}}}\sum_{i=1}^{N_{\mathcal{D}_{s}}}(a_{i}-\bar{a})\frac{P(s=1)}{{P}(s=1|x_{i},a_{i})}d_{\textbf{w}(\textbf{x}_{i})}.$
(6)

We enforce $C_{\hat{\mathcal{D}}_{s}}(t,\textbf{w})\leq\sigma,\sigma\in R^{+}$
to achieve the fair classification. With the boundary fairness, we can rewrite
the robust and fairness-aware loss function as:

$\begin{aligned} &\min_{\textbf{w}\in\mathcal{W}}\hskip
2.84544pt\max_{{P}(s=1|x_{i},a_{i})}\hskip
2.84544ptL(\textbf{w},{P})=\frac{1}{N_{\mathcal{D}_{s}}}\sum_{i=1}^{N_{\mathcal{D}_{s}}}\frac{P(s=1)}{{P}(s=1|x_{i},a_{i})}l(f(x_{i},a_{i}),y_{i})\\\
&\text{subject to}\hskip
2.84544pt|P(s=1|x_{i},a_{i})-\hat{P}(s=1|x_{i},a_{i})|\leq\epsilon\\\ &\hskip
42.67912pt|\frac{1}{N_{\mathcal{D}_{s}}}\sum_{i=1}^{N_{\mathcal{D}_{s}}}(a_{i}-\bar{a})\frac{P(s=1)}{{P}(s=1|x_{i},a_{i})}d_{\textbf{w}(\textbf{x}_{i})}|\leq\sigma\end{aligned}$

(7)

###  4.1 Solving Robust Fairness-aware Optimization

The optimization of Eq. 7 involves two sets of parameters w and
${P}(s=1|x_{i},a_{i})$. According to its minimax formulation, it is preferable
to obtain the optimal solution in an iterative manner by optimizing w and
${P}(s=1|x_{i},a_{i})$ alternatively. First, we fix ${P}(s=1|x_{1},a_{i})$ and
decompose the part of Eq. 7 only related to w as:

$\displaystyle\min_{\textbf{w}\in\mathcal{W}}\hskip
2.84544ptL(\textbf{w})=\frac{1}{N_{\mathcal{D}_{s}}}\sum_{i=1}^{N_{\mathcal{D}_{s}}}\frac{P(s=1)}{{P}(s=1|x_{i},a_{i})}l(f(x_{i},a_{i}),y_{i})$
(8) $\displaystyle\text{subject to}\hskip
14.22636pt|\frac{1}{N_{\mathcal{D}_{s}}}\sum_{i=1}^{N_{\mathcal{D}_{s}}}(a_{i}-\bar{a})\frac{P(s=1)}{{P}(s=1|x_{i},a_{i})}d_{\textbf{w}(\textbf{x}_{i})}|\leq\sigma$

It can be seen that with the fixed ${P}(s=1|x_{i},a_{i})$, the optimization of
w subjects to the linear fairness constraint. In fact, the optimal w is
determined by the choice of loss function $l$ and learning model $f$. For
example, if the chosen model is regression model, then the optimization of w
subject to the linear constraints belongs to the family of quadratic
programming, which can be solved efficiently by the available tools. Instead,
if we choose some complex models, e.g. deep learning models, then the loss
function parameterized by w is non-convex, and it is quite challenging to
apply the commonly used optimization techniques, e.g. gradient decent, to
optimize w with the existence of linear constraints. Therefore, for the
generalization purpose, we choose to transform the fairness constraint as a
penalty term and add to $L(\textbf{w})$, which can be expressed as:

$\displaystyle\min_{\textbf{w}\in\mathcal{W}}\hskip
2.84544ptL(\textbf{w})=\frac{1}{N_{\mathcal{D}_{s}}}\sum_{i=1}^{N_{\mathcal{D}_{s}}}\frac{P(s=1)}{{P}(s=1|x_{i},a_{i})}l(f(x_{i},a_{i}),y_{i})$
(9)
$\displaystyle+\beta(\frac{1}{N_{\mathcal{D}_{s}}}\sum_{i=1}^{N_{\mathcal{D}_{s}}}(a_{i}-\bar{a})\frac{P(s=1)}{{P}(s=1|x_{i},a_{i})}d_{\textbf{w}(\textbf{x}_{i})}-\sigma)^{2}$

where $\beta$ is a hyperparameter that controls the trade-off between the
utility and fairness. By the transformation, standard optimization techniques
such as stochastic gradient decent can be used to solve Eq. 9.

Second, we can fix w and decompose the part of Eq. 7 only related to
${P}(s=1|x_{i},a_{i})$ as the following:

$\begin{aligned} &\max_{{P}(s=1|x_{i},a_{i})}\hskip
2.84544ptL({P})=\frac{1}{N_{\mathcal{D}_{s}}}\sum_{i=1}^{N_{\mathcal{D}_{s}}}\frac{P(s=1)}{{P}(s=1|x_{i},a_{i})}l(f(x_{i},a_{i}),y_{i})\\\
&\text{subject to}\hskip
2.84544pt|P(s=1|x_{i},a_{i})-\hat{P}(s=1|x_{i},a_{i})|\leq\epsilon\\\ &\hskip
42.67912pt|\frac{1}{N_{\mathcal{D}_{s}}}\sum_{i=1}^{N_{\mathcal{D}_{s}}}(a_{i}-\bar{a})\frac{P(s=1)}{{P}(s=1|x_{i},a_{i})}d_{\textbf{w}(\textbf{x}_{i})}|\leq\sigma\end{aligned}$

(10)

The objective of Eq. 10 is a linear combination of
$\frac{P(s=1)}{{P}(s=1|x_{i},a_{i})}$ as we can treat
$\frac{P(s=1)}{{P}(s=1|x_{i},a_{i})}$ to be one variable. $P(s=1)$ is a
constant and does not affect the optimization. For the first constraint,
$|P(s=1|x_{i},a_{i})-\hat{P}(s=1|x_{i},a_{i})|\leq\epsilon$, we can obtain the
range of $\frac{P(s=1)}{{P}(s=1|x_{i},a_{i})}$ after we estimate the range of
each $P(s=1|x_{i},a_{i})$. The second constraint
$|\frac{1}{N_{\mathcal{D}_{s}}}\sum_{i=1}^{N_{\mathcal{D}_{s}}}(a_{i}-\bar{a})\frac{P(s=1)}{{P}(s=1|x_{i},a_{i})}d_{\textbf{w}(\textbf{x}_{i})}|\leq\sigma$
in Eq. 10 is linear with respect to $\frac{P(s=1)}{{P}(s=1|x_{i},a_{i})}$ when
w is fixed. Therefore, the optimization of
$\frac{P(s=1)}{{P}(s=1|x_{i},a_{i})}$ is a standard linear programming and we
can directly apply linear programming tool to get the optimal solution of
$\frac{P(s=1)}{{P}(s=1|x_{i},a_{i})}$ without any additional relaxation.

###  4.2 RFLearn1: Sample Bias Correction with $\mathcal{D}$

In the previous section we formulate the robust and fairness-aware problem
under the assumption that the estimated $\hat{P}(s=1|x,a)$ is obtained, but
without addressing how to estimate it. In this section, we assume the
unlabeled $\mathcal{D}$ is available and estimate the true $P(s=1|x,a)$ by
using $\mathcal{D}$ and $\mathcal{D}_{s}$ together. For a particular data
record $t$, the ratio between the number of times $t$ in $\mathcal{D}$ and the
number of times $t$ in $\mathcal{D}_{s}$ in terms of $(a,x)$ is an estimation
value for $P(s=1|x,a)$. Formally, for $t\in\mathcal{D}$, let $\mathcal{D}^{t}$
denote the subset of $\mathcal{D}$ containing exactly all the instances of $t$
and $n_{t}=|\mathcal{D}^{t}|$. Similarly, let $\mathcal{D}_{s}^{t}$ denote the
subset of $\mathcal{D}_{s}$ containing exactly all the instances of $t$ and
$m_{t}=|\mathcal{D}_{s}^{t}|$. We then have
$\hat{P}(s=1|x,a)=\frac{m_{t}}{n_{t}}$.

Lemma 1 Cortes et al. (2008) Let $\delta>0$, then, with probability at least
$1-\delta$, the following inequality holds for all $t\in\mathcal{D}_{s}$:

$|P(s=1|x,a)-\frac{m_{t}}{n_{t}}|\leq\sqrt{\frac{\text{ln}2m^{\prime}+\text{ln}\frac{1}{\delta}}{p_{0}N_{\mathcal{D}}}}$
(11)

where $m^{\prime}$ denotes the number of unique points in $\mathcal{D}_{s}$
and $p_{0}=\text{min}_{t\in\mathcal{D}}P(t)\neq 0$.

For notation convenience, we define $\theta(x_{i},a_{i})=P(s=1|x_{i},a_{i})$
and $\hat{\theta}(x_{i},a_{i})=\hat{P}(s=1|x_{i},a_{i})$, where
$\hat{\theta}(x_{i},a_{i})$ is the empirical value based on the frequency
estimation. Lemma 1 states that
$|\theta(x_{i},a_{i})-\hat{\theta}(x_{i},a_{i})|$ is upper bounded by the
right term in Eq. 11. Then we can apply the robust fairness aware framework by
setting $\theta(x_{i},a_{i})$ within $\epsilon$ range of estimated
$\hat{\theta}(x_{i},a_{i})$, where $\epsilon$ is the right term of Eq. 11.
According to the Theorem 2 in Cortes et al. (2008), the generalization error
between the true distribution $\theta(x_{i},a_{i})$ and distribution using the
estimated $\hat{\theta}(x_{i},a_{i})$ is expressed as:

$|L_{\theta}(\textbf{w})-L_{\hat{\theta}}(\textbf{w})|<\mu\sqrt{\frac{\text{ln}2m^{\prime}+\text{ln}\frac{1}{\delta}}{p_{0}N_{\mathcal{D}}}}$
(12)

where $\mu$ is a constant determined by $\sigma$ (Lemma 1) and hyperparameter
$\beta$ (Eq. 9). Suppose the maximum value of $L_{\hat{\theta}}(\textbf{w})$
is defined as $L_{\hat{\theta}}(\textbf{w})_{max}$ and our robust fairness-
aware optimization is to minimize $L_{\hat{\theta}}(\textbf{w})_{max}$ per
iteration. The loss $L_{\hat{\theta}}(\textbf{w})_{max}$ consists of both the
prediction loss and fairness loss. Therefore, the minimization of upper bound
of the generalization error of true distribution can provide robustness in
terms of both prediction and fairness.

Table 1: Model performance under data distribution shift (Adult and Dutch)
Acc: accuracy

Methods | Adult Dataset | Dutch Dataset
---|---|---
Training Acc | Test Acc | Training $RD$ | Test $RD$ | Training Acc | Test Acc | Training $RD$ | Test $RD$
LR (unbiased) | 0.8124 | 0.8126 | 0.1562 | 0.1373 | 0.7493 | 0.7669 | 0.1498 | 0.1395
LR | 0.8041 | 0.7882 | 0.1344 | 0.1228 | 0.7018 | 0.6821 | 0.0378 | 0.1012
FairLR | 0.7823 | 0.7622 | 0.0231 | 0.0956 | 0.7006 | 0.6624 | 0.0289 | 0.0991
Wen et al. (2014) | 0.7883 | 0.8048 | 0.1348 | 0.1333 | 0.6812 | 0.7044 | 0.1421 | 0.1394
$\textit{RFLearn}^{1-}$ | 0.7412 | 0.7875 | 0.0351 | 0.1048 | 0.6501 | 0.6879 | 0.0315 | 0.0809
$\textit{RFLearn}^{1}$ | 0.7484 | 0.7816 | 0.0281 | 0.0416 | 0.6673 | 0.6910 | 0.0317 | 0.0405
$\textit{RFLearn}^{2-}$ | 0.7473 | 0.7771 | 0.0321 | 0.0963 | 0.6457 | 0.6809 | 0.0411 | 0.0973
$\textit{RFLearn}^{2}$ | 0.7336 | 0.7678 | 0.0197 | 0.0238 | 0.6479 | 0.6755 | 0.0321 | 0.0373

Table 2: Model performance of $\textit{RFLearn}^{1}$ under sample selection
bias with different $\delta$ (Adult and Dutch). Acc: accuracy

$\delta$ | Adult Dataset | Dutch Dataset
---|---|---
Training Acc | Test Acc | Training $RD$ | Test $RD$ | Training Acc | Test Acc | Training $RD$ | Test $RD$
0.025 | 0.7181 | 0.7601 | 0.0189 | 0.0219 | 0.6521 | 0.6812 | 0.0291 | 0.0326
0.05 | 0.7217 | 0.7673 | 0.0239 | 0.0398 | 0.6521 | 0.6812 | 0.0321 | 0.0326
0.1 | 0.7484 | 0.7816 | 0.0307 | 0.0416 | 0.6673 | 0.6910 | 0.0378 | 0.0405
0.15 | 0.7239 | 0.7768 | 0.0277 | 0.0333 | 0.6701 | 0.6994 | 0.0275 | 0.0379

Table 3: Model performance of $\textit{RFLearn}^{2}$ under sample selection
bias with different $\rho$ (Adult and Dutch). Acc: accuracy

$\rho$ | Adult Dataset | Dutch Dataset
---|---|---
Training Acc | Test Acc | Training $RD$ | Test $RD$ | Training Acc | Test Acc | Training $RD$ | Test $RD$
0.2 | 0.7229 | 0.7558 | 0.0178 | 0.0114 | 0.6401 | 0.6543 | 0.0175 | 0.0214
0.4 | 0.7336 | 0.7628 | 0.0197 | 0.0238 | 0.6479 | 0.6755 | 0.0321 | 0.0373
0.6 | 0.7428 | 0.7724 | 0.0269 | 0.0361 | 0.6544 | 0.6792 | 0.0301 | 0.0314

###  4.3 RFLearn2: Sample Bias Correction without $\mathcal{D}$

In this section, we focus on the scenario without unlabeled test data
$\mathcal{D}$. The challenge is how to use $\mathcal{D}_{s}$ alone to estimate
the true sample selection probability so that we can construct
$\hat{\mathcal{D}}_{s}$ to resemble $\mathcal{D}$. In general, the exact
relationship between $\mathcal{D}_{s}$ and $\mathcal{D}$ is unknown. Without
additional assumptions, it is impossible to build a model to resemble the true
$\mathcal{D}$ with only access to the observed $\mathcal{D}_{s}$.

We assume that 1) there exist $K$ clusters in $\mathcal{D}_{s}$; 2) the
samples in the same cluster have the same selection probability; and 3) the
selection probability of each sample is within a range of the uniform
selection probability. Under these assumptions, the ratio
$\frac{P(s=1)}{{P}(s=1|x_{i},a_{i})}$ for each sample from the same cluster is
the same. The ratio vector for $K$ clusters is defined as
$\mathbf{r}=(r_{1},r_{2},\cdots,r_{K})$. We robustify the estimation by
approximating $r$ within a Wasserstein ball $B_{\rho}$ Blanchet et al. (2019)
around the uniform ratio $\mathbf{r}_{u}$, where all of the values in
$\mathbf{r}_{u}$ is 1. Formally we have $|\mathbf{r}-\mathbf{r}_{u}|\leq\rho$,
where $\rho$ is the radius of the Wasserstein ball. Suppose $x_{i}$ belongs to
the $k$-th cluster where $k\in[K]$, we can write the robust loss of $f$ with
only $\mathcal{D}_{s}$ available as the following:

$\displaystyle\min_{\textbf{w}\in\mathcal{W}}\hskip
2.84544pt\max_{\mathbf{r}\in B_{\rho}}\hskip
2.84544ptL(\textbf{w},\mathbf{r})=\frac{1}{N_{\mathcal{D}_{s}}}\sum_{i=1}^{N_{\mathcal{D}_{s}}}r_{k}l(f(x_{i},a_{i}),y_{i})$
(13) $\displaystyle\text{subject to}\hskip
28.45274pt|\mathbf{r}-\mathbf{r}_{u}|\leq\rho$ $\displaystyle\hskip
65.44142pt|\frac{1}{N_{\mathcal{D}_{s}}}\sum_{i=1}^{N_{\mathcal{D}_{s}}}(a_{i}-\bar{a})r_{k}d_{\textbf{w}(\textbf{x}_{i})}|\leq\sigma$

The above formulation also saves huge amounts of computational cost as it
reduces the number of constraints from the training data size level to the
cluster size level.

##  5 Experiment

Dataset. We use two benchmark datasets, Adult Dua and Graff (2017) and Dutch
Žliobaite et al. (2011), to evaluate our proposed algorithms, RFLearn1 and
RFLearn2. Adult dataset consists of individual’s information such as age,
education level, gender, occupation, race, and so forth. The task for Adult
dataset is to predict whether an individual’s income is over 50k or not based
on the collected personal information. Dutch dataset is a collection of Census
data in Netherlands, where each data sample has different attributes like
Adult datasets, including education level, gender, race, marital status and so
forth. The task is also a binary classification to predict whether an
individual belongs to low income or high income group. In both datasets, we
set gender as the protected attribute and use one-hot encoding to convert the
categorical attributes to vectors and apply normalization to covert numerical
attributes into the range $[0,1]$. After preprocessing, Adult has 45222
records and 40 features, while Dutch has 60420 records and 35 features.

Experimental Setting. We follow the biased data generation approach in
Laforgue and Clémençon (2019) and choose to select the data based on the
education level (married status) for Adult (Dutch). For Adult, we create the
biased training data by selecting 18157 records from the first 35000 records
(randomly choosing 70% of people with 10+ years education and 30% of people
with less than 10 years education) and use the rest 10222 records as the test
data. Similarly for Dutch, we create the biased training data by selecting
20928 examples from the first 45000 records (randomly choosing 75% of married
people and 25% of unmarried people) and use the rest 15420 records as the test
data.

Baselines. We choose the logistic regression (LR) to implement and evaluate
different algorithms. We consider the following baselines to compare with our
proposed RFLearn1 and RFLearn2: (a) LR without fairness constraint (LR); (b)
LR with fairness constraint (FairLR); (c) robust LR in Wen et al. (2014) that
uses kernel functions to reweigh samples under covariate shift but ignores the
fairness constraint. For RFLearn1 (RFLearn2), we also consider its variation
RFLearn1- (RFLearn2-) that optimizes the robust loss with unweighted fairness
constraint.

Metrics and Hyperparameters. We evaluate the performance in terms of
prediction accuracy and risk difference ($RD$). We consider a classifier is
fair if $RD(f)\leq 0.05$. The hyperparameter $\beta$ that controls the
accuracy-fairness trade-off is set as 1. The radius $\rho$ is set as 0.4,
$\delta$ is set as 0.1, and $\sigma$ is set as 0.2. For
$\textit{RFLearn}^{2-}$ and $\textit{RFLearn}^{2}$, we apply K-means Hartigan
and Wong (1979) to cluster the training data and set the number of clusters
300. We run all experiments 20 times and report the average results.

###  5.1 Accuracy vs. Fairness

We report the accuracy and fairness on both training data and test data for
each method in our results. Note that our goal is to achieve robust accuracy
and fairness on test data under the sample selection bias. Results on both
training and test data help explain why baselines focusing on the training
data itself fail achieving good accuracy and fairness on test data.

Accuracy. Table 1 shows prediction accuracy and risk difference of each model
on training and test data for both Adult and Dutch. The first row LR
(unbiased) indicates the performance of the logistic regression model with
unbiased training data. We randomly select the same number of data from the
original datasets. The results of the rest rows are all conducted on our
biased training data discussed in the experimental setting section. We can
draw several important points from the experimental results.

First, the testing accuracy of LR is 0.7882 for Adult and 0.6821 for Dutch,
whereas the accuracy of LR (unbiased) is 0.8126 for Adult and 0.7669 for
Dutch. It demonstrates that the model prediction performance degrades under
sample selection bias. Second, with the use of robust learning, the prediction
accuracy for Adult (Ducth) of RFLearn1- is 0.7875 (0.6879) and RFLearn2- is
0.7771 (0.6809), which outperforms FairLR by 0.0253 (0.0255) and 0.0109
(0.0185). As the fairness constraints of RFLearn1- and RFLearn2- are the same
with FairLR, it demonstrates that the robust learning in Eq. 7 can provide
robust prediction under the sample selection bias. Third, the testing accuracy
from robust learning methods is higher than the training accuracy, which
further demonstrates the advantage of robust learning under the sample
selection bias. Fourth, the accuracy of RFLearn1 is higher than that of
RFLearn2. It is reasonable as we leverage the unlabeled test data in our
RFLearn1.

Fairness. In this section, we focus on fairness comparison. We summarize below
the interesting observations from Table 1. First, all of FairLR, RFLearn1- and
RFLearn2- can only achieve fairness on the training data with $RD\leq 0.05$,
but none of these approaches can guarantee the fairness on the test data.
RFLearn1- and RFLearn2-, which uses the unweighted fairness constraint, also
fail to achieve fairness on the test data. The method proposed by Wen et al.
(2014) only considers the robustness of prediction error but ignores the
fairness, so that it cannot achieve the fairness on the test data as well.
Second, RFLearn1 and RFLearn2 can achieve fairness on both training and test
data. The result is consistent with our algorithms as we enforce the fairness
under any possible adverse distribution. Therefore, the classifier is also
robust in terms of meeting fairness requirement on the test data even it is
trained on the biased sampling data.

###  5.2 Effects of the Hyperparameters

In this section, we conduct sensitivity analysis of our RFLearn1 and RFLearn2
with different hyperparameters. Due to space limits, we focus on two key
parameters $\delta$ and $\rho$. Table 2 shows the performance of RFLearn1 with
different $\delta$ values. Note that in Lemma 1 the estimation error of the
sample selection probability is upper bounded with the probability greater
than $1-\delta$. The upper bound (the right side in Eq. 11) increases with the
decreasing $\delta$. A larger upper bound indicates that the adversary can
generate more possible distributions during the robust optimization, hence
helping achieve better prediction accuracy on test data. However, when the
upper bound is too large, excessive possible distributions may reduce the
prediction accuracy on test data. The experimental results in Table 2 match
our above analysis.

Table 3 shows the result of RFLearn2 under different radius $\rho$. We can see
that the testing accuracy increases with the increasing $\rho$. Larger $\rho$
indicates more possible generated distributions which are more likely to cover
the test distribution and improve the model performance. Moreover, the
proposed RFLearn1 and RFLearn2 can achieve both fairness on the training and
test data with different $\delta$ and $\rho$. It is beneficial as the fairness
performance of our proposed algorithms are not sensitive to the varying
$\delta$ and $\rho$.

##  6 Related Work

There has been extensive research on classification under sample selection
bias, e.g., Zadrozny (2004); Huang et al. (2006). Laforgue and Clémençon
(2019) provide a methodology to address selection bias issues in statistical
learning and extend the paradigmatic ERM approach to the sample selection bias
setting. Robust classification under sample selection bias has also been
studied recently in Hu et al. (2018); Liu and Ziebart (2014); Wen et al.
(2014). Wen et al. (2014) consider covariate shift between the training and
test data and apply Gaussian kernel functions to reweigh the training examples
and correct the shift. They propose a minimax robust framework to minimize the
most adverse distribution and provide robust classification when it comes to
the test data. Similarly, Liu and Ziebart (2014) propose a robust bias-aware
probabilistic classifier that can deal with different test data distributions
using a minimax estimation formulation.

Most recently, there have been a few papers that study fairness from the
distributionally robust perspective. Taskesen et al. (2020) propose a
distributionally robust logistic regression model with an unfairness penalty.
They assume the unknown true test distribution is contained in a Wasserstein
ball centered at the empirical distribution on the observed training data. The
proposed model, which robustifies the fair logistic regression against all
distributions in the ball, is equivalent to a tractable convex problem when
unfairness is quantified under the log-probabilistic equalized opportunities
criterion. However the approach robustifies the distribution at the individual
data level and overlooks the overall distribution. Rezaei et al. (2020)
propose the use of ambiguity set to derive the fair classifier based on the
principles of distributional robustness. The proposed approach incorporates
fairness criteria into a worst case logarithmic loss minimization but ignores
the distribution shift. Yurochkin et al. (2020) develop an individual fair
distributionally robust classifier with a Wasserstein ambiguity set. However,
the approach does not admit a tractable convex reformulation.

There has been research on fairness aware domain adaptation and transfer
learning of fairness metrics, e.g., Schumann et al. (2019); Coston et al.
(2019); Lahoti et al. (2020); Kallus and Zhou (2018). In particular, Schumann
et al. (2019) study the fair transfer learning from source domain to target
domain and provide a fairness bound on the target domain of the predictor
trained on the source domain data. Coston et al. (2019) study the fair
transfer learning with missing protected attributes under the covariate shift.
Similarly, Kallus and Zhou (2018) use covariate shift correction when
computing fairness metrics to address bias in label collection. The above
works also adopt the idea of reweighing the source examples to resemble the
target domain examples during the training. However, they do not address the
robustness in learning under sample selection bias.

Related but different from our work is fair learning from label biased
training data Blum and Stangl (2020); Jiang and Nachum (2020) and fair
representation learning Madras et al. (2018); Song et al. (2019); Zhao and
Gordon (2019). For example, Jiang and Nachum (2020) develop a framework to
model how label bias can arise in a dataset, assuming that there exists an
unbiased ground truth, and develop a bias correcting approach based on re-
weighting the training examples. Du et al. (2021) propose a robust framework
to achieve fairness in federated learning under covariate shift between the
training data and test data. It applies Gaussian kernel to generate the
adversary distribution and minimizes the worst adversary distribution loss
during training. Zhao and Gordon (2019) investigate the relationship between
fair representation and decision making with theoretical guarantee. They
employ the adversarial technique to extract the fair representation. Singh et
al. (2021) study the covariate shift in the domain adaption and assumes a
known causal graph of the data generating process and a context variable
causing the shift. Rezaei et al. (2021) apply minimax optimization to achieve
robustness fairness under covariate shift, where the fairness violation
penalty term between the target input distribution and adversary’s conditional
label distribution is incorporated during the optimization. Our work is
different from above works as we go from the sample selection probability
estimation and minimize the worst case loss while enforcing fairness on re-
weighted samples.

##  7 Conclusion

We have proposed a robust and fair learning framework to deal with the sample
selection bias. Our framework adopts the reweighing estimation approach for
bias correction and the minimax robust estimation for achieving robustness on
prediction accuracy and fairness on test data. We further developed two
algorithms to handle sample selection bias when test data is both available
and unavailable.

Experimental results showed our algorithms can achieve both good prediction
accuracy and fairness on test data. In our future work, we will investigate
the sample selection bias under missing not at random, i.e., the sample
selection probability also depends on the label. We will also study how to
enforce other fairness notions such as equal opportunityHardt et al. (2016) .

##  8 Acknowledgement

This work was supported in part by NSF 1920920, 1940093, and 1946391,

## References

  * Blanchet et al. (2019) Jose Blanchet, Yang Kang, and Karthyek Murthy.  Robust wasserstein profile inference and applications to machine learning.  Journal of Applied Probability, 2019.
  * Blum and Stangl (2020) Avrim Blum and Kevin Stangl.  Recovering from biased data: Can fairness constraints improve accuracy?  In FORC, 2020.
  * Cortes et al. (2008) Corinna Cortes, Mehryar Mohri, Michael Riley, and Afshin Rostamizadeh.  Sample selection bias correction theory.  In ALT, 2008.
  * Coston et al. (2019) Amanda Coston, Karthikeyan Natesan Ramamurthy, Dennis Wei, Kush R Varshney, Skyler Speakman, Zairah Mustahsan, and Supriyo Chakraborty.  Fair transfer learning with missing protected attributes.  In AIES, 2019.
  * Du et al. (2021) Wei Du, Depeng Xu, Xintao Wu, and Hanghang Tong.  Fairness-aware agnostic federated learning.  In Proceedings of the 2021 SIAM International Conference on Data Mining (SDM), pages 181–189. SIAM, 2021.
  * Dua and Graff (2017) Dheeru Dua and Casey Graff.  UCI machine learning repository, 2017.
  * Hardt et al. (2016) Moritz Hardt, Eric Price, and Nati Srebro.  Equality of opportunity in supervised learning.  In Advances in neural information processing systems, pages 3315–3323, 2016.
  * Hartigan and Wong (1979) John A Hartigan and Manchek A Wong.  Algorithm as 136: A k-means clustering algorithm.  Journal of the royal statistical society. series c (applied statistics), 1979.
  * Hu et al. (2018) Weihua Hu, Gang Niu, Issei Sato, and Masashi Sugiyama.  Does distributionally robust supervised learning give robust classifiers?  In ICML, 2018.
  * Huang et al. (2006) Jiayuan Huang, Alexander J. Smola, Arthur Gretton, Karsten M. Borgwardt, and Bernhard Schölkopf.  Correcting sample selection bias by unlabeled data.  In NeurIPS, 2006.
  * Jiang and Nachum (2020) Heinrich Jiang and Ofir Nachum.  Identifying and correcting label bias in machine learning.  In AISTATS, 2020.
  * Kallus and Zhou (2018) Nathan Kallus and Angela Zhou.  Residual unfairness in fair machine learning from prejudiced data.  In ICML, 2018.
  * Laforgue and Clémençon (2019) Pierre Laforgue and Stephan Clémençon.  Statistical learning from biased training samples.  arXiv preprint arXiv:1906.12304, 2019.
  * Lahoti et al. (2020) Preethi Lahoti, Alex Beutel, Jilin Chen, Kang Lee, Flavien Prost, Nithum Thain, Xuezhi Wang, and Ed H Chi.  Fairness without demographics through adversarially reweighted learning.  In NeurIPS, 2020.
  * Liu and Ziebart (2014) Anqi Liu and Brian Ziebart.  Robust classification under sample selection bias.  In NeurIPS, 2014.
  * Madras et al. (2018) David Madras, Elliot Creager, Toniann Pitassi, and Richard Zemel.  Learning adversarially fair and transferable representations.  In ICML, 2018.
  * Rezaei et al. (2020) Ashkan Rezaei, Rizal Fathony, Omid Memarrast, and Brian D. Ziebart.  Fairness for robust log loss classification.  In AAAI, 2020.
  * Rezaei et al. (2021) Ashkan Rezaei, Anqi Liu, Omid Memarrast, and Brian Ziebart.  Robust fairness under covariate shift.  In AAAI, 2021.
  * Schumann et al. (2019) Candice Schumann, Xuezhi Wang, Alex Beutel, Jilin Chen, Hai Qian, and Ed H Chi.  Transfer of machine learning fairness across domains.  arXiv preprint arXiv:1906.09688, 2019.
  * Singh et al. (2021) Harvineet Singh, Rina Singh, Vishwali Mhasawade, and Rumi Chunara.  Fairness violations and mitigation under covariate shift.  In Proceedings of the 2021 ACM Conference on Fairness, Accountability, and Transparency, pages 3–13, 2021.
  * Song et al. (2019) Jiaming Song, Pratyusha Kalluri, Aditya Grover, Shengjia Zhao, and Stefano Ermon.  Learning controllable fair representations.  In AISTATS, 2019.
  * Taskesen et al. (2020) Bahar Taskesen, Viet Anh Nguyen, Daniel Kuhn, and Jose Blanchet.  A distributionally robust approach to fair classification.  arXiv preprint arXiv:2007.09530, 2020.
  * Wen et al. (2014) Junfeng Wen, Chun-Nam Yu, and Russell Greiner.  Robust learning under uncertain test distributions: Relating covariate shift to model misspecification.  In ICML, 2014.
  * Yurochkin et al. (2020) Mikhail Yurochkin, Amanda Bower, and Yuekai Sun.  Training individually fair ML models with sensitive subspace robustness.  In ICLR, 2020.
  * Zadrozny (2004) Bianca Zadrozny.  Learning and evaluating classifiers under sample selection bias.  In ICML, 2004.
  * Zafar et al. (2017) Muhammad Bilal Zafar, Isabel Valera, Manuel Gomez Rogriguez, and Krishna P Gummadi.  Fairness constraints: Mechanisms for fair classification.  In AISTATS, 2017.
  * Zhao and Gordon (2019) Han Zhao and Geoff Gordon.  Inherent tradeoffs in learning fair representations.  In NeurIPS, 2019.
  * Žliobaite et al. (2011) Indre Žliobaite, Faisal Kamiran, and Toon Calders.  Handling conditional discrimination.  In IEEE ICDM, 2011.