# Electronic and magnetic properties of silicene monolayer under bi-axial
mechanical strain: first principles study

M.A.Jafari Department of Mesoscopic Physics, ISQI, Faculty of Physics, Adam
Mickiewicz University, ul. Uniwersytetu Poznańskiego 2, 61-614 Poznań, Poland
A.A. Kordbacheh Department of Physics, Iran University of Science and
Technology, 16846-13114 Tehran, Iran  A. Dyrdal Department of Mesoscopic
Physics, ISQI, Faculty of Physics, Adam Mickiewicz University, ul.
Uniwersytetu Poznańskiego 2, 61-614 Poznań, Poland

###### Abstract

Mechanical control of electronic and magnetic properties of 2D Van-der-Waals
heterostructures gives new possibilities for further development of
spintronics and information-related technologies. Using the density functional
theory, we investigate the structural, electronic and magnetic properties of
silicene monolayer with substituted Chromium atoms and under a small biaxial
strain ($-6\%<\epsilon<8\%$). Our results indicate that the Cr-doped silicene
nanosheets without strain have magnetic metallic, half-metallic or
semiconducting properties depending on the type of substitution. We also show
that the magnetic moments associated with the monomer and vertical dimer
substitutions change very weakly with strain. However, the magnetic moment
associated with the horizontal dimer substitution decreases when either
compressive or tensile strain is applied to the system. Additionally, we show
that the largest semiconductor band-gap is approximately 0.13 eV under zero
strain for the vertical Cr-doped silicene. Finally, biaxial compressive strain
leads to irregular changes in the magnetic moment for Cr vertical dimer
substitution.

DFT — Silicene — Strain — Electronic properties — Magnetic Properties

##  1 Introduction

Two-dimensional (2D) crystals are currently of great interest for both applied
and fundamental research. The most prominent example of this kind materials is
graphene. However, the class of 2D materials is very large and is continuously
growing. It contains single-layered materials such as those belonging to the
group IV of the periodic table (silicene, germanene and stanene) and to the
group V (arsenene, antimonene, bismuthene), as well as layered 2D materials
like Transition Metal Dichalcogenides and MXenes. Among them silicene seems to
be a promising material due to its compatibility with existing silicon-based
electronic devices. Because of sp3 hybridization, silicene is the 2D buckled
hexagonal lattice of silicon atoms, and is considered as a one of alternative
materials to graphene Greve et al. (2010); Enriquez et al. (2012); Feng et al.
(2012); Lalmi et al. (2010); Tao et al. (2015). Research activities on
silicene significantly increased after its successful synthesis under UHV
conditions on several substrates, like for instance on Ag(111), ZrB2(0001),
MoS2(0001), and Ir(111) Kumar & Ahluwalia (2013, 2012c, 2012a, 2012b); Mao et
al. (2018). Silicene displays several interesting characteristics, which have
been revealed by recent experimental and theoretical investigations. These
include for instance: (i) a remarkable spin–orbit coupling parameter, that
leads to the energy gap of 1.5 meV Chen (2016); Chowdhury & Jana (2016); Ding
et al. (2017) at the Dirac point, which is much larger than that in graphene
(24 $\mu$eV) Enriquez et al. (2012); Drummond et al. (2012); Ezawa (2012);
(ii) electrically tunable bandgap; (iii) the phase transition from a spin Hall
topological insulator to a band insulator Dyrdał & Barnaś (2012); (iv) the
strain-induced tunable bandgap Gmitra et al. (2009); Guo et al. (2015, 2013),
and (v) promising electric and thermoelectric characteristics Wierzbicki et
al. (2015a, b); Weymann et al. (2017); Krompiewski (2021); Krompiewski &
Cuniberti (2017); Ghasemi et al. (2019); Jafari et al. (2020). The energy gap
in silicene makes it promising for applications, however the realization of
stabile monolayer of silicene is still problematic Low et al. (2011). For
example, to achieve high on-to-off current ratios and a perfect switching
capability, the material exploited for Field-Effect-Transistors (FETs) is
usually required to have a fairly large bandgap Mao et al. (2018),
significantly larger than that mentioned above for silicene.

Electronic and magnetic properties of silicene can be tuned by impurities,
i.e., magnetic atoms built into the monolayer structure. Importantly, recent
achievements of nanotechnology enable precise arrangements of impurities
(including regular lattices), so by doping one can modify electronic
properties in a controllable way. The single-side adsorption of alkali metal
atoms on silicene has been reported to give rise to a bandgap of approximately
0.5 eV Zhang et al. (2013); Le Lay (2015), that is much larger than that due
to intrinsic spin-orbit interaction. If both sides of the single-layer
silicene have been saturated with hydrogen atoms, the formed composite system
has been shown to be a kind of a nonmagnetic semiconductor Cao et al. (2015).
Based on the previous studies, the ferromagnetic characteristics of silicene
can be attributed to the single-side hydrogenation Zhang et al. (2012).

Figure 1: (a) Hexagonal crystal structure of silicene, with $d$ being the atom
spacing and $\Delta$ denoting the buckling parameter. The unit cell is
indicated by he red dashed lines, and the basis consists of two Si atoms,
labelled with A and B. (b-d) Types of impurity substitutions: monomer (b),
horizontal dimer (c), and vertical dimmer (d).

Strain engineering enables external control of electronic characteristics of
semiconductor heterostructures and nanomaterials. This technique is widely
applied to nano-electro-mechanical and nano-opto-mechanical systems, as well
as to MOSFETs Chu et al. (2009); Lee et al. (2005). In turn, the in-plane
strain in silicene leads to a modification of the electronic structure and its
transport characteristics Qin et al. (2012); Farokhnezhad et al. (2017).

An important question is whether a significant gap in the spectrum of silicene
can be induced by the strain, especially at low values of strain. If this is
the case, tunable silicene devices would be of great practical importance.
According to earlier density functional theory (DFT) calculations, a gap in
the spectrum of silicene can be open under arbitrary uniaxial strain. Its
magnitude varies non-monotonically with the strain. These findings were
supported by other ab-initio (without empirical parameters and from first
principles) calculation Mohan et al. (2014). For similar strain magnitudes,
however, the two gap calculations were not in agreement. Recently, Pereira et
al. Pereira et al. (2009) have questioned the accuracy of those conclusions by
applying the Tight Binding (TB) approach. According to their findings, a
spectral gap can only be achieved for +20% uniaxial deformations. Furthermore,
this effect highly depends on the deformation route regarding the rudimentary
lattice. The aforementioned findings are in accordance with the studies of
Hasegawa et al. Hasegawa et al. (2006), indicating that there is a robust
gapless Dirac spectrum with regard to arbitrary and not extremely large
changes in the nearest neighbour hopping parameters. Moreover, employing the
TB model, Wunsch et al. Wunsch et al. (2008) found that the semi-metallic
phase appears for hopping parameter expansion.

The results of ab-initio calculations Farjam & Rafii-Tabar (2009) are in
agreement with the gapless situation presented in Faccio et al. (2009). The
inconsistency between various ab-initio calculations ware partially related to
the fact that due to strain the Dirac points shift from the high symmetry
points of the Brillouin zone. This resulted in arriving at the wrong
conclusion that a bandgap is achievable for any strain. In addition, Faccio et
al. also performed DFT research and calculated the impact of $\approx 2\%$
strain in silicene nanoribbon [43]. Keeping in mind that mechanical strain can
substantially change the physical properties of silicene, we have performed
detailed ab-initio calculations of the magnetic and electronic characteristics
of silicene with inserted dimers and monomers of Cr atoms. Then, we have
analysed the influence of a biaxial strain for up to $\approx\pm 8\%$
deformations. We have shown that magnetic and electronic characteristics of
silicene with substitutional impurity atoms can be easily controlled by
various kinds of strains (i.e., substrate-induced strains or external
mechanical forces). It is worth to note, that the gap due to strain and doping
is significantly larger than the gap induced by the intrinsic spin-orbit
interaction in undoped and unstrained silicene.

##  2 Methodology and strain-dependent structural properties

To study electronic and magnetic properties of silicene under mechanical
strain we have used the DFT method Grimme et al. (2010) within the Perdew-
Burke-Ernzerhof (PBE) generalized gradient approximation (GGA) Grimme (2006)
form of the exchange-correlation functional, as implemented in the QuantumATK
code package ver. S-2021.06 Stradi et al. (2016); Smidstrup et al. (2020). The
PseudoDojo collection of optimized norm-conserving Vanderbilt (ONCV)
pseudopotentials and ultra-basis set have been used for the optimization of
structures and for further calculations van Setten et al. (2018). For the
Brillouin zone integration we have taken 8$\times$8$\times$1 Monkhorst-Pack
k-points in self-consisting calculation (SCF), and the mesh cut-off of energy
has been set to 450 Ry. Structures ware relaxed until the forces on each atom
were less than 0.05 eV/$\AA$ and relative convergence for the Self-Consistent
Field (SCF) energy is reached until $10^{-5}$ eV/$\AA$. We have used 15 $\AA$
vacuum region to prevent interaction of two neighboring layers along the
$c$-axis Ernzerhof & Scuseria (1999); Yin & Cohen (1982); Burke (2012).

The atomic structure of silicene should be characterized before modelling the
electronic structure L. C. (2015). The first structure optimization process
for a single layer of Si was reported by Takeda and Shiraishi Takeda &
Shiraishi (1994). By analogy to graphite, they defined a hexagonal lattice for
Si atoms (with a periodicity perpendicular to the plane with a large vacuum
layer of minimally 10 $\AA$), and then varied the in-plane lattice constant
and positions of the basis atoms (Figure 1(a)) inside the unit cell, while
keeping constant the imposed D3d symmetry 52. According to their results, the
buckled structure has a lower total energy, with a local minimum for $a=3.855$
$\AA$ and a deformation angle of 9.9∘, when compared to the energy of a flat
structure L. C. (2015).

In the present study, the 4$\times$4 monolayer of silicene was characterized
systematically (thus the unit cells were repeated up to four times in the $x$
and $y$ directions) using DFT Kansara et al. (2018). The Si atom bounded to
three nearest neighbour surrounding Si atoms with the Si-Si bond length of
$2.28\,\AA$ and lattice parameter $a$ of $3.84\,\AA$ was assumed prior to
structural relaxation. The optimized lattice parameter, $a=3.86\,\AA$,
correlates quite well with other data, even though the standard GGA functional
method overestimates it. The buckling parameter $\Delta$ is $0.46\,\AA$, which
is also consistent with other studies Lebègue & Eriksson (2009); Björkman
(2014); Yan et al. (2015); Ni et al. (2012); Liu et al. (2012).

In this paper we analyse three different types of substitution in silicene
monolayer by Chromium adatoms as presented in Figure 1. These are: (i) monomer
substitution, where one Si atom in the supercell is substituted by Cr atom
($3.2\%$ substitution), as presented in Figure 1 (b); (ii) horizontal dimer
(HDimer) substitution with two neighbouring Si atoms substituted by Cr atoms
($6.25\%$ substitution), see Figure 1 (c); (iii) vertical dimer (VDimer)
substitution, where one Si atom is replaced by two Cr atoms ($3.2\%$
substitution), as shown in Figure 1 (d). These three structures were modeled
within the optimized 4$\times$4 supercell geometries, and we analysed
behaviour of the electronic and magnetic properties with the strain.

Figure 2: Variation of the Cr-Si and Cr-Cr bond lengths, $d_{X-X}$
(Fig.2.(a)), as well as of the buckling parameter (b) under the biaxial
tensile and compressive strain for the monomer, horizontal dimer, and vertical
dimer substitutions.

The strain is defined as a deformation resulting from external loads or
forces, that may be calculated by the following equation: ${\epsilon={\Delta
a}/a_{0}}$ with ${\Delta a=a-a_{0}}$, where $a$ is the lattice parameter of
the strained silicene and ${a_{0}=3.86\,\AA}$ is the lattice constant of the
unstrained silicene Qin et al. (2012); Mohan et al. (2014); Kaloni et al.
(2013). The model of strained unit cell for the tensile and compression
strains is achieved by varying the lattice constant along the lattice vectors
by the following substitution $a\rightarrow\epsilon a$. Accordingly, the
uniaxial tensile strain or compression ($\Delta a\gtrless 0$ respectively) is
realized by modification of the lattice constant in the $x$ direction, that is
by modification of the lattice vector $\mathbf{a}_{1}$ (mechanical force
applied along the lattice vector $\mathbf{a}_{1}$), whereas the biaxial
tensile or compressive strain is described by modification of both
$\mathbf{a}_{1}$ and $\mathbf{a}_{2}$ lattice vectors (forces are oriented
along both lattice vectors).

The presence of biaxial strain affects the buckling parameter, $\Delta$, as
well as the Cr-Si and Cr-Cr bonds in the silicene monolayer with monomer and
dimer substitutions, respectively. Figure 2 presents the basic parameters of
the relaxed structures as a function of strain and also for all the
substitutions under consideration.

For the monomer substitution (see Figures 2(a)), the bond length between Cr
and Si atoms, $d_{\mathrm{Cr-Si}}$, slightly increases with the strain.
However, there is no clear universal behavior of the buckling parameter with
strain, though one can see that this parameter reaches a maximum for a
specific strain equal $2\%$, and then decreases with increasing magnitude of
either tensile or compressive strain.

For the silicene with HDimer substitution, the tensile strain reduces
monotonically the bond length $d_{\mathrm{Cr-Cr}}$ and the buckling parameter.
However, for the compressive strain one observes increase of the bond length
for Cr-Cr dimers. In turn, the buckling parameter for Cr-Cr HDimers varies
nonmonotonously with the magnitude of compressive strain, i.e., it reaches a
minimum at some magnitude of the strain, see Figure 2(b).

Figure 3: The electronic energy spectrum of silicene under selected values of
biaxial strain (from $-4\%$ to $+4\%$) in the presence of spin-orbit
interaction, and for Cr doping with the three types of substitution: (a-e)
monomer, (f-j) HDimer, and (k-o) VDimer.

In turn, for the VDimer substitution, the strain only very slightly affects
the bound lengths reducing it when the strain takes absolute values larger
than 2$\%$. The corresponding buckling parameter decreases with increasing
tensile strain and increases with increasing magnitude of the compressive
strain up to $4\%$, where it reaches a maximal value, and then decreases with
a further increase of the magnitude of compressive strain.

At this point, one should note that high strain values are experimentally
difficult to obtain. Mohan et al. (2014); Mohiuddin et al. (2009); Tsoukleri
et al. (2009); Huang et al. (2010); Bertolazzi et al. (2011); Fischetti & Laux
(1996). The recent reports about graphene-like monoatomic crystals indicate
that the strain around $4\%$ can be relatively easily obtained. Accordingly,
in this paper we have restricted our considerations to the strain ranging from
$-6\%$ to $8\%$.

##  3 Strain-dependent electronic and magnetic properties

It is well known that strain has a significant impact on electronic and
magnetic properties of 2D crystals Roldán et al. (2015); Wang et al. (2015);
Si et al. (2016); Khatibi et al. (2019a, b). One of the consequences of the
strain in 2D crystal is the electronic bandgap engineering, i.e., strain-
induced bandgap opening Wierzbicki et al. (2015a) or a direct–indirect–direct
bandgap transition in green phosphorene Peng et al. (2014); Shao et al.
(2017).

Figure 3 presents the band structures of strained and unstrained silicene
doped with Cr for the three different types of substitution (Monomer, HDimer
and VDimer), as discussed in Section 2. The band structure has been calculated
along the high-symmetry points of the Brillouin zone, i.e., along the
$\Gamma$-M–K–$\Gamma$ path. In turn, Table 1 collects information about the
band gap in silicene under strain and with different types of Cr substitution.
The electronic band structure of undoped silicene monolayer under strain is
presented in A. Here one needs to remind, that strain in undoped silicene does
not open a gap. A small gap appears only due to spin-orbit interaction. When
neglecting this interaction, the gap in undoped silicene remains equal to zero
(at the Dirac points), see also Figure 5 in Appendix A. Apart from this, due
to hybridization of the 3d-impurity states and those of pure silicene, the
band structure becomes remarkably modified by doping. Each state of the doped
system includes in general contributions due to 3d-transition metals as well
as due to silicon atoms. To show this explicitly, we have presented the so-
called fat-bands structure, where the two contributions are indicated
explicitly with different colors. In B we present fat-bands calculated for
silicene monolayer doped with Cr-atoms (i.e., the band structure projected
over orbitals of Silicone and Chromium atoms). The corresponding results are
shown in Figure 6, and from this figure one can estimate whether a particular
band contributes to conductivity or not. If it is due to impurities only and
is dispersionless around the Fermi level, it does not contribute to
conductivity. If however it is dispersive around the Fermi level, then even
though the silicon contribution is small, it contributes to conductivity.
These features have been taken into account when determining the band gaps. In
B we show the fat-bands for silicene with Cr-Monomers, Cr-HDimers and Cr-
VDimers.

The silicene with Cr-monomers is either metallic or semimetallic, with the
Fermi energy crossing the valence bands. In the presence of compressive strain
the energy gap becomes opened in the spectrum above the Fermi energy, while
the Fermi level is still inside the valence bands. In the presence of tensile
strain the system moves from semimetallic to metallic one. In turn, for
silicene with Cr-HDimer substitution, the electronic structure is only
slightly affected by the strain. The most promising effect of strain on the
electronic structure can be observed for silicene monolayer with Cr-VDimer
substitution. Without strain the system is a semiconductor with the energy gap
equal 0.13 eV. Applying compressive strain one can close the energy gap and
move the Fermi level to the valence bands. The tensile strain, in turn, leads
to reduction of the energy gap for the 4$\%$ strain, and leads to its complete
closure for strain $\geq 6$ $\%$.

Figure 4: Magnetic moment behaviour as a function of the applied biaxial strain for the Cr substitution obtained based on calculations with and without spin-orbit coupling (SOC). The green curves (no SOC) correspond to the collinear calculations discussed in C. Table 1: Band Gap ( eV) changing for Cr substitution during the biaxial strain and applied spin-orbit interaction Strain (%) | -6% | -4% | -2% | 0% | 2% | 4% | 6% | 8%
---|---|---|---|---|---|---|---|---
Monomer-Cr | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0
HDimer-Cr | 0.1 | 0 | 0 | 0 | 0 | 0 | 0 | 0
VDimer-Cr | 0 | 0 | 0.12 | 0.13 | 0 | 0.05 | 0 | 0

Our calculations also reveal the impact of biaxial strain on the magnetic
characteristics of the silicene monolayer doped with the specified above
Monomers, HDimers and VDimers of Cr atoms. The corresponding results are
presented in Figure 4, where spin polarization of silicene monolayer doped
with Cr atoms is shown as a function of strain (the three types of
substitution are presented). Situations in the presence of spin-orbit
interaction and that in the absence of spin-orbit coupling are shown there. In
the latter case, the calculation procedure is described in C. For silicene
with the Cr-monomers, the magnetic moment varies monotonously with the strain
when the spin-orbit coupling is included, while in its absence, the magnetic
moment increases with the magnitude of strain, both tensile and compressive.
Moreover, the difference between these two situations is relatively large. In
turn, for the case of Cr-HDimer, the magnetic moment decreases with the
compressive and tensile strain, and the difference between the cases with and
without spin-orbit interaction is small. For the silicene monolayer doped with
Cr-VDimers, the magnetic moment only weakly depends on the strain for strain
larger than $-4\%$, and the difference between the case with and without spin-
orbit interaction is small except the strain below $-4\%$, where this
difference is relatively large. It is worth noting that the largest magnetic
moment for the unstrained system is for silicene monolayer with Cr-HDimers,
where the changes in the magnetic moment due to strain are also most
pronounced.

##  4 Conclusions

In this work we presented detailed study of electronic and magnetic properties
of silicene doped with Cr atoms in one of the three doping schemes, i.e.,
monomer, HDimer, and VDimer substitutions. Numerical results based on the DFT
calculations clearly show that the way of substitution may substantially
change the structural, electronic and magnetic behaviour of the silicene under
strain. The interplay of doping and strain may be used to engineer band gap,
and thus also character of transport properties from metallic to half-metallic
or semiconducting. Such a strain-induced engineering of transport properties
is important from the practical point of view as it may be used in various
spintronics and/or logic devices. It is expected, that controlling current and
magnetic state with a strain opens new route for nanoelectronics of future
generation.

##  5 Acknowledgments

We thank Prof. J. Barnaś for valuable discussions and reading this manuscript.
This work has been supported by the Norwegian Financial Mechanism 2014- 2021
under the Polish-Norwegian Research Project NCN GRIEG “2Dtronics” no.
2019/34/H/ST3/00515.

Figure 5: The electronic band structure of undoped silicene monolayer under
biaxial strain from $-8\%$ to $+8\%$ and in the absence of spin-orbit
interaction.

##  Appendix A The electronic band structure of undoped silicene

To understand the interplay of the effects due to strain and doping, it is
advisory to analyse the impact of strain on the undoped silicene. In Figure 5
we present the electronic structure of pure silicene monolayer subject to a
biaxial strain from $-8\%$ to $+8\%$, and in the absence of spin-orbit
coupling. It is evident, that the strain has a significant a impact on the
band structure. However, the band gap remains zero as in the absence of
strain. The valence and conduction bands tough each other at the Dirac K
points, independently of the strain, which preserves semi-metallic behavior of
the system with increasing strain, either tensile or compressive. The strain
does not open a gap, however, it modifies asymmetry of the Dirac cones.

##  Appendix B The fat-band structures

Figure 6: A fat-band representation is used to indicate the projected weights
in a silicene monolayer with substituted Chromium atoms in the (top) monomer,
(middle) HDimer, and (bottom) VDimer forms. The yellow and blue colors
represent the transition metal and silicon atoms contributions, respectively.

As already mentioned above, due to hybridization of the 3d-impurity states and
those of pure silicene, the band structure becomes remarkably modified by
doping. Each state of the doped system includes then, in general,
contributions from the 3d-transition metals as well as due to the silicon
atoms. We show this explicitly in Figure 6, where the two contributions are
indicated with different colors for silicene monolayer doped with Cr-atoms. We
show there the fat-bands for Cr-Monomers, Cr-HDimers and Cr-VDimers. From this
figure one can evaluate whether a particular band contributes to transport or
not, and this was taken into account when determining the band gaps presented
in Table 1.

##  Appendix C The electronic band structure of silicene doped by Cr atoms
from collinear calculations

In this section we present electronic spectrum of silicene monolayer doped by
Cr atoms in the presence of biaxial strain. As before we consider the three
types of substitutions discussed in the main text. The electronic structure
has been obtained from collinear (spin polarized) calculations. This procedure
is applicable in the absence of spin-orbit interaction, where spin-up and
spin-down states are well defined. Figure 7 presents the corresponding
electronic band dispersions, where the solid blue and dashed red lines
correspond to the spin-up and spin-down states, respectively.

Figure 7: The electronic energy spectrum of silicene under selected values of
biaxial strain, from $-4\%$ to $+4\%$, for Cr doping in case of three types of
substitution: (a-e) monomer, (f-j) HDimer, and (k-o) VDimer. The solid (blue)
and dashed (red) lines correspond to the spin-up and spin-down, respectively.

## References

  * Bertolazzi et al. (2011) Bertolazzi, S., Brivio, J., & Kis, A. 2011, ACS Nano, 5, 9703, doi: 10.1021/nn203879f
  * Björkman (2014) Björkman, T. 2014, The Journal of Chemical Physics, 141, 074708, doi: 10.1063/1.4893329
  * Burke (2012) Burke, K. 2012, The Journal of Chemical Physics, 136, 150901, doi: 10.1063/1.4704546
  * Cao et al. (2015) Cao, G., Zhang, Y., & Cao, J. 2015, Physics Letters A, 379, 1475, doi: 10.1016/j.physleta.2015.03.021
  * Chen (2016) Chen, S.-H. 2016, Journal of Magnetism and Magnetic Materials, 405, 317, doi: 10.1016/j.jmmm.2015.12.088
  * Chowdhury & Jana (2016) Chowdhury, S., & Jana, D. 2016, Reports on Progress in Physics, 79, 126501, doi: 10.1088/0034-4885/79/12/126501
  * Chu et al. (2009) Chu, M., Sun, Y., Aghoram, U., & Thompson, S. E. 2009, Annual Review of Materials Research, 39, 203, doi: 10.1146/annurev-matsci-082908-145312
  * Ding et al. (2017) Ding, N., Wang, H., Chen, X., & Lawrence Wu, C.-M. 2017, RSC Advances, 7, 10306, doi: 10.1039/C6RA27291F
  * Drummond et al. (2012) Drummond, N. D., Zólyomi, V., & Fal’ko, V. I. 2012, Physical Review B, 85, 075423, doi: 10.1103/PhysRevB.85.075423
  * Dyrdał & Barnaś (2012) Dyrdał, A., & Barnaś, J. 2012, physica status solidi (RRL) - Rapid Research Letters, 6, 340, doi: 10.1002/pssr.201206202
  * Enriquez et al. (2012) Enriquez, H., Vizzini, S., Kara, A., Lalmi, B., & Oughaddou, H. 2012, Journal of Physics: Condensed Matter, 24, 314211, doi: 10.1088/0953-8984/24/31/314211
  * Ernzerhof & Scuseria (1999) Ernzerhof, M., & Scuseria, G. E. 1999, The Journal of Chemical Physics, 110, 5029, doi: 10.1063/1.478401
  * Ezawa (2012) Ezawa, M. 2012, Physical Review Letters, 109, 055502, doi: 10.1103/PhysRevLett.109.055502
  * Faccio et al. (2009) Faccio, R., Denis, P. A., Pardo, H., Goyenola, C., & Mombrú, Á. W. 2009, Journal of Physics: Condensed Matter, 21, 285304, doi: 10.1088/0953-8984/21/28/285304
  * Farjam & Rafii-Tabar (2009) Farjam, M., & Rafii-Tabar, H. 2009, Physical Review B, 80, 167401, doi: 10.1103/PhysRevB.80.167401
  * Farokhnezhad et al. (2017) Farokhnezhad, M., Esmaeilzadeh, M., & Shakouri, K. 2017, Physical Review B, 96, 205416, doi: 10.1103/PhysRevB.96.205416
  * Feng et al. (2012) Feng, B., Ding, Z., Meng, S., et al. 2012, Nano Letters, 12, 3507, doi: 10.1021/nl301047g
  * Fischetti & Laux (1996) Fischetti, M. V., & Laux, S. E. 1996, Journal of Applied Physics, 80, 2234, doi: 10.1063/1.363052
  * Ghasemi et al. (2019) Ghasemi, N., Ahmadkhan Kordbacheh, A., & Berahman, M. 2019, Journal of Magnetism and Magnetic Materials, 473, 306, doi: 10.1016/j.jmmm.2018.10.059
  * Gmitra et al. (2009) Gmitra, M., Konschuh, S., Ertler, C., Ambrosch-Draxl, C., & Fabian, J. 2009, Physical Review B, 80, 235431, doi: 10.1103/PhysRevB.80.235431
  * Greve et al. (2010) Greve, H., Woltermann, E., Quenzer, H.-J., Wagner, B., & Quandt, E. 2010, Applied Physics Letters, 96, 182501, doi: 10.1063/1.3377908
  * Grimme (2006) Grimme, S. 2006, Journal of Computational Chemistry, 27, 1787, doi: 10.1002/jcc.20495
  * Grimme et al. (2010) Grimme, S., Antony, J., Ehrlich, S., & Krieg, H. 2010, The Journal of Chemical Physics, 132, 154104, doi: 10.1063/1.3382344
  * Guo et al. (2015) Guo, X.-X., Guo, P., Zheng, J.-M., Cao, L.-K., & Zhao, P.-J. 2015, Applied Surface Science, 341, 69, doi: 10.1016/j.apsusc.2015.03.002
  * Guo et al. (2013) Guo, Z.-X., Furuya, S., Iwata, J.-i., & Oshiyama, A. 2013, Physical Review B, 87, 235435, doi: 10.1103/PhysRevB.87.235435
  * Hasegawa et al. (2006) Hasegawa, Y., Konno, R., Nakano, H., & Kohmoto, M. 2006, Physical Review B, 74, 033413, doi: 10.1103/PhysRevB.74.033413
  * Huang et al. (2010) Huang, M., Yan, H., Heinz, T. F., & Hone, J. 2010, Nano Letters, 10, 4074, doi: 10.1021/nl102123c
  * Jafari et al. (2020) Jafari, M. A., Kordbacheh, A. A., Mahdian, S., & Ghasemi, N. 2020, Physica E: Low-dimensional Systems and Nanostructures, 117, 113855, doi: 10.1016/j.physe.2019.113855
  * Kaloni et al. (2013) Kaloni, T. P., Cheng, Y. C., & Schwingenschlögl, U. 2013, Journal of Applied Physics, 113, 104305, doi: 10.1063/1.4794812
  * Kansara et al. (2018) Kansara, S., Gupta, S. K., & Sonvane, Y. 2018, Computational Materials Science, 141, 235, doi: 10.1016/j.commatsci.2017.09.037
  * Khatibi et al. (2019a) Khatibi, Z., Namiranian, A., & Panahi, S. 2019a, Physica B: Condensed Matter, 565, 33, doi: 10.1016/j.physb.2018.11.029
  * Khatibi et al. (2019b) Khatibi, Z., Namiranian, A., & Parhizgar, F. 2019b, Diamond and Related Materials, 92, 228, doi: 10.1016/j.diamond.2018.12.007
  * Krompiewski (2021) Krompiewski, S. 2021, Journal of Magnetism and Magnetic Materials, 534, 168036, doi: 10.1016/j.jmmm.2021.168036
  * Krompiewski & Cuniberti (2017) Krompiewski, S., & Cuniberti, G. 2017, Physical Review B, 96, 155447, doi: 10.1103/PhysRevB.96.155447
  * Kumar & Ahluwalia (2012a) Kumar, A., & Ahluwalia, P. 2012a, Materials Chemistry and Physics, 135, 755, doi: 10.1016/j.matchemphys.2012.05.055
  * Kumar & Ahluwalia (2012b) —. 2012b, Physica B: Condensed Matter, 407, 4627, doi: 10.1016/j.physb.2012.08.034
  * Kumar & Ahluwalia (2013) —. 2013, Physica B: Condensed Matter, 419, 66, doi: 10.1016/j.physb.2013.03.029
  * Kumar & Ahluwalia (2012c) Kumar, A., & Ahluwalia, P. K. 2012c, The European Physical Journal B, 85, 186, doi: 10.1140/epjb/e2012-30070-x
  * L. C. (2015) L. C., L. Y. V. 2015, Chinese Physics B, 24, 087309, doi: 10.1088/1674-1056/24/8/087309
  * Lalmi et al. (2010) Lalmi, B., Oughaddou, H., Enriquez, H., et al. 2010, Applied Physics Letters, 97, 223109, doi: 10.1063/1.3524215
  * Le Lay (2015) Le Lay, G. 2015, Nature Nanotechnology, 10, 202, doi: 10.1038/nnano.2015.10
  * Lebègue & Eriksson (2009) Lebègue, S., & Eriksson, O. 2009, Physical Review B, 79, 115409, doi: 10.1103/PhysRevB.79.115409
  * Lee et al. (2005) Lee, M. L., Fitzgerald, E. A., Bulsara, M. T., Currie, M. T., & Lochtefeld, A. 2005, Journal of Applied Physics, 97, 011101, doi: 10.1063/1.1819976
  * Liu et al. (2012) Liu, G., Wu, M. S., Ouyang, C. Y., & Xu, B. 2012, EPL (Europhysics Letters), 99, 17010, doi: 10.1209/0295-5075/99/17010
  * Low et al. (2011) Low, T., Guinea, F., & Katsnelson, M. I. 2011, Physical Review B, 83, 195436, doi: 10.1103/PhysRevB.83.195436
  * Mao et al. (2018) Mao, Y., Xu, H., Yuan, J., & Zhong, J. 2018, Journal of Physics: Condensed Matter, 30, 365001, doi: 10.1088/1361-648X/aad6f9
  * Mohan et al. (2014) Mohan, B., Kumar, A., & Ahluwalia, P. 2014, Physica E: Low-dimensional Systems and Nanostructures, 61, 40, doi: 10.1016/j.physe.2014.03.013
  * Mohiuddin et al. (2009) Mohiuddin, T. M. G., Lombardo, A., Nair, R. R., et al. 2009, Physical Review B, 79, 205433, doi: 10.1103/PhysRevB.79.205433
  * Ni et al. (2012) Ni, Z., Liu, Q., Tang, K., et al. 2012, Nano Letters, 12, 113, doi: 10.1021/nl203065e
  * Peng et al. (2014) Peng, X., Wei, Q., & Copple, A. 2014, Physical Review B, 90, 085402, doi: 10.1103/PhysRevB.90.085402
  * Pereira et al. (2009) Pereira, V. M., Castro Neto, A. H., & Peres, N. M. R. 2009, Physical Review B, 80, 045401, doi: 10.1103/PhysRevB.80.045401
  * Qin et al. (2012) Qin, R., Wang, C.-H., Zhu, W., & Zhang, Y. 2012, AIP Advances, 2, 022159, doi: 10.1063/1.4732134
  * Roldán et al. (2015) Roldán, R., Castellanos-Gomez, A., Cappelluti, E., & Guinea, F. 2015, Journal of Physics: Condensed Matter, 27, 313201, doi: 10.1088/0953-8984/27/31/313201
  * Shao et al. (2017) Shao, C., Yu, X., Yang, N., Yue, Y., & Bao, H. 2017, Nanoscale and Microscale Thermophysical Engineering, 21, 201, doi: 10.1080/15567265.2017.1286421
  * Si et al. (2016) Si, C., Sun, Z., & Liu, F. 2016, Nanoscale, 8, 3207, doi: 10.1039/C5NR07755A
  * Smidstrup et al. (2020) Smidstrup, S., Markussen, T., Vancraeyveld, P., et al. 2020, Journal of Physics: Condensed Matter, 32, 015901, doi: 10.1088/1361-648X/ab4007
  * Stradi et al. (2016) Stradi, D., Martinez, U., Blom, A., Brandbyge, M., & Stokbro, K. 2016, Physical Review B, 93, 155302, doi: 10.1103/PhysRevB.93.155302
  * Takeda & Shiraishi (1994) Takeda, K., & Shiraishi, K. 1994, Physical Review B, 50, 14916, doi: 10.1103/PhysRevB.50.14916
  * Tao et al. (2015) Tao, L., Cinquanta, E., Chiappe, D., et al. 2015, Nature Nanotechnology, 10, 227, doi: 10.1038/nnano.2014.325
  * Tsoukleri et al. (2009) Tsoukleri, G., Parthenios, J., Papagelis, K., et al. 2009, Small, 5, 2397, doi: 10.1002/smll.200900802
  * van Setten et al. (2018) van Setten, M., Giantomassi, M., Bousquet, E., et al. 2018, Computer Physics Communications, 226, 39, doi: 10.1016/j.cpc.2018.01.012
  * Wang et al. (2015) Wang, X., Tantiwanichapan, K., Christopher, J. W., Paiella, R., & Swan, A. K. 2015, Nano Letters, 15, 5969, doi: 10.1021/acs.nanolett.5b02107
  * Weymann et al. (2017) Weymann, I., Zwierzycki, M., & Krompiewski, S. 2017, Physical Review B, 96, 115452, doi: 10.1103/PhysRevB.96.115452
  * Wierzbicki et al. (2015a) Wierzbicki, M., Barnaś, J., & Swirkowicz, R. 2015a, Physical Review B, 91, 165417, doi: 10.1103/PhysRevB.91.165417
  * Wierzbicki et al. (2015b) —. 2015b, Journal of Physics: Condensed Matter, 27, 485301, doi: 10.1088/0953-8984/27/48/485301
  * Wunsch et al. (2008) Wunsch, B., Guinea, F., & Sols, F. 2008, New Journal of Physics, 10, 103027, doi: 10.1088/1367-2630/10/10/103027
  * Yan et al. (2015) Yan, J.-A., Gao, S.-P., Stein, R., & Coard, G. 2015, Physical Review B, 91, 245403, doi: 10.1103/PhysRevB.91.245403
  * Yin & Cohen (1982) Yin, M. T., & Cohen, M. L. 1982, Physical Review B, 25, 7403, doi: 10.1103/PhysRevB.25.7403
  * Zhang et al. (2012) Zhang, P., Li, X., Hu, C., Wu, S., & Zhu, Z. 2012, Physics Letters A, 376, 1230, doi: 10.1016/j.physleta.2012.02.030
  * Zhang et al. (2013) Zhang, X.-L., Liu, L.-F., & Liu, W.-M. 2013, Scientific Reports, 3, 2908, doi: 10.1038/srep02908


# On the Fairness of Generative Adversarial Networks (GANs)

Patrik Joslin Kenfack1, Daniil Dmitrievich Arapov2, Rasheed Hussain3, S.M.
Ahsan Kazmi4, and Adil Khan5,
{1p.kenfack<EMAIL_ADDRESS>{3r.hussain,
<EMAIL_ADDRESS>Innopolis University, Innopolis, Russia

###### Abstract

Generative adversarial networks (GANs) are one of the greatest advances in AI
in recent years. With their ability to directly learn the probability
distribution of data, and then sample synthetic realistic data. Many
applications have emerged, using GANs to solve classical problems in machine
learning, such as data augmentation, class imbalance problems, and fair
representation learning. In this paper, we analyze and highlight fairness
concerns of GANs. In this regard, we show empirically that GANs models may
inherently prefer certain groups during the training process and therefore
they’re not able to homogeneously generate data from different groups during
the testing phase. Furthermore, we propose solutions to solve this issue by
conditioning the GAN model towards samples’ group or using ensemble method
(boosting) to allow the GAN model to leverage distributed structure of data
during the training phase and generate groups at equal rate during the testing
phase.

###### Index Terms:

Generative Adversarial Networks, Fairness, Group Imbalance, Representation
Bias.

##  I Introduction

Machine learning (ML) aims at reproducing human intelligence, for example by
learning patterns in data and trying to generalize them. When data are
labelled the training process is supervised, i.e. each training instance has a
label and the goal of the machine learning model is to learn patterns from
these data and predict labels of new instances. When labels are not provided,
the training process is said to be unsupervised and the model will try to
discover patterns in data. ML models are widely used nowadays to solve
different tasks, such as pattern recognition [1], natural language processing
[2], anomaly detection [3], action recognition[4, 5, 6], scene
classification[7], etc. However, in general the success of these models depend
on features used into the training process [8, 9]. Although these features
might be provided via features engineering, which requires a lot of work and
in certain domains difficult to extract manually, the need for the ML models
to extract meaningful features from the data by themselves has become a hot
topic within the Artificial Intelligence (AI) community [8, 1]. This is where
representation learning (typically Deep Leaning) come in. Basically, the goal
is to identify and extract useful information from data to facilitate or
improve the predictions, or even generate new data. Generative model is one of
the most promising approach in this direction. The main idea behind generative
model, is to capture the underlying distribution of the data in order to be
able to sample most realistic examples from that learned distribution.

Among the most popular generative approaches, we have Variational Auto Ecoders
(VAEs) [10, 11] that consist in two neural networks: an encoder and a decoder
trained jointly. The encoder tries to encode the prior distribution of the
data into a latent space. The decoder tries to reconstruct original examples
using examples sampled from the learned distribution with the goal to minimize
the reconstruction loss. Although VAEs are simple to train, they suffer from
the assumption of an approximate posterior distribution of the observed data.
Thus, this makes VAEs less efficient in generating data that most reflect the
real distribution of the observed data (e.g poor images quality). Generative
Adversarial Networks (GANs) [12] solve the shortcoming of VAEs by trying to
identify the real distribution of the observed data using an approach inspired
by the Game Theory, the goal to find Nash equilibrium between the two
networks, Generator and Discriminator. The generator generates most realistic
examples as possible in order to deceive the discriminator, which the aim is
to distinguish between the real and the generated (fake) data. Generator and
discriminator are trained in competition until Nash equilibrium is reached,
which means that the generator was able to generate data similar to the actual
observed data such that the discriminator wasn’t able to make the difference
between them. Indeed, GANs and its different variants are considered as the
most interesting breakthrough in AI for the last decade, because of its
outstanding data generation capacity, and and its ability to generate diverse
types of data. i.e it allows to generate most realistic images [13], natural
language generation [14], music generation [15], medical data generation [16],
fair representation learning [17] etc.

More importantly, the wide use of AI in different domain in real life has
shown that ML models might exhibit unintended behaviour such as learning
biases in data, even amplified them or generate new biases. i.e a model
trained using data containing sensitive attributes such gender with
disproportionate representation, will tends to be biased toward one groups
[18]. Thus, fairness has become another important research focus within the ML
community. The fairness of important technologies such as GANs must be
particularly analysed. In general, dataset may contain different groups of
data with different distributions, so it is important to analyse and ensure
that rather than prioritizing a single group, the generator of GANs models is
capable of generating examples of different groups at an equal rate. Several
fairness notions have proposed in the literature. Among them, statistical
parity and individual fairness [18] are widely used. Individual fairness
requires that similar individual must receive the same outcome, while
statistical parity requires that classifier predictions outcome must be
independent to the protected attributes. Bias in data can exist in many shapes
and forms, some of which can lead to unfairness in different downstream
learning tasks [19]. Representation bias is a common form of bias that leads
to unfairness decisions, it happens from the way we define and sample from a
population. i.e. if certain groups in the population are not represented or
underrepresented.

In paper this we extensively analyse GANs models with respect to their ability
to generate examples from different groups present in dataset at an equal
rate. To do that, we first present experiments that show that is in most
cases, GANs are unable to generate example from different groups and always
converge to a single group distribution.

The key contributions of this paper are the following:

  * •

We highlight and demonstrate the shortcomings of GANs in reproducing data from
different groups, which may lead to fairness concerns i.e generate biased
synthetic data.

  * •

Point out the negative impact of inherent biases existing in GANs generation
process, on downstream tasks such as classification.

  * •

We propose a GAN boosting strategy to generate data samples from different
groups at equal rate, by creating diverse ensemble of generators.

The rest of this paper in organised as follows: Section II introduces
different GANs models and the theory behind. Section III provides the
experiment results, that highlight the inability of GANs to generate examples
belonging to different group distributions even thought they are were well
balanced in the dataset. Section V presents possible solutions and research
directions to fix this issue and finally the discussions and the conclusion.

##  II Overview of generative adversarial networks

Inspired by the game theory, GANs model consist in two neural networks
(generator and discriminator) trained in adversarial manner. The generator’s
(G) goal is to learn the underlying distribution ($p_{g}$) of the dataset by
sampling as most realistic examples as possible such the discriminator (D),
which the goal is to distinguish between generated examples and real examples,
will not be able to make to difference between examples $G(z)$ sampled from
$G$ using the random noise vector $z$ and real examples $x$. Thus, $G$ and $D$
play a minimax game with value function $V(G,D)$:

$\begin{split}\begin{aligned}
\underset{G}{\mathrm{min}}\>\underset{D}{\mathrm{max}}\>V(D,G)=\;&\mathbb{E}_{x\sim
p_{data}(x)}[\mathrm{log}\>D(x)]\\\ &+\mathbb{E}_{z\sim
p_{z}}[1-\mathrm{log}\>D(G(z))]\end{aligned}\end{split}$ (1)

The parameters in G and D will be updated simultaneously during the training
until $D(G(z))=0.5$, which means that the global optimal solution in achieved
and D cannot identify the different between these two distributions. However,
this original GANs model has presented some limitations and several extended
variants have been proposed. Among these limitations, the problem of mode
collapse is the one that has attracted the most attention. This problem occurs
when the model can only fit a few modes of the data distribution, while
ignoring the majority of them. In this regards, Arjovsky et. al [20] proposed
Wasserstein GAN (WGAN) to improve stability of traditional GANs and resolve
problems like mode collapse.

The original GANs used Multi Layers Perceptron to train the generator, since
Convolutional Neural Networks (CNN) have shown the most interesting results in
extracting feature from images, [21] proposed Deep Conditional Generative
Adversarial Networks (DCGANs). Although DCGANs is able to learn good
representation of images and provides great performances in image generation
tasks, it suffers from the fact that the generator can collapse in training
mode, and thus produces limited varieties of samples [21]. Mirza and Osindero
[22] proposed Conditional Adversarial Networks (CGANs), which extends GANs by
adding extra information $c$ in both the generator and the discriminator, such
that the generator could generate examples constrained by $c$ e.g. class
conditional samples. $c$ can be class label, or any kind of auxiliary
information. Therefore, the objective function is defined as for GANs as
follow:

$\begin{split}\begin{aligned}
\underset{G}{\mathrm{min}}\>\underset{D}{\mathrm{max}}\>V(D,G)=\;&\mathbb{E}_{x\sim
p_{data}(x)}[\mathrm{log}\>D(x|c)]\\\ &+\mathbb{E}_{z\sim
p_{z}}[1-\mathrm{log}\>D(G(z|c))]\end{aligned}\end{split}$ (2)

It has been shown adding side information can significantly improve the
quality of generated samples. Several extensions of CGANs have been proposed
for diverse purpose, for instance Auxiliary GAN (ACGAN) [23] in which the
extra information $c$ is not added to the discriminator but instead the
discrimination perform additional tasks such as predicting the side
information $c$ i.e outputs the class label of training data, therefore ACGAN
use modified version of the loss function to increase the predictive
performance for the class label.

Stacked GANs (SGAN) [24] uses a particular architecture, in which several
encoders (DNN), generators and discriminators are put together. Encoders are
stacked and pre-trained in a bottom-up manner. The input image is passed
through the first encoder, which output a high-level representation and pass
it to next encoder and so on. The uppermost encoder will output the class
label. Provided with pre-trained encoder, generators are stacked and trained
independently and jointly 111Samples generated by a preview generator is used
as input of the next generator in top-down manner. Each generator takes as
inputs high level representation (from the encoder) and a noise vector and
outputs lower-level representation. At each level, one discriminator
distinguishes generated representation from ”real” representation. The main
advantage of SGAN is that it introduces intermediate supervision using
representation discriminator at different representation level.

##  III The shortcomings of GANs in generating data from different group

In the section we discuss applicability of GANs model in generating data in a
settings where the training data contains different groups with balance or
imbalanced representation. We first describe the experimental setup and then
present the results on different dataset.

###  III-A Experimental setup

This study focuses on two widely used dataset: The MNIST [25] dataset, which
contains 70,000 labelled examples of 28 x 28 images of hand-written digits,
and the SVHN [26] dataset, which contains 73,257 labelled examples (10
classes) of 32 x 32 colored images of house number digits. CelibA is a large-
scale and diverse face attribute dataset, which contains 202,599 images of
10,177 identities.

Figure 1: Example of 2 groups created from MNIST dataset. White digits having
white background and black digits with black background Figure 2: Example of
groups created from SVHN dataset. First two rows show samples from the first
group (normal samples) and last two rows the second groups (darkened)

To create different groups, in MNIST dataset, we artificially inverted colors
of certain images from black to white (Fig. 1) with different white-to-all
ratio (30%, 50%, 70%). We trained different generators on the datasets with
different representation ratio of groups, and analyse their ability to recover
the distributions of both groups at the same rate. Similarly, for SVHN
dataset, we darkened certain images (Fig. 2). We have considered samples
brightness as a sensitive attribute, forming new group out of SVHN samples,
divided by 2.

###  III-B Results on MNIST dataset

On MNIST dataset, we trained SGAN model with equal representation of different
groups in the dataset (black and white digits) and analysed the distribution
of groups inside the data generated by the generator. We used mean of the
sample to decide which group each example belongs to: if the mean is greater
than 0.5, then the sample is considered white and black otherwise. On overall,
even thought groups were equally represented in the training dataset, the
model was not able to generate white and black digits at equal rate. Figure 3
presents joint distribution of generated digits means. We trained generators
10 times, sampling 100 images per each of 10 classes. Figure 3 shows
distribution of samples over all 10 experiments.

For instance Fig. 3(a) shows that majority of samples are white, while in Fig.
3(b) majority are black. In most cases the group distribution is not well
balanced. The desired case is the one in which the group distributions are the
same (as in Fig. 3(j)). However throughout our experiments, this case has
rarely occurred.

a)$\displaystyle{0}$$\displaystyle{2}$$\displaystyle{4}$$\displaystyle{6}$$\displaystyle{8}$Densityblackwhiteb)blackwhitec)blackwhited)blackwhitee)blackwhite$\displaystyle{0.0}$$\displaystyle{0.5}$$\displaystyle{1.0}$f)$\displaystyle{0}$$\displaystyle{2}$$\displaystyle{4}$$\displaystyle{6}$$\displaystyle{8}$Densityblackwhite$\displaystyle{0.0}$$\displaystyle{0.5}$$\displaystyle{1.0}$g)blackwhite$\displaystyle{0.0}$$\displaystyle{0.5}$$\displaystyle{1.0}$h)blackwhite$\displaystyle{0.0}$$\displaystyle{0.5}$$\displaystyle{1.0}$i)blackwhite$\displaystyle{0.0}$$\displaystyle{0.5}$$\displaystyle{1.0}$j)blackwhite

Figure 3: Distribution of digits generated by Stacked GAN trained on the
modified MNIST dataset with balanced group representations: black digits
($mean<0.5$); white digits ($mean>0.5$). The generator evenly produces digits
from both groups, but not in most cases (a, b, c, d, f, g).

The discrepancy is even more serious if the groups’ representations are not
well balanced in training set. We designed different experiment settings
where, the GANs model was trained with imbalance groups representations. In
the first setting, the model was trained using a dataset containing 70% of
white digits and 30% of black digits, and the opposite in the second setting.
In most cases, out of a thousand examples generated, most are from the well
represented group. This behaviour shows groups are not treated equally by the
model. Figure 4 shows the group distributions of examples generated using the
trained model with the imbalanced group representation.

a)$\displaystyle{0}$$\displaystyle{2}$$\displaystyle{4}$$\displaystyle{6}$$\displaystyle{8}$$\displaystyle{10}$Densityblackb)blackwhitec)blackd)blacke)blackwhite$\displaystyle{0.0}$$\displaystyle{0.5}$$\displaystyle{1.0}$f)$\displaystyle{0}$$\displaystyle{2}$$\displaystyle{4}$$\displaystyle{6}$$\displaystyle{8}$$\displaystyle{10}$Densitywhite$\displaystyle{0.0}$$\displaystyle{0.5}$$\displaystyle{1.0}$g)white$\displaystyle{0.0}$$\displaystyle{0.5}$$\displaystyle{1.0}$h)white$\displaystyle{0.0}$$\displaystyle{0.5}$$\displaystyle{1.0}$i)whiteblack$\displaystyle{0.0}$$\displaystyle{0.5}$$\displaystyle{1.0}$j)white

Figure 4: Distribution of digits generated using GAN trained on MNIST dataset
with imbalance group representation. (a, b, c, d, e) dataset with 70% of black
digits vs 30% of white digits. (f, g, h, i, j) dataset with 30% of black
digits vs 70% of white digits. The generator mainly generates digits that
belong to the well representation group.

###  III-C Results on SVHN dataset

On SVHN datset we trained almost the same SGAN, as reported in the Fig. 5, we
trained 5 different generators using the same dataset. Group representations
were balanced within the training data. However, during the testing phase, the
examples sampled from the trained generator had a different distribution. This
suggests that the generator converged to a particular group during training
(Fig. 5).

a)$\displaystyle{0}$$\displaystyle{5}$$\displaystyle{10}$Densityb)$\displaystyle{0}$$\displaystyle{5}$$\displaystyle{10}$Densityc)$\displaystyle{0}$$\displaystyle{5}$$\displaystyle{10}$Densityd)$\displaystyle{0}$$\displaystyle{5}$$\displaystyle{10}$Density$\displaystyle{0.0}$$\displaystyle{0.2}$$\displaystyle{0.4}$$\displaystyle{0.6}$$\displaystyle{0.8}$$\displaystyle{1.0}$e)$\displaystyle{0}$$\displaystyle{5}$$\displaystyle{10}$Density

Figure 5: Results on the SVHN dataset. In most cases the generated examples
are concentrated around one group.

###  III-D CelibA dataset

CelibA dataset contains 61.4% female faces versus 38.6% male faces. We
generated different images using Progressive Growing of GANs (PGAN) model,
which is good in generating high resolution images [13], and it turns out that
well-represented groups (female faces) are the most generated by the
generator, while there is a lack of dark-skinned faces throughout several runs
of the trained generator (Figure 6). Moreover, most of the few black faces
generated were of worse quality.

Figure 6: Images generated using PGAN. Overall the generator predominately
produces pictures of females (well represented groups) with lighter skins
tones.

##  IV Discussions

In this section, we examine the impact that the inherent biases produced by
GANs models can have when they are used to generate data for other downstream
tasks, such as data augmentation, class imbalance problem and fair
representation learning.

###  IV-A Class imbalance problem & Data augmentation

GANs model can be used to increase the size of training dataset when there is
not enough data available. For example, dataset of credit card fraud will
normally contains more records of normal transactions than fraudulent ones.
Similarly, database of medical information will contains higher number of
healthy patients than affected ones. For classification tasks, when different
classes are not uniformly distributed within the dataset, the accuracy can be
significantly impacted negatively. Indeed, classifiers using this kind of
dataset are biased towards the underrepresented classes. The good generative
properties of GAN models gives the possibility to restore the dataset balance
by generating examples from minority-class (oversampling).

Another useful application of GAN models for data augmentation is when due to
privacy concerns, real dataset cannot be used directly. For example, medical
records could contain a lot of sensitive information about patients that
cannot be shared or are protected. GAN can be use in this case to generate
synthetic dataset, that could be then used for a given task e.g
classification.

There are various applications of GANs in this way. Tanaka et. al [27]
analysed the performances of GANs model when used to balance the dataset and
generate synthetic data for classification tasks, they showed that synthetic
data generated by GAN can in some cases lead to better accuracy performance
than real dataset. For instance, Giovanni et. al [28] used to restore the
balance of an imbalanced dataset. Other applications to balance dataset for
Speech Emotion Recognition [29].s

In this paper, we argue that despite the fact GAN provides interesting
property for data augmentation, their inability to generate different groups
that are uniformly distributed may rise fairness concerns. As the results of
the previous section showed, GAN model can inherently tends to converge to
prefer certain groups in the dataset and therefor generates a much larger
number of instances from these groups. Thus, when used for dataset
augmentation, this may leads to the creation of biased dataset, i.e. dataset
in which different groups are not equally represented. It has been shown that
classifiers trained using this kind of dataset may lead to unfair decisions
[19].

###  IV-B Using GANs for fair representation learning

Adversarial training has been used to learn ”fair” representation of data. The
goal of this approach is to mitigate unfairness by obstructing information
concerning a protected attribute (Gender, age, religion etc…) as much as
possible, while preserving other useful information [30]. The objective of the
leaned representation is to achieve desired fairness notion such as
statistical parity or individual fairness in downstream tasks as
classification. The potential problem with approaches that use adversarial
learning to mitigate unfairness is the bias toward subgroups. In fact, when
the adversarial model attempts to mitigate unfairness by removing all
information about a given sensitive attribute, there is a risk that the
generator will only converge to certain subgroups. For instance, if the
sensitive is gender, with each group having subgroups (for example, pregnant
women) then the generator may focus on the main subgroups (non-pregnant women)
and then unfairness will shift away from a male-female problem to a pregnant
and non-pregnant problem.

##  V Proposed solutions

In this section, we discuss possible solutions that could help solve the
problem presented in the previous sections.

###  V-A Using Conditional GANs

One possible solution to solve the stated issue is to use Conditional GANs
[22]. Instead of conditioning the generator and discriminator on the class
label $y$, one can condition them with the groups label and necessary class
label. The trained generator will then allows to generate samples conditioned
by group label. e.g. generate images of imaginary actors of given gender
(group) like male or female. This solution needs to be explored as quality of
generated data. A limitation of this approach is that biases may shift to
subgroups.

###  V-B Using model ensemble

Another interesting direction to explore is ensemble learning. The main idea
behind ensemble model is to train different model and combine them to produce
one model that outperform all of them. Boosting is an ensemble method
technique widely for classification tasks. The idea is to create successive
model during the training process, in which a newly created model attempts to
correct the errors from the preview model. Models are added to the ensemble
until errors are completely eliminated on training set or the maximum number
of models is reached. In section this subsection, we describe a technique
works similarly to boosting algorithm, to build ensemble of generators that
combined together, could generate data with an equal representation rate of
different groups. During the training process, different generators can be
created base on the previous ones and added to the ensemble. More precisely,
while creating new generator $G_{i}$ based on the set of the already existing
generators $\textbf{G}=\\{G_{1},G_{2},...,G_{i-1}\\}$, one should consider the
set of corresponding samples generated by each existing generator $G_{i}$. i.e
$\textbf{Q}=\\{Q_{1},Q_{2},...,Q_{i-1}\\}$, where $Q_{i}$ is the set of
samples generated by the generator $G_{i}$ i.e.
$Q_{i}=\\{x_{1},x_{2},...,x_{m}\\},\;\forall j\in\\{1,...,i-1\\}$; where $m$
is a hyper-parameter.

We can then add a regularization term to the loss function of generators, that
will enforces the distance between the newly and previously generated samples.
For a given set of fake samples $\textbf{F}=\\{f_{1},f_{2},...,f_{m}\\}$,
sampled by the generator at the current batch, the regularization term
$\Theta$ can define as following:

$\Theta=\frac{\Phi(\textbf{Q})}{\sum_{Q_{i}\in\textbf{Q}}\sum_{x\in
Q_{i}}\>\sum_{f\in\textbf{F}}\parallel x-f\parallel}$ (3)

Where $\Phi$ is a given function that depends on Q used as normalization term.

Thus the new loss of a generator $G_{i}$ is define as follow:

$\mathcal{L}^{(G_{i})}=-\mathbb{E}_{z}\mathrm{log}\;(1-\mathrm{D}(\mathrm{G}(z)))+\Theta_{i}$
(4)

During the testing phase, the objective is to generate samples from all
existing groups (and possibly subgroups). To do so, we should not use
generators, whose generation space is already covered by already generated
samples ($\textbf{g}=\\{g_{1},g_{2},...,g_{p}\\}$), but rather generators that
can generate samples different from those already generated. So, given a
probability density functions $\\{P_{1},P_{2},...,P_{n}\\}$ for all the
generators $\\{G_{1},G_{2},...,G_{n}\\}$ and probability P for the generator
to be chosen, we have $\textbf{P}(G_{i})<\textbf{P}(G_{j})$ iff
$\sum_{g\in\textbf{g}}P_{i}(g)>\sum_{g\in\textbf{g}}P_{j}(g)$.

Although we can not be sure about the form of the probability density function
for the different generators, we may use a Normal distribution as an estimator
for the function. In this case,
$P_{i}=\mathcal{N}(E[X_{i}],\sqrt{Var[X_{i}]})$.

More specifically, probability for some generator $G_{i}$ to be chosen is
given by the following equation:

$\textbf{P}(G_{i})=\frac{\displaystyle\prod_{f\in\textbf{f}}e^{1-P_{i}(f)}}{\displaystyle\sum_{P\in\textbf{P}}\prod_{f\in\textbf{f}}e^{1-P(f)}}$
(5)

In other word, $\textbf{P}(G_{i})$ is defined as the softmax over probability
that there is no point at $f$.

##  VI Related work

The existing works, which are similar to ours, are those that seek to solve
the GAN mode collapse problem in GANs, by improving the generalization
performance of the model, in order to generate diverse samples. For instance,
Qi et. al [31] proposed a regularization method for mode seeking to address
the mode collapse problem in CGANs. However, diversity of generated samples is
different from the goal of fairly generating data from different groups. In
other words, a diverse set of samples may also contain underrepresented groups
samples.

Xu et. al [32] proposed FairGAN, a GAN architecture to generate ”fair” data
with respect to the statistic parity, i.e remove all the information about
sensitive attribute from generated data, while retaining as much information
as possible. In their architecture, they have included another discriminator,
that distinguish whether the generated samples are from protected group or
unprotected group. Although their work also focuses on fairness in GANs, the
goal is different from ours in that we are trying to alleviate the problem of
representational bias, rather than achieving a certain notion of fairness.
However, in future work, we will conduct comprehensive empirically comparisons
of our boosting approach and FairGAN approach in terms of accuracy-fairness
tradeoff.

## Conclusion

In this paper, we highlighted fairness concern of GANs model. We have shown
that when different groups are present in the dataset, the GAN model is not
able to generate them homogeneously during the test phase, which can lead to
representation bias. We discussed existing applications of GANs and argued
that techniques using GANs to generate data for downstream tasks should
consider this issue in order to avoid generating biased data, inherently
introduced by GANs. We proposed a boosting strategy that can help to solve
this issue. This strategy will be further explored in our future work and will
be compared to other approaches in terms of fairness and accuracy.

## References

  * [1] W. Liu, Z. Wang, X. Liu, N. Zeng, Y. Liu, and F. E. Alsaadi, “A survey of deep neural network architectures and their applications,” _Neurocomputing_ , vol. 234, pp. 11–26, 2017.
  * [2] A. Vaswani, N. Shazeer, N. Parmar, J. Uszkoreit, L. Jones, A. N. Gomez, L. Kaiser, and I. Polosukhin, “Attention is all you need,” _arXiv preprint arXiv:1706.03762_ , 2017.
  * [3] A. R. Rivera, A. Khan, I. E. I. Bekkouch, and T. S. Sheikh, “Anomaly detection based on zero-shot outlier synthesis and hierarchical feature distillation,” _IEEE Transactions on Neural Networks and Learning Systems_ , 2020.
  * [4] Y. Gavrilin and A. Khan, “Across-sensor feature learning for energy-efficient activity recognition on mobile devices,” in _2019 International Joint Conference on Neural Networks (IJCNN)_. IEEE, 2019, pp. 1–7.
  * [5] K. Sozykin, S. Protasov, A. Khan, R. Hussain, and J. Lee, “Multi-label class-imbalanced action recognition in hockey videos via 3d convolutional neural networks,” in _2018 19th IEEE/ACIS International Conference on Software Engineering, Artificial Intelligence, Networking and Parallel/Distributed Computing (SNPD)_. IEEE, 2018, pp. 146–151.
  * [6] A. M. Khan, Y.-K. Lee, S. Lee, and T.-S. Kim, “Accelerometer’s position independent physical activity recognition system for long-term activity monitoring in the elderly,” _Medical & biological engineering & computing_, vol. 48, no. 12, pp. 1271–1279, 2010.
  * [7] S. Protasov, A. M. Khan, K. Sozykin, and M. Ahmad, “Using deep features for video scene detection and annotation,” _Signal, Image and Video Processing_ , vol. 12, no. 5, pp. 991–999, 2018.
  * [8] Y. Bengio, A. Courville, and P. Vincent, “Representation learning: A review and new perspectives,” _IEEE transactions on pattern analysis and machine intelligence_ , vol. 35, no. 8, pp. 1798–1828, 2013.
  * [9] A. Khan and K. Fraz, “Post-training iterative hierarchical data augmentation for deep networks,” _Advances in Neural Information Processing Systems_ , vol. 33, 2020.
  * [10] D. P. Kingma and M. Welling, “Auto-encoding variational bayes,” _arXiv preprint arXiv:1312.6114_ , 2013.
  * [11] D. J. Rezende, S. Mohamed, and D. Wierstra, “Stochastic backpropagation and approximate inference in deep generative models,” in _International conference on machine learning_. PMLR, 2014, pp. 1278–1286.
  * [12] I. J. Goodfellow, J. Pouget-Abadie, M. Mirza, B. Xu, D. Warde-Farley, S. Ozair, A. Courville, and Y. Bengio, “Generative adversarial networks,” _arXiv preprint arXiv:1406.2661_ , 2014.
  * [13] T. Karras, T. Aila, S. Laine, and J. Lehtinen, “Progressive growing of gans for improved quality, stability, and variation,” _arXiv preprint arXiv:1710.10196_ , 2017.
  * [14] J. Guo, S. Lu, H. Cai, W. Zhang, Y. Yu, and J. Wang, “Long text generation via adversarial training with leaked information,” in _Proceedings of the AAAI Conference on Artificial Intelligence_ , vol. 32, no. 1, 2018.
  * [15] O. Mogren, “C-rnn-gan: Continuous recurrent neural networks with adversarial training,” _arXiv preprint arXiv:1611.09904_ , 2016.
  * [16] C. Esteban, S. L. Hyland, and G. Rätsch, “Real-valued (medical) time series generation with recurrent conditional gans,” _arXiv preprint arXiv:1706.02633_ , 2017.
  * [17] D. Madras, E. Creager, T. Pitassi, and R. Zemel, “Learning adversarially fair and transferable representations,” in _International Conference on Machine Learning_. PMLR, 2018, pp. 3384–3393.
  * [18] C. Dwork, M. Hardt, T. Pitassi, O. Reingold, and R. Zemel, “Fairness through awareness,” in _Proceedings of the 3rd innovations in theoretical computer science conference_ , 2012, pp. 214–226.
  * [19] N. Mehrabi, F. Morstatter, N. Saxena, K. Lerman, and A. Galstyan, “A survey on bias and fairness in machine learning,” _arXiv preprint arXiv:1908.09635_ , 2019.
  * [20] M. Arjovsky, S. Chintala, and L. Bottou, “Wasserstein generative adversarial networks,” in _International conference on machine learning_. PMLR, 2017, pp. 214–223.
  * [21] A. Radford, L. Metz, and S. Chintala, “Unsupervised representation learning with deep convolutional generative adversarial networks,” _arXiv preprint arXiv:1511.06434_ , 2015.
  * [22] M. Mirza and S. Osindero, “Conditional generative adversarial nets,” _arXiv preprint arXiv:1411.1784_ , 2014.
  * [23] A. Odena, C. Olah, and J. Shlens, “Conditional image synthesis with auxiliary classifier gans,” in _International conference on machine learning_. PMLR, 2017, pp. 2642–2651.
  * [24] X. Huang, Y. Li, O. Poursaeed, J. Hopcroft, and S. Belongie, “Stacked generative adversarial networks,” in _Proceedings of the IEEE conference on computer vision and pattern recognition_ , 2017, pp. 5077–5086.
  * [25] Y. LeCun, L. Bottou, Y. Bengio, and P. Haffner, “Gradient-based learning applied to document recognition,” _Proceedings of the IEEE_ , vol. 86, no. 11, pp. 2278–2324, 1998.
  * [26] Y. Netzer, T. Wang, A. Coates, A. Bissacco, B. Wu, and A. Y. Ng, “Reading digits in natural images with unsupervised feature learning,” 2011.
  * [27] F. H. K. d. S. Tanaka and C. Aranha, “Data augmentation using gans,” _arXiv preprint arXiv:1904.09135_ , 2019.
  * [28] G. Mariani, F. Scheidegger, R. Istrate, C. Bekas, and C. Malossi, “Bagan: Data augmentation with balancing gan,” _arXiv preprint arXiv:1803.09655_ , 2018\.
  * [29] A. Chatziagapi, G. Paraskevopoulos, D. Sgouropoulos, G. Pantazopoulos, M. Nikandrou, T. Giannakopoulos, A. Katsamanis, A. Potamianos, and S. Narayanan, “Data augmentation using gans for speech emotion recognition.” in _Interspeech_ , 2019, pp. 171–175.
  * [30] R. Feng, Y. Yang, Y. Lyu, C. Tan, Y. Sun, and C. Wang, “Learning fair representations via an adversarial framework,” _arXiv preprint arXiv:1904.13341_ , 2019.
  * [31] Q. Mao, H.-Y. Lee, H.-Y. Tseng, S. Ma, and M.-H. Yang, “Mode seeking generative adversarial networks for diverse image synthesis,” in _Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition_ , 2019, pp. 1429–1437.
  * [32] D. Xu, S. Yuan, L. Zhang, and X. Wu, “Fairgan: Fairness-aware generative adversarial networks,” in _2018 IEEE International Conference on Big Data (Big Data)_. IEEE, 2018, pp. 570–575.