# Atomic ionization by scalar dark matter and solar scalars

H. B. Tran Tan1,2 A. Derevianko1 V. A. Dzuba2 V. V. Flambaum2,3 1Department of
Physics, University of Nevada, Reno, Nevada 89557, USA  2School of Physics,
University of New South Wales, Sydney 2052, Australia  3Helmholtz Institute
Mainz, Johannes Gutenberg University, 55099 Mainz, Germany

###### Abstract

We calculate the cross-sections of atomic ionization by absorption of scalar
particles in the energy range from a few eV to 100 keV. We consider both
nonrelativistic particles (dark matter candidates) and relativistic particles
which may be produced inside Sun. We provide numerical results for atoms
relevant for direct dark matter searches (O, Na, Ar, Ca, Ge, I, Xe, W and Tl).
We identify a crucial flaw in previous calculations and show that they
overestimated the ionization cross sections by several orders of magnitude due
to violation of the orthogonality of the bound and continuum electron wave
functions. Using our computed cross-sections, we interpret the recent data
from the Xenon1T experiment, establishing the first direct bounds on coupling
of scalars to electrons. We argue that the Xenon1T excess can be explained by
the emission of scalars from the Sun. While our finding is in a similar
tension with astrophysical bounds as the solar axion hypothesis, we establish
direct limits on scalar DM for the $\sim 1-10\,\mathrm{keV}$ mass range. We
also update axio-ionization cross-sections. Numerical data files are provided.

The nature of Dark Matter (DM) remains an unsolved problem of modern physics.
Current experiments searching for the simplest form of the weakly interacting
massive particles (WIMPs) have exhausted their predicted parameter space
without obtaining unequivocal signals [1, 2, 3, 4, 5, 6, 7, 8]. With
experimental tests for supersymmetric theories, which supply WIMP candidates
[9, 10, 11, 12, 13, 14, 15, 16], also experiencing difficulties, there is a
growing interest in other DM candidates, including pseudo-scalars (axions and
axion-like particles [17, 18, 19, 20, 21, 22, 23, 24, 25, 26, 27, 28, 29, 30,
31, 32]) and scalars. In particular, the intriguing excess rate in the recent
Xenon1T experiment [33] was attributed to solar axions (at the $3.5\sigma$
level). While this interpretation remains in tension with astrophysical bounds
[34], here we examine if the scalar particles could account for the observed
Xenon1T excess. Not only we show that the previous work [35] substantially
overestimated the cross-sections of atomic ionization by scalars and provide
cross-section data for a variety of detectors, we also demonstrate that the
Xenon1T excess can be explained by the emission of scalars from the Sun. While
our finding is in a similar tension with astrophysical bounds as the solar
axion hypothesis, we establish direct limits on scalar DM for the $\sim
1-10\,\mathrm{keV}$ mass range.

Examples of scalars are abundant and include the scalar familon, the
sgoldstino, the dilaton, the relaxon, moduli and Higgs-portal DM. Among these,
the Higgs-portal scalar DM has become particularly well motivated since the
discovery of the Higgs particle at the Large Hadron Collider [36]. Detection
techniques for ultralight DM scalars of mass $m\ll 1\,\mathrm{eV}$ rely on a
variety of techniques: atomic clocks [37, 38, 39, 40, 41, 42], resonant-mass
detectors [43], accelerometers [44], atomic gravitational wave detectors [45],
laser and maser interferometry [39, 46, 47, 48], atom interferometers [49],
pulsar timing and nongravitational lensing [50]. Interactions between heavier
scalars and electrons can drive detectable bound-bound transitions in atomic
and molecular systems [51] if their energies match the transition frequencies.
Here we focus on DM scalars of mass $m\sim O(\mathrm{keV})$ which can drive
bound-continuum transitions, leading to ionization of atoms. This ionization
channel contributes to the detection rates of DM particle detectors. Because
DM halo particles are nonrelativistic, ultralight DM scalar candidates can not
be probed directly in particle detectors due to their small energies.
Nevertheless, these detectors may be sensitive to the fluxes of ultralight
scalars produced in the Sun. Solar scalars may have enough energy to ionize
the detector’s atoms, leading to measurable signals.

In this work we consider the ionization of atoms by scalar particles. This
process was considered alongside the axioelectric effect in an attempt to
explain the signal modulation observed by DAMA/NaI [35]. It was later pointed
out that Ref. [35] underestimated the axioelectric effects by several orders
of magnitude due to the omission of the leading term in the axion-electron
Hamiltonian [52]. Here, we show that Ref. [35] overestimated the scalar
ionization process by several orders of magnitude due to the use of electron
plane waves which do not obey orthogonality conditions to electron bound
states. Furthermore, the calculation in Ref. [35] used a simple model of atoms
which ignored relativistic and many-body effects. As a result, a relativistic
Hartree-Fock (HF) atomic calculation of the ionization cross section is needed
and is performed, for the first time, in this paper. The results of this work
are to be used in the experiments searching for DM and solar particles using
underground detectors.

Theory - The Lagrangian density of a scalar field $\phi$ coupled to an
electron field $\psi$ may be written in the form

$\mathcal{L}_{\phi\bar{e}e}=\sqrt{\hbar
c}g_{\phi\bar{e}e}\phi\bar{\psi}\psi\,,$ (1)

where $g_{\phi ee}$ is a dimensionless coupling constant. We consider the
ionization process where an atomic electron in the bound state absorbs a
scalar particle $\phi$ with energy $\epsilon$ and is ejected into the
continuum. The ionization cross section may be written in the form

$\sigma_{\phi}=g_{\phi\bar{e}e}^{2}(c/v)Q(\epsilon)a_{0}^{2}\,,$ (2)

where $c$ is the speed of light, $v$ is the scalar particle’s velocity in the
laboratory frame and $a_{0}\approx 5.29\times 10^{-11}\,{\rm m}$ is the Bohr
radius. In standard halo models of nonrelativistic DM, a typical velocity is
$v\sim 10^{-3}c$. We also consider the case of ultrarelativistic scalars,
which may be produced in the solar interior. Therefore, in addition to the
nonrelativistic scalar case, we performed calculations for $m=0$. All
intermediate cases are between the curves for the nonrelativistic case and
$m=0$ case in the graphs below.

The dimensionless form-factor $Q(\epsilon)$ may be presented as a multipolar
expansion (see the appendix for a derivation)

$Q(\epsilon)=\frac{\pi\hbar^{2}c^{2}}{\epsilon
a_{0}^{2}}\sum_{bc}\sum_{L=0}^{\infty}(2L+1)\left|\langle
b||\upsilon_{L}||c\rangle\right|^{2}\,,$ (3)

where the reduced matrix element $\langle b||\upsilon_{L}||c\rangle$ is given
by

$\displaystyle\langle b||\upsilon_{L}||c\rangle$
$\displaystyle=(-1)^{j_{b}-1/2}\sqrt{(2j_{b}+1)(2j_{c}+1)}$ (4)
$\displaystyle\times\begin{pmatrix}j_{b}&j_{c}&L\\\
-1/2&1/2&0\end{pmatrix}\Pi(l_{b}+L+l_{c})$
$\displaystyle\times\int\left(f_{\epsilon_{b}}^{\kappa_{b}}f_{\epsilon_{c}}^{\kappa_{c}}-\alpha^{2}g_{\epsilon_{b}}^{\kappa_{b}}g_{\epsilon_{c}}^{\kappa_{c}}\right)j_{L}(kr)dr\,.$

Here, the functions $f$ and $g$ are the upper and lower radial components of
the electron wave function

$\psi({\bf
r})=\frac{1}{r}\begin{pmatrix}f_{\epsilon}^{\kappa}(r)\Omega_{m}^{\kappa}\\\
i\alpha g_{\epsilon}^{\kappa}(r)\Omega_{m}^{-\kappa}\end{pmatrix}\,,$ (5)

where $\alpha$ is the fine structure constant, $\epsilon_{b}$ is the
electronic bound state’s energy, $j_{b}$ its total angular momentum, $l_{b}$
its orbital angular momentum and $\kappa_{b}\equiv(j_{b}+1/2)(-1)^{l_{b}+1}$.
The quantities $\epsilon_{c}$, $j_{c}$, $l_{c}$ and $\kappa_{c}$ are similarly
defined for the continuum state. We assumed that the bound state wave
functions are normalized to unity whereas the continuum wave functions are
normalized to the $\delta$-function of energy,
$\delta(\epsilon_{c}-\epsilon^{\prime}_{c})$. Note that the continuum state
energy necessarily satisfies the energy conservation condition
$\epsilon_{c}=\epsilon_{b}+\epsilon$. The function $\Pi(x)$ imposes parity
selection rules; it returns 1 if $x$ is even and zero if $x$ is odd. The
quantity $k=\sqrt{(\epsilon/c)^{2}-(mc)^{2}}/\hbar=\epsilon v/(\hbar c^{2})$
is the wave number of the scalar particle and $j_{L}(...)$ is the spherical
Bessel function of order $L$. The summation over $L$ saturates very rapidly
and we cut it at $L=3$. The electronic bound and continuum wave functions
needed for the radial integral in Eq. (4) are calculated using the
relativistic HF method. The HF energies of all core states for several atoms
of interest may be found in the Supplemental Materials.

It is worth emphasizing the failure of the photo-ionization-derived intuition
(see, e.g. [53]) about the relative importance of various multi-polar
contributions to Eq. (3). We find that the monopole $L=0$ contribution is
suppressed due to the following ‘orthogonality’ arguments. The integral in the
monopole ($L=0$) contribution to the matrix element in Eq. (4) may be
presented as

$\displaystyle\int\left(f_{\epsilon_{b}}^{\kappa_{b}}f_{\epsilon_{c}}^{\kappa_{c}}-\alpha^{2}g_{\epsilon_{b}}^{\kappa_{b}}g_{\epsilon_{c}}^{\kappa_{c}}\right)j_{0}(kr)dr=2\alpha^{2}\int
g_{\epsilon_{b}}^{\kappa_{b}}g_{\epsilon_{c}}^{\kappa_{c}}dr$ (6)
$\displaystyle+\int\left(f_{\epsilon_{b}}^{\kappa_{b}}f_{\epsilon_{c}}^{\kappa_{c}}-\alpha^{2}g_{\epsilon_{b}}^{\kappa_{b}}g_{\epsilon_{c}}^{\kappa_{c}}\right)(j_{0}(kr)-1)dr\,,$

where we have used the orthogonality condition between the bound and continuum
radial wave functions,
$\int\left(f_{\epsilon_{b}}^{\kappa_{b}}f_{\epsilon_{c}}^{\kappa_{c}}+\alpha^{2}g_{\epsilon_{b}}^{\kappa_{b}}g_{\epsilon_{c}}^{\kappa_{c}}\right)dr=0$.
In the nonrelativistic approximation, the first term in the right hand side of
Eq. (6) vanishes and the second term is small since
$|j_{0}(kr)-1|\approx(kr)^{2}/6$. For massive nonrelativistic scalar
$kr\approx 0$. For massless scalar, $kr\ll 1$ in the interested energy range.

Ref. [35] calculated the integral in Eq. (6) in the nonrelativistic electron
limit and obtained nonzero result when $j_{0}(kr)\approx 1$. This is because
Ref. [35] used, for the outgoing electron, plane waves instead of proper
continuum wave functions, violating the orthogonality condition. As a result
Ref. [35] strongly overestimated the cross section. In Fig. 1, we show the
results of computing the form factor $Q(\epsilon)$ for the ionization of Xe by
massive scalars using (a) HF continuum wave function, (b) free continuum wave
function and (c) free continuum wave function with orthogonality condition
enforced manually. It is clear that the naive use of free continuum wave
function gives incorrect results 111We point out that a similar
‘orthogonality’ condition also exists for the axioelectric effect, affecting
the $L=1$ multipole, as may be seen in Eq. (A18) in Ref. [68]. For axion
energy of 1 keV and above, as was considered in Ref. [68], the $L=1$ term is
subleading and ‘orthogonality’ has no significance. In the sub-eV region,
however, it should be explicitly imposed to avoid numerical inaccuracy. The
existence of the ‘orthogonality’ condition also means that using free wave
functions for the continuum also gives very wrong results..

Figure 1: Comparison of the form factor $Q(\epsilon)$ for ionization of Xe by
massive scalars obtained by using HF continuum wave function (red line), free
continuum wave function (black line) and free continuum wave function with
orthogonality condition imposed (blue line).

The next order term with $L=1$ in Eq. (4) is proportional to $\int
f_{\epsilon_{b}}^{\kappa_{b}}f_{\epsilon_{c}}^{\kappa_{c}}rdr$ which is the
same as the radial integral appearing in the photoionization cross section
$\sigma_{\gamma}$. For ultrarelativistic scalars, this $L=1$ term dominates
over the small $L=0$ term and one has (see the appendix)

$\sigma_{\phi}(m=0)/\sigma_{\gamma}(\epsilon_{\gamma}=\epsilon)\approx
g_{\phi\bar{e}e}^{2}/(4\pi\alpha)\,.$ (7)

On the other hand, for massive scalars, the $L=1$ contribution is suppressed,
compared to its massless counterpart, by the factor $v/c\approx 10^{-3}$,

$\sigma_{\phi}^{L=1}(mc^{2}\approx\epsilon)/\sigma_{\gamma}(\epsilon_{\gamma}=\epsilon)\approx
g_{\phi\bar{e}e}^{2}v/(4\pi\alpha c)\,.$ (8)

This suppression factor makes the massive $L=1$ term somewhat smaller than the
massive $L=0$ term.

Using Eq. (7) and experimental data on photoionization cross sections [55, 56,
57], we performed a test for our numerical calculations, obtaining agreement
within a few percents accuracy, except for the near the thresholds of
ionization where the difference is about 10%.

Results - We computed the form-factor $Q(\epsilon)$ in the expression for the
ionization cross section (2) for several atoms currently used in DM search
experiments including O, Na, Ar, Ca, Ge, I, Xe, W and Tl [2, 58, 3, 59, 5, 60,
4, 61, 62, 63, 33, 64, 24, 65]. The results are presented in Fig. 2. In our
calculations, correlation corrections and field-theoretic effects beyond the
relativistic HF approximation are ignored. The accuracy of this approximation
is few percents due to the dominating contribution from the inner core states.
For these states the correlation corrections are small due to a strong nuclear
field. The initial core state is calculated in a self-consistent potential
including all electrons whereas the final electron state in the continuum is
calculated in the potential of the ionized core. We use Eq. (6) to avoid
problems with the orthogonality condition. For energies above 100 eV, there is
practically no difference between the results obtained this way and those
obtained when both initial and final states are calculated in the same
potential. For smaller energies, however, the deviations are significant and
use of more accurate potentials combined with Eq. (6) is important.

We computed the form-factor $Q$ assuming that outgoing electrons with any
nonzero energy are detectable. However, current experiments can only detect
recoil electrons with energy 1 keV or above (see, for example, Ref. [33]).
Thus, we also computed a reduced factor $\tilde{Q}$ which receives
contributions only from those subshells which give rise to outgoing electrons
with energy at least 1 keV. The result from this calculation for $\tilde{Q}$
may be directly used to interpret recent DM search results (see, for example,
Refs. [33, 66]).

Figure 2: Dimensionless form-factors in the ionization cross sections of O,
Na, Ar, Ca, Ge, I, Xe, W and Tl by scalar particles of mass $m$ and energy
$\epsilon$. Thin black line - $Q$ for $m=0$; thin dashed black line - $Q$ for
$mc^{2}=\epsilon$; thick red line - $\tilde{Q}$ for $m=0$; thick dashed blue
line - $\tilde{Q}$ for $mc^{2}=\epsilon$. The leftmost sides of the graphs
correspond with the lowest energies that can excite an electron. For all
energies smaller than these, the factors $Q$ and $\tilde{Q}$ have value zero.
The numerical data used to plot these graphs and others are presented in the
Supplemental Materials.

It is illustrative to compare the dimensionless factor $Q(\epsilon)$ for the
ionization by scalar particle with the dimensionless factor $K(\epsilon)$ of
the axioelectric effect, defined via [67, 68]

$\sigma_{a}=(\epsilon_{0}/f_{a})^{2}(c/v)K(\epsilon)a_{0}^{2}\,,$ (9)

where $\sigma_{a}$ is the axioelectric cross section and $\epsilon_{0}=27.21$
eV is the Hartree energy. As shown in Ref. [68], $K(\epsilon)$ is generally
the largest when the energy of the incoming axion is large enough to excite
the $1s$, $2s$ and $2p$ core electrons. In contrast, one observes from Fig. 2
that $Q(\epsilon)$ generally peaks for sub-keV scalar particles. This fact may
be readily verified in the case of a massless axion and a massless scalar
particle. Using Eq. (7) and the relation (see Ref. [52])

$\sigma_{a}(m_{a}=0)/\sigma_{\gamma}(\epsilon_{\gamma}=\epsilon)\approx\epsilon^{2}/(2\pi\alpha
f_{a}^{2})\,,$ (10)

(here $\epsilon$ is the axion energy ) one obtains

$Q(m=0)/K(m_{a}=0)\approx\epsilon_{0}^{2}/(2\epsilon^{2})\,,$ (11)

which shows that at high energies $Q(\epsilon)$ is suppressed in comparison
with $K(\epsilon)$. We tested the relation (11) numerically and found
agreement within a few percents accuracy. The numerical data for the form-
factor $K$ and its ‘cutoff’ version $\tilde{K}$ are also presented in the
Supplemental Materials.

One may now place limits on the electron-scalar coupling constant by assuming,
for example, that the excess events recently recorded by the Xenon1T
experiment [33], whose aim was to detect ionization by solar axion and DM
ALPs, were a result of ionization by scalars.

Considering first the case where the scalar particles saturate the local cold
DM density $\rho_{\rm DM}\sim 0.3\,{\rm GeV/cm}^{3}$. In this case, the scalar
flux is $\Phi_{\rm DM}^{\phi}=v\rho_{\rm DM}/(mc^{2})$ where $v\sim 10^{-3}c$
and the expected ionization signal peaks at the scalar energy $\epsilon\approx
mc^{2}$, with an event rate given by

$R\approx\frac{4.8}{A}\frac{\tilde{Q}(m=\frac{\epsilon}{c^{2}})}{{\rm
year}}\left(\frac{g_{\phi\bar{e}e}}{10^{-17}}\right)^{2}\left(\frac{{\rm
keV}}{mc^{2}}\right)\left(\frac{M}{{\rm ton}}\right)\,,$ (12)

where $A$ is the average atomic mass number of the detector medium and $M$ is
the medium’s total mass ($A\approx 131$ and $M=1\,{\rm ton}$ for Xenon1T).
Note that we have used the ‘cutoff’ form factor $\tilde{Q}$ to account for the
energy threshold of the detector, taken to be 1 keV.

The Xenon1T experiment reported an event rate of about
$23.5/(\mathrm{ton}\times\mathrm{year})$ at around 2 keV. From the
Supplemental Materials, we have $\tilde{Q}_{\rm Xe}(\epsilon=mc^{2}=2\,{\rm
keV})\approx 1.94\times 10^{-3}$. Substituting these values into Eq. (12), we
find that the Xenon1T result is consistent with the value

$\left|g_{\phi\bar{e}e}\right|_{\rm DM}\approx 8.2\times 10^{-15}\,,$ (13)

assuming that it is caused solely by scalar DM. Furthermore, the Xenon1T
result may be interpreted as imposing constraint on
$\left|g_{\phi\bar{e}e}\right|$ for different scalar mass. Using the
dependence on $\epsilon$ of $R$ [33] and $\tilde{Q}$ (this paper), we plot the
exclusion curve for $\left|g_{\phi\bar{e}e}\right|_{\rm DM}$ in Fig. 3.

Figure 3: Exclusion region for the $\left|g_{\phi\bar{e}e}\right|_{\rm DM}$
coupling strength as implied by Xenon1T experiment, assuming that the Xenon1T
signal was caused by ionization by scalar dark matter.

Next, we consider the case where the ionizing scalars are of solar origin,
assuming $v\approx c$. Assuming that the dominant mechanisms for producing of
solar scalars are the atomic recombination and deexcitation, Bremsstrahlung
and Compton-like (ABC) processes, one may estimate the solar scalar flux from
solar opacity, as was done for axion in Ref. [69]. Actually, we only need to
estimate the ratio of the scalar and axion matrix elements. Using Eqs. (7) and
(10), we may write the ratio between the axion and scalar emission cross
sections, and thus the corresponding fluxes, as

$\Phi_{\rm solar}^{\phi}/\Phi_{\rm
solar}^{a}=2g_{\phi\bar{e}e}^{2}m_{e}^{2}/(g_{a\bar{e}e}^{2}\epsilon^{2})\,,$
(14)

where $g_{a\bar{e}e}\equiv 2m_{e}/f_{a}$. Note that although we derived Eqs.
(7), (10) and (14) for the case of bound-free electron transitions (ionization
or recombination), they also hold for the cases of free-free (Bremsstrahlung
and Compton-like processes) and bound-bound (deexcitation) transitions.

Ref. [69] gave, for $g_{a\bar{e}e}=10^{-13}$, the value $\Phi_{\rm
solar}^{a}\approx 0.95\times 10^{20}/({\rm year\,m}^{2})$ at incoming axion
energy of $2\,{\rm keV}$222Inferred from the blue solid line in Fig. 6 in Ref.
[69].. Using this value and Eq. (14), one may write the rate of ionization by
solar scalars as

$R\approx\frac{8.3}{A}\frac{\tilde{Q}(m=0)}{{\rm
year}}\left(\frac{g_{\phi\bar{e}e}}{10^{-15}}\right)^{4}\left(\frac{{\rm
keV}}{\epsilon}\right)^{2}\left(\frac{M}{{\rm ton}}\right)\,.$ (15)

Substituting into Eq. (15) the Xenon1T event rate of
$23.5/(\mathrm{ton}\times\mathrm{year})$ and the value $\tilde{Q}_{\rm
Xe}(m=0,\epsilon=2\,{\rm keV})\approx 0.144$, one finds that the Xenon1T
result is consistent with the value

$\left|g_{\phi\bar{e}e}\right|_{\rm solar}\approx 1.0\times 10^{-14}\,.$ (16)

Using the dependence on $\epsilon$ of $R$ [33], $\tilde{Q}(m=0)$ and the solar
axion flux [69], we also derived limits on $\left|g_{\phi\bar{e}e}\right|_{\rm
solar}$ as presented in Fig. 4.

Figure 4: Exclusion region for the $\left|g_{\phi\bar{e}e}\right|_{\rm solar}$
coupling strength as implied by Xenon1T experiment, assuming that the Xenon1T
signal was caused by ionization by solar scalar.

Let us now compare the limits on $g_{\phi\bar{e}e}$ with those placed by other
DM searches and astrophysical observations. For this purpose, it is useful to
convert from $g_{\phi\bar{e}e}$ to the electron mass modulus $d_{m_{e}}$,
defined via

$g_{\phi\bar{e}e}=\sqrt{4\pi}d_{m_{e}}m_{e}/m_{P}\,,$ (17)

where $m_{P}\approx 1.22\times 10^{19}\,\mathrm{GeV}$ is the Planck mass. The
constraint on $\left|d_{m_{e}}\right|_{\rm DM}$ may be inferred from that on
$\left|g_{\phi\bar{e}e}\right|_{\rm DM}$ presented in Fig. 3. The constraint
on $\left|d_{m_{e}}\right|_{\rm solar}$ may be inferred from that on
$\left|g_{\phi\bar{e}e}\right|_{\rm solar}$ at scalar energy $2\,{\rm keV}$,
where the Xenon1T signal is the strongest, yielding

$\left|d_{m_{e}}\right|_{\rm solar}\leq 6.8\times 10^{7}\,.$ (18)

Note that Eq. (18) is independent of the scalar mass $m$, subject only to the
requirement that $mc^{2}\ll\epsilon=2\,{\rm keV}$.

In Fig. 5 we plot our constraints on $\left|d_{m_{e}}\right|$ alongside with
those imposed by other scalar DM searches and astrophysical considerations. We
see that the Xenon1T limits on $\left|d_{m_{e}}\right|_{\rm DM}$ and
$\left|d_{m_{e}}\right|_{\rm solar}$ cut deep into the natural parameter space
for a $10\,{\rm TeV}$ cutoff (the region below the green line). They are
always better than fifth-force limits, are about an order of magnitude less
stringent than the red-giant cooling limit and are comparable with or better
than horizontal-branch cooling limits.

Figure 5: Comparison of our limits on the electron mass modulus
$\left|d_{m_{e}}\right|$ of scalar DM (thick red) and solar scalar (thick
blue) implied by Xenon1T results with constraints from fifth-force searches
(thin black) [71] red-giant cooling (thick purple) and horizontal-branch (HB)
cooling (thick and thin orange) [72] and naturalness argument (green) for a
$10\,{\rm TeV}$ cutoff.

One source of the observed excess rate in the Xenon1T experiment was
attributed to the solar ABC axions [33]. Although the Xenon1T derived
constraints on the axion-electron coupling strength is a factor of 5-10 weaker
than those from astrophysical analyses, Ref. [33] argues that this tension
could be relieved by underestimated systematic uncertainties in astrophysical
analyses or estimates in solar fluxes (see, however, Ref. [34]). Since the ABC
axion solar fluxes can be directly scaled to scalar fluxes, see Eq. (14), we
can draw a similar conclusion: the excess rate in Xenon1T can be also
attributed to the solar scalars. As Fig. 5 shows this interpretation is also
in a similar tension with current astrophysical bounds. Finally, the scalar
signal may also be detected by looking for diurnal and annual modulation in
the same way as with solar axion and galactic dark matter.

Acknowledgments - We thank A. Arvanitaki for helpful discussions. This work
was supported in part by the Australian Research Council Grants No.
DP190100974 and DP200100150, the Gutenberg Fellowship and U.S. National
Science Foundation grant PHY-1912465. A.D. is grateful to the University of
New South Wales for hospitality during his visit supported in part by the
Gordon Godfrey fellowship.

Appendix - In this appendix, we derive formulae (2) and (3) for the ionization
cross section by the absorption of a scalar particle, whose wave function may
be represent as

$\phi=\phi_{0}e^{i({\bf k}\cdot{\bf r}-\omega t)}\,$ (A.1)

with the dispersion relation $\omega=c\sqrt{(mc/\hbar)^{2}+k^{2}}$ and
$\phi_{0}$ being the normalization constant. The standard prescription for
evaluating cross sections due to the Lagrangian (1) thus requires computing
matrix elements of the corresponding interaction Hamiltonian

$H_{\phi}=-\sqrt{\hbar c}g_{\phi\bar{e}e}\phi_{0}e^{i{\bf k}\cdot{\bf
r}}\gamma_{0}\,.$ (A.2)

The wave functions of the electronic bound and continuum states may be
presented in the form of Eq. (5). Since we are interested only in the total
ionization cross section without regarding to the angular distribution of the
ejected electrons, we may use, for the electronic continuum states, those with
definite energies and angular momenta. A summation over these states gives the
same result as that over those with definite linear momenta.

The total rate at which the atom absorbs a scalar particle and emits an
electron is given by Fermi’s golden rule

$\displaystyle W$
$\displaystyle=\frac{2\pi}{\hbar}\sum_{bc}\left|\bra{b}H_{\phi}\ket{c}\right|^{2}$
(A.3) $\displaystyle=2\pi cg_{\phi ee}^{2}\phi_{0}^{2}\sum_{bc}\left|\int
e^{i{\bf k}\cdot{\bf
r}}\psi_{b}^{\dagger}\gamma_{0}\psi_{c}d^{3}r\right|^{2}\,,$

where the energy conservation condition $\epsilon_{c}=\epsilon_{b}+\epsilon$
is implicit. Choosing the reference frame so that ${\bf k}=k{\bf e}_{z}$ and
using the expansion [73]

$e^{ikz}=\sum_{L=0}^{\infty}(-i)^{L}(2L+1)j_{L}(kr)C_{0}^{L}({\bf n})\,,$
(A.4)

where ${\bf n}\equiv{\bf r}/r$ and $C_{0}^{L}({\bf
n})\equiv\sqrt{4\pi/(2L+1)}Y_{0}^{L}({\bf n})$, one may put the matrix element
in Eq. (A.3) in the form

$\displaystyle\int e^{i{\bf k}\cdot{\bf
r}}\psi_{b}^{\dagger}\gamma_{0}\psi_{c}d^{3}r$
$\displaystyle=\sum_{L=0}^{\infty}(-i)^{L}(2L+1)$ (A.5)
$\displaystyle\times\int\psi_{b}^{\dagger}\upsilon_{L0}\psi_{c}d^{3}r\,,$

where $\upsilon_{L0}\equiv\gamma_{0}j_{L}(kr)C_{0}^{L}({\bf n})$. Using
Wigner-Eckart theorem, one may write the integral on the right hand side of
Eq. (A.5) as

$(-1)^{j_{b}-m_{b}}\begin{pmatrix}j_{b}&j_{c}&1\\\
-m_{b}&m_{c}&0\end{pmatrix}\langle b||\upsilon_{L}||c\rangle\,,$ (A.6)

where the reduced matrix element is defined as

$\displaystyle\langle b||\upsilon_{L}||c\rangle$
$\displaystyle\equiv\langle\kappa_{b}||C_{L}||\kappa_{c}\rangle$ (A.7)
$\displaystyle\times\int\left(f_{\epsilon_{b}}^{\kappa_{b}}f_{\epsilon_{c}}^{\kappa_{c}}-\alpha^{2}g_{\epsilon_{b}}^{\kappa_{b}}g_{\epsilon_{c}}^{\kappa_{c}}\right)j_{L}(kr)dr\,.$

Squaring Eq. (A.5) and substituting it into Eq. (A.3) and dividing by the flux
of the incoming scalar particles

$j\equiv\frac{\left|\phi^{*}\nabla\phi-(\nabla\phi^{*})\phi\right|}{\hbar}=\frac{2\epsilon}{\hbar^{2}c}\frac{v}{c}\phi^{2}_{0}\,,$
(A.8)

one obtains the result Eq. (2). Equation (4) follows from Eq. (A.7) by using
the identity [74]

$\displaystyle\langle\kappa_{b}||C_{L}||\kappa_{c}\rangle$
$\displaystyle=(-1)^{j_{b}-1/2}\sqrt{(2j_{b}+1)(2j_{c}+1)}$ (A.9)
$\displaystyle\times\begin{pmatrix}j_{b}&j_{c}&L\\\
-1/2&1/2&0\end{pmatrix}\Pi(l_{b}+L+l_{c})\,.$

It is useful to compare the ionization by scalar cross section (2) with the
photoionization cross section. Assuming that the photon and the scalar have
the same energy $\epsilon_{\gamma}=\epsilon$, we have

$\sigma_{\gamma}=\frac{4\pi^{2}\alpha\epsilon}{3}\sum_{bc}\left|\langle
b||\xi_{0}||c\rangle\right|^{2}\,,$ (A.10)

where

$\displaystyle\langle b||\xi_{0}||c\rangle$
$\displaystyle=\langle\kappa_{b}||C_{1}||\kappa_{c}\rangle$ (A.11)
$\displaystyle\times\int\left(f_{\epsilon_{b}}^{\kappa_{b}}f_{\epsilon_{c}}^{\kappa_{c}}+\alpha^{2}g_{\epsilon_{b}}^{\kappa_{b}}g_{\epsilon_{c}}^{\kappa_{c}}\right)j_{0}\left(\frac{\epsilon
r}{\hbar c}\right)rdr\,.$

As mention in the main text, the $L=0$ term in Eq. (A.7) vanishes in the non-
relativistic limit. Here, we consider only the term with $L=1$ in the
nonrelativistic electron limit. Assuming further that $\frac{\epsilon r}{\hbar
c}\ll 1$ then $j_{0}\left(\frac{\epsilon r}{\hbar c}\right)\approx 1$ and
$j_{1}(kr)\approx kr/3$ so

$\sigma_{\phi}\approx\frac{\pi c^{3}g_{\phi ee}^{2}p^{2}}{3\epsilon
v}\sum_{bc}\left|\langle\kappa_{b}||C_{1}||\kappa_{c}\rangle\int
f_{\epsilon_{b}}^{\kappa_{b}}f_{\epsilon_{c}}^{\kappa_{c}}rdr\right|^{2}\,,$
(A.12)

where $p=\hbar k$ and

$\sigma_{\gamma}\approx\frac{4\pi^{2}\alpha\epsilon}{3}\sum_{bc}\left|\langle\kappa_{b}||C_{1}||\kappa_{c}\rangle\int
f_{\epsilon_{b}}^{\kappa_{b}}f_{\epsilon_{c}}^{\kappa_{c}}rdr\right|^{2}\,.$
(A.13)

Taking the ratio of Eqs. (A.12) and (A.13), one obtains the result (8) of the
main text.

## References

  * Angloher  _et al._ [2014] G. Angloher, A. Bento, C. Bucci, L. Canonica, A. Erb, F. von Feilitzsch, N. F. Iachellini, P. Gorla, A. Gütlein, D. Hauff, _et al._ (CRESST collaboration), Eur. Phys. J. C 74, 1 (2014).
  * Aalseth  _et al._ [2013] C. E. Aalseth, P. S. Barbeau, J. Colaresi, J. I. Collar, J. Diaz Leon, J. E. Fast, N. E. Fields, T. W. Hossbach, A. Knecht, M. S. Kos, M. G. Marino, H. S. Miley, M. L. Miller, J. L. Orrell, and K. M. Yocum (CoGeNT Collaboration), Phys. Rev. D 88, 012002 (2013).
  * Armengaud  _et al._ [2016] E. Armengaud _et al._ (EDELWEISS Collaboration), J. Cosmo. Astropart. Phys. 2016, 019 (2016).
  * Bernabei  _et al._ [2018] R. Bernabei, P. Belli, A. Bussolotti, F. Cappella, V. Caracciolo, R. Cerulli, C. J. Dai, A. D’Angelo, A. Di Marco, H. L. He, A. Incicchitti, X. H. Ma, A. Mattei, V. Merlo, F. Montecchia, X. D. Sheng, and Z. P. Ye, Universe 4, 116 (2018).
  * Agnese  _et al._ [2018] R. Agnese _et al._ (SuperCDMS Collaboration), Phys. Rev. Lett. 121, 051301 (2018).
  * Aprile  _et al._ [2018] E. Aprile _et al._ (XENON Collaboration), Phys. Rev. Lett. 121, 111302 (2018).
  * Akerib  _et al._ [2019] D. S. Akerib _et al._ (LUX Collaboration), Phys. Rev. Lett. 122, 131301 (2019).
  * Wang  _et al._ [2020] Q. Wang _et al._ (PandaX-II Collaboration), Chin. Phys. C 44, 125001 (2020).
  * Grossman and Haber [1997] Y. Grossman and H. E. Haber, Phys. Rev. Lett. 78, 3438 (1997).
  * Hall  _et al._ [1998] L. J. Hall, T. Moroi, and H. Murayama, Phys. Lett. B 424, 305 (1998).
  * Bottino  _et al._ [2003] A. Bottino, N. Fornengo, and S. Scopel, Phys. Rev. D 67, 063519 (2003).
  * Bottino  _et al._ [2004] A. Bottino, F. Donato, N. Fornengo, and S. Scopel, Phys. Rev. D 69, 037302 (2004).
  * Arina and Fornengo [2007] C. Arina and N. Fornengo, J. High Ener. Phys. 2007, 029 (2007).
  * Cerdeño  _et al._ [2009] D. G. Cerdeño, C. Muñoz, and O. Seto, Phys. Rev. D 79, 023510 (2009).
  * Kakizaki  _et al._ [2015] M. Kakizaki, E. K. Park, J. H. Park, and A. Santa, Phys. Lett. B 749, 44 (2015).
  * Ghosh  _et al._ [2018] P. Ghosh, I. Lara, D. E. López-Fogliani, C. Muñoz, and R. Ruiz de Austri, Int. J. Mod. Phys. A 33, 1850110 (2018).
  * Preskill  _et al._ [1983] J. Preskill, M. B. Wise, and F. Wilczek, Phys. Lett. B 120, 127 (1983).
  * Abbott and Sikivie [1983] L. Abbott and P. Sikivie, Phys. Lett. B 120, 133 (1983).
  * Dine and Fischler [1983] M. Dine and W. Fischler, Phys. Lett. B 120, 137 (1983).
  * Mizumoto  _et al._ [2013] T. Mizumoto _et al._ , J. Cosmo. Astropart. Phys. 2013, 013 (2013).
  * Anastassopoulos  _et al._ [2017] V. Anastassopoulos _et al._ (CAST collaboration), Nat. Phys. 13, 584 EP (2017).
  * Gattone  _et al._ [1999] A. Gattone _et al._ , Nucl. Phys. B - Proc. Suppl. 70, 59 (1999).
  * Morales  _et al._ [2002] A. Morales _et al._ , Astropart. Phys. 16, 325 (2002).
  * Agnese  _et al._ [2014] R. Agnese _et al._ (SuperCDMS Collaboration), Phys. Rev. Lett. 112, 241302 (2014).
  * Hoskins  _et al._ [2011] J. Hoskins, J. Hwang, C. Martin, P. Sikivie, N. S. Sullivan, D. B. Tanner, M. Hotz, L. J. Rosenberg, G. Rybka, A. Wagner, S. J. Asztalos, G. Carosi, C. Hagmann, D. Kinion, K. van Bibber, R. Bradley, and J. Clarke, Phys. Rev. D 84, 121302(R) (2011).
  * Brubaker  _et al._ [2017] B. M. Brubaker, L. Zhong, Y. V. Gurevich, S. B. Cahn, S. K. Lamoreaux, M. Simanovskaia, _et al._ , Phys. Rev. Lett. 118, 061302 (2017).
  * McAllister  _et al._ [2017] B. T. McAllister _et al._ , Phys. Dark Univ. 18, 67 (2017).
  * Ehret  _et al._ [2010] K. Ehret _et al._ , Phys. Lett. B 689, 149 (2010).
  * Pugnat  _et al._ [2008] P. Pugnat _et al._ (OSQAR Collaboration), Phys. Rev. D 78, 092003 (2008).
  * Chou  _et al._ [2008] A. S. Chou _et al._ , Phys. Rev. Lett. 100, 080402 (2008).
  * Della Valle  _et al._ [2016] F. Della Valle _et al._ , Eur. Phys. J. C 76, 24 (2016).
  * Chen  _et al._ [2007] S. J. Chen _et al._ , Mod. Phys. Lett. A 22, 2815 (2007).
  * Aprile  _et al._ [2020] E. Aprile _et al._ (XENON Collaboration), Phys. Rev. D 102, 072004 (2020).
  * Di Luzio  _et al._ [2020] L. Di Luzio, M. Fedele, M. Giannotti, F. Mescia, and E. Nardi, Phys. Rev. Lett. 125, 131804 (2020), 2006.12487 .
  * Bernabei  _et al._ [2006] R. Bernabei, P. Belli, F. Montecchia, F. Nozzoli, F. Cappella, A. Incicchitti, D. Prosperi, R. Cerulli, C. J. Dai, H. L. He, H. H. Kuang, J. M. Ma, and Z. P. Ye, Int. J. Mod. Phys. A 21, 1445 (2006).
  * Aad  _et al._ [2012] G. Aad _et al._ , Phys. Lett. B 716, 1 (2012).
  * Arvanitaki  _et al._ [2015] A. Arvanitaki, J. Huang, and K. Van Tilburg, Phys. Rev. D 91, 015015 (2015).
  * Van Tilburg  _et al._ [2015] K. Van Tilburg, N. Leefer, L. Bougas, and D. Budker, Phys. Rev. Lett. 115, 011802 (2015).
  * Stadnik and Flambaum [2015a] Y. V. Stadnik and V. V. Flambaum, Phys. Rev. Lett. 115, 201301 (2015a).
  * Stadnik and Flambaum [2016a] Y. V. Stadnik and V. V. Flambaum, Phys. Rev. A 94, 022111 (2016a).
  * Hees  _et al._ [2016] A. Hees, J. Guéna, M. Abgrall, S. Bize, and P. Wolf, Phys. Rev. Lett. 117, 061301 (2016).
  * Kennedy  _et al._ [2020] C. J. Kennedy, E. Oelker, J. M. Robinson, T. Bothwell, D. Kedar, W. R. Milner, G. E. Marti, A. Derevianko, and J. Ye, Phys. Rev. Lett. 125, 201302 (2020).
  * Arvanitaki  _et al._ [2016] A. Arvanitaki, S. Dimopoulos, and K. Van Tilburg, Phys. Rev. Lett. 116, 031102 (2016).
  * Graham  _et al._ [2016] P. W. Graham, D. E. Kaplan, J. Mardon, S. Rajendran, and W. A. Terrano, Phys. Rev. D 93, 075029 (2016).
  * Arvanitaki  _et al._ [2018a] A. Arvanitaki, P. W. Graham, J. M. Hogan, S. Rajendran, and K. Van Tilburg, Phys. Rev. D 97, 075020 (2018a).
  * Stadnik and Flambaum [2015b] Y. V. Stadnik and V. V. Flambaum, Phys. Rev. Lett. 114, 161301 (2015b).
  * Stadnik and Flambaum [2016b] Y. V. Stadnik and V. V. Flambaum, Phys. Rev. A 93, 063630 (2016b).
  * Geraci  _et al._ [2019] A. A. Geraci, C. Bradley, D. Gao, J. Weinstein, and A. Derevianko, Phys. Rev. Lett. 123, 031304 (2019).
  * Geraci and Derevianko [2016] A. A. Geraci and A. Derevianko, Phys. Rev. Lett. 117, 261301 (2016).
  * Stadnik and Flambaum [2014] Y. V. Stadnik and V. V. Flambaum, Phys. Rev. Lett. 113, 151301 (2014).
  * Arvanitaki  _et al._ [2018b] A. Arvanitaki, S. Dimopoulos, and K. Van Tilburg, Phys. Rev. X 8, 041001 (2018b), 1709.05354 .
  * Pospelov  _et al._ [2008] M. Pospelov, A. Ritz, and M. Voloshin, Phys. Rev. D 78, 115012 (2008).
  * Derevianko  _et al._ [2000] A. Derevianko, O. Hemmers, S. Oblad, P. Glans, H. Wang, S. B. Whitfield, R. Wehlitz, I. A. Sellin, W. R. Johnson, and D. W. Lindle, Phys. Rev. Lett. 84, 2116 (2000).
  * Note [1] We point out that a similar ‘orthogonality’ condition also exists for the axioelectric effect, affecting the $L=1$ multipole, as may be seen in Eq. (A18) in Ref. [68]. For axion energy of 1 keV and above, as was considered in Ref. [68], the $L=1$ term is subleading and ‘orthogonality’ has no significance. In the sub-eV region, however, it should be explicitly imposed to avoid numerical inaccuracy. The existence of the ‘orthogonality’ condition also means that using free wave functions for the continuum also gives very wrong results.
  * Wuilleumier [1972] F. Wuilleumier, Phys. Rev. A 6, 2067 (1972).
  * Veigele [1973] W. Veigele, At. Data Nucl. Data Tables 5, 51 (1973).
  * Marr and West [1976] G. Marr and J. West, At. Data Nucl. Data Tables 18, 497 (1976).
  * Bernabei  _et al._ [2014] R. Bernabei, P. Belli, F. Cappella, V. Caracciolo, S. Castellano, R. Cerulli, C. J. Dai, A. d’Angelo, S. d’Angelo, A. Di Marco, _et al._ , Adv. High Energy Phys. 2014, 605659 (2014).
  * Calvo  _et al._ [2017] J. Calvo, C. Cantini, P. Crivelli, M. Daniel, S. D. Luise, A. Gendotti, S. Horikawa, B. Montes, W. Mu, S. Murphy, G. Natterer, K. Nguyen, L. Periale, Y. Quan, B. Radics, C. Regenfus, L. Romero, A. Rubbia, R. Santorelli, F. Sergiampietri, T. Viant, and S. Wu, J. Cosmo. Astropart. Phys. 2017, 003 (2017).
  * Armengaud  _et al._ [2013] E. Armengaud _et al._ , J. Cosmo. and Astropart. Phys. 2013, 067 (2013).
  * Aalseth  _et al._ [2018] C. E. Aalseth _et al._ , Eur. Phys. J. Plus 133 (2018).
  * Antonello  _et al._ [2019] M. Antonello _et al._ , Astropart. Phys. 106, 1 (2019).
  * Adhikari  _et al._ [2020] P. Adhikari _et al._ (DEAP Collaboration), Phys. Rev. D 102, 082001 (2020).
  * Amaral  _et al._ [2020] D. W. Amaral _et al._ (SuperCDMS Collaboration), Phys. Rev. D 102, 091101 (2020).
  * Abdelhameed  _et al._ [2019] A. H. Abdelhameed _et al._ (CRESST Collaboration), Phys. Rev. D 100, 102002 (2019).
  * Adhikari  _et al._ [2021] G. Adhikari _et al._ , (2021), arXiv:2104.03537 [hep-ex] .
  * Dzuba  _et al._ [2010] V. A. Dzuba, V. V. Flambaum, and M. Pospelov, Phys. Rev. D 81, 103520 (2010).
  * Derevianko  _et al._ [2010] A. Derevianko, V. A. Dzuba, V. V. Flambaum, and M. Pospelov, Phys. Rev. D 82, 065006 (2010).
  * Redondo [2013] J. Redondo, J. Cosmo. Astropart. Phys. 2013, 008 (2013).
  * Note [2] Inferred from the blue solid line in Fig. 6 in Ref. [69].
  * Adelberger  _et al._ [2003] E. Adelberger, B. Heckel, and A. Nelson, Annu. Rev. Nucl. Part. Sci. 53, 77 (2003).
  * Hardy and Lasenby [2017] E. Hardy and R. Lasenby, J. High Ener. Phys. 2017, 033 (2017).
  * Rose and Holladay [1961] M. E. Rose and W. Holladay,  _Relativistic electron theory_ (John Wiley & Sons, Inc, 1961).
  * Edmonds [1996] A. R. Edmonds,  _Angular momentum in quantum mechanics_ (Princeton University Press, 1996).


# DP-CGAN : Differentially Private Synthetic Data and Label Generation

Reihaneh Torkzadehmahani
University of California, Santa Cruz
<EMAIL_ADDRESS>Peter Kairouz
Google AI
<EMAIL_ADDRESS>Benedict Paten
University of California, Santa Cruz
<EMAIL_ADDRESS>
###### Abstract

Generative Adversarial Networks (GANs) are one of the well-known models to
generate synthetic data including images, especially for research communities
that cannot use original sensitive datasets because they are not publicly
accessible. One of the main challenges in this area is to preserve the privacy
of individuals who participate in the training of the GAN models. To address
this challenge, we introduce a Differentially Private Conditional GAN (DP-
CGAN) training framework based on a new clipping and perturbation strategy,
which improves the performance of the model while preserving privacy of the
training dataset. DP-CGAN generates both synthetic data and corresponding
labels and leverages the recently introduced Rényi differential privacy
accountant to track the spent privacy budget. The experimental results show
that DP-CGAN can generate visually and empirically promising results on the
MNIST dataset with a single-digit epsilon parameter in differential privacy.

##  1 Introduction

Recent studies have shown that deep neural networks (DNNs) can achieve state-
of-the-art performance in various applications such as image recognition[1,
2], natural language processing[3], speech recognition[4, 5] and complex video
games[6, 7]. It has not only achieved exceptional accuracy in different tasks
but also surpassed human-level performance in some of them[6, 8]. DNNs have
also been leveraged in health-related studies ranging from medical images[9,
10, 11, 12, 13] to human genome analyses[14, 15, 16].

Generative Adversarial Networks (GANs)[17] form a well-researched class of
generative models[18, 19, 20, 21]. They can learn the distribution of the
training data and generate synthetic data with a distribution very similar to
the distribution of the training data. GAN models are particularly used by
research communities to generate the synthetic datasets in cases where they
cannot directly access sensitive datasets. However, using sensitive data to
train GAN models raises privacy concerns for participating individuals.
Indeed, recent works show that most machine learning models, including GAN
models, are vulnerable to a slew of attacks (from model inversion attacks to
membership inference attacks) that can expose significant information about
training data[22, 23, 24, 25].

Differential Privacy (DP) [26, 27] is a common technique to protect the
privacy of ML models trained on sensitive data. However, in spite of its
popularity, there have been very few recent studies on training GANs in a
differentially private way[28, 29, 30, 31, 32]. The standard procedure
leveraged by these recent studies to enforce DP is to first clip the l2 norm
of the gradients of the sum of the discriminator’s loss on real and fake data
and then add Gaussian noise to the clipped gradients. To keep track of the
privacy budget, they typically use the Moment Accountant (MA) technique[33].
One of the limitations of these recent works is that they focus exclusively on
generating synthetic data (e.g., images) without corresponding labels – an
aspect that renders the synthetically generated data useless for supervised
learning applications. More importantly, training high quality GANs with a
single digit epsilon parameter (for differential privacy) has been absent so
far even for the simplest of all tasks: generating MNIST-like digits.

In this work, we propose a Differentially Private Conditional GAN (DP-CGAN)
training framework, which can preserve the privacy of conditional GAN models
using DP[26, 27]. The main idea in DP-CGAN is that it clips the gradients of
discriminator loss on real and fake data separately, which allows the designer
to better control the sensitivity of the model to real (sensitive) data.
Moreover, DP-CGAN can generate not only synthetic data but also corresponding
labels. Further, DP-CGAN employs the newly introduced Rényi Differential
Privacy (RDP) Accountant[34] to track the privacy budget. In comparison to the
classical MA technique, RDP accounting provides a tighter bound on the privacy
budget, allowing for the addition of less noise without compromising the
privacy guarantees.

DP-CGAN framework has three main components: conditional generator network,
differentially private discriminator network, and privacy accountant. At each
step of the training process, the discriminator network is trained in a
differentially private manner in which the gradients of loss on real and fake
data are clipped separately. Afterwards, the sum of these two set of clipped
gradients are computed and noised by adding Gaussian noise to them. Then, the
privacy accountant, which is based on the RDP accountant[34], is updated by
accumulating the spent privacy budget at each step. Next, the generator
network is trained with a non-private optimizer. At any given point in time,
if the privacy budget exceeds the target one, the training process is halted
and the conditional generator network is ready for the creation of synthetic
data and labels.

We make the following contributions in this work:

  * •

We propose DP-CGAN based on a new gradient clipping and noising procedure,
which improves the performance compared to the standard procedure to preserve
privacy. To the best of our knowledge, DP-CGAN is the first differentially
private GAN framework than can generate both the synthetic data and
corresponding labels with promising results. It leverages the recently
introduced RDP accountant and TensorFlow
Privacy111https://github.com/tensorflow/privacy package (by Google) to keep
track of the privacy budget.

  * •

We provide preliminary experimental results showing that DP-CGAN can generate
good visual and empirical results on MNIST dataset with single-digit epsilon
parameter. This suggests that our work can be viewed as the first stepping
stone towards training high quality GANs with strong DP guarantees.

  * •

We use the differnetially private conditional generative model to create
synthetic data and labels which are used (together) in the training of machine
learning models. We test the accuracy of the learned models on real data and
show that they perform well. We get an area under the ROC (AUROC) of $87.57\%$
using DP-CGANs compared to $92.17\%$ if we were to train the classifier
directly on real data.

The remainder of the paper is organized as follows: Section 2 provides a
background on GAN, CGAN, and differential privacy. Section 3 overviews the
previous related work in the area of preserving the privacy of deep learning
models. Section 4 describes the DP-CGAN framework in detail. Section 5
provides the experimental results and Section 6 concludes the paper with a
brief conclusion.

##  2 Preliminaries

In this section, we review Generative Adversarial Networks(GAN), Conditional
Generative Adversarial Networks(CGAN) and differential privacy concepts used
in DP-CGAN.

###  2.1 GAN and CGAN

Nowadays, there is a great interest in using generative models to create
synthetic data that looks like the original one. Generative Adversarial
Network(GAN) proposed by Goodfellow et. al [17] is one the primary methods to
learn generative models for images. GANs consist of two main components: a
generator and a discriminator. The generator takes noise as input and
generates synthetic data by capturing the original data distribution while the
discriminator takes the synthetic data (generator’s output) as well as
original data (training set) and learns to discriminate between the real
(training) and fake (synthetic) data distribution. The discriminator returns
two possible values as output which is the assigned score to a test sample
representing whether it is real or fake data. The generator and discriminator
always try hard to be as accurate as possible and the more the generator
improves the quality of the fake data, it gets harder for discriminator to
distinguish the difference between the original and fake data. These two
components always play a game and are trained simultaneously.

Suppose $p_{z}(z)$ is the probability distribution that random noise $z$ is
taken from, $G(z)$ is the generator network that takes the random noise $z$ as
input and $D(x)$ is the discriminator network that takes the generator’s
output as well as the input data $x$ taken form the distribution
$p_{data}(x)$. The game that the generator and discriminator play to achieve a
trade-off, encapsulates in the following objective function, $V(D,G)$, of a
minimax game:

$\begin{split}\min_{G}\max_{D}V(D,G)=\\\
\mathop{\displaystyle\mathbb{E}}_{x\sim p_{data}(x)}[log(D(x))]+\\\
\mathop{\mathbb{E}}_{z\sim p_{z}(z)}[log(1-D(G(z)))]\end{split}$ (1)

Conditional GAN[35] is an extension of GAN in which both generator and
discriminator are conditioned on some side information, “y” that can be any
kind of extra information like class labels or data from other modalities. The
objective function of a minimax game for CGAN is as the following:

$\begin{split}\min_{G}\max_{D}V(D,G)=\\\
\mathop{\displaystyle\mathbb{E}}_{x\sim p_{data}(x)}[log(D(x|y))]+\\\
\mathop{\mathbb{E}}_{z\sim p_{z}(z)}[log(1-D(G(z|y))))]\end{split}$ (2)

###  2.2 Differential Privacy

Differential privacy[26, 27] is a mathematical framework to express the level
of privacy preservation of individuals in a statistical databases. It provides
strong privacy guarantees for algorithms on aggregate databases. Intuitively,
in differential privacy, the user should learn about population as a whole but
not about particular individual. In other words, if we replace individual $I$
with another random member of the population, the user should learn the same
thing about the dataset in presence or absence of individual $I$. Differential
privacy has become an actual standard in data protection in both academia and
industry[36] (Apple[37], Google[38] and US Census[39]).

Definition 1. (differential privacy) A randomized mechanism $M$ over a set of
databases $D$, satisfies $(\epsilon,\delta)$-differential privacy if for any
two adjacent databases $d,d^{{}^{\prime}}\in D$, with only one different
sample, and for any subset of output $S\in R$, the following inequality holds:

$Pr[M(d)\in S]\leq e^{\epsilon}Pr[M(d^{\prime})\in S]+\delta$ (3)

In pure differential privacy, $\delta=0$ and the additive term $\delta$ does
not exist while in approximate differential privacy [26], $\delta$ is used for
approximation in the cases that pure differential privacy is broken. $\delta$
is the probability that privacy loss is not bounded by $\epsilon$ and its
optimal value is smaller than $\frac{1}{|d|}$ (inverse of the database size).

Differential privacy is resistant to post-processing. That is, any arbitrary
randomized mapping of an $(\epsilon,\delta)$-differentially private algorithm,
is differentially private as well.

Theorem 1. (post-processing) Given a randomized algorithm $M:D\xrightarrow{}R$
that is $(\epsilon,\delta)$-differentially private and an arbitrary randomized
mapping $f:R\xrightarrow{}R^{{}^{\prime}}$, $f\circ
M:D\xrightarrow{}R^{{}^{\prime}}$ is $(\epsilon,\delta)$-differentially
private.

A routine approach to privatizing the output of a real-valued function
$f:D\xrightarrow{}\mathbb{R}$ is to add noise with variance in the scale of
$f$’s sensitivity, $S_{f}$, to the output. The sensitivity of a function $f$
is defined as the maximum absolute distance $|f(d)-f(d^{{}^{\prime}})|$ ($d$
and $d^{{}^{\prime}}$ are adjacent databases). In formal notion:

$S_{f}\equiv\max_{d\sim d^{{}^{\prime}}}{|f(d)-f(d^{{}^{\prime}})|},$ (4)

Gaussian noise is one of the popular kinds of noise employed in differential
privacy, in which $f(d)$ is perturbed by Gaussian noise
$N(0,{S_{f}}^{2}.\sigma^{2})$. That is:

$M(d)\equiv f(d)+N(0,{S_{f}}^{2}.\sigma^{2})$ (5)

Composability is one of the interesting properties of differetnial privacy
that makes it possible to combine multiple differentially private mechanisms
into one. A standard analysis implies the composition of $k$ mechanisms that
each of them are $(\epsilon,\delta)$-differentially private, is at least
$(k\epsilon,k\delta)$-differentially private[26, 27, 40]. One of the possible
ways of accounting differential privacy in composition of additive-noise
mechanisms is to use Moment Account technique introduced by Abadi et. al[33],
which provides strong estimates of privacy loss compared to various versions
of composition theorem [26, 40, 41, 42, 43] including strong composition
theorem [43]. RDP accountant [34] is a new approach based on a new definition
of privacy, Rényi differential privacy, which provides a tighter bound for
privacy loss in comparison with Moment Accountant.

##  3 Related Work

Some previous studies have proposed approaches to addressing the problem of
preserving privacy in Deep Learning. Shokri et al.[22] developed a distributed
approach in which multiple parties train a model on their local training set
independently. Then, each party selects a set of key parameters, and shares
them with the other parties. Although this method has high training accuracy
without sharing the input parameters, Abadi et al.[33] showed that the overall
privacy loss for each party exceeds several thousands on MNIST dataset using
Moment Accountant technique they introduced.

Moment accountant mechanism[33] can be used to track the overall spent privacy
budget, $(\epsilon,\delta)$, for composing Gaussian Mechanisms with random
sampling (e.g. training process in Stochastic Gradient Descent). This method
provides a much tighter estimation for privacy loss compared to standard
composition theorem[44]. It computes the log moments of the random variable
indicating privacy loss and then calculates the tail bound using moments bound
and standard Markov inequality. The result is privacy loss estimation in terms
of differential privacy. In addition to Moment Accountant technique, Abadi et
al.[33] proposed a method to make the Stochastic Gradient Descent(SGD) process
differentially private.

Private Aggregation of Teacher Ensembles (PATE)[45] is a framework that
leverages the moment accountant mechanism to trace the privacy leakage of
knowledge transfer task using differential privacy. It presents a
differentially private semi-supervised learning method in which the training
data is split into multiple disjoint sets and the teacher models are trained
independently. The teacher ensemble predicts the labels after perturbing
counts of teachers’ votes by Laplace noise while the student model is trained
on public data as well as labeled data from the teacher model and can be
published publicly. Although this method outperforms Shokri et al.[22] work in
terms of both accuracy and privacy, it assumes the model has access to public
data which may not be the case in practice. Moreover, the teacher ensemble
just responds to the queries for which the consensus among teachers is
sufficiently high.

Some other previous researches focused on preserving privacy of GANs in
particular. DPGAN method[28], enforces differential privacy during the
training process of the discriminator by adding Gaussian noise to the gradient
of Wasserstein distance in WGAN algorithm and uses post-processing theorem to
guarantee differential privacy for the generator. However, it is unclear how
the overall privacy budget is accounted, the results do not look promising
even on MNIST dataset and there is no methodology for creating labels for
synthetic images.

Similar to DPGAN method, PATE-GAN approach[30] enforces privacy by making the
discriminator differentially private. In PATE-GAN, the discriminator is
replaced with modified version of PATE[45] in which the student model allows
back-propagation to the generator and there is no need to have access to
public training data. It employs the generated data to train different
classifiers and evaluate the quality of generated data by testing these
classifiers on real test data. The limitation of PATE-GAN is that it assigns
binary labels for synthetic data, and therefore, it is not applicable for
multi-label datasets. Moreover, the datasets used to evaluate the model are
small. The other work is a DPGAN framework for time series, continuous, and
discrete data[29]. This framework is alike the previous DPGAN work[28] except
it employs moments accountant approach to account the privacy budget and clips
the discriminator gradients while reducing the clipping parameter over time
(adaptive clipping).

Unlike DPGAN method[28], our proposed method leverages RDP accountant
technique to follow the consumed privacy budget, $(\epsilon,\delta)$ and
generates not only synthetic data but also the labels using a Conditional GAN
model. In contrast to PATE-GAN[30] which generates only binary labels, our
model generates multi-class labels. Finally, in DPGAN frameworks[28, 29] the
discriminator gradients are clipped and perturbed by adding Gaussian noise to
gradients of the discriminator loss, while in our framework, Gaussian noise is
added to the accumulation of clipped gradients of discriminator loss on real
data and clipped gradients of discriminator loss on fake data.

##  4 Our Approach

As mentioned before, DP-CGAN can generate the synthetic data as well as the
corresponding labels while preserve privacy of training samples. To this end,
the DP-CGAN makes the training process private by injecting random Gaussian
noise into the optimization process of the discriminator network. Based on
post-processing theorem[46] making the generative network differentially
private results in having a differentially private generator too. DP-CGAN
tracks the spent privacy loss using RDP accounting technique[34], which
provides tighter estimation on privacy loss in comparison with moment
accountant technique. The training procedure stops if the spent privacy budget
$(\epsilon,\delta)$ goes beyond the target ones.

1Input: Examples $\\{x_{1},x_{2},...,x_{N}\\}$, labels
$\\{y_{1},y_{2},...,y_{N}\\}$, target epsilon $\epsilon$, target delta
$\delta$, noise scale $\sigma$, clip norm bound $C$, learning rate $lr$, batch
size $bs$

2Output: Differentially private Generator that generates synthetic data and
labels

3should$\\_$terminate = False

4while  _$(step\leq max\\_step\And!\ should\\_terminate)$_ do

5 \- Sample random batch $(X^{t},Y^{t})$ of size $bs$ with probability $bs/N$
from data distribution $p_{data}{(X)}$

6 \- Sample noise batch $Z_{t}$ of size $bs$ from noise prior $p_{z}{(z)}$

/* Update the Discriminator Network */

7

8 $d\\_loss\\_real\xleftarrow[]{}\log(D(X^{t}))$

9 $d\\_loss\\_fake\xleftarrow[]{}\log(1-D(G(Z^{t})))$

10 Compute per-example gradients of discriminator loss on real data $X_{t}$
and clip them

11 for  _$i\in X_{t}$_ do

12 Compute
${grad_{d\\_real}}^{t}\xleftarrow[]{}\nabla_{\theta_{d}}d\\_{loss}\\_real({{\theta_{d}}^{t}},X_{i})$

13

14
${grad_{d\\_real}}^{t}={grad_{d\\_real}}^{t}/max(1,\frac{||grad_{d\\_real}||_{2}}{C})$
Compute per-example gradients of discriminator loss on fake data $Z_{t}$ and
clip them

15 for  _$i\in Z_{t}$_ do

16 Compute
${grad_{d\\_fake}}^{t}\xleftarrow[]{}\nabla_{\theta_{d}}d\\_{loss}\\_fake({{\theta_{d}}^{t}},Z_{i})$

17

18
${grad_{d\\_fake}}^{t}={grad_{d\\_fake}}^{t}/max(1,\frac{||grad_{d\\_fake}||_{2}}{C})$

19 Compute the overall gradients of discriminator and add Gaussian Noise to
them

20
${grad_{{d}}}^{t}\xleftarrow[]{}{\frac{1}{bs}{\sum{grad_{{d\\_real}}}^{t}+{grad_{{d\\_fake}}}^{t}+N(0,\sigma^{2}C^{2}I)}}$

21 Take the gradient Descent step for discriminator

22
${{\theta_{d}}_{t+1}}\xleftarrow[]{}SGD(grads\\_d^{t},{{\theta_{d}}_{t}},lr))$

/* Update RDP Accountant */

23

24 Accumulate the spent privacy budget using RDP Accountant

/* Update the Generator Network */

25

26 $g\\_loss\xleftarrow[]{}log(1-D(G(Z^{t})))$

27 Compute gradients of generator loss

28 Compute
${grad\\_g}^{t}\xleftarrow[]{}\nabla_{\theta_{g}}g\\_loss({{\theta_{g}}^{t}},Z_{i})$

29 Take the gradient Descent step for generator

30 ${{\theta_{g}}^{t+1}}\xleftarrow[]{}ADAM({grad\\_g}^{t},{\theta_{g}}^{t})$

31 if  _ $spent\\_epsilon>\epsilon$ OR $spent\\_delta>\delta$_ then

32

/* Running out of privacy budget */

33

34 should$\\_$terminate = True

35

Algorithm 1 DP-CGAN

DP-CGAN makes the optimization process of discriminator loss (discriminator
training) differentially private by computing the per-example gradients of the
discriminator loss on both real and fake data, clipping the per-example
gradients on real data and fake data separately, summing up two sets of the
clipped gradients, perturbing the clipped gradients by adding Gaussian noise
$N(0,\sigma^{2}C^{2})$, $\sigma$ is noise multiplier and $C$ is clipping
value, to them, and finally applying the perturbed gradients.

Algorithm 1 outlines the training process of DP-CGAN. According to the
algorithm, the model updates the discriminator network and the generator
network as long as the number of iterations is less than maximum iteration
count and the spent privacy budget is less than the target $\epsilon$. At each
step, it minimizes the discriminator loss function by computing the
discriminator gradients of loss on real data and clipping them by $L_{2}$-norm
(lines 9-12), computing the discriminator gradients of loss on fake data(lines
13-15) and clipping them by $L_{2}$-norm, compute the overall clipped
gradients of discriminator by adding these two sets of clipped gradients,
adding Gaussian noise to them and taking average over all the perturbed
clipped per-example gradients in the batch(line 16-17), and finally applying
the gradients (line 18). The model tracks the spent privacy budget by
accumulating the spent privacy budget and updating the RDP accountant every
time that noise is injected into the model(line 20). Then, the generator the
gradients of generator loss are computed and applied so that the generator
network gets trained(line 21-25). The last step is to check the overall spent
privacy budget so far. If the spent $\epsilon$ or the spent $\delta$ has
exceeded the target values, training is stopped, otherwise it continues (line
26-27).

##  5 Experimental Results

We compare the performance of DP-CGAN to CGAN with no privacy and CGAN trained
with standard differentially private approach.The CGAN architecture used in
all models is a vanilla CGAN in which both generator and discriminator consist
of two fully connected layers.The generator takes random noise sample $z$ and
the corresponding label $y$ as inputs while the discriminator inputs are real
training sample $x$ and its label $y$. Figure 1 depicts the generator and
discriminator architecture of the vanilla CGAN.

Figure 1: Vanilla CGAN Generator and Discriminator Architecture

Differentially private CGAN models use the new privacy package of TensorFlow
Privacy (by Google), a python library that includes the implementation of few
differentially private optimizers as well as the privacy accountants to keep
track of the privacy loss. They leverage differentially private Gradient
Descent as optimizer and RDP accountant as privacy accountant from this
package.

The dataset used used in the evaluation is MNIST handwritten dataset
containing 60k training samples and 10k test samples. In the experiments,
batch size is set to 600, $\delta=10^{-5}$ and learning rate is set by an
adapative approach in which the initial learning rate is 0.15, it is decreased
to 0.052 in iteration 10K and is fixed on 0.052 for the rest iterations.

We trained Logistic Regression and Multi-Layer Perceptron classifiers using
the synthetic data and labels generated by the models and tested the
classifier on real test data. Closer performance of the classifier trained on
synthetic data generated by differntially private models and on real data
indicates that the model has captured the real data distribution better. We
measured the performance of the classifier using the Area under ROC curve
metric (AuROC). In the evaluation process, the generative model takes the 60k
MNIST training data and the labels as input and generates 60k synthetic
labeled data.Then, the classifier is trained on the generated data. Finally,
performance of the trained classifier is evaluated on the 10k test data using
AuROC metric.

Table 1. lists the results of AuROC for the three models as well as the case
in which classifiers are trained on real data. According to the table, the
AuROC of DP-CGAN is higher than CGAN trained with basic differentially private
method, indicating that new clipping and perturbing technique used in DP-CGAN
improves the performance. On the other hand, the AuROC of DP-CGAN is about
$5\%$ lower than that for real data and this is the price we pay to have
privacy.

| Real | CGAN | DP-CGAN |  CGAN with
---|---|---|---|---
|  |  |  | basic DP
LR | $92.17\%$ | $91.10\%$ | $\bm{87.57\%}$ | $83.42\%$
MLP | $97.60\%$ | $91.06\%$ | $\bm{88.16\%}$ | $83.29\%$
Table 1: Comparing AuROC for Logistic Regression(LR) and Multi-Layer
Perceptron(MLP), which are trained on real data, data generated by CGAN (non-
private), DP-CGAN and CGAN with basic differentially private approach using
$\epsilon=9.6$ , and $\delta={10^{-5}}$

We also visualized the images generated by the models (Figure 2) . In the
figure, the most left column shows the results for DP-CGAN, the left column
represents the results for CGAN with no privacy, and the right column depicts
the synthetic images generated by CGAN with basic differentially private
approach. According to the figure, the quality of the images generated by DP-
CGAN is better than CGAN with basic differentially private approach but worse
than CGAN with no privacy.

(a)

(b)

(c)

Figure 2: (a) DP-CGAN, (b) CGAN with no privacy, (c) CGAN with basic
differentially private approach

##  6 Conclusion

In this research, we proposed DP-CGAN framework that is a differentially
private GAN model capable of generating both synthetic data and corresponding
labels. The main idea behind DP-CGAN is that it clips the gradients of
discriminator loss on real and fake data separately, sums up two sets of
gradients, and adds Gaussian noise to the sum. DP-CGAN employs RDP account
technique to track the spent privacy budget. The experimental results showed
that DP-CGAN improves the performance compared to basic DP-CGAN and generates
promising results on MNIST dataset.

The architectures we used for the generator and discriminator are rather
simple. We are going to consider deep CGAN architectures with multiple
convolutional layers to improve the quality of the synthetic data while
spending the same privacy budget as we did for vanilla CGAN. Moreover, our
results are still preliminary and we are going to show high quality
differentially private CGANs on more challenging datasets such as CIFAR100 and
CelebA/B. Finally, our preliminary results are very promising and we can
extend our methodology to tackle the mentioned challenges.

## References

  * [1] K. He, X. Zhang, S. Ren, and J. Sun, “Deep residual learning for image recognition,” in Proceedings of the IEEE conference on computer vision and pattern recognition, pp. 770–778, 2016.
  * [2] A. Krizhevsky, I. Sutskever, and G. E. Hinton, “Imagenet classification with deep convolutional neural networks,” in Advances in neural information processing systems, pp. 1097–1105, 2012.
  * [3] R. Collobert and J. Weston, “A unified architecture for natural language processing: Deep neural networks with multitask learning,” in Proceedings of the 25th international conference on Machine learning, pp. 160–167, ACM, 2008.
  * [4] G. Hinton, L. Deng, D. Yu, G. E. Dahl, A.-r. Mohamed, N. Jaitly, A. Senior, V. Vanhoucke, P. Nguyen, T. N. Sainath, et al., “Deep neural networks for acoustic modeling in speech recognition: The shared views of four research groups,” IEEE Signal processing magazine, vol. 29, pp. 82–97, Jan. 2012.
  * [5] A. Graves, A.-r. Mohamed, and G. Hinton, “Speech recognition with deep recurrent neural networks,” in Acoustics, speech and signal processing (icassp), 2013 ieee international conference on, pp. 6645–6649, IEEE, 2013.
  * [6] D. Silver, A. Huang, C. J. Maddison, A. Guez, L. Sifre, G. Van Den Driessche, J. Schrittwieser, I. Antonoglou, V. Panneershelvam, M. Lanctot, et al., “Mastering the game of go with deep neural networks and tree search,” Nature, vol. 529, p. 484, Apr. 2016.
  * [7] V. Mnih, K. Kavukcuoglu, D. Silver, A. A. Rusu, J. Veness, M. G. Bellemare, A. Graves, M. Riedmiller, A. K. Fidjeland, G. Ostrovski, et al., “Human-level control through deep reinforcement learning,” Nature, vol. 518, no. 7540, p. 529, 2015.
  * [8] K. He, X. Zhang, S. Ren, and J. Sun, “Delving deep into rectifiers: Surpassing human-level performance on imagenet classification,” in Proceedings of the IEEE international conference on computer vision, pp. 1026–1034, 2015.
  * [9] V. Gulshan, L. Peng, M. Coram, M. C. Stumpe, D. Wu, A. Narayanaswamy, S. Venugopalan, K. Widner, T. Madams, J. Cuadros, et al., “Development and validation of a deep learning algorithm for detection of diabetic retinopathy in retinal fundus photographs,” Jama, vol. 316, no. 22, pp. 2402–2410, 2016.
  * [10] D. Shen, G. Wu, and H.-I. Suk, “Deep learning in medical image analysis,” Annual review of biomedical engineering, vol. 19, pp. 221–248, 2017.
  * [11] D. Ravı, C. Wong, F. Deligianni, M. Berthelot, J. Andreu-Perez, B. Lo, and G.-Z. Yang, “Deep learning for health informatics,” IEEE journal of biomedical and health informatics, vol. 21, no. 1, pp. 4–21, 2017.
  * [12] H. Greenspan, B. Van Ginneken, and R. M. Summers, “Guest editorial deep learning in medical imaging: Overview and future promise of an exciting new technique,” IEEE Transactions on Medical Imaging, vol. 35, no. 5, pp. 1153–1159, 2016.
  * [13] C. Angermueller, T. Pärnamaa, L. Parts, and O. Stegle, “Deep learning for computational biology,” Molecular systems biology, vol. 12, no. 7, p. 878, 2016.
  * [14] B. Alipanahi, A. Delong, M. T. Weirauch, and B. J. Frey, “Predicting the sequence specificities of dna-and rna-binding proteins by deep learning,” Nature biotechnology, vol. 33, no. 8, p. 831, 2015.
  * [15] L. J. L. A. D. D. D. Killoran, Nathan and B. J. Frey, “Generating and designing dna with deep generative models,” arXiv preprint arXiv:1712.06148, 2017.
  * [16] H. Zeng, M. D. Edwards, G. Liu, and D. K. Gifford, “Convolutional neural network architectures for predicting dna–protein binding,” Bioinformatics, vol. 32, no. 12, pp. i121–i127, 2016.
  * [17] I. Goodfellow, J. Pouget-Abadie, M. Mirza, B. Xu, D. Warde-Farley, S. Ozair, A. Courville, and Y. Bengio, “Generative adversarial nets,” in Advances in neural information processing systems, pp. 2672–2680, 2014.
  * [18] P. Isola, J.-Y. Zhu, T. Zhou, and A. A. Efros, “Image-to-image translation with conditional adversarial networks,” in The IEEE Conference on Computer Vision and Pattern Recognition (CVPR), July 2017.
  * [19] G. Antipov, M. Baccouche, and J. Dugelay, “Face aging with conditional generative adversarial networks,” in 2017 IEEE International Conference on Image Processing (ICIP), pp. 2089–2093, Sep. 2017.
  * [20] C. Ledig, L. Theis, F. Huszar, J. Caballero, A. Cunningham, A. Acosta, A. Aitken, A. Tejani, J. Totz, Z. Wang, and W. Shi, “Photo-realistic single image super-resolution using a generative adversarial network,” in The IEEE Conference on Computer Vision and Pattern Recognition (CVPR), July 2017\.
  * [21] G. Antipov, M. Baccouche, and J.-L. Dugelay, “Face aging with conditional generative adversarial networks,” in 2017 IEEE International Conference on Image Processing (ICIP), pp. 2089–2093, IEEE, 2017.
  * [22] R. Shokri, M. Stronati, C. Song, and V. Shmatikov, “Membership inference attacks against machine learning models,” in Security and Privacy (SP), 2017 IEEE Symposium on, pp. 3–18, IEEE, 2017.
  * [23] J. Hayes, L. Melis, G. Danezis, and E. De Cristofaro, “Logan: evaluating privacy leakage of generative models using generative adversarial networks,” arXiv preprint arXiv:1705.07663, 2017.
  * [24] M. Fredrikson, S. Jha, and T. Ristenpart, “Model inversion attacks that exploit confidence information and basic countermeasures,” in Proceedings of the 22nd ACM SIGSAC Conference on Computer and Communications Security, pp. 1322–1333, ACM, 2015.
  * [25] C. Zhang, S. Bengio, M. Hardt, B. Recht, and O. Vinyals, “Understanding deep learning requires rethinking generalization,” arXiv preprint arXiv:1611.03530, 2016.
  * [26] F. M. I. M. Cynthia Dwork, Krishnaram Kenthapadi and M. Naor, “Our data,ourselves: Privacy via distributed noise generation,” EUROCRYPT, 2006\.
  * [27] K. N. Cynthia Dwork, Frank McSherry and A. Smith, “Calibrating noise to sensitivity in private data analysis,” in Theory of Cryptography Conference, p. 265–284, Springer, 2006.
  * [28] L. Xie, K. Lin, S. Wang, F. Wang, and J. Zhou, “Differentially private generative adversarial network,” arXiv preprint arXiv:1802.06739, 2018\.
  * [29] L. Frigerio, A. S. de Oliveira, L. Gomez, and P. Duverger, “Differentially private generative adversarial networks for time series, continuous, and discrete open data,” arXiv preprint arXiv:1901.02477, 2019.
  * [30] J. Yoon, J. Jordon, and M. van der Schaar, “Pate-gan: Generating synthetic data with differential privacy guarantees,” ””, 2018.
  * [31] B. K. Beaulieu-Jones, Z. S. Wu, C. Williams, R. Lee, S. P. Bhavnani, J. B. Byrd, and C. S. Greene, “Privacy-preserving generative deep neural networks support clinical data sharing,” BioRxiv, p. 159756, 2018.
  * [32] Q. Chen, C. Xiang, M. Xue, B. Li, N. Borisov, D. Kaarfar, and H. Zhu, “Differentially private data generative models,” arXiv preprint arXiv:1812.02274, 2018.
  * [33] M. Abadi, A. Chu, I. Goodfellow, H. B. McMahan, I. Mironov, K. Talwar, and L. Zhang, “Deep learning with differential privacy,” in Proceedings of the 2016 ACM SIGSAC Conference on Computer and Communications Security, pp. 308–318, ACM, 2016.
  * [34] I. Mironov, “Reny differential privacy,” in 2017 IEEE 30th Computer Security Foundations Symposium (CSF), pp. 263–275, IEEE, 2017.
  * [35] M. Mirza and S. Osindero, “Conditional generative adversarial nets,” arXiv preprint arXiv:1411.1784, 2014.
  * [36] A. Myers and G. Nelson, “Differential privacy: Raising the bar,” 1 Geo. L. Tech. Rev., vol. 1, pp. 135–142, November 2016.
  * [37] U. S. V. G. K. J. F. V. R. S. Abhradeep Thakurta, Andrew Vyrros and D. Davidson, “Learning new words,” US Patent no. 9,594,741 B1, 2017.
  * [38] Ú. Erlingsson, V. Pihur, and A. Korolova, “Rappor: Randomized aggregatable privacy-preserving ordinal response,” in Proceedings of the 2014 ACM SIGSAC conference on computer and communications security, pp. 1054–1067, 2014\.
  * [39] A. D. L. Phyllis E. Singer Daniel Kifer Jerome P. Reiter Ashwin Machanavajjhala Simson L. Garfinkel Scot A.Dahl Matthew Graham Vishesh Karwa Hang Kim Philip Leclerc Ian M. Schmutte William N. Sexton Lars Vilhuber Aref N. Dajani and J. M. Abowd, “The modernization of statistical disclosure limitation at the u.s. census bureau,” ””, 2018.
  * [40] C. Dwork and J. Lei, “Differential privacy and robust statistics.,” in STOC, vol. 9, pp. 371–380, 2009.
  * [41] P. Kairouz, S. Oh, and P. Viswanath, “The composition theorem for differential privacy,” IEEE Transactions on Information Theory, vol. 63, no. 6, pp. 4037–4049, 2017.
  * [42] C. Dwork and G. N. Rothblum, “Concentrated differential privacy,” arXiv preprint arXiv:1603.01887, 2016.
  * [43] C. Dwork, G. N. Rothblum, and S. Vadhan, “Boosting and differential privacy,” in 2010 IEEE 51st Annual Symposium on Foundations of Computer Science, pp. 51–60, IEEE, 2010.
  * [44] C. Dwork, A. Roth, et al., “The algorithmic foundations of differential privacy,” Foundations and Trends® in Theoretical Computer Science, vol. 9, no. 3–4, pp. 211–407, 2014.
  * [45] N. Papernot, M. Abadi, U. Erlingsson, I. Goodfellow, and K. Talwar, “Semi-supervised knowledge transfer for deep learning from private training data,” arXiv preprint arXiv:1610.05755, 2016.
  * [46] C. Dwork, A. Roth, et al., “The algorithmic foundations of differential privacy,” Foundations and Trends® in Theoretical Computer Science, vol. 9, no. 3–4, pp. 211–407, 2014.