# Elucidating the oscillation instability of sessile drops triggered by
surface acoustic waves

Nicolas Chastrette1,2 Michaël Baudoin3,4 Philippe Brunet2 Laurent Royon5 Régis
Wunenburger1<EMAIL_ADDRESS>1Sorbonne Université,
CNRS, Institut Jean Le Rond d’Alembert, F-75005 Paris, France
2Université de Paris, MSC, UMR 7057, CNRS, F-75013 Paris, France
3Univ. Lille, CNRS, Centrale Lille, ISEN, Univ. Polytechnique Hauts de France,
UMR 8520, IEMN, F-59000 Lille, France
4Institut Universitaire de France, 1 rue Descartes, 75231 Paris Cedex 05,
France
5Université de Paris, LIED, UMR 8236, CNRS, F-75013 Paris, France

###### Abstract

The oscillation instability of sessile drops is ubiquitous in surface acoustic
wave (SAW)-powered digital microfluidics. Yet, the physics underlying these
phenomena has not been elucidated owing to the interplay between
hydrodynamics, acoustics and capillarity. We decipher the instability by
combining high-speed imaging with pressure measurements. We rationalize the
observed behaviour with a model inspired from optomechanics, which couples an
intracavity acoustic mode excited by the SAW to a surface deformation
eigenmode through amplitude modulation and delayed radiation pressure
feedback.

Manipulating microparticles or fluid samples is a key issue in microfluidics
for e.g. high throughput screening, bottom-up processing, selectivity or
detection sensitivity enhancement or risk reduction Tabelling (2005).
Actuation by surface acoustic waves (SAW) has been early identified as a
versatile and efficient tool in both microchannel and drop microfluidics
Friend and Yeo (2011); Yeo and Friend (2014); Connacher  _et al._ (2018);
Riaud  _et al._ (2016). The interaction of a SAW with a sessile drop results
in various phenomena depending on the liquid viscosity, drop size, SAW
frequency, phase and intensity distribution: drop trapping Alvarez  _et al._
(2008), oscillations and transport Wixforth (2003); Renaudin  _et al._ (2006);
Brunet  _et al._ (2010); Baudoin  _et al._ (2012); Bussonnière  _et al._
(2016), liquid atomisation Shiokawa  _et al._ (1989, 1990); Chono  _et al._
(2004); Qi  _et al._ (2008); Tan  _et al._ (2009), particle transport or
segregation Tan  _et al._ (2007); Rezk  _et al._ (2014), mixing Frommelt  _et
al._ (2008) and heating Kondoh  _et al._ (2009); Reboud  _et al._ (2012);
Shilton  _et al._ (2015). The physical mechanisms at play combine
hydrodynamics, acoustics, capillarity and wetting. In this letter, we address
the $\sim\\!\\!10^{2}$ Hz surface oscillation instability of sessile drops
insonicated by $\sim\\!\\!10^{6}$ Hz SAWs, a counter-intuitive phenomenon that
has been observed to precede surface wave turbulence Blamey  _et al._ (2013)
as well as two key-applications of SAWs in microfluidics, namely atomisation
Qi  _et al._ (2009); Friend and Yeo (2011) and drop transport Brunet  _et al._
(2010); Baudoin  _et al._ (2012), and whose elucidation as been quoted as one
of the major fundamental challenges in Yeo and Friend’s reviews on
acoustofluidics Friend and Yeo (2011); Yeo and Friend (2014).

When a SAW irradiates a sessile drop, it is partially converted to a bulk
longitudinal wave, which remains confined within the drop acting as a cavity.
Here we operate at moderate acoustic excitation frequency $f_{\text{ae}}\simeq
1$ MHz such that $\lambda/R\simeq 1$ ($c\simeq 1.5~{}\text{km.s}^{-1}$ the
sound velocity in water, $\lambda=c/f_{\text{ae}}$ the acoustic wavelength and
$R\simeq 1.6$ mm the drop radius). This enables to excite a single acoustic
mode in the cavity, while at higher frequencies, high modal density leads to
mode overlapping and in turn to chaotic behaviour Riaud  _et al._ (2017).
Furthermore, a drop is deformable and behaves as a mechanical oscillator whose
stiffness is associated to surface tension and mass to liquid inertia Rayleigh
(1879); Lamb (1932). By combining high-speed imaging with acoustic pressure
measurements and using an instability model inspired from optomechanics, we
demonstrate that drop oscillations result from the mutual interaction between
the mechanical oscillator and the confined wave field, through amplitude
modulation and delayed radiation pressure feedback, a scheme reminiscent of
the parametric instability of gravitational wave interferometric detectors
Abramovici  _et al._ (1992), force sensing microlevers Jourdan  _et al._
(2008); Favero and Karrai (2009) and opto-mechanical oscillators designed for
quantum intrication Metzger  _et al._ (2008).

Figure 1: A $10.0~{}\mu\text{L}$ water drop deposited on a glass slab is
excited by a harmonic progressive Rayleigh wave. A needle hydrophone, whose
tip is brought into contact with the liquid surface, measures the intracavity
pressure $p(t)$. Full pictures of the slowly evaporating drop are taken at
$30$ fps. Oscillations of the drop free surface close to its apex (red frame
on the main picture) are recorded at $1000$ fps using a high-speed camera in a
narrow field (shown in inset).

A sessile water drop of volume $V=10.0~{}\mu\text{L}$ partially wets a $10$
mm-thick glass slab with a contact angle $\theta{{}_{\mathrm{w}}}$ close to
$90^{\circ}$ and a contact line radius $R_{w}=1.65$ mm (see Fig. 1 and SM ).
The drop is irradiated by a plane, progressive Rayleigh wave SM . The tip of a
needle hydrophone with $200~{}\mu\text{m}$-diameter active element is put in
contact with the liquid, allowing for a measurement of the intracavity
acoustic pressure $p(t)$. Full pictures of the drop are acquired using a
camera at $30$ fps, while the oscillations of its free surface in a narrow
field close to its apex (Fig. 1) are recorded at $2.2~{}\mu\text{m/pixel}$
resolution using a high-speed camera at $1000$ fps. The experiments consist in
exciting a drop with a harmonic SAW at carrier frequency $f_{\mathrm{ae}}$
close to one of the resonance frequencies $f_{\mathrm{ar}}$ of the acoustic
cavity. Amplitude ramps are applied in order to detect the onset of the
oscillation instability.

Figure 2: (a) Red curve: Narrow-band spectrum $|u(f)|$ of the frequency-swept
voltage periodically applied to the transducer to scan the drop acoustic
resonances during its evaporation. Spectrum $\hat{P}(f)=|p(f)/u(f)|$ of the
drop pressure response immediately after its deposition (thin black curve) and
after $3$ min of evaporation (bold black curve). Blue curve: best fit by Eq.
(1). $f_{2,1}$ and $f_{0,2}$ are the numerically predicted acoustic
eigenfrequencies of a hemispherical drop with radius $R=1.68$ mm. (b) Acoustic
resonance frequency $f_{\rm ar}$ vs $H$ during the evaporation of three drops
(connected red, black, blue symbols). Solid and dashed curves: theoretical
predictions, see text. Inset: pressure distribution of the forced $m=\pm 2$
eigenmodes. (c) Corresponding acoustic quality factor $Q_{\rm ar}$. Bold
symbols: measurements during which the instability is observed.

The linear acoustic response of the cavity formed by the drop is determined by
supplying the transducer with a voltage signal $u(t)$ whose frequency linearly
sweeps over the range $0.8-0.9$ MHz, with low amplitude $U=5$ V and $20$ ms
duration (its spectral amplitude $|u(f)|$ is displayed in Fig. 2(a)). The
spectral amplitude $\hat{P}(f)=|p(f)/u(f)|$ of the pressure response of a
freshly deposited drop (height $H_{0}\\!=\\!1.72$ mm) is shown in Fig. 2(a),
where $p(f)$ is the Fourier transform of $p(t)$. $\hat{P}$ exhibits two peaks
in the range $0.8-0.9$ MHz respectively at $f_{\mathrm{ar}}\simeq 0.81$ MHz
and $f^{{}^{\prime}}_{\mathrm{ar}}\simeq 0.87$ MHz. Considering the drop as
hemispherical and assuming (i) no displacement of the rigid glass surface and
(ii) pressure node at the drop free surface, the acoustic eigenmodes can be
expressed using spherical waves Baudoin and Thomas (2020) and their
eigenfrequencies $f_{\ell,q}$ satisfy $j_{\ell}\left(2\pi
Rf_{\ell,q}/c\right)=0$ where $R=(3V/(2\pi))^{1/3}=1.68$ mm, $j_{\ell}$ is the
spherical Bessel function of the first kind, $\ell$ a positive integer and $q$
the number of the root of $j_{\ell}$ in ascending order SM . Two
eigenfrequencies $f_{2,1}=0.82$ MHz (corresponding to $3$ degenerate modes
symmetrical to the equatorial plane, labelled by $m=0,\pm 2$ SM ) and
$f_{0,2}=0.89$ MHz fall in the range $0.8-0.9$ MHz and are found to be close
to the measured peak frequencies $f_{\mathrm{ar}}$ and
$f^{{}^{\prime}}_{\mathrm{ar}}$. This demonstrates that these peaks do
correspond to the resonant forcing of cavity eigenmodes.

Due to evaporation, the height $H$ of the drop apex decreases in time while
its contact line remains pinned during the first $5$ min, resulting in a
continuous increase of $f_{\mathrm{ar}}$, see Fig. 2(a, b). A numerical
calculation (using Comsol software) of the acoustic eigenmodes of a sessile
drop with fixed $R_{w}$, initial volume $V$ and initial height $H_{0}$
evidences that, when the drop is not hemispherical ($H\\!\neq\\!R_{w}$), the
eigenfrequencies of the $(\ell=2,q=1)$ modes split SM . The comparison between
the variations of $f_{\mathrm{ar}}$ with $H$ measured during several
experiments and the computed variations of the eigenfrequencies of the $m=\pm
2$ and $m=0$ modes versus $H$, shown in Fig. 2(b), allows us to unambiguously
identify $f_{\mathrm{ar}}$ as the resonance frequency of the
$(\ell=2,q=1,m=\pm 2)$ modes, whose pressure field is shown in inset of Fig.
2(b).

As shown in Fig. 2(a), during evaporation the pressure response is accurately
described by a one-dimensional (1D) resonator model:

$\hat{P}(f)=\hat{P}_{\text{i}}\left|1+\left[1-\frac{\pi}{2Q_{\text{ar}}}\exp\left(i\pi\frac{f}{f_{\text{ar}}}\right)\right]\right|^{-1}$
(1)

where $\hat{P}_{\text{i}}=(3.5\pm 1.5)~{}\text{kPa}\cdot\text{V}^{-1}$ is the
magnitude related to the excitation of the acoustic wave refracted through the
drop by mode conversion and $Q_{\text{ar}}$ is the quality factor. Fig. 2(c)
displays the variations vs $H$ of $Q_{\text{ar}}$ measured over several
experiments. Its fluctuations lie in the range $80-250$ and may be ascribed to
the slight variations of the triple line shapes along the substrate and needle
tip from one experiment to another.

We now address the oscillation instability. Due to evaporation,
$f_{\mathrm{ar}}$ increases at a typical rate of $3.5~{}\text{kHz.min}^{-1}$.
Instead of continuously adjusting the frequency $f_{\text{ae}}$ of the
acoustic excitation for maintaining the same interference conditions in the
drop, we set $f_{\text{ae}}$ slightly above the initial value of
$f_{\mathrm{ar}}$ (see Fig. 2(a)) i.e. $f_{\text{ae}}=0.84$ MHz. Hence,
$f_{\mathrm{ar}}$ crosses $f_{\mathrm{ae}}$ after $3\\!-\\!4$ min and the drop
acoustic resonance is scanned in $5$ min, which leads us to analyze how the
drop unstable behavior changes with time. Accordingly, every $15$ s, (i) the
linear acoustic response of the cavity is measured as described above, (ii)
then the drop is insonicated using a harmonic SAW with fixed frequency
$f_{\mathrm{ae}}$, amplitude $U$ increasing as $\sqrt{t}$ from $0$ V to $55$ V
in $1$ s, as sketched in Fig. 3(a), so that the radiation pressure (RP)
exerted by the intracavity acoustic field on the drop surface, which is
proportional to $U^{2}$, increases linearly in time, (iii) while $p(t)$ and
the drop surface height $h(t)$, defined in Fig. 1 and shown in Fig. 3(b), are
recorded, (iv) the remaining time being dedicated to data transfer.

Figure 3: (a) Amplitude $U(t)$ of the sine voltage signal prescribed to the
transducer, which is 15 s-periodic, 1 s-long (note the breaks on time scale).
Inset: zoom on the frequency-swept, small-amplitude signal used to assess the
cavity acoustic response. (b) Relative height $h(t)$ (arbitrary origin) of the
drop surface imaged using the high-speed camera, undergoing a slow decrease
due to evaporation and possibly oscillations during excitation. (c)
Enlargements of the parts of the signals framed in blue in (a) and (b),
evidencing the surface oscillation instability triggered for $U\geq U_{\rm
th}$. (d) Spectrogram of $h(t)$ evidencing a monochromatic oscillation at
frequency $f_{\rm so}\simeq 230$ Hz. Inset: corresponding drop deformation
pattern measured from above SM . (e) Amplitude $P(t)$ of the intracavity
pressure $p(t)$, displaying modulation starting from $t_{\rm th}$. (f)
Spectrogram of $p$ showing the appearance of satellite peaks shifted from the
carrier frequency $f_{\rm ae}$ by $\pm f_{\rm so}$ starting from $t_{\rm th}$.
Inset: sketch of the feedback of the drop deformation on the intracavity
field.

As it can be understood from Fig. 2(a), during evaporation, starting from non-
resonant conditions (“blue detuning” in optical physics), the drifting
resonance frequency $f_{\mathrm{ar}}$ crosses $f_{\mathrm{ae}}$ for a given
height $H^{\text{res}}\simeq 1.63$ mm, realizing a resonant forcing. When
$f_{\mathrm{ar}}$ overruns $f_{\mathrm{ae}}$, the cavity is driven away from
resonance (“red detuning”). Accordingly, as shown in Figs. 4(a-c), when $H$
decreases, the magnitude of the intracavity pressure response to excitation
$\hat{P}(f_{\text{ae}})$ first increases, then reaches a maximum when
$f_{\mathrm{ar}}=f_{\mathrm{ae}}$, and finally decreases. The effect of the
evaporation on the drop eigenmode excitation and the way to take advantage of
it are reminiscent of a previous study on concentration patterns of colloids
in SAW-excited drops Li  _et al._ (2008).

Figure 4: (a-c) Left scale, joined black symbols: $H$-dependent pressure
response $\hat{P}$ at excitation frequency $f_{\mathrm{ae}}$. Right scale,
joined red symbols: instability threshold voltage $U_{\rm th}$ vs $H$. Blue
curve: best fit of $U_{\rm th}(h)$ by Eqs. (2, 4). (d) Solid curves: measured
surface oscillation frequency $f_{\text{so}}$ vs $U$ ($\geq U_{\text{th}}$)
for several values of $H$. Dashed curves: corresponding predictions from Eq.
(5).

Meanwhile, as shown in Fig. 3(b, c), $h$ slowly decreases and undergoes
oscillations during excitation when $U$ exceeds a threshold $U_{\mathrm{th}}$.
The variations of $U_{\mathrm{th}}$ vs $H$, measured during several
experiments, some of which displayed in Fig. 4(a-c), reveal that two
instability tongues reproducibly show up on the right-hand side
($f_{\mathrm{ae}}>f_{\mathrm{ar}}$) and left-hand side
($f_{\mathrm{ae}}<f_{\mathrm{ar}}$) of the peak on the curve of variation of
$\hat{P}(f_{\text{ae}})$ vs $H$, demonstrating that the instability occurs
close to (and not at) an eigenfrequency of the acoustic cavity. Besides, Fig.
2(c) shows that the occurrence of the instability is not correlated to the
variations of $Q_{\text{ar}}$.

The spectrogram of $h(t)$ shown in Fig. 3(d) evidences that, at the
instability onset, $h$ shows harmonic oscillations of frequency
$f_{\mathrm{so}}\simeq 230$ Hz. An independent recording of the deformations
of the image of a grid visualized through the drop SM  allowed us to measure
the free-surface deformations and to identify the corresponding forced surface
eigenmode, whose experimental pattern is shown in inset of Fig. 3(d) SM ;
Bostwick and Steen (2014). We conclude that the instability involves a single
surface eigenmode.

To gain insight into the instability mechanism, we now consider the time
evolution of the amplitude $P$ of $p(t)$, shown in Fig. 3(e): $P$ smoothly
increases with $U$ up to the instability threshold. Once the surface
oscillates, $P$ exhibits oscillations at $f_{\mathrm{so}}$, whose amplitude
increases with $U$. This modulation can be explained as follows: the drop
behaves as a 1D resonator with an effective length modulated by the free
surface oscillations $h(t)=A\cos{(\omega_{\mathrm{so}}t)}$ ($\omega=2\pi f$
throughout the text), see inset in Fig. 3(f). Hence, $f_{\text{ar}}$ is
modulated at frequency $f_{\text{so}}$ with amplitude
$A\frac{df_{\text{ar}}}{dh}$, where $\frac{df_{\text{ar}}}{dh}$, which
quantifies the sensitivity of $f_{\text{ar}}$ to drop deformations, is
negative since a 1D resonator eigenfrequency decreases with its length. Since
the delay of adaptation of $p(t)$ to changes in interference conditions
$\tau_{\text{ar}}=Q_{\text{ar}}/\omega_{\text{ar}}\simeq 20~{}\mu\text{s}$ is
much shorter than their period $f_{\text{so}}^{-1}\simeq 4$ ms, $P(t)$
quasistatically follows these changes and in turn adopts a similar modulation
at frequency $f_{\text{so}}$, with amplitude $\Delta
P=U\frac{\partial\hat{P}}{\partial
f_{\text{ar}}}(f_{\text{ae}})A\frac{df_{\text{ar}}}{dh}$. This is
experimentally confirmed by the spectrogram of $p$ shown in Fig. 3(f), which
displays a pair of satellite peaks shifted from $f_{\text{ae}}$ by $\pm
f_{\mathrm{so}}$ SM .

Finally, we address the feedback of the amplitude-modulated intracavity field
on the surface oscillations. The aforementioned RP, defined as the time-
average over $f_{\text{ae}}^{-1}$ of the pressure $p(t)$ exerted on the drop
free surface, results in a normal stress oriented outwards and scaling as
$\Pi=P^{2}/(\rho c^{2})$, where $\rho$ is the water density Herrey (1955);
Borgnis (1953). The first ingredient of the proposed feedback is that the
intracavity pressure modulation amplitude $\Delta P$ induced by the surface
oscillations results in RP oscillations of amplitude $\Delta\Pi=2P\,\Delta
P/(\rho c^{2})$ at instability onset, which consequently have the same
frequency $f_{\text{so}}$ as the surface oscillations. These RP oscillations
may damp or amplify the surface oscillations depending on their phase
difference. Close to resonance, the amplitude $h$ of the surface eigenmode
forced by RP oscillations follows the dynamics of a forced mass-spring system:
$\ddot{h}+\frac{\omega_{so}}{Q_{\text{so}}}\dot{h}+\omega_{\text{so}}^{2}h=\frac{F(h)}{m}$,
where $\dot{h}$ is the time derivative of $h$ and $Q_{\text{so}}=(40\pm 15)$
is the independently measured quality factor of the resonance of the surface
eigenmode SM . Evaluating the effective wavenumber of the surface eigenmode as
$k_{\text{so}}=\frac{4}{R}$ SM , $F\propto k_{\text{so}}^{-2}\Delta\Pi$ is the
modulated acoustic radiation force exerted on a portion of drop surface of
characteristic size equal to one deformation wavelength
$\lambda_{\text{so}}=2\pi/k_{\text{so}}$ and $m\propto\rho k_{\text{so}}^{-3}$
the corresponding mass of moving water.

At the instability onset, we can linearize the variations of $F$ around
equilibrium: $\frac{F(h)}{m}=\frac{K_{\text{a}}}{m}h$ where
$K_{\text{a}}=F^{{}^{\prime}}(0)$ accounts for a modification of the surface
stiffness induced by RP (“acoustic spring” effect Issenmann  _et al._ (2013)).
Thus, $h$ satisfies:
$\ddot{h}+\frac{\omega_{\text{so}}}{Q_{\text{so}}}\dot{h}+\left(\omega_{\text{so}}^{2}-\frac{K_{\text{a}}}{m}\right)h=0$
where

$\frac{K_{\text{a}}}{m}=\frac{k_{\text{so}}}{\rho^{2}c^{2}}U^{2}\hat{P}\frac{\partial\hat{P}}{\partial
f_{\text{ar}}}(f_{\text{ae}})\frac{df_{\text{ar}}}{dh}$ (2)

within an unknown factor. First, we focus on the right-hand side of the peak
of the $\hat{P}(f_{\text{ae}})$ vs $H$ curve shown in Fig. 4(a-c)
($f_{\mathrm{ae}}>f_{\mathrm{ar}}$), along which
$\frac{\partial\hat{P}}{\partial f_{\text{ar}}}(f_{\text{ae}})>0$ since a
decrease of $f_{\text{ar}}$ due to a crest ($h>0$) detunes the cavity, see
Fig. 2(a). Hence $K_{\text{a}}<0$ and the eigenmode dynamics is that of a
stiffer, yet damped free harmonic oscillator exhibiting no instability. A
missed point is the finite delay $\tau_{\text{ar}}$ of adaptation of $P$, and
hence of $\Pi$, to the changes of the intracavity interference conditions
caused by the surface oscillations. In the harmonic regime, this results in a
phase lag of $F$ with respect to $h$ : $K_{\text{a}}$ is to be changed to
$K_{\text{a}}\exp(-i\phi)$, where
$\phi=\omega\tau_{\text{ar}}\simeq\omega_{\text{so}}\tau_{\text{ar}}\simeq
0.03\ll 1$. Thus, in the harmonic regime, $h$ satisfies:

$\ddot{h}+\left(\frac{\omega_{\text{so}}}{Q_{\text{so}}}+\frac{K_{\text{a}}}{m\omega}\sin\phi\right)\dot{h}+\left(\omega_{\text{so}}^{2}-\frac{K_{\text{a}}}{m}\cos\phi\right)h=0.$
(3)

Since $\sin\phi>0$ and $K_{a}<0$, the damping term in Eq. (3) is reduced by
the delayed feedback. The condition for instability is a negative damping
occurring for $U\geq U_{\text{th}}$ such that:

$\frac{|K_{\text{a}}(U_{\text{th}})|}{m}=\frac{\omega_{\text{ar}}\omega_{\text{so}}}{Q_{\text{ar}}Q_{\text{so}}}$
(4)

given $\omega\simeq\omega_{\text{so}}$ and $\sin\phi\simeq\phi$. Furthermore,
as shown by Eq. (3), the acoustic spring effect increases the oscillation
frequency $\omega^{{}^{\prime}}_{\text{so}}$:

$\omega^{{}^{\prime}}_{\text{so}}=\sqrt{\omega_{\text{so}}^{2}-\frac{K_{\text{a}}}{m}}$
(5)

since $\cos\phi\simeq 1$.

The instantaneous oscillation frequency $f^{{}^{\prime}}_{\text{so}}$ is
extracted from the spectrograms of $h(t)$ measured for several values of $H$.
Fig. 4(d) evidences the increase of $f^{{}^{\prime}}_{\text{so}}$ with $U$,
which is in agreement with the stiffening predicted by (5). The negative
offset of $f^{{}^{\prime}}_{\text{so}}$ with respect to $f_{\text{so}}=230$ Hz
may be ascribed to the RP-induced static deformation of the drop, see SM . To
test the quantitative validity of this model for
$f_{\mathrm{ae}}>f_{\mathrm{ar}}$, we compare Eqs. (4) and (5) to the
experimental data using Eq. (1) fitted to the resonance curve at each value of
$H$, Eq. (2) and $Q_{\text{ar}}=100$, $\alpha$ (defined as
$\frac{df_{\text{ar}}}{dh}=-\alpha\frac{f_{\text{ar}}}{H}$) being the only
fitting parameter. As shown in Figs. 4(a-c), the measured and predicted
instability tongues $U_{\text{th}}(H)$ are in quantitative agreement. The same
goes for the measured and predicted rates of variation of
$f^{{}^{\prime}}_{\text{so}}(U)$, as shown in Fig. 4(d). Moreover, the values
of $\alpha$ corresponding to the best fits of $U(H)$ and of the rates of
variation of $f_{\rm so}(U)$ are scattered in a narrow range (between $2\cdot
10^{-4}$ and $10^{-3}$). Thus, the model reproduces quantitatively and self-
consistently two main observables of the instability, namely its threshold and
the RP-induced stiffening.

Finally, we consider the left-hand side of the peak of the
$\hat{P}(f_{\text{ae}})$ vs $H$ curve ($f_{\mathrm{ae}}<f_{\mathrm{ar}}$),
along which $\frac{\partial\hat{P}}{\partial f_{\text{ar}}}(f_{\text{ae}})<0$.
As shown in SM , the average RP non-linearly deforms the drop as in Issenmann
_et al._ (2006). As a result, each time the oscillation instability is
observed, the steady deformation satisfies $\frac{\partial\hat{P}}{\partial
f_{\text{ar}}}(f_{\text{ae}})>0$. This basically explains why oscillations are
also observed on the left-hand side of the peak of the
$\hat{P}(f_{\text{ae}})$ vs $H$ curve.

We note that the phase-locking mechanism proposed in Mahravan  _et al._ (2016,
2020) for explaining the instability of low-frequency surface eigenmodes
($f_{\text{so}}\ll f_{\text{ar}}$) predicts that (i) the components along the
liquid surface of the acoustic and surface eigenmodes coincide and (ii) the
larger the acoustic amplitude, the larger the amplitude of the surface
oscillation. This is not experimentally observed here, see Figs. 4, 3d and 2b
(even the symmetries of the acoustic and surface eigenmodes even do not
coincide), thus making this model inadequate for explaining our observations,
possibly because it does not take into account the feedback of the drop shape
on the acoustic field.

In the light of these results, first we suggest that SAW-driven surface
turbulence Blamey  _et al._ (2013) may originate from the independent
excitation of numerous surface eigenmodes above different thresholds since
many surface eigenmodes can in principle interact with the intracavity
acoustic field, as shown in SM . Moreover, we note that the contact-line
pinning, which usually hinders drop transport, can be overcome by low-
frequency drop oscillations Baudoin  _et al._ (2012); Bussonnière  _et al._
(2016), thus making of this oscillation instability an essential ingredient of
SAW-induced transport. Indeed, since intracavity resonances are expected to
persist up to $40$ MHz Brunet  _et al._ (2010), we expect the above unraveled
instability mechanism to hold in the higher frequency range where drop
transport is usually achieved, possibly cooperating with acoustic streaming.
Finally, the instability can be also triggered by a hybrid combination of
surface and bulk waves Rezk  _et al._ (2016) and is therefore generic, as
shown by the experiments dedicated to the imaging of the drop surface
deformations presented in SM .

###### Acknowledgements.

M.B. first observed the phenomenon. R.W. proposed and designed the experiments
and supervised the project. N.C. built the setups and carried out the
experiments and their analysis. P.B. and M.B. provided experimental support
and advice. M.B. performed the numerical computations. All authors contributed
to the interpretation of the results. R.W. proposed and derived the model and
wrote the manuscript. M.B. and P.B. helped shape the research and manuscript.
The authors thank J.L. Thomas, A. Riaud and O. Bou-Matar for their help in the
use of their vibrometer, J. Marchall for his help in instrumentation and J.L.
Thomas, J. Pierre and A. Bussonnière for helpful advices.

## References

  * Tabelling (2005) P. Tabelling,  _Introduction to Microfluidics_ , edited by O. U. Press (2005).
  * Friend and Yeo (2011) J. Friend and L. Y. Yeo, Rev. Mod. Phys. 83, 647 (2011).
  * Yeo and Friend (2014) L. Yeo and J. Friend, Ann. Rev. Fluid Mech. 46, 379 (2014).
  * Connacher  _et al._ (2018) W. Connacher, N. Zhang, A. Huang, J. Mei, S. Zhang, T. Gopesh, and J. Friend, Lab Chip 18, 1952 (2018).
  * Riaud  _et al._ (2016) A. Riaud, M. Baudoin, J.-L. Thomas, and O. Bou Matar, IEEE T. Ultrason. Ferr. 63, 1601 (2016).
  * Alvarez  _et al._ (2008) M. Alvarez, J. Friend, and L. Yeo, Langmuir 24, 10629 (2008).
  * Wixforth (2003) A. Wixforth, Superlattices Microstruct. 33, 389 (2003).
  * Renaudin  _et al._ (2006) A. Renaudin, P. Tabourier, V. Zang, J. Camart, and C. Druon, Sens. Actuators B 113, 389 (2006).
  * Brunet  _et al._ (2010) P. Brunet, M. Baudoin, O. B. Matar, and F. Zoueshtiagh, Phys. Rev. E 81, 036315 (2010).
  * Baudoin  _et al._ (2012) M. Baudoin, P. Brunet, O. Bou Matar, and E. Herth, Appl. Phys. Lett. 100, 154102 (2012).
  * Bussonnière  _et al._ (2016) A. Bussonnière, M. Baudoin, P. Brunet, and O. Bou Matar, Phys. Rev. E 93, 053106 (2016).
  * Shiokawa  _et al._ (1989) S. Shiokawa, Y. Matsui, and T. Ueda, IEEE Ultrason. Symp. 1, 643 (1989).
  * Shiokawa  _et al._ (1990) S. Shiokawa, Y. Matsui, and T. Ueda, Jpn. J. Appl. Phys. 29, 137 (1990).
  * Chono  _et al._ (2004) K. Chono, N. Shimizu, Y. Matsui, J. Kondoh, and S. Shiokawa, Japan. J. Appl. Phys. 43, 2987 (2004).
  * Qi  _et al._ (2008) A. Qi, L. Yeo, and J. Friend, Phys. Fluids 20, 073103 (2008).
  * Tan  _et al._ (2009) M. Tan, L. Yeo, and J. Friend, EPL 87 (2009).
  * Tan  _et al._ (2007) M. Tan, J. Friend, and L. Yeo, Lab Chip 7, 618 (2007).
  * Rezk  _et al._ (2014) A. Rezk, L. Yeo, and J. Friend, Langmuir 30, 11243 (2014).
  * Frommelt  _et al._ (2008) T. Frommelt, M. Kostur, M.and Wenzel-Schäfer, P. Talkner, P. Hänggi, and A. Wixforth, Phys. Rev. Lett. 100, 034502 (2008).
  * Kondoh  _et al._ (2009) J. Kondoh, N. Shimizu, Y. Matsui, M. Sugimoto, and S. Shiokawa, Sens. Actuators A 149, 292 (2009).
  * Reboud  _et al._ (2012) J. Reboud, Y. Bourquin, G. Wilson, G. Pall, M. Jiwaji, A. Pitt, A. Graham, A. Waters, and J. Cooper, Proc. Nat. Ac. Sci. USA 109, 15162 (2012).
  * Shilton  _et al._ (2015) R. J. Shilton, V. Mattoli, M. Travagliati, M. Agostini, A. Desii, F. Beltram, and M. Cecchini, Adv. Funct. Mat. 25, 5895 (2015).
  * Blamey  _et al._ (2013) J. Blamey, L. Y. Yeo, and J. R. Friend, Langmuir 29, 3835 (2013).
  * Qi  _et al._ (2009) A. Qi, J. R. Friend, L. Y. Yeo, D. A. Morton, M. P. McIntosh, and L. Spiccia, Lab Chip 9, 2184 (2009).
  * Riaud  _et al._ (2017) A. Riaud, M. Baudoin, O. Bou Matar, J.-L. Thomas, and P. Brunet, J. Fluid Mech. 821, 384 (2017).
  * Rayleigh (1879) L. Rayleigh, Proc. Roy. Soc. London 29, 71 (1879).
  * Lamb (1932) H. Lamb,  _Hydrodynamics_ , edited by C. U. Press (1932).
  * Abramovici  _et al._ (1992) A. Abramovici, W. Althouse, R. W. P. Drever, Y. Gursel, S. Kawamura, F. J. Raab, D. Shoemaker, L. Sievers, R. Spero, K. S. Thorne, R. E. Vogt, R. Weiss, S. E. Whitcomb, and M. Zucker, Science 256, 325 (1992).
  * Jourdan  _et al._ (2008) G. Jourdan, F. Comin, and J. Chevrier, Phys. Rev. Lett. 101, 133904 (2008).
  * Favero and Karrai (2009) I. Favero and K. Karrai, Nature Photonics 3, 201 (2009).
  * Metzger  _et al._ (2008) C. Metzger, M. Ludwig, C. Neuenhahn, A. Ortlieb, I. Favero, K. Karrai, and F. Marquardt, Phys. Rev. Lett. 101, 133903 (2008).
  * (32) See Supplemental Material [url] for information on experiments, acoutic eigenmodes, droplet deformation imaging, unstable surface eigenmodes, bistability of static droplet deformations, which includes Refs. Gumerov and Duraiswami (2004); Frankot and Chellappa (1988); Chang  _et al._ (2015); Bloomfield (2004); Chraïbi  _et al._ (2008).
  * Baudoin and Thomas (2020) M. Baudoin and J.-L. Thomas, Annu. Rev. Fluid Mech. 52, 205 (2020).
  * Li  _et al._ (2008) H. Li, J. R. Friend, and L. Y. Yeo, Phys. Rev. Lett. 101, 084502 (2008).
  * Bostwick and Steen (2014) J. Bostwick and P. Steen, J. Fluid Mech. 760, 5 (2014).
  * Herrey (1955) E. M. Herrey, J. Acoust. Soc. Am. 27, 891 (1955).
  * Borgnis (1953) F. E. Borgnis, Rev. Mod. Phys. 25, 653 (1953).
  * Issenmann  _et al._ (2013) B. Issenmann, A. Auberon, R. Wunenburger, and J.-P. Delville, Eur. Phys. J. E 36, 39 (2013).
  * Issenmann  _et al._ (2006) B. Issenmann, R. Wunenburger, S. Manneville, and J.-P. Delville, Phys. Rev. Lett. 97, 074502 (2006).
  * Mahravan  _et al._ (2016) E. Mahravan, H. Naderan, and E. Damangir, Ultrasonics 72, 184 (2016).
  * Mahravan  _et al._ (2020) E. Mahravan, H. Naderan, and E. Damangir, Phys. Fluids 32, 092111 (2020).
  * Rezk  _et al._ (2016) A. Rezk, J. K. Tan, and L. Yeo, Adv. Mater. 28, 1970 (2016).
  * Gumerov and Duraiswami (2004) N. Gumerov and R. Duraiswami,  _Fast multipole methods for the Helmholtz equation in three dimensions_ (Elsevier, 2004).
  * Frankot and Chellappa (1988) R. T. Frankot and R. Chellappa, IEEE Trans. Pattern Anal. Mach. Intell. 10, 439 (1988).
  * Chang  _et al._ (2015) C.-T. Chang, J. Bostwick, S. Daniel, and P. Steen, J. Fluid Mech. 768, 442 (2015).
  * Bloomfield (2004) P. Bloomfield,  _Fourier analysis of time series: an introduction_ (John Wiley & Sons, 2004).
  * Chraïbi  _et al._ (2008) H. Chraïbi, D. Lasseux, E. Arquis, R. Wunenburger, and J.-P. Delville, Phys. Rev. E 77, 066706 (2008).


# Characterizing Stereotypical Bias from Privacy-preserving Pre-Training

Stefan Arnold  Rene Gröbner  Annika Schreiner
Friedrich-Alexander-Universität Erlangen-Nürnberg
Lange Gasse 20, 90403 Nürnberg, Germany
(stefan.st.arnold, rene.edgar.gröbner<EMAIL_ADDRESS>
###### Abstract

Differential Privacy (DP) can be applied to raw text by exploiting the spatial
arrangement of words in an embedding space. We investigate the implications of
such text privatization on Language Models (LMs) and their tendency towards
stereotypical associations. Since previous studies documented that linguistic
proficiency correlates with stereotypical bias, one could assume that
techniques for text privatization, which are known to degrade language
modeling capabilities, would cancel out undesirable biases. By testing BERT
models trained on texts containing biased statements primed with varying
degrees of privacy, our study reveals that while stereotypical bias generally
diminishes when privacy is tightened, text privatization does not uniformly
equate to diminishing bias across all social domains. This highlights the need
for careful diagnosis of bias in LMs that undergo text privatization.

##  1 Introduction

Language Models (LMs) (Devlin et al., 2019; Radford et al., 2019) are trained
on large corpora of text that may contain confidential information. Since such
information can be recovered from word embeddings (Song and Raghunathan, 2020;
Thomas et al., 2020) and language models (Carlini et al., 2019; Nasr et al.,
2023), privacy emerged as an active concern for building trust and complying
with stringent regulations on privacy protection.

To protect against unintended disclosure of information, Differential Privacy
(DP) (Dwork et al., 2006) has been integrated into machine learning (Abadi et
al., 2016) and language models (McCann et al., 2017; Shi et al., 2022; Du et
al., 2023). DP formalizes privacy through a notion of indistinguishability so
that the model outputs are not affected by the addition or removal of an entry
in the training corpus. This is accomplished by injecting additive noise on
gradients during model training.

Due to scaling issues associated with DP on LMs during perturbation of per-
sample gradient updates (Abadi et al., 2016), there is a trend towards
perturbing the raw text (Fernandes et al., 2019; Feyisetan et al., 2020; Yue
et al., 2021; Chen et al., 2023).

By exploiting the geometric proximity of words in word embeddings (Mikolov et
al., 2013), Feyisetan et al. (2020) proposed a probabilistic mechanism
grounded in metric DP (Chatzikokolakis et al., 2013) to perturb all words in a
text while ensuring plausible deniability (Bindschaedler et al., 2017) of the
text regarding its provenance and content.

However, several studies documented that mechanisms for embedding words in a
high-dimensional space harbor (Bolukbasi et al., 2016; Caliskan et al., 2017;
Garg et al., 2018; Manzini et al., 2019) and transfer (Papakyriakopoulos et
al., 2020) unwanted stereotypes and prejudices present in a text corpus.

#### Contribution.

Building on the rich body of research exploring privacy-fairness trade-offs
(Bagdasaryan et al., 2019; Farrand et al., 2020; Hansen et al., 2022), this
study addresses the implications of text privatization on biased associations
in LMs. Specifically, we pre-train BERT (Devlin et al., 2019) models with
masked language modeling and next sentence prediction on webscraped text
modified under varying levels of privacy. We then score the stereotypical bias
following the context association test of Nadeem et al. (2021) and stereotype
pairs benchmark of Nangia et al. (2020). Our findings reveal a nuanced
landscape where stereotypical bias generally diminishes as privacy guarantees
are tightened. This is in line with prior research indicating that LMs with
impaired language modeling capabilities tend to exhibit less stereotypical
associations (Nadeem et al., 2021). However, this diminution is not uniform
across all social categories as biases associated with certain attributes show
varying trends of stability, amplification, and attenuation. We thus advocate
for careful bias measurement when deploying privacy-preserving LMs.

##  2 Background

To ensure a consistent understanding of privacy and fairness in machine
learning, we provide the foundations of differential privacy and a brief
definition of stereotypical bias along with related work.

###  2.1 Differential Privacy

Differential Privacy (DP) (Dwork et al., 2006) originated in the field of
statistical databases and was adapted to machine learning (Abadi et al.,
2016). DP formalizes privacy through the indistinguishability of model outputs
with respect to the presence or absence of a record in the dataset. The notion
of indistinguishability is achieved through noise and can be controlled by the
privacy budget $\varepsilon\in(0,\infty]$, with privacy guarantees diminishing
as $\varepsilon\rightarrow\infty$.

Despite evidence of preventing information disclosure, the perturbations
caused by noise can have detrimental (Jayaraman and Evans, 2019) and disparate
(Bagdasaryan et al., 2019; Farrand et al., 2020; Hansen et al., 2022) effects
on the behavior of machine learning models. By assessing the accuracy of
differentially private machine learning models for (underrepresented)
subgroups, Bagdasaryan et al. (2019) find a disparate impact regarding gender
and ethnicity in both vision and text.

To prevent the risk of authorship disclosure, text rewriting is an appealing
strategy that applies noise at word level or sentence level by leveraging word
embeddings (Mikolov et al., 2013) or sequence-to-sequence models (Vaswani et
al., 2017). Each approach comes with distinct mechanisms and implications for
balancing utility and privacy.

#### Embedding-based Text Rewriting.

Feyisetan et al. (2020) pioneered a mechanism for text rewriting termed
Madlib. Madlib exploits the distance of words in embedding spaces (Mikolov et
al., 2013) to substitute all words in a text with another word within a radius
controlled by the privacy budget $\varepsilon$. Since this substitution
mechanism scales the notion of indistinguishability by a distance, it
satisfies the axioms of metric DP (Chatzikokolakis et al., 2013).

Building on a word embedding, the substitution involves three steps at word
level: (1) retrieving the continuous representations of words from the
embedding space, (2) adding noise to the representations calibrated using a
multivariate distribution, and (3) mapping the noisy representation back onto
the discrete space of vocabulary by employing a nearest neighbor
approximation. While the probabilistic nature of these substitutions assures
plausible deniability (Bindschaedler et al., 2017), substitutions based on the
distance between words alleviate the curse of dimensionality typical of
randomized response (Warner, 1965).

However, privatizing text through perturbations at word level imposes notable
limitations. Since the privacy guarantee in this approach depend on the
geometry of the embedding space, it necessitates meticulous calibration of the
noise magnitude (Xu et al., 2020). For dense regions of the embedding space,
excessive noise may obscure suitable substitutions. For sparse regions of the
embedding space, minimal noise may not provide sufficient protection against
reconstruction. In addition the to noise calibration, perturbations at word
level, albeit retaining the meaning of a text, encounter difficulties in
maintaining the coherence of the text, such as grammar (Mattern et al., 2022),
ambiguity (Arnold et al., 2023), and hierarchy (Feyisetan et al., 2019).

#### Autoencoder-based Text Rewriting.

Instead of privatization over word embeddings, an orthogonal approach utilizes
sequence-to-sequence models built on recurrent (Bo et al., 2021; Krishna et
al., 2021; Weggenmann et al., 2022) and transformer (Igamberdiev and Habernal,
2023) architectures. Common to these approaches is that noise is added to the
encoder representations of text and the decoder learns to convert these noisy
representations into text but without stylistic identifiers.

By perturbing the text at sentence level, this approach presents unique
challenges compared to perturbing texts at word level. For instance,
Igamberdiev et al. (2022) criticized that the utility is contingent upon the
resemblance between the texts on which the sequence-to-sequence model was
optimized and the texts that are subjected to privacy-preserving paraphrasing.
This limitation in generalizability renders this form of text rewriting
infeasible for the privatization of pretext at scale.

###  2.2 Stereotypical Bias

Bias in machine learning is viewed as prior information that informs
algorithmic learning (Mitchell, 1980). When the prior information is
predicated on stereotypes and prejudices, bias transcends this neutral
definition and manifests in a disproportionate weight in favor of or against a
social group.

The origins of these problematic biases are often rooted in the raw data used
to develop machine learning models (Caliskan et al., 2017). Implicit or
explicit stereotypes based on characteristics such as gender and race can
cause the models to perpetuate and propagate these biases. This can
significantly affect perception and decision making. The issue with
stereotypical bias is particularly acute in the context of language models due
to their extensive training on vast corpora that reflect biases present in
human language. This bias magnifies the potential to influence its tone
(Dhamala et al., 2021) and content (Abid et al., 2021), resulting in negative
effects on individuals and society at large.

Using tests for association analogies, prior research demonstrated that
embeddings harbor stereotypical biases related to gender (Bolukbasi et al.,
2016; Kurita et al., 2019; Chaloner and Maldonado, 2019) and race (Manzini et
al., 2019). Specifically, Caliskan et al. (2017) showed that terms related to
career are associated with male names rather than female names, whereas
unpleasant terms are associated with ethnic minorities. Garg et al. (2018)
elaborate on the temporal dimension of bias in word embeddings by observing
changes in gender and ethnic stereotypes over a century. This diachronic
analysis indicates that while certain stereotypes have diminished over time,
others remain robustly encoded in language. By investigating bias diffusion,
Papakyriakopoulos et al. (2020) showed that biases contained in word
embeddings can permeate natural language understanding, while Abid et al.
(2021) report stereotypes in language generation such as violence for certain
religious groups.

Unlike these studies on bias in raw data, we examine the bias that stems from
text privatization.

##  3 Methodology

To test our hypothesis on amplification of stereotypical bias through text
privatization, we need to define (1) a language model, (2) the mechanism for
text privatization, and (3) a bias measurement.

###  3.1 Language Model

Following Qu et al. (2021), we use a BERT model (Devlin et al., 2019)
leveraging masked language modeling and next sentence prediction tasks for
pre-training. The choice of BERT is motivated by its widespread adoption and
proven effectiveness in capturing contextual relationships within text.

For pre-training, we selected a webscraped replication of WebText (Radford et
al., 2019), which compared to WikiText (Merity et al., 2016), covers a broader
spectrum of topics, styles, and viewpoints. This diversity renders WebText
particularly suited for examining the transfer of stereotypical biases from
the pre-text corpus. For fine-tuning, we reproduced the experiments of
Bagdasaryan et al. (2019) but found no stereotypical bias other than a
disparate impact due to sampling bias.

To assess the alterations in stereotypical bias by text privatization, we
trained a BERT model devoid of any privacy interventions, serving as a control
to score amplification and attenuation, and three additional copies of the
BERT model under varying degrees of privacy guarantees. Since all BERT models
are identical in terms of architecture and optimization (differing solely in
the degree of text privatization), this setup warrants a controlled comparison
that isolates the effects of text privatization on the anchoring of
stereotypical bias.

###  3.2 Text Privatization

To privatize the WebText corpus, we operationalize the Madlib mechanism
developed by Feyisetan et al. (2019) for text privatization at word level.
Madlib necessitates the utilization of continuous representations supplied by
a word embedding. We integrate Madlib with GloVe (Pennington et al., 2014).
GloVe supplies a $400000$-words vocabulary, each mapped to a $300$-dimensional
representation. The choice of GloVe is motivated by the richness of its
semantic space, making it an ideal candidate for privacy-preserving text
privatization.

Since the privacy guarantee of Madlib is rooted in metric DP, we need to
calibrate the noise parameter $\varepsilon$ according to the metric space of
GloVe. This calibration involves an estimation of the plausible deniability
(Bindschaedler et al., 2017) through two proxy statistics (Feyisetan et al.,
2020):

  * •

$N_{w}=\mathbb{P}\\{M(w)=w\\}$ measures the number of identical words that
stem from perturbing a word given a privacy budget $\varepsilon$. We estimate
$N_{w}$ by counting the occurrence of unaltered words after querying a random
subset of $10000$ words for a total of $1000$ times.

  * •

$S_{w}=|\mathbb{P}\\{M(w)=w^{{}^{\prime}}\\}|$ measures the number of unique
words that stem from perturbing a word given a privacy budget $\varepsilon$.
We estimate $S_{w}$ by calculating the effective support of a word after
querying the same random subset of $10000$ words for a total of $1000$ times.

We can relate the proxy statistics to the privacy budget. Adding more noise
corresponds to a tighter privacy guarantee. This is indicated by a smaller
value for $\varepsilon$ and results in a diverse set of perturbed words (low
$N_{w}$ and high $S_{w}$). Adding less noise reflects a weaker privacy
guarantee. This is characterized by a larger value for $\varepsilon$ and
results in more frequent unperturbed words (high $N_{w}$ and low $S_{w}$).

(a) $N_{w}$ refers to the number of perturbed words that are identical to a
queried word.

(b) $S_{w}$ refers to the number of perturbed words that are unique from a
queried word.

Figure 1:  Plausible deniability statistics approximated for a randomly
compiled vocabulary of $10000$ words, each word privatized over a number of
$1000$ queries.  Table 1: Example sentence derived from Webtext and privatized
for three independent runs of Madlib (Feyisetan et al., 2020) using a privacy
budget $\varepsilon$ of $10$.

Tokens | Substiutions
---|---
Port-au-Prince | rosita, xiangfan, tejgaon
, | and, as, ,
Haiti | vanuatu, cuba, haiti
( | (, 45, according
CNN | informed, journalist, speaker
) | –, ), 2000
– | likely, –, two
Earthquake | quake, earthquake, stress
victims | killings, murdered, deaths
, | agrees, things, went
writhing | desolation, stayers ,tiredness
in | out, in, first
pain | frustration, fractures, pain
and | have, over, with
grasping | interplay, spit, dangling
at | at, the, as
life | proud, day, loves
, | and, took, 45
watched | watched, lined, raised
doctors | medical, researchers, surgeons
and | including, as, alongside
nurses | pharmacists, nurses, physicians
walk | walks, sideways, walked
away | gone, away, when
from | from, around, off
a | an, than, one
field | games, yards, field
hospital | school, nursing, staff
Friday | week, thursday, saturday
night | night, hours, watch
after | after, afterwards, before
a | a, first, one
Belgian | danish, macedonian, french
medical | medical, hospital, psychiatric
team | division, helm, cup
evacuated | evacuated, ferried, homeless
the | the, 1984, on
area | town, area, park
, | accused, 6, :
saying | asking, iranians, saying
it | since, as, is
was | that, only, subsequently
concerned | suspicious, expect, insist
about | nearly, just, about
security | beijing, actions, personnel
. | still, then, .

Figure 1 presents the distribution of $N_{w}$ and $S_{w}$. Since $N_{w}$
($S_{w}$) should be positively (negatively) skewed to assure a reasonable
privacy guarantee, we adopt privacy budgets of $\varepsilon=\\{5,10\\}$,
corresponding to a high and low level of privacy protection, respectively.
Table 1 illustrates an example obtained by querying Madlib using a privacy
budget $\varepsilon$ of $10$. Notice the fidelity while some variation asserts
compliance with privacy requirements.

###  3.3 Bias Measurement

Characterizing bias embedded within models typically relies on carefully
crafted datasets. Several datasets exist to measure bias in word embeddings
(Caliskan et al., 2017; May et al., 2019) and language models trained with
masked (Nangia et al., 2020; Nadeem et al., 2021) and causal language modeling
objective (Dhamala et al., 2021).

We adopt the StereoSet dataset designed by Nadeem et al. (2021). Given
associative contexts, this dataset is intended to measure the tendency to
default to stereotypical or anti-stereotypical associations. StereoSet
provides meticulously crafted stimuli for bias measurement regarding gender,
profession, race, and religion at two distinct levels:

#### Intrasentence.

The intrasentence task measures bias for sentence-level reasoning. It is
formulated as a fill-mask task. Given a context sentence describing a social
group, the task is to fill in a masked attribute corresponding to a
stereotype, an anti-stereotype, and an unrelated option. The propensity for
stereotypical associations is gauged by the likelihood of assigning each of
these options.

#### Intersentence.

The intersentence task measures bias for discourse-level reasoning. It is
formulated as a next-sentence task. Given a context sentence pertaining to a
social group, followed by three sentences embodying a stereotype, an anti-
stereotype, and an unrelated attribute, the assessment of stereotypical bias
hinges on which of these sentences is instantiated as the most likely
continuation.

To capture social biases at more differentiated levels, we complement our
investigation with the CrowS-Pairs benchmark designed by Nangia et al. (2020).
This benchmark consists of pairs of minimally distant sentences dealing with
bias about gender identity, ethnic affiliation, age, nationality, religion,
sexual orientation, socioeconomic status, physical appearance, and disability.
The first sentence in each pair demonstrates a stereotype about a social
group, while the second sentence in each pair violates it. This allows to
score the bias in a language model by measuring how frequently it prefers a
statement that portrays a social group stereotypically compared to an
alternative portrayal of the same situation with a different social identity.

Despite some criticism due to issues with model calibration (Desai and
Durrett, 2020), we determine the preferences using pseudo-likelihood scoring
(Salazar et al., 2020). We iterate over each sentence, masking a word at a
time (except for the words that identify a social group), and accumulate the
log-likelihoods of the masks in a sum for comparison.

##  4 Experiments

Prior to initiating our bias measurement, we conducted a preliminary sanity
check by examining the pseudo-perplexity scores of BERT models trained under
varying degrees of privacy. Pseudo-perplexity serves an indicator of a LM’s
ability to accurately model the probability distribution of words within a
text corpus, thereby reflecting the model’s proficiency to comprehend the
linguistic structures encountered during its training.

We use a 10% subset of WikiText for computing the pseudo-perplexities.
Evaluated at privacy levels specified by the privacy parameter $\varepsilon$,
the pseudo-perplexity scores were 93.51 with no privacy interventions, 502.67
with moderate privacy settings, and 2056.43 under conditions of high privacy.
Consistent with previous evidence that introducing noise at word-level
compromises the linguistic proficiency of LMs (Mattern et al., 2022), these
results demonstrate a substantial degradation as the level of privacy
augmentation increases.

The observed degradation raises an interesting question of whether private LMs
harbor stereotypical biases despite diminished language modeling capabilities.
This question forms the basis for our subsequent analysis of the undesirable
biases in LMs stemming from text privatization.

###  4.1 Stereotype Results from StereoSet

Table 2: Percentage preference of stereotypical associations derived from StereoSet, where scores above 0.5 indicate pro-stereotypical bias and scores below 0.5 indicate anti-stereotypical bias. Effect sizes compared to the baseline value according to Cohens $d$ in brackets. Epsilon | $\infty$ | 10 |  5
---|---|---|---
Intrasentence |  |  |
Gender | .6196 | .5490 ($\downarrow.14$) |  .5020 ($\downarrow.24$)
Race | .6060 | .5135 ($\downarrow.19$) |  .4709 ($\downarrow.27$)
Religion | .5897 |  .6538 ($\uparrow.13$) |  .6538 ($\uparrow.13$)
Profession | .6062 | .5679 ($\downarrow.08$) |  .5259 ($\downarrow.16$)
Average | .6054 | .5711 ($\downarrow.07$) |  .5382 ($\downarrow.14$)
Intersentence |  |  |
Gender | .5868 |  .5909 ($\uparrow.01$) |  .5248 ($\downarrow.12$)
Race | .5318 | .5287 ($\downarrow.01$) |  .5461 ($\uparrow.03$)
Religion | .5641 | .5513 ($\downarrow.03$) |  .5385 ($\downarrow.05$)
Profession | .6070 | .5272 ($\downarrow.16$) |  .4813 ($\downarrow.25$)
Average | .5724 | .5495 ($\downarrow.05$) |  .5227 ($\downarrow.10$)

To measure the bias resulting from text privatization at sentence and
discourse level, we commence our analysis by detailing the stereotype scores
derived from the StereoSet benchmark. The stereotype score is defined by the
percentage of examples for which the LM assigns a higher probability to the
pro-stereotypical word as opposed to the anti-stereotypical word. As such,
scores closer to 0.5 are indicative of unbiased associations.

Table 2 presents the averaged stereotype scores grouped by intrasentence and
intersentence tasks and segmented by social categories 111Since Madlib
involves a probabilistic mechanisms, one could argue that the bias patterns of
the privacy budget $\varepsilon$ on social categories is caused by the
randomness of text privatization. To test whether the observed patterns stem
from randomness, we reproduced all experiments using three distinct seeds. The
variance across different configurations suggests that these patterns are
inherent to the privatization process and not merely artifacts of random
perturbations.. Several key trends inform our understanding of the impact of
text privatization on stereotypical bias. We observe that results from the
intrasentence task aligns with those from the intersentence task, showing that
the stereotype scores decline as the privacy level intensifies. For the
intrasentence tasks, the averaged stereotype scores decreased from 0.6054 to
0.5711 and 0.5382 as the privacy budget was tightened to 10 and 5,
respectively. For the intersentence tasks, the stereotype scores decreased
similarity from 0.5724 to 0.5495 and 0.5227, respectively. However, the fall
in stereotype scores is overall more pronounced in the intrasentence task than
in the intersentence task. This disparity implies that mask language modeling
is affected more acutely than next sentence prediction, which requires a
broader context to build stereotypical association.

While text privatization generally reduces stereotypical biases, we find
inconsistent pattern when breaking down the stereotype scores by social
categories. This indicates that the impact of text privatization is not
uniformly spread across social groups.

###  4.2 Stereotype Results from CrowS-Pairs

Table 3: Percentage preference of stereotypes derived from CrowS-Pairs, where
scores closer to 0.5 are indicative of unbiased associations. Effect sizes of
text privatization compared to the baseline value in brackets.

Epsilon | $\infty$ | 10 |  5
---|---|---|---
Gender | .5229 |  .5878 ($\uparrow.13$) |  .5267 ($\uparrow.01$)
Age | .4943 | .4943 ($\uparrow.00$) |  .5402 ($\uparrow.09$)
Race | .5233 | .5446 ($\uparrow.04$) |  .5640 ($\uparrow.08$)
Religion | .6000 | .5905 ($\downarrow.02$) |  .5905 ($\downarrow.02$)
Nationality | .5283 |  .5535 ($\uparrow.05$) |  .5346 ($\uparrow.01$)
Occupation | .5465 | .5407 ($\downarrow.01$) |  .4535 ($\downarrow.19$)
Sexuality | .6786 | .6190 ($\downarrow.12$) |  .5119 ($\downarrow.34$)
Disability | .6167 | .6000 ($\downarrow.03$) |  .5500 ($\downarrow.13$)
Appearance | .4762 |  .6190 ($\uparrow.29$) |  .4921 ($\uparrow.03$)

To explore the manifestation of stereotypical bias across a broader range of
social categories, we broadened our analysis to include CrowS-Pairs. Table 3
confirms that there is no overarching trend regarding the degree of text
privatization and the manifestation of stereotypical biases.

Following the general observation of decreasing stereotype scores as the
privacy budget tightens, further scrutiny into social categories reveals a
complex and heterogeneous response to text privatization. We discern social
categories that are constant (e.g., religion), amplified (e.g., age, race),
and attenuated (e.g., occupation, sexuality, disability). This suggests that
some social categories are detached from the influences of textual
perturbations while others seem less robust. Further complicating the
interactions is that some social categories (e.g., gender, nationality,
appearance) experience fluctuating responses. The categories show an increase
in stereotype scores as privacy settings are intensified before stabilizing or
reverting at the strictest levels of privacy. Except for sexual orientation
($\downarrow.34$) and physical appearance appearance ($\uparrow.29$), the
effect sizes are negligible. This variability underscores the intricate
dynamics between text privatization and LMs, suggesting that minor
modifications in the privacy parameters can have significant and diverse
impacts on stereotypical biases across different social constructs.

##  5 Conclusion

The interaction dynamics that govern the manifestation of bias in LMs are
equivocal (Hansen et al., 2022). Prior research indicates that stereotypical
bias is related to language proficiency in LMs (Nadeem et al., 2021). Since
text privatization is known to impair language modeling capabilities
(Feyisetan et al., 2020), one would expect a general diminution of
stereotypical bias. However, the word embeddings used for text privatization
are documented to harbor (Bolukbasi et al., 2016; Caliskan et al., 2017) and
transfer (Papakyriakopoulos et al., 2020) stereotypical biases. This duality
raises questions about whether text privatization leads to an amplification or
an attenuation of stereotypical biases. By probing a LMs tendency to default
to stereotypical or anti-stereotypical associations, we aimed to elucidate the
relationship between text privatization and the amplification or attenuation
of biases. We find that different social domains react differently to privacy
settings and recommend to carefully assess stereotypical bias after training a
LM on a privatized corpus of text.

##  6 Limitations

This study has several limitations that warrant consideration. Our experiments
are based on WebText. While this corpus provides a broad range of topics and
styles, it is possible that the derived insights, such as the general
reduction in stereotypical bias and the unequal reduction across social
groups, are influenced by spurious correlations (Schwartz and Stanovsky, 2022)
inherent in the dataset. In addition to the flaws caused by the training
corpus, our reliance on GloVe embeddings for text privatization introduces
another potential source of inherent biases. Future research should address
these limitations by incorporating a more diverse set of datasets and explore
how alternative embeddings affect the persistence of stereotypical bias after
privatization.

## References

  * Abadi et al. (2016) Martin Abadi, Andy Chu, Ian Goodfellow, H Brendan McMahan, Ilya Mironov, Kunal Talwar, and Li Zhang. 2016.  Deep learning with differential privacy.  In _Proceedings of the 2016 ACM SIGSAC conference on computer and communications security_ , pages 308–318.
  * Abid et al. (2021) Abubakar Abid, Maheen Farooqi, and James Zou. 2021.  Persistent anti-muslim bias in large language models.  In _Proceedings of the 2021 AAAI/ACM Conference on AI, Ethics, and Society_ , pages 298–306.
  * Arnold et al. (2023) Stefan Arnold, Dilara Yesilbas, and Sven Weinzierl. 2023.  Driving context into text-to-text privatization.  In _Proceedings of the 3rd Workshop on Trustworthy Natural Language Processing (TrustNLP 2023)_ , pages 15–25, Toronto, Canada. Association for Computational Linguistics.
  * Bagdasaryan et al. (2019) Eugene Bagdasaryan, Omid Poursaeed, and Vitaly Shmatikov. 2019.  Differential privacy has disparate impact on model accuracy.  _Advances in neural information processing systems_ , 32.
  * Bindschaedler et al. (2017) Vincent Bindschaedler, Reza Shokri, and Carl A Gunter. 2017.  Plausible deniability for privacy-preserving data synthesis.  _arXiv preprint arXiv:1708.07975_.
  * Bo et al. (2021) Haohan Bo, Steven H. H. Ding, Benjamin C. M. Fung, and Farkhund Iqbal. 2021.  ER-AE: Differentially private text generation for authorship anonymization.  In _Proceedings of the 2021 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies_ , pages 3997–4007, Online. Association for Computational Linguistics.
  * Bolukbasi et al. (2016) Tolga Bolukbasi, Kai-Wei Chang, James Y Zou, Venkatesh Saligrama, and Adam T Kalai. 2016.  Man is to computer programmer as woman is to homemaker? debiasing word embeddings.  _Advances in neural information processing systems_ , 29.
  * Caliskan et al. (2017) Aylin Caliskan, Joanna J Bryson, and Arvind Narayanan. 2017.  Semantics derived automatically from language corpora contain human-like biases.  _Science_ , 356(6334):183–186.
  * Carlini et al. (2019) Nicholas Carlini, Chang Liu, Úlfar Erlingsson, Jernej Kos, and Dawn Song. 2019.  The secret sharer: Evaluating and testing unintended memorization in neural networks.  In _28th USENIX Security Symposium (USENIX Security 19)_ , pages 267–284.
  * Chaloner and Maldonado (2019) Kaytlin Chaloner and Alfredo Maldonado. 2019.  Measuring gender bias in word embeddings across domains and discovering new gender bias word categories.  In _Proceedings of the First Workshop on Gender Bias in Natural Language Processing_ , pages 25–32, Florence, Italy. Association for Computational Linguistics.
  * Chatzikokolakis et al. (2013) Konstantinos Chatzikokolakis, Miguel E Andrés, Nicolás Emilio Bordenabe, and Catuscia Palamidessi. 2013.  Broadening the scope of differential privacy using metrics.  In _International Symposium on Privacy Enhancing Technologies Symposium_ , pages 82–102. Springer.
  * Chen et al. (2023) Sai Chen, Fengran Mo, Yanhao Wang, Cen Chen, Jian-Yun Nie, Chengyu Wang, and Jamie Cui. 2023.  A customized text sanitization mechanism with differential privacy.  In _Findings of the Association for Computational Linguistics: ACL 2023_ , pages 5747–5758, Toronto, Canada. Association for Computational Linguistics.
  * Desai and Durrett (2020) Shrey Desai and Greg Durrett. 2020.  Calibration of pre-trained transformers.  In _Proceedings of the 2020 Conference on Empirical Methods in Natural Language Processing (EMNLP)_ , pages 295–302, Online. Association for Computational Linguistics.
  * Devlin et al. (2019) Jacob Devlin, Ming-Wei Chang, Kenton Lee, and Kristina Toutanova. 2019.  BERT: Pre-training of deep bidirectional transformers for language understanding.  In _Proceedings of the 2019 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, Volume 1 (Long and Short Papers)_ , pages 4171–4186, Minneapolis, Minnesota. Association for Computational Linguistics.
  * Dhamala et al. (2021) Jwala Dhamala, Tony Sun, Varun Kumar, Satyapriya Krishna, Yada Pruksachatkun, Kai-Wei Chang, and Rahul Gupta. 2021.  Bold: Dataset and metrics for measuring biases in open-ended language generation.  In _Proceedings of the 2021 ACM conference on fairness, accountability, and transparency_ , pages 862–872.
  * Du et al. (2023) Minxin Du, Xiang Yue, Sherman SM Chow, Tianhao Wang, Chenyu Huang, and Huan Sun. 2023.  Dp-forward: Fine-tuning and inference on language models with differential privacy in forward pass.  In _Proceedings of the 2023 ACM SIGSAC Conference on Computer and Communications Security_ , pages 2665–2679.
  * Dwork et al. (2006) Cynthia Dwork, Frank McSherry, Kobbi Nissim, and Adam Smith. 2006.  Calibrating noise to sensitivity in private data analysis.  In _Theory of cryptography conference_ , pages 265–284. Springer.
  * Farrand et al. (2020) Tom Farrand, Fatemehsadat Mireshghallah, Sahib Singh, and Andrew Trask. 2020.  Neither private nor fair: Impact of data imbalance on utility and fairness in differential privacy.  In _Proceedings of the 2020 workshop on privacy-preserving machine learning in practice_ , pages 15–19.
  * Fernandes et al. (2019) Natasha Fernandes, Mark Dras, and Annabelle McIver. 2019.  Generalised differential privacy for text document processing.  In _International Conference on Principles of Security and Trust_ , pages 123–148. Springer, Cham.
  * Feyisetan et al. (2020) Oluwaseyi Feyisetan, Borja Balle, Thomas Drake, and Tom Diethe. 2020.  Privacy-and utility-preserving textual analysis via calibrated multivariate perturbations.  In _Proceedings of the 13th International Conference on Web Search and Data Mining_ , pages 178–186.
  * Feyisetan et al. (2019) Oluwaseyi Feyisetan, Tom Diethe, and Thomas Drake. 2019.  Leveraging hierarchical representations for preserving privacy and utility in text.  In _2019 IEEE International Conference on Data Mining (ICDM)_ , pages 210–219. IEEE.
  * Garg et al. (2018) Nikhil Garg, Londa Schiebinger, Dan Jurafsky, and James Zou. 2018.  Word embeddings quantify 100 years of gender and ethnic stereotypes.  _Proceedings of the National Academy of Sciences_ , 115(16):E3635–E3644.
  * Hansen et al. (2022) Victor Petren Bach Hansen, Atula Tejaswi Neerkaje, Ramit Sawhney, Lucie Flek, and Anders Sogaard. 2022.  The impact of differential privacy on group disparity mitigation.  In _Proceedings of the Fourth Workshop on Privacy in Natural Language Processing_ , pages 12–12, Seattle, United States. Association for Computational Linguistics.
  * Igamberdiev et al. (2022) Timour Igamberdiev, Thomas Arnold, and Ivan Habernal. 2022.  DP-rewrite: Towards reproducibility and transparency in differentially private text rewriting.  In _Proceedings of the 29th International Conference on Computational Linguistics_ , pages 2927–2933, Gyeongju, Republic of Korea. International Committee on Computational Linguistics.
  * Igamberdiev and Habernal (2023) Timour Igamberdiev and Ivan Habernal. 2023.  DP-BART for privatized text rewriting under local differential privacy.  In _Findings of the Association for Computational Linguistics: ACL 2023_ , pages 13914–13934, Toronto, Canada. Association for Computational Linguistics.
  * Jayaraman and Evans (2019) Bargav Jayaraman and David Evans. 2019.  Evaluating differentially private machine learning in practice.  In _28th USENIX Security Symposium (USENIX Security 19)_ , pages 1895–1912.
  * Krishna et al. (2021) Satyapriya Krishna, Rahul Gupta, and Christophe Dupuy. 2021.  ADePT: Auto-encoder based differentially private text transformation.  In _Proceedings of the 16th Conference of the European Chapter of the Association for Computational Linguistics: Main Volume_ , pages 2435–2439, Online. Association for Computational Linguistics.
  * Kurita et al. (2019) Keita Kurita, Nidhi Vyas, Ayush Pareek, Alan W Black, and Yulia Tsvetkov. 2019.  Measuring bias in contextualized word representations.  In _Proceedings of the First Workshop on Gender Bias in Natural Language Processing_ , pages 166–172, Florence, Italy. Association for Computational Linguistics.
  * Manzini et al. (2019) Thomas Manzini, Yao Chong Lim, Yulia Tsvetkov, and Alan W Black. 2019.  Black is to criminal as caucasian is to police: Detecting and removing multiclass bias in word embeddings.  _arXiv preprint arXiv:1904.04047_.
  * Mattern et al. (2022) Justus Mattern, Benjamin Weggenmann, and Florian Kerschbaum. 2022.  The limits of word level differential privacy.  _arXiv preprint arXiv:2205.02130_.
  * May et al. (2019) Chandler May, Alex Wang, Shikha Bordia, Samuel R. Bowman, and Rachel Rudinger. 2019.  On measuring social biases in sentence encoders.  In _Proceedings of the 2019 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, Volume 1 (Long and Short Papers)_ , pages 622–628, Minneapolis, Minnesota. Association for Computational Linguistics.
  * McCann et al. (2017) Bryan McCann, James Bradbury, Caiming Xiong, and Richard Socher. 2017.  Learned in translation: Contextualized word vectors.  _Advances in neural information processing systems_ , 30.
  * Merity et al. (2016) Stephen Merity, Caiming Xiong, James Bradbury, and Richard Socher. 2016.  Pointer sentinel mixture models.  _arXiv preprint arXiv:1609.07843_.
  * Mikolov et al. (2013) Tomas Mikolov, Kai Chen, Greg Corrado, and Jeffrey Dean. 2013.  Efficient estimation of word representations in vector space.  _arXiv preprint arXiv:1301.3781_.
  * Mitchell (1980) Tom M Mitchell. 1980.  The need for biases in learning generalizations.
  * Nadeem et al. (2021) Moin Nadeem, Anna Bethke, and Siva Reddy. 2021.  StereoSet: Measuring stereotypical bias in pretrained language models.  In _Proceedings of the 59th Annual Meeting of the Association for Computational Linguistics and the 11th International Joint Conference on Natural Language Processing (Volume 1: Long Papers)_ , pages 5356–5371, Online. Association for Computational Linguistics.
  * Nangia et al. (2020) Nikita Nangia, Clara Vania, Rasika Bhalerao, and Samuel R. Bowman. 2020.  CrowS-pairs: A challenge dataset for measuring social biases in masked language models.  In _Proceedings of the 2020 Conference on Empirical Methods in Natural Language Processing (EMNLP)_ , pages 1953–1967, Online. Association for Computational Linguistics.
  * Nasr et al. (2023) Milad Nasr, Nicholas Carlini, Jonathan Hayase, Matthew Jagielski, A Feder Cooper, Daphne Ippolito, Christopher A Choquette-Choo, Eric Wallace, Florian Tramèr, and Katherine Lee. 2023.  Scalable extraction of training data from (production) language models.  _arXiv preprint arXiv:2311.17035_.
  * Papakyriakopoulos et al. (2020) Orestis Papakyriakopoulos, Simon Hegelich, Juan Carlos Medina Serrano, and Fabienne Marco. 2020.  Bias in word embeddings.  In _Proceedings of the 2020 conference on fairness, accountability, and transparency_ , pages 446–457.
  * Pennington et al. (2014) Jeffrey Pennington, Richard Socher, and Christopher D Manning. 2014.  Glove: Global vectors for word representation.  In _Proceedings of the 2014 conference on empirical methods in natural language processing (EMNLP)_ , pages 1532–1543.
  * Qu et al. (2021) Chen Qu, Weize Kong, Liu Yang, Mingyang Zhang, Michael Bendersky, and Marc Najork. 2021.  Natural language understanding with privacy-preserving bert.  In _Proceedings of the 30th ACM International Conference on Information & Knowledge Management_, pages 1488–1497.
  * Radford et al. (2019) Alec Radford, Jeffrey Wu, Rewon Child, David Luan, Dario Amodei, Ilya Sutskever, et al. 2019.  Language models are unsupervised multitask learners.  _OpenAI blog_ , 1(8):9.
  * Salazar et al. (2020) Julian Salazar, Davis Liang, Toan Q. Nguyen, and Katrin Kirchhoff. 2020.  Masked language model scoring.  In _Proceedings of the 58th Annual Meeting of the Association for Computational Linguistics_ , pages 2699–2712, Online. Association for Computational Linguistics.
  * Schwartz and Stanovsky (2022) Roy Schwartz and Gabriel Stanovsky. 2022.  On the limitations of dataset balancing: The lost battle against spurious correlations.  In _Findings of the Association for Computational Linguistics: NAACL 2022_ , pages 2182–2194, Seattle, United States. Association for Computational Linguistics.
  * Shi et al. (2022) Weiyan Shi, Aiqi Cui, Evan Li, Ruoxi Jia, and Zhou Yu. 2022.  Selective differential privacy for language modeling.  In _Proceedings of the 2022 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies_ , pages 2848–2859, Seattle, United States. Association for Computational Linguistics.
  * Song and Raghunathan (2020) Congzheng Song and Ananth Raghunathan. 2020.  Information leakage in embedding models.  In _Proceedings of the 2020 ACM SIGSAC Conference on Computer and Communications Security_ , pages 377–390.
  * Thomas et al. (2020) Aleena Thomas, David Ifeoluwa Adelani, Ali Davody, Aditya Mogadala, and Dietrich Klakow. 2020.  Investigating the impact of pre-trained word embeddings on memorization in neural networks.  In _International Conference on Text, Speech, and Dialogue_ , pages 273–281. Springer.
  * Vaswani et al. (2017) Ashish Vaswani, Noam Shazeer, Niki Parmar, Jakob Uszkoreit, Llion Jones, Aidan N Gomez, Łukasz Kaiser, and Illia Polosukhin. 2017.  Attention is all you need.  _Advances in neural information processing systems_ , 30.
  * Warner (1965) Stanley L Warner. 1965.  Randomized response: A survey technique for eliminating evasive answer bias.  _Journal of the American Statistical Association_ , 60(309):63–69.
  * Weggenmann et al. (2022) Benjamin Weggenmann, Valentin Rublack, Michael Andrejczuk, Justus Mattern, and Florian Kerschbaum. 2022.  Dp-vae: Human-readable text anonymization for online reviews with differentially private variational autoencoders.  In _Proceedings of the ACM Web Conference 2022_ , WWW ’22, page 721–731, New York, NY, USA. Association for Computing Machinery.
  * Xu et al. (2020) Zekun Xu, Abhinav Aggarwal, Oluwaseyi Feyisetan, and Nathanael Teissier. 2020.  A differentially private text perturbation method using a regularized mahalanobis metric.  _arXiv preprint arXiv:2010.11947_.
  * Yue et al. (2021) Xiang Yue, Minxin Du, Tianhao Wang, Yaliang Li, Huan Sun, and Sherman S. M. Chow. 2021.  Differential privacy for text analytics via natural text sanitization.  In _Findings of the Association for Computational Linguistics: ACL-IJCNLP 2021_ , pages 3853–3866, Online. Association for Computational Linguistics.