CPHT-RR108.122019

# Diluting SUSY flavour problem on the Landscape

Emilian Dudas  CPHT, Ecole Polytechnique, 91128 Palaiseau cedex, France
Priyanka Lamba  Centre for High Energy Physics, Indian Institute of Science,
Bangalore- 560012, India  Sudhir K Vempati  Centre for High Energy Physics,
Indian Institute of Science, Bangalore- 560012, India

###### Abstract

We consider an explicit effective field theory example based on the Bousso-
Polchinski framework with a large number N of hidden sectors contributing to
supersymmetry breaking. Each contribution comes from four form quantized
fluxes, multiplied by random couplings. The soft terms in the observable
sector in this case become random variables, with mean values and standard
deviations which are computable. We show that this setup naturally leads to a
solution of the flavor problem in low-energy supersymmetry if N is
sufficiently large. We investigate the consequences for flavor violating
processes at low-energy and for dark matter.

##  I Introduction and Motivation

Supersymmetry breaking in MSSM (Minimal Supersymmetric Standard Model) is
introduced in terms of explicit soft breaking terms. These are large in number
$\sim$ 105, most of which violate flavor and CP symmetries. Phenomenologically
there are strong constraints on the flavor off-diagonal entries, requiring
them to be suppressed (compared to the flavour diagonal ones) by one to
several orders of magnitude. The flavor violation constraints on the first two
generations are significantly stronger compared to the ones involving the
third generation. These bounds are well documented in the literature
Gabbiani:1996hi  (for reviews, see Dimopoulos:1995ju ; Misiak:1997ei ;
Masiero:2005ua ).

To solve the problem with flavour violating soft terms, several solutions have
been proposed. If supersymmetry breaking is mediated purely in terms of gauge
interactions, the resulting soft terms would not contain any flavor violation
Dine:1993yw ; Dine:1994vc ; Dine:1995ag ; Giudice:1998bp . However, the
discovery of the Higgs boson puts constraints on such models. If the Higgs
boson is of supersymmetric origin, one would expect the mass of the lightest
CP even Higgs boson in Minimal Supersymmetric Standard Model (MSSM) to be
rather light. This already puts severe constraints on the supersymmetric
parameter space, in particular that of the third generation up-type squarks
(the stops): they are either required to mix almost maximally or to be heavy,
between 3 to 4 TeV (See for example, Bagnaschi:2018igf ). Several
supersymmetry breaking models like minimal gauge mediation and its variations
are disfavoured in the light of the Higgs discovery Draper:2011aa  or require
a rather heavy spectra in the range of multi-TeV Bagnaschi:2018igf .

In string or supergravity based models, it has been long known that it in
general scenarios it is hard to escape flavour violation, unless some specific
conditions are chosen Brignole:1993dj ; Brignole:1997dp . For example, if the
Kähler potential of the matter fields is canonical and independent of the
moduli/hidden sector fields, one could expect an universal, flavor-independent
form for the soft terms as in minimal Supergravity. On the other hand, the
problem can also be avoided if supersymmetry breaking is dominantly dilaton
mediated Kaplunovsky:1993rd . Other solutions include decoupling of the first
two generations Cohen:1996vb  or imposing flavour symmetries (See for example,
Antusch:2008jf ; Pomarol:1995xc ; Dine:1993np ; Dudas:1995eq  and references
there in).

In the present letter, we would like to address these issues from a different
point of view, insipred by the landscape of string theory vacua. We will
consider a large number $N$ of sectors contributing to supersymmetry breaking.
Large number of sequestered hidden sectors have also been considered recently
in Cheung:2010mc ; Cheung:2010qf ; Benakli:2007zza , in models with multiple
(pseudo)goldstini. Other works which have addressed supersymmetric soft
spectrum phenomenology from the landscape following Denef:2004ze  include
Baer:2019tee ; Baer:2019xww ; Baer:2017uvn ; Baer:2017cck . In particular a
solution to flavour and CP problems in the landscape through heavy first two
generations was proposed in Baer:2019zfl .

In the present work, we consider quantized four form fluxes a la Bousso-
Polchinski Bousso:2000xa . Each sector contributes in a quantized way, with a
quantum that will be taken to be below the electroweak scale. Due to the large
number of contributions, the observable soft terms become random variables
with Normal-type distributions around an average value. The setup has also the
virtue of minimizing fine-tuning of the electroweak scale, due to the small
contributions of each sector. We find that our setup can address at the same
time the flavor problem of low-energy SUSY by generating FCNC effects
proportional to standard deviation of soft terms from their mean value, which
are parametrically suppressed as $1/{\sqrt{N}}$. By performing a RG analysis
from high to low-energy, the setup makes also concrete predictions for low-
energy flavor observables.

The letter is organized as follows. In Section 2, we review the Bousso-
Polchinski setup and follow it in Section 3 with a review of four-forms
fluxes. In the same section, we derive the soft terms and also show the impact
on the flavour violating soft terms in the limit of large $N$. We also set up
the boundary conditions for the scanning. In Section 4 we discuss the
numerical results and present constraints from $K^{0}-\bar{K}^{0}$
oscillations and $\mu\to e+\gamma$. The framework also has interesting
implications for dark matter which we discuss briefly at the end of the
section. We end with a small section of conclusions and outlook.

##  II The Bousso-Polchinski setup

The logic and setup we put forward is mainly originating from Bousso and
Polchinski approach to the cosmological constant Bousso:2000xa . We are
regarding the implications of the string theory landscape for observable
sector soft SUSY breaking terms. We will assume a large $N\gg 1$ number of
SUSY-breaking sectors communicating through gravitational couplings to the
Supersymmetric Standard Model (SSM). Such models could naturally appear in
string theory, where there may be several independent sources of supersymmetry
breaking.

Higher-dimensional operators and gravitational interactions lead to
interactions between the moduli and the visible sector. One writes an
effective action for the visible sector fields at a high scale, treating the
hidden sector fields as non-dynamical background fields. This is justified if
they are very heavy compared to the observable fields. One then writes a set
of renormalization group equations for the higher-dimensional operators and
evolves them into the infrared, ignoring the hidden sector dynamics.
Supersymmetry breaking F and D components of the background hidden sector
fields then give rise to visible sector soft supersymmetry breaking
parameters.
Usually SUSY breaking is parametrized in terms of a single hidden sector
field. This gives to a spectra at high scale (up to O(1) parameters) in terms
of the scale of SUSY breaking Fα. In our case, we consider instead MSSM
interacting with N hidden sectors at the Planck scale. In the models we are
considering, auxiliary fields of the hidden sector fields contain quantized
four-form fluxes, with discrete charges contributing to supersymmetry
breaking, as in the Bousso-Polchinski solution to the cosmological constant
problem.

The main features of our framework are:

  * •

Integer quanta parameterising the soft supersymmetry breaking contribution
from each hid- den sector

  * •

Minimal number of parameters representing coupling between hidden sector
fields and MSSM. Since these couplings depend on moduli vev’s and
interactions, we parametrize them by random continuous parameters taking
values inside a compact interval around zero.

  * •

Assume gravity mediation for simplicity. A similar scan can be done for gauge
mediation, although the details will be quantitatively different.

  * •

We consider a flat probability distribution of flux in each hidden sector.
Since each flux is a random variable, due to the central limit theorem, this
will lead to Normal-type distributions for the soft terms.

While we will impose the cancellation of the cosmological constant à la
Bousso-Polchinski, we do not necessarily use our framework to address the
cosmological constant problem. Instead we use the framework as a network of
hidden sectors each contributing individually to soft supersymmetry breaking.

##  III Four-forms and fluxes

Three form gauge potentials with (non-dynamical) four-form field strengths
were considered longtime ago for addressing the cosmological constant problem
Aurilia:1980xj ; Witten:1983ux ; Henneaux:1984ji ; Brown:1987dd ; Brown:1988kg
; Duff:1989ah ; Feng:2000if ; dvali:2001sm , the gauge hierarchy problem
Dvali:2003br ; Dvali:2004tma ; Herraez:2016dxn ; Giudice:2019iwl (see also
Lee:2019efp ), the strong CP problem Dvali:2005an ; Dvali:2005zk , inflation
Kaloper:2008fb ; Kaloper:2011jz ; Kaloper:2014zba ; Dudas:2014pva  and
supersymmetry breaking Farakos:2016hly . On the other hand, it turned out to
play an important role in the landscape of string theory compactifications
Farakos:2017jme ; Herraez:2018vae ; Lanza:2019xxg  (for a recent review see
e.g. Lanza:2019nfa ). Here we briefly review the main points of a theory
containing three-forms with quantized form-forms field strengths.

Let us start from a lagrangian containing some scalar fields $\varphi_{i}$ and
three-form fields $C_{mnp}^{\alpha}$, with the action

${\cal S}_{0}=\int d^{4}x\
\\{-\frac{1}{2}(\partial\varphi_{i})^{2}-\Lambda_{0}-\frac{1}{2\times
4!}F_{mnpq}^{\alpha,2}+\frac{1}{24}\ f_{\alpha}(\varphi_{i})\
\epsilon^{mnpq}F_{mnpq}^{\alpha}\\}\ ,$ (1)

where

$F_{mnpq}^{\alpha}=\partial_{m}C_{npq}^{\alpha}+{\rm 3\ perms.}\ .$ (2)

For future convenience we define

$F^{\alpha}=\frac{1}{4!}\epsilon^{mnpq}F_{mnpq}^{\alpha}\ ,\
F_{mnpq}^{\alpha}=-\epsilon_{mnpq}F^{\alpha}\ .$ (3)

The lagrangian (1) has actually to be supplemented with a boundary term

${\cal S}_{b}=\frac{1}{6}\int d^{4}x\
\partial_{m}\left(F^{mnpq}_{\alpha}C_{npq}^{\alpha}-f_{\alpha}(\varphi_{i})\epsilon^{mnpq}C_{npq}^{\alpha}\right)\
.$ (4)

The total action is

$\displaystyle{\cal S}={\cal S}_{0}+{\cal S}_{b}=\int d^{4}x\
\\{-\frac{1}{2}(\partial\varphi_{i})^{2}-\Lambda_{0}-\frac{1}{2\times
4!}F_{mnpq}^{\alpha,2}-\frac{1}{6}\
\epsilon^{mnpq}\partial_{m}f_{\alpha}(\varphi_{i})\ C_{npq}^{\alpha}\\}$
$\displaystyle+\frac{1}{6}\int d^{4}x\
\partial_{m}\left(F^{mnpq}_{\alpha}C_{npq}^{\alpha}\right)\ .$ (5)

A massless three-form gauge field in four spacetime dimensions has no on-shell
degrees of freedom. As such, it can be integrated out via its field eqs.

$\partial^{m}F_{mnpq}^{\alpha}\ =\ \
\epsilon_{mnpq}\partial^{m}f_{\alpha}(\varphi_{i})\ ,$ (6)

whose solution is given by

$F_{\alpha}=-f_{\alpha}(\varphi_{i})+c_{\alpha}\ ,$ (7)

where $c_{\alpha}$ is a constant, which is to be interpreted as a flux. It was
argued in Bousso:2000xa  that $c_{\alpha}$ are quantized in units of the
fundamental membrane coupling $c_{\alpha}=m_{\alpha}e$, fact that has
important consequences for the landscape of string theory. After doing so, the
final lagrangian takes the form

${\cal S}=\int d^{4}x\
\\{-\frac{1}{2}(\partial\varphi_{i})^{2}-\Lambda_{0}-\frac{1}{2}\sum_{\alpha}(f_{\alpha}(\varphi_{i})-c_{\alpha})^{2}\\}\
.$ (8)

The final resulting cosmological constant is therefore scanned by the flux

$\Lambda=\Lambda_{0}+\frac{1}{2}\sum_{\alpha}(f_{\alpha}(\varphi_{i})-c_{\alpha})^{2}\
.$ (9)

Notice that the boundary term ${\cal S}_{b}$ is crucial in obtaining the
correct action. Ignoring it leads to the wrong sign of the last term in (8),
fact that created confusion in the past.

###  III.1 Supersymmetric formulation

The embedding of four-form fluxes in supersymmetry and supergravity proceeds
by introducing three-form multiplets, defined as the real superfields
Gates:1980ay ; Burgess:1995kp ; Burgess:1995aa ; Binetruy:1995hq ;
Binetruy:1996xw ; Binetruy:2000zx ; Groh:2012tf

$\displaystyle
U_{\alpha}={\bar{U}}_{\alpha}=B_{\alpha}+i(\theta\chi_{\alpha}-{\bar{\theta}}{\bar{\chi}}_{\alpha})+\theta^{2}{\bar{M}}_{\alpha}+{\bar{\theta}}^{2}{M_{\alpha}}+\frac{1}{3}\theta\sigma^{m}{\bar{\theta}}\epsilon_{mnpq}C^{npq}_{\alpha}+$
$\displaystyle\theta^{2}{\bar{\theta}}(\sqrt{2}{\bar{\lambda}}_{\alpha}+\frac{1}{2}{\bar{\sigma}}^{m}\partial_{m}\chi_{\alpha})+{\bar{\theta}}^{2}\theta(\sqrt{2}{\lambda}_{\alpha}-\frac{1}{2}{\sigma}^{m}\partial_{m}{\bar{\chi}}_{\alpha})+\theta^{2}{\bar{\theta}}^{2}(D_{\alpha}-\frac{1}{4}\Box
B_{\alpha})\ .$ (10)

The difference between $U_{\alpha}$ and a regular vector superfield $V$ is the
replacement of the vector potential $V_{m}$ by a three-form
$C^{npq}_{\alpha}$. In order to find correct kinetic terms, the analog of the
chiral field strength superfield $W_{\alpha}$ for a vector multiplet is
replaced by the chiral superfield Gates:1980ay

$T_{\alpha}=-\frac{1}{4}{\bar{D}}^{2}U_{\alpha}\quad,\quad
T_{\alpha}(y^{m},\theta)=M_{\alpha}+\sqrt{2}\theta\lambda_{\alpha}+\theta^{2}(D_{\alpha}+iF_{\alpha})\
,$ (11)

with $F_{\alpha}$ defined as in (3). The definition (11) is invariant under
the gauge transformation $U_{\alpha}\to U_{\alpha}-L_{\alpha}$, where
$L_{\alpha}$ are linear multiplets. Correspondingly, lagrangians expressed as
a function of $T_{\alpha}$ will have this gauge freedom. One can therefore
choose a gauge in which $B_{\alpha}=\chi_{\alpha}=0$ in (10) and the physical
fields are complex scalars $M_{\alpha}$ and Weyl fermions $\lambda_{\alpha}$.

Notice that for the purpose of finding the correct on-shell lagrangian and
scalar potential, there is a simpler formulation in which $T_{\alpha}$ are
treated as standard chiral superfields with $D_{\alpha}+iF_{\alpha}$ as
auxiliary fields, no boundary terms are included, but the superpotential of
the theory is changed according to Binetruy:1996xw ; Binetruy:2000zx ;
Groh:2012tf

$W(\phi_{i},T_{\alpha})\to W^{\prime}(\phi_{i},T_{\alpha})\ =\
W(\phi_{i},S_{\alpha})+ic_{\alpha}T_{\alpha}\ ,$ (12)

where $c_{\alpha}$ are the quantized fluxes. The linear terms in the
superpotential shift linearly the auxiliary fields. In supergravity, the
(F-term) scalar potential can be written as ($M_{P}=1$ in what follows)

$V=K_{\alpha\bar{\beta}}F^{\alpha}F^{\bar{\beta}}-3m_{3/2}^{2}\quad,\quad{\rm
where}\quad
F^{\alpha}=e^{\frac{K}{2}}K^{\alpha\bar{\beta}}\overline{D_{\beta}W}\ .$ (13)

The linear flux terms shift therefore the auxiliary fields according to

$F^{\alpha}=e^{\frac{K}{2}}K^{\alpha\bar{\beta}}\overline{D_{\beta}W^{\prime}}=e^{\frac{K}{2}}K^{\alpha\bar{\beta}}\overline{D_{\beta}W}-ie^{\frac{K}{2}}K^{\alpha\bar{\gamma}}({\bar{c}}_{\bar{\gamma}}+K_{\bar{\gamma}}{\bar{T}}_{\bar{\beta}}{\bar{c}}_{\bar{\beta}})\
,$ (14)

leading to a scanning of the cosmological constant.

###  III.2 Soft terms in supergravity

We start from a supergravity lagrangian containing hidden sector (moduli)
fields $T_{\alpha}$, whose auxiliary fields contain the four-form fluxes we
introduced previously, coupled to matter fields called $Q_{i}$ in what
follows. The Kahler potential and superpotential are defined by

$\displaystyle
K={\hat{K}}(T_{\alpha},{\bar{T}}_{\alpha})+K_{i\bar{j}}(T_{\alpha},{\bar{T}}_{\alpha})Q^{i}{\bar{Q}}^{\bar{j}}+\frac{1}{2}\left(Z_{ij}(T_{\alpha},{\bar{T}}_{\alpha})Q^{i}{Q}^{j}+{\rm
h.c.}\right)\ ,$ $\displaystyle
W={\hat{W}}(T_{\alpha})+\frac{1}{2}{\tilde{\mu}}_{ij}(T_{\alpha})Q^{i}{Q}^{j}+\frac{1}{3}{\tilde{Y}}_{ijk}(T_{\alpha})Q^{i}{Q}^{j}Q^{k}+\cdots\
.$ (15)

The low-energy softly broken supersymmetric lagrangian is defined by the
superpotential and soft scalar potential

$\displaystyle W_{\rm
eff}=\frac{1}{2}{\mu}_{ij}Q^{i}{Q}^{j}+\frac{1}{3}{Y}_{ijk}Q^{i}{Q}^{j}Q^{k}\
,$ $\displaystyle{\cal L}_{\rm
soft}=-m_{i\bar{j}}^{2}q^{i}q^{\bar{j}}-\left(\frac{1}{2}{B}_{ij}q^{i}{q}^{j}+\frac{1}{3}{A}_{ijk}q^{i}{q}^{j}q^{k}+\frac{1}{2}M_{a}\lambda_{a}\lambda_{a}+{\rm
h.c.}\right)\ ,\ $ (16)

where $Y_{ijk}=e^{K/2}{\tilde{Y}}_{ijk}$. After imposing the cancellation of
the cosmological constant, the various soft terms and the supersymmetric
masses are given by Kaplunovsky:1993rd ; Brignole:1993dj ; Brignole:1997dp ;
Ferrara:1994kg ; Dudas:2005vv

$\displaystyle M_{a}=\frac{1}{2}g_{a}^{2}F^{\alpha}\partial_{\alpha}f_{a}\ ,$
$\displaystyle
m_{i\bar{j}}^{2}=m_{3/2}^{2}K_{i\bar{j}}-F^{\alpha}F^{\bar{\beta}}R_{i\bar{j}\alpha\bar{\beta}}\quad,\quad{\rm
where}\quad
R_{i\bar{j}\alpha\bar{\beta}}=\partial_{\alpha}\partial_{\bar{\beta}}K_{i\bar{j}}-K^{m\bar{n}}\partial_{\alpha}K_{i\bar{n}}\partial_{\bar{\beta}}K_{m\bar{j}}\
,$ $\displaystyle A_{ijk}=(m_{3/2}-F^{\alpha}\partial_{\alpha}\log
m_{3/2})Y_{ijk}+F^{\alpha}\partial_{\alpha}Y_{ijk}-3F^{\alpha}\Gamma_{\alpha(i}^{l}Y_{ljk)}\
,$ (17)

where we have introduced also the Kahler connexion

$\Gamma_{IJ}^{K}=K^{K\bar{L}}\partial_{I}K_{J\bar{L}}\ .$ (18)

$B_{ij}$ terms are not displayed since they will not be scanned in what
follows. Similarly the $\mu$ term is determined by the radiative electroweak
symmetry breaking conditions at the weak scale.

###  III.3 Scanning soft terms and gravitino mass

Taking into account the scanning of auxiliary fields from four-forms fluxes,
in what follows we use the simplified scanning

$F_{\alpha}=m_{\alpha}{\tilde{m}}M_{P}\ ,$ (19)

where $m_{\alpha}$ are integers and $\tilde{m}M_{P}$ is the quantum of
scanning. Taking into account the cancellation of the cosmological constant,
and setting the matter fields wavefunctions in a canonical form, the formulae
for the scanning will be taken to be

$\displaystyle
m_{3/2}=\tilde{m}(g_{0}+\sum_{\alpha}g_{\alpha}m_{\alpha})\quad,\quad
m_{3/2}^{2}=\frac{1}{3}\sum_{\alpha=1}^{N}\frac{F_{\alpha}^{2}}{M_{P}^{2}}=\frac{1}{3}{\tilde{m}}^{2}\sum_{\alpha}{m_{\alpha}^{2}}\
,\ $
$\displaystyle(m_{0}^{2})_{i\bar{j}}=m_{3/2}^{2}\delta_{i\bar{j}}+{\tilde{m}}^{2}\sum_{\alpha}d_{\alpha,i\bar{j}}m_{\alpha}^{2}\
,\ $ $\displaystyle
M_{1/2}^{a}={\tilde{m}}\sum_{\alpha}s_{\alpha}^{a}m_{\alpha}\ ,\ $
$\displaystyle
A_{ijk}=m_{3/2}y_{ijk}+{\tilde{m}}\sum_{\alpha}a_{\alpha,ijk}m_{\alpha}\ ,\ $
(20)

where $\alpha=1\cdots N$, $-M\leq m_{\alpha}\leq M$ (integers), $-d_{0}\leq
a_{\alpha},d_{\alpha},s_{\alpha},g_{\alpha}\leq d_{0}$ (continuous). These
last couplings are taking to be continuous in order to take into account the
couplings of the hidden sector fields with the MSSM ones, which are dependent
on the hidden sector vev’s and interactions. Note that the soft terms defined
above are in the so-called super CKM basis which is important for the flavor
discussion below.

The scanning of the gravitino mass, combined with the cancellation of the
cosmological constant (the Deser-Zumino relation) implies a constrained among
the fluxes

$g_{0}^{2}+2g_{0}\sum_{\alpha}g_{\alpha}m_{\alpha}+\sum_{\alpha,\beta}g_{\alpha}g_{\beta}m_{\alpha}m_{\beta}=\frac{1}{3}\sum_{\alpha}m_{\alpha}^{2}\
.$ (21)

Taking the average value of (21) this implies in particular

$g_{0}^{2}=\sum_{\alpha}\overline{(1/3-g_{\alpha}^{2})m_{\alpha}^{2}}\simeq\frac{N}{9}(1-d_{0}^{2})M^{2}\quad,\quad\overline{m_{3/2}}={\tilde{m}}g_{0}\sim
O(\sqrt{N}){\tilde{m}}\ .$ (22)

where we have used the large flux limit $M\gg 1$. We can also compute

$\overline{m_{3/2}^{2}}=\frac{\tilde{m}^{2}}{3}\sum_{\alpha}\overline{m_{\alpha}^{2}}\sim\frac{1}{9}NM^{2}{\tilde{m}^{2}}\
,$ (23)

The mean values of the soft terms are therefore computed to be

$\overline{(m_{0}^{2})_{i\bar{j}}}=\overline{m_{3/2}^{2}}\delta_{i\bar{j}},\quad\quad\overline{A_{ijk}}=\overline{m_{3/2}}y_{ijk},\quad\quad\overline{M_{1/2}^{a}}=0\
.$ (24)

There are two type of averages : one over the flux quanta $m_{\alpha}$ and the
other over the (continuous) couplings $d_{\alpha}$. Being independent
variables, one can use formulae of the type

$\overline{f_{1}(d_{\alpha})f_{2}(m_{\alpha})}=\overline{f_{1}(d_{\alpha})}\times\overline{f_{2}(m_{\alpha})}=\frac{1}{2d_{0}}\int_{-d_{0}}^{d_{0}}dxf_{1}(x)\
\times\ \frac{1}{2M+1}\sum_{m_{\alpha}=-M}^{M}f_{2}(m_{\alpha})\ .$ (25)

By using such formulae, one finds

$\displaystyle(\Delta m_{0}^{2})^{2}=(\Delta
m_{3/2}^{2})^{2}+{\tilde{m}}^{4}\sum_{\alpha}\overline{d_{\alpha}^{2}m_{\alpha}^{4}}\simeq\frac{NM^{4}}{15}\tilde{m}^{4}\left(\frac{4}{27}+d_{0}^{2}\right)\
,$ $\displaystyle{\rm where}\quad(\Delta
m_{3/2}^{2})^{2}=\frac{{\tilde{m}}^{4}}{9}\sum_{\alpha}[\overline{m_{\alpha}^{4}}-(\overline{m_{\alpha}^{2}})^{2}]\
.$ (26)

Consequently, one finds

$(\delta_{ij})_{LL/RR}\equiv\frac{\delta
m_{0}^{2}}{\overline{m_{0}^{2}}}\simeq\frac{1}{\sqrt{N}}\sqrt{\frac{1}{5}\left(4+27d_{0}^{2}\right)}\
.$ (27)

The off-diagonal entries, which have zero average values, are governed by the
standard deviation $\delta m_{0}^{2}$. One concludes then that they are
suppressed compared to the diagonal entries. For a large number of hidden
sector $N\geq 10^{6}$, the flavor problem of MSSM is therefore solved. While
this discussion is considering the flavor violating entries at the
supergravity scale, in practice at the weak scale, as we will see in the next
section, $N\sim 100$ would be sufficient to absolve strong constraints from
$\Delta m_{K}$. For the constraint from $\mu\to e+\gamma$, however, $N\sim
100$ is not sufficient and a larger value of $N$ should be chosen.

It should be noted however that the above discussion is pertaining to
definition of $\delta_{ij}$ at the high scale. At the weak scale, for the
leptonic sector (in the absence of right handed neutrinos), there is no
significant change in the mean values, where as for the hadronic (squark)
sector, due to the large gluino contributions to the squark masses in RG
running, the $\delta_{ij}^{q,u,d}$ would be further suppressed by a factor
from $7$ up to an order of magnitude.

For the gaugino masses, one finds

$\Delta
M_{1/2}^{2}={\tilde{m}}^{2}\sum_{\alpha}\overline{s_{\alpha}^{2}m_{\alpha}^{2}}\simeq
N{\tilde{m}}^{2}\frac{d_{0}^{2}M^{2}}{9}\ .$ (28)

Therefore one finds the standard deviation

$\Delta M_{1/2}=d_{0}\sqrt{\overline{m_{3/2}^{2}}}\ .$ (29)

For A-terms, let us consider for definiteness

$A^{u}=m_{3/2}y_{D}^{u}+{\tilde{m}}\sum_{\alpha}a_{\alpha}^{u}m_{\alpha}\ ,$
(30)

where in the mass basis for fermions and scalar-fermion-gaugino couplings are
diagonal, $y_{D}^{u}$ is diagonal in the flavor space. If $a_{\alpha}^{u}\sim
y^{u}\tilde{a}_{\alpha}$, then one expects the flavor violation in this case
to be under control. However, if this is not the case, we can use the same
arguments as above. One then finds the standard deviation

$\displaystyle(\Delta
A^{u})^{2}=\overline{m_{3/2}^{2}}(y_{D}^{u})^{2}+{\tilde{m}}^{2}\sum_{\alpha}\overline{(a_{\alpha}^{u})^{2}m_{\alpha}^{2}}-\left(\overline{m_{3/2}}y_{D}^{u}\right)^{2}$
$\displaystyle\simeq\frac{NM^{2}}{9}{\tilde{m}}^{2}d_{0}^{2}\left(1+(y_{D}^{u})^{2}\right)\
.$ (31)

The A-terms are such that additional flavor violation (other than from Yukawa
couplings) would be from the variance of the distribution. Thus we have

$(\delta_{ij}^{u})_{LR/RL}\equiv{\Delta
A^{u}v_{u}\over\overline{m_{0}^{2}}}\sim{3d_{0}v_{u}\over\sqrt{N}\tilde{m}M}\
.$ (32)

From the above it is clear that, similarly to the case of scalar masses, there
is a suppression $1/\sqrt{N}$ coming from the large number of hidden-sector
fields.

Notice that our starting expressions for soft terms (17) and the scanning we
performed above is different compared to one based on a naive spurion-type
parameterization of soft terms:

$\sum_{\alpha=1}^{N}\frac{s_{\alpha}^{a}}{M_{P}}\int d^{2}\theta
T_{\alpha}W^{a}W^{a}\quad\to\quad\quad
M^{a}_{1/2}=\frac{1}{M_{P}}\sum_{\alpha=1}^{N}s_{\alpha}^{a}F_{T_{\alpha}}\ ,$
(33) $\sum_{\alpha=1}^{N}\frac{d_{\alpha,ij}}{M_{P}^{2}}\int d^{4}\theta
T^{\dagger}_{\alpha}T_{\alpha}Q^{\dagger}_{i}Q_{j}\quad\to\quad\quad
m_{\tilde{f}_{ij}}^{2}=\frac{1}{M_{P}^{2}}\sum_{\alpha=1}^{N}d_{\alpha,ij}F_{T_{\alpha}}^{\dagger}F_{T_{\alpha}}\
,$ (34) $\sum_{\alpha=1}^{N}\frac{a_{\alpha,ijk}}{M_{P}}\int d^{2}\theta
T_{\alpha}Q_{i}Q_{j}Q_{k}\quad\to\quad\quad
A_{ijk}=\frac{1}{M_{P}}\sum_{\alpha=1}^{N}a_{\alpha,ijk}F_{T_{\alpha}}\ ,$
(35) $\sum_{\alpha=1}^{N}\frac{b_{\alpha}}{M_{P}^{2}}\int d^{4}\theta
T_{\alpha}T^{\dagger}_{\alpha}H_{u}H_{d}\quad\to\quad\quad
B_{H_{u}H_{d}}=\frac{1}{M_{P}^{2}}\sum_{\alpha=1}^{N}b_{\alpha}F_{T_{\alpha}}F_{T_{\alpha}}^{\dagger}\
,$ (36) $\sum_{\alpha=1}^{N}\frac{q_{\alpha}}{M_{P}}\int d^{4}\theta
T^{\dagger}_{\alpha}H_{u}H_{d}\quad\to\quad\quad\mu=\frac{1}{M_{P}}\sum_{\alpha=1}^{N}q_{\alpha}F_{T_{\alpha}}^{\dagger}\
.$ (37)

The difference is that in the SUGRA expressions (17) the flavor-blind
contributions proportional to $m_{3/2}^{2}$ scan coherently (add up) in soft
terms, whereas the other contributions, which are similar to the global SUSY
expressions (37), being multiplied by random couplings scanned around zero,
average to zero. A similar scan we performed above, but starting from (37)
would not lead a suppression of FCNC effects, unlike our scan above.

##  IV Numerical Analysis

Using eqs.(20) as boundary conditions at the high scale, we perform a
numerical analysis of the resulting soft spectrum at the weak scale and
studied the phenomenology. For the numerical analysis, we have considered $N$
to be 100, with $m_{\alpha}$ varying discretely and randomly from -100 to 100.
We explored taking $\tilde{m}$ to be 20 GeV. The maximum value of $m_{3/2}$ is
roughly about 6 TeV. The parameters
$d_{\alpha},s_{\alpha},a_{\alpha},g_{\alpha}$ are varied between {-1/4,1/4}. A
larger value for the $d_{0}$ parameters would lead to significant number of
the points ruled out due to tachyonic masses at the weak scale. We believe
that larger values $d_{0}\sim{\cal O}(1)$ would not significantly alter the
results presented here. Finally we set $\tan\beta=10$. We show that these
values of $N$ are enough to demonstrate the $1/\sqrt{N}$ suppression on the
flavor violating off-diagonal entries. We use Suseflav Chowdhury:2011zr  for
computation of the spectrum and computing the flavor observables.

Figure 1:  A scatter plot showing the variation of maximum value of
$\delta_{ij}$ of the type $LL/RR$ with respect to number of hidden sectors,
$N$ at the high scale (left side). A histogram of the $\delta_{ij}$ is
presented. As expected the mean is very close to zero and the variance is as
computed in the text.

The variation of the off-diagonal flavor entries in the sfermion mass matrices
is presented in Figs.(1) and (1). In Fig.(1) we present the scatter plot of a
typical $\delta_{ij}$ as defined in eq.(27). From the plot it is clear that
the $\delta_{ij}$ does fall off as $1/\sqrt{N}$. The second figure show the
same data in terms of a histogram, where as we can see the mean value is close
to zero and the variance is as expected from the formulae in eqs.(26,27).

The high scale distributions are then evolved to the weak scale where the full
soft supersymmetric spectrum is computed. Radiative electroweak symmetry
breaking conditions are imposed. Experimental constraints from LHC and the
Higgs mass are also taken in consideration. As is standard practice we
consider one $\delta_{ij}$ at a time. In the present letter, we consider the
two of the strongest constraints, i.e. the mass difference between the neutral
$K$-mesons, $\Delta M_{K}$ and the leptonic rare decay $\mu\to e+\gamma$. A
more detailed analysis with rest of the flavor processes will be presented
elsewhere edplskv .

Figure 2: Regions of the parameter space which satisfy the bounds from LHC,
Higgs mass and other phenomenological bounds. We have chosen $N$ to be 100 and
$\tilde{m}$ to be 20 GeV. The distribution of $\delta$ values is as per the
Eqs. (27) and (32). The left side plot is for $LL$ type mass insertion whereas
the right hand side is for $LR$ type mass insertion. All points satisfy the
experimental constraint from $\Delta M_{K}$.

At the weak scale, the diagonal entries would be enhanced due to
renormalisation group equation running, while the inter-generational entries
of the squark matrices would only receive corrections suppressed by the
product of Yukawa couplings and CKM angles Ciuchini:2007ha . Due to this the
$\delta_{ij}$ would be further suppressed roughly by an additional factor
which is proportional to the gluino mass corrections and roughly independent
of the number of hidden sector fields. In figs. 2,2 we present the regions of
the parameter space allowed by $\Delta M_{k}$ constraint as a function of the
gluino mass. It should be noted here that we have taken the weak scale values
of the mass insertions of eq.(27), where all the parameters appearing on the
RHS are computed at the weak scale. The left figure is for the $LL$ mass
insertion where as the right figure is for the $LR$ mass insertion. As can be
seen from the figure, all the points lie below the experimentally measured
value of $\Delta M_{K}$ Tanabashi:2018oca . The spectrum at the weak scale for
the first two generations is about 5-6 TeV and the gluino mass is shown in the
figure after taking into consideration the limits from LHC. For this spectrum
and a diluted $\delta\lesssim 10^{-1}$ the constraint from $\Delta M_{K}$ is
satisfied.

Figure 3:  Regions of the parameter space constrained by the leptonic rare
decay $\mu\to e+\gamma$ for $(\delta_{12})_{LL}$ (on the left) and for
$(\delta_{12})_{LR}$ (on the right). The horizontal (red) line is the present
experimental limit from MEG experiment. As can be seen, a large part of the
parameter space survives the experimental limit for N=100.

The leptonic rare process $\mu\to e+\gamma$ is however more strongly
constraining for the same set of parameters, i.e, $N=100$ and $\tilde{m}=20$
GeV. In Figs. 3 and 3 we present results of the scanning as a function of the
$\delta$ parameter and $\mu$. The left figure is for a $LL$ type mass
insertion whereas the right figure is for $RR$ type mass insertion. As one can
see from the figures, a significantly large region of the parameter space is
still compatible with the latest result from the MEG experimentTheMEG:2016wtm
, but the constraints from $LR$ are significantly stronger, as expected. A
larger $N$ value $\gtrsim 10^{5}$ would lead to complete dilution of the
$\delta$.

Finally we have also looked for regions with neutralino dark matter which
could lead to correct relic density while satisfying the constraints from
direct detection and flavour. As can be seen from Fig 4, there are two
branches which satisfy relic density as well as the direct detection result.
The first branch has dark matter masses $\lesssim$ 100 GeV and the lightest
neutralino is a pure bino. In the second branch the neutralino has a region in
which it is a pure bino and another region where there is a significant
admixture from wino and higgsino. The regions where the neutralino are pure
bino have significant co-annihilations with the chargino as can be seen from
the Fig 4. These regions arise due to the non-universality in the gaugino
masses at the high scale due to the $s_{\alpha}$ parameters. On the other
hand, regions with bino-higgsino mixing arise due to cancellations in
$M_{1/2}$ in contributions from various fluxes of different spurion fields. As
the charges/fluxes ${m}_{\alpha}$ take both signs, for significantly large $N$
there is an enhanced probability of cancellations between the charges leading
to small $M_{1/2}$ at the high scale. We numerically found that this
probability is significantly high for $N\gtrsim 30$. Due to the universal
nature of the gravitino mass, such cancellations do not occur in the soft
scalar mass terms. A low value for $\mu$ is very probable in these regions
leading to significant bino-higgsino mixing. Together they lead to regions
with physically viable regions of neutralino dark matter. More details of
these regions will be presented elsewhere edplskv .

Figure 4: Regions of the parameter space which satisfy the relic density and
the direct detection results from Xenon 1T. On the left we have shown the spin
independent cross-section with respect to the lightest neutralino mass and the
Bino component of the lightest neutralino. On the right, we show the same,
with the mass difference between chargino and Bino.

##  V Conclusions and Outlook

We presented a novel solution to the supersymmetric flavor problem in the
presence of large number of hidden sector (spurion) fields. Such a scenario
naturally arises in the string landscape. The result does not depend on the
explicit details of the string construction, but crucially on the form of the
soft terms in the supergravity potential in the presence of a large number of
hidden sector fields, eqs.(20). They naturally lead to a suppression of the
flavour violating entries as $1/\sqrt{N}$. At the weak scale, there is further
suppression due to the renormalisation group running, especially for the
hadronic mass insertions. We have shown that numerically $N=100$ is sufficient
to remove the constraints from $\Delta M_{K}$, whereas a much larger $N$ would
be required to eliminate completely the constraints on the leptonic sector
from $\mu\to e+\gamma$. Conversely, a discovery of such leptonic processes in
forthcoming experiments could be a smoking gun of such a scenario. The four
fluxes contribution to the soft terms presented here provides an interesting
framework to further study the implications for low energy phenomenology.

###### Acknowledgements.

This work is supported by the CNRS LIA (Laboratoire International Associé)
THEP (Theoretical High Energy Physics) and the INFRE-HEPNET (IndoFrench
Network on High Energy Physics) of CEFIPRA/IFCPAR (Indo-French Centre for the
Promotion of Advanced Research). We also thank CEFIPRA for the individual
project grant ”Glimpses of New Physics.” ED is supported in part by the ANR
grant Black-dS-String ANR-16-CE31-0004-01 and PL and SKV are also supported by
the Department of Science and Technology, Govt of India Project ”Nature of New
Physics”.

## References

  * (1) F. Gabbiani, E. Gabrielli, A. Masiero, and L. Silvestrini, “A Complete analysis of FCNC and CP constraints in general SUSY extensions of the standard model,” Nucl. Phys., vol. B477, pp. 321–352, 1996.
  * (2) S. Dimopoulos and D. W. Sutter, “The Supersymmetric flavor problem,” Nucl. Phys., vol. B452, pp. 496–512, 1995.
  * (3) M. Misiak, S. Pokorski, and J. Rosiek, “Supersymmetry and FCNC effects,” Adv. Ser. Direct. High Energy Phys., vol. 15, pp. 795–828, 1998.  [,795(1997)].
  * (4) A. Masiero, S. K. Vempati, and O. Vives, “Flavour physics and grand unification,” in Particle physics beyond the standard model. Proceedings, Summer School on Theoretical Physics, 84th Session, Les Houches, France, August 1-26, 2005, pp. 1–78, 2005.
  * (5) M. Dine and A. E. Nelson, “Dynamical supersymmetry breaking at low-energies,” Phys. Rev., vol. D48, pp. 1277–1287, 1993.
  * (6) M. Dine, A. E. Nelson, and Y. Shirman, “Low-energy dynamical supersymmetry breaking simplified,” Phys. Rev., vol. D51, pp. 1362–1370, 1995.
  * (7) M. Dine, A. E. Nelson, Y. Nir, and Y. Shirman, “New tools for low-energy dynamical supersymmetry breaking,” Phys. Rev., vol. D53, pp. 2658–2669, 1996.
  * (8) G. F. Giudice and R. Rattazzi, “Theories with gauge mediated supersymmetry breaking,” Phys. Rept., vol. 322, pp. 419–499, 1999.
  * (9) E. Bagnaschi et al., “Supersymmetric Models in Light of Improved Higgs Mass Calculations,” Eur. Phys. J., vol. C79, no. 2, p. 149, 2019.
  * (10) P. Draper, P. Meade, M. Reece, and D. Shih, “Implications of a 125 GeV Higgs for the MSSM and Low-Scale SUSY Breaking,” Phys. Rev., vol. D85, p. 095007, 2012.
  * (11) A. Brignole, L. E. Ibanez, and C. Munoz, “Towards a theory of soft terms for the supersymmetric Standard Model,” Nucl. Phys., vol. B422, pp. 125–171, 1994.  [Erratum: Nucl. Phys.B436,747(1995)].
  * (12) A. Brignole, L. E. Ibanez, and C. Munoz, “Soft supersymmetry breaking terms from supergravity and superstring models,” Adv. Ser. Direct. High Energy Phys., vol. 18, pp. 125–148, 1998.
  * (13) V. S. Kaplunovsky and J. Louis, “Model independent analysis of soft terms in effective supergravity and in string theory,” Phys. Lett., vol. B306, pp. 269–275, 1993.
  * (14) A. G. Cohen, D. B. Kaplan, and A. E. Nelson, “The More minimal supersymmetric standard model,” Phys. Lett., vol. B388, pp. 588–598, 1996.
  * (15) S. Antusch, S. F. King, M. Malinsky, and G. G. Ross, “Solving the SUSY Flavour and CP Problems with Non-Abelian Family Symmetry and Supergravity,” Phys. Lett., vol. B670, pp. 383–389, 2009.
  * (16) A. Pomarol and D. Tommasini, “Horizontal symmetries for the supersymmetric flavor problem,” Nucl. Phys., vol. B466, pp. 3–24, 1996.
  * (17) M. Dine, R. G. Leigh, and A. Kagan, “Flavor symmetries and the problem of squark degeneracy,” Phys. Rev., vol. D48, pp. 4269–4274, 1993.
  * (18) E. Dudas, S. Pokorski, and C. A. Savoy, “Soft scalar masses in supergravity with horizontal U(1)-x gauge symmetry,” Phys. Lett., vol. B369, pp. 255–261, 1996.
  * (19) C. Cheung, Y. Nomura, and J. Thaler, “Goldstini,” JHEP, vol. 03, p. 073, 2010.
  * (20) C. Cheung, J. Mardon, Y. Nomura, and J. Thaler, “A Definitive Signal of Multiple Supersymmetry Breaking,” JHEP, vol. 07, p. 035, 2010.
  * (21) K. Benakli and C. Moura, “Brane-Worlds Pseudo-Goldstinos,” Nucl. Phys., vol. B791, pp. 125–163, 2008.
  * (22) F. Denef and M. R. Douglas, “Distributions of flux vacua,” JHEP, vol. 05, p. 072, 2004.
  * (23) H. Baer, V. Barger, and D. Sengupta, “Mirage mediation from the landscape,” 2019\.
  * (24) H. Baer, V. Barger, S. Salam, H. Serce, and K. Sinha, “LHC SUSY and WIMP dark matter searches confront the string theory landscape,” JHEP, vol. 04, p. 043, 2019.
  * (25) H. Baer, V. Barger, H. Serce, and K. Sinha, “Higgs and superparticle mass predictions from the landscape,” JHEP, vol. 03, p. 002, 2018.
  * (26) H. Baer, V. Barger, M. Savoy, H. Serce, and X. Tata, “Superparticle phenomenology from the natural mini-landscape,” JHEP, vol. 06, p. 101, 2017.
  * (27) H. Baer, V. Barger, and D. Sengupta, “A landscape solution to the SUSY flavor and CP problems,” Phys. Rev. Res., vol. 1, no. 3, p. 033179, 2019.  [Phys. Rev. Research.1,033179(2019)].
  * (28) R. Bousso and J. Polchinski, “Quantization of four form fluxes and dynamical neutralization of the cosmological constant,” JHEP, vol. 06, p. 006, 2000\.
  * (29) A. Aurilia, H. Nicolai, and P. K. Townsend, “Hidden Constants: The Theta Parameter of QCD and the Cosmological Constant of N=8 Supergravity,” Nucl. Phys., vol. B176, pp. 509–522, 1980.
  * (30) E. Witten, “FERMION QUANTUM NUMBERS IN KALUZA-KLEIN THEORY,” 1983.  [Conf. Proc.C8306011,227(1983)].
  * (31) M. Henneaux and C. Teitelboim, “THE COSMOLOGICAL CONSTANT AS A CANONICAL VARIABLE,” Phys. Lett., vol. 143B, pp. 415–420, 1984.
  * (32) J. D. Brown and C. Teitelboim, “Dynamical Neutralization of the Cosmological Constant,” Phys. Lett., vol. B195, pp. 177–182, 1987.
  * (33) J. D. Brown and C. Teitelboim, “Neutralization of the Cosmological Constant by Membrane Creation,” Nucl. Phys., vol. B297, pp. 787–836, 1988.
  * (34) M. J. Duff, “The Cosmological Constant Is Possibly Zero, but the Proof Is Probably Wrong,” Phys. Lett., vol. B226, p. 36, 1989.  [Conf. Proc.C8903131,403(1989)].
  * (35) J. L. Feng, J. March-Russell, S. Sethi, and F. Wilczek, “Saltatory relaxation of the cosmological constant,” Nucl. Phys., vol. B602, pp. 307–328, 2001\.
  * (36) G. R. Dvali and A. Vilenkin, “Field theory models for variable cosmological constant,” Phys. Rev., vol. D64, p. 063509, 2001.
  * (37) G. Dvali and A. Vilenkin, “Cosmic attractors and gauge hierarchy,” Phys. Rev., vol. D70, p. 063501, 2004.
  * (38) G. Dvali, “Large hierarchies from attractor vacua,” Phys. Rev., vol. D74, p. 025018, 2006.
  * (39) A. Herraez and L. E. Ibanez, “An Axion-induced SM/MSSM Higgs Landscape and the Weak Gravity Conjecture,” JHEP, vol. 02, p. 109, 2017.
  * (40) G. F. Giudice, A. Kehagias, and A. Riotto, “The Selfish Higgs,” JHEP, vol. 10, p. 199, 2019.
  * (41) H. M. Lee, “Relaxation of Higgs mass and cosmological constant with four-form fluxes and reheating,” 2019.
  * (42) G. Dvali, “Three-form gauging of axion symmetries and gravity,” 2005.
  * (43) G. Dvali, “A Vacuum accumulation solution to the strong CP problem,” Phys. Rev., vol. D74, p. 025019, 2006.
  * (44) N. Kaloper and L. Sorbo, “A Natural Framework for Chaotic Inflation,” Phys. Rev. Lett., vol. 102, p. 121301, 2009.
  * (45) N. Kaloper, A. Lawrence, and L. Sorbo, “An Ignoble Approach to Large Field Inflation,” JCAP, vol. 1103, p. 023, 2011.
  * (46) N. Kaloper and A. Lawrence, “Natural chaotic inflation and ultraviolet sensitivity,” Phys. Rev., vol. D90, no. 2, p. 023506, 2014.
  * (47) E. Dudas, “Three-form multiplet and Inflation,” JHEP, vol. 12, p. 014, 2014.
  * (48) F. Farakos, A. Kehagias, D. Racco, and A. Riotto, “Scanning of the Supersymmetry Breaking Scale and the Gravitino Mass in Supergravity,” JHEP, vol. 06, p. 120, 2016.
  * (49) F. Farakos, S. Lanza, L. Martucci, and D. Sorokin, “Three-forms in Supergravity and Flux Compactifications,” Eur. Phys. J., vol. C77, no. 9, p. 602, 2017.
  * (50) A. Herraez, L. E. Ibanez, F. Marchesano, and G. Zoccarato, “The Type IIA Flux Potential, 4-forms and Freed-Witten anomalies,” JHEP, vol. 09, p. 018, 2018.
  * (51) S. Lanza, F. Marchesano, L. Martucci, and D. Sorokin, “How many fluxes fit in an EFT?,” JHEP, vol. 10, p. 110, 2019.
  * (52) S. Lanza, Exploring the Landscape of effective field theories.  PhD thesis, 2019.
  * (53) S. J. Gates, Jr., “SUPER P FORM GAUGE SUPERFIELDS,” Nucl. Phys., vol. B184, pp. 381–390, 1981.
  * (54) C. P. Burgess, J. P. Derendinger, F. Quevedo, and M. Quiros, “Gaugino condensates and chiral linear duality: An Effective Lagrangian analysis,” Phys. Lett., vol. B348, pp. 428–442, 1995.
  * (55) C. P. Burgess, J. P. Derendinger, F. Quevedo, and M. Quiros, “On gaugino condensation with field dependent gauge couplings,” Annals Phys., vol. 250, pp. 193–233, 1996.
  * (56) P. Binetruy, M. K. Gaillard, and T. R. Taylor, “Dynamical supersymmetric breaking and the linear multiplet,” Nucl. Phys., vol. B455, pp. 97–108, 1995.
  * (57) P. Binetruy, F. Pillon, G. Girardi, and R. Grimm, “The Three form multiplet in supergravity,” Nucl. Phys., vol. B477, pp. 175–202, 1996.
  * (58) P. Binetruy, G. Girardi, and R. Grimm, “Supergravity couplings: A Geometric formulation,” Phys. Rept., vol. 343, pp. 255–462, 2001.
  * (59) K. Groh, J. Louis, and J. Sommerfeld, “Duality and Couplings of 3-Form-Multiplets in N=1 Supersymmetry,” JHEP, vol. 05, p. 001, 2013.
  * (60) S. Ferrara, C. Kounnas, and F. Zwirner, “Mass formulae and natural hierarchy in string effective supergravities,” Nucl. Phys., vol. B429, pp. 589–625, 1994.  [Erratum: Nucl. Phys.B433,255(1995)].
  * (61) E. Dudas and S. K. Vempati, “Large D-terms, hierarchical soft spectra and moduli stabilisation,” Nucl. Phys., vol. B727, pp. 139–162, 2005.
  * (62) D. Chowdhury, R. Garani, and S. K. Vempati, “SUSEFLAV: Program for supersymmetric mass spectra with seesaw mechanism and rare lepton flavor violating decays,” Comput. Phys. Commun., vol. 184, pp. 899–918, 2013\.
  * (63) E. Dudas, P. Lamba, and S. K. Vempati, “To appear,”
  * (64) M. Ciuchini, A. Masiero, P. Paradisi, L. Silvestrini, S. K. Vempati, and O. Vives, “Soft SUSY breaking grand unification: Leptons versus quarks on the flavor playground,” Nucl. Phys., vol. B783, pp. 112–142, 2007.
  * (65) M. Tanabashi et al., “Review of Particle Physics,” Phys. Rev., vol. D98, no. 3, p. 030001, 2018.
  * (66) A. M. Baldini et al., “Search for the lepton flavour violating decay $\mu^{+}\rightarrow\mathrm{e}^{+}\gamma$ with the full dataset of the MEG experiment,” Eur. Phys. J., vol. C76, no. 8, p. 434, 2016.


# Generative AI for Synthetic Data Generation: Methods, Challenges and the
Future

Xu Guo, , and Yiqiang Chen Xu Guo is with Nanyang Technological University
(NTU), Singapore and Yiqiang Chen is with Institute of Computing Technology,
Chinese Academy of Sciences.Emails<EMAIL_ADDRESS>
###### Abstract

The recent surge in research focused on generating synthetic data from large
language models (LLMs), especially for scenarios with limited data
availability, marks a notable shift in Generative Artificial Intelligence
(AI). Their ability to perform comparably to real-world data positions this
approach as a compelling solution to low-resource challenges. This paper
delves into advanced technologies that leverage these gigantic LLMs for the
generation of task-specific training data. We outline methodologies,
evaluation techniques, and practical applications, discuss the current
limitations, and suggest potential pathways for future research.

###### Index Terms:

Generative AI, Synthetic Data Generation, Large Language Models.

##  I Introduction

The introduction of Transformer [1] in 2017, followed by groundbreaking LLMs
like OpenAI’s GPT [2] and Google’s BERT [3], marked the beginning of a new era
in language understanding and generation. More recently, generative LLMs
(e.g., GPT-3[4], LlaMa[5] and ChatGPT[6]) have propelled this evolution to
unprecedented heights, seamlessly converging with Generative AI and heralding
a fresh era in the realm of synthetic data generation[7, 8, 9, 10, 11, 12,
13].

The origins of Generative AI can be traced back to pivotal models such as
Generative Adversarial Networks[14] (GANs) and Variational Autoencoders[15]
(VAEs), which demonstrated the ability to generate realistic images and
signals[16]. However, it wasn’t until the advent of LLMs in recent years that
Generative AI truly began to flourish. These LLMs, trained on vast datasets,
showcased an unprecedented ability to produce coherent and contextually
relevant text, pushing the boundaries of what AI could achieve in language-
related tasks. The convergence of Generative AI and LLMs in the realm of
synthetic data creation represents not merely a technological advancement, but
a profound paradigm shift in our approach to data creation and the training of
AI models.

Why do we need synthetic data? The necessity for synthetic data arises from
the inherent limitations of general-purpose Large Language Models (LLMs) in
specialized and private domains, despite their significant achievements across
various benchmarks. For instance, ClinicalBERT[17], adapted from BERT through
pre-training on clinical texts, demonstrates superior performance in
predicting hospital readmissions compared to the original BERT[18], which was
trained on Wikipedia and BookCorpus[19] text data. This highlights a crucial
challenge: specialized domains often rely on domain-specific data that is not
readily available or open to the public, thereby underscoring the importance
of synthetic data in bridging these gaps.

Synergy between LLMs and synthetic data generation. Large Language Models
(LLMs) for synthetic data generation marks a significant frontier in the field
of AI. LLMs, such as ChatGPT, have revolutionized our approach to
understanding and generating human-like text, providing a mechanism to create
rich, contextually relevant synthetic data on an unprecedented scale. This
synergy is pivotal in addressing data scarcity and privacy concerns,
particularly in domains where real data is either limited or sensitive. By
generating text that closely mirrors human language, LLMs facilitate the
creation of robust, varied datasets necessary for training and refining AI
models across various applications, from healthcare[20], eduction[21] to
business management[22]. Moreover, this collaboration opens new avenues for
ethical AI development, allowing researchers to bypass the biases and ethical
dilemmas often inherent in real-world datasets. The integration of LLMs in
synthetic data generation not only pushes the boundaries of what’s achievable
in AI but also ensures a more responsible and inclusive approach to AI
development, aligning with evolving ethical standards and societal needs.

Other related survey papers. Comprehensive surveys for Generative AI and LLMs
exist, each revisits related works from a different perspective: Generative AI
surveys provide a holistic view of this area starting from Generative
Adversarial Networks (GANs) to ChatGPT [23] and models developed for synthetic
data generation in the past decade [24], with a special focus on text-to-image
[25] or text-to-speech [26] generation as well as practical applications in
Education [27] and Healthcare [28]; Surveys for LLMs provide systematic
categorization [29] for NLP tasks [30] and methods to adapt these LLMs to
specific domains [31] through model optimization and personalization
perspectives [32]. Surveys on LLMs for text generation [33] focus on
developing generative LLMs including model architecture choices and training
techniques and do not contain gigantic LLMs released in the past two years.
Unlike these survey papers, this paper mainly focuses on recent technologies
that employ generative LLMs without training them for synthetic training data
generation and elicit their potential impact on practical adoption.

Outline of this paper. The following of this paper is organized as follows.
Section II introduces recent methods for generating synthetic data from LLMs.
Specifically, we summarize prompt engineering techniques that are particularly
designed for probing LLMs to obtain desired data in sub-section II-A while in
sub-section II-B, we talk about how to employ parameter-efficient methods to
adapt LLMs for generating task-related data; In sub-sections II-C and II-D we
introduce methods that can measure the quality of the synthetic dataset and
how to effectively make use of the data for training. Section III details the
application of synthetic data, focusing on its utilization in low-resource
tasks in Sub-Section III-A and practical deployment scenarios in Sub-Section
III-B. Additionally, Sub-Section III-C provides a specific case study on the
use of synthetic data within medical domains. Finally, in Section IV, we
underscore some prominent challenges in synthetic data and discuss potential
avenues for future research.

Figure 1: A general comparison between using LLMs for label-specific synthetic
data generation (a) and label words prediction (b). In both cases, the LLMs
are frozen and a task-related prompt is provided to condition the LLMs for
task adaptation. $\langle X\rangle$ represents the text data and $\langle
Y\rangle$ represents the label words.

##  II Generating synthetic training data from LLMs

Figure 1 shows the major difference between using generative LLMs for
synthetic data generation and the predominant $\mathrm{Prompting}$ technique
[2, 34] that directly applies LLMs for label prediction. In short,
$\mathrm{Prompting}$ requires deploying the LLM model in practice to predict
the label words $\langle Y\rangle$ (e.g., negative) from the input text data
$\langle X\rangle$ with additional constraints from the prompt, e.g., “the
sentiment of the movie review” indicates that the context is a movie review
and the label shall describe its sentiment. On the contrary, synthetic data
generation requires LLMs to generate text data $\langle X\rangle$ based on
label-conditional prompts. It is the synthetic data distilled from LLMs rather
than the LLMs themselves that will be applied in downstream applications,
enabling more diverse and unlimited use cases based on synthetic data. Table I
lists the newly emerging methods for generating task-specific training data
from LLMs proposed in the past two years.

Method | Generator | Classifier | Benchmark
---|---|---|---
ZeroGen [9] |  GPT2-XL [35] |  LSTM[36] |  SST-2[37], IMDb[38], QNLI[39]
|  |  DistilBERT [40] |  RTE[41], SQuAD[39]
|  |  |  AdversarialQA[42]
ZeroGen+ [10] |  GPT2-XL[35] |  LSTM[36] |  IMDb[38], SST-2[37], Amazon[43]
|  |  DistilBERT [40] |  Rotten Tomatoes[44], Yelp[45]
|  |  |  Subj[46], AGNews[45], DBpedia[45]
SuperGen [8] |  CTRL[47] |  COCO-LM[48] |  GLUE[49]
|  |  RoBERTa[50] |
|  |  GPT-2[35] |
FewGen [7] |  CTRL[47] |  RoBERTa[50] |  GLUE[49]
ReGen [12] |  Condenser[51] |  RoBERTa[50] |  AGNews[45],DBpedia[45], MR[44]
|  |  |  NYT[52], Yahoo[45], Amazon[43]
|  |  |  Yelp[45], SST-2[37], IMDb[38]
ProGen [11] |  GPT2-XL[35] |  LSTM[36] |  SST-2[37], IMDb[38], Elec[43]
|  |  DistilBERT [40] |  Rotten Tomatoes[44], Yelp[45]
AttrPrompt [53] |  ChatGPT[6] |  BERT[3] |  NYT[52], Amazon[54]
|  |  DistilBERT [40] |  Reddit[55], StackExchange[55]
MixPrompt [13] |  FLAN-T5 XXL [56] |  GODEL [57] |  NLU++[58],TOPv2[59]
|  |  |  CrossNER [60]
TABLE I: Data generation methods. Generator refers to LLMs that are used for
synthetic data generation. Classifier refers to small-scale models that are
trained on the synthetic data. These methods are limited to NLP models and
tasks.

###  II-A Prompt engineering

Designing an informative prompt is the key to effective data generation with
LLMs. A simple and straightforward approach is to embed the label information
in the prompt to refrain LLMs from generating label-agnostic data as described
in Figure 1 (a). However, due to the limited number of words in labels and the
limited task information in the prompt, the data generated by LLMs still can
be unrelated to the task and lack diversity, limiting the size of the
synthetic dataset that can be generated from the same LLM. As such, more
advanced prompt engineering techniques are expected to circumvent the
limitations of traditional ones.

Attribute-controlled prompt. A clear definition for a specific task can be
obtained by specifying a set of attributes. Take News classification as an
example, one piece of News article can differ from another by providing the
details of $\mathrm{location}$, $\mathrm{topic}$, $\mathrm{text}$
$\mathrm{genre}$ and so on. Inspired by this, MSP [13] employs a mixture of
attributes in the prompt template to obtain desired synthetic data. In
AttrPrompt [53], authors show that such attribute-specific prompts can be
directly extracted from ChatGPT and then applied to query ChatGPT for
generating attribute-specific data. By expanding the simple class-conditional
prompt with more attribute constraints, we can gather more diverse synthetic
data from LLMs while ensuring relevance to the given task.

Verbalizer. The verbalizer technique was originally proposed to enhance
$\mathrm{Prompting}$ performance, where the target label words are expanded
with their neighbouring words that hold the same semantic meanings [61, 62].
This strategy can be directly utilized to promote diverse data generation by
expanding the class-conditional prompt into a set of semantically similar
prompts. Besides, the verbalizer values can be extracted from LLMs themselves.
For example, MetaPrompt [63] first obtains an expanded prompt from ChatGPT and
further applies the enriched prompt to prompt LLMs for data generation.

###  II-B Parameter-efficient task adaptation

Parameter-efficient approaches in the era of LLMs generally refer to the
tuning methods that only tune a small set of an LLM’s parameters (e.g., bias
terms [64], embeddings or last layer) or an extra set of parameters that are
inserted to LLMs (e.g., Adapters [65, 66], Prompt Tuning [67, 68], Prefix
Tuning [69] and LoRA [70]). In the tuning process, the parameters of the LLM
backbone are not updated and only the small set of trainable parameters are
learned on task-specific datasets to achieve domain adaptation. More
parameter-efficient methods can be found in the survey [71]. The advantage of
parameter-efficient methods is that they grasp new task information while
retaining powerful pre-trained knowledge.

To enable a general LLM to generate data for a specific task style, one
promising approach is to aggregate a few-shot dataset (e.g., eight instances
per class) and perform parameter-efficient adaptation for the LLM [68]. The
method, FewGen [7], demonstrates that by tuning a few set of prefix vectors
prepended to the CTRL model (1.6 Billion parameters) on few-shot datasets, the
PrefixCTRL can generate more task-related training data. Similarly, MSP [13]
trains a set of soft prompt embeddings on few-shot task-specific training data
and then applies the trained soft prompts to condition the FLAN-T5 [56]
(T5[34] further trained on instruction tuning datasets) for text generation.
Compared with zero-shot generation, a small budget for few-shot task data can
allow the general-purpose LLMs to quickly adapt to the target task under the
parameter-efficient learning paradigm.

###  II-C Measuring data quality

The quality of synthetic data is often measured by quantitative metrics. In
ZeroGen [9], authors measured the quality of the generated data from three
perspectives: $\mathrm{diversity}$, $\mathrm{correctnes}$, and
$\mathrm{naturalness}$. $\mathrm{Diversity}$ measures the difference between a
chunk of text and another in the generated instances. For example,
4-$\mathrm{gram}$ Self-BLEU computes BLEU scores on every four consecutive
tokens in the generated texts. $\mathrm{Correctnes}$ measures whether the data
instance is related to the given label. Existing approaches for measuring
$\mathrm{correctnes}$ can be divided into two categories: automatic evaluation
and human evaluation. Automatic evaluation methods train a model (e.g.,
RoBERTa-large) on the oracle training dataset in a fully-supervised full-model
fine-tuning manner, and then apply the model to calculate the percentage of
correctly predicted samples on the synthetic dataset. Human evaluation
requires the availability of human annotators who will be assigned a random
subset of the synthetic dataset and asked to judge whether the content is
related to the label. $\mathrm{Naturalness}$ measurement requires human
evaluators who can assess whether the generated text is fluent and similar to
human-written texts by selecting a score from a given range.

To obtain high-quality synthetic data, ProGen [11] proposes to incorporate a
quality estimation module in the data generation pipeline, where the firstly
generated synthetic data are evaluated by a task-specific model that was
trained on oracle data in advance. Then, the most influential synthetic
samples are selected as in-context examples to prompt GPT2-XL [35] to generate
a new set of synthetic data.

###  II-D Training with synthetic data

In the process of training with synthetic data generated from LLMs, challenges
such as inherent biases and hallucinations in the LLMs can introduce noise
into the dataset, despite meticulous prompt design and supervised training. To
mitigate these issues, the implementation of regularization techniques is
crucial for stabilizing training with noisy datasets. Innovations like
ZeroGen+ [10] suggest the use of a small weight network trained through
bilevel optimization to autonomously determine sample weights. Additionally,
FewGen [7] incorporates a self-supervised training approach using temporal
ensembling [72]. This method has been shown to offer superior performance
enhancements compared to label smoothing [73] when training downstream
classifiers on synthetic data, highlighting its effectiveness in dealing with
the unique challenges posed by synthetic datasets. Other techniques such as
gradual annealing [74] also demonstrates to be effective in enhancing the
learning performance on synthetic data.

##  III Applications

Synthetic data generated from LLMs can be used in a wide range of
applications. In this section, we first introduce how to solve the long-
standing low-resource and long-tail problems with synthetic data and its use
cases for fast inference and deployment. Then, we present two practical
examples of applying synthetic data in medical and education scenarios.

###  III-A Low-resource and long-tail problems

Low-resource problems are generally trapped by the lack of sufficient data and
in some cases particularly impacted by long-tail classes in practice [75].
Traditional research has predominantly leveraged transfer learning techniques
[76, 68] to enhance performance in low-resource settings. Yet, these methods
hinge on the availability of relevant source-domain datasets, which may not
always be accessible. The impressive generative capabilities of LLMs and the
production of highly realistic synthetic data signal a significant potential
to reshape the traditional landscape of low-resource and long-tail problems.

A primary challenge in merging the research directions of synthetic data
generation and low-resource learning tasks is navigating the distribution
disparity between real and synthetic data, as well as optimizing the use of
synthetic data in training scenarios. Noteworthy approaches to address these
issues include the application of regularization techniques. For instance,
FewGen employs temporal ensembling [7], and CAMEL utilizes gradual learning
[74]. Additionally, innovative data selection techniques, as explored in Du et
al. (2023) [77], offer valuable insights. These methods are instrumental in
harnessing the full potential of synthetic data to enhance learning
performance, particularly in environments where real data is limited or
imbalanced.

###  III-B Fast inference and lightweight deployment

Finetuning pre-trained language models on downstream tasks has been the
predominant approach starting from the release of BERT [18]. However, the
growing size of these language models, while enhancing performance, imposes
practical burdens on organizations requiring swift inference and prompt
responses. The shift towards synthetic data generation opens up a realm of
possibilities for downstream applications. By generating a curated synthetic
dataset, it becomes feasible to train smaller, less complex models, as
demonstrated in [9, 10, 11]. This approach not only facilitates easier
deployment but also ensures faster inference, addressing the critical need for
efficiency in real-world applications.

###  III-C Medical Scenarios

The medical domain presents unique challenges due to the confidential nature
of patient data and the relative scarcity of medical data compared to the
abundance of information available online. The use of LLMs and multi-modal
LLMs has shown promising potential in medical domains such as dental diagnosis
[78], radiograph analysis [79], and so on [80, 81]. The exceptional data
comprehension and generation capabilities of LLMs position synthetic data
generation as an especially promising research avenue in the medical domain.

Data augmentation. Synthetic data generation can help some medical tasks that
lack sufficient data to train a strong predictive model. For instance, studies
in [79] demonstrated that augmenting real datasets with synthetic chest
radiograph images generated by latent diffusion models[82] can enhance
classification performance. In medical language processing, Tang et al. (2023)
[83] demonstrated that tailored prompts provided to ChatGPT can yield task-
specific synthetic data, significantly boosting the performance in tasks like
biological named entity recognition and relation extraction. Additionally,
GatorTronGPT, as explored in Peng et al. (2023) [20], which involved training
GPT-3 from scratch on a dataset amalgamating 277-billion words from English
and clinical texts, exhibited remarkable proficiency in generating synthetic
clinical text. This data surpassed real data in performance across various
biomedical tasks, including relation extraction and question answering,
showcasing the potential of synthetic data in transforming medical AI
applications.

Missing value imputation. Medical data can be sparse in that patients may take
different or do not take some examinations, leading to imbalanced attributes.
Missing value imputation (MVI) methods are helpful in enhancing the density of
medical attribute values [84]. Traditional MVI approaches typically involve
random sampling from specified value ranges, as noted in Luo et al. (2022)
[85], essentially serving as a form of random data augmentation for certain
attributes. With the advent of multi-modal LLMs, Ozbey et al. (2023) [86]
demonstrate that in cross-modality translation tasks, missing images under
specific attributes can be effectively imputed using synthetic images
generated from diffusion models. Such synthetic data, compared to traditional
random imputation methods, offer more diverse information, thereby helping to
mitigate the issue of overfitting in attributes with limited data.

##  IV Challenges with Synthetic Data and Future Directions

Many domains suffer from a lack of quality data, especially when it comes to
rare events or minority classes. LLMs can augment existing datasets, creating
balanced and comprehensive data sets that improve the training and performance
of machine learning models. In this section, we highlight some challenges in
the creation and use of synthetic data and discuss promising research
directions.

###  IV-A Overcoming Data Limitations

Synthetic data generated from LLMs inherently faces several data limitations
that must be acknowledged and addressed.

Correctness and Diversity. In Section II, we summarized existing approaches
for monitoring the data quality and promoting data diversity in generation.
They demonstrated effectiveness but do not entirely solved the problem. The
challenge of ensuring the quality and accuracy of the generated data still
remains profound. As an inherent nature, LLMs may inadvertently propagate
inaccuracies or biases present in their pre-training data [87, 88], leading to
outputs that may not always align with factual or unbiased information.
Additionally, the intra-class and inter-class data diversity and domain
representativeness are a concern, especially in specialized or niche domains.

Hallucination. Synthetic data generated by Large Language Models (LLMs) can
sometimes be not only inaccurate but completely fictitious or disconnected
from reality, a phenomenon often referred to as ”hallucination” [89, 90]. For
instance, image generation based on specific captions can result in outputs
with unrealistic features, such as a soldier depicted with three hands, as
noted in the studies [74] for cross-modality generation. This hallucination
issue is frequently linked to the quality of the training data, particularly
if it contains inaccuracies that the LLM then overfits during the pre-training
phase. The challenge is compounded due to the difficulty of either fine-tuning
LLMs or modifying their pre-training data. Therefore, there’s a pressing need
to develop new, more effective strategies to detect and address hallucination
[91] in the context of synthetic data generation, ensuring the reliability and
authenticity of the output.

###  IV-B Data privacy and ethical concerns

While synthetic data offers a way to leverage the power of AI without
compromising individual privacy[92], the ethical implications of using
synthetic data, particularly in sensitive domains, raise questions about
privacy and consent, as the boundaries between real and synthetic data blur.
Research in [93] demonstrates that it is possible to extract specific
information from the datasets used in training LLMs. Consequently, there
exists a risk that synthetic data generation might inadvertently reveal
elements of the underlying training data [94], some of which might be subject
to licensing agreements. This scenario poses not only privacy issues but also
potential financial implications for users, highlighting the need for careful
management and consideration in the use and dissemination of synthetic data
generated by LLMs.

Moreover, uploading data to LLM APIs also remains a data privacy concern. For
instance, employing LLMs in clinical text mining poses significant privacy
risks related to uploading patient information to LLM APIs [83]. This
challenge necessitates a careful balance between leveraging the benefits of AI
and respecting the confidentiality and privacy of individuals, particularly in
healthcare and other sensitive areas. Addressing these concerns requires not
just technological solutions, but also robust policy frameworks and ethical
guidelines to ensure responsible use of synthetic data and AI technologies.

##  V Conclusion

This paper reviews recent research on utilizing generative LLMs for synthetic
data generation. With a focus on gigantic LLMs which are fixed for inference,
we elicit the complexities of producing high-quality and diverse synthetic
data and present some recent effective strategies to navigate these
challenges, including attribute-controlled prompt engineering and
verbalization strategies. Additionally, we also introduce some practical
training techniques for training downstream models on the synthetic data
presuming the data quality is inadequate. Then, we introduce some application
scenarios for the use of synthetic data generation, extending from general
low-resource issues to more specialized medical contexts. Finally, we conclude
by spotlighting the significant ongoing challenges in the realm of synthetic
data and proposing potential avenues for future research in this evolving
field.

## Acknowledgments

Xu Guo thanks the Wallenberg-NTU Presidential Postdoctoral Fellowship.

## References

  * [1] A. Vaswani, N. Shazeer, N. Parmar, J. Uszkoreit, L. Jones, A. N. Gomez, L. Kaiser, and I. Polosukhin, “Attention is all you need,” 2023.
  * [2] T. Brown, B. Mann, N. Ryder, M. Subbiah, J. D. Kaplan, P. Dhariwal, A. Neelakantan, P. Shyam, G. Sastry, A. Askell _et al._ , “Language models are few-shot learners,” _Advances in neural information processing systems_ , vol. 33, pp. 1877–1901, 2020.
  * [3] J. Devlin, M.-W. Chang, K. Lee, and K. Toutanova, “BERT: Pre-training of deep bidirectional transformers for language understanding,” in _Proceedings of the 2019 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, Volume 1 (Long and Short Papers)_ , J. Burstein, C. Doran, and T. Solorio, Eds. Minneapolis, Minnesota: Association for Computational Linguistics, Jun. 2019, pp. 4171–4186. [Online]. Available: https://aclanthology.org/N19-1423
  * [4] T. Kojima, S. S. Gu, M. Reid, Y. Matsuo, and Y. Iwasawa, “Large language models are zero-shot reasoners,” 2023.
  * [5] H. Touvron, T. Lavril, G. Izacard, X. Martinet, M.-A. Lachaux, T. Lacroix, B. Rozière, N. Goyal, E. Hambro, F. Azhar, A. Rodriguez, A. Joulin, E. Grave, and G. Lample, “Llama: Open and efficient foundation language models,” 2023.
  * [6] OpenAI, “Introducing chatgpt,” 2023.
  * [7] Y. Meng, M. Michalski, J. Huang, Y. Zhang, T. Abdelzaher, and J. Han, “Tuning language models as training data generators for augmentation-enhanced few-shot learning,” in _International Conference on Machine Learning_. PMLR, 2023, pp. 24 457–24 477.
  * [8] Y. Meng, J. Huang, Y. Zhang, and J. Han, “Generating training data with language models: Towards zero-shot language understanding,” in _Advances in Neural Information Processing Systems_ , A. H. Oh, A. Agarwal, D. Belgrave, and K. Cho, Eds., 2022. [Online]. Available: https://openreview.net/forum?id=4G1Sfp_1sz7
  * [9] J. Ye, J. Gao, Q. Li, H. Xu, J. Feng, Z. Wu, T. Yu, and L. Kong, “ZeroGen: Efficient zero-shot learning via dataset generation,” in _Proceedings of the 2022 Conference on Empirical Methods in Natural Language Processing_ , Y. Goldberg, Z. Kozareva, and Y. Zhang, Eds. Abu Dhabi, United Arab Emirates: Association for Computational Linguistics, Dec. 2022, pp. 11 653–11 669. [Online]. Available: https://aclanthology.org/2022.emnlp-main.801
  * [10] J. Gao, R. Pi, L. Yong, H. Xu, J. Ye, Z. Wu, W. ZHANG, X. Liang, Z. Li, and L. Kong, “Self-guided noise-free data generation for efficient zero-shot learning,” in _The Eleventh International Conference on Learning Representations_ , 2023. [Online]. Available: https://openreview.net/forum?id=h5OpjGd_lo6
  * [11] J. Ye, J. Gao, Z. Wu, J. Feng, T. Yu, and L. Kong, “ProGen: Progressive zero-shot dataset generation via in-context feedback,” in _Findings of the Association for Computational Linguistics: EMNLP 2022_ , Y. Goldberg, Z. Kozareva, and Y. Zhang, Eds. Abu Dhabi, United Arab Emirates: Association for Computational Linguistics, Dec. 2022, pp. 3671–3683. [Online]. Available: https://aclanthology.org/2022.findings-emnlp.269
  * [12] Y. Yu, Y. Zhuang, R. Zhang, Y. Meng, J. Shen, and C. Zhang, “ReGen: Zero-shot text classification via training data generation with progressive dense retrieval,” in _Findings of the Association for Computational Linguistics: ACL 2023_ , A. Rogers, J. Boyd-Graber, and N. Okazaki, Eds. Toronto, Canada: Association for Computational Linguistics, Jul. 2023, pp. 11 782–11 805. [Online]. Available: https://aclanthology.org/2023.findings-acl.748
  * [13] D. Chen, C. Lee, Y. Lu, D. Rosati, and Z. Yu, “Mixture of soft prompts for controllable data generation,” in _Findings of the Association for Computational Linguistics: EMNLP 2023_ , H. Bouamor, J. Pino, and K. Bali, Eds. Singapore: Association for Computational Linguistics, Dec. 2023, pp. 14 815–14 833. [Online]. Available: https://aclanthology.org/2023.findings-emnlp.988
  * [14] I. J. Goodfellow, J. Pouget-Abadie, M. Mirza, B. Xu, D. Warde-Farley, S. Ozair, A. Courville, and Y. Bengio, “Generative adversarial networks,” 2014.
  * [15] D. P. Kingma and M. Welling, “Auto-encoding variational bayes,” 2022.
  * [16] Y. Wu, J. Donahue, D. Balduzzi, K. Simonyan, and T. Lillicrap, “Logan: Latent optimisation for generative adversarial networks,” 2020.
  * [17] K. Huang, J. Altosaar, and R. Ranganath, “Clinicalbert: Modeling clinical notes and predicting hospital readmission,” _arXiv preprint arXiv:1904.05342_ , 2019.
  * [18] J. Devlin, M.-W. Chang, K. Lee, and K. Toutanova, “Bert: Pre-training of deep bidirectional transformers for language understanding,” _arXiv preprint arXiv:1810.04805_ , 2018.
  * [19] Y. Zhu, R. Kiros, R. Zemel, R. Salakhutdinov, R. Urtasun, A. Torralba, and S. Fidler, “Aligning books and movies: Towards story-like visual explanations by watching movies and reading books,” in _Proceedings of the IEEE international conference on computer vision_ , 2015, pp. 19–27.
  * [20] C. Peng, X. Yang, A. Chen, K. E. Smith, N. PourNejatian, A. B. Costa, C. Martin, M. G. Flores, Y. Zhang, T. Magoc _et al._ , “A study of generative large language model for medical research and healthcare,” _arXiv preprint arXiv:2305.13523_ , 2023.
  * [21] S. Moore, R. Tong, A. Singh, Z. Liu, X. Hu, Y. Lu, J. Liang, C. Cao, H. Khosravi, P. Denny _et al._ , “Empowering education with llms-the next-gen interface and content generation,” in _International Conference on Artificial Intelligence in Education_. Springer, 2023, pp. 32–37.
  * [22] N. Rane, “Role and challenges of chatgpt and similar generative artificial intelligence in business management,” _Available at SSRN 4603227_ , 2023.
  * [23] Y. Cao, S. Li, Y. Liu, Z. Yan, Y. Dai, P. S. Yu, and L. Sun, “A comprehensive survey of ai-generated content (aigc): A history of generative ai from gan to chatgpt,” _arXiv preprint arXiv:2303.04226_ , 2023.
  * [24] A. Bauer, S. Trapp, M. Stenger, R. Leppich, S. Kounev, M. Leznik, K. Chard, and I. Foster, “Comprehensive exploration of synthetic data generation: A survey,” _arXiv preprint arXiv:2401.02524_ , 2024.
  * [25] C. Zhang, C. Zhang, M. Zhang, and I. S. Kweon, “Text-to-image diffusion model in generative ai: A survey,” _arXiv preprint arXiv:2303.07909_ , 2023.
  * [26] C. Zhang, C. Zhang, S. Zheng, M. Zhang, M. Qamar, S.-H. Bae, and I. S. Kweon, “A survey on audio diffusion models: Text to speech synthesis and enhancement in generative ai,” _arXiv preprint arXiv:2303.13336_ , vol. 2, 2023.
  * [27] D. Baidoo-Anu and L. O. Ansah, “Education in the era of generative artificial intelligence (ai): Understanding the potential benefits of chatgpt in promoting teaching and learning,” _Journal of AI_ , vol. 7, no. 1, pp. 52–62, 2023.
  * [28] P. Yu, H. Xu, X. Hu, and C. Deng, “Leveraging generative ai and large language models: A comprehensive roadmap for healthcare integration,” in _Healthcare_ , vol. 11, no. 20. MDPI, 2023, p. 2776.
  * [29] X. Qiu, T. Sun, Y. Xu, Y. Shao, N. Dai, and X. Huang, “Pre-trained models for natural language processing: A survey,” _Science China Technological Sciences_ , vol. 63, no. 10, pp. 1872–1897, 2020.
  * [30] B. Min, H. Ross, E. Sulem, A. P. B. Veyseh, T. H. Nguyen, O. Sainz, E. Agirre, I. Heintz, and D. Roth, “Recent advances in natural language processing via large pre-trained language models: A survey,” _ACM Computing Surveys_ , vol. 56, no. 2, pp. 1–40, 2023.
  * [31] X. Guo, “Data-efficient domain adaptation for pretrained language models,” 2023.
  * [32] X. Guo and H. Yu, “On the domain adaptation and generalization of pretrained language models: A survey,” _arXiv preprint arXiv:2211.03154_ , 2022.
  * [33] J. Li, T. Tang, W. X. Zhao, J.-Y. Nie, and J.-R. Wen, “Pretrained language models for text generation: A survey,” _arXiv preprint arXiv:2201.05273_ , 2022.
  * [34] C. Raffel, N. Shazeer, A. Roberts, K. Lee, S. Narang, M. Matena, Y. Zhou, W. Li, and P. J. Liu, “Exploring the limits of transfer learning with a unified text-to-text transformer,” _The Journal of Machine Learning Research_ , vol. 21, no. 1, pp. 5485–5551, 2020.
  * [35] A. Radford, J. Wu, R. Child, D. Luan, D. Amodei, I. Sutskever _et al._ , “Language models are unsupervised multitask learners,” _OpenAI blog_ , vol. 1, no. 8, p. 9, 2019.
  * [36] S. Hochreiter and J. Schmidhuber, “Long short-term memory,” _Neural computation_ , vol. 9, no. 8, pp. 1735–1780, 1997.
  * [37] R. Socher, A. Perelygin, J. Wu, J. Chuang, C. D. Manning, A. Ng, and C. Potts, “Recursive deep models for semantic compositionality over a sentiment treebank,” in _Proceedings of the 2013 Conference on Empirical Methods in Natural Language Processing_ , D. Yarowsky, T. Baldwin, A. Korhonen, K. Livescu, and S. Bethard, Eds. Seattle, Washington, USA: Association for Computational Linguistics, Oct. 2013, pp. 1631–1642. [Online]. Available: https://aclanthology.org/D13-1170
  * [38] A. L. Maas, R. E. Daly, P. T. Pham, D. Huang, A. Y. Ng, and C. Potts, “Learning word vectors for sentiment analysis,” in _Proceedings of the 49th Annual Meeting of the Association for Computational Linguistics: Human Language Technologies_ , D. Lin, Y. Matsumoto, and R. Mihalcea, Eds. Portland, Oregon, USA: Association for Computational Linguistics, Jun. 2011, pp. 142–150. [Online]. Available: https://aclanthology.org/P11-1015
  * [39] P. Rajpurkar, J. Zhang, K. Lopyrev, and P. Liang, “SQuAD: 100,000+ questions for machine comprehension of text,” in _Proceedings of the 2016 Conference on Empirical Methods in Natural Language Processing_ , J. Su, K. Duh, and X. Carreras, Eds. Austin, Texas: Association for Computational Linguistics, Nov. 2016, pp. 2383–2392. [Online]. Available: https://aclanthology.org/D16-1264
  * [40] V. Sanh, L. Debut, J. Chaumond, and T. Wolf, “Distilbert, a distilled version of bert: smaller, faster, cheaper and lighter,” 2020.
  * [41] I. Dagan, O. Glickman, and B. Magnini, “The pascal recognising textual entailment challenge,” in _Machine learning challenges workshop_. Springer, 2005, pp. 177–190.
  * [42] M. Bartolo, A. Roberts, J. Welbl, S. Riedel, and P. Stenetorp, “Beat the AI: Investigating adversarial human annotation for reading comprehension,” _Transactions of the Association for Computational Linguistics_ , vol. 8, pp. 662–678, 2020. [Online]. Available: https://aclanthology.org/2020.tacl-1.43
  * [43] J. McAuley and J. Leskovec, “Hidden factors and hidden topics: understanding rating dimensions with review text,” in _Proceedings of the 7th ACM conference on Recommender systems_ , 2013, pp. 165–172.
  * [44] B. Pang and L. Lee, “Seeing stars: Exploiting class relationships for sentiment categorization with respect to rating scales,” in _Proceedings of the 43rd Annual Meeting of the Association for Computational Linguistics (ACL’05)_ , K. Knight, H. T. Ng, and K. Oflazer, Eds. Ann Arbor, Michigan: Association for Computational Linguistics, Jun. 2005, pp. 115–124. [Online]. Available: https://aclanthology.org/P05-1015
  * [45] X. Zhang, J. Zhao, and Y. LeCun, “Character-level convolutional networks for text classification,” _Advances in neural information processing systems_ , vol. 28, 2015.
  * [46] B. Pang and L. Lee, “A sentimental education: Sentiment analysis using subjectivity summarization based on minimum cuts,” in _Proceedings of the 42nd Annual Meeting of the Association for Computational Linguistics (ACL-04)_ , Barcelona, Spain, Jul. 2004, pp. 271–278. [Online]. Available: https://aclanthology.org/P04-1035
  * [47] N. S. Keskar, B. McCann, L. R. Varshney, C. Xiong, and R. Socher, “Ctrl: A conditional transformer language model for controllable generation,” _arXiv preprint arXiv:1909.05858_ , 2019.
  * [48] Y. Meng, C. Xiong, P. Bajaj, P. Bennett, J. Han, X. Song _et al._ , “Coco-lm: Correcting and contrasting text sequences for language model pretraining,” _Advances in Neural Information Processing Systems_ , vol. 34, pp. 23 102–23 114, 2021.
  * [49] A. Wang, A. Singh, J. Michael, F. Hill, O. Levy, and S. Bowman, “GLUE: A multi-task benchmark and analysis platform for natural language understanding,” in _Proceedings of the 2018 EMNLP Workshop BlackboxNLP: Analyzing and Interpreting Neural Networks for NLP_ , T. Linzen, G. Chrupała, and A. Alishahi, Eds. Brussels, Belgium: Association for Computational Linguistics, Nov. 2018, pp. 353–355. [Online]. Available: https://aclanthology.org/W18-5446
  * [50] Y. Liu, M. Ott, N. Goyal, J. Du, M. Joshi, D. Chen, O. Levy, M. Lewis, L. Zettlemoyer, and V. Stoyanov, “Roberta: A robustly optimized bert pretraining approach,” _arXiv preprint arXiv:1907.11692_ , 2019.
  * [51] L. Gao and J. Callan, “Condenser: a pre-training architecture for dense retrieval,” in _Proceedings of the 2021 Conference on Empirical Methods in Natural Language Processing_ , M.-F. Moens, X. Huang, L. Specia, and S. W.-t. Yih, Eds. Online and Punta Cana, Dominican Republic: Association for Computational Linguistics, Nov. 2021, pp. 981–993. [Online]. Available: https://aclanthology.org/2021.emnlp-main.75
  * [52] Y. Meng, J. Shen, C. Zhang, and J. Han, “Weakly-supervised hierarchical text classification,” _Proceedings of the AAAI Conference on Artificial Intelligence_ , vol. 33, no. 01, pp. 6826–6833, Jul. 2019. [Online]. Available: https://ojs.aaai.org/index.php/AAAI/article/view/4658
  * [53] Y. Yu, Y. Zhuang, J. Zhang, Y. Meng, A. Ratner, R. Krishna, J. Shen, and C. Zhang, “Large language model as attributed training data generator: A tale of diversity and bias,” in _Thirty-seventh Conference on Neural Information Processing Systems Datasets and Benchmarks Track_ , 2023. [Online]. Available: https://openreview.net/forum?id=6hZIfAY9GD
  * [54] J. Blitzer, M. Dredze, and F. Pereira, “Biographies, Bollywood, boom-boxes and blenders: Domain adaptation for sentiment classification,” in _Proceedings of the 45th Annual Meeting of the Association of Computational Linguistics_ , A. Zaenen and A. van den Bosch, Eds. Prague, Czech Republic: Association for Computational Linguistics, Jun. 2007, pp. 440–447. [Online]. Available: https://aclanthology.org/P07-1056
  * [55] G. Geigle, N. Reimers, A. Rücklé, and I. Gurevych, “Tweac: Transformer with extendable qa agent classifiers,” 2021.
  * [56] H. W. Chung, L. Hou, S. Longpre, B. Zoph, Y. Tay, W. Fedus, Y. Li, X. Wang, M. Dehghani, S. Brahma, A. Webson, S. S. Gu, Z. Dai, M. Suzgun, X. Chen, A. Chowdhery, A. Castro-Ros, M. Pellat, K. Robinson, D. Valter, S. Narang, G. Mishra, A. Yu, V. Zhao, Y. Huang, A. Dai, H. Yu, S. Petrov, E. H. Chi, J. Dean, J. Devlin, A. Roberts, D. Zhou, Q. V. Le, and J. Wei, “Scaling instruction-finetuned language models,” 2022.
  * [57] B. Peng, M. Galley, P. He, C. Brockett, L. Liden, E. Nouri, Z. Yu, B. Dolan, and J. Gao, “Godel: Large-scale pre-training for goal-directed dialog,” 2022.
  * [58] I. Casanueva, I. Vulić, G. Spithourakis, and P. Budzianowski, “NLU++: A multi-label, slot-rich, generalisable dataset for natural language understanding in task-oriented dialogue,” in _Findings of the Association for Computational Linguistics: NAACL 2022_ , M. Carpuat, M.-C. de Marneffe, and I. V. Meza Ruiz, Eds. Seattle, United States: Association for Computational Linguistics, Jul. 2022, pp. 1998–2013. [Online]. Available: https://aclanthology.org/2022.findings-naacl.154
  * [59] X. Chen, A. Ghoshal, Y. Mehdad, L. Zettlemoyer, and S. Gupta, “Low-resource domain adaptation for compositional task-oriented semantic parsing,” in _Proceedings of the 2020 Conference on Empirical Methods in Natural Language Processing (EMNLP)_ , B. Webber, T. Cohn, Y. He, and Y. Liu, Eds. Online: Association for Computational Linguistics, Nov. 2020, pp. 5090–5100. [Online]. Available: https://aclanthology.org/2020.emnlp-main.413
  * [60] Z. Liu, Y. Xu, T. Yu, W. Dai, Z. Ji, S. Cahyawijaya, A. Madotto, and P. Fung, “Crossner: Evaluating cross-domain named entity recognition,” _Proceedings of the AAAI Conference on Artificial Intelligence_ , vol. 35, no. 15, pp. 13 452–13 460, May 2021. [Online]. Available: https://ojs.aaai.org/index.php/AAAI/article/view/17587
  * [61] G. Cui, S. Hu, N. Ding, L. Huang, and Z. Liu, “Prototypical verbalizer for prompt-based few-shot tuning,” in _Proceedings of the 60th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers)_ , S. Muresan, P. Nakov, and A. Villavicencio, Eds. Dublin, Ireland: Association for Computational Linguistics, May 2022, pp. 7014–7024. [Online]. Available: https://aclanthology.org/2022.acl-long.483
  * [62] S. Hu, N. Ding, H. Wang, Z. Liu, J. Wang, J. Li, W. Wu, and M. Sun, “Knowledgeable prompt-tuning: Incorporating knowledge into prompt verbalizer for text classification,” 2022.
  * [63] L. Reynolds and K. McDonell, “Prompt programming for large language models: Beyond the few-shot paradigm,” 2021.
  * [64] E. Ben Zaken, Y. Goldberg, and S. Ravfogel, “BitFit: Simple parameter-efficient fine-tuning for transformer-based masked language-models,” in _Proceedings of the 60th Annual Meeting of the Association for Computational Linguistics (Volume 2: Short Papers)_ , S. Muresan, P. Nakov, and A. Villavicencio, Eds. Dublin, Ireland: Association for Computational Linguistics, May 2022, pp. 1–9. [Online]. Available: https://aclanthology.org/2022.acl-short.1
  * [65] N. Houlsby, A. Giurgiu, S. Jastrzebski, B. Morrone, Q. De Laroussilhe, A. Gesmundo, M. Attariyan, and S. Gelly, “Parameter-efficient transfer learning for NLP,” in _Proceedings of the 36th International Conference on Machine Learning_ , ser. Proceedings of Machine Learning Research, K. Chaudhuri and R. Salakhutdinov, Eds., vol. 97. PMLR, 09–15 Jun 2019, pp. 2790–2799. [Online]. Available: https://proceedings.mlr.press/v97/houlsby19a.html
  * [66] M. Liu, X. Guo, H. Jiakai, J. Chen, F. Zhou, and S. Hui, “InteMATs: Integrating granularity-specific multilingual adapters for cross-lingual transfer,” in _Findings of the Association for Computational Linguistics: EMNLP 2023_ , H. Bouamor, J. Pino, and K. Bali, Eds. Singapore: Association for Computational Linguistics, Dec. 2023, pp. 5035–5049. [Online]. Available: https://aclanthology.org/2023.findings-emnlp.335
  * [67] B. Lester, R. Al-Rfou, and N. Constant, “The power of scale for parameter-efficient prompt tuning,” in _Proceedings of the 2021 Conference on Empirical Methods in Natural Language Processing_ , M.-F. Moens, X. Huang, L. Specia, and S. W.-t. Yih, Eds. Online and Punta Cana, Dominican Republic: Association for Computational Linguistics, Nov. 2021, pp. 3045–3059. [Online]. Available: https://aclanthology.org/2021.emnlp-main.243
  * [68] X. Guo, B. Li, and H. Yu, “Improving the sample efficiency of prompt tuning with domain adaptation,” in _Findings of the Association for Computational Linguistics: EMNLP 2022_ , Y. Goldberg, Z. Kozareva, and Y. Zhang, Eds. Abu Dhabi, United Arab Emirates: Association for Computational Linguistics, Dec. 2022, pp. 3523–3537. [Online]. Available: https://aclanthology.org/2022.findings-emnlp.258
  * [69] X. L. Li and P. Liang, “Prefix-tuning: Optimizing continuous prompts for generation,” in _Proceedings of the 59th Annual Meeting of the Association for Computational Linguistics and the 11th International Joint Conference on Natural Language Processing (Volume 1: Long Papers)_ , C. Zong, F. Xia, W. Li, and R. Navigli, Eds. Online: Association for Computational Linguistics, Aug. 2021, pp. 4582–4597. [Online]. Available: https://aclanthology.org/2021.acl-long.353
  * [70] E. J. Hu, yelong shen, P. Wallis, Z. Allen-Zhu, Y. Li, S. Wang, L. Wang, and W. Chen, “LoRA: Low-rank adaptation of large language models,” in _International Conference on Learning Representations_ , 2022. [Online]. Available: https://openreview.net/forum?id=nZeVKeeFYf9
  * [71] N. Ding, Y. Qin, G. Yang, F. Wei, Z. Yang, Y. Su, S. Hu, Y. Chen, C.-M. Chan, W. Chen, J. Yi, W. Zhao, X. Wang, Z. Liu, H.-T. Zheng, J. Chen, Y. Liu, J. Tang, J. Li, and M. Sun, “Delta tuning: A comprehensive study of parameter efficient methods for pre-trained language models,” 2022.
  * [72] S. Laine and T. Aila, “Temporal ensembling for semi-supervised learning,” _arXiv preprint arXiv:1610.02242_ , 2016.
  * [73] R. Müller, S. Kornblith, and G. Hinton, “When does label smoothing help?” 2020.
  * [74] Z. Du, Y. Li, X. Guo, Y. Sun, and B. Li, “Training multimedia event extraction with generated images and captions,” _arXiv preprint arXiv:2306.08966_ , 2023.
  * [75] A. M. H. Tiong, J. Li, G. Lin, B. Li, C. Xiong, and S. C. H. Hoi, “Improving tail-class representation with centroid contrastive learning,” 2023.
  * [76] X. Guo, B. Li, H. Yu, and C. Miao, “Latent-optimized adversarial neural transfer for sarcasm detection,” in _Proceedings of the 2021 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies_ , K. Toutanova, A. Rumshisky, L. Zettlemoyer, D. Hakkani-Tur, I. Beltagy, S. Bethard, R. Cotterell, T. Chakraborty, and Y. Zhou, Eds. Online: Association for Computational Linguistics, Jun. 2021, pp. 5394–5407. [Online]. Available: https://aclanthology.org/2021.naacl-main.425
  * [77] Z. Du, H. Li, X. Guo, and B. Li, “Training on synthetic data beats real data in multimodal relation extraction,” 2023.
  * [78] H. Huang, O. Zheng, D. Wang, J. Yin, Z. Wang, S. Ding, H. Yin, C. Xu, R. Yang, Q. Zheng _et al._ , “Chatgpt for shaping the future of dentistry: the potential of multi-modal large language model,” _International Journal of Oral Science_ , vol. 15, no. 1, p. 29, 2023.
  * [79] K. Packhäuser, L. Folle, F. Thamm, and A. Maier, “Generation of anonymous chest radiographs using latent diffusion models for training thoracic abnormality classification systems,” in _2023 IEEE 20th International Symposium on Biomedical Imaging (ISBI)_. IEEE, 2023, pp. 1–5.
  * [80] K. Singhal, S. Azizi, T. Tu, S. S. Mahdavi, J. Wei, H. W. Chung, N. Scales, A. Tanwani, H. Cole-Lewis, S. Pfohl _et al._ , “Large language models encode clinical knowledge,” _Nature_ , vol. 620, no. 7972, pp. 172–180, 2023.
  * [81] A. J. Thirunavukarasu, D. S. J. Ting, K. Elangovan, L. Gutierrez, T. F. Tan, and D. S. W. Ting, “Large language models in medicine,” _Nature medicine_ , vol. 29, no. 8, pp. 1930–1940, 2023.
  * [82] R. Rombach, A. Blattmann, D. Lorenz, P. Esser, and B. Ommer, “High-resolution image synthesis with latent diffusion models,” 2022.
  * [83] R. Tang, X. Han, X. Jiang, and X. Hu, “Does synthetic data generation of llms help clinical text mining?” _arXiv preprint arXiv:2303.04360_ , 2023.
  * [84] M. Liu, S. Li, H. Yuan, M. E. H. Ong, Y. Ning, F. Xie, S. E. Saffari, Y. Shang, V. Volovici, B. Chakraborty _et al._ , “Handling missing values in healthcare data: A systematic review of deep learning-based imputation techniques,” _Artificial Intelligence in Medicine_ , p. 102587, 2023.
  * [85] F. Luo, H. Qian, D. Wang, X. Guo, Y. Sun, E. S. Lee, H. H. Teong, R. T. R. Lai, and C. Miao, “Missing value imputation for diabetes prediction,” in _2022 International Joint Conference on Neural Networks (IJCNN)_. IEEE, 2022, pp. 1–8.
  * [86] M. Özbey, O. Dalmaz, S. U. Dar, H. A. Bedel, Ş. Özturk, A. Güngör, and T. Çukur, “Unsupervised medical image translation with adversarial diffusion models,” _IEEE Transactions on Medical Imaging_ , 2023.
  * [87] P. P. Liang, C. Wu, L.-P. Morency, and R. Salakhutdinov, “Towards understanding and mitigating social biases in language models,” in _International Conference on Machine Learning_. PMLR, 2021, pp. 6565–6576.
  * [88] H. Kotek, R. Dockum, and D. Sun, “Gender bias and stereotypes in large language models,” in _Proceedings of The ACM Collective Intelligence Conference_ , 2023, pp. 12–24.
  * [89] Z. Ji, N. Lee, R. Frieske, T. Yu, D. Su, Y. Xu, E. Ishii, Y. J. Bang, A. Madotto, and P. Fung, “Survey of hallucination in natural language generation,” _ACM Computing Surveys_ , vol. 55, no. 12, pp. 1–38, 2023.
  * [90] Y. Zhang, Y. Li, L. Cui, D. Cai, L. Liu, T. Fu, X. Huang, E. Zhao, Y. Zhang, Y. Chen _et al._ , “Siren’s song in the ai ocean: A survey on hallucination in large language models,” _arXiv preprint arXiv:2309.01219_ , 2023.
  * [91] W. Xu, S. Agrawal, E. Briakou, M. J. Martindale, and M. Carpuat, “Understanding and detecting hallucinations in neural machine translation via model introspection,” _Transactions of the Association for Computational Linguistics_ , vol. 11, pp. 546–564, 2023.
  * [92] M. Fang, M. Huber, and N. Damer, “Synthaspoof: Developing face presentation attack detection based on privacy-friendly synthetic data,” in _Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition_ , 2023, pp. 1061–1070.
  * [93] N. Carlini, F. Tramer, E. Wallace, M. Jagielski, A. Herbert-Voss, K. Lee, A. Roberts, T. Brown, D. Song, U. Erlingsson, A. Oprea, and C. Raffel, “Extracting training data from large language models,” 2021.
  * [94] R. T. McCoy, P. Smolensky, T. Linzen, J. Gao, and A. Celikyilmaz, “How much do language models copy from their training data? evaluating linguistic novelty in text generation using raven,” _Transactions of the Association for Computational Linguistics_ , vol. 11, pp. 652–670, 2023.