# Dilepton Decay of Low-mass $\rho$ Mesons

K. Gallmeister<EMAIL_ADDRESS>Institut für
Theoretische Physik, Universität Giessen, Giessen, Germany  Helmholtz Research
Academy Hesse for FAIR (HFHF), Campus Giessen, Giessen, Germany  U. Mosel
Institut für Theoretische Physik, Universität Giessen, Giessen, Germany
Helmholtz Research Academy Hesse for FAIR (HFHF), Campus Giessen, Giessen,
Germany  L. von Smekal  Institut für Theoretische Physik, Universität Giessen,
Giessen, Germany  Helmholtz Research Academy Hesse for FAIR (HFHF), Campus
Giessen, Giessen, Germany

###### Abstract

The HADES collaboration has extracted dilepton mass spectra for $\pi^{-}$
induced reactions on the proton from a comparison of data taken on C and CH2
targets. The spectra were interpreted in terms of different versions of vector
meson dominance. Here we present results obtained from the theory and
generator GiBUU. We first check the subtraction procedure used and then
discuss the obtained mass spectra for the proton target. We point out that any
conclusions on the version of VMD requires the knowledge of the $\rho$
spectral function in the interesting mass region.

##  I Introduction

The radiative decay of nucleon resonances is governed by electromagnetic
transition form factors which take the finite extension of the resonance and
the nucleon into account. A very successful description of such photon-hadron
couplings and has been achieved within the vector-meson-dominance (VMD) model
Nambu and Sakurai (1962). In this model the coupling takes place through an
intermediate $\rho$ meson. As discussed in some detail in O’Connell  _et al._
(1997) there are two versions of VMD. In the more widely used version VMD2 (in
the notation of O’Connell  _et al._ (1997)) the coupling of the photon takes
place only through the $\rho$ meson, whereas in version VMD1 the coupling
amplitude employs in addition a coherently added term in which the photon
couples directly to the hadron. Both methods are fully equivalent if certain
relations between the coupling constants involved are met O’Connell  _et al._
(1997). Since these relations in nature hold only approximately the HADES
collaboration has recently tried to find experimental signatures for one or
the other VMD version Abou Yassine  _et al._ (2022).

The two VMD versions differ in their predicted dilepton invariant mass
($M_{e^{+}e^{-}}$) distribution mainly at small masses below the $2m_{\pi}$
threshold where the pion electromagnetic formfactor is not accessible. One
possibility to explore this low mass range is thus given by the Dalitz decay
of a nucleon resonance $N^{*}\to Ne^{+}e^{-}$; this decay is governed by the
electromagnetic transition formfactor in the time-like region. A conclusive
comparison of data with the VMD versions should be possible if the
$\rho$-spectral function in the mass range used for the comparison is known.

Therefore, merging the experimental information on $\rho$ production in the
$\pi^{-}+p\to\rho+n$ reaction from Ref. Adamczewski-Musch  _et al._ (2020)
with measured dilepton yields in the reaction $\pi^{-}+p\to e^{+}e^{-}+n$ from
Ref. Abou Yassine  _et al._ (2022) seems to offer an interesting possibility
to access also the low-mass region and to explore the validity of VMD there.

The recent HADES experiment aims to determine the dilepton yield in the
reaction $\pi^{-}p$ at dilepton invariant masses of about 100 MeV to 400 MeV,
i.e. around and below the $2\pi$ threshold Abou Yassine  _et al._ (2022). The
experiment was performed at an incoming pion momentum of $p_{\pi}=0.69$ GeV,
corresponding to $\sqrt{s}=1.49$ GeV. It did not directly use a proton target,
but instead obtained data for CH2 and (with lower statistics) for C. A
comparison of both then leads to the dilepton mass spectrum for H. Invariant
mass cuts, assuming a quasi-free reaction process, are used to isolate the
$n+e^{+}e^{-}$ final state.

The further analysis uses results of an earlier publication Adamczewski-Musch
_et al._ (2020) on a measurement and analysis of 2$\pi$ production in the
$\pi^{-}+p$ reaction. A partial wave analysis (PWA) of these results employing
the Bonn-Gatchina $K$-matrix model Bonn-Gatchina Partial Wave Analysis ;
Anisovich  _et al._ (2005) then led to the cross section for $\rho$ production
primarily through the D13 N(1520) resonance.

The data obtained in Abou Yassine  _et al._ (2022) do not show the strong rise
($\propto 1/M_{e^{+}e^{-}}^{3}$) towards small dilepton invariant masses
contained in the VMD2 model Larionov  _et al._ (2021). On the other hand, a
good fit to the data could be obtained by using VMD1 and fitting the relative
strength of the two components with a free parameter.

For such a comparison of the VMD versions the $\rho$ spectral function has to
be known in the region of interest. Crucial for this comparison is, therefore,
the continuation of the $\rho$ spectral function from an energy range, where
it has been measured by the $2\pi$ decay, to lower masses. The latter is
influenced by the decay N$(1520)\to\rho+n$, i.e. by a hadronic interaction
vertex independent of the electromagnetic coupling of the $\rho$ to the
virtual photon. In VMD1 an additional assumption about the electromagnetic
transition formfactor is required.

The purpose of this present paper is twofold. First, we investigate the steps
leading from the actual measurements, which were performed on heavier targets
(C, CH2), to the spectrum on a proton. Second, we have a closer look at the
conditions necessary to decide between VMD1 and VMD2.

##  II Model

We describe the reactions $\pi^{-}+C$ and $\pi^{-}+p$ within the quantum-
kinetic GiBUU theory and event generator. Both the theoretical foundations as
well as all the elementary reaction input and the numerical algorithms are
described in detail in Ref. Buss  _et al._ (2012). The source code used for
the present calculations can be obtained from GiBUU Website . All the special
equations for dilepton production are given in Ref. Larionov  _et al._ (2021).
As described there we use the VMD2 variant for the dilepton-decay of a $\rho$
meson.

Essential input for the calculations are the properties of the hadronic
interaction channels; for the problem at hand the relevant channel is
$\pi^{-}+p\to n+\rho$. In the present version of GiBUU these are taken from
the partial wave analysis of Manley and Saleski Manley and Saleski
(1992).111The HADES analysis of the 2$\pi$ cross sections in Adamczewski-Musch
_et al._ (2020) used the Bonn-Gatchina analysis which leads to somewhat
different decay widths. Contrary to the non-relativistic spectral function
employed there in the present calculations we use a relativistic form which –
when integrated over $m$ – is normalized to 1 for constant widths

$A(m)=\frac{2}{\pi}\,\frac{m^{2}\Gamma(m)}{(m^{2}-M^{2})^{2}+m^{2}\Gamma(m)^{2}}~{}.$
(1)

Here $M$ stands for the peak mass of either the $\rho$ meson or the N(1520)
resonance. The width $\Gamma$ is either that equal to the sum of dilepton and
pion decay widths for the $\rho$ meson or that of the nucleon resonance. We
use this Breit-Wigner form for the spectral function of the $\rho$ meson also
below the $2\pi$ threshold even though the physical character of this meson
becomes doubtful in this mass range.

In the Manley-Saleski analysis the decay width of a nucleon resonance $N^{*}$
with mass $M$ into $N\rho$ is given by the product of the phase-space element
with a Blatt-Weisskopf (BW) formfactor $B_{l}$,

$\Gamma^{*}(W,m)=\Gamma^{*}_{0}\,\frac{q_{\rho}(W,m)}{W}B_{l}^{2}(q_{\rho}(W,m)\,R)~{}.$
(2)

Here $q_{\rho}$ is the c.m. momentum of the decay products,

$\displaystyle q_{\rho}(W,m)$ $\displaystyle=$ (3) $\displaystyle\frac{1}{2W}$
$\displaystyle\sqrt{(W^{2}-(M_{N}+m)^{2})(W^{2}-(M_{N}-m)^{2})},$

$W$ is the invariant mass of the decaying $N^{*}$, $m$ is the running mass of
the $\rho$ meson and $M_{N}$ is that of the nucleon. The BW formfactor
regularizes the width at large $m$. It does not, however, limit the high-
momentum (low-mass) behavior.

In Fig. 1 we show the width for the decay $N(1520)\to\rho N$ at the invariant
mass at which the HADES experiment was performed.

Figure 1: Decay width of Eq. (2) for the decay $N(1520)\to N\rho$ as a
function of the $\rho$ mass $m$ at fixed $W=1.49$ GeV.

Being dominated by phase-space this width obviously favors the production of
high-momentum, very-low-mass $\rho$ mesons. This is in contrast to usually
used formfactors which are used to mimick the finite extension of hadrons and
thus cut off the high-momentum parts of a transition.

The Manley-Saleski analysis does not describe explicitly the secondary decay
of primary decay daughters. For the present calculations we use the expression
Post  _et al._ (2004)

$\displaystyle\Gamma_{\rho\to\pi^{+}\pi^{-}}(m)$ $\displaystyle=$
$\displaystyle\Gamma_{0}\,\frac{M_{\rho}^{2}}{m^{2}}\frac{q_{\pi}^{3}(m)}{q_{\pi}^{3}(M_{\rho})}\frac{1+[q_{\pi}(M_{\rho})R]^{2}}{1+[q_{\pi}(m)R]^{2}}$
(4) $\displaystyle\times\Theta(m-2m_{\pi})~{},$

with

$q_{\pi}(m)=\frac{1}{2}\sqrt{m^{2}-4m_{\pi}^{2}}~{},$ (5)

being the c.m. momentum of the pions.222Note that this definition of the
$\rho$ decay width from Post  _et al._ (2004) differs by a factor $M_{\rho}/m$
from the one used in Larionov  _et al._ (2021). For the dilepton spectra from
heavy-ion collisions this difference is irrelevant. Here $m$ is the ’running
mass’ of the $\rho$ meson and $M_{\rho}$ is its nominal mass at the peak of
the spectral function. Finally, the dilepton decay width is given by

$\displaystyle\Gamma_{\rho\to e^{+}e^{-}}(m)$ $\displaystyle=$ $\displaystyle
C_{\rho}\frac{M_{\rho}^{4}}{m^{3}}(1+2m_{e}^{2}/m^{2})\sqrt{1-4m_{e}^{2}/m^{2}}$
(6) $\displaystyle\mbox{}\times\Theta(m-2m_{e})~{}.$

Here $m_{e}$ is the electron mass and $C=9.078\times 10^{-6}$ is an empirical
coupling constant Larionov  _et al._ (2021).

##  III Results

We start our discussion with pointing out that the beam energy in this
experiment is so low that only the low-mass tail of the $\rho$ meson is
populated in the reaction $\pi^{-}+p\to n+\rho$ and even for the N(1520) the
full peak cannot be reached. This is illustrated in Fig. 2 showing the
spectral functions for the N(1520) resonance and the $\rho$ meson, the latter
shifted by the nucleon mass.

The experiment was run at an $\pi N$ invariant mass of 1.46 GeV (indicated by
a vertical bar in Fig. 2). For the present experiment even the very low mass
region below the $2\pi$ threshold of the $\rho$ meson ($m<1.22$ GeV) is
essential because it is here that the predictions of VMD1 and VMD2 differ most
in the analysis of Abou Yassine  _et al._ (2022). Fig. 2 shows that this
region lies in the extremely low low-mass tail of the $\rho$.

Figure 2: The spectral functions of the N(1520) resonance and the $\rho$
meson, the latter shifted by the nucleon mass. The vertical line at $m=1.46$
GeV shows the energy of the HADES experiment; the line at $m=1.22$ GeV denotes
the $2\pi$ threshold of the $\rho$ meson.

Here one has to keep in mind that the Breit-Wigner distribution for the $\rho$
meson emerges from a Taylor expansion of the $\pi\pi$ scattering amplitudes
around the peak mass and thus becomes the less reliable the farther the masses
are removed from the maximum.

###  III.1 $\pi^{-}p$ cross sections

In Fig. 3 we show for reference the calculated various dilepton contributions
for the reaction $\pi^{-}+p\rightarrow n+e^{+}e^{-}$, with the proton isolated
and at rest.

Figure 3: Invariant mass spectrum of dileptons in the reaction $\pi^{-}+p$ at
0.56 GeV lab energy. The individual contributions are indicated in the figure.

In the relevant mass range, starting at the upper end of the $\pi^{0}$ Dalitz
decay at around 0.14 GeV and extending to about 0.4 GeV, well above the two-
pion threshold, the dominant contributions are the radiative $\rho$ decay and
the $\eta$ Dalitz decay; at the lowest masses in this range also $\pi N$
bremsstrahlung is non-negligible.

We have checked the absolute magnitude of these cross sections by comparing
the cross section for $\pi^{-}+p\to\pi^{0}+n$ with experiment Baldini  _et
al._ (1988) (see Fig. 4); excellent agreement is reached.

Figure 4: Inclusive cross section for $\pi^{-}+p\to\pi^{0}+n$ as a function of
the incoming pion’s lab momentum. Data can be found in Baldini  _et al._
(1988).

This fixes the height of the $\pi^{0}$ Dalitz peak to the correct value.

###  III.2 Extraction of $\pi^{-}p$ cross sections
in the HADES experiment

The HADES experiment was not run on a proton target, but instead used an
approximate subtraction procedure based on measured missing mass distributions
for CH2 and C to isolate the proton contribution Adamczewski-Musch  _et al._
(2020). By cuts on these distributions, furthermore, the experiment has tried
to identify the particular contribution $\pi^{-}+p\to n+\rho\to n+e^{+}e^{-}$,
assuming a quasifree interaction with the protons in the target nucleus.

To check this procedure we first show in Fig. 5 a comparison of the missing
mass distribution obtained with GiBUU for the CH2 target. The results were
obtained by using the experimental HADES acceptance cuts333The HADES
acceptance cuts used were: $\theta_{i}=18\degree-85\degree,p_{i}>100$ MeV,
opening angle between leptons $>9\degree$..

Figure 5: Missing mass distribution for $\pi^{-}+CH_{2}$ for events with an
invariant mass larger than 140 MeV. Shown are the contribution of the $\rho$
meson (dashed, blue curve), of the $\eta$ meson (dotted, red), the $\pi N$
bremsstrahlung and the sum of all contributions (solid, red). The results of
the calculation include the effects of angular cuts according to the HADES
acceptance. The data points are from Abou Yassine  _et al._ (2022), they are
efficiency, but not acceptance corrected. The two vertical lines show the
positions of the missing-mass cuts employed by the experiment.

In agreement with the data the missing mass distribution shows two clearly
distinct peaks, one centered at the free neutron mass and another one at 1.15
GeV, representing the $\eta$ Dalitz-decay contribution. Both peaks also show
up in the experimental distribution. There is, however, a significant
disagreement in absolute height of the cross section; the GiBUU cross section
is larger than then experimental values by about a factor 3. The missing mass
distribution shows a clear peak at the neutron mass $M_{n}=0.938$ GeV while
the result of a simulation shown in Fig. 1 of Ref. Abou Yassine  _et al._
(2022) peaks at a higher mass. Part of this discrepancy in the mass is due to
a detector-efficiency related smearing. The overall magnitude of the cross
sections is related to additional acceptance limitations of the HADES
detector. Unfortunately, a filter routine that takes these acceptance
properties into account is still not available.

In the GiBUU calculations the CH2 cross section is simply given by the sum of
cross sections for C and for $2\times H$. In Fig. 6 we show first the dilepton
spectrum for CH2 since this spectrum is directly measured and does not suffer
from possible inaccuracies in the extraction of the proton cross section. The
calculation describes the data very well for masses above about 0.4 GeV, i.e.
above the $2\pi$ threshold. Going to smaller masses it becomes larger than the
data by about a factor 3. Surprising is the fact that this overestimate also
shows up in the $\pi^{0}$ Dalitz peak region since the elementary charge
exchange cross section for $\pi^{-}+p\to n+\pi^{0}$ is described very well.

The HADES analysis has then used a similar, but statistically less
constrained, distribution for a C target to deduce a factor of about 2.9
between the C and p targets Abou Yassine  _et al._ (2022). This factor is used
to normalize the final data on the proton target.

In Fig. 7 we show the dilepton invariant mass distribution for a 12C target
together with that for H. The cross section is described quite well for the
higher masses but overshoots the measured values for masses below about 0.4
GeV. The final state interactions have only a very small influence justifying
the assumption of a quasifree reaction process. The spectra are obtained after
performing the invariant mass cuts as in the experiment. It is seen that the
resulting distribution, given by the dashed red line, is higher than the
distribution obtained from the H target (blue solid curve) by about a factor
of 1.75, significantly smaller than the factor 2.9 used in the HADES analysis.
That this factor is less than the number of protons in the target (6) is due
to initial state interactions. That it is also lower than the factor obtained
in the HADES analysis we attribute to the fact that the results for C shown
here contain both the Fermi-motion of the nucleons and their binding energy.
It can also be seen that some $\eta$ Dalitz decay contribution survives. While
it is well suppressed at the higher masses, below about 200 MeV it becomes
comparable in magnitude to the data. We speculate that the slight upwards
trend seen in the data at the lowest masses could be due to this $\eta$
contamination.

Figure 6: Dilepton invariant mass distribution for $\pi^{-}+CH_{2}$. Shown are
the contribution of the $\rho$ meson (dashed, blue curve), of the $\eta$ meson
(dotted, red), the $\pi N$ bremsstrahlung and the sum of all contributions
(solid, red). The results of the calculation include the effects of angular
cuts according to the HADES acceptance. The data points are from Abou Yassine
_et al._ (2022), they are corrected for detector efficiencies, but restricted
to the HADES acceptance cuts given above. Thin black lines show results from
Abou Yassine  _et al._ (2022). Figure 7: Invariant mass spectrum of dileptons
for the $\pi^{-}C$ reaction at 0.56 GeV (dashed, green curve); the $\pi^{0}$
Dalitz decay contribution is subtracted. Shown also is the mass distribution
after a missing mass cut for the final state $n+e^{+}e^{-}$ (dashed red curve,
labeled Ccut) as well as the same for a proton target (blue, solid curve,
labeled H). The black data points are taken from Adamczewski-Musch  _et al._
(2020); they are acceptance corrected. The lowest dashed-dotted curve gives
the contribution from $\eta$ Dalitz decay after the invariant mass cut.

Summarizing the results discussed so far we find that the method used by the
HADES collaboration to isolate the $\rho$ contribution to the dilepton
spectrum works reasonably well with some $\eta$ contribution remaining at the
smallest masses. Any detailed comparison, however, requires the knowledge of
the HADES acceptance and efficiency.

###  III.3 Dilepton invariant mass distribution for H

After having discussed the preparation of the dilepton spectrum for H,
starting from data for 12C and CH2, we now discuss the mass-dependence of the
dilepton spectrum. As shown by the blue, solid curve in Fig. 7 the spectrum
for H is on average quite well described. However, it is somewhat smaller (by
about 30%) for the masses between 0.3 and 0.5 GeV and larger (by about 100%)
for the masses around 0.2 GeV.

In a model that links an observed dilepton yield to the $\rho$ meson (as in
VMD2, but note that also in VMD1 the $\rho$ strength enters) the overall
magnitude of the dilepton yield is directly proportional to the number of
$\rho$ mesons produced. Fig. 8 shows that the extracted experimental mass
distribution lies by about a factor of 1.3 above the GiBUU calculation.

Figure 8: The $\rho$ meson mass distribution for the $\pi^{-}+p$ reaction.
Black points give the $\rho$ contribution extracted from the $2\pi$ data
measured by HADES Adamczewski-Musch  _et al._ (2020) by means of the Bonn-
Gatchina partial wave analysis. The solid (red) curve gives the contribution
obtained from a GiBUU calculation.

This experimental result depends on the particular PWA used to extract the
$\rho$-contribution from the measured $2\pi$ cross sections (Bonn-Gatchina for
the HADES data, Manley for the GiBUU calculation). Tuning the $\rho$
production cross section to the Bonn-Gatchina values used by HADES the
dilepton invariant mass spectrum would improve the description of the data
above the $2\pi$ threshold while at the same time increasing the discrepancy
at smaller masses.

Note that in the region where the $\rho$ spectral function is known, i.e.
above the $2\pi$ threshold, the agreement of the calculation with the data is
quite reasonable. A comparison with the experimental results shown in Fig. 2b
of Abou Yassine  _et al._ (2022) shows that the tuned VMD1 calculation there
essentially agrees with the results obtained here for H.

##  IV Discussion

The very low mass behavior, below the $2\pi$ threshold, of the $N^{*}\to N$
transition form factor is of particular interest here. Faessler et al.
Faessler  _et al._ (2003), noting the too large values obtained for the
radiative decay of nucleon resonances, had pointed out that this behavior
could be cured by adding radial excitations of the $\rho$ meson. In effect,
such addition changes the power of the pion form factor from monopole to
dipole (or higher). Also the authors of Ref. Larionov  _et al._ (2021) had
mentioned that the simple VMD2 version of Eq. (6) is doubtful at small $m$.

In Ref. Abou Yassine  _et al._ (2022) the discrepancy of the measured dilepton
invariant mass distribution with the results of the VMD2 model (blue curve in
Fig. 7) has been interpreted as a failure of the VMD2 version of vector meson
dominance. It was indeed shown that agreement with the data could be obtained
by using VMD1 of vector meson dominance and tuning the mixing parameters, that
relate the direct photon-coupling amplitude to that for the $\rho$ coupling,
to the data Zétényi  _et al._ (2021). Alternatively, also a transition form
factor model developed by Ramalho and Pena Ramalho and Peña (2017) worked
quite well. The latter, based on a quark-core and pion-cloud model, was
actually predicted before the data became available.

Both of these models used an extrapolation of the $\rho$ spectral function
into the region below the $2\pi$ threshold. Below the $2\pi$ threshold,
however, the $\rho$ meson production cross section has not been measured. The
cross section there is a convolution of the $\rho$ spectral function $A(m)$
and the dilepton decay width

$\frac{d\sigma}{dm}\propto A(m)\,\Gamma_{\rho\to e^{+}e^{-}}~{}.$ (7)

Any discrepancy for $m<2m_{\pi}$ must, therefore, be attributed not
necessarily to the dilepton decay width, but can also be due to a change in
the spectral function $A(m)$ which in this mass region has not been measured.

On the other hand, using the Breit-Wigner distribution of the $\rho$ meson
also in the subthreshold region (as used in the GiBUU calculations, but also
in the experimental analysis) then requires to change the Dalitz decay of the
N(1520) resonance. As mentioned already earlier, at the end of Sect. 1, the
$N(1520)\to N\rho$ decay vertex contains only a phase-space factor, the Blatt-
Weisskopf formfactor does not appear in this case because the transition is an
$s$-wave process. In order to account for a finite size of the hadronic
transition one could thus introduce an additional formfactor which limits the
population of high-momentum, very-low-mass $\rho$’s. We note that similar
formfactors appear also in an application of VMD1, both at the $N^{*}N\rho$
vertex and the $N^{*}N\gamma$ vertex.

The additional form factor that multiplies the decay width of the N(1520)
resonance affects the population of the $\rho$ at small masses. There it can
as well be seen as a multiplicative factor to the dilepton decay width. In
effect then this factor just makes the constant $C_{\rho}$ in Eq. (6) mass
dependent.

##  V Conclusions

By comparing with a model (GiBUU) which contains both Fermi-motion and nuclear
binding we have shown that the isolation of dilepton spectra on H out of data
obtained for CH2 and C is reasonably reliable. In addition the identification
of the $\rho$ component in the dilepton spectrum, achieved by missing mass
cuts, works well enough. In the absence of a realistic acceptance filter,
however, quantitative differences between the normalization of the C and the H
data still remain.

The obtained dilepton spectra deviate from the naive prediction of the vector
meson dominance model VMD2. While the authors of Abou Yassine  _et al._ (2022)
have taken this as evidence against the validity of VMD2 in the present paper
we have pointed out that these data can also be explained by either a
formfactor at the $N^{*}N\rho$ vertex or, equivalently, by a mass dependence
of the constant $C$ in the dilepton spectrum with still using VMD2. This
explanation, as well as others based on VMD1, assumes that the Breit-Wigner
distribution is still valid at very (unmeasured) low masses, far away from the
peak mass. Alternatively the data could also be explained – within VMD2 – by a
change in the spectral function of the $\rho$ meson at these low masses.

It is thus clear that a distinction between VMD1 and VMD2 can only be made if
the $\rho$ meson spectral function is experimentally known. In the HADES
experiment this is true only in the region above the $2\pi$ decay threshold.
It is there, however, where VMD1 and VMD2 hardly differ even in the tune shown
in Abou Yassine  _et al._ (2022).

Finally, we note that while the GiBUU calculation indeed ascribes the dilepton
yield mostly to a decay of a $\rho$ meson, originating in the N(1520) decay to
the nucleon, it is not at all clear if this is also the case in the
experimental data. Indeed, the dileptons observed in the $\pi^{-}+p\to
n+e^{+}e^{-}$ reaction can originate in a number of different elementary
processes involving different hadronic transition form factors (see Fig. 40 in
Ref. Perez-Y-Jorba and Renard (1977)).

###### Acknowledgements.

We are grateful for a close collaboration with Alexei Larionov in the early
stage of this investigation. We also gratefully acknowledge helpful
discussions with Stefan Leupold, Volker Metag, Beatrice Ramstein and Piotr
Salabura. This work was supported by BMBF grant no. 05P21RGFCA.

## References

  * Nambu and Sakurai (1962) Y. Nambu and J. J. Sakurai, Phys. Rev. Lett. 8, 79 (1962).
  * O’Connell  _et al._ (1997) H. B. O’Connell, B. C. Pearce, A. W. Thomas, and A. G. Williams, Prog. Part. Nucl. Phys. 39, 201 (1997), arXiv:hep-ph/9501251 .
  * Abou Yassine  _et al._ (2022) R. Abou Yassine _et al._ (HADES), (2022), arXiv:2205.15914 [nucl-ex] .
  * Adamczewski-Musch  _et al._ (2020) J. Adamczewski-Musch _et al._ (HADES), Phys. Rev. C 102, 024001 (2020), arXiv:2004.08265 [nucl-ex] .
  * (5) Bonn-Gatchina Partial Wave Analysis, http://pwa.hiskp.uni-bonn.de/.
  * Anisovich  _et al._ (2005) A. Anisovich, E. Klempt, A. Sarantsev, and U. Thoma, Eur. Phys. J. A 24, 111 (2005), arXiv:hep-ph/0407211 .
  * Larionov  _et al._ (2021) A. B. Larionov, U. Mosel, and L. von Smekal, Phys. Rev. C 102, 064913 (2021), arXiv:2009.11702 [nucl-th] .
  * Buss  _et al._ (2012) O. Buss, T. Gaitanos, K. Gallmeister, H. van Hees, M. Kaskulov, _et al._ , Phys.Rept. 512, 1 (2012), arXiv:1106.1344 [hep-ph] .
  * (9) GiBUU Website, http://gibuu.hepforge.org.
  * Manley and Saleski (1992) D. M. Manley and E. M. Saleski, Phys. Rev. D45, 4002 (1992).
  * Post  _et al._ (2004) M. Post, S. Leupold, and U. Mosel, Nucl. Phys. A 741, 81 (2004), arXiv:nucl-th/0309085 .
  * Baldini  _et al._ (1988) A. Baldini, V. Flaminio, W. G. Moorhead, and D. R. O. Morrison,  _Total Cross-Sections for Reactions of High Energy Particles (Including Elastic, Topological, Inclusive and Exclusive Reactions)_, edited by H. Schopper, Landolt-Boernstein - Group I Elementary Particles, Nuclei and Atoms, Vol. 12a (Springer, 1988).
  * Faessler  _et al._ (2003) A. Faessler, C. Fuchs, M. I. Krivoruchenko, and B. V. Martemyanov, J. Phys. G 29, 603 (2003), arXiv:nucl-th/0010056 .
  * Zétényi  _et al._ (2021) M. Zétényi, D. Nitt, M. Buballa, and T. Galatyuk, Phys. Rev. C 104, 015201 (2021), arXiv:2012.07546 [nucl-th] .
  * Ramalho and Peña (2017) G. Ramalho and M. T. Peña, Phys. Rev. D 95, 014003 (2017), arXiv:1610.08788 [nucl-th] .
  * Perez-Y-Jorba and Renard (1977) J. P. Perez-Y-Jorba and F. M. Renard, Phys. Rept. 31, 1 (1977).


# Initial Exploration of Zero-Shot Privacy Utility Tradeoffs in Tabular Data
Using GPT-4

Bishwas Mandal  Department of Computer Science
Kansas State University
Manhattan, KS 66506, USA
<EMAIL_ADDRESS>George Amariucai  Department of Computer Science
Kansas State University
Manhattan, KS 66506, USA
<EMAIL_ADDRESS>Shuangqing Wei  Division of Electrical & Computer
Engineering
Louisiana State University
Baton Rouge, LA 70803, USA
<EMAIL_ADDRESS>
###### Abstract

We investigate the application of large language models (LLMs), specifically
GPT-4, to scenarios involving the tradeoff between privacy and utility in
tabular data. Our approach entails prompting GPT-4 by transforming tabular
data points into textual format, followed by the inclusion of precise
sanitization instructions in a zero-shot manner. The primary objective is to
sanitize the tabular data in such a way that it hinders existing machine
learning models from accurately inferring private features while allowing
models to accurately infer utility-related attributes. We explore various
sanitization instructions. Notably, we discover that this relatively simple
approach yields performance comparable to more complex adversarial
optimization methods used for managing privacy-utility tradeoffs. Furthermore,
while the prompts successfully obscure private features from the detection
capabilities of existing machine learning models, we observe that this
obscuration alone does not necessarily meet a range of fairness metrics.
Nevertheless, our research indicates the potential effectiveness of LLMs in
adhering to these fairness metrics, with some of our experimental results
aligning with those achieved by well-established adversarial optimization
techniques.

###### Index Terms:

privacy-utility tradeoff, large language models, inference privacy,
adversarial optimization, fairness.

##  I Introduction

The rapid growth observed in the deep learning field can be attributed to a
range of factors, such as advancements in architectural designs, improved
convergence capabilities, and the widespread availability of open-source
models, among others. However, the primary catalyst driving the significant
growth of deep learning is the accessibility of extensive datasets and the
continuous enhancement of computational capabilities [1]. One specific
architectural advancement has made a substantial contribution to the effective
utilization of GPUs for efficient computational usage: the Transformer
architecture [2]. The transformer architecture serves as the foundational
framework for numerous recent advancements, including the development of
various Large Language Models (LLMs). These LLMs have popularized the practice
of fine-tuning language models to a much greater extent than in the past.

LLMs are characterized by their massive architecture, with models having
billions of parameter (e.g GPT-3 model with 175 billion parameters [3]).
Training these models involves working with extensive datasets that tap into a
wide array of information from diverse data sources. By capitalizing on
insights derived from these varied sources, LLMs have a tendency to amass a
comprehensive range of knowledge, rendering them highly adaptable to a diverse
array of tasks. Therefore, they have found application across various domains,
including applications in sentiment analysis, text generation, code
generation, multimodal tasks, and more. They have also proven effective in
wide array of tasks in low-data scenarios such as zero-shot and few-shot
settings [3]. LLMs have not only demonstrated their effectiveness in handling
unstructured data but have also exhibited remarkable performance when applied
to structural tabular datasets within zero-shot and few-shot settings [4].

LLMs pose a significant challenge when it comes to preserving privacy. One of
the primary concerns is their inadvertent potential to reveal information
about the training data [5]. This vulnerability implies that malicious
individuals could potentially access the training data, which may contain
sensitive information, leading to privacy breaches. Additionally, LLMs have
made substantial advancements in their ability to draw inferences, potentially
allowing them to deduce various personal attributes of users. [6] demonstrate
how large language models can deduce sensitive information from users during
the inference phase. Therefore, privacy issues regarding LLMs go beyond just
extracting training data; they also involve privacy violations due to LLMs’
powerful inference capabilities.

Several studies [7, 8] have proposed methods, to protect LLMs from revealing
their training data. These methods focus on modifying the model’s parameters
using different techniques to enhance privacy. However, as of our current
knowledge, there hasn’t been prior research exploring whether LLMs can
effectively be used to safeguard user privacy concerning their ability to make
inferences. Therefore, our paper shifts its focus towards investigating the
potential of LLMs to leverage their inference abilities and statistical
insights to maintain user privacy, especially when dealing with tabular
datasets. Our main objective is to use LLMs, especially GPT-4, to sanitize
datasets in a way that hinders the extraction of sensitive user information
while still retaining the ability to extract useful features. To the best of
our knowledge, this study represents the first attempt to explore how LLMs can
be employed to enhance the privacy of tabular datasets while preserving their
usefulness.

Figure 1: Overview of the proposed data sanitization technique using GPT-4.
Our approach involves initializing the GPT-4 model with a prompt that includes
the conversion of tabular data points into textual format, followed by the
incorporation of sanitization instructions. Additionally, we incorporate
formatting instructions into the prompt to ensure that the generated outputs
are suitable for programmatic utilization. The presented prompt above pertains
to the prompt designated for Task 1.

##  II Related Work

Within the sphere of privacy, two fundamental categories have been
established: data privacy and inference privacy [9, 10]. Data privacy
primarily focuses on preserving the original, unaltered data in its raw state.
Inference privacy deals with the protection of sensitive information that can
be derived or deduced from the disclosed raw data, often as a consequence of
correlations. In this context, raw data denotes the initial data collected or
generated, which remains unaltered and represents the data in its fundamental
state before undergoing any form of examination, modification, or inference.

In the broader discourse concerning privacy implications in LLMs, a central
concern revolves around the identification of the training data. This aligns
with the paradigm of data privacy, wherein the actual raw data contains
sensitive information, and the extraction of this data poses a substantial
privacy risk. [5, 11, 12], show diverse methods for extracting training data
from language models. To mitigate the risk of extraction of private training
data, [7] introduce a method for privately fine-tuning large language models
using Differential Privacy, and [8] further demonstrate the fine-tuning of
LLMs with enhanced privacy. In a similar vein, [13] propose a privacy neuron
detector designed to identify neurons associated with private information and
subsequently modify these identified privacy neurons by setting their
activations to zero. Furthermore, [14] enhance the privacy of language models
through reinforcement learning, utilizing negative similarity scores to
promote paraphrasing and reduce dependence on the original training data. [15]
utilize a straightforward sanitization step aimed at mitigating privacy risks
in retrieval-based language models.

It is important to note that these methods primarily address the issue of data
privacy and do not directly tackle the core focus of our research, which
centers on inference privacy. [6] demonstrate privacy concerns related to LLMs
extend beyond mere memorization of training data and how the improved
efficiency of LLMs in inference can be leveraged to infer private attributes
from user texts during inference. While LLMs have predominantly focused on
privacy from an unstructured data perspective (e.g text, image, etc.), [4]
show LLMs can be utilized for classification tasks in zero-shot and few-shot
settings in tabular datasets as well. The statistical knowledge acquired by
LLMs extends beyond textual data, potentially posing privacy risks in tabular
datasets, as demonstrated by [6], where GPT-4 is capable to infer private
attributes such as birth place, race, education level, income, and gender with
a high accuracy from the ACS Income Dataset [16] in a zero-shot setting.

Various adversarial optimization techniques [17, 18, 19, 20, 21, 22, 23, 24,
25, 26, 27, 28, 29], have been utilized to tackle the inherent trade-offs
between safeguarding inference privacy and preserving data utility. These
techniques either introduce additional noise into their generator network or
latent variables or employ loss functions that achieve distortion without
explicitly adding noise. They have proven their effectiveness in thwarting
Machine Learning (ML) models from inferring sensitive attributes like race,
gender, income, and others depending on the application. Moreover, these
methods are adaptable and can be applied to different data types, including
tables, images, and text (representations). The primary focus of these
methodologies has revolved around ensuring inference privacy, with the aim of
preventing the revelation of sensitive attributes or meeting fairness criteria
within ML models. However, as pointed out by [29], a notable portion of this
research has not thoroughly addressed the unique characteristics of the data
and the requisite post-processing steps when dealing with tabular datasets
that incorporate categorical features. They emphasize that sanitized datasets
generated through diverse privacy mechanisms often necessitate additional
post-processing efforts to restore the data to its original format.

##  III Problem Formulation and Methodology

Problem: Let’s consider the scenario of an external service provider tasked
with the responsibility of sanitizing a dataset to both protect user privacy
and ensure the desired utility is preserved. This provider has access to a raw
database containing a data vector, denoted as $D$, which consists of various
features represented as $\\{d_{1},d_{2},d_{3},\ldots,d_{n}\\}$. The primary
objective for this provider is to sanitize the dataset $D$ in such a way that
when users release their sanitized data, denoted as $\hat{D}$, it enables
accurate inference of utility features, denoted as $D_{U}$, while
simultaneously making it difficult to accurately infer private features,
denoted as $D_{P}$. Initially, the dataset $D$ exhibits correlations with both
$D_{U}$ and $D_{P}$ and it’s important to note that $D_{U}\notin D$ and
$D_{P}\notin D$. Failing to properly sanitize and releasing $D$ publicly could
lead to the accurate inference of private features, thus compromising user
privacy. In a public domain, potential attackers might utilize various pre-
trained machine learning models to deduce private features from the data.
Therefore, it is crucial for the sanitizing function $f$ to be robust against
multiple models to ensure effective privacy protection. In the context of this
paper, _privacy is construed as the pretrained model’s incapacity to
accurately deduce private attributes, whereas utility is characterized as the
pretrained model’s competence in accurately deducing utility attributes_.

Prior research has primarily focused on adversarial optimization approaches to
identify an effective sanitizing function $f$, or other theoretical techniques
involving noise addition [30]. In contrast, our approach explores the
potential of using GPT-4 as a viable sanitizing function $f$.

Proposed Methodology: We now detail the methodology for employing GPT-4 as a
sanitizing function $f$. In this specific context, the function $f$ is a
composite of two distinct functions, namely $p$ and $g$, expressed as
$f(.)=g(p(.))$. Here, $p$ is responsible for the prompting mechanism, while
$g$ represents the pre-trained GPT-4 model. The prompting function $p$
operates by taking the original tabular data point as its input and generating
a prompt. The prompting function $p$ encompasses several sequential steps
elaborated below:

  1. 1.

Translation to Text: The first phase involves converting tabular data into
text. [4] demonstrate that different techniques for this conversion produce
similar outcomes, as long as all the information from the tabular data is
retained. Therefore, we adopt a straightforward technique for this conversion.
The exact prompt is illustrated in Fig. 1, where the variables {age,
workclass, …, occupation} represent all features in vector $D$.

  2. 2.

Suffix with Supervision: This stage involves annotating the data $D$ with
accurate private and utility labels $D_{P}$ and $D_{U}$, respectively. This
annotated information is subsequently incorporated into the prompt, as
depicted in Fig. 1. In this illustration, $D_{P}$ corresponds to gender, while
$D_{U}$ corresponds to income.

  3. 3.

Sanitization Instruction: This portion involves the addition of prompts that
instruct the model to sanitize the dataset $D$ in order to achieve the
specified objectives related to privacy and utility. We conducted an
evaluation using two distinct prompts, namely $P1$ and $P2$. The precise
prompts are displayed in Fig. 1.

  4. 4.

Output Formatting Instruction: The final part of the prompt entails
incorporating instructions for arranging the output in a specific sequence,
ensuring that the outcome from the GPT-4 function $g$ aligns with our desired
format and can be seamlessly added to the database as $\hat{D}$.

Upon completion of the function $p$ transforming the tabular data into a
comprehensive prompt that encompasses all necessary guidelines for data
sanitization, the resulting output of $p$ is then supplied to the function
$g$. The role of function $g$ is to produce the sanitized dataset, denoted as
$\hat{D}$. In formal terms, this process is represented by the equation
$\hat{D}=g(p(D,D_{U},D_{P}))$, indicating the sequential application of
functions $p$ and $g$ to achieve the final sanitized dataset.

###  III-A Assumptions and Threat Model

In our conceptual framework, we establish a foundational trust relationship
between the data owner and the third-party service provider. This trust
empowers the data owner to confidently share raw data with the service
provider, who is responsible for data sanitization. Analysts or potential
attackers seeking to extract private or utility features from the data
$\hat{D}$ possess an auxiliary dataset at their disposal. This auxiliary
dataset enables them to initially train various machine learning models that
can predict the private and utility features. After these models are trained,
analysts proceed to attempt the inference of private and utility features for
the users whose data is contained within $\hat{D}$.

###  III-B Existing Sanitization Mechanisms (Baselines)

Figure 2: ALFR and UAE-PUPET architecture

In order to evaluate the effectiveness of GPT-4 as a privacy-preserving tool,
we draw comparisons with two established privacy mechanisms that employ
adversarial optimization techniques: ALFR [17], and UAE-PUPET [29]. These
methods are characterized by a generator-discriminator framework, depicted in
Fig. 2. Within this architecture, the generator is a generative model
comprising an encoder-decoder pair, while the discriminator is made up of
neural network models tasked with classifying private and utility features.
The discriminator’s role is crucial, providing the necessary loss function
that ensures the sanitized data $\hat{D}$ conceals private features while
maintaining the inferability of utility features.

For our analysis, we employ the official implementation [31] of UAE-PUPET to
produce the results presented in our paper. In the case of ALFR, we adopt the
sanitization algorithm outlined in their paper and integrate it with the model
architecture used in UAE-PUPET to ensure consistency. Algorithm 1 demonstrates
the ALFR sanitization mechanism. In this algorithm, $f$, $c_{p}$, and $c_{u}$
correspond to functions associated with generator, private feature classifier,
utility feature classifier networks. In parallel $\gamma$, $\gamma_{p}$, and
$\gamma_{u}$ denote the weights of these functions, respectively.
Additionally, $\alpha$, $\lambda_{p}$, and $\lambda_{u}$ are scalars utilized
to prioritize specific loss components in the final loss function L.

Algorithm 1 ALFR training algorithm

$\gamma,\gamma_{p},\gamma_{u}\leftarrow$ initialize weights

$U\leftarrow True$

repeat

$\hat{D}\leftarrow f(D)$, $\hat{D}_{P}\leftarrow c_{p}(\hat{D})$,
$\hat{D}_{U}\leftarrow c_{u}(\hat{D})$

$C\leftarrow\text{MSE}(\hat{D},D)$

$l_{p}\leftarrow\text{cross\\_entropy}(\hat{D}_{P},D_{P})$

$l_{u}\leftarrow\text{cross\\_entropy}(\hat{D}_{U},D_{U})$

$L\leftarrow\alpha C-\lambda_{p}l_{p}+\lambda_{u}l_{u}$

if $U$ then

Update $\gamma_{p}$ to _maximize_ the loss L

else

Update ($\gamma,\gamma_{u}$) to _minimize_ the loss L

end if

$U\leftarrow$ not $U$

until Deadline

##  IV Experimental Setup

###  IV-A Dataset

We conduct our experiment using the UCI Adult dataset [32]. Our dataset
selection process involved reviewing various tabular datasets initially used
for fairness studies, such as COMPAS, Bank Marketing, Communities and Crime,
Student Performance, and German Credit Dataset, as well as others mentioned in
[4] like Blood, Callhousing, Car, Diabetes, Heart, and Jungle. The selection
of the UCI Adult dataset was motivated by several considerations. Firstly, we
required a dataset that encompassed a minimum of two dependent variables
capable of serving as private and utility features. Secondly, our research
objective involved the development of machine learning models that would not
be able to accurately infer private features following data sanitization.
Consequently, it was imperative to have at least two dependent variables with
high initial accuracy to enable us to assess whether there was a notable
alteration in model accuracy subsequent to data sanitization.

Furthermore, we sought a dataset with high zero-shot classification scores
when utilized with LLMs, signifying that the language model possess
statistical information pertaining to that dataset. To fulfill all these
criteria, the UCI Adult dataset emerged as the only suitable choice. Although
the ACS Income Dataset met these criteria as well, we ultimately opted
exclusively for the UCI Adult dataset due to its similar nature and the fact
that the inclusion of the ACS dataset would not significantly diversify our
experiments. In order to introduce diversity into our research endeavors, we
devised two distinct tasks, designated as Task 1 and Task 2.

#### Task 1

Task 1 focuses on the UCI Adult dataset, designating Gender as the private
feature and the prediction of whether an individual’s income exceeds $50,000
as the utility feature, similar to the work in [29]. In this task, all
features apart from the private and utility ones are classified as variables
requiring sanitization, denoted as $D$ in our problem formulation.

#### Task 2

In Task 2, although the features designated for sanitization ($D$) stay the
same, there’s a role reversal between the private and utility features. In
this context, the private feature is now the classification task determining
whether an individual’s income exceeds $50,000, and the Gender is considered
the utility feature. This modification is intentionally made to demonstrate
that the outcomes of our research are not influenced by the specific selection
of a feature as private or another as utility and therefore, confirms the
effectiveness of our proposed methodology, irrespective of which variable is
designated as private or utility.

###  IV-B Performance Evaluation Metrics

To evaluate the efficacy of our sanitization mechanism, we proceed to compute
the accuracy and F1 scores for an array of classifier models. Initially, we
calculate these scores using the dataset in its unsanitized state.
Subsequently, we assess the performance of these models after the data has
undergone sanitization. Our expectation is that the accuracy and F1 scores for
the utility feature should exhibit minimal decline in comparison to their
original scores when the data remains unsanitized. However, for the private
feature, our objective is to achieve a significant reduction in these scores.

Additionally, we employ privacy-utility tradeoff metrics introduced in [33].
They define privacy leakage for attribute $p$ as follows:

$M_{p}=\frac{c_{a}(p)-c_{r}(p)}{c_{n}(p)-c_{r}(p)}$ (1)

Similarly, utility performance of attribute $u$, is defined as follows:

$M_{u}=\frac{c_{a}(u)-c_{r}(u)}{c_{n}(u)-c_{r}(u)}$ (2)

We ensure that these metrics are constrained within the range of 0 to 1. In
Eq. 1, Eq. 2, $c_{n}(.)$ represents the accuracy of a classifier in predicting
either the private feature $p$ or the utility feature $u$ using raw,
unsanitized data. $c_{a}(.)$ denotes the accuracy after the sanitization
process, while $c_{r}(.)$ signifies the accuracy associated with random
guessing. A lower value of $M_{p}$ indicates enhanced privacy, as it implies a
closer resemblance to random guessing. Conversely, a higher value of $M_{u}$
is preferable, signifying a lower drop in utility. It is noteworthy that the
accuracy for random guessing in predicting Income is 0.74, while for Gender,
it is 0.69 (these are unbalanced classification problems).

Privacy |  Models | Task 1 | Task 2
---|---|---|---
Gender (Private) | Income (Utility) | Income (Private) | Gender (Utility)
Acc | F1 | Acc | F1 | Acc | F1 | Acc | F1
No PM |  LR | $0.83_{.002}$ | $0.84_{.002}$ | $0.85_{.003}$ | $0.84_{.003}$ | $0.85_{.003}$ | $0.84_{.003}$ | $0.83_{.002}$ | $0.84_{.002}$
RF | $0.84_{.002}$ | $0.84_{.003}$ | $0.87_{.002}$ | $0.85_{.003}$ | $0.87_{.002}$ | $0.85_{.003}$ | $0.84_{.002}$ | $0.84_{.003}$
XGB | $0.84_{.002}$ | $0.84_{.003}$ | $0.88_{.003}$ | $0.87_{.003}$ | $0.88_{.003}$ | $0.87_{.003}$ | $0.84_{.002}$ | $0.84_{.003}$
NN | $0.83_{.002}$ | $0.81_{.002}$ | $0.85_{.003}$ | $0.84_{.003}$ | $0.85_{.003}$ | $0.84_{.003}$ | $0.83_{.002}$ | $0.81_{.002}$
GPT-4 (C) | $0.80_{.002}$ | $0.78_{.002}$ | $0.80_{.003}$ | $0.80_{.003}$ | $0.80_{.003}$ | $0.80_{.003}$ | $0.80_{.002}$ | $0.78_{.002}$
Summary | $0.84_{.002}$ | $0.84_{.002}$ | $0.88_{.003}$ | $0.87_{.003}$ | $0.88_{.003}$ | $0.87_{.003}$ | $0.84_{.002}$ | $0.84_{.002}$
ALFR |  LR | $0.65_{.006}$ | $0.64_{.007}$ | $0.81_{.006}$ | $0.79_{.004}$ | $0.75_{.007}$ | $0.70_{.005}$ | $0.81_{.003}$ | $0.81_{.009}$
RF | $0.65_{.010}$ | $0.63_{.009}$ | $0.81_{.007}$ | $0.79_{.006}$ | $0.75_{.006}$ | $0.68_{.008}$ | $0.80_{.005}$ | $0.80_{.005}$
XGB | $0.64_{.005}$ | $0.63_{.007}$ | $0.81_{.008}$ | $0.80_{.005}$ | $0.72_{.007}$ | $0.68_{.009}$ | $0.80_{.006}$ | $0.80_{.007}$
NN | $0.65_{.007}$ | $0.65_{.008}$ | $0.81_{.010}$ | $0.79_{.008}$ | $0.75_{.006}$ | $0.71_{.007}$ | $0.79_{.004}$ | $0.80_{.007}$
GPT-4 (C) | $0.65_{.006}$ | $0.62_{.007}$ | $0.75_{.005}$ | $0.75_{.005}$ | $0.66_{.008}$ | $0.67_{.012}$ | $0.78_{.007}$ | $0.75_{.011}$
Summary | $\mathbf{0.65}_{.006}$ | $\mathbf{0.65}_{.008}$ | $0.81_{.006}$ | $0.80_{.005}$ | $0.75_{.006}$ | $0.71_{.007}$ | $0.81_{.003}$ | $0.81_{.009}$
UAE-PUPET |  LR | $0.67_{.009}$ | $0.65_{.008}$ | $0.79_{.007}$ | $0.78_{.005}$ | $0.73_{.011}$ | $0.67_{.005}$ | $0.81_{.005}$ | $0.81_{.004}$
RF | $0.67_{.006}$ | $0.64_{.011}$ | $0.80_{.006}$ | $0.79_{.006}$ | $0.74_{.008}$ | $0.67_{.006}$ | $0.81_{.004}$ | $0.81_{.006}$
XGB | $0.66_{.008}$ | $0.64_{.010}$ | $0.78_{.004}$ | $0.77_{.004}$ | $0.72_{.006}$ | $0.69_{.009}$ | $0.82_{.005}$ | $0.82_{.003}$
NN | $0.66_{.007}$ | $0.64_{.008}$ | $0.79_{.008}$ | $0.78_{.008}$ | $0.71_{.006}$ | $0.68_{.009}$ | $0.81_{.007}$ | $0.81_{.008}$
GPT-4 (C) | $0.65_{.008}$ | $0.60_{.007}$ | $0.75_{.006}$ | $0.75_{.006}$ | $0.60_{.009}$ | $0.62_{.010}$ | $0.78_{.006}$ | $0.75_{.006}$
Summary | $0.67_{.006}$ | $\mathbf{0.65}_{.008}$ | $0.80_{.006}$ | $0.79_{.006}$ | $0.74_{.008}$ | $0.69_{.009}$ | $\mathbf{0.82}_{.005}$ | $\mathbf{0.82}_{.003}$
GPT4 (P1) |  LR | $0.36_{.006}$ | $0.30_{.005}$ | $0.88_{.004}$ | $0.88_{.004}$ | $0.46_{.008}$ | $0.49_{.008}$ | $0.81_{.004}$ | $0.82_{.007}$
RF | $0.65_{.005}$ | $0.66_{.007}$ | $0.79_{.006}$ | $0.74_{.005}$ | $0.63_{.005}$ | $0.58_{.010}$ | $0.81_{.008}$ | $0.81_{.006}$
XGB | $0.35_{.008}$ | $0.55_{.006}$ | $0.86_{.005}$ | $0.86_{.005}$ | $0.67_{.006}$ | $0.67_{.006}$ | $0.79_{.007}$ | $0.80_{.007}$
NN | $0.47_{.008}$ | $0.46_{.005}$ | $0.79_{.005}$ | $0.80_{.005}$ | $0.47_{.011}$ | $0.50_{.009}$ | $0.79_{.004}$ | $0.80_{.005}$
GPT-4 (C) | $0.60_{.007}$ | $0.61_{.005}$ | $0.89_{.004}$ | $0.89_{.005}$ | $0.35_{.013}$ | $0.39_{.009}$ | $0.79_{.006}$ | $0.78_{.007}$
Summary | $\mathbf{0.65}_{.005}$ | $0.66_{.007}$ | $\mathbf{0.89}_{.004}$ | $\mathbf{0.89}_{.005}$ | $\mathbf{0.67}_{.006}$ | $\mathbf{0.67}_{.006}$ | $0.81_{.004}$ | $\mathbf{0.82}_{.007}$
GPT4 (P2) |  LR | $0.75_{.004}$ | $0.76_{.005}$ | $0.88_{.005}$ | $0.88_{.005}$ | $0.71_{.006}$ | $0.72_{.008}$ | $0.79_{.003}$ | $0.80_{.005}$
RF | $0.73_{.004}$ | $0.74_{.007}$ | $0.87_{.007}$ | $0.87_{.006}$ | $0.72_{.011}$ | $0.73_{.009}$ | $0.78_{.006}$ | $0.79_{.007}$
XGB | $0.74_{.008}$ | $0.75_{.007}$ | $0.85_{.005}$ | $0.86_{.005}$ | $0.74_{.011}$ | $0.75_{.012}$ | $0.77_{.007}$ | $0.78_{.007}$
NN | $0.73_{.006}$ | $0.74_{.006}$ | $0.86_{.008}$ | $0.86_{.009}$ | $0.69_{.010}$ | $0.71_{.007}$ | $0.78_{.005}$ | $0.78_{.005}$
GPT-4 (C) | $0.73_{.004}$ | $0.73_{.005}$ | $0.83_{.005}$ | $0.83_{.006}$ | $0.62_{.008}$ | $0.64_{.010}$ | $0.77_{.005}$ | $0.77_{.006}$
Summary | $0.75_{.004}$ | $0.76_{.005}$ | $0.88_{.005}$ | $0.88_{.005}$ | $0.74_{.011}$ | $0.75_{.012}$ | $0.79_{.003}$ | $0.80_{.005}$
TABLE I:  Accuracy and F1 scores of multiple machine learning models evaluated
both before and after the application of the sanitization mechanism. No PM
denotes the scores obtained prior to the implementation of any sanitization
mechanism.

Since existing adversarial optimization techniques are commonly applied to
achieve fair representation learning, we also investigate fairness metrics to
evaluate whether our mechanism complies with various fairness standards. We
consider three fairness metrics: equalized odds, equalized opportunity, and
demographic parity. In brief, equalized odds mandates that a classifier
maintains equal true positive rates and false positive rates across different
sensitive groups. Equal opportunity, a more relaxed version of equalized odds,
is satisfied if the classifier exhibits equal true positive rates for all
sensitive groups, without specific consideration for false positive rates.
Similarly, demographic parity, also referred to as statistical parity,
necessitates that the decision rate (the rate of predicting the positive
class) remains consistent across all groups, irrespective of the base rates of
the outcome within those groups [34]. Note that, lower scores for these
metrics indicate better fairness.

###  IV-C Implementation Details

In our research, we access the GPT-4 model through OpenAI’s API, specifically
utilizing the gpt-4-1106-preview model and its chat completions functionality.
We utilize the default system prompt. The model is currently subject to a
daily token limit of 500,000, within which our prompt equates to roughly 710
data points. Moreover, we also use GPT-4 as a zero-shot classifier. The exact
prompt used for classification tasks is shown in Fig. 3, again with the
default system prompt. The inspiration for the prompt’s prefix comes from [6],
and as noted in [4], varying prompts don’t significantly alter results,
provided that all necessary information is clearly conveyed to the model. This
classification prompt facilitates the use of about 2600 data points within the
500,000-token limit. Our reported results are based on a temperature parameter
set to 0.1, and we found that varying this parameter did not significantly
alter the core outcomes of our study.

Figure 3: GPT-4 zero-shot classification prompt, GPT-4 (C)

Figure 4: Fairness metrics for Task 1 and Task 2. The X-axis represents
different ML models, and the Y-axis represents the fairness scores for those
ML models, without sanitization (No PM), and after using different
sanitization techniques.

##  V Result and Discussions

In this section, we present the results obtained from our proposed
methodology. We conduct a total of five independent experiments using
different random seeds to calculate the mean and standard deviations of the
results. Our test dataset comprises 1000 data points, a selection made with
consideration to the associated costs with using the Open AI API and daily
token limits. Table I shows the mean accuracy and F1 scores obtained by
various models before and after the implementation of the sanitization
mechanism for both tasks (standard deviation in the subscript). The Summary in
each segment, such as No PM, ALFR, etc., indicates the highest accuracy and F1
scores achieved among the machine learning models employed, including Logistic
Regression (LR), Random Forest (RF), XGBoost (XGB), Feed Forward Neural
Network (NN), and the GPT-4 Classifier (GPT-4 (C)). The highest scores are
considered as the benchmark scores in inferring the private and utility
features.

For Task 1, the model predicts private feature with an accuracy of 0.84
without any sanitization mechanism, and utility feature with an accuracy of
0.88. After applying ALFR and UAE-PUPET, the accuracy for private feature
significantly decreases to 0.65 and 0.67 respectively, while the accuracy for
utility feature remains relatively stable at 0.81 and 0.80. Our proposed
method, GPT-4 (P1) - which incorporates the P1 prompt as shown in Fig. 1 \-
yields result similar to both existing sanitization mechanisms ALFR and UAE-
PUPET, reducing the accuracy for inferring private features to 0.65. For
utility feature, it surpasses existing methods with an accuracy of 0.89.
However, our GPT-4 (P2) approach shows a higher privacy leakage, with an
accuracy of 0.75 for private feature, yet it maintains better utility
(accuracy of 0.88) than other adversarial techniques. For Task 2, GPT-4 (P1)
shows better privacy protection and similar utility compared to existing
adversarial techniques. For a comparison of F1 scores and the individual
results of classifier models, kindly refer Table I.

PM | Task 1 | Task 2
---|---|---
$M_{p}$ | $M_{u}$ | $M_{p}$ | $M_{u}$
ALFR | 0.00 | 0.50 | 0.07 | 0.80
UAE-PUPET | 0.00 | 0.42 | 0.00 | 0.87
GPT-4 (P1) | 0.00 | 1.00 | 0.00 | 0.80
GPT-4 (P2) | 0.40 | 1.00 | 0.00 | 0.67
TABLE II: Privacy Leakage and Utility Performance results.

Table II showcases the performance concerning privacy leakage $(M_{P})$ and
utility performance $(M_{U})$. For Task 1, our GPT-4 (P1) approach
demonstrates result with no privacy leakage and no utility drop, surpassing
the performance of existing techniques. Conversely, GPT-4 (P2) exhibits a
higher degree of privacy leakage, although it maintains utility performance
without a drop. For Task 2, our proposed techniques exhibit performance levels
comparable to those of existing methods.

Figure 5: Visualization of distortion (noise) applied to continuous variables
by different sanitization mechanisms for Task 1 and Task 2. Upper row shows
distortion applied for Task 1 and lower row shows distortion applied for Task
2.

(a) Task 1

(b) Task 2

Figure 6: Label flips for categorical variables for both tasks.

Numerous existing optimization strategies focus on sanitizing datasets to
enhance fairness in current models. It’s crucial to evaluate whether our
sanitization method can similarly promote fairness. Fig. 4 depict fairness
metrics for Task 1 and Task 2, respectively. Each line in these plots
represent a different sanitization mechanism, with the exception of the No PM
line, which illustrates the fairness levels of various models before any
sanitization. While the ALFR and UAE-PUPET methods improve fairness over not
using any sanitization across all three fairness metrics, our GPT-4 (P1)
mechanism falls short in providing comparable fairness. A closer examination
of Prompt (P1) reveals it doesn’t explicitly address fairness, which might
explain its relatively lower fairness performance. However, it does show
promise in certain scenarios, achieving the best fairness scores with some
classification models (e.g., GPT-4 (C) for Equalized Odds and Equalized
Opportunity, RF for Demographic Parity).

Since prompt P1 didn’t include any mention of fairness, we introduce prompt
P2, which specifically instructs the GPT-4 model to take fairness into
account. This prompt shows notable improvement in Task 1 for equalized odds
and equalized opportunity, achieving results on par with ALFR. However, for
Task 2 (also for Task 1, demographic parity metric), the introduction of the
prompt P2 does not enhance fairness, and in many cases, the scores worsen,
except for the GPT-4 (C) classifier model, where fairness metrics are
comparable to existing techniques.

This observation suggests that while our models don’t uniformly perform
similar to existing techniques in fairness metrics, there is potential in
leveraging LLMs with new improved LLMs in the future.

In our study, Prompt (P1) is tailored to sanitize the dataset by reducing
accuracy, while Prompt P2 is designed to improve the fairness of the models.
Consistent with their objectives, our results indeed show that P1 offers
enhanced privacy protection, whereas P2 better addresses fairness constraints.
Additionally, we experimented with another prompt that combines elements of P1
and P2, aiming to harness the strengths of both. However, this combined prompt
did not yield superior results. The specific wording of this prompt (for Task
1) was: My Income category can be correctly inferred, but my Gender cannot be
correctly inferred (to fulfill fairness metrics). This phrase was intended to
balance privacy protection and fairness enhancement, but it did not lead to
the anticipated improvements in the models’ performance.

Furthermore, we try to gain further understanding of the distortion (noise)
introduced using our proposed methodology compared to existing techniques.
Therefore, we compare the noise introduced in both existing and proposed
mechanisms, to identify any potential patterns. Fig. 5 illustrates the noise
levels in continuous variables, excluding the variables capital-gain and
capital-loss for the sake of brevity. The upper row of the figure depicts the
noise for Task 1, while the lower row corresponds to Task 2. For continuous
variables, it is observed that the noise added by our proposed mechanisms is
generally less than that by the ALFR and UAE-PUPET methods, with the
distributions retaining similar shapes.

In the context of categorical variables, noise is quantified as label flips –
essentially assessing whether labels change before and after the application
of the sanitization mechanism. Figures 6 and 6 display the counts of label
flips for each categorical variable in Task 1 and Task 2, respectively. We
note that for categorical variables, the ALFR and UAE-PUPET methods result in
fewer label flips compared to our GPT-4 (P1) and GPT-4 (P2) approaches, where
for some variables almost all the categories are flipped.

Our research further seeks to determine whether actual private and utility
labels are necessary for effective dataset sanitization. To explore this, we
experimented with an unsupervised approach, as depicted in Fig. 1, without
supervision block, where the models were not provided with actual labels
(including no range for income), to assess if they could autonomously discern
the true values and sanitize the dataset accordingly. We find that including
true label values, i.e., the supervised approach is important.

| Private feature | Utility feature
---|---|---
| Acc | Precision | F1 | Acc | Precision | F1
Task 1 | 0.68 | 0.73 | 0.69 | 0.78 | 0.78 | 0.78
Task 2 | 0.69 | 0.59 | 0.62 | 0.60 | 0.61 | 0.60
TABLE III: GPT-4 (P1) sanitization without supervision.

Table III displays the highest accuracy, precision, and F1 scores achieved
across the five classification models used in our study. While the outcomes
for Task 1 were somewhat acceptable, the results for Task 2 clearly indicate
the necessity of true labels for effective sanitization. It’s also noteworthy
that the highest accuracy for the utility feature in Task 1, as mentioned in
the table, was achieved by only a single model. For the remaining models,
there was a notable absence of preserved information for the utility feature,
underscoring the challenges faced by unsupervised method in maintaining
utility while protecting privacy.

##  VI Conclusion

In our study, we present an initial study of GPT-4 based sanitization
mechanism that operates without the need for additional training, relying
solely on the implementation of specifically designed prompts. Our findings
reveal that this approach offers a level of privacy protection against various
machine learning models that is comparable to existing techniques, despite its
relative simplicity. While our method demonstrates efficacy in privacy
protection, it does not consistently meet fairness constraints to the same
extent as existing methods. However, certain results within our study do show
promise, exhibiting comparable outcomes to those of established techniques.
Given the rapid advancement in model development, we are optimistic that
future iterations of such models will further enhance the capabilities of our
proposed method, potentially addressing its current limitations in ensuring
fairness.

## References

  * [1] I. H. Sarker, “Deep learning: A comprehensive overview on techniques, taxonomy, applications and research directions,” _Sn Computer Science_ , vol. 2, 2021. [Online]. Available: https://api.semanticscholar.org/CorpusID:237241776
  * [2] A. Vaswani, N. Shazeer, N. Parmar, J. Uszkoreit, L. Jones, A. N. Gomez, L. u. Kaiser, and I. Polosukhin, “Attention is all you need,” in _Advances in Neural Information Processing Systems_ , I. Guyon, U. V. Luxburg, S. Bengio, H. Wallach, R. Fergus, S. Vishwanathan, and R. Garnett, Eds., vol. 30. Curran Associates, Inc., 2017. [Online]. Available: https://proceedings.neurips.cc/paper_files/paper/2017/file/3f5ee243547dee91fbd053c1c4a845aa-Paper.pdf
  * [3] T. B. Brown, B. Mann, N. Ryder, M. Subbiah, J. Kaplan, P. Dhariwal, A. Neelakantan, P. Shyam, G. Sastry, A. Askell, S. Agarwal, A. Herbert-Voss, G. Krueger, T. Henighan, R. Child, A. Ramesh, D. M. Ziegler, J. Wu, C. Winter, C. Hesse, M. Chen, E. Sigler, M. Litwin, S. Gray, B. Chess, J. Clark, C. Berner, S. McCandlish, A. Radford, I. Sutskever, and D. Amodei, “Language models are few-shot learners,” 2020.
  * [4] S. Hegselmann, A. Buendia, H. Lang, M. Agrawal, X. Jiang, and D. Sontag, “Tabllm: Few-shot classification of tabular data with large language models,” 2023.
  * [5] N. Carlini, F. Tramer, E. Wallace, M. Jagielski, A. Herbert-Voss, K. Lee, A. Roberts, T. Brown, D. Song, U. Erlingsson, A. Oprea, and C. Raffel, “Extracting training data from large language models,” 2021.
  * [6] R. Staab, M. Vero, M. Balunović, and M. Vechev, “Beyond memorization: Violating privacy via inference with large language models,” 2023.
  * [7] R. Behnia, M. R. Ebrahimi, J. Pacheco, and B. Padmanabhan, “Ew-tune: A framework for privately fine-tuning large language models with differential privacy,” in _2022 IEEE International Conference on Data Mining Workshops (ICDMW)_ , 2022, pp. 560–566.
  * [8] C. Peris, C. Dupuy, J. Majmudar, R. Parikh, S. Smaili, R. Zemel, and R. Gupta, “Privacy in the time of language models,” in _Proceedings of the Sixteenth ACM International Conference on Web Search and Data Mining_ , ser. WSDM ’23. New York, NY, USA: Association for Computing Machinery, 2023, p. 1291–1292. [Online]. Available: https://doi.org/10.1145/3539597.3575792
  * [9] M. Sun, W. P. Tay, and X. He, “Toward information privacy for the internet of things: A nonparametric learning approach,” _IEEE Transactions on Signal Processing_ , vol. 66, no. 7, pp. 1734–1747, 2018.
  * [10] M. Sun and W. P. Tay, “On the relationship between inference and data privacy in decentralized iot networks,” _IEEE Transactions on Information Forensics and Security_ , vol. 15, pp. 852–866, 2020.
  * [11] M. Nasr, N. Carlini, J. Hayase, M. Jagielski, A. F. Cooper, D. Ippolito, C. A. Choquette-Choo, E. Wallace, F. Tramèr, and K. Lee, “Scalable extraction of training data from (production) language models,” 2023.
  * [12] M. Li, J. Wang, J. Wang, and S. Neel, “Mope: Model perturbation-based privacy attacks on language models,” 2023.
  * [13] X. Wu, J. Li, M. Xu, W. Dong, S. Wu, C. Bian, and D. Xiong, “Depn: Detecting and editing privacy neurons in pretrained language models,” 2023.
  * [14] A. Kassem, O. Mahmoud, and S. Saad, “Preserving privacy through dememorization: An unlearning technique for mitigating memorization risks in language models,” in _Proceedings of the 2023 Conference on Empirical Methods in Natural Language Processing_ , H. Bouamor, J. Pino, and K. Bali, Eds. Singapore: Association for Computational Linguistics, Dec. 2023, pp. 4360–4379. [Online]. Available: https://aclanthology.org/2023.emnlp-main.265
  * [15] Y. Huang, S. Gupta, Z. Zhong, K. Li, and D. Chen, “Privacy implications of retrieval-based language models,” 2023.
  * [16] F. Ding, M. Hardt, J. Miller, and L. Schmidt, “Retiring adult: New datasets for fair machine learning,” 2022.
  * [17] H. Edwards and A. Storkey, “Censoring representations with an adversary,” 2016.
  * [18] C. Huang, P. Kairouz, X. Chen, L. Sankar, and R. Rajagopal, “Context-aware generative adversarial privacy,” _Entropy_ , vol. 19, no. 12, 2017. [Online]. Available: https://www.mdpi.com/1099-4300/19/12/656
  * [19] D. Madras, E. Creager, T. Pitassi, and R. Zemel, “Learning adversarially fair and transferable representations,” 2018.
  * [20] F. Pittaluga, S. J. Koppal, and A. Chakrabarti, “Learning privacy preserving encodings through adversarial training,” 2018.
  * [21] S. A. Osia, A. Taheri, A. S. Shamsabadi, K. Katevas, H. Haddadi, and H. R. Rabiee, “Deep private-feature extraction,” _IEEE Transactions on Knowledge and Data Engineering_ , vol. 32, no. 1, pp. 54–66, 2020.
  * [22] C. Huang, P. Kairouz, X. Chen, L. Sankar, and R. Rajagopal, “Generative adversarial privacy,” 2019.
  * [23] X. Chen, T. Navidi, S. Ermon, and R. Rajagopal, “Distributed generation of privacy preserving data with user customization,” 2019.
  * [24] Z. Wu, Z. Wang, Z. Wang, and H. Jin, “Towards privacy-preserving visual recognition via adversarial training: A pilot study,” 2020.
  * [25] A. Morales, J. Fierrez, R. Vera-Rodriguez, and R. Tolosana, “Sensitivenets: Learning agnostic representations with application to face images,” 2020.
  * [26] T. Xiao, Y.-H. Tsai, K. Sohn, M. Chandraker, and M.-H. Yang, “Adversarial learning of privacy-preserving and task-oriented representations,” _Proceedings of the AAAI Conference on Artificial Intelligence_ , vol. 34, no. 07, pp. 12 434–12 441, Apr. 2020. [Online]. Available: https://ojs.aaai.org/index.php/AAAI/article/view/6930
  * [27] E. Erdemir, P. L. Dragotti, and D. Gunduz, “Active privacy-utility trade-off against a hypothesis testing adversary,” 2021.
  * [28] Z. Wu, H. Wang, Z. Wang, H. Jin, and Z. Wang, “Privacy-preserving deep action recognition: An adversarial learning framework and a new dataset,” 2021.
  * [29] B. Mandal, G. Amariucai, and S. Wei, “Uncertainty-autoencoder-based privacy and utility preserving data type conscious transformation,” in _2022 International Joint Conference on Neural Networks (IJCNN)_ , 2022, pp. 1–8.
  * [30] C. Sharma, B. Mandal, and G. Amariucai, “A practical approach to navigating the tradeoff between privacy and precise utility,” in _ICC 2021 - IEEE International Conference on Communications_ , 2021, pp. 1–6.
  * [31] “GitHub - bishwasmandal246/uae-pupet: Uncertainty-Autoencoder-Based Privacy and Utility Preserving Data Type Conscious Transformation — github.com,” https://github.com/bishwasmandal246/uae-pupet, [Accessed 27-01-2024].
  * [32] B. Becker and R. Kohavi, “Adult,” UCI Machine Learning Repository, 1996, DOI: https://doi.org/10.24432/C5XW20.
  * [33] L. Zhang, Y. Chen, A. Li, B. Wang, Y. Chen, F. Li, J. Cao, and B. Niu, “Interpreting disparate privacy-utility tradeoff in adversarial learning via attribute correlation,” in _2023 IEEE/CVF Winter Conference on Applications of Computer Vision (WACV)_ , 2023, pp. 4690–4698.
  * [34] P. Garg, J. Villasenor, and V. Foggo, “Fairness metrics: A comparative analysis,” 2020.