# Potentials, exchange, and correlation in attosecond photoemission delays

Hakon Volkmann(a)111E-mail<EMAIL_ADDRESS>Vinay P.
Majety(b)222E-mail<EMAIL_ADDRESS>and Armin Scrinzi(c)333E-mail:
<EMAIL_ADDRESS>
(a)Institut für Physik, Humboldt-Universität, Berlin, Germany
(b)Indian Institute of Technology Tirupati, Tirupati, India
(c)Ludwig Maximilian Universität, Munich, Germany

###### Abstract

We investigate attosecond time delays in the emission of photoelectrons using
a hierarchy of models of the $CO_{2}$ molecule including the strong field
approximation, Coulomb-scattering, short-range parts of the molecular
potential, Hartree and Hartree-Fock descriptions. In addition, we present an
ab initio calculation based on quantum-chemical structure in combination with
strong-field techniques, which fully includes multi-electron exchange and
correlation. Every single of these model constituents is found to modify
delays on the scale of 10 as or more, with exchange and correlation having the
most pronounced effect.

##  1 Introduction

Time delays in photoelectron emission are a highly sensitive probe of the
dynamics of electron ejection and subsequent scattering. Streaking [1] and
RABITT [2, 3] are two experimental techniques that can measure time delays
with resolutions on the scale of 10 as (1 as=$10^{-18}s$). The high precision
of these measurements may uncover a variety of mechanisms that contribute to
photoemission. An interpretation in terms of “tunneling times”[4] has created
a long debate, with recent measurements suggesting the absence of any such
time delay in the tunnel process [5]. The whole debate has also been hampered
by serious conceptual questions related to the definition and interpretation
of any such delay time (see, for example, [6, 7, 8, 9]). In course of that
debate, awareness has been heightened that irrespective of delays that might
be associated with the photo absorption process, scattering delays dominate
the measurements.

Independent of any interpretation, the measured delays may provide a sensitive
probe into bound-continuum matrix elements of complex correlated systems. That
becomes transparent for the RABITT technique, where a near infrared (IR) pulse
is superposed with a high harmonic pulse in the extreme ultraviolet (XUV) to
produce side bands whose amplitudes are sensitive to the relative phase
between the pulses. Already in the original proposal of the technique [2] a
system specific “atomic phase” of the bound-continuum transition was
introduced. With progress of attosecond techniques, that phase has been moved
into the focus of interest and it is re-interpreted as a time delay within the
optical cycle of the IR pulse. At sufficiently low intensities the process is
a perturbative two-photon transition that solely depends on the initial bound
and final scattering states of the field free system. This inherently
stationary nature of the process precludes the interpretation as a time-
ordered process of photo absorption followed by scattering, but it provides a
highly sensitive probe into bound and scattering solutions.

It was noted that the picture may become more complex in presence of
resonances [8] as the process needs to be studied on the level of quasi-
degenerate perturbation theory. Even in absence of such complications,
scattering solutions need to be obtained taking into account correlation. In a
recent publication [10] it was noted that for the description of delays in
photoemission from $CO_{2}$ a single scattering channel is insufficient and
channel coupling must be included.

In view of this complexity, fully correlated ab initio scattering solutions
may be used in the computation of attosecond delays. This should ultimately
lead to convergence of theoretical with experimental results. Such a
verification of ab initio calculations or complex models against precision
measurements may be gratifying, but a more simplified description would be
desirable for interpreting observations in terms of physical mechanisms such
as the role of the initial state, system specific single-particle scattering,
the more universal Coulomb scattering, and exchange and correlation.

In the present paper we explore which level of approximation may be admissible
when the goal is to compute delays with accuracies of $\sim 10\,as$. We
quantify the effect of individual approximations using a hierarchy of closely
related models for the $CO_{2}$ molecule. Anticipating the conclusions, we
find that attosecond delays on that small scale crucially depend on every
single of the effects listed above and any modeling short of the ab initio
level is unlikely to produce conclusive results with accuracies of $\sim
10\,as$.

##  2 RABITT for the time-dependent Schrödinger equation

We discuss delays that are measured by the RABITT technique, for which we
summarize the procedures here and refer to literature for a discussion, e.g.
Ref. [11]. In that technique one measures photo emission spectra, where
single-photon ionization by a pulse of odd high harmonics ejects an electron
in presence of a weak field at the fundamental frequency. This generates side
bands in between the odd harmonic peaks by adding a single photon of the
fundamental to one peak or, alternatively, subtracting a photon from the next
higher peak. The interference of the two pathways causes a variation of the
side-band intensity with the relative phase between harmonics and fundamental,
which in the time-domain can be interpreted as a delay in the harmonic photo-
emission.

Specifically, we compute the photo-emission spectra in the combined
fundamental near-IR and high harmonic fields with vector potential

$\displaystyle\int_{-\infty}^{t}d\tau\vec{\mathcal{E}}(t)=:{\vec{A}}(t)=$ (1)
$\displaystyle{\vec{A}}_{f}(t)\sin(\omega
t+\phi)+{\vec{A}}_{h}(t)\left[\sin((2n-1)\omega t)+\sin((2n+1)\omega
t)\right]$

at a fundamental wave length of $\lambda=800\,nm$ and its 13th and 15th
harmonic ($n=7$). Peak amplitudes $\max_{t}|{\vec{A}}_{f}(t)|$ and
$\max_{t}|{\vec{A}}_{h}(t)|$ are chosen such that 2-photon perturbation theory
is applicable. In that limit, the exact amplitudes become unimportant and can
be factored out. Similarly, neither the exact pulse shapes nor durations
matter, if only the pulses are long enough to have a narrow band-width that
allows a clear separation of the harmonic peaks from the side band peak
created by the fundamental. In practice, we use pulses with FWHM duration of
3-4 $T$, where $T=\lambda/c\approx 2.6fs$ is the optical period of the
fundamental. The photon energy of the fundamental is
$\omega=2\pi\hbar/T\approx 1.55\,eV\approx 0.057\,au$, where atomic units (au)
are defined through $m_{e}=\hbar=e^{2}=1$.

For initial state energy $E_{c}$ w.r.t. a given ionization channel, the side
band appears at electron energy $E_{c}+2n\omega$. In a time-dependent
description, the relevant part of the two-photon perturbative wave function at
time $t$ depends on the phase shift $\phi$ between the fundamental and the
harmonics as follows:

$\displaystyle\Psi(t)$ $\displaystyle=\sin{\phi}\int_{-\infty}^{t}d\tau
U_{0}(t-\tau)\cos{\omega\tau}(i{\vec{A}}_{f}(\tau)\cdot\vec{\nabla})U_{0}(\tau)\Phi_{h}(\tau)$
$\displaystyle+\cos{\phi}\int_{-\infty}^{t}d\tau
U_{0}(t-\tau)\sin{\omega\tau}(i{\vec{A}}_{f}(\tau)\cdot\vec{\nabla})U_{0}(\tau)\Phi_{h}(\tau).$
(2)

Here we denote by $U_{0}(t)$ the time-evolution without the field and by
$\Phi_{h}$ the wave packet generated by the harmonic vector potential
${\vec{A}}_{h}(t)$ in first order of perturbation theory. Two time-dependent
calculations suffice for determining the modulation of the side band
amplitude. A maximal side band peak at $\phi=0$ may be interpreted as
undelayed emission of the photo-electron, in which case a maximum at non-zero
$\phi$ corresponds to a delay of $\Delta T=T\times\phi/(2\pi)$.

As the physical interpretation of RABITT fundamentally relies on perturbation
theory, it can be equivalently formulated using stationary scattering theory,
where instead of emission delays one deals with phases in the bound-continuum
dipole transition matrix elements. Such a formulation does not reduce the
computational challenge, as for obtaining correct phases one needs scattering
solutions that are very accurate in the vicinity of the nuclei. We have chosen
the time-dependent approach for the simple reason that we have a powerful
solver for the time dependent Schrödinger equation (TDSE) at our disposal
[12]. The time-dependent approach has the added benefit that one can explore
the intensity limits for perturbation theory and estimate the intensities that
are admissible in a RABITT experiment.

Perturbation theory also simplifies calculations of the polarization
dependence of the side-band: yields for all polarization directions of
fundamental and harmonic at given phase shift $\phi$ can be reconstructed from
a maximum of 4 calculations at different polarizations by the formula

$\displaystyle\Psi_{\alpha\beta}(t)=$
$\displaystyle\cos\alpha\cos\beta\Psi^{(zz)}(t)+\sin\alpha\cos\beta\Psi^{(xz)}(t)$
$\displaystyle+\cos\alpha\sin\beta\Psi^{(zx)}(t)+\sin\alpha\sin\beta\Psi^{(xx)}(t).$
(3)

where $\alpha$ and $\beta$ are the polarization angles of fundamental and
harmonic, respectively and polarization is assumed in the $xz$-plane. The
$\Psi^{(ab)}$ are

$\Psi^{(uv)}(t)=\int_{-\infty}^{t}d\tau
U_{0}(t-\tau)\sin{\omega\tau}A^{(u)}_{f}(t)U_{0}(\tau)\Phi^{(v)}_{h}(\tau),\qquad
u,v\in\\{x,z\\}.$ (4)

The subscripts refer to the respective polarization directions of fundamental
and harmonic fields. In the present paper we discuss only the case of parallel
linear polarization ${\vec{A}}_{f}\parallel{\vec{A}}_{h}$,
$\alpha=\beta=:\gamma$. In that case only the linear combination
$\Psi^{(xz)}+\Psi^{(zx)}$ enters in Eq. (3), such that wave packets
$\Psi_{\gamma\gamma}$ computed at three different $\gamma$ suffice for
determining the result at all parallel polarization directions.

###  2.1 The haCC model of the multi-electron system

For the purpose of this study we consider the positions of nuclei as fixed and
admit only single electronic excitation and single-electron emission. With
these constraints one can make an ansatz for the $N$-electron wave function
$\Psi$ of the time-dependent Schrödinger equation at time $t$ as

$\Psi({\vec{r}}_{1},\ldots,{\vec{r}}_{N};t)=\Psi_{b}({\vec{r}}_{1},\ldots,{\vec{r}}_{N};t)+\sum_{c}\mathcal{A}[\chi_{c}({\vec{r}}_{1};t)\Phi_{c}({\vec{r}}_{2},\ldots,{\vec{r}}_{N})],$
(5)

where $\Psi_{b}$ is an in general time-dependent bound neutral wave function,
$\Phi_{c}$ are stationary bound states of the ion, which define the ionization
channels, and $\chi_{c}$ are single-electron wave functions with bound as well
as unbound content. For notational brevity, spin is subsumed into the
${\vec{r}}_{i}$ coordinate. The notation $\mathcal{A}[\ldots]$ indicates anti-
symmetrization w.r.t. exchange of the electrons. Photo-electron spectra are
obtained by asymptotically analyzing $\chi_{c}$ at large times in terms of
scattering states. In practice one may freeze $\Psi_{b}$ at the neutral ground
state, admitting only the single-electron excitations that are contained in
$\mathcal{A}[\chi_{c}\Phi_{c}]$ and limit the emission channels to a small
number. This circumscribes the “hybrid anti-symmetrized Coupled Channels”
(haCC) approach of Ref. [13].

The haCC ansatz was used previously for computing ionization in a variety of
non-perturbative strong field problems, where important effects of exchange
and correlation were observed [14, 15]. RABITT in contrast is inherently
perturbative, but in turn the XUV emission is more sensitive to the short
range behavior of the solutions, such that conclusions from the strong-field
regime cannot be carried over to the present problem.

In the present paper we discuss ionization into the HOMO, HOMO-1, and HOMO-2
channels. These are for ionic states where the $X^{2}\Pi_{g}$, $A^{2}\Pi_{u}$
and $B^{2}\Sigma_{u}$ orbitals, respectively, are removed, further abbreviated
as $X,A$ and $B$. We use real $\Pi$ orbitals, where orbitals with the node in
the polarization plane are labeled as $X_{n},A_{n}$ and the ones with an anti-
node as $X_{a},A_{a}$. The polarization plane is $xz$, i.e. at azimuthal angle
$\phi=0$. In experiments, a statistical mixture of the two states would be
observed, but we keep the channels apart here for additional illustration of
channel sensitivity.

###  2.2 Simplified models

While the ab initio approach above can provide the correct numbers, it does
little to clarify the mechanisms at work. Therefore we explore to which extent
the approach may be simplified using more intuitive models. We use a hierarchy
of models with Hartree-Fock as the most complete one, a Hartree model
discarding anti-symmetrization, and a set of single-electron models with the
“strong field approximation” (SFA) as its simplest representative, whose only
system-specific input is the electron’s initial orbital.

In all models, the electron is emitted from the same initial orbital. For that
we separate the orbital $\varphi_{c}$ from the rest of the space by writing

$H(t)=|\varphi_{c}\rangle
E_{c}\langle\varphi_{c}|+QH_{c}Q+i{\vec{A}}(t)\cdot\vec{\nabla},\quad
Q:=1-|\varphi_{c}\rangle\langle\varphi_{c}|$ (6)

The channel-specific Hamiltonian $H_{c}$ and the projector $Q$ are adjusted
for the various models, while the initial energy $E_{c}$ is kept at the known
value for the given channel.

####  2.2.1 A Hartree model for RABITT

The Hartree ansatz is a reduction of the haCC ansatz (5), where anti-
symmetrization $\mathcal{A}[\ldots]$ is omitted and product states are used
for $\Psi_{b}$ and $\Phi_{c}$. Within the perturbative conditions of RABITT,
the harmonic field induces transitions to the continuum, and the fundamental
modulates the continuum as to produce side bands. Under these conditions, any
impact of channel coupling on the two-photon side band only appears in
processes that involve more than two photons. Consequently, we will consider
all but a one orbital as frozen.

We use for $\Psi_{b}$ a product of single electron orbitals
$\varphi_{c}({\vec{r}})$ and the ionic hole states $\Phi_{c}=a_{c}\Psi_{b}$,
where $a_{c}$ denotes removal of $\varphi_{c}$. Integrating over the ionic
degrees of freedom one obtains, in atomic units, single-electron Hamiltonians
$H_{c}(t)=-\Delta/2+W_{c}$ for channel $c$, with the nuclear plus Hartree
potentials

$W_{c}({\vec{r}})=V({\vec{r}})+\sum_{c\neq
c^{\prime}}\int_{\mathbb{R}}d^{3}r^{\prime}\,\frac{|\varphi_{c^{\prime}}|^{2}}{|{\vec{r}}-{\vec{r}}^{\prime}|}.$
(7)

A plausible choice for the $\varphi_{c}$ are the energetically lowest Hartree-
Fock molecular orbitals from the CI representations of $\Psi_{b}$ and
$\Phi_{c}$ in (5). This Hartree ansatz is our basis for exploring the
relevance of common approximations or improvements of the model for attosecond
delays. The interaction with the laser field $i{\vec{A}}\cdot\vec{\nabla}$
acts on the complete space and in particular connects the initial orbital
$\varphi_{c}$ to the rest of the single-electron space.

####  2.2.2 A Hartree-Fock model for RABITT

The HF ansatz extends the Hartree ansatz by full anti-symmetrization of the
wave function. It is related to Eq. (5) by using the HF ground state for
$|\Psi\rangle$ and the channel functions
$|\Phi_{c}\rangle=a_{c}|\Psi_{b}\rangle$, where $a_{c}$ is the annihilation
operator for $\varphi_{c}$. By symmetry, none of the occupied orbitals
$\varphi_{c}^{\prime}$ is accessible for the ejected electron, which amounts
to extending the projector $Q$ in Eq. (6) to all orbitals as

$Q_{HF}=1-\sum_{c}|\varphi_{c}^{\prime}\rangle\langle\varphi_{c}^{\prime}|.$
(8)

This reduces the dynamical freedom of the emitted electron comparing to the
Hartree ansatz and creates an additional “nonlocal potential”.

The main technical complication of HF calculations in general is the
appearance of an exchange operator, which couples different ionization
channels. Within our assumption of all electrons in their Hartree-Fock
orbitals except for laser-induced perturbations, this appears only in the next
perturbative order beyond the two-photon limit underlying RABITT. Therefore,
on this lowest level of perturbation theory exchange can be neglected and we
are left with the orthogonalization $Q_{HF}$ to all $\phi_{c}^{\prime}$ as the
only effect of anti-symmetrization.

####  2.2.3 SFA-like models with short and long-range potentials

Models that further simplify the Hamiltonian (6) have been of great use for
understanding the physics of photoionization in general and of strong field
physics in particular. In SFA one assumes that the electron is in some initial
bound state $\varphi_{c}$, but otherwise electronic motion in the laser field
is free. With a few extra approximations, one can solve the system with
analytic methods (or very nearly so). Avoiding these additional
approximations, we solve the TDSE where motion outside $\varphi_{c}$ is free,
i.e. we choose $H_{c}=-\Delta/2$. Analytic solutions usually disregard the
need for orthogonalization by $Q$ when accounting for the dynamics after
ionization, but still produce results very similar to the Hamiltonian (6).

The key approximation of SFA — the absence of any potential — is also its main
source of error. Clearly, the potential introduces time-delays in the
emission. Delays by the potential are commonly separated into delays by the
short range part of the potential, that could be considered more genuine, and
a universal contribution due to the long-range Coulomb potential [16]. One can
analyze the relative importance of the contributions using the the Hamiltonian
(6), where one adds a potentials with different long- and short-range
behavior, while keeping initial state $|\varphi_{c}\rangle$ and binding energy
$E_{c}$ unchanged.

We are interested in effects of short-range interactions, of the Coulomb
potential, and possibly of longer range non-Coulombic parts of the potential.
Somewhat arbitrarily we place the boundary between short- and long-range at a
distance of $\approx 4$ atomic units from the $C$-atom in $CO_{2}$, which is a
bit less than twice the $C\\!-\\!O$ bond length of $2.197\,au$. We model these
three cases by adding different potentials $QVQ$ outside the initial state in
(6) that are purely Coulombic, molecular at short range and Coulombic at
intermediate range, or Coulombic at short range and molecular at intermediate
range:

$\displaystyle V_{q}$ $\displaystyle=$ $\displaystyle-\frac{1}{r}$ (9)
$\displaystyle V_{s}$ $\displaystyle=$ $\displaystyle
m(r)W_{c}({\vec{r}})-\frac{1-m(r)}{r}$ (10) $\displaystyle V_{\ell}$
$\displaystyle=$ $\displaystyle[1-m(r)]W_{c}({\vec{r}})-\frac{m(r)}{r}.$ (11)

Here $m(r)$ is a mask function that is $\equiv 1$ for $r<R_{m}$ and $\equiv 0$
for $r>R_{n}$ with a differentiably smooth transition from 1 to 0 on
$[R_{m},R_{n}]$.

##  3 Numerical results and discussion

All calculations presented below are based on the same set of orbitals
$\varphi_{c}$ expanded in a single-center basis using spherical harmonics

$\varphi_{c}({\vec{r}})=\sum_{l=|m|}^{L}Y_{l}^{m}(\phi,\theta)R_{lm}(r),$ (12)

where for $R_{ml}(r)$ a finite-element discrete variable (FE-DVR)
representation is used with $m=0$ and $m=\pm 1$ for the $\Sigma$ and $\Pi$
orbitals, respectively. The same expansion is used for the $\chi_{c}$ with a
range of $m=-3,\ldots,3$, which accounts for two-photon transitions from the
initial $|m|\leq 1$ states. Photo-electron spectra are computed using tSurff
[17, 18]. Computations were performed with the tRecX code as reported in Ref.
[12], where the reader can find details on computational implementation and an
overview of computational methods.

The single-center spherical expansion is notoriously poorly convergent for a
multi-center molecule, but the discretization errors do not affect conclusions
of the relative importance of short- and long-range effects and of the
orthogonalization by Eq. (8), if the same expansion is used across all models.
We used a comparatively small number of $L=9$ for all model calculations, but
verified that the conclusions do not change with a larger value of $L=29$.

For computing spectra we use tSurff with radius $R_{c}=35$, which implies that
beyond that radius all effects of the molecular potential are neglected [17,
18]. The cutoff is well beyond the $C-O$ bond distance of $2.197\,au$, such
that we are in the Coulombic tail of the molecular potential and safely
outside the reach of any occupied orbital $\varphi_{c}$. Note that absolute
delays between Coulomb and free motion cannot be defined, as these diverge
logarithmically with the propagation distance. Rather, in our comparisons we
assess the effects at short- and medium range interactions, where “medium
range” means distances where non-Coulombic contributions to the potential
would be well-approximated by multipole potentials.

The discretization in the radial coordinate is dense in the vicinity of the
Coulomb singularities with a total of 40 functions on the interval $[0,5]$,
which is followed by elements of size 5 up to $R_{c}$ with FE-DVR order of
$15$ on each. That radial discretization is fully converged, leaving the
limitation in angular momenta as the main discretization error. Outside
$R_{c}$ we use irECS [19] for absorption, which implements exterior complex
scaling using an exponentially damped polynomial basis. This gives perfect
absorption in the present setting.

The computation of photoelectron spectra by tSurff is numerically robust with
rapid convergence and negligible numerical noise such that equations (2) and
(3) are fulfilled faithfully for any choice of phases $\phi$ and polarization
angles $\alpha,\beta$. This agreement also shows that for the given system the
selected intensities of $10^{10}\,W/cm^{2}$ for the fundamental and
$10^{11}\,W/cm^{2}$ for the harmonics are safely in the perturbative range.
The example spectrum in Fig. 1 shows harmonic peaks and side band peaks over a
wide dynamic range.

Figure 1:  Photo-ionization peaks into the HOMO channel of $CO_{2}$ for an 800
nm pulse at $10^{10}\,W/cm^{2}$ combined with the 13th and 15th harmonic at
$10^{11}\,W/cm^{2}$. The side-band at $8\,eV$ between the two dominating
harmonic peaks is used for RABITT. Calculation for the Hartree-Fock model.

###  3.1 Hartree and Hartree-Fock

The Hartree model Eqs. (6) and (7) is arguably the most accessible for an
intuitive interpretation, as system-specific properties appear only in the
initial orbital and as single-electron motion in a potential. We expect it to
represent — at least on a qualitative level — the behavior of emission delays.
Its minimally required extension is to Hartree-Fock on the perturbative level
by replacing $Q$ with $Q_{HF}$, Eq. (8). Fig. 2 shows attosecond delays for
ionization from the HOMO, HOMO-1 ($\Pi_{g}$ and $\Pi_{u}$) and HOMO-2
($\Sigma_{g}$) orbitals. As to be expected, orientation dependence of yields
is opposite for the $\Sigma$ and $\Pi$ symmetries, with the yield in parallel
alignment being maximal for the $\Sigma$-state and minimal for the $\Pi$
states. Delay times of the channels differ significantly, and also alignment
dependence has different characteristics. It is important to observe that in
Hartree approximation absolute delays change by up to 50 as or more relative
to Hartree-Fock, which partly appears as an overall offset, but also as a
modulation of alignment dependence. There appears to be a preponderance for
anti-symmetrization to reduce alignment dependence, which might be interpreted
as an effective screening of the Hartree potential by excluding the electron
from the more attractive regions, but such an observation would need a more
systematic confirmation with a larger set of channels and more molecular
species. The important conclusion of this comparison is that any
interpretation of delays on the level of a few tens of attoseconds must
include anti-symmetrization.

Figure 2:  Alignment-dependence of delays in Hartree-Fock description of
$CO_{2}$ for the HOMO ($X^{2}\Pi_{g}$), HOMO-1 ($A^{2}\Pi_{u}$) and HOMO-2
($C^{2}\Sigma_{u}$) channels. The degenerate $\Pi$ states are chosen with a
node ($X_{n},A_{n}$) and anti-node ($X_{a},A_{a}$) in the polarization plane,
respectively. Size of the plot symbols indicates the relative yields within
each curve. The dashed and dotted lines matching the colors of the respective
channels are obtained in Hartree-approximation, where dotted refers to $X_{a}$
and $A_{a}$, respectively.

The figures shown were obtained using only 10 angular momenta in our single-
center expansion of the wave function. While this is clearly insufficient for
fully converged results, it does not affect our conclusions. Figure 3 shows
the convergence with up to 30 angular momenta without qualitative change of
the results.

Figure 3:  Convergence of delays in the HOMO and HOMO-1 channels for 10, 20
and 30 angular momenta. The solid lines with markers are for maximal angular
momenta, dotted lines for 10. Largest deviations appear at parallel alignment
where yields are low. For the HOMO, results with 20 angular momenta are also
included, but not distinguishable from the converged curve on the resolution
of the graph.

###  3.2 Short- and long-range scattering

One expects that delays originate from scattering in the immediate vicinity of
the nuclei roughly on the scale where the respective orbitals are located. We
identify the relevant range of the potential by computing delays for the
screened Hartree potentials of Eqs. (10) and (11). With the finding above that
anti-symmetrization reduces the effect of the potential and alignment
dependence, we include the results with and without the anti-symmetrization.

Fig. 4 summarizes the results for these models. We see that within accuracy
$\sim 10\,as$, delays are caused by $V_{s}$, if we define the transition form
short to long range in the interval [3,5]. Conversely, delays due to the long
range potential $V_{\ell}$ are barely distinguishable from purely Coulombic
delays. This rapid transition of the potential to Coulombic behavior is to be
expected for a non-polar molecule.

Even in absence of any potential in the continuum (SFA) one observes
alignment-dependent delays. As the Fourier transform of our orbitals is
strictly real, the phase shift is solely due to the projector $Q$ acting as a
non-local potential, which is the minimal tribute one must pay in a consistent
theory to the presence of an spatially extended initial bound state.

Figure 4:  Spatial origin of RABITT delays in emission from the HOMO.
Truncated $V_{s}$, Eq. (10), in HF (green squares) and Hartree (red crosses)
approximations. Lines next to the symbols are with the full Hartree potential.
Cyan diamonds: long range potential $V_{\ell}$, Eq. (11), Coulomb is the line
next to it. Bullets are for zero potential without anti-symmetrization (SFA).

###  3.3 Correlation and exchange

Knowing that attosecond delays are highly sensitive to short-range modulations
of the single electron potentials, an important role of correlation is to be
expected. The simplest manifestation of correlation is the modification of the
initial state on a scale that one finds in Dyson orbitals. As short-range
modulations are most important for XUV ionization and delays, admixtures of
tightly bound orbitals may impact delays.

The most prominent effect of correlation comparing to the Hartree and Hartree-
Fock description discussed so far is that channels become coupled. In Ref.
[10] channel coupling was identified as crucial for the description attosecond
delays. Physically this may be expected, as correlation means that a given
electron is influenced by the specific positions of other electrons, rather
than just by their average field, again implying local modifications of forces
on every electron.

The haCC ansatz (5) admits electron correlation on a CI level and fully
includes exchange. For the present calculations we describe the neutral
initial state $\Psi_{b}$ and the lowest 6 ionic states $\Phi_{c}$ on the CI
singles doubles level with an atom centered minimally augmented cc-pvtz basis
as obtained from the COLUMBUS package [20]. For further details on computation
and method we refer the reader to Refs. [13, 15]. Fig. 5 shows delays in the
two degenerate HOMO channels as obtained with and without exchange and
correlation. The degenerated channels are distinguished by the dominant
channel orbitals with a node ($X_{n}$) or anti-node ($X_{a}$) in the
polarization plane, respectively. We observe some modification of yields and
dramatic changes in the computed delays. In all cases yields are minimal at
parallel alignment and increase monotonically towards perpendicular alignment,
the increase is less pronounced for the haCC calculation. Delays in the
$X_{a}$ channel are minimal around 40 degrees alignment while delays in the
$X_{n}$ channel decrease monotonically with the aliment angle.

On the scale of the observed effects, initial state correlation as represented
by a Dyson orbital is negligible: Fig. 5 includes a HF calculation with
$\varphi_{c}$ replaced with the corresponding Dyson orbital. The small impact
is consistent with the fact, that the most important admixtures in the Dyson
orbital is from more diffuse orbitals, while all contributions from more
tightly bound orbitals remain below the level of $10^{-5}$.

It must be mentioned that due to the full inclusion of exchange, haCC
computations are very resource intensive and systematic convergence studies
have not been possible yet. While pushing convergence of the haCC calculation
may change results on the level of many tens of attoseconds, any convergence
towards closer resemblance with the Hartree-Fock calculation would be purely
accidental: as discussed in Sec. 2.2.2, Hartree-Fock appears as the limiting
case of haCC in the independent particle picture. A systematic convergence
study of the haCC calculation to assess the reliability of haCC delays ab
initio will be presented separately.

Figure 5:  The importance of correlation in photoemission delays for the X
channels. Squares: haCC, Eq. (5), bullets: Hartree-Fock, lines without
symbols: Hartree-Fock using the Dyson orbital for the initial orbital.

##  4 Summary and conclusions

Phase shifts in RABITT setups that correspond to delays of 10 or more
attoseconds were found to depend on every detail of modeling of the physical
process. This includes elementary aspects like the exact shape of the
effective potential seen by the emitted electron, which precludes any
approximation that does not fully take into account electron screening on the
level of the Hartree potential. Similarly, the simple constraint of the
emitted electron’s dynamics by Pauli exclusion must be imposed. The far
dominant effects appear locally in the vicinity of the nuclei, which explains
the pronounced effect of correlation on delays seen in the present simulation.
Correlation effects are also highlighted by the importance of channel coupling
reported earlier in Ref. [10]. In view of this high sensitivity, one must also
examine the validity the common frozen nuclei approximation when computing of
delays with high accuracy.

It appears, that any theory-experiment comparison of delays on the level of a
few tens of attoseconds requires ab initio level theory. This casts doubt on
the usefulness of attosecond delay measurements for independent insight into
electronic or nuclear dynamics in molecules. On the other hand, it establishes
an outstanding scientific challenge and a highly sensitive test for fully
correlated multi-electron calculations and the matching precision experiments.

## References

  * [1] Vladislav Yakovlev, Ferdinand Bammer, and Armin Scrinzi.  Attosecond streaking measurements.  Journal of Modern Optics, 52(2):395–410, January 2005.
  * [2] Valérie Véniard, Richard Taïeb, and Alfred Maquet.  Phase dependence of (n+1)-color (n$>$1) ir-uv photoionization of atoms with higher harmonics.  Phys. Rev. A, 54:721–728, 1996.
  * [3] P. M. Paul, E. S. Toma, P. Breger, G. Mullot, F. Auge, Ph. Balcou, H. G. Muller, and P. Agostini.  Observation of a train of attosecond pulses from high harmonic generation.  Science, 292:1689–1692, 2001.
  * [4] P. Eckle, A N Pfeiffer, C Cirelli, A Staudte, R Doerner, H G Muller, M Büttiker, and U Keller.  Attosecond Ionization and Tunneling Delay Time Measurements in Helium.  Science, 322(December):1525–1529, 2008.
  * [5] U. Satya Sainadh, Han Xu, Xiaoshan Wang, A. Atia-Tul-Noor, William C. Wallace, Nicolas Douguet, Alexander Bray, Igor Ivanov, Klaus Bartschat, Anatoli Kheifets, R. T. Sang, and I. V. Litvinyuk.  Attosecond angular streaking and tunnelling time in atomic hydrogen.  Nature, 568(7750):75+, Apr 4 2019.
  * [6] R. Buettiker, M. and Landauer.  Traversal time for tunneling.  Physical Review Letters, 49(23):1739–1742, 1982.
  * [7] L. Torlina, F. Morales, J. Kaushal, I. Ivanov, A. Kheifets, A. Zielinski, A. Scrinzi, H.G. Muller, S. Sukiasyan, M. Ivanov, and O. Smirnova.  Interpreting attoclock measurements of tunnelling times.  Nature Physics, 11:503–508, 2015.
  * [8] L. Argenti, Á. Jiménez-Galán, J. Caillat, R. Taïeb, A. Maquet, and F. Martín.  Control of photoemission delay in resonant two-photon transitions.  Phys. Rev. A, 95:043426, Apr 2017.
  * [9] Ulf Saalmann and Jan M. Rost.  Proper time delays measured by optical streaking.  Phys. Rev. Lett., 125:113202, Sep 2020.
  * [10] Andrei Kamalov, Anna L. Wang, Philip H. Bucksbaum, Daniel J. Haxton, and James P. Cryan.  Electron correlation effects in attosecond photoionization of $CO_{2}$.  Phys. Rev. A, 102:023118, Aug 2020.
  * [11] Renate Pazourek, Stefan Nagele, and Joachim Burgdoerfer.  Attosecond chronoscopy of photoemission.  Rev. Mod. Phys., 87(3), Aug 12 2015.
  * [12] A. Scrinzi.  tRecX — an environment for solving time-dependent Schrödinger-like problems.  (preprint) https://arxiv.org/abs/2101.08171.
  * [13] Vinay Pramod Majety, Alejandro Zielinski, and Armin Scrinzi.  Photoionization of few electron systems: a hybrid coupled channels approach.  New. J. Phys., 17, Jun 1 2015.
  * [14] Vinay Pramod Majety and Armin Scrinzi.  Dynamic exchange in the strong field ionization of molecules.  Phys. Rev. Lett., 115:103002, Sep 2015.
  * [15] Vinay Pramod Majety and Armin Scrinzi.  Multielectron effects in strong-field ionization of $CO_{2}$: Impact on differential photoelectron spectra.  Phys. Rev. A, 96:053421, Nov 2017.
  * [16] J M Dahlström, A L’Huillier, and A Maquet.  Introduction to attosecond delays in photoionization.  Journal of Physics B: Atomic, Molecular and Optical Physics, 45(18):183001, Aug 2012.
  * [17] Liang Tao and Armin Scrinzi.  Photo-electron momentum spectra from minimal volumes: the time-dependent surface flux method.  New Journal of Physics, 14(1):013021, Jan 2012.
  * [18] Armin Scrinzi.  t-surff: fully differential two-electron photo-emission spectra.  New Journal of Physics, 14(8):085008, 2012.
  * [19] Armin Scrinzi.  Infinite-range exterior complex scaling as a perfect absorber in time-dependent problems.  Phys. Rev. A, 81(5):053845, May 2010.
  * [20] Hans Lischka, Thomas Mueller, Peter G. Szalay, Isaiah Shavitt, Russell M. Pitzer, and Ron Shepard.  COLUMBUS — a program system for advanced multireference theory calculations.  Wiley Interdiscip. Rev.-Comp. Mol. Science, 1(2):191, 2011\.


11institutetext: Los Alamos National Laboratory

# Heavy Flavor and Quarkonia Results
from the PHENIX Experiment

Krista Smith on behalf of the PHENIX Collaboration 11<EMAIL_ADDRESS>
###### Abstract

The PHENIX experiment at RHIC has a unique large rapidity coverage
($1.2<|y|<2.2$) for heavy flavor studies in heavy-ion collisions. This
kinematic region has a smaller particle density and may undergo different
nuclear effects before and after the hard process when compared to mid-
rapidity production. The latest PHENIX runs contain a large data set which
allows, for the first time, the study of heavy flavor and $J/\psi$ flow at the
large rapidity region in Au$+$Au collisions at $\sqrt{s_{{}_{NN}}}=$200 GeV.
This measurement has the potential to reveal a medium evolution distinct from
that known at mid-rapidity.

##  1 Introduction

Three recent heavy flavor and quarkonia analyses by the PHENIX Collaboration
aim to investigate the following questions: Do we see evidence for multi-
parton interactions at RHIC energies? Are there final state effects on
charmonium production in $p$$+$$p$ collisions? Is there evidence of mass
ordering for charged hadron vs. open heavy flavor $v_{2}$ at forward rapidity?
Here we present the results from these ongoing studies and discuss the
conclusions.

##  2 Data Set & Experimental Setup

The $p$$+$$p$ data set used in two of the analyses was recorded in run year
2015 at a center of mass energy $\sqrt{s_{{}_{NN}}}$=200 GeV, while the
Au$+$Au data set was recorded in 2014, also at the same center of mass energy
per nucleon pair. The integrated luminosity is 47 pb-1 for the 2015 $p$$+$$p$
data set PHENIX:2019gix  and 2.3 nb-1 for the 2014 Au$+$Au data set
PHENIX:2022wim .

All analyses have utilized the PHENIX Muon Arms, which includes the Forward
Vertex Silicon Detector (FVTX), the Muon Tracker (MuTr), the Muon Arms magnet,
and the Muon Identifier (MuID). The FVTX consists of four layers of silicon
strip stations for precision measurement of track trajectory and charged
particle multiplicity. The MuTr measures the momentum of charged tracks, and
the MuID contains five layers of steel absorbers for hadron and muon
separation. At backward rapidity, the South Muon Arm covers -2.2 $<$ $\eta$
$<$ -1.2 in pseudorapidity while the forward arm, slightly larger, covers 1.2
$<$ $\eta$ $<$ 2.4 in pseudorapidity.

##  3 Multiplicity Dependent $J/\psi$ Production in $p$$+$$p$ Collisions

The first PHENIX measurement of relative $J/\psi$ yields $R$ versus the
normalized event charged particle multiplicity $N_{ch}$/$\langle
N_{ch}\rangle$ in $p$$+$$p$ collisions at $\sqrt{s}$=200 GeV is shown in
Figure 1. The charged particle tracks have been measured in three different
rapidity regions: forward, mid, and backward rapidity, as shown in the diagram
on the left of Figure 1. At RHIC energies, there are, on average, $\sim$5
charged particle tracks produced per Minimum Bias event; therefore, the
$J/\psi$ lepton pairs contribute to a large percentage of the total charged
particle tracks. To avoid biasing the charged particle multiplicity, the
PHENIX Experiment has found that $J/\psi$ tracks should be subtracted from
(i.e., not included in) the charged particle multiplicity if the $J/\psi$ is
measured at the same rapidity Shi:2023gnw . At LHC energies, however, the
$J/\psi$ lepton pairs do not constitute a large percentage of the overall
charged particle tracks, and any bias from double counting (i.e., including
the $J/\psi$ decay products in the multiplicity) is negligible.

The PHENIX results (blue, green data points) are shown in the right plot after
the $J/\psi$ tracks have been subtracted from the charged particle
multiplicity. The dependence on multiplicity becomes less pronounced compared
to the ALICE ALICE:2020msa  and STAR STAR:2018smh  results after this effect
has been considered. However, the multiplicity dependence does increase with
increasing particle multiplicity, which is consistent with expectations of
multi-parton interactions.

In Figure 2, the $J/\psi$ multiplicity dependence is shown before (magenta
data points) and after (red data points) the $J/\psi$ tracks have been
subtracted. Both measurements are compared to pythia8 Detroit tune PYTHIA:81 ;
Aguilar:2021sfa  calculations both with (right) and without (left) multi-
parton interactions turned on. This study has concluded that the $J/\psi$
multiplicity dependence is well described by the pythia8 Detroit tune and that
multi-parton interactions are required to reproduce PHENIX data.

Figure 1: Left: Diagram showing the rapidity $y$ of the measured $J/\psi$
compared to the measured charged particle tracks $N_{ch}$. Right: PHENIX
$J/\psi$ multiplicity measurements at forward (green data points) and backward
(blue data points) are compared to ALICE ALICE:2020msa  and STAR STAR:2018smh
measurements at mid-rapidity in magenta and gold data points, and to ALICE
measurements (teal data points) at forward rapidity. Note that the ALICE and
STAR charged particle tracks are measured at mid-rapidity. Figure 2: The
$J/\psi$ multiplicity dependence is shown before (magenta data points) and
after (red data points) the $J/\psi$ tracks are subtracted. Both measurements
are compared to pythia8 Detroit tune PYTHIA:81 ; Aguilar:2021sfa  calculations
with (right) and without (left) multi-parton interactions turned on.

##  4 $\psi(2S)$ to $J/\psi$ Ratio in $p$$+$$p$ Collisions

Multiplicity-dependent studies in small system collisions can provide a
testing ground for examining the onset of quark-gluon plasma-like effects.
Here the $\psi(2S)$ to $J/\psi$ ratio is measured in $p$$+$$p$ collisions as a
function of charged particle multiplicity at forward and backward rapidity.
Figure 3 shows the PHENIX measurements (red, black, blue data points) compared
to ALICE measurements (green data points). The left, middle, and right plots
correspond to the PHENIX charged particle tracks measured at forward, mid, and
backward rapidity, respectively. Note that ALICE results have charged particle
multiplicity measured at mid-rapidity only ALICE:2022gpu . We find that the
PHENIX ($\sqrt{s_{{}_{NN}}}$=200 GeV) and ALICE ($\sqrt{s_{{}_{NN}}}$=13 TeV)
results are consistent, showing a weak multiplicity dependence more or less
consistent with unity. No evidence is found for $\psi(2S)$ final state effects
in $p$$+$$p$ collisions at RHIC energies.

Figure 3: The PHENIX $\psi(2S)$ to $J/\psi$ ratio in $p$$+$$p$ collisions as a
function of charged particle multiplicity (red, black, blue data points) is
compared to ALICE measurements (green data points) at forward rapidity
ALICE:2022gpu . The PHENIX charged particle tracks are measured at forward
rapidity (left), mid-rapidity (middle), and backward rapidity (right).

##  5 Open Heavy Flavor $v_{2}$ in Au$+$Au Collisions

The rapidity dependence of elliptic flow measurements is of interest because
particle production at mid versus forward rapidity could undergo different
quark gluon plasma related effects, such as temperature and pressure
gradients. The first-ever RHIC measurement of open heavy flavor elliptic flow
at forward rapidity is shown in Figure 4. The charged hadron elliptic flow
(red data points, left) and open heavy flavor elliptic flow (black data
points, right) in Au$+$Au collisions as a function of $p_{T}$ are compared to
corresponding PHENIX elliptic flow measurements (blue data points) at mid-
rapidity PHENIX:2014yml ; PHENIX:2006iih . We find that the open heavy flavor
$v_{2}$ is consistent with PHENIX mid-rapidity results. The light charged
hadron $v_{2}$ (left) shows larger elliptic flow than the heavy-flavor $v_{2}$
over the entire $p_{T}$ range of the data, and is also consistent with PHENIX
mid-rapidity results. From these measurements of charged hadron and open heavy
flavor $v_{2}$, we conclude that similar quark gluon plasma effects are
present at both mid and forward rapidities.

Figure 4: Charged hadron elliptic flow (red data points, left) and open heavy
flavor elliptic flow (black data points, right) in Au$+$Au collisions at
forward rapidity as a function of $p_{T}$ are compared to corresponding PHENIX
elliptic flow measurements (blue data points) at mid-rapidity PHENIX:2014yml ;
PHENIX:2006iih .

##  6 Conclusion

In conclusion, for $p$$+$$p$ collision system results, the $J/\psi$
multiplicity dependence increases with increasing particle multiplicity.
Additionally, PHENIX data is well described by the PYTHIA Detroit tune with
Multi Parton Interactions turned on. We conclude there is evidence for Multi
Parton Interactions at RHIC energies. The $\psi(2S)$ to $J/\psi$ ratio shows a
weak dependence on multiplicity, and therefore no evidence is found for
$\psi(2S)$ final state effects in $p$$+$$p$ collisions at RHIC. In Au$+$Au
collisions, the first RHIC measurement of open heavy flavor $v_{2}$ at forward
rapidity is presented. The results are consistent with PHENIX mid-rapidity
measurements, suggesting similar quark gluon plasma effects (including
temperature/pressure gradients) are present at both rapidities.

## References

  * (1) C. Aidala et al. (PHENIX), Phys. Rev. C 101, 034910 (2020), 1906.09928
  * (2) U.A. Acharya et al. (PHENIX) (2022), 2203.17058
  * (3) Z. Shi, Universe 9, 322 (2023)
  * (4) S. Acharya et al. (ALICE), Phys. Lett. B 810, 135758 (2020), 2005.11123
  * (5) J. Adam et al. (STAR), Phys. Lett. B 786, 87 (2018), 1805.03745
  * (6) T. Sj$\mathrm{\ddot{o}}$strand, S. Mrenna, P.Z. Skands, Comput. Phys. Commun. 178, 852 (2008)
  * (7) M.R. Aguilar, Z. Chang, R.K. Elayavalli, R. Fatemi, Y. He, Y. Ji, D. Kalinkin, M. Kelsey, I. Mooney, V. Verkest, Phys. Rev. D 105, 016011 (2022), 2110.09447
  * (8) S. Acharya et al. (ALICE), JHEP 06, 147 (2023), 2204.10253
  * (9) A. Adare et al. (PHENIX), Phys. Rev. C 92, 034913 (2015), 1412.1043
  * (10) A. Adare et al. (PHENIX), Phys. Rev. Lett. 98, 172301 (2007), nucl-ex/0611018


# Toward Human Readable Prompt Tuning:
Kubrick’s The Shining is a good movie, and a good prompt too?

Weijia Shi Xiaochuang Han11footnotemark: 1
Hila Gonen Ari Holtzman Yulia Tsvetkov Luke Zettlemoyer
Paul G. Allen School of Computer Science & Engineering,
University of Washington, Seattle, WA
{swj0419, xhan77, hilagnn, ahai, yuliats<EMAIL_ADDRESS> Equal
contribution. Order randomly determined.

###### Abstract

Large language models can perform new tasks in a zero-shot fashion, given
natural language prompts that specify the desired behavior. Such prompts are
typically hand engineered, but can also be learned with gradient-based methods
from labeled data. However, it is underexplored what factors make the prompts
effective, especially when the prompts are natural language. In this paper, we
investigate common attributes shared by effective prompts. We first propose a
human readable prompt tuning method (FluentPrompt) based on Langevin dynamics
that incorporates a fluency constraint to find a diverse distribution of
effective and fluent prompts. Our analysis reveals that effective prompts are
topically related to the task domain and calibrate the prior probability of
label words. Based on these findings, we also propose a method for generating
prompts using only unlabeled data, outperforming strong baselines by an
average of 7.0% accuracy across three tasks.

##  1 Introduction

Large language models can perform new tasks by simply conditioning on a
prompt–a short sequence of text specific to the task. Such natural langauge
prompts are either carefully hand engineered (e.g., manual prompt engineering,
Kojima et al. 2022) or automatically learned from labeled data (e.g.,
gradient-based prompt tuning, Shin et al. 2020). Despite their effectiveness,
it remains unclear what makes these prompts work and what attributes effective
prompts share in common. In this paper, we aim to identify key characteristics
of effective prompting, and use this knowledge to generate effective and human
readable prompts without any labeled data.

There are two main challenges for performing this type of analysis. First,
manual prompt tuning produces a limited number of effective prompts for each
task, making it difficult to infer common features of good prompts where
contrast with less effective prompts is needed. Additionally, the prompts
found by gradient-based tuning methods are often disfluent and unnatural,
making them difficult to interpret (e.g., AutoPrompt in Figure 1).

Figure 1:  Compared with previous discrete prompt tuning method AutoPrompt
Shin et al. (2020) which generates gibberish prompts, FluentPrompt can
identify effective and more readable prompts that are topically relevant to
the task domain and calibrate the prior probability of label words.

To overcome these challenges, we first propose a human readable prompt tuning
method called FluentPrompt based on a constrained decoding method.
FluentPrompt uses Langevin dynamics to generate a set of human readable
prompts for any task. Our method adds a progressive noise to the tuning
procedure to obtain a distribution of effective prompts, while also
maintaining the fluency of the prompts through a perplexity constraint. As
shown in Figure 1, compared to the baseline gibberish prompts produced by
AutoPrompt, FluentPrompt generates prompts that are more fluent (i.e., lower
perplexity) and perform competitively. The resulting fluent prompts not only
facilitate our further analysis, but can also lead to better trust and
engagement from both researchers and end users.

After obtaining a diverse set of effective and human-readable prompts, we
analyze the factors that contribute to the effectiveness of prompts.
Specifically, we show that effective prompts are both (1) topically related to
the task domain and (2) more _calibrated_ to the task verbalizers. Calibration
measures how balanced the label word distribution of the prompted model is
given an example-independent domain string Holtzman et al. (2019).

Based on our findings, we propose a novel method Unsupervised FluentPrompt,
for automatically searching for effective prompts using only unlabeled data.
Unsupervised FluentPrompt optimizes the prompts for both better calibration
and better domain relevance. Our experimental results show that Unsupervised
FluentPrompt outperforms strong zero-shot baseline Holtzman et al. (2021) by
7.0% in accuracy. We summarize our contributions as follows:

  * •

We introduce FluentPrompt, a human-readable prompt tuning method that can
generate a diverse set of effective and fluent prompts (§3). This method not
only serves as the foundation for our analysis, but also helps bridge the gap
between manual prompt engineering and gradient-based prompt tuning.

  * •

We analyze the factors that contribute to the effectiveness of prompts and
show that topic relatedness and calibration of the prompts are key to their
success (§4).

  * •

Inspired by our findings, we introduce a new method for discovering effective
prompts without the need for labeled data (§5).

##  2 Related Work

###  2.1 Prompt Tuning

#### Continuous Prompt

Continuous prompts are continuous vectors inserted to the task input for a
prompted language model Qin and Eisner (2021); Ding et al. (2021); Lester et
al. (2021); Liu et al. (2021). Such continuous prompts are typically tuned by
gradient-based methods, which are guided by the tasks training examples with
labels. While these prompts usually improve the model performance, their
continuous nature makes them difficult for humans to understand or interpret
Khashabi et al. (2021); Hambardzumyan et al. (2021).

#### Discrete Prompt

Discrete prompts are composed of discrete tokens from natural language
vocabulary. Such prompts can be either written by human or searched
automatically. Human-written prompts Kojima et al. (2022); Wang et al. (2022);
Sanh et al. (2021) typically consist of meaningful texts such as task
descriptions Schick and Schütze (2021) or instructions (e.g., “let’s think
step by step”, Kojima et al. 2022), which are not only human readable but also
aligns with their understanding of the task. In-context demonstration examples
can also be considered as human-written prompts Brown et al. (2020); Liu et
al. (2022) but is not a focus of this work.

Prior work has also focused on searching the discrete prompts automatically.
One prominent way for this search can be gradient-based similar to the
continuous prompt setup but with projections to a discrete vocabulary Shin et
al. (2020). The drawback of this method is that the resulting prompts are
usually disfluent and difficult to read. Other work searching for discrete
prompts include edit-based enumeration Prasad et al. (2022), reinforcement
learning Deng et al. (2022), and large language model continuation and
filtering Zhou et al. (2022). The goal for these prompt tuning methods is
mainly to achieve competitive task performance without modifying language
model parameters.

The purpose of our work is to analyze what aspects of the tuned natural
language prompts make them effective for zero-shot language models. To
facilitate such analysis, we need prompt readability as in human-written
prompts and also a large search space as in gradient-based discrete prompt
tuning. FluentPrompt bridges the gap and provides a distribution of effective,
diverse, and human-readable prompts.

###  2.2 Analyses of Prompts

A growing body of literature tries to understand the mechanisms behind prompts
via various perspectives. For example, prompts in the form of in-context
examples are analyzed under perturbations w.r.t. order, label, editing, etc.
Lu et al. (2022); Min et al. (2022); Chen et al. (2022). Human-written
instructions Mishra et al. (2021) have also been studied and show weak
sensitivity to semantic-changing perturbations Webson and Pavlick (2021).
Gonen et al. (2022) use paraphrasing and back-translation on a set of human-
written prompts and analyze the correlation between their perplexity and
performance.

Our work focuses on natural language prompts derived from gradient-based
prompt tuning. Khashabi et al. (2021) tune continuous prompts and shows
effective continuous prompts may transfer poorly to their nearest discrete
prompts. In contrast, we perform prompt tuning in the discrete space directly
with FluentPrompt, demonstrating the feasibility of searching for readable
prompts using gradient-based method. This approach gives us a more faithful
understanding of the factors that contribute to the effectiveness of natural
language prompts.

##  3 FluentPrompt

FluentPrompt generates a diverse set of human-readable prompts. Our goal is
not only to identify a single best-performing prompt, but also to explore the
relationship between the features of the prompts and their performance.

###  3.1 Background: continuous prompt tuning

Given an input example $\boldsymbol{x}$ with an output label $y\in Y$, we can
prompt an autoregressive language model with parameters $\theta$ as follows.
We reformulate the task as a language modeling problem by inserting a task-
specific template $\boldsymbol{t}$ to $\boldsymbol{x}$ and defining a
verbalizer $v$ mapping from a label $y$ to a label word (i.e, token in the
LM’s vocabulary). The probability of the label is estimated by:

$\displaystyle
p_{\theta}(v(y)\mid\boldsymbol{x},\boldsymbol{t})=\frac{\exp\text{logit}_{\theta}(v(y)\mid\boldsymbol{x},\boldsymbol{t})}{\sum_{y^{\prime}}\exp\text{logit}_{\theta}(v(y^{\prime})\mid\boldsymbol{x},\boldsymbol{t})}$

Lester et al. (2021) add a sequence of $M$ soft embeddings
$\tilde{\boldsymbol{e}}_{0:M}$ (simplified as $\tilde{\boldsymbol{e}}$; 0:$M$
refers to the positional subscript for the sequence from position 0 to $M-1$)
in front of the input. Therefore, the probability of the label is computed by
$p_{\theta}(v(y)\mid\tilde{\boldsymbol{e}},\boldsymbol{x},\boldsymbol{t})$,
where $\tilde{\boldsymbol{e}}$ is embeddings that bypass the word embedding
layer of the LM $\theta$ and is learned based on a set of training data. These
learned embeddings are sometimes referred to as soft prompts, and the learning
of such prompts as soft prompt tuning. For example, if stochastic gradient
descent (SGD) is used as an optimizer, the soft prompt
$\tilde{\boldsymbol{e}}$ is updated as

$\displaystyle\tilde{\boldsymbol{e}}^{i}=\tilde{\boldsymbol{e}}^{i-1}-\eta\nabla_{\tilde{\boldsymbol{e}}}(-\log
p_{\theta}(v(y)\mid\tilde{\boldsymbol{e}}^{i-1},\boldsymbol{x},\boldsymbol{t}))$

where $i$ is the timestep superscript, referring to $i$-th optimization step.

###  3.2 Discrete prompt tuning with Langevin dynamics

There are two challenges for the soft prompt tuning reviewed above. First, the
resulting embeddings cannot be mapped to the natural language vocabulary.
Khashabi et al. (2021) show that naively mapping an effecive soft prompt to
their nearest tokens significantly drops the performance. Second, we only
obtain a single embedding instead of a range of embeddings with varying levels
of performance. This makes it difficult to analyze the characteristics of the
prompts and compare their effectiveness in specific tasks for the language
model.

Following Kumar et al. (2022), we use Langevin dynamics to sample discrete
prompts that lead to a better performing model in the task. Overall, the
method is similar to SGD but adds a progressive Gaussian noise to the
embeddings, with the scale decreasing over time. Additionally, at each
optimization step, the updated embedding is projected to the nearest embedding
in the LM vocabulary.

$\displaystyle\tilde{\boldsymbol{e}}^{i}=\text{Proj}_{\mathbf{E}}[$
$\displaystyle\tilde{\boldsymbol{e}}^{i-1}-\eta\nabla_{\tilde{\boldsymbol{e}}}\mathcal{E}(\tilde{\boldsymbol{e}}^{i-1})+\sqrt{2\eta\beta_{i}}\boldsymbol{z}]$

where:

  * •

$\mathcal{E}$ is an energy function (lower is better),
$\mathcal{E}(\tilde{\boldsymbol{e}}^{i-1})=-\log
p_{\theta}(v(y)\mid\tilde{\boldsymbol{e}}^{i-1},\boldsymbol{x},\boldsymbol{t})$.

  * •

$\boldsymbol{z}$ is a Gaussian noise,
$\boldsymbol{z}\sim\mathcal{N}(0,I_{|\tilde{e}|})$.

  * •

$\beta$ is the variance of the noise following a geometric progression,
$\beta_{\text{start}}>\beta_{i}>\beta_{\text{end}}\xrightarrow{}0$.

  * •

$\mathbf{E}$ is the embedding table (layer) of the LM $\theta$, one embedding
for each token in the vocabulary.

  * •

$\text{Proj}_{\mathbf{E}}$ is a projection operation finding a nearest
neighbor for each soft embedding in the LM’s vocabulary,
$\text{Proj}_{\mathbf{E}}(\tilde{e})=\text{argmin}_{e_{v}\in\mathbf{E}}(\lVert
e_{v}-\tilde{e}\rVert_{2})$.

Without the progressive noise in Langevin dynamics, our prompt search
procedure is gradient-based and shares a similar intuition with AutoPrompt
(Shin et al., 2020). Both methods use the gradient of the loss w.r.t. the
embeddings, though AutoPrompt applies greedy substitution whereas we use
projected gradient descent, aligning with soft prompt tuning and enabling the
subsequent prompt sampling. AutoPrompt also incorporates verbalizer word
selection, which is not a focus of the analysis in this work. We use our
gradient-based, discrete prompt tuning method without Langevin dynamics as a
baseline, referred to as $\text{AutoPrompt}_{\text{SGD}}$.

###  3.3 Fluency constraint

Sampling from projected Langevin dynamics ensures that the tuned prompt
contains natural langauge tokens. However, with no extra constraints, they can
form a disfluent sentence.

We explicitly incorporate a fluency objective to the Langevin energy function.
This objective resembles the regular perplexity loss, but the labels (next
token in the prompt) are not ground-truth. Instead, we measure an embedding-
based sequence probability according to Kumar et al. (2022). For simplicity,
below we drop the timestep superscript on the prompt embeddings and only keep
the positional subscript.

The first step is to obtain the probability of generating the embedding at
position $m$ (i.e., $\tilde{e}_{m}$) based on the previous $m-1$ embeddings
(i.e., $\tilde{\boldsymbol{e}}_{0:m}$). We extract the last hidden state from
the LM (i.e., output embedding) at position $m-1$:
$h_{\theta,m-1}=h_{\theta}(\tilde{\boldsymbol{e}}_{0:m})$. Then the
probability is:

$\displaystyle
p_{\theta}(\tilde{e}_{m}\mid\tilde{\boldsymbol{e}}_{0:m})=\frac{\exp(h_{\theta,m-1}\cdot\tilde{e}_{m})}{\sum_{e_{v}\in\mathbf{E}}\exp(h_{\theta,m-1}\cdot
e_{v})}$

where we equivalently compute the logits for each embedding’s corresponded
vocabulary and take the softmax.111This is equivalently computing the logits
since $e_{v}$ and the projected $\tilde{e}_{m}$ from the last optimization
step are both in the embedding table. Subsequently, the sequence probability
is
$p_{\theta}(\tilde{\boldsymbol{e}}_{0:M})=\prod_{m=1}^{M-1}p_{\theta}(\tilde{e}_{m}\mid\tilde{\boldsymbol{e}}_{0:m})$.

We define a prompt fluency loss as the negative log-likelihood of the prompt
embeddings, $-\log p_{\theta}(\tilde{\boldsymbol{e}}_{0:M})$. Along with the
task labeling loss (§3.2), we modify our energy function as:

$\displaystyle\mathcal{E}(\tilde{\boldsymbol{e}}_{0:M})=$
$\displaystyle-\lambda_{\text{task}}\log
p_{\theta}(v(y)\mid\tilde{\boldsymbol{e}}_{0:M},\boldsymbol{x},\boldsymbol{t})$
$\displaystyle-\lambda_{\text{fluency}}\log
p_{\theta}(\tilde{\boldsymbol{e}}_{0:M})$

where $\lambda_{\text{task}}+\lambda_{\text{fluency}}=1$. Through the whole
FluentPrompt tuning procedure, the language model parameters $\theta$ is fixed
while the embeddings $\tilde{\boldsymbol{e}}_{0:M}$ are tuned.

###  3.4 Experimental Setup

#### Target tasks

We evaluate performance on two sentiment analysis tasks: Amazon Polarity
McAuley and Leskovec (2013) and SST-2 Socher et al. (2013), and one topic
classification task: AGNEWS Zhang et al. (2015). These tasks were selected
since vanilla soft prompt tuning Lester et al. (2021) substantially improves
model performance. In contrast, tasks like RTE Dagan et al. (2005) are more
difficult; soft prompt tuning did not yield a significant improvement (57.4%
accuracy from prompt tuning compared with 52.1% from random guess) in our
pilot study, and we therefore did not pursue further analysis using
FluentPrompt. The verbalizer words and templates used for each task are listed
in Table 8.

Prompt | Acc. | PPL
---|---|---
SST-2
Empty Prompt | 66.5 | -
$\text{AutoPrompt}_{\text{SGD}}$ |  |
Compl disgustingÃÂÃÂ Rated jer | 87.6 | $>10^{6}$
FluentPrompt |  |
Kubrick, "The Shining | 87.5 | 13.1
Paramount, "The Shining | 86.8 | 12.2
Kubrick\’s "The Man | 86.3 | 9.3
disappointing.\n\n" | 84.4 | 4.1
AMAZON
Empty Prompt | 75.8 | -
$\text{AutoPrompt}_{\text{SGD}}$ |  |
Reviewed experien audition lashesrible | 82.2 | $>10^{6}$
FluentPrompt |  |
scathing.\n\n" | 83.1 | 5.1
upset.\n\n" | 82.6 | 3.67
cigars: \n\n | 82.4 | 20.9
mascara\n\n\n | 82.2 | 47.1
AGNEWS
Empty Prompt | 49.7 | -
$\text{AutoPrompt}_{\text{SGD}}$ |  |
EStreamFramenetflixnetflixobookgenre | 69.3 | $>10^{5}$
FluentPrompt |  |
netflix/genre/netflix | 71.1 | 281.0
netflix AnimeMoviegenre\n | 70.1 | 1925.0
Synopsis\n\nThe story is | 69.2 | 9.6
pmwiki.php/main/Superhero | 65.0 | 2.4

Table 1: Accuracy (Acc.) and Perplexity (PPL) of prompts. Both FluentPrompt
and $\text{AutoPrompt}_{\text{SGD}}$ use $M$=5 tunable tokens. FluentPrompt
shows comparable performance to the $\text{AutoPrompt}_{\text{SGD}}$ but with
significantly lower perplexity. Prompts discovered by FluentPrompt show domain
relevance and potential caliberation for model outputs.

| SST-2 | Amazon | AGNews
---|---|---|---
|  | $\log(\text{perplexity})$
---
| Accuracy
---
$\log(\text{perplexity})$ |  | Accuracy
---
$\log(\text{perplexity})$ | Accuracy
$\lambda_{\text{fluency}}=0$ | 13.75 $\pm$ 1.81 | 87.55 $\pm$ 0.95 | 14.32 $\pm$ 1.31 | 75.31 $\pm$ 1.76 | 15.04 $\pm$ 3.30 | 74.56 $\pm$ 1.65
$\lambda_{\text{fluency}}=0.003$ | 9.86 $\pm$ 2.41 | 88.86 $\pm$ 0.67 | 10.44 $\pm$ 2.09 | 86.37 $\pm$ 0.68 | 10.13 $\pm$ 1.13 | 76.43 $\pm$ 1.05
Table 2: Accuracy and perplexity of the prompts tuned with and without the
readability constraint $\lambda_{\text{fluency}}$. For
$\lambda_{\text{fluency}}>0$, we report the best value
($\lambda_{\text{fluency}}=$0.003) across 4 learning rates and 5 random seeds
with $M=$10\. All $t$-tests of perplexity and accuracy show $p\leq$ 0.0001.

#### Model

We optimize prompts for GPT-2 large (774M parameters, Radford et al. 2019)
using FluentPrompt. We use a batch size of 16 and train for 5,000 steps with
an AdamW optimizer (Loshchilov and Hutter, 2018). We select the best prompt
based on the validation performance. For our method FluentPrompt, we use a
learning rate $\eta\in$ {0.3, 1.0, 3.0, 10.0}, $\beta_{\text{start}}=$ 1.0,
$\beta_{\text{end}}=$ 0.0001, $\lambda_{\text{fluency}}\in$ {0.003, 0.01,
0.03, 0.1, 0.3}. We search for both 5-token prompts ($M=5$) and 10 token
($M=10$) prompts and use five random seeds for each hyperparameter setup.
Additionally, we perform experiments with
$\beta_{\text{start}}=\beta_{\text{end}}=0$ (i.e, no progressive noise) and
$\lambda_{\text{fluency}}=0$ (i.e, no fluency constraint) as ablations to
FluentPrompt purposed for analysis.

###  3.5 Results

Table 3.4 shows the accuracy and perplexity of empty prompt (i.e., no
$\tilde{\boldsymbol{e}}$), $\text{AutoPrompt}_{\text{SGD}}$ and FluentPrompt,
along with example prompts for each method. We see that FluentPrompt performs
comparably to $\text{AutoPrompt}_{\text{SGD}}$ and significantly better than
the empty prompt. In terms of readability, FluentPrompt generates more fluent
prompts than $\text{AutoPrompt}_{\text{SGD}}$.

To further understand the contributions of FluentPrompt, we now investigate
the effects of its two key modifications on top of
$\text{AutoPrompt}_{\text{SGD}}$: the noise $\boldsymbol{z}$ in Langevin
dynamics and the weight $\lambda_{\text{fluency}}$ for prompt fluency.

#### Effect of $\lambda_{\text{fluency}}$

In Table 2 we show the performance with and without the fluency constraint
($\lambda_{\text{fluency}}=0.003$ and $\lambda_{\text{fluency}}=0$) and the
log-perplexity of the discovered prompts. The fluency constraint effectively
leads to significantly lower perplexity and also better accuracy ($p\leq$
0.0001 in all $t$-tests).222On human-written prompts, Gonen et al. (2022)
report a similar finding. Prompts with lower perplexity are desired for their
potentially better readability for downstream analyses.

#### Effect of $\boldsymbol{z}$

The progressive noise $\boldsymbol{z}$ helps find a diverse set of prompts
while not compromising overall performance. In Table 3 we show the best and
average prompt performance with and without the noise $\boldsymbol{z}$ (i.e.,
$\beta>0$ and $\beta=0$). We measure the diversity of prompts by Dist-1, a
unigram distinctiveness metric (Li et al., 2016). We find that the prompts
obtained with $\boldsymbol{z}$ ($\beta>0$) are more diverse and overall have
an on-par performance with the setup without $\boldsymbol{z}$ ($\beta=0$).

| SST-2 | Amazon | AGNews
---|---|---|---
|  | Max
---
| Mean
---
Dist-1 | Max |  | Mean
---
Dist-1 | Max | Mean | Dist-1
$\beta=0$ | 90.2 | 86.5 | 72.6 | 87.7 | 85.1 | 57.9 | 82.6 | 71.6 | 81.7
$\beta>0$ | 89.6 | 85.5 | 77.6 | 88.7 | 85.4 | 61.2 | 80.7 | 74.1 | 82.2
Table 3: Prompt performance and diversity with and without the progressive
noise $\boldsymbol{z}$ ($\beta>0$ and $\beta=0$).

##  4 What makes good prompts?

In this section, we analyze common attributes of the effective tuned prompts.
Specifically, we study the 10-token prompts found by FluentPrompt on SST-2,
Amazon and AGNEWS.

Figure 2:  Frequency of prompts (y-axis) at different entropy level (x-axis).
We compare effective prompts with the empty and human-written prompt.

Figure 3: Correlation between task performance and label word entropy.
Spearman rank correlation coefficients for SST-2, Amazon and AGNEWS are +0.61,
+0.75 and +0.43. All p-values are less than 0.0001.

###  4.1 Effective prompts calibrate the output distribution over label words

Language models are known to be biased towards label words that are common in
its pretraining distribution Holtzman et al. (2021); Zhao et al. (2021). In
this section, we aim to investigate whether effective prompts found by prompt
tuning implicitly adjust for the bias (calibration). To measure this bias, we
follow Holtzman et al. (2021) to use task-specific domain string
$\boldsymbol{d}$ as the test input and compute the entropy of the labels. As
listed in Table 4, the task-specific domain strings do not imply any label
information. Therefore, we expect the output of the language model to be more
uniform over the label words when only conditioned on the domain string. The
entropy of the label words is computed as follows:

Task | Domain String
---|---
SST-2 | This is a movie review
Amazon | This is an Amazon product review
AGNEWS | This is a news
Table 4: Tasks and their task-specific domain strings.

$\displaystyle H(y)=\mathbb{E}_{y\in Y}[-\log p(y)]=$
$\displaystyle-\sum_{y\in
Y}{p_{\theta}(v(y)\mid\tilde{\boldsymbol{e}},\boldsymbol{d},\boldsymbol{t})\log
p_{\theta}(v(y)\mid\tilde{\boldsymbol{e}},\boldsymbol{d},\boldsymbol{t})}$

The higher the entropy is, the more balanced (calibrated) the label words
distribution is.

As listed in Table 3.4, some effective prompts found by FluentPrompt for
sentiment analysis contain negative sentiment words (e.g., “disappointing” and
“complained” in prompts for SST-2 ), which may implicitly reduce the
probabilty of positive label and calibrate the label word distribution. To
validate this hypothesis, we filter a set of effective prompts by FluentPrompt
and compute the entropy of the label predictions conditioned on the
concatenation of the prompt and the task-specific domain string. Figure 2
shows the density plot comparing the label word entropy of effective prompts,
along with empty and human-written prompts taken from Bach et al. (2022). We
observe that the entropy of effective prompts has a higher mode than the
entropy of empty and human-written prompts with lower accuracy.

To further explore the relation between the task performance and calibration,
we compute correlation between the task accuracy and the label word entropy of
all prompts obtained by our algorithm FluentPrompt and report Spearman’s rank
correlation. From Figure 3, we observe that the label word entropy exhibits
significant positive correlations with the task accuracy (all $p<$ 0.0001).
The Spearman’s coefficients are +0.61, +0.75 and +0.43 for SST-2, Amazon and
AGNEWS, respectively.

###  4.2 Effective prompts are topically related to the task domain

#### Qualitative Analysis

As shown in Table 3.4, most of the effective prompts obtained by FluentPrompt
contain domain-related words. For example, the prompt Kubrick, "The Shining in
SST-2 contains movie director names and movie titles, relevant to the domain
of movie reviews. Similarly, the prompt mascara\n\n and cigars\n\n found for
Amazon contain product names relevant to the domain of product reviews.
Additionally, AGNEWS is a news topic classification task. Some of the
effective prompts in AGNEWS contain topic classification-related words such as
“genre”, while others contain URLs that link to websites such as
netflix333www.netflix.com and pmwiki.444www.pmwiki.org The target pages of
these URLs also contain topic classification-related information, such as the
prompt pmwiki/pmwiki.php/Main/Superhero which links to a wiki page containing
the following information: “ _Genre: Action Adventure Comedy Commercials_ ”.

#### Quantitative Analysis

Based on our qualitative analysis, we hypothesize that effective prompts are
topically related to the task domain. To validate this hypothesis, we compare
domain word frequency in effective prompts and random sentences. First, we
select a set of domain words for each task (see Table 5), which consist of the
task label words (e.g., “positive” and “negative” for SST-2) and common words
in the task domain (e.g., “movie” and “film” for the movie domain of SST-2).
Since our prompts are very short (10 tokens), we augment each prompt with its
continuation generated by GPT-3 Brown et al. (2020), based on the assumption
that the continuation by the large LM follows the same domain as the prompt.
For each prompt, we sample 5 distinct continuations from GPT-3 using nucleus
sampling $p=0.95$ at a length of 100 tokens. We compare the top 10 effective
prompts with 10 random sentences from PILE Gao et al. (2020) augmented by the
same continuations. We then count the domain words in the concatenation of the
prompt and its continuation.

Table 6 lists the average accuracy of and number of domain words in the
effective and random prompts with the continuations. The accuracy of effective
prompts is higher than that of random sentences on all three datasets.
Moreover, the domain words frequency of effective prompts is significantly
higher than that of random sentences with p-values of 0.004, 0.003, and 0.0002
for SST-2, Amazon, and AGNEWS, respectively. Both our qualitative and
quantitative analysis provide strong evidence that effective prompts obtained
by our prompt tuning are topically related to the task’s domain.

Task |  Domain Words
---|---
SST-2 |  movie, film, cinima, director, positive, negative
Amazon |  book, amazon, product, furniture, positive, negative
AGNEWS |  topic, category, politics, sports, business, technology
Table 5: Domain words for each task.  | SST-2 | Amazon | AGNEWS
---|---|---|---
| Acc. | Freq. | Acc. | Freq. | Acc. | Freq.
Effective | 89.4 | 23.4 | 86.5 | 5.8 | 77.6 | 3.7
Random | 67.2 | 1.3 | 74.2 | 2.2 | 49.3 | 0.8
Table 6: Average domain words frequency (Freq.) and average accuracy (Acc.)
for effective and random prompts. Effective prompts and their continuation
contain subsantially more domain words than random prompts. The p-values from
the paired t-test for SST-2, Amazon, and AGNEWS were 0.004, 0.003, and 0.0002,
respectively.

##  5 Unsupervised FluentPrompt

Our findings in Section 4 show the effective tuned prompts do calibration and
have a high domain relevance to the task. These two attributes are both highly
predictive and do not require ground-truth labels to compute. Therefore, in
this section, we extend FluentPrompt to explicitly tune the prompts towards
better calibration and domain relevance, without using the task labels.

###  5.1 Method

#### Calibration loss

In Section 4.1 we find a strong positive correlation between the degree of
calibration (i.e., entropy) and performance of the prompts. We therefore
explicitly optimize the prompt towards greater calibration, with an (negative)
entropy loss defined below.

$\displaystyle\mathcal{L}_{\text{entropy}}(\tilde{\boldsymbol{e}})$
$\displaystyle=\mathbb{E}_{y\in Y}[\log\mathbb{E}_{\boldsymbol{x}\in
X}p_{\theta}(v(y)\mid\tilde{\boldsymbol{e}},\boldsymbol{x},\boldsymbol{t})]$

Intuitively the entropy loss encourages the prompt to help model generate more
balanced predictions at a group level.

#### Domain relevance loss

In Section 4.2 we find effective prompts overall are more related to the task
domain, as defined by augmented data and keyword matches. To explicitly
incorporate the domain relevance to the prompts, we extend the existing
fluency (perplexity) loss in Section 3.3, modeling the perplexity of both the
prepending prompt and the input example:

$\displaystyle\mathcal{L}_{\text{domain}}(\tilde{\boldsymbol{e}})=$
$\displaystyle-\log p_{\theta}(\tilde{\boldsymbol{e}}_{0:M})$
$\displaystyle-\sum_{i}\log
p_{\theta}(x_{i}\mid\tilde{\boldsymbol{e}},\boldsymbol{x}_{<i})$
$\displaystyle-\sum_{j}\log
p_{\theta}(t_{j}\mid\tilde{\boldsymbol{e}},\boldsymbol{x},\boldsymbol{t}_{<j})$

Intuitively, $\log p_{\theta}(\boldsymbol{x}\mid\tilde{\boldsymbol{e}})-\log
p_{\theta}(\boldsymbol{x})$ would measure the pointwise mutual information
between the task data $\boldsymbol{x}$ and the tuned prompt $\boldsymbol{x}$,
with the part $\log p_{\theta}(\boldsymbol{x})$ not involved in the prompt
optimization.

Overall, our unsupervised energy function $\mathcal{E}$ is updated to:

$\displaystyle\mathcal{E}(\tilde{\boldsymbol{e}}_{0:M})=$
$\displaystyle-\lambda_{\text{calibration}}\mathcal{L}_{\text{entropy}}(\tilde{\boldsymbol{e}})$
$\displaystyle-\lambda_{\text{domain}}\mathcal{L}_{\text{domain}}(\tilde{\boldsymbol{e}})$

where $\lambda_{\text{calibration}}+\lambda_{\text{domain}}=1$.

#### Hyperparameters

Inheriting the notations of FluentPrompt, we consider the following
hyperparameters: $\eta\in$ {1.0, 3.0}, $\beta_{\text{start}}=$ 1.0,
$\beta_{\text{end}}=$ 0.0001, $\lambda_{\text{domain}}\in$ {0, 0.0003, 0.001,
0.003, 0.01, 0.05, 0.2, 0.5}, $M=$ 10\. We use five random seeds for each
setup and report the average performance.

###  5.2 Results

In Table 7, we compare the performance of our proposed method, Unsupervised
FluentPrompt, with two other unsupervised methods, the empty prompt and PMI
calibration $\text{PMI}_{\text{DC}}$ Holtzman et al. (2021) on three datasets.
Our results show that Unsupervised FluentPrompt consistently outperforms
$\text{PMI}_{\text{DC}}$ with an average improvement of 7.0% across the
datasets. This demonstrates the incorporated calibration and domain
information are helpful to finding effective prompts.

| SST-2 | Amazon | AGNEWS
---|---|---|---
Unsupervised |  |  |
Emtpy | 66.5 | 75.8 | 49.7
$\text{PMI}_{\text{DC}}$ | 85.6 | 76.2 | 64.1
Unsup. F.P. | 88.2 | 85.3 | 68.0
Table 7: Accuracy of different unsupervised prompting methods on the three
datasets. Unsup. F.P. refers to our Unsupervised FluentPrompt.

##  6 Conclusion

In this paper, we investigate the factors that contribute to the effectiveness
of prompts. To facilitate this study, we develop a human-readable prompt
tuning method FluentPrompt and apply it to the GPT-2 large model to generate
effective and readable prompts. Our analysis reveals that effective prompts
are topically related to the task domain and calibrate the prior probability
of label words.

Although the prompts generated by FluentPrompt are effective and readable,
they still carry limited semantic meanings. For instance, we did not find any
prompts directly indicating the task definition or instructions. One potential
reason is that the GPT-2 large model is not instruction-tuned. Future work can
apply FluentPrompt to an instruction-tuned model to see if instruction-like
prompts can be discovered.

## References

  * Bach et al. (2022) Stephen Bach, Victor Sanh, Zheng Xin Yong, Albert Webson, Colin Raffel, Nihal V Nayak, Abheesht Sharma, Taewoon Kim, M Saiful Bari, Thibault Févry, et al. 2022.  Promptsource: An integrated development environment and repository for natural language prompts.  In _Proceedings of the 60th Annual Meeting of the Association for Computational Linguistics: System Demonstrations_ , pages 93–104.
  * Brown et al. (2020) Tom Brown, Benjamin Mann, Nick Ryder, Melanie Subbiah, Jared D Kaplan, Prafulla Dhariwal, Arvind Neelakantan, Pranav Shyam, Girish Sastry, Amanda Askell, et al. 2020.  Language models are few-shot learners.  _Advances in neural information processing systems_ , 33:1877–1901.
  * Chen et al. (2022) Yanda Chen, Chen Zhao, Zhou Yu, Kathleen McKeown, and He He. 2022.  On the relation between sensitivity and accuracy in in-context learning.  _arXiv preprint arXiv:2209.07661_.
  * Dagan et al. (2005) Ido Dagan, Oren Glickman, and Bernardo Magnini. 2005.  The pascal recognising textual entailment challenge.  In _Machine learning challenges workshop_ , pages 177–190. Springer.
  * Deng et al. (2022) Mingkai Deng, Jianyu Wang, Cheng-Ping Hsieh, Yihan Wang, Han Guo, Tianmin Shu, Meng Song, Eric P Xing, and Zhiting Hu. 2022.  Rlprompt: Optimizing discrete text prompts with reinforcement learning.  _arXiv preprint arXiv:2205.12548_.
  * Ding et al. (2021) Ning Ding, Shengding Hu, Weilin Zhao, Yulin Chen, Zhiyuan Liu, Hai-Tao Zheng, and Maosong Sun. 2021.  Openprompt: An open-source framework for prompt-learning.  _CoRR_ , abs/2111.01998.
  * Gao et al. (2020) Leo Gao, Stella Biderman, Sid Black, Laurence Golding, Travis Hoppe, Charles Foster, Jason Phang, Horace He, Anish Thite, Noa Nabeshima, et al. 2020.  The pile: An 800gb dataset of diverse text for language modeling.  _arXiv preprint arXiv:2101.00027_.
  * Gonen et al. (2022) Hila Gonen, Srini Iyer, Terra Blevins, Noah A Smith, and Luke Zettlemoyer. 2022\.  Demystifying prompts in language models via perplexity estimation.  _arXiv preprint arXiv:2212.04037_.
  * Hambardzumyan et al. (2021) Karen Hambardzumyan, Hrant Khachatrian, and Jonathan May. 2021.  WARP: Word-level Adversarial ReProgramming.  In _Proceedings of the 59th Annual Meeting of the Association for Computational Linguistics and the 11th International Joint Conference on Natural Language Processing (Volume 1: Long Papers)_ , pages 4921–4933, Online. Association for Computational Linguistics.
  * Holtzman et al. (2019) Ari Holtzman, Jan Buys, Li Du, Maxwell Forbes, and Yejin Choi. 2019.  The curious case of neural text degeneration.  In _International Conference on Learning Representations_.
  * Holtzman et al. (2021) Ari Holtzman, Peter West, Vered Shwartz, Yejin Choi, and Luke Zettlemoyer. 2021\.  Surface form competition: Why the highest probability answer isn’t always right.  In _Proceedings of the 2021 Conference on Empirical Methods in Natural Language Processing_ , pages 7038–7051, Online and Punta Cana, Dominican Republic. Association for Computational Linguistics.
  * Khashabi et al. (2021) Daniel Khashabi, Shane Lyu, Sewon Min, Lianhui Qin, Kyle Richardson, Sean Welleck, Hannaneh Hajishirzi, Tushar Khot, Ashish Sabharwal, Sameer Singh, and Yejin Choi. 2021.  Prompt waywardness: The curious case of discretized interpretation of continuous prompts.
  * Kojima et al. (2022) Takeshi Kojima, Shixiang Shane Gu, Machel Reid, Yutaka Matsuo, and Yusuke Iwasawa. 2022.  Large language models are zero-shot reasoners.  _arXiv preprint arXiv:2205.11916_.
  * Kumar et al. (2022) Sachin Kumar, Biswajit Paria, and Yulia Tsvetkov. 2022.  Gradient-based constrained sampling from language models.  _arXiv preprint arXiv:2205.12558_.
  * Lester et al. (2021) Brian Lester, Rami Al-Rfou, and Noah Constant. 2021.  The power of scale for parameter-efficient prompt tuning.  In _Proceedings of the 2021 Conference on Empirical Methods in Natural Language Processing_ , pages 3045–3059.
  * Li et al. (2016) Jiwei Li, Michel Galley, Chris Brockett, Jianfeng Gao, and Bill Dolan. 2016.  A diversity-promoting objective function for neural conversation models.  In _Proceedings of the 2016 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies_ , pages 110–119, San Diego, California. Association for Computational Linguistics.
  * Liu et al. (2022) Jiachang Liu, Dinghan Shen, Yizhe Zhang, William B Dolan, Lawrence Carin, and Weizhu Chen. 2022.  What makes good in-context examples for gpt-3?  In _Proceedings of Deep Learning Inside Out (DeeLIO 2022): The 3rd Workshop on Knowledge Extraction and Integration for Deep Learning Architectures_ , pages 100–114.
  * Liu et al. (2021) Xiao Liu, Yanan Zheng, Zhengxiao Du, Ming Ding, Yujie Qian, Zhilin Yang, and Jie Tang. 2021.  Gpt understands, too.  _arXiv preprint arXiv:2103.10385_.
  * Loshchilov and Hutter (2018) Ilya Loshchilov and Frank Hutter. 2018.  Decoupled weight decay regularization.  In _International Conference on Learning Representations_.
  * Lu et al. (2022) Yao Lu, Max Bartolo, Alastair Moore, Sebastian Riedel, and Pontus Stenetorp. 2022\.  Fantastically ordered prompts and where to find them: Overcoming few-shot prompt order sensitivity.  In _Proceedings of the 60th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers)_ , pages 8086–8098.
  * McAuley and Leskovec (2013) Julian McAuley and Jure Leskovec. 2013.  Hidden factors and hidden topics: understanding rating dimensions with review text.  In _Proceedings of the 7th ACM conference on Recommender systems_ , pages 165–172.
  * Min et al. (2022) Sewon Min, Xinxi Lyu, Ari Holtzman, Mikel Artetxe, Mike Lewis, Hannaneh Hajishirzi, and Luke Zettlemoyer. 2022.  Rethinking the role of demonstrations: What makes in-context learning work?  _arXiv preprint arXiv:2202.12837_.
  * Mishra et al. (2021) Swaroop Mishra, Daniel Khashabi, Chitta Baral, and Hannaneh Hajishirzi. 2021.  Cross-task generalization via natural language crowdsourcing instructions.  _arXiv preprint arXiv:2104.08773_.
  * Prasad et al. (2022) Archiki Prasad, Peter Hase, Xiang Zhou, and Mohit Bansal. 2022.  Grips: Gradient-free, edit-based instruction search for prompting large language models.  _arXiv preprint arXiv:2203.07281_.
  * Qin and Eisner (2021) Guanghui Qin and Jason Eisner. 2021.  Learning how to ask: Querying LMs with mixtures of soft prompts.  In _Proceedings of the 2021 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies_ , pages 5203–5212, Online. Association for Computational Linguistics.
  * Radford et al. (2019) Alec Radford, Jeffrey Wu, Rewon Child, David Luan, Dario Amodei, Ilya Sutskever, et al. 2019.  Language models are unsupervised multitask learners.  _OpenAI blog_ , 1(8):9.
  * Sanh et al. (2021) Victor Sanh, Albert Webson, Colin Raffel, Stephen H. Bach, Lintang Sutawika, Zaid Alyafeai, Antoine Chaffin, Arnaud Stiegler, Teven Le Scao, Arun Raja, Manan Dey, M. Saiful Bari, Canwen Xu, Urmish Thakker, Shanya Sharma, Eliza Szczechla, Taewoon Kim, Gunjan Chhablani, Nihal V. Nayak, Debajyoti Datta, Jonathan Chang, Mike Tian-Jian Jiang, Han Wang, Matteo Manica, Sheng Shen, Zheng Xin Yong, Harshit Pandey, Rachel Bawden, Thomas Wang, Trishala Neeraj, Jos Rozen, Abheesht Sharma, Andrea Santilli, Thibault Févry, Jason Alan Fries, Ryan Teehan, Stella Biderman, Leo Gao, Tali Bers, Thomas Wolf, and Alexander M. Rush. 2021.  Multitask prompted training enables zero-shot task generalization.  _CoRR_ , abs/2110.08207.
  * Schick and Schütze (2021) Timo Schick and Hinrich Schütze. 2021.  Exploiting cloze-questions for few-shot text classification and natural language inference.  In _Proceedings of the 16th Conference of the European Chapter of the Association for Computational Linguistics: Main Volume_ , pages 255–269, Online. Association for Computational Linguistics.
  * Shin et al. (2020) Taylor Shin, Yasaman Razeghi, Robert L Logan IV, Eric Wallace, and Sameer Singh. 2020.  Autoprompt: Eliciting knowledge from language models with automatically generated prompts.  In _Proceedings of the 2020 Conference on Empirical Methods in Natural Language Processing (EMNLP)_ , pages 4222–4235.
  * Socher et al. (2013) Richard Socher, Alex Perelygin, Jean Wu, Jason Chuang, Christopher D. Manning, Andrew Ng, and Christopher Potts. 2013.  Recursive deep models for semantic compositionality over a sentiment treebank.  In _Proceedings of the 2013 Conference on Empirical Methods in Natural Language Processing_ , pages 1631–1642, Seattle, Washington, USA. Association for Computational Linguistics.
  * Wang et al. (2022) Yizhong Wang, Swaroop Mishra, Pegah Alipoormolabashi, Yeganeh Kordi, Amirreza Mirzaei, Anjana Arunkumar, Arjun Ashok, Arut Selvan Dhanasekaran, Atharva Naik, David Stap, Eshaan Pathak, Giannis Karamanolakis, Haizhi Gary Lai, Ishan Purohit, Ishani Mondal, Jacob Anderson, Kirby Kuznia, Krima Doshi, Maitreya Patel, Kuntal Kumar Pal, Mehrad Moradshahi, Mihir Parmar, Mirali Purohit, Neeraj Varshney, Phani Rohitha Kaza, Pulkit Verma, Ravsehaj Singh Puri, Rushang Karia, Shailaja Keyur Sampat, Savan Doshi, Siddhartha Mishra, Sujan Reddy, Sumanta Patro, Tanay Dixit, Xudong Shen, Chitta Baral, Yejin Choi, Noah A. Smith, Hannaneh Hajishirzi, and Daniel Khashabi. 2022.  Super-naturalinstructions: Generalization via declarative instructions on 1600+ nlp tasks.
  * Webson and Pavlick (2021) Albert Webson and Ellie Pavlick. 2021.  Do prompt-based models really understand the meaning of their prompts?  _arXiv preprint arXiv:2109.01247_.
  * Zhang et al. (2015) Xiang Zhang, Junbo Jake Zhao, and Yann LeCun. 2015.  Character-level convolutional networks for text classification.  In _NIPS_.
  * Zhao et al. (2021) Zihao Zhao, Eric Wallace, Shi Feng, Dan Klein, and Sameer Singh. 2021.  Calibrate before use: Improving few-shot performance of language models.  In _International Conference on Machine Learning_ , pages 12697–12706. PMLR.
  * Zhou et al. (2022) Yongchao Zhou, Andrei Ioan Muresanu, Ziwen Han, Keiran Paster, Silviu Pitis, Harris Chan, and Jimmy Ba. 2022.  Large language models are human-level prompt engineers.  _arXiv preprint arXiv:2211.01910_.

##  Appendix A Verbalizer and templates

Table 8 shows an example input, template and the verbalizer used for each
task.

Task |  Templates | Verbalizers
---|---|---
SST-2 |  Illuminating if overly talky documentary. It was | positive, negative
Amazon |  Terrible service. It was | positive, negative
AGNEWS |  Economic growth in Japan slows down as the country experiences. It is about | politics, sports, business, technology
Table 8: The template, example (colored black) and verbalizer used for each
dataset.