# Strong-field ionization of chiral molecules with bicircular laser fields :
sub-barrier dynamics, interference, and vortices.

S. Beaulieu,1 S. Larroque,1 D. Descamps,1 B. Fabre,1 S. Petit,1, R. Taïeb,2 B.
Pons,1 Y. Mairesse1 1 Université de Bordeaux – CNRS – CEA, CELIA, UMR5107,
Talence, France  2 Sorbonne Université, CNRS, Laboratoire de Chimie Physique –
Matière et Rayonnement, LCPMR, 75005 Paris, France

###### Abstract

Strong-field ionization by counter-rotating two-color laser fields produces
quantum interference between photoelectrons emitted on the leading and
trailing edges of the laser field oscillations. We show that in chiral
molecules, this interference is asymmetric along the light propagation
direction and strongly enhances the sensitivity of the attoclock scheme to
molecular chirality. Calculations in a toy-model molecule with a short-range
chiral potential show that this enhanced sensitivity already emerges at the
exit of the tunnel. We investigate the possible sources of chiral sensitivity
in the tunneling process, and find that the interference between electron
vortices plays a crucial role in the chiral response.

##  I Introduction

The dynamics of under-the-barrier particles in quantum tunneling has gained a
lot of attention in the past few years Ramos  _et al._ (2020). Strong-field
light-matter interaction provides an interesting playground to investigate
tunneling on the attosecond timescale Kaushal and Smirnova (2018a); Hofmann
_et al._ (2019); Kheifets (2020). When an atom or a molecule is submitted to a
strong infrared laser field, the potential barrier evolves over a timescale of
a few hundreds of attoseconds. The electrons can tunnel through the barrier
when it is sufficiently low, and the dynamical evolution of the barrier
dictates the properties of the emerging wavepacket – momentum and phase
Lewenstein  _et al._ (1994a); Ivanov  _et al._ (2005). If the direction of the
laser field evolves in time (i.e. if the field is not linearly polarized), the
2D evolution of the barrier imprints vectorial properties on the tunneling
electrons, determining the initial direction of their motion in the continuum.
In that context, molecular chirality offers an interesting approach to reveal
sub-barrier effects in strong-field ionization Bloch  _et al._ (2021).

When a chiral molecule is ionized by circularly polarized light, the ionic
potential induces a forward-backward asymmetry in the ejected electron
wavepacket along the light propagation direction, an effect called
PhotoElectron Circular Dichroism (PECD) Ritchie (1975); Powis (2000); Böwering
_et al._ (2001); Nahon  _et al._ (2015); Lux  _et al._ (2012); Lehmann  _et
al._ (2013); Beaulieu  _et al._ (2016). In the single-photon ionization
regime, PECD is imprinted during the scattering of the electron off the short-
range chiral potential. In the strong-field ionization (SFI) regime, the
electron is released though a sequential process : (i) tunneling through the
potential barrier lowered by the laser field; (ii) acceleration by the laser
field. The electron emerges from the tunnel a few atomic units away from the
core, in a region where the ionic potential is mostly isotropic. However, PECD
is still observed in strong-field ionization Beaulieu  _et al._ (2016); Müller
_et al._ (2020), and can be imprinted through both steps of the SFI process.

We have recently performed a joint experimental and theoretical investigation
using the attoclock technique Eckle  _et al._ (2008) to decouple the influence
of short and long-range effects in chiral photoionization Bloch  _et al._
(2021). The attoclock consists in measuring the momentum distribution of
photoelectrons produced by ionizing a target with a strong elliptically (or
more complex polarization-shaped Eicke and Lein (2019)) laser field. The
rotation of the laser field induces an angular streaking of the electron
distribution, which maps the electron interaction with the ionic potential
Torlina  _et al._ (2015); Sainadh  _et al._ (2019) and possibly its finite
tunneling time Hofmann  _et al._ (2019). In chiral molecules, experiments
using elliptically polarized Fehre  _et al._ (2019a) or co-rotating bicircular
laser fields Bloch  _et al._ (2021) established that the angular streaking
produces different attoclock angles for electrons ejected forward and backward
along the light propagation direction. This asymmetry was found to vanish in
calculations when the long-range part of the ionic potential was screened
Bloch  _et al._ (2021), even though the number of electrons ejected forward
and backward still differed. The attoclock thus seemed rather insensitive to
short range effects in chiral photoionization.

In this article, we investigate the ionization of chiral molecules by
bicircular bichromatic fields. In section II, we show that characteristic
interference patterns appear in the photoelectron momentum distributions when
the two components of the field are counter-rotating. Section III describes
measurements of strong-field ionization of chiral molecules using co- and
counter-rotating bicircular fields, which demonstrate the high sensitivity of
the interference pattern to chirality. Simulations in a short-range chiral
potential confirm the importance of interference in the chiral response,
raising the question of the origin of the chiral sensitivity in the tunneling
process. This question is addressed in Section IV, using simulations within
the Strong-Field Approximation (SFA) Lewenstein  _et al._ (1994a) to
investigate the sub-barrier dynamics in various field configurations –
circular, co-rotating and counter-rotating. We then model the chiral
photoionization using different sources of chirosensitivity, and reveal the
influence of interference of photoelectron vortices in counter-rotating
fields. Last, in Section V we conclude our study by showing that changing the
ratio between the two components of the bicircular fields can increase the
contrast of the interference, resulting in very strong forward/backward
asymmetries in the angular distributions close to destructive interference
locations.

##  II Interference in counter-rotating bicircular photoionization

Strong-field ionization generally releases two sets of electron trajectories.
The electrons released before the maximum of the laser field move away from
the core without turning back. By contrast the electrons released after the
maximum of the field start their motion in one direction before reversing it,
and can thus revisit the ionic core before being accelerated away by the laser
field. These two families of electron trajectories are generally labeled
”direct” and ”indirect”, respectively de Bohan  _et al._ (2002). In an
elliptically polarized laser field, direct and indirect electrons are
angularly separated by the rotation of the laser field, and end up with
different final momenta. By contrast, in counter-rotating bicircular fields,
the trajectories released by consecutive maxima of the laser field can overlap
in momentum space, leading to a quantum interference process. To illustrate
this, we consider a bicircular field consisting of a circularly polarized
field at fundamental frequency $\omega=1.55$ eV (800 nm wavelength) and
intensity $I_{\omega}=3\times 10^{13}$ W/cm2, and its counter-rotating second
harmonic whose intensity $I_{2\omega}$ is defined by the ratio
$r=I_{2\omega}/I_{\omega}=0.1$ (Fig. 1(a)). Figure 1 shows the photoelectron
momentum distribution obtained within the strong-field approximation using the
saddle-point method Lewenstein  _et al._ (1994a), for a target with 7.2 eV
ionization potential. Since the influence of the ionic potential on the
electron dynamics is neglected, the final momentum of the electron
$\mathbf{p}_{f}$ is dictated by the value of the vector potential $\mathbf{A}$
at the time of ionization $t_{i}$ and the initial momentum of the electron at
the exit of the tunnel $\mathbf{p}_{i}$:
$\mathbf{p}_{f}=-\mathbf{A}(t_{i})+\mathbf{p}_{i}$. The counter-rotating laser
field has a C3 symmetry, maximizing three times per laser period. It
consequently releases three wavepackets per period, whose momentum-resolved
modulus in the laser polarization plane is shown in Fig. 1(b). The wavepackets
follow the shape of $\mathbf{-A}(t)$ (depicted as a green line), and each
wavepacket is divided in two parts, corresponding to direct and indirect
electrons, released respectively before and after the maxima of the filed
oscillations. The direct and indirect wavepackets from adjacent laser sub-
cycles overlap in momentum space, and are thus expected to interfere. To
determine the influence of this interference, we compare in Fig. 1(c,d) the
photoelectron angular distributions produced by a multicycle laser pulse, in
which the contribution of the three third cycles are incoherently (b) or
coherently (c) summed, i.e. obtained respectively by summing the modulus
square of the three wavepackets or by calculating the modulus square of the
sum of the complex wavepackets. The distributions are made of concentric
rings, corresponding to the above-threshold ionization (ATI) peaks Agostini
_et al._ (1979). The second ATI peak maximizes around azimuthal angle
$\varphi_{0}=90^{\circ}[120^{\circ}]$ in the incoherent signal, and
$\varphi_{0}=150^{\circ}[120^{\circ}]$ in the coherent signal (locations
marked by dots in Fig. 1(c,d)). The interference between direct and indirect
electrons thus completely reverses the azimuthal angular dependence of the
photoelectron yield. The interference can be further studied by investigating
the dependence of the electron azimuthal distribution as a function of the
electron ejection angle $\theta$, defined in Fig. 1(a). In the incoherent
signal (Fig. 1(e)), the position of the three lobes in the first ATI peak is
independent of $\theta$. By contrast, in the coherent signal, a clear phase
jump appears in the distribution around $\theta=45^{\circ}$. The interference
also induces a splitting of the azimuthal profile of higher ATI peaks into 6
lobes.

Figure 1: Principle of photoelectron interferometry in a counter-rotating
bicircular field shown as red line in (a). (b) Modulus of the wavepackets
released around each peak of the electric field, within the SFA, overlapped
with the opposite of the vector potential (green). The direct and indirect
parts of each wavepacket are depicted respectively in plain or transparent.
(c-d) Photoelectron momentum distribution, projected in the laser polarization
plane, obtained by incoherently (c) or coherently (d) summing the
contributions of the three maxima of a four-cycle laser pulse. The dots mark
the azimuthal maxima of the second ATI peak. (e-h) Azimuthal profile of the
photoelectron momentum distribution as a function of the electron ejection
angle $\theta$ with respect to the polarization plane, obtained incoherently
(e) and coherently (f) for the second ATI peak, and coherently for ATI3 (g)
and ATI4 (h).

This simple study demonstrates the importance of interference in strong field
ionization by counter-rotating bicircular fields, even within a framework in
which the influence of the ionic potential is neglected. The constructive or
destructive nature of the interference is dictated by the phase between direct
and indirect electrons. The different ionization dynamics of direct and
indirect electrons is sufficient to induce a strong $\theta$-dependence of
this phase, which transposes into a $60^{\circ}$ jump of the azimuthal
distribution at a specific ejection angle $\theta$ for each ATI peak. Within
the SFA, the lobes of the photoelectron azimuthal distribution are necessarily
aligned the directions defined by the field, i.e. along
$\varphi_{0}=90^{\circ}[120^{\circ}]$ or
$\varphi_{0}=150^{\circ}[120^{\circ}]$. Beyond the SFA, the influence of the
ionic potential on the departing electrons is expected to induce an angular
shift $\varphi^{ion}(\theta)$ of the lobes, as observed in attoclock
measurements Eckle  _et al._ (2008); Torlina  _et al._ (2015); Sainadh  _et
al._ (2019); Hofmann  _et al._ (2019), but should also affect the interference
pattern because of scattering phase shifts.

##  III Chiral attoclock in co- and counter rotating fields

###  III.1 Measurements

In order to experimentally assess the sensitivity of the interferometric
attoclock, we measured the 3D photoelectron angular distributions produced by
photoionizing fenchone molecules with bicircular fields. We used the 1 kHz 25
fs Ti:Sa Aurore laser system at CELIA Fedorov  _et al._ (2020). The bicircular
field was generated in a Mach-Zehnder interferometer and focused in chiral
molecules introduced in the interaction region of a Velocity Map Imaging
Spectrometer as described in Bloch  _et al._ (2021). The 3D photoelectron
angular distribution of the electrons was reconstructed from 31 projections
using a tomographic inversion method based on inverse Radon transform
Wollenhaupt  _et al._ (2009); Mancuso  _et al._ (2015). The chirosensitive
part of the signal was extracted by subtracting the 3D photoelectron angular
distribution obtained at a given helicity of the bicircular field to the
mirror image of that obtained at opposite field helicity, as detailed in Bloch
_et al._ (2021).

Figure 2: Interferometric attoclock measurements in (+)-fenchone using
bicircular fields with $I_{\omega}\approx 3\times 10^{13}$ W$\cdot$cm-2,
$r=I_{2\omega}/I_{\omega}=0.1$ and right circular polarization of the
fundamental. (a-e) Photoelectron signal of the first five ATI peaks in the
case of counter-rotating field configuration, as a function of the electron
ejection angle $\theta$ and streaking angle $\varphi$. The signal is
normalized at each $\theta$ by its $\varphi$-averaged value. The red lines
represent the maximizing azimuthal angle $\varphi_{0}(\theta)$ extracted by
Fourier analysis. (f-j) Chiral angular shift $\Delta\varphi_{0}(\theta)$
between electrons ejected forward and backward in counter-rotating (red) and
co-rotating (blue) laser fields. (k-o) Forward/backward asymmetry in the
photoelectron yield, using counter-rotating (red) or co-rotating (blue)
fields.

Figure 2 shows the azimuthal profile of the first five ATI peaks produced in
(+)-fenchone molecules using a counter-rotating bicircular field defined by
$I_{\omega}\approx 3\times 10^{13}$ W.cm-2 and $r=0.1$, as a function of the
electron ejection angle $\theta$. At each $\theta$, the ATI yield is
normalized by its $\varphi$-average value to enhance visibility. All ATI peaks
show a clear threefold pattern along the azimuthal angle $\varphi$,
corresponding to the ejection of three electron bunches per period of the
field. The azimuthal profile of the first ATI peak is rather similar to the
one obtained in SFA calculations, the lobes presenting a phase jump. The phase
jump occurs at lower angles ($\theta\approx\pm 30^{\circ}$) than in the SFA
($\theta\approx\pm 45^{\circ}$) and its magnitude is larger than $60^{\circ}$.
This reflects the sensitivity of the interferometric scheme to the influence
of the ionic potential on the electron trajectories. Higher ATI peaks show a
more complex evolution as a function of the ejection angle.

The chirosensitive part of the signal is obtained by comparing the angle
$\varphi$ maximizing the azimuthal distribution (red line in Fig. 1(a-e)) in
the forward and backward directions, and calculating the chiral angular shift
$\Delta\varphi_{0}(\theta)=\varphi_{0}(\theta)-\varphi_{0}(-\theta)$. All ATI
peaks show a clear chiral angular shift, with a strong dependence on the
electron ejection angle. For low ATI peaks, for which the evolution of the
pattern is rather smooth, $\Delta\varphi_{0}$ is of the order of a few
degrees, and switches sign across the phase jump in the interference. For
higher ATI peaks, the chiral angular shift reaches high values (15∘ for ATI5),
around the ejection angle at which a sharp phase jump is observed in the
interference. This is due to the fact that the position of the phase jump
slightly differs for electrons ejected in the forward and backward directions.
For comparison, we plot the results of attoclock measurements with a co-
rotating bicircular field (blue lines in fig. 2 f-j), presented in Bloch  _et
al._ (2021). In this field configuration, the electron momentum distribution
is sharper and the signal vanishes for $\theta>40^{\circ}$. For the first ATI
peak, the chiral angular shift reaches values above 10∘ at $\theta\sim
30^{\circ}$, but it drops below 3∘ for higher ATI peaks. Within the
investigated regime, the counter-rotating scheme thus possesses a superior
sensitivity to chirality.

We can compare the attoclock measurements to a more conventional detection
scheme which consists in determining the number of electrons ejected forward F
($\theta>0$) and backward B ($\theta<0$), integrated over $\varphi$, and
calculating the Forward/Backward Asymmetry as FBA=2(F-B)/(F+B) (Fig. 2(k-o)).
The FBA signals obtained in co- and counter-rotating fields show the same
order of magnitude and overall trends, but the counter-rotating case produces
additional modulations of the FBA as a function of electron ejection angle,
which we interpret as signatures of the interference process.

###  III.2 TDSE simulations using a short-range potential

To shed light on the origin of the chiral sensitivity of SFI in bicircular
fields, we performed quantum-mechanical calculations solving the Time-
Dependent Schrödinger Equation (TSDE) using a toy-model chiral molecule. The
details of the model can be found in Bloch  _et al._ (2021). We employed the
same laser field as in the experiments, defined by $I_{\omega}=3\times
10^{13}$ W$\cdot$cm-2 and $r=0.1$, and carried out the same analysis as for
the experimental data. In order to isolate the influence of short-range
effects, we damped the long-range chiral potential beyond a distance
$r_{0}=3.5$ a.u. by multiplying it by an isotropic Yukawa cut-off term
$\exp^{-(r-r_{0})}$ Torlina  _et al._ (2015); Bloch  _et al._ (2021).

The angular distributions of the first four ATI peaks are shown in Fig. 3.
They resemble the results of SFA calculations (Fig. 1), which reflects the
efficiency of the potential screening by the above-described Yukawa cut-off
term. Figure 3(e-l) shows the analysis of the chiral attoclock, in co-rotating
(blue) and counter-rotating (red) fields. When co-rotating fields are used,
the chiral angular shift decreases with increasing electron kinetic energy
($\Delta\varphi_{0}<1^{\circ}$ for ATI3 and $\Delta\varphi_{0}<0.1^{\circ}$
for ATI4), as observed in Bloch  _et al._ (2021). By contrast, in counter-
rotating fields a clear angular shift is measured for all ATI peaks. The shift
maximizes at the location of the phase jump in the interference pattern, as
shown in Fig. 3(a), reaching 9∘ for the third ATI peak.

Figure 3: Interferometric attoclock calculations in toy-model chiral molecules
with a screened long-range potential, subject to bicircular fields with
$I=3\times 10^{13}$ W$\cdot$cm-2, $r=I_{2\omega}/I_{\omega}=0.1$ and right
circular polarization of the fundamental. (a-d) Photoelectron signal of the
first four ATI peaks in the case of counter-rotating field configuration, as a
function of the electron ejection angle $\theta$ and streaking angle
$\varphi$. The signal is normalized at each $\theta$ by its $\varphi$-averaged
value. The red lines represent the maximizing azimuthal angle
$\varphi_{0}(\theta)$ extracted by Fourier analysis. (e-h) Chiral angular
shift $\Delta\varphi_{0}(\theta)$ between electrons ejected forward and
backward in counter-rotating (red) and co-rotating (blue) laser fields. (i-l)
Forward/backward asymmetry in the photoelectron yield, using counter-rotating
(red) or co-rotating (blue) fields.

The forward/backward asymmetries are of the same order of magnitude in co- and
counter-rotating field configurations, even if their signs are opposite for
the intermediate ATI2-3 peaks. The asymmetry exhibits a sign switch in
counter-rotating field configuration, at electron ejection angles where the
chiral attoclock shows interference patterns.

The persistence of a clear chiral angular shift in the SFI of molecules with a
screened long-range potential is intriguing. In the absence of potential
during the electron acceleration of the continuum, the azimuthal angle
maximizing the electron distributions should be $\varphi_{0}=90^{\circ}$ or
$150^{\circ}$, depending on the constructive or destructive nature of the
interference between direct and indirect electrons. A phase shift between
direct and indirect electrons emerging from the tunnel would change the
position of the phase jump, but cannot rotate the distribution away from these
$\varphi_{0}$ values within the SFA. On the other hand, a difference in
amplitude between direct and indirect electrons would induce an effective
rotation of the electron momentum distribution. To understand the origin of
the interferometric chiral attoclock shift, we take a closer look at the SFA
calculations by monitoring the electron trajectories.

##  IV Strong-field electron dynamics in bicircular fields

###  IV.1 Electron trajectories in the complex plane

The SFA framework Lewenstein  _et al._ (1994b) provides a complete description
of the electron motion, including the sub-barrier dynamics. The electron
motion starts at time $t_{i}=t_{i}^{\prime}+it_{i}^{\prime\prime}$, where
$t_{i}^{\prime}$ and $t_{i}^{\prime\prime}$ are respectively the real and
imaginary parts of the time. As time evolves along the imaginary axis from
$t=t_{i}^{\prime}+it_{i}^{\prime\prime}$ to the real ionization time
$t=t_{i}^{\prime}$, the motion occurs under the potential barrier, in the
classically-forbidden region, and is described in the complex plane Smirnova
and Ivanov (2014). The electron emerges from the tunnel with an initial
velocity $\mathbf{v}(t_{i}^{\prime})$ and is accelerated by the laser field to
reach its final momentum $\mathbf{p_{f}}$. Figure 4(a) shows the momentum
distribution obtained by ionizing a target with 7.2 eV ionization potential by
a counter-rotating bicircular field with $I_{\omega}=3\times 10^{13}$ W.cm-2
and $r=0.1$. We restricted the calculation to a single wavepacket emitted by
the first (continuous lines, WP1) or second (dashed lines, WP2) third of the
whole laser cycle, and repeated it over two consecutive cycles of the
fundamental field to produce ATI peaks. The colored crosses mark the final
momentum of electrons from the first five ATI peaks, which are produced with
the same probability by the first or second wavepacket. The selected electrons
belonging to the first wavepacket are emitted after the peak of the laser
field, and are thus indirect, while the selected electrons belonging to the
second wavepacket are direct. In order to monitor the differences between the
electron ejection dynamics associated with both these wavepackets, we plot in
Fig. 4(b) the real part of their trajectories. The electrons start their
motion in the sub-barrier region in the direction opposite to the laser field
$\mathbf{E}(t^{\prime}_{i})$ (represented as arrows), and exit the tunnel
around 6.5 a.u. from the origin (radius of the black circle). The sub-barrier
trajectories are slightly curved, with opposite handedness for direct and
indirect electrons. The electron motion in the continuum is very different for
the two families, in particular due to their different velocities at the exit
of the tunnel. The velocity exit component $v_{\parallel}(t^{\prime}_{i})$,
parallel to the laser electric field $\mathbf{E}(t^{\prime}_{i})$, is positive
for direct trajectories and negative for indirect ones. The transverse exit
velocity $v_{\perp}$ changes sign between low and high order ATI peaks, but
shows the same sign for direct and indirect electrons. In the imaginary plane
(Fig. 4(c)), where the electron motion is restricted to the sub-barrier
region, the trajectories are strongly curved. Interestingly, the indirect
imaginary trajectories are mirror images of the direct ones. This is due to
the fact that while direct and indirect electrons enter the barrier with the
same imaginary angle, they are submitted to opposite forces because of a sign
change of the imaginary component of the laser field, which results in
opposite imaginary torques.

Figure 4: Electron trajectories in a counter-rotating bicircular field. (a)
Final momentum distribution of the electron wavepacket produced by the first
(continuous lines) and second (dashed lines) maxima of the laser field. (b)
Real part of the electron trajectories producing different ATI peaks (from the
first in blue to the fifth in orange), with a final momentum corresponding to
the crosses in (a). The continuous lines are trajectories of the first
wavepacket and are emitted on the trailing edge of the laser field
oscillation. The dashed trajectories belong to the second wavepacket and are
emitted on the leading edge of the field. The arrows represent the electric
field at time $t_{i}^{\prime}$ when the electron exits the tunnel. The black
circle has a 6.5 a.u. radius and represents the approximate extent of the
tunnel for the selected trajectories. (c) Imaginary part of the electron
trajectories. The imaginary motion is confined to the sub-barrier region.

###  IV.2 Characteristics of the complex electron motion

Figure 5: SFA calculations of photoionization of a target with 7.2 eV
ionization potential by a circularly polarized 800 nm field at $3\times
10^{13}$ W.cm-2 (first row), a co-rotating bicircular field with
$I_{\omega}=3\times 10^{13}$ W.cm-2 and $r=0.1$ (second row), and a counter-
rotating field with the same parameters (third row). The first column
corresponds to the photoelectron angular distribution in the polarization
plane. The second and third columns represent the components of the exit
tunnel velocities, parallel and perpendicular to the laser field,
respectively. The fourth column depicts the electron angular momentum along
the direction of light propagation as the electron enters the continuum. The
curvatures of real and imaginary electron trajectories under the barrier are
displayed in the fifth and sixth columns, respectively, while the last column
shows the imaginary part of the entrance angle of the electron into the
barrier.

In order to generalize the observations made on these few trajectories, we
calculated the key parameters of the electron motion for photoionization by
single color circularly polarized fundamental field, as well as co- and
counter-rotating bichromatic fields. In the latter case we restricted the
calculation to a single wavepacket emitted by one third of the whole laser
cycle, to enhance visibility. The first column of Fig. 5 (a,h,o) shows the
photoelectron angular distributions in the polarization plane, and the next
columns show the different parameters of the tunneling process that are
discussed below. Note that a mask is applied to represent the data only in the
areas where the photoelectron yield in the polarization plane is above 10$\%$
of its maximum value.

The first striking result that appears in Fig. 5(a,h,o) is that the electron
distribution obtained in circular and co-rotating fields extends to higher
momenta (i.e. is made of higher ATI peaks) than the one obtained in counter-
rotating field. This is the result of the large velocity at the exit of the
tunnel. This exit velocity is calculated by
$\mathbf{v}(t_{i}^{\prime})=\mathbf{p_{f}}+\mathbf{A}(t_{i}^{\prime})$, and
decomposed in components parallel $v_{\parallel}(t_{i}^{\prime})$ (Fig.
5(b,i,p)) and perpendicular $v_{\perp}(t_{i}^{\prime})$ (Fig. 5(d,k,r)) to the
direction of the electric field $\mathbf{E}(t^{\prime}_{i})$. In the case of
circular polarization (Fig. 5(b,c)), the exit velocity only consists of a
perpendicular component which results from an outward centrifugal effect
captured by uniform adiabatic asymptotics Ohmi  _et al._ (2015) and described
by a Coriolis term in the rotating field frame Dubois  _et al._ (2024). When a
bicircular field is used (second and third rows of Fig. 5), a parallel
velocity component appears at the exit of the tunnel. This velocity originates
from the non-adiabaticity of the tunneling process due to the temporal
variation of the magnitude of the electric field Barth and Smirnova (2011);
Ohmi  _et al._ (2015); Han  _et al._ (2017); Liu  _et al._ (2019). This has
been observed in high-harmonic interferometry experiments, in which the
velocity of the electrons emerging from the tunnel after the peak of the laser
field has been measured Pedatzur  _et al._ (2015). Our results show that the
parallel exit velocity is negative for electrons ejected in an increasing
laser field (direct electrons), and positive for electrons ejected in a
decreasing laser field (indirect electrons). The indirect electrons thus start
their motion in the continuum with a velocity opposed to the force acting on
them. The parallel exit velocity reaches larger values in counter-rotating
fields (up to 0.35 a.u.) compared to co-rotating fields (up to 0.1 a.u.),
which is the sign of a prominent role of non-adiabatic effects in counter-
rotating field configuration. By contrast, the perpendicular exit velocity is
smaller in counter-rotating fields, as observed experimentally Eckart  _et
al._ (2018), and changes sign across the classical
$\mathbf{p_{f}}=-{\mathbf{A}}(t_{i}^{\prime})$ curve.

The perpendicular exit velocity is related to the angular momentum of the
electron as it emerges in the continuum,
${\mathbf{L}}(t_{i}^{\prime})={\mathbf{r}}(t_{i}^{\prime})\times{\mathbf{v}}(t_{i}^{\prime})$.
The component of ${\mathbf{L}}(t_{i}^{\prime})$ along $z$,
$L_{z}(t_{i}^{\prime})$, is shown in the fourth column of Fig. 5.
$L_{z}(t_{i}^{\prime})$, like $v_{\perp}(t_{i}^{\prime})$, changes sign across
the classical $\mathbf{p_{f}}=-{\mathbf{A}}(t_{i}^{\prime})$ curve but is the
same for direct and indirect electrons in counter-rotating field
configuration. Therefore, this quantity does not enable to discriminate
between electron trajectories at the exit of the tunnel in spite of the
opposite handednesses of direct and indirect trajectories observed in Fig. 4.
We thus directly quantify these handednesses by calculating the normalized
curvatures of the real and imaginary trajectories:
$C_{\Re}=\int_{t_{i}^{\prime}+it_{i}^{\prime\prime}}^{t_{i}^{\prime}}\left[\frac{\Re(\mathbf{r})}{\left|\Re(\mathbf{r})\right|}\times\frac{d\Re(\mathbf{r})}{dt}dt\right]$
and
$C_{\Im}=\int_{t_{i}^{\prime}+it_{i}^{\prime\prime}}^{t_{i}^{\prime}}\left[\frac{\Im(\mathbf{r})}{\left|\Im(\mathbf{r})\right|}\times\frac{d\Im(\mathbf{r})}{dt}dt\right]$,
respectively. The results are shown in the fifth and sixth columns of Fig. 5.
The curvature is more important in the imaginary plane, and switches sign
between electrons ejected before and after the field maxima in bicircular
field configurations. This is consistent with the trajectories displayed in
Fig. 4. An additional sign switch shows up in $C_{\Im}$ across the
$\mathbf{p_{f}}=-{\mathbf{A}}(t_{i}^{\prime})$ line in the case of counter-
rotating fields, driven by the similar behavior of $v_{\perp}(t_{i}^{\prime})$
shown in Fig. 5. Such a handedness switch in the imaginary electron
trajectories produced by counter-rotating bicircular fields was previously
observed by Ayuso and Smirnova Ayuso  _et al._ (2017), who focused on
recolliding (i.e. indirect) electron trajectories. These authors related this
switch to a change in the initial conditions of the electron motion – a sign
switch of the imaginary part of the electron entrance angle in the barrier
$\Im(\Phi(t_{i}))=\Im(\text{atan}(v_{x}(t_{i})/v_{y}(t_{i})))$. We thus plot
this imaginary angle in the last column of Fig. 5. The angle shows a sign
change between low and high final momenta, matching the one observed in the
handedness of the imaginary trajectories. However, $\Im(\Phi)$ has the same
sign for direct and indirect electrons, whereas $C_{\Im}$ reverses.

###  IV.3 Modeling chiral photoionization

The detailed analysis of the SFA dynamics sheds light on the possible sources
of chirosensitivity of the tunneling process. In a previous work Bloch  _et
al._ (2021), we introduced a perturbed SFA model with a forward/backward
asymmetric phase and amplitude modulation of the electrons emerging from the
tunnel. Here we take a step further and investigate the dynamical origin of
the process. Within a simple mechanistic picture of photoelectron circular
dichroism, confirmed by classical calculations Beaulieu  _et al._ (2016), the
chiral potential converts the electron rotation motion in the polarization
plane into translation along the propagation axis, in a similar manner as the
thread of a nut converts the rotation of a bolt into translation Powis  _et
al._ (2008). Drawing from this, we can expect a chirosensitive momentum
transfer $\delta p_{z}$ resulting from the rotation dynamics under the
barrier. In this section, we estimate the effect of such a momentum transfer
in co- and counter-rotating attoclock measurements. We introduce a momentum
offset of the ionized electrons as $\delta p_{z}=\delta
p_{z}^{0}.\mathcal{M}$, where $\mathcal{M}$ is the source of chirosensitive
momentum transfer. Among the sources explored in Fig. 5, we focus on the real
ones, i.e. $L_{z}(t_{i}^{\prime})$ and $C_{\Re}(t_{i}^{\prime})$, since the
imaginary ones, $\Im(\Phi(t_{i}))$ and $C_{\Im}(t_{i}^{\prime})$, are rather
linked to tunnel ionization probabilities as the standard imaginary part of
the electron phase Lewenstein  _et al._ (1994a); Ayuso  _et al._ (2017).
$\delta p_{z}^{0}$ is a proportionality factor such that each component of the
3D momentum distribution is shifted along $p_{z}$ according to the $\delta
p_{z}^{0}$-scaled magnitude of $\mathcal{M}$ which depends on the electron
trajectory. In counter-rotating fields, the three wavepackets emitted by the
three thirds of the laser period are shifted independently. The wavepacket
momentum distributions are interpolated on the same momentum grid, and the
total distribution is obtained by coherently summing the interpolated
contributions.

Figure 6: SFA-based simulations in a co-rotating bicircular field with
$I_{\omega}=3\times 10^{13}$ W.cm-2 and $r=0.1$. Chiral attoclock shift (a-d)
and forward/backward asymmetry in the electron yield (e-h) using a
chirosensitive momentum shift due to the real trajectory curvature under the
barrier, $\mathcal{M}=C_{\Re}$ (red), or to the angular momentum at the exit
of the tunnel, $\mathcal{M}=L_{z}(t_{i}^{\prime})$ (blue). The black dashed
line is the result of the TDSE calculation in screened chiral molecules.

Figure 6 shows the results of the simulations obtained in a co-rotating field,
in which no interference effect is at play. The main conclusions from the TDSE
calculations, reproduced as dashed lines in Fig. 6, were that the chiral
attoclock shift almost vanishes for high enough energy electrons, i.e. when
the potential screening is efficient, but that a forward/backward asymmetry in
the electron yield persist, and even increases with electron energy. Let us
examine the ability of the SFA-based calculations to reproduce these trends.
If a chirosensitive momentum transfer is induced by the curvature of the sub-
barrier trajectories, i.e. $\mathcal{M}=C_{\Re}$, then the opposite curvatures
of direct and indirect electrons produce opposite momentum shifts. This
induces a forward/backward asymmetry in the attoclock angle, visible in Fig.
6. For these calculations, the value of $\delta p_{z}^{0}$ is such that the
maximum momentum transfer experienced by the electrons is $\delta
p_{z}^{max}\approx 10^{-3}$ a.u.. The chiral attoclock shift reaches
$-4^{\circ}$ for electrons of the first ATI peak ejected around $-50^{\circ}$,
and decreases with increasing electron kinetic energy. This overall trend is
in agreement with the results of the TDSE calculations in the chiral molecules
with a screened long-range potential, shown as dashed line in Fig. 6(a-d).
However, the opposite curvatures of direct and indirect electrons result in a
compensation effect in the forward/backward asymmetry in the number of ejected
electrons: the FBA is zero for all ATI peaks (Fig. 6 (e-h)). By contrast,
assuming that the chirosensitive momentum transfer is proportional to the
electron angular momentum at the exit of the tunnel, i.e.
$\mathcal{M}=L_{z}(t_{i}^{\prime})$, produces the same modulation for direct
and indirect electrons, and thus no attoclock shift (Fig. 6 (a-d)) but a
forward/backward asymmetry in the electron yield (Fig. 6 (e-h)). The FBA
reaches $-4\%$ at $-50^{\circ}$ ejection angle for the first ATI peak, and
switches sign for higher ATI peaks. For ATI4, it maximizes at $-28^{\circ}$
with value of $2\%$. For the two highest ATI peaks, the results are in
qualitative agreement with the results of the TDSE calculations in screened
chiral molecules. Assuming that a chirosensitive momentum shift is imprinted
because of the sub-barrier electron rotation motion, proportional to the
angular momentum at the exit of the tunnel, thus seems to provide a reasonable
description of chiral tunneling. The situation would be qualitatively similar
if we assumed that the imaginary entrance angle in the tunnel was the source
of chirosensitivity, as proposed by Ordonez and Smirnova (2018) and recently
used by Hofmann et al. to describe chiral photoionization in bicircular fields
Hofmann  _et al._ (2024).

Figure 7: SFA-based simulations in counter-rotating bicircular fields. Chiral
attoclock shift (a-d) and forward/backward asymmetry in the electron yield
(e-h) using a chirosensitive momentum shift $\mathcal{M}=C_{\Re}$ (red) or
$\mathcal{M}=L_{z}(t_{i}^{\prime})$ (blue). The black dashed line is the
result of the TDSE calculation in chiral molecules with a screened long-range
potential.

We repeated the calculations using counter-rotating fields and keeping the
same values of $\delta p_{z}^{0}$. The results, shown in Fig. 7, present a
striking feature: while in the co-rotating case, the modulation produced
either a chiral attoclock shift but no FBA ($\mathcal{M}=C_{\Re}$ hypothesis),
or a FBA but no chiral attoclock shift ($\mathcal{M}=L_{z}(t_{i}^{\prime})$
hypothesis), we observe a chiral attoclock shift and a forward/backward
asymmetry in the photoelectron yield in both cases. The signals produced by
the two hypothesis are indeed qualitatively similar. The attoclock shift shows
a quasi-linear slope at small electron ejection angles. For ATI 2, the slope
has the same sign as the one obtained in the TDSE calculations, but the signs
are opposite for higher ATI peaks. Jumps are observed at well-defined values
of $\theta$, where $\Delta\varphi$ exceeds 10∘. The jumps are sharper than the
ones observed in the TDSE calculations, and two jumps are present for ATI 2-4
in the SFA-based model, while a single, broader one is observed in TDSE. We
believe that this discrepancy is due to orientation-averaging effects. Our
SFA-based model assumes that the influence of the chirality in tunneling
results in a momentum transfer $\delta p_{z}^{0}$. In practice it is likely
that different molecular orientations produce different values of $\delta
p_{z}^{0}$, in a similar manner as the chiral response in photoionization
depends on molecular orientation Tia  _et al._ (2017); Fehre  _et al._
(2019b), which leads to a broadening of the response. Regarding the FBA, the
order of magnitude of the SFA-based calculation matches the values observed in
TDSE, but the signs and detailed evolutions are not well reproduced. In a
previous study, we have shown that the phase of the wavepackets emerging from
the tunnel was influenced by the chirality of the molecular potential Bloch
_et al._ (2021). This forward/backward phase asymmetry of the wavepackets is
not taken into account in the present model, and is likely to explain the
discrepancy between the SFA-based calculations and the TDSE in counter-
rotating fields, where interference effects play an essential role.

The difference between Fig. 6 and Fig. 7 confirms the importance of
interference effects in strong field ionization by counter-rotating bicircular
fields, and raises an interesting question: how can interference effects
induce a rotation of the photoelectron momentum distribution, i.e. an
attoclock shift? As we mentioned before, a phase shift between direct and
indirect electrons emerging from the tunnel is expected to change the position
of the interference phase jump, but cannot rotate the distribution away from
$\varphi_{0}=90^{\circ}$ or $150^{\circ}$ within the SFA. This means that an
additional process is at play in chiral tunneling. To understand the origin of
the chiral interferometric attoclock shift, we examine in Fig. 8 the phase of
the photoelectron wavepacket produced by the counter-rotating bicircular
field. Fig. 8(b) shows a cut of the wavepacket phase in the $p_{z}=0.12$ a.u.
plane. The phase presents a clear vortex structure, reflecting the temporal
evolution of the action accumulated by the electrons in the continuum. For a
given ATI peak, i.e. a given radius $p_{x}^{2}+p_{y}^{2}$, the angular phase
evolution is characteristic of a spiral wavefront, carrying orbital angular
momentum Eickhoff  _et al._ (2020); Maxwell  _et al._ (2020). Comparing the
phase of the wavepacket obtained in two planes, $p_{z}=0.12$ (Fig. 8(b)) and
$p_{z}=0.18$ a.u. (Fig. 8(f)), reveals a high sensitivity of the vortex
structure to the electron momentum, which can be visualized by plotting the
phase difference between the two $p_{z}$ planes (Fig. 8(d)). In a pure SFA
calculation, this phase difference has no effect, and the vorticity of the
electron wavepacket is invisible in the electron momentum distribution, which
is the modulus square of the wavepacket. However, if the chirality of the
potential induces a $\delta p_{z}$ shift of the electrons emerging from the
tunnel, then electrons corresponding to different $p_{z}$ in the pure SFA case
end up at the same $p_{z}$. In that case, their different phases can lead to a
rotation of the interference pattern. To quantify this effect, we plot in Fig.
8(e) the phase difference between the electron wavepacket calculated assuming
$\mathcal{M}=L_{z}(t_{i}^{\prime})$ and a pure SFA wavepacket. The phase
difference is about 20 mrad, and shows a twisted angular structure. Thus, even
if the chirosensitive modulation $\mathcal{M}=L_{z}(t_{i}^{\prime})$ is the
same for electrons born on the leading and trailing edge of the field, and
cannot produce an attoclock shift of each electron wavepacket, the phase twist
resulting from the electron angular momentum in the continuum induces a
rotation of the interference pattern, which appears as a chiral attoclock
shift. Similarly, in the $\mathcal{M}=C_{\Re}$ hypothesis, the opposite
modulations of direct and indirect electrons does not result in a FBA in the
yield of each wavepacket, but produces one in the interference pattern because
of the phase vortex.

Figure 8: Intensity (a) and phase (b) of the photoelectron wavepacket ejected
by a counter-rotating bicircular field calculated by SFA, in the $p_{z}=0.12$
a.u. plane. (c) Phase of the wavepacket in the $p_{z}=0.18$ a.u. plane. (d)
Phase difference between these two planes. (e) Phase of the wavepacket (in
rad) obtained by modulating the SFA signal by a chirosensitive momentum
transfer due to the electron angular momentum at the exit of the tunnel
[$\mathcal{M}=L_{z}(t_{i}^{\prime})$], in the $p_{z}=0.18$ a.u. plane. (f).
Phase difference between the modulated and unmodulated wavepacket in the
$p_{z}=0.18$ a.u. plane. A mask is applied to the phase maps to hide the
signal in areas where the photoelectron yield is below $5\%$ of its maximum.

##  V Conclusion and perspectives

We have shown that interference effects between electrons ejected on the
leading and trailing edges of the field oscillations play a crucial role in
photoionization by counter-rotating bicircular fields. In the strong-field
ionization of chiral molecules, this interference has enabled us to increase
the sensitivity of the attoclock scheme and to identify characteristic
chirosenstive jumps along the electron ejection direction. The TDSE
calculations in chiral molecules with a screened long-range potential have
shown that this feature can be produced by short-range effects, i.e. be a
consequence of non-adiabatic tunnel-ionization. We thus used SFA calculations
to investigate the possible sources of chirosensitivity of the tunneling
process. This study revealed the importance of non-adiabatic effects in the
ionization by bicircular fields, induced by the field rotation as well as by
its amplitude modulations. While some parameters of the complex sub-barrier
electron motion, such as the electron exit angular momentum, are identical for
direct and indirect electrons, others, such as the electron trajectory
curvature, were found to switch sign. We tested the impact of these two
categories of parameters on the chirosensitive response, and found that
interference effects are determinant in shaping the attoclock angle and
forward/backward asymmetry, because of the vorticity of the electron wavefront
imposed by the rotating laser field.

The interferometric chiral attoclock scheme thus enables to reveal the
influence of the electron phase vortex in the strong field ionization. The
twist, vorticity and angular orbital momentum of photoelectrons has been the
subject of several recent studies. Ngoko Djiokap et al. have observed
interference vortices in calculations of the photoionization by two delayed
counter-rotating circularly polarized attosecond pulses Ngoko Djiokap  _et
al._ (2015). Such vortices were measured by Pengel et al. in the multiphoton
regime, using two delayed counter-rotating laser fields Pengel  _et al._
(2017). This configuration has been extensively studied theoretically in the
strong field regime by Maxwell et al. Maxwell  _et al._ (2020), using the SFA
as well as more sophisticated models. In chiral molecules, Planas et al. have
calculated that strong field ionization by linearly polarized light produced
enantio-sensitive electron vortices Planas  _et al._ (2022). These vortices
are predicted to produce modifications of the electron-ion differential cross
section Tolstikhin and Morishita (2019), opening new perspectives for the
recently demonstrated chiral laser-induced electron diffraction Rajak  _et
al._ (2024).

Another interesting aspect of the interferometric nature of photoionization in
counter-rotating bicircular fields is its enhanced sensitivity to chirality.
This sensitivity can be further increased by adjusting the intensity ratio $r$
between second harmonic and fundamental components. In the results presented
above, this ratio was rather low (0.1), such that the dynamics of the two
families of electron trajectories in the continuum was very similar. When this
ratio is increased, the indirect electrons revisit the ionic core region after
being released, ending up with an enhanced sensitivity to the influence of the
ionic potential. This effect is further increased by the initial momentum of
the electron at the exit of the tunnel Kaushal and Smirnova (2018a); Geyer
_et al._ (2023). In that situation, we can expect an increased structural
sensitivity of the interference between direct and indirect electrons. Eckart
et al. have established that this interference could be studied by monitoring
the electron momentum distribution along the light propagation direction.
Figure 9(a) shows measurements of the projection of the low-energy part of the
distribution onto the polarization plane, for photoionization of camphor
molecules by a counter-rotating bicircular field at $I=8\times 10^{12}$
W$\cdot$cm-2 and $r=1$. When the longitudinal momentum $(p_{x},p_{y})$ is
large, the transverse $p_{z}$-distribution displayed in Fig. 9(b) has a
symmetric Gaussian shape. The FBA is close to zero. The electrons ejected in
this range are thus insensitive to the chirality of the molecular potential.
In the region of intermediate longitudinal momentum, exemplified by
$(p_{x},p_{y})=(0.11,0.03)$ a.u. in Fig. 9(c), modulations of the transverse
distribution appear, reflecting the interference between direct and indirect
electrons. These modulations are forward/backward asymmetric, and thus
sensitive to chirality. The FBA reaches $7\%$, and reverses perfectly when
switching the enantiomer to (-)-camphor. At very low longitudinal momentum
$(p_{x},p_{y})\approx(0,0)$, the transverse distribution is sharply peaked
about $p_{z}=0$, indicating a strong Coulomb focusing (Fig. 9(d)). Coulomb
focusing was theoretically shown to increase the sensitivity of
photoionization to molecular chirality Rozen  _et al._ (2021). This is
confirmed by these measurements, which show that the momentum distribution is
highly asymmetric along $p_{z}$, with a FBA oscillating as $p_{z}$ varies and
reaching 15$\%$. These oscillations are the signature of the interference
between direct and indirect electrons. Increasing the laser intensity to
$I=1.4\times 10^{13}$ W$\cdot$cm-2 with $r=0.4$ leads to an increase of the
contribution of the rescattering electrons, which clearly appear on the
transverse momentum distribution (Fig. 9(e)), with a FBA reaching 17$\%$.
Last, to illustrate the structural sensitivity of the scheme, we repeated the
measurements in fenchone. The asymmetric interference pattern is even more
contrasted. The interference between direct and delayed electrons is almost
fully destructive in the backward direction, but not in the forward one. This
leads to a very high value of FBA, reaching 120$\%$ – the maximum possible FBA
value being 200$\%$ with the definition employed.

Last, it would be interesting to investigate the importance of the sub-barrier
dynamical parameters, such as the trajectory curvature, on spin-polarization
effects Barth and Smirnova (2013); Hartung  _et al._ (2016), in the line of
the pioneering work carried out by Kaushal and Smirnova Kaushal and Smirnova
(2018a, b, c) and Ayuso et al.Ayuso  _et al._ (2017). Measuring spin-resolved
quantum interference patterns arising in counter-rotating fields could provide
an interesting approach to probe dynamical spin effects in strong-field
ionization.

Figure 9: Experimental observation of chirosensitive transverse interference
in (+)-camphor and (+)-fenchone, ionized with counter-rotating bicircular
fields with right circular polarization of the fundamental. (a) Projection of
the photoelectron angular distribution in the laser polarization plane, in
(+)-camphor at $I=8\times 10^{12}$ W$\cdot$cm-2 with
$r=I_{2\omega}/I_{\omega}=1$. (b-d) Forward/Backward asymmetries in
(+)-camphor (continuous lines) and (-)-camphor (dashed lines) and transverse
momentum distributions (shaded areas) at $(p_{x},p_{y})=(0.02,0)$,
$(0.11,0.03)$, and $(0.12,0.21)$ a.u., which correspond to the three spots
marked in panel (a). (e) Same as (d), but at $I=1.4\times 10^{13}$
W$\cdot$cm-2 with $r=0.4$. (f) Same as (e), but in (+)-fenchone.

We thank Franck Blais, Rodrigue Bouillaud and Laurent Merzeau for technical
assistance. We acknowledge David Ayuso, Valérie Blanchet and Olga Smirnova for
fruitful discussions. This project has received funding from the Agence
Nationale de la Recherche (ANR) – Shotime (ANR-21-CE30-038-01), and from the
European Research Council (ERC) under the European Union’s Horizon 2020
research and innovation programme no. 682978 - EXCITERS. Funded by the
European Union (ERC Starting Grant UTOPIQ ERC-2022-STG No.101076639). Views
and opinions expressed are however those of the authors only and do not
necessarily reflect those of the European Union. Neither the European Union
nor the granting authority can be held responsible for them.

## References

  * Ramos  _et al._ (2020) R. Ramos, D. Spierings, I. Racicot, and A. M. Steinberg, Nature 583, 529 (2020).
  * Kaushal and Smirnova (2018a) J. Kaushal and O. Smirnova, Journal of Physics B: Atomic, Molecular and Optical Physics 51, 174002 (2018a).
  * Hofmann  _et al._ (2019) C. Hofmann, A. S. Landsman, and U. Keller, Journal of Modern Optics 66, 1052 (2019).
  * Kheifets (2020) A. S. Kheifets, Journal of Physics B: Atomic, Molecular and Optical Physics 53, 072001 (2020).
  * Lewenstein  _et al._ (1994a) M. Lewenstein, Ph. Balcou, M. Yu. Ivanov, A. L’Huillier, and P. B. Corkum, Physical Review A 49, 2117 (1994a).
  * Ivanov  _et al._ (2005) M. Y. Ivanov, M. Spanner, and O. Smirnova, Journal of Modern Optics 52, 165 (2005).
  * Bloch  _et al._ (2021) E. Bloch, S. Larroque, S. Rozen, S. Beaulieu, A. Comby, S. Beauvarlet, D. Descamps, B. Fabre, S. Petit, R. Taïeb, A. J. Uzan, V. Blanchet, N. Dudovich, B. Pons, and Y. Mairesse, Physical Review X 11, 041056 (2021).
  * Ritchie (1975) B. Ritchie, Physical Review A 12, 567 (1975).
  * Powis (2000) I. Powis, The Journal of Chemical Physics 112, 301 (2000).
  * Böwering  _et al._ (2001) N. Böwering, T. Lischke, B. Schmidtke, N. Müller, T. Khalil, and U. Heinzmann, Physical Review Letters 86, 1187 (2001).
  * Nahon  _et al._ (2015) L. Nahon, G. A. Garcia, and I. Powis, Journal of Electron Spectroscopy and Related Phenomena Gas Phase Spectroscopic and Dynamical Studies at Free-Electron Lasers and Other Short Wavelength Sources, 204, Part B, 322 (2015).
  * Lux  _et al._ (2012) C. Lux, M. Wollenhaupt, T. Bolze, Q. Liang, J. Köhler, C. Sarpe, and T. Baumert, Angewandte Chemie International Edition 51, 5001 (2012).
  * Lehmann  _et al._ (2013) C. S. Lehmann, N. B. Ram, I. Powis, and M. H. M. Janssen, The Journal of Chemical Physics 139, 234307 (2013).
  * Beaulieu  _et al._ (2016) S. Beaulieu, A. Ferré, R. Géneaux, R. Canonge, D. Descamps, B. Fabre, N. Fedorov, F. Légaré, S. Petit, T. Ruchon, V Blanchet, Y. Mairesse, and B. Pons, New Journal of Physics 18, 102002 (2016).
  * Müller  _et al._ (2020) A. D. Müller, E. Kutscher, A. N. Artemyev, and P. V. Demekhin, The Journal of Chemical Physics 152, 044302 (2020), arxiv:2001.05029 .
  * Eckle  _et al._ (2008) P. Eckle, M. Smolarski, P. Schlup, J. Biegert, A. Staudte, M. Schöffler, H. G. Muller, R. Dörner, and U. Keller, Nature Physics 4, 565 (2008).
  * Eicke and Lein (2019) N. Eicke and M. Lein, Physical Review A 99, 031402 (2019).
  * Torlina  _et al._ (2015) L. Torlina, F. Morales, J. Kaushal, I. Ivanov, A. Kheifets, A. Zielinski, A. Scrinzi, H. G. Muller, S. Sukiasyan, M. Ivanov, and O. Smirnova, Nature Physics 11, 503 (2015).
  * Sainadh  _et al._ (2019) U. S. Sainadh, H. Xu, X. Wang, A. Atia-Tul-Noor, W. C. Wallace, N. Douguet, A. Bray, I. Ivanov, K. Bartschat, A. Kheifets, R. T. Sang, and I. V. Litvinyuk, Nature 568, 75 (2019).
  * Fehre  _et al._ (2019a) K. Fehre, S. Eckart, M. Kunitski, C. Janke, D. Trabert, J. Rist, M. Weller, A. Hartung, M. Pitzer, L. Ph. H. Schmidt, T. Jahnke, R. Dörner, and M. S. Schöffler, Physical Review Research 1, 033045 (2019a).
  * de Bohan  _et al._ (2002) A. de Bohan, B. Piraux, L. Ponce, R. Taïeb, V. Véniard, and A. Maquet, Physical Review Letters 89, 113002 (2002).
  * Agostini  _et al._ (1979) P. Agostini, F. Fabre, G. Mainfray, G. Petite, and N. K. Rahman, Phys. Rev. Lett. 42, 1127 (1979).
  * Fedorov  _et al._ (2020) N. Fedorov, S. Beaulieu, A. Belsky, V. Blanchet, R. Bouillaud, M. De Anda Villa, A. Filippov, C. Fourment, J. Gaudin, R. E. Grisenti, E. Lamour, A. Lévy, S. Macé, Y. Mairesse, P. Martin, P. Martinez, P. Noé, I. Papagiannouli, M. Patanen, S. Petit, D. Vernhet, K. Veyrinas, and D. Descamps, Review of Scientific Instruments 91, 105104 (2020).
  * Wollenhaupt  _et al._ (2009) M. Wollenhaupt, M. Krug, J. Köhler, T. Bayer, C. Sarpe-Tudoran, and T. Baumert, Applied Physics B 95, 647 (2009).
  * Mancuso  _et al._ (2015) C. A. Mancuso, D. D. Hickstein, P. Grychtol, R. Knut, O. Kfir, X.-M. Tong, F. Dollar, D. Zusin, M. Gopalakrishnan, C. Gentry, E. Turgut, J. L. Ellis, M.-C. Chen, A. Fleischer, O. Cohen, H. C. Kapteyn, and M. M. Murnane, Physical Review A 91, 031402 (2015).
  * Lewenstein  _et al._ (1994b) M. Lewenstein, Ph. Balcou, M. Yu. Ivanov, A. L’Huillier, and P. B. Corkum, Physical Review A 49, 2117 (1994b).
  * Smirnova and Ivanov (2014) O. Smirnova and M. Ivanov, in  _Attosecond and XUV Physics_ (John Wiley & Sons, Ltd, 2014) Chap. 7, pp. 201–256.
  * Ohmi  _et al._ (2015) M. Ohmi, O. I. Tolstikhin, and T. Morishita, Physical Review A 92, 043402 (2015).
  * Dubois  _et al._ (2024) J. Dubois, C. Lévêque, J. Caillat, R. Taïeb, U. Saalmann, and J.-M. Rost, Physical Review A 109, 013112 (2024).
  * Barth and Smirnova (2011) I. Barth and O. Smirnova, Physical Review A 84, 063415 (2011).
  * Han  _et al._ (2017) M. Han, P. Ge, Y. Shao, M.-M. Liu, Y. Deng, C. Wu, Q. Gong, and Y. Liu, Physical Review Letters 119, 073201 (2017).
  * Liu  _et al._ (2019) X. Liu, K. Amini, T. Steinle, A. Sanchez, M. Shaikh, B. Belsa, J. Steinmetzer, A.-T. Le, R. Moshammer, T. Pfeifer, J. Ullrich, R. Moszynski, C. D. Lin, S. Gräfe, and J. Biegert, The Journal of Chemical Physics 151, 024306 (2019).
  * Pedatzur  _et al._ (2015) O. Pedatzur, G. Orenstein, V. Serbinenko, H. Soifer, B. D. Bruner, A. J. Uzan, D. S. Brambila, A. G. Harvey, L. Torlina, F. Morales, O. Smirnova, and N. Dudovich, Nature Physics 11, 815 (2015).
  * Eckart  _et al._ (2018) S. Eckart, M. Kunitski, M. Richter, A. Hartung, J. Rist, F. Trinter, K. Fehre, N. Schlott, K. Henrichs, L. P. H. Schmidt, T. Jahnke, M. Schöffler, K. Liu, I. Barth, J. Kaushal, F. Morales, M. Ivanov, O. Smirnova, and R. Dörner, Nature Physics , 1 (2018).
  * Ayuso  _et al._ (2017) D. Ayuso, A. Jiménez-Galán, F. Morales, M. Ivanov, and O. Smirnova, New Journal of Physics 19, 073007 (2017).
  * Powis  _et al._ (2008) I. Powis, C. J. Harding, G. A. Garcia, and L. Nahon, ChemPhysChem 9, 475 (2008).
  * Ordonez and Smirnova (2018) A. F. Ordonez and O. Smirnova, Physical Review A 98, 063428 (2018).
  * Hofmann  _et al._ (2024) M. Hofmann, D. Trabert, A. Geyer, N. Anders, J. Kruse, J. Rist, L. P. H. Schmidt, T. Jahnke, M. Kunitski, M. S. Schöffler, S. Eckart, and R. Dörner, “Sub-cycle resolved strong field ionization of chiral molecules and the origin of chiral photoelectron asymmetries,”  (2024), arxiv:2402.03215 [physics, physics:quant-ph] .
  * Tia  _et al._ (2017) M. Tia, M. Pitzer, G. Kastirke, J. Gatzke, H.-K. Kim, F. Trinter, J. Rist, A. Hartung, D. Trabert, J. Siebert, K. Henrichs, J. Becht, S. Zeller, H. Gassert, F. Wiegandt, R. Wallauer, A. Kuhlins, C. Schober, T. Bauer, N. Wechselberger, P. Burzynski, J. Neff, M. Weller, D. Metz, M. Kircher, M. Waitz, J. B. Williams, L. P. H. Schmidt, A. D. Müller, A. Knie, A. Hans, L. Ben Ltaief, A. Ehresmann, R. Berger, H. Fukuzawa, K. Ueda, H. Schmidt-Böcking, R. Dörner, T. Jahnke, P. V. Demekhin, and M. Schöffler, The Journal of Physical Chemistry Letters 8, 2780 (2017).
  * Fehre  _et al._ (2019b) K. Fehre, S. Eckart, M. Kunitski, C. Janke, D. Trabert, J. Rist, M. Weller, A. Hartung, L. P. H. Schmidt, T. Jahnke, R. Dörner, and M. Schöffler, The Journal of Physical Chemistry A 123, 6491 (2019b).
  * Eickhoff  _et al._ (2020) K. Eickhoff, C. Rathje, D. Köhnke, S. Kerbstadt, L. Englert, T. Bayer, S. Schäfer, and M. Wollenhaupt, New Journal of Physics 22, 103045 (2020).
  * Maxwell  _et al._ (2020) A. S. Maxwell, G. S. J. Armstrong, M. F. Ciappina, E. Pisanty, Y. Kang, A. C. Brown, M. Lewenstein, and C. F. d. M. Faria, Faraday Discussions (2020), 10.1039/D0FD00105H.
  * Ngoko Djiokap  _et al._ (2015) J. M. Ngoko Djiokap, S. X. Hu, L. B. Madsen, N. L. Manakov, A. V. Meremianin, and A. F. Starace, Physical Review Letters 115, 113004 (2015).
  * Pengel  _et al._ (2017) D. Pengel, S. Kerbstadt, D. Johannmeyer, L. Englert, T. Bayer, and M. Wollenhaupt, Physical Review Letters 118, 053003 (2017).
  * Planas  _et al._ (2022) X. B. Planas, A. Ordóñez, M. Lewenstein, and A. S. Maxwell, Physical Review Letters 129, 233201 (2022).
  * Tolstikhin and Morishita (2019) O. I. Tolstikhin and T. Morishita, Physical Review A 99, 063415 (2019).
  * Rajak  _et al._ (2024) D. Rajak, S. Beauvarlet, O. Kneller, A. Comby, R. Cireasa, D. Descamps, B. Fabre, J. D. Gorfinkiel, J. Higuet, S. Petit, S. Rozen, H. Ruf, N. Thiré, V. Blanchet, N. Dudovich, B. Pons, and Y. Mairesse, Physical Review X 14, 011015 (2024).
  * Geyer  _et al._ (2023) A. Geyer, D. Trabert, M. Hofmann, N. Anders, M. S. Schöffler, L. Ph. H. Schmidt, T. Jahnke, M. Kunitski, R. Dörner, and S. Eckart, Physical Review Research 5, 033094 (2023).
  * Rozen  _et al._ (2021) S. Rozen, S. Larroque, N. Dudovich, Y. Mairesse, and B. Pons, J. Phys. B 54, 184002 (2021).
  * Barth and Smirnova (2013) I. Barth and O. Smirnova, Physical Review A 88, 013401 (2013).
  * Hartung  _et al._ (2016) A. Hartung, F. Morales, M. Kunitski, K. Henrichs, A. Laucke, M. Richter, T. Jahnke, A. Kalinin, M. Schöffler, L. P. H. Schmidt, M. Ivanov, O. Smirnova, and R. Dörner, Nature Photonics 10, 526 (2016).
  * Kaushal and Smirnova (2018b) J. Kaushal and O. Smirnova, Journal of Physics B: Atomic, Molecular and Optical Physics 51, 174003 (2018b).
  * Kaushal and Smirnova (2018c) J. Kaushal and O. Smirnova, Journal of Physics B: Atomic, Molecular and Optical Physics 51, 174001 (2018c).


# Rethinking Compression: Reduced Order Modelling of Latent Features in Large
Language Models

Arnav Chavan 1,2, Nahush Lele∗1, Deepak Gupta2
1Nyun AI 2Transmute AI Lab
<EMAIL_ADDRESS><EMAIL_ADDRESS>
Equal contribution. Work done while Nahush was an Intern at Nyun AI

###### Abstract

Due to the substantial scale of Large Language Models (LLMs), the direct
application of conventional compression methodologies proves impractical. The
computational demands associated with even minimal gradient updates present
challenges, particularly on consumer-grade hardware. This paper introduces an
innovative approach for the parametric and practical compression of LLMs based
on reduced order modelling, which entails low-rank decomposition within the
feature space and re-parameterization in the weight space. Notably, this
compression technique operates in a layer-wise manner, obviating the need for
a GPU device and enabling the compression of billion-scale models within
stringent constraints of both memory and time. Our method represents a
significant advancement in model compression by leveraging matrix
decomposition, demonstrating superior efficacy compared to the prevailing
state-of-the-art structured pruning method.

##  1 Introduction

Recent advances in generative modeling have led to a notable increase in the
construction of large language models (LLMs), some of which consist of
hundreds of billions of parameters. Despite their commendable accuracy, the
associated computational demands are considerable, particularly in terms of
GPU memory for inference. In practical applications, there is a growing need
to compress these models while minimizing the accompanying performance
degradation.

Promising approaches to compress LLMs include pruning (Frantar & Alistarh,
2023; Sun et al., 2023; Ma et al., 2023), quantization (Frantar et al., 2022;
Dettmers et al., 2023) and knowledge distillation (Wu et al., 2023; Gu et al.,
2023). Current LLM quantization methods require specific hardware-level
support and are not able to reduce MACs and speed up inference time due to
expensive quant-dequant operations in LLMs. Knowledge distillation has been
shown to perform well in a training-aware fashion on standard deep learning
models. However, the massive computational resources needed for distillation
limits the applicability of such approaches. Recently, (Ma et al., 2023)
presented a structured pruning approach designed for LLMs. While this approach
is capable of pruning the LLMs with no need for fine-tuning, the drop in
performance is significant, and clearly there is a need to explore further in
this direction. Moreover, the pruning strategy is not universal and
significant effort is needed per neural architecture to identify the prunable
structures.

In this paper, we present a novel, practical and training-free approach to
model compression which is specifically for large models including LLMs.
Referred further as LLM-ROM, our approach performs localized reduced order
modelling of the latent features through low-rank decomposition in the feature
space and re-parameterization in the weight space. Since LLM-ROM operates
layerwise, it does not require any massive model updates and can be executed
on small GPU/CPU resources. The simplicity of LLM-ROM facilitates the
compression of billion-scale models within stringent constraints of both
memory and time. Our early experiments demonstrate that LLM-ROM outperforms
existing approaches and can compress LLMs without any fine-tuning.

##  2 Method

LLM-ROM builds reduced-order model (ROM) layerwise, and for a model with $L$
layers, the decomposition of the latent feature maps is done in a sequential
manner using a calibration data $\mathbf{X}\in\mathbb{R}^{B\times d_{1}}$,
where $B$ and $d_{1}$ denote the batch-size and number of input channels,
respectively. For the $i^{th}$ layer, denoted as $L_{i}$ with weights
$\mathbf{W}_{i}\in\mathbb{R}^{d_{2}\times d_{1}}$ where $d_{2}$ denotes the
output channels, we compute the feature map
$\mathbf{Y}_{i}=\mathbf{W}_{i}\mathbf{X}_{i}\in\mathbb{R}^{B\times d_{2}}$.
Following this, the principal components of $\mathbf{Y}_{i}$ are then computed
through eigenvalue decomposition of the symmetric covariance matrix of
$Y_{i}$. These components can be represented as $V_{j}\in\mathbb{R}^{d_{2}}$
$\enskip\forall\enskip j\in[1,d_{2}]$, and the principal component matrix can
be represented as $\mathbf{V}\in\mathbb{R}^{d_{2}\times d_{2}}$, with each row
denoting a principal component arranged in the descending order of their
eigenvalue.

Depending upon the target rank of the layer, we select only the top $r$
principal components ranked by their respective eigenvalues. Thus, we index
$V_{r}=\mathbf{V}[1\rightarrow r,:]\in\mathbb{R}^{r\times d_{2}}$. Thus, the
ROM of this layer can be denoted as
$\mathbf{Y}_{i}=\mathbf{V}_{r}^{T}\mathbf{V}_{r}\mathbf{W}_{i}\mathbf{X}_{i}$.
Upon re-parameterization into low-rank matrices,
$\mathbf{W}_{i1}=\mathbf{V}_{r}^{T}\in\mathbb{R}^{d_{2}\times r}$ and
$\mathbf{W}_{i2}=\mathbf{V}_{r}\mathbf{W}_{i}\in\mathbb{R}^{r\times d_{1}}$,
the layer can be decomposed into a sequential combination of two smaller
linear layers with weights $\mathbf{W}_{i1}$ and $\mathbf{W}_{i2}$
respectively. We consider the ROM of the previous layer to generate inputs for
the next layer so that the next layers have prior information of the error
introduced in the previous layers for decomposition. Note that the ROM
operations are performed on CPU with no requirement for a GPU, and the
computational cost associated with it is very small.

###  2.1 Layerwise rank computation

The LLaMA-7B (Touvron et al., 2023) model consists of 32 identical decoder
modules (these modules comprise >96% of the total model parameters), each of
which consists of seven decomposable weight matrices. Our initial studies
showcased that setting a uniform compression budget for all the modules from
the very beginning of the model lead to a significant deterioration in model
performance; for this reason, we restrict the application of our compression
process to a subset of modules. Further, decomposition of the layers of a
module introduces errors in the outputs of that layer which get compounded as
we move forward in the network; to minimize this, we only compress modules
towards the end of the model. Based on these heuristics we perform experiments
compressing varying numbers of modules from the end depending on the budget
that we need to satisfy for the entire model. The specific number of modules
to be compressed is empirically determined for each budget. For instance, to
achieve an overall budget of 80%, we conducted experiments compressing only
the last 8 modules uniformly with a budget of 0.20, the last 12 modules with a
budget of 0.46, and the last 16 modules with a budget of 0.60. Our findings
indicated that compressing the last 12 modules yielded the most favorable
results for 80% budget. Similar experiments for 90% and 50% budget yield the
best results when we compress the last 8 modules with a budget of 0.60 and the
last 24 modules with a budget of 0.33 respectively, the results of which are
listed in Table 1. Each module is originally composed of 4 weight matrices
$\mathbf{W}_{i}\in\mathbb{R}^{4096\times 4096}$ from the self attention block
and 3 weight matrices $\mathbf{W}_{j}\in\mathbb{R}^{4096\times 11008}$ from
the feed-forward network (although one of these is transposed, it does not
change the computed rank). At the end of compression, each weight matrix in
the self-attention block is decomposed into two low-rank matrices:
$\mathbf{W}_{i1}\in\mathbb{R}^{4096\times r}$ and
$\mathbf{W}_{i2}\in\mathbb{R}^{r\times 4096}$, where $r$ takes values of 1228,
954, and 675. Additionally, the weight matrices in the feed-forward network
are decomposed into $\mathbf{W}_{j1}\in\mathbb{R}^{4096\times r}$ and
$\mathbf{W}_{j2}\in\mathbb{R}^{r\times 11008}$ with $r$ values of 1791, 1373,
and 985. These decompositions correspond to three module budget settings of
60%, 46%, and 33%, resulting in overall model budgets of 90%, 80%, and 50%,
respectively.

##  3 Experiments

###  3.1 Zero-shot performance

To evaluate the performance of the model in a task-agnostic setting, we employ
LLaMA’s (Touvron et al., 2023) assessment methodology, performing zero shot
task classification across common sense reasoning datasets, including BoolQ
(Clark et al., 2019), PIQA (Bisk et al., 2020), HellaSwag (Zellers et al.,
2019), WinoGrande (Sakaguchi et al., 2021), ARC-easy (Clark et al., 2018), and
ARC-challenge (Clark et al., 2018). We use a batch-size of 512 for the
calibration data (Section 3.3) from the training splits of the aforementioned
datasets, ensuring no data leakage, and set a maximum sequence length of 128.
We set target compression rates of 80%, and 50% and compare with LLM-
Pruner111We pick the best performing pruning method from LLM-Pruner _i.e._
Block Pruning. with and without fine-tuning in Table 1.

Our LLM-ROM method consistently outperforms LLM-Pruner at 80% and 50%
compression without any fine-tuning. It is noteworthy that at 80% budget our
method even outperforms fine-tuned LLM-Pruner model, signifying that ROM is
able to better extract smaller neural structures and weights from larger
counterparts without any gradient updates on the extracted weights.

Method | Finetune | #Params | #MACs | BoolQ | PIQA | HellaSwag | WinoGrande | ARC-e | ARC-c | Average
---|---|---|---|---|---|---|---|---|---|---
LLaMA-7B | - | 6.7B | 423.93G | 76.5 | 79.8 | 76.1 | 70.1 | 72.8 | 47.6 | 70.5
LLM-Pruner | ✗ | 5.4B | 339.60G | 59.4 | 75.6 | 65.3 | 61.3 | 59.2 | 37.1 | 59.7
LLM-Pruner | ✓ | 5.4B | 339.60G | 69.5 | 76.4 | 68.1 | 65.1 | 63.4 | 37.9 | 63.4
LLM-ROM | ✗ | 5.4B | 339.99G | 74.5 | 73.8 | 66.6 | 68.1 | 67.2 | 39.8 | 65.0
LLM-Pruner | ✗ | 3.4B | 206.59G | 52.3 | 59.6 | 35.6 | 53.2 | 33.5 | 27.2 | 43.6
LLM-Pruner | ✓ | 3.4B | 206.59G | 60.3 | 69.3 | 47.1 | 53.4 | 46.0 | 29.2 | 50.9
LLM-ROM | ✗ | 3.5B | 215.61G | 62.0 | 62.5 | 35.3 | 57.7 | 39.3 | 27.6 | 47.4

Table 1: Comprehensive comparison of our method with LLM-Pruner on LLaMA-7B
model.

###  3.2 Effect of Batch Size and Sequence Length

The eigenvalue decomposition of the covariance matrix and the subsequent
selection of the principal components require a computation of the outputs of
that layer. The batch used to compute this output is a key factor that can
influence the generalizability of the layers obtained after decomposition.
Principal components computed on a larger sample size will exhibit closer
alignment with those of the true distribution. To corroborate this hypothesis,
we conducted experiments along two orthogonal directions: one by solely
varying the batch-size, and the other with the variation of sequence length,
the results for the same are presented in Table 2 and 3 respectively.

Batch Size | BoolQ | PIQA | HellaSwag | WinoGrande | ARC-e | ARC-c | Average
---|---|---|---|---|---|---|---
512 | 74.5 | 73.8 | 66.6 | 68.1 | 67.2 | 39.8 | 65.0
128 | 72.6 | 72.4 | 63.2 | 66.3 | 61.1 | 37.7 | 62.2
32 | 70.2 | 68.4 | 58.7 | 67.2 | 55.7 | 35.7 | 59.3

Table 2: Effect of batch size on model performance at a sequence length 128.

Seq. Length | BoolQ | PIQA | HellaSwag | WinoGrande | ARC-e | ARC-c | Average
---|---|---|---|---|---|---|---
128 | 74.5 | 73.8 | 66.6 | 68.1 | 67.2 | 39.8 | 65.0
64 | 66.6 | 74.4 | 65.7 | 67.6 | 65.4 | 40.1 | 63.3
32 | 66.2 | 73.4 | 65.1 | 67.8 | 64.6 | 39.5 | 62.7

Table 3: Effect of sequence length on model performance at batch size 512.

From Tables 2 and 3 it is evident that a larger batch is beneficial and
results in significantly better model generalization and at the same time
longer sequence length also aids in maintaining model performance post
compression.

###  3.3 Choice of calibration dataset

Given that the activations of data from the calibration dataset are used to
calculate the covariance matrix, which is subsequently utilized for
eigendecomposition, it is reasonable to infer that the model’s performance is
sensitive to the choice of this dataset. This inference is supported by our
findings in the conducted studies where we use three different datasets,
namely ARC-challenge (Clark et al., 2018), BookCorpus(Zhu et al., 2015) and a
combination of all the common sense task prompts i.e each batch contains an
equal number of samples from the six common sense reasoning tasks’ datasets
used for benchmarking namely BoolQ (Clark et al., 2019), PIQA (Bisk et al.,
2020), HellaSwag (Zellers et al., 2019), WinoGrande (Sakaguchi et al., 2021),
ARC-easy (Clark et al., 2018), and ARC-challenge (Clark et al., 2018) , as our
calibration datasets at a budget of $80\%$ keeping other hyperparameters such
as batch-size and sequence length constant. When creating the calibration
datasets we choose samples from a data split which is disjoint from the set
upon which evaluation is performed, ensuring there is no data leak. The
results of these studies are compiled in Table 4.

Dataset | BoolQ | PIQA | HellaSwag | WinoGrande | ARC-e | ARC-c | Average
---|---|---|---|---|---|---|---
Combination | 74.5 | 73.8 | 66.6 | 68.1 | 67.2 | 39.8 | 65.0
ARC-c | 64.6 | 72.5 | 63.8 | 67.0 | 67.8 | 40.9 | 62.8
Book Corpus | 63.2 | 73.6 | 65.6 | 67.7 | 63.6 | 38.0 | 61.9

Table 4: Comparison of model performance with respect to choice of calibration
dataset

The results presented above show the influence of the calibration dataset
choice on model performance. It is unsurprising that the dataset, which
consists of the combination of all common sense tasks used for benchmarking,
exhibits the most favorable relative performance.

##  4 Computational Cost

We conduct ROM of LLaMA-7B (Touvron et al., 2023) on a CPU server with 128 GB
RAM and 48-core/96-thread processor. Our current implementation loads the
complete model at once; however, it is trivial to perform ROM layerwise and
hence can be done in under 10 GB of peak RAM given that only inputs and
weights of current layer are loaded and processed into the memory. On an
average it takes 13 seconds to perform ROM of each layer of LLaMA-7B (Touvron
et al., 2023) which has a total of 224 layers. Overall, it takes 15.8 minutes,
21.8 minutes and 28.9 minutes for 90%, 80% and 50% compression rates
respectively.

##  5 Conclusion

In this paper we presented a new direction for LLM compression leveraging
reduced order modeling of the latent features. Based on the concept of
identifying the finite set of most useful latent feature modes, LLM-ROM is
capable of compressing LLMs without the need for any fine-tuning. With no
requirement of a GPU during the compression process, LLM-ROM can be
efficiently run on a simple CPU machine. Moreover, unlike pruning, LLM-ROM is
very generic and does not require manual interference for different model
architectures. Based on the presented results, we hope to have paved way for a
novel approach to design compressed LLMs in a resource-efficient manner.

## References

  * Bisk et al. (2020) Yonatan Bisk, Rowan Zellers, Jianfeng Gao, Yejin Choi, et al.  Piqa: Reasoning about physical commonsense in natural language, 2020.
  * Clark et al. (2019) Christopher Clark, Kenton Lee, Ming-Wei Chang, Tom Kwiatkowski, Michael Collins, and Kristina Toutanova.  BoolQ: Exploring the surprising difficulty of natural yes/no questions.  In Jill Burstein, Christy Doran, and Thamar Solorio (eds.), _Proceedings of the 2019 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, Volume 1 (Long and Short Papers)_ , pp. 2924–2936, Minneapolis, Minnesota, June 2019. Association for Computational Linguistics.  doi: 10.18653/v1/N19-1300.  URL https://aclanthology.org/N19-1300.
  * Clark et al. (2018) Peter Clark, Isaac Cowhey, Oren Etzioni, Tushar Khot, Ashish Sabharwal, Carissa Schoenick, and Oyvind Tafjord.  Think you have solved question answering? try arc, the ai2 reasoning challenge.  _arXiv preprint arXiv:1803.05457_ , 2018.
  * Dettmers et al. (2023) Tim Dettmers, Artidoro Pagnoni, Ari Holtzman, and Luke Zettlemoyer.  Qlora: Efficient finetuning of quantized llms.  _arXiv preprint arXiv:2305.14314_ , 2023.
  * Frantar & Alistarh (2023) Elias Frantar and Dan Alistarh.  Sparsegpt: Massive language models can be accurately pruned in one-shot.  In _International Conference on Machine Learning_ , pp. 10323–10337. PMLR, 2023.
  * Frantar et al. (2022) Elias Frantar, Saleh Ashkboos, Torsten Hoefler, and Dan Alistarh.  Gptq: Accurate post-training quantization for generative pre-trained transformers.  _arXiv preprint arXiv:2210.17323_ , 2022.
  * Gu et al. (2023) Yuxian Gu, Li Dong, Furu Wei, and Minlie Huang.  Knowledge distillation of large language models.  _arXiv preprint arXiv:2306.08543_ , 2023.
  * Ma et al. (2023) X. Ma, G. Fang, and X. Wang.  Llm-pruner: On the structural pruning of large language models.  _NeurIPS_ , 2023.
  * Sakaguchi et al. (2021) Keisuke Sakaguchi, Ronan Le Bras, Chandra Bhagavatula, and Yejin Choi.  Winogrande: An adversarial winograd schema challenge at scale.  _Commun. ACM_ , 64(9):99–106, aug 2021.  ISSN 0001-0782.  doi: 10.1145/3474381.  URL https://doi.org/10.1145/3474381.
  * Sun et al. (2023) Mingjie Sun, Zhuang Liu, Anna Bair, and J Zico Kolter.  A simple and effective pruning approach for large language models.  _arXiv preprint arXiv:2306.11695_ , 2023.
  * Touvron et al. (2023) Hugo Touvron, Thibaut Lavril, Gautier Izacard, Xavier Martinet, Marie-Anne Lachaux, Timothée Lacroix, Baptiste Rozière, Naman Goyal, Eric Hambro, Faisal Azhar, Aurelien Rodriguez, Armand Joulin, Edouard Grave, and Guillaume Lample.  Llama: Open and efficient foundation language models, 2023.
  * Wu et al. (2023) Minghao Wu, Abdul Waheed, Chiyu Zhang, Muhammad Abdul-Mageed, and Alham Fikri Aji.  Lamini-lm: A diverse herd of distilled models from large-scale instructions.  _arXiv preprint arXiv:2304.14402_ , 2023.
  * Zellers et al. (2019) Rowan Zellers, Ari Holtzman, Yonatan Bisk, Ali Farhadi, and Yejin Choi.  Hellaswag: Can a machine really finish your sentence?  In _Proceedings of the 57th Annual Meeting of the Association for Computational Linguistics_ , pp. 4791–4800, 2019.
  * Zhu et al. (2015) Yukun Zhu, Ryan Kiros, Richard S. Zemel, Ruslan Salakhutdinov, Raquel Urtasun, Antonio Torralba, and Sanja Fidler.  Aligning books and movies: Towards story-like visual explanations by watching movies and reading books.  _CoRR_ , abs/1506.06724, 2015.  URL http://arxiv.org/abs/1506.06724.