# Molecule-molecule and atom-molecule collisions with ultracold RbCs molecules

1Philip D. Gregory, 1111Present address: Department of Physics, University of
Oxford, Oxford, United Kingdom, OX1 3PU.Jacob A. Blackmore, 2Matthew D. Frye,
1Luke M. Fernley, 1Sarah L. Bromley, 2Jeremy M. Hutson, and 1Simon L. Cornish
1Joint Quantum Centre (JQC) Durham-Newcastle, Department of Physics,
Durham University, Durham, United Kingdom, DH1 3LE.  2Joint Quantum Centre
(JQC) Durham-Newcastle, Department of Chemistry,
Durham University, Durham, United Kingdom, DH1 3LE.

###### Abstract

Understanding ultracold collisions involving molecules is of fundamental
importance for current experiments, where inelastic collisions typically limit
the lifetime of molecular ensembles in optical traps. Here we present a broad
study of optically trapped ultracold RbCs molecules in collisions with one
another, in reactive collisions with Rb atoms, and in nonreactive collisions
with Cs atoms. For experiments with RbCs alone, we show that by modulating the
intensity of the optical trap, such that the molecules spend 75% of each
modulation cycle in the dark, we partially suppress collisional loss of the
molecules. This is evidence for optical excitation of molecule pairs mediated
via sticky collisions. We find that the suppression is less effective for
molecules not prepared in the spin-stretched hyperfine ground state. This may
be due either to longer lifetimes for complexes or to laser-free decay
pathways. For atom-molecule mixtures, RbCs+Rb and RbCs+Cs, we demonstrate that
the rate of collisional loss of molecules scales linearly with the density of
atoms. This indicates that, in both cases, the loss of molecules is rate-
limited by two-body atom-molecule processes. For both mixtures, we measure
loss rates that are below the thermally averaged universal limit.

Ultracold polar molecules have been proposed for applications in the fields of
quantum computing [1, 2, 3, 4, 5, 6, 7], quantum simulation [8, 9, 10, 11, 12,
13, 14], quantum-state-controlled chemistry [15, 16, 17, 18], and precision
measurements [19, 20, 21, 22, 23, 24, 25, 26]. Experiments are now able to
produce a wide variety of ultracold polar molecules in the ground state by
association of atom pairs in a pre-cooled atomic mixture [27, 28, 29, 30, 31,
32, 33, 34, 35, 36] or by direct laser cooling of the molecules [37, 38, 39,
40, 41, 42, 43, 44, 45, 46, 47]. In these experiments, the densities of the
gases are often high enough that the effects of collisions are measurable and
typically limit the trap lifetime of the molecular gas. Understanding
ultracold collisions involving molecules is both fundamentally interesting,
due to their complexity, and crucial for further developing the techniques
needed to control collisional losses during experiments [48, 49, 50, 51, 52,
53, 54].

All ultracold molecules investigated to date experience fast collisional
losses from optical traps characterised by second-order kinetics [55, 56, 32,
57, 28, 30, 58, 59, 60, 61, 62, 63]. The molecules can be broadly categorised
as either reactive or nonreactive in photon-free two-body collisions; for a
pair of identical nonreactive molecules ($XY$) composed of the atomic species
$X$ and $Y$, the atom-exchange reactions of the form

$2XY\rightarrow X_{2}+Y_{2},$ $2XY\rightarrow X_{2}Y+Y,$ $2XY\rightarrow
X+Y_{2}X$

are all endothermic [64]. In contrast, for reactive molecules at least one of
these reactions is exothermic. Accordingly, these atom-exchange reactions can
cause fast loss of reactive molecules, but not nonreactive molecules. Despite
this, two-body collision rates at or near the universal limit [65, 66], where
all molecules that reach the short-range part of the interaction potential are
lost, have been observed in all experiments so far, independent of whether the
molecules are reactive [55, 56, 32, 57] or nonreactive [28, 30, 58, 59, 60,
34, 62, 61, 63]. In particular, Ye  _et al._ [58] have compared the loss of
23Na87Rb molecules in ground and first-excited vibrational states and
confirmed high loss and heating rates regardless of the energetics of the
exchange reactions.

A two-step process has been proposed to explain the fast loss of nonreactive
molecules from optical traps. First, during the collision a long-lived two-
molecule collision complex is formed [67, 68, 69, 70]; this is commonly
referred to as a ‘sticky collision’. The lifetime of the complex is commonly
estimated as

$\tau_{\mathrm{c}}=\frac{2\pi\hbar\rho}{N_{0}},$ (1)

where $\rho$ is the density of the accessible rovibrational states and $N_{0}$
is the number of open channels, with $N_{0}=1$ for nonreactive molecules in
their absolute ground states. This is based on Rice-Ramsperger-Kassel-Marcus
(RRKM) theory [71], which effectively assumes that the motion is ergodic, i.e.
that energy is fully randomised in the complex. Second, once the complex is
formed it may be removed from the trap due to electronic excitation by the
trap light leading to permanent loss of the molecule pair from the sample
[69].

We have previously established the validity of this two-step loss process for
RbCs molecules. We demonstrated that loss of RbCs molecules from an optical
trap is rate-limited by a two-body process [60], consistent with the ‘sticky
collision’ hypothesis that the loss is mediated by the formation of long-lived
two-molecule collision complexes. More recently, we showed that optical
excitation is the dominant mechanism by which the (RbCs)2 complexes are
removed from the gas [72]. We observed a suppression of the collisional loss
by applying square-wave modulation to the intensity of the optical trap to
form a time-averaged potential where 75% of each modulation cycle is dark. By
varying the frequency of the modulation, we measured a lifetime for the
complex in the dark of 0.53(6) ms [72], within a factor of $\sim 2$ of the
RRKM prediction of 0.253 ms [69]. A similar level of agreement between theory
and experiment has been found for the lifetime of the (KRb)2 complex formed in
reactive KRb+KRb collisions [73]. However, experiments in intensity-modulated
traps with nonreactive 23Na39K, 23Na40K, and 23Na87Rb have observed no
suppression of the collisional loss [63, 61]. Moreover, fast losses also exist
in experiments with 23Na40K confined to a repulsive box potential, where the
molecules spend the majority of the time in the dark [63]. These results
indicate either much faster optical excitation rates than expected or sticking
times that are at least an order of magnitude greater than the RRKM
prediction.

Atom-molecule collisions offer a compromise between the relative simplicity of
collisions between alkali-metal atoms and the complexity of collisions between
diatomic molecules. For molecules formed by association, it is convenient to
study collisions between the associated molecule and the constituents of the
initial atomic mixture. In this case, atom-molecule combinations $(X+XY)$ can
be defined as nonreactive if the atom-exchange reaction

$X+XY\rightarrow X_{2}+Y$

is endothermic. However, as was found for molecule-molecule collisions, the
reactivity of the combination alone does not appear to determine the rate of
collisional loss observed in experiments [55, 34, 74, 75, 76, 77, 78, 79]. For
example, reactive atom-molecule collisions in mixtures of triplet 23Na6Li
molecules and 23Na atoms are sufficiently suppressed for the fully stretched
hyperfine states that efficient sympathetic cooling of the molecules is
possible [74]. Experiments studying nonreactive 23Na39K+39K collisions have
measured a hyperfine-dependent two-body loss rate far below the universal
limit [79]. However, Nichols  _et al._ [77] recently measured a photon-free
lifetime for complexes in nonreactive collisions for a 40K87Rb+87Rb mixture to
be $\sim 10^{5}$ times higher than predicted by RRKM; it is therefore possible
that optical excitation of long-lived two-body collision complexes also plays
a role in the fast losses observed in other atom-molecule collisions. Yang
_et al._ [34] have reported the existence of Feshbach resonances between
23Na40K + 40K. These resonances have been attributed to long-range states of
the triatomic complex in which the atoms and molecules retain their individual
character [80]. One of these Feshbach resonances was recently used to form
weakly-bound 23Na40K2 molecules by rf association [81]. Magnetic Feshbach
resonances have also been observed for reactive 23Na6Li$+^{23}$Na collisions,
where the loss rate can be modulated by more than a factor of a hundred [78].

There remain many open questions in this field. First, we do not yet know the
extent to which laser-induced loss via complexes is dominant. There are some
systems in which there is no energetically allowed two-body pathway that leads
to trap loss; at densities too low for three-body collisions, laser absorption
seems to be the only available loss mechanism. However, for reactive systems
and some systems involving excited atomic or molecular states, laser-free
reactive or elastic processes may compete. Such losses may themselves occur
either directly or via complexes. When complexes are involved, the factors
that determine their lifetimes are poorly understood. Indeed, it is not even
certain whether the complexes behave chaotically, with widths that are well
described by RRKM theory. If they are chaotic, it is unclear whether the
electron and nuclear spins are coupled into the chaotic bath, or are to some
extent decoupled from it.

Motivated by these open questions, in this article we study collisional losses
in an ultracold gas of 87Rb133Cs molecules (hereafter RbCs), and in ultracold
mixtures of RbCs+87Rb and RbCs+133Cs. We first examine RbCs+RbCs collisions.
In section 1.1, we outline a model for loss of molecules from an optical trap
via optical excitation and inelastic loss of two-molecule collision complexes.
We show how the use of an intensity-modulated trap may be used to probe these
loss mechanisms. In section 1.2, we present our method for creating an
ultracold gas of RbCs molecules. In section 1.3, we show results comparing
collisional loss of molecules from continuous-wave and modulated traps, for
molecules prepared in three different hyperfine states. We observe the largest
suppression of collisional losses in the modulated trap for molecules in the
hyperfine ground state. We then proceed to study molecule loss in a reactive
mixture of RbCs+Rb and a nonreactive mixture of RbCs+Cs in section 2. We begin
by explaining our experimental method for measuring atom-molecule collision
rates in our apparatus. In section 2.1, we show that the molecule loss rates
for both the reactive and nonreactive collisions depend linearly on the atomic
density; this confirms that the collisional loss in both mixtures has a two-
body rate-limiting step. In section 2.2, we extract two-body rate coefficients
from our measurements of molecule loss. We find rate coefficients that are
lower than the limit of universal loss, but are similar for reactive and
nonreactive collisions. Finally, in section 2.3, we compare loss from the
nonreactive mixture of RbCs+Cs in intensity-modulated and CW traps. We find no
significant change in the two-body rate coefficient when the optical trap is
modulated and interpret this observation using our rate-equation model for the
optical excitation of collision complexes.

##  1 RbCs+RbCs Collisions

###  1.1 A rate-equation model for loss via bimolecular collision complexes

We begin by examining molecule-molecule collisions in an optical trap, where
the intensity of the light is modulated as a square-wave such that the
molecules spend 75% of each modulation cycle in the dark. In doing so, we
partially suppress the optical excitation of two-body collision complexes,
which is believed to be the dominant mechanism for collisional loss for pairs
of molecules in the absolute ground state. When the trap light is off,
complexes can form and break apart without the risk of destructive optical
excitation. This leads to a reduction in the loss rate, with the maximum
fractional reduction in loss simply equal to the duty cycle of the modulation.

We model the rate of change of the densities of ‘free’ molecules
${n}_{\mathrm{m}}$ and bimolecular complexes ${n}_{\mathrm{c}}$ with the rate
equations

$\dot{n}_{\mathrm{m}}=-k_{2}n_{\mathrm{m}}^{2}+\frac{2}{\tau_{\mathrm{-1}}}n_{\mathrm{c}},\\\
\dot{n}_{c}=+\frac{1}{2}k_{2}n_{\mathrm{m}}^{2}-\frac{1}{\tau_{\mathrm{-1}}}n_{\mathrm{c}}-\frac{1}{\tau_{\mathrm{inel}}}n_{\mathrm{c}}-k_{\mathrm{laser}}I(t)n_{\mathrm{c}}.$
(2)

This differs from previous treatments [60, 63, 61] in the inclusion of a term
for inelastic loss of complexes. Here $k_{2}$ describes the rate of formation
of complexes and $k_{\mathrm{laser}}$ is the photon scattering rate of the
complexes per unit intensity $I(t)$. The lifetime $\tau_{\mathrm{-1}}$ is the
$1/e$ time for dissociation of the complexes back to free molecules in the
initially prepared state, and $\tau_{\mathrm{inel}}$ describes the loss of
molecules via a non-optical mechanism, such as conversion to a state other
than the one in which the molecules are initially prepared. The lifetime of
the complex in the dark is then
$1/\tau_{\mathrm{c}}=1/\tau_{-1}+1/\tau_{\mathrm{inel}}$, and the probability
that the molecules return to the initial state is
$\Phi=\tau_{\mathrm{inel}}/(\tau_{-1}+\tau_{\mathrm{inel}})$. We have
previously measured $k_{2}$ by examining the rate of loss of molecules from a
continuous-wave trap, which is rate-limited by the formation of complexes. The
molecules have a temperature of $2$ $\mu$K and $k_{2}=5.4\times 10^{-11}$ cm3
s-1 [60, 72], which is about a factor of two lower than the thermally averaged
universal rate [66]. In addition, we set $k_{\mathrm{laser}}=3\times 10^{3}$
W-1 cm2 s-1 as measured in our previous experiments [60] on molecules in their
absolute ground state (where $\Phi=1)$. The behaviour of our model is however
insensitive to this value provided that $k_{\mathrm{laser}}I(t)\gg
1/\tau_{\mathrm{c}}$ when the trap light is on, corresponding to the situation
where the loss of complexes is strongly saturated.

Figure 1: Suppression of loss of molecules using an intensity-modulated trap.
We present results of the model described by Eq. 2 for a trap modulation
frequency of $f_{\mathrm{mod}}=1$ kHz and a $25\%$ duty cycle. The other
parameters are set to the values determined in our previous work [72], namely
$k_{2}=5.4\times 10^{-11}$ cm2 s-1, $k_{\mathrm{laser}}=3\times 10^{3}$ W-1
cm2 s-1, $I=2\times 10^{4}$ W cm-2, and $n_{\mathrm{m,0}}=10^{11}$ cm-3. (a)
Comparison between the variation in density of molecules $n_{\mathrm{m}}$ in a
continuous-wave trap (solid line) and a modulated trap (dashed) for the case
where $\Phi=1$ and $\tau_{\mathrm{c}}=\tau_{\mathrm{-1}}=0.53$ ms. The inset
highlights the change in density at short timescales. The shaded regions
indicate time when the trap light is on. (b) Effect of varying the decay time
for complexes back to the initially prepared state $\tau_{-1}$ on the
suppression of loss in the modulated trap. We plot the fractional difference
between the density in CW ($n_{\mathrm{CW}}$) and modulated
($n_{\mathrm{mod}}$) traps after a 200 ms hold time. The top horizontal axis
expresses $\tau_{-1}$ as a fraction of the dark time in the trap
$t_{\mathrm{dark}}$. The solid, dashed, and dotted lines shows the case for
$\Phi=1$, 0.5, and 0.1, respectively. The vertical dashed line indicates the
value $\tau_{\mathrm{-1}}$ used in (a).

In Fig. 1, we show the effect of modulating the trap intensity as a square
wave by solving these rate equations for continuous-wave (CW) and modulated
traps. For the modulated trap, the intensity is modulated at a frequency of 1
kHz and the molecules spend 75% of each cycle in the dark. In Figure 1(a), we
fix $\Phi=1$ and $\tau_{\mathrm{c}}=\tau_{-1}=0.53$ ms, as previously measured
for molecules in the spin-stretched hyperfine ground state [72]. In this case
we expect a slower rate of molecule loss in the modulated trap (dashed line)
when compared to the CW trap (solid line). In Fig. 1(b), we examine the effect
of varying $\tau_{\mathrm{-1}}$ and $\Phi$, for a fixed hold time in the trap
of 200 ms. We find that significant suppression of the loss occurs only if
$\tau_{-1}<\tau_{\mathrm{inel}}$ and $\tau_{-1}<t_{\mathrm{dark}}$, where
$t_{\mathrm{dark}}$ is the duration the molecules spend in the dark during
each cycle of the trap modulation.

The dark time can be varied in experiments by changing the frequency of the
modulation, and we expect any suppression to become greater as the modulation
frequency is reduced and the dark time increases correspondingly. Measurements
in modulated traps thus allow one to distinguish optical loss from other loss
mechanisms only if the lifetime of the complex is significantly shorter than
the dark time and the optical loss dominates over any laser-free loss.

###  1.2 Creating ultracold ground-state RbCs molecules

For our experiments, we create a sample of molecules [82, 83, 29, 84, 85]
starting from an ultracold atomic mixture of Rb and Cs atoms confined to a
magnetically levitated crossed optical dipole trap (with a wavelength,
$\lambda=1550$ nm) [82]. We first form weakly bound molecules from the atomic
mixture using magnetoassociation on an interspecies Feshbach resonance at 197
G [83]. Following this, we remove the remaining atoms from the trap using the
Stern-Gerlach effect [83]. We then transfer the molecules into an optical trap
where the intensities of the beams are modulated as a square wave. This is
achieved by ramping up the intensity of the modulated trap and switching off
the 1550 nm CW trap and magnetic levitation gradient.

The modulated trap is formed from a single beam $(\lambda=1064$ nm) in a bow-
tie configuration. The trap frequencies experienced by molecules in this trap
are $[\omega_{x},\omega_{y},\omega_{z}]/2\pi=[96(2),160(3),185(3)]$ Hz. The
square-wave intensity modulation is achieved by blocking the light using an
optical chopper wheel. Using this method, we can modulate the trap intensity
with cycle frequencies of up to 5 kHz. We find significant trap loss for
modulation frequencies below $1$ kHz, consistent with loss due to parametric
heating resonances, as occurs in time-dependent potentials that are not fully
in the time-averaged trap regime [86, 87, 88].

We transfer the molecules in the modulated trap to the ${}^{1}\Sigma$
rovibrational ground state using STImulated Raman Adiabatic Passage (STIRAP)
[29, 84, 85]. The STIRAP is performed at a magnetic field of $B=181.6$ G
during the trap dark time. Immediately after STIRAP, the sample typically
consists of up to 5000 molecules with a mean density of $\sim 10^{11}$ cm-3.
STIRAP prepares the molecules in a single hyperfine level of the rovibrational
ground state.

For RbCs at $B=181.6$ G the rovibrational ground state consists of 32
hyperfine states spread across 1.3 MHz with separations between neighbouring
states ranging from 10 to 100 kHz as shown in Fig. 2(a). We label these
hyperfine states by $(n,m_{f,\mathrm{RbCs}})_{k}$ where $n$ is the quantum
number for rotational angular momentum,
$m_{F}=m_{\mathrm{Rb}}+m_{\mathrm{Cs}}+m_{n}$ is the sum of the angular
momentum projections for the nuclear spins $m_{\mathrm{Rb}},m_{\mathrm{Cs}}$
and the rotation of the molecule $m_{n}$, and $k$ is an index that counts up
the states in order of increasing energy for a given value of $n$ and
$m_{f,\mathrm{RbCs}}$. For magnetic fields above $90$ G the spin-stretched
state $(0,5)_{0}$ state is the hyperfine ground state of RbCs. With the
STIRAP, we are able to populate either the $(0,5)_{0}$ or $(0,4)_{1}$
hyperfine states directly. In this work, we also prepare molecules in
$(0,4)_{0}$, which we achieve using a pair of one-photon $\pi$-pulses to drive
transitions in the molecule coherently between $n=0$ and $n=1$ [89]. The
states used in this work are highlighted in Fig. 2(a).

###  1.3 RbCs+RbCs in an intensity-modulated trap

Our understanding of the formation and optical excitation of molecule-molecule
collision complexes is far from complete. This is evidenced by observations in
23Na39K, 23Na40K, and 23Na87Rb [63, 61] where no suppression of loss in
modulated traps was seen, despite RRKM predictions of lifetimes for complexes
[69] that are much shorter than the dark time in the trap. We have previously
observed a suppression of loss for RbCs molecules in the spin-stretched
hyperfine ground state, so here we explore collisions of molecules prepared in
different hyperfine states, where other loss channels and mechanisms could
affect the lifetime of the complex.

To measure the effect of the dark time in the modulated trap, we measure the
number of molecules remaining after a 200 ms hold in the modulated trap, both
with ($N_{\mathrm{mod+CW}}$) and without ($N_{\mathrm{mod}}$) an additional CW
source of 1550 nm light. The CW light is derived from the 1550 nm trap used to
prepare the Feshbach molecules, and has total peak intensity of $\sim 3\times
10^{2}$ W cm-2. This intensity is sufficiently low that it does not
significantly affect the trap frequencies or the trap depth experienced by the
molecules, but high enough to remove complexes continuously from the trap
[72]. By performing a comparative measurement, we remove any sensitivity to
the effects of residual heating and associated evaporative loss that may arise
due to the modulation of the trap intensity. To measure the molecule number,
we reverse the STIRAP and association process, to break the molecules back
apart into their constituent atoms for detection by absorption imaging. For
each measurement of loss, we perform 50 interleaved measurements of
$N_{\mathrm{mod}}$ and $N_{\mathrm{mod+CW}}$ and extract a mean and standard
error from the resulting distributions.

Figure 2: (a) Hyperfine Zeeman structure of RbCs in the rovibrational ground
state. The states used in this work are highlighted in bold colours and
labelled by $(n,m_{f,\mathrm{RbCs}})_{k}$ as defined in the main text. (b)
Fractional change in the molecule number
$N_{\mathrm{mod}}/N_{\mathrm{mod+CW}}-1$ as a function of the modulation
frequency of the trap light intensity. Results are shown for molecules in
$(0,5)_{0}$ in red, $(0,4)_{0}$ in blue, and $(0,4)_{1}$ in green, with all
measurements performed at a magnetic field of 181.6 G. We observe a smaller
suppression of the collisional loss for molecules not occupying the hyperfine
ground state $(0,5)_{0}$. The lines show fits to the results using the
numerical model in Eq. 2 for $\Phi=1$, with the 1$\sigma$ uncertainty in the
lifetime of the complex in the dark indicated by the shaded regions for each
fit.

We characterise the suppression of loss due to the dark time by calculating
the fractional difference in the molecule number
$N_{\mathrm{mod}}/N_{\mathrm{mod+CW}}-1$. This is shown as a function of
modulation frequency in Fig. 2(b) for molecules prepared in several different
hyperfine states. For molecules in $(0,5)_{0}$ we observe a suppression of
loss characterised by $N_{\mathrm{mod}}/N_{\mathrm{mod+CW}}-1>0$. The
suppression is greatest at the lowest modulation frequencies we are able to
reach where $t_{\rm{dark}}/\tau_{\rm{-1}}$ is greatest, see Fig. 1(b)). We fit
the results using the rate equation model (Eq. 2), making the assumption that
the molecules remain at a fixed temperature. We fix $\Phi=1$ because the
$(0,5)_{0}$ state is both spin-stretched and is the lowest-energy hyperfine
state; the molecules do not have sufficient kinetic energy to leave the
complex in any other state. As such, the lifetime of the collision complex in
the dark is the only free parameter in the fitting. We find an optimal value
for the lifetime of the (RbCs)2 collision complex is 0.8(3) ms in the dark,
where the number in brackets is the 1$\sigma$ uncertainty. This is consistent
with our previously measured value of 0.53(6) ms [72].

Fig. 2(b) also shows similar measurements for molecules prepared in the
higher-energy hyperfine states $(0,4)_{0}$ and $(0,4)_{1}$. In these cases we
observe a lower suppression of the loss. We first analyse results with the
assumption $\tau_{\mathrm{inel}}\rightarrow\infty,\Phi=1$. In this limit, our
results would suggest that the lifetime of the complex in the dark depends on
the hyperfine state, which might be associated with an increase in the
effective density of states. This change might be caused by the increased
number of nuclear spin arrangements that have the same value of the total spin
projection $M_{F}=m_{f,1}+m_{f,2}$ as the incoming pair. For the $(0,4)_{0}$
state our results are consistent with $\tau_{\mathrm{c}}=\tau_{-1}=2.1(1.3)$
ms. In contrast, for the $(0,4)_{1}$ state our results are consistent with no
suppression of loss and we can place only a lower limit on the lifetime of the
complex in the dark, $\tau_{\mathrm{c}}=\tau_{-1}>3.3$ ms (at the 68%
confidence level).

An alternative interpretation of the reduced suppression for the states with
$m_{f,\mathrm{RbCs}}=4$ is the presence of one or more additional loss
channels. Collisions between molecules in either state have at least one
energetically accessible inelastic channel that conserves $M_{F}$. Decay of
complexes to form molecules in different states would be observed as loss in
our experiments, because we detect molecules only in the specific hyperfine
state in which they are prepared. To test this interpretation, we restrict
$\tau_{-1}$ to the range of values in the 68% confidence interval found for
the $(0,5)_{0}$ state, $\tau_{\mathrm{-1}}=0.8(3)$ ms, and fit the results for
$(0,4)_{0}$ and $(0,4)_{1}$ with $\tau_{\mathrm{inel}}$ as a free parameter.

For molecules prepared in $(0,4)_{0}$ there is one energetically available
channel for spin exchange, with the molecules exiting the complex in
$(0,5)_{0}+(0,3)_{0}$. This combination of states is higher in energy than the
prepared state by only $k_{\mathrm{B}}\times 0.16$ $\mu$K, which is much
smaller than the temperature of the molecules $T=2\,\mu$K. For molecules in
$(0,4)_{0}$, our results find an optimum value
$\tau_{\mathrm{inel}}=0.2^{+0.5}_{-0.1}$, leading to $\Phi=0.2^{+0.4}_{-0.1}$.
For molecules in $(0,4)_{1}$, several spin exchange channels are available. In
addition, there is the possibility of exchange between nuclear spins in the
same molecule such that it exits in the lower-energy $(0,4)_{0}$ state. For
the $(0,4)_{1}$ state, we find $\tau_{\mathrm{inel}}<0.13$ ms and $\Phi<0.08$
at the 68% confidence level.

This analysis demonstrates that our experimental results for excited molecular
states can be explained either by a longer lifetime $\tau_{-1}$ or by the
presence of laser-free inelastic decay. We cannot at present distinguish
between these explanations.

##  2 Collisions of RbCs with Rb or Cs

We now examine collisions between RbCs molecules and the constituents of the
atomic mixture in which they are prepared. To achieve this, the Rb-Cs mixture
is prepared in a purely optical potential ($\lambda=1550$ nm), with no
magnetic levitation to support against gravity. We then perform
magnetoassociation on the same Feshbach resonance as before ($B_{0}=197$ G),
but the transfer to the $(0,5)_{0}$ ground state instead takes place 1 ms
after magnetoassociation, while the atoms are still present in the trap. The 1
ms hold is necessary to allow the magnetic field to reach $181.6$ G and become
sufficiently stable for efficient STIRAP. With the molecules in the ground
state, one atomic species is removed by ramping the magnetic field to 21.3 G
and turning on the appropriate repump and cooling light from the magneto-
optical trap for 3 ms. After the unwanted atoms are removed, we either hold
the atom-molecule mixture in the optical trap at 21.3 G, or ramp the magnetic
field to 181.6 G over a further 2 ms for the lifetime measurement. At the end
of the measurement, prior to dissociation and imaging, the magnetic field is
jumped back to 21.3 G to remove the atomic species left after the first
removal pulse. We also measure molecular lifetimes without atoms using this
sequence by removing both species of atoms in the first removal step. This
provides a useful consistency check with the measurements discussed in section
1 to verify the efficacy of the atom-removal procedure. We also use such
measurements to account for background loss of molecules, as described later.

Our experimental sequence produces up to $\sim 8\times 10^{5}$ ($3\times
10^{5}$) Rb (Cs) atoms at a temperature of 1.1(1) $\upmu$K, and mean density
of typically $\sim 10^{12}$ cm3 s-1. Each species is prepared in the hyperfine
ground state $(f_{\mathrm{Rb}}=1,m_{f,\mathrm{Rb}}=1)$ for Rb and
$(f_{\mathrm{Cs}}=3,m_{f,\mathrm{Cs}}=3)$ for Cs. The temperature and density
of the atoms remain constant over the duration of the experiments (typical
timescales of $\sim$10 ms). The molecules on the other hand have a temperature
of $1.3(3)$ $\upmu$K, and and mean starting density $\sim 10^{10}$ cm-3. The
reduced starting density for the molecules in these measurements is caused by
a combination of lower trap frequencies and loss of molecules from inelastic
collisions between atoms and Feshbach molecules during the association
procedure.

Figure 3: Example molecule loss measurements performed at 181.6 G for
molecules alone and for mixtures of RbCs+Rb and RbCs+Cs. The molecules are
prepared in the $(n=0,m_{f,\mathrm{RbCs}}=5)_{0}$ state which is the hyperfine
ground state of the molecule at this magnetic field. Collisional loss of RbCs
molecules alone is shown by the black squares with a linear fit to the results
indicated by the dashed line. Empty red circles show loss of RbCs molecules
when prepared with ground-state Rb atoms
$(f_{\mathrm{Rb}}=1,m_{f,\mathrm{Rb}}=1)$, such that the loss is dominated by
reactive RbCs+Rb collisions. Filled blue circles show loss of RbCs molecules
with ground-state Cs atoms $(f_{\mathrm{Cs}}=3,m_{f,\mathrm{Cs}}=3)$, where
loss is dominated by nonreactive RbCs+Cs collisions. The solid lines are
exponential fits incorporating the molecule-only loss following Eq. 3 as
explained in the main text.

The lifetime of RbCs molecules in the presence of each species of atom is
shown in Fig. 3. To account for the background variation in the number of
molecules due to molecule-molecule collisions, we also measure a lifetime for
molecules alone. As the atom-molecule loss is much faster than the molecule-
only loss in all measurements presented, we fit the molecule-only loss with a
straight line with starting number $N_{0}$ and gradient $m$ as shown. We then
fit the variation in the number of molecules $N_{\mathrm{m}}$ for each atom-
molecule combination with an exponential function,

$N_{\mathrm{m}}(t)=N_{0}\left(1-mt\right)\exp{\left(\frac{-t}{\tau}\right)}.$
(3)

Here, $t$ is the hold time in the trap, $\tau$ is the $1/e$ lifetime for the
atom-molecule collisions, $N_{0}$ is the initial number of molecules, and the
first term in brackets represents the normalisation to the molecule-only
background. For the curves shown in Fig. 3, we extract $1/e$ lifetimes of
16(1) ms for reactive RbCs$+$Rb collisions and 30(2) ms for nonreactive
RbCs$+$Cs collisons. However, as the densities of Rb and Cs are different,
direct comparison of these time constants is not immediately useful; we
present density-normalised rate coefficients, which may be properly compared,
in section 2.3.

###  2.1 Density dependence

Studying the change in molecule loss rate as a function of atom density can
yield insight into the kinetics of the underlying loss mechanism. For reactive
RbCs+Rb collisions, the energetically allowed atom-exchange reaction is likely
to be the dominant loss mechanism, although it is possible that laser-induced
loss may also be important [77]. As this is a two-body (atom+molecule)
process, we expect that the rate of loss of molecules will depend linearly
upon the density of atoms. For nonreactive RbCs+Cs collisions however, the
expectation is less clear. Possible mechanisms for loss in the nonreactive
mixture include (but may not be limited to) optical excitation of two-body
(atom-molecule) complexes and three-body (atom-atom-molecule) collisions. We
expect the loss rate associated with these processes to depend on the atom
density, or the square of the atom density, respectively.

Figure 4: Dependence of the molecule loss rate on the mean atomic density. We
control the atomic density by pulsing on near-resonant light that reduces the
number of atoms remaining in the trap. (a) Loss rate of RbCs molecules in
$(n=0,m_{f,\mathrm{RbCs}}=4)_{1}$ as a function of the Rb density. The solid
line is a linear fit to the results constrained to go through the origin with
gradient $3.2(2)\times 10^{11}$ cm3 s-1. (b) Loss rate of RbCs molecules in
$(0,5)_{0}$ as a function of the Cs density. The solid line is a linear fit to
the results constrained to go through the origin with gradient $2.05(7)\times
10^{11}$ cm3 s-1.

To vary the atom density we use resonant light to remove a fraction of the
chosen atomic species from the trap. This is performed in parallel with the
removal of the unwanted atomic species after STIRAP. The additional light is
tuned to be resonant with the
$5^{2}\mathrm{S}_{1/2}(f_{\mathrm{Rb}}=1,m_{f,\mathrm{Rb}}=1)\rightarrow
5^{2}\mathrm{P}_{3/2}(2,2)$ electronic transition for Rb and the
$6^{2}\mathrm{S}_{1/2}(f_{\mathrm{Cs}}=3,m_{f,\mathrm{Cs}}=3)\rightarrow
6^{2}\mathrm{P}_{3/2}(4,4)$ for Cs. The light is horizontally polarised and
delivered in a collimated beam of radius $\sim 1$ mm that propagates
orthogonal (designated $x$ axis) to the 21.3 G magnetic field (along the $z$
axis). Throughout the resonant light pulse, we also switch on cooling light
from the magneto-optical trap which removes any atoms that spontaneously decay
into the $f_{\mathrm{Rb}}=2$ state of Rb and the $f_{\mathrm{Cs}}=4$ state of
Cs. We observe loss of atoms with an exponential time constant of 88(2)
$\upmu$s for Rb and 6.4(2) $\upmu$s for Cs. The difference in removal rate is
primarily due to the different laser intensities used for the different
species. Using this method, we do not observe any significant build-up of
population in other atomic hyperfine states, such that the atoms that survive
the resonant light pulse remain in the target $(1,1)$ state for Rb and $(3,3)$
state for Cs. However, we do observe that the temperature of the atoms
increases with the duration of the light pulse, with approximately a factor of
2 greater increase in the direction of the laser propagation. This change in
temperature is taken into account when calculating the atomic density.

We present measurements of molecule loss rate as a function of mean atom
density in Fig. 4(a,b). In Fig. 4(a), we examine the reactive combination
RbCs+Rb. Specifically, we measure collisions between molecules prepared in
$(0,4)_{1}$ with ground-state Rb atoms. As expected, we observe a linear
dependence of the molecule loss rate on atom density. A linear fit to the
results, constrained to pass through the origin, yields a gradient of
$3.3(2)\times 10^{11}$ cm3 s-1. In Fig. 4(b), we show a measurement using the
nonreactive combination of RbCs+Cs. In this case the molecules are prepared in
$(0,5)_{0}$ so that both the Cs atoms and RbCs molecules occupy their
respective hyperfine ground states. For the nonreactive combination we also
find a linear dependence of the molecule loss rate on the atom density, this
time with a gradient of $2.05(7)\times 10^{11}$ cm3 s-1. This indicates that
the loss mechanisms for both reactive and nonreactive collisions have rate-
limiting steps that depends on a two-body RbCs+atom collision.

###  2.2 Second-order rate coefficients for collisions of RbCs with Rb or Cs

Figure 5: Second-order loss rate coefficients $k^{\mathrm{a,m}}_{2}$ measured
for RbCs molecules collisions with ground-state (a) Rb (b) Cs atoms.
Measurements were performed at magnetic fields of 21.3 G and 181.6 G. Filled
and empty circles show the result of measurements performed with RbCs
molecules in the $(n=0,m_{f,\mathrm{RbCs}}=5)_{0}$ and $(0,4)_{1}$ states
respectively. The coloured solid horizontal line indicates the average of the
two loss rates measured for molecules in the $(0,5)_{0}$ state, and the dashed
line is the average for molecules in the $(0,4)_{1}$ state. In all
measurements the atoms occupy their hyperfine ground states, either
($f_{\mathrm{Rb}}=1,m_{f,\mathrm{Rb}}=1$) or
($f_{\mathrm{Cs}}=3,m_{f,\mathrm{Cs}}=3$) for Rb and Cs respectively. The
black dashed line indicates the thermally averaged universal rate for the
particular atom-molecule combination.

As both reactive and nonreactive atom-molecule collisions appear to be rate-
limited by second-order kinetics, we can quantify the loss rate using a
second-order rate coefficient $k_{2}^{\mathrm{a,m}}$. To extract a second-
order rate coefficient from a measurement of molecule loss, we model the loss
of molecules due to atom-molecule collisions with the rate equation

$\dot{n}_{\mathrm{m}}(\textbf{r},t)=-k^{\mathrm{a,m}}_{2}n_{\mathrm{a}}(\textbf{r},t)n_{\mathrm{m}}(\textbf{r},t),$
(4)

where $n_{\mathrm{a}},n_{\mathrm{m}}$ represent the densities of atoms and
molecules, respectively. The rate of change of the number of molecules can be
obtained by integrating Eq. 4, giving

$\dot{N}_{\mathrm{m}}(t)=-k^{\mathrm{a,m}}_{2}\int{n_{\mathrm{a}}(\textbf{r},t)n_{\mathrm{m}}(\textbf{r},t)}{d^{3}\textbf{r}}.$
(5)

The term in the integral contains the overlap between the distributions of
atoms and molecules, and we can use this integral to define a mean
interspecies density

$\bar{n}_{\mathrm{a,m}}=\frac{1}{N_{\mathrm{m}}(t)}\int{n_{\mathrm{a}}(\textbf{r},t)n_{\mathrm{m}}(\textbf{r},t)}{d^{3}\textbf{r}}.$
(6)

If we assume the atoms and molecules are thermally distributed in the harmonic
region of the trap with temperatures $T_{\mathrm{a}}$ and $T_{\mathrm{m}}$,
respectively, then,

$\bar{n}_{\mathrm{a,m}}=N_{\mathrm{a}}F_{z}(\Delta_{z})\left[\frac{m_{\mathrm{m}}\omega_{\mathrm{m}}^{2}m_{\mathrm{a}}\omega_{\mathrm{a}}^{2}}{2\pi
k_{\mathrm{B}}(m_{\mathrm{m}}\omega_{\mathrm{m}}^{2}T_{\mathrm{a}}+m_{\mathrm{a}}\omega_{\mathrm{a}}^{2}T_{\mathrm{m}})}\right]^{3/2},$
(7)

where $N_{\mathrm{a}}$ is the number of atoms (which remains constant over the
duration of the measurement), $k_{\mathrm{B}}$ is the Boltzmann constant,
$m_{\mathrm{a}}$, $m_{\mathrm{m}}$ are the masses and
$\omega_{\mathrm{a}},\omega_{\mathrm{m}}$ are the geometrically-averaged trap
frequencies experienced by the atoms and molecules, respectively.
$F_{z}(\Delta_{z})$ describes the reduction in overlap due to the difference
in gravitational sag $\Delta_{z}$ between the two clouds [90],

$F_{z}(\Delta_{z})=\exp\left[-\frac{m_{\mathrm{m}}\omega_{\mathrm{z,m}}^{2}m_{\mathrm{a}}\omega_{\mathrm{z,a}}^{2}\Delta_{z}^{2}}{2k_{\mathrm{B}}(m_{\mathrm{a}}T_{\mathrm{m}}\omega_{\mathrm{z,a}}^{2}+m_{\mathrm{m}}T_{\mathrm{a}}\omega_{\mathrm{z,m}}^{2})}\right],$
(8)

where $\omega_{\mathrm{z,a}}$, $\omega_{\mathrm{z,m}}$ are the vertical trap
frequencies of the atoms and molecules, respectively. The largest difference
in gravitational sag in our experiments, $\Delta_{z}=2.7$ $\upmu$m, is between
Rb and RbCs. In this case, we find $F_{z}(2.7\,\upmu\mathrm{m})=0.98$; the
difference in gravitational sag changes the interspecies density by only 2%,
which is much less than the typical uncertainty in either the atom or the
molecule density alone (each typically $\sim 10\%$). We therefore neglect the
$F_{z}(\Delta_{z})$ term in Eq. 7. Returning to Eq. 5, we have

$\dot{N}_{\mathrm{m}}(t)=-k^{\mathrm{a,m}}_{2}\bar{n}_{\mathrm{a,m}}N_{\mathrm{m}}.$
(9)

We have confirmed experimentally that the atom number and temperature do not
change appreciably over the course of the measurements presented. If we
additionally assume that the molecule temperature does not change
significantly, then $\bar{n}_{\mathrm{a,m}}$ is a constant. This produces
pseudo-first-order kinetics with the solution

$N_{\mathrm{m}}(t)=N_{0}\exp{\left(-k^{\mathrm{a,m}}_{2}\bar{n}_{\mathrm{a,m}}t\right)},$
(10)

where $N_{0}$ is the initial number of molecules. This allows us to extract
two-body rate coefficients from the measured exponential time constants.

We first compare the loss rates measured with molecules prepared in the
hyperfine ground state $(0,5)_{0}$ at 181.6 G. Example molecule loss
measurements are presented in Fig. 3. For the reactive combination RbCs+Rb ,
we find $k_{2}^{\mathrm{a,m}}=2.0(4)\times 10^{-11}$ cm3 s-1, while for the
nonreactive combination RbCs+Cs we find $k_{2}^{\mathrm{a,m}}=1.8(3)\times
10^{-11}$ cm3 s-1. Additional loss rates measured at a magnetic field of 21.3
G and for molecules prepared in $(0,4)_{1}$ are presented in Fig. 5. We
observe no significant variation with the large change in magnetic field, and
only marginally higher loss rates for atom-molecule collisions involving
molecules prepared in $(0,4)_{1}$.

Figure 6: Thermally averaged two-body loss rate coefficient $k_{2}$ from the
QDT model, as a function of loss parameter $y$ and short-range phase shift
$\delta^{\mathrm{s}}$ for RbCs+Rb (left) and RbCs+Cs (right). Solid and dashed
black lines show the experimental value and uncertainty.

We compare the loss rates measured in experiments to a single-channel model
based on quantum defect theory (QDT) [65, 66]. The model and its underlying
theory have been described at length elsewhere [91, 92, 66], so we omit
further description here; we have previously applied it to RbCs+RbCs and
Rb+CaF collisions [60, 75, 76]. The long-range interactions are approximated
by their leading term $-C_{6}R^{-6}$ and the short-range interactions are
modelled by an absorbing boundary condition. We use values of $C_{6}$ from
Żuchowski _et al._ [93]. The boundary condition is parameterized by the loss
parameter $0\leq y\leq 1$ of Idziaszek and Julienne [65] and a short-range
phase shift $\delta^{\mathrm{s}}$, which is related to the scattering length
and controls interference effects including resonances. In the limit of $y=1$
all flux that is transmitted past the long-range potential is lost; this is
termed the universal limit. The thermally averaged universal loss rates at the
current experimental temperatures are $6.0\times 10^{-11}$ cm3 s-1 for RbCs+Rb
and $5.0\times 10^{-11}$ cm3 s-1 for RbCs+Cs; these are indicated by
horizontal dashed lines in Fig. 5.

The results of the QDT model as a function of $y$ and $\delta^{\mathrm{s}}$,
for RbCs+Rb and RbCs+Cs, are shown in Fig. 6, with the experimental results
and their uncertainties. The present results limit the loss parameter to less
than 0.35, with the most likely range being $0.2<y<0.3$ for both species. It
is notable that the ranges are very similar for the two systems, even though
one is potentially reactive and the other is non-reactive. The ranges also
overlap with the result for RbCs+RbCs, $y=0.26(3)$ [60]. All three results are
consistent with a recent prediction [94] that rapid loss from collision
complexes would result in an effective loss rate described by $y=0.25$.

Figure 7: The number of RbCs molecules remaining in
$(n=0,m_{f,\mathrm{RbCs}}=5)_{0}$ after a fixed hold time with Rb atoms in the
($f_{\mathrm{Rb}}=1,m_{f,\mathrm{Rb}}=1$) state (upper panels) and Cs atoms in
the $(f_{\mathrm{Cs}}=3,m_{f,\mathrm{Cs}}=3)$ state (lower panels) at 21.3 G
(left panels) and 181.6 G (right panels). In each case the magnetic field is
scanned $\pm 5$ G around the centre field $B_{0}$ in $\sim 0.1$ G steps. The
atom-molecule mixture is held in the trap for 10 ms (6 ms for the data with Rb
at $\sim 181.6$ G). The mean value for each panel is shown as a solid
horizontal line, 1$\sigma$ and 2$\sigma$ intervals are shown as dashed and
dotted lines respectively. The data are also shown as a histogram and compared
to a normal distribution, normalised to give the probability density $p$. We
see no significant variation from normally distributed noise.

Finally, motivated by the observation of Feshbach resonances in 23Na40K + 40K
collisions [34], we check for the possibility of resonant behaviour close to
the magnetic fields at which we perform our experiments. To do this, we hold
the atom-molecule mixture in the trap for a fixed period of time (6 to 10 ms)
and vary the magnetic field by $\pm 5$ G in steps of $\sim 0.1$ G, as shown in
Fig. 7 for molecules prepared in $(0,5)_{0}$. To determine if there is any
variation from background we make a histogram of the results and extract a
mean and standard deviation. For all four measurements we observe that only
$\sim 5\%$ of the points are outside the interval defined by the mean $\pm
2\sigma$, as would be expected for normally distributed noise. Additionally,
we see no obvious large dips in the molecule number that would indicate the
presence of resonances.

###  2.3 RbCs + Cs in an intensity-modulated trap

It is possible that the dominant loss process for atom + molecule collisions
is the same fast optical excitation of two-body collision complexes that we
have observed for molecule + molecule collisions [60, 72]. In the RRKM limit,
the laser-free lifetime of the RbCs + Cs collision complex in the dark is
expected to be a factor of $2\times 10^{4}$ shorter [69, 95] than the lifetime
of $\sim 0.5$ ms for the (RbCs)2 complex. If the lifetime of the atom-molecule
complex is so short, we may expect that there is not enough time for
significant laser excitation before the complex dissociates, or that at least
this process may not be saturated. However, recent experiments [77] studying
nonreactive collisions of KRb with Rb found a photon-free lifetime for the
complex of 0.39(6) ms, which is 5 orders of magnitude longer than the RRKM
prediction, along with evidence for loss associated with optical excitation of
the complexes. The reason for such long-lived complexes in KRb+Rb and whether
optical excitation of complexes can contribute significantly to loss in
collisions between other combinations of atoms and molecules are open
questions.

Figure 8: Molecule loss in a mixture of RbCs in the
$(n=0,m_{f,\mathrm{RbCs}}=5)_{0}$ state and Cs in the
$(f_{\mathrm{Cs}}=3,m_{f,\mathrm{Cs}}=3)$ state at 181.6 G confined in CW
(filled circles) and intensity-modulated (empty circles) traps. The lines show
fitted exponential decay curves with $1/e$ lifetimes of 17.4(1.2) ms in the CW
trap and 21.0(1.5) ms in the modulated trap.

To investigate this possibility, we have measured the lifetime of RbCs
molecules, in the ground state $(0,5)_{0}$, in the presence of Cs atoms, with
and without 1 kHz square-wave intensity modulation, in the $\lambda=1064$ nm
optical trap described in section 1. As for the experiments on RbCs alone, the
square-wave modulation is set such that the atom-molecule mixture spends 75%
of each cycle in the dark. The peak intensity in the modulated trap is 4 times
that of the CW trap, so that the trap frequencies and trap depths are the same
for both trap configurations.

A comparison of the molecule lifetimes for the CW and modulated traps is shown
in Fig. 8. We find two-body rate coefficients of
$k_{\mathrm{mod}}=2.1(6)\times 10^{-11}$ cm3 s-1 for the modulated trap, and
$k_{\mathrm{CW}}=2.6(6)\times 10^{-11}$ cm3 s-1 for the CW trap. As such,
there is no statistically significant difference in the loss rates.

Figure 9: Suppression of the loss of molecules due to atom-molecule collisions
in an intensity-modulated trap. We model the variation in the molecular
density as described by Eq. 11, and extract the ratio of loss rates for CW and
modulated traps $(k_{\mathrm{CW}}/k_{\mathrm{mod}}-1)$. As presented, our
results are independent of the values of $k^{\mathrm{a,m}}_{2}$ and
$n_{\mathrm{a}}$ used in the model. The modulation frequency of the trap is
fixed at 1 kHz. We assume no inelastic decay such that the lifetime of the
complex $\tau_{\mathrm{c}}=\tau_{-1}$. (a) Suppression of loss for unsaturated
optical excitation of complexes as a function of
$k_{\mathrm{laser}}I_{\mathrm{avg}}\tau_{-1}$, in the limit that $\tau_{-1}\ll
t_{\mathrm{dark}}$. (b) Suppression of loss for saturated optical excitation
of complexes as a function of $\tau_{-1}$, in the limit that
$k_{\mathrm{I}}I_{\mathrm{avg}}\gg 1/\tau_{-1}$. For both calculations, the
experimentally measured ratio $k_{\mathrm{CW}}/k_{\mathrm{mod}}-1=0.2(5)$ is
shown by the horizontal solid line, with the $1\sigma$ uncertainty indicated
by the dotted line. The shaded regions show the parameter space which is
consistent with our experiments at the 68% confidence level.

To examine our results in the context of optical excitation of atom+molecule
complexes, we construct the rate-equation model

$\dot{n}_{\mathrm{m}}=-k^{\mathrm{a,m}}_{2}n_{\mathrm{a}}n_{\mathrm{m}}+\frac{1}{\tau_{\mathrm{-1}}}n_{\mathrm{c}},\\\
\dot{n}_{c}=+k^{\mathrm{a,m}}_{2}n_{\mathrm{a}}n_{\mathrm{m}}-\frac{1}{\tau_{\mathrm{-1}}}n_{\mathrm{c}}-k_{\mathrm{laser}}I(t)n_{\mathrm{c}}.$
(11)

This describes the loss of molecule density due to the atom-molecule
collisions and the dissociation and optical excitation of the associated atom-
molecule collision complexes. We neglect the formation (and loss) of complexes
resulting from molecule-molecule collisions owing to the much longer
associated timescale. Additionally, there are no inelastic decay channels for
the state combination used in the measurement, such that the lifetime of the
complex $\tau_{\mathrm{c}}=\tau_{-1}$. The predictions of this model are shown
in Fig. 9 and lead to two interpretations of the experimental observations.

In Fig. 9(a) we show the predicted ratio
$(k_{\mathrm{CW}}/k_{\mathrm{mod}}-1)$ as a function of
$k_{\mathrm{laser}}I_{\mathrm{avg}}\tau_{-1}$ under the assumption that the
lifetime of the complex is much shorter than the trap dark time ($\tau_{-1}\ll
t_{\mathrm{dark}}$) so that the suppression of loss in the modulated trap is
independent of the trap modulation frequency. Here $I_{\mathrm{avg}}$ is the
average trap intensity experienced by the molecules in either trap. For
modulation where the trap light is off for 75% of each cycle, we expect
$k_{\mathrm{CW}}/4<k_{\mathrm{mod}}<k_{\mathrm{CW}}$ such that the ratio
$(k_{\mathrm{CW}}/k_{\mathrm{mod}}-1)$ can take values between 0 and 3. The
ratio extracted from our experiments is
$(k_{\mathrm{CW}}/k_{\mathrm{mod}}-1)=0.2(5)$. This is indicated by the
horizontal solid and dashed lines in Fig. 9. The grey shaded region in Fig.
9(a) indicates the range $k_{\mathrm{laser}}I_{\mathrm{avg}}\tau_{-1}<0.3$
consistent with our experimental results at the 68% confidence level. This
corresponds to our results being consistent with optical excitation that is
not saturated. This could be due to the predicted short lifetime of the
complex, but also depends on the unknown laser scattering rate for the atom-
molecule complexes.

In Fig. 9(b) we examine an alternative interpretation of the experimental
results, namely that the lack of suppression in the loss results from the
formation of complexes with long photon-free lifetimes. In this case, we
assume the optical excitation of the complexes is saturated
$k_{\mathrm{laser}}I_{\mathrm{avg}}\gg 1/\tau_{-1}$. Little or no suppression
can then occur if the dark time is not sufficiently long for a significant
number of complexes to decay before the next bright trap pulse. Fig. 9(b)
shows the prediction of our model in this limit as a function of $\tau_{-1}$.
Our results are consistent with a lifetime of the complex $\tau_{-1}>0.4$ ms,
again at the 68% confidence level. This would be similar to the recently
measured lifetime for complexes in KRb+Rb collisions [77], but is 5 orders of
magnitude longer than the RRKM prediction.

##  3 Conclusion

We have studied collisional loss of optically trapped RbCs molecules alone and
in mixtures with Rb and Cs atoms. For RbCs molecules alone, we have
demonstrated that collisional loss may be partially suppressed by modulating
the intensity of the optical trap, such that the molecules spend 75% of each
modulation cycle in the dark. For molecules in the spin-stretched absolute
ground state, the results confirm that optical excitation of long-lived two-
molecule complexes plays a dominant role in the collisional loss. However, the
suppression is diminished for molecules in higher-energy hyperfine states.
This may indicate changes in the effective density of states or competition
from other collisional loss mechanisms such as spin exchange.

For RbCs molecules prepared in either RbCs+Rb or RbCs+Cs mixtures, we have
demonstrated that the collisional loss shows second-order kinetics and we have
extracted two-body loss rate coefficients. We have interpreted these using a
model based on quantum defect theory. The resulting loss parameters are well
below the universal limit, and are similar for reactive collisions with Rb and
nonreactive collisions with Cs. For the nonreactive collisions, we have
compared the loss rate for mixtures in CW and modulated traps. We observe no
significant change in the loss rate associated with the trap light being
switched off.

Understanding ultracold molecular collisions with diatomic molecules remains
an important frontier for quantum state-controlled chemistry, with many
fundamental questions currently unanswered. The inherent complexity of these
systems presents many challenges ahead.

## Acknowledgements

This work was supported by U.K. Engineering and Physical Sciences Research
Council (EPSRC) Grants EP/P01058X/1 and EP/P008275/1.

## References

## References

  * [1] Demille D 2002 Phys. Rev. Lett. 88 067901
  * [2] Yelin S F, Kirby K and Côté R 2006 Phys. Rev. A 74 050301
  * [3] Zhu J, Kais S, Wei Q, Herschbach D and Friedrich B 2013 J. Chem. Phys. 138 024104
  * [4] Herrera F, Cao Y, Kais S and Whaley K B 2014 New J. Phys. 16 075001
  * [5] Ni K K, Rosenband T and Grimes D D 2018 Chem. Sci. 9 6830–6838
  * [6] Sawant R, Blackmore J A, Gregory P D, Mur-Petit J, Jaksch D, Aldegunde J, Hutson J M, Tarbutt M R and Cornish S L 2020 New J. Phys. 22 013027
  * [7] Hughes M, Frye M D, Sawant R, Bhole G, Jones J A, Cornish S L, Tarbutt M R, Hutson J M, Jaksch D and Mur-Petit J 2020 Phys. Rev. A 101 062308
  * [8] Barnett R, Petrov D, Lukin M and Demler E 2006 Phys. Rev. Lett. 96 190401
  * [9] Micheli A, Brennen G K and Zoller P 2006 Nat. Phys. 2 341
  * [10] Büchler H P, Demler E, Lukin M, Micheli A, Prokof’ev N, Pupillo G and Zoller P 2007 Phys. Rev. Lett. 98 060404
  * [11] Gorshkov A V, Manmana S R, Chen G, Demler E, Lukin M D and Rey A M 2011 Phys. Rev. A 84 033619
  * [12] Macià A, Hufnagl D, Mazzanti F, Boronat J and Zillich R E 2012 Phys. Rev. Lett. 109 235307
  * [13] Manmana S R, Stoudenmire E M, Hazzard K R A, Rey A M and Gorshkov A V 2013 Phys. Rev. B 87 081106
  * [14] Gorshkov A V, Hazzard K R A and Rey A M 2013 Mol. Phys. 111 1908–1916
  * [15] Heazlewood B R and Softley T P 2021 Nature Reviews Chemistry 5 125–140 ISSN 2397-3358
  * [16] Krems R V 2005 International Reviews in Physical Chemistry 24 99–118
  * [17] Krems R V 2008 Physical Chemistry Chemical Physics 10 4079
  * [18] Balakrishnan N 2016 Journal of Chemical Physics 145 150901
  * [19] DeMille D, Doyle J M and Sushkov A O 2017 Science 357 990–994
  * [20] Safronova M S, Budker D, DeMille D, Kimball D F J, Derevianko A and Clark C W 2018 Rev. Mod. Phys. 90(2) 025008
  * [21] Tarbutt M R, Sauer B E, Hudson J J and Hinds E A 2013 New Journal of Physics 15 053034
  * [22] Aggarwal P, Bethlem H L, Borschevsky A, Denis M, Esajas K, Haase P A B, Hao Y, Hoekstra S, Jungmann K, Meijknecht T B, Mooij M C, Timmermans R G E, Ubachs W, Willmann L, Zapara A and collaboration T N e 2018 The European Physical Journal D 72 197 ISSN 1434-6079
  * [23] Hutzler N R 2020 Quantum Science and Technology 5 044011
  * [24] Augenbraun B L, Lasner Z D, Frenett A, Sawaoka H, Miller C, Steimle T C and Doyle J M 2020 New Journal of Physics 22 022003
  * [25] Zelevinsky T, Kotochigova S and Ye J 2008 Phys. Rev. Lett. 100 043201
  * [26] Schiller S, Bakalov D and Korobov V 2014 Phys. Rev. Lett. 113 023004
  * [27] Ni K K, Ospelkaus S, de Miranda M H G, Pe’er A, Neyenhuis B, Zirbel J J, Kotochigova S, Julienne P S, Jin D S and Ye J 2008 Science 322 231–235
  * [28] Takekoshi T, Reichsöllner L, Schindewolf A, Hutson J M, Le Sueur C R, Dulieu O, Ferlaino F, Grimm R and Nägerl H C 2014 Physical Review Letters 113 205301
  * [29] Molony P K, Gregory P D, Ji Z, Lu B, Köppinger M P, Le Sueur C R, Blackley C L, Hutson J M and Cornish S L 2014 Physical Review Letters 113(25) 255301
  * [30] Park J W, Will S A and Zwierlein M W 2015 Physical Review Letters 114 205302
  * [31] Guo M, Zhu B, Lu B, Ye X, Wang F, Vexiau R, Bouloufa-Maafa N, Quéméner G, Dulieu O and Wang D 2016 Physical Review Letters 116 205303
  * [32] Rvachov T M, Son H, Sommer A T, Ebadi S, Park J J, Zwierlein M W, Ketterle W and Jamison A O 2017 Physical Review Letters 119 143001
  * [33] Seeßelberg F, Buchheim N, Lu Z K, Schneider T, Luo X Y, Tiemann E, Bloch I and Gohle C 2018 Physical Review A 97 013405
  * [34] Yang H, Zhang D C, Liu L, Liu Y X, Nan J, Zhao B and Pan J W 2019 Science 363 261–264
  * [35] Voges K K, Gersema P, Meyer zum Alten Borgloh M, Schulze T A, Hartmann T, Zenesini A and Ospelkaus S 2020 Physical Review Letters 125(8) 083401
  * [36] Cairncross W B, Zhang J T, Picard L R B, Yu Y, Wang K and Ni K K 2021 Phys. Rev. Lett. 126(12) 123402
  * [37] Shuman E S, Barry J F and DeMille D 2010 Nature 467 820–823
  * [38] Hummon M T, Yeo M, Stuhl B K, Collopy A L, Xia Y and Ye J 2013 Physical Review Letters 110 143001
  * [39] Zhelyazkova V, Cournol A, Wall T E, Matsushima A, Hudson J J, Hinds E A, Tarbutt M R and Sauer B E 2014 Physical Review A 89 053416
  * [40] Barry J F, McCarron D J, Norrgard E B, Steinecker M H and DeMille D 2014 Nature 512 286–289
  * [41] McCarron D J, Norrgard E B, Steinecker M H and DeMille D 2015 New Journal of Physics 17 035014
  * [42] Norrgard E B, McCarron D J, Steinecker M H, Tarbutt M R and DeMille D 2016 Physical Review Letters 116 063004
  * [43] Kozyryev I, Baum L, Matsuda K, Augenbraun B L, Anderegg L, Sedlack A P and Doyle J M 2017 Physical Review Letters 118(17) 173201
  * [44] Truppe S, Williams H J, Hambach M, Caldwell L, Fitch N J, Hinds E A, Sauer B E and Tarbutt M R 2017 Nature Physics 13 1173–1176
  * [45] Lim J, Almond J R, Trigatzis M A, Devlin J A, Fitch N J, Sauer B E, Tarbutt M R and Hinds E A 2018 Physical Review Letters 120(12) 123201
  * [46] Anderegg L, Augenbraun B L, Bao Y, Burchesky S, Cheuk L W, Ketterle W and Doyle J M 2018 Nature Physics 14 890–893
  * [47] Ding S, Wu Y, Finneran I A, Burau J J and Ye J 2020 Phys. Rev. X 10(2) 021049
  * [48] Zhu B, Gadway B, Foss-Feig M, Schachenmayer J, Wall M L, Hazzard K R A, Yan B, Moses S A, Covey J P, Jin D S, Ye J, Holland M and Rey A M 2014 Phys. Rev. Lett. 112(7) 070404
  * [49] Karman T and Hutson J M 2018 Phys. Rev. Lett. 121(16) 163401
  * [50] Lassablière L and Quéméner G 2018 Phys. Rev. Lett. 121(16) 163402
  * [51] Karman T and Hutson J M 2019 Phys. Rev. A 100(5) 052704
  * [52] Matsuda K, De Marco L, Li J R, Tobias W G, Valtolina G, Qu’emémener G and Ye J 2020 Science 370(6522) 1324–1327
  * [53] William J R, Tobias W G, Matsuda K, Miller C, Valtolina G, De Marco L, Wang R R W, Lassabliére L, Quéméner G, Bohn J L and Ye J 2021 arXiv:2103.06246
  * [54] Anderegg L, Burchesky S, Bao Y, Yu S S, Karman T, Chae E, Ni K K, Ketterle W and Doyle J M 2021 arXiv:2102.04365
  * [55] Ospelkaus S, Ni K K, Wang D, de Miranda M H G, Neyenhuis B, Quéméner G, Julienne P S, Bohn J L, Jin D S and Ye J 2010 Science 327 853–857
  * [56] Ni K K, Ospelkaus S, Wang D, Quéméner G, Neyenhuis B, de Miranda M H G, Bohn J L, Ye J and Jin D S 2010 Nature 464 1324–1328
  * [57] Cheuk L W, Anderegg L, Bao Y, Burchesky S, Yu S S, Ketterle W, Ni K K and Doyle J M 2020 Phys. Rev. Lett. 125(4) 043401
  * [58] Ye X, Guo Guo M, González-Martínez M L, Quéméner G and Wang D 2018 Science Advances 4 eaaq0083
  * [59] Guo M, Ye X, He J, González-Martínez M L, Vexiau R, Quéméner G and Wang D 2018 Phys. Rev. X 8 041044
  * [60] Gregory P D, Frye M D, Blackmore J A, Bridge E M, Sawant R, Hutson J M and Cornish S L 2019 Nature Communications 10 3104
  * [61] Gersema P, Voges K K, Meyer zum Alten Borgloh M, Koch L, Hartmann T, Zenesini A, Ospelkaus S, Lin J, He J and Wang D 2021 arXiv:2103.00510
  * [62] He J, Ye X, Lin J, Guo M, Quéméner G and Wang D 2021 Phys. Rev. Research 3(1) 013016
  * [63] Bause R, Schindewolf A, Tao R, Duda M, Chen X Y, Quéméner G, Karman T, Christianen A, Bloch I and Luo X Y 2021 Phys. Rev. Research 3 033013
  * [64] Żuchowski P S and Hutson J M 2010 Phys. Rev. A 81(6) 060703
  * [65] Idziaszek Z and Julienne P S 2010 Phys. Rev. Lett. 104 113202
  * [66] Frye M D, Julienne P S and Hutson J M 2015 New J. Phys. 17 045019
  * [67] Mayle M, Ruzic B P and Bohn J L 2012 Phys. Rev. A 85 062712
  * [68] Mayle M, Quéméner G, Ruzic B P and Bohn J L 2013 Phys. Rev. A 87 012709
  * [69] Christianen A, Karman T and Groenenboom G C 2019 Phys. Rev. A 100 032708
  * [70] Christianen A, Zwierlein M W, Groenenboom G C and Karman T 2019 Phys. Rev. Lett. 123 123402
  * [71] Levine R D 2005 Molecular Reaction Dynamics (Cambridge University Press)
  * [72] Gregory P D, Blackmore J A, Bromley S L and Cornish S L 2020 Physical Review Letters 124(16) 163402
  * [73] Liu Y, Hu M G, Nichols M A, D G D, Karman T, Guo H and Ni K K 2020 Nature Physics 16 1132
  * [74] Son H, Park J J, Ketterle W and Jamison A O 2020 Nature 580 197–200
  * [75] Jurgilas S, Chakraborty A, Rich C J H, Caldwell L, Williams H J, Fitch N J, Sauer B E, Frye M D, Hutson J M and Tarbutt M R 2021 Phys. Rev. Lett. 126(15) 153401
  * [76] Jurgilas S, Chakraborty A, Rich C J H, Sauer B E, Frye M D, Hutson J M and Tarbutt M R 2021 New J. Phys. 23 075004
  * [77] Nichols M A, Liu Y X, Zhu L, Hu M G, Liu Y and Ni K K 2021 arXiv:2105.14960
  * [78] of reactive collisions by quantum interference C 2021 arXiv:2109.03944v1
  * [79] Voges K K, Gersema P, Hartmann T, Ospelkaus S and Zenesini A 2021 arXiv:2109.03605v1
  * [80] Wang X Y, Frye M D, Su Z, Cao J, Liu L, Zhang D C, Yang H, Hutson J M, Zhao B, Bai C L and Pan J W 2021 arXiv:2103.07130v1
  * [81] Yang H, Wang X Y, Su Z, Cao J, Zhang D C, Rui J, Zhao B, Bai C L and Pan J W 2021 arXiv:2104.11424v1
  * [82] McCarron D J, Cho H W, Jenkin D L, Köppinger M P and Cornish S L 2011 Physical Review A 84 011603(R)
  * [83] Köppinger M P, McCarron D J, Jenkin D L, Molony P K, Cho H W, Cornish S L, Le Sueur C R, Blackley C L and Hutson J M 2014 Physical Review A 89 033604
  * [84] Gregory P D, Molony P K, Köppinger M P, Kumar A, Ji Z, Lu B, Marchant A L and Cornish S L 2015 New Journal of Physics 17 055006
  * [85] Molony P K, Gregory P D, Kumar A, Le Sueur C R, Hutson J M and Cornish S L 2016 ChemPhysChem. 17 3811–3817
  * [86] Schnelle S K, van Ooijen E D, Davis M J, Heckenberg N R and Rubinsztein-Dunlop H 2008 Opt. Express 16 1405–1412
  * [87] Henderson K, Ryu C, MacCormick C and Boshier M G 2009 New J. Phys. 11 043030
  * [88] Roberts K O, McKellar T, Fekete J, Rakonjac A, Deb A B and Kjærgaard N 2014 Opt. Lett. 39 2012
  * [89] Gregory P D, Aldegunde J, Hutson J M and Cornish S L 2016 Physical Review A 94 041403(R)
  * [90] Guttridge A, Hopkins S A, Kemp S L, Frye M D, Hutson J M and Cornish S L 2017 Phys. Rev. A 96(1) 012704
  * [91] Gao B 1998 Phys. Rev. A 58 1728–1734
  * [92] Gao B 2008 Phys. Rev. A 78 012702
  * [93] Żuchowski P, Kosicki M, Kodrycka M and Soldán P 2013 Phys. Rev. A 87 022706
  * [94] Christianen A, Groenenboom G C and Karman T 2021 arXiv:2108.072724v1
  * [95] Frye M D and Hutson J M 2021 arXiv:2109.07435


# Critical Scenario Generation for Developing Trustworthy Autonomy

Wenhao Ding
Carnegie Mellon University

##  I Motivation and Problem Statement

Autonomous systems, such as self-driving vehicles, quadrupeds, and robot
manipulators, are largely enabled by the rapid development of artificial
intelligence. However, such systems involve several trustworthy challenges
such as safety, robustness, and generalization, due to their deployment in
open-ended and real-time environments. To evaluate and improve
trustworthiness, simulations or so-called digital twins are largely utilized
for system development with low cost and high efficiency. One important thing
in virtual simulations is scenarios that consist of static and dynamic
objects, specific tasks, and evaluation metrics.

However, designing diverse, realistic, and effective scenarios is still a
challenging problem. One straightforward way is creating scenarios through
human design, which is time-consuming and limited by the experience of experts
[21]. Another method commonly used in self-driving areas is log replay. This
method collects scenario data in the real world and then replays it in
simulations [23] or adds random perturbations [1, 25]. Although the replay
scenarios are realistic, most of the collected scenarios are redundant since
they are all ordinary scenarios that only consider a small portion of critical
cases. The desired scenarios should cover all cases in the real world,
especially rare but critical events with extremely low probability. We provide
three examples in Figure 1: safety-critical accident for self-driving vehicles
[4], robustness-critical terrain for quadrupedal robots [16], and
generalization-critic tasks for robot arm [13]. As shown in the left part of
Figure 2, ordinary scenarios have a high probability happen but are usually
redundant for training learning-based algorithms. Critical scenarios are rare
but important to test autonomous systems under risky conditions and
unpredictable perturbations, which reveal their trustworthiness [22].

There are 5 aspects we should consider for generating critical scenarios:

  * •

Reality. Since the ultimate goal is to deploy autonomous systems in the real
world, we should ensure the scenarios have the chance to happen in real
situations.

  * •

Efficiency. Critical scenarios are usually rare in the real world. The
generation needs to consider the efficiency and increase the density of
critical scenarios.

  * •

Diversity. Critical scenarios are diverse. The generation algorithm should be
able to discover and generate as many different critical scenarios as
possible.

  * •

Transferability. Scenarios are dynamic due to the interaction between
autonomous systems and their surrounding objects. The scenarios we generate
should be adaptive to different autonomous systems.

  * •

Controllability. We may want to reproduce or repeat one type of critical
scenario rather than random ones. The generation method should be able to
follow instructions or conditions to generate corresponding scenarios.

My research objective is to generate critical scenarios in simulations to help
evaluate and improve the trustworthiness of autonomy, including but not
limited to safety, robustness, generalization, privacy, fairness, and
security. In particular, I leverage the huge success of deep generative models
[20, 19] and the increased scalability of datasets in robotics areas, for
example, self-driving [2] and manipulation task [14]. My research is built
upon multiple machine-learning tools, including adversarial examples,
reinforcement learning, and causal discovery. Additionally, recent powerful
simulation and digital twin platforms [12, 17] almost bridge the gap between
simulation and the real world, which enables the fidelity of scenarios to the
greatest extent.

Figure 1: Examples of critical scenarios in autonomy. Figure 2: Left: Critical
scenarios have a low probability to happen but are important for improving
trustworthiness Middle: My contributed approaches can be divided into three
categories. Right: Analysis of three types of methods from five perspectives.

##  II Contributed Approaches

My previous research has contributed several important approaches toward the
critical scenario generation area [10]. These approaches fall into three
categories according to the source of the information for generations, i.e.,
data-driven generation, adversarial generation, and knowledge-based generation
(shown in the middle of Figure 2).

###  II-A Data-driven Generation

We first consider algorithms that only leverage information from datasets. For
example, we control several vehicles to pass an intersection by making them
follow the trajectories recorded in the same intersection in the real world.
We can use density estimation models $p_{\theta}(x)$ parameterized by $\theta$
to learn the distribution of scenarios, which enables the generation of unseen
scenarios, where $x$ represents the general scenario. I select Variational
Auto-encoder (VAE) [15] as the estimation model, which has an encoder $q(z|x)$
and a decoder $p(x|z)$. The encoder projects the scenario data to a low-
dimensional latent space, where we can randomly draw latent codes. These codes
are then converted back to scenarios by using the decoder [3]. This framework
provides a simple way to generate new scenarios but still has two drawbacks:
(1) we do not know the critical level of generated scenarios (2) generated
scenarios are not conditioned on road maps. To solve these two problems, I
propose a new framework [5] to learn the joint distribution of ordinary data
and collision data. After training, we can do linear interpolation in the
latent space to control the critical level of generation scenarios. The model
also takes road maps as conditions to force scenarios to satisfy constraints.

###  II-B Adversarial Generation

We then consider a more efficient way for generation, which actively creates
critical scenarios by attacking autonomy. For example, we control the
steepness of the terrain to make the quadrupedal robot fail in locomotion
tasks. Although robots learn to avoid falling into the ground, the scenario
generation methods can gradually increase the difficulty to attack the robot.
This framework, named adversarial generation, is essentially a game between
two components, one is the generator, and the other is the robot. We propose a
Reinforcement Learning method in [4] to maximize the collision rate between a
self-driving vehicle and cyclists. This simple framework efficiently generates
the position and velocity of cyclists to make self-driving vehicles fail.
However, scenarios generated by this method usually lack diversity and reality
due to the naive objective. In [6], we modify the adversarial generation
framework with two improvements. We leverage a multi-modal flow-based model
[11] to represent the distribution of the scenario and also train a prior
distribution of the real-world dataset as a constraint.

###  II-C Knowledge-based Generation

Notice that scenarios are constructed in the physical world, therefore, need
to satisfy physical laws and specific rules. The samples in the density we
estimate or the adversarial examples we generate could easily violate these
constraints. In addition, domain knowledge also improves the efficiency of the
generation. In my previous research, I consider knowledge injection from two
perspectives. The first one is designing a tree-structure representation of
the scenario and back-propagating knowledge from leaf nodes to the root node
[7], which forces the generated scenarios to satisfy given constraints. The
second method is from a causality perspective, which aims to analyze scenarios
and find the reasons that cause critical events. In the first step, I assume
the causality underlying the critical scenario is given as a graph. Then, I
propose an autoregressive generation model to efficiently generate diverse
critical scenarios guided by this causal graph [8]. Next, I investigate how to
automatically discover causal graphs from an interventional dataset and use
the discovered causality for better generation [9].

##  III Future Directions

Develop new generation methods. In the right part of Figure 2, I summarize the
trade-offs of three types of generation methods. None of them is perfect to
satisfy all requirements but the chance to combine the advantages of different
categories still exists. In particular, large models and datasets provide
powerful implicit neural models, while causal reasoning and knowledge
injection enable explicit symbolic learning. The combined neural-symbolic
system has great potential to improve current critical scenario generation
methods.

Joint learning of trustworthy agent and scenarios. The trustworthy problem
cannot be solved only by developing a perfect agent. All robots have
limitations and there always exist scenarios that cannot be solved. The
ultimate goal of critical scenario generation is not to make the robot fail
but to make it realize the boundary of feasible solutions. To this end, we can
focus on the joint learning of agents and scenarios, which leads to several
directions such as automatic curriculum learning [18] and adversarial training
[24].

Practical value of critical scenario generation. To push the academic research
results to real-world applications, I am also working on a benchmark [26] that
contains thousands of critical traffic scenarios for self-driving vehicles.
This benchmark includes several important scenario-generation methods for
evaluating object detection, trajectory prediction, and vehicle control
algorithms. This benchmark will be extended to other areas where
trustworthiness is also highly required as in self-driving.

## References

  * Bansal et al. [2018] Mayank Bansal, Alex Krizhevsky, and Abhijit Ogale.  Chauffeurnet: Learning to drive by imitating the best and synthesizing the worst.  _arXiv preprint arXiv:1812.03079_ , 2018.
  * Caesar et al. [2020] Holger Caesar, Varun Bankiti, Alex H Lang, Sourabh Vora, Venice Erin Liong, Qiang Xu, Anush Krishnan, Yu Pan, Giancarlo Baldan, and Oscar Beijbom.  nuscenes: A multimodal dataset for autonomous driving.  In _Proceedings of the IEEE/CVF conference on computer vision and pattern recognition_ , pages 11621–11631, 2020.
  * Ding et al. [2019] Wenhao Ding, Wenshuo Wang, and Ding Zhao.  A new multi-vehicle trajectory generator to simulate vehicle-to-vehicle encounters.  In _International Conference on Robotics and Automation (ICRA)_. IEEE, 2019.
  * Ding et al. [2020a] Wenhao Ding, Baiming Chen, Minjun Xu, and Ding Zhao.  Learning to collide: An adaptive safety-critical scenarios generating method.  In _IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS)_. IEEE, 2020a.
  * Ding et al. [2020b] Wenhao Ding, Mengdi Xu, and Ding Zhao.  Cmts: Conditional multiple trajectory synthesizer for generating safety-critical driving scenarios.  In _International Conference on Robotics and Automation (ICRA)_. IEEE, 2020b.
  * Ding et al. [2021a] Wenhao Ding, Baiming Chen, Bo Li, Kim Ji Eun, and Ding Zhao.  Multimodal safety-critical scenarios generation for decision-making algorithms evaluation.  _IEEE Robotics and Automation Letters_ , 2021a.
  * Ding et al. [2021b] Wenhao Ding, Haohong Lin, Bo Li, Kim Ji Eun, and Ding Zhao.  Semantically adversarial driving scenario generation with explicit knowledge integration.  _arXiv preprint arXiv:2106.04066_ , 2021b.
  * Ding et al. [2022a] Wenhao Ding, Haohong Lin, Bo Li, and Ding Zhao.  Causalaf: Causal autoregressive flow for safety-critical driving scenario generation.  In _6th Annual Conference on Robot Learning_ , 2022a.
  * Ding et al. [2022b] Wenhao Ding, Haohong Lin, Bo Li, and Ding Zhao.  Generalizing goal-conditioned reinforcement learning with variational causal reasoning.  In _Neural Information Processing Systems (NeurIPS)_ , 2022b.
  * Ding et al. [2023] Wenhao Ding, Chejian Xu, Mansur Arief, Haohong Lin, Bo Li, and Ding Zhao.  A survey on safety-critical driving scenario generation—a methodological perspective.  _IEEE Transactions on Intelligent Transportation Systems_ , 2023.
  * Dinh et al. [2016] Laurent Dinh, Jascha Sohl-Dickstein, and Samy Bengio.  Density estimation using real nvp.  _arXiv preprint arXiv:1605.08803_ , 2016.
  * Dosovitskiy et al. [2017] Alexey Dosovitskiy, German Ros, Felipe Codevilla, Antonio Lopez, and Vladlen Koltun.  Carla: An open urban driving simulator.  In _Conference on robot learning_ , pages 1–16. PMLR, 2017.
  * Fang et al. [2021] Kuan Fang, Yuke Zhu, Silvio Savarese, and Li Fei-Fei.  Discovering generalizable skills via automated generation of diverse tasks.  _arXiv preprint arXiv:2106.13935_ , 2021.
  * James et al. [2020] Stephen James, Zicong Ma, David Rovick Arrojo, and Andrew J Davison.  Rlbench: The robot learning benchmark & learning environment.  _IEEE Robotics and Automation Letters_ , 5(2):3019–3026, 2020.
  * Kingma and Welling [2013] Diederik P Kingma and Max Welling.  Auto-encoding variational bayes.  _arXiv preprint arXiv:1312.6114_ , 2013.
  * Made Aswin Nahrendra et al. [2023] I Made Aswin Nahrendra, Byeongho Yu, and Hyun Myung.  Dreamwaq: Learning robust quadrupedal locomotion with implicit terrain imagination via deep reinforcement learning.  _arXiv e-prints_ , pages arXiv–2301, 2023.
  * Makoviychuk et al. [2021] Viktor Makoviychuk, Lukasz Wawrzyniak, Yunrong Guo, Michelle Lu, Kier Storey, Miles Macklin, David Hoeller, Nikita Rudin, Arthur Allshire, Ankur Handa, et al.  Isaac gym: High performance gpu-based physics simulation for robot learning.  _arXiv preprint arXiv:2108.10470_ , 2021.
  * Narvekar et al. [2020] Sanmit Narvekar, Bei Peng, Matteo Leonetti, Jivko Sinapov, Matthew E Taylor, and Peter Stone.  Curriculum learning for reinforcement learning domains: A framework and survey.  _The Journal of Machine Learning Research_ , 21(1):7382–7431, 2020.
  * Ramesh et al. [2022] Aditya Ramesh, Prafulla Dhariwal, Alex Nichol, Casey Chu, and Mark Chen.  Hierarchical text-conditional image generation with clip latents.  _arXiv preprint arXiv:2204.06125_ , 2022.
  * Roose [2022] K Roose.  The brilliance and weirdness of chatgpt.  _The New York Times_ , 2022.
  * [21] German Ros, Vladlen Koltun, Felipe Codevilla, and Antonio Lopez.  Carla autonomous driving challenge 2022.  [EB/OL].  https://leaderboard.carla.org/challenge/.
  * Salem et al. [2015] Maha Salem, Gabriella Lakatos, Farshid Amirabdollahian, and Kerstin Dautenhahn.  Towards safe and trustworthy social robots: ethical challenges and practical issues.  In _Social Robotics: 7th International Conference, ICSR 2015, Paris, France, October 26-30, 2015, Proceedings 7_ , pages 584–593. Springer, 2015\.
  * Scanlon et al. [2021] John M Scanlon, Kristofer D Kusano, Tom Daniel, Christopher Alderson, Alexander Ogle, and Trent Victor.  Waymo simulated driving behavior in reconstructed fatal crashes within an autonomous vehicle operating domain.  _Accident Analysis & Prevention_, 163:106454, 2021.
  * Shafahi et al. [2019] Ali Shafahi, Mahyar Najibi, Mohammad Amin Ghiasi, Zheng Xu, John Dickerson, Christoph Studer, Larry S Davis, Gavin Taylor, and Tom Goldstein.  Adversarial training for free!  _Advances in Neural Information Processing Systems_ , 32, 2019.
  * Tremblay et al. [2018] Jonathan Tremblay, Aayush Prakash, David Acuna, Mark Brophy, Varun Jampani, Cem Anil, Thang To, Eric Cameracci, Shaad Boochoon, and Stan Birchfield.  Training deep networks with synthetic data: Bridging the reality gap by domain randomization.  In _Proceedings of the IEEE conference on computer vision and pattern recognition workshops_ , pages 969–977, 2018.
  * Xu et al. [2022] Chejian Xu, Wenhao Ding, Weijie Lyu, Zuxin Liu, Shuai Wang, Yihan He, Hanjiang Hu, Ding Zhao, and Bo Li.  Safebench: A benchmarking platform for safety evaluation of autonomous vehicles.  In _Neural Information Processing Systems (NeurIPS)_ , 2022.