# Anti-Zeno cooling of spin baths by quantum probe measurements

Durga Bhaktavatsala Rao Dasari  3\. Physics Institute, University of
Stuttgart, Center for Applied Quantum Technologies, IQST, MPI for Solid State
Research, Stuttgart 70569, Germany.  Sen Yang  Department of Physics, The Hong
Kong University of Science and Technology, Clear Water Bay, Hong Kong, China.
Amit Finkler  Department of Chemical Physics, Weizmann Institute of Science,
Rehovot, Israel  Gershon Kurizki  Department of Chemical Physics, Weizmann
Institute of Science, Rehovot, Israel  Joerg Wrachtrup  3\. Physics Institute,
University of Stuttgart, Center for Applied Quantum Technologies, IQST, MPI
for Solid State Research, Stuttgart 70569, Germany.

(August 27, 2024)

###### Abstract

We put forth, theoretically and experimentally, the possibility of drastically
cooling down (purifying) thermal ensembles (“baths”) of solid-state spins via
a sequence of projective measurements of a probe spin that couples to the bath
in an arbitrary fashion. If the measurement intervals are chosen to correspond
to the anti-Zeno regime of the probe-bath exchange, then a short sequence of
measurements with selected outcomes is found to have an appreciable success
probability. Such a sequence is shown to condition the bath evolution so that
it can dramatically enhance the bath-state purity and yield a low-entropy
steady state of the bath. This purified bath state persists after the
measurements, and can be chosen, on demand, to allow for Zeno- or anti-Zeno-
like evolution of quantum systems coupled to the purified bath. The
experimental setup for observing these effects consists of a nitrogen vacancy
(NV) center in diamond at low temperature that acts as a probe of the
surrounding nuclear spin bath. The NV sigle-shot measurements are induced by
optical fields at microsecond intervals.

quantum measurements, spin-bath cooling, anti-Zeno effect

##  I Introduction

Quantum system interactions with their environment, alias a bath with many
degrees of freedom, are foundationally interesting since they chart the path
from quantum coherent dynamics to behavior described by statistical-mechanics
or thermodynamics as the complexity of the system-bath compound increases ref1
; ref2 ; ref3 ; ref4 ; ref5 ; ref6 ; ref7 ; ref8 ; ref9 . On the applied side,
adequate control over such interactions is a prerequisite for all emerging
quantum technologies ref10 ; ref11 ; ref12 ; ref13 ; ref14 ; ref16 ; ref17 ;
ref18 ; ref19 ; ref20 ; ref21 ; ref22 ; ref23 ; ref24 ; ref25 ; ref26 ; ref27
; ref28 ; ref29 ; ref30 ; ref31 ; ref32 . In particular, fault-tolerant
quantum computation ref11 ; ref12  requires the ability to protect the
information encoded in the computational qubits from leaking into the
environment (bath), thus causing their decoherence. The desirable, often
inachievable, goal is to completely decouple the system from the bath ref32 ;
ref33 ; ref34 ; ref35 ; ref36 ; ref37 ; ref38 ; ref39 . More realistically,
the adverse effects of decoherence can be suppressed by control pulses at a
rate faster than the inverse memory (correlation) time of the bath ref40 ;
ref41 ; ref42 ; ref43 ; ref44 ; ref45 ; ref46 . Yet, existing system-bath
interaction control is severely constrained by the paradigm whereby the bath
is immutable and control is only applicable to the system ref37 ; ref38 ;
ref39 ; ref40 ; ref41 ; ref42 ; ref43 . Correspondingly, theory commonly
resorts to master equations that describe only the dynamics of the system (its
reduced density matrix) ref2 ; ref40 ; ref46 . In such a description, the bath
can be emulated for most purposes by a classical noisy, fluctuating field. The
justification is that in typical experimental settings one can only achieve a
high degree of control and readout precision of the system, but not of the
bath.

A major stride beyond this paradigm towards deeper understanding of quantum
aspects of system-bath interactions has been the discovery that the Quantum
Zeno effect (QZE) ref47 ; ref48 ; ref49  and its inverse, the Anti-Zeno (AZE)
effect, can profoundly effect open quantum systems ref50 ; ref51 ; ref52 ;
ref53 ; ref54 . These effects occur on time-scales such that system-bath
interaction dynamics may allow for partial revival of the system coherence
caused by energy and entropy backflow from the bath to the system. The most
familiar manifestations of the QZE or the AZE are, respectively, the slowdown
or speedup of the open system-state evolution when subjected to measurements
at appropriate rates ref50 ; ref51 ; ref52 . Both effects have been
extensively investigated and verified for a variety of systems ref46 ; ref55 ;
ref56 ; ref57 ; ref58 ; ref59 .

Not less significant has been the discovery that QZE dynamics can heat up a
qubit by equalizing the bath-induced upward and downward transition rates,
whereas less frequent measurements conforming to the AZE regime can cool the
qubit down by predominantly enhancing downward transitions (relaxation to the
ground state) ref60 ; ref61 . Such cooling and heating effects have been
experimentally verified by some of us for interacting spin-systems using
nuclear magnetic resonance (NMR) techniques for their quantum simulation ref62
.

Thus far, the study of the QZE and AZE has been focused only on the evolution
of the system, and an unaffected bath, a scenario relevant to weak system-bath
coupling, for which the second- order non-Markovian master equations reveal
the dynamics under repeated non-selective (unread) measurements (or phase
modulations) of the system ref46 ; ref50 ; ref51 ; ref60 ; ref61 ; ref62 .
Here, by contrast, we consider strong system-bath coupling for which a
selective (projective) measurement of the system can drastically disturb the
bath state depending on its readout outcome. This disturbance becomes stronger
as the bath gets smaller. In such cases one has to rely on the exact solution
of the Hamiltonian, which is obtainable for only a few system-bath interaction
models. We here theoretically investigate such a model of the dephasing of a
two-level system (probe spin) by a spin bath with conserved spin, and
experimentally realize it for a degenerate (non-interacting) spin bath.We show
how to employ the coupled system-bath evolution under consecutive selective
measurements to control (purify) the bath state. In turn, such a purified bath
state is shown to allow, on demand, either dramatically prolonged coherence
time of qubits in the QZE regime, or, conversely rapid oscillations of qubit
cohernece in the AZE regime. Measurement induced Quantum State Engineering of
spin and oscillator baths has been studied theoretically to generate exotic
entangled states ref63  and for ground state cooling/squeezing of cantilever
modes ref64 . While these studies only focus on the preparation of the target
state the role of QZE and AZE have not been explored in such preparations.

The concrete protocol consists in the preparation of the probe system in a
superposition of energy states and observation of the decay of this
superposition via the probe interaction with the adjacent spin bath. The idea
is to repeatedly check via frequent selective measurements whether or not the
probe is still in its initial state and thereby control the bath state.

Figure 1: Bath purification by conditional measurements and its posterior
(persistent) effects (a) Schematic description of our scheme: Conditional
measurements of the state of the central probe-spin (S) collapse the spin-bath
toward a low entropy state with resolvable spins. (b) An experimentally
obtained measurement sequence of the central spin displaying the outcomes of a
measurement, where $0$ corresponds to zero-photon and $1$ to one-photon
detection. (c) Modification of the bath spectrum before and after the
measurements. (d) Coherence decay of the probe electron-spin coupled to the
original (red) and purified (blue) spin-bath (experiment and theory).

##  II Model

We consider the dipolar interaction of an (effectively) spin-$1/2$ probe
system ($S$) with $N$ identical spin-$1/2$ systems (the bath – B). The
inherent quantization axis of S and the S-B resonance mismatch (detuning)
render their interaction dispersive, which leads to decoherence (pure
dephasing) of the $S$-spin. A common variant of this problem involves spin-
spin interactions in the bath ref44 . The dynamics generated by all such
Hamiltonians can be exactly solved for any distribution of S-B couplings and
any eigenvalue spectrum of B. Here we assume, for simplicity, that S is
coupled to a non-interacting spin bath B in the star configuration (Fig. 1a).
The Hamiltonian describing the S-B interaction is then given by

$\displaystyle H$ $\displaystyle=$ $\displaystyle S^{z}\otimes\hat{B},$ (1)
$\displaystyle\hat{B}$ $\displaystyle=$
$\displaystyle\sum_{k}\vec{g}_{k}\cdot\vec{I}_{k}.$

Here $S^{z}$ is the S-spin component along its quantization axis, k labels the
B-spins, $\vec{g}_{k}$ are the dipolar couplings between S and the $k$-th
nuclear spin, and $\vec{I}_{k}$ its spin (Pauli) operator. The typically
random spatial locations of the B-spins result in an inhomogeneous
distribution of the couplings $\vec{g}_{k}$ such that B has no preferred
spatial symmetries.

In the $S^{z}$ basis of S spanned by states ($|{e}\rangle,|{g}\rangle$), the
above Hamiltonian can be rewritten as

$\displaystyle H=\hat{B}_{e}{|e\rangle}{\langle
e|}+\hat{B}_{g}{|g\rangle}{\langle g|},\leavevmode\nobreak\
\leavevmode\nobreak\ \hat{B}_{e(g)}=\pm\sum_{k}\vec{g}_{k}(r)\cdot\vec{I}_{k}$
(2)

where $\hat{B}_{e(g)}$ are the spin-bath energy operators associated with the
energy states ${|e(g)\rangle}$ in S. The Hamiltonian in Eq. (2) gives rise in
the interaction picture to the closed-form time-evolution operator of the
“supersystem” $\bf{S+B}$

$U_{S+B}(t)=U_{e}(t)|e\rangle\langle e|\,+\,U_{g}(t)|{g}\rangle\langle g|$ (3)

where $U_{e(g)}(t)={\exp{(it\hat{B}_{e(g)})}}$ govern the dynamics of B in a
manner conditional on the state of $S$. In what follows we review standard
dephasing that assumes an unchanging statistical state of B and contrast it
with the conditional dynamics of B, which is shown to differ dramatically.

##  III Free induction decay and its Zeno or anti-Zeno modifications

In standard treatments, the effect of B on S is mimicked by a classical, noisy
(random) magnetic field, assuming the initial B-state to be completely
depolarized (mixed): the S-B interaction given above does not lead then to any
bath dynamics, i.e.,

$\rho_{B}(t)=\rho_{B}(0),\leavevmode\nobreak\ \leavevmode\nobreak\
\rho_{B}(0)=\sum_{m}P_{j}(0){|j\rangle}{\langle
j|},P_{j}(0)=\sqrt{\frac{\pi}{2N}}\exp{(-j^{2}/2N)}$ (4)

where $j$ labels the collective B-states. As the $S-B$ interaction causes pure
dephasing (decoherence) of $S$, the dynamics only occurs between superposition
states of $S$ in the $e(g)$ basis, say,
${|\pm\rangle}=\frac{1}{\sqrt{2}}[{|e\rangle}\pm{|g\rangle}]$. If we
initialize the probe in the ${|+\rangle}$ state, we can measure its free-
induction decay (FID) ref39 , caused by its mixing with the orthogonal
superposition state ${|-\rangle}$ via a bath-induced random phase. The
ensemble average of their populations corresponds to the probabilities of
measuring these states at time $t$, given by

$P_{+}(t)\equiv 1-P_{-}(t)=\frac{1}{2}[1+{\rm e}^{-(t/T_{2})^{2}}].$ (5)

Equivalently, the probe-spin coherence can be measured by the decaying mean
value of the ${\langle S^{x}\rangle}$ component of its spin, which displays
Gaussian decay, given by

${\langle S^{x}\rangle}(t)\approx{\rm e^{-(t/T_{2})^{2}}}.$ (6)

It is known that the dephasing time $T_{2}$ can be modified by dynamical
control of $S$ ref39 . A common (but not always optimal) example of such
control are dynamical - decoupling pulse sequences ref35 ; ref38 ; ref39 ;
ref65 ; ref66 . According to the Kofman-Kurizki (KK) universal formula ref40 ;
ref46 ; ref50 ; ref51 any dynamically-controlled decoherence of $S$ evolves as

${\langle S^{x}\rangle}(t)\sim\exp{\Big{[}-\int
G(\omega)F(t,\omega)d\omega\Big{]}}.$ (7)

Here $F(t,\omega)$ is the time-dependent control power-spectrum, which may
arise from unitary (pulsed or continuous) operations or from frequent
measurements and $G(\omega)$ is the bath-coupling spectrum. In principle, one
may adapt the $F(t,\omega)$ spectrum to the $G(\omega)$ spectrum so as to
either suppress or enhance the decoherence in Eq. (7) , corresponding to the
QZE or AZE regime, respectively, as is done in bath-optimized minimal energy
control (BOMEC) ref42 ; ref43 .

Here we set out to overcome the limitations of any form of conventional
dynamical control of $S$, be it DD or BOMEC: (i) the extremely high rate of
pulses required to effectively control decoherence in the high- temperature or
broad-band (white noise) limit of B, which are properties that cannot be
affected by conventional control; (ii) the need to maintain the dynamical
control during the time that other (quantum-logic or quantum- sensing)
operations are performed on the qubits of interest (although this difficulty
can be mitigated by appropriate control ref67 . To this end, we resort to
unconventional bath control via conditional measurements of the probe.

##  IV Bath dynamics caused by conditional measurements

Let us assume that $\tau$ is the interval between consecutive measurements of
S, which must exceed the minimal (resolution) time of probing/ reading out its
state. If these intervals greatly exceed the bath correlation (memory) time
$t_{c}\ll\tau$, then the regime is Markovian, i.e. the results of each
measurement of S are independent of its predecessors. This means that $S$
interacts every time with $B$ that is in the same statistical (thermal) state.
Then, upon many repetitions of the FID measurement, one can extract the
average coherence at a specific time to infer the value of the dephasing time
$T_{2}$. We, in contrast, wish to venture into the opposite strongly non-
Markovian limit, that corresponds to highly correlated results of consecutive
measurements. Namely, the readout at the time $t=m\tau$ ($m$ is an integer)
depends on the outcomes of all its predecessors. In this limit, the $B$-state
evolution is governed by the non-unitary evolution operator corresponding to
$S$ projections onto either of the states in Eq. (5), given by

$V_{\pm}(t)=(U_{e}(t)\pm U_{g}(t))/2,\leavevmode\nobreak\ \leavevmode\nobreak\
U_{e(g)}={\rm e}^{\pm i\omega_{j}t}{|j\rangle}{\langle j|}.$ (8)

The conditional evolution of $B$ assumes the form

$\rho^{\pm}_{B}(t)\equiv
V_{\pm}(t)\rho_{B}(0)V^{\dagger}_{\pm}(t)=\sum_{j}P_{j}(t){|j\rangle}{\langle
j|}$ (9)

where ${|j\rangle}$ are the collective eigenstates of the bath operator in Eq.
(2), and $P_{j}(t)$ are conditionally modified populations. Repeated
application of the conditional evolution operator in Eqs. (8), (9) can yield a
purified state of $B$ that as shown in Fig. (1) can drastically change the
standard FID of $S$.

To get an insight into this purification process, let us consider that
$m$-fold measurements of $S$ are consecutively performed in the
${|+\rangle},{|-\rangle}$ basis. There are $2^{m}$ measurement sequences with
different possible outcomes. The projections on the ${|+\rangle}$ or
${|-\rangle}$ state in each of these (single-shot) measurements are designated
as ${}^{\prime}0^{\prime}$, ${}^{\prime}1^{\prime}$ respectively. Each such
sequence, denoted by $M_{n,m}$, can be represented as an n-fold string of ‘1’
and m-fold string of ‘0’. The bath evolution is conditional on a particular
sequence or string, which, in parameter space, can be viewed as a trajectory –
dubbed henceforth “conditional trajectory” (CT).

For any CT associated to $M_{n,m}$, the corresponding non-unitary B-evolution
operator is

$V_{n,m}(t)=\cos^{m-n}(\hat{B}t)\sin^{n}(\hat{B}t).$ (10)

The $j$-distribution that underlies the B-state is progressively narrowed down
by the evolution operator (10) compared to that of the initial state, Eq. (1),
in a manner that depends on the realized CT. This comes about since, at a
chosen time $t$, different $j$-states will have either maximized or minimized
probabilities, depending on the arguments of the respective cosine or sine
functions.

In what follows we focus on the CT $M_{0,m}=\\{{0,0,\cdots 0\\}}$,
corresponding to an m-fold string of 0 results that is associated with the
bath state

$\rho_{B}(t,M_{0,m})=\frac{V_{0,m}(t)\rho_{B}(0)V_{0,m}^{\dagger}(t)}{{\rm
Tr}[V_{0,m}(t)\rho_{B}(0)V_{0,m}^{\dagger}(t)]},\hskip
5.69046ptV_{0,m}=\cos^{m}(\hat{B}t).$ (11)

The success probability $P_{0,m}(t)={{\rm
Tr}[V_{0,m}(t)\rho_{B}(0)V_{0,m}^{\dagger}(t)]}$ is maximized for
${\langle\hat{B}\rangle}(t)=l\pi$ $(l\in 1,2\cdots)$. Importantly, as the
$j$-distribution is gradually narrowed down, this probability does not
decrease rapidly with $m$, unlike the probability of an uncorrelated CT
sequence (see Experiment). Thus, by recording such an $m$-fold CT with
appreciable success probability, one ensures the projection of $B$ onto a
specific state $\rho_{B}$. The changes caused by B on S and vice versa along
this CT continue until a steady state of B is reached, that has much higher
purity than its initial high-temperature state. The steady states are
uncorrelated and are given by

$\rho_{B}(M_{0,m},t\rightarrow\infty)=\frac{1}{2^{N^{\prime}}}\sum_{k=-N^{\prime}/2}^{N^{\prime}/2}[{|{j^{\prime}}\rangle}{\langle{j^{\prime}}|}+{|-{j^{\prime}}\rangle}{\langle-{j^{\prime}}|}],$
(12)

where ${|\pm{j^{\prime}}\rangle}$ are the B- eigenstates that have survived
the purification by the selected CT and ${N^{\prime}}\ll N$.

Surprisingly, even without optimal feedback that keeps readjusting the time-
interval between measurements according to their outcomes (as proposed by some
of us Harel_1996), the convergence to the steady state turns out to be rather
fast and the success probability of reaching the desired steady state is
fairly high. The reason is the sinusoidal $j$-dependent S-B exchange that
allows to select certain $j$-states by choosing the appropriate interval. Let
us choose for the purification the shortest time-interval of resolvable
measurement, which is $\tau=1/\bar{g}$, where
$\bar{g}=\sqrt{\sum_{k}g_{k}^{2}}$. The steady state of $B$ following such
purification is a statistical mixture of total- magnetization (macroscopic)
states

$\rho_{B}(t\rightarrow\infty)=\frac{1}{2}[{|00\cdots 0\rangle}{\langle
00\cdots 0|}+{|11\cdots 1\rangle}{\langle 11\cdots 1|}].$ (13)

We have thus dramatically raised the bath purity from $1/2^{N}$ to $1/2$. Yet,
the individual spins are all unpolarized, and the polarization is not local
but global (collective).

The steady-state of $B$ following such purification is a persistent imprint
left by the corresponding CT of this state. This imprint is revealed by
initializing the same or another qubit in the $|+\rangle$ state and tracking
the subsequent evolution of its mean coherence by repeated measurements with
selected time-intervals. Remarkably, the QZE or AZE regimes can be stretched
on demand over much longer time intervals than before (Fig. 2) because of the
extreme non-Markovianity of the steady-state of B in Eqs. (12), (13).

Figure 2: Probe FID before and after bath purification: (a) Energy-level
diagram of the central probe-spin depicting the basis states and the readout.
Shown below is the pulse-sequence for four measurements. (b) Experimental and
theoretical FID of the probe spin before (blue) and after (red) four
measurements with identical (0) results (CT $M_{0,4}$). The effective FID rate
of the corresponding conditional bath state is $4$ times lower than the
original (blue) decay rate. (c) ODMR of the spin-bath spectrum before (blue)
and after (red) the conditional measurements, CT $M_{0,4}$.

###  IV.1 Experiment

In the experiment the $S$-spin is a low strain NV center in diamond. A small
magnetic field is applied along the NV axis [111] to counter the earth’s
magnetic field, thereby eliminating the net $B$-field at the location of the
NV center. Hence the ${|\pm 1\rangle}$ states in Fig. 2(a) are degenerate.
Only the bright superposition state $|+\rangle$ couples to to ${|0\rangle}$,
where the two states ${|\pm\rangle}$
($|\pm\rangle=\frac{[|+1\rangle\pm|-1\rangle]}{\sqrt{2}}$) form the two-level
probe-spin ($S$) subspace described above. For a projective readout of the
probe state, we map the ${|+\rangle}$ state to the ancillary state
${|0\rangle}$ via a microwave transition. At a low temperature ($4K$),
according to the optical selection rules, we achieve high-fidelity single-shot
readout. We can clearly observe our theoretical predictions as the time needed
for the purification of the bath and its posterior effects on $S$, as the
nuclear-spin coherence times, reaches minutes at low temperature.

Our protocol starts with the initialization of the electron spin to the state
${|0\rangle}$ by resonantly exciting it in the $A1$ transition ref69 ,
followed by a microwave transition, ${|0\rangle}\rightarrow{|+\rangle}$. We
let the prepared state acquire a phase over time $\tau$ due to its interaction
with the bath and then map the population back to the ancillary state
${|0\rangle}$ and perform single shot readout using the $E_{x}$ excitation.

Figure 3: Zeno and Anti-Zeno effects on the evolution of the probe-spin
following bath purification(a) The experimental pulse diagram depicting the
Microwave pulses (blue) interleaved with repeated initialization ($A_{1}$) and
measurements ($E_{x}$), for preparation of the bath state $M_{n,m}$. Following
this the FID of the probe spin is measured again and compared with the
original FID. (b, c) The FID of the probe spin interacting with the
conditional bath state $M_{0,4}$ (red), $M_{4,0}$ (green) is compared with the
average FID obtained from all possible conditional bath state preparations
(blue). The same is repeated is two different time scales $\tau=600$ns (b) and
$\tau=1200$ns. The solid-lines are fits to the experimental data.

To confirm the actual purification of the bath toward a low-entropy state we
perform two different experiments, wherein: (i) we measure and compare the
probe-spin FID (Eq. (2)) before and after the bath purification by conditional
measurements; and (ii) we measure the bath spectrum via the standard procedure
of the Optically Detected Magnetic Resonance (ODMR) that records the spin
noise spectrum of the probe-spin, before and after the conditional
measurements. The results shown in Fig. 2, 3 reveal the prolonged coherence
time of the probe-spin and the corresponding narrowing of its spin noise
spectrum, after four conditional measurements that purify the bath to a low-
entropy state. Having chosen the measurement time-scale ($\tau<T_{2}^{*}$) to
be in the Zeno (QZE) regime, the observation of a measurement string $M_{0,4}$
projects the bath state to a mixture of the effective zero-field state ($j=0$)
and the high-$j$ states (Eq. (13)). Since the density of states that
correspond to the zero bath-field is exponentially larger, by the factor
${}^{N}C_{N/2}$, than the high-$j$ states, the probability of the bath to
collapse toward the $j=0$ is high.

Figure 4: Life-time of the purified collective bath-state. (a) The pulse
sequence used for the measurement of the bath life-time. After the conditional
bath state preparation $M_{0,4}$, we allow a free time $t$ and then measure
again the probability for obtaining the result an identical measurement
result. (b) The probability of obtaining the result $0$ after a long wait-time
$t$ for a bath state preparations $M_{0,4}$ (red) is shown for the measurement
times $\tau=1200$ns.These probabilities are also compared with the case when
no apriori conditional bath state preparation. The effective life-time of the
bath extends towards the $T_{1}^{e}$ life-time of the probe.

To change the probe dynamics from the Zeno (QZE) to the Anti-Zeno (AZE)
regime, we repeat the experiments with a different CT, $M_{4,0}$, and also a
different measurement interval $\tau$. In Fig. 3 we show the FID samplings of
the probe-spin following different conditional preparations of the bath by
repeated selective measurement samplings of the probe-spin. We see two
dramatically different decay profiles of the probe-spin coherence: either a
slowdown of the decay (red-QZE) or an underdamped oscillation (blue-AZE). In
the AZE regime, the probe coherence exhibits underdamped oscillation at the
mean frequency of the bath field $\omega_{B}$, reflecting the bath preparation
in a non-zero $j$-state. The best fits to these AZE-like experimental data
take the form $C(t)=1/2+ae^{-(t/T)^{2}}\cos(\omega_{B}t)$, where $a$ is a
fitting parameter. For example in Fig. 3, the bath-field obtained from such
fitting for measurements time scales $\tau=600ns$ and $\tau=1200$ns is
approximately $\omega_{B}=200$kHz and $\omega_{B}=80$kHz, and $a\approx 1$.

We next repeat the same experiment with a much longer time interval between
samplings $t_{s}\geq T_{2}^{*}$ that is well beyond the Zeno or anti-Zeno time
scale. Remarkably, we observe similar effects to Fig. 3, but now they are
dependent only on the CT, $M_{0,4},M_{4,0}$ and not on the sampling time-scale
$t_{s}$. Thus, whereas the original QZE and AZE depend on time-resolution and
not on energy resolution, and are caused by non-selective independent
measurements, here we observe the opposite: their counter-parts depend on
energy resolution rather than on time-resolution, and are caused by state-
selective measurements.

These novel modifications of the QZE and AZE are shown in Fig. 4 where long
time delay is introduced between the bath purification by conditional
measurements and the subsequent samplings of the probe-spin coherence. These
sequences help to measure the life-time of the purified collective bath state.
One can see that the measurement of the probe-spin result (coherence) is
almost unchanged upto a milli second and then slowly decays towards the fully
mixed state i.e., $C(t)=0.5$. This corresponds to a very long life-time of the
collective bath spin-state prepared by the conditional measurements of the
probe-spin.

##  V Discussion

We have demonstrated, theoretically and experimentally, the ability to purify
the initially fully mixed spin-bath state to a low-entropy steady-state by
conditional measurements of a spin-probe coupled to it. We have also revealed
the remarkable, hitherto unknown, persistent posterior effects of such
purification on other probe spins. The demonstrated effects are not observable
unless we track a particular conditional measurement sequence. Otherwise,
following $m$ projective measurements of the spin-probe S, its mean coherence
exhibits decay that is an average over the $2^{m}$ possible measurement
outcomes which we dub here conditional trajectories (CT). One can verify ref70
that the corresponding averaged state of the bath $B$ is then the same as the
unmeasured state at time $t$ i.e., $\sum_{N}\rho_{B}(M_{N},t)=\rho_{B}(t)$.
This confirms the ergodicity of the interacting $S+B$ (system $+$ bath). Thus,
averaging over all possible FID patterns resulting from all different CT
yields the FID of $S$ in the absence of any measurements. By contrast, we have
here searched to unravel individual CTs that correspond to specific states of
B and hence to particular magnetic (dipolar) field distributions generated by
the bath. The FID time $T_{2}^{*}$, is actually an average of faster and
slower decays of the S-coherence caused by the bath: There are bath states
that almost freeze the decay (obeying QZE evolution) and those that , on the
contrary, speed it up (under AZE evolution).All such effects become observable
upon resolving the CTs of S, as demonstrated here. We wish to highlight the
role of conditional measurements that lead to AZE oscillations of the probe
spin, as they allow to spectroscopically identify strongly-interacting spins.
Such spins are otherwise identified through complex correlation spectroscopy
methods ref71 ; ref72 . For example, from the underdamped (AZE) oscillations
in Fig. 3, we identify the presence of a strongly-interacting spin with
coupling strength $g\sim 200kHz$. Remarkably, we have shown the ability to
purify an initially unpolarized (high-temperature) bath-state to a low-entropy
steady state that is controlled by selecting a CT with fairly high success
probability and limited number of measurement. Due to the rise in the success
probability of the selected result as the $B$-distribution narrows down. This
favorable scaling of spin-bath purification with the number of successful
outcomes is akin to that of quantum state engineering by conditional
measurements previously introduced by some of us to control the evolution of
quantum observables ref68 ; ref73 ; ref74 . One can optimize the present
protocol by adopting similar protocols ref68 .

The ability, proposed and demonstrated here, to cool down (purify) the bath to
a low-entropy steady state, allows us to preserve spin coherence or cause its
oscillations over much longer times than with a thermal bath. This may
substantially benefit quantum information processing or sensing, since, unlike
existing dynamical control methods (such as dynamical decoupling (DD) ref35 ;
ref37 ; ref38 ; ref39  or the bath-optimized minimal-energy control (BOMEC)
devised by us ref42 ; ref43 , this measurement-induced steady-state remains
imprinted on the bath long after the control has been switched off.

###### Acknowledgements.

D.D.B.Rao and J.W would like to acknowledge the support from DFG (FOR2724),
ERC project SMeL, DFG SFB/TR21, EU ASTERIQS, QIA, Max Planck Society, the
Volkswagenstiftung, the Baden- Wuerttemberg Foundation. S.Y acknowledges the
financial support from Hong Kong RGC (GRF/14304618). A.F would like to
acknowledge the historic generosity of the Harold Perlman Family, research
grants from the Abramson Family Center for Young Scientists, the Ilse Katz
Institute for Material Sciences and Magnetic Resonance Research and the
Willner Family Leadership Institute for the Weizmann Institute of Science.

## References

  * (1) F. Schwabl,Statistica Mechanics(Springer, Berlin, 2006).
  * (2) H.-P. Breuer and F. Petruccione, The Theory of Open Quantum Systems (Oxford University Press, 2002).
  * (3) W. H. Zurek, Rev. Mod. Phys. 75, 715 (2003).
  * (4) W. H. Zurek, Nat. Phys. 5, 181 (2009).
  * (5) G. Lindblad, Non-Equilibrium Entropy and Irreversibility (D. Reidel, Holland, 1983).
  * (6) A. Polkovnikov, K. Sengupta, A. Silva, and M. Vengalattore, Rev. Mod. Phys. 83, 863 (2011).
  * (7) P. Reimann, New Journal of Physics 12, 055027 (2010).
  * (8) M. Rigol, V. Dunjko, and M. Olshanii, Nature 452, 854 (2008).
  * (9) E. Joos, H. D. Zeh, C. Kiefer, D. J. Giulini, J. Kupsch, and I.-O. Stamatescu, Decoherence and the appearance of a classical world in quantum theory (Springer Science $\&$ Business Media, 2013).
  * (10) G. Kurizki, P. Bertet, Y. Kubo, K. Moelmer, D. Petrosyan, P. Rabl, and J. Schmiedmayer, Proc. Nat. Acad. Sci. 112, 3866 (2015).
  * (11) M. A. Nielsen and I. L. Chuang, Quantum computation and quantum information (Cambridge University Press, Cambridge, 2010).
  * (12) P. Kaye, R. La amme, M. Mosca, et al., An introduction to quantum computing (Oxford university press, 2007).
  * (13) G. Balasubramanian, I. Y. Chan, R. Kolesov, M. Al-Hmoud, J. Tisler, C. Shin, C. Kim, A. Wojcik, P. R. Hemmer, A. Krueger, T. Hanke, A. Leitenstorfer, R. Bratschitsch, F. Jelezko, and J. Wrachtrup, Nature 455, 648 (2008).
  * (14) J. R. Maze, P. L. Stanwix, J. S. Hodges, S. Hong, J. M. Taylor, P. Cappellaro, L. Jiang, M. V. G. Dutt, E. Togan, A. S. Zibrov, A. Yacoby, R. L. Walsworth, and M. D. Lukin, Nature 455, 644 (2008).
  * (15) T. Staudacher, F. Shi, S. Pezzagna, J. Meijer, J. Du, C. A. Meriles, F. Reinhard, and J. Wrachtrup, Science 339, 561 (2013).
  * (16) P. Neumann et al., Science 320, 1326 (2008).
  * (17) M. V. G. Dutt, L. Childress, L. Jiang, E. Togan, J. Maze, F. Jelezko, A. S. Zibrov, P. R. Hemmer, and M. D. Lukin, Science 316, 1312 (2007).
  * (18) F. Dolde, H. Fedder, M. Doherty, and T. Nobauer, Nat. Phys. 7, 459 (2011).
  * (19) J. M. Taylor, P. Cappellaro, L. Childress, L. Jiang, D. Budker, P. R. Hemmer, A. Yacoby, R. Walsworth, and M. D. Lukin, Nat Phys 4, 810 (2008).
  * (20) P. C. Maurer, G. Kucsko, C. Latta, L. Jiang, N. Y. Yao, S. D. Bennett, F. Pastawski, D. Hunger, N. Chisholm, M. Markham, D. J. Twitchen, J. I. Cirac, and M. D. Lukin, Science 336, 1283 (2012).
  * (21) M. H. Abobeih, J. Randall, C. E. Bradley, H. P. Bartling, M. A. Bakker, M. J. Degen, M. Markham, D. J. Twitchen, and T. H. Taminiau, Nature 576, 411 (2019).
  * (22) L. Robledo, L. Childress, H. Bernien, B. Hensen, P. F. A. Alkemade, and R. Hanson, Nature 477, 574 (2011).
  * (23) H. Bernien, B. Hensen, W. Pfa, G. Koolstra, M. S. Blok, L. Robledo, T. H. Taminiau, M. Markham, D. J. Twitchen, L. Childress, and R. Hanson, Nature 497, 86 (2013).
  * (24) K. S. Cujia, J. M. Boss, K. Herb, J. Zopes, and C. L. Degen, Nature 571, 230 (2019).
  * (25) L. Childress, M. V. Gurudev Dutt, J. M. Taylor, A. S. Zibrov, F. Jelezko, J. Wrachtrup, P. R. Hemmer, and M. D. Lukin, Science 314, 281 (2006).
  * (26) M. S. Grinolds, S. Hong, P. Maletinsky, L. Luan, M. D. Lukin, R. L. Walsworth, and A. Yacoby, Nat. Phys. 9, 215 (2013).
  * (27) N. Aslam, M. Pfender, P. Neumann, R. Reuter, A. Zappe, F. Favaro de Oliveira, A. Denisenko, H. Sumiya, S. Onoda, J. Isoya, and J. Wrachtrup, Science 357, 67 (2017).
  * (28) D. R. Glenn, D. B. Bucher, J. Lee, M. D. Lukin, H. Park, and R. L. Walsworth, Nature 555, 351 (2018).
  * (29) D. Le Sage, K. Arai, D. Glenn, S. DeVience, L. Pham, L. Rahn-Lee, M. Lukin, A. Yacoby, A. Komeili, and R. Walsworth, Nature 496, 486 (2013).
  * (30) L. P. McGuinness, Y. Yan, A. Stacey, D. A. Simpson, L. T. Hall, D. Maclaurin, S. Prawer, P. Mulvaney, J. Wrachtrup, F. Caruso, R. E. Scholten, and L. C. L. Hollenberg, Nature Nanotech. 6, 358 (2011).
  * (31) J. F. Barry, M. J. Turner, J. M. Schloss, D. R. Glenn, Y. Song, M. D. Lukin, H. Park, and R. L. Walsworth, Proc. of the Nat. Acad. of Sci. 113, 14133 (2016).
  * (32) G. Kucsko, P. C. Maurer, N. Y. Yao, M. Kubo, H. J. Noh, P. K. Lo, H. Park, and M. D. Lukin, Nature 500, 54 (2013).
  * (33) H. M. Wiseman and G. J. Milburn, Quantum Measurement and Control (Cambridge University Press, 2009).
  * (34) C. Brif, R. Chakrabarti, and H. Rabitz, New Journal of Physics 12, 075008 (2010).
  * (35) C. P. Koch, Journal of Physics: Condensed Matter 28, 213001 (2016).
  * (36) G. S. Agarwal, M. Scully, and H. Walther, Phys. Rev. A 63, 044101 (2001).
  * (37) L. Viola and S. Lloyd, Phys. Rev. A 58, 2733 (1998).
  * (38) D. Vitali and P. Tombesi, Phys. Rev. A 65, 012305 (2001).
  * (39) C. P. Slichter, Principles of magnetic resonance, Vol. 1 (Springer Science $\&$ Business Media, 2013).
  * (40) A. G. Kofman and G. Kurizki, Phys. Rev. Lett. 93, 130406 (2004).
  * (41) G. Gordon, G. Kurizki, and D. A. Lidar, Phys. Rev. Lett. 101, 010403 (2008).
  * (42) J. Clausen, G. Bensky, and G. Kurizki, Phys. Rev. Lett. 104, 040401 (2010).
  * (43) J. Clausen, G. Bensky, and G. Kurizki, Phys. Rev. A 85, 052105 (2012).
  * (44) A. Zwick, G. A. Alvarez, G. Bensky, and G. Kurizki, New J. Phys. 16, 065021 ((2014)).
  * (45) B. M. Escher, G. Bensky, J. Clausen, and G. Kurizki, J. Phys. B 44, 154015 ((2011)).
  * (46) G. Gordon, N. Erez, and G. Kurizki, J. Phys. B: At. Mol. Opt. Phys. 40, S75 (2007).
  * (47) B. Misra and E. C. G. Sudarshan, Journal of Mathematical Physics 18, 756 (1977).
  * (48) W. M. Itano, D. J. Heinzen, J. J. Bollinger, and D. J. Wineland, Phys. Rev. A 41, 2295 (1990).
  * (49) P. Kwiat, H. Weinfurter, T. Herzog, A. Zeilinger, and M. A. Kasevich, Phys. Rev. Lett. 74, 4763 (1995).
  * (50) A. G. Kofman and G. Kurizki, Nature 405, 546 (2000).
  * (51) A. G. Kofman and G. Kurizki, Phys. Rev. Lett. 87, 270405 (2001).
  * (52) A. G. Kofman and G. Kurizki, Phys. Rev. A 54, R3750 (1996).
  * (53) M. C. Fischer, B. Gutierrez-Medina, and M. G. Raizen, Phys. Rev. Lett. 87, 040402 (2001).
  * (54) P. Facchi and S. Pascazio, Phys. Rev. Lett. 89, 080401 (2002).
  * (55) D. D. B. Rao and G. Kurizki, Phys. Rev. A 83, 032105 (2011).
  * (56) A. Barone, G. Kurizki, and A. G. Kofman, Phys. Rev. Lett. 92, 200403 (2004).
  * (57) M. M. Mueller, S. Gherardini, and F. Caruso, Annalen der Physik 529, 1600206 (2017).
  * (58) Y. Khodorkovsky, G. Kurizki, and A. Vardi, Phys. Rev. Lett. 100, 220403 (2008).
  * (59) P. Biagioni, G. D. Valle, M. Ornigotti, M. Finazzi, L. Duo, P. Laporta, and S. Longhi, Opt. Express 16, 3762 (2008).
  * (60) G. Gordon, D. D. B. Rao, and G. Kurizki, New Journal of Physics 12, 053033 (2010).
  * (61) N. Erez, G. Gordon, M. Nest, and G. Kurizki, Nature 452, 724 (2008).
  * (62) G. A. Alvarez, D. D. B. Rao, L. Frydman, and G. Kurizki, Phys. Rev. Lett. 105, 160401 (2010).
  * (63) J. N. Greiner, D. B. R. Dasari, and J.Wrachtrup, Sci. Reps. 7 (2017).
  * (64) D. D. B. Rao, S. A. Momenzadeh, and J. Wrachtrup, Physical Review Letters 117 (2016).
  * (65) L. Viola and S. Lloyd, Phys. Rev. A 58, 2733 (1998).
  * (66) N. Bar-Gill, L. Pham, A. Jarmola, D. Budker, and R. Walsworth, Nat. Commns. 4, 1743 (2013).
  * (67) G. Gordon and G. Kurizki, Phys. Rev. A 76, 042310 (2007).
  * (68) G. Harel, G. Kurizki, J. K. McIver, and E. Coutsias, Phys. Rev. A 53, 4534 (1996).
  * (69) S. Yang, Y.Wang, D. D. B. Rao, T. H. Tran, A. S. Momenzadeh, M. Markham, D. J. Twitchen, P. Wang, W. Yang, R. Stoehr, P. Neumann, H. Kosaka, and J. Wrachtrup, Nature Photonics 10, 507 (2016).
  * (70) D. D. Bhaktavatsala Rao, S. Yang, S. Jesenski, E. Tekin, F. Kaiser, and J. Wrachtrup, Phys. Rev. A 100, 022307 (2019).
  * (71) C. O. Bretschneider, G. A. Alvarez, G. Kurizki, and L. Frydman, Phys. Rev. Lett. 108, 140403 (2012).
  * (72) C. E. Bradley, J. Randall, M. H. Abobeih, R. C. Berrevoets, M. J. Degen, M. A. Bakker, M. Markham, D. J. Twitchen, and T. H. Taminiau, Phys. Rev. X 9, 031045 (2019).
  * (73) T. Opatrny, G. Kurizki, and D.-G. Welsch, Phys. Rev. A 61, 032302 (2000).
  * (74) B. M. Garraway, B. Sherman, H. Moya-Cessa, P. L. Knight, and G. Kurizki, Phys. Rev. A 49, 535 (1994).


* Raghu et al. [2017] Maithra Raghu, Ben Poole, Jon Kleinberg, Surya Ganguli, and Jascha Sohl-Dickstein.  On the expressive power of deep neural networks.  In _Proceedings of the 34th International Conference on Machine Learning (ICML)_ , pages 2847–2854. PMLR, 2017.
  * Rahaman et al. [2019] Nasim Rahaman, Aristide Baratin, Devansh Arpit, Felix Draxler, Min Lin, Fred Hamprecht, Yoshua Bengio, and Aaron Courville.  On the spectral bias of neural networks.  In _Proceedings of the 36th International Conference on Machine Learning (ICML)_ , pages 5301–5310. PMLR, 2019.
  * Rahimi and Recht [2008] Ali Rahimi and Benjamin Recht.  Random features for large-scale kernel machines.  In _Advances in neural information processing systems_ , pages 1177–1184, 2008.
  * Raileanu and Fergus [2021] Roberta Raileanu and Rob Fergus.  Decoupling value and policy for generalization in reinforcement learning.  _International Conference on Machine Learning (ICML)_ , pages 8787–8798, 2021.
  * Raileanu et al. [2021] Roberta Raileanu, Maxwell Goldstein, Denis Yarats, Ilya Kostrikov, and Rob Fergus.  Automatic data augmentation for generalization in reinforcement learning.  In _Advances in Neural Information Processing Systems_ , 2021.
  * Raj et al. [2017] Anant Raj, Abhishek Kumar, Youssef Mroueh, Tom Fletcher, and Bernhard Schoelkopf.  Local Group Invariant Representations via Orbit Embeddings.  In Aarti Singh and Jerry Zhu, editors, _Proceedings of the 20th International Conference on Artificial Intelligence and Statistics_ , volume 54 of _Proceedings of Machine Learning Research_ , pages 1225–1235. PMLR, 2017.
  * Rakelly et al. [2019] Kate Rakelly, Aurick Zhou, Chelsea Finn, Sergey Levine, and Deirdre Quillen.  Efficient off-policy meta-reinforcement learning via probabilistic context variables.  In _Proceedings of the 36th International Conference on Machine Learning (ICML)_ , pages 5331–5340. PMLR, 2019.
  * Rasmussen [2003] Carl Edward Rasmussen.  Gaussian processes in machine learning (icml).  In _Summer School on Machine Learning (ICML)_ , pages 63–71. Springer, 2003.
  * Rasmussen and Ghahramani [2001] Carl Edward Rasmussen and Zoubin Ghahramani.  Occam’s razor.  In _Advances in neural information processing systems_ , pages 294–300, 2001.
  * Ravanbakhsh et al. [2017] Siamak Ravanbakhsh, Jeff Schneider, and Barnabas Poczos.  Equivariance through parameter-sharing.  In _Proceedings of the 34th International Conference on Machine Learning-Volume 70_ , pages 2892–2901. JMLR. org, 2017.
  * Robbins and Monro [1951] Herbert Robbins and Sutton Monro.  A stochastic approximation method.  _The annals of mathematical statistics_ , pages 400–407, 1951.
  * Rojas-Carulla et al. [2018] Mateo Rojas-Carulla, Bernhard Schölkopf, Richard Turner, and Jonas Peters.  Invariant models for causal transfer learning.  _The Journal of Machine Learning Research_ , 19(1):1309–1342, 2018.
  * Ru et al. [2021] Binxin Ru, Clare Lyle, Lisa Schut, Mark van der Wilk, and Yarin Gal.  Revisiting the train loss: an efficient performance estimator for neural architecture search.  _Advances in Neural Information Processing Systems (Spotlight Presentation)_ , 34, 2021.
  * Rusu et al. [2016] Andrei A Rusu, Sergio Gomez Colmenarejo, Çaglar Gülçehre, Guillaume Desjardins, James Kirkpatrick, Razvan Pascanu, Volodymyr Mnih, Koray Kavukcuoglu, and Raia Hadsell.  Policy distillation.  In _ICLR (Poster)_ , 2016.
  * Salamon and Bello [2017] Justin Salamon and Juan Pablo Bello.  Deep convolutional neural networks and data augmentation for environmental sound classification.  _IEEE Signal Processing Letters_ , 24(3):279–283, 2017.
  * Samvelyan et al. [2021] Mikayel Samvelyan, Robert Kirk, Vitaly Kurin, Jack Parker-Holder, Minqi Jiang, Eric Hambro, Fabio Petroni, Heinrich Kuttler, Edward Grefenstette, and Tim Rocktäschel.  Minihack the planet: A sandbox for open-ended reinforcement learning research.  In _Thirty-fifth Conference on Neural Information Processing Systems Datasets and Benchmarks Track (Round 1)_ , 2021.
  * Santurkar et al. [2018] Shibani Santurkar, Dimitris Tsipras, Andrew Ilyas, and Aleksander Madry.  How does batch normalization help optimization?  _Advances in neural information processing systems_ , 31, 2018.
  * Schaul et al. [2019] Tom Schaul, Diana Borsa, Joseph Modayil, and Razvan Pascanu.  Ray interference: a source of plateaus in deep reinforcement learning.  _Workshop on Reinforcement Learning and Decision Making_ , 2019.
  * Schmitt et al. [2018] Simon Schmitt, Jonathan J Hudson, Augustin Zidek, Simon Osindero, Carl Doersch, Wojciech M Czarnecki, Joel Z Leibo, Heinrich Kuttler, Andrew Zisserman, Karen Simonyan, et al.  Kickstarting deep reinforcement learning.  _arXiv preprint arXiv:1803.03835_ , 2018.
  * Schrittwieser et al. [2020] Julian Schrittwieser, Ioannis Antonoglou, Thomas Hubert, Karen Simonyan, Laurent Sifre, Simon Schmitt, Arthur Guez, Edward Lockhart, Demis Hassabis, Thore Graepel, et al.  Mastering atari, go, chess and shogi by planning with a learned model.  _Nature_ , 588(7839):604–609, 2020.
  * Schulman et al. [2015] John Schulman, Sergey Levine, Pieter Abbeel, Michael Jordan, and Philipp Moritz.  Trust region policy optimization.  In _Proceedings of the 32nd International Conference on Machine Learning (ICML)_ , pages 1889–1897. PMLR, 2015.
  * Schulman et al. [2017] John Schulman, Filip Wolski, Prafulla Dhariwal, Alec Radford, and Oleg Klimov.  Proximal policy optimization algorithms.  _arXiv preprint arXiv:1707.06347_ , 2017.
  * Schwarz et al. [2018] Jonathan Schwarz, Wojciech Czarnecki, Jelena Luketina, Agnieszka Grabska-Barwinska, Yee Whye Teh, Razvan Pascanu, and Raia Hadsell.  Progress & compress: A scalable framework for continual learning (icml).  In _Proceedings of the 35th International Conference on Machine Learning (ICML)_ , 2018.
  * Serre [1977] Jean-Pierre Serre.  _Linear representations of finite groups_ , volume 42.  Springer, 1977.
  * Shalit et al. [2017] Uri Shalit, Fredrik D Johansson, and David Sontag.  Estimating individual treatment effect: generalization bounds and algorithms.  In _Proceedings of the 34th International Conference on Machine Learning-Volume 70_ , pages 3076–3085. JMLR. org, 2017.
  * Shao et al. [2022] Lin Shao, Yifan You, Mengyuan Yan, Qingyun Sun, and Jeannette Bohg.  GRAC: self-guided and self-regularized actor-critic.  _Conference on Robot Learning_ , pages 267–276, 2022.
  * Sharkey and Sharkey [1995] Noel E Sharkey and Amanda JC Sharkey.  An analysis of catastrophic interference.  _Connection Science_ , 1995.
  * Silver and Mercer [2002] Daniel L Silver and Robert E Mercer.  The task rehearsal method of life-long learning: Overcoming impoverished data.  In _Conference of the Canadian Society for Computational Studies of Intelligence_ , pages 90–101, 2002.
  * Smith and Le [2018] Samuel L. Smith and Quoc V. Le.  A bayesian perspective on generalization and stochastic gradient descent.  In _International Conference on Learning Representations (ICLR)_ , 2018.
  * Smith et al. [2020] Samuel L Smith, Benoit Dherin, David Barrett, and Soham De.  On the origin of implicit regularization in stochastic gradient descent.  In _International Conference on Learning Representations (ICLR)_ , 2020.
  * Sodhani et al. [2021] Shagun Sodhani, Amy Zhang, and Joelle Pineau.  Multi-task reinforcement learning with context-based tepresentations.  In _International Conference on Machine Learning (ICML)_ , pages 9767–9779. PMLR, 2021.
  * Song et al. [2020] Xingyou Song, Yiding Jiang, Stephen Tu, Yilun Du, and Behnam Neyshabur.  Observational overfitting in reinforcement learning.  In _International Conference on Learning Representations (ICLR)_ , 2020.
  * Srivastava et al. [2014] Nitish Srivastava, Geoffrey Hinton, Alex Krizhevsky, Ilya Sutskever, and Ruslan Salakhutdinov.  Dropout: a simple way to prevent neural networks from overfitting.  _The journal of machine learning research_ , 15(1):1929–1958, 2014.
  * Stachenfeld et al. [2014] Kimberly L Stachenfeld, Matthew Botvinick, and Samuel J Gershman.  Design principles of the hippocampal cognitive map.  In _Advances in Neural Information Processing Systems_ , 2014.
  * Stachenfeld et al. [2017] Kimberly L Stachenfeld, Matthew M Botvinick, and Samuel J Gershman.  The hippocampus as a predictive map.  _Nature neuroscience_ , 20(11):1643–1653, 2017\.
  * Suter et al. [2019] Raphael Suter, Djordje Miladinovic, Bernhard Schölkopf, and Stefan Bauer.  Robustly disentangled causal mechanisms: Validating deep representations for interventional robustness.  In _Proceedings of the 36th International Conference on Machine Learning (ICML)_ , pages 6056–6065. PMLR, 2019.
  * Sutton and Barto [2018] Richard S Sutton and Andrew G Barto.  _Reinforcement learning: An introduction_.  MIT press, 2018.
  * Sutton et al. [2000] Richard S Sutton, David A McAllester, Satinder P Singh, and Yishay Mansour.  Policy gradient methods for reinforcement learning with function approximation.  In _Advances in neural information processing systems_ , pages 1057–1063, 2000.
  * Sutton et al. [2008] Richard S Sutton, Csaba Szepesvári, and Hamid Reza Maei.  A convergent $O(n)$ algorithm for off-policy temporal-difference learning with linear function approximation.  _Advances in Neural Information Processing Systems_ , 2008.
  * Tassa et al. [2018] Yuval Tassa, Yotam Doron, Alistair Muldal, Tom Erez, Yazhe Li, Diego de Las Casas, David Budden, Abbas Abdolmaleki, Josh Merel, Andrew Lefrancq, Timothy Lillicrap, and Martin Riedmiller.  DeepMind control suite.  Technical report, DeepMind, January 2018.
  * Taylor et al. [2008] Jonathan Taylor, Doina Precup, and Prakash Panagaden.  Bounding performance loss in approximate mdp homomorphisms.  _Advances in Neural Information Processing Systems_ , 21, 2008.
  * Teh et al. [2017] Yee Teh, Victor Bapst, Wojciech M. Czarnecki, John Quan, James Kirkpatrick, Raia Hadsell, Nicolas Heess, and Razvan Pascanu.  Distral: Robust multitask reinforcement learning.  In I. Guyon, U. V. Luxburg, S. Bengio, H. Wallach, R. Fergus, S. Vishwanathan, and R. Garnett, editors, _Advances in Neural Information Processing Systems 30_ , pages 4496–4506. Curran Associates, Inc., 2017.
  * Tishby and Zaslavsky [2015] Naftali Tishby and Noga Zaslavsky.  Deep learning and the information bottleneck principle.  In _2015 ieee information theory workshop (itw)_ , pages 1–5. IEEE, 2015.
  * Tsitsiklis and Van Roy [1996] John Tsitsiklis and Benjamin Van Roy.  Analysis of temporal-diffference learning with function approximation.  _Advances in neural information processing systems_ , 9, 1996.
  * Tsitsiklis [1994] John N Tsitsiklis.  Asynchronous stochastic approximation and Q-learning (icml).  _Machine Learning (ICML)_ , 16(3):185–202, 1994\.
  * Tzeng et al. [2017] E. Tzeng, J. Hoffman, K. Saenko, and T. Darrell.  Adversarial discriminative domain adaptation.  In _2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR)_ , pages 2962–2971, Los Alamitos, CA, USA, jul 2017. IEEE Computer Society.  10.1109/CVPR.2017.316.
  * Ullrich et al. [2017] Karen Ullrich, Edward Meeds, and Max Welling.  Soft weight-sharing for neural network compression.  In _International Conference on Learning Representations (ICLR)_ , 2017.
  * Valle-Pérez et al. [2018] Guillermo Valle-Pérez, Chico Q Camargo, and Ard A Louis.  Deep learning generalizes because the parameter-function map is biased towards simple functions.  _International Conference on Learning Representations_ , 2018.
  * van der Pol et al. [2020a] Elise van der Pol, Thomas Kipf, Frans A Oliehoek, and Max Welling.  Plannable approximations to mdp homomorphisms: Equivariance under actions.  _Proceedings of the 19th International Conference on Autonomous Agents and MultiAgent Systems_ , pages 1431–1439, 2020a.
  * van der Pol et al. [2020b] Elise van der Pol, Daniel Worrall, Herke van Hoof, Frans Oliehoek, and Max Welling.  Mdp homomorphic networks: Group symmetries in reinforcement learning.  _Advances in Neural Information Processing Systems_ , 33:4199–4210, 2020b.
  * van der Wilk et al. [2018] Mark van der Wilk, Matthias Bauer, ST John, and James Hensman.  Learning invariances using the marginal likelihood.  In _Advances in Neural Information Processing Systems_ , pages 9938–9948, 2018.
  * Van Hasselt et al. [2016] Hado Van Hasselt, Arthur Guez, and David Silver.  Deep reinforcement learning with double Q-learning (icml).  In _AAAI Conference on Artificial Intelligence_ , 2016.
  * Van Hasselt et al. [2018] Hado Van Hasselt, Yotam Doron, Florian Strub, Matteo Hessel, Nicolas Sonnerat, and Joseph Modayil.  Deep reinforcement learning and the deadly triad.  _arXiv preprint arXiv:1812.02648_ , 2018.
  * Vapnik [1968] Vladimir Vapnik.  On the uniform convergence of frequencies of occurrence of events to their probabilities.  _Theory of Probability and its Applications_ , 16, 1968.
  * Vapnik [1991] Vladimir Vapnik.  Principles of risk minimization for learning theory.  _Advances in neural information processing systems_ , 4, 1991.
  * Vapnik [1999] Vladimir Vapnik.  _The nature of statistical learning theory_.  Springer science & business media, 1999.
  * Vaswani et al. [2017] Ashish Vaswani, Noam Shazeer, Niki Parmar, Jakob Uszkoreit, Llion Jones, Aidan N Gomez, Ł ukasz Kaiser, and Illia Polosukhin.  Attention is all you need.  _Advances in Neural Information Processing Systems_ , 2017.
  * Veeriah et al. [2019] Vivek Veeriah, Matteo Hessel, Zhongwen Xu, Richard Lewis, Janarthanan Rajendran, Junhyuk Oh, Hado van Hasselt, David Silver, and Satinder Singh.  Discovery of useful questions as auxiliary tasks.  In _Advances in Neural Information Processing Systems_ , 2019.
  * Virtanen et al. [2020] Pauli Virtanen, Ralf Gommers, Travis E. Oliphant, Matt Haberland, Tyler Reddy, David Cournapeau, Evgeni Burovski, Pearu Peterson, Warren Weckesser, Jonathan Bright, Stéfan J. van der Walt, Matthew Brett, Joshua Wilson, K. Jarrod Millman, Nikolay Mayorov, Andrew R. J. Nelson, Eric Jones, Robert Kern, Eric Larson, C J Carey, İlhan Polat, Yu Feng, Eric W. Moore, Jake VanderPlas, Denis Laxalde, Josef Perktold, Robert Cimrman, Ian Henriksen, E. A. Quintero, Charles R. Harris, Anne M. Archibald, Antônio H. Ribeiro, Fabian Pedregosa, Paul van Mulbregt, and SciPy 1.0 Contributors.  SciPy 1.0: Fundamental Algorithms for Scientific Computing in Python.  _Nature Methods_ , 17:261–272, 2020.
  * Wang et al. [2021] B Wang, C Lyle, and M Kwiatkowska.  Provable guarantees on the robustness of decision rules to causal interventions.  In _Proceedings of the International Joint Conference on Artificial Intelligence_ , 2021.
  * Wang et al. [2020] Yining Wang, Ruosong Wang, Simon Shaolei Du, and Akshay Krishnamurthy.  Optimism in reinforcement learning with generalized linear function approximation.  In _International Conference on Learning Representations (ICLR)_ , 2020.
  * Watkins and Dayan [1992] Christopher JCH Watkins and Peter Dayan.  Q-learning.  _Machine Learning (ICML)_ , 8(3-4):279–292, 1992\.
  * Wei and Ma [2019] Colin Wei and Tengyu Ma.  Improved sample complexities for deep neural networks and robust classification via an all-layer margin.  In _International Conference on Learning Representations (ICLR)_ , 2019.
  * Welling and Teh [2011] Max Welling and Yee W Teh.  Bayesian learning via stochastic gradient langevin dynamics.  In _Proceedings of the 28th International Conference on Machine Learning (ICML-11)_ , pages 681–688, 2011.
  * Wilson and Izmailov [2020] Andrew Gordon Wilson and Pavel Izmailov.  Bayesian deep learning and a probabilistic perspective of generalization.  _Advances in neural information processing systems_ , 33:4697–4708, 2020.
  * Wood and Shawe-Taylor [1996] Jeffrey Wood and John Shawe-Taylor.  Representation theory and invariant neural networks.  _Discrete Applied Mathematics_ , 69(1):33–60, 1996.
  * Wu et al. [2020] Sen Wu, Hongyang Zhang, Gregory Valiant, and Christopher Re.  On the generalization effects of linear transformations in data augmentation.  In Hal Daumé III and Aarti Singh, editors, _Proceedings of the 37th International Conference on Machine Learning (ICML)_ , volume 119 of _Proceedings of Machine Learning Research_ , pages 10410–10420. PMLR, 13–18 Jul 2020.
  * Yang and Schoenholz [2017] Ge Yang and Samuel Schoenholz.  Mean field residual networks: On the edge of chaos.  In I. Guyon, U. Von Luxburg, S. Bengio, H. Wallach, R. Fergus, S. Vishwanathan, and R. Garnett, editors, _Advances in Neural Information Processing Systems_ , volume 30. Curran Associates, Inc., 2017.
  * Yang et al. [2022] Ge Yang, Anurag Ajay, and Pulkit Agrawal.  Overcoming the spectral bias of neural value approximation.  In _International Conference on Learning Representations (ICLR)_ , 2022.
  * Yang and Hu [2021] Greg Yang and Edward J Hu.  Tensor programs iv: Feature learning in infinite-width neural networks.  In _Proceedings of the 38th International Conference on Machine Learning (ICML)_ , pages 11727–11737. PMLR, 2021.
  * Yarats and Kostrikov [2020] Denis Yarats and Ilya Kostrikov.  Soft actor-critic (sac) implementation in pytorch.  https://github.com/denisyarats/pytorch_sac, 2020.
  * Ye and Lim [2016] Ke Ye and Lek-Heng Lim.  Schubert varieties and distances between subspaces of different dimensions.  _SIAM Journal on Matrix Analysis and Applications_ , 37(3):1176–1197, 2016.
  * Yin et al. [2019] Mingzhang Yin, George Tucker, Mingyuan Zhou, Sergey Levine, and Chelsea Finn.  Meta-learning without memorization.  _International Conference on Learning Representations (ICLR)_ , 2019\.
  * Zaidi et al. [2022] Sheheryar Zaidi, Tudor Berariu, Hyunjik Kim, Jörg Bornschein, Claudia Clopath, Yee Whye Teh, and Razvan Pascanu.  When does re-initialization work?  _arXiv preprint arXiv:2206.10011_ , 2022.
  * Zhang et al. [2018a] Amy Zhang, Nicolas Ballas, and Joelle Pineau.  A dissection of overfitting and generalization in continuous reinforcement learning.  _arXiv preprint_ , 2018a.
  * Zhang et al. [2018b] Amy Zhang, Yuxin Wu, and Joelle Pineau.  Natural environment benchmarks for reinforcement learning.  _arXiv preprint_ , abs/1811.06032, 2018b.
  * Zhang et al. [2019] Amy Zhang, Zachary C. Lipton, Luis Pineda, Kamyar Azizzadenesheli, Anima Anandkumar, Laurent Itti, Joelle Pineau, and Tommaso Furlanello.  Learning causal state representations of partially observable environments.  _Reinforcement Learning and Decision Making_ , 2019.
  * Zhang et al. [2020a] Amy Zhang, Clare Lyle, Shagun Sodhani, Angelos Filos, Marta Kwiatkowska, Joelle Pineau, Yarin Gal, and Doina Precup.  Invariant causal prediction for block MDPs.  In _Proceedings of the 37th International Conference on Machine Learning (ICML)_ , 2020a.
  * Zhang et al. [2020b] Amy Zhang, Rowan Thomas McAllister, Roberto Calandra, Yarin Gal, and Sergey Levine.  Learning invariant representations for reinforcement learning without reconstruction.  In _International Conference on Learning Representations (ICLR)_ , 2020b.
  * Zhang et al. [2017] Chiyuan Zhang, Samy Bengio, Moritz Hardt, Benjamin Recht, and Oriol Vinyals.  Understanding deep learning requires rethinking generalization.  In _International Conference on Learning Representations (ICLR)_ , 2017.
  * Zhang et al. [2018c] Chiyuan Zhang, Oriol Vinyals, Remi Munos, and Samy Bengio.  A study on overfitting in deep reinforcement learning.  _arXiv preprint_ , 2018c.
  * Zhao et al. [2018] R. Zhao, D. Wang, R. Yan, K. Mao, F. Shen, and J. Wang.  Machine health monitoring using local feature-based gated recurrent unit networks.  _IEEE Transactions on Industrial Electronics_ , 65(2):1539–1548, Feb 2018.
  * Zhou and Troyanskaya [2015] Jian Zhou and Olga G Troyanskaya.  Predicting effects of noncoding variants with deep learning-based sequence model.  _Nature Methods_ , 12(10):931, 2015.
  * Zhou et al. [2019] Wenda Zhou, Victor Veitch, Morgane Austern, Ryan P. Adams, and Peter Orbanz.  Non-vacuous generalization bounds at the imagenet scale: a PAC-Bayesian compression approach.  _International Conference on Learning Representations (ICLR)_ , 2019\.
  * Zilly et al. [2021] Julian G. Zilly, Alessandro Achille, Andrea Censi, and Emilio Frazzoli.  On plasticity, invariance, and mutually frozen weights in sequential task learning (icml).  In _Advances in Neural Information Processing Systems_ , 2021.
  * Zintgraf et al. [2019] Luisa Zintgraf, Kyriacos Shiarli, Vitaly Kurin, Katja Hofmann, and Shimon Whiteson.  Fast context adaptation via meta-learning (icml).  In _International Conference on Machine Learning (ICML)_ , pages 7693–7702. PMLR, 2019.
  * Zou et al. [2012] Will Zou, Shenghuo Zhu, Kai Yu, and Andrew Y Ng.  Deep learning of invariant features via simulated fixations in video.  In _Advances in Neural Information Processing Systems_ , pages 3203–3211, 2012.


# HydroNet: Benchmark Tasks for Preserving Intermolecular Interactions and
Structural Motifs in Predictive and Generative Models for Molecular Data

Sutanay Choudhury
<EMAIL_ADDRESS>
&Jenna A. Bilbrey
<EMAIL_ADDRESS>
&Logan Ward
<EMAIL_ADDRESS>
&Sotiris S. Xantheas
<EMAIL_ADDRESS>
&Ian Foster
<EMAIL_ADDRESS>
&Joseph P. Heindel
<EMAIL_ADDRESS>
&Ben Blaiszik
<EMAIL_ADDRESS>
&Marcus E. Schwarting
<EMAIL_ADDRESS>

###### Abstract

Intermolecular and long-range interactions are central to phenomena as diverse
as gene regulation, topological states of quantum materials, electrolyte
transport in batteries, and the universal solvation properties of water. We
present a set of challenge problems 111https://exalearn.github.io/hydronet for
preserving intermolecular interactions and structural motifs in machine-
learning approaches to chemical problems, through the use of a recently
published dataset of 4.95 million water clusters held together by hydrogen
bonding interactions and resulting in longer range structural patterns. The
dataset provides spatial coordinates as well as two types of graph
representations, to accommodate a variety of machine-learning practices.

##  1 Introduction

The application of machine-learning (ML) techniques such as supervised
learning and generative models in chemistry is an active research area. ML-
driven prediction of chemical properties and generation of molecular
structures with tailored properties have emerged as attractive alternatives to
expensive computational methods [20, 24, 23, 32, 31, 7, 14, 16, 22]. Though
increasingly used, graph representations of molecules often do not explicitly
include non-covalent interactions such as hydrogen bonding, which poses
difficulties when examining systems with intermolecular and/or long-range
interactions [10]. To facilitate the development of such methods, we discuss a
set of challenge problems and suggest an approach based on a recently
published database of low-energy water cluster minima lying within 5 kcal/mol
from the putative minimum of each cluster size [17].

Scientific motivation: A water cluster is a discrete hydrogen bonded network
of water molecules. While most interactions are short-range (i.e., between
neighboring molecules) [19], there also exist substantial (~20%) many-body,
longer-range interactions (i.e., with next-nearest and more distant neighbors)
[30]. Understanding many-body and long-range hydrogen bonding interactions is
key to answering long-standing scientific questions such as the macroscopic
properties of liquid water, ice, and aqueous systems (e.g., heat capacity,
density, dielectric constant, compressibility) [13]. These interactions are
responsible for the bulk and interfacial properties of liquid water and ice,
as well as solvation processes, and are key to the realization of diverse
applications, from drug delivery to protein folding and the design of quantum
materials and novel electrolytes for batteries [15, 28, 9].

Figure 1: Data model: geometries provided as spatial coordinates. In the graph
representation nodes are atoms (Atomic) or water molecules (Coarse) and edges
are covalent and/or hydrogen bonds.

HydroNet summary: The dataset of 4.95 million water cluster minima is the
largest collection of water cluster minima reported to date [17]. Originally
created to advance the development of interaction potentials in chemical
physics, the dataset is composed of clusters of isomers differing in the
underlying hydrogen bonding network. Each cluster is described by its
potential energy, Cartesian coordinates of each atom, and two graph-based
representations of their bonding arrangement (Fig. 1): an _atomic interaction
graph_ that captures both intramolecular and intermolecular bonding patterns
and a _coarse graph_ that captures only the intermolecular structure of the
cluster.

ML tasks on the dataset: 1) Molecular Property Prediction: given a water
cluster with specified spatial coordinate information or bonding structure,
predict its energy. 2) Molecule Generation: given _N_ water molecules,
generate candidate structures that conform to structural measures of low-
energy configurations described in the key challenges below.

Key challenges: A defining feature of water clusters is that numerous
dissimilar structures can have quite similar energies. In addition, for a
given set of spatial orientations of oxygen atoms in the cluster (oxygen
network), there exist numerous hydrogen bonding networks (Coarse graphs)
depending on the arrangement of the hydrogen atoms, which form hydrogen bonds
according to the Bernal–Fowler rules [3]. The structural properties of low-
energy hydrogen bonding networks – characterized by graph-theoretical measures
such as degree distribution, shortest path length, and distribution of
polygons (Fig. 2) – vary systematically with cluster size [4]. A generative
method that produces a water cluster network should be mindful of these
properties, as clusters far outside the distributions are likely to be much
higher in energy and therefore of less interest.

Figure 2: Graph-based measures capturing intermolecular interactions and
structural motifs in water cluster networks. Clear patterns such as the
average number of neighbors (left), the shortest path between two water
molecules (middle), and the predominance of pentamer and hexamer polygons over
tetramer polygons (right) emerge as the cluster size increases [4].

##  2 Dataset Description

The potential energy of each cluster was obtained using the ab initio-based
Thole-type, flexible, polarizable interaction potential for water (TTM2.1-F)
[6, 8]. Clusters were generated with the Monte Carlo Temperature Basin Paving
(MCTBP) sampling method to produce a dense sampling of low-energy water
clusters containing 3–30 water molecules per cluster [17]. The water cluster
minima dataset [2] is represented in the three formats shown in Fig. 1, each
as both line-delimited JavaScript Object Notation (JSON) and Tensorflow
Protobufs. Table 1 lists the information available for each sample. The
records are separated into predefined train (80%), validation (10%), and test
sets (10%), where clusters are maintained in the same subset for each data
format. We also provide code to compute graph descriptors for structural motif
tracking [1].

Table 1: Records in HydroNet are stored as dictionary objects with the below keys. Key |  Type |  Description
---|---|---
energy |  float |  Energy of the cluster, in kcal/mol.
n_water |  int |  Number of water molecules.
n_atom |  int |  Number of atoms or, for coarse graphs, nodes.
atom |  list of ints |  Types of each atom or node in the graph. Atomic: 0 is oxygen, 1 is hydrogen; coarse: all nodes are the same type.
z |  list of ints |  Geometry only. Atomic number of each atom.
coords |  $N\times 3$ array of floats |  Geometry only. XYZ coordinates of atoms, in angstrom.
n_bond |  list of ints |  Graph only. Number of edges in the graph.
bond |  list of ints |  Graph only. Types of edges. Atomic: 0 is covalent, 1 is hydrogen; coarse: 0 is donor, 1 is acceptor.
connectivity |  $N\times 2$ array of ints |  Graph only. Connectivity between nodes in the graph.

##  3 Machine-Learning Tasks

We introduce the defined property prediction and generative modeling tasks on
the water cluster dataset and provide details on baseline implementations for
the first task.

###  3.1 Cluster Potential Energy Prediction Task

This task is to predict the potential energy of a water cluster without the
use of expensive ab initio methods, in two settings. In the geometry-to-energy
[21] setting, we assume the geometry of the structure is known accurately. In
the graph-to-energy [11, 26] setting, we require predictions to be made from
the connectivity of the water molecules only.

We approach these tasks using neural network potentials, in which the original
state of each atom ($h_{v}$) and bond ($\alpha_{vw}$) is represented as a
vector embedding based on atomic number and bond type. These states are
modified by successive message layers. Each message layer uses a multi-layer
perceptron to compute a message from the atom state ($h_{v}$), a neighboring
atom state ($h_{w}$), and the connecting bond ($\alpha_{vw}$). The atom and
bond states are updated according to the following equations, which generalize
both the geometry-to-energy and graph-to-energy prediction settings described
below. In the first, an atom’s neighbors are those within a radial cutoff
distance; in the second, neighbors are represented by the adjacency matrix of
the network.

$m^{t+1}_{v}=\sum_{w\in\emph{Nbors}(v)}M_{t}(h^{t}_{v},h^{t}_{w},\alpha^{t}_{vw})\
\ \ \ \ \ \ \ \ h^{t+1}_{v}=h^{t}_{v}+m^{t+1}_{v}\ \ \ \ \ \ \ \ \
\alpha^{t+1}_{vw}=\alpha^{t}_{vw}+M_{t}(h^{t}_{v},h^{t}_{w},\alpha^{t}_{vw})$

Figure 3: Performance of (top) SchNet trained on 500,000 water clusters, _N_
=11–29, and tested on 10,500 clusters, _N_ =10–30, and (bottom) MPNN trained
on HydroNet training set and evaluated on test set. SchNet performance data
from [4].

We sum the energy per atom values produced by the penultimate layer to obtain
the full cluster energy.

For the geometry-to-energy problem, we previously [4] trained a continuous-
filter convolutional neural network (SchNet, PyTorch implementation [20]) on
500,000 water clusters of size _N_ =11–29. The training set was stratified by
cluster size, from which 10% were reserved for validation during training. The
test set was composed of 10,500 clusters, with 500 from each cluster size
N=10–30. Clusters of size _N_ =10, 30 allowed us to examine the effects of
energy prediction on clusters smaller and larger than those in the training
set. Using mean squared error (MSE) as the loss, we achieved a final training
loss of 0.0030 (kcal/mol)2 and final validation loss of 0.0035 (kcal/mol)2.
The mean absolute error (MAE) of the test set predictions was 0.0427 kcal/mol.
The network took $\sim$65 hours to train using 4 NVIDIA V100 GPUs.

For the graph-to-energy problem, we use a message-passing neural network
(MPNN) inspired by [11] to create a reference implementation [26]. A first
performance study shows that models trained on coarse networks achieve losses
an order of magnitude superior to those trained on atomic networks, but
exhibit significantly worse performance on the test set. Such generalization
issues suggest opportunities to explore regularization techniques or new
network designs to take advantage of coarse graphs.

Our best models have a MAE of $\sim$0.5 kcal/mol/water. Errors are nearly
equal for clusters of sizes 10–30, giving us confidence that the MPNN is
capturing the changes in favored geometries as clusters increase in size.
However, these errors are around a factor of 100 higher than those achieved
with SchNet, which incorporates knowledge of the atomic coordinates. In short,
we find conventional MPNN approaches to be promising solutions for energy
prediction, but significant work is needed by the ML community to find models
suitable for evaluating systems with intermolecular and/or long-range
interactions.

###  3.2 Structural Measure-Preserving Molecular Generation Task

We define the task as follows: given _N_ water molecules, generate a
geometric/atomic/coarse representation that a) satisfies certain graph-
theoretic structural measures derived from its hydrogen bonding network and b)
minimizes the cluster energy by optimizing the relative spatial arrangements
of atoms and molecules. As an example, Fig. 4 shows a sequence of
transformations that would yield a more stable water cluster from a static
number of molecules.

Figure 4: Generation of a low-energy water network from a higher energy
structure.

Structural measures vary with cluster size, as shown in Fig. 2. The mean
degree (number of neighbors) of water molecules in a hydrogen bonded network
increases and eventually plateaus when water molecules become saturated with
hydrogen bonds. The shortest path length also increases with cluster size,
through the growth rate decays. The number of cycles in the hydrogen bonded
network also show trends as clusters grow in size: trimers (3-cycles) and
tetramers (4-cycles) become less dominant, while pentamers (5-cycles) and
hexamers (6-cycles) emerge as key building blocks. We also observe a strong
evolution in geometric structure (Fig. 5). The water cluster structures start
as quasi-planar graphs (_N_ =3-5), evolve into cubic graphs (_N_ =7–16), and
then into cage-like structures (_N_ =17 onward), with some structures being
quasi-symmetric (i.e. for _N_ =20).

##  4 Relationship with Other Work

Figure 5: Geometric shape progression of low-energy structures with cluster
size _n_. Figure reproduced with permission from [17].

Generative models and deep reinforcement learning methods that produce
molecular structures with desired properties are active research areas [31,
32, 7, 12, 16, 22]. However, most ML-benchmark datasets [18, 27, 29] used by
these studies map from spatial or graph structure to properties of interest
(e.g, drug-likeliness, binding affinity between a drug molecule and a target
protein). Modeling the interaction of heterogeneous functional groups and
reasoning about potential reaction mechanisms is the primary focus for such
tasks. Our dataset and science challenge tasks require methods that prioritize
satisfying topological constraints, addressing challenges with stereoisomers,
and modeling variations in interaction patterns at multiple scales.

The multi-scale nature of the interactions of our dataset also fills a void in
existing scientific datasets. Datasets which map molecular geometry or
molecular graphs, available for years, capture challenges in which properties
are driven by short-range covalent interactions [25, 29, 18]. There are also
datasets of condensed phases (e.g., AGNI [5]) in which long-range, many-body
interactions become important. Our dataset presents a complex mixture of
short-range covalent, intermolecular hydrogen bonding, and extended many-body
effects. We propose this dataset as a basis for the challenges of learning
such effects from 3D geometries and inferring them from bonding structure.

##  5 Conclusions

We present challenge problems and a corresponding dataset to advance machine-
learning on molecular data. An important feature of the dataset discussed here
is the incorporation of intermolecular interactions, which allows for two
distinct graph representations in addition to the spatial coordinates. Our
initial benchmarks illustrate a large performance gap between neural networks
based on atomic coordinates and those which use only the bonding structure,
which make this benchmark particularly relevant for advancing graph neural
network-based generative and prediction models. Moreover, this dataset is
useful for the creation of novel machine-learning methods that preserve
structural patterns across a wide size regime and can address the key
challenges of property prediction and molecular generation.

##  6 Broader Impact

The problems addressed here have the potential to advance methods for modeling
biological and chemical systems, leading to vast speed-ups in computation that
would accelerate accurate simulations of large, complex systems. These speed-
ups will allow domain researchers to more quickly assess the properties of and
generate potential structures for novel molecular systems with phenomena
dependant on long-range interactions. Such research is the beginning stage to
making practicable, real-world scientific advancements.

##  7 Acknowledgements

S.C., J.A.B., L.W., S.S.X., I.F., J.P.H., and M.S. were supported by the DOE
Exascale Computing Project, ExaLearn Co-design Center. B.B. was was supported
by the National Science Foundation under NSF Award Number: 1931306
“Collaborative Research: Framework: Machine Learning Materials Innovation
Infrastructure.” This research used resources of the Argonne Leadership
Computing Facility, which is a DOE Office of Science User Facility supported
under Contract DE-AC02-06CH11357. Pacific Northwest National Laboratory (PNNL)
is a multi-program national laboratory operated for the US Department of
Energy by Battelle. Argonne National Laboratory’s work was supported by the
U.S. Department of Energy, Assistant Secretary for Environmental Management,
Office of Science and Technology, under contract DE-AC02-06CH11357.

## References

  * [1] Code repository for computing structural motifs.  https://github.com/exalearn/molecular-graph-descriptors.
  * [2] Database of water clusters.  https://sites.uw.edu/wdbase.
  * [3] J. D. Bernal and R. H. Fowler.  A theory of water and ionic solution, with particular reference to hydrogen and hydroxyl ions.  The Journal of Chemical Physics, 1(8):515–548, 1933.
  * [4] Jenna A. Bilbrey, Joseph P. Heindel, Malachi Schram, Pradipta Bandyopadhyay, Sotiris S. Xantheas, and Sutanay Choudhury.  A look inside the black box: Using graph-theoretical descriptors to interpret a Continuous-Filter Convolutional Neural Network (CF-CNN) trained on the global and local minimum energy structures of neutral water clusters.  The Journal of Chemical Physics, 153(2):024302, 2020.
  * [5] V. Botu, R. Batra, J. Chapman, and R. Ramprasad.  Machine learning force fields: Construction, validation, and outlook.  The Journal of Physical Chemistry C, 121(1):511–522, December 2016\.
  * [6] Christian J Burnham and Sotiris S Xantheas.  Development of transferable interaction models for water. IV. A flexible, all-atom polarizable potential (TTM2-F) based on geometry dependent charges derived from an ab initio monomer dipole moment surface.  The Journal of Chemical Physics, 116(12):5115–5124, 2002.
  * [7] Nicola De Cao and Thomas Kipf.  Molgan: An implicit generative model for small molecular graphs.  arXiv preprint arXiv:1805.11973, 2018.
  * [8] George S Fanourgakis and Sotiris S Xantheas.  The flexible, polarizable, Thole-type interaction potential for water (TTM2-F) revisited.  The Journal of Physical Chemistry A, 110(11):4100–4106, 2006.
  * [9] Roger H French, V Adrian Parsegian, Rudolf Podgornik, Rick F Rajter, Anand Jagota, Jian Luo, Dilip Asthagiri, Manoj K Chaudhury, Yet-ming Chiang, Steve Granick, et al.  Long range interactions in nanoscale science.  Reviews of Modern Physics, 82(2):1887, 2010.
  * [10] Niklas Gebauer, Michael Gastegger, and Kristof Schütt.  Symmetry-adapted generation of 3d point sets for the targeted discovery of molecules.  In Advances in Neural Information Processing Systems, pages 7566–7578, 2019.
  * [11] Justin Gilmer, Samuel S Schoenholz, Patrick F Riley, Oriol Vinyals, and George E Dahl.  Neural message passing for quantum chemistry.  arXiv preprint arXiv:1704.01212, 2017.
  * [12] Wengong Jin, Regina Barzilay, and Tommi Jaakkola.  Composing molecules with multiple property constraints.  arXiv preprint arXiv:2002.03244, 2020.
  * [13] Frank N Keutsch and Richard J Saykally.  Water clusters: Untangling the mysteries of the liquid, one molecule at a time.  Proceedings of the National Academy of Sciences, 98(19):10533–10540, 2001.
  * [14] Yash Khemchandani, Steve O’Hagan, Soumitra Samanta, Neil Swainston, Timothy J Roberts, Danushka Bollegala, and Douglas B Kell.  Deepgraphmol, a multi-objective, computational strategy for generating molecules with desirable properties: a graph convolution and reinforcement learning approach.  bioRxiv, 2020.
  * [15] Erez Lieberman-Aiden, Nynke L Van Berkum, Louise Williams, Maxim Imakaev, Tobias Ragoczy, Agnes Telling, Ido Amit, Bryan R Lajoie, Peter J Sabo, Michael O Dorschner, et al.  Comprehensive mapping of long-range interactions reveals folding principles of the human genome.  Science, 326(5950):289–293, 2009.
  * [16] Kaushalya Madhawa, Katushiko Ishiguro, Kosuke Nakago, and Motoki Abe.  Graphnvp: An invertible flow model for generating molecular graphs.  arXiv preprint arXiv:1905.11600, 2019.
  * [17] Avijit Rakshit, Pradipta Bandyopadhyay, Joseph P. Heindel, and Sotiris S. Xantheas.  Atlas of putative minima and low-lying energy networks of water clusters n = 3–25.  The Journal of Chemical Physics, 151(21):214307, 2019.
  * [18] Raghunathan Ramakrishnan, Pavlo O Dral, Matthias Rupp, and O Anatole von Lilienfeld.  Quantum chemistry structures and properties of 134 kilo molecules.  Scientific Data, 1, 2014.
  * [19] Laurent Schaeffer.  Chapter 14 - the role of functional groups in drug–receptor interactions.  In Camille Georges Wermuth, David Aldous, Pierre Raboisson, and Didier Rognan, editors, The Practice of Medicinal Chemistry (Fourth Edition), pages 359 – 378. Academic Press, San Diego, fourth edition edition, 2008.
  * [20] K. T. Schütt, P. Kessel, M. Gastegger, K. A. Nicoli, A. Tkatchenko, and K.-R. Müller.  Schnetpack: A deep learning toolbox for atomistic systems.  Journal of Chemical Theory and Computation, 15(1):448–455, 2019\.
  * [21] K. T. Schütt, H. E. Sauceda, P.-J. Kindermans, A. Tkatchenko, and K.-R. Müller.  Schnet – a deep learning architecture for molecules and materials.  The Journal of Chemical Physics, 148(24):241722, 2018.
  * [22] Chence Shi, Minkai Xu, Zhaocheng Zhu, Weinan Zhang, Ming Zhang, and Jian Tang.  Graphaf: a flow-based autoregressive model for molecular graph generation.  ICLR, 2020.
  * [23] Andreas Singraber, Tobias Morawietz, Jörg Behler, and Christoph Dellago.  Parallel multistream training of high-dimensional neural network potentials.  Journal of Chemical Theory and Computation, 15(5):3075–3092, 2019\.  PMID: 30995035.
  * [24] J. S. Smith, O. Isayev, and A. E. Roitberg.  Ani-1: an extensible neural network potential with dft accuracy at force field computational cost.  Chem. Sci., 8:3192–3203, 2017.
  * [25] Justin S. Smith, Olexandr Isayev, and Adrian E. Roitberg.  ANI-1, a data set of 20 million calculated off-equilibrium conformations for organic molecules.  Scientific Data, 4(1), December 2017.
  * [26] Peter C St. John, Caleb Phillips, Travis W Kemper, A Nolan Wilson, Yanfei Guan, Michael F Crowley, Mark R Nimlos, and Ross E Larsen.  Message-passing neural networks for high-throughput polymer screening.  The Journal of Chemical Physics, 150(23):234111, 2019.
  * [27] Teague Sterling and John J Irwin.  ZINC 15–ligand discovery for everyone.  Journal of Chemical Information and <odeling, 55(11):2324–2337, 2015.
  * [28] Nevena Veljkovic, Sanja Glisic, Vladimir Perovic, and Veljko Veljkovic.  The role of long-range intermolecular interactions in discovery of new drugs.  Expert Opinion on Drug Discovery, 6(12):1263–1270, 2011.
  * [29] Zhenqin Wu, Bharath Ramsundar, Evan N Feinberg, Joseph Gomes, Caleb Geniesse, Aneesh S Pappu, Karl Leswing, and Vijay Pande.  MoleculeNet: A benchmark for molecular machine learning.  Chemical Science, 9(2):513–530, 2018.
  * [30] Sotiris S Xantheas.  Cooperativity and hydrogen bonding network in water clusters.  Chemical Physics, 258(2):225 – 231, 2000.
  * [31] Jiaxuan You, Bowen Liu, Zhitao Ying, Vijay Pande, and Jure Leskovec.  Graph convolutional policy network for goal-directed molecular graph generation.  In Advances in Neural Information Processing Systems, pages 6410–6421, 2018.
  * [32] Zhenpeng Zhou, Steven Kearnes, Li Li, Richard N. Zare, and Patrick Riley.  Optimization of molecules via deep reinforcement learning.  Scientific Reports, 9(1):10752, 2019.