Lyapunov exponents in $\mathbf{{\cal N}=2}$ supersymmetric Jackiw-Teitelboim
gravity

Ruben Campos Delgado and Stefan Förste

Bethe Center for Theoretical Physics
and
Physikalisches Institut der Universität Bonn,
Nussallee 12, 53115 Bonn, Germany

††footnotetext<EMAIL_ADDRESS><EMAIL_ADDRESS>
Abstract

We study $\mathcal{N}=2$ supersymmetric Jackiw-Teitelboim (JT) gravity at
finite temperature coupled to matter. The matter fields are related to
superconformal primaries by AdS/CFT duality. Due to broken super
reparametrisation invariance in the SCFT dual, there are corrections to
superconformal correlators. These are generated by the exchange of super-
Schwarzian modes which is dual to the exchange of 2D supergravity modes. We
compute corrections to four-point functions for superconformal primaries and
analyse the behaviour of out-of-time-ordered correlators. In particular, four-
point functions of two pairs of primaries with mutually vanishing two-point
functions are considered. By decomposing the corresponding supermultiplet into
its components, we find different Lyapunov exponents. The value of the
Lyapunov exponents depends on whether the correction is due to graviton,
gravitini or graviphoton exchange. If mutual two-point functions do not vanish
all components grow with maximal Lyapunov exponent.

##  1 Introduction

Lower dimensional instances of the AdSD/CFTD-1 correspondence [1] have
attracted a wide interest in recent years. The case $D=2$ is important from
the point of view of black hole physics because AdS2 is the near-horizon
geometry of extremal (e.g. supersymmetric) black holes with non-zero entropy.
A known problem is that pure gravity in AdS2 is inconsistent with the
existence of finite energy excitations above the AdS2 vacuum [2, 3, 4]. The
problem can be overcome by considering a deformed version of the AdS2 space.
In Euclidean signature, AdS2 is the hyperbolic disk. The idea is to slightly
deform the boundary of the disk by means of a UV brane, i.e. a curve that
reduces to the old boundary as soon as a small parameter is sent to zero,
without changing the AdS2 geometry. In this nearly AdS2 space (nAdS2) it is
possible to define a consistent 2-dimensional theory of gravity, the Jackiw-
Teitelboim (JT) theory [5, 6], coupled to a dilaton field. The dilaton field
enters as a Lagrange multiplier enforcing negative constant curvature. After
imposing the constraint, JT gravity reduces to an effective action involving
the Schwarzian derivative of a reparametrisation of the coordinate describing
the nAdS2 boundary [7, 8, 9, 10]:

$S_{\text{sch}}=-C\int du\,\text{Sch}\left[t(u);u\right],$ (1)

where $C$ is inversely proportinal to Newton’s constant $G_{N}$ and

$\text{Sch}\left[t(u);u\right]=\frac{2\dot{t}t^{(3)}-3\ddot{t\,}^{2}}{2\dot{t\,}^{2}}.$
(2)

The JT theory has surprisingly some features in common with a completely
different model, originally studied in condensed matter physics, the SYK model
[11, 12]. It describes $N$ fermions interacting via random couplings. The
model is solvable in the large $N$ limit, is chaotic and is conformally
invariant in the low energy regime [12, 13, 14, 15]. Moving away from the IR
region breaks the conformal symmetry, resulting in an effective action for
time reparametrisations given again by the Schwarzian. This corroborates JT
gravity as the holographic dual of the SYK low-energy region. To be precise,
this is an example of a nearly AdS2/nearly CFT1 correspondence. In the same
spirit of the usual “holographic dictionary” of the canonical AdS/CFT
correspondence [16], matter fields in nAdS2 correspond to conformal primaries
in the conformal limit of the SYK model. More precisely, correlation functions
of conformal primaries are obtained by taking variational derivatives of the
gravity partition function with respect to boundary conditions of matter
fields. The breaking of conformal symmetry is characterised by a Schwarzian
action for reparametrisations of the boundary. These are responsible for
corrections to correlation functions signaling the occurence of quantum chaos.
In a chaotic system, small differences in the initial conditions lead to large
and unpredictable differences in the dynamics. In a quantum theory, a
diagnosis of chaos is offered by the behaviour of a special class of
correlators, the out-of-time-ordered correlators (OTOC). If the system is
chaotic they show an exponential behaviour with rate given by a real number
$\lambda$ called Lyapunov exponent. For a causal and unitary QFT it was argued
that the Lyapunov exponent is bounded by a maximal value [17]. Calculations
carried in JT gravity [9] and in the SYK model [12, 18] gave the maximal value
of Lyapunov exponent for both theories, thereby supporting the duality between
them.

Both the JT theory and the SYK model can be generalised by adding
supersymmetry. In [19] it was shown that the $\mathcal{N}=1$ and
$\mathcal{N}=2$ supersymmetric versions of the SYK model lead to super-
Schwarzian actions. Similarly to the purely bosonic theory, the super-
Schwarzian also arises as an effective action for a UV regulator brane in
nAdS2 supergravity, for both the $\mathcal{N}=1$ [20] and $\mathcal{N}=2$ [21]
supersymmetric JT theories. This suggests that the JT/SYK duality is still
preserved after adding supersymmetry.

Lyapunov exponents were considered for ${\cal N}=1$ JT super gravity in [22,
23]. For ${\cal N}=2$ they were computed in [24] within the SYK model. Here,
we carry out the holographic computation for ${\cal N}=2$ and obtain results
similar to [22]. Whenever the leading correction to the four-point functions
contains graviton exchange the Lyapunov exponent saturates the bound
$2\pi/\beta$. There are also components of the four-point functions for which
the leading correction does not contain graviton exchange. If gravitino
exchange contributes, their Lyapunov exponent will be $\pi/\beta$ whereas for
combinations with graviphoton contributions only there will be just linear
growth with time.

We briefly outline the procedure that we are going to follow. Firstly, we put
the theory at finite temperature and compute the corresponding super-
Schwarzian action. Secondly, we expand the reparametrisations of the super-
boundary in terms of small fluctuations. Subsequently, we put matter fields
into the nAdS2 space and compute the four-point functions of the dual
operators, contracting the appearances of the fluctuations using their
propagators. Using its definition, we are eventually able to extract the
Lyapunov exponents from the four-point functions. The same procedure was
carried out for the purely bosonic JT theory in [9] and reviewed in [25]. The
paper is organised as follows. In Section 2 we review the $\mathcal{N}=2$
superspace formalism and compute the super-Schwarzian at finite temperature.
In Section 3 we derive the propagators of the fluctuations. In Section 4 we
couple the theory to matter. In Section 5 we compute the thermal four-point
functions for operators dual to the matter fields and determine the Lyapunov
exponents.

##  2 The $\mathcal{N}=2$ superspace formalism

In the purely bosonic JT theory, the field $t(u)$ is the only dynamical
variable describing reparametrisations of the boundary in the nAdS2 space. In
a supersymmetric theory, we have to add one or more fermionic fields. The
superspace formalism offers a simple way to keep track of all the relevant
degrees of freedom and to make supersymmetry manifest. Within this framework,
space is described in terms of the usual bosonic coordinates and by additional
Grassmann variables. In a generic $\mathcal{N}=m$ theory, super-
reparametrisations of the boundary are described by an $(m+1)$-tuple
$\left(t,\xi_{1},\dots,\xi_{m}\right)$ where $t$ respectively $\xi_{i}$ is a
commuting respectively anti-commuting superfield satisfying additional
constraints which can be viewed as one-dimensional versions of superconformal
Killing equations.

###  2.1 $\mathcal{N}=2$ superspace and super-Schwarzian

We briefly review definitions of ${\cal N}=2$ super-diffeomorphisms following
conventions in [19]. The $\mathcal{N}=2$ one-dimensional superspace, or super-
line, is parametrised by a bosonic variable $u$ and Grassmann variables
$\theta$ and $\overline{\theta}$. The super-translation group consists of the
transformations

$u\mapsto\tau=u+\epsilon+\theta\overline{\eta}+\overline{\theta}\eta,\hskip
11.38109pt\theta\mapsto\xi=\theta+\eta,\hskip
11.38109pt\overline{\theta}\mapsto\overline{\xi}=\overline{\theta}+\overline{\eta}.$
(3)

where $\epsilon(u)$ is a bosonic function and $\eta(u)$, $\overline{\eta}(u)$
are fermionic functions. Super-derivative operators are defined as

$D_{\theta}=\frac{\partial}{\partial\theta}+\overline{\theta}\frac{\partial}{\partial
u},\hskip
17.07164pt\overline{D}_{\overline{\theta}}=\frac{\partial}{\partial\overline{\theta}}+\theta\frac{\partial}{\partial
u}.$ (4)

Their anticommutator is

$\\{D_{\theta},\overline{D}_{\overline{\theta}}\\}=2\partial_{u}.$ (5)

Fermions and scalars can be packaged into a chiral superfield, i.e. a field
$\Psi\left(u,\theta,\overline{\theta}\right)$ constrained to satisfy

$\overline{D}_{\overline{\theta}}\Psi\left(u,\theta,\overline{\theta}\right)=0,$
(6)

which is solved by

$\Psi\left(u,\theta,\overline{\theta}\right)=\Gamma\left(u+\theta\overline{\theta}\right)+\theta
b(u)$ (7)

for a fermion $\Gamma$ and boson $b$. The conjugate $\overline{\Psi}$ and
$D_{\theta}\Psi$ are instead anti-chiral field, namely they are annihilated by
$D_{\theta}$.

Transormations in (3) are super-reparametrisations if they satisfy the
constraints

$\begin{split}&D_{\theta}\overline{\xi}=0,\hskip
22.76219ptD_{\theta}\tau=\overline{\xi}D_{\theta}\xi,\\\
&\overline{D}_{\overline{\theta}}\xi=0,\hskip
22.76219pt\overline{D}_{\overline{\theta}}\tau=\xi\overline{D}_{\overline{\theta}}\overline{\xi}.\end{split}$
(8)

Super-reparametrisations are described by an effective action which can be
obtained from a supersymmetric SYK model [19] as well as from supersymmetric
JT gravity [21]

$S_{\text{eff}}=-\frac{1}{2\pi G_{N}}\int_{\partial M}dud\theta
d\overline{\theta}\,\phi_{b}\text{Sch}\left[\tau,\xi,\overline{\xi};u,\theta,\overline{\theta}\right],$
(9)

where $\phi_{b}$ is the boundary value for the leading component of the
dilaton $\Phi$. For convenience we rename

$\kappa=-\frac{\phi_{b}}{2\pi G_{N}}$ (10)

and consider constant $\phi_{b}$ which can be achieved by an appropriate
choice of the boundary. The functional
$\text{Sch}\left[\tau,\xi,\overline{\xi};u,\theta,\overline{\theta}\right]$ is
the $\mathcal{N}=2$ super-Schwarzian derivative [19]

$\text{Sch}\left[\tau,\xi,\overline{\xi};u,\theta,\overline{\theta}\right]=\frac{\partial_{u}\overline{D}_{\overline{\theta}}\overline{\xi}}{\overline{D}_{\overline{\theta}}\overline{\xi}}-\frac{\partial_{u}D_{\theta}\xi}{D_{\theta}\xi}-2\frac{\partial_{u}\xi\partial_{u}\overline{\xi}}{\left(D_{\theta}\xi\right)\left(\overline{D}_{\overline{\theta}}\overline{\xi}\right)},$
(11)

where $\tau$, $\xi$, $\overline{\xi}$ are super-reparametrisations of the
boundary of nAdS2 satisfying the constraints (8).

###  2.2 The $\mathcal{N}=2$ super-Schwarzian at finite temperature

We now put the theory at finite temperature $T=1/\beta$. This is achieved by
reparametrising the boundary time to a tangent,
$\tau\left(u\right)\mapsto\tan{\frac{2\pi}{\beta}\frac{\tau\left(u\right)}{2}}$
[9]. For simplicity we fix $\beta=2\pi$. Assuming that
$\left(\tau,\xi,\overline{\xi}\right)$ are super-reparametrisations it is easy
to check that

$\tau^{\prime}=\tan\frac{\tau}{2}\,\,\,,\,\,\,\xi^{\prime}=\frac{\xi}{\sqrt{2}\cos\frac{\tau}{2}}\,\,\,,\,\,\,\overline{\xi}^{\prime}=\frac{\overline{\xi}}{\sqrt{2}\cos\frac{\tau}{2}}.$
(12)

satisfy (8). The super-Schwarzian transforms according to the chain rule [19]

$\text{Sch}\left[\tau^{\prime},\xi^{\prime},\overline{\xi}^{\prime};u,\theta,\overline{\theta}\right]=\left(D_{\theta}\xi\right)\left(\overline{D}_{\overline{\theta}}\overline{\xi}\right)\text{Sch}\left[\tau^{\prime},\xi^{\prime},\overline{\xi}^{\prime};\tau,\xi,\overline{\xi}\right]+\text{Sch}\left[\tau,\xi,\overline{\xi};u,\theta,\overline{\theta}\right].$
(13)

This allows us to express the effective action in terms of fluctuations around
the zero temperature solution
$\left(\tau,\xi,\overline{\xi}\right)=\left(u,\theta,\overline{\theta}\right)$.
Fluctuations are described by two bosonic fields $\epsilon\left(u\right)$ and
$\sigma\left(u\right)$ and two fermionic fields $\eta\left(u\right)$ and
$\overline{\eta}\left(u\right)$. Solving (8) up to quadratic order in the
fluctuations yields 111This expression is obtained by considering the
infinitesimal limit of reparametrisation given in [19] in terms of independent
fields.

$\tau(u)=u+\epsilon(u)+\theta\overline{\eta}(u)\left[1+\dot{\epsilon}(u)\right]+\overline{\theta}\eta(u)\left[1+\dot{\epsilon}(u)\right]+\theta\overline{\theta}\left[\eta(u)\dot{\overline{\eta}}(u)-\overline{\eta}(u)\dot{\eta}(u)\right],$
(14)
$\begin{split}&\xi(u)=\eta(u)\left[1+\text{i}\sigma(u)+\frac{1}{2}\dot{\epsilon}(u)\right]+\theta\bigg{\\{}1+\overline{\eta}(u)\dot{\eta}(u)+\text{i}\sigma(u)-\frac{1}{2}\sigma^{2}(u)-\frac{1}{8}\dot{\epsilon}^{2}(u)+\\\
&+\frac{1}{2}\dot{\epsilon}(u)\left[1+\text{i}\sigma(u)\right]\bigg{\\}}+\theta\overline{\theta}\left\\{\dot{\eta}(u)\left[1+\text{i}\sigma(u)+\frac{1}{2}\dot{\epsilon}(u)\right]+\eta(u)\left[\text{i}\dot{\sigma}(u)+\frac{1}{2}\ddot{\epsilon}(u)\right]\right\\},\end{split}$
(15)
$\begin{split}\overline{\xi}(u)=\overline{\eta}(u)\left[1-\text{i}\sigma(u)+\frac{1}{2}\dot{\epsilon}(u)\right]+\overline{\theta}\bigg{\\{}1+\eta(u)\dot{\overline{\eta}}(u)-\text{i}\sigma(u)-\frac{1}{2}\sigma^{2}(u)-\frac{1}{8}\dot{\epsilon}^{2}(u)+\\\
+\frac{1}{2}\dot{\epsilon}(u)\left[1-\text{i}\sigma(u)\right]\bigg{\\}}-\theta\overline{\theta}\left\\{\dot{\overline{\eta}}(u)\left[1-\text{i}\sigma(u)+\frac{1}{2}\dot{\epsilon}(u)\right]+\overline{\eta}(u)\left[-\text{i}\dot{\sigma}(u)+\frac{1}{2}\ddot{\epsilon}(u)\right]\right\\}.\end{split}$
(16)

The field $\epsilon(u)$, which also appears in the purely bosonic JT gravity,
is interpreted as a boundary graviton [9]; the fermionic fields $\eta(u)$ and
$\overline{\eta}(u)$ are boundary gravitinos; finally, $\sigma(u)$ represents
the boundary degree of freedom of a graviphoton [26]. We can now express the
effective action for finite temperature super-reparametrisations in terms of
this gravity supermultiplet. Neglecting total derivatives and contributions of
higher than quadratic order in the fluctuations, we obtain

$\displaystyle S^{\mathcal{N}=2}_{\text{sch},\,\beta=2\pi}$
$\displaystyle=\kappa\int dud\theta
d\overline{\theta}\,\,\text{Sch}\left[\tau^{\prime},\xi^{\prime},\overline{\xi}^{\prime};u,\theta,\overline{\theta}\right]$
$\displaystyle=C\int_{0}^{2\pi}du\,\,\left\\{\frac{1}{2}\left[\ddot{\epsilon}^{2}(u)-\dot{\epsilon}^{2}(u)\right]+2\dot{\sigma}^{2}(u)+\left[\eta(u)\dot{\overline{\eta}}(u)-4\dot{\eta}(u)\ddot{\overline{\eta}}(u)\right]\right\\}.$
(17)

Here, we have used (12), (13) and (14), (15), (16). This action has been given
already in [27]. The form of the $\epsilon$ dependent contribution coincides
with the one obtained in the purely bosonic setting [9].

##  3 Propagators of the fluctuations

The propagator for $\epsilon$ was already obtained in [9]. Here, we closely
follow the derivation detailed in [25]. In a thermal field theory, time $u$ is
periodic. In our case, the period is $\beta=2\pi$. We start by expanding
fluctuations into Fourier series,

$\displaystyle\epsilon(u)=\sum_{n\in\mathbb{Z}}\epsilon_{n}\text{e}^{\text{i}nu}$
$\displaystyle,\,\,\,\sigma\left(u\right)=\sum_{n\in{\mathbb{Z}}}\sigma_{n}\text{e}^{\text{i}nu},$
$\displaystyle\eta(u)=\sum_{m\in\mathbb{Z}+\frac{1}{2}}\eta_{m}\text{e}^{\text{i}mu}$
$\displaystyle,\,\,\,\overline{\eta}(u)=\sum_{m\in\mathbb{Z}+\frac{1}{2}}\overline{\eta}_{m}\text{e}^{-\text{i}mu},$
(18)

where we have imposed periodic and anti-periodic boundary conditions on
bosonic and fermionic fields, respectively [28]. Plugging (18) into (17) gives

$S^{\mathcal{N}=2}_{\text{sch},\,\beta=2\pi}=\pi
C\left\\{\sum_{n\in\mathbb{Z}}\left[(n^{4}-n^{2})\epsilon_{n}\epsilon_{-n}+2n^{2}\sigma_{n}\sigma_{n}\right]+\sum_{m\in{\mathbb{Z}}+1/2}m\left(m^{2}-\frac{1}{4}\right)\eta_{m}\overline{\eta}_{m}\right\\}.$
(19)

In Fourier space, the propagators are

$\langle\epsilon_{n}\epsilon_{k}\rangle=\frac{\delta_{n+k}}{\pi
C}\frac{1}{n^{2}(n^{2}-1)},\,\,\,\,\,\,\langle\sigma_{n}\sigma_{k}\rangle=\frac{\delta_{n+k}}{4\pi
C}\frac{1}{n^{2}},\,\,\,\,\,\,\langle\eta_{m}\overline{\eta}_{k}\rangle=\frac{\delta_{m-k}}{2\pi\text{i}C}\frac{1}{m\left(4m^{2}-1\right)}.$
(20)

Divergencies due to zeros in the denominators are due to zero modes. These are
removed by gauge fixing the supersymmetric extension of
SL$\left(2,{\mathbb{R}}\right)$ called SU$\left(\left.1,1\right|1\right)$.
This is achieved by just not summing over the associated Fourier modes when
transforming back to position space. The corresponding sums can be expressed
by contour integrals,

$\displaystyle\langle\epsilon(u)\epsilon(0)\rangle$
$\displaystyle=\frac{1}{\pi C}\oint_{{\cal
C}_{\epsilon}}dz\,\,\frac{1}{\text{e}^{2\pi\text{i}z}-1}\frac{\text{e}^{izu}}{z^{2}(z^{2}-1)},\,\,\,\,\,\,\langle\sigma(u)\sigma(0)\rangle=\frac{1}{4\pi
C}\oint_{{\cal
C}_{\sigma}}dz\,\,\frac{1}{\text{e}^{2\pi\text{i}z}-1}\frac{\text{e}^{izu}}{z^{2}}$
$\displaystyle\langle\eta(u)\overline{\eta}(0)\rangle$
$\displaystyle=\frac{1}{4\pi\text{i}C}\oint_{{\cal
C}_{\eta}}dz\,\,\frac{1}{\text{e}^{4\pi\text{i}z}-1}\frac{\text{e}^{\text{i}zu}}{z\left(z^{2}-\frac{1}{4}\right)},$
(21)

where the integration contours are sets of small circles running counter
clockwise around integer $z$’s excluding zeros in the denominators in the
second factors of the corresponding integrand. By contour deformation the
integrals can be related to integrals in which the contour runs around the
previously excluded poles. These can be evaluated by means of the residue
formula. One obtains

$\displaystyle\langle\epsilon(u)\epsilon(u^{\prime})\rangle=$ (22)
$\displaystyle\frac{1}{2\pi C}\\!\left[-\frac{(\lvert
u-u^{\prime}\rvert-\pi)^{2}}{2}+(\lvert u-u^{\prime}\rvert-\pi)\sin\lvert
u-u^{\prime}\rvert+1+\frac{\pi^{2}}{6}+\frac{5}{2}\cos\lvert
u-u^{\prime}\rvert\right],$
$\langle\sigma(u)\sigma(u^{\prime})\rangle=\frac{1}{48\pi C}\left[3\lvert
u-u^{\prime}\rvert^{2}-6\pi\lvert u-u^{\prime}\rvert+2\pi^{2}\right],$ (23)
$\langle\eta(u)\overline{\eta}(u^{\prime})\rangle=\frac{1}{4\pi
C}\left[(2\pi-\lvert u-u^{\prime}\rvert)\cos\frac{\lvert
u-u^{\prime}\rvert}{2}+3\sin\frac{\lvert u-u^{\prime}\rvert}{2}\right],$ (24)

where we have used shift symmetry in $u$ to obtain the dependence on the
second argument $u^{\prime}$.

##  4 Coupling to matter

Superconformal invariance fixes the form of correlators. For instance, the
two-point function for a superconformal primary $\mathcal{O}$ of dimension
$\Delta$ has the form

$\langle\mathcal{O}(u_{1})\overline{\mathcal{O}}(u_{2})\rangle\sim\frac{1}{\Xi^{2\Delta}},$
(25)

where $\Xi$ is invariant under supertranslations. Furthermore, the two-point
function of chiral (anti-chiral) superfields is also chiral (anti-chiral).
Thus, if we are interested in the correlators for chiral and anti-chiral
matter fields in $\mathcal{N}=2$ supersymmetry, $\Xi$ does not only have to be
invariant under the super-translations (3) but also has to be chiral in one
coordinate and anti-chiral in the other. The correct denominator that
satisfies these requirements is [24]

$\Xi=u_{1}-u_{2}-2\overline{\theta}_{1}\theta_{2}-\theta_{1}\overline{\theta}_{1}-\theta_{2}\overline{\theta}_{2}.$
(26)

Following the general rules of AdS/CFT duality, superconformal symmetry fixes
the on-shell value of the bulk matter action [16]

$S_{\text{matter}}=\mu\int
d\tau^{\prime}_{1}d\tau^{\prime}_{2}d\xi^{\prime}_{2}d\overline{\xi^{\prime}}_{1}\,\,\frac{\Psi(\tau^{\prime}_{2},\xi^{\prime}_{2})\overline{\Psi}(\tau^{\prime}_{1},\overline{\xi^{\prime}}_{1})}{\left|\tau^{\prime}_{1}-\tau^{\prime}_{2}-2\overline{\xi^{\prime}}_{1}\xi^{\prime}_{2}-\xi^{\prime}_{1}\overline{\xi^{\prime}}_{1}-\xi^{\prime}_{2}\overline{\xi^{\prime}}_{2}\right|^{2\Delta}},$
(27)

where $\mu$ is some constant and $\Psi(\tau^{\prime}_{2},\xi^{\prime}_{2})$,
$\overline{\Psi}(\tau^{\prime}_{1},\overline{\xi^{\prime}}_{1})$ are
respectively chiral and anti-chiral matter multiplets corresponding to the
boundary values of fields living in the bulk of nAdS2. On the CFT side, these
are sources coupling to superconformal primaries of dimension $\Delta$. The
zero temperature correlator is obtained by taking variational derivatives of
the exponential of $S_{\text{matter}}$ with respect to
$\Psi\left(\tau^{\prime}_{i},\xi^{\prime}_{i},\overline{\xi}^{\prime}_{i}\right)$
and
$\overline{\Psi}\left(\tau^{\prime}_{j},\xi^{\prime}_{j},\overline{\xi}^{\prime}_{j}\right)$
and viewing $\tau^{\prime},\xi^{\prime},\overline{\xi}^{\prime}$ as superspace
coordinates. The finite temperature correlator follows from variational
derivatives with respect to superfields depending on the unprimed coordinates
$\tau,\xi,\overline{\xi}$ where primed and unprimed quantities are related as
in (12). Finally, corrections due to broken superconformal invariance can be
taken into account when considering variational derivatives with respect to
superfields depending on $u,\theta,\overline{\theta}$ and the relation to
$\tau,\xi,\overline{\xi}$ is given in (14), (15), (16). In order to take the
variational derivatives with respect to fields depending on
$u,\theta,\overline{\theta}$ we use the fact that they are related by super-
reparametrisations to
$\tau^{\prime},\theta^{\prime},\overline{\theta}^{\prime}$. There will be a
Berezian from the superspace measure [19] whereas the transformation
properties of the superfields $\Psi$ and $\overline{\Psi}$ follow from the
AdS/CFT dictionary equating them with currents coupling to superconformal
primaries. Taking all these contributions into account we obtain

$S^{\mathcal{N}=2}_{\text{matter}}=\mu\int
d{u_{ac}}_{1}d\overline{\theta}_{1}d{u_{c}}_{2}d\theta_{2}\,\,\frac{\Big{(}D_{\overline{\theta}_{2}}\overline{\xi}^{\prime}_{2}\Big{)}^{2\Delta}\Big{(}D_{\theta_{1}}\xi^{\prime}_{1}\Big{)}^{2\Delta}\Psi\left({u_{c}}_{2},\theta_{2}\right)\overline{\Psi}\left({u_{ac}}_{1},\overline{\theta}_{1}\right)}{\left|\tau^{\prime}_{1}-\tau^{\prime}_{2}-2\overline{\xi^{\prime}}_{1}\xi^{\prime}_{2}-\xi^{\prime}_{1}\overline{\xi^{\prime}}_{1}-\xi^{\prime}_{2}\overline{\xi^{\prime}}_{2}\right|^{2\Delta}},$
(28)

where the primed quantities are viewed as functions of
$u,\theta,\overline{\theta}$ with the corresponding label. Further,
$u_{c}=u-\theta\overline{\theta}$ and $u_{ac}=u+\theta\overline{\theta}$ are
the chiral and anti-chiral combinations on which associated superfields
depend.

##  5 Four-point functions

In this section we compute the four-point functions at finite temperature for
operators $V$ and $\overline{V}$ of conformal dimension $\Delta$, dual to the
matter fields $\overline{\Psi}$ and $\Psi$. Four-point functions at zero
temperature were already computed in [26]. It is useful to introduce a second
pair of superconformal primaries $W$ and $\overline{W}$ which have vanishing
two-point functions with $\overline{V}$ and $V$. On the supergravity side this
corresponds to adding $\tilde{S}_{\text{matter}}$ which is equal to
$S_{\text{matter}}$ but where the matter multiplets are labelled with an
additional tilde. The generating functional of correlators for superconformal
operators of the boundary theory coincides with the partition function of the
gravity theory in which the currents are identified with boundary values of
matter superfields

$\begin{gathered}Z\left[\Psi,\overline{\Psi},\tilde{\Psi},\tilde{\overline{\Psi}}\right]=\Big{\langle}\exp\left[\int
dud\theta
d\overline{\theta}\left(\Psi\overline{V}+\overline{\Psi}V+\tilde{\Psi}\overline{W}+\tilde{\overline{\Psi}}W\right)\right]\Big{\rangle}_{\text{SCFT}}\\\
\stackrel{{\scriptstyle\text{AdS/CFT}}}{{=}}\int{\cal D}^{\prime}\epsilon{\cal
D}^{\prime}\sigma{\cal D}^{\prime}\eta{\cal
D}^{\prime}\overline{\eta}\,\,\text{e}^{-S^{\mathcal{N}=2}_{\text{sch},\,\beta=2\pi}-S^{\mathcal{N}=2}_{\text{matter}}-\tilde{S}^{\mathcal{N}=2}_{\text{matter}}}.\end{gathered}$
(29)

From the generating functional one can e.g. obtain four-point functions by
taking variational derivaties,

$\langle{\cal
T}\overline{V}V\overline{W}W\rangle=\frac{\delta^{4}Z}{\delta\Psi\delta\overline{\Psi}\delta\tilde{\Psi}\delta\tilde{\overline{\Psi}}}\bigg{\rvert}_{\Psi,\overline{\Psi},\tilde{\Psi},\tilde{\overline{\Psi}}=0}=\left(\frac{\delta^{2}Z}{\delta\Psi\delta\overline{\Psi}}\right)\left(\frac{\delta^{2}Z}{\delta\tilde{\Psi}\delta\tilde{\overline{\Psi}}}\right)\bigg{\rvert}_{\Psi,\overline{\Psi}\tilde{\Psi},\tilde{\overline{\Psi}}=0}.$
(30)

where ${\cal T}$ denotes time ordering.

One finds

$\langle{\cal
T}\overline{V}\left(u_{ac1},\overline{\theta}_{1}\right)V\left(u_{c2},\theta_{2}\right)\overline{W}\left(u_{ac3},\overline{\theta}_{3}\right)W\left(u_{c4},\theta_{4}\right)\rangle\\\
=\int{\cal D}^{\prime}\epsilon{\cal D}^{\prime}\sigma{\cal
D}^{\prime}\eta{\cal
D}^{\prime}\overline{\eta}\,\,NJ\left({u_{ac}}_{1},{{u_{c}}}_{2}\right)J\left({u_{ac}}_{3},{{u_{c}}}_{4}\right)\text{e}^{-S_{\text{Schw}}},$
(31)

where

$N^{-1}=4^{2\Delta}\sin^{2\Delta}\frac{{u_{ac}}_{1}-{{u_{c}}}_{2}}{2}\sin^{2\Delta}\frac{{u_{ac}}_{3}-{{u_{c}}}_{4}}{2}$
(32)

and

$J\\!\left({u_{ac}}_{1},\overline{\theta}_{1},{{u_{c}}}_{2},\theta_{2}\right)=A\\!\left({u_{ac}}_{1},{{u_{c}}}_{2}\right)+B\\!\left({u_{ac}}_{1},{{u_{c}}}_{2}\right)\overline{\theta}_{1}+C\\!\left({u_{ac}}_{1},{{u_{c}}}_{2}\right)\theta_{2}+D\\!\left({u_{ac}}_{1},{{u_{c}}}_{2}\right)\overline{\theta}_{1}\theta_{2}$
(33)

and the coefficients can be read off (28). Dropping the $a$ and $ac$ labels
and introducing the convenient notation $u_{ij}=u_{i}-u_{j}$, we find

$\displaystyle A\left(u_{1},u_{2}\right)=$ $\displaystyle
1+\Delta\left[\frac{-\epsilon\left(u_{1}\right)+\epsilon\left(u_{2}\right)}{\tan\frac{u_{12}}{2}}+\dot{\epsilon}(u_{1})+\dot{\epsilon}(u_{2})+2\text{i}\left(\sigma(u_{1})-\sigma(u_{2})\right)\right],$
(34) $\displaystyle B\left(u_{1},u_{2}\right)=$ $\displaystyle
2\Delta\left[\frac{\eta(u_{1})}{\tan\frac{u_{12}}{2}}-\frac{\eta(u_{2})}{\sin\frac{u_{12}}{2}}-2\dot{\eta}(u_{1})\right],$
(35) $\displaystyle C\left(u_{1},u_{2}\right)=$ $\displaystyle
2\Delta\left[\frac{\overline{\eta}(u_{1})}{\sin\frac{u_{12}}{2}}-\frac{\overline{\eta}(u_{2})}{\tan\frac{u_{12}}{2}}-2\dot{\overline{\eta}}(u_{2})\right],$
(36) $\displaystyle D\left(u_{1},u_{2}\right)=$
$\displaystyle\frac{2\Delta}{\sin\frac{u_{12}}{2}}\Big{[}1-(1+2\Delta)\frac{\epsilon(u_{1})-\epsilon(u_{2})}{2\tan\frac{u_{12}}{2}}+(1+2\Delta)\left(\dot{\epsilon}(u_{1})+\dot{\epsilon}(u_{2})\right)$
$\displaystyle+2\text{i}\left(-1+2\Delta\right)\left(\sigma(u_{1})-\sigma(u_{2})\right)\Big{]}.$
(37)

We decompose the superconformal primaries as

$\begin{gathered}V\left(u_{2},\theta_{2}\right)=V_{\theta}\left({{u_{c}}}_{2}\right)\theta_{2}+V_{\phi}\left({{u_{c}}}_{2}\right)\equiv
V_{\theta_{2}}\theta_{2}+V_{\phi_{2}}\\\
\overline{V}\left(u_{1},\theta_{1}\right)=\overline{V}_{\overline{\theta}}\left({u_{ac}}_{1}\right)\overline{\theta}_{1}+\overline{V}_{\overline{\phi}}\left({u_{ac}}_{1}\right)\equiv\overline{V}_{\overline{\theta}_{1}}\overline{\theta}_{1}+\overline{V}_{\overline{\phi}_{1}},\end{gathered}$
(38)

with similar expressions holding for $W$, $\overline{W}$. By comparing both
sides of (30) we are then led to the following non-vanishing connected four-
point functions,

$\displaystyle\langle{\cal
T}\overline{V}_{\overline{\phi}_{1}}V_{\phi_{2}}\overline{W}_{\overline{\phi}_{3}}W_{\phi_{4}}\rangle=$
$\displaystyle\langle{\cal
T}\overline{V}_{\overline{\theta_{1}}}V_{\theta_{2}}\overline{W}_{\overline{\phi}_{3}}W_{\phi_{4}}\rangle=$
$\displaystyle N\wick{\c{D}(u_{1},u_{2})\c{A}(u_{3},u_{4})},$ (39)
$\displaystyle\langle{\cal
T}\overline{V}_{\overline{\theta}_{1}}V_{\theta_{2}}\overline{W}_{\overline{\theta}_{3}}W_{\theta_{4}}\rangle=$
$\displaystyle N\wick{\c{D}(u_{1},u_{2})\c{D}(u_{3},u_{4})},$ (40)
$\displaystyle\langle{\cal
T}\overline{V}_{\overline{\theta}_{1}}V_{\phi_{2}}\overline{W}_{\overline{\phi}_{3}}W_{\theta_{4}}\rangle=$
$\displaystyle-N\wick{\c{B}(u_{1},u_{2})\c{C}(u_{3},u_{4})},$ (41)
$\displaystyle\langle{\cal
T}\overline{V}_{\overline{\phi}_{1}}V_{\theta_{2}}\overline{W}_{\overline{\theta}_{3}}W_{\phi_{4}}\rangle=$
$\displaystyle-N\wick{\c{C}(u_{1},u_{2})\c{B}(u_{3},u_{4})}.$ (42)

Horizontal brackets denote Wick contractions of super-Schwarzian fluctuations:
only contributions quadratic in fluctuations are contained in the connected
four-point function. The pair of fluctuations is replaced by its propagator.
Since the propagators depend on absolute values of their arguments the results
depend on the chosen ordering. The arrangement which, upon canonical
continuation to Minkowski time, leads to out of time order correlators (OTOCs)
is $u_{1}>u_{3}>u_{2}>u_{4}$. Before continuation to Minkowski time one
obtains

$\displaystyle\langle\overline{V}_{\overline{\phi}_{1}}\overline{W}_{\overline{\phi}_{3}}V_{\phi_{2}}W_{\phi_{4}}\rangle=$
$\displaystyle\frac{N\Delta^{2}}{2\pi
C}\bigg{[}\left(\frac{u_{12}}{\tan\frac{u_{12}}{2}}-2\right)\left(\frac{u_{34}}{\tan\frac{u_{34}}{2}}-2\right)+2\pi\frac{\sin\frac{u_{12}+u_{34}}{2}-\sin\frac{u_{23}+u_{14}}{2}}{\sin\frac{u_{12}}{2}\sin\frac{u_{34}}{2}}$
$\displaystyle+2\pi\frac{u_{23}}{\tan\frac{u_{12}}{2}\tan\frac{u_{34}}{2}}+u_{12}u_{34}+2\pi
u_{23}\bigg{]},$ (43)
$\displaystyle\langle\overline{V}_{\overline{\theta_{1}}}\overline{W}_{\overline{\phi}_{3}}V_{\theta_{2}}W_{\phi_{4}}\rangle=$
$\displaystyle-\frac{N\Delta^{2}}{2\pi
C}\frac{1}{\sin\frac{u_{34}}{2}}\bigg{\\{}(2\Delta+1)\bigg{[}\left(\frac{u_{12}}{\tan\frac{u_{12}}{2}}-2\right)\left(\frac{u_{34}}{\tan\frac{u_{34}}{2}}-2\right)$
$\displaystyle+2\pi\frac{\sin\frac{u_{12}+u_{34}}{2}-\sin\frac{u_{23}+u_{14}}{2}}{\sin\frac{u_{12}}{2}\sin\frac{u_{34}}{2}}\bigg{]}+(2\Delta-1)\left[u_{12}u_{34}+2\pi
u_{23}\right]\bigg{\\}},$ (44)
$\displaystyle\langle\overline{V}_{\overline{\theta}_{1}}\overline{W}_{\overline{\theta}_{3}}V_{\theta_{2}}W_{\theta_{4}}\rangle=$
$\displaystyle\frac{N\Delta^{2}}{2\pi
C}\frac{1}{\sin\frac{u_{12}}{2}\sin\frac{u_{34}}{2}}\bigg{\\{}(2\Delta+1)^{2}\bigg{[}\left(\frac{u_{12}}{\tan\frac{u_{12}}{2}}-2\right)\left(\frac{u_{34}}{\tan\frac{u_{34}}{2}}-2\right)$
$\displaystyle+2\pi\frac{\sin\frac{u_{12}+u_{34}}{2}-\sin\frac{u_{23}+u_{14}}{2}}{\sin\frac{u_{12}}{2}\sin\frac{u_{34}}{2}}\bigg{]}+(2\Delta-1)^{2}\left[u_{12}u_{34}+2\pi
u_{23}\right]\bigg{\\}},$ (45)
$\displaystyle\langle\overline{V}_{\overline{\theta}_{1}}\overline{W}_{\overline{\phi}_{3}}V_{\phi_{2}}W_{\theta_{4}}\rangle=$
$\displaystyle-\frac{2N\Delta^{2}}{\pi
C}\frac{1}{\sin\frac{u_{12}}{2}\sin\frac{u_{34}}{2}}\left[u_{23}\cos\frac{u_{23}}{2}-3\sin\frac{u_{23}}{2}\right],$
(46)
$\displaystyle\langle\overline{V}_{\overline{\phi}_{1}}\overline{W}_{\overline{\theta}_{3}}V_{\theta_{2}}W_{\phi_{4}}\rangle=$
$\displaystyle-\frac{2N\Delta^{2}}{\pi
C}\frac{1}{\sin\frac{u_{12}}{2}\sin\frac{u_{34}}{2}}\Big{[}u_{32}\cos\frac{u_{14}}{2}+\sin\frac{u_{14}}{2}$
$\displaystyle\quad-\sin\frac{u_{12}+u_{42}}{2}+\sin\frac{u_{13}+u_{43}}{2}\Big{]}.$
(47)

To obtain the real time OTOCs we proceed by analytic continuation [9].
Reinstalling also temperature dependence we replace

$u_{1}=\frac{2\pi\text{i}}{\beta}a\,\,\,,\,\,\,u_{2}=0\,\,\,,\,\,\,u_{3}=\frac{2\pi\text{i}}{\beta}\left(b+\hat{u}\right)\,\,\,,\,\,\,u_{4}=\frac{2\pi\text{i}}{\beta}\hat{u}$
(48)

in equations (43)-(47). Furthermore, one is interested in the region with
$a,b$ of the order of the dissipation time $\beta$ whereas $\hat{u}$ is much
larger than the dissipation time while still much shorter than the scrambling
time, i.e.
$\frac{2\pi}{\beta}\ll\hat{u}\ll\frac{2\pi}{\beta}\log\frac{C}{\beta}$.
Keeping only the leading contributions (growing exponentially with $\hat{u}$)
one finds

$\displaystyle\langle\overline{V}_{\overline{\phi}}(0)\overline{W}_{\overline{\phi}}\left(\hat{u}\right)V_{\phi}(0)W_{\phi}(\hat{u})\rangle_{\beta}$
$\displaystyle\sim\frac{\Delta^{2}}{C}\text{e}^{\frac{2\pi}{\beta}\hat{u}},$
(49)
$\displaystyle\langle\overline{V}_{\overline{\theta}}(0)\overline{W}_{\overline{\theta}}\left(\hat{u}\right)V_{\theta}(0)W_{\theta}(\hat{u})\rangle_{\beta}$
$\displaystyle\sim\frac{\Delta^{2}}{C}\left(1+2\Delta\right)^{2}\text{e}^{\frac{2\pi}{\beta}\hat{u}},$
(50)
$\displaystyle\langle\overline{V}_{\overline{\phi}}(0)\overline{W}_{\overline{\theta}}\left(\hat{u}\right)V_{\phi}(0)W_{\theta}(\hat{u})\rangle_{\beta}$
$\displaystyle\sim\frac{\Delta^{2}}{C}\left(1+2\Delta\right)\text{e}^{\frac{2\pi}{\beta}\hat{u}},$
(51)
$\displaystyle\langle\overline{V}_{\overline{\theta}}(0)\overline{W}_{\overline{\phi}}\left(\hat{u})\right)V_{\theta}(0)W_{\phi}(\hat{u})\rangle_{\beta}$
$\displaystyle\sim\frac{\Delta^{2}}{C}\left(1+2\Delta\right)\text{e}^{\frac{2\pi}{\beta}\hat{u}}.$
(52)

The exponential growth of the above correlators originates from graviton
($\epsilon$) exchange. The corresponding Lyapunov exponent is

$\lambda^{(2)}=\frac{2\pi}{\beta},$ (53)

i.e. it saturates the bound of [17]. The superscript two indicates that this
behaviour is due to graviton (spin two particle) exchange. Graviton exchange
does not contribute to the four-point functions (41) and (42) at the given
order in $\frac{1}{C}$. For those one finds from gravitini exchange

$\langle\overline{V}_{\overline{\phi}}(0)\overline{W}_{\overline{\theta}}(\hat{u})V_{\theta}(0)W_{\phi}(\hat{u})\rangle_{\beta}\sim\langle\overline{V}_{\overline{\theta}}(0)\overline{W}_{\overline{\phi}}(\hat{u})V_{\phi}(0)W_{\theta}(\hat{u})\rangle_{\beta}\sim\frac{\Delta}{C}\hat{u}\text{e}^{\frac{\pi}{\beta}\hat{u}},$
(54)

implying a Lyapunov exponent

$\lambda^{(3/2)}=\frac{\pi}{\beta}.$ (55)

This agrees with the pattern found in the context of ${\cal N}=1$
supersymmetry [29, 22],

$\lambda^{(s)}=\frac{2\pi}{\beta}\left(s-1\right),$ (56)

where $s$ is the spin of the gravitational mode being exchanged. From (23) we
observe that combinations to which only the exchange of a graviphoton $\sigma$
(spin one) contributes do not show exponential behaviour in agreement with
(56). Notice also that for the result (54) the introduction of the second
primary $W$ with vanishing two-point function, $\langle
W\overline{V}\rangle=0$, is crucial. If e.g. we had considered the 4-point
functions with $V$, $\overline{V}$’s only, there would be additional terms
with

$J\left(u_{ac1},u_{c4}\right)J\left(u_{ac3},u_{c2}\right)$

in (31). These would result in graviton exchange contributing to (54) with
$W$, $\overline{W}$ replaced by $V$, $\overline{V}$. Then also these
components would show exponential growth with maximal Lyapunov exponent.

##  6 Summary and Outlook

In this paper we studied the $\mathcal{N}=2$ supersymmetric Jackiw-Teitelboim
theory at finite temperature and computed the Lyapunov exponents. We found
that their value depends on the particular component of the four-point
function within the supermultiplet. There are compenents with maximal Lyapunov
exponent $2\pi/\beta$. For them the leading correction contains graviton
exchange. For other components the graviton does not couple to corresponding
bi-locals. Due to gravitini exchange the leading contribution yields a
Lyapunov exponent $\pi/\beta$. One could also consider superpositions with
graviphoton exchange only. They do not show exponential time dependence
(chaotic behaviour). Finally, we argued that for non vanishing two-point
function, $\langle\overline{V}W\rangle\not=0$, all components have maximal
Lyapunov exponent.

The ${\cal N}=2$ case is interesting. For instance, it is related to four
dimensionial 1/4 BPS black holes [30, 31, 26], whereas, to our knowledge, a
higher dimensional solution leading to ${\cal N}=1$ JT gravity in its near
horizon limit has not been identified so far.

Several generalisations of the present calculation are possible. One could add
flavour symmetry as in [22], or consider higher amounts of sypersymmetry, e.g.
$N=3,4$ super-Schwarzian actions. Furthermore, subleading corrections in $1/C$
might be interesting. For the bosonic case these have been analysed in [32],
for ${\cal N}=1$ some contributions are given in [23].

## Acknowledgements

This work was supported by “Bonn-Cologne Graduate School for Physics and
Astronomy” (BCGS).

## References

  * [1] J.M. Maldacena, _The Large N limit of superconformal field theories and supergravity_ , _Int. J. Theor. Phys._ 38 (1999) 1113 [hep-th/9711200].
  * [2] T.M. Fiola, J. Preskill, A. Strominger and S.P. Trivedi, _Black hole thermodynamics and information loss in two-dimensions_ , _Phys. Rev. D_ 50 (1994) 3987 [hep-th/9403137].
  * [3] J.M. Maldacena, J. Michelson and A. Strominger, _Anti-de Sitter fragmentation_ , _JHEP_ 02 (1999) 011 [hep-th/9812073].
  * [4] A. Almheiri and J. Polchinski, _Models of AdS 2 backreaction and holography_, _JHEP_ 11 (2015) 014 [1402.6334].
  * [5] R. Jackiw, _Lower Dimensional Gravity_ , _Nucl. Phys. B_ 252 (1985) 343.
  * [6] C. Teitelboim, _Gravitation and Hamiltonian Structure in Two Space-Time Dimensions_ , _Phys. Lett. B_ 126 (1983) 41.
  * [7] G. Turiaci and H. Verlinde, _On CFT and Quantum Chaos_ , _JHEP_ 12 (2016) 110 [1603.03020].
  * [8] K. Jensen, _Chaos in AdS 2 Holography_, _Phys. Rev. Lett._ 117 (2016) 111601 [1605.06098].
  * [9] J. Maldacena, D. Stanford and Z. Yang, _Conformal symmetry and its breaking in two dimensional Nearly Anti-de-Sitter space_ , _PTEP_ 2016 (2016) 12C104 [1606.01857].
  * [10] M. Cvetič and I. Papadimitriou, _AdS 2 holographic dictionary_, _JHEP_ 12 (2016) 008 [1608.07018].
  * [11] S. Sachdev and J. Ye, _Gapless spin fluid ground state in a random, quantum Heisenberg magnet_ , _Phys. Rev. Lett._ 70 (1993) 3339 [cond-mat/9212030].
  * [12] A. Kitaev, _A Simple Model Of Quantum Holography_ , in _Talks at KITP_ , April 7, 2015 and May 27, 2015 [http://online.kitp.ucsb.edu/online/entangled15/kitaev/, http://online.kitp.ucsb.edu/online/entangled15/kitaev2/].
  * [13] O. Parcollet and A. Georges, _Non-Fermi-liquid regime of a doped Mott insulator_ , _Phys. Rev. B_ 59 (1999) 5341.
  * [14] S. Sachdev, _Holographic metals and the fractionalized Fermi liquid_ , _Phys. Rev. Lett._ 105 (2010) 151602 [1006.3794].
  * [15] S. Sachdev, _Strange metals and the AdS/CFT correspondence_ , _J. Stat. Mech._ 1011 (2010) P11022 [1010.0682].
  * [16] E. Witten, _Anti-de Sitter space and holography_ , _Adv. Theor. Math. Phys._ 2 (1998) 253 [hep-th/9802150].
  * [17] J. Maldacena, S.H. Shenker and D. Stanford, _A bound on chaos_ , _JHEP_ 08 (2016) 106 [1503.01409].
  * [18] J. Maldacena and D. Stanford, _Remarks on the Sachdev-Ye-Kitaev model_ , _Phys. Rev. D_ 94 (2016) 106002 [1604.07818].
  * [19] W. Fu, D. Gaiotto, J. Maldacena and S. Sachdev, _Supersymmetric Sachdev-Ye-Kitaev models_ , _Phys. Rev. D_ 95 (2017) 026009 [1610.08917].
  * [20] S. Förste and I. Golla, _Nearly AdS 2 sugra and the super-Schwarzian_, _Phys. Lett. B_ 771 (2017) 157 [1703.10969].
  * [21] S. Förste, J. Kames-King and M. Wiesner, _Towards the Holographic Dual of N = 2 SYK_ , _JHEP_ 03 (2018) 028 [1712.07398].
  * [22] P. Narayan and J. Yoon, _Supersymmetric SYK Model with Global Symmetry_ , _JHEP_ 08 (2018) 159 [1712.02647].
  * [23] T.G. Mertens, _Degenerate operators in JT and Liouville (super)gravity_ , _JHEP_ 04 (2021) 245 [2007.00998].
  * [24] K. Bulycheva, _$\mathcal{N}=2$ SYK model in the superspace formalism_, _JHEP_ 04 (2018) 036 [1801.09006].
  * [25] G. Sárosi, _AdS 2 holography and the SYK model_, _PoS_ Modave2017 (2018) 001 [1711.08482].
  * [26] S. Förste, A. Gerhardus and J. Kames-King, _Supersymmetric black holes and the SJT/nSCFT 1 correspondence_, _JHEP_ 01 (2021) 186 [2007.12393].
  * [27] C. Peng and S. Stanojevic, _Soft modes in $\mathcal{N}=2$ SYK model_, _JHEP_ 01 (2021) 082 [2006.13961].
  * [28] R.F. Dashen, B. Hasslacher and A. Neveu, _Semiclassical Bound States in an Asymptotically Free Theory_ , _Phys. Rev. D_ 12 (1975) 2443.
  * [29] E. Perlmutter, _Bounding the Space of Holographic CFTs with Chaos_ , _JHEP_ 10 (2016) 069 [1602.08272].
  * [30] S.L. Cacciatori and D. Klemm, _Supersymmetric AdS(4) black holes and attractors_ , _JHEP_ 01 (2010) 085 [0911.4926].
  * [31] K. Hristov and S. Vandoren, _Static supersymmetric black holes in AdS 4 with spherical symmetry_, _JHEP_ 04 (2011) 047 [1012.4314].
  * [32] Y.-H. Qi, S.-J. Sin and J. Yoon, _Quantum Correction to Chaos in Schwarzian Theory_ , _JHEP_ 11 (2019) 035 [1906.00996].


# Adaptive Sampling of $k$-space in Magnetic Resonance
for Rapid Pathology Prediction

Chen-Yu Yen  Raghav Singhal  Umang Sharma  Rajesh Ranganath  Sumit Chopra
Lerrel Pinto

###### Abstract

Magnetic Resonance (MR) imaging, despite its proven diagnostic utility,
remains an inaccessible imaging modality for disease surveillance at the
population level. A major factor rendering MR inaccessible is lengthy scan
times. An MR scanner collects measurements associated with the underlying
anatomy in the Fourier space, also known as the k-space. Creating a high-
fidelity image requires collecting large quantities of such measurements,
increasing the scan time. Traditionally to accelerate an MR scan, image
reconstruction from under-sampled k-space data is the method of choice.
However, recent works show the feasibility of bypassing image reconstruction
and directly learning to detect disease directly from a sparser learned subset
of the k-space measurements. In this work, we propose Adaptive Sampling for MR
(ASMR), a sampling method that learns an adaptive policy to sequentially
select k-space samples to optimize for target disease detection. On $6$ out of
$8$ pathology classification tasks spanning the Knee, Brain, and Prostate MR
scans, ASMR reaches within $2\%$ of the performance of a fully sampled
classifier while using only $8\%$ of the k-space, as well as outperforming
prior state-of-the-art work in k-space sampling such as EMRT, LOUPE, and DPS.

Reinforcement Learning, Healthcare, MRI, Machine Learning, ICML

##  1 Introduction

Magnetic Resonance (MR) imaging is a diagnostic medical imaging tool that
exhibits superior soft-tissue contrast in its generated images. This has led
to MR scans being the gold standard for diagnosing a range of cancers (Winawer
et al., 1997; Ilic et al., 2018; Elmore et al., 1998), and musculoskeletal
disorders (Dean Deyle, 2011; Elvenes et al., 2000). An MR machine sequentially
measures the responses to radio-frequency pulses of a human body placed in a
magnetic field to generate cross-sectional images (slices) of the anatomy
(Liang & Lauterbur, 2000; Zbontar et al., 2018). The measurements are acquired
in the Fourier space, also known as the k-space (Liang & Lauterbur, 2000) in
the MR community. The spatial image is then constructed by applying a multi-
dimensional Fourier transform to the k-space data. High-resolution MR images
require a large number of k-space measurements, leading to increased scan
times (up to $40$ minutes, depending on the anatomy) (Zbontar et al., 2018).

Figure 1: Overview of ASMR. Given a set of prior samples from the MR scanner,
ASMR proposes the next sample to collect.These sampling steps are repeated for
a fixed set of iterations $T$, at which point the collected samples are used
by a classifier to predict the presence of a fixed set of pathologies. ASMR is
trained to optimize classification performance and sidesteps reconstruction
altogether.

Reducing MR scan times is an active area of research. A common approach is to
under-sample the k-space and then use image reconstruction techniques such as
compressed sensing (Lustig et al., 2008; Zbontar et al., 2018; Donoho, 2006)
and deep learning (Sriram et al., 2020; Chung & Ye, 2022) to reconstruct the
underlying image accurately. A recent challenge organized by NYU Langone
(Knoll et al., 2020) shows that at a $12.5\%$ sampling rate, reconstructed
images started missing pathologies. This is understandable as reconstruction
metrics, such as SSIM (Wang et al., 2004), measure the similarity between
moments of localized patches rather than optimizing for diagnostic
performance. To overcome this, recent work (Singhal et al., 2023; Tolpadi et
al., 2023) shows that we can directly detect the presence of a pathology and
bypass reconstruction altogether. Furthermore, Singhal et al. (2023) show that
directly classifying from a sparse subset of the k-space yields similar
performance to using the entire k-space. However, both Singhal et al. (2023);
Tolpadi et al. (2023) use a heuristic approach to find the sampling pattern.

Typically, the sampling patterns are heuristic or hand-crafted (Lustig et al.,
2007), based on the assumption that natural images have rapidly decaying
Fourier coefficients (Lustig et al., 2007). However, Bahadir et al. (2019);
Jin et al. (2019); Pineda et al. (2020); Huijben et al. (2019) show that
learning the sampling pattern to optimize for the underlying task offers
significant improvements over heuristic sampling patterns. In practice, a
heuristic sampling pattern, such as an equi-spaced pattern (Johnson et al.,
2022), at a fixed sampling rate is used for each patient (Zbontar et al.,
2018). However, a heuristic pattern cannot adapt to the collection of
previously acquired $k$-space samples.

In this work, we introduce Adaptive Sampling for Magnetic Resonance (ASMR).
Using the fact that the k-space data collection is a sequential process, we
pose the problem of learning the sampling pattern as a decision-making
problem, which can then be optimized using reinforcement learning (RL).
Concretely, at every sampling time step, the collection of previous samples
serves as the observation, the choice of the next sample serves as the action,
and the reward is computed by measuring the log-likelihood between the labels
and the sub-sampled $k$-space. See Figure 1 for an overview of ASMR. The
policy for selecting actions is optimized using Proximal Policy Optimization.
Unlike prior work that uses RL for reconstruction (Bakker et al., 2020; Jin et
al., 2019; Pineda et al., 2020), ASMR is trained to maximize the
classification performance.

To understand the performance of ASMR, we run experiments on the FastMRI
dataset (Mathieu et al., 2013; Zhao et al., 2021), consisting of volumetric
brain, knee and abdominal scans. For simplicity, we use Cartesian sampling
(Liang & Lauterbur, 2000; Lustig et al., 2008) that selects columns of the
$2$D complex-valued k-space matrix. Our experiments reveal the following
findings:

  1. 1.

ASMR outperforms state-of-the-art non-adaptive sampling patterns such as EMRT
(Singhal et al., 2023) on 6 out of 8 classification tasks. Compared to learned
probabilistic sampling patterns like LOUPE (Bahadir et al., 2019) and DPS,
ASMR achieves an improvement of atleast 2.5% in the AUC metrics on 7 out of 8
tasks (See Figure 5).

  2. 2.

ASMR outperforms adaptive sampling methods (Bakker et al., 2020) that are
optimized for reconstruction, providing an absolute gain of 2.88% and 7% in
the AUC metrics on the Knee and Brain datasets respectively (See Figure 6).

  3. 3.

Compared to an image-based classifier that uses the entire k-space data as
input, ASMR comes within $2\%$ of its performance on 6 out of 8 tasks with a
$12.5\times$ acceleration factor (See Figure 5).

Our code can be found at adaptive-sampling-mr.github.io.

##  2 Related Work

ASMR builds upon several prior works in k-space-based prediction, mask
sampling strategies, and adaptive learning.

#### Pathology prediction from sparse k-space:

Singhal et al. (2023) show that directly classifying from sparsely acquired
k-space data yields classification performance similar to an image-based model
that uses as input images reconstructed using the entire k-space data. The
sampling pattern they use to acquire the sparse k-space data is optimized
using a non-adaptive variable density sampling prior (Lustig et al., 2007).
Similarly, Tolpadi et al. (2023) show that directly generating segmentations
as the main objective with a secondary reconstruction objective can also
accelerate pathology prediction without any significant drop in performance.
Interestingly, Tolpadi et al. (2023) show that there was no correlation
between segmentation and reconstruction performance. In both works, the
sampling patterns used were heuristic-based, non-adaptive, and not optimized
for the downstream task.

#### Learned non-adaptive sampling methods:

Several prior works (Bahadir et al., 2019; Huijben et al., 2019; Weiss et al.,
2020) have looked at learning a probabilistic model over sampling patterns
jointly with a reconstruction model. Gözcü et al. (2019); Weiss et al. (2019)
learn a fixed sampling pattern using a pre-trained reconstruction model and
the reconstruction error as a scoring rule for sampling patterns. The sampling
patterns learned in these methods are non-adaptive and only produce sampling
patterns for a fixed sampling budget. In Figure 4 we compare ASMR against
sampling patterns optimized for classification using loupe (Bahadir et al.,
2019) and dps (Huijben et al., 2019).

#### Learned adaptive sampling methods:

More recently, Bakker et al. (2020); Pineda et al. (2020); Zhang et al.
(2019); Jin et al. (2019) pose learning an adaptive sampling pattern as an rl
problem. For example Bakker et al. (2020) first trains a reconstruction model
and then uses the reconstruction loss as a reward to train the policy. Zhang
et al. (2019) also presents an active acquisition method, where instead of
learning a policy model, they jointly train a reconstruction model and
evaluator model. Similarly, Jin et al. (2019) jointly learn a reconstruction
model and policy network using Monte-Carlo tree search, similar to AlphaGo
(Silver et al., 2016). All these works use reconstruction metrics as the
reward, which is often sub-optimal for pathology classification (Singhal et
al., 2023). Figure 6 shows that ASMR, which is directly optimized for
classification improves upon reconstruction-based sampling.

#### Reinforcement learning for adaptive sampling:

Reinforcement learning for adaptive sampling is a promising area of research
in the fields of gaming (Silver et al., 2016), finance (Hambly et al., 2023),
and chatbots (Achiam et al., 2023). For instance, in the robotics domain, it
has been used to improve controllers (Zucker et al., 2008; Pan et al., 2022),
and path planning (Pan et al., 2022). Additionally, it has applications in
autonomous balloon navigation (Bellemare et al., 2020), chip design and
hardware optimization (Kumar et al., 2022; Chen et al., 2023). However, unlike
applications where a large amount of data is available or new data can be
easily synthesized, one of our challenges is the limited dataset size. Unlike
scenarios where the reward is well-defined, another challenge is the choice of
reward suitable for fast pathology classification.

##  3 Background and Setup

In this section, we provide background for understanding ASMR, which includes
understanding how an MR scanner works, details on k-space classification, and
adaptive sampling with reinforcement learning.

Figure 2: Cartesian Sampling on k-space at different sampling rates. The
leftmost image shows the complete k-space followed 2.5%, 5% and 10% sampling
rates (from left to right)

###  3.1 Magnetic Resonance (MR) Environment

MR imaging functions indirectly, acquiring frequency and phase measurements of
the anatomy being imaged instead of direct spatial data. A receiver coil,
situated near the target area, acquires a collection of measurements encoding
the human body’s response to a combination of applied electromagnetic fields,
including a static spatially homogeneous magnetic field, spatially varying
magnetic fields (gradients), and radio-frequency pulses (the observed
measurements) (Liang & Lauterbur, 2000). These measurements of the body’s
response to different radio-frequency pulses correspond to points in the
k-space and are stored in a matrix form called the k-space matrix. A k-space
matrix is a complex-valued matrix $\mathbf{x}\in\mathbb{C}^{d_{r}\times
d_{c}}$, which is the Fourier Transform of the underlying $2$D MR image.
Hence, an image can be reconstructed using an Inverse Fourier Transform of the
k-space.

We represent the k-space sampling pattern using a two-dimensional binary mask
$\mathbf{s}\in\prod_{i,j=1}^{d_{r},d_{c}}\\{0,1\\}$, such that
$\mathbf{s}_{i,j}=1$ indicates that $\mathbf{x}_{i,j}$ has been sampled.
Typically, the k-space is retrospectively under-sampled by multiplying the
full k-space $\mathbf{x}$ with the binary mask $\mathbf{s}$ (Zbontar et al.,
2018), which we denote as $\mathbf{x}_{\mathbf{s}}=\mathbf{x}\odot\mathbf{s}$,
where $\odot$ stands for element-wise multiplication.

The k-space measurements are acquired sequentially. A MR scanner has physical
restraints on how it can sample. Some examples of physically feasible
trajectories include Cartesian, spiral, and radial (Zbontar et al., 2018;
Weiss et al., 2019). In this work, we consider Cartesian sampling, which
involves sampling columns of the k-space matrix sequentially. See Figure 2 for
examples of Cartesian sampling on k-space. It can take a long time to acquire
measurements sufficient for reconstructing a diagnostic quality image, thereby
increasing the cost of the scan and making them infeasible for population-
level screenings.

In the next section, we discuss recent work that bypasses image reconstruction
and directly predicts the presence or absence of a set of pathologies.

###  3.2 Classification from k-space data

Given a labeled dataset $(\\{(\mathbf{x}_{i},\mathbf{y}_{i})\\}_{i=1}^{N})$
consisting of k-space data $\mathbf{x}$ and pathology labels $\mathbf{y}$, we
follow Singhal et al. (2023) and directly train a classifier from the
retrospectively under-sampled k-space $\mathbf{x}_{\mathbf{s}}$ using the
kspace-net classifier $q_{\phi}(\mathbf{y}\mid\mathbf{x}_{\mathbf{s}})$.
Importantly, the kspace-net classifier $q_{\phi}$ takes the complex-valued
k-space as input and does not rely on reconstructed images. Similar to Jethani
et al. (2021, 2023) and Singhal et al. (2023) we train the kspace-net
classifier using randomly masked out k-space data, where the masks are drawn
independently from the variable density prior Lustig et al. (2007). Models
trained in such a way can then be used to score masks. For instance, we can
estimate the mutual information (Cover, 1999) between
$\mathbf{x}_{\mathbf{s}}$ and the label $\mathbf{y}$, upto a constant, using
the classifier $q_{\theta}$:

$\displaystyle\mathbf{V}_{\theta}(\mathbf{s})=\mathbb{E}_{q(\mathbf{x}_{\mathbf{s}})q(\mathbf{y}\mid\mathbf{x}_{\mathbf{s}})}\log{q_{\theta}(\mathbf{y}\mid\mathbf{x}_{\mathbf{s}})}$
(1)

In the Section 4, we show how this classifier can be used to design rewards
for the policy.

###  3.3 Reinforcement Learning

The reinforcement learning problem formulates the decision-making problem as
Markov Decision Process (MDP)
$(\mathcal{S},\mathcal{A},p_{0}(s),p(s^{\prime}|s,a),R(s,a),\gamma,T)$, where
$a$ is an action, $s$ is the state, $\mathcal{S}$ is a state space,
$\mathcal{A}$ is an action space, $p_{0}(s)$ is a distribution of initial
states, $p(s^{\prime}|s,a)$ is the environment dynamics, $R(s,a)$ is a reward
function, $\gamma$ is a discount factor, and $T$ is the task horizon. The
agent interacts with the MDP according to a policy $\pi(a|s)$. The goal of
reinforcement learning is to obtain a policy that maximizes the cumulative
discounted returns:
$\pi^{*}=\arg\max_{\pi}\mathbb{E}_{\pi}\left[\sum_{t=0}^{T}\gamma^{t}R(s_{t},a_{t})\right]$.
For more details on RL we refer the reader to Sutton & Barto (2018).

#### Proximal Policy Optimization (PPO):

PPO (Schulman et al., 2017) is an RL algorithm that learns a policy
$\pi_{\theta}$ and a value function $V_{\phi}$ with the goal of finding an
optimal policy for an MDP. An actor proposes an action to the environment, and
a critic estimates a value for the action proposed by the actor. To this end,
PPO optimizes a surrogate objective

$L=\mathbb{E}\left[\min(l_{t}(\theta)\hat{A}_{t},\text{clip}(l_{t}(\theta),1-\varepsilon,1+\varepsilon)\hat{A}_{t})\right]$
(2)

where
$l_{t}(\theta)=\frac{\pi_{\theta}(a_{t}|s_{t})}{\pi_{\text{old}}(a_{t}|s_{t})}$
denotes the likelihood ratio between the new and old policies, and
$\hat{A}_{t}=R_{t}+\gamma V_{\phi}(s_{t+1})-V_{\phi}(s_{t})$ denotes an
advantage function at timestep $t$. During training, PPO penalizes large
changes to the policy to improve the stability of learning. For this work, we
build our agent on top of an open-source implementation of PPO in (Huang et
al., 2022).

##  4 Adaptive Sampling for Magnetic Resonance

In this section, we introduce Adaptive Sampling for Magnetic Resonance (ASMR).
ASMR learns an adaptive sampling method for direct k-space classification by
training an RL agent to select k-space samples sequentially. Unlike standard
RL settings with access to an online environment, we retrospectively under-
sample the k-space by applying a binary mask $\mathbf{s}$ to the fully-sampled
k-space $\mathbf{x}$. Furthermore, the action space and reward function
require additional consideration given the constraints of the MR pathology
prediction problem, such as data imbalance and the input being in the Fourier
domain.

###  4.1 Formulating k-space selection as an RL Problem

To frame the selection of k-space measurements as a RL problem, we simulate
environments using a dataset
$\mathcal{D}=\\{(\mathbf{x}_{i},\mathbf{y}_{i})\\}_{i=1}^{n}$, where each
k-space sample $\mathbf{x}_{i}\in\mathbb{C}^{d_{r}\times d_{c}}$ represents
one environment and has an associated pathology label $\mathbf{y}_{i}$. We
employ a Cartesian sampling pattern, where an acquisition corresponds to a
column of the k-space matrix.

Within each environment, the goal of the RL agent is to sequentially select a
set of $T=|d_{c}|\times\alpha$ k-space columns to maximize classification
performance, where $T$ represents the sampling budget and serves as our
episode length. The agent is tasked to take action
$a_{t}\in\\{0,1,\dots,d_{c}\\}$ at each time step $t$, which corresponds to
selecting a column from the k-space matrix. From an implementation standpoint,
we zero-fill columns that have not been selected by the agent yet. Hence, the
task for the RL agent can also be framed as a mask selection problem where the
objective is to obtain a mask $M_{T}$ that maximizes classification
performance. Starting with an initial empty mask $M_{0}$, the final mask
$M_{T}$ is obtained by iteratively updating the masks as $M_{t-1}[a_{t}]=1$.
The actions $a_{t}$ are produced by the policy $\pi_{\theta}$, which acts on
the current state $s_{t}$. We next describe the state and action spaces.

Figure 3: ASMR Training. ASMR takes an initial sub-sampled k-space
$\mathbf{s}_{t}$ as input and proposes the next sample $\mathbf{a}_{t}$ to
generate the next state $\mathbf{s}_{t+1}$. This state $\mathbf{s}_{t+1}$ is
used by the reward model to compute the log-likelihood
$q_{\phi}(\mathbf{y}\mid\mathbf{x}_{\mathbf{s}_{t+1}})$, the reward for the
actor. We repeat these steps for a fixed number of iterations $T$.

#### State space for ASMR:

Let $\mathbf{x}^{F}\in\mathbb{C}^{d_{r}\times d_{c}}$ represent the fully
acquired k-space data for a random sample from $\mathcal{D}$. The input state
at time $t$ is defined as $s_{t}=M_{t}\odot\mathbf{x}^{F}$, where $M_{t}$ is
the mask, a binary vector defined as $M_{t}\in\prod_{i=1}^{d_{c}}\\{0,1\\}$.
Initially, at $t=0$ we set $M_{0}=0$ and $s_{0}=0$. We emphasize that
$\mathbf{x}^{F}$ is utilized merely for notational simplicity. The full
k-space data, $\mathbf{x}^{F}$, is never disclosed to the agent, who can only
observe the acquired data. Naturally, during inference, the agent can only
observe the k-space data it has acquired till step $t$. Given an action
$a_{t}$ and state $s_{t+1}$, the next state at time $t+1$ is updated as:

$\displaystyle s_{t+1}$ $\displaystyle=M_{t+1}\odot\mathbf{x}^{F}$ (3)

#### Action space for ASMR:

We use a discrete action space, which is denoted as the set
$\mathcal{A}=\\{1,\ldots,d_{c}\\}$ where each element $i$ of $\mathcal{A}$
corresponds to selecting the $i^{\text{th}}$ k-space column from a given
sample. Unlike common RL settings, our action space dynamically shrinks as we
iterate through the episode. To handle this shrinkage, we need to iteratively
mask out already selected actions, which is discussed next.

#### Dynamic masking of action space:

To account for the reduced action space after each selection, we maintain a
set $\Omega_{t-1}=\\{a_{1},\dots,a_{t-1}\\}$ to remove actions already
selected in previous time steps of the current episode. At the step $t$, we
update the set $\Omega_{t}=\Omega_{t-1}\cup\\{a_{t}\\}$ and reduce the action
space $\mathcal{A}_{t}=\mathcal{A}_{t-1}\setminus\\{\Omega_{t-1}\\}$. We
generate a mask $M_{t}$ from the set $\Omega_{t}$ as:

$\displaystyle M_{t}$
$\displaystyle=\left(m_{ij}\right)_{i=1}^{d_{r}},_{j=1}^{d_{c}}\quad\text{where}\quad
m_{ij}=\begin{cases}1&\text{if }j\in\Omega_{t}\\\
0&\text{otherwise}\end{cases}$

An alternative approach to address the dynamic masking issue is to assign a
negative reward for selecting a previously acquired measurement. However, this
is inefficient in practice, and our experiments in Section 5.4 indicate a
strong advantage for our explicit masking technique.

###  4.2 Reward Model

Typically, RL problem settings have access to an oracle reward model directly
as part of the environment, for instance, the game score in Atari (Silver et
al., 2016). The agent then uses the rewards to optimize the policy parameters
$\theta$. In the ASMR setting, we do not have ‘oracle’ rewards but instead
have access to underlying pathology labels. However, it is not immediately
obvious how to use them to provide intermediate rewards for our agent.

To address the lack of an oracle reward model, we learn a mapping between the
undersampled k-space data and a quantifiable measure of how well the acquired
undersampled k-space relates to the labels. As such, we repurpose the models
used for direct k-space classification by EMRT (Singhal et al., 2023) to
compute the log-likelihood between the state $\mathbf{s}_{t}$ and the label
$\mathbf{y}$.

Since our agent sequentially acquires k-space data we need our reward model to
be able to provide rewards across varying undersampling rates. While a
different classifier for each sampling rate can be used, following Jethani et
al. (2021, 2023) we train the classifier using random sampling patterns across
sampling rates. Such a training approach allows us to train a single
classifier that is able to evaluate samples across all rates. For a given
state $s_{t}$ at time $t$, the reward model is defined as

$R(s_{t},a_{t})=\log q_{\phi}(\mathbf{y}|s_{t+1})$

where $q_{\phi}$ is a classifier and $s_{t+1}$ is obtained as in Equation 3.
The classifier is frozen while training the policy.

#### Policy architecture:

Following the PPO algorithm (Schulman et al., 2017), we adopt the actor-critic
architecture. Since our input is in the Fourier domain, we adapt the kspace-
net in Singhal et al. (2023) for the actor and critic models. The kspace-net
first performs a convolution in the Fourier domain (Mathieu et al., 2013). The
output of the convolution, which is complex-valued, is then transformed into
real values by taking the magnitude of its inverse Fourier transform. These
real-valued features are then fed into a backbone network, where any modern
convolutional network such as a ResNet (He et al., 2016a, b) can be plugged
in. In this work, we use a ResNet-18 as the backbone. The features from the
backbone are used as input to the actor and critic networks. The actor-network
processes these features into a 2-layer feed-forward layer to produce a
categorical distribution over the action space.

#### Balancing the training environment:

Traditionally, training environments are sampled uniformly; however, medical
datasets can have a highly imbalanced label distribution. For classification
problems, this label imbalance can cause the classifier to predict the
majority class and still achieve good classification performance. In such
situations, a common remedy is to either use importance weighting or up-
sampling. In this work, we train the policy by sampling environments in a
manner that ensures the agent sees data from all classes with equal
probability. For a given dataset labels $\\{\mathbf{y}_{i}\\}_{i=1}^{N}$, we
sample an environment $\mathbf{x}$ with probability proportional to the
inverse of its label frequency.

##  5 Experiments

| Knee Dataset | Brain Dataset | Prostate Dataset
---|---|---|---
| ACL | Mensc. Tear | Abnormal | Edema | Enlg. Ventricles | Mass | Abnormal | CS-PCA
Train slices | 29100 (3.6%) | 29100 (11.55%) | 29100 (15.9%) | 9420 (1.89%) | 9420 (1.66%) | 9420 (1.83%) | 9420 (24.06%) | 6649 (5%)
Validation slices | 6298 (2.49%) | 6298 (11.08%) | 6298 (15.39%) | 3148 (2.06%) | 3148 (1.43%) | 3148 (2.6%) | 3148 (20.71%) | 1431 (4.5%)
Test slices | 6281 (3.58%) | 6281 (11.94%) | 6281 (16.75%) | 3162 (2.44%) | 3162 (2.3%) | 3162 (2.4%) | 3162 (23.88%) | 1462 (6%)

Table 1: Number of slices in the training, validation and test splits for each
task. The number in the bracket is the percentage of slices with a pathology.

Our experiments are designed to answer the following questions:

  * •

How does ASMR compare to non-adaptive learned sampling methods?

  * •

How do ASMR policies compare to policies optimized for image reconstruction?

  * •

How important are the design choices to ASMR’s performance?

Figure 4: AUROCs obtained by ASMR compared to Learned Non-Adaptive Methods;
the horizontal dotted line in red denotes the performance of an image-based
classifier (which uses the entire k-space data). ASMR outperforms LOUPE and
DPS for 6 out of 8 tasks, and outperforms EMRT on 7 out of 8 tasks. All
results are computed over 5 seeds, and plotted with their means and standard
deviations. Figure 5: AUROCs obtained by ASMR compared to sequential sampling
methods such as VDS random and non-adaptive greedy sequence. In the low
sampling regime, ASMR consistently outperforms both baselines, while either
outperforming or matching their performance at higher sampling rates. All
results are computed over 5 seeds, and plotted with their means and standard
deviations.

#### Datasets:

We test ASMR on three different datasets consisting of knee, brain and
prostate MR scans:

  * •

Knee scans: For the knee dataset, we use the FastMRI knee dataset (Zbontar et
al., 2018) with slice-level labels provided by Zhao et al. (2021). The
training, validation, and test splits contain 816, 176, and 175 volumes,
respectively. Following Bien et al. (2018); Singhal et al. (2023), we predict
the presence of Meniscal Tears and ACL sprains in each slice. Additionally, we
also predict an extra category called abnormal, which is a collection of
pathologies that are less frequently observed (Bien et al., 2018) in the
dataset.

  * •

Brain scans: We use the FastMRI brain dataset with labels from Zhao et al.
(2021). Our training, validation, and test splits contain 600, 200, and 201
volumes, respectively. We predict the presence of Enlarged Ventricles, Mass,
Edema, and an abnormal category.

  * •

Prostate scans: For the prostate MR scans, we detect the presence of
clinically significant prostate cancer (CS-PCA). CS-PCA is defined as a lesion
within the prostate for which a radiologist assigns a Prostate Imaging
Reporting and Data System (PI-RADS) score (Weinreb et al., 2016) greater or
equal to 3. The data was split into 218, 48, and 47 volumes for training,
validation, and testing, respectively.

For each dataset, we predict the presence of pathologies in each slice. To
prevent label leakage we split each dataset at the volume level, ensuring
slices from the same volume do not spill across different splits. Slice level
splits along with positivity rates are provided in Table 1

Due to computational constraints, we use the single-coil k-space generated
from the multi-coil k-space matrix. Similar to Zbontar et al. (2018), we use
the emulated single-coil (Tygert & Zbontar, 2020) to convert the multi-coil
brain and prostate into single-coil. The emulated single-coil method takes a
complex-valued linear combination of coils to produce a single matrix, where
the combination of weights is learned per sample. For details see (Tygert &
Zbontar, 2020; Zbontar et al., 2018). With additional computional resources,
ASMR can be extended to multi-coil by just adapting the kspace-net to accept
multi-coil k-space as input.

#### Mask Distributions:

In Appendix E, we plot the distribution of masks selected by ASMR on each of
the datasets.

#### Evaluation metrics.

For comparisons on imbalanced datasets, a commonly used metric is the area
under the receiver operator curve (auroc). In Appendix A, we provide
classification metrics other than auroc.

#### Baselines:

We compare ASMR against several methods, including both adaptive and non-
adaptive methods.

  1. 1.

EMRT: EMRT learns a single classifier and a non-adaptive pattern for each
sampling rate.

  2. 2.

Fully-Sampled Image-based Classifier: A classifier trained on images generated
from the fully-sampled k-space. This is the standard input for DL models
performing image analysis.

  3. 3.

Learned Non-Adaptive Probabilistic Methods: For a fixed sampling rate, we
repurpose loupe (Bahadir et al., 2019) and dps (Huijben et al., 2019) to learn
a classifier and a probabilistic sampling mask jointly.

  4. 4.

Random Policy: We use a random policy that samples k-space columns using the
variable density sampling (VDS) prior (Lustig et al., 2007).

  5. 5.

Reconstruction Optimized Policy: We use the greedy policy method in (Bakker et
al., 2020), using the same 16 channel U-Net (Ronneberger et al., 2015) as the
reconstruction model using single-coil data.

  6. 6.

Non-Adaptive Greedy Sequence: A sequential non-adaptive method that uses the
same kspace-net used in ASMR to do forward greedy selection Macedo et al.
(2019), producing a sequence of masks $\mathbf{s}_{1},\dots,\mathbf{s}_{d}$.

#### Pathology classifier:

For classifiers that use the k-space as input we use the kspace-net
architecture (Singhal et al., 2023) with either a ResNet-18 or ResNet-50 as
the backbone architecture. For methods that take an image as input, we use a
ResNet-50.

###  5.1 Comparing ASMR to non-adaptive methods

Following (Singhal et al., 2023), we train EMRT with the kspace-net classifier
using random masks sampled from the VDS prior. Then, using the scoring rule
defined in Equation 1 we sample $K=100$ masks at a fixed sampling rate from
the VDS prior and select the mask that maximizes the score rule. For gradient-
based mask learning, we adapt DPS and LOUPE to learn a probabilistic mask and
a classifier jointly. All these methods learn a classifier and a non-adaptive
sampling pattern for a fixed sampling rate. We train EMRT, DPS, and LOUPE at
5%, 8%, 10%, and 12.5% sampling rates. Figure 4 compares the performance of
ASMR over these non-adaptive methods. As an upper bound, we also provide the
performance of an image-based classifier that uses the Inverse Fourier
Transform of the fully sampled k-space as input. We note that ASMR approaches
the performance of a fully sampled classifier using just 8% of the samples.

###  5.2 Comparing ASMR to sequential sampling methods

ASMR sequentially collects k-space samples, unlike non-adaptive methods like
DPS and LOUPE. As such, we compare ASMR to other methods that perform
sequential sampling. Following Macedo et al. (2019), we use the pre-trained
classifier $q_{\phi}$ as in Section 4.2 and the scoring rule in Equation 1 to
iteratively build a sequence
$\mathbf{s}_{t+1}=\mathbf{s}_{t}+\arg\max_{j}\mathbf{V}(\mathbf{s}_{t}+\mathbf{e}_{j})$,
starting with $\mathbf{s}_{0}=0$. To evaluate the scoring rule, we use a
balanced subset of the validation set to build this sequence. This is similar
to the greedy non-adaptive oracle considered in Bakker et al. (2020). The
greedy sequence suffers from high variance in classification metrics as can be
observed across all datasets in Figure 5, and plateaus beyond a certain
sampling rate.

On average, ASMR achieves 1.87%, 7.01%, and 9.82% absolute gains on AUROC over
greedy solutions across the knee, brain, and prostate datasets, respectively.
The VDS random policy, greedy sequence, and ASMR use the same pre-trained
classifier for evaluation. Figure 5 shows a significant performance gap
between ASMR and VDS, particularly in the low sampling rate regime ($\leq 8\%$
sampling rate). On average, we see a 1.33%, 2%, and 2.25% gain for ASMR over
VDS on the knee, brain, and prostate datasets, respectively.

###  5.3 Comparing ASMR to reconstruction optimized policies

To show the benefits of learning policies for classification as opposed to
image reconstruction, we follow the open source implementation of Bakker et
al. (2020) to train a policy model to reconstruct scan images from
undersampled k-space data. Since Bakker et al. (2020) requires specifying a
target sampling rate, we train separate reconstruction-based policies for each
sampling rate. Masks selected by the reconstruction-based policy are then
evaluated using the same classifiers used to evaluate ASMR. Figure 6 compares
the two policies in terms of the classification performance of different
sampling rates. ASMR consistently outperforms reconstruction-based policies
across all four pathologies in the brain dataset, achieving an average of 7%
performance gains. Additional results for knee data and prostate data are
included in Appendix B.1.

Figure 6: AUROCs obtained by ASMR compared to a reconstruction-based policy on
the Brain test dataset. ASMR significantly outperforms these policies across
all sampling rates, indicating the limitations of reconstruction objectives
towards pathology prediction.

###  5.4 Importance of design decisions for ASMR

To understand the important design decisions of ASMR, we perform two ablations
on our training regime choices, using a label-balanced training environment
and dynamically masking the action space. We perform these experiments on the
knee dataset only and evaluate selected samples by each ablated policy using
the same set of classifiers that are used by ASMR. First, we investigate the
effect of using a label-balanced training environment. For this, we implement
a “Naive Environment Sampling”, that samples the dataset uniformly during the
training. Next, we ablate on the dynamic action space masking during training.
Instead, we implement a reward penalty of $-1$ when the policy selects a
previously acquired k-space measurement. Figure 7 shows the results of our
ablations. We hypothesize that without the label-balanced training
environment, the learned policy is biased towards normal patients, causing
performance drops across all sampling rates evaluated. Our design choices are
crucial to improve the performance of ASMR over a vanilla PPO framework.

Figure 7: Ablation analysis on the Knee dataset for different components of
ASMR, where significant performance gain is a achieved.

##  6 Conclusion and Limitations

We have presented ASMR, an RL-based adaptive sampling technique for pathology
prediction. ASMR improves over prior state-of-the-art methods in MR sampling
and is a step towards enabling MR-based screening at a population scale as it
offers strong pathology prediction using $12\times$ fewer k-space samples.
However, several challenges still remain before widespread adoption.

#### Limitations:

ASMR currently only uses single-coil k-space data while many modern MR
scanners now collect multi-coil k-space data (Deshmane et al., 2012). We
believe that ASMR can also be applied to the multi-coil setting as the under-
sampling pattern for each coil is the same. Next, we note that due to limited
volumetric data, we worked with slice-level classifiers rather than volume-
level classifiers. Extending ASMR to volumetric classification or pathology
segmentation is an exciting future direction.

## Impact Statement

Methods that accelerate MR scans and enable wide-spread use of MR technology
for screening need to be thoroughly validated prior to clinical adoption.
Suddenly increasing the screening population can lead to false positives
causing patient trauma and significant cost increases to the healthcare system
(Kilpeläinen et al., 2010; Lafata et al., 2004). Therefore, prior to clinical
adoption, methods such as ASMR need to be improved and undergo extensive
clinical trials.

## Acknowledgements

The computational requirements for this work were supported in part by the
resources and personnel of the NYU Langone High Performance Computing (HPC)
Core. LP is supported by the Packard Fellowship. We also thank Shenglong Wang
and the NYU High Performance Computing for their support.

## References

  * Achiam et al. (2023) Achiam, J., Adler, S., Agarwal, S., Ahmad, L., Akkaya, I., Aleman, F. L., Almeida, D., Altenschmidt, J., Altman, S., Anadkat, S., et al.  Gpt-4 technical report.  _arXiv preprint arXiv:2303.08774_ , 2023.
  * Bahadir et al. (2019) Bahadir, C. D., Dalca, A. V., and Sabuncu, M. R.  Learning-based optimization of the under-sampling pattern in mri.  In _International Conference on Information Processing in Medical Imaging_ , pp. 780–792. Springer, 2019.
  * Bakker et al. (2020) Bakker, T., van Hoof, H., and Welling, M.  Experimental design for mri by greedy policy search.  _Advances in Neural Information Processing Systems_ , 33, 2020.
  * Bellemare et al. (2020) Bellemare, M. G., Candido, S., Castro, P. S., Gong, J., Machado, M. C., Moitra, S., Ponda, S. S., and Wang, Z.  Autonomous navigation of stratospheric balloons using reinforcement learning.  _Nature_ , 588(7836):77–82, 2020.
  * Bien et al. (2018) Bien, N., Rajpurkar, P., Ball, R. L., Irvin, J., Park, A., Jones, E., Bereket, M., Patel, B. N., Yeom, K. W., Shpanskaya, K., et al.  Deep-learning-assisted diagnosis for knee magnetic resonance imaging: development and retrospective validation of mrnet.  _PLoS medicine_ , 15(11):e1002699, 2018.
  * Chen et al. (2023) Chen, H., Zhang, X., Huang, K., and Koushanfar, F.  Adatest: Reinforcement learning and adaptive sampling for on-chip hardware trojan detection.  _ACM Transactions on Embedded Computing Systems_ , 22(2):1–23, 2023.
  * Chung & Ye (2022) Chung, H. and Ye, J. C.  Score-based diffusion models for accelerated mri.  _Medical image analysis_ , 80:102479, 2022.
  * Cover (1999) Cover, T. M.  _Elements of information theory_.  John Wiley & Sons, 1999.
  * Dean Deyle (2011) Dean Deyle, G.  The role of mri in musculoskeletal practice: a clinical perspective.  _Journal of Manual & Manipulative Therapy_, 19(3):152–161, 2011.
  * Deshmane et al. (2012) Deshmane, A., Gulani, V., Griswold, M. A., and Seiberlich, N.  Parallel mr imaging.  _Journal of Magnetic Resonance Imaging_ , 36(1):55–72, 2012.
  * Donoho (2006) Donoho, D. L.  Compressed sensing.  _IEEE Transactions on information theory_ , 52(4):1289–1306, 2006.
  * Elmore et al. (1998) Elmore, J. G., Barton, M. B., Moceri, V. M., Polk, S., Arena, P. J., and Fletcher, S. W.  Ten-year risk of false positive screening mammograms and clinical breast examinations.  _New England Journal of Medicine_ , 338(16):1089–1096, 1998.
  * Elvenes et al. (2000) Elvenes, J., Jerome, C., Reikerås, O., and Johansen, O.  Magnetic resonance imaging as a screening procedure to avoid arthroscopy for meniscal tears.  _Archives of orthopaedic and trauma surgery_ , 120(1-2):14–16, 2000.
  * Gözcü et al. (2019) Gözcü, B., Sanchez, T., and Cevher, V.  Rethinking sampling in parallel mri: A data-driven approach.  In _2019 27th European Signal Processing Conference (EUSIPCO)_ , pp. 1–5. IEEE, 2019.
  * Hambly et al. (2023) Hambly, B., Xu, R., and Yang, H.  Recent advances in reinforcement learning in finance.  _Mathematical Finance_ , 33(3):437–503, 2023.  doi: https://doi.org/10.1111/mafi.12382.  URL https://onlinelibrary.wiley.com/doi/abs/10.1111/mafi.12382.
  * He et al. (2016a) He, K., Zhang, X., Ren, S., and Sun, J.  Deep residual learning for image recognition.  In _Proceedings of the IEEE conference on computer vision and pattern recognition_ , pp. 770–778, 2016a.
  * He et al. (2016b) He, K., Zhang, X., Ren, S., and Sun, J.  Identity mappings in deep residual networks.  In _European conference on computer vision_ , pp. 630–645. Springer, 2016b.
  * Huang et al. (2022) Huang, S., Dossa, R. F. J., Ye, C., Braga, J., Chakraborty, D., Mehta, K., and Araújo, J. G.  Cleanrl: High-quality single-file implementations of deep reinforcement learning algorithms.  _Journal of Machine Learning Research_ , 23(274):1–18, 2022.  URL http://jmlr.org/papers/v23/21-1342.html.
  * Huijben et al. (2019) Huijben, I. A., Veeling, B. S., and van Sloun, R. J.  Deep probabilistic subsampling for task-adaptive compressed sensing.  In _International Conference on Learning Representations_ , 2019.
  * Ilic et al. (2018) Ilic, D., Djulbegovic, M., Jung, J. H., Hwang, E. C., Zhou, Q., Cleves, A., Agoritsas, T., and Dahm, P.  Prostate cancer screening with prostate-specific antigen (psa) test: a systematic review and meta-analysis.  _Bmj_ , 362, 2018.
  * Jethani et al. (2021) Jethani, N., Sudarshan, M., Aphinyanaphongs, Y., and Ranganath, R.  Have we learned to explain?: How interpretability methods can learn to encode predictions in their interpretations.  In _International Conference on Artificial Intelligence and Statistics_ , pp. 1459–1467. PMLR, 2021.
  * Jethani et al. (2023) Jethani, N., Saporta, A., and Ranganath, R.  Don’t be fooled: label leakage in explanation methods and the importance of their quantitative evaluation.  In _International Conference on Artificial Intelligence and Statistics_ , pp. 8925–8953. PMLR, 2023.
  * Jin et al. (2019) Jin, K. H., Unser, M., and Yi, K. M.  Self-supervised deep active accelerated mri.  _arXiv preprint arXiv:1901.04547_ , 2019.
  * Johnson et al. (2022) Johnson, P. M., Tong, A., Donthireddy, A., Melamud, K., Petrocelli, R., Smereka, P., Qian, K., Keerthivasan, M. B., Chandarana, H., and Knoll, F.  Deep learning reconstruction enables highly accelerated biparametric mr imaging of the prostate.  _Journal of Magnetic Resonance Imaging_ , 56(1):184–195, 2022.
  * Kilpeläinen et al. (2010) Kilpeläinen, T., Tammela, T., Määttänen, L., Kujala, P., Stenman, U.-H., Ala-Opas, M., Murtola, T., and Auvinen, A.  False-positive screening results in the finnish prostate cancer screening trial.  _British journal of cancer_ , 102(3):469–474, 2010.
  * Knoll et al. (2020) Knoll, F., Murrell, T., Sriram, A., Yakubova, N., Zbontar, J., Rabbat, M., Defazio, A., Muckley, M. J., Sodickson, D. K., Zitnick, C. L., et al.  Advancing machine learning for mr image reconstruction with an open competition: Overview of the 2019 fastmri challenge.  _Magnetic resonance in medicine_ , 84(6):3054–3070, 2020.
  * Kumar et al. (2022) Kumar, A., Yazdanbakhsh, A., Hashemi, M., Swersky, K., and Levine, S.  Data-driven offline optimization for architecting hardware accelerators.  In _International Conference on Learning Representations_ , 2022.  URL https://openreview.net/forum?id=GsH-K1VIyy.
  * Lafata et al. (2004) Lafata, J. E., Simpkins, J., Lamerato, L., Poisson, L., Divine, G., and Johnson, C. C.  The economic impact of false-positive cancer screens.  _Cancer Epidemiology and Prevention Biomarkers_ , 13(12):2126–2132, 2004.
  * Liang & Lauterbur (2000) Liang, Z.-P. and Lauterbur, P. C.  _Principles of magnetic resonance imaging_.  SPIE Optical Engineering Press Bellingham, 2000.
  * Lustig et al. (2007) Lustig, M., Donoho, D., and Pauly, J. M.  Sparse mri: The application of compressed sensing for rapid mr imaging.  _Magnetic Resonance in Medicine: An Official Journal of the International Society for Magnetic Resonance in Medicine_ , 58(6):1182–1195, 2007.
  * Lustig et al. (2008) Lustig, M., Donoho, D. L., Santos, J. M., and Pauly, J. M.  Compressed sensing mri.  _IEEE signal processing magazine_ , 25(2):72–82, 2008.
  * Macedo et al. (2019) Macedo, F., Oliveira, M. R., Pacheco, A., and Valadas, R.  Theoretical foundations of forward feature selection methods based on mutual information.  _Neurocomputing_ , 325:67–89, 2019.
  * Mathieu et al. (2013) Mathieu, M., Henaff, M., and LeCun, Y.  Fast training of convolutional networks through ffts.  _arXiv preprint arXiv:1312.5851_ , 2013.
  * Pan et al. (2022) Pan, L., Manjanna, S., and Hsieh, M. A.  Marlas: Multi agent reinforcement learning for cooperated adaptive sampling.  _arXiv preprint arXiv:2207.07751_ , 2022.
  * Pineda et al. (2020) Pineda, L., Basu, S., Romero, A., Calandra, R., and Drozdzal, M.  Active mr k-space sampling with reinforcement learning.  In _International Conference on Medical Image Computing and Computer-Assisted Intervention_ , pp. 23–33. Springer, 2020.
  * Ronneberger et al. (2015) Ronneberger, O., Fischer, P., and Brox, T.  U-net: Convolutional networks for biomedical image segmentation.  In _International Conference on Medical image computing and computer-assisted intervention_ , pp. 234–241. Springer, 2015.
  * Schulman et al. (2017) Schulman, J., Wolski, F., Dhariwal, P., Radford, A., and Klimov, O.  Proximal policy optimization algorithms.  _arXiv preprint arXiv:1707.06347_ , 2017.
  * Silver et al. (2016) Silver, D., Huang, A., Maddison, C. J., Guez, A., Sifre, L., Van Den Driessche, G., Schrittwieser, J., Antonoglou, I., Panneershelvam, V., Lanctot, M., et al.  Mastering the game of go with deep neural networks and tree search.  _nature_ , 529(7587):484–489, 2016.
  * Singhal et al. (2023) Singhal, R., Sudarshan, M., Mahishi, A., Kaushik, S., Ginocchio, L., Tong, A., Chandarana, H., Sodickson, D. K., Ranganath, R., and Chopra, S.  On the feasibility of machine learning augmented magnetic resonance for point-of-care identification of disease.  _arXiv preprint arXiv:2301.11962_ , 2023.
  * Sriram et al. (2020) Sriram, A., Zbontar, J., Murrell, T., Defazio, A., Zitnick, C. L., Yakubova, N., Knoll, F., and Johnson, P.  End-to-end variational networks for accelerated mri reconstruction.  In _International Conference on Medical Image Computing and Computer-Assisted Intervention_ , pp. 64–73. Springer, 2020.
  * Sutton & Barto (2018) Sutton, R. S. and Barto, A. G.  _Reinforcement learning: An introduction_.  MIT press, 2018.
  * Tolpadi et al. (2023) Tolpadi, A. A., Bharadwaj, U., Gao, K. T., Bhattacharjee, R., Gassert, F. G., Luitjens, J., Giesler, P., Morshuis, J. N., Fischer, P., Hein, M., et al.  K2s challenge: From undersampled k-space to automatic segmentation.  _Bioengineering_ , 10(2):267, 2023.
  * Tygert & Zbontar (2020) Tygert, M. and Zbontar, J.  Simulating single-coil mri from the responses of multiple coils.  _Communications in Applied Mathematics and Computational Science_ , 15(2):115–127, 2020.
  * Wang et al. (2004) Wang, Z., Bovik, A. C., Sheikh, H. R., and Simoncelli, E. P.  Image quality assessment: from error visibility to structural similarity.  _IEEE transactions on image processing_ , 13(4):600–612, 2004.
  * Weinreb et al. (2016) Weinreb, J. C., Barentsz, J. O., Choyke, P. L., Cornud, F., Haider, M. A., Macura, K. J., Margolis, D., Schnall, M. D., Shtern, F., Tempany, C. M., et al.  Pi-rads prostate imaging–reporting and data system: 2015, version 2.  _European urology_ , 69(1):16–40, 2016.
  * Weiss et al. (2019) Weiss, T., Senouf, O., Vedula, S., Michailovich, O., Zibulevsky, M., and Bronstein, A.  Pilot: Physics-informed learned optimized trajectories for accelerated mri.  _arXiv preprint arXiv:1909.05773_ , 2019.
  * Weiss et al. (2020) Weiss, T., Vedula, S., Senouf, O., Michailovich, O., Zibulevsky, M., and Bronstein, A.  Joint learning of cartesian under sampling andre construction for accelerated mri.  In _ICASSP 2020-2020 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP)_ , pp. 8653–8657. IEEE, 2020.
  * Winawer et al. (1997) Winawer, S. J., Fletcher, R. H., Miller, L., Godlee, F., Stolar, M., Mulrow, C., Woolf, S., Glick, S., Ganiats, T., Bond, J., et al.  Colorectal cancer screening: clinical guidelines and rationale.  _Gastroenterology_ , 112(2):594–642, 1997.
  * Zbontar et al. (2018) Zbontar, J., Knoll, F., Sriram, A., Murrell, T., Huang, Z., Muckley, M. J., Defazio, A., Stern, R., Johnson, P., Bruno, M., et al.  fastmri: An open dataset and benchmarks for accelerated mri.  _arXiv preprint arXiv:1811.08839_ , 2018.
  * Zhang et al. (2019) Zhang, Z., Romero, A., Muckley, M. J., Vincent, P., Yang, L., and Drozdzal, M.  Reducing uncertainty in undersampled mri reconstruction with active acquisition.  In _Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition_ , pp. 2049–2058, 2019.
  * Zhao et al. (2021) Zhao, R., Yaman, B., Zhang, Y., Stewart, R., Dixon, A., Knoll, F., Huang, Z., Lui, Y. W., Hansen, M. S., and Lungren, M. P.  fastmri+: Clinical pathology annotations for knee and brain fully sampled multi-coil mri data.  _arXiv preprint arXiv:2109.03812_ , 2021.
  * Zucker et al. (2008) Zucker, M., Kuffner, J., and Bagnell, J. A.  Adaptive workspace biasing for sampling-based planners.  In _2008 IEEE International Conference on Robotics and Automation_ , pp. 3757–3762. IEEE, 2008.

## Appendix

##  Appendix A Comparison to Learning-based Methods

Here we present comparisons of ASMR to methods that learn a non-adaptive mask.
Table 2 shows the AUC metrics across 3 pathologies in the Knee dataset. We
provide the mean and standard deviations across 5 seeds. Table 3 and Table 4
provide the same results for pathologies in the Brain and Prostate datasets
respectively.

Pathology - Sampling Rate (%) | DPS | EMRT | LOUPE | ASMR (Ours)
---|---|---|---|---
Abnormal - 5 | 82.0 ± 0.6 | 81.5 ± 1.0 | 81.8 ± 1.1 | 86.8 ± 0.2
Abnormal - 8 | 82.7 ± 0.8 | 81.1 ± 0.7 | 83.9 ± 1.0 | 87.7 ± 0.4
Abnormal - 10 | 83.7 ± 1.0 | 80.3 ± 1.7 | 83.5 ± 0.9 | 88.1 ± 0.2
Abnormal - 12.5 | 82.4 ± 2.9 | 82.0 ± 1.5 | 84.6 ± 0.9 | 88.3 ± 0.2
ACL - 5 | 91.3 ± 0.5 | 88.7 ± 0.9 | 91.0 ± 1.1 | 93.0 ± 0.4
ACL - 8 | 91.3 ± 0.7 | 88.5 ± 2.7 | 91.4 ± 0.9 | 93.8 ± 0.4
ACL - 10 | 91.9 ± 0.5 | 89.3 ± 1.3 | 91.8 ± 0.2 | 93.9 ± 0.3
ACL - 12.5 | 91.5 ± 1.9 | 89.4 ± 0.5 | 92.1 ± 1.3 | 94.0 ± 0.4
Mensc. Tear - 5 | 91.2 ± 0.2 | 90.3 ± 0.9 | 91.2 ± 0.4 | 94.2 ± 0.2
Mensc. Tear - 8 | 91.2 ± 0.2 | 91.3 ± 1.7 | 92.1 ± 0.6 | 94.7 ± 0.2
Mensc. Tear - 10 | 91.8 ± 0.4 | 90.9 ± 1.3 | 92.1 ± 0.4 | 94.9 ± 0.2
Mensc. Tear - 12.5 | 91.7 ± 1.0 | 91.7 ± 1.0 | 92.6 ± 0.5 | 95.1 ± 0.2
Table 2: Test AUC for identifying pathologies under different sampling rates on the Knee dataset Pathology - Sampling Rate (%) | DPS | EMRT | LOUPE | ASMR (Ours)
---|---|---|---|---
Edema - 5 | 76.5 ± 2.5 | 82.0 ± 1.2 | 80.1 ± 5.1 | 84.4 ± 2.2
Edema - 8 | 74.7 ± 3.3 | 82.1 ± 0.9 | 81.1 ± 2.6 | 86.8 ± 1.2
Edema - 10 | 77.7 ± 2.3 | 85.4 ± 1.5 | 82.5 ± 2.5 | 87.2 ± 1.6
Edema - 12.5 | 74.5 ± 1.9 | 86.6 ± 2.5 | 85.1 ± 1.4 | 87.8 ± 1.6
Enlg. Ventricles - 5 | 76.9 ± 7.3 | 91.2 ± 1.4 | 86.9 ± 3.0 | 89.1 ± 4.1
Enlg. Ventricles - 8 | 76.5 ± 4.2 | 89.4 ± 2.1 | 88.7 ± 1.9 | 90.6 ± 3.1
Enlg. Ventricles - 10 | 74.2 ± 2.7 | 88.0 ± 3.7 | 87.0 ± 5.7 | 91.3 ± 2.9
Enlg. Ventricles - 12.5 | 78.9 ± 6.4 | 90.4 ± 1.5 | 90.5 ± 2.2 | 91.8 ± 2.7
Mass - 5 | 74.3 ± 1.2 | 80.5 ± 1.3 | 77.6 ± 4.4 | 86.2 ± 3.6
Mass - 8 | 73.4 ± 2.1 | 82.4 ± 2.0 | 83.5 ± 1.7 | 87.7 ± 3.1
Mass - 10 | 74.9 ± 2.6 | 84.9 ± 1.0 | 82.2 ± 3.1 | 87.7 ± 2.5
Mass - 12.5 | 73.8 ± 3.7 | 85.3 ± 2.5 | 86.0 ± 3.0 | 88.0 ± 2.8
Abnormal - 5 | 64.9 ± 3.2 | 72.9 ± 0.9 | 70.0 ± 2.2 | 72.2 ± 1.0
Abnormal - 8 | 66.1 ± 1.5 | 75.0 ± 0.4 | 70.9 ± 2.5 | 74.9 ± 1.1
Abnormal- 10 | 66.8 ± 3.8 | 75.8 ± 0.9 | 73.3 ± 2.0 | 75.6 ± 1.3
Abnormal - 12.5 | 67.7 ± 1.2 | 76.8 ± 0.9 | 74.1 ± 1.4 | 76.3 ± 1.5
Table 3: Test AUC for identifying pathologies under different sampling rates on the Brain dataset Pathology - Sampling Rate (%) | EMRT | DPS | LOUPE | ASMR (Ours)
---|---|---|---|---
CS-PCA - 5 | 83.8 ± 1.5 | 66.14 ± 5.8 | 75.00 ± 4.1 | 78.46 ± 2.3
CS-PCA - 8 | 83.5 ± 1.9 | 63.30 ± 6.6 | 79.68 ± 2.7 | 78.73 ± 2.4
CS-PCA - 10 | 83.4 ± 2.1 | 66.41 ± 7.4 | 82.83 ± 0.6 | 79.06 ± 1.9
CS-PCA - 12.5 | 82.7 ± 1.4 | 69.73 ± 4.7 | 81.70 ± 1.4 | 80.24 ± 2.2
Table 4: Test AUCs for identifying pathologies under different sampling rates
on the Prostate dataset

##  Appendix B Additional Experiments

###  B.1 Comparison to Reconstruction Optimized Policies on Knee and Prostate
Dataset

To demonstrate the utility of optimizing a policy for classification
performance, following Bakker et al. (2020) we also train policies to optimize
for reconstruction errors. Using k-space subsets selected by the
reconstruction optimized policy, we evaluate these subsets using our pathology
classifier. Section B.1 shows the results of these experiments and indicate
that reconstruction-based objectives are not necessarily suited for
classification.

##  Appendix C Additional Classification Metrics

###  C.1 Knee Results of ASMR

Pathology - Sampling Rate (%) | Balanced Accuracy | Sensitivity | Specifity | NPV
---|---|---|---|---
ACL - 5 | 86.1 ± 0.5 | 89.8 ± 0.0 | 82.4 ± 1.1 | 99.5 ± 0.0
ACL - 8 | 86.5 ± 0.7 | 89.8 ± 0.0 | 83.2 ± 1.5 | 99.5 ± 0.0
ACL - 10 | 86.8 ± 0.6 | 89.8 ± 0.0 | 83.9 ± 1.2 | 99.6 ± 0.0
ACL - 12.5 | 86.7 ± 0.3 | 89.8 ± 0.0 | 83.7 ± 0.7 | 99.5 ± 0.0
Mensc. Tear - 5 | 86.8 ± 0.3 | 89.9 ± 0.1 | 83.7 ± 0.5 | 98.4 ± 0.0
Mensc. Tear - 8 | 87.6 ± 0.5 | 89.9 ± 0.0 | 85.3 ± 1.0 | 98.4 ± 0.0
Mensc. Tear - 10 | 88.0 ± 0.3 | 89.9 ± 0.0 | 86.1 ± 0.6 | 98.4 ± 0.0
Mensc. Tear - 12.5 | 88.0 ± 0.5 | 89.9 ± 0.0 | 86.2 ± 1.0 | 98.4 ± 0.0
Abnoraml - 5 | 76.5 ± 0.9 | 89.9 ± 0.0 | 63.2 ± 1.8 | 96.9 ± 0.1
Abnoraml - 8 | 77.6 ± 0.8 | 89.9 ± 0.0 | 65.3 ± 1.6 | 97.0 ± 0.1
Abnoraml - 10 | 77.7 ± 0.4 | 89.9 ± 0.0 | 65.4 ± 0.9 | 97.0 ± 0.0
Abnoraml - 12.5 | 77.6 ± 0.6 | 89.9 ± 0.0 | 65.4 ± 1.2 | 97.0 ± 0.1

###  C.2 Brain Results of ASMR

Pathology - Sampling Rate (%) | Balanced Accuracy | Sensitivity | Specifity | NPV
---|---|---|---|---
Edema - 5 | 74.2 ± 2.5 | 89.8 ± 0.0 | 82.4 ± 1.1 | 99.5 ± 0.0
Edema - 8 | 76.9 ± 1.1 | 89.8 ± 0.0 | 83.2 ± 1.5 | 99.5 ± 0.0
Edema - 10 | 77.6 ± 2.1 | 89.8 ± 0.0 | 83.9 ± 1.2 | 99.6 ± 0.0
Edema - 12.5 | 78.7 ± 1.9 | 89.8 ± 0.0 | 83.7 ± 0.7 | 99.5 ± 0.0
Enlg. Ventricles - 5 | 78.1 ± 7.1 | 89.9 ± 0.1 | 83.7 ± 0.5 | 98.4 ± 0.0
Enlg. Ventricles - 8 | 78.9 ± 7.2 | 89.9 ± 0.0 | 85.3 ± 1.0 | 98.4 ± 0.0
Enlg. Ventricles - 10 | 79.8 ± 7.3 | 89.9 ± 0.0 | 86.1 ± 0.6 | 98.4 ± 0.0
Enlg. Ventricles - 12.5 | 81.2 ± 7.3 | 89.9 ± 0.0 | 86.2 ± 1.0 | 98.4 ± 0.0
Mass - 5 | 74.6 ± 3.7 | 89.9 ± 0.0 | 63.2 ± 1.8 | 96.9 ± 0.1
Mass - 8 | 76.5 ± 3.3 | 89.9 ± 0.0 | 65.3 ± 1.6 | 97.0 ± 0.1
Mass - 10 | 77.1 ± 2.3 | 89.9 ± 0.0 | 65.4 ± 0.9 | 97.0 ± 0.0
Mass - 12.5 | 77.0 ± 1.8 | 89.9 ± 0.0 | 65.4 ± 1.2 | 97.0 ± 0.1
Abnormal - 5 | 62.1 ± 2.0 | 72.9 ± 0.9 | 70.0 ± 2.2 | 72.2 ± 1.0
Abnormal - 8 | 64.1 ± 1.3 | 75.0 ± 0.4 | 70.9 ± 2.5 | 74.9 ± 1.1
Abnormal- 10 | 64.9 ± 1.8 | 75.8 ± 0.9 | 73.3 ± 2.0 | 75.6 ± 1.3
Abnormal - 12.5 | 65.4 ± 1.6 | 76.8 ± 0.9 | 74.1 ± 1.4 | 76.3 ± 1.5

###  C.3 Prostate Results of ASMR

Pathology - Sampling Rate (%) | Balanced Accuracy | Sensitivity | Specifity | NPV
---|---|---|---|---
CS-PCA - 5 | 68.3 ± 2.0 | 89.7 ± 0.0 | 47.0 ± 3.9 | 98.4 ± 0.1
CS-PCA - 8 | 70.0 ± 3.7 | 89.7 ± 0.0 | 50.2 ± 7.5 | 98.5 ± 0.2
CS-PCA - 10 | 68.4 ± 3.0 | 89.7 ± 0.0 | 47.2 ± 6.0 | 98.4 ± 0.2
CS-PCA - 12.5 | 68.0 ± 2.5 | 89.7 ± 0.0 | 46.4 ± 5.1 | 98.4 ± 0.2

##  Appendix D Training Details

###  D.1 Training Parameters

The k-space data input to agent is in complex domain with height and width of
${d_{r}\times d_{c}}$. The ${d_{r}\times d_{c}}$ for knee, brain, prostate are
$(768\times 400)$, $(640\times 400)$, and $(320\times 451)$, respectively.
Hyperparameters used to train the policy are provided in Table 5.

Parameter | Value
---|---
Optimizer | Adamw
Learning rate | 1e-04
weight decay | 1e-04
discount factor | 0.99
gae_lambda | 0.95
clip ratio | 0.2
entropy cost | 0.01
grad norm clipping | 0.5
value function coefficient | 0.5
parallelized rollout | 128
Table 5: Hyperparameters of our agent

##  Appendix E Policy selected Mask Distributions

Below we plot a heatmap of the k-space column selections made by ASMR on the
Knee, Brain and Prostate datasets. We evaluate our policy to have a sampling
rate of $12.5\%$ and generate heat maps representing the fraction of samples
in the test set for which a column was picked.