# Recurrent flow patterns as a basis for turbulence: predicting statistics
from structures

Jacob Page1 Peter Norgaard2 Michael P. Brenner2,3 Rich R. Kerswell4 1School of
Mathematics, University of Edinburgh, Edinburgh, EH9 3FD, UK  2Google
Research, Mountain View, CA 94043  3School of Engineering and Applied
Sciences, Harvard University, Cambridge MA 02138  4DAMTP, Centre for
Mathematical Sciences, University of Cambridge, Cambridge, CB3 0WA, UK

###### Abstract

A dynamical systems approach to turbulence envisions the flow as a trajectory
through a high-dimensional state space transiently visiting the neighbourhoods
of unstable simple invariant solutions (E. Hopf, Commun. Appl. Maths 1, 303,
1948). The grand hope has always been to turn this appealing picture into a
predictive framework where the statistics of the flow follows from one
appropriately-weighted sum of the statistics of each simple invariant
solution. Two outstanding obstacles have prevented this goal from being
achieved: $(1)$ paucity of known solutions and $(2)$ as yet no rational theory
to predict the required weights. Here we describe a method to substantially
solve the former and a data-driven approach to overcome the latter. In doing
so, we provide the first compelling evidence that the statistics of a fully
developed turbulent flow can be reconstructed with a set of unstable periodic
orbits (UPOs). The new method for finding solutions is based on automatic
differentiation in which high-quality guesses are constructed by minimising a
trajectory-dependent loss function with gradient descent and then these are
fully converged using a Newton solve. Unlike past work, the approach does not
require observation of a ‘near recurrence’ on a turbulent orbit and can be
used to target solutions with specific physical properties via the addition of
appropriate terms to the loss. Scores of new solutions are found with short
periods in turbulent, two-dimensional Kolmogorov flow. These can be used to
reproduce a variety of statistical quantities after ‘learning’ weights for
each periodic orbit by fitting to one statistic only. To our knowledge, this
is the first time the statistics of a spatio-temporally-chaotic system have
been successfully reproduced with a set of simple invariant states, and
provides a fascinating connection between self-sustaining dynamical processes
and the more well-known statistical properties of turbulence.

##  I Introduction

In recent decades, ideas from dynamical systems theory have greatly improved
our understanding of transitional and weakly turbulent shear flows (Kawahara
_et al._ , 2012; Graham and Floryan, 2021). As examples, the onset of
turbulence in pipes has been connected to the emergence of finite-amplitude
travelling wave solutions beyond some critical Reynolds number, $Re$, in
saddle node bifurcations (Kerswell, 2005; Eckhardt  _et al._ , 2007), while
the later discovery of unstable periodic orbits (UPOs) in so-called ‘minimal’
turbulent configurations (Kawahara and Kida, 2001; Cvitanovic and Gibson,
2010) has revealed some of the self-sustaining mechanisms at play in wall
bounded turbulence (Hamilton  _et al._ , 1995; Waleffe, 1997; Hall and
Sherwin, 2010). The dynamical systems view of fully developed turbulence
imagines a state space littered with simple invariant solutions (equilibria,
travelling waves, UPOs) whose entangled stable and unstable manifolds create a
scaffold for the turbulent flow trajectory as it bounces between the solutions
Hopf (1948); Eckhardt  _et al._ (2002); Kerswell (2005); Kerswell and Tutty
(2007); Gibson  _et al._ (2008); Cvitanovic and Gibson (2010); Chandler and
Kerswell (2013); Suri  _et al._ (2020); Yalniz  _et al._ (2021); Krygier  _et
al._ (2021). This picture suggests a predictive theory based upon a suitably
weighted sum of the properties of the visited invariant solutions where the
weights reflect the relative time spent in the neighbourhood of the solution
(Cvitanović  _et al._ , 2016). However, finding enough of the important
solutions let alone identifying the appropriate weights has proved
prohibitively expensive. This is especially so at high $Re$ where the set of
invariant solutions proliferates dramatically and each becomes increasingly
unstable, hampering their identification.

A central and well-known issue is the sensitivity of the Newton-Raphson root-
finding algorithm to the quality of the initial guess in high dimensional
problems. Past work has mainly relied on ‘recurrent flow analysis’ to generate
good enough guesses for UPOs where a turbulent orbit is required to shadow a
UPO for at least one full period (Kawahara and Kida, 2001; Viswanath, 2007;
Cvitanovic and Gibson, 2010; Chandler and Kerswell, 2013; Lucas and Kerswell,
2015; Yalniz  _et al._ , 2021). In practice, this imposes a limit on how
unstable a UPO can be which becomes severe as $Re$ increases (Chandler and
Kerswell, 2013). A second issue is the strategy for recognising when the flow
has nearly recurred. Typically this is done simply with an Euclidean norm of
the difference between initial and final states and then the threshold for
near recurrence has to be set quite high. Finally, the third shortcoming is
that many of the dynamically relevant periodic orbits remain undiscovered,
particularly those with a larger than average dissipation rate (Chandler and
Kerswell, 2013; Lucas and Kerswell, 2015). These difficulties have motivated a
number of alternative approaches to guess generation Lan and Cvitanovic
(2004); Lucas and Kerswell (2015); Farazmand (2016); Page and Kerswell (2020);
Page  _et al._ (2021); Parker and Schneider (2022) or even UPO identification
Lucas and Yasuda (2022) but none have been sufficiently transformational to
open up the problem yet.

Even armed with a complete set of UPOs, the more profound theoretical question
is predicting the weights for how each UPO should be counted in a ‘UPO
expansion’ of the turbulent flow. Periodic orbit theory (Artuso  _et al._ ,
1990a, b; Cvitanovic, 1991; Cvitanović  _et al._ , 2016) gives a theoretical
prescription for these weights which has proved effective in low dimensional
dynamical systems coming from ODEs or even some PDEs (e.g. Christiansen  _et
al._ , 1997). However it remains an open question whether this theory says
anything about the Navier-Stokes equations. An early attempt to apply this
theory to the 2D Navier Stokes equations showed no greater skill than a
control experiment of using equal weighting although the set of solutions
available was clearly too small Chandler and Kerswell (2013); Lucas and
Kerswell (2015). Looking beyond a predictive theory for the weights, the
‘lesser’ question of whether this expansion exists at all is still a
fundamental question in fluid mechanics.

The purpose of this paper is to present evidence that the expansion does seem
to exist in the sense that educing the weights from the DNS data of one
statistic allows a variety of other DNS statistics to be usefully predicted
using those same data-derived set of weights. As should be clear from the
above discussion, a crucial ingredient in carrying out such a program is
identifying a sufficiently representative set of UPOs. To do this, we employ a
new method for UPO detection which does not require careful construction of an
initial guess, and which yields large numbers of dynamically relevant UPOs. In
order to do this, we adapt a recently developed fully differentiable flow
solver (Kochkov  _et al._ , 2021), which allows us to find high-quality
guesses for UPOs by performing gradient descent on a loss function involving
entire solution trajectories. This allows explicit searches for periodic
orbits with certain properties (e.g. high dissipation rates) and can
successfully converge large numbers of UPOs starting from arbitrary turbulent
snapshots. The collection of UPOs assembled proves sufficient to robustly
reconstruct statistics of the turbulent attractor thereby demonstrating the
usefulness of an UPO expansion.

##  II Computational setup

###  II.1 Kolmogorov flow

We consider monochromatically forced, two-dimensional turbulence in a doubly-
periodic domain (‘Kolmogorov’ flow), for which the governing equations are

$\displaystyle\partial_{t}\mathbf{u}+\mathbf{u}\cdot\boldsymbol{\nabla}\mathbf{u}$
$\displaystyle=-\boldsymbol{\nabla}p+\frac{1}{Re}\Delta\mathbf{u}+\sin(ny)\hat{\mathbf{x}},$
(1a) $\displaystyle\boldsymbol{\nabla}\cdot\mathbf{u}$ $\displaystyle=0.$ (1b)

where the Reynolds number is defined as
$Re:=\sqrt{\chi}(L_{y}/2\pi)^{3/2}/\nu$, with $\chi$ the forcing amplitude and
$L_{y}$ the height of the computational domain. Throughout we set
$L_{x}=L_{y}$ and use the popular forcing wavelength $n=4$ (e.g. Platt  _et
al._ , 1991; Chandler and Kerswell, 2013; Farazmand, 2016; Lucas and Kerswell,
2015; Parker and Schneider, 2022; Lucas and Yasuda, 2022). We consider two
Reynolds numbers ($Re=40$ and $Re=100$) where self-sustaining turbulence is
observed with that at $Re=100$ clearly in the asymptotic regime Chandler and
Kerswell (2013). At $Re=40$ around $50$ UPOs have been found previously, all
with low average dissipation rates, while only $9$ UPOs have been converged at
$Re=100$ (Chandler and Kerswell, 2013). We also work with the equations
written in terms of the out-of-plane vorticity,
$\omega:=(\boldsymbol{\nabla}\times\mathbf{u})\cdot\hat{\mathbf{z}}$,

$\partial_{t}\omega+\mathbf{u}\cdot\boldsymbol{\nabla}\omega=\frac{1}{Re}\Delta\omega-n\cos(ny)$
(2)

from which the induced velocity can be obtained from the solution of a Poisson
equation, $\Delta\psi=-\omega$, where the streamfunction $\psi$ yields the
induced velocity components via $u=\partial_{y}\psi$, $v=-\partial_{x}\psi$.

The governing equations (1) and (2) are equivariant under continuous shifts in
the horizontal direction,

$\mathscr{T}^{\alpha}:[u,v,\omega](x,y)\to[u,v,\omega](x+\alpha,y)$

and consequently many of the simple invariant solutions are relative
equilibria (travelling waves) or periodic orbits. There are also discrete
shift-reflect and rotational symmetries, but these are not exploited here. We
seek relative unstable periodic orbits (RPOs) with some period $T$ and shift
$\alpha$ which satisfy

$\mathscr{T}^{\alpha}\mathbf{f}^{T}(\mathbf{u})-\mathbf{u}=\mathbf{0},$ (3)

where $\mathbf{f}^{t}$ is the time-forward map of equation (1) or (2).

###  II.2 Automatic differentiation for periodic orbits

Figure 1: (Left) Dissipation against period of all converged orbits with
periods $T<8$ at $Re=40$ (average dissipation rate is shown with a white
square). The four previously known solutions are identified with vertical
dashed lines at $T\in\\{2.83,2.92,5.38,6.72\\}$. (Right) Energy production
versus dissipation for all converged solutions at $Re=40$, including some
longer orbits not shown on the left (all converged UPOs are listed in the
Appendix). The probability density for the turbulent state (computed from a
long calculation run for $t_{l}=2.5\times 10^{5}$) is shown in grey. Contour
levels are spaced logarithmically with a minimum value of $10^{-6}$. All
values are normalised by the laminar value $D_{l}=Re/(2n^{2})$.

We solve equations (1) and (2) using jax-cfd, a fully differentiable flow
solver where gradients of the time-forward map, $\mathbf{f}^{T}(\mathbf{u})$,
with respect to initial conditions can be computed via automatic
differentiation to machine precision (Kochkov  _et al._ , 2021). It is this
capability that forms the basis of the new periodic orbit search strategy. We
make use of both the ‘standard’, finite-difference, primitive variable
formulation (Kochkov  _et al._ , 2021) and the spectral, vorticity version
(Dresdner  _et al._ , 2022) of jax-cfd. The former is used to construct robust
periodic orbit guesses, for reasons discussed below, and the latter for final
convergence in a Newton solver and comparison to previously reported results
from the literature (which were all obtained in spectral codes Chandler and
Kerswell, 2013; Parker and Schneider, 2022).

Periodic orbits are sought by minimising a loss function which is just a
scaled norm of equation (3):

$\mathscr{L}:=\frac{\|\mathscr{T}^{\alpha}\mathbf{f}^{T}(\mathbf{u})-\mathbf{u}\|}{\|\mathbf{u}\|}.$
(4)

The loss depends on the initial condition, $\mathbf{u}$, and an unknown shift,
$\alpha$, and period, $T$. Gradients with respect to all of these variables
can be computed efficiently using the JAX library (Bradbury  _et al._ , 2018).
Typically we deem that guesses for which we can reduce the loss to
$\mathscr{L}\leq 0.015$ are suitable for passing to the Newton solve – direct
convergence with the optimiser is too slow and so it is used as an effective
preconditioner on guesses for Newton.

At $Re=40$ we will also explicitly target certain periods, $T^{*}$, and
attempt to find periodic orbits with average dissipations above some
thresholds $D^{*}$ (such UPOs were largely missing from previous results
Chandler and Kerswell, 2013; Lucas and Kerswell, 2015; Parker and Schneider,
2022). We do this by adding appropriate terms to the loss,

$\displaystyle\mathscr{L}_{T}$
$\displaystyle:=\frac{\|\mathscr{T}^{\alpha}\mathbf{f}^{T}(\mathbf{u})-\mathbf{u}\|}{\|\mathbf{u}\|}+\gamma\,(T-T^{*})^{2},$
(5a) $\displaystyle\mathscr{L}_{D}$
$\displaystyle:=\frac{\|\mathscr{T}^{\alpha}\mathbf{f}^{T}(\mathbf{u})-\mathbf{u}\|}{\|\mathbf{u}\|}+\kappa\,\sigma\left(\frac{D^{*}-\langle
D\rangle_{T}}{\delta}\right),$ (5b)

where the hyperparameters $\gamma=10^{-2}$, $\kappa=100$ and $\delta=10^{-2}$;
$\sigma(\bullet)$ denotes a sigmoid function leading to very harsh
penalisation if the dissipation time average falls below the threshold
$D^{*}$. The penalisation on the target periods is relatively weak, with
$|T-T^{*}|=O(1)$ resulting in a $O(10^{-2})$ contribution to the loss. When
explicitly searching for high dissipation events we relax our threshold on
Newton-worthy guesses to $\mathscr{L}_{D}=0.05$. We use an AdaGrad (Duchi  _et
al._ , 2011) optimiser with initial learning rate $\eta=0.35$ throughout
(learning rate adapts for individual features).

The primitive variable formulation is much more effective at generating good
guesses because the optimiser is free to add a _constant_ background vertical
velocity, $v_{0}$, as it updates the velocity field $\mathbf{u}^{n}$ in the
gradient descent. The constant cross-stream velocity $v_{0}$ fundamentally
changes the system from the basic ‘Kolmogorov’ flow, but it also prevents the
optimiser from getting stuck in shallow local minima – which is a common
feature when searching directly with the spectral, vorticity formulation of
jax-cfd. Similar observations have been made in a recent attempt to find
periodic orbits in a variational formulation, where non-solenoidal velocity
fields were used as initial guesses (Parker and Schneider, 2022). No constant
background flow is possible in the spectral version of jax-cfd because we
solve only for the rotational induced velocity field at each timestep.

For short periods (e.g. $T\lesssim 10$ at $Re=40$), the weak background
vertical velocity only weakly affects the UPO and the spectral Newton solve is
capable of converging the nearby $v_{0}=0$ solution in a few steps. For longer
periods the weak vertical flow has more of an impact, but can often
successfully be efficiently removed with an additional optimisation run
penalising the vertical velocity,

$\mathscr{L}_{V}:=\frac{\|\mathscr{T}^{\alpha}\mathbf{f}^{T}(\mathbf{u})-\mathbf{u}\|}{\|\mathbf{u}\|}+\mu\,v_{0}^{2},$
(6)

where we set $\mu=10^{3}$ and use a smaller learning rate (typically
$\eta=10^{-2}$) to carefully deform the near-closed loop into one with near-
zero vertical velocity.

##  III Detecting unstable periodic orbits

###  III.1 Density of states at $Re=40$

We first explore the more well-studied problem of Kolmogorov flow at $Re=40$.
This configuration has been examined by a number of previous authors (Chandler
and Kerswell, 2013; Lucas and Kerswell, 2015; Farazmand, 2016; Page  _et al._
, 2021; Parker and Schneider, 2022) though we still know very little about the
density of states as a function of their period, $\rho(T)$, and have not
managed to identify any localised high dissipation UPOs. Motivated by both
this and the importance of ‘prime cycles’ in periodic orbit theory (Cvitanović
_et al._ , 2016), we conduct an intensive sweep over short periods
$T\in[2,10]$ using the period-targeting loss function (5a). We increment the
target $T^{*}$ in values of $0.5$ within this range and seed $50$ optimisation
calculations at each $T^{*}$. Each calculation is initialised with a random
snapshot from the turbulent attractor, an initial guess for the period
$T^{0}=T^{*}$ and zero initial shift, $\alpha^{0}=0$. Previously four
solutions were known to exist in this range, with periods
$T\in\\{2.83,2.92,5.38,6.72\\}$.

We also initialise separate searches for high dissipation solutions using loss
function (5b). We perform three computations searching for solutions with
average dissipation rates above threshold values
$D^{*}\in\\{0.12,0.15,0.2\\}$. The computations are initialised in a similar
way to those described previously, though with the starting snapshots
constrained to have dissipation values $D>D^{*}$. We fix the initial period in
this search at $T^{0}=4$ and the shift again at $\alpha^{0}=0$.

Large numbers of solutions are converged within the variable $v_{0}$
formulation, though success is not uniform across target $T^{*}$. For example,
we find many solutions when $T^{*}\in\\{5,5.5,6\\}$, which when introduced
into the $v_{0}=0$ spectral solver all collapse to the well known UPO at
$T=5.38$ (similar behaviour is found close to $T=2.83$ and its integer
multiples). Finding these common solutions is expected, but we do also find
very large numbers of new solutions which were previously unknown. Indeed, for
$T<8$ we converge 38 unique solutions (these are all detailed in the
Appendix), including many high dissipation states that have been inaccessible
to previous search methods.

Figure 2: Spanwise vorticity are extracted at four points equispaced-in-time
over three UPOs at $Re=40$. (Top) Shortest orbit at $Re=40$ – the previously
known $T=2.83$ solution with average dissipation rate $\langle
D/D_{l}\rangle=0.095$. (Middle) A new high dissipation UPO with $T=2.90$ and
average dissipation rate $\langle D/D_{l}\rangle=0.246$. (Bottom) A new high
dissipation UPO with $T=3.27$ and average dissipation rate $\langle
D/D_{l}\rangle=0.285$. In all cases contours run for $-10$ to $10$.

Remarkably, these short-period solutions appear to span nearly the full range
of production and dissipation events in the overall flow (see the right panel
in figure 2). What is missing is actually the low dissipation events – these
are associated with slower dynamics and the UPOs tend to have longer periods.
To find these states, we also searched for solutions with target periods
$T^{*}\in\\{12.5,15,17.5,20\\}$. Generally, the optimiser struggles for these
longer orbits, as $\mathbf{f}^{T}(\mathbf{u})$ is likely far away on the
solution manifold when starting from random initial conditions and gradient
information is not particularly helpful. However, we do obtain several longer
solutions – most are listed in Chandler and Kerswell (2013) – which are all
low dissipation. This aspect of the work would benefit from a more considered
approach to initial condition selection to ensure that
$\mathbf{f}^{T}(\mathbf{u})$ is ‘nearby’ on the solution manifold without
necessarily requiring a near recurrence.

We examine some of the $Re=40$ UPOs in figure 2, where we report snapshots of
spanwise vorticity at four points along the orbit. The low dissipation
solutions (the $T=2.83$ UPO is shown in this figure) all share a common
structure, with vortical structures sitting on top of a pair of slanted
stripes of vorticity (which are reminiscent of the first non-trivial
equilbrium solution in this configuration, see Page  _et al._ , 2021). In the
$T=2.83$ case the period corresponds to a complete rotation of an elliptical
vortex patch located to the right of the panel; the orbit with $T=5.38$ (not
shown) is similar but involves the rotation of a like-signed vortex pair.

In contrast, the high dissipation structures display a much greater variety of
dynamics. For instance, one of the two ‘bursting’ UPOs reported in figure 2
features a crystal like structure with four large-amplitude vortex cores that
are maintained over the period. The other high dissipation solution in figure
2 features a strong dipole structure that makes its way through the domain
from left to right. Unsurprisingly, many of the new solutions have higher-
dimensional unstable manifolds than those that have been previously documented
(see full details in the Appendix).

###  III.2 Periodic orbits at $Re=100$

Figure 3: Energy production rate against dissipation rate at $Re=100$. Grey
background is the PDF computed from a long turbulent computation with
$T=2.5\times 10^{5}$. Contour levels are spaced logarithmically with a minimum
value of $10^{-6}$. Closed loops are the two-dimensional projections of the 43
converged UPOs. All values are normalised by the laminar value
$D_{l}=Re/(2n^{2})$. All periodic orbits along with relevant properties
(period, shift, Floquet exponents) are listed in the Appendix.

We now turn our attention to the strongly turbulent case at $Re=100$, where
only a handful of solutions have been obtained by any previous method
(Chandler and Kerswell, 2013). The 9 solutions obtained previously at this
value of $Re$ were all of short period, $T<5$. The majority of solutions we
report here were computed from five runs using the basic loss function (4)
without additional physics.

We initialise five computations with starting periods
$T^{0}\in\\{1,2,3,4,5\\}$, each with 50 initial guesses, along with an
additional computation of $250$ guesses at $T^{0}=2.5$, the latter computation
motivated by the success rate of the $T^{0}=2$ and $T^{0}=3$ calculations. We
also perform a high dissipation search using loss function (5b) for periodic
orbits with average dissipations $\langle D/D_{l}\rangle\geq 0.03$, though
find this to be much less effective than the equivalent computations at
$Re=40$. In contrast, the standard loss function (4) returns a wide range of
periodic orbits without augmentation, rather than the same subset of
solutions. Overall, we observe a success rate for converging solutions of
between $5-15$%, depending on the choice of starting period. It is important
to emphasise that the starting point in the process is a _random_ snapshot
from a turbulent calculation, and should be contrasted to other methods with a
success rate close to zero where guesses were more carefully constructed (e.g.
the recurrent flow analysis in Chandler and Kerswell, 2013, produced 9
solutions from an analysis of $10^{5}$ time units of data).

Figure 4: Spanwise vorticity are extracted at four points equispaced-in-time
over four UPOs at $Re=100$. From top-to-bottom the UPOs have the following
periods and average dissipation rates: $(T,\langle
D/D_{l}\rangle)=(1.424,0.057)$, $(1.794,0.053)$, $(4.212,0.027)$ and
$(1.164,0.078)$ (for full details of converged solutions see Appendix).
Vorticity contour levels run between $\pm 10$.

Our computations have so far yielded 43 unique UPOs at $Re=100$, and the
states are summarised in figure 3 (and listed in full in the Appendix). Here,
we see that most of the UPOs appear to be highly localised in phase space,
nearly all appearing as small closed loops in the two-dimensional projection.
The states appear even more localised when visualised in terms of their
kinetic energy, $E$ (not shown), and full coverage of the associated turbulent
PDFs will require many more states than the equivalent computations at
$Re=40$.

We visualise some of the converged UPOs in figure 4. The solutions show a wide
variety of different dynamical behaviours. Often, the states are dominated by
two large vortex patches (expected due to the inverse cascade Boffetta and
Ecke, 2012) – see middle two rows of figure 4. There is wide variation within
this type of solution, in addition to the behaviour in figure (e.g. where we
see a strong stationary vortex and a co-rotating pair) there are also states
with three or more like-signed co-rotating vortices – see further details in
the Appendix. The more strongly dissipative states (top and bottom rows of
figure 4) show an even greater variety of vorticity dynamics, and further work
will be required to assess the various ‘classes’ of UPO discovered by our new
approach.

##  IV Statistics from periodic orbits

Figure 5: UPO-based predictions of statistics. Top row shows dissipation rate
(left), production rate (centre) and energy (right) at $Re=40$. The dashed
blue lines are ‘ground truth’ PDFs obtained from a long turbulent calculation
($T=2.5\times 10^{5}$) and the filled grey curves are the UPO reconstructions.
The weights used in the UPO based PDFs are determined via a fit to the
dissipation PDF only using equation (8). Bottom row comparses the mean
velocity profile (left) and the root-mean-square velocity flucutations
(centre, $u$ and $v$ are left and right in the panel) – averaged over the
streamwise direction, discrete symmetries and time. Dashed blue and orange
curves are the DNS ‘ground truth’, black curves the UPO reconstruction;
(right) the weights in the expansion (7) plotted against the (real part of
the) sum of growing Floquet exponents $\sum_{j}\sigma_{j}$, $\sigma_{j}>0$,
for each UPO.

Given the wide range of converged UPOs found, we now proceed to construct a
periodic orbit expansion at $Re=40$. The dissipation probability density
function (PDF) of the DNS, $f(D)$, was chosen to fit the weights of the UPO
expansion

$q(D,\mathbf{w}):=\sum_{j}w_{j}f_{j}(D),$ (7)

where $f_{j}(D)$ is the individual dissipation PDF of UPO $j$ and $w_{j}\geq
0\;\forall j$ with $\sum_{j}w_{j}=1$. Once these weights $w_{j}$ are
determined, the UPO expansion (7) can be reused with UPO PDFs of a different
statistic to make a prediction of the equivalent DNS PDF. How good this
procedure proves to be over statistics of interest then tests how useful the
UPO expansion concept is.

To compute the weights $w_{j}$, we seek to minimise the KL divergence between
the true and reconstructed PDFs for the dissipation, while also maintaining
the required normalisation of the UPO-based PDF. The associated loss function
for the reconstruction is

$\mathscr{L}_{\mathbf{w}}:=-\int
f(D)\log\left(\frac{q(D,\mathbf{w})}{f(D)}\right)dD+\beta\big{|}\sum_{j}|w_{j}|-1\big{|}.$
(8)

We use stochastic gradient descent to find the optimal weights with a
numerical approximation to the integral and the true PDF evaluated at 201
equally-spaced points across the range of the dissipation $D$ (constraining
the weights to be $w_{j}>0$ during optimisation was found not necessary).

To demonstrate the performance of the UPO expansion so constructed, figure 5
reports PDFs of the energy production and kinetic energy computed from a very
long trajectory ($T=2.5\times 10^{5}$) at $Re=40$ overlayed with UPO-based
PDFs computed via expressions like (7) along with the dissipation PDF match
used to define the weights. The reproduction of the dissipation itself is
remarkably complete – although we are missing some of the very lowest values
associated with the longer orbits (as discussed previously). The prediction
for the energy production rate is also pleasingly complete, with the UPOs able
to successfully reconstruct the high-$I$ tail of the distribution. This is
perhaps the simplest test beyond the starting dissipation PDF, because the
time average of these two quantities is the same for a statistically steady
state, $\langle D\rangle=\langle I\rangle$. The more challenging PDF is the
kinetic energy, $E$, and there are some gaps in the UPO reconstruction.
However, the bulk of the distribution is reproduced to a fairly high standard
(in particular compare to past attempts to apply periodic orbit theory
Chandler and Kerswell, 2013). The missing states at $E/E_{l}\sim 0.35$ can be
plausibly linked to the missing lowest dissipation orbits, so resolving issues
with one PDF would also indirectly fix issues with other reconstructions.

The UPO predictions (again with the same weights) for the mean velocity
profile and root-mean-square velocity fluctuations are also tested in figure 5
(averaged over discrete symmetries as well as the streamwise coordinate and
time, see Chandler and Kerswell, 2013; Farazmand, 2016). The UPO-prediction
for the mean profile is indistinguishable from the turbulent mean, while the
fluctuation levels are also remarkably accurate (errors $O(3\%)$ and $O(1\%)$
in $u_{rms}$ and $v_{rms}$ respectively). Again, this represents a dramatic
improvement in UPO-based prediction over the past state-of-the-art.

Finally, we also examine the weights in our reconstruction (7) as a function
of the instability of the underlying UPOs in the final panel of figure 5 (as
measured by the sum of the growth rates from Floquet exponents). The weights
are anti-correlated with the level of instability, where the highly unstable
states are much less important in the reconstruction. This reflects the fact
that the highly dissipative UPOs (which contribute to the very weak right tail
of the distribution) tend to be much more unstable.

##  V Conclusion

In this paper, we have introduced a new method of finding and converging
unstable periodic orbits in turbulent flows. The method relies on automatic
differentiation to perform gradient descent on a trajectory-dependent loss
function. With this approach, UPOs with specific physical properties can be
targeted. Moreover, the approach performs well when initialised with random
initial conditions, and does not suffer from the restrictions of past search
techniques (e.g. the need for a near recurrence). Crucially, the method
performs well at high $Re$ where previous attempts to find UPOs have been
ineffective.

At modest $Re=40$, the search revealed very large numbers of new solutions
with short periods – both high- and low-dissipation – that had been previously
undetectable. There are large numbers of UPOs that are relevant to high energy
“bursting” dynamics, where the community had previously assumed that these
events were contained in longer orbits only, or in the heteroclinic
connections between solutions (Farazmand, 2016). At high $Re=100$, we again
found very large numbers of short solutions, though the distinction between
low/high dissipation events is more ambiguous there. The states at high $Re$
appear to be highly localised in state space, and display a wealth of
interesting vortical dynamics. The library of UPOs assembled at $Re=40$ covers
nearly the full range of production and dissipation events seen by the fully
turbulent state. This allows for a relatively robust reproduction of the
dissipation PDF with a superposition of the PDFs of the individual UPOs, an
approach inspired by periodic orbit theory (Cvitanović  _et al._ , 2016).
Encouragingly, the optimal weights determined via a fit to the dissipation PDF
are then able to reproduce other statistical quantities with UPO statistics –
albeit with obvious gaps. At $Re=40$, the missing UPOs are actually the
longer, low dissipation ones, and our approach could benefit from a more
considered method of initial snapshot selection for the longer orbits. This is
also the case for all orbits at $Re=100$, and guess generation is something
that will need to be refined before the approach can be deployed in three
dimensional computations. In the present configuration, an improved snapshot
selection could involve something as simple as only selecting snapshots which
return to a similar region of state space – not a near recurrence but a weaker
requirement e.g. determined by a neural network as in Page  _et al._ (2021) –
or by searching over discrete symmetries.

Most importantly, a steady improvement is observed in the reproduction of the
statistics at $Re=40$ as we have converged more states, and it is likely that
this trend will continue and should be verified at $Re=100$. All this suggests
that a representation of turbulence with a UPO expansion is a viable modelling
approach, which opens up exciting new vistas for prediction and control.

##  Appendix A Periodic orbit details

Here we report details of the UPOs we have found via automatic differentiation
at both $Re=40$ and $Re=100$ in tables 1 and 2 respectively, including their
leading Floquet exponent and the dimension of the unstable manifold.

In addition, we also report snapshots of spanwise vorticity for five further
states at $Re=100$ in figure 6. These include localised co-rotating three-
vortex states, as well as large quiescent vortex patches.

Table 1: Periodic orbits at $Re=40$. Known solutions as listed in Chandler and Kerswell (2013) are listed in the ‘UPO’ column; $T$ is the period, $\alpha$ the shift. $N$ indicates the number of unstable directions and $\sigma_{r}$ is the growth rate in the leading Floquet exponent. We also report the average dissipation normalised by the laminar value, $\langle D/D_{l}\rangle$. UPO |  $T$ |  $\alpha$ |  $N$ |  $\sigma_{r}$ |  $\langle D/D_{l}\rangle$
---|---|---|---|---|---
P2 |  2.83 |  0.0 |  5 |  0.223 |  0.095
|  2.901 |  0.486 |  13 |  0.356 |  0.246
P3 |  2.917 |  0.0 |  7 |  0.236 |  0.099
|  3.176 |  0.424 |  15 |  0.504 |  0.251
|  3.275 |  0.588 |  17 |  0.462 |  0.285
|  3.396 |  0.926 |  18 |  0.479 |  0.273
|  3.488 |  1.142 |  12 |  0.444 |  0.224
|  3.452 |  0.362 |  11 |  0.247 |  0.117
|  3.749 |  0.0 |  11 |  0.393 |  0.236
|  3.841 |  0.945 |  14 |  0.497 |  0.26
|  4.15 |  0.384 |  15 |  0.581 |  0.246
|  4.359 |  0.288 |  14 |  0.366 |  0.243
|  4.488 |  1.28 |  20 |  0.526 |  0.247
|  4.6 |  0.615 |  10 |  0.341 |  0.194
|  4.616 |  0.545 |  12 |  2.081 |  0.115
|  4.706 |  0.173 |  10 |  0.253 |  0.156
|  4.745 |  0.842 |  13 |  0.378 |  0.223
|  5.062 |  1.324 |  10 |  0.253 |  0.192
|  5.234 |  0.526 |  15 |  0.559 |  0.253
|  5.241 |  1.206 |  11 |  0.366 |  0.223
P1 |  5.38 |  0.0 |  7 |  0.191 |  0.093
|  5.423 |  0.372 |  13 |  0.348 |  0.201
|  5.789 |  0.236 |  12 |  0.394 |  0.199
|  5.992 |  0.099 |  12 |  0.274 |  0.173
|  6.352 |  0.156 |  6 |  0.34 |  0.111
|  6.397 |  0.439 |  11 |  0.19 |  0.141
|  6.523 |  0.0 |  6 |  0.201 |  0.107
|  6.639 |  0.151 |  7 |  0.34 |  0.113
R4 |  6.72 |  0.106 |  8 |  0.343 |  0.111
|  6.876 |  0.188 |  15 |  0.534 |  0.254
|  6.961 |  0.0 |  5 |  1.386 |  0.113
|  7.002 |  0.194 |  6 |  1.376 |  0.108
|  7.156 |  0.032 |  4 |  0.292 |  0.109
|  7.375 |  0.0 |  7 |  0.301 |  0.118
|  7.391 |  0.081 |  8 |  0.219 |  0.118
|  7.516 |  0.193 |  9 |  0.24 |  0.127
|  7.735 |  0.101 |  7 |  0.28 |  0.115
|  8.067 |  0.742 |  7 |  0.243 |  0.116
R19 |  12.207 |  0.243 |  2 |  0.07 |  0.072
|  18.771 |  0.457 |  3 |  0.105 |  0.084
R22 |  19.723 |  0.222 |  4 |  0.172 |  0.083
R24 |  19.779 |  0.248 |  6 |  0.202 |  0.081
R6 |  20.808 |  0.06 |  3 |  0.172 |  0.083
|  22.597 |  0.257 |  4 |  0.209 |  0.081
R34 |  23.157 |  0.265 |  3 |  0.113 |  0.084
Table 2: Periodic orbits at $Re=100$. Known solutions as listed in Chandler and Kerswell (2013) are listed in the ‘UPO’ column; $T$ is the period, $\alpha$ the shift. $N$ indicates the number of unstable directions and $\sigma_{r}$ is the growth rate in the leading Floquet exponent. We also report the average dissipation normalised by the laminar value, $\langle D/D_{l}\rangle$. UPO |  $T$ |  $\alpha$ |  $N$ |  $\sigma_{r}$ |  $\langle D/D_{l}\rangle$
---|---|---|---|---|---
|  1.164 |  0.768 |  14 |  1.061 |  0.078
|  1.287 |  0.163 |  19 |  1.037 |  0.049
|  1.345 |  0.007 |  7 |  1.113 |  0.028
|  1.356 |  0.126 |  18 |  0.896 |  0.038
|  1.424 |  0.815 |  12 |  0.683 |  0.057
|  1.711 |  0.207 |  9 |  0.268 |  0.03
|  1.723 |  0.204 |  15 |  0.525 |  0.031
|  1.729 |  0.206 |  14 |  0.246 |  0.031
|  1.746 |  0.027 |  20 |  0.468 |  0.042
|  1.794 |  0.012 |  16 |  1.582 |  0.053
R16 |  1.938 |  0.121 |  6 |  0.271 |  0.023
|  1.939 |  0.0 |  30 |  1.724 |  0.08
|  1.941 |  0.425 |  13 |  0.369 |  0.031
|  1.946 |  0.0 |  16 |  1.615 |  0.055
|  1.966 |  0.435 |  20 |  1.536 |  0.068
R15 |  1.984 |  0.122 |  17 |  0.556 |  0.024
|  2.098 |  0.118 |  9 |  0.303 |  0.022
|  2.123 |  0.319 |  11 |  0.349 |  0.029
|  2.136 |  0.025 |  11 |  0.423 |  0.027
|  2.163 |  0.382 |  18 |  0.365 |  0.027
|  2.196 |  0.166 |  13 |  0.439 |  0.027
|  2.2 |  0.012 |  13 |  0.454 |  0.025
|  2.29 |  0.013 |  13 |  0.394 |  0.023
|  2.301 |  0.037 |  10 |  0.43 |  0.023
|  2.441 |  0.002 |  10 |  0.346 |  0.021
|  2.59 |  0.067 |  13 |  0.539 |  0.02
|  2.592 |  0.069 |  13 |  0.482 |  0.024
|  2.877 |  0.056 |  25 |  1.229 |  0.04
|  2.742 |  0.492 |  11 |  0.444 |  0.022
|  3.231 |  0.12 |  11 |  0.663 |  0.025
|  3.34 |  0.048 |  10 |  0.167 |  0.017
|  3.498 |  0.077 |  11 |  0.307 |  0.033
|  3.51 |  0.068 |  12 |  0.216 |  0.028
R17 |  3.827 |  0.008 |  16 |  0.818 |  0.027
|  3.84 |  0.078 |  13 |  0.306 |  0.023
|  3.869 |  0.148 |  12 |  0.229 |  0.026
|  4.212 |  0.765 |  13 |  0.335 |  0.027
|  4.361 |  0.199 |  12 |  0.599 |  0.021
|  4.464 |  0.079 |  12 |  0.407 |  0.025
|  4.497 |  0.109 |  15 |  0.368 |  0.025
R14 |  4.526 |  0.071 |  9 |  0.105 |  0.014
|  4.635 |  0.386 |  15 |  0.259 |  0.024
|  4.654 |  0.073 |  9 |  0.098 |  0.014

Figure 6: Spanwise vorticity are extracted at four points equispaced-in-time
over five UPOs at $Re=100$. From top-to-bottom the UPOs have the following
periods and average dissipation rates: $(T,\langle
D/D_{l}\rangle)=(1.356,0.038)$, $(1.723,0.021)$, $(1.966,0.068)$,
$(2.196,0.027)$ and $(2.590,0.02)$ (for full details of converged solutions
see table 2). Vorticity contour levels run between $\pm 10$.

## References

  * Kawahara  _et al._ (2012) G. Kawahara, M. Uhlmann, and L. van Veen, “The significance of simple invariant solutions in turbulent flows,” Annual Review of Fluid Mechanics 44, 203–225 (2012).
  * Graham and Floryan (2021) Michael D. Graham and Daniel Floryan, “Exact coherent states and the nonlinear dynamics of wall-bounded turbulent flows,” Annual Review of Fluid Mechanics 53, 227–253 (2021).
  * Kerswell (2005) R. R. Kerswell, “ Recent progress in understanding the transition to turbulence in a pipe,” Nonlinearity 18, R17–R44 (2005).
  * Eckhardt  _et al._ (2007) B. Eckhardt, T. M. Schneider, B. Hof, and J. Westerweel, “ Turbulence transition in pipe flow ,” Annual Review of Fluid Mechanics 39, 447–468 (2007).
  * Kawahara and Kida (2001) G. Kawahara and S. Kida, “Periodic motion embedded in plane Couette turbulence: regeneration cycle and burst,” Journal of Fluid Mechanics 449, 291–300 (2001).
  * Cvitanovic and Gibson (2010) P. Cvitanovic and J. F. Gibson, “ Geometry of the turbulence in wall-bounded shear flows: periodic orbits ,” Physica Scripta T142, 014007 (2010).
  * Hamilton  _et al._ (1995) J. M. Hamilton, J. Kim, and F. Waleffe, “Regeneration mechanisms of near-wall turbulence structures,” Journal of Fluid Mechanics 287, 317–348 (1995).
  * Waleffe (1997) F. Waleffe, “ On a self-sustaining process in shear flows,” Physics of Fluids 9, 883–900 (1997).
  * Hall and Sherwin (2010) P. Hall and S. Sherwin, “Streamwise vortices in shear flows: harbingers of transition and the skeleton of coherent structures,” Journal of Fluid Mechanics 661, 178–205 (2010).
  * Hopf (1948) E. Hopf, “A mathematical example displaying features of turbulence,” Commun. Pure Appl. Math. 1, 303–322 (1948).
  * Eckhardt  _et al._ (2002) B. Eckhardt, H. Faisst, A. Schmiegel, and J. Schumacher, “Turbulence transition in shear flows,” Advances in Turbulence IX: Proceedings 9th European Turbulence Conference (Southampton) (ed. I.P. Castro et al.) CISME , 701 (2002).
  * Kerswell and Tutty (2007) R. R. Kerswell and O. R. Tutty, “Recurrence of travelling waves in transitional pipe flow,” J. Fluid Mech. 584, 69–102 (2007).
  * Gibson  _et al._ (2008) J. F. Gibson, J. Halcrow, and P. Cvitanovic, “Visualizing the geometry of state space in plane Couette flow,” Journal of Fluid Mechanics 611, 107–130 (2008).
  * Chandler and Kerswell (2013) G. J. Chandler and R. R. Kerswell, “Invariant recurrent solutions embedded in a turbulent two-dimensional Kolmogorov flow,” Journal of Fluid Mechanics 722, 554–595 (2013).
  * Suri  _et al._ (2020) B. Suri, L. Kageorge, R. O. Grigoriev, and M. F. Schatz, “Capturing turbulent dynamics and statistics in experiments with unstable periodic orbits,” Phys. Rev. Lett. 125, 064501 (2020).
  * Yalniz  _et al._ (2021) G. Yalniz, B. Hof, and N. B. Budanur, “Coarse graining the state space of a turbulent flow using periodic orbits,” Phys. Rev. Lett. 126, 244502 (2021).
  * Krygier  _et al._ (2021) M. C. Krygier, J. L Pughe-Sanford, and R. O. Grigoriev, “Exact coherent structures and shadowing in turbulent Taylor-Couette flow,” J. Fluid Mech. 923 (2021).
  * Cvitanović  _et al._ (2016) P. Cvitanović, R. Artuso, R. Mainieri, G. Tanner, and G. Vattay,  _Chaos: Classical and Quantum_ (Niels Bohr Inst., Copenhagen, 2016).
  * Viswanath (2007) D. Viswanath, “Recurrent motions within plane Couette turbulence,” Journal of Fluid Mechanics 580, 339–358 (2007).
  * Lucas and Kerswell (2015) D. Lucas and R. R. Kerswell, “Recurrent flow analysis in spatiotemporally chaotic 2-dimensional Kolmogorov flow,” Physics of Fluids 27, 045106 (2015).
  * Lan and Cvitanovic (2004) Y. H. Lan and P. Cvitanovic, “Variational method for finding periodic orbits in a general flow,” Phys. Rev. E 69, 016217 (2004).
  * Farazmand (2016) M. Farazmand, “An adjoint-based approach for finding invariant solutions of Navier–Stokes equations,” Journal of Fluid Mechanics 795, 278–312 (2016).
  * Page and Kerswell (2020) Jacob Page and Rich R. Kerswell, “Searching turbulence for periodic orbits with dynamic mode decomposition,” Journal of Fluid Mechanics 886, A28 (2020).
  * Page  _et al._ (2021) J. Page, M. P. Brenner, and R. R. Kerswell, “Revealing the state space of turbulence using machine learning,” Physical Review Fluids 6, 034402 (2021).
  * Parker and Schneider (2022) J.P. Parker and T.M. Schneider, “Variational methods for finding periodic orbits in the incompressible navier–stokes equations,” Journal of Fluid Mechanics 941, A17 (2022).
  * Lucas and Yasuda (2022) D. Lucas and T. Yasuda, “Stabilization of exact coherent structures in two-dimensional turbulence using time-delayed feedback,” Phys. Rev. Fluids 7, 014401 (2022).
  * Artuso  _et al._ (1990a) R. Artuso, E. Aurell, and P. Cvitanovic, “Recycling of strange sets: I cycle expansions,” Nonlinearity 3, 325–359 (1990a).
  * Artuso  _et al._ (1990b) R. Artuso, E. Aurell, and P. Cvitanovic, “Recycling of strange sets: II applications,” Nonlinearity 3, 361–386 (1990b).
  * Cvitanovic (1991) P. Cvitanovic, “Periodic orbits as the skeleton of classical and quantum chaos,” Physica D 51, 138–151 (1991).
  * Christiansen  _et al._ (1997) F. Christiansen, P. Cvitanovic, and V. Putkaradze, “Spatiotemporal chaos in terms of unstable recurrent patterns,” Nonlinearity 10, 55–70 (1997).
  * Kochkov  _et al._ (2021) Dmitrii Kochkov, Jamie A. Smith, Ayya Alieva, Qing Wang, Michael P. Brenner, and Stephan Hoyer, “Machine learning–accelerated computational fluid dynamics,” Proceedings of the National Academy of Sciences 118 (2021).
  * Platt  _et al._ (1991) N. Platt, L. Sirovich, and N. Fitzmaurice, “An investigation of chaotic Kolmogorov flows,” Physics of Fluids 3, 681–696 (1991).
  * Dresdner  _et al._ (2022) Gideon Dresdner, Dmitrii Kochkov, Peter Norgaard, Leonardo Zepeda-Núñez, Jamie A. Smith, Michael P. Brenner, and Stephan Hoyer, “Learning to correct spectral methods for simulating turbulent flows,”  (2022).
  * Bradbury  _et al._ (2018) James Bradbury, Roy Frostig, Peter Hawkins, Matthew James Johnson, Chris Leary, Dougal Maclaurin, George Necula, Adam Paszke, Jake VanderPlas, Skye Wanderman-Milne, and Qiao Zhang, “JAX: composable transformations of Python+NumPy programs,”  (2018).
  * Duchi  _et al._ (2011) John Duchi, Elad Hazan, and Yoram Singer, “Adaptive subgradient methods for online learning and stochastic optimization,” Journal of Machine Learning Research 12, 2121–2159 (2011).
  * Boffetta and Ecke (2012) Guido Boffetta and Robert E. Ecke, “Two-dimensional turbulence,” Annual Review of Fluid Mechanics 44, 427–451 (2012).


# Two-carrier Magnetoresistance: Applications to Ca3Ru2O7

Lakshmi Das1 Yang Xu1 Tian Shang1 Alexander Steppke1 Masafumi Horio1 Jaewon
Choi1 Simon Jöhr1 Karin von Arx1 Jasmin Mueller1 Dominik Biscette1 Xiaofu
Zhang1 Andreas Schilling1 Veronica Granata3 Rosalba Fittipaldi2,3 Antonio
Vecchione2,3 and Johan Chang1 1 Physik-Institut1 Physik-Institut Universität
Zürich Universität Zürich Winterthurerstrasse 190 Winterthurerstrasse 190
CH-8057 Zürich CH-8057 Zürich Switzerland
2 CNR-SPIN Switzerland
2 CNR-SPIN I-84084 Fisciano I-84084 Fisciano Salerno Salerno Italy
3 Dipartimento di Fisica ”E.R. Caianiello” Italy
3 Dipartimento di Fisica ”E.R. Caianiello” Università di Salerno Università di
Salerno I-84084 Fisciano I-84084 Fisciano Salerno Salerno Italy Italy

###### Abstract

Ambipolar transport is a commonly occurring theme in semimetals and
semiconductors. Here we present an analytical formulation of the conductivity
for a two-band system. Electron and hole carrier densities and their
respective conductivities are mapped into a two-dimensional unit-less phase
space. Provided that one of the carrier densities is known, the dimensionless
phase space can be probed through magnetoresistance measurements. This
formulation of the two-band model for conductivity is applied to
magnetoresistance experiments on Ca3Ru2O7. While previous such measurements
focused on the low-temperature limit, we cover a broad temperature range and
find negative magnetoresistance in an intermediate interval below the
electronic transition at 48 K. The low-temperature magnetoresistance in
Ca3Ru2O7 is consistent with a two-band structure. However, the model fails to
describe the full temperature and magnetic field dependence. Negative
magnetoresistance found in an intermediate temperature range is, for example,
not captured by this model. We thus conclude that the electronic and magnetic
structure in this intermediate temperature range render the system beyond the
most simple two-band model.

##  1 Introduction

The combination of electron and hole charge carriers is a common theme of
condensed matter physics [(1)]. It is also essential for many applications
such as semiconductor junctions, excitonic solar cells and light emitting
diodes. Transport properties of semimetals are defined by electron and hole
charge carriers. In its simplest form, a semimetal consists of just one
electron- and one hole-like band crossing the Fermi level. Exposed to a
magnetic field, the electrons and holes experience a Lorentz force which
results in a negative or positive Hall effect depending on whether
conductivity is dominated by electrons or holes. Compensated ambipolar
transport properties stem from the detailed balance between electron and hole
mobilities [(2)]. Most semimetals/metals have more than two bands crossing the
Fermi level. Nevertheless, a two-band model can still be of some relevance and
is widely used to discuss transport experiments [(3), (4), (5), (6), (7), (8),
(9), (10), (11)]. In some multiband systems, only two Fermi surface sheets
host high-mobility carriers – making the two-band model approximately viable.
Orbital selectivity found in systems with strong Hunds coupling [(12)] may
also trigger selective Fermi surface gapping while other sheets are untouched
– thus providing another route to two-band physics. Finally, electronic
reconstruction of single band systems is a common motif that can generate two-
band structures [(13), (14), (15)]. The wide applicability justifies a
detailed analysis of the two-band model.

Here we show that the electron and hole transport properties can be described
by two dimensionless parameters $\alpha$ and $\beta$. This ($\alpha$,$\beta$)
phase space connects smoothly the ambipolar transport with the mono-carrier
dominated regime. For constant filling, the system must move along
($\alpha$,$\beta$) specific contour lines. If the carrier density of – say the
electrons – is known from angle resolved photoemission spectroscopy (ARPES)
[(16)] or quantum oscillation [(17)] experiments, the temperature dependent
parameters $\alpha$ and $\beta$ can be determined from a magnetotransport
experiment. We apply this new formulation of the two-band model to
magnetoresistance experiments of the semimetallic phase of Ca3Ru2O7. While
previous studies focused entirely on the low-temperature limit [(18)], we
cover a broad temperature range and find negative magnetoresistance for 30 K
$<T<48$ K. For selected temperatures ($T\approx 10$ K), the observed low-field
magnetoresistance is captured well by the two-carrier model. However, as a
function of temperature (warming) the ($\alpha$,$\beta$) parametrisation is
inconsistent with a constant filling. For the case of Ca3Ru2O7, the two-band
model fails to explain the full temperature and magnetic field dependence of
the magnetoresistance. The most pronounced limitation of the two-band model is
its inability to explain the negative magnetoresistance found for 30 K $<T<48$
K. We discuss these discrepancies in terms of a two-stage Fermi surface
reconstruction. Below 48 K, the system is a semimetal but with a Fermi surface
including more than two sheets. A second reconstruction across 30 K simplifies
the Fermi surface such that it hosts only a single electron- and hole-like
pocket. Within this structure reasonable consistency with the two band model
is reached at low temperatures.

Figure 1: (Color online) Magnetotransport properties of Ca3Ru2O7. (a,b) Zero-
field resistivity and thermopower versus temperature. The vertical dashed
lines indicate three temperature scales: $T^{*}=30$ K, $T_{s}=48$ K and
$T_{N}=56$ K. These temperature scales represent respectively the onset of the
low-temperature metallic transport properties and the AFM-a and AFM-b phases.
The $T_{s}$ temperature scale is also associated with a Fermi surface
reconstruction that changes the sign of the thermopower from positive to
negative [(24)]. (c) Hall resistivity $\rho_{xy}$ versus magnetic field for
temperatures as indicated. The inset shows the temperature evolution of the
Hall coefficient, $R_{H}$. (d) Magnetoresistance plotted as
$[\rho_{xx}(H)-\rho_{xx}(0)]/\rho_{xx}(0)$ versus magnetic field squared for
temperatures as indicated. Solid grey lines are linear fits.

##  2 Methods

High quality single crystals of Ca3Ru2O7 were grown by the floating zone
techniques [(19), (20)]. The orthorhombic (Bb21m) crystals were detwinned, by
pressing along an orthorhombic axis, using a thermo-mechanical detwinning
device [(21)] in combination with a polarized light microscope. The
longitudinal resistivity (along the $b$-axis), Hall, magnetization and
thermoelectric measurements were carried out in a Quantum Design Magnetic
Property Measurement system and a Physical Property Measurement System with
magnetic field $H$ applied along the crystallographic $c$-axis. For the
thermoelectric experiments, temperature gradients along the orthorhombic
$b$-axis (the longer lattice parameter) were recorded using Cernox chips
[(22)].

##  3 Results

In Figs. 1 a,b, we are plotting the zero magnetic field resistivity $\rho$
(along b \- axis) and thermopower $S$ versus temperature. The temperature
dependence of both curves are in agreement with all previous experiments
[(23), (24), (25)]. Resistivity displays a weak drop across the AFM-a Néel
ordering temperature $T_{N}=56$ K and a sharp upturn below the spin-flip
transition $T_{s}=48$ K into the AFM-b phase. The latter transition is
associated with a Fermi surface reconstruction, signaled by a sharp sign
change of the Seebeck coefficient [(24)] (Fig. 1b). Below 30 K, the
resistivity recovers a metallic Fermi-liquid-like ($\rho\propto T^{2}$)
temperature dependence (Fig. 1a). Hall resistivity isotherms ($\rho_{xy}$
versus magnetic field $H$) – shown in Fig. 1c for temperatures as indicated –
are also consistent with previous reports. Negative Hall resistivity is found
for $T<T_{s}$ [(26)].

Magnetoresistance plotted as $[\rho_{xx}(H)-\rho_{xx}(0)]/\rho_{xx}(0)$ versus
$H^{2}$ is displayed in Fig. 1d for temperatures as indicated. The low-
temperature positive magnetoresistance is consistent with previous studies
[(27), (24)], all of which showed,
$[\rho_{xx}(H)-\rho_{xx}(0)]/\rho_{xx}(0)=(\mu_{MR}H)^{2}$ with mobility
$\mu_{MR}\approx 0.1$ T-1. Here we focus on the temperature dependent
magnetoresistance in Ca3Ru2O7. For $T>T_{s}$, a small positive
magnetoresistance is found whereas for 30 K$<T<T_{s}$ it is negative before
turning to large positive values in the $T\rightarrow 0$ limit. The derivative
$d\rho_{xx}/dH$ is shown in Fig. 2a as a function of temperature and magnetic
field. The onset of negative magnetoresistance and history dependent magnetic
susceptibility (Fig. 2b) clearly correlate. Both effects appear below $T_{s}$.

Figure 2:  (Color online) Magnetic field derivative of resistivity
d${\rho}{{}_{xx}}$/dH and magnetic susceptibility. (a) Magnetic field
derivative of resistivity, d${\rho}{{}_{xx}}$/dH plotted in false color scale,
as a function of magnetic field and temperature. Red (blue) color indicates
positive (negative) magnetoresistance. (b) Susceptibility recorded with an
applied field of 0.5 T versus temperature in field-cooled (FC - dotted black)
and zero-field-cooled (ZFC - blue curve) mode. The bifurcation below 48 K
demonstrates the existence of a ferromagnetic coupling [(49)]. (c) and (d)
shows the low-temperature magnetization, resistivity and Hall isotherms. (c)
Hall and magnetization isotherms plotted as a function of magnetic field $H$
for temperatures as indicated. (d) Resistivity isotherm versus $H^{2}$. The
dashed red line represents the two-carrier model including higher order terms
in $H^{2}$.

The low-temperature magnetization and Hall resistivity isotherms display
different field dependence (Fig. 2c). The magnetization $M$ is linear in
magnetic field whereas $\rho_{xy}$ is displaying non-linear field dependence
suggesting that these two quantities are not directly coupled. The linear
trend of the low temperature magnetization data (Fig. LABEL:fig:70028Fig2c)
excludes the possibility of an anomalous Hall effect. We also point out that
although $\rho_{xx}\propto H^{2}$ in the low-field ($H\rightarrow 0$) regime,
significant deviations are observed in larger magnetic field (see Fig. 2d and
Reference.[(27)]). In what follows, we discuss the low-temperature large
positive and intermediate-temperature negative magnetoresistance behaviour
within the two-band model.

##  4 Modelling

For metals with two Fermi surface sheets of different band curvature, the
magnetoresistance is given by [(28), (3), (4), (5), (6)]:

${\rho}{{}_{xx}}(H)=\frac{({\sigma}{{}_{h}}+{\sigma}{{}_{e}})+{\sigma}{{}_{h}}{\sigma}{{}_{e}}({\sigma}{{}_{h}}R{{}_{h}}^{2}+{\sigma}{{}_{e}}\lvert{R{{}_{e}}}\rvert^{2})H^{2}}{({\sigma}{{}_{h}}+{\sigma}{{}_{e}})^{2}+{\sigma}{{}_{h}}^{2}{\sigma}{{}_{e}}^{2}(R{{}_{h}}-\lvert
R{{}_{e}}\rvert)^{2}H^{2}}~{}~{},$ (1)

$\frac{{\rho}{{}_{xy}}(H)}{H}=\frac{{\sigma}{{}_{h}}^{2}R{{}_{h}}-{\sigma}{{}_{e}}^{2}\lvert
R{{}_{e}}\rvert-{\sigma}{{}_{h}}^{2}{\sigma}{{}_{e}}^{2}R{{}_{h}}\lvert
R{{}_{e}}\rvert(R{{}_{h}}-\lvert
R{{}_{e}}\rvert)H^{2}}{({\sigma}{{}_{h}}+{\sigma}{{}_{e}})^{2}+{\sigma}{{}_{h}}^{2}{\sigma}{{}_{e}}^{2}(R{{}_{h}}-\lvert
R{{}_{e}}\rvert)^{2}H^{2}}~{}~{}.$ (2)

The model consists of four parameters $\sigma_{e},\sigma_{h}$, $R_{e}$ and
$R_{h}$. The hole and electron conductivities are denoted by
($\sigma_{h},\sigma_{e}$) and the hole and electron Hall coefficients are
indicated by ($R_{h},R_{e}$). Given that $\sigma_{i}=n_{i}e\mu_{i}$ and
$R_{i}=\pm 1/(n_{i}e)$ where $i={e,h}$, the model can also be expressed in
terms of electron and hole carrier density ($n_{e}$, $n_{h}$) and mobility
($\mu_{e}$, $\mu_{h}$). This model generally assumes that the mobilities are
independent of the applied magnetic field. Analysis of $\rho_{xx}$ and
$\rho_{xy}$ isotherms in terms of the four parameters
($\sigma_{e},\sigma_{h}$,$R_{e}$,$R_{h}$) is typically associated with
ambiguity [(6)]. Although $\sigma_{e}$ and $\sigma_{h}$ are coupled
parameters, since in zero-field,
$\rho_{xx}=(\sigma_{e}+\sigma_{h})^{-1}=\sigma^{-1}$, the problem has three
free parameters.

To make progress, we assume that the electron carrier density $n_{e}$ is
known, for example, from ARPES [(16)] or quantum oscillation [(17)]
experiments. The problem then reduces to two equations with two unknowns
($\sigma_{h}$ and $n_{h}$), lifting the ambiguity. In the low-field
$H\rightarrow 0$ limit, the problem can be expressed in terms of two
dimensionless parameters $\alpha$ and $\beta$:

$\alpha=\frac{R_{H}}{\lvert{R_{e}}\rvert}=\frac{\rho_{xy}}{H\lvert{R_{e}}\rvert}=\frac{{\sigma}{{}_{h}}^{2}R{{}_{h}}/\lvert{R{{}_{e}}}\rvert-{\sigma}{{}_{e}}^{2}}{\sigma^{2}}~{}~{},$
(3)

and

$\beta=\frac{3\mathcal{C}}{\sigma\lvert{R_{e}}\rvert^{2}}=3{\sigma}{{}_{h}}{\sigma}{{}_{e}}\frac{\left[{\sigma}{{}_{h}}R{{}_{h}}/\lvert{R{{}_{e}}}\rvert+{\sigma}{{}_{e}}\right]^{2}}{\sigma^{4}}~{}~{}.$
(4)

with $\mathcal{C}$ defined by $\rho_{xx}=\rho_{0}+\mathcal{C}H^{2}$ where
$\rho_{0}$ is the residual resistivity. Solving with respect to $\sigma_{h}$
and $R_{h}$ yields $\sigma_{h}/\sigma=\mathcal{F}(\alpha,\beta)$,
$R_{h}/R{{}_{e}}$=
$\mathcal{G}(\alpha,\beta)=[\alpha+(1-\mathcal{F})^{2}]/\mathcal{F}^{2}$ and
$\mu_{h}/\mu_{e}=\mathcal{G}\mathcal{F}/(1-\mathcal{F})$ where the analytical
function $\mathcal{F}$ is expressed in the appendix. These three ratios
($n_{e}/n_{h}$, $\sigma_{h}/\sigma$ and $\mu_{h}/\mu_{e}$) are plotted in Fig.
3 within the ”phase space” of the two dimensionless parameters $\alpha$ and
$\beta$.

These parameters are measurable (modulus $R_{e}$) and typically temperature
dependent. Perhaps $\alpha=R_{H}/\lvert{R_{e}}\rvert$ is the most intuitive
parameter. Within an electron-hole two-band model, it varies in the interval
$[-1,\infty]$. Ambipolar transport, defined by comparable electron and hole
contributions to the conductivity, typically leads to $|\alpha|\sim 0$. By
contrast, for $\alpha\approx-1$ conductivity is dominated by electrons. The
second dimensionless parameter $\beta$ is given by the ratio between the
magnetoresistance coefficient $\mathcal{C}$ and the conductivity modulus
$R_{e}^{2}$.

We notice that the two-carrier model was recently expressed in a similar
scheme with two dimensionless parameters [(6)]. In the work of Eguchi and
Paschen (Ref. 6), magnetoresistance is expressed in terms of dimensionless
carrier and mobility ratios. Here, we use the inverse approach where the
carrier and mobility ratios are expressed in terms of dimensionless parameters
$\alpha$ and $\beta$ obtained from magnetoresistance. This formulation has
some advantages. With our notation, solutions have the following properties.
First, for $\mathcal{F}$ to take real values, $\beta>\alpha^{2}$ and hence
$C/(\sigma R_{H}^{2})>{1/3}$ must be satisfied. As $C$, $\sigma$ and $R_{H}$
are all measurable quantities, this provides a test as to whether a two-band
model is applicable. We notice that this is independent of our initial input
for $R_{e}$. Second, constant filling implies that
$n_{e}+n_{h}=n_{e}(1+\mathcal{G}^{-1})$ is invariant. Therefore, if $n_{e}$
remains constant, the system must stay on a $\mathcal{G}$-contour. These
contour lines ”flow” from the electron to the hole dominated limit via the
ambipolar BelPRL2003  regime (see Fig. 3a).

Figure 3: (Color online) Two-band model solutions. (a-c) Carrier density ratio
$\mathcal{G}=n_{e}/n_{h}$, conductivity ratio $\mathcal{F}=\sigma_{h}/\sigma$,
and mobility ratio $\mu_{h}/\mu_{e}$ versus the dimensionless parameters
$\alpha$ and $\sqrt{\beta}$. Dashed lines are contours of the respective
quantities. Solutions are found for $\beta>\alpha^{2}$ (lower bound is red
line). The $n_{e}/n_{h}=1$ contour line, given by $\beta=3(1-\alpha^{2})/4$
indicates the border between hole and electron ”doped” regimes. The parameter
space around $\beta=\alpha=1$ represents the situation $n_{e}>n_{h}$ and
$\mu_{h}\gg\mu_{e}$ whereas $\beta=-\alpha=1$ describes $\mu_{h}>\mu_{e}$ and
$n_{e}\gg n_{h}$. Hole rich ($n_{h}\gg n_{e}$) solutions with either large
hole or electron mobility is found in the $\beta,\alpha\rightarrow 0$ limit.
Red and blue circles in (a-c) represents the phase space position of Ca3Ru2O7
(CRO327) at $T=10$ K and YBa2Cu4O8 (Y124) at $T=20$ K RourkePRB2010
respectively.

For example, increasingly negative Hall coefficient implies
$\alpha\rightarrow-1$ and hence $\beta$ should decrease for the system to stay
on a constant $n_{e}/n_{h}$ contour. Since both $\sqrt{\beta}$ and $\alpha$
scale with $1/R_{e}$, the shape of the contour lines are independent of
$R_{e}$. A third solution property stems from the $n_{e}/n_{h}=1$ contour line
EomPRB2012 ; PisoniPRB2016 ; RULLIERALBENQUE2016164  given by
$\beta=3(1-\alpha^{2})/4$. If a system is known to be hole-doped, i.e.
$n_{h}>n_{e}$ ($\mathcal{G}<1$), it must satisfy $|\alpha|<\sqrt{3/7}$ and
$\beta<3(1-\alpha^{2})/4$. Deviation from any of the three properties implies:
(a) non-constant filling, (b) the system is not having a two-band structure or
(c) the two-band model assumptions are too simplistic.

##  5 Discussion

Before analysing the magnetoresistance of Ca3Ru2O7, we start by illustrating
the use of the aforementioned solution properties. In doing so, we consider
results obtained on hole and electron doped cuprates. For hole doped
YBa2Cu3O7-x and YBa2Cu4O8, quantum oscillation experiments yields an electron-
like Fermi surface sheet with an area corresponding to $R_{e}=-29~{}$mm3C-1
YellandPRL2007 ; Doiron-Leyraud-Nat2007 . Since the system is hole-doped, we
expect $n_{h}>n_{e}$. Hall effect experiments suggest that $\alpha\approx-1$
LebeoufNature2007 . Already here a contradiction emerges as
$|\alpha|>\sqrt{3/7}$ is impossible for a two-carrier hole-doped system.
Evaluation of $\beta$ positions the system in the $n_{e}\gg n_{h}$ region
where $\sigma_{h}\rightarrow 0$ RourkePRB2010 . In fact $\beta>\alpha^{2}$ is
not strictly satisfied. A plausible explanation for these contradictory
results is that the Fermi surface structure contains more than two sheets.
Since the Fermi surface is likely reconstructed by an incommensurate charge-
density-wave order Wu11 ; GhiringhelliSCI12 ; chang12 , a more complex multi-
band structure is expected MillisPRB2007 . On the electron doped side
($n_{e}>n_{h}$), by contrast, the Fermi surface is expected to fold around the
antiferromagnetic zone boundary leading to a two-band structure HePNAS2019
(see Fig. 4a).

Figure 4: (Color online) Fermi surfaces of two-carrier systems. Blue and red
sheets represent respectively hole and electron-like band curvature. (a) The
Fermi surface that appears in electron doped cuprates after
folding/reconstruction around the anti-ferromagnetic zone boundary. (b) Low-
temperature ($T<30$ K) Fermi surface of Ca3Ru2O7, revealed by ARPES
experiments Horio2019 . (c) Fermi surface structure of Ca3Ru2O7 in the
($30<T<48$ K) temperature interval. ARPES experiments indicates a more complex
structure around the electron pocket.

Quantum oscillation Hall experiments on Pr1.86Ce0.14CuO4 (PCCO) Breznay  and
Nd1.85Ce0.15CuO4 (NCCO) HelmPRL2009 ; HelmPRB2015  reveal a small hole pocket
with $n_{h}\approx 4.0\times 10^{25}/m^{3}$ giving $R_{H}\ll R_{h}$. To
compare to our model results, we use momentarily $n_{e}\leftrightarrow n_{h}$
and $\sigma_{e}\leftrightarrow\sigma_{h}$. We therefore have $|\alpha|\ll 1$
and evaluation of the magnetoresistance for both NCCO and PCCO Breznay ;
GollnikPRB1998  yields $\beta\approx 0.03$ less than $3(1-\alpha^{2})/4$ as
expected. These systems are thus not incompatible with a two-band model.

Next, we discuss our magnetoresistance experiments on Ca3Ru2O7. Recent ARPES
data suggest that Ca3Ru2O7, at base temperature, is a semimetal with a single
electron and two identical hole pockets Horio2019 ; markovi2020 ;
PuggioniPRR2020  (see Figure. 4b). The electron pocket - corresponds to a
carrier density $n_{e}=7.8\times 10^{24}/m^{3}$, consistent with the dominant
quantum oscillation frequency KikugawaJPSJ2010 . The electron Hall coefficient
$R_{e}=-1/(en_{e})$ is therefore known within the confidence provided by ARPES
and quantum oscillation experiments. Our low magnetic-field magnetoresistance
data permit extraction of $R_{H}$, $\sigma$ and $\mathcal{C}$ as a function of
temperature. At $T=10$ K, we find $\alpha=-0.21$ and $\beta=0.85$ and with
that, all model parameters are now known:
$\sigma_{h}=0.37\sigma=0.6\sigma_{e}$, and $\lvert R_{h}/R_{e}\rvert=1.26$
(see Fig. 3). The electron-hole carrier ratio is consistent with the low-
temperature ARPES derived Fermi surface Horio2019 . Our results also imply
that the electron carriers are more mobile $\mu_{e}=1.33\mu_{h}$ than the
holes. This is consistent with the fact that the largest quantum oscillation
amplitude KikugawaJPSJ2010  is found for the frequency corresponding to the
electron pocket in the ARPES experiment Horio2019 ; markovi2020 . So far, the
two-band model seems to sensibly describe the magnetoresistance experiment.
However, there are at least three problematic issues with the two-band model.
(i) Even in the temperature range 2 K – 30 K, $\alpha$ and $\beta$ vary in a
fashion that is inconsistent with a constant $n_{e}/n_{h}$. (ii) For
intermediate temperatures negative magnetoresistance is observed. (iii) We
used the low-field limit $H\rightarrow 0$ for our analysis. On top of this,
higher order magnetic field terms can be derived. However, these higher-order
terms are not large enough to account for the deviation from $\rho_{xx}\propto
H^{2}$ under higher fields (Fig. 2d).

In what follows, we discuss possible reasons for these discrepancies. We start
by stressing that the underlying assumption that carrier mobility is magnetic-
field independent is not the sole culprit. Field dependent mobilities could
explain (iii) but not (i) and (ii). We also point out that carrier density
ratio $n_{e}/n_{h}$ may, in fact, be temperature dependent. Recent density-
functional-theory (DFT) calculations suggest a strong coupling between
detailed lattice structure and a temperature dependent Coulomb interaction
PuggioniPRR2020 . Early neutron diffraction experiments indeed demonstrated
the temperature dependence of ruthenium-oxygen bond angles
yoshida_crystal_2005 . It is therefore not inconceivable that the chemical
potential and even the low-energy electronic structure is temperature
dependent. In fact, recent ARPES studies reported a two stage reconstruction
with characteristic temperatures of $T^{*}=30$ K and $T_{s}=48$ K Horio2019 ;
markovi2020 . Since the negative magnetoresistance is found for
$T^{*}<T<T_{s}$, it is very likely linked to the intermediately reconstructed
electronic structure (see Figure. 4c). ARPES experiments suggest that these
Fermi surface consists for more than two sheets Horio2019 . Furthermore, out-
of-plane resistivity indicate that a finite inter-layer interaction is still
present yoshida_quasi-two-dimensional_2004 . As such the electronic structure
contains complexity beyond the most simple two-band model. In addition, our
magnetization data (see Figure. 2b) suggests the existence of weak
ferromagnetism (on top of the antiferromagnetic coupling) below $T_{s}$. It is
not uncommon that magnetic correlations generate a negative magnetoresistance
PhysRevB.66.024433 ; yamada1972negative  that is comparable or larger than the
orbital component. It is also not unusual that the ferromagnetic contribution
is largest just below the ordering temperature Destraznpj2020 . The negative
magnetoresistance in Ca3Ru2O7 therefore is likely linked both to the
electronic structure and ferromagnetic properties.

##  6 Conclusions

In summary, we have carried out a magnetoresistance study of Ca3Ru2O7. As a
function of temperature, three different regimes are identified: weak positive
magnetoresistance for $T>48$ K, negative magnetoresistance for 30 K $<T<48$ K
and large positive magnetoresistance below $30$ K. These characteristic
temperatures are directly linked to reconstructions of the low-energy
electronic structure. We analysed the low-temperature magnetoresistance within
a two-band model. An analytical solution to the two-band model for
conductivity is developed and expressed in terms of two dimensionless (but
measurable) parameters. Whereas reasonable values of electron/hole carrier
density and mobility is found for selected temperatures ($T\approx 10$ K), the
two band model is not capturing the full magnetic field and temperature
dependence. The most pronounced limitation of the model is its inability to
explain negative magnetoresistance. On top of a two-stage electronic
reconstruction, we argue that the Fermi surface structure – and with that the
electron/hole carrier density – is temperature dependent. This in combination
with ferromagnetism generates the complicated magnetoresistance in Ca3Ru2O7.

###### Acknowledgements.

We thank Benoit Fauque for fruitful discussions. L.D., Y. X, M.H., K.v.A,
T.S., and J.C. acknowledge support by the Swiss National Science Foundation.
L. D. was partially funded through Excellence Scholarship by the Swiss
Government.

## Appendix A

In the main text, the dimensionless parameters $\alpha$ and $\beta$ are
expressed in terms of ${\sigma}{{}_{h}}/{\sigma}=~{}\mathcal{F}(\alpha,\beta)$
and $R{{}_{h}}/R{{}_{e}}=\mathcal{G}(\alpha,\beta)$ – Equations (3) and (4).
Solving with respect to $\mathcal{F}(\alpha,\beta)$ and
$\mathcal{G}(\alpha,\beta)$ yields:

$\mathcal{F}(\alpha,\beta)=A_{1}+\displaystyle{\frac{A_{2}}{(A_{3}+A_{4})^{1/3}}}+\frac{2^{2/3}}{6}\left(A_{3}+A_{4}\right)^{1/3},$
(5)

and

$\mathcal{G}(\alpha,\beta)=\frac{{\alpha}+[1-\mathcal{F}(\alpha,\beta)]^{2}}{[\mathcal{F}(\alpha,\beta)]^{2}}~{}~{}.$
(6)

with

$\displaystyle A_{1}$ $\displaystyle=1+\frac{2}{3}{\alpha},$ (7)
$\displaystyle A_{2}$
$\displaystyle=\frac{\sqrt[3]{2}}{3}\left({\alpha^{2}}-\beta\right),$
$\displaystyle A_{3}$ $\displaystyle=-[2{\alpha^{3}}+3\beta(2\alpha+3)],$
$\displaystyle A_{4}$
$\displaystyle=\sqrt{3}\beta\sqrt{\frac{12\alpha^{3}(1+\alpha)}{\beta}+(27+36\alpha+8\alpha^{2})+\frac{4\beta}{3}}~{}~{}.$

It is now possible to express the longitudinal and transverse resistance in
terms of $\mathcal{F}$ and $\mathcal{G}$:

$\rho_{xx}=\frac{1}{\sigma}\frac{1+\mathcal{F}(1+\frac{\mathcal{F}\mathcal{G}^{2}}{1-\mathcal{F}})(\mu_{e}H)^{2}}{1+(\mathcal{F}(1-\mathcal{G})\mu_{e}H)^{2}},$
(8)

$\rho_{xy}=\mathcal{F}\mathcal{G}\frac{1-\left(\frac{1-\mathcal{F}}{\sqrt{\mathcal{G}F}}\right)^{2}+(1-\mathcal{G})(\mu_{e}H)^{2}}{1+(\mathcal{F}(1-\mathcal{G})\mu_{e}H)^{2}}R_{e}H.$
(9)

For a semimetal with $\mathcal{G}=1$ ($n_{e}=n_{h}$), Equations 8 and 9
simplify to $\Delta\rho(H)/\rho(0)=\mu_{e}\mu_{h}B^{2}$ and
$\rho_{xy}=(1-\mu_{e}/\mu_{h})R_{e}H$. Therefore, no saturation of
magnetoresistance or Hall effect is expected. For $\mathcal{G}\neq 1$,
however, the magnetoresistance should eventually saturate and non-linear
$\rho_{xy}$ isotherms should occur in the high-field limit. As shown in Fig.
2d, we indeed observe saturation of the $\Delta\rho(H)/\rho(0)\propto H^{2}$
and $\rho_{xy}$ as magnetic field strength is increased. In the same field
range, the magnetization scales perfectly with $H$. Hence ferromagnetism as a
source for the saturating magnetoresistance can thus be excluded. As we find
$\mathcal{G}$ close to unity, this saturation effect is stronger than what can
be explained by the two-carrier model. It is possible to fit the entire field
range, however the resulting parameters
($\sigma_{h}$,$\sigma_{e}$,$n_{h}$,$n_{e}$) are difficult to reconcile with
the quantum oscillation and ARPES experiments. We are therefore concluding
that the electron mobility is magnetic field dependent for $H>4$ T.

## References

  * (1) B. Fauqué B, X. Yang, W. Tabis, M. Shen, Z. Zhu, C. Proust, Y. Fuseya and K. Behnia, Phys. Rev. Materials 2, 114201 (2018).
  * (2) R. Bel, K. Behnia and H. Berger, Phys. Rev. Lett. 91 066602 (2003).
  * (3) Li P, Balakirev F F and Greene R L 2007 Phys. Rev. Lett. 99(4) 047003
  * (4) Rourke P M C, Bangura A F, Proust C, Levallois J, Doiron-Leyraud N, LeBoeuf D, Taillefer L, Adachi S, Sutherland M L and Hussey N E 2010 Phys. Rev. B 82(2) 020514
  * (5) Rana K G, Dejene F K, Kumar N, Rajamathi C R, Sklarek K, Felser C and Parkin S S P 2018 Nano Lett. 18 6591–6596 pMID: 30241438
  * (6) Eguchi G and Paschen S 2019 Phys. Rev. B 99(16) 165128
  * (7) Watts S M, Wirth S, Von Molnar S, Barry A and Coey J 2000 Phys. Rev. B 61 9621
  * (8) Tokumoto T, Jobiliong E, Choi E, Oshima Y and Brooks J 2004 Solid State Commun.  129 599–604
  * (9) Peramaiyan G, Sankar R, Muthuselvam I P and Lee W L 2018 Sci. Rep.  8 1–8
  * (10) Wang A, Graf D, Liu Y, Du Q, Zheng J, Lei H and Petrovic C 2017 Phys. Rev. B 96(12) 121107
  * (11) Fournier P, Jiang X, Jiang W, Mao S N, Venkatesan T, Lobb C J and Greene R L 1997 Phys. Rev. B 56(21) 14149–14156
  * (12) Georges A, de’ Medici L and Mravlje J 2013 Annu. Rev. Condens. Matter Phys 4 137–178
  * (13) He J, Rotundu C R, Scheurer M S, He Y, Hashimoto M, Xu K J, Wang Y, Huang E W, Jia T, Chen S, Moritz B, Lu D, Lee Y S, Devereaux T P and Shen Z X 2019 Proc. Natl. Acad. Sci. U.S.A. 116 3449–3453 ISSN 0027-8424 (Preprint
  * (14) Sachdev S, Metlitski M A and Punk M 2012 J. Phys.: Condens. Matter 24 294205
  * (15) Chang J, Sassa Y, Guerrero S, Månsson M, Shi M, Pailhés S, Bendounan A, Mottl R, Claesson T, Tjernberg O, Patthey L, Ido M, Oda M, Momono N, Mudry C and Mesot J 2008 New J. Phys.  10 103016
  * (16) Damascelli A, Hussain Z and Shen Z X 2003 Rev. Mod. Phys. 75(2) 473–541
  * (17) Sebastian S E and Proust C 2015 Annu. Rev. Condens. Matter Phys. 6 411–430 (Preprint
  * (18) Kikugawa N, Winfried Rost A, William Hicks C, John Schofield A and Peter Mackenzie A 2010 J. Phys. Soc. Jpn. 79 024704
  * (19) Fukazawa H, Nakatsuji S and Maeno Y 2000 Physica B 281 613–614
  * (20) Nakatsuji S and Maeno Y 2001 J. Solid State Chem. 156 26 – 31
  * (21) E Burkhardt Z G Y and Schmid H 1995 Rev. Sci. Instrum. 66(7) 3888
  * (22) Destraz D, Das L, Tsirkin S S, Xu Y, Neupert T, Chang J, Schilling A, Grushin A G, Kohlbrecher J, Keller L, Puphal P, Pomjakushina E and White J S 2020 npj Quantum Mater. 5 5
  * (23) Yoshida Y, Nagai I, Ikeda S I, Shirakawa N, Kosaka M and Môri N 2004 Phys. Rev. B 69 220411 ISSN 1098-0121, 1550-235X
  * (24) Xing H, Wen L, Shen C, He J, Cai X, Peng J, Wang S, Tian M, Xu Z A, Ku W, Mao Z and Liu Y 2018 Phys. Rev. B 97(4) 041113
  * (25) Iwata K, Kosaka M, Katano S, Môri N, Yoshida Y and Shirakawa N 2007 J. Magn. Magn. Mater.  310 1125 – 1127 ISSN 0304-8853 proceedings of the 17th International Conference on Magnetism
  * (26) Yoshida Y, Ikeda S I and Shirakawa N 2007 J. Phys. Soc. Jpn.  76 085002
  * (27) Kikugawa N, Winfried Rost A, William Hicks C, John Schofield A and Peter Mackenzie A 2010 J. Phys. Soc. Jpn. 79 024704
  * (28) Ashcroft N and Mermin N 1976 Solid State Physics (Fort Worth: Saunders College Publishing)
  * (29) Eom M J, Na S W, Hoch C, Kremer R K and Kim J S 2012 Phys. Rev. B 85(2) 024536
  * (30) Pisoni A, Szirmai P, Katrych S, Náfrádi B, Gaál R, Karpinski J and Forró L 2016 Phys. Rev. B 93(9) 094519
  * (31) Rullier-Albenque F 2016 C R Phys. 17 164 – 187 ISSN 1631-0705 iron-based superconductors / Supraconducteurs à base de fer
  * (32) Yelland E A, Singleton J, Mielke C H, Harrison N, Balakirev F F, Dabrowski B and Cooper J R 2008 Phys. Rev. Lett. 100(4) 047003
  * (33) Doiron-Leyraud N, Proust C, LeBoeuf D, Levallois J, Bonnemaison J B, Liang R, Bonn D A, Hardy W N and Taillefer L 2007 Nature 447 565–568
  * (34) LeBoeuf D, Doiron-Leyraud N, Levallois J, Daou R, Bonnemaison J B, Hussey N E, Balicas L, Ramshaw B J, Liang R, Bonn D A, Hardy W N, Adachi S, Proust C and Taillefer L 2007 Nature 450 533–536
  * (35) Wu T, Mayaffre H, Krämer S, Horvatic M, Berthier C, Hardy W N, Liang R, Bonn D A and Julien M H 2011 Nature 477 191
  * (36) Ghiringhelli G, Tacon M L, Minola M, Blanco-Canosa S, Mazzoli C, Brookes N B, Luca G M D, Frano A, Hawthorn D G, He F, Loew T, Sala M M, Peets D C, Salluzzo M, Schierle E, Sutarto R, Sawatzky G A, Weschke E, Keimer B and Braicovich L 2012 Science 337 821–825
  * (37) Chang J, Blackburn E, Holmes A T, Christensen N B, Larsen J, Mesot J, Liang R, Bonn D A, Hardy W N, Watenphul A, v Zimmermann M, Forgan E M and Hayden S M 2012 Nat. Phys. 8 871–876
  * (38) Millis A J and Norman M R 2007 Phys. Rev. B 76(22) 220503
  * (39) Breznay N P, Hayes I M, Ramshaw B J, McDonald R D, Krockenberger Y, Ikeda A, Irie H, Yamamoto H and Analytis J G 2016 Phys. Rev. B 94(10) 104514
  * (40) Helm T, Kartsovnik M V, Bartkowiak M, Bittner N, Lambacher M, Erb A, Wosnitza J and Gross R 2009 Phys. Rev. Lett. 103(15) 157002
  * (41) Helm T, Kartsovnik M V, Proust C, Vignolle B, Putzke C, Kampert E, Sheikin I, Choi E S, Brooks J S, Bittner N, Biberacher W, Erb A, Wosnitza J and Gross R 2015 Phys. Rev. B 92(9) 094501
  * (42) Gollnik F and Naito M 1998 Phys. Rev. B 58(17) 11734–11752
  * (43) Horio M, Wang Q, Granata V, Kramer K P, Sassa Y, Jöhr S, Sutter D, Bold A, Das L, Xu Y, Frison R, Fittipaldi R, Kim T K, Cacho C, Rault J E, Fèvre P L, Bertran F, Plumb N C, Shi M, Vecchione A, Fischer M H and Chang J 2019 Electron-driven ${C}_{2}$-symmetric Dirac semimetal uncovered in Ca3Ru2O7
  * (44) Marković I, Watson M D, Clark O J, Mazzola F, Morales E A, Hooley C A, Rosner H, Polley C M, Balasubramanian T, Mukherjee S, Kikugawa N, Sokolov D A, Mackenzie A P and King P D C 2020 Proc. Natl. Acad. Sci. U.S.A.  117 15524–15529
  * (45) Puggioni D, Horio M, Chang J and Rondinelli J.M 2020 Phys. Rev. Research 2(2) 023141
  * (46) Yoshida Y, Ikeda S.I, Matsuhata H, Shirakawa N, Lee C. H and Katano S 2005 Phys. Rev. B 72 054412 ISSN 1098-0121, 1550-235X
  * (47) Raquet B, Viret M, Sondergard E, Cespedes O and Mamy R 2002 Phys. Rev. B 66(2) 024433
  * (48) Yamada H and Takada S 1972 Prog. Theor. Phys.  48 1828–1848
  * (49) Sokolov D, Kikugawa N, Helm T, Borrmann H, Burkhardt U, Cubitt R, White J, Ressouche E, Bleuel M, Kummer K, Mackenzie A. P , and Rößler U. K. 2019 Nat. Phys.  15 671–677


# Direct Cardiac Segmentation from Undersampled K-space Using Transformers

###### Abstract

The prevailing deep learning-based methods of predicting cardiac segmentation
involve reconstructed magnetic resonance (MR) images. The heavy dependency of
segmentation approaches on image quality significantly limits the acceleration
rate in fast MR reconstruction. Moreover, the practice of treating
reconstruction and segmentation as separate sequential processes leads to
artifact generation and information loss in the intermediate stage. These
issues pose a great risk to achieving high-quality outcomes. To leverage the
redundant k-space information overlooked in this dual-step pipeline, we
introduce a novel approach to directly deriving segmentations from sparse
k-space samples using a transformer (DiSK). DiSK operates by globally
extracting latent features from 2D+time k-space data with attention blocks and
subsequently predicting the segmentation label of query points. We evaluate
our model under various acceleration factors (ranging from 4 to 64) and
compare against two image-based segmentation baselines. Our model consistently
outperforms the baselines in Dice and Hausdorff distances across foreground
classes for all presented sampling rates.

Index Terms—  cardiac magnetic resonance imaging, segmentation, transformer,
k-space

##  1 Introduction

Cardiac magnetic resonance (CMR) segmentation is vital for deriving key
clinical metrics like ejection fraction and strain. The current deep learning-
based workflow for the cardiac segmentation process is comprised of three key
steps: data acquisition, reconstruction, and segmentation. In clinical
practice, data acquisition involves sampling signals in frequency domain
(k-space) to mitigate lengthy acquisition times caused by physiological and
physical constraints. Reconstruction methods derive to restore the original MR
images by exploiting the prior knowledge in either k-space [1, 2] or image
domain [3, 4]. Research in cardiac segmentation primarily focuses on
leveraging contextual inter-slice shape information [5], spatial details from
adjacent slice [6], or temporal data [7].

However, treating reconstruction and segmentation as separate serial tasks
presents two major limitations. Firstly, errors and artifacts introduced
during the earlier reconstruction phase can adversely impact the subsequent
segmentation step, particularly prevalent at higher sampling rates. Secondly,
the distinct objectives of reconstruction and segmentation may lead to a
suboptimal solution for the final task. Conventionally, extensive care is
taken to maximize the quality of images at pixel level during reconstruction,
whereas segmentation tasks predominantly rely on high-level anatomical
information presented in CMR images. Thus, viewing these two steps as a
unified process holds great potential for enhancing segmentation performance
at higher sampling rates.

Only a few works have endeavored to blur the boundary between reconstruction
and segmentation. Schlemper et al. [8] have trained segmentation models to be
robust to reconstruction artifacts directly from zero-filled reconstructed
images. Other networks [9, 10] have fused reconstruction and segmentation
processes through end-to-end training settings. Acar et al. [11] have employed
a segmentation model as a proxy task to emphasize anatomical structure during
the reconstruction training.

However, initiating the reconstruction from zero-filled k-space data
introduces artifacts when employing the inverse fast Fourier transform (FFT).
Furthermore, having an explicit image reconstruction step requires more
information (in the form of k-space samples) than is required to directly
infer a segmentation map. Notably, Rempe et al. [12] have achieved skull
segmentation directly from k-space data, bypassing an intermediate
reconstruction by predicting masks in frequency domain. Nonetheless, they only
use fully-sampled k-space to carry out the segmentation, which might deviate
from standard clinical practices.

Fig. 1: Overview of DiSK architecture. $N$, $M$, and $P$ represent the number
of sampled points in k-space, latent vectors, and query points in image
domain, respectively. The encoder is made up of $L$ Perceiver layers [13]
which alternates cross-attention and latent self-attention blocks.
Specifically, cross-attention blocks project global features of the set of
input k-space samples $K$ into a fixed-dimensional latent bottleneck of $M$
vectors. Self-attention between latent vectors contextualizes the extracted
global features between the vectors. The decoder is made up of $L$ cross-
attention layers which condition queried image-domain coordinates with the
encoder’s latent vectors into segmentation class probabilities.

In this study, we propose DiSK, a novel approach to predict segmentation maps
directly from undersampled (not zero-filled) k-space measurements using
attention-based architecture. We demonstrate that operating from the source
frequency data, bypassing the reconstruction step, is both sufficient and
highly efficient in accurately recovering cardiac anatomy, even at high
acceleration factors (up to 64). Our contributions can be summarized as
follows:

  * •

We achieve 2D+time CMR image segmentation directly from undersampled k-space
measurements without an intermediate image reconstruction step or an explicit
inverse Fourier transform.

  * •

We utilize a transformer architecture to extract k-space features globally and
subsequently predict segmentation maps in image space.

  * •

Our model shows superior Dice and Hausdorff distance scores compared to our
baselines under acceleration factors ranging from 4 to 64.

##  2 Method

In this work, we utilize a transformer-based architecture that employs global
attention to leverage all available k-space measurements into the output
segmentation.

Due to the nature of the Fourier Transform (FT), local information in the
image domain is correlated to the whole k-space domain. However, the
convolutional kernels found in convolutional neural networks have an implicit
bias towards local features. This puts a limit on their ability to integrate
global information, causing them to inefficiently encode latent features from
the k-space domain.

On the other hand, we believe that transformers offer the necessary properties
to appropriately process the nature of k-space: (1) the mechanism considers
global correlations, (2) inputs of arbitrary sparsity caused by undersampling
patterns are supported, and (3) feature extraction is invariant to the
presented ordering of data samples. These properties offer the opportunity to
process global relationships between k-space measurements, with no requirement
to zero-fill or interpolate the input grid.

An overview of the architecture is presented in Figure 1. DiSK uses an
encoder-decoder design to extract latent features from k-space and
subsequently decodes coordinates into segmentation maps.

We define the encoder as $E_{\bm{\theta}}(\left[V_{k}^{u},X_{k}^{u}\right])$,
where $V_{k}^{u}\in\mathbb{C}^{N\times 1}$ is the set of $N$ sampled k-space
values, $X_{k}^{u}\in\mathbb{R}^{N\times 3}$ are the coordinates of each given
k-space value, and $[\cdot,\cdot]$ represents element-wise concatenation. We
use the $u$ superscript to denote the undersampled nature of the input set
$\left|V_{k}^{u}\right|\leq\left|V_{k}\right|$. The encoder uses the Perceiver
architecture [13] to iteratively aggregate global k-space information. The
alternating design between cross-attention (CA) and self-attention (SA) blocks
circumvents the $\mathcal{O}(N^{2})$ memory complexity of naive SA between
input tokens of standard transformers. Given that the number of k-space
samples in our data ranges from tens to hundreds of thousands of points, this
architecture allows us to efficiently scale up the number of sample points
that can be processed. The CA blocks project global k-space information into a
fixed set of latent vectors $H\in\mathbb{R}^{M\times d}$, while the self-
attention (SA) blocks contextualize features between latent vectors.

The decoder $D_{\bm{\phi}}(\bm{h},\bm{x})$ is comprised of a sequence of CA
blocks that allow for any queried image-domain coordinate
$\bm{x}\in\mathbb{R}^{3}$ to be conditioned by the latent vectors into
decoding a probabilistic segmentation vector
$\hat{\bm{s}}_{\bm{x}}\in\mathbb{R}^{C}$ of $C$ classes. The segmentation
output is supervised using Dice and binary cross-entropy (BCE) losses. The
overall model $G$ can be formulated as:

$G_{\bm{\Theta}}(\left[X_{k}^{u},V_{k}^{u}\right],\bm{x})=D_{\bm{\phi}}(E_{\bm{\theta}}(\left[X_{k}^{u},V_{k}^{u}\right]),\bm{x})=\hat{\bm{s}}_{\bm{x}}\>,$
(1)

where $\hat{\bm{s}}_{\bm{x}}$ is the predicted segmentation class
probabilities.

We apply the positional encoding outlined in [14] individually to each
coordinate and value element, which increases the network’s sensitivity to
small changes in the input.

##  3 Experiments and result

Dataset. Our dataset is comprised of 1200 mid-ventricular slices of cardiac
short-axis scans from the UK-Biobank study [15]. The dataset is arranged into
1000/100/100 splits for training, validation, and testing sets. Each scan has
50 frames with an average image size of approximately 80x80 pixels per frame.
As only the magnitude images were available, we create synthetic k-space data
for each 2D+time scan by applying additional Gaussian B0 variations on the fly
in order to remove the conjugate symmetry of k-space. Similarly, Cartesian
undersampling masks are generated in real-time independently for each frame by
sampling normally distributed lines centered on the DC component.

Implementation details. We set the number of layers $L$ in the transformer to
4 and token sizes to 128 for both the encoder and decoder. Fully-connected
layers at each attention block had 128 nodes. We use a total of 128 latent
vectors (also of size 128), where initialization prior to the first layer is
optimizable with respect to the overall loss. The segmentation head is a
linear layer followed by a softmax non-linearity for 4 classes. We conducted
all experiments on a single NVIDIA RTX A4000 GPU using the Pytorch library.
Code for training our model is made publicly
available111https://github.com/Yundi-Zhang/DiSK.git.

Baselines. To the best of our knowledge, no previous work has attempted to
produce cardiac segmentations directly from k-space samples. We thus implement
two baselines from [8] that have shown promising cardiac segmentation
performance on images reconstructed from high acceleration factors via zero-
filling: Syn-Net and LI-Net.

Results. We train and evaluate all models under acceleration factors (4, 8,
16, 32, 64). Performances are quantified by taking average Dice over the three
foreground classes, as well as the maximum Hausdorff distance over the
foreground classes. Results are reported in Table 1, where the scores are
averaged over three foreground classes to enhance better anatomical accuracy.
Figure 2 shows a qualitative example of all models predicting segmentations
from sparse k-space measurements. The corresponding undersampled k-space
measurements and zero-filled reconstructed images are shown on the side. Our
model consistently outperforms the baselines by approximately 0.1 in Dice
scores and demonstrates a reduction of roughly 2 pixels in Hausdorff distances
across foreground classes for all presented sampling rates.

Table 1: Dice scores and Hausdorff distances over a testing set of 100
subjects.

Acc. |  | Syn-Net | LI-Net | Ours
---|---|---|---|---
Dice $\uparrow$ | $0.749_{\pm 0.260}$ | $0.805_{\pm 0.198}$ | $\bm{0.902_{\pm 0.089}}$
4$\times$ | HD $\downarrow$ | $6.809_{\pm 2.776}$ | $6.557_{\pm 3.160}$ | $\bm{4.797_{\pm 2.064}}$
| Dice $\uparrow$ | $0.748_{\pm 0.258}$ | $0.809_{\pm 0.192}$ | $\bm{0.902_{\pm 0.089}}$
8$\times$ | HD $\downarrow$ | $6.794_{\pm 2.868}$ | $7.019_{\pm 3.521}$ | $\bm{4.772_{\pm 2.253}}$
| Dice $\uparrow$ | $0.742_{\pm 0.264}$ | $0.800_{\pm 0.197}$ | $\bm{0.903_{\pm 0.085}}$
16$\times$ | HD $\downarrow$ | $6.792_{\pm 2.818}$ | $6.841_{\pm 2.971}$ | $\bm{4.509_{\pm 2.068}}$
| Dice $\uparrow$ | $0.723_{\pm 0.287}$ | $0.752_{\pm 0.242}$ | $\bm{0.902_{\pm 0.086}}$
32$\times$ | HD $\downarrow$ | $7.383_{\pm 3.122}$ | $7.531_{\pm 3.131}$ | $\bm{4.665_{\pm 2.054}}$
| Dice $\uparrow$ | $0.733_{\pm 0.261}$ | $0.799_{\pm 0.190}$ | $\bm{0.902_{\pm 0.085}}$
64$\times$ | HD $\downarrow$ | $7.543_{\pm 2.972}$ | $6.706_{\pm 2.567}$ | $\bm{4.911_{\pm 2.356}}$

Acc.GTSyn-NetLI-NetOursZero-
filling$4\times$$8\times$$16\times$$32\times$$64\times$

Fig. 2: Test set segmentations predicted from different models over varying
acceleration factors. The last column visualizes the undersampled k-space
measurements in a time frame.

Discussion. In summary, we propose a network aimed at deriving segmentation
from undersampled k-space data, omitting the image reconstruction stage. We
demonstrate the superior performance of DiSK compared to our baselines on
2D+time cardiac dataset.

Interestingly, both our approach and the baselines show little degradation
under the higher acceleration rates. We believe the models are prone to
memorizing the general segmentation labels, by either leveraging the image
borders in the case of the image-based baselines, or by correlating
coordinates with segmentation labels like in the case of our model. In some
sense, the models can be seen as memorizing an implicit prior over the typical
anatomy in the dataset and learning how to deform its default prediction with
subject-specific information.

In future work, we aim to expand upon the current work to conduct full-view
short-axis cardiac MRI by processing larger numbers of k-space samples. We
also hope to investigate how the reliance on shape memorization is influenced
by the introduction of rotations and translations of the input. Moreover, we
would like to explore other downstream tasks such as cardiac function
assessment and disease classification.

##  4 Conclusion

To the best of our knowledge, this is the first study that explores the direct
prediction of segmentation maps directly from sparse under-sampled k-space
measurements. Compared to conventional segmentation models, the proposed model
shows a novel data processing pipeline for medical downstream tasks, where
global feature extraction is learnt directly from k-space data without the
potential introduction of artifacts or the loss of information due to
intermediate image reconstruction steps.

##  5 Acknowledgements

This research study was conducted retrospectively using human subject data
made available in open access by the UK Biobank Resource under Application
Number 87802. Ethical approval was not required as confirmed by the license
attached with the open access data.

This work is funded in part by the European Research Council (ERC) project
Deep4MI (884622) and the Munich Center for Machine Learning.

## References

  * [1] M. A. Griswold, P. M. Jakob, R. M. Heidemann, M. Nittka, V. Jellus, J. Wang, B. Kiefer, and A. Haase,  “Generalized autocalibrating partially parallel acquisitions (GRAPPA),”  ISMRM, vol. 47, no. 6, pp. 1202–1210, 2002.
  * [2] J. Pan, S. Shit, Ö. Turgut, W. Huang, H. B. Li, N. Stolt-Ansó, T. Küstner, K. Hammernik, and D. Rueckert,  “Global k-Space Interpolation for Dynamic MRI Reconstruction Using Masked Image Modeling,”  pp. 228–238, 2023.
  * [3] R. Otazo, E. Candes, and D. K. Sodickson,  “Low-rank plus sparse matrix decomposition for accelerated dynamic MRI with separation of background and dynamic components,”  ISMRM, vol. 73, no. 3, pp. 1125–1136, 2015.
  * [4] K. Hammernik, T. Klatzer, E. Kobler, M. P. Recht, D. K. Sodickson, T. Pock, and F. Knoll,  “Learning a variational network for reconstruction of accelerated MRI data,”  ISMRM, vol. 79, no. 6, pp. 3055–3071, 2018.
  * [5] C. Zotti, Z. Luo, A. Lalande, and P.-Marc Jodoin,  “Convolutional neural network with shape prior applied to cardiac MRI segmentation,”  IEEE J-BHI, vol. 23, no. 3, pp. 1119–1128, 2018.
  * [6] Q. Zheng, H. Delingette, N. Duchateau, and N. Ayache,  “3-D consistent and robust segmentation of cardiac images by deep learning with spatial propagation,”  IEEE transactions on medical imaging, vol. 37, no. 9, pp. 2137–2148, 2018.
  * [7] C. Qin, W. Bai, J. Schlemper, S. E. Petersen, S. K. Piechnik, S. Neubauer, and D. Rueckert,  “Joint learning of motion estimation and segmentation for cardiac MR image sequences,”  in MICCAI. Springer, 2018, pp. 472–480.
  * [8] J. Schlemper, O. Oktay, W. Bai, D. C. Castro, J. Duan, C. Qin, J. V. Hajnal, and D. Rueckert,  “Cardiac MR segmentation from undersampled k-space using deep latent representation learning,”  pp. 259–267, 2018.
  * [9] Q. Huang, D. Yang, J. Yi, L. Axel, and D. Metaxas,  “FR-Net: Joint reconstruction and segmentation in compressed sensing cardiac MRI,”  pp. 352–360, 2019.
  * [10] A. A Tolpadi, U. Bharadwaj, K. T. Gao, R. Bhattacharjee, F. G. Gassert, J. Luitjens, P. Giesler, J. N. Morshuis, P. Fischer, M. Hein, et al.,  “K2S Challenge: From Undersampled K-Space to Automatic Segmentation,”  Bioengineering, vol. 10, no. 2, pp. 267, 2023.
  * [11] M. Acar, T. Çukur, and İ. Öksüz,  “Segmentation-aware MRI reconstruction,”  pp. 53–61, 2022.
  * [12] M. Rempe, F. Mentzel, K. L. Pomykala, J. Haubold, F. Nensa, K. Kröninger, J. Egger, and J. Kleesiek,  “k-strip: A novel segmentation algorithm in k-space for the application of skull stripping,”  Computer Methods and Programs in Biomedicine, p. 107912, 2023.
  * [13] A. Jaegle, F. Gimeno, A. Brock, O. Vinyals, A. Zisserman, and J. Carreira,  “Perceiver: General Perception with Iterative Attention,”  pp. 4651–4664, 18–24 Jul 2021.
  * [14] B. Mildenhall, P. P. Srinivasan, M. Tancik, J. T. Barron, R. Ramamoorthi, and R. Ng,  “NeRF: Representing scenes as neural radiance fields for view synthesis,”  Communications of the ACM, vol. 65, no. 1, pp. 99–106, 2021.
  * [15] S. E. Petersen, P. M. Matthews, J. M. Francis, M. D. Robson, F. Zemrak, R. Boubertakh, A. A. Young, S. Hudson, P. Weale, S. Garratt, et al.,  “UK Biobank’s cardiovascular magnetic resonance protocol,”  JCMR, pp. 1–7, 2015.