# Predicting dynamic, motion-related changes in $\text{B}_{\text{0}}$ field in
the brain at a 7 T MRI using a subject-specific fine-tuned U-net

Stanislav Motyka, Paul Weiser, Beata Bachrata, Lukas Hingerl, Bernhard
Strasser, Gilbert Hangel, Eva Niess, Dario Goranovic, Fabian Niess, Maxim
Zaitsev, Simon Daniel Robinson, Georg Langs, Siegfried Trattnig, Wolfgang
Bogner S.M., P.W., B.B., L.H., B.S., G.H., E.N., D.G., F.N., S.R., S.T, and
W.B. are with High Field MR Center, Department of Biomedical Imaging and
Image-guided Therapy, Medical University of Vienna, Vienna, Austria, email:
<EMAIL_ADDRESS>and W.B. are with Christian Doppler
Laboratory for Clinical Molecular MR Imaging, Medical University of Vienna,
Vienna, Austria; G.H. is with Department of Neurosurgery, Medical University
of Vienna, Vienna, Austria; M.Z. is with Department of Radiology – Medical
Physics, University of Freiburg, Faculty of Medicine, University of Freiburg –
Medical Centre, Freiburg, Germany; P.W. and G.L. are with Computational
Imaging Research Lab, Department of Biomedical Imaging and Image-guided
Therapy, Medical University of Vienna, Vienna, Austria; B.B. and S.T. are with
Karl Landsteiner Institute for Clinical Molecular MR in Musculoskeletal
Imaging, Vienna, Austria; B.B. is with Department of Medical Engineering,
Carinthia University of Applied Sciences, Klagenfurt, AustriaThe financial
support by the Austrian Science Fund (FWF P 34198 and FWF TAI-676) is
gratefully acknowledged.

###### Abstract

Subject movement during the magnetic resonance examination is inevitable and
causes not only image artefacts but also deteriorates the homogeneity of the
main magnetic field ($B_{0}$), which is a prerequisite for high quality data.
Thus, characterization of changes to $B_{0}$, e.g. induced by patient
movement, is important for MR applications that are prone to $B_{0}$
inhomogeneities. We propose a deep learning based method to predict such
changes within the brain from the change of the head position to facilitate
retrospective or even real-time correction. A 3D U-net was trained on in vivo
brain $7\text{\,}\mathrm{T}$ MRI data. The input consisted of $B_{0}$ maps and
anatomical images at an initial position, and anatomical images at a different
head position (obtained by applying a rigid-body transformation on the initial
anatomical image). The output consisted of $B_{0}$ maps at the new head
positions. We further fine-tuned the network weights to each subject by
measuring a limited number of head positions of the given subject, and trained
the U-net with these data. Our approach was compared to established dynamic
$B_{0}$ field mapping via interleaved navigators, which suffer from limited
spatial resolution and the need for undesirable sequence modifications.
Qualitative and quantitative comparison showed similar performance between an
interleaved navigator-equivalent method and proposed method. We therefore
conclude that it is feasible to predict $B_{0}$ maps from rigid subject
movement and, when combined with external tracking hardware, this information
could be used to improve the quality of magnetic resonance acquisitions
without the use of navigators.

###### Index Terms:

$B_{0}$ inhomogeneities, U-net, patient movement, artificial neural network,
deep learning, Magnetic resonance imaging, motion correction

##  I Introduction

All in vivo Magnetic Resonance Imaging (MRI) examinations are sensitive to
subject motion. Those requiring prolonged measurement times, are particularly
susceptible [1, 2]. A change in the subject’s position causes motion artefacts
and decreases the homogeneity of the static magnetic field ($B_{0}$) [3, 4].
Changes in $B_{0}$ are increasingly pronounced at ultra-high-field MR scanners
($B_{0}\geq$7\text{\,}\mathrm{T}$)$ [5].

A spatially homogeneous—or at least temporarily stable—$B_{0}$ field is a
prerequisite for several MRI methods. For instance, in MR spectroscopy (MRS),
intra-voxel $B_{0}$ inhomogeneities and temporal frequency changes degrade the
spectral resolution, which translates into reduced chemical specificity [6].
In MRS imaging, they aggravate artefacts arising from extracranial lipid and
unsuppressed water signals [7]. In fast imaging, $B_{0}$ inhomogeneities cause
nonlinear image distortions (e.g., for echo planar imaging) or image blurring
(e.g., for spiral acquisitions) [8]. For Chemical exchange saturation transfer
(CEST), $B_{0}$ inhomogeneities induce frequency offsets [9] which cause
systematic errors in quantification. Long MRI sequences or those with many
repetitions are even more vulnerable to subject motion. To tackle those
issues, several MR-based and external tracking methods have been proposed,
which provide information about the change of the patient position, and some
of them are capable to map the $B_{0}$ distribution.

MR-based tracking methods consist of short MR sequences (termed navigators)
for example based on fast gradient echo scans with echo-planar imaging (EPI)
read-out [10]. Navigators are temporally interleaved with the main (parent)
sequence that requires correction [11]. Simple navigators can monitor subject
position and more advanced volumetric navigators (vNavs) can even map changes
of the $B_{0}$ field over time [12, 13]. However, for vNavs, the acquisition
alone can be as long as $500\text{\,}\mathrm{m}\mathrm{s}$ [14] and can thus
not be easily inserted into the majority of sequences, especially not those
with short TR (frequently under $10\text{\,}\mathrm{m}\mathrm{s}$) [10].
Simpler navigators are easier to implement, but they can only measure global
frequency drift and cannot capture the spatial distribution of $B_{0}$ changes
[3]. Finally, self-navigation allows motion to be monitored e.g. based on
repeated resampling of the k-space center via the parent sequence. Self-
navigation does not require additional scans, but reduces the SNR efficiency
of the sequence and has limited or no ability to characterize $B_{0}$ field
changes depending on the contrast of the main sequence [15].

External tracking methods use additional hardware. Their advantage over
navigators is that motion (detected for example by optical tracking [16, 17])
or changes of the $B_{0}$ field (detected for example by NMR probes [18]) are
acquired independently from the MR scanner and are thus compatible with every
sequence. However, optical or similar tracking systems do not provide
information about $B_{0}$ field changes. NMR probes can track the $B_{0}$
field, but only outside the subject, and translating this to accurate $B_{0}$
estimates inside the body is challenging or needs to be combined with
conventional $B_{0}$ mapping [19]. While optical tracking is already
established as a clinically approved commercial product, an NMR probe system
is a highly specialized and costly piece of equipment that is not generally
supplied as part of an MRI system.

Thus, an approach that combines the benefits of external motion tracking
(i.e., independence from the parent sequence, highly accurate tracking with
high temporal resolution) with that of internal navigators (i.e., accurate
dynamic volumetric $B_{0}$ mapping) without their disadvantages, is highly
desirable. Ideally, it would allow improved real-time (or retrospective)
correction of both motion and $B_{0}$ instabilities.

In recent years, deep learning methods have proved to be successful in
uncovering hidden patterns in image data, which can be leveraged to solve
complex problems, provided that sufficient training data are available [20].
For MRI methods, the image reconstruction [21] as well as the segmentation
[22] and many other problems [23] can be potentially overcome by deep
learning-based methods.

In this study, we propose a neural network (NN) approach to predict changes of
the $B_{0}$ field within the brain from observed changes of the head position
and orientation. A U-net is used to predict a $B_{0}$ map from the following
input: (i) anatomical MRI at the initial position, (ii) initial $B_{0}$ map,
and (iii) head pose change at a certain time point described via six degrees
of freedom. The $B_{0}$ maps are predicted for each known head position/time
point. A general set of weights of the U-net is estimated using the data of 11
volunteers. These weights are then fine-tuned for each volunteer using the
acquired $B_{0}$-maps of six head positions for that volunteer to include
subject-specific information and thus improve the $B_{0}$-prediction. The
whole proposed method would therefore include the measurement of an anatomical
MRI sequence, and the $B_{0}$-sequence of six head positions to refine the
general network weights for the specific subject in a short training
($\sim$1\text{\,}\text{min}$1$) while the subject is in the scanner. These
weights are then used to predict the $B_{0}$-changes caused by motion, which
can be used to correct the data.

Figure 1: 3D U-net architecture used in the study. The input to the network
has three features: (i) $B_{0}$ map of the initial position, (ii) Anatomical
reference of the initial position, and (iii) Anatomical reference of a new
position. The output has one feature: $B_{0}$ map of the new position.

##  II Methods

###  II-A Experimental data

All measurements were carried out on a $7\text{\,}\mathrm{T}$ Magnetom+ MR
Scanner (Siemens Healthineers, Erlangen, Germany) with a 32-channel head coil
(Nova Medical, Wilmington, MA). 15 healthy volunteers (11 males and 4 females)
were included in this study. The study was approved by the Ethics Committee of
the Medical University of Vienna and written informed consent was obtained
from all volunteers.

For each volunteer, an MP2RAGE image [24] was acquired as anatomical reference
with nominal resolution of $1.1\times 1.1\times
1.$1\text{\,}\mathrm{m}\mathrm{m}$$, FOV $=220\times
220\times$220\text{\,}\mathrm{m}\mathrm{m}$$, TE/TR
$=3.28/$5000\text{\,}\mathrm{m}\mathrm{s}$$, TI
$=$700\text{\,}\mathrm{m}\mathrm{s}$$, TI2
$=$2700\text{\,}\mathrm{m}\mathrm{s}$$, GRAPPA factor $=4$, TA =
$4:$57\text{\,}\mathrm{m}\mathrm{i}\mathrm{n}\mathrm{s}$$.

The $B_{0}$ maps were acquired at 30 random head positions per volunteer. All
volunteers were asked to change their head positions randomly, to cover the
possible range within the head coil. The first head position was identical to
that for the MP2RAGE scans. At each head position, two sequences for $B_{0}$
mapping were run:

(i) 2D multi-echo gradient echo (GRE) sequence with nominal resolution of
$1.9\times 1.$9\text{\,}\mathrm{m}\mathrm{m}$$, FOV
$=240\times$240\text{\,}\mathrm{m}\mathrm{m}$$, GRAPPA factor $=4$, $80$
slices with $2\text{\,}\mathrm{m}\mathrm{m}$ thickness, TR
$=$1410\text{\,}\mathrm{m}\mathrm{s}$$,
$\text{TE}_{1-5}=3/6/9/12/$15\text{\,}\mathrm{m}\mathrm{s}$$, Flip angle
$=55^{\circ}$, TA $=$59\text{\,}\mathrm{s}$$;

(ii) 3D dual-echo echo planar imaging (EPI) sequence with nominal resolution
of $8.0\times 8.0\times$8.0\text{\,}\mathrm{m}\mathrm{m}$$, FOV
$=256\times$256\text{\,}\mathrm{m}\mathrm{m}$$, $32$ slices, EPI factor $=16$,
TR $=$9.0\text{\,}\mathrm{m}\mathrm{s}$$,
$\text{TE}_{1-2}=3.8/$4.8\text{\,}\mathrm{m}\mathrm{s}$$, Flip angle
$=2^{\circ}$, TA $=$0.6\text{\,}\mathrm{s}$$.

Figure 2: Comparison of $B_{0}$ maps of four approaches. On the very left, the
anatomical reference is depicted of one volunteer at the initial and the new
head position. For both positions three slices from three planes are plotted.
The second column consists of $B_{0}$ maps at the initial position and the
residua to the ground truth $B_{0}$ map of the new position. The next four
columns depict $B_{0}$ maps of three approaches: (i) EPI-based approach (EPI),
(ii) predicted $B_{0}$ map from not fine-tuned NN (PR0), (iii) predicted
$B_{0}$ map from fine-tuned NN (PRFT); and the ground truth $B_{0}$ maps (GT)
plus their residua to the ground truth $B_{0}$ map at the new position.

###  II-B Experimental data for physics-driven augmentation

In a separate experiment, measurements with a spherical phantom were used to
map the 1st and the 2nd order spherical harmonics of the shimming system of
the $7\text{\,}\mathrm{T}$ Magnetom+ MR Scanner via the same multi-echo GRE
sequence described above. $B_{0}$ shimming was performed using the standard
automatic shim procedure and the initial $B_{0}$ map was measured. The current
amplitudes for each spherical harmonic term were manually altered four times
from its initial $B_{0}$ shim setting in a linear fashion
($-100\text{\,}\mu\mathrm{T}\mathrm{/}\mathrm{m}^{n}$,
$-50\text{\,}\mu\mathrm{T}\mathrm{/}\mathrm{m}^{n}$,
$50\text{\,}\mu\mathrm{T}\mathrm{/}\mathrm{m}^{n}$ and
$100\text{\,}\mu\mathrm{T}\mathrm{/}\mathrm{m}^{n}$, where $n$ is the order of
the spherical harmonic). After each modification, another $B_{0}$ map was
acquired. These data were later used for data augmentation in the neural
network training.

###  II-C Pre-processing of experimental data

For each head position, $B_{0}$ maps were calculated from the GRE sequence and
the EPI-based sequence. GRE-based $B_{0}$ maps were calculated from the
magnitude and phase images coil combined by ASPIRE [25], and phase unwrapped
using ROMEO [26]. $B_{0}$ maps from the 2TE-EPI sequence were calculated as
Hermitian inner product [27]. GRE-based $B_{0}$ maps were acquired in high
spatial resolution with multiple echo time, thus served as the gold standard
method to estimate $B_{0}$ inhomogeneity. For the neural network training,
GRE-based $B_{0}$ maps were considered the ground truth. Low spatial
resolution EPI-based $B_{0}$ maps are equivalent to the dual-echo navigators,
which can be used to estimated $B_{0}$ inhomogeneity in the dead time of the
parent sequence [12].

MP2RAGE data were transformed to different head positions using Flirt from the
FSL toolbox [28] by applying transformation matrices from the co-registration
of the first position to the other positions using the magnitudes of the first
echoes. MP2RAGE datasets were also used to calculate brain masks (BET, FSL
toolbox [28]), which were transformed in the same way as described above.

Spherical harmonics of the the first and the second order of the shimming
system (i.e.X, Y, Z, XY, ZY, Z2, ZX, X2-Y2) [5] were characterized using the
five $B_{0}$ maps with different shim current amplitudes. The measured $B_{0}$
field associated with each shim term was fitted with the respective analytical
spherical harmonic function [29] by a nonlinear curve fitting solver in Matlab
[30].

The training dataset consisted of the 319 instances from 11 volunteers. Each
instance contained the input: (i) anatomical MRI (i.e., T1-weighted MP2RAGE)
at the initial position, (ii) $B_{0}$ map at initial position, and (iii) the
same anatomical MRI, but after applying the 6DoF transformation to the new
position. The output consisted of the $B_{0}$ map at the new position.

Augmentation of the training dataset was performed during each epoch of the
network training by adding the same, randomly-scaled spherical harmonic
$B_{0}$ fields to both input and output $B_{0}$ maps of one instance. The
spherical harmonics are normally used for $B_{0}$ shimming of the volume-of-
interest. Thus, that data augmentation is physically meaningful since the
changes of the $B_{0}$-maps with motion should not depend on the $B_{0}$
shimming, and by adding spherical harmonics $B_{0}$ fields, the acquisition of
the same volunteer under different shimming conditions is simulated.

The test dataset consisted of the data of 4 volunteers. The pre-processing was
similar to that performed on the training dataset.

###  II-D Architecture and training of neural network

All calculations were performed on a DGX station equipped with Tesla V100 GPU
cards (Nvidia, Santa Clara, CA, US). The PyTorch DL framework [31] was used.

The U-net architecture was used [32] because of the ability to extract
features from the input data at different spatial resolutions which are later
used in the decoder part of the network to form a prediction. The network had
4 levels, with the encoder part at each level containing the two 3D
convolution layers each followed by the leakyRelu activation. The spatial
resolution was decreased with a max-pooling layer by a factor of two. The
bottom of the U-net consists of two 3D convolution and one 3D transposed
convolution layers, each followed by leakyRelu activation. The decoder part,
at each level, consists of two 3D transposed convolution each followed by
leaky Relu. The spatial resolution was increase by a trilinear interpolation
by factor of 2. All convolutional layers had a kernel size of 5 in all three
spatial dimensions and convolutions were performed with a stride of one. The
skipped connection was performed as a concatenation of features from the
encoder part to the features of the same spatial resolution in the decoder
part. At the end, 3D convolution was performed with a kernel size of one. The
architecture is depicted in Figure 1.

The training was performed for 2000 epochs with a mini batch of 10. The Adam
optimizer was used with a learning rate of 1e-5 and weight decay of 1e-7. For
each epoch during the training, the order of the training dataset was randomly
permuted and each instance was augmented by the randomly scaled spherical
harmonics. The mean-squared error of the unwrapped $B_{0}$ map and the
prediction formed a loss function.

###  II-E Fine-tuning to specific subject

The U-net trained on the training dataset was fine-tuned to each subject with
a very short training (50 epochs). The first six head positions from all
volunteers in the test dataset were separated for the fine-tuning training and
the 23 head position were kept for evaluation. The Adam optimizer was used
with learning rate of 1e-6, and weight decay of 1e-7.

###  II-F Evaluation

The accuracy of the subject-specific, fined-tuned U-net (PRFT) was compared to
three approaches: (i) no-correction (NC), for which the $B_{0}$ map was not
updated and directly compared to the initial head position; (ii) prediction of
NN, which was not fine-tuned (PR0); (iii) EPI-based approach (EPI), in which
the $B_{0}$ maps were measured at the new position with a navigator-like
sequence set up. The $B_{0}$ maps of all four approaches were compared against
the ground truth data (GRE-based $B_{0}$ maps) and residua maps were
calculated.

The $B_{0}$ maps and the residua maps were compared qualitatively. The
quantitative analysis was performed using the absolute values of residua maps
within the brain mask. For each head position in the testing dataset, median
and interquartile range was calculated of the difference to the ground truth.
Boxplots of these values were created, which summarize the approach overall
performance on the test dataset.

The fine-tuning of the NN for a specific subject was analyzed in terms of the
required minimum number of head positions used for fine-tuning training and
the number of epochs. Fine-tuning was tested with 3, 4, 5, and 6 head
positions and in each case the fine-tuning was performed for 50 epochs. The
number of epochs was tested with 6 head position and 5, 10, 20, 35, 50, 75,
100, 150 and 200 epochs were tested. The quantitative analysis was run over
the residua maps of each fine-tuning test in the same fashion as describe
above.

###  II-G Reproducible research

The code is available on Github [link to the repository will be avaliable].

##  III Results

###  III-A Accuracy of network prediction

The qualitative comparison of $B_{0}$ maps of 4 approaches and their residua
to the ground truth $B_{0}$ maps are depicted along with the anatomical
references in Figure 2. The results are presented in three orthogonal planes.
In the axial and coronal planes, residua maps between the initial and the new
position show a clear left-right gradient of the error, which is caused by
patient movement. In the sagittal plane, a high amplitude hotspot of error is
visible in the frontal lobe.

The gradient as well as the hotspot in the frontal lobe are not visible for
the EPI approach and PRFT. The residua maps of the PR0 contain the slow
spatial gradient in all three orthogonal planes.

Figure 3: Comparison of the medians (section A) and the IQRs (section B) of
the residua maps between four tested apporaches and the ground truth. The
apporachers are: No-correction (NC), prediction by the non-fine-tuned NN
(PR0), EPI-based $B_{0}$ mapping (EPI), and prediction by the fine-tuned NN
(PRFT). Figure 4: Quantitative results of one volunteer. The boxplots of the
absolute $B_{0}$ residua to the ground truth for 4 tested approaches at 23
head positions are presented. No-correction (NC), Prediction by non-fine-tuned
NN (PR0), EPI-based $B_{0}$ mapping (EPI) and prediction by fine-tuned NN
(PRFT) are compared. The fine-tuning was performed with 6 brain volumes in 50
epochs.

Quantitative comparison of overall performance of the four approaches is
depicted in Figure 3. The medians and the IQRs of the absolute values of
residua maps are compared. The EPI approach as well as PRFT yield lower
absolute residua compared to the NC approach. For the NC approach, the median
of the median of the absolute residua was $7.59\text{\,}\mathrm{H}\mathrm{z}$,
while for the EPI approach it was significantly lower,
$3.48\text{\,}\mathrm{H}\mathrm{z}$ (p-value $<<0.0001$), as well as for PRFT,
$3.45\text{\,}\mathrm{H}\mathrm{z}$ (p-value $<<0.0001$). There was no
significance difference between the EPI approach and the PRFT (p-value
$=0.69$).

For the NC approach the median of the IQR of the absolute residua was
$10.94\text{\,}\mathrm{H}\mathrm{z}$, which was significantly higher compared
to the other three approaches: $8.37\text{\,}\mathrm{H}\mathrm{z}$ (p-value
$=2.21e-5$) for PR0; $4.82\text{\,}\mathrm{H}\mathrm{z}$ (p-value $<<0.0001$)
for EPI approach, $4.48\text{\,}\mathrm{H}\mathrm{z}$ (p-value $<<0.0001$) for
PRFT. The PRFT results are significantly lower to the PR0 (p-value
$<<0.0001$). There was no significant difference between PRFT and the EPI
approach (p-value $=0.57$).

A quantitative comparison of methods for one volunteer is depicted in Figure
4. 23 head positions are evaluated. Boxplots of absolute residua maps for each
method at each head position are plotted. For the NC approach and the PR0 the
medians of absolute residua are above $5\text{\,}\mathrm{H}\mathrm{z}$ in all
cases. For the EPI approach and the PRFT, the medians in all case are below 5
Hz. Moreover, the average of upper quartiles is 5.58 Hz for the EPI approach
and 5.19 Hz for the PRFT.

###  III-B Analysis of fine-tuning procedure

Quantitative results of the fine-tuning evaluation are depicted in Figure 5 in
terms of number of epochs and in Figure 6 for the number of brain volumes used
for the fine-tuning.

The effect of using a different number of epochs for the fine-tuning was
evaluated in a range between 5 and 200 epochs. In Figure 5, section A, a trend
of decreasing of the median of the absolute median residua can be observed in
a range between 5 and 50 epochs. In a range between 50 and 200 epochs, the
difference in the median values were not observed.

The IQR metric followed the same trend, as shown in Figure 5, section B. Only
in the the range between 5 and 50 epoch of fine-tuning training, the values
were decreasing. For the fine-tuning, which lasted longer then 50 epochs,
there were no differences compared to the case of 50 epochs of the fine-
tuning.

The number of volumes used for the fine-tuning training were compared in a
range 3 to 6 brain volumes. There were no major differences for the median of
the absolute median residua, depicted in Figure 6, section A, nor for the IQRs
of the absolute median residua, depicted in Figure 6, section B.

Figure 5: Quantitative comparison of the number of epochs for fine tuning
training. A range of epoch was between 5 and 200. The results of fine-tuning
are plotted along side other three methods: No-correction (NC), EPI-base
$B_{0}$ mapping (EPI), and prediction by non-fine-tuned NN (PR0). Section A:
Comparison of the medians of absolute residua maps. Section B: Comparison of
the IQRs of the absolute residua maps. Figure 6: Quantitative comparison of
the number of volumes for fine tuning training. The range of used volumes was
between 3 and 6. The results of fine-tuning are plotted along side other three
methods: No-correction (NC), Epi-base $B_{0}$ mapping (EPI), and prediction by
non-fine-tuned NN (PR0). Section A: Comparison of the medians of absolute
residua maps. Section B: Comparison of the IQRs of the absolute residua maps.

##  IV Discussion

This work presents the proof-of-principle investigation of a method to predict
motion-induced $B_{0}$ changes via a NN from available rigid body head motion
logs (e.g., obtained from external motion tracking), an initial $B_{0}$ map
and an initial anatomical image. The prediction of the $B_{0}$ maps was
carried out with a U-net trained with the experimentally acquired and
augmented data of 11 volunteers. For each volunteer in the test dataset, the
network was fine-tuned with a small number of subject-specific data and a
limited number of epochs. The performance of the network was compared with
three other approaches using a test dataset of four volunteers.

$B_{0}$ (as well as $B_{1}$) inhomogeneities can cause severe artifacts in
reconstructed images. Several other methods were therefore proposed in the
past to estimate field inhomogeneities and correct for them [33, 34, 35, 36,
37]. Such calibration methods typically require a specific calibration scan at
the beginning of each MRI acquisition protocol. The results are then used to
set the parameters of the following sequences. In case of subject movement,
the $B_{0}$ information is not updated, which is analogous to the no-
correction approach. Repeated $B_{0}$ mapping throughout the acquisition
protocol is therefore necessary to account for any temporal instability (e.g.,
patient movement related $B_{0}$ changes). vNavs—typically interleaved with
the main (parent) sequence—are able to provide dynamic $B_{0}$ estimates, but
would often increase the total scan time and lower the SNR-per-unit-time
efficiency in an unacceptable way [13]. In some specific cases (e.g., fMRI),
the $B_{0}$ maps can be calculated from the phase of EPI images [38], but such
approaches are not very flexible and often require making certain compromises
in the parent sequence [39]. In contrast, our NN approach is capable to
predict the change of the $B_{0}$ with high temporal resolution without
measuring extra data during the subsequent sequences in the particular
volunteer’s MRI acquisitions. If motion information can be acquired
externally, e.g. by optical tracking or directly from the MR data, no sequence
parameter changes are necessary for our proposed NN method, which relies only
on an accurate knowledge of the transformation matrices describing the rigid
body motion. The fine-tuning procedure depends on the limited number of
subject-specific data, which can be acquired at the beginning of the MR
investigation protocol.

In recent years, deep learning methods have been applied in the reconstruction
of MRI data [23], mainly assuming that the acquired k-space data are artefact-
free. A small number of methods have been proposed which reduce or remove some
artifacts arising from $B_{0}$ inhomogeneities but this is, to the best of our
knowledge, the first which directly predicts $B_{0}$ map inhomogeneities. In
the context of distortions in EPI images caused by $B_{0}$ inhomogeneities,
deep-learning-based methods have been proposed to directly predict distortion
free images [40, 41]. Another deep-learning based approach was designed to
compensate for the artifact due to $B_{0}$ fluctuations arising from
respiration in multi-slice GRE by predicting the phase error term from the
corrupted images [42]. In contrast, our approach is less direct, but more
flexible.

We have used subject-specific fined-tuned NN to predict $B_{0}$ maps. In
general, this PRFT approach outperformed PR0 (i.e., without fine-tuning) and
the NC approach. The results of PRFT are similar to those obtained with the
EPI approach. The NC method yielded the highest medians of $B_{0}$ residua as
well as IQRs. The $B_{0}$ residua maps shows the left-right gradient of error
in the axial and the coronal plane. The sagittal plane frequently shows error
hotspots in the frontal lobe. Those effects are in agreement with a previously
published analysis [43].

The PR0 approach yielded slightly better results compared to the NC approach.
The quantitative comparison showed improvement in the medians and the IRQs of
$B_{0}$ residua maps. From the investigation of methods per single head
position, the PR0 had similar performance for each position, no matter how
severe the error of the NC approach was. The residua maps of the PR0 shows a
reverse gradient of the error compared to the NC approach. However, the
$B_{0}$ maps themselves have similar features compared to the PRFT. The main
difference is in their magnitude.

PRFT results were similar to those with the EPI approach in several
investigations. The $B_{0}$ maps of both methods are comparable to the ground
truth. Their $B_{0}$ maps residua did not contain the left-right gradient and
the frontal lobe hotpot, which are typical $B_{0}$ inhomogeneities originating
from subject movement. Quantitative results showed the same results for the
median and IQRs of the absolute $B_{0}$ map residua in the overall comparison
of the methods as well as in the comparison of methods per head volume.

The EPI approach and the fine-tuning required acquisition of additional
subject-specific data. However, while the EPI approach acquires data during
the whole scan, the data for fine-tuning are only acquired at the beginning of
the MR protocol as a prescan. Once the network is fine-tuned for a specific
subject, only the tracking of movement is required. These motion logs could be
acquired via several internal or external methods [10]. The one most
independent from the MR acquisitions, and hence the most versatile, is optical
tracking, in which information about the movement is sampled at a very high
temporal rate up to $80\text{\,}\mathrm{H}\mathrm{z}$ [17]. Another approach
with minimal interference to the MR acquisition is to use navigators based on
highly-undersampled kSpace [44], which takes only
$2.3\text{\,}\mathrm{m}\mathrm{s}$ to detect the rigid-body movement.

Acquiring subject-specific additional data (e.g. as prescan) for the MRI
reconstruction is common practice for conventional methods. E.g. the parallel
imaging method GRAPPA requires ACS lines to reconstruct the missing kSpace
points [45]. Similarly, SENSE requires measured coil sensitivity profiles to
disentangle aliased MRI images [46]. Subject-specific NN were also proposed by
Akçakaya et al. [47] to perform MRI reconstructions.

Our investigation of the subject-specific NN fine-tuning suggested that the
amount of the additional training datasets can be as little as three volumes.
The amount of head positions used for training were investigated and no
significant differences were shown in the range 3 to 6 datasets. However,
there were no special instruction for the fine-tuning data. Further
optimization could, thus, lead to improved results. The fine-tuning should be
performed for a sufficient time, however after some point the improvement is
saturated. In our case, 50 epochs (which took approximately 1 minute without
any specific optimization) were sufficient for the fine-tuning training with
the 6 head positions.

###  IV-A Limitations and Outlook

The paper presents a proof-of-principle and there are many details which can
be improved. The training dataset was created from only 11 volunteers. For
each volunteer only $B_{0}$ maps at 30 positions were acquired and although
the distribution of test and training data was similar, more data on a much
more diverse group of subjects (e.g., different head sizes) will be necessary
to achieve optimal results. The training dataset was augmented by a physics-
driven concept. However, the number of head positions remained unchanged. The
augmentation, thus, simulates only the differences in the $B_{0}$ shimming of
the volume-of-interest. Also other augmentation approaches and tests on
generalizability should be considered. The subject-specific fine-tuning was
performed with six volumes in 50 epochs. No special instruction were given,
which can be improved by a tailored process of fine-tuning sampling for a
given coil.

Future research should involve a combination with external motion tracking
hardware and evaluating the benefits for $B_{0}$\- and motion-sensitive MRI
sequences. Optical tracking could provide updates of the patient position with
temporal resolution up to 85 Hz. The proposed method requires calibration
sampling for the fine-tuning and short training. After that the input to the
NN consist of anatomical images, the initial $B_{0}$ map from the beginning of
the measurement protocol, and information of the rigid movement. The
prediction of the $B_{0}$ map is completely independent from the MR scanner.
Thus, the valuable information about the change of the $B_{0}$ map due to
subjects movement, which is usually available only from lengthy volumetric
navigators, could be predicted with the same temporal resolution as is
available from rigid-body motion logs. In the future these predicted $B_{0}$
maps could even be used for real-time updating both the MRI volume and the
$B_{0}$ shim parameters together during the acquisition of the MR data. This
can ultimately lead to significantly improved data quality for a range of
$B_{0}$-sensitive MRI methods.

##  V Conclusion

This paper presents the proof-of-principle implementation of a new deep
learning-based and subject-specific approach to predicting the change of the
$B_{0}$ maps due to patient movement. Results were compared to the ground
truth and the established EPI approach, which is equivalent to using lengthy
volumetric navigators. Our results suggest that the prediction of $B_{0}$ maps
is feasible and highly accurate. In combination with external tracking, a
considerable improvement in data quality of $B_{0}$-sensitive MRI methods
could be expected.

## References

  * [1] E. Heckova, M. Považan, B. Strasser, P. Krumpolec, P. Hnilicová, G. Hangel, P. Moser, O. C. Andronesi, A. van der Kouwe, P. Valkovic, B. Ukropcova, S. Trattnig, and W. Bogner, “Real-time correction of motion and imager instability artifacts during 3d $\gamma$-aminobutyric acid-edited mr spectroscopic imaging,” Radiology, vol. 286, pp. 666–675, 2 2018.
  * [2] M. Zaitsev, J. Maclaren, and M. Herbst, “Motion artefacts in mri: a complex problem with many partial solutions,” Journal of magnetic resonance imaging : JMRI, vol. 42, p. 887, 10 2015.
  * [3] W. Bogner, A. T. Hess, B. Gagoski, M. D. Tisdall, A. van der Kouwe, S. Trattnig, B. R. Rosen, and O. C. Andronesi, “Real-time motion- and b0-correction for laser-localized spiral-accelerated 3d-mrsi of the brain at 3t,” NeuroImage, vol. 88, pp. 22–31, 3 2014.
  * [4] J. Liu, J. A. de Zwart, P. van Gelderen, J. Murphy-Boesch, and J. H. Duyn, “Effect of head motion on mri b 0 field distribution,” Magnetic resonance in medicine, vol. 80, pp. 2538–2548, 12 2018.
  * [5] C. Juchem and R. A. de Graaf, “B0 magnetic field homogeneity and shimming for in vivo magnetic resonance spectroscopy,” Analytical Biochemistry, vol. 529, pp. 17–29, 7 2017.
  * [6] S. Motyka, P. Moser, L. Hingerl, G. Hangel, E. Heckova, B. Strasser, K. Eckstein, S. D. Robinson, B. Poser, S. Gruber, S. Trattnig, and W. Bogner, “The influence of spatial resolution on the spectral quality and quantification accuracy of whole-brain mrsi at 1.5t, 3t, 7t, and 9.4t,” Magnetic Resonance in Medicine, 2019.
  * [7] A. A. Maudsley, O. C. Andronesi, P. B. Barker, A. Bizzi, W. Bogner, A. Henning, S. J. Nelson, S. Posse, D. C. Shungu, and B. J. Soher, “Advanced magnetic resonance spectroscopic neuroimaging: Experts’ consensus recommendations,” NMR in Biomedicine, 2020.
  * [8] M. Wu, L. Chang, L. Walker, H. Lemaitre, A. Barnett, S. Marenco, and C. Pierpaoli, “Comparison of epi distortion correction methods in diffusion tensor mri using a novel framework,” Medical image computing and computer-assisted intervention : MICCAI … International Conference on Medical Image Computing and Computer-Assisted Intervention, vol. 11, pp. 321–329, 2008.
  * [9] P. Z. Sun, C. T. Farrar, and A. G. Sorensen, “Correction for artifacts induced by b0 and b1 field inhomogeneities in ph-sensitive chemical exchange saturation transfer (cest) imaging,” Magnetic Resonance in Medicine, vol. 58, pp. 1207–1215, 12 2007.
  * [10] O. C. Andronesi, P. K. Bhattacharyya, W. Bogner, I.-Y. Choi, A. T. Hess, P. Lee, E. M. Meintjes, M. D. Tisdall, M. Zaitzev, and A. van der Kouwe, “Motion correction methods for mrs: experts’ consensus recommendations,” NMR in Biomedicine, vol. 34, p. e4364, 5 2021.
  * [11] N. White, C. Roddey, A. Shankaranarayanan, E. Han, D. Rettmann, J. Santos, J. Kuperman, and A. Dale, “Promo: Real-time prospective motion correction in mri using image-based tracking,” Magnetic Resonance in Medicine, vol. 63, pp. 91–105, 1 2010.
  * [12] A. T. Hess, M. D. Tisdall, O. C. Andronesi, E. M. Meintjes, and A. J. W. van der Kouwe, “Real-time motion and b0 corrected single voxel spectroscopy using volumetric navigators,” Magnetic Resonance in Medicine, vol. 66, pp. 314–323, 8 2011.
  * [13] P. Moser, K. Eckstein, L. Hingerl, M. Weber, S. Motyka, B. Strasser, A. Kouwe, S. Robinson, S. Trattnig, and W. Bogner, “Intra‐session and inter‐subject variability of 3d‐fid‐mrsi using single‐echo volumetric epi navigators at 3t,” Magnetic Resonance in Medicine, vol. 83, pp. 1920–1929, 6 2020.
  * [14] J. Liu, P. van Gelderen, J. A. de Zwart, and J. H. Duyn, “Reducing motion sensitivity in 3d high-resolution t2*-weighted mri by navigator-based motion and nonlinear magnetic field correction,” NeuroImage, vol. 206, p. 116332, 2 2020.
  * [15] D.-H. Kim, E. Adalsteinsson, and D. M. Spielman, “Spiral readout gradients for the reduction of motion artifacts in chemical shift imaging,” Magnetic Resonance in Medicine, vol. 51, pp. 458–463, 3 2004.
  * [16] M. Zaitsev, C. Dold, G. Sakas, J. Hennig, and O. Speck, “Magnetic resonance imaging of freely moving objects: prospective real-time motion correction using an external optical motion tracking system,” NeuroImage, vol. 31, pp. 1038–1050, 7 2006.
  * [17] J. Maclaren, B. S. Armstrong, R. T. Barrows, K. A. Danishad, T. Ernst, C. L. Foster, K. Gumus, M. Herbst, I. Y. Kadashevich, T. P. Kusik, Q. Li, C. Lovell-Smith, T. Prieto, P. Schulze, O. Speck, D. Stucht, and M. Zaitsev, “Measurement and correction of microscopic head motion during magnetic resonance imaging of the brain,” PLoS ONE, vol. 7, 11 2012.
  * [18] N. D. Zanche, C. Barmet, J. A. Nordmeyer-Massner, and K. P. Pruessmann, “Nmr probes for measuring magnetic fields and field dynamics in mr systems,” Magnetic Resonance in Medicine, vol. 60, pp. 176–186, 7 2008.
  * [19] C. Barmet, N. D. Zanche, and K. P. Pruessmann, “Spatiotemporal magnetic field monitoring for mr,” Magnetic Resonance in Medicine, vol. 60, pp. 187–197, 7 2008.
  * [20] Y. LeCun, Y. Bengio, and G. Hinton, “Deep learning,” Nature, vol. 521, pp. 436–444, 5 2015.
  * [21] F. Knoll, T. Murrell, A. Sriram, N. Yakubova, J. Zbontar, M. Rabbat, A. Defazio, M. J. Muckley, D. K. Sodickson, C. L. Zitnick, and M. P. Recht, “Advancing machine learning for mr image reconstruction with an open competition: Overview of the 2019 fastmri challenge,” Magnetic Resonance in Medicine, vol. 84, pp. 3054–3070, 1 2020.
  * [22] X. Liu, L. Song, S. Liu, Y. Zhang, C. Feliu, and D. Burgos, “A review of deep-learning-based medical image segmentation methods,” Sustainability, 2021.
  * [23] A. S. Lundervold and A. Lundervold, “An overview of deep learning in medical imaging focusing on mri,” Zeitschrift fur Medizinische Physik, vol. 29, pp. 102–127, 5 2019.
  * [24] J. P. Marques and R. Gruetter, “New developments and applications of the mp2rage sequence–focusing the contrast and high spatial resolution r1 mapping.,” PloS one, vol. 8, p. e69294, 2013.
  * [25] K. Eckstein, B. Dymerska, B. Bachrata, W. Bogner, K. Poljanc, S. Trattnig, and S. D. Robinson, “Computationally efficient combination of multi-channel phase data from multi-echo acquisitions (aspire),” Magnetic Resonance in Medicine, vol. 79, pp. 2996–3006, 6 2018.
  * [26] B. Dymerska, K. Eckstein, B. Bachrata, B. Siow, S. Trattnig, K. Shmueli, and S. D. Robinson, “Phase unwrapping with a rapid opensource minimum spanning tree algorithm (romeo),” Magnetic Resonance in Medicine, vol. 85, pp. 2294–2308, 4 2021.
  * [27] M. A. Bernstein, M. Grgic, T. J. Brosnan, and N. J. Pelc, “Reconstructions of phase contrast, phased array multicoil data,” Magnetic Resonance in Medicine, vol. 32, pp. 330–334, 9 1994.
  * [28] M. Jenkinson, P. Bannister, M. Brady, and S. Smith, “Improved optimization for the robust and accurate linear registration and motion correction of brain images,” NeuroImage, 2002.
  * [29] M. J. Caola, “Solid harmonics and their addition theorems,” J. Phys. A: Math. Gen, vol. 11, 1978.
  * [30] MATLAB, version 9.3.0.713579 (R2017b).  Natick, Massachusetts: The MathWorks Inc., 2017.
  * [31] A. Paszke, S. Gross, S. Chintala, G. Chanan, E. Yang, Z. D. Facebook, A. I. Research, Z. Lin, A. Desmaison, L. Antiga, O. Srl, and A. Lerer, “Automatic differentiation in pytorch,” NIPS-W, 2017.
  * [32] O. Ronneberger, P. Fischer, and T. Brox, “U-net: Convolutional networks for biomedical image segmentation,” Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics), vol. 9351, pp. 234–241, 2015.
  * [33] S. Robinson and J. Jovicich, “B0 mapping with multi-channel rf coils at high field,” Magnetic Resonance in Medicine, vol. 66, pp. 976–988, 10 2011.
  * [34] P. Jezzard and R. S. Balaban, “Correction for geometric distortion in echo planar images from b0 field variations,” Magnetic Resonance in Medicine, vol. 34, pp. 65–73, 7 1995.
  * [35] N.-K. Chen and A. M. Wyrwicz, “Correction for epi distortions using multi-echo gradient-echo imaging,” Magnetic Resonance in Medicine, vol. 41, pp. 1206–1213, 1999.
  * [36] P. Reber, E. Wong, R. Buxton, and L. Frank, “Correction of off resonance-related distortion in echo-planar imaging using epi-based field maps,” Magnetic resonance in medicine, vol. 39, pp. 328–330, 1998.
  * [37] K. Sekihara, S. Matsui, and H. Kohno, “Nmr imaging for magnets with large nonuniformities.,” IEEE Transactions on Medical Imaging, vol. MI-4, pp. 193–199, 1985.
  * [38] B. Dymerska, B. A. Poser, M. Barth, S. Trattnig, and S. D. Robinson, “A method for the dynamic correction of b0-related distortions in single-echo epi at 7 t,” NeuroImage, vol. 168, pp. 321–331, 3 2018.
  * [39] B. Dymerska, B. A. Poser, W. Bogner, E. Visser, K. Eckstein, P. Cardoso, M. Barth, S. Trattnig, and S. D. Robinson, “Correcting dynamic distortions in 7t echo planar imaging using a jittered echo time sequence,” Magnetic resonance in medicine, vol. 76, pp. 1388–1399, 11 2016.
  * [40] Z. Hu, Y. Wang, Z. Zhang, J. Zhang, H. Zhang, C. Guo, Y. Sun, and H. Guo, “Distortion correction of single-shot epi enabled by deep-learning,” NeuroImage, vol. 221, p. 117170, 11 2020.
  * [41] K. G. S. Id, J. Blaber, C. Hansen, L. Cai, B. R. Id, A. W. Anderson, S. Smith, P. Kanakaraj, T. Rex, S. M. Resnick, A. T. Shaferid, L. E. Cutting, N. Woodward, D. Zald, and B. A. Landman, “Distortion correction of diffusion weighted mri without reverse phase-encoding scans or field-maps,” PlosOne, 2020.
  * [42] H. An, H. G. Shin, S. Ji, W. Jung, S. Oh, D. Shin, J. Park, and J. Lee, “Deepresp: Deep learning solution for respiration-induced b0 fluctuation artifacts in multi-slice gre,” NeuroImage, vol. 224, p. 117432, 1 2021\.
  * [43] A. T. Hess, O. C. Andronesi, M. D. Tisdall, A. G. Sorensen, A. J. van der Kouwe, and E. M. Meintjes, “Real-time motion and b 0 correction for localized adiabatic selective refocusing (laser) mrsi using echo planar imaging volumetric navigators,” NMR in Biomedicine, vol. 25, pp. 347–358, 2 2012.
  * [44] T. Ulrich, M. Riedel, and K. Pruessmann, “Prospective head motion correction using orbital k-space navigators and a linear perturbation model,” Proc. Joint Annual Meeting ISMRM-ESMRMB, London, UK, p. 0353, 2022.
  * [45] M. A. Griswold, P. M. Jakob, R. M. Heidemann, M. Nittka, V. Jellus, J. Wang, B. Kiefer, and A. Haase, “Generalized autocalibrating partially parallel acquisitions (grappa),” Magnetic Resonance in Medicine, vol. 47, pp. 1202–1210, 2002.
  * [46] K. P. Pruessmann, M. Weiger, M. B. Scheidegger, and P. Boesiger, “Sense: Sensitivity encoding for fast mri,” Magnetic Resonance in Medicine, vol. 42, pp. 952–962, 11 1999.
  * [47] M. Akçakaya, S. Moeller, S. Weingärtner, and K. Uğurbil, “Scan-specific robust artificial-neural-networks for k-space interpolation (raki) reconstruction: Database-free deep learning for fast imaging,” Magnetic Resonance in Medicine, vol. 81, pp. 439–453, 1 2019.