# Generalizable synthetic MRI with physics-informed convolutional networks

Luuk Jacobs1,2, Stefano Mandija1,2, Hongyan Liu1,2, Cornelis A.T. van den
Berg1,2, Alessandro Sbrizzi1,2 and Matteo Maspero1,20

1Department of Radiotherapy, UMC Utrecht, Utrecht, The Netherlands
2Computational Imaging Group for MR Diagnostics and Therapy, UMC Utrecht,
Utrecht, The Netherlands Corresponding author. E-mail:
<EMAIL_ADDRESS><EMAIL_ADDRESS>
###### Abstract

In this study, we develop a physics-informed deep learning-based method to
synthesize multiple brain magnetic resonance imaging (MRI) contrasts from a
single five-minute acquisition and investigate its ability to generalize to
arbitrary contrasts to accelerate neuroimaging protocols. A dataset of fifty-
five subjects acquired with a standard MRI protocol and a five-minute
transient-state sequence was used to develop a physics-informed deep learning-
based method. The model, based on a generative adversarial network, maps data
acquired from the five-minute scan to “effective” quantitative parameter maps,
here named q*-maps, by using its generated PD, T1, and T2 values in a signal
model to synthesize four standard contrasts (proton density-weighted,
T1-weighted, T2-weighted, and T2-weighted fluid-attenuated inversion
recovery), from which losses are computed. The q*-maps are compared to
literature values and the synthetic contrasts are compared to an end-to-end
deep learning-based method proposed by literature. The generalizability of the
proposed method is investigated for five volunteers by synthesizing three non-
standard contrasts unseen during training and comparing these to respective
ground truth acquisitions via contrast-to-noise ratio and quantitative
assessment. The physics-informed method was able to match the high-quality
synthMRI of the end-to-end method for the four standard contrasts, with mean
$\pm$ standard deviation structural similarity metrics above $0.75\pm 0.08$
and peak signal-to-noise ratios above $22.4\pm 1.9$ and $22.6\pm 2.1$.
Additionally, the physics-informed method provided retrospective contrast
adjustment, with visually similar signal contrast and comparable contrast-to-
noise ratios to the ground truth acquisitions for three sequences unused for
model training, demonstrating its generalizability and potential application
to accelerate neuroimaging protocols.

Key words: Synthetic MRI, quantitative MRI, deep learning, generative
adversarial network

##  1 Introduction

Conventionally, multiple complementary contrast-weighted magnetic resonance
imaging (MRI) images are separately acquired for disease diagnosis. For
example, standard brain imaging examinations include PD-weighted (PDw),
T1-weighted (T1w), T2-weighted (T2w), and T2-weighted fluid-attenuated
inversion recovery (T2-FLAIR) contrasts among many other acquisitions [1]. The
qualitative nature of conventional MRI impedes the comparison of signal
intensities between examinations from different time points, patients, and
vendors. Although different scanners output qualitatively similar images, they
exhibit considerable variation when viewed quantitatively [2]. Over the last
decades, attempts to make MRI quantitative have been pursued. Quantitative MRI
(qMRI) aims to facilitate standardized MRI-based measurements, reduce bias,
and increase reproducibility [3]. In the last decades, efforts have been
dedicated to developing qMRI techniques mapping biophysical tissue parameters
such as T1, T2, and PD (q-maps) within a clinically viable acquisition and
reconstruction time, i.e., 3-5 minutes or less. Examples of other tissue
parameters are T2*, apparent diffusion coefficients, flow, perfusion, and
stiffness, but these are not considered in this work. Three examples of such
fast qMRI techniques are MR fingerprinting (MRF) [4], Magnetic Resonance Spin
TomogrAphy in Time-domain (MR-STAT) [5], and fitting of a signal model to
multi-dynamic multi-echo (MDME) MRI [6].

To still provide radiologists with the standard contrasts routinely used for
neurological diagnoses, the reconstructed q-maps can be used to generate so-
called “synthetic MRI” (synthMRI), making use of signal models [7]. This way,
synthMRI provides intrinsically registered, multi-contrast images from a
single scan, reducing examination time compared to conventional MRI. MRF [8,
9], MR-STAT [10, 11], and model fitting to MDME data [6] have all been
explored to facilitate synthMRI. The latter commercial solution has even been
implemented in clinics. Several studies showed that this commercial solution
may facilitate patients’ diagnosis on par with conventional contrasts [12],
with applications for gliomas [13], brain metastases [14], Sturge-Weber
syndrome [15], multiple sclerosis [16], and stroke [17]. However, synthetic
T2-FLAIR contrasts remain challenging for synthMRI [18, 19]. The quality of
synthetic T2-FLAIR contrasts can be hindered by oversimplified signal models
that do not model effects like partial volume, magnetization transfer, and
flow [20, 21, 16]. The subsequent artifacts can result in coarse
hyperintensities and a lack of contrast between the lesion and surrounding
tissues, making it necessary to acquire an additional conventional T2-FLAIR to
confirm the diagnosis and lose the promised decrease in scan time from
synthMRI [22].

Deep learning (DL) is a subfield of machine learning that focuses on
developing models that learn abstract data representations using data-driven
training strategies [23]. DL has achieved great success for various medical
imaging tasks such as segmentation [24], MRI reconstruction [25], super-
resolution [26], and image modality conversion [27] and has recently been
proposed to facilitate synthMRI. Generative adversarial networks (GANs) [28]
are a class of DL that has shown promising results for a variety of medical
applications, particularly for image synthesis and image-to-image translation
[29]. Specifically, due to its fast inference times and representation
capability, it has the potential to overcome imperfections in qMRI
reconstructions and signal models in a data-driven manner [30, 31]. So far,
only DL-based synthMRI methods that exclusively synthesize specific contrasts
provided during training have been extensively investigated. These methods
cannot generalize to unseen contrasts during training, meaning additional
acquisitions are still needed if contrast tweaking or non-standard contrasts
are necessary to answer clinical demands.

This work111Code will be made publicly available on
https://gitlab.com/computational-imaging-lab/qstarMRI investigates the use of
DL to obtain synthMRI from a single full-brain, five-minute scan for four
routinely acquired contrasts and generalizability to unseen contrasts during
training. We propose incorporating a physics-based signal model into the
framework to achieve this generalizability. We will compare the physics-
informed DL-based synthMRI constrasts to separately acquired ground truths and
investigate whether generalizability to contrasts unseen during training for a
volunteer is feasible.

Figure 1: Schematic representation of possible synthMRI approaches. The
standard synthMRI approach (solid black arrows) starts with qMRI
reconstruction, from which synthMRI is obtained via a signal model. End-to-end
DL-based methods (dotted red arrow) have been proposed to skip the qMRI
reconstruction and signal model. Our proposed physics-informed method (striped
blue arrows) aims to address the lack of generalizability of the end-to-end
approach by outputting effective q-maps (q*-maps) and feeding these to the
signal model to obtain synthMRI.

##  2 Theory

The standard synthMRI approach (Fig. 1, solid black arrows) consists of a qMRI
reconstruction that results in q-maps and a voxel-wise signal model to
synthesize contrasts from the q-maps. Various qMRI techniques have been
proposed, and generally, an analytical solution to the Bloch equations is used
as the signal model [8, 10, 6], as described by:

$\mathrm{S}=\mathrm{PD}\cdot
e^{-\mathrm{TE}/\mathrm{T}_{2}}\cdot\frac{1-\left[1-\cos\left(\theta\right)\right]e^{-\mathrm{TI}/\mathrm{T}_{1}}-\cos\left(\theta\right)e^{-\mathrm{TD}/\mathrm{T}_{1}}}{1-\cos\left(\mathrm{FA}\right)\cos\left(\theta\right)e^{-\mathrm{TD}/\mathrm{T}_{1}}}$
(1)

with echo time $\mathrm{TE}$, saturation pulse angle $\theta$
($\theta=180^{\circ}$ for inversion recovery (IR) sequences, otherwise
$\theta=0^{\circ}$), inversion time $\mathrm{TI}$, delay time $\mathrm{TD}$
($\mathrm{TD}=\mathrm{TR}-\mathrm{ETL}\cdot\mathrm{ESP}$ for turbo-spin echo
(TSE) sequence, $\mathrm{TD}=\mathrm{TR}$ for spin-echo (SE) sequence, with
ETL = echo train length and ESP = echo spacing), and flip angle $\mathrm{FA}$
[6, 32].

Recently, supervised end-to-end DL-based methods (Fig. 1, dotted red arrow)
have been proposed [30, 31], which train a neural network (NN) to directly
synthesize contrasts from the acquired data by optimizing the following
objective function:

$\boldsymbol{w^{*}}=\arg\min_{\boldsymbol{w}}\mathcal{L}\big{(}\boldsymbol{y},\text{NN}(\boldsymbol{x},\boldsymbol{w})\big{)}$
(2)

with weights $\boldsymbol{w}$, loss function $\mathcal{L}$, contrasts
$\boldsymbol{y}$, and acquired data $\boldsymbol{x}$. For example, K. Wang et
al. [30] proposed synthesizing T1w, T2w, and T2-FLAIR contrasts from MRF
acquisition data using DL, allowing the model to learn physiological effects
in a data-driven manner. However, this restricts the synthesis during
inference to only the contrasts $\boldsymbol{y}$, which are part of the paired
dataset. Synthesizing different contrasts would require additional MRI scans
to expand the dataset, which can be very expensive [33]. Furthermore, the NN
would need to be retrained and re-evaluated with the new dataset.

To provide generalizability to unseen contrasts for DL-based synthMRI methods,
we propose a physics-informed method (Fig. 1, striped blue arrows). The model
first maps the acquired data to quantitative PD, T1, and T2 values, so-called
“effective q-maps” (q*-maps), which are then fed into the signal model
described by Eq. (1). The model is trained to optimize for the following
objective function:

$\boldsymbol{w^{*}}=\arg\min_{\boldsymbol{w}}\mathcal{L}\bigg{(}\boldsymbol{y},\text{S}\big{(}\text{NN}(\boldsymbol{x},\boldsymbol{w}),\text{TE},\text{TR},\text{TI}\big{)}\bigg{)}$
(3)

By performing contrast synthesis via q*-maps, all resulting synthMRI contrasts
are enforced to be jointly consistent with the signal model. Similar to the
standard synthMRI methods, a computationally inexpensive signal model is used
for synthesis. However, our physics-informed DL-based network is trained to
incorporate the signal model imperfections; effects that the signal model does
not incorporate can still be captured in a data-driven manner via the q*-maps.
Because of the effective nature of the q*-maps, their quantitative PD, T1, and
T2 values may differ from q-maps reconstructed using standard qMRI methods,
which could harm synthMRI quality. We hypothesize that with this physics-
informed training setup, we can use the q*-maps to provide generalizability to
unseen contrasts during inference by varying the desired sequence parameters
in the signal model. Additionally, contrast synthesis via q*-maps is more
interpretable compared to the "black box" end-to-end methods, which improves
trust and conduces clinical adaption.

Table 1: Sequence parameters of the MR-STAT acquisition, standard contrasts,
and unseen contrasts. For all sequences, geometric parameters were kept
constant: field of view of $224$x$224$x$134$ mm3, acquisition and
reconstruction resolution of $1$x$1$x$3$ mm3, and $30$ slices with $1.5$ mm
gaps for all scans.

|  Acquisition name |  TR |  TE |  TI |  FA |  ETL |  ESP |  TA
---|---|---|---|---|---|---|---|---
|  |  [ms] |  [ms] |  [ms] |  [degrees] |  [-] |  [ms] |  [min:sec]
MR-STAT acquisition |  Multi2D spoiled Cartesian GRE |  8.9 |  4.7 |  N/A |  0-90 |  N/A |  N/A |  4:50
Standard |  T1w MS SE |  864 |  14 |  N/A |  70 |  1 |  N/A |  3:16
contrasts |  T2w MS TSE |  3254 |  80 |  N/A |  90 |  15 |  10 |  1:44
|  PDw MS TSE |  2800 |  20 |  N/A |  90 |  14 |  9 |  1:41
|  T2-FLAIR MS TSE |  10000 |  120 |  2800 |  90 |  24 |  9.6 |  3:40
Unseen |  T1/T2w MS TSE |  2000 |  50 |  N/A |  70 |  14 |  6.7 |  3:00
contrasts |  TI400 MS TSE |  10000 |  120 |  400 |  90 |  24 |  9.6 |  3:40
|  DIR MS TSE |  10608 |  25 |  325/3400* |  90 |  17 |  6.8 |  5:39

Abbrevriations: GRE = gradient-echo; MS = multi-slice; DIR = double inversion
recovery; TSE = turbo spin-echo; TR = repetition time; TE = echo time; TI =
inversion time; FA = flip angle; ETL = echo train length; ESP = echo spacing;
TA = acquisition time. *Two inversion pulses are applied with different TIs

##  3 Methods

###  3.1 Data acquisition and processing

In this study, we collected data from forty patients (brain tumor (11), stroke
(10), epilepsy (10), and multiple sclerosis (9)) and ten volunteers (10)
included in an institutional review board-approved study to assess the
clinical usability of MR-STAT-based synthMRI [11]. Subjects were imaged on a 3
T scanner (Ingenia MR-RT, Philips, Best, The Netherlands) and 15-channel head
coil (HeadSpine, Philips, Best, The Netherlands). The acquired sequences
included MR-STAT [5, 34] and four standard neuro contrast acquisitions (PDw,
T1w, T2w, and T2-FLAIR), whose imaging protocols are summarized in Tab. 1. The
MR-STAT sequence is a five-minute transient-state multi-2D spoiled gradient-
echo with a slowly varying flip angle preceded by a non-selective 180∘
inversion pulse. Flip angle variations were subdivided into five acquisition
subsets, obtaining five separate k-spaces. Two stroke patients were excluded
due to severe motion artifacts in the standard contrast acquisitions. To
assess the feasibility of the physics-informed approach to synthesize unseen
contrasts, five additional volunteers were imaged with the MR-STAT, four
standard contrasts, and three different sequences with increasing deviation
from the standard contrasts to assess the framework flexibility (also
summarized in Tab. 1): (i) T12w as an intermediate contrast between the T1w
and T2w contrasts; (ii) TI400 as a T2w IR sequence like the T2-FLAIR contrast,
but with a different TI; (iii) a double IR (DIR) [35] as a completely new
sequence.

The 15-channel MR-STAT data (each capturing five k-spaces) were compressed
into a single virtual coil using singular value decomposition [36]. For pre-
processing purposes, q-maps were reconstructed using the MR-STAT framework
[5], from which “MR-STAT contrasts” (PDw, T1w, T2w, T2-FLAIR) were synthesized
using the signal model [10]. White matter (WM), gray matter (GM), and
cerebrospinal fluid (CSF) were segmented from the MR-STAT contrasts using the
predefined tissue probability maps in SPM12 (v7771) [37]. All other pre-
processing steps were performed using Python (v3.7.4) [38]. The conventional
contrasts were normalized by scaling the WM intensity mode to the average mode
among all subjects and intensities were clipped based on estimated values that
clipped most of the highest skull intensities. To simplify the contrast
synthesis starting from the image domain, the five complex k-spaces of the
virtual coil were fast Fourier transformed, creating five complex “fast
Fourier transform (FFT) images”. The magnitudes of the FFT images were min-max
scaled to [0, 1] per volume, and the real and imaginary components were used
as separate real-valued input channels. We compared using raw FFT as input
against using MR-STAT q-maps in a preliminary analysis reported in the
supplementary materials, demonstrating that starting from raw FFT is
beneficial for synthesis. Based on this study, only investigations using raw
FFT images as input are therefore reported. Finally, the conventional
contrasts were rigidly registered per slice to their respective MR-STAT
contrast that is intrinsically registered to the FFT images since these are
generated from the same k-spaces. The registration was performed with
SimpleElastix (v2.0.0rc2) [39], adopting an adaptive stochastic gradient
descent optimizer [40], the Mattes mutual information similarity metric [41],
and a recursive pyramid strategy with isotropic smoothing and down-sampling of
8, 4, 2, and 1 for the four resolutions.

Figure 2: Schematic representation of the synthMRI methods. The five complex
FFT images (x) were fed into the two architectures to output four contrasts
($\hat{\textbf{y}}$) directly (end-to-end) or via q*-maps followed by the
physics model M (physics-informed).

###  3.2 Deep learning synthesis

Two models were adopted: 1) end-to-end and 2) physics-informed (proposed).
GANs were employed for both methods due to their success in various medical
image translation tasks [29]. Specifically, conditional GANs (cGANs) [42] were
used, where either a synthesized or conventional contrast was concatenated to
the FFT images for conditioning [43]. The architectures of the generator and
discriminators of the end-to-end and physics-informed approaches were kept
similar (Fig. 2), consisting of a ResUNet generator [44] with three down-
sampling operations (33M and 13M parameters, respectively) and four contrast-
specific conditional PatchGAN discriminators [43] (2.8M parameters each). The
end-to-end approach mapped the FFT images to the four conventional contrasts
using a separate decoder (one output channel) for each contrast, similar to
the work by K. Wang et al. [30]. Contrarily, the physics-informed method
mapped the FFT images to the q*-maps using a single decode (three output
channels) from which contrasts were synthesized in a voxel-wise manner using
the signal model described in Eq. (1). The resulting signal value is min-max
scaled to [0, 1] per slice and used as the signal intensities. Synthetic T1
and T2 q*-maps are clipped to 5 and 1.5 s, respectively, after observing that
these values were never exceeded in the training population. All networks used
the Kaiming weight initialization [45] and rectified linear unit (ReLU)
activations [46], where the leaky variant [47] with slope 0.2 was adopted for
the discriminators. As described in (4), the content loss compares the
synthesized contrasts $\hat{y}$ to the conventional ones $y$, for each
contrast $i$.

$\mathcal{L}_{\textit{cont,
i}}=\lambda_{1}\|\hat{y}_{i}-y_{i}\|_{1}+\lambda_{2}\|\hat{y}_{i}-y_{i}\|_{2}-\lambda_{3}\cdot\operatorname{SSIM}(\hat{y}_{i},y_{i})+\lambda_{4}\left\|\left(\phi(\hat{y}_{i})-\phi(y_{i})\right)\right\|_{2}$
(4)

The voxel-wise terms comprised a weighted linear combination of L1 and L2
losses. A structural similarity metric (SSIM) [48] loss was added to
incorporate local structures and optimize perceptual quality. To further
optimize the perceptual similarity between the generated and conventional
contrasts, we also minimized the L2-norm of the features extracted from the
14th convolutional layer of a pre-trained VGG-19 net ($\phi$) [49]. Adding the
least-squares adversarial losses gives us the full objective functions for the
generator $\mathrm{G}$ and discriminators $\mathrm{D}_{i}$, described by

$\min_{G}\sum_{i=1}^{4}\Big{(}\gamma_{1}\mathbb{E}\big{[}(1-D_{i}(x,\hat{y}_{i}))^{2}\big{]}+\mathcal{L}_{\textit{cont,
i}}\Big{)}$ (5)

and

$\min_{D_{i}}\gamma_{2}\mathbb{E}\big{[}D_{i}(x,\hat{y}_{i})^{2}\big{]}+\gamma_{3}\mathbb{E}\big{[}(1-D_{i}(x,y_{i}))^{2}\big{]}$
(6)

respectively [50].

Five-fold cross-validation was performed, randomly splitting the 48 subjects
(30 slices each) [11] into 34, 5, and 9 subjects for training, validation, and
testing, respectively. All folds have an equal distribution of pathologies in
the training and test sets. Hyperparameter optimization was performed on a
single fold containing only healthy volunteers in the validation set. The
training was repeated for the four remaining folds (with 35 and 8 subjects in
the train and test set, respectively, for two of the folds) using these same
hyperparameters. Each subject is in the test set once, except for the five
held-out validation subjects, which were only used for hyperparameter
optimization, giving a total of 43 subjects in the test set.

The loss weights, types of augmentations, and batch size were manually
optimized for the physics-informed approach and were adopted for the end-to-
end approach. For this optimization, the SSIM and a sharpness estimation (the
variance of the image convolved with a 3x3 Laplacian kernel) [51] were used as
metrics, where SSIM was the primary metric considered when SSIM and sharpness
disagreed. The discriminators were updated twice per generator update and
weights [$\lambda_{1},\lambda_{2},\lambda_{3},\lambda_{4}$] and
[$\gamma_{1},\gamma_{2},\gamma_{3}$] were set to [1, 1, 4, 0.005] and [0.05,
0.025, 0.025]. During training, the images were augmented by applying flipping
and translation ($\pm 20$ voxels), both horizontally and vertically, and
rotation ($\pm 15^{\circ}$). In preliminary experiments, we observed that
unhealthy tissues were underrepresented compared to healthy tissue during
training. To counteract this imbalance, we doubled the chance that a slice
with pathology could occur. All methods were implemented in PyTorch v1.9.0
[52] and training with a batch size of 1 took about 38 and 26 hours for the
end-to-end and physics-informed approach, respectively, on an Nvidia Tesla
P100 GPU and Intel E5-2690v4 CPUs.

A cosine learning rate decay over 300 epochs, with a restart after 100 epochs
to prevent overfitting, was adopted for the generator and discriminators,
where the final epoch was always used for evaluation. The initial learning
rate values were determined using separate automatic Bayesian searches for the
physics-informed and end-to-end approaches. These optimizations, performed in
Weight&Biases [53], minimized the sum of the content losses as defined in Eq.
(4) for learning rates between 5e-5 and 1e-2 (logarithmic uniform
distribution) using ten searches. The optimal learning rate for the physics-
informed and end-to-end approaches were 1e-3 and 2.5e-3, respectively.

###  3.3 Experiments

####  3.3.1 Synthetic q*-maps

The values of test volunteer’s q*-maps were compared to literature values for
WM and GM [54, 55]. Due to the signal equation’s linear dependency on PD (see
Eq. (1)) and the applied min-max scaling for contrast synthesis (see Section
3.2), the PD q*-maps were not considered quantitative and were compared to
literature using the GM/WM ratio instead, to eliminate the effect of different
scaling factors.

####  3.3.2 Synthetic contrasts

To assess whether the physics-informed method is able to match the performance
of the end-to-end approach for the conventional contrasts, quantitative and
qualitative comparisons were performed. Firstly, the physics-informed method
was quantitatively compared to the end-to-end method in terms of the test
set’s SSIM and peak signal-to-noise ratio (PSNR) values. The lower and upper
five slices were left out for this and all upcoming experiments due to
increased noise in the conventional contrasts. A Shapiro-Wilk test [56] showed
that the image similarity metrics were not normally distributed. Therefore, a
non-parametric paired Wilcoxon signed-rank test [57] was used to compare the
physics-informed method to the end-to-end method (two-tailed with a
significance level of $p\leq 0.05$), where the mean SSIM and PSNR of each
subject were considered to satisfy the independent sample assumption. All
statistical analyses were performed using SciPy (v1.3.1) [58] and statsmodels
(v0.13.2) [59]. Secondly, a visual comparison is shown for a multiple
sclerosis patient to assess robustness of the methods synthesizing lesions of
various shapes and sizes.

A more detailed qualitative comparison was performed focusing on T2-FLAIR
contrasts, as these have been reported to be the most challenging contrasts
[22]: The occurrence of motion artifacts, CSF flow, checkerboard artifacts,
and unrealistic lesion suppression in T2-FLAIR contrasts were reported, as
first classified by a non-clinical intern and revised by an experienced MRI
physicist, together with example images of the observed artifacts. Finally,
the mean $\pm$ standard deviation inference times over all the test sets to
obtain synthMRI for a single subject were reported for both methods.

To assess the generalizability of the physics-informed approach to unseen
contrasts, contrasts from three additional sequences were synthesized (Tab. 1)
for five volunteers (see Tab. 1): (i) T12w, (ii) TI400, and (iii) DIR. The DIR
contrast was synthesized using the following signal model:

$\mathrm{S}=\mathrm{PD}\cdot
e^{-\mathrm{TE}/\mathrm{T}_{2}}\cdot\left(1-2e^{-\mathrm{TI}_{1}/\mathrm{T}_{1}}+2e^{-\mathrm{TI}_{2}/\mathrm{T}_{1}}-e^{-\mathrm{TD}/\mathrm{T}_{1}}\right)$
(7)

with $\mathrm{TI}_{1}$ the time between the first $180^{\circ}$ pulse and the
excitation pulse (suppressing CSF) and $\mathrm{TI}_{2}$ the time between the
second $180^{\circ}$ pulse and the excitation pulse (suppressing WM). The
synthetic contrasts were skull-stripped and visually compared to separately
acquired ground truth acquisitions. For quantitative image comparison, the
mean $\pm$ standard deviation contrast-to-noise ratios (CNRs) were compared
for five test volunteers, defined as the mean signal intensity difference
between two regions of interest divided by the image noise (estimated by
computing the standard deviation of intensities in a manually selected
homogeneous square patch of WM). The regions of interest were chosen to be GM-
WM for T12w, CSF-WM for TI400, and GM-WM for DIR.

Figure 3: Q*-maps for a volunteer. A transverse slice example of the q*-maps
for a volunteer. The GM/WM ratio is compared to Ref. [55] and the T1 and T2
values to the median value in Ref. [54]. The standard deviations of the PD
ratio and T1 and T2 values describe inter-subject and intra-subject
variability, respectively. The PD q*-maps and MR-STAT maps are normalized to
the 75th percentile of the CSF volume for visualization purposes. The superior
sagittal sinus is highlighted in the PD maps.

##  4 Results

###  4.1 Synthetic q*-maps

The PD GM/WM ratios of the q*-maps are close to the ratio reported in
literature [55] and present lower values where there are inflow effects, such
as in the superior sagittal sinus (Fig. 3). The T1 values in WM are close to
the reported values, whereas the values in GM differ substantially [54]. The
T2 values of WM and GM are underestimated compared to the literature values
[54].

Figure 4: Quantitative comparison of the end-to-end and physics-informed
approach. The SSIM and PSNR of the contrasts synthesized using the end-to-end
(blue) and physics-informed (red) approach are visualized using boxplots and
statistically compared. *($0.01<p\leq 0.05$) **($0.001<p\leq 0.01$) ***($p\leq
0.001$) n.s. = not significant ($p>0.05$).

Figure 5: SynthMRI on multiple sclerosis. Contrasts are shown for conventional
acquisition (top row), end-to-end (middle row), and physics-informed (bottom
row) approaches, where the right columns zooming in on patches of the T2-FLAIR
contrast with red and blue arrows highlighting missed and hypointense/blurred
lesions, respectively.

###  4.2 Synthetic contrasts

A quantitative comparison demonstrated that the physics-informed method
performed comparable to the end-to-end method for the PDw, T1w, T2w, and
T2-FLAIR contrasts in terms of SSIM and PSNR (Fig. 4). The biggest difference
in performance is in the T1w contrasts, having SSIM values of $0.88\pm 0.03$
and $0.85\pm 0.03$ and PSNR values of $25.5\pm 2.0$ and $24.0\pm 1.6$ for the
end-to-end and physics-informed approaches, respectively. The inference times
for all four contrasts averaged over all test subjects were $2.38\pm 0.03$ s
and $1.34\pm 0.02$ s for the end-to-end and physics-informed approaches,
respectively.

The end-to-end and physics-informed approaches do not display large visual
differences, for example, as demonstrated for a multiple sclerosis patient in
Fig. 5. The conventional and synthetic PDw and T1w images lack contrast
between the lesions and the surrounding healthy tissue. The CSF of the
physics-informed T1w contrast was consistently hyperintense compared to the
conventional contrast. Both synthetic T2w contrasts accurately capture almost
all lesions present in the conventional contrast, but for the T2-FLAIR
contrasts, some smaller lesions were missed (Fig. 5, red arrows) and
hypointense or blurred (Fig. 5, blue arrows).

Figure 6: Qualitative comparison of the conventional contrasts and synthMRI
methods focusing on T2-FLAIR only. In the left three columns, zoomed-in
structures of volunteers with a) motion artifacts in the conventional
contrast, b) flow artifacts in all contrasts, c) checkerboard artifacts in the
brainstem of the synthMRI contrasts, and d) blurry basal ganglia structures in
the synthMRI contrasts. In the right three columns, zoomed-in lesions are
shown of patients with e) epilepsy, f) multiple sclerosis, g) stroke, and h)
tumor.

Here, a more detailed analysis of the T2-FLAIR contrasts is provided, where
the artifact counts are determined using manual inspection of the images.
Motion artifacts were detected for 17 out of 43 test subjects (40%) in the
conventional contrasts, which were not observed for the synthMRI of any
subject (Fig. 6a). CSF flow artifacts are present for 43 subjects (100%) in
the synthMRI (Fig. 6b). Grid-like (chessboard) artifacts were introduced in
the brainstem for 23 (53%) and 4 (9%) subjects for the physics-informed and
end-to-end approach, respectively (Fig. 6c). Similar chessboard patterns were
observed in the parenchyma of 4 (9%) subjects for the physics-informed method.
Finally, synthMRI (especially the end-to-end approach) can appear blurrier
than the conventional contrasts, for example, the basal ganglia structures
(Fig. 6d). This finding has been quantitatively confirmed in a study reported
in the supplementary materials investigating image sharpness. Regarding
pathologies, both synthMRI methods sometimes miss or result in hypointense
lesions with a smaller volume, for example, in epilepsy or multiple sclerosis
patients (Fig. 6e-f). Also, more prominent, complicated pathologies such as a
stroke can result in inaccuracies. Although the gross shape is captured,
hypointensities and blurriness were observed (Fig. 6g). Finally, both synthMRI
methods were found to result in an unrealistic suppression of tumor contrast
for 4 out of 10 patients (40%) (Fig. 6h).

The standard end-to-end approach can only synthesize contrasts seen during
training. Contrarily, the experiment results suggest that the physics-informed
approach can synthesize any desired contrast from the q*-maps (Fig. 7).
Similar to the T1w contrast, the physics-informed synthetic T12w has a
hyperintense CSF compared to the conventional T12w contrast. For the physics-
informed synthetic TI400 and DIR contrasts, the signal suppression is almost
identical to the ground truth contrasts, although slight hyperintensities can
be observed in the WM of the synthetic DIR contrasts. The CNR values of the
synthetic T12w and TI400 contrasts were close to the values of their
respective ground truths: 2.65 $\pm$ 0.12 versus 2.24 $\pm$ 0.21 and 56.9
$\pm$ 11.4 versus 54.8 $\pm$ 7.96, respectively. The CNR values of the
synthetic DIR contrast were considerably lower than for its ground truth: 12.8
$\pm$ 3.31 versus 17.6 $\pm$ 1.79.

Figure 7: SynthMRI on a volunteer with unseen contrasts. Standard contrasts
seen during training (left) and unseen, skull-stripped contrasts (right) are
shown for the conventional contrasts and the end-to-end and physics-informed
approaches. Window leveling is kept identical among the same contrasts.

##  5 Discussion

This work investigated a physics-informed GAN-based framework synthesizing MRI
from a single five-minute acquisition, obtaining four standard neurological
sequences in about 2 seconds. For the PDw, T1w, T2w, and T2 FLAIR contrasts,
we showed that the physics-informed method closely resembles those of the end-
to-end approach, visually and in terms of SSIM and PSNR. For the T2-FLAIR
contrasts, both methods could result in hypointensities for smaller lesions
and more checkerboard artifacts appeared for the physics-informed approach.
Future work may focus on minimizing and finding the causes of these artifacts.
However, we demonstrated that the physics-informed method resulted in sharper
T2-FLAIR contrasts and facilitated synthetization of unseen contrasts, with
only minor visual deviations and comparable CNR values compared to the
conventional, separately acquired ground truth contrasts. The inferior CNR of
the synthetic DIR contrasts compared to its ground truth can be explained by
the suboptimal signal suppression leading to a higher WM tissue inhomogeneity
and thus a higher noise, lowering the CNR. Further work is required to
evaluate the clinical relevance of these trade-offs offered by the proposed
physics-informed approach.

This work is a continuation of our previous work, where the physics-informed
framework was originally proposed [60, 61]. In this work, the framework was
further developed and the evaluation was greatly expanded via cross-
validation, correlation of q*-maps with baseline q-maps, quantitative artifact
analysis, and a more detailed analysis of the generalizability. DL-based end-
to-end methods have been proposed to improve synthMRI quality by using MRF or
MDME acquisition data as input to a GAN and bypassing the oversimplified
signal model [30, 31]. K. Wang et al. [30] and G. Wang et al. [31] adopted a
single generator to synthesize all contrasts to exploit complementary
information from different contrasts, forming our end-to-end method. In terms
of network design, the single-branch and multi-branch U-net [24] architectures
from K. Wang et al. [30] were adapted for the physics-informed and end-to-end
approaches, respectively, where we implemented residual units to simplify
training [44]. The real and imaginary components of the FFT images were used
as separate real-valued input channels. Alternatively, complex-valued networks
could be explored to accurately represent the magnitude and phase to improve
the network’s performance [62]. The translations performed by the generators
were kept within the image domain, similar to K. Wang et al. [30] and G. Wang
et al. [31], preventing the need for fully-connected layers in the generator
to estimate the Fourier transform, which is memory intensive and require more
data to avoid overfitting [63]. Similar to our proposed framework, Pal et al.
[64] proposed to use a deep image prior to denoise three SE contrasts and fit
a signal model to extract q-maps from the denoised contrasts, facilitating
personalized synthMRI by omitting the need for an external training dataset.
However, the high inference time is undesirable in a clinical setting and the
methods were trained and evaluated using exclusively synthetic data. Denck et
al. [65] proposed the first GAN to allow retrospective contrast adjustments to
synthesize missing or corrupted knee MRI contrasts from existing contrasts,
where MR physics was incorporated implicitly via style transfer. Our proposed
method obtains this feature by explicitly modeling MR physics, similar to the
physical priors used by Moya-Sáez et al. [66, 67]. Ref. [66] was trained using
exclusively synthetic data and Ref. [67] was finetuned using a relatively
small and homogeneous in-vivo dataset, whereas we used a large cohort of in-
vivo data containing heterogeneous pathologies. Furthermore, we pushed the
generalizability of our method to contrasts which have not been observed by
the network. This is the first GAN-based synthMRI framework allowing
retrospective contrast adjustments from a single acquisition.

The previously proposed MDME-based and MRF-based synthMRI frameworks suffer
from sub-optimal T2-FLAIR contrasts compared to conventional acquisitions,
generally attributed to an oversimplified signal model [16, 20, 21]. Signal
model extensions incorporating partial volume and flow effects have shown
promising results to improve the T2-FLAIR contrasts for MRF-based synthMRI
[68, 69]. Partial volume and magnetization transfer modeling could be explored
for our physics-informed method. We showed that our data-driven approach
already resolved flow effects without requiring explicit modeling, for
example, in the superior sagittal sinus. Grid-like structures in the
parenchyma were reported by G. Wang et al. [31] when adding adversarial and
perceptual losses. Future work should investigate the influence of these loss
terms on our observed checkerboard artifacts. Although GANs may introduce
hallucinations [70], these were not observed in our methods.

Considering that the adopted similarity metrics have been suggested not to be
ideal surrogate measures of MRI quality as determined by radiologist
evaluation [71], a large-scale clinical study like the one performed by
Kleinloog et al. [11] should be initiated to validate the proposed method
further. Although the dataset was relatively small, a heterogeneous
pathological cohort has been considered, exposing the networks to a diverse
distribution of lesions, making the dataset representative of the real-world
data that the network would infer in the clinic and improving generalization
performance [33].

Although the MR-STAT acquisition is used as input for our synthMRI framework,
substituting a different acquisition that encodes sufficient information to
characterize the tissue parameters (such as MRF or MDME data) would be
conceptually straightforward, making the proposed framework independent of MR-
STAT and generally applicable to any acquisition. Furthermore, the data were
acquired on a single scanner, so inter-scanner variability should be
investigated before clinical use. Currently, the q*-maps are used to
facilitate synthetization of unseen contrasts. In future work, we plan to
investigate the q*-maps’ diagnostic application as a fast, surrogate qMRI
method to standardize MRI-based measurements, reduce bias, and increase
reproducibility [3].

Acquiring a single, five-minute acquisition and synthesizing the contrasts
reduces scan time compared to the conventional, separate contrast
acquisitions. This way, synthMRI reduces examination costs, increases patient
throughput, and makes MRI a more accessible imaging modality. The proposed
incorporation of the signal model provides added interpretability, which is
important for clinical adaptation of deep learning-based methods [33].
Furthermore, the provided retrospective contrast adjustment gives imaging
specialists the flexibility to select the desired post-scan contrasts, which
may decrease the number of patient recalls [72], possibly resulting in
improved diagnostics. For example, it has been shown that synthesizing DIR
contrasts may improve the detection of multiple sclerosis plaques [16].
Additionally, a shorter scan time is advantageous for pediatric brain imaging
[73], potentially decreasing the use of general anesthesia and preventing
potential long-term complications in the developing brain of children [72,
74].

##  6 Conclusion

We demonstrated that the proposed physics-informed method synthesizes high-
quality standard contrasts from a single full-brain five-minute acquisition.
Also, we proved the feasibility of generating synthMRI of unseen sequences
during training. The proposed method is able to match the quality of PDw, T1w,
T2w, and T2 FLAIR contrasts of an end-to-end framework but provides additional
flexibility in synthesizing additional contrasts.

## Acknowledgments

We want to thank Sarah Jacobs for the clinical feedback. This work would not
have been possible without the involvement of Anja van der Kolk, Beyza Koktas,
and Sarah Jacobs, who contributed to patient selection and inclusion. We also
gratefully acknowledge the support of NVIDIA Corporation with the donation of
the Quadro RTX 5000 GPU used for the prototyping part of this research.

### Author contributions

L.J. sketched the idea, implemented the methods, and performed the analyses.
M.M. participated in developing the idea. S.M., H.L., and M.M. revised the
methods and analyses. C.B. and A.S. supervised and facilitated the work. All
authors discussed the results and contributed to the writing of the
manuscript.

### Financial disclosure

None reported.

### Conflict of interest

The authors declare no potential conflict of interests.

## References

  * [1] Langen, K.J., Galldiks, N., Hattingen, E., and Shah, N.J. (2017), Advances in neuro-oncology imaging, doi:10.1038/nrneurol.2017.44. URL https://www.nature.com/articles/nrneurol.2017.44.
  * [2] Cashmore, M.T., McCann, A.J., Wastling, S.J., McGrath, C., Thornton, J., and Hall, M.G. (2021), Clinical quantitative MRI and the need for metrology, doi:10.1259/bjr.20201215. URL https://www.birpublications.org/doi/10.1259/bjr.20201215.
  * [3] Margaret Cheng, H.L., Stikov, N., Ghugre, N.R., and Wright, G.A. (2012), Practical medical applications of quantitative MR relaxometry, doi:10.1002/jmri.23718. URL www.wileyblackwellcme.com.
  * [4] Ma, D., Gulani, V., Seiberlich, N., Liu, K., Sunshine, J.L., Duerk, J.L., and Griswold, M.A. (2013) Magnetic resonance fingerprinting. _Nature_ , 495 (7440), 187–192, doi:10.1038/nature11971. URL https://www.nature.com/articles/nature11971.
  * [5] Sbrizzi, A., Heide, O.v.d., Cloos, M., Toorn, A.v.d., Hoogduin, H., Luijten, P.R., and van den Berg, C.A. (2018) Fast quantitative MRI as a nonlinear tomography problem. _Magnetic Resonance Imaging_ , 46, 56–63, doi:10.1016/j.mri.2017.10.015.
  * [6] Warntjes, J., Leinhard, O.D., West, J., and Lundberg, P. (2008) Rapid magnetic resonance quantification on the brain: Optimization for clinical usage. _Magnetic Resonance in Medicine_ , 60 (2), 320–329, doi:10.1002/mrm.21635. URL http://doi.wiley.com/10.1002/mrm.21635.
  * [7] Ji, S., Yang, D., Lee, J., Choi, S.H., Kim, H., and Kang, K.M. (2020), Synthetic MRI: Technologies and Applications in Neuroradiology, doi:10.1002/jmri.27440. URL https://onlinelibrary.wiley.com/doi/10.1002/jmri.27440.
  * [8] Jiang, Y., Ma, D., Seiberlich, N., Gulani, V., and Griswold, M.A. (2015) MR fingerprinting using fast imaging with steady state precession (FISP) with spiral readout. _Magnetic Resonance in Medicine_ , 74 (6), 1621–1631, doi:10.1002/mrm.25559.
  * [9] Mehta, B.B., Ma, D., Pierre, E.Y., Jiang, Y., Coppo, S., and Griswold, M.A. (2018) Image reconstruction algorithm for motion insensitive MR Fingerprinting (MRF): MORF. _Magnetic Resonance in Medicine_ , 80 (6), 2485–2500, doi:10.1002/mrm.27227. URL http://doi.wiley.com/10.1002/mrm.27227.
  * [10] Mandija, S., D’Agata, F., Liu, H., van der Heide, O., Koktas, B., van den berg, C.A., Hendrikse, J., van der Kolk, A., and Sbrizzi, A. (2020) A five-minute multi-parametric high-resolution whole-brain MR-STAT exam: first results from a clinical trial. _proceedings of ISMRM_.
  * [11] Kleinloog, J.P.D., Mandija, S., D’Agata, F., Liu, H., Heide, O.v.d., Koktas, B., Jacobs, S.M., Berg, C.A.T.v.d., Hendrikse, J., Kolk, A.G.v.d., and Sbrizzi, A. (2022) Synthetic MRI with Magnetic Resonance Spin TomogrAphy in Time-Domain (MR-STAT): Results from a Prospective Cross-Sectional Clinical Trial. _Journal of Magnetic Resonance Imaging_ , doi:10.1002/JMRI.28425. URL https://onlinelibrary.wiley.com/doi/full/10.1002/jmri.28425https://onlinelibrary.wiley.com/doi/abs/10.1002/jmri.28425https://onlinelibrary.wiley.com/doi/10.1002/jmri.28425.
  * [12] Ryu, K.H., Baek, H.J., Moon, J.I., Choi, B.H., Park, S.E., Ha, J.Y., Jeon, K.N., Bae, K., Choi, D.S., Cho, S.B., Lee, Y., and Heo, Y.J. (2020) Initial clinical experience of synthetic MRI as a routine neuroimaging protocol in daily practice: A single-center study. _Journal of Neuroradiology_ , 47 (2), 151–160, doi:10.1016/j.neurad.2019.03.002. URL https://doi.org/10.1016/j.neurad.2019.03.002.
  * [13] Blystad, I., Marcel Warntjes, J.B., Smedby, O., Lundberg, P., Larsson, E.M., and Tisell, A. (2017) Quantitative MRI for analysis of peritumoral edema in malignant gliomas. _PLoS ONE_ , 12 (5), e0177 135, doi:10.1371/journal.pone.0177135. URL https://journals.plos.org/plosone/article?id=10.1371/journal.pone.0177135.
  * [14] Hagiwara, A., Hori, M., Suzuki, M., Andica, C., Nakazawa, M., Tsuruta, K., Takano, N., Sato, S., Hamasaki, N., Yoshida, M., Kumamaru, K.K., Ohtomo, K., and Aoki, S. (2016) Contrast-enhanced synthetic MRI for the detection of brain metastases. _Acta Radiologica Open_ , 5 (2), 205846011562 675, doi:10.1177/2058460115626757. URL https://journals.sagepub.com/doi/full/10.1177/2058460115626757.
  * [15] Andica, C., Hagiwara, A., Nakazawa, M., Tsuruta, K., Takano, N., Hori, M., Suzuki, H., Sugano, H., Arai, H., and Aoki, S. (2016) The advantage of synthetic MRI for the visualization of early white matter change in an infant with Sturge-Weber syndrome. _Magnetic Resonance in Medical Sciences_ , pp. ci–2015.
  * [16] Hagiwara, A., Hori, M., Yokoyama, K., Takemura, M.Y., Andica, C., Tabata, T., Kamagata, K., Suzuki, M., Kumamaru, K.K., Nakazawa, M., Takano, N., Kawasaki, H., Hamasaki, N., Kunimatsu, A., and Aoki, S. (2017) Synthetic MRI in the detection of multiple sclerosis plaques. _American Journal of Neuroradiology_ , 38 (2), 257–263, doi:10.3174/ajnr.A5012. URL http://dx.doi.org/10.3174/ajnr.A5012.
  * [17] André, J., Barrit, S., and Jissendi, P. (2022) Synthetic MRI for stroke: a qualitative and quantitative pilot study. _Scientific reports_ , 12 (1), 11 552, doi:10.1038/s41598-022-15204-8. URL /pmc/articles/PMC9262877//pmc/articles/PMC9262877/?report=abstracthttps://www.ncbi.nlm.nih.gov/pmc/articles/PMC9262877/http://www.ncbi.nlm.nih.gov/pubmed/35798771%0Ahttp://www.pubmedcentral.nih.gov/articlerender.fcgi?artid=PMC9262877.
  * [18] Gulani, V., Schmitt, P., Griswold, M.A., Webb, A.G., and Jakob, P.M. (2004) Towards a single-sequence neurologic magnetic resonance imaging examination: multiple-contrast images from an IR TrueFISP experiment. _Investigative radiology_ , 39 (12), 767–774.
  * [19] Redpath, T.W., Smith, F.W., and Hutchison, J.M. (2014) Magnetic resonance image synthesis from an interleaved saturation recovery/inversion recovery sequence. _http://dx.doi.org/10.1259/0007-1285-61-727-619_ , 61 (727), 619–624, doi:10.1259/0007-1285-61-727-619. URL https://www.birpublications.org/doi/abs/10.1259/0007-1285-61-727-619.
  * [20] Granberg, T., Uppman, M., Hashim, F., Cananau, C., Nordin, L.E., Shams, S., Berglund, J., Forslin, Y., Aspelin, P., Fredrikson, S., and Kristoffersen-Wiberg, M. (2016) Clinical feasibility of synthetic MRI in multiple sclerosis: A diagnostic and volumetric validation study. _American Journal of Neuroradiology_ , 37 (6), 1023–1029, doi:10.3174/ajnr.A4665. URL http://dx.doi.org/10.3174/ajnr.A4665.
  * [21] Hagiwara, A., Warntjes, M., Hori, M., Andica, C., Nakazawa, M., Kumamaru, K.K., Abe, O., and Aoki, S. (2017), SyMRI of the Brain: Rapid Quantification of Relaxation Rates and Proton Density, with Synthetic MRI, Automatic Brain Segmentation, and Myelin Measurement, doi:10.1097/RLI.0000000000000365. URL /pmc/articles/PMC5596834//pmc/articles/PMC5596834/?report=abstracthttps://www.ncbi.nlm.nih.gov/pmc/articles/PMC5596834/.
  * [22] Tanenbaum, L.N., Tsiouris, A.J., Johnson, A.N., Naidich, T.P., DeLano, M.C., Melhem, E.R., Quarterman, P., Parameswaran, S.X., Shankaranarayanan, A., Goyen, M., and Field, A.S. (2017) Synthetic MRI for clinical neuroimaging: Results of the magnetic resonance image compilation (MAGiC) prospective, multicenter, multireader trial. _American Journal of Neuroradiology_ , 38 (6), 1103–1110, doi:10.3174/ajnr.A5227. URL http://dx.doi.org/10.3174/ajnr.A5227.
  * [23] Lecun, Y., Bengio, Y., and Hinton, G. (2015), Deep learning, doi:10.1038/nature14539. URL https://www.nature.com/articles/nature14539.
  * [24] Ronneberger, O., Fischer, P., and Brox, T. (2015) U-net: Convolutional networks for biomedical image segmentation. _Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics)_ , 9351, 234–241, doi:10.1007/978-3-319-24574-4{\\_}28. URL http://lmb.informatik.uni-freiburg.de/http://lmb.informatik.uni-freiburg.de/people/ronneber/u-net.
  * [25] Wang, S., Su, Z., Ying, L., Peng, X., Zhu, S., Liang, F., Feng, D., and Liang, D. (2016) Accelerating magnetic resonance imaging via deep learning. _Proceedings - International Symposium on Biomedical Imaging_ , 2016-June, 514–517, doi:10.1109/ISBI.2016.7493320.
  * [26] Chaudhari, A.S., Fang, Z., Kogan, F., Wood, J., Stevens, K.J., Gibbons, E.K., Lee, J.H., Gold, G.E., and Hargreaves, B.A. (2018) Super-resolution musculoskeletal MRI using deep learning. _Magnetic Resonance in Medicine_ , 80 (5), 2139–2154, doi:10.1002/mrm.27178. URL https://onlinelibrary.wiley.com/doi/full/10.1002/mrm.27178https://onlinelibrary.wiley.com/doi/abs/10.1002/mrm.27178https://onlinelibrary.wiley.com/doi/10.1002/mrm.27178.
  * [27] Wolterink, J.M., Dinkla, A.M., Savenije, M.H., Seevinck, P.R., van den Berg, C.A., and Išgum, I. (2017) Deep MR to CT synthesis using unpaired data. _Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics)_ , 10557 LNCS, 14–23, doi:10.1007/978-3-319-68127-6{\\_}2. URL https://link.springer.com/chapter/10.1007/978-3-319-68127-6_2.
  * [28] Goodfellow, I., Pouget-Abadie, J., Mirza, M., Xu, B., Warde-Farley, D., Ozair, S., Courville, A., and Bengio, Y. (2014) Generative adversarial nets. _Advances in neural information processing systems_ , 27.
  * [29] Yi, X., Walia, E., and Babyn, P. (2019) Generative adversarial network in medical imaging: A review. _Medical Image Analysis_ , 58, 101 552, doi:10.1016/j.media.2019.101552.
  * [30] Wang, K., Doneva, M., Amthor, T., Keil, V.C., Karasan, E., Tan, F., Tamir, J.I., Yu, S.X., and Lustig, M. (2020) High Fidelity Direct-Contrast Synthesis from Magnetic Resonance Fingerprinting in Diagnostic Imaging. _Proceedings of the 28th Annual Meeting of ISMRM_ , 867.
  * [31] Wang, G., Gong, E., Banerjee, S., Martin, D., Tong, E., Choi, J., Chen, H., Wintermark, M., Pauly, J.M., and Zaharchuk, G. (2020) Synthesize High-Quality Multi-Contrast Magnetic Resonance Imaging From Multi-Echo Acquisition Using Multi-Task Deep Generative Model. _IEEE transactions on medical imaging_ , 39 (10), 3089–3099, doi:10.1109/TMI.2020.2987026.
  * [32] Meara, S.J. and Barker, G.J. (2005) Evolution of the longitudinal magnetization for pulse sequences using a fast spin-echo readout: Application to fluid-attenuated inversion-recovery and double inversion-recovery sequences. _Magnetic Resonance in Medicine_ , 54 (1), 241–245, doi:10.1002/mrm.20541.
  * [33] Lundervold, A.S. and Lundervold, A. (2019), An overview of deep learning in medical imaging focusing on MRI, doi:10.1016/j.zemedi.2018.11.002.
  * [34] van der Heide, O., Sbrizzi, A., Luijten, P.R., and van den Berg, C.A. (2020) High-resolution in vivo MR-STAT using a matrix-free and parallelized reconstruction algorithm. _NMR in Biomedicine_ , 33 (4), doi:10.1002/nbm.4251.
  * [35] Turetschek, K., Wunderbaldinger, P., Bankier, A.A., Zontsich, T., Graf, O., Mallek, R., and Hittmair, K. (1998) Double inversion recovery imaging of the brain: initial experience and comparison with fluid attenuated inversion recovery imaging. _Magnetic resonance imaging_ , 16 (2), 127–135.
  * [36] Golub, G.H. and Van Loan, C.F. (2013) _Matrix computations_ , JHU press.
  * [37] Penny, W.D., Friston, K.J., Ashburner, J.T., Kiebel, S.J., and Nichols, T.E. (2011) _Statistical parametric mapping: the analysis of functional brain images_ , Elsevier.
  * [38] Van Rossum, G. and Drake, F.L. (2009) _Python 3 Reference Manual_ , CreateSpace, Scotts Valley, CA.
  * [39] Marstal, K., Berendsen, F., Staring, M., and Klein, S. (2016) SimpleElastix: A User-Friendly, Multi-Lingual Library for Medical Image Registration. _Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition (CVPR) Workshops_.
  * [40] Klein, S., Pluim, J.P., Staring, M., and Viergever, M.A. (2009) Adaptive stochastic gradient descent optimisation for image registration. _International Journal of Computer Vision_ , 81 (3), 227–239, doi:10.1007/s11263-008-0168-y. URL https://link.springer.com/article/10.1007/s11263-008-0168-y.
  * [41] Mattes, D., Haynor, D.R., Vesselle, H., Lewellyn, T.K., and Eubank, W. (2001) Nonrigid multimodality image registration. _Medical Imaging 2001: Image Processing_ , 4322, 1609–1620, doi:10.1117/12.431046. URL https://www.spiedigitallibrary.org/conference-proceedings-of-spie/4322/0000/Nonrigid-multimodality-image-registration/10.1117/12.431046.fullhttps://www.spiedigitallibrary.org/conference-proceedings-of-spie/4322/0000/Nonrigid-multimodality-image-registrat.
  * [42] Mirza, M. and Osindero, S. (2014) Conditional generative adversarial nets. _arXiv preprint arXiv:1411.1784_.
  * [43] Isola, P., Zhu, J.Y., Zhou, T., and Efros, A.A. (2017) Image-to-image translation with conditional adversarial networks. _Proceedings - 30th IEEE Conference on Computer Vision and Pattern Recognition, CVPR 2017_ , 2017-Janua, 5967–5976, doi:10.1109/CVPR.2017.632. URL https://github.com/phillipi/pix2pix.
  * [44] Zhang, Z., Liu, Q., and Wang, Y. (2018) Road Extraction by Deep Residual U-Net. _IEEE Geoscience and Remote Sensing Letters_ , 15 (5), 749–753, doi:10.1109/LGRS.2018.2802944. URL https://www.cs.toronto.edu/.
  * [45] He, K., Zhang, X., Ren, S., and Sun, J. (2015) Delving deep into rectifiers: Surpassing human-level performance on imagenet classification. _Proceedings of the IEEE International Conference on Computer Vision_ , 2015 Inter, 1026–1034, doi:10.1109/ICCV.2015.123.
  * [46] Nair, V. and Hinton, G.E. (2010) Rectified linear units improve Restricted Boltzmann machines. _ICML 2010 - Proceedings, 27th International Conference on Machine Learning_ , pp. 807–814.
  * [47] Maas, A.L., Hannun, A.Y., Ng, A.Y., and others (2013) Rectifier nonlinearities improve neural network acoustic models. _Proc. icml_ , 30 (1), 3.
  * [48] Wang, Z., Bovik, A.C., Sheikh, H.R., and Simoncelli, E.P. (2004) Image quality assessment: From error visibility to structural similarity. _IEEE Transactions on Image Processing_ , 13 (4), 600–612, doi:10.1109/TIP.2003.819861.
  * [49] Simonyan, K. and Zisserman, A. (2015) Very deep convolutional networks for large-scale image recognition. _3rd International Conference on Learning Representations, ICLR 2015 - Conference Track Proceedings_.
  * [50] Mao, X., Li, Q., Xie, H., Lau, R.Y., Wang, Z., and Smolley, S.P. (2017) Least Squares Generative Adversarial Networks. _Proceedings of the IEEE International Conference on Computer Vision_ , 2017-Octob, 2813–2821, doi:10.1109/ICCV.2017.304.
  * [51] Pech-Pacheco, J.L., Cristöbal, G., Chamorro-Martínez, J., and Fernândez-Valdivia, J. (2000) Diatom autofocusing in brightfield microscopy: A comparative study. _Proceedings - International Conference on Pattern Recognition_ , 15 (3), 314–317, doi:10.1109/ICPR.2000.903548.
  * [52] Paszke, A., Gross, S., Massa, F., Lerer, A., Bradbury, J., Chanan, G., Killeen, T., Lin, Z., Gimelshein, N., Antiga, L., Desmaison, A., Kopf, A., Yang, E., DeVito, Z., Raison, M., Tejani, A., Chilamkurthy, S., Steiner, B., Fang, L., Bai, J., and Chintala, S. (2019) PyTorch: An Imperative Style, High-Performance Deep Learning Library. _Advances in Neural Information Processing Systems_ , 32.
  * [53] Biewald, L. (2020), Experiment Tracking with Weights and Biases. URL https://www.wandb.com/.
  * [54] Bojorquez, J.Z., Bricq, S., Acquitter, C., Brunotte, F., Walker, P.M., and Lalande, A. (2017), What are normal relaxation times of tissues at 3 T?, doi:10.1016/j.mri.2016.08.021. URL http://dx.doi.org/10.1016/j.mri.2016.08.021.
  * [55] Hagiwara, A., Hori, M., Cohen-Adad, J., Nakazawa, M., Suzuki, Y., Kasahara, A., Horita, M., Haruyama, T., Andica, C., Maekawa, T., Kamagata, K., Kumamaru, K.K., Abe, O., and Aoki, S. (2019) Linearity, Bias, Intrascanner Repeatability, and Interscanner Reproducibility of Quantitative Multidynamic Multiecho Sequence for Rapid Simultaneous Relaxometry at 3 T: A Validation Study with a Standardized Phantom and Healthy Controls. _Investigative Radiology_ , 54 (1), 39–47, doi:10.1097/RLI.0000000000000510. URL https://journals.lww.com/investigativeradiology/Fulltext/2019/01000/Linearity,_Bias,_Intrascanner_Repeatability,_and.6.aspx.
  * [56] Shapiro, S.S. and Wilk, M.B. (1965) An Analysis of Variance Test for Normality (Complete Samples). _Biometrika_ , 52 (3/4), 591, doi:10.2307/2333709. URL https://www.jstor.org/stable/2333709?origin=crossref.
  * [57] Wilcoxon, F. (1945) Individual Comparisons by Ranking Methods. _Biometrics Bulletin_ , 1 (6), 80, doi:10.2307/3001968.
  * [58] Virtanen, P., Gommers, R., Oliphant, T.E., Haberland, M., Reddy, T., Cournapeau, D., Burovski, E., Peterson, P., Weckesser, W., Bright, J., and others (2020) SciPy 1.0: fundamental algorithms for scientific computing in Python. _Nature methods_ , 17 (3), 261–272.
  * [59] Seabold, S. and Perktold, J. (2010) Statsmodels: Econometric and statistical modeling with python. _Proceedings of the 9th Python in Science Conference_ , 57, 61.
  * [60] Jacobs, L., Mandija, S., Liu, H., Sbrizzi, A., van den berg, C.A., and Maspero, M. (2022) Generalizable synthetic multi-contrast MRI generation using physics-informed convolutional networks. _proceedings of ISMRM_.
  * [61] Jacobs, L. (2022) Generalizable synthetic multi-contrast MRI using physics-informed convolutional networks. _[Unpublished master’s thesis, Technical University Eindhoven]_.
  * [62] Cole, E., Cheng, J., Pauly, J., and Vasanawala, S. (2021) Analysis of deep complex-valued convolutional neural networks for MRI reconstruction and phase-focused applications. _Magnetic Resonance in Medicine_ , 86 (2), 1093–1109, doi:10.1002/mrm.28733. URL https://onlinelibrary.wiley.com/doi/full/10.1002/mrm.28733https://onlinelibrary.wiley.com/doi/abs/10.1002/mrm.28733https://onlinelibrary.wiley.com/doi/10.1002/mrm.28733.
  * [63] Eo, T., Shin, H., Jun, Y., Kim, T., and Hwang, D. (2020) Accelerating Cartesian MRI by domain-transform manifold learning in phase-encoding direction. _Medical Image Analysis_ , 63, 101 689, doi:10.1016/J.MEDIA.2020.101689.
  * [64] Pal, S., Dutta, S., and Maitra, R. (2022) Personalized synthetic mr imaging with deep learning enhancements. _Magnetic Resonance in Medicine_.
  * [65] Denck, J., Guehring, J., Maier, A., and Rothgang, E. (2021) MR-contrast-aware image-to-image translations with generative adversarial networks. _International Journal of Computer Assisted Radiology and Surgery_ , pp. 1–10, doi:10.1007/s11548-021-02433-x. URL https://link.springer.com/article/10.1007/s11548-021-02433-x.
  * [66] Moya-Sáez, E., Peña-Nogales, O., Luis-García, R.d., and Alberola-López, C. (2021) A deep learning approach for synthetic MRI based on two routine sequences and training with synthetic data. _Computer Methods and Programs in Biomedicine_ , 210, 106 371, doi:10.1016/j.cmpb.2021.106371.
  * [67] Moya-Sáez, E., Luis-García, R.d., and Alberola-López, C. (2021) A self-supervised deep learning approach to synthesize weighted images and T1, T2, and PD parametric maps based on MR physics priors. _proceedings of ISMRM_.
  * [68] Cencini, M., Buonincontri, G., Biagi, L., Gómez, P.A., Schulte, R.F., and Tosetti, M. (2019) Chasing True FLAIR: a three-component Magnetic Resonance Fingerprinting approach to synthetic MRI. _Proceedings of the 27th Annual Meeting of ISMRM_ , p. 816.
  * [69] Deshmane, A., McGivney, D., Badve, C., Yu, A., Jiang, Y., Ma, D., and Griswold, M.A. (2016) Accurate synthetic FLAIR images using partial volume corrected MR fingerprinting. _Proceedings of the Annual Meeting and Exhibition of International Society for Magnetic Resonance in Medicine_ , pp. 7–13.
  * [70] Cohen, J.P., Luck, M., and Honari, S. (2018) Distribution matching losses can hallucinate features in medical image translation. _Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics)_ , 11070 LNCS, 529–536, doi:10.1007/978-3-030-00928-1{\\_}60. URL https://doi.org/10.1007/978-3-030-00928-1_60.
  * [71] Mason, A., Rioux, J., Clarke, S.E., Costa, A., Schmidt, M., Keough, V., Huynh, T., and Beyea, S. (2020) Comparison of Objective Image Quality Metrics to Expert Radiologists’ Scoring of Diagnostic Quality of MR Images. _IEEE Transactions on Medical Imaging_ , 39 (4), 1064–1072, doi:10.1109/TMI.2019.2930338. URL https://pubmed.ncbi.nlm.nih.gov/31535985/.
  * [72] Gonçalves, F.G., Serai, S.D., and Zuccoli, G. (2018), Synthetic brain MRI: Review of current concepts and future directions, doi:10.1097/RMR.0000000000000189. URL https://journals.lww.com/topicsinmri/Fulltext/2018/12000/Synthetic_Brain_MRI__Review_of_Current_Concepts.1.aspx.
  * [73] West, H., Leach, J.L., Jones, B.V., Care, M., Radhakrishnan, R., Merrow, A.C., Alvarado, E., and Serai, S.D. (2017) Clinical validation of synthetic brain MRI in children: initial experience. _Neuroradiology_ , 59 (1), 43–50, doi:10.1007/s00234-016-1765-z. URL https://link.springer.com/article/10.1007/s00234-016-1765-z.
  * [74] Ing, C., Wall, M.M., DiMaggio, C.J., Whitehouse, A.J., Hegarty, M.K., Sun, M., Von Ungern-Sternberg, B.S., Li, G., and Sun, L.S. (2017) Latent class analysis of neurodevelopmental deficit after exposure to anesthesia in early childhood. _Journal of Neurosurgical Anesthesiology_ , 29 (3), 264–273, doi:10.1097/ANA.0000000000000303. URL https://pubmed.ncbi.nlm.nih.gov/27077892/.