2024 Full Paper – MIDL 2024 – 272 Wentian XuContributed equally1
<EMAIL_ADDRESS>
Matthew Moffat1<EMAIL_ADDRESS>
Thalia Seale1<EMAIL_ADDRESS>
Ziyun Liang1<EMAIL_ADDRESS>
Felix Wagner1<EMAIL_ADDRESS>
Daniel Whitehouse2<EMAIL_ADDRESS>
David Menon2<EMAIL_ADDRESS>
Virginia Newcombe2<EMAIL_ADDRESS>
Natalie Voets3<EMAIL_ADDRESS>
Abhirup Banerjee1<EMAIL_ADDRESS>
Konstantinos Kamnitsas1<EMAIL_ADDRESS>
1 Department of Engineering Science, University of Oxford, Oxford, United
Kingdom
2 Department of Medicine, University of Cambridge, Cambridge, United Kingdom
3 Nuffield Department of Clinical Neurosciences, University of Oxford, Oxford,
United Kingdom

# Feasibility and benefits of joint learning from MRI databases with different
brain diseases and modalities for segmentation

###### Abstract

Models for segmentation of brain lesions in multi-modal MRI are commonly
trained for a specific pathology using a single database with a predefined set
of MRI modalities, determined by a protocol for the specific disease. This
work explores the following open questions: Is it feasible to train a model
using multiple databases that contain varying sets of MRI modalities and
annotations for different brain pathologies? Will this joint learning benefit
performance on the sets of modalities and pathologies available during
training? Will it enable analysis of new databases with different sets of
modalities and pathologies? We develop and compare different methods and show
that promising results can be achieved with appropriate, simple and practical
alterations to the model and training framework. We experiment with 7
databases containing 5 types of brain pathologies and different sets of MRI
modalities. Results demonstrate, for the first time, that joint training on
multi-modal MRI databases with different brain pathologies and sets of
modalities is feasible and offers practical benefits. It enables a single
model to segment pathologies encountered during training in diverse sets of
modalities, while facilitating segmentation of new types of pathologies such
as via follow-up fine-tuning. The insights this study provides into the
potential and limitations of this paradigm should prove useful for guiding
future advances in the direction. Code and pretrained models:
https://github.com/WenTXuL/MultiUnet

††editors: Accepted for publication at MIDL 2024

##  1 Introduction

Segmentation of pathologies in Magnetic Resonance Imaging (MRI) of the brain
plays a crucial role in disease diagnosis and treatment. Deep learning has
marked a significant leap in medical image segmentation. Existing works for
segmentation of brain lesions in MRI [Kamnitsas et al.(2017)Kamnitsas, Ledig,
Newcombe, Simpson, Kane, Menon, Rueckert, and Glocker, Isensee et
al.(2021)Isensee, Jaeger, Kohl, Petersen, and Maier-Hein, Chen et
al.(2019b)Chen, Liu, Peng, Sun, and Qiao], however, are commonly trained to
segment a specific pathology, using a database acquired with a predefined set
of imaging modalities determined by a protocol for the specific disease. One
database may contain several MRI modalities, but the standard methods require
the same set of modalities available during training and inference. Therefore
databases with diverse sets of modalities are not jointly leveraged.
Performance of a model trained on a single database is heavily influenced by
the database’s size. Instead, learning from multiple databases could enhance
generalization, while training using varying modality sets and pathologies
could enable segmenting different sets of modalities and pathologies with a
single model. Thus, exploring how to train models using multiple databases
with heterogeneous modalities is a promising avenue. Towards this goal, this
study investigates the following open questions:

  * Q1:

Can we develop a lesion segmentation model that can learn from different
multi-modal MRI databases when each has a different set of modalities and
pathologies?

  * Q2:

Can databases with different sets of modalities mutually benefit via joint
training?

  * Q3:

Can training on different databases help the model segment pathologies that
have been seen during training but on sets of modalities unlike those used in
training?

  * Q4:

How will the jointly trained model perform when encountering pathologies not
presented during its initial training, either directly or through subsequent
fine-tuning?

Several studies have previously investigated the concept of joint training on
multiple medical databases. [Yan et al.(2020)Yan, Cai, Zheng, Harrison, Jin,
Tang, Tang, Huang, Xiao, and Lu] and [Ulrich et al.(2023)Ulrich, Isensee,
Wald, Zenk, Baumgartner, and Maier-Hein] trained a model on multiple databases
for lesion detection and structure segmentation, but they only trained on one
modality (CT). [Moeskops et al.(2016)Moeskops, Wolterink, Van Der Velden,
Gilhuijs, Leiner, Viergever, and Išgum] trained a model on databases of
different anatomies in T1 and CT but did not explore multi-modal brain MRI.
Recently, [Wang et al.(2023)Wang, Guo, Ye, Deng, Cheng, Li, Chen, Su, Huang,
Shen, et al.] built a foundation model for 3D medical images based on SAM
[Kirillov et al.(2023)Kirillov, Mintun, Ravi, Mao, Rolland, Gustafson, Xiao,
Whitehead, Berg, Lo, et al.], trained on a large number of medical databases
with different modalities using a large transformer as backbone. That model
can only process a single modality at a time, it does not learn relations
between modalities, and it requires the user to click or draw a box around the
object for segmentation. Similar is the training of UniverSeg [Butoi et
al.(2023)Butoi, Ortiz, Ma, Sabuncu, Guttag, and Dalca], a model that handles
one modality at a time, and requires manual segmentations given at inference-
time to segment a new database. In contrast, our work is specifically focused
on finding effective methods for handling diverse sets of multi-modal MRI
data, without manual inputs.

Our study also relates to works that focused on the setting where a single
training database is available with specific modalities, aiming to reduce
performance degradation when modalities are missing at test time [Havaei et
al.(2016)Havaei, Guizard, Chapados, and Bengio, Hu et al.(2020)Hu, Maillard,
Zhang, Ciceri, La Barbera, Bloch, and Gori, Islam et al.(2021)Islam,
Wijethilake, and Ren, Zhou et al.(2021)Zhou, Canu, Vera, and Ruan]. Some such
works [Islam et al.(2021)Islam, Wijethilake, and Ren, Zhou et al.(2021)Zhou,
Canu, Vera, and Ruan] require introduction of complex auxiliary components,
such as generative models. Others enable the segmentation model to embed
inputs even if modalities are missing, such as HeMIS [Havaei et
al.(2016)Havaei, Guizard, Chapados, and Bengio]. Our study draws inspiration
from these but focuses on how to handle multiple, diverse training databases
with different sets of MRI modalities and pathologies.

Also related are works that studied how to segment new pathologies not seen
during training. Examples are domain generalization works leveraging
information from other pathologies [Gu et al.(2021)Gu, Zhang, Huang, Lei,
Wang, and Zhang], or transferring pre-trained knowledge to segment new
database [Chen et al.(2019a)Chen, Ma, and Zheng], or unsupervised methods
[Pinaya et al.(2022a)Pinaya, Graham, et al., Pinaya et al.(2022b)Pinaya,
Tudosiu, et al.] that can segment pathologies without training explicitly for
it. Our work instead explores the feasibility of approaching this problem with
a model jointly-trained on heterogeneous sets of modalities.

The main contributions of this study can be summarized as follows:

  * •

We explore and demonstrates the feasibility and benefits of jointly training
with databases with heterogeneous sets of MRI modalities and types of brain
lesions.

  * •

We explore several methods for this challenge and show promising results with
appropriate, simple (thus practical) alterations to the model and training
strategy.

  * •

We approach the stated open questions (Q1-4) by conducting experiments with 7
databases in multiple settings, including segmentation on diverse sets of
modalities, and segmenting types of pathologies seen and unseen during
training, thus providing valueable insights into the properties of this
unexplored training paradigm.

##  2 Materials and methods

###  2.1 Handling varying modalities with Unet and modality drops: Multi-Unet

Can we enable the well-established Unet to learn from databases with distinct
sets of MRI modalities (Tab. 1) without extensive alterations, to retain its
practicality? For this, we use a Residual Unet [Zhang et al.(2018)Zhang, Liu,
and Wang] as base model (Fig. 1) and modify it as follows.

Handling diverse modality sets: Assume we aim to train a model on a collection
$D$ of $N$ training databases, $D\\!=\\!\\{D_{1},...,D_{N}\\}$. Assume
database $D_{i}$ has set of MRI modalities $\mathbb{M}_{i}$, where
$C_{i}=|\mathbb{M}_{i}|$ their number. To process inputs _solely_ from
database $D_{i}$, a Unet would be designed using input layers with $C_{i}$
input channels. Instead, to enable a Unet to process jointly all training
databases $D_{i}$ in $D$, with any distinct set of modalities
$\mathbb{M}_{i}$, we design it with number of input channels $C$ equal to the
number of _unique_ modalities across all $N$ databases,
$C=|\mathbb{M}_{1}\cup...\cup\mathbb{M}_{N}|$. When processing samples from
databases that do not have certain modalities, the corresponding channels are
filled with zeros (Fig. 1). We term the Unet trained with the above strategy
as Multi-Unet.

Enhancing Generalization with Random Drop of Modalities: The above alteration
enables joint training on MRI databases with diverse modality sets. This
model, however, may only learn to segment specific combinations of modalities
$\\{\mathbb{M}_{1},...,\mathbb{M}_{N}\\}$, those available in the training
databases. It may also associate the specific combination of modalities
$\mathbb{M}_{i}$ in database $i$ with the specific pathology annotated in
database $i$, unable to segment it if given another combination. Our
desiderata, instead, is to train a model that can segment any arbitrary
modality combination at test time. To enable this, we drop modalities when
training. For each training sample from database $D_{i}$ with $C_{i}$ number
of modalities, we sample an integer number $n$ of modalities to drop from the
range $[0,C_{i}\\!-\\!1]$. Then we randomly choose $n$ out of the $C_{i}$
modalities of that sample and replace them with blank images. This enforces
the model to segment each pathology under diverse sets of modalities and
encourages feature extraction from all modalities rather than the most
prevalent.

###  2.2 Embedding each modality separately: LFUnet and MAFUnet

We develop and study two alternative, more complex methods that employ
separate embeddings per modality. These are inspired by previous works that
focused on training with a single database while aiming to enhance
generalization when modalities are missing at test time, as opposed to our
study that targets learning from databases with different sets of modalities.
Inspired by HeMIS [Havaei et al.(2016)Havaei, Guizard, Chapados, and Bengio]
that uses a pathway per modality and fuses them by outputting their $average$
and $variance$, we create a Unet-based variant that we term Late-Fusion-Unet
(LFUnet), Fig. LABEL:fig:figLFUnetandMAFUnet, that applies a Unet per modality
and fuses their embeddings at the penultimate layer. Same Unet weights are
shared to avoid overfitting. In early experiments, we found that results are
improved by using fusion via weighted averaging with attention weights, rather
than the averaging and variance used in HeMIS. For $C$ total possible
modalities, and $\mathbf{z}_{i}\in\mathbb{R}^{H\times W\times D}$ the
embedding of i-th modality with H,W,D spatial dimensions, the attention
weights are
$\mathbf{a}=softmax(conv(conv([\mathbf{z}_{1},...,\mathbf{z}_{C}])))\in\mathbb{R}^{H\times
W\times D\times C}$, where softmax normalizes attention across C modalities.
Fusion’s output is then the dot product of attention with the concatenated
embeddings,
$\mathbf{z}_{fused}=\mathbf{a}\cdot[\mathbf{z}_{1},...,\mathbf{z}_{C}]$.
LFUnet requires high memory and computation due to one Unet replica per
modality. We therefore developed a further variant, termed Multi-scale
Attention Fusion Unet (MAFUnet). This uses an encoder per modality and fuses
embeddings at each scale, as shown in Fig. LABEL:fig:figLFUnetandMAFUnet.
Modality dropping (Sec. 2.1) is also employed in these two methods.

Databases | PD | FLAIR | SWI | T1 | T1c | T2 | DWI
---|---|---|---|---|---|---|---
BRATS |  | ✓ |  | ✓ | ✓ | ✓ |
MSSEG | ✓ | ✓ |  | ✓ | ✓ | ✓ |
ATLAS |  |  |  | ✓ |  |  |
TBI |  | ✓ | ✓ | ✓ |  | ✓ |
WMH |  | ✓ |  | ✓ |  |  |
ISLES |  | ✓ |  | ✓ |  | ✓ | ✓
Tumor_2 |  |  |  | ✓ |  |  |

Table 1: Modalities in each database

Figure 1: Unet training with modality dropping.

fig:figLFUnetandMAFUnet

Figure 2: Architectures that embed each modality separately: LFUnet and
MAFUnet

###  2.3 Databases

Seven brain MRI databases are used, including 484 tumor cases from BRATS 2016
[Bakas et al.(2017)Bakas, Akbari, Sotiras, Bilello, Rozycki, Kirby, Freymann,
Farahani, and Davatzikos], 53 Multiple Sclerosis cases from MSSEG 2016
[Commowick et al.(2018)Commowick, Istace, Kain, Laurent, Leray, Simon, Pop,
Girard, Ameli, Ferré, et al.], 655 stroke lesion cases from ATLAS v2.0 [Liew
et al.(2022)Liew, Lo, Donnelly, Zavaliangos-Petropulu, Jeong, Barisano,
Hutton, Simon, Juliano, Suri, et al.], 281 traumatic brain injury cases (TBI)
from our institutions, 60 White Matter Hyperintensity cases from the WMH
challenge [Kuijf et al.(2022)Kuijf, Biesbroek, de Bresser, Heinen, Chen, van
der Flier, Barkhof, Viergever, and Biessels], 28 stroke lesion cases from SISS
subtask of ISLES 2015 [Maier et al.(2017)Maier, Menze, Von der Gablentz, Häni,
Heinrich, Liebrand, Winzeck, Basit, Bentley, Chen, et al.], and another tumor
database of 51 cases collected at our institutions (Tumor_2). tab:modalities
shows modalities available in each database. For training we use 444, 459, 37,
156, and 42 cases from BRATS, MSSEG, ATLAS, TBI, and WMH respectively, with
remaining cases used for evaluation. ISLES and Tumor_2 are used for evaluation
only.

Preprocessing: We skull-strip, resample to $1\\!\times\\!1\\!\times\\!1$ mm
and apply z-score intensity normalization to all data. All lesion classes are
merged in a single ‘lesion’ label for all experiments.

##  3 Experiments and Results

Learning from brain lesion MRI databases with heterogeneous modality sets has
not been explored before. Thus we firstly set to identify promising models
among those developed (Sec. 2) and then use them to assess the benefits of
learning from diverse databases.

Training setup: To address class imbalance during training we over-sample
small databases to the frequency of the largest. We train with batch size of 2
for 600 epochs with learning rate $0.001$, decayed to 0.0001 after 150 epochs.

###  3.1 Which architecture is more robust to missing modalities?

Our ultimate desiderata is to develop a model that can learn from databases
with heterogeneous modality sets and can segment new data with modality sets
that differ from those seen during training. A first step towards this is to
identify architectures that perform best on a single database when modalities
are dropped. We show such experiments in tab:missing_mod. We find that to
reduce performance degradation with missing modalities, training with modality
dropping is essential. In this training setting, the most favourable models
are MAFUnet and Multi-Unet, all three achieve similar performance when all
modalities are present, but LFUnet degrades the most with missing modalities,
while requiring highest memory. Therefore in followup experiments we focus on
MAFUnet and Multi-Unet.

tab:missing_mod  Average Dice and Difference LFUnet MAFUnet Multi-Unet
Training with all modalities 0.665 (-0.348) 0.653 (-0.333) 0.658 (-0.528)
Training with modality dropping 0.650 (-0.194) 0.640 (-0.123) 0.649 (-0.125)

Table 2: Performance when training on single database and robustness to
missing modalities: We train each model on each of the five databases
separately (BRATS, TBI, MSSEG, ATLAS, WMH) in two ways: using all available
modalities during training (top) or randomly dropping modalities (bottom). We
then evaluate each model on the corresponding database using all available
modalities, and report average Dice over all databases (out of brackets). To
evaluate robustness to missing modalities, we test each model with all
possible combinations of modalities (dropping 1 or more) on each database ,
and report how much Dice decreased on average across all databases and all
combinations of dropped modalities, in comparison to using all modalities (in
brackets). Training with modality drop is necessary to improve performance
with missing modalities, therefore we concentrate on bottom row. Therein, all
architectures achieve similar Dice with all modalities availble but LFUnet
drops the most with missing modalities. Hence Multi-Unet or MAFUnet are
preferred.

###  3.2 Can we train one model on multiple databases with varying modalities?

We now set out to train a single model using 5 databases (MSSEG, TBI, WMH,
BRATS, ATLAS). We assess whether it is possible to obtain one model that can
segment all databases well, regardless that each has a distinct set of
modalities and pathology. For this, we train Multi-Unet and MAFUnet with all 5
databases. We use each to segment all databases. We compare their performance
with instances of Residual Unets trained separately on each database in
tab:traintogehter. Results verify that training with multiple databases is
feasible, regardless the distinct sets of modalities. The simple (thus
practical) Multi-Unet is particularly effective! It gives results comparable
or better than the database-specific models, with the great practical benefit
that knowledge is wrapped in a single model. In following sections we focus on
Multi-Unet and explore what more benefits are unlocked by such model that has
condensed knowledge from varying sets of modalities and pathologies.

tab:traintogehter  Methods MSSEG TBI WMH BRATS ATLAS Avg Unet-SD (all mods)
0.659 0.515 0.721 0.908 0.487 0.658 Unet-SD (drop) 0.657 0.517 0.695 0.908
0.487 0.653 MultiUnet-MD (drop) 0.676 0.521 0.725 0.906 0.485 0.663 MAFUnet-MD
(drop) 0.674 0.515 0.708 0.891 0.468 0.651

Table 3: Performance when training on multiple databases: Separate models were
trained on each Single Database (SD) and evaluated on the same database,
compared with models jointly trained on Multiple Databases (MD), shown either
all available modalities during training (all mods) or random modality drop
(drop). All available modalities are provided for testing. For each
experiment, average Dice of 3 runs (different RNG seeds) is reported. Most
importantly, one MD model can segment all databases regardless their different
modality sets, in contrast to SD that can only segment the database seen
during training. MultiUnet-MD performs best on average, showing benefits of
transfer learning especially on smaller databases (MSSEG, TBI, WMH).

###  3.3 Segmenting “known” pathologies in databases with different modality
sets

Models that can learn from diverse databases and sets of modalities could
potentially unlock the practical benefit of enabling segmentation of new
databases that have the same pathologies as some of the training databases but
have different sets of modalities than them. tab:diff_mod shows related
experiments. The Multi-Unet previously trained on 5 databases with modality
drop has only seen stroke in T1 (from ATLAS) during training. Results
demonstrate that this model can use the additional modalities (T2, Flair) of
ISLES to better segment stroke therein, thanks to having processed other types
of pathologies in these modalities during training. The same model can also
segment tumor in a new database that contains only T1 better than a model
trained solely on BRATS using modality drop to enable inference on just T1.
These results may not reach state-of-the-art performance on these databases,
but they demonstrate the promising benefits from multi-database training when
the challenge of handling distinct sets of modalities is overcome.

tab:diff_mod

Table 4:  Segmenting “known” pathologies in new databases with different sets
of modalities: Stroke segmentation model trained solely on ATLAS, which
contains only T1, cannot use T2 and FLAIR available in ISLES (first row).
Multi-Unet trained on (ATLAS, MSSEG, WMH, TBI, BRATS) with modality drop
(fourth row) has seen stroke only in ATLAS’s T1 during training, but can use
ISLES’s T2 and FLAIR for better segmentation, as T2 and FLAIR have been seen
in other training databases. The same model segments tumor in unseen database
Tumor_2, which contains only T1, better than a model trained only on BRATS,
which contains T1, T1c, T2, FLAIR (second row). Average Dice of 3 runs
(different RNG seeds) reported.

Training Inference | Stroke_2 (ISLES) | Tumor_2
---|---|---
(T1) | (T1,T2) | (FLAIR, T1, T2) | (T1)
Stroke_1 (ATLAS) | 0.053 | N/A | N/A |
Tumor_1 (BRATS, drop mods) |  |  |  | 0.646
All Databases (all mods) | 0.046 | 0.064 | 0.471 | 0.538
All Databases (drop mods) | 0.348 | 0.424 | 0.559 | 0.697

###  3.4 Can joint training of multiple databases facilitate segmentation of
pathologies unseen during training, directly or after fine-tuning?

Segment unknown pathologies: We now explore whether learning from multiple
brain lesion databases can facilitate direct segmentation of types of
pathologies unseen during training. Related results shown in
tab:unknown_lesions. We compare results with state-of-the-art unsupervised
segmentation models, to compare capabilities of segmenting new type of
pathology. Regardless the simplicity of Multi-Unet, which is a standard Unet
trained with simple modifications, it outperforms complex recent unsupervised
methods, which demonstrates the potential from this training paradigm in
extracting knowledge useful for new segmentation tasks. Regardless, results
are significantly below what is possible with models supervised with labels
for the target pathologies. We therefore now set out to assess if these Multi-
Unet instances can leverage supervision via fine-tuning for the new
pathologies.

tab:unknown_lesions  Methods Test mod. BRATS WMH TBI ATLAS MSSEG Unet-SD
(upper bound) all 0.908 0.721 0.515 0.487 0.659 Transf. VQ-VAE Flair 0.537*
0.429* - - - DDPM Flair or T1 0.469* 0.272* 0.253 0.225 0.121 MultiUNet (drop
mods) Flair or T1 0.614 0.454 0.324 0.241 0.540 MultiUNet (drop mods) all
0.718 0.470 0.334 0.241 0.326

Table 5: Segmenting types of lesions never seen during training: Using the 5
shown databases, 5 instances of Multi-Unet are trained, each on 4 databases,
using the 5th for evaluating model’s ability to segment the unseen pathology.
Dice reported for the held-out database. Multi-Unet outperforms unsupervised
models (Transformer VQ-VAE [Pinaya et al.(2022b)Pinaya, Tudosiu, et al.] and
DDPM [Pinaya et al.(2022a)Pinaya, Graham, et al.]) that have not seen these
pathologies, demonstrating the benefits of learning from diverse databases.
VQ-VAE and DDPM use single modalities (FLAIR, except for Atlas that only
includes T1). Hence we report results of Multi-Unet when tested with the same
inputs, and with all possible modalities. Results by Unet-SD trained on each
target database (Tab. LABEL:tab:traintogehter) are shown as upper bound for
reference.

Segmenting new pathologies after fine-tuning: We here study if Multi-Unets
pre-trained on diverse databases can benefit segmentation of new pathologies
if used as starting checkpoints for fine-tuning to the target database. Table
LABEL:tab:finetune shows related experiments. Results demonstrate the prior
knowledge from diverse databases consistently benefits fine-tuning to new
pathologies in comparison to training from scratch, both with limited and
plenty of labels. We demonstrate this also with a more sophisticated fine-
tuning method based on progressive nets [Rusu et al.(2016)Rusu, Rabinowitz,
Desjardins, Soyer, Kirkpatrick, Kavukcuoglu, Pascanu, and Hadsell], by
learning a new Unet ’column’ that leverages prior knowledge via lateral
connections from the pre-trained column.

tab:finetune  None Limited Data All Data Target 0-shot Scratch Finetune Progr.
Scratch Finetune Progr. BRATS 0.718 0.825 0.850 0.871 0.908 0.911 0.913 ATLAS
0.241 0.268 0.389 0.365 0.487 0.487 0.509 MSSEG 0.326 0.644 0.670 0.673 0.659
0.690 0.706 TBI 0.334 0.375 0.437 0.447 0.515 0.524 0.529 WMH 0.470 0.677
0.695 0.703 0.721 0.736 0.758

Table 6: Benefits for fine-tuning: 5 Multi-Unets are trained on 4 out of 5
databases and evaluated on the 5th target database (0-shot). Performance does
not reach supervised models trained from scratch with labels of the target
database. However, knowledge acquired from the diverse prior databases can
benefit fine-tuning. Standard fine-tuning of Multi-Unets out-performs models
trained from scratch, both when using limited labels (50 for TBI, 20 for
others) or labels of whole training split of target database. We also report
results from a more advanced fine-tuning method using Progressive Nets
(Progr).

##  4 Conclusion

This study investigated several models to enable, for the first time, joint
training on multiple brain lesion MRI databases with heterogeneous sets of
modalities. We demonstrate feasibility and benefits of such training paradigm
on various tasks, including segmenting multiple pathologies with a single
model, segmenting them in new databases and modality sets, and benefits for
transfer learning to learn segmenting new pathologies. Interestingly, these
benefits were realised with minimal alterations to a standard Unet, which
allows practical applications. This study did not aim to raise state-of-the-
art on the individual databases. Rather, it demonstrates the feasibility and
benefits of this training paradigm for multi-modal brain MRI databases, which
we hope will inspire further work in this direction. Future work could
investigate more advanced frameworks that can extract invariant information to
do segmentation on untrained pathologies and modalities without fine-tuning.

TS and FW are supported by the EPSRC Centre for Doctoral Training in Health
Data Science (EP/S02428X/1). ZL is supported by scholarship provided by the
EPSRC Doctoral Training Partnerships programme [EP/W524311/1]. NV is supported
by the NIHR Oxford Health Biomedical Research Centre (NIHR203316). VN, NIHR
Rosetrees Trust Advanced Fellowship, NIHR302544, is funded in partnership by
the NIHR and Rosetrees Trust. The views expressed are those of the authors and
not necessarily those of the NIHR, Rosetrees Trust or the Department of Health
and Social Care. The authors also acknowledge the use of the University of
Oxford Advanced Research Computing (ARC) facility in carrying out this work
(http://dx.doi.org/10.5281/zenodo.22558).

## References

  * [Bakas et al.(2017)Bakas, Akbari, Sotiras, Bilello, Rozycki, Kirby, Freymann, Farahani, and Davatzikos] Spyridon Bakas, Hamed Akbari, Aristeidis Sotiras, Michel Bilello, Martin Rozycki, Justin S Kirby, John B Freymann, Keyvan Farahani, and Christos Davatzikos.  Advancing the cancer genome atlas glioma mri collections with expert segmentation labels and radiomic features.  _Scientific data_ , 4(1):1–13, 2017.
  * [Butoi et al.(2023)Butoi, Ortiz, Ma, Sabuncu, Guttag, and Dalca] Victor Ion Butoi, Jose Javier Gonzalez Ortiz, Tianyu Ma, Mert R Sabuncu, John Guttag, and Adrian V Dalca.  Universeg: Universal medical image segmentation.  In _Proceedings of the IEEE/CVF International Conference on Computer Vision_ , pages 21438–21451, 2023.
  * [Chen et al.(2019a)Chen, Ma, and Zheng] Sihong Chen, Kai Ma, and Yefeng Zheng.  Med3d: Transfer learning for 3d medical image analysis.  _arXiv preprint arXiv:1904.00625_ , 2019a.
  * [Chen et al.(2019b)Chen, Liu, Peng, Sun, and Qiao] Wei Chen, Boqiang Liu, Suting Peng, Jiawei Sun, and Xu Qiao.  S3d-unet: separable 3d u-net for brain tumor segmentation.  In _Brainlesion: Glioma, Multiple Sclerosis, Stroke and Traumatic Brain Injuries: 4th International Workshop, BrainLes 2018, Held in Conjunction with MICCAI 2018, Granada, Spain, September 16, 2018, Revised Selected Papers, Part II 4_ , pages 358–368. Springer, 2019b.
  * [Commowick et al.(2018)Commowick, Istace, Kain, Laurent, Leray, Simon, Pop, Girard, Ameli, Ferré, et al.] Olivier Commowick, Audrey Istace, Michael Kain, Baptiste Laurent, Florent Leray, Mathieu Simon, Sorina Camarasu Pop, Pascal Girard, Roxana Ameli, Jean-Christophe Ferré, et al.  Objective evaluation of multiple sclerosis lesion segmentation using a data management and processing infrastructure.  _Scientific reports_ , 8(1):13650, 2018.
  * [Gu et al.(2021)Gu, Zhang, Huang, Lei, Wang, and Zhang] Ran Gu, Jingyang Zhang, Rui Huang, Wenhui Lei, Guotai Wang, and Shaoting Zhang.  Domain composition and attention for unseen-domain generalizable medical image segmentation.  In _Medical Image Computing and Computer Assisted Intervention–MICCAI 2021: 24th International Conference, Strasbourg, France, September 27–October 1, 2021, Proceedings, Part III 24_ , pages 241–250. Springer, 2021.
  * [Havaei et al.(2016)Havaei, Guizard, Chapados, and Bengio] Mohammad Havaei, Nicolas Guizard, Nicolas Chapados, and Yoshua Bengio.  Hemis: Hetero-modal image segmentation.  In _Medical Image Computing and Computer-Assisted Intervention–MICCAI 2016: 19th International Conference, Athens, Greece, October 17-21, 2016, Proceedings, Part II 19_ , pages 469–477. Springer, 2016.
  * [Hu et al.(2020)Hu, Maillard, Zhang, Ciceri, La Barbera, Bloch, and Gori] Minhao Hu, Matthis Maillard, Ya Zhang, Tommaso Ciceri, Giammarco La Barbera, Isabelle Bloch, and Pietro Gori.  Knowledge distillation from multi-modal to mono-modal segmentation networks.  In _Medical Image Computing and Computer Assisted Intervention–MICCAI 2020: 23rd International Conference, Lima, Peru, October 4–8, 2020, Proceedings, Part I 23_ , pages 772–781. Springer, 2020.
  * [Isensee et al.(2021)Isensee, Jaeger, Kohl, Petersen, and Maier-Hein] Fabian Isensee, Paul F Jaeger, Simon AA Kohl, Jens Petersen, and Klaus H Maier-Hein.  nnu-net: a self-configuring method for deep learning-based biomedical image segmentation.  _Nature methods_ , 18(2):203–211, 2021.
  * [Islam et al.(2021)Islam, Wijethilake, and Ren] Mobarakol Islam, Navodini Wijethilake, and Hongliang Ren.  Glioblastoma multiforme prognosis: Mri missing modality generation, segmentation and radiogenomic survival prediction.  _Computerized Medical Imaging and Graphics_ , 91:101906, 2021.
  * [Kamnitsas et al.(2017)Kamnitsas, Ledig, Newcombe, Simpson, Kane, Menon, Rueckert, and Glocker] Konstantinos Kamnitsas, Christian Ledig, Virginia FJ Newcombe, Joanna P Simpson, Andrew D Kane, David K Menon, Daniel Rueckert, and Ben Glocker.  Efficient multi-scale 3d cnn with fully connected crf for accurate brain lesion segmentation.  _Medical image analysis_ , 36:61–78, 2017.
  * [Kirillov et al.(2023)Kirillov, Mintun, Ravi, Mao, Rolland, Gustafson, Xiao, Whitehead, Berg, Lo, et al.] Alexander Kirillov, Eric Mintun, Nikhila Ravi, Hanzi Mao, Chloe Rolland, Laura Gustafson, Tete Xiao, Spencer Whitehead, Alexander C Berg, Wan-Yen Lo, et al.  Segment anything.  _arXiv preprint arXiv:2304.02643_ , 2023.
  * [Kuijf et al.(2022)Kuijf, Biesbroek, de Bresser, Heinen, Chen, van der Flier, Barkhof, Viergever, and Biessels] Hugo Kuijf, Matthijs Biesbroek, Jeroen de Bresser, Rutger Heinen, Christopher Chen, Wiesje van der Flier, Barkhof, Max Viergever, and Geert Jan Biessels.  Data of the White Matter Hyperintensity (WMH) Segmentation Challenge, 2022.  URL https://doi.org/10.34894/AECRSD.
  * [Liew et al.(2022)Liew, Lo, Donnelly, Zavaliangos-Petropulu, Jeong, Barisano, Hutton, Simon, Juliano, Suri, et al.] Sook-Lei Liew, Bethany P Lo, Miranda R Donnelly, Artemis Zavaliangos-Petropulu, Jessica N Jeong, Giuseppe Barisano, Alexandre Hutton, Julia P Simon, Julia M Juliano, Anisha Suri, et al.  A large, curated, open-source stroke neuroimaging dataset to improve lesion segmentation algorithms.  _Scientific data_ , 9(1):320, 2022.
  * [Maier et al.(2017)Maier, Menze, Von der Gablentz, Häni, Heinrich, Liebrand, Winzeck, Basit, Bentley, Chen, et al.] Oskar Maier, Bjoern H Menze, Janina Von der Gablentz, Levin Häni, Mattias P Heinrich, Matthias Liebrand, Stefan Winzeck, Abdul Basit, Paul Bentley, Liang Chen, et al.  Isles 2015-a public evaluation benchmark for ischemic stroke lesion segmentation from multispectral mri.  _Medical image analysis_ , 35:250–269, 2017.
  * [Moeskops et al.(2016)Moeskops, Wolterink, Van Der Velden, Gilhuijs, Leiner, Viergever, and Išgum] Pim Moeskops, Jelmer M Wolterink, Bas HM Van Der Velden, Kenneth GA Gilhuijs, Tim Leiner, Max A Viergever, and Ivana Išgum.  Deep learning for multi-task medical image segmentation in multiple modalities.  In _Medical Image Computing and Computer-Assisted Intervention–MICCAI 2016: 19th International Conference, Athens, Greece, October 17-21, 2016, Proceedings, Part II 19_ , pages 478–486. Springer, 2016.
  * [Pinaya et al.(2022a)Pinaya, Graham, et al.] Walter H. L. Pinaya, Mark S. Graham, et al.  Fast unsupervised brain anomaly detection and segmentation with diffusion models.  In Linwei Wang, Qi Dou, P. Thomas Fletcher, Stefanie Speidel, and Shuo Li, editors, _Medical Image Computing and Computer Assisted Intervention – MICCAI 2022_ , pages 705–714, Cham, 2022a. Springer Nature Switzerland.
  * [Pinaya et al.(2022b)Pinaya, Tudosiu, et al.] Walter H.L. Pinaya, Petru-Daniel Tudosiu, et al.  Unsupervised brain imaging 3d anomaly detection and segmentation with transformers.  _Medical Image Analysis_ , 79:102475, 2022b.
  * [Rusu et al.(2016)Rusu, Rabinowitz, Desjardins, Soyer, Kirkpatrick, Kavukcuoglu, Pascanu, and Hadsell] Andrei A Rusu, Neil C Rabinowitz, Guillaume Desjardins, Hubert Soyer, James Kirkpatrick, Koray Kavukcuoglu, Razvan Pascanu, and Raia Hadsell.  Progressive neural networks.  _arXiv preprint arXiv:1606.04671_ , 2016.
  * [Ulrich et al.(2023)Ulrich, Isensee, Wald, Zenk, Baumgartner, and Maier-Hein] Constantin Ulrich, Fabian Isensee, Tassilo Wald, Maximilian Zenk, Michael Baumgartner, and Klaus H Maier-Hein.  Multitalent: A multi-dataset approach to medical image segmentation.  _arXiv preprint arXiv:2303.14444_ , 2023.
  * [Wang et al.(2023)Wang, Guo, Ye, Deng, Cheng, Li, Chen, Su, Huang, Shen, et al.] Haoyu Wang, Sizheng Guo, Jin Ye, Zhongying Deng, Junlong Cheng, Tianbin Li, Jianpin Chen, Yanzhou Su, Ziyan Huang, Yiqing Shen, et al.  Sam-med3d.  _arXiv preprint arXiv:2310.15161_ , 2023.
  * [Yan et al.(2020)Yan, Cai, Zheng, Harrison, Jin, Tang, Tang, Huang, Xiao, and Lu] Ke Yan, Jinzheng Cai, Youjing Zheng, Adam P Harrison, Dakai Jin, Youbao Tang, Yuxing Tang, Lingyun Huang, Jing Xiao, and Le Lu.  Learning from multiple datasets with heterogeneous and partial labels for universal lesion detection in ct.  _IEEE Transactions on Medical Imaging_ , 40(10):2759–2770, 2020.
  * [Zhang et al.(2018)Zhang, Liu, and Wang] Zhengxin Zhang, Qingjie Liu, and Yunhong Wang.  Road extraction by deep residual u-net.  _IEEE Geoscience and Remote Sensing Letters_ , 15(5):749–753, 2018.  10.1109/LGRS.2018.2802944.
  * [Zhou et al.(2021)Zhou, Canu, Vera, and Ruan] Tongxue Zhou, Stéphane Canu, Pierre Vera, and Su Ruan.  Feature-enhanced generation and multi-modality fusion based deep neural network for brain tumor segmentation with missing mr modalities.  _Neurocomputing_ , 466:102–112, 2021.

##  Appendix A Additional metrics

In this section, we provide additional metrics for the experiments described
in Sec. 3.3 and Sec. 3.2. We calculate the commonly used metrics of
Sensitivity (i.e. Recall), Precision and Average Symmetric Surface Distance
(ASSD).

tab:traintogehter_sensi_prec and tab:traintogehter_assd show these metrics for
the experiments of Sec. 3.3 that were previously analysed in
tab:traintogehter. tab:traintogehter_sensi_prec shows that on average across
all databases, MultiUnet-MD (joint training on multiple databases) achieves
highest Sensitivity (low FNs) and highest Precision (low FPs) in comparison to
models trained on each database separately. Additionally,
tab:traintogehter_assd shows that it also achieves the best (lowest) average
ASSD. These results demonstrate the consistent improvements of joint training
on multiple databases.

tab:traintogehter_sensi_prec  Methods MSSEG TBI WMH BRATS ATLAS Avg Unet-SD
(all mods) 67.9/80.9 66.8/67.8 73.7/78.9 92.2/92.3 63.1/81.5 72.7/80.3 Unet-SD
(drop) 55.3/83.4 65.0/63.1 71.0/75.2 90.5/93.1 63.1/81.5 69.0/79.3 MultiUnet-
MD (drop) 71.5/82.4 66.4/72.7 76.9/79.5 90.4/93.0 64.6/84.7 74.0/82.5 MAFUnet-
MD (drop) 68.7/81.2 66.9/70.7 73.0/79.0 87.8/92.6 63.9/83.0 72.06/81.3

Table 7: Sensitivity and Precision of models trained on a single database
(Unet-SD) with all modalities (all mods) or by dropping modalities during
training (drop), in comparison to models trained jointly on all databases.
Experimental settings are the same as tab:traintogehter. Sensitivity and
precision are reported in the format “sensitivity/precision”. On average
across all databases (“Avg” column), MultiUnet-MD that is trained on all
available databases with modality drop achieves highest sensitivity and
precision, demonstrating the consistent benefits by this learning paradigm.

tab:traintogehter_assd  Methods MSSEG TBI WMH BRATS ATLAS Avg Unet-SD (all
mods) 3.87 10.30 1.36 1.67 11.35 5.71 Unet-SD (drop) 4.49 11.99 2.08 1.52
11.35 6.29 MultiUnet-MD (drop) 3.71 10.42 1.44 1.67 9.25 5.30 MAFUnet-MD
(drop) 3.72 11.54 1.56 1.99 11.07 5.98

Table 8: Average symmetric surface distance (ASSD) of models trained on a
single database (Unet-SD) with all modalities (all mods) or by dropping
modalities during training (drop), in comparison to models trained jointly on
all databases. The experiment settings are the same as tab:traintogehter.
MultiUnet-MD that is trained on all available databases with modality drop
achieves the best results (lowest ASSD) in all datatabases, demonstrating the
consistent improvements by this learning paradigm.

tab:diff_mod_sensi_prec and tab:diff_mod_assd present these metrics for the
experiments described in Sec. 3.3 that investigate generalisation to new
databases with new sets of modalities. These were previously analysed with the
Dice metric in tab:diff_mod. The model trained with the studied learning
framework (“All databases (drop mods)”) achieves improved Sensitivity in all
but one experiment, improved Precision in all experiments, and improved ASSD
in all experiments. This analysis reinforce the findings that the studied
framework is beneficial.

tab:diff_mod_sensi_prec  Training Inference Stroke_2 (ISLES) Tumor_2 (T1)
(T1,T2) (FLAIR, T1, T2) (T1) Stroke_1 (ATLAS) 9.7/44.5 N/A N/A Tumor_1 (BRATS,
drop mods) 87.5/62.0 All Databases (all mods) 4.5/66.8 5.1/27.0 80.0/53.2
64.1/69.8 All Databases (drop mods) 51.5/64.1 64.9/74.8 81.9/72.8 82.7/79.7

Table 9: Sensitivity and Precision on segmenting “known” pathologies in new
databases with different sets of modalities using one model. The experiment
settings are the same as tab:diff_mod, but we calculate different metrics.
Sensitivity and precision are reported in the format “sensitivity/precision”.

tab:diff_mod_assd  Training Inference Stroke_2 (ISLES) Tumor_2 (T1) (T1,T2)
(FLAIR, T1, T2) (T1) Stroke_1 (ATLAS) 37.16 N/A N/A Tumor_1 (BRATS, drop mods)
11.00 All Databases (all mods) 42.35 58.04 24.11 13.76 All Databases (drop
mods) 25.44 16.36 14.4 7.28

Table 10: Average symmetric surface distance(ASSD) when segmenting “known”
pathologies in new databases with different sets of modalities using one
model. The experiment settings are the same as tab:diff_mod. Best (lower) in
bold.

##  Appendix B Fine-tuning on new databases that contain modality not
available in original training data

When fine-tuning a pretrained model on a new database, the new data may
include imaging modalities absent in the original training databases. For
example, in experiments of Sec. 3.4 and Table 6, a model trained on ATLAS,
BRATS, MSSEG and WMH but not the TBI database has not seen Susceptibility
Weighted Imaging (SWI) modality during training, as it is only available in
the TBI data. Therefore, when this pre-trained model is used as basis for
fine-tuning on the TBI database, an input channel needs to be assigned to this
new SWI modality. In this case, if there is an unused input channel, which was
pre-trained for a modality that is not available in the new, target database
for which the model will be fine-tuned, the new modality can be assigned to
such an unused channel. During fine-tuning, the corresponding weights will be
adjusted accordingly. In our example, the SWI modality is assigned to the PD
channel, as the latter is not available in the TBI database. If the new
database contains total number of modalities larger than the number available
channels from the pre-training stage, the filters of the model’s first layer
can be expanded with additional, randomly initialised weights, to account for
the additional channels.

##  Appendix C Details on Progressive neural networks for fine-tuning

In Section 3.4, We besides standard fine-tuning with further training the
original model parameters on the new target database of interest, we also
applied the framework of Progressive Networks [Rusu et al.(2016)Rusu,
Rabinowitz, Desjardins, Soyer, Kirkpatrick, Kavukcuoglu, Pascanu, and Hadsell]
for improved results. This original work developed this framework for
continual learning, for a model to sequentially learn a new, different task.
Knowledge is transferred from previous tasks to improve convergence speed. The
model architecture aims to prevent catastrophic forgetting by instantiating a
new neural network (a column) for each task being solved. Knowledge transfer
between tasks is facilitated by lateral connections of features from the
previously learned column to the new column. In this work, we use a similar
framework for fine-tuning. We set two models (columns) with the same Unet
architecture used in other experiments. The first column is the model pre-
trained on the original databases. Its parameters are kept frozen, thereby
retaining the knowledge learned from the original databases. The second column
is a Unet with parameters initialised randomly, and will be trained using the
new, target database. Following [Rusu et al.(2016)Rusu, Rabinowitz,
Desjardins, Soyer, Kirkpatrick, Kavukcuoglu, Pascanu, and Hadsell], the second
column leverages features extracted from the first column. To do this,
activations from the layers of the first column before each downsampling
(convolution kernel with stride greater than one) and upsampling layer are
transferred to the second column. These features are concatenated with
corresponding features in the same position in the second column, and are then
processed by the following convolutional layer along with the feature
activations from the previous layer of the second column. During both the
fine-tuning and inference phases, images are inputted simultaneously into both
columns.


# Possible unconventional order parameter in single crystals of SrPt3P
superconductor

Kyuil Cho  Ames Laboratory, Ames, Iowa 50011, USA<EMAIL_ADDRESS>S. Teknowijoyo
Ames Laboratory, Ames, Iowa 50011, USA  Department of Physics & Astronomy,
Iowa State University, Ames, Iowa 50011, USA  S. Ghimire  Ames Laboratory,
Ames, Iowa 50011, USA  Department of Physics & Astronomy, Iowa State
University, Ames, Iowa 50011, USA  E. H. Krenkel  Ames Laboratory, Ames, Iowa
50011, USA  Department of Physics & Astronomy, Iowa State University, Ames,
Iowa 50011, USA  M. A. Tanatar  Ames Laboratory, Ames, Iowa 50011, USA
Department of Physics & Astronomy, Iowa State University, Ames, Iowa 50011,
USA  N. D. Zhigadlo  CrystMat Company, CH-8037 Zurich, Switzerland  R.
Prozorov  Ames Laboratory, Ames, Iowa 50011, USA  Department of Physics &
Astronomy, Iowa State University, Ames, Iowa 50011, USA

###### Abstract

Anisotropic properties of single crystals of SrPt3P were studied using London
penetration depth and electrical resistivity measurements. The upper critical
field, $H_{c2}(T)$, was determined from four-probe electrical resistivity
measurements for three orthogonal directions of a magnetic field with respect
to the crystal. The London penetration depth, $\lambda(T)$, was determined
from the magnetic susceptibility of the Meissner-London state measured using a
tunnel-diode resonator technique. Whereas $H_{c2}(T)$ and the normal-state
$\rho(T)$ are practically identical for all three magnetic field orientations,
the London penetration depth shows significant unidirectional anisotropy. The
low-temperature $\lambda(T)$ is exponentially attenuated when a small
excitation radiofrequency magnetic field, $H_{rf}$, is applied along the
$c^{\prime\prime}-$direction, in which case screening currents flow in the
$a^{\prime\prime}b^{\prime\prime}-$ plane, while for the other two
orientations, $H_{rf}\parallel a^{\prime\prime}$ and $H_{rf}\parallel
b^{\prime\prime}$, the London penetration depth shows a much weaker,
$\lambda(T)\sim T^{2}$, variation. Such unusual and contrasting behavior of
the two anisotropies,
$\gamma_{H}\left(T\right)=H_{c2,ab}/H_{c2,c}=\xi_{ab}/\xi_{c}$ and
$\gamma_{\lambda}\left(T\right)=\lambda_{c}/\lambda_{ab}$, imposes significant
constraints on the possible order parameter. Although our measurements are
insufficient to derive conclusively the superconducting gap anisotropy,
qualitatively, order parameter with two point nodes and a modulation in the
perpendicular direction is consistent with the experimental observations.

##  I Introduction

Superconductivity in platinum-based phosphides, APt3P (A = Sr, Ca, La), was
discovered by Takayama et al. in 2012 [1]. Similar to non-centrosymmetric
superconductors LaPt3Si and CePt3Si, these compounds have distorted anti-
perovskite structure but preserve the inversion symmetry. The APt3P compounds
show apparently large variation of the electron-phonon (EP) coupling strength
from weak to strong with the EP coupling constant values of, $\lambda^{EP}=$
0.57, 0.86 and 1.33, for LaPt3P, CaPt3P and SrPt3P, respectively [2, 3].

SrPt3P has superconducting transition temperature, $T_{c}\approx 8.4$ K, the
highest among the 5d electron superconductors. From the heat capacity
measurements, Takayama et al. concluded that SrPt3P is an $s-$wave
superconductor. Its superconductivity is in a strong coupling limit with the
characteristic ratio, $\Delta\left(0\right)/k_{B}T_{c}\approx 2.5$, notably
exceeding the weak-coupling value of $1.76$ [1]. This feature was attributed
to the presence of soft phonon modes. The conclusion about the $s-$wave
character of the superconducting gap was further supported by the nuclear
magnetic resonance (NMR) measurements of the Knight shift, though no Hebel-
Slichter peak was observed [4]. In an $s-$wave scenario, this feature can also
be due to the enhanced phonon damping. Nonlinear magnetic field dependence of
the Hall resistivity was interpreted as coming from the multiple pockets of
the Fermi surface, potentially hinting at the multi-band superconductivity
[1]. Hu et al. investigated the effect of Pd-doping in polycrystalline SrPt3P
and found that there is a complex interplay between electron correlations,
electron-phonon coupling, and spin-orbit coupling [5]. In principle, such
features may support unconventional order parameters [6, 7].

To probe the nature of superconducting pairing in SrPt3P, London penetration
depth was measured in polycrystalline samples using transverse-field $\mu$SR
[8]. Combined with the measurements of the critical field, the authors
proposed that SrPt3P is a two-band superconductor with equal gaps but
different coherence lengths in different Fermi surface sheets. However, such a
scenario is impossible considering that there is only one characteristic
length scale (one $\xi$) for the spatial variation of the order parameter [7,
6].

On the theoretical side, from first principle calculations, Chen et al.
suggested possible unusual superconductivity in SrPt3P caused by a charge
density wave and strong spin-orbit coupling [9]. In contrast, Subedi et al.
concluded that SrPt3P is a conventional $s-$wave superconductor in which spin-
orbit coupling plays only “a marginal role” [2]. Many more theoretical works
studying the electronic structure and phonon modes followed [10, 11, 12, 13,
14, 15].

In this situation, it is important not only to establish the overall
thermodynamic behavior of SrPt3P, but also to determine the anisotropy of the
superconducting order parameter. This cannot be done on polycrystalline
samples and requires crystals of sufficiently large size in all directions.
The growth of SrPt3P single crystals is non-trivial and requires high-
pressure, high-temperature synthesis, similar to MgB2 [16]. First single
crystals of SrPt3P were synthesized in 2016 [17], and they are used in this
study.

In this paper, we report our investigation of the anisotropic London
penetration depth, $\lambda(T)$, the upper critical field, $H_{c2}(T)$, and
electrical resistivity, $\rho(T)$, in single crystals SrPt3P. Based on the
results, we suggest that SrPt3P may, indeed, support an unconventional order
parameter with point nodes.

##  II Experimental

Single crystals of SrPt3P were grown under high pressure in a cubic anvil cell
and were mechanically separated from the flux as described in detail elsewhere
[17]. It has nearly cubic crystal structure, $a=b=5.8$ Å, $c=5.4$ Å[17].
Unfortunately, such difficult conditions result in samples whose facets are
not oriented in prime crystallographic directions. We therefore assign three
orthogonal axes based on sample shapes: $a^{\prime}$-, $b^{\prime}$-, and
$c^{\prime}$-axes assigned for a resistivity measurement sample, and
$a^{\prime\prime}$-, $b^{\prime\prime}$-, and $c^{\prime\prime}$-axes for a
London penetration depth measurement sample. The shortest directions are
labeled as $c^{\prime}$ and $c^{\prime\prime}$, the longest directions for
$a^{\prime}$ and $a^{\prime\prime}$. Since the response in each of those
directions is a linear combination of the responses in “true” $a,b,c-$axes,
observation of electronic anisotropy would mean a true anisotropic response of
the crystal.

A crystal for the resistivity measurement was 0.6 mm long and 0.1$\times$0.1
mm2 in cross-section. Silver wires of 50 $\mu$m diameter were soldered to the
sample using tin flux [18, 19], in a standard four-probe resistivity
configuration. The contact resistance was in the m$\Omega$ range. Resistivity
measurements were performed in a Quantum Design PPMS down to 1.8 K in magnetic
fields up to 9 T in configurations $H\parallel a^{\prime}-$axis, $H\parallel
b^{\prime}-$axis and $H\parallel c^{\prime}-$axis. Sample orientation was
controlled by attaching the sample to different sides of a plastic
parallelopiped, see Ref.[20] for details. The upper critical field $H_{c2}$
was determined from electrical resistivity measurements using different
criteria as described in the text below.

The temperature variations of the London penetration depth were measured by
using a self-oscillating tunnel diode resonator (TDR) when sample temperature
varied down to 400 mK ($\sim 0.05~{}T_{c}$) [21, 22]. The TDR circuit
resonates at 14 MHz and the frequency shift is measured with one part per
billion (ppb) precision. Its inductor coil generates an ac magnetic field,
$H_{rf}<20~{}\text{mOe}$, so that the sample is always in the Meissner state
at the temperatures of interest. The size of the sample was 600 $\pm$ 5 $\mu$m
($a^{\prime\prime}-$axis), 250 $\mu$m $\pm$ 5 $\mu$m
($b^{\prime\prime}-$axis), 155 $\pm$ 5 $\mu$m ($c^{\prime\prime}-$axis). The
sample was mounted on a 1 mm diameter sapphire rod and inserted into an
inductor coil of 2 mm diameter. The coil and the sample were in vacuum in a
3He cryostat with sample and TDR circuit temperatures actively stabilized by
independent closed-loop LakeShore temperature controllers. The London
penetration depth was measured on the same crystal in three different
configurations; $H_{rf}\parallel a^{\prime\prime},\>b^{\prime\prime}$, and
$c^{\prime\prime}-$axes. The resonant frequency of the $LC-$ tank circuit
containing the sample depends on the total inductance of the sample in the
coil, and it is straightforward to show that the frequency shift compared to
the empty resonator is proportional to the total magnetic susceptibility of a
sample up to a calibration factor that depends on the sample dimensions,
demagnetizing factor, and parameters of the coil. The calibration constant is
established for each experimental run by mechanically pulling the sample out
of the coil at the base temperature of about 400 mK. Technical details of the
technique are provided elsewhere [22, 23, 24, 25].

##  III Results

###  III.1 Upper critical field

Figure 1 shows the temperature-dependent resistivity of SrPt3P single crystal
and compares it with the polycrystalline sample data reported by Takayama _et
al._[1]. As shown in the upper-left inset, the single crystalline sample of
the present study has much lower resistivity over the whole temperature range.
The normalized resistivity in the main panel shows similar temperature
dependence between single crystal and polycrystalline samples, but the
residual resistivity ratio is higher in a single crystal by a factor of two,
RRR $\approx 10$. The right-bottom inset zooms at the superconducting
transition. Note that the transition temperature, $T_{c}$, is also somewhat
higher in the single crystalline sample. Assuming that a single crystal has a
lower scattering rate than a polycrystalline one, this implies a violation of
the Anderson theorem, which would be compatible with an anisotropic or nodal
gap.

Figure 1: (Color online) Temperature dependent normalized resistivity in a
single crystal of SrPt3P (blue circles) for current along the
$a^{\prime}-$axis. It is compared to the results obtained in a polycrystalline
sample by Takayama et al. (green squares) [1]. The upper inset shows the non-
normalized data, and the lower inset zooms at the superconducting transition.
Figure 2: (Color online) Four-probe resistivity, $\rho\left(T\right)$, of
SrPt3P single crystal measured in different magnetic fields (values indicated
next to the curves) in three orthogonal orientations, $H\parallel a^{\prime}$
(dot-dashed), $b^{\prime}$ (dashed), and $c^{\prime}-$axes (solid).

The temperature-dependent resistivities, $\rho\left(T\right)$, measured in the
vicinity of the superconducting transition in magnetic fields applied in three
different orientations, $H$ $\parallel
a^{\prime},\>b^{\prime}\>,\>c^{\prime}-$axes, respectively, are plotted in
Fig. 2. A very close to isotropic behavior is self-evident. Perhaps, only
$c^{\prime}-$axis curves become more rounded than the other two orientations
in fields above 2 T. Fig. 3 summarizes the temperature-dependent upper
critical field, $H_{c2}(T)$, estimated using four different criteria as shown
in the inset in Fig. 3(b). Top panel, Fig. 3(a), shows $H_{c2}(T)$ defined by
the deviation, onset, and offset criteria. Expectedly, the onset criterion
produces values close to the literature data on polycrystalline samples [1,
8], shown by open crossed symbols in both panels of Fig. 3. The commonly used
mid-point transition data are plotted in Fig. 3(b) by different symbols and
the same colors defined in panel (a). We conclude that $H_{c2}(T)$ is very
similar between all three orientations regardless of the criterion used.

Figure 3: (Color online) Upper critical field, $H_{c2}(T)$ of SrPt3P single
crystal measured with magnetic field applied along three principal directions,
$H\parallel a^{\prime}-$axis (green symbols), $H\parallel b^{\prime}-$axis
(red symbols) and $H\parallel c^{\prime}-$axis (blue symbols). The inset in
panel (b) shows four different criteria used to define $H_{c2}$. Upper panel
(a) shows deviation (circles), onset (squares) and offset (triangles)
$H_{c2}$. Lower panel (b) shows commonly used midpoint-defined $H_{c2}$. The
results are compared with the data on polycrystalline samples, from Takayama
et al. (crossed circles) [1] and Khasanov et al. (crossed squares) [8].

Returning to $H_{c2}\left(T\right)$, we note that often made assertion that
its upward (positive) curvature implies multi-band superconductivity as
claimed, for example, in Ref.[8], is not supported by the theory [26, 27, 28].
Similar behavior, especially in the limited temperature interval, may be the
result of complications of the Fermi surface topology, anisotropy of the order
parameter, or nonmagnetic scattering even in a single-band material [27]. In
fact, usually $H_{c2}(T)$ in multi-band superconductors shows “normal” concave
behavior and no difference in shape from the single-band result [26].
Similarly, temperature-dependent anisotropy, $\gamma_{H}(T)=\xi_{ab}/\xi_{c}$,
does not imply multi-band superconductivity and is commonly found in single-
band superconductors in different circumstances [28].

On the other hand, there may be consequences of multi-band character on the
response to scattering as discussed below. The important conclusion from the
resistivity measurements is that, as expected from the crystal structure, this
material is practically isotropic. Of course, the electronic band structure is
quite complicated, but for the analysis of transport and thermodynamic
properties, we may approximate it by a Fermi sphere, an approach justified by
the lack of anisotropy of the upper critical fields.

###  III.2 London penetration depth

In a tetragonal system, London penetration depth has two components, for
fields penetration along the tetragonal plane, $\lambda_{a}$, and for field
penetration along the tetragonal axis, $\lambda_{c}$. When a small magnetic
field is applied in the plane, both $a-$ and $c-$ components contribute to the
measured signal, whereas for the magnetic field along the $c-$axis, only the
in-plane penetration depth. In tetragonal SrPt3P ($a=b\neq c$), therefore,
intrinsically, there are two distinct coherence lengths,
$\xi_{a}=\xi_{b}\neq\xi_{c}$, and two values of the London penetration depth,
$\lambda_{a}=\lambda_{b}\neq\lambda_{c}$. Since our crystals facets are not
oriented properly along the prime crystallographic directions, we use
$a^{\prime}-,b^{\prime}-,c^{\prime}-$ axes for one crystal, and
$a^{\prime\prime}-,b^{\prime\prime}-,c^{\prime\prime}-$ axes for another.
However, the resultant measured quantities are the linear combinations of the
crystallographic quantities, therefore we have two distinct characteristic
lengths of both types, $\xi$ and $\lambda$.

We now examine the temperature variation of the London penetration depth,
$\lambda(T)$, which is linked directly to the structure of superconducting gap
as it depends sensitively on the thermally excited quasiparticles. High-
resolution magnetic susceptibility was measured at zero applied dc field in
three different orientations of small excitation field, $H_{rf}<20$ mOe, along
$a^{\prime\prime}-$, $b^{\prime\prime}-$ and $c^{\prime\prime}-$axes as
discussed in the experimental section above.

Figure 4: (color online) Temperature dependence of the London penetration
depth measured for three orthogonal orientations: $\lambda_{H_{rf}\parallel
a^{\prime\prime}}(T)$ (red squares), $\lambda_{H_{rf}\parallel
b^{\prime\prime}}(T)$ (green triangles), and $\lambda_{H_{rf}\parallel
c^{\prime\prime}}(T)$ (blue circles). Inset zooms at the low-temperature
region of the same curves, green and blue symbols, respectively. The black
solid lines are the fits in the range of $T\leq T_{c}$/3, to the power-law:
$\Delta\lambda\sim T^{n}$.

Figure 4 shows the low-temperature variation of the London penetration depth;
$\lambda_{H_{rf}\parallel a^{\prime\prime}}(T)$ (red squares),
$\lambda_{H_{rf}\parallel b^{\prime\prime}}(T)$ (green triangles), and
$\lambda_{H_{rf}\parallel c^{\prime\prime}}(T)$ (blue circles). As clearly
shown in the inset, the low-temperature part of the London penetration depth
exhibits distinct behavior depending on the magnetic field orientation. London
penetration depth depends on the integral of the projection of the Fermi
velocity over the whole Fermi surface and having nodes or anisotropy in some
parts of the order parameter has an effect on all components of $\lambda$ [24,
25]. In addition, scattering may play a significant role and, depending on the
order parameter structure, even non-magnetic (potential) scattering can be
pair-breaking, which significantly affects the penetration depth and
effectively turns exponential into a power-law behavior at low temperatures
[24, 25]. Therefore, we use a power-law fitting, $\Delta\lambda\sim T^{n}$, to
quantify the degree of creation of quasiparticles with increasing temperature
in different directions. The inset in Fig. 4 shows the large exponent,
$n=4.9$, for $\lambda_{H_{rf}\parallel c^{\prime\prime}}(T)$. Numerically, the
exponents above $n=4$ are indistinguishable from exponential and indicate a
fully gapped superconducting order parameter. On the other hand, the lower
exponents were obtained for $\lambda_{H_{rf}\parallel a^{\prime\prime}}$$(T)$
($n=2.5$) and $\lambda_{H_{rf}\parallel b^{\prime\prime}}$$(T)$ ($n=2.2$),
signifying either line-nodal gap in the dirty limit, the multi-band sign-
changing order parameter, such as $s_{\pm}$ in the dirty limit, or a point
node [24, 25]. However, if the order parameter had a line node or was an
$s_{\pm}$ type on a three-dimensional Fermi surface, the measurements for all
three orientations would show similar low power $n$. Our data seem to suggest
a case with the point node.

Figure 5: (color online) (a) Low-temperature part of the London penetration
depth for ${H_{rf}}\parallel c^{\prime\prime}$. The red solid line is a single
s-wave gap BCS fit. (b) Normalized superfluid density of the same data in the
panel (a), $\rho_{s}(T)=\left(\lambda(0)/\lambda(T)\right)^{2}$, plotted with
$\lambda_{0}$ = 197 nm (blue circle, this work) and $\lambda_{0}$ = 134 nm
(green triangle, from a polycrystal, Ref 8).

For the data of high exponent $n$ = 4.9 ($\lambda_{H_{rf}}\parallel
c^{\prime\prime}$), we attempted a BCS fitting with a standard expression,
$\Delta\lambda(T)=\lambda(0)\sqrt{\pi\delta(0)/2T}\exp(-\delta(0)/T)$. The
best fitting is shown in Fig 5 (a) with $\lambda_{0}$ = 197 nm $\pm$ 35 nm. In
the panel (b), its superfluid density,
$\rho_{s}(T)=\left(\lambda(0)/\lambda(T)\right)^{2}$, was plotted which is
consistent with a single gap BCS case. For comparison, the superfluid density
was also plotted with $\lambda_{0}$ = 134 nm obtained from a polycrystalline
sample (ref. 8).

##  IV Discussion

###  IV.1 The relations between different anisotropies

In tetragonal crystal with $a=b\neq c$, the anisotropies of the upper critical
field, $\gamma_{H}=H_{c2,ab}/H_{c2,c}=\xi_{ab}/\xi_{c}$, and of the London
penetration depth, $\gamma_{\lambda}=\lambda_{c}/\lambda_{ab}$, depend on the
the anisotropy of the Fermi surface and of the order parameter [26, 29, 28].
Here, $H_{c2,c}=\phi_{0}/2\pi\xi_{ab}^{2}$ is the upper critical field
measured with a magnetic field applied along the $c-$axis and $\xi_{ab}$ is
the coherence length in the transverse, in-plane direction,
$H_{c2,ab}=\phi_{0}/2\pi\xi_{c}\xi_{ab}$ is the in-plane upper critical field;
$\lambda_{c}$ is the London penetration depth of the in-plane magnetic field
with screening super-currents flowing along the $c-$axis, and $\lambda_{ab}$
is for the magnetic field along the $c-$axis and screening currents in the
$ab-$plane. In the text we use index $a$ with the understanding that it is
equivalent to $b$, hence $ab$. These anisotropies are generally temperature-
dependent and may increase or decrease on warming or cooling, with
$\gamma_{H}\left(T\right)$ and $\gamma_{\lambda}\left(T\right)$ often going in
the opposite directions [28, 29]. This is easier to see if we assume a
commonly used separation of variables ansatz,
$\Delta\left(T,\mathbf{k}\right)=\Psi\left(T\right)\Omega\left(\mathbf{k}\right)$,
with the normalization $\left\langle\Omega^{2}\right\rangle_{FS}=1$, where
$\left\langle...\right\rangle_{FS}$ denotes the averaging over the Fermi
surface. For example, isotropic $s-$wave is described by $\Omega=1$, whereas a
$d-$wave order parameter is given by $\Omega=\sqrt{2}\cos 2\varphi$. At
$T_{c}$, in clean case, we always have
$\gamma_{H}^{2}\left(T_{c}\right)=\gamma_{\lambda}^{2}\left(T_{c}\right)=\left\langle\Omega^{2}v_{a}^{2}\right\rangle/\left\langle\Omega^{2}v_{c}^{2}\right\rangle$,
so the anisotropy depends on the symmetry of the order parameter described by
$\Omega\left(\mathbf{k}\right)$ [28]. The electrical conductivity is,
$\sigma_{ik}=2e^{2}N\left(0\right)D_{ik},$ where $D_{ik}=v_{i}v_{k}\tau$ is
the diffusivity tensor. In these equations, $v_{i}$ is the relevant component
of the Fermi velocity, $N\left(0\right)$ is the density of states at the Fermi
level, and $\tau$ is the transport scattering time of the normal metal.
Therefore, the anisotropy of resistivity is, roughly,
$\gamma_{\rho}\left(T_{c}\right)=\left\langle
v_{a}^{2}\right\rangle/\left\langle v_{c}^{2}\right\rangle$ assuming isotropic
$\tau$. Therefore, for isotropic order parameter, $\Omega=1$, regardless of
the Fermi surface anisotropy,
$\gamma_{H}^{2}\left(T_{c}\right)=\gamma_{\lambda}^{2}\left(T_{c}\right)=\gamma_{\rho}\left(T_{c}\right)$.
However, in general,
$\gamma_{H}^{2}\left(T_{c}\right)=\gamma_{\lambda}^{2}\left(T_{c}\right)\neq\gamma_{\rho}\left(T_{c}\right).$
There is one more possibility. In the dirty limit, the anisotropy of the order
parameter washes away, and the equality of all three $\gamma$ is restored.
However, there is no reason to assume that we are at the dirty limit.

Electronic bandstructure calculations predict isotropic electrical resistivity
of the normal state [11], and we observed it in our experiments. Combined with
the measured nearly isotropic upper critical field at $T_{c}$, this would
imply isotropic $\Omega=1$ or a dirty limit. However, significantly
anisotropic London penetration depth represents a problem for this
interpretation. In particular, its nearly exponential attenuation at the low
temperature when the excitation magnetic field, $H_{rf}$, is applied along the
$c^{\prime\prime}$ axis, but substantially lower exponents along the other two
orthogonal directions, not far from $n=2$, when the power-law fitting,
$\lambda=AT^{n}$, is performed.

In our view, the only way to understand such behavior within a standard theory
of superconductivity without invoking some complicated scenarios is to find an
order parameter for which
$\gamma_{H}^{2}\left(T_{c}\right)=\gamma_{\lambda}^{2}\left(T_{c}\right)=\gamma_{\rho}\left(T_{c}\right)=1.$
Importantly, it does not imply that $\gamma_{\lambda}\left(T\right)$ is
isotropic at the intermediate temperatures. Several order parameters were
previously analyzed for their resulting anisotropies [28, 29], and some of
them satisfy the requirement.

Figure 6: (Color online) The variation of the London penetration depth in two
orientations for the order parameter, $\Omega/\Omega_{0}=\sin\theta-0.4529\cos
2\varphi$, where the numeric coefficient was chosen so that the system is
isotropic at $T=0$ and $T_{c}$. This order parameter, with two polar point
nodes, in the $k-$ space is shown in the inset.

Of course, we do not know the type of the order parameter, and our crystals
are not properly oriented, so this is just one plausible order parameter fully
compatible with our observations. Specifically we consider a composite order
parameter, $\Omega\propto\sin\theta-a\cos 2\varphi$, which is a combination of
a $d-$wave - like variation in the $ab-$plane (azimuthal angle $\varphi$) and
having two point nodes at the poles, at the polar angle, $\theta=0,\pi$, as
shown schematically in Fig.6. Point nodes are known to result in a power-law
behavior for out-of-plane penetration depth, whereas showing nearly
exponential variation in the plane [30, 29]. The reason that we chose this
more complicated form of the order parameter, and not just
$\Omega\propto\sin\theta$ is that at $a=0.4529$, it produces a non-monotonic
$\gamma_{\lambda}\left(T\right)$ shown in the inset in Fig.6, that starts at
$T=0$, and ends at $T_{c}$ at the isotropic value of 1. As shown in Fig.6,
this order parameter results in a significant difference in
$\lambda\left(T\right)$ between different orientations exhibiting behavior
similar to our observations.

Our observations of presumably point-node superconducting gap structure should
be contrasted with the chiral topological superconducting state observed in
polycrystalline samples of the closely related LaPt3P [31]. The two materials
are isostructural but differ by the electron count. In the LaPt3P the
temperature-dependent superfluid density is consistent with nodal gap,
potentially with horizontal line node and polar point nodes, as determined
from $\mu$SR measurements. Changing electron count moves the system to clearly
point-node behavior, with line nodes erased. This observation clearly suggests
that Fermi surface topology change plays important role in the superconducting
pairing of the compounds. Further investigations may be of great importance.

##  V Conclusions

Based on the measurements of the upper critical fields, the London penetration
depths, and their anisotropies, our results open up a possibility of a
nontrivial order parameter in SrPt3P, possibly with point nodes somewhere on
the isotropic Fermi surface. Taking into account considerable difficulty of
making single crystals of this compound, this conclusion is a step forward
compared to the polycrystalline samples where such observation would be
impossible. We hope our work will motivate further effort to grow and measure
better single crystals of SrPt3P with well-defined orientations of the facets.

###### Acknowledgements.

We thank Seongyoung Kong and Kirill Kovnir for help with crystallographic
orientation, and Linlin Wang for useful discussions of the electronic
structure. Work in Ames was supported by the U.S. Department of Energy (DOE),
Office of Science, Basic Energy Sciences, Materials Science and Engineering
Division. Ames Laboratory is operated for the U.S. DOE by Iowa State
University under contract DE-AC02-07CH11358. N.D.Z. acknowledges support from
the Laboratory for Solid State Physics ETH Zurich and the Department of
Chemistry and Biochemistry of the University of Bern where crystal growth
studies were initiated.

##  VI Data Availability

All data that support the findings of this study are included within the
article (and any supplementary information files).

## References

  * Takayama  _et al._ [2012] T. Takayama, K. Kuwano, D. Hirai, Y. Katsura, A. Yamamoto, and H. Takagi, Phys. Rev. B 108, 237001 (2012).
  * Subedi  _et al._ [2013] A. Subedi, L. Ortenzi, and L. Boeri, Phys. Rev. B 87, 144504 (2013).
  * Aperis  _et al._ [2020] A. Aperis, E. V. Morooka, and P. M. Oppeneer, Annals of Physics 417, 168095 (2020).
  * Shiroka  _et al._ [2015] T. Shiroka, M. Pikulski, N. D. Zhigadlo, B. Batlogg, J. Mesot, and H.-R. Ott, Phys. Rev. B 91, 245143 (2015).
  * Hu  _et al._ [2016] K. Hu, B. Gao, Q. Ji, Y. Ma, W. Li, X. Xu, H. Zhang, G. Mu, F. Huang, C. Cai, X. Xie, and M. Jiang, Phys. Rev. B 93, 214510 (2016).
  * Ichioka  _et al._ [2017] M. Ichioka, V. G. Kogan, and J. Schmalian, Phys. Rev. B 95, 064512 (2017).
  * Kogan and Schmalian [2011] V. G. Kogan and J. Schmalian, Phys. Rev. B 83, 054515 (2011).
  * Khasanov  _et al._ [2014] R. Khasanov, A. Amato, P. K. Biswas, H. Luetkens, N. D. Zhigadlo, and B. Batlogg, Phys. Rev. B 90, 140507 (2014).
  * Chen  _et al._ [2012] H. Chen, X. Xu, C. Cao, and J. Dai, Phys. Rev. B 86, 125116 (2012).
  * Nekrasov and V. Sadovskii [2012] I. Nekrasov and M. V. Sadovskii, JETP Letters 96, 243 (2012).
  * Kang  _et al._ [2013] C.-J. Kang, K.-H. Ahn, K.-W. Lee, and B. Il Min, Journal of the Physical Society of Japan 82, 053703 (2013).
  * Szczecsniak  _et al._ [2014] R. Szczecsniak, A. P. Durajski, and L. Herok, Physica Scripta 89, 125701 (2014).
  * Zhang  _et al._ [2016] X.-Q. Zhang, Z.-Y. Zeng, Y. Cheng, and G.-F. Ji, RSC Adv. 6, 27060 (2016).
  * Zocco  _et al._ [2015] D. A. Zocco, S. Krannich, R. Heid, K.-P. Bohnen, T. Wolf, T. Forrest, A. Bosak, and F. Weber, Phys. Rev. B 92, 220504 (2015).
  * Jawdat  _et al._ [2015] B. I. Jawdat, B. Lv, X. Zhu, Y. Xue, and C.-w. Chu, Phys. Rev. B 91, 094514 (2015).
  * Karpinski  _et al._ [2007] J. Karpinski, N. Zhigadlo, S. Katrych, R. Puzniak, K. Rogacki, and R. Gonnelli, Physica C: Superconductivity 456, 3 (2007).
  * Zhigadlo [2016] N. D. Zhigadlo, Journal of Crystal Growth 455, 94 (2016).
  * Tanatar  _et al._ [2010] M. A. Tanatar, N. Ni, S. L. Bud’ko, P. C. Canfield, and R. Prozorov, Superc. Sci. Technol. 23, 054002 (2010).
  * Tanatar  _et al._ [2011] M. Tanatar, R. Prozorov, N. Ni, S. Bud’ko, and P. Canfield, U.S. Patent 8,450,246 , 10pp. (2011).
  * Liu  _et al._ [2013] Y. Liu, M. A. Tanatar, V. G. Kogan, H. Kim, T. A. Lograsso, and R. Prozorov, Phys. Rev. B 87, 134513 (2013).
  * Van Degrift [1975] C. T. Van Degrift, Review of Scientific Instruments 46, 599 (1975).
  * Prozorov  _et al._ [2000a] R. Prozorov, R. W. Giannetta, A. Carrington, and F. M. Araujo-Moreira, Phys. Rev. B 62, 115 (2000a).
  * Prozorov  _et al._ [2000b] R. Prozorov, R. W. Giannetta, A. Carrington, P. Fournier, R. L. Greene, P. Guptasarma, D. G. Hinks, and A. R. Banks, Appl. Phys. Lett. 77, 4202 (2000b).
  * Prozorov and Giannetta [2006] R. Prozorov and R. W. Giannetta, Superconductor Science and Technology 19, R41 (2006).
  * Prozorov and Kogan [2011] R. Prozorov and V. G. Kogan, Reports on Progress in Physics 74, 124505 (2011).
  * Kogan and Prozorov [2012] V. G. Kogan and R. Prozorov, Reports on Progress in Physics 75, 114502 (2012).
  * Kogan and Prozorov [2013] V. G. Kogan and R. Prozorov, Phys. Rev. B 88, 024503 (2013).
  * Kogan  _et al._ [2019] V. G. Kogan, R. Prozorov, and A. E. Koshelev, Phys. Rev. B 100, 014518 (2019).
  * Kogan and Prozorov [2021] V. G. Kogan and R. Prozorov, Phys. Rev. B 103, 054502 (2021).
  * Einzel  _et al._ [1986] D. Einzel, P. J. Hirschfeld, F. Gross, B. S. Chandrasekhar, K. Andres, H. R. Ott, J. Beuers, Z. Fisk, and J. L. Smith, Physical Review Letters 56, 2513 (1986).
  * Biswas  _et al._ [2021] P. K. Biswas, S. K. Ghosh, J. Z. Zhao, D. A. Mayoh, N. D. Zhigadlo, X. Xu, C. Baines, A. D. Hillier, G. Balakrishnan, and M. R. Lees, Nature Communications 12 (2021).