11institutetext: Department of Computing, Imperial College London, London, UK
22institutetext: Data Science Institute, Imperial College London, London, UK
22email<EMAIL_ADDRESS>33institutetext: Department of Brain Sciences,
Imperial College London, London, UK 44institutetext: Friedrich–Alexander
University Erlangen–Nürnberg, DE 55institutetext: UK Dementia Research
Institute, Imperial College London, London, UK

# LesionMix: A Lesion-Level Data Augmentation Method for Medical Image
Segmentation

Berke Doga Basaran(✉) 1122 Weitong Zhang 11 Mengyun Qiao 2233
Bernhard Kainz 1144 Paul M. Matthews 3355 Wenjia Bai 112233

###### Abstract

Data augmentation has become a de facto component of deep learning-based
medical image segmentation methods. Most data augmentation techniques used in
medical imaging focus on spatial and intensity transformations to improve the
diversity of training images. They are often designed at the image level,
augmenting the full image, and do not pay attention to specific abnormalities
within the image. Here, we present LesionMix, a novel and simple lesion-aware
data augmentation method. It performs augmentation at the lesion level,
increasing the diversity of lesion shape, location, intensity and load
distribution, and allowing both lesion populating and inpainting. Experiments
on different modalities and different lesion datasets, including four brain MR
lesion datasets and one liver CT lesion dataset, demonstrate that LesionMix
achieves promising performance in lesion image segmentation, outperforming
several recent Mix-based data augmentation methods. The code will be released
at https://github.com/dogabasaran/lesionmix.

###### Keywords:

Data augmentation Lesion populating Lesion inpainting Image synthesis Lesion
image segmentation.

##  1 Introduction

Availability of labelled medical imaging data has been a long-term challenge
for developing robust machine learning methods for medical image segmentation.
In particular, when dealing with lesions or abnormality detection, datasets
often follow a long-tail distribution [11, 21], which means there can be a
variety of categories for abnormal cases but with each category only
containing very few samples. In medical imaging, most data augmentation
methods are developed at the image level, aiming to increase the diversity of
the full image [8]. They often lack the capability to model specific
abnormalities in the images. Recently, several disease-specific augmentation
methods have been proposed for brain tumors, multiple sclerosis, and skin
lesions [19, 2, 1]. Unfortunately, the majority of these methods are either
disease or organ-specific, or are difficult to train and implement due to
their complexity.

In this work, we propose LesionMix, a novel and simple lesion-level data
augmentation method for medical image segmentation. LesionMix is able to
populate lesions with various properties, including shape, location, intensity
and lesion load, as well as inpaint existing lesions by using a dual-branch
iterative 3D framework. With LesionMix, we are able to train lesion
segmentation models in a low-data setting, even if there are very few samples
of lesion images. We perform a comprehensive evaluation of LesionMix using
different imaging modalities and datasets, including four brain MR lesion
datasets and one liver CT lesion dataset. Experiments show that LesionMix
achieves promising lesion segmentation performance on various datasets and
outperforms several state-of-the-art (SOTA) data augmentation methods.

###  1.1 Related Works

####  1.1.1 Non-generative data augmentation.

Traditional data augmentation (TDA) techniques are widely used for training
medical image segmentation models [12]. TDA include flipping, rotating,
scaling, intensity changes, and elastic deformations. These augmentations do
not dramatically change the lesion properties, such as the shape and location
of lesions with respect to the surrounding tissue. Zhang et al. proposed
CarveMix, derived from CutMix [26], which uses a lesion-aware Mix-based
technique to carve lesion regions from one image and insert them into another
image [28]. Zhu et al. developed another Mix-based data augmentation method,
SelfMix, which performs augmentation by mixing tumours with non-tumour regions
[29]. Lebbos et al. introduced semantic mixing for rare lesions in ultrasound
images [16]. Zhang et al. presented ObjectAug, an object-level augmentation
method for semantic image segmentation [27]. These methods provide valuable
insights for lesion-level data augmentation. However, they directly mix
original lesion masks for augmentation without augmenting individual lesion
volumes, with no attention to location of the augmentation, or the lesion load
of augmented images.

####  1.1.2 Generative data augmentation.

Generative methods provide an alternative way for data augmentation by
performing abnormality synthesis. Salem et al. synthesises multiple sclerosis
lesions using an encoder-decoder U-Net structure [22]. Bissoto, Jin, and Li et
al. utilise generative adversarial networks (GANs) to synthesise skin lesions
or brain tumours [5, 14, 17]. Reinhold et al. creates lesions with a
predetermined lesion load using a structural casual model [20]. Xia et al.
employs an adversarial framework for subject-specific pathological to healthy
image synthesis, referred to as pseudo-healthy synthesis [25]. Similarly,
Basaran et al. performs lesion image synthesis and pseudo-healthy synthesis by
using cyclic attention-based generators [3]. Lin et al. proposes InsMix, a
data augmentation method for nuclei segmentation, by employing a Copy-Paste-
Smooth principle with a smooth-GAN for achieving contextual smoothness [18].
While generative augmentation methods have potential for diverse abnormality
generation, they are often disease-specific and not easy to extend to
different applications and datasets.

###  1.2 Contributions

There are three main contributions of this work: (1) We propose a novel non-
deep data augmentation method, which augments images at the lesion level and
accounts for lesion shape, location, intensity as well as load distribution.
(2) The method is easy to implement and can be added to a segmentation
pipeline to complement traditional data augmentations. (3) It is generic and
can be applied to datasets of various modalities (MRI, CT, etc).

##  2 Method

###  2.1 LesionMix

The objective is to develop an efficient and easy-to-implement augmentation
method that is aware of lesions in medical images, accounting for the spatial
and load distribution of the lesions. Figure 1 illustrates the proposed
LesionMix method, which consists of two branches, namely for lesion populating
and lesion inpainting. LesionMix takes a lesion image, X, and its
corresponding lesion mask, Y, as input, and generates an augmented lesion
image, X’, and lesion mask, Y’, as output, which achieves a target lesion
load, $\mathrm{v_{tar}}$. If the target lesion load, $\mathrm{v_{tar}}$, is
greater than the current load, $\mathrm{v_{cur}}$, the lesion load is
increased via the populating branch. Otherwise, the lesion load is decreased
via the inpainting branch. To generate diverse lesion samples, lesion-level
augmentations are performed during populating. To maintain the fidelity of the
samples, lesions are augmented according to learnt spatial and load
distributions.

Figure 1: Illustration of the augmentation process of LesionMix. It consists
of lesion populating (top) and lesion inpainting (bottom) branches to
iteratively augment images to a desired lesion load.

###  2.2 Lesion populating

####  2.2.1 Lesion-level augmentation.

Given the input image, X, and its lesion mask, Y, a lesion is randomly
selected and augmented. We apply 3D spatial augmentations, brightness
augmentations (multiplicative), and Gaussian noise augmentations. Lesion-level
spatial augmentations include flipping, rotating, resizing, and elastic
deformation. Augmentations are applied by extracting the selected 3D lesion
volume, applying the augmentation, and inserting the augmented lesion back
into the image. By iteratively inserting lesions into the images, augmented
lesions can overlap one another, allowing for unique lesion formations.
Augmentation parameters are set empirically and provided in Table 1.

Table 1: Data augmentation parameters for LesionMix. $p$ denotes the probability of augmentation. For augmentations where a range is given, the parameter is determined by selecting a value by uniformly sampling from the range. Augmentation | Details
---|---
Flipping | In X,Y,Z dimensions, $p=0.5$ for each dimension
Rotating | In X,Y,Z dimensions, $p=0.5$ for each dimension, range =[1$\degree$, 89$\degree$]
Resizing | Dimension multiplication, range=[0.5, 1.8]
Elastic deformation11footnotemark: 1 | Random deformation grid, $\sigma$ range= [3, 7]
Brightness | Intensity value multiplication, range = [0.9, 1.1]
Gaussian noise | Addition of $\mathcal{N}(0,1)$
Inpainting | Fast marching method

The original image, lesion mask, and the augmented lesion region are mixed
using the following equations to generate the augmented image and mask,

$\text{X'}=\text{X}\odot(1-\text{M})+\text{F}\odot\text{M}$ (1)
$\text{Y'}=\text{Y}\odot(1-\text{M})+\text{M},$ (2)

where F denotes the augmented lesion intensity image, M denotes the mask of
the augmented lesion region, and $\odot$ denotes element-wise multiplication.
To generate X’ we use a soft mask M, in which the boundary pixels of the
lesion mask are weighted by 0.66 and the inner pixels are weighted by 1. This
allows the lesion boundary to blend more naturally with the input image.
Lesion populating can be performed iteratively. At each iteration, lesion-
level augmentation is applied to a randomly selected lesion and inserted into
the image, until the lesion load, $\mathrm{v_{cur}}$, reaches the target
lesion load, $\mathrm{v_{tar}}$.

####  2.2.2 Lesion likelihood map.

The augmented lesion is inserted into the original image at a location sampled
from a spatial heatmap, termed the lesion likelihood map, which describes the
probability that a lesion appears at a specific spatial location in the
anatomy. The map is learnt by summing the labels of the images to produce a
lesion heatmap and normalising it into a probability map. The map is computed
once for each organ dataset before model training. For brain datasets,
augmented lesions can occur on both the white matter and gray matter. Although
white matter lesions may be more common, gray matter lesions have been
recorded in clinical literature [6].

###  2.3 Lesion inpainting

Given the input image and lesion mask, a 3D lesion volume is randomly
selected. 2D axial slices of the volume are inpainted using the fast marching
method [24], which fills in the intensities within the lesion mask with
neighbouring intensities from the normal region, formulated by,

$I(p)=\frac{\sum_{q\in N(p)}w(p,q)[I(q)+\nabla I(q)(p-q)]}{\sum_{q\in
N(p)}w(p,q)},$ (3)

where $I$ denotes the intensity, $p$ denotes a pixel within the lesion mask,
$q\in N(p)$ denotes pixels in the neighbourhood of $q$ that belong to the
normal region, $\nabla I(q)$ denotes the image gradient at $q$ and $w(p,q)$
denotes a weighting function determined by the distance and direction from $q$
to $p$ [24]. After inpainting, we insert the inpainted slices back into the
original image. 2D inpainting is implemented on axial slices of the lesions,
due to the simplicity in implementation and fast computation. To ensure 3D
continuity of the inpainted lesion, Gaussian blurring is performed on the
boundary of inpainted lesions along all three dimensions,

$\text{X'}=G(f(\text{X},\text{M}))\odot\partial\text{M}+f(\text{X},\text{M})\odot(1-\partial\text{M})$
(4) $\text{Y'}=\text{Y}-\text{M},$ (5)

where $f(\text{X},\text{M})$ denotes the inpainting function using fast
marching, $G$ denotes the Gaussian blurring function, and $\partial\text{M}$
denotes the boundary of the lesion mask. Lesion inpainting can be performed
iteratively for randomly selected lesions, until the lesion load,
$\mathrm{v_{cur}}$, reaches the target lesion load, $\mathrm{v_{tar}}$.

###  2.4 Lesion load distribution

Unlike other Mix-based methods, LesionMix allows for the generation of
datasets with varying lesion load distribution. The load distribution, $P(v)$,
is a probability distribution function for lesion volume, $v$. It
characterises the degree of severity of the disease. We experiment with six
different lesion load distributions: low, medium, high, uniform, Gaussian, and
Real. Real denotes the real distribution learnt from the data. The other five
are parametric distribution functions, with parameters described in Figure 2.

Figure 2: Illustration of the brain lesion datasets’ lesion load distribution
(light blue), named Dataset Load, and the six load distributions (dark blue)
for augmentation used by LesionMix. Low load is a uniform distribution sampled
between 5 and 25 percentiles of the Dataset Load. Medium load is a uniform
distribution sampled between 37.5 and 62.5 percentiles of the Dataset Load.
High load is a uniform distribution sampled between 75 and 95 percentiles of
the Dataset Load. Uniform load is a uniform distribution sampled between 5 and
95 percentiles of the Dataset Load. Gaussian load samples from a Gaussian
distribution with the same mean and variance as the Dataset load. Real load
samples directly from the Dataset load distribution. This process is repeated
for the LiTS dataset.

For each image to be augmented, we sample the target lesion load,
$\mathrm{v_{tar}}$, from the distribution and apply lesion populating or
inpainting iteratively to achieve this target. If $\mathrm{v_{tar}}$ is lower
than $\mathrm{v_{cur}}$, lesion inpainting is applied; if $\mathrm{v_{tar}}$
is greater than $\mathrm{v_{cur}}$, lesion populating is applied. We present
examples of inpainting and populating as examples of LesionMix in low load and
high load distribution setting examples, respectively, in Figure 3. We present
the algorithm of LesionMix in Algorithm 1.

Figure 3: Original image, annotation and augmented data by LesionMix with low
and high load image examples for both brain (first three rows) and liver
(fourth row) datasets. Red denotes brain or liver lesions. Green denotes the
liver. Low load example demonstrates performance of inpainted lesions,
indicated by yellow arrows. High load example shows populated lesions,
indicated by blue arrows.

Input Training images and annotations
{$\mathrm{(X_{1},Y_{1}),...,(X_{N},Y_{N})}$}; the desired number of augmented
images $T$, the desired load distribution $P(v)$.

Output Augmented training data
{$\mathrm{(X^{\prime}_{1},Y^{\prime}_{1}),...,(X^{\prime}_{T},Y^{\prime}_{T})}$}

for  _t=1,2,…,T_ do

Sample target load from the distribution, $\mathrm{v_{tar}}\sim P(v)$

if  _$\mathrm{v_{cur}} <\mathrm{v_{tar}}$_ then

while  _$\mathrm{v_{cur}} <\mathrm{v_{tar}}$_ do

1) Randomly select lesion from $\mathrm{(X_{i},Y_{i})}$

2) Sample lesion location from the lesion likelihood map

3) Apply lesion-level augmentations and generate F and M

4) Apply mixing in Eq. 1 and 2

end while

else

while  _$\mathrm{v_{cur}} >\mathrm{v_{tar}}$_ do

1) Randomly select lesion from $\mathrm{(X_{i},Y_{i})}$ and extract axial
slice

2) Apply inpainting in Eq. 4 and 5 and reinsert slices

end while

end if

return $\mathrm{(X^{\prime}_{i},Y^{\prime}_{i})}$

end for

Algorithm 1 LesionMix: Lesion-level augmentation

###  2.5 Properties of LesionMix

We compare LesionMix with other Mix-based data augmentation methods, including
CutMix [26], CarveMix [28] and SelfMix [29]. LesionMix offers greater control
in augmentation, compared to CarveMix and SelfMix. LesionMix is spatially-
aware, utilising the lesion likelihood map for drawing locations and thus
mixes different backgrounds with the lesion. LesionMix performs shape and
intensity augmentations at the lesion level, thus increasing sample diversity.
Apart from populating lesions, it is also able to inpaint lesions and control
the lesion load distribution. We further summarise in Table 2, and present a
qualitative comparison against CutMix and CarveMix in Figure 4.

Table 2: Qualitative comparison of the properties of LesionMix with other Mix-based augmentation methods. Property | CutMix [25] | CarveMix [26] | SelfMix [27] | LesionMix
---|---|---|---|---
Lesion-aware |  | ✓ | ✓ | ✓
Spatially-aware |  |  |  | ✓
Lesion-background mixing |  |  | ✓ | ✓
Lesion-level augmentation |  |  |  | ✓
Lesion inpainting |  |  |  | ✓
Lesion load control |  |  |  | ✓
Figure 4: Original image, annotation and augmented data by Mix-based methods
(lesions: red, liver: green). CutMix produces discontinuities in the image.
CarveMix can place lesions outside the organ, indicated by arrows.

##  3 Experiments

###  3.1 Data

As a generic method for lesion data augmentation, LesionMix is evaluated on
brain lesion MR images and liver lesion CT images.

####  3.1.1 Brain lesion data.

Four brain lesion datasets are used, the MICCAI 2008 multiple sclerosis (MS)
lesion dataset (MS2008, $n$=20) [23], ISBI 2015 longitudinal MS lesion dataset
(MS2015, $n$=21) [7], MICCAI 2016 MS lesion dataset (MS2016, $n$=15) [9], and
MICCAI 2017 white matter hyperintensity dataset (WMH2017, $n_{train}$=60,
$n_{test}$=110) [15]. We use the WMH2017 training set for training a lesion
segmentation network with the proposed augmentation method and evaluate its
performance on the WMH2017 test set and MS2008, MS2015, MS2016 datasets. For
all datasets, FLAIR images are used and resampled to $1\times 1\times 1$ mm3
voxel spacing, followed by brain extraction using FSL [13] and rigid
registration into the MNI space [10].

####  3.1.2 Liver lesion data.

We use the MICCAI 2017 liver tumor segmentation dataset (LiTS) [4]. The
training set contains CT scans for 131 subjects, which are split into batch 1
($n$=28) and batch 2 ($n$=103) by the challenge organisers. We use the LiTS
batch 1 dataset for training a liver lesion segmentation network and evaluate
its performance on the LiTS batch 2 dataset. The in-plane image resolution
ranges from 0.56mm to 1.0mm, and 0.45mm to 6.0mm in slice thickness. The LiTS
dataset has high variance of data size and organ shape, therefore we use the
normalised label map of the liver as the lesion likelihood map. This ensures
the placed lesion is within the liver.

###  3.2 Implementation details

The proposed method is developed on PyTorch. All augmentation methods are
evaluated with the same segmentation model, nnU-Net with 3D full resolution
configuration, and trained for 1,000 epochs on NVIDIA Tesla T4 GPUs.

###  3.3 Results

####  3.3.1 Lesion load distribution.

We perform an ablation study to select the optimal lesion load distribution
for LesionMix. We simulate a low-data setting by selecting just one training
image from WMH2017, perform data augmentation using LesionMix to generate 100
augmented images, and train a segmentation network. Table 3 reports the lesion
segmentation performance when six different lesion load distributions are
used, and compared against the None method, which is trained with a single
image without augmentation.

Table 3: Mean and standard deviations of lesion segmentation Dice scores
($\%$), when different lesion load distributions are used for LesionMix. Best
results are in bold.

Test set | None | Low | Medium | High | Uniform | Gaussian | Real
---|---|---|---|---|---|---|---
MS2008 | $16.46_{15.43}$ | $22.90_{18.59}$ | $22.23_{17.63}$ | $23.57_{18.72}$ | $\mathbf{24.07_{19.04}}$ | $22.22_{17.59}$ | $22.66_{17.84}$
MS2015 | $37.58_{16.05}$ | $\mathbf{40.51_{9.05}}$ | $37.09_{12.20}$ | $40.37_{15.64}$ | $39.38_{15.68}$ | $36.64_{14.34}$ | $37.68_{14.21}$
MS2016 | $24.35_{20.07}$ | $36.75_{21.33}$ | $38.10_{20.37}$ | $49.08_{18.05}$ | $\mathbf{50.72_{18.96}}$ | $41.16_{20.04}$ | $50.32_{16.43}$
WMH2017 | $39.30_{25.16}$ | $49.79_{24.88}$ | $51.59_{23.80}$ | $58.99_{21.26}$ | $\mathbf{59.42_{21.25}}$ | $53.23_{22.85}$ | $55.24_{21.56}$
LiTS | $3.34_{4.94}$ | $9.40_{6.25}$ | $12.18_{10.34}$ | $13.33_{5.94}$ | $\mathbf{13.65_{8.02}}$ | $12.88_{7.20}$ | $11.99_{7.82}$

####  3.3.2 Comparison to other data augmentation methods.

Following the ablation study, we choose the uniform lesion load distribution
for the remaining experiments. We compare LesionMix to SOTA data augmentation
methods, including traditional data augmentations (TDA), which come default
with nnU-Net [12], CutMix [26] and CarveMix [28]. TDA includes rotation,
scaling, mirroring, elastic deformation, intensity perturbation and simulation
of low resolution. We add CutMix, CarveMix, or the proposed LesionMix onto
TDA. We re-implement CutMix [26] for 3D medical images, and use the public
code for CarveMix [28]. We are unable to compare against SelfMix [29] due to
unavailability of public code. For fair comparison, all methods use nnU-net as
the segmentation network and augment the WMH2017 training set for brain
lesions and the LiTS batch 1 dataset for liver lesions by five times. We
conduct experiments when different sizes of the training data is used. Table 4
reports the lesion segmentation Dice scores for different data augmentation
methods. LesionMix improves lesion segmentation against SOTA methods in the
majority of experiments. We notice greater statistical significance in
experiments with smaller dataset sizes. We present example segmentations
against benchmark methods in Figure 5.

Table 4: Mean and standard deviations of lesion segmentation Dice scores
($\%$), at different sizes of training data. Best results are in bold.
Asterisks indicate statistical significance (∗: p$\leq$ 0.05, ∗∗: p $\leq$
0.01, ∗∗∗: p $\leq$ 0.005) when using a paired Student’s t-test comparing
LesionMix’s performance to baseline methods.

Size | Test set | TDA[12] | CutMix[26] | CarveMix[28] | LesionMix
---|---|---|---|---|---
| MS2008 | $36.72_{18.07}^{*}$ | $37.67_{19.09}$ | $36.69_{17.82}^{*}$ | $\mathbf{38.30_{17.67}}$
| MS2015 | $71.59_{11.14}$ | $\mathbf{72.81_{7.53}}$ | $71.33_{9.96}$ | $72.33_{11.41}$
100% | MS2016 | $57.85_{17.50}$ | $63.22_{15.04}$ | $\mathbf{65.85_{14.26}}$ | $65.62_{14.56}$
| WMH2017 | $79.13_{10.59}$ | $80.14_{9.69}$ | $79.74_{9.94}$ | $\mathbf{80.95_{9.45}}$
| LiTS | $61.93_{24.30}$ | $58.39_{29.40}^{*}$ | $60.20_{29.04}$ | $\mathbf{63.51_{24.97}}$
| MS2008 | $35.11_{21.16}$ | $35.83_{19.55}$ | $32.71_{19.95}^{*}$ | $\mathbf{36.04_{18.93}}$
| MS2015 | $70.59_{11.14}$ | $71.77_{7.16}$ | $67.44_{9.78}^{**}$ | $\mathbf{71.82_{7.40}}$
50% | MS2016 | $55.75_{17.82}^{***}$ | $57.90_{14.24}^{**}$ | $62.27_{16.64}$ | $\mathbf{62.64_{16.26}}$
| WMH2017 | $73.65_{17.30}$ | $74.26_{12.69}$ | $72.45_{18.25}^{*}$ | $\mathbf{75.65_{17.60}}$
| LiTS | $52.40_{29.21}$ | $49.60_{31.32}^{*}$ | $51.98_{27.99}$ | $\mathbf{52.59_{29.18}}$
| MS2008 | $34.35_{21.49}$ | $33.86_{20.21}$ | $30.64_{21.17}^{*}$ | $\mathbf{34.51_{20.04}}$
| MS2015 | $66.09_{8.82}^{**}$ | $67.69_{6.10}^{*}$ | $67.12_{8.13}^{*}$ | $\mathbf{70.73_{8.20}}$
25% | MS2016 | $55.72_{18.78}^{**}$ | $58.24_{15.67}$ | $60.64_{17.67}$ | $\mathbf{60.94_{16.66}}$
| WMH2017 | $71.50_{17.88}^{*}$ | $72.02_{14.29}$ | $72.12_{16.95}$ | $\mathbf{73.92_{15.81}}$
| LiTS | $29.88_{26.52}^{***}$ | $36.35_{30.12}^{*}$ | $36.77_{29.45}$ | $\mathbf{39.25_{30.01}}$
| MS2008 | $28.77_{17.77}^{**}$ | $31.32_{18.28}$ | $29.57_{21.66}^{*}$ | $\mathbf{32.28_{22.11}}$
| MS2015 | $63.97_{9.97}^{**}$ | $64.97_{10.15}^{*}$ | $58.10_{13.35}^{***}$ | $\mathbf{67.13_{10.73}}$
10% | MS2016 | $43.07_{17.03}^{***}$ | $49.58_{13.65}^{***}$ | $50.34_{13.71}^{***}$ | $\mathbf{61.02_{15.56}}$
| WMH2017 | $71.16_{16.70}$ | $70.18_{14.79}^{*}$ | $67.80_{19.97}^{**}$ | $\mathbf{72.03_{15.48}}$
| LiTS | $24.23_{25.76}^{***}$ | $27.99_{29.26}^{*}$ | $20.53_{26.84}^{***}$ | $\mathbf{30.12_{27.47}}$

Figure 5: Qualitative comparison of segmentation performance when 10% of
dataset size is used. Models with LesionMix detect more lesions and segment
them more accurately.

##  4 Conclusion

We present LesionMix, a simple lesion-level data augmentation method. It is
aware of the lesion likelihood distribution and produces augmented data with
varying lesion load. LesionMix improves segmentation performance against other
Mix-based augmentation methods across datasets of different modalities and
organs. It is modality- and organ-agnostic and can serve as a useful tool for
medical image segmentation.

####  4.0.1 Acknowledgements

This work is supported by the UKRI CDT in AI for Healthcare
http://ai4health.io (Grant No. EP/S023283/1).

## References

  * [1] Abdelhalim, I.S.A., Mohamed, M.F., Mahdy, Y.B.: Data augmentation for skin lesion using self-attention based progressive generative adversarial network. Expert Systems with Applications (2021)
  * [2] Barile, B., Marzullo, A., Stamile, C., Durand-Dubief, F., Sappey-Marinier, D.: Data augmentation using generative adversarial neural networks on brain structural connectivity in multiple sclerosis. Computer Methods and Programs in Biomedicine (2021)
  * [3] Basaran, B.D., Qiao, M., Matthews, P.M., Bai, W.: Subject-specific lesion generation and pseudo-healthy synthesis for multiple sclerosis brain images. In: Simulation and Synthesis in Medical Imaging (2022)
  * [4] Bilic, P., Christ, P., Li, H.B., et al.: The liver tumor segmentation benchmark (LiTS). Medical Image Analysis (2023)
  * [5] Bissoto, A., Perez, F., Valle, E., Avila, S.: Skin lesion synthesis with generative adversarial networks. In: OR 2.0 Context-Aware Operating Theaters, Computer Assisted Robotic Endoscopy, Clinical Image-Based Procedures, and Skin Image Analysis (2018)
  * [6] Calabrese, M., Favaretto, A., Martini, V., Gallo, P.: Grey matter lesions in MS. Prion (2013)
  * [7] Carass, A., Roy, S., Jog, A., et al.: Longitudinal multiple sclerosis lesion segmentation: Resource and challenge. NeuroImage (2017)
  * [8] Chlap, P., Min, H., Vandenberg, N., Dowling, J.A., Holloway, L., Haworth, A.: A review of medical image data augmentation techniques for deep learning applications. Journal of Medical Imaging and Radiation Oncology (2021)
  * [9] Commowick, O., Istace, A., Kain, M., et al.: Objective evaluation of multiple sclerosis lesion segmentation using a data management and processing infrastructure. Scientific Reports (2018)
  * [10] Fonov, V., Evans, A., McKinstry, R., Almli, C., Collins, D.: Unbiased nonlinear average age-appropriate brain templates from birth to adulthood. NeuroImage (2009)
  * [11] Galdran, A., Carneiro, G., González Ballester, M.A.: Balanced-mixup for highly imbalanced medical image classification. In: Medical Image Computing and Computer Assisted Intervention (2021)
  * [12] Isensee, F., Jaeger, P.F., Kohl, S.A.A., Petersen, J., Maier-Hein, K.H.: nnU-Net: a self-configuring method for deep learning-based biomedical image segmentation. Nature Methods (2021)
  * [13] Jenkinson, M., Beckmann, C.F., Behrens, T.E., Woolrich, M.W., Smith, S.M.: FSL. NeuroImage (2012)
  * [14] Jin, Q., Cui, H., Sun, C., Meng, Z., Su, R.: Free-form tumor synthesis in computed tomography images via richer generative adversarial network. Knowledge-Based Systems (2021)
  * [15] Kuijf, H.J., Biesbroek, J.M., De Bresser, J., et al.: Standardized assessment of automatic segmentation of white matter hyperintensities and results of the WMH segmentation challenge. IEEE Transactions on Medical Imaging (2019)
  * [16] Lebbos, C., Barcroft, J., Tan, J., Müller, J., Baugh, M., et al.: Adnexal mass segmentation with ultrasound data synthesis. In: Simplifying Medical Ultrasound: Third International Workshop, ASMUS 2022 (2022)
  * [17] Li, Q., Yu, Z., Wang, Y., Zheng, H.: TumorGAN: A multi-modal data augmentation framework for brain tumor segmentation. Sensors (2020)
  * [18] Lin, Y., Wang, Z., Cheng, K.T., Chen, H.: InsMix: Towards realistic generative data augmentation for nuclei instance segmentation. In: Medical Image Computing and Computer Assisted Intervention (2022)
  * [19] Mok, T.C.W., Chung, A.C.S.: Learning data augmentation for brain tumor segmentation with coarse-to-fine generative adversarial networks. In: Brainlesion: Glioma, Multiple Sclerosis, Stroke and Traumatic Brain Injuries (2019)
  * [20] Reinhold, J.C., Carass, A., Prince, J.L.: A structural causal model for MR images of multiple sclerosis. In: Medical Image Computing and Computer Assisted Intervention (2021)
  * [21] Roy, A.G., Ren, J., Azizi, S., Loh, A., Natarajan, V., et al.: Does your dermatology classifier know what it doesn’t know? Detecting the long-tail of unseen conditions. Medical Image Analysis (2022)
  * [22] Salem, M., Valverde, S., Cabezas, M., et al.: Multiple sclerosis lesion synthesis in MRI using an encoder-decoder U-net. IEEE Access (2019)
  * [23] Styner, M., Lee, J., Chin, B., et al.: 3D segmentation in the clinic: A grand challenge II: MS lesion segmentation. The MIDAS Journal (2008)
  * [24] Telea, A.: An image inpainting technique based on the fast marching method. Journal of Graphics Tools (2004)
  * [25] Xia, T., Chartsias, A., Tsaftaris, S.A.: Pseudo-healthy synthesis with pathology disentanglement and adversarial learning. Medical Image Analysis (2020)
  * [26] Yun, S., Han, D., Oh, S.J., Chun, S., Choe, J., Yoo, Y.: CutMix: Regularization strategy to train strong classifiers with localizable features. In: IEEE/CVF International Conference on Computer Vision (2019)
  * [27] Zhang, J., Zhang, Y., Xu, X.: Objectaug: Object-level data augmentation for semantic image segmentation. 2021 International Joint Conference on Neural Networks (IJCNN) (2021)
  * [28] Zhang, X., Liu, C., Ou, N., et al.: CarveMix: A simple data augmentation method for brain lesion segmentation. In: Medical Image Computing and Computer Assisted Intervention (2021)
  * [29] Zhu, Q., Wang, Y., Yin, L., Yang, J., Liao, F., Li, S.: SelfMix: A self-adaptive data augmentation method for lesion segmentation. In: Medical Image Computing and Computer Assisted Intervention (2022)


Currently at ]Max-Planck Institute for Sustainable Materials, Max-Planck
Strasse 1, 40237 Düsseldorf, Germany

#  Dislocation cartography: Representations and unsupervised classification of
dislocation networks with unique fingerprints

Benjamin Udofia<EMAIL_ADDRESS>Interdisciplinary Centre for Advanced
Materials Simulation (ICAMS), Materials Informatics and Data Science, Ruhr-
Universität Bochum, Universitätsstr. 150, 44801 Bochum, Germany  Tushar Jogi
<EMAIL_ADDRESS>[  Markus Stricker<EMAIL_ADDRESS>Interdisciplinary
Centre for Advanced Materials Simulation (ICAMS), Materials Informatics and
Data Science, Ruhr-Universität Bochum, Universitätsstr. 150, 44801 Bochum,
Germany

###### Abstract

Detecting structure in data is the first step to arrive at meaningful
representations for systems. This is particularly challenging for dislocation
networks evolving as a consequence of plastic deformation of crystalline
systems. Our study employs Isomap, a manifold learning technique, to unveil
the intrinsic structure of high-dimensional density field data of dislocation
structures from different compression axis. The resulting maps provide a
systematic framework for quantitatively comparing dislocation structures,
offering unique fingerprints based on density fields. Our novel, unbiased
approach contributes to the quantitative classification of dislocation
structures which can be systematically extended.

##  I Introduction

Since the first proposal of dislocations as the carrier of plastic
deformationTaylor (1934); Polanyi (1934); Orowan (1934) and subsequent
experimental confirmation,Menter (1956); Hirsch, Horne, and Whelan (1956) many
efforts went into characterizing dislocation structures under different
conditions to better understand the relationship between loading, motion,
dislocation structure, and macroscopic properties: Cyclic loading leads to
dislocation patterning with wall and cell structures Mughrabi (1983); Hussein
and El-Awady (2016) or persistent slip bands Kubin and Sauzay (2016) as well
as certain local configurations of dislocations.El-Achkar and Weygand (2018)
Indentation loading results in specific patterns, depending on the indenter
geometry and surface orientation, Gagel, Weygand, and Gumbsch (2016) as does
bending Motz and Dunstan (2012), torsion Ziemann  _et al._ (2015); Stricker
_et al._ (2022) or tension loading in different crystallographic orientations
Stricker and Weygand (2015) which favor or oppose different configurations,
depending on activation of slip systems and their interaction through
junctions. Sills  _et al._ (2018); Stricker  _et al._ (2018); Akhondzadeh  _et
al._ (2021) Body-centered cubic crystals show different structures because of
the difference in edge and screw dislocation character mobility; Weygand  _et
al._ (2015) microstructures with multiple phases like those of superalloys
result in yet another dislocation structure. Gao  _et al._ (2015); Jogi and
Bhattacharya (2021) In addition, dislocation mobility and, therefore,
structure is affected by temperature Chen  _et al._ (2016) or local chemical
order Sudmanns and El-Awady (2021) as well as the aspect ratio in small scale
specimen. Senger  _et al._ (2011)

It is undisputed that the macroscopic properties are a function of the (local)
dislocation structure(s). Suzuki, Takeuchi, and Yoshinaga (2013); Stricker
_et al._ (2018); Akhondzadeh  _et al._ (2023); Deka  _et al._ (2023)
Consequently, substantial efforts have gone into both experimental Zhang  _et
al._ (2022); Sills and Medlin (2022); Yildirim  _et al._ (2023); Wang  _et
al._ (2023) and computational Steinberger, Gatti, and Sandfeld (2016);
Steinberger, Leimberger, and Sandfeld (2016); Demirci  _et al._ (2023); Katzer
_et al._ (2024) characterization and to develop strategies for their
representation. Robertson and Kalidindi (2021) In its essence, this is the
search for variables rather than confounding variables.Hiemer, Fan, and Zaiser
(2023) Discrete dislocation dynamics simulations provide a wealth of complex,
evolving network data, however it is notoriously difficult to analyze despite
intense mathematical efforts with a hierarchy of tensorial quantities to
capture geometry and topology of dislocation networks.Weger, Gupta, and
Hochrainer (2021) Notably, Steinberger et al. Steinberger, Leimberger, and
Sandfeld (2016) developed a scheme which establishes a correlation between
subvolumes of discrete dislocation simulations based on such tensorial
features derived from continuum dislocation dynamics theory (CDD) Hochrainer
_et al._ (2014) and specimen size. More specifically, they chose classes based
on specimen sizes of simulations from which the (sub)volume was originally
extracted. This approach, while insightful for predicting specimen sizes, may
not be as directly applicable when characterizing the complex features of
specific dislocation structures themselves. Another approach for the
representation of dislocations is the development of a dislocation ontology,
called DISO. Ihsan, Fathalla, and Sandfeld (2023) However, this also does not
solve the problem of characterizing dislocation structures systematically.
Despite all efforts, the quantitative assessment of similarities and
differences between dislocation structures from different loading scenarios,
orientation, crystal structures, etc., remains a challenge and requires a
methodological advancement.

To address this challenge of quantitative dislocation structure
characterization, we present a methodology for uniquely creating fingerprints
of individual dislocation structures. These fingerprints not only serve as
unique identifiers but also encode the intricate relationships between each
other. An analogy would be atomistic systems. For these, specifically in the
context of the development of machine-learned interatomic potentials based on
appropriate representations, this has largely been solved Deringer  _et al._
(2021); Musil  _et al._ (2021a); Qamar  _et al._ (2023) and much of the effort
of the community is focused on efficient implementations. Bochkarev  _et al._
(2022) No such convergence of agreement for representations exists for
dislocation systems. Our contribution represents an initial step towards the
formalization of the development of representations for dislocation
structures. In this first step, we propose an unbiased approach of
characterizing dislocation density fields using the concept of embeddings. Our
method proves to be a powerful tool for the quantitative comparison of
dislocation structures, enabling the identification of both differences and
similarities. Our approach is general and can be extended systematically to
include other features of the dislocation network in addition or instead of
the total dislocation density field we chose here. We demonstrate the
generality of our approach using two different discrete dislocation dynamics
(DDD) codes and foresee that our approach accommodates data not only from
simulations but potentially also from experimental sources, aligning with
recent advances in experimental 3D characterizations of dislocations. Yildirim
_et al._ (2023) Finally, our dislocation maps will be useful in machine
learning approaches where a balanced training data set of diverse structures
is crucial: our maps can be combined with unbiased sample selection strategies
like farthest point sampling. Cersonsky  _et al._ (2021)

The paper is organized with details about the generation of the data using
discrete dislocation dynamics and the analysis using embeddings in section II.
We then present the resulting dislocation structure maps in section III,
discuss implications of discretization, open vs. periodic boundary conditions,
variations of compression axes, the choice of hyperparameters, and the concept
of ‘similarity’ of dislocation structures in section IV. The paper closes with
concluding remarks about the implications and a future path towards the
development of unbiased dislocation structure characterization in section V.

##  II Methods

The basis for the dislocation network characterization is a dataset of
discrete dislocation dynamics simulations, openly available on Zenodo
including the script for interacting with it.Udofia and Stricker (2024); Jogi,
Udofia, and Stricker (2024) For these, we will briefly introduce the
methodical background and postprocessing steps: DDD output of the dislocation
structure is a set of points and segments. This is a consequence of
discretizing curved lines in space and is unsuitable as input to manifold
learning techniques because the number of nodes and segments varies across
timesteps and simulations. A solution to this is to convert curved dislocation
lines into density fields with a user-specified discretization. Subsequently,
we introduce our approach for quantitative characterization employing the
concept of manifold learning resulting in embeddings.

###  II.1 Discrete dislocation dynamics

DDD is a computational method for modeling the collective behavior of
dislocations in materials including their motion and interactions. In DDD,
dislocations are represented as lines which are discretized using nodes and
segments. Additional inputs constitutes all possible interactions between
dislocations such as junctions. Here, we use two different implementations of
DDD to create the dislocation structure dataset. Specifically, ParaDiS
Arsenlis  _et al._ (2007) and the “Karlsruhe Code”. Weygand  _et al._ (2002);
Weygand and Gumbsch (2005) The distinction between the two implementations is
mainly w.r.t. surfaces: periodic vs. open surfaces. In the following, we
provide the parameters common to both frameworks and parameters which are
specific to each implementation.

###  II.2 Simulation setup

(a) Karlsruhe Code (free surfaces).

(b) ParaDiS code (periodic boundary conditions).

Figure 1:  Snapshot of simulated dislocation structures in perspective view
initialized with randomly distributed Frank-Read sources. (a) Aspect ratio of
$2$ for the open surface boundary condition with a low starting density of
dislocation network. (b) Aspect ratio of $1$ for the periodic boundary
condition with a high starting density of dislocation network. Strain rate-
controlled displacement is applied in negative y direction.

####  II.2.1 Common conditions

All samples are compressed using displacement control with a constant strain
rate of $5000\text{\,}{\mathrm{s}}^{-1}$, typical for DDD. Compression loading
was chosen based on the prevalent use of compression tests in small-scale
plasticity experiments. Frick  _et al._ (2010); Maaß and Uchic (2012); Weekes
_et al._ (2015); Malyar  _et al._ (2017) Two input parameters are
systematically varied: compression direction and initial dislocation density
for a variety of resulting structures. Compression is applied in directions
$[100]$, $[110]$, $[111]$ and $[234]$ resulting in varying numbers of active
slip systems and distinct dislocation structures. Additionally, we explore
deviations from perfect orientations and adjust the compression axis by
increments of $\pm$$5\text{\,}\mathrm{\SIUnitSymbolDegree}$ and $\pm$
$10\text{\,}\mathrm{\SIUnitSymbolDegree}$. Density variations are a ‘low’
initial density at $2.5\text{\times}{10}^{13}\text{\,}{\mathrm{m}}^{-2}$ and a
‘high’ initial density of $1\text{\times}{10}^{14}\text{\,}{\mathrm{m}}^{-2}$.
With this, we capture both the plastic regime characterized by the activation
of a single source and the multiplication regime where dislocation
interactions dominate the network evolution. The resulting dislocation
structures are then postprocessed into dislocation density fields with varying
discretization. Grid size of the density field is approximately the average
dislocation distance $\bar{d}=\rho^{-1/2}$ at the end of the compression
simulations.

####  II.2.2 Material parameters for open surfaces

For the open boundary condition, we use material parameters mimicking face-
centered cubic (fcc) isotropic aluminum: a lattice parameter of
$a=$0.404\text{\,}\mathrm{nm}$$, Poisson’s ratio $\nu=0.347$, and shear
modulus $G=$27\text{\,}\mathrm{GPa}$$. The box size of the fcc base sample
measures $240\times 480\times 240$ $\mathrm{nm}$3, an aspect ratio of $2$. The
initial dislocation structure consists of randomly distributed Frank–Read
sources with a length of $100\text{\,}\mathrm{nm}$, subject to a variation of
$\pm 20$% within the volume (cf. Fig. 1). All simulations were compressed to a
total strain of approximately $1$%.

####  II.2.3 Material parameters for periodic boundary conditions

For the periodic boundary condition (PBC), material parameters corresponding
to pure copper are used: $a=$ $0.361\text{\,}\mathrm{nm}$, $G=$
$56.9\text{\,}\mathrm{GPa}$, and $\nu=0.324$. Frank–Read sources were
distributed within a cube-shaped volume with an aspect ratio of one (see Fig.
1). Simulations were conducted using a box size of $500\times 500\times 500$
$\mathrm{nm}$3, containing Frank-Read sources with lengths ranging from
$256\text{\,}\mathrm{nm}281\text{\,}\mathrm{nm}$. All simulation are performed
up to total compressive strains ranging from $0.2\text{\,}\%4\text{\,}\%$ for
different loading directions.

####  II.2.4 Postprocessing: density fields

All the simulations were postprocessed into density fields to achieve a fixed
input size for the manifold learning algorithms and facilitate meaningful
comparisons among the different simulations. For simulations conducted under
periodic boundary conditions, discretizations of $10\times 10\times 10$ and
$20\times 20\times 20$ were applied to the density fields, resulting in
resolutions of $50\text{\,}\mathrm{nm}$ and $25\text{\,}\mathrm{nm}$,
respectively. For open boundary simulations we used discretizations of
$8\times 16\times 8$ and $16\times 32\times 16$, resulting in spatial
resolutions of $30\text{\,}\mathrm{nm}$ and $15\text{\,}\mathrm{nm}$,
respectively.

###  II.3 Embeddings through manifold learning

The field representations of the previous step of dislocation structures
result in a high-dimensional representation ranging from $1000$ to $8192$
values depending on discretization. Our objective is to find structure in
these datasets for effective characterization and visualization of the
similarities and differences resulting from different initializations,
compression axes, and open vs. periodic surfaces. We therefore apply several
manifold learning techniques to our dataset to create mappings from the high-
dimensional dislocation density fields to the low-dimensional embeddings, here
called dislocation structure maps. These embeddings serve as a representation
of a specific dislocation structure. Manifold learning techniques have become
a cornerstone of modern machine learning, offering a powerful approach to
represent data in a more informative and compact form. In a general sense,
embeddings are a mathematical representation of an object or entity in a
lower-dimensional space while preserving the relationship to other entities in
the dataset. The goal is to capture the important features, characteristics,
or relationships of objects in a condensed form that can be easily processed
by algorithms. The basic idea behind embeddings is to reduce complex and
usually high-dimensional representations of objects in a simplified and
compact form, which can be used for efficient computation and analysis, such
as data visualization, Iwata  _et al._ (2004) dimensionality reduction, Roweis
and Saul (2000) and machine learning. LeCun, Bengio, and Hinton (2015)
Embeddings are used in a wide range of fields, including computer vision,
Sitzmann  _et al._ (2019) graph analysis, Goyal and Ferrara (2018) and
recommendation systems. Zhang  _et al._ (2016)

We employ the manifold learning technique Isometric Mapping for the generation
of two-dimensional embeddings of the high-dimensional dislocation density
dataset. We choose this algorithm based on superior performance and physical
interpretability compared to other embedding schemes (cf. Supplementary Fig. 1
for comparison with other algorithms we assessed).

#### Isomap embeddings

Isomap is a non-linear dimensionality reduction method designed to capture the
intrinsic geometry of high-dimensional data. It constructs a weighted graph
based on nearest neighbors with edges representing distances. Geodesic
distances between all pairs of points are estimated on the graph, and
classical multidimensional scaling is employed to embed the data into a lower-
dimensional space, preserving the underlying structure of non-linear
manifolds. The choice of the number of nearest neighbors is a critical factor
that significantly influences the outcomes, exerting a substantial impact on
the effectiveness of Isomap. This approach has been widely used in various
fields to reveal the inherent geometry and relationship of complex entities,
offering insights into their underlying structure and relationship.Tenenbaum,
De Silva, and Langford (2000)

Isomap can be conceptualized as an extension of multi-dimensional scaling
(MDS) or kernel principal component analysis (PCA). Kruskal (1964); Borg and
Groenen (2005) The Isomap algorithm, for instance, implemented in the popular
library scikit-learn, Pedregosa  _et al._ (2011) navigates through three
distinct stages: locating nearest neighbors, uncovering shortest paths in
graphs, and executing partial eigenvalue decomposition. Computational
complexity is characterized by input dimensions, neighbor count, and desired
output dimensions. Tenenbaum, De Silva, and Langford (2000)

##  III Results

Embedding maps of the dislocation microstructures for the various loading
directions are shown in Figures 2(a) to 5(b). We present the embedding maps
along two general tracks: open vs. periodic boundaries. Within the open
boundary section we compare different activations by compression along the
$[100]$, $[110]$, $[111]$, and $[234]$ crystallographic directions of fcc
crystals. Along these orientations, we vary the initial dislocation density
and the coarse vs. fine discretization of the density field. In addition, we
present embedding maps for compression simulations varying
$\pm$5\text{\,}\mathrm{\SIUnitSymbolDegree}$$ and
$\pm$10\text{\,}\mathrm{\SIUnitSymbolDegree}$$ from the perfect
crystallographic alignment of the compression axes. For the periodic boundary
conditions, we present results for perfectly aligned compression axes as above
using a coarse discretization for the low-density initialization and a fine
discretization for the high-density initialization. The periodic boundary
conditions case serves to show that our analysis is generally applicable to
dislocation structures.

###  III.1 Open surfaces

####  III.1.1 Low resolution

Figure 2 shows embeddings maps for simulations with the compression axes
perfectly aligned with crystallographic directions using the coarse
discretization density fields. The low density map (Fig. 2(a)) shows three
diverging orientations with increasing total strain (faint to dark color) from
a common central point indicating a more similar structure at initialization
and dissimmilar structures after compression. The $[110]$ compression axis
trajectory stays at the common central starting point. The high density map
(Fig. 2(b)) shows a less pronounced common starting point but also a generally
diverging behavior with increasing total strain.

(a) Low density case.

(b) High density case.

Figure 2:  Isomap embedding maps for perfect crystal orientations at a
discretization of $8\times 16\times 8$, with $k$ nearest neighbors set to
$30$. (a) Initial density of
$2.5\text{\times}{10}^{13}\text{\,}{\mathrm{m}}^{-2}$. (b) Initial density of
$1\text{\times}{10}^{14}\text{\,}{\mathrm{m}}^{-2}$.

####  III.1.2 High resolution

Figure 3 shows the same data as Figure 2, however using a higher resolution
dislocation density field to represent discrete dislocation structures from
the low and high density initializations. The previously observed diverging
behavior from a common starting point is more pronounced, in particular for
the high-density case.

(a) Low density case.

(b) High density case.

Figure 3:  Isomap embedding results for perfect crystal orientations at a
discretization of $16\times 32\times 16$, with $k$ nearest neighbors set to
$50$. (a) Initial density of
$2.5\text{\times}{10}^{13}\text{\,}{\mathrm{m}}^{-2}$. (b) Initial density of
$1\text{\times}{10}^{14}\text{\,}{\mathrm{m}}^{-2}$.

####  III.1.3 Variations in crystal orientation

In addition to the previously presented perfectly-aligned compression axes
trajectories, Figure 4 shows trajectories with
$\pm$5\text{\,}\mathrm{\SIUnitSymbolDegree}$$ and
$\pm$10\text{\,}\mathrm{\SIUnitSymbolDegree}$$ deviation of the compression
axes from the perfect crystallographic orientation. Deviations from the
crystallographic axes are indicated using individual labels. In the low-
density initialization case (Fig. 4(a)) with coarse discretization fields,
similarly-oriented compression axes cluster together but vary. E.g. the
$[100]+10$ case is closer to the $[110]-5$ case than to the $[234]+10$ case
(cf. also Supplementary Fig. 2). In addition, similarities within one
orientation group $[111]+5$, $[111]+10$, etc. are visible on the lower right,
indicating that similar dislocation structures develop despite the variation
in compression axes. The high-density case (Fig. 4(b)) shows three strongly
diverging trajectories, $[111]+-0$, $[100]+-0$, and $[111]-10$, whereas the
other trajectories cluster in the region around the common starting point.

(a) Low density case.

(b) High density case.

Figure 4:  Isomap embedding results for perfect crystal orientations ($\pm 0$)
and their deviations at angles of $\pm 5$ and $\pm 10$ degrees. (a)
Discretization of $8\times 16\times 8$, with $k$ nearest neighbors set to
$30$. (b) Discretization of $16\times 32\times 16$, with $k$ nearest neighbors
set to $50$.

###  III.2 Periodic boundary conditions

In addition to the application of the manifold learning techniques to
simulations using open surfaces, we also applied it to trajectories with
periodic boundary conditions. This was done to facilitate comparison with
other simulation tools or frameworks. The maps were generated for both the
low-resolution, low-density case Fig. 5(a) and the high-resolution, high-
density case Fig. 5(b). Dislocation trajectories are grouped into four
different clusters corresponding to the loading directions (see legends),
which start at a common central point. The trajectories then distinctly
deviate with increasing total strain (faint to darker colors). In the low-
density case (Fig. 5(a)), the $[111]$ and $[100]$, the directions overlap
significantly; in the high-density case (Fig. 5(b)), all but the $[111]$
trajectory diverge whereas the latter stays close to the common starting
point.

(a) Low density case.

(b) High density case.

Figure 5:  Isomap embedding results for perfect crystal orientations under
PBC. (a) Discretization of $10\times 10\times 10$, with $k$ nearest neighbors
set to 30. (b) Discretization of $20\times 20\times 20$, with $k$ nearest
neighbors set to 50.

##  IV Discussion

Embedding coordinates are a novel representation of dislocation networks and
provide a unique view on the structural relationship of different discrete
dislocation structures from various trajectories. Clear correlations are
visible within individual trajectories (cf. Fig 2) as sequential simulation
steps are mapped in sequence. Further, different compression axes which lead
to different glide system activations generally have different embedding
coordinates in the maps. Embeddings provide an unbiased numerical value for
characterization and comparison of dislocation structures. However, as with
most techniques the outcome depends on several user-supplied choices.

It starts with the choice of representation of a discrete dislocation
structure in its original form of nodes and segments from DDD. A conversion
step is necessary since any manifold learning technique requires the same
number of dimensions for each datapoint to be used which is generally not the
case for the representation of dislocation networks in form of segments and
nodes. In this first study, we chose the simplest form of field
representation, dislocation density fields. But any choice of representation
of discrete dislocation structures in form of field variables, including
expansions of higher-order alignment tensors, Weger, Gupta, and Hochrainer
(2021) should be explored in the future. Note that for generality of the
representation any currently existing choice of representation in form of
continuum fields would still require some kind of rotation-invariance since
this is generally not the case: Suh  _et al._ (2020) Imagine exchanging the x
and z axes in Figure 1, the dislocation structure would be the same, but its
representation in form of a field would be markedly different. A possible
solution could be the use of correlations fields per structure. In the
following, we discuss our approach and its implications based on a fixed
dataset as well as fixing the choice to represent dislocation structures as
dislocation density fields.

###  IV.1 Discretization

The results obtained from the low-resolution ($8\times 16\times 8$)
discretization reveal a clear distinction in dislocation structures for the
low-density case as depicted in Fig. 2(a), whereas such clarity is not evident
for the high-density case (see Fig. 2(b)). Conversely, the high-resolution
($16\times 32\times 16$ discretization) results, illustrated in Fig. 3,
distinctly delineate the dislocation structures for both the low and high-
density cases. These findings highlight the significance of resolution in
discerning the differences in dislocation structures. Specifically, the
$8\times 16\times 8$ discretization proves adequate for distinguishing
dislocation structures in the low-density case (see Fig. 2(a)), while the
$16\times 32\times 16$ discretization is necessary for the high-density case
(see Fig. 3(b)). Furthermore, these conclusions align with the comparison of
resolution sizes and the average dislocation distances. By incorporating the
maximum obtained dislocation density values of
$4.38\text{\times}{10}^{13}\text{\,}{\mathrm{m}}^{-2}$ and
$1.93\text{\times}{10}^{14}\text{\,}{\mathrm{m}}^{-2}$ into the average
dislocation distance relation (see Section II.2), we determined the average
dislocation distances of $0.15\text{\,}\mathrm{\SIUnitSymbolMicro m}$ and
$0.072\text{\,}\mathrm{\SIUnitSymbolMicro m}$ for simulations with low and
high starting densities, respectively. Notably, there is a fivefold difference
in the low-density case between the voxel size of the low resolution
($0.03\text{\,}\upmu\mathrm{m}$) and the obtained average dislocation distance
($0.15\text{\,}\upmu\mathrm{m}$), as well as a similar relationship for the
high-density case between the voxel size of the high resolution
($0.015\text{\,}\upmu\mathrm{m}$) and the obtained average dislocation
distance ($0.072\text{\,}\upmu\mathrm{m}$). The observed fivefold difference
between voxel sizes and average dislocation distances provides a sufficient
resolution to distinguish different dislocation structures. A similar trend is
observed for the PBC case, implying that the captured dislocation structures
are accurately represented within the resolution limits. Other field
representations of the discrete representation, e.g. with higher dimensional
tensors Weger, Gupta, and Hochrainer (2021), will likely work with lower
resolutions.Steinberger, Song, and Sandfeld (2019)

###  IV.2 Open surface vs. periodic boundary conditions

Various embedding techniques are explored to reduce the dimensionality of the
generated datasets. Comparing results from other techniques (cf. Supplementary
Fig. 1) demonstrate that the Isomap embedding approach is the most
appropriate. This is due to its ability to not only identify the distinct
compression axes in the datasets but also classify the data points according
to the simulation steps – the Isomap result is amenable to physical
interpretation. The divergence of each crystal orientation (relative to the
compression axis) from the simulation onset to the final simulation step
indicates the progression of deformation across the sample, where the
divergence is visually conveyed by a change in color intensity within each
category.

The observed behavior in the low-density cases, depicted in both the low-
resolution (Fig. 2(a)) and high-resolution (Fig. 3(a)) dislocation maps, can
be attributed to the presence of a spiral Frank-Read source. This source
expands and exits the volume, only to re-emerge in a repetitive manner with
increasing compression. Consequently, self-similar structures emerge (see
Supplementary Fig. 3 for a 3-dimensional embedding view), which are
effectively captured.

In contrast, the high-density cases shown in Fig. 2(b) (low resolution) and
Fig. 3(b) (high resolution) display a different behavior: an increasing
dissimilarity over time with now repeating structures. Deformation and
evolution of the structure is dominated by multiple dislocation interactions
leading to the formation of intricate networks which are well captured in the
embedding coordinates.

The maps obtained dislocation trajectories from a different DDD implementation
with periodic boundary conditions (ParaDiS, see Fig. 5) results in similar
trends for both low and high-density cases. In Figure 5(a), self-similar
structures are observed over time in the low-density case, attributed to the
cyclic motion of one or two activated Frank-Read sources within the volume,
while Figure 5(b) illustrates distinct dislocation structures over time in the
high-density case due to interactions many dislocations. These findings are
well aligned with results of the using open surface boundary conditions and
demonstrate that the developed classification scheme is not dependent on the
specific simulation method used and can be extrapolated to other data sources,
such as experimental frameworks.

###  IV.3 Orientation variations

We examined the effect of misorientations by creating dislocation maps for the
perfectly oriented crystal with deviations of
$\pm$5\text{\,}\mathrm{\SIUnitSymbolDegree}$$ and
$\pm$10\text{\,}\mathrm{\SIUnitSymbolDegree}$$ from the crystallographically-
aligned compression axis, for both low and high-density initializations. In
the low-density case with low resolution (Fig. 4(a)), microstructures of +5°
and +10° misorientations from the $[111]$ orientation overlap, indicating
similar dislocation structures. Similarly, microstructures of
$-$10\text{\,}\mathrm{\SIUnitSymbolDegree}$$ and
$-$5\text{\,}\mathrm{\SIUnitSymbolDegree}$$ misorientations from the $[100]$
orientation overlap, suggesting similarity in their dislocation structures.
The observed overlap is clearly visible in Supplementary Fig. 2. This
indicates that despite the different nomenclatures for misorientations of
$+$5\text{\,}\mathrm{\SIUnitSymbolDegree}$$ and
$+$10\text{\,}\mathrm{\SIUnitSymbolDegree}$$ in the $[111]$ orientation, and
$-$10\text{\,}\mathrm{\SIUnitSymbolDegree}$$ and
$-$5\text{\,}\mathrm{\SIUnitSymbolDegree}$$ in the $[100]$ orientation, the
trajectories and evolving structures are similar and are correctly projected
into embedding space as such.

The results for the high-density case with high resolution depict a complex
dislocation map, where most dislocation structures are mapped to the center of
the map (see Fig. 4(b)). This suggests that the chosen resolution for the
field representations is likely insufficient to distinguish dislocation
structures resulting from different compression axes adequately. It also
underscores the need for additional descriptors such as curvatureSteinberger,
Song, and Sandfeld (2019) or higher order alignment tensorsWeger, Gupta, and
Hochrainer (2021) to represent discrete dislocation networks as fields with
fixed dimensions.

###  IV.4 Methods

The dislocation maps obtained depend on the dataset, the embedding algorithm,
and its hyperparameters. They are unique to the current dataset, and even
slight changes in any of these factors results in a different map. The
following subsections highlight the reasons behind the selections for the
orientations and hyperparameters. Additionally, we discuss constraints imposed
on the embeddings.

####  IV.4.1 Compression axes

Figure 6 shows the calculated Schmid factors for the analyzed perfect
orientations (±0°) and their misorientations at angles of
$\pm$5\text{\,}\mathrm{\SIUnitSymbolDegree}$$ and
$\pm$10\text{\,}\mathrm{\SIUnitSymbolDegree}$$, sorted according to magnitude
for the various slip systems.

Figure 6: Calculated Schmid factors for all 12 glide systems sorted according
to magnitude: perfect orientations
($\pm$0\text{\,}\mathrm{\SIUnitSymbolDegree}$$) and deviations of angles of
$\pm$10\text{\,}\mathrm{\SIUnitSymbolDegree}$$ and
$\pm$5\text{\,}\mathrm{\SIUnitSymbolDegree}$$.

Four crystal orientations were analyzed with respect to the compression axis.
These orientations encompass eight, six, four, and one dominant glide
system(s) for the $[100]$, $[111]$, $[110]$, and $[234]$ orientations,
respectively, as shown in Fig. 6, thereby ensuring the generation of markedly
different microstructures.

The Schmid factors order and magnitude illustrate the basic fact that a
deviation from a given perfect orientation result in different active slip
systems because the symmetry is disturbed. This observation is consistent with
the findings shown in Fig. 4(a), where the band widens in the embedding space
as different misorientations are explored, suggesting that rotating away from
the perfect orientation leads to a distinct distribution of the Isomap
embeddings.

A $-10\text{\,}\mathrm{\SIUnitSymbolDegree}$ variation in $[111]$ and $[234]$
compression as depicted in Figure 6 shows a similar distribution, resulting in
a comparable activation of dislocations. This is further supported by the map
illustrated in Figure 4(a), where the dislocation structures for the
-$10\text{\,}\mathrm{\SIUnitSymbolDegree}$ variation in $[111]$ and $[234]$
compression have similar embedding coordinates: We interpret the alignment of
trajectories as a measure for the activation of similar glide systems,
resulting in comparable dislocation structures. However, this observation is
also true for the +$10\text{\,}\mathrm{\SIUnitSymbolDegree}$ variation in
$[111]$ and $[234]$ compression, where a different distribution of the
activated slip systems as shown in Fig. 6 results in very different
dislocation microstructures (see Fig. 4(a)). We attribute this similarity to
our choice of field representation. Therefore we expect that a field
representation including more information about the dislocation structure in
each voxel other than the total density would result in in a clearer
separation in embedding space.

####  IV.4.2 Selection of hyperparameters

The final choice of hyperparameters for the Isomap algorithm is based on an
optimization to determine the ideal number of $k$-nearest neighbors. We
explored different values of the nearest neighbor parameter and observed that
as the number of nearest neighbors increases, the dataset progressively
separates into four distinct categories, representing various crystal
orientations. Beyond $k=30$, these categories begin to exhibit similarities in
the low-resolution case. In the high-resolution case, using the same
parameters as for the low-resolution case ($k=30$) results in maps that are
less amenable to physical interpretation, as the $[110]$ orientation starts
off from elsewhere and not the center on the map (see Supplementary Fig. 4).
Consequently, we determined the optimal number of nearest neighbors for
achieving distinct categorization to be $k=30$ and $k=50$ for the low and
high-resolution cases, respectively. Optimization of the hyperparameters
currently is a manual process. In the future, this optimization should be
automated and also based on data.

####  IV.4.3 Constraints

One constraint imposed on the embeddings, informed by domain knowledge, is the
expectation of similar dislocation structures at the starting point: each
structure is initially populated with the same number of Frank-Read sources.
Conversely, it is anticipated that the end point will display markedly
different dislocation structures across various orientations because they move
on differently oriented crystallographic planes.

Figure 7:  Similarity matrix illustrating the cosine similarity between
various orientations at simulation timesteps of 5000, 10000, 15000, 20000, and
25000 for the low-density low-resolution case.

Figure 7 shows a similarity matrix using the cosine similarity measure of the
density fields for various crystal orientations ($[100]$, $[111]$, $[110]$,
and $[234]$) at simulation steps 5000 , 10000, 15000, 20000, and 25000,
corresponding to total strains of approximately 0.2 %, 0.4 %, 0.6 %, 0.8 %,
and 1 %, respectively. Each simulation step comprises five pixels representing
orientation variations: $\pm$10\text{\,}\mathrm{\SIUnitSymbolDegree}$$
$\pm$5\text{\,}\mathrm{\SIUnitSymbolDegree}$$ and
$0\text{\,}\mathrm{\SIUnitSymbolDegree}$. A comparison of the density fields
of the perfect crystal orientations shows that within each orientation, the
fields are more similar to each other than to those of the other orientations.
This observation aligns with the patterns in our embeddings in Section III. As
a result, the manual selection of hyperparameters reflects our expectations
regarding the distinctiveness of orientations in embedding space.

##  V Conclusion

The main message of our study is that dislocation structures can be
characterized and compared in novel ways: Manifold learning techniques provide
an unbiased and unsupervised view on inherent structure in data. Finding and
understanding inherent structure is the basis to establish representations
which can be used to model complex systems. This concept applies to
dislocation systems as much as to any other system. The most critical step
before applying manifold learning techniques is the transformation of discrete
dislocation systems represented as nodes and segments to a fixed dimension. We
choose the simplest form of a field representation in form of dislocation
density fields. However, ideal representations are physically motivated,
mathematically robust, exploit symmetries of the described system, efficient
to calculate and uniquely describe the state of a system Hiemer, Fan, and
Zaiser (2023); Musil  _et al._ (2021b). For atomistic systems with degrees of
freedom of atom positions, chemistry, and unit cells in the context of
interatomic potential development (force fields), this has largely been solved
Deringer  _et al._ (2021); Musil  _et al._ (2021a). Much of the effort of the
community now revolves around their efficient implementation and training
strategies. For dislocation systems, no agreement exists on which continuum
representation is the most appropriate for characterization and modeling. In
particular, because the complex networks of lines are notoriously difficult to
analyze, Weger, Gupta, and Hochrainer (2021) e.g. evidenced by the recent
addition of evolution laws for junction point densities Starkey, Hochrainer,
and El-Azab (2022) in the context of continuum dislocation dynamics Hochrainer
_et al._ (2014). Unbiased approaches like the one presented here can provide a
new path to understanding the structure of complex dislocation networks which
can then be correlated with their properties and used in modeling.

In our contribution we establish a methodology for quantitatively comparing
dislocation structures, ultimately creating dislocation structure maps and,
therefore necessarily, a “coordinate system” for dislocation structures. Our
approach lays the groundwork for building a navigation system for the large
variety of existing dislocation structures and their characterization. The
representative set of dislocation trajectories used here comprises several
compression axes and densities as well as open and periodic boundary
conditions. Subsequently, the discrete structures were postprocessed into
dislocation density fields which were then mapped using Isomap embeddings.

The findings of this study demonstrate the ability to differentiate evolving
dislocation structures undergoing significant changes over time, particularly
in the high-density case. Additionally, the results highlight the capacity to
quantify the similarity between dislocation structures that exhibit self-
similarity despite evolving, as observed in the low-density case. These
resulting maps provide valuable insights into the dynamic nature of
dislocation microstructures and their relationship and offer a means to
characterize and compare their evolution.

Even with the simplest method of characterizing dislocation structure — solely
employing a scalar field of dislocation density on a limited dataset — it is
possible to extract maps that align with our (intuitive) physical
understanding: distinct compression axes lead to distinct structures. However,
the sensitivity of the embeddings to (hyper)parameters implies that the
representation of dislocation solely through density field descriptors should
be improved. Thus, it seems likely that more meaningful representations of
dislocation networks including alignment tensors Weger, Gupta, and Hochrainer
(2021) or network properties based on graph measures into the representation
of dislocation structures. Our approach could ultimately provide a new path
for the representation and modeling of dislocation systems.

## Supplementary Material

The supplementary material for this manuscript contains four figures.
Supplementary Fig. 1 shows the results from all the manifold learning
techniques explored in this research study. Supplementary Fig. 2 clearly
depicts the distinct plots of each crystal orientation and loading direction,
properly distinguishing the overlaps observed in Fig. 4(a). Supplementary Fig.
3 presents a visualization of evolving self-similar structures in perfect
crystal orientations using low-density, coarse discretization under open
surface boundary conditions. Supplementary Fig. 4 presents the Isomap
embedding result for the high-density, fine discretization case, using $k=30$
as the number of nearest neighbors and open boundary conditions.

###### Acknowledgements.

Benjamin Udofia and Markus Stricker gratefully acknowledge funding by the
German Research Foundation (DFG) through project STR 1729/1-1, project number
469020538.

## Author Declarations

### Conflict of Interest

The authors have no conflicts to disclose.

### Author Contributions

Benjamin Udofia: data curation (lead); formal analysis (lead), methodology
(equal); software (lead); validation (lead); visualization (lead); writing –
original draft (lead); writing – review and editing (support). Tushar Jogi:
data curation (support); formal analysis (support), methodology (support);
software (support); validation (support); visualization (support); writing –
original draft (support); writing – review and editing (support). Markus
Stricker: data curaction (support); formal analysis (support); methodology
(equal); project administration (lead); software (support); supervision
(lead); validation (support); visualization (support); writing – original
draft (support); writing – review and editing (lead).

## Data Availability Statement

The data that support the findings of this study are openly available in
Zenodo at https://doi.org/10.5281/zenodo.11354118, Udofia and Stricker (2024)
and https://zenodo.org/doi/10.5281/zenodo.11154055Jogi, Udofia, and Stricker
(2024). The development of the scripts for interacting with the data is
accessible on GitLab: https://gitlab.ruhr-uni-
bochum.de/udofibzd/ddd_embeddings.

## References

  * Taylor (1934) G. I. Taylor, “The mechanism of plastic deformation of crystals. Part I. Theoretical,” Proceedings of the Royal Society of London, Series A, Containing Paper of a Mathematical and Physical Character 145, 362–387 (1934).
  * Polanyi (1934) M. Polanyi, “Über eine Art Gitterstörung, die einen Kristall plastisch machen könnte,” Zeitschrift für Physik 89, 660–664 (1934).
  * Orowan (1934) E. Orowan, “Zur Kristallplastizität. I,” Zeitschrift für Physik 89, 605–613 (1934).
  * Menter (1956) J. W. Menter, “The direct study by electron microscopy of crystal lattices and their imperfections,” Proceedings of the Royal Society of London A: Mathematical, Physical and Engineering Sciences 236, 119–135 (1956), http://rspa.royalsocietypublishing.org/content/236/1204/119.full.pdf .
  * Hirsch, Horne, and Whelan (1956) P. B. Hirsch, R. W. Horne, and M. J. Whelan, “Lxviii. direct observations of the arrangement and motion of dislocations in aluminium,” Philosophical Magazine 1, 677–684 (1956), http://dx.doi.org/10.1080/14786435608244003 .
  * Mughrabi (1983) H. Mughrabi, “Dislocation wall and cell structures and long-range internal stresses in deformed metal crystals,” Acta Metallurgica 31, 1367–1379 (1983).
  * Hussein and El-Awady (2016) A. M. Hussein and J. A. El-Awady, “Quantifying dislocation microstructure evolution and cyclic hardening in fatigued face-centered cubic single crystals,” Journal of the Mechanics and Physics of Solids 91, 126 – 144 (2016).
  * Kubin and Sauzay (2016) L. Kubin and M. Sauzay, “Persistent slip bands: Similitude and its consequences,” Acta Materialia 104, 295 – 302 (2016).
  * El-Achkar and Weygand (2018) T. El-Achkar and D. Weygand, “Discrete dislocation dynamics study of dislocation microstructure during cyclic loading,” in  _Fatigue of Materials at Very High Numbers of Loading Cycles: Experimental Techniques, Mechanisms, Modeling and Fatigue Life Assessment_, edited by H.-J. Christ (Springer Fachmedien Wiesbaden, Wiesbaden, 2018) pp. 395–416.
  * Gagel, Weygand, and Gumbsch (2016) J. Gagel, D. Weygand, and P. Gumbsch, “Formation of extended prismatic dislocation structures under indentation,” Acta Materialia 111, 399 – 406 (2016).
  * Motz and Dunstan (2012) C. Motz and D. Dunstan, “Observation of the critical thickness phenomenon in dislocation dynamics simulation of microbeam bending,” Acta Materialia 60, 1603–1609 (2012).
  * Ziemann  _et al._ (2015) M. Ziemann, Y. Chen, O. Kraft, E. Bayerschen, S. Wulfinghoff, C. Kirchlechner, N. Tamura, T. Böhlke, M. Walter, and P. Gruber, “Deformation patterns in cross-sections of twisted bamboo-structured Au microwires,” Acta Materialia 97, 216 – 222 (2015).
  * Stricker  _et al._ (2022) M. Stricker, M. Ziemann, M. Walter, S. M. Weygand, P. Gruber, and D. Weygand, “Dislocation structure analysis in the strain gradient of torsion loading: a comparison between modelling and experiment,” Modelling and Simulation in Materials Science and Engineering 30, 035007 (2022).
  * Stricker and Weygand (2015) M. Stricker and D. Weygand, “Dislocation multiplication mechanisms - glissile junctions and their role on the plastic deformation at the microscale,” Acta Materialia 99, 130–139 (2015).
  * Sills  _et al._ (2018) R. B. Sills, N. Bertin, A. Aghaei, and W. Cai, “Dislocation networks and the microstructural origin of strain hardening,” Phys. Rev. Lett. 121, 085501 (2018).
  * Stricker  _et al._ (2018) M. Stricker, M. Sudmanns, K. Schulz, T. Hochrainer, and D. Weygand, “Dislocation multiplication in stage ii deformation of fcc multi-slip single crystals,” Journal of the Mechanics and Physics of Solids 119, 319–333 (2018).
  * Akhondzadeh  _et al._ (2021) S. Akhondzadeh, N. Bertin, R. B. Sills, and W. Cai, “Slip-free multiplication and complexity of dislocation networks in fcc metals,” Materials Theory 5, 2 (2021).
  * Weygand  _et al._ (2015) D. Weygand, M. Mrovec, T. Hochrainer, and P. Gumbsch, “Multiscale simulation of plasticity in bcc metals,” Annual Review of Materials Research 45, 369–390 (2015), http://dx.doi.org/10.1146/annurev-matsci-070214-020852 .
  * Gao  _et al._ (2015) S. Gao, M. Fivel, A. Ma, and A. Hartmaier, “Influence of misfit stresses on dislocation glide in single crystal superalloys: A three-dimensional discrete dislocation dynamics study,” Journal of the Mechanics and Physics of Solids 76, 276–290 (2015).
  * Jogi and Bhattacharya (2021) T. Jogi and S. Bhattacharya, “Interfacial dislocation network in precipitation strengthened alloys during creep: a discrete dislocation dynamics (ddd) study in three dimensions,” Modelling and Simulation in Materials Science and Engineering 29, 035010 (2021).
  * Chen  _et al._ (2016) H. Chen, A. Kauffmann, B. Gorr, D. Schliephake, C. Seemüller, J. Wagner, H.-J. Christ, and M. Heilmaier, “Microstructure and mechanical properties at elevated temperatures of a new Al-containing refractory high-entropy alloy Nb-Mo-Cr-Ti-Al,” Journal of Alloys and Compounds 661, 206 – 215 (2016).
  * Sudmanns and El-Awady (2021) M. Sudmanns and J. A. El-Awady, “The effect of local chemical ordering on dislocation activity in multi-principle element alloys: A three-dimensional discrete dislocation dynamics study,” Acta Materialia 220, 117307 (2021).
  * Senger  _et al._ (2011) J. Senger, D. Weygand, C. Motz, P. Gumbsch, and O. Kraft, “Aspect ratio and stochastic effects in the plasticity of uniformly loaded micrometer-sized specimens,” Acta Materialia 59, 2937–2947 (2011).
  * Suzuki, Takeuchi, and Yoshinaga (2013) T. Suzuki, S. Takeuchi, and H. Yoshinaga,  _Dislocation dynamics and plasticity_ , Vol. 12 (Springer Science & Business Media, 2013).
  * Akhondzadeh  _et al._ (2023) S. Akhondzadeh, M. Kang, R. B. Sills, K. Ramesh, and W. Cai, “Direct comparison between experiments and dislocation dynamics simulations of high rate deformation of single crystal copper,” Acta Materialia 250, 118851 (2023).
  * Deka  _et al._ (2023) N. Deka, C. Alleman, D. L. Medlin, and R. B. Sills, “Energy and stochasticity: the yin and yang of dislocation patterning,” Materials Research Letters 11, 289–295 (2023), https://doi.org/10.1080/21663831.2022.2149283 .
  * Zhang  _et al._ (2022) C. Zhang, H. Song, D. Oliveros, A. Fraczkiewicz, M. Legros, and S. Sandfeld, “Data-mining of in-situ TEM experiments: On the dynamics of dislocations in CoCrFeMnNi alloys,” Acta Materialia 241, 118394 (2022).
  * Sills and Medlin (2022) R. B. Sills and D. L. Medlin, “Semi-Automated, Object-Based Tomography of Dislocation Structures,” Microscopy and Microanalysis 28, 633–645 (2022), https://academic.oup.com/mam/article-pdf/28/3/633/48199402/mam0633.pdf .
  * Yildirim  _et al._ (2023) C. Yildirim, H. F. Poulsen, G. Winther, C. Detlefs, P. H. Huang, and L. E. Dresselhaus-Marais, “Extensive 3d mapping of dislocation structures in bulk aluminum,” Scientific Reports 13, 3834 (2023).
  * Wang  _et al._ (2023) F. Wang, J.-C. Stinville, M. Charpagne, M. P. Echlin, S. R. Agnew, T. M. Pollock, M. D. Graef, and D. S. Gianola, “Dislocation cells in additively manufactured metallic alloys characterized by electron backscatter diffraction pattern sharpness,” Materials Characterization 197, 112673 (2023).
  * Steinberger, Gatti, and Sandfeld (2016) D. Steinberger, R. Gatti, and S. Sandfeld, “A universal approach towards computational characterization of dislocation microstructure,” JOM 68, 2065–2072 (2016).
  * Steinberger, Leimberger, and Sandfeld (2016) D. Steinberger, M. Leimberger, and S. Sandfeld, “D2c — converting and compressing discrete dislocation microstructure data,” TMS 2016 145th Annual Meeting & Exhibition: Supplemental Proceedings , 531–538 (2016).
  * Demirci  _et al._ (2023) A. Demirci, D. Steinberger, M. Stricker, N. Merkert, D. Weygand, and S. Sandfeld, “Statistical analysis of discrete dislocation dynamics simulations: initial structures, cross-slip and microstructure evolution,” Modelling and Simulation in Materials Science and Engineering 31, 075003 (2023).
  * Katzer  _et al._ (2024) B. Katzer, D. Betsche, K. Böhm, D. Weygand, and K. Schulz, “A graph database for feature characterization of dislocation networks,” Scripta Materialia 240, 115841 (2024).
  * Robertson and Kalidindi (2021) A. E. Robertson and S. R. Kalidindi, “Digital representation and quantification of discrete dislocation structures,” JOM (2021).
  * Hiemer, Fan, and Zaiser (2023) S. Hiemer, H. Fan, and M. Zaiser, “Relating plasticity to dislocation properties by data analysis: scaling vs. machine learning approaches,” Materials Theory 7, 1 (2023).
  * Weger, Gupta, and Hochrainer (2021) B. Weger, S. Gupta, and T. Hochrainer, “Analysing discrete dislocation data using alignment and curvature tensors,” Comptes Rendus. Physique (2021), 10.5802/crphys.60, online first.
  * Hochrainer  _et al._ (2014) T. Hochrainer, S. Sandfeld, M. Zaiser, and P. Gumbsch, “Continuum dislocation dynamics: Towards a physical theory of crystal plasticity,” Journal of the Mechanics and Physics of Solids 63, 167–178 (2014).
  * Ihsan, Fathalla, and Sandfeld (2023) A. Z. Ihsan, S. Fathalla, and S. Sandfeld, “Diso: A domain ontology for modeling dislocations in crystalline materials,” in  _Proceedings of the 38th ACM/SIGAPP Symposium on Applied Computing_ (2023) pp. 1746–1753.
  * Deringer  _et al._ (2021) V. L. Deringer, A. P. Bartók, N. Bernstein, D. M. Wilkins, M. Ceriotti, and G. Csányi, “Gaussian process regression for materials and molecules,” Chem. Rev. 121, 10073–10141 (2021).
  * Musil  _et al._ (2021a) F. Musil, A. Grisafi, A. P. Bartók, C. Ortner, G. Csányi, and M. Ceriotti, “Physics-inspired structural representations for molecules and materials,” Chem. Rev. 121, 9759–9815 (2021a).
  * Qamar  _et al._ (2023) M. Qamar, M. Mrovec, Y. Lysogorskiy, A. Bochkarev, and R. Drautz, “Atomic cluster expansion for quantum-accurate large-scale simulations of carbon,” J. Chem. Theory Comput. 19, 5151–5167 (2023).
  * Bochkarev  _et al._ (2022) A. Bochkarev, Y. Lysogorskiy, S. Menon, M. Qamar, M. Mrovec, and R. Drautz, “Efficient parametrization of the atomic cluster expansion,” Phys. Rev. Materials 6, 013804 (2022).
  * Cersonsky  _et al._ (2021) R. K. Cersonsky, B. A. Helfrecht, E. A. Engel, S. Kliavinek, and M. Ceriotti, “Improving sample and feature selection with principal covariates regression,” Machine Learning: Science and Technology 2, 035038 (2021).
  * Udofia and Stricker (2024) B. Udofia and M. Stricker, “Density field data & VTK files for dislocation embedding analysis,”  (2024), version v1.
  * Jogi, Udofia, and Stricker (2024) T. Jogi, B. Udofia, and M. Stricker, “Density field data, VTK files and Jupyter notebook for dislocation embedding analysis (with periodic boundary condition),”  (2024), version v3.
  * Arsenlis  _et al._ (2007) A. Arsenlis, W. Cai, M. Tang, M. Rhee, T. Oppelstrup, G. Hommes, T. G. Pierce, and V. V. Bulatov, “Enabling strain hardening simulations with dislocation dynamics,” Modelling and Simulation in Materials Science and Engineering 15, 553 (2007).
  * Weygand  _et al._ (2002) D. Weygand, L. H. Friedman, E. Van der Giessen, and A. Needleman, “Aspects of boundary-value problem solutions with three-dimensional dislocation dynamics,” Modelling and Simulation in Materials Science and Engineering 10, 437 (2002).
  * Weygand and Gumbsch (2005) D. Weygand and P. Gumbsch, “Study of dislocation reactions and rearrangements under different loading conditions,” Materials Science and Engineering: A 400, 158–161 (2005).
  * Frick  _et al._ (2010) C. P. Frick, B. G. Clark, A. S. Schneider, R. Maaß, S. Van Petegem, and H. Van Swygenhoven, “On the plasticity of small-scale nickel–titanium shape memory alloys,” Scripta Materialia 62, 492–495 (2010).
  * Maaß and Uchic (2012) R. Maaß and M. Uchic, “In-situ characterization of the dislocation-structure evolution in ni micro-pillars,” Acta materialia 60, 1027–1037 (2012).
  * Weekes  _et al._ (2015) H. Weekes, V. Vorontsov, I. Dolbnya, J. Plummer, F. Giuliani, T. Britton, and D. Dye, “In situ micropillar deformation of hydrides in zircaloy-4,” Acta Materialia 92, 81–96 (2015).
  * Malyar  _et al._ (2017) N. Malyar, J.-S. Micha, G. Dehm, and C. Kirchlechner, “Size effect in bi-crystalline micropillars with a penetrable high angle grain boundary,” Acta Materialia 129, 312–320 (2017).
  * Iwata  _et al._ (2004) T. Iwata, K. Saito, N. Ueda, S. Stromsten, T. Griffiths, and J. Tenenbaum, “Parametric embedding for class visualization,” Advances in neural information processing systems 17 (2004).
  * Roweis and Saul (2000) S. T. Roweis and L. K. Saul, “Nonlinear dimensionality reduction by locally linear embedding,” science 290, 2323–2326 (2000).
  * LeCun, Bengio, and Hinton (2015) Y. LeCun, Y. Bengio, and G. Hinton, “Deep learning,” nature 521, 436–444 (2015).
  * Sitzmann  _et al._ (2019) V. Sitzmann, J. Thies, F. Heide, M. Nießner, G. Wetzstein, and M. Zollhofer, “Deepvoxels: Learning persistent 3d feature embeddings,” in  _Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition_ (2019) pp. 2437–2446.
  * Goyal and Ferrara (2018) P. Goyal and E. Ferrara, “Graph embedding techniques, applications, and performance: A survey,” Knowledge-Based Systems 151, 78–94 (2018).
  * Zhang  _et al._ (2016) F. Zhang, N. J. Yuan, D. Lian, X. Xie, and W. Ma, “Collaborative knowledge base embedding for recommender systems,” in  _Proceedings of the 22nd ACM SIGKDD international conference on knowledge discovery and data mining_ (2016) pp. 353–362.
  * Tenenbaum, De Silva, and Langford (2000) J. B. Tenenbaum, V. De Silva, and J. C. Langford, “A global geometric framework for nonlinear dimensionality reduction,” science 290, 2319–2323 (2000).
  * Kruskal (1964) J. B. Kruskal, “Nonmetric multidimensional scaling: a numerical method,” Psychometrika 29, 115–129 (1964).
  * Borg and Groenen (2005) I. Borg and P. J. F. Groenen,  _Modern multidimensional scaling: Theory and applications_ (Springer Science & Business Media, 2005).
  * Pedregosa  _et al._ (2011) F. Pedregosa, G. Varoquaux, A. Gramfort, V. Michel, B. Thirion, O. Grisel, M. Blondel, P. Prettenhofer, R. Weiss, V. Dubourg, J. Vanderplas, A. Passos, D. Cournapeau, M. Brucher, M. Perrot, and E. Duchesnay, “Scikit-learn: Machine learning in Python,” Journal of Machine Learning Research 12, 2825–2830 (2011).
  * Suh  _et al._ (2020) C. Suh, C. Fare, J. A. Warren, and E. O. Pyzer-Knapp, “Evolving the materials genome: How machine learning is fueling the next generation of materials discovery,” Annual Review of Materials Research 50, null (2020), https://doi.org/10.1146/annurev-matsci-082019-105100 .
  * Steinberger, Song, and Sandfeld (2019) D. Steinberger, H. Song, and S. Sandfeld, “Machine learning-based classification of dislocation microstructures,” Frontiers in Materials 6, 141 (2019).
  * Musil  _et al._ (2021b) F. Musil, M. Veit, A. Goscinski, G. Fraux, M. J. Willatt, M. Stricker, T. Junge, and M. Ceriotti, “Efficient implementation of atom-density representations,” The Journal of Chemical Physics 154, 114109 (2021b).
  * Starkey, Hochrainer, and El-Azab (2022) K. Starkey, T. Hochrainer, and A. El-Azab, “Development of mean-field continuum dislocation kinematics with junction reactions using de rham currents and graph theory,” Journal of the Mechanics and Physics of Solids 158, 104685 (2022).