# Point Cloud Transformers applied to Collider Physics

Vinicius Mikuni1, Florencia Canelli1 1 University of Zurich,
Winterthurerstrasse 190, 8057 Zurich, Switzerland
<EMAIL_ADDRESS>
###### Abstract

Methods for processing point cloud information have seen a great success in
collider physics applications. One recent breakthrough in machine learning is
the usage of Transformer networks to learn semantic relationships between
sequences in language processing. In this work, we apply a modified
Transformer network called Point Cloud Transformer as a method to incorporate
the advantages of the Transformer architecture to an unordered set of
particles resulting from collision events. To compare the performance with
other strategies, we study jet-tagging applications for highly-boosted
particles.

##  1 Introduction

The interactions between elementary particles is described by the Standard
Model (SM) of particle physics. Particle colliders are then used to study
these interactions by comparing experimental signatures to SM predictions. At
every collision, hundreds of particles can be created, detected, and
reconstructed by particle detectors. Extracting relevant physics quantities
from this space is a challenging task that is often accomplished through the
usage of physics-motivated summary statistics, that reduce the data
dimensionality to manageable quantities. A recent approach is to interpret the
set of reconstructed particles as points in a point cloud. Point clouds
represent a set of unordered objects, described in a well-defined space, often
used for applications in self-driving vehicles, robotics, and augmented
reality, to name a few. With this approach, information from each bunch
crossing in a particle collider is interpreted as a point cloud, where the
goal is to use this high-dimensional set of reconstructed particles to extract
relevant information. However, extracting information from point clouds can
also be a challenging task. One novel approach is to use the Transformers
architecture [1] to learn the semantic relationship between objects.
Transformers yielded a great success in recent years applied to natural
language processing (NLP), often showing a superior performance when compared
to previous well-established methods. The advantage of this architecture is
the capability of learning semantic affinities between objects without losing
information during long sequences. Transformers are also easily
parallelizable, a huge computational advantage over sequential architectures
like gated recurrent [2] and long short-term memory [3] neural networks.
Applications of the Transformer network have already been applied to examples
outside NLP problems, with examples in image recognition [4, 5].

However, the original Transformer architecture is not readily applicable to
point clouds. In fact, since point clouds are intrinsically unordered, the
Transformer structure has to be modified to define a self-attention operation
that is invariant to permutations of the inputs. A recent approach introduced
in [6] addresses these issues through the development of Point Cloud
Transformer (PCT). In this approach, the input points of the point cloud are
first passed through a feature extractor to create a high dimensional
representation of the particles features. The transformed data is then used as
an input to a self-attention module. This module introduces attention
coefficients per particle that learn the relative pairwise importance of each
point in the set.

In this work, we will first describe the key features of the PCT architecture,
and then evaluate it in the context of a high energy physics task, in the form
of jet-tagging. Due to the nature of our example task the segmentation module
of the PCT is not required. Results are compared with other benchmark
implementations using three different public datasets.

##  2 Related works

Neural network architectures that treat collision events as point clouds have
recently grown in number given their state-of-the-art performance when applied
to different collider physics problems. A few examples of such applications
are jet-tagging [7, 8], secondary vertex finding [9], event and object
reconstruction [10, 11, 12, 13, 14], and jet parton assignment [15]. A
comprehensive review of the different methods is described in [16].

Two particular algorithms for jet-tagging will be relevant for the following
discussions of the PCT implementation. These are the ParticleNet [17] and
ABCNet [18] architectures. The former introduces the EdgeConv operation for
jet tagging, initially developed in [3]. This operation uses a k-nearest
neighbors approach to create local patches inside a point cloud. The local
information is then used to create high level features for each point that
retains the information of the local neighborhood. ABCNet, on the other hand,
uses the local information to define an attention mechanism that encoded the
neighborhood importance for each particle. This idea was first introduced in
[19] and applied in [20]. A similar concept of attention mechanisms are
defined for PCT, where a self-attention layer is used to provide the
relationship importance between all particles in the set.

Jet-tagging is a common task used to benchmark different algorithms applied to
collider physics. While a number of algorithms have been proposed in recent
years, special attention will be given to algorithms with results in public
datasets. In [17], results are presented for both quark gluon and top tagging
datasets, while [21] introduces a multiclassification sample containing five
different jet categories. The description of each dataset is discussed in Sec.
5.

##  3 PCT network

The Transformer implementation applied to point clouds requires two main
building blocks: the feature extractor and the self-attention (SA) layers. The
feature extractor is used to map the input point cloud
$F_{in}\in\mathbb{R}^{N\times\mathrm{d_{in}}}$ to a higher dimensional
representation Fe $\in\mathbb{R}^{N\times\mathrm{d_{out}}}$. This step is used
to achieve a higher level of abstraction for each point present in the point
cloud. In this work, two different strategies are compared. An architecture
consisting of stacked, one-dimensional convolutional (Conv1D) layers, and a
second option, based on EdgeConv blocks. The EdgeConv block consists of an
EdgeConv operation [3] followed by 2 two-dimensional convolutional (Conv2D)
layers and an average pooling operation across all neighbors. To ensure the
permutation invariance between particles is maintained, all convolutional
layers are implemented with stride and kernel size of 1 and unless otherwise
stated, are followed by a batch normalization operation and ReLU activation
function. All convolutional layers are implemented such that the particle
information is preserved, i.e for N particles with F features, the
convolutional operations only modify F while N is unchanged. The
EdgeConvolution operation uses a k-nearest neighbors approach to define a
vicinity for each point in the point cloud. This enhances the ability of the
network to extract information from a local neighborhood around each point.

The first strategy is referred to simple PCT (SPCT) while the second will be
referred to just PCT.

The second main building block is the usage of an offset attention defined as
a self-attention (SA) layer. The output of the feature extractor Fe is used as
the input of the first SA layer. The goal of the SA layer is to determine the
relationship between all particles of the point cloud through an offset
attention mechanism. This approach differs from the one taken in ABCNet, where
a self-attention and neighboring attention coefficients are defined for a
neighborhood of each particle.

In the same terms defined in the original Transformer [1] work, three
different matrices, all built from linear transformations of the original
inputs. These matrices are called query (Q), key (K), and value (V). Attention
weights are created by matching Q and K through matrix multiplication. These
attention weights, after normalization, represent the weighted importance
between each pair of particles. The self-attention is then the result of the
weighted elements of V, defined as the result of the matrix multiplication
between the attention weights and the value matrix. The linear transformations
are accomplished through the usage of Conv1D layers such that:

${}\mathrm{Q,K,V}{}$ | ${}=\mathrm{F}_{e}.\mathrm{(W_{q},W_{k},W_{v})}{}$
---|---
${}\mathrm{Q,K}{}$ | ${}\in\mathbb{R}^{N\times\mathrm{d_{a}}},\mathrm{V}\in\mathbb{R}^{N\times\mathrm{d_{out}}}.{}$
(1)

The dimension $\mathrm{d_{a}}$ of the Q and K matrices is independent from the
dimension $\mathrm{d_{out}}$. In this work, and similarly to the choice used
in the original PCT implementation, $\mathrm{d_{a}}$ is fixed to
$\mathrm{d_{out}}/4$. While other options can be used, we have not observed
further improvements for different choices of $\mathrm{d_{a}}$. To maintain
the linear operations in these definitions, no batch normalization or
activation function are used. The matrices $\mathrm{(W_{q},W_{k},W_{v})}$
contain the trainable linear coefficients introduced by the convolutional
operation. The attention weights (A) are then calculated by first multiplying
the query matrix with the transpose of the key matrix followed by a softmax
operation:

$\mathrm{A}=\mathrm{Softmax}(\mathrm{Q.K}^{\mathrm{T}})/N,\mathrm{A}\in\mathbb{R}^{N\times
N}.$ (2)

The softmax operation is applied to each row of A to normalize the
coefficients for all points, while the columns are normalized using the L1
norm (represented with N in the equation), after the Softmax operation. This
decision follows the approach introduced in the PCT model, different from the
original Transformer implementation that used $1/\sqrt{d}$, with $d$ the
embedding dimension.

The attention weights are then multiplied by the value matrix, resulting in
the self-attention Fsa with

$\mathrm{F}_{sa}=\mathrm{A\cdot
V},\mathrm{F}_{sa}\in\mathbb{R}^{N\times\mathrm{d_{out}}}.$ (3)

While the original Transformer architecture uses Fsa to derive an absolute
attention per object, it was observed by the PCT authors, and later revisited
in Section 6, that the usage of an offset-attention might result in a superior
classification performance. The offset-attention defines the self-attention
coefficients as per particle modifications of the inputs, rather than an
absolute quantity per particle. To calculate the offset-attention, first the
difference between Fe and Fsa is calculated as the initial offset. This
initial offset is then passed through a Conv1D layer with same output
dimension $\mathrm{d_{out}}$. The result of this layer is now the offset added
to the initial inputs Fe. Different levels of abstraction can be achieved by
stacking multiple SA layers, using the output of each SA layer as the input
for the next.

To complete the general architecture, the SA layers are combined through a
simple concatenation over the feature dimension, followed by a mean
aggregation, resulting in the overall means of each feature across all the
particles, differently from the original PCT implementation that instead uses
a maximum pooling operation. The output of this operation is passed through
fully connected layers before reaching the output layer, normalized through a
softmax operation. Similarly to the convolutional layers, all fully connected
layers besides the output layers are followed by a batch normalization and
ReLU activation function.

The general PCT network and the main building blocks are shown in Fig. 1. The
training details are explained in Sec. 4

Figure 1: General network architecture (left), feature extractor (middle), and
self-attention layer description (right). din, dout, and da represent the
input and output feature sizes and the dimension used in the definition of the
key and query matrices. The ReLu activation function, followed by a batch
normalization operation are implied in all convolutional and fully connected
layers, besides the ones used in the SA layers to define the linear
transformations.

##  4 Training details

The PCT implementation is done using Tensorflow v1.14 [22]. A Nvidia GTX 1080
Ti graphics card is used for the training and evaluation steps. For all
architectures, the Adam optimiser [23] is used with a learning rate starting
from 0.001 and decreasing by a factor 2 every 20 epochs, to a minimum of 1e-6.
The training is performed with a mini batch size of 64 and a maximum number of
200 epochs. If no improvement is observed in the evaluation set for 15
consecutive epochs, the training is stopped. The epoch with the lowest
classification loss on the test set is stored for further evaluation.

##  5 Jet classification

Different performance metrics are compared for (S)PCT applied to a jet
classification task on different public datasets. Jets are collimated sprays
of particles resulting from the hadronization and fragmentation of energetic
partons. Jets can show distinguishing radiation patterns depending on the
elementary particle that has initiated the jet. Traditional methods use this
information to define physics-motivated observables [24] that can distinguish
different jet categories.

The PCT architecture uses two EdgeConv blocks, each defining the k-nearest
neighbors of each point with k = 20. The initial distances are calculated in
the pseudorapidity-azimuth ($\eta-\phi$) space of the form $\Delta
R=\sqrt{\Delta\eta^{2}+\Delta\phi^{2}}$. The distances used for the second
EdgeConv block are calculated using the full-feature space produced in the
output of the last EdgeConv block.

Besides the feature extractor, PCT uses three SA layers while SPCT uses two.
The output of all SA layers are concatenated in the feature dimension for both
PCT and SPCT. The output of the last EdgeConv block is also added during
concatenation with a skip connection. The detailed architectures used during
training for PCT and SPCT are shown in Fig. 2.

Figure 2: SPCT (left) and PCT (middle) architectures used for all jet-tagging
classification tasks. The EdgeConv block structure used in PCT is shown in the
right. Numbers in parenthesis represent layer sizes.

PCT and SPCT receive as inputs the particles found inside the jets. The input
features vary between applications, depending on the available content for
each public dataset. For all comparisons, up to 100 particles per jet are
used. If more particles were found inside a jet, the event is truncated,
otherwise zero-padded up to 100. The choice of 100 particles cover the
majority of the jets stored in all datasets without truncation (more than 99%
of the cases for all datasets). To handle the zero-padded particles, the
original PCT implementation has been modified such that these particles are
masked during the k-nearest neighbors calculation and are replaced by large
negative values to the matrix A defined in Eq. 2 such that the following
softmax operation will result in zeros for zero-padded entries.

###  5.1 HLS4ML LHC Jet dataset

For this study, samples containing simulated jets originating from W bosons, Z
bosons, top quarks, light quarks, and gluons produced at $\sqrt{s}$ = 13 TeV
proton-proton collisions are used. The samples are available at [25]. This
dataset is created and configured using a parametric description of a generic
LHC detector, described in [26, 27]. The jets are clustered with the
anti-$k_{T}$ algorithm [28] with radius parameter R = 0.8, while also
requiring that the jet’s transverse momentum is around 1 TeV, which ensures
that most of the decay products of the generated particles are found inside a
single jet. The training and testing sets contain 567k and 63k jets
respectively. The performance comparison is reported using the official
evaluation set, containing 240k jets.

For each jet, the momenta of all particle constituents are available. For each
particle, a set of 16 kinematic features are used, matching the ones used in
[21], to facilitate the comparison. All input features available in the
dataset are used without modification, with exception to the momenta and
energy of the particles, which are taken after a logarithmic function is
applied to limit the feature range.

The area under the curve (AUC) for each class is calculated in the one-vs-rest
strategy. The results of the AUC are shown in Table 1 while the true positive
rate (TPR) for a fixed false positive rate (FPR) at 10% and 1% are shown in
Table 2. PCT has the overall higher AUC compared to other algorithms, followed
closely by SPCT and JEDI-net. For fixed FPR values, PCT also shows a good
performance for all categories, achieving the best result for almost all
categories and FPR values.

Table 1: Area under the curve and accuracy for each jet category reported on the HLS4ML LHC Jet dataset. Results for all algorithms are taken as the average of 10 trainings with random network initialization. If the uncertainty is not quoted then the variation is negligible compared to the expected value. Bold results represent the algorithm with highest performance. All results besides (S)PCT are taken from [21]. Algorithm | Gluon | Light quark | W boson | Z boson | Top quark
---|---|---|---|---|---
|  | AUC |  |  |
DNN | 0.9384 | 0.9026 | 0.9537 | 0.9459 | 0.9620
GRU | 0.9040 | 0.8962 | 0.9192 | 0.9042 | 0.9350
CNN | 0.8945 | 0.9007 | 0.9102 | 0.8994 | 0.9494
JEDI-net | 0.9529 | 0.9301 | 0.9739 | 0.9679 | 0.9683
JEDI-net with $\sum O$ | 0.9528 | 0.9290 | 0.9695 | 0.9649 | 0.9677
SPCT | 0.9537 | 0.9326 | 0.9740 | 0.9779 | 0.9693
PCT | 0.9623 | 0.9414 | 0.9789 | 0.9814 | 0.9757
Table 2: True positive rate (TPR) for fixed values of the false positive rate (FPR). Results reported are the average of 10 independent trainings with random network initialization. Bold results represent the algorithm with largest TPR value. All results besides (S)PCT are taken from [21]. Algorithm | Gluon | Light quark | W boson | Z boson | Top quark
---|---|---|---|---|---
TPR for FPR=10% |  |  |  |  |
DNN | $0.830\pm 0.002$ | $0.715\pm 0.002$ | $0.855\pm 0.001$ | $0.833\pm 0.002$ | $0.917\pm 0.001$
GRU | $0.740\pm 0.014$ | $0.746\pm 0.011$ | $0.812\pm 0.035$ | $0.753\pm 0.036$ | $0.867\pm 0.006$
CNN | $0.700\pm 0.008$ | $0.740\pm 0.003$ | $0.760\pm 0.005$ | $0.721\pm 0.006$ | $0.889\pm 0.001$
JEDI-net | $0.878\pm 0.001$ | $0.822\pm 0.001$ | 0.938 $\pm$ 0.001 | $0.910\pm 0.001$ | $0.930\pm 0.001$
JEDI-net with $\sum O$ | $0.879\pm 0.001$ | $0.818\pm 0.001$ | $0.927\pm 0.001$ | $0.903\pm 0.001$ | $0.931\pm 0.001$
SPCT | $0.864\pm 0.003$ | $0.807\pm 0.002$ | $0.921\pm 0.001$ | $0.940\pm 0.002$ | $0.926\pm 0.002$
PCT | 0.891 $\pm$ 0.001 | 0.833 $\pm$ 0.001 | $0.932\pm 0.001$ | 0.946 $\pm$ 0.001 | 0.941 $\pm$ 0.001
TPR for FPR=1% |  |  |  |  |
DNN | $0.420\pm 0.002$ | $0.178\pm 0.002$ | $0.656\pm 0.002$ | $0.715\pm 0.001$ | $0.651\pm 0.003$
GRU | $0.273\pm 0.018$ | $0.220\pm 0.037$ | $0.249\pm 0.057$ | $0.386\pm 0.060$ | $0.426\pm 0.020$
CNN | $0.257\pm 0.005$ | $0.254\pm 0.007$ | $0.232\pm 0.006$ | $0.291\pm 0.005$ | $0.504\pm 0.005$
JEDI-net | $0.485\pm 0.001$ | 0.302 $\pm$ 0.001 | $0.704\pm 0.001$ | $0.769\pm 0.001$ | $0.633\pm 0.001$
JEDI-net with $\sum O$ | $0.482\pm 0.001$ | $0.301\pm 0.001$ | $0.658\pm 0.001$ | $0.729\pm 0.001$ | $0.632\pm 0.001$
SPCT | $0.467\pm 0.002$ | $0.283\pm 0.002$ | $0.809\pm 0.003$ | $0.753\pm 0.002$ | $0.628\pm 0.003$
PCT | 0.513 $\pm$ 0.002 | $0.298\pm 0.002$ | 0.834 $\pm$ 0.001 | 0.781 $\pm$ 0.001 | 0.700 $\pm$ 0.003

###  5.2 Top tagging dataset

The top tagging dataset consists of jets containing the hadronic decay
products of top quarks together with jets generated through QCD dijet events.
The dataset is available at [29]. The events are generated with Pythia8 [30]
with detector simulation done through Delphes [31]. The jets are clustered
with the anti-$k_{T}$ algorithm with radius parameter R = 0.8. Only jets with
transverse momentum $p_{T}\in[550,650]$ GeV and rapidity $|y|<2$ are kept. The
official training, testing, and evaluation splitting are used, containing
1.2M/400k/400k events respectively. For each particle, a set of 7 input
features is used, based only on the momenta of each particle clustered inside
the jet. The input features per particle are the same ones used in [17] to
facilitate the comparison between algorithms. Considering jets containing top
quarks as the signal, the AUC and background rejection power, defined as the
inverse of the background efficiency for a fixed signal efficiency, are listed
in Tab. 3, with a reduced number of algorithms as reported in [17]. A more
complete, although slightly outdated list is available at [32].

Table 3: Comparison between the performance reported for different classification algorithms on the top tagging dataset. The uncertainty quoted corresponds to the standard deviation of nine trainings with different random weight initialization. If the uncertainty is not quoted then the variation is negligible compared to the expected value. Bold results represent the algorithm with highest performance. | Acc | AUC | 1/$\epsilon_{B}$ ($\epsilon_{S}=0.5$) | 1/$\epsilon_{B}$ ($\epsilon_{S}=0.3$)
---|---|---|---|---
ResNeXt-50 [17] | 0.936 | 0.9837 | 302$\pm$5 | 1147$\pm$58
P-CNN [17] | 0.930 | 0.9803 | 201$\pm$4 | 759$\pm$24
PFN [33] | - | 0.9819 | 247$\pm$3 | 888$\pm$17
ParticleNet-Lite [17] | 0.937 | 0.9844 | 325$\pm$5 | 1262$\pm$49
ParticleNet [17] | 0.940 | 0.9858 | 397$\pm$7 | 1615$\pm$93
JEDI-net [21] | 0.9263 | 0.9786 | - | 590.4
JEDI-net with $\sum O$ [21] | 0.9300 | 0.9807 | - | 774.6
SPCT | 0.928 | 0.9799 | 201$\pm$9 | 725$\pm$54
PCT | 0.940 | 0.9855 | 392$\pm$7 | 1533$\pm$101

###  5.3 Quark and gluon dataset

The dataset used for the studies are available from [33], providing for each
particle the same information present in the top tagging dataset (the momenta
of the particles) with additional information that identifies the particle
type as either electron, muon, charged hadron, neutral hadron, or photon. It
consists of stable particles clustered into jets, excluding neutrinos, using
the anti-$k_{T}$ algorithm with R = 0.4. The quark-initiated sample (treated
as signal) is generated using a Z($\nu\nu$) + $(u,d,s)$ while the gluon-
initiated data (treated as background) are generated using Z($\nu\nu$) +$g$
processes. Both samples are generated using Pythia8 [30] without detector
effects. Jets are required to have transverse momentum $p_{T}\in[500,550]$ GeV
and rapidity $|y|<1.7$ for the reconstruction. For the training, testing and
evaluation, the recommended splitting is used with 1.6M/200k/200k events
respectively. Each particle contains the four momentum and the expected
particles type (electron, muon, photon, or charged/neutral hadrons). For each
particle, a set of 13 kinematic features is used. These features are chosen to
match the ones used in [17, 18]. The AUC and background rejection power are
listed in Tab. 4.

Table 4: Comparison between the performance reported for different classification algorithms on the quark and gluon dataset. The uncertainty quoted corresponds to the standard deviation of nine trainings with different random weight initialization. If the uncertainty is not quoted then the variation is negligible compared to the expected value. Bold results represent the algorithm with best performance. | Acc | AUC | 1/$\epsilon_{B}$ ($\epsilon_{S}=0.5$) | 1/$\epsilon_{B}$ ($\epsilon_{S}=0.3$)
---|---|---|---|---
ResNeXt-50 [17] | 0.821 | 0.9060 | 30.9 | 80.8
P-CNN [17] | 0.827 | 0.9002 | 34.7 | 91.0
PFN [33] | - | 0.9005 | 34.7$\pm$0.4 | -
ParticleNet-Lite [17] | 0.835 | 0.9079 | 37.1 | 94.5
ParticleNet [17] | 0.840 | 0.9116 | 39.8$\pm$0.2 | 98.6$\pm$1.3
ABCNet [18] | 0.840 | 0.9126 | 42.6$\pm$0.4 | 118.4$\pm$1.5
SPCT | 0.815 | 0.8910 | 31.6$\pm$0.3 | 93.0$\pm$1.2
PCT | 0.841 | 0.9140 | 43.2$\pm$0.7 | 118.0$\pm$2.2

##  6 Ablation study

To understand the benefits of the self-attention and the architecture choices
for the development of (S)PCT applied to jet-tagging, we provide the
comparison between the results attained by (S)PCT shown in the previous
sections with different implementation choices. We start by comparing the
benefit the SA layers bring to the classification by training the (S)PCT model
without the SA layers, effectively taking the output of the feature extractor
and passing it through the mean aggregation layer before the fully connected
layers. While describing the PCT architecture, an offset-attention was used
instead of the self-attention introduced in the original Transformer
architecture. The comparison of both options has also been performed, where
Fsa, introduced in Eq. 3, is used as the output of the SA layer. These
comparisons are summarized in Tables 5 and 6 with the HLS4ML LHC Jet dataset,
Table 7 with the top tagging dataset, and 8 with the quark and gluon dataset.
The values reported for the different scenarios are the result of a single
training (hence no uncertainties provided), but is has been verified that the
results shown are stable with additional trainings.

Table 5: Area under the curve for each jet category of the HLS4ML LHC Jet dataset for different implementations of the (S)PCT architecture. Values in parenthesis represent the percentage difference with the standard implementation. Algorithm | Gluon | Light quark | W boson | Z boson | Top quark
---|---|---|---|---|---
|  |  | AUC |  |
SPCT | 0.9537 | 0.9326 | 0.9740 | 0.9779 | 0.9693
SPCT w/o attention | 0.9399 (-1.4%) | 0.918 (-1.6%) | 0.9501 (-2.5%) | 0.9641 (-1.4%) | 0.956 (-1.4%))
SPCT w/o offset-attention | 0.9543 (+0.1%) | 0.932 (-0.1%) | 0.9737 (-0.0%) | 0.9772 (-0.1%) | 0.9691 (-0.0%)
PCT | 0.9623 | 0.9414 | 0.9789 | 0.9814 | 0.9757
PCT w/o attention | 0.9578 (-0.5%) | 0.9362 (-0.6%) | 0.9736 (-0.5%) | 0.9774 (-0.4%) | 0.9707 (-0.5%)
PCT w/o offset-attention | 0.9615 (-0.1%) | 0.9404 (-0.1%) | 0.9785 (-0.0%) | 0.9812 (-0.0%) | 0.9749 (-0.1%)
Table 6: True positive rate (TPR) for fixed values of the false positive rate (FPR) for different implementations of the (S)PCT architecture evaluated in the HLS4ML LHC Jet dataset. Values in parenthesis represent the percentage difference with the standard implementation. Algorithm | Gluon | Light quark | W boson | Z boson | Top quark
---|---|---|---|---|---
TPR for FPR=10% |  |  |  |  |
SPCT | $0.864$ | $0.807$ | $0.921$ | $0.940$ | $0.926$
SPCT w/o attention | 0.826 (-4.4%) | 0.775 (-4.0%) | 0.82 (-11.0%) | 0.911 (-3.1%) | 0.902 (-2.6%)
SPCT w/o offset-attention | 0.866 (+0.2%) | 0.805 (-0.2%) | 0.92 (-0.1%) | 0.938 (-0.2%) | 0.924 (-0.2%)
PCT | 0.891 | 0.833 | $0.932$ | 0.946 | 0.941
PCT w/o attention | 0.878 (-1.5%) | 0.819 (-1.7%) | 0.922 (-1.1%) | 0.937 (-1.0%) | 0.929 (-1.3%)
PCT w/o offset-attention | 0.867 (-2.7%) | 0.829 (-0.5%) | 0.932 (-0.0%) | 0.946 (-0.0%) | 0.939 (-0.2%)
TPR for FPR=1% |  |  |  |  |
SPCT | $0.467$ | $0.283$ | 0.809 | $0.753$ | $0.628$
SPCT w/o attention | 0.422 (-9.6%) | 0.263 (-7.1%) | 0.697 (-13.8%) | 0.594 (-21.1%) | 0.51 (-18.8%)
SPCT w/o offset-attention | 0.469 (+0.4%) | 0.282 (-0.4%) | 0.806 (-0.4%) | 0.747 (-0.8%) | 0.63 (+0.3%)
PCT | 0.513 | $0.298$ | 0.834 | 0.781 | 0.700
PCT w/o attention | 0.493 (-3.9%) | 0.289 (-3.0%) | 0.797 (-4.4%) | 0.745 (-4.6%) | 0.648 (-7.4%)
PCT w/o offset-attention | 0.507 (-1.2%) | 0.297 (-0.3%) | 0.833 (+0.1%) | 0.782 (–0.1%) | 0.683 (-2.4%)
Table 7: Performance comparison for different (S)PCT implementations on the top tagging dataset. Values in parenthesis represent the percentage difference with the standard implementation. | Acc | AUC |  1/$\epsilon_{B}$ ($\epsilon_{S}=0.5$) |  1/$\epsilon_{B}$ ($\epsilon_{S}=0.3$)
---|---|---|---|---
SPCT | 0.928 | 0.9799 | 201 | 725
SPCT w/o attention | 0.91 (-1.9%) | 0.964 (-1.6%) | 52 (-74.1%) | 130 (-82.1%)
SPCT w/o offset-attention | 0.926 (-0.2%) | 0.9789 (-0.1%) | 183 (-9.0%) | 645 (-11.0%)
PCT | 0.940 | 0.9855 | 392 | 1533
PCT w/o attention | 0.938 (-0.2%) | 0.9846 (-0.1%) | 330 (-15.8%) | 1177 (-23.2%)
PCT w/o offset-attention | 0.939 (-0.1%) | 0.9849 (-0.1%) | 350 (-10.7%) | 1343 (-12.4%)
Table 8: Performance comparison for different (S)PCT implementations on the quark gluon dataset. Values in parenthesis represent the percentage difference with the standard implementation. | Acc | AUC |  1/$\epsilon_{B}$ ($\epsilon_{S}=0.5$) |  1/$\epsilon_{B}$ ($\epsilon_{S}=0.3$)
---|---|---|---|---
SPCT | 0.815 | 0.8910 | 31.6 | 93.0
SPCT w/o attention | 0.803 (-1.5%) | 0.8786 (-1.4%) | 27.1 (-14.2%) | 79.8 (-14.2%)
SPCT w/o offset-attention | 0.801 (-1.7%) | 0.8765 (-1.6%) | 26.9 (-14.9%) | 78.5 (-15.6%)
PCT | 0.841 | 0.9140 | 43.2 | 118.0
PCT w/o attention | 0.826 (-1.8%) | 0.9012 (-1.4%) | 34.0 (-21.3%) | 97.4 (-17.5%)
PCT w/o offset-attention | 0.823 (-2.1%) | 0.898 (-1.8%) | 31.6 (-26.9%) | 86.9 (-26.4%)

The implementation of PCT without the attention modules results in an
architecture that is similar to the ParticleNet-Lite implementation from [17],
also resulting in similar performance for both top tagging and quark gluon
datasets. The differences for SPCT with and without the attention modules are
much bigger compared to PCT. Without the attention layers, SPCT does not have
access to the information shared between particles, resulting in worse
performance. Since for PCT the neighboring information is already used in the
feature extractor, this effect is mitigated. Compared to the original self-
attention implementation, the offset-attention shows better performance, which
is more noticeable in the quark gluon dataset. The quark gluon dataset uses
the same input features per particle as the top tagging dataset, but with an
additional particle identifier (PID) information, encoded as binary entries to
categorize the particles into five categories: muons, electrons, photons,
charged hadrons, and neutral hadrons. Given that results without attention for
the quark gluon dataset also seem to yield better performance compared to the
standard attention, this might indicate that binary input features are not
correctly handled by the standard attention implementation. In the offset-
attention, the module calculates an offset of the inputs per particle, relying
less on the absolute value of the PID flag.

##  7 Computational complexity

Besides the algorithm performance, the computational cost is also an important
figure of merit. To compare the amount of computational resources required to
evaluate each model, the number of trainable weights and the number of
floating point operations (FLOPs) are computed. The comparison of these
quantities for different algorithms are shown in Tab. 9.

Table 9: Number of trainable weights and floating point operations (FLOPs) for each model under consideration. Missing entries are left for implementations that do not report an official value. Algorithm | Weights | FLOPs
---|---|---
ResNeXt-50 [17] | 1.46M | -
P-CNN [17] | 348k | -
PFN [33] | 82k | -
ParticleNet-Lite [17] | 26k | -
ParticleNet [17] | 366k | -
ABCNet [18] | 230k | -
DNN [21] | 14.7k | 27k
GRU [21] | 15.6k | 46k
CNN [21] | 205.5k | 400k
JEDI-net [21] | 33.6k | 116M
JEDI-net with $\sum O$ [21] | 8.8k | 458M
SPCT | 7.0k | 2.4M
PCT | 193.3k | 266M

While PCT shows a better overall AUC compared to SPCT, the improvement in
performance from the usage of EdgeConv blocks comes with a cost in
computational complexity. SPCT, on the other hand, provides a good balance
between performance and computational cost, resulting in more than 100 times
less FLOPs and almost 30 times less trainable weights compared to PCT.
Although JEDI-net shows similar performance to SPCT, the implementation takes
as inputs 150 particles with the model based on $\bar{O}$ sums appearing with
a large FLOP count of 458M. This number however can be decreased with a sparse
implementation, since many of these operations are $\times 0$ and $\times 1$
products.

##  8 Visualization

The SA module defines the relative importance between all points in the set
through the attention weights. We can use this information to identify the
regions inside a jet that have high importance for a chosen particle. To
visualize the particle importance, the HLS4ML LHC jet dataset is used to
create a pixelated image of a jet in the transverse plane. The average jet
image of 100k examples in the evaluation set is used. For each image, a simple
preprocessing strategy is applied to align the different images. First, the
whole jet is translated such that the particle with the highest transverse
momentum in the jet is centered at (0,0). This particle is also used as the
reference particle from where attention weights are shown. Next, the full jet
image is rotated, making the second most energetic particle aligned with the
positive y-coordinate. Lastly, the image is flipped in the x-coordinate in
case the third most energetic particle is located on the negative x-axis,
otherwise the image is left as is. These transformations are also used in
other jet image studies such as [34, 18]. The pixel intensity for each jet
image is taken from the attention weights after the softmax operation is
applied, expressing the particle importance with respect to the most energetic
particle in the event. A comparison of the extracted images for each SA layer
and for each jet category is shown in Fig. 3 .

Figure 3: Average jet image for each jet category (columns) and for each self-
attention layer (rows). The pixel intensities represent the overall particle
importance compared to the most energetic particle in the jet.

The different SA layers are able to extract different information for each
jet. In particular, the jet substructure is exploited, resulting in an
increased relevance to harder subjets in the case of Z boson, W boson, and top
quark initiated jets. On the other hand, light quark and gluon initiated jets
have a more homogeneous radiation pattern, resulting also in a more
homogeneous picture.

##  9 Summary of experimental results

In this section, the summary of the comparisons made using different datasets
is given. In the HLS4ML dataset, both PCT and SPCT show state-of-the-art
performance, resulting in superior AUC compared to other approaches. Although
the AUC provides a general idea of the performance, the TPR at fixed FPR
thresholds represent a more realistic application of the algorithm in high
energy physics, where one is interested in maximizing TPR while having control
of the background level. in these comparisons again PCT and SPCT show a
superior performance for the majority of the jet categories. In the top
tagging and quark gluon datasets, a similar picture is shown, with PCT and
SPCT again performing well compared to other algorithms. In the top tagging
dataset, the results achieved with PCT and ParticleNet are similar and
compatible within training uncertainties. This observation suggests that the
information encoded through EdgeConv layers might already provide enough model
abstraction to perform the classification. On the other hand, in the quark
gluon dataset, the performance of PCT compared to ParticleNet shows a 20%
improvement in background rejection power at 30% signal efficiency. This
performance is similar to the one reported by the ABCNet architecture that
implements attention through graph attention pooling layers [35], evidencing
some of the benefits of the usage of attention mechanisms.

##  10 Conclusion

In this work, a new method based on the Transformer architecture was applied
to a high energy physics application. The point cloud transformer (PCT)
modifies the usual Transformer architecture to be applied to a set of
unordered points present in a point cloud. This method has the advantage of
extracting semantic affinities between the points through the development of a
self-attention mechanism. We evaluate the performance of this architecture
applied to several jet-tagging datasets by testing two different
implementations, one that exploits the neighborhood information through
EdgeConv operations and a simpler form that connects all points through
convolutional layers called simple PCT (SPCT). Both approaches have shown
state-of-the-art performance compared to other publicly available results.
While the classification performance of SPCT is slightly lower compared to the
standard PCT, the number of floating point operations required to evaluate the
model decreases by almost a factor 20. This reduced computational complexity
can be exploited in environments with limited computing resources or
applications that require fast inference responses.

A different advantage of (S)PCT is the visualization of the self-attention
coefficients to understand which points have a greater importance through the
classification task. Traditional methods often define physics-motivated
observables to distinguish the different types of jets. PCT, on the other
hand, exploits subjet information by learning affinities on a particle-by-
particle basis, resulting in images with distinct features for jets of
different decay modes.

##  11 Acknowledgements

The authors would like to thank Jean-Roch Vlimant for helpful comments during
the development of this work. This research was supported in part by the Swiss
National Science Foundation (SNF) under contract No. 200020-182037 and
Forschungskredit of the Universityof Zurich, grant no. FK-20-097.

##  12 References

## References

  * [1] A. Vaswani, N. Shazeer, N. Parmar, J. Uszkoreit, L. Jones, A. N. Gomez, L. u. Kaiser, and I. Polosukhin, Attention is all you need, in Advances in Neural Information Processing Systems (I. Guyon, U. V. Luxburg, S. Bengio, H. Wallach, R. Fergus, S. Vishwanathan, and R. Garnett, eds.), vol. 30, Curran Associates, Inc., 2017.
  * [2] J. Chung, C. Gulcehre, K. Cho, and Y. Bengio, Empirical evaluation of gated recurrent neural networks on sequence modeling, in NIPS 2014 Workshop on Deep Learning, December 2014, 2014.
  * [3] Y. Wang, Y. Sun, Z. Liu, S. E. Sarma, M. M. Bronstein, and J. M. Solomon, Dynamic graph cnn for learning on point clouds, ACM Transactions on Graphics (TOG) (2019).
  * [4] B. Wu, C. Xu, X. Dai, A. Wan, P. Zhang, Z. Yan, M. Tomizuka, J. Gonzalez, K. Keutzer, and P. Vajda, Visual transformers: Token-based image representation and processing for computer vision, 2020.
  * [5] A. Dosovitskiy, L. Beyer, A. Kolesnikov, D. Weissenborn, X. Zhai, T. Unterthiner, M. Dehghani, M. Minderer, G. Heigold, S. Gelly, J. Uszkoreit, and N. Houlsby, An image is worth 16x16 words: Transformers for image recognition at scale, 2020.
  * [6] M.-H. Guo, J.-X. Cai, Z.-N. Liu, T.-J. Mu, R. R. Martin, and S.-M. Hu, Pct: Point cloud transformer, 2020.
  * [7] P. T. Komiske, E. M. Metodiev, and J. Thaler, Energy Flow Networks: Deep Sets for Particle Jets, JHEP 01 (2019) 121, [arXiv:1810.05165].
  * [8] M. J. Dolan and A. Ore, Equivariant Energy Flow Networks for Jet Tagging, arXiv:2012.00964.
  * [9] J. Shlomi, S. Ganguly, E. Gross, K. Cranmer, Y. Lipman, H. Serviansky, H. Maron, and N. Segol, Secondary Vertex Finding in Jets with Neural Networks, arXiv:2008.02831.
  * [10] S. R. Qasim, J. Kieseler, Y. Iiyama, and M. Pierini, Learning representations of irregular particle-detector geometry with distance-weighted graph networks, Eur. Phys. J. C 79 (2019), no. 7 608, [arXiv:1902.07987].
  * [11] M. J. Fenton, A. Shmakov, T.-W. Ho, S.-C. Hsu, D. Whiteson, and P. Baldi, Permutationless Many-Jet Event Reconstruction with Symmetry Preserving Attention Networks, arXiv:2010.09206.
  * [12] J. Duarte and J.-R. Vlimant, Graph Neural Networks for Particle Tracking and Reconstruction, arXiv:2012.01249.
  * [13] X. Ju et al., Graph Neural Networks for Particle Reconstruction in High Energy Physics detectors, in 33rd Annual Conference on Neural Information Processing Systems, 3, 2020.  arXiv:2003.11603.
  * [14] J. Pata, J. Duarte, J.-R. Vlimant, M. Pierini, and M. Spiropulu, MLPF: Efficient machine-learned particle-flow reconstruction using graph neural networks, arXiv:2101.08578.
  * [15] J. S. H. Lee, I. Park, I. J. Watson, and S. Yang, Zero-Permutation Jet-Parton Assignment using a Self-Attention Network, arXiv:2012.03542.
  * [16] J. Shlomi, P. Battaglia, and J.-R. Vlimant, Graph Neural Networks in Particle Physics, arXiv:2007.13681.
  * [17] H. Qu and L. Gouskos, Jet tagging via particle clouds, Phys. Rev. D 101 (Mar, 2020) 056019.
  * [18] V. Mikuni and F. Canelli, ABCNet: An attention-based method for particle tagging, Eur. Phys. J. Plus 135 (2020), no. 6 463, [arXiv:2001.05311].
  * [19] P. Velickovic, G. Cucurull, A. Casanova, A. Romero, P. Lio, and Y. Bengio, Graph attention networks, 2017.
  * [20] C. Chen, L. Z. Fragonara, and A. Tsourdos, GAPNet: Graph Attention based Point Neural Network for Exploiting Local Feature of Point Cloud, arXiv:1905.08705.
  * [21] E. A. Moreno, O. Cerri, J. M. Duarte, H. B. Newman, T. Q. Nguyen, A. Periwal, M. Pierini, A. Serikova, M. Spiropulu, and J.-R. Vlimant, JEDI-net: a jet identification algorithm based on interaction networks, arXiv:1908.05318.
  * [22] M. Abadi, A. Agarwal, P. Barham, E. Brevdo, Z. Chen, C. Citro, G. S. Corrado, A. Davis, J. Dean, M. Devin, S. Ghemawat, I. Goodfellow, A. Harp, G. Irving, M. Isard, Y. Jia, R. Jozefowicz, L. Kaiser, M. Kudlur, J. Levenberg, D. Mané, R. Monga, S. Moore, D. Murray, C. Olah, M. Schuster, J. Shlens, B. Steiner, I. Sutskever, K. Talwar, P. Tucker, V. Vanhoucke, V. Vasudevan, F. Viégas, O. Vinyals, P. Warden, M. Wattenberg, M. Wicke, Y. Yu, and X. Zheng, TensorFlow: Large-scale machine learning on heterogeneous systems, 2015.  Software available from tensorflow.org.
  * [23] D. P. Kingma and J. Ba, Adam: A Method for Stochastic Optimization, arXiv e-prints (Dec, 2014) arXiv:1412.6980, [arXiv:1412.6980].
  * [24] J. Thaler and K. Van Tilburg, Identifying Boosted Objects with N-subjettiness, JHEP 03 (2011) 015, [arXiv:1011.2268].
  * [25] M. Pierini, J. M. Duarte, N. Tran, and M. Freytsis, Hls4ml lhc jet dataset (100 particles), Jan., 2020.
  * [26] E. Coleman, M. Freytsis, A. Hinzmann, M. Narain, J. Thaler, N. Tran, and C. Vernieri, The importance of calorimetry for highly-boosted jet substructure, JINST 13 (2018), no. 01 T01003, [arXiv:1709.08705].
  * [27] J. Duarte et al., Fast inference of deep neural networks in FPGAs for particle physics, JINST 13 (2018), no. 07 P07027, [arXiv:1804.06913].
  * [28] M. Cacciari, G. P. Salam, and G. Soyez, The anti-$k_{T}$ jet clustering algorithm, JHEP 04 (2008) 063, [arXiv:0802.1189].
  * [29] G. Kasieczka, T. Plehn, J. Thompson, and M. Russel, Top quark tagging reference dataset, Mar., 2019.
  * [30] T. Sjöstrand, S. Ask, J. R. Christiansen, R. Corke, N. Desai, P. Ilten, S. Mrenna, S. Prestel, C. O. Rasmussen, and P. Z. Skands, An Introduction to PYTHIA 8.2, Comput. Phys. Commun. 191 (2015) 159–177, [arXiv:1410.3012].
  * [31] DELPHES 3 Collaboration, J. de Favereau, C. Delaere, P. Demin, A. Giammanco, V. Lemaître, A. Mertens, and M. Selvaggi, DELPHES 3, A modular framework for fast simulation of a generic collider experiment, JHEP 02 (2014) 057, [arXiv:1307.6346].
  * [32] A. Butter et al., The Machine Learning Landscape of Top Taggers, SciPost Phys. 7 (2019) 014, [arXiv:1902.09914].
  * [33] P. T. Komiske, E. M. Metodiev, and J. Thaler, Energy flow networks: deep sets for particle jets, Journal of High Energy Physics 2019 (Jan, 2019).
  * [34] P. T. Komiske, E. M. Metodiev, and M. D. Schwartz, Deep learning in color: towards automated quark/gluon jet discrimination, JHEP 01 (2017) 110, [arXiv:1612.01551].
  * [35] P. Velickovic, G. Cucurull, A. Casanova, A. Romero, P. Liò, and Y. Bengio, Graph attention networks, in 6th International Conference on Learning Representations, ICLR 2018, Vancouver, BC, Canada, April 30 - May 3, 2018, Conference Track Proceedings, OpenReview.net, 2018.


# Thermal and Magnetoelastic Properties of the van der Waals Ferromagnet
Fe3-δGeTe2: Anisotropic Spontaneous Magnetostriction and Ferromagnetic Magnon
Excitations

Reinhard K. Kremer<EMAIL_ADDRESS>Max Planck Institute for Solid State
Research, Heisenbergstrasse 1, D-70569 Stuttgart, Germany  Eva Brücher
<EMAIL_ADDRESS>Max Planck Institute for Solid State Research,
Heisenbergstrasse 1, D-70569 Stuttgart, Germany

###### Abstract

By determining the lattice parameters as a function of temperature of the
hexagonal van der Waals ferromagnet Fe2.92(1)Ge1.02(3)Te2 we obtain the
temperature dependence of the spontaneous in-plane magnetostriction in the
ferromagnetic and the linear thermal expansion coefficients in the
paramagnetic state. The spontaneous magnetostriction is clearly seen in the
temperature dependence of the in-plane lattice parameter $a(T)$, but less well
pronounced perpendicular to the planes along $c$. Below $T_{\rm C}$ the
spontaneous magnetostriction follows the square of the magnetization and leads
to an expansion of the hexagonal layers. Extrapolating to $T\rightarrow$ 0 K
we obtain a spontaneous in-plane saturation magnetostriction of $\lambda_{{\rm
sp},a}(T\rightarrow 0)\approx-220~{}\times~{}10^{-6}$. In the paramagnetic
state the linear thermal expansion coefficients amount to 13.9(1)$\times$10-6
K-1 and to 23.2(2)$\times$10-6 K-1 for the in-plane and out-of-plane
direction, respectively, indicating a linear volume thermal expansion
coefficient of 50.8(4)$\times$10-6K-1 which we use to estimate the volume
thermal expansion contribution to the heat capacity determined at constant
pressure. A Sommerfeld-type linear term in the low-temperature heat capacities
can be quantitatively ascribed to 2dim ferromagnetic magnon excitations.

##  I Introduction

Two-dimensional (2dim) van der Waals (vdW) ferromagnets with sufficiently high
Curie temperatures lately have attracted special attention with respect to
their applicability in modern nanoelectronic and spintronic devices. Among a
number of prominent systems, the ternary iron germanium telluride, Fe3-δGeTe2
($\delta\approx 0.1$) (FGT), with a Curie temperature, $T_{\rm C}$, close to
220 K has been intensively investigated. After the initial synthesis of
polycrystalline specimen and the basic structural and magnetic
characterization,[1] accessibility of larger crystalline samples enabled a
broad variety of experiments with a focus on mono- and multilayer samples.
These proved FGT to be a highly promising platform for studying the complex
interplay of magnetic and electronic properties in reduced dimensions up to
the point of monolayer devices.

FGT exhibits a number of intriguing magneto-electronic properties: Below
$T_{\rm C}$ FGT is an itinerant ferromagnet.[1, 2] Though, an
antiferromagnetic transition at $\sim$150 K has also been conjectured.[3]
Using density functional calculations Jang et al. concluded that Fe defects
and hole doping are the key to drive FGT into the ferromagnetic phase, whereas
they proposed that stoichiometric FGT to be antiferromagnetic.[4]

A large anomalous Hall effect already at small polarizing external magnetic
fields has been found for bulk samples.[5, 6, 7, 8] Exfoliated monolayers
still show uniaxial anisotropy and robust ferromagnetism, however with $T_{\rm
C}$ reduced to $\sim$130 K.[9, 10] Broadest interest attracted the electrolyte
gating experiments by Deng et al.. They showed that already small gate
voltages applied to flakes of FGT with Li+ intercalated in between the Te
double layers can tune $T_{\rm C}$ from 100 K up to room temperature.[11]
Laser-induced spin and charge photocurrents in single-layer FGT were predicted
by first-principles calculations suggesting applications of FGT in opto-
spintronics.[12] Néel- or Bloch-type skyrmions have been reported to occur in
FGT monolayers.[13, 14, 15, 16, 17] Xu et al. argued that unusual terms in the
Hamiltonian connecting four spins can account for these observations.[18]
Recently, successful generation and manipulation of terahertz spin-current has
been reported by Chen et al..[19]

Despite the booming interest in the magnetic and magnetoelectric properties of
FGT, the lattice properties and especially magnetoelastic coupling still
remained scarcely explored. By using density-functional theory Zhuang et al.
found that the orbital moment of the Fe atoms is sizeable, causing a large
magnetic anisotropy energy increasing with tensile strain, and a large in-
plane magnetostrictive coefficient of -559$\times$10-6 for monolayer FGT.[9]
In Raman spectroscopy measurements Milosavljević et al. observed fingerprints
of spin-phonon coupling at around 152 K and also anomalous behavior of the
Raman frequencies and mode linewidths at the ferromagnetic transition.[20]

FGT crystallizes in the hexagonal crystal system with lattice parameters
$a\approx$ 3.99 Å and $c\approx$ 16.34 Å. The crystal structure of FGT
contains slabs of Fe3Ge sandwiched by vdW bonded Te double layers.[1] The Fe
atoms occupy two different crystallographic sites. Notably, the Fe2 site
(Wyckoff position 2c) in samples prepared under stoichiometric conditions is
not fully occupied (typically $\delta\sim$ 0.1), whereas the Fe1 site, within
error bars, exhibits no deficiency. Increasing $\delta$, i.e. enlarging the
Fe2 deficiency reduces $T_{\rm C}$. The in-plane lattice parameter $a$
decreases with increasing $\delta$, whereas the out-of-plane lattice parameter
$c$ increases for larger Fe2 deficiency.[21] Strong uniaxial anisotropy aligns
the Fe magnetic moments along the $c$ easy axis making FGT an auspicious
platform for magnetic data storage devices.[22, 23, 9, 24] The ordered
magnetic moments of the Fe atoms for $\delta$ = 0.1 amount to 2.2 $\mu_{\rm
B}$ for the Fe1 and 1.5 $\mu_{\rm B}$ for the Fe2 crystallographic sites,
respectively.[21] For $\delta\sim$ 0.25 both ordered moments converge to a
common value of $\sim$ 1.4 $\mu_{\rm B}$.[21, 9]

Here, we report on measurements of the zero-field spontaneous magnetostriction
when FGT enters into the ferromagnetic state. Magnetostriction, i.e. the
deformation of the geometrical shape of a ferromagnetic specimen during the
magnetization process can be evoked e.g. by an external magnetic field.
Especially magnetic field induced magnetostriction is of great technological
importance but also fundamental to model the magnetization process itself and
the formation of the domain structure. Magnetostriction induced by an external
magnetic field depends on the orientation of the external magnetic field with
respect to the orientation of domain magnetization and the direction of the
exchange interaction between the magnetic moments.[25] Spontaneous
magnetostriction where the shape change, $\lambda$=$\delta l(T)/l(T)$, is
initiated by the increase of the spontaneous magnetization $M(T<T_{\rm
c},H=0)$ below $T_{\rm C}$ in vanishing external magnetic field $H$ i.e. by
the internal ferromagnetic domain formation can provide very valuable
information, e.g. about spontaneous reorientation processes and changes in the
domain structure. Technologically magnetostrictive effects may become
important for the interaction of FGT monolayers deposited on substrates.

In addition, we review preceding heat capacity experiments which have found
large linear terms in the low and high temperature specific heats which the
authors attributed to magnetic contributions.[2] Especially in the
paramagnetic regime, the published heat capacity data substantially overshoot
the Dulong-Petit limit, suggesting a critical reconsideration of the thermal
properties. By using the linear thermal volume expansion coefficients in the
paramagnetic state to determine the lattice expansion contribution to the heat
capacities measured at constant pressure, $C_{p}$, we revise these findings.

The Sommerfeld-like linear term in the low-temperature heat capacities can be
quantitatively attributed to 2dim ferromagnetic magnon excitations. A
comparison with the spin stiffness constants found by inelastic neutron
scattering gives quantitative agreement.

##  II Experimental

Samples of FGT were prepared from stoichiometric mixtures of powders of the
elements, Fe (Alfa Aesar, purity 99.998%), Ge (Thermo Fisher, purity 99.999%)
and Te (Thermo Fisher, purity 99.999%) using a minute amount of iodine as
mineralizer. The starting materials were thoroughly mixed and reacted in
evacuated quartz glass tubes in a two-zone furnace heated to temperatures
between 750∘C and 650∘C. Phase purity and composition of the samples was
checked by energy-dispersive spectroscopy (EDX) employing a Tescan Vega4 LMU
SEM equipped with an Oxford X-MaxNN20 detector and by x-ray powder diffraction
(XRPD) using Mo$K\alpha_{1}$ and Cu$K\alpha_{1}$ radiation. The magnetic
properties of the sample were determined by powder and single crystal dc
magnetization (MPMS, Quantum Design) and specific heat measurements (PPMS,
Quantum Design).

The lattice parameters of a polycrystalline sample of FGT were determined in
zero external magnetic field as a function of temperature from Rietveld
profile analysis[26] of XRPD patterns collected on a Bruker D8 Discovery x-ray
diffractometer (Bragg-Brentano scattering geometry) using Cu$K\alpha_{1}$
radiation. A PHENIX (Oxford Cryosystems) closed cycle cooling cryostat was
used to set the temperature of the sample. Each XRPD pattern was collected at
stabilized temperature. The particle size of the powder was adjusted to 63
$\mu$m or less by straining the powder through sieves of the respective mesh
size. The powder sample was thermally anchored with ApiezonN vacuum grease to
the sample holder platform, equipped with an inlay of a Si wafer specially
oriented to suppress background scattering from the sample holder.

##  III Results and Discussion

###  III.1 Sample Characterization

Figure 1 displays a SEM picture of a typical crystal analyzed with energy
dispersive x-ray spectroscopy. Three spots on an $a$-$b$ surface were tested
and very good agreement was found for the element concentration, proving
homogeneity of the element concentration. EDX analyses carried out on crystals
of several other FGT sample preparations indicated a spread of the Fe2
concentration of 0.82 $\leq(1-\delta)\leq$ 0.92.

Figure 1: SEM image of a crystal of our sample with the rectangles marking the
areas where EDX element analysis was performed. The variation of element
concentration across the analyzed areas amounted to less than 0.5%, 1.4% and
0.07% for Fe, Ge and Te, respectively. The lower inset shows the SEM spectrum
collected at spot no. 1.

XRPD on crushed crystals and polycrystalline samples of FGT proved phase
purity of our specimen. Figure 2 displays a typical room-temperature XRPD
pattern analyzed with a Rietveld profile refinement assuming the structure
model proposed by Deiseroth et al. (space group $P$63/$mmc$, no. 194).[1] At
room temperature the refined lattice parameters amount to $a$ = 3.99939(8) Å
and $c$ = 16.3225(4) Å, in good agreement with the values reported by
Deiseroth et al.. In the refinements the Ge and Fe1 lattice sites were assumed
to be fully occupied, whereas for Fe2 the occupancy converged to 0.918(11),in
good accord with the EDX result. At room temperature the $z$-positional
parameter of the Te and Fe1 atoms (Wyckoff positions 4f and 4e, respectively)
were refined to 0.5902(2) and 0.6702(3), also in good agreement with the
values obtained from single crystal structure determination.[1]

Figure 2: (color online) XRD pattern of FGT (Cu$K\alpha_{1}$ radiation)
collected at $295$ K. The red circles represent the experimental data, the
solid black line show the result of the Rietveld profile refinement and the
vertical green bars mark the positions of the Bragg reflections used to
calculate the refined pattern.

The Curie temperatures of the FGT samples were determined from heat capacity
and magnetization measurements. Figure 3 summarizes the magnetic
susceptibilities and the specific heat of the sample with composition
Fe2.92(1)Ge1.00(3)Te2. The magnetization data exhibit the characteristic
splitting of the zero-field cooled (zfc) and field - cooled (fc) branches
reported before.[1] The zfc - fc hysteresis closes with increasing magnetic
field. The steepest descent of the magnetization is found at $\sim$218 K (see
inset (b) in Figure 3) where also the $\lambda$-type anomaly in the specific
heat is observed (Figure 3(d)). At room temperature the heat capacity linearly
approaches a value of $\sim$156 J/molK, consistent with the Dulong-Petit value
of 5.92$\times$3$R$, where $R$ is the molar gas constant, but substantially
lower than the findings reported by Bin Chen et al..[2] The linear increase of
the heat capacity will be discussed in detail below. At low temperatures in
the ferromagnetic state the heat capacities follow a power law (Figure 3(e))

$C_{{mol}}/T=\gamma+\beta T^{2},$ (1)

with $\gamma$ = 109(1) mJ/molK2, close to what has been observed by Bin Chen
et al..[2] The slope $\beta$ = 1.25(6) mJ/molK4 implies a Debye temperature,
$\Theta_{\rm Deb}(T\rightarrow$ 0 K) of 210(2) K. The majority of the linear
contribution to the heat capacity can be ascribed to two-dimensional
ferromagnetic magnons, as will be analyzed in detail below.

Figure 3: (color online) (upper panel, black circles) (a) Molar heat capacity
of a crystal of Fe2.92(1)Ge1.02(3)Te2. The blue dots exemplify the reduction
of the Curie temperature for a sample with element composition Fe2.85GeTe2
(data downshifted by 10 J/molK for better comparison). (b) displays the power
law behavior at low temperatures.(lower panel) (c) zfc and fc magnetic
susceptibilities (indicated by the arrows) of the FGT sample of composition
Fe2.92(1)Ge1.02(3)Te2 measured with external magnetic fields of 100 Oe, 1 kOe
and 10 kOe. The inset (d) displays the derivative of the magnetization
measured with an external field of 100 Oe as a function of temperature. The
(upper) inset (e) shows the magnetic moment per Fe atom. The (red) solid line
is a guide to the eye.

The Curie temperature $T_{\rm C}$ depends on the Fe2 content. An increase of
$\delta$ leads to a reduction of the Curie temperature (see blue dots in
Figure 3(a)). Figure 4(a) displays the variation of the Curie temperature as a
function of the composition around $\delta\approx$ 0.1. A decrease of $T_{\rm
C}$ is paralleled by a decrease of the volume of the crystallographic unit
cell (Fig. 4(b)), the latter induced by the decrease of the in-plane lattice
parameter $a$.

Figure 4: (color online) (a) Curie temperature, $T_{\rm C}$ as a function of
the Fe2 deficiency, $\delta$ of our samples. $\delta$ was determined from the
Rietveld refinements of the XRPD patterns collected using Mo$K\alpha_{1}$
radiation and from EDX analyses. (b) Volume of the crystallographic unit cell
as a function of $\delta$. The (red) dashed lines are guides to the eye.

###  III.2 Spontaneous Magnetostriction

A fraction of the same sample with composition Fe2.92(1)Ge1.02(3)Te2 which had
never been exposed to an external field was subsequently used for the
temperature dependent XRPD measurements. Figure 5 displays the reduced lattice
parameters $a^{*}(T)$=$a(T)$/$a$(295 K) and $c^{*}(T)$=$c(T)$/$c$(295 K),
relative to their room temperature values, as derived from the Rietveld
profile refinements of the temperature dependent XRPD patterns. With
decreasing temperature the relative contraction of the lattice parameters
perpendicular to the layers is about a factor of two larger than the in-plane
contraction. In the paramagnetic regime the cell volume and the lattice
parameters decrease linearly with temperature with rates ($dV_{\rm
cell}/dT$)/$V_{\rm cell}$(295 K) = -50.8(4)$\times$10-6 K-1, and
-13.9(2)$\times$10-6 K-1, and - 23.2(2)$\times$10-6 K-1, for $a$ and $c$,
respectively.

Figure 5: (color online) Temperature dependence of the lattice parameters
$a^{*}(T)$=$a(T)$/$a(295K)$ and of $c^{*}(T)$=$c(T)$/$(295K)$.

Figure 6 displays the temperature dependence of the $z$-positional parameters
of the Te and the Fe1 atoms. Whereas $z$(Fe1) shows a faint increase with
decreasing temperature, still within error bars, the $z$ positional parameter
of the Te atoms decreases and levels off at low temperatures.

Figure 6: $z$ positional parameters of the (a) Fe1 and (b) the Te atoms
(Wyckoff positions 4e and 4f, respectively) of Fe2.92(1)Ge1.02(3)Te2 as a
function of temperature.

Figure 7 compiles the volume of the hexagonal unit cell and the lattice
parameters $a(T)$ and $c(T)$ as a function of the temperature. Whereas the in-
plane lattice parameter $a(T)$ exhibits a noticeable shoulder near $T\sim
T_{\rm C}$, $c(T)$ bents away from the high-temperature linear behavior before
leveling off at low temperatures. A noticeable response to the onset of
ferromagnetic order is not seen for the out-of-plane lattice parameter $c$.
The shoulder in $a(T)$ is ascribed to a negative spontaneous in-plane
magnetostriction, i.e. an expansion of the hexagonal planes, induced by
ferromagnetic ordering.

Extrapolating the linear behavior of $a(T)$ from the paramagnetic regime and
subtracting from the data below $T_{\rm C}$, reveals a spontaneous in-plane
magnetostriction starting below $\sim$223 K, which saturates below $\sim$50 K.
The temperature dependence of the spontaneous magnetostriction (see Figure 8)
is reminiscent of a continuous phase transition with a critical temperature of
223(1) K, matching $T_{\rm C}$, as determined from the magnetization and
specific heat experiments.

Figure 7: (color online) Temperature dependence of the lattice parameters $a$
(a), $c$ (b), and the cell volume (c) of Fe2.92(1)Ge1.02(3)Te2. Error bars are
of the size of the symbols. The solid (red) lines represent linear fits
between 235 K and 295 K of the temperature dependence of the respective
quantities. The relative slopes are given in the text.

A tentative extrapolation of the spontaneous magnetostriction to
$T\rightarrow$ 0 K (see the black dashed line in Fig. 8) indicates a
spontaneous in-plane saturation magnetostriction, $\lambda_{{\rm sp},a}(0))$,
of

$\lambda_{{\rm sp},a}(T\rightarrow 0)\approx-2.2\times 10^{-4}.$

Figure 8: (color online) Spontaneous in-plane magnetostriction of
Fe2.92(1)Ge1.02(3)Te2 as a function of temperature. The red solid line is a
fit to a critical power law, $\tau^{\beta}$ with the reduced temperature
$\tau$ = 1 - ($T/T_{\rm C}$) with a critical temperature, $T_{\rm C}$ 222.4(8)
K and a critical exponent $\beta$ = 0.48(2). The black dashed line represents
an extrapolation to $T\rightarrow$ 0 K.

Attempts to model the temperature dependence of the lattice parameters to a
Debye function over the whole temperature range lead to unstable fits and were
not meaningful. For the same reason the out-of-plane spontaneous
magnetostriction was difficult to ascertain since there is no visible anomaly
in the temperature dependence of the $c$-lattice parameter at $T_{\rm C}$.

In order to relate the spontaneous magnetostriction to the temperature
dependence of the magnetization we carried out isothermal magnetization
measurements on a thin crystal platelet of Fe2.92(1)Ge1.02(3)Te2 with the
magnetic field aligned along the $c$-axis. After a correction for the
demagnetizing field, we derived the temperature dependence of the zero-field
magnetization from modified Arrott Belov plots[27, 28] (see inset in Fig. 9).

Figure 9: (color online) Temperature dependence of the zero-field spontaneous
magnetization of Fe2.92(1)Ge1.02(3)Te2 derived from modified Arrott Belov
plots displayed (lower inset) adopting critical exponents $\beta$ = 0.33 and
$\gamma$ = 1.38. The main frame shows a fit of the magnetization with a power-
law temperature dependence (solid red) line according to eq. (2).

The zero-field magnetization can be very well fitted to a critical power law
(see Fig. 9) given by

$M(T)=M_{0}(1-T/T_{\rm C})^{\beta},$ (2)

with a critical exponent $\beta$ = 0.32(1), consistent with findings reported
earlier by Liu et al..[29, 30]

Fig. 10 displays the temperature dependence of the spontaneous
magnetostriction as a function of the magnetization.

Figure 10: (color online) Log-log plot of the spontaneous in-plane
magnetostriction as a function of the magnetization. The blue dashed line
corresponds to a power law with an exponent 1.5.

The log-log plot reveals a power law behavior of the spontaneous
magnetostriction according to

$\lambda_{\rm sp,a}(T)=[M_{0}(1-T/T_{\rm C})^{\beta}]^{1.5(1)}.$ (3)

Taking into consideration the critical exponent $\beta$ = 0.32 for the
magnetization, eq.(3) indicates a critical exponent for the spontaneous in-
plane magnetostriction of $\sim$0.48, i.e. close to the critical exponent
expected for mean field behavior.

###  III.3 Thermal Lattice Expansion Contribution to the Specific Heat

In a first investigation of the heat capacities of FGT Bin Chen et al. had
found large linear contributions at low and high temperatures, hitherto no
conclusively explained.[2] At room temperature they found a value of $\sim$200
J/molK and a stark linear increase above $T_{\rm C}$, which they attributed to
electronic and magnetic contributions.[2]

In contrast to these earlier findings our heat capacity data measured at
constant pressure, $C_{p}$, (see Fig. 3(d)) tend to a value of $\sim$156
J/molK, only moderately exceeding the Dulong-Petit limit for the heat capacity
at constant volume, $C_{v}(T\rightarrow\infty)\times 3NR=$ 148 J/molK, with
$N$ = 5.92 being the number of atoms per formula unit and $R$ the molar gas
constant. At 350 K our $C_{p}$ data attain a value of $\sim$158 J/molK,
substantially lower than that reported by Bin Chen et al..

Figure 11: (color online) High-temperature heat capacities at constant
pressure, $C_{p}$, of Fe2.92(1)Ge1.02(3)Te2. The solid (red) line is a fit to
eq.(5) with parameters given in the text.

In the following, using our thermal expansion data for $T>T_{\rm C}$, we
estimate the electronic and the lattice thermal expansion contribution to the
heat capacities at high temperatures. The difference between the heat
capacities measured at constant pressure and at constant volume is given by

$C_{p}-C_{v}=\alpha_{v}^{2}(T)BV_{{mol}}T,$ (4)

where $\alpha_{v}(T)$ is the temperature dependent coefficient of the volume
thermal expansion, $B$ is the bulk modulus and $V_{mol}$ the molar volume.

In the paramagnetic regime above $T_{\rm C}$ and up to room temperature the
lattice expands approximately linearly with increasing temperature. The linear
thermal expansion coefficients amount to $\alpha_{a}$ = 13.9(1)$\times$10-6
K-1 and to $\alpha_{c}$ = 23.2(2)$\times$10-6 K-1 for the in-plane and out-of-
plane direction, respectively, resulting in a linear volume thermal expansion
coefficient of $\alpha_{v}$ = 50.8(4)$\times$10-6. The in-plane linear thermal
expansion coefficient of Fe2.92(1)Ge1.02(3)Te2 in the paramagnetic state
compares well with that of elementary iron for which an expansion coefficient
of 11.6$\times$10-6 was observed at room temperature.[31]

In order to fit the heat capacity at high temperatures displayed in Figure 11
we used a series expansion for the Debye contribution[32] and included a term
linear in temperature, $E~{}T$,

$C_{p}(T)=5.92\times
3\,R\,\,(1-\frac{1}{20}\,\frac{\Theta_{\infty}^{2}}{T^{2}}+\frac{1}{560}\,\frac{\Theta_{\infty}^{4}}{T^{4}})+E~{}T,$
(5)

with $R$ being the gas constant, and $\Theta_{\infty}$ the Debye temperature.
$E~{}T$ comprises the thermal expansion contribution given by Eq. (4) but also
a linear electronic (Sommerfeld) term $\gamma$. A fit of the heat capacities
of several runs (Fig. 11) using data for $T>$ 225 K yields

$E=37(2)~{}\rm{mJ/molK^{2}},$

and

$\Theta_{\infty}=222(3)~{}\rm{K},$

the latter being very close to the Debye temperature $\Theta_{\rm
Debye}(T\rightarrow$ 0 K) which amounts to 210 K.

The Sommerfeld-term, $\gamma$, from the heat capacity of the conduction
electrons can be estimated from

$\gamma=\frac{\pi^{2}k_{\rm B}^{2}}{3}N(E_{\rm F}),$ (6)

with the Boltzmann constant $k_{\rm B}$ and the electronic density of states
$N(E_{\rm F})$ at the Fermi level. The electronic densities of states for two
spin directions at $E_{\rm F}$ per formula unit obtained from density
functional calculations has been reported to 6(2) states/eV.[33, 34] Using
this value and Eq. (6) one arrives at a conduction electron term of 14
mJ/molK2 and at a value of 23 mJ/molK2 for the thermal expansion contribution.
Adopting the volume thermal expansion coefficient $\alpha_{\rm{Vol}}$ =
50.8$\times$10-6 K-1 and the molar volume $V_{mol}$ = 6.785$\times$10-5
m3mol-1 [1] results in a Bulk modulus $B$ of

$B\sim 130(40)\rm{GPa},$

wherein the sizeable error bar arises from the uncertainty of the electronic
density of states at the Fermi energy. Our finding for the bulk modulus is by
a factor of two larger than the value of $\sim$70 GPa reported in
literature[35, 34] The difference between our value and the reported results
might be partially due to the negligence of possible electron-phonon
enhancement of the electronic Sommerfeld term. We also note that for their
high-pressure Mössbauer measurements O’Hara et al. used FGT samples with a
$T_{\rm C}$ of 155 K, indicating a substantial deficit on the Fe2 site which
may also affect their value for the bulk modulus.[35]

Even accepting the full value of $E\approx$ 37 mJ/molK2 as due to the electron
contribution, the excess heat capacity of almost 50 J/molK at room temperature
over the Dulong-Petit limit reported by Bin Chen et al.[2] is far too large to
be reconciled with our findings.

###  III.4 2dim Ferromagnetic Magnon Heat Capacity

We finally comment on the linear contribution to the low temperature heat
capacity which amounts to $\sim$109 mJ/molK2, again too large to be attributed
to the conduction electron contribution. However, the increase linear in
temperature, also reported by Bin Chen et al. with a similar magnitude, can be
readily reconciled if we take ferromagnetic magnon excitations in a 2dim
lattice into account. Adopting a quadratic dispersion for long-wavelength
magnons in a 2dim ferromagnet the magnon contribution to the heat capacity at
low temperatures varies linearly with temperature.[36] From inelastic neutron
scattering data, Song Bao et al. have derived spin wave stiffness constants
for FGT between 57 and 69 meVÅ2 for the in-plane spin waves.[37] For the
magnon heat capacity, $C_{\rm mag}$, of a 2dim honeycomb lattice, Grosu et al.
arrived at[38]

$C_{\rm{mag}}=R\frac{{3\sqrt{3}\,\,\pi^{2}}a^{2}}{24D}T,$ (7)

where $R$ is the molar gas constant, $a\approx$ 4Å the in-plane lattice
parameter and $D$ the spin wave stiffness constant. With $D~{}\approx$ 63
meVÅ2 one estimates a magnon contribution linear in temperature of

$C_{\rm{mag}}/T=123~{}\rm{mJ/molK^{2}}.$

Adding the conduction electron contribution of $\sim$14 mJ/molK2, this finding
is in fair agreement with our experimental observation of 109 mJ/molK2 found
from the Sommerfeld plot at low temperatures (see Fig. 3).

##  IV Summary and Conclusion

Using temperature dependent XRPD measurements we have determined the hexagonal
lattice parameters of the 2dim van der Waals ferromagnet Fe2.92(1)Ge1.02(3)Te2
which exhibits a Curie temperature of $\sim$ 217 K. Spontaneous
magnetostriction induced by the transition into the ferromagnetic state leads
to an expansion of the vdW coupled layers, whereas a noticeable effect of the
spontaneous magnetostriction perpendicular to the layers is not seen in the
$c$-lattice parameters. For $T\rightarrow$ 0 K, the in-plane spontaneous
magnetostriction can be extrapolated to $\sim$-220$\times$10-6 K-1 by more
than a factor of two smaller than the ab-initio calculations Zhuang et al..

Using the linear volume thermal expansion coefficient in the paramagnetic
regime we estimate the difference of the specific heats determined at constant
pressure and constant volume. The majority of the linear Sommerfeld-type
contribution to the low-temperature heat capacity can be quantitatively
ascribed to 2dim ferromagnetic magnon excitations.

In summary, our experimental results evidence a substantial in-plane
magnetostriction for a sample of FGT with composition Fe2.92(1)Ge1.02(3)Te2
having a Curie temperature of 217 K. A careful reconsideration of the heat
capacities of such sample enables us to correct and understand previously
reported linear contributions to the low- and high-temperature heat capacity
data of FGT. Particularly noteworthy is the low-temperature magnon heat
capacity contribution, linear in temperature, which can be quantitatively
attributed to the layered ferromagnetic character of FGT.

###### Acknowledgements.

Acknowledgments We thank V. Duppel for performing the SEM/EDS measurements and
Hj. Deiseroth for valuable suggestions as to the preparation of FGT. A useful
communication with Houlong L. Zhuang is gratefully acknowledged.

## References

  * Deiseroth  _et al._ [2006] H.-J. Deiseroth, K. Aleksandrov, C. Reiner, L. Kienle, and R. K. Kremer, $\mathrm{Fe_{3}GeTe_{2}}$ and $\mathrm{Ni_{3}GeTe_{2}}$ – two new layered transition-metal compounds: Crystal structures, hrtem investigations, and magnetic and electrical properties, Eur. J. Inorg. Chem. 2006, 1561 (2006).
  * Chen  _et al._ [2013] B. Chen, J. Yang, H. Wang, M. Imai, H. Ohta, C. Michioka, K. Yoshimura, and M. Fang, Magnetic properties of layered itinerant electron ferromagnet $\mathrm{Fe_{3}GeTe_{2}}$, J. Phys. Soc. Jpn. 82, 124711 (2013), https://doi.org/10.7566/JPSJ.82.124711 .
  * Yi  _et al._ [2017] J. Yi, H. Zhuang, Q. Zou, Z. Wu, G. Cao, S. Tang, S. A. Calder, P. R. C. Kent, D. Mandrus, and Z. Gai, Competing antiferromagnetism in a quasi-2d itinerant ferromagnet: $\mathrm{Fe_{3}GeTe_{2}}$, 2D Materials 4, 10.1088/2053-1583/4/1/011005 (2017).
  * Jang  _et al._ [2020] S. W. Jang, H. Yoon, M. Y. Jeong, S. Ryee, H.-S. Kim, and M. J. Han, Origin of ferromagnetism and the effect of doping on $\mathrm{Fe_{3}GeTe_{2}}$, Nanoscale 12, 13501 (2020).
  * Tan  _et al._ [2018] C. Tan, J. Lee, S.-G. Jung, T. Park, S. Albarakati, J. Partridge, M. R. Field, D. G. McCulloch, L. Wang, and C. Lee, Hard magnetic properties in nanoflake van der waals $\mathrm{Fe_{3}GeTe_{2}}$, Nat. Commun. 9, 10.1038/s41467-018-04018-w (2018).
  * Wang  _et al._ [2018] Z. Wang, D. Sapkota, T. Taniguchi, K. Watanabe, D. Mandrus, and A. F. Morpurgo, Tunneling spin valves based on $\mathrm{Fe_{3}GeTe_{2}}$/hbn/$\mathrm{Fe_{3}GeTe_{2}}$ van der Waals heterostructures, Nano Letters 18, 4303 (2018).
  * Kim  _et al._ [2018] K. Kim, J. Seo, E. Lee, K.-T. Ko, B. S. Kim, B. G. Jang, J. M. Ok, J. Lee, Y. J. Jo, W. Kang, J. H. Shim, C. Kim, H. W. Yeom, B. Il Min, and J. S. Yang, Bohm-Jung Kim, Large anomalous hall current induced by topological nodal lines in a ferromagnetic van der waals semimetal, Nature Materials 17, 794 (2018).
  * Liu  _et al._ [2018] Y. Liu, E. Stavitski, K. Attenkofer, and C. Petrovic, Anomalous hall effect in the van der waals bonded ferromagnet $\mathrm{Fe_{3-x}GeTe_{2}}$, Phys. Rev. B 97, 165415 (2018).
  * Zhuang  _et al._ [2016] H. L. Zhuang, P. R. C. Kent, and R. G. Hennig, Strong anisotropy and magnetostriction in the two-dimensional stoner ferromagnet $\mathrm{Fe_{3}GeTe_{2}}$, Phys. Rev. B 93, 134407 (2016).
  * Fei  _et al._ [2018] Z. Fei, B. Huang, P. Malinowski, W. Wang, T. Song, J. Sanchez, W. Yao, D. Xiao, X. Zhu, A. F. May, W. Wu, D. H. Cobden, J.-H. Chu, and X. Xu, Two-dimensional itinerant ferromagnetism in atomically thin $\mathrm{Fe_{3}GeTe_{2}}$, Nature Materials 17, 778 (2018).
  * Deng  _et al._ [2018] Y. Deng, Y. Yu, Y. Song, J. Zhang, N. Z. Wang, Z. Sun, Y. Yi, Y. Z. Wu, S. Wu, J. Zhu, J. Wang, X. H. Chen, and Y. Zhang, Gate-tunable room-temperature ferromagnetism in two-dimensional $\mathrm{Fe_{3}GeTe_{2}}$, Nature 563, 94 (2018).
  * Merte  _et al._ [2021] M. Merte, F. Freimuth, T. Adamantopoulos, D. Go, T. G. Saunderson, M. Kläui, L. Plucinski, O. Gomonay, S. Blügel, and Y. Mokrousov, Photocurrents of charge and spin in monolayer $\mathrm{Fe_{3}GeTe_{2}}$, Phys. Rev. B 104, L220405 (2021).
  * Ding  _et al._ [2020] B. Ding, Z. Li, G. Xu, H. Li, Z. Hou, E. Liu, X. Xi, F. Xu, Y. Yao, and W. Wang, Observation of magnetic skyrmion bubbles in a van der waals ferromagnet $\mathrm{Fe_{3}GeTe_{2}}$, Nano Letters 20, 868 (2020), pMID: 31869236, https://doi.org/10.1021/acs.nanolett.9b03453 .
  * Meijer  _et al._ [2020] M. J. Meijer, J. Lucassen, R. A. Duine, H. J. Swagten, B. Koopmans, R. Lavrijsen, and M. H. D. Guimarães, Chiral spin spirals at the surface of the van der waals ferromagnet fe3gete2, Nano Letters 20, 8563 (2020), pMID: 33238096, https://doi.org/10.1021/acs.nanolett.0c03111 .
  * Wu  _et al._ [2020] Y. Wu, S. Zhang, J. Zhang, W. Wang, Y. L. Zhu, J. Hu, G. Yin, K. Wong, C. Fang, C. Wan, X. Han, Q. Shao, T. Taniguchi, K. Watanabe, J. Zang, Z. Mao, X. Zhang, and K. L. Wang, Néel-type skyrmion in $\mathrm{WTe_{2}}$/$\mathrm{Fe_{3}GeTe_{2}}$ van der waals heterostructure, Nat. Commun. 11, 10.1038/s41467-020-17566-x (2020).
  * Yang  _et al._ [2020] M. Yang, Q. Li, V. R. Chopdekar, R. Dhall, J. Turner, J. D. Carlström, C. Ophus, C. Klewe, P. Shafer, A. T. N’Diaye, J. W. Choi, G. Chen, Y. Z. Wu, C. Hwang, F. Wang, and Z. Q. Qiu, Creation of skyrmions in van der waals ferromagnet $\mathrm{Fe_{3}GeTe_{2}}$ on (co/pd)(n) superlattice, Sci. Adv. 6, 10.1126/sciadv.abb5157 (2020).
  * Park  _et al._ [2021] T.-E. Park, L. Peng, J. Liang, A. Hallal, F. S. Yasin, X. Zhang, K. M. Song, S. J. Kim, K. Kim, M. Weigand, G. Schütz, S. Finizio, J. Raabe, K. Garcia, J. Xia, Y. Zhou, M. Ezawa, X. Liu, J. Chang, H. C. Koo, Y. D. Kim, M. Chshiev, A. Fert, H. Yang, X. Yu, and S. Woo, Néel-type skyrmions and their current-induced motion in van der waals ferromagnet-based heterostructures, Phys. Rev. B 103, 104410 (2021).
  * Xu  _et al._ [2022] C. Xu, X. Li, P. Chen, Y. Zhang, H. Xiang, and L. Bellaiche, Assembling diverse skyrmionic phases in $\mathrm{Fe_{3}GeTe_{2}}$ monolayers, Adv. Mater. 34, 10.1002/adma.202107779 (2022).
  * Chen  _et al._ [2022] X. Chen, H. Wang, H. Liu, C. Wang, G. Wei, C. Fang, H. Wang, C. Geng, S. Liu, P. Li, H. Yu, W. Zhao, J. Miao, Y. Li, L. Wang, T. Nie, J. Zhao, and X. Wu, Generation and control of terahertz spin currents in topology-induced 2d ferromagnetic $\mathrm{Fe_{3}GeTe_{2}}$/$\mathrm{Bi_{2}Te_{3}}$ heterostructures, Adv. Mater. 34, 10.1002/adma.202106172 (2022).
  * Milosavljević  _et al._ [2019] A. Milosavljević, A. Šolajić, S. Djurdji ć Mijin, J. Pešić, B. Višić, Y. Liu, C. Petrovic, N. Lazarević, and Z. V. Popović, Lattice dynamics and phase transitions in $\mathrm{Fe_{3-x}GeTe_{2}}$, Phys. Rev. B 99, 214304 (2019).
  * May  _et al._ [2016] A. F. May, S. Calder, C. Cantoni, H. Cao, and M. A. McGuire, Magnetic structure and phase stability of the van der waals bonded ferromagnet $\mathrm{Fe_{3-x}GeTe_{2}}$, Phys. Rev. B 93, 014411 (2016).
  * Verchenko  _et al._ [2015] V. Y. Verchenko, A. A. Tsirlin, A. V. Sobolev, I. A. Presniakov, and A. V. Shevelkov, Ferromagnetic order, strong magnetocrystalline anisotropy, and magnetocaloric effect in the layered telluride $\mathrm{Fe_{(3-\delta)}GeTe_{2}}$, Inorg. Chem. 54, 8598 (2015).
  * Zhu  _et al._ [2016] J.-X. Zhu, M. Janoschek, D. S. Chaves, J. C. Cezar, T. Durakiewicz, F. Ronning, Y. Sassa, M. Mansson, B. L. Scott, N. Wakeham, E. D. Bauer, and J. D. Thompson, Electronic correlation and magnetism in the ferromagnetic metal $\mathrm{Fe_{3}GeTe_{2}}$, Phys. Rev. B 93, 144404 (2016).
  * León-Brito  _et al._ [2016] N. León-Brito, E. D. Bauer, F. Ronning, J. D. Thompson, and R. Movshovich, Magnetic microstructure and magnetic properties of uniaxial itinerant ferromagnet $\mathrm{Fe_{3}GeTe_{2}}$, J. Appl. Phys. 120, 083903 (2016), https://doi.org/10.1063/1.4961592 .
  * Chikazumi [1997] S. Chikazumi,  _Physics of Ferromagnetism_ (Oxford University Press, 1997).
  * Rodríguez-Carvajal [1993] J. Rodríguez-Carvajal, Recent advances in magnetic structure determination by neutron powder diffraction, Physica B: Condens. Matter 192, 55 (1993).
  * Reisser  _et al._ [1995] R. Reisser, R. K. Kremer, and A. Simon, Maagnetic phase-transition in the metal-rich rare-earth carbide halides $\mathrm{Gd_{2}XC}$ $\mathrm{(X=BR,I)}$, Physical Review B 52, 3546 (1995).
  * Reiser  _et al._ [1995] R. Reiser, R. K. Kremer, and A. Simon, 3d-xy critical-behavior of the layered metal-rich halides gd2ife2, $\mathrm{Gd_{2}ICo_{2}}$ and $\mathrm{Gd_{2}BrFe_{2}}$, Physica B 204, 265 (1995).
  * Liu  _et al._ [2017a] Y. Liu, V. N. Ivanovski, and C. Petrovic, Critical behavior of the van derwaals bonded ferromagnet $\mathrm{Fe_{2.72}GeTe_{2}}$, Physical Review B 96, 10.1103/PhysRevB.96.144429 (2017a).
  * Liu  _et al._ [2017b] B. Liu, Y. Zou, S. Zhou, L. Zhang, Z. Wang, H. Li, Z. Qu, and Y. Zhang, Critical behavior of the van der waals bonded high $\mathrm{T_{C}}$ ferromagnet $\mathrm{Fe_{2.72}GeTe_{2}}$, Scientific Reports 7, 10.1038/s41598-017-06671-5 (2017b).
  * Touloukian  _et al._ [1977] Y. S. Touloukian, R. K. Kirby, E. R. Taylor, and T. Y. R. Lee,  _Thermophysical Properties of Matter \- the TPRC Data Series. Volume 13. Thermal Expansion - Nonmetallic Solids_ (John Wiley and Sons Ltd, 1977).
  * Tolman [1938] R. C. Tolman,  _The Principles of Statistical Mechanics_ (Oxford University Press, Oxford, 1938).
  * Kim  _et al._ [2021] D. Kim, C. Lee, B. G. Jang, K. Kim, and J. H. Shim, Drastic change of magnetic anisotropy in $\mathrm{Fe_{3}GeTe_{2}}$ and $\mathrm{Fe_{4}GeTe_{2}}$ monolayers under electric field studied by density functional theory, Scientific Reports 11, 11567 (2021), pMID: 35442017, https://doi.org/10.1021/acsnano.1c09150 .
  * Dang  _et al._ [2023] N.-T. Dang, D. P. Kozlenko, O. N. Lis, S. E. Kichanov, Y. V. Lukin, N. O. Golosova, B. N. Savenko, D.-L. Duong, T.-L. Phan, T.-A. Tran, and M.-H. Phan, High pressure-driven magnetic disorder and structural transformation in $\mathrm{Fe_{3}GeTe_{2}}$: Emergence of a magnetic quantum critical point, Adv. Sci. , 2206842 (2023), https://onlinelibrary.wiley.com/doi/pdf/ 10.1002/advs.202206842 .
  * O’Hara  _et al._ [2020] D. J. O’Hara, Z. E. Brubaker, R. L. Stillwell, E. F. O’Bannon, A. A. Baker, D. Weber, L. B. B. Aji, J. E. Goldberger, R. K. Kawakami, R. J. Zieve, J. R. Jeffries, and S. K. McCall, Suppression of magnetic ordering in fe-deficient $\mathrm{Fe_{3-x}GeTe_{2}}$ from application of pressure, Phys. Rev. B 102, 054405 (2020).
  * De Jongh and Miedema [2001] L. J. De Jongh and A. R. Miedema, Experiments on simple magnetic model systems, Adv. Phys. 50, 947 (2001), https://doi.org/10.1080/00018730110101412 .
  * Bao  _et al._ [2022] S. Bao, W. Wang, Y. Shangguan, Z. Cai, Z.-Y. Dong, Z. Huang, W. Si, Z. Ma, R. Kajimoto, K. Ikeuchi, S.-i. Yano, S.-L. Yu, X. Wan, J.-X. Li, and J. Wen, Neutron spectroscopy evidence on the dual nature of magnetic excitations in a van der waals metallic ferromagnet $\mathrm{Fe_{2.72}GeTe_{2}}$, Phys. Rev. X 12, 011022 (2022).
  * Grosu and Crisan [2020] I. Grosu and M. Crisan, Ferromagnetic order in two-dimensional spin systems with dipolar interaction, J. Supercond. Nov. Magn. 33, 1073 (2020).