# Ultra-Low Power Keyword Spotting at the Edge

Mehmet Gorkem Ulkar  Analog Devices Inc.
Istanbul, Turkey
<EMAIL_ADDRESS>Osman Erman Okman  Analog Devices Inc.
Istanbul, Turkey
<EMAIL_ADDRESS>
###### Abstract

Keyword spotting (KWS) has become an indispensable part of many intelligent
devices surrounding us, as audio is one of the most efficient ways of
interacting with these devices. The accuracy and performance of KWS solutions
have been the main focus of the researchers, and thanks to deep learning,
substantial progress has been made in this domain. However, as the use of KWS
spreads into IoT devices, energy efficiency becomes a very critical
requirement besides the performance. We believe KWS solutions that would seek
power optimization both in the hardware and the neural network (NN) model
architecture are advantageous over many solutions in the literature where
mostly the architecture side of the problem is considered. In this work, we
designed an optimized KWS CNN model by considering end-to-end energy
efficiency for the deployment at MAX78000, an ultra-low-power CNN accelerator.
With the combined hardware and model optimization approach, we achieve 96.3%
accuracy for 12 classes while only consuming 251 uJ per inference. We compare
our results with other small-footprint neural network-based KWS solutions in
the literature. Additionally, we share the energy consumption of our model in
power-optimized ARM Cortex-M4F to depict the effectiveness of the chosen
hardware for the sake of clarity. ††Preprint.

###### Index Terms:

speech commands, keyword spotting, wake word detection, convolutional neural
networks, once-for-all, neural architecture, mfcc

##  I Introduction

Speech-based interfaces have been commonly used for smart devices like
intelligent loudspeakers or mobile assistants, especially with the recent
improvements in deep neural network (DNN) technologies. Large vocabulary
speech recognition systems run big networks that require a lot of resources
and high processing power; therefore, they are generally built as cloud-based
solutions. On-device keyword spotting (KWS) is used in such systems to
activate or support a limited number of interactions (i.e., commands) to
reduce standby energy and latency, preserve user privacy, and avoid data
leakage during data transmission.

Due to the low-latency requirement of KWS systems, many researchers focused on
solutions with a small footprint in terms of computation and parameters. The
authors in [1] proposed using a CNN with limited size instead of a DNN for the
KWS application. In [2], the authors proposed the use of residual layers and
dilations in CNN to increase the accuracy while satisfying the small footprint
requirement. Graph convolutional networks are proposed in [3] to capture non-
local relations in KWS. Zhang et al.proposed a depthwise-separable CNN for MCU
deployment in [4]. All of these mentioned methods use Mel-Frequency Cepstrum
Coefficients (MFCC) to preprocess the raw audio first and then run their
models on these features. While they are competing to get more compact and
accurate models, the computation and power consumed in the MFCC remain stable.
The authors in [5] proposes using Tsetlin Machines instead of NN to run on
MFCC features, and they show that MFCC consumes most of the total energy of
their end-to-end KWS system. This illustrates the significance of eliminating
MFCC or other similar preprocessing from the KWS pipeline. Authors in [6]
proposes using Sinc-convolutions in the feature extraction layer of their CNN
network instead of MFCC. However, Sinc-convolutions are not supported by most
of the known NN accelerators, if not none. Therefore this block needs to be
implemented in the MCU if edge deployment is targeted.

Contrary to most literature where the number of operations and parameter
number are the main concerns, energy is the most vital consideration for any
battery-powered edge solution. With the introduction of NN accelerators, the
metrics like multiply-accumulate operations (MAC) do not substitute for energy
usage as the energy usage of those operations differs vastly depending on
whether those operations can be done in an ultra-efficient NN accelerator or
not. The main focus of this paper is to propose a highly accurate and ultra-
energy-efficient KWS solution. For the lowest possible end-to-end energy
usage, both the hardware and model architecture need to be optimized as shown
in Fig. 1. The utilization of specialized low-power hardware for neural
network inference is the general solution if custom hardware is not to be
designed for the given application. However, most of the time, neural networks
are not the single components of the keyword spotting systems. Preprocessing
the audio signal or postprocessing the model outputs could be required besides
the neural network inference operation. Removing those parts that cannot be
run by an efficient accelerator and delegating their tasks to neural network
blocks would be a viable method for increasing overall energy efficiency.
Another room for improvement in the architecture side is to obtain the best
architecture in terms of performance and energy for the given limitations of
the deployment hardware.

In this study, we propose a CNN-based neural network architecture for KWS
problems, which reduces the required energy consumption by removing the
commonly used Mel-Spectrum transformation and utilizing an optimized neural
network accelerator, MAX78000. We also employ a neural architecture search
(NAS) approach and modify due to the limitations of the specialized hardware
to optimize the accuracy of the final model. To summarize, our contributions
can be listed as follows:

  * •

In the hardware side, an ultra-low-power CNN accelerator, MAX78000, is
utilized to implement the solution to present the overall energy reduction.

  * •

In the NN architecture front, a network architecture is proposed for KWS
applications that run on time-domain signals to reduce the required
computation resources while preserving the accuracy.

  * •

Once-For-All (OFA) based NAS approach is implemented and used to maximize the
performance of the proposed network architecture while considering the
MAX78000 hardware limitations.

  * •

Proposed solution is compared with other keyword spotting methods in the
literature regarding performance and energy consumption.

Figure 1: Energy optimization methods

##  II Neural Network Architecture

Figure 2: KWS approaches

###  II-A CNN Model with Raw Audio Input

In many of the keyword spotting applications, MFCC is commonly utilized
features [1], [2], [4] which represents the speech signal with a set of known
and relevant components. MFCC are achieved by decomposition of the audio
signal using a filter bank. It provides the Discrete Cosine Transform (DCT) of
a real logarithm of the short-term energy on the Mel frequency scale. More
specifically, the computation pipeline of MFCC includes windowing the speech
signal into frames, performing Fast Fourier Transform (FFT) to find the power
spectrum of each frame, filter bank processing using the Mel scale, and
finally DCT on the log scale of the power spectrum. Therefore, the
computational load of this preprocessing step is generally much higher than
the inference of the trained networks.

If the models that require MFCC processing are deployed to NN accelerators,
MFCC has to be implemented in a general-purpose MCU outside of the
accelerator, as shown in Fig. 2 B. However, this reduces the overall
efficiency as an important portion of computation does not take place in the
ultra-low-power NN accelerator. There are also some efforts in the literature
to reduce the power consumption of MFCC processing, but it still requires more
than two times more power than the neural network inference [7].

In this study, we propose to minimize the preprocessing step by elimination of
the transformation to generate MFCC to reduce energy consumption effectively,
as shown in Fig. 2 C. The raw audio signals are fed to the neural network
after reshaping the signal, where the short-term and long-term variations can
be observed. We used 1.024s length utterances sampled at 16kHz, and each of
them are reshaped to 128x128x1 by folding the raw audio signal to the signal
channels as shown in Fig. 3. Note that this strategy has no processing cost
for our NN accelerator as it can be achieved by changing the order of the
memory locations where the streaming input data is copied.

Figure 3: Proposed data folding strategy. 16384x1 signal is reshaped to
128x128x1.

The advantage of this data structure is using 1D convolutional operators it is
possible to extract features that represent both short time and long time
relations. Therefore, in this study, we designed a base network seen in Fig. 4
which is composed of six 1-D convolutional layer units separated by pooling
layers and a linear layer at the end. The units are composed of two or three
convolutional layers is defined as 128 channel 5x5 kernels, and will then be
modified by the NAS as described in the following section.

###  II-B Neural Architecture Search

Neural architecture search (NAS) is an increasingly popular research topic in
the literature [8]. By automating the exploration process, neural architecture
search (NAS) methods aim to find the best neural architecture for a given set
of requirements. The models that are found by contemporary NAS methods are
difficult to obtain using manual design in terms of complexity and achieved
accuracy. In this paper, we employed a weight-sharing-based NAS technique,
Once-For-All (OFA) [9]. The authors in [9] propose training a super-network
and deploying parts of it to diverse hardware without any need for retraining.
This super-network is called “Once-for-All” network, and it requires a
training process where all sub-networks are trained sufficiently to be
deployed directly. Since training all sub-networks is computationally
prohibitive, the authors came up with an idea of progressive shrinking. In
this training method, the training starts with training the super-network with
maximum kernel size, depth and width first and smaller sub-networks that share
parameters with super-network become available for sampling progressively. By
doing so, the training of interfering sub-networks at the same time is
avoided. If the search space consists, different kernel, depth, and width
values, they are added to sampling space sequentially. Once the whole training
step is completed, the evolutionary search is performed to find the best sub-
network for a given set of requirements. The constraints of targeted NN
accelerator like memory are considered in this search step. In the Fig. 4, the
designed super-network for the KWS is shown. This super-network contains
convolutional layers with maximum width and kernel sizes, and the depth of the
network is set to maximum. Specifically, the designed super-network contains
14 identical Conv1D layers with 128 channels and 5 as kernel lengths. Upon OFA
training and evolutionary search, the sub-network selected automatically can
be shown in the same Fig. 4 with parameter overrides in green font. In case
any layer of the automatically selected network has the same parameter value
with the super network, it is not indicated with green, so the red default
values are used. It is noteworthy that the sub-network is much smaller than
the super-network where its parameters are shared with. Another observation is
that in this case, OFA did not prefer to eliminate any layer from the super
network, although it could do so. Rather than decreasing the depth of the
model, it chooses to slim the layers or shorten the kernels.

According to the OFA scheme, the model is ready to deploy after training and
search steps. However, the model needs to be quantized if it is to be deployed
to an NN accelerator or any MCU. For this purpose, we employed quantization-
aware training (QAT) rather than post-quantization as QAT yields networks with
higher performances.

All the codes that have been developed for this study are open-sourced and can
be found at [10].

Figure 4: KWS super-network for OFA

##  III Experiments & Results

###  III-A Dataset

The proposed approach is evaluated using the Google Speech Commands Dataset v2
[11], which contains 65000 1-second long utterances of 35 short words by
thousands of different people, as well as some background noises. Based on the
recent researches for KWS applications [12, 2, 4, 13], the dataset is
relabeled to form 12 classes: ”Yes”, ”No”, ”Up”, ”Down”, ”Left”, ”Right”,
”On”, ”Off”, ”Stop”, ”Go” along with the ”Silence” which contains no speech
utterances and the ”Unknown” which contains the data from the remaining 25
keywords in the original dataset. The dataset is split into training,
validation, and test set in the ratio of 80:10:10 while the audio clips from
the same person are kept in the same set.

An augmentation procedure is also followed to generate the training data to
increase the robustness and the performance of the neural network model. This
procedure is composed of signal stretching, utterance shifting, and noise
addition, respectively. As a first step, the utterance is stretched with a
random factor selected from $\mathcal{U}$(0.8, 1.3). A random time shift, $t$
$ms$, is then applied to the audio signal, where
$t$$\backsim\$(\mathcal{U})$(-0.1, 0.1). Lastly, a zero-mean white Gaussian
noise is added to the audio signal with variance $\sigma^{2}$, which is a
uniformly distributed random variable between 0 and 1.

During training, every sample in the dataset is used, so there are more
samples from the ”Unknown” class than the other class samples. In order to
prevent the model from getting biased, the class weight of the ”Unknown”
samples is reduced proportionally to the amount of the other class samples.

###  III-B Model Training

In this study, model training includes two steps. As the first step, the
supernet given in Fig. 4 is trained using Stochastic Gradient Descent (SGD)
optimizer with a learning rate of 0.001 for 15000 epochs. Knowledge
distillation is also used during the elastic training stages as described in
the original OFA paper [9]. It fuses two loss terms using both the soft labels
of the supernet just before initialization of the elastic training stages, and
the real labels [14]. In Fig. 5, the progress of the training is given in
terms of the objective loss with respect to both training epochs and time
spent. The elastic kernel, elastic depth, and elastic width stages are also
given in this graph. An evolutionary search algorithm is run using this model
to find the best-performed model given in Fig. 5. Note that the details of the
search algorithm and selected parameter values are given in our project
repository [10].

The selected model parameters must be quantized to at most 8-bits integers in
order to be deployed to our edge platform described in the following section.
However, post-quantization of the trained networks causes high-performance
degradation, and it is much higher when the model is deeper or includes batch
normalization, according to our experiments. Therefore, we use the second step
of training, Quantization Aware Training (QAT) of the selected model, to fine-
tune the model parameters while minimizing the induced errors due to the
quantization. The QAT approach implements the fake (simulated) quantization
approach in [15]. In this training step, an Adam optimizer is used with an
initial learning rate of 0.001, gradually halved 5 times starting from the
100th epoch. The model is trained for 200 epochs, and the QAT procedure is
initiated after epoch 150. In this study, all model parameters are quantized
to 8-bits, and existing batch normalization layers are folded into the weights
with the initialization of QAT.

Figure 5: Objective Loss vs. Number of Epochs for training OFA supernet

###  III-C Neural Network Accelerator

As a deployment platform of the proposed CNN model, MAX78000 is chosen.
MAX78000 is an Ultra-Low-Power Arm Cortex-M4 processor with FPU-based MCU with
CNN accelerator. By spending only microjoules of energy to execute inferences,
MAX78000 targets battery-powered neural network-based applications at the edge
[16]. The CNN engine supports both Conv1D and Conv2D as convolution
operations, 2D transposed convolution layer, linear layer, pooling layers, and
element-wise operations as addition, subtraction, binary OR, binary XOR.

###  III-D Accuracy Results

Table I summarizes the results of our findings of the proposed KWS approach
with the selected small footprint models in the literature. We use class
accuracy as the evaluation metric to be consistent with the literature.
However, these studies consider the footprint of the approach by only taking
the model complexity into account without considering the feature extraction
step, i.e., MFCC calculation. Therefore, we present the table with a field
that shows whether the approach uses MFCC or not. The number of parameters in
the model, as well as the precision of those parameters and activations, are
given in the table.

TABLE I: Comparison of accuracy results on the Speech Command Dataset Model | MFCC | Precision | #Param | Model Size | Acc
---|---|---|---|---|---
[1] | yes | fp-32 | 244.2K | 976.8 KB | 90.2%
[2] | yes | fp-32 | 238K | 952 KB | 95.8%
[3] | yes | fp-32 | 61K | 244 KB | 96.4%
[4] | yes | int-8 | 497.6K | 497.6 KB | 95.4%
[6] | no | fp-32 | 122K | 488 KB | 96.6%
ours | no | int-8 | 419.8K | 419.8 KB | 96.3%

The proposed model in this study provides an accuracy of over 96%, which is
above or comparable with the state-of-the-art small footprint KWS models
despite not utilizing pre-calculated MFCCs. Having similar accuracies with the
state-of-the-art proves that specialized hardware for Neural Net inference can
easily be used to run the pre-trained models for KWS applications on the edge
as the model runs on the raw audio signal.

###  III-E Energy Measurements

In order to analyze the efficiency provided by MAX78000 and our approach, two
sets of measurements are presented in this study. The first measurement is
made to calculate the latency and the energy consumption of the MAX78000 for
the proposed model. The same model is also implemented for an ultra-low-power
ARM Cortex M4F microprocessor, and the measurements are given in II. It should
be noted that the same model consumes more than 40 times more energy when
deployed in the ARM Cortex M4F processor compared to MAX78000 CNN accelerator.
One major element in this low consumption figure is that our model does not
need any preprocessing outside the accelerator; therefore, it can better
leverage the accelerator’s energy efficiency.

TABLE II: Comparison of latency and energy measurements of the proposed model for different hardware Device | Latency (ms/inf) | Energy (uJ/inf)
---|---|---
MAX78000 | 3.5 | 251
ARM Cortex M4F | 905 | 11200

Secondly, we measure the energy and latency for MFCC calculation to show the
efficiency of the proposed approach for the state-of-the-art approaches as
they use MFCC transformation as an initial feature extraction procedure. 40
MFCC features extracted from a speech frame of length 40ms with a stride of
20ms, ending with a 49x40 data for a 1s long audio. Note that this is the same
configuration used in [4] and similar to the others used in the literature.
The described MFCC feature extraction is measured to consume 826 uJ, more than
three times what our end-to-end KWS model consumes in the MAX78000
accelerator. This means any conventional model in the literature that is built
upon MFCC features consumes much more energy than our solution, even if it is
deployed in the convolution efficient hardware.

##  IV Conclusion

In this paper, we present a highly accurate and energy-efficient KWS solution
that runs on battery-powered devices. For this purpose, we propose a novel CNN
model that works on raw audio by eliminating commonly used MFCC preprocessing.
By doing so, the whole of the proposed network becomes deployable to the NN
accelerators. In order to find the best architecture, we adopted a weight-
sharing-based NAS approach, OFA, and we show that our accuracy is comparable
with the state-of-the-art small footprint KWS solutions. Lastly, we deploy the
model to a NN accelerator, MAX78000, and present its effectiveness in terms of
energy and latency.

##  V Acknowledgements

The authors would like to thank Robert Muchsel, Brian Rush and other members
from the AI Development Group at Analog Devices that contributed to this work.

## References

  * [1] T. N. Sainath and C. Parada, “Convolutional neural networks for small-footprint keyword spotting,” in _Sixteenth Annual Conference of the International Speech Communication Association_ , 2015.
  * [2] R. Tang and J. Lin, “Deep residual learning for small-footprint keyword spotting,” in _2018 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP)_ , 2018, pp. 5484–5488.
  * [3] X. Chen, S. Yin, D. Song, P. Ouyang, L. Liu, and S. Wei, “Small-footprint keyword spotting with graph convolutional network,” in _2019 IEEE Automatic Speech Recognition and Understanding Workshop (ASRU)_ , 2019, pp. 539–546.
  * [4] Y. Zhang, N. Suda, L. Lai, and V. Chandra, “Hello edge: Keyword spotting on microcontrollers,” 2018.
  * [5] J. Lei, T. Rahman, R. Shafik, A. Wheeldon, A. Yakovlev, O.-C. Granmo, F. Kawsar, and A. Mathur, “Low-power audio keyword spotting using tsetlin machines,” _Journal of Low Power Electronics and Applications_ , vol. 11, no. 2, 2021. [Online]. Available: https://www.mdpi.com/2079-9268/11/2/18
  * [6] S. Mittermaier, L. Kürzinger, B. Waschneck, and G. Rigoll, “Small-footprint keyword spotting on raw audio data with sinc-convolutions,” in _ICASSP 2020 - 2020 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP)_ , 2020, pp. 7454–7458.
  * [7] W. Shan, M. Yang, T. Wang, Y. Lu, H. Cai, L. Zhu, J. Xu, C. Wu, L. Shi, and J. Yang, “A 510-nw wake-up keyword-spotting chip using serial-fft-based mfcc and binarized depthwise separable cnn in 28-nm cmos,” _IEEE Journal of Solid-State Circuits_ , vol. 56, no. 1, pp. 151–164, 2021.
  * [8] P. Ren, Y. Xiao, X. Chang, P.-Y. Huang, Z. Li, X. Chen, and X. Wang, “A comprehensive survey of neural architecture search: Challenges and solutions,” 2021.
  * [9] H. Cai, C. Gan, T. Wang, Z. Zhang, and S. Han, “Once for all: Train one network and specialize it for efficient deployment,” in _International Conference on Learning Representations_ , 2020. [Online]. Available: https://arxiv.org/pdf/1908.09791.pdf
  * [10] “Max7800x model training,” https://github.com/MaximIntegratedAI/ai8x-training, 2021.
  * [11] P. Warden, “Speech commands: A dataset for limited-vocabulary speech recognition,” 2018.
  * [12] R. Tang and J. Lin, “Honk: A pytorch reimplementation of convolutional neural networks for keyword spotting,” 2017.
  * [13] T. Mo, Y. Yu, M. Salameh, D. Niu, and S. Jui, “Neural Architecture Search for Keyword Spotting,” in _Proc. Interspeech 2020_ , 2020, pp. 1982–1986. [Online]. Available: http://dx.doi.org/10.21437/Interspeech.2020-3132
  * [14] G. Hinton, O. Vinyals, and J. Dean, “Distilling the knowledge in a neural network,” 2015.
  * [15] B. Jacob, S. Kligys, B. Chen, M. Zhu, M. Tang, A. Howard, H. Adam, and D. Kalenichenko, “Quantization and training of neural networks for efficient integer-arithmetic-only inference,” in _Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition (CVPR)_ , June 2018.
  * [16] _MAX78000 Datasheet - Artificial Intelligence Microcontroller with UltraLow-Power Convolutional Neural Network Accelerator_ , Maxim Integrated, 5 2021, rev. 1.


# Intrinsic geometry of collider events and nearest neighbour based weighted
filtration

Jyotiranjan Beuria

###### Abstract

Collider observations have mainly been studied on an event-by-event basis,
leveraging several kinematic techniques. However, the intrinsic topological
imprints of the ensemble of new physics events can be strikingly different
from the SM background ensemble. Traditional topological data analysis (TDA)
is known for its stability against small perturbations. However, a plethora of
rich information encoded in the clustering of ensembles is often lost due to
the unweighted filtration of simplicial complexes. Taking a singlet extended
model as an example, this work illustrates the rich global properties
associated with the so-called distance-to-measure (DTM) filtration on Alpha
complexes using weights determined from k-nearest neighbours.

###### keywords:

DTM filtration , Persistent Homology , Topological Data Analysis , Event
Geometry

††journal: Physics Letters B

[first]organization=IKSMHA Center, Indian Institute of Technology,
Mandi,country=India [second]organization=Indian Knowledge Systems Center, ISS
Delhi,country=India

##  1 Introduction

The Standard Model (SM) of particle physics has proven highly effective in
explaining the behavior of elementary particles. The identification of the
Higgs boson, a neutral scalar particle, marked a significant achievement for
the SM [1, 2]. Despite its success, the SM fails to address several crucial
unanswered questions. Consequently, searching for models beyond the SM (BSM)
has prompted various upgrades to the Large Hadron Collider (LHC) and the
emergence of other future colliders.

The exploration of Beyond the Standard Model (BSM) phenomenology involves
constraining the parameter space of new physics models. This is achieved
through collider simulations at the patron, hadron, and detector levels,
incorporating signal and background analyses. Physicists have introduced a
range of kinematic variables [3] for this purpose, relying on kinematic cuts
applied on an event-by-event basis. In recent years, several efforts have been
made to analyze the global properties of events using techniques such as
Voronoi and Delaunay tessellations [4, 5, 6], network distance metrics [7],
and multi-event ML classifiers [8]. Although Topological Data Analysis (TDA)
[9, 10, 11, 12, 13, 14, 15, 16, 17, 18] is being increasingly used in the
broader field of data science for examining the intrinsic global properties of
system, it has yet to receive sufficient momentum [18] in the broader particle
physics community.

TDA is a mathematical and computational approach that applies tools from
algebraic topology to analyze the intrinsic topology of complex datasets. This
allows for a more robust understanding of the underlying space and
relationships within the dataset. TDA is particularly effective in studying
the global properties and connectivity of data points. It introduces concepts
such as persistent homology, which helps identify significant topological
features that persist across different scales. By representing data as a
topological space, TDA enables the extraction of valuable information about
clusters, voids, and other topological structures that may not be apparent
through other analytical methods. In practical applications, TDA has found
utility in fields such as biology, neuroscience, machine learning, and
materials science. It offers a robust framework for uncovering hidden patterns
and structures in diverse datasets, providing insights that traditional
methods might overlook.

Particle colliders like LHC collect massive amounts of data featuring various
production and decay cascades of elementary particles. Physicists have studied
the phenomenology of several SM extensions, viz., Supersymmetry [19, 20, 21,
22] and Minimal Universal Extra Dimension [23, 24, 25], Two Higgs Doublet
models [26, 27], etc. to name a few. In a recent work [18], we have delineated
a framework to study the topological properties of the ensemble of collider
events using unweighted persistent homology.

Since the traditional TDA technique uses unweighted simplices, a great deal of
information encoded in the global geometry of the dataset is likely to be
unrecoverable. By assigning weights to simplices, TDA can distinguish between
strong and weak connections, emphasizing essential features and filtering out
noise. This allows for a more robust identification of persistent homology,
which, in turn, enhances the ability to extract meaningful information about
the global and local structures of the dataset. Thus, TDA could be
instrumental in uncovering new physics scenarios at the particle colliders.

In this work, we explore the topological properties of the ensemble of events
with weighted filtration of simplicial complex using the so-called distance-
to-measure (DTM) [28] weights for vertices and edges. DTM filtration is known
to be robust against the presence of outliers in the data [28]. Depending on
the number of nearest neighbours, this filtration highlights different regions
of the global geometry of the ensemble. We illustrate this method for a
minimal extension of the SM by a real singlet scalar [29, 30, 31, 32, 33, 34,
35, 36]. We choose a specific variant of the model where the singlet scalar
becomes the stable particle under a global $Z_{2}$ symmetry [31, 33], making
it a candidate for dark matter. This is sometimes called the Singlet Scalar
Dark Matter (SSDM) model. The method outlined in this work is equally
applicable to any other BSM scenarios.

The organization of the paper is as follows. In section 2, we present some
mathematical preliminaries of topological data analysis. We give the basic
framework of analysis in section 3. We discuss the topological features and
their variation with the nearest neighbour fraction in section 4. In section
5, we present machine learning-based classification to distinguish the
benchmark scenarios. We conclude the discussion in section 6.

##  2 Topological Data Analysis

Figure 1: Simplices (plural of simplex) are the combinatorial building blocks
of a simplicial complex. For illustration, 0-,1-,2-, and 3-simplex are shown
from left to right.

One of the prominent tools used for topological data analysis is persistent
homology. The foundational geometric structure for studying persistent
homology is the simplicial complex. Simplicial complexes provide a method for
constructing topological spaces using fundamental combinatorial building
blocks, simplifying the treatment of the continuous geometry of topological
spaces. Instead, it involves the more manageable tasks of combinatorics and
counting. These elementary building blocks, known as simplices and illustrated
in Figure 1, are formed by taking the convex hull of independent points. A
$k$-dimensional simplex is generated by this process, involving $k+1$ points.
For instance, a 0-simplex is a point, a 1-simplex is a line segment, a
2-simplex is a filled triangle, and a 3-simplex is a filled tetrahedron. This
construction can be extended to higher-dimensional polytopes. In an
$n$-dimensional simplex, the simplices with dimensions $k<n$ constitute its
faces. Consequently, for a 2-simplex (triangle), its edges (1-simplex) serve
as the faces.

We recommend the reader to refer to our earlier work [18] for a brief
introduction to the underlying mathematics and its application in particle
physics phenomenology. One may also consult excellent reviews on TDA available
online [37, 15, 14, 17]. Additionally, readers can find valuable insights in
the publicly available worked-out real-life examples [38, 39].

###  2.1 Homology groups

The $k$-th homology group, denoted as $H_{k}$, is defined as the quotient
group, representing cycles modulo boundaries. Mathematically, it is expressed
as:

$H_{k}=\frac{Z_{k}}{B_{k}}=\frac{\text{ker}(\partial_{k})}{\text{im}(\partial_{k+1})}$
(1)

Here, $H_{k}(K)$ is the quotient vector space whose generators are given by
$k$-cycles that are not boundaries of any $(k+1)$-simplices. The rank of
$H_{k}(K)$ is referred to as the $k$-th Betti number, denoted as
$\beta_{k}(K)$. The Betti number $\beta_{k}(K)$ signifies the number of
$k$-dimensional holes in the simplicial complex $K$ that are not boundaries of
any $(k+1)$-simplices. For instance, $\beta_{0}(K)$ represents the number of
connected components in $K$. It is important to note that Betti numbers,
particularly $\beta_{k}$, serve as crucial properties for our subsequent
analysis.

Betti numbers also play a role in defining the Euler characteristic ($\chi$),
a topological invariant of the simplicial complex, given by:

$\chi=\sum_{k=0}^{n}(-1)^{k}\beta_{k}$ (2)

Spectrum | $M_{s}^{2}$ | $\lambda_{sh}$ | $\lambda_{s}$ |  $m_{s}$ (GeV) | $BR_{h\rightarrow s\>s}$ |  $\sigma_{ZH\rightarrow 2\ell+\mathrm{MET}}$ (fb) |  $\sigma_{WH\rightarrow 1\ell+\mathrm{MET}}$ (fb)
---|---|---|---|---|---|---|---
A | -550 | 0.01 | 0.5 | 10.2 | 0.36 | 12.48 | 75.13
B | 2600 | 0.02 | 0.5 | 60.0 | 0.37 | 12.64 | 76.07
Table 1: A and B are two benchmark scenarios in the SSDM having low and high
singlet DM mass, respectively.

###  2.2 Weighted Alpha filtration with DTM

The conventional homology approach may not provide comprehensive information
for a given point cloud data set $X$. However, by adopting a multi-scale
perspective on homology through a filtration parameter, it becomes possible to
capture changes in the homology of $X$. This approach considers a family of
simplicial complexes $K^{\delta}$, parametrized by $\delta\in\mathbb{R}$,
derived from the set $X$. In this context, a simplicial complex
$K^{\delta_{i}}$ at step $i$ is a subset of $K^{\delta_{j}}$ at step $j$ for
$i\leq j$. This family of nested simplicial complexes is referred to as a
filtration, and $\delta$ is termed a filtration parameter that evolves at each
$i$-th step. This multi-scale perspective allows for identifying topological
features that persist over varying scales, offering more reliable insights
into the point cloud data.

We have chosen the Alpha complex throughout this work. It is essentially a
Vietoris–Rips complex that is constructed from the finite cells of a Delaunay
triangulation. Thus, it generates a lesser number of simplices compared to the
Vietoris–Rips complex, and it is also computationally efficient for large
datasets. In our earlier work [18], we have used unweighted simplices of Alpha
complex for which the filtration parameter is the radial distance from the
nodes. However, there is a loss of much interesting topological information
[40] because all higher-order simplicial connections are treated equally.
However, in a weighted simplicial complex, one assigns weights to the
simplices while maintaining the structure of a topological space, i.e., it is
closed under the inclusion of faces of simplices. In this case, the natural
choice of filtration parameter is the weight of a particular type of simplex.
Thus, one may assign low weights to the actual features and large weights to
the outliers and noise so that the actual features can show up at an early
stage of filtration.

The empirical distance-to-measure (DTM) function [28] has been advocated for
its robustness against outliers in the data. We use this weighting function
for vertices and edges in the Alpha complex. The vertex weight function is
given by

$w(x)=\left(\frac{1}{n+1}\sum_{k=1}^{n}d(x,x_{k})^{r}\right)^{\frac{1}{r}},$
(3)

where $n$ is the number of nearest neighbours, $d(x,x_{k})$ is the distance
metric. Throughout this paper, we use the usual Euclidean distance with $r=2$.
$f=\frac{n}{N}$ is the fraction of neareast neighbours, where $N$ is the total
number of vertices. As per the DTM scheme [28], the edge weights
$w(x_{i},x_{j})$ are given as follows.

$w(x_{i},x_{j})=\begin{cases}max(w_{i},w_{j}),&\text{if
}d(x_{i},x_{j})\leq\left|w_{i}^{p}-w_{j}^{p}\right|^{\frac{1}{p}}\\\
t,&\text{if }otherwise,\end{cases}$ (4)

where $t$ is the only positive root of

$d(x_{i},x_{j})=\left(t^{p}-w_{i}^{p}\right)^{\frac{1}{p}}+\left(t^{p}-w_{j}^{p}\right)^{\frac{1}{p}}$
(5)

In this study, we will employ the aforementioned DTM weights (with $p=1$) as
the filtration parameter for constructing persistent homology. It has been
observed that increasing values of $p$ have a tendency to simplify the
persistence diagram, and varying values of $f$ can emphasize different aspects
of the dataset [28].

After this concise introduction to persistent homology, we are prepared to
delve into the physics discussion. Our next focus will be on the collider
simulation and the subsequent analysis using weighted persistent homology.

##  3 Framework of Analysis

###  3.1 $Z_{2}$ symmetric real singlet extension of the SM

We study a simple SM extension using a real $SU(2)_{\mathrm{L}}$ singlet
scalar field $S$. In addition, we impose $Z_{2}$ symmetry on the scalar sector
so that all SM fields are even under $S$, while $S$ is odd. As the stable
singlet scalar particle is guaranteed by $Z_{2}$ symmetry, this model also
functions as the most straightforward extension of the SM with a scalar dark
matter (DM) candidate. We use this model to show that the topological
signature of the persistent homology of the ensemble of collider signals of a
BSM model may differ from that of the SM. The tree-level scalar potential is
given by

$V(H,S)=-\frac{\mu^{2}}{2}H^{\dagger}H-\frac{\lambda_{H}}{2}(H^{\dagger}H)^{2}-\frac{M_{S}^{2}}{2}S^{2}-\frac{\lambda_{s}}{2}S^{4}-\lambda_{sh}H^{\dagger}HS^{2}$
(6)

In the vicinity of the vacuum expectation value ($vev$), the parameterization
of the neutral component of the Standard Model Higgs doublet
($\mathcal{H}_{0}$) and the real singlet scalar ($S$) is expressed as

$\mathcal{H}_{0}=\frac{v+h}{\sqrt{2}},\;S=\frac{v_{s}+s}{\sqrt{2}},$ (7)

Here, $v=246$ GeV ($v_{s}$) represents the vacuum expectation value ($vev$)
for $\mathcal{H}_{0}$ ($S$). The $Z_{2}$ invariance of the scalar sector
mandates a zero $vev$ for the real singlet scalar, denoted as $v_{s}=\langle
S\rangle=0$. Consequently, the tree-level mass of the real singlet $S$ is
determined by

$m_{s}^{2}=M_{S}^{2}+\lambda_{sh}v^{2}$ (8)

We obtain the particle spectra from SARAH v4.5.1 [41, 42] generated SPheno
v4.0.5 [43, 44]. We choose two benchmark spectra (see table 1) such that
$BR_{h\rightarrow s\>s}$ is approximately the same for both except for the
mass of the singlet Higgs. This ensures a similar cross-section for the
invisible decay modes of the SM Higgs. However, there is likely to be a
difference between the phase space of the two spectra owing to the large
difference between their masses. As later discussion will unfold, we will see
a difference in the topological properties of both spectra.

(a)

(b)

(c)

Figure 2: $S_{k}^{pe}$ is the persistent entropy for the $k$-th homology group
and $f$ is the fraction of $k$-nearest neighbours considered for DTM
filtration.

(a)

(b)

(c)

Figure 3: $A_{k}$ is the persistent amplitude for the $k$-th homology group.

(a)

(b)

(c)

Figure 4: $BA_{k}$ is the area under the Betti curve for the $k$-th homology
group.

###  3.2 Collider Simulation

We consider resonant production of $ZH$ and $W^{\pm}H$ two benchmark spectra
of the SSDM. The Higgs boson is made to decay invisibly under $h\rightarrow
s\;s$, where $s$ is the singlet scalar. The vector bosons are made to decay
leptonically. We also consider $1\ell+\mathrm{MET}$ and $2\ell+\mathrm{MET}$
SM background processes via resonant production of $Z$ and $W^{\pm}$ gauge
bosons.

Event samples are generated at the lowest order (LO) in perturbation theory
using MadGraph5 aMC@NLO v3.5.1 [45, 46] with the nn23lo1 [47] parton
distribution function at $\sqrt{s}=13$ TeV. The generated patron-level events
undergo showering with Pythia v8.309 [48]. To avoid double counting of events
in the simulated samples, especially in the presence of extra hard partonic
jets and the parton shower, the event generator utilizes the MLM matching
technique with the variables xqcut and qcut set at appropriate values. The
cross sections for all processes are estimated using an NLO K-factor of 1.2.

The FastJet (v3.3.4) [49, 50] package, integrated into Delphes v3.5.0 [51], is
employed for jet finding. The anti-$k_{T}$ jet algorithm is utilized with a
cone size of 0.5, requiring a minimum $p_{T}^{jet}$ of 20 GeV and limiting the
pseudorapidity to $|\eta_{jet}|<2.5$.

Following the default parameter settings of Delphes v3.5.0, the reconstruction
of leptons (electrons and muons) involves a minimum $p_{T}^{l}$ of 0.5 GeV and
$|\eta_{jet}|<2.5$. For electrons and muons, the track isolation requirement
entails removing jets within an angular distance $\Delta R\leq 0.5$ from the
lepton. To enhance the purity of electrons, it is required that the ratio of
the total $p_{T}$ of stray tracks within the cones of their identification to
their own $p_{T}$ is less than 0.12. Similarly, the corresponding ratio for
muons is set at 0.25.

(a)

(b)

(c)

(d)

(e)

(f)

Figure 5:  The Betti curves for $WH$ mode for benchmark A (top row) and
benchmark B (bottom row) with three representative values of $f$
($=0.02,0.05,0.1$). The legends $H_{k}$ stand for the $k-$th homology group,
i.e., $k-$dimensional holes.

(a)

(b)

(c)

(d)

(e)

(f)

Figure 6:  The Betti curves for $ZH$ mode for benchmark A (top row) and
benchmark B (bottom row) with three representative values of $f$
($=0.02,0.05,0.1$). The legends $H_{k}$ stand for the $k-$th homology group,
i.e., $k-$dimensional holes.

###  3.3 Point cloud data from collider observations

After the fast detector level simulation, we collect the reconstructed
particles event-wise and apply some basic kinematic cuts. For the $ZH$
resonant mode, we demand $p_{T}^{\ell_{1}}>30$ GeV and $p_{T}^{\ell_{2}}>30$
GeV. The event selection also requires $|m_{inv}(\ell_{1},\ell_{2})-m_{Z}|\leq
10$ GeV and $|\Delta R(\ell_{1},\ell_{2})|\leq 2.0$. However, for the $WH$
resonant mode, we require $N(\ell)=1$, $N(b_{jet})\geq 1$ and
$p_{T}^{\ell_{1}}>30$ GeV.

Approaching the problem from a data science perspective guided by collider
physics, we extract coordinates ($\eta$, $\phi$, $p_{T}$, $m_{inv}$) from
event files stored in the LHCO format. These coordinates serve as the
representation of the entire data, forming a point cloud. Given that these
four variables cover significantly different value ranges, we normalize each
variable to the [-1, 1] range. The number of events is normalized to the
integrated luminosity multiplied by the effective cross-section of the
considered process. We opt for an integrated luminosity of 100 fb-1 throughout
the study. The topological data analysis is performed using the python
interface of Gudhi v3.8.0 [52] library.

(a)

(b)

Figure 7: Mean binary classification accuracy of distinguishing benchmark A
and B under $WH$ ($ZH$) mode is shown on left (right).

##  4 Topological features

The goal of this study is to explore the dependence of topological properties
of the ensemble of events on the nearest neighbours. By adjusting the fraction
$f$ of nearest neighbours, the so-called DTM-based weights can be made to
highlight different regions of the topological space formed out of the
collider events. Below, we analyze several topological features, viz.,
persistent entropy, persistent amplitude, Betti curves and Betti areas.

###  4.1 Persistent Entropy

Let $D={(b_{i},d_{i})}$ represent the set of all birth-death pairs associated
with the $k$-th order homology group in the persistence diagram, where
$d_{i}<\infty$. The $k$-th order persistence entropy is expressed as

$\displaystyle S_{k}^{pe}$ $\displaystyle=-\sum_{i}p_{i}\log(p_{i}),$ (9)

where $p_{i}=\frac{d_{i}-b_{i}}{L_{D}}$ and $L_{D}=\sum_{i}{(d_{i}-b_{i})}$.

We present persistent entropy ($S_{k}^{pe}$) and its dependence on the
fraction of nearest neighbours in figure 2. In figure 2(a), we see $ZH$ mode
for benchmark A and B have larger zeroth-order entropy compared to $WH$ mode.
Interestingly, the $S_{0}^{pe}$ for di-lepton (mono lepton) SM background has
the lowest (highest) magnitude. So far as the first homology group is
concerned, we see a sharp drop in $S_{1}^{pe}$ for all modes at $f\approx
0.05$ except for $2\ell+\mathrm{MET}$. However, for $S_{2}^{pe}$ all modes
except for $2\ell+\mathrm{MET}$ have similar magnitude. There appears to be a
minimum for all modes at $f\approx 0.1$.

###  4.2 Persistent Amplitude

The persistent amplitude is defined on the set $D$ of persistent
($birth$,$death$) pairs as a function

$\displaystyle A:D\rightarrow\mathbb{R},$

for which there exists a vectorization

$\displaystyle\Phi:D\rightarrow V,$

with $V$ being a normed space, such that

$\displaystyle A(x)$ $\displaystyle=|\Phi(x)|$ (10)

for all $x\in D$. We present persistent amplitude in figure 3. The Wasserstein
metric is chosen for the computation of distances. Persistent amplitude for
the zeroth homology group, $A_{0}$, has an almost opposite trend compared to
$S_{0}^{pe}$. $H_{0}$ corresponds to the connected components in the
simplicial complex. Thus, it seems $2\ell+\mathrm{MET}$ has the largest
density of connected components leading to largest amplitude and most stable
persistent diagraph for $H_{0}$. This is not the case for higher-order
persistent amplitudes. For $A_{1}$ and $A_{2}$, we see minima at $f\approx
0.1$. We also find that $S_{2}^{pe}$ (see figure 2(c)) and $A_{2}$ (see figure
3(c)) for $WH$ mode associated with benchmark B has the lowest value.

###  4.3 Betti curves and Betti Area

Betti area is the area under the curves of the Betti number across the
filtration parameter. We present the Betti areas in figure 4. In figure 4(a),
(b) and (c), we see the $2\ell+\mathrm{MET}$ background process has the
highest Betti area followed by the SM background process,
$1\ell+\mathrm{MET}$. As before, the $WH$ mode for spectrum B has the lowest
value for all orders of homology dimensions. $BA_{1}$ (see figure 4(b)) and
$BA_{2}$ (see figure 4(c)) for all modes attain minimum value around $f\approx
0.1$.

In figure 5 and figure 6, we present Betti curves for all decay modes for
three representative $f$ ($=0.02,0.05,0.1$) values. As we increase the
fraction of nearest neighbours, we see that the cut-off value for filtration
weight also shifts. This may be intuitively understood from the fact that the
DTM measure discussed in equation 3 is essentially a function of the Euclidean
metric and $f$. Thus, the first simplicial complex in the filtration comes
from a larger number of neighbours, leading to a larger weight $w_{i}$. In
figure 5 and figure 6, we also notice that Betti curves for zeroth and first
homology group peak at filtration weight $\approx 0.2$ irrespective of the
fraction of the nearest neighbours.

##  5 Machine learning classification

We perform Support Vector Machine (SVM) based binary classification using each
of the three features discussed above to distinguish the new physics scenarios
(benchmark A and B). We find $100\%$ accuracy in $WH$ mode and around
$70\%-80\%$ accuracy for $ZH$ mode. Figure 7 presents the variation of the
mean accuracy for both $WH$ and $ZH$ modes with respect to the variation in
$f$. We also find that for a higher fraction of nearest neighbours in $ZH$
mode, the mean classification accuracy is slightly higher for persistent
entropy ($S^{pe}_{k}$) compared to Betti area ($BA_{k}$) and persistent
amplitude ($A_{k}$).

To enhance the reliability of this result, we perform 15 iterations of the
classification task to determine the mean accuracies presented in figure 7.
This meticulous approach serves to diminish the influence of random variations
in the data, ensuring the robustness of our findings.

##  6 Conclusion

In the presence of many proposed BSM models, it is all the more difficult to
distinguish the most viable one when we see some signature in collider data.
However, global topological properties of the observations at colliders could
be an important approach to uncover new physics scenarios. In this preliminary
work, we have demonstrated that with suitable weights attached to the
simplices, it is possible to highlight important features associated with the
topological clusters in the collider data.

From a philosophical standpoint, these topological properties signify that the
system, in its entirety, holds equal importance as its individual components.
The essence of fundamental laws of Nature might be discerned within the
intricate global relations and geometry linked to observations at the
colliders. This study marks an initial exploration in that direction.

## Declaration of competing interest

The authors declare that they have no known competing financial interests or
personal relationships that could have appeared to influence the work reported
in this paper.

## Acknowledgements

JB thanks IKSMHA Center, IIT Mandi and IKS Center, ISS Delhi, for their
support while part of the work was completed.

## References

  * Aad et al. [2012] G. Aad, T. Abajyan, B. Abbott, J. Abdallah, S. A. Khalek, A. A. Abdelalim, R. Aben, B. Abi, M. Abolins, O. AbouZeid, et al.,  Observation of a new particle in the search for the standard model higgs boson with the atlas detector at the lhc,  Physics Letters B 716 (2012) 1–29.
  * Chatrchyan et al. [2012] S. Chatrchyan, V. Khachatryan, A. M. Sirunyan, A. Tumasyan, W. Adam, E. Aguilo, T. Bergauer, M. Dragicevic, J. Erö, C. Fabjan, et al.,  Observation of a new boson at a mass of 125 gev with the cms experiment at the lhc,  Physics Letters B 716 (2012) 30–61.
  * Franceschini et al. [2022] R. Franceschini, D. Kim, K. Kong, K. T. Matchev, M. Park, P. Shyamsundar,  Kinematic Variables and Feature Engineering for Particle Phenomenology (2022). arXiv:2206.13431.
  * Debnath et al. [2016a] D. Debnath, J. S. Gainer, D. Kim, K. T. Matchev,  Edge Detecting New Physics the Voronoi Way,  EPL 114 (2016a) 41001. doi:10.1209/0295-5075/114/41001. arXiv:1506.04141.
  * Debnath et al. [2016b] D. Debnath, J. S. Gainer, C. Kilic, D. Kim, K. T. Matchev, Y.-P. Yang,  Identifying Phase Space Boundaries with Voronoi Tessellations,  Eur. Phys. J. C 76 (2016b) 645. doi:10.1140/epjc/s10052-016-4431-z. arXiv:1606.02721.
  * Matchev et al. [2020] K. T. Matchev, A. Roman, P. Shyamsundar,  Finding wombling boundaries in LHC data with Voronoi and Delaunay tessellations,  JHEP 12 (2020) 137\. doi:10.1007/JHEP12(2020)137. arXiv:2006.06582.
  * Mullin et al. [2021] A. Mullin, S. Nicholls, H. Pacey, M. Parker, M. White, S. Williams,  Does SUSY have friends? A new approach for LHC event analysis,  JHEP 02 (2021) 160\. doi:10.1007/JHEP02(2021)160. arXiv:1912.10625.
  * Nachman and Thaler [2021] B. Nachman, J. Thaler,  Learning from many collider events at once,  Phys. Rev. D 103 (2021) 116013. doi:10.1103/PhysRevD.103.116013. arXiv:2101.07263.
  * Taylor et al. [2015] D. Taylor, F. Klimm, H. A. Harrington, M. Kramár, K. Mischaikow, M. A. Porter, P. J. Mucha,  Topological data analysis of contagion maps for examining spreading processes on networks,  Nature communications 6 (2015) 7723.
  * Topaz et al. [2015] C. M. Topaz, L. Ziegelmeier, T. Halverson,  Topological data analysis of biological aggregation models,  PloS one 10 (2015) e0126383.
  * Lloyd et al. [2016] S. Lloyd, S. Garnerone, P. Zanardi,  Quantum algorithms for topological and geometric analysis of data,  Nature communications 7 (2016) 10138.
  * Gidea and Katz [2018] M. Gidea, Y. Katz,  Topological data analysis of financial time series: Landscapes of crashes,  Physica A: Statistical Mechanics and its Applications 491 (2018) 820–834.
  * Saggar et al. [2018] M. Saggar, O. Sporns, J. Gonzalez-Castillo, P. A. Bandettini, G. Carlsson, G. Glover, A. L. Reiss,  Towards a new approach to reveal dynamical organization of the brain using topological data analysis,  Nature communications 9 (2018) 1399.
  * Sizemore et al. [2019] A. E. Sizemore, J. E. Phillips-Cremins, R. Ghrist, D. S. Bassett,  The importance of the whole: topological data analysis for the network neuroscientist,  Network Neuroscience 3 (2019) 656–673.
  * Murugan and Robertson [2019] J. Murugan, D. Robertson,  An Introduction to Topological Data Analysis for Physicists: From LGM to FRBs (2019). arXiv:1904.11044.
  * Cole and Shiu [2019] A. Cole, G. Shiu,  Topological data analysis for the string landscape,  Journal of High Energy Physics 2019 (2019) 1–31.
  * Chazal and Michel [2021] F. Chazal, B. Michel,  An introduction to topological data analysis: fundamental and practical aspects for data scientists,  Frontiers in artificial intelligence 4 (2021) 108.
  * Beuria [2023] J. Beuria,  Persistent homology of collider observations: When (w) hole matters,  Physics Letters B 846 (2023) 138188.
  * Csaki [1996] C. Csaki,  The minimal supersymmetric standard model,  Modern Physics Letters A 11 (1996) 599–613.
  * Martin [1998] S. P. Martin,  A supersymmetry primer,  in: Perspectives on supersymmetry, World Scientific, 1998, pp. 1–98.
  * Baer and Tata [2006] H. Baer, X. Tata, Weak scale supersymmetry: From superfields to scattering events, Cambridge University Press, 2006.
  * Ellwanger et al. [2010] U. Ellwanger, C. Hugonie, A. M. Teixeira,  The next-to-minimal supersymmetric standard model,  Physics Reports 496 (2010) 1–77.
  * Cembranos et al. [2007] J. A. Cembranos, J. L. Feng, L. E. Strigari,  Exotic collider signals from the complete phase diagram of minimal universal extra dimensions,  Physical Review D 75 (2007) 036004.
  * Datta et al. [2010] A. Datta, K. Kong, K. T. Matchev,  Minimal universal extra dimensions in calchep/comphep,  New Journal of Physics 12 (2010) 075017.
  * Beuria et al. [2018] J. Beuria, A. Datta, D. Debnath, K. T. Matchev,  Lhc collider phenomenology of minimal universal extra dimensions,  Computer Physics Communications 226 (2018) 187–205.
  * Davidson and Haber [2005] S. Davidson, H. E. Haber,  Basis-independent methods for the two-higgs-doublet model,  Physical Review D 72 (2005) 035004.
  * Branco et al. [2012] G. C. Branco, P. Ferreira, L. Lavoura, M. Rebelo, M. Sher, J. P. Silva,  Theory and phenomenology of two-higgs-doublet models,  Physics reports 516 (2012) 1–102.
  * Anai et al. [2020] H. Anai, F. Chazal, M. Glisse, Y. Ike, H. Inakoshi, R. Tinarrage, Y. Umeda,  Dtm-based filtrations,  in: Topological Data Analysis: The Abel Symposium 2018, Springer, 2020, pp. 33–66.
  * Ham et al. [2005] S. Ham, Y. Jeong, S. Oh,  Electroweak phase transition in an extension of the standard model with a real higgs singlet,  Journal of Physics G: Nuclear and Particle Physics 31 (2005) 857.
  * Barger et al. [2008] V. Barger, P. Langacker, M. McCaskey, M. J. Ramsey-Musolf, G. Shaughnessy,  Cern lhc phenomenology of an extended standard model with a real scalar singlet,  Physical Review D 77 (2008) 035005.
  * Guo and Wu [2010] W.-L. Guo, Y.-L. Wu,  The real singlet scalar dark matter model,  Journal of High Energy Physics 2010 (2010) 1–13.
  * Enqvist et al. [2014] K. Enqvist, S. Nurmi, T. Tenkanen, K. Tuominen,  Standard model with a real singlet scalar and inflation,  Journal of Cosmology and Astroparticle Physics 2014 (2014) 035.
  * Feng et al. [2015] L. Feng, S. Profumo, L. Ubaldi,  Closing in on singlet scalar dark matter: Lux, invisible higgs decays and gamma-ray lines,  Journal of High Energy Physics 2015 (2015) 1–13.
  * Kanemura et al. [2016] S. Kanemura, M. Kikuchi, K. Yagyu,  Radiative corrections to the higgs boson couplings in the model with an additional real singlet scalar field,  Nuclear Physics B 907 (2016) 286–322.
  * Kurup and Perelstein [2017] G. Kurup, M. Perelstein,  Dynamics of electroweak phase transition in singlet-scalar extension of the standard model,  Physical Review D 96 (2017) 015036.
  * Chiang et al. [2019] C.-W. Chiang, Y.-T. Li, E. Senaha,  Revisiting electroweak phase transition in the standard model with a real singlet scalar,  Physics Letters B 789 (2019) 154–159.
  * Carlsson [2009] G. Carlsson,  Topology and data,  Bulletin of the American Mathematical Society 46 (2009) 255–308.
  * giotto-tda library [2023] giotto-tda library, Tutorials and examples, 2023\. URL: https://giotto-ai.github.io/gtda-docs/0.5.1/notebooks.
  * library – Topological data analysis and geometric inference in higher dimensions [2023] G. library – Topological data analysis, geometric inference in higher dimensions, Tda-tutorial, 2023. URL: https://github.com/GUDHI/TDA-tutorial.
  * Baccini et al. [2022] F. Baccini, F. Geraci, G. Bianconi,  Weighted simplicial complexes and their representation power of higher-order network data and topology,  Physical Review E 106 (2022) 034319.
  * Staub [2014] F. Staub,  SARAH 4 : A tool for (not only SUSY) model builders,  Comput. Phys. Commun. 185 (2014) 1773–1790. doi:10.1016/j.cpc.2014.02.018. arXiv:1309.7223.
  * Staub [2015] F. Staub,  Exploring new models in all detail with SARAH,  Adv. High Energy Phys. 2015 (2015) 840780. doi:10.1155/2015/840780. arXiv:1503.04200.
  * Porod [2003] W. Porod,  SPheno, a program for calculating supersymmetric spectra, SUSY particle decays and SUSY particle production at e+ e- colliders,  Comput. Phys. Commun. 153 (2003) 275–315. doi:10.1016/S0010-4655(03)00222-4. arXiv:hep-ph/0301101.
  * Porod and Staub [2012] W. Porod, F. Staub,  SPheno 3.1: Extensions including flavour, CP-phases and models beyond the MSSM,  Comput. Phys. Commun. 183 (2012) 2458–2469. doi:10.1016/j.cpc.2012.05.021. arXiv:1104.1573.
  * Alwall et al. [2014] J. Alwall, R. Frederix, S. Frixione, V. Hirschi, F. Maltoni, O. Mattelaer, H. S. Shao, T. Stelzer, P. Torrielli, M. Zaro,  The automated computation of tree-level and next-to-leading order differential cross sections, and their matching to parton shower simulations,  JHEP 07 (2014) 079\. doi:10.1007/JHEP07(2014)079. arXiv:1405.0301.
  * Frederix et al. [2018] R. Frederix, S. Frixione, V. Hirschi, D. Pagani, H. S. Shao, M. Zaro,  The automation of next-to-leading order electroweak calculations,  JHEP 07 (2018) 185\. doi:10.1007/JHEP11(2021)085. arXiv:1804.10017, [Erratum: JHEP 11, 085 (2021)].
  * Ball et al. [2017] R. D. Ball, V. Bertone, S. Carrazza, L. D. Debbio, S. Forte, P. Groth-Merrild, A. Guffanti, N. P. Hartland, Z. Kassabov, J. I. Latorre, et al.,  Parton distributions from high-precision collider data: Nnpdf collaboration,  The European Physical Journal C 77 (2017) 1–75.
  * Bierlich et al. [2022] C. Bierlich, et al.,  A comprehensive guide to the physics and usage of PYTHIA 8.3 (2022). doi:10.21468/SciPostPhysCodeb.8. arXiv:2203.11601.
  * Cacciari and Salam [2006] M. Cacciari, G. P. Salam,  Dispelling the $N^{3}$ myth for the $k_{t}$ jet-finder,  Phys. Lett. B 641 (2006) 57–61. doi:10.1016/j.physletb.2006.08.037. arXiv:hep-ph/0512210.
  * Cacciari et al. [2012] M. Cacciari, G. P. Salam, G. Soyez,  FastJet User Manual,  Eur. Phys. J. C 72 (2012) 1896. doi:10.1140/epjc/s10052-012-1896-2. arXiv:1111.6097.
  * de Favereau et al. [2014] J. de Favereau, C. Delaere, P. Demin, A. Giammanco, V. Lemaître, A. Mertens, M. Selvaggi (DELPHES 3),  DELPHES 3, A modular framework for fast simulation of a generic collider experiment,  JHEP 02 (2014) 057\. doi:10.1007/JHEP02(2014)057. arXiv:1307.6346.
  * Maria et al. [2014] C. Maria, J.-D. Boissonnat, M. Glisse, M. Yvinec,  The gudhi library: Simplicial complexes and persistent homology,  in: Mathematical Software–ICMS 2014: 4th International Congress, Seoul, South Korea, August 5-9, 2014. Proceedings 4, Springer, 2014, pp. 167–174.