# Convolutional Neural Networks for the classification of glitches in
gravitational-wave data streams

Tiago Fernandes<EMAIL_ADDRESS>Departamento de Física da Universidade
de Aveiro, Campus de Santiago, 3810-183 Aveiro, Portugal  Samuel Vieira
<EMAIL_ADDRESS>Departamento de Física da Universidade de Aveiro, Campus
de Santiago, 3810-183 Aveiro, Portugal  Antonio Onofre<EMAIL_ADDRESS>Centro de Física das Universidades do Minho e do Porto (CF-UM-UP),
Universidade do Minho, 4710-057 Braga, Portugal  Juan Calderón Bustillo
<EMAIL_ADDRESS>Instituto Galego de Física de Altas Enerxías,
Universidade de Santiago de Compostela, 15782 Santiago de Compostela, Galicia,
Spain  Department of Physics, The Chinese University of Hong Kong, Shatin,
N.T., Hong Kong  Alejandro Torres-Forné<EMAIL_ADDRESS>Departamento
de Astronomía y Astrofísica, Universitat de València, C/ Dr Moliner 50, 46100,
Burjassot (València), Spain  Observatorio Astronómico, Universitat de
València, C/ Catedrático José Beltrán 2, 46980, Paterna (València), Spain
José A. Font<EMAIL_ADDRESS>Departamento de Astronomía y Astrofísica,
Universitat de València, C/ Dr Moliner 50, 46100, Burjassot (València), Spain
Observatorio Astronómico, Universitat de València, C/ Catedrático José Beltrán
2, 46980, Paterna (València), Spain

###### Abstract

We investigate the use of Convolutional Neural Networks (including the modern
ConvNeXt network family) to classify transient noise signals (i.e. glitches)
and gravitational waves in data from the Advanced LIGO detectors. First, we
use models with a supervised learning approach, both trained from scratch
using the Gravity Spy dataset and employing transfer learning by fine-tuning
pre-trained models in this dataset. Second, we also explore a self-supervised
approach, pre-training models with automatically generated pseudo-labels. Our
findings are very close to existing results for the same dataset, reaching
values for the F1 score of 97.18% (94.15%) for the best supervised (self-
supervised) model. We further test the models using actual gravitational-wave
signals from LIGO-Virgo’s O3 run. Although trained using data from previous
runs (O1 and O2), the models show good performance, in particular when using
transfer learning. We find that transfer learning improves the scores without
the need for any training on real signals apart from the less than 50 chirp
examples from hardware injections present in the Gravity Spy dataset. This
motivates the use of transfer learning not only for glitch classification but
also for signal classification.

##  I Introduction

The era of Gravitational-Wave (GW) Astronomy started in 2015 with the
detection of the GWs emitted in the coalescence of two black holes [1] by the
Advanced LIGO detectors [2]. To date, the LIGO-Virgo-KAGRA (LVK) collaboration
[2, 3, 4] has reported results from three observing runs (O1, O2, and O3)
which comprise 90 confident detections. All signals observed are consistent
with being produced in compact binary coalescences (CBCs), namely mergers of
binary black holes (BBH), binary neutron stars (BNS), and binaries of a
neutron star and a black hole (NSBH) [5, 6, 7, 8, 9]. With the start of the
fourth observing run (O4) in May 2023, the number of confident detections is
expected to significantly increase, yielding an estimated CBC rate of about
one detection per day [10].

GW detectors operate in extremely low noise conditions which may be disrupted
by the ground motion surrounding the detectors or earthquakes, storms or even
anthropogenic sources of noise [11]. More than 200 thousand auxiliary channels
are constantly monitored to minimize instrumental noise [12]. These include
angular drift of optics, light transmitted through mirrors as well as
actuation signals used to control optic position in order to ensure optical
cavity resonance [11, 12]. With a higher detection rate, transient noises of
instrumental or environmental origin, commonly dubbed “glitches” [13], will
become increasingly more of a concern for the LVK detectors. Since glitches
can mimic actual GW signals they hinder the sensitivity of the detectors by
increasing the false-alarm rate of true GW signals. Reducing the rate of
glitches requires a deep understanding of their nature and physical origin,
which in some cases is unclear. Blip glitches constitute a particularly
harmful example for the LIGO detectors, where an average rate of two such
glitches per hour of data was measured during O1 and O2 [14].

Many different approaches to classify noise transients and subtract them from
strain data have been developed over the years [15, 16, 17, 18, 19, 20, 21,
22, 23, 24, 25, 26, 27, 28, 29, 30, 31, 32, 33, 34]. Current LVK efforts
include e.g. BayesWave [17] and gwsubtract [25], which model glitches using
sine-Gaussian wavelets and use a linear subtraction algorithm to remove them
from the data stream. Moreover, GW searches (e.g. [35, 36, 37]) also use
strategies to veto glitches, as e.g. the $\chi^{2}$ time-frequency
discriminator of [15] or a sine-Gaussian veto [24]. Glitch classification and
mitigation using probabilistic principal component analysis has been put
forward in [18, 20, 31]. Moreover, the Gravity Spy citizen-science project
[13] has allowed to confidently divide Advanced LIGO glitches into classes,
and provides a reliably classified glitch catalog that can be used by noise
classification and subtraction methods. Glitches from the Advanced Virgo
detector have also been introduced in more recent versions of Gravity Spy. A
similar citizen-science project focused on the Advanced Virgo detector,
GwitchHunters [34], introduced the localization of glitches in time-frequency
space, as well as the search for correlations in the detector’s auxiliary
channels. A variety of methods that are becoming increasingly important for
glitch characterization are Machine Learning (ML) methods. Those have already
been extensively applied to this topic (see [38] and references therein). For
instance, ML and Deep Learning (DL) methods have been used to predict the
existence of glitches using only auxiliary channel data [16, 28] or using both
the GW strain and auxiliary channels [29]. Furthermore, dictionary learning
techniques have been used to reconstruct and remove blips from GW data streams
in [26, 27].

Another interesting line of work frames the glitch classification problem as a
computer vision task, where glitches are converted from a time-series to
spectrogram images, suitable for classification taking advantage of the rapid
advances of DL methods for computer vision. In Ref. [23] this approach was
successfully tested with simulated glitches, using a custom-made Convolutional
Neural Network (CNN). After this initial success, images from the Gravity Spy
dataset have also been used to train CNNs for glitch classification. First,
custom-made CNNs with two to five layers were trained from scratch for 200
epochs, and individual models achieved satisfactory results, with accuracies
over 95% [39]. Subsequently, deeper pre-trained networks with more powerful
architectures were trained on the same dataset, and it was found that transfer
learning reduced training time and improved results, reaching the state-of-
the-art accuracy of 98.8% in the Gravity Spy dataset [21]. In that work,
popular CNN architectures like Inception [40, 41], ResNets [42], and VGGs [43]
were trained using transfer learning, for up to 100 epochs, and it was found
that, for some architectures, transfer learning allowed to achieve accuracies
higher than 98% within 10 epochs.

Recent advances in computer vision research, like the discovery of better
training recipes [44, 45, 46, 47] or more powerful architectures [48, 49, 50,
51], could potentially improve glitch classification even more. With this in
mind, this paper focuses on the classification of glitches in image format
(spectrograms), using the latest available techniques to attempt to improve
the state-of-the-art. In particular, we aim at improving present-day results
by using the 1cycle training policy [44], trying a more modern and powerful
CNN architecture family, the ConvNeXt [48], training models using both
supervised and self-supervised methods, and using the typical three subset
split (train/validation/test) for better assessment of the model-
generalization performance. We also analyze in this paper the use of transfer
learning applied to O3 data. It should be stressed that the networks are
trained using glitches from the Gravity Spy catalog, which only includes O1
and O2 data, and they are not retrained or fine-tuned on O3 data. We find that
transfer learning significantly increases the flexibility of the algorithms
and their performance. This approach might be advantageous for future GW
observing runs, as it is not as sensitive to changes in the detector
background noise.

This paper is organized as follows: Section II discusses the supervised and
self-supervised DL methods that we employ for our study. In Section III we
present the dataset used. Section IV contains our results on glitch
classification including, as well, a preliminary analysis of the performance
of our methods with O3 data and actual GW signals. Finally, in Section V we
present our main conclusions and outline our plans for future work.

##  II Deep Learning methods

Deep Learning is a subfield of Machine Learning that relies on artificial
neural networks with many intermediate layers, each of which builds on the
representations learned by the previous layer in order to automatically
develop increasingly complex and useful representations of the data [52].
While shallow learning methods, i.e. methods with very few layers, rely on
carefully handcrafted features to produce useful results, DL requires no
feature engineering, as it can autonomously find what are the useful features,
considerably simplifying ML workflows.

According to the experience111Here, “experience” is used with same meaning as
in Tom Mitchell’s Machine Learning definition [53]. the algorithms acquire
during the training phase, DL methods can be categorized as unsupervised,
supervised or self-supervised algorithms. In all cases, the algorithms
experience an entire dataset, $\boldsymbol{X}$, which is a collection of $m$
data points ${\bf\it x}^{(i)}=(x_{0},x_{1},...,x_{n})$, each of which has $n$
features. Unlike unsupervised algorithms, in supervised learning algorithms
each example ${\bf\it x}^{(i)}$ has, in addition, an associated label,
$y^{(i)}$, which is the target for the prediction of the point. Similarly,
self-supervised learning uses the same dataset composed of $n$ samples, with
targets for the predictions indicated by the labels. However, the labels of
those same samples are replaced by automatically generated labels, referred to
as pseudo-labels. In this work, both supervised and self-supervised algorithms
are used.

###  II.1 Supervised Deep Learning

The ResNet architecture family [42] is used in this investigation as a
baseline for the classification task. ResNets are a family of CNNs [54, 55]
which use residual connections, i.e. direct paths between non-adjacent layers,
to allow training deeper models. In the original implementation, ResNets end
with a fully connected layer with 1000 units, corresponding to the number of
classes in the ImageNet dataset [56]. For other applications, the number of
neurons in this last layer is changed to match the number of classes.

The models were trained using the fastai library [57], taking advantage of its
implementation of the 1cycle training policy [44], and training using the
AdamW optimizer [47]. In addition, the learning rates were selected using the
learning-rate finder [44]. Mixed precision training [45] was also used to
reduce the memory requirements and speed up training.

In this work we employ the cross-entropy loss, which for the case of a single
example, is given by

$\mathcal{L}({\bf\it\theta})=-\sum_{k=1}^{K}{y}_{k}\log(\hat{p}_{k}).$ (1)

Above, ${y}_{k}$ denotes the target probability of class $k$, typically either
equal to 0 or 1, $K$ denotes the total number of classes (equal to the number
of neurons in the output layer), and $\hat{p}_{k}$ denotes the probability
that the example belongs to class $k$. A weighted loss function was also
tried, to penalize more the model for mistakes in the less represented
classes, by introducing a class-dependent weight, $w_{k}$, in the cross-
entropy loss:

$\mathcal{L}({\bf\it\theta})=-\sum_{k=1}^{K}w_{k}\leavevmode\nobreak\
{y}_{k}\log(\hat{p}_{k}).$ (2)

This approach was tried using class weights inversely proportional to the
number of samples in each class,

$w_{k}=\frac{1}{N_{k}},$ (3)

with $N_{k}$ being the total number of samples of class $k$. In addition, the
Effective Number of Samples (ENS) approach [58], which obtains the weights
associated with each class using

$w_{k}=\frac{1-\beta}{1-\beta^{N_{k}}},$ (4)

was also tried. Here, $\beta$ is a hyperparameter that allows to control the
re-weighting degree. Its value was fixed to $\beta=0.99$, following the
suggestion in [58]. In both cases, the class weights are normalized by
dividing each class weight by the sum of the class weights and multiplying by
the number of classes, in order to create weights closer to unity.

Moreover, the use of the focal loss function, introduced in Ref. [59], was
also tested, since it adds to the cross-entropy loss a modulating factor which
down-weights easy examples, that is, examples where the model is more
confident. The focal loss is defined by

$\mathcal{L}({\bf\it\theta})=-\sum_{k=1}^{K}w_{k}\leavevmode\nobreak\
(1-\hat{p}_{k})^{\gamma}\leavevmode\nobreak\ y_{k}\log(\hat{p}_{k}),$ (5)

where $\gamma$ is the focusing parameter, which was kept to the default value
of 2.0 [59].

In addition to training ResNets from scratch, transfer learning [60] has also
been considered in our investigation. Transfer learning is a technique that
allows to take advantage of knowledge learned in one domain, typically from a
task that has many labelled examples, and to apply it to a different but
related task which may have a limited number of examples. This technique is
one of the cornerstones of the fastai library, as it allows to train more
accurate models more quickly, and using less data, as compared to training the
same models from scratch [61]. We found it interesting to explore transfer
learning here to also understand if the networks (pre-trained in datasets
which are different from the ones they are applied to) improve performance (in
comparison to when no transfer learning is used), also allowing them to be
more robust against changes in the detector background.

In the default fastai transfer-learning approach, after assigning the
pretrained weights, the final layer is replaced with a fully connected layer
with the correct number of outputs, and it is initialized with entirely random
weights. Then, the networks are trained with fastai’s fine_tune training
schedule [62], which first trains the last layer with the others kept frozen,
and then trains the whole network using lower learning rates, while also
taking advantage of discriminative learning rates.

For the networks trained with transfer learning, the ConvNeXt network family
[48] was also tried. The design of this CNN family is inspired on the
successful Vision Transformers such as ViT [49], allowing it to achieve very
competitive performances (see e.g. [63]). More details about the architectures
used in this work can be found in the appendix of [64].

Figure 1: Scheme of the self-supervised DL framework during training. In the
dashed green box (a), labels are automatically generated for a pretext task
(i.e. the image transformation imposed on the data is to be predicted by the
model). In the dashed purple box (b), the architecture’s last layer is changed
to solve a related problem using labeled data.

###  II.2 Self-supervised Deep Learning

Self-supervised DL is a recently introduced method in the field of ML [65].
This method involves training a model for a pretext task before training the
model for the desired (downstream) task, as sketched in Figure 1. For
instance, a model can be trained to detect what kind of visual transformation
is applied to an image and, after that, a transfer of knowledge be done to
another model that is trained with labeled data for the downstream task. To
solve the pretext task, the samples contained in the dataset must have their
pseudo-labels generated, as observed in box (a) of Figure 1. In this case,
each one is associated with the type of visual transformation applied to the
image. The model is then trained, allowing it to learn features/patterns about
the images in the context of this so-called pretext task. The training section
is approached in the same way as in supervised learning, with the objective of
minimizing the loss function in order to match the model’s prediction to the
targets, in this case given by the pseudo-labels.

Figure 2: Examples of the different glitch classes from the Gravity Spy
dataset. Note that the classes None_of_the_Above and No_Glitch are not
represented in the figure. Moreover, the timescale is not equal for all images
and the grey-scale spectrograms are shown with a colour map for better
visualization.

Following this first training section, transfer learning is applied by
removing the last layer of the trained model and replacing it with a randomly
initiated layer of the same type. This updated model is then trained for the
downstream task, as shown in box (b) of Figure 1, where the dataset used to
feed the neural network already has human-made labels. In theory, the pretext
task allows the model to learn certain useful patterns from the images, making
it so that the downstream model is easier to train. Usually the pretext task
tackles a problem that is a subcategory of the problem associated with the
pretext task. This technique is more appropriate if there is a high amount of
samples, but only few of those are labelled.

The architecture we chose for the self-supervised approach was the ResNet18,
with randomly initialized parameters. This model is trained using a dataset
composed of images that only correspond to a specific time-span window. Data
augmentation is used to increase the variety in the dataset and avoid
overfitting. Similarly to the supervised DL methods discussed before, the
AdamW optimizer and cross-entropy loss function are used during training,
using the 1cycle training policy and the learning rate finder to select a
learning rate.

###  II.3 Model evaluation

The metric we select to compare possible model configurations is the F1 score.
This is the harmonic mean of precision (the ratio of correct positive
predictions to the total number of positive predictions) and recall (the ratio
between the number of correct positive predictions and the total number of
positive examples)

$\textrm{F1
score}=\frac{2}{\frac{1}{\textrm{precision}}+\frac{1}{\textrm{recall}}}=\frac{\textrm{TP}}{\textrm{TP}+\frac{\textrm{FN}+\textrm{FP}}{2}}.$
(6)

Above, TP, FP and FN respectively denote the total number, true positives,
false positives and false negatives.

The F1 score can be computed for each class but this raises the issue of
having multiple metrics. In order to have one single overall metric, the
macro-averaged F1 score is used instead. This metric is calculated by plugging
in the macro-averaged precision and macro-averaged recall, which are
respectively the precision and recall averaged across all classes, in Eq. (6).

As models that train faster are easier to optimize and can also result in
faster inference times, a new metric was created to penalize models that take
longer to train:

$\textrm{combined\\_F1\\_time}=\textrm{F1\\_score}-\textrm{total\\_runtime}/3\times
10^{4}.$ (7)

The numerical value in the equation was conveniently chosen so that an
increase of 0.2 percentage points in the F1 score will only be accepted if it
adds less than 60 seconds to the training time. Thus, this metric allows to
search for better models without increasing too much the training time.

The metrics were computed on the validation dataset for the candidate models,
with only the best configurations being chosen for evaluation on the test set.
The use of three separate subsets in the model training and evaluation process
is of utmost importance. The inclusion of an intermediate validation set
allows to evaluate different model candidates on this set and select them
accordingly based on their performance. Then, the selected models can be
evaluated on the test set, which had not been used to guide previous
decisions, to provide a more accurate estimation of the models’
generalization. Had the intermediate set not been used, the test set would
provide an over-confident estimate of the models’ performance, as they had
been optimized for that same dataset.

##  III Dataset

The Gravity Spy dataset, introduced in [13, 39], is a collection of
spectrograms of (mostly) glitches identified during the O1 and O2 observing
runs in the Advanced LIGO detectors. This dataset presents a multi-class
classification problem, where the goal is to assign the only correct glitch
class to each noise sample. Each sample contains four spectrogram images
centered at the same instant in time but with different durations of 0.5, 1.0,
2.0 and 4.0 seconds, all represented as $140\times 170$ pixels grey-scale
images, as seen in Figure 2. In this work we directly use the public images
from Gravity Spy. It can be observed that the glitch classes vary widely in
duration, frequency range and shape. Nevertheless, some classes can be easily
mistaken for each other due to their similar morphology, like the Blip and
Tomte classes [31], or if they are not visualized using the appropriate time
window. For instance, a Repeating_Blips sample may be identified as a Blip if
the window is too small.

In the version of the dataset used in this
work222https://zenodo.org/record/1476156, v1.0, there are 8583 glitch samples
distributed unevenly over 22 different classes, which are thoroughly described
in [13]. The majority of examples are Blips, with almost 1900 samples, while
there are five minority classes (Paired_Doves, Wandering_Line, Air_Compressor,
Chirp, and None_of_the_Above) with less than 190 examples (10% of the number
of Blips), the less represented being the Paired_Doves, with only 27 examples.
The Chirp class does not actually represent glitches but instead “hardware
injections”, which are simulated signals created to resemble real GWs from
CBCs, and used for the testing and calibration of the detectors [39]. The
Gravity Spy dataset is already split, using stratified sampling to ensure
similar distributions in each subset, into training (70%), validation (15%)
and test (15%) sets, to facilitate the comparison of different methods.

##  IV Results

###  IV.1 Supervised models

We start discussing the results of glitch classification for supervised DL
models. For a quick search for a baseline classifier the ResNet18 and ResNet34
architectures were chosen, with an adapted number of inputs for the first
convolutional layer, and using random weight initialization. Both
architectures were trained from scratch with fastai’s fit_one_cycle routine
over 15 epochs with the steepest point from the learning rate finder as the
maximum learning rate; with the other hyperparameters set to fastai’s
defaults.

Several approaches were tested regarding the amount and format of the
information provided to the model. In the simplest approach, the model had
only access to one of the four spectrograms for each example, always with the
same duration. This corresponds to views labelled single1 (0.5 s), single2
(1.0 s), single3 (2.0 s) and single4 (4.0 s). Thus, these single-view models
accepted batches of grey-scale (1-channel) $140\times 170$ (height $\times$
width) pixels images. The second approach was a merged view model inspired in
[66], which consists of placing the four single view $140\times 170$ images
next to each other, forming a $280\times 340$ pixels image. Finally, we
implemented an approach based on encoding information related to a different
time duration in each image channel, as introduced in [67]. Encoded views with
all combinations of two to four time durations were compared.

A hyperparameter grid search was run over the two model architectures and all
the possible views, in order to select the best views and choose a baseline
model. The results in terms of each view are shown in Figure 3. It was found
that the single views were outperformed by almost all the other views. The
highest average F1 scores were obtained by encoded134. Remarkably, this
outperformed merged view and encoded1234, which use all the available
information. Taking the total training time in consideration using the
combined_f1_time metric from Eq. (7), we conclude that encoded134 is the best
view, as the merged view takes about 6 more seconds per epoch (a 70% increase)
than view encoded134, while achieving slightly worse performance regarding the
F1 score. Therefore, encoded134 was selected as the view used for the rest of
the supervised models.

Figure 3: Comparison of the different views regarding combined_f1_time
computed over the validation set. The horizontal bars show the average of the
result of the two architectures, ResNet18 and ResNet34.

In addition, the results of the grid search were also evaluated in terms of
the chosen architecture, as depicted in Figure 4. This figure shows box plots
of the combined_F1_time for the validation set333In box plots (which we employ
in several figures in this paper) the lower limit of the box corresponds to
the $25^{\mathrm{th}}$ percentile of the data, and the upper limit to the
$75^{\mathrm{th}}$ percentile. The line which goes through the box represents
the median of the observations. The whiskers represent the maximum and minimum
(excluding outliers) of the quantity being displayed, and the individual
points (if any) are outliers, i.e. points that are more than 1.5 inter-
quartile distance away from the closest quartile.. We found that, on average,
ResNet18 achieves better results both on the F1 score and the training time,
resulting in a median combined_F1_time 0.6 percentage points higher than
ResNet34. Thus, the ResNet18 was chosen for the baseline model.

Figure 4: Box plots with the comparison of the architectures regarding
combined_f1_time, computed over the validation set.

After deciding the baseline configuration, five models were trained for 15
epochs each, using the same configuration as before, and selecting the
ResNet18 architecture and the encoded134 view. The F1 scores of the five
training runs range between 96.7% and 98.1%, and the best one was chosen as
the baseline model.

An attempt was made to tune the learning-rate choice method, the number of
epochs and the batch size, with the goal of further optimizing the model’s
performance. Hence, a Bayesian sweep was performed over 50 configurations of
the following three hyperparameters: (a) number of epochs, between 8 and 25;
(b) batch size, sampling from the set $[32,64,128,256]$; and (c) learning rate
finder function, sampling from the four available methods in fastai (steep,
valley, slide, minimum). It was found that no model obtained an F1 score
higher than the baseline, although some models reached a slightly better
combined_f1_time score by maintaining a comparable F1 score while training for
less epochs.

The weighted loss function from Eq. (2) was also tried, in order to further
penalize the model for mistakes in the least represented classes. The re-
weighting strategies were compared using the baseline configuration in
addition to a configuration equal to the baseline but with the ResNet34
architecture instead of the ResNet18. Each configuration/strategy combination
was run for 10 times, yielding the results shown in Figure 5. When the inverse
weighting strategy is introduced, the performance of the baseline decreases,
while the ResNet34’s F1 scores slightly increase. The effective re-weighting
strategy seems to result in similar performance for the baseline in comparison
to the models without re-weighting, while the ResNet34 models, once again,
slightly improve their performance.

Figure 5: Box plots of the F1 scores on the validation set for three different
weighting strategies using three different configurations.

Next, the focal loss from Eq. (5) was tried by training the baseline
configuration using this loss five times for each of the previous weighting
strategies (including the absence of re-weighting). The results are shown in
Figure 6. Focal loss does not seem to work with the inverse re-weighting
strategy and yields similar performance without re-weighting and using the
effective number of samples strategy. The results with focal loss are worse
than the results with cross-entropy in all cases.

Figure 6: Box plots of the F1 scores on the validation set for the cross-
entropy loss and focal loss functions, using the baseline configuration with
three different re-weighting strategies.

###  IV.2 Supervised models with transfer learning

After being unable to improve the baseline model while training from scratch,
an approach using transfer learning was attempted. Initially, models pre-
trained on the ImageNet dataset using the ResNet18 and ResNet34 architectures
were trained with fastai’s fine_tune routine with the same configuration as
the baseline model. As the results suggested that the ResNet18 architecture
may not be the best one for transfer learning, other architectures from the
ResNet family were also tried (ResNet26 and ResNet50) as well as architectures
from the ConvNeXt family (ConvNeXt_Nano and ConvNeXt_Tiny). Having chosen
these six architectures, a grid sweep was performed in order to compare them.
In particular, we trained each model for 4, 9 and 14 epochs (in addition to
the initial frozen epoch), using two different learning rate functions, steep
and minimum, while keeping the other hyperparameters at the default values. In
total, each architecture was trained six times, with the results shown in
Figure 7. It was observed that ResNet18 produced the best single run, but it
appears to be worse than all the other architectures when averages and medians
over multiple training runs are considered. The performance of the ResNet
family seems to peak with the ResNet34 architecture but it is surpassed, on
average, by the ConvNeXt family. ConvNeXt_Nano is close to ConvNeXt_Tiny in
terms of F1 score but is preferable when the training time is considered,
resulting in higher combined_f1_time values. For these reasons, ConvNeXt_Nano
was chosen as the architecture for subsequent optimizations.

Figure 7: Box plots with the comparison of the architectures regarding
combined_f1_time, computed over the validation set.

A Bayesian sweep was then performed to optimize the hyperparameters of the
transfer learning model, with the ConvNeXt_Nano architecture. Fifty
configurations were evaluated, given the following parameter space: (a) number
of frozen_epochs, with a uniform integer distribution between 1 and 5; (b)
number of unfrozen epochs, using a uniform integer distribution between 1 and
10; (c) batch size, sampling from the set $[32,64,128,256]$; (d) learning rate
finder function, sampling from the four available methods in fastai (slide,
valley, steep. minimum); (e) loss function: either cross-entropy or focal
loss; and (f) re-weighting strategy, which could be none, inverse of class
frequency, or ENS.

The six overall best configurations of the sweep as well as the three best
models trained for 6 epochs or less were more thoroughly evaluated by re-
training them five times each. From these, the three most promising
configurations in terms of F1 score were selected, which have the following
hyperparameters: (a) tl_best5: 2 frozen and 8 unfrozen epochs, batch size of
64, minimum learning rate selection function and cross-entropy loss with
inverse class re-weighting; (b) tl_fast1: the same hyperparameters as tl_best5
except for a quicker training with 1 frozen and 5 unfrozen epochs; and (c)
tl_best6: 1 frozen and 9 unfrozen epochs, batch size of 256, steep learning
rate selection function and focal loss with ENS re-weighting. They were
trained five more times, totaling ten models for each, and the results are
shown in Figure 8.

Figure 8: Box plots of the F1 scores on the validation set of the ten runs for
the baseline and the best three configurations selected from the second
Bayesian sweep.

The highest score was obtained by the best run of the tl_best5 configuration,
which beat the baseline’s best run. The tl_best5 and tl_fast1 configurations
achieved F1 scores higher than the baseline median in all runs. Moreover,
tl_fast1 has the highest median among all models.

The model with the highest overall score, corresponding to the best run of
tl_best5, was chosen as the transfer learning classifier. In comparison with
the baseline model, it only increases the F1 score from 98.07% to 98.21%, but
the transfer learning configuration appears to result in more stable
performances across multiple training trials, with its worst performance
higher than the baseline median. In fact, the baseline configuration achieves
an F1 score of $(97.4\pm 0.5)\%$, less than tl_best5’s $(97.8\pm 0.2)\%$ and
tl_fast1’s $(97.9\pm 0.2)\%$. Note that the previous F1 scores are reported in
the form $(\mu\pm\sigma)\%$, where $\mu$ is the distribution average and
$\sigma$ its standard deviation. We highlight that these models were very
quick to train, with training times between 1 and 2 minutes using a NVIDIA RTX
A4000 GPU (16 GB VRAM).

###  IV.3 Self-supervised model

In contrast to supervised learning, self-supervised learning is a two-phase
process that partly uses automatically generated labels, i.e. pseudo-labels,
during the training of the architecture. This can be considered an advantage
for datasets where data is abundant but the labeling of the samples is scarce
or incorrect. As stated in Section II.2, the self-supervised learning
framework first needs to solve the pretext task and then the downstream task.
This means that this approach utilizes two distinct training phases, each one
using its own independent dataset. The first training phase gives rise to the
first model (I), which solves the pretext task. In the second phase, we apply
transfer learning to the previous model, to obtain the second and final model
(II), which solves the downstream task.

After applying the image transformations (in the first phase) or image
augmentations (in the second phase), all the images for both datasets are then
resized to 128$\times$128 pixels through interpolation. This is done to make
the models train faster since the input’s size is smaller. Just like in
supervised models, the F1 score and accuracy are the metrics being examined.
Both metrics are only applied to the validation dataset. In both phases the
batch size is set to 64.

Model (I) was trained from scratch using pseudo-labels, with a dataset
composed of single view images from all four time-span windows (0.5 s, 1.0 s,
2.0 s, and 4.0 s). Each signal has thus four independent single view images
associated with it. The visual transformation is picked from a list of
operations like random crops (crop), horizontal and vertical flips
(flipVertical and flipHorizontal), color jitters (colorize), and rotations of
90º and 270º (rotate90 and rotate270). The no-transformation case was also
used (labeled as noTransformation). For each image, one visual transformation
is randomly selected from the list. Once the visual transformation is applied,
a pseudo-label is associated with it. For instance, if the vertical flip
option is selected from the list, the resulting image will be flipped
vertically and then resized to 128 pixels by 128 pixels. The resulting pseudo-
label is denominated as the flipVertical class.

For simplicity, model (II) only utilizes data with 4-second time windows,
although we note that certain types of glitches require longer windows to be
recognized. Similar to the previous dataset, all images are single views. As a
consequence, the dataset used to train model (I) has four times more images
for each epoch compared to the dataset from model (II). During the training
phase of this model, the original labels are used instead of the pseudo-
labels. In this phase, data augmentation is applied to avoid overfitting, as
explained below. Before resizing the image, each one is rotated between -5 and
5 degrees, then undergoes a color jitter, and finally a random crop is
applied. Note that since the original labels are used, there is no longer a
correlation between the class and the type of augmentation applied to the
image.

For both models, the learning rate finder is used to determine the values of
the learning rate. For model (I) the learning rate is $6.92\times 10^{-3}$,
and for model (II) is $5.75\times 10^{-4}$. Since the 1cycle policy is used
during training, these values actually correspond to the maximum learning
rate, instead of a single fixed learning rate value for all epochs.

Model (I) was trained during 10 epochs since both the values of the loss and
the metrics quickly converged after the first few epochs, as shown in Figure
9(a) and (c). Model (II), however, was trained for 30 epochs. The evolution of
the metrics for this model, as well as the loss, are represented in Figure
9(b) and (d). The values start to converge somewhere between the
$10^{\mathrm{th}}$ and the $15^{\mathrm{th}}$ epoch. In the first five epochs,
only the parameters of the last layer were allowed to change with a maximum
learning rate determined by the learning rate finder. In the remaining 25
epochs, all architecture parameters can be modified, with the maximum learning
rate being $5.75\times 10^{-5}$, a 10 times smaller rate than in the first
five epochs. The resulting final model (II) has an F1 score of 92.74% and an
accuracy of 96.74%, evaluated on the validation dataset.

Figure 9: Epoch evolution of the accuracy and macro-averaged F1 score for
training of model I (a) and model II (b). The corresponding losses are
represented in panels (c) and (d).

###  IV.4 Evaluation of the best models on the test set

The best models found in the previous sections are now evaluated on the test
dataset, for both supervised and self-supervised approaches. It is worth
recalling that this dataset was not used during the training of the neural
networks.

From the supervised models, both the baseline and tl_best5 models were
selected to be evaluated on the test dataset, based on their performance on
the validation dataset. We found that the F1 score of the baseline slightly
dropped to 97.18%, while tl_best5, the best model on the validation dataset,
performed worse than the baseline on the test dataset, with an F1 score of
96.84%. Nevertheless, the F1 scores obtained on data which was never seen by
the models nor used to perform choices regarding the hyperparameter
optimization, are still very high for both models. The confusion matrix of the
baseline model, using the predictions on the test dataset, is presented in
Figure 10. Both precision and recall are, at least, 95% for 19 out of the 22
classes of glitches. The model appears to have more difficulties in the
Air_Compressor and None_of_the_Above classes. This may be explained by the
fact that, for these classes, the number of images in the training set is
significantly smaller than for the other classes. Note that the
None_of_the_Above class, in particular, has a very diverse distribution since
it encompasses every glitch outside the defined classes. This class has been
deprecated from the glitch classes for O3 data [68].

Figure 10: Normalized confusion matrix of the predictions of the baseline
model over the test dataset, for the supervised approach.

For the self-supervised model, the corresponding test dataset confusion matrix
is shown in Figure 11. The model shows that the recall and precision scores
are, at least, 95% for 14 out of the 22 glitch classes. This is slightly worse
than the result obtained under the supervised approach, as expected. In
particular, the model struggles to identify several other classes beyond those
already signaled for the supervised approach above, like the Wandering_Line
and Repeating_Blips classes. This happens due to the fact that, for
simplicity, these models were only trained with datasets consisting of
4-second wide signals in contrast to some of the previous models that were
trained with encoded views. As a result of the resized augmentation, the
performance of the model might also be affected, as reducing the number of
pixels will reduce the image’s details. For these reasons, this model did not
perform as well as the supervised trained models. However, the performance is
still high, with an overall accuracy of 96.50% (relative to all classes) and
an macro-averaged F1 score of 94.15% on the test dataset. For completeness,
the confusion matrix of the model trained with pseudo-labels, before transfer
learning is applied (first training phase), is also shown in Figure 12. We
note that the model has some difficulties distinguishing between the pseudo-
labels resize and flipHorizontal, which makes sense since some glitches have
vertical symmetry.

Figure 11: Normalized confusion matrix of the predictions of the self-
supervised trained model (II), for the test dataset. Figure 12: Normalized
confusion matrix of the predictions of the self-supervised trained model (I)
over the test dataset, before transfer learning is applied

The two best supervised models, as well as the best self-supervised model,
were compared to the literature in terms of their F1 score and accuracy. The
results are shown in Table 1. Both supervised models obtained competitive
results, with accuracies higher than the merged view CNNs reported in [66,
39], and the baseline model’s performance was also better than the hard fusion
ensemble of CNNs [39]. This model, which uses a ResNet18 architecture trained
from scratch, is only slightly worse than the fine-tuned ResNet50 from [21],
with a decrease in F1 score of less than 0.2 percentage points. It should be
stressed, however, that Ref. [21] used a different dataset split, without a
validation dataset, resulting in more training data and a different test set.
More importantly, the absence of a validation set could imply that the results
reported by [21] are over-confident. They report the test set performance from
the epochs selected because they had the higher test set performance
themselves. On the other hand, our results from self-supervised model scores
show slightly lower values with respect to supervised approaches but the
reported numbers are still interestingly high and promising, as they are close
to the best performance achieved in Ref. [39] using a single view.

Table 1: Performance of different models on the Gravity Spy dataset. Model | F1 score (%) | accuracy (%) | Notes
---|---|---|---
single2 view CNN [39] | not reported | 96.81 | custom shallow CNN using only one view
merged view CNN [66] | not reported | 96.89 | different dataset version (20 classes)
merged view CNN [39] | not reported | 97.67 | improved version of [66]
hard fusion ensemble [39] | not reported | 98.21 | combines four CNNs
fine-tuned ResNet50 [67, 21] | 97.65 | 98.84 | different split (no validation set)
baseline [this work] | 97.18 | 98.68 | ResNet18 trained from scratch
tl_best5 [this work] | 96.84 | 98.14 | fine-tuned ConvNeXt_Nano
self-supervised model [this work] | 94.15 | 96.50 | self-supervised trained ResNet18

###  IV.5 Evaluation of the best models on GW signals

As mentioned before, the Gravity Spy dataset contains a few examples of the
Chirp class, which are simulated GW signals from CBCs. In addition, the best
models have shown perfect precision and recall for chirp classification in the
Gravity Spy test dataset (cf. Figure 10). These two reasons motivated us to
conduct a brief investigation of whether the models trained in this work would
be able to identify real detections of GWs as Chirps, using data from the
Advanced LIGO detectors collected during the third LVK observing run (O3).

The strain data for the confident detections in the O3 run, for each Advanced
LIGO detector, Hanford (“H1”) and Livingston (“L1”), was obtained from the
Gravitational Wave Open Science Center (GWOSC) [69]. For each data sample, a
Q-graph of the strain data in the 5 seconds around the GPS time of the
detection was obtained using the q_transform method from the GWpy library
[70]. Furthermore, four crops of this Q-graph, centered at the time with the
highest energy, were obtained, one for each different single view duration in
the Gravity Spy dataset. From the resulting 178 samples (89 events times, 2
detectors), many were discarded due to the GW signal being buried in the
background noise, with no visible chirp, resulting in 50 samples with clearly
visible chirps.

The views with 0.5, 2.0 and 4.0 seconds of duration were combined taking
advantage of the RGB color channels, as done previously, to obtain the
encoded134 view. The resulting images were passed through the baseline and
tl_best5 model. The results are shown in panel (a) of Figure 13. The
performance of the two supervised models tested with real GW signals from the
O3 run is noticeably different, despite both having perfect performance in the
chirp class of the test dataset. On the one hand, the baseline model is very
dependent on the image creation pipeline. For this reason, an extra alignment
step for the colour channels was necessary to improve its performance. Even
for the aligned images, the model achieves a mere 10% recall, assigning most
images to the None_of_the_Above class. On the other hand, the transfer
learning model achieves a 52% recall for the aligned images, correctly
identifying 26 out of the 50 GW samples. Moreover, the model is much more
robust to samples which are different from the ones used to train it (i.e.
with different background noise conditions), being practically unaffected by
the colour channel misalignment. One possible explanation for the better
performance and behaviour with samples slightly out of distribution could be
the benefit of pre-trained features, which were reported in [71] to produce a
boost in generalization even after fine-tuning.

Panel (b) of Figure 13 displays an illustrative sample of the O3 GW signals
used to evaluate the model, as well as the respective tl_best5 model
predictions. As expected, we found that the model performs well with chirps
similar to those present in the Gravity Spy dataset (top row), typically from
BBH mergers, but has more difficulties with different types of chirps (bottom
row) which are either much longer (in the case of BNS coalescences) or much
narrower (in the case of high-mass BBH mergers) than the chirps present in the
training dataset.

Finally we discuss the results for the self-supervised model, applied to the
4-second view images. The results are plotted in Figure 14. Panel (a) of this
figure indicates that most of the classified images are wrongly predicted as
Blips. This is likely due to the fact that Chirps and Blips in the 4-second
window images are quite similar to one another, as observed in panel (b) of
Figure 14. In addition, the relatively small number of samples of Chirps in
the dataset used for training might contribute to justify the very low GW
detection rate. Nevertheless, the model has a similar recall value than the
baseline supervised model. However, the value is significantly smaller than
that achieved by the tl_best5 model.

Further studies to improve the performance of both the supervised and the
self-supervised models by using an appropriate training on O3 data will be
conducted in the future. We note, however, that no reliably labelled O3
dataset is yet available in the Gravity Spy project. The labels from the O3
dataset reported in [68] were automatically obtained using a model trained
mostly using O1/O2. Until this dataset becomes available, the next step for
self-supervised models will be to use encoded views similar to the supervised
models. This could allow us to understand if self-supervised methods are worth
pursuing.

(a)

(b)

Figure 13: Results of the predictions of the supervised models for O3 GW
images. Left panel (a): Confusion matrices with a single row each
corresponding to the Chirp class. Right panel (b): examples of the tl_best5
model predictions.

(a)

(b)

Figure 14: Results of the predictions of the self-supervised model for O3 GW
images. Left panel (a): A confusion matrix with only one row corresponding to
the Chirp class. The majority of samples are classified as Blips. Right panel
(b): examples of the model predictions together with its input images.

##  V Conclusion

The classification and mitigation of transient sources of noise, or
“glitches”, in GW data streams is a crucial task in detector characterization
and GW data analysis. In this paper we have investigated the use of
Convolutional Neural Networks to classify glitches included in the Gravity Spy
dataset, corresponding to the O1 and O2 data-taking periods of the Advanced
LIGO and Advanced Virgo detectors. We have used both supervised and self-
supervised deep learning. First, we have trained models using a supervised
learning approach, both trained from scratch and using transfer learning and
hence fine-tuning already pre-trained models in the Gravity Spy dataset. The
comparison with the best baseline model when using transfer learning shows a
moderate increase of the F1 score (our metric of choice to compare different
model configurations) from 98.07% to 98.21%. However, the transfer learning
configuration results in more stable performances across multiple training
trials. In the second part of this work we have assessed the use of self-
supervised training. In this case models have been pre-trained with pseudo-
labels corresponding to image transformations applied to the original images,
and then fine-tuned with the original labels.

The results obtained from our best models are very close to the state-of-the-
art results reported in the literature using the Gravity Spy dataset. Our best
baseline and transfer learning supervised models reach accuracies higher than
the merged view CNNs reported in [66, 39]. Moreover, the baseline model’s
performance is also better than the hard fusion ensemble of CNNs [39] and only
slightly worse than the fine-tuned ResNet50 results from [21], despite the
fact we use a significantly simpler ResNet18 architecture trained from
scratch. On the other hand, the scores of our self-supervised models show
slightly lower values with respect to supervised approaches, yielding an F1
score of 94.15%. This value, however, is close to the best performance
achieved in Ref. [39] using a single view.

In the last part of this study we have tested the models using actual GW
signals from LIGO-Virgo’s O3 run. We have found that despite the models having
been trained using data from previous runs (O1 and O2), they show good
performance, in particular the supervised model with transfer learning. When
using transfer learning the scores improve in a significant way without the
need for any training on actual GW signals (other than the less than 50 chirp
examples from hardware injections present in the Gravity Spy dataset). This
finding motivates the use of transfer learning not only for glitch
classification but also for GW classification, as the model flexibility and
undemanding generalization might help detect signals slightly different from
the ones used during training. Transfer learning may thus provide a better
coverage for glitch classification and GW detection with our ever-changing GW
detection network.

To end, we note that an experiment worth performing elsewhere would be to
fine-tune the transfer-learning model on O3 data. For this, we would need to
classify O3 glitches for each class (including the new ones as e.g. Fast
Scattering [72]) by hand, both for training and for testing. The task of
producing a reliable O3 glitch dataset (like there is for O1 and O2) would
probably take significant effort, but perhaps the outputs of our models, the
Gravity Spy team’s model, and the human labelling from the Gravity Spy citizen
science project can help.

###### Acknowledgements.

We thank Osvaldo Freitas and Solange Nunes for fruitful discussions during the
course of this work. We also thank Christopher Berry, Jools Clarke, Tom
Dooney, Melissa López, Jade Powell, Max Razzano, and Agata Trovato for useful
comments. AO is supported by the FCT project CERN/FIS-PAR/0029/2019. JCB is
supported by a fellowship from “la Caixa” Foundation (ID 100010434) and from
the European Union’s Horizon 2020 research and innovation programme under the
Marie Skłodowska-Curie grant agreement No 847648 (fellowship code
LCF/BQ/PI20/11760016). JCB is also supported by the research grant
PID2020-118635GB-I00 from the Spain-Ministerio de Ciencia e Innovación. ATF
and JAF are supported by the Spanish Agencia Estatal de Investigación (Grant
PID2021-125485NB-C21) funded by MCIN/AEI/10.13039/501100011033 and ERDF A way
of making Europe). Further support is provided by the EU’s Horizon 2020
research and innovation (RISE) programme H2020-MSCA-RISE-2017 (FunFiCO-777740)
and by the European Horizon Europe staff exchange (SE) programme HORIZON-
MSCA-2021-SE-01 (NewFunFiCO-10108625). This material is based upon work
supported by NSF’s LIGO Laboratory which is a major facility fully funded by
the National Science Foundation.

## References

  * Abbott et al. [2016a] B. P. Abbott et al. (LIGO Scientific, Virgo), Phys. Rev. Lett. 116, 061102 (2016a), eprint 1602.03837.
  * Aasi et al. [2015] J. Aasi et al. (LIGO Scientific), Class. Quant. Grav. 32, 074001 (2015), eprint 1411.4547.
  * Acernese et al. [2015] F. Acernese et al. (Virgo), Class. Quant. Grav. 32, 024001 (2015), eprint 1408.3978.
  * Akutsu and et al. [2021] T. Akutsu and et al., Progress of Theoretical and Experimental Physics 2021, 05A101 (2021), eprint 2005.05574.
  * Abbott et al. [2017] B. P. Abbott et al. (LIGO Scientific, Virgo), Phys. Rev. Lett. 119, 161101 (2017), eprint 1710.05832.
  * Abbott et al. [2019] B. P. Abbott et al. (LIGO Scientific, Virgo), Phys. Rev. X 9, 031040 (2019), eprint 1811.12907.
  * Abbott et al. [2021a] R. Abbott et al. (LIGO Scientific, Virgo), Phys. Rev. X 11, 021053 (2021a), eprint 2010.14527.
  * Abbott et al. [2021b] R. Abbott et al. (LIGO Scientific, Virgo) (2021b), eprint 2108.01045.
  * Abbott et al. [2021c] R. Abbott et al. (LIGO Scientific, Virgo, KAGRA) (2021c), eprint 2111.03606.
  * [10] _LIGO/Virgo/KAGRA Public Alerts User Guide: Observing Capabilities_ , https://emfollow.docs.ligo.org/userguide/capabilities.html, Accessed: 20/03/2023.
  * Abbott et al. [2020] B. P. Abbott et al. (LIGO Scientific, Virgo), Class. Quant. Grav. 37, 055002 (2020), eprint 1908.11170.
  * Abbott et al. [2016b] B. P. Abbott et al. (LIGO Scientific, Virgo), Class. Quant. Grav. 33, 134001 (2016b), eprint 1602.03844.
  * Zevin et al. [2017] M. Zevin et al., Classical and Quantum Gravity 34, 064003 (2017), eprint 1611.04596.
  * Cabero et al. [2019] M. Cabero, A. Lundgren, A. H. Nitz, T. Dent, D. Barker, E. Goetz, J. S. Kissel, L. K. Nuttall, P. Schale, R. Schofield, et al., Classical and Quantum Gravity 36, 155010 (2019), eprint 1901.05093.
  * Allen [2005] B. Allen, Phys. Rev. D 71, 062001 (2005), eprint gr-qc/0405045.
  * Biswas et al. [2013] R. Biswas, L. Blackburn, J. Cao, R. Essick, K. A. Hodge, E. Katsavounidis, K. Kim, Y.-M. Kim, E.-O. Le Bigot, C.-H. Lee, et al., Phys. Rev. D 88, 062003 (2013), eprint 1303.6984.
  * Cornish and Littenberg [2015] N. J. Cornish and T. B. Littenberg, Classical and Quantum Gravity 32, 135012 (2015), eprint 1410.3835.
  * Powell et al. [2015] J. Powell, D. Trifirò, E. Cuoco, I. S. Heng, and M. Cavaglià, Classical and Quantum Gravity 32, 215012 (2015), eprint 1505.01299.
  * Mukund et al. [2017] N. Mukund, S. Abraham, S. Kandhasamy, S. Mitra, and N. S. Philip, Phys. Rev. D 95, 104059 (2017), eprint 1609.07259.
  * Powell et al. [2017] J. Powell, A. Torres-Forné, R. Lynch, D. Trifirò, E. Cuoco, M. Cavaglià, I. S. Heng, and J. A. Font, Classical and Quantum Gravity 34, 034002 (2017), eprint 1609.06262.
  * George et al. [2018] D. George, H. Shen, and E. Huerta, Physical Review D 97, 101501 (2018).
  * Llorens-Monteagudo et al. [2019] M. Llorens-Monteagudo, A. Torres-Forné, J. A. Font, and A. Marquina, Class. Quant. Grav. 36, 075005 (2019), eprint 1811.03867.
  * Razzano and Cuoco [2018] M. Razzano and E. Cuoco, Classical and Quantum Gravity 35, 095016 (2018), eprint 1803.09933.
  * Nitz [2018] A. H. Nitz, Classical and Quantum Gravity 35, 035016 (2018), eprint 1709.08974.
  * Davis et al. [2019] D. Davis, T. Massinger, A. Lundgren, J. C. Driggers, A. L. Urban, and L. Nuttall, Classical and Quantum Gravity 36, 055011 (2019), eprint 1809.05348.
  * Llorens-Monteagudo et al. [2019] M. Llorens-Monteagudo, A. Torres-Forné, J. A. Font, and A. Marquina, Classical and Quantum Gravity 36, 075005 (2019), eprint 1811.03867.
  * Torres-Forné et al. [2020] A. Torres-Forné, E. Cuoco, J. A. Font, and A. Marquina, Phys. Rev. D 102, 023011 (2020), eprint 2002.11668.
  * Ormiston et al. [2020] R. Ormiston, T. Nguyen, M. Coughlin, R. X. Adhikari, and E. Katsavounidis, Physical Review Research 2, 033066 (2020), eprint 2005.06534.
  * Colgan et al. [2020] R. E. Colgan, K. R. Corley, Y. Lau, I. Bartos, J. N. Wright, Z. Márka, and S. Márka, Phys. Rev. D 101, 102003 (2020), eprint 1911.11831.
  * Chatziioannou et al. [2021] K. Chatziioannou, N. Cornish, M. Wijngaarden, and T. B. Littenberg, Phys. Rev. D 103, 044013 (2021), eprint 2101.01200.
  * Merritt et al. [2021] J. D. Merritt, B. Farr, R. Hur, B. Edelman, and Z. Doctor, Phys. Rev. D 104, 102004 (2021), eprint 2108.12044.
  * Davis et al. [2022] D. Davis, T. B. Littenberg, I. M. Romero-Shaw, M. Millhouse, J. McIver, F. Di Renzo, and G. Ashton, Classical and Quantum Gravity 39, 245013 (2022), eprint 2207.03429.
  * Ding et al. [2022] J. Ding, R. T. Ng, and J. McIver, Classical and Quantum Gravity 39, 135011 (2022), eprint 2111.09465.
  * Razzano et al. [2023] M. Razzano, F. Di Renzo, F. Fidecaro, G. Hemming, and S. Katsanevas, Nuclear Instruments and Methods in Physics Research A 1048, 167959 (2023), eprint 2301.05112.
  * Usman et al. [2016] S. A. Usman et al., Class. Quant. Grav. 33, 215004 (2016), eprint 1508.02357.
  * Messick et al. [2017] C. Messick et al., Phys. Rev. D 95, 042001 (2017), eprint 1604.04324.
  * Chandra et al. [2022] K. Chandra, J. C. Bustillo, A. Pai, and I. Harry, Physical Review D 106 (2022).
  * Cuoco et al. [2020] E. Cuoco, J. Powell, M. Cavaglià, K. Ackley, M. Bejger, C. Chatterjee, M. Coughlin, S. Coughlin, P. Easter, R. Essick, et al., Machine Learning: Science and Technology 2, arXiv:2005.03745 (2020), eprint 2005.03745.
  * Bahaadini et al. [2018] S. Bahaadini et al., Information Sciences 444, 172 (2018).
  * Ioffe and Szegedy [2015] S. Ioffe and C. Szegedy, 32nd International Conference on Machine Learning 1, 448 (2015), eprint 1502.03167.
  * Szegedy et al. [2016] C. Szegedy, V. Vanhoucke, S. Ioffe, J. Shlens, and Z. Wojna, in _IEEE Conference on Computer Vision and Pattern Recognition_ (IEEE, 2016), vol. 2016-Decem, pp. 2818–2826, eprint 1512.00567.
  * He et al. [2016] K. He et al., Proceedings of the 2016 IEEE Conference on Computer Vision and Pattern Recognition pp. 770–778 (2016), eprint 1512.03385.
  * Simonyan and Zisserman [2014] K. Simonyan and A. Zisserman, 3rd International Conference on Learning Representations pp. 1–14 (2014), eprint 1409.1556.
  * Smith [2018] L. Smith (2018), eprint 1803.09820.
  * Micikevicius et al. [2017] P. Micikevicius et al., 6th International Conference on Learning Representations (2017), eprint 1710.03740.
  * Wightman et al. [2021] R. Wightman, H. Touvron, and H. Jégou (2021), eprint 2110.00476.
  * Loshchilov and Hutter [2019] I. Loshchilov and F. Hutter, 7th International Conference on Learning Representations (2019), eprint 1711.05101.
  * Liu et al. [2022a] Z. Liu et al. (2022a), eprint 2201.03545.
  * Dosovitskiy et al. [2020] A. Dosovitskiy et al. (2020), eprint 2010.11929.
  * Liu et al. [2022b] Z. Liu et al., in _Conference on Computer Vision and Pattern Recognition_ (2022b), pp. 11999–12009, eprint 2111.09883.
  * Tan and Le [2021] M. Tan and Q. Le, in _Proceedings of the 38th International Conference on Machine Learning_ (2021), vol. 139 of _Proceedings of Machine Learning Research_ , pp. 10096–10106, eprint 2104.00298.
  * Lecun et al. [2015] Y. Lecun, Y. Bengio, and G. Hinton, Nature 521, 436 (2015).
  * Mitchell [1997] T. M. Mitchell, _Machine learning_ , vol. 1 (McGraw-Hill, 1997), ISBN 0070428077\.
  * LeCun et al. [1989] Y. LeCun et al., Neural computation 1, 541 (1989).
  * LeCun et al. [1998] Y. LeCun et al., Proceedings of the IEEE 86, 2278 (1998).
  * Deng et al. [2009] J. Deng et al., Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition 9, 248 (2009).
  * Howard and Gugger [2020a] J. Howard and S. Gugger, Information (Switzerland) 11, 1 (2020a), eprint 2002.04688.
  * Cui et al. [2019] Y. Cui et al., Proceedings of the 2019 IEEE Conference on Computer Vision and Pattern Recognition pp. 9260–9269 (2019), eprint 1901.05555.
  * Lin et al. [2020] T. Lin et al., IEEE Transactions on Pattern Analysis and Machine Intelligence 42, 318 (2020), eprint 1708.02002.
  * Zhuang et al. [2021] F. Zhuang et al., Proceedings of the IEEE 109, 43 (2021), eprint 1911.02685.
  * Howard and Gugger [2020b] J. Howard and S. Gugger, _Deep Learning for Coders with Fastai and Pytorch: AI Applications Without a PhD_ (O’Reilly Media, 2020b), ISBN 9781492045526.
  * [62] _fastai: Learner.fine_tune_ , https://docs.fast.ai/callback.schedule.html##learner.fine_tune, Accessed: 06/07/2022.
  * [63] J. Howard, _The best vision models for fine-tuning_ , https://www.kaggle.com/code/jhoward/the-best-vision-models-for-fine-tuning/notebook, Accessed: 17/07/2022.
  * Fernandes [2022] T. Fernandes, Master’s thesis, Universidade de Aveiro (2022), URL https://ria.ua.pt/handle/10773/36277.
  * Jing and Tian [2020] L. Jing and Y. Tian, IEEE Transactions on Pattern Analysis and Machine Intelligence 43, 4037 (2020).
  * Bahaadini et al. [2017] S. Bahaadini et al., Proceedings of the 2017 IEEE Conference on Acoustics, Speech and Signal Processing pp. 2931–2935 (2017), eprint 1705.00034.
  * George et al. [2017] D. George, H. Shen, and E. Huerta (2017), eprint 1706.07446.
  * Glanzer et al. [2023] J. Glanzer, S. Banagiri, S. B. Coughlin, S. Soni, M. Zevin, C. P. L. Berry, O. Patane, S. Bahaadini, N. Rohani, K. Crowston, et al., Classical and Quantum Gravity 40, 065004 (2023), eprint 2208.12849.
  * Abbott et al. [2021d] R. Abbott et al. (LIGO Scientific, Virgo), SoftwareX 13 (2021d), eprint 1912.11716.
  * Macleod et al. [2021] D. Macleod et al., SoftwareX 13, 100657 (2021).
  * Yosinski et al. [2014] J. Yosinski et al., Proceedings of the 27th International Conference on Neural Information Processing Systems 2, 3320 (2014), eprint 1411.1792.
  * Soni et al. [2021] S. Soni, C. P. L. Berry, S. B. Coughlin, M. Harandi, C. B. Jackson, K. Crowston, C. Østerlund, O. Patane, A. K. Katsaggelos, L. Trouille, et al., Classical and Quantum Gravity 38, 195016 (2021), eprint 2103.12104.