# cDVGAN: One Flexible Model for Multi-class Gravitational Wave Signal and
Glitch Generation

Tom Dooney1 R. Lyana Curier1 Daniel Stanley Tan1 Melissa Lopez2,3 Chris Van
Den Broeck2,3 Stefano Bromuri1 1Faculty of Science, Open Universiteit,
Valkenburgerweg 177, 6419 AT Heerlen, The Netherlands  2Institute for
Gravitational and Subatomic Physics (GRASP), Utrecht University,
Princetonplein 1, 3584 CC, Utrecht, The Netherlands  3Nikhef, Science Park
105, 1098 XG, Amsterdam, The Netherlands.

###### Abstract

Simulating realistic time-domain observations of gravitational waves (GWs) and
other events of interest in GW detectors, such as transient noise bursts
called glitches, can help in advancing GW data analysis. Simulated data can be
used in downstream data analysis tasks by augmenting datasets for signal
searches, balancing data sets for machine learning applications, validating
detection schemes and constructing mock data challenges. In this work, we
present Conditional Derivative GAN (cDVGAN), a novel conditional model in the
Generative Adversarial Network framework for simulating multiple classes of
time-domain observations that represent gravitational waves (GWs) and detector
glitches. cDVGAN can also generate generalized hybrid samples that span the
variation between classes through class interpolation in the conditioned class
vector. cDVGAN introduces an additional player into the typical 2-player
adversarial game of GANs, where an auxiliary discriminator analyzes the first-
order derivative time-series. Our results show that this provides synthetic
data that better captures the features of the original data. cDVGAN conditions
on three classes in the time-domain, two denoised from LIGO blip and tomte
glitch events from its 3rd observing run (O3), and the third representing
binary black hole (BBH) mergers. Our proposed cDVGAN outperforms 4 different
baseline GAN models in replicating the features of the three classes.
Specifically, our experiments show that training convolutional neural networks
(CNNs) with our cDVGAN-generated data improves the detection of samples
embedded in detector noise beyond the synthetic data from other state-of-the-
art GAN models. Our best synthetic dataset yields as much as a 4.2% increase
in area-under-the-curve (AUC) performance, maintaining the same CNN
architecture, compared to synthetic datasets from baseline GANs. Moreover,
training the CNN with class-interpolated hybrid samples from our cDVGAN
outperforms CNNs trained only on the standard classes, when identifying real
samples embedded in LIGO detector background between signal-to-noise ratios
ranging from 1 to 16 (4% AUC improvement for cDVGAN). Lastly, we illustrate an
application of cDVGAN in a data augmentation example, showing that it is
competitive with traditional augmentation approaches.

††preprint: APS/123-QED

##  I Introduction

The first detection of a Gravitational Wave (GW) from a binary black hole
(BBH) merger in 2015 ushered in a new era of astronomy and cosmology [1].
Since then, over three observing runs (O1, O2, O3), advanced LIGO [2] and
Virgo [3] detectors have made confident detections of 90 compact binary
coalescence (CBC) events, as reported in the Gravitational Wave Transient
Catalogues GWTC-1, GWTC-2, GWTC-3 [4, 5, 6]. With the introduction of KAGRA
[7], Japan’s underground detector, towards the end of O3 and the O4 run
currently underway, hundreds of more detections are expected from the enhanced
sensitivity of GW detectors [8].

Ongoing upgrades to advanced detector systems will give rise to new challenges
in Gravitational Wave (GW) data analysis, particularly with the introduction
of next-generation GW detectors, such as the Einstein Telescope (ET) [9] and
Cosmic Explorer [10]. GW detection rates from all sources are expected to
significantly increase i.e. it is estimated the ET will detect on the order of
$8\times 10^{4}\,\text{y}^{-1}$ BBH mergers [11] and $7\times
10^{4}\,\text{y}^{-1}$ Binary Neutron Star (BNS) mergers [12]. This could lead
to over 400 compact binary coalescence (CBC) events daily. The enhanced
detectors are also expected to detect new astrophysical sources of GWs [13,
14].

Aside from genuine astrophysical signals, the heightened sensitivity of GW
detectors is expected to exacerbate issues relating to ‘glitches’ [15, 16, 17,
8]. Glitches are non-Gaussian transient noise artefacts that can resemble
astrophysical signals, hindering GW data analysis and increasing false
positives [18, 19, 20, 21, 22, 23, 24, 25, 26, 27]. Glitches are unmodelled
noise events, stemming from environmental or instrumental factors, with some
sources remaining unidentified [28]. Unlike modelled CBC events, which are
detected through matched-filtering [29, 30, 31], glitches are detected by
model-free detection algorithms that scrutinize excess power in the time-
frequency (spectrogram) representation to distinguish them from the detector
background [32].

To meet the challenges of advanced detector systems, machine learning
algorithms have become increasingly popular in the GW physics community [33,
34, 35, 36]. In the case of glitches, studies have largely focused on
spectrogram representations for identification due to their unmodelled nature.
For example, Gravity Spy [37, 38] have made significant strides in
characterising spectrogram representations of glitches by combining machine
learning and citizen science. Multiple studies have employed machine learning
extensively to improve classification accuracy on Gravity Spy spectrograms
[39, 40, 41]. Others have leveraged the Generative Advsersarial Network (GAN)
[42] framework for generating Gravity Spy spectrograms to augment glitch
datasets for further improvement [43, 44]. However, relying on computationally
expensive spectrogram transformations to identify and simulate unmodelled
events like glitches may not always be feasible as GW detector technology
improves.

In this study, we develop a generative modelling framework for diverse classes
of time-domain observations in GW detectors. Generating time-domain
representations of GWs and glitches offers various advantages, such as their
flexibility for experimental purposes and low dimensionality requiring less
computational expense. Simulated data can be used in downstream applications
such as data augmentation and class balancing for machine learning
applications, validating detection schemes via software injections [45, 46,
47] and constructing mock data challenges. Furthermore, where unmodelled
transients like glitches can be isolated from the background, transforming
time-series to spectrograms is straightforward, while the reverse is
challenging due to background noise captured in spectrograms.

An approach to simulating time-domain glitch events is implemented using the
gengli glitch generator [48]. The authors implement a Wasserstein GAN (WGAN)
[49] on blip glitches extracted from detector backgrounds using BayesWave
[50]. They show that it is possible to isolate blips from their surroundings
and learn their underlying distribution in the time-domain with WGANs. Another
study [51] implements a conditional GAN (cGAN) called McGANn to simulate 5
waveform classes analogous to GW bursts. Aside from generating five distinct
classes, their GAN can generate class-interpolated, hybrid samples.

In this work, we propose a novel conditional Derivative GAN (cDVGAN) that
simulates different classes of time-domain observations of LIGO glitch classes
and/or astrophysical waveforms. Aside from learning realistic distributions of
diverse classes in one flexible model, cDVGAN can generate hybrid samples that
traverse the variation between the learned classes by manipulating the user-
controlled class vector. After the training phase, the models can generate
hundreds of thousands of in-class or hybrid samples in a matter of seconds,
per the user’s preference. Among other applications, cDVGAN is intended for
use in a next-generation glitch generator, expanding the scope beyond that of
gengli [48].

To show the utility of our generated (synthetic) data, we implement an
experiment that uses it for training a convolutional neural network (CNN)
detection algorithm to identify real data from the GAN training data
distribution in additive LIGO detector noise. Our results indicate that cDVGAN
can better capture the features of the data than other state-of-the-art GAN
models by incorporating adversarial feedback on the first-order derivatives
using an auxiliary discriminator. Furthermore, we show that GAN-generated
hybrid samples can be useful for training detection algorithms to identify
real data in noise beyond the standard GW and glitch classes.

This paper is structured as follows; in section II, we discuss concepts
relating to GANs. In section III, we present the cDVGAN architecture and
training schemes, the datasets and preprocessing, and experimental details. In
section IV, we present the experimental results involving a simple search for
real GWs and glitches in additive detector background using synthetic data
from 5 different GAN models. Finally, section V discusses the conclusions of
this research.

##  II Generative Adversarial Networks

Figure 1: Diagrams of a typical cGAN architecture (left), comprising one
discriminator, and cDVGAN (right), comprising two discriminators. Class
vectors $c$ (real) and $\hat{c}$ (fake), are fed to all model components in
both cases. An intermediate derivative calculation is observed in the cDVGAN
plot, where the derivative of the synthetic sample is calculated. cDVGAN2
includes yet another discriminator applied to second-order derivatives. In
cDVGAN and cDVGAN2, the total generator loss is calculated as a linear
combination of the discriminator losses applied to synthetic samples.

###  II.1 Wasserstein GANs

GANs are a class of machine learning algorithms that generate realistic
synthetic data. They consist of two neural networks: a discriminator (also
known as a critic) that distinguishes between real and synthetic data; and a
generator that generates synthetic data that can fool the discriminator. GAN
approaches suffer from stability issues such as vanishing gradients
stimulating numerous studies centred around methods to stabilize the training
process. Wasserstein GAN [52] is a particular variant that addresses these
issues. It uses the Wasserstein-1 distance ($W1$) as the loss function to
measure the similarity between the real and synthetic distributions. $W1$ is
fully differentiable and increases monotonically while never saturating,
removing the issue of vanishing gradients. Under this paradigm, the
optimization problem can be formulated as

$\theta_{opt}=arg\min_{\theta}\max_{\phi:||D(x,\phi)||_{l}\leq
1}L(\phi,\theta)$ (1)

where the maximum is taken over all 1-Lipschitz functions $D$ and with $L$
defined as

$L(\phi,\theta)=\mathbb{E}_{x\sim P_{x}}[D(x,\phi)]-\mathbb{E}_{\hat{x}\sim
P_{\hat{x}}}[D(\hat{x},\phi)]$ (2)

where $\hat{x}=G(z,\theta)$ and $z$ is a batch of the generator’s latent
vector. $D$ and $G$ refer to the discriminator and the generator with
parameters $\phi$ and $\theta$, respectively. $\mathbb{E}_{x\sim P_{x}}$
averages over a batch of real samples $x$ from the real distribution $P_{x}$,
while $\mathbb{E}_{\hat{x}\sim P_{\hat{x}}}$ averages over a batch of
generated samples $\hat{x}$ from the synthetic distribution $P_{\hat{x}}$.
Equation 1 requires a constraint of 1-Lipschitz continuity on $D$ [52]. This
can be accomplished by adding a regularization penalty called the gradient
penalty ($GP$) to the discriminator loss [53, 54, 55]. The discriminator loss
then becomes

$L_{D}=-L(\phi,\theta)+\lambda GP(\phi)$ (3)

with

$GP(\phi)=\mathbb{E}_{\hat{x}\sim
P_{\hat{x}}}\left[(||\nabla_{x}D(\hat{x},\phi)||_{2}-1)^{2}\right]$ (4)

and where $\lambda$ represents the regularization hyperparameter, $||.||_{2}$
represents the $L^{2}$-norm and $\hat{x}$ is a randomly sampled point between
the real and synthetic data. When updating the generator, errors are
propagated through the entire network, from $D$ to $G$. Naturally, for the
generator updates are made only on generated samples from $G$. The generator
loss is written as

$L_{G}(\phi,\theta)=-\mathbb{E}_{\hat{x}\sim P_{\hat{x}}}[D(\hat{x},\phi)]$
(5)

Figure 2: A comparison of the discriminators from the original cGAN paper
(used for McGANn and McDVGANn) and the projection discriminator (used for
cWGAN, cDVGAN, cDVGAN2).

###  II.2 Conditional GANs

Conditional GANs (cGANs) [56] allow finer control over the generated data by
providing extra information to both the generator and discriminator. For
example, we can specify the class of the generated data by providing class
label, $c$. The training data and class labels are taken from a joint
distribution $P_{data}(x,c)$. When generating synthetic samples, the class
vector $\hat{c}$ and the GAN latent vector distribution $P_{z}$, are sampled
independently.

In the original cGAN paper [56], conditional information was provided to the
network by naively concatenating the class information to the first layers of
the GAN components. In the case of time-series, this corresponds to
concatenating it to the $z$ vector before passing it to the generator, and the
input sample before passing it to the discriminator.

Miyato and Koyama [57] developed a more effective conditioning method using a
projection in the discriminator between the conditional features and the
features extracted from the input before the discriminator output. The
projection output measures the similarity between the condition and the
discriminator’s feature vector and is added to the discriminator output. The
output of the discriminator is modified from

$D(x,c)=g(h(x,c))$ (6)

to

$D(x,c)=c^{T}Vh(x)+g(h(x))$ (7)

where $V$ is an embedding matrix, $h$ is the discriminator’s feature vector
and $g$ is the discriminator output layer (see Figure 2). This approach has
been shown to improve the quality of class conditional generation using GANs.
Note that the generator is still conditioned by concatenating the class-
embedded vector to the latent input, similar to the original cGAN paper.

##  III Methods

###  III.1 Conditional Derivative GAN (cDVGAN)

The adversarial training process for GANs is known to be volatile. Models can
suffer mode collapse, where only a few realistic samples are learned, while
they can also fail to converge at all. These issues often occur when one model
component (generator/discriminator) begins to dominate the other during the
training phase. These problems are exacerbated when conditioning on multi-
modal distributions of GW and glitch time-series. In this section, we
introduce two new cGAN designs, cDVGAN and cDVGAN2, that can help overcome
these limitations.

In cDVGAN, two discriminators are applied to two different representations of
the data instead of the usual single discriminator. The first discriminator is
applied to the original samples as in a conventional GAN, while the second
discriminator is applied to the corresponding derivative samples. A high-level
diagram of cDVGAN can be seen in Figure 1. We have shown in our previous work
that the derivative discriminator leads to increased training stability of the
model components and minimizes high-frequency artefacts in the GAN output,
generating smoother and more faithful data [58]. Under this scheme, the
generator loss is calculated as a linear combination of the two discriminator
outputs. Equation 5 can be rewritten as

$L_{G}(\phi_{1},\phi_{2},\theta)=-\eta_{1}\mathbb{E}_{\hat{x}_{1}\sim
P_{\hat{x}_{1}}}[D_{1}(\hat{x}_{1},\phi_{1},\hat{c})]\\\
-\eta_{2}\mathbb{E}_{\hat{x}_{2}\sim
P_{\hat{x}_{2}}}[D_{2}(\hat{x}_{2},\phi_{2},\hat{c})]$ (8)

where $D_{1}$ and $D_{2}$, represent the first and second discriminator
respectively. Here $\hat{x}_{1}$ represents synthetic samples while
$\hat{x}_{2}=d\hat{x}_{1}/dt$ represents the corresponding derivatives and
$\hat{c}$ represents the class vector for the synthetic samples.
$P_{\hat{x}_{1}}$ and $P_{\hat{x}_{2}}$ are the distributions of the two
representations of generated data, and $\eta_{1}$ and $\eta_{2}$ are
hyperparameters that control the relative strength of the discriminator
losses. The cDVGAN method is not restricted to two discriminators. We extend
cDVGAN to cDVGAN2, which includes a third discriminator that is applied to
second-order derivatives. The models are identical except for the addition of
the second-order derivative discriminator in cDVGAN2, while the second-order
derivative discriminator is identical to the first-order discriminator except
for the input size. Additional representations of the data (eg. time-frequency
representations) can also be provided to additional discriminators, depending
on the problem. For k discriminators applied to k representations of the data,
the generator loss is written generally as

$L_{G}(\phi_{1},...,\phi_{k},\theta)=-\sum_{i=1}^{k}\eta_{i}\mathbb{E}_{\hat{x}_{i}\sim
P_{\hat{x}_{i}}}[D_{i}(\hat{x}_{i},\phi_{i},\hat{c})]$ (9)

Our cDVGAN and cDVGAN2 models are conditioned via projection, as described in
section II.2. Since hyperparameter optimization is not the focus of this
research, the hyperparameters $\eta_{1}=\eta_{2}=0.5$ in cDVGAN and
$\eta_{1}=\eta_{2}=\eta_{3}=0.33$ in cDVGAN2, meaning all discriminators
contribute equally to the respective generator losses. The discriminators are
updated 5 times for each generator update. The full model architecture can be
viewed in the Appendix (VII.1)111Python code can be found at
https://git.ligo.org/tom.dooney/cdvgan_paper..

###  III.2 Training Data and Preprocessing

Figure 3: Examples of blip (top), tomte (middle) and BBH signals (bottom) used
to train GAN models

The GAN models in this study are conditioned on three different classes; two
classes that are derived from Gravity Spy glitch classes called blip and
tomte, and a third BBH signal class. The astrophysical nature that the BBH
signals represent is not considered important during experimentation, and they
are treated as another class of transient, glitch-like time-series for
experimental purposes. Examples of the three classes can be observed in Figure
3. Blip glitches have a characteristic time-frequency morphology of a
symmetric teardrop shape in the range $[30,500]\,$Hz with short-durations
($\sim 0.04\,$s). They appear in LIGO Livingston and LIGO Hanford, Virgo and
GEO $600$ [59]. Due to their abundance and form, they hinder both the
unmodeled burst and modelled CBC searches, with particular emphasis on compact
binaries with large total mass, highly asymmetric component masses, and spins
anti-aligned with the orbital angular momentum [18, 27]. Tomte glitches are
also short-duration ($\sim 0.25\,$s) with characteristic triangular
morphology. Since both blip and tomte glitches have no clear correlation to
the auxiliary channels, they cannot be removed from astrophysical searches.
The BBH class represents the inspiral and merger of a binary black hole
system. All samples are of length $1,024$ and have a sampling rate of
$4,096\,$Hz, corresponding to $0.25\,$s of data. The GANs are trained on
$7,500$ samples ($2,500$ samples from each class).

The blip and tomte datasets are constructed using confidences from Gravity Spy
applied to the glitch triggers in LIGO’s third observing run (O3) [60]. Only
blip and tomte events with Gravity Spy confidences of $c^{1}_{GS}\geq 0.9$ for
their respective class are used in this study, and are extracted using GWpy’s
[61] fetch_open_data method, which provides an interface to the GWOSC222The
Gravitational Wave Open Science Centre [62] data archive. The glitches are
surrounded by stationary and uncorrelated noise, which would hinder the
learning of GAN models.

To avoid the computational expense of BayesWave, as done in [48], the glitches
are isolated from the background using Savitsky-Golay [63] filters. This
requires the following preprocessing steps:

  1. 1.

Firstly, $20\,$s of strain data is extracted, centred around the glitch GPS
time provided by Gravity Spy.

  2. 2.

The data is whitened, and a bandpass filter between $(20,350)\,$Hz is applied
to the $20\,$s of data.

  3. 3.

The data is cropped at 8,192 datapoints, centred around the GPS time of the
glitch, corresponding to $2\,$s of data.

  4. 4.

A window of 100 data points is isolated around the centre of the glitch, and
two consecutive Savitsky-Golay filters with a polynomial of order 3 are
applied to each side around the glitch centre iteratively, with window sizes
of 501 and 301 respectively.

  5. 5.

The two smoothed sides and the unsmoothed central peak are then concatenated,
followed by applying 3 additional Savitsky-Golay filters iteratively with
window sizes 41, 31 and 21 to the entire concatenated sample. The entire
sample is finally cropped at 1,024 data points around the event GPS time,
centred around 0 and rescaled to $(-1,1)$.

(a) Whitened and bandpassed sample

(b) Filtering either side of the peak

(c) A fully preprocessed blip (before rescaling)

Figure 4: Visualizations of the preprocessing steps applied to a blip glitch
event.

The results of the above preprocessing steps are shown in Figure 4. The
extensive use of Savitsky-Golay filters ensures that high-frequency noise
artefacts are removed while preserving the overall shape. An analysis was made
to investigate whether the characteristics of blips and tomtes are preserved
after filtering (Appendix VII.5). Gravity Spy generally classifies the
preprocessed samples as their correct class when embedded in detector
background for the signal-to-noise ratio (SNR) ranges considered in
experiments (see section III.3), indicating that their morphologies are
analogous to glitches.

Figure 5: A plot of the first 3 principal components of the original samples.
The separability of the three classes in this compressed representation
indicates the diversity of the data.

All BBH signals are simulated with PyCBC [64] using the IMRPhenomD waveform
routine from LALSuite [65], which generates the inspiral, merger and ringdown
of a BBH waveform. The component masses are restricted to the range of
$[30,160]M_{\odot}$ with a spin of zero and fixing $m_{1}>m_{2}$ and using
only plus polarization.

We show the diversity of the three classes in Figure 5, where the dataset is
represented by three separable clusters in a reduced principal component
space.

###  III.3 Experimental Procedure

####  III.3.1 GAN Benchmarks

This work uses ablation studies to compare cDVGAN and its cDVGAN2 extension
with three other baseline GAN models in their ability to generate data useful
for training detection algorithms (see section III.3.3). An ablation study in
machine learning is a systematic experimentation technique used to understand
the contribution of individual components of a model to its overall
performance. In this case, it involves selectively disabling auxiliary first
and second-order derivative discriminators during the training of GAN models
to investigate if they are successful in improving the features captured in
synthetic GW signals and glitches. To construct an appropriate ablation study,
the first benchmark is a conditional variant of a vanilla Wasserstein GAN
(cWGAN), which comprises the same architecture as cDVGAN except for the
derivative discriminator. This allows us to investigate the effect of
including a first-order derivative discriminator during GAN training. cDVGAN2
allows us to investigate the effect of including a second-order derivative
discriminator on top of the first-order derivative discriminator.

The second benchmark is McGANn [51] since it successfully replicates the
features of simulated GW bursts. As a third benchmark, we developed a modified
McGANn model, called McDVGANn, that uses a second auxiliary discriminator
applied to first-order derivatives, similar to cDVGAN. The architecture is
identical to McGANn except for the addition of a derivative discriminator. In
McDVGANn, the derivative discriminator is identical to the base discriminator
except for the input size. This benchmark investigates whether the idea of
derivative discriminators can generalize to other GAN models. For a more
detailed description of the baselines (see Appendix VII.6).

McGANn and McDVGANn are conditioned using concatenation similar to their paper
[51] and are trained with an Adam optimizer, binary cross-entropy loss
function and learning rate $2\times 10^{-4}$. Conversely, cWGAN and cDVGAN2
are conditioned via projection similarly to cDVGAN and are trained with
RMSProp, a Wasserstein loss function and a learning rate of $2\times 10^{-4}$.
All models are trained for 500 epochs with a batch size of 512. Unless
otherwise specified, all GANs are trained using the same standard
hyperparameters as prior works.

####  III.3.2 GAN-generated datasets

We construct 3 different datasets from each GAN by sampling the respective
generator’s class space, as suggested by [51]. The three variants of GAN-
generated datasets are as follows:

  1. 1.

Vertex: The vertex class space corresponds to the vertices of the three-
dimensional class space. The locations of the vertex class space are the same
as the training set class space locations used to train the GAN models and are
the closest representation to the training set. The vertex class vector is
one-hot encoded corresponding to one of the three GAN training classes eg. [1,
0, 0] corresponds to the blip class.

  2. 2.

Simplex: The simplex class space corresponds to points on a $k=2$ simplex (2D
triangle) in the case of three classes. Class vectors are constructed by
sampling points uniformly on this simplex. The simplex can be considered the
simplest surface that intersects all three training classes, and all simplex
class vectors sum to 1. Variations are observed in the samples, with some
having characteristics that strongly resemble the training classes, due to one
class dominating the others. The simplex dataset is a superset of the vertex
dataset and represents synthetic data outside the training data distribution.

  3. 3.

Uniform: For the uniform dataset, each entry in the class vector is uniformly
sampled from U$[0,1]$, corresponding to sampling uniformly within a cube with
dimensions 1x1x1. The uniform dataset is a superset of the simplex and vertex
datasets and, like the simplex dataset, represents data from outside of the
training data distribution. The uniform dataset exhibits the largest variety
since it explores regions of the class space further than the simplex dataset
relative to the training set vertices.

Figure 6 shows examples from each of the cDVGAN-generated datasets (also in
Appendix VII.3). Figure 7 shows plots of the first 3 principal components
(PCs) of real and GAN-generated samples from cDVGAN. The vertex samples
(Figure 7(a)) generally match their corresponding classes from the real data
distribution. Figure 7(b) shows that the simplex and uniform hybrid datasets
populate spaces between the class clusters, while Figure 7(c) shows that the
uniform dataset covers a larger part of the PC space than the simplex dataset.
This is intuitive since the simplex space is a subset of the uniform space.

Figure 6: Examples of vertex (top), simplex (middle) and uniform samples
(bottom) from cDVGAN. The corresponding class vector is shown above each
sample.

(a) Real and Vertex datasets

(b) All datasets

(c) Hybrid datasets

Figure 7: The first 3 principal components (PCs) of real and GAN-generated
samples from cDVGAN. The vertex samples from cDVGAN generally match the real
samples in the PC space while hybrid samples populate intermediate regions
between the clusters for the 3 classes. Figure 7(c) shows that the uniform
dataset covers a larger space than the simplex dataset.

####  III.3.3 Downstream search with CNNs

The experiments followed in this study investigate the effectiveness of GAN-
generated data for training CNNs to detect real data in additive Gaussian
noise. CNNs are a class of deep learning model commonly used in computer
vision. However, they can also be applied to other spatially adjacent data
types like time-series [66] and have been applied to GW detector strain data
to detect merging black holes [34].

The CNNs’ objective is to perform the binary classification of two classes:
samples in additive detector noise and detector noise only, examples of which
are shown in Figure 8. It takes a time-series input of dimension $1,024$,
representing $0.25\,$s of LIGO strain data. The CNN architecture is kept
constant for training with each dataset and can be viewed in the Appendix
(VII.2).

Better synthetic data will result in CNNs with better detection efficiency on
a real data subset from the GAN training data distribution. We also
investigate if training CNNs with GAN-generated hybrid datasets (simplex and
uniform) can improve the detection efficiency beyond training solely with the
standard three classes.

The training and testing samples are injected additively into detector noise
from Hanford (H1) or Livingston detectors (L1) during O3 for each of the above
datasets. The detector noise is extracted from GWOSC, using trigger times for
Gravity Spy’s No_glitch class, extracing $14\,$s around each No_glitch GPS
time. Although it is not investigated whether other glitches are present in
the noise, using Gravity Spy’s No_glitch class should guarantee that most of
the samples contain stationary noise after whitening.

The background is sampled at $4,096\,$Hz, similar to the GAN training data.
The LIGO detector noise is first whitened using PyCBC’s whitening function,
with the first and last $2.5\,$s removed due to artefacts at the noise
boundaries. The preprocessed detector noise is then split into chunks of
length $1,024$, the same dimensionality as the GAN input and output. Following
this procedure, we accumulate just over $250,000$ background samples in total.
Since the GAN training data and output are scaled between $[-1,1]$ the samples
are scaled to a signal-to-noise ratio (SNR) ratio that is sampled uniformly on
U$[1,16]$ before injecting them into the preprocessed detector noise. This is
done by first computing $\rho_{opt}$ for each generated sample according to

$\rho_{opt}^{2}=4\int^{f_{max}}_{f_{min}}\frac{|\hat{h}(f)|^{2}}{S_{n}(f)}df$
(10)

where $\hat{h}(f)$ and $S_{n}(f)$ are the Fourier transform of the input
sample (blip, tomte, BBH) and the detector noise PSD respectively [67] (which
we set to unity for convenience as we are working with whitened data). The
sample can then be scaled to the desired $\rho_{opt}$ on U$[1,16]$.

(a) Pure LIGO detector noise (0 class)

(b) Detector noise + glitch (1 class)

Figure 8: Examples of the two classes predicted by CNN models. The injected
sample on the right is scaled to an SNR of 8 before injection (shown in orange
for clarity).

For each GAN, we train three separate CNNs on the three generated datasets
(vertex, simplex, uniform). For the vertex dataset the three different vertex
locations in the class space are sampled with equal probability. For the
uniform and simplex datasets, samples are drawn uniformly from their
respective spaces.

(a) cDVGAN

(b) cDVGAN2

(c) cWGAN

(d) McGANn

(e) McDVGANn

Figure 9: ROC-Curves for the different GAN-generated datasets from each GAN. Dataset | cDVGAN (ours) | cDVGAN2 (ours) | cWGAN | McGANn | McDVGANn (ours)
---|---|---|---|---|---
Vertex-Trained | 0.771 $\pm$ 0.012 | 0.758 $\pm$ 0.008 | 0.762 $\pm$ 0.018 | 0.768 $\pm$ 0.016 | 0.768 $\pm$ 0.022
Simplex-Trained |  0.802 $\pm$ 0.019 | 0.789 $\pm$ 0.010 |  0.788 $\pm$ 0.009 | 0.759 $\pm$ 0.010 |  0.786 $\pm$ 0.012
Uniform-Trained | 0.797 $\pm$ 0.022 |  0.791 $\pm$ 0.009 | 0.777 $\pm$ 0.014 |  0.770 $\pm$ 0.014 | 0.778 $\pm$ 0.012
Table 1: The Area-Under-Curve (AUC) yielded on a real test set by CNNs trained
on each synthetic dataset from each GAN. The results represent the mean AUC
over 5 iterations, where the bounds are calculated using the standard
deviations over the 5 iterations. The best result overall is shown in bold
text, while the best result per GAN is shown in italic text.

Each training dataset comprises $100,000$ samples, with 50% glitch/signal plus
Gaussian noise and 50% Gaussian noise only. We test each CNN on a real data
subset comprising $7,500$ samples ($2,500$ from each class), sampled randomly
from a distribution of $20,850$ samples ($6,950$ from each class). This
results in a test dataset of 15,000 samples in total (50% LIGO noise only) in
each iteration of training/testing. We ensure that none of the GAN training
data appear in the test set, although they are taken from the same
distribution.

The results are presented using the area-under-the-curve (AUC) metric. AUC is
a metric commonly used in machine learning to evaluate the performance of a
classification model, particularly in the context of binary classification
problems. It is associated with the Receiver Operating Characteristic (ROC)
curve (see Figure 9), which is a graphical representation of the trade-off
between true positive rate (sensitivity) and false positive rate
(1-specificity) across different thresholds. It provides a single metric
between 0 and 1 that summarizes the overall discriminatory power of a model
across different classification thresholds. We repeat the experiment 5 times
and report the mean results, randomly generating the training data and
randomly sampling the backgrounds and test data in each iteration. We keep the
real test datasets constant for each GAN-generated dataset in each iteration.

##  IV Results

###  IV.1 Training with GAN data

Dataset |  cDVGAN (ours) | cDVGAN2 (ours) | cWGAN | McGANn | McDVGANn (ours)
---|---|---|---|---|---
Vertex-Trained | 0.689 $\pm$ 0.009 | 0.680 $\pm$ 0.006 | 0.685 $\pm$ 0.017 |  0.687 $\pm$ 0.010 | 0.668 $\pm$ 0.020
Simplex-Trained | 0.698 $\pm$ 0.013 | 0.686 $\pm$ 0.007 |  0.695 $\pm$ 0.010 | 0.673 $\pm$ 0.008 |  0.676 $\pm$ 0.010
Uniform-Trained |  0.702 $\pm$ 0.014 |  0.693 $\pm$ 0.008 | 0.692 $\pm$ 0.009 | 0.680 $\pm$ 0.013 | 0.671 $\pm$ 0.006
Table 2: The AUC for test samples under an SNR of 8. The results represent the mean AUC over 5 iterations, where the bounds are calculated using the standard deviations over the 5 iterations. The best result overall is shown in bold text, while the best result per GAN is shown in italic text. Dataset |  cDVGAN (ours) | cDVGAN2 (ours) | cWGAN | McGANn | McDVGANn (ours)
---|---|---|---|---|---
Vertex-Trained | 0.844 $\pm$ 0.015 | 0.827 $\pm$ 0.010 | 0.830 $\pm$ 0.022 | 0.841 $\pm$ 0.021 | 0.856 $\pm$ 0.027
Simplex-Trained |  0.893 $\pm$ 0.029 |  0.881 $\pm$ 0.013 |  0.867 $\pm$ 0.010 | 0.836 $\pm$ 0.012 |  0.885 $\pm$ 0.013
Uniform-Trained | 0.883 $\pm$ 0.031 | 0.878 $\pm$ 0.011 | 0.851 $\pm$ 0.020 |  0.852 $\pm$ 0.020 | 0.873 $\pm$ 0.014
Table 3: The AUC for samples above an SNR of 8. The results represent the mean
AUC over 5 iterations, where the bounds are calculated using the standard
deviations over the 5 iterations. The best result overall is shown in bold
text, while the best result per GAN is shown in italic text.

The AUC values for each CNN model over the entire SNR range (1-16) are shown
in Table 1. We also scale the test sets between SNRs of 1-8 and 8-16 and
record the AUC of the same CNNs to investigate performance for quieter and
louder samples, which can be seen in Tables 2 and 3 respectively. Examples of
ROC curves from one of the testing iterations are shown in Figure 9. All
tables show that the simplex and uniform datasets from cDVGAN yield the
highest AUC results for CNNs, with the simplex dataset yielding the highest
overall AUC.

Ablation Studies. The ablation between cDVGAN and cWGAN in Table 1 shows that
the adversarial feedback from the first-order derivative discriminator
improves the features of the synthetic data in all three datasets. This
suggests that the first-order derivative discriminator was successful in
improving the GAN output. cDVGAN2 was unsuccessful in improving upon cDVGAN’s
results, yielding a slightly lower performance, but improves upon the
performance of cWGAN’s simplex and uniform datasets, particularly for the
higher SNR range (Table 3).

The AUC performance yielded from McDVGANn’s datasets is competitive among the
GAN models. Both its simplex and uniform datasets yield better AUC performance
than McGANn’s counterparts, with the simplex dataset yielding the highest
overall AUC performance between the two models. The vertex datasets from
McGANn and McDVGANn yield comparable performance. This suggests that the
derivative discriminator can be effective in a more traditional GAN
architecture.

These results show that incorporating derivative discriminators can improve
the synthetic data in multiple GAN architectures, and indicate that analyzing
first-order derivatives in a separate auxiliary discriminator is superior to
using both first and second-order derivative discriminators for modelling the
dataset covered in this study (in the cWGAN architecture). For all GAN models,
hybrid datasets provide the best overall AUC performance for the CNN. This
suggests that GAN-generated hybrid samples are useful for searching for
multiple classes of real data when obscured by the detector background. This
might offer interesting applications in glitch searches, particularly for
those with no clear correlation to auxiliary channels. Such glitches could be
conditioned into cDVGAN to generate hybrid samples specific to a subset of
LIGO glitch classes for use in a glitch detection algorithm. Combining all
three GAN-generated datasets for training CNNs may improve upon these results
yet again, although this is left to future work.

SNR | 100:0 | 75:25 | 50:50 | 25:75 | 0:100
---|---|---|---|---|---
1-16 | 0.900 $\pm$ 0.001 | 0.898 $\pm$ 0.002 | 0.893 $\pm$ 0.002 | 0.887 $\pm$ 0.002 | 0.802 $\pm$ 0.019
1-8 | 0.799 $\pm$ 0.001 | 0.792 $\pm$ 0.005 | 0.787 $\pm$ 0.003 | 0.777 $\pm$ 0.003 | 0.698 $\pm$ 0.013
8-16 | 0.988 $\pm$ 0.001 | 0.987 $\pm$ 0.001 | 0.987 $\pm$ 0.001 | 0.985 $\pm$ 0.001 | 0.893 $\pm$ 0.029
Table 4: AUC values over three SNR ranges for different proportions of
real:synthetic samples for a training set fixed at 100,000 samples. The
results are represented by the mean AUC and standard deviation over 5
iterations.

###  IV.2 Combining real and cDVGAN data for improved training

In this section, we augment real datasets with GAN-generated data to improve
upon the classification performance in the previous section. We augment the
real data with the simplex dataset from cDVGAN since it yields the best
performance. We compare cDVGAN data with traditional duplication of real
training samples for data augmentation.

Maintaining a real hold-out test set of 7,500 samples (2,500 from each class)
as in section IV.1, we use all remaining samples from the real distribution
for training, which amounts to 13,350 samples (4,450 from each class). Fixing
the training data size again at 100,000 (50,000 glitch/signal + noise samples,
50,000 noise-only samples), we vary the proportion of real and GAN-generated
samples. Since there is no limit to generating cDVGAN data, we duplicate the
real training data before injection to reach the required number of samples.
This is done to control the effects of the background noise on the training of
CNNs, since CNNs are sensitive to the number of different backgrounds seen
during training.

The results in Table 4 show that the performance drops only slightly with
smaller proportions of real data. The CNN performance remains competitive even
with only 25% of real training samples with only a 1% drop in overall AUC
performance compared to only using real data. The second and third rows of
Table 4 show that this decrease in AUC performance occurs mostly for lower SNR
($<$8) samples. The decrease in performance for louder samples is minimal
($<$1% decrease). This indicates that the synthetic data is competitive for
augmenting the training data for CNNs when including a relatively small amount
of real data in the training set. Although there is a more substantial
decrease in performance when using 100% synthetic samples, this might be
improved by including other synthetic datasets from cDVGAN in the training
schedule.

##  V Conclusion

###  V.1 Conclusions

Time-domain generative modelling in the GAN framework has shown potential to
improve GW data analysis. Using GANs to learn distributions of GWs and other
events of interest such as detector glitches can be useful for data
augmentation tasks, validating detection schemes for unmodelled waveforms such
as [32, 68, 69], or be used to construct mock data challenges. This work
presents a novel conditional GAN, called cDVGAN, for generating distinct time-
domain classes, including two classes of unmodelled glitches and one class of
modelled BBH signals. cDVGAN uses additional adversarial feedback on the
first-order derivatives of training samples in an auxiliary discriminator and
generates realistic samples that span the variation within each class. It also
allows for the explicit control of the mixing of classes. Thus, it is capable
of generating generalized hybrid samples that are outside of the limited
training distribution and span the variation between classes by sampling the
continuous class space.

We use ablation studies to show the effectiveness of using auxiliary
discriminators to analyze sample derivatives in an experiment that uses GAN-
generated data to train convolutional neural networks (CNNs) to detect real
samples in LIGO detector noise. An ablation study between cDVGAN and its
vanilla cWGAN counterpart shows that the additional adversarial feedback from
the first-order derivative discriminator yields generated data that is more
useful for training a CNN detection algorithm.

The ablation study between cDVGAN2 and cDVGAN reveals that second-order
derivative discrimination did not improve the performance under this problem
scheme, although the performance of cDVGAN2 is competitive with other baseline
GANs. Another ablation study between our McDVGANn model and McGANn [51]
indicates that the method can be effective under a traditional cGAN
architecture. These results suggest that providing adversarial feedback on
derivatives on top of the original samples can improve the learning of GANs on
continuous time-series, and in particular, events of interest to the GW
physics community.

Furthermore, our experiments demonstrate the effectiveness of GAN-generated
hybrid samples for training detection algorithms. The best overall synthetic
dataset for training CNNs was cDVGAN’s simplex dataset, while hybrid datasets
from other GANs yielded better training sets than the standard vertex dataset.
Lastly, our final experiment combines GAN-generated data with real data to
improve the performance of CNN models for glitch and signal searches, showing
cDVGAN as a viable approach for data augmentation.

Using algorithms such as BayesWave to extract glitch events would yield more
reliable glitch datasets for training cDVGAN, improving the output. The
approach could then be extended to more classes of glitches, expanding the
scope of what cDVGAN can generate. Generalized hybrid datasets that span a
larger class space might also offer more effective data augmentation.

Since hyperparameter optimization was not the focus of this research,
investigations could be made into better architectures for cDVGAN. For
example, optimization of the $\eta$ (Equation 8) hyperparameters controlling
the contribution of each discriminator to the generator loss might yield
better generated data. Including a consistency term, as in [70], may also
improve the generated data from cDVGAN. Expanding cDVGAN to other
representations of the data, might also improve the quality of the generated
data. Finally, research into better CNNs or other detection algorithms that
can make use of the GAN-generated data might also result in efficient and
scalable analysis solutions towards the next-generation detectors.

##  VI Acknowledgements

This research was conducted within the ET Technologies project (PROJ-03612)
which is partly funded by EFRO, the Province of Limburg and The Dutch Ministry
of Economic Affairs and Climate Policy within the REACT-EU Programme of OP
Zuid. The authors are grateful for contributions by members of the ET
Technologies research team, in particular; Stefano Schmidt, Andrew Miller and
Sarah Caudill. This material is based upon work supported by NSF’s LIGO
Laboratory which is a major facility fully funded by the National Science
Foundation. The authors are grateful for computational resources provided by
the LIGO Laboratory and supported by the National Science Foundation Grants
No. PHY-0757058 and No. PHY-0823459.

## References

  * Abbott  _et al._ [2016a] B. P. Abbott _et al._ (LIGO Scientific Collaboration and Virgo Collaboration), Observation of gravitational waves from a binary black hole merger, Phys. Rev. Lett. 116, 061102 (2016a).
  * LIGO Scientific Collaboration and others [2015] LIGO Scientific Collaboration and others, Advanced LIGO, Class. Quant. Grav. 32, 074001 (2015), arXiv:1411.4547 [gr-qc] .
  * Acernese  _et al._ [2015] F. Acernese _et al._ (VIRGO), Advanced Virgo: a second-generation interferometric gravitational wave detector, Class. Quant. Grav. 32, 024001 (2015), arXiv:1408.3978 [gr-qc] .
  * Abbott  _et al._ [2019] B. P. Abbott _et al._ (LIGO Scientific Collaboration and Virgo Collaboration), Gwtc-1: A gravitational-wave transient catalog of compact binary mergers observed by ligo and virgo during the first and second observing runs, Phys. Rev. X 9, 031040 (2019).
  * Abbott  _et al._ [2021a] R. Abbott _et al._ (LIGO Scientific Collaboration and Virgo Collaboration), Gwtc-2: Compact binary coalescences observed by ligo and virgo during the first half of the third observing run, Phys. Rev. X 11, 021053 (2021a).
  * Abbott  _et al._ [2023a] R. Abbott _et al._ (LIGO Scientific Collaboration, Virgo Collaboration, and KAGRA Collaboration), Gwtc-3: Compact binary coalescences observed by ligo and virgo during the second part of the third observing run, Phys. Rev. X 13, 041039 (2023a).
  * Akutsu  _et al._ [2020] T. Akutsu _et al._ , Overview of KAGRA: Detector design and construction history, Progress of Theoretical and Experimental Physics 2021, 05A101 (2020), https://academic.oup.com/ptep/article-pdf/2021/5/05A101/37974994/ptaa125.pdf .
  * Abbott  _et al._ [2020a] B. P. Abbott _et al._ , Prospects for observing and localizing gravitational-wave transients with advanced ligo, advanced virgo and kagra, Living Reviews in Relativity 23, 10.1007/s41114-020-00026-9 (2020a).
  * Hild  _et al._ [2011] S. Hild, M. Abernathy, F. Acernese, P. Amaro-Seoane, N. Andersson, K. Arun, F. Barone, B. Barr, M. Barsuglia, and M. B. e, Sensitivity studies for third-generation gravitational wave observatories, Class. Quant. Grav. 28, 094013 (2011).
  * Hall [2022] E. D. Hall , Cosmic explorer: A next-generation ground-based gravitational-wave observatory, Galaxies 10, 10.3390/galaxies10040090 (2022).
  * Iacovelli  _et al._ [2022] F. Iacovelli _et al._ , Forecasting the detection capabilities of third-generation gravitational-wave detectors using GWFAST, The Astrophysical Journal 941, 208 (2022).
  * Kalogera  _et al._ [2021] V. Kalogera _et al._ , The next generation global gravitational wave observatory: The science book (2021).
  * Fryer and New [2011] C. Fryer and K. New, Gravitational waves from gravitational collapse, Living Reviews in Relativity 14 (2011).
  * Baiotti  _et al._ [2007] L. Baiotti, I. Hawke, L. Rezzolla, and E. Schnetter, Details on the gravitational-wave emission from rotating gravitational collapse in 3d (2007).
  * Davis  _et al._ [2021] D. Davis _et al._ , Ligo detector characterization in the second and third observing runs, Classical and Quantum Gravity 38, 135014 (2021).
  * Acernese  _et al._ [2023] F. Acernese _et al._ (Virgo), Virgo detector characterization and data quality: results from the O3 run, Class. Quant. Grav. 40, 185006 (2023), arXiv:2210.15633 [gr-qc] .
  * Akutsu  _et al._ [2021] T. Akutsu _et al._ , Overview of KAGRA: Calibration, detector characterization, physical environmental monitors, and the geophysics interferometer, Progress of Theoretical and Experimental Physics 2021, 05A102 (2021), https://academic.oup.com/ptep/article-pdf/2021/5/05A102/38109702/ptab018.pdf .
  * Abbott  _et al._ [2018] B. P. Abbott _et al._ (LIGO Scientific, Virgo), Effects of data quality vetoes on a search for compact binary coalescences in Advanced LIGO’s first observing run, Class. Quant. Grav. 35, 065010 (2018), arXiv:1710.02185 [gr-qc] .
  * Abbott  _et al._ [2022] R. Abbott _et al._ (KAGRA, LIGO Scientific, VIRGO), All-sky search for continuous gravitational waves from isolated neutron stars using Advanced LIGO and Advanced Virgo O3 data, Phys. Rev. D 106, 102008 (2022), arXiv:2201.00697 [gr-qc] .
  * Steltner  _et al._ [2023] B. Steltner _et al._ , Deep Einstein@Home all-sky search for continuous gravitational waves in LIGO O3 public data, (2023), arXiv:2303.04109 [gr-qc] .
  * Abbott  _et al._ [2021b] R. Abbott _et al._ (KAGRA, Virgo, LIGO Scientific), Upper limits on the isotropic gravitational-wave background from Advanced LIGO and Advanced Virgo’s third observing run, Phys. Rev. D 104, 022004 (2021b), arXiv:2101.12130 [gr-qc] .
  * Steltner  _et al._ [2022] B. Steltner, M. A. Papa, and H. B. Eggenstein, Identification and removal of non-Gaussian noise transients for gravitational-wave searches, Phys. Rev. D 105, 022005 (2022), arXiv:2105.09933 [gr-qc] .
  * Pankow  _et al._ [2018] C. Pankow _et al._ , Mitigation of the instrumental noise transient in gravitational-wave data surrounding gw170817, Phys. Rev. D 98, 084016 (2018).
  * Davis  _et al._ [2019] D. Davis _et al._ , Improving the sensitivity of advanced ligo using noise subtraction, Class. Quant. Grav. 36, 055011 (2019).
  * Driggers  _et al._ [2019] J. C. Driggers _et al._ , Improving astrophysical parameter estimation via offline noise subtraction for advanced ligo, Phys. Rev. D 99, 042001 (2019).
  * Blackburn  _et al._ [2008] L. Blackburn _et al._ , The LSC Glitch Group: Monitoring Noise Transients during the fifth LIGO Science Run, Class. Quant. Grav. 25, 184004 (2008), arXiv:0804.0800 [gr-qc] .
  * Abbott  _et al._ [2016b] B. P. Abbott _et al._ (LIGO Scientific, Virgo), Characterization of transient noise in Advanced LIGO relevant to gravitational wave signal GW150914, Class. Quant. Grav. 33, 134001 (2016b), arXiv:1602.03844 [gr-qc] .
  * Abbott  _et al._ [2020b] B. P. Abbott _et al._ , A guide to LIGO–virgo detector noise and extraction of transient gravitational-wave signals, Class. Quant. Grav. 37, 055002 (2020b).
  * Owen and Sathyaprakash [1999] B. J. Owen and B. S. Sathyaprakash, Matched filtering of gravitational waves from inspiraling compact binaries: Computational cost and template placement, Phys. Rev. D 60, 10.1103/physrevd.60.022002 (1999).
  * Usman  _et al._ [2016] S. A. Usman _et al._ , The PyCBC search for gravitational waves from compact binary coalescence, Class. Quant. Grav. 33, 215004 (2016).
  * Cannon  _et al._ [2021] K. Cannon _et al._ , Gstlal: A software framework for gravitational wave discovery, SoftwareX 14, 100680 (2021).
  * Robinet  _et al._ [2020] F. Robinet _et al._ , Omicron: A tool to characterize transient noise in gravitational-wave detectors, SoftwareX 12, 100620 (2020).
  * Gabbard  _et al._ [2018] H. Gabbard, M. Williams, F. Hayes, and C. Messenger, Matching matched filtering with deep networks for gravitational-wave astronomy, Phys. Rev. Lett. 120, 141103 (2018).
  * Gebhard  _et al._ [2019] T. D. Gebhard, N. Kilbertus, I. Harry, and B. Schölkopf, Convolutional neural networks: A magic bullet for gravitational-wave detection?, Phys. Rev. D 100, 063015 (2019).
  * Krastev [2020] P. G. Krastev, Real-time detection of gravitational waves from binary neutron stars using artificial neural networks, Physics Letters B 803, 135330 (2020).
  * Cuoco  _et al._ [2020] E. Cuoco _et al._ , Enhancing gravitational-wave science with machine learning, Machine Learning: Science and Technology 2, 011002 (2020).
  * Zevin  _et al._ [2017] M. Zevin _et al._ , Gravity spy: integrating advanced LIGO detector characterization, machine learning, and citizen science, Class. Quant. Grav. 34, 064003 (2017).
  * Wu  _et al._ [2024] Y. Wu, M. Zevin, C. P. L. Berry, K. Crowston, C. Østerlund, Z. Doctor, S. Banagiri, C. B. Jackson, V. Kalogera, and A. K. Katsaggelos, Advancing glitch classification in gravity spy: Multi-view fusion with attention-based machine learning for advanced ligo’s fourth observing run (2024), arXiv:2401.12913 [gr-qc] .
  * Bahaadini  _et al._ [2017] S. Bahaadini, N. Rohani, S. Coughlin, M. Zevin, V. Kalogera, and A. K. Katsaggelos, Deep multi-view models for glitch classification, 2017 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP) , 2931 (2017).
  * Bahaadini  _et al._ [2018] S. Bahaadini, V. Noroozi, N. Rohani, S. Coughlin, M. Zevin, J. Smith, V. Kalogera, and A. Katsaggelos, Machine learning for gravity spy: Glitch classification and dataset, Information Sciences 444, 172 (2018).
  * George  _et al._ [2018] D. George, H. Shen, and E. Huerta, Classification and unsupervised clustering of LIGO data with deep transfer learning, Phys. Rev. D 97, 10.1103/phvysrevd.97.101501 (2018).
  * Goodfellow  _et al._ [2014] I. J. Goodfellow _et al._ , Generative adversarial networks (2014).
  * Powell  _et al._ [2022] J. Powell, L. Sun, K. Gereb, P. D. Lasky, and M. Dollmann, Generating transient noise artifacts in gravitational-wave detector data with generative adversarial networks (2022).
  * Yan  _et al._ [2022] J. Yan, A. P. Leung, and D. C. Y. Hui, On improving the performance of glitch classification for gravitational wave detection by using generative adversarial networks (2022).
  * Abadie  _et al._ [2010a] J. Abadie _et al._ , Search for gravitational waves from compact binary coalescence in ligo and virgo data from s5 and vsr1, Physical Review D 82, 10.1103/physrevd.82.102001 (2010a).
  * Abadie  _et al._ [2010b] J. Abadie _et al._ , All-sky search for gravitational-wave bursts in the first joint ligo-geo-virgo run, Physical Review D 81, 10.1103/physrevd.81.102001 (2010b).
  * Biwer  _et al._ [2017] C. Biwer _et al._ , Validating gravitational-wave detections: The advanced ligo hardware injection system, Phys. Rev. D 95, 10.1103/physrevd.95.062002 (2017).
  * Lopez  _et al._ [2022] M. Lopez _et al._ , Simulating transient noise bursts in ligo with generative adversarial networks (2022).
  * Arjovsky  _et al._ [2017a] M. Arjovsky, S. Chintala, and L. Bottou, Wasserstein gan (2017a).
  * Cornish and Littenberg [2015] N. J. Cornish and T. B. Littenberg, Bayeswave: Bayesian inference for gravitational wave bursts and instrument glitches, Class. Quant. Grav. 32, 135012 (2015).
  * McGinn  _et al._ [2021] J. McGinn _et al._ , Generalised gravitational wave burst generation with generative adversarial networks, Class. Quant. Grav. 38, 155005 (2021).
  * Arjovsky  _et al._ [2017b] M. Arjovsky, S. Chintala, and L. Bottou, Wasserstein generative adversarial networks, in  _Proceedings of the 34th International Conference on Machine Learning_, Proceedings of Machine Learning Research, Vol. 70, edited by D. Precup and Y. W. Teh (PMLR, 2017) pp. 214–223.
  * Gulrajani  _et al._ [2017] I. Gulrajani _et al._ , Improved training of wasserstein gans (2017).
  * Salimans  _et al._ [2016] T. Salimans _et al._ , Improved techniques for training gans (2016).
  * Karras  _et al._ [2018] T. Karras _et al._ , Progressive growing of GANs for improved quality, stability, and variation, in  _International Conference on Learning Representations_ (2018).
  * Mirza and Osindero [2014] M. Mirza and S. Osindero, Conditional generative adversarial nets (2014), arXiv:1411.1784 [cs.LG] .
  * Miyato and Koyama [2018] T. Miyato and M. Koyama, cgans with projection discriminator (2018), arXiv:1802.05637 [cs.LG] .
  * Dooney  _et al._ [2022] T. Dooney, S. Bromuri, and L. Curier, Dvgan: Stabilize wasserstein gan training for time-domain gravitational wave physics, in  _2022 IEEE International Conference on Big Data (Big Data)_ (2022) pp. 5468–5477.
  * Cabero  _et al._ [2019] M. Cabero _et al._ , Blip glitches in Advanced LIGO data, Class. Quant. Grav. 36, 15 (2019), arXiv:1901.05093 [physics.ins-det] .
  * Glanzer  _et al._ [2022] J. Glanzer _et al._ , Data quality up to the third observing run of advanced ligo: Gravity spy glitch classifications (2022).
  * Macleod  _et al._ [2021] D. M. Macleod, J. S. Areeda, S. B. Coughlin, T. J. Massinger, and A. L. Urban, GWpy: A Python package for gravitational-wave astrophysics, SoftwareX 13, 100657 (2021).
  * Abbott  _et al._ [2023b] R. Abbott _et al._ (KAGRA, VIRGO, LIGO Scientific), Open Data from the Third Observing Run of LIGO, Virgo, KAGRA, and GEO, Astrophys. J. Suppl. 267, 29 (2023b), arXiv:2302.03676 [gr-qc] .
  * Gallagher [2020] N. Gallagher, Savitzky-golay smoothing and differentiation filter (2020).
  * Nitz  _et al._ [2022] A. Nitz _et al._ , gwastro/pycbc: v2.0.4 release of pycbc (2022).
  * LIGO Scientific Collaboration [2018] LIGO Scientific Collaboration, LIGO Algorithm Library - LALSuite, free software (GPL) (2018).
  * Fawaz  _et al._ [2019] H. I. Fawaz _et al._ , Deep learning for time series classification: a review, Data Mining and Knowledge Discovery 33, 917 (2019).
  * Allen  _et al._ [2012] B. Allen, W. G. Anderson, P. R. Brady, D. A. Brown, and J. D. E. Creighton, Findchirp: An algorithm for detection of gravitational waves from inspiraling compact binaries, Physical Review D 85, 10.1103/physrevd.85.122006 (2012).
  * Drago  _et al._ [2021] M. Drago, S. Klimenko, C. Lazzaro, E. Milotti, G. Mitselmakher, V. Necula, B. O’Brian, G. A. Prodi, F. Salemi, M. Szczepanczyk, S. Tiwari, V. Tiwari, G. V, G. Vedovato, and I. Yakushin, coherent waveburst, a pipeline for unmodeled gravitational-wave data analysis, SoftwareX 14, 100678 (2021).
  * Klimenko  _et al._ [2008] S. Klimenko _et al._ , A coherent method for detection of gravitational wave bursts, Class. Quant. Grav. 25, 114029 (2008).
  * Xiang  _et al._ [2018] W. Xiang _et al._ , Improving the improved training of wasserstein gans: A consistency term and its dual effect (2018).

##  VII Appendix

###  VII.1 cDVGAN Architecture

|  | Discriminator |  |  | (3.5M param.)
---|---|---|---|---|---
Operation | Output shape | Kernel size | Stride | Dropout | Activation
Input | (1024) | - | - | 0 | -
Reshape | (64,16) | - | - | 0 | -
Convolutional | (64,128) | 14 | 2 | 0.5 | Leaky ReLU
Convolutional | (32,128) | 14 | 2 | 0.5 | Leaky ReLU
Convolutional | (16, 256) | 14 | 2 | 0.5 | Leaky ReLU
Convolutional | (8, 256) | 14 | 2 | 0.5 | Leaky ReLU
Convolutional | (4, 512) | 14 | 2 | 0.5 | Leaky ReLU
Global Avg. Pooling | (512) | - | - | 0.5 | -
Avg. Pooling Dense | (128) | - | - | 0.2 | Leaky ReLU
Dense | (1) | - | - | 0 | Linear
Class Input | (3) | - | - | - | -
Class Dense | (128) | - | - | 0 | Linear
Scalar Product | (1) | - | - | - | -
Dense + Scalar Product | (1) | - | - | - | -
|  | DV Discriminator |  |  | (1.1M param.)
Operation | Output shape | Kernel size | Stride | Dropout | Activation
Input | (1023) | - | - | 0 | -
Dense | (512) | - | - | 0 | Leaky ReLU
Reshape | (32,16) | - | - | 0 | -
Convolutional | (32, 64) | 5 | 2 | 0.5 | Leaky ReLU
Convolutional | (16,128) | 5 | 2 | 0.5 | Leaky ReLU
Convolutional | (8,256) | 5 | 2 | 0.5 | Leaky ReLU
Convolutional | (4,256) | 5 | 2 | 0.5 | Leaky ReLU
Global Avg. Pooling | (256) | - | - | 0.5 | -
Avg. Pooling Dense | (128) | - | - | 0.2 | Leaky ReLU
Dense | (1) | - | - | 0 | Linear
Class Input | (3) | - | - | - | -
Class Dense | (128) | - | - | 0 | Linear
Scalar Product | (1) | - | - | - | -
Dense + Scalar Product | (1) | - | - | - | -
|  | Generator |  |  | 3.5M param.
Operation | Output shape | Kernel size | Stride | BN | Activation
Latent input | (100) | - | - | ✗ | -
Class Input | (3) | - | - | ✗ | -
Class Dense | (32) | - | - | ✗ | -
Concatenate | (132) | - | - | ✗ | -
Dense | (1024) | - | - | ✗ | ReLU
Reshape | (32,32) | - | - | ✗ | -
Transposed conv. | (64,512) | 18 | 2 | ✓ | ReLU
Transposed conv. | (128,256) | 18 | 2 | ✓ | ReLU
Transposed conv. | (256,128) | 18 | 2 | ✓ | ReLU
Transposed conv. | (512, 64) | 18 | 2 | ✓ | ReLU
Transposed conv. | (1024,1) | 18 | 2 | ✗ | Linear
Flatten | (1024) | - | - | ✗ | -
Optimizer | RMSprop($\alpha=0.0001$) |  |  |  |
Batch size | 512 |  |  |  |
Epochs | 500 |  |  |  |
Loss | Wasserstein |  |  |  |
Table 5: The architecture and hyperparameters describing cDVGAN, which
consists of a base discriminator, a derivative (DV) discriminator and a
generator convolutional network. The additional discriminator of cDVGAN2
follows the same architecture as the DV disciminator but with an input shape
of 1022.

###  VII.2 CNN Architecture

Operation | Output shape | Kernel size | Stride | Dropout | Activation
---|---|---|---|---|---
Input | (1024) | - | - | 0 | -
Reshape | (1024,1) | - | - | 0 | -
Convolutional | (512,256) | 5 | 2 | 0.5 | Leaky ReLU
Convolutional | (256,128) | 5 | 2 | 0.5 | Leaky ReLU
Convolutional | (128,64) | 5 | 2 | 0.5 | Leaky ReLU
Convolutional | (64,32) | 5 | 2 | 0.5 | Leaky ReLU
Flatten | (2048) | - | - | 0.5 | -
Dense | (512) | - | - | 0 | Leaky ReLU
Dense | (1) | - | - | 0 | Sigmoid
Optimizer | Adam($\alpha=0.001$) |  |  |  |
Batch size | 64 |  |  |  |
Epochs | 20 |  |  |  |
Loss | Binary Crossentropy |  |  |  |
Table 6: The architecture of the CNN (1.2M parameters) used during
experiments.

###  VII.3 Vertex, Simplex and Uniform Datasets

Figure 10: Standard 3-class vertex generations from cDVGAN. Figure 11: Simplex
generations from cDVGAN. Figure 12: Uniform generations from cDVGAN.

###  VII.4 Class Interpolation

Figure 13: Interpolation between blip and BBH classes for cDVGAN (1st row),
cDVGAN2 (2nd row), cWGAN (3rd row), McGANn (4th row) and McDVGANn (5th row).
The class input is shown at the top of each column, while the latent input of
the generator is kept constant.

###  VII.5 Gravity Spy Analysis of GAN Training Data

As we have discussed in section III.3.3, we have scaled some examples of blip
and tomte glitches $g(t)$, adding them to the noise $n(t)$ from the third
observing (O3) run in the GPS time range $[1262540000,1262540040]$. Note that
the glitches have been extracted from their original noise following the
procedure previously presented in section III.2. Afterwards, the time-series
$s(t)=g(t)+n(t)$ was classified with Gravity Spy, providing a class label and
a class confidence $c_{GS}$.

In Fig. 14 we present the results of classifying three different denoised
blips re-injected in O3 noise. In the top panel, we plot the the confidence
$c_{GS}$ as a function of the optimal SNR $\rho_{opt}$, defined in Eq. 10. In
the middle panel, we show the time-series of the blip as outputted by the pre-
processing, and in the bottom panel, we show the spectrogram or time-frequency
representation of the glitch embedded in O3 real noise at $\rho_{opt}=18.32$.

(a)

(b)

(c)

Figure 14: Top panel: Three examples of blip glitches classified by Gravity
Spy, with the classification confidences as a function of SNR. Middle panel:
The time-series representation of the classified glitch. Bottom panel: The
corresponding spectrogram representation of the classified glitch after
injecting into whitened detector noise.

We can observe in Fig. 14 that Gravity Spy classifies the time-series as
No$\\_$Glitch for SNR $\lesssim 7$, which is expected as this algorithm only
learns glitches with an SNR $\leq 7.5$. While these examples are labelled as
blips at SNR $\sim 10$, as the SNR increases they get misclassified as
Blip$\\_$Low$\\_$Frequency or Low$\\_$Frequency$\\_$Burst, which can be
explained by the use of band-pass filtering that attenuates the high-frequency
contribution of blips.

Similarly, in Fig. 15 we can also observe that the time-series gets classified
as No$\\_$Glitch, but this time at an SNR $\lesssim 5$. Then, for
$5\lesssim\text{ SNR }<10$ it gets briefly classified as
Blip$\\_$Low$\\_$Frequency, to then be classified as Tomte for most of the SNR
range, meaning that the features of tomtes are better preserved than the blip
class after data pre-processing.

(a)

(b)

(c)

Figure 15: Top panel: Three examples of tomte glitches classified by Gravity
Spy, with the classification confidences as a function of SNR. Middle panel:
The time-series representation of the classified glitch, Bottom panel: The
corresponding spectrogram representation of the classified glitch after
injecting into whitened detector noise.

###  VII.6 McDVGANn Additional Analysis

Figure 16: A comparison of McDVGANn CNN performance trained with a
discriminator update schedule of 1 for each generator update and a
discriminator update schedule of 3 times for each generator update, which is
used for the original McGANn model. These results suggest that McDVGANn
overfits the data with the original McGANn update schedule and that using a
discriminator update schedule of 1 is superior.


# Gravitational lensing modification of the high redshift galaxy luminosity
function

G. Ferrami 1,2 ID J. Stuart B. Wyithe 1,2 ID
1School of Physics, University of Melbourne, Parkville, VIC 3010, Australia
2ARC Centre of Excellence for All-Sky Astrophysics in 3 Dimensions (ASTRO 3D)
E-mail<EMAIL_ADDRESS>
(Accepted XXX. Received YYY; in original form ZZZ)

###### Abstract

The bright end of the rest-frame UV luminosity function (UVLF) of high-
redshift galaxies is modified by gravitational lensing magnification bias.
Motivated by recent discoveries of very high-z galaxies with JWST, we study
the dependence of magnification bias on the finite size of sources at
$6<z<14$. We calculate the magnification probability distributions and use
these to calculate the magnification bias assuming a rest-frame Schechter UVLF
for galaxies at redshift $6<z<14$. We find that the finite size of bright
high-redshift galaxies together with lens ellipticity significantly suppresses
magnification bias, producing an observed bright end which declines more
sharply than the power-law resulting from assumption of point sources. By
assuming a luminosity-size relation for the source population and comparing
with the observed $z=6$ galaxy luminosity function from Harikane et al.
(2022), we show that the UVLF can be used to set mild constraints on the
galaxies intrinsic size, favoring smaller galaxies compared to the fiducial
luminosity-size relation. In the future, wide surveys using Euclid and Roman
Space Telescope will place stronger constraints. We also tabulate the maximum
magnification possible as a function of source size and lens ellipticity.

###### keywords:

gravitational lensing: strong – galaxies: high-redshift – galaxies: luminosity
function – galaxies: evolution

††pubyear: 2022††pagerange: Gravitational lensing modification of the high
redshift galaxy luminosity function–References

##  1 Introduction

Study of the rest-frame UV luminosity function (UVLF) is fundamental to
understanding the role of high-redshift galaxies during the epoch of
reionization and their evolution through cosmic time. In recent years, surveys
have measured the UVLF over a large absolute magnitude interval up to
$z\approx 9$ (e.g. Bouwens et al. 2022, Harikane et al. 2022, Atek et al.
2018, Livermore et al. 2017, Finkelstein et al. 2015). These works suggest
that the rest-frame UVLF is well fitted by a Schechter function (a power law
at faint luminosities and an exponential cut-off at bright luminosities,
Schechter 1976) or a double power-law, whose parameters evolve with redshift.
The number counts and flux measurements of high-redshift sources are distorted
by gravitational lensing due to foreground structures (Barkana & Loeb 2000,
Wyithe et al. 2011, Mason et al. 2015). In particular, considering a Schechter
profile for the intrinsic UVLF, the magnification of more numerous fainter
galaxies leads to an apparent increase in the observed number of bright
galaxies, an effect called magnification bias (Turner et al. 1984). Other
studies focused on modeling the low magnification regime for virialized and
non-virialized lens structures (Fialkov & Loeb 2015). Barone-Nugent et al.
(2015) directly measured the magnification bias on a sample of Lyman-break
galaxies at $4\leq z\leq 8$ in the eXtreme Deep Field (XDF) and the Cosmic
Assembly Near-infrared Deep Extragalactic Legacy Survey (CANDELS) fields.
Modification of the statistics of the bright end of the observed high-redshift
galaxy UVLF are dominated by galaxy-scale lenses, which are well described by
isothermal ellipsoids (Keeton et al. 1997). Background galaxy sources are
finite-sized with a luminosity profile characterized by a characteristic
radius. Both non-spherical lenses and extended sources affect the predictions
of a lensing model (e.g. see Schneider et al. 1992 for a theoretical
introduction and Oguri 2002 for an application on the statistics of strong
lensing radial arcs). However, discussion of lensing statistics for high-z
galaxies has been restricted to spherical lenses and point sources.
In this Letter we study the combined effect of elliptical lenses and extended
sources on the bright end of the luminosity function for high-redshift
galaxies. In Section 2, we obtain magnification probability distributions for
extended sources assuming a singular isothermal ellipsoid (SIE) lens model. In
Section 3, we calculate apparent luminosity functions and the magnification
bias assuming a rest-frame Schechter UVLF for galaxies at redshift $6<z<14$.
We then compare our models with some previous observational results for the UV
luminosity functions for galaxies at $z=6$ and show how this could lead to
constraints on the intrinsic size distribution for high-redshift galaxies.
Discussion and conclusions are presented in Section 4. Throughout this paper,
we adopt $H_{0}=70$ km s-1 Mpc-1, $\Omega_{0}=0.3$, $\Omega_{\Lambda}=0.7$.

Figure 1: Magnification probability distribution for the brightest image
produced by SIE lenses with axis ratios $f=0.3$ (left), $f=0.7$ (center) and
$f=1$ (SIS model, right). The magnification probability distributions for
point-like sources share the $\derivative{P}{\mu}\sim\mu^{-3}$ trend
characteristic of the SIS profile (e.g. see Wyithe et al. 2011), while the
magnification probability distributions for extended sources show a cutoff
after a certain maximum value of magnification inversely related to the
dimensionless half-light radius of the sources $R_{hl}^{a}$.

##  2 Magnification probability distributions

The dimensionless lens equation relating source position ${\bf\it y}$ to image
positions ${\bf\it x}$ through the deflection angle ${\bf\it\alpha}({\bf\it
x})$ is

${\bf\it y}={\bf\it x}-{\bf\it\alpha}({\bf\it x})\text{ ,}$ (1)

where ${\bf\it x}$ and ${\bf\it y}$ are expressed in units of the Einstein
radius, which, for a singular isothermal sphere lens model (SIS, see for
example Schneider et al. 1992), is

$\theta_{E}(\sigma,z)=4\pi\frac{D_{ls}}{D_{s}}\left(\frac{\sigma}{c}\right)^{2}=1^{\prime\prime}\frac{D_{ls}}{D_{s}}\left(\frac{\sigma}{186\text{
km/s}}\right)^{2}\text{ .}$ (2)

Here $\sigma$ is the velocity dispersion. Therefore $\theta_{E}\sim
1^{\prime\prime}$ in galaxy-scale lenses with a geometry such that
${D_{ls}}/{D_{s}}\approx 1$. The cosmological distances $D_{l}$, $D_{s}$,
$D_{ls}$ are respectively the angular diameter distances between the observer
and the deflector, the observer and the source, and the deflector and the
source.
To calculate the magnification, we consider a singular isothermal ellipsoid
(SIE, see Kormann et al. 1994) model for the lens, characterized by the
parameter $f$ defined as the ratio between the minor and major axes of the
elliptical isodensity contours111The other model parameters are ignored here,
since the connection with the value of the surface mass density is removed by
the dimensionless scaling, and our analysis is not sensible to the absolute
position and rotation of the lens model.. Due to the singularity of the
surface mass density, a SIE lens produces 1, 2 or 4 images of a point source,
depending on the position of the source with respect to lens.
Gravitational lensing conserves surface-brightness $I$ but stretches the solid
angle that an unlensed source subtends in the sky. The ratio of the observed
and intrinsic (unlensed) fluxes gives the magnification $\mu$. For an extended
source, one integrates the point-like magnification over the surface-
brightness profile of the source $I({\bf\it y})$ to obtain

$\mu_{e}=\frac{\int_{R^{2}}\differential^{2}yI({\bf\it y})\mu_{p}({\bf\it
y})}{\int_{R^{2}}\differential^{2}yI({\bf\it y})}\text{ .}$ (3)

An extended source therefore has a lower or equal magnification than a point-
source through the same lens. For example, considering a circular source with
a constant surface-brightness near a linear caustic, it has been shown that
there is a maximum magnification at a finite distance from the caustic (see
Appendix of Schneider & Weiss 1987). This implies that the magnification
probability distribution has a cutoff after $\mu_{max}$.
Our choice of modeling elliptical lenses and extended sources improves upon
the previous statistical lensing studies on the luminosity function of high-
redshift galaxies, which considered point-like sources and a singular
isothermal sphere model for the lenses (Wyithe et al. 2011, Mason et al.
2015). The effect of the finite size of a source becomes significant when it
is comparable with its distance from the caustic. We expect bright galaxies at
high-redshift to have angular sizes $\lesssim 0.5^{\prime\prime}$ (e.g. see
Liu et al. 2017 for a semi-analytical model of high-z galaxies size), which is
smaller than but comparable to the Einstein radius $\theta_{E}$.
We implemented the numerical solution for the SIE lens model discussed in
Kormann et al. (1994), initializing a dimensionless source plane with a
$l^{2}$ square area divided in $1000\times 1000$ pixels. We chose the value to
be $l=4.2$, to have both high resolution in the high magnification regions and
explore the low magnification regions away from the caustic. We solve the lens
equation assuming different choices of the lens axis ratio $f$ for a point-
like source in each pixel, and retrieved a map of the magnification values for
the brightest of the images of each source. We then applied Eq. (3)
considering a circular source with a constant surface-brightness and different
choices for the values of the dimensionless half-light radius $R^{a}_{hl}$ of
the source, obtained by scaling its physical size $R_{hl}$ in the source plane
as $R_{hl}^{a}=\frac{R_{hl}}{\theta_{E}}\frac{D_{s}}{D_{l}}$. From these
magnification maps, we obtained the magnification probability distributions
for the brightest image as a function of $f$ and $R^{a}_{hl}$.
Some of those are represented in Fig. 1. The magnification probability
distributions for point-like sources show the expected power-law
characteristic of the SIS profile (e.g. see Pei 1995, Wyithe et al. 2011),
while the magnification probability distributions for extended sources present
a cutoff above a certain maximum value of magnification. This maximum
increases toward small sources and spherical lenses, and will be of interest
in particular observed cases. The values of maximum magnification as a
function of source size and lens ellipticity are listed in Table 1.

Table 1: Maximum magnification for an extended source. | Source dimensions [arcsec $\times(1"/\theta_{E})\times(D_{s}/D_{l})$]
---|---
|  0.05 |  0.10 |  0.15 |  0.20 |  0.40 |  0.60 |  0.80 |  1.00
Lens axes ratio $f\times 10$ |  1 |  5.74 |  3.62 |  2.87 |  2.46 |  1.95 |  1.49 |  1.23 |  1.18
2 |  9.12 |  5.97 |  4.74 |  4.06 |  2.66 |  2.03 |  1.68 |  1.49
3 |  10.23 |  6.70 |  5.32 |  4.39 |  2.98 |  2.46 |  2.11 |  1.81
4 |  11.05 |  7.24 |  5.53 |  4.56 |  3.22 |  2.56 |  2.28 |  1.95
5 |  12.41 |  7.82 |  5.97 |  4.92 |  3.35 |  2.66 |  2.28 |  2.19
6 |  13.40 |  8.44 |  6.45 |  5.74 |  3.48 |  2.87 |  2.66 |  2.46
7 |  14.47 |  8.77 |  7.52 |  5.53 |  4.06 |  3.48 |  2.98 |  2.66
8 |  16.24 |  11.49 |  7.52 |  6.20 |  4.92 |  3.91 |  3.22 |  2.76
9 |  22.97 |  12.89 |  11.94 |  9.85 |  5.74 |  4.22 |  3.35 |  2.87
SIS |  42.54 |  21.27 |  14.47 |  10.64 |  5.97 |  4.22 |  3.48 |  2.98

##  3 Lensed luminosity functions of extended sources

The a-priori probability for a source to be lensed by a foreground galaxy into
multiple images is defined as the optical depth $\tau_{m}$ (e.g., see Wyithe
et al. 2011 and references therein), defined as

$\begin{split}\tau_{m}(z_{s})=\int_{0}^{z_{s}}\differential
z_{l}\int_{0}^{\infty}\differential\sigma\Phi(\sigma,z_{l})(1+z_{l})^{3}\frac{c\differential
t}{\differential z_{l}}\pi D_{l}^{2}\theta_{E}^{2}(\sigma,z_{l})\text{
,}\end{split}$ (4)

where $z_{s}$ and $z_{l}$ are respectively the source and lens redshifts,
$\Phi(\sigma,z)$ is the velocity dispersion function. Flux limited samples are
also subject to magnification bias, which increases the relative probability
that detected galaxies are gravitationally lensed, and concentrates sources in
a flux limited sample around foreground objects (Webster et al. 1988, Wyithe
et al. 2011).
The observed luminosity function can therefore be approximated as the sum of
the apparent luminosity function of the fraction of lensed galaxies and the
intrinsic UVLF

$\Psi_{obs}(L,R_{hl},f)\approx\left(1-\tau_{m}l^{2}\right)\Psi\left(L\right)+\Psi_{lensed}(L,R_{hl},f)\text{
.}$ (5)

We ignored the effective small demagnification in the unlensed UVLF in the
right-hand side due to the conservation of flux over the whole sky since its
effect is negligible over the bright-end of the apparent UVLF. We adopted the
standard Schechter function (Schechter 1976) for the intrinsic luminosity
function

$\Psi(L)\differential
L=\Psi_{\star}\left(\frac{L}{L_{\star}}\right)^{\alpha}\exp\left(-\frac{L}{L_{\star}}\right)\frac{\differential
L}{L_{\star}}\text{ .}$ (6)

The evolution of the Schechter function parameters with respect to redshift is
taken from Bouwens et al. (2022) (in particular, see Fig. 6 and Sect. 3.4 in
that paper).
The apparent luminosity function of the fraction of lensed galaxies for a
fixed value of ellipticity $f$ of the lens population is then

$\begin{split}\Psi_{lensed}(L,R_{hl},f)&=\int_{0}^{z_{s}}\differential
z_{l}\int_{0}^{\infty}\differential\sigma\int_{0}^{\infty}\differential\mu\Phi(\sigma,z_{l})(1+z_{l})^{3}\frac{c\differential
t}{\differential z_{l}}\\\
&\times\mathcal{A}D_{l}^{2}\theta_{E}^{2}(\sigma,z_{l})\frac{1}{\mu}\derivative{P}{\mu}\left(R_{hl},f,\sigma,z_{l},z_{s}\right)\Psi\left({\frac{L}{\mu}}\right)\text{
,}\end{split}$ (7)

where $\mathcal{A}=\left({D_{l}}/{D_{s}}\times l\right)^{2}$ is the
dimensionless source plane area used in the previous section to derive
$\frac{\textit{d}P}{\textit{d}\mu}$. Note that this expression for the bias is
different than the one in Wyithe et al. (2011), as we can’t separate the
optical depth term from the integral since in the case of finite sources the
magnification probability distribution is a function of the geometry and mass
of the lenses.
We adopt the Mason et al. (2015) velocity dispersion function and its
evolution with redshift

$\Phi(\sigma,z)=\frac{\ln(10)}{p}\frac{\Phi^{\star}_{S}(z)}{\sigma(1+z)^{\beta}}\left(\frac{\sigma}{\sigma^{\star}}\right)^{p^{-1}(1+\alpha_{S})}\exp\left[-\left(\frac{\sigma}{\sigma^{\star}}\right)^{p^{-1}}\right]\text{
,}$ (8)

with their best fit values for the parameters $p=0.24$, $\beta=0.20$,
$\alpha_{S}=-0.54$, $\sigma^{\star}=216\text{ km/s}$, and
$\Phi^{\star}_{S}(z)=3.75\times 10^{-3}(1+z)^{-2.46}\text{Mpc}^{-3}$. This is
close to the local Velocity Dispersion Function measured by the SDSS (Choi et
al. 2007).
The relation between galaxy size and luminosity is commonly fitted by a power
law

$R_{hl}=R_{0}\left(\frac{L}{L_{0}}\right)^{\gamma}\text{ ,}$ (9)

where $R_{0}$ is the effective radius at $L_{0}$, and $\gamma$ is the slope.
Here $L_{0}$ is the characteristic UV luminosity for Lyman-break galaxies at
$z=3$, which corresponds to magnitude $M_{0}=-21$ (Steidel et al. 1999). As a
reference, we adopted the best-fit values of $R_{0}=0.61$ kpc and
$\gamma=0.25$ at $z=7$ found in Liu et al. (2017). In this work we favoured
the use of this simple relation over more complex luminosity-size relations
(e.g. Genel et al. 2018).
The fraction of observed lensed galaxies per luminosity interval at the bright
end is well approximated by the ratio $\Psi_{lensed}/\Psi_{obs}$, as shown in
the top panel of Fig. 2. In the lower panel of Fig. 2 we show the ratio of LFs
with respect to a point source case. We see that extended sources can suppress
the effect of magnification bias by several orders of magnitude.

Figure 2: (Upper) Ratio between the lensed UVLF and observed UVLF
$\Psi_{lensed}/\Psi_{obs}$, which in the bright end approximates the fraction
of observed lensed galaxies per luminosity interval for different L-size
relations at $z=7$ (solid, thin dashed and dotted lines are for redshift
$z=11$ and $z=14$). (Lower) Ratio between the apparent UVLF for a point-source
and the apparent UVLF for extended sources with different choices for the
value of $\gamma$ in the L-size relation, with $R_{0}=0.11"$. On the left side
we show the results assuming a population of SIE lenses with axis ratio
$f=0.4$, in the center with $f=0.7$, and on the right assuming a population of
spherical lenses.

###  3.1 Comparison with observed luminosity functions

We next consider the lensed UVLF for a population of lenses with varying
ellipticities. Based on the ellipticity distribution for the deflector
population, we do a weighted sum over $f$ to obtain the predicted observed
UVLF

$\Psi_{obs}(L,R_{hl})=\int\differential fp(f)\Psi_{obs}(L,R_{hl},f)\text{ .}$
(10)

We included the dependency on the ellipticity distribution of lenses using the
geometry constraints up to $z=2$ provided by the SDSS (van der Wel et al.
2014) and assuming that the ellipticity of the lensing potential is
approximated by the projected star distribution ellipticity 222The assumption
is reasonable considering a constant mass-to-light ratio and a weak
contribution of the dark matter halo in the central region where strong
lensing happens. Within the distributions listed by van der Wel et al. 2014,
we choose the one associated with the stellar mass bin $10<M_{\star}<10.5$ at
redshift $1.5<z<2$ as it represents the galaxy population of the most
effective lenses (due to their strength and cosmological distances geometry).
The final distribution $p(f)$ is approximately a Gaussian centered on $\langle
f\rangle=0.5$ with a standard deviation of $\sigma_{f}=0.3$.

Figure 3: Lensed luminosity functions of extended sources assuming different
choices of luminosity-size relation at redshift $z=6$. The intrinsic UVLF is
shown in black, and it is chosen to be a Schechter function with parameters
obtained in Bouwens et al. 2022, as the best fit from the data from Bouwens et
al. 2021 (magenta squares). The green line represents the apparent UVLF
expected for a population of point-like sources. The yellow, magenta and cyan
lines represent a luminosity-size relation with $R_{0}=0.11",0.05",0.02"$
respectively. The dashed, dashed-dotted and dotted lines represent a
luminosity-size relation with $\gamma=0.25,0.15,0.05$ respectively. For
comparison, we show observational results for the UV luminosity function for
galaxies obtained from dropout sources in Harikane et al. 2022 (red dots). The
inset plot shows the value of the Bayes factor comparing the extended source
model and the point source model (higher values corresponds to more likely
models).

In Fig. 3 we compare our prediction for the galaxy UVLF at $z=6$ to
observational results at that redshift from Harikane et al. (2022) and Bouwens
et al. (2021). The inset plot shows the value of the Bayes factor comparing
the extended source model and the point source model. The effect of extended
sources is to suppress the apparent brightening of the UVLF. This is due to
the cutoff in the magnification probability distribution shown in Fig. 1,
which convolved with the exponential shape of the Schechter profile gives the
deviation from the power-law bright end expected from point-like sources. This
provides a constraint on the intrinsic size of high-z galaxies because of the
small number of observations of bright ($M_{AB}\lesssim M_{AB,\star}-2$)
sources. The brightest data-point is compatible with a non-detection,
therefore we do not penalize the models that predict a lower density at that
magnitude, but the second brightest point marginally favors galaxies that are
smaller compared to the predicted L-size relation parameters values from
observed high-z galaxies (Kawamata et al. 2018, Yang et al. 2022, or the
clumpy and extended morphologies found in Bowler et al. 2017), simulations
(Liu et al. 2017, Marshall et al. 2019, or the dust-attenuated $L_{UV}$-size
relation in Marshall et al. 2022). That is, for a galaxy with $M_{AB}=-21$ at
$z=6$, we find a size of $\approx 115$ pc instead of $\approx 800$ kpc found
in Liu et al. (2017) and Kawamata et al. (2018). Yang et al. (2022) reports a
size of around $\approx 400$ kpc in rest frame optical, and $\approx 200$ pc
in rest frame UV, using a sample of galaxies with $z>7$ from early JWST data.
The lensed Schechter presented in Harikane et al. (2022) (following Wyithe et
al. 2011) is lower than the one predicted in our work, due to the difference
in the adopted magnification probability distribution.
In the future we expect the UVLF at the bright end will probe the source size.
To illustrate, in Fig. 4 we show the observed LFs at $7<z<14$ for both a
constant source size and a size-luminosity relation, along with a comparison
to some current and future surveys aiming to probe the first galaxies volume
density at the bright end of the UVLF. We see that the LFs from future wide
field high-z surveys could be strongly affected by gravitational lensing.
We note that observations of the bright end of the galaxy luminosity function
may suffer AGN contamination (Harikane et al. 2022; see also Leethochawalit et
al. 2022 and Bagley et al. 2022 for potential evidence of such contamination
at redshift $z=8$ and $z=9$). Additional studies of the intrinsic AGNs
luminosity function are needed to correctly disentangle the two population. It
is worth remarking that AGNs would behave as point-like sources. However
lensing bias would play a negligible effect on them assuming an intrinsic
double power-law luminosity function profile (e.g. see Wyithe & Loeb 2003, Qin
et al. 2017, and Ren & Trenti 2021).
Furthermore, recent studies by Mason et al. (2022) and Ferrara et al. (2022)
predict that the bright end of the UV luminosity function of high-z galaxies
is strongly affected by dust attenuation. Significant dust attenuation has
also been observationally confirmed in bright high-redshift galaxies (e.g.
Bowler et al. 2022). Since this modification on the UVLF profile happens in
the rest-frame of the source, it will introduce a steepening in the intrinsic
UVLF bright end, thus increasing the effects of gravitational lensing. If on
the other hand the slope of the galaxy size-luminosity relation depends on
dust attenuation (Wu 2018, Marshall et al. 2022), we expect to see a change in
the observed bright-end slope at different observed rest-frame wavelengths.

Figure 4: Predictions for the observed LFs caused by a foreground population
of elliptical lenses for sources with constant half-light radius (left) and
assuming a luminosity-size relation (right), for three values of redshift:
$z=7$ (solid), $z=11$ (dashed) and $z=14$ (dotted). The intrinsic UVLF is
shown in black, and it is chosen to be a Schechter function with parameters
obtained in Bouwens et al. 2022. The green line represents the apparent UVLF
expected for a population of point-like sources. The blue, red, and brown
lines represent a source with $R_{hl}=35,100,300$ (left) or alternatively with
different choices for the value of $\gamma$ in the L-size relation
$\gamma=0.1,0.25,0.4$, with $R_{0}=0.11"$. We show the regions of magnitude
and volume observable by future surveys, calculated for $z=7$: the order of
magnitude of a typical medium-deep JWST Cycle 1 field (in this case Windhorst
et al. 2022, yellow), the High Latitude Wide Area Survey for the Roman Space
Telescope (green), and the Wide Survey from Euclid (cyan). Since the survey
depths shown here are calculated around $z=7$, the Roman Space Telescope will
see the bright end only for for $z\lesssim 9$. Euclid Wide Survey on the other
hand, is not deep enough in magnitude to see the bright end, assuming galaxies
are extended sources with some L-size relation.

##  4 Conclusions

In this Letter we calculate the effects of source size of bright high-redshift
galaxies and lens ellipticity on gravitational lensing statistics. We show
that these significantly suppress magnification bias, producing an apparent
bright end with a sharper decline than previously considered. Comparing our
model for the observed UVLF to $z=6$ data from Harikane et al. (2022), which
includes a departure from a Schechter profile at the bright end, we show that
the luminosity function can be used to set constraints on the galaxies
intrinsic size. This limited sample favors galaxies that are smaller compared
to other predicted luminosity-size relation parameters values (Kawamata et al.
2018, Liu et al. 2017, Marshall et al. 2019, or the dust-attenuated
$L_{UV}$-size relation in Marshall et al. 2022). However, increasing the
sample size will be necessary to make a reliable measurement.
In the future, the difference in lensed LFs due to finite source size effects
is expected to be measurable by the Roman Space Telescope High Latitude Wide
Area Survey, and would allow to constrain the size of galaxies in the epoch of
reionization.

## Acknowledgments

The authors thank Charlotte Mason for useful suggestions and discussions. This
research was supported by the Australian Research Council Centre of Excellence
for All Sky Astrophysics in 3 Dimensions (ASTRO 3D), through project number
CE170100013.

## Data Availability

The modeled data discussed in this Letter will be shared upon reasonable
request to the corresponding authors.

## References

  * Atek et al. (2018) Atek H., Richard J., Kneib J.-P., Schaerer D., 2018, MNRAS, 479, 5184
  * Bagley et al. (2022) Bagley M. B., et al., 2022, arXiv e-prints, p. arXiv:2205.12980
  * Barkana & Loeb (2000) Barkana R., Loeb A., 2000, ApJ, 531, 613
  * Barone-Nugent et al. (2015) Barone-Nugent R. L., Wyithe J. S. B., Trenti M., Treu T., Oesch P., Bouwens R., Illingworth G. D., Schmidt K. B., 2015, MNRAS, 450, 1224
  * Bouwens et al. (2021) Bouwens R. J., et al., 2021, AJ, 162, 47
  * Bouwens et al. (2022) Bouwens R. J., Illingworth G. D., Ellis R. S., Oesch P. A., Stefanon M., 2022, arXiv e-prints, p. arXiv:2205.11526
  * Bowler et al. (2017) Bowler R. A. A., Dunlop J. S., McLure R. J., McLeod D. J., 2017, MNRAS, 466, 3612
  * Bowler et al. (2022) Bowler R. A. A., Cullen F., McLure R. J., Dunlop J. S., Avison A., 2022, MNRAS, 510, 5088
  * Choi et al. (2007) Choi Y.-Y., Park C., Vogeley M. S., 2007, ApJ, 658, 884
  * Ferrara et al. (2022) Ferrara A., Pallottini A., Dayal P., 2022, arXiv e-prints, p. arXiv:2208.00720
  * Fialkov & Loeb (2015) Fialkov A., Loeb A., 2015, ApJ, 806, 256
  * Finkelstein et al. (2015) Finkelstein S. L., et al., 2015, ApJ, 810, 71
  * Genel et al. (2018) Genel S., et al., 2018, MNRAS, 474, 3976
  * Harikane et al. (2022) Harikane Y., et al., 2022, ApJS, 259, 20
  * Kawamata et al. (2018) Kawamata R., Ishigaki M., Shimasaku K., Oguri M., Ouchi M., Tanigawa S., 2018, ApJ, 855, 4
  * Keeton et al. (1997) Keeton C. R., Kochanek C. S., Seljak U., 1997, ApJ, 482, 604
  * Kormann et al. (1994) Kormann R., Schneider P., Bartelmann M., 1994, A&A, 284, 285
  * Leethochawalit et al. (2022) Leethochawalit N., Roberts-Borsani G., Morishita T., Trenti M., Treu T., 2022, arXiv e-prints, p. arXiv:2205.15388
  * Liu et al. (2017) Liu C., Mutch S. J., Poole G. B., Angel P. W., Duffy A. R., Geil P. M., Mesinger A., Wyithe J. S. B., 2017, MNRAS, 465, 3134
  * Livermore et al. (2017) Livermore R. C., Finkelstein S. L., Lotz J. M., 2017, ApJ, 835, 113
  * Marshall et al. (2019) Marshall M. A., Mutch S. J., Qin Y., Poole G. B., Wyithe J. S. B., 2019, MNRAS, 488, 1941
  * Marshall et al. (2022) Marshall M. A., Wilkins S., Di Matteo T., Roper W. J., Vijayan A. P., Ni Y., Feng Y., Croft R. A. C., 2022, MNRAS, 511, 5475
  * Mason et al. (2015) Mason C. A., et al., 2015, ApJ, 805, 79
  * Mason et al. (2022) Mason C. A., Trenti M., Treu T., 2022, arXiv e-prints, p. arXiv:2207.14808
  * Oguri (2002) Oguri M., 2002, ApJ, 573, 51
  * Pei (1995) Pei Y. C., 1995, ApJ, 440, 485
  * Qin et al. (2017) Qin Y., et al., 2017, MNRAS, 472, 2009
  * Ren & Trenti (2021) Ren K., Trenti M., 2021, ApJ, 923, 110
  * Schechter (1976) Schechter P., 1976, ApJ, 203, 297
  * Schneider & Weiss (1987) Schneider P., Weiss A., 1987, A&A, 171, 49
  * Schneider et al. (1992) Schneider P., Ehlers J., Falco E. E., 1992, Gravitational Lenses, doi:10.1007/978-3-662-03758-4.
  * Steidel et al. (1999) Steidel C. C., Adelberger K. L., Giavalisco M., Dickinson M., Pettini M., 1999, ApJ, 519, 1
  * Turner et al. (1984) Turner E. L., Ostriker J. P., Gott J. R. I., 1984, ApJ, 284, 1
  * Webster et al. (1988) Webster R. L., Hewett P. C., Harding M. E., Wegner G. A., 1988, Nature, 336, 358
  * Windhorst et al. (2022) Windhorst R. A., et al., 2022, arXiv e-prints, p. arXiv:2209.04119
  * Wu (2018) Wu P.-F., 2018, MNRAS, 473, 5468
  * Wyithe & Loeb (2003) Wyithe J. S. B., Loeb A., 2003, ApJ, 595, 614
  * Wyithe et al. (2011) Wyithe J. S. B., Yan H., Windhorst R. A., Mao S., 2011, Nature, 469, 181
  * Yang et al. (2022) Yang L., et al., 2022, ApJ, 938, L17
  * van der Wel et al. (2014) van der Wel A., et al., 2014, ApJ, 792, L6