††institutetext: Département de physique nucléaire et corpusculaire,
Université de Genève, 1211 Genève, Switzerland

# Improving new physics searches with diffusion models for event observables
and jet constituents

Debajyoti Sengupta  Matthew Leigh  John Andrew Raine  Samuel Klein  Tobias
Golling<EMAIL_ADDRESS><EMAIL_ADDRESS><EMAIL_ADDRESS>
###### Abstract

We introduce a new technique called Drapes to enhance the sensitivity in
searches for new physics at the LHC. By training diffusion models on side-band
data, we show how background templates for the signal region can be generated
either directly from noise, or by partially applying the diffusion process to
existing data. In the partial diffusion case, data can be drawn from side-band
regions, with the inverse diffusion performed for new target conditional
values, or from the signal region, preserving the distribution over the
conditional property that defines the signal region. We apply this technique
to the hunt for resonances using the LHCO di-jet dataset, and achieve state-
of-the-art performance for background template generation using high level
input features. We also show how Drapes can be applied to low level inputs
with jet constituents, reducing the model dependence on the choice of input
observables. Using jet constituents we can further improve sensitivity to the
signal process, but observe a loss in performance where the signal
significance before applying any selection is below 4$\sigma$.

##  1 Introduction

The standard model of particle physics is one of the most successful
scientific theories ever, accurately describing the interactions of elementary
particles for three of the fundamental forces of nature in a quantum field
theory. However, it is known to be incomplete as it cannot describe gravity,
provide an explanation for dark matter, or address the (anti)matter imbalance
in the universe, amongst other observed phenomena. At the energy and intensity
frontier, experiments at the Large Hadron Collider Evans:2008zzb  at CERN,
such as the ATLAS and CMS experiments ATLAS:2008xda ; CMS:2008xjf , hunt for
new physics phenomena which could provide an insight into physics beyond the
standard model (BSM).

Dedicated analyses optimised for a wide variety of BSM models have been
performed, but as yet no significant evidence of new physics has been
uncovered ATLAS:2021ilc ; ATLAS:2021zqc ; ATLAS:2021yfa ; CMS:summary1 ;
CMS:summary2 ; CMS:summary3 . Recently, focus has turned to searches with
broader sensitivity in the context of anomaly detection. Instead of aiming for
high sensitivity to a small subset of models, these approaches target
sensitivity to a much wider spectrum of possible new physics scenarios, at the
expense of sensitivity to any one process Kasieczka:2021xcg ;
Aarrestad:2021oeb .

A cornerstone of these model agnostic searches is the bump hunt, which
identifies localised excesses of data above a falling background, typically as
a resonance in an invariant mass. One of the drawbacks to the standard bump
hunt is that the sensitivity is limited to the invariant mass after applying a
selection to define a signal enriched sample. To enhance the sensitivity,
machine learning approaches which improve the sensitivity are studied, in
particular, weakly supervised classifiers (CWoLa) trained to separate signal
region data from a reference background sample cwola ; cwolabump . For this
approach it is important that the reference background sample closely matches
the true background data in the signal region, otherwise the classifier can
learn to separate events based on the background mismodelling, rather than
differences between signal and background data. Several machine learning
methods have been developed for this utilising normalizing flows or
multidimensional sample reweighting anode ; cathode ; salad ; curtains ;
curtainsf4f ; feta ; LaCathode . These methods, however, only form a subset of
the array of anomaly detection techniques being developed in high energy
physics DAgnolo:2018cun ; DAgnolo:2019vbw ; Farina:2018fyg ; Heimel:2018mkt ;
Roy:2019jae ; Cerri:2018anq ; Blance:2019ibf ; Hajer:2018kqm ;
DeSimone:2018efk ; Mullin:2019mmh ; Dillon:2019cqt ; Aguilar-Saavedra:2017rzt
; Romao:2019dvs ; Romao:2020ojy ; Amram:2020ykb ; Cheng:2020dal ;
Khosa:2020qrz ; Thaprasop:2020mzp ; Alexander:2020mbx ; Mikuni:2020qds ;
vanBeekveld:2020txa ; Park:2020pak ; Faroughy:2020gas ; Chakravarti:2021svb ;
Batson:2021agz ; Blance:2021gcs ; Bortolato:2021zic ; Collins:2021nxn ;
Dillon:2021nxw ; Finke:2021sdf ; Shih:2021kbt ; Atkinson:2021nlt ;
Kahn:2021drv ; Dorigo:2021iyy ; Caron:2021wmq ; Govorkova:2021hqu ;
Kasieczka:2021tew ; Volkovich:2021txe ; Govorkova:2021utb ; Ostdiek:2021bem ;
Fraser:2021lxm ; Jawahar:2021vyu ; Herrero-Garcia:2021goa ; Aguilar-
Saavedra:2021utu ; Tombs:2021wae ; Lester:2021aks ; Mikuni:2021nwn ;
Chekanov:2021pus ; dAgnolo:2021aun ; Canelli:2021aps ; Ngairangbam:2021yma ;
Bradshaw:2022qev ; Aguilar-Saavedra:2022ejy ; Buss:2022lxw ; Alvi:2022fkk ;
Dillon:2022tmm ; Birman:2022xzu ; Letizia:2022xbe ; Fanelli:2022xwl ;
Finke:2022lsu ; Verheyen:2022tov ; Dillon:2022mkq ; Caron:2022wrw ;
Park:2022zov ; Kamenik:2022qxs ; Kasieczka:2022naq ; Araz:2022zxk ;
Schuhmacher:2023pro ; Roche:2023int ; Vaslin:2023lig ; ATLAS:2023azi ;
Chekanov:2023uot ; CMSECAL:2023fvz ; Bickendorf:2023nej ; Freytsis:2023cjr ;
Metodiev:2023izu .

Recently, diffusion models Song2020 ; Karras2022 ; yang2023diffusion  and
continuous normalizing flows chen2019neural_CNF ; liu2022flow_flowmatching1 ;
albergo2023building_flowmatching2 ; lipman2023flow_flowmatching3  have
demonstrated exceptional performance in high energy physics applications
CaloScore ; pcjedi ; fpcd ; Shmakov:2023kjj ; Butter:2023fov ; pcdroid ; glam
; Mikuni:2023tqg ; Buhmann:2023kdg ; epicjedi ; Buhmann:2023acn ;
Heimel:2023mvw ; Devlin:2023jzp ; Heimel:2023ngj ; Butter:2023ira ;
Birk:2023efj , notably for reproducing the underlying kinematics of both
calorimeter and particle showers. These models have outperformed previous
state-of-the-art approaches using normalizing flows and generative adversarial
networks in both tasks.

In this work we introduce a new method, Drapes (Denoising resonant anomalies
by perturbing existing samples), which leverages state of the art diffusion
models to generate reference background samples in a fully data driven
approach for resonant anomaly searches. After training Drapes on side-band
data, reference background samples can be generated in the signal region
either by sampling from noise, or by partially applying the forward and
reverse diffusion processes on existing data drawn from either the side-bands,
the signal region, or another reference sample. We apply Drapes to the LHCO
R&D dataset LHCOlympics  and demonstrate state-of-the art performance in
training CWoLa classifiers for a range of signal fractions. Due to the
flexibility of diffusion models, we can apply Drapes to both high level
features and low level jet constituents.

This work represents one of the first applications of diffusion models111Here
we can also include continuous normalizing flows. See Ref. epicjedi  for a
comparison with diffusion models, which introduces the umbrella term
continuous time generative models. for the task of sample generation in weakly
supervised searches for new physics. Concurrent with the development of this
work a similar approach was developed, applying diffusion models to di-jet
generation for weakly supervised searches using low level jet constituents
Buhmann:2023acn . However, in addition to low level generation, our work
studies the application to high level variables as well as the first
application of partial diffusion in high energy physics. Other previous work
applying diffusion models for anomaly detection has seen a diffusion model
trained as an unsupervised tagger for identifying anomalous jets
Mikuni:2023tok .

##  2 Dataset

The LHCO R&D dataset LHCOlympics  has become a standard candle of performance
for resonant anomaly detection approaches. It comprises background data from
di-jet events produced in QCD interactions, and signal data from the all-
hadronic decay of a massive resonant BSM particle into two lighter BSM
particles. The masses of the three BSM particles in the signal events are
$m_{W^{\prime}}=3.5$ TeV, $m_{X}=500$ GeV, and $m_{Y}=100$ GeV.

Signal and background events are generated with Pythia 8.219 Sjostrand:2014zea
with detector simulation performed with Delphes 3.4.1 deFavereau:2013fsa .
Large radius jets, with a radius parameter $R=1.0$, are reconstructed using
the anti-$k_{t}$ clustering algorithm AntiKt  using the FastJet
Cacciari:2011ma  package. There are in total 1 million background and 100,000
signal events.

The LHCO Black Box 2 sample, which also comprises 1 million events, is used as
an alternative background sample. The generation chain follows the same
prescription, except Herwig++ Bahr:2008pv  is used to generate the events,
with a slightly modified detector parametrisation.

Events are required to have exactly two jets, where at least one jet has
$p_{\mathrm{T}}^{J}>1.2$ TeV, and are ordered by decreasing mass. In order to
compare Drapes to other methods, we use the same high level input features as
in Refs. curtains ; LaCathode ; feta ; interplay , namely

$m_{JJ},\,m_{J_{1}},\,\Delta
m_{J}=m_{J_{1}}-m_{J_{2}},\,\tau_{21}^{J_{1}},\,\tau_{21}^{J_{2}},\,\Delta
R_{JJ}=\sqrt{\Delta\eta^{2}+\Delta\phi^{2}}.$

Furthermore, with diffusion models it is possible to operate on input sets,
such as the constituent particles within a jet, with permutation equivariance.
As such, the model can be trained without enforcing any order on the
constituents within the jet. Therefore, in addition to the default feature
set, we also consider the leading 128 constituents in each jet. They are
represented by their transverse momentum as a fraction of the jet transverse
momentum and their coordinates in $\eta$-$\phi$ space relative to the central
axis of the jet ($p_{\mathrm{T}}/p_{\mathrm{T}}^{J}$, $\Delta\eta$,
$\Delta\phi$).

The background data follow a falling spectrum in $m_{JJ}$ above 2.7 TeV, with
signal events localised as a resonance in $m_{JJ}$ with a peak at 3.5 TeV
(corresponding to $m_{W^{\prime}}$), and mostly contained within a signal
window $m_{JJ}\in[3300,3700)$, which we define as our signal region (SR). To
improve the statistical accuracy when evaluating model performance, additional
background events falling within the SR are also available for the high level
input features david_shih_2021_5759087 .

##  3 Method

In Drapes, we train diffusion models to learn a conditional generative model
of the data from side-bands (SB) surrounding our signal region. The model is
conditioned on $m_{JJ}$ and, under the assumption that the side-bands contain
only background data, can be assumed to be a generative model for the
background process across the full $m_{JJ}$ range, including the SR. The
Drapes models are trained on either the standard high level variables or the
jet constituents of the two jets in the event.

For a nominal choice of side-bands we consider the region
$m_{JJ}\in[2800,6000)$, excluding the signal region. Separate Drapes models
are trained for a wide range of injected signal levels.

###  3.1 Diffusion model

We follow the ‘EDM’ noise scheduler and network pre-conditioning introduced in
Ref. Karras2022 . Here, the probability ordinary differential equation
defining the reverse diffusion process is given by

$\mathop{}\\!\mathrm{d}x_{t}=-t\nabla_{x}\log p(x;t)\mathop{}\\!\mathrm{d}t,$

where $p(x;t)$ is the distribution over the data at time $t$, where
$t=\sigma\in[0,80)$ (where $\sigma$ is referred to as the noise strength).

For high level input variables we employ a simple ResNet resnet  architecture
comprising four hidden layers, each with 64 nodes, for the diffusion network.
The noise strength $\sigma$ is processed with a cosine embedding layer before
being concatenated alongside $m_{JJ}$ as the conditional vector. Skip and
scaling connections, $c_{\mathrm{in}}(\sigma)$, $c_{\mathrm{out}}(\sigma)$ and
$c_{\mathrm{skip}}(\sigma)$ are used in conjunction with the network,
resulting in an output of

$D_{\theta}(x;\sigma,m_{JJ})=c_{\mathrm{out}}(\sigma)F_{\theta}(c_{\mathrm{in}}(\sigma)x;\sigma,m_{JJ})+c_{\mathrm{skip}}(\sigma)x.$

This results in a training objective of

$\mathcal{L}=||D_{\theta}(x;\sigma,m_{JJ})-x_{0}||^{2}_{2},$

which is simply the distance between the network output and the true target,
namely the original noised input data, $x_{0}$. During training, we sample the
noise rate following a shifted log-normal distribution
$\log(\sigma)\sim\mathcal{N}\left(-1.2,1.2\right)$, and all inputs are scaled
to have a mean of zero and unit variance. The Drapes models are each trained
for 700 epochs222Convergence is achieved after far fewer epochs, within an
hour of training. with a learning rate rate of $10^{-4}$, with a linear warmup
from zero over the first 50,000 training steps. A schematic overview of the
network architecture during training for the case of high level features is
shown in Fig. 1.

Figure 1: Schematic overview of the Drapes architecture during training when
using high level variables. Data are drawn from the side-bands of the $m_{JJ}$
distribution, and their features are passed through the diffusion network
$F_{\theta}$ after being perturbed with noise with strength $\sigma$. The
network is conditioned on $m_{JJ}$ and $\sigma$. Skip connections modify the
input and output of the network.

###  3.2 Background sample generation

Considering a single Drapes model, there are several options which can be used
to generate reference background samples by performing the reverse diffusion
process.

It is possible to use Drapes as a purely conditional generative model,
generating new data for target values of $m_{JJ}$. To generate new samples
noise is drawn from a multidimensional gaussian distribution, and the reverse
diffusion process is performed starting at $\sigma=80$. We refer to this
method as Drapes $\emptyset$. This generation method can be considered a
diffusion analogue to the Cathode approach cathode , where samples are
generated for target $m_{JJ}$ values with a normalizing flow starting from
noise.

As with all diffusion models, the solver plays a key role in the fidelity and
generation speed of samples. Here we find performance is saturated with 50
integration steps, with no discernable difference between the Heun and Euler
ordinary differential equation solvers. Nonetheless, to ensure that the choice
of solver plays a subleading role, we choose to use the Heun 2nd solver with
200 integration steps at $\sigma=80$ when generating all samples.

However, in addition to generating from pure noise, it is also possible to
generate a reference background template by taking existing samples and using
them as the inputs for partial diffusion. In partial diffusion, the forward
diffusion process is performed by adding noise to the samples corresponding to
a target noise strength $\sigma^{\prime}$. Then, the reverse diffusion process
is performed for a target value $m_{JJ}$, starting at
$\sigma=\sigma^{\prime}$. In this case, fewer passes are required to recover
the output samples, as the diffusion process is starting partway along the
trajectory. The number of diffusion steps $N$ and $\sigma^{\prime}$ are
related333As $N$ is required to be an integer, we instead use the inverse of
the equation to determine the values of $\sigma^{\prime}$ which can be used.
by

$N=N_{\text{max}}\cdot{\sigma^{\prime}}^{\frac{1}{\rho}}-\frac{\sigma_{\text{max}}^{\frac{1}{\rho}}}{\sigma_{\text{\vphantom{max}min}}^{\frac{1}{\rho}}-\sigma_{\text{\vphantom{min}max}}^{\frac{1}{\rho}}}$
(1)

where $N_{\text{max}}=200$ and $\sigma_{max}=\sigma=80$ is the total number of
reverse diffusion steps and corresponding noise strength during training. Here
$\sigma_{\text{min}}=10^{-5}$ and $\rho=7$ are hyperparameters of the
diffusion model.

We consider three different approaches for partial diffusion, which differ
based on the source of the initial data.

Drapes SR

Here, partial diffusion is applied to the data in the signal region with a
target noise strength $\sigma^{\prime}$. They are subsequently denoised using
the reverse diffusion process using the original $m_{JJ}$ values.

Drapes SB

Here, data are drawn from the side-bands and partial diffusion is applied for
a target $m_{JJ}$ value which lies in the SR, rather than their initial
$m_{JJ}$ value. This case most closely resembles the Curtains method which
uses normalizing flows curtains ; curtainsf4f  to transform sideband data to
target values of $m_{JJ}$ within the signal region.

Drapes MC

Here partial diffusion is applied to data in the signal region which are drawn
from another reference sample, for example Monte Carlo simulation. Here we use
the alternative background sample from the LHC Olympics. This case most
closely resembles the Feta approach feta  which uses normalizing flows to
transform signal region data from another sample to the signal region of the
nominal sample.

For the case of Drapes $\emptyset$ and Drapes SB, we sample target values of
$m_{JJ}$ from a four-parameter polynomial

$f(z)=p_{1}\left(1-z\right)^{p_{2}}z^{p_{3}+p_{4}\log z},$ (2)

where $z=m_{JJ}/\sqrt{s}$. The parameters $p_{i}$ are extracted by performing
a fit to the side-band data.

In all cases it is possible to oversample the reference background sample by
generating more data than are present in the SR. For Drapes SR and Drapes MC
this is done by sampling different noise in the forward diffusion process
multiple times for the same events. For Drapes $\emptyset$ and Drapes SB, both
the noise in the forward diffusion process and target $m_{JJ}$ values are
sampled independently multiple times for each event.

###  3.3 Drapes for jet constituents

Due to the flexibility of diffusion models in comparison to normalizing flows,
Drapes can easily be adapted to any type of input. To apply Drapes to low
level jet constituents, we generate the constituents of each jet in isolation
using the same diffusion model. The diffusion model builds upon the
architecture and prescription used in Ref. pcdroid  with the denoising
architecture built from a series of transformer encoder (TE) blocks
vaswani2023attention , without a further optimisation of the hyperparameters.
The jet four-momentum and number of jet constituents
($p_{4}=({p}_{\mathrm{T}},\eta,\phi,m_{j},N_{const})$)444For notational
convenience we include $N_{const}$ in the definition of the four-vector. are
provided as conditional information. However, in order to obtain the
conditional information $p_{4}$ for each jet, we require a second diffusion
model conditional on $m_{JJ}$. This network is identical to the Drapes
architecture used for the high level features. The two networks comprising the
low level Drapes model are shown in Fig. 2.

Figure 2: Schematic overview of the Drapes architecture during training when
applied to jet constituents. The approach comprises two components, the di-jet
kinematic model (left) and the per-jet model (right). Data are drawn from the
side-bands of the $m_{JJ}$ distribution. To learn the di-jet kinematics as a
function of $m_{JJ}$, the four-momentum and number of jet constituents for
each jet $p_{4}$ are passed through the kinematic diffusion network $G_{\phi}$
after being perturbed with noise with strength $\sigma$. The network is
conditioned on $m_{JJ}$ and $\sigma$. To learn the jet constituents as a
function of $p_{4}$, the constituents of each jet are passed through the jet
diffusion network $H_{\psi}$ after being perturbed with noise with strength
$\sigma$. The network is conditioned on $p_{4}$ and $\sigma$. In both networks
skip connections modify the input and output of the network, and partial
diffusion can also be employed.

In background QCD events we do not expect significant correlations between the
substructure of the two jets, which are produced as either two gluons, two
quarks, or one of each, predominantly via t-channel.555At the LHC, both pairs
of gluons and pairs of quarks are more likely to be produced than a gluon-
quark pair. However, due to the similarity between quarks and gluons as well
as contributions from additional radiation and next to leading order
production mechanisms, we expect this to be a subleading effect, especially in
comparison to any potential resonant signal. However, for any resonant signal
process the two daughter particles will be related and their substructure will
be strongly correlated. By applying Drapes to each jet in isolation, in the
case where there are signal events in the sidebands the substructure of signal
jets could be learned, but the correlations between the substructure of the
two jets would not be. Alternatively, if Drapes were applied to the
constituents in both jets simultaneously, the correlations between the
substructure of signal jets would be learned. As a result, when generating
reference templates for the signal region, considering each jet independently
should have fewer signal like events, where both jets have signal like
substructure, resulting in better sensitivity than when the two jets are not
independent.

Even in the case where there are no signal events present in the sidebands,
applying Drapes to each jet in isolation should still be beneficial for
partial diffusion. After the addition of noise and applying independent
denoising processes to each jet, the correlations between the substructure of
two signal jets in an event would be decreased.

###  3.4 Weakly-supervised classifiers

To test the performance of the various approaches, we train CWoLa classifiers
to separate the signal region data from the generated templates. The SR data
can either be purely background events, or contain a controlled number of
signal data, which can also contaminate the side-bands. As the classifiers
themselves are not the focus on this work, we use the same hyperparameters and
architecture used in Refs. curtains ; curtainsf4f  without further
optimisation for the high level features, despite promising performance
improvements observed with decision trees Finke:2023ltw ; Freytsis:2023cjr  or
for mitigating sculpting after applying cuts LaCathode . This enables easier
direct comparison with CurtainsF4F. A five-fold cross validation approach is
employed to train all classifiers, including the fully supervised
configuration.

Following the CWoLa classifiers for the high level features, we keep the
architecture small also for the low level approach with Drapes applied to
generate jet constituents. We use a classifier which is permutation invariant
to both the jet constituents, but also the two jets. To achieve this we use
transformer encoders operating on the jet constituents with self-attention
vaswani2023attention  layers. The output is then pooled with a learnable class
token cross-attention touvron2021going  layer, weighting the contribution from
each constituent; this ensures permutation invariance over the jet
constituents. The same transformer is applied to the constituents of both
jets, with the outputs combined in a summation in order to ensuring
permutation invariance between the two jets. The resulting vector is
subsequently processed with a small multi-layer perceptron.

Figure 3: Schematic overview of the weakly supervised classifier for the jet
constituents. The same transformer network is applied to the constituents of
both jets, with the outputs combined in a summation in order to ensuring
permutation invariance between the two jets.

###  3.5 Comparison with prior work

As previously discussed, various modes of sample generation with Drapes are
analogous to existing approaches but using diffusion models rather than
normalizing flows. However, during the development of this work Ref.
Buhmann:2023acn  applied flow matching and diffusion models to the task of
dijet generation in a signal region. This approach is equivalent to Drapes
$\emptyset$ applied to the low level jet constituents, however, this work
differentiates itself in several key regards with respect to Ref.
Buhmann:2023acn .

With Drapes, we introduce and study the use of partial diffusion as a new
means of generating samples, rather than generating only from noise as with
Drapes $\emptyset$. Furthermore, we demonstrate that diffusion models can
outperform normalizing flow based approaches on the same features, and that
there are advantages beyond the more flexible choice of architecture that
enables jet constituent generation. We compare the performance of Drapes
applied to high and low level variables across a wide range of injected signal
events, and make comparisons for the case of partial diffusion for both sets
of inputs.

##  4 Results

In order to measure how well Drapes generates reference background samples, we
focus on a single signal region defined as $m_{JJ}\in[3300,3700)$ with various
levels of signal contamination. The main measure of performance of the
subsequent classifier is the significance improvement, given by
${}^{\epsilon_{S}}\\!/_{\sqrt{\epsilon_{B}}}$, where $\epsilon_{S}$ and
$\epsilon_{B}$ are the signal and background efficiencies respectively.
Similar to $\epsilon_{B}$ in a receiver operator characteristic (ROC) curve,
we calculate the significance improvement across the full range of background
rejection values (${}^{1}\\!/_{\epsilon_{B}}$).

We compare the performance of Drapes in several scenarios to a CurtainsF4F
model trained using the full side-band width, which was shown in Ref.
curtainsf4f  to deliver state of the art performance. For reference we also
show the performance of a supervised classifier trained to separate the signal
process from the QCD background on an independent set of data, and an
idealised CWoLa classifier. In the idealised classifier, simulated QCD
background events are used as the reference background sample, with
oversampling equal to the amount used in Drapes and CurtainsF4F (Over-
Idealised).

###  4.1 Simple diffusion

As it is the simplest approach, we first look at the performance of Drapes
$\emptyset$. In Fig. 4 we look at the performance of the CWoLa classifiers for
two levels of signal injection. Where 1,000 signal events are present in the
data, we see that Drapes $\emptyset$ achieves state-of-the-art performance as
a purely generative model. Here, Drapes $\emptyset$ has higher significance
improvement values than the Over-Idealised classifier only due to the
different profile of the ROC curve. The Over-Idealised classifier has greater
separation power, with an AUC of 0.89 in comparison to 0.85 with Drapes
$\emptyset$. Similarly we see that Drapes $\emptyset$ reaches state-of-the-art
performance where 3,000 signal events have been injected. However, we note
here that both Drapes $\emptyset$ and CurtainsF4F reach saturation in
significance improvement at high levels of background rejection, achieving the
same level of performance of the supervised classifier.

Figure 4: Background rejection as a function of signal efficiency (left) and
significance improvement as a function of background rejection (right) for
Drapes $\emptyset$ (purple), CurtainsF4F (orange), Supervised (black), and
Over-Idealised (green). All methods are trained on the sample with 1,000 (top)
and 3,000 (bottom) injected signal events, and a signal region $3300\leq
m_{JJ}<3700$ GeV. The lines show the mean value of fifty independent
classifiers, with the shaded band representing a 68% uncertainty.

In Fig. 5 we look at the performance of Drapes $\emptyset$ as the number of
signal events present in the data changes. In comparison to CurtainsF4F we see
that Drapes $\emptyset$ is able to remain more sensitive at lower
${}^{S}\\!/_{B}$. However, Drapes $\emptyset$ drops off in performance below
injected significances of 1$\sigma$ at background rejections of $10^{3}$,
where fewer than 350 signal events are present in the signal region. This is
just before the Over-Idealised classifier indicates the limit of the CWoLa
weakly supervised approach. The same trend is observed for a background
rejection factor of $5\times 10^{3}$.

Figure 5: Significance improvement at fixed background rejection rates of
$10^{3}$ (left) and $5\times 10^{3}$ (right) as a function of the number of
signal events in the signal region, $3300\leq m_{JJ}<3700$ GeV, for Drapes
$\emptyset$ (purple), CurtainsF4F (orange), Supervised (black), and Over-
Idealised (green). The injected significance measured from the number of
signal and background events in the signal region before applying a cut
$\frac{S}{\sqrt{B}}$ is also shown. The lines show the mean value of fifty
independent classifiers, with the shaded band representing a 68% uncertainty.

###  4.2 Partial diffusion

When using Drapes with performing partial diffusion, it is critical that the
generated reference distributions still match the true background
distribution. For example, when starting from side-band data with a value of
$\sigma^{\prime}$ too close to zero, the overall distributions will not change
and a CWoLa classifier will identify the differences between the two
distributions arising from the correlations to $m_{JJ}$ instead of differences
between the signal present in one class, and the approximation of the
background data in the other. Similarly, when starting from the signal region
data themselves, if $\sigma^{\prime}$ is too small, the samples will be
statistically identical and the classifier will not be able to separate them,
leading to a classifier unable to separate signal events from the background.

We measure the performance of the partial diffusion generation as we change
$\sigma^{\prime}$ by training Drapes on a sample with no signal present. By
generating background reference templates for a wide range of
$\sigma^{\prime}$ we can then test how close they match the true background by
training a classifier to separate them. If the area under the ROC curve of the
classifier is close to 0.5, then the reference samples are good approximations
of the background. However, if the values are too large, a CWoLa classifier
would instead focus on this mismodelling of the background.

In addition, we compare the performance of Drapes $\emptyset$ with partial
diffusion, starting with data drawn from noise but with fewer denoising steps
and the same values of $\sigma^{\prime}$. The AUC scores for all four Drapes
methods are compared in Fig. 6. We can see that all three methods starting
from reference data, Drapes SR, Drapes SB, and Drapes MC, result in reference
samples much closer to the true background distribution than when starting
from noise with Drapes $\emptyset$. This shows that there is a benefit for
starting with some data rather than just pure noise, but only up until a
point, as after some value of $\sigma^{\prime}$ the input data are almost
indistinguishable from noise. By construction, Drapes SR is closest to the
true background distribution as they are used as the starting samples. For
Drapes SR and Drapes SB $\sigma^{\prime}$ values above 0.62 result in
reference templates with AUC values less than 0.55, which are generally close
enough to result in reasonable CWoLa classifiers in the presence of signal.

Figure 6: AUC values of classifiers trained to separate the Drapes reference
samples from data in the case where no signal has been injected. The AUC is
calculated as a function of the initial $\sigma^{\prime}$ used to generate
samples using Drapes SR (blue), Drapes MC (magenta) and Drapes SB (red). By
construction, all methods converge to the same outputs as $\sigma^{\prime}$
increases, and at a value of $\sigma^{\prime}=80$ all methods are equivalent
when using the same $m_{JJ}$ values. An AUC of 0.505 is observed at the
convergence of the methods. The lines show the mean value of fifty independent
classifiers, with the shaded band representing a 68% uncertainty, which are
negligible at this scale.

In Fig. 7 the performance of the partial diffusion approaches are compared as
we vary $\sigma^{\prime}$ in the case of 1,000 and 3,000 injected signal
events. As expected, in the case of fewer signal events the overall
performance is more sensitive to the modelling of the background reference
template. For 1,000 signal events we see that both Drapes SR and Drapes SB
maintain competitive performance for values of $\sigma^{\prime}$ down to 0.6,
where the AUC scores are also below 0.55, however for lower values the
performance degrades. Interestingly, Drapes MC has lower overall performance
and requires more denoising steps to reach competitive performance. Drapes SR
and Drapes SB have similar performance for 1,000 injected signal events,
though Drapes SR can maintain performance with far fewer steps in the presence
of more signal in the SR.

Figure 7: Significance improvement as a function of noise rate for Drapes SR
(red), Drapes SB (blue), and Drapes MC (magenta). All generation methods use
the same diffusion model trained on the sample with 1,000 (top) and 3,000
(bottom) injected signal events, and a signal region $3300\leq m_{JJ}<3700$
GeV. Reference values for Drapes $\emptyset$ (purple), Supervised (black) and
Over-Idealised (green) are shown in addition. The lines show the mean value of
fifty independent classifiers, with the shaded band representing a 68%
uncertainty.

In order not to optimise for $\sigma^{\prime}$ for all amounts of signal and
subsequent studies we choose a single value for $\sigma^{\prime}$ using the
AUC in the no signal case shown in Fig. 6. Here we see that the agreement of
the partial diffusion approaches with true background data saturates at around
$\sigma^{\prime}=2.24$. This also corresponds to where the performance has
saturated for all three methods at the case of 1,000 signal events in Fig. 7.
It should be noted that that this may not be the most optimal choice for all
methods at low signal levels.

Figure 8: Significance improvement at fixed background rejection rates of
$10^{3}$ (left) and $5\times 10^{3}$ (right) as a function of the number of
signal events in the signal region, $3300\leq m_{JJ}<3700$ GeV, for
CurtainsF4F (orange), Drapes $\emptyset$ (purple), Drapes SR (red), Drapes SB
(blue), Drapes MC (magenta), Supervised (black), and Over-Idealised (green).
The injected significance measured from the number of signal and background
events in the signal region before applying a cut $\frac{S}{\sqrt{B}}$ is also
shown. The lines show the mean value of fifty independent classifiers, with
the shaded band representing a 68% uncertainty. A dashed line is shown at 1 to
indicate where an improvement over the initial significance is achieved.

In Fig. 8 the significance performance at fixed background rejection rates is
compared to Drapes $\emptyset$ for a range of signal values. Here we see that
Drapes SR comes closest to the overall performance of Drapes $\emptyset$, with
all methods becoming more similar as the injected ${}^{S}\\!/_{\sqrt{B}}$
increases, though Drapes $\emptyset$ performs best for all values. However,
Drapes SR and Drapes SB remain competitive, with less than half the number of
denoising steps as Drapes $\emptyset$.

###  4.3 Drapes with jet constituents

When using low level inputs (LLV) we focus on the two best drapes generation
approaches, Drapes $\emptyset$ and Drapes SR. We compare the performance to
Drapes trained on high level variables (HLV) and a supervised classifier
trained on the same low level inputs.

Figure 9: Background rejection as a function of signal efficiency (left) and
significance improvement as a function of background rejection (right) for
Drapes $\emptyset$ (purple) and Drapes SR (red). Models trained on the low
level jet constituents are shown in a solid line (LLV), with models using the
high level variables (HLV) are shown in a dashed line. All methods are trained
on the sample with 3,000 injected signal events, and a signal region $3300\leq
m_{JJ}<3700$ GeV. The lines show the mean value of fifty independent
classifiers, with the shaded band representing a 68% uncertainty. A supervised
classifier (black) trained on low level jet constituents is shown for
reference.

Fig. 9 shows the performance for the case of 3,000 injected signal events. We
can see that Drapes trained on low level inputs substantially outperforms the
high level features. Here Drapes $\emptyset$ again results in overall better
performance compared to partial diffusion, with Drapes SR suffering a non-
negligible performance reduction not observed for HLV. This is most notable in
the ROC curve, where at high signal efficiencies Drapes SR LLV has
significantly lower background rejection.

Figure 10: Impact on performance of varying the level of partial diffusion
$\sigma^{\prime}$ in the case where there are no signal events (left) and
3,000 injected signal events (right). For the no signal case we show the AUC
values of classifiers trained to separate the Drapes SR reference samples from
true background data. For the case where signal events are present we show the
significance improvement at a background rejection of 5,000. Models trained on
the low level jet constituents are shown in a solid line (LLV), with models
using the high level variables (HLV) are shown in a dashed line. All methods
are trained on the sample with 3,000 injected signal events, and a signal
region $3300\leq m_{JJ}<3700$ GeV. The lines show the mean value of fifty
independent classifiers, with the shaded band representing a 68% uncertainty.
For the significance improvement, a supervised classifier (black) trained on
low level jet constituents is shown for reference.

For Drapes SR a value of $\sigma^{\prime}=8.9$ is used for LLV with
$\sigma^{\prime}=0.62$ used for HLV as before. The motivation for this can be
seen in Fig. 10, where Drapes SR has little to no sensitivity below
$\sigma^{\prime}=2$. This suggests that the partial diffusion approaches have
less direct benefit when applied to LLV in comparison to HLV. However, this
could be strongly dependent on how Drapes SR is applied when using the jet
constituents. In this work, the jet kinematics used to condition the Drapes
model are unmodified from the signal region data, and partial diffusion is
only applied to the jet constituents. Another option, which has not been
studied in this work, would be to perform partial diffusion on both $p_{4}$
and the jet constituents. This could improve the sensitivity of the partial
diffusion method, and would be necessary for using partial diffusion with
Drapes SB.

The performance of Drapes $\emptyset$ (LLV) as the amount of signal present
changes is shown in Fig. 11. Here we can see that although the performance
achieved by Drapes $\emptyset$ LLV substantially outperforms HLV where more
signal is present, the performance drops off below 2,000 injected signal
events. Drapes $\emptyset$ trained on HLV outperforms the jet constituent
approach in the region of interest where the injected significance is lower
than 3. This observation is consistent with what is observed in Ref.
Buhmann:2023acn . This drop in performance, however, can be expected and is
reproducible in the idealised setting. CWoLa classifiers based on neural
networks have been observed to lose sensitivity as the dimensionality of the
input increases, especially with inputs which are not sensitive to signal
processes LaCathode ; Finke:2023ltw ; Freytsis:2023cjr . This effect is
amplified as the number of signal events in the training data decreases. In
moving from five high level variables to jet constituents with Drapes, the
dimensionality of the input to the weakly supervised neural networks has
increased by more than a factor of 150.

Figure 11: Significance improvement at fixed background rejection rates of
$10^{3}$ (left) and $5\times 10^{3}$ (right) as a function of the number of
signal events in the signal region, $3300\leq m_{JJ}<3700$ GeV, for Drapes
$\emptyset$ (purple) using low level jet constituents (LLV, solid line) or
high level variables (HLV, dashed line). The injected significance measured
from the number of signal and background events in the signal region before
applying a cut $(\frac{S}{\sqrt{B}})$ is also shown. The lines show the mean
value of fifty independent classifiers, with the shaded band representing a
68% uncertainty. A supervised classifier (black) trained on low level jet
constituents is shown for reference.

##  5 Conclusion

In this work we have introduced Drapes, a new approach for generating
background reference samples for use in resonant anomaly searches, using
diffusion models with deterministic samplers. This approach can be used on
both high level features as well as low level jet constituents.

For high level features, state-of-the-art performance is achieved on the LHCO
R&D dataset when using Drapes as a purely data-driven generative model. Even
better performance could be achieved by combining the generated background
templates with boosted decision trees, rather than neural networks, for the
CWoLa classifier.

Furthermore, we demonstrate an advantage of the flexibility of diffusion
models by applying Drapes to the generation of low level jet constituents.
This approach is more model agnostic than using high level variables, as it
does not impose a choice of observables which may be sensitive to new physics
processes. Substantial improvements are observed for many levels of signal
events. However, for initial levels of signal significance below $4\sigma$ the
performance degrades and is outperformed by the high level features.

In addition to pure template generation, we demonstrate the first application
of partial diffusion in high energy physics. Partial diffusion is a promising
approach to generate new samples from an initial reference dataset, however on
this dataset it does not reach the same level of performance as Drapes
$\emptyset$. This is likely due to the nature of the application in high
energy physics. In other fields, anomalies can be characterised as single-
sample out-of-distribution events. In this case, distance based metrics
between the original sample and the partially diffused sample can be used as
an anomaly score. However, in the case of resonant anomalies, we are searching
for over-densities in feature space, localised in one observable ($m_{JJ}$).
In this case, what is important is that we compare the differences in
densities of our datasets. With partial diffusion, if the initial samples are
all in distribution, by performing the forward and reverse diffusion process,
the initial density of the initial samples will be preserved. This approach
could still have merit when combined with another anomaly metric than the
classifier based approaches studied here. For example exploiting the
displacement of samples in the signal region with Drapes SB, or assigning
labels during training to side-band and signal region data which can be
reversed at inference.

## Acknowledgements

The authors would like to acknowledge funding through the SNSF Sinergia grant
CRSII5_193716 “Robust Deep Density Models for High-Energy Particle Physics and
Solar Flare Analysis (RODEM)” and the SNSF project grant 200020_212127 “At the
two upgrade frontiers: machine learning and the ITk Pixel detector”. They
would also like to acknowledge the funding from the Swiss Government
Excellence Scholarships for Foreign Scholars.

## References

  * (1) L. Evans and P. Bryant, _LHC Machine_ , _JINST_ 3 (2008) S08001.
  * (2) ATLAS collaboration, _The ATLAS Experiment at the CERN Large Hadron Collider_ , _JINST_ 3 (2008) S08003.
  * (3) CMS collaboration, _The CMS Experiment at the CERN LHC_ , _JINST_ 3 (2008) S08004.
  * (4) ATLAS Collaboration, _SUSY Summary Plots June 2021_ , Jun, 2021.  Available at https://cds.cern.ch/record/2771785, accessed 29th August 2023.
  * (5) ATLAS Collaboration, _Summary Plots from ATLAS Searches for Pair-Produced Leptoquarks_ , Jun, 2021.  Available at https://cds.cern.ch/record/2771726, accessed 29th August 2023.
  * (6) ATLAS Collaboration, _Summary Plots for Heavy Particle Searches and Long-lived Particle Searches - July 2021_ , Jul, 2021.  Available at https://cds.cern.ch/record/2777015, accessed 29th August 2023.
  * (7) CMS Collaboration, _CMS EXO sumary plots at 13TeV_ , 2022.  Accessed Available at https://twiki.cern.ch/twiki/bin/view/CMSPublic/SummaryPlotsEXO13TeV, accessed 29th August 2023.
  * (8) CMS Collaboration, _CMS B2G physics results summary_ , 2022.  Available at https://twiki.cern.ch/twiki/bin/view/CMSPublic/PhysicsResultsB2G, accessed 29th August 2023.
  * (9) CMS Collaboration, _CMS SUS physics results summary_ , 2022.  Available at https://twiki.cern.ch/twiki/bin/view/CMSPublic/PhysicsResultsSUS, accessed 29th August 2023.
  * (10) G. Kasieczka et al., _The LHC Olympics 2020 a community challenge for anomaly detection in high energy physics_ , _Rept. Prog. Phys._ 84 (2021) 124201 [2101.08320].
  * (11) T. Aarrestad et al., _The Dark Machines Anomaly Score Challenge: Benchmark Data and Model Independent Event Classification for the Large Hadron Collider_ , _SciPost Phys._ 12 (2022) 043 [2105.14027].
  * (12) E.M. Metodiev, B. Nachman and J. Thaler, _Classification without labels: Learning from mixed samples in high energy physics_ , _JHEP_ 10 (2017) 174 [1708.02949].
  * (13) J.H. Collins, K. Howe and B. Nachman, _Extending the search for new resonances with machine learning_ , _Phys. Rev. D_ 99 (2019) 014038 [1902.02634].
  * (14) B. Nachman and D. Shih, _Anomaly Detection with Density Estimation_ , _Phys. Rev. D_ 101 (2020) 075042 [2001.04990].
  * (15) A. Hallin, J. Isaacson, G. Kasieczka, C. Krause, B. Nachman, T. Quadfasel et al., _Classifying anomalies through outer density estimation_ , _Phys. Rev. D_ 106 (2022) 055006.
  * (16) A. Andreassen, B. Nachman and D. Shih, _Simulation Assisted Likelihood-free Anomaly Detection_ , _Phys. Rev. D_ 101 (2020) 095004 [2001.05001].
  * (17) J.A. Raine, S. Klein, D. Sengupta and T. Golling, _CURTAINs for your sliding window: Constructing unobserved regions by transforming adjacent intervals_ , _Front. Big Data_ 6 (2023) 899345 [2203.09470].
  * (18) D. Sengupta, S. Klein, J.A. Raine and T. Golling, _CURTAINs Flows For Flows: Constructing Unobserved Regions with Maximum Likelihood Estimation_ , 2305.04646.
  * (19) T. Golling, S. Klein, R. Mastandrea and B. Nachman, _Flow-enhanced transportation for anomaly detection_ , _Phys. Rev. D_ 107 (2023) 096025 [2212.11285].
  * (20) A. Hallin, G. Kasieczka, T. Quadfasel, D. Shih and M. Sommerhalder, _Resonant anomaly detection without background sculpting_ , 2210.14924.
  * (21) R.T. D’Agnolo and A. Wulzer, _Learning New Physics from a Machine_ , _Phys. Rev. D_ 99 (2019) 015014 [1806.02350].
  * (22) R.T. D’Agnolo, G. Grosso, M. Pierini, A. Wulzer and M. Zanetti, _Learning multivariate new physics_ , _Eur. Phys. J. C_ 81 (2021) 89 [1912.12155].
  * (23) M. Farina, Y. Nakai and D. Shih, _Searching for New Physics with Deep Autoencoders_ , _Phys. Rev. D_ 101 (2020) 075021 [1808.08992].
  * (24) T. Heimel, G. Kasieczka, T. Plehn and J.M. Thompson, _QCD or What?_ , _SciPost Phys._ 6 (2019) 030 [1808.08979].
  * (25) T.S. Roy and A.H. Vijay, _A robust anomaly finder based on autoencoders_ , 1903.02032.
  * (26) O. Cerri, T.Q. Nguyen, M. Pierini, M. Spiropulu and J.-R. Vlimant, _Variational Autoencoders for New Physics Mining at the Large Hadron Collider_ , _JHEP_ 05 (2019) 036 [1811.10276].
  * (27) A. Blance, M. Spannowsky and P. Waite, _Adversarially-trained autoencoders for robust unsupervised new physics searches_ , _JHEP_ 10 (2019) 047 [1905.10384].
  * (28) J. Hajer, Y.-Y. Li, T. Liu and H. Wang, _Novelty Detection Meets Collider Physics_ , _Phys. Rev. D_ 101 (2020) 076015 [1807.10261].
  * (29) A. De Simone and T. Jacques, _Guiding New Physics Searches with Unsupervised Learning_ , _Eur. Phys. J. C_ 79 (2019) 289 [1807.06038].
  * (30) A. Mullin, S. Nicholls, H. Pacey, M. Parker, M. White and S. Williams, _Does SUSY have friends? A new approach for LHC event analysis_ , _JHEP_ 02 (2021) 160 [1912.10625].
  * (31) B.M. Dillon, D.A. Faroughy and J.F. Kamenik, _Uncovering latent jet substructure_ , _Phys. Rev. D_ 100 (2019) 056002 [1904.04200].
  * (32) J.A. Aguilar-Saavedra, J.H. Collins and R.K. Mishra, _A generic anti-QCD jet tagger_ , _JHEP_ 11 (2017) 163 [1709.01087].
  * (33) M. Romão Crispim, N.F. Castro, R. Pedro and T. Vale, _Transferability of Deep Learning Models in Searches for New Physics at Colliders_ , _Phys. Rev. D_ 101 (2020) 035042 [1912.04220].
  * (34) M. Crispim Romão, N.F. Castro, J.G. Milhano, R. Pedro and T. Vale, _Use of a generalized energy Mover’s distance in the search for rare phenomena at colliders_ , _Eur. Phys. J. C_ 81 (2021) 192 [2004.09360].
  * (35) O. Amram and C.M. Suarez, _Tag N’ Train: a technique to train improved classifiers on unlabeled data_ , _JHEP_ 01 (2021) 153 [2002.12376].
  * (36) T. Cheng, J.-F. Arguin, J. Leissner-Martin, J. Pilette and T. Golling, _Variational autoencoders for anomalous jet tagging_ , _Phys. Rev. D_ 107 (2023) 016002 [2007.01850].
  * (37) C.K. Khosa and V. Sanz, _Anomaly Awareness_ , _SciPost Phys._ 15 (2023) 053 [2007.14462].
  * (38) P. Thaprasop, K. Zhou, J. Steinheimer and C. Herold, _Unsupervised Outlier Detection in Heavy-Ion Collisions_ , _Phys. Scripta_ 96 (2021) 064003 [2007.15830].
  * (39) S. Alexander, S. Gleyzer, H. Parul, P. Reddy, M.W. Toomey, E. Usai et al., _Decoding Dark Matter Substructure without Supervision_ , 2008.12731.
  * (40) V. Mikuni and F. Canelli, _Unsupervised clustering for collider physics_ , _Phys. Rev. D_ 103 (2021) 092007 [2010.07106].
  * (41) M. van Beekveld, S. Caron, L. Hendriks, P. Jackson, A. Leinweber, S. Otten et al., _Combining outlier analysis algorithms to identify new physics at the LHC_ , _JHEP_ 09 (2021) 024 [2010.07940].
  * (42) S.E. Park, D. Rankin, S.-M. Udrescu, M. Yunus and P. Harris, _Quasi Anomalous Knowledge: Searching for new physics with embedded knowledge_ , _JHEP_ 21 (2020) 030 [2011.03550].
  * (43) D.A. Faroughy, _Uncovering hidden new physics patterns in collider events using Bayesian probabilistic models_ , _PoS_ ICHEP2020 (2021) 238 [2012.08579].
  * (44) P. Chakravarti, M. Kuusela, J. Lei and L. Wasserman, _Model-Independent Detection of New Physics Signals Using Interpretable Semi-Supervised Classifier Tests_ , 2102.07679.
  * (45) J. Batson, C.G. Haaf, Y. Kahn and D.A. Roberts, _Topological Obstructions to Autoencoding_ , _JHEP_ 04 (2021) 280 [2102.08380].
  * (46) A. Blance and M. Spannowsky, _Unsupervised event classification with graphs on classical and photonic quantum computers_ , _JHEP_ 21 (2020) 170 [2103.03897].
  * (47) B. Bortolato, A. Smolkovič, B.M. Dillon and J.F. Kamenik, _Bump hunting in latent space_ , _Phys. Rev. D_ 105 (2022) 115009 [2103.06595].
  * (48) J.H. Collins, P. Martín-Ramiro, B. Nachman and D. Shih, _Comparing weak- and unsupervised methods for resonant anomaly detection_ , _Eur. Phys. J. C_ 81 (2021) 617 [2104.02092].
  * (49) B.M. Dillon, T. Plehn, C. Sauer and P. Sorrenson, _Better Latent Spaces for Better Autoencoders_ , _SciPost Phys._ 11 (2021) 061 [2104.08291].
  * (50) T. Finke, M. Krämer, A. Morandini, A. Mück and I. Oleksiyuk, _Autoencoders for unsupervised anomaly detection in high energy physics_ , _JHEP_ 06 (2021) 161 [2104.09051].
  * (51) D. Shih, M.R. Buckley, L. Necib and J. Tamanas, _via machinae: Searching for stellar streams using unsupervised machine learning_ , _Mon. Not. Roy. Astron. Soc._ 509 (2021) 5992 [2104.12789].
  * (52) O. Atkinson, A. Bhardwaj, C. Englert, V.S. Ngairangbam and M. Spannowsky, _Anomaly detection with convolutional Graph Neural Networks_ , _JHEP_ 08 (2021) 080 [2105.07988].
  * (53) A. Kahn, J. Gonski, I. Ochoa, D. Williams and G. Brooijmans, _Anomalous jet identification via sequence modeling_ , _JINST_ 16 (2021) P08012 [2105.09274].
  * (54) T. Dorigo, M. Fumanelli, C. Maccani, M. Mojsovska, G.C. Strong and B. Scarpa, _RanBox: anomaly detection in the copula space_ , _JHEP_ 01 (2023) 008 [2106.05747].
  * (55) S. Caron, L. Hendriks and R. Verheyen, _Rare and Different: Anomaly Scores from a combination of likelihood and out-of-distribution models to detect new physics at the LHC_ , _SciPost Phys._ 12 (2022) 077 [2106.10164].
  * (56) E. Govorkova, E. Puljak, T. Aarrestad, M. Pierini, K.A. Woźniak and J. Ngadiuba, _LHC physics dataset for unsupervised New Physics detection at 40 MHz_ , _Sci. Data_ 9 (2022) 118 [2107.02157].
  * (57) G. Kasieczka, B. Nachman and D. Shih, _New Methods and Datasets for Group Anomaly Detection From Fundamental Physics_ , in _Conference on Knowledge Discovery and Data Mining_ , 7, 2021 [2107.02821].
  * (58) S. Volkovich, F. De Vito Halevy and S. Bressler, _A data-directed paradigm for BSM searches: the bump-hunting example_ , _Eur. Phys. J. C_ 82 (2022) 265 [2107.11573].
  * (59) E. Govorkova et al., _Autoencoders on field-programmable gate arrays for real-time, unsupervised new physics detection at 40 MHz at the Large Hadron Collider_ , _Nature Mach. Intell._ 4 (2022) 154 [2108.03986].
  * (60) B. Ostdiek, _Deep Set Auto Encoders for Anomaly Detection in Particle Physics_ , _SciPost Phys._ 12 (2022) 045 [2109.01695].
  * (61) K. Fraser, S. Homiller, R.K. Mishra, B. Ostdiek and M.D. Schwartz, _Challenges for unsupervised anomaly detection in particle physics_ , _JHEP_ 03 (2022) 066 [2110.06948].
  * (62) P. Jawahar, T. Aarrestad, N. Chernyavskaya, M. Pierini, K.A. Wozniak, J. Ngadiuba et al., _Improving Variational Autoencoders for New Physics Detection at the LHC With Normalizing Flows_ , _Front. Big Data_ 5 (2022) 803685 [2110.08508].
  * (63) J. Herrero-Garcia, R. Patrick and A. Scaffidi, _A semi-supervised approach to dark matter searches in direct detection data with machine learning_ , _JCAP_ 02 (2022) 039 [2110.12248].
  * (64) J.A. Aguilar-Saavedra, _Anomaly detection from mass unspecific jet tagging_ , _Eur. Phys. J. C_ 82 (2022) 130 [2111.02647].
  * (65) R. Tombs and C.G. Lester, _A method to challenge symmetries in data with self-supervised learning_ , _JINST_ 17 (2022) P08024 [2111.05442].
  * (66) C.G. Lester and R. Tombs, _Using unsupervised learning to detect broken symmetries, with relevance to searches for parity violation in nature. (Previously: ”Stressed GANs snag desserts”)_ , 2111.00616.
  * (67) V. Mikuni, B. Nachman and D. Shih, _Online-compatible unsupervised nonresonant anomaly detection_ , _Phys. Rev. D_ 105 (2022) 055006 [2111.06417].
  * (68) S. Chekanov and W. Hopkins, _Event-Based Anomaly Detection for Searches for New Physics_ , _Universe_ 8 (2022) 494 [2111.12119].
  * (69) R.T. d’Agnolo, G. Grosso, M. Pierini, A. Wulzer and M. Zanetti, _Learning new physics from an imperfect machine_ , _Eur. Phys. J. C_ 82 (2022) 275 [2111.13633].
  * (70) F. Canelli, A. de Cosa, L.L. Pottier, J. Niedziela, K. Pedro and M. Pierini, _Autoencoders for semivisible jet detection_ , _JHEP_ 02 (2022) 074 [2112.02864].
  * (71) V.S. Ngairangbam, M. Spannowsky and M. Takeuchi, _Anomaly detection in high-energy physics using a quantum autoencoder_ , _Phys. Rev. D_ 105 (2022) 095004 [2112.04958].
  * (72) L. Bradshaw, S. Chang and B. Ostdiek, _Creating simple, interpretable anomaly detectors for new physics in jet substructure_ , _Phys. Rev. D_ 106 (2022) 035014 [2203.01343].
  * (73) J.A. Aguilar-Saavedra, _Taming modeling uncertainties with mass unspecific supervised tagging_ , _Eur. Phys. J. C_ 82 (2022) 270 [2201.11143].
  * (74) T. Buss, B.M. Dillon, T. Finke, M. Krämer, A. Morandini, A. Mück et al., _What’s anomalous in LHC jets?_ , _SciPost Phys._ 15 (2023) 168 [2202.00686].
  * (75) S. Alvi, C.W. Bauer and B. Nachman, _Quantum anomaly detection for collider physics_ , _JHEP_ 02 (2023) 220 [2206.08391].
  * (76) B.M. Dillon, R. Mastandrea and B. Nachman, _Self-supervised anomaly detection for new physics_ , _Phys. Rev. D_ 106 (2022) 056005 [2205.10380].
  * (77) M. Birman, B. Nachman, R. Sebbah, G. Sela, O. Turetz and S. Bressler, _Data-directed search for new physics based on symmetries of the SM_ , _Eur. Phys. J. C_ 82 (2022) 508 [2203.07529].
  * (78) M. Letizia, G. Losapio, M. Rando, G. Grosso, A. Wulzer, M. Pierini et al., _Learning new physics efficiently with nonparametric methods_ , _Eur. Phys. J. C_ 82 (2022) 879 [2204.02317].
  * (79) C. Fanelli, J. Giroux and Z. Papandreou, _‘Flux+Mutability’: a conditional generative approach to one-class classification and anomaly detection_ , _Mach. Learn. Sci. Tech._ 3 (2022) 045012 [2204.08609].
  * (80) T. Finke, M. Krämer, M. Lipp and A. Mück, _Boosting mono-jet searches with model-agnostic machine learning_ , _JHEP_ 08 (2022) 015 [2204.11889].
  * (81) R. Verheyen, _Event Generation and Density Estimation with Surjective Normalizing Flows_ , _SciPost Phys._ 13 (2022) 047 [2205.01697].
  * (82) B.M. Dillon, L. Favaro, T. Plehn, P. Sorrenson and M. Krämer, _A Normalized Autoencoder for LHC Triggers_ , 2206.14225.
  * (83) S. Caron, R.R. de Austri and Z. Zhang, _Mixture-of-Theories training: can we find new physics and anomalies better by mixing physical theories?_ , _JHEP_ 03 (2023) 004 [2207.07631].
  * (84) S.E. Park, P. Harris and B. Ostdiek, _Neural embedding: learning the embedding of the manifold of physics data_ , _JHEP_ 07 (2023) 108 [2208.05484].
  * (85) J.F. Kamenik and M. Szewc, _Null hypothesis test for anomaly detection_ , _Phys. Lett. B_ 840 (2023) 137836 [2210.02226].
  * (86) G. Kasieczka, R. Mastandrea, V. Mikuni, B. Nachman, M. Pettee and D. Shih, _Anomaly detection under coordinate transformations_ , _Phys. Rev. D_ 107 (2023) 015009 [2209.06225].
  * (87) J.Y. Araz and M. Spannowsky, _Quantum-probabilistic Hamiltonian learning for generative modelling & anomaly detection_, 2211.03803.
  * (88) J. Schuhmacher, L. Boggia, V. Belis, E. Puljak, M. Grossi, M. Pierini et al., _Unravelling physics beyond the standard model with classical and quantum anomaly detection_ , _Mach. Learn. Sci. Tech._ 4 (2023) 045031 [2301.10787].
  * (89) S. Roche, Q. Bayer, B. Carlson, W. Ouligian, P. Serhiayenka, J. Stelzer et al., _Nanosecond anomaly detection with decision trees for high energy physics and real-time application to exotic Higgs decays_ , 2304.03836.
  * (90) L. Vaslin, V. Barra and J. Donini, _GAN-AE: an anomaly detection algorithm for New Physics search in LHC data_ , _Eur. Phys. J. C_ 83 (2023) 1008 [2305.15179].
  * (91) ATLAS Collaboration, _Anomaly detection search for new resonances decaying into a Higgs boson and a generic new particle $X$ in hadronic final states using $\sqrt{s}=13$ TeV $pp$ collisions with the ATLAS detector_, _Phys. Rev. D_ 108 (2023) 052009 [2306.03637].
  * (92) S.V. Chekanov and R. Zhang, _Enhancing the hunt for new phenomena in dijet final-states using anomaly detection filter at High-Luminosity Large Hadron Collider_ , 2308.02671.
  * (93) CMS ECAL collaboration, _Autoencoder-based Anomaly Detection System for Online Data Quality Monitoring of the CMS Electromagnetic Calorimeter_ , 2309.10157.
  * (94) G. Bickendorf, M. Drees, G. Kasieczka, C. Krause and D. Shih, _Combining Resonant and Tail-based Anomaly Detection_ , 2309.12918.
  * (95) M. Freytsis, M. Perelstein and Y.C. San, _Anomaly Detection in Presence of Irrelevant Features_ , 2310.13057.
  * (96) E.M. Metodiev, J. Thaler and R. Wynne, _Anomaly Detection in Collider Physics via Factorized Observables_ , 2312.00119.
  * (97) Y. Song et al., _Score-based generative modeling through stochastic differential equations_ , 2011.13456.
  * (98) T. Karras et al., _Elucidating the design space of diffusion-based generative models_ , 2206.00364.
  * (99) L. Yang, Z. Zhang, Y. Song, S. Hong, R. Xu, Y. Zhao et al., _Diffusion models: A comprehensive survey of methods and applications_ , 2209.00796.
  * (100) R.T.Q. Chen, Y. Rubanova, J. Bettencourt and D. Duvenaud, _Neural ordinary differential equations_ , 1806.07366.
  * (101) X. Liu, C. Gong and Q. Liu, _Flow straight and fast: Learning to generate and transfer data with rectified flow_ , 2209.03003.
  * (102) M.S. Albergo and E. Vanden-Eijnden, _Building normalizing flows with stochastic interpolants_ , 2209.15571.
  * (103) Y. Lipman, R.T.Q. Chen, H. Ben-Hamu, M. Nickel and M. Le, _Flow matching for generative modeling_ , 2210.02747.
  * (104) V. Mikuni and B. Nachman, _Score-based generative models for calorimeter shower simulation_ , _Phys. Rev. D_ 106 (2022) 092009.
  * (105) M. Leigh, D. Sengupta, G. Quétant, J.A. Raine, K. Zoch and T. Golling, _PC-JeDi: Diffusion for Particle Cloud Generation in High Energy Physics_ , 2303.05376.
  * (106) V. Mikuni, B. Nachman and M. Pettee, _Fast Point Cloud Generation with Diffusion Models in High Energy Physics_ , 2304.01266.
  * (107) A. Shmakov, K. Greif, M. Fenton, A. Ghosh, P. Baldi and D. Whiteson, _End-To-End Latent Variational Diffusion Models for Inverse Problems in High Energy Physics_ , 2305.10399.
  * (108) A. Butter, N. Huetsch, S.P. Schweitzer, T. Plehn, P. Sorrenson and J. Spinner, _Jet Diffusion versus JetGPT – Modern Networks for the LHC_ , 2305.10475.
  * (109) M. Leigh, D. Sengupta, J.A. Raine, G. Quétant and T. Golling, _PC-Droid: Faster diffusion and improved quality for particle cloud generation_ , 2307.06836.
  * (110) O. Amram and K. Pedro, _CaloDiffusion with GLaM for High Fidelity Calorimeter Simulation_ , 2308.03876.
  * (111) V. Mikuni and B. Nachman, _CaloScore v2: Single-shot Calorimeter Shower Simulation with Diffusion Models_ , 2308.03847.
  * (112) E. Buhmann, F. Gaede, G. Kasieczka, A. Korol, W. Korcari, K. Krüger et al., _CaloClouds II: Ultra-Fast Geometry-Independent Highly-Granular Calorimeter Simulation_ , 2309.05704.
  * (113) E. Buhmann, C. Ewen, D.A. Faroughy, T. Golling, G. Kasieczka, M. Leigh et al., _EPiC-ly Fast Particle Cloud Generation with Flow-Matching and Diffusion_ , 2310.00049.
  * (114) E. Buhmann, C. Ewen, G. Kasieczka, V. Mikuni, B. Nachman and D. Shih, _Full Phase Space Resonant Anomaly Detection_ , 2310.06897.
  * (115) T. Heimel, N. Huetsch, R. Winterhalder, T. Plehn and A. Butter, _Precision-Machine Learning for the Matrix Element Method_ , 2310.07752.
  * (116) P. Devlin, J.-W. Qiu, F. Ringer and N. Sato, _Diffusion model approach to simulating electron-proton scattering events_ , 2310.16308.
  * (117) T. Heimel, N. Huetsch, F. Maltoni, O. Mattelaer, T. Plehn and R. Winterhalder, _The MadNIS Reloaded_ , 2311.01548.
  * (118) A. Butter, T. Jezo, M. Klasen, M. Kuschick, S.P. Schweitzer and T. Plehn, _Kicking it Off(-shell) with Direct Diffusion_ , 2311.17175.
  * (119) J. Birk, E. Buhmann, C. Ewen, G. Kasieczka and D. Shih, _Flow Matching Beyond Kinematics: Generating Jets with Particle-ID and Trajectory Displacement Information_ , 2312.00123.
  * (120) G. Kasieczka, B. Nachman and D. Shih, _Official Datasets for LHC Olympics 2020 Anomaly Detection Challenge (Version v6)_ , 2019.  10.5281/zenodo.4536624.
  * (121) V. Mikuni and B. Nachman, _High-dimensional and Permutation Invariant Anomaly Detection_ , 2306.03933.
  * (122) T. Sjöstrand, S. Ask, J.R. Christiansen, R. Corke, N. Desai, P. Ilten et al., _An introduction to PYTHIA 8.2_ , _Comput. Phys. Commun._ 191 (2015) 159 [1410.3012].
  * (123) J. de Favereau, C. Delaere, P. Demin, A. Giammanco, V. Lemaître, A. Mertens et al., _DELPHES 3, A modular framework for fast simulation of a generic collider experiment_ , _JHEP_ 02 (2014) 057 [1307.6346].
  * (124) M. Cacciari, G.P. Salam and G. Soyez, _The anti-kt jet clustering algorithm_ , _JHEP_ 04 (2008) 063.
  * (125) M. Cacciari, G.P. Salam and G. Soyez, _FastJet User Manual_ , _Eur. Phys. J. C_ 72 (2012) 1896 [1111.6097].
  * (126) M. Bahr et al., _Herwig++ Physics and Manual_ , _Eur. Phys. J. C_ 58 (2008) 639 [0803.0883].
  * (127) T. Golling, G. Kasieczka, C. Krause, R. Mastandrea, B. Nachman, J.A. Raine et al., _The Interplay of Machine Learning–based Resonant Anomaly Detection Methods_ , 2307.11157.
  * (128) D. Shih, _Additional QCD Background Events for LHCO2020 R &D (signal region only)_, Dec., 2021.  10.5281/zenodo.5759087.
  * (129) K. He, X. Zhang, S. Ren and J. Sun, _Deep residual learning for image recognition_ , 1512.03385.
  * (130) A. Vaswani, N. Shazeer, N. Parmar, J. Uszkoreit, L. Jones, A.N. Gomez et al., _Attention is all you need_ , 1706.03762.
  * (131) T. Finke, M. Hein, G. Kasieczka, M. Krämer, A. Mück, P. Prangchaikul et al., _Back To The Roots: Tree-Based Algorithms for Weakly Supervised Anomaly Detection_ , 2309.13111.
  * (132) H. Touvron, M. Cord, A. Sablayrolles, G. Synnaeve and H. Jégou, _Going deeper with image transformers_ , 2103.17239.
  * (133) A. Butter, S. Diefenbacher, G. Kasieczka, B. Nachman and T. Plehn, _GANplifying event samples_ , _SciPost Phys._ 10 (2021) 139 [2008.06545].
  * (134) S. Bieringer, A. Butter, S. Diefenbacher, E. Eren, F. Gaede, D. Hundhausen et al., _Calomplification — the power of generative calorimeter models_ , _JINST_ 17 (2022) P09028 [2202.07352].

##  Appendix A Impact of side-band width

As seen in Ref. curtainsf4f , the amount of data used to train the models
producing reference background samples, has non-negligible impact on the
performance of the CWoLa classifiers. This is strongly influenced by the
widths of the side-bands around the signal region. It can be beneficial to
have a local training window around the signal region if it is expected that
the background composition may not be homogenous across a wide range of
$m_{JJ}$.

Furthermore, by using narrower side-bands with the SR centred on the signal
peak in $m_{JJ}$, we can also study the impact of increasing the relative
amount of signal data present in the training region without changing the data
in the SR to train the CWoLa classifier

Figure 12: Significance improvement as a function of background rejection for
Drapes $\emptyset$ (purple), Drapes SR (blue), Drapes SB (red), and
CurtainsF4F (orange), trained on side-band data for three different side-band
widths. All methods are trained on the sample with 3,000 injected signal
events, and a signal region $3300\leq m_{JJ}<3700$ GeV. The lines show the
mean value of fifty independent classifiers, with the shaded band representing
a 68% uncertainty.

The impact of SB width on the performance of Drapes and CurtainsF4F are
studied for two narrow side-bands in comparison to the nominal maximum width.
Fig. 12 show the performance for the case where there are 3,000 signal events
in the data. Here we see that all Drapes methods are more sensitive to the
side-band width than CurtainsF4F, demonstrating similar behaviour to Cathode
observed in Ref. curtainsf4f . However, Drapes SR retains slightly more of its
performance at high background rejection values.

##  Appendix B Studying oversampling

###  B.1 Impact of oversampling background template

In Fig. 13 the impact of oversampling the background reference samples with
Drapes $\emptyset$ i studied for 1,000 and 3,000 injected signal events. We
observe that a saturation of performance is achieved at a factor of four for
3,000 signal events, however in the case of 1,000 signal events more training
data are required to saturate the performance. Here saturation is achieved at
a factor of 12. For reference, the over-idealised curve has an oversampling
factor of five.

Figure 13: Significance improvement as a function of background rejection for
Drapes $\emptyset$ for different levels of background oversampling, compared
to Over-Idealised (green) and Supervised (black). All methods are trained on
the sample with 1,000 injected signal events (left) and 3,000 injected signal
events (right), and a signal region $3300\leq m_{JJ}<3700$ GeV. The lines show
the mean value of fifty independent classifiers, with the shaded band
representing a 68% uncertainty.

###  B.2 Oversampling signal region data

With Drapes we observe that oversampling the background template is able to
bring substantial improvement to the overall performance of the CWoLa
classifier. This is something which has also been observed and exploited in
many other approaches cathode ; curtains ; LaCathode ; feta ; curtainsf4f ;
interplay , and also brings performance gains in the idealised case. However,
this oversampling is only performed on one class in the classifier, with the
real signal region data untouched.

Another observation is that even in the idealised case that the performance
degrades as the number of signal events present in the signal region
decreases. By training a Drapes model on the data in the signal region, we can
learn the conditional density for the combination of both signal and
background events as a function of $m_{JJ}$. Using this model, GenSR, we can
sampling multiple events for each value of $m_{JJ}$ in the signal region and
see is able to improve the sensitivity of the CWoLa classifiers in the case of
few signal events. The CWoLa classifiers are subsequently trained to separate
this oversampled signal region data from the oversampled Drapes background
reference samples. Using generative models to inflate training statistics has
been observed to improve the performance relative to limited available
statistics Butter:2020qhk ; Bieringer:2022cbs . However, to our knowledge this
is the first test of this observation in the context of weakly supervised
learning.

To study this we compare the nominal Drapes $\emptyset$ against a setup in
which a second Drapes $\emptyset$ model has been trained inclusive of the
signal region, and is used to oversample the signal region data. The SB data
is also included when training the new model, labelled Drapes GenSR, to
minimise the chance of the CWoLa classifier only identifying differences in
the background modelling arising from different training statistics.
Unfortunately, in Fig. 14 we find that the classifiers trained using Drapes
GenSR perform worse than when using the real signal region data. This is
especially the case for low amounts of injected signal, and suggests the
Drapes model is not able to sufficiently capture the distribution over signal
events when the training dataset comprises mostly background data, and at
higher amounts of signal only approaches the upper limit of performance
already reached with Drapes $\emptyset$.

Figure 14: Significance improvement as a function of background rejection for
1,000 (left) and 3,000 (right) injected signal events for the nominal Drapes
$\emptyset$ approach (purple, solid) and when Drapes GenSR is used to
oversample the signal region data (purple, dashed). The lines show the mean
value of fifty independent classifiers, with the shaded band representing a
68% uncertainty.

##  Appendix C Additional figures

###  C.1 Partial diffusion

When using partial diffusion noise is added to initial events which are
subseqeuntly denoised. The mean absolute shift between the input data and
denoised output data is shown as a function of $\sigma^{\prime}$ in Fig. 15.
In other domains, this displacement can be used as a single event
classification score, with out of distribution often resulting in larger
displacements. However, as can be seen for $m_{j_{1}}$, the signal events have
a smaller displacement for all values of $\sigma^{\prime}$.

Figure 15: Displacement of input jets in the signal region after adding then
removing noise with various noise rates using Drapes SR. A noise rate of 0
represents the input data without any noise perturbation or diffusion steps,
and a rate of 80 corresponds to the case where the inputs are completely
noise. The same diffusion model is used which was trained on the sample with
3,000 injected signal events.

In Fig. 16 distributions for signal and background data after applying the
forward and inverse diffusion processes in partial diffusion are compared to
the true signal and background distributions. In all cases, background events
result in generated data following the same ground truth distribution.
However, in the case of signal events, as $\sigma^{\prime}$ increases, the
resulting distributions become more background like, however even at
$\sigma^{\prime}=2.24$ there remain differences between the generated data
using signal as a starting point, and the background true distribution.

Figure 16: Events generated with Drapes SR for increasing values of
$\sigma^{\prime}$ for $m_{J}^{1}$, $\tau_{21}^{j_{1}}$ and $\Delta R_{jj}$,
shown separately for the case where initial events are from the signal (Gen
Signal, brown) and background (Gen Background, blue) processes. The true
signal (red) and background (green) distributions are shown for reference.

The significance improvement and receiver operator characteristic curves
comparing Drapes $\emptyset$ to the partial diffusion methods are shown in
Fig. 17. Here a value of $\sigma^{\prime}=2.24$ is used for all three partial
diffusion methods. At 1,000 injected signal events, Drapes SB is able to reach
the highest level of performance. However, for the same oversampling factor
this results in more events in the background reference sample, as each side-
band event is considered when generating the background template. When using
the same number of raw training data, Drapes $\emptyset$ achieves even higher
levels of performance, as evidenced by the oversampling study in Fig. 13,
where Drapes $\emptyset$ can reach a max significance improvement of close to
15.

Figure 17: Background rejection as a function of signal efficiency (left) and
significance improvement as a function of background rejection (right) for
Drapes $\emptyset$ (purple), Drapes SR (blue), Drapes SB (red), Drapes MC
(magenta), and Supervised (black). Drapes SR, Drapes SB, and Drapes MC use
initial samples with a noise rate of $0.6$, and Drapes $\emptyset$ generates
samples purely from noise. All methods are trained on the sample with 1,000
(top) and 3,000 (bottom) injected signal events, and a signal region $3300\leq
m_{JJ}<3700$ GeV. The lines show the mean value of fifty independent
classifiers, with the shaded band representing a 68% uncertainty.

###  C.2 Maximum significance improvement

In other studies, the maximum significance improvement is used as a measure of
performance, despite its sensitivity to statistical fluctuations at very high
background rejection. The maximum significance improvement as a function of
the number of injected signal events for all Drapes methods is shown in Fig.
18. The maximum significance improvement as a function of $\sigma^{\prime}$
for Drapes with partial diffusion is shown in Fig. 19.

Figure 18: Maximum significance improvement as a function of the number of
signal events in the signal region, $3300\leq m_{JJ}<3700$ GeV, for
CurtainsF4F (orange), Drapes $\emptyset$ (purple), Drapes SR (red), Drapes SB
(blue), Drapes MC (magenta), Supervised (black), and Over-Idealised (green).
The lines show the mean value of fifty independent classifiers, with the
shaded band representing a 68% uncertainty.

Figure 19: Maximum significance improvement as a function of noise rate for
Drapes SR (red), Drapes SB (blue), and Drapes MC (magenta). All generation
methods use the same diffusion model trained on the sample with 1,000 (left)
and 3,000 (right) injected signal events, and a signal region $3300\leq
m_{JJ}<3700$ GeV. CurtainsF4F (orange), Supervised (black) and Over-Idealised
(green) are shown for reference. The lines show the mean value of fifty
independent classifiers, with the shaded band representing a 68% uncertainty.