# Black-box Adversarial Attacks on Network-wide Multi-step Traffic State
Prediction Models

Bibek Poudel and Weizi Li Bibek Poudel and Weizi Li are with the Department of
Computer Science, University of Memphis, Memphis, TN 38152, USA
<EMAIL_ADDRESS>
###### Abstract

Traffic state prediction is necessary for many Intelligent Transportation
Systems applications. Recent developments of the topic have focused on
network-wide, multi-step prediction, where state of the art performance is
achieved via deep learning models, in particular, graph neural network-based
models. While the prediction accuracy of deep learning models is high, these
models’ robustness has raised many safety concerns, given that imperceptible
perturbations added to input can substantially degrade the model performance.
In this work, we propose an adversarial attack framework by treating the
prediction model as a black-box, i.e., assuming no knowledge of the model
architecture, training data, and (hyper)parameters. However, we assume that
the adversary can oracle the prediction model with any input and obtain
corresponding output. Next, the adversary can train a substitute model using
input-output pairs and generate adversarial signals based on the substitute
model. To test the attack effectiveness, two state of the art, graph neural
network-based models (GCGRNN [1] and DCRNN [2]) are examined. As a result, the
adversary can degrade the target model’s prediction accuracy up to $54\%$. In
comparison, two conventional statistical models (linear regression and
historical average) are also examined. While these two models do not produce
high prediction accuracy, they are either influenced negligibly (less than
$3\%$) or are immune to the adversary’s attack.

##  I Introduction

Traffic state prediction is a crucial component of Intelligent Transportation
Systems (ITS) and has many applications in traffic control and management.
Various statistical and machine learning methods, ranging from traditional
techniques [3, 4, 5] to deep learning models [6, 7, 8, 1], have been applied
to improve the prediction accuracy. The spatial-temporal resolution of
prediction has also evolved from a single timestep on one link [9, 10, 11] to
multiple timesteps on the entire road network [12, 13, 14, 15, 1]. In this
work, we focus on network-wide, multi-step prediction models as they represent
the latest advancements regarding the subject.

While accuracy is an apparent measure of prediction, robustness is another
desired feature. A model that can produce high prediction accuracy but acts
violently to small perturbations on input has limited use in practice. A
better model should be robust under adversarial attacks [16]. The demand of
robustness is more pronounced during critical conditions such as the pandemic
[17, 18] and the deep learning era: while deep learning models can deliver
state of the art performance, they are known to be fragile to adversarial
examples—small crafted perturbations to the input can lead to highly
unpredictable behaviors of the model [19]. As an example, a state of the art
deep learning model can mis-classify a stop sign as a 45 mph speed limit sign
with minor changes to the sign [20]. These types of attacks have serious
safety implications for vehicles equipped with such a technology, e.g.,
autonomous driving.

The adversarial attack in ITS has been studied under the context of
recognition tasks for traffic signs [20] and licence plates [21], and control
and coordination of the vehicle platoon [22]. However, its influence on
traffic state prediction has not been receiving equal attention [16]. In this
paper, we propose an adversarial attack framework to degrade network-wide,
multi-step traffic state prediction models by treating the model as a black-
box, i.e., we assume no knowledge of the model architecture, training data,
and (hyper)parameters. Nevertheless, we assume the adversary can oracle a
deployed model (i.e., the target model) using arbitrary input to obtain its
output. The input-output pairs are then used to train a substitute model to
mimic the target model’s behaviors. Next, adversarial signals are generated
based on the substitute model via Fast Gradient Sign Method (FGSM) [23] and
Basic Iterative Method (BIM) [24], respectively. Using the transferability
property [25], the adversary then attacks the target model using the produced
adversarial signals with the goal to degrade the target model’s prediction
performance.

We test our black-box adversarial attack framework on two deep learning
models, namely graph convolutional gated recurrent neural network (GCGRNN) [1]
and diffusion convolutional recurrent neural Network (DCRNN) [2], which
correspond to state of the art performance on network-wide, multi-step traffic
state prediction. In comparison, we also test two traditional methods, namely
linear regression and historical average. Using substitute models trained on
the hourly traffic flow data from the Caltrans Performance Measurement System
(PeMS) [26], we show that the adversary can degrade the performance of GCGRNN
up to $26.41\%$ and DCRNN up to $54.07\%$. Compared to deep learning models,
traditional models are robust against adversarial attacks with less than $3\%$
or no performance degradation.

##  II Related Work

In this section, we first introduce relevant deep learning-based studies of
network-wide multi-step traffic state prediction. Then, we discuss related
studies of adversarial attacks in ITS.

###  II-A Traffic State Prediction

Traffic state prediction models need to capture both spatial and temporal
dependencies embedded in traffic data for high prediction accuracy [27]. Many
deep learning models have been developed for this task. One approach is to use
convolutional neural network (CNN). As an example, the traffic state of a
city-wide network is converted to a grid map which acts as an “image” to be
processed by CNN [28]. Since traffic data have varying structures, researchers
have adopted a more nature structure—graph—to represent traffic data (e.g.,
treat traffic sensors as nodes). This has motivated the use of graph
convolutional neural network (GCNN) [29]. One example is diffusion
convolutional recurrent neural Network (DCRNN) [2], which combines GCNN with
recurrent neural network (RNN) to capture the spatial-temporal dependencies.
More recently, graph convolutional gated recurrent neural network (GCGRNN) is
proposed to integrate data-driven graph filter and gated recurrent neural
network for capturing hidden correlations among traffic sensors without
requiring a predefined graph representation [1]. In this work, we treat GCGRNN
and DCRNN, along with two traditional methods (linear regression and
historical average), as our target models, and examine their adversarial
vulnerability.

###  II-B Adversarial Attacks in ITS

Adversarial attacks have been studied extensively for recognition tasks in
ITS. Examples include mis-classifying traffic signs at various angles and
distances to a vehicle [20] and mis-classifying license plates to an optical
character recognition system [21]. For control tasks in ITS, there exist
studies concerning the coordination of autonomous vehicles. In particular, a
single adversarial vehicle is found capable of destabilizing an entire
vehicular platoon, causing possible catastrophic accidents [22]. More
recently, adversarial attacks are crafted for in-vehicle networks, where a
black-box attacker successfully degrades the performance of the deep learning
model [30]. While adversarial attacks in ITS have received increasing
attention in recent years, to the best of our knowledge, studies that explore
the adversarial vulnerability of traffic state prediction models are scarce.
In this work, we develop a black-box adversarial attack framework for network-
wide, multi-step traffic state prediction models.

##  III Preliminaries

In this section, we briefly introduce the concept of adversarial attack, the
attack algorithms used in this work, and the traffic state prediction models
considered for attack.

###  III-A Network-wide Multi-step Traffic State Prediction

Both deep learning and traditional methods have been used to conduct network-
wide, multi-step traffic state prediction. For our attacks, we use two deep
learning models and two traditional models as target models. While deep
learning models provide state of the art performance, traditional models are
easier to interpret and implement. Among traditional models, linear regression
is conceptually simple and requires a few parameters to train and historical
average is non-parameterized, which requires no training and uses less data
and computation for prediction. These models have been widely used in traffic
forecasting [31, 32] and benchmarking [33, 1].

####  III-A1 Diffusion convolutional recurrent neural network (DCRNN)

DCRNN [2] requires a predefined spatial graph of sensor networks represented
using the adjacency matrix:

$A_{ij}=\text{exp}\frac{-dist(s_{i},s_{j})}{{\sigma}^{2}},$ (1)

where $dist(s_{i},s_{j})$ is the spatial distance between the sensors $s_{i}$
and $s_{j}$, and $\sigma$ is the standard deviation of the distance. DCRNN
predicts traffic state by capturing a) spatial features of traffic data via
random walk on the graph and b) temporal features though an encoder-decoder
architecture.

####  III-A2 Graph convolutional gated recurrent neural network (GCGRNN)

While graph convolution on DCRNN assumes a strong correlation between two
sensors that are topologically-close, GCGRNN [1] does not make the same
assumption. It achieves state of the art performance in traffic state
prediction by integrating data-driven graph filter and gated recurrent neural
network to automatically learn the adjacency matrix and capture hidden
correlations among traffic sensors. In addition, by using gated recurrent unit
cells, GCGRNN enjoys better training efficiency.

####  III-A3 Linear Regression (LR)

LR estimates the traffic state of the next timestep as a linearly weighted sum
of traffic states from historical timesteps. In a network-wide prediction
task, each sensor in the network is trained with the ordinary least squares
objective (no regularization) and combined over the prediction horizon for
estimation.

####  III-A4 Historical Average (HA)

HA assumes traffic state is periodic: the traffic state of the current period
is the average traffic state of previous periods. For example, if we take one
week as a period and consider four periods, the predicted traffic state of
current Monday is taken as the average state of past four Mondays.

###  III-B Adversarial Attack

An adversarial attack on deep neural network $f$ is conducted through crafting
an adversarial signal $\operatorname*{\mathbf{x}}^{*}$ by imposing minimal
perturbation $\delta_{\operatorname*{\mathbf{x}}}$ to the original signal
$\operatorname*{\mathbf{x}}$. $\operatorname*{\mathbf{x}}^{*}$ can be obtained
via solving the following optimization problem [19]:

$\begin{array}[]{rl}{\text{minimize}}&D(\operatorname*{\mathbf{x}},\operatorname*{\mathbf{x}}+\delta_{\operatorname*{\mathbf{x}}})\\\
\text{s.t.}&f(\operatorname*{\mathbf{x}})\neq
f(\operatorname*{\mathbf{x}}^{*}),\end{array}$ (2)

where $D$ is a distance metric such as ${L_{0}}$, ${L_{2}}$ or ${L_{\infty}}$.
Multiple methods can be used to solve Eq. 2. Two are examined in this work
which are introduced in the following.

####  III-B1 Fast Gradient Sign Method (FGSM)

FGSM [23] is designed to produce an adversarial signal in a fast, non-
iterative manner. It computes the gradient of the cost function $J$ w.r.t
input $\operatorname*{\mathbf{x}}$ and scales the sign of the gradient by a
${L_{\infty}}$ constraint for generating
$\delta_{\operatorname*{\mathbf{x}}}$:

$\mathbf{x}^{*}=\operatorname*{\mathbf{x}}+{\varepsilon}{\cdot}sign\left({\nabla}_{x}J\left(\theta,\operatorname*{\mathbf{x}},y\right)\right),$
(3)

where ${\theta}$ represents the parameters of $f$; $y$ is the target; and
${\varepsilon}$ is the ${L_{\infty}}$ constraint parameter controlling the
attack magnitude. Increasing the value of ${\varepsilon}$ will increase the
attack effectiveness but also make the adversarial example more
distinguishable (i.e., having large $\delta_{\operatorname*{\mathbf{x}}}$).

####  III-B2 Basic Iterative Method (BIM)

BIM [24] is a refined, iterative method of FGSM that generates an adversarial
signal. At each iteration, BIM will take a small step $\alpha$ and clip the
result using ${\varepsilon}$:

$\mathbf{x}^{*}_{n+1}=clip_{\operatorname*{\mathbf{x}},{\varepsilon}}\left(\mathbf{x}_{n}^{*}+\alpha\cdot
sign\left(\nabla_{\operatorname*{\mathbf{x}}}J\left(\theta,{\mathbf{x}^{*}_{n}},y\right)\right)\right),$
(4)

where $\theta$ denotes the parameters of $f$. Compared to FGSM, BIM can
produce a less distinguishable adversarial signal compared to the original
signal $\operatorname*{\mathbf{x}}$ at a cost of more computation.

##  IV Methodology

Our attack takes the form of a black-box threat model, where an adversary is
assumed to possess no knowledge about the target model’s architecture,
training data, and (hyper)parameters. However, the adversary can oracle the
target model by feeding any input data to it and obtain corresponding output.
The goal of the adversary is then to perturb the input data so that the target
model’s prediction accuracy will be degraded. The perturbed input is the
adversarial signal and can be obtained by solving the optimization program in
Eq. 2 based on a substitute model that mimics the target model’s behavior.

Figure 1: Systematic diagram of our framework. Network-wide traffic flow data
is pre-processed and split into two parts, one is used by the adversary to
oracle the target model to obtain input-output pairs for training the
substitute model; the other is used by the adversary to generate adversarial
signals based on the trained substitute model. The attack effectiveness is
conducted on the target model using original signals and their corresponding
adversarial signals.

The systematic diagram of our framework is shown in Fig. 1. The network-wide
traffic flow data is pre-processed and split into input data for oracling the
target model and ground-truth data for later evaluation of the attack
effectiveness. The adversary feeds the input data into the target model and
obtains corresponding output predictions. These input-output pairs are then
used to train and tune a substitute model such that the substitute model
produces similar prediction performance as the target model. Once this is
done, adversarial signals are generated using the substitute model to attack
the target model. In the following, we detail these steps.

TABLE I: LEFT: Average L2 distance between the 2 616 original signals and
their adversarial signals as well as average L2 distance between the target
model’s predictions on 2 616 original signals and their corresponding
adversarial signals. All values are normalized by $10^{8}$. RIGHT: Performance
evaluation of target models before and after the attack. Deep Learning models
suffer severe degradation in terms of their prediction performance, compared
to traditional models. DCRNN has the highest performance degradation among all
models.

| Change on original signal (L2) | Change on prediction (L2) | Pre-attack RMSE | Post-attack RMSE | RMSE Degradation(%)
---|---|---|---|---|---
Model | FGSM | BIM | FGSM | BIM | N/A | FGSM | BIM | FGSM | BIM
GCGRNN | 3.35 | 1.67 | 3.27 | 1.80 | 529.88 | 669.87 | 607.14 | 26.41% | 14.58%
DCRNN | 3.35 | 1.45 | 17.5 | 10.1 | 770.04 | 1186.41 | 1013.66 | 54.07% | 31.63%
LR | 3.35 | 1.45 | 3.16 | 1.38 | 1870.43 | 1916.83 | 1891.21 | 2.48% | 1.11%
HA | 132 | 129 | 0 | 0 | 935.46 | 935.46 | 935.46 | 0 | 0

####  IV-1 Data pre-processing

The traffic flow data is obtained from PeMS [26], which consists of hourly
data from Jan 1, 2018 to Jun 30, 2019 of 150 sensors in Los Angeles,
California. To prepare this network-wide data for multi-step prediction, 12
timesteps of data in the past are treated as inputs and next 12 timesteps of
data are treated as outputs (i.e., ground truth). In total, we obtain 13 081
network-wide, 12-step traffic flow input-output data.

####  IV-2 Substitute model training

In order to train the substitute model, 9 157 (sampled from 13 081) input data
is used to oracle the target model to obtain the corresponding predictions
(which are different from the ground-truth output data for the 9 157 input
data). These input-output pairs form our training data for the substitute
model, which are z-score normalized using:

$\operatorname*{\mathbf{x}}=(\operatorname*{\mathbf{x}}-{\mu})/{\sigma},$ (5)

where ${\mu}$ and ${\sigma}$ are the mean and standard deviation of the 9 157
input data, respectively.

The substitute model is chosen to be the 50-layered residual network
(ResNet-50) [34] as it achieves state of the art performance in time series
classification on various UCR datasets [35]. To make the model more suitable
for our purpose, we alter the model’s architecture by removing the pooling
operation from initial layers to better preserve temporal correlations
embedded in traffic data, and change the input and output layers to match the
prediction task of 150 sensors across 12 timesteps. For each target model
examined (four in total), a substitute model with randomly initialized
parameters and the same set of hyperparameters is trained, using the 9 157
input-output pairs.

####  IV-3 Adversarial attack

In total, 2 616 adversarial signals are generated using both Fast Gradient
Sign Method (FGSM) [23] and Basic Iterative Method (BIM) [24] by constraining
the optimization program given in Eq. 2 with the maximum and minimum values of
normalized inputs obtained from Eq. 5. This ensures that the adversarial
signals are physically plausible, i.e., no negative traffic flows should
occur. The adversarial signals generated on the substitute model are fed into
the target model to obtain the “post-attack predictions” as the attack
results. To measure the attack effectiveness, the inputs that are used to
generate these adversarial signals are also fed into the target model to
obtain “pre-attack predictions.” Root mean squared error (RMSE) is used as an
evaluation metric to measure the model performance:

$\textrm{RMSE}=\sqrt{\frac{1}{S\times F\times
N}{\sum_{{i}}^{S}}{\sum_{{d}}^{F}}{\sum_{{n}}^{N}}{(y^{i}_{dn}-\hat{y}^{i}_{dn})}^{2}},$
(6)

where $S$ = 2 616 is the number of adversarial signals; $F$ = 12 is the number
of timesteps; $N$ = 150 is the number of sensors; $y_{dn}$ is the ground-truth
data from PeMS and $\hat{y}_{dn}$ is either the “post-attack prediction” or
“pre-attack predcition”. Denoting $\text{RMSE}_{pre}$ the difference between
“pre-attack predictions” and the ground-truth, and $\text{RMSE}_{post}$ the
difference between “post-attack predictions” and the ground-truth, the
performance degradation is calculated as

$\textrm{RMSE
Degradation}=\frac{\textrm{RMSE}_{post}-\textrm{RMSE}_{pre}}{\textrm{RMSE}_{pre}}.$
(7)

Figure 2: Examples of original signals and their corresponding adversarial
signals’ impact on model prediction results. For both GCGRNN and DCRNN, we can
see that the adversarial signals can cause large deviation between the
prediction results and the ground truth than the original signals. Figure 3:
The RMSE distributions on both original signals and their adversarial signals
(AS) of four target models ($2~{}616$ in total). Between GCGRNN and DCRNN,
DCRNN shows larger distribution shift in RMSE values, which demonstrates the
comparative robustness of GCGRNN against adversarial attacks. For LR, the
shift is marginal; for HA, no shift is observed. These results demonstrate
that traditional models can be more robust or even immune against adversarial
attacks.

##  V Experiments

In this section, we first describe the experiment set-up and then present and
discuss the experiment results. All experiments are conducted using an
Intel(R) Core(TM) i7-10700 CPU, a Nvidia RTX 2080 SUPER GPU, and 32G RAM.
PyTorch [36] is used to implement the substitute model and Advertorch [37] is
used to generate adversarial signals.

The training data obtained by oracling the four pre-trained target models is
respectively used to train four substitute models under the objective to
minimize RMSE. The values of the hyperparameters are as follows: learning rate
$=0.005$, optimizer $=$ Adam [38], batch size $=24$, number of epochs $=50$.
To generate adversarial signals, RMSE is used as the cost function for both
FGSM (with $\varepsilon=0.2$) and BIM (with $\alpha=0.05,~{}\varepsilon=0.2$,
number of iterations $=10$). The upper and lower limit constraints on the
optimization program in Eq. 2 are set to $3.49$ and $-1.89$, respectively.

Table I LEFT shows the results of using L2 distance as the metric to measure
1) the difference between the original signal and its corresponding
adversarial signal and 2) the difference between the target model’s prediction
on the original signal and the corresponding adversarial signal. Except for
HA, the changes produced on the original signal by FGSM ($3.35\times 10^{8}$)
are almost doubled than that of BIM ($1.67\times 10^{8}$, $1.45\times
10^{8}$), indicating that the adversarial signal from BIM are in general
closer to the original signals than those from FGSM. For HA, the changes
produced by FGSM ($1.32\times 10^{10}$) and BIM ($1.29\times 10^{10}$) to the
original signals are much higher.

This may due to the inability of the substitute model to mimic HA’s behaviour:
HA makes a periodic assumption on traffic state prediction i.e., to make a
prediction on next 12 timesteps from an input of 12 historical timesteps, it
does not use the input flow values, rather it looks at the hours of day, and
day of the week (assuming period = week) of the input and averages the flow
values over the same hours of day on the same day of the week from previous
weeks. The substitute model makes predictions based on the learned features
from the flow values of 12 historical timesteps, which does not contain
periodic information. In other words, there is no way to tell if two network-
wide multi-step inputs belong to the same period (e.g., the same hours of the
same day of week). As a result, the substitute model fails to learn HA’s
behavior, causing FGSM and BIM to produce large changes to the input signal
while maximizing RMSE.

Regarding the changes on predictions, for GCGRNN and LR, changes under FGSM
($3.27\times 10^{8}$, $3.16\times 10^{8}$) are almost doubled than that of the
changes under BIM ($1.80\times 10^{8}$, $1.38\times 10^{8}$). These changes
are similar to the changes produced on the original signals to craft
adversarial signals using FGSM and BIM respectively. In contrast, no changes
are produced on the predictions for HA, this is again explained by the
periodic assumption of HA since HA does not consider the flow values in the
current input rather it considers the historical periods that it belongs.
Hence, changes produced on current input cannot influence the prediction.

DCRNN suffers the largest changes at $17.5\times 10^{8}$ under FGSM and
$10.1\times 10^{8}$ under BIM, indicating that the substitute model highly
resembles DCRNN. Fig. 2 shows the predictions of GCGRNN and DCRNN on both
original signals and their corresponding adversarial signals from either FGSM
or BIM. As shown, the adversarial signals can cause much larger deviation
between the model prediction and the ground truth than that of the original
signals.

Table I RIGHT gives information about the target model predictions before the
attack, after the attack, and RMSE degradation. Under no attack, GCGRNN has
the smallest RMSE of $529.88$, which has increased to
$699.87~{}(26.41\%~{}\text{RMSE degradation})$ using FGSM and
$607.14~{}(14.58\%~{}\text{RMSE degradation})$ using BIM under adversarial
attacks, respectively. The highest performance drop is for DCRNN where the
RMSE of $770.4$ under no attack has increased to
$1186.41~{}(54.07\%~{}\text{RMSE degradation})$ using FGSM and
$1013.66~{}(31.63\%~{}\text{RMSE degradation})$ using BIM under adversarial
attacks, respectively. The RMSE distributions on both original signals and
their adversarial signals are shown in Fig. 3. DCRNN corresponds to more
dramatic distribution shift to higher RMSE values than GCGRNN, which
demonstrates the comparative robustness of GCGRNN against adversarial attacks
than DCRNN.

For traditional models’ performance shown in Table I RIGHT, LR operates at
high RMSE under no attack ($1870.43$) because of the linearity assumption
(linearly combines historical timesteps for prediction). The RMSE has
increased to $1916.83~{}(2.48\%~{}\text{RMSE degradation})$ using FGSM and
$1891.21~{}(1.11\%~{}\text{RMSE degradation})$ using BIM under adversarial
attacks, respectively. The “RMSE degradation” values for LR are relatively
small compared to that for GCGRNN and DCRNN. From Table I LEFT, we can see
that although the “change on prediction” under both FGSM and BIM for LR are
similar to “change on prediction” for GCGRNN, the difference between “Pre-
attack RMSE” and “Post-attack RMSE” is higher for GCGRNN than LR. This
indicates that for LR, although the predictions on adversarial signals are far
from predictions on original signals, their distances to the ground truth
remain similar, i.e., the LR’s prediction already has high RMSE due to the
model limitation in capturing temporal correlations embedded in historical
traffic flow. In case of HA, the performance remains invariant to any attack,
i.e., RMSE remains the same for original signals and adversarial signals due
to the periodicity assumption explained before.

The lower prediction performance of both traditional models under no attack,
due to their inability to accurately capture spatial and temporal correlations
embedded in traffic data (owing to their simplistic assumptions of either
linearity (LR) or periodicity (HA)), may be a contributor to their robustness.
As can seen from Fig. 3, there exists marginal distribution shift to higher
RMSE values for LR and no distribution shift for HA.

##  VI Conclusion and Future Work

In this work, we propose a black-box adversarial attack framework where an
adversary can attack traffic state prediction models using publicly available
dataset to oracle a target model, train a substitute model on prediction
results, and attack the target model using the produced adversarial signals
based on the substitute model. We have analyzed two deep learning models and
two traditional models and find that traditional models are more robust to the
proposed adversarial attack. Among the deep learning models, we find that
GCGRNN is comparatively more robust against adversarial attacks than DCRNN.

There are many future research directions of this work. First of all, the
input data for oracling the target model is complete and accurate. It would be
interesting to study the influence of compressing data [39] or even missing
data [40] on the attack. Secondly, our work could be extended to adversarial
attacks on other traffic measurements such as speed and travel time. Another
interesting topic is to study the impact of adversarial attack on existing
techniques to navigate and coordinate autonomous vehicles [41, 42], which
heavily rely on accurate prediction of traffic states to operate. This line of
research can benefit from the use of virtual traffic platforms [43, 44] for
experimenting, since the real-world deployment of the algorithms has safety
implications.

Last but not least, deep learning models (e.g., GCGRNN and DCRNN) are data-
driven, which in general do not consider domain knowledge such as short-term
seasonality (e.g., weekdays vs. weekend), long-term seasonality (e.g., Spring
vs. Summer), and special events (e.g., holidays). Given that the real-world
traffic is associated with such information, it would be worthwhile to explore
their inclusion as a part of the input.

## References

  * [1] L. Lin, W. Li, and L. Zhu, “Network-wide multi-step traffic volume prediction using graph convolutional gated recurrent neural network,” _Technical Report_ , 2021.
  * [2] Y. Li, R. Yu, C. Shahabi, and Y. Liu, “Diffusion convolutional recurrent neural network: Data-driven traffic forecasting,” in _International Conference on Learning Representations (ICLR)_ , 2018.
  * [3] M. S. Ahmed and A. R. Cook, _Analysis of freeway traffic time-series data by using Box-Jenkins techniques_ , 1979, no. 722.
  * [4] O. Anacleto, C. Queen, and C. J. Albers, “Multivariate forecasting of road traffic flows in the presence of heteroscedasticity and measurement errors,” _Journal of the Royal Statistical Society: Series C (Applied Statistics)_ , vol. 62, no. 2, pp. 251–270, 2013.
  * [5] L. Zhang, Q. Liu, W. Yang, N. Wei, and D. Dong, “An improved k-nearest neighbor model for short-term traffic flow prediction,” _Procedia-Social and Behavioral Sciences_ , vol. 96, pp. 653–662, 2013.
  * [6] Y. Lv, Y. Duan, W. Kang, Z. Li, and F.-Y. Wang, “Traffic flow prediction with big data: a deep learning approach,” _IEEE Transactions on Intelligent Transportation Systems_ , vol. 16, no. 2, pp. 865–873, 2014.
  * [7] L. Lin, W. Li, and S. Peeta, “Predicting station-level bike-sharing demands using graph convolutional neural network,” in _Transportation Research Board 98th Annual Meeting (TRB)_ , 2019.
  * [8] L. Lin, W. Li, H. Bi, and L. Qin, “Vehicle trajectory prediction using LSTMs with spatial-temporal attention mechanisms,” _IEEE Intelligent Transportation Systems Magazine_ , 2021.
  * [9] S. R. Chandra and H. Al-Deek, “Predictions of freeway traffic speeds and volumes using vector autoregressive models,” _Journal of Intelligent Transportation Systems_ , vol. 13, no. 2, pp. 53–72, 2009.
  * [10] X. Ma, Z. Tao, Y. Wang, H. Yu, and Y. Wang, “Long short-term memory neural network for traffic speed prediction using remote microwave sensor data,” _Transportation Research Part C: Emerging Technologies_ , vol. 54, pp. 187–197, 2015.
  * [11] L. Lin, J. C. Handley, Y. Gu, L. Zhu, X. Wen, and A. W. Sadek, “Quantifying uncertainty in short-term traffic prediction and its application to optimal staffing plan development,” _Transportation Research Part C: Emerging Technologies_ , vol. 92, pp. 323–348, 2018.
  * [12] P. Cai, Y. Wang, G. Lu, P. Chen, C. Ding, and J. Sun, “A spatiotemporal correlative k-nearest neighbor model for short-term traffic multistep forecasting,” _Transportation Research Part C: Emerging Technologies_ , vol. 62, pp. 21–34, 2016.
  * [13] W. Li, D. Nie, D. Wilkie, and M. C. Lin, “Citywide estimation of traffic dynamics via sparse GPS traces,” _IEEE Intelligent Transportation Systems Magazine_ , vol. 9, no. 3, pp. 100–113, 2017.
  * [14] W. Li, M. Jiang, Y. Chen, and M. C. Lin, “Estimating urban traffic states using iterative refinement and wardrop equilibria,” _IET Intelligent Transport Systems_ , vol. 12, no. 8, pp. 875–883, 2018.
  * [15] Y. Luo, Q. Liu, H. Zhu, H. Fan, T. Song, C. W. Yu, and B. Du, “Multistep flow prediction on car-sharing systems: A multi-graph convolutional neural network with attention mechanism,” 2019.
  * [16] A. K. Haghighat, V. Ravichandra-Mouli, P. Chakraborty, Y. Esfandiari, S. Arabi, and A. Sharma, “Applications of deep learning in intelligent transportation systems,” _Journal of Big Data Analytics in Transportation_ , vol. 2, no. 2, pp. 115–145, 2020.
  * [17] S. Wang, K. Wei, L. Lin, and W. Li, “Spatial-temporal analysis of COVID-19’s impact on human mobility: the case of the united states,” in _The 20th and 21st Joint COTA International Conference of Transportation Professionals_ , 2021.
  * [18] L. Lin, F. Shi, and W. Li, “Assessing inequality, irregularity, and severity regarding road traffic safety during covid-19,” _Scientific Reports_ , vol. 11, p. 13147, 2021.
  * [19] C. Szegedy, W. Zaremba, I. Sutskever, J. Bruna, D. Erhan, I. Goodfellow, and R. Fergus, “Intriguing properties of neural networks,” _arXiv preprint arXiv:1312.6199_ , 2013.
  * [20] K. Eykholt, I. Evtimov, E. Fernandes, B. Li, A. Rahmati, C. Xiao, A. Prakash, T. Kohno, and D. Song, “Robust physical-world attacks on deep learning visual classification,” in _Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition_ , 2018, pp. 1625–1634.
  * [21] C. Song and V. Shmatikov, “Fooling ocr systems with adversarial text images,” _arXiv preprint arXiv:1802.05385_ , 2018.
  * [22] S. Dadras, R. M. Gerdes, and R. Sharma, “Vehicular platooning in an adversarial environment,” in _Proceedings of the 10th ACM Symposium on Information, Computer and Communications Security_ , 2015, pp. 167–178.
  * [23] I. J. Goodfellow, J. Shlens, and C. Szegedy, “Explaining and harnessing adversarial examples,” _arXiv preprint arXiv:1412.6572_ , 2014.
  * [24] A. Kurakin, I. Goodfellow, and S. Bengio, “Adversarial machine learning at scale,” _arXiv preprint arXiv:1611.01236_ , 2016.
  * [25] N. Papernot, P. McDaniel, and I. Goodfellow, “Transferability in machine learning: from phenomena to black-box attacks using adversarial samples,” _arXiv preprint arXiv:1605.07277_ , 2016.
  * [26] “Caltrans performance measurement system (pems),” http://pems.dot.ca.gov, accessed: 2021-02-28.
  * [27] L. Zhao, Y. Song, C. Zhang, Y. Liu, P. Wang, T. Lin, M. Deng, and H. Li, “T-gcn: A temporal graph convolutional network for traffic prediction,” _IEEE Transactions on Intelligent Transportation Systems_ , vol. 21, no. 9, pp. 3848–3858, 2019.
  * [28] X. Ma, Z. Dai, Z. He, J. Ma, Y. Wang, and Y. Wang, “Learning traffic as images: a deep convolutional neural network for large-scale transportation network speed prediction,” _Sensors_ , vol. 17, no. 4, p. 818, 2017.
  * [29] Z. Cui, K. Henrickson, R. Ke, and Y. Wang, “Traffic graph convolutional recurrent neural network: A deep learning framework for network-scale traffic learning and forecasting,” _IEEE Transactions on Intelligent Transportation Systems_ , 2019.
  * [30] Y. Wang, D. W. M. Chia, and Y. Ha, “Vulnerability of deep learning model based anomaly detection in vehicle network,” in _2020 IEEE 63rd International Midwest Symposium on Circuits and Systems (MWSCAS)_. IEEE, 2020, pp. 293–296.
  * [31] R. A. Rixey, “Station-level forecasting of bikesharing ridership: Station network effects in three us systems,” _Transportation research record_ , vol. 2387, no. 1, pp. 46–55, 2013.
  * [32] Y. Kamarianakis, W. Shen, and L. Wynter, “Real-time road traffic forecasting using regime-switching space-time models and adaptive lasso,” _Applied stochastic models in business and industry_ , vol. 28, no. 4, pp. 297–315, 2012\.
  * [33] B. L. Smith and M. J. Demetsky, “Traffic flow forecasting: comparison of modeling approaches,” _Journal of transportation engineering_ , vol. 123, no. 4, pp. 261–266, 1997.
  * [34] K. He, X. Zhang, S. Ren, and J. Sun, “Deep residual learning for image recognition,” in _Proceedings of the IEEE conference on computer vision and pattern recognition_ , 2016, pp. 770–778.
  * [35] Z. Wang, W. Yan, and T. Oates, “Time series classification from scratch with deep neural networks: A strong baseline,” in _2017 International joint conference on neural networks (IJCNN)_. IEEE, 2017, pp. 1578–1585.
  * [36] A. Paszke, S. Gross, S. Chintala, G. Chanan, E. Yang, Z. DeVito, Z. Lin, A. Desmaison, L. Antiga, and A. Lerer, “Automatic differentiation in pytorch,” 2017.
  * [37] G. W. Ding, L. Wang, and X. Jin, “Advertorch v0. 1: An adversarial robustness toolbox based on pytorch,” _arXiv preprint arXiv:1902.07623_ , 2019.
  * [38] D. P. Kingma and J. Ba, “Adam: A method for stochastic optimization,” in _International Conference on Learning Representations_ , 2015.
  * [39] L. Lin, W. Li, and S. Peeta, “Efficient data collection and accurate travel time estimation in a connected vehicle environment via real-time compressive sensing,” _Journal of Big Data Analytics in Transportation_ , vol. 1, no. 2, pp. 95–107, 2019.
  * [40] W. Li, D. Wolinski, and M. C. Lin, “City-scale traffic animation using statistical learning and metamodel-based optimization,” _ACM Trans. Graph._ , vol. 36, no. 6, pp. 200:1–200:12, Nov. 2017.
  * [41] C. Wu, A. M. Bayen, and A. Mehta, “Stabilizing traffic with autonomous vehicles,” in _2018 IEEE International Conference on Robotics and Automation (ICRA)_. IEEE, 2018, pp. 6012–6018.
  * [42] W. Li, D. Wolinski, and M. C. Lin, “ADAPS: Autonomous driving via principled simulations,” in _IEEE International Conference on Robotics and Automation (ICRA)_ , 2019, pp. 7625–7631.
  * [43] D. Wilkie, J. Sewall, W. Li, and M. C. Lin, “Virtualized traffic at metropolitan scales,” _Frontiers in Robotics and AI_ , vol. 2, p. 11, 2015\.
  * [44] Q. Chao, H. Bi, W. Li, T. Mao, Z. Wang, M. C. Lin, and Z. Deng, “A survey on visual traffic simulation: Models, evaluations, and applications in autonomous driving,” _Computer Graphics Forum_ , vol. 39, no. 1, pp. 287–308, 2020.


Measurements of inclusive and differential cross-sections of combined
$t\bar{t}\gamma$ and $tW\gamma$ production in the $e\mkern-2.0mu\mu$ channel
at 13 TeV with the ATLAS detector Inclusive and differential cross-sections
for the production of top quarks in association with a photon are measured
with proton–proton collision data corresponding to an integrated luminosity of
139 fb-1. The data were collected by the ATLAS detector at the LHC during Run
2 between 2015 and 2018 at a centre-of-mass energy of 13 TeV. The measurements
are performed in a fiducial volume defined at parton level. Events with
exactly one photon, one electron and one muon of opposite sign, and at least
two jets, of which at least one is $b$-tagged, are selected. The fiducial
cross-section is measured to be $39.6\,^{+2.7}_{-2.3}\,\textrm{fb}$.
Differential cross-sections as functions of several observables are compared
with state-of-the-art Monte Carlo simulations and next-to-leading-order
theoretical calculations. These include cross-sections as functions of photon
kinematic variables, angular variables related to the photon and the leptons,
and angular separations between the two leptons in the event. All measurements
are in agreement with the predictions from the Standard Model. TOPQ-2020-03
JHEP CERN-EP-2020-100 JHEP 09 (2020) 049 10.1007/JHEP09(2020)049

##  1 Introduction

Precise measurements of top-quark production and decay properties provide
crucial information for testing the predictions of the Standard Model (SM) and
its possible extensions. In particular, the study of the associated production
of a top-quark pair ($t\bar{t}$) with a high-energy photon probes the
$t\gamma$ electroweak coupling. Furthermore, measurements of the inclusive and
differential cross-sections of this process are of particular interest because
these topologies are sensitive, for instance, to new physics through anomalous
dipole moments of the top quark [Baur:2004uw, Bouzas:2012av, Schulze:2016qas]
and in the context of effective field theories [Bylund:2016phk].

First evidence for the production of $t\bar{t}$ in association with a photon
($t\bar{t}\gamma$) was reported by the CDF Collaboration [Aaltonen:2011sp],
while the observation of the $t\bar{t}\gamma$ process was established by the
ATLAS Collaboration in proton–proton ($pp$) collisions at
$\sqrt{s}=7\leavevmode\nobreak\ \text{Te\kern-1.00006ptV}$ [TOPQ-2012-07].
Both the ATLAS and CMS Collaborations measured the $t\bar{t}\gamma$ cross-
section at $\sqrt{s}=8\leavevmode\nobreak\ \text{Te\kern-1.00006ptV}$
[TOPQ-2015-21, CMS-TOP-14-008]. First measurements of the inclusive and
differential cross-sections at $\sqrt{s}=13\leavevmode\nobreak\
\text{Te\kern-1.00006ptV}$ were performed by the ATLAS Collaboration
[TOPQ-2017-14].

This paper presents a measurement of the fiducial inclusive and differential
combined $t\bar{t}\gamma$ \+ $tW\gamma$ production cross-sections in the final
state with one electron and one muon, referred to as the $e\mkern-2.0mu\mu$
channel. Events where the electrons and muons arise from the leptonic decays
of $\tau$-leptons are considered as background. The measurement is performed
using the full data set recorded at the LHC between 2015 and 2018 at a centre-
of-mass energy of $\sqrt{s}=13\leavevmode\nobreak\ \text{Te\kern-1.00006ptV}$
and corresponding to an integrated luminosity of 139 fb-1. The fiducial
inclusive cross-section is measured using a profile likelihood fit to the
distribution of $S_{\text{T}}$, defined as the scalar sum of all transverse
momenta in the event, including leptons, photons, jets and missing transverse
momentum. The differential cross-sections, absolute and normalised to unity,
are measured in the same fiducial region as the inclusive cross-section, as
functions of photon kinematic variables, angular variables related to the
photon and the leptons, and angular separations between the two leptons in the
event.

Compared to the previous $t\bar{t}\gamma$ ATLAS analysis with 13 TeV data
[TOPQ-2017-14], only the $e\mu$ channel is considered since it provides a
clean final state with a small background contribution and, thus, no
multivariate analysis techniques are needed to separate signal and background
processes. Additionally, the cross-sections are measured at parton level
rather than at particle level to allow comparison with the theory calculation
in Refs. [Bevilacqua:2018woc, Bevilacqua:2018dny]. The calculation constitutes
the first full computation for $t\bar{t}$ production with a hard final-state
photon in hadronic collisions at next-to-leading order (NLO) in quantum
chromodynamics (QCD), $pp\rightarrow bWbW\gamma$, including all resonant and
non-resonant diagrams, interferences, and off-shell effects of the top quarks
and the $W$ bosons. Therefore, in this paper the combined cross-section of
resonant $t\bar{t}\gamma$ and non-resonant $tW\gamma$ production is measured,
referred to as signal in the following. Example Feynman diagrams at leading
order in QCD for $t\bar{t}\gamma$ and $tW\gamma$ production are shown in
Figure 1.

The paper is organised as follows. The ATLAS detector is briefly introduced in
Section 2. Details of the event-simulation generators and their theoretical
predictions are given in Section 3. The event selection and the analysis
strategy are presented in Sections 4 and 5. The systematic uncertainties are
described in Section 6. The results for the fiducial inclusive and
differential cross-sections are presented in Sections 7 and
LABEL:sec:differential, respectively. Finally, a summary is given in Section
LABEL:sec:conclusion.

Figure 1:  Example Feynman diagrams at leading order for $t\bar{t}\gamma$
(left) and $tW\gamma$ production (right) in the $e\mu$ channel. The top-quark
mass resonances are marked with double-lined arrows, while $W$ bosons are
marked in red.

##  2 ATLAS detector

ATLAS [PERF-2007-01, ATLAS-TDR-19, Abbott:2018ikt] is a multipurpose detector
with a forward–backward symmetric cylindrical geometry with respect to the LHC
beam axis.111 ATLAS uses a right-handed coordinate system with its origin at
the nominal interaction point (IP) in the centre of the detector and the
$z$-axis along the beam pipe. The $x$-axis points from the IP to the centre of
the LHC ring, and the $y$-axis points upwards. Cylindrical coordinates
$(r,\phi)$ are used in the transverse plane, $\phi$ being the azimuthal angle
around the $z$-axis. The pseudorapidity is defined in terms of the polar angle
$\theta$ as $\eta=-\ln\tan(\theta/2)$. Angular distance is measured in units
of $\Delta R\equiv\sqrt{(\Delta\eta)^{2}+(\Delta\phi)^{2}}$. The innermost
layers consist of tracking detectors in the pseudorapidity range $|\eta|<2.5$.
This inner detector (ID) is surrounded by a thin superconducting solenoid that
provides a $2\text{\,}\mathrm{T}$ axial magnetic field. It is enclosed by the
electromagnetic and hadronic calorimeters, which cover $|\eta|<4.9$. The
outermost layers of ATLAS consist of an external muon spectrometer within
$|\eta|<2.7$, incorporating three large toroidal magnetic assemblies with
eight coils each. The field integral of the toroids ranges between 2.0 and 6.0
Tm for most of the acceptance. The muon spectrometer includes precision
tracking chambers and fast detectors for triggering. A two-level trigger
system [TRIG-2016-01] reduces the recorded event rate to an average of
$1\text{\,}\mathrm{kHz}$.

##  3 Signal and background modelling

The estimation of signal and background contributions relies on the modelling
of these processes with simulated events produced with Monte Carlo (MC) event
generators. The response of the ATLAS detector was simulated [SOFT-2010-01]
with Geant4 [Agostinelli:2002hh]. For some of the estimates of modelling
uncertainties, the fast-simulation package AtlFast-II was used instead of the
full detector simulation. Additional $pp$ interactions (pile-up) were
generated with Pythia 8 [Sjostrand:2006za:mod, Sjostrand:2014zea] using a set
of tuned parameters called the A3 tune [ATL-PHYS-PUB-2016-017] and the
NNPDF2.3LO parton distribution function (PDF) set [Pumplin:2002vw].
Corrections to the pile-up profile, selection efficiencies, energy scales and
resolutions derived from dedicated data samples are applied to the MC
simulation to improve agreement with data.

This analysis uses both _inclusive_ samples, in which processes were generated
at matrix-element (ME) level without explicitly including a photon in the
final state, and _dedicated_ samples for certain processes, where photons were
included in the ME-level generation step. Dedicated samples with a photon in
the ME were generated for the $t\bar{t}\gamma$ and $tW\gamma$ final states, as
well as for $V\gamma$ processes with additional jets. Here, $V$ denotes either
a $W$ or a $Z$ boson. Although no photons were generated at ME level in the
inclusive samples, initial- and final-state radiation of photons is accounted
for by the showering algorithm. Combining inclusive and dedicated samples for
the modelling of processes might result in double-counting photon radiation in
certain phase-space regions. As a consequence, a procedure to remove overlaps
between the inclusive and dedicated samples was performed. Photon radiation
simulated at ME level in dedicated samples achieves higher accuracy than the
photon radiation in the showering algorithm. On the other hand, kinematic
requirements are applied to the kinematic properties of the photons at ME
level in the dedicated samples. In the overlap-removal procedure, all events
from the dedicated samples are kept while events from the inclusive samples
are discarded if they contain a parton-level photon that fulfils the dedicated
samples’ kinematic requirements of
$p_{\text{T}}(\gamma)>$15\text{\,}\text{Ge\kern-1.00006ptV}$$ and $\Delta
R(\gamma,\ell)>0.2$, where $p_{\text{T}}(\gamma)$ is the photon’s transverse
momentum and $\Delta R(\gamma,\ell)$ is the angular distance between the
photon and any charged lepton.

The dedicated sample for the $t\bar{t}\gamma$ signal process was simulated
using the MadGraph5_aMC@NLO generator (v2.3.3) [Alwall:2014hca] and the
NNPDF2.3LO PDF set at leading order (LO) in QCD. The events were generated as
a doubly resonant $2\to 7$ process, e.g. as $pp\to b\ell\nu b\ell\nu\gamma$,
thus, diagrams where the photon is radiated from the initial state (in the
case of quark–antiquark annihilation), intermediate top quarks, the
$b\text{-quarks}$, and the intermediate $W$ bosons, as well as the decay
products of the $W$ bosons, are included. To prevent divergences, the photon
was required to have $p_{\text{T}}>15\leavevmode\nobreak\
\text{Ge\kern-1.00006ptV}$ and $|\eta|<5.0$ and the leptons to satisfy
$|\eta|<5.0$. The $\Delta R$ between the photon and any of the charged
particles among the seven final-state particles were required to be greater
than 0.2. The top-quark mass in this and all other samples was set to . The
renormalisation and the factorisation scales were set to
0.5$\times\sum_{i}\sqrt{m^{2}_{i}+p^{2}_{\text{T},i}}$, where the sum runs
over all the particles generated from the ME calculation. The event generation
was interfaced to Pythia 8 (v8.212) using the A14 tune [ATL-PHYS-PUB-2014-021]
to model parton showers, hadronisation, fragmentation and the underlying
event. Heavy-flavour hadron decays were modelled with EvtGen [Lange:2001uf];
this program was used for all samples, except for those generated using the
Sherpa MC program [Gleisberg:2008ta:mod, Hoeche:2009rj]. In the latter case,
heavy-flavour decays were modelled directly with Sherpa.

Two dedicated samples for the $tW\gamma$ process were generated with the
MadGraph5_aMC@NLO generator as well. The first one was produced at LO in the
five-flavour scheme for the $2\rightarrow 3$ process (e.g. $pp\to tW\gamma$)
assuming a stable top quark. The second set of events was generated at LO as a
$2\to 6$ process (e.g. $pp\to b\ell\nu\ell\nu\gamma$) in the five-flavour
scheme, where the photon is radiated from any other charged final-state
particle. In the five-flavour scheme, the $b\text{-quarks}$ are treated as
massless and the LO representation of the process includes a $b\text{-quark}$
in the initial state. The two sets of events are complementary and, once
combined, provide a full simulation of the $tW\gamma$ process. Both samples
make use of the NNPDF2.3LO PDF set and were interfaced to Pythia 8 (v8.212)
for parton showering using the A14 tune. The photon was also required to have
$p_{\text{T}}>15\leavevmode\nobreak\ \text{Ge\kern-1.00006ptV}$ and
$|\eta|<5.0$ and to be separated by $\Delta R>0.2$ from any parton. Although
possible interference effects between $t\bar{t}\gamma$ and $tW\gamma$ are
still missing in the simulated LO samples, the $tW\gamma$ process is treated
as part of the signal in this analysis.

Events with $W\gamma$ and $Z\gamma$ final states (with additional jets) were
simulated as dedicated samples. The $W\gamma$ processes were simulated with
Sherpa 2.2.2 at NLO accuracy in QCD using the NNPDF3.0NNLO PDF set, whereas
$Z\gamma$ events were generated with Sherpa 2.2.4 at LO in QCD with the same
PDF set. The samples are normalised to the cross-sections given by the
corresponding MC simulation. The Sherpa generator performs all steps of the
event generation, from the hard process to the observable particles. All
samples were matched and merged by the Sherpa-internal parton showering based
on Catani–Seymour dipoles [Gleisberg:2008fv, Schumann:2007mg] using the
MEPS@NLO prescription [Hoeche:2011fd, Catani:2001cc, Hoeche:2012yf:mod].
Virtual corrections for the NLO accuracy in QCD in the matrix element were
provided by the OpenLoops library [Cascioli:2011va:mod, Denner:2016kdg].

Inclusive $t\bar{t}$ production processes were simulated at matrix-element
level at NLO accuracy in QCD using Powheg-Box v2 [Nason:2004rx,
Frixione:2007vw, Alioli:2010xd]. The calculation used the NNPDF3.0NLO PDF set
[Ball:2014uwa]. The parton shower was generated with Pythia 8 (v8.230), for
which the A14 tune [ATL-PHYS-PUB-2012-003] was used. The $t\bar{t}$ events are
normalised to a cross-section value calculated with the Top++2.0 program at
next-to-next-to-leading order (NNLO) in perturbative QCD, including soft-gluon
resummation to next-to-next-to-leading-logarithm order (see Ref.
[Czakon:2011xx:mod] and references therein).

Events with inclusive $W$\- and $Z$-boson production in association with
additional jets were simulated with Sherpa 2.2.1 [Gleisberg:2008ta:mod,
Hoeche:2009rj] at NLO in QCD. The NNPDF3.0NLO PDF set was used in conjunction
with a dedicated tune provided by the Sherpa authors. The samples are
normalised to the NNLO cross-section in QCD [STDM-2015-03].

Events with two directly produced vector bosons, i.e. $\mathit{WW}$,
$\mathit{WZ}$ and $\mathit{ZZ}$, were generated with Sherpa versions 2.2.2
(purely leptonic decays) and 2.2.1 (all others) at LO in QCD. The NNPDF3.0NNLO
PDF set was used in conjunction with a dedicated tune provided by the Sherpa
authors. The samples are normalised to NLO accuracy cross-sections in QCD
[Campbell:1999ah].

Events with a $t\bar{t}$ pair and an associated $W$ or $Z$ boson ($t\bar{t}V$)
were simulated at NLO at the ME level with MadGraph5_aMC@NLO using the
NNPDF3.0NLO PDF set. The ME generator was interfaced to Pythia 8 (v8.210), for
which the A14 tune was used in conjunction with the NNPDF2.3LO PDF set. The
samples are normalised to NLO in QCD and electroweak theory
[deFlorian:2016spz].

The background processes are sorted into three categories based on the origin
of the reconstructed photon required in the event selection. The three are
estimated from MC simulation by categorising events from all considered
samples that are not classified as signal events. The MC simulations for all
categories include processes without prompt photons such as $t\bar{t}$,
$W$+jets, $Z$+jets, diboson and $t\bar{t}V$ production, as well as background
processes with an additional prompt photon. The first category is labelled _h-
fake_ and contains any type of hadronic fakes that mimic a photon signature in
the detector. This category includes not only photon signatures faked by
hadronic energy depositions in the electromagnetic calorimeter, but also
hadron decays involving photons, for example $\pi^{0}\to\gamma\gamma$ decays.
It also includes processes with a prompt photon, where the prompt photon is
not reconstructed in the detector or does not pass the selection requirements,
but a h-fake photon does. Studies performed with data-driven techniques
following the approach described in Ref. [TOPQ-2017-14] show that possible
data-driven corrections have a negligible effect on the distribution shapes of
relevant observables. Possible differences in the total expected number of
events are covered by a normalisation uncertainty as described in Section 6.
The second category is labelled _e-fake_ and contains processes with an
electron mimicking a photon signature in the calorimeter. Similarly to the
h-fake category, this category includes contributions from processes without a
prompt photon but with an e-fake photon, as well as processes with a prompt
photon in the simulation but an e-fake photon in the reconstruction. This
category represents a minor background contribution. The third category is
called _prompt $\gamma$ background_ and contains any type of background
process with a prompt photon. The background contribution from $t\bar{t}$
production with a photon produced in an additional $pp$ interaction in the
same bunch crossing was found to be negligible. This was estimated by
comparing the significance of the distance in $z$ between the photon’s origin
and the primary vertex in data and simulation.

The $t\bar{t}\gamma$ and $tW\gamma$ events where one or both $W$ bosons decay
into $\tau$-leptons, which then subsequently decay into $e$ or $\mu$, are
categorised as _Other_ $t\bar{t}\gamma$/$tW\gamma$, and not as
$e\mkern-2.0mu\mu$ signal, following the definition of signal events in the
theory calculation in Refs. [Bevilacqua:2018woc, Bevilacqua:2018dny]. Single-
lepton events, where a second lepton is faked by hadronic energy depositions,
are also included in the category _Other_ $t\bar{t}\gamma$/$tW\gamma$. The
contribution of $t\bar{t}\gamma$ single-lepton events was found to be
negligible in the $e\mkern-2.0mu\mu$ final state in the previous measurement
[TOPQ-2017-14] and it is therefore estimated from the MC simulation.

##  4 Event selection

The data set used in this analysis corresponds to the 139 fb-1 of integrated
luminosity collected with the ATLAS detector during the Run 2 period. Each
event in data and simulation is required to have at least one reconstructed
primary vertex with at least two associated reconstructed tracks. Furthermore,
only events where at least one of the single-electron [Aad:2019wsl] or single-
muon [Aad:2020uyd] triggers was fired are selected.

The main physics objects considered in this analysis are electrons, muons,
photons, jets, $b\text{-jets}$ and missing transverse momentum. Electrons are
reconstructed from energy deposits in the electromagnetic calorimeter
associated with reconstructed tracks in the ID system. They are identified
with a combined likelihood technique [EGAM-2018-01] using a ‘tight’ working
point, and are required to be isolated based on calorimeter and tracking
quantities. The $p_{\text{T}}$\- and $\eta$-dependent isolation criteria yield
an efficiency of 90% for electrons with $p_{\text{T}}=25$ GeV and 99% for
those with $p_{\text{T}}=60$ GeV. The origin of the electron track has to be
compatible with the primary vertex. Electrons are calibrated with the method
described in Ref. [EGAM-2018-01]. They are selected if they fulfil
$p_{\text{T}}>$25\text{\,}\mathrm{GeV}$$ and $|\eta_{\mathrm{clus}}|<2.47$,
excluding the calorimeter barrel/endcap transition region
$1.37<|\eta_{\mathrm{clus}}|<1.52$. 222$\eta_{\mathrm{clus}}$ denotes the
pseudorapidity of the calorimeter cell cluster associated with the electron.

Muons are reconstructed with an algorithm that combines the track segments in
the various layers of the muon spectrometer and the tracks in the ID system.
The reconstruction, identification and calibration methods are described in
Ref. [PERF-2015-10]. Muons are required to be isolated according to track- and
calorimeter-based criteria similar to those applied to electrons. Only muons
with calibrated $p_{\text{T}}>$25\text{\,}\mathrm{GeV}$$ and $|\eta|<2.5$ and
passing ‘medium’ quality requirements are considered. The muon track is also
required to originate from the primary collision vertex.

Photons are reconstructed from energy deposits in the central region of the
electromagnetic calorimeters. If the cluster considered is not matched to any
reconstructed track in the ID system, the photon candidate is classified as
unconverted. If the cluster is matched with one or two reconstructed tracks
that are consistent with originating from a photon conversion and if, in
addition, a conversion vertex can be found, the photon candidate is classified
as converted. Both kinds of photons are considered in this analysis. Photons
are reconstructed and identified as described in Ref. [PERF-2017-02] and their
energies are calibrated with the method described in Ref. [PERF-2017-03]. They
are subject to a tight isolation requirement defined as
$\left.E_{\text{T}}^{\mathrm{iso}}\right|_{\Delta R<0.4}<0.022\cdot
E_{\text{T}}(\gamma)+$2.45\text{\,}\mathrm{GeV}$$ in conjunction with
$\left.p_{\text{T}}^{\mathrm{iso}}\right|_{\Delta R<0.2}<0.05\cdot
E_{\text{T}}(\gamma)$, where $E_{\text{T}}^{\mathrm{iso}}$ refers to the
calorimeter isolation within $\Delta R<0.4$ around the direction of the photon
candidate and $p_{\text{T}}^{\mathrm{iso}}$ is the track isolation within
$\Delta R<0.2$ [PERF-2017-02]. Only photons with calibrated
$E_{\text{T}}>$20\text{\,}\mathrm{GeV}$$ and $|\eta_{\mathrm{clus}}|<2.37$,
excluding the calorimeter transition region
$1.37<|\eta_{\mathrm{clus}}|<1.52$, are considered.

Jets are reconstructed using the anti-$k_{t}$ algorithm [Cacciari:2008gp] in
the FastJet implementation [Fastjet] with a distance parameter $R=0.4$. They
are reconstructed from topological clusters of cells in the calorimeter [ATL-
PHYS-PUB-2015-036]. The jet energy scale and jet energy resolution are
calibrated using information from both simulation and data [PERF-2016-04]. The
jets are required to have $p_{\text{T}}>$25\text{\,}\mathrm{GeV}$$ and
$|\eta|<2.5$. Jets with a large contribution from pile-up vertices are
identified with the _Jet Vertex Tagger_ [ATLAS-CONF-2014-018] and rejected.

The $b\text{-tagging}$ algorithm (MV2c10) applied to the selected jets to
identify those from $b\text{-quark}$ hadronisation [ATL-PHYS-PUB-2017-013]
labelled as $b\text{-jets}$ is based on a boosted decision tree combining
information from other algorithms using track impact parameters and secondary
vertices, and a multi-vertex reconstruction algorithm. A working point with a
selection efficiency of 85% on simulated $t\bar{t}$ events is used,
corresponding to rejection factors of 3.1 and 35 for jets initiated by charm
quarks and light-flavour partons, respectively. The flavour-tagging efficiency
for $b\text{-jets}$, as well as for $c$-jets and light-flavour jets, is
calibrated as described in Ref. [PERF-2016-05].

The reconstructed missing transverse momentum $E_{\text{T}}^{\text{miss}}$
[ATLAS-CONF-2018-023, PERF-2016-07] is computed as the negative vector sum
over all reconstructed, fully calibrated physics objects, including photons,
and the remaining unclustered energy, also called the _soft term_. The soft
term is estimated from low-$p_{\text{T}}$ tracks associated with the primary
vertex but not with any reconstructed object.

An overlap-removal procedure is applied to avoid the reconstruction of the
same energy clusters or tracks as different objects. First, electron
candidates sharing their track with a muon candidate are removed and jets
within a $\Delta R=0.2$ cone around any remaining electron are excluded.
Secondly, electrons within a $\Delta R=0.4$ cone around any remaining jet are
removed. If the distance between a jet and any muon candidate is $\Delta
R<0.4$, the muon candidate is discarded if the jet has more than two
associated tracks, otherwise the jet is removed. Finally, photons within a
$\Delta R=0.4$ cone around any remaining electron or muon are removed and then
jets within a $\Delta R=0.4$ cone around any remaining photon are excluded.

The selected events must have exactly one electron and exactly one muon, each
with $p_{\text{T}}>$25\text{\,}\mathrm{GeV}$$. At least one of these leptons
has to be matched to a fired single-lepton trigger. Since the $p_{\text{T}}$
threshold of the single-lepton triggers was increased over the different data-
taking periods due to increased collisions rates, the offline $p_{\text{T}}$
thresholds for these electrons and muons that are matched to a fired single-
lepton trigger are chosen to be $25\leavevmode\nobreak\
\text{Ge\kern-1.00006ptV}$ in 2015, $27\leavevmode\nobreak\
\text{Ge\kern-1.00006ptV}$ in 2016, and $28\leavevmode\nobreak\
\text{Ge\kern-1.00006ptV}$ in 2017 and 2018 in order to lie above the trigger
thresholds. Electrons and muons must have opposite-sign charges and the
$e\mkern-2.0mu\mu$ invariant mass is required to be higher than
$15\leavevmode\nobreak\ \text{Ge\kern-1.00006ptV}$. The event is required to
have at least two jets and at least one of the jets must be $b\text{-tagged}$.
In addition, all events must contain exactly one reconstructed photon
fulfilling the condition that $\Delta R$ between the selected photon and any
of the leptons is greater than 0.4.

The observed event yields after selection are listed in Table 1 for the
different signal and background categories described in Section 3. The LO
cross-section of the MC samples underestimates the expected number of signal
events; therefore, for illustration purposes the combination of all
$t\bar{t}\gamma$ and $tW\gamma$ categories is normalised to match the event
yields in data. Correction factors for detector effects (described in Section
6) are applied, when needed, to improve the description of the data by the
simulation.

Table 1: Event yields before the profile likelihood fit of the signal and background processes to data after the full selection. All categories are estimated from MC simulation and include correction factors for detector effects as described in Section 6. The combination of all $t\bar{t}\gamma$ and $tW\gamma$ categories is scaled to match the event yields in data. The quoted uncertainties correspond to the total statistical and systematic uncertainties (cf. Section 6) added in quadrature. | Events
---|---
$t\bar{t}\gamma$ $e\mu$ |  $2391$${}\pm{}$ | $130$
$tW\gamma$ $e\mu$ |  $156$${}\pm{}$ | $15$
_Other_ $t\bar{t}\gamma$/$tW\gamma$ |  $279$${}\pm{}$ | $15$
h-fake |  $78$${}\pm{}$ | $40$
e-fake |  $23$${}\pm{}$ | $12$
Prompt $\gamma$ bkg. |  $87$${}\pm{}$ | $40$
Total |  $3014$${}\pm{}$ | $160$
Data | 3014

The modelling of signal and background processes is inspected through the
comparison of distributions. A selection of these distributions showing a
comparison between the MC simulation before the profile likelihood fit and
data is presented in Figure 2. The combination of all $t\bar{t}\gamma$ and
$tW\gamma$ categories is normalised to match the event yields in data as done
in Table 1 to allow a comparison of the shapes of the kinematic variables. All
systematic uncertainties that are introduced in Section 6 are included in
these distributions and their sum in quadrature, which assumes they are fully
uncorrelated, is illustrated by the shaded error bands.

Figure 2: Distributions of the transverse momentum of the electron, the muon
and all jets (top row), and the number of jets, $E_{\text{T}}^{\text{miss}}$
and $S_{\text{T}}$ (bottom row) after event selection and before the profile
likelihood fit. The combination of all $t\bar{t}\gamma$ and $tW\gamma$
categories is scaled to match the event yields in data. The shaded bands
correspond to the statistical and systematic uncertainties (cf. Section 6)
added in quadrature. Overflow events are included in the last bin of each
distribution. In the case of the $S_{\text{T}}$ distribution, the underflow
events are included in the first bin. The lower part of each plot shows the
ratio of the data to the prediction.

##  5 Analysis strategy

The inclusive and differential cross-sections are measured in the fiducial
region described in Section 5.1 and the same sources of background
contributions and systematic uncertainties are considered. In the fiducial
inclusive cross-section the $S_{\text{T}}$ distribution is fitted and the
post-fit background yields and systematic uncertainties are used to extract
the signal cross-section, while no fit is performed for the determination of
the differential cross-sections.

###  5.1 Fiducial region definition

The cross-sections are reported at parton level in a fiducial region, defined
by the kinematic properties of the signal process, in which all selected
final-state objects are produced within the detector acceptance. This is done
in a way that mimics the event selection as defined in the theoretical
calculation. Objects at parton level are taken from the MC simulation history.
Photons and leptons are selected as stable particles after final-state
radiation. The leptons ($\ell=e,\mu$) must originate from $W$-boson decays and
they are dressed with nearby photons within a cone of size of $\Delta R=0.1$
around them and must have $p_{\text{T}}>25\leavevmode\nobreak\
\text{Ge\kern-1.00006ptV}$ and $|\eta|<$ 2.5. Only events with exactly one
electron and one muon are considered. Events with leptons originating from an
intermediate $\tau$-lepton in the top-quark decay chain are not considered.
The $b$-jets at parton level in the calculation from Refs.
[Bevilacqua:2018woc, Bevilacqua:2018dny] are jets clustered with the
anti-$k_{t}$ algorithm with a distance parameter of $R=0.4$. Since showering
and hadronisation effects are not considered in this calculation, the jets
correspond to the $b$-quarks from the top-quark decay (with an additional
parton in the cases where the NLO real emission leads to a parton close by a
$b$-quark). To mimic this definition in the LO MC simulation, parton-level
$b$-jets are defined as follows. The anti-$k_{t}$ algorithm with a distance
parameter $R=0.4$ is applied to all partons that are radiated from the two
$b$-quarks (including the $b$-quarks themselves) and from the two initial
partons. The jets that include a $b$-quark from the decay of a top quark are
selected as $b$-jets. The event is kept if there are two $b$-jets satisfying
$p_{\text{T}}>25\leavevmode\nobreak\ \text{Ge\kern-1.00006ptV}$ and
$|\eta|<2.5$. Exactly one photon with $E_{\text{T}}>20\leavevmode\nobreak\
\text{Ge\kern-1.00006ptV}$ and $|\eta|<2.37$ is required. Photons are required
to be isolated from nearby jets by imposing a modified cone approach as
described in Ref. [Frixione:1998jh], as it is also done in the theory
calculation in Refs. [Bevilacqua:2018woc, Bevilacqua:2018dny], to ensure soft
and collinear safety. The event is dropped if any of the following
requirements is not fulfilled: $\Delta R(\gamma,\ell)>0.4$, $\Delta
R(e,\mu)>0.4$, $\Delta R(b,b)>0.4$ or $\Delta R(\ell,b)>0.4$.

###  5.2 Fiducial inclusive cross-section

The fiducial inclusive cross-section is extracted using a binned profile
likelihood fit to the full $S_{\text{T}}$ distribution. The distribution of
$S_{\text{T}}$ provides good separation between signal and background and was
found to be less sensitive to systematic uncertainties than other
distributions considered, such as the jet multiplicity or the $p_{\text{T}}$
of individual jets. The expected signal and background distributions are
modelled in the fit using template distributions taken from the simulated
samples. The parameter of interest, the fiducial cross-section
$\sigma_{\text{fid}}$, is related to the number of signal events in bin $i$ of
the $S_{\text{T}}$ distribution as:

$N_{i}^{s}=L\times\sigma_{\text{fid}}\times C\times f_{i}^{S_{\text{T}}}.$

The term $L$ is the integrated luminosity, $f_{i}^{S_{\text{T}}}$ is the
fraction of generated signal events falling into bin $i$ of the $S_{\text{T}}$
distribution after fiducial requirements are applied, and $C$ is the
correction factor for the signal efficiency $\epsilon$ and for migration into
the fiducial region $f_{\textrm{out}}$, defined as follows:

$\displaystyle f_{\textrm{out}}=\frac{N_{\textrm{reco}}^{\textrm{non-
fid}}}{N_{\textrm{reco}}}\,,\quad\quad\epsilon=\frac{N_{\textrm{reco}}^{\textrm{fid}}}{N_{\textrm{MC}}^{\textrm{fid}}}\quad\quad\Rightarrow
C=\frac{\epsilon}{1-f_{\textrm{out}}}=\frac{N_{\textrm{reco}}}{N_{\textrm{MC}}^{\textrm{fid}}}\,,$

where $N_{\textrm{reco}}$ is the number of simulated signal events passing the
event selection described in Section 4, $N_{\textrm{MC}}^{\textrm{fid}}$ is
the corresponding number of signal events generated in the fiducial region
defined in Section 5.1, and $N_{\textrm{reco}}^{\textrm{fid}}$ and
$N_{\textrm{reco}}^{\textrm{non-fid}}$ are the numbers of signal events that
pass the event selection and are generated within and outside the fiducial
region, respectively. The efficiency and outside migration are obtained from
simulated $t\bar{t}\gamma$ and $tW\gamma$ events. The correction factor is
estimated from the signal simulation to be $C=0.462\pm 0.002$ (statistical
uncertainty only).

The likelihood function $\mathcal{L}$, based on Poisson statistics, is given
by:

$\mathcal{L}=\prod_{i}P\left(N_{i}^{\text{obs}}|N_{i}^{s}(\vec{\theta})+\sum_{b}N_{i}^{b}(\vec{\theta})\right)\times\prod_{t}G(0|\theta_{t},1),$

where $N_{i}^{\text{obs}}$, $N_{i}^{s}$, and $N_{i}^{b}$ are the observed
number of events in data, the predicted number of signal events, and the
estimated number of background events in bin $i$ of the $S_{\text{T}}$
distribution, respectively. The rates of those $t\bar{t}\gamma$ and $tW\gamma$
events not counted as part of the signal and categorised as _Other_
$t\bar{t}\gamma$/$tW\gamma$ are scaled with the same parameter as the signal
events in the fit, i.e. no independent production cross-section is assumed for
these parts of the simulated $t\bar{t}\gamma$/$tW\gamma$ process. The vector
$\vec{\theta}$, of components $\theta_{t}$, represents the nuisance parameters
that describe the sources of systematic uncertainties. Each nuisance parameter
$\theta_{t}$ is constrained by a Gaussian distribution, $G(0|\theta_{t},1)$.
The width of the Gaussian function corresponds to a change of $\pm$1 standard
deviation of the corresponding quantity in the likelihood. For systematic
uncertainties related to the finite number of simulated MC events, the
Gaussian terms in the likelihood are replaced by Poisson terms. The cross-
section is measured by profiling the nuisance parameters and minimising
$-2\ln\mathcal{L}$ [Cowan:2010js:fixed].

###  5.3 Absolute and normalised differential cross-sections

The measurements of the absolute and normalised differential cross-sections
are performed as functions of the $p_{\text{T}}$ and $|\eta|$ of the photon,
and of angular variables between the photon and the leptons: $\Delta R$
between the photon and the closest lepton $\Delta
R(\gamma,\ell)_{\mathrm{min}}$, as well as $\Delta\phi(\ell,\ell)$ and
$|\Delta\eta(\ell,\ell)|$ between the two leptons. The kinematic properties of
the photon are sensitive to the $t\gamma$ coupling. In particular, $\Delta
R(\gamma,\ell)_{\mathrm{min}}$ is related to the angle between the top quark
and the radiated photon, which could give insight into the structure of this
coupling. The distributions of $\Delta\phi(\ell,\ell)$ and
$|\Delta\eta(\ell,\ell)|$ are sensitive to the $t\bar{t}$ spin correlation.
The corresponding distributions in data and SM simulations are compared in
Figure 3. The simulation describes reasonably well the data within the
uncertainties although it favours smaller $\Delta
R(\gamma,\ell)_{\mathrm{min}}$ and larger $\Delta\phi(\ell,\ell)$ values than
the observed ones.

Figure 3: Distributions of the photon $p_{\text{T}}$ and $|\eta|$ in the top
row, and $\Delta R(\gamma,\ell)_{\mathrm{min}}$, $\Delta\phi(\ell,\ell)$ and
$|\Delta\eta(\ell,\ell)|$ in the bottom row after event selection and before
the profile likelihood fit. The combination of all $t\bar{t}\gamma$ and
$tW\gamma$ categories is scaled to match the event yields in data. The shaded
bands correspond to the statistical and systematic uncertainties (cf. Section
6) added in quadrature. When overflow events are present, they are included in
the last bin of the distribution. The lower part of each plot shows the ratio
of the data to the prediction.

The data are corrected for detector resolution and acceptance effects to
parton level in the fiducial phase space using an iterative matrix unfolding
that uses Bayes’ theorem [DAgostini:1994fjx] implemented in the RooUnfold
package [Adye:2011gm]. The differential cross-section is defined as:

$\frac{\mathrm{d}\sigma}{\mathrm{d}X_{k}}=\frac{1}{L\times\Delta
X_{k}\times\epsilon_{k}}\times\sum_{j}M_{jk}^{-1}\times(N^{\mathrm{obs}}_{j}-N^{b}_{j})\times
f_{e\mu,j}\times(1-f_{\mathrm{out},j})\,.$

The indices $j$ and $k$ represent the bin indices of the observable $X$ at
detector and parton levels, respectively. The variable $N^{\textrm{obs}}_{j}$
is the number of observed events, and $N^{b}_{j}$ is the number of estimated
non-$t\bar{t}\gamma$/$tW\gamma$ background events (pre-fit) in bin $j$ at
detector level. The contribution from the _Other_ $t\bar{t}\gamma$/$tW\gamma$
category is taken into account by correcting the remaining number of observed
events by the signal fraction, $f_{e\mu,j}$, defined as the ratio of the
number of selected $t\bar{t}\gamma$ and $tW\gamma$ $e\mkern-2.0mu\mu$ events
to the total number of selected $t\bar{t}\gamma$ and $tW\gamma$ events, as
determined from simulation. This avoids the dependence on the signal cross-
section used for the normalisation. The efficiency $\epsilon_{k}$ is the
fraction of signal events generated at parton level in bin $k$ of the fiducial
region that are reconstructed and selected at detector level. The total
integrated luminosity is denoted by $L$, and $\Delta X_{k}$ represents the bin
width. The migration matrix $M_{kj}$ describes the detector response and
expresses the probability for an event in bin $k$ at parton level to be
reconstructed in bin $j$ at detector level, calculated from events passing
both the fiducial-region selection and the event selection. The outside-
migration fraction $f_{\mathrm{out},j}$ is the fraction of signal events
generated outside the fiducial region but reconstructed and selected in bin
$j$ at detector level. The normalised differential cross-section is derived by
dividing the absolute result by the total cross-section, obtained by
integrating over all bins of the observable.

The signal MC samples are used to determine $\epsilon_{k}$,
$f_{\mathrm{out},j}$, and $M_{kj}$. The unfolding method relies on the
Bayesian probability formula, starting from a given prior of the parton-level
distribution and iteratively updating it with the posterior distribution. The
binning choices of the unfolded observables take into account the detector
resolution and the expected statistical uncertainty. The bin width has to be
larger than twice the resolution, and the statistical uncertainty is required
to be around or below 10% across all bins, with the latter being the limiting
factor in most of the cases. The resolution of the lepton and photon momenta
is very high and, therefore, the fraction of events migrating from one bin to
another is small. In all bins, the purity, defined as the fraction of
reconstructed events that originate from the same bin at parton level, is
larger than 80%, and it is above 90% for all observables except for the
$p_{\text{T}}$ of the photon. The number of iterations chosen is two, which
provides good convergence of the unfolding distribution and a statistically
stable result. For illustration purposes, the migration matrix is presented in
the left panel of Figure 4, while the right panel shows the efficiency,
outside-migration fraction and the resulting $C$ correction factor obtained
for the distribution of the photon $p_{\text{T}}$. The performance of the
unfolding procedure is tested for possible biases from the choice of input
model. It was verified that when reweighting the shape of the signal
simulation by up to 50% bin-by-bin with respect to the nominal shape, the
unfolding procedure based on the nominal response matrix reproduces the
altered shapes.

Figure 4: Left: migration matrix relating the photon $p_{\text{T}}$ at the
reconstruction and parton levels in the fiducial phase space, normalised by
column and shown as percentages. Right: signal reconstruction and selection
efficiency ($\epsilon$), ($1-f_{\mathrm{out}}$) fraction and resulting $C$
correction factor as a function of the photon $p_{\text{T}}$.

##  6 Systematic uncertainties

Various systematic uncertainties arising from detector effects are considered,
along with theoretical uncertainties. Signal and background predictions are
both subject to these uncertainties.

###  6.1 Experimental uncertainties

Experimental systematic uncertainties affect the normalisation and shape of
the distributions of the simulated signal and background samples. These
include reconstruction and identification efficiency uncertainties, as well as
uncertainties in the energy and momentum scale and resolution for the
reconstructed physics objects in the analysis, including leptons, photons,
jets and $E_{\text{T}}^{\text{miss}}$. In addition, uncertainties in the
flavour-tagging of jets, the jet vertex tagger (JVT) discriminant, the
integrated luminosity value and the pile-up simulation are considered.

The photon identification and isolation efficiencies as well as the
efficiencies of the lepton reconstruction, identification, isolation, and
trigger in the MC samples are all corrected using scale factors to match the
corresponding values in data. Similarly, corrections to the lepton and photon
momentum scale and resolution are applied in simulation [PERF-2017-03,
PERF-2015-10]. All these corrections, which are $p_{\text{T}}$ and $\eta$
dependent, are varied within their uncertainties.

The jet energy scale (JES) uncertainty is derived using a combination of
simulations, test-beam data and in situ measurements [PERF-2016-04].
Additional contributions from jet-flavour composition,
$\eta$-intercalibration, punch-through, single-particle response, calorimeter
response to different jet flavours, and pile-up are taken into account,
resulting in 30 uncorrelated JES uncertainty subcomponents, of which 29 are
non-zero in a given event depending on the type of simulation used. The most
relevant JES uncertainties are related to the pile-up correction (JES pile-up
correction) and modelling aspects of the in situ calibration (JES in situ
calibration). The jet energy resolution (JER) in simulation is smeared by the
measured JER uncertainty [ATL-PHYS-PUB-2015-015] split into eight uncorrelated
sources. The uncertainty associated with the JVT discriminant is obtained by
varying the efficiency correction factors (labelled jet vertex tagging in the
results, cf. Figure 5).

The uncertainties related to the $b$-jet tagging calibration are determined
separately for $b$-jets, $c$-jets and light-flavour jets [Aaboud:2018xwy,
ATLAS-CONF-2018-001, ATLAS-CONF-2018-006]. For each jet category, the
uncertainties are decomposed into several uncorrelated components. The
corrections are varied by their measured uncertainties.

The uncertainties associated with energy scales and resolutions of photons,
leptons and jets are propagated to the $E_{\text{T}}^{\text{miss}}$.
Additional uncertainties originate from the modelling of its soft term [ATLAS-
CONF-2013-082].

The uncertainty in the combined 2015–2018 integrated luminosity is 1.7%
[ATLAS-CONF-2019-021], obtained using the LUCID-2 detector [LUCID2] for the
primary luminosity measurements.

The uncertainty associated with the modelling of pile-up in the simulation is
assessed by varying the pile-up reweighting in the simulation within its
uncertainties.

###  6.2 Signal and background modelling uncertainties

The $t\bar{t}\gamma$ signal modelling uncertainties include the uncertainties
owing to the choice of QCD scales, parton shower, amount of initial-state
radiation (ISR), and PDF set. The effect of the QCD scale uncertainty is
evaluated by varying the renormalisation and factorisation scales separately
up and down by a factor of two from their nominal chosen values. The
uncertainty from the parton shower and hadronisation ($t\bar{t}\gamma$ PS
model) is estimated by comparing the $t\bar{t}\gamma$ nominal samples,
produced with MadGraph5_aMC@NLO + Pythia 8, with an alternative sample
interfaced to Herwig 7 [Bahr:2008pv, Bellm:2015jjp]. The ISR uncertainty
($t\bar{t}\gamma$ ISR) is studied by comparing the nominal MadGraph5_aMC@NLO +
Pythia 8 sample with the results of varying the A14 tune parameter for
radiation [ATL-PHYS-PUB-2014-021]. The PDF uncertainty ($t\bar{t}\gamma$ PDF)
is evaluated using the standard deviation in each bin of the respective
distribution formed by the set of 100 replicas of the NNPDF set
[Pumplin:2002vw].

For the $tW\gamma$ process the uncertainties due to the choice of
renormalisation and factorisation scales are also estimated by varying them up
and down separately by a factor of two relative to the nominal sample value. A
systematic uncertainty from the parton shower and hadronisation model is
considered by comparing Pythia 8 and Herwig 7 both interfaced to
MadGraph5_aMC@NLO. The $tW\gamma$ modelling uncertainties are treated as
uncorrelated with the $t\bar{t}\gamma$ signal modelling uncertainties.

The $tW\gamma$ process was generated in the five-flavour scheme at leading
order in QCD and one of the two $b\text{-quarks}$ is not included in the
matrix-element generation step. This $b\text{-quark}$, expected to be produced
in the initial state through the PDF, is only found in a fraction of the
events at parton level in the MC simulation. The fractions of generated
$tW\gamma$ events without a second $b\text{-quark}$ were found to be around
30% and 50% for the MC samples interfaced with Herwig and Pythia,
respectively. Therefore, an additional uncertainty associated with this
possibly lost $b\text{-quark}$ is assigned ($tW\gamma$ parton definition) as
follows. Relative to the nominal $tW\gamma$ simulation, the parton-level event
yields are doubled, assuming all $b$-jets are found, while the number of
reconstructed events is kept constant. This leads to a variation of the
correction factor $C$ of 2.8%.

Several uncertainties in the modelling of $t\bar{t}$ processes, which give a
dominant contribution to the h-fake and prompt $\gamma$ background categories,
are considered as shape-only uncertainties. The uncertainties associated with
the parton shower and hadronisation are estimated by comparing the nominal
simulation with alternative showering by Herwig 7\. Uncertainties in the
modelling of final-state radiation are estimated by evaluating the effects of
varying four different parameters in the Powheg \+ Pythia 8 generator set-up
described in the following. Uncertainties due to the renormalisation and
factorisation scales are estimated by varying them up and down independently
by a factor of two relative to the default scale choice. These scale
variations are implemented with corresponding weights which are available as
part of the nominal MC sample. Uncertainties due to the value of
$\alpha_{\text{S}}$ used in the ISR parton shower modelling are estimated by
comparing the nominal Powheg \+ Pythia 8 simulation with alternative samples
that correspond to higher and lower radiation parameter settings in the A14
tune, controlled by the var3c parameter in Pythia 8\. This parameter is varied
within its uncertainties corresponding to variations of $\alpha_{S}(m_{Z})$
between 0.115 and 0.140. An additional ISR uncertainty is obtained by
comparing the nominal sample with an additional one where the
$h_{\mathrm{damp}}$ parameter, which controls the $p_{\text{T}}$ of the first
additional emission, is varied by a factor of two as supported by measurements
reported in Ref. [ATL-PHYS-PUB-2018-009].

In addition to those background modelling uncertainties, global normalisation
uncertainties of 50% are assigned to the following three categories: h-fake
photons, e-fake photons and prompt $\gamma$ background [TOPQ-2017-14]
(h-fakes, e-fakes, and prompt $\gamma$ normalisation).

###  6.3 Treatment of the systematic uncertainties in the measurements

As stated in Section 5, the impact of systematic uncertainties on the fiducial
inclusive cross-section measurement is taken into account via nuisance
parameters in the likelihood function. The nuisance parameters $\vec{\theta}$
are profiled in the maximum-likelihood fit. Variations of the nuisance
parameters can affect the rate of events as well as the shape of the
$S_{\text{T}}$ distribution. In the case of signal modelling uncertainties,
the rate uncertainty is composed of variations of the efficiency $\epsilon$
and the fraction $f_{\mathrm{out}}$. All MC samples used to evaluate signal
modelling uncertainties are scaled to the same number of events in the
fiducial phase space, $N_{\textrm{MC}}^{\textrm{fid}}$. The only uncertainty
that is not included as a nuisance parameter in the profile likelihood fit is
the uncertainty from the $tW\gamma$ parton definition. This uncertainty does
not affect the number of reconstructed events in the corresponding template in
the profile likelihood fit. It comprises only an uncertainty in the number of
generated events in the fiducial phase space. Thus, the $tW\gamma$ parton
definition uncertainty is added in quadrature to the post-fit uncertainty of
the profile likelihood fit.

To reduce the sensitivity to statistical fluctuations due to the limited
number of events in the MC samples used in systematic variations, _smoothing_
techniques are applied to the MC templates used to evaluate the signal and
background modelling systematic uncertainties in the template fit.
Additionally, the systematic uncertainties are symmetrised, taking the average
of the up- and down-variation as the uncertainty. In the cases where both
variations have the same sign or only one variation is available (e.g. the
uncertainty from the parton shower and hadronisation signal modelling) the
largest variation or the available one, respectively, is taken as both the up-
and down-variations for the corresponding source. The ISR uncertainty suffers
from statistical fluctuations in the available $t\bar{t}\gamma$ MC samples, so
a more conservative approach is chosen for the symmetrisation. In this case,
the largest of the two variations is taken and mirrored around the nominal
prediction.

In the case of the differential cross-section measurements, each systematic
uncertainty is determined individually in each bin of the measurement by
varying the corresponding efficiency, resolution, and model parameter within
its uncertainty. The same symmetrisation approach described for the fiducial
inclusive cross-section is used for this measurement. For each variation, the
measured differential cross-section is recalculated and the deviation from the
nominal result per bin is taken as the systematic uncertainty. The overall
uncertainty in the measurement is then derived by adding all contributions in
quadrature, assuming the sources of systematic uncertainty to be fully
uncorrelated.

Sources of systematic uncertainty relating only to the background prediction
are evaluated by shifting the nominal distribution of the corresponding
background process by its associated uncertainty. For the experimental
uncertainties, the input is varied by the corresponding shift, which typically
affects both the shape and normalisation of signal and background process
distributions. The resulting distribution is unfolded and compared with the
nominal unfolded distribution and the difference is assigned as an
uncertainty. The systematic uncertainties due to signal modelling are
evaluated by varying the signal corrections, i.e. the migration matrix
$M_{kj}$, the efficiency $\epsilon_{k}$ and the fraction $f_{\mathrm{out},j}$,
by the corresponding model parameter uncertainty and calculating the
difference between the resulting unfolded distributions and the nominal ones.

##  7 Fiducial inclusive cross-section measurement

The number of signal events is extracted using a profile likelihood fit to the
$S_{\text{T}}$ distribution and is translated into the signal cross-section in
the fiducial phase space given by the kinematic boundaries of the signal as
described in Section 5.

The best-fit values of the nuisance parameters ranked highest in impact are
shown in Figure 5 along with their impact on the result. Rate and shape
uncertainties from the $t\bar{t}\gamma$ PS model and $t\bar{t}\gamma$ ISR
variations are treated as separate nuisance parameters. This approach prevents
pulls on the rate uncertainty due to differences in the shape of the
$S_{\text{T}}$ distribution between the data and simulation, in particular in
the tail where the data overshoot the prediction and the fit compensates for
this discrepancy by pulling the nuisance parameter of the $t\bar{t}\gamma$ PS
model shape uncertainty. The impact of the individual nuisance parameters is
evaluated as the difference between the reference best-fit value of the cross-
section and the one obtained when fixing the corresponding nuisance parameter
under scrutiny to its best-fit value and its $\pm$ one standard deviation
$(\pm 1\sigma)$. Table 7 shows the systematic uncertainties and their relative
impact on the measurement of the fiducial inclusive cross-section. The effect
of each category of uncertainties is calculated from the variance
$(\sigma^{2})$ difference between the total uncertainty in the measured
fiducial cross-section and the uncertainty from the fit with the corresponding
nuisance parameters fixed to their fitted values. The uncertainties in the
signal modelling, especially the rate uncertainties from the $t\bar{t}\gamma$
PS model and the ISR variation, have the largest impact on the result.

Figure 5:  Ranking of the systematic uncertainties included in the profile likelihood fit used in the fiducial inclusive cross-section measurement. The blue and turquoise bands indicate the post-fit impact on the fit result, whereas the outlined blue and turquoise rectangles show the pre-fit impact. The difference between the two reflects the constraint of the nuisance parameter due to correlations in the fit. Most nuisance parameters are not or only marginally constrained. The impact is overlaid with the post-fit values of the nuisance parameters (pulls) shown by the black dots. The black lines represent the post-fit uncertainties normalised to the pre-fit uncertainties. For uncertainties parameterised with more than one nuisance parameter, the index (1) refers to the leading component.  Table 2: Illustrative summary of the systematic uncertainties on the fiducial inclusive cross-section measurement grouped into different categories and their relative impact on the measurement (symmetrised). The categories ‘$t\bar{t}\gamma$/$tW\gamma$ modelling’ and ‘Background modelling’ include all corresponding systematic uncertainties described in Section 6.2. The ‘$tW\gamma$ parton definition’ uncertainty is listed separately since it does not enter the profile likelihood fit directly as described in Section 6.3. The category ‘Photons’ corresponds to the uncertainties related to photon identification and isolation as well as photon energy scale and resolution. ‘Jets’ includes the total uncertainty from the JES, JER and JVT discriminant, while the $b$-tagging-related uncertainties are given in a separate category (‘Flavour-tagging’). The category ‘Leptons’ represents the uncertainties related to lepton identification, isolation and energy/momentum calibration.  Category | Uncertainty
---|---
$t\bar{t}\gamma$/$tW\gamma$ modelling |