[1]Muhammad Haroon

# Avengers Ensemble! Improving Transferability of Authorship Obfuscation

Muhammad Fareed Zaffar  Lahore University of Management Sciences, E-mail:
<EMAIL_ADDRESS>Padmini Srinivasan  The University of Iowa, E-mail:
<EMAIL_ADDRESS>Zubair Shafiq  University of California, Davis,
E-mail<EMAIL_ADDRESS>
###### Abstract

Stylometric approaches have been shown to be quite effective for real-world
authorship attribution. To mitigate the privacy threat posed by authorship
attribution, researchers have proposed automated authorship obfuscation
approaches that aim to conceal the stylometric artefacts that give away the
identity of an anonymous document’s author. Recent work has focused on
authorship obfuscation approaches that rely on black-box access to an
attribution classifier to evade attribution while preserving semantics.
However, to be useful under a realistic threat model, it is important that
these obfuscation approaches work well even when the adversary’s attribution
classifier is different from the one used internally by the obfuscator.
Unfortunately, existing authorship obfuscation approaches do not transfer well
to unseen attribution classifiers. In this paper, we propose an ensemble-based
approach for transferable authorship obfuscation. Our experiments show that if
an obfuscator can evade an ensemble attribution classifier, which is based on
multiple base attribution classifiers, it is more likely to transfer to
different attribution classifiers. Our analysis shows that ensemble-based
authorship obfuscation achieves better transferability because it combines the
knowledge from each of the base attribution classifiers by essentially
averaging their decision boundaries.

##  1 Introduction

Authorship obfuscation is the process of concealing stylometric pointers in a
text document that may reveal the identity of its author. The problem has
become increasingly relevant today considering the erosion of privacy due to
recent advances in the performance of state-of-the-art authorship attribution
approaches. Sophisticated machine learning models can determine the author of
a given text document [27, 44] using hand-crafted stylometric features [2, 8,
35, 12, 3] or automated features such as word embeddings [42, 23]. State-of-
the-art authorship attribution approaches have achieved impressive results in
a multitude of settings ranging from social media posts [4, 37, 41] to large-
scale settings with up to a 100,000 possible authors [36].

The desire to maintain anonymity in this increasingly hostile environment
motivates the need for effective authorship obfuscation methods. Obfuscation
approaches can be broadly divided into two groups: those that do not rely on
feedback from an authorship attribution classifier and those that do require
such feedback.

In the first group, there are a number of efforts especially from the PAN
digital text forensics initiative [1]. These authorship obfuscation approaches
mostly use rule based transformations (e.g., splitting or joining sentences)
guided by some general criteria, such as moving the text towards some average
point or moving it away from the author’s writing patterns, text
simplification, machine translation, etc. [29, 9, 28, 40]. These approaches
generally struggle with achieving the appropriate trade off between evasion
effectiveness and preserving text semantics.

In the second group, obfuscators that rely on access to an authorship
attribution classifier are more relevant to our research. In a seminal work,
MacDonald et al. [35] proposed Anonymouth – an obfuscator that relies on
access to the attributor JStylo to guide manual text obfuscation. A4NT [43]
proposed a generative adversarial network (GAN) based automated approach to
obfuscation and also requires access to the attribution classifier. More
recently, Mutant-X [33] used a genetic algorithm and ParChoice [22] used
combinatorial paraphrasing for automated obfuscation, and both require access
to an attribution classifier. These methods have shown promise in effectively
evading attribution classifiers while reasonably preserving text semantics.

While prior authorship obfuscation methods can suitably trade off between
evading attribution and preserving semantics, they do not work well when the
adversary uses a different attribution classifier than the one used internally
by the obfuscator [33, 22, 43]. However, it is important that authorship
obfuscators can protect the author’s identity even when the adversary uses a
different attribution classifier. In other words, obfuscation should transfer
to previously unseen attribution classifiers.

The lack of transferability is essentially because of the mismatch between the
obfuscator’s internal classifier and the adversary’s attribution classifier.
To address the transferability issue, our key insight is that if an obfuscator
can evade a meta-classifier, which is based on multiple base classifiers that
target different feature subspaces, it is more likely to evade an unseen
attribution classifier. Building on this insight, we propose an ensemble-based
approach for transferable authorship obfuscation. We explore the design space
of the ensemble using different feature subspaces, base classifiers, and
aggregation techniques. The experimental evaluation shows our ensemble based
authorship obfuscation approach yields state-of-the-art transferability
results. We find that obfuscation by our ensemble approach achieves an
1.7$\times$ and 2.1$\times$ better transferability in terms of the attack
success rate (ASR) than the baseline RFC and SVM attributors. The ensemble
achieves an average METEOR score of 0.36, which is comparable with the RFC at
0.42 and the SVM at 0.40.

We summarize our key contributions and findings as follows:

  1. 1.

We explore the problem of transferability of authorship obfuscation against
unseen attribution classifiers.

  2. 2.

We propose an ensemble approach that consists of multiple base classifiers,
each capturing different feature subspaces, to guide automated text
obfuscation.

  3. 3.

We evaluate the evasion effectiveness, semantic preservation, and
transferability of the ensemble obfuscator and show that it achieves much
better transferability against unseen attribution classifiers than prior
approaches.

##  2 Preliminaries & Methods

Fig. 1: Overview of the threat model involving an obfuscator and multiple
adversaries. The obfuscator relies on access to an internal attribution
classifier. The obfuscator consists of two components: a generator and an
obfuscator. The generator generates an obfuscation and queries the internal
classifier for feedback on its probability of detection. They repeat this for
$M$ iterations and then, if evaded, the final document is verified against the
adversaries.

###  2.1 Authorship Attribution vs. Obfuscation

Stylometry is the analysis of an author’s writing style that helps distinguish
them from other authors. For example, writeprints [2] is a well-known
stylometric feature set that has been used to analyze the writing style for
the sake for authorship attribution. The primary goal of authorship
obfuscation is to evade attribution by concealing such stylometric features in
the document while retaining its original meaning. Early approaches such as
Anonymouth [35] highlighted the distinctive stylometric properties of the text
that could then be modified by the user to evade attribution. Follow up work
at PAN-CLEF aimed to automatically obfuscate documents using simple predefined
rules. For example, Mansoorizadeh et al. [34] used WordNet to identify
synonyms for the most commonly used words by the author and replaced them with
similar words. Castro et al. [9] used sentence simplification techniques, such
as replacing contractions with their expansions, to obfuscate a document.
Keswani et al. [29] used round-trip translation ($English\rightarrow
German\rightarrow French\rightarrow English$) to obfuscate a document. While
these automated approaches managed to evade attribution, they severely
compromised the obfuscated text’s semantics. These approaches, rather
unsuccessfully, navigate the trade-off between evading attribution and
preserving semantics [33].

Recent work such as A4NT [43], Mutant-X[33], and ParChoice [22] employ more
sophisticated adversarial obfuscation to evade authorship attribution
classifiers. Their threat model assumes that the obfuscator can query the
adversary’s attribution classifier to guide obfuscation. For example, A4NT
uses a generative adversarial network (GAN) for obfuscation that requires
white-box access to the adversary’s attribution classifier. Mutant-X uses a
genetic algorithm for obfuscation that requires black-box access to the
adversary’s attribution classifier. ParChoice uses combinatorial paraphrasing
for obfuscation that requires black-box access to adversary’s attribution
classifier. While these obfuscation approaches achieve a better tradeoff
between attribution evasion and preserving semantics, they all assume
white/black box access to the the adversary’s attribution classifier. This key
assumption limits their effectiveness in the real world because the
adversary’s attribution classifier might be different or unknown. For example,
the evasion effectiveness of Mutant-X drops drastically when the adversary
uses a different attribution classifier than assumed by Mutant-X [33].
Similarly, an adversarially retrained attribution classifier is resistant to
obfuscation by ParChoice using the original classifier [22]. This lack of
transferability to unseen attribution classifiers has major ramifications in
the real world as the obfuscator’s effectiveness is questionable when the
adversary happens to use a different attribution classifier. Figure 1 provides
an overview of this threat model involving the obfuscator and multiple unseen
adversaries.

###  2.2 Problem Statement

The obfuscator seeks to obfuscate the stylometric properties of an input
document $D$ of author $A$ by modifying its text to produce an obfuscated
document $D^{\prime}$ such that the attributor incorrectly classifies the
obfuscated document to another author $A^{\prime}\neq A$. The state-of-the-art
authorship obfuscators mainly consist of two components: a generator and an
internal authorship attribution classifier $C$. The generator modifies the
input document based on some rules and queries the internal classifier to
predict whether these modifications would degrade the likelihood of successful
authorship attribution. The two components work in tandem for $M$ iterations
to progressively obfuscate the input document by generating new obfuscation
samples and measuring the degradation in authorship attribution by $C$. It is
noteworthy that the adversary might use a different authorship attribution
classifier $C^{\prime}\neq C$. There could, in fact, be multiple adversaries
in this setting, with each using a different attribution classifier
$C^{\prime}$ than the obfuscator’s internal classifier $C$. The primary goal
of the obfuscator is to obfuscate an input document $D\rightarrow D^{\prime}$
using its internal classifier $C$ such that it evades attribution by the
adversary classifier $C^{\prime}$. This problem is also referred to as
transferability in the field of adversarial machine learning.

###  2.3 Approach

Intuition. The obfuscator relies on feedback from its internal classifier as a
proxy to identify suitable transformations that can help evade attribution by
the adversary’s classifier. These transformations essentially aim to move the
document to the wrong side of the decision boundary, which partition different
author classes, of the obfuscator’s internal classifier. Since these
transformations are specific to the decision boundary of the obfuscator’s
internal classifier, they may not achieve the same result on the adversary’s
attribution classifier. When they do not, the obfuscated document would evade
the attribution classifier of the obfuscator but not that of the adversary.
These differences in the decision boundaries of the two classifiers could be
because of the differences in their emphasis on different features; thus,
transformations that targeted a certain feature emphasized by one classifier
might be rendered useless for the other classifier. To address this issue, our
key insight is that if an obfuscator can evade a meta-classifier, whose
decision boundary is based on the decision boundaries of multiple base
classifiers, it is more likely to evade those base classifiers. We hypothesize
that a meta-classifier consisting of multiple base classifiers, each
emphasizing different features, will better capture the relative importance of
various features. Intuitively, as the internal attribution classifier, this
ensemble of base classifiers can provide a more nuanced view of the entire
feature space and can classify the document in a manner that essentially
averages the decision boundaries of the base classifiers.

Fig. 2: Ensemble architecture for the feature subspacing technique. The
original feature space is split into subspaces that are then used to train the
base classifiers. The outputs of the base classifiers are then aggregated for
the final prediction.

Ensemble Approach. An ensemble is a learning algorithm that takes a set of
classifiers and uses their individual outputs to make the final classification
for a given input. The classifiers in this set are referred to as the base
classifiers for the ensemble. The number of base classifiers affects how the
model fits to the training set: too few and the model is likely to underfit
and too many will likely result in overfitting. An efficient number can be
determined using cross-validation though it is a time-consuming exercise to
train multiple ensembles and validate their results. [24] The base classifiers
can be different classifiers trained on the same training set, or they could
be the same and trained on different subsets of the training set (a technique
known as bagging) or even on subspaces of the feature set [48].

The outputs of the base classifiers are then polled by the ensemble through
either a majority vote or by training another classifier (a technique called
stacking) [16]. A majority vote gives uniform weight to the output of each
base classifier whereas stacking causes the weights to vary, as it can learn
to downplay classifiers that are inaccurate more often. While these base
classifiers might not be very accurate on their own, the ensemble together can
capitalize on their knowledge and make more accurate predictions.

We construct our ensemble using the feature subspace method and describe it as
follows. A subspace is a subset of the entire universal set of features that
are available to the classifier. We train the base classifiers of the ensemble
on different subspaces of the feature set. The goal of using a subspace of
features is to train a base learner that is specialized in that distinct and
local set of features. This is motivated by our findings of feature importance
and decision boundaries which we discuss later in Section 4.2. The subspaces
can be selected randomly [47], through sampling [50], or through feature
selection techniques [17].

Figure 2 illustrates the architecture of our proposed ensemble. The original
feature space is divided into multiple subspaces which are then used to train
the base classifiers. The outputs from these base classifiers are then
aggregated to produce the final classification of the ensemble for the given
input.

##  3 Experimental Setup

In this section, we state the assumptions and describe the setup for our
experiments. Specifically, we describe the dataset we use, the attribution
classifiers used by the obfuscator and the adversaries, the layout of the
experiments, and finally the evaluation metrics used to assess the results.

###  3.1 Data

The Extended Brennan Greenstadt Corpus [7] comprises of writing samples
submitted by various authors through Amazon’s Mechanical Turk (AMT) platform.
The corpus is unique because it was collected expressly for the purpose of
adversarial stylometry in text and was vetted against a strict set of
guidelines imposed by AMT and the authors themselves to ensure quality. The
guidelines required that the submissions be professionally written, be free of
anything other than the writing itself (i.e., citations, URLs, headings,
etc.), and contained at least 6500 words. The imposition of these strict
guidelines ensured that the submissions were of high quality, reflected the
author’s particular writing style, and that there was sufficient data to train
an attribution classifier. Out of the 100 submissions, the authors selected
the 45 that most closely followed the guidelines and then split them into
nearly 500 word passages, averaging 15 documents per author, to create the
final corpus. In our experiments, we divide the corpus into groups of 5
authors based on document length and report results on these groupings by
further splitting them into a 80% training and 20% testing set.

###  3.2 Obfuscator’s Attribution Classifiers

Baseline obfuscator: We use Mutant-X as the baseline obfuscator for our
experiments as its generator only requires black-box access to the attribution
classifier. [33] This loose coupling between the two components allows the
attribution classifier to be easily swapped for another. Additionally, we use
the Writeprints [2] feature set throughout the experiments to train the
internal attribution classifiers for Mutant-X. This feature set incorporates
lexical and syntactic features to capture the stylometric properties of the
author’s writing style. The lexical features include character-level and word-
level features such as total words, average length of words, proportions of
different character classes, among others. The syntactic features include POS
tags, use of function words, and various punctuations.

The fitness function for Mutant-X takes into account the detection probability
of a given attribution classifier $C$ and the semantic similarity between the
original and the obfuscated document. We train the two classifiers that were
originally used for Mutant-X, a random forest classifier and a support vector
machine, on the Writeprints feature set to serve as baselines for comparing
the performance of the ensemble.

Writeprints-Static + Ensemble: We use the same feature set as our baselines to
train the ensemble and construct it by training base classifiers on subspaces
of the entire feature set. We use a linear SVM as the base classifier owing to
its stability and demonstrated use as a base classifier for an ensemble in
prior work [48]. We use the random subspace method [47] to construct the
feature subspaces by randomly choosing distinct features to train each base
classifier. The ensemble then reduces the results from these internal
classifiers by polling their individual predictions through a majority vote
because it gives a uniform weight to all the base classifiers. We configure
the remaining parameters for the ensemble architecture by conducting small-
scale experiments in a variety of settings. Just as before, we use the
training portion of the EBG 5 dataset to select appropriate values for the
following hyper-parameters:

  * –

number of internal classifiers: $I_{c}\in\\{5,10,15\\}$

  * –

length of subspaces: $L_{s}\in\\{30,50,80\\}$

The results from these experiments show that while a lower value of $L_{s}$
yields inaccurate individual classifiers, the accuracy of the overall ensemble
is much higher. This ascertains the notion that a robust and highly accurate
model can be created from a grouping of weak learners. In light of this, we
conservatively set the value of $L_{s}=30$ and $I_{c}=10$ as we noticed that
higher values yield similar results. These settings are retained throughout
the entirety of our experiments.

###  3.3 Adversary’s Attribution Classifiers

To assess the transferability of the obfuscated samples to other classifiers,
we train a series of classifiers and measure the performance of the baselines
and the ensemble. Specifically, we train multiple classifiers that use
different types of techniques to measure the cross-technique transferability
[38] of the method used to generate the samples. These classifiers are also
trained on the Writeprints feature set and are as follows: k-nearest neighbors
(KNN), naive-bayes (NB), multilayer-perceptron (MLP), logistic regression (LR)
in addition to the already trained random forest classifier (RFC), support
vector machine (SVM), and the ensemble (Ens) itself. Additionally, we
incorporate counter-measures for the findings by Gröndahl et al. [22] that
using an internal classifier results in highly specific transformations which
are not only non-transferable to other different classifiers but also to the
same classifier if it is retrained. We accomplish this by training multiple
versions of the classifiers that exhibit randomness (RFC, Ensemble, and MLP)
during training and then report the average transferability result in their
respective columns.

In addition to these Writeprints based classifiers, we also measure the
transferability of the samples to JGAAP [26], a well-known system for
authorship attribution that provides a wide-array of features and classifiers,
and another MLP model trained on the Basic-9 feature set[35]. This setting
allows us to explore the performance of the ensemble against an adversary that
uses a different feature set and classifier implementation. We borrow the
configuration recommended by Juola et al [27] for JGAAP. The final
configuration for JGAAP is listed in Table 1.

Pre-processing |  Features |  Classifiers
---|---|---
[]Lowercase all Strip punctuation Normalize whitespaces |  Word unigrams Word bigrams Character bigrams Character trigrams Function words Sentence length |  Weka SMO (SVM) Weka Naive Bayes (NB) Linear Discriminant Analysis (LDA)
Table 1: Configuration for JGAAP.

###  3.4 Design of Transferability Experiments

We conduct several experiments with each corresponding to a particular
internal classifier that is used by Mutant-X. In each of these experiments,
all Mutant-X parameters are kept consistent and only its internal classifier
is replaced. We configure Mutant-X parameters based on the findings from the
original pilot experiments performed on the EBG 5 dataset. The values for the
different Mutant-X parameters are specified in Table 2. For each experiment,
we report the average METEOR score of the obfuscated documents, the
transferability rate across our fixed set of adversaries, and the overall
attack success rate of that technique.

Setting | Value
---|---
[]Number of word replacements | 5
Number of iterations | 25
Number of runs per document | 10
Weight assigned to attribution confidence in fitness function | 0.75
Number of top individuals retained in each iteration | 5
Number of document mutants | 5
Table 2: Configuration of hyper-parameters for Mutant-X.

In its default setting, Mutant-X stops obfuscation once the mutated document
has been misclassified by its attribution classifier. We alter this behavior
to allow Mutant-X to continue obfuscation until all $M$ iterations have been
performed. This alteration is partly inspired by the idea that stopping early
because of one successful misclassification is detrimental to the overall goal
of transferability to a wider set of adversaries.

###  3.5 Evaluation Metrics

Originally, Mutant-X was evaluated through two metrics that measured the
safety and soundness of the obfuscation. While effective for measuring
obfuscation across one adversary, they fail to quantify the transferability of
the obfuscated samples to multiple adversaries. To alleviate this, we utilize
a third metric called the Attack Success Rate (ASR) to capture this
information and slightly modify the safety metric to accommodate it. The final
metrics for evaluation are as follows.

  1. 1.

Evasion Effectiveness: An obfuscated document generated from an internal
attribution classifier effectively evades an adversary if it is misclassified
by that particular adversary. We refer to this property of the internal
classifier as its transferability to an adversary and report it as the
percentage of obfuscated documents produced by that classifier that were
misclassified by the adversary. For an adversary $i$ that misclassifies
$a_{i}$ out of $n$ obfuscated documents generated by the internal classifier,
we measure transferability as:

$T_{i}=\frac{a_{i}}{n}\times 100$ (1)

  2. 2.

Attack Success Rate: The attack success rate [11] measures the overall
transferability of the obfuscated documents across the entire set of
adversaries. It is an average of all the transferability scores reported for
that specific internal classifier. Given a fixed set of adversaries $m$, the
attack success rate of classifier $A$ can be reported as:

$ASR_{A}=\frac{\sum_{i=1}^{m}T_{i}}{m}$ (2)

  3. 3.

Semantic Similarity: An obfuscated document has to maintain semantic
similarity to the original document. As originally used to evaluate Mutant-X,
we use the METEOR score [14] to assess this similarity. The score lies in the
range [0, 1] with 1 indicating perfect similarity and 0 indicating the
opposite. The final score reported is the average METEOR score of the
obfuscated documents, where a higher score implies that the final documents
were similar to the original ones.

##  4 Results

###  4.1 Evaluation

Mutant-X Writeprints-Static JGAAP Basic-9 ASR METEOR Classifier RFC* SVM KNN
NB MLP* LR Ens* SVM LDA NB MLP* SVM 1.6 93.7 18.5 20.6 10.1 1.6 7.4 5.0 15.0
10.5 16.9 18.3 0.40 RFC 28.2 26.2 19.4 18.4 14.6 5.8 29.1 24.0 28.0 20.0 25.2
21.7 0.42 Ensemble 18.4 61.0 41.6 52.9 21.9 15.8 71.9 32.0 39.0 32.0 31.0 38.0
0.36

Table 3: Transferability results for multiple Mutant-X settings. Rows are the
writeprints-static classifier used for obfuscation. Columns are the
adversary’s feature set and classifier. The asterisk * indicates that the
classifier exhibits randomness during training. Cells underneath a random
classifier report the average against multiple instances of that particular
classifier. The attack success rate is summarized in the ASR column and the
average METEOR scores are reported in the last column.

The main results of the experiments are presented in Table 3. The rows
correspond to the internal classifier used by Mutant-X and the subsequent
columns correspond to the classifier used by the adversary and the feature set
they are trained on. The cell values are the percentage of documents generated
using that method that were misclassified by the adversary’s classifier, i.e.,
the transferability of that method. The final two columns contain the attack
success rate (mean of transferability to adversaries) and the mean METEOR
score of the technique. Additionally, we reiterate the counter-measure
discussed in Section 3.3 and train multiple versions of classifiers that
exhibit randomness during training. As such, the columns for the Ensemble,
MLP, and RFC report the average transferability to different versions of the
classifiers.

Impact of attribution classifier: The transferability achieved by SVM ranges
from 1.6% ($SVM\rightarrow RFC$) to 93.7% ($SVM\rightarrow SVM$), whereas for
RFC it ranges from 5.8% ($RFC\rightarrow LR$) to 29.1% ($RFC\rightarrow
Ensemble$). In comparison, the ensemble achieves transferability ranging from
15.8% ($Ensemble\rightarrow LR$) to 71.9% ($Ensemble\rightarrow Ensemble$).

We notice that the cases where the baselines perform better are where the
internal classifier and adversary’s classifier are the same ($SVM\rightarrow
SVM,RFC\rightarrow RFC$); cases which are fairly trivial and advantageous to
Mutant-X. In fact, the 93.7% in the case of $SVM\rightarrow SVM$ is
unrealistic because the adversary’s classifier is exactly the same as
Mutant-X, whereas $RFC\rightarrow RFC$ and $Ensemble\rightarrow Ensemble$
scenarios have been normalized by training multiple instances of the adversary
and reporting their average (see Section 3.3). Regardless, the ensemble still
manages to outperform the other baseline in these trivial cases.

The effects of re-training the ensemble and RFC are also evident in these
cases. The average transferability of $RFC\rightarrow RFC$ is quite low at
28.2%, corroborating the findings of Gröndahl et al. [22]. However, this
doesn’t appear to be true for the ensemble as it still reports a fairly high
transferability at 71.9% indicating its robustness.

In the non-trivial cases where the adversary’s classifier is different from
the internal classifier, the ensemble fares far better than the baselines. In
the case of KNN, the ensemble achieves a transferability of 41.6% compared to
the RFC at 19.4%. In the case of NB, it achieves a transferability of 52.9%,
which is 32.3% higher than the SVM at 20.6%. On average, the ensemble achieves
21% higher transferability than the SVM and 13.8% higher than the RFC across
the set of adversaries where the internal classifier is different.

A comparison of the overall performance (trivial and non-trivial) between the
ensemble and the baselines shows that the ensemble outperforms both across the
wide range of adversarial settings. The overall attack success rate of the
ensemble at 38.0% is the best compared to the baselines, which is 1.7$\times$
higher than the transferability of the RFC at 21.7% and 2.1$\times$ higher
than the transferability of the SVM at 18.3%.

The ensemble does not perform as well as the other methods when we compare the
METEOR scores. The samples generated by RFC and SVM retain better semantic
similarity to the original documents with an average METEOR score of 0.42 and
0.40, respectively. In contrast, the ensemble reports an average score of 0.36
indicating that the generated samples differed more from their original
selves. We attribute this lower score to the problem of balance between
protecting the author’s identity and being true to the original content. We
believe that the effort to ensure transferability requires more substantial
changes to be made to the document which leads to lower similarity between the
source and the obfuscated text, and consequently a lower average METEOR score.

The superior effectiveness of the ensemble as an internal classifier is
undeniable when compared to the baselines. The high attack success rate and a
comparable METEOR score make it a reliable alternative to other conventional
classifiers for use alongside an obfuscator like Mutant-X.

Impact of feature set: Mutant-X may have an inherent advantage when the
obfuscator and adversary’s classifiers are trained on the same feature set as
this likely provides the obfuscator unfair insight into how the adversary
operates. We test this by observing results from experiments where the
adversary is trained on a different feature set and classification technique
than the internal classifier.

Within the JGAAP setting, the SVM does not perform as well as the other two
settings. It surprisingly performs the worst in the $SVM\rightarrow SVM$ case,
only achieving a transferability of 5%. We attribute this to the difference
between the kernel functions of the two SVMs; as opposed to a linear kernel
used in the internal classifier, JGAAP’s default setting uses a RBF kernel. In
comparison, the ensemble and the RFC achieve higher degrees of transferability
yielding an attack success rate of 34.3% and 24% respectively with the
ensemble outperforming the RFC by 10.3%.

We see similar results in the Basic-9 setting, where the ensemble achieves a
transferability that is 6% higher than RFC and almost twice as high the SVM.
This affirms the idea that the ensemble performs just as well against
adversaries trained on a different feature set and outperforms other
conventional classifiers.

###  4.2 Discussion

We now study the various decisions we’ve made concerning the ensemble and try
to understand their impact on its inner workings.

Impact of ensemble diversity: It is widely understood that combining a diverse
set of individual classifiers leads to a more robust ensemble [15]. While the
ensemble is diverse in the sense that the base classifiers are trained on
distinct subspaces, we instead focus on the predictions of the base
classifiers to measure diversity. Intuitively, the diversity of an ensemble is
the difference in the predictions of the individual members that constitute it
[30]. To try and understand how the diversity of the ensemble impacts
obfuscation, we train multiple ensembles with varying degrees of diversity and
compare their performance with each other.

To measure the diversity of an ensemble, we use the non-pairwise metric of
entropy $E$ [30]. The entropy value of an ensemble lies in the range $[0,1]$,
a value closer to 0 means that the individual classifiers mostly agree, and a
value closer to 1 means that they mostly disagree with one another. This
measure assumes that a diverse set of classifiers will disagree with one
another as opposed to correlated classifiers which will agree more often.

We train multiple ensembles and control for their diversity by selecting
appropriate base classifiers. Specifically, we create 4 bins of entropy values
$E\in\\{0,0.25,0.5,0.75\\}$ and train 10 ensembles for each bin, each ensemble
having approximately the same entropy as the bin it is assigned to. We then
conduct 40 experimental runs of Mutant-X, and in each run use one of these
ensembles as the internal classifier to obfuscate documents and measure the
attack success rate of these documents against our set of adversaries.

Fig. 3: The attack success rates of the ensembles belonging to each entropy
bin. The y-axis represents the attack success rates while x-axis ticks
represent the entropy value of the bin. Note the inverse relation between ASR
and Entropy: samples sourced from more diverse ensembles fail to transfer well
to other adversaries.

Figure 3 shows the results from these experiments. The y-axis represents the
attack success rates while x-axis ticks represent the entropy value of the
bin. Contrary to our intuition, we notice that ensembles with higher values of
entropy (more diverse) had lower transferability while ensembles with lower
entropy (less diverse) performed comparatively better. We explain our
interpretation of these results as follows.

Recalling that Mutant-X uses the confidence score of the internal classifier
to make decisions regarding obfuscation, we investigate the impact of
diversity on the accuracy of the ensemble and the confidence of the classifier
in its classifications. To increase the diversity of an ensemble, we need to
promote disagreement between the individual classifiers that comprise it.
Consequently, this makes the ensemble overall less confident in its final
classification even if it is correct. Since Mutant-X uses this confidence
score as an indicator of attribution, a lower score leads to poorer decision
making on Mutant-X’s part and reduces the quality of obfuscation. We note that
this problem is likely unique to how Mutant-X operates and not necessarily an
artefact of using ensembles. In future work, it will be interesting to explore
how diversity impacts ensemble based transferability in different settings
that are not bound by this restriction.

Fig. 4: The decision boundaries of the base classifiers inside the ensemble.

Impact of feature subspaces: Our main set of experiments consider an ensemble
of base classifiers trained on randomly selected subspaces of the feature set.
We now consider more systematic approaches for constructing the subspaces and
see how they affect transferability.

At a higher level, the Writeprints feature set incorporates lexical and
syntactic features that are qualitatively distinct. This distinction indicates
the presence of a contextual subspace within the feature set. More
specifically, there are 9 distinct subspaces and are as follows: frequency of
special characters, letters, digits, parts-of-speech tags and punctuation,
most common letter bigrams and trigrams, percentage of certain function words,
and the ratio of unique words (hapax ratio) [2]. We train the base classifiers
on this division of subspaces and measure the attack success rate of the
resultant ensemble and note that, in contrast to the random subspace method,
this yields base classifiers that have different values of $L_{s}$.

Additionally, we explore feature selection techniques to construct the
subspaces. Using a one-way ANOVA test [18], we measure the dependency between
the features and the author label, and use the highest-ranking features to
train the first base classifier. We repeat this for the next base classifier
by considering the remaining set of features and so on for the rest of the
classifiers. This yields base classifiers of varying performance as the
initial ones are highly accurate but the accuracy gradually drops as the
remaining features are insufficient predictors . For this particular
experiment, we set $I_{c}=8$ and $L_{s}=20$ for a consistent distribution of
features.

The results of the experiments are as follows: the contextual subspace
ensemble yields an ASR of 37.1% whereas the feature selection subspace
ensemble yields an ASR of 34.7%. We see that the results for the subspace
ensemble are comparable with those of the random subspace ensemble which had
an ASR of 38.0%.

Considering the security aspect of the techniques, the contextual subspaces
approach is relatively more risky. Since the features composing these
subspaces are easy to identify, an adversary can undo the effects of
obfuscation through adversarial training: building a classifier to recognize
obfuscation by training it on the obfuscated documents generated by the
internal classifier [21]. In contrast, an ensemble built using random
subspaces offers a good balance: achieving a commensurable degree of
transferability and providing a good defence against adversarial training as
its random nature is unpredictable for the adversary.

Feature importance and decision boundaries: While the goal of all classifiers
is to map the input document to an author, there are fundamental differences
in the way they operate and actually classify the data. These differences
highlight the notion of feature importance: some features are more important
for a particular type of classifier than to another. We now interpret these
models to identify the features they consider important and see how it affects
transferability.

RFC is a collection of decision trees that also counts the votes of the
individual outputs of the trees to make the final classification. The decision
trees consist of several nodes that split the training set into subsets based
on the values of certain features. Arguably, features that are used more often
for splitting and split a sizable portion of the training set compared to
others bear greater significance for the model. This is known as the Gini
Importance of that feature [6] and it is the number of times a particular
feature was used for a split weighted by the number of samples it splits.

SVM classifies the data by learning a hyperplane between the data points that
separates the different class boundaries. In a linear SVM, this hyperplane
represents the points at which the distance between the class boundaries is
maximum. Since the coefficients of this hyperplane are associated with the
features, their absolute values represent the significance of the
corresponding feature relative to the other features. In a multi-class
setting, the SVM has multiple hyperplanes separating each of the classes and
each hyperplane has its own set of coefficients.

RFC SVM []posTagFrequency - Space functionWordsPercentage - by
frequencyOfLetters - a functionWordsPercentage - and functionWordsPercentage -
is functionWordsPercentage - was frequencyOfDigits - 1 functionWordsPercentage
- in functionWordsPercentage - had functionWordsPercentage - that

Table 4: Top 5 important features for baseline classifiers.

We assess the differences between what features are important for the RFC and
SVM. Table 4 lists the top 5 features for the baseline RFC and one of the SVM
hyperplanes. We note that these are different for the two classifiers;
moreover, this trend holds even beyond the top 5 features. In a high-
dimensional feature space such as Writeprints, this difference in feature
emphasis by the classifier amounts to some features losing their relative
importance and thus the obfuscator does not consider their relevance. This
highlights a fundamental flaw in the obfuscator: the obfuscation will always
be tuned to the features preferred by its internal classifier and fail to
transfer to a different classifier emphasizing different feature.

Our approach of using feature subspaces in an ensemble alleviates this flaw to
an extent; base classifiers trained on smaller random sets of features
emphasize the importance of those features. The base classifier then
specializes in its localized subspace of features and, while it may not be
accurate, it is representative of a certain aspect of the feature space that
might be of significance to an adversary’s classifier.

Taking a look at the decision boundaries of the base classifiers in Figure 4
helps explain how they improve transferability. We use Principal Component
Analysis (PCA) [49] to reduce the higher-dimensional feature space to two-
dimensions for the plotting the boundaries. The data points are the documents
from the test set projected into the PCA dimensions. The colored regions in
the background represent the decision regions of the classifier for that
particular label, i.e., points that fall in those regions are classified
according to that label. We stress that this two-dimensional projection is
merely an approximation of the actual high-dimensional feature space so some
misalignments are expected. Looking at the decision boundaries of the base
classifiers, we see that they vary significantly and that some of the
classifiers perform better at classifying a particular author than others. The
decision boundaries also highlight the limited access the classifiers have to
the entire feature space, observable by the disjoint between the same decision
region. While the projection takes into account the entire feature space, the
decision region is only based on the subspace the particular classifier was
concerned with.

Figure 5 shows the decision boundary of the ensemble formed from these base
classifiers. We see that the decision region of the ensemble more closely
encapsulates the data points than the base classifiers. Since the ensemble
classifies according to the majority vote of the base classifiers, its
decision boundary is approximately the average of all their decision
boundaries. The voting mechanism also ensures that the base classifiers are
weighted equally so as to not downplay the role of a certain subspace.
Therefore, the ensemble capitalizes on the individual knowledge of the base
classifiers and effectively serves as a middle-ground for the obfuscator to
compare against.

Fig. 5: The decision boundary of the ensemble. It provides a middle-ground
between the boundaries of the base classifiers and serves as an appropriate
attribution classifier.

##  5 Related Work

We survey related research on the transferability of adversarial attacks
designed to evade machine learning classifiers.

There is a rich body of literature in the image classification context on
transferability of adversarial attacks in both white-box and black-box
settings. Biggio et al. [5] and Szegedy et al. [46] first showed that an
adversary can launch attacks by creating minor perturbations in the input that
cause machine learning models to misclassify it. Follow up work has studied
the practically of these adversarial attacks in the real world by studying
whether they can transfer even when the adversary might not have complete
access to the machine learning classifier (e.g., [32, 39, 45, 13]). For
example, Papernot et al. [39] proposed a black-box attack against a variety of
machine learning approaches including deep neural networks, logistic
regression, SVM, decision tree, and nearest neighbors, outperforming existing
attacks in terms of transferability. Suciu et al. [45] and Demontis et al.
[13] studied if and why adversarial attacks (do not) transfer in real-world
settings. They showed that the target model’s complexity and its alignment
with the adversary’s source model significantly impact the transferability of
adversarial attacks.

Adversarial attacks in the continuous vision/image domain are different than
adversarial attacks in the discrete text domain. Much of prior work on
adversarial attacks is focused on the vision domain and cannot be easily
adapted to the text domain [51]. Adversarial attacks on text classification
models mostly work by simply misspelling certain words [19, 31]. While these
attacks are effective, they are easy to counter by standard pre-processing
steps such as fixing misspelled, out-of-vocabulary words. Jin et al.
(TextFooler) [25] and Garg et al. (BAE) [20] proposed black-box adversarial
attacks on text classification models by replacing certain words using word
embeddings and language models, respectively. The evaluation showed that these
black-box adversarial attacks at best only moderately transfer to unseen
models.

Recent adversarial attacks on machine learning based authorship attribution
models employing feedback from authorship classifiers are also quite similar.
Mahmood et al. (Mutant-X) proposed a black-box adversarial attack that
replaced selected words using word embeddings based on a genetic algorithm
[33]. Grondahl et al. [22] also proposed a similar black-box adversarial
attack (ParChoice) that used paraphrasing to replace selected texts. While
these adversarial attack approaches are effective at authorship obfuscation,
they do not transfer well against unseen authorship classifiers. Transferable
authorship obfuscation in such settings remains an open challenge that we
address in our work.

Another relevant line of research has investigated using ensembles to improve
transferability of adversarial attacks. For example, Liu et al. [32] showed
that if an adversarial attack succeeds in evading an ensemble of models it
will have better transferability because the source ensemble model and the
target models are more likely to share decision boundaries. Most recently, Che
et al. [10] studied the effectiveness of different ensemble strategies in
improving transferability of adversarial attacks. They also conclude that an
attack model that evades an ensemble of multiple source models is more likely
to transfer to different target models.

##  6 Conclusion

In the arms race between authorship attribution and obfuscation, it is crucial
that obfuscation can transfer when an adversary deploys a different attributor
than the one assumed by the obfuscator. In this paper we showed that an
ensemble that uses multiple base attribution classifiers, each exploiting
random portions of the feature space, is able to achieve better
transferability by a factor of 1.7$\times$ and 2.1$\times$. Moreover, we
showed that this success holds even when the adversary’s attributor operates
off a different feature space. We also found that ensemble diversity in terms
of disagreement is not crucial for transferability as it only hinders the
obfuscator due to a decrease in the ensemble’s probability of detection.

## References

  * pan [2018] Author Obfuscation.  https://pan.webis.de/clef18/pan18-web/author-obfuscation.html, 2018\.
  * Abbasi and Chen [2008] Ahmed Abbasi and Hsinchun Chen.  Writeprints: A stylometric approach to identity-level identification and similarity detection in cyberspace.  _ACM Transactions on Information Systems (TOIS)_ , 26(2):7, 2008.
  * Afroz et al. [2014] Sadia Afroz, Aylin Caliskan Islam, Ariel Stolerman, Rachel Greenstadt, and Damon McCoy.  Doppelgänger finder: Taking stylometry to the underground.  In _2014 IEEE Symposium on Security and Privacy_ , pages 212–226. IEEE, 2014.
  * Almishari et al. [2014] Mishari Almishari, Dali Kaafar, Ekin Oguz, and Gene Tsudik.  Stylometric linkability of tweets.  In _Proceedings of the 13th Workshop on Privacy in the Electronic Society_ , pages 205–208, 2014.
  * Biggio et al. [2013] Battista Biggio, Igino Corona, Davide Maiorca, Blaine Nelson, Nedim Srndic, Pavel Laskov, Giorgio Giacinto, and Fabio Roli.  Evasion Attacks against Machine Learning at Test Time.  In _Joint European Conference on Machine Learning and Knowledge Discovery in Databases (ECML PKDD)_ , 2013.
  * Breiman [2017] L. Breiman.  _Classification and Regression Trees_.  CRC Press, 2017.  ISBN 9781351460484.  URL https://books.google.com.pk/books?id=gLs6DwAAQBAJ.
  * Brennan et al. [2012a] Michael Brennan, Sadia Afroz, and Rachel Greenstadt.  Adversarial stylometry: Circumventing authorship recognition to preserve privacy and anonymity.  _ACM Transactions on Information and System Security (TISSEC)_ , 15, 11 2012a.  10.1145/2382448.2382450.
  * Brennan et al. [2012b] Michael Brennan, Sadia Afroz, and Rachel Greenstadt.  Adversarial stylometry: Circumventing authorship recognition to preserve privacy and anonymity.  _ACM Transactions on Information and System Security (TISSEC)_ , 15(3):1–22, 2012b.
  * Castro-Castro et al. [2017] Daniel Castro-Castro, R. O. Bueno, and R. Muñoz.  Author masking by sentence transformation.  In _CLEF_ , 2017.
  * Che et al. [2020] Zhaohui Che, Ali Borji, Guangtao Zhai, Suiyi Ling, Jing Li, and Patrick Le Callet.  A New Ensemble Adversarial Attack Powered by Long-Term Gradient Memories.  In _AAAI Conference on Artificial Intelligence (AAAI-20)_ , 2020.
  * Chen and Vorobeychik [2018] Yifan Chen and Yevgeniy Vorobeychik.  Regularized ensembles and transferability in adversarial learning.  _CoRR_ , abs/1812.01821, 2018.  URL http://arxiv.org/abs/1812.01821.
  * Clark and Hannon [2007] Jonathan H Clark and Charles J Hannon.  An algorithm for identifying authors using synonyms.  In _Eighth Mexican International Conference on Current Trends in Computer Science (ENC 2007)_ , pages 99–104. IEEE, 2007.
  * Demontis et al. [2019] Ambra Demontis, Marco Melis, Maura Pintor, Matthew Jagielski, Battista Biggio, Alina Oprea, Cristina Nita-Rotaru, and Fabio Roli.  Why Do Adversarial Attacks Transfer? Explaining Transferability of Evasion and Poisoning Attacks.  In _USENIX Security Symposium_ , 2019.
  * Denkowski and Lavie [2014] Michael Denkowski and Alon Lavie.  Meteor universal: Language specific translation evaluation for any target language.  In _Proceedings of the Ninth Workshop on Statistical Machine Translation_ , pages 376–380, Baltimore, Maryland, USA, June 2014. Association for Computational Linguistics.  10.3115/v1/W14-3348.  URL https://www.aclweb.org/anthology/W14-3348.
  * Dietterich [2000a] Thomas G. Dietterich.  Ensemble methods in machine learning.  In _Multiple Classifier Systems_ , pages 1–15, Berlin, Heidelberg, 2000a. Springer Berlin Heidelberg.
  * Dietterich [2000b] Thomas G. Dietterich.  Ensemble methods in machine learning.  In _Proceedings of the First International Workshop on Multiple Classifier Systems_ , MCS ’00, page 1–15, Berlin, Heidelberg, 2000b. Springer-Verlag.  ISBN 3540677046.
  * Dyer et al. [2013] Eva L. Dyer, Aswin C. Sankaranarayanan, and Richard G. Baraniuk.  Greedy feature selection for subspace clustering.  _J. Mach. Learn. Res._ , 14(1):2487–2517, January 2013.  ISSN 1532-4435.
  * Elssied et al. [2014] Nadir Elssied, Assoc Prof. Dr. Othman Ibrahim, and Ahmed Hamza Osman.  A novel feature selection based on one-way anova f-test for e-mail spam classification.  _Research Journal of Applied Sciences, Engineering and Technology_ , 7:625–638, 01 2014.  10.19026/rjaset.7.299.
  * Gao et al. [2018] Ji Gao, Jack Lanchantin, Mary Lou Soffa, and Yanjun Qi.  Black-box Generation of Adversarial Text Sequences to Evade Deep Learning Classifiers.  In _IEEE Security and Privacy Workshop on Deep Learning and Security_ , 2018.
  * Garg and Ramakrishnan [2020] Siddhant Garg and Goutham Ramakrishnan.  BAE: BERT-based Adversarial Examples for Text Classification.  In _Conference on Empirical Methods in Natural Language Processing (EMNLP)_ , 2020.
  * Goodfellow et al. [2014] Ian J. Goodfellow, Jonathon Shlens, and Christian Szegedy.  Explaining and Harnessing Adversarial Examples.  _arXiv e-prints_ , art. arXiv:1412.6572, Dec 2014.
  * Gröndahl and Asokan [01 Oct. 2020] Tommi Gröndahl and N. Asokan.  Effective writing style transfer via combinatorial paraphrasing.  _Proceedings on Privacy Enhancing Technologies_ , 2020(4):175 – 195, 01 Oct. 2020.  https://doi.org/10.2478/popets-2020-0068.  URL https://content.sciendo.com/view/journals/popets/2020/4/article-p175.xml.
  * Howard and Ruder [2018] Jeremy Howard and Sebastian Ruder.  Universal language model fine-tuning for text classification.  _arXiv preprint arXiv:1801.06146_ , 2018.
  * Htike [2016] Kyaw Htike.  Efficient determination of the number of weak learners in adaboost.  _Journal of Experimental & Theoretical Artificial Intelligence_, 29:1–16, 12 2016.  10.1080/0952813X.2016.1266038.
  * Jin et al. [2020] Di Jin, Zhijing Jin, Joey Tianyi Zhou, and Peter Szolovits.  Is BERT Really Robust? A Strong Baseline for Natural Language Attack on Text Classification and Entailment.  In _AAAI 2020_ , 2020.
  * Juola [2009] P. Juola.  Jgaap: A system for comparative evaluation of authorship attribution.  2009\.
  * Juola and Vescovi [2010] Patrick Juola and Darren Vescovi.  Empirical evaluation of authorship obfuscation using jgaap.  In _Proceedings of the 3rd ACM workshop on Artificial Intelligence and Security_ , pages 14–18, 2010.
  * Karadzhov et al. [2017] Georgi Karadzhov, Tsvetomila Mihaylova, Yasen Kiprov, Georgi Georgiev, Ivan Koychev, and Preslav Nakov.  The case for being average: A mediocrity approach to style masking and author obfuscation.  pages 173–185, 07 2017.  ISBN 978-3-319-65812-4.  10.1007/978-3-319-65813-1_18.
  * Keswani et al. [2016] Yashwant Keswani, H. Trivedi, Parth Mehta, and P. Majumder.  Author masking through translation.  In _CLEF_ , 2016.
  * Kuncheva and Whitaker [2004] L. Kuncheva and C. Whitaker.  Measures of diversity in classifier ensembles and their relationship with the ensemble accuracy.  _Machine Learning_ , 51:181–207, 2004.
  * Li et al. [2019] Jinfeng Li, Shouling Ji, Tianyu Du, Bo Li, and Ting Wang.  TextBugger: Generating Adversarial Text Against Real-world Applications.  In _Network and Distributed Systems Security (NDSS) Symposium_ , 2019\.
  * Liu et al. [2017] Yanpei Liu, Xinyun Chen, Chang Liu, and Dawn Song.  Delving into transferable adversarial examples and black-box attacks.  _International Conference on Learning Representations_ , 2017.
  * Mahmood et al. [2019] Asad Mahmood, Faizan Ahmad, Zubair Shafiq, Padmini Srinivasan, and Fareed Zaffar.  A girl has no name: Automated authorship obfuscation using mutant-x.  _PoPETs_ , 2019(4):54–71, 2019.  10.2478/popets-2019-0058.  URL https://doi.org/10.2478/popets-2019-0058.
  * Mansoorizadeh et al. [2016] Muharram Mansoorizadeh, Taher Rahgooy, Mohammad Aminian, and M. Eskandari.  Author obfuscation using wordnet and language models.  In _CLEF_ , 2016.
  * McDonald et al. [2012] Andrew WE McDonald, Sadia Afroz, Aylin Caliskan, Ariel Stolerman, and Rachel Greenstadt.  Use fewer instances of the letter “i”: Toward writing style anonymization.  In _International Symposium on Privacy Enhancing Technologies Symposium_ , pages 299–318. Springer, 2012.
  * Narayanan et al. [2012] Arvind Narayanan, Hristo Paskov, Neil Zhenqiang Gong, John Bethencourt, Emil Stefanov, Eui Chul Richard Shin, and Dawn Song.  On the feasibility of internet-scale author identification.  In _2012 IEEE Symposium on Security and Privacy_ , pages 300–314. IEEE, 2012.
  * Overdorf and Greenstadt [2016] Rebekah Overdorf and Rachel Greenstadt.  Blogs, twitter feeds, and reddit comments: Cross-domain authorship attribution.  _Proceedings on Privacy Enhancing Technologies_ , 2016(3):155–171, 2016.
  * Papernot et al. [2016] Nicolas Papernot, Patrick D. McDaniel, and Ian J. Goodfellow.  Transferability in machine learning: from phenomena to black-box attacks using adversarial samples.  _ArXiv_ , abs/1605.07277, 2016.
  * Papernot et al. [2017] Nicolas Papernot, Patrick McDaniel, Ian Goodfellow, Somesh Jha, Z. Berkay Celik, and Ananthram Swami.  Practical Black-Box Attacks against Machine Learning.  In _ACM Asia Conference on Computer and Communications Security (AsiaCCS)_ , 2017.
  * Potthast et al. [2016] Martin Potthast, Matthias Hagen, and Benno Stein.  Author obfuscation: Attacking the state of the art in authorship verification.  In _CLEF_ , 2016.
  * Rajapaksha et al. [2017] Praboda Rajapaksha, Reza Farahbakhsh, and Noël Crespi.  Identifying content originator in social networks.  In _GLOBECOM 2017-2017 IEEE Global Communications Conference_ , pages 1–6. IEEE, 2017.
  * Ruder et al. [2016] Sebastian Ruder, Parsa Ghaffari, and John G Breslin.  Character-level and multi-channel convolutional neural networks for large-scale authorship attribution.  _arXiv preprint arXiv:1609.06686_ , 2016.
  * Shetty et al. [2017] Rakshith Shetty, Bernt Schiele, and Mario Fritz.  A${}^{\mbox{4}}$nt: Author attribute anonymity by adversarial training of neural machine translation.  _CoRR_ , abs/1711.01921, 2017.  URL http://arxiv.org/abs/1711.01921.
  * Stolerman et al. [2014] Ariel Stolerman, Rebekah Overdorf, Sadia Afroz, and Rachel Greenstadt.  Breaking the closed-world assumption in stylometric authorship attribution.  In _IFIP International Conference on Digital Forensics_ , pages 185–205. Springer, 2014.
  * Suciu et al. [2018] Octavian Suciu, Radu Marginean, Yigitcan Kaya, Hal Daume III, and Tudor Dumitras.  When Does Machine Learning FAIL? Generalized Transferability for Evasion and Poisoning Attacks.  In _USENIX Security Symposium_ , 2018.
  * Szegedy et al. [2014] Christian Szegedy, Wojciech Zaremba, Ilya Sutskever, Joan Bruna, Dumitru Erhan, Ian Goodfellow, and Rob Fergus.  Intriguing properties of neural networks.  In _International Conference on Learning Representations (ICLR)_ , 2014.
  * Tin Kam Ho [1998] Tin Kam Ho.  The random subspace method for constructing decision forests.  _IEEE Transactions on Pattern Analysis and Machine Intelligence_ , 20(8):832–844, 1998.  10.1109/34.709601.
  * Ting et al. [2011] Kai Ming Ting, Jonathan R Wells, Swee Chuan Tan, Shyh Wei Teng, and Geoffrey I Webb.  Feature-subspace aggregating: ensembles for stable and unstable learners.  _Machine Learning_ , 82(3):375–397, 2011.
  * Wold et al. [1987] Svante Wold, Kim Esbensen, and Paul Geladi.  Principal component analysis.  _Chemometrics and Intelligent Laboratory Systems_ , 2(1):37 – 52, 1987.  ISSN 0169-7439.  https://doi.org/10.1016/0169-7439(87)80084-9.  URL http://www.sciencedirect.com/science/article/pii/0169743987800849.  Proceedings of the Multivariate Statistical Workshop for Geologists and Geochemists.
  * Ye et al. [2013] Yunming Ye, Qingyao Wu, Joshua Zhexue Huang, Michael K. Ng, and Xutao Li.  Stratified sampling for feature subspace selection in random forests for high dimensional data.  _Pattern Recognition_ , 46(3):769 – 787, 2013\.  ISSN 0031-3203.  https://doi.org/10.1016/j.patcog.2012.09.005.  URL http://www.sciencedirect.com/science/article/pii/S0031320312003974.
  * Zhang et al. [2020] Wei Emma Zhang, Quan Z. Sheng, Ahoud Alhazmi, and Chenliang Li.  Adversarial Attacks on Deep Learning Models in Natural Language Processing: A Survey.  In _ACM Transactions on Intelligent Systems and Technology_ , 2020\.


[a]Simone Zoia

# Two-loop five-particle scattering amplitudes

###### Abstract

I discuss the recent advances in the computation of two-loop scattering
amplitudes for five-particle processes. The latter are fundamental ingredients
to obtain predictions at the next-to-next-to-leading order (NNLO) in QCD for
many interesting LHC processes. I discuss the state-of-the-art technology for
computing scattering amplitudes analytically, and present new results relevant
for the LHC phenomenology.

##  1 Introduction

Improvements in systematic uncertainties and the increasingly large amount of
data collected at the Large Hadron Collider (LHC) mean that measurements at
the percent-level precision will soon be available for a wide class of
observables. Exploiting fully this wealth of data requires that we keep the
theoretical uncertainties in line with the experimental ones. This is a multi-
faceted challenge which requires improvements on all levels of theoretical
predictions. One crucial step to this end is the computation of fixed-order
predictions at least at the Next-to-Next-to-Leading Order (NNLO) in QCD. This
is particularly challenging for processes involving many physical scales,
which are however of great interest since the high energy of the interactions
at the LHC can lead to the production of many particles.

The current frontier for multi-particle NNLO predictions are $2\to 3$
processes. The main bottleneck is the computation of the two-loop five-
particle scattering amplitudes required for the double-virtual corrections.
Over the last years we have seen important progress in this direction,
building upon the campaign which mastered the $2\to 2$ processes and a
deepened understanding of the polylogarithmic special functions appearing in
the amplitudes. In the case of fully massless final states, all partonic
processes needed for three-jet (in the leading colour approximation) [1, 2],
three-photon [3, 4, 5, 6], di-photon + jet [7, 8], and photon + di-jet [9]
production at the LHC have been computed. More recently, several leading-
colour amplitudes involving an external massive vector boson [10, 11, 12, 13,
14, 15] have also become available, and the first steps have been taken for
processes with internal massive propagators [16, 17] (see Matteo Becchetti’s
talk [18]). Remarkably, for most of these $2\to 3$ processes, theoretical
predictions at NNLO in QCD followed within months from the computation of the
two-loop amplitudes [3, 19, 20, 21, 22, 23, 15, 24, 25, 9, 26]. On the one
hand, this shows that the availability of the two-loop amplitudes is indeed
the bottleneck for such computations. On the other hand, this proves that the
analytic expressions obtained for them meet the high demands of phenomenology
in terms of stability and efficiency of the numerical evaluation.

In order to understand what kind of difficulties need to be overcome, it is
instructive to have a look at how the analytic expression of a loop amplitude
looks like. It is a function of the particles’ momenta,111For simplicity we
here neglect the dependence on the space-time dimension $D$. which we denote
by $p$, with the following general structure,

$\displaystyle\mathcal{A}(p)=\sum_{i}c_{i}(p)\times f_{i}(p)\,,$ (1)

where $f_{i}(p)$ are special functions, i.e. functions with branch cuts such
as logarithms and polylogarithms, and $c_{i}(p)$ are rational coefficients.
This separation allows us to distinguish two types of complexity: algebraic
and analytic. The _algebraic complexity_ affects the rational coefficients:
while they are simple from the analytic point of view, their expressions can
be exceedingly large. To give an idea, in the massless five-particle processes
they are functions of five independent variables and, written as ratios of
polynomials, their maximum polynomial degree can exceed 100. Manipulating
these coefficients symbolically is thus difficult from the computational point
of view. For the special functions the difficulty is not only computational,
but also conceptual. Understanding what class of special functions is relevant
for a given amplitude is often challenging, and evaluating them even at a
single phase-space point may be problematic due to their intricate branch-cut
structure. Developing the mathematical technology to bring under control this
_analytic complexity_ is thus imperative in order to obtain analytic results
that can be used for phenomenology.

For the fully massless five-particle processes and for the five-particle
processes with a single external massive particle, the problem of the analytic
complexity is completely solved at two-loop order. The solution consists in
representing the appearing Feynman integrals in terms of bases of
algebraically independent special functions, called _pentagon functions_ [27,
28, 29, 30, 31], which allow for a compact analytic form as well as a stable
and efficient numerical evaluation. Dmitry Chicherin discusses thoroughly this
topic in his talk [32].

In my talk, I focus on the techniques we developed to tame the algebraic
complexity, and use as concrete example the two-loop amplitudes required for
the NNLO QCD corrections to isolated photon production in association with two
jets in hadron collisions [9].

##  2 Amplitude workflow

In this section I present a workflow for computing scattering amplitudes
analytically based on finite-field arithmetic [33, 34]. It was developed in a
long series of works by a multitude of authors including myself, and has
proven efficient throughout a wide range of cutting-edge applications [34, 35,
36, 37, 38, 10, 39, 12, 13, 16, 14, 15, 40, 9]. I will give here a brief and
general overview of the method, considering the computation of a partial
helicity amplitude in general.222This methodology can be similarly applied to
the computation of any quantity involving Feynman integrals. I refer the
readers who are not familiar with the modern on-shell methods for scattering
amplitudes to refs. [41, 42, 43].

The starting point is the expression of the chosen helicity amplitude $A$ in
terms of Feynman diagrams, which we generate using QGRAF [44]. It is a
function of a certain set of independent kinematic variables, which we label
cumulatively by $\vec{x}$, and of the dimensional regulator
$\epsilon=(4-D)/2$, where $D$ is the number of space-time dimensions. We use a
momentum-twistor parametrisation for the kinematics [45, 46], which allows us
to represent also the helicity spinors and rationalises certain square roots
that appear in the kinematics of more than four particles. We process the
Feynman-diagram expression to extract the chosen partial amplitude, and
rewrite it as a linear combination of rational coefficients $c_{i}$ and scalar
Feynman integrals $I_{i}$,

$\displaystyle
A\left(\vec{x},\epsilon\right)=\sum_{i}c_{i}(\vec{x},\epsilon)\,I_{i}(\vec{x},\epsilon)\,.$
(2)

The scalar Feynman integrals satisfy linear relations, for instance the
integration-by-parts identities (IBPs) [47, 48]. Only a subset of them, called
master integrals (MIs), are linearly independent. The next step is then to
rewrite all integrals in terms of MIs, which I denote by $G_{j}$:

$\displaystyle
I_{i}\left(\vec{x},\epsilon\right)=\sum_{j}W_{ij}(\vec{x},\epsilon)\,G_{j}(\vec{x},\epsilon)\,.$
(3)

This task is in principle straightforward using Laporta’s algorithm [49]. It
amounts to the generation and solution of a large set of linear relations
among the scalar integrals. Using eq. 3 we can then express the amplitude in
terms of MIs:

$\displaystyle
A\left(\vec{x},\epsilon\right)=\sum_{i}d_{i}(\vec{x},\epsilon)\,G_{i}(\vec{x},\epsilon)\,.$
(4)

In practice, however, these steps suffer from a common plague of symbolic
computations: intermediate expression swell. While the input (the Feynman
diagrams) and the final result for the amplitude are comparatively compact —
in particular the latter, as it is constrained by many physical properties —
the intermediate expressions grow to the point of hindering the computation.

Two simple but crucial insights allow us to overcome this issue. First, we do
not need to know analytically the intermediate expressions, such as the
coefficients $W_{ij}$ in eq. 3; we only care about the final result, the
amplitude. Second, the rational coefficients collapse to numbers once they are
evaluated numerically, deflating the expression swell. Leveraging these two
points, ref. [34] pioneered an approach which trades the symbolic manipulation
of the rational functions for numerical evaluations over finite fields — that
is, using integers modulo a (large) prime number — throughout the entire
computation. Using finite fields allows us to avoid the loss of accuracy
inherent to floating-point numbers, as well as the computationally expensive
arbitrary-precision arithmetic required to work with exact rational numbers.
The analytic expression can then be recovered from sufficiently many numerical
samples using multivariate functional reconstruction algorithms.

Our entire workflow therefore becomes a “black box” which, for given values of
the kinematic variables $\vec{x}$ and of $\epsilon$, returns the values of the
coefficients of the MIs $d_{i}$ in eq. 4, performing all computations
numerically over finite fields. To this end, we use the multi-purpose
Mathematica/C++ framework FiniteFlow [38]. In principle we could now
reconstruct the analytic expression of the MI coefficients $d_{i}$ by
evaluating the black box sufficiently many times. The number of numerical
samples depends on the complexity of the rational functions which need to be
reconstructed. In order to make the most of the finite-field approach,
therefore, one should reconstruct something that is as simple as possible. To
this end, we also Laurent-expand around $\epsilon=0$ the coefficients
$d_{i}(\vec{x},\epsilon)$ and the MIs $G_{i}(\vec{x},\epsilon)$, truncating
the expansion at the required order. While the expansion of the coefficients
can be performed within the finite-field setup, expanding the MIs requires
having analytic expressions in terms of special functions for them.

The analytic computation of the MIs is typically the most difficult step. As
we are here focusing on the problem of the algebraic complexity, I will assume
that analytic expressions for the MIs are available, and refer to Dmitry
Chicherin’s talk [32] for a discussion of this issue. For our purposes, it
suffices to mention two important aspects. First, in order to completely
separate the algebraic from the analytic complexity, and the rational
coefficients from the special functions in the amplitudes, the MIs must be
chosen so that they satisfy differential equations in the canonical form [50]
(see refs. [51, 43] for an introduction to this concept). In practice, this
means that the MIs are given — order by order in $\epsilon$ — by polynomials
in the special functions (and transcendental constants) with constant rational
coefficients. A toy example for a Feynman integral of this form with a single
kinematic variable $z$ would look like

$\displaystyle G_{j}(z,\epsilon)\approx
1+\epsilon\,\left(\log(z)-\mathrm{i}\pi\right)+\epsilon^{2}\,\left(\mathrm{Li}_{2}(z)+\log^{2}(z)+\frac{\pi^{2}}{6}\right)+\mathcal{O}\left(\epsilon^{3}\right)\,,$
(5)

where $\mathrm{Li}_{2}$ is the dilogarithm. The second important aspect for
our purposes is that special functions satisfy functional identities. A simple
example is given by the dilogarithm’s inversion formula,

$\displaystyle\mathrm{Li}_{2}(z)+\frac{1}{2}\log^{2}(-z)+\mathrm{Li}_{2}\left(\frac{1}{z}\right)+\frac{\pi^{2}}{6}=0\,,$
(6)

for $z<0$. Unlike the rational coefficients, the special functions are treated
symbolically, and the identities among them must be known exactly in order to
reconstruct a final result that is as simple as possible. This is why it is
important that we express the MIs in terms of a basis333Strictly speaking, we
should talk of a generating set rather than a basis. of _algebraically
independent_ special functions. This gives a unique representation for all MIs
and the amplitude where all cancellations and simplifications due to the
special functions are built in.

Denoting the list of special functions which form a basis by
$\vec{f}(\vec{x})$, our final expression for the helicity partial amplitude
has the form

$\displaystyle A\left(\vec{x},\epsilon\right)=\sum_{w=-2L}^{w_{\rm
max}}\epsilon^{w}\sum_{i}e_{i}^{(w)}(\vec{x})\
\mathrm{mon}_{i}\left[\vec{f}(\vec{x})\right]\,,$ (7)

where $w_{\mathrm{max}}$ depends on the intended application of our results,
and $\mathrm{mon}_{i}\left[\vec{f}(\vec{x})\right]$ are monomials in the
special functions. We may now evaluate the rational coefficients
$e_{i}^{(w)}(\vec{x})$ over finite fields, and reconstruct their analytic
expression using FiniteFlow’s functional reconstruction algorithms. While
evaluating the coefficients at a single phase-space point $\vec{x}$ is
typically feasible, evaluating them sufficiently many times for the
reconstruction to succeed can still be prohibitive. In section 3 I will
discuss a number of strategies which allow us to leverage our physical
understanding of the analytic structure of the amplitudes to optimise
substantially the reconstruction.

##  3 Functional reconstruction strategy

In this section I review a number of techniques to optimise the functional
reconstruction of a set of rational coefficients $\\{r_{i}(\vec{x})\\}$ from
their numerical values over finite fields. A proxy for the difficulty of this
problem is given by the total time it takes to collect the samples required
for the reconstruction.444The time required to perform the actual
reconstruction from the numerical samples is negligible. This can be estimated
as

$\displaystyle\text{total evaluation time}\approx\frac{\text{evaluation time
of one point}\ \times\ \\#\ \text{points}}{\\#\ \text{CPUs}}\,.$ (8)

Each of the factors on the right-hand side offers a different line of attack.

First of all, the evaluations can be parallelised, which is why the total
reconstruction time is inversely proportional to the number of available CPUs.
The availability of computing resources is thus a crucial factor in this game.

The evaluation time of the coefficients at one random point in the finite
field is mainly affected by the evaluation of the solution to the IBPs. The
traditional way to generate the IBPs introduces integrals which have higher
propagator powers with respect to those appearing in the amplitudes. These
“spurious” integrals increase considerably both the size and the number of
variables of the linear system of equations we need to solve to perform the
reduction of the amplitudes. This makes the evaluation of the solution over
finite fields slower, and increases the memory load, this way reducing the
number of CPUs which can be used in parallel on a single machine. To overcome
these issues, we generated relations among integrals without higher propagator
powers [52, 53, 54]. We made use of the Baikov parameterisation of Feynman
integrals [55, 56], where IBP relations typically contain both higher
propagator powers and dimensionally-shifted integrals. We derived equations
without dimensionally-shifted integrals as closed-form solutions to polynomial
equations called syzygy equations [57]. Next, we put these solutions into a
sparse matrix, and eliminated the higher propagator powers through Gaussian
elimination [58]. We then reconstructed new template equations using
FiniteFlow’s sparse solver and, from them, generated a substantially smaller
system of equations. This final system of equations is fed into our finite-
field workflow, where we solve it on the fly using the traditional Laporta
algorithm [49] to reduce the amplitudes onto MIs.

Finally, reducing the number of sample points required for the reconstruction
is where we can leverage the most our physical understanding of the analytic
structure of scattering amplitudes. This number is strictly related to the
complexity of the rational coefficients to be reconstructed: the higher are
the numerator/denominator polynomial degrees of the coefficients, the more
points are necessary. We must therefore reduce the complexity of the
coefficients which need to be reconstructed. We employ a multi-step strategy,
discussed thoroughly in ref. [59].

Step 1: We fit the linear relations among the $\\{r_{i}(\vec{x})\\}$, i.e. we
find the $c_{i}\in\mathbb{Q}$ such that

$\displaystyle\sum_{i}c_{i}\,r_{i}(\vec{x})=0\quad\forall\ \vec{x}\,.$ (9)

The fit requires a number of evaluations of the coefficients equal to the
number of linearly independent coefficients plus a few extra evaluations for
checks. This is typically considerably smaller than the number of evaluations
required to reconstruct the rational coefficients. We then solve the obtained
linear relations so as to express the more complicated coefficients in terms
of simpler ones. Only the latter then need to be reconstructed.

Step 2: The denominators of the coefficients are intimately related to the
singularities of the amplitudes and of the Feynman integrals, which are well
understood. We can thus make the ansatz

$\displaystyle
r_{i}(\vec{x})=\frac{\mathrm{num}_{i}(\vec{x})}{\prod_{j}W_{j}(\vec{x})^{q_{ij}}}\,,\qquad\quad
q_{ij}\in\mathbb{Z}\,,$ (10)

where $\mathrm{num}_{i}(\vec{x})$ is a polynomial, and the possible factors
$W_{j}(\vec{x})$ are known from the computation of the Feynman integrals (the
letters of the pentagon alphabet [60]).555Spinor factors ($\langle ij\rangle$,
$[ij]$) also need to be included in the ansatz to compute objects with non-
trivial helicity scaling. We determine the integers $q_{ij}$ by reconstructing
the coefficients analytically on a random univariate phase-space slice [61],
which is much cheaper than the full multivariate reconstruction. From the
reconstruction on the univariate slice we can also determine the degrees of
the numerators, which remain to be reconstructed.

Step 3: We perform a univariate partial fraction decomposition of the
coefficients $r_{i}(\vec{x})$ with respect to one of the variables on the fly.
In other words, we make an ansatz for the decomposition, and reconstruct the
coefficients of the ansatz by solving a fit. The variable for the univariate
partial fraction decomposition is chosen so as to minimise the degrees of the
coefficients of the ansatz, which depend on one less variable. For example,
consider the following rational function:

$\displaystyle
r(x,y)=\frac{-2x^{4}-4x^{3}y+5x^{2}y^{2}-xy^{3}+4y^{4}}{(x-y)y^{2}(x^{2}+y^{2})}\,.$
(11)

Its partial fraction decomposition with respect to $y$ is given by

$\displaystyle
r(x,y)=-\frac{2x}{y^{2}}-\frac{6}{y}+\frac{1}{x-y}+\frac{3y}{x^{2}+y^{2}}\,.$
(12)

We then make the following ansatz for the partial fraction decomposition of
$r(x,y)$,

$\displaystyle
r(x,y)=\frac{q_{1}(x)}{y^{2}}+\frac{q_{2}(x)}{y}+\frac{q_{3}(x)}{x-y}+\frac{q_{4}(x)+q_{5}(x)y}{x^{2}+y^{2}}\,,$
(13)

and reconstruct the coefficients $q_{i}(x)$ from the numerical evaluations of
the original function $r(x,y)$. By comparing eqs. (12) and (13) we see that

$\displaystyle q_{1}(x)=-2x\,,\qquad q_{2}(x)=-6\,,\qquad q_{3}(x)=1\,,\qquad
q_{4}(x)=0\,,\qquad q_{5}(x)=3\,.$ (14)

As a result, the coefficients of the ansatz have maximum numerator|denominator
degrees 1|0 in one variable. This is to be contrasted with the degrees 4|5 in
two variables of the expression with a common denominator in eq. (11).
Constructing the ansatz in eq. (13) requires the knowledge of the denominator
and of the degree in $y$ of the numerator of $r(x,y)$. The former is known
from the previous step, and the latter can be determined by reconstructing
$r(x,y)$ on a univariate slice where we vary $y$ and keep $x$ constant.

Step 4: We carry out another factor-matching (step 2), this time on the
coefficients of the ansätze for the partial fraction decompositions (the
$q_{i}(x)$’s in eq. (13) for the toy example above). We enlarge the list of
factors $W_{i}(\vec{x})$ in the ansatz (10) to include also the spurious
factors introduced by the univariate partial fraction decomposition in the
previous step.

Finally, we multiply away all factors which have been determined, and
reconstruct what remains using FiniteFlow’s built-in functional reconstruction
algorithms. In the most complicated cases, the evaluation of the coefficients
in just one prime field is not sufficient to lift all constants from the
finite field to the field of rational numbers. In such cases, it is necessary
to combine the reconstructions over multiple prime fields using the Chinese
remainder theorem. A thorough discussion of finite field methods in the
context of scattering amplitudes can be found in ref. [38].

I conclude by showcasing the effectiveness of this strategy in the recent
computation of the two-loop helicity amplitudes required for the double-
virtual NNLO QCD corrections to isolated photon production in association with
two jets at the LHC [9]. The most complicated partonic channel is the
2-quark-2-gluon channel ($0\to\bar{q}qgg\gamma$). The amplitudes for this
process depend on 5 independent variables. We set one to $1$ throughout the
computation, and recover it at the end from dimensional analysis. We decompose
each helicity amplitude into various partial amplitudes based on the colour
dependence, the number of closed fermion loops, and what the photon couples
to. The reduction of the polynomial degrees at each step of the reconstruction
is shown in table 1 for the most complicated partial amplitudes. As can be
seen, the maximum polynomial degree goes down from 94 in 4 variables to 22 in
3 variables thanks to our reconstruction strategy, leading to a substantial
drop in the number of sample points required to complete the reconstruction.

|  | 4 variables | 3 variables
---|---|---|---
amplitude | helicity | common denominator | step 1 | step 2 | step 3 | step 4
$A^{(2),1}_{34;q}$ | $-++-+$ | 94|91 | 74|71 | 74|0 | 22|18 | 22|0
$A^{(2),1}_{34;q}$ | $-+-++$ | 93|89 | 90|86 | 90|0 | 24|14 | 18|0
$A^{(2),N_{c}^{2}}_{34;q}$ | $-+-++$ | 58|55 | 54|51 | 53|0 | 20|16 | 20|0
Table 1:  Highest numerator|denominator degrees of the rational coefficients
of the two most complicated $0\to\bar{q}qgg\gamma$ two-loop partial amplitudes
$A^{(2)}$. The superscripts denote the relevant power of $N_{c}$, while the
subscripts give further information about the partial amplitude (see ref.
[9]). The last row shows the most complicated leading-colour partial amplitude
for comparison.

##  4 Conclusions

In this talk I presented an approach to compute scattering amplitudes
analytically which replaces the symbolic manipulations of the intermediate
expressions with numerical evaluations in a finite field, this way
sidestepping the plague of intermediate expression swell. In particular, I
discussed a number of strategies which leverage our understanding of the
analytic structure of scattering amplitudes to optimise the functional
reconstruction of the result from the numerical samples in the finite field.
This approach is well suited for the computation of high-multiplicity
amplitudes. Indeed, finite field methods such as the ones I presented underly
the recent progress in the computation of two-loop five-particle scattering
amplitudes in QCD. The computation of the latter is one of the main bottleneck
towards obtaining NNLO QCD predictions for $2\to 3$ processes, which are in
high demand for the LHC programme.

As an example, I showcased the effectiveness of our approach in the
computation of the full-colour two-loop five-particle amplitudes required for
the NNLO QCD corrections to isolated photon production in association with two
jets at the LHC [9]. This work marked two important milestones. First, it
completed the study at NNLO in QCD of all $2\to 3$ processes of interest for
the LHC whose amplitudes involve massless particles only. Second, it was the
first calculation of a $2\to 3$ hadron-collider process at NNLO in QCD that
did not rely on the leading-colour approximation. I refer to Michał Czakon’s
talk [62] for a discussion of the phenomenology of this process.

## Acknowledgments

I thank Simon Badger, Michał Czakon, Heribertus Bayu Hartanto, Ryan Moodie,
Tiziano Peraro and Rene Poncelet for the great work done together in ref. [9].
This project received funding from the European Research Council (ERC) under
the European Union’s Horizon 2020 research and innovation programme High
precision multi-jet dynamics at the LHC (grant agreement No. 772099), and from
the European Union’s Horizon research and innovation programme under the Marie
Skłodowska-Curie grant agreement No. 101105486.

## References

  * [1] S. Abreu, J. Dormans, F. Febres Cordero, H. Ita, B. Page and V. Sotnikov, _Analytic Form of the Planar Two-Loop Five-Parton Scattering Amplitudes in QCD_ , _JHEP_ 05 (2019) 084 [1904.00945].
  * [2] S. Abreu, F. Febres Cordero, H. Ita, B. Page and V. Sotnikov, _Leading-color two-loop QCD corrections for three-jet production at hadron colliders_ , _JHEP_ 07 (2021) 095 [2102.13609].
  * [3] H.A. Chawdhry, M.L. Czakon, A. Mitov and R. Poncelet, _NNLO QCD corrections to three-photon production at the LHC_ , _JHEP_ 02 (2020) 057 [1911.00479].
  * [4] S. Abreu, B. Page, E. Pascual and V. Sotnikov, _Leading-Color Two-Loop QCD Corrections for Three-Photon Production at Hadron Colliders_ , _JHEP_ 01 (2021) 078 [2010.15834].
  * [5] H.A. Chawdhry, M. Czakon, A. Mitov and R. Poncelet, _Two-loop leading-color helicity amplitudes for three-photon production at the LHC_ , _JHEP_ 06 (2021) 150 [2012.13553].
  * [6] S. Abreu, G. De Laurentis, H. Ita, M. Klinkert, B. Page and V. Sotnikov, _Two-Loop QCD Corrections for Three-Photon Production at Hadron Colliders_ , 2305.17056.
  * [7] B. Agarwal, F. Buccioni, A. von Manteuffel and L. Tancredi, _Two-loop leading colour QCD corrections to $q\bar{q}\to\gamma\gamma g$ and $qg\to\gamma\gamma q$_, _JHEP_ 04 (2021) 201 [2102.01820].
  * [8] H.A. Chawdhry, M. Czakon, A. Mitov and R. Poncelet, _Two-loop leading-colour QCD helicity amplitudes for two-photon plus jet production at the LHC_ , _JHEP_ 07 (2021) 164 [2103.04319].
  * [9] S. Badger, M. Czakon, H.B. Hartanto, R. Moodie, T. Peraro, R. Poncelet et al., _Isolated photon production in association with a jet pair through next-to-next-to-leading order in QCD_ , 2304.06682.
  * [10] H.B. Hartanto, S. Badger, C. Brønnum-Hansen and T. Peraro, _A numerical evaluation of planar two-loop helicity amplitudes for a W-boson plus four partons_ , _JHEP_ 09 (2019) 119 [1906.11862].
  * [11] S. Abreu, F. Febres Cordero, H. Ita, M. Klinkert, B. Page and V. Sotnikov, _Leading-color two-loop amplitudes for four partons and a W boson in QCD_ , _JHEP_ 04 (2022) 042 [2110.07541].
  * [12] S. Badger, H.B. Hartanto and S. Zoia, _Two-Loop QCD Corrections to Wbb¯ Production at Hadron Colliders_ , _Phys. Rev. Lett._ 127 (2021) 012001 [2102.02516].
  * [13] S. Badger, H.B. Hartanto, J. Kryś and S. Zoia, _Two-loop leading-colour QCD helicity amplitudes for Higgs boson production in association with a bottom-quark pair at the LHC_ , _JHEP_ 11 (2021) 012 [2107.14733].
  * [14] S. Badger, H.B. Hartanto, J. Kryś and S. Zoia, _Two-loop leading colour helicity amplitudes for W ±$\gamma$ \+ j production at the LHC_, _JHEP_ 05 (2022) 035 [2201.04075].
  * [15] H.B. Hartanto, R. Poncelet, A. Popescu and S. Zoia, _Next-to-next-to-leading order QCD corrections to Wbb¯ production at the LHC_ , _Phys. Rev. D_ 106 (2022) 074016 [2205.01687].
  * [16] S. Badger, M. Becchetti, E. Chaubey, R. Marzucca and F. Sarandrea, _One-loop QCD helicity amplitudes for pp → $t\overline{t}j$ to O($\varepsilon$2)_, _JHEP_ 06 (2022) 066 [2201.12188].
  * [17] S. Badger, M. Becchetti, E. Chaubey and R. Marzucca, _Two-loop master integrals for a planar topology contributing to pp → $t\overline{t}j$_, _JHEP_ 01 (2023) 156 [2210.17477].
  * [18] M. Becchetti, _Two-loop Feynman integrals for top-quark pair plus jet production_ , _RADCOR 2023 – 16th International Symposium on Radiative Corrections: Applications of Quantum Field Theory to Phenomenology_ (2023) .
  * [19] S. Kallweit, V. Sotnikov and M. Wiesemann, _Triphoton production at hadron colliders in NNLO QCD_ , _Phys. Lett. B_ 812 (2021) 136013 [2010.04681].
  * [20] H.A. Chawdhry, M. Czakon, A. Mitov and R. Poncelet, _NNLO QCD corrections to diphoton production with an additional jet at the LHC_ , _JHEP_ 09 (2021) 093 [2105.06940].
  * [21] M. Czakon, A. Mitov and R. Poncelet, _Next-to-Next-to-Leading Order Study of Three-Jet Production at the LHC_ , _Phys. Rev. Lett._ 127 (2021) 152001 [2106.05331].
  * [22] S. Badger, T. Gehrmann, M. Marcoli and R. Moodie, _Next-to-leading order QCD corrections to diphoton-plus-jet production through gluon fusion at the LHC_ , _Phys. Lett. B_ 824 (2022) 136802 [2109.12003].
  * [23] X. Chen, T. Gehrmann, E.W.N. Glover, A. Huss and M. Marcoli, _Automation of antenna subtraction in colour space: gluonic processes_ , _JHEP_ 10 (2022) 099 [2203.13531].
  * [24] H.B. Hartanto, R. Poncelet, A. Popescu and S. Zoia, _Flavour anti- $k_{\text{T}}$ algorithm applied to $Wb\bar{b}$ production at the LHC_, 2209.03280.
  * [25] L. Buonocore, S. Devoto, S. Kallweit, J. Mazzitelli, L. Rottoli and C. Savoini, _Associated production of a W boson and massive bottom quarks at next-to-next-to-leading order in QCD_ , _Phys. Rev. D_ 107 (2023) 074032 [2212.04954].
  * [26] M. Alvarez, J. Cantero, M. Czakon, J. Llorente, A. Mitov and R. Poncelet, _NNLO QCD corrections to event shapes at the LHC_ , _JHEP_ 03 (2023) 129 [2301.01086].
  * [27] T. Gehrmann, J.M. Henn and N.A. Lo Presti, _Pentagon functions for massless planar scattering amplitudes_ , _JHEP_ 10 (2018) 103 [1807.09812].
  * [28] D. Chicherin, T. Gehrmann, J.M. Henn, P. Wasser, Y. Zhang and S. Zoia, _All Master Integrals for Three-Jet Production at Next-to-Next-to-Leading Order_ , _Phys. Rev. Lett._ 123 (2019) 041603 [1812.11160].
  * [29] D. Chicherin and V. Sotnikov, _Pentagon Functions for Scattering of Five Massless Particles_ , _JHEP_ 20 (2020) 167 [2009.07803].
  * [30] D. Chicherin, V. Sotnikov and S. Zoia, _Pentagon functions for one-mass planar scattering amplitudes_ , _JHEP_ 01 (2022) 096 [2110.10111].
  * [31] S. Abreu, D. Chicherin, H. Ita, B. Page, V. Sotnikov, W. Tschernow et al., _All Two-Loop Feynman Integrals for Five-Point One-Mass Scattering_ , 2306.15431.
  * [32] D. Chicherin, _Pentagon Functions for One-Mass Scattering_ , _RADCOR 2023 – 16th International Symposium on Radiative Corrections: Applications of Quantum Field Theory to Phenomenology_ (2023) .
  * [33] A. von Manteuffel and R.M. Schabinger, _A novel approach to integration by parts reduction_ , _Phys. Lett. B_ 744 (2015) 101 [1406.4513].
  * [34] T. Peraro, _Scattering amplitudes over finite fields and multivariate functional reconstruction_ , _JHEP_ 12 (2016) 030 [1608.01902].
  * [35] S. Badger, C. Brønnum-Hansen, H.B. Hartanto and T. Peraro, _First look at two-loop five-gluon scattering in QCD_ , _Phys. Rev. Lett._ 120 (2018) 092001 [1712.02229].
  * [36] S. Badger, C. Brønnum-Hansen, H.B. Hartanto and T. Peraro, _Analytic helicity amplitudes for two-loop five-gluon scattering: the single-minus case_ , _JHEP_ 01 (2019) 186 [1811.11699].
  * [37] S. Badger, D. Chicherin, T. Gehrmann, G. Heinrich, J.M. Henn, T. Peraro et al., _Analytic form of the full two-loop five-gluon all-plus helicity amplitude_ , _Phys. Rev. Lett._ 123 (2019) 071601 [1905.03733].
  * [38] T. Peraro, _FiniteFlow: multivariate functional reconstruction using finite fields and dataflow graphs_ , _JHEP_ 07 (2019) 031 [1905.08019].
  * [39] S. Badger, E. Chaubey, H.B. Hartanto and R. Marzucca, _Two-loop leading colour QCD helicity amplitudes for top quark pair production in the gluon fusion channel_ , _JHEP_ 06 (2021) 163 [2102.13450].
  * [40] S. Badger, J. Kryś, R. Moodie and S. Zoia, _Lepton-pair scattering with an off-shell and an on-shell photon at two loops in massless QED_ , 2307.03098.
  * [41] M.L. Mangano and S.J. Parke, _Multiparton amplitudes in gauge theories_ , _Phys. Rept._ 200 (1991) 301 [hep-th/0509223].
  * [42] L.J. Dixon, _Calculating scattering amplitudes efficiently_ , in _Theoretical Advanced Study Institute in Elementary Particle Physics (TASI 95): QCD and Beyond_ , pp. 539–584, 1, 1996 [hep-ph/9601359].
  * [43] S. Badger, J. Henn, J. Plefka and S. Zoia, _Scattering Amplitudes in Quantum Field Theory_ , 2306.05976.
  * [44] P. Nogueira, _Automatic Feynman graph generation_ , _J. Comput. Phys._ 105 (1993) 279.
  * [45] A. Hodges, _Eliminating spurious poles from gauge-theoretic amplitudes_ , _JHEP_ 05 (2013) 135 [0905.1473].
  * [46] S. Badger, _Automating QCD amplitudes with on-shell methods_ , _J. Phys. Conf. Ser._ 762 (2016) 012057 [1605.02172].
  * [47] F.V. Tkachov, _A Theorem on Analytical Calculability of Four Loop Renormalization Group Functions_ , _Phys. Lett. B_ 100 (1981) 65.
  * [48] K.G. Chetyrkin and F.V. Tkachov, _Integration by Parts: The Algorithm to Calculate beta Functions in 4 Loops_ , _Nucl. Phys. B_ 192 (1981) 159.
  * [49] S. Laporta, _High precision calculation of multiloop Feynman integrals by difference equations_ , _Int. J. Mod. Phys. A_ 15 (2000) 5087 [hep-ph/0102033].
  * [50] J.M. Henn, _Multiloop integrals in dimensional regularization made simple_ , _Phys. Rev. Lett._ 110 (2013) 251601 [1304.1806].
  * [51] J.M. Henn, _Lectures on differential equations for Feynman integrals_ , _J. Phys. A_ 48 (2015) 153001 [1412.2296].
  * [52] J. Gluza, K. Kajda and D.A. Kosower, _Towards a Basis for Planar Two-Loop Integrals_ , _Phys. Rev. D_ 83 (2011) 045012 [1009.0472].
  * [53] H. Ita, _Two-loop Integrand Decomposition into Master Integrals and Surface Terms_ , _Phys. Rev. D_ 94 (2016) 116015 [1510.05626].
  * [54] K.J. Larsen and Y. Zhang, _Integration-by-parts reductions from unitarity cuts and algebraic geometry_ , _Phys. Rev. D_ 93 (2016) 041701 [1511.01071].
  * [55] P.A. Baikov, _Explicit solutions of the three loop vacuum integral recurrence relations_ , _Phys. Lett. B_ 385 (1996) 404 [hep-ph/9603267].
  * [56] P.A. Baikov, _Explicit solutions of n loop vacuum integral recurrence relations_ , hep-ph/9604254.
  * [57] J. Böhm, A. Georgoudis, K.J. Larsen, M. Schulze and Y. Zhang, _Complete sets of logarithmic vector fields for integration-by-parts identities of Feynman integrals_ , _Phys. Rev. D_ 98 (2018) 025023 [1712.09737].
  * [58] A. von Manteuffel and R.M. Schabinger, _Quark and gluon form factors in four loop QCD: The $N_{f}^{2}$ and $N_{q\gamma}N_{f}$ contributions_, _Phys. Rev. D_ 99 (2019) 094014 [1902.08208].
  * [59] S. Badger, C. Brønnum-Hansen, D. Chicherin, T. Gehrmann, H.B. Hartanto, J. Henn et al., _Virtual QCD corrections to gluon-initiated diphoton plus jet production at hadron colliders_ , _JHEP_ 11 (2021) 083 [2106.08664].
  * [60] D. Chicherin, J. Henn and V. Mitev, _Bootstrapping pentagon functions_ , _JHEP_ 05 (2018) 164 [1712.09610].
  * [61] S. Abreu, J. Dormans, F. Febres Cordero, H. Ita and B. Page, _Analytic Form of Planar Two-Loop Five-Gluon Scattering Amplitudes in QCD_ , _Phys. Rev. Lett._ 122 (2019) 082002 [1812.04586].
  * [62] M. Czakon, _Jet and Photon Physics at high perturbative orders in QCD_ , _RADCOR 2023 – 16th International Symposium on Radiative Corrections: Applications of Quantum Field Theory to Phenomenology_ (2023) .