# Improving conversion rate prediction via self-supervised pre-training in
online advertising

Alex Shtoff  Yahoo Research
Haifa, Israel
<EMAIL_ADDRESS>Yohay Kaplan  Yahoo Research
Haifa, Israel
<EMAIL_ADDRESS>Ariel Raviv  Yahoo Research
Haifa, Israel
<EMAIL_ADDRESS>
###### Abstract

The task of predicting conversion rates (CVR) lies at the heart of online
advertising systems aiming to optimize bids to meet advertiser performance
requirements. Even with the recent rise of deep neural networks, these
predictions are often made by factorization machines (FM), especially in
commercial settings where inference latency is key. These models are trained
using the logistic regression framework on labeled tabular data formed from
past user activity that is relevant to the task at hand.

Many advertisers only care about click-attributed conversions, which are
conversions that occurred after a user has clicked on an ad. A major challenge
in training models that predict conversions-given-clicks comes from data
sparsity - clicks are rare, conversions attributed to clicks are even rarer.
However, mitigating sparsity by adding conversions that are not click-
attributed to the training set impairs model calibration, causing the mean
prediction to no longer converge to the actual CVR. Since calibration is
critical to achieving advertiser goals, this is infeasible.

In this work we use the well-known idea of self-supervised pre-training, and
use an auxiliary auto-encoder model trained on all conversion events, both
click-attributed and not, as a feature extractor to enrich the main CVR
prediction model. Since the main model does not train on non click-attributed
conversions, this does not impair calibration. We adapt the basic self-
supervised pre-training idea to our online advertising setup by using a loss
function designed for tabular data, facilitating continual learning by
ensuring auto-encoder stability, and incorporating a neural network into a
large-scale real-time ad auction that ranks tens of thousands of ads, while
conforming to the strict latency constraints, and without incurring a major
engineering cost. We evaluate our approach and show improvements both offline,
during training, and in an online A/B test. Following its success in A/B
tests, our solution is now fully deployed to the Yahoo native advertising
system, and its impact is measured in millions of dollars annually.

††publicationid: pubid: 979-8-3503-2445-7/23/$31.00 ©2023 IEEE

##  I Introduction

The Yahoo native ads marketplace serves users with native ads that resemble
the surrounding content (see Figure 1). It shows billions of daily impressions
with a run-rate of several hundreds of millions USD each year. In order to
rank native ads for incoming users and their specific context according to the
cost-per-click price type, the expected revenue of each ad is computed as a
product of the advertiser’s bid and the predicted click probability. The click
probability is produced by a feature enhanced collaborative filtering
algorithm called Offset [1].

Figure 1: A native ad on Yahoo homepage that resembles the surrounding
content.

Advertiser bids, that should reflect the value of an ad click to the
advertiser, are either specified manually, or chosen programmatically
according to several strategies designed to meet various advertiser goals. At
the heart of programmatic bid optimization lie Offset models that compute, for
each ad, the predicted conversion rate (pCVR) given a click for a given user
and a context. For example, one of the strategies allows advertisers to
specify the sum they are willing to pay for a conversion, also known as the
target cost per acquisition (tCPA). The bid, in that case, is

$\operatorname{bid}=\operatorname{pCVR}\cdot\operatorname{tCPA}.$

Clearly, calibration of the $\operatorname{pCVR}$ model, in the sense that its
average prediction approximates the true CVR on any traffic segment, is
crucial. Consistent over-prediction causes over-bidding, which results in the
advertisers paying more than they desire per conversion. Under-prediction
causes under-bidding, which results in the advertisers winning less auctions
and paying less for the ones they won, meaning the advertisers are losing
exposure, and we are losing revenue.

Offset is a variant of a factorization machine [23] that is trained using the
logistic regression framework involving positive and negative events. Offset
trains incrementally on an infinite stream of data, with periodic checkpoints
for hyper-parameter tuning [2] and deployment to the production environment.
The incremental training methodology was adopted to make sure that the delay
between an event happening and a model trained on that event being deployed to
production is as small as possible. In many systems [26, 16, 34], including
ours, model freshness is of paramount importance for the performance of
content recommendation systems deployed in a changing environment.

Training CVR prediction model poses many key challenges, one of which is the
data sparsity issue. Accurate models require a large amount of data, but the
required amount of data when training conversion models is not always
available. For example, typically a few percents of the impressions result in
clicks, and a few percents of those result in a conversion. Thus, the number
of clicks and conversions for training is quite small, and the coverage of
various feature combinations, such as ”25-30 years old people using a mobile
device”, is even smaller.

An initial idea that might come to mind is enriching the training set with
additional data, such as conversions that are not attributed to clicks, or
impression data in addition to clicks. However, since we require the model for
the regression task of CVR prediction, rather than a classification task, it
is unclear how the labels of such events should be assigned to produce a
calibrated model.

To avoid feeding the main CVR prediction model with data that may impair
calibration, we use the celebrated approach of self-supervised pre-training.
We pre-train an auto-encoder model [7, 14] on _all_ conversions, both related
and unrelated to the task at hand. Then, we train the main CVR prediction
model _only_ on the relevant data while using the encoder’s output, which we
refer to as the _code_ , as an additional feature, meaning that the we attach
task-specific prediction layers on top of the encoder. Since these layers are
trained only on the task-specific data, no bias is introduced. Moreover, since
the code is low-dimensional, training a low dimensional model is potentially
immune to data sparsity.

The objective of our design is to provide the CVR prediction model with
valuable information about user conversion patterns available in the
additional data present in non click-attributed conversions, without accessing
it directly. The results of the online and offline experiments in this paper
support our thesis by demonstrating improved performance of our CVR prediction
model, both offline and in an online A/B test.

The setting of online advertising and incremental training pose unique
challenges that require adapting the standard self-supervised pre-training
methodology. Hence, we present several design decisions for tailoring the
methodology towards a system performing auctions under strict latency
constraints, driven by a factorization machine variant that is incrementally
trained on tabular data. Concretely, the decisions include the choice of a
loss function, careful feature selection, and a model architecture that is
driven by a set of criteria and metrics designed to evaluate an incrementally
training auto-encoder.

To summarize, the main contributions of our paper are:

  * •

An application of self-supervised pre-training to utilize additional data from
conversion events that a CVR model cannot access directly during training.

  * •

A set of techniques for training and using auto-encoders with incrementally
trained factorization machine variants that are used in ranking under strict
latency constraints.

##  II Related work

Our work primarily addresses the task of conversion prediction[33]. This task
is similar to click prediction, the ’core’ task of online ad models, and often
uses similar approaches. Its distinct challenges, such as the click-to-
conversion delay, the difficulty in attribution to a specific event, and the
attribution gap between different platforms and advertisers have been tackled
in [10, 18, 27, 17] and many subsequent works.

In this work we adapt a self-supervised pre-training technique to the task of
conversion rate prediction in a strict latency constraints settings. Self-
supervised pre-training is the technique that leverages self-supervised
learning [19, 11] to generate feature representations, typically using a deep
neural network, that are (somewhat) agnostic of the overall prediction task
and subsequently injecting them in some fashion to the dedicated predictive
model. In previous works this main model has also been some type of deep-
neural networks model [28, 30, 35]. In [20] a universal user representation is
learned from multiple forms of data and then applied to multiple predictive
models. [36] considers a sequential ordering of predictive tasks and injects
representations learned from multiple tasks into various points of a self-
attentive network. [13, 31] suggest ways of taking the standard deep learning
predictive flow (i.e., embedding layer into cross layers into predictive
output) and injecting intermediate output of one task’s model into the other
(and vice versa) to boost the performance of both.

In contrast to recent works that tackle this task using deep neural networks,
our underlying model is a variant of a factorization machine [23],
specifically Offset [1, 2]. This is also the underlying model for most
prediction tasks in Yahoo’s native advertising network.

##  III Our approach

In this section we describe the techniques we use to integrate an auto-encoder
into the CVR prediction models in a way that facilitates incremental training,
and enables real-time ad auctions under strict latency constraints.

###  III-A Training a CVR model with an autoencoder

Like many probability-based regressions models, Offset models compute their
output (in this case, a $\operatorname{pCVR}$) with the sigmoid function:

$\operatorname{pCVR}(\Omega)=\frac{1}{1+e^{-f(\Omega)}}$

where $\Omega$ is the user,ad and contextual properties of the given event.
The $f$ used by Offset can broken into:

$f(\Omega)=\langle\mathbf{U},\mathbf{A}\rangle+\sum_{i\in S}w_{i}x_{i}+b$ (1)

where $\mathbf{U}$ is a latent representation vector of the user and
contextual features, $\mathbf{A}$ is a latent representation of the ad
features, $S$ is a set of potential additional features, $x_{i}$ is an
indicator the feature $i$, $w_{i}$ is the learned weight for feature $i$ and
$b$ is a global bias. The continual model training process is responsible for
generating the latent representation for all possible combinations of
user,contextual and ad features as well as the various $w_{i}$’s and the
global bias term. For more on how Offset constructs the various latent vectors
from $\Omega$ and its parameters please see [1] and [2]. $S$ is used for
features that aren’t separable between the user and ad, such as how often and
how recently this user has seen this ad, see [3] for more. The training
process uses the logloss function as its error function:

$\mathcal{L}(\Omega,y,t)=-(1-y)\log\left(1-\operatorname{pCVR}(\Omega)\right)-y\log\operatorname{pCVR}(\Omega)$

where y is an indicator for a positive event (in our case, click attributed
conversion)

We quickly note that this form of computation has benefits for the ability to
handle real-time inference. Each ad has it pre-computed latent representation,
while each user has a singular latent vector (calculated at inference time).

We want to introduce the code generated by the auto-encoder into this
computation. Since the code depends on both user and ad features, it makes
sense to include it in the same way that $S$ is used, so as not to intrude on
the portions of the computation that are user/ad isolated. So the resulting
$f$ is

$f(\Omega)=\langle\mathbf{U},\mathbf{A}\rangle+\sum_{i\in
S}w_{i}x_{i}+b+\langle\mathbf{W},C(\Omega)\rangle$ (2)

where $C(\Omega)$ is the code associated with a specific event, and
$\mathbf{W}$ is vector of weight learned by the model as part of the training
process. The resulting system is illustrated in Figure 2. Learning a linear
function of the code is our way to re-use the existing training and serving
infrastructure that is built for Offset models, that already include a linear
term. Although at first glance it might look like having a limited
representation power, we show in later sections how we significantly enhance
it using a classical method in machine learning.

Figure 2: Schematic illustration of our self-supervised pre-training
framework. On the left - the auto-encoder that is trained on all conversions,
both click-attributed and not. On the right - the CVR prediction model that is
trained on clicks and click-attributed conversions, and uses the encoder’s
code for its training data as an additional feature.

So a single training interval of the model does the following:

  1. 1.

in interval $t$ \- load previous Offset model $M_{t-1}$ and previous encoder
$Enc_{t-1}$

  2. 2.

For each event $\Omega$, calculate the code $C=Enc_{t-1}(\Omega)$

  3. 3.

(CVR training) adapt model parameters (including $\mathbf{W}$) using SGD
according to $f(\Omega,C)$ using logloss error

  4. 4.

(Autoencoder trainig) adapt parameters of the auto-encoder, whose encoder is
$Enc_{t}$.

###  III-B Architectural challenges

Despite the architecture’s apparent simplicity and ease of integration into an
existing serving system, it poses several challenges. We describe them here,
and present our solution in the following sub-sections. The first challenge
stems from incremental training. The basic premise is that the model has long-
term memory, and its parameters at interval $t-1$ are a good initialization
point for training at interval $t$. Therefore, the codes $C(\Omega)$ need to
be stable, in the sense that the expectation
$\mathbb{E}_{\Omega}[Enc_{t-2}(\Omega)-Enc_{t-1}(\Omega)]$ is small.
Otherwise, the premise is broken, since the vector $W$ that Offset learned in
the previous interval encodes little information that is useful at interval
$t$.

Second, the expressive power of the linear function $\mathbf{W}^{T}C(\Omega)$
is weak. Indeed, typically the prediction layers on top of an auto-encoder is,
by itself, a neural network of several layers. This is because the separation
between positive and negative samples in the encoder’s latent space is often
non-linear.

The final challenge comes from our need to perform fast real-time auctions. In
our system, the time it takes to use a neural network for every item in the
auction introduces latency that is orders of magnitude beyond our latency
constraints.

###  III-C Auto-encoder architecture and design

Suppose our data-set comprises of $C$ categorical columns where column $i$ has
one of $n_{i}$ possible values. The encoder’s first layer is an embedding
layer - each column has an embedding table of dimension $n_{i}\times d$. Next,
the embedding vectors are concatenated to form a vector of length $C\cdot d$,
which is then passed to a multi-layer perceptron (MLP) network to produce the
code. The hidden layers use ReLU activations, while the layer that produces
the code uses a $\tanh$ activation to produce code that is bounded in the unit
box.

Since the encoder’s input is composed of categorical features, we treat the
decoder as a set of multi-class classifiers with a classifier of $n_{i}$
classes for every column. To that end, it consists of an MLP that transforms
the code to a vector of size $\sum_{i=1}^{C}n_{i}$, that we treat as $C$
blocks of size $n_{i}$ each. The last layer applies a log-softmax activation
to each block to produce valid multi-class logits. The reconstruction loss is,
naturally, the average of cross-entropy losses applied to each block. Our
architecture is illustrated in Figure 3. We denote the loss of auto-encoder
$M$ on sample $\Omega$ as $\operatorname{RecLoss}(M,\Omega)$, and the code it
produces by $\operatorname{Code}(M,\Omega)$. We note that the naïve method of
reconstructing a concatenation of one-hot encoded feature indicators using the
L2 loss did not result in any improvement of the downstream CVR model.

Figure 3: Auto-encoder model architecture. For clarity, illustrated with $C=3$
tabular columns. The input is used to choose embeddings, that are
concatenated, and fed to a multi-layer perceptron to produce the code. The
decoder, in turn, is a multi-layer perceptron that acts as a set of
classifiers with cross-entropy losses, that are averaged to obtain the output.

Just like Offset, our auto-encoder is trained in an online manner on data sub-
divided into intervals $D_{1},D_{2},\dots$, training on each event only once.
The process produces a sequence of auto-encoder models $M_{1},M_{2},\dots$. On
our data, the optimizer that produced the best results was stochastic
gradient-descent with momentum [24, 21]. We found the online paradigm to
produce satisfactory results, and our use of the paradigm is driven both by
the practical success of Offset and theoretical results about generalization
of online learning [9]. We note that in the online learning paradigm there is
no test set separate from the training set, and instead each sample is first
evaluated and then trained upon.

The architecture we describe is the result of a process that was driven by
empirical evaluation of several metrics that represent its quality in terms of
learning data patterns, stability, and generalization. We gradually increased
the complexity of the encoder until we found its performance to be
satisfactory, while meeting the strict latency constraints. Beyond embedding,
code, and hidden layer dimensions, we also tried different ways of combining
the column embeddings: addition, product, sum of pairwise component-wise
products (à la FM), sum of pairwise outer product matrices, and concatenation.
The best results according to the metrics described below were obtained with
concatenation.

We designed our model with several desired properties in mind. Below, we
describe these properties and the metrics we used to measure the model’s
performance against them.

#### Small reconstruction loss

We aim at a reconstruction loss that is much smaller than that of uniformly
distributed columns:

$\operatorname{RecLoss}_{t}\equiv\operatorname{\mathbb{E}}_{\Omega\sim
D_{t}}[\operatorname{RecLoss}(M_{t},\Omega)]\ll\frac{1}{C}\sum_{i=1}^{C}\ln(n_{i}).$

#### Stability

The code produced by the encoder of the current interval should be close to
the one produced by the encoder of the previous interval. Therefore, the
following quantity should be as small as possible:

$\mathrm{Diff}_{t}=\operatorname{\mathbb{E}}_{\Omega\sim
D_{t}}[\|\operatorname{Code}(M_{t},\Omega)-\operatorname{Code}(M_{t-1},\Omega)\|].$

#### Interval generalization

Previous interval’s auto-encoder should generalize well to the samples in the
current interval, and the following quantity should be close to 1:

$\mathrm{Gen}_{t}=\frac{\operatorname{\mathbb{E}}_{\Omega\sim
D_{t}}[\operatorname{RecLoss}(M_{t},\Omega)]}{\operatorname{\mathbb{E}}_{\Omega\sim
D_{t}}[\operatorname{RecLoss}(M_{t-1},\Omega)]}.$

#### Learning meaningful patterns

The encoder should reconstruct real data well, and random data badly. The
random data-set $R_{t}$ is of the same length as $D_{t}$, but with column
values chosen uniformly at random from the column’s dictionary. The
reconstruction loss on $R_{t}$ is denoted by
$\operatorname{RecLoss}^{R}_{t}\equiv\operatorname{\mathbb{E}}_{\Omega\sim
R_{t}}[\operatorname{RecLoss}(M_{t},\Omega)]$. The random loss ratio, defined
by

$\mathrm{RandRatio}_{t}=\frac{\operatorname{RecLoss}_{t}}{\operatorname{RecLoss}^{R}_{t}},$

should be close to zero, preferably $<10^{-2}$ \- the reconstruction of real
data should be _orders of magnitude_ better than that of random data.

We would like to point out that interval generalization $\mathrm{Gen}_{t}$ is
tightly related to stability $\mathrm{Diff}_{t}$. In fact, generalization
implies stability. Intuitively, if the model from the last interval
generalizes well to the current interval’s data, then its loss gradients w.r.t
the data of the current interval are small, and its parameters will not change
significantly. In the evaluation section we show that this phenomenon happens
in practice.

We observed that the most profound positive effect on all of the above metrics
was obtained from embedding concatenation, rather than other strategies of
handling the embedding vectors, and from increasing the embedding dimension.
Increasing the number of hidden layers beyond a certain amount had little
effect, and so was increasing their dimension. At first the above seems to be
in contrast to classical machine-learning, where increasing model complexity
results in over-fitting and reduces generalization. However, it is known [5,
15, 6] that over-parameterized networks, which are networks that are complex
enough to achieve near zero training loss, generalize well. As is apparent in
our evaluation section, our auto-encoder’s training loss is indeed close to
zero. This is despite the fact that it is quite small - we have 7 categorical
columns, the embedding dimension is 20, the code dimension is 12, and the
total number of parameters of our encoder is roughly $10^{5}$.

###  III-D Nonlinear separability

Having learned an encoder, there is no guarantee that a linear function can
differentiate well between positive and negative samples in the encoder’s
embedding space. Classical machine learning methodology suggests using kernel
methods [4, 8, 25] as the go-to technique for creating linear models that
represent non-linear functions. However, kernel methods require having the
entire training set in advance, and thus do not fit incremental training. As a
remedy, we use random Fourier features (RFFs) [22] as an approximation of the
Gaussian radial basis function kernel. This technique suits incremental
training well, since it dictates a simple formula for transforming the raw
$\operatorname{Code}(\Omega)$ into the feature vector $C(\Omega)$ in Equation
(2) with randomly generated matrix $\mathbf{P}$ and vector $\mathbf{q}$:

$C(\Omega)=\cos(\mathbf{P}\operatorname{Code}(\Omega)+\mathbf{q}).$

To verify that RFFs have significantly more representation power, we ran a
simple experiment of training a linear CVR model based on the encoder’s code,
with and without RFFs. We observed that RFFs provide a substantial improvement
of 15% in logloss and 3.6% in AUC. The model improves as the number of rows of
$\mathbf{P}$ increases, until it reaches a plateau at 200 rows. This means
that computing $C(\Omega)$ requires an additional multiplication by by a
$200\times 12$ matrix, which does not incur any significant latency cost.

###  III-E Latency constraints

During an ad auction we rank tens of thousands of ads in a matter of
milliseconds. Neural network inference for every ad is prohibitive, because it
introduced a significant latency cost far beyond the acceptable range in our
system. The above is true even for small encoders of a few tens of thousands
of parameters. In theory, if we had only user features in the auto-encoder, we
could use the code only _once_ per auction. However, an auto-encoder that was
trained without features of the ads that were responsible for conversions did
not improve the downstream CVR model.

To avoid inference for every ad, we use only high-level ad taxonomy features
in our auto-encoder. In our system, we have a two-level taxonomy hierarchy -
the first level breaks down ads by high-level categories, such as Finance or
Tourism, and the second level by a few dozen categories. Hence, during
ranking, we need to compute the product $W^{T}C(\Omega)$ from Equation (2)
only once for every low-level category, and add the result to the score
$f(\Omega)$ we compute for every ad according to its taxonomy.

##  IV Evaluation

We performed several experiments on our CVR models that predict the
probability of a conversion given a click. For training our CVR prediction
model, we treat clicks as negative events, and conversions attributed to
clicks on our properties as positive events. In addition, we also have
conversions that are attributed to ad views. All models train on daily
intervals.

In this section we first corroborate our thesis that directly adding view-
attributed conversions to the training set produces a less accurate model.
Then, we evaluate our auto-encoder according to the metrics in Section III-C.
Next, we show that equipping a CVR model with an auto-encoder improves the
model’s accuracy during training. Then, we verify that the incurred
computational cost has no visible effect on the auction latency, and
demonstrate the improved performance of the model on real traffic via an A/B
test against the production model.

###  IV-A Training on view-attributed conversions

The paper’s premise is that training on conversions that are not attributed to
clicks, in addition to those that are attributed to clicks, degrades model
accuracy. We trained a model that includes view-attributed conversions as
positive events, in addition to the relevant events. In both models, the
logloss was evaluated only for clicks and conversions attributed to clicks.
The logloss achieved by the model that includes view-attributed conversion is
between 3% and 7% higher (depending on the day) than that of the model which
does not include them, which makes such a model much _worse_.

###  IV-B Auto-encoder evaluation

Here we evaluated the desired properties of our auto-encoder using the metrics
$\operatorname{RecLoss}_{t}$, $\operatorname{RecLoss}^{R}_{t}$,
$\mathrm{Gen}_{t}$, and $\mathrm{Diff}_{t}$ from Section III-C.

In order to validate that our auto-encoder learns meaningful patterns, rather
than just the identity function, we measure $\operatorname{RecLoss}_{t}$ and
$\operatorname{RecLoss}^{R}_{t}$ for an auto-encoder trained on a month worth
of data. We plot the results in Figure 4. It is evident that as the model
trains on more data, its performance on true data improves, on random data
degrades, the reconstruction loss approaches zero, and the
$\mathrm{RandRatio}_{t}$ approaches $10^{-4}$, as desired.

Figure 4: Reconstruction losses over real and random data, of an auto-encoder
trained on a month worth of data.

In order to verify that our auto-encoder is stable and generalizes well
between consecutive days, we measure $\mathrm{Gen}_{t}$ and
$\mathrm{Diff}_{t}$ over a few weeks worth of data. The results are plotted in
Figure 5, where each point is the result of training on a day of data. As is
apparent, most of the values of $\mathrm{Gen}_{t}$ are close to 1, which means
that our auto-encoder often generalizes well. Moreover, as we claimed in
Section III-C, generalization and stability are tightly coupled. Indeed, most
of the points lie along a straight regression line, and there are no points
where $\mathrm{Gen}_{t}$ is small but $\mathrm{Diff}_{t}$ is large.

Figure 5: Daily generalization metric $\mathrm{Gen}_{t}$ (closer to 1 is
better) versus daily stability metric $\mathrm{Diff}_{t}$ (lower is better).

###  IV-C CVR model offline lifts

We train a new CVR model using our auto-encoder based approach, and measure
the logloss difference between the new model and the production model at the
time. In contrast to a CVR model that is directly trained on view-attributed
conversions, the auto-encoder based model has an improved logloss, as is
apparent from Figure 6.

Figure 6: Daily logloss difference (negative is better), in percents, between
a model enriched by an auto-encoder, and the production model at the time.
Marker size is proportional to the evaluation set size.

###  IV-D Incurred latency tests

We deployed the new workflow to our production system by periodically loading
the encoder that is trained with the model, and using it in ad auctions
according to the formula in Equation (2). Since we use only high level ad
taxonomy features, our ad auction computes the inner product
$\langle\mathbf{W},C(\Omega)\rangle$ only once for each ad category, and uses
the cached results for all the ads participating in the auction. We compare
the latency incurred by auctions where our new model is used to auctions with
the production model at that time in Figure 7. Indeed, the size of the model
and the usage of high level features paid off - there is essentially no
visible latency difference between the new and the product model at the time.

Figure 7: Average hourly latency difference, in milliseconds, between the new
encoder-augmented model and the production model at the time. Negative values
are better.

###  IV-E Online A/B tests

#### Setup

A CVR model performance is measured along two axes - the revenue measured by
the average cost per thousand impressions (CPM), and the percentage of
advertiser spend belonging to campaigns that attain advertiser cost per
acquisition (CPA) goals. The objective is obtaining improvement in at least
one metric without impairing the other. This is since a better model can
either correct under-prediction, which results in lower bids and lower
revenue, or over-prediction, which results in missing the advertiser CPA goal.
To evaluate the online performance, we launched an online bucket serving 50%
of Yahoo native traffic and measured lifts of both metrics against the
production model. The evaluation was conducted over a week and included
billions of impressions.

#### Results

On average, the online test bucket showed $0.6$% CPM lift and $0.67$% revenue
lift overall. Such a CPM (or revenue) lift translates to many millions of USD
in yearly revenue once the solution is deployed to all traffic. The results
also indicate a $+10.1$% increase to the spend that is under oCPC influence,
with a slightly larger percentage of spend belonging to campaigns hitting the
CPA goal ($83.45$% vs $82.96$% in test and control bucket receptively). In
short, the results show that the new model increases the overall revenue while
maintaining the advertiser’s CPA goals. Following these positive results, the
new model was fully deployed and serves 100% of the Yahoo Gemini native
traffic.

##  V Discussion and future work

We described a framework for reducing the effect of data sparsity in CVR
prediction by incorporating conversions that are considered un-labaled via
self-supervised pre-training. Moreover, we adapted the basic idea to systems
based on incrementally-trained factorization machines that are used in ad
auctions with strict latency constraints. We have shown that even a simple
auto-encoder with a small number of parameters already provides a significant
improvement. Therefore, natural future directions, beyond adding more
features, should aim to squeeze more information out of the un-labaled
conversions by further improvements of model architecture and training
techniques.

One direction is looking for better ways to incorporate the encoder’s output
into the factorization model without incurring a significant latency increase.
Currently, we are using the random Fourier features technique as a ‘projector
network’. It can be viewed as a shallow one-layer network whose activation is
the cosine function, and its weights are random, rather than learned. However,
the projector does not have to be a random one-layer network. It can be an
arbitrary deep neural network with arbitrary activation functions and
learnable weights. As long as this network is small enough to facilitate fast
inference, it can potentially squeeze out more performance out of the encoder
network. Moreover, the projector’s incorporation into the factorization model
does not have to be in the form of an additive term, but can be in the form of
another latent vector. For example, instead modifying the score in Equation
(1) as in (2), it can be alternatively modified as

$f(\Omega)=\langle\mathbf{U},\mathbf{A},P(\operatorname{Code}(\Omega))\rangle+\sum_{i\in
S}w_{i}x_{i}+b,$

where
$\langle\mathbf{x},\mathbf{y},\mathbf{z}\rangle=\sum_{i=1}^{n}x_{i}y_{i}z_{i}$
denotes a ”triple” inner product, and $P(\cdot)$ is the projector network
whose output dimension is equal to the dimension of the user and ad vectors.

Another prominent direction is using more advanced self-supervised learning
techniques, such as masked auto-encoders [12], or even drifting away from the
auto-encoder framework and use techniques such as Barlow Twins [32]. Another
improvement can potentially come from better model architectures, such as
using small transformers [29] to process feature embedding vectors.

## References

  * [1] Michal Aharon, Natalie Aizenberg, Edward Bortnikov, Ronny Lempel, Roi Adadi, Tomer Benyamini, Liron Levin, Ran Roth, and Ohad Serfaty.  Off-set: one-pass factorization of feature sets for online recommendation in persistent cold start settings.  In Proc. RecSys’2013, pages 375–378, 2013.
  * [2] Michal Aharon, Amit Kagian, and Oren Somekh.  Adaptive online hyper-parameters tuning for ad event-prediction models.  In Proceedings of the 26th International Conference on World Wide Web Companion, pages 672–679. International World Wide Web Conferences Steering Committee, 2017.
  * [3] Michal Aharon, Yohay Kaplan, Rina Levy, Oren Somekh, Ayelet Blanc, Neetai Eshel, Avi Shahar, Assaf Singer, and Alex Zlotnik.  Soft frequency capping for improved ad click prediction in yahoo gemini native.  In Proceedings of the 28th ACM International Conference on Information and Knowledge Management, CIKM ’19, page 2793–2801, New York, NY, USA, 2019. Association for Computing Machinery.
  * [4] A Aizerman.  Theoretical foundations of the potential function method in pattern recognition learning.  Automation and remote control, 25:821–837, 1964.
  * [5] Zeyuan Allen-Zhu, Yuanzhi Li, and Yingyu Liang.  Learning and generalization in overparameterized neural networks, going beyond two layers.  In H. Wallach, H. Larochelle, A. Beygelzimer, F. d'Alché-Buc, E. Fox, and R. Garnett, editors, Advances in Neural Information Processing Systems, volume 32. Curran Associates, Inc., 2019.
  * [6] Sanjeev Arora, Rong Ge, Behnam Neyshabur, and Yi Zhang.  Stronger generalization bounds for deep nets via a compression approach.  In Jennifer Dy and Andreas Krause, editors, Proceedings of the 35th International Conference on Machine Learning, volume 80 of Proceedings of Machine Learning Research, pages 254–263. PMLR, 10–15 Jul 2018\.
  * [7] Dana H. Ballard.  Modular learning in neural networks.  In Proceedings of the Sixth National Conference on Artificial Intelligence - Volume 1, AAAI’87, page 279–284. AAAI Press, 1987.
  * [8] Bernhard E. Boser, Isabelle M. Guyon, and Vladimir N. Vapnik.  A training algorithm for optimal margin classifiers.  In Proceedings of the Fifth Annual Workshop on Computational Learning Theory, COLT ’92, page 144–152, New York, NY, USA, 1992. Association for Computing Machinery.
  * [9] Nicolò Cesa-bianchi, Alex Conconi, and Claudio Gentile.  On the generalization ability of on-line learning algorithms.  In T. Dietterich, S. Becker, and Z. Ghahramani, editors, Advances in Neural Information Processing Systems, volume 14. MIT Press, 2001\.
  * [10] Olivier Chapelle.  Modeling delayed feedback in display advertising.  In Proceedings of the 20th ACM SIGKDD international conference on Knowledge discovery and data mining, pages 1097–1105, 2014.
  * [11] Jacob Devlin, Ming-Wei Chang, Kenton Lee, and Kristina Toutanova.  Bert: Pre-training of deep bidirectional transformers for language understanding.  arXiv preprint arXiv:1810.04805, 2018.
  * [12] Mathieu Germain, Karol Gregor, Iain Murray, and Hugo Larochelle.  Made: Masked autoencoder for distribution estimation.  In Francis Bach and David Blei, editors, Proceedings of the 32nd International Conference on Machine Learning, volume 37 of Proceedings of Machine Learning Research, pages 881–889, Lille, France, 07–09 Jul 2015\. PMLR.
  * [13] Guangneng Hu, Yu Zhang, and Qiang Yang.  Conet: Collaborative cross networks for cross-domain recommendation.  In Proceedings of the 27th ACM international conference on information and knowledge management, pages 667–676, 2018.
  * [14] Mark A Kramer.  Nonlinear principal component analysis using autoassociative neural networks.  AIChE journal, 37(2):233–243, 1991.
  * [15] Roi Livni, Shai Shalev-Shwartz, and Ohad Shamir.  On the computational efficiency of training neural networks.  In Z. Ghahramani, M. Welling, C. Cortes, N. Lawrence, and K.Q. Weinberger, editors, Advances in Neural Information Processing Systems, volume 27. Curran Associates, Inc., 2014.
  * [16] Andreas Lommatzsch, Benjamin Kille, and Sahin Albayrak.  Incorporating context and trends in news recommender systems.  In Proceedings of the international conference on web intelligence, pages 1062–1068, 2017.
  * [17] Quan Lu, Shengjun Pan, Liang Wang, Junwei Pan, Fengdan Wan, and Hongxia Yang.  A practical framework of conversion rate prediction for online display advertising.  In Proceedings of the ADKDD’17, pages 1–9. 2017.
  * [18] Mohammad Mahdian and Kerem Tomak.  Pay-per-action model for online advertising.  In Proceedings of the 1st international workshop on Data mining and audience intelligence for advertising, pages 1–6, 2007.
  * [19] Tomas Mikolov, Ilya Sutskever, Kai Chen, Greg S Corrado, and Jeff Dean.  Distributed representations of words and phrases and their compositionality.  Advances in neural information processing systems, 26, 2013.
  * [20] Yabo Ni, Dan Ou, Shichen Liu, Xiang Li, Wenwu Ou, Anxiang Zeng, and Luo Si.  Perceive your users in depth: Learning universal user representations from multiple e-commerce tasks.  In Proceedings of the 24th ACM SIGKDD International Conference on Knowledge Discovery & Data Mining, pages 596–605, 2018.
  * [21] Ning Qian.  On the momentum term in gradient descent learning algorithms.  Neural Networks, 12(1):145–151, 1999.
  * [22] Ali Rahimi and Benjamin Recht.  Random features for large-scale kernel machines.  In J. Platt, D. Koller, Y. Singer, and S. Roweis, editors, Advances in Neural Information Processing Systems, volume 20. Curran Associates, Inc., 2007.
  * [23] Steffen Rendle.  Factorization machines.  In 2010 IEEE International Conference on Data Mining, pages 995–1000. IEEE, 2010.
  * [24] Herbert Robbins and Sutton Monro.  A Stochastic Approximation Method.  The Annals of Mathematical Statistics, 22(3):400 – 407, 1951.
  * [25] Bernhard Schölkopf, Alexander Smola, and Klaus-Robert Müller.  Nonlinear component analysis as a kernel eigenvalue problem.  Neural Computation, 10(5):1299–1319, 1998.
  * [26] David Sculley, Gary Holt, Daniel Golovin, Eugene Davydov, Todd Phillips, Dietmar Ebner, Vinay Chaudhary, Michael Young, Jean-Francois Crespo, and Dan Dennison.  Hidden technical debt in machine learning systems.  Advances in neural information processing systems, 28:2503–2511, 2015.
  * [27] Xuhui Shao and Lexin Li.  Data-driven multi-touch attribution models.  In Proceedings of the 17th ACM SIGKDD international conference on Knowledge discovery and data mining, pages 258–264, 2011.
  * [28] Fei Sun, Jun Liu, Jian Wu, Changhua Pei, Xiao Lin, Wenwu Ou, and Peng Jiang.  Bert4rec: Sequential recommendation with bidirectional encoder representations from transformer.  In Proceedings of the 28th ACM international conference on information and knowledge management, pages 1441–1450, 2019.
  * [29] Ashish Vaswani, Noam Shazeer, Niki Parmar, Jakob Uszkoreit, Llion Jones, Aidan N Gomez, Ł ukasz Kaiser, and Illia Polosukhin.  Attention is all you need.  In I. Guyon, U. Von Luxburg, S. Bengio, H. Wallach, R. Fergus, S. Vishwanathan, and R. Garnett, editors, Advances in Neural Information Processing Systems, volume 30. Curran Associates, Inc., 2017.
  * [30] Fajie Yuan, Xiangnan He, Alexandros Karatzoglou, and Liguang Zhang.  Parameter-efficient transfer from sequential behaviors for user modeling and recommendation.  In Proceedings of the 43rd International ACM SIGIR conference on research and development in Information Retrieval, pages 1469–1478, 2020.
  * [31] Feng Yuan, Lina Yao, and Boualem Benatallah.  Darec: Deep domain adaptation for cross-domain recommendation via transferring rating patterns.  arXiv preprint arXiv:1905.10760, 2019.
  * [32] Jure Zbontar, Li Jing, Ishan Misra, Yann LeCun, and Stephane Deny.  Barlow twins: Self-supervised learning via redundancy reduction.  In Marina Meila and Tong Zhang, editors, Proceedings of the 38th International Conference on Machine Learning, volume 139 of Proceedings of Machine Learning Research, pages 12310–12320. PMLR, 18–24 Jul 2021.
  * [33] Weinan Zhang, Shuai Yuan, and Jun Wang.  Optimal real-time bidding for display advertising.  In Proceedings of the 20th ACM SIGKDD international conference on Knowledge discovery and data mining, pages 1077–1086, 2014.
  * [34] Yang Zhang, Fuli Feng, Chenxu Wang, Xiangnan He, Meng Wang, Yan Li, and Yongdong Zhang.  How to retrain recommender system? a sequential meta-learning method.  In Proceedings of the 43rd International ACM SIGIR Conference on Research and Development in Information Retrieval, pages 1479–1488, 2020.
  * [35] Yujing Zhang, Zhangming Chan, Shuhao Xu, Weijie Bian, Shuguang Han, Hongbo Deng, and Bo Zheng.  Keep: An industrial pre-training framework for online recommendation via knowledge extraction and plugging.  In Proceedings of the 31st ACM International Conference on Information & Knowledge Management, pages 3684–3693, 2022.
  * [36] Kun Zhou, Hui Wang, Wayne Xin Zhao, Yutao Zhu, Sirui Wang, Fuzheng Zhang, Zhongyuan Wang, and Ji-Rong Wen.  S3-rec: Self-supervised learning for sequential recommendation with mutual information maximization.  In Proceedings of the 29th ACM international conference on information & knowledge management, pages 1893–1902, 2020.


[a]Jérémie Quarroz

# Non-factorisable contributions to $t$-channel single-top production at the
LHC and FCC

Christian Brønnum-Hansen  Chiara Signorile-Signorile  Chen-Yu Wang

###### Abstract

Single top quark is mainly produced through the $t$-channel W boson exchange
$q+b\rightarrow q^{\prime}+t$ at LHC. This process probes Wtb vertex directly
and can be used to measure the CKM matrix element $V_{tb}$ or to constrain the
bottom quark PDF. The non-factorisable contributions are the last missing
piece of the NNLO corrections. In these proceedings, we discuss in a first
part the ostensible importance of such corrections and the obtention of the
different relevant amplitudes. In the second part, recently published results
at the energy of the LHC are compared to new results for proton-proton
collisions at $100\,{\rm TeV}$, the energy of the FCC.

TTP22-050, P3H-22-080

##  1 Introduction

The physics of the top quark is unique. Indeed, the lifetime of the top quark,
the heaviest particle of the Standard Model, is so short that it does not
hadronise. As a consequence, non-perturbative effects do not need to be
considered to get accurate predictions for processes in which top quarks are
produced at colliders. In addition, since the top-quark coupling to the Higgs
boson is close to one, its study may provide important insights into the
electroweak symmetry breaking.

At the LHC, top quarks are mostly produced in $t\bar{t}$ pairs. Nevertheless,
single-top production also happens frequently. Indeed, the total cross-section
for single-top production is about four times smaller than the one for
$t\bar{t}$ pair production. The NLO QCD and NLO electroweak corrections to
single-top production with decay of the top quark are known since more than
ten years now [1, 2, 3, 4, 5, 6]. More recently the NNLO QCD corrections have
been computed in the factorisable approximation [7, 8, 9, 10]. The mixed QCD-
electroweak corrections and soft-gluon resummation have also been studied [11,
12].

The single-top quark production is interesting as it allows a direct
determination of the CKM matrix element $V_{bt}$ [13, 14, 15, 16, 17]. This
leads to an indirect determination of the decay rate of the top quark as it
will mostly decay into a bottom quark by emitting a W boson. In addition, it
offers a way to constrain the bottom-quark PDF.

At hadron collider, single top quarks can be produced in three different ways
with rather different rates. Indeed, $70\%$ of the single-top quarks at the
LHC originate from the so-called $t$-channel production mode. It will be the
subject of our study.

In these proceedings, we discuss the non-factorisable corrections to the
$t$-channel single-top quark production. In the first part, we discuss the
ostensible importance of such corrections and the obtention of the different
relevant amplitudes. In the second part, recently published results at the
energy of the LHC are compared to new results for proton-proton collisions at
$100\,{\rm TeV}$, the energy of the FCC [18, 19].

##  2 Non-factorisable corrections

The Born amplitude for the $t$-channel single-top quark production is made of
two fermion lines connected by a $W$ boson. Non-factorisable corrections are
described by diagrams where gluons are exchanged between these two fermions
line, see Fig. 1. On the left, the gluons are emitted and absorbed by the same
quark line and therefore the diagram is part of the factorisable corrections.
On the right, a diagram contributing to the non-factorisable two-loop
amplitude is shown. The non-factorisable corrections are usually neglected
because they are colour-suppressed [7, 10]. It is easy to compute the color
factor of these two diagrams once they are projected on the Born amplitude.
The resulting color factors are $\frac{1}{4}(N_{c}^{2}-1)^{2}$ for the
factorisable diagram and $\frac{1}{4}(N_{c}^{2}-1)$ for the non-factorisable
one respectively. Therefore, the non-factorisable corrections are suppressed
by a factor $N_{c}^{2}-1=8$. In addition, one cannot rely on an estimate of
these corrections at NLO since the different interferences vanish at this
order because of colour conservation.

(a) Factorisable diagram contributing to the double-virtual amplitude.

(b) Non-factorisable diagram contributing to the double-virtual amplitude.

Figure 1: In the factorisable approximation, diagrams made of a cross-talk
between the two fermion lines are neglected since they are suppressed by a
factor $N_{c}^{2}-1=8$ at NNLO compared to diagrams where the corrections are
confined to the same quark line.

Nevertheless, there are several reasons to believe that the non-factorisable
should not be ignored. First of all, the NNLO QCD corrections to the
$t$-channel single-top production in the factorisable approximation have been
computed recently and appear to be small, namely $\mathcal{O}(1\%)$ [10]. In
addition, there could also be a $\pi^{2}$ enhancement due to the Glauber or
Coulomb phase. This is a virtual effect which is linked to the infinite
perturbative interaction range between the two quark lines exchanging gluons.
In principle, it does not require a scattering to occur. This effect has been
studied in the context of two electrons exchanging an arbitrary number of
photons in the limit where the centre-of-mass energy is large compared to the
momentum transfer [20]. We note that the presence of the $\pi^{2}$ enhancement
has been explicitly shown for the non-factorisable NNLO QCD corrections to the
vector boson fusion in the eikonal approximation [21]. If this enhancement
factor $\pi^{2}\sim 10$ exist for the single-top production, it could clearly
compensate for the colour suppression.

There is another consequence of this effect; namely that the role of the
virtual corrections is expected to be dominant in non-factorisable
corrections. This can be shown by expanding the integrated cross section in
the transverse momentum of the top quark $p_{\perp}^{t}$

$\sigma=\sigma_{0}+\frac{p_{\perp}^{t}}{\sqrt{s}}\>\sigma_{1}+\mathcal{O}\left(\left(p_{\perp}^{t}/\sqrt{s}\right)^{2}\right)\,.$
(1)

This expansion is valid for the $t$-channel single-top production as the
typical value of the transverse momentum of the top-quark is
$p_{\perp}^{t}\sim 40\>{\rm GeV}$ and $\sqrt{s}\sim 300\>{\rm GeV}$. As the
$\pi^{2}$ enhancement occurs when $p_{\perp}^{t}\rightarrow 0$, it will affect
the term $\sigma_{0}$ in Eq. (1). However, in this limit, the real emission
vanishes up to corrections due to the mass of the top quark. Indeed, to
contribute to $\sigma_{0}$, the emission has to be collinear or soft. Since
non-factorisable amplitude are free of collinear divergences, we only need to
consider emission of a soft gluon. In this limit, any amplitude factorises.
For instance, the five-point amplitude can be written as

$\displaystyle\begin{split}\lim_{E_{5}\rightarrow
0}\mathcal{A}^{(0)}(1_{q},2_{b},3_{q}^{\prime},4_{t},5_{g})=&-g_{s}^{2}C_{F}(\frac{p_{1}^{\mu}}{k\cdot
p_{1}}-\frac{p_{3}^{\mu}}{p_{5}\cdot p_{3}}+\frac{p_{2}^{\mu}}{p_{5}\cdot
p_{2}}-\frac{p_{4}^{\mu}}{p_{5}\cdot p_{4}})\epsilon^{*}_{\mu}(p_{5})\\\
&\times\mathcal{A}^{(0)}(1_{q},2_{b},3_{q}^{\prime},4_{t})\,,\end{split}$ (2)

where $\mathcal{A}^{(0)}(1_{q},2_{b},3_{q}^{\prime},4_{t})$ is the Born
amplitude and $\epsilon^{*}_{\mu}(p_{5})$ is the polarisation vector for the
emitted gluon $5_{g}$. We see that, in the limit $p^{t}_{\perp}\rightarrow 0$
and for massless momenta, the soft function vanishes as
$p_{1}^{\mu}=p_{3}^{\mu}$ and $p_{2}^{\mu}=p_{4}^{\mu}$. In the case of
single-top production, $p_{4}^{\mu}$ is massive and this argument holds only
when the centre-of-mass energy is much larger than the top mass.

##  3 Real-virtual amplitude

The calculation of the real-virtual amplitude turns out to be non-trivial due
to the presence of multiple mass scales. We generate the 24 diagrams with
QGRAF and we treat them with FORM. We work in the ’t Hooft-Veltman scheme
where the external momenta live in four dimensions and the internal ones live
in $d=4-2\epsilon$. The $\epsilon-$dependence of the different spinor
structures can be extracted by splitting the gamma matrices into four-
dimensional and $\epsilon$-dimensional parts

$\gamma^{\mu}=\gamma^{\bar{\mu}}+\gamma^{\tilde{\mu}}\,,$ (3)

where

$\\{\gamma^{\bar{\mu}},\gamma_{5}\\}=0\qquad\qquad[\gamma^{\bar{\mu}},\gamma_{5}]=0\,.$
(4)

It is now clear that one needs at least two gamma matrices living in
$-2\epsilon$ space to produce an $\epsilon$ term. For instance, the following
spinor structure

$\bar{u}_{t}(p_{4})\gamma^{\mu}\gamma^{\nu}u_{b}(p_{2})=\bar{u}_{t}(p_{4})\gamma^{\bar{\mu}}\gamma^{\bar{\nu}}u_{b}(p_{2})+g^{\tilde{\mu}\tilde{\nu}}\bar{u}_{t}(p_{4})u_{b}(p_{2})\,,$
(5)

can be expressed as pure four dimensions spinor chains. We projected the two
matrices $\gamma^{\tilde{\mu}}\gamma^{\tilde{\nu}}$ on the metric tensor
$g^{\tilde{\mu}\tilde{\nu}}$ as it is the only object that can be build from
two indices living in $-2\epsilon$ dimension provided that all other momenta
are purely four-dimensional.

The treatment of $\gamma_{5}$ within ’t Hooft-Veltman scheme is generally
difficult. However, the case of the single-top production is convenient.
Indeed, the left-handed projector from the quark-quark-W boson vertex can be
moved to the external states from the beginning. Then, we decide to remove
these projectors and to work with helicity amplitudes in $d$ dimension.
Finally, we select only the left-handed external state.

The next step is to decompose the massive momentum $p_{4}$ into a reference
vector $r$ and a new massless momentum $4^{\flat}$

$P_{L}u_{t}(p_{4})=|4^{\flat}]+\frac{m_{t}}{\langle
4^{\flat}r\rangle}|r\rangle\qquad\qquad
P_{R}u_{t}(p_{4})=|4^{\flat}\rangle+\frac{m_{t}}{[4^{\flat}r]}|r]\,,$ (6)

where $P_{L}=\frac{1-\gamma_{5}}{2}$ stands for the left-handed projector and
$P_{R}=\frac{1+\gamma_{5}}{2}$ the right-handed one. The reference vector $r$
is an arbitrary massless momentum. It can be choosen conveniently to simplify
the expressions. After simplification, we are left with two spinor structures
per helicity configuration. There are four of them as both the emitted gluon
and the top quark can be left- or right-handed.

On the other hand, one needs to treat the form factors. The challenge comes
from the reduction of five-point rank-three integrals. As an example, we
consider a rank $r\leq 3$ tensor pentagon integral

$\int\frac{d^{d}k}{(2\pi)^{d}}\frac{k^{\mu_{1}}\dots
k^{\mu_{r}}}{\prod_{i=1}^{5}\left[(k+q_{i})^{2}-m_{i}^{2}\right]}\,,$ (7)

where all momenta are considered to be incoming, so that
$\sum_{i=1}^{5}p_{i}=0$, and the flowing momenta are defined as
$q_{i}=\sum_{j=1}^{i}p_{j}$. We introduce the _van Neerven-Vermaseren_ basis
$\\{v_{i}\\}_{i=1\dots 4}$ which spans the physical momentum space. We refer
the reader to the following reference for a detailed discussion [22]. The
basis vectors are defined to fulfill the following relation

$v_{i}\cdot p_{j}=\delta_{ij}\,.$ (8)

We emphasise that, since the five incoming momenta are not independent, the
dot product of the vector $v_{i}$ to the left-over momentum is fully-defined
by the four others. The loop momentum can be expressed in this new basis as

$k^{\mu}=\sum_{i=1}^{4}(k\cdot p_{i})v_{i}^{\mu}+k^{\tilde{\mu}}\,,$ (9)

where $k^{\tilde{\mu}}$ stands for the $-2\epsilon$-dimension part of the loop
momentum $k^{\mu}$. Since five-point rank-three integrals do not contain
rational terms, the former can be dropped. The factor $k\cdot p_{i}$ can be
expressed as

$k\cdot
p_{i}=\frac{1}{2}\left[(k+q_{i})^{2}-m_{i}^{2}\right]-\frac{1}{2}\left[(k+q_{i-1})^{2}-m_{i-1}^{2}\right]+\frac{1}{2}\left[m_{i}^{2}-m_{i-1}^{2}-p_{i}^{2}-2p_{i}\cdot
q_{i-1}\right]\,.$ (10)

The first two terms in Eq. (10) lead to two rank $r-1$ boxes which we can
treat with a standard Passarino-Veltman reduction. The last term corresponds
to a pentagon of rank $r-1$; to deal with it, one can repeat this procedure.
Once we reached the point of scalar pentagon, they can be expressed as a
combination of five scalar box integrals, one for each pinched propagator of
the pentagon integrals up to $\mathcal{O}(\epsilon)$ terms [23].

The real-virtual amplitude is now expressed in terms of 109 scalar box,
triangle and bubble integrals. To improve the numerical stability of the
amplitude, we want to reduce the size of the integral coefficients. The most
complicated ones come with the box integrals. We decided to switch to a basis
of finite box integrals in order to set the dimensional regulator present in
their coefficient to zero. As an example, we consider the following box
integral

$I_{4,1}=\int\frac{\mathrm{d}^{d}k}{(2\pi)^{d}}\,\frac{1}{k^{2}(k-p_{1})^{2}(k-p_{1}-p_{2})^{2}(k-p_{1}-p_{2}+p_{5})^{2}}\;.$
(11)

This integral is divergent as any of the propagator goes on-shell. It can be
regulated through a numerator insertion which vanishes in these same limits

$\displaystyle\mathrm{tr}\left((-\not{p}_{1})(\not{k}-\not{p}_{1})(\not{k}-\not{p}_{1}-\not{p}_{2})(\not{p}_{5})\right)=\,$
$\displaystyle-s_{12}\,(s_{12}+s_{15}-s_{34})+(s_{12}+s_{15}-s_{34})\,k^{2}$
$\displaystyle-(s_{12}-s_{34})\,(k-p_{1})^{2}+(s_{12}+s_{15})\,(k-p_{1}-p_{2})^{2}$
$\displaystyle-s_{12}\,(k-p_{1}-p_{2}+p_{5})^{2}\,.$ (12)

After the trace is calculated, we observe that four terms on the right-hand
side of Eq. (12) correspond to the four different propagator of the considered
box integrals Eq. (11). These terms will lead to triangle integrals. The last
term is independent of $k$. This factor multiplies the initial divergent box
integral. Using this relation, we can therefore express the divergent box
integral as a sum of one finite box integral that involves the trace that
appear on the left-hand side of Eq. (12) and four divergent triangle
integrals. This redefinition can be done for any of the divergent boxe
integrals present in the amplitude.

As all the divergences are now in the triangle integrals and since the pole
structure for non-factorisable corrections at NNLO is very simple, we observe
that the coefficients of the triangle integrals either become independent of
the dimensional regulator $\epsilon$ or simply vanish.

##  4 Double-virtual amplitude

The double-virtual amplitude is obtained numerically through the auxiliary
mass flow method. We refer the reader to Ref. [24] for a detailed explanation.
The analytic reduction of the two-loop amplitude keeping the full dependence
on the two Mandelstam variables $s$ and $t$ and the two masses $m_{W}$ and
$m_{t}$ is possible within 4 days on 20 cores. We end up with 428 masters
integrals split into 18 families. The evaluation of the master integrals
through the auxiliary mass flow method can be performed to any desired
accuracy. We were able to evaluate the 428 master integrals at a typical phase
space point to 20 digits accuracy within 30 minutes on a single core.

| $\epsilon^{-2}$ | $\epsilon^{-1}$
---|---|---
$\langle\mathcal{A}^{(0)}|\mathcal{A}_{\rm nf}^{(2)}\rangle$ | $-229.094040865466{\color[rgb]{1,0,0}\definecolor[named]{pgfstrokecolor}{rgb}{1,0,0}0}-8.978163333241{\color[rgb]{1,0,0}\definecolor[named]{pgfstrokecolor}{rgb}{1,0,0}640}i$ | $-301.18029889447{\color[rgb]{1,0,0}\definecolor[named]{pgfstrokecolor}{rgb}{1,0,0}64}-264.17735965295{\color[rgb]{1,0,0}\definecolor[named]{pgfstrokecolor}{rgb}{1,0,0}05}i$
IR poles | $-229.0940408654665-8.978163333241973i$ | $-301.1802988944791-264.1773596529535i$
Table 1: Comparison of the poles of the two-loop amplitude with the ones
predicited using Catani’s operator.

In table Tab.1, we present the poles obtained for the two-loop amplitude
compared to the one predicted by Catani’s operator [25]. We observe that we
have about 15 digits match at $\epsilon^{-2}$ and 14 digits match at
$\epsilon^{-1}$. We expect therefore that the finite part is correct to about
13 digits, as we lose one digit per $\epsilon$ order.

To evaluate the cross section, we prepare a Vegas grid starting with the Born
squared cross section. We extract then 10 sets of $10^{4}$ points from this
grid and we evaluate the amplitude for each of these points. Since we have 10
independent sets, we can use the spread of the NNLO double-virtual correction
to estimate the accuracy of our result, namely $\mathcal{O}(2\%)$.

##  5 Results at the LHC

We consider proton-proton collisions at 13 TeV. We use the PDF set CT14. The
leading-order cross section is computed with then leading-order PDFs whereas
the NNLO cross section is computed with NNLO PDFs. We present the integrated
cross section at a fixed factorisation scale $\mu_{F}=m_{t}$

$\frac{\sigma_{pp\rightarrow X+t}}{1\,{\rm
pb}}=117.96+0.26\left(\frac{\alpha_{s}(\mu_{R})}{0.108}\right)^{2}\,.$ (13)

At $\mu_{R}=m_{t}$, the non-factorisable corrections to the leading-order
cross section amount to $0.2\%$. The non-factorisable cross section displays a
trivial dependence on the renormalisation scale. As the non-factorisable
corrections appear for the first time at NNLO, they are independent of LO, NLO
and NNLO _factorisable_ contributions. As a consequence, at this order, there
is no indication of a good scale choice. Analogously to deep inelastic
scattering, the renormalisation scale may be chosen to be the typical
transverse momentum of the top quark, $\mu_{R}=40\,{\rm GeV}$. In this case,
the non-factorisable corrections become close to $0.35\%$. In comparison, the
NNLO factorisable corrections to the NLO cross section are about $0.7\%$ [10].

The impact of the non-factorisable corrections on the top-quark transverse
momentum distribution is presented on the left pane in Fig. 2. The blue solid
line represents the Born cross section. The red dashed line corresponds to the
non-factorisable corrections at $\mu\equiv\mu_{F}=\mu_{R}=m_{t}$. The
variation of $\mu$ by a factor 2 is depicted by the red region around the red
dashed line. A green dotted line shows NNLO corrections computed with the
scale $\mu=40\;{\rm GeV}$. We observe that the non-factorisable corrections
exhibit a significant dependendence on the transverse momentum of the top
quark.

(a) 13 TeV

(b) 100 TeV

Figure 2: The top quark transverse momentum distribution. The blue solid line
represents the Born cross section. The red dashed line corresponds to the non-
factorisable corrections at $\mu\equiv\mu_{F}=\mu_{R}=m_{t}$. The variation of
$\mu$ by a factor 2 is depicted by the red region around the red dashed line.
We also present with a green dotted line at the scale $\mu=40\;{\rm GeV}$.

Another interesting feature is that the non-factorisable corrections vanish
around at $50\,{\rm GeV}$ for any scale value. On the other hand, the
factorisable corrections vanish at $30\;{\rm GeV}$ [10]. The consequence is
that there is a part of the phase space at low $p_{t}^{\perp}$, close to the
peak of the $p_{t}^{\perp}$ distribution, where the non-factorisable
corrections are _dominant_ compared to factorisable corrections.

##  6 Non-factorisable corrections at the FCC

In this section, we present results for proton-proton collision at $100\>{\rm
TeV}$. We use the same PDF set as in the previous section. The integrated
cross-section reads

$\frac{\sigma_{pp\rightarrow X+t}}{1\,{\rm
pb}}=2367.0+3.8\left(\frac{\alpha_{s}(\mu_{R})}{0.108}\right)^{2}\,.$ (14)

At $100\,{\rm TeV}$, we observe that at $\mu_{R}=m_{t}$, the NNLO corrections
amount to $0.16\%$. At $\mu_{R}=40\,{\rm GeV}$, these corrections become
$0.25\%$.

|  | $\mu_{R}=m_{t}$ | $\mu_{R}=40\>{\rm GeV}$
---|---|---|---
$p_{\perp}^{t,\text{cut}}$ |  $\sigma_{\text{LO}}$ (pb) |  $\sigma^{{\rm nf}}_{\text{NNLO}}$ (pb) | $\delta_{\text{NNLO}}\>[\%]$ |  $\sigma^{{\rm nf}}_{\text{NNLO}}$ (pb) | $\delta_{\text{NNLO}}\>[\%]$
0 GeV | $2367.02$ | $3.79_{\;\;\>0.84}^{-0.63}$ | $0.16_{\;\;\>0.04}^{-0.03}$ | $5.95$ | $0.25$
20 GeV | $2317.03$ | $3.89_{\;\;\>0.86}^{-0.64}$ | $0.17_{\;\;\>0.04}^{-0.03}$ | $6.11$ | $0.26$
40 GeV | $2216.61$ | $4.14_{\;\;\>0.92}^{-0.69}$ | $0.19_{\;\;\>0.04}^{-0.03}$ | $6.50$ | $0.29$
60 GeV | $2121.88$ | $4.28_{\;\;\>0.95}^{-0.71}$ | $0.20_{\;\;\>0.04}^{-0.03}$ | $6.71$ | $0.32$
Table 2: Born and NNLO non-factorisable cross-sections computed with a cut on
the transverse momentum of the top quark at fixed factorisation scale
$\mu_{F}=m_{t}$ at $100\,{\rm TeV}$. The first column corresponds to the value
of the cut. The second column gives the leading-order cross section. The third
and the fourth column present the NNLO non-factorisable corrections at scale
$\mu_{R}=m_{t}$ with its variation by a factor 2. The two last column
describes the same quantities at renormalisation scale $\mu_{R}=40\,{\rm
GeV}$.

In the right pane of Fig. 2, the distribution of the top-quark transverse
momentum is plotted. We emphasise that we change unit from ${\rm fb}$ at
$13\,{\rm TeV}$ to ${\rm pb}$ on this figure at $100\,{\rm TeV}$. The shape of
the distribution is similar. The peak of the distribution is still centred
around $p_{t}^{\perp}=40\,{\rm GeV}$. The NNLO non-factorisable corrections
change sign around $70\,{\rm GeV}$ whereas at $13\,{\rm TeV}$, it was at
$50\,{\rm GeV}$.

In Tab. 2, we present the leading-order cross section and the NNLO non-
factorisable cross section subject to different cuts on the transverse
momentum of the top quark. The factorisation scale is fixed to
$\mu_{F}=m_{t}$. In the first two columns, we present the leading-order cross
section at four different cut values. The third and the fourth column
corresponds to $\mu_{R}=m_{t}$. We vary the renormalisation scale by a factor
2. The last two columns corresponds to $\mu_{R}=40\,{\rm GeV}$.

(a) Distribution of the top-quark rapidity.

(b) Distribution of the leading-jet rapidity.

Figure 3: Distribution of the rapidity of the top quark (left pane) and of the
leading jet (right pane) at 100 TeV. The red dashed line corresponds to the
scale $\mu\equiv\mu_{F}=\mu_{R}=m_{t}$. The green dotted line to $\mu=40\,{\rm
GeV}$.

In Fig. 3, we present the rapidity of the top quark on the left pane and the
rapidity of the leading jet on the right pane. The corrections are nearly
constant around $0.2\%$. On the right pane, one can read the distribution of
the leading-jet rapidity. The corrections to this observable turn out to be
also almost flat and amount to $0.35\%$. The behaviour of these two rapidity
distributions is different from the ones at $13\,{\rm TeV}$. Indeed, at this
centre-of-mass energy, the corrections are drastically decreasing around $y=2$
and even negative for rapidities above $y=3.5$ [26].

(a) Leading-jet transverse momentum.

(b) Sum of transverse momenta $H$.

Figure 4: Distribution of the transverse momentum of the leading jet (left
pane) and of the sum of the transverse momenta defined in Eq. (15) (right
pane) at 100 TeV. The red dashed line corresponds to the scale
$\mu\equiv\mu_{F}=\mu_{R}=m_{t}$. The green dotted line to $\mu=40\,{\rm
GeV}$.

In Fig. 4, we present the distribution of the leading jet transverse momentum
on the left pane and the sum of the transverse momenta

$H=\sum_{i=1}^{n_{\text{jet}}}p_{\perp}^{\text{jet},i}\,,$ (15)

on the right pane. The shape of the NNLO non-factorisable corrections on these
two observables is similar to the ones at $13\,{\rm TeV}$. It is interesting
to note that the hierarchy between the different scales is inverted compared
to $13\,{\rm TeV}$ results. Indeed, in Fig. 4, the corrections at
$\mu=40\,{\rm GeV}$ represented by a green dotted line are smaller than the
ones at $\mu=m_{t}/2$, $\mu=m_{t}$ and $\mu=2m_{t}$ in red dashed line.

##  7 Conclusions

We report on the computation of _non-factorisable_ corrections to the
$t$-channel single-top production. The double-virtual amplitude has been
numerically evaluated using the auxiliary mass flow method. This procedure is
sufficiently robust to produce phenomenologically relevant results. Due to
multiple mass scales appearing in the one-loop five-point amplitude, its
reduction to master integrals and its stable and efficient numerical
evaluation turn out to be non-trivial. Since the NNLO factorisable corrections
to the NLO cross section are small, the non-factorisable corrections are found
to be quite comparable. We provide new results for non-factorisable
corrections for proton-proton collisions at $100\,{\rm TeV}$. The integrated
cross-section is changed by few permilles and the corrections to the different
distributions have the tendency to be much flatter than at $13\,{\rm TeV}$. It
is hard to estimate how the factorisable corrections will be affected by such
an increase in the beam energy through an heuristic argument. Such results
would be desirable to understand the enhancement of the non-factorisable
corrections when the centre-of-mass energy becomes large.

## References

  * [1] G. Bordes and B. van Eijk, _Calculating QCD corrections to single top production in hadronic interactions_ , _Nucl. Phys. B_ 435 (1995) 23–58.
  * [2] J.M. Campbell, R.K. Ellis and F. Tramontano, _Single top production and decay at next-to-leading order_ , _Phys. Rev. D_ 70 (2004) 094012 [hep-ph/0408158].
  * [3] Q.-H. Cao and C.P. Yuan, _Single top quark production and decay at next-to-leading order in hadron collision_ , _Phys. Rev. D_ 71 (2005) 054022 [hep-ph/0408180].
  * [4] Q.-H. Cao, R. Schwienhorst, J.A. Benitez, R. Brock and C.P. Yuan, _Next-to-leading order corrections to single top quark production and decay at the Tevatron: 2. $t^{-}$ channel process_, _Phys. Rev. D_ 72 (2005) 094027 [hep-ph/0504230].
  * [5] B.W. Harris, E. Laenen, L. Phaf, Z. Sullivan and S. Weinzierl, _The Fully Differential Single Top Quark Cross-Section in Next to Leading Order QCD_ , _Phys. Rev. D_ 66 (2002) 054024 [hep-ph/0207055].
  * [6] R. Schwienhorst, C.P. Yuan, C. Mueller and Q.-H. Cao, _Single top quark production and decay in the $t$-channel at next-to-leading order at the LHC_, _Phys. Rev. D_ 83 (2011) 034019 [1012.5132].
  * [7] M. Brucherseifer, F. Caola and K. Melnikov, _On the NNLO QCD corrections to single-top production at the LHC_ , _Phys. Lett. B_ 736 (2014) 58–63 [1404.7116].
  * [8] E.L. Berger, J. Gao, C.P. Yuan and H.X. Zhu, _NNLO QCD Corrections to t-channel Single Top-Quark Production and Decay_ , _Phys. Rev. D_ 94 (2016) 071501 [1606.08463].
  * [9] E.L. Berger, J. Gao and H.X. Zhu, _Differential Distributions for t-channel Single Top-Quark Production and Decay at Next-to-Next-to-Leading Order in QCD_ , _JHEP_ 11 (2017) 158 [1708.09405].
  * [10] J. Campbell, T. Neumann and Z. Sullivan, _Single-top-quark production in the $t$-channel at NNLO_, _JHEP_ 02 (2021) 040 [2012.01574].
  * [11] Q.-H. Cao, P. Sun, B. Yan, C.P. Yuan and F. Yuan, _Transverse Momentum Resummation for $t$-channel single top quark production at the LHC_, _Phys. Rev. D_ 98 (2018) 054032 [1801.09656].
  * [12] R. Frederix, D. Pagani and I. Tsinikos, _Precise predictions for single-top production: the impact of EW corrections and QCD shower on the $t$-channel signature_, _JHEP_ 09 (2019) 122 [1907.12586].
  * [13] S.D. Rindani and P. Sharma, _Probing anomalous tbW couplings in single-top production using top polarization at the Large Hadron Collider_ , _JHEP_ 11 (2011) 082 [1107.2597].
  * [14] Q.-H. Cao, B. Yan, J.-H. Yu and C. Zhang, _A General Analysis of Wtb anomalous Couplings_ , _Chin. Phys. C_ 41 (2017) 063101 [1504.03785].
  * [15] G.A. González-Sprinberg and J. Vidal, _The top quark right coupling in the tbW-vertex_ , _Eur. Phys. J. C_ 75 (2015) 615 [1510.02153].
  * [16] CDF, D0 collaboration, _Tevatron Combination of Single-Top-Quark Cross Sections and Determination of the Magnitude of the Cabibbo-Kobayashi-Maskawa Matrix Element $\bf V_{tb}$_, _Phys. Rev. Lett._ 115 (2015) 152003 [1503.05027].
  * [17] CMS collaboration, _Measurement of the t-channel single-top-quark production cross section and of the $\mid V_{tb}\mid$ CKM matrix element in pp collisions at $\sqrt{s}$= 8 TeV_, _JHEP_ 06 (2014) 090 [1403.7366].
  * [18] FCC collaboration, _FCC-hh: The Hadron Collider: Future Circular Collider Conceptual Design Report Volume 3_ , _Eur. Phys. J. ST_ 228 (2019) 755.
  * [19] M. Mangano, _Physics at the FCC-hh, a 100 TeV pp collider_ , 1710.06353.
  * [20] S.-J. Chang and S.-K. Ma, _Multiphoton exchange amplitudes at infinite energy_ , _Phys. Rev._ 188 (1969) 2385.
  * [21] T. Liu, K. Melnikov and A.A. Penin, _Nonfactorizable QCD Effects in Higgs Boson Production via Vector Boson Fusion_ , _Phys. Rev. Lett._ 123 (2019) 122002 [1906.10899].
  * [22] R.K. Ellis, Z. Kunszt, K. Melnikov and G. Zanderighi, _One-loop calculations in quantum field theory: from Feynman diagrams to unitarity cuts_ , _Phys. Rept._ 518 (2012) 141 [1105.4319].
  * [23] Z. Bern, L.J. Dixon and D.A. Kosower, _Dimensionally regulated pentagon integrals_ , _Nucl. Phys. B_ 412 (1994) 751 [hep-ph/9306240].
  * [24] C. Brønnum-Hansen, K. Melnikov, J. Quarroz and C.-Y. Wang, _On non-factorisable contributions to t-channel single-top production_ , _JHEP_ 11 (2021) 130 [2108.09222].
  * [25] S. Catani, _The Singular behavior of QCD amplitudes at two loop order_ , _Phys. Lett. B_ 427 (1998) 161–171 [hep-ph/9802439].
  * [26] C. Brønnum-Hansen, K. Melnikov, J. Quarroz, C. Signorile-Signorile and C.-Y. Wang, _Non-factorisable contribution to t-channel single-top production_ , _JHEP_ 06 (2022) 061 [2204.05770].