# VidTr: Video Transformer Without Convolutions

Yanyi Zhang 1,2 , Xinyu Li 1111we measure latency of X3D using the authors’
code and fast depth convolution patch:
https://github.com/facebookresearch/SlowFast/blob/master/projects/x3d/README.md,
which only has models for X3D-M and X3D-L and not the XL and XXL variants,
Chunhui Liu 1, Bing Shuai 1, Yi Zhu 1,
Biagio Brattoli 1, Hao Chen 1, Ivan Marsic 2 and Joseph Tighe 1
1 Amazon Web Service; 2 Rutgers University
<EMAIL_ADDRESS><EMAIL_ADDRESS>Equally Contributed.

###### Abstract

We introduce Video Transformer (VidTr) with separable-attention for video
classification. Comparing with commonly used 3D networks, VidTr is able to
aggregate spatio-temporal information via stacked attentions and provide
better performance with higher efficiency. We first introduce the vanilla
video transformer and show that transformer module is able to perform spatio-
temporal modeling from raw pixels, but with heavy memory usage. We then
present VidTr which reduces the memory cost by 3.3$\times$ while keeping the
same performance. To further optimize the model, we propose the standard
deviation based topK pooling for attention ($pool_{topK\\_std}$), which
reduces the computation by dropping non-informative features along temporal
dimension. VidTr achieves state-of-the-art performance on five commonly used
datasets with lower computational requirement, showing both the efficiency and
effectiveness of our design. Finally, error analysis and visualization show
that VidTr is especially good at predicting actions that require long-term
temporal reasoning.

##  1 Introduction

We introduce Video Transformer (VidTr) with separable-attention, one of the
first transformer-based video action classification architecture that performs
global spatio-temporal feature aggregation. Convolution-based architectures
have dominated the video classification literature in recent years [55, 19,
32], and although successful, the convolution-based approaches have two
drawbacks: 1. they have limited receptive field on each layer and 2.
information is slowly aggregated through stacked convolution layers, which is
inefficient and might be ineffective [55, 31]. Attention is a potential
candidate to overcome these limitations as it has a large receptive field
which can be leveraged for spatio-temporal modeling. Previous works use
attention to modeling long-range spatio-temporal features in videos but still
rely on convoluational backbones [55, 31]. Inspired by recent successful
applications of transformers on NLP [52, 12] and computer vision [14, 47], we
propose a transformer-based video network that directly applies attentions on
raw video pixels for video classification, aiming at higher efficiency and
better performance.

We first introduce a vanilla video transformer that directly learns spatio-
temporal features from raw-pixel inputs via vision transformer [14], showing
that it is possible to perform pixel-level spatio-temporal modeling. However,
as discussed in [56], the transformer has $\mathbb{O}(n^{2})$ complexity with
respect to the sequence length. The vanilla video transformer is memory
consuming, as training on a 16-frame clip ($224\times 224$) with only batch
size of 1 requires more than 16GB GPU memory, which makes it infeasible on
most commercial devices. Inspired by the R(2+1)D convolution that breaks down
3D convolution kernel to a spatial kernel and a temproal kernel [50], we
further introduce our separable-attention, which performs spatial and temporal
attention separately. This reduces the memory consumption by 3.3$\times$ with
no drop in accuracy. We can further reduce the memory and computational
requirements of our system by exploiting the fact that a large portion of many
videos have redundant information temporally. This notion has been explored in
the context of convolutional networks to reduce computation previously [32].
We build on this intuition and propose a standard deviation based topK pooling
operation ($topK\\_std$ pooling), which reduces the sequence length and
encourages the transformer network to focus on representative frames.

We evaluated our VidTr on 6 most commonly used datasets, including Kinetics
400/700, Charades, Something-something V2, UCF-101 and HMDB-51. Our model
achieved state-of-the-art (SOTA) or comparable performance on five datasets
with lower computational requirements and latency compared to previous SOTA
approaches. Our error analysis and ablation experiments show that the VidTr
works significantly better than I3D on activities that requires longer
temporal reasoning (e.g. making a cake vs. eating a cake), which aligns well
with our intuition. This also inspires us to ensemble the VidTr with the I3D
convolutional network as features from global and local modeling methods
should be complementary. We show that simply combining the VidTr with a I3D50
model (8 frames input) via ensemble can lead to roughly a 2% performance
improvement on Kinetics 400. We further illustrate how and why the VidTr works
by visualizing the separable-attention using attention rollout [1], and show
that the spatial-attention is able to focus on informative patches while
temporal attention is able to reduce the duplicated/non-informative temporal
instances. Our contributions are:

  1. 1.

Video transformer:  We propose to efficiently and effectively aggregate
spatio-temporal information with stacked attentions as opposed to convolution
based approaches. We introduce vanilla video transformer as proof of concept
with SOTA comparable performance on video classification.

  2. 2.

VidTr:  We introduce VidTr and its permutations, including the VidTr with SOTA
performance and the compact-VidTr with significantly reduced computational
costs using the proposed standard deviation based pooling method.

  3. 3.

Results and model weights:  We provide detailed results and analysis on 6
commonly used datasets which can be used as reference for future research. Our
pre-trained model can be used for many down-streaming tasks.

##  2 Related Work

###  2.1 Action Classification

The early research on video based action recognition relies on 2D convolutions
[28]. The LSTM [25] was later proposed to model the image feature based on
ConvNet features [63, 51, 30]. However, the combination of ConvNet and LSTM
did not lead to significantly better performance. Instead of relying on RNNs,
the segment based method TSN [54] and its permutations [22, 35, 64] were
proposed with good performance.

Although 2D network was proved successful, the spatio-temporal modeling was
still separated. Using 3D convolution for spatio-temporal modeling was
initially proposed in [26] and further extended to the C3D network [48].
However, training 3D convnet from scratch was hard, initializing the 3D
convnet weights by inflate from 2D networks was initially proposed in I3D [7]
and soon proved applicable with different type of 2D network [24, 10, 58]. The
I3D was used as backbone for many following work including two-stream network
[55, 19], the networks with focus on temporal modeling [32, 31, 59], and the
3D networks with refined 3D convolution kernels [27, 33, 39, 44].

The 3D networks are proved effective but often not efficient, the 3D networks
with better performance often requires larger kernels or deeper structures.
The recent research demonstrates that depth convolution significantly reduce
the computation [49], but depth convolution also increase the network
inference latency. TSM [37] and TAM [17] proposed a more efficient backbone
for temporal modeling, however, such design couldn’t achieve SOTA performance
on Kinetics dataset. The neural architecture search was proposed for action
recognition [18, 43] recently with competitive performance, however, the high
latency and limited generalizability remain to be improved.

The previous methods heavily rely on convolution to aggregate features spatio-
temporally, which is not efficient. A few previous work tried to perform
global spatio-temporal modeling [55, 31] but still limited by the convolution
backbone. The proposed VidTr is fundamentally different from previous works
based on convolutions, the VidTr doesn’t require heavily stacked convolutions
[59] for feature aggregation but efficiently learn feature globally via
attention from first layer. Besides, the VidTr don’t rely on sliding
convolutions and depth convolutions, which runs at less FLOPs and lower
latency compared with 3D convolutions [59, 18].

###  2.2 Vision Transformer

The transformers [52] was previously proposed for NLP tasks [13] and recently
adopted for computer vision tasks. The transformers were roughly used in three
different ways in previous works: 1.To bridge the gap between different
modalities, e.g. video captioning [65], video retrieval [20] and dialog system
[36]. 2\. To aggregate convolutional features for down-streaming tasks, e.g.
object detection [11, 5], pose estimation [61], semantic segmentation [15] and
action recognition [21]. 3\. To perform feature learning on raw pixels, e.g.
most recently image classification [14, 47].

Action recognition with self-attention on convolution features [21] is proved
successful, however, convolution also generates local feature and gives
redundant computations. Different from [21] and inspired by very recent work
on applying transformer on raw pixels [14, 47], we pioneer the work on
aggregating spatio-temporal feature from raw videos without relying on
convolution features. Different from very recent work [41] that extract
spatial feature with vision transformer on every video frames and then
aggregate feature with attention, our proposed method jointly learns spatio-
temporal feature with lower computational cost and higher performance. Our
work differs from the concurrent work [4], we present a split attention with
better performance without requiring larger video resolution nor extra long
clip length. Some more recent work [4, 16, 4, 42, 2, 40] further studied the
multi-scale and different attention factorization methods.

##  3 Video Transformer

We introduce the Video Transformer starting with the vanilla video transformer
(section 3.1) which illustrates our idea of video action recognition without
convolutions. We then present VidTr by first introducing separable-attention
(section 3.2), and then the attention pooling to drop non-representative
information temporally (section 3.2).

Figure 1:  Spatio-temporal separable-attention video transformer (VidTr). The
model takes pixels patches as input and learns the spatial temporal feature
via proposed separable-attention. The green shaded block denotes the down-
sample module which can be inserted into VidTr for higher efficiency. $\tau$
denotes the temporal dimension after downsampling.

###  3.1 Vanilla Video Transformer

Following previous efforts in NLP [13] and image classification [14], we adopt
the transformer [52] encoder structure for action recognition that operates on
raw pixels. Given a video clip $V\in\mathbb{R}^{C\times T\times W\times H}$,
where $T$ denotes the clip length, $W$ and $H$ denote the video frame width
and height, and $C$ denotes the number of channel, we first convert $V$ to a
sequence of $s\times s$ spatial patches, and apply a linear embedding to each
patch, namely $S\in\mathbb{R}^{T\frac{H}{s}\frac{W}{s}\times
C^{{}^{\prime}}}$, where $C^{{}^{\prime}}$ is the channel dimension after the
linear embedding. We add a 1D learnable positional embedding [13, 14] to $S$
and following previous work [13, 14], append a class token as well, whose
purpose is to aggregate features from the whole sequence for classification.
This results in $S^{\prime}\in\mathbb{R}^{(\frac{TWH}{s^{2}}+1)\times
C^{\prime}}$, where $S^{{}^{\prime}}_{0}\in\mathbb{R}^{1\times C^{\prime}}$ is
the attached class token. $S^{\prime}$ is feed into our transformer encoder
structure detailed next.

As Figure 1 middle shows, we expand the previous successful ViT transformer
architecture for 3D feature learning. Specifically, we stack 12 encoder
layers, with each encoder layer consisting of an 8-head self-attention layer
and two dense layers with 768 and 3072 hidden units. Different from
transformers for 2D images, each attention layer learns a spatio-temporal
affinity map
$Attn\in\mathbb{R}^{(\frac{TWH}{s^{2}}+1)\times(\frac{TWH}{s^{2}}+1)}$.

###  3.2 VidTr

In Table 2 we show that this simple formulation is capable of learning 3D
motion features on a sequence of local patches. However, as explained in [3],
the affinity attention matrix
$Attn\in\mathbb{R}^{(\frac{TWH}{s^{2}}+1)\times(\frac{TWH}{s^{2}}+1)}$ needs
to be stored in memory for back propagating, and thus the memory consumption
is quadratically related to the sequence length. We can see that the vanilla
video transformer increases memory usage for the affinity map from
$\mathbb{O}(W^{2}H^{2})$ to $\mathbb{O}(T^{2}W^{2}H^{2})$, leading to
$T^{2}\times$ memory usage for training, which makes it impractical on most
available GPU devices. We now address this inefficiency with a separable
attention architecture.

####  3.2.1 Separable-Attention

To address these memory constraints, we introduce a multi-head separable-
attention (MSA) by decoupling the 3D self-attention to a spatial attention
$\operatorname{MSA}_{s}$ and a temporal attention $\operatorname{MSA}_{t}$
(Figure 1):

$\operatorname{MSA}(S)=\operatorname{MSA}_{s}(\operatorname{MSA}_{t}(S))$ (1)

Different from the vanilla video transformer that applies 1D sequential
modeling on $S$, we decouple $S$ to a 2D sequence
$\hat{S}\in\mathbb{R}^{(T+1)\times(\frac{WH}{s^{2}}+1)\times C^{\prime}}$ with
positional embedding and two types of class tokens that append additional
tokens along the spatial and temporal dimensions. Here, the spatial class
tokens gather information from spatial patches in a single frame using spatial
attention, and the temporal class tokens gather information from patches
across frames (at same location) using temporal attention. Then the
intersection of the spatial and temporal class tokens $\hat{S}^{(0,0,:)}$ is
used for the final classification. To decouple 1D self-attention functions on
2D sequential features $\hat{S}$, we first operate on each spatial location
($i$) independently, applying temporal attention as:

$\displaystyle\hat{S}_{t}^{(:,i,:)}$
$\displaystyle=\operatorname{MSA_{t}}(k=q=v=\hat{S}^{(:,i,:)})$ (2)
$\displaystyle=pool(Attn_{t})\cdot v_{t}$ (3)
$\displaystyle=pool(\operatorname{Softmax}(q_{t}\cdot k_{t}^{T}))\cdot v_{t}$
(4)

where $\hat{S}_{t}\in\mathbb{R}^{(\tau+1)\times(\frac{WH}{s^{2}}+1)\times C}$
is the output of $\operatorname{MSA_{t}}$, = $pool$ denotes the down-sampling
method to reduce temporal dimension (from $T$ to $\tau$, $\tau=T$ when no
down-sampling is performed) that will be described later, $q_{t}$, $k_{t}$,
and $v_{t}$ denote key, query, and value features after applying independent
linear functions (LN) on $\hat{S}$:

$\footnotesize
q_{t}=\operatorname{LN}_{q}(\hat{S}^{(:,i,:)});k_{t}=\operatorname{LN}_{k}(\hat{S}^{(:,i,:)});v_{t}=\operatorname{LN}_{v}(\hat{S}^{(:,i,:)})$
(5)

Moreover, $Attn_{t}\in\mathbb{R}^{(\tau+1)\times(T+1)}$ represent a temporal
attention obtained from matrix multiplication between $q_{t}$ and $k_{t}$.
Following $\operatorname{MSA_{s}}$, we apply a similar 1D sequential self-
attention $\operatorname{MSA_{s}}$ on spatial dimension:

$\displaystyle\hat{S}_{st}^{(i,:,:)}$
$\displaystyle=\operatorname{MSA_{s}}(k=q=v=\hat{S}_{t}^{(i,:,:)})$ (6)
$\displaystyle=Attn_{s}\cdot v_{s}$ (7)
$\displaystyle=\operatorname{Softmax}(q_{s}\cdot k_{s}^{T})\cdot v_{s}$ (8)

where $\hat{S}_{st}\in\mathbb{R}^{(\tau+1)\times(\frac{WH}{s^{2}}+1)\times C}$
is the output of $\operatorname{MSA_{s}}$, $q_{s}$, $k_{s}$, and $v_{s}$
denotes key, query, and value features after applying independent linear
functions on $\hat{S}_{t}$.
$Attn_{s}\in\mathbb{R}^{(\frac{WH}{s^{2}}+1)\times(\frac{WH}{s^{2}}+1)}$
represent a spatial-wise affinity map. We do not apply down-sampling on the
spatial attention as we saw a significant performance drop in our preliminary
experiments.

Our spatio-temporal split attention decreased the memory usage of the
transformer layer by reducing the affinity matrix from
$\mathbb{O}(T^{2}W^{2}H^{2})$ to $\mathbb{O}(\tau^{2}+W^{2}H^{2})$. This
allows us to explore longer temporal sequence lengths that were infeasible on
modern hardware with the vanilla transformer.

####  3.2.2 Temporal Down-sampling method

Video content usually contains redundant information [31], with multiple
frames depicting near identical content over time. We introduce compact VidTr
(C-VidTr) by applying temporal down-sampling within our transformer
architecture to remove some of this redundancy. We study different temporal
down-sampling methods ($pool$ in Eq. 3) including temporal average pooling and
1D convolutions with stride 2, which reduce the temporal dimension by half
(details in Table 5d).

A limitation of these pooling the methods is that they uniformly aggregate
information across time but often in video clips the informative frames are
not uniformly distributed. We adopted the idea of non-uniform temporal feature
aggregation from previous work [31]. Different from previous work [31] that
directly down-sample the query using average pooling, we found that in our
proposed network, the temporal attention highly activates on a small set of
temporal features when the clip is informative, while the attention equally
distributed over the length of the clip when the clip caries little additional
semantic information. Building on this intuition, we propose a topK based
pooling ($topK\\_std$ pooling) that orders instances by the standard deviation
of each row in the attention matrix:

$\small\operatorname{pool}_{topK\\_std}(Attn_{t}^{(1:,:)})=Attn_{t}^{\left(\operatorname{topK}\left(\sigma(Attn_{t}^{(1:,:)})\right),:\right)}$
(9)

where $\sigma\in\mathbb{R}^{T}$ is row-wise standard deviation of
$Attn_{t}^{(1:,:)}$ as:

$\displaystyle\small\sigma^{(i)}$
$\displaystyle=\frac{1}{T}\sqrt{\sum_{i=1}^{T}(Attn_{t}^{(i,:)}-\mu)^{2}}$
(10) $\displaystyle\mu^{(i)}$
$\displaystyle=\frac{1}{T}\sum_{i=1}^{T}Attn_{t}^{(i,:)}$ (11)

where $\mu\in\mathbb{R}^{T}$ is the mean of $Attn_{t}^{(1:,:)}$. Note that the
$topK\\_std$ pooling was applied to the affinity matrix excludes the token
($Attn_{t}^{(0,:,:)}$) as we always preserve token for information
aggregation. Our experiments show that $topK\\_std$ pooling gives better
performance than average pooling or convolution. The $topK\\_std$ pooling can
be intuitively understood as selecting the frames with strong localized
attention and removing frames with uniform attention.

###  3.3 Implementation Details

Model | $clip\\_len$ | $sr$ | Down-sample Layer | $\tau$
---|---|---|---|---
VidTr-S | 8 | 8 | - | -
VidTr-M | 16 | 4 | - | -
VidTr-L | 32 | 2 | - | -
C-VidTr-S | 8 | 8 | [1,2,4] | [6,4,2]
C-VidTr-M | 16 | 4 | [1,2,4] | [8,4,2]
Table 1: Detailed configuration of different VidTr permutations. $clip\\_len$
denotes the sampled clip length and $sr$ stands for the sample rate. We
uniformly sample $clip\\_len$ frames out of $clip\\_len\times sr$ consecutive
frames. The configurations are empirically selected, details in Compact VidTr
down-sampling twice starting from layer $1$ and skipping different number of
layers..

Model Instantiating: Based on the input clip length and sample rate, we
introduce three base VidTr models (VidTr-S,VidTr-M and VidTr-L). By applying
the different pooling strategies we introduce two compact VidTr permutations
(C-VidTr-S, and C-VidTr-M). To normalize the feature space, we apply layer
normalization before and after the residual connection of each transformer
layer and adopt the GELU activation as suggested in [14]. Detailed
configurations can be found in Table 1. We empirically determined the
configuration for different clip length to produce a set of models from low
FLOPs and low latency to high accuracy (details in Compact VidTr down-sampling
twice starting from layer $1$ and skipping different number of layers.).

During training we initialize our model weights from ViT-B [14]. To avoid over
fitting, we adopted the commonly used augmentation strategies including random
crop, random horizontal flip (except for Something-something dataset). We
trained the model using 64 Tesla V100 GPUs, with batch size of 6 per-GPU (for
VidTr-S) and weight decay of 1e-5. We adopted SGD as the optimizer but found
the Adam optimizer also gives us the same performance. We trained our network
for 50 epochs in total with initial learning rate of 0.01, and reduced it by
10 times after epochs 25 and 40. It takes about 12 hours for VidTr-S model to
converge, the training process also scales well with fewer GPUs (e.g. 8 GPUs
for 4 days). During inference we adopted the commonly used 30-crop evaluation
for VidTr and compact VidTr, with 10 uniformly sampled temporal segments and 3
uniformly sampled spatial crop on each temporal segment [55]. It is worth
mentioning that we can further boost the inference speed of compact VidTr by
adopting a single pass inference mechanise, this is because the attention
mechanism captures global information more effectively than 3D convolution. We
do this by training a model with frames sampled in TSN [54] style, and
uniformly sampling $N$ frames in inference (details in supplemental
materials).

##  4 Experimental Results

###  4.1 Datasets

We evaluate our method on six of the most widely used datasets. Kinetics 400
[8] and Kinetics 700 [6] consists of approximately 240K/650K training videos
and 20K/35K validation videos trimmed to 10 seconds from 400/700 human action
categories. We report top-1 and top-5 classification accuracy on the
validation sets. Something-Something V2 [23] dataset consists of 174 actions
and contains 168.9K training videos and 24.7K evaluation videos. We report
top-1 accuracy following previous works [37] evaluation setup. Charades [45]
has 9.8k training videos and 1.8k validation videos spanning about 30 seconds
on average. Charades contains 157 multi-label classes with longer activities,
performance is measured in mean Average Precision (mAP). UCF-101 [46] and
HMDB-51 [29] are two smaller datasets. UCF-101 contains 13320 videos with an
average length of 180 frames per video and 101 action categories. The HMDB-51
has 6,766 videos and 51 action categories. We report the top-1 classification
on the validation videos based on split 1 for both dataset.

###  4.2 Kinetics 400 Results

####  4.2.1 Comparison To SOTA

Model | Input | GFLOPs | Lat. | top-1 | top-5
---|---|---|---|---|---
I3D50 [60] | $32\times 2$ | 167 | 74.4 | 75.0 | 92.2
I3D101 [60] | $32\times 2$ | 342 | 118.3 | 77.4 | 92.7
NL50 [55] | $32\times 2$ | 282 | 53.3 | 76.5 | 92.6
NL101 [55] | $32\times 2$ | 544 | 134.1 | 77.7 | 93.3
TEA50 [34] | $16\times 2$ | 70 | - | 76.1 | 92.5
TEINet [39] | $16\times 2$ | 66 | 49.5 | 76.2 | 92.5
CIDC [32] | $32\times 2$ | 101 | 82.3 | 75.5 | 92.1
SF50 8$\times$8 [19] |  (32+8)$\times 2$ | 66 | 49.3 | 77.0 | 92.6
SF101 8$\times$8 [19] |  (32+8)$\times 2$ | 106 | 71.9 | 77.5 | 92.3
SF101 16$\times$8 [19] |  (64+16)$\times 2$ | 213 | 124.3 | 78.9 | 93.5
TPN50 [60] | $32\times 2$ | 199 | 89.3 | 77.7 | 93.3
TPN101 [60] | $32\times 2$ | 374 | 133.4 | 78.9 | 93.9
CorrNet50 [53] | $32\times 2$ | 115 | - | 77.2 | N/A
CorrNet101 [53] | $32\times 2$ | 187 | - | 78.5 | N/A
X3D-XXL [18] | $16\times 5$ | 196 | - | 80.4 | 94.6
Vanilla-Tr | $8\times 8$ | 89 | 32.8 | 77.5 | 93.2
VidTr-S | $8\times 8$ | 89 | 36.2 | 77.7 | 93.3
VidTr-M | $16\times 4$ | 179 | 61.1 | 78.6 | 93.5
VidTr-L | $32\times 2$ | 351 | 110.2 | 79.1 | 93.9
En-I3D-50-101 | $32\times 2$ | 509 | 192.7 | 77.7 | 93.2
En-I3D-TPN-101 | $32\times 2$ | 541 | 207.8 | 79.1 | 94.0
En-VidTr-S | $8\times 8$ | 130 | 73.2 | 79.4 | 94.0
En-VidTr-M | $16\times 4$ | 220 | 98.1 | 79.7 | 94.2
En-VidTr-L | $32\times 2$ | 392 | 147.2 | 80.5 | 94.6

Table 2: Results on Kinetics-400 dataset. We report top-1 accuracy(%) on the
validation set. The ‘Input’ column indicates what frames of the 64 frame clip
are actually sent to the network. $n\times s$ input indicates we feed $n$
frames to the network sampled every $s$ frames. Lat. stands for the latency on
single crop.

We report results on the validation set of Kinetics 400 in Table 2, including
the top-1 and top-5 accuracy, GFLOPs (Giga Floating-Point Operations) and
latency (ms) required to compute results on one view.

As shown in Table 2, the VidTr achieved the SOTA performance compared to
previous I3D based SOTA architectures with lower GFLOPs and latency. The VidTr
significantly outperform previous SOTA methods at roughly same computational
budget, e.g. at 200 GFLOPs, the VidTr-M outperform I3D50 by $3.6\%$, NL50 by
$2.1\%$,and TPN50 by $0.9\%$. At similar accuracy levels, VidTr is
significantly more computationally efficient than other works, e.g. at $78\%$
top-1 accuracy, the VidTr-S has 6$\times$ fewer FLOPs than NL-101, 2$\times$
fewer FLOPs than TPN and $12\%$ fewer FLOPs than Slowfast-101. We also see
that our VidTr outperforms I3D based networks at higher sample rate (e.g.
$s=8$, TPN achieved $76.1\%$ top-1 accuracy), this denotes, the global
attention learns temporal information more effectively than 3D convolutions.
X3D-XXL from architecture search is the only network that outperforms our
VidTr. We plan to use architecture search techniques for attention based
architecture in future work.

####  4.2.2 Compact VidTr

We evaluate the effectiveness of our compact VidTr with the proposed temporal
down-sampling method (Table 1). The results (Table 1) show that the proposed
down-sampling strategy removes roughly $56\%$ of the computation required by
VidTr with only $2\%$ performance drop in accuracy. The compact VidTr complete
the VidTr family from small models (only 39GFLOPs) to high performance models
(up to 79.1% accuracy). Compared with previous SOTA compact models [34, 39],
our compact VidTr achieves better or similar performance with lower FLOPs and
latency, including: TEA (+0.6% with 16% fewer FLOPs) and TEINet (+0.5% with
11% fewer FLOPs).

Model | Input | Res. | GFLOPs | Latency(ms) | top-1
---|---|---|---|---|---
TSM [37] |  $8f$TSN | 256 | 69 | $29$ | 74.7
TEA [34] |  16$\times$4 | 256 | 70 | - | 76.1
3DEffi-B4 [18] |  16$\times$5 | 224 | 7 | - | 72.4
TEINet [39] |  16$\times$4 | 256 | 33 | $36$ | 74.9
X3D-M [18] |  16$\times$5 | 224 | 5 | $40.9$ | 74.6
X3D-L [18] |  16$\times$5 | 312 | 19 | $59.4$ | 76.8
C-VidTr-S |  8$\times$8 | 224 | 39 | $17.5$ | 75.7
C-VidTr-M |  16$\times$4 | 224 | 59 | $26.1$ | 76.7

Table 3: Comparison of VidTr to other fast networks. We present the number of
views used for evaluation and FLOPs required for each view. The latency
denotes the total time required to get the reported top-1 score.111we measure
latency of X3D using the authors’ code and fast depth convolution patch:
https://github.com/facebookresearch/SlowFast/blob/master/projects/x3d/README.md,
which only has models for X3D-M and X3D-L and not the XL and XXL variants

####  4.2.3 Error and Ensemble Analysis

Top 5 (+) | Acc. gain
---|---
making a cake | +26.0%
catching fish | +21.2%
catching baseball | +20.8%
stretching arm | +19.1%
spraying | \+ 18.0 %
(a) Top 5 classes that VidTr works better than I3D.

Top 5 (-) | Acc. gain
---|---
shaking head | -21.7%
dunking basketball | -20.8%
lunge | -19.9%
playing guitar | -19.9%
tap dancing | -16.3%
(b) Top 5 classes that I3D works better than VidTr.

Table 4:  Quantitative analysis on Kinetics-400 dataset. The performance gain
is defined as the disparity of the top-1 accuracy between VidTr network and
that of I3D.

We compare the errors made by VidTr-S and the I3D50 network to better
understand the local networks’ (I3D) and global networks’ (VidTr) behavior. We
provide the top-5 activities that our VidTr-S gain most significant
improvement over the I3D50. We find that our VidTr-S outperformed the I3D on
the activities that requires long-term video contexts to be recognized. For
example, our VidTr-S outperformed the I3D50 on “making a cake” by 26% in
accuracy. The I3D50 overfits to “cakes” and often recognize making a cake as
eating a cake. We also analyze the top-5 activities where I3D does better than
our VidTr-S (Table 4). Our VidTr-S performs poorly on the activities that need
to capture fast and local motions. For example, our VidTr-S performs 21% worse
in accuracy on “shaking head”.

Inspired by the findings in our error analysis, we ensembled our VidTr with a
light weight I3D50 network by averaging the output values between the two
networks. The results (Table 2) show that the the I3D model and transformer
model complements each other and the ensemble model roughly lead to 2%
performance improvement on Kinetics 400 with limited additional FLOPs (37G).
The performance gained by ensembling the VidTr with I3D is significantly
better than the improvement by combine two 3D networks (Table 2).

####  4.2.4 Ablations

Model | FP. | top-1
---|---|---
Cubic (4$\times$162) | 23G | 73.1
Cubic (2$\times$162) | 45G | 75.5
Square (1$\times$162) | 89G | 77.7
Square (1$\times$322) | 21G | 71.2

(a) Comparison between different patching strategies.

Model | Mem. | top-1
---|---|---
WH | 2.1GB | 74.7
WHT | 7.6GB | 77.5
WH + T | 2.3GB | 77.7
W + H + T. | 1.5GB | 72.3

(b) Comparison between different factorization.

Init. from | FP. | top-1
---|---|---
T2T [62] | 34G | 76.3
ViT-B [14] | 89G | 77.7
ViT-L [14] | 358 | 77.5

(c) Comparison between different backbones.

Configurations | top-1 | top-5
---|---|---
Temp. Avg. Pool. | 74.9 | 91.6
1D Conv. [62] | 75.4 | 92.3
STD Pool. | 75.7 | 92.2

(d) Comparison between different down-sample methods.

Layer | $\tau$ | FP. | top-1
---|---|---|---
$[0,2]$ | $[4,2]$ | 26G | 72.9
$[1,3]$ | $[4,2]$ | 32G | 74.9
$[2,4]$ | $[4,2]$ | 47G | 74.9
$[6,8]$ | $[4,2]$ | 60G | 75.3

(e) Compact VidTr down-sampling twice at layer $k$ and $k+2$.

Layer | $\tau$ | FP. | top-1
---|---|---|---
$[1,2]$ | $[4,2]$ | 30G | 73.9
$[1,3]$ | $[4,2]$ | 32G | 74.9
$[1,4]$ | $[4,2]$ | 33G | 75.0
$[1,5]$ | $[4,2]$ | 34G | 75.2

(f) Compact VidTr down-sampling twice starting from layer $1$ and skipping
different number of layers.

Table 5:  Ablation studies on Kinetics 400 dataset. We use an VidTr-S backbone
with 8 frames input for (a,b) and C-VidTr-S for (c,d). The evaluation is
performed on 30 views with 8 frame input unless specified. FP. stands for
FLOPs.

We perform all ablation experiments with our VidTr-S model on Kinetics 400. We
used $8\times 224\times 224$ input with a frame sample rate of 8, and 30-view
evaluation.
Patching strategies:  We first compare the cubic patch ($4\times 16^{2}$),
where the video is represented as a sequence of spatio-temporal patches, with
the square patch ($1\times 16^{2}$), where the video is represented as a
sequence of spatial patches. Our results (Table 5a) show that the model using
cubic patches with longer temporal size has fewer FLOPs but results in
significant performance drop (73.1 vs. 75.5). The model using square patches
significantly outperform all cubic patch based models, likely because the
linear embedding is not enough to represent the shot-term temporal association
in the cubic. We further compared the performance of using different patch
sizes ($1\times 16^{2}$ vs. $1\times 32^{2}$), using $32^{2}$ patches lead to
$4\times$ decreasing of the sequence length, which decreases memory
consumption of the affinity matrices by $16\times$, however, using $16^{2}$
patches significantly outperform the model using $32^{2}$ patches (77.7 vs.
71.2). We did not evaluate the model using smaller patching sizes (e.g.,
$8\times 8$) because of the high memory consumption.
Attention Factorization: We compare different factorization for attention
design, including spatial modeling only (WH), jointly spatio-temporal modeling
module (WHT, vanilla-Tr), spatio-temporal separable-attention (WH + T, VidTr),
and axial separable-attention (W + H + T). We first evaluate an spatio-only
transformer. We average the class token for each input frame for our final
output. Our results (Table 5b) show that the spatio-only transformer requires
less memory but has worse performance compare with spatio-temporal attention
models. This shows that temporal modeling is critical for attention based
architectures. The joint spatio-temporal transformer significantly outperforms
the spatio-only transformer but requires a restrictive amount of memory
($T^{2}$ times for the affinity matrices). Our VidTr using spatio-temporal
separable-attention requires $3.3\times$ less memory with no accuracy drop. We
further evaluate the axial separable-attention (W + H + T), which requires the
least memory. The results (Table 5b) show that the axial separable-attention
has a significant performance drop likely due to breaking the X and Y spatial
dimensions.
Sequence down-sampling comparison:  We compare different down-sampling
strategy including temporal average pooling, 1D temporal convolution and the
proposed STD-based topK pooling method. The results (Table 5d) show that our
proposed STD-based down-sampling method outperformed the temporal average
pooling and the convolution-based down-sampling strategies that uniformly
aggregate information over time.

Figure 2: The comparison between different models on accuracy, FLOPs and
latency.

Backbone generalization:  We evaluate our VidTr initialized with different
models, including T2T [62], ViT-B, and ViT-L. The results on Table 5c show
that our VidTr achieves reasonable performance across all backbones. The VidTr
using T2T as the backbone has the lowest FLOPs but also the lowest accuracy.
The Vit-L-based VidTr achieve similar performance with the Vit-B-based VidTr
even with $3\times$ FLOPs. As showed in previous work [14], transformer-based
network are more likely to over-fit and Kinetics-400 is relatively small for
Vit-L-based VidTr.
Where to down-sample:  Finally we study where to perform temporal down-
sampling. We perform temporal down-sampling at different layers (Table 5e).
Our results (Table 5e) show that starting to perform down-sampling after the
first encoder layer has the best trade-off between the performance and FLOPs.
Starting to perform down-sampling at very beginning leads to the fewest FLOPs
but has a significant performance drop (72.9 vs. 74.9). Performing down-
sampling later only has slight performance improvement but requires higher
FLOPs. We then analyze how many layers to skip between two down-sample layers.
Based on the results in Table 5f, skipping one layer between two down-sample
operations has the best trade-off. Performing down-sampling on consecutive
layers (0 skip layers) has lowest FLOPs but the performance decreases (73.9
vs. 74.9). Skipping more layers did not show significant performance
improvement but does have higher FLOPs.

Model | Input | K700 | Chad | SS | UCF | HM
---|---|---|---|---|---|---
I3D [7] |  32$\times$2 | 58.7 | 32.9 | 50.0 | 95.1 | 74.3
TSM [37] | 8(TSN) | - | - | 59.3 | 94.5 | 70.7
I3D101 [59] |  32 $\times$ 4 | 40.3 | - | - | - |
CSN152 [49] | $32\times 2$ | 70.1 | - | - | - | -
TEINet[39] | 16 (TSN) | - | - | 62.1 | 96.7 | 73.3
SF101 [19] |  64$\times$2 | 70.2 | - | 60.9 | - | -
SF101-NL [19] |  64$\times$2 | 70.6 | 45.2 | - | - | -
X3D-XL [18] |  16 $\times$ 5 | - | 47.1 | - | - | -
VidTr-M |  16 $\times$ 4 | 69.5 | - | 61.9 | 96.6 | 74.4
VidTr-L |  32 $\times$ 2 | 70.2 | 43.5 | 63.0 | 96.7 | 74.4
En-VidTr-L |  32 $\times$ 2 | 70.8 | 47.3 | - | - | -
Table 6: Results on Kinetics-700 dataset (K700), Charades dataset (Chad),
something-something-V2 dataset (SS), UCF-101 and HMDB (HM) dataset. The
evaluation metrics are mean average precision (mAP) in percentage for Charades
(32$\times$4 input is used), top-1 accuracy for Kinetics 700, something-
something-V2 (TSN styled dataloader is used), UCF and HMDB.

####  4.2.5 Run-time Analysis

We further analyzed the trade-off between latency, FLOPs and accuracy. We note
that the VidTr achieved the best balance between these factors (Figure 2). The
VidTr-S achieve similar performance but significantly fewer FLOPs compare with
I3D101-NL ($5\times$ fewer FLOPs), Slowfast101 $8\times 8$ ($12\%$ fewer
FLOPs), TPN101 ($2\times$ fewer FLOPs), and CorrNet50 ($20\times$ fewer
FLOPs). Note that the X3D has very low FLOPs but high latency due to the use
of depth convolution. Our experiments show that the X3D-L has about
$3.6\times$ higher latency comparing with VidTr-S (Figure 2).

###  4.3 More Results

##### Kinetics-700 Results:

Our experiments show a consistent performance trend on Kinetics 700 (Table 6).
The VidTr-S significantly outperformed the baseline I3D model (+9%), the
VidTr-M achieved the performance comparable to Slowfast101 $8\times 8$ and the
VidTr-L is comparable to previous SOTA slowfast101-nonlocal. There is a small
performance gap between our model and Slowfast-NL [19], because Slowfast is
pre-trained on both Kinetics 400 and 600 while we only pre-trained on Kinetics
400. Previous findings that VidTr and I3D are being complementary is
consistent on Kinetics 700, ensemble VidTr-L with I3D leads to +0.6%
performance boost.
Charades Results:  We compare our VidTr with previous SOTA models on Charades.
Our VidTr-L outperformed previous SOTA methods LFB and NUTA101, and achieved
the performance comparable to Slowfast101-NL (Table 6). The results on
Charades demonstrates that our VidTr generalizes well to multi-label activity
datasets. Our VidTr performs worse than the current SOTA networks (X3D-XL) on
Charades likely due to overfitting. As discussed in previous work [14], the
transformer-based networks overfit easier than convolution-based models, and
Charades is relatively small. We observed a similar finding with our ensemble,
ensembling our VidTr with a I3D network (40.3 mAP) achieved SOTA performance.
Something-something V2 Results:  We observe that the VidTr does not work well
on the something-something dataset (Table 6), likely because pure transformer
based approaches do not model local motion as well as convolutions. This
aligns with our observation in our error analysis. Further improving local
motion modeling ability is an area of future work.
UCF and HMDB Results:  Finally we train our VidTr on two small dataset UCF-101
and HMDB-51 to test if VidTr generalizes to smaller datasets. The VidTr
achieved SOTA comparable performance with 6 epochs of training (96.6% on UCF
and 74.4% on HMDB), showing that the model generalize well on small dataset
(Table 6).

##  5 Visualization and Understanding VidTr

(a) The spatial and temporal attention in Vidtr. The attention is able to
focus on the informative frames and regions.

(b) The rollout attentions from different layers of VidTr.

(c) Comparison of I3D activations and VidTr attentions.

Figure 3: Visualization of spatial and temporal attention of VidTr and
comparison with I3D activation.

We first visualized the VidTr’s separable-attention with attention roll-out
method [1] (Figure 3a). We find that the spatial attention is able to focus on
informative regions and temporal attention is able to skip the duplicated/non-
representative information temporally. We then visualized the attention at
4th, 8th and 12th layer of VidTr (Figure 3b), we found the spatial attention
is stronger on deeper layers. The attention does not capture meaningful
temporal instances at early stages because the temporal feature relies on the
spatial information to determine informative temporal instances. Finally we
compared the I3D activation map and rollout attention from VidTr (Figure 3c).
The I3D mis-classified the catching fish as sailing, as the I3D attention
focused on the people sitting behind and water. The VidTr is able to make the
correct prediction and the attention showed that the VidTr is able to focus on
the action related regions across time.

##  6 Conclusion

In this paper, we present video transformer with separable-attention, an novel
stacked attention based architecture for video action recognition. Our
experimental results show that the proposed VidTr achieves state-of-the-art or
comparable performance on five public action recognition datasets. The
experiments and error analysis show that the VidTr is especially good at
modeling the actions that requires long-term reasoning. Further combining the
advantage of VidTr and convolution for better local-global action modeling
[57, 38] and adopt self-supervised training [9] on large-scaled data will be
our future work.

## References

  * [1] Samira Abnar and Willem Zuidema.  Quantifying attention flow in transformers.  In Proceedings of the 58th Annual Meeting of the Association for Computational Linguistics, pages 4190–4197, 2020.
  * [2] Anurag Arnab, Mostafa Dehghani, Georg Heigold, Chen Sun, Mario Lučić, and Cordelia Schmid.  Vivit: A video vision transformer.  arXiv preprint arXiv:2103.15691, 2021.
  * [3] Iz Beltagy, Matthew E Peters, and Arman Cohan.  Longformer: The long-document transformer.  arXiv preprint arXiv:2004.05150, 2020.
  * [4] Gedas Bertasius, Heng Wang, and Lorenzo Torresani.  Is space-time attention all you need for video understanding?  arXiv preprint arXiv:2102.05095, 2021.
  * [5] Nicolas Carion, Francisco Massa, Gabriel Synnaeve, Nicolas Usunier, Alexander Kirillov, and Sergey Zagoruyko.  End-to-end object detection with transformers.  In European Conference on Computer Vision, pages 213–229. Springer, 2020.
  * [6] Joao Carreira, Eric Noland, Chloe Hillier, and Andrew Zisserman.  A short note on the kinetics-700 human action dataset.  arXiv preprint arXiv:1907.06987, 2019.
  * [7] Joao Carreira and Andrew Zisserman.  Quo vadis, action recognition? a new model and the kinetics dataset.  In proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pages 6299–6308, 2017.
  * [8] J. Carreira and A. Zisserman.  Quo vadis, action recognition? a new model and the kinetics dataset.  In 2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR), pages 4724–4733, July 2017.
  * [9] Xinlei Chen, Saining Xie, and Kaiming He.  An empirical study of training self-supervised visual transformers.  arXiv preprint arXiv:2104.02057, 2021.
  * [10] Yunpeng Chen, Yannis Kalantidis, Jianshu Li, Shuicheng Yan, and Jiashi Feng.  Multi-fiber networks for video recognition.  In Proceedings of the european conference on computer vision (ECCV), pages 352–367, 2018.
  * [11] Zhigang Dai, Bolun Cai, Yugeng Lin, and Junying Chen.  Up-detr: Unsupervised pre-training for object detection with transformers.  arXiv preprint arXiv:2011.09094, 2020.
  * [12] Jacob Devlin, Ming-Wei Chang, Kenton Lee, and Kristina Toutanova.  Bert: Pre-training of deep bidirectional transformers for language understanding.  arXiv preprint arXiv:1810.04805, 2018.
  * [13] Jacob Devlin, Ming-Wei Chang, Kenton Lee, and Kristina Toutanova.  Bert: Pre-training of deep bidirectional transformers for language understanding.  In Proceedings of the 2019 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, Volume 1 (Long and Short Papers), pages 4171–4186, 2019.
  * [14] Alexey Dosovitskiy, Lucas Beyer, Alexander Kolesnikov, Dirk Weissenborn, Xiaohua Zhai, Thomas Unterthiner, Mostafa Dehghani, Matthias Minderer, Georg Heigold, Sylvain Gelly, et al.  An image is worth 16x16 words: Transformers for image recognition at scale.  arXiv preprint arXiv:2010.11929, 2020.
  * [15] Brendan Duke, Abdalla Ahmed, Christian Wolf, Parham Aarabi, and Graham W Taylor.  Sstvos: Sparse spatiotemporal transformers for video object segmentation.  arXiv preprint arXiv:2101.08833, 2021.
  * [16] Haoqi Fan, Bo Xiong, Karttikeya Mangalam, Yanghao Li, Zhicheng Yan, Jitendra Malik, and Christoph Feichtenhofer.  Multiscale vision transformers.  arXiv preprint arXiv:2104.11227, 2021.
  * [17] Quanfu Fan, Chun-Fu Chen, Hilde Kuehne, Marco Pistoia, and David Cox.  More is less: Learning efficient video representations by big-little network and depthwise temporal aggregation.  arXiv preprint arXiv:1912.00869, 2019.
  * [18] Christoph Feichtenhofer.  X3d: Expanding architectures for efficient video recognition.  In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 203–213, 2020.
  * [19] Christoph Feichtenhofer, Haoqi Fan, Jitendra Malik, and Kaiming He.  Slowfast networks for video recognition.  In Proceedings of the IEEE International Conference on Computer Vision, pages 6202–6211, 2019.
  * [20] Valentin Gabeur, Chen Sun, Karteek Alahari, and Cordelia Schmid.  Multi-modal transformer for video retrieval.  In European Conference on Computer Vision (ECCV), volume 5. Springer, 2020.
  * [21] Rohit Girdhar, Joao Carreira, Carl Doersch, and Andrew Zisserman.  Video action transformer network.  In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 244–253, 2019.
  * [22] Rohit Girdhar, Deva Ramanan, Abhinav Gupta, Josef Sivic, and Bryan Russell.  ActionVLAD: Learning Spatio-Temporal Aggregation for Action Classification.  In The IEEE Conference on Computer Vision and Pattern Recognition (CVPR), 2017.
  * [23] Raghav Goyal, Samira Ebrahimi Kahou, Vincent Michalski, Joanna Materzynska, Susanne Westphal, Heuna Kim, Valentin Haenel, Ingo Fruend, Peter Yianilos, Moritz Mueller-Freitag, et al.  The” something something” video database for learning and evaluating visual common sense.  In ICCV, volume 1, page 3, 2017.
  * [24] Kensho Hara, Hirokatsu Kataoka, and Yutaka Satoh.  Can spatiotemporal 3d cnns retrace the history of 2d cnns and imagenet?  In Proceedings of the IEEE conference on Computer Vision and Pattern Recognition, pages 6546–6555, 2018.
  * [25] Sepp Hochreiter and Jürgen Schmidhuber.  Long short-term memory.  Neural computation, 9(8):1735–1780, 1997.
  * [26] Shuiwang Ji, Wei Xu, Ming Yang, and Kai Yu.  3d convolutional neural networks for human action recognition.  IEEE transactions on pattern analysis and machine intelligence, 35(1):221–231, 2012.
  * [27] Boyuan Jiang, MengMeng Wang, Weihao Gan, Wei Wu, and Junjie Yan.  STM: SpatioTemporal and Motion Encoding for Action Recognition.  In The IEEE International Conference on Computer Vision (ICCV), 2019\.
  * [28] Andrej Karpathy, George Toderici, Sanketh Shetty, Thomas Leung, Rahul Sukthankar, and Li Fei-Fei.  Large-scale video classification with convolutional neural networks.  In Proceedings of the IEEE conference on Computer Vision and Pattern Recognition, pages 1725–1732, 2014.
  * [29] Hildegard Kuehne, Hueihan Jhuang, Estíbaliz Garrote, Tomaso Poggio, and Thomas Serre.  Hmdb: a large video database for human motion recognition.  In 2011 International Conference on Computer Vision, pages 2556–2563. IEEE, 2011.
  * [30] Qing Li, Zhaofan Qiu, Ting Yao, Tao Mei, Yong Rui, and Jiebo Luo.  Action recognition by learning deep multi-granular spatio-temporal video representation.  In Proceedings of the 2016 ACM on International Conference on Multimedia Retrieval, pages 159–166, 2016.
  * [31] Xinyu Li, Chunhui Liu, Bing Shuai, Yi Zhu, Hao Chen, and Joseph Tighe.  Nuta: Non-uniform temporal aggregation for action recognition.  arXiv preprint arXiv:2012.08041, 2020.
  * [32] Xinyu Li, Bing Shuai, and Joseph Tighe.  Directional temporal modeling for action recognition.  In European Conference on Computer Vision, pages 275–291. Springer, 2020.
  * [33] Yan Li, Bin Ji, Xintian Shi, Jianguo Zhang, Bin Kang, and Limin Wang.  TEA: Temporal Excitation and Aggregation for Action Recognition.  In The IEEE Conference on Computer Vision and Pattern Recognition (CVPR), 2020.
  * [34] Yan Li, Bin Ji, Xintian Shi, Jianguo Zhang, Bin Kang, and Limin Wang.  Tea: Temporal excitation and aggregation for action recognition.  In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 909–918, 2020.
  * [35] Yingwei Li, Weixin Li, Vijay Mahadevan, and Nuno Vasconcelos.  VLAD3: Encoding Dynamics of Deep Features for Action Recognition.  In The IEEE Conference on Computer Vision and Pattern Recognition (CVPR), 2016.
  * [36] Zekang Li, Zongjia Li, Jinchao Zhang, Yang Feng, Cheng Niu, and Jie Zhou.  Bridging text and video: A universal multimodal transformer for video-audio scene-aware dialog.  arXiv preprint arXiv:2002.00163, 2020.
  * [37] Ji Lin, Chuang Gan, and Song Han.  Tsm: Temporal shift module for efficient video understanding.  In Proceedings of the IEEE/CVF International Conference on Computer Vision, pages 7083–7093, 2019.
  * [38] Ze Liu, Yutong Lin, Yue Cao, Han Hu, Yixuan Wei, Zheng Zhang, Stephen Lin, and Baining Guo.  Swin transformer: Hierarchical vision transformer using shifted windows.  arXiv preprint arXiv:2103.14030, 2021.
  * [39] Zhaoyang Liu, Donghao Luo, Yabiao Wang, Limin Wang, Ying Tai, Chengjie Wang, Jilin Li, Feiyue Huang, and Tong Lu.  TEINet: Towards an Efficient Architecture for Video Recognition.  In The Conference on Artificial Intelligence (AAAI), 2020.
  * [40] Ze Liu, Jia Ning, Yue Cao, Yixuan Wei, Zheng Zhang, Stephen Lin, and Han Hu.  Video swin transformer.  arXiv preprint arXiv:2106.13230, 2021.
  * [41] Daniel Neimark, Omri Bar, Maya Zohar, and Dotan Asselmann.  Video transformer network.  arXiv preprint arXiv:2102.00719, 2021.
  * [42] Mandela Patrick, Dylan Campbell, Yuki M Asano, Ishan Misra Florian Metze, Christoph Feichtenhofer, Andrea Vedaldi, Jo Henriques, et al.  Keeping your eye on the ball: Trajectory attention in video transformers.  arXiv preprint arXiv:2106.05392, 2021.
  * [43] AJ Piergiovanni, Anelia Angelova, and Michael S Ryoo.  Tiny video networks.  arXiv preprint arXiv:1910.06961, 2019.
  * [44] Hao Shao, Shengju Qian, and Yu Liu.  Temporal interlacing network.  In Proceedings of the AAAI Conference on Artificial Intelligence, volume 34, pages 11966–11973, 2020.
  * [45] Gunnar A. Sigurdsson, Gül Varol, Xiaolong Wang, Ivan Laptev, Ali Farhadi, and Abhinav Gupta.  Hollywood in homes: Crowdsourcing data collection for activity understanding.  ArXiv e-prints, 2016.
  * [46] Khurram Soomro, Amir Roshan Zamir, and M Shah.  A dataset of 101 human action classes from videos in the wild.  Center for Research in Computer Vision, 2012.
  * [47] Hugo Touvron, Matthieu Cord, Matthijs Douze, Francisco Massa, Alexandre Sablayrolles, and Hervé Jégou.  Training data-efficient image transformers & distillation through attention.  arXiv preprint arXiv:2012.12877, 2020.
  * [48] Du Tran, Lubomir Bourdev, Rob Fergus, Lorenzo Torresani, and Manohar Paluri.  Learning spatiotemporal features with 3d convolutional networks.  In Proceedings of the IEEE international conference on computer vision, pages 4489–4497, 2015.
  * [49] Du Tran, Heng Wang, Lorenzo Torresani, and Matt Feiszli.  Video classification with channel-separated convolutional networks.  In Proceedings of the IEEE/CVF International Conference on Computer Vision, pages 5552–5561, 2019.
  * [50] Du Tran, Heng Wang, Lorenzo Torresani, Jamie Ray, Yann LeCun, and Manohar Paluri.  A closer look at spatiotemporal convolutions for action recognition.  In Proceedings of the IEEE conference on Computer Vision and Pattern Recognition, pages 6450–6459, 2018.
  * [51] Amin Ullah, Jamil Ahmad, Khan Muhammad, Muhammad Sajjad, and Sung Wook Baik.  Action recognition in video sequences using deep bi-directional lstm with cnn features.  IEEE access, 6:1155–1166, 2017.
  * [52] Ashish Vaswani, Noam Shazeer, Niki Parmar, Jakob Uszkoreit, Llion Jones, Aidan N Gomez, Lukasz Kaiser, and Illia Polosukhin.  Attention is all you need.  arXiv preprint arXiv:1706.03762, 2017.
  * [53] Heng Wang, Du Tran, Lorenzo Torresani, and Matt Feiszli.  Video modeling with correlation networks.  In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 352–361, 2020.
  * [54] Limin Wang, Yuanjun Xiong, Zhe Wang, Yu Qiao, Dahua Lin, Xiaoou Tang, and Luc Van Gool.  Temporal Segment Networks: Towards Good Practices for Deep Action Recognition.  In The European Conference on Computer Vision (ECCV), 2016.
  * [55] Xiaolong Wang, Ross Girshick, Abhinav Gupta, and Kaiming He.  Non-local neural networks.  In Proceedings of the IEEE conference on computer vision and pattern recognition, pages 7794–7803, 2018.
  * [56] Chao-Yuan Wu, Christoph Feichtenhofer, Haoqi Fan, Kaiming He, Philipp Krahenbuhl, and Ross Girshick.  Long-term feature banks for detailed video understanding.  In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 284–293, 2019.
  * [57] Haiping Wu, Bin Xiao, Noel Codella, Mengchen Liu, Xiyang Dai, Lu Yuan, and Lei Zhang.  Cvt: Introducing convolutions to vision transformers.  arXiv preprint arXiv:2103.15808, 2021.
  * [58] Saining Xie, Ross Girshick, Piotr Dollár, Zhuowen Tu, and Kaiming He.  Aggregated residual transformations for deep neural networks.  In Proceedings of the IEEE conference on computer vision and pattern recognition, pages 1492–1500, 2017.
  * [59] Ceyuan Yang, Yinghao Xu, Jianping Shi, Bo Dai, and Bolei Zhou.  Temporal Pyramid Network for Action Recognition.  In The IEEE Conference on Computer Vision and Pattern Recognition (CVPR), 2020.
  * [60] Ceyuan Yang, Yinghao Xu, Jianping Shi, Bo Dai, and Bolei Zhou.  Temporal pyramid network for action recognition.  In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 591–600, 2020.
  * [61] Sen Yang, Zhibin Quan, Mu Nie, and Wankou Yang.  Transpose: Towards explainable human pose estimation by transformer.  arXiv preprint arXiv:2012.14214, 2020.
  * [62] Li Yuan, Yunpeng Chen, Tao Wang, Weihao Yu, Yujun Shi, Francis EH Tay, Jiashi Feng, and Shuicheng Yan.  Tokens-to-token vit: Training vision transformers from scratch on imagenet.  arXiv preprint arXiv:2101.11986, 2021.
  * [63] Joe Yue-Hei Ng, Matthew Hausknecht, Sudheendra Vijayanarasimhan, Oriol Vinyals, Rajat Monga, and George Toderici.  Beyond short snippets: Deep networks for video classification.  In Proceedings of the IEEE conference on computer vision and pattern recognition, pages 4694–4702, 2015.
  * [64] Bolei Zhou, Alex Andonian, Aude Oliva, and Antonio Torralba.  Temporal Relational Reasoning in Videos.  In The European Conference on Computer Vision (ECCV), 2018.
  * [65] Luowei Zhou, Yingbo Zhou, Jason J Corso, Richard Socher, and Caiming Xiong.  End-to-end dense video captioning with masked transformer.  In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pages 8739–8748, 2018.


##  Chapter High precision Drell-Yan predictions

The level of accuracy reached by the high energy physics experiments imposes
the need for high precision in the theoretical predictions. The prime process
for precision benchmarking is the Drell-Yan process, this can be both measured
and predicted with high accuracy. The calculation needs to take into account
extra radiation in the final state and the most important contribution comes
from QCD radiation. The experimental results are often extracted within the
fiducial region, that corresponds to the application of kinematic cuts on the
final state leptons. For a meaningful comparison with the experimental data,
the prediction must also implement fiducial cuts. To do this the fully
exclusive cross-section for the vector boson production is needed. This is
known up to NNLO in QCD for Drell-Yan process [nnlo-exclusive-dy-catani, nnlo-
dy-fewz, nnlo-dy-mcfm]. The NNLO calculation considers up to two additional
emissions and must deal with the cancellation of the related soft and
collinear singularities. To this purpose two main approaches have been
developed. These are often referred to as _slicing_ and _local_ singularity
subtraction schemes. Thanks to the success of these methods, the NNLO QCD
accuracy for DY is now the state of the art and the predictions can be
obtained with various public codes [nnlo-dy-fewz, nnlo-dy-mcfm, MATRIX-dy-
nnlo, dyturbo]. More recently also inclusive N3LO results have been obtained
[dy-n3lo]. Despite the level of precision claimed by the NNLO codes, it has
been noticed that the results differ between each other by an amount that can
reach the percent level. This has been noticed for example by the ATLAS
collaboration in performing the $W/Z$ cross section measurement at
$7\umeas{TeV}$ [atlas-w-z-7tev]. The disagreement is understood to be related
to the presence of fiducial cuts applied to final state leptons and the
different subtraction schemes adopted in the calculation [fpc-tackman, moch-
benchmark-nnlo]. The disagreement can be as high as few percent, in some cases
this is of the same order as the pure NNLO correction term and, most
importantly, is not negligible compared to the experimental uncertainty of
some DY measurements. In the case of the ATLAS $W/Z$ measurement mentioned
above, for example, the maximum precision reached is of $0.5\%$. These
differences in the theory predictions can have an impact on PDF extraction
where Drell-Yan differential measurements play an important role in
constraining the $u-$ and $d-$quark valence distributions. In this chapter a
benchmark study of predictions for the DY process produced with different
codes adopting different subtraction schemes is reported. In section 1 the
idea behind the different subtraction approaches is described. In section 1
the problem of linear power correction that biases the results of slicing
methods, in the presence of fiducial cuts, is explored. The results of the
benchmark comparison are reported in section 2. The predictions are produced
in the fiducial region used in the ATLAS $W/Z$ $7\umeas{TeV}$ cross section
measurement [atlas-w-z-7tev]. The same calculations are used in section 3 for
a PDF study where the agreement of the predictions in combination with
different PDFs and the data is assessed.

###  1 Subtraction methods

In section LABEL:sec:high_order_qcd, Figure LABEL:fig:dy-nlo, the different
diagrams contributing to the NLO QCD correction for the DY process were
presented; in general, two kind of terms are identified: virtual loop
correction to the LO vertex and real emission correction. Both contributions
present collinear and soft (or infrared (IR)) singularities. General results
guarantee the cancellation of the IR singularity when summing up the two
contribution [Lee-Nauenberg, kinoshita], and the universal factorisation of
the collinear poles, that can be absorbed into a renormalization of the PDFs.
These results are valid at each perturbative order. At NNLO the classes of
divergent contributions are three: double real emission, real-virtual
contribution, and double loop correction. Despite the cancellation of the IR
singularities the problem of calculating the exclusive cross section is
challenging: fully analytical results for a general phase space integration
are not available and numerical results cannot be applied straightforwardly
due to the divergent nature of the integrand. In the virtual and real
correction terms, the number of final state particles is different, and the
phase space integration has to be performed separately, preventing direct
cancellation in the sum of the two terms. Two classes of techniques, _local
subtraction_ and _slicing_ methods, have then been developed to isolate and
regularize the divergent terms in the integrands. In the next sections the
idea behind the two methods is presented.

####  Slicing methods

Slicing methods identify an IR safe observable that resolves additional
emissions. This observable is used to slice the phase space integration into
two regions, one containing the singular term and the other in which the cross
section is regular. The regular part can be integrated safely using standard
techniques, the singular part is instead approximated thanks to the known
universal structure of the singularities. Typical observables are the
transverse momentum $q_{T}$ of a color singlet [nnlo-exclusive-dy-
catani],[nnlo-exclusive-higgs-catani] and N-jettiness [tackmann-njettiness].
In the Drell-Yan process $q_{T}$ is the transverse momentum of the vector
boson. The slicing parameter is usually defined as

$\tau=\frac{q_{T}^{2}}{Q^{2}}$ (1)

$Q^{2}$ is the hard scale of the born level process, the invariant mass of the
exchanged boson in the DY case. When $\tau\rightarrow 0$ the cross section
$d\sigma/d\tau$ reaches the singular limit. The cross section integration is
sliced by introducing a threshold value $\tau_{\text{cut}}$

$\sigma=\int_{0}^{\tau_{\text{cut}}}\text{d}\tau\frac{\text{d}\sigma}{\text{d}\tau}+\int_{\tau_{\text{cut}}}\text{d}\tau\frac{\text{d}\sigma}{\text{d}\tau}$
(2)

The first term in the right hand side contains the singular term, while the
second term is regular and can be integrated with standard techniques. The
integrand, $d\sigma/d\tau$, can be divided into two contributions, the
singular one, that contains the singular contribution as $\tau\rightarrow 0$
and a non-singular one

$\frac{d\sigma}{d\tau}=\frac{d\sigma^{\text{sing}}}{d\tau}+\frac{d\sigma^{\text{nonsing}}}{d\tau}.$
(3)

The slicing method relies on the fact that the singular contribution, that
dominate the low $\tau$ integration part, can be predicted from the
factorisation theorem, the structure of the IR singularities and the KLN
theorem. The structure is of the type

$\frac{d\sigma^{\text{sing}}}{d\tau}\sim\delta(\tau)+\sum_{i}\left[\frac{\ln\tau}{\tau}\right]_{+}$
(4)

The $+$ distribution, whose definition was given in equation LABEL:eq:plus-
distribution, indeed represents the structure of the cancellation between the
real and virtual IR divergences. This knowledge can be used to approximate the
cross section at low $\tau$. Integrating the singular term up to the threshold
$\tau_{cut}$ one obtains something of the order

$\sigma^{\text{sing}}(\tau_{cut})=\int_{0}^{\tau_{cut}}\frac{d\sigma^{\text{sing}}}{d\tau}\sim
1+\sum_{j\geq 0}\ln^{j+1}\tau_{\text{cut}}.$ (5)

$\sigma^{\text{nonsing}}$ instead includes terms that are at least $\tau$
integrable, or that go to zero when $\tau\rightarrow 0$. The structure is of
type

$\split&\frac{d\sigma^{\text{nonsing}}}{d\tau}\sim\sum_{m>0}\sum_{j\geq
0}\tau^{m-1}\ln^{j}\tau\\\
\sigma^{\text{nonsing}}(\tau_{\text{cut}})=\int_{0}^{\tau_{\text{cut}}}d\tau\frac{d\sigma^{\text{nonsing}}}{d\tau}\sim\sum_{m>0}\sum_{j\geq
0}\tau^{m}\ln^{j}\tau\sim O(\tau)$ (6)

These are considered power corrections ($O(\tau_{\text{cut}}^{m}),\,m>0$) to
the singular term. The cross section in eq. 2 is calculated by adding and
subtracting a subtraction term $\sigma^{sub}$

$\sigma=\sigma^{sub}(\tau_{\text{cut}})+\int_{\tau_{\text{cut}}}d\tau\frac{d\sigma}{d\tau}+\Delta\sigma^{sub}(\tau_{\text{cut}}).$
(7)

$\sigma^{sub}(\tau_{\text{cut}})$ approximates the cross section at low
$\tau$, while the term
$\Delta\sigma^{sub}(\tau_{\text{cut}})=\sigma(\tau_{\text{cut}})-\sigma^{sub}(\tau_{\text{cut}})$,
that is neglected when doing the computation, represents the error of this
approximation. $\sigma^{sub}(\tau_{\text{cut}})$ should include at least the
singular term of equation 5,

$\sigma^{sub}(\tau_{cut})=\sigma^{\text{sing}}(\tau_{cut})[1+O(\tau_{\text{cut}})]$
(8)

so that the error in the calculation is a power correction

$\Delta\sigma^{sub}(\tau_{\text{cut}})\sim O(\tau^{m}_{\text{cut}})$ (9)

The actual value of $m$ depends on how many terms of
$\sigma^{\text{nonsing}}(\tau_{\text{cut}})$, equation 6, can be included in
$\sigma^{sub}$. In the case of the inclusive Drell-Yan cross section the
$O(\tau^{1}_{\text{cut}})$ is known at NLO and can be included in
$\sigma^{sub}$. The neglected term $\Delta\sigma$ is a power correction
$O(\tau^{2}_{\text{cut}})$. The presence of fiducial cuts on the final state
leptons can introduce additional power dependence of
$\text{d}\sigma^{\text{nonsing}}/\text{d}\tau$ on $q_{T}$ and lower the
precision of the approximation.

####  Fiducial power corrections

The power corrections to the singular term, equation 6, represent the
intrinsic error of slicing subtraction methods. This error is of order
$O((q^{2}_{T,\,\text{cut}}/Q^{2})^{2})$ in the case of DY predictions and for
inclusive calculation. A choice of the slicing threshold
$\tau_{\text{cut}}=(q^{2}_{T,\,\text{cut}}/Q^{2})$ small enough makes this
bias negligible. The presence of fiducial cuts on the final state leptons,
however, can introduce additional power $q_{T}$ correction that cause a
further bias in the predictions. These fiducial power corrections (FPC) arise
when considering the final state phase space integration. Using a schematic
notation, the phase space integration reads as

$\frac{d\sigma}{d\tau}=\int d\Phi_{F}\frac{d\sigma}{d\tau}(\Phi_{F})$ (10)

where $d\Phi_{F}$ represent the phase space integration on the final state
leptons. The singular term of the cross section depends only on the Born phase
space, where no additional emission are considered,

$\frac{d\sigma^{\text{sing}}}{d\tau}=\int
d\Phi^{\text{Born}}_{F}\frac{d\sigma^{\text{sing}}}{d\tau}(d\Phi).$ (11)

When applying fiducial cuts a dependence of the acceptance, $\Phi_{F}=\int
d\Phi_{F}$, on the boson transverse momentum is introduced. The reason is
shown intuitively in Figure where the $p_{T}$ cut on the leptons coming from
the $Z$ decay are shown schematically. When the $Z$ is at rest both leptons
pass the requirement $p_{T\,1,2}>p_{T}^{\text{min}}$. This is no longer the
case when the $Z$ boson is given a small transverse momentum.

[width = 0.95]pheno-study/fid-power-correction-sketch.pdf

Figure : Schematic representation of the rising of $q_{T}$ power corrections
due to fiducial cuts on the final state leptons. On left, the case in which
the $Z$ boson is at rest and the two lepton are passing the fiducial cuts is
shown. When the $Z$ is given a small transverse boost, Figure on the right,
one of the two leptons fail to pass the selection.

The error introduced in the slicing method can be estimated by comparing the
acceptance of the final state with the Born one

$\Delta\Phi_{F}=\int d\Phi_{F}(q_{T})-d\Phi_{F}^{\text{Born}}(q_{T}=0)$ (12)

It has been shown [fpc-tackman] that

$\Delta\Phi_{F}\sim\frac{1}{q_{T}^{2}}\left(\frac{q_{T}^{2}}{Q^{2}}\right)^{m}$
(13)

where the value of $m$ depends on the fiducial cut configuration. In the case
of a $p_{T}^{\ell}$ symmetric cut,
$p_{T}^{\ell_{1},\,\ell_{2}}>p_{T}^{\text{min}}$, it is found that $m=1/2$,
that means

$\left.\Delta\Phi_{F}\right|_{p_{T}^{\ell_{1},\ell_{2}}>p_{T}^{\text{min}}}\sim\frac{1}{q_{T}^{2}}\left(\frac{q_{T}}{Q}\right)$
(14)

This linear dependence introduces a bias in the cross section calculation,
equation 7, that is dominant with respect to the inclusive power corrections,
equation 9. In the case of additional lepton pseudorapidity cuts, for example
a symmetric cut $|\eta_{\ell_{1},\ell_{2}}|<\eta_{\text{min}}$, linear power
correction also arise, but effects can be different according to the di-lepton
rapidity configuration. A more detailed analysis of the relation between
different fiducial cut configurations and the power corrections can be found
in Refs [moch-benchmark-nnlo, salam-two-body-decay]. A simple solution in the
DY case to avoid this bias is to change the selection configuration. A
selection in which different cut thresholds are applied to on the lepton and
anti-lepton $p_{T}$, also known as _staggered_ cuts, is not affected by linear
$q_{T}$ biases [matrix-fully-diff-NNLO, salam-two-body-decay]. The symmetric
cuts are anyway still common in DY measurements and relevant measurements have
been performed with this configuration [atlas-w-z-7tev]. A way to mitigate the
effects of the linear power corrections is then needed. A first possibility is
to lower the $q_{T,\,\text{cut}}$ threshold value, or to have an extrapolation
method to small $q_{T,\,\text{cut}}$ values. Both of these approaches affect
the stability and the performance of the calculation. Furthermore the
introduction of a dependence of the cross section on $q_{T}$ introduces some
instability in the series expansion due to non perturbative small-$q_{T}$
effects [salam-two-body-decay]. These affect any fixed order calculations,
regardless of the subtraction scheme. A more feasible approach is then to
include the $q_{T}$ resummation effects in the predictions. It has been shown
in Ref. [tackman-fpc-resummation] that the implementation of a recoil
prescription of the boson transverse momentum as done in the $q_{T}$
resummation [pt-resum-catani-2015, pt-resum-becher, pt-resum-bertone-2019,
n3ll-radish, n3ll-wan-li-ju-2020, dyturbo], allows the linear power
corrections to be resummed at each perturbative order. The residual fiducial
biases are quadratic in $q_{T}$ and so of the same order as in the inclusive
case. Starting from this result it is also possible to include the linear
power correction in $q_{T}$ at a given order in $\alpha_{S}$. This gives a
fixed order result that can be used for a benchmark study with the local
subtraction method predictions. This procedure has been implemented in public
code such as MATRIX [matrix-fpc] or DYTurbo [dyturbo-fpc]. In section 2 a
benchmark study of predictions for the DY process in the case of fiducial cuts
is presented and the DYTurbo code is used.

####  Local subtraction methods

The other strategy to deal with the divergences is the local subtraction
method. This aims to correct for the singular contributions in the cross
section point by point at the integrand level, so that the phase space
integration can be freely performed, without the need of approximations. In
the case of NLO calculations, the cross section can be split into the real and
virtual contributions, whose phase space integration is performed over a
different number of particles in the final state

$\sigma^{NLO}=\int d\sigma=\int_{N+1}d\sigma^{\text{R}}+\int_{N}d\sigma^{V}$
(15)

A subtraction for the real term that encodes all its IR divergences is now
constructed. Adding it to the real term eliminates the singularities, so that
the $N+1$ phase space integration can be performed

$\split\sigma^{NLO}&=\int_{N+1}\text{d}\sigma^{\text{R}}-\int_{N+1}\text{d}\sigma^{\text{S}}+\int_{N+1}\text{d}\sigma^{\text{S}}+\int_{N}\text{d}\sigma^{\text{V}}\\\
=\int_{N+1}\left[\text{d}\sigma^{\text{R}}-\text{d}\sigma^{\text{S}}\right]+\int_{N+1}\text{d}\sigma^{\text{S}}+\int_{N}\text{d}\sigma^{\text{V}}$
(16)

The subtraction term is also built in a way that the integral on the extra
particle can be factorized and the term can be combined with the virtual
contribution. In particular it can be combined with the loop integral in the
virtual term from which the infrared singularity arises, hence allowing the
cancellation

$\sigma^{NLO}=\int_{N+1}\left[\text{d}\sigma^{\text{R}}-\text{d}\sigma^{\text{S}}\right]+\int_{N}\left[\int_{\text{loop}}\text{d}\sigma^{\text{V}}+\int_{1}\text{d}\sigma^{\text{S}}\right].$
(17)

The definition of the subtraction term presents a certain freedom, in
particular in its finite contribution. Different definition choices is what
distinguish the local subtraction methods available for the calculation. At
NLO the FKS [nlo-FKS-sub-scheme] and the Catani-Seymour [nlo-catani-seymur-
sub-scheme] schemes are implemented. At NNLO the level of complexity is
significantly higher due to the overlapping of singularities. One of the
established method is antenna subtraction [antenna-nnlo, antenna-nnlo-
hadronic, antenna-nnlo-infrared], this is implemented in the NNLOJET code.
NNLO Drell-Yan predictions from this program are used in the next section for
the benchmark study. An alternative approach is the _sector decomposition_ as
implemented in the simulation code FEWZ [nnlo-dy-fewz] or in the STRIPPER code
[stripper-I, stripper-II, stripper-III].

###  2 NLO and NNLO benchmark study

In this section a benchmark study of NLO and NNLO QCD predictions for the DY
process is presented. The comparison is performed between the results of two
codes: DYTurbo and NNLOJET. These use different subtraction schemes, DYTurbo
implements a slicing subtraction in $q_{T}$ [dyturbo], while NNLOJET uses a
local subtraction based on the antenna subtraction method. A treatment to
include the linear $q_{T}$ power corrections due to the fiducial cuts is
implemented in DYTurbo [dyturbo-fpc] and is benchmarked in this section.
Furthermore, the DYTurbo offers also the possibility to include the $q_{T}$
resummation effects in the predictions. Also these predictions are used in the
comparison. The benchmark is performed in the context of predictions for the
ATLAS $7\umeas{TeV}$ $W/Z$ measurement. This offers an interesting case in
which to evaluate the differences in the theory as this data set reaches an
high experimental precision and contributes significantly in the PDF
determination. The same predictions will be used in section 3 for a comparison
of the different theory definitions with the data.

####  Fiducial region

The predictions are produced in the fiducial region of the ATLAS
$7\umeas{TeV}$ $W/Z$ cross section measurement [atlas-w-z-7tev]. Only the
$Z/\gamma^{*}\rightarrow\ell\ell$ cross section is used in the benchmark
study. The ATLAS measurement is performed in two channels. A central channel
that selects leptons at central pseudorapidity, $|\eta_{\ell}|<2.5$, and a
forward channel in which one of the two leptons is required to be produced at
high pseudorapidity, $2.5<|\eta_{\ell}|<4.9$. The kinematics requirement
applied for the cross section measurement are

$\split\text{Central}\,\,Z/\gamma^{*}\rightarrow\ell\ell:\,\,&p_{T,\,\ell}>20\umeas{GeV},\\\
|\eta_{\ell}|<2.5,\\\ 46<m_{\ell\ell}<150\umeas{GeV}\\\
\text{Forward}\,\,Z/\gamma^{*}\rightarrow\ell\ell:\,\,p_{T,\,\ell}>20\umeas{GeV},\\\
\text{onelepton}\,|\eta|<2.5,\,\text{theotherlepton}\,\,2.5<|\eta|<4.9,\\\
\,66<m_{\ell\ell}<150\umeas{GeV}$ (18)

The cross section is measured in three different bins of the dilepton
invariant mass

$m_{\ell\ell}=[46,66,116,140]\umeas{GeV}$ (19)

these are referred to as _low mass_ , _$Z$ -peak_ and _high mass_ bins. In the
low mass bin only the central channel is measured. In each mass bin the cross
section is measured differentially in the dilepton rapidity $|y_{\ell\ell}|$.
In the $Z$-peak region, central channel, the following bins edges are used

$|y_{\ell\ell}|=[0.0,0.2,0.4,0.8,1.0,1.2,1.4,1.6,1.8,2.0,2.2,2.4]$ (20)

In the low and high mass bins a binning twice as coarse is used. When
measuring the cross section in the forward channel instead, bins with the
following boundaries are chosen

$|y_{\ell\ell}|=[1.2,1.4,1.6,1.8,2.0,2.2,2.4,2.8,3.2,3.6].$ (21)

The same fiducial region and binning schemes are used in producing the
predictions. The measurement implements symmetric cuts on the lepton and anti-
lepton $p_{T}$ and both asymmetric and symmetric cuts (according to the
channel) on the lepton pseudorapidity $\eta$. As discussed in section 1, an
increase in the fiducial power correction when calculating the cross section
with slicing subtraction methods is expected in all these configuration .

####  Simulation setup

The DYTurbo and NNLOJET prediction are calculated with the same input
parameters. The $G_{\mu}$ scheme is used: $G_{F}$, $m_{W}$, $m_{Z}$ are the
input values while $\sin^{2}\theta_{W}$ and the QED coupling $\alpha(m_{Z})$
are calculated as output. In general, the Standard Model input parameters are
set to the following values [PDG]

$\split G_{F}&=1.1663787\times 10^{-5}\umeas{GeV}^{-2},\\\
m_{Z}=91.1876\umeas{GeV},\\\ m_{W}=79.936\umeas{GeV},\\\
\Gamma_{Z}=2.4950\umeas{GeV},\\\ \Gamma_{W}=2.091\umeas{GeV}.\\\ $ (22)

The CKM matrix elemets are set to [PDG]

$\alignedat{2}|V_{ud}|&=0.97427,\quad|V_{cd}|=0.2252,\\\
|V_{us}|=0.2253,\quad|V_{cs}|=0.97344,\\\
|V_{ub}|=0.00351,\quad|V_{cb}|=0.0412\\\ $ (23)

The input PDFs are taken from the NNPDF3.1 set [nnpdf31]. The values of the
renormalization and factorization scale $\mu_{R}$ and $\mu_{F}$ is set to in
the dilepton invariant mass in each event, $\mu_{R}=\mu_{F}=m_{\ell\ell}$.
DYTurbo implements a slicing subtraction scheme in $q_{T}$. The vaule of the
slicing cut-off is set to $(q_{T}/Q)_{\text{cut}}=0.008$. Additional
parameters to include the resummation effects need to be set. The first is the
resummation scale $\mu_{\text{Res}}$ that is set, as the renormalization and
factorization scales, to the dilepton invariant mass $m_{\ell\ell}$. The
resummation damping function that regulates the vanishing of the resummation
effects at high $q_{T}$ is set to a Gaussian:
$f(x)=\exp(-(k\,m_{\ell\ell}-q_{T})^{2}/\Delta\,m_{\ell\ell}^{2})$ with
$k=0.75$ and $\Delta=0.5$. A term to parametrize non-perturbative QCD effects
also needs to be included. This is a Gaussian form factor in the space of the
impact parameter $b$, $G^{\text}{{NP}}(b)=\exp(-g_{1}b^{2})$ with $g_{1}=0.8$.
A comparison of the predictions at LO, where no difference due to the
subtraction schemes is expected, is carried out to test the simulation setup.
Agreement at the level of 0.1per mille is found.

####  NLO comparison

The predictions at NLO QCD are compared. DYTurbo implements the $q_{T}$
slicing also at this perturbative order and differences with respect to the
NNLOJET results are expected. The fiducial power correction term implemented
in DYTurbo is considered, this helps to recover good agreement between the two
subtraction scheme calculations. In figure 2 the NLO comparison for the
$Z$-peak bin predictions, in the central and forward channel is shown. A
disagreement beetween the fixed order DYTurbo and NNLOJET results is observed.
In the central channel the differences are of the order of 6 per mille, these
are more severe in the forward channel where the ratio departs from one as
much as $7\%$. The inclusion of the FPC term in the DYTurbo predictions
improves the agreement at the level better than $0.5$ per mille. Similar
observation can be made by comparing the predictions in the high mass bin,
Figure 2. The effect of fiducial cuts are partially smaller than in the
$Z$-peak bin and again, adding the FPC term to the DYTurbo predictions,
increases the agreement at a per mille level. In the low mass bin 2 the effect
of using different subtraction methods results only in a 2 per mille
difference at central rapididty, including the FPC further improves the
agreement.

[width = 0.49]pheno-study/nlo-comparison/zypeak_cc.pdf [width = 0.49]pheno-
study/nlo-comparison/zypeak_cf.pdf

Figure : NLO QCD predictions for the ATLAS $Z$-peak central (left) and forward
(right) channel. The fixed order DYTurbo prediction shows a disagreement with
respect to the NNLOJET. Good agreement is recovered after including the FPC
term.

[width = 0.49]pheno-study/nlo-comparison/zyhigh_cc.pdf [width = 0.49]pheno-
study/nlo-comparison/zyhigh_cf.pdf

Figure : NLO QCD predictions for the ATLAS high mass central (left) and
forward (right) channel. The fixed order DYTurbo prediction shows a
disagreement with respect to the NNLOJET is observe. Good agreement is
recovered after including the FPC term. In the forward channel some residual
discrepancies are observed at very high rapidity.

[width = 0.49]pheno-study/nlo-comparison/zylow_cc.pdf

Figure : NLO QCD predictions for the ATLAS low mass central channel. The fixed
order DYTurbo prediction is in slight disagreement with respect to the
NNLOJET. Including the FPC term the agreement is further improved.

####  NNLO comparison

A benchmark study at NNLO precision is performed in this section. In the
comparison, also the DYTurbo results that include the $q_{T}$ resummation
corrections are reported, as they will be used in the next section for the PDF
study. These prediction correspond to a NNLO+NNLL accuracy. In Figure 2 the
comparison of the NNLO $Z$-peak bin predictions is reported. Both in the
forward and central channel differences between NNLOJET and the fixed order
DYTURBO results are observed. These are of the the same order as those noticed
at NLO, Figure 2 (but in the opposite direction: at NLO DYTurbo undershoots
the NNLOJET cross section, here it overshoots it). Adding the FPC term,
DYTurbo is able to recover a per mille level agreement. The NNLO+NNLL result
deviates from the fixed order by few per mille in the central channel, with
some shape differences for $|y_{\ell\ell}|>1.2$. In the forward channel the
resummation seems to have a bigger effect, and the deviation gets as high as
$3\%$.

[width = 0.49]pheno-study/nnlo-comparison/zypeak_cc.pdf [width = 0.49]pheno-
study/nnlo-comparison/zypeak_cf.pdf

Figure : Comparison of the NNLO QCD calculations from NNLOJET and DYTURBO. The
predictions are for the $Z$-peak bin in the central (left) and forward (right)
channel.

In the high mass bin, Figure 2, the inclusion of the FPC term doesn’t improve
the agreement of the DYTurbo results with the NNLOJET ones. For both in the
central and forward it actually slightly increases the differences. The reason
for this behaviour is yet to be understood. The resummation effects lead to
few per mille deviations in the central channel and up to per cent differences
in the forward channel.

[width = 0.49]pheno-study/nnlo-comparison/zyhigh_cc.pdf [width = 0.49]pheno-
study/nnlo-comparison/zyhigh_cf.pdf

Figure : Comparison of the NNLO QCD calculations from NNLOJET and DYTURBO. The
predictions are for the high mass bin in the central (left) and forward
(right) channel.

The comparison for the low mass bin predictions is reported in Figure 2. An
improvement in the agreement when including the FPC term is observed this
time. The differences are reduced to a $5$per mille level. The resummed
DYTurbo result has a shape similar to the fixed order one, but a $2.5\%$
offset difference.

[width = 0.55]pheno-study/nnlo-comparison/zylow_cc.pdf

Figure : Comparison of the NNLO QCD predictions for the low mass bin.

###  3 Data theory comparison

Thanks to the experimental precision and the wide phase space considered, the
ATLAS data presented in section 2 provide a significant constraint on the PDFs
[atlas-w-z-7tev]. These measurements are usually included in determining
modern PDF sets. In this section a study of the compatibility of some modern
PDF sets with the ATLAS data when changing the underlying theory is reported.
The study is performed with _xfitter_ [xfitter], an open source software that
offers a framework for PDF determination. The data theory compatibility is
carried out through a $\chi^{2}$ test. The $\chi^{2}$ definition takes into
account the correlated systematic uncertainties, both experimental and
theoretical (from the PDFs), introducing nuisance parameters that are
extracted through a $\chi^{2}$ minimization. The $\chi^{2}$ at the minimum is
equivalent to the more standard definition
$\chi^{2}=\sum_{ij}(\sigma^{\text{exp}}_{i}-\sigma^{\text{th}}_{j})V^{-1}_{ij}(\sigma^{\text{exp}}_{j}-\sigma^{\text{th}}_{j})$,
where $V_{ij}$ is the uncertainty covariance matrix, see Ref. [correlated-
syst-chi2, chi2-pdfUncert-lagrangianMul] for the proof. Using the nuisance
parameter approach is computationally more feasible because one avoids the
inversion of the matrix $V_{ij}$. Furthermore it allows us to distinguish
between the different sources of systematics. The $\chi^{2}$ is defined as

$\split\chi^{2}(\mathbf{b}_{\text{exp}},\mathbf{b}_{\text{th}})=&\sum_{i=1}^{N_{\text{data}}}\frac{\left[\sigma^{\text{exp}}_{i}-\sigma^{\text{th}}_{i}(1-\sum_{k}\gamma_{ik}^{\text{th}}b_{k,\text{th}}-\sum_{j}\gamma_{ij}^{\text{exp}}b_{j,\text{exp}})\right]^{2}}{\Delta_{i}^{2}}\\\
+\sum_{i}\log\frac{\delta^{2}_{i,\text{uncor}}(\sigma^{\text{th}}_{i})^{2}+\delta^{2}_{i,\text{stat}}\sigma^{\text{exp}}_{i}\sigma^{\text{th}}_{i}}{\delta^{2}_{i,\text{uncor}}(\sigma^{\text{exp}}_{i})^{2}+\delta^{2}_{i,\text{stat}}(\sigma^{\text{exp}}_{i})^{2}}\\\
+\sum_{j=1}^{N_{\text{exp.sys}}}b^{2}_{j,\text{exp}}+\sum_{k=1}^{N_{\text{th.sys}}}b^{2}_{k,\text{th}}.$
(24)

In the formula, the index $i$ runs over all $N_{\text{data}}$ points. For each
point, the data measurement $\sigma_{i}^{\text{exp}}$ and theory point
$\sigma_{i}^{\text{th}}$ are compared taking into account both the
experimental uncertainties and theoretical uncertainties coming from PDF
variations. The uncorrelated experimental uncertainties are given by
$\Delta_{i}$. The correlated uncertainty component is included introducing the
nuisance parameter vectors $\mathbf{b_{\text{exp}}}$ and
$\mathbf{b_{\text{th}}}$. The impact of the uncertainty source $j$ on the
data(theory) point $i$ is described by the matrices
$\gamma^{\text{data(th)}}_{ij}$ . Both the correlated and uncorrelated
uncertainties are treated as multiplicative (the uncertainties are
proportional to central prediction values). The $\chi^{2}$ function is
minimized as a function of $\mathbf{b_{\text{exp}}}$ and
$\mathbf{b_{\text{exp}}}$. The contribution to the $\chi^{2}$ for the
determination of the nuisance parameters is given by the two sums of
$b^{2}_{j}$ over the number of correlated uncertainty sources
$N_{\text{exp.sys}}$, $N_{\text{th.sys}}$. This term is referred to as
_correlated $\chi^{2}$_ contribution when reporting the results of the study.
The $\log$ term instead is added to minimize biases. Because the $\chi^{2}$ is
quadratic in $\mathbf{b}$ the minimization can be performed analytically. The
$\chi^{2}$ at its minimum gives a test of compatibility between data and
theory. The values of the nuisance parameters $b_{k,\,\text{th}}$ at this
minimum can be interpreted as an optimization of the PDFs to describe the
data, and allows us to determine a new set of PDFs. This procedure is called
PDF profiling [pdf-profiling, pdf-profile-nlo-camarda] and can be used to test
in a quantitative way the impact of new data measurement on a PDF set without
performing a new PDF fit. In this study we limit our self to use the
$\chi^{2}$ at the minimum as a way to check the agreement with the data. The
theory points $\sigma^{\text{th}}_{i}$ are calculated using the input PDF and
the NLO APPLGRID [applgrid] interfaced with the MCFM parton level generator
[MCFM-I, MCFM-II, MCFM-III]. The NNLO accuracy is reached via $k$-factors
($kf$) calculated in each observable bin, as the ratio between the higher and
lower order QCD calculation

$kf_{i}=\frac{\sigma_{i}^{NNLO}}{\sigma_{i}^{NLO}}.$ (25)

The NNLO DYTurbo results presented in the previous section are used to
calculate the $k$-factors.

####  Results

Some modern sets of PDFs are used for the comparison with the ATLAS data and
using the NNLO theory predictions from DYTurbo. The PDF sets considered are
NNPDF3.1$\\_$nnlo$\\_$as$\\_$0118 [nnpdf31], NNPDF4.0 [NNPDF40], CT18NNLO,
CT18ANNLO [ct18nnlo] and MSHT20nnlo$\\_$as118 [MSHT20]. In Table 3 the
$\chi^{2}$ results using the NNPDF3.1 and NNPDF4.0 for the comparison are
reported. In table 3 the results using the CT18 and CT18A PDF sets are shown.
Finally the results using MSHT20 PDF set are reported in Table 3. In each of
the result tables, the $\chi^{2}$ contribution from each data set, the first
term on the right hand side of eq. 24, is reported, together with the
corresponding number of degrees of freedom. At the bottom of the table the
correlated and total $\chi^{2}$ are listed. Different theory predictions from
DYTurbo are tested: the fixed order NNLO calculation, the prediction including
the FPC term, and the $q_{T}$ resummed calculation. The results seem to be in
line with what exposed in section 1: the data-theory agreement improves when
including the FPC term in the NNLO predictions, obtaining a reasonable
$\chi^{2}$ over degrees of freedom. A further improvement is obtained when
considering the $q_{T}$ resummation effects (third column in Tables 3, 3, 3),
resulting in a $\chi^{2}/ndf\sim 1$. This behaviour of the results is
noticeable with each of the PDF sets under consideration.

Table : Results of compariosn of the NNPDF3.1 and NNPDF4.0 PDF sets with the
ATLAS 7 TeV datasets, using different NNLO theories definitions. The NNLO
precision is reached with KFactor calcultated from the DYTurbo predicitons.

NNPDF3.1 NNLO NNLO+FPC NNLO+NNLL Data set ATLAS 7 TeV $y_{Z}$ low mass CC 14 /
6 9.8 / 6 8.5 / 6 ATLAS 7 TeV $y_{Z}$ $Z$-peak CC 8.4 / 12 6.7 / 12 6.3 / 12
ATLAS 7 TeV $y_{Z}$ $Z$-peak CF 5.2 / 9 5.4 / 9 7.1 / 9 ATLAS 7 TeV $y_{Z}$
high mass CC 5.7 / 6 5.8 / 6 5.3 / 6 ATLAS 7 TeV $y_{Z}$ high mass CF 4.8 / 6
4.8 / 6 4.6 / 6 Correlated $\chi^{2}$ 16 15 9.7 Log penalty $\chi^{2}$ -2.28
-2.28 -2.72 Total $\chi^{2}$ / dof 52 / 39 45 / 39 39 / 39 $\chi^{2}$ p-value
0.09 0.23 0.48 NNPDF4.0 NNLO NNLO+FPC NNLO+NNLL Data set ATLAS 7 TeV $y_{Z}$
low mass CC 15.2/6 10.2/6 8.7/6 ATLAS 7 TeV $y_{Z}$ $Z$-peak CC 12.1/12 8.3/12
6.5/12 ATLAS 7 TeV $y_{Z}$ $Z$-peak CF 9.8/9 5.7/9 7.3/9 ATLAS 7 TeV $y_{Z}$
high mass CC 5.1/6 5.7/6 5.3/6 ATLAS 7 TeV $y_{Z}$ high mass CF 4.6/6 5/6
4.8/6 Correlated $\chi^{2}$ 8.02 12.6 6.6 Log Penalty $\chi^{2}$ -2.1 -1.6 -2
Total $\chi^{2}/$dof 52.7/39 46/39 37.1/39 $\chi^{2}$ p-value 0.07 0.20 0.56

Table : Results of the comparison of the CT18 and CT18A PDF sets with the ATLAS 7 TeV datasets, using different NNLO theory definitions. The NNLO precision is reached with KFactor calcultated from the DYTurbo predicitons. | CT18 NNLO |  |
---|---|---|---
| NNLO | NNLO+FPC | NNLO+NNLL
Datasets |  |  |
ATLAS 7 TeV $y_{Z}$ low mass CC | 7.2/6 | 5.7/6 | 5.9/6
ATLAS 7 TeV $y_{Z}$ $Z$-peak CC | 12.6/12 | 8.3/12 | 7.1/12
ATLAS 7 TeV $y_{Z}$ $Z$-peak CF | 8.9/9 | 6/9 | 7.2/9
ATLAS 7 TeV $y_{Z}$ high mass CC | 5.3/6 | 5.8/6 | 5.4/6
ATLAS 7 TeV $y_{Z}$ high mass CF | 4.6/6 | 5/6 | 4.6/6
Correlated $\chi^{2}$ | 13.6 | 13.3 | 10.1
Log Penalty $\chi^{2}$ | -3.5 | -3 | -3
Total $\chi^{2}$ | 48.7/39 | 41.1/39 | 36.9/39
$\chi^{2}$ p-value | 0.13 | 0.38 | 0.57
| CT18A NNLO |  |
| NNLO | NNLO+FPC | NNLO+NNLL
Datasets |  |  |
ATLAS 7 TeV $y_{Z}$ low mass CC | 7.4/6 | 5.9/6 | 6.1/6
ATLAS 7 TeV $y_{Z}$ $Z$-peak CC | 12.4/12 | 8.2/12 | 6.9/12
ATLAS 7 TeV $y_{Z}$ $Z$-peak CF | 8.7/9 | 6/9 | 7.2/9
ATLAS 7 TeV $y_{Z}$ high mass CC | 5.4/6 | 5.7/6 | 5.5/6
ATLAS 7 TeV $y_{Z}$ high mass CF | 4.5/6 | 5/6 | 5/6
Correlated $\chi^{2}$ | 13.2 | 13.1 | 9.7
Log Penalty $\chi^{2}$ | -3.5 | -2.9 | -3.4
Total $\chi^{2}$ | 48.1/39 | 41.1/39 | 36.6/39
$\chi^{2}$ p-value | 0.15 | 0.38 | 0.58
Table : Results of the comparisong of the MSHT20 PDF set with the ATLAS 7 TeV datasets, using different NNLO theory definitions. The NNLO precision is reached with KFactor calcultated from the DYTurbo predicitons. | MSHT20 |  |
---|---|---|---
| NNLO | NNLO+FPC | NNLO+NNLL
Dataset |  |  |
ATLAS 7 TeV $y_{Z}$ low mass CC | 19.9/6 | 12.1/6 | 11.3/6
ATLAS 7 TeV $y_{Z}$ $Z$-peak CC | 14.0/12 | 7.7/12 | 7/12
ATLAS 7 TeV $y_{Z}$ $Z$-peak CF | 10.9/9 | 6.5/9 | 8.1/9
ATLAS 7 TeV $y_{Z}$ $Z$-mass CC | 5.2/6 | 5.9/6 | 5.3/6
ATLAS 7 TeV $y_{Z}$ high mass CF | 4.9/6 | 5.3/6 | 5.1/6
Correlated $\chi^{2}$ | 20.3 | 15.9 | 14.1
Log Penalty $\chi^{2}$ | -3.6 | -3.1 | -3.5
Total $\chi^{2}$ | $71.7/39$ | $50.4/39$ | $47.3/39$
$\chi^{2}$ p-value | $0.001$ | $0.11$ | $0.17$

###  4 Conclusion and outlook

In this chapter, a study regarding high precision QCD predictions for DY was
introduced and the differences between the two main calculation methods, local
(or slicing) subtraction and global subtraction methods, in the presence of
fiducial cuts on the final state leptons, have been explored. Some cut
configurations lead the phase space acceptance to have a dependence on the
vector boson transverse momentum, $q_{T}$, at small values. This induces
linear $q_{T}$ biases in the slicing method predictions and furthermore
introduces an instability in the convergence of the series expansion,
regardless of the subtraction scheme used. In section 2 it has been shown how
the inclusion of a fiducial power correction (FPC) term improves the nominal
accuracy of the results. This correction allows us to perform benchmark
comparison between global and local subtraction scheme calculations. The FPC
term comes from the resummation results. The resummation program offers a way
to resum the linear $q_{T}$ bias at each perturbative order and improves the
stability of the power expansion. The downside of including resummation effect
is the introduction of additional parameters in the calculation. A further
investigation should include the theoretical uncertainty effects related to
this new parameters. The theory predictions of chapter  are calculated in the
same fiducial region as in the ATLAS $Z$ measurement at $7\umeas{TeV}$ [atlas-
w-z-7tev], and are used in section 3 for a comparison study with the data. The
comparison tests different PDF sets, and the various theory predictions. In
particular the fixed order NNLO predictions, the prediction including the FPC
term and the resummed results are tested. With the latter two, an improvement
of the agreement $\chi^{2}$ is observed. The ATLAS collaboration performed
also a $W^{+/-}$ measurement in the same analysis; to complete the picture,
these data and the corresponding predictions could be added to the PDF study
of section 3. The results of this study is in preparation [dis-22-conference,
my-pubbl-fpc]. The $q_{T}$ resummation effects can be important in the
presence of lepton fiducial cuts. So far these effects have been ignored when
determining the parton distribution functions (PDF). To fully assess the
variation in the results a PDF fit should be performed varying the underlying
theory and comparing the the different outcomes. The study can be a combined
fit of deep inelastic scattering data from the HERA experiment [H1-zeus-
combination] and the most relevant Drell-Yan data sets. For some of the DY
data it would be important to evaluate the effects of resummation. Other than
the ATLAS $W/Z$ measurements at $7\umeas{TeV}$, interesting DY measurements to
include are for example the ATLAS $W/Z$ measurements at $5\umeas{TeV}$ and
$8\umeas{Tev}$ [atlas-wz-5tev, atlas-wz-8tev], the CMS $W$ and $Z$ forward-
backward asymmetry measurements at $7$ and $8\umeas{TeV}$ [cms-z-fbasym-8tev,
cms-w-fbasym-7tev] and also the Drell-Yan analysis performed at Tevatron
collider by the D0 [D0-DY-Zee, D0-Amu-Wmunu, D0-A_W, D0-A_e] and CDF [CDF-DY-
Zee-1.8TeV, CDF-DY-Zee-1.96TeV, CDF-A_W, CDF-W-Ae] experiments.