# An Empirical Analysis of the Impact of Data Augmentation on Knowledge
Distillation

Deepan Das  Haley Massa  Abhimanyu Kulkarni  Theodoros Rekatsinas

###### Abstract

Generalization Performance of Deep Learning models trained using Empirical
Risk Minimization can be improved significantly by using Data Augmentation
strategies such as simple transformations, or using Mixed Samples. We attempt
to empirically analyze the impact of such strategies on the transfer of
generalization between teacher and student models in a distillation setup. We
observe that if a teacher is trained using any of the mixed sample
augmentation strategies, such as MixUp or CutMix, the student model distilled
from it is impaired in its generalization capabilities. We hypothesize that
such strategies limit a model’s capability to learn example-specific features,
leading to a loss in quality of the supervision signal during distillation. We
present a novel Class-Discrimination metric to quantitatively measure this
dichotomy in performance and link it to the discriminative capacity induced by
the different strategies on a network’s latent space.

##  1 Introduction

Figure 1: Overview of the results of Mix Up, Cut Mix, Cut Out and simple
transformations on CIFAR-10 Test Set when distilled into two different student
models from a ResNet Teacher Model. Note that Mixed Sample Augmentation
strategies help improve teacher performance, but corresponding student
performance is impaired.

Deep Neural Networks (DNN) now have the ability to generate sophisticated
models that capture the intricacies of information in large amounts of data.
This ability has lent spectacular success to Deep Learning models in many
domains. However, often as the ability for the machine learning models to
encapsulate information needed to express this complexity increases, so does
the size of the machine learning model. These complex models may be too
demanding to run on more reasonable hardware, including personal computers or
mobile devices. One solution to this problem is to use knowledge-based
distillation, which trains a smaller, more efficient model that approximates
the performance of the original, more cumbersome model. A knowledge
distillation objective can enable us to train a smaller, lightweight model
without using any external annotations on given data, but instead utilising
the prediction labels generated by a pre-trained cumbersome teacher model.
There are plenty of other techniques that achieve high performance in
compressed models including model quantization (Zhou et al., 2017), model
pruning (Han et al., 2015), and more recently, lottery tickets (Jonathan
Frankle, 2018).

Figure 2: Pictorial representation of the data samples generated using the
several data augmentation strategies. Starting from the left, we note that the
original image is that of a firetruck. Standard transformations such as
flipping and cropping produces the augmented image. Cutout introduces regional
dropout in the image, whereas in Mix Up and Cut Mix, images and their labels
are combined proportionally. For instance, here we see that a car is combined
along with the firetruck.

With the recent surge of deep learning research, there is a natural need to
create knowledge-based distillation techniques that mimic more complex
algorithms. At the same time, however, there is little basis for evaluating
the impact of existing strategies on the latent representation space generated
by the teacher model that allows distillation training on the student model.
This raises the question: what is the impact of generalization in teacher
networks? While many forms of explicit regularization have been added to
large, cumbersome models, it will be interesting to analyze the impact of the
implicit regularization techniques when applied to such neural networks
through the different optimization techniques. Implicit regularization has
shown that as neural networks increase in size, they are actually able to
create solutions with lower complexity (Neyshabur, 2017). Since there is no
explicit regularization in the model’s objective function, this ability to
generalize is built in through well studied techniques such as normalization,
gradient descent, and the choice of weight kernels, like the convolution
filter. Some of these regularizers have become fairly popular and have become
a standard addition to any Deep Learning model optimization.

Regularization performed on neural networks have many different aspects. In a
distillation setup, where a smaller model(student) attempts to mimic the
softened softmax output of a large, cumbersome model(teacher), student
performance is shown to improve, but there still exists a significant
generalization gap between the two models. It is indeed important to try and
minimize this generalization gap, but it is also important to consider the
viability of using one or many of these implicit regularizers in the
distillation process. In this experiment, we will be creating a framework that
measures the impact of using such implicit regularizers on the generalization
gap between teacher and student models.

One such form of implicit regularization is the Vicinal Risk Minimization
(VRM) principle (Vapnik, 2000). In this work, we will evaluate the impact data
augmentation inspired by VRM techniques have in the transfer of generalization
between teacher and student models. However, this is a general framework, and
can also be used to evaluate other types of generalization in the future. In
any natural setting, we encounter noise in the presented data that enables us
to gain additional information about relatively similar data. Traditionally,
data augmentation and even Cut Out (DeVries & Taylor, 2017), is viewed as one
such example, where the model is trained on some simple transformations of the
data. However, the scope of a vicinity is not explicitly defined in this case.
Mixed Sample Data Augmentation Techniques like Mix Up (Zhang et al., 2017) and
Cut Mix (Yun et al., 2019) or FMix (Harris et al., 2020) provide a new outlook
on the concept of vicinity. Considered as standalone techniques, they can lead
to state of the art results in standard Deep Learning tasks, but it will be
interesting to note the impact each technique has on the supervision signal
from the teacher model to the student model. We hypothesize that even though
Data Augmentation techniques provide good regularization, they impair the
distillation process because of several implicit qualitative biases in the
techniques. This impairment is much more pronounced in the Mixed Sample Data
Augmentation Techniques.

To prove this hypothesis, we will be using a framework to evaluate four data
augmentation techniques inspired by VRM (traditional augmentation, Mix Up, Cut
Out and Cut Mix) by creating teacher networks on ResNet-18 architectures.
Student networks will then be distilled and evaluated against appropriate data
sets to investigate their generalization capabilities. Our contributions are
thus summarized as follows:

  * •

We demonstrate that popular data augmentation techniques, and especially Mixed
Sample techniques, such as MixUp and CutMix when applied on a teacher model,
can impair the transfer of generalization capabilities onto a student model in
a distillation setting.

  * •

We present a novel similarity-based metric to help explain some qualitative
traits inherent in the latent representations of such models. These findings
are also backed by a traditional KL-Divergence based metric that operates on
the probability distribution of the model predictions.

  * •

We also analyze the performance of these distilled models under distributional
shift, and demonstrate the adversarial impact of Mixed Sample Augmentation
strategies on the distillation objective.

  * •

We present empirical proof that data augmentation techniques tend to
increasingly make models more discriminative and regularizes on example-
specific features pertinent to the image.

##  2 Related Work

Knowledge Distillation: The idea behind knowledge distillation was first
introduced in 2006 (Buciluǎ et al., 2006), but made popular by (Hinton et al.,
2015) in 2015, as a novel technique for model compression. It works by
teaching a student network to mimic, step by step, the behavior of a larger
network. This process works by first obtaining a more cumbersome teacher model
on the original data. The complex features thus extracted by the teacher model
are then transferred to a simpler model or a model of similar size (Furlanello
et al., 2018) by using softened probability scores from the cumbersome models.
It is important to note that the relative probability of incorrect cases to
the probability of the correct class and each other are useful in
understanding how complex models generalize the data. For example, in the
well-known MNIST dataset, it is helpful to understand not only that the
correct classification is a 2, but also which of these 2’s look like 3’s and
the 2’s that look like 7’s. Based on this idea, several improved distillation
techniques have been proposed (Tarvainen & Valpola, 2017; Li et al., 2017; Xie
et al., 2019; Heo et al., 2019) that address different aspects of improving
distillation performance. More recent work attempts to find the crucial
aspects that determine the quality of a distilled model (Phuong & Lampert,
2019).

Regularization: The ability of the cumbersome model to learn the nuances in
probability can be seen as the model’s ability to generalize the data.
Generalization, can be interpreted as the capacity of a model to adapt to new,
unseen data, drawn from the same distribution the model was trained on.
Several works analyse the generalization performance of the numerous implicit
regularization methods in Deep Learning. Moreover, several other papers
propose different techniques of introducing noise to improve generalization
performance of highly parameterized neural networks, (Srivastava et al.,
2014b; Blundell et al., 2015; Ioffe & Szegedy, 2015; Vapnik, 2000). Some other
works also analyse the relationship between certain qualities of a learned
model and the measured performance metrics (Neyshabur, 2017; Tsipras et al.,
2018; Peterson et al., 2019). Knowledge distillation as a technique for model
compression is unique due to its similarity to human learning, and is thus an
interesting downstream task to analyze more about a neural networks ability to
generalize. A lot of research has been done to improve the generalization of
the student by re-defining a novel teacher architecture, such as adding
sequence-level techniques and ensemble teachers (Simard et al., 1996).
However, this work is more interesting in exploring the impact of
generalization techniques meant to enhance the existing teacher architecture.
Some recent works attempt to explain the relationship between implicit
regularization techniques and knowledge distillation. (Müller et al., 2019)
used a similar approach to ours while using a novel latent representation
strategy to model the adversarial impact of label smoothing on distillation,
while (Arani et al., 2019) analyse the beneficial impact of using trial-to-
trial variability during distillation. (Cho & Hariharan, 2019) analyse the
impact of Early Stopping on Distillation, but none refer to data augmentation
strategies. Specific to Data Augmentation techniques, there have been attempts
to explain the latent effect of data augmentation using mathematical
formulations, such as in (Chen et al., 2019; Thulasidasan et al., 2019; He et
al., 2019), and for Mixed Sample Data Augmentation Techniques in (Harris et
al., 2020). In the following sections, we describe the experimental setup used
in this paper, and then attempt to explain the interesting results obtained.

##  3 Experimental Setup

In this section, we will describe in detail the experimental setup used in
this paper, ranging from the VRM techniques considered for the experiments, to
the generalization measures we used to evaluate the student and teacher
networks.

###  3.1 Comparison Methods

The principle behind VRM was introduced in (Vapnik, 2000), and has found its
application in Machine Learning model training via several different tools.
Using the standard Empirical Risk Minimization principle, the loss objective
is optimized only on the training samples, whereas in VRM, virtual data points
are also sampled from the vicinity of the real data points. Thus, whereas ERM
can be thought of as minimizing the expectation of the loss objective with
respect to an available empirical distribution $P_{emp}(\mathbf{x},y)$, VRM
can be thought of as a natural improvement where the density estimates on each
sample is replaced by some estimate of the density in the neighborhood of the
sample. Thus, the optimization objective is now regularized by an uncertainty
estimate in the sampling task as follows:

$R_{vicinal}(f)=\frac{1}{n}\sum_{i=1}^{n}\int l(f(\mathbf{x}),y)dP_{x_{i}}(x)$

The following augmentation strategies can be thought of as extended VRM
techniques and are used in the experiments to follow. These strategies are
represented pictorially in 2.

Table 1: Performance of the different Teacher and Student Models on CIFAR-10 Test Set. The KLD Metric is the distance between Human labeled confidence scores and Model prediction probabilities. Expected Calibration Error(ECE) measures prediction quality | Teacher | LeNet Student | AlexNet Student
---|---|---|---
Models | Accuracy | KLD | ECE | Accuracy | KLD | ECE | Accuracy | KLD | ECE
Baseline | 0.852 | 0.656 | 0.205 | 0.652 | 1.002 | 0.15 | 0.769 | 0.710 | 0.149
Augment | 0.940 | 0.466 | 0.255 | 0.631 | 0.951 | 0.070 | 0.777 | 0.735 | 0.210
Cutout | 0.880 | 0.220 | 0.237 | 0.644 | 0.987 | 0.109 | 0.785 | 0.726 | 0.249
Mixup | 0.868 | 0.641 | 0.170 | 0.633 | 0.991 | 0.033 | 0.714 | 0.836 | 0.130
Cutmix | 0.954 | 0.524 | 0.252 | 0.621 | 0.987 | 0.28 | 0.720 | 0.776 | 0.062
Table 2: Performance on CINIC-Imagenet Test Set.

Models | Teacher | LeNet-Accuracy | AlexNet-Accuracy
---|---|---|---
Baseline | 0.600 | 0.457 | 0.544
Augment | 0.687 | 0.439 | 0.549
Cutout | 0.630 | 0.451 | 0.555
Mixup | 0.614 | 0.444 | 0.498
Cutmix | 0.716 | 0.439 | 0.499

Standard Transformations: Data Augmentation techniques such as flipping,
splitting, scaling, rotating, cropping, etc. are some of the most common
techniques to augment data for image classification techniques and find their
use in several state of the art results. While general data augmentation leads
to improvement in generalization by creating several invariant features, it is
important to note that these augmentations are data set dependent and require
domain expertise. In this work, a few traditional data augmentation techniques
are tested, that include random cropping and random flipping along the
horizontal axis.

Cut Out: Cut Out is a generalization technique inspired by the Dropout
regularization technique (Srivastava et al., 2014a). While Dropout has been
shown to improve generalization of the model, it blindly reduces the capacity
of the model by adding in sensitive hyper-parameters (Hernandez-Garcia &
Konig, 2019). Therefore, Cut Out works as a data augmentation technique,
pulling from previous works that show that dropping continuous areas has shown
improvement in generalization (Golnaz Ghiasi, 2018). A selected number of
randomly sized continuous sections are removed from the image to create a
modified image for training, as given by (DeVries & Taylor, 2017).

Mix Up: A relatively newer strategy to implement data augmentation is to
create a convex combination of data samples and their label. The idea was
first introduced in (Zhang et al., 2017) and the virtual data-label pair can
be generated as follows:

$\hat{x}=\lambda x_{i}+(1-\lambda)x_{j}$

$\hat{y}=\lambda y_{i}+(1-\lambda)y_{j}$

where $\lambda\sim\beta(\alpha,\alpha)$ is the mixture percentage of each
image. This creates a target value that is a mix of the two original target
values. By using an implicit bias that linear interpolations of data should
lead to predictions that are linearly interpolated in the target space, Mix Up
enables generation of well-calibrated models whose generalization performance
is slightly better (Thulasidasan et al., 2019).

Cut Mix: Introduced by (Yun et al., 2019), Cut Mix was inspired by
augmentation techniques that blend the classes of images (Yuji Tokozume,
2018), like Mix Up, and techniques that cut out regions of images, like Cut
Out. The new virtual sample is given as:

$\hat{x}=\mathbf{M}\odot x_{i}+(1-\mathbf{M})\odot x_{j}$

$\hat{y}=\lambda y_{i}+(1-\lambda)y_{j}$

where $\mathbf{M}$ is a binary mask that contains the information of where to
drop and fill the image, $\odot$ denotes element-wise multiplication and
$\lambda\sim\beta$(1, 1) is the combination ratio. The target label is
generated similarly to Mix Up, but the authors claim that it improves upon
both Cut Out and Mix Up by not removing informative pixels and generalizing
over more natural samples.

We understand that none of these techniques can be truly considered to be pure
VRM techniques in the absolute sense, but only capture the essence of VRM.
However, these techniques are some of the most popular regularization
techniques to be found in the state of the art results for most Deep Learning
tasks. Moreover, a lot of recent work attempts to understand the qualitative
abilities of such techniques (He et al., 2019; Gontijo-Lopes et al., 2020)

###  3.2 Knowledge Distillation

As described above, we attempt to understand the impact of these augmentation
strategies in a simple Knowledge distillation setup, where a smaller model is
trained by forcing it to mimic the tempered probability distributions of a
larger, cumbersome model. When neural networks use a softmax output layer to
convert the logit $z_{i}$ for an example $x_{i}$ into a probability score
$p_{i}$, we can soften it using a temperature parameter $T$ in the following
manner:

$p_{i}=\frac{exp(z_{i}/T)}{\sum_{j}^{N}exp(z_{j}/T)}$

One can conveniently use unlabeled data to train a smaller model, by
completely depending on the cumbersome model’s softened outputs, or use label
information, if available while using the following loss formulation using
$\tau$ as the temperature parameter.

$\mathcal{L}(\mathbf{x},y)=(1-\alpha)\mathcal{L}_{CE}(f_{S}(\mathbf{x}),y)+\alpha\tau^{2}\mathcal{D}_{KL}(f_{S}^{\tau}(\mathbf{x}),f_{T}^{\tau}(\mathbf{x}))$

###  3.3 Evaluation

After having trained a set of teacher and student models using the above
mentioned augmentation and distillation strategy, it is important to set up a
well-defined and intuitive evaluation strategy that can explain the impairment
in transfer of distilled knowledge.

####  3.3.1 Test Datasets

A key aspect of measuring generalization performance is to analyse performance
of the models on not only unseen data, but also test data with some natural
variations in it. For this, we use the following data sets to analyse the
performance of the models trained on CIFAR-10.

CIFAR-10: This is used to measure the performance of the model on unseen data
lying within the seen distribution.

CINIC-10: This is used for the out-of-sample generalization test. This data
set is collected by (Darlow et al., 2018) and contains both CIFAR-10 and
ImageNet images in its test fold. However, we just use the 70,000 ImageNet
images that have been bucketized into CIFAR-10 classes.

Figure 3: Average human confidence distribution across ground truth classes in
CIFAR-10. Diagonal elements, that have the highest confidence have been masked
to reveal implicit patterns between different classes

CIFAR-10H: This is called the Human Labeled CIFAR Test Set which is collected
by (Peterson et al., 2019) and contains the exact same images as the CIFAR-10
images, but instead of one-hot encoded target labels, this dataset provides
the original probability distributions based on the labelings provided by the
human annotators. This enables us to measure the closeness of a model’s
predictions with human beliefs about a data sample.

###  3.4 Generalization Measures

We evaluate each model on the basis of standard confusion metrics, such as
Accuracy, F-1 Score, Precision and Recall. However, accuracy ignores the
probability assigned to the predicted class label as well as the probabilities
assigned to the incorrect classes. This information is key to understanding
the degree of generalization that a model has achieved. Since we have access
to the probability distributions for each CIFAR test sample, we can easily
compare a model’s softmax score vector with the human-labeled distribution. To
better understand this, note that in Figure 3, which represents the average
confidence scores that humans have in each class, certain implicit patterns
exist, that reveal a more generalized story for a prediction. For instance,
humans are more likely to confuse a dog with a cat than a car. This
generalization should also exist in the predicted models. This difference in
model prediction and human-level confidence can be easily measured by a KL-
Divergence between the model prediction and human-labeled scores, as
formulated below, where $p_{f}(x_{i})$ represents the model output and
$p_{h}(x_{i})$ represents the human labeled confidence scores.

$\mathcal{L}_{CE}=\sum_{i=1}^{N}\mathcal{D_{KL}}(\mathbf{p}_{f}(x_{i}),\mathbf{p}_{h}(x_{i}))$

In essence, this is nothing but the cross-entropy loss between the model
predictions and human labels. However, it would also be interesting to note
the KL-Divergences between different classes. Since, it is not feasible to do
that on a sample-level, it would be interesting to compute the Divergences
between the averaged probability distribution for each ground truth class for
both the model prediction scores and human labeled scores. To understand
quality of predictions, we also plot reliability diagrams of the different
models and compare the model calibration using Expected Calibration Error,
which is given as follows:

$ECE=\sum_{m=1}^{M}\frac{|B_{m}|}{N}|acc(B_{m})-conf(B_{m})|$

We also propose a novel metric to explain the discriminative power of the
different models we train. This operates on the penultimate layer embeddings
generated by the model and can be thought of as a measure of how well-
separated the embedding manifold is. It takes into account both the intra-
class and the inter-class similarity and defines the discriminative power of
the model as the difference between them. If intra-class similarity is high,
class representations are cohesive and more compressed. If inter-class
similarity is low, class representations are less adhesive and are far away
from each other. An optimal classifier will tend to have high cohesivity and
low adhesion, and thus higher discriminative power. To compute this metric, we
first standardize the embeddings, and define the cohesion and adhesion metrics
as inter- and intra-class similarities, respectively using a cosine similarity
function $S$.

$C^{(i)}=\frac{1}{N_{C}(N_{C}-1)}\sum_{i=1}^{N_{C}-1}\sum_{j=i+1}^{N_{C}}S(d_{i},d_{j})$
$A^{(i,j)}=\frac{1}{N_{1}N_{2}}\sum_{i=1}^{N_{1}}\sum_{j=1}^{N_{2}}S(d_{i},d_{j})$

Where, $N_{k}$ represents the number of instances in Class $k$. Thus, the
class-discrimination can then be computed as:

$D=\frac{1}{\sqrt{d}}[\frac{1}{K}\sum_{i=1}^{K}C^{(i)}-\frac{2}{K(K-1)}\sum_{i=1}^{K-1}\sum_{j=i+1}^{K}A^{(i,j)}]$

Where, $K$ represents the total number of classes and $d$ represents the
dimensionality of the embeddings.

(a) Baseline

(b) CutMix

(c) Cutout

(d) Augmentation

Figure 4: Latent Space Visualization from Penultimate Layer, dimensionally
reduced using tSNE. Note the well-separated class representations in the
augmented models as compared to the baseline model

##  4 Experiments

Five unique experimental setups were created using the framework and
evaluation techniques from above. Teacher models were trained using
augmentation techniques and distilled into the student models. This was
evaluated with different complexity student models, and an additional data set
for comparison. Additionally, we also performed a small ablation study, where
we compared the effect of our augmentation techniques on each step of the
knowledge distillation process.

The models were implemented in PyTorch, with different parameter values for
the different augmentation techniques. Knowledge distillation was performed
with a temperature value of 20 and gamma of 0.5. When performing augmentation,
baseline and standard data augmentation did not require explicit parameters.
Mix Up was implemented with a beta value of 1, and Cut Out was implemented
with 16 holes to be randomly places in the images. Finally, Cut Mix used alpha
and beta parameters of 0.5 and 1, respectively. These values were chosen based
on original paper implementation of the techniques, and could be fine tuned in
later work.

###  4.1 CIFAR 10 - Augmented ResNet Distilled to LeNet

####  4.1.1 SetUp

Five ResNet18 models (Kaiming He, 2015) were trained using the data
augmentation techniques discussed above, on the CIFAR-10 training dataset. The
student networks were built from the LeNet architecture (Y. LeCun & Haffner,
1998) with no data augmentation techniques added.

####  4.1.2 Results

As can be seen in 1, the baseline student model out performed other models in
terms of accuracy. The baseline model also outperformed all other student
models in terms of Precision, Recall and F1 Score on the CIFAR-10 and CINIC
data set, which can be found in the appendix 6.

###  4.2 CIFAR 10 - Augmented ResNet Distilled to AlexNet

####  4.2.1 Set Up

Another experiment distilled the ResNet18 teacher networks, from above, into a
more complex student architecture, to evaluate if it’s increased complexity
would better capture the nuance of the Teacher network generalization. The
architecture chosen was the AlexNet (Krizhevsky et al., 2012).

####  4.2.2 Results

This experiment emphasized the same trend from the previous experiment that
enhanced generalization did not lead to equally enhanced distillation, as can
be seen in 1, 6. However, it is important to note that the augmentation
techniques based on assumed sampling distance, such as standard augmentation
and Cut Out, outperformed the baseline model. While, augmentation techniques
that incorporate VRM sampling, Mix Up and Cut Mix, still under-performed the
baseline teacher-student model. It is of interest that possibly the larger
AlexNet model was able to pick up on more nuance. Additionally, we propose
that this difference between more standard augmentation and Mix Sample
augmentation techniques is related to the varying interpolative nature of the
techniques. This finding will be explored and further evaluated in the next
section.

###  4.3 MNIST - Augmented AlexNet Distilled to LeNet

####  4.3.1 Set Up

This work also used the MNIST data set to validate these findings on another
data set. For this validation, we just compared the baseline ERM technique
with our Mixed Sample augmentation techniques - Mix Up and Cut Mix. This was
due to the fact that the standard augmentation techniques as we defined them
have been shown to inhibit MNIST training (He et al., 2019). We trained three
teacher networks using the AlexNet architecture on the MNIST data using the
augmentation techniques. Information from these models was then distilled into
baseline LeNet models, which were then evaluated.

####  4.3.2 Results

The results can be seen in the table below. While the accuracies were very
close to one another, it is shown that the validation loss shows more
significant decrease for the baseline model. This validation was done with
fairly simple models and data set, and it would be interesting to further
explore this concept with more complex models and data sets for further work.

Table 3: Performance of the different Teacher and Student Models on MNIST Test
Set.

| AlexNet Teacher | LeNet Student
---|---|---
Models | Accuracy | Loss | Accuracy | Loss
Baseline | 0.9958 | 0.020 | 0.9868 | 0.039
Mixup | 0.9960 | 0.010 | 0.9886 | 0.074
Cutmix | 0.9938 | 0.104 | 0.9861 | 0.107

###  4.4 CIFAR 10 - Ablation Study - ResNet distilled to LeNet

####  4.4.1 Set Up

To further explore the effect of data augmentation on distillation, we
evaluated how augmentation impacted each step of the distillation process.
This evaluation was done with ResNet18 teachers and LeNet students, with one
of the outcomes (Augmented Teacher - Student) being the first experiment
described above. We also evaluated the set ups of a Baseline ResNet teacher
distilled into LeNet students trained with augmentation techniques. Finally,
we evaluated augmented ResNet teachers distilled into augmented LeNet
students.

(a) Teacher Model

(b) AlexNet Student Models

(c) LeNet Student Models

Figure 5: Reliability curves for different models. It is observed that data
augmentation generates calibrated predictions

####  4.4.2 Results

This study showed that no matter the set up, data augmentation techniques hurt
the performance of the student models. We suspect that much of this has to do
with the interpolation of the latent space by the augmentation techniques.
These results can be seen in 6, 8, 7. In our previous experiments, we found
that the teacher manifold is the key driver in distillation, so it is no
surprise that the techniques with augmented teachers performed poorly.
However, it is interesting that adding augmentation to the student networks
also inhibited distillation. However, more work may need to be done with more
complex student networks to further understand the impact on augmentation in
student networks.

##  5 Explaining the Dichotomy

It is clear from Table 1 and from Figure 1 that when simpler models are
distilled from cumbersome models trained with Data Augmentation,
generalization performance is generally impaired. This is more pronounced for
mixed sample data augmentation strategies like Mix Up and Cut Mix. This
reversal of behavior is quite counter-intuitive, as we expect better
generalized teacher models to transfer their capabilities to their respective
students. We try to explain this in the following ways, with varying degrees
of success.

Latent Space Representations: We visualize the latent representations
generated under the influence of these different augmentation strategies. The
teacher model representations as seen in 4 depict a clearer picture that can
help explain this reversal in performance. We plot the tSNE-reduced
embeddings(Maaten & Hinton, 2008) from the penultimate layer of the teacher
ResNet models for four different classes in the CIFAR-10 Test Set. We chose
two distinct pairs of semantically different classes. Shown in orange and
green are two semantically similar classes, that represent vehicles(cars and
trucks), whereas shown in blue and red are another pair of semantically
similar classes (cat and bird), which are different from vehicles. This
combination enables us to analyse relationships better, as a well-generalized
model should be able to not only form correct clusters, but cluster similar
classes near one another. Plotting all 10 classes might lead to
undistinguishable inter-class distances, and so we focus on these
representations. We note that for Cut Mix that uses a linear interpolation of
both images and labels, the clusters are much more compressed and each
semantic group lies much farther away from each other than any other strategy.
Moreover, there is little interaction between the classes, as not a lot of
points lie between the clusters. On the other hand, the baseline model
manifold presents itself as much more uniform, wherein a lot of points lie on
the boundaries of the class clusters and there is a gradual change in
representational capability of the model between the classes. The two
different semantic groups are closer in the baseline, when compared to any
other augmentation strategy. This separability between classes and semantic
groups exists in Cutout and transformation based Augmentations, but is not as
pronounced as the interpolative Cut Mix.

Class-Separability: To quantify the behavior observed in the latent space
visualiztions, we propose the use of a very basic class-separability metric
that simply measures the difference in a model’s confidence structure for
different classes. In an ideal scenario where the model is supremely confident
of each image, each class’ confidence distribution is well separated and this
can theoretically happen if a model is let to run for long enough using
gradient descent algorithm. But, in real scenarios, models also assign
probabilities to incorrect classes and as claimed by (Hinton et al., 2015),
this relative probability structure is crucial in the generalization
performance during distillation. The mathematical formulation of this class-
separability metric can be written as:

$S_{f}=\frac{1}{C^{2}}\sum_{i=1}^{C}\sum_{j=1}^{C}\mathcal{D_{KL}}(p_{f,i},p_{f,j})$

Where, $p_{f,k}\in\mathrm{R}^{C}$ represents the average model prediction
distribution for the class $k$, and $C$ represents the number of classes.

Table 4: Class Separability Score $S_{f}$

Models | Teacher | LeNet | AlexNet
---|---|---|---
Baseline | 3.73 | 3.21 | 3.07
Augment | 5.19 | 3.14 | 3.23
Cutout | 4.30 | 3.14 | 3.41
Mixup | 5.32 | 2.92 | 2.17
Cutmix | 5.33 | 2.89 | 2.27

Class Discrimination: Accuracy Performance can be a deceptive metric to
evaluate generalization performance, as it depends on the one-hot
representation of a prediction probability distribution. Similarly, the
Similarity Score formulation we use above can be made even more robust by
considering the latent space embeddings instead of the probability structure.
We develop a novel metric, while taking inspiration from Discriminant Analysis
techniques that help explain the model behavior in a better fashion with the
current setup. This metric, called class-discrimination has been explained in
Section 3.4 and involves a joint optimization of both intra-class similarity
and inter-class similarity objectives in a latent space. We find the class-
discrimination, intra-class similarity, and inter-class similarity metrics for
the different models and find that data augmentation acts as a regularizer by
creating more discriminant models. In simpler terms, a model with loose, yet
well-separated(low intra-class similarity, low inter-class similarity) class
representations will make a better teacher than a model with comparatively
tighter and well-separated class representations(high intra-class similarity,
low inter-class similarity), while not losing out on original accuracy
performance.

Table 5: Class Discrimination Evaluation for Teacher Models

Metrics | Cohesion | Adhesion | Discrimination
---|---|---|---
Baseline | 0.739 | -0.041 | 0.246
Augment | 0.886 | -0.049 | 0.296
Cutout | 0.783 | -0.043 | 0.261
Mixup | 0.793 | -0.042 | 0.255
Cutmix | 0.917 | -0.050 | 0.306

As is evident from the above set of analysis, we can attempt to explain the
adversarial impact of such augmentation strategies on distillation as that of
the discriminative powers that these models gain during training. This is
consistent with the ideas presented in (He et al., 2019), wherein data
augmentation strategies are believed to be regularization strategies that
focus more on class-specific major features, while regularizing the example
specific minor features. This regularization is more pronounced in
interpolative techniques like Cut Mix and Mix Up as we believe they don’t
really add any new information to the model. By attempting to combine class-
specific labels and corresponding images, they attempt to regularize on finer,
example specific information and generate overall class-specific concepts as
features. This is great for the model’s performance on test data and data from
a slightly different distribution, but distillation quality depends largely on
the amount of information encoded in all the latent features encoded by the
teacher model. Thus, if the generated features within any given class have
greater variance, they are able to encode more information about the class’
relationship with other classes and are expected to generate more generalized
probability distributions. This is a key factor in generating better quality
students. This also helps explaining the superior performance of Cutout and
Augmentation that add new information by retaining the same label and
transforming the image.

(a) Baseline Teacher

(b) CutMix Teacher

Figure 6: Relative KL-Divergence Confusion Matrix

Figure 7: Expected Calibration Error for the different Models. Smaller models
like LeNet usually have better calibration performance and previous studies
also show that linear interpolation of data samples always leads to better
model calibration

Prediction Quality: To test this idea of generating high quality probability
distributions, we make use of the CIFAR-10 Human labeled datasets, and measure
the quality of the average model confidence distributions against the human
confidence estimates. This can again be represented as a KL-Divergence between
the probability distributions of a model’s average prediction probabilities
for a given class against the average human distribution. We present this
information in a Confusion Matrix like visualization in 6. Each cell
represents the KL-Divergence between the human estimate on that class and the
model’s estimate. The KL-Divergences are all scaled uniformly and are color
coded, so higher values correspond to brighter pixel values. We are more
interested in the non-diagonal elements as they reveal the mutual information
between classes and note that the Cut Mix Matrix is much brighter in those
pixels when compared to the Baseline. This points out the fact that the mutual
information between different classes is better encoded in the Baseline than
Cut Mix-trained model. This enables the creation of a superior representation
manifold for distillation to take place.

Model Calibration: We also measure calibration performance across the several
models, and note that interpolative techniques generate better calibrated
models consistently across teacher and student models. This is evident from
the reliability diagrams in 5 and the Expected Calibration Error metrics in 1
and 7. However, it has been hypothesized in (Guo et al., 2017) that smaller
models like LeNet generally tend to exhibit better Calibration performance
than overparameterized, modern models like ResNet. This trend is visible as
the reliability curves for all models tend to hug the ideal straight line
closer as the model complexity decreases. However, no direct relationship can
be found between the calibration performance of a student and the augmentation
applied on its teacher model.

Model Generalization

Generalization is often defined in machine learning by the model’s ability to
adapt and classify on unseen data. However, with our analysis of the latent
representation space, we found that as these techniques increase the intra-
class discrimination, they may inhibit the ability to create probability
distributions that mimic informative generalization for distillation into a
student model.

We analyzed this concept further through the use of confusion matrices. These
confusion matrices compared the predicted label probabilities and actual
classes. Both the baseline and Mix Up models show that they classify well
overall, with a strong diagonal line in the un-masked confusion matrices. When
that diagonal is masked,however, we can see that the probabilities between
classes not chosen are far more spread apart in the baseline model. These can
be compared to 3, which visualized the human generated predicted label
probabilities.

It is important to note that these matrices have different color scales, with
the Masked Baseline going up to .1 and the Masked Cut Mix ranging to 0.07. The
matrices show that they are both able to pick up on some of the nuances
between data, but it can be seen that the masked baseline shows more
generalization between classes with brighter colors between related classes.
This ability to create an informative probabilistic distribution between
different class types is the backbone for effective knowledge distillation.

##  6 Future Work

Moving ahead, a quantification of the regularization effect of mixed data
augmentation strategies like Cut Mix and Mix Up should be formulated. An
extension of this work would be to develop a novel augmentation strategy that
is able to retain or improve the latent embedding manifold qualities for the
given data set, and consequently, generate more generalized model predictions.
A key missing aspect in this study is the fact that the training data set used
to distill student models is the same as the one used to train the teacher
models. A set of experiments could be conducted on a held out distillation
data set. Another aspect of measuring true generalization performance could be
the relative performance of these models under adversarial attack, and would
be an interesting metric to analyse.

## References

  * Arani et al. (2019) Arani, E., Sarfraz, F., and Zonooz, B.  Improving generalization and robustness with noisy collaboration in knowledge distillation, 2019.
  * Blundell et al. (2015) Blundell, C., Cornebise, J., Kavukcuoglu, K., and Wierstra, D.  Weight uncertainty in neural networks, 2015.
  * Buciluǎ et al. (2006) Buciluǎ, C., Caruana, R., and Niculescu-Mizil, A.  Model compression.  In _Proceedings of the 12th ACM SIGKDD international conference on Knowledge discovery and data mining_ , pp. 535–541, 2006.
  * Chen et al. (2019) Chen, S., Dobriban, E., and Lee, J. H.  A group-theoretic framework for data augmentation, 2019.
  * Cho & Hariharan (2019) Cho, J. H. and Hariharan, B.  On the efficacy of knowledge distillation.  In _2019 IEEE/CVF International Conference on Computer Vision (ICCV)_ , pp. 4793–4801, 2019.
  * Darlow et al. (2018) Darlow, L. N., Crowley, E. J., Antoniou, A., and Storkey, A. J.  Cinic-10 is not imagenet or cifar-10, 2018.
  * DeVries & Taylor (2017) DeVries, T. and Taylor, G. W.  Improved regularization of convolutional neural networks with cutout, 2017\.
  * Furlanello et al. (2018) Furlanello, T., Lipton, Z. C., Tschannen, M., Itti, L., and Anandkumar, A.  Born again neural networks, 2018.
  * Golnaz Ghiasi (2018) Golnaz Ghiasi, Tsung-Yi Lin, Q. V. L.  Dropblock: A regularization method for convolutional networks, 2018.
  * Gontijo-Lopes et al. (2020) Gontijo-Lopes, R., Smullin, S. J., Cubuk, E. D., and Dyer, E.  Affinity and diversity: Quantifying mechanisms of data augmentation, 2020\.
  * Guo et al. (2017) Guo, C., Pleiss, G., Sun, Y., and Weinberger, K. Q.  On calibration of modern neural networks, 2017.
  * Han et al. (2015) Han, S., Mao, H., and Dally, W. J.  Deep compression: Compressing deep neural networks with pruning, trained quantization and huffman coding, 2015.
  * Harris et al. (2020) Harris, E., Marcu, A., Painter, M., Niranjan, M., Prügel-Bennett, A., and Hare, J.  Understanding and enhancing mixed sample data augmentation, 2020.
  * He et al. (2019) He, Z., Xie, L., Chen, X., Zhang, Y., Wang, Y., and Tian, Q.  Data augmentation revisited: Rethinking the distribution gap between clean and augmented data, 2019.
  * Heo et al. (2019) Heo, B., Lee, M., Yun, S., and Choi, J. Y.  Knowledge distillation with adversarial samples supporting decision boundary.  _Proceedings of the AAAI Conference on Artificial Intelligence_ , 33:3771–3778, Jul 2019.  ISSN 2159-5399.  doi: 10.1609/aaai.v33i01.33013771.  URL http://dx.doi.org/10.1609/aaai.v33i01.33013771.
  * Hernandez-Garcia & Konig (2019) Hernandez-Garcia, A. and Konig, P.  Data augmentation instead of explicit regularization, 2019.
  * Hinton et al. (2015) Hinton, G., Vinyals, O., and Dean, J.  Distilling the knowledge in a neural network, 2015.
  * Ioffe & Szegedy (2015) Ioffe, S. and Szegedy, C.  Batch normalization: Accelerating deep network training by reducing internal covariate shift, 2015.
  * Jonathan Frankle (2018) Jonathan Frankle, M. C.  The lottery ticket hypothesis: Finding sparse, trainable neural networks, 2018.
  * Kaiming He (2015) Kaiming He, Xiangyu Zhang, S. R. J. S.  Deep residual learning for image recognition, 2015.
  * Krizhevsky et al. (2012) Krizhevsky, A., Sutskever, I., and Hinton, G. E.  Imagenet classification with deep convolutional neural networks.  In Pereira, F., Burges, C. J. C., Bottou, L., and Weinberger, K. Q. (eds.), _Advances in Neural Information Processing Systems 25_ , pp. 1097–1105. Curran Associates, Inc., 2012.  URL http://papers.nips.cc/paper/4824-imagenet-classification-with-deep-convolutional-neural-networks.pdf.
  * Li et al. (2017) Li, Y., Yang, J., Song, Y., Cao, L., Luo, J., and Li, L.-J.  Learning from noisy labels with distillation.  _2017 IEEE International Conference on Computer Vision (ICCV)_ , Oct 2017.  doi: 10.1109/iccv.2017.211.  URL http://dx.doi.org/10.1109/ICCV.2017.211.
  * Maaten & Hinton (2008) Maaten, L. v. d. and Hinton, G.  Visualizing data using t-sne.  _Journal of machine learning research_ , 9(Nov):2579–2605, 2008.
  * Müller et al. (2019) Müller, R., Kornblith, S., and Hinton, G.  When does label smoothing help?, 2019.
  * Neyshabur (2017) Neyshabur, B.  Implicit regularization in deep learning.  _arXiv preprint arXiv:1709.01953_ , 2017.
  * Peterson et al. (2019) Peterson, J., Battleday, R., Griffiths, T., and Russakovsky, O.  Human uncertainty makes classification more robust.  _2019 IEEE/CVF International Conference on Computer Vision (ICCV)_ , Oct 2019.  doi: 10.1109/iccv.2019.00971.  URL http://dx.doi.org/10.1109/ICCV.2019.00971.
  * Phuong & Lampert (2019) Phuong, M. and Lampert, C.  Towards understanding knowledge distillation.  In _International Conference on Machine Learning_ , pp. 5142–5151, 2019.
  * Simard et al. (1996) Simard, P. Y., LeCun, Y., Denker, J. S., and Victorri, B.  Transformation invariance in pattern recognition-tangent distance and tangent propagation.  In _Neural Networks: Tricks of the Trade_ , 1996.
  * Srivastava et al. (2014a) Srivastava, N., Hinton, G., Krizhevsky, A., Sutskever, I., and Salakhutdinov, R.  Dropout: A simple way to prevent neural networks from overfitting.  _Journal of Machine Learning Research_ , 15(56):1929–1958, 2014a.  URL http://jmlr.org/papers/v15/srivastava14a.html.
  * Srivastava et al. (2014b) Srivastava, N., Hinton, G. E., Krizhevsky, A., Sutskever, I., and Salakhutdinov, R.  Dropout: a simple way to prevent neural networks from overfitting.  _J. Mach. Learn. Res._ , 15:1929–1958, 2014b.
  * Tarvainen & Valpola (2017) Tarvainen, A. and Valpola, H.  Mean teachers are better role models: Weight-averaged consistency targets improve semi-supervised deep learning results, 2017.
  * Thulasidasan et al. (2019) Thulasidasan, S., Chennupati, G., Bilmes, J., Bhattacharya, T., and Michalak, S.  On mixup training: Improved calibration and predictive uncertainty for deep neural networks, 2019.
  * Tsipras et al. (2018) Tsipras, D., Santurkar, S., Engstrom, L., Turner, A., and Madry, A.  Robustness may be at odds with accuracy, 2018.
  * Vapnik (2000) Vapnik, V. N.  The vicinal risk minimization principle and the svms.  In _The nature of statistical learning theory_ , pp. 267–290. Springer, 2000.
  * Xie et al. (2019) Xie, Q., Luong, M.-T., Hovy, E., and Le, Q. V.  Self-training with noisy student improves imagenet classification, 2019\.
  * Y. LeCun & Haffner (1998) Y. LeCun, L. Bottou, Y. B. and Haffner, P.  Gradient-based learning applied to document recognition.  _Proceedings of the IEEE_ , November 1998.
  * Yuji Tokozume (2018) Yuji Tokozume, Yoshitaka Ushiku, T. H.  Between-class learning for image classification, 2018.
  * Yun et al. (2019) Yun, S., Han, D., Chun, S., Oh, S. J., Yoo, Y., and Choe, J.  Cutmix: Regularization strategy to train strong classifiers with localizable features.  _2019 IEEE/CVF International Conference on Computer Vision (ICCV)_ , Oct 2019.  doi: 10.1109/iccv.2019.00612.  URL http://dx.doi.org/10.1109/ICCV.2019.00612.
  * Zhang et al. (2017) Zhang, H., Cisse, M., Dauphin, Y. N., and Lopez-Paz, D.  mixup: Beyond empirical risk minimization, 2017.
  * Zhou et al. (2017) Zhou, Z., Mertikopoulos, P., Bambos, N., Boyd, S., and Glynn, P.  On the convergence of mirror descent beyond stochastic convex programming, 2017.

##  Appendix A Appendix Figures

Table 6: Appendix Table: All Student Metrics trained from ResNet18 CIFAR Teachers | Model | Test Set | Accuracy | Precision | Recall | F1-Score
---|---|---|---|---|---|---
Baseline | LeNet | CIFAR-10 | 0.652 | 0.648 | 0.652 | 0.650
CINIC | 0.456 | 0.452 | 0.457 | 0.449
AlexNet | CIFAR-10 | 0.768 | 0.767 | 0.769 | 0.768
CINIC | 0.544 | 0.550 | 0.544 | 0.529
Augmentation | LeNet | CIFAR-10 | 0.631 | 0.632 | 0.631 | 0.474
CINIC | 0.439 | 0.434 | 0.435 | 0.435
AlexNet | CIFAR-10 | 0.777 | 0.776 | 0.777 | 0.766
CINIC | 0.549 | 0.557 | 0.549 | 0.535
Cut Out | LeNet | CIFAR-10 | 0.644 | 0.643 | 0.644 | 0.643
CINIC | 0.451 | 0.447 | 0.451 | 0.445
AlexNet | CIFAR-10 | 0.785 | 0.783 | 0.785 | 0.784
CINIC | 0.554 | 0.562 | 0.554 | 0.540
Mix Up | LeNet | CIFAR-10 | 0.633 | 0.631 | 0.633 | 0.632
CINIC | 0.444 | 0.435 | 0.443 | 0.433
AlexNet | CIFAR-10 | 0.714 | 0.712 | 0.714 | 0.712
CINIC | 0.497 | 0.501 | 0.498 | 0.477
Cut Mix | LeNet | CIFAR-10 | 0.621 | 0.617 | 0.621 | 0.619
CINIC | 0.439 | 0.434 | 0.439 | 0.435
AlexNet | CIFAR-10 | 0.720 | 0.717 | 0.720 | 0.718
CINIC | 0.498 | 0.502 | 0.498 | 0.478

Table 7: Appendix Table: All Baseline Teacher with Augmented Student Test Metrics trained from CIFAR-10 | Model | Test Set | Accuracy | Precision | Recall | F1-Score
---|---|---|---|---|---|---
Baseline | LeNet | CIFAR-10 | 0.652 | 0.648 | 0.652 | 0.650
CINIC | 0.456 | 0.452 | 0.457 | 0.449
Augmentation | LeNet | CIFAR-10 | 0.641 | 0.639 | 0.641 | 0.639
CINIC | 0.451 | 0.446 | 0.451 | 0.444
Cut Out | LeNet | CIFAR-10 | 0.635 | 0.630 | 0.635 | 0.632
CINIC | 0.446 | 0.446 | 0.439 | 0.441
Mix Up | LeNet | CIFAR-10 | 0.647 | 0.645 | 0.647 | 0.647
CINIC | 0.457 | 0.455 | 0.457 | 0.452
Cut Mix | LeNet | CIFAR-10 | 0.629 | 0.626 | 0.629 | 0.627
CINIC | 0.439 | 0.433 | 0.439 | 0.434
Table 8: Appendix Table: All ResNet Teacher - Student with same Augmentation Test Metrics trained from CIFAR-10 | Model | Test Set | Accuracy | Precision | Recall | F1-Score
---|---|---|---|---|---|---
Baseline | LeNet | CIFAR-10 | 0.652 | 0.648 | 0.652 | 0.650
CINIC | 0.456 | 0.452 | 0.457 | 0.449
Augmentation | LeNet | CIFAR-10 | 0.641 | 0.637 | 0.641 | 0.638
CINIC | 0.445 | 0.441 | 0.445 | 0.441
Cut Out | LeNet | CIFAR-10 | 0.636 | 0.643 | 0.636 | 0.638
CINIC | 0.440 | 0.451 | 0.440 | 0.440
Mix Up | LeNet | CIFAR-10 | 0.626 | 0.622 | 0.626 | 0.624
CINIC | 0.440 | 0.437 | 0.440 | 0.431
Cut Mix | LeNet | CIFAR-10 | 0.635 | 0.631 | 0.635 | 0.633
CINIC | 0.446 | 0.443 | 0.446 | 0.441
Table 9: Appendix Table: All Teacher Test Metrics trained from CIFAR-10 | Model | Test Set | Accuracy | Precision | Recall | F1-Score
---|---|---|---|---|---|---
Baseline | ResNet | CIFAR-10 | 0.852 | 0.851 | 0.852 | 0.851
CINIC | 0.600 | 0.602 | 0.600 | 0.593
Augmentation | ResNet | CIFAR-10 | 0.940 | 0.940 | 0.940 | 0.940
CINIC | 0.630 | 0.620 | 0.628 | 0.629
Cut Out | ResNet | CIFAR-10 | 0.879 | 0.878 | 0.880 | 0.879
CINIC | 0.629 | 0.620 | 0.628 | 0.630
Mix Up | ResNet | CIFAR-10 | 0.867 | 0.954 | 0.867 | 0.953
CINIC | 0.603 | 0.606 | 0.603 | 0.598
Cut Mix | ResNet | CIFAR-10 | 0.954 | 0.869 | 0.954 | 0.868
CINIC | 0.710 | 0.707 | 0.720 | 0.710


# Generation of quasi-monoenergetic positron beams in chirped laser fields

S. Tang<EMAIL_ADDRESS>Department of Physics, College of Information
Science and Engineering, Ocean University of China, Qingdao, Shandong, 266100,
China

###### Abstract

High energy photons can decay to electron-positron pairs via the nonlinear
Breit-Wheeler process when colliding with an intense laser pulse. The energy
spectrum of the produced particles is broadened because of the variation of
their effective mass in the course of the laser pulse. Applying a suitable
chirp to the laser pulse can narrow the energy distribution of the generated
electrons and positrons. We present a scenario where a high-energy electron
beam is collided with a chirped laser pulse to generate a beam of quasi-
monoenergetic $\gamma$-photons, which then decay in a second chirped, UV pulse
to produce a quasi-monoenergetic source of high-energy electrons and
positrons.

##  I Introduction

When a beam of charged particles collides with an intense laser pulse, the
spectrum of produced photons, via the process referred to as the nonlinear
Compton scattering (NLC) Ritus (1985); Di Piazza et al. (2012), is sensitive
to the shape of the pulse. Employing a many-cycle laser pulse will lead to
outgoing photon spectra similar to those in a monochromatic background
Nikishov and Ritus (1964); King and Tang (2020): well-defined harmonic fringes
in lightfront momenta and emission angle. Collision with short laser pulses
will lead to a broadening of outgoing particle harmonic peaks Boca and
Florescu (2009); Seipt and Kämpfer (2011); Mackenroth and Di Piazza (2011) and
richer spectral structures: infra-red structure Ilderton and MacLeod (2020);
King (2020a), asymmetry in emission angle Krajewska and Kamiński (2012), and
pronounced interference phenomena Wistisen (2014); Ilderton et al. (2020). The
spectral broadening can be attributed to the inhomogeneous effective mass of
the charged particle moving in the intense laser pulse Brown and Kibble
(1964); Kibble (1966); Reiss (2014); Raicher and Hatsagortsyan (2020): the
variation of the effective mass modifies the velocity of the changed particle
during the scattering Hartemann et al. (1996); Heinzl et al. (2010); Albert et
al. (2011); Seipt et al. (2014). It is also known, that if one can prescribe
the chirp of the laser pulse, that is, a nonlinear dependency on the phase,
then this relativistic broadening of particle spectra can be compensated for,
to generate a narrowband source of high energy photons Terzić et al. (2014);
Seipt et al. (2014, 2019).

The decay of a probe photon to an electron-positron pair in an intense
electromagnetic field, is often referred to as the nonlinear Breit-Wheeler
process (NBW) Breit and Wheeler (1934); Reiss (1962); Nikishov and Ritus
(1964), and has been measured experimentally in the landmark E144 experiment
over two decades ago Burke et al. (1997); Bamber et al. (1999). The
phenomenology of the process has been investigated theoretically in various
types of laser field. First in monochromatic Nikishov and Ritus (1964), and
constant crossed fields Nikishov and Ritus (1964); Narozhnyĭ (1969) and more
recently in finite laser pulses Jansen et al. (2016); Wistisen (2020); Titov
et al. (2020), as well as two-colour Jansen and M ller (2013) and double-pulse
fields Jansen and Muller (2017); Titov et al. (2018); Ilderton (2020). As in
NLC process, harmonic structure also arises in the outgoing electron-positron
pair spectra operating in a many-cycle laser pulse. However, this structure is
only clearly discernible when the centre-of-mass energy reaches the threshold
of $2mc^{2}$ already with only a low number of laser photons, where $m$ is
electron (positron) rest mass and $c$ is the speed of light.

Analogous to the spectral broadening in the NLC process, the variation of the
electron-positron pair’s effective mass in the course of the intense laser
pulse also induce a broadening in their energy spectra. The current paper is a
proof of principle calculation to show that a suitable nonlinear chirp of a
laser pulse can also be employed to counterbalance the spectral broadening in
the NBW process, and proposes a simple two-step scenario to provide a quasi-
monoenergetic source positrons. The existence of a quasi-monoenergetic
positrons source would be useful in the electron-positron colliding
experiments Cabibbo and Gatto (1961); Myers and Picasso (1990); Tang et al.
(2019).

The study of laser chirp’s effect on positron spectra, is partly motivated by
upcoming high-energy experiments LUXE at DESY Hartin et al. (2019); Abramowicz
et al. (2019) and E320 at FACET-II Joshi et al. (2018); Meuren (2019), where
photons with energies $\mathcal{O}(10~{}\textrm{GeV})$ are planned to be
generated, either directly in the laser pulse through Compton scattering of
the electrons (LUXE and E320), or from a separated bremsstrahlung and inverse
Compton source (LUXE). The centre-of-mass energy can be effectively increased,
and hence the multi-photon harmonic region of the Breit-Wheeler process
approached, by generating higher-order harmonics of the interaction laser,
using e.g. relativistic plasmas Rödel et al. (2012); Tang et al. (2017); Tang
and Kumar (2019).

The paper is organised as follows. In Sec. II, we present the spectrum of
produced positrons in the NBW process, and investigate the contributions from
the stationary phase points. We then analysis the broadening of the positron
spectrum and propose a special laser frequency chirp to counteract the
spectral broadening. In Sec. III we demonstrate numerical implementations of
our chirp prescription in narrowing the positron spectra from a single high-
energy photon and from the $\gamma$-ray obtained through the NLC process of a
high-energy electron. We conclude in Sec. IV.

##  II Theoretical framework

We consider the scenario in which a high-energy photon with momentum $\ell$
colliding (almost) head-on with a laser pulse produces a pair of electron and
positron. The laser pulse is simplified as a plane wave with scaled vector
potential $a^{\mu}=eA^{\mu}(\phi)$ and wavevector
$k^{\mu}=\omega_{0}(1,0,0,1)$ where $\phi=k\cdot x$ and $\omega_{0}$ is the
laser frequency at the initial phase point $\phi_{i}$ at which the laser is
turned on. The interaction energy is characterised by
$\eta_{\ell}=k\cdot\ell/m^{2}$. We use natural units $\hbar=c=1$ throughout
and the fine structure constant is $\alpha=e^{2}\approx 1/137$.

The angular-resolved spectrum of the produced positron can be formulated as

$\displaystyle\frac{\mathrm{d}^{3}\textrm{P}}{\mathrm{d}s\mathrm{d}^{2}q_{{\scriptscriptstyle\perp}}}$
$\displaystyle=\alpha\frac{|I|^{2}+\left(SI^{*}+IS^{*}-2F\cdot
F^{*}\right)g}{(2\pi)^{2}\eta^{2}_{\ell}(1-s)s}\,,$ (1)

where $g\equiv[s^{2}+(1-s)^{2}]/[4s(1-s)]$. We sum over the spin of the
outgoing particles and average over the polarisation of the incoming photon.
The spectrum (1) is parameterised by the three components of the positron
momentum $p$: these are $s=k\cdot p/k\cdot\ell$, the fraction of the photon
light-front momentum taken by the positron, and
$q_{{\scriptscriptstyle\perp}}=(q_{x},q_{y})$,
$q_{x,y}=p_{x,y}/m-s\ell_{x,y}/m$, positron momenta in the plane perpendicular
to the laser propagating direction. $q_{{\scriptscriptstyle\perp}}$ reflects
the angular spread of the produced positron around the photon incident
direction. In our parameter region, the angular spread is extremely narrow,
and for head-on collisions $\ell_{{\scriptscriptstyle\perp}}=0$,
$q_{{\scriptscriptstyle\perp}}\approx\gamma_{p}\theta_{p}(\cos\psi,\sin\psi)$
where $\theta_{p}$ and $\psi$ are the polar and azimuthal angles of the
positron, and $\gamma_{p}$ is the positron energy factor.

The functions $I$, $F$ and $S$ are defined as

$\displaystyle I$
$\displaystyle=\int\mathrm{d}\phi\left(1-\frac{\ell\cdot\pi_{p}}{\ell\cdot
p}\right)e^{i\Phi(\phi)}\,,$ $\displaystyle F^{\mu}$
$\displaystyle=\frac{1}{m}\int\mathrm{d}\phi~{}a^{\mu}(\phi)~{}e^{i\Phi(\phi)}\,,$
$\displaystyle S$
$\displaystyle=\frac{1}{m^{2}}\int\mathrm{d}\phi~{}a(\phi)\cdot
a(\phi)~{}e^{i\Phi(\phi)}\,,$

with the exponent:

$\displaystyle\Phi(\phi)=\int^{\phi}_{\phi_{i}}\mathrm{d}\phi^{\prime}\frac{\ell\cdot\pi_{p}(\phi^{\prime})}{m^{2}\eta_{\ell}(1-s)}\,,$
(2)

where $\pi_{p}$ is the instantaneous momentum of the positron in the field:

$\displaystyle\pi_{p}=p+a-\frac{p\cdot a}{k\cdot p}k-\frac{a^{2}}{2k\cdot
p}k\,.$ (3)

The completed derivation of the NBW pair production probability could start
from an S-matrix element with Volkov wavefunctions Volkov (1935) and has been
well documented in the literature, see for example King (2020b) for an
introduction and Ilderton et al. (2020) for an analogous presentation for NLC.
Because of the charge symmetry, the spectrum (1) can also be applied to the
produced electron by just changing the corresponding definitions for the
electron. After doing the transverse integral over
$q_{{\scriptscriptstyle\perp}}$ , the spectrum (1) shows the symmetry:
$P(s)=P(1-s)$.

###  II.1 Quasi-monoenergetic positron beams from chirped laser pulses

We now choose, as an example, the vector potential $a^{\mu}$ with circular
polarisation:

$\displaystyle a^{\mu}(\phi)$
$\displaystyle=m\xi\left[0,\cos\Psi(\phi),\sin\Psi(\phi),0\right]f(\phi)\,,$
(4)

in which $\xi$ and $f(\phi)$ are the normalised pulse amplitude and envelope,
$\mathrm{d}\Psi(\phi)/\mathrm{d}\phi=\omega(\phi)/\omega_{0}$ is the chirped
frequency of the pulse. At the initial phase $\phi_{i}=0$, $f(\phi_{i})=0$ and
$\omega(\phi_{i})=\omega_{0}$.

We request (i) that the pulse duration is sufficiently long and the variation
of the pulse local amplitude is much slower than the laser frequency, and (ii)
that the variation of the local frequency $\omega(\phi)$ is on the same time
scale as the pulse local amplitude (This will be clear later.):
$\omega^{\prime}(\phi),~{}f^{\prime}(\phi)<<1$. Under these conditions, we can
then apply the slowly-varying approximation that terms of order
$f^{\prime}(\phi)$ [$\omega^{\prime}(\phi)$] can be neglected Heinzl et al.
(2020).

The exponent (2) can be expressed approximately as

$\displaystyle\Phi(\phi)$
$\displaystyle\approx\kappa(\phi)\phi-\zeta(\phi)\sin\left[\Psi(\phi)-\psi\right]$
(5)

where

$\displaystyle\kappa=$ $\displaystyle\frac{\ell\cdot
p}{m^{2}\eta_{\ell}(1-s)}+\frac{\xi^{2}}{2\eta_{\ell}(1-s)s}\frac{1}{\phi}\int^{\phi}_{\phi_{i}}\mathrm{d}\tilde{\phi}f^{2}(\tilde{\phi})\,,$
$\displaystyle\zeta=$ $\displaystyle\frac{\xi
f(\phi)}{\eta_{\ell}(1-s)s}\frac{\omega_{0}}{\omega(\phi)}|q_{{\scriptscriptstyle\perp}}|\,.$

With the Fourier expansions:

$\displaystyle e^{-i\zeta\sin(\Psi-\psi)}$
$\displaystyle=\sum_{n=-\infty}^{+\infty}J_{n}(\zeta)e^{in\psi}e^{-in\Psi}\,,$
(6)

where $J_{n}(\zeta)$ is the Bessel function of the first kind, the functions
$S$ can be expanded approximately as a series of harmonics:

$\displaystyle S$
$\displaystyle\approx-\xi^{2}\sum_{n=-\infty}^{+\infty}e^{in\psi}\int~{}\mathrm{d}\phi~{}f^{2}(\phi)~{}J_{n}(\zeta)e^{i\Omega(\phi)}\,,$
(7)

where $\Omega(\phi)=\kappa(\phi)\phi-n\Psi(\phi)$, the harmonic order $n$
means the net number of the laser photons absorbed from the background field.
The dependence of the argument $\zeta$ on the pulse envelope $f(\phi)$
indicates that the contribution of each harmonic varies during the course of
the pulse and the high order harmonics contribute only at the pulse centre
where $f(\phi)\rightarrow 1$. The functions $I$ and $F$ can be calculated in
the same way as (7) and obtained with the exactly same exponent term (and the
different pre-exponents).

From (7), one can see that the main contribution to the functions $I$, $F$ and
$S$, and therefore to the final spectrum (1), comes from the stationary phase
point where

$\displaystyle\frac{\partial}{\partial\phi}\Omega(\phi)=\frac{q^{2}_{{\scriptscriptstyle\perp}}+m^{2}_{*}/m^{2}}{2\eta_{\ell}(1-s)s}-n\frac{\omega(\phi)}{\omega_{0}}=0\,.$
(8)

where $m_{*}(\phi)=m[1+\xi^{2}f^{2}(\phi)]^{1/2}$ denotes the effective mass
of the produced positrons in the laser pulse Kibble (1966); Reiss (2014).

Let us first consider the standard case with constant frequency:
$\omega(\phi)=\omega_{0}$. The stationary condition (8) implies a chirp in the
positron energy varying with the pulse envelop,
$\mathrm{d}s/\mathrm{d}\phi\neq 0$: For the $n$th harmonic, the positron
energy is in the spectral region
$s^{\prime}_{n,{\scriptscriptstyle-}}(\phi)\leq s\leq
s^{\prime}_{n,{\scriptscriptstyle+}}(\phi)$, where

$\displaystyle
s^{\prime}_{n,\pm}(\phi)=[1\pm\sqrt{1-2m^{2}_{*}/(n\eta_{\ell}m^{2})}~{}]/2\,,$
(9)

shifting between the linear [$f(\phi)=0$] and nonlinear [$f(\phi)=1$] BW
spectral lines. This shifting stems indeed from the variation of the
positron’s effective mass $m_{*}(\phi)$ during the course of laser pulse Seipt
et al. (2014, 2019).

To exclude the energy chirp in the positron spectrum, one simple idea from (9)
is to adapt the energy parameter $\eta_{\ell}$ by prescribing the laser
frequency with a specific chirp: $\eta_{\ell}\sim\omega(\phi)\sim
1+\xi^{2}f^{2}(\phi)$, to compensate the variation of the particle effective
mass in the laser pulse. As one can see, this frequency chirp is modulated by
the field intensity and on the same time scale as the pulse envelope,
satisfying the request before.

Based on the stationary condition (8), this frequency chirp can be prescribed
by solving the differential equation:

$\displaystyle\frac{\mathrm{d}\omega}{\mathrm{d}\phi}=\frac{\omega_{0}}{2n\eta_{\ell}(1-s)s}\frac{\mathrm{d}}{\mathrm{d}\phi}[q^{2}_{{\scriptscriptstyle\perp}}+1+\xi^{2}f^{2}(\phi)]\,,$
(10)

with the initial conditions: $f(\phi_{i})=0$ and
$\omega(\phi_{i})=\omega_{0}$, and acquired with its explicit expression:

$\displaystyle\omega(\phi)=\omega_{0}\left[1+\xi^{2}f^{2}(\phi)/(q_{{\scriptscriptstyle\perp}}^{2}+1)\right].$
(11)

From (11), one can see a remarkable fact that this chirp prescription is
irrelevant to the harmonic order $n$, which indicates this spectral broadening
can be removed from all the harmonic lines at the same time. One should also
note that the complete counterbalance of the broadening can only happen at a
particular outgoing angle:
$|q_{{\scriptscriptstyle\perp}}|\sim\gamma_{p}\theta_{p}$ specified by the
chirp (11). For a realistic detector with nonzero angular acceptance
$\Delta\theta>0$, the spectral peaks of the probed positrons would shift
slightly from the harmonic lines with a finite energy spread (See the results
later in Fig. 3).

Similar discussions can also be applied to linearly polarised field
backgrounds: $a^{\mu}(\phi)=m\xi\left[0,\cos\Psi(\phi),0,0\right]f(\phi)$, and
the prescription of frequency chirp (11) is exactly the same, except that
$\xi^{2}\rightarrow\xi^{2}/2$.

##  III Numerical result

In this section, we first present a numerical example of a head-on collision
between a $13.1\,\textrm{GeV}$ photon and a laser pulse with and without the
frequency chirp, and then we consider the scenario in which the high-energy
photon is replaced with a beam of $\gamma$-photons obtained from the NLC
process of a $16.5~{}\textrm{GeV}$ electron.

We are interested in the on-axis positrons collimated in the direction of the
incoming photon $q_{{\scriptscriptstyle\perp}}\rightarrow 0$ for higher yield,
and thus apply the frequency chirp:

$\displaystyle\omega(\phi)=\omega_{0}\left[1+\xi^{2}f^{2}(\phi)\right].$ (12)

Inserting back into the stationary condition (8), one can then get the
unchirped positron spectrum peaked at the linear BW harmonic line

$\displaystyle s_{n,\pm}=[1\pm\sqrt{1-2/(n\eta_{\ell})}~{}]/2$ (13)

at the angle $\theta_{p}=0$. To produce a narrow-band positron beam at the
cone angle $\theta_{p}\sim|q_{{\scriptscriptstyle\perp}}|/\gamma_{p}$, one can
employ the frequency chirp (11).

To improve the interaction energy parameter $\eta_{l}$, we employ the laser
pulse with the UV frequency $\omega_{0}=15.5\,\textrm{eV}$ and the envelope
$f(\phi)=\sin^{2}(\phi/2N)$ where $0<\phi<2N\pi$ and $N=16$. The (areal)
energy of a plane laser pulse can be calculated as:

$\displaystyle E$
$\displaystyle=-\frac{\omega_{0}}{4\pi\alpha\lambdabar^{2}_{e}}\int\mathrm{d}\phi\left(\frac{1}{m}\frac{\mathrm{d}a}{\mathrm{d}\phi}\right)^{2}\,,$
(14)

in which $\lambdabar_{e}=1/m=386.16~{}\textrm{fm}$ is the electron’s reduced
Compton wavelength. For circularly polarised laser pulses with the frequency
chirp (12), one can obtain

$\displaystyle
E=\frac{\omega_{0}\xi^{2}}{4\alpha\lambdabar^{2}_{e}}\left(\frac{3}{4}+\xi^{2}\frac{35}{32}+\xi^{4}\frac{231}{512}\right)N\,.$
(15)

The first term in the bracket corresponds to the laser pulse with a constant
frequency: $\omega(\phi)=\omega_{0}$.

The choice of particle energy parameters is motivated by the upcoming high-
energy experiments such as LUXE at DESY Hartin et al. (2019); Abramowicz et
al. (2019) and E320 at FACET-II Joshi et al. (2018); Meuren (2019). The strong
UV laser pulse can be obtained through the plasma harmonic generation driven
by an ultrahigh-power optical laser pulse Rödel et al. (2012); Tang et al.
(2017); Tang and Kumar (2019). The frequency chirp (12) can be plugged into
the laser spectrum via the coherent superposition of two linearly and
oppositely chirped laser pulses with suitable time delay Seipt et al. (2019).

###  III.1 NBW

Figure 1: Upper two panels: Angular-energy distribution
$\mathrm{d}^{2}\textrm{P}/[\mathrm{d}s~{}\mathrm{d}(\gamma_{p}\theta_{p})]$ of
the produced positrons via the NBW process from a high-energy photon
($13.1~{}\textrm{GeV}$) in a circularly polarised laser pulse without (a) and
with (b) the frequency chirp. Bottom two panels: Comparison of the positron
energy spectra in the laser pulse with (red line) and without (blue line) the
frequency chirp within the whole angular spread: $\Delta\theta_{p}=\pi$ (c)
and a narrow acceptance: $\Delta\theta_{p}=32~{}\mu\textrm{rad}$ (d). In (d),
the spectral curve for the unchirped case is multiplied by a factor $20$ for
visibility. The vertical dashed lines show the location of the first three
harmonics in the NBW process: $s_{2,+}=0.80$, $s_{3,+}=0.88$ and
$s_{4,+}=0.91$. The chirped laser pulse ($\xi=1$,
$\omega(\phi)=\omega_{0}[1+\xi^{2}f^{2}(\phi)]$) has the same energy as the
unchirped laser pulse ($\xi=1.75$, $\omega(\phi)=\omega_{0}$).

Fig. 1 depicts the narrowing of the positron spectra from the chirped laser
pulse benchmarked with the results from a constant-frequency pulse. As shown
in Fig. 1 (a) and (b), the frequency chirp (12) can effectively compensate the
broadening of each harmonic line: For the unchirped case in (a), harmonic
lines are broadened with plenty of subsidiary peaks and overlap together to be
a continuum spectral domain in the positron angular-energy distribution.
However for the chirped case in (b), the angular-energy distribution is
comprised of a number of well-separated harmonic lines. Around each harmonic
line, the distribution is narrowed to be a single peak at the small outgoing
angle $\theta_{p}\rightarrow 0$ and is slightly broadened at larger scattering
angles with some subpeaks, which comes from the interference between
contributions from different stationary points. At the same time, the
amplitude of the harmonic lines in the chirped case are significantly
improved.

From Fig. 1 (c), one can see that, with the same laser energy, the chirped
laser pulse can produce, approximately, the same number of positrons as the
laser pulse with constant frequency. The difference between the positron
spectra appears around the harmonic lines $s_{n,\pm}$ (13), where the harmonic
order $n$ must be $\geq 2$ because of our parameter setup. This difference
becomes significant in Fig. 1 (d) where the angular-energy distribution is
integrated within a narrow angular spread
$\theta_{p}<\Delta\theta_{p}/2=16~{}\mu\textrm{rad}$: the positron spectrum
from the chirped pulse background spikes around the harmonic lines $s_{n\geq
2,+}$, see the vertical dashed lines in Fig. 1 (d), and has a much higher
amplitude than that from the unchirped laser pulse which gives a much lower
and broader spectrum in the on-axis direction. We label the location of the
first three harmonics: $s_{2,+}=0.80$, $s_{3,+}=0.88$ and $s_{4,+}=0.91$ in
the chirped case. As one can see, the first spectral peak has a much higher
amplitude and narrower energy spread $\Delta s/s\approx 1.7\%$ than other
higher-order peaks, where $\Delta s$ means the full width at the half maximum
of the peak.

As one can see that the $x$-axes of the panels in Fig. 1 (a)-(c) are plotted
in the region $0.5<s<1$ because of the symmetry:
$\textrm{P}(s)=\textrm{P}(1-s)$, see (1). However, the spectrum in Fig. 1 (d),
which is plotted for the whole spectral region $0<s<1$, shows an asymmetric
distribution in the higher energy region $s>0.5$. This is because the lower-
energy positrons distribute in a broader angular region:
$\gamma_{p}\rightarrow 0$ leading to $\theta_{p}\rightarrow\pi/2$. Therefore,
the lower-energy positrons could be simply excluded from the generated high-
energy positron beam by an angular selection.

###  III.2 NLC + NBW

GeV $\gamma$-rays generated by high-energy electron beams via the NLC process
has been analysed in detail in King and Tang (2020); Tang et al. (2020). The
spectrum of the emitted $\gamma$-rays is presented in Ilderton et al. (2020)
in the same way as (1). With a similar stationary phase analysis, one can
prove that a beam of quasi-monoenergetic $\gamma$-photons can be obtained in a
well-chirped laser background Seipt et al. (2014). Replacing the seed photon
used in Fig. 1 with these acquired high-energy $\gamma$-photons, a source of
narrow-band positrons can be obtained. This two-step scenario can be regarded
as a part of the trident process Ilderton (2011); King and Ruhl (2013); King
and Fedotov (2018) in which only real photons contribute.

Figure 2: (a) Angular-energy distribution
$\mathrm{d}^{2}\textrm{P}_{\gamma}/(\mathrm{d}r~{}\mathrm{d}\theta_{\ell})$ of
the $\gamma$-photons generated through the NLC process from a high-energy
electron $E_{e}=16.5~{}\textrm{GeV}$. (b) Energy distribution of the emitted
$\gamma$-photons within the whole angular spread: $\Delta\theta_{\ell}=\pi$
(blue dashed) and a narrow acceptance:
$\Delta\theta_{\ell}=16~{}\mu\textrm{rad}$ (red solid). On the top axis is
shown the corresponding change in the photon energy $E_{\ell}\approx rE_{e}$
from $7.5~{}\textrm{GeV}$ to $16~{}\textrm{GeV}$. The vertical dashed lines
show the location of the first two harmonics in the NLC process: $r_{1}=0.80$
and $r_{2}=0.89$ corresponding to the photon energy
$E_{\ell}=13.1~{}\textrm{GeV}$ and $14.6~{}\textrm{GeV}$. The laser parameters
are the same as Fig. 1 (b).

In Fig. 2, we plot the distributions of the emitted photons from a
$16.5~{}\textrm{GeV}$ electron head-on colliding with the laser pulse
parameterized same as in Fig. 1 (b). With the same technique as the NBW case,
the frequency chirp can effectively compensate the broadening of the photon
spectrum, results in well-separated harmonic lines in the photon angular-
energy distribution
$\mathrm{d}^{2}\textrm{P}_{\gamma}/\mathrm{d}\theta_{\ell}\mathrm{d}r$ in Fig.
2 (a), especially for small angle scatterings $\theta_{\ell}\rightarrow 0$
Seipt et al. (2014, 2019), where $r=k\cdot\ell/k\cdot p_{e}$ is the fraction
of the light-front momentum taken by the scattered photon from the seed
electron, $p_{e}$ is the electron momentum and $\theta_{\ell}$ is the polar
angle of the scattered photon.

With a narrow acceptance: $\Delta\theta_{\ell}=16~{}\mu\textrm{rad}$
collimated in the on-axis direction in Fig. 2 (b), the collected photons
distribute tightly around the first harmonic line $r_{1}=0.80$, corresponding
to the energy $E_{\ell}=13.1~{}\textrm{GeV}$, with the energy spread $\Delta
r/r\approx 1.3\%$, and sub-peak around the second harmonic line $r_{2}=0.89$
at $E_{\ell}=14.6~{}\textrm{GeV}$, see the vertical dashed lines in Fig. 2
(b), where $r_{v}=2v\eta_{e}/(2v\eta_{e}+1)$, $\eta_{e}=k\cdot p_{e}/m^{2}$,
$v\geq 1$ denotes the net number of laser photons absorbed in the NLC process.
The other higher-order subpeaks are too small to view. With a broader angular
selector $\Delta\theta_{\ell}=\pi$, the energy spread of the collected
$\gamma$-photons would be much larger, shown as the blue dashed line in Fig. 2
(b).

Figure 3: (a) Energy distribution $\mathrm{d}\textrm{P}/\mathrm{d}t$ of the
positrons generated by the on-axis $\gamma$-photons obtained through the NLC
process of a high-energy electron $E_{e}=16.5~{}\textrm{GeV}$. The energy
region of the produced positron $E_{p}$ is shown on the top axis. The vertical
black dashed lines denote the location of each combined harmonic:
$t_{1,2,3,4}=(0.637,~{}0.700,~{}0.726,~{}0.732)$ corresponding, respectively,
to the energy $E_{p}=(10.5,~{}11.5,~{}12.0,~{}12.1)~{}\textrm{GeV}$. (b)
Energy spread and peak location of the first harmonic peak with the change of
the acceptance $\Delta\theta_{p}$. The horizontal dashed line denotes the
theoretical location of the first combined harmonic: $t_{1}$. The energy
spectrum of the $\gamma$-photons used in the calculation is plotted as the red
solid line in Fig. 2 (b).

Making use of the obtained $\gamma$-ray spectrum :
$\rho_{\gamma}(r)=\mathrm{d}\textrm{P}_{\gamma}/\mathrm{d}r$ in Fig. 2 (b), we
can calculate the total number of the generated positrons:

P
$\displaystyle=\frac{\alpha}{(2\pi\eta_{e})^{2}}\int^{1}_{0}\mathrm{d}t\int^{1}_{t}\mathrm{d}r\rho_{\gamma}(r)h(r,t)$
(16)

where $t=k\cdot p/k\cdot p_{e}$ denotes the fraction of the light-front
momentum transferring from the seed electron to the produced positron, and

$\displaystyle
h=\int\mathrm{d}^{2}q_{{\scriptscriptstyle\perp}}\frac{|I|^{2}+(SI^{*}+IS^{*}-2F\cdot
F^{*})g}{(r-t)rt}$ (17)

in which $g\equiv[t^{2}+(r-t)^{2}]/4t(r-t)$ is re-defined.

Here, we ignore the small angular spread of the collected $\gamma$-photons to
simplify the numerical calculations. The selection of small-angle photons can
be done in experiments by adjusting the separation between the two laser
pulses used in each process. The induced numerical errors can be offset by
using a much broader acceptance to collect the produced positrons:
$\Delta\theta_{p}\gg\Delta\theta_{\ell}$, see the result in Fig. 3.

In Fig. 3 (a), we plot the yield of the positrons from the on-axis
$\gamma$-photons ($\Delta\theta_{\ell}=16~{}\mu\textrm{rad}$) obtained through
the NLC process discussed in Fig. 2. With the acceptance
$\Delta\theta_{p}=32~{}\mu\textrm{rad}$ along the direction of the seed
electron, most of the positrons are collected in a narrow energy region peaked
at $t=0.625$ with a narrow energy spread $\Delta t/t\approx 2.2\%$ and a much
higher amplitude than other subpeaks in the higher energy region. All of these
peaks can be related to the combined harmonic lines:
$t_{u}=r_{v}s_{n,+}(r_{v})$, where $s_{n,+}(r_{v})$ is the value of the NBW
harmonic line calculated with the NLC harmonic energy
$\eta_{\ell}=\eta_{e}r_{v}$, see the vertical dashed lines in Fig. 3 (a): The
dominant peak is relevant to the first combined harmonic line
$t_{1}=r_{1}s_{2,+}=0.637$ corresponding to the energy $E_{p}\approx
10.5~{}\textrm{GeV}$; The one appears around $t=0.691$ corresponds to the
second combination $t_{2}=r_{1}s_{3,+}\approx 0.700$ with the energy
$E_{p}\approx 11.5~{}\textrm{GeV}$, and the one appears around $t=0.719$ may
come from the sum of the combinations: $t_{3}=r_{1}s_{4,+}=0.726$ and
$t_{4}=r_{2}s_{2,+}=0.732$. The red-shift of each peak location is because of
the broad acceptance $\Delta\theta_{p}$ and can be reduced by narrowing the
acceptance, see the red dotted-dashed line in Fig. 3 (b): the dominant peak
moves asymptotically back to the location of the first combined harmonic line
$t_{1}$ with a decreasing acceptance.

To improve the brilliance of the positron beam, one needs to increase the
detector acceptance: with a larger acceptance
$\Delta\theta_{p}=48~{}\mu\textrm{rad}$ in Fig. 3 (a), the amplitude of the
spectral peak becomes much higher, and at the same time, its energy spread is
broadened to be $3.8\%$. As shown in Fig. 3 (b), the energy spread of the
probed positrons increases with the raising of the detector acceptance
$\Delta\theta_{p}$. With a broad acceptance $\Delta\theta_{p}\approx
80~{}\mu\textrm{rad}$, one can acquire a positron beam with the energy spread
around $10\%$, and with a relatively narrow acceptance
$\Delta\theta_{p}<56~{}\mu\textrm{rad}$, the energy spread of the positron
beam can be simply controlled to be less than $5\%$.

##  IV Conclusion

We investigated the nonlinear Breit-Wheeler process in a chirped laser
background with intensity $\xi\sim 1$. Via the standard stationary-phase
analysis, we elaborated the broadening of the produced positron spectrum
stemming from its inhomogeneous effective mass during the course of the laser
pulse, and proved that with a suitable frequency chirp, this broadening can be
completely compensated in a specified direction. We present a proof-of-
principle calculation in which a beam of quasi-monoenergetic $\gamma$-photons
are obtained from a chirped laser pulse via the nonlinear Compton scattering
process and then are used to produce electron-positron pair in the second
chirped laser pulse to provide a quasi-monoenergetic source of positrons. The
produced positrons are tightly gathered in a narrow energy region around the
combined harmonic lines from the relevant processes. By adjusting the detector
acceptance, the energy spread of the obtained positrons can be well
controlled.

In our calculations, we ignore the energy spread of the seed electron beam,
which would be crucial if it is in the same level of or much broader than the
predicted positron energy spread. To obtain the predicted narrow-band
positrons, high-quality electron beams with limited energy spread are needed
Meuren (2019). We employ the high-power laser pulse with the UV frequency
which is critical in our discussion: the UV laser frequency guarantees the
scattering of high-energy $\gamma$-photons, and then significantly improve the
yield of the electrons and positrons by opening the low-order harmonic
channels for the following nonlinear Breit-Wheeler process.

As an outlook, the considered two-step scenario is potential to generate
highly polarised positron beams: high-energy $\gamma$-photons scattered by the
seed electrons are more probable in the polarisation state parallel to the
background field Tang et al. (2020), and polarised $\gamma$-photons are likely
to decay to electron-positron pairs in particular spin states Seipt and King
(2020).

##  V Acknowledgments

The author thanks Dr. A. Ilderton and Dr. B. King for useful discussion and
careful reading of the manuscript. The author thanks the support from the
Young Talents Project at Ocean University of China.

## References

  * Ritus (1985) V. Ritus, Journal of Russian Laser Research 6, 497 (1985), URL https://doi.org/10.1007/BF01120220.
  * Di Piazza et al. (2012) A. Di Piazza, C. Müller, K. Z. Hatsagortsyan, and C. H. Keitel, Rev. Mod. Phys. 84, 1177 (2012), URL https://link.aps.org/doi/10.1103/RevModPhys.84.1177.
  * Nikishov and Ritus (1964) A. I. Nikishov and V. I. Ritus, Sov. Phys. JETP 19, 529 (1964).
  * King and Tang (2020) B. King and S. Tang, Phys. Rev. A 102, 022809 (2020), URL https://link.aps.org/doi/10.1103/PhysRevA.102.022809.
  * Boca and Florescu (2009) M. Boca and V. Florescu, Phys. Rev. A80, 053403 (2009).
  * Seipt and Kämpfer (2011) D. Seipt and B. Kämpfer, Phys. Rev. A 83, 022101 (2011), URL https://link.aps.org/doi/10.1103/PhysRevA.83.022101.
  * Mackenroth and Di Piazza (2011) F. Mackenroth and A. Di Piazza, Phys. Rev. A 83, 032106 (2011).
  * Ilderton and MacLeod (2020) A. Ilderton and A. J. MacLeod, Journal of High Energy Physics 2020 (2020), URL https://doi.org/10.1007%2Fjhep04%282020%29078.
  * King (2020a) B. King, arXiv preprint arXiv:2012.05920 (2020a).
  * Krajewska and Kamiński (2012) K. Krajewska and J. Z. Kamiński, Phys. Rev. A 85, 062102 (2012), URL https://link.aps.org/doi/10.1103/PhysRevA.85.062102.
  * Wistisen (2014) T. N. Wistisen, Phys. Rev. D 90, 125008 (2014), URL https://link.aps.org/doi/10.1103/PhysRevD.90.125008.
  * Ilderton et al. (2020) A. Ilderton, B. King, and S. Tang, Physics Letters B 804, 135410 (2020), ISSN 0370-2693, URL http://www.sciencedirect.com/science/article/pii/S0370269320302148.
  * Brown and Kibble (1964) L. S. Brown and T. W. B. Kibble, Phys. Rep. 133, A705 (1964).
  * Kibble (1966) T. W. B. Kibble, Phys. Rev. 150, 1060 (1966), URL https://link.aps.org/doi/10.1103/PhysRev.150.1060.
  * Reiss (2014) H. R. Reiss, Phys. Rev. A 89, 022116 (2014), URL https://link.aps.org/doi/10.1103/PhysRevA.89.022116.
  * Raicher and Hatsagortsyan (2020) E. Raicher and K. Z. Hatsagortsyan, Phys. Rev. Research 2, 013240 (2020), URL https://link.aps.org/doi/10.1103/PhysRevResearch.2.013240.
  * Hartemann et al. (1996) F. Hartemann, A. Troha, N. Luhmann, and Z. Toffano, Physical review. E, Statistical physics, plasmas, fluids, and related interdisciplinary topics 54, 2956 (1996).
  * Heinzl et al. (2010) T. Heinzl, D. Seipt, and B. Kämpfer, Phys. Rev. A 81, 022125 (2010), URL https://link.aps.org/doi/10.1103/PhysRevA.81.022125.
  * Albert et al. (2011) F. Albert, S. Anderson, D. Gibson, R. Marsh, S. Wu, C. Siders, C. Barty, and F. Hartemann, Physical Review Special Topics - Accelerators and Beams 14, 050703 (2011).
  * Seipt et al. (2014) D. Seipt, S. Rykovanov, A. Surzhykov, and s. Fritzsche, Physical Review A 91 (2014).
  * Terzić et al. (2014) B. Terzić, K. Deitrick, A. Hofler, and G. Krafft, Physical review letters 112, 074801 (2014).
  * Seipt et al. (2019) D. Seipt, V. Kharin, and S. Rykovanov, Physical Review Letters 122 (2019).
  * Breit and Wheeler (1934) G. Breit and J. A. Wheeler, Phys. Rep. 46, 1087 (1934).
  * Reiss (1962) H. R. Reiss, Journal of Mathematical Physics 3, 59 (1962), eprint https://doi.org/10.1063/1.1703787, URL https://doi.org/10.1063/1.1703787.
  * Burke et al. (1997) D. L. Burke et al., Phys. Rev. Lett. 79, 1626 (1997).
  * Bamber et al. (1999) C. Bamber, S. J. Boege, T. Koffas, T. Kotseroglou, A. C. Melissinos, D. D. Meyerhofer, D. A. Reis, W. Ragg, C. Bula, K. T. McDonald, et al., Phys. Rev. D 60, 092004 (1999), URL https://link.aps.org/doi/10.1103/PhysRevD.60.092004.
  * Narozhnyĭ (1969) N. B. Narozhnyĭ, Sov. Phys. JETP 28, 371 (1969).
  * Jansen et al. (2016) M. Jansen, J. Kamiński, K. Krajewska, and C. Müller, Physical Review D 94 (2016).
  * Wistisen (2020) T. Wistisen, Physical Review D 101 (2020).
  * Titov et al. (2020) A. Titov, A. Otto, and B. Kämpfer, The European Physical Journal D 74 (2020).
  * Jansen and M ller (2013) M. Jansen and C. M ller, Physical Review A 88 (2013).
  * Jansen and Muller (2017) M. J. Jansen and C. Muller, Physics Letters B 766, 71 (2017), ISSN 0370-2693, URL http://www.sciencedirect.com/science/article/pii/S0370269316308024.
  * Titov et al. (2018) A. I. Titov, B. Kämpfer, and H. Takabe, Phys. Rev. D98, 036022 (2018), eprint 1807.04547.
  * Ilderton (2020) A. Ilderton, Phys. Rev. D101, 016006 (2020), eprint 1910.03012.
  * Cabibbo and Gatto (1961) N. Cabibbo and R. Gatto, Phys. Rev. 124, 1577 (1961), URL https://link.aps.org/doi/10.1103/PhysRev.124.1577.
  * Myers and Picasso (1990) S. Myers and E. Picasso, Contemporary Physics 31, 387 (1990), eprint https://doi.org/10.1080/00107519008213789, URL https://doi.org/10.1080/00107519008213789.
  * Tang et al. (2019) S. Tang, A. Ilderton, and B. King, Phys. Rev. A 100, 062119 (2019), URL https://link.aps.org/doi/10.1103/PhysRevA.100.062119.
  * Hartin et al. (2019) A. Hartin, A. Ringwald, and N. Tapia, Phys. Rev. D99, 036008 (2019), eprint 1807.10670.
  * Abramowicz et al. (2019) H. Abramowicz, M. Altarelli, R. Aßmann, T. Behnke, Y. Benhammou, O. Borysov, M. Borysova, R. Brinkmann, F. Burkart, K. Büßer, et al., arXiv preprint arXiv:1909.00860 (2019).
  * Joshi et al. (2018) C. Joshi, E. Adli, W. An, C. E. Clayton, S. Corde, S. Gessner, M. J. Hogan, M. Litos, W. Lu, K. A. Marsh, et al., Plasma Physics and Controlled Fusion 60, 034001 (2018), URL https://doi.org/10.1088/1361-6587/aaa2e3.
  * Meuren (2019) S. Meuren, in _Third Conference on Extremely High Intensity Laser Physics (ExHILP)_ (2019).
  * Rödel et al. (2012) C. Rödel, D. an der Brügge, J. Bierbach, M. Yeung, T. Hahn, B. Dromey, S. Herzer, S. Fuchs, A. G. Pour, E. Eckner, et al., Phys. Rev. Lett. 109, 125002 (2012), URL http://link.aps.org/doi/10.1103/PhysRevLett.109.125002.
  * Tang et al. (2017) S. Tang, N. Kumar, and C. H. Keitel, Phys. Rev. E 95, 051201 (2017), URL https://link.aps.org/doi/10.1103/PhysRevE.95.051201.
  * Tang and Kumar (2019) S. Tang and N. Kumar, Plasma Physics and Controlled Fusion 61, 025013 (2019), URL https://doi.org/10.1088%2F1361-6587%2Faaf378.
  * Volkov (1935) D. M. Volkov, Z. Phys. 94, 250 (1935).
  * King (2020b) B. King, Physical Review A 101 (2020b).
  * Heinzl et al. (2020) T. Heinzl, B. King, and A. J. MacLeod, Phys. Rev. A 102, 063110 (2020), URL https://link.aps.org/doi/10.1103/PhysRevA.102.063110.
  * Tang et al. (2020) S. Tang, B. King, and H. Hu, Physics Letters B 809, 135701 (2020), ISSN 0370-2693, URL http://www.sciencedirect.com/science/article/pii/S0370269320305049.
  * Ilderton (2011) A. Ilderton, Phys. Rev. Lett. 106, 020404 (2011).
  * King and Ruhl (2013) B. King and H. Ruhl, Phys. Rev. D 88, 013005 (2013).
  * King and Fedotov (2018) B. King and A. M. Fedotov, Phys. Rev. D98, 016005 (2018), eprint 1801.07300.
  * Seipt and King (2020) D. Seipt and B. King, Phys. Rev. A 102, 052805 (2020), URL https://link.aps.org/doi/10.1103/PhysRevA.102.052805.