# Drastically Reducing the Number of Trainable Parameters in Deep CNNs by
Inter-layer Kernel-sharing

Alireza Azadbakht1 1 Department of Computer Science, Faculty of Mathematical
Sciences, Shahid Beheshti University, Tehran, Iran
2 Artificial and Natural Intelligence Toulouse Institute (ANITI), Toulouse,
France
3 CerCo UMR 5549, CNRS Université Toulouse 3, Toulouse, France Saeed Reza
Kheradpisheh1,111Corresponding Author
Email addresses:
<EMAIL_ADDRESS>(AA),
<EMAIL_ADDRESS>(SRK),
<EMAIL_ADDRESS>(IKH),
<EMAIL_ADDRESS>(TM) 1 Department of Computer Science, Faculty of
Mathematical Sciences, Shahid Beheshti University, Tehran, Iran
2 Artificial and Natural Intelligence Toulouse Institute (ANITI), Toulouse,
France
3 CerCo UMR 5549, CNRS Université Toulouse 3, Toulouse, France Ismail
Khalfaoui-Hassani2 1 Department of Computer Science, Faculty of Mathematical
Sciences, Shahid Beheshti University, Tehran, Iran
2 Artificial and Natural Intelligence Toulouse Institute (ANITI), Toulouse,
France
3 CerCo UMR 5549, CNRS Université Toulouse 3, Toulouse, France Timothée
Masquelier3 1 Department of Computer Science, Faculty of Mathematical
Sciences, Shahid Beheshti University, Tehran, Iran
2 Artificial and Natural Intelligence Toulouse Institute (ANITI), Toulouse,
France
3 CerCo UMR 5549, CNRS Université Toulouse 3, Toulouse, France

###### Abstract

Deep convolutional neural networks (DCNNs) have become the state-of-the-art
(SOTA) approach for many computer vision tasks: image classification, object
detection, semantic segmentation, etc. However, most SOTA networks are too
large for edge computing. Here, we suggest a simple way to reduce the number
of trainable parameters and thus the memory footprint: sharing kernels between
multiple convolutional layers. Kernel-sharing is only possible between
“isomorphic” layers, i.e. layers having the same kernel size, input and output
channels. This is typically the case inside each stage of a DCNN. Our
experiments on CIFAR-10 and CIFAR-100, using the ConvMixer and SE-ResNet
architectures show that the number of parameters of these models can
drastically be reduced with minimal cost on accuracy. The resulting networks
are appealing for certain edge computing applications that are subject to
severe memory constraints, and even more interesting if leveraging “frozen
weights” hardware accelerators. Kernel-sharing is also an efficient
regularization method, which can reduce overfitting. The codes are publicly
available at https://github.com/AlirezaAzadbakht/kernel-sharing

##  1 Introduction

Modern deep learning is pushing the boundaries of artificial intelligence with
increasingly complex models. These, in turn, come at a skyrocketing cost in
terms of data, energy, memory, computing power, and time to train and use
them. Although empirical model-scaling laws exist, such as the one proposed in
EfficientNet [1], the optimal architectures found by these laws still need
tens of millions of parameters to reach reasonable accuracies on established
tasks and benchmarks [2] [3] [4] [5] [6]. As a result, and despite the huge
success of large deep models, they are still not easily usable in resource-
constrained systems, such as edge devices and embedded systems [7].

A first solution for adapting large CNNs to resource-limited systems is to use
compact architectures such as MobileNet [8] designed to minimize the number of
computational operations and trainable parameters. Other techniques such as
parameter quantization [9] [10] and pruning [11] [12] also reduce the memory
footprint and computational demand of large deep models by reducing the model
size.

To drastically reduce the number of trainable parameters, we propose sharing
parameters between the network layers. Literally, the same set of parameters
is being used in several layers while it plays a different role at each layer.
This helps to have a smaller set of trainable parameters, without down-scaling
the network size. In other words, the network can be folded into layers with
shared parameters and deepened without increasing the number of parameters.

When used in CNNs, this technique can be used by sharing kernels between the
isomorphic convolutional layers having the same configuration (i.e. kernel
size and the number of input and output channels). In traditional CNNs, weight
sharing was limited to neurons in the same feature map. With kernel-sharing, a
kernel could be duplicated in feature maps at different layers. To update a
shared kernel with backpropagation, the gradients should be accumulated across
its different feature maps throughout the network.

With kernel-sharing, the learning algorithm does not search for layer-specific
kernels, rather it looks for shared kernels that can play different roles at
different depths of the network. This could be considered as a form of
regularization on the network complexity. In usual regularization techniques
such as L2-norm [13], the complexity of over-parameterized networks is
controlled by putting pressure on the majority of parameters to get near zero
values. While with shared kernels, the number of parameters is significantly
reduced and kernels are forced to maximally exploit their learning capacity.

We applied the proposed kernel-sharing to two different deep CNN architectures
of ConvMixer [14] and SE-ResNet [15] on CIFAR-10 and CIFAR-100 datasets [16].
The first layer of ConvMixer performs a convolution with large kernels and
stride, then, it is followed by a cascade of isomorphic interlaying depthwise
and pointwise convolutional layers. The SE-ResNet architecture has several
convolutional stages with squeeze-and-excitation modules [15] in between,
where, each stage is consist of several isomorphic convolutional layers.

In the extreme case of applying kernel-sharing to all the isomorphic layers,
the classification accuracy of the ConvMixer model dropped only by 1.8% and
3.6% on CIFAR-10 and CIFAR-100 datasets, while the number of its trainable
parameters was reduced by 13.4 and 9.6 times with respect to the baseline
model. Similarly, the accuracy drop in SE-ResNet with kernel-sharing was
negligible, while, the number of its trainable parameters was drastically
fewer.

Also, our results indicate that kernel-sharing can reduce the challenge of
overfitting when we keep the network depth or the number of trainable
parameters the same as the baseline model.

##  2 Method

State-of-the-art CNNs are usually made of tens of convolutional layers stacked
on each other. Usually, some of these layers have similar configurations, such
as the same number of channels and kernel size. In multi-stage deep CNNs,
usually, the model has several stages of consecutive convolutional layers with
the same configuration (see Fig. 1a). In other words, layer configurations are
similar in each stage and vary across stages. For example, the ConvMixer model
[14] has only one stage of convolutional layers, and the SE-ResNet [15] model
has three or four stages depending on the given dataset. Even in deep CNNs
with multi-block stages, layers may have different configurations within the
same block but they are similar to layers in other blocks (see Fig. 1b).

We introduce kernel-sharing or inter-layer weight-sharing for deep CNNs to
reduce the number of trainable parameters and eventually reduce the memory
footprint, which is especially useful in memory-constrained situations. In the
context of CNNs, weight-sharing is an intra-layer concept and refers to using
the same kernel at different locations. However, these kernels differ between
the layers. Here, we propose to go beyond and train deep CNNs with shared
kernels (i.e., kernel weights) among the layers with the same configuration,
which we are going to call “isomorphic layers” hereafter (see Fig. 2).

(a)

(b)

Figure 1:  Multi-stage deep CNNs with isomorphic layers have the same number
of channels and kernel sizes. a) Isomorphic layers in one stage, where the
number of channels and kernel sizes can vary among the stages. b) Isomorphic
layers in multi-block stages, where the number of channels and kernel sizes
are different in one block but the same between blocks.

We define a sharing group as a set of isomorphic layers sharing their kernels,
located in the same (Fig. 1a) or different stages (Fig. 1b). Also, one might
partition a set of isomorphic layers into two or even more sharing groups. For
example, $n$ isomorphic layers in Fig. 1a could be divided into two sharing
groups of $n/2$ isomorphic layers.

During the forward pass, isomorphic layers in a sharing group use the same set
of trainable parameters (i.e., shared kernels). Hence, we should keep these
parameters the same after the weight updates following the backward pass.
During the training, each layer computes its gradients. Therefore, similar to
what happens for the shared weights in CNNs, for each shared kernel, we should
accumulate its gradients over its feature maps in all the isomorphic layers of
its sharing group.

More formally: let $w^{g}$ be the weights of a shared kernel between $n$
isomorphic layers ($l=1,2,...,n$) of the sharing group $g$. Its gradient is:

$\frac{\partial L}{\partial w^{g}}=\sum^{n}_{l=1}\frac{\partial L}{\partial
w^{g}_{l}},$ (1)

where $w^{g}_{l}$ denotes the use of the shared kernel $w^{g}$ in the $l^{th}$
isomorphic layer, and $L$ is the loss function of the neural network.

Figure 2: Two consecutive isomorphic layers with shared kernels. The same set
of weights is being used in these isomorphic layers.

Each isomorphic layer in a sharing group computes its own gradients with
respect to the shared kernels. Therefore, the computational cost of error-
backpropagation in training CNNs with shared kernels is the same as in
conventional CNNs. However, the number of trainable parameters is drastically
lower with shared kernels.

The proposed method can be applied to most modern deep learning models by
sharing kernels among groups of isomorphic layers. There are no constraints on
how many isomorphic layers lie in a sharing group. For example, in a CNN with
twenty isomorphic layers, we can have two sharing groups, each containing ten
isomorphic layers, or four sharing groups, each containing five isomorphic
layers. Kernel-sharing can be even more flexible when applying it to some
groups and not others. For example, in the CNN mentioned above, with two
groups of ten isomorphic layers, one can apply kernel-sharing to the first
group but not the second group. If kernel-sharing applies only to a small part
of the network or to a small sharing group, the overall set of trainable
parameters will increase. Hence, we have a trade-off here, the more kernel-
sharing you have, the fewer trainable parameters you will deal with.

##  3 Experimental Results

Here, we use CIFAR-10 and CIFAR-100 datasets [16], which are benchmark image
classification tasks widely used in evaluating deep learning models. The
difference between these two datasets is the number of classes, as the name
implicates CIFAR-10 contains ten classes, and CIFAR-100 consists of 100
classes. In the following, we evaluate kernel sharing in two recent deep CNN
architectures (ConvMixer [14] and SE-ResNet [15]) on these two datasets.

###  3.1 Implementation Details

The ConvMixer model [14] was initially proposed to investigate if the
outstanding performance of Visual Transformer models [4] was due to their
inherently more powerful architecture or because of using patches as an early
representation. The model is straightforward: it directly works on patches
from input, creating a patch embedding, separating the mixing of spatial and
channel dimensions (using depthwise and pointwise convolutions), and
maintaining equal channel number and resolution throughout the network.
ConvMixer is a single-stage deep CNN consisting of consecutive CNN blocks,
each made of a depthwise layer followed by a pointwise layer. The two main
hyper-parameters are the number of CNN blocks and the number of channels per
block.

In our experiments, we shared kernels among each of the two sharing groups of
the depthwise and pointwise convolutional layers. For example,
ConvMixer-384/20 is a model with 20 CNN blocks in depth and 384 channels in
each depthwise and pointwise layer, respectively. Therefore, we have two
sharing groups, one consists of 20 depthwise layers, and the other one
consists of 20 pointwise layers. If we apply kernel-sharing for these two
sharing groups, we call this full sharing or two-kernel experiment (see Fig.
3). We can reduce the granularity of kernel-sharing and split these 20 blocks
in half and form four sharing groups, two depthwise and two pointwise sharing
groups, each of size ten. We call it the four-kernel experiment. If we go
further and split the CNN blocks into four parts, we would have eight sharing
groups, called the eight-kernel experiment.

SE-ResNet [15] is a variant of ResNet [17] that employs squeeze-and-excitation
(SE) blocks [15] in order to perform dynamic channel-wise feature
recalibration. SE-ResNet is a multi-stage ResNet with a SE module at the end
of each block of convolutional layers within a stage, which computes a
weighted representation of its input channels. The corresponding weight of
each channel is a scaling factor computed by a series of fully connected
layers.

Depending on the dataset, the SE-ResNet model could have a different number of
stages, where each stage consists of a stack of isomorphic convolutional
layers. The SE-ResNet models for both CIFAR-10 and CIFAR-100 consist of three
stages. We set the convolutional layers in these three stages to have 64, 128,
and 256 channels, respectively.

To demonstrate the effectiveness of kernel-sharing with a stage-wise setting
in SE-ResNet experiments, we consider isomorphic layers in each stage as a
sharing group. For example, SE-ResNet-d6 has three sharing groups, each
consisting of six isomorphic convolutional layers. We investigate kernel-
sharing in all the three stages (i.e. sharing groups). We call this experiment
as full or all-stages kernel-sharing. In the second experiment, kernel-sharing
was applied only to the second and third stages. Then in the third experiment,
we only applied kernel-sharing to the third stage and did not enforce kernel-
sharing to other stages.

###  3.2 ConvMixer Model

We trained ConvMixer models on both CIFAR datasets using AdamW optimizer[18]
with Cosine learning rate scheduler [19] for 200 epochs. Experiments were
conducted in batches of 128 images with a learning rate of 0.01.

(a) CIFAR-10 dataset.

(b) CIFAR-100 dataset.

Figure 3: The results of ConvMixer-384/$N$ models on the CIFAR family
datasets. The depth of the network, $N$, is set to 4, 8, 12, 16, 20, and 24.
The radius of each dot represents the number of FLOPs required to run that
model. As the depth of the model increases, its computational cost also
increases. Note that the eight-kernel model with a depth of four is the same
as the baseline, so we ignore this model. The computational demand of models
varies in the range of 1.4 to 9 GFLOPs.

To demonstrate the effect of kernel-sharing on the ConvMixer model, we set the
number of channels in all the layers to 384. Fig. 3a illustrates the results
of the iso-channel experiment, where we fixed the number of channels and
varied the depth of the model. As seen, for the CIFAR-10 dataset, the optimum
depths for two-kernel, four-kernel, eight-kernel, and baseline are 20, 20, 16,
and 16, respectively.

The overall accuracy of the optimum full kernel-sharing model (i.e., two-
kernel model) is 1.8% lower than the optimum baseline model (95.4% vs. 97.2%),
however, its trainable parameters are 13.4 times fewer ($\sim$216K vs.
$\sim$2.9M). This means that the kernel-sharing model significantly reduces
the memory footprint with only a small cost on accuracy. Note that their
numbers of computations are not so different (7.4 GFLOPs vs. 6 GFLOPs).

In the iso-channel study, we investigated the trade-off between the number of
sharing kernels and model performance. If we use four kernels in kernel-
sharing, its optimum model is 1.3% lower than the optimum baseline (95.9% vs.
97.2%), with 7.3 times fewer parameters ($\sim$395K vs. $\sim$2.9M). With
eight kernels, this margin reduces to 0.9% (96.3% vs. 97.2%) with 3.9 times
fewer parameters ($\sim$748K vs. $\sim$2.9M). Evidently, as the number of
sharing groups increases from two to eight, the gap between the baseline and
kernel-sharing model shrinks in terms of accuracy and memory footprint.
Regarding this trade-off, one can choose to use lower memory or get higher
accuracy. Of course, this accuracy drop, even in the extreme case of full
kernel-sharing, is not more than 1.8% while the model size is drastically
smaller.

The eight-kernel ConvMixer-384/16 performs similarly to the ConvMixer-384/8
baseline. This equality means that if there are no strong constraints on
response time or computation power, but the memory footprint is the main
challenge, one can employ the kernel-shared model and approximately reduce the
number of trainable parameters by 50% with almost no drop in the accuracy.

As seen in Fig. 3(b), the iso-channel study on CIFAR-100 follows the same
pattern as CIFAR-10. In this dataset, the optimum depths for two-kernel, four-
kernel, eight-kernel, and baseline are 20, 16, 12, and 16, respectively. In
comparison with the optimum models on CIFAR-10, it is visible that all the
kernel-shared models get to optimum performance with a shallower depth.

We can also observe the trade-off between the number of shared kernels and
model performance in the CIFAR-100 dataset. With two kernels, the difference
between the optimum kernel-shared model and optimum baseline is 3.6% (77.6%
vs. 81.2%) with 11.7 times fewer parameters ($\sim$250K vs. $\sim$2.9M). If we
use four kernels, the difference is 2.3% (78.9% vs. 81.2%) with 6.9 times
fewer parameters ($\sim$424K vs. $\sim$2.9M), and with eight kernels, this
margin reduces to 1.3% (79.9% vs. 81.2%) with 3.8 times fewer number of
parameters ($\sim$776K vs. $\sim$2.9M).

With kernel-sharing, we can make the model deeper with a negligible increase
in the number of trainable parameters. This small increase is caused by the
batch normalization layers, whose parameters are not shared. Indeed, we
investigated the effect of sharing batch normalization layers parameters, but
it also led to a considerable performance drop. Therefore it is not
recommended to share batch normalization layers.

Model | Depth | No. Channels | No. Parameters | GFLOPs | Accuracy(%)
---|---|---|---|---|---
Baseline | 4 | 128 | 112138 | 0.22 | 90.28
Kernel-Shared | 4 | 283 | 112644 | 0.86 | 91.29
Baseline | 8 | 128 | 222218 | 0.46 | 94.28
Kernel-Shared | 8 | 410 | 221820 | 3.36 | 94.76
Table 1: The iso-parameter study of ConvMixer model on CIFAR-10 dataset with
full kernel-sharing(i.e. two-kernels)

The kernel-sharing model drastically decreases the number of parameters
compared to the baseline model. In the next experiment we performed a iso-
parameter study, in which we compared kernel-shared models with the same
number of parameters as the baseline model. To do so, we chose two baselines
(ConvMixer-128/4 and ConvMixer-128/8) and compared them with kernel-shared
ConvMixer models, which have almost the same number of parameters as the
baselines on the CIFAR-10 dataset. Note that we fixed the depth of the model
and increased the number of channels so that overall trainable parameters
remain in the same order as the baseline (see Table. 1).

The kernel-shared ConvMixer-283/4 approximately has the same number of
parameters as the baseline ConvMixer-128/4 with 1% better accuracy on the
CIFAR-10 dataset, although with more FLOPs. Also, for baseline ConvMixer-128/8
and kernel-shared ConvMixer-410/8, which have almost the same number of
parameters, the kernel-shared model outperforms the baseline by a margin of
0.5%.

(a)

(b)

Figure 4: Result of the iso-depth study on ConvMixer-$C$/20 models over the
CIFAR-100 dataset, where $C$ is 128, 256, 384, 512, 640, 768, 896, 1024, 1152.
The radius of each dot represents the number of FLOPs requied to run that
model. Obviously, the model computational cost increases with the number of
channels. The accuracy of the best baseline model is shown by the red line to
make a better comparison. The computational demand of models varies in the
range of 1.2 to 58.4 GFLOPs.

In the iso-depth study, we chose ConvMixer models with the depth fixed at 20
and varied the number of channels from 128 to 1152 with steps of 128. We
trained these models on the CIFAR-100 dataset with and without applying
kernel-sharing. As shown in Fig. 4a, from ConvMixer-768/20 onwards, kernel-
shared models outperform baselines in terms of accuracy. Notably,
ConvMixer-768/20 has 16.6 times fewer trainable parameters than the baseline.
Any increase in channels from 768 would cause the baseline models to overfit.
However, kernel-sharing can help the model to avoid overfitting (see Fig. 4b).
It seems that kernel-sharing can be considered a regularization method, as it
reduces the model complexity and increases its generalization.

###  3.3 SE-ResNet Model

Experiments of SE-ResNet were conducted with SGD optimizer and MultiStepLR
learning rate scheduler with batches of 128 for 256 epochs and a learning rate
of 0.1.

(a) CIFAR-10 dataset.

(b) CIFAR-100 dataset.

Figure 5: Result of SE-ResNet models on the CIFAR family datasets, where the
depth of each stage is set to 2, 3, 4, 6, 8, and 10. The radius of each dot
represents the number of FLOPs required to run that model. The computational
demand of models varies in the range of 0.8 to 4.4 GFLOPs.

In the original SE paper[15], SE-ResNet-110, and SE-ResNet-164 were used for
CIFAR family datasets. These two models consist of 18 and 27 convolutional
layers at each stage, so the overall networks have 54 and 81 layers in depth,
respectively.

Here, we decreased the depth of each stage and instead increased the number of
channels in our experiments. Indeed, we set the number of channels in the
three stages of our SE-ResNet models respectively to 64, 128, and 256 and
varied the depth of each stage from two to ten. Regarding the use of more
channels in higher stages, the overall number of trainable parameters
increases throughout the network. Therefore, most of the trainable parameters
lie in deeper stages. For the iso-channel study, we only investigate the
effect of sharing kernels on specific stages. In the SE-ResNet model, each
stage could be considered as a sharing group. As the first stage consists of
the least number of parameters, sharing kernels only in the first stage is not
that beneficial. Therefore, we performed three kernel-sharing experiments:
sharing in all stages, sharing in the second and third stages, and sharing
only in the third stage.

As seen in Fig. 5a, on the CIFAR-10 dataset, the optimum baseline is SE-
ResNet-d8, with eight isomorphic layers in each stage. SE-ResNet-d6 is the
best model when we apply kernel sharing only to the third stage. The optimum
model for applying kernel sharing to all stages or only the second and third
stages is SE-ResNet-d4. Seemingly with applying kernel-sharing in more stages,
the optimum model is shallower.

By applying kernel-sharing over all stages, the optimum accuracy drops by 1.3%
with respect to the baseline (94.5% vs. 95.8%), but with a 9.7 times fewer
number of trainable parameters ($\sim$1.2M vs. $\sim$12.1M) and about 0.5
times fewer FLOPs. A comparison between kernel-sharing in the second and third
stages and baseline reveals that accuracy drops by 1.1% (94.7% vs. 95.8%) with
8.3 times fewer trainable parameters ($\sim$1.4M vs. $\sim$12.1M) and 0.35
times fewer FLOPs. The same comparison for kernel-sharing in the third stage
has similar results, Where, accuracy drops by 0.7% (95.1% vs. 95.8%) with 3.9
times fewer trainable parameters ($\sim$3.1M vs. $\sim$12.1M) and again with
0.26 times fewer FLOPs. The accuracy gap between models with kernel-sharing in
all stages and only the second and third stages is negligible. Indeed, when we
excluded the first stage from kernel-sharing, the number of trainable
parameters remained approximately the same as in the all-stages kernel-sharing
model. However, we gained a slightly (0.2%) better accuracy.

It can be seen that, as for the ComvMixer models, when we apply kernel-sharing
to more stages, the accuracy gap between the kernel-sharing and baseline SE-
ResNet models slightly increases with 1.3% in the most extreme case. On the
other hand, the number of parameters of kernel-shared models drastically
decreases to 1.2M from 12.1M.

The SE-ResNet-d6 with kernel-sharing only in the third stage has the same
accuracy as the SE-ResNet-d3 baseline. Therefore, one can employ the kernel-
shared model and save approximately 28% trainable parameters without any drop
in accuracy.

On the CIFAR-100 dataset, the optimum depth for the baseline is eight (SE-
ResNet-d8). For all-stages kernel-sharing, the optimum depth is half the
baseline (SE-ResNet-d4). When we apply kernel-sharing to the second and third
stages, the optimum depth is three (SE-ResNet-d3), and for kernel sharing only
in the third stage, the optimum depth is six (SE-ResNet-d6).

The iso-channel study on the CIFAR-100 dataset demonstrates that after
applying kernel-sharing over all stages, the overall accuracy drops by 3.7%
(from 77.8% to 74.1%) but with 9.6 times smaller number of trainable
parameters ($\sim$12.1M vs. $\sim$1.3M) and with about half the FLOPs.
Comparison between baseline and kernel-sharing in the second and third stages
reveals that overall accuracy drops by 2.8% (77.8% vs. 75%) with 8.64 times
smaller number of trainable parameters ($\sim$12.1M vs. $\sim$1.4M) and fewer
FLOPs. The same comparison for kernel-sharing only in the third stage has
similar results. The overall accuracy drops by 1.3% (77.8% vs. 76.5%) with a
3.9 times smaller number of trainable parameters ($\sim$12.1M vs. $\sim$3.1M)
and still efficient in terms of required FLOPs.

##  4 Discussion

In this paper, we proposed inter-layer kernel-sharing for convolutional deep
neural networks. This new method helps to reduce the number of parameters in
deep CNNs with a slight drop in accuracy. This idea is a generalization of
weight sharing in conventional CNNs. Traditionally in CNNs, weights are shared
among neurons in a feature map, but here we pushed it further and applied
weight sharing between the isomorphic layers. In this way, isomorphic layers
in a sharing group share the same set of kernels. With kernel-sharing, a
single layer is not free to learn a set of kernels that would be only
effective in that specific layer. Instead, isomorphic layers in a sharing
group should find a more general set of kernels that would be effective for
all the layers in that group.

It seems that kernel-sharing could be thought of as a particular case of
unfolded recurrent neural networks, where at each time step, there is no input
token, and the model only does its computation on the feedback of the previous
time step, and the output is the final state.

As seen in the iso-parameter study on ConvMixer models, we can conclude that
kernel-sharing on shallower networks is more effective. This phenomenon could
be justified by the fact that when the depth of the model increases, the
kernel-shared model faces a more challenging problem in finding the best
kernel that would work if used repeatedly in many layers. When the depth
increases, too many gradient updates apply on the same kernel, and some of the
effects smooth out, which negatively affects the overall performance.

In hardware implementations, a small non-programmable circuit can be designed
to do the computation needed for a neural network [20] [21] [22]. This
hardwired model implementation is more efficient and faster than the same
model on a programable circuit [23] [24]. Many optimizations like fixed shift-
add scalers [25], zero-overhead weight pruning [26], optimized intermediate
precision [23], zero DRAM bandwidth [23] [27], and minimal activation storage
[23] could be used in these kinds of kits. The hardware implementation of a
kernel-shared model is much more efficient and accessible. For a model with a
single set of kernels shared among the whole network, one only needs to design
a CNN layer on the circuit and use it repeatedly to run the whole network. In
this way, the silicon area needed for implementing a neural network
drastically decreases, and the memory footprint would be significantly
smaller.

In the iso-parameter case, applying kernel-sharing improves the model’s
accuracy, but the resulting model has more FLOPs than the baseline
counterpart. For example, in terms of accuracy, the kernel-shared
ConvMixer-283/4 is better than the ConvMixer-128/4 baseline with the same
number of parameters. This behavior of kernel-sharing implies that if
computational demand is not an issue, we can use kernel-sharing and achieve
better performance. In practice, the benefits of hardware implementation of
the kernel-shared model can compensate for its higher computational demand.

As seen in the iso-depth study with fixed depth, when we increase the width of
the network (number of channels), at some point, baseline models start to
overfit. However, kernel-shared counterparts outperform them with fewer
trainable parameters and the same number of FLOPs. In this way, we can
consider kernel-sharing as a regularization method. Therefore, CNN blocks
consisting of convolutional layers with shared kernels could be used as the
building blocks of large networks with a lower risk of overfitting.

In iso-channel studies on ConvMixer and SE-ResNet models on both CIFAR-10 and
CIFAR-100 datasets, we faced a trade-off between the number of trainable
parameters and the classification accuracy. If we enforce kernel-sharing more
extensively, the model would be drastically smaller in terms of trainable
parameters, however, the accuracy would drop a bit. In the worst case, the
accuracy dropped by 1.8% in CIFAR-10 and 3.7% in CIFAR-100 datasets. Regarding
this trade-off, we have to pay attention to our memory constraints or the
available silicon area in hardware implementations. We should first fix the
number of trainable parameters and the feasible computation power, then,
choose the proper depth and the number of shared kernels.

The main question is why we need kernel-sharing and where it would be helpful.
Most accurate deep neural networks have a large number of trainable
parameters, and they are too heavy to be implemented on a small silicon area
of an edge device [7]. Kernel-sharing can be the solution to this problem. We
could achieve a model with drastically lower trainable parameters but with a
slight drop in accuracy. Note that kernel-sharing is very flexible, and one
can choose how and where to apply it.

## References

  * [1] M. Tan and Q. Le, “Efficientnet: Rethinking model scaling for convolutional neural networks,” in _International conference on machine learning_. PMLR, 2019, pp. 6105–6114.
  * [2] D. Mahajan, R. Girshick, V. Ramanathan, K. He, M. Paluri, Y. Li, A. Bharambe, and L. Van Der Maaten, “Exploring the limits of weakly supervised pretraining,” in _Proceedings of the European conference on computer vision (ECCV)_ , 2018, pp. 181–196.
  * [3] Z. Liu, H. Mao, C.-Y. Wu, C. Feichtenhofer, T. Darrell, and S. Xie, “A convnet for the 2020s,” in _Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition_ , 2022, pp. 11 976–11 986.
  * [4] A. Dosovitskiy, L. Beyer, A. Kolesnikov, D. Weissenborn, X. Zhai, T. Unterthiner, M. Dehghani, M. Minderer, G. Heigold, S. Gelly _et al._ , “An image is worth 16x16 words: Transformers for image recognition at scale,” _arXiv preprint arXiv:2010.11929_ , 2020.
  * [5] C. Jia, Y. Yang, Y. Xia, Y.-T. Chen, Z. Parekh, H. Pham, Q. Le, Y.-H. Sung, Z. Li, and T. Duerig, “Scaling up visual and vision-language representation learning with noisy text supervision,” in _International Conference on Machine Learning_. PMLR, 2021, pp. 4904–4916.
  * [6] Z. Dai, H. Liu, Q. V. Le, and M. Tan, “Coatnet: Marrying convolution and attention for all data sizes,” _Advances in Neural Information Processing Systems_ , vol. 34, pp. 3965–3977, 2021.
  * [7] N. D. Lane, S. Bhattacharya, A. Mathur, P. Georgiev, C. Forlivesi, and F. Kawsar, “Squeezing deep learning into mobile and embedded devices,” _IEEE Pervasive Computing_ , vol. 16, no. 3, pp. 82–88, 2017.
  * [8] A. G. Howard, M. Zhu, B. Chen, D. Kalenichenko, W. Wang, T. Weyand, M. Andreetto, and H. Adam, “Mobilenets: Efficient convolutional neural networks for mobile vision applications,” _arXiv preprint arXiv:1704.04861_ , 2017.
  * [9] I. Hubara, M. Courbariaux, D. Soudry, R. El-Yaniv, and Y. Bengio, “Quantized neural networks: Training neural networks with low precision weights and activations,” _The Journal of Machine Learning Research_ , vol. 18, no. 1, pp. 6869–6898, 2017.
  * [10] J. Wu, C. Leng, Y. Wang, Q. Hu, and J. Cheng, “Quantized convolutional neural networks for mobile devices,” in _Proceedings of the IEEE conference on computer vision and pattern recognition_ , 2016, pp. 4820–4828.
  * [11] T.-J. Yang, Y.-H. Chen, and V. Sze, “Designing energy-efficient convolutional neural networks using energy-aware pruning,” in _Proceedings of the IEEE conference on computer vision and pattern recognition_ , 2017, pp. 5687–5695.
  * [12] P. Molchanov, S. Tyree, T. Karras, T. Aila, and J. Kautz, “Pruning convolutional neural networks for resource efficient inference,” _arXiv preprint arXiv:1611.06440_ , 2016.
  * [13] A. Krogh and J. Hertz, “A simple weight decay can improve generalization,” _Advances in neural information processing systems_ , vol. 4, 1991.
  * [14] A. Trockman and J. Z. Kolter, “Patches are all you need?” _arXiv preprint arXiv:2201.09792_ , 2022.
  * [15] J. Hu, L. Shen, and G. Sun, “Squeeze-and-excitation networks,” in _Proceedings of the IEEE conference on computer vision and pattern recognition_ , 2018, pp. 7132–7141.
  * [16] A. Krizhevsky and G. Hinton, “Learning multiple layers of features from tiny images,” _(Technical Report) University of Toronto_ , 2009.
  * [17] K. He, X. Zhang, S. Ren, and J. Sun, “Deep residual learning for image recognition,” in _Proceedings of the IEEE conference on computer vision and pattern recognition_ , 2016, pp. 770–778.
  * [18] I. Loshchilov and F. Hutter, “Decoupled weight decay regularization,” _arXiv preprint arXiv:1711.05101_ , 2017.
  * [19] ——, “Sgdr: Stochastic gradient descent with warm restarts,” _arXiv preprint arXiv:1608.03983_ , 2016.
  * [20] B. Barry, C. Brick, F. Connor, D. Donohoe, D. Moloney, R. Richmond, M. O’Riordan, and V. Toma, “Always-on vision processing unit for mobile applications,” _IEEE Micro_ , vol. 35, no. 2, pp. 56–66, 2015.
  * [21] B. Reagen, P. Whatmough, R. Adolf, S. Rama, H. Lee, S. K. Lee, J. M. Hernández-Lobato, G.-Y. Wei, and D. Brooks, “Minerva: Enabling low-power, highly-accurate deep neural network accelerators,” in _2016 ACM/IEEE 43rd Annual International Symposium on Computer Architecture (ISCA)_. IEEE, 2016, pp. 267–278.
  * [22] M. Buckler, P. Bedoukian, S. Jayasuriya, and A. Sampson, “Eva2: Exploiting temporal redundancy in live computer vision,” in _2018 ACM/IEEE 45th Annual International Symposium on Computer Architecture (ISCA)_. IEEE, 2018, pp. 533–546.
  * [23] P. N. Whatmough, C. Zhou, P. Hansen, S. K. Venkataramanaiah, J.-s. Seo, and M. Mattina, “Fixynn: Efficient hardware for mobile computer vision via transfer learning,” _arXiv preprint arXiv:1902.11128_ , 2019.
  * [24] A. Samajdar, Y. Zhu, P. Whatmough, M. Mattina, and T. Krishna, “Scale-sim: Systolic cnn accelerator simulator,” _arXiv preprint arXiv:1811.02883_ , 2018\.
  * [25] K. D. Cooper, L. T. Simpson, and C. A. Vick, “Operator strength reduction,” _ACM Trans. Program. Lang. Syst._ , vol. 23, no. 5, p. 603–625, sep 2001\.
  * [26] A. Parashar, M. Rhu, A. Mukkara, A. Puglielli, R. Venkatesan, B. Khailany, J. Emer, S. W. Keckler, and W. J. Dally, “Scnn: An accelerator for compressed-sparse convolutional neural networks,” _ACM SIGARCH computer architecture news_ , vol. 45, no. 2, pp. 27–40, 2017.
  * [27] H.-T. Shih and T.-S. Chang, “Zebra: memory bandwidth reduction for cnn accelerators with zero block regularization of activation maps,” in _2020 IEEE International Symposium on Circuits and Systems (ISCAS)_. IEEE, 2020, pp. 1–5.


# Model studies of fluctuations in the background for jets in heavy ion
collisions

Charles Hughes  Antonio Carlos Oliveira Da Silva  Christine Nattrass
University of Tennessee, Knoxville, TN, USA-37996.

###### Abstract

Jets produced in high energy heavy ion collisions are quenched by the
production of the quark gluon plasma. Measurements of these jets are
influenced by the methods used to suppress and subtract the large, fluctuating
background and the assumptions inherent in these methods. We compare the
measurements of the background in Pb+Pb collisions at $\sqrt{s_{\mathrm{NN}}}$
= 2.76 TeV by the ALICE collaboration [1] to calculations in TennGen (a data-
driven random background generator) and PYTHIA Angantyr. The standard
deviation of the energy in random cones in TennGen is approximately in
agreement with the form predicted in the ALICE paper, with deviations of 1–6%.
The standard deviation of energy in random cones in Angantyr exceeds the same
predictions by approximately 40%. Deviations in both models can be explained
by the assumption that the single particle $d^{2}N/dydp_{T}$ is a Gamma
distribution in the derivation of the prediction. This indicates that model
comparisons are potentially sensitive to the treatment of the background.

###### pacs:

25.75.-q,25.75.Gz,25.75.Bh

##  I Introduction

A hot, dense, strongly interacting liquid of quarks and gluons called the
Quark Gluon Plasma (QGP) is briefly created in high energy heavy ion
collisions [2, 3, 4, 5]. Two of the key signatures of the formation of the QGP
are hydrodynamical flow and jet quenching. The strong azimuthal asymmetry in
the final state particles’ momenta is a signature of hydrodynamical flow.
There are many measurements of jets which can, in principle, provide
quantitative constraints on the properties of the medium [6]. While there have
been some constraints on the properties of the medium due from measurements of
jets [7], the era of quantitative measurements of jets is just beginning.

Improving quantitative constraints on the medium using jet measurements
requires a quantitative understanding of the background. The correlations in
the background due to flow lead to an anisotropic background, which can in
turn influence jet measurements. At the Relativistic Heavy Ion Collider
(RHIC), mixed events were able to successfully describe the background in
measurements of hadron-jet correlations [8], indicating that the background is
dominated by random combinations of particles. Studies of the background at
the Large Hadron Collider (LHC) by the ALICE collaboration found that the
distribution of background energy density measured by using random cones with
the leading jet removed were described well by predictions for a random
background with correlations due to flow [1].

We study the measurements in [1] in two models. We compare to a data-driven
random background generator, TennGen [9], which uses the measured single
particle spectra and flow to generate a realistic background without any jets.
We also use PYTHIA Angantyr [10], a Monte Carlo generator based on PYTHIA
8.2[10, 11], which models heavy ion collisions as a superposition of nucleon-
nucleon collisions.

##  II Simulations

###  II.1 TennGen

The measured single particle double differential spectra for $\pi^{\pm}$, K±,
p and $\bar{p}$ from [12] are fit to a Boltzmann-Gibbs Blast Wave distribution
[13] [14]

$\begin{split}\dfrac{d^{2}N}{dp_{T}dy}=Np_{T}\int_{0}^{1}r^{\prime{}}dr^{\prime{}}\left(\sqrt{m^{2}+p_{T}^{2}}\right)\cdot\\\
I_{0}\left(\dfrac{p_{T}\sinh\left(\tanh^{-1}\left(\beta_{s}{r}^{\prime{}^{n}}\right)\right)}{T_{kin.}}\right)\cdot\\\
K_{1}\left(\dfrac{\sqrt{m^{2}+p_{T}^{2}}\cosh\left(\tanh^{-1}\left(\beta_{s}{r}^{\prime{}^{n}}\right)\right)}{T_{kin.}}\right)\end{split}$
(1)

where $p_{T}$ is the transverse momentum, $y$ is the rapidity, $N$ is the
normalization, $m$ is the mass of the particle, $\beta_{s}$ is the surface
velocity, $n$ is an exponent describing the evolution of the velocity profile,
and $T_{kin.}$ is the kinetic freeze out temperature. The $I_{0}$ and $K_{1}$
are modified Bessel functions. The reduced radius, $r^{\prime{}}$, is
integrated over from 0 to 1. The multiplicity of each particle species is
determined from ratios [15] and is scaled up assuming a constant charged
particle multiplicity per unit pseudorapidity, $dN_{ch}/d\eta$. This is a
reasonable approximation for the pseudorapidity region used in this analysis,
$-1.0<\eta<1.0$. The multiplicities are determined from measurements of the
charged particle multiplicities in [16]. Only the centrality bins in [15] are
available (0–5%, 5–10%, 10–20%, 20–30%, 30–40%, and 40–50%). Only charged
hadrons are generated for this analysis.

The azimuthal asymmetry in heavy ion collisions is decomposed using

$\dfrac{dN}{d\phi}=\dfrac{N_{0}}{2\pi}\sum_{n=1}^{5}2v_{n}\cos(n(\phi-\Psi_{n}))$
(2)

where $N_{0}$ is the number of particles, the $v_{n}$ are constants between 0
and 0.5, and $\phi$ is the azimuthal position of the track. The symmetry
planes $\Psi_{n}$ are set to zero for even $n$ and are randomly thrown from a
flat distribution for the odd $n$, roughly matching correlations observed in
data [17]. A random $p_{T}$ is thrown from the distribution in eq. 1, which is
then used to determine the $v_{n}$. When the $v_{n}$ are included, the
$p_{T}$-dependent $v_{n}$ from [18] are fit to a polynomial for $n>2$. For
$n=1$, a rapidity-even $v_{1}$ comparable to $v_{2}$ and $v_{3}$ has been
observed [19, 20, 21], but it is difficult to measure and is still poorly
constrained. To roughly match these measurements, we use
$v_{1}(p_{T})=v_{2}(p_{T})-0.02$, which will give a negative $v_{1}$ for low
$p_{T}$ and a positive $v_{1}$ for high $p_{T}$, roughly conserving momentum.
The azimuthal coordinate is then randomly drawn from eq. 2. TennGen is also
run with the $v_{n}$ set to zero. The pseudorapidity ($\eta$) is randomly
drawn from a uniform distribution for $\mid\eta\mid<$ 0.9. Event-by-event
fluctuations in the multiplicity are not included. For each centrality bin and
combination of $v_{n}$, 60k events are generated. For the 0–10% centrality
bin, the 0–5% and 5–10% bins are combined. The code for TennGen is available
on Github [9]

###  II.2 Angantyr

PYTHIA Angantyr [10] is a Monte Carlo model for heavy ion collisions included
in PYTHIA 8[10, 11]. It is primarily a superposition of nucleon-nucleon
collisions and includes inelastic collisions, single-diffractive, double-
diffractive, and absorptive collisions using a model with fluctuating radii.
The fluctuating nucleon radii result in a fluctuating nucleon-nucleon cross
section. This further results in multiplicity fluctuations. Angantyr includes
hard scatterings, event-by-event multiplicity fluctuations, and multiparton
interactions. Angantyr does not contain flow or jet quenching. As such, it is
a good baseline for collisions in the absence of a QGP.

Default parameters are used and $20\cdot 10^{3}$ minimum bias Pb+Pb collisions
at $\sqrt{s_{\mathrm{NN}}}$ = 2.76 TeV and $20\cdot 10^{3}$ minimum bias Au+Au
collisions at $\sqrt{s_{\mathrm{NN}}}$ = 200 GeV were generated. The
centrality was determined using the centrality class implemented in Rivet
[22], which uses the multiplicity in the forward pseudorapidity regions
matching the ALICE V0-A and V0-C acceptance and bins the events in terms of
the multiplicity in these regions in Angantyr.

###  II.3 Reconstruction efficiency

The measurements in [1] did not include corrections for detector effects so we
implement an approximate single track reconstruction efficiency, the dominant
effect, to make these model calculations more realistic. We use a
parameterized $p_{T}$-dependent efficiency roughly matching the efficiency of
the ALICE detector in [23].

##  III Results

###  III.1 Background density $\rho$

Table 1:  FastJet Parameters Used $R_{param}$ | 0.4
---|---
ghost max. rapidity | 2.0
repeat | 1
ghost area | 0.005
grid scatter | 1.0
$p_{T}$ scatter | 0.1
$<p_{T}^{ghost}>$ | $10^{-100}GeV/c$

To match the analysis in [1], the background density $\rho$ is estimated using
the $k_{T}$ jet finding algorithm implemented in FastJet [24] with the $p_{T}$
recombination scheme and a resolution parameter of $R=0.4$. Reconstructed
charged particles with $p_{T}$ $>$ 0.15 GeV/$c$ are input into the jet finder
and ghost particles are used to estimate the jet area, $A$. Jet finding
parameters are summarized in tab. 1.

For jet candidates with $|\eta|<0.5$, the median $p_{T}^{jet}/A$ is used to
estimate the background momentum density $\rho$ for each event (as in [25]).
For Angantyr, the two leading jet candidates are excluded from the sample when
calculating the median, as done in [1]. Leading jets are not excluded in
TennGen because it contains no jets.

Figure 1 shows $\rho$ versus the reconstructed number of tracks
$N_{input}^{raw}$ for TennGen and Angantyr. These are fit to a straight line
with the parameters given in tab. 2 and compared to fits from [1]. The
multiplicity dependence is comparable to ALICE data in both models. Note that
the data cover a wider range of multiplicities because TennGen only includes
fixed multiplicities and Angantyr underestimates the multiplicity distribution
by 5–10% [10]. This trivial difference in the multiplicity means that neither
model is directly comparable to the data. We therefore emphasize comparisons
to expectations for a random background in the following sections.

Table 2:  $\rho$ versus multiplicity fit parameters | slope | intercept
---|---|---
Angantyr | 0.0611 $\pm$ 0.0003 | -1.20 $\pm$ 0.23
Background Generator | 0.0610 $\pm$ 0.0029 | -1.31 $\pm$ 2.38
ALICE Data ([1]) | 0.0623 $\pm$ 0.0002 | -3.3 $\pm$ 0.3
Figure 1: Median event by event $\rho$ vs. $N_{input}^{raw}$ for TennGen and
Angantyr Pb+Pb collisions at $\sqrt{s_{\mathrm{NN}}}$ = 2.76 TeV. The line is
from the fit of a straight line to Angantyr. Parameters from fits to a
straight line are in tab. 2.

###  III.2 Distribution of $\delta p_{T}$

Two random cones with a radius $R=0.4$ are drawn within $|\eta|<$ 0.5 for each
event. The $p_{T}$ of all reconstructed charged hadrons in the cone are added
and the background density $\rho$ estimated from jets found with the $k_{T}$
jet finder is subtracted to get

$\delta p_{T}=p_{T,cone}-A_{cone}\rho$ (3)

where $A_{cone}=\pi R^{2}$. The distribution of $\delta p_{T}$ is a measure of
the fluctuations in the background.

Figure 2: Comparison of TennGen and Angantyr to 0–10% central Pb+Pb collisions
at $\sqrt{s_{\mathrm{NN}}}$ = 2.76 TeV data from [1]. In both Angantyr and the
ALICE data, the leading jet has been exlcuded.

Figure 2 shows the distribution of $\delta p_{T}$ in ALICE data in Pb+Pb
collisions at $\sqrt{s_{\mathrm{NN}}}$ = 2.76 TeV [1], TennGen, and Angantyr.
The leading jet has been excluded from both the ALICE data and Angantyr. Even
though TennGen uses ALICE single particle spectra and $v_{n}$, the
distributions do not overlap. This is in part because TennGen uses the average
multiplicity and does not include fluctuations in the number of particles,
leading to a somewhat narrower distribution than the data. Furthermore,
TennGen contains no jets.

The distribution of the sum of momenta from a random sample of particles was
discussed in [26], where it was applied to distributions of transverse energy
in events. These derivations were applied in [1] to the problem of random
cones. In [26], the single particle spectrum is approximated as a Gamma
distribution

$\frac{d^{2}N}{dydp_{T}}\propto\frac{k}{\Gamma(p)}(kp_{T})^{p-1}e^{-kp_{T}}$
(4)

where $p\approx 2$ and $k$ are constants and $\Gamma(p)=p!$ if $p$ is an
integer. The $N$-fold convolution of this distribution is itself another Gamma
distribution with a mean given by $N\langle p_{T}\rangle$ and standard
deviation $\sqrt{N}\sigma_{p_{T}}$. The $\delta p_{T}$ distribution in fig. 2
can therefore be fit to a Gamma distribution to extract the width.

When there are Poissonian fluctuations in the number of particles in the
sample, the distribution is a sum of Gamma distributions, with a standard
deviation given by

$\sigma_{\delta p_{T}}=\sqrt{N\sigma^{2}_{p_{T}}+N\langle p_{T}\rangle^{2}}.$
(5)

For both TennGen with $v_{n}$ =0 and Angantyr, the distribution of the number
of particles in the random cone were consistent with a Poissonian
distribution. The presence of the hydrodynamic flow in eq. 2 leads to non-
Poissonian number fluctuations. If the fluctuations from each term are
approximated as uncorrelated, the width is given by

$\sigma_{\delta
p_{T}}=\sqrt{N\sigma^{2}_{p_{T}}+(N+N^{2}\sum_{n=1}^{\infty}v_{n}^{2})\langle
p_{T}\rangle^{2}}.$ (6)

This approximates the $v_{n}$ as constant for all momenta and uncorrelated
with each other. In [1], only $n=2$ and $n=3$ terms were considered.

Note that $N$ in eq. 5 and eq. 6 is the number of particles in the random
cone, not the charged particle multiplicity in the event. Furthermore the
event-by-event subtraction of $\rho$ slightly reduces the width observed in a
sample of events because this reduces the fluctuations in $N$ due to
differences in the reconstructed multiplicity.

###  III.3 Width of the $\delta p_{T}$ distribution

####  III.3.1 TennGen

Figure 3: Comparison of the $\delta p_{T}$ distribution’s width in TennGen
with $v_{n}$ =0 compared to eq. 5 and non-zero $v_{n}$ compared to eq. 6.
TennGen is generated from fits to single particle $p_{T}$ spectra measured in
$\sqrt{s_{\mathrm{NN}}}$ = 2.76 TeV Pb+Pb collisions with ALICE.

Figure 3 shows $\sigma_{\delta p_{T}}$ in TennGen with $v_{n}$ =0 compared to
eq. 5 and with non-zero $v_{n}$ compared to eq. 6. The predictions from eq. 5
and eq. 6 use the multiplicities, $\langle p_{T}\rangle$, and $\sigma_{p_{T}}$
in TennGen. The slight deviations seen here are qualitatively consistent with
[1], but the absence of any correlations other than flow makes the discrepancy
easier to interpret in TennGen. The derivation of eq. 5 assumed that the
single particle spectra were a Gamma distribution while TennGen uses a Blast
wave, which explains the 2% deviation between TennGen with $v_{n}$ =0 and eq.
5. This indicates that the width is somewhat dependent on the shape of the
spectrum. The derivation of eq. 6 assumed both that the $v_{n}$ are
independent of $p_{T}$ and that there are no correlations between number
fluctuations due to flow, explaining the deviations as high as 6% between this
prediction and TennGen with non-zero flow.

####  III.3.2 Angantyr

Figure 4 compares the $\delta p_{T}$ widths in Angantyr with the no jets, the
leading jet, and the leading two jets excluded from the sample to eq. 5. Jets
are excluded by requiring the separation between the axis of the random cone
and the jet axis $\Delta
R=\sqrt{(\phi_{jet}-\phi_{cone})^{2}+(\eta_{jet}-\eta_{cone})^{2}}>1.0$. The
predictions from eq. 5 use the multiplicities, $\langle p_{T}\rangle$, and
$\sigma_{p_{T}}$ in Angantyr. The same events are analyzed in each case, so
their uncertainties are mostly correlated. The widths in Angantyr are up to
40% higher than that predicted by eq. 5, and the discrepancy only gets
slightly better when jets are excluded.

Figure 5 shows the $\delta p_{T}$ widths in Au+Au collisions at
$\sqrt{s_{\mathrm{NN}}}$ = 200 GeV and Pb+Pb collisions at
$\sqrt{s_{\mathrm{NN}}}$ = 2.76 TeV. The predictions from eq. 5 use the
multiplicities, $\langle p_{T}\rangle$, and $\sigma_{p_{T}}$ in at each
energy. The lower energy should have fewer jets than the higher energy, but
the discrepancy between Angantyr and eq. 5 is comparable at both energies.
This indicates that jets are not the dominant cause of the difference.

Figure 6 shows a comparison of the $\delta p_{T}$ widths calculated with
randomized track $\phi$ to predictions from eq. 5. Even after randomizing the
tracks in the Angantyr simulations, the difference remains. This randomization
breaks all all correlations, indicating that the primary cause of this
discrepancy is the difference between the single particle spectrum in Angantyr
and a Gamma distribution.

Figure 4: Comparison of the $\delta p_{T}$ distribution’s width in Angantyr
for Pb+Pb collisions at $\sqrt{s_{\mathrm{NN}}}$ = 2.76 TeV with eq. 5 with
zero, one, and two leading jets omitted from the sample. Figure 5: Comparison
of the $\delta p_{T}$ distribution’s width in Angantyr for Au+Au collisions at
$\sqrt{s_{\mathrm{NN}}}$ = 200 GeV and Pb+Pb collisions at
$\sqrt{s_{\mathrm{NN}}}$ = 2.76 TeV compared to eq. 5. Figure 6: Comparison of
the $\delta p_{T}$ distribution’s width in Angantyr for Pb+Pb collisions at
$\sqrt{s_{\mathrm{NN}}}$ = 2.76 TeV with tracks with randomized azimuthal
angles compared to eq. 5.

##  IV Conclusions

While our studies broadly support the conclusions in [1] that the background
fluctuations are dominated by random combinations of particles, we find that
this width is sensitive to both details of the hydrodynamical background and
the shape of the single particle momentum spectrum. These effects are less
than 6% for TennGen, a data-driven random background generator, but up to 40%
in PYTHIA Angantyr.

As measurements of jets in heavy ion collisions become higher precision, it is
important to make sure that model comparisons are comparable to data. Some of
the details of flow correlations would be difficult to fully describe in
background subtraction methods. Area-based subtraction techniques such as
those used by ALICE with a data-driven determination of the fluctuations [27,
28] and the $\eta$-reflection method used by CMS [29, 30] should be robust to
these effects. It is less clear how these subtle effects in the width of
fluctuations in the background would be incorporated into mixed events [8] or
impacted by the iterative subtraction techniques used by CMS [31] and ATLAS
[32]. Many models, such as Angantyr, may not accurately reproduce the
background in heavy ion collisions. Implementation of the full experimental
method in model calculations, using tools such as Rivet, is essential for
robust and meaningful comparisons between models and data.

##  V Acknowledgements

We are grateful to Christian Klein-Bösing, Marco van Leeuwen, Leif Lönnblad,
Mike Tannenbaum, and Gary Westfall for productive discussions and Alexandre
Shabetai for feedback on the manuscript. This work was supported in part by
funding from the Division of Nuclear Physics of the U.S. Department of Energy
under Grant No. DE-FG02-96ER40982 and from the National Science Foundation
under Grant No. OAC-1550300. We also acknowledge support from the UTK and ORNL
Joint Institute for Computational Sciences Advanced Computing Facility.

## References

  * [1] The ALICE collaboration.  Measurement of event background fluctuations for charged particle jet reconstruction in pb-pb collisions at $\sqrt{s_{NN}}$ = 2.67 tev.  JHEP, 2012(3):53, Mar 2012.
  * [2] K. Adcox et al.  Formation of dense partonic matter in relativistic nucleus nucleus collisions at RHIC: Experimental evaluation by the PHENIX collaboration.  Nucl. Phys., A757:184–283, 2005.
  * [3] John Adams et al.  Experimental and theoretical challenges in the search for the quark gluon plasma: The STAR collaboration’s critical assessment of the evidence from RHIC collisions.  Nucl. Phys., A757:102–183, 2005.
  * [4] B. B. Back et al.  The PHOBOS perspective on discoveries at RHIC.  Nucl. Phys., A757:28–101, 2005.
  * [5] I. Arsene et al.  Quark Gluon Plasma an Color Glass Condensate at RHIC? The perspective from the BRAHMS experiment.  Nucl. Phys., A757:1–27, 2005.
  * [6] Megan Connors, Christine Nattrass, Rosi Reed, and Sevil Salur.  Jet measurements in heavy ion physics.  Rev. Mod. Phys., 90:025005, 2018.
  * [7] Karen M. Burke et al.  Extracting the jet transport coefficient from jet quenching in high-energy heavy-ion collisions.  Phys.Rev., C90(1):014909, 2014.
  * [8] L. Adamczyk et al.  Measurements of jet quenching with semi-inclusive hadron+jet distributions in Au+Au collisions at $\sqrt{s_{NN}}$ = 200 GeV.  Phys. Rev., C96(2):024905, 2017.
  * [9] Charles Hughes and Christine Nattrass.  TennGen Source Code and Documentation.  https://github.com/chughes90/TennGen.  Accessed: 2020-05-05.
  * [10] Christian Bierlich, Gösta Gustafson, Leif Lönnblad, and Harsh Shah.  The Angantyr model for Heavy-Ion Collisions in PYTHIA8.  JHEP, 10:134, 2018.
  * [11] Torbjorn Sjostrand, Stephen Mrenna, and Peter Z. Skands.  PYTHIA 6.4 Physics and Manual.  JHEP, 05:026, 2006.
  * [12] Betty Abelev et al.  Centrality Dependence of Charged Particle Production at Large Transverse Momentum in Pb–Pb Collisions at $\sqrt{s_{\rm{NN}}}=2.76$ TeV.  Phys. Lett., B720:52–62, 2013.
  * [13] O. Ristea, A. Jipa, C. Ristea, T. Esanu, M. Calin, A. Barzu, A. Scurtu, and I. Abu-Quoad.  Study of the freeze-out process in heavy ion collisions at relativistic energies.  J. Phys. Conf. Ser., 420:012041, 2013.
  * [14] Ekkard Schnedermann, Josef Sollfrank, and Ulrich W. Heinz.  Thermal phenomenology of hadrons from 200-A/GeV S+S collisions.  Phys. Rev. C, 48:2462–2475, 1993.
  * [15] Betty Abelev et al.  Centrality dependence of $\pi$, K, p production in Pb-Pb collisions at $\sqrt{s_{NN}}$ = 2.76 TeV.  Phys. Rev., C88:044910, 2013.
  * [16] Kenneth Aamodt et al.  Centrality dependence of the charged-particle multiplicity density at mid-rapidity in Pb-Pb collisions at $\sqrt{s_{NN}}$ = 2.76 TeV.  Phys. Rev. Lett., 106(CERN-PH-EP-2010-071. CERN-PH-EP-2010-071):032301. 14 p, Dec 2010.
  * [17] Georges Aad et al.  Measurement of event-plane correlations in $\sqrt{s_{NN}}=2.76$ TeV lead-lead collisions with the ATLAS detector.  Phys. Rev. C, 90(2):024905, 2014.
  * [18] Jaroslav Adam et al.  Higher harmonic flow coefficients of identified hadrons in Pb-Pb collisions at $\sqrt{s_{\rm NN}}$ = 2.76 TeV.  JHEP, 09:164, 2016.
  * [19] Matthew Luzum and Jean-Yves Ollitrault.  Directed flow at midrapidity in heavy-ion collisions.  Phys. Rev. Lett., 106:102301, 2011.
  * [20] Georges Aad et al.  Measurement of the azimuthal anisotropy for charged particle production in $\sqrt{s_{NN}}=2.76$ TeV lead-lead collisions with the ATLAS detector.  Phys. Rev., C86:014907, 2012.
  * [21] Ekaterina Retinskaya, Matthew Luzum, and Jean-Yves Ollitrault.  Directed flow at midrapidity in $\sqrt{s_{NN}}=2.76$ TeV Pb+Pb collisions.  Phys. Rev. Lett., 108:252302, 2012.
  * [22] Andy Buckley, Jonathan Butterworth, David Grellscheid, Hendrik Hoeth, Leif Lonnblad, James Monk, Holger Schulz, and Frank Siegert.  Rivet user manual, 2010.
  * [23] Betty Bezverkhny Abelev et al.  Performance of the ALICE Experiment at the CERN LHC.  Int. J. Mod. Phys. A, 29:1430044, 2014.
  * [24] Matteo Cacciari, Gavin P. Salam, and Gregory Soyez.  Fastjet user manual.  The European Physical Journal C, 72(3), Mar 2012.
  * [25] Matteo Cacciari and Gavin P. Salam.  Pileup subtraction using jet areas.  Phys. Lett. B, 659:119–126, 2008.
  * [26] M.J. Tannenbaum.  The distribution function of the event-by-event average pt for statistically independent emission.  Physics Letters B, 498(1):29 – 34, 2001.
  * [27] B. Abelev et al.  Measurement of charged jet suppression in Pb-Pb collisions at $\sqrt{s_{NN}}$ = 2.76 TeV.  JHEP, 03:013, 2014.
  * [28] Jaroslav Adam et al.  Measurement of jet suppression in central Pb-Pb collisions at $\sqrt{s_{\rm NN}}$ = 2.76 TeV.  Phys. Lett. B, 746:1–14, 2015.
  * [29] Serguei Chatrchyan et al.  Measurement of jet fragmentation in PbPb and pp collisions at $\sqrt{s_{NN}}=2.76$ TeV.  Phys.Rev., C90(2):024908, 2014.
  * [30] Serguei Chatrchyan et al.  Measurement of jet fragmentation into charged particles in $pp$ and PbPb collisions at $\sqrt{s_{NN}}=2.76$ TeV.  JHEP, 1210:087, 2012.
  * [31] Vardan Khachatryan et al.  Measurement of inclusive jet cross sections in $pp$ and PbPb collisions at $\sqrt{s_{NN}}=$ 2.76 TeV.  Phys. Rev., C96(1):015202, 2017.
  * [32] Georges Aad et al.  Measurement of the jet radius and transverse momentum dependence of inclusive jet suppression in lead-lead collisions at $\sqrt{s_{NN}}$= 2.76 TeV with the ATLAS detector.  Phys.Lett., B719:220–241, 2013.