# Using linear initialisation to improve speed of convergence and fully-
trained error in Autoencoders

Marcel Marais  Artefact Ltd,
6th Floor, 51 Eastcheap, London EC3M 1DT  Máté Hartstein  Artefact Ltd,
6th Floor, 51 Eastcheap, London EC3M 1DT  George Čevora Correspondence:
<EMAIL_ADDRESS>Artefact Ltd,
6th Floor, 51 Eastcheap, London EC3M 1DT

###### Abstract

Good weight initialisation is an important step in successful training of
Artificial Neural Networks [1, 2]. Over time a number of improvements have
been proposed to this process [3, 4]. In this paper we introduce a novel
weight initialisation technique called the _Straddled Matrix Initialiser_.
This initialisation technique is motivated by our assumption that major,
global-scale relationships in data are linear with only smaller effects
requiring complex non-linearities. Combination of Straddled Matrix and ReLU
activation function initialises a Neural Network as a de facto linear model,
which we postulate should be a better starting point for optimisation given
our assumptions. We test this by training autoencoders on three datasets using
Straddled Matrix and seven other state-of-the-art weight initialisation
techniques. In all our experiments the Straddeled Matrix Initialiser clearly
outperforms all other methods.

Modern autoencoders are artificial neural networks (ANNs) that are mainly used
for dimensionality reduction [2, 5] or anomaly detection [6, 7]. Autoencoders
consist of a series of hidden layers that transform the input data. The
objective of the transformation being that its output matches input as closely
as possible, i.e. achieve low reconstruction error [8].

A major research topic in ANNs is weight initialisation [1]. In the specific
area of autoencoders layer-wise pre-training has been recognised as an
improved weight initialisation method [2], but with the introduction of
Glorot, and He initialisation techniques designed for other types of ANNs [3,
4], it has became obsolete. The motivation for the development of these very
techniques is that the optimal initialisation is dependent on the architecture
of the ANN [4]. However, to our knowledge there has not been any novel
research into weight initialisation in the context of autonecoders, an
omission we address here.

Many problems solved by autoencoders are to a large degree linear.
Consequently, we put forward that initialising autoencoders to perform the
linear transformation of variables would be a better starting point for
gradient descent algorithms than highly nonlinear transformation resulting
from commonly used initialisation techniques such as Glorot, and He [3, 4].
Furthermore, we postulate that the interactions between individual variables
do not have larger role than the variables themselves. This lead us to
conclude that a good starting point for gradient descent should minimise the
overlap of variables, while all variables should have the same weight. When
considering ReLU-based autoencoders with the same number of units in all
layers these two conclusions lead to initialisation using an Identity matrix,
which indeed in this specific case lead to perfect reconstruction without any
optimisation of weights. However, this type of autoencoder has only very
limited uses, such as Fair Adversarial Networks [9], as it doesn’t establish
useful encodings, but merely copies input through a series of identities.
General autoencoder architectures utilise non-square weight matrices, which
precludes the usage of the Identity matrix. We, therefore, introduce
_Straddled Matrix_ , an initialisation technique that achieves our aims
generalized to any autoencoder architecture.

There are fundamentally three ways that the Identity matrix can be adapted to
non-square matrices. First, padding with zeroes as shown in Figure 1(a) and
implemented in the current version of TensorFlow (v2.10.0). Second, the
Recurrent Identity which repeats the Identity matrix as many times as it will
fit, then pads the rest with zeroes as shown in Figure 1(b). Recurrent
Identity was an undocumented feature of Keras removed by pull request #11887
and is the most similar initialisation method to the Straddled Matrix which we
propose in this paper (Figure 1(c)). Rationale behind implementation of
Recurrent Identity into Keras is unknown to us, while its removal seems to
lack any reasoning besides discrepancy with TensorFlow.

$\begin{bmatrix}1&0&0\\\ 0&1&0\\\ 0&0&1\\\ 0&0&0\\\ 0&0&0\\\ 0&0&0\\\ 0&0&0\\\
0&0&0\\\ \end{bmatrix}$

(a) Identity

$\begin{bmatrix}1&0&0\\\ 0&1&0\\\ 0&0&1\\\ 1&0&0\\\ 0&1&0\\\ 0&0&1\\\ 0&0&0\\\
0&0&0\\\ \end{bmatrix}$

(b) Recurrent Identity

$\begin{bmatrix}1&0&0\\\ 0&1&0\\\ 0&0&1\\\ 1&0&0\\\ 0&1&0\\\ 0&0&1\\\ 1&0&0\\\
0&1&0\\\ \end{bmatrix}$

(c) Straddled Matrix

Figure 1: Examples of the three possible ways to initialise a non-square
weight matrix with identity.

### Straddled Matrix

Straddled Matrix initialisation diagonally fills ones for all rows in a
repeated manner as shown in Figure 1(c), thus, unlike Identity or Recurrent
Identity, there is no zero padding. Formally, for a matrix $W_{m\times n}$,
Recurrent Identity is equal to the Straddled Matrix if and only if $m\geq n$
and $m$ is a multiple of $n$. For an $m\times n$ matrix if $m=n$ (a square
matrix) the Straddled Matrix is the same as the Identity $I_{n}$ or $I_{m}$.
However, in the case of a rectangular matrix, specifically when $m>n$, the
diagonal of ones continues on the next row, resulting in a single element of
the matrix being one for each row as shown in Figure 2(a). In the $m<n$ case
we simply pad the extra columns with zeros as shown in Figure 2(b). It should
be clarified that ${\text{Straddled}}_{3\times 5}\neq\text{Straddled}_{5\times
3}^{T}$ neither $||{\text{Straddled}}_{3\times
5}||\neq||\text{Straddled}_{5\times 3}||$ as shown in Figure 2.

$\begin{bmatrix}1&0&0\\\ 0&1&0\\\ 0&0&1\\\ 1&0&0\\\ 0&1&0\\\ \end{bmatrix}$

(a) 5x3 Straddled Matrix

$\begin{bmatrix}1&0&0&0&0\\\ 0&1&0&0&0\\\ 0&0&1&0&0\\\ \end{bmatrix}$

(b) 3x5 Straddled Matrix

Figure 2: The difference between a 3x5 and 5x3 Straddled Matrix

The result of the Straddled Matrix initialisation is that every feature
receives an equal weight, regardless of the the size of the next hidden layer.
When using the ReLU activation function and min-max scaling of features the
first forward pass of the model will have no non-linearities and no
interactions between features.

## Methods

To compare the performance of various initialisation techniques we test all
initialisers using the same autoencoder architecture on three distinct tasks.
The structure of the autoencoder was kept as simple as possible to eliminate
confounding effects of the architecture. To maximise the difference between
Identity, Recurrent Identity and Straddled Matrix we chose to use in the
contractive layer to have $50\%+1$ the number of neurons of the preceding
layer. The full architecture is described in Figure 3. Root Mean Squared Error
(RMSE) of the reconstructed input with respect to the original input was used
as a cost function with standard gradient descent as an optimiser.

|  Input Layer |  Layer 1 |  Layer 2 |  Layer 3 |  Output Layer
---|---|---|---|---|---
Size |  Dataset dependent |  64 |  33 |  64 |  Dataset dependent
Activation function |  N/A |  ReLU |  ReLU |  ReLU |  Sigmoid
Figure 3: The architecture of autoencoder used for comparison of different weight initialisation approaches. Dataset |  Train/test split (%) |  Batch size |  Num. of epochs |  Learning rate |  Num. of runs
---|---|---|---|---|---
Synthetic |  80/20 |  Full batch |  1000 |  0.1 |  10
MNIST |  80/20 |  256 |  1000 |  0.1 |  10
Swarm Behaviour |  80/20 |  Full batch |  1500 |  0.1 |  10
Figure 4: The hyper-parameters used for training during the experiemnts

To assess the performance we ran 8 different initialisers listed in Figure 5
on three different datasets described in detail in Section Data & Results,
while keeping the autoencoder architecture constant as per Figure 3. To ensure
results are comparable across initialisers all hyper-parameters have been kept
the same and in order to mitigate the variance introduced by the stochastic
initialisers we ran each experiment multiple times to produce confidence
intervals. To give every initialiser a fair chance at convergence they have
all been trained for the same number of epochs - leading to overtraining in
some instances. The setup for each experiment is shown in Figure 4.

Initialiser |  Notes
---|---
Straddled |  Novel initialisation proposed in this paper shown in Figure 1(c).
Glorotuniform |  $W\sim U\big{(}-\sqrt{6/(i+o)},\sqrt{6/(i+o)}\big{)}$
Glorotnormal |  $W\sim N\big{(}0,\sqrt{2/(i+o)}\big{)}$
Identity |  Zero padded Identity matrix as shown in Figure 1(a).
Henormal |  $W\sim N\big{(}0,\sqrt{2/i}\big{)}$
Heuniform |  $W\sim U\big{(}-\sqrt{6/i},\sqrt{6/i}\big{)}$
Orthogonal |  Orthogonal matrix obtained from the QR decomposition of a matrix of random numbers drawn from a normal distribution. (TensorFlow 2.11.0 documentation)
Random |  Default TensorFlow 2.11.0 initialiser sampling from a normal distribution.
Figure 5: The weight initialisers used for performance comparison. $i$ and $o$
refer respectively to the number of inputs and outputs to a layer.

To ensure consistency when comparing the performance of the autoencoder using
different initialisers we have a defined a condition that if met will be
considered the _convergence_ point for that initialiser. At epoch $t$ with
loss $l_{t}$ in the loss curve for the autoencoder with a given intialiser.
The autoencoder is said to be converged at $t$ if the loss at every epoch
between $t$ and $t+\alpha$ is bounded above by $l_{t}+\epsilon$ and below by
$l_{t}-\epsilon$. The parameters were selected for each dataset to provide
meaningful representation of convergence, the specific values are provided
alongside the relevant results.

The loss values used are epoch-level means taken across the number of runs
specified in Figure 4.

## Data & Results

To benchmark the performance of the autoencoder using the Straddled Matrix
initialiser against the other initialisers we tested all initialisers on a
variety of datasets. Both synthetic and real world datasets with different
amounts of features and observations as well as varying levels of non-
linearity were used to ensure a holistic overview of the benefits and
drawbacks of the different initialisers.

As a preprocessing step all features were scaled to a range $[0,1]$,
effectively making the reconstruction error a percentage and thus comparable
across datasets.

#### Synthetic Data

We used a synthetic dataset with 100 features and 5000 records. Each record
was created from a randomly initialised vector of dimension 20, which was
transformed using linear and non-linear functions to create a vector of
dimension 100 with varying amounts of linearity between elements. The goal was
to create a dataset that encodes well and benefits the Straddled Matrix
initialisation, which uses the linear transformation of variables as the
starting point. The best description of the dataset is the code used to
generate it, provided in our public repository.

As shown in Figure 6(a), the Straddled Matrix initialiser showed quicker
convergence and a slightly lower loss than all other initialisers we tested
against on the synthetic dataset. Five out of the seven comparisons between
losses reached at the end of the learning came out highly significantly
better, and for all seven comparisons the convergence was best and fastest for
the Straddled Matrix initialiser as reported in Figure 6(b).

(a) Loss as a function of training epoch for each initialiser. The loss values
are the average of 10 runs with the associated confidence intervals shown by
the semi-transparent areas. Both convergence speed and quality are markedly
improved by using Straddled Matrix as opposed to other initialisers.

Initialiser | Converged Epochs | Converged Loss | p-value
---|---|---|---
Straddled | 294.0 | 0.097866 |
Glorotuniform | 448.0 | 0.100448 | $<0.001$
Glorotnormal | 434.0 | 0.100689 | $<0.001$
Identity | 387.0 | 0.109341 | $0.924$
Henormal | 586.0 | 0.099029 | $<0.001$
Heuniform | 525.0 | 0.099770 | $<0.001$
Orthogonal | 540.0 | 0.100072 | $<0.001$
Random | N/A | N/A | $0.059$
(b) Convergence metrics reported as averages across 10 runs, with convergence
criteria of $\epsilon=0.001$, and $\alpha=100$, with N/A recorded when they
were not met throughout the training. The p-values reported correspond to the
results of single tailed t-test on the loss at the final epoch of the training
across the ten runs. Straddled Matrix converged faster and reached a lower
converged loss than all other initialisers.

Figure 6: Performance of various initialisers on the synthetic dataset.

#### MNIST

The MNIST dataset is a collection of 70,000 handwritten digits in the form of
28x28 greyscale images. The original split contains 60,000 examples in the
training set and 10,000 in the the test set. The images were flattened into
vectors of size 784, and normalised to be between 0 and 1 - this preprocessing
was replicated from a Keras autoencoder blog post [10].

It was expected that the high amount of of non-linearity in the MNIST dataset
would have a negative impact on the performance of the Straddled Matrix.
However, as shown in Figure 7(a) Straddled Matrix initialisation performed
comparably to the best initialisers for the first 500 epochs, and actually
converged earlier and at a lower loss than the other three best-performing
methods as reported in Figure 7(b). The improvement in the final loss for
Straddled Matrix initialisation was also confirmed to high significance.

(a) Loss as a function of training epoch for each initialiser. The loss values
are the average of 10 runs with the associated confidence intervals shown by
the semi-transparent areas. Both convergence speed and quality were best for
Straddled Matrix initialisation.

Initialiser | Converged Epochs | Converged Loss | p-value
---|---|---|---
Straddled | 611.0 | 0.099854 |
Glorotuniform | 668.0 | 0.110882 | $<0.001$
Glorotnormal | 705.0 | 0.109809 | $<0.001$
Identity | 652.0 | 0.270451 | $<0.001$
Henormal | 566.0 | 0.148061 | $<0.001$
Heuniform | 666.0 | 0.146084 | $<0.001$
Orthogonal | 702.0 | 0.107960 | $<0.001$
Random | N/A | N/A | $<0.001$
(b) Convergence metrics reported as averages across 10 runs, with with
convergence criteria of $\epsilon=0.005$, and $\alpha=250$, with N/A recorded
when they were not met throughout the training. The p-values reported
correspond to the results of single tailed t-test on the loss at the final
epoch of the training across the ten runs. Straddled Matrix reached a lower
converged loss than all other initialisers.

Figure 7: Performance of various initialisers on the MNIST dataset.

#### Swarm Behaviour

The Swarm Behaviour dataset was constructed by running a survey of human
perception of simulated swarming behaviour. The participants were asked to
rate to what extent the behaviour could be considered flocking, aligned and
grouped. The features of this dataset relate to the position, velocity and
other movement properties of each entity in the swarm [11].

The Swarm Behaviour dataset consists of 2400 features and thus would require
the most compression by the autoencoder when being reduced to a latent space
with 33 dimensions. The Straddled Matrix initialiser demonstrated a rapid
decrease in loss within the first 50 epochs and went on to reach a lower loss
than any other initialiser as shown in Figure 8(a), with convergence results
presented in Figure 8(b). The loss at the end of training was highly
significantly better than any other method, and, notably, a low loss was
achieved in just 20 epochs, 300 epochs earlier than any other method.

(a) Loss as a function of training epoch for each initialiser. The loss values
are the average of 10 runs with the associated confidence intervals shown by
the semi-transparent areas. Both convergence speed and quality are markedly
improved by using Straddled Matrix as opposed to other initialisers. While
Identity initialisation converged faster, the loss of converged state was
worse.

Initialiser | Converged Epochs | Converged Loss | p-value
---|---|---|---
Straddled | 836 | 0.175701 |
Glorotuniform | 873 | 0.185304 | $<0.001$
Glorotnormal | 860 | 0.185110 | $<0.001$
Identity | 360 | 0.186394 | $<0.001$
Henormal | N/A | N/A | $<0.001$
Heuniform | N/A | N/A | $<0.001$
Orthogonal | 785 | 0.185592 | $<0.001$
Random | N/A | N/A | $<0.001$
(b) Convergence metrics reported as averages across 10 runs, with convergence
criteria of $\epsilon=0.005$, and $\alpha=500$, with N/A recorded when they
were not met throughout the training. The p-values reported correspond to the
results of single tailed t-test on the loss at the final epoch of the training
across the ten runs. Straddled Matrix reached a lower converged loss than all
other initialisers.

Figure 8: Performance of various initialisers on the Swarm Behaviour dataset.

## Discussion & Conclusions

In this article we have introduced and tested a novel weight initialisation
technique for Artificial Neural Networks - the _Straddled Matrix_. Straddled
Matrix is a generalization of Identity matrix to non-square matrices. This is
particularly useful for autoencoders, which is the architecture used in this
article.

The motivation for Straddled Matrix initialiser is the assumption that most
relationships in data as well as the transformation performed by this
initialiser are linear, therefore minimising reconstruction loss should be an
generally smoother process leading to lower error, and faster convergence.
Identity Matrix initialisation does have these same properties for certain
architectures, but comes with areas of zeroes whenever the weight matrix is
not square.

To verify these hypotheses we have trained a simple fixed-architecture
autoencoder on three datasets, ten times each, using seven state-of-the-art
initialisation techniques as well as the Straddled Matrix. In all cases except
two (out of 21 comparisons) the Straddled Matrix achieved highly significantly
better (p-value of $<0.001$) and faster convergence than all of the other
initialisers. The two cases where convergence was not significantly better
(p-value$>0.001$) were the comparison between the Straddled Matrix and
Identity, and Straddled Matrix and Random initialisations on the synthetic
dataset we have devised. Yet, even in these two cases Straddled Matrix
initialisation converged faster and reached a lower loss than all other
methods (see Figure 6(a)). These results therefore constitute a novel
improvement to training autoencoders, saving precious computation time as well
as providing more accurate reconstruction results.

We see a large number of opportunities for further investigation regarding the
Straddled Matrix initialisation of weights in Artificial Neural Networks.

  1. 1.

Introduce stochasticity to facilitate escapes from local minima by
randomisation where the ones are located in the matrix.

  2. 2.

Scale the ones in the matrix to achieve even activations across hidden units -
a refinement that was shown to provide improvements to other initialisation
techniques.

  3. 3.

Try different network architectures e.g. feed-forward, convolutional networks
and others.

  4. 4.

Analyse linear separability of latent space using a support vector machine
(SVM). While the results clearly favour Straddled Matrix, they do not directly
support our hypothesis. Demonstration of better linear separability of latent
space would provide more support for our hypotheses.

##### Acknowledgements:

Special thank you belongs to Nikola Loncar and Michał Gnacik for reviewing the
code accompanying this article.

##### Data & code availability:

The code for the Straddled Matrix and the experiments to reproduce the results
presented here are publicly available on GitHub
hhttps://github.com/artefactory-uk/autoencoder-paper.

## References

  * [1] Meenal V Narkhede, Prashant P Bartakke, and Mukul S Sutaone.  A review on weight initialization strategies for neural networks.  Artificial intelligence review, 55(1):291–322, 2022.
  * [2] G. E. Hinton and R. R. Salakhutdinov.  Reducing the dimensionality of data with neural networks.  Science, 313(5786):504–507, 2006.
  * [3] Xavier Glorot and Yoshua Bengio.  Understanding the difficulty of training deep feedforward neural networks.  In Yee Whye Teh and Mike Titterington, editors, Proceedings of the Thirteenth International Conference on Artificial Intelligence and Statistics, volume 9 of Proceedings of Machine Learning Research, pages 249–256, Chia Laguna Resort, Sardinia, Italy, 2010. PMLR.
  * [4] Kaiming He, Xiangyu Zhang, Shaoqing Ren, and Jian Sun.  Delving deep into rectifiers: Surpassing human-level performance on imagenet classification.  CoRR, abs/1502.01852, 2015.
  * [5] Quentin Fournier and Daniel Aloise.  Empirical comparison between autoencoders and traditional dimensionality reduction methods.  In 2019 IEEE Second International Conference on Artificial Intelligence and Knowledge Engineering (AIKE), pages 211–214. IEEE, 2019.
  * [6] Varun Chandola, Arindam Banerjee, and Vipin Kumar.  Anomaly detection: A survey.  ACM computing surveys (CSUR), 41(3):1–58, 2009.
  * [7] Jinwon An and Sungzoon Cho.  Variational autoencoder based anomaly detection using reconstruction probability.  Special Lecture on IE, 2(1):1–18, 2015.
  * [8] Dor Bank, Noam Koenigstein, and Raja Giryes.  Autoencoders.  arXiv preprint arXiv:2003.05991, 2020.
  * [9] George Cevora.  Fair adversarial networks.  arXiv preprint arXiv:2002.12144, 2020.
  * [10] Francois Chollet.  Building Autoencoders in Keras, 2016.
  * [11] Shadi Abpeikar and Kathryn Kasmarik.  Motion behaviour recognition dataset collected from human perception of collective motion behaviour.  Data in Brief, 47:108976, 2023.


# Big model only for hard audios: Sample dependent Whisper model selection for
efficient inferences

###### Abstract

Recent progress in Automatic Speech Recognition (ASR) has been coupled with a
substantial increase in the model sizes, which may now contain billions of
parameters, leading to slow inferences even with adapted hardware. In this
context, several ASR models exist in various sizes, with different inference
costs leading to different performance levels. Based on the observation that
smaller models perform optimally on large parts of testing corpora, we propose
to train a decision module, that would allow, given an audio sample, to use
the smallest sufficient model leading to a good transcription. We apply our
approach to two Whisper models with different sizes. By keeping the decision
process computationally efficient, we build a decision module that allows
substantial computational savings with reduced performance drops.

Index Terms—  Speech recognition, efficiency

##  1 Introduction

(a) Absolute performances

(b) Relative performances

Fig. 1: Absolute and relative performances of Whisper models on CommonVoice
test set. The four models are Whisper Tiny (39M parameters), Base (74M), Small
(244M) and Medium (759M). Each cell $i,j$ in Figure (b) represents the
percentage of utterances where the model $i$ performs at least as good as the
model $j$.

Recent progress in neural-based automatic speech recognition (ASR) has been
driven by new modelling architectures, data collection and processing but also
by larger models that have recently exceeded a billion parameters [1]. Such
advances have promised enhanced accuracy and capabilities, yet they have also
ushered in escalating computational demands. These ASR models are usually
available in a certain range of sizes with varying performance levels. For
instance, Whisper models [1] are available in 6 sizes from Tiny (39M
parameters) to Large (1.5B parameters), Nvidia FastConformers [2] range from
Large (118M parameters) to XXLarge (1.2B parameters) and self-supervised
models like Hubert [3] or WavLM [4] are generally available in Base and Large
versions. Systematically, following the deep learning trend across modalities,
larger version models, although generally trained on the same datasets,
perform substantially better than their reduced-size counterparts. This is
shown in Figure 1 (a), where the mean Word Error Rates (WER) of four Whisper
models with different sizes on the test set of CommonVoice [5] are presented.
The mean WER drops from $28.1$ with the “Tiny” version to $10.2$ with the
“Medium” one.

However, as shown in Figure 1 (b), this performance drop may not concern a
significant part of the testing points. In this figure, every cell $(i,j)$
shows the proportion of samples in the CommonVoice test set where model $i$
performs better or equally to model $j$. For instance, the third cell in the
first line (cell $(0,2)$) states that for $52\%$ of the testing samples, the
Tiny model (39M) performs equally or better than the Small one (244M) while
bearing more than $6$ times fewer parameters. Based on this observation, this
work explores whether we can predict if audio samples will fall into this
category. By doing so, audio samples that would not benefit from the costly
inference of a large model can be assigned to a smaller one in order to reduce
the total computational load.

More precisely, this study aims to develop a decision module that, given an
audio sample, chooses a Whisper model version that has the lowest inference
cost without WER degradation. Due to the complexity of the task, in this
paper, we only focus on deciding if an audio sample should be decoded with
Whisper Tiny (39M) or with Whisper Small (244M). These two model versions are
relevant candidates as they exhibit high differences in both WER, inference
costs and latency.

A few works [6, 7] have already attempted to choose among several ASR model
versions using WER prediction. Given the textual output of an ASR model, they
explored the prediction of the sentence-level WER. Yet, these methods are not
aiming to reduce inference costs but rather to decide whether an audio sample
should be retreated by a more complex ASR model. Indeed, the most efficient
techniques predict WER using full ASR pipelines based on either acoustic
encoding and language model (LM) beam search [6, 7]. Such methods that rely on
costly beam searches cannot be used in our case where the aim is to reduce the
computational load.

Another close line of work is dynamic or early-exiting approaches. Instead of
saving computation by choosing between separate ASR models, these methods have
attempted to make forward passes lighter by skipping some of the last
transformer layers of an ASR model [8, 9]. Decision to exits is based on
entropy or representation-similarity thresholds. However, early-exiting, as
developed in these works, can only save layer computation in the ASR encoder,
while, as shown in Table 1, for attentional encoder-decoder architectures,
most of the computations occur in the beam search decoding.

The closest work to our effort is from Lugosch et al. [10], who propose to
save computation cost at inference time by choosing between a large and a
small ASR decoder. However, their method relies only on the log-likelihood of
the encoder which is one of the baselines of our work.

This paper explores possibilities to build a decision module that allows
efficient selection between different ASR model sizes while keeping high
performance. Our contributions are threefold :

  * •

We successfully reduce inference costs for a negligible WER degradation. In
addition, our method can be used to interpolate between model sizes, saving
the need for costly training of intermediate models.

  * •

We explore different inputs and architectures for the decision module and
compare them to several baselines and toplines.

  * •

The codebase111https://github.com/hugomalard/Big-model-only-for-hard-
audios.git, developed within the SpeechBrain [11] framework, is released for
further investigations.

##  2 Methods

This section describes our main pipeline represented in Figure 2. For a given
audio sample, we first extract speech features and connect the output to a
decider module. This latter is responsible for choosing to transcribe the
audio with either a cheap model, Whisper Tiny, or a more expensive one,
Whisper Small.

Fig. 2: The decision module is composed of a feature extractor that encodes
the speech signal into latents that are given in input to the decider. Based
on the output of the decider, the audio sample is transcribed either by
Whisper Tiny or Whisper Small.

###  2.1 Feature extractor

One of the questions we tackle in this work is which speech features are
needed to predict how hard to transcribe is an audio sample. We explore two
levels of representations, depending on their closeness to the raw signal
waveform. We call low-level features hand-crafted representations like Mel
spectrograms or MFCCs while high-level features are typically the outputs of a
transformer encoder trained with self-supervision [4, 12] or text supervision
[1]. Intuitively, it is reasonable to think that high-level features will
provide better input to the decider module. Indeed, while low-level features
will only provide a frequency analysis of the speech input, higher-level
features provide high-quality zero-shot phonetic encoding [13] and contain
various additional information such as speaker and gender identity [14] as
well as audio backgrounds [15].

However, regarding our objective of minimizing computational cost, low-level
features may be a favorable choice compared to high-level ones. Yet, we
observed that the computational cost of representing speech with high-level
features is negligible compared to the cost of the full ASR pipeline. Indeed,
attention-based encoder-decoder models, like Whisper, are composed of an
encoder and a decoder model that have virtually the same number of parameters.
The encoder represents speech into latents and the decoder turns the latents
into text with a beam search. While the encoding only costs one forward pass
through the encoder, the decoding costs a large number of forward passes
through the decoder. For instance, we present in Table 1 the computational
cost of encoding and decoding the test set of the CommonVoice [5] with either
Whisper Tiny or Whisper Small. As expected, it appears that most of the
computation is done in the beam-search decoding.

Therefore, considering both the quality of high-level features and the
relatively low cost of encoding speech with a transformer stack, we decided to
use the Whisper Small encoder as our feature extractor. We ablate this choice
in the results section at Table 2.

Model | Encoding | Beam-search decoding
---|---|---
Whisper Tiny | 0.01T | 0.22T
Whisper Small | 0.13T | 2.49T

Table 1: Total MAC (multiply–accumulate) operations on CommonVoice test set
for the encoder and the decoder of Whisper Tiny and Small using a beam search
with n=8

###  2.2 The decider

As shown previously in Figure 1 (b), Whisper Tiny performs as well or even
better than Whisper Small in $52\%$ of the sentences in the CommonVoice
dataset. The decider from Figure 1 is a neural model that exploits this
observation. Specifically, for an audio sample $a$, the decider is trained on
the output of the frozen feature extractor to predict 1 or 0 according to the
following equation.

$g_{\mathcal{M}_{T},\mathcal{M}_{S}}(a)=\mathbbm{1}_{WER(\mathcal{M}_{T}(a))>WER(\mathcal{M}_{S}(a))}$
(1)

Where $\mathcal{M}_{T}$ and $\mathcal{M}_{S}$ are respectively Whisper Tiny
and Whisper Small. It means that the model should learn to predict 0 is the
true WER obtained with $\mathcal{M}_{T}$ is lower than the one of
$\mathcal{M}_{S}$.

In a computationally aware context, using a lightweight module as decider is
crucial, as its computational cost will be systematically paid at each
inference. Using a transformer stack for the decider would induce a high
inference cost due to the high dimensionality of the output of the feature
extractor. A convolutional stack, scaling linearly with the sequence length,
does not suffer from those side effects. Moreover, the locality bias of the
convolutions may be pertinent in this context since errors may occur at
localized segments of the speech sample. Therefore, we opted for a one-
dimensional small ResNet [16] for the decider module architecture.

Instead of simply connecting the feature extractor output to the decider
input, we learn a weighted sum of the feature extractor layers as in [17].
This method exploits the fact that different layers of a transformer stack
encode different types of information from the audio signal [18].

Here is a summary of our pipeline at inference time for transcribing one given
audio sample, $a$. First, $a$ is turned into latents using Whisper Small
encoder. The decider computes a learned weighted average of the encoder layers
before doing a forward pass in a small 1D ResNet. If the decider output value
falls below a threshold $h$, the beam search is performed in the Whisper Small
decoder. Otherwise, $a$ is encoded and beam-search decoded with Whisper Tiny.

##  3 Experiments

In this section, we describe the datasets and hyperparameters used for the
training of the decider model. In addition, we present baselines and toplines
that act as comparison points to our method.

###  3.1 Datasets and setups

In this study, two datasets are considered: LibriSpeech [19] and CommonVoice
7.0 [5]. The LibriSpeech corpus is composed of read English speech recordings
with 960 hours for training, two dev splits dev-clean and dev-other and two
test splits test-clean and test-other of 5 hours each. CommonVoice is a
collection of speech samples from worldwide users recording themselves with
their own devices covering a large variety of age, gender and accents. The
English part of the dataset presents roughly 1260 hours of recorded audio.

Our decider module is a small ResNet [16], with 3 ResBlocks of two
convolutional layers, each with 256 feature maps. The output of the ResNet is
average-pooled before going through a linear layer with one sigmoid output
neuron. In addition, the decider has one learnable weight per feature
extractor layer. The feature extractor remains frozen during the decoder
training. Training is done with binary cross entropy using Adam[20] optimizer
and a learning rate of $10^{-5}$ with cosine annealing.

###  3.2 Baselines and Toplines

In order to compare our decision module from Section 2 to simpler methods with
threshold-based decisions, we present in this subsection three considered
baselines.

First, based on the well-known impact of noise on ASR quality, the output of a
blind Signal to Noise Ratio (SNR) estimator [21] is used for the decision
module. Here, the decider relies on a simple threshold, determined by equal
error rate: it runs Whisper Small if the computed SNR is lower than the
threshold, otherwise it uses Whisper Tiny. A second baseline consists of using
an accent detection model [22] to assign audios of rarer accents to the larger
model. Indeed, [23] shows a significant effect of English accents on ASR
performances. For this baseline, the decision module consists in selecting
Whisper Tiny if the English accent detected is either American, British or
Canadian.

Finally, inspired by Lugosch and al. [10], a third baseline explores the use
of ASR encoder-decoder logits as a confidence measure. Precisely, using
Whisper Tiny, we perform a full greedy decoding, compute the entropy of the
logit probabilities for each time-step, then aggregate with mean pooling over
the time dimension. Here again, the decider is a threshold on entropy values
that is set on a validation set as for the SNR baseline.

Regarding toplines, we propose two decision modules that are voluntarily
unrealistic in order to show the potential computational savings in an ideal
case scenario. The first topline is an oracle that knows the true value of
equation 1 for any audio sample. For the second topline, we assume that the
WER of Whisper Tiny is known in advance for any audio sample. The decider here
is a threshold on WER values set on a validation set to determine when to use
the larger Whisper model. The threshold is chosen such that it corresponds to
the larger value for which there is no false negative (i.e choosing “Tiny”
while it performs less well than “Small”).

##  4 Results and Discussion

###  4.1 Accuracy of decision modules

Table 2 presents the decision modules accuracies on equation 1. It consists,
for a collection of audio samples, in the percentage of times a decision
module correctly assigns audio samples to the appropriate Whisper model as
defined in Equation 1.

The three first rows present the accuracy of the three baselines. The two
first ones achieve accuracies close to random performance, which shows that
neither SNR nor accent seems to capture the difficulty of the audio sample.
For SNR, we hypothesize that this is due to the fact that the recordings come
from relatively clean backgrounds, leading to very noisy and poorly
informative estimation from the blind SNR estimator. On the contrary, logit-
level entropy, although being computationally costly gives significantly
better results reaching $64.5\%$ accuracy. It seems to indicate that the model
internal states contain useful information about the difficulty of decoding
audio samples.

The second part of the table considers different feature extractors to our
ResNet decider. As expected, higher-level features perform better than
80-dimensional Mel Spectrograms. The encoder of Whisper Small model performs
better than the Wav2vec2.0 in our setting with $68.2\%$ accuracy, compared to
$63.9\%$ for Wav2Vec2.0 features. This suggests that model-related features
are the best adapted to the decision task.

Finally, the oracle based on a threshold over the WER of Whisper Tiny produces
strong results, showing that the WER of smaller Whisper version can be used to
allocate audio samples to large models efficiently. However, blind WER
prediction remains a noisy and computationally costly endeavour [7].

We ablate the architectural choice of the decider model in Table 3. First,
removing the learned weighted sum of encoder layers slightly degrades the
accuracy. Second, using a one-layer transformer network with a roughly equal
number of parameters to the ResNet, apart from raising the cost of the
decision, also degrades accuracy scores by a couple of percent. Finally,
inspired by [15], we implemented a small TL-transformer architecture which is
composed of one transformer layer on the encoder layers outputs and another
one on their pooled (time-wise) representations. This approach scores even
lower than the simple one-layer transformer architecture.

###  4.2 WER/MACs trade-off

Table 4 shows the trade-off WER/MACs obtained with our pipeline compared to
transcribing speech using Whisper models. MACs, which stands for
multiply–accumulate operations, is our measure of computational cost. It is
important to note that the MACs column shows the cost of the full pipeline for
transcribing the CommonVoice test set, including the cost of the decision
module when there is one. Table 4 starts with the WER/MACs of simply
transcribing speech using the different Whisper models with beam search. Then,
we include the logit entropy of Whisper Tiny, which is the best baseline from
Table 2. This baseline increases the WER by an absolute $0.7$ points while
reducing the model computational cost by $150G$. Comes next, our main pipeline
that uses a decision module composed of Whisper Small encoder and Resnet. For
this latter, we provide scores at two different thresholds, $0.3$ and $0.5$.
By comparison with simply running Whisper Small, selecting a threshold of
$0.5$ on the sigmoid output of our decision module gives a $16\%$ higher WER
while resulting in a $35\%$ decrease in MACs. Using a threshold of $0.3$
increases the WER by an absolute $0.37$ points while reducing the model
computational cost by $310G$ MACs (i.e. $12\%$ of the total cost). Finally,
the last 2 lines show the hypothetical improvements that can be obtained using
a perfect decider, or a threshold, based on a perfect estimation of the WER of
$\mathcal{M}_{T}$. Not only they allow to reduce the MACs, but they also
reduce significantly the computational load. These toplines confirm the
potential of the approach and call for further research on model size
assignment.

Figure 3 shows the performances (WER) and computational costs (MACs) of the
intermediate models obtained using our main pipeline (i.e. Whisper Small
encoder and ResNet). Almost all the points are under the plotted diagonal,
which means that the resulting drop in performance (relative to the larger
model) is systematically smaller than the gain in computational cost. Whisper
Base is included in Figure 3 as it is an intermediate model between Whisper
Tiny and Small. Its WER/MACs is only slightly better than our selection
approach. This shows, that the method presented yields nearly equivalent
performance to that of an intermediate model trained entirely from scratch,
saving very costly training.

Feature extractor | Decider | test-clean$\uparrow$ | test-other$\uparrow$ | CV test$\uparrow$
---|---|---|---|---
SNR [21] | thresh. | 50.7 | 47.2 | 47.0
Accent [22] | thresh. | n/a | n/a | 52.0
$\mathcal{M}_{T}$ logit entropy | thresh. | 64.4 | 63.7 | 64.5
Mel f-banks | ResNet | 62.5 | 55.2 | 60.0
Wav2Vec2.0 Base | ResNet | 52.3 | 57.7 | 63.9
$\mathcal{M}_{T}$ encoder | ResNet | 65.1 | 65.0 | 66.4
$\mathcal{M}_{S}$ encoder | ResNet | 68.0 | 66.6 | 68.2
WER $\mathcal{M}_{T}$ | thresh. | 84.8 | 80.7 | 80.3

Table 2: Accuracies (higher the better) of the several decision modules on
LibriSpeech and CommonVoice test sets. SNR, Accent and Logit Entropy are
baseline models that only require to fit a threshold on a validation set. When
the Decider is ResNet, a dedicated ResNet is trained on each of the different
feature extractors. The last line is a topline model based on an oracle that
provides the WER of Whisper Tiny. $\mathcal{M}_{T}$ and $\mathcal{M}_{S}$ are
respectively Whisper Tiny and Small.

Decider | Test-Clean$\uparrow$ | Test-Other$\uparrow$
---|---|---
ResNet | 68.0 | 66.6
ResNet w/o weighted | 66.4 | 66.0
Transformer | 66.2 | 65.4
TL-Transformer | 64.4 | 55.2

Table 3: Accuracy (higher the better) obtained by various decider models on
LibriSpeech test sets using Whisper Small encoder as a feature extractor. ’w/o
weighted’ means that the decider does not learn a weighted average of the
encoder layers. Transformer has one transformer layer and TL-transformer is a
1 layer replication of [15].

Method | WER$\downarrow$ | MACs$\downarrow$
---|---|---
$\mathcal{M}_{T}$ | 28.1 | 0.23T
$\mathcal{M}_{Base}$ | 20.7 | 0.60T
$\mathcal{M}_{S}$ | 13.3 | 2.62T
$\mathcal{M}_{T}$ logit entropy | 14.0 | 2.47T
encoder $\mathcal{M}_{S}$ \+ ResNet @$0.5$ | 15.4 | 1.72T
encoder $\mathcal{M}_{S}$ \+ ResNet @$0.3$ | 13.7 | 2.31T
WER $\mathcal{M}_{T}$ Oracle | 12.93 | 1.954T
Oracle | 12.27 | 1.468T

Table 4: Average WER and MACs (lower the better) associated with each method
on the CommonVoice test set. The table starts with the performances of
$\mathcal{M}_{T}$, $\mathcal{M}_{Base}$ and $\mathcal{M}_{S}$ which are full
ASR pipeline of respectively Whisper Tiny, Base and Small. $\mathcal{M}_{T}$
logit entropy is our baseline. encoder $\mathcal{M}_{S}$\+ ResNet is our main
contribution for which we give performances at two different theshold value
(0.3 and 0.5). The last 2 lines are our toplines.

Fig. 3: WER/MACs values on the CommonVoice test set for different pipelines.
The blue dots are the values for multiple thresholds using our best decision
module (Whisper Small encoder and ResNet). The crosses correspond to the
Whisper models Tiny, Base and Small respectively. Finally, the black line is a
linear interpolation between the MACs and WER of Whisper Tiny and Whisper
Small

###  4.3 Discussion

The failure of the baselines and the encoder layers being the best performing
input, tend to show that the errors are very dependent of the ASR model,
rather than complexities inherent to the audio signal that would make any ASR
model fail to transcribe.

To investigate this, we compute correlation values between the WER of a
Conformer Large [24] model and of a Wav2Vec2.0 Base model fine tuned on
LibriSpeech, with the WER of Whisper tiny, on the LibriSpeech test sets
(combined clean and other).
The Pearson correlation coefficient between the WER of the Conformer model and
the WER of Whisper Tiny reaches only $0.44$, while the Spearman correlation is
only $0.41$. Similarly, these two correlation quantities between the WER of
Wav2Vec2.0 and Whisper Tiny reach respectively $0.51$ and $0.45$. The low
correlation values and the weak monotonic relationship seem to indicate that
models have different intrinsic failure cases. It confirms, as our results
have first shown, that a successful model selection approach needs model-
related inputs.

##  5 Conclusion

In this study, we explored a new, computationally efficient approach, that
selects for an audio sample the most efficient model among two of different
sizes. It can be applied to interpolate between two trained models of fixed
sizes without additional training, reducing the relative computational cost
more than it degrades performances.

##  6 Acknowledgements

This work was funded in part by the European Research Council
(ERC-2011-AdG-295810 BOOTPHON), the Agence Nationale pour la Recherche
(ANR-17-EURE-0017 Frontcog, ANR-10-IDEX-0001-02 PSL*, ANR-19-P3IA-0001 PRAIRIE
3IA Institute) and grants from CIFAR (Learning in Machines and Brains), Meta
AI Research (Research Grant), Google (Faculty Research Award), Microsoft
Research (Azure Credits and Grant), and Amazon Web Service (AWS Research
Credits). Furthermore, this work was performed using HPC resources from GENCI-
IDRIS (Grant 2021-[AD011011217])

## References

  * [1] Alec Radford, Jong Wook Kim, Tao Xu, Greg Brockman, Christine McLeavey, and Ilya Sutskever,  “Robust speech recognition via large-scale weak supervision,” 2022.
  * [2] Dima Rekesh, Nithin Rao Koluguri, Samuel Kriman, Somshubra Majumdar, Vahid Noroozi, He Huang, Oleksii Hrinchuk, Krishna Puvvada, Ankur Kumar, Jagadeesh Balam, and Boris Ginsburg,  “Fast conformer with linearly scalable attention for efficient speech recognition,” 2023.
  * [3] Wei-Ning Hsu, Benjamin Bolte, Yao-Hung Hubert Tsai, Kushal Lakhotia, Ruslan Salakhutdinov, and Abdelrahman Mohamed,  “Hubert: Self-supervised speech representation learning by masked prediction of hidden units,”  CoRR, vol. abs/2106.07447, 2021.
  * [4] Sanyuan Chen, Chengyi Wang, Zhengyang Chen, Yu Wu, Shujie Liu, Zhuo Chen, Jinyu Li, Naoyuki Kanda, Takuya Yoshioka, Xiong Xiao, Jian Wu, Long Zhou, Shuo Ren, Yanmin Qian, Yao Qian, Jian Wu, Michael Zeng, and Furu Wei,  “Wavlm: Large-scale self-supervised pre-training for full stack speech processing,”  CoRR, vol. abs/2110.13900, 2021.
  * [5] Rosana Ardila, Megan Branson, Kelly Davis, Michael Henretty, Michael Kohler, Josh Meyer, Reuben Morais, Lindsay Saunders, Francis M. Tyers, and Gregor Weber,  “Common voice: A massively-multilingual speech corpus,” 2020.
  * [6] Ahmed Ali and Steve Renals,  “Word error rate estimation without asr output: e-wer2,” 2020.
  * [7] Shammur Absar Chowdhury and Ahmed Ali,  “Multilingual word error rate estimation: e-wer3,” 2023.
  * [8] Salah Zaiem, Robin Algayres, Titouan Parcollet, Slim Essid, and Mirco Ravanelli,  “Fine-tuning strategies for faster inference using speech self-supervised models: A comparative study,” 2023.
  * [9] Ji Won Yoon, Beom Jun Woo, and Nam Soo Kim,  “Hubert-ee: Early exiting hubert for efficient speech recognition,” 2022\.
  * [10] Loren Lugosch, Derek Nowrouzezahrai, and Brett H. Meyer,  “Surprisal-triggered conditional computation with neural networks,” 2020\.
  * [11] Mirco Ravanelli, Titouan Parcollet, Peter Plantinga, Aku Rouhe, Samuele Cornell, Loren Lugosch, Cem Subakan, Nauman Dawalatabad, Abdelwahab Heba, Jianyuan Zhong, et al.,  “Speechbrain: A general-purpose speech toolkit,”  arXiv preprint arXiv:2106.04624, 2021.
  * [12] Alexei Baevski, Henry Zhou, Abdelrahman Mohamed, and Michael Auli,  “wav2vec 2.0: A framework for self-supervised learning of speech representations,” 2020.
  * [13] Morgane Rivière, Armand Joulin, Pierre-Emmanuel Mazaré, and Emmanuel Dupoux,  “Unsupervised pretraining transfers well across languages,” 2020.
  * [14] Maureen de Seyssel, Marvin Lavechin, Yossi Adi, Emmanuel Dupoux, and Guillaume Wisniewski,  “Probing phoneme, language and speaker information in unsupervised speech representations,”  in Interspeech 2022. sep 2022, ISCA.
  * [15] Yuan Gong, Sameer Khurana, Leonid Karlinsky, and James Glass,  “Whisper-at: Noise-robust automatic speech recognizers are also strong general audio event taggers,” 2023.
  * [16] Kaiming He, X. Zhang, Shaoqing Ren, and Jian Sun,  “Deep residual learning for image recognition,”  2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR), pp. 770–778, 2015.
  * [17] Shu-Wen Yang, Po-Han Chi, Yung-Sung Chuang, Cheng-I Jeff Lai, Kushal Lakhotia, Yist Y. Lin, Andy T. Liu, Jiatong Shi, Xuankai Chang, Guan-Ting Lin, Tzu-Hsien Huang, Wei-Cheng Tseng, Ko-tik Lee, Da-Rong Liu, Zili Huang, Shuyan Dong, Shang-Wen Li, Shinji Watanabe, Abdelrahman Mohamed, and Hung-yi Lee,  “SUPERB: speech processing universal performance benchmark,”  CoRR, vol. abs/2105.01051, 2021.
  * [18] Salah Zaiem, Youcef Kemiche, Titouan Parcollet, Slim Essid, and Mirco Ravanelli,  “Speech self-supervised representations benchmarking: a case for larger probing heads,”  arXiv preprint arXiv:2308.14456, 2023.
  * [19] Vassil Panayotov, Guoguo Chen, Daniel Povey, and Sanjeev Khudanpur,  “Librispeech: An asr corpus based on public domain audio books,”  in 2015 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP), 2015, pp. 5206–5210.
  * [20] Diederik P. Kingma and Jimmy Ba,  “Adam: A method for stochastic optimization,” 2017.
  * [21] Marvin Lavechin, Marianne Métais, Hadrien Titeux, Alodie Boissonnet, Jade Copet, Morgane Rivière, Elika Bergelson, Alejandrina Cristia, Emmanuel Dupoux, and Hervé Bredin,  “Brouhaha: multi-task training for voice activity detection, speech-to-noise ratio, and c50 room acoustics estimation,” 2023.
  * [22] Juan Zuluaga-Gomez, Sara Ahmed, Danielius Visockas, and Cem Subakan,  “Commonaccent: Exploring large acoustic pretrained models for accent classification based on common voice,” 2023.
  * [23] Mikel K. Ngueajio and Gloria Washington,  “Hey ASR system! why aren’t you more inclusive?,”  in Lecture Notes in Computer Science, pp. 421–440. Springer Nature Switzerland, 2022.
  * [24] Anmol Gulati, James Qin, Chung-Cheng Chiu, Niki Parmar, Yu Zhang, Jiahui Yu, Wei Han, Shibo Wang, Zhengdong Zhang, Yonghui Wu, and Ruoming Pang,  “Conformer: Convolution-augmented transformer for speech recognition,” 2020.


# The sixth order cumulant of net-proton number in Binomial distribution at
$\sqrt{s_{NN}}=$ 200 GeV††thanks: This work was supported by the postdoctral
science and technology project of Hubei Province under Grant No. 2018Z27, and
the Fundamental Research Funds for the Central Universities under Grant No.
CCNU19ZN019.

Li-Zhu Chen1;1) Ye-Yin Zhao2 Jin Wu 2 Zhi-Ming Li2 Yuan-Fang Wu2
<EMAIL_ADDRESS>1School of Physics and Optoelectronic Engineering, Nanjing
University of Information Science and Technology, Nanjing 210044, China
2Key Laboratory of Quark and Lepton Physics (MOE) and Institute of Particle
Physics, Central China Normal University, Wuhan 430079, China

###### Abstract

It is proposed that ratios of the sixth order to the second order cumulant
($C_{6}/C_{2}$) of conserved quantities are sensitive to the chiral crossover
transition. Recently, the negative $C_{6}/C_{2}$ was obtained both in
theoretical Lattice QCD and experiments at $\sqrt{s_{NN}}=$ 200 GeV. In this
study, we investigate the behavior of net-proton $C_{6}/C_{2}$ in statistical
Binomial distribution (BD) at $\sqrt{s_{NN}}=$ 200 GeV in Au + Au collisions.
With the BD parameters extracted from RHIC/STAR, it is found that
$C_{6}/C_{2}$ can be negative. Furthermore, the obtained $C_{6}/C_{2}$ becomes
smaller when applying the same magnitude of experimental statistics and
calculation method to simulations. In 0-10% centrality, there is a significant
difference between the simulated result and theoretical expectation. Based on
the extracted parameters and experimentally collected statistics, the baseline
of net-proton $C_{6}/C_{2}$ in BD is presented.

###### keywords:

The sixth order cumulant, Net-proton number, Binomial distribution, Statistics

###### pacs:

2

5.75.Gz, 25.75.Nq

##  1 Introduction

One of the main goals of the Beam Energy Scan (BES) Program at the
Relativistic Heavy Ion Collider (RHIC) is to explore the phase diagram of the
Quantum Chromodynamics (QCD) [2]. At vanishing and small values of the
chemical potentials for conserved charges, such as net-baryon, net-charge and
net-strangeness, Lattice QCD has predicted that it occurs a smooth crossover
from the hadronic phase to the QGP phase [3]. Based on Lattice QCD
calculations with physical values of light and strange quark masses, the
negative signal of ratio of the sixth order to the second order
($C_{6}/C_{2}$) is observed in the crossover region [4, 5, 6, 7]. The QCD-
assisted low-energy effective theory and the QCD based models, such as the
Polyakov loop extended quark-meson (PQM) and the Nambu-Jona-Lasinio (PNJL)
models, also support that the sixth order cumulant is negative near the chiral
crossover transition [8, 9, 10, 11, 12]. Consequently, $C_{6}/C_{2}$ is a
powerful observable to study the QCD phase diagram in experiment.

The STAR Collaboration had reported the preliminary results of net-proton
$C_{6}/C_{2}$ [13, 14, 15]. In 0-40% centrality, $C_{6}/C_{2}$ is negative at
$\sqrt{s_{NN}}$ = 200 GeV, which is consistent with the Lattice QCD
calculation. However, $C_{6}/C_{2}$ is positive and close to unity at
$\sqrt{s_{NN}}$ = 54.4 GeV, unlike the QCD predicted negative behavior. In the
future, with the sufficient statistics accumulated, the analysis of the net-
proton $C_{6}/C_{2}$ is possible by the ALICE collaboration at LHC [16].
Before illustrating the physics of measured $C_{6}/C_{2}$, the contributions
of non-phase transition related influences should be subtracted, such as the
conservation of the total baryon number, experimental acceptances in terms of
kinematic variables, efficiency corrections, the difference of $C_{6}/C_{2}$
in net-proton and net-baryon and so on [17, 18, 19, 20]. It was shown that
$C_{6}/C_{2}$ can be negative for $\sqrt{s_{NN}}\leq$ 40 GeV due to the baryon
number conservation [21]. In this paper, we will focus on discussions of the
BD contribution and influence of the statistics used for data analysis at
$\sqrt{s_{NN}}$ = 200 GeV.

In nuclear collisions, some basic statistical distributions, such as the
Poisson distribution, BD and Negative Binomial distribution (NBD), are
frequently used to describe the shape of the multiplicity distributions [22,
23, 24, 25]. Theoretically, in the free hadron gas in equilibrium, the
resonance gas in the hadron phase obeys the Poisson distributions. It was also
argued that the proton number is given by the superposition of the binomial
distribution of the baryon number due to the isospin randomization [26].
Consequently, studies of the cumulants in different statistical distributions
can extract information related to the nature of the particle production
mechanism.

Recently, cumulants in these statistical distributions were widely taken as
baselines to help us understand the experimentally measured cumulants, such
the the cumulants of the net-proton, net-charge, net-kaon and net-$\Lambda$
[27, 28, 29, 30, 31, 32, 33]. To date, the studies of cumulants in BD/NBD have
been up to the fourth order. It is found that the BD/NBD baselines can
quantitatively explain the cumulants of net-charge, net-kaon and net-$\Lambda$
in experiment. The BD can also describe the fourth order cumulant of the net-
proton number at $\sqrt{s_{NN}}\geq$ 39 GeV. Consequently, it is also
interesting to study the net-proton $C_{6}/C_{2}$ in BD.

With the data accumulated, STAR has about 850M events for analysis of
$C_{6}/C_{2}$ at $\sqrt{s_{NN}}$ = 200 GeV [14]. However, the data used in the
analysis include three independent parts: (a) around 420M events for 0-80%
centrality collected from the year 2010 minimum bias (MB) trigger, (b) around
110M events for 0-10% centrality from the year 2010 central trigger, and (c)
around 320M events for 0-80% centrality from the year 2011 MB trigger. In
principle, $C_{6}/C_{2}$ in each centrality should be firstly calculated by
these three independent parts separately. For each part, $C_{6}/C_{2}$ is
obtained by the method of centrality bin width correction (CBWC) [34, 35, 36].
It means $C_{6}/C_{2}$ needs to be calculated first in each Refmult3 bin
($N_{ch}$) to reduce the initial size fluctuation. Then, $C_{6}/C_{2}$ is
averaged over all $N_{ch}$ in a given centrality.

A critical issue is that the statistics in each of $N_{ch}$ are still
considerably limited even with 320M MB events. Our previous studies showed
that we must check whether the statistics in each of $N_{ch}$ are sufficient
to satisfy the Central Limit Theorem (CLT) when applying the CBWC method in
cumulants analysis [37, 38, 39]. It showed that 90M MB events are not
sufficient for analysis of net-proton $C_{6}/C_{2}$ at $\sqrt{s_{NN}}=11.5$
GeV in the UrQMD model. These statistics are of the same order of magnitude as
that in each data collection at $\sqrt{s_{NN}}=$ 200 GeV at RHIC/STAR. In
addition, the required statistics are also related to the detail shape and
width of the net-proton multiplicity distributions. Therefore, we need to re-
examine if we can obtain reliable $C_{6}/C_{2}$ with 320M MB events.

(Color Online) The left panel shows $\varepsilon_{p}$ and
$\varepsilon_{\bar{p}}$ in different centralities at $\sqrt{s_{NN}}=$ 200 GeV
in Au + Au collisions measured by RHIC/STAR [36]. The protons and anti-protons
are identified at mid-rapidity $(|y|<0.5)$ within $0.4<p_{T}<2.0$ GeV/$c$. The
right panel is the $\varepsilon$ dependence of $C_{6}/C_{2}$.

In this paper, we will start off the discussions from behavior of net-proton
$C_{6}/C_{2}$ in BD. At $\sqrt{s_{NN}}$ = 200 GeV, it is found that the
negative $C_{6}/C_{2}$ can be obtained with the experimentally measured
parameters. To study influence of the statistics on $C_{6}/C_{2}$, a toy
simulation about validation of the CLT in $C_{6}/C_{2}$ is demonstrated in
Section 3. To make a direct comparison with experimental data, more detail
procedures of simulations are described in Section 4. With the experimentally
collected statistics and extracted BD parameters, we show that there is a
marked drop in $C_{6}/C_{2}$ in 0-10% centrality in Section 5. The baseline of
$C_{6}/C_{2}$ is also demonstrated. Finally, the summary is presented in
Section 6.

##  2 $C_{6}/C_{2}$ in BD

In probability theory and statistics, the BD is the discrete probability
distribution of the number of successes in a sequence of $n$ independent
binary experiments, where the results of each experiment is true with
probability $p$ and false with probability $q=1-p$ [40]. The probability that
the binomial random variable $x$ takes on values in its range can be expressed
using the binomial probability function:

$P(x)=\left(\begin{array}[]{c}n\\\
x\end{array}\right)p^{x}(1-p)^{n-x}=\frac{n!}{x!(n-x)!}p^{x}(1-p)^{n-x},$ (1)

where $x$ corresponds to the number of protons or anti-protons in each event.

Experimentally, if we know the mean $\mu$ and variance $\sigma^{2}$
($\sigma^{2}<\mu$ for a BD, while $\sigma^{2}>\mu$ for a NBD) of the
distribution, then the input parameters of $p$ and $n$ are:

$p=1-\frac{\sigma^{2}}{\mu}=1-\varepsilon,$ (2)

and

$n=\frac{\mu}{p}=\frac{\mu}{1-\varepsilon},$ (3)

where $\varepsilon=\frac{\sigma^{2}}{\mu}$.

With given $\mu$ and $\varepsilon$, the expectations of cumulants from the
second to the sixth order can be written as:

$\displaystyle C_{2}$ $\displaystyle=$ $\displaystyle
n\left(p-p^{2}\right)=\varepsilon\mu$ (4) $\displaystyle C_{3}$
$\displaystyle=$ $\displaystyle
n\left(p-3p^{2}+2p^{3}\right)=\varepsilon\mu\left(-1+2\varepsilon\right)$ (5)
$\displaystyle C_{4}$ $\displaystyle=$ $\displaystyle
n\left(p-7p^{2}+12p^{3}-6p^{4}\right)=\varepsilon\mu\left(1-6\varepsilon+6\varepsilon^{2}\right)$
(6) $\displaystyle C_{5}$ $\displaystyle=$ $\displaystyle
n\left(p-15p^{2}+50p^{3}-60p^{4}+24p^{5}\right)$ (7) $\displaystyle=$
$\displaystyle\varepsilon\mu\left(-1+14\varepsilon-36\varepsilon^{2}+24\varepsilon^{3}\right)$
$\displaystyle C_{6}$ $\displaystyle=$ $\displaystyle
n\left(p-31p^{2}+180p^{3}-390p^{4}+360p^{5}-120p^{6}\right)$ (8)
$\displaystyle=$
$\displaystyle\varepsilon\mu\left(1-30\varepsilon+150\varepsilon^{2}-240\varepsilon^{3}+120\varepsilon^{4}\right)$

If the numbers of protons and anti-protons are independently produced as BD,
the net-proton $C_{6}/C_{2}$ can be expressed as:

$C_{6}/C_{2}=\frac{C_{6}^{p}+C_{6}^{\bar{p}}}{C_{2}^{p}+C_{2}^{\bar{p}}}.$ (9)

Generally, the expected $C_{6}/C_{2}$ is related to $\varepsilon_{p}$,
$\varepsilon_{\bar{p}}$, $\mu_{p}$, and $\mu_{\bar{p}}$. Based on these four
parameters, one can obtain the expected $C_{6}/C_{2}$ in each centrality. In
contrast, the experimental studies had shown that $\varepsilon_{p}$ and
$\varepsilon_{\bar{p}}$ are close to each other at $\sqrt{s_{NN}}=200$ GeV in
Au + Au collisions shown in Fig. 1(a) [36]. The error contains the statistical
and systematical uncertainties, which is performed by
$\sigma_{\varepsilon}=\sqrt{\sigma^{2}_{stat}+\sigma^{2}_{sys}}$. The protons
and anti-protons are selected at mid-rapidity $(|y|<0.5)$ within
$0.4<p_{T}<2.0$ GeV/c. It shows $\varepsilon_{p}$ and $\varepsilon_{\bar{p}}$
extracted from STAR are consistent with each other. Within
1$\sigma_{\varepsilon}$ of uncertainty, the centrality dependence of
$\varepsilon$ is weak. To make an appropriate approximation, we assume
$\varepsilon=\varepsilon_{p}=\varepsilon_{\bar{p}}$ in this paper. In this
case, the expectation of net-proton $C_{6}/C_{2}$ can be written as:

$C_{6}/C_{2}=1-30\varepsilon+150\varepsilon^{2}-240\varepsilon^{3}+120\varepsilon^{4}.$
(10)

Eq. (10) shows the expectation of $C_{6}/C_{2}$ is only dependent on
$\varepsilon$. The effects of $\mu_{p}$ and $\mu_{\bar{p}}$ are cancelled. The
detailed $\varepsilon$ dependence of $C_{6}/C_{2}$ is shown in Fig. 1(b). It
shows $C_{6}/C_{2}$ drops drastically with the decrease of $\varepsilon$. It
is already negative with $\varepsilon<0.958$, which is within the
experimentally measured range. In addition, $C_{6}/C_{2}$ has a broad value
range and it can change from positive to negative, within
1$\sigma_{\varepsilon}$ uncertainty of $\varepsilon$. Consequently, it is not
suitable to directly give the expectation only based on the unique measured
value of $\varepsilon$, without considering its uncertainty. Instead, we must
set an interval of $\varepsilon$ to study the behavior of $C_{6}/C_{2}$. Fig.
1(a) shows the upper and lower values of the error bar touch $\varepsilon$ at
approximately 0.99 and 0.94, respectively. If we assume the range of
$\varepsilon$ is between 0.94 and 0.99, then the expected values of
$C_{6}/C_{2}$ are from -0.31 to 0.71. The negative $C_{6}/C_{2}$ can be
obtained in the pure statistical BD. Only the negative signal is not enough to
be taken as an indication of a smooth crossover transition.

Here the obtained $C_{6}/C_{2}$ is the ideal theoretical expectation. In the
experiment, the statistics are still a critical issue for the analysis of
$C_{6}/C_{2}$. The satisfaction of the CLT requires to be carefully checked
before the data analysis.

##  3 Central Limit Theorem in $C_{6}/C_{2}$

(Color Online) Statistics dependence of $\left<Mean\right>$ and
$\left<C_{6}/C_{2}\right>$, respectively.

(Color Online) The left panel shows normalized probability distribution of
Refmult3 in UrQMD model in Au + Au collisions at $\sqrt{s_{NN}}=$ 200 GeV. The
middle and right panels show $\left<N_{p}\right>$ and
$\left<N_{\bar{p}}\right>$ as a function of Refmult3, respectively.

It is known that if the statistics are sufficient for validation of CLT [41],
then the experimentally measured results should always be consistent with the
true value within 1$\sigma$, 2$\sigma$ and 3$\sigma$ of uncertainties with
probability of 68.3$\%$, 95.5$\%$ and 99.7$\%$, respectively. For most of the
measurements, such as the mean of the net-proton number, the required
statistics for CLT are easily achieved in experiments. In this case, within
the uncertainties, the mean of the measured observable needs to be a real
value and independent of the statistics. i.e.,

$\left<X\right>_{n_{1}}=\left<X\right>_{n_{2}}=\left<X\right>_{n_{3}},$ (11)

where $X$ is the measured observable. The subscripts $n_{1}$, $n_{2}$ and
$n_{3}$ denote different statistics. As an example, Fig. 3(a) shows the
simulated $\left<Mean\right>$ of the net-proton number as a function of the
statistics. The BD is applied in simulations with parameters:
$\mu_{p}=12.66,\mu_{\bar{p}}=7.5,\varepsilon_{p}=\varepsilon_{\bar{p}}=0.97$.
$\mu_{p}$ and $\mu_{\bar{p}}$ are mean values of proton and anti-proton in
$0-10\%$ centrality with $0.4<p_{T}<2.0$ GeV/c and $|y|<0.5$. For each data
point, we randomly and independently generate 50 sub-samples with the fixed
statistics to calculate $\left<Mean\right>$ as:

$\left<Mean\right>=\frac{\sum_{i=1}^{i=N}(Mean)_{i}}{N},$ (12)

where $(Mean)_{i}$ is the averaged number of net-proton in $i_{th}$ sub-
sample. There are two methods to estimate error($\left<Mean\right>$). One is
obtained by the formula of the error propagation:

$error(\left<Mean\right>)=\frac{\sqrt{\sum_{i=1}^{i=N}error(Mean)_{i}^{2}}}{N}.$
(13)

We can also first calculate the width of $\left<Mean\right>$ based on these
$N$ results. Then error($\left<Mean\right>$) is the width of
$\left<Mean\right>$ divided by $\sqrt{N}$. A Good agreement is obtained based
on these two methods. In this paper, the formula of error propagation is used
to measure the statistical uncertainty.

The simulated statistics in Fig. 3(a) are from 50 to 5,000 events. The black
dashed line is the theoretical expectation. For mean of the net-proton number
analysis, Fig. 3(a) clearly shows that 50 events are sufficient for the
validation of the CLT. That is why we do not require to check whether the
statistics are sufficient for most of the observables.

However, it is a challenge for analysis of the high-order cumulants which are
up to the fifth order, sixth order, or even the eighth order [37, 38, 39].
When the CBWC method is applied in cumulants calculations, the statistics in
each $N_{ch}$ are significantly limited in $0-10\%$ centrality even with a few
hundred million MB events. As a crude estimation, supposing there are 1000
$N_{ch}$ bins in 0-10$\%$ centrality, the averaged events are only
approximately 10,000 in each $N_{ch}$ with 100M MB events. If 10, 000 events
are not sufficient for CLT in $C_{6}/C_{2}$ calculations, the value obtained
by the CBWC method is not reliable in 0-10% centrality with 100M MB events.

By using the same simulated parameters as Fig. 3(a), while the simulated sub-
sample $N$ is significantly larger than 50, the statistics dependence of
$\left<C_{6}/C_{2}\right>$ from 10,000 to 3M events are demonstrated in Fig.
3(b). Below 0.1M events in each $N_{ch}$, $\left<C_{6}/C_{2}\right>$ is
systematically smaller than 0 and theoretical BD expectations. Up to 0.5M
events in each $N_{ch}$, $\left<C_{6}/C_{2}\right>$ is consistent with
theoretical expectations within statistical uncertainties.

As we mentioned, the required statistics are also related to the detail shape
and width of the net-proton multiplicity distributions [37, 38, 39]. The
behavior of the statistical dependency in different centralities requires a
careful case-by-case study. To directly compare, we study the statistics
dependence of $C_{6}/C_{2}$ in BD with the same calculation method as
RHIC/STAR.

##  4 Simulation Setup

(Color Online) $C_{6}/C_{2}$ and their statistical errors of 200 samples in
0-10% and 30-40% centrality, respectively. Each of $C_{6}/C_{2}$ is calculated
by the CBWC method with 300M MB events. The protons and anti-protons are
independently and randomly generated from the BD with $\varepsilon=0.97$. The
input parameters for simulations are taken from UrQMD model at
$\sqrt{s_{NN}}=$ 200 GeV in Au + Au collisions.

To study the statistics dependence of $C_{6}/C_{2}$ measurement based on the
CBWC method, the input parameters for simulations are taken from the UrQMD
model at $\sqrt{s_{NN}}=$ 200 GeV in Au + Au collisions [42, 43].

Fig. 3(a) shows the normalized probability distribution of Refmult3, which is
used for the centrality selection. To avoid the auto-correlation and improve
the centrality resolution, Refmult3 is defined by using the number of charged
$\pi$ and $K$ in the final state within pseudo-rapidity $|\eta|<$1.0 [36].
Figs. 3(b) and (c) show the means of proton and anti-proton numbers,
$\left<N_{p}\right>$ and $\left<N_{\bar{p}}\right>$, as a function of
Refmult3, respectively. As a comparable study to STAR measurements, the
protons and anti-protons in the UrQMD model are carried out at mid-rapidity
$(|y|<0.5)$ in the transverse momentum range $0.4<p_{T}<2.0$ GeV/$c$. These
values of $\left<N_{p}\right>$ and $\left<N_{\bar{p}}\right>$ are taken as the
input parameters in the following simulations.

For a straightforward comparison between the simulated results and theoretical
expectations, we assume that the numbers of protons and anti-protons
independently follow BD with
$\varepsilon=\varepsilon_{p}=\varepsilon_{\bar{p}}=0.97$ in each Refmult3 bin.
Finally, the influence of statistics on $C_{6}/C_{2}$ with the whole interval
of $\varepsilon$, from 0.94 to 0.99, is studied. The required number of MB
events is 300 million (M). Under these assumptions, the simulated procedures
are implemented as follows:

(a) Multiplying the normalized probability of Refmult3 by 300M. The
corresponding number of events in $j_{th}$ Refmult3 bin is $n_{j}$.

(b) According to $n_{j}$, the values of $\left<N_{p}\right>$ and
$\left<N_{\bar{p}}\right>$ in $j_{th}$ Refmult3 bin, the numbers of protons
and anti-protons are independently and randomly generated event-by-event by BD
in each Refmult3 bin.

(c) Calculating $\left(C_{6}/C_{2}\right)_{j}$ and its error in each Refmult3
bin. Its error is estimated by the delta theorem method [44]. Then the
centrality dependence of $C_{6}/C_{2}$ and error($C_{6}/C_{2}$) in each
centrality are obtained based on the CBWC method.

(d) Repeating the procedures of (b) and (c) $N$ times. Here, $N$ should be
large enough to reduce the uncertainty of averaged $C_{6}/C_{2}$,
$\left<C_{6}/C_{2}\right>$. We set $N=1000$ in the following.

(e) Calculating the centrality dependence of $\left<C_{6}/C_{2}\right>$ and
error($\left<C_{6}/C_{2}\right>$). The formula of error propagation is used to
estimate the statistical uncertainty.

To demonstrate more details about the simulations, Figs. 4(a) and (b) show 200
results of 1000 samples in 0-10% and 30-40% centrality, respectively. To
reduce the statistical error and get a more stable result, the value of
$\left<C_{6}/C_{2}\right>$, shown by the black solid line in each panel, is
derived from the total 1000 samples. The black dashed lines are 1$\sigma$
limits of $C_{6}/C_{2}$. Both of these two plots clearly demonstrate that
$C_{6}/C_{2}$ randomly fluctuates around $\left<C_{6}/C_{2}\right>$. In Fig.
4(a), the probabilities of $C_{6}/C_{2}$ that lie outside 1$\sigma$ and
2$\sigma$ of $\left<C_{6}/C_{2}\right>$ are about 31.0% and 5.0%,
respectively. Meanwhile, 30.0% and 4.0% of the observations lie outside of the
1$\sigma$ and 2$\sigma$ standard deviations shown in Fig. 4(b). Theoretically,
for $N$ independent observations $(x_{1},x_{2},\cdots,x_{N})$ with the same
expectation, they are approximated with a normal distribution. In this case,
31.7% and 4.5% of the observations are outside 1$\sigma$ and 2$\sigma$
standard deviations of the expectation, respectively. Consequently, our
results are consistent with the theoretical studies. It confirms the validity
of our simulations.

Figure 1:  Statistical dependence of $\left<C_{6}/C_{2}\right>$ in BD with
$\varepsilon=0.97$. The statistical error is calculated from formula of the
error propagation. The black solid lines are the corresponding theoretical BD
expectations derived from Eq. (10).

The red solid line in each panel is the theoretical BD expectation. It is
about 0.233 with $\varepsilon=0.97$. In principle, if the statistics in each
Refmult3 bin are sufficient, the final $\left<C_{6}/C_{2}\right>$ should be
consistent with 0.233 in all centralities. However, Figs. 4(a) and (b) show
the disagreement of $C_{6}/C_{2}$ between the simulated
$\left<C_{6}/C_{2}\right>$ and theoretical BD expectation. The difference of
$C_{6}/C_{2}$ in 0-10% centrality is more significant than that in 30-40%
centrality. Comprehensive studies of the statistics dependence of
$C_{6}/C_{2}$ are shown in the following section.

##  5 Statistics dependence of $\left<C_{6}/C_{2}\right>$ in BD

As we mentioned, the experimental data collected for $C_{6}/C_{2}$ analysis
include three different runs: 300M MB events, 420M MB events, and 110M from
central trigger. Here 100M events from central trigger can be reference to
1000M MB events in our simulations. As a direct comparison, we can study the
statistical dependence of $\left<C_{6}/C_{2}\right>$ with the magnitude of a
few hundred million.

By considering $\varepsilon=0.97$ to the simulations, Figs. 1(a) to (f) show
statistical dependence of $\left<C_{6}/C_{2}\right>$ in different centralities
from 100M to 1000M MB events. In 0-10% centrality, Fig. 1(a) shows that all of
the results are systematically smaller than the black solid line, which is the
theoretical expectation. Below 500M MB events, the values of
$\left<C_{6}/C_{2}\right>$ first increase as the statistics increase and they
are all negative. From 600M to 1000M MB events, $\left<C_{6}/C_{2}\right>$ is
consistent with zero with 2$\sigma$ of the statistical uncertainty.
Consequently, with the current statistics at RHIC/STAR, $C_{6}/C_{2}$ may be
under-estimated in 0-10% centrality at $\sqrt{s_{NN}}=$ 200 GeV.

In 10-20% centrality, Fig. 1(b) shows that the values of
$\left<C_{6}/C_{2}\right>$ are still systematically smaller than that in BD.
However, it becomes positive above 200M MB events. It suggests that the effect
of the statistics becomes weaker. In 20-30% and 30-40% centralities,
$\left<C_{6}/C_{2}\right>$ is only slightly smaller than that in BD, as shown
in Figs. 1(c) and (d). Finally, they are in agreement with each other in
peripheral collisions shown in Figs. 1(e) and (f).

Fig. 1 shows that the influence of statistics on $\left<C_{6}/C_{2}\right>$ is
strongest in central collisions, comparing to the mid-central and peripheral
collisions. In 0-10% centrality, $\left<C_{6}/C_{2}\right>$ is still smaller
than the expectation even with 1000M MB events. $\left<C_{6}/C_{2}\right>$ is
still about 0 and we do not observe the convergence even with 1,000M MB
events. It is due to the wider distribution of net-proton number and the
smaller statistics in each $N_{ch}$ in $0-10\%$ centrality. It can be further
understood from Fig. 3(b), in which the simulated parameters are also
extracted from mean values of proton and anti-proton in $0-10\%$ centrality in
the UrQMD model. As the same estimation as in section 2, with 1,000M MB
events, the averaged statistics in each $N_{ch}$ is around 0.1M, assuming
there are 1,000 $N_{ch}$ bins in $0-10\%$ centrality. Fig. 3(b) also shows
that $\left<C_{6}/C_{2}\right>$ is also about 0 with 0.1M events in each
$N_{ch}$, which is consistent with that shown in Fig. 1(a). In contrast, Fig.
3(b) also shows that $\left<C_{6}/C_{2}\right>$ is consistent with the
theoretical expectations above 0.5M events, which suggests that 5,000M MB
events are required if the CBWC method in each $N_{ch}$ is applied. That is
why the convergence is not observed in Fig. 1(a).

One possible alternative is to reduce the number of bins when applying CBWC
method, such as the centrality bin width of each $\delta 0.5\%$, $\delta
1.0\%$, and $\delta 2.5\%$ and so on. It is beyond the scope of this paper to
examine through the transport model or the experimental data directly.

(Color Online) Centrality dependence of $\left<C_{6}/C_{2}\right>$ in BD with
$\varepsilon$ from 0.94 to 0.99 and in SD, respectively. Each
$\left<C_{6}/C_{2}\right>$ is averaged from 10,000 randomly and independently
simulated samples, with 300M MB events in each sample.

Because the influence of the statistics cannot be neglected, it is necessary
to study $C_{6}/C_{2}$ in BD with the current statistics at RHIC/STAR. With
300M MB events, the blue filled band of Fig. 5 shows centrality dependence of
$\left<C_{6}/C_{2}\right>$ with $\varepsilon$ from 0.94 to 0.99. The solid and
dashed lines are theoretically expected $C_{6}/C_{2}$ with $\varepsilon=0.99$
and 0.94, respectively. In 0-10% and 10-20% centralities, the simulated
results are systematically smaller than those with the ideal theoretical
expectations. Particularly, there is a marked drop in
$\left<C_{6}/C_{2}\right>$ in 0-10% centrality. With $\varepsilon=0.99$, the
simulated $\left<C_{6}/C_{2}\right>$ is still negative, while the theoretical
expectation is about 0.71. Meanwhile, this phenomenon is also observed with
the same simulation method in Skellam distribution (SD), shown by circular
solid points in Fig. 5. The simulated $\left<C_{6}/C_{2}\right>$ is consistent
with zero while it must be unity in SD. It further confirm that $C_{6}/C_{2}$
can be under-estimated in 0-10% centrality. With insufficient statistics,
$C_{6}/C_{2}$ can vary from a positive value to a negative one.

Together with the same statistics as RHIC/STAR, the negative $C_{6}/C_{2}$ is
more significant in BD. The blue filled band shown in Fig. 5 almost covers the
current preliminary measured $C_{6}/C_{2}$ within experimental large
uncertainties [13]. It can better describe the experimental measured
$C_{6}/C_{2}$, comparing to the SD baseline. Nonetheless, it cannot completely
reproduce the experimentally measured results. First, the experimental data
show $C_{6}/C_{2}$ decreases from peripheral to central collisions [13]. In
our simulations, the variation range of $\left<C_{6}/C_{2}\right>$ is too
large, although $\varepsilon$ is just from 0.94 to 0.99. It is difficult to
study the feature of centrality dependence of $C_{6}/C_{2}$. Second, besides
the statistics and BD, $C_{6}/C_{2}$ can still be affected by many other
complex contributions. We must correct those contributions well before
relating the experimental measurements to our calculations. Third, we assume
the proton and anti-proton are produced independently. However, they must have
correlation, such as from resonance decay, which is not considered in this
paper. A direct comparison between the experimentally measured results and
this baseline requires further studies. In contrast, our studies also imply
that we must exclude factors of non-phase transition related influences well
before connecting the measured $C_{6}/C_{2}$ to the theoretical Lattice QCD
calculations.

##  6 Summary

We studied the behavior of the net-proton $C_{6}/C_{2}$ in BD in Au + Au
collisions at $\sqrt{s_{NN}}=$ 200 GeV. By a reasonable approximation of
$\varepsilon=\varepsilon_{p}=\varepsilon_{\bar{p}}$, the net-proton
$C_{6}/C_{2}$ is only dependent on $\varepsilon$ in BD. With protons and anti-
protons carried out in the mid-rapidity $|y|<0.5$ and $0.4<p_{T}<2.0$ GeV/$c$
in experiment, $\varepsilon_{p}$ and $\varepsilon_{\bar{p}}$ are close to each
other and the values are between 0.94 and 0.99 in all centralities. In this
region, $\varepsilon$ has a very significant effect on $C_{6}/C_{2}$. The
values of $C_{6}/C_{2}$ decrease from 0.71 to -0.31 with $\varepsilon$ from
0.99 down to 0.94. The negative $C_{6}/C_{2}$ is observed in the pure
statistical BD. It suggests that the negative $C_{6}/C_{2}$ is not sufficient
to be considered as an indication of a smooth crossover transition.

We also simulated the statistics dependence of $\left<C_{6}/C_{2}\right>$
based on CBWC calculation method. Our method can compare the simulated results
and theoretical expectations in a straightforward manner. The significantly
dropped signal is observed in 0-10% centrality. With 300M MB events, the
simulated $C_{6}/C_{2}$ is negative, while it is about 0.71 with
$\varepsilon=0.99$ in BD. This phenomenon is also observed with the same
method of simulation in SD. The simulated $\left<C_{6}/C_{2}\right>$ is
consistent with zero while the theoretical expectation is unity. Moreover,
with 1000M MB events, the simulated $\left<C_{6}/C_{2}\right>$ is consistent
with zero, while the theoretical expectation is about 0.233 with
$\varepsilon=0.97$ in BD. Consequently, with the current statistics at
RHIC/STAR, $C_{6}/C_{2}$ may be under-estimated in 0-10% centrality at
$\sqrt{s_{NN}}=$ 200 GeV.

With the experimental collected statistics, the negative $C_{6}/C_{2}$ is more
significant in BD. It shows the values of $C_{6}/C_{2}$ have a broad range
which can change from positive to negative. Comparing the baselines of
$C_{6}/C_{2}$ in SD and BD, the BD could better describe the experimental
measured $C_{6}/C_{2}$. Within large uncertainties at RHIC/STAR, the obtained
baseline almost covers the current preliminary measured $C_{6}/C_{2}$.
Consequently, the exclusions of the non-phase transition related influences
are required when using $C_{6}/C_{2}$ to study the chiral phase transition.

## References

  * [1]
  * [2] M. M. Aggarwal et al., (STAR Collaboration), arXiv: 1007.2613 [nucl-ex].
  * [3] Y. Aoki, G. Endrodi, Z. Fodor et al. Nature, 443: 675 (2006).
  * [4] M. Cheng et al., Phys. Rev. D 79, 074505 (2009).
  * [5] C. Schmidt, prog. Theor. Phys. Suppl. 186, 563 (2010).
  * [6] A. Bazavov et al., (HotQCD Collaboration) Phys. Rev. D 95, 054504 (2017).
  * [7] A. Bazavov et al., (HotQCD Collaboration) Phys. Rev. D 101, 074502 (2020).
  * [8] Kenji Morita et al., Phys. Rev. C 88 034903, (2013).
  * [9] V. Skokov, B. Friman and K. Redlich, Phys. Lett. B 708, 179 (2012).
  * [10] B. Friman et al., Eur. Phys. J. C 71, 1694 (2011).
  * [11] Wei-jie Fu, Yu-xin Liu, and Yue-Liang Wu, Phys. Rev. D 81, 014028 (2010).
  * [12] Wei-jie Fu et al., arXiv: 2101.06035 [hep-ph].
  * [13] Toshihiro Nonaka (for the STAR Collaboration), Nucl. Phys. A 1005, 121882 (2021).
  * [14] Roli Esha (for the STAR Collaboration), Nucl. Phys. A 967, 457 (2017).
  * [15] M. S. Abdallah et al., (STAR Collaboration), arXiv: 2105.14698 [nucl-ex].
  * [16] Z. Citron et al., (ALICE Collaboration) CERN Yellow Rep. Monogr. 7, 1159 (2019). arXiv: 1812.06772 [hep-ph].
  * [17] Volker Koch, Volodymyr Vovchenko, arXiv: 2102.08506 [nucl-th].
  * [18] Adam Bzdak, Volker Koch and Vladimir Skokov, Phys. Rev. C 87, 014901 (2013).
  * [19] P. Garg et al., Phys. Lett. B 726, 691 (2013).
  * [20] Adam Bzdak, Romain Holzmann and Volker Koch, Phys. Rev. C 94, 064907 (2016).
  * [21] P. Braun-Munzinger et al., Nucl. Phys. A 1008, 122141 (2021).
  * [22] P. Braun-Munzinger et al., Phys. Rev. C 84, 064911 (2011).
  * [23] P. Braun-Munzinger et al., Phys. Lett. B 747, 292 (2015).
  * [24] B. Abelev et al., (ALICE Collaboration) Eur. Phys. J. C 75, 146 (2015).
  * [25] Premomoy Ghosh, Phys. Rev. D 85, 054017 (2012).
  * [26] M. Kitazawa, M. Asakawa, Phys. Rev. C 86 (2012).
  * [27] Terence J. Tarnowsky, Gary D. Westfall. Phys. Lett. B 724, 51 (2013).
  * [28] Xiaofeng Luo, Bedangadas Mohanty, Nu Xu, Nucl. Phys. A 931, 808 (2014).
  * [29] L. Adamczyk et al., (STAR Collaboration), Phys. Rev. Lett 112, 032302 (2014).
  * [30] L. Adamczyk et al., (STAR Collaboration), Phys. Rev. Lett 113, 092301 (2014).
  * [31] A. Adare et al., (PHENIX Collaboration), Phys. Rev. C 93, 011901(R) (2016).
  * [32] L. Adamczyk et al., (STAR Collaboration), Phys. Lett. B 785, 551 (2018).
  * [33] J. Adam et al., (STAR Collaboration), Phys. Rev. C 102, 024903 (2020).
  * [34] X. Luo (For the STAR Collaboration), J. Phys. : Conf. Ser. 316, 012003 (2011).
  * [35] Xiaofeng Luo, Nu Xu. Nucl. Sci. Tech. 28, 112 (2017).
  * [36] M. S. Abdallah et al., (STAR Collaboration), arXiv: 2101.12413 [nucl-ex].
  * [37] Lizhu Chen, Zhiming Li and Yuanfang Wu. J. Phys. G 41, 105107 (2014).
  * [38] Lizhu Chen et al., J. Phys. G 42, 065103 (2015).
  * [39] Lizhu Chen, Zhiming Li, Fenping Cui, and Yuanfang Wu. Nucl. Phys. A 957 60 (2017).
  * [40] Papoulis, A. Probability, Random Variables, and Stochastic Processes, 2nd ed. New York, 1984.
  * [41] John A. Rice, Mathematical Statistics and Data Analysis, third ed., Duxbury Press, ISBN 0-534-39942-8, 2007.
  * [42] S. A. Bass et al., Prog. Part. Nucl. Phys. 41, 255 (1998).
  * [43] M. Bleicher et al., J. Phys. G25, 1859 (1999).
  * [44] X. Luo, J. Phys. G 39, 025008 (2012).