# Probabilistic Self-supervised Learning via
Scoring Rules Minimization

Amirhossein Vahidi
LMU Munich &
MCML &Simon Schoßer
LMU Munich &Lisa Wimmer
LMU Munich &
MCML &Yawei Li
LMU Munich &
MCML &Bernd Bischl
LMU Munich &
MCML &Eyke Hüllermeier
LMU Munich &
MCML &Mina Rezaei
LMU Munich &
MCML

###### Abstract

In this paper, we propose a novel probabilistic self-supervised learning via
Scoring Rule Minimization (ProSMIN), which leverages the power of
probabilistic models to enhance representation quality and mitigate collapsing
representations. Our proposed approach involves two neural networks; the
online network and the target network, which collaborate and learn the diverse
distribution of representations from each other through knowledge
distillation. By presenting the input samples in two augmented formats, the
online network is trained to predict the target network representation of the
same sample under a different augmented view. The two networks are trained via
our new loss function based on proper scoring rules. We provide a theoretical
justification for ProSMIN’s convergence, demonstrating the strict propriety of
its modified scoring rule. This insight validates the method’s optimization
process and contributes to its robustness and effectiveness in improving
representation quality. We evaluate our probabilistic model on various
downstream tasks, such as in-distribution generalization, out-of-distribution
detection, dataset corruption, low-shot learning, and transfer learning. Our
method achieves superior accuracy and calibration, surpassing the self-
supervised baseline in a wide range of experiments on large-scale datasets
like ImageNet-O and ImageNet-C, ProSMIN demonstrates its scalability and real-
world applicability.

##  1 Introduction

In the field of machine learning, probabilistic approaches have emerged as a
powerful toolkit, offering advantages such as uncertainty quantification,
robustness, and flexibility, especially in domains with inherent complexity
and uncertainty Dutordoir et al. (2021); Peharz et al. (2020); Ghahramani
(2015). While deterministic models have their merits, a crucial frontier lies
in extending the benefits of probabilistic methods to self-supervised learning
(SSL) for representation reliability. Representation reliability provides a
more comprehensive understanding of the underlying data, which is particularly
valuable in domains where prediction errors can have serious consequences,
such as disease diagnosis Ozdemir et al. (2019); Rezaei et al. (2022), climate
prediction Gneiting and Raftery (2007), computational finance Dawid and Musio
(2014), and economic forecasting Gneiting and Katzfuss (2014).

Current SSL methods have made impressive progress, but a notable gap exists in
their evaluation of representational reliability through a probabilistic lens.
Incorporating probabilistic modeling into SSL training provides a way to not
only predict outcomes but also provide distributions of possible predictions
along with associated probabilities Ho et al. (2020); Nichol and Dhariwal
(2021). This distinctive feature enables SSL models to capture the inherent
uncertainties in real-world data and provide a more complete understanding of
underlying patterns. Encouragingly, recent advances in SSL, exemplified by
methods such as Oquab et al. (2023), have demonstrated the potential to
generate effective features across diverse image distributions and tasks
without the need for fine-tuning, confirming the viability of this paradigm
shift.

As the demand for more reliable representations escalates, the integration of
probabilistic principles into SSL is well poised to improve representation
quality, align predictions with real-world complexity, and thereby advance the
frontiers of machine learning. In this paper, we present ProSMIN, a novel
probabilistic formulation of self-supervised learning that contributes to the
understanding and improvement of probabilistic modeling in this domain. Our
proposed method involves two deep neural networks, an online and a target
network, each learning a different representation of input samples. The online
network maps input samples to a probability distribution inferred from the
representation of the online encoder part. We train the online network in such
a manner that samples from its output distribution predict the target
network’s representation on a second augmented view of the input. The loss
realized by this prediction is expressed via a modified scoring rule, which
incentivizes the recovery of the true distribution.

Our contributions are:

  * •

We introduce a novel probabilistic definition of representation reliability
for self-supervised learning. The probabilistic definition provides a deeper
understanding of the quality and trustworthiness of the learned
representations in guiding subsequent tasks. To the best of our knowledge,
this is the first comprehensive study to investigate probabilistic formulation
in the self-supervised representation domain.

  * •

Our probabilistic approach effectively mitigates collapsing representation by
encouraging the online and target networks to explore a diverse range of
representations, thus avoiding convergence to a limited set which results in
more comprehensive representation space that better encapsulates the
intricacies of the data distribution.

  * •

We provide a rigorous theoretical foundation for the convergence of our
proposed algorithm. Specifically, we demonstrate the strict propriety of our
modified scoring rule, substantiating the convergence of the optimization
process. This theoretical insight not only underscores the robustness of our
approach but also provides a principled explanation for its effectiveness in
improving representation quality.

  * •

Through extensive empirical analysis, we validate the effectiveness of our
approach in diverse scenarios. Our method achieves competitive predictive
performance and calibration on various tasks such as in-distribution (IND),
out-of-distribution (OOD), and corrupted datasets, demonstrating
generalization capabilities. Moreover, we demonstrate the superiority of our
method in semi-supervised and low-shot learning scenarios. Additionally, our
framework establishes a superior trade-off between predictive performance and
robustness when compared to deterministic baselines. This outcome is
particularly notable on large-scale datasets such as ImageNet-O and
ImageNet-C, underscoring the scalability and effectiveness of our method in
real-world, high-dimensional settings.

##  2 Background and Related work

Self-supervised methods are designed to tackle unsupervised problems by
training on a _pretext task_ that utilizes the data itself to generate labels,
effectively employing supervised methods to solve unsupervised problems Grill
et al. (2020); Chen et al. (2020); Jang et al. (2023); Caron et al. (2021);
Zhou et al. (2021); Zbontar et al. (2021); Bardes et al. (2021); Chen et al.
(2021). The resulting representations learned from the pretext task can serve
as a foundation for _downstream supervised tasks_ , such as image
classification or object detection. Alternatively, the extracted
representation can be directly utilized for downstream applications, such as
detecting anomalies and OOD data Tran et al. (2022). Recent studies Oquab et
al. (2023); Zhou et al. (2021) provided evidence that performing self-
supervised pretext task learning on a large-scale and diverse dataset can
extract features that are effective across different image distributions and
tasks without the need for fine-tuning. Following this, we introduce a novel
probabilistic self-supervised framework aiming to learn robust representation
over parameters using _self-distillation_ and by _minimizing the scoring
rule_.

Self-distillation is a variant of knowledge distillation Hinton et al. (2015)
in which a larger model (_teacher_) is used to distill knowledge into a
smaller model (_student_) of the same architecture Caron et al. (2021); Zhou
et al. (2021). Given an input sample $\bm{x}$, the student network
$f_{\bm{\theta}}$ is trained on the soft labels provided by the teacher
network $f_{\bm{\xi}}$. Self-distillation combines self-supervised learning
with knowledge distillation and was introduced by DINO Caron et al. (2021).
The two networks share the same architecture but take different augmentations
of the input sample and output different representation vectors. The knowledge
is distilled from the teacher network $f_{\bm{\xi}}$ to student
$f_{\bm{\theta}}$ by minimizing the cross-entropy between the respective
representation vectors. The parameters of the student network $\bm{\theta}$
are obtained from an exponential moving average (EMA) of the parameters of the
teacher network $\bm{\xi}$, thus reducing computational cost by confining
backpropagation to the teacher network.

A scoring rule is a function used to evaluate the accuracy of a probabilistic
prediction Gneiting and Raftery (2007); Der Kiureghian and Ditlevsen (2009).
It quantifies the divergence between the predicted probability distribution
and the true distribution of the event. The concept of a scoring rule is
fundamental to many areas of machine learning, including probabilistic
classification Parry (2016) and decision theory Dawid and Musio (2014). A
proper scoring rule Gneiting and Katzfuss (2014) is one that incentivizes
truthful reporting of the probabilities by the forecaster, i.e., the
forecaster is incentivized to report the correct probability distribution
Pacchiardi et al. (2021). The use of proper scoring rules has had significant
implications in areas such as online learning V’yugin and Trunov (2019),
generative neural networks Pacchiardi and Dutta (2022), and uncertainty
quantification Bengs et al. (2023); Gruber and Buettner (2022). This paper
utilizes scoring rules and adapts them for the purpose of pretext task
learning of a self-supervised framework. This adaptation is inspired by the
endeavor to infuse probabilistic learning principles into the self-supervised
learning domain.

##  3 Problem formulation

##### Avoiding collapsing representation

One of the major challenges in self-supervised learning is collapsing
representation where learned representations converge to a limited set of
points in the representation space. In other words, the model fails to capture
the full diversity and richness of the underlying data distribution. This can
lead to reduced representation quality, impaired generalization to new tasks
or domains, and limited capacity to handle variations in the data. Most recent
studies addressed this problem with contrastive learning through effective
augmentation Chen et al. (2020), negative sample strategies Wang et al.
(2021), ensemble approach Vahidi et al. (2023), regularization techniques
Rezaei et al. (2023), and self-distilation Caron et al. (2021). In this paper,
we formulate the collapsing representation problem through a probabilistic
lens, aiming to provide a comprehensive and nuanced solution that not only
addresses the limitations of deterministic approaches but also harnesses the
power of uncertainty quantification and broader representation distributions.
This pivotal shift offers promising avenues for achieving representation
reliability and superior generalization capabilities in self-supervised
learning scenarios.

##### Scoring rules

A scoring rule Gneiting and Raftery (2007) is a function that evaluates how
well a predicted distribution $P$ over a random variable $\mathbf{X}$ aligns
with the actually observed realizations $\bm{x}$ of $\mathbf{X}$. We define
the loss111 The original proposal in Gneiting and Raftery (2007) defines
scoring rules in terms of a gain that is to be maximized. We adhere to the
convention in deep learning of expressing the objective via a loss function
that we seek to minimize.  of predicting distribution $P$ while observing
$\bm{x}$ as $S(P,\bm{x})$. Assuming that $\mathbf{X}$ follows some true
distribution $Q$, the _expected_ scoring rule measuring the loss of predicting
$P$ can be expressed as $S(P,Q)\triangleq\mathbb{E}_{\mathbf{X}\sim
Q}S(P,\mathbf{X})$. A scoring rule $S$ is _proper_ with respect to a set of
distributions $\mathcal{P}$ if for all $P,Q\in\mathcal{P}$ it holds that
$S(Q,Q)\leq S(P,Q)$, thus incentivizing the prediction of the true
distribution $Q$. If the former holds with equality, i.e., the expected score
$S(P,Q)$ is uniquely minimized in $Q$ at $Q=P$, then the scoring rule is
called _strictly proper_. In practice, the expectation with respect to $Q$ is
usually replaced by an empirical mean over a finite amount of samples. We will
denote the resulting scoring rule as $\hat{S}$. There exist many types of
scoring rules, including entire parameterized families of strictly proper
scoring rules. We provide more details on some of them that we use in our
experiments in Appendix 9.

Figure 1: Illustration of our proposed architecture (ProSMin). Given a batch
$\bm{X}$ of input samples, two different augmented samples $\tilde{\bm{x}}$
and $\tilde{\bm{x}}^{\prime}$ are taken by an online network with
$\bm{\theta}$ and target network with $\bm{\xi}$ parameters respectively. Our
objective is to minimize the proposed scoring rule between $P_{\bm{\theta}}$,
$P_{\bm{\xi}}$.

##  4 Method

Consider a randomly sampled mini-batch of training data
$\bm{X}\triangleq\left[\bm{x}_{1},\ldots,\bm{x}_{n}\right]^{N}\in\mathbb{R}^{N\times
D}$ and transformation functions $\tau,\tau^{\prime}$ acting on the data. To
enhance the training process, the transformation functions produce two
augmented views $\tilde{\bm{x}}\triangleq\tau(\bm{x})$ and
$\tilde{\bm{x}}^{\prime}\triangleq\tau^{\prime}(\bm{x})$ for each sample in
$\bm{X}$. These augmented views are generated by sampling $\tau,\tau^{\prime}$
from a distribution of suitable data transformations, such as partially
masking image patches (He et al., 2022) or applying image augmentation
techniques (Chen et al., 2020).

The first augmented view $\tilde{\bm{x}}$ is fed to the encoder of online
network $f_{\bm{\theta}}$ that outputs a _representation_
$y_{\bm{\theta}}\triangleq f_{\bm{\theta}}(\tilde{\bm{x}})$. This is followed
by passing it subsequently through a projector $g_{\bm{\theta}}$ and predictor
$q_{\bm{\theta}}$, such that $t_{\bm{\theta}}\triangleq
g_{\bm{\theta}}(y_{\bm{\theta}})$ and $z_{\bm{\theta}}\triangleq
q_{\bm{\theta}}(t_{\bm{\theta}})$. We collect all trainable parameters of the
online network in $\bm{\theta}$. Similarly, the encoder of target network
$f_{\bm{\xi}}$ takes the second augmented view and outputs
$y_{\bm{\xi}}\triangleq f_{\bm{\xi}}(\tilde{\bm{x}}^{\prime})$, followed by
the projector network $g_{\bm{\xi}}$ producing $t_{\bm{\xi}}\triangleq
g_{\bm{\xi}}(y_{\bm{\xi}})$, where the trainable parameters are denoted as
$\bm{\xi}$. It is important to note that the predictor is applied exclusively
to the online network.

In order to introduce probabilistic self-distillation training, we employ a
scoring rule Gneiting and Raftery (2007) as our loss function. To accomplish
this, it is necessary to generate samples from the online network. One way of
producing samples from a neural network architecture, while still enabling
backpropagation with respect to the latent representation $\bm{z}$, is to use
the reparametrization trick Kingma et al. (2015). Here, we assume that the
output of the predictor network
$z_{\bm{\theta}}=q_{\bm{\theta}}(t_{\bm{\theta}})$ follows an underlying
normal distribution with mean $\mu$ and standard deviation $\sigma$. We
generate $r\in\mathbb{N}$ samples from the output of the linear layers
following the prediction head by sampling random noise $\epsilon^{i}_{j}\sim
N(0,1)$ for each augmented view of the $i$-th data point, such that the $j$-th
sample, with $j\in 1,...,r$, is given by:
$\bm{z}^{i}_{j}=\mu+\sigma\odot\epsilon^{i}_{j}.$

We thus obtain samples $\bm{z}^{i}_{j}$ by shifting and scaling the random
noise samples $\epsilon^{i}_{j}$ by the outputs $(\mu,\sigma)$ of a neural
network with trainable parameters $\bm{\theta}$, and the loss incurred by
these samples can be backpropagated to update $\bm{\theta}$ during training.
We update the online network parameters by minimizing the scoring rule as
follows:

$\hat{\bm{\theta}}:=\operatorname*{arg\,min}_{\bm{\theta}}J(\bm{\theta}),\quad
J(\bm{\theta})=\hat{S}(P_{\bm{\theta}},P_{\bm{\xi}}):=\mathbb{E}_{z_{\bm{\xi}}\sim
P_{\bm{\xi}}}[P_{\bm{\theta}},z_{\bm{\xi}}],$ (1)

where $P_{\bm{\xi}}$ denotes the target distribution and $z_{\bm{\xi}}$
denotes the target output and $P_{\bm{\theta}}$ represents the online induced
multivariate normal distribution. Our custom scoring rule loss over
$P_{\bm{\theta}}$ and $P_{\bm{\xi}}$ is defined as

$\leavevmode\nobreak\
\hat{S}(P_{\bm{\theta}},P_{\bm{\xi}})=\dfrac{1}{N}\sum_{i=1}^{N}\left|\\!\left[\dfrac{2\lambda}{r}\sum_{j=1}^{r}\|\bm{z}_{j}^{i}-\bm{z}_{\xi}^{i}\|^{\beta}_{2}-\frac{1-\lambda}{r(r-1)}\sum_{j\neq
k}\|\bm{z}_{j}^{i}-\bm{z}_{k}^{i}\|^{\beta}_{2}\\!\right]\right|,$ (2)

where $\bm{z}_{\xi}^{i}$ represents the target prediction for the $i$-the
input sample. $\beta\in(0,2)$ and $\lambda\in(0,1)$ are hyperparameters.
Detailed information is provided in Appendix 9.5.6.

By the principle of knowledge distillation, the parameters of the target
network are updated through the EMA of the weights from the online network
Grill et al. (2020); Caron et al. (2021), saving the need for backpropagation
and thus reducing computation time considerably.

$\displaystyle\bm{\xi}_{t}$
$\displaystyle=\alpha\bm{\theta}_{t}+(1-\alpha)\bm{\xi}_{t-1},\quad
t=1,2,\dots,\quad\alpha\in[0,1]$ (3)

The initial weights $\bm{\xi}_{0}$ are obtained through random initialization.

Next, we explain the details of our objective function based on scoring rules
4.2. We refer to Section 9 for the background and different variations of
scoring rules. Then, we show that, by utilizing a strictly proper scoring rule
and retaining the ability to calculate gradients as usual (as proven in
9.5.4), we can infer that our algorithm will converge towards the desired
minimum (see Section 4.3).

###  4.1 Avoiding collapse

There are different methods in self-supervised learning to avoid collapse by
adding batch normalization Grill et al. (2020), prediction layer Grill et al.
(2020), and sharpening and centering Caron et al. (2021). Our framework can
benefit from more layers in the online network and centering on the target
outputs to improve the results. Still, our method inherently can not collapse
to the same representation because of its probabilistic nature. Furthermore,
our loss function converges to zero without collapsing.

###  4.2 Objective function

Many scoring rules decompose into two terms, one of which is a function of two
samples $\bm{z}_{j},\bm{z}_{k}$ from the predicted distribution $P$, while the
other is a function of $\bm{z}_{j}$ and the realized observation
$\bm{z}_{\bm{\xi}}$. Our objective function is an adjusted version of the
former, where the two components are posed as a convex combination with
component weights controlled via hyperparameter $\lambda\in(0,0.5]$. This
adjustment of the scoring rules can be useful to adjust the focus of the loss
function on either part of the scoring rule. The hyperparameter $\lambda$
helps to improve the performance by shifting the focus of the loss function.
Two notable examples of scoring rules adhering to this form are the _energy
score_ and the _kernel score_ Gneiting and Raftery (2007).

With the above notation, we define the energy score as:

$\leavevmode\nobreak\
S_{\mathrm{E}}(P_{\bm{\theta}},\bm{z}_{\bm{\xi}})=2\cdot\mathbb{E}_{P_{\bm{\theta}}}\left[\|\bm{z}_{j}-\bm{z}_{\bm{\xi}}\|_{2}^{\beta}\right]-\mathbb{E}_{P_{\bm{\theta}}}\left[\left\|\bm{z}_{j}-\bm{z}_{k}\right\|_{2}^{\beta}\right]=:S_{\mathrm{E}}^{1}(P_{\bm{\theta}},\bm{z}_{\bm{\xi}})+S_{\mathrm{E}}^{2}(P_{\bm{\theta}},\bm{z}_{\bm{\xi}})$
(4)

Analogously, we write the kernel score as:

$\leavevmode\nobreak\
S_{\mathrm{K}}(P_{\bm{\theta}},\bm{z}_{\bm{\xi}})=\mathbb{E}_{P_{\bm{\theta}}}\left[k(\bm{z}_{j},\bm{z}_{k})\right]-2\cdot\mathbb{E}_{P_{\bm{\theta}}}\left[k(\bm{z}_{j},\bm{z}_{\bm{\xi}})\right]=:S_{\mathrm{K}}^{2}(P_{\bm{\theta}},\bm{z}_{\bm{\xi}})+S_{\mathrm{K}}^{1}(P_{\bm{\theta}},\bm{z}_{\bm{\xi}}),$
(5)

with suitable kernel function $k(\cdot,\cdot)$.

For simplicity, we only write
$S(P_{\bm{\theta}},\bm{z}_{\bm{\xi}})=S^{1}(P_{\bm{\theta}},\bm{z}_{\bm{\xi}})+S^{2}(P_{\bm{\theta}},\bm{z}_{\bm{\xi}})$
in the following for both scores. With $\lambda\in(0,1)$, we define the
general form of our objective function as follows:

$S^{*}(P_{\bm{\theta}},\bm{z}_{\bm{\xi}}):=\left|\lambda
S^{1}(P_{\bm{\theta}},\bm{z}_{\bm{\xi}})+(1-\lambda)S^{2}(P_{\bm{\theta}},\bm{z}_{\bm{\xi}})\right|.$
(6)

The proposed objective function for energy and kernel score is strictly proper
in our setup. We provide proof in following Section 4.3.

###  4.3 Theoretical justification

This section provides the mathematical justification of our algorithm. We
establish that the custom loss is strictly proper in our setup. In subsection
9.5.4, we demonstrate that the expectation and gradient of our theoretical
derivative can be interchanged, thereby enabling us to derive the gradients as
usual. Due to the use of the samples within the scoring rule as well as the
use of non-differential activation functions, this step becomes necessary.
This is followed by subsection 9.5.5, where we present an unbiased estimate of
the gradient. By undertaking these three steps, we ensure that our algorithm
converges effectively.

##### Strict propriety of the proposed objective function

Let $\lambda\in(0,1)$. We only show the proof for the energy score. The proof
for the kernel score can be done analogously. We need to show that

$S^{*}(P_{\bm{\xi}},P_{\bm{\xi}})<S^{*}(P_{\bm{\theta}},P_{\bm{\xi}})\quad\forall
P_{\bm{\theta}},P_{\bm{\xi}}\in\mathcal{P}\quad s.t.\quad P_{\bm{\theta}}\neq
P_{\bm{\xi}}$ (7)

We define the following based on Eq. 1:

$\displaystyle S^{*}(P_{\bm{\theta}},P_{\bm{\xi}})$
$\displaystyle=\left|\mathbb{E}_{\bm{z}^{i}_{\bm{\xi}}\sim
P_{\bm{\xi}}}S^{*}(P_{\bm{\theta}},\bm{z}^{i}_{\bm{\xi}})\right|$
$\displaystyle=\left|\mathbb{E}_{\bm{z}^{i}_{\bm{\xi}}\sim
P_{\bm{\xi}}}\left[\lambda\mathbb{E}_{\bm{z}^{i}_{j}\sim
P_{\bm{\theta}}}\left[2\|\bm{z}^{i}_{j}-\bm{z}^{i}_{\bm{\xi}}\|_{2}^{\beta}\right]-(1-\lambda)\mathbb{E}_{\bm{z}^{i}_{j},\bm{z}^{i}_{k}\sim
P_{\bm{\xi}}}\left[\left\|\bm{z}^{i}_{j}-\bm{z}^{i}_{k}\right\|_{2}^{\beta}\right]\right]\right|$
$\displaystyle>\left|\mathbb{E}_{\bm{z}^{i}_{\bm{\xi}}\sim
P_{\bm{\xi}}}\left[\lambda\mathbb{E}_{\bm{z}^{i}_{j}\sim
P_{\bm{\xi}}}\left[2\|\bm{z}^{i}_{j}-\bm{z}^{i}_{\bm{\xi}}\|_{2}^{\beta}\right]-(1-\lambda)\mathbb{E}_{\bm{z}^{i}_{j},\bm{z}^{i}_{k}\sim
P_{\bm{\xi}}}\left[\left\|\bm{z}^{i}_{j}-\bm{z}^{i}_{k}\right\|_{2}^{\beta}\right]\right]\right|$
$\displaystyle=S^{*}(P_{\bm{\xi}},P_{\bm{\xi}})=0$

$S^{*}(P_{\bm{\xi}},P_{\bm{\xi}})$ can be estimated as zero and hence is
smaller than $S^{*}(P_{\bm{\theta}},P_{\bm{\xi}})$ for every
$P_{\bm{\theta}}\neq P_{\bm{\xi}}$. For a specific observation $\bm{x}_{i}$,
there is only a single target prediction $\bm{z}_{\bm{\xi}}$, such that
$\bm{z}_{\bm{\xi}}$ essentially follows a Dirac distribution. This concludes
the proof.

By utilizing a strictly proper scoring rule and retaining the ability to
calculate gradients as usual (as proven in 9.5.4), we can infer that our
algorithm will converge towards the desired minimum.

##  5 Implementation details and experimental setup

Image augmentation We define a random transformation function $\bm{T}$ that
applies a combination of multi-crop, horizontal flip, color jittering, and
grayscale. Similar to Caron et al. (2021), we perform multi-crops with a
random size from $0.8$ to $1.0$ of the original area and a random aspect ratio
from $3/4$ to $4/3$ of the original aspect ratio. We define color-jittering of
$(0.8,0.8,0.8,0.2)$, and Gaussian blurring with $0.5$ probability and
$\zeta=(0.1,2.0)$.

Deep self-supervised network architecture  The _online neural network_ is
constructed from a backbone $f$, which can be either ViT Dosovitskiy et al.
(2021) or ResNet He et al. (2016), and a projection head $g$ followed by a
prediction $q$. The backbone $f$ output is used as features for downstream
tasks. The projection consists of a 3-layer multilayer perceptron (MLP) with a
hidden dimension of 2048, followed by 2 normalizations and a weight-normalized
fully connected layer with $K$ dimensions, similar to the design used in the
DINO projection head. We use a predictor with two layers of MLPs with a hidden
dimension of 12000, with nonlinearity GELU in between. Notably, ViT
architectures do not use batch normalization (BN) by default, so we do not use
BN in the projection or prediction when applying ViT.

The _target network_ has the same backbone and projection as the online
network and it learns through self-distillation. Similar to DINO Caron et al.
(2021), after the online network parameters are updated, an EMA of the online
parameters (i.e., a momentum encoder) is used to update the target parameters.
The EMA prevents the target parameters from updating too quickly. After
parameter updates the target also gets a new centering parameter.

Optimization  Our pretraining process involves training the models on the
ImageNet training dataset Deng et al. (2009) using the adamw optimizer
Loshchilov and Hutter (2017a) and a batch size of 512, distributed across 8
GPUs utilizing Nvidia Tesla A100 with ViT-S/16 architecture. We adopt a linear
scaling rule to determine the base value of the learning rate, which is ramped
up linearly during the first 30 epochs. Specifically, the learning rate is set
to $lr=0.0005*\text{batchsize}/256$. After the warmup phase, we decay the
learning rate using a cosine schedule Loshchilov and Hutter (2017b). The
weight decay also follows a cosine schedule, decreasing from $0.04$ to $0.4$.
The centering (smooth parameter) is $0.9$.

Datasets  The datasets utilized in our experiments are as follows: The
ImageNet Deng et al. (2009) dataset with 1.28 million training images and
50,000 validation images with the size of $256\times 256$ contains 1,000
classes. The ImageNet-O dataset Srivastava et al. (2022) comprises images
belonging to classes that are not present in the ImageNet-1k dataset. It is
considered a challenging benchmark for evaluating the robustness of the
models, as it requires models to generalize to a diverse range of visual
conditions and handle variations that are not typically encountered in
standard training datasets. The ImageNet-C Hendrycks and Dietterich (2019) is
a benchmark for evaluating the robustness of the models against common
corruptions and perturbations that can occur in real-world scenarios. It
consists of more than 30,000 images derived from the ImageNet dataset, with
each image being corrupted in one of 15 different ways, including noise, blur,
weather conditions, and digital artifacts. CIFAR-10/100 Krizhevsky (2009) are
subsets of the tiny images dataset. Both datasets include 50,000 images for
training and 10,000 validation images of size $32\times 32$ with 10 and 100
classes, respectively. The Oxford 102 Flower Nilsback and Zisserman (2008)
consists of 102 flower categories, each class including between 40 and 258
images. The images have large scale, pose, and light variations. In addition,
there are categories that have large variations within the category and
several very similar categories. iNaturalist-2018 Van Horn et al. (2018)
(iNat) comprises a vast collection of 675,170 training and validation images,
classified into 5,089 distinct fine-grained categories found in the natural
world. It is worth noting that the iNat dataset exhibits a significant
imbalance, as the number of images varies greatly across different categories.

Tasks  We evaluate the performance of ProSMin representations after self-
supervised pretraining on the ImageNet on the basis of In-Domain (IND)
generalization, OOD detection, semi-supervised learning, corrupted dataset
evaluation, as well as transfer learning to other datasets and tasks.

Evaluation metrics  We report the prediction performance with the following
metrics: Top-1 accuracy $\uparrow$: refers to the proportion of test
observations that are correctly predicted by the model’s output as belonging
to the correct class. AUROC $\uparrow$: the area under the ROC curve
represents the relationship between false-positive and false-negative rates
for various classification thresholds. In this case, the positive and negative
classes refer to whether an observation is in or out of a given distribution,
respectively, and the ROC curve is plotted as the threshold for classifying an
observation as "positive" is gradually increased. Negative log-likelihood
(NLL) $\downarrow$: measures the probability of observing the given test data
given the estimated model parameters, multiplied by -1. This measure
quantifies the degree to which the model’s estimated parameters fit the test
observations. Expected calibration error (ECE) $\downarrow$ (Naeini et al.,
2015): calculated as the mean absolute difference between the accuracy and
confidence of the model’s predictions, where confidence is defined as the
highest posterior probability among the predicted classes. The difference is
calculated across equally-spaced confidence intervals or bins and is weighted
by the relative number of samples in each bin. A lower value of ECE indicates
better calibration of the model. mean Calibration Error (mCE) $\downarrow$ is
a metric used to evaluate the calibration of a classification model, similar
to ECE. It is calculated as the mean of the absolute differences between the
predicted and true probabilities of a given class across all classes. A lower
value of mCE indicates better calibration of the model.

##  6 Results and discussion

In-distribution generalization  IND generalization (or linear evaluation)
measures how well a model’s confidence aligns with its accuracy. To assess and
compare the predictive abilities of our proposed model on in-distribution
datasets, we freeze the encoder of the online network, denoted as
$f_{\bm{\theta}}$, after performing unsupervised pretraining. Then, we train a
supervised linear classifier using a fully connected layer followed by
softmax, which is placed on top of $f_{\bm{\theta}}$ after removing the
projection and prediction network. The desired outcome is high predictive
scores and low uncertainty scores. In Table 1, a comprehensive comparison is
presented between our approach and other self-supervised methods. The results
indicate that our method attained the second-highest accuracy with 300 epochs
while outperforming all others in terms of calibration, as evidenced by the
lowest Expected Calibration Error (ECE) and Negative Log-Likelihood (NLL)
scores.

Out-of-distribution detection  The ability of a model to recognize test
samples from classes that were not present during training is evaluated using
OOD detection, as discussed in Geng et al. (2020). We conduct experiments on
ImageNet-O Srivastava et al. (2022) to assess the generalization of the model
from IND to OOD datasets, as well as to predict the uncertainty of the models
on OOD datasets. Note that evaluation is performed directly after self-
supervised pretraining without a fine-tuning step. The y-axis of Figure 2
shows and compares the results for the OOD task in terms of AUROC. Remarkably,
our method, trained for 300 epochs, exhibits competitive performance
comparable to iBOT, which was trained for 800 epochs. This outcome aligns with
our expectations as we directly use the probabilistic latent representation
for this task.

Figure 2: OOD detection and corrupted dataset evaluation, Methods with higher
AUROC and lower mCE are better. Our method and iBOT have the best performance
across both evaluations.  Table 1: IND Generalization (or Linear Evaluation):
Top-1 accuracy, $\kappa$-NN, ECE, and NLL averaged over in-distribution on
test samples of the ImageNet dataset where the encoder is ViT-S/16 over 800
epochs. The best score for each metric is shown in bold, and the second-best
is underlined.

Method | Top-1 Acc (%) ($\uparrow$) |  $\kappa$-NN ($\uparrow$) | NLL ($\downarrow$) | ECE ($\downarrow$)
---|---|---|---|---
DINO Caron et al. (2021) | 76.8 | 74.5 | 0.919 | 0.015
MOCO-V3 Chen et al. (2021) | 73.2 | 64.7 | 1.152 | 0.027
i-BOT Zhou et al. (2021) | 77.9 | 75.2 | 0.918 | 0.013
ProSMin | 78.4 | 76.2 | 0.900 | 0.006

Corrupted dataset evaluation  An essential aspect of model robustness is its
capacity to produce precise predictions when the test data distribution
changes. We examine model robustness under the context of _covariate shift_.
Figure 2 presents the improved performance metrics. Our method achieves better
results than the baseline and comparable predictive performance to the
baseline in terms of mCE.

Semi-supervised and Low-shot learning on ImageNet

Following the semi-supervised protocol established in Chen et al. (2020), we
employ fixed 1% and 10% splits of labeled training data from ImageNet. In
Table 2, we compare our performance against several concurrent models,
including the baseline (DINO). Our results demonstrate that our features
trained for 300 epochs are on par with DINO. Moreover, we achieve better
results for 10% compared to state-of-the-art methods despite our network being
pretrained for significantly less time (300 epochs versus 800 epochs).

We assess our model’s efficacy on a low-shot image classification task, where
we train logistic regression on frozen weights with 1% and 10% labels. It’s
important to note that this experiment was performed on frozen weights,
without finetuning. Table 2 shows our features are on par with state-of-the-
art semi-supervised models which these methods trained for 800 epochs.

Table 2: Low-shot and semi-supervised evaluation: Top-1 accuracy (ACC), ECE,
and NLL for semi-supervised on ImageNet classification using 1% and 10%
training examples fine-tuning. and Low-shot results with frozen ViT features.

Method | 1% | 10% | Architecture | Parameters
---|---|---|---|---
Semi-supervised |  |  |  |
DINO Caron et al. (2021) | 60.3 | 74.3 | ViT-S/16 | 21
i-BOT Zhou et al. (2021) | 61.9 | 75.1 | ViT-S/16 | 21
ProSMin (ours) | 62.1 | 75.6 | ViT-S/16 | 21
Low-shot learning |  |  |  |
DINO Caron et al. (2021) | 64.5 | 72.2 | ViT-S/16 | 21
i-BOT Zhou et al. (2021) | 65.9 | 73.4 | ViT-S/16 | 21
ProSMin (ours) | 66.1 | 73.8 | ViT-S/16 | 21

Transfer learning evaluation  We further assess the generalization capacity of
the learned representation on learning a new dataset. We followed the same
transfer learning protocol explained in Caron et al. (2021). To this end, we
evaluate the performance of our model pretrained on ImageNet Deng et al.
(2009) to CIFAR10/100 Krizhevsky (2009), an imbalanced naturalist dataset
(iNat-18) Van Horn et al. (2018), and the Flower dataset Nilsback and
Zisserman (2008). According to the results shown in Table 3, we observe that
our method provides a robust solution when transferring to the new dataset.

Table 3: Transfer to new dataset evaluation: Transfer learning by finetuning
pretrained models on different datasets. We report top-1 accuracy. Self-
supervised pretraining with ProSMin transfers better than supervised
pretraining.

Method | CIFAR-10 | CIFAR-100 | iNat-18 | Flowers
---|---|---|---|---
DINO Caron et al. (2021) | 99.0 | 90.5 | 72 | 98.5
i-BOT Zhou et al. (2021) | 99.1 | 90.7 | 73.7 | 98.6
ProSMin (ours) | 99.0 | 90.2 | 72.5 | 98.5

##  7 Ablation study

To gain a deeper understanding of the behavior and performance of our proposed
method, we conduct several ablation studies to explore various aspects of our
approach. Specifically, we investigate the following factors: different
scoring rules as an objective function, the hyperparameter of our loss
function ($\lambda$), the number of samples used for generating latent
representations, the size of the embedding, the effect of the momentum
hyperparameter, the impact BN, and prediction network (PL). These
investigations aim to provide insights and intuition regarding our approach.

(a)

(b)

(c)

(d)

Figure 3: Study of hyperparameters of our proposed ProSMin (a) $\lambda$, (b)
Number of samples, (c) Momentum coefficient ($\alpha$), and (d) Size of
embedding obtained by 100 epochs on ImageNet dataset.

Impact of scoring rule  We conduct a series of experiments to explore
alternative scoring rules, including kernel scoring rules and various
variations of energy scoring rules, for our objective function. The results,
as presented in Table 4, demonstrate that the kernel scoring rule exhibits
instability. Since the kernel score becomes negative and it is not anymore a
proper scoring rule (See our proof in Section. 4.3). However, the energy score
with $\beta=1$ ($L1$ loss) yields the best performance among the tested
scoring rules. We provide a theoretical explanation for $L1$ and $L2$ (see
Section 9.5.3).

Table 4: Important component for training. We investigate the effect of each
component on the Linear evaluation performance. The first line shows the best
combination. PL. is the prediction layer. BN. is Batch normalization layer

Energy (L1 loss) | Kernel score | Energy (L2-loss) | BN. | PL. | Accuracy
---|---|---|---|---|---
✓ | ✗ | ✗ | ✗ | ✓ | 73.2
✗ | ✓ | ✗ | ✗ | ✓ | 1.1
✗ | ✗ | ✓ | ✗ | ✓ | 43
✓ | ✗ | ✗ | ✓ | ✓ | 68.5
✓ | ✗ | ✗ | ✗ | ✗ | 71.2
✓ | ✗ | ✗ | ✗ | ✓ | 71

Table 5: Evaluation of Computational Efficiency We conduct a thorough analysis
of the computational efficiency of our novel probabilistic approach in
comparison to alternative self-supervised methods. This evaluation encompasses
memory utilization and computational expenditure.

Method (ViT-s) | parameters (M) | im/s | time/ 300-epochs (hr) | number of GPUs | memory (G)
---|---|---|---|---|---
DINO Caron et al. (2021) | 21 | 1007 | 72.6 | 16 | 15.4
i-BOT Zhou et al. (2021) | 21 | 1007 | 73.8 | 16 | 19.5
ProSMin (ours) | 21 | 1007 | 98 | 8 | 21.1

Study of hyperparameters  Figure 3a depicts the influence of the
hyperparameter $\lambda$ utilized in our proposed loss function, which
controls the impact of $S^{1}(P_{\bm{\theta}},\bm{z}_{\bm{\xi}})$ and
$S^{2}(P_{\bm{\theta}},\bm{z}_{\bm{\xi}})$ on the objective function.

An ablation analysis was conducted to investigate the effect of increasing the
number of samples from $q_{\bm{\theta}}$, as illustrated in Figure 3b. The
results demonstrate that employing four samples yields satisfactory
performance.

Figure 3c showcases the outcomes obtained from the knowledge distillation
rate. In previous approaches, the exponential moving average parameter
initiated from a value relatively close to 1 (e.g., 0.996 Grill et al. (2020),
Caron et al. (2021)). However, in our case, $\alpha$ commences from 0.9,
implying a faster pace of knowledge distillation.

Furthermore, we examined the impact of different sizes for the embedding
vector, as presented in Fig. 3d. The results obtained after 100 epochs reveal
that increasing the embedding size leads to improved performance. However, it
should be noted that larger embedding sizes necessitate additional
computational resources, thus our choice of size was based on the available
computational capacity.

Table 4 provides insights into the influence of batch normalization in the
prevention of representation collapse Grill et al. (2020). As our framework
operates in a probabilistic setting, the inclusion of batch normalization is
unnecessary for averting collapse. Additionally, the prediction layer (PL
$q_{\theta}$) enhances performance by facilitating improved feature extraction
in online networks.

Analysis of computational cost We further assess the effectiveness of our
proposed approach and compare it with DINO and iBOT in Table 5. The presented
values are obtained from the data reported in the DINO and iBOT papers.
However, these papers do not include information regarding the total number of
parameters.

##  8 Conclusion

In this paper, we presented _ProSMin_ as a novel probabilistic self-supervised
framework that involves two neural networks which collaborate and learn from
each other using an augmented format. Our framework is trained by minimizing a
proposed scoring rule. We provided theoretical justification and showed that
our modified loss is strictly proper. We evaluated ProSMin across different
tasks, including in-distribution generalization, out-of-distribution
detection, dataset corruption, transfer learning, and semi-supervised
learning. The results demonstrate that our method achieves superior
performance in terms of accuracy and calibration, thus showing the
effectiveness of our proposed approach.

##### Broader impact and limitations

This study has the potential to inspire new algorithms and stimulate
theoretical and experimental exploration. The algorithm presented here can be
used for many different probabilistic downstream tasks, including (but not
limited to) uncertainty quantification, density estimation, image retrieval,
probabilistic unsupervised clustering, program debugging, image generation,
music analysis, and ranking. In addition, we believe that our extended concept
probabilistic framework opens many interesting avenues for future development
in self-supervised learning, and addresses many problems of existing models,
such as avoiding representation collapse.

However, there are several limitations. One limitation of our model compared
to other learning methods (such as supervised learning) is that self-
supervised learning may require more computational resources and training
time. However, considering that our proposed method does not require manual
annotation, which is usually very expensive, we would argue that this trade-
off is acceptable. In addition, due to limited computational resources, we
provide results for fewer epochs compared to other methods. Furthermore, we
believe that the results can be improved with extensive hyperparameter
optimization.

## References

  * Bardes et al. (2021) A. Bardes, J. Ponce, and Y. LeCun.  Vicreg: Variance-invariance-covariance regularization for self-supervised learning.  _ICLR_ , 2021.
  * Bengs et al. (2023) V. Bengs, E. Hüllermeier, and W. Waegeman.  On second-order scoring rules for epistemic uncertainty quantification.  _arXiv preprint arXiv:2301.12736_ , 2023.
  * Caron et al. (2021) M. Caron, H. Touvron, I. Misra, H. Jégou, J. Mairal, P. Bojanowski, and A. Joulin.  Emerging properties in self-supervised vision transformers.  In _Proceedings of the IEEE/CVF International Conference on Computer Vision_ , pages 9650–9660, 2021.
  * Chen et al. (2020) T. Chen, S. Kornblith, M. Norouzi, and G. Hinton.  A simple framework for contrastive learning of visual representations.  In _International conference on machine learning_ , pages 1597–1607. PMLR, 2020.
  * Chen et al. (2021) X. Chen, S. Xie, and K. He.  An empirical study of training self-supervised vision transformers.  In _Proceedings of the IEEE/CVF International Conference on Computer Vision_ , pages 9640–9649, 2021.
  * Dawid and Musio (2014) A. P. Dawid and M. Musio.  Theory and applications of proper scoring rules.  _Metron_ , 72(2):169–183, 2014.
  * Deng et al. (2009) J. Deng, W. Dong, R. Socher, L.-J. Li, K. Li, and L. Fei-Fei.  Imagenet: A large-scale hierarchical image database.  In _2009 IEEE conference on computer vision and pattern recognition_ , pages 248–255. Ieee, 2009.
  * Der Kiureghian and Ditlevsen (2009) A. Der Kiureghian and O. Ditlevsen.  Aleatory or epistemic? does it matter?  _Structural Safety_ , 31(2):105–112, 2009.
  * Dosovitskiy et al. (2021) A. Dosovitskiy, L. Beyer, A. Kolesnikov, D. Weissenborn, X. Zhai, T. Unterthiner, M. Dehghani, M. Minderer, G. Heigold, S. Gelly, J. Uszkoreit, and N. Houlsby.  An Image is Worth 16x16 Words: Transformers for Image Recognition at Scale.  In _Proceedings of the 9th International Conference on Learning Representations (ICLR)_ , 2021.
  * Dutordoir et al. (2021) V. Dutordoir, J. Hensman, M. van der Wilk, C. H. Ek, Z. Ghahramani, and N. Durrande.  Deep neural networks as point estimates for deep gaussian processes.  _Advances in Neural Information Processing Systems_ , 34:9443–9455, 2021.
  * Geng et al. (2020) C. Geng, S.-j. Huang, and S. Chen.  Recent advances in open set recognition: A survey.  _IEEE transactions on pattern analysis and machine intelligence_ , 43(10):3614–3631, 2020.
  * Ghahramani (2015) Z. Ghahramani.  Probabilistic machine learning and artificial intelligence.  _Nature_ , 521(7553):452–459, 2015.
  * Gneiting and Katzfuss (2014) T. Gneiting and M. Katzfuss.  Probabilistic forecasting.  _Annual Review of Statistics and Its Application_ , 1:125–151, 2014.
  * Gneiting and Raftery (2007) T. Gneiting and A. E. Raftery.  Strictly proper scoring rules, prediction, and estimation.  _Journal of the American statistical Association_ , 102(477):359–378, 2007.
  * Grill et al. (2020) J.-B. Grill, F. Strub, F. Altché, C. Tallec, P. Richemond, E. Buchatskaya, C. Doersch, B. Avila Pires, Z. Guo, M. Gheshlaghi Azar, et al.  Bootstrap your own latent-a new approach to self-supervised learning.  _Advances in neural information processing systems_ , 33:21271–21284, 2020.
  * Gruber and Buettner (2022) S. Gruber and F. Buettner.  Better uncertainty calibration via proper scores for classification and beyond.  _Advances in Neural Information Processing Systems_ , 35:8618–8632, 2022.
  * He et al. (2016) K. He, X. Zhang, S. Ren, and J. Sun.  Deep residual learning for image recognition.  In _Proceedings of the IEEE conference on computer vision and pattern recognition_ , pages 770–778, 2016.
  * He et al. (2022) K. He, X. Chen, S. Xie, Y. Li, P. Dollár, and R. Girshick.  Masked autoencoders are scalable vision learners.  In _Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition_ , pages 16000–16009, 2022.
  * Hendrycks and Dietterich (2019) D. Hendrycks and T. G. Dietterich.  Benchmarking neural network robustness to common corruptions and perturbations.  In _7th International Conference on Learning Representations, ICLR 2019, New Orleans, LA, USA, May 6-9, 2019_. OpenReview.net, 2019.  URL https://openreview.net/forum?id=HJz6tiCqYm.
  * Hinton et al. (2015) G. Hinton, O. Vinyals, and J. Dean.  Distilling the knowledge in a neural network.  _arXiv preprint arXiv:1503.02531_ , 2015.
  * Ho et al. (2020) J. Ho, A. Jain, and P. Abbeel.  Denoising diffusion probabilistic models.  _Advances in Neural Information Processing Systems_ , 33:6840–6851, 2020.
  * Jang et al. (2023) J. Jang, S. Kim, K. Yoo, C. Kong, J. Kim, and N. Kwak.  Self-distilled self-supervised representation learning.  In _Proceedings of the IEEE/CVF Winter Conference on Applications of Computer Vision_ , pages 2829–2839, 2023.
  * Kingma et al. (2015) D. P. Kingma, T. Salimans, and M. Welling.  Variational dropout and the local reparameterization trick.  _Advances in neural information processing systems_ , 28, 2015.
  * Krizhevsky (2009) A. Krizhevsky.  Learning multiple layers of features from tiny images.  Technical report, University of Toronto, 2009.
  * Loshchilov and Hutter (2017a) I. Loshchilov and F. Hutter.  Decoupled weight decay regularization.  _ICLR_ , 2017a.
  * Loshchilov and Hutter (2017b) I. Loshchilov and F. Hutter.  SGDR: stochastic gradient descent with warm restarts.  In _5th International Conference on Learning Representations, ICLR 2017, Toulon, France, April 24-26, 2017, Conference Track Proceedings_. OpenReview.net, 2017b.  URL https://openreview.net/forum?id=Skq89Scxx.
  * Naeini et al. (2015) M. P. Naeini, G. F. Cooper, and M. Hauskrecht.  Obtaining well calibrated probabilities using bayesian binning.  In _Proceedings of AAAI’15_ , page 2901–2907. AAAI Press, 2015.
  * Nichol and Dhariwal (2021) A. Q. Nichol and P. Dhariwal.  Improved denoising diffusion probabilistic models.  In _International Conference on Machine Learning_ , pages 8162–8171. PMLR, 2021.
  * Nilsback and Zisserman (2008) M.-E. Nilsback and A. Zisserman.  Automated flower classification over a large number of classes.  In _2008 Sixth Indian Conference on Computer Vision, Graphics & Image Processing_, pages 722–729. IEEE, 2008.
  * Oquab et al. (2023) M. Oquab, T. Darcet, T. Moutakanni, H. Vo, M. Szafraniec, V. Khalidov, P. Fernandez, D. Haziza, F. Massa, A. El-Nouby, et al.  Dinov2: Learning robust visual features without supervision.  _arXiv preprint arXiv:2304.07193_ , 2023.
  * Ozdemir et al. (2019) O. Ozdemir, R. L. Russell, and A. A. Berlin.  A 3d probabilistic deep learning system for detection and diagnosis of lung cancer using low-dose ct scans.  _IEEE transactions on medical imaging_ , 39(5):1419–1429, 2019.
  * Pacchiardi and Dutta (2022) L. Pacchiardi and R. Dutta.  Likelihood-free inference with generative neural networks via scoring rule minimization.  _arXiv preprint arXiv:2205.15784_ , 2022.
  * Pacchiardi et al. (2021) L. Pacchiardi, R. Adewoyin, P. Dueben, and R. Dutta.  Probabilistic forecasting with generative networks via scoring rule minimization.  _Preprint, URL https://arxiv. org/abs/2112.08217_ , 2021.
  * Parry (2016) M. Parry.  Linear scoring rules for probabilistic binary classification.  _Electronic Journal of Statistics_ , 10:1596–1607, 2016\.
  * Peharz et al. (2020) R. Peharz, S. Lang, A. Vergari, K. Stelzner, A. Molina, M. Trapp, G. Van den Broeck, K. Kersting, and Z. Ghahramani.  Einsum networks: Fast and scalable learning of tractable probabilistic circuits.  In _International Conference on Machine Learning_ , pages 7563–7574. PMLR, 2020.
  * Rezaei et al. (2022) M. Rezaei, J. J. Näppi, B. Bischl, and H. Yoshida.  Bayesian uncertainty estimation for detection of long-tail and unseen conditions in abdominal images.  In _Medical Imaging 2022: Computer-Aided Diagnosis_ , volume 12033, pages 270–276. SPIE, 2022.
  * Rezaei et al. (2023) M. Rezaei, F. Soleymani, B. Bischl, and S. Azizi.  Deep bregman divergence for self-supervised representations learning.  _Computer Vision and Image Understanding_ , page 103801, 2023.
  * Srivastava et al. (2022) A. Srivastava, S. Jain, and M. Thigle.  Out of distribution detection on imagenet-o.  _arXiv preprint arXiv:2201.09352_ , 2022.
  * Tran et al. (2022) D. Tran, J. Liu, M. W. Dusenberry, D. Phan, M. Collier, J. Ren, K. Han, Z. Wang, Z. Mariet, H. Hu, et al.  Plex: Towards reliability using pretrained large model extensions.  _arXiv preprint arXiv:2207.07411_ , 2022.
  * Vahidi et al. (2023) A. Vahidi, L. Wimmer, H. A. Gündüz, B. Bischl, E. Hüllermeier, and M. Rezaei.  Diversified ensemble of independent sub-networks for robust self-supervised representation learning.  _arXiv preprint arXiv:2308.14705_ , 2023.
  * Van Horn et al. (2018) G. Van Horn, O. Mac Aodha, Y. Song, Y. Cui, C. Sun, A. Shepard, H. Adam, P. Perona, and S. Belongie.  The inaturalist species classification and detection dataset.  In _Proceedings of the IEEE conference on computer vision and pattern recognition_ , pages 8769–8778, 2018.
  * V’yugin and Trunov (2019) V. V. V’yugin and V. G. Trunov.  Online learning with continuous ranked probability score.  In _Conformal and Probabilistic Prediction and Applications_ , pages 163–177. PMLR, 2019.
  * Wang et al. (2021) W. Wang, W. Zhou, J. Bao, D. Chen, and H. Li.  Instance-wise hard negative example generation for contrastive learning in unpaired image-to-image translation.  In _Proceedings of the IEEE/CVF International Conference on Computer Vision_ , pages 14020–14029, 2021.
  * Zbontar et al. (2021) J. Zbontar, L. Jing, I. Misra, Y. LeCun, and S. Deny.  Barlow twins: Self-supervised learning via redundancy reduction.  In _International Conference on Machine Learning_ , pages 12310–12320. PMLR, 2021.
  * Zhou et al. (2021) J. Zhou, C. Wei, H. Wang, W. Shen, C. Xie, A. Yuille, and T. Kong.  ibot: Image bert pre-training with online tokenizer.  _ICLR_ , 2021.

##  9 Scoring Rules

The notation employed in this study aligns with the work of Gneiting et al.
Gneiting and Raftery (2007) regarding scoring rules $S(P,\bm{x})$, where $P$
represents the predictive distribution and $\bm{x}$ denotes the observed data.
By assuming that $\mathbf{X}$ follows a true distribution $Q$. With this
assumption, we first explain different variations of scoring rules, such as
the expected scoring rule 9.1, the proper scoring rule 9.2, the energy scoring
rule 9.3, and the kernel scoring rule 9.4.

We then discuss the unbiased estimation properties of the scoring rule 9.5 for
kernel score 9.5.1 and energy score 9.5.2, which we used as the objective
function in our study.

In subsection 9.5.4, we establish the interchangeability of the expectation
and gradient in our theoretical derivative, enabling us to derive gradients in
a conventional manner. This is crucial due to the incorporation of samples
within the scoring rule and the utilization of non-differential activation
functions. Subsequently, in subsection 9.5.5, we introduce an unbiased
estimate of the gradient. By completing these three steps, we guarantee
effective convergence of our algorithm (refer to 4.3).

###  9.1 Expected scoring rule

We define the scoring rule $S(P,x)$ as a function of the distribution $P$ and
the observation $x$ of $\mathbf{X}$. The expected scoring rule is defined as:

$\displaystyle S(P,Q):=\mathbb{E}_{\mathbf{X}\sim Q}S(P,\mathbf{X}).$ (8)

###  9.2 Proper scoring rule

A scoring rule $S$ is called _proper_ w.r.t a set of distributions
$\mathcal{P}$, if for all $P,Q\in\mathcal{P}$ the expected score $S(P,Q)$ is
minimized in $Q$ at $Q=P$. A scoring rule $S$ is called strictly proper if
there exists the unique minimum $S(P,Q)>S(Q,Q)$ for $Q\neq P$.

###  9.3 Energy score

The energy score is defined by:

$\displaystyle
S_{\mathrm{E}}^{(\beta)}(P,\mathbf{x})=2\cdot\mathbb{E}\left[\|\tilde{\mathbf{X}}-\mathbf{x}\|_{2}^{\beta}\right]-\mathbb{E}\left[\left\|\tilde{\mathbf{X}}-\tilde{\mathbf{X}}^{\prime}\right\|_{2}^{\beta}\right],\quad\tilde{\mathbf{X}}\perp\\!\\!\\!\perp\tilde{\mathbf{X}}^{\prime}\sim
P,$ (9)

where $\beta\in(0,2)$. It is a strictly proper scoring rule for the class of
probability measures $\mathcal{P}$ such that
$\mathbb{E}_{\tilde{\mathbf{X}}\sim P}\|\tilde{\mathbf{X}}\|^{\beta}<\infty$.
An unbiased estimate can be obtained by replacing the expectations in
$S_{\mathrm{E}}^{(\beta)}$ with empirical means over draws from $P$.

###  9.4 Kernel score

Let $k(\cdot,\cdot)$ be a positive definite kernel. The kernel score for $k$
is defined by

$\displaystyle
S_{k}(P,\mathbf{x})=\mathbb{E}\left[k\left(\tilde{\mathbf{X}},\tilde{\mathbf{X}}^{\prime}\right)\right]-2\cdot\mathbb{E}[k(\tilde{\mathbf{X}},\mathbf{x})],\quad\tilde{\mathbf{X}}\perp\\!\\!\\!\perp\tilde{\mathbf{X}}^{\prime}\sim
P.$ (10)

The kernel score is a proper scoring rule for the class of probability
distribution $P$ for which holds, that
$\mathbb{E}_{\tilde{\mathbf{X}},\tilde{\mathbf{X}}^{\prime}\sim
P}\left[k\left(\tilde{\mathbf{X}},\tilde{\mathbf{X}}^{\prime}\right)\right]<\infty$.
Under the condition that the kernel Maximum Mean Discrepancy is a metric, the
kernel score is strictly proper. This condition is satisfied by the Gaussian
kernel, which will be used in this work. Let $\gamma$ be a scalar bandwidth.
We define the Gaussian kernel as follows:

$\displaystyle
k(\tilde{\mathbf{x}},\mathbf{x})=\exp\left(-\frac{\|\tilde{\mathbf{x}}-\mathbf{x}\|_{2}^{2}}{2\gamma^{2}}\right).$

We get an unbiased estimate by replacing the expectations in $S_{\mathrm{k}}$
with the empirical means overdraws from $P$.

###  9.5 Unbiased estimators of the expected scoring rule

Let $N$ be the batch size and $r$ be the number of samples drawn from the
online-induced distribution. Let $z_{\theta}=(z_{1},...,z_{r})$ be the sample
drawn from the online. We recall the definition of the Expected Scoring Rule:

$\displaystyle S(P_{\theta},P_{\xi}):=\mathbb{E}_{z_{\xi}\sim
P_{\xi}}[S(P_{\theta},z_{\xi})],$

where $z_{\xi}$ denotes the output of the target network while $P_{\theta}$
represents the online induced multivariate normal distribution.

####  9.5.1 Kernel score

We can write the estimated kernel score as follows:

$\displaystyle\hat{S}(P_{\theta},P_{\xi})=\dfrac{1}{N}\sum_{i=1}^{N}\\!\left[\frac{1}{r(r-1)}\sum_{j,k=1,j\neq
k}k(z_{j}^{i},z_{k}^{i})-\dfrac{2}{r}\sum_{j=1}^{r}k(z_{j}^{i},z_{\xi}^{i})\\!\right]$

####  9.5.2 Energy score

Consider $\beta\in(0,2)$ and $N$ be the batch size. The Estimated Energy Score
can be written as:

$\displaystyle\hat{S}(P_{\theta},P_{\xi})=\dfrac{1}{N}\sum_{i=1}^{N}\\!\left[\dfrac{2}{r}\sum_{j=1}^{r}\|z_{j}^{i}-z_{\xi}^{i}\|^{\beta}_{2}-\frac{1}{r(r-1)}\sum_{j,k=1,j\neq
k}\|z_{j}^{i}-z_{k}^{i}\|^{\beta}_{2}\\!\right]$

####  9.5.3 Partial derivatives of the estimated energy score

Let $\beta\in(0,2)$ and let $N$ be the batch size. Let $x_{i}$ be the input
data for $i\in{1,...,N}$. Let $r\in\mathbb{N}$ be the number of samples. Let
$\mu_{i}=f^{(1)}_{\theta}(x_{i})$ and let
$\sigma^{2}_{i}=f^{(2)}_{\theta}(x_{i})$, where $f^{(j)}$ denotes the
separated branches for mean and variance in the last layer of the online
network for $j\in{1,2}$. We can calculate the partial derivatives needed for
stochastic gradient descent as follows. Then, the formula for the mean is:

$\displaystyle\dfrac{\partial}{\partial\mu}\hat{S}(P_{\theta},P_{\xi})$
$\displaystyle=\dfrac{\partial}{\partial\mu}\dfrac{1}{N}\sum_{i=1}^{N}\left[\dfrac{2}{r}\sum_{j=1}^{r}\left\|\mu_{i}+\epsilon_{j}\sqrt{\sigma^{2}_{i}}-\phi_{\xi}^{i}\right\|^{\beta}_{2}\right.$
$\displaystyle\quad\left.-\frac{1}{r(r-1)}\sum_{j,k=1,j\neq
k}\left\|\mu_{i}+\epsilon_{j}\sqrt{\sigma^{2}_{i}}-\mu_{i}+\epsilon_{k}\sqrt{\sigma^{2}_{i}}\right\|^{\beta}_{2}\right]$
$\displaystyle=\dfrac{1}{N}\sum_{i=1}^{N}\dfrac{2}{r}\sum_{j=1}^{r}\beta\left\|\mu_{i}+\epsilon_{j}\sqrt{\sigma^{2}_{i}}-\phi_{\xi}^{i}\right\|^{\beta-1}_{2}\dfrac{\partial}{\partial\mu}\left[\mu_{i}+\epsilon_{j}\sqrt{\sigma^{2}_{i}}-\phi_{\xi}^{i}\right]$
$\displaystyle=\dfrac{1}{N}\sum_{i=1}^{N}\dfrac{2}{r}\sum_{j=1}^{r}\beta\left\|f^{(1)}_{\theta}(x_{i})+\epsilon_{j}\sqrt{f^{(2)}_{\theta}(x_{i})}-\phi_{\xi}^{i}\right\|^{\beta-1}_{2}\nabla_{\theta}f^{(1)}_{\theta}(x_{i})$

while the formula for the variance is:

$\displaystyle\dfrac{\partial}{\partial\sigma^{2}}\hat{S}(P_{\theta},P_{\xi})$
$\displaystyle=\dfrac{\partial}{\partial\sigma^{2}}\dfrac{1}{N}\sum_{i=1}^{N}\left[\dfrac{2}{r}\sum_{j=1}^{r}\left\|\mu_{i}+\epsilon_{j}\sqrt{\sigma^{2}_{i}}-\phi_{\xi}^{i}\right\|^{\beta}_{2}\right.$
$\displaystyle\quad\left.-\frac{1}{r(r-1)}\sum_{j,k=1,j\neq
k}\left\|\mu_{i}+\epsilon_{j}\sqrt{\sigma^{2}_{i}}-\mu_{i}+\epsilon_{k}\sqrt{\sigma^{2}_{i}}\right\|^{\beta}_{2}\right]$
$\displaystyle=\left[\dfrac{\beta}{N}\sum_{i=1}^{N}\left[\dfrac{2}{r}\sum_{j=1}^{r}\left\|f^{(1)}_{\theta}(x_{i})+\epsilon_{j}\sqrt{f^{(2)}_{\theta}(x_{i})}-\phi_{\xi}^{i}\right\|^{\beta-1}_{2}\right.\right.$
$\displaystyle\quad\left.\left.-\frac{1}{r(r-1)}\sum_{j,k=1,j\neq
k}\left\|\sqrt{f^{(2)}_{\theta}(x_{i})}\left(\epsilon_{j}-\epsilon_{k}\right)\right\|^{\beta-1}_{2}\right]\right]\nabla_{\theta}f^{(2)}_{\theta}(x_{i})$

####  9.5.4 Exchangeability of gradient and expectation

This proof follows a similar argumentation as the proof performed in
Pacchiardi and Dutta (2022). We want to solve

$\hat{\bm{\theta}}:=\operatorname*{arg\,min}_{\bm{\theta}}J(\bm{\theta}),\quad
J(\bm{\theta})=\dfrac{1}{N}\sum_{i=1}^{N}S(P_{\bm{\theta}},\bm{z}^{i}_{\bm{\xi}})$
(11)

Finding a minimum in our network architecture is done via stochastic gradient
descent (SGD) or an algorithm exploiting the same properties as SGD. Recall
that the scoring rule is defined as an expectation over samples from
$P_{\bm{\theta}}$. Additionally, we can describe the scoring rule as some
function $h$ of independent inputs
$\bm{z}^{i}_{j}\perp\\!\\!\\!\perp\bm{z}^{i}_{k}$. Hence we can write:

$S(P_{\bm{\theta}},\bm{z}^{i}_{\bm{\xi}})=\mathbb{E}_{\bm{z}^{i}_{j},\bm{z}^{i}_{k}\sim
P_{\bm{\theta}}}\left[h\left(\bm{z}^{i}_{j},\bm{z}^{i}_{k},\bm{z}^{i}_{\bm{\xi}}\right)\right]$
(12)

Let $\mu_{i},\sigma^{2}_{i}$ be the estimated mean and variance, respectively,
of the online network $f_{\bm{\theta}}$ for the $i$-th input sample
$\bm{x}_{i}$. With $P_{\bm{\theta}}$ being the distribution induced by the
online network and $g_{\bm{\theta}}$ being its transformation, we use the
reparametrization trick and get

$J(\bm{\theta})=\dfrac{1}{N}\sum_{i=1}^{N}\mathbb{E}_{\epsilon^{i}_{j},\epsilon^{i}_{k}\sim\mathcal{N}(0,1)}\left[h(g_{\bm{\theta}}(\epsilon^{i}_{j},\mu_{i},\sigma^{2}_{i}),g_{\bm{\theta}}(\epsilon^{i}_{k},\mu_{i},\sigma^{2}_{i}),\bm{z}^{i}_{\bm{\xi}})\right]$
(13)

We now can write the derivative as follows:

$\displaystyle\nabla_{\bm{\theta}}J(\bm{\theta})$
$\displaystyle=\nabla_{\bm{\theta}}\dfrac{1}{N}\sum_{i=1}^{N}\mathbb{E}_{\epsilon^{i}_{j},\epsilon^{i}_{k}\sim\mathcal{N}(0,1)}\left[h(g_{\bm{\theta}}(\epsilon^{i}_{j},\mu_{i},\sigma^{2}_{i}),g_{\bm{\theta}}(\epsilon^{i}_{k},\mu_{i},\sigma^{2}_{i}),\bm{z}^{i}_{\bm{\xi}})\right]$
$\displaystyle=\dfrac{1}{n}\sum_{i=1}^{n}\mathbb{E}_{\epsilon^{i}_{j},\epsilon^{i}_{k}\sim\mathcal{N}(0,1)}\left[\nabla_{\bm{\theta}}h(g_{\bm{\theta}}(\epsilon^{i}_{j},\mu_{i},\sigma^{2}_{i}),g_{\bm{\theta}}(\epsilon^{i}_{k},\mu_{i},\sigma^{2}_{i}),\bm{z}^{i}_{\bm{\xi}})\right]$

The exchange between expectation and gradient is not trivial because of the
non-differentiability of functions (e.g., ReLU) within the network function
$g_{\bm{\theta}}$. We can still perform this step by using Theorem 5 from
Pacchiardi and Dutta (2022), considering mild conditions on the neural network
architecture of $g_{\bm{\theta}}$.

####  9.5.5 Unbiased estimate of the gradient

Let $\mathcal{B}$ be a random subset (batch) of the data set. Using 9.5.4, we
can obtain unbiased estimates of $\nabla_{\bm{\theta}}J(\bm{\theta})$ using
samples $\epsilon^{j}_{i}\sim\mathcal{N}(0,1),j=1,...,r$ for each
$i\in\\{1,...,N\\}$:

$\widehat{\nabla_{\bm{\theta}}J(\bm{\theta})}=\dfrac{1}{|\mathcal{B}|}\sum_{i\in\mathcal{B}}\dfrac{1}{r(r-1)}\sum_{j,k=1,j\neq
k}^{r}\nabla_{\bm{\theta}}h(g_{\bm{\theta}}(\epsilon^{k}_{i},\mu_{i},\sigma^{2}_{i}),g_{\bm{\theta}}(\epsilon^{j}_{i},\mu_{i},\sigma^{2}_{i}),\bm{z}^{i}_{\bm{\xi}})$
(14)

####  9.5.6 Detailed implementation

In addition to the detail in Section 5, it’s important to mention that the
target network takes two global augmented samples while the online network
takes two global augmented samples and 16 local augmentations for multi-
cropping samples. Then, the $r$ in Eq. 2 refers to the number of augmentation
samples multiplied by the number of samples. Furthermore, in Eq. 2 $z_{\xi}$
denotes one of the global augmented samples.


# Energy dependent ratios of level-density parameters in superheavy nuclei

A. Rahmatinejad, T. M. Shneidman, G. Adamian, N. V. Antonenko
<EMAIL_ADDRESS>Joint Institute for Nuclear Research, Dubna, 141980,
Russia  P. Jachimowicz  Institute of Physics, University of Zielona Góra,
Szafrana 4a, 65516 Zielona Góra, Poland  M. Kowal  National Centre for Nuclear
Research, Pasteura 7, 02-093 Warsaw, Poland

###### Abstract

The nuclear level densities and level-density parameters in fissioning nuclei
at their saddle points of fission barriers - $a_{f}$, as well as those for
neutron - $a_{n}$, proton - $a_{p}$ , and $\alpha$-particle - $a_{\alpha}$
emission residues at the ground states are calculated for isotopic chains of
superheavy nuclei with $Z$=112-120. The calculations are performed with the
superfluid formalism using the single-particle energies obtained from the
diagonalization of the deformed Woods-Saxon potential. Spectra were generated
at global minima of the adiabatic potential energy surfaces, found by the
multidimensional minimization method, and at the proper saddle points, found
by the ”immersion water flow” technique on multidimensional energy grids, with
allowed the reflection and axial symmetry breaking. The influence of shell
effects on the energy dependence of the ratios of level-density parameters
corresponding to residues of the considered decay modes to those of neutron
emission is studied. We have shown that, in contrast to the $a_{f}/a_{n}$
ratio, the $a_{p}/a_{n}$ and $a_{\alpha}/a_{n}$ ratios do not show
characteristic maxima depending on the excitation energy of the compound
nucleus being formed. In the case of alpha decay, we identified the collective
enhancement caused by cluster degrees of freedom to play quite an important
role. The energetic course of the variability of the level density parameters
before reaching the asymptotic value, not taken into account so far, will be
of great importance for the estimation of the probabilities of de-excitation
cascades via light particles emission in competition with splitting and thus
for the determination of the survival probabilities and finally for the total
production cross-sections of superheavy nuclei in channels with their (light
particles) participation.

###### pacs:

21.10.Ma, 21.10.Pc, 24.60.Dr, 24.75.+i
Key words: microscopic-macroscopic model, fission barrier, level-density
parameter, survival probability, superheavy nuclei

##  I Introduction

The experiments on complete fusion reactions with 48Ca beams and actinide
targets were successfully carried out at FLNR (Dubna), GSI (Darmstadt), and
LBNL (Berkeley) in order to synthesize superheavy nuclei (SHN) with the charge
numbers $Z=112-118$ [1, 2, 3, 4, 5, 6, 7]. The formation of high-$Z$
evaporation residues in the complete fusion reactions can be roughly treated
as a three-stage process: capture, fusion, and survival against fission and
charged particle emissions. To analyze the competition between different decay
processes in the hot compound nucleus and to calculate evaporation residues
cross-sections, one should reliably know the level densities of the final
states in various decay modes. This includes the level densities of the nuclei
produced after neutron, proton, or $\alpha$-particle emissions and the level
densities of the fissioning nucleus at the saddle point (SP) of the fission
barrier.

The nuclear level densities (NLD) can be calculated with sophisticated
combinatorial models either built on mean-field theory [8, 9] or taking into
account effective nucleon-nucleon interactions [10, 11]. In Ref. [12],
explicitly treating parity, angular momentum, pairing, and shell correlations
as well as rotational and vibrational collective enhancements, the
combinatorial method has been applied to estimate NLD at the fission SP of
actinides. A recently developed combinatorial method [13] has been applied for
treating the energy and angular-momentum dependencies of nuclear shape
evolution in the fission process. Another technique is the direct microscopic
calculation of NLD with the shell model Monte Carlo method used in Ref. [14]
for the iron region and the rare-earth nucleus 162Dy. Unlike the mentioned
models [8, 9, 10, 11, 12, 13, 14], in which large-scale computations are
required, the thermodynamical models based on the partition function method
provide a simpler formalism. With its analytical formula, the Fermi-gas model
remains the simplest and widely used model for the calculation of level
densities [15]. From the Fermi-gas prescription for NLD, the level-density
parameter $a$ is a crucial factor indicating the relative importance of each
decay mode. In many applications, the average value of the level-density
parameter is often assumed to depend linearly on the mass number $A$ [16]. But
in a nucleus that has an internal structure, the level-density parameter
depends on the distribution of levels or shell effects. Damping of shell
effects with increasing energy causes the level-density parameter to reach its
asymptotic value. In the cases of large shell correction at the ground state
(GS), the shell effect in $a$ persists up to high energies. Within the
thermodynamic superfluid model [17], the level densities of superheavy nuclei
(SHN) at the ground state and the SP were systematically studied and the
energy and shell-correction dependencies of level-density parameters were
compared with those of phenomenological approach [18]. Using all binding
energies, fission thresholds as well as shell corrections at the GS and SP
from the microscopic-macroscopic model [19], it has been shown in Ref. [17]
that the ratio of the level-density parameter at the SP to the one at the GS
is crucial for the survival probability of excited SHN. The present work is
mainly dedicated to the analysis of energy dependence of the relative level-
density parameters corresponding to the residues of different decay channels
including neutron, proton, and $\alpha$-particle emissions, at their GS as
well as fission mode at the SP with using the same mass tables [19]. Isospin
and shell-correction dependencies of level-density parameter ratios at given
excitation energies are compared in different decay channels. For this aim,
the level densities and level-density parameters for SHN with $Z$=112-120 are
calculated within the thermodynamic superfluid formalism [20, 21] which allows
us to take into account the temperature dependence of pairing and shell
effects. Application of this formalism was examined in Ref. [22] through the
comparison with the available experimental level densities for Dy and Mo
isotopes. Previously the superfluid model was applied to study the level
densities of SHN and dinuclear systems using the single-particle spectra
obtained with the modified two-center shell model [23, 24].

##  II Level-density parameter

In this work, the single-particle energies are obtained in the microscopic-
macroscopic model based on the deformed single-particle Woods-Saxon potential
[25] and Yukawa-plus exponential macroscopic energy [26] with parameters
specified in [27]. Both non-axial and mass-asymmetrical shapes are allowed in
the parametrization of the shape of the nuclear surface. SP are found via
immersion water flow method on multi-dimensional spaces of deformations while
GS by conjugate gradient minimization method. For odd and odd-odd nuclei,
adiabatic potential energy surfaces were calculated by additional minimization
over configurations with one blocked neutron or/and proton on a level from the
10th below to the 10th above the Fermi level, for details see [28, 30, 29,
19]. Thus obtained decay thresholds, i.e. fission barriers, separation
energies, shell corrections, with an additional estimated Coulomb barriers for
charged particles, are taken into account in the next step in which we use the
statistical formalism allowing us to estimate the level-density parameters
with the deformed single-particle spectra. Based on the superfluid formalism
[21] and with an assumption of thermal equilibrium between neutron (the number
of neutrons is $N$) and proton subsystems, the constants of the pairing
interaction for neutrons ($G_{N}$) and protons ($G_{Z}$) are adjusted to
obtain the GS pairing gaps ($\Delta_{N}$ and $\Delta_{Z}$) at zero temperature
with Bardeen-Cooper-Schrieffer (BCS) equations:

$Z=\sum_{k}\left(1-\frac{\varepsilon^{Z}_{k}-\lambda_{Z}}{E^{Z}_{k}}\tanh\frac{\beta
E^{Z}_{k}}{2}\right),$ (1)
$N=\sum_{k}\left(1-\frac{\varepsilon^{N}_{k}-\lambda_{N}}{E^{N}_{k}}\tanh\frac{\beta
E^{N}_{k}}{2}\right),$ (2)
$\frac{2}{G_{N,Z}}=\sum_{k}\frac{1}{E^{N,Z}_{k}}\tanh\frac{\beta
E^{N,Z}_{k}}{2},$ (3)

where $Z$ and $N$ are the numbers of protons and neutrons in the nucleus,
respectively. The quasiparticle energies
$E^{N,Z}_{k}=\sqrt{(\varepsilon^{N,Z}_{k}-\lambda_{N,Z})^{2}+\Delta_{N,Z}^{2}}$
are obtained using the single-particle energies ($\varepsilon^{N,Z}_{k}$)
calculated with the deformed Woods-Saxon potential. In our calculations, in
the cases of $\Delta_{N,Z}<0.25$ MeV the pairing gap is taken as 0.25 MeV.
Using the values of pairing constants obtained, the pairing gaps and chemical
potentials $\lambda$ are determined by solving Eqs. (1), (2), and (3) at given
temperatures $T=1/\beta$. Then, using these values, the excitation energies
$U=U_{Z}+U_{N}$, entropies $S=S_{Z}+S_{N}$ and intrinsic level densities
$\rho$ are calculated as

$E_{Z,N}(T)=\sum_{k}\varepsilon_{k}\left(1-\frac{\varepsilon^{Z,N}_{k}-\lambda_{Z,N}}{E^{Z,N}_{k}}\tanh\frac{\beta
E^{Z,N}_{k}}{2}\right)-\frac{\Delta^{2}_{Z,N}}{G_{Z,N}},$ (4)
$U_{Z,N}(T)=E_{Z,N}(T)-E_{Z,N}(0),$ (5)
$S_{Z,N}(T)=\sum_{k}\left\\{\ln[1+\exp(-\beta E^{Z,N}_{k})]+\frac{\beta
E^{Z,N}_{k}}{1+\exp(\beta E^{Z,N}_{k})}\right\\},$ (6)
$\rho=\frac{\exp{(S)}}{(2\pi)^{\frac{3}{2}}\sqrt{D}},$ (7)

where

$D=\begin{vmatrix}\frac{\partial^{2}S}{\partial\beta^{2}}&\frac{\partial^{2}S}{\partial\beta\partial\mu_{Z}}&\frac{\partial^{2}S}{\partial\beta\partial\mu_{N}}\\\
\frac{\partial^{2}S}{\partial\beta\partial\mu_{Z}}&\frac{\partial^{2}S}{\partial\mu_{Z}^{2}}&0\\\
\frac{\partial^{2}S}{\partial\beta\partial\mu_{N}}&0&\frac{\partial^{2}S}{\partial\mu_{N}^{2}}\end{vmatrix}$
(8)

is the determinant of the matrix comprised of the second derivatives of the
entropy with respect to $\beta$ and $\mu_{Z,N}=\beta\lambda_{Z,N}$. From the
condition of thermal equilibrium between neutron and proton subsystems the
terms $\partial S^{2}/\partial\mu_{N}\partial\mu_{Z}$ are assumed to be zero.

Figure 1: Excitation energy dependence of the intrinsic level densities
calculated for: (a) 292Fl, and (b) 300120 with the single-particle spectra
obtained at their GS (red lines) and SP (blue lines). The inserted panels
(green lines) show the ratio of the level densities at the GS to those at the
SP. Here, the excitation energy $U$ is with respect to the corresponding
potential energy.

The calculations are performed for the isotopic chains of nuclei with
$Z$=112-120 to obtain the information needed to estimate the evaporation
residues cross sections in neutron, proton, and $\alpha$-particle emission
channels. In order to obtain the level densities of nuclei in fission channel,
the calculations were performed for the same nuclei using the single-particle
energies obtained with the Woods-Saxon potential at the SP. In the BCS
calculations at SP, the pairing constants were taken as in the GS. In Fig. 1,
the excitation energy dependencies of the intrinsic level densities are
presented for 292Fl, and 300120, at their fission SP and GS. The ratios of
these level densities are shown as well. Because of higher density of single-
particle states at the SP, the NLD at the SP are larger than those at the GS
at the same excitation energy. Note that at in the calculations of fission
width the excitation energies are not the same at the GS and SP. For a
comparison between various decays, the energy cost to overcome the fission SP
should be taken into consideration.

Fitting the calculated intrinsic level density as a function of excitation
energy in the interval between 10 and 100 MeV with the back-shifted Fermi gas
expression [15]

$\rho_{FG}(U)=\frac{\sqrt{\pi}}{12a^{1/4}(U-\Delta)^{5/4}}\exp({2\sqrt{a(U-\Delta)}}),$
(9)

the energy dependent level-density parameter $a(U)$ is obtained. The energy
back-shifts in Eq. (9) are taken as $\Delta=12/\sqrt{A},0,-12/\sqrt{A}$ for
even-even, odd-$A$ and odd-odd isotopes, respectively. Note that in the cases
of particle emissions resulting in the residues with very close values of
shell-corrections, different definition of energy back-shifts can affect the
energy dependence of level-density parameter ratios at low energies. However,
it is not the case of the level-density parameter ratios in the energy range
of interest $U>25$ MeV.

In Fig. 2, the calculated level-density parameters are shown for the neutron,
proton, and $\alpha$-particle decay channels as well as for fission mode
versus the excitation energy of mother nuclei: 288Mc, 296Og and ${}^{297}119$.
The energy costs due to fission barrier $B_{f}$, neutron separation energy
$B_{n}=V_{c}^{p}-Q_{p}$, and proton $B_{p}$ and $\alpha$-particle
$B_{\alpha}=V_{c}^{\alpha}-Q_{\alpha}$ energy thresholds, where the Coulomb
barriers for charged particles

$\displaystyle
V_{c}^{p,\alpha}=\frac{1.44(Z-Z_{p,\alpha})Z_{p,\alpha}}{C_{p,\alpha}[(A-A_{p,\alpha})^{1/3}+A_{p,\alpha}^{1/3}]}$
(10)

and $Q_{p,\alpha}$-values from the microscopic-macroscopic model [19] are
taken into consideration. In Eq. (10), $A_{p}=Z_{p}=1$,
$A_{\alpha}=2Z_{\alpha}=4$, and the constants $C_{p,\alpha}$ are taken as
$C_{p}=1.7$ fm and $C_{\alpha}=1.57$ fm [31].

Figure 2: Level density parameters obtained for the residual nuclei
corresponding to neutron, proton, and $\alpha$-particle decay channels as well
as fission versus excitation energy of mother nuclei: (a) 288Mc, (b) 296Og,
and (c) ${}^{297}119$. $B_{f}$ is the fission barrier whereas $B_{\tau}$
($\tau=n,p,\alpha$) are the particle emission energy thresholds corresponding
to each decay channel, respectively.

As pairing effect is expected to be damped out at energies above the neutron
separation, the energy dependence of level-density parameters in Fig. 2 is
mainly explained by the damping of shell correction with excitation energy.
The level-density parameters calculated for fissioning nuclei at the SP with
smaller shell corrections than in the GS reach their asymptotic values when
the energies are already above 30 MeV. However, the shell effects at the GS
survive up to energies higher than 50 MeV. As seen in Fig. 2, different
damping rates of the shell effects result in particular energy dependent
ratios of the level-density parameters in each decay channels. This effect is
important to be taken into account in the study of competition between
different decay modes at the excitation energies at which the shell effects
are not completely damped out.

##  III  Ratios of level-density parameters

Survival of excited SHN under neutron emission is strongly affected by the
ratios of level-density parameters

$\displaystyle\frac{a_{f}}{a_{n}}=\frac{a_{sp}(A,U-B_{f})}{a_{gs}(A-1,U-B_{n})},$
(11)
$\displaystyle\frac{a_{p}}{a_{n}}=\frac{a_{gs}(A-1,U-B_{p})}{a_{gs}(A-1,U-B_{n})},$
(12)

and

$\displaystyle\frac{a_{\alpha}}{a_{n}}=\frac{a_{gs}(A-4,U-B_{\alpha})}{a_{gs}(A-1,U-B_{n})}.$
(13)

The dependencies of $a_{f}/a_{n}$ on excitation energy are displayed in Fig. 3
for 279-291Nh and 291-299Og isotopic chains. The value of $a_{f}/a_{n}$ grows
fast to a maximum and then slowly falls to asymptotic value less than 1.1.
Depending on the difference between the shell-corrections at the SP and at the
GS as well as on their damping rates, the values of $a_{f}/a_{n}$ for
different isotopes reach maxima at different energies but lower than 25 MeV.
The behavior of $a_{f}/a_{n}$ as a function of $U$ depends on the difference
between neutron separation energy and the SP height as well as on the shell
and pairing effects.

Figure 3: Excitation energy dependence of $a_{f}/a_{n}$ calculated from Eq.
(11) for 279-291Nh nuclei - panels: (a), (b) and 291-299Og - panels: (c), (d).
The mass number $A$ of the given isotope is indicated by the selected color.

For example, the curve for 296Og in Fig. 3 (d) has a maximum at 15 MeV and
monotonically decreases with energy to its asymptotic. As seen in Fig. 2 (b),
various energy dependencies of level-density parameters, which is due to
different decay thresholds, can lead to a maximum of $a_{f}/a_{n}$ at higher
energies. These drawings convince us that the effect of the level density
difference: the SP to that of the $A-1$ nucleus will be particularly important
for the channel with the emission of one or two neutrons, where the excitation
energy of the system is not too high. For example, for the Og, we expect the
maximum of excitation function for the $1n$–channel ($2n$–channel) at
$U\approx 15$ MeV ($U\approx 25$ MeV). For these values, the system is far
from the asymptotic value where the density parameters converge to a value
close to about 1.1.

Figure 4: Panel (a) - the ratios of shell corrections at the SP $E_{sh}^{f}$
to those at the GS residual nuclei after neutron separation $E_{sh}^{n}$.
Panel (b) - the values of the GS shell corrections of residual nuclei after
neutron separation $E_{sh}^{n}$. The $Z=113$ isotopes are shown by red color
while $Z=118$ by blue one. Panels: (c) and (d) - the level-density parameter
ratios $a_{f}/a_{n}$ from Eq. (11) obtained at 15 MeV (turquoise), 30 MeV
(green), 50 MeV (orange), and 80 MeV (brown) for: (c) Nh and (d) Og isotopic
chain versus their isospin numbers.

At $U>30$ MeV the SP shell corrections are mostly damped and the energy
dependence of $a_{f}/a_{n}$ is ruled by the GS shell correction. This is seen
in Fig. 4 for Nh and Og isotopic chains, respectively. In these figures, the
isospin ($N-Z$) dependence of the ratio of shell corrections at the fission SP
to those at the GS are displayed in panel (a), the shell corrections at the GS
of nuclei after neutron emission are displayed in panel (b), and the ratios
$a_{f}/a_{n}$ are shown in panels (c) and (d) versus isospin value.

In Fig. 5 and 6, the energy dependencies of $a_{p}/a_{n}$ and
$a_{\alpha}/a_{n}$ are shown for the isotopic chains of nuclei with
$Z=115-120$. Since the GS have close rates of shell-correction damping, the
relative difference between the level-density parameters in particle emission
residues uniformly decreases with excitation energy. Therefore, in comparison
with $a_{f}/a_{n}$ the energy dependent values of $a_{p}/a_{n}$ and
$a_{\alpha}/a_{n}$ do not demonstrate maxima at low energies. For most of
isotopes considered the level-density parameter ratios increase with
excitation energy and reach the asymptotic values between 0.95 and 1.

Taking proton emission barrier as $B_{p}\pm 1.5$ MeV for 292Fl, we show the
sensitivity of $a_{p}/a_{n}$ to the decay threshold in Fig. 7 (a). As seen,
about $\mp 10\%$ deviation of $a_{p}/a_{n}$ from the one in the case of proton
emission barrier $B_{p}$ at 20 MeV decreases to zero with increasing
excitation energy. The same effect for $a_{\alpha}/a_{n}$ is displayed in Fig.
7 (b). Taking $\alpha$-particle emission barrier as $B_{\alpha}$, and
$B_{\alpha}\pm 1.5$ MeV, we find about $\mp 8\%$ difference of
$a_{\alpha}/a_{n}$ at $U=20$ MeV that decreases to zero with increasing
excitation energy. At enough high excitation energies the relative importance
of charged particles emission with respect to neutron emission is less
sensitive to the value of the Coulomb barrier. However, at low excitation
energy, especially in multi-step processes in which the energy is reduced by
emitted particles, the variation of the Coulomb barrier can play an important
role in the estimation of survival probability.

The energy dependence of the level-density parameter ratio is affected by the
difference of the decay thresholds and its isospin dependence is mainly
affected by the difference of shell-corrections. The relation between the
shell-correction ratios (panels a) and the level-density parameter ratios
$a_{\alpha}/a_{n}$ and $a_{p}/a_{n}$ (panels b,c, and d) calculated for the
isotopic chains of nuclei with $Z=115-120$ is shown in Figs. 8–11 versus
isospin. The isospin dependencies presented in Figs. 8–11 for $a_{p}/a_{n}$
and $a_{\alpha}/a_{n}$ remains almost unchanged at $U>30$ MeV. As seen, the
positions of maxima of $a_{p}/a_{n}$ and $a_{\alpha}/a_{n}$ coincide with
those of minima of $E^{p}_{sh}/E^{n}_{sh}$ and $E^{\alpha}_{sh}/E^{n}_{sh}$,
respectively. This gives an idea how to estimate the relative survival
probability under proton ($\alpha$-particle) emission in neighboring isotopes.

Figure 5: Excitation energy dependence of $a_{p}/a_{n}$ calculated with Eq.
(12) for 282-296Lv - panels: (a), (b), 295-300119 - panels: (c), (d), and
295-302120 - panels: (e), (f). The mass number $A$ of the given isotope is
indicated by the selected color. Figure 6: Excitation energy dependence of
$a_{\alpha}/a_{n}$ calculated with Eq. (13) for 282-295Mc - panels: (a), (b),
291-298Ts - panels: (c), (d), and 291-299Og - panels: (e), (f). The mass
number $A$ of the given isotope is indicated by the selected color. Figure 7:
Panel (a) - excitation energy dependencies of $a_{p}/a_{n}$ calculated with
Eq. (12), and panel (b) - $a_{\alpha}/a_{n}$ calculated with Eq. (13), for
292Fl taking the particle emission barriers as: $B_{p,\alpha}$ (red lines),
$B_{p,\alpha}+1.5$ MeV (blue lines), and $B_{p,\alpha}-1.5$ MeV (green lines).
Figure 8: For $Z=115$ (red lines), $Z=117$ (blue lines), and $Z=119$ (green
lines) isotopic chains, (a) the ratios of shell corrections corresponding to
the residual nuclei after proton emission to those after neutron emission, and
their level-density parameter ratios at: (b) 30 MeV, (c) 50 MeV, and (d) 80
MeV are shown versus their isospin. Figure 9: The same as Fig. 8, but for
$Z=116$ (red lines), $Z=118$ (blue lines), and $Z=120$ (green lines) isotopic
chains. Figure 10: For $Z=115$ (red lines), $Z=117$ (blue lines), and $Z=119$
(green lines) isotopic chains, (a) the ratios of shell corrections
corresponding to the residual nuclei after $\alpha$-particle emission to those
after neutron emission, and their level-density parameter ratios at: (b) 30
MeV, (c) 50 MeV, and (d) 80 MeV are shown versus their isospin. Figure 11: The
same as Fig. 10, but for $Z=116$ (red lines), $Z=118$ (blue lines), and
$Z=120$ (green lines) isotopic chains.

Assuming that the collective enhancement of the level densities of residual
nuclei are close to each other, we neglect these effects in the calculations
in Figs. 3-6. So, only the ratios of internal level densities were considered.
However, in the case of fission and $\alpha$-emission channels there are
additional collective states. In fission they are related to collective
rotation and vibration and usually taken into account through the enhancement
of level density [18]. This enhancement can effectively lead to the ratios
$a_{f}/a_{n}$ larger than those in Fig. 3. So, the phenomenological larger
$a_{f}/a_{n}$ effectively accounts for the collective enhancement of level
density. If the effect of collective enhancement of level density is well
known in fission, we suggest to take into account the collective effects on
$\alpha$-emission channel in a similar way.

The level density for $\alpha$-emission channel can be enhanced because of
additional collective states related to the cluster degrees of freedom [32,
33]. Assuming the excitations in mass asymmetry motion and relative vibrations
of $\alpha$-particle and daughter nucleus, the enhancement factor can be
estimated as in Ref. [22]

$\displaystyle K_{\alpha}(\beta)=\sum_{c}\exp\left(-\beta
U_{c}\right)\tau_{c},$ (14)

where, $U_{c}=\hbar\omega_{ma}n_{ma}+\hbar\omega_{b}(2n_{b}+|K|)$ and
$\tau_{c}=2|K|+1$ are the collective excitation energies and degeneracies,
respectively [32, 33]. Here, $\hbar\omega_{ma}$ is the frequency of vibrations
in mass asymmetry while $\hbar\omega_{b}$ describes the relative vibrations of
$\alpha$ particle and daughter nucleus, and $n_{b}$, $n_{ma}$, and $K$ are the
corresponding quantum numbers. Excitation energy dependencies of the ratios
$a_{\alpha}/a_{n}$ calculated for 282-295Mc, 291-298Ts, and 291-299Og nuclei
with account of $K_{\alpha}(\beta)$ in the level densities are shown in Fig.
12. Taking $\hbar\omega_{ma}=0.5$ MeV, and $\hbar\omega_{b}=0.3$ MeV we find
$K_{\alpha}(\beta)$ values varying between 10 and 50, but in the energy range
of our interest, one can take effectively into account these collective
effects with $K_{\alpha}\approx$20\. This leads not only to an increase in the
absolute value of $a_{\alpha}/a_{n}$, but also to a functional change in the
dependence of $a_{\alpha}/a_{n}$ on $U$. For example, the maximum of the ratio
$a_{\alpha}/a_{n}$ occurs in the region of energies of 20–30 MeV. Note that
the ratios $a_{\alpha}/a_{n}$ reach asymptotic values larger than unity.

Figure 12: Excitation energy dependencies of ratios $a_{\alpha}/a_{n}$
calculated for 282-295Mc nuclei - panels: (a), (b), 291-298Ts - panels: (c),
(d), and 291-299Og - panels: (e), (f), with account of $K_{\alpha}(\beta)$
from Eq. (14) in the level densities of $\alpha$-channel final states. The
mass number $A$ of the given isotope is indicated by the selected color.

##  IV Summary

The intrinsic level densities of SHN with $Z=112-120$ at the ground state and
at the saddle point are calculated within the thermodynamic superfluid
formalism. The single-particle energies, shell corrections and nuclear masses
used in these calculations were obtained within the multidimensional
microscopic-macroscopic model [19]. The energy dependent level-density
parameters of the considered nuclei were obtained by fitting the calculated
intrinsic level densities by the Fermi-gas expression. Then, the energy
dependent ratio of level-density parameters corresponding to the nuclei at the
fission saddle point $a_{f}$, and proton $a_{p}$ and $\alpha$-particle
$a_{\alpha}$ emission residues at their ground state to the ones obtained for
the daughter nuclei after neutron emission $a_{n}$ were calculated. Generally,
the level-density parameter ratios increase with excitation energy and reach
an asymptotic value less than 1.1 for $a_{f}/a_{n}$, and less than unity for
$a_{p,\alpha}/a_{n}$. The account of collective effects due to cluster degrees
of freedom in the level densities of $\alpha$–emission residue enhances the
ratio $a_{\alpha}/a_{n}$ to the values larger than unity. So, the values of
$a_{\alpha}/a_{n}$ and $a_{f}/a_{n}$ larger than those obtained
microscopically effectively account the collective enhancement of level
density.

Our analysis shows important effects of decay thresholds and shell correction
on the energy dependence of level-density parameter ratios before they reach
their asymptotic. Because of large difference in the shell corrections at the
saddle point and at the ground state as well as different rates of their
damping with excitation energy, the ratios $a_{f}/a_{n}$ have a peak at energy
lower than 30 MeV. The saddle-point shell corrections are mostly damped at
higher energies and the isospin dependence of $a_{f}/a_{n}$ is mainly related
to the ground-state shell corrections of residual nuclei after neutron
emission. Channels with the emission of one or two neutrons seem to be
particularly sensitive to the change of $a_{f}/a_{n}$, where the available
excitation energy does not have to be large and where the system is far from
the asymptotic value. As these channels ($1n$, $2n$) are characteristic of the
cold synthesis scenario, it is a strong hint that there also, and perhaps
above all, this difference ($a_{f}$ to $a_{n}$) will be very important.

Because of close values of shell corrections and their damping rates at the
ground states, the ratios $a_{p}/a_{n}$ and $a_{\alpha}/a_{n}$ monotonically
change with excitation energy.

The information and analysis performed in this article can be useful for the
calculation of energy dependent ratios of the widths for the considered decay
modes and further applied to evaluate corresponding cross sections and
probability of survival against fission and charged particle emissions.

## Acknowledgements

T.M.S, G.G.A., and N.V.A. were supported by the Ministry of Science and Higher
Education of the Russian Federation (Contract No. 075-10-2020-117). M.K. was
co-financed by the National Science Centre under Contract No.
UMO-2013/08/M/ST2/00257 (LEA COPIGAL). The Polish-JINR cooperation program is
acknowledged.

## References

  * [1] Yu.Ts. Oganessian, J. Phys. G 34, R165 (2007).
  * [2] Yu.Ts. Oganessian et al., Phys. Rev. Lett. 104, 142502 (2010); Phys. Rev. C 87, 014302 (2013); 87, 034605 (2013); 87, 054621 (2013); V.K. Utyonkov et al., Phys. Rev. C 92, 034609 (2015); 97, 014320 (2018).
  * [3] Yu.Ts. Oganessian and V.K. Utyonkov, Nucl. Phys. A 944, 62 (2015); Rep. Prog. Phys. 78, 036301 (2015).
  * [4] S. Hofmann et al., Eur. Phys. J. A 32, 251 (2007); 48, 62 (2012); 52, 180 (2016).
  * [5] R. Eichler et al., Nature 447, 72 (2007).
  * [6] L. Stavsetra et al., Phys. Rev. Lett. 103, 132502 (2009).
  * [7] Ch. Düllmann et al., Phys. Rev. Lett. 104, 252701 (2010); J.M. Gates et al., Phys. Rev. C 83, 054618 (2011). J.M. Khuyagbaatar et al., Phys. Rev. Lett. 112, 172501 (2014).
  * [8] S. Hilaire, and S. Goriely, Nucl. Phys. A 779 63 (2006).
  * [9] S. Hilaire, J. P. Delaroche, and M. Girod, Eur. Phys. J. A 12 169 (2001).
  * [10] Y. Alhassid, G. F. Bertsch, S. Liu, and H. Nakada, Phys. Rev. Lett. 84 4313 (2000).
  * [11] Y. Alhassid, S. Liu, and H. Nakada, Phys. Rev. Lett. 99 162504 (2007).
  * [12] S. Goriely, S. Hilaire, and A. J. Koning, Phys. Rev. C 78, 064307 (2008).
  * [13] D. E. Ward, B. G. Carlsson, T. Døssing, P. Möller, J. Randrup, and S. Åberg, Phys. Rev. C 95, 024618 (2017).
  * [14] Y. Alhassid, M. Bonett-Matiz, S. Liu, and H. Nakada, Phys. Rev. C 92, 024307 (2015).
  * [15] H. Bethe, Rev. Mod. Phys. 9, 69 (1937).
  * [16] Yu. V. Sokolov, Level densities of atomic nuclei (Energoizdat, Moscow, 1990).
  * [17] A. Rahmatinejad, A. N. Bezbakh, T. M. Shneidman, G. Adamian, N. V. Antonenko, P. Jachimowicz, and M. Kowal, Phys. Rev. C 103, 034309, (2021).
  * [18] A. B. Ignatyuk, G. N. Smirenkin, A. S. Tishin, Yad. Fiz.21, 485 (1975).
  * [19] P. Jachimowicz, M. Kowal, J. Skalski, Properties of heaviest nuclei with $98\leq$ Z $\leq 126$ and $134\leq$ N $\leq 192$, Atomic Data and Nuclear Data Tables 138 (2021) 101393.  doi:https://doi.org/10.1016/j.adt.2020.101393.
  * [20] P. Decowski, W. Grochulski, A. Marcinkowski, K. Siwek, Z. Wilhelmi, Nucl. Phys. A 110, 129 (1968).
  * [21] G.D. Adeev and P.A. Cherdantsev, Yad. Fiz. 21, 491 (1975).
  * [22] A. Rahmatinejad, T. M. Shneidman, N. V. Antonenko, A. N. Bezbakh, G. G. Adamian, and L. A. Malov, Phys. Rev. C 101, 054315, (2020).
  * [23] A. N. Bezbakh, T. M. Shneidman, G. G. Adamian, N. V. Antonenko, Eur. Phys. J. A 50, 97 (2014).
  * [24] A. N. Bezbakh, T. M. Shneidman, G. G. Adamian, N. V. Antonenko, and S. G. Zhou, Eur. Phys. J. A 52, 353 (2016).
  * [25] S. Ćwiok, J. Dudek, W. Nazarewicz, J. Skalski and T. Werner, Comput. Phys. Commun. 46, 379 (1987).
  * [26] H. J. Krappe, J. R. Nix and A. J. Sierk, Phys. Rev. C 20, 992 (1979).
  * [27] I. Muntian, Z. Patyk and A. Sobiczewski, Acta Phys. Pol. B 32, 691 (2001).
  * [28] M. Kowal, P. Jachimowicz, and A. Sobiczewski, Phys. Rev. C 82, 014303 (2010).
  * [29] P. Jachimowicz, M. Kowal, and J. Skalski, Phys. Rev. C 95, 034329 (2017).
  * [30] P. Jachimowicz, M. Kowal, and J. Skalski, Phys. Rev. C 89 , 024304 (2014).
  * [31] Juhee Hong, G. G. Adamian, and N. V. Antonenko, Phys. Rev. C 94, 044606, (2016).
  * [32] T. M. Shneidman, G. G. Adamian, N. V. Antonenko, R. V. Jolos, and Shan-Gui Zhou, Phys. Rev. C 92, 034302 (2015).
  * [33] T. M. Shneidman, G. G. Adamian, N. V. Antonenko, R. V. Jolos, and W. Scheid, Phys. Rev. C 67, 014313 (2003).