#  BERTer: The Efficient One
Stanford CS224N Default Project

Pradyumna Saligram
Department of Computer Science
Stanford University
<EMAIL_ADDRESS>
Andrew Lanpouthakoun
Department of Computer Science
Stanford University
<EMAIL_ADDRESS>

###### Abstract

We explore advanced fine-tuning techniques to boost BERT’s performance in
sentiment analysis, paraphrase detection, and semantic textual similarity. Our
approach leverages SMART regularization to combat overfitting, improves
hyperparameter choices, employs a cross-embedding Siamese architecture for
improved sentence embeddings, and introduces innovative early exiting methods.
Our fine-tuning findings currently reveal substantial improvements in model
efficiency and effectiveness when combining multiple fine-tuning
architectures, achieving a state-of-the-art performance score of on the test
set, surpassing current benchmarks and highlighting BERT’s adaptability in
multifaceted linguistic tasks.

##  1 Introduction

With the advent of transformers in 2017 (Vaswani et al., 2017) the field of
Natural Language Processing has improved immensely. Progressions such as Large
Language Models, particularly ChatGPT, have caused the world to see the full
capability of Machine Learning in changing lives. This ability by large
language models is largely explained by progressions made in various
downstream tasks and research in Multitask Learning(MTL). This research method
allowed models to learn various different tasks at once. The joining of
knowledge of multiple tasks pushed models to learn more complex and difficult
tasks by using this information at once. However, the efficacy and efficiency
of models doing various divergent tasks varies greatly. The goal of this
project is to increase the capability of BERT model (Devlin et al., 2018) on
three tasks in conjunction.

This research focuses on improving three tasks: sentence sentiment
classification, paraphrase detection, and sentence semantic similarity. Our
approach first focuses on improvement of sentiment classification. After
training this model, we choose to train on large datasets to learn paraphrase
detection and sentence semantic similarity in conjunction with further
training on sentiment classification. Training for these three tasks is done
simultaneously, to produce a truly multi-skilled model. This is done using
various different methods. The goal of this project is not to simply train our
model to accomplish these tasks; rather, the goal of this project is to boost
accuracy and efficiency. Therefore, we implement SMART a regularization method
(Jiang et al., 2019) in hopes of reducing overfitting, we implement Cosine
Embedding Loss, and early exiting to improve computational Efficiency.

Our findings suggest that SMART regularization yields great initial
improvements for Paraphrase detection, but does not continue as training
continues and does not perform as well in Sentiment Classification and Textual
Similarity Correlation. Therefore, we mostly paired SMART regularization with
Paraphrase Detection

Because SMART showed great initial improvements, we found that it pairs very
well with our other main fine-tuning technique of early exiting. (Xin et al.,
2020) Xin et al. (2021)

##  2 Related Work

Researchers aim to improve multi-task models in the hopes of leveraging
transfer learning. This is fairly intuitive, when a model performs well on one
task, it seems fairly simple for it to perform well on a similar task.
However, the results of a transformer model in similar tasks has surprisingly
poor results; therefore, we must work to have our model perform well on a
multitude tasks. Our work aims to train the Bidirectional Encoder
Representations from Transformers (BERT) Devlin et al. (2018) on these
multiple tasks, we use ideas from other papers to fine tune a model to do so.

(Jiang et al., 2019) shows that regularization tools such as perturbations can
combat overfitting by adding additional noise. This is similar to the dropout
layer (Srivastava et al., 2014) that is already within the original BERT
model. However, SMART is a much more advanced regularization technique that is
adaptive, generalizes to multiple tasks, smooths the distribution, and uses
gradients. We use SMART regularization over other regularization techniques
because of its computational efficiency.

We utilize early exiting techniques in the hopes of maximizing our
computational efficiency in unison with accuracy. BERxiT (Xin et al., 2021)
introduces a confidence-based mechanism to exit inference early if the model
is sufficiently confident in its prediction, offering the perfect mechanism
for our goal. DeeBERT (Xin et al., 2020) further extends this by incorporating
a dynamic inference mechanism that adjusts the exit point based on task
complexity and input characteristics.

We worked with a few different model architectures for tasks with multiple
sentences. Namely, we looked at concatenation prior to or after embedding.
(Schroff et al., 2015) speaks about a Siamese Architecture that that passes
each sentence into parallel models and then compares Cosine Similarity between
the two embeddings. This was also the method utilized within the very
successful SBERT paper for their Textual Similarity analysis. (Jiang et al.,
2019). Additionally, we look towards the original BERT paper (Devlin et al.,
2018) which concatenates sentences beforehand and uses a separation token
prior to embedding. Both of these have proven results and we experiment with
both during training.

##  3 Approach

We developed a comprehensive minBERT Transformer model, comprising 12
transformer layers where each layer includes a self-attention mechanism. The
transformation within each BERT layer, denoted as $BL(h)$, is formulated as:

$BL(h)=LN(h+FFN(LN(h+MH(h))))$

where $LN$ denotes layer normalization, $SA$ represents the self-attention
function, $FFN$ stands for the feed-forward network, while $MH$ is the multi-
head attention component.

###  3.1 Baseline Multitask Classifier

Our baseline multitask BERT model leverages the pre-trained weights of the
base minBERT for feature extraction across all tasks. The model architecture
is extended with three task-specific heads, each comprising a linear layer,
tailored to each respective task output requirement:

  * •

Sentiment Analysis (SST): Produces a 5-class output using a linear layer,
applying cross-entropy loss to quantify the difference between predicted
probability distributions and one-hot encoded target vectors.

  * •

Paraphrase Detection (Para): Initially used MSELoss but later optimized using
cosine embedding loss for better performance.

  * •

Semantic Textual Similarity (STS): Produces a single logit interpreted as a
continuous similarity score, using MSELoss against the labels.

The BERT model configuration includes a hidden size of 768, divided across 12
attention heads, resulting in an attention head size of 64 while dropout with
a probability of 0.3 is also applied.

Training is conducted sequentially within an epoch, each consisting of a loop
that fully trains each task. Initially, the base BERT model parameters are
frozen, establishing a lower baseline for accuracy after which single-task
fine-tuning is performed for each task, updating all model parameters to
achieve higher accuracy. The fine-tuning strategy is applied to both the SST
and CFIMDB datasets, creating separate models for each task. Finally, a fully
fine-tuned multi-task model is trained, updating 100% of the pre-trained
model’s parameters to generate a comprehensive model for all downstream tasks.

This approach allows us to establish both lower and higher bounds of accuracy,
demonstrating the efficacy of parameter-efficient fine-tuning. The model’s
"pretrain" mode freezes BERT parameters, while the "fine-tune" mode updates
all parameters based on task-specific data. For this baseline model, a batch
size of 8 is used for all tasks, and training is conducted over 10 epochs.

###  3.2 SMART Regularization and Alternative Optimizers

To mitigate overfitting, particularly notable in the SST-5 dataset, we
incorporated the SMART regularization technique. SMART consists of two main
components: Smoothness-inducing Adversarial Regularization and Bregman
Proximal Point Optimization, with our implementation primarily focusing on the
former. This method modifies the fine-tuning optimization process to enhance
model smoothness and reduce overfitting, leading to better generalization to
the target domain. Specifically, the optimization objective is reformulated to
integrate a smoothness-inducing adversarial regularizer along with the
standard loss function, defined as:

$\min_{\theta}F(\theta)=L(\theta)+\lambda_{s}R_{s}(\theta),$

where $L(\theta)$ represents the loss function for our downstream task,
$\lambda_{s}$ is a tuning parameter, and $R_{s}(\theta)$ is the smoothness-
inducing adversarial regularizer, defined as:

$R_{s}(\theta)=\frac{1}{n}\sum_{i=1}^{n}\max_{\|\mathbf{x}_{i}^{\prime}-\mathbf{x}_{i}\|_{p}\leq\epsilon}l_{s}(f(\mathbf{x}_{i}^{\prime};\theta),f(\mathbf{x}_{i},\theta)).$

For classification tasks, $l_{s}$ is selected as the symmetrized KL-divergence
loss, and for regression tasks, it is chosen as the mean-squared-error loss.

Initially, we implemented two methods aimed at pruning for computational
efficiency and preventing overfitting. We applied SMART regularization across
all three tasks but observed a positive accuracy change only for paraphrase
detection. This outcome was unexpected given the substantial size of the
CFIMDB dataset. Nevertheless, we continued to employ SMART exclusively for
paraphrase detection.

For optimization, we explored RMSprop and SGD as alternatives to AdamW. The
RMSprop algorithm accelerates the optimization process by maintaining the
moving average of squared gradients for each weight and dividing the gradient
by the square root of the mean square mathematically expressed as:

$v(w,t):=\gamma v(w,t-1)+(1-\gamma)(\nabla Q_{i}(w))^{2}$
$w:=w-\frac{\eta}{\sqrt{v(w,t)}}\nabla Q_{i}(w)$

where $\gamma$ is the forgetting factor. Stochastic Gradient Descent updates
the weights using the formula:

$w:=w-\eta\nabla Q_{i}(w)$

###  3.3 Early Exiting Mechanisms and Novel Fine-Tuning Strategy

To reduce extensive training times, which often extended to approximately 20
hours, we incorporated an Early-Exiting Architecture, reducing training time
to around 18 hours on an NVIDIA T4 GPU. Our early-exiting model is based on
established techniques, introducing innovative fine-tuning strategies.

The backbone model consists of a multi-layer pre-trained BERT architecture
with classifiers attached to each layer, enabling accelerated inference
through early exiting. The hidden state of the [CLS] token at the $i$-th
layer, denoted as $h_{i}$, is defined by:

$h_{i}=f_{i}(x;\theta_{1},\ldots,\theta_{i}),$

where $x$ is the input sequence, $\theta_{i}$ represents the parameters of the
$i$-th transformer layer, and $f_{i}$ maps the input to the $i$-th layer’s
hidden state. Each layer $i$ employs a classifier with a loss function:

$L_{i}(x,y)=H(y,g_{i}(h_{i};w_{i})),$

where $x$ and $y$ are the input and label, $g_{i}$ is the classifier at the
$i$-th layer, $w_{i}$ are the classifier’s parameters, and $H$ is the task-
specific loss function. To enable early exiting, we implemented two methods:
Confidence Threshold and Learning to Exit (LTE). Confidence Threshold exits
early if the prediction confidence at an intermediate layer exceeds a set
threshold, measured by the maximum probability of the output prediction for
classification tasks. For regression tasks or when confidence distribution is
unavailable, we used the LTE method. The LTE module, a one-layer fully
connected network, estimates the certainty level $u_{i}$ at the $i$-th layer:

$u_{i}=\sigma(c^{\top}h_{i}+b),$

where $\sigma$ is the sigmoid function, $c$ is the weight vector, and $b$ is
the bias term. The LTE module’s loss function is the mean squared error
between $u_{i}$ and the ground truth certainty level $\tilde{u}_{i}$:

$J_{i}=\|u_{i}-\tilde{u}_{i}\|^{2}_{2}.$

The ground truth certainty level for classification is whether the classifier
makes the correct prediction:

$\tilde{u}_{i}=1[\arg\max_{j}g_{i}^{(j)}(h_{i};w_{i})=y],$

and for regression, it is negatively related to the prediction’s absolute
error:

$\tilde{u}_{i}=1-\tanh(|g_{i}(h_{i};w_{i})-y|).$

We trained the LTE module jointly with the rest of the model by substituting
$L_{i}$ with $L_{i}+J_{i}$ in the loss function.

To improve the balance between immediate inference and gradual feature
extraction, we propose a novel fine-tuning strategy named Sequential Layer
Focus (SLF). This strategy combines elements of Joint and Two-stage
strategies, ensuring optimal training of early and late layers sequentially.

  * •

Stage 1: Sequential Focus \- Fine-tune each layer individually, updating only
the layer-specific classifier and the backbone model parameters corresponding
to that layer. This stage ensures that each layer independently learns to
provide effective intermediate representations.

  * •

Stage 2: Global Refinement \- Jointly fine-tune all layers and classifiers,
optimizing the entire network to harmonize the contributions from all layers
for the final output.

This approach ensures that each layer is first trained to optimal performance
in isolation before jointly refining the entire network. The Sequential Layer
Focus (SLF) strategy balances the needs of intermediate and final layers,
potentially leading to better overall model performance and efficiency.

###  3.4 Order Alteration of Word Embeddings and Concatenation

Traditionally, BERT concatenates sequences before embedding them. In our
extended approach, we reverse this order: sequences are first passed through
the embedding layer and then concatenated. This modification aims to preserve
more contextual information prior to concatenation, potentially enhancing
model performance.

In the typical BERT architecture, the input processing includes:

  * •

Input Representation: Summing token embeddings, segmentation embeddings, and
position embeddings.

  * •

Sequence Packing: Sequences are packed with special tokens [CLS] and [SEP].

Our modified approach can be mathematically represented as follows:

$E=\text{Embedding}(X)\quad\text{for each sequence }X$
$C=[E_{A};E_{B}]\quad\text{where }E_{A}\text{ and }E_{B}\text{ are embeddings
of sequences A and B respectively}$

We hypothesize that this approach retains more granular contextual information
before the sequences are merged, leading to improved downstream performance.
The embeddings from this modified process are then fed into the BERT layers
for further processing.

Novel Fine-Tuning Strategy

In addition to the embedding modification, we introduced a novel fine-tuning
strategy inspired by the Joint and Alternating strategies. Our strategy,
termed "Smart Alternating," dynamically switches between two objectives based
on performance metrics observed during training epochs:

  * •

Stage 1: Optimize the final layer’s classifier and the BERT backbone using the
loss function $L_{n}$.

  * •

Stage 2: Optimize intermediate classifiers jointly with the final classifier
using the sum of loss functions $\sum_{i=1}^{n}L_{i}$.

This strategy alternates between these stages depending on the validation
performance after each epoch, ensuring that both intermediate and final
classifiers are well-tuned without sacrificing the overall model quality. This
balance aims to improve both immediate inference at intermediate layers and
gradual feature extraction for the final classifier.

##  4 Experiments

###  4.1 Data

We undertake training for three downstream tasks: Sentiment Analysis,
Paraphrase Detection, and Text Similarity, employing diverse datasets for each
task.

####  4.1.1 Sentiment Analysis

For Sentiment Analysis, our BERT model is trained using two datasets: the
Stanford Sentiment Treebank (SST) and the CFIMDB Dataset. Both datasets focus
on movie reviews to gauge sentiment. Specifically, the SST dataset is
partitioned into 8,544 training examples, 1,101 development examples, and
2,210 testing examples. In contrast, the CFIMDB dataset consists of 1,701
training examples, 245 development examples, and 488 testing examples. While
SST labels phrases with five sentiment categories, CFIMDB labels sentences as
either negative or positive. Model evaluation is based on the proportion of
correctly labeled sentences.

####  4.1.2 Paraphrase Detection

To train for Paraphrase Detection, we utilize the Quora Dataset, which
includes 283,010 training examples, 40,429 development examples, and 80,859
testing examples. This dataset features pairs of sentences with binary labels
indicating whether they are paraphrases. Model performance is measured by the
proportion of correctly identified paraphrase pairs.

####  4.1.3 Sentence Textual Similarity Analysis

For Sentence Textual Similarity Analysis, we employ the SemEval STS Benchmark
Dataset, comprising 6,040 training examples, 863 development examples, and
1,725 testing examples. The dataset assigns labels ranging from 0 to 5 to each
sentence pair, indicating the degree of correlation. We use Pearson
Correlation Coefficients to evaluate performance, comparing the predicted and
true labels.

###  4.2 Evaluation Method

Supervised training is employed, with labeled datasets guiding model learning
and evaluation. The evaluation process involves making predictions and
comparing them to true labels. We assess model quality by accuracy on SST and
Paraphrasing tasks, and by Pearson Correlation on STS. The overall "evaluation
score" is calculated as the average of these metrics. Given our focus on
computational efficiency, we also measure training time and consider it in our
evaluations.

###  4.3 Experimental Details

Our experimental framework explores four dependent variables: Early-Exiting
Methods, SMART Regularization, Cosine Embeddings, and concatenation order.
Typically, training time averages 15 hours for a full model on an NVIDIA T4
GPU; however, using Cosine Embeddings significantly reduces this duration.
Throughout our experiments, we maintain consistent model configurations and
learning rates. In total, we evaluate 12 different training models, testing
various combinations of these methods to determine their effectiveness.

###  4.4 Results

Method | Type | SST | CFIMDB
---|---|---|---
Pretrain | last-linear-layer | 0.390 | 0.780
Finetune | full-model | 0.519 | 0.967
Table 1: Single task MinBERT classifier dev accuracy on Sentiment Analysis Model | Dev Score | SST | Paraphrase | STS | Training Time (hh:mm)
---|---|---|---|---|---
Baseline | 0.428 | 0.314 | 0.369 | 0.199 | 15:36
Full SMART | 0.433 | 0.253 | 0.485 | 0.123 | 15:57
Early Exiting | 0.476 | 0.313 | 0.768 | 0.184 | 13:36
Full SMART + Early Exiting | 0.482 | 0.253 | 0.632 | 0.123 | 13:47
Conc Before Embed + Early Exit | 0.595 | 0.520 | 0.625 | 0.792 | 13:49
Cos Loss + Early Exiting | 0.585 | 0.501 | 0.368 | 0.771 | 04:20
SLF | 0.592 | 0.505 | 0.640 | 0.721 | 12:53
SLF + Early Exiting | 0.650 | 0.544 | 0.658 | 0.740 | 11:47
SLF + Conc Before Embed | 0.671 | 0.523 | 0.651 | 0.801 | 12:14
SLF + Smart Alt | 0.676 | 0.545 | 0.614 | 0.790 | 13:54
SLF + Smart Alt + Early Exit | 0.683 | 0.510 | 0.632 | 0.818 | 10:41
SLF + Smart Alt + Early Exit + Cos Loss | 0.679 | 0.510 | 0.604 | 0.818 | 05:43
Table 2: Model Performance Comparison

$1$$2$$3$$4$$5$$6$$7$$8$$9$$10$$0.45$$0.5$$0.55$$0.6$$0.65$$0.7$$0.75$$0.8$$0.85$$0.9$$0.95$$1$EpochAccuracySST
Accuracy over epochs Figure 1: SST Accuracy over epochs

$10^{-7}$$10^{-6}$$10^{-5}$$10^{-4}$$10^{-3}$$0.4$$0.5$$0.6$$0.7$$0.8$$0.9$$1$Learning
RateAccuracySST accuracy by Learning Rates Figure 2: SST accuracy by Learning
Rates

$8$$10$$12$$14$$16$$18$$20$$22$$24$$0.5$$0.6$$0.7$$0.8$$0.9$$1$Batch
SizeAccuracyDev Score by Batch Size Figure 3: Dev Score by Batch Size

$0.18$$0.2$$0.22$$0.24$$0.26$$0.28$$0.3$$0.32$$0.34$$0.36$$0.38$$0.4$$0.42$$0.2$$0.3$$0.4$$0.5$$0.6$$0.7$$0.8$$0.9$$1$Hidden
Dropout ProbabilityAccuracyDev Score by Hidden Dropout Probability Figure 4:
Dev Score by Dropout Probability

Our results align with our expectations in terms of the relationships between
different model variations. The combination of SMART and SLF techniques
yielded the best overall performance across our training sets. Specifically,
the SLF + Smart Alternating + Early Exiting model achieved the highest overall
dev score of 0.683, with individual task accuracies of 0.510 for SST, 0.632
for Paraphrase Detection, and 0.818 for STS. These results, though not at the
top of the leaderboard in absolute terms, demonstrate a compelling balance
between accuracy and computational efficiency.

Notably, our early exiting methods, while slightly less accurate in isolation,
significantly improved computational efficiency, reducing training times by up
to 5%. For instance, the cosine loss model with early exiting reduced training
time to just 4 hours and 20 minutes, illustrating the trade-off between
accuracy and efficiency.

Overall, our findings underscore the potential of parameter-efficient fine-
tuning methods to balance accuracy and computational cost effectively. While
some accuracy trade-offs were observed, the substantial reduction in training
times offers a compelling advantage for practical applications.

##  5 Analysis

###  5.1 Overall Performance of Extensions

Our results align with our expectations in terms of the relationships between
different model variations. The combination of SMART and SLF techniques
yielded the best overall performance across our training sets. Specifically,
the SLF + Smart Alternating + Early Exiting model achieved the highest overall
dev score of 0.683, with individual task accuracies of 0.510 for SST, 0.632
for Paraphrase Detection, and 0.818 for STS. These results, though not at the
top of the leaderboard in absolute terms, demonstrate a compelling balance
between accuracy and computational efficiency.

Notably, our early exiting methods, while slightly less accurate in isolation,
significantly improved computational efficiency, reducing training times by up
to 5%. For instance, the cosine loss model with early exiting reduced training
time to just 4 hours and 20 minutes, illustrating the trade-off between
accuracy and efficiency.

Overall, our findings underscore the potential of parameter-efficient fine-
tuning methods to balance accuracy and computational cost effectively. While
some accuracy trade-offs were observed, the substantial reduction in training
times offers a compelling advantage for practical applications.

Paraphrase Detection scores seemed to stall at a common number in many of our
models. We found that there was a very clear answer; all predictions were made
to be 0. When inspecting the predictions file, it seems that our model is just
predicting that a sentence is not a paraphrase of another for all sentences.
This proves to have negative results and displays a sense of overfitting in a
sense.

This is incredibly interesting, as we expected our focus on SMART
regularization purely on paraphrase detection to negate any overfitting.
However, our model seems to have learned its training set slightly and only
predicts 0 because there are more pairs that are not paraphrases than ones
that are.

We found the change of concatenation order to be incredibly intuitive and
clear as to why it increased accuracy so significantly. We want our sentence
embeddings to utilize context within the sentence to create them; therefore,
it seems so intuitive for a larger sentence that is simply just two
concatenated together to have greater and more fruitful context than one that
is just two separate embeddings. This embedding has more fruitful information
and gives the model more to work with. This makes it much easier for the model
to train on.

Cosine Embedding Loss was surprisingly ineffective at fighting our error of
only predicting 0. Instead, our model only predicted 1 and ended up having the
complement accuracy.

The most interesting result we found was the effect of early exiting. We found
that early exiting had an insignificant effect on results; however, early
exiting had quite a powerful effect on computational efficiency, cutting down
our runtimes by 5%, a significant amount.

###  5.2 Hyperparameter Tuning and Analysis

Hyperparameter tuning played a critical role in optimizing our model
performance. By examining various configurations, we identified patterns that
led to an optimal setup. Figures 2 and 2 illustrate the impact of epochs and
learning rates on accuracy.

Our results showed that a learning rate of $1.3\times 10^{-5}$, a batch size
of 16, 5 epochs, and a dropout rate of 0.3 consistently provided the best
performance. Figure 4 indicates that a batch size of 16 yields the highest dev
score, balancing efficiency and accuracy. As seen in Figure 4, a 0.3 dropout
rate optimally regularizes the model, preventing overfitting while maintaining
performance.

These hyperparameter configurations were critical in achieving the improved
dev scores across various tasks, illustrating the importance of fine-tuning to
enhance model performance.

##  6 Conclusion

Our research highlights the importance of computational efficiency without
sacrificing significant accuracy metrics. By employing innovative fine-tuning
strategies like Early Exiting with BERxiT, Cosine Embeddings Loss, and
sequential fine-tuning techniques, we struck a commendable optimized fusion of
performance and efficiency.

Despite its theoretical promise, SMART regularization yielded mixed outcomes:
while it initially improved paraphrase detection accuracy, its performance
plateaued and underperformed in sentiment classification and textual
similarity tasks, suggesting the need for further selective application.

The Sequential Layer Focus (SLF) strategy, paired with Smart Alternating and
Early Exiting, achieved the highest overall development score of 0.683. Task-
specific accuracies included 0.510 for SST, 0.632 for Paraphrase Detection,
and 0.818 for STS. These results were obtained while significantly reducing
training times, demonstrating the efficiency of our approach. For instance,
models utilizing cosine loss with early exiting reduced training times to just
4 hours and 20 minutes.

Additionally, our novel fine-tuning strategies, particularly altering word
embedding order and Smart Alternating techniques, proved highly effective.
Adjusting the embedding order enhanced context utilization, thereby improving
model performance.

Future research should investigate different embedding strategies and novel
transformer architectures to enhance model generalization across various
linguistic contexts. Moreover, focusing on model explainability and
interpretability will be crucial for identifying and mitigating potential
biases in predictions, ensuring fairness and robustness.

##  7 Ethics Statement

There are several ethical and societal risks associated with this project that
are universal across various NLP tasks, including transformers, LSTMs, and
large language models.

The foremost issue is data bias. Many NLP systems rely on broadly sourced data
that may not accurately represent the ideologies, cultures, and moral values
of diverse populations. This can result in models failing to account for
textual similarities among slang or culturally specific language, leading to
poor performance in tasks like paraphrase detection when comparing texts from
different socioeconomic backgrounds. Moreover, the model may misinterpret
sentiment in culturally nuanced sentences. This skew towards more readily
available data can lead to the misrepresentation of underrepresented groups.
To address this, it is imperative to curate more inclusive datasets that
reflect the diversity of the user base.

Additionally, our model confronts the ethical challenge of computational power
consumption. The growing concern over global warming underscores the need for
sustainable computing practices. The exponential increase in computational
demands directly correlates with higher energy consumption, often sourced from
nonrenewable and polluting resources. By focusing on creating efficient models
that balance accuracy with reduced computational demands, we aim to contribute
to the fight against climate change. Our approach to enhancing efficiency
without sacrificing performance highlights the necessity of integrating
environmental considerations into the development of advanced computational
systems.

The heightened need for interpretability in NLP models is paramount in
addressing biases and ensuring fairness. By enhancing the transparency of our
models, we can better understand and mitigate potential biases in their
predictions. This involves developing methods that allow users to comprehend
and trust the decision-making processes of these models. Interpretable models
can provide insights into the underlying reasons for their predictions,
thereby enabling the identification and correction of biases. Ensuring
fairness and robustness in NLP systems is critical for their ethical
deployment across various applications. could you write the related works for
early exiting

## References

  * Devlin et al. (2018) Jacob Devlin, Ming-Wei Chang, Kenton Lee, and Kristina Toutanova. 2018.  BERT: pre-training of deep bidirectional transformers for language understanding.  _CoRR_ , abs/1810.04805.
  * Jiang et al. (2019) Haoming Jiang, Pengcheng He, Weizhu Chen, Xiaodong Liu, Jianfeng Gao, and Tuo Zhao. 2019.  SMART: robust and efficient fine-tuning for pre-trained natural language models through principled regularized optimization.  _CoRR_ , abs/1911.03437.
  * Schroff et al. (2015) Florian Schroff, Dmitry Kalenichenko, and James Philbin. 2015.  Facenet: A unified embedding for face recognition and clustering.  _CoRR_ , abs/1503.03832.
  * Srivastava et al. (2014) Nitish Srivastava, Geoffrey Hinton, Alex Krizhevsky, Ilya Sutskever, and Ruslan Salakhutdinov. 2014.  Dropout: A simple way to prevent neural networks from overfitting.  _Journal of Machine Learning Research_ , 15(56):1929–1958.
  * Vaswani et al. (2017) Ashish Vaswani, Noam Shazeer, Niki Parmar, Jakob Uszkoreit, Llion Jones, Aidan N. Gomez, Lukasz Kaiser, and Illia Polosukhin. 2017.  Attention is all you need.  _CoRR_ , abs/1706.03762.
  * Xin et al. (2020) Ji Xin, Raphael Tang, Jaejun Lee, Yaoliang Yu, and Jimmy Lin. 2020.  DeeBERT: Dynamic early exiting for accelerating BERT inference.  In _Proceedings of the 58th Annual Meeting of the Association for Computational Linguistics_ , pages 2246–2251, Online. Association for Computational Linguistics.
  * Xin et al. (2021) Ji Xin, Raphael Tang, Yaoliang Yu, and Jimmy Lin. 2021.  BERxiT: Early exiting for BERT with better fine-tuning and extension to regression.  In _Proceedings of the 16th Conference of the European Chapter of the Association for Computational Linguistics: Main Volume_ , pages 91–104, Online. Association for Computational Linguistics.


# Hydrodynamic Decay of Decorated Quantum Vortex Rings

L. Moriconi  Instituto de Física, Universidade Federal do Rio de Janeiro,
C.P. 68528, 21945-970, Rio de Janeiro, RJ, Brazil

###### Abstract

The decay of quantum vortex rings in counterflow regimes, visualized in Helium
II with the help of solid hydrogen particles trapped to their cores, has been
a puzzling issue within the usual description of superfluid vortex dynamics,
grounded on the hypothesis that a vortex filament is, effectively, an extended
massless object subject to a canceling superposition of Magnus and mutual
friction forces. We discuss, from a general energy-budget point of view, a
phenomenological solution of this problem, which relies on viscous and quantum
dissipation mechanisms, the later associated to the backreaction of vortex
singular structures on the surrounding two-component fluid mixture.

##  I Introduction

Superfluid (quantum) vortices, conjectured by Onsager in 1949 onsager  and
further discussed by Feynman a few years later feynman , are well-defined flow
structures, which have atomic-sized singular cores, and dynamical evolution
ruled, in the zero temperature limit, by the classical Euler equation donnelly
. Due to macroscopic quantum coherence, the circulation around superfluid
vortices is quantized in integer multiples of $h/m$ ($h$ is the Planck
constant and $m$ is the mass of the superfluid particle constituents). These
topologically stable excitations are assumed to play fundamental roles in the
description of both equilibrium and out-of-equilibrium superfluid phenomena.
Actually, while the suppression of superfluid phase coherence near the
critical temperature is related to the proliferation of quantum vortex loops
induced by thermal fluctuations williams ; shenoy , turbulence below the
lambda point has been generally depicted as a complex tangle of superfluid
vortices halp_tsu ; cooper_etal . A proper modeling of quantum vortex
evolution is thus believed to be the key for a phenomenological understanding
of quantum turbulent regimes.

Even though objectively defined from the topological properties of the
condensate wavefunction, and detected long ago by means of ingenious
mechanical experiments vinen58 ; whit_zimm , the direct observation of
superfluid vortices has been a challenging task. The breakthrough experiments
of Yarmchuk, Gordon and Packard YGP  provided clear visualizations of
stationary arrays of He-II quantum vortices in a rotating tank. However, their
observation technique, based on electric discharges produced from electron
bubbles trapped in the vortex cores williams_packard , where pressure is
depleted, is not suitable for the investigation of evolving vortex filaments.

After a time gap of almost three decades, a turning point in the problem of
quantum vortex visualization was reached through the use of micro-sized solid
hydrogen or deuterium particles bewley_etal ; chago_sci ; guoa , which are
often trapped in quantum vortex cores. This approach has paved the way for an
ongoing wave of new experiments, as the visualization of vortices near the
superfluid transition bewley_etal , the study of quantum vortex reconnection
bewley_etal2 , the decay of quantum vortex rings bewley_sree  – our main focus
in this work – and the observation of Kelvin waves in vortex filaments
fonda_etal . Interesting open challenges are furthermore related to the amount
of dynamic information that one can obtain from the recorded trajectories of
tracking particles in superfluid turbulent regimes zhang_sci ; serg_bar_kivo ;
kivo_wilkin ; duda ; mastracci_guo ; mori ; guo_etal .

The visualization of particle loaded vortices yields, furthermore, an
interesting stage for the validation (or not) of the vortex line structural
approach to quantum vortex dynamics pioneered by Schwarz schwarz , which
stands at a central place in the literature. A careful experiment showed, in
this context, that the application of the structural modeling framework to the
decay of a quantum vortex ring decorated by attached solid microscopic
particles is plagued with difficulties bewley_sree . In that study, an
estimate of the vortex ring decay time scale was attempted, under the
assumptions that

(i) the vortex ring core size should be modified by the presence of the
attached particles,

(ii) the vortex filament should experience an additional drag force due to the
presence of normal flow around the attached particles, and that

(iii) the existing background counterflow should not affect the vortex ring
evolution.

While assumption (i) is perfectly reasonable, (ii) and (iii) are ad hoc
hypotheses of difficult justification. Also, it turns out that even along
these lines the predicted time evolution of the vortex ring radius does not
accurately match observations.

A main point of criticism to the usual vortex line structural description is
that it completely neglects backreaction effects of quantum vortex filaments
on the surrounding normal component of the condensate. They are particularly
relevant in situations where vortex reconnection events are assumed to play a
significant dynamical role barenghi_etalbook ; villois_etal . One expects,
actually, normal flow to be always induced by quantum vortex filaments. As a
surprising illustration of this fact, it has been suggested that superfluid
vortex rings exhibit an interesting triple comoving flow structure composed of
two vortex rings of normal fluid coupled to the much thinner quantum vortex
ring kivotides_etal ; galan_etal . Our aim in this work is to show that
similar backreaction effects offer a phenomenologically consistent and
accurate solution of the vortex ring decay puzzle. Our discussion is based on
a general energy-budget perspective, which bypasses specific modeling details
and places emphasis on the role of normal viscous dissipation and
normal/superfluid mutual interaction effects.

This paper is organized as follows. In Sec. II, we briefly review the basic
ideas of the vortex line structural approach and point out modeling
difficulties related to the phenomenon of particle-decorated vortex ring
decay. In sec. III, we address a general phenomenological discussion of the
vortex ring evolution along energy budget lines, which takes into account the
coupling between the superfluid and the normal components of the flow. We
devise, in this way, a time evolution equation for the vortex ring radius
which leads to a fine agreement with experimental data. Finally, we summarize,
in Sec. IV, our results and indicate directions of further research.

##  II Structural Modeling Issues

The motion of a quantum vortex filament in a superfluid condensate which has
assigned velocity fields ${\bf{v_{n}}}({\bf{r}},t)$ and
${\bf{v_{s}}}({\bf{r}},t)$ for its normal and superfluid components,
respectively, is subject to Magnus and drag forces donnelly ; schwarz . Thus,
if a point $P$ of the vortex filament where the local unit tangent vector is
${\bm{\hat{\omega}}}_{\bf{s}}$ (oriented along the vorticity field) moves with
velocity ${\bf{v_{L}}}$, the Magnus and drag forces forces per unit length at
$P$ are, respectively,

$\displaystyle{\bf{F_{M}}}$ $\displaystyle=$
$\displaystyle\rho_{s}\kappa{\bm{\hat{\omega}}}_{\bf{s}}\times({\bf{v_{L}}}-{\bf{v_{s}}})\
,\ $ (1) $\displaystyle{\bf{F_{D}}}$ $\displaystyle=$ $\displaystyle
D_{L}{\bm{\hat{\omega}}}_{\bf{s}}\times({\bf{v_{n}}}-{\bf{v_{L}}})+D{\bm{\hat{\omega}}}_{\bf{s}}\times\left[{\bm{\hat{\omega}}}_{\bf{s}}\times({\bf{v_{L}}}-{\bf{v_{n}}})\right]\
,\ $ (2)

where $\rho_{s}$ is the superfluid density, $\kappa\simeq 10^{-7}$ m2/s is the
quantum of circulation, and $D_{L}$ and $D$ are phenomenological drag
coefficients that model the interaction between the normal and superfluid
components of the condensate. Assuming that a quantum vortex filament has
negligible inertia, we obtain

${\bf{F_{M}}}+{\bf{F_{D}}}=0\ ,\ $ (3)

which in view of Eqs. (1) and (2) gives, for the vortex filament velocity
schwarz ,

${\bf{v_{L}}}={\bf{v_{s}}}+\alpha{\bm{\hat{\omega}}}_{\bf{s}}\times({\bf{v_{n}}}-{\bf{v_{s}}})-\alpha^{\prime}{\bm{\hat{\omega}}}_{\bf{s}}\times\left[{\bm{\hat{\omega}}}_{\bf{s}}\times({\bf{v_{n}}}-{\bf{v_{s}}})\right]\
,\ $ (4)

where

$\alpha\equiv\frac{\rho_{s}\kappa
D}{D^{2}+D_{0}^{2}}\hbox{~{}~{}~{}and~{}~{}~{}}\alpha^{\prime}\equiv
1-\frac{\rho_{s}\kappa D_{0}}{D^{2}+D_{0}^{2}}$ (5)

are the so-called mutual friction coefficients, with

$D_{0}\equiv\rho_{s}\kappa-D_{L}\ .\ $ (6)

Eq. (3) is a fundamental postulate in the structural description of the vortex
line, to be regarded more as a working hypothesis, rather than a well-
established truth. The conjecture that vortex filaments are massless objects
has been in fact the subject of relevant questioning toikka-brand ; simula .

The decay of a decorated, fully-visualized, quantum vortex ring with initial
radius $R=400$ $\mu$m and lifetime of about 6 s, subject to an approximately
uniform background counterflow of velocity $\simeq 500$ $\mu$m/s, was
meticulously observed by Bewley and Sreenivasan bewley_sree . The Bewley-
Sreenivasan (BS) experiment was performed at the temperature $T=2.06$ K, where
the He-II mutual friction coefficients are $\alpha\simeq 0.37$ and
$\alpha^{\prime}\simeq 3\times 10^{-3}$ bar-donn .

Once Eqs. (4-6) are supposed to describe the evolution of vortex filaments, it
is natural to presume, as a first modeling attempt, that the decay of
decorated vortex rings could be ruled by similar equations, with appropriate
redefinitions of the drag coefficient parameters and a reasonable choice of
the counterflow velocity field (which may be difficult to measure in
practice). This is the point of view taken in Ref. bewley_sree , which we
momentarily adopt for the sake of critical analysis.

In the particular situation of the BS experiment, the decaying vortex ring was
not seen to rotate or to have its circular shape deformed during its
evolution. It follows, then, as a consequence of Eq. (4), that
${\bf{v_{ns}}}={\bf{v_{n}}}-{\bf{v_{s}}}$ should be a vector field normal to
the vortex ring plane. Denoting by ${\bf{\hat{n}}}$ a unit vector which is
parallel to the self-induced vortex ring translation velocity, we may write

${\bf{v_{ns}}}\equiv
v_{ns}{\bf{\hat{n}}}=(U_{ns}-v^{\circ}_{s}){\bf{\hat{n}}}\ ,\ $ (7)

where $U_{ns}$ is the approximately uniform background counterflow velocity
and $v^{\circ}_{s}>0$, with $v^{\circ}_{s}\ll|U_{ns}|$, is the self-induced
vortex ring velocity. Taking into account that $\alpha^{\prime}\ll 1$ and that
${\bf{v_{s}}}$ is also approximately uniform on the vortex ring filament, it
is not difficult to get, from Eq. (4), the evolution equation for the vortex
ring radius $R$,

$\frac{dR}{dt}=\alpha(U_{ns}-v^{\circ}_{s})\ .\ $ (8)

Quantum vortex rings have hollow cores, where the superfluid condensate
vanishes. For a core of radius $r_{c}\ll R$, the self-induced vortex ring
translation velocity is given by saffman

$v_{s}^{\circ}=\frac{\kappa}{4\pi
R}\left[\ln\left(\frac{8R}{r_{c}}\right)-\frac{1}{2}\right]\ .\ $ (9)

The vortex ring core radius $r_{c}$ depends on the temperature. For
temperatures close to $2$ K, as reported in the BS experiment, we estimate
$r_{c}\simeq 10$ $\mathring{\mathrm{A}}$ hall .

Before proceeding with the application of (8) and (9) to the problem of
decorated vortex ring decay, it is important to emphasize a couple of related
phenomenological points:

(i) Particles that are trapped to quantum vortices are constrained to
passively follow their host vortex filaments. In fact, bouyant/pressure forces
or even forces associated to the normal component of the condensate, like the
viscous drag, are negligible if compared to the strong restoring superfluid
pressure forces that act on the particles near vortex cores.

(ii) Vortex filaments can be populated by a largely variable number of
attached particles. We can distinguish two limiting cases in our analysis,
related to sparse or dense vortex ring decorations, as it is illustrated in
Fig. 1. Theses decoration regimes can be parametrized by the packing parameter

$p\equiv\frac{Nr_{p}}{\pi R}\ ,\ $ (10)

where $N$ is the number of attached particles and $r_{p}$ is their mean
radius.

We examine, in the following, what the vortex line structural description
would tell us about vortex ring decays for the $p\ll 1$ and $p\gg 1$ cases.

Figure 1: Sparse (a) and dense (b) particle decorations of quantum vortex
rings. In the packed case (b) (only a vortex ring arc is depicted), the
effective vortex core radius $r_{c}$ can be much larger than the vortex
filament core radius. Note that these pictures are merely illustrative, since
particles’ linear sizes are typically $10^{3}-10^{4}$ times larger than the
ones of vortex filament cores.

Sparsely Decorated Vortex Rings ($p\ll 1$)

When the vortex ring filament is sparsely decorated, the background and self-
induced superfluid flow velocity fields are only weakly perturbed by the
attached particles, so that we just have to combine Eqs. (8) and (9) to define
and numerically solve

$\frac{dR}{dt}=\alpha\left\\{U_{ns}-\frac{\kappa}{4\pi
R}\left[\ln\left(\frac{8R}{r_{c}}\right)-\frac{1}{2}\right]\right\\}\ .\ $
(11)

Results are shown in Fig. 2, for a number of counterflow velocities, together
with a comparison to experimental data. As it gets clear, this modeling
scenario is not satisfactory. Visual inspections suggest, as a matter of fact,
that the decaying vortex ring is densely decorated, the situation we examine
now.

Figure 2: Vortex ring radius as a function of time. Circles represent
experimental data taken from Ref. bewley_sree . Solid lines are the decaying
profiles predicted from Eq. (11) for a set of counterflow velocities $U_{ns}$:
in $\mu$m/s, from the left to the right in the picture, $U_{ns}=0$, $150$,
$200$, $230$, $250$. Figure 3: Vortex ring radius as a function of time.
Circles represent experimental data taken from Ref. bewley_sree . Solid lines
are the decaying profiles predicted from Eq. (14) for a set of counterflow
velocities $U_{ns}$: in $\mu$m/s, from the left to the right in the picture,
$U_{ns}=-250$, $-120$, $-50$, $0$, $30$.

Densely Decorated Vortex Rings ($p\gg 1$)

A densely packed group of particles provides some non-negligible volume
$V_{\circ}$ to the vortex ring core, which is conserved along its decay. In
this case, the vortex ring radius and its core radius are related through

$r_{c}^{2}=\frac{V_{\circ}}{2\pi^{2}R}\ .\ $ (12)

This geometrical constraint implies that

$\frac{R}{r_{c}}=cR^{\frac{3}{2}}\ ,\ $ (13)

where $c=\sqrt{2\pi^{2}/V_{\circ}}\simeq\pi\times
10^{-3}\mu$m${}^{-\frac{3}{2}}$, as roughly estimated from snapshots of the
decaying vortex ring bewley_sree .

It was hypothesized in Ref. bewley_sree  that the normal low Reynolds number
flow around the observed densely decorated vortex ring could modify the drag
coefficient parameter $D$ in (5), rescaling the mutual friction coefficient
$\alpha$ to the effective larger value $\tilde{\alpha}=1.3\alpha$. Replacing
in Eq. (11) $\alpha$ by $\tilde{\alpha}$ and $R/r_{c}$ by $cR^{3/2}$, as
prescribed in (13), we obtain

$\frac{dR}{dt}=\tilde{\alpha}\left\\{U_{ns}-\frac{\kappa}{4\pi
R}\left[\ln\left(8cR^{\frac{3}{2}}\right)-\frac{1}{2}\right]\right\\}\ .\ $
(14)

Solutions of (14) for a set of background counterflow velocities are shown in
Fig. 3. A reasonable approximation to the experimental data seems to be
achieved now up to the time instant where $R\simeq 150$ $\mu$m. The agreement,
however, is based on the assumption of vanishing background counterflow, a
condition of hard phenomenological support. We have also verified that
solutions with alternative values of $\tilde{\alpha}/\alpha$, within
acceptable ranges, do not improve results in a meaningful way.

The message we take from the above modeling attempts is that the usual
structural vortex line approach – including slight formulation variations – is
not enough per se to reproduce the evolution of densely decorated quantum
vortices. After all, this should not be a matter of great surprise. As it has
been pointed out in the literature, the extension of Landau’s two-fluid model
to cope with the existence of superfluid circulation, associated to correlated
vortex filament bundles donnelly  or even singular quantum vortices
kivotides_etal ; galan_etal , implies that energy and momentum should be
exchanged between the normal and superfluid components of the condensate,
underlying the existence of yet unsuspected phenomena.

Given the complex (and dynamic) boundary conditions involved in the
geometrical characterization of a densely packed vortex ring, direct numerical
simulations of the coupled normal and superfluid flows would be extremely
costly for this particular problem. We address in this work, instead, a
heuristic strategy to devise the general equation, in place of (11) or (14),
that should rule the radius evolution of densely decorated vortex rings. The
argument, carried out in the next section, is essentially based on the energy
balance equation derived from a two-fluid hydrodynamic model.

##  III Energy-Budget Approach

Let $\rho_{n}$ and $\mu_{n}$ be, respectively, the density and dynamic
viscosity of the normal component of the condensate. We take, as a model of
Helium II hydrodynamics in the isothermal approximation, the Hall-Vinen-
Bekarevich-Khalatnikov (HVBK) coupled set of equations for the
normal/superfluid mixture barenghi_etalbook ; galan_etal ,

$\displaystyle\rho_{n}\left(\partial_{t}{\bf{v_{n}}}+{\bf{v_{n}}}{\bm{\cdot\nabla}}{\bf{v_{n}}}\right)=-{\bm{\nabla}}P_{n}+\mu_{n}\nabla^{2}{\bf{v_{n}}}+{\bf{F_{ns}}}\
,\ $ (15)
$\displaystyle\rho_{s}\left(\partial_{t}{\bf{v_{s}}}+{\bf{v_{s}}}{\bm{\cdot\nabla}}{\bf{v_{s}}}\right)=-{\bm{\nabla}}P_{s}-{\bf{F_{ns}}}\
,\ $ (16)

supplemented by the double incompressibility condition,

${\bm{\nabla\cdot}}{\bf{v_{s}}}={\bm{\nabla\cdot}}{\bf{v_{n}}}=0\ .\ $ (17)

Above, $P_{n}$ and $P_{s}$ are the partial pressures associated to the Helium
II normal and superfluid components. In our specific problem, the force per
unit volume, ${\bf{F_{ns}}}$, that the superfluid component exerts on the
normal one, is assumed to have singular support on the vortex filament pieces
that hold the particles in the vortex ring (see see again Fig. 1). Notice that
the force density ${\bf{F_{sn}}}=-{\bf{F_{ns}}}$ can be readily computed from
the substitution of (4) into (2), using (5) and the fact that the singular
superfluid vorticity field ${\bm{\nabla}}\times{\bf{v_{s}}}$ can be
represented in a small neighborhood of the vortex filament around its position
${\bf{r_{v}}}$ as

${\bm{\omega}}_{\bf{s}}({\bf{r_{v}}}+{\bm{\xi}})=\kappa{\bm{\hat{\omega}}}_{\bf{s}}\delta^{2}({\bm{\xi}})\
,\ $ (18)

where ${\bm{\xi}}\cdot{\bm{\hat{\omega}}}_{\bf{s}}=0$. We obtain, thus,

$\displaystyle{\bf{F_{ns}}}=\alpha\rho_{s}{\bm{\hat{\omega}}}_{\bf{s}}\times\left[{\bm{\omega}}_{\bf{s}}\times\left({\bf{v_{n}}}-{\bf{v_{s}}}\right)\right]+\alpha^{\prime}\rho_{s}{\bm{\omega}}_{\bf{s}}\times\left({\bf{v_{n}}}-{\bf{v_{s}}}\right)\
.\ $ (19)

It is important to remark that Eq. (19) holds for the vortex filament segments
that bridge particles in the decorated vortex ring, as depicted in Fig. 1.
These various vortex bridges are labelled henceforth by a given set of lines
$\\{C_{i}\\}$. Furthermore, the vorticity field lines which are spread over
the relatively large particle surfaces are not supposed to provide relevant
contributions to the interaction force between the normal and superfluid flow
components.

Direct numerical solutions of Eqs. (15) and (16) with the constraints (17) and
the definition (19) are far from simple even for the case of bare vortex rings
kivotides_etal ; galan_etal . Considerable further complications are expected
to arise in the case of decorated vortex filaments, since one would have to
worry about the implementation of dynamic boundary conditions and
supplementary prescriptions related to vortex reconnection events.

It is interesting, therefore, to see how far we can proceed, having focus on
general properties of the dynamical equations (15) and (16) and resorting as
much as possible to a minimum number of phenomenological assumptions. To
start, we write, inspired by the Reynolds decomposition procedure pope , the
normal and superfluid velocity fields as

${\bf{v_{n}}}={\bf{U_{n}}}+{\bf{u_{n}}}\ ,\
{\bf{v_{s}}}={\bf{U_{s}}}+{\bf{u_{s}}}\ ,\ $ (20)

where

${\bm{\nabla}}\cdot{\bf{u_{s}}}={\bm{\nabla}}\cdot{\bf{u_{n}}}=0\ .\ $ (21)

The velocity fields ${\bf{U_{n}}}$ and ${\bf{U_{s}}}$, assumed to be time-
independent and uniform, represent the background counterflow in the absence
of quantum vortex filaments. The fields ${\bf{u_{n}}}$ and ${\bf{u_{s}}}$ are,
on their turn, the velocity perturbations associated to the dynamic evolution
of the decorated vortex ring. We, now,

(i) substitute (20) in Eqs. (15) and (16) and contract them with
${\bf{u_{n}}}$ and ${\bf{u_{s}}}$, respectively;

(ii) sum up the contracted equations and integrate the resulting scalar
equation over all the space, taking (17) into account.

In this way, we find, up to second order in ${\bf{u_{n}}}$ and ${\bf{u_{s}}}$,
the energy-budget equation,

$\displaystyle\frac{d}{dt}\left(E_{s}+E_{n}\right)=$ (22) $\displaystyle=$
$\displaystyle-\mu_{n}\int
d^{3}{\bf{r}}{\hbox{Tr}}\left[\left({\bm{\nabla}}\otimes{\bf{u_{n}}}\right)^{2}\right]-\alpha\rho_{s}\kappa\sum_{i}\int_{C_{i}}ds({\bf{u_{n}}}-{\bf{u_{s}}})_{\perp}^{2}+$
$\displaystyle-$
$\displaystyle\alpha\rho_{s}\kappa\sum_{i}\int_{C_{i}}ds({\bf{u_{n}}}-{\bf{u_{s}}})_{\perp}\cdot({\bf{U_{n}}}-{\bf{U_{s}}})_{\perp}+$
$\displaystyle+$
$\displaystyle\alpha^{\prime}\rho_{s}\kappa\sum_{i}\int_{C_{i}}ds({\bf{u_{n}}}-{\bf{u_{s}}})\cdot[{\bm{\hat{\omega}}}_{\bf{s}}\times({\bf{U_{n}}}-{\bf{U_{s}}})]\
,\ $

where the notation $\perp$ indicates the vector component which is normal to
${\bm{\hat{\omega}}}_{\bf{s}}$ and

$\displaystyle E_{s}=\frac{1}{2}\rho_{s}\int
d^{3}{\bf{r}}[{\bf{u_{s}}}({\bf{r}},t)]^{2}\ ,\ $ (23) $\displaystyle
E_{n}=\frac{1}{2}\rho_{n}\int d^{3}{\bf{r}}[{\bf{u_{n}}}({\bf{r}},t)]^{2}\ ,\
$ (24)

are the superfluid and normal kinetic energy contributions related to the
perturbation fields ${\bf{u_{s}}}$ and ${\bf{u_{n}}}$.

Assuming that a quantum vortex ring of time-dependent radius $R(t)$ is coupled
to vortex ring-like normal flow structures at similar length scales
kivotides_etal , it is not difficult to show, from the fluid dynamic Biot-
Savart law saffman , that at large distances ($r\gg R$) velocity perturbations
have the self-similar form

$\displaystyle{\bf{u_{s}}}({\bf{r}},t)=\frac{\kappa}{R(t)}{\bf{u}}^{\bm{(0)}}_{\bf{s}}({\bf{r}}/R(t))\
,\ $ (25)
$\displaystyle{\bf{u_{n}}}({\bf{r}},t)=\frac{\kappa}{R(t)}{\bf{u}}^{\bm{(0)}}_{\bf{n}}({\bf{r}}/R(t))\
,\ $ (26)

where ${\bf{u}}^{\bm{(0)}}_{\bf{s}}({\bf{r}}/R(t))$ and
${\bf{u}}^{\bm{(0)}}_{\bf{s}}({\bf{r}}/R(t))$ are dimensionless fields that
depend on structural details of the normal and superfluid vortex rings (number
of structures, relative positions, etc.). Eqs. (25) and (26) can be used to
derive rough dimensional estimates of the several contributions in (22). The
superfluid kinetic energy (23), for instance, would be estimated as

$E_{s}=\frac{1}{2}\rho_{s}\kappa^{2}R\int
d^{3}{\bf{r}}[{\bf{u}}^{\bm{(0)}}_{\bf{s}}({\bf{r}})]^{2}\propto\rho_{s}\kappa^{2}R\
.\ $ (27)

However, this estimate is found to lose accuracy for vortex rings which have
their core radius $r_{c}$ much smaller than $R$. A better approximation is
(hollow vortices) given as saffman

$E_{s}=\frac{1}{2}\rho_{s}\kappa^{2}R\left[\ln\left(\frac{8}{e^{2}}\right)+\ln\left(\frac{R}{r_{c}}\right)\right]\
.\ $ (28)

The above $\ln(R/r_{c})$ contribution comes essentially from the fast growth
of the velocity field intensity and approximate axial symmetry near the cores
of slender vortex ring filaments, the ones which have $r_{c}\ll R$. An
analogous log-correction term is observed for the vortex ring translation
velocity, Eq. (9).

In order to cope with both the far and the near-core vortex ring field
asymptotics, we put forward estimates for the various terms in (22) which
contain, besides the predictions obtained from (25) and (26), further log-
correction contributions, having in mind that the only relevant length scales
in the system are $R$ and $r_{c}$. More concretely, we write

$\displaystyle\star~{}~{}~{}E_{n}=\rho_{s}\kappa^{2}R\left[a_{1}+a_{2}\ln\left(\frac{R}{r_{c}}\right)\right]\
,\ $ (29) $\displaystyle\star~{}~{}~{}\mu_{n}\int
d^{3}{\bf{r}}{\hbox{Tr}}\left[\left({\bm{\nabla}}\otimes{\bf{u_{n}}}\right)^{2}\right]=\frac{\nu}{R}\rho_{s}\kappa^{2}\left[b_{1}+b_{2}\ln\left(\frac{R}{r_{c}}\right)\right]\
,\ $ (30)
$\displaystyle\star~{}~{}~{}\rho_{s}\kappa\sum_{i}\int_{C_{i}}ds({\bf{u_{n}}}-{\bf{u_{s}}})_{\perp}^{2}=\frac{\nu}{R}\rho_{s}\kappa^{2}\left[c_{1}+c_{2}\ln\left(\frac{R}{r_{c}}\right)\right]\
,\ $ (31)
$\displaystyle\star~{}~{}~{}\rho_{s}\kappa\sum_{i}\int_{C_{i}}ds\left\\{({\bf{u_{n}}}-{\bf{u_{s}}})_{\perp}\cdot({\bf{U_{n}}}-{\bf{U_{s}}})_{\perp}\right\\}=U_{ns}\rho_{s}\kappa^{2}\left[d_{1}+d_{2}\ln\left(\frac{R}{r_{c}}\right)\right]\
,\ $ (32)
$\displaystyle\star~{}~{}~{}\rho_{s}\kappa\sum_{i}\int_{C_{i}}ds({\bf{u_{n}}}-{\bf{u_{s}}})\cdot[{\bm{\hat{\omega}}}_{\bf{s}}\times({\bf{U_{n}}}-{\bf{U_{s}}})]=U_{ns}\rho_{s}\kappa^{2}\left[e_{1}+e_{2}\ln\left(\frac{R}{r_{c}}\right)\right]\
,\ $ (33)

where $a_{1}$, $a_{2}$, …, $e_{1}$, $e_{2}$ are unknown dimensionless
coefficients to be discussed in a moment and $\nu\equiv\mu_{n}/\rho_{n}$ is
the kinematic viscosity of the normal fluid. The attentive reader will notice
that in Eqs. (29) and (30), the estimates are expressed in terms of $\rho_{s}$
instead of $\rho_{n}$. Also, the apparently extraneous kinematic viscosity
prefactor $\nu$ comes into play in Eq. (31). We note, accordingly, that all of
these definitions are actually just convenient ways to establish standardized
estimates, which are in good terms with dimensional analysis principles.

The profusion of arbitrary coefficients in Eqs. (29-33) could induce one to
think, in a first moment, that the energy-budget line of reasoning carried out
so far has reached a hopeless dead end. However, there is an interesting
phenomenological solution to this issue, based on the fact that as the
superfluid vortex ring collapses and its energy (28) decays, all the terms in
(22) concomitantly decay as well. It is reasonable to conjecture, thus, that
all the unknown coefficients in (29-33) have their ratios $a_{1}/a_{2}$,
$b_{1}/b_{2}$, …, $e_{1}/e_{2}$ approximately locked to the value prescribed
by the known coefficients of (28). In more expicit terms, we postulate that

$\frac{a_{1}}{a_{2}}\simeq\frac{b_{1}}{b_{2}}\simeq{\hbox{ ...
}}\simeq\frac{e_{1}}{e_{2}}\simeq\ln\left(\frac{8}{e^{2}}\right)\ .\ $ (34)

Taking (29-33) and (34) into account (replacing approximations by equalities),
it follows, from (22), that

$\frac{dE_{s}}{dt}=-\gamma(R)E_{s}\ ,\ $ (35)

where, introducing a pair of dimensionless unknowns $\beta_{1}$ and
$\beta_{2}$,

$\gamma(R)\equiv\beta_{1}\frac{U_{ns}}{R}+\beta_{2}\frac{\nu}{R^{2}}\ .\ $
(36)

The coefficients $\beta_{1}$ and $\beta_{2}$, which are linear functions of
$a_{2}$, $b_{2}$, …, $e_{2}$, have now the status of the only two independent
adjusting phenomenological parameters that are necessary to model the radius
evolution of the decorated vortex ring decay.

Figure 4: Vortex ring radius as a function of time. Circles represent
experimental data taken from Ref. bewley_sree . The solid line is obtained
from the numerical solution of Eq. (38) with the optimal parameters
$\beta_{1}=0.14$ and $\beta_{2}=0.36$ and initial condition $R(0)=400$ $\mu$m.

The two terms in the RHS of the radius-dependent decay rate $\gamma(R)$, Eq.
(36), have appealing physical interpretations: energy can be removed from the
coupled normal-superfluid vortex rings by background drag/Magnus forcing
(decay rates $\propto U_{ns}/R$) combined with normal viscous dissipation and
excitation of normal flow near the superfluid vortex ring (decay rates
$\propto\nu/R^{2}$).

As already discussed in Sec. II, the vortex ring core radius is not constant,
but it grows in time as the vortex ring decays, due to volume conservation. We
revisit here the estimate of the parameter $c$, introduced in Eq. (13), under
the light of the present discussion. For this sake, we assume that the
asymptotic radius $R_{\infty}$ that characterizes the complete collapse of the
superfluid vortex ring is the one that leads to $E_{s}=0$ in (28), that is,

$\frac{R_{\infty}}{r_{c}}=\frac{e^{2}}{8}\ .\ $ (37)

From the decay experiment data, one gets $R_{\infty}\simeq 30$ $\mu$m, which
gives, from (13) and (37), $c=5.6\times 10^{-3}\mu$m${}^{-\frac{3}{2}}$. For
the kinematic viscosity of the normal component of Helium II we take, at the
temperature of $T=2.06$ K, $\nu=1.08\times 10^{-8}$m${}^{2}/$s bar-donn . The
counterflow velocity is estimated as $U_{ns}=500$ $\mu$m/s bewley_sree .

Using (13) and (28), the energy dissipation equation (35) can be reshuffled as

$\displaystyle\frac{dR}{dt}=-R\gamma(R)\left[\frac{\partial\ln(E)}{\partial\ln(R)}\right]^{-1}$
$\displaystyle=-\left(\beta_{1}U_{ns}+\beta_{2}\frac{\nu}{R}\right)\left[\frac{\ln(8kR^{\frac{3}{2}})-2}{\ln(8kR^{\frac{3}{2}})-\frac{1}{2}}\right]\
.\ $ (38)

An excellent fitting to the data (set by least squares regression) is shown in
Fig. 4, produced from the numerical solution of (38) with $\beta_{1}=0.14$ and
$\beta_{2}=0.36$.

It is worth mentioning that direct HVBK numerical simulations of bare vortex
rings galan_etal  were found to lead to similar linear time-dependent decaying
profiles for $R(t)$. It is possible that the phenomenological discussion
addressed in this paper can be extended to the case of decaying vortex rings
which are free of attached particles or sparsely decorated.

##  IV Conclusions

We have investigated, within an essentially phenomenological framework, the
decay of densely decorated quantum vortex rings. Relying upon energy balance
arguments, we have been able to accurately reproduce the vortex ring radius
measurements taken in the BS experiment bewley_sree . It turns out that
backreaction effects, modeled through the HVBK coupled hydrodynamical
equations barenghi_etalbook ; galan_etal , are a fundamental ingredient to
quantify the exchange of kinetic energy between the superfluid vortex ring and
its surrounding normal (viscous) fluid.

The assumption that quantum vortices are massless is by no means obvious even
at zero temperature, since quantum excitations may play a role in processes
associated to superfluid linear momentum variations. As a consequence, vortex
filaments may interact and evolve in completely different ways as the ones
predicted from the Schwarz’s structural approach schwarz .

It would be interesting to investigate, under the light of the present
findings, the modifications that should be implemented in the standard
structural description of vortex dynamics, as encoded in Eq. (3), so as to
establish alternative ways to model quantum vortex motion. One gets back, in
this connection, to the long-time debated topic of quantum vortex inertia
toikka-brand ; simula .

In order to derive the radius evolution equation (38), we had to resort to
simple and reasonable hypotheses, like (34), that could be validated through
further numerical simulations of the HVBK equations. We do not mean
necessarily the numerical study of particle-laden superflows, which may be too
heavy for the analysis of realistic cases, but simplified numerical solutions
where, as an effective boundary condition, the vortex ring core is prescribed
to have a fixed volume as the vortex ring decays.

###### Acknowledgements.

This work has been partially supported by CNPq under grant number
307659/2015-1 .

## References

  * (1) L. Onsager, Il Nuovo Cimento 6, 279 (1949).
  * (2) R.P. Feynman, Prog. Low Temp. Phys. 1, 17 (1955).
  * (3) R.J. Donnelly, Quantized Vortices in Helium II, Cambridge University Press (1991).
  * (4) G.A. Williams, Phys. Rev. Lett. 59, 1926 (1987).
  * (5) S.R. Shenoy, Phys. Rev. B 40, 5056 (1989).
  * (6) M. Tsubota and W.P. Halperin (Editors), Progress in Low Temperature Physics: Quantum Turbulence, Vol. XVI, Elsevier (2008).
  * (7) R.G. Cooper, M. Mesgarnezhad, A.W. Baggaley, and C.F. Barenghi, Sci. Rep. 9, 10545 (2019).
  * (8) W.F. Vinen, Nature 181, 1524 (1958).
  * (9) S.C. Whitmore and W. Zimmermann, Phys. Rev. 166, 181 (1968).
  * (10) E.J. Yarmchuk, M.J.V. Gordon, and R.E. Packard, Phys. Rev. Lett. 43, 214 (1979).
  * (11) G.A. Williams and R.E. Packard, Phys. Rev. Lett. 33, 280 (1974).
  * (12) G.P. Bewley, D.P. Lathrop, and K.R. Sreenivasan, Nature 441, 588 (2006).
  * (13) T.V. Chagovets and S.W. Van Sciver, Phys. Fluids 23, 107102 (2011).
  * (14) W. Guoa, M. La Mantiac, D.P. Lathrop, and S.W. Van Sciver, Proc. Natl. Acad. Sci. USA 111, 4653 (2014).
  * (15) G.P. Bewley, M.S. Paoletti, K.R. Sreenivasan, D.P. Lathrop, Proc. Natl. Acad. Sci. USA 105, 13707 (2008).
  * (16) G.P. Bewley and K.R. Sreenivasan, J. Low Temp. Phys. 156, 84 (2009).
  * (17) E. Fonda, D.P. Meichle, N.T. Ouellette, S. Hormoz, and D.P. Lathrop, Proc. Natl. Acad. Sci. USA 25, 4707 (2014).
  * (18) T. Zhang and S.W. Van Sciver, J. Low Temp. Phys. 138, 865 (2005).
  * (19) Y.A. Sergeev, C.F. Barenghi, and D. Kivotides, Phys. Rev. B 74, 184506 (2006).
  * (20) D. Kivotides and S.L. Wilkin, J. Fluid Mech. 605, 367 (2008).
  * (21) D. Duda, Quantum Turbulence in Superfluid Helium Studied by Particle Tracking Velocimetry Visualization Technique, PhD Dissertation, Charles University in Prague (2017).
  * (22) B. Mastracci and W. Guo, Phys. Rev. Fluids 3, 063304 (2018).
  * (23) L. Moriconi, Phys. Rev. E 100, 043102 (2019).
  * (24) Y. Tang, S. Bao, T. Kanai, and W. Guo, Phys. Rev. Fluids 5, 084602 (2020).
  * (25) K.W. Schwarz, Phys. Rev. B 31, 5782 (1985).
  * (26) C.F. Barenghi, Introduction to Superfluid Vortices and Turbulence, in Quantized Vortex Dynamics and Superfluid Turbulence, edited by C.F. Barenghi, R.J. Donnelly, and W.F. Vinen, Springer-Verlag (2001).
  * (27) A. Villois, D. Proment, and G. Krstulovic, Phys. Rev. E 93, 061103(R) (2016).
  * (28) D. Kivotides, C.F. Barenghi, D.C. Samuels, Science 290, 777 (2000).
  * (29) L. Galantucci, A.W. Baggaley, C.F. Barenghi, and G. Krstulovic, Eur. Phys. J. Plus 135, 547 (2020).
  * (30) L.A. Toikka and J. Brand, New J. Phys. 19, 023029 (2017).
  * (31) T. Simula, Phys. Rev. A 97, 023609 (2018).
  * (32) C.F. Barenghi and R.J. Donnely, J. Phys. Chem. Ref. Data 27, 1217 (1998).
  * (33) P.G. Saffman, Vortex Dynamics, Cambridge University Press (1995).
  * (34) H.E. Hall, Adv. Phys. 9, 89 (1960).
  * (35) S.B. Pope, Turbulent Flows, Cambridge University Press (2000).