# nanoT5: A PyTorch Framework for Pre-training and Fine-tuning
T5-style Models with Limited Resources

Piotr Nawrot
University of Edinburgh
https://github.com/PiotrNawrot/nanoT5

###### Abstract

State-of-the-art language models like T5 have revolutionized the NLP
landscape, but their computational demands hinder a large portion of the
research community. To address this challenge, we present nanoT5, a specially-
optimized PyTorch framework for efficient pre-training and fine-tuning of T5
models. Drawing on insights from optimizer differences and prioritizing
efficiency, nanoT5 allows a T5-Base model to be pre-trained on a single GPU in
just 16 hours, without any loss in performance. With the introduction of this
open-source framework, we hope to widen the accessibility to language
modelling research and cater to the community’s demand for more user-friendly
T5 (Encoder-Decoder) implementations. Our contributions, including
configurations, codebase, software/hardware insights, and pre-trained models,
are available to the public, aiming to strike a balance between research
accessibility and resource constraints in NLP.

##  1 Introduction

The transformative power of large pre-trained language models such as GPT-3
Brown et al. (2020), T5 Raffel et al. (2019), and PaLM Chowdhery et al. (2022)
is undeniable. However, their hefty computational requirements remain a
barrier for many researchers. Notably, models like T5 require extensive
datasets and significant computational resources for their pre-training Raffel
et al. (2019). Furthermore, many open-source implementations lean heavily on
TPU accelerators Shazeer (2020), which are not as available to the academic
community as GPUs.

Recognizing this gap, we introduce nanoT5, a resource-efficient, open-source
PyTorch framework designed for the pre-training and fine-tuning of T5 models.
Inspired by pioneering efforts such as nanoGPT Karpathy (2021) and Cramming
Geiping and Goldstein (2022), nanoT5 uniquely concentrates on enhancing the
training pipeline specifically for T5 encoder-decoder models. Our framework
includes optimized configurations and scripts, enabling researchers to pre-
train a T5-Base model with 248M parameters on a single GPU in just 16 hours.
Every facet, from data preprocessing and model architecture to the learning
rate schedule, has been tuned for both efficiency and adaptability. With
nanoT5, users can seamlessly initiate model training within minutes of
accessing our GitHub repository.

This paper underscores two main innovations: First, we delve into the nuances
between the Adam and Adafactor optimizer performances as detailed in Havinga ,
suggesting a version of AdamW Loshchilov and Hutter (2017), augmented with
matrix-wise learning rate scaling based on root mean square. This variant
showcases better speed and robustness compared to the default Adafactor
Shazeer and Stern (2018). Second, we demonstrate that T5 models trained with
nanoT5, housing around 250M parameters, can achieve performance akin to the
publicly-available checkpoints while requiring 150x less pre-training data.

Our primary motivation stems from the growing demand for reproducible and
tuned baselines (Kaddour et al., 2023), enabling fast and small-scale
hypothesis validation in the evolving realm of large pre-trained Transformers.
With nanoT5, we address a gap highlighted by community requests
111https://github.com/google-research/text-to-text-transfer-
transformer/issues/172222https://github.com/huggingface/transformers/issues/18030333https://github.com/huggingface/transformers/issues/5079,
providing an approachable platform for working with T5 (Encoder-Decoder)
architectures. To our understanding, nanoT5 pioneers the effort to reproduce
T5 v1.1 pre-training using PyTorch, deviating from prior Jax/Flax
implementations. We invite the community to explore our training guidelines,
codebase, and pre-trained models, all of which are available at
https://github.com/PiotrNawrot/nanoT5.

##  2 Related Work

The landscape of open-source repositories tailored for efficient pre-training
of Transformer language models is vast. Notably, nanoGPT Karpathy (2021) sheds
light on decoder-only models, while Cramming Geiping and Goldstein (2022)
homes in on the optimal pre-training of the encoder-only BERT architecture
Devlin et al. (2019). Contrastingly, with nanoT5, we sought to bridge the
existing gap by providing a standalone research template tailored for the
T5-style (Encoder-Decoder) models.

To expedite the training process of nanoT5 we incorporated various
optimizations. These encompass mixed precision training (Micikevicius et al.,
2017), compiled runtimes (Narang et al., 2021), and more. Additionally, we
delved into the potential of efficient training methodologies such as recent
optimizers (Chen et al., 2023; Liu et al., 2023), and fast attention mechanism
(Dao et al., 2022), which are elaborated further in Section 4.3. It’s crucial
to note that while we evaluated various efficient algorithms, we consciously
opted against those, such as (Nawrot et al., 2022; Shazeer et al., 2017), that
would modify the core model structure. Instead, our intent with nanoT5 was to
cultivate a straightforward baseline for further research endeavors. The
standout contribution of our work in terms of efficient training algorithms is
our in-depth exploration into the RMS matrix scaling to improve AdamW
(Loshchilov and Hutter, 2017) convergence.

##  3 Methodology

Our validation strategy seeks to replicate the T5-base pre-training outcomes
detailed in (Shazeer, 2020) and the fine-tuning results of Tk-Instruct on the
Super Natural-Instructions (SNI) meta-dataset (Wang et al., 2022).

###  3.1 Training pipeline

We’ve devised a comprehensive training pipeline prioritizing efficient data
management, low-level optimizations, and coding simplicity, all while
preserving the core model and training logic:

  * •

Dataset Handling: Given the extensive volume of the C4 dataset, which exceeds
300GB, our repository implements concurrent data downloading with model
training. This optimization speeds up the commencement of T5 model pre-
training to a few minutes.

  * •

Exposure and Simplicity: Our methodology aims to strike a balance between
adaptability and abstraction. With tools such as the HuggingFace Accelerator
(Sylvain Gugger, 2022), we abstract tasks like checkpoint management and
tensor operations. Experiment tracking is realized via neptune.ai (Neptune
team, 2019), and we’ve employed hydra (Yadan, 2019) for coordinated
hyperparameter handling.

  * •

Efficiency: We’ve leveraged the optimizations of PyTorch 2.0 (Paszke et al.,
2019), and employed mixed-precision training in line with established
optimization guidelines
444https://huggingface.co/docs/transformers/perf_train_gpu_one555https://pytorch.org/tutorials/recipes/recipes/tuning_guide.html.

  * •

Flexibility: Our repository is designed with adaptability in mind, offering
support for multi-GPU training, gradient accumulation, and gradient
checkpointing. This ensures users can reproduce our results on a variety of
GPUs beyond the A100 and can experiment with configurations larger than the
T5-base size emphasized in this study. Additionally, we provide support for
both CPUs and Apple’s ARM M1 chips.

###  3.2 Pre-training

Our experiments strictly follow the T5-v1.1-base training configuration
(Shazeer, 2020), where the model itself comprises of roughly 248M parameters.
The C4 dataset (Raffel et al., 2019), sourced directly from Huggingface,
undergoes tokenization via the Wordpiece tokenizer (Schuster and Nakajima,
2012), with the original model’s vocabulary. During pre-processing, 15% of
input data is masked using sentinel tokens, setting the neural network’s
target as the prediction of these tokens, leveraging its decoder. Consistent
with the original study, we’ve set the batch size at 128 examples, with inputs
of 512 tokens and outputs of 114 tokens. Optimization is facilitated through
the Adafactor optimizer (Shazeer and Stern, 2018), combined with the Inverse-
Square-Root (ISR) learning rate schedule. The model is trained for $2^{16}$
steps. For more details please refer to the original work.

###  3.3 Fine-tuning

Our fine-tuning employs the Super Natural-Instructions (SNI) meta-dataset
(Wang et al., 2022), which has been previously used for fine-tuning models
like FlanT5 (Chung et al., 2022), BLOOM (Scao et al., 2022), and Tk-Instruct
(Wang et al., 2022). To assess the correctness of our fine-tuning setup, and
the efficiency of our pre-training, we decided to reproduce the Tk-Instruct
methodology.

Figure 1: Downstream performance of models across various pre-training
durations, including existing T5-base variants accessible through Huggingface
Hub.

###  3.4 Reproducibility

Ensuring that our work can be reliably replicated is a core focus of our
methodology. To facilitate this, we have taken the following measures:

  * •

Model Weights: We make the model’s weights available on the HuggingFace Hub.
These can be downloaded and used for fine-tuning on the SNI dataset with
nanoT5.

  * •

Loss Curves: We openly share both the pre-training and fine-tuning loss curves
to provide insight into the model’s learning dynamics.

  * •

Hyperparameters: All hyperparameters used in our experiments have been
released.

  * •

Environment and Hardware: In our repository we offer comprehensive
instructions on how to recreate our environment, including detailed
information about our hardware. This encompasses specifications of our CPU and
GPU, as well as the relevant driver versions.

  * •

Statistical Robustness: To ensure the validity of our results, each experiment
was conducted three times with different random seeds.

##  4 Results

###  4.1 Reproducing Pre-Training

By following the original experimental setup described above, we achieved a
negative log-likelihood of $1.995$ on the held-out set, which is slightly
inferior to the reference.

In exploring alternative optimization methods, we tested the AdamW optimizer
as a potential replacement for the original Adafactor. While AdamW
theoretically promises greater training stability by directly estimating the
second moment of the gradients (as opposed to Adafactor’s low-rank
approximation), our training with AdamW diverged. This behavior mirrors
findings from a study on T5 pre-training Havinga . Upon further investigation,
we identified that matrix-wise learning rate (LR) scaling using its root mean
square (RMS) was the crucial element ensuring Adafactor’s convergence. After
augmenting AdamW with RMS scaling, it not only converged but also exhibited
improved stability during pre-training and operated slightly faster, thanks to
the direct retrieval of the second moment from memory instead of approximating
it.

Nonetheless, when combined with the Inverse-Square-Root LR schedule, AdamW’s
performance was still outpaced by Adafactor. By replacing the ISR schedule
with a Cosine LR Schedule, we achieved a superior negative log-likelihood of
1.953 on the held-out set, significantly surpassing Adafactor with the ISR
schedule. The specific results of these experiments can be found in Table 2. A
comparison of the training loss curves using different optimizers (Adafactor
vs. AdamW) and schedules (ISR vs. Cosine) is provided in Figure 2.

Mixed Precision | Torch 2.0 compile | Grad Acc | Time per 1 Pre-training step | Total Pre-training time
---|---|---|---|---
FP32 | No | 2 |  $\sim$4.10s |  $\sim$74.6h
TF32 | No | 2 |  $\sim$1.39s |  $\sim$25.3h
BF16 | No | 2 |  $\sim$1.30s |  $\sim$23.7h
TF32 | Yes | 2 |  $\sim$0.95s |  $\sim$17.3h
BF16 | Yes | 1 |  $\sim$0.56s |  $\sim$10.2h

Table 1: Efficiency metrics across various configuration settings during pre-training, with the "Total Pre-training Time" column referencing $2^{16}$ steps following the default config. | Inverse-Square-Root | Cosine
---|---|---
Adafactor | 1.995 | 1.993
AdamW | 2.040 | 1.953
Table 2: Comparison of negative log-likelihood scores on the held-out set of
C4 using different optimization methods and learning rate schedules. Figure 2:
Training loss curves contrasting different optimizers and learning rate
schedules.

###  4.2 Fine-Tuning Performance Across Different Pre-Training Durations

Our fine-tuning configuration strictly aligns with that of Tk-Instruct.
However, there remains some ambiguity regarding whether Tk-Instruct was
initialized from a regular checkpoint (google/t5-v1_1-base) or from a version
specifically tailored for Language Modelling (google/t5-base-lm-adapt). To
cover all bases, we evaluated both, and successfully reproduced the original
results.

Figure 1 presents a performance comparison of the model we trained in various
time increments (ranging from 4 to 24 hours) against the original T5-base-v1.1
model weights from Huggingface Hub and its language modeling-adapted version.
Notably, our model, trained for 16 hours on a single GPU, lagged by only 0.2
RougeL on average compared to the original T5-base-v1.1. This is an impressive
result given the vast disparity in training data (the T5 paper indicates
training on approximately 150x more data than we did). The language modeling-
adapted checkpoint outperformed both the original T5-base-v1.1 model and ours,
but this language modeling model adaptation extends beyond the scope of this
study. A single fine-tuning step in our setup took approximately 0.18s,
culminating in roughly an hour for the entire fine-tuning process.

###  4.3 Efficiency Statistics

Table 1 showcases the efficiency metrics from our pre-training experiments. It
details the time taken for a single pre-training step and the overall pre-
training time based on our default configuration. A noteworthy point is our
need to increase the gradient accummulation steps to fit the model with a
precision other than BF16.

#### Attempts at Boosting Efficiency

In our pursuit of efficiency, we experimented with various strategies, albeit
with limited success:

  * •

Optimization Algorithms: We assessed the performance of recent optimizers like
Lion Chen et al. (2023) and Sophia Liu et al. (2023). However, neither
outperformed the AdamW with RMS scaling.

  * •

Positional Embeddings: We tried replacing T5’s learned relative positional
embeddings with ALiBi Press et al. (2021). Although such a switch had the
potential to reduce the number of parameters, leading to faster training and
inference rates, and paving the way for integrating Flash Attention Dao et al.
(2022) (currently limited to non-parametric bias), our trials revealed that
training with ALiBi was more volatile and yielded suboptimal pre-training
loss.

  * •

FP16 Precision: Unfortunately, all our attempts using FP16 precision
consistently diverged.

##  5 Conclusions

In this study, we demonstrated the feasibility of pre-training a substantial
model like T5 under resource constraints, specifically using a single A100 GPU
within a 24-hour timeframe. Through selection of optimization methods and
configurations, we achieved results comparable to large-scale training
settings. Our intention in sharing the codebase, configurations, and training
logs is to bridge the gap between research accessibility and computational
resource limitations in the NLP domain. We invite and welcome community
suggestions to further refine and enhance our approach.

Moving forward, we aim to enrich our codebase by incorporating additional
training objectives, such as those suggested by Tworkowski et al. (2023); Tay
et al. (2022), in hopes of further optimizing the training pipeline.

## Acknowledgements

This work was supported by the UKRI Centre for Doctoral Training in Natural
Language Processing, funded by the UKRI (grant EP/S022481/1) and the
University of Edinburgh, School of Informatics and School of Philosophy,
Psychology & Language Sciences.

## References

  * Brown et al. (2020) Tom B. Brown, Benjamin Mann, Nick Ryder, Melanie Subbiah, Jared Kaplan, Prafulla Dhariwal, Arvind Neelakantan, Pranav Shyam, Girish Sastry, Amanda Askell, Sandhini Agarwal, Ariel Herbert-Voss, Gretchen Krueger, T. J. Henighan, Rewon Child, Aditya Ramesh, Daniel M. Ziegler, Jeff Wu, Clemens Winter, Christopher Hesse, Mark Chen, Eric Sigler, Mateusz Litwin, Scott Gray, Benjamin Chess, Jack Clark, Christopher Berner, Sam McCandlish, Alec Radford, Ilya Sutskever, and Dario Amodei. 2020.  Language models are few-shot learners.  _ArXiv_ , abs/2005.14165.
  * Chen et al. (2023) Xiangning Chen, Chen Liang, Da Huang, Esteban Real, Kaiyuan Wang, Yao Liu, Hieu Pham, Xuanyi Dong, Thang Luong, Cho-Jui Hsieh, Yifeng Lu, and Quoc V. Le. 2023\.  Symbolic discovery of optimization algorithms.  _ArXiv_ , abs/2302.06675.
  * Chowdhery et al. (2022) Aakanksha Chowdhery, Sharan Narang, Jacob Devlin, Maarten Bosma, Gaurav Mishra, Adam Roberts, Paul Barham, Hyung Won Chung, Charles Sutton, Sebastian Gehrmann, Parker Schuh, Kensen Shi, Sasha Tsvyashchenko, Joshua Maynez, Abhishek Rao, Parker Barnes, Yi Tay, Noam M. Shazeer, Vinodkumar Prabhakaran, Emily Reif, Nan Du, Benton C. Hutchinson, Reiner Pope, James Bradbury, Jacob Austin, Michael Isard, Guy Gur-Ari, Pengcheng Yin, Toju Duke, Anselm Levskaya, Sanjay Ghemawat, Sunipa Dev, Henryk Michalewski, Xavier García, Vedant Misra, Kevin Robinson, Liam Fedus, Denny Zhou, Daphne Ippolito, David Luan, Hyeontaek Lim, Barret Zoph, Alexander Spiridonov, Ryan Sepassi, David Dohan, Shivani Agrawal, Mark Omernick, Andrew M. Dai, Thanumalayan Sankaranarayana Pillai, Marie Pellat, Aitor Lewkowycz, Erica Moreira, Rewon Child, Oleksandr Polozov, Katherine Lee, Zongwei Zhou, Xuezhi Wang, Brennan Saeta, Mark Díaz, Orhan Firat, Michele Catasta, Jason Wei, Kathleen S. Meier-Hellstern, Douglas Eck, Jeff Dean, Slav Petrov, and Noah Fiedel. 2022.  Palm: Scaling language modeling with pathways.  _ArXiv_ , abs/2204.02311.
  * Chung et al. (2022) Hyung Won Chung, Le Hou, S. Longpre, Barret Zoph, Yi Tay, William Fedus, Eric Li, Xuezhi Wang, Mostafa Dehghani, Siddhartha Brahma, Albert Webson, Shixiang Shane Gu, Zhuyun Dai, Mirac Suzgun, Xinyun Chen, Aakanksha Chowdhery, Dasha Valter, Sharan Narang, Gaurav Mishra, Adams Wei Yu, Vincent Zhao, Yanping Huang, Andrew M. Dai, Hongkun Yu, Slav Petrov, Ed Huai hsin Chi, Jeff Dean, Jacob Devlin, Adam Roberts, Denny Zhou, Quoc V. Le, and Jason Wei. 2022.  Scaling instruction-finetuned language models.  _ArXiv_ , abs/2210.11416.
  * Dao et al. (2022) Tri Dao, Daniel Y. Fu, Stefano Ermon, Atri Rudra, and Christopher R’e. 2022.  Flashattention: Fast and memory-efficient exact attention with io-awareness.  _ArXiv_ , abs/2205.14135.
  * Devlin et al. (2019) Jacob Devlin, Ming-Wei Chang, Kenton Lee, and Kristina Toutanova. 2019.  Bert: Pre-training of deep bidirectional transformers for language understanding.  _ArXiv_ , abs/1810.04805.
  * Geiping and Goldstein (2022) Jonas Geiping and Tom Goldstein. 2022.  Cramming: Training a language model on a single gpu in one day.  _ArXiv_ , abs/2212.14034.
  * (8) Yeb Havinga.  Pre-training dutch t5 models.
  * Kaddour et al. (2023) Jean Kaddour, Oscar Key, Piotr Nawrot, Pasquale Minervini, and Matt J. Kusner. 2023\.  No train no gain: Revisiting efficient training algorithms for transformer-based language models.  _ArXiv_ , abs/2307.06440.
  * Karpathy (2021) Andrej Karpathy. 2021.  nanogpt.  https://github.com/karpathy/nanoGPT.  GitHub repository.
  * Liu et al. (2023) Hong Liu, Zhiyuan Li, David Leo Wright Hall, Percy Liang, and Tengyu Ma. 2023.  Sophia: A scalable stochastic second-order optimizer for language model pre-training.  _ArXiv_ , abs/2305.14342.
  * Loshchilov and Hutter (2017) Ilya Loshchilov and Frank Hutter. 2017.  Decoupled weight decay regularization.  In _International Conference on Learning Representations_.
  * Micikevicius et al. (2017) Paulius Micikevicius, Sharan Narang, Jonah Alben, Gregory Frederick Diamos, Erich Elsen, David García, Boris Ginsburg, Michael Houston, Oleksii Kuchaiev, Ganesh Venkatesh, and Hao Wu. 2017.  Mixed precision training.  _ArXiv_ , abs/1710.03740.
  * Narang et al. (2021) Sharan Narang, Hyung Won Chung, Yi Tay, William Fedus, Thibault Févry, Michael Matena, Karishma Malkan, Noah Fiedel, Noam M. Shazeer, Zhenzhong Lan, Yanqi Zhou, Wei Li, Nan Ding, Jake Marcus, Adam Roberts, and Colin Raffel. 2021\.  Do transformer modifications transfer across implementations and applications? _ArXiv_ , abs/2102.11972.
  * Nawrot et al. (2022) Piotr Nawrot, Jan Chorowski, Adrian La’ncucki, and E. Ponti. 2022.  Efficient transformers with dynamic token pooling.  In _Annual Meeting of the Association for Computational Linguistics_.
  * Neptune team (2019) Neptune team. 2019.  neptune.ai.
  * Paszke et al. (2019) Adam Paszke, Sam Gross, Francisco Massa, Adam Lerer, James Bradbury, Gregory Chanan, Trevor Killeen, Zeming Lin, Natalia Gimelshein, Luca Antiga, Alban Desmaison, Andreas Kopf, Edward Yang, Zachary DeVito, Martin Raison, Alykhan Tejani, Sasank Chilamkurthy, Benoit Steiner, Lu Fang, Junjie Bai, and Soumith Chintala. 2019.  PyTorch: An Imperative Style, High-Performance Deep Learning Library.  In _Advances in Neural Information Processing Systems 32_ , pages 8024–8035. Curran Associates, Inc.
  * Press et al. (2021) Ofir Press, Noah A. Smith, and Mike Lewis. 2021.  Train short, test long: Attention with linear biases enables input length extrapolation.  _ArXiv_ , abs/2108.12409.
  * Raffel et al. (2019) Colin Raffel, Noam M. Shazeer, Adam Roberts, Katherine Lee, Sharan Narang, Michael Matena, Yanqi Zhou, Wei Li, and Peter J. Liu. 2019.  Exploring the limits of transfer learning with a unified text-to-text transformer.  _ArXiv_ , abs/1910.10683.
  * Scao et al. (2022) Teven Le Scao, Angela Fan, Christopher Akiki, Elizabeth-Jane Pavlick, Suzana Ili’c, Daniel Hesslow, Roman Castagn’e, Alexandra Sasha Luccioni, Franccois Yvon, Matthias Gallé, Jonathan Tow, Alexander M. Rush, Stella Rose Biderman, Albert Webson, Pawan Sasanka Ammanamanchi, Thomas Wang, Benoît Sagot, Niklas Muennighoff, Albert Villanova del Moral, Olatunji Ruwase, Rachel Bawden, Stas Bekman, Angelina McMillan-Major, Iz Beltagy, Huu Nguyen, Lucile Saulnier, Samson Tan, Pedro Ortiz Suarez, Victor Sanh, Hugo Laurenccon, Yacine Jernite, Julien Launay, Margaret Mitchell, Colin Raffel, Aaron Gokaslan, Adi Simhi, Aitor Soroa Etxabe, Alham Fikri Aji, Amit Alfassy, Anna Rogers, Ariel Kreisberg Nitzav, Canwen Xu, Chenghao Mou, Chris C. Emezue, Christopher Klamm, Colin Leong, Daniel Alexander van Strien, David Ifeoluwa Adelani, Dragomir R. Radev, Eduardo Gonz’alez Ponferrada, Efrat Levkovizh, Ethan Kim, Eyal Bar Natan, Francesco De Toni, Gérard Dupont, Germán Kruszewski, Giada Pistilli, Hady ElSahar, Hamza Benyamina, Hieu Trung Tran, Ian Yu, Idris Abdulmumin, Isaac Johnson, Itziar Gonzalez-Dios, Javier de la Rosa, Jenny Chim, Jesse Dodge, Jian Zhu, Jonathan Chang, Jorg Frohberg, Josephine L. Tobing, Joydeep Bhattacharjee, Khalid Almubarak, Kimbo Chen, Kyle Lo, Leandro von Werra, Leon Weber, Long Phan, Loubna Ben Allal, Ludovic Tanguy, Manan Dey, Manuel Romero Muñoz, Maraim Masoud, Mar’ia Grandury, Mario vSavsko, Max Huang, Maximin Coavoux, Mayank Singh, Mike Tian-Jian Jiang, Minh Chien Vu, Mohammad Ali Jauhar, Mustafa Ghaleb, Nishant Subramani, Nora Kassner, Nurulaqilla Khamis, Olivier Nguyen, Omar Espejel, Ona de Gibert, Paulo Villegas, Peter Henderson, Pierre Colombo, Priscilla Amuok, Quentin Lhoest, Rheza Harliman, Rishi Bommasani, Roberto L’opez, Rui Ribeiro, Salomey Osei, Sampo Pyysalo, Sebastian Nagel, Shamik Bose, Shamsuddeen Hassan Muhammad, Shanya Sharma, S. Longpre, Somaieh Nikpoor, Stanislav Silberberg, Suhas Pai, Sydney Zink, Tiago Timponi Torrent, Timo Schick, Tristan Thrush, Valentin Danchev, Vassilina Nikoulina, Veronika Laippala, Violette Lepercq, Vrinda Prabhu, Zaid Alyafeai, Zeerak Talat, Arun Raja, Benjamin Heinzerling, Chenglei Si, Elizabeth Salesky, Sabrina J. Mielke, Wilson Y. Lee, Abheesht Sharma, Andrea Santilli, Antoine Chaffin, Arnaud Stiegler, Debajyoti Datta, Eliza Szczechla, Gunjan Chhablani, Han Wang, Harshit Pandey, Hendrik Strobelt, Jason Alan Fries, Jos Rozen, Leo Gao, Lintang Sutawika, M Saiful Bari, Maged S. Al-shaibani, Matteo Manica, Nihal V. Nayak, Ryan Teehan, Samuel Albanie, Sheng Shen, Srulik Ben-David, Stephen H. Bach, Taewoon Kim, Tali Bers, Thibault Févry, Trishala Neeraj, Urmish Thakker, Vikas Raunak, Xiang Tang, Zheng Xin Yong, Zhiqing Sun, Shaked Brody, Y Uri, Hadar Tojarieh, Adam Roberts, Hyung Won Chung, Jaesung Tae, Jason Phang, Ofir Press, Conglong Li, Deepak Narayanan, Hatim Bourfoune, Jared Casper, Jeff Rasley, Max Ryabinin, Mayank Mishra, Minjia Zhang, Mohammad Shoeybi, Myriam Peyrounette, Nicolas Patry, Nouamane Tazi, Omar Sanseviero, Patrick von Platen, Pierre Cornette, Pierre Franccois Lavall’ee, Rémi Lacroix, Samyam Rajbhandari, Sanchit Gandhi, Shaden Smith, Stéphane Requena, Suraj Patil, Tim Dettmers, Ahmed Baruwa, Amanpreet Singh, Anastasia Cheveleva, Anne-Laure Ligozat, Arjun Subramonian, Aur’elie N’ev’eol, Charles Lovering, Daniel H Garrette, Deepak R. Tunuguntla, Ehud Reiter, Ekaterina Taktasheva, Ekaterina Voloshina, Eli Bogdanov, Genta Indra Winata, Hailey Schoelkopf, Jan-Christoph Kalo, Jekaterina Novikova, Jessica Zosa Forde, Xiangru Tang, Jungo Kasai, Ken Kawamura, Liam Hazan, Marine Carpuat, Miruna Clinciu, Najoung Kim, Newton Cheng, Oleg Serikov, Omer Antverg, Oskar van der Wal, Rui Zhang, Ruochen Zhang, Sebastian Gehrmann, S. Osher Pais, Tatiana Shavrina, Thomas Scialom, Tian Yun, Tomasz Limisiewicz, Verena Rieser, Vitaly Protasov, Vladislav Mikhailov, Yada Pruksachatkun, Yonatan Belinkov, Zachary Bamberger, Zdenvek Kasner, Alice Rueda, Amanda Pestana, Amir Feizpour, Ammar Khan, Amy Faranak, Ananda Santa Rosa Santos, Anthony Hevia, Antigona Unldreaj, Arash Aghagol, Arezoo Abdollahi, Aycha Tammour, Azadeh HajiHosseini, Bahareh Behroozi, Benjamin Olusola Ajibade, Bharat Kumar Saxena, Carlos Muñoz Ferrandis, Danish Contractor, David M. Lansky, Davis David, Douwe Kiela, Duong Anh Nguyen, Edward Tan, Emily Baylor, Ezinwanne Ozoani, Fatim T Mirza, Frankline Ononiwu, Habib Rezanejad, H.A. Jones, Indrani Bhattacharya, Irene Solaiman, Irina Sedenko, Isar Nejadgholi, Jan Passmore, Joshua Seltzer, Julio Bonis Sanz, Karen Fort, Lívia Macedo Dutra, Mairon Samagaio, Maraim Elbadri, Margot Mieskes, Marissa Gerchick, Martha Akinlolu, Michael McKenna, Mike Qiu, M. K. K. Ghauri, Mykola Burynok, Nafis Abrar, Nazneen Rajani, Nour Elkott, Nourhan Fahmy, Olanrewaju Samuel, Ran An, R. P. Kromann, Ryan Hao, Samira Alizadeh, Sarmad Shubber, Silas L. Wang, Sourav Roy, Sylvain Viguier, Thanh-Cong Le, Tobi Oyebade, Trieu Nguyen Hai Le, Yoyo Yang, Zachary Kyle Nguyen, Abhinav Ramesh Kashyap, Alfredo Palasciano, Alison Callahan, Anima Shukla, Antonio Miranda-Escalada, Ayush Kumar Singh, Benjamin Beilharz, Bo Wang, Caio Matheus Fonseca de Brito, Chenxi Zhou, Chirag Jain, Chuxin Xu, Clémentine Fourrier, Daniel Le’on Perin’an, Daniel Molano, Dian Yu, Enrique Manjavacas, Fabio Barth, Florian Fuhrimann, Gabriel Altay, Giyaseddin Bayrak, Gully A. Burns, Helena U. Vrabec, Iman I.B. Bello, Isha Dash, Ji Soo Kang, John Giorgi, Jonas Golde, Jose David Posada, Karthi Sivaraman, Lokesh Bulchandani, Lu Liu, Luisa Shinzato, Madeleine Hahn de Bykhovetz, Maiko Takeuchi, Marc Pàmies, María Andrea Castillo, Marianna Nezhurina, Mario Sanger, Matthias Samwald, Michael Cullan, Michael Weinberg, M Wolf, Mina Mihaljcic, Minna Liu, Moritz Freidank, Myungsun Kang, Natasha Seelam, Nathan Dahlberg, Nicholas Michio Broad, Nikolaus Muellner, Pascale Fung, Patricia Haller, R. Chandrasekhar, R. Eisenberg, Robert Martin, Rodrigo L. Canalli, Rosaline Su, Ruisi Su, Samuel Cahyawijaya, Samuele Garda, Shlok S Deshmukh, Shubhanshu Mishra, Sid Kiblawi, Simon Ott, Sinee Sang-aroonsiri, Srishti Kumar, Stefan Schweter, Sushil Pratap Bharati, T. A. Laud, Th’eo Gigant, Tomoya Kainuma, Wojciech Kusa, Yanis Labrak, Yashasvi Bajaj, Y. Venkatraman, Yifan Xu, Ying Xu, Yun chao Xu, Zhee Xao Tan, Zhongli Xie, Zifan Ye, Mathilde Bras, Younes Belkada, and Thomas Wolf. 2022.  Bloom: A 176b-parameter open-access multilingual language model.  _ArXiv_ , abs/2211.05100.
  * Schuster and Nakajima (2012) Mike Schuster and Kaisuke Nakajima. 2012.  Japanese and Korean voice search.  In _2012 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP)_ , pages 5149–5152.
  * Shazeer (2020) Noam M. Shazeer. 2020.  Glu variants improve transformer.  _ArXiv_ , abs/2002.05202.
  * Shazeer et al. (2017) Noam M. Shazeer, Azalia Mirhoseini, Krzysztof Maziarz, Andy Davis, Quoc V. Le, Geoffrey E. Hinton, and Jeff Dean. 2017.  Outrageously large neural networks: The sparsely-gated mixture-of-experts layer.  _ArXiv_ , abs/1701.06538.
  * Shazeer and Stern (2018) Noam M. Shazeer and Mitchell Stern. 2018.  Adafactor: Adaptive learning rates with sublinear memory cost.  _ArXiv_ , abs/1804.04235.
  * Sylvain Gugger (2022) Thomas Wolf Philipp Schmid Zachary Mueller Sourab Mangrulkar Marc Sun Benjamin Bossan Sylvain Gugger, Lysandre Debut. 2022.  Accelerate: Training and inference at scale made simple, efficient and adaptable.  https://github.com/huggingface/accelerate.
  * Tay et al. (2022) Yi Tay, Mostafa Dehghani, Vinh Q. Tran, Xavier García, Jason Wei, Xuezhi Wang, Hyung Won Chung, Dara Bahri, Tal Schuster, Huaixiu Steven Zheng, Denny Zhou, Neil Houlsby, and Donald Metzler. 2022.  Ul2: Unifying language learning paradigms.  In _International Conference on Learning Representations_.
  * Tworkowski et al. (2023) Szymon Tworkowski, Konrad Staniszewski, Mikolaj Pacek, Yuhuai Wu, Henryk Michalewski, and Piotr Milo’s. 2023.  Focused transformer: Contrastive training for context scaling.  _ArXiv_ , abs/2307.03170.
  * Wang et al. (2022) Yizhong Wang, Swaroop Mishra, Pegah Alipoormolabashi, Yeganeh Kordi, Amirreza Mirzaei, Anjana Arunkumar, Arjun Ashok, Arut Selvan Dhanasekaran, Atharva Naik, David Stap, Eshaan Pathak, Giannis Karamanolakis, Haizhi Gary Lai, Ishan Purohit, Ishani Mondal, Jacob Anderson, Kirby Kuznia, Krima Doshi, Maitreya Patel, Kuntal Kumar Pal, M. Moradshahi, Mihir Parmar, Mirali Purohit, Neeraj Varshney, Phani Rohitha Kaza, Pulkit Verma, Ravsehaj Singh Puri, Rushang Karia, Shailaja Keyur Sampat, Savan Doshi, Siddharth Deepak Mishra, Sujan Reddy, Sumanta Patro, Tanay Dixit, Xudong Shen, Chitta Baral, Yejin Choi, Noah A. Smith, Hanna Hajishirzi, and Daniel Khashabi. 2022.  Super-naturalinstructions: Generalization via declarative instructions on 1600+ nlp tasks.  In _Conference on Empirical Methods in Natural Language Processing_.
  * Yadan (2019) Omry Yadan. 2019.  Hydra - a framework for elegantly configuring complex applications.  Github.


# On the analytically-improved running coupling in QCD

F. T. Brandt<EMAIL_ADDRESS>Instituto de Física, Universidade de São Paulo,
São Paulo, SP 05508-090, Brazil  J. Frenkel<EMAIL_ADDRESS>Instituto de
Física, Universidade de São Paulo, São Paulo, SP 05508-090, Brazil  D. G. C.
McKeon<EMAIL_ADDRESS>Department of Applied Mathematics, The University of
Western Ontario, London, Ontario N6A 5B7, Canada  Department of Mathematics
and Computer Science, Algoma University, Sault Ste. Marie, Ontario P6A 2G4,
Canada  G. S. S. Sakoda<EMAIL_ADDRESS>Instituto de Física,
Universidade de São Paulo, São Paulo, SP 05508-090, Brazil

###### Abstract

We examine, in the ‘t Hooft renormalization scheme, the analytic running
coupling $\bar{\alpha}_{t}(Q^{2})$ in QCD, using the two-loop $\beta$-function
with positive expansion parameters $\beta_{0}$ and $\beta_{1}$. An exact
integral representation is derived for this causal coupling, which is fully
expressed in terms of the imaginary part of the Lambert function $W$. This
integral form manifestly accounts for the universal value of the infrared
limit $\bar{\alpha}_{t}(Q^{2}=0)=4\pi/\beta_{0}$.

gauge field theories, dispersion relations, renormalization group

###### pacs:

05.10.Cc

##  I Introduction

In perturbative calculations of physical processes, there appear at higher
orders ultraviolet divergences, which require a procedure (called the
renormalization scheme) for their subtraction. At finite order in perturbation
theory, a number of ambiguities occur as the renormalization procedure is not
uniquely specified and, in particular, amplitudes calculated in perturbation
theory appear to depend on an arbitrary renormalization scale (usually denoted
by $\mu$). However, since these amplitudes are connected to physical
observables, these cannot depend on such a parameter (or any other parameter
that characterize the renormalization scheme). This can be achieved if the
coupling becomes a function of $\mu$, which leads to the independence of
observables on the choice of this scale parameter. The renormalization group
(RG) is closely related to the scale invariance of physical systems.
Furthermore, it improves the properties of the perturbative series in the
ultraviolet region, by summing the radiative corrections that occur in higher
orders of perturbation theory r1 ; r2 ; r3 ; r4 ; r5 ; Lavrov:2012xz . Due to
asymptotic freedom r6 , physical quantities in QCD at high momentum squared
$Q^{2}$ can be evaluated by solving the RG equations in terms of a power
series involving the perturbative running coupling $\alpha(Q^{2})$.

It is well known that such a running coupling leads to unphysical
singularities like the Landau pole r7 , which renders perturbation theory
useless for low-energy processes in QCD. A resolution of this problem was
proposed a long time ago r8 , when the RG method was unified with the
requirement of analyticity. This led to the concept of an analytic running
coupling $\bar{\alpha}(Q^{2})$ that is free of unphysical singularities.
Subsequently, other methods have been developed through the use of dispersion
relations that reflect the causality principle r9 ; r10 ; r11 ; r12 , and
successfully applied in QCD. The analytic approach preserves the ultraviolet
behavior in the ultraviolet region, which ensures asymptotic freedom but leads
to essential modifications of the behaviour in the infrared region. There are
no unphysical singularities in the low energy region. An important feature of
this approach is the presence of a universal value of the analytic coupling
$\bar{\alpha}(Q^{2}=0)=4\pi/\beta_{0}$, where $\beta_{0}$ is the coefficient
of the $\beta$-function at one loop. Several arguments for the universality of
the analytic running coupling have been previously given in in the literature,
from various points of view r9 ; r10 ; r11 ; r12 ; r13 ; r14 ; r15 ; r16 ; r17
; Milton:1997miMilton:1998jy ; r19aa . Another relevant property of the
analytic approach is that it allows us to define the causal running coupling
$\bar{\alpha}(Q^{2})$ both in the space-like as well as in the time-like
domains, in a consistent way. These features enable meaningful calculations of
hadronic processes in QCD, like the inelastic lepton-hadron scattering and the
electron-positron annihilation into hadrons as described, for example, in the
reviews Prosperi:2006hx ; Deur:2016tte  and the references cited therein.

Moreover, there are also other analytic methods for improving the QCD
perturbation theory Deur:2016tte . These generally involve some criterion for
making a choice of the renormalization scale $\mu$, or other parameters that
characterize a renormalization scheme, so that the perturbative result best
approximates the exact (scheme-independent) result. In this context, we
mention the “Optimized Perturbation Theory”, which consists in improving the
convergence of the perturbative expansion by choosing the value of $\mu$
according to a criterion of minimum sensitivity Stevenson:1980du ;
Stevenson:1981vj . We also point out the “Fastest Apparent Convergence”
technique Grunberg:1980ja ; Grunberg:1982fw , that amounts to selecting the
scale $\mu$ so that the next-to leading and higher order coefficients of the
perturbative series are set to zero. This optimization procedure is related to
effective charges Grunberg:1980ja ; Grunberg:1982fw ; Brodsky:2011zza ;
Brodsky:2011ig , which consist in defining new running couplings that are more
directly connected with physical observables. In addition, we mention the
“Principle of Maximum Conformality” Deur:2016tte  which improves the QCD
predictive power by removing the dependence of physical predictions on the
choice of the renormalization scheme. There has also been much work towards
unifying the different approaches, including the AdS-CFT/Dyson-Schwinger
methods deTeramond:2008ht ; Cui:2019dwv , which may lead to a consistent
universal running coupling in QCD.

The purpose of this letter is to give an exact integral representation for the
causal $\bar{\alpha}(Q^{2})$ coupling in the ’t Hooft scheme, which leads in a
simple and direct way to the universal value of its infrared limit. In section
II we succinctly review some features of RG equations and of the ‘t Hooft
renormalization scheme r18  where the $\beta$-function has exactly the two-
loop form, which will be useful subsequently. This approach removes the
arbitrariness that occurs in other renormalization schemes. Here we also give
the known form of the perturbative running coupling at two-loops in terms of
the Lambert function r19 . In section III we derive an exact integral form
(Eq. (21)) for the analytic running coupling, which is entirely expressed in
terms of the imaginary part of Lambert’s function. A useful analytic
approximation of this integral form that is good to within $7.7$% of the exact
expression is given in Eq. (22). We conclude this note with a summary of the
results in section IV, where we also briefly discuss an application of this
approach to the $e^{+}e^{-}$ annihilation into hadrons. A more systematic
analysis of the role of the ’t Hooft coupling, when considering the
renormalization scheme dependence in perturbative QCD, is given in Appendix C.

##  II The perturbative running coupling

The perturbative running coupling is defined by the RG differential equation

$Q^{2}\frac{d}{dQ^{2}}\alpha(Q^{2})=-\beta(\alpha(Q^{2}));\;\;\;\alpha(\mu^{2})=\alpha,$
(1)

where $\mu$ is the renormalization point, $\alpha=g^{2}/4\pi$ and $g$ is the
renormalized coupling constant.

The $\beta$-function may be written in the form

$\beta(\alpha(Q^{2}))=\frac{\beta_{0}}{4\pi}\alpha^{2}(Q^{2})+\frac{\beta_{1}}{(4\pi)^{2}}\alpha^{3}(Q^{2})+\frac{\beta_{2}}{(4\pi)^{3}}\alpha^{4}(Q^{2})+\cdots,$
(2)

where, up to two-loop order ($N_{f}$ being the number of active quark flavors)

$\beta_{0}=11-2N_{f}/3;\;\;\;\beta_{1}=102-38N_{f}/3.$ (3)

G. ‘t Hooft showed r18  that it is possible to choose a coupling $\alpha_{t}$
such that the $\beta$-function has just the 2-loop form, which is invariant
under changes of the renormalization scheme. This condition leads to the
relation r20 ; r21

$\alpha={\alpha_{t}}+\left(\frac{1}{4\pi}\right)^{2}\frac{\beta_{2}}{\beta_{0}}\alpha_{t}^{3}+\cdots.$
(4)

It has been argued r20 ; r22 ; r23  that an all-order summation of the terms
which depend explicitly on $\mu$ and the expansion parameters
$\beta_{2},\dots$, yields a $\beta$-function that is consistent with ‘t Hooft
scheme.

At two loops, a straightforward integration of Eq. (1) gives r9 ; r10 ; r11 ;
r12

$\ln\left(\frac{Q^{2}}{\Lambda^{2}}\right)=\frac{1}{a(Q^{2}/\Lambda^{2})}-\frac{\beta_{1}}{\beta_{0}^{2}}\ln\left(1+\frac{\beta_{0}^{2}}{\beta_{1}}\frac{1}{a(Q^{2}/\Lambda^{2})}\right),$
(5)

where $a(Q^{2}/\Lambda^{2})\equiv(\beta_{0}/4\pi)\,\alpha(Q^{2}/\Lambda^{2})$
and $\Lambda$ is the QCD scale parameter, which is associated with the
$\alpha(1)=\infty$ boundary condition in Eq. (1).

An inversion of Eq. (5) can be written in terms of the Lambert multi-valued
function $W$ r19 , which is defined as

$W(z)\exp[W(z)]=z$ (6)

where

$z(Q^{2})=-\frac{1}{e}\left(\frac{Q^{2}}{\Lambda^{2}}\right)^{-\beta^{2}_{0}/\beta_{1}}$
(7)

($e$ is the Euler number). One can now verify that an exact solution of the
Eq. (5) is given by r15 ; r17

$a(z)=-\frac{\beta_{0}^{2}}{\beta_{1}}\frac{1}{1+W(z)}.$ (8)

We note that in the ‘t Hooft scheme, this expression would give the complete
solution. The requirement that $a(Q^{2})$ is real and positive for positive
$Q^{2}$ and that it should vanish in the limit $Q^{2}\rightarrow\infty$,
determines the appropriate branch of the Lambert function. In this work we
consider QCD with $N_{f}\leq 6$, which implies that $\beta_{0}$ and
$\beta_{1}$ in Eq. (3) are positive. In this case, the function $z(Q^{2})$ in
Eq. (7) is negative and the correct physical branch that ensures asymptotic
freedom is $W_{-1}(z)$, which becomes negative and infinite in the limit
$z\rightarrow 0$.

##  III The analytic running coupling

This causal coupling may be constructed through a dispersion relation as r9 ;
r10 ; r11 ; r12

$\bar{a}_{t}(Q^{2}/\Lambda^{2})\equiv\frac{1}{\pi}\int_{0}^{\infty}d\sigma\frac{\Im
a_{t}(-\sigma)}{\sigma+Q^{2}},$ (9)

where $Q^{2}>0$ for space-like momentum transfer.

To one-loop order, the perturbative running coupling has the form

$a_{t}^{(1)}(Q^{2}/\Lambda^{2})=\frac{1}{\ln(Q^{2}/\Lambda^{2})}.$ (10)

Performing the analytical continuation $\sigma\rightarrow\sigma+i\epsilon$ and
evaluating the imaginary part of $a_{t}(-\sigma)$ in Eq. (9), yields the one-
loop analytic running coupling function

$\bar{a}_{t}^{(1)}(Q^{2}/\Lambda^{2})=\int_{0}^{\infty}d\sigma\frac{1}{\sigma+Q^{2}}\frac{1}{\ln^{2}(\sigma/\Lambda^{2})+\pi^{2}}=\frac{1}{\ln(Q^{2}/\Lambda^{2})}+\frac{1}{1-Q^{2}/\Lambda^{2}}.$
(11)

The first term is the usual perturbative running coupling at one-loop. The
second term ensures the correct analytic properties, by cancelling the Landau
pole present in the first term. The above expression has a physical cut when
the real part of $Q^{2}$ is negative, and no other singularities, so that it
is consistent with causality. We also note that $\bar{a}_{t}^{(1)}(0)=1$.

We next proceed to the two-loop case, by using the expression given in Eq. (8)
for the perturbative running coupling. To this end, one needs to evaluate the
imaginary part of this coupling. One then gets in Eq. (9) a complicated
integrand, because the integration variable $\sigma$ occurs implicitly in the
Lambert function $W$. For this reason, it is convenient to transform this
equation into an equivalent integral, where the integration variable is just
the imaginary part of $W$. To this end, we will proceed in two steps as
follows. We first change the integration variable to $z$, as given by Eq. (7),
with $Q^{2}$ replaced by $\sigma$

$z(\sigma)=-\frac{1}{e}\left(\frac{\sigma}{\Lambda^{2}}\right)^{-\beta_{0}^{2}/\beta_{1}}$
(12)

so that the integration will occur along the negative real $z$-axis

$\bar{a}_{t}(Q^{2}/\Lambda^{2})=-\frac{e}{\pi}\int_{-\infty}^{0}dz\frac{(-ez)^{-\beta_{1}/\beta_{0}^{2}-1}}{(-ez)^{-\beta_{1}/\beta_{0}^{2}}+Q^{2}/\Lambda^{2}}\Im\frac{1}{1+W(z(-\sigma))}.$
(13)

We now pass to calculate the imaginary part of $1/[1+W(z(-\sigma))]$, using
Eq. (12) and making the analytic continuation
$\sigma\rightarrow\sigma+i\epsilon$. In this way, we find that we need to
evaluate the Lambert function $W$ at the point
$Z(\sigma)=z(\sigma)\exp(i\pi\beta_{0}^{2}/\beta_{1})$. Thus, we write

$Z=x+iy$ (14)

and

$W=\tau+i\eta.$ (15)

Using the above equations together with the relation $W(Z)\exp[W(Z)]=Z$, and
equating the imaginary parts of this relation, one finds that

$x=z(\sigma)\cos(\pi\beta_{0}^{2}/\beta_{1})=e^{\tau}(\tau\cos\eta-\eta\sin\eta)$
(16)

and

$y=z(\sigma)\sin(\pi\beta_{0}^{2}/\beta_{1})=e^{\tau}(\eta\cos\eta+\tau\sin\eta).$
(17)

Dividing the last two equations and using a trigonometric identity, one
obtains

$\tau=-\eta\cot(\eta-\pi\beta_{0}^{2}/\beta_{1}).$ (18)

Substituting this relation in Eq. (16) and using a trigonometric identity, we
get

$z(\sigma(\eta))=-\frac{\eta}{\sin(\eta-\pi\beta_{0}^{2}/\beta_{1})}\exp-(\eta\cot(\eta-\pi\beta_{0}^{2}/\beta_{1})).$
(19)

We can now evaluate the imaginary contribution in Eq. (13), by using the
relations (8), (12), (14– 18), obtaining

$\Im\frac{1}{1+W(z)}=\Im\frac{1}{1-\eta\cot(\eta-\pi\beta_{0}^{2}/\beta_{1})+i\eta}=-\frac{\eta}{1-2\eta\cot(\eta-\pi\beta_{0}^{2}/\beta_{1})+\eta^{2}/\sin^{2}(\eta-\pi\beta_{0}^{2}/\beta_{1})}.$
(20)

Substituting this relation in Eq. (13) and changing the variable of
integration from $z(\eta)$ to $\eta$ by using Eq. (19), one obtains after a
straightforward calculation, the following integral form for the analytic
running coupling

$\bar{a}_{t}(Q^{2}/\Lambda^{2})=\frac{1}{\pi}\int_{\pi\beta_{0}^{2}/\beta_{1}}^{\pi+\pi\beta_{0}^{2}/\beta_{1}}d\eta\frac{1}{1+(Q^{2}/\Lambda^{2})[-ez(\eta)]^{\beta_{1}/\beta_{0}^{2}}}.$
(21)

The above result expresses the causal coupling $\bar{a}(Q^{2}/\Lambda^{2})$ in
terms of an integral involving the imaginary part $\eta$ of the Lambert
function, which corresponds to the branch $W_{1}$ r19 . This exact integral
representation directly shows that, at the point $Q^{2}=0$, this coupling
equals to $1$, which implies that $\bar{\alpha}_{t}(Q^{2}=0)=4\pi/\beta_{0}$,
in accordance with the universal value obtained by other methods.

The above form of $\bar{a}_{t}(Q^{2}/\Lambda^{2})$ shows that it has a
negative derivative whose magnitude decreases from infinity to zero as the
value of $Q^{2}/\Lambda^{2}$ increases from zero to infinity. Moreover, the
second derivative of $\bar{a}_{t}(Q^{2})$ is everywhere positive. Thus, one
may expect the integral to behave as depicted in Fig. 1, obtained by a
numerical integration. The graphs shown in Fig. 1 illustrate the behaviour of
this integral for $\beta_{1}/\beta_{0}^{2}=64/81$ and
$\beta_{1}/\beta_{0}^{2}=26/49$, corresponding respectively to $N_{f}=3$ and
$N_{f}=6$ in the Eq. (3). One can see from this figure that for large values
of $Q^{2}/\Lambda^{2}$, the analytic running coupling becomes, like the
perturbative running coupling, of order $1/\ln(Q^{2}/\Lambda^{2})$. This may
be understood by noticing that in this case, the main contribution to the
integral comes from the region where the integrand in Eq. (21) is of order
$1$. But one can easily check that this requires values of $\eta$ near
$\pi\beta_{0}^{2}/\beta_{1}$, in a range of order $1/\ln(Q^{2}/\Lambda^{2})$.
One then gets a slowly varying integrand, so the integral becomes proportional
to the range of such a region. This explains the logarithmic form of the
result obtained numerically for large values of $Q^{2}/\Lambda^{2}$. The
figure also shows that, in the infrared limit, there are no higher-loop
corrections.

The result shown in Fig. 1 fully agrees with that obtained numerically at two
loops, for $N_{f}=3$, in Ref. r17 . It is also consistent with the result
obtained in Ref. r19aa  by a different method, which does not involve the
exact Lambert solution of Eq. (5). One may verify that in the limit
$\beta_{1}\rightarrow 0$, Eq. (21) reduces to Eq. (11), as expected (see
Appendix A).

Figure 1: The behaviour of the integral in Eq. (21) for
$\beta_{1}/\beta_{0}^{2}=64/81$ (solid) and $26/49$ (dashed) corresponding
respectively to $N_{f}=3$ and $N_{f}=6$. The analytic one-loop result is
plotted as a dotted line.

Although the integration in Eq. (21) cannot be performed in closed form, a
useful analytic approximation may be obtained by making in the dispersion
relation (9) the shift $\sigma\rightarrow\sigma-Q^{2}$ so that
$a_{t}(-\sigma)\rightarrow a_{t}(Q^{2}-\sigma)$. Then, in view of the fact
that $a_{t}(Q^{2}-\sigma)$ is a slowly varying function of the logarithmic
type, one may neglect here the $Q^{2}$ dependence. This procedure is correct
for small $Q^{2}$ and is also valid for large values of $Q^{2}$, since in the
ultraviolet region the imaginary part of
$[a_{t}(-\sigma)-a_{t}(Q^{2}-\sigma)]$ leads to corrections of order
$1/\ln^{3}(Q^{2})$. In this way, we obtain the approximate analytic expression
(see Appendix B)

$\bar{a}_{t}(Q^{2}/\Lambda^{2})\simeq 1-\frac{1}{\pi}\Im\ln
W_{1}[Z(Q^{2}/\Lambda^{2})]$ (22)

where

$Z(Q^{2}/\Lambda^{2})=-\frac{1}{e}(Q^{2}/\Lambda^{2})^{-\beta_{0}^{2}/\beta_{1}}\exp(i\pi\beta_{0}^{2}/\beta_{1}).$
(23)

The above result has the correct limits both in the infrared as well as in the
ultraviolet regions. Moreover, the numerical values displayed in table 1 show
that in the intermediate region this approximation is accurate to within
$7.7\%$ (see also Ref. r17 ).

$Q^{2}/\Lambda^{2}$ | Eq. (21) | Eq. (22) | Relative difference
---|---|---|---
0.00 | 1.0000 | 1.0000 | 0.000
1.00 | 0.3518 | 0.3411 | 0.030
10.0 | 0.2286 | 0.2112 | 0.076
20.0 | 0.2013 | 0.1858 | 0.077
40.0 | 0.1782 | 0.1651 | 0.074
80.0 | 0.1590 | 0.1482 | 0.068
100. | 0.1535 | 0.1435 | 0.066
200. | 0.1384 | 0.1303 | 0.059
400. | 0.1258 | 0.1193 | 0.052
800. | 0.1152 | 0.1100 | 0.045
$10^{6}$ | 0.0616 | 0.0607 | 0.015
$10^{7}$ | 0.0536 | 0.0531 | 0.009
Table 1: Numerical results from Eq. (21) and the approximation given by Eq.
(22), for $N_{f}=3$. The third column shows that the results agree to within
7.7%. The last two lines show the level of accuracy of Eq. (22) for very large
values of $Q^{2}/\Lambda^{2}$.

##  IV Discussion

We have examined, in the ‘t Hooft scheme, the behaviour of the analytic
running coupling $\bar{\alpha}_{t}(Q^{2})$ in QCD, for $\beta_{0}>0$ and
$\beta_{1}>0$. This coupling is independent of the renormalization procedure.
This causal coupling, that is free from unphysical singularities, preserves
the ultraviolet behaviour of the perturbative running coupling which ensures
the asymptotic freedom of the theory, but modifies its behaviour in the
infrared region. A relevant property of this approach is that the value of the
analytic running coupling at $Q^{2}=0$ is determined just by the one-loop
corrections as $\bar{\alpha}_{t}(Q^{2}=0)=4\pi/\beta_{0}$. An exact integral
representation of the analytic running coupling, that is expressed in terms of
the Lambert function, is given in Eq. (21) which manifestly leads to this
universal infrared limit.

We note here that the equation (5) may, alternatively, be solved by the
iteration method r9 ; r10 ; r11 ; r12 . Although this procedure is accurate
for large values of $Q^{2}/\Lambda^{2}$ when $a(Q^{2}/\Lambda^{2})$ is small,
it violates the analytical attributes of $a(Q^{2}/\Lambda^{2})$ near the point
$Q^{2}/\Lambda^{2}=1$ r17 . The present approach is consistent with the
requirement of analyticity and thus is convenient to investigate further the
analytical properties of the causal coupling
$\bar{\alpha}_{t}(Q^{2}/\Lambda^{2})$.

These aspects may be useful to improve the calculation of physical quantities
evaluated through the RG equations. For example, let us consider the ratio $R$
which is related to the total cross section for the $e^{+}e^{-}$ annihilation
into hadrons. This is a function of the center of mass energy squared $s$ and
of the renormalized coupling constant $\alpha$. At high energies, this ratio
may be written in terms of a perturbative series as r24 ; r25

$R(s/\mu^{2},\alpha(\mu^{2}/\Lambda^{2}))=\sum_{i}Q_{i}^{2}\left[1+t_{1}(s/\mu^{2})\alpha(\mu^{2}/\Lambda^{2})+t_{2}(s/\mu^{2})\alpha^{2}(\mu^{2}/\Lambda^{2})+\cdots\right],$
(24)

where $Q_{i}e$ is the charge of the $i$-th quark. The coefficients $t_{1}$,
$t_{2}$ $\cdots$ include in general large logarithms like $\ln(s/\mu^{2})$.
The RG equation for this ratio implies that $R$ is independent of the
renormalization point $\mu$, having the form (see Appendix C )

$R(s/\mu^{2},\alpha(\mu^{2}/\Lambda^{2}))=R(1,\alpha(s/\Lambda^{2})),$ (25)

where $\alpha(s)$ is the perturbative running coupling. One can thus express
$R$ as follows

$R=\sum_{i}Q_{i}^{2}\left[1+t_{1}(1)\alpha(s/\Lambda^{2})+t_{2}(1)\alpha^{2}(s/\Lambda^{2})+\cdots\right].$
(26)

When $\alpha(s/\Lambda^{2})$ is given in the ‘t Hooft renormalization scheme,
then $t_{1}$, $t_{2}$, etc, are renormalization scheme invariants r20 . The
simplest way to improve the series for the ratio $R$ consists of replacing in
the Eq. (26), the perturbative running coupling by the analytic running
coupling $\bar{\alpha}(s/\Lambda^{2})$ defined in the time-like region.

An enhanced, but more involved, procedure has been elaborated in Refs. r12 ;
Milton:1997miMilton:1998jy ; r17 ; Prosperi:2006hx . It turns out that the
time-like and the space-like analytical running couplings may be related by
the linear integral transformation ($q^{2}>0$)

$\bar{\alpha}(s)=\frac{i}{2\pi}\int_{s-i\epsilon}^{s+i\epsilon}\frac{dq^{2}}{q^{2}}\bar{\alpha}(-q^{2}),$
(27)

where the contour integral is computed along a path in the analiticity region
of the $\bar{\alpha}(-q^{2})$ function. These analytic running couplings have
a common infrared stable point. In the ultraviolet limit, such couplings also
have the same asymptotic behaviour. However, these functions generally differ
in the intermediate energy range.

Relations between RG-invariant quantities defined in the Minkowski and
Euclidean domains may be established by making use of linear integral
transformations. An example is the measurable ratio $R(s)$ which can be
similarly related to the hadronic polarization function $\Pi(-q^{2})$,
calculable perturbatively in the Euclidean region.

A key feature of the analytic perturbation theory is the transformation of
series involving powers of the space-like running coupling into functional
expansions involving the time-like running coupling. These expansion
functions, which are free of unphysical singularities and have the universal
value at the infrared stable point, yield an enhanced convergence of the
analytic perturbation theory. Such a procedure has been performed for the
$\tau$-lepton and $\Upsilon$ decays, for hadronic form factors as well as for
$e^{+}e^{-}$ annihilation into hadrons, leading to results which are in a
rather good agreement with the experimental observations r12 ; Prosperi:2006hx
.

###### Acknowledgements.

We would like to thank CNPq (Brazil) for financial support.

##  Appendix A The exact analytic coupling in the limit $\beta_{1}\rightarrow
0$

From Eq. (21) one gets, in the limit $\beta_{1}\rightarrow 0$, the integral
($\phi=\eta-\pi\beta_{0}^{2}/\beta_{1}$)

$I(Q^{2}/\Lambda^{2})=\frac{1}{\pi}\int_{0}^{\pi}d\phi\frac{1}{1+(Q^{2}/\Lambda^{2})\exp(-\pi\cot\phi)}.$
(28)

It is now convenient to make the change of variable $x=-\pi\cot\phi$ so that
Eq. (28) becomes

$I(Q^{2}/\Lambda^{2})=\int_{-\infty}^{\infty}\frac{1}{1+(Q^{2}/\Lambda^{2})e^{x}}\frac{dx}{x^{2}+\pi^{2}}.$
(29)

Finally, making the change of variable $x=-\ln(\sigma/\Lambda^{2})$, we obtain

$I(Q^{2}/\Lambda^{2})=\int_{0}^{\infty}\frac{1}{\sigma+Q^{2}}\frac{d\sigma}{\ln^{2}(\sigma/\Lambda^{2})+\pi^{2}},$
(30)

which is equal to Eq. (11). This is expected because in the limit
$\beta_{1}\rightarrow 0$, the exact analytic coupling should reduce to that
obtained at one-loop.

Alternatively, using a contour of integration in the complex plane which
encloses only the poles inside the strip $0<\Im z<2\pi$, Eq. (29) can be
written as (see Eq. (7.5.5) of ComplexVariables )

$I(Q^{2}/\Lambda^{2})=\frac{1}{1-(Q^{2}/\Lambda^{2})}+{\rm
Res}_{z=z_{0}}\left[\frac{1}{1+(Q^{2}/\Lambda^{2})e^{z}}\frac{1}{z-\pi
i}\right]=\frac{1}{1-(Q^{2}/\Lambda^{2})}+\frac{1}{\ln(Q^{2}/\Lambda^{2})},$
(31)

where $z_{0}=-\ln(Q^{2}/\Lambda^{2})+\pi i$ is the zero of
${1+(Q^{2}/\Lambda^{2})e^{z}}$.

##  Appendix B An approximate analytic result

Making the shift $\sigma\rightarrow\sigma-Q^{2}$ in the dispersion relation
(9) and neglecting the $Q^{2}$ dependence in the perturbative coupling
$a_{t}$, we obtain

$\bar{a}_{t}(Q^{2}/\Lambda^{2})\simeq\frac{1}{\pi}\Im\int_{Q^{2}}^{\infty}\frac{d\sigma}{\sigma}a_{t}(-\sigma).$
(32)

Changing the variable of integration to $z(\sigma)$ given in Eq. (12) and
using the relation (8), one gets the integral

$\bar{a}_{t}(Q^{2}/\Lambda^{2})\simeq\frac{1}{\pi}\Im\int_{Z(Q^{2})}^{Z(\infty)}\frac{dZ}{Z}\frac{1}{1+W(Z)},$
(33)

where $Z(\sigma)=z(\sigma)\exp(i\pi\beta_{0}^{2}/\beta_{1})$.

Employing the identity r19

$\frac{dW(Z)}{dZ}=\frac{1}{Z}\frac{W}{1+W}$ (34)

one obtains from Eq. (33) the following result

$\bar{a}_{t}(Q^{2}/\Lambda^{2})\simeq\frac{1}{\pi}\Im\int_{W_{1}[Z(Q^{2})]}^{W_{1}[Z(\infty)]}\frac{dW}{W}=1-\frac{1}{\pi}\Im\ln
W_{1}[Z(Q^{2}/\Lambda^{2})],$ (35)

where we used the asymptotic form of the Lambert function $W_{1}$ r19 , which
leads to the form given in Eq. (22).

A simple example is provided by the one-loop analytic running coupling, in
which case the Eq. (32) yields

$\bar{a}_{t}^{(1)}(Q^{2}/\Lambda^{2})\simeq\int_{Q^{2}}^{\infty}\frac{d\sigma}{\sigma}\frac{1}{\ln^{2}(\sigma/\Lambda^{2})+\pi^{2}}=\frac{1}{2}-\frac{1}{\pi}\arctan\left[\frac{1}{\pi}\ln\left(\frac{Q^{2}}{\Lambda^{2}}\right)\right].$
(36)

One may verify that at $Q^{2}=0$, Eq. (36) gives a result equal to $1$, as
expected.

For very large values of $Q^{2}/\Lambda^{2}$, one can expand Eq. (36) as
follows

$\bar{a}_{t}^{(1)}(Q^{2}/\Lambda^{2})\simeq\frac{1}{\ln(Q^{2}/\Lambda^{2})}-\frac{\pi^{2}}{3}\frac{1}{\ln^{3}(Q^{2}/\Lambda^{2})}+\dots,$
(37)

which differs from the exact one-loop result (11), apart from very small power
correction like $\Lambda^{2}/Q^{2}$, by terms of order
$1/\ln^{3}(Q^{2}/\Lambda^{2})$. This is a general feature since in the
ultraviolet region all running couplings have a similar behaviour, due to
asymptotic freedom.

##  Appendix C Scheme independence

As has been mentioned in the Introduction, in Refs. Deur:2016tte ;
Stevenson:1980du ; Stevenson:1981vj ; Grunberg:1980ja ; Grunberg:1982fw ;
Brodsky:2011zza ; Brodsky:2011ig ; deTeramond:2008ht , various criteria have
been proposed for selecting a renormalization scheme so that a perturbative
result best approximates the exact (scheme-independent) result. In this
Appendix, rather than discussing any specific choice of renormalization scheme
in QCD used to compute $R(s)$ (the cross section for $e^{+}e^{-}\rightarrow$
hadrons) we show how the RG equation allows one to sum all higher order
contributions to $R(s)$ that explicitly involve parameters that characterize
the renormalization scheme being used, and that this summation leads to
cancellation between this explicit dependence on these parameters and implicit
dependence on these parameters through the QCD coupling $\alpha$. $R(s)$ is
finally expressed as a power series in the ’t Hooft coupling evaluated at
$s/\Lambda^{2}$ with coefficients that are independent of the renormaliztion
scheme being used.

Let us consider the ratio $R$ which is a function of the center of mass energy
squared $s$ and of the renormalized coupling constant $\alpha$. At high
energies, this ratio may be written in terms of a perturbative series as

$R(s/\mu^{2},\alpha(\mu^{2}/\Lambda^{2}))=\sum_{i}Q_{i}^{2}\left[1+\sum_{n=0}^{\infty}\sum_{m=0}^{n}T_{n,m}\ln^{m}\left(\frac{s}{\mu^{2}}\right)\alpha^{n+1}\left(\frac{\mu^{2}}{\Lambda^{2}}\right)\right],$
(38)

where $Q_{i}\,e$ is the charge on the $i^{th}$ quark. If we group the terms in
Eq. (38) so that

$A_{n}(\alpha)=\sum_{k=0}^{\infty}T_{n+k,n}\alpha^{n+k+1}$ (39)

then

$R=\sum_{i}Q^{2}_{i}\sum_{n=0}^{\infty}A_{n}(\alpha(\mu^{2}/\Lambda^{2}))\ln^{n}\left(\frac{s}{\mu^{2}}\right).$
(40)

The RG equation results in $A_{n+1}$ being expressed in terms of $A_{n}$; when
this relation is iterated, $R$ is expressed entirely in terms of
$A_{0}(\alpha)$ with $\alpha$ being evaluated at $s/\Lambda^{2}$. We then have
r20

$R(s/\mu^{2},\alpha(\mu^{2}/\Lambda^{2}))=\sum_{i}Q_{i}^{2}A_{0}(\alpha(s/\Lambda^{2}))=\sum_{i}Q_{i}^{2}\left[1+\sum_{n=0}^{\infty}\tau_{n}\alpha^{n+1}(s/\Lambda^{2})\right];\;\;\;\;\;(\tau_{n}=T_{n,0}).$
(41)

As expected, the RG equation leads to an expression for $R$ that is
independent of $\mu$. (This approach can also be used if there are massive
fields present r39 .) There is no need for devising some criterion for
choosing $\mu$ so that when considering a finite number of terms in the series
of Eq. (38), the perturbative result best approximates the exact result. In
Ref. r20  it is shown that the RG equation allows one to actually sum those
terms in $R$ that have an explicit dependence on $\mu$ through
$\ln(s/\mu^{2})$, and that once this is done, this explicit dependence on
$\mu^{2}$ cancels against the implicit dependence on $\mu^{2}$ in Eq. (38)
that resides in $\alpha(\mu^{2}/\Lambda^{2})$ (see also Akrami:2019sru ).

At this stage, the coupling $\alpha$ in Eq. (38) is dependent on the
renormalization scheme used, with the scheme being parametrized by
coefficients $\beta_{i}$ ($i\geq 2$) in Eq. (2), with
$\partial\alpha/\partial\beta_{i}$ depending on $\beta_{i}$ Stevenson:1981vj .
As a result, it is not feasible to effect the sort of summation that leads
from Eq. (38) to Eq. (40), so the explicit and implicit dependence of $R$ on
$\beta_{i}$ cannot be shown to explicitly cancel. However, in Refs. r21 ; r22
, it is shown that if one replaces the coupling $\alpha$ by a parameter $z$
which is effectively the ’t Hooft coupling, renormalization scheme dependence
on the parameters $\beta_{i}$ ($i\geq 2$) is replaced by dependence on a
single parameter $\gamma$. Again, the RG equation can be used to show that all
dependence on $\gamma$ cancels in $R$, just as dependence of $R$ on $\mu$
cancels. In addition, the expansion parameters $\tau_{n}$ in Eq. (40) can be
explicitly shown to be renormalization scheme independent r20 . This is why we
focus on the ’t Hooft coupling when considering the analytic running coupling
in section III.

## References

  * (1) E. C. G. Stueckelberg and A. Petermann, Helv. Phys. Acta 26, 499 (1953).
  * (2) M. Gell-Mann and F. E. Low, Phys. Rev. 95, 1300 (1954).
  * (3) N. N. Bogoliubov and D. V. Shirkov, Nuovo Cim. 3, 845 (1956).
  * (4) S. Weinberg, Phys. Rev. D 8, 3497 (1973).
  * (5) G. ‘t Hooft, Nucl. Phys. B61, 455 (1973).
  * (6) P. M. Lavrov and I. L. Shapiro, JHEP 06, 086 (2013).
  * (7) S. Weinberg, The Quantum Theory of Fields - Vol. 2 (Cambridge University Press, Cambridge, 1996).
  * (8) L. D. Landau, Nucl. Phys. 13, 181 (1959).
  * (9) N. N. Bogoliubov, A. A. Logunov and D. V. Shirkov, Zh. Eksp. Teor. Fiz. 37, 805 (1959).
  * (10) D. V. Shirkov and I. L. Solovtsov, Phys. Rev. Lett. 79, 1209 (1997).
  * (11) D. V. Shirkov, Theor. Math. Phys. 119, 438 (1999).
  * (12) I. L. Solovtsov and D. V. Shirkov, Theor. Math. Phys. 120, 1220 (1999).
  * (13) D. V. Shirkov and I. L. Solovtsov, Theor. Math. Phys. 150, 132 (2007).
  * (14) G. Grunberg, Power corrections and Landau singularity, (hep-ph 9705290).
  * (15) R. Akhoury and V. I. Zakharov, AIP Conf. Proc. 415, 274 (1997).
  * (16) E. Gardi, G. Grunberg and M. Karliner, JHEP 07, 007 (1998).
  * (17) E. Gardi and M. Karliner, Nucl. Phys. B 529, 383 (1998).
  * (18) K. A. Milton, I. L. Solovtsov and O. P. Solovtsova, Phys. Lett. B 415, 104-110 (1997); “Analytic perturbative approach to QCD,” ICHEP 98, (arXiv:hep-ph/9808457).
  * (19) B. A. Magradze, Conf. Proc. C 980518, 158 (1999); arXiv:hep-ph/0010070 (2000).
  * (20) N. Zenine, The analytic running coupling of QCD at two-loop level, Proceedings of the 3rd Nuclear and Particle Physics Conference (NUPPAC-2001).
  * (21) G. M. Prosperi, M. Raciti and C. Simolo, Prog. Part. Nucl. Phys. 58 (2007), 387-438.
  * (22) A. Deur, S. J. Brodsky and G. F. de Téramond, Nucl. Phys. 90 (2016), 1.
  * (23) P. M. Stevenson, Phys. Lett. B 100 (1981), 61-64.
  * (24) P. M. Stevenson, Phys. Rev. D 23 (1981), 2916.
  * (25) G. Grunberg, Phys. Lett. B 95 (1980), 70 [erratum: Phys. Lett. B 110 (1982), 501].
  * (26) G. Grunberg, Phys. Rev. D 29 (1984), 2315-2338.
  * (27) S. J. Brodsky and L. Di Giustino, SLAC-PUB-14425.
  * (28) S. J. Brodsky and L. Di Giustino, Phys. Rev. D 86 (2012), 085026.
  * (29) S. J. Brodsky and G. F. de Teramond, Phys. Rev. Lett. 96, 201601 (2006); Phys. Rev. Lett. 102, 081601 (2009).
  * (30) Z. F. Cui, J. L. Zhang, D. Binosi, F. de Soto, C. Mezrag, J. Papavassiliou, C. D. Roberts, J. Rodríguez-Quintero, J. Segovia and S. Zafeiropoulos, Chin. Phys. C 44 (2020) no.8, 083102.
  * (31) G. ‘t Hooft, Subnucl. Ser. 15, 943 (1979).
  * (32) R. M. Corless, G. H. Gonnet, D. E. G. Hare, D. J. Jeffrey and D. E. Knuth, Adv. Comput. Math. 5, 329-359 (1996).
  * (33) F. Chishtie, D. G. C. McKeon and T. N. Sherry, Phys. Rev. D94, 054031 (2016).
  * (34) I. M. Suslov, On ‘t Hooft’s representation of the $\beta$-function, (hep-ph 0605115).
  * (35) D. Boito, M. Jamin and R. Miravitllas, Phys. Rev. Lett. 117, 152001 (2016).
  * (36) F. A. Chishtie and D. G. C. Mckeon, Can. J. Phys. 99, no.10, 883-888 (2021) [arXiv:2009.08589 [hep-ph]].
  * (37) T. Muta, Foundations of Quantum Chromodynamics (World Scientific, Singapore, 1987).
  * (38) W. Celmaster and R. J. Gonsalves, Phys. Rev. Lett. 44, 560 (1980).
  * (39) M. Ya. Antimirov, Andrei A. Kolyshkin and Rémi Vaillancourt, Complex Variables (Academic Press, 1998).
  * (40) F. A. Chishtie, D. G. C. McKeon and T. N. Sherry, Can. J. Phys. 99, no.8, 622-633 (2021) [arXiv:1708.04219 [hep-ph]].
  * (41) M. Akrami and A. Mirjalili, Phys. Rev. D 101, no.3, 034007 (2020) [arXiv:1912.08938 [hep-ph]].


# Interpretation of immunofluorescence slides by deep learning techniques:
anti-nuclear antibodies case study

Oumar Khlelfa
Military Academy
Fondouk Jedid, Nabeul 8012, Tunisia
<EMAIL_ADDRESS>
Aymen Yahyaoui
Military Academy
Fondouk Jedid, Nabeul 8012, Tunisia
Science and Technology for Defense Lab (STD), Ministry of National Defense,
Tunisia
<EMAIL_ADDRESS>
Mouna Ben Azaiz
Immunology Department, The Principal Military Hospital of Instruction of Tunis
Tunisia
Anwer Ncibi
Military Academy
Fondouk Jedid, Nabeul 8012, Tunisia
Ezzedine Gazouani
Immunology Department, The Principal Military Hospital of Instruction of Tunis
Tunisia
Adel Ammar
Robotics and Internet-of-Things Laboratory, Prince Sultan University, Riyadh,
Saudi Arabia
<EMAIL_ADDRESS>Wadii Boulila
Robotics and Internet-of-Things Laboratory, Prince Sultan University, Riyadh,
Saudi Arabia
RIADI Laboratory, National School of Computer Sciences, University of Manouba,
Manouba, Tunisia
<EMAIL_ADDRESS>
###### Abstract

Nowadays, diseases are increasing in numbers and severity by the hour.
Immunity diseases, affecting 8% of the world population in 2017 according to
the World Health Organization (WHO), is a field in medicine worth attention
due to the high rate of disease occurrence classified under this category.
This work presents an up-to-date review of state-of-the-art immune diseases
healthcare solutions. We focus on tackling the issue with modern solutions
such as Deep Learning to detect anomalies in the early stages hence providing
health practitioners with efficient tools. We rely on advanced deep learning
techniques such as Convolutional Neural Networks (CNN) to fulfill our
objective of providing an efficient tool while providing a proficient analysis
of this solution. The proposed solution was tested and evaluated by the
immunology department in the Principal Military Hospital of Instruction of
Tunis, which considered it a very helpful tool.

_Keywords_ CNN $\cdot$ e-health $\cdot$ immune systems $\cdot$
immunofluorescence slides $\cdot$ antinuclear antibodies $\cdot$ artificial
intelligence $\cdot$ deep learning.

##  1 Introduction

In the current era, the significant spread of chronic diseases is much higher
than ever before, especially those related to immune problems [1]. On this
note, the fact that the rate of affection by immune diseases is increasing
extensively underlines the need to provide a solution for the doctors to help
save time and provide them with precision while treating the
immunofluorescence slides. In fact, the analysis of immunofluorescence [2]
results based solely on the knowledge of a single doctor is a method that
lacks precision, and also could be a significant source of potential
misdiagnosis due to the massive amount of details present in the Indirect
immunofluorescence (IIF) images [3]. This creates dangerous risks for
immunologists working under enormous responsibility and likewise for the
patient, who is the most significant beneficiary. Therefore, the presented
solution is very effective in this matter, as it helps to save time, leading
to faster results; thus, more patients can be treated by a single expert
doctor. On the other hand, the certainty factor also plays a major part in
this study, as the error rate is very minimal thanks to this model, which made
it possible to reach a 94.48% accuracy for diagnosing positive and negative
IIF images. This will lead to another level of performance that will help
newly graduated doctors to go through their medical career with more
confidence based on the automated results. This work is carried out in
collaboration with the immunology department of the Principal Military
Hospital of Instruction of Tunis. The objective is to offer a tool in the form
of a computer application, where the greatest benefits are to effectively
assist the doctors of the immunology department in diagnosing autoimmune
diseases through the analysis of IIF images. This should make interpreting IIF
images much easier in terms of time and level of certainty.

The main contributions of this work are as follows:

  * •

It presents an overview of the different state-of-the-art techniques, methods,
and tools related to AI and the interpretation of immunofluorescence sides.

  * •

It treats the multiple cases of immunofluorescence slides using a combination
of a DL model and a NASNet model leading to accurate results.

  * •

It presents the interactive tool developed for the specific needs of the
medical staff at the military hospital of Tunis immunology department. This
tool was developed, deployed, and tested on many patients. The medical staff
confirmed it as an interesting and useful tool for IIF interpretation.

The science of immunology was revealed in the late 18th century, and from that
moment on, it can be admitted that it gained its rightful recognition as a
branch of knowledge.
Autoimmune diseases, one of the most common issues in immunology, is a disease
that results from a dysfunction of the immune system leading it to attack the
normal constituents of the body: Normally, the immune system preserves the
cells of the body. Whereas during autoimmune disease, it identifies them as
foreign agents and attacks them. Statistics have shown that 5 to 8% of the
world’s population is affected by this type of disease [5].
Immunofluorescence technique: It is a method that consists of detecting and
locating, by fluorescence emission, a protein of interest produced by an
Antigen, with the help of a specific antibody related to the agent in
question. Thus, it makes it possible to determine the presence or absence of a
protein and its localization in the cell or tissue analyzed. In
immunofluorescence, two types of labeling can be carried out as shown in
Figure 1:

  * •

Direct immunofluorescence [4] where the fluorescent protein, called
fluorochrome, is not coupled to a secondary antibody but to the primary
antibody.

  * •

Indirect immunofluorescence [2] where a primary antibody binds to the antigen.
Then, a secondary antibody with a strong affinity for the primary antibody is
added.

Figure 1: Direct vs Indirect Immunofluorescence [16].

##  2 Related Work

Indirect immunofluorescence (IIF) is considered the reference test for the
detection of autoimmune diseases. As a well-established and challenging issue
in the field of medical image analysis, HEp-2 image classification has become
one of the growing centers of interest in the last decade. For this reason,
three international IIF image classification competitions were held in 2012,
2014, and 2016. These competitions were very significant in this progress by
facilitating the collection of a large amount of data. As a classical image
classification problem, traditional machine learning techniques have been
greatly applied to HEp2 image classification.

###  2.1 Machine Learning-based solutions

In this context, several research works have been considered using various
methods such as gradient features with intensity order pooling by [6] who were
capable of reaching 74.39% cell level accuracy and 85.71% image level accuracy
on the ICPR dataset. Also, multiple linear projection descriptors were
considered by [7], who adopt the feature learning method to learn the
appropriate descriptor from the image data itself. They were able to reach
66.6% classification accuracy. Machine Learning (ML) can learn from the data,
while non-learning-based techniques depend on rules that depend critically on
domain knowledge. However, traditional machine learning techniques rely on
predefining feature representations, which is a crucial step and necessitates
complicated engineering [8].

###  2.2 Deep Learning based solutions

Several deep learning-based approaches proposed in the literature have shown
great progress in healthcare for disease diagnosis [9, 10, 11, 13, 12, 14,
15]. Deep learning was considered in two subfields of HEp2 image
classification (HEp2IC) that have attracted the attention of researchers: the
classification of individual HEp-2 cells and the classification of HEp-2
specimens.

####  2.2.1 Cell-level HEp-2 image classification (CL-HEP2IC) methods:

there are two methods for Deep Neural Networks (DNN) to be used; DNN as a
feature extractor and DNN as a classifier. As shown in Figure 2, the amount of
research based on the last method is more important than the other. In fact,
more than 30 publications adopt this method on three of the most popular
datasets (ICPR 2012, I3A, and SNPHEp-2). In contrast, less than ten
publications are adopting the method of using the DNN as a feature extractor.

Figure 2: Number of publications based on the use of DNNs [17].

1) CL-HEp2IC methods that use DNN as feature extractor:
Extraction is frequently used in HE2IC approaches based on handcrafted
features. It allows extracting features from the input image to carry out the
classification performed generally with support vector machines or k-nearest
neighbors. There are two kinds of feature extraction: feature extraction from
a pre-trained DNN model and feature extraction from a fine-tuned DNN model:
First, in [18], authors used a pre-trained CNN model but with an intensity-
aware classification schema with two steps accompanied by an approach to pull
out the features that can differentiate between the classes which allowed them
to reach 77.1% average class accuracy. One year later, authors in [19]
proposed a much simpler idea of replacing the SIFT with the CNN features,
enabling them to achieve a 98% classification accuracy. Second, feature
extraction from a fine-tuned DNN model consists of improving a pre-trained
model. One of the methods in this context involves a modern pooling strategy
to overcome and remove the fixed-size constraint of the CNNs known by resizing
the input images. They used a CaffeNet model to extract features and then a
K-spatial pooling to support the HEp2 cell images with arbitrary sizes. This
method allowed them to reach an accuracy of 98.41% [17]. Unlike all previous
works, authors of [21] used the convolutional auto-encoder (CAE) as a feature
extractor. In fact, it is not only CAE, but they train two different CAEs, one
for normal images with RGB colors and another one for the gradient images,
which are images that have been subject to a directional change in intensity
or color. The first is used to learn the geometric properties of HEp-2 cells,
while the second is used to learn the local intensity changes in HEp-2 cells.
They were able to reach 98.27% accuracy with the SNPHEp-2 [21] dataset and
98.89 with the I3A dataset [21].

2) CL-HEp2IC methods that use DNN as a classifier:
The following methods combine the feature extraction step and the
classification step as one part. In fact, in this kind of method, the features
of the input images are automatically extracted and relying on these features,
the classification phase is achieved.
The cell-level HEp2 image classification that uses DNN as a classifier is
divided into three groups. The following methods combine the feature
extraction step and the classification step as one part. In fact, in this kind
of method, the features of the input images are automatically extracted and
relying on these features, the classification phase is achieved. The cell-
level HEp2 image classification that uses DNN as a classifier is divided into
three groups. Generic DNN-based approaches exploit popular DNN architectures,
primarily designed for general image classification tasks like ImageNet
classification, to perform cell-level HEp2 image classification. The focus on
this type of method began in 2015 with [22] who used an AlexNet, a variant of
CNN used in the competition of image classification of the ImageNet Database
in 2012, reinforced by some pre-processing techniques such as image
enhancement and data augmentation. They were able to achieve 80.3% [22]
accuracy with the ICPR 2012 dataset. A later study conducted by [23]
concerning other generic CNN models, specially ImageNet, GoogleNet, and
LetNet-5, shows that even without any pre-processing methods used in the
procedure, GoogleNet exceeds the performance of the two remaining methods and
gets as far as 95.53% accuracy. In [23], the authors focused on three other
generic CNNs, which are VGG-16, ResNet-50, and InceptionV3. Their work
confirms that this last generic CNN, named InceptionV3 is more efficient in
the HEp2 image classification than the other models considered in their
research without any pre-processing procedures with a 98.28% accuracy [24]. As
mentioned earlier, the generic CNN is strengthened with techniques such as
image enhancing and DA, another idea brought by [25] who suggested to use of
the generative adversarial network (GAN), which generates images with a
different composition than the initial images different from rotating or
reversing, and train generic CNN like the GoogleNet model with those images
provided. It reaches 98.6% accuracy [25]. Despite its high accuracy, this
method shows its weaknesses while facing a large intra-class variation of
data. Second, Generic DNN-based methods with partial changes in layers or
training schemes consist of making smaller changes in the generic model. Two
major studies were shown in this context: The first one is proposed by [26],
who added two additional convolutional layers with a 1x1 filter before the
LeNet5 CNN architecture. It aims to boost the number of feature channels to
facilitate the task of the classification layers to attain a 79.13% [26]
accuracy, which outperforms the accuracy of the basic LeNet-5 by a significant
amount. The second method by the same group of researchers two years later in
2018: they chose to concentrate on the ResNet model and merge the first
layers’ predictions into the final classification layers using a bridging
mechanism to go far as 97.14% accuracy on the ICPR 2012 dataset and 98.42%
[28] accuracy on the I3A dataset. Despite the high accuracy achieved in this
study, it is accompanied by an increasing number of network parameters.

####  2.2.2 Specimen-level HEp-2 image classification (SL-HEP2IC) methods :

The specimen-level methods deal with the images of the HEP2 as an entire block
and classify the whole image. These methods were decomposed into two main
types: single-cell processing-based SL-HEP2IC methods and multi-cell
processing-based methods.

1) Single-cell processing-based SL-HEP2IC methods:
Single-cell processing methods consist of feeding this kind of model with a
specimen image as input. It will be split into singular cell images using
ground truth labeling at the cell level, such as bounding box annotations or
segmentation masks. Then a majority voting strategy is employed to obtain the
specimen-level result by accumulating the cell results. It is considered the
extension of the Cell-level HEp2 image classification. According to the
utilization of DNNs, we can split this part of the related work concerning the
HEp2 image classification into two main parts:

  * •

Feature extraction-based methods: Those methods are much the same as those
previously discussed. They are divided into two parts: feature extraction from
each cell of the specimen and then passed into a classifier to obtain the cell
class. In this context, the authors [26] proposed a modification to the
LeNet-5 model by adding a 1x1 convolutional block. Then, after the cell
classes are identified, a majority voting strategy (MVS) is applied by
choosing the most dominant cell label as the specimen label. This strategy
leads them to reach 95.83% accuracy [26]. Confusion can be present while
applying the majority voting strategy. That is why another approach is used to
overcome such a critical issue: representing the population histogram (PH).

  * •

Pure DNN-based methods: The pure DNN means that the DNN is used as a direct
classifier in this kind of method. They are similar to the methods discussed
previously. One of the studies concerning this type of method is the [26]
study, which used modified LeNet-5 for the classification issue for each cell
image obtained from specimen images and reached a 79.13% accuracy [26].

The drawbacks of the single-cell processing based on specimen level are:

  * •

The requirement of single cell classification to obtain the specimen
classification.

  * •

When numerous cells are presented in a specimen image, the single cell
processing based on specimen level faces many problems while first of all
segmenting and also while classifying specimen-cells.

Figure 3: Level of concordance between two seniors [20].

2) Multi-cell processing-based methods:
Multi-cell processing methods treat a whole specimen image at once. They are
more efficient and can be divided into two types:

  * •

Pixel-wise prediction-based methods: In this prediction, each pixel of the
specimen image is contributed to a class label. Then, the results of these
pixels are fed to a majority voting method to classify the whole specimen. The
basis of this method is the fully convolutional network, a variant of the CNN
that replaces the fully connected layers with convolutional ones.

  * •

Image-wise prediction-based methods: In this type of method, only one
prediction is given to the whole specimen image. A recent method is introduced
in this context which resizes the specimen image. Consequently, the local
information disappears. That is why handcrafted features are used to make the
model stronger, showing a good classification performance that outperforms
some other methods.

##  3 Proposed approach

In this section, the proposed approach is described, and we mention details
about the dataset as well as model architectures.

###  3.1 Immunofluorescence slides interpretation

Medical decisions must be made carefully and with high precision. In this
regard, the detection of the IIF image class is very crucial because of two
factors :

  * •

The need for a double reading of these images, which is not always possible in
many cases.

  * •

Even with the availability of two senior physicians, the difficulty of
analyzing IIF images is always present due to the great detail in this type of
image.

Figure 3 visualizes the level of concordance between two seniors Immunologist
readers. We note that they had, on average, 71% of concordance despite their
expertise in the field.

###  3.2 Global approach process

As shown in Figure 4, the user (who is either the doctor or the technician)
must be able to:

  * •

Identify whether the input image provided is positive or negative in the first
instance.

  * •

Provide prediction of the exact class of the IIF image if it is positive: The
developed application, whose purpose is assisting Immunologists in diagnosing
different cases, must be able to provide predictions with nearly perfect
accuracy of the anomalies present in the patient’s IIF images.

Figure 4: Proposed approach.

###  3.3 Dataset

For Binary classification, the dataset is a mixture of an open-source dataset
called AIDA HEp-2 and a set of IIF images taken from the department of
immunology of the military hospital of Tunis. The AIDA HEp-2 dataset is a part
of the complete AIDA database, resulting from international cooperation
between Tunisia and Italy. In Table 1, the number of patients from whom the
IIF images were taken is mentioned, as well as the positive and the negative
patients. The total number of images acquired is equal to 2080, about 2 images
from each patient.

Table 1: Dataset Description Number of patients | Positive fluorescence intensity | Negative fluorescence intensity
---|---|---
1000 | 470 | 530

For multi-class classification, a dataset containing 2668 IIF images
distributed into 7 classes is used to train, validate and test the model.
Figure 5 describes the representation of how these images are divided.

Figure 5: Distribution of samples over classes.

###  3.4 Model Architecture

Two models were proposed for the prediction of the IIF image class. The first
one is dedicated to the binary classification of the image provided as input:
an IIF image is either positive (presence of antibody) or negative
(otherwise). The model consists of multiple layers, one enveloping the other
so that the output of the first becomes the input of the second and so on,
until the last layer, which is the output of the whole model. In this work,
the first model is sequential, i.e., the output of each layer is the input of
the next layer. It is composed of the layers put in the following arrangement:

  * •

6 sets of Convolutional followed by a Max Pooling layer.

  * •

A Flatten layer (with the shape of 256).

  * •

3 Dense layers (with the respective shapes of 64, 128, and 1).

For the second model, the proposed pre-trained NASNet neural network provides
accurate results. We enhanced the architecture by adding a dense layer to the
model which led to better results.

In this section, the obtained results through the training and testing phases
are presented.

###  3.5 Training

For the binary classification, a model was trained using two classes: positive
and negative. Figure 6 shows a screen capture of results during the training
phase. Then, the performance was evaluated using a range of metrics. Next, the
proposed approach was extended to the multiclass model, which was trained on a
total of 7 different classes, as shown in Figure 7, and again evaluates the
model’s performance using a range of metrics.

####  3.5.1 Binary Model

To investigate the performance of the proposed approach, we began by training
a binary classification model with two classes: positive and negative. The
positive class represented instances of the existence antibody in the cell,
while the negative class represented instances that did not exhibit the
phenomenon. A dataset of 2080 samples for training and testing the model and
evaluating its performance was used.

Figure 6: Training the binary classification model

####  3.5.2 Multi Class Classification

A Multi-class model was trained to distinguish the different interpretable
classes. There are seven classes, as explained previously, and the model was
trained using Google Colab Pro as its training exceeded the locally provided
resources.

Figure 7: Training the multi-class classification model

###  3.6 Testing Results

The first model achieved an accuracy of 94.48% on a challenging classification
task. The model was trained using a convolutional neural network architecture
as explained in the previous section on a dataset of 2080 images. Then it was
evaluated using a standard test set and obtained a precision-recall curve,
which showed that the model’s accuracy was consistent across a range of
thresholds. The results of this study demonstrate the effectiveness of this
approach and provide a foundation for future research.

The second model achieved an accuracy of 69.61% on the multiclass
classification task. The second model was trained using a more complex neural
network architecture with additional layers and a larger number of parameters.
A similar dataset of 2668 images was used for training and testing the second
model. The evaluation of the second model’s performance showed that it had
lower accuracy compared to the first model, which may be due to the lack of
images or the increased complexity of the architecture.

##  4 Conclusion

In this work, an overview of the different state-of-the-art techniques,
methods, and tools related to AI for the Interpretation of Immunofluorescence
slides is depicted. Also, a solution that treats the multiple cases of
immunofluorescence slide interpretation is presented using a combination of a
DL model and a NASNet model leading to accurate results, especially in binary
classification. In future works, a fusion between the two methods, cell level
and specimen level, is a must-try approach to reach better results. In fact,
with the use of the cell-level method, a population diagram representing the
distribution of the cells within the sample can be produced. Then, with the
use of the prediction based on the specimen-level method, a comparison of the
two results will lead to a better verification of the decision-making process.
Finally, we note that AI is a powerful tool in the Immunology field, however,
human beings are exposed to a variety of diseases that target their immunity,
therefore, new AI solutions evolving with theses diseases are required.

## Acknowledgment

The authors would like to thank Prince Sultan University for financially
supporting the conference attendance fees.

## References

  * [1] Bagatini Al., 2018. Immune system and chronic diseases 2018. Journal of immunology research, 2018.
  * [2] Van Hoovels Al. "Variation in antinuclear antibody detection by automated indirect immunofluorescence analysis." Annals of the Rheumatic Diseases 78, no. 6 (2019): e48-e48.
  * [3] Cascio Al. "Deep CNN for IIF images classification in autoimmune diagnostics." Applied Sciences 9, no. 8 (2019): 1618.
  * [4] Jain Al. "Role of direct immunofluorescence microscopy in spectrum of diffuse proliferative glomerulonephritis: A single-center study." Journal of Microscopy and Ultrastructure 9, no. 4 (2021): 177.
  * [5] Olivier Boyer and Sophie Candon, F.:Autoimmune diseases: The breakdown of self-tolerance. https://www.inserm.fr/information-en-sante/dossiers-information/maladies-auto-immunes (2021)
  * [6] Linlin Shen, Jiaming Lin Al.: Hep-2 image classification using intensity order pooling based features and bag of words. Pattern Recognition, 47(7):2419–2427, (2014)
  * [7] Lingqiao Liu and Lei Wang F.: Hep-2 cell image classification with mul- tiple linear descriptors. Pattern Recognition,47(7):2400–24082 (2014)
  * [8] Qawqzeh, Y., Bajahzar, A., Jemmali, M., Otoom, M. & Thaljaoui, A. Classification of diabetes using photoplethysmogram (PPG) waveform analysis: Logistic regression modeling. BioMed Research International. 2020 (2020)
  * [9] Driss, K., Boulila, W., Batool, A. & Ahmad, J. A novel approach for classifying diabetes’ patients based on imputation and machine learning. 2020 International Conference On UK-China Emerging Technologies (UCET). pp. 1-4 (2020)
  * [10] Al-Sarem, M., Alsaeedi, A., Saeed, F., Boulila, W. & AmeerBakhsh, O. A novel hybrid deep learning model for detecting COVID-19-related rumors on social media based on LSTM and concatenated parallel CNNs. Applied Sciences. 11, 7940 (2021)
  * [11] Al-Sarem, M., Saeed, F., Boulila, W., Emara, A., Al-Mohaimeed, M. & Errais, M. Feature selection and classification using CatBoost method for improving the performance of predicting Parkinson’s disease. Advances On Smart And Soft Computing: Proceedings Of ICACIn 2020. pp. 189-199 (2021)
  * [12] Ben Atitallah, S., Driss, M., Boulila, W. & Ben Ghezala, H. Randomly initialized convolutional neural network for the recognition of COVID-19 using X-ray images. International Journal Of Imaging Systems And Technology. 32, 55-73 (2022)
  * [13] Rasool, M., Ismail, N., Boulila, W., Ammar, A., Samma, H., Yafooz, W. & Emara, A. A Hybrid Deep Learning Model for Brain Tumour Classification. Entropy. 24, 799 (2022)
  * [14] Jemmali, M., Melhim, L., Alourani, A. & Alam, M. Equity distribution of quality evaluation reports to doctors in health care organizations. PeerJ Computer Science. 8 pp. e819 (2022)
  * [15] Alam, M., Melhim, L., Ahmad, M. & Jemmali, M. Public attitude towards covid-19 vaccination: validation of covid-vaccination attitude scale (c-vas). Journal Of Multidisciplinary Healthcare. pp. 941-954 (2022)
  * [16] https://www.abcam.com/secondary-antibodies/direct-vs-indirect-immunofluorescence
  * [17] Xian-Hua Han Al. Hep-2 cell classification using k-support spatial pooling in deep cnns. In Deep Learning and Data Labeling for Medical Applications, pages 3–11. Springer, 2016
  * [18] Ha Tran Hong Phan Al: Transfer learning of a convolutional neural network for hep-2 cell image classification. In 2016 IEEE 13th International Symposium on Biomedical Imaging (ISBI), pages 1208–1211. IEEE, (2016)
  * [19] Mengchi Lu Al. Hep- 2 cell image classification method based on very deep convolutional networks with small datasets. In Ninth International Conference on Dig- ital Image Processing (ICDIP 2017), volume 10420, page 1042040. Inter- national Society for Optics and Photonics, (2017)
  * [20] Amel Benammar Elgaaied Al. Computer-assisted classification patterns in autoimmune di- agnostics: the aida project. BioMed research international, (2016).
  * [21] Caleb Vununu, Suk-Hwan Lee, Oh-Jun Kwon, and Ki-Ryong Kwon. A dynamic learning method for the classification of the hep-2 cell images. Electronics, 8(8):850, 2019
  * [22] Neslihan Bayramoglu Al. Human epithelial type 2 cell classification with convolutional neural networks. In 2015 IEEE 15th International Conference on Bioinformatics and Bioengi- neering (BIBE), pages 1–6. IEEE, 2015
  * [23] Larissa Ferreira Rodrigues Al. Hep-2 cell image classification based on convolutional neural networks. In 2017 Workshop of Computer Vision (WVC), pages 13–18. IEEE,2017
  * [24] Larissa Ferreira Rodrigues Al. Comparing convolutional neural networks and preprocessing techniques for hep-2 cell classification in immunofluorescence images. Computers in biology and medicine, 116:103542, 2020
  * [25] Tomáš Majtner Al. On the effectiveness of generative adversarial networks as hep-2 image augmentation tool. In Scandinavian Conference on Image Analysis, pages 439–451. Springer, 2019.
  * [26] Hongwei Li Al. Deep cnns for hep-2 cells classification: A cross-specimen analysis. arXiv preprint arXiv:1604.05816, 2016.
  * [27] Siqi Li, Huiyan Jiang, and Wenbo Pang. Joint multiple fully connected convolutional neural network with extreme learning machine for hepatocellular carcinoma nuclei grading. Computers in biology and medicine, 84:156–167, 2017
  * [28] Haijun Lei, Tao Han, Feng Zhou, Zhen Yu, Jing Qin, Ahmed Elazab, and Baiying Lei. A deeply supervised residual network for hep-2 cell classification via cross-modal transfer learning. Pattern Recognition, 79:290–302, 2018
  * [29] Jingxin Liu, Bolei Xu, Linlin Shen, Jon Garibaldi, and Guoping Qiu. Hep-2 cell classification based on a deep autoencoding-classification convolutional neural network. In 2017 IEEE 14th International Symposium on Biomedical Imaging (ISBI 2017), pages 1019–1023. IEEE, 2017
  * [30] Yuexiang Li and Linlin Shen. A deep residual inception network for hep-2 cell classification. In Deep Learning in Medical Image Analysis and Multimodal Learning for Clinical Decision Support, pages 12–20. Springer, 2017.