# Federating to Grow Transformers with Constrained Resources without Model
Sharing

Shikun Shen
Shandong University
<EMAIL_ADDRESS>
&Yifei Zou
Shandong University
<EMAIL_ADDRESS>
&Yuan Yuan
Shandong University
<EMAIL_ADDRESS>
&Yanwei Zheng
Shandong University
<EMAIL_ADDRESS>
&Peng Li
University of Aizu
<EMAIL_ADDRESS>&Xiuzhen Cheng
Shandong University
<EMAIL_ADDRESS>
&Dongxiao Yu
Shandong University
<EMAIL_ADDRESS>
###### Abstract

The high resource consumption of large-scale models discourages resource-
constrained users from developing their customized transformers. To this end,
this paper considers a federated framework named Fed-Grow for multiple
participants to cooperatively scale a transformer from their pre-trained small
models. Under the Fed-Grow, a Dual-LiGO (Dual Linear Growth Operator)
architecture is designed to help participants expand their pre-trained small
models to a transformer. In Dual-LiGO, the Local-LiGO part is used to address
the heterogeneity problem caused by the various pre-trained models, and the
Global-LiGO part is shared to exchange the implicit knowledge from the pre-
trained models, local data, and training process of participants. Instead of
model sharing, only sharing the Global-LiGO strengthens the privacy of our
approach. Compared with several state-of-the-art methods in simulation, our
approach has higher accuracy, better precision, and lower resource consumption
on computations and communications. To the best of our knowledge, most of the
previous model-scaling works are centralized, and our work is the first one
that cooperatively grows a transformer from multiple pre-trained heterogeneous
models with the user privacy protected in terms of local data and models. We
hope that our approach can extend the transformers to the broadly distributed
scenarios and encourage more resource-constrained users to enjoy the bonus
taken by the large-scale transformers.

##  1 Introduction

In recent years, the transformer with billions of parameters [25] has shown
its high performance on addressing the various complex tasks in the domains of
Natural Language Processing (NLP) [8; 21; 4] and Computer Vision (CV) [9; 18;
24]. However, the training of such large-scale transformers has a high
requirement on the computing, storage, and data resources [4; 2], and
pitilessly reject those resource-constrained users from enjoying the bonus
taken by transformer [35]. Thus, an efficient and resource-friendly approach
to obtain a transformer has been a hot topic in recent years.

Due to its significance, some relevant works have been proposed to reduce the
resource consumption of training a transformer. Some noteworthy strategies
include mixed precision training [22], large batch optimization [30], and
dropping layers [31] or tokens [13]. A common ground of the abovementioned
approaches is that they all train transformers from scratch, which is not
resource-saving if the targeted transformer is a scaled-up version of a pre-
trained one.111For example, the GPT-2 Large model with 774 million parameters
is trained from scratch despite being a scaled-up version of GPT-2 Base with
117 million parameters. Compared with the scratch-based methods, a more
efficient idea is to reuse the smaller models to initialize a larger
transformer, leveraging the implicit knowledge embedded in the pre-trained
models. Based on this idea, a series of works have been proposed [12; 29; 5;
20; 26; 27; 10]. For instance, the Linear Growth Operator (LiGO) in [26] is
used to learn a linear mapping from the parameters of a small model to a large
one, which reduces the computational cost of scratch-based training by up to
50%, and outperforms several model-based baselines. Figure 1 (a) and (b) are
given to illustrate the scratch-based approach and pre-trained model-based
approach, respectively. Even though the existing methods have demonstrated the
effectiveness of reusing smaller pre-trained models to train larger ones, all
of them are centralized under the single-machine environment. The
corresponding problem on the distributed side still requires further
investigation.

In this paper, we study the cooperative training of a transformer with several
participants, each of which has a heterogeneous pre-trained model and is
resource-constrained to expand a transformer individually. For example, the
data of each participant is not sufficient to train a transformer with high
performance by itself. To expand a transformer from its own pre-trained model,
the participants choose to share some useful knowledge with each other, as is
illustrated in Figure 1 (d). Compared with the existing scratch-based (Figure
1 (c)) and centralized model-based approaches, our approach has several
advantages. First, it allows for the integration of diverse knowledge from the
pre-trained models and dataset of different participants, potentially
resulting in a robust and comprehensive large model. Second, it enables
distributed learning for high efficiency, which can significantly accelerate
the training process by utilizing the computational resources of multiple
users. Third, it offers an alternative approach for the resource-constrained
users to cooperatively train a transformer.

Figure 1: Illustration of different methods for efficient model training. The
upper left subfigure shows the conventional methods of training a large model
from scratch. The lower left subfigure shows the model reusing methods of
exploiting a small pre-trained model. The upper right figure shows the
conventional federated learning (FL) approach. The lower right subfigure shows
our proposed framework: Fed-Grow with Dual-LiGO.

Even though the cooperative training of transformers has such advantages,
designing a distributed training algorithm is not an easy task. The biggest
challenge comes from the model heterogeneity, i.e., each participant may have
different architectures, sizes, and parameters of their pre-trained models.
Additionally, the participants may be unwilling to share their data and even
pre-trained models with others, since the models are also the property.
Solving the model heterogeneity problem under the constraint of privacy
protection requires a deep thinking on the following question: What should be
shared in the collaborative training process for knowledge integration,
distributed training acceleration, and privacy protection?

To answer this question, we introduce Fed-Grow framework with Dual-LiGO
architecture. In Fed-Grow, multiple participants cooperatively grow a
transformer from their pre-trained small models. The Dual-LiGO architecture
enables the extension of heterogeneous small pre-trained models to large
models in a federated framework and is based on the idea of LiGO (Linear
Growth Operator) from [26] in a centralized environment. The key idea of Dual-
LiGO is to design two LiGO operators: Local-LiGO and Global-LiGO. In the
first, each participant uses the Local-LiGO to expand its heterogeneous pre-
trained model to a homogeneous intermediate one. Then, the Global-LiGO is used
by each participant to expand its intermediate model to a homogeneous
transformer. In each local training process, gradient descent method is used
to update the Local-LiGO and Global-LiGO. After multiple times local training,
the Global-LiGO will be shared with others to achieve distributed training
acceleration. Such a local training and federating process will be repeated
for sufficient times until a transformer with high performance is obtained.
Since only the Global-LiGO is shared, the local data and pre-trained model can
be rarely recovered by the others, which addresses the privacy concerns in
collaborative learning. Our main contributions are summarized as follows:

  1. 1.

We introduce Fed-Grow, a federated framework for multiple users to grow a
transformer from their heterogeneous pre-trained models. To the best of our
knowledge, this is the first work tackling the transformer expansion problem
under a distributed and heterogeneous setting. Compared with the existing
centralized works requiring sufficient data and computing resources on a
single machine, Fed-Grow makes full use of the fragmented resources and the
pre-trained models of multiple participants to cooperatively grow a
transformer. With our Fed-Grow, those resource-constrained users who cannot
afford a transformer individually can also enjoy the bonus taken by
transformers.

  2. 2.

Under our Fed-Grow framework, the Dual Linear Growth Operator (Dual-LiGO) is
designed for each participant to expand its pre-trained small model to a
transformer. Specifically, the Dual-LiGO consists of two parts: Local-LiGO and
Global-LiGO, both of which are trained locally but only the latter one will be
shared. With the help of Global-LiGO, the participants share their knowledge
in model expansion, which speeds up the training process and improves the
final performance of the transformer. The Local-LiGO is designed to address
the heterogeneity problem caused by pre-trained models with various
architectures and sizes. Since the Local-LiGO is not shared, the adversary
lacks the complete knowledge to recover the local data and pre-trained models
of participants, which addresses the privacy concerns in federated learning.

  3. 3.

Extensive experiments are conducted to validate the effectiveness of our Fed-
Grow framework with Dual-LiGO on multiple datasets. Compared with the existing
method [26] in stand-alone settings, Dual-LiGO boosts the accuracy or
precision by average 52.43%. Compared with the conventional federated learning
that directly trains a transformer from scratch [24; 8], Dual-LiGO reduces the
trainable parameters by 59.25% and the communication cost by 73.01%, achieving
higher training efficiency and lower resource consumption on computation and
communication.

Roadmap: Section 2 reviews the related works. The problem setup and our Fed-
Grow framework with Dual-LiGO are presented in section 3. Section 4 reports
the numerical results in simulation. In Section 5, we conclude this paper and
discuss the future work.

##  2 Related Work

The efficient training of transformers has been a topic of interest in recent
years. Initial strategies include mixed precision training [22], large batch
optimization [30], progressive layer dropping [31], token dropping [13],
sparse training [6], and knowledge inheritance [20], which aim to lower the
computational cost, memory usage, and communication overhead of training a
transformer, while preserve or even improve the performance of transformer on
downstream tasks. A common ground of these strategies is that they all train
large models from scratch, thereby discarding the valuable knowledge acquired
by their smaller counterparts. Recognizing this, recent research has also
focused on reusing small pre-trained models to initialize a large model before
regular training, which leverages the implicit knowledge of the small model to
improve the convergence speed of the large model. For example, StackBERT [12]
doubles the depth of BERT model by replicating the parameters of a small
model. MSLT [29] starts with a pre-trained small model and gradually increases
the depth of the model by adding new encoder layers. Compared with the above
two depth-only works, Width-and-depth methods [7; 5; 26; 10] increase both the
width and the depth of the model by adding more neurons and layers to the pre-
trained small model. In detail, Net2Net [7] expands the width of neural
networks by randomly copying neurons while preserving output values via
normalization. In [5], bert2BERT learns linear mappings to scale the pre-
trained model by minimizing the task loss during model expansion. LiGO [26]
learns to grow pre-trained transformers by factorizing the linear
transformation as a composition of width- and depth-growth operators. TripLe
[10] partially scales a model before training, while growing the rest of the
new parameters during training by copying both the warmed-up weights with the
optimizer states from existing weights. By leveraging the implicit knowledge
of the pre-trained small models, these model-reusing methods improve the
training efficiency of large models, saving computational cost and resources.

Despite these advancements, a significant research gap remains. Most of these
approaches are centralized and require considerable data and computational
resources on a single machine. For example, 64 dragonfish TPUs and ImageNet-1k
are required in TripLe [10], which prevents those resource-constrained users
from discovering the delights of large models. Furthermore, there is a lack of
relevant research in distributed scenarios. So, will it be possible for
multiple resource-constrained users to cooperatively expand a transformer from
their pre-trained small models is still an open question in the existing
research.

##  3 Fed-Grow with Dual-LiGO

In this section, we provide the problem setup and an overview of our approach
Fed-Grow with Dual-LiGO, in which multiple resource-constrained participants
collaboratively scale a transformer from their pre-trained small models.
During this process, the data and pre-trained models of the participants are
not shared, which enhances the privacy but also makes the design harder.

###  3.1 Problem Setup

We consider a setup with $n$ participants (also termed as clients in federated
learning), and a parameter server to communicate with the clients for
knowledge sharing. Importantly, the server has no access to private data or
pre-trained small models of the clients, thereby ensuring privacy in federated
learning. Each client $i$ has its own data
$\mathcal{D}_{i}=\\{x_{i}\in\mathbb{R}^{d},y_{i}\in\mathbb{R}\\}_{i=1}^{|\mathcal{D}_{i}|}$
and a pre-trained small model $\Theta_{i}^{(small)}$. Considering that the
participants in Fed-Grow may come from different institutions, we assume that
the data on each client is not independently and identically distributed (non-
IID), to reflect the realistic and diverse data distributions across different
clients. Besides, the pre-trained small models of clients are heterogeneous,
which means the number of layers and dimensions of those small models are
various. Following the notations of [26], we denote the parameters of a neural
network with $L$ layers and $D$ dimensions as
$\Theta_{L,D}\in\mathbb{R}^{LD\times D}$. The LiGO in the original work [26]
learns a linear mapping $\mathbf{M}:\mathbb{R}^{L_{1}D_{1}\times
D_{1}}\to\mathbb{R}^{L_{2}D_{2}\times D_{2}}$ from the parameters of a smaller
pre-trained model $\Theta_{L_{1},D_{1}}$ to initialize a larger one
$\Theta_{L_{2},D_{2}}$, i.e.,
$\Theta_{L_{2},D_{2}}=\mathbf{M}(\Theta_{L_{1},D_{1}}$) where $L_{1}<L_{2}$
and $D_{1}<D_{2}$. LiGO effectively reuses the knowledge from the smaller
model to accelerate the training of the larger model.

The goal of clients in Fed-Grow is to utilize their local model
$\Theta_{i}^{(small)}$ to collaboratively scale a transformer
$\Theta^{(large)}$ by sharing useful experience or knowledge with others. Let
$\ell(\Theta;x,y)$ denote the loss function for given model $\Theta$ and data
point $(x,y)$, such as the cross-entropy loss and the mean-squared loss. The
underlying optimization goal of our Fed-Grow can be formalized as follows:

$\underset{\Theta^{(large)}}{\min}\left\\{F(\Theta^{(large)}):=\sum_{i=1}^{n}\frac{|\mathcal{D}_{i}|}{\sum_{j=1}^{n}|\mathcal{D}_{j}|}F_{i}(\Theta^{(large)})\right\\}$
(1)

where
$F_{i}(\Theta^{(large)})=\mathbb{E}_{(x,y)\sim\mathcal{D}_{i}}[\ell(\Theta^{(large)};x,y)]$
denotes the average loss of $\Theta^{(large)}$ over local dataset
$\mathcal{D}_{i}$. The objective of the optimization problem is to find
$\Theta^{(large)}$ that minimizes the weighted sum of the average losses
overall local datasets, which ensures that the large model performs well on
all clients’ data, taking into account the size of each client’s dataset.

The important notations in problem setup and our method are presented in
Appendix A.1.

###  3.2 Overview of Our Method

Building upon the problem setup, we now turn our attention to the challenges
posed by the heterogeneous model settings in a multi-user collaborative
scenario. Specifically, we need to figure out what to share in the federated
learning framework, to guarantee efficiency and privacy simultaneously.

To address these challenges, we introduce the Dual-LiGO architecture. The key
idea behind Dual-LiGO is to leverage two operators, the Local-LiGO and the
Global-LiGO, to scale a homogeneous global model from multiple heterogeneous
pre-trained models.

In the first step, the Local-LiGO is designed to handle the heterogeneity of
the small models. It expands each client’s small model into a homogeneous
intermediate one. This process ensures that all intermediate models have the
same architecture, which is crucial for knowledge sharing in the next step.

In the second step, the Global-LiGO takes the homogeneous intermediate models
and expands them into a homogeneous large model. This operator is learned
collaboratively by all clients, ensuring that the knowledge from all small
models is effectively integrated into the large model without compromising the
privacy of the pre-trained small model stored in each client.

Through this two-step process, Dual-LiGO can effectively aggregate
heterogeneous small models into a unified large model, while preserving the
privacy of each client with respect to its small model and local data.
Furthermore, by reusing the knowledge from the small models, Dual-LiGO
accelerates the training process and enhances the final performance of the
large model.

###  3.3 Dual-LiGO Architecture

The Dual-LiGO architecture consists of two main components: the Local-LiGO and
the Global-LiGO.

Local-LiGO. The Local-LiGO tackles the heterogeneity of the small models. For
each client $i$, it expands the pre-trained small model $\Theta_{i}^{(small)}$
into a homogeneous intermediate model $\Theta_{i}^{(inter)}$ that has the same
architecture across all clients. Let’s denote the intermediate model as
$\Theta_{L_{inter},D_{inter}}\in\mathbb{R}^{L_{inter}D_{inter}\times
D_{inter}}$. The Local-LiGO in each client learns a linear mapping
$\mathbf{M}_{i}^{(local)}:\mathbb{R}^{L_{i}D_{i}\times
D_{i}}\to\mathbb{R}^{L_{inter}D_{inter}\times D_{inter}}$ from the pre-trained
small model $\Theta_{i}^{(small)}$ to initialize the intermediate model
$\Theta_{i}^{(inter)}$, i.e.,

$\displaystyle\Theta_{i}^{(inter)}=\mathbf{M}_{i}^{(local)}(\Theta_{i}^{(small)}).$
(2)

The optimization goal of the Local-LiGO is to minimize the loss on the local
dataset under the constraint that the parameters of the small model are fixed.
This can be formalized as follows:

$\underset{\mathbf{M}_{i}^{(local)}}{\min}J(\mathbf{M}_{i}^{(local)}):=\left\\{\mathbb{E}_{(x,y)\sim\mathcal{D}_{i}}[\ell(\mathbf{M}_{i}^{(local)}(\Theta_{i}^{(small)});x,y)]\right\\}$
(3)

To achieve this goal, each client updates the $\mathbf{M}_{i}^{(local)}$ using
the following gradient descent rule:

$\mathbf{M}_{i}^{(local)}\leftarrow\mathbf{M}_{i}^{(local)}-\eta_{l}\nabla_{\mathbf{M}_{i}^{(local)}}J(\mathbf{M}_{i}^{(local)})$
(4)

where $\eta_{l}$ is the learning rate for updating $\mathbf{M}_{i}^{(local)}$.

Global-LiGO. The Global-LiGO scales the homogeneous intermediate models to a
homogeneous large model. It learns a linear mapping
$\mathbf{M}^{global}:\mathbb{R}^{L_{inter}D_{inter}\times
D_{inter}}\to\mathbb{R}^{L_{large}D_{large}\times D_{large}}$ from the
parameters of the intermediate model $\Theta_{L_{inter},D_{inter}}$ to
initialize the large model $\Theta_{i}^{(large)}$, i.e.,

$\displaystyle\Theta_{i}^{(large)}=\mathbf{M}_{i}^{(global)}(\Theta_{i}^{(inter)}).$
(5)

The Global-LiGO is what we actually share in each communication round. By
sharing this operator instead of the local models themselves, we can ensure
the privacy of each client with respect to its pre-trained model and local
data. Similar to Local-LiGO, the optimization goal of Global-LiGO for each
client can be formulated as follows:

$\underset{\mathbf{M}_{i}^{(global)}}{\min}J(\mathbf{M}_{i}^{(global)}):=\left\\{\mathbb{E}_{(x,y)\sim\mathcal{D}_{i}}[\ell(\mathbf{M}_{i}^{(global)}(\Theta_{i}^{(inter)});x,y)]\right\\}$
(6)

Likewise, each client updates the $\mathbf{M}_{i}^{(global)}$ using the
following gradient descent rule:

$\mathbf{M}_{i}^{(global)}\leftarrow\mathbf{M}_{i}^{(global)}-\eta_{g}\nabla_{\mathbf{M}_{i}^{(global)}}J(\mathbf{M}_{i}^{(global)})$
(7)

where $\eta_{g}$ is the learning rate for updating
$\mathbf{M}_{g}^{(global)}$.

The workflow of Dual-LiGO is illustrated in Figure 2, and its pseudocode is
given in Appendix A.2. For each client, it firstly train the
$\mathbf{M}_{i}^{(local)}$ locally for $E_{l}$ epochs, and get the
intermediate homogeneous model via $\mathbf{M}_{i}^{(local)}$. Next, it trains
Global-LiGO locally for $E_{g}$ epochs and sends the Global-LiGO to the
parameter server for aggregation. The aggregated Global-LiGO will be used as
the starting point of the next $E_{g}$ epochs local training. Such a local
training and aggregation process will be repeated for $R$ times. Finally,
based on the intermediate model and Global-LiGO, each client gets a
homogeneous transformer.

Figure 2: Workflow of Dual-LiGO.

##  4 Experiments

In this section, we first introduce the experiment setup and then show the
numerical results demonstrating the effectiveness of our Fed-Grow with Dual-
LiGO.

###  4.1 Datasets and Models

We consider the following 3 tasks: text classification, sequence tagging and
image classification. For text classification tasks, we use the 20 Newsgroups
[15] and AG News datasets [32], employing the BERT [8] model. For sequence
tagging tasks, we use the WikiNER dataset [23], also utilizing the BERT [8]
model. For image classification tasks, we use the CIFAR-10 [14], CIFAR-100
[14], and Flowers-102 [19] datasets, using the Vision Transformer (ViT) [9]
model. The detailed description about the dataset is available in the Appendix
B.1

For each client’s small model, i.e., $\Theta^{(small)}$, we design three
different sizes to simulate the heterogeneous setting of the small models.
Details about the sizes of small models $\Theta^{(small)}$, intermediate
models $\Theta^{(inter)}$, and large models $\Theta^{(large)}$ are provided in
Appendix B.2.

Table 1: Comparison of accuracy/precision and standard deviation between Fed-
Grow (Agg) and NoAgg [26] under different settings.

|  |  Accuracy/Precision (%) $\uparrow$
---|---|---
|  | 10 Clients | 20 Clients
|  | IID | Non-IID | IID | Non-IID
Model | Task | Agg | NoAgg | $\Delta$ | Agg | NoAgg | $\Delta$ | Agg | NoAgg | $\Delta$ | Agg | NoAgg | $\Delta$
| 20news | 65.295 | 49.177 | 16.118 | 53.983 | 37.759 | 16.224 | 48.258 | 45.470 | 2.788 | 27.787 | 18.693 | 9.094
| agnews | 90.158 | 87.474 | 2.684 | 85.671 | 73.079 | 12.592 | 86.053 | 86.737 | -0.684 | 84.250 | 49.303 | 34.947
BERT | wikiner | 42.581 | 28.324 | 14.257 | 43.878 | 33.539 | 10.339 | 43.693 | 28.803 | 14.890 | 42.558 | 27.300 | 15.258
| CIFAR-10 | 76.717 | 67.977 | 8.740 | 70.186 | 48.771 | 21.415 | 78.223 | 70.020 | 8.203 | 71.914 | 53.730 | 18.184
| CIFAR-100 | 53.178 | 38.127 | 15.051 | 50.109 | 26.904 | 23.205 | 54.004 | 39.141 | 14.863 | 51.270 | 28.457 | 22.813
ViT | Flowers102 | 38.555 | 25.000 | 13.555 | 37.928 | 20.326 | 17.602 | 31.548 | 12.448 | 19.100 | 31.904 | 10.349 | 21.555
|  |  Standard Deviation $\downarrow$
|  | 10 Clients | 20 Clients
|  | IID | Non-IID | IID | Non-IID
Model | Task | Agg | NoAgg | $\Delta$ | Agg | NoAgg | $\Delta$ | Agg | NoAgg | $\Delta$ | Agg | NoAgg | $\Delta$
| 20news | 3.546 | 24.219 | -20.673 | 11.960 | 23.161 | -11.201 | 3.518 | 14.605 | -11.087 | 3.518 | 6.112 | -2.594
| agnews | 1.001 | 4.362 | -3.361 | 4.350 | 28.025 | -23.675 | 2.100 | 1.573 | 0.527 | 14.605 | 16.340 | -1.735
BERT | wikiner | 3.513 | 9.069 | -5.556 | 1.360 | 11.288 | -9.928 | 2.441 | 9.689 | -7.248 | 2.241 | 11.109 | -8.868
| CIFAR-10 | 1.482 | 2.908 | -1.426 | 1.870 | 5.168 | -3.298 | 1.251 | 2.280 | -1.029 | 1.321 | 5.225 | -3.904
| CIFAR-100 | 1.045 | 1.848 | -0.803 | 1.332 | 3.099 | -1.767 | 1.401 | 1.795 | -0.394 | 1.551 | 2.282 | -0.731
ViT | Flowers102 | 1.419 | 1.657 | -0.238 | 1.399 | 2.609 | -1.210 | 1.467 | 2.305 | -0.838 | 1.038 | 1.638 | -0.600

(a) 20news

(b) agnews

(c) wikiner

(d) CIFAR-10

(e) CIFAR-100

(f) Flowers-102

Figure 3: Comparison of Dual-LiGO (Agg) and NoAgg on six datasets under IID
and non-IID settings with 10 clients. The subfigures show the mean accuracy
(or precision) of the methods on each dataset. Each subfigure shows four
lines: IID_AGG, IID_NoAgg, NIID_AGG, NIID_NoAgg, which represent the test
accuracy (or precision) of the methods under different settings.

###  4.2 Results Overview

Baselines. We compare our proposed method, Fed-Grow with Dual-LiGO, against
two baseline methods. The first one is each client’s own expansion of the
large model (without aggregation) [26], denoted as "NoAgg" in the tables. This
baseline represents the scenario where each client uses its own pre-trained
model to initialize a large model and trains it locally without any
communication or knowledge sharing with other clients. We compare this
baseline with our method on the top-1 accuracy or precision metrics, to show
that our Fed-Grow framework really works on improving accuracy, precision, and
training speed. The second one is the conventional federated learning that
directly trains a large transformer from scratch [9; 8], denoted as "Scratch"
in the tables. This baseline represents the scenario where each client starts
with a randomly initialized large model and trains with federated averaging on
the large model. We compare this baseline with our method on the number of
trainable parameters and communication cost metrics, to show that our Dual-
LiGO architecture is resource saving than the full model sharing.

Performance Analysis. We test our models with different numbers of clients,
specifically 10 and 20 clients. We analyze the performance of our Fed-Grow
from three aspects: accuracy/precision, stability, and resource consumption.
In Table 1 and Figure 3, we compare our method with the NoAgg method on the
accuracy and stability metrics. The results on the number of trainable
parameters and communication cost metrics are shown in Table 2, which compares
our method with the Scratch method. We also show the average test
accuracy/precision over communication rounds under different settings in
Appendix C.1.

Fed-Grow improves accuracy/precision. Our proposed method, Fed-Grow with Dual-
LiGO, generally outperforms the baseline method NoAgg [26] in terms of
accuracy and precision across the absolute majority of datasets, both under
IID and non-IID conditions, and for both 10 and 20 clients. This is indicated
by the positive $\Delta$ values in the accuracy/precision part of Table 1,
which represent the improvement in accuracy achieved by Fed-Grow over the
baseline method. There is only 1 exception to be reported. In the agnews
dataset under IID conditions with 20 clients, the NoAgg method achieves a
slightly higher accuracy (86.737) than Fed-Grow (86.053) and a lower standard
deviation. This could be attributed to the relatively simple nature of the
agnews dataset, which has only 4 classes of data. In the IID setting, every
client has enough amount of data for training a robust model, which might have
given the NoAgg method an advantage. The above results suggest that Fed-Grow
with Dual-LiGO can effectively integrate the diverse knowledge from different
participants and achieve a robust and comprehensive large model.

Fed-Grow reduces the standard deviation of clients’ performance. In terms of
stability, measured by the standard deviation of accuracy/precision across
clients, Dual-LiGO also appears to perform better than NoAgg. This is
indicated by the negative $\Delta$ values in the standard deviation part of
Table 1, which suggests that Fed-Grow with Dual-LiGO has a lower standard
deviation, and hence more stable performance, compared to the baseline method.
This suggests that Dual-LiGO can reduce the performance gap among different
clients and ensure a consistent quality of the model.

Table 2: Comparison on number of trainable parameters and communication cost
between Dual-LiGO and Scratch [8; 9]. The table shows the results for
different components. The communication cost is measured by the number of
parameters that need to be transmitted between each device and the server in
each training round.

Method | Model | # Trainable Params | Comm Cost
---|---|---|---
| Small | 0 | 0
| Local-LiGO | 0.737M | 0
Dual-LiGO#1 | Global-LiGO | 2.089M | 4.178M
| Small | 0 | 0
| Local-LiGO | 1.065M | 0
Dual-LiGO#2 | Global-LiGO | 2.089M | 4.178M
| Small | 0 | 0
| Local-LiGO | 1.393M | 0
Dual-LiGO#2 | Global-LiGO | 2.089M | 4.178M
Scratch | Large | 7.740M | 15.480M
Averaged result of Dual-LiGO | 3.154M | 4.178M
Improvements on Scratch | 59.25% | 73.01%

Dual-LiGO reduces the resource consumption on computation and communication.
As shown in Table 2, Dual-LiGO has fewer trainable parameters and lower
communication cost than Scratch. Specifically, Fed-Grow with Dual-LiGO reduces
the number of trainable parameters and the communication cost by 59.25% and
73.01%, respectively, which indicates that our Fed-Grow with Dual-LiGO
requires less computational resources and transfers less data in the training
process than Scratch. This can reduce the hardware thresholds for the
participants, save bandwidth and energy cost in training, and also improve the
privacy and security of the data since less parameters are shared. These
results demonstrate that our Fed-Grow with Dual-LiGO is more efficient and
resource-friendly than Scratch.

In summary, our experimental results have shown that our method has three main
advantages over the baseline methods. First, Fed-Grow with Dual-LiGO improves
the accuracy and precision of the model across various tasks and conditions,
indicating higher performance. Second, Fed-Grow with Dual-LiGO enhances the
stability of the model’s performance across different clients, indicating more
consistency. Third, Fed-Grow with Dual-LiGO reduces the resource consumption
on computation and communication, indicating lower threshold and more green
for training a transformer. These findings suggest that our Fed-Grow with
Dual-LiGO is a promising approach, especially for those users with limited
data and computing resources.

###  4.3 Detailed Analysis

In this part, we analyze the detailed performance of Fed-Grow with Dual-LiGO
under different client settings. Additionally, the resource consumption of
Fed-Grow with Dual-LiGO on computation and communication are also observed.

Table 3: Mean accuracy of Dual-LiGO (Agg) and NoAgg on the CIFAR-100 dataset
under IID and non-IID settings with different numbers of clients.

|  Accuracy (%) $\uparrow$
---|---
| IID | Non-IID
Client Number | Agg | NoAgg | $\Delta$ | Agg | NoAgg | $\Delta$
10 | 53.178 | 38.127 | 15.051 | 50.109 | 26.904 | 23.205
20 | 54.004 | 39.141 | 14.863 | 51.270 | 28.457 | 22.813
30 | 47.57 | 31.341 | 16.229 | 43.727 | 21.363 | 22.364
40 | 42.061 | 28.499 | 13.562 | 37.209 | 18.48 | 18.729
50 | 37.478 | 32.854 | 4.624 | 25.922 | 17.122 | 8.800

Table 4: Mean accuracy of Dual-LiGO (Agg) and NoAgg on the agnews dataset
under IID and non-IID settings with different numbers of clients.

|  Accuracy (%) $\uparrow$
---|---
| IID | Non-IID
Client Number | Agg | NoAgg | $\Delta$ | Agg | NoAgg | $\Delta$
10 | 90.158 | 87.474 | 2.684 | 85.671 | 73.079 | 12.592
20 | 86.053 | 86.737 | -0.684 | 84.250 | 49.303 | 34.947
30 | 88.000 | 87.197 | 0.803 | 87.632 | 50.263 | 37.369
40 | 87.329 | 85.737 | 1.592 | 82.961 | 38.579 | 44.382
50 | 70.232 | 66.484 | 3.748 | 68.126 | 34.137 | 33.989

Different Number of Clients. We conduct a robustness analysis to test the
performance of Fed-Grow with Dual-LiGO under different client settings. We use
two datasets, CIFAR-100 and agnews, and test Fed-Grow under both IID and non-
IID data distributions. We vary the number of clients from 10 to 50. The
results are shown in Tables 3 and 4 and the accuracy curves are provided in
Appendix C.1, which show that Fed-Grow with Dual-LiGO is robust to different
client settings, as it consistently outperforms baseline in both IID and Non-
IID settings, except for one case where baseline has slightly higher accuracy
than Fed-Grow when the number of clients is 20 in the IID setting of agnews.
This has been explained in Section 4.2. In most cases, Fed-Grow has a clear
advantage over baseline, especially in the Non-IID setting, where the gap is
very large. The largest improvement of Fed-Grow with Dual-LiGO over baseline
is achieved when the number of clients is 40, with 1.592% and 44.382% higher
accuracy in IID and Non-IID settings of agnews, respectively. The smallest
improvement is observed when the number of clients is 20, with -0.684% and
34.947% higher accuracy in IID and Non-IID settings of agnews, respectively.
This demonstrates that a client with Dual-LiGO can effectively leverage the
information from other clients to improve the performance of its transformer,
while the baseline suffers from limited and heterogeneous data. The numerical
results shows that Dual-LiGO is robust to data heterogeneity and sparsity and
can adapt to different client settings.

Detailed resource consumption on Dual-LiGO.

In our approach, each client only has to store its small model, and train the
Local-LiGO and Global-LiGO to obtain its transformer. Since there are three
kind of small models (#1, #2, #3) in our simulation, the Dual-LiGO also vary
on clients with small models #1, #2, #3 with respect to trainable parameters
and communication cost. The observed trainable parameters and communication
cost for the Dual-LiGO#1, Dual-LiGO#2, Dual-LiGO#3 are presented in the Table
2, from which we can see that the average trainable parameters and
communication cost for Dual-LiGO are 3.154M and 4.178M, respectively. Compared
with the results $7.740$M and $15.480$M for Scratch, our Dual-LiGO has
$59.25\%$ and $73.01\%$ improvements on trainable parameters and communication
cost.

##  5 Conclusion

To make the resource-constrained users can also afford the costly-training
transformers, this paper studies a federated framework named Fed-Grow, in
which multiple participants cooperatively expand a large-scale transformer
from their pre-trained small models. To efficiently exchange the useful
knowledge and protect the privacy of users in model scaling, the Dual-LiGO
architecture is designed based on Fed-Grow, with its Local-LiGO part used to
tackle the heterogeneity problem caused by the pre-trained models, and its
Global-LiGO part shared by participants to exchange the implicit knowledge
from the pre-trained models and the experience learned in model scaling.
Compared with the classical federated learning, our approach no longer shares
the models, let alone the raw data, which strengthens the privacy of our
approach. The numerical results from the simulation show that our approach
outperforms several baselines in terms of accuracy, precision, stability, and
resource consumption on computations and communications. Extending our work to
train customized transformers with heterogeneous architecture will be our work
in the future.

## References

  * [1]
  * Achiam et al. [2023] Josh Achiam, Steven Adler, Sandhini Agarwal, Lama Ahmad, Ilge Akkaya, Florencia Leoni Aleman, Diogo Almeida, Janko Altenschmidt, Sam Altman, Shyamal Anadkat, et al. 2023\.  GPT-4 Technical Report.  arXiv:2303.08774 [cs.CL]
  * Beutel et al. [2020] Daniel J Beutel, Taner Topal, Akhil Mathur, Xinchi Qiu, Javier Fernandez-Marques, Yan Gao, Lorenzo Sani, Hei Li Kwing, Titouan Parcollet, Pedro PB de Gusmão, and Nicholas D Lane. 2020.  Flower: A Friendly Federated Learning Research Framework.  _arXiv preprint arXiv:2007.14390_ (2020).
  * Brown et al. [2020] Tom Brown, Benjamin Mann, Nick Ryder, Melanie Subbiah, Jared D Kaplan, Prafulla Dhariwal, Arvind Neelakantan, Pranav Shyam, Girish Sastry, Amanda Askell, et al. 2020\.  Language models are few-shot learners.  _Advances in neural information processing systems_ 33 (2020), 1877–1901.
  * Chen et al. [2021b] Cheng Chen, Yichun Yin, Lifeng Shang, Xin Jiang, Yujia Qin, Fengyu Wang, Zhi Wang, Xiao Chen, Zhiyuan Liu, and Qun Liu. 2021b.  bert2bert: Towards reusable pretrained language models.  _arXiv preprint arXiv:2110.07143_ (2021).
  * Chen et al. [2021a] Tianlong Chen, Yu Cheng, Zhe Gan, Lu Yuan, Lei Zhang, and Zhangyang Wang. 2021a.  Chasing Sparsity in Vision Transformers: An End-to-End Exploration.  arXiv:2106.04533 [cs.CV]
  * Chen et al. [2015] Tianqi Chen, Ian Goodfellow, and Jonathon Shlens. 2015.  Net2net: Accelerating learning via knowledge transfer.  _arXiv preprint arXiv:1511.05641_ (2015).
  * Devlin et al. [2018] Jacob Devlin, Ming-Wei Chang, Kenton Lee, and Kristina Toutanova. 2018.  Bert: Pre-training of deep bidirectional transformers for language understanding.  _arXiv preprint arXiv:1810.04805_ (2018).
  * Dosovitskiy et al. [2020] Alexey Dosovitskiy, Lucas Beyer, Alexander Kolesnikov, Dirk Weissenborn, Xiaohua Zhai, Thomas Unterthiner, Mostafa Dehghani, Matthias Minderer, Georg Heigold, Sylvain Gelly, et al. 2020\.  An image is worth 16x16 words: Transformers for image recognition at scale.  _arXiv preprint arXiv:2010.11929_ (2020).
  * Fu et al. [2023] Cheng Fu, Hanxian Huang, Zixuan Jiang, Yun Ni, Lifeng Nai, Gang Wu, Liqun Cheng, Yanqi Zhou, Sheng Li, Andrew Li, et al. 2023\.  TripLe: Revisiting Pretrained Model Reuse and Progressive Learning for Efficient Vision Transformer Scaling and Searching. In _Proceedings of the IEEE/CVF International Conference on Computer Vision_. 17153–17163.
  * Gao et al. [2024] Mengtong Gao, Yifei Zou, Zuyuan Zhang, Xiuzhen Cheng, and Dongxiao Yu. 2024.  Cooperative Backdoor Attack in Decentralized Reinforcement Learning with Theoretical Guarantee.  _arXiv preprint arXiv:2405.15245_ (2024).
  * Gong et al. [2019] Linyuan Gong, Di He, Zhuohan Li, Tao Qin, Liwei Wang, and Tieyan Liu. 2019.  Efficient training of BERT by progressively stacking. In _International conference on machine learning_. PMLR, 2337–2346.
  * Hou et al. [2022] Le Hou, Richard Yuanzhe Pang, Tianyi Zhou, Yuexin Wu, Xinying Song, Xiaodan Song, and Denny Zhou. 2022.  Token dropping for efficient BERT pretraining.  _arXiv preprint arXiv:2203.13240_ (2022).
  * Krizhevsky et al. [2009] Alex Krizhevsky, Geoffrey Hinton, et al. 2009\.  Learning multiple layers of features from tiny images.  (2009).
  * Lang [1995] Ken Lang. 1995.  Newsweeder: Learning to filter netnews. In _Proceedings of the Twelfth International Conference on Machine Learning_. 331–339.
  * Li et al. [2022] Qinbin Li, Yiqun Diao, Quan Chen, and Bingsheng He. 2022.  Federated learning on non-iid data silos: An experimental study. In _2022 IEEE 38th International Conference on Data Engineering (ICDE)_. IEEE, 965–978.
  * Lin et al. [2022] Bill Yuchen Lin, Chaoyang He, Zihang Zeng, Hulin Wang, Yufen Huang, Christophe Dupuy, Rahul Gupta, Mahdi Soltanolkotabi, Xiang Ren, and Salman Avestimehr. 2022.  FedNLP: Benchmarking Federated Learning Methods for Natural Language Processing Tasks.  arXiv:2104.08815 [cs.CL]
  * Liu et al. [2021] Ze Liu, Yutong Lin, Yue Cao, Han Hu, Yixuan Wei, Zheng Zhang, Stephen Lin, and Baining Guo. 2021.  Swin transformer: Hierarchical vision transformer using shifted windows. In _Proceedings of the IEEE/CVF international conference on computer vision_. 10012–10022.
  * Nilsback and Zisserman [2008] Maria-Elena Nilsback and Andrew Zisserman. 2008.  Automated flower classification over a large number of classes. In _2008 Sixth Indian conference on computer vision, graphics & image processing_. IEEE, 722–729.
  * Qin et al. [2021] Yujia Qin, Yankai Lin, Jing Yi, Jiajie Zhang, Xu Han, Zhengyan Zhang, Yusheng Su, Zhiyuan Liu, Peng Li, Maosong Sun, et al. 2021\.  Knowledge inheritance for pre-trained language models.  _arXiv preprint arXiv:2105.13880_ (2021).
  * Radford et al. [2019] Alec Radford, Jeffrey Wu, Rewon Child, David Luan, Dario Amodei, Ilya Sutskever, et al. 2019\.  Language models are unsupervised multitask learners.  _OpenAI blog_ 1, 8 (2019), 9.
  * Shoeybi et al. [2019] Mohammad Shoeybi, Mostofa Patwary, Raul Puri, Patrick LeGresley, Jared Casper, and Bryan Catanzaro. 2019.  Megatron-lm: Training multi-billion parameter language models using model parallelism.  _arXiv preprint arXiv:1909.08053_ (2019).
  * Tedeschi et al. [2021] Simone Tedeschi, Valentino Maiorca, Niccolò Campolungo, Francesco Cecconi, and Roberto Navigli. 2021.  WikiNEuRal: Combined neural and knowledge-based silver data creation for multilingual NER. In _Findings of the Association for Computational Linguistics: EMNLP 2021_. 2521–2533.
  * Touvron et al. [2021] Hugo Touvron, Matthieu Cord, Matthijs Douze, Francisco Massa, Alexandre Sablayrolles, and Hervé Jégou. 2021.  Training data-efficient image transformers & distillation through attention. In _International conference on machine learning_. PMLR, 10347–10357.
  * Vaswani et al. [2017] Ashish Vaswani, Noam Shazeer, Niki Parmar, Jakob Uszkoreit, Llion Jones, Aidan N Gomez, Ł ukasz Kaiser, and Illia Polosukhin. 2017.  Attention is All you Need. In _Advances in Neural Information Processing Systems_ , I. Guyon, U. Von Luxburg, S. Bengio, H. Wallach, R. Fergus, S. Vishwanathan, and R. Garnett (Eds.), Vol. 30. Curran Associates, Inc.  https://proceedings.neurips.cc/paper_files/paper/2017/file/3f5ee243547dee91fbd053c1c4a845aa-Paper.pdf
  * Wang et al. [2023a] Peihao Wang, Rameswar Panda, Lucas Torroba Hennigen, Philip Greengard, Leonid Karlinsky, Rogerio Feris, David Daniel Cox, Zhangyang Wang, and Yoon Kim. 2023a.  Learning to Grow Pretrained Models for Efficient Transformer Training. In _The Eleventh International Conference on Learning Representations_.  https://openreview.net/forum?id=cDYRS5iZ16f
  * Wang et al. [2023b] Yite Wang, Jiahao Su, Hanlin Lu, Cong Xie, Tianyi Liu, Jianbo Yuan, Haibin Lin, Ruoyu Sun, and Hongxia Yang. 2023b.  LEMON: Lossless model expansion.  arXiv:2310.07999 [cs.LG]
  * Yan et al. [2024] Kunda Yan, Sen Cui, Abudukelimu Wuerkaixi, Jingfeng Zhang, Bo Han, Gang Niu, Masashi Sugiyama, and Changshui Zhang. 2024.  Balancing Similarity and Complementarity for Federated Learning.  _arXiv preprint arXiv:2405.09892_ (2024).
  * Yang et al. [2020] Cheng Yang, Shengnan Wang, Chao Yang, Yuechuan Li, Ru He, and Jingqiao Zhang. 2020.  Progressively stacking 2.0: A multi-stage layerwise training method for bert training speedup.  _arXiv preprint arXiv:2011.13635_ (2020).
  * You et al. [2019] Yang You, Jing Li, Sashank Reddi, Jonathan Hseu, Sanjiv Kumar, Srinadh Bhojanapalli, Xiaodan Song, James Demmel, Kurt Keutzer, and Cho-Jui Hsieh. 2019.  Large batch optimization for deep learning: Training bert in 76 minutes.  _arXiv preprint arXiv:1904.00962_ (2019).
  * Zhang and He [2020] Minjia Zhang and Yuxiong He. 2020.  Accelerating training of transformer-based language models with progressive layer dropping.  _Advances in Neural Information Processing Systems_ 33 (2020), 14011–14023.
  * Zhang et al. [2015] Xiang Zhang, Junbo Zhao, and Yann LeCun. 2015.  Character-level convolutional networks for text classification.  _Advances in neural information processing systems_ 28 (2015).
  * Zhang et al. [2024a] Zuyuan Zhang, Mahdi Imani, and Tian Lan. 2024a.  Modeling Other Players with Bayesian Beliefs for Games with Incomplete Information.  _arXiv preprint arXiv:2405.14122_ (2024).
  * Zhang et al. [2024b] Zuyuan Zhang, Hanhan Zhou, Mahdi Imani, Taeyoung Lee, and Tian Lan. 2024b.  Collaborative ai teaming in unknown environments via active goal deduction.  _arXiv preprint arXiv:2403.15341_ (2024).
  * Zhuang et al. [2023] Bohan Zhuang, Jing Liu, Zizheng Pan, Haoyu He, Yuetian Weng, and Chunhua Shen. 2023.  A survey on efficient training of transformers.  _arXiv preprint arXiv:2302.01107_ (2023).
  * Zou et al. [2024] Yifei Zou, Zuyuan Zhang, Congwei Zhang, Yanwei Zheng, Dongxiao Yu, and Jiguo Yu. 2024.  A Distributed Abstract MAC Layer for Cooperative Learning on Internet of Vehicles.  _IEEE Transactions on Intelligent Transportation Systems_ (2024).

##  Appendix A Detailed Description of Dual-LiGO

###  A.1 Notations

The notations to articulate the Dual-LiGO framework within our federated
learning context are shown in Table 5.

Table 5: Table of notations. Notation |  Description
---|---
$n$ |  Number of clients
$\mathcal{D}_{i}$ |  Dataset of client $i$
$\Theta_{L,D}$ |  A neural network with $L$ layers and $D$ dimensions
$\Theta_{i}^{(small)}$ |  Pre-trained small model of client $i$
$\Theta_{i}^{(inter)}$ |  Intermediate model of client $i$
$\Theta_{i}^{(large)}$ |  Large model of client $i$
$\mathbf{M}_{i}^{(local)}$ |  Local-LiGO of client $i$
$\mathbf{M}_{i}^{(global)}$ |  Global-LiGO of client $i$
$\ell(\Theta;x,y)$ |  Loss function given model $\Theta$ and data point $(x,y)$
$F(\Theta^{(large)})$ |  Objective function of the large model
$F_{i}(\Theta^{(large)})$ |  Average loss of the large model over dataset $\mathcal{D}_{i}$
$\eta_{l}$ |  The learning rate of Local-LiGO
$\eta_{g}$ |  The learning rate of Global-LiGO
$E_{l}$ |  Number of epochs for training the Local-LiGO
$E_{g}$ |  Number of epochs for training the Global-LiGO
$R$ |  Number of rounds for aggregation round

###  A.2 Pseudocode of Dual-LiGO

The pseudocode for the implementation of the Dual-LiGO framework within the
Fed-Grow federated learning system is shown in Algorithm 2 and Algorithm 1.
The algorithms detail the iterative process of training, updating, and
aggregating the models, which culminates in the construction of a robust and
large global model.

Algorithm 1 Fed-Grow with Dual-LiGO for Client $i$

1: Input: Pre-trained small model $\Theta_{i}^{(small)}$;

2: Local dataset $\mathcal{D}_{i}$; Local-LiGO training epoch $E_{l}$;

3: Federated round $R$; Global-LiGO training epoch $E_{g}$;

4: Output: The global large model $\Theta_{i}^{(large)}$; // Train the Local-
LiGO for $E_{l}$ epochs.

5: for $E_{l}$ epochs do

6: Train Local-LiGO $\mathbf{M}_{i}^{(local)}$ on local data $\mathcal{D}_{i}$
with equation 3 as the goal and equation 4 as the gradient descent rule;

7: end for // Train and upload the Global-LiGO for $R$ rounds.

8: for $r=1$ to $R$ do

9: for $E_{g}$ epochs do

10: Train $\mathbf{M}_{i}^{(global)}$ on local data $\mathcal{D}_{i}$ with
equation 6 as the goal and equation 7 as the gradient descent rule;

11: end for

12: Upload its Global-LiGO $\mathbf{M}_{i}^{(global)}$ to parameter server;

13: $\mathbf{M}_{i}^{(global)}\leftarrow\text{the aggregated result from
parameter server}$ ;

14: end for

15: Obtain global large model $\Theta_{i}^{(large)}$ via
$\mathbf{M}_{i}^{(global)}$;

Algorithm 2 Fed-Grow with Dual-LiGO for Parameter Server

1: for $r=1$ to $R$ do

2: Receive $M_{i}^{(global)}$ from all clients;

3:
$\mathbf{M}^{(global)}\leftarrow\sum_{i=1}^{n}\frac{|\mathcal{D}_{i}|}{\sum_{j=1}^{n}|\mathcal{D}_{j}|}\mathbf{M}_{i}^{(global)}$;

4: Send $\mathbf{M}^{(global)}$ to all clients;

5: end for

##  Appendix B Details of Experimental Setup

###  B.1 Dataset Descriptions

  * •

20 Newsgroups: The 20 Newsgroups dataset [15] is a collection of approximately
20,000 newsgroup documents, partitioned nearly across 20 different newsgroups.

  * •

AG News: The AG News dataset [32] is a sub-dataset of AG’s corpus of news
articles, containing 30,000 training and 1,900 test samples per class, with a
total of 120,000 training samples and 7,600 testing samples.

  * •

WikiNER: The WikiNER dataset [23] is a large collection of sentences from
Wikipedia, predicting 4 tags: ’PER’ for person names, ’LOC’ for location
names, ’ORG’ for organization names, and ’MISC’ for other names.

  * •

CIFAR-10 and CIFAR-100: The CIFAR-10 and CIFAR-100 datasets [14] are labeled
subsets of the 80 million tiny images dataset, designed for image
classification tasks.

  * •

Flowers-102: The Flowers-102 dataset [19] is an image classification dataset
consisting of 102 flower categories commonly occurring in the United Kingdom.

###  B.2 Configurations of Different Transformer Models

The detailed architecture of small models $\Theta^{(small)}$, intermediate
models $\Theta^{(inter)}$, and large models $\Theta^{(large)}$ can be found in
Table 6. For both BERT and ViT, we use the configuration shown in Table 6. For
each client’s small model, we randomly and uniformly sample one of the three
heterogeneous model architectures in Table 6 and train it as a pre-trained
small model for that client.

Table 6: Configurations of different transformer models. Name | #Hidden | #Layers | #Heads
---|---|---|---
small#1 | 256 | 2 | 8
small#2 | 256 | 3 | 8
small#3 | 256 | 4 | 8
intermediate | 320 | 4 | 8
large | 384 | 6 | 8

###  B.3 Implementation Details

Our experimental platform is built upon Flower [3] and FedNLP [17] with
Pytorch as the training backend. All simulations are conducted on $4\times$
NVIDIA RTX 4090 GPUs with 320GB memory. We conduct our simulations under
various conditions to thoroughly evaluate the performance of our models. Our
simulations span 6 different tasks, providing a comprehensive evaluation of
our models’ performance across diverse domains. For different datasets, we set
different hyper-parameters $E_{l}$, $E_{g}$, and $R$ due to the varying
dataset sizes. For each client, we train the small model on the local dataset
for $E_{l}$ rounds as the pre-trained small model. Following [16; 17], we use
the Dirichlet distribution with $\beta=0.5$ to generate non-IID data and
$\beta=100$ to generate IID data, with a fixed random seed. We test our models
with different numbers of clients, specifically 10 and 20 clients. The
detailed configurations for each task are provided in Table 7. We referenced
the code of [34; 36; 33; 11].

###  B.4 Metrics

We use several key metrics to evaluate the performance of our models. For
tasks involving image and text classification, we report the top-1 accuracy on
the test set. For the sequence tagging task, we report precision on the test
set. In addition to these task-specific metrics, we assess the stability of
our models’ performance across different clients by calculating and reporting
the standard deviation of the aforementioned metrics. Moreover, we report the
number of trainable parameters and communication cost, to reflect the resource
consumption on computation and communication resources.

Table 7: Hyperparameters for each task. Task | Batch Size | LR | $E_{l}$ | $E_{g}$ | $R$ | Optimizer
---|---|---|---|---|---|---
20news | 32 | 3e-4 | 100 | 10 | 200 | AdamW
agnews | 32 | 1e-4 | 10 | 1 | 100 | AdamW
wikiner | 32 | 1e-4 | 50 | 3 | 100 | AdamW
CIFAR-10 | 64 | 5e-4 | 50 | 4 | 200 | AdamW
CIFAR-100 | 64 | 5e-4 | 50 | 4 | 200 | AdamW
Flowers-102 | 64 | 5e-4 | 200 | 40 | 100 | AdamW

##  Appendix C Additional Experimental Results

###  C.1 Curves of Test Accuracy/Precision During Training Process

Fig 4 exhibit evolution of the mean accuracy/precision over global
communication stpes in 20-client settings. From which, we can find that our
method outperforms baseline in terms of convergence speed and final
performance. The baseline methods are more prone to overfitting (Fig 4 (c)) or
even failing to converge (Fig 4 (b)) due to the reduced amount of data shared
by each client, our method effectively avoids these problems.

Fig 5 and Fig 6 show the performance of our method on a larger number of
clients (30-50 clients). As the number of clients increases, less data is
stored on each client, resulting in slower convergence, but our method still
significantly outperforms baseline in terms of convergence speed and final
performance.

(a) 20news

(b) agnews

(c) wikiner

(d) CIFAR-10

(e) CIFAR-100

(f) Flowers-102

Figure 4: Comparison of Dual-LiGO (Agg) and NoAgg on six datasets under IID
and non-IID settings with 20 clients.

(a) 30 clients

(b) 40 clients

(c) 50 clients

Figure 5: Comparison of Dual-LiGO (Agg) and NoAgg on agnews under IID and non-
IID settings with 30-50 clients.

(a) 30 clients

(b) 40 clients

(c) 50 clients

Figure 6: Comparison of Dual-LiGO (Agg) and NoAgg on CIFAR-100 under IID and
non-IID settings with 30-50 clients.

###  C.2 Effects of Small Model Heterogeneity

In the original experiment, we designed three heterogeneous small models. In
order to verify the robustness of our method to the degree of heterogeneity of
the small models, in this section, we designed 2 additional heterogeneous
cases and ran the experiments in the CIFAR-100 dataset, IID, 10-client
settings. The model heterogeneity cases are shown in Table 8, Table 9 and the
final experimental results are presented in Table 10. From the table, we can
see that regardless of the heterogeneity case, our method has a significant
improvement over the baseline method, which proves the tolerance of our method
for small model heterogeneity.

Table 8: Configurations of additional heterogeneity case 1. Name | #Hidden | #Layers | #Heads
---|---|---|---
small#1 | 256 | 2 | 8
small#2 | 256 | 3 | 8
small#3 | 256 | 4 | 8
small#4 | 192 | 2 | 8
small#5 | 192 | 3 | 8
small#6 | 192 | 4 | 8
intermediate | 320 | 4 | 8
large | 384 | 6 | 8
Table 9: Configurations of additional heterogeneity case 2. Name | #Hidden | #Layers | #Heads
---|---|---|---
small#1 | 256 | 2 | 8
small#2 | 256 | 3 | 8
small#3 | 256 | 4 | 8
small#4 | 256 | 5 | 8
small#5 | 256 | 6 | 8
intermediate | 320 | 7 | 8
large | 384 | 8 | 8
Table 10: Mean accuracy of Dual-LiGO (Agg) and NoAgg on the CIFAR-100 dataset under IID, 10-client settings with different heterogeneity of small models. Heterogeneity Case |  Accuracy (%) $\uparrow$
---|---
Agg | NoAgg | $\Delta$
Original Case | 53.178 | 38.127 | 15.051
Additional Case 1 | 57.812 | 38.306 | 19.506
Additional Case 2 | 53.949 | 47.568 | 6.381

###  C.3 Comparison with FedAvg with Only Large Model

In this section, we compare our approach with FedAvg, specifically, we resize
the large model to ensure roughly the same amount of communication as our
approach, and then train the large model, upload and aggregate it (There are
no small models and intermediate models stored in clients); Table 11 shows the
results of the comparison in 20news dataset, IID and 10-client settings. Due
to the limitation of communication resources, the parameter number of the
large model is limited, and the performance of our algorithm has a significant
advantage over directly training a large model under the same communication
resources. Moreover, compared to directly training a large model, our approach
is able to utilize the implicit knowledge of the pre-trained small model on
each client to further accelerate the training process.

Table 11: Comparative results of communication cost and accuracy between FedAvg and our approach on the 20news dataset under IID conditions with 10 clients. Method | Comm Cost | Accuracy (%) $\uparrow$
---|---|---
FedAvg | 4.034M | 53.638
Ours | 4.178M | 65.295

###  C.4 Visualization of Intermediate Models’ Similarity

In our investigation of privacy preservation within our proposed framework, we
explore the pair-wise similarity between intermediate models contributed by
different clients in agnews dataset, non-IID and 10-clients setting. The
similarity matrix, computed using cosine similarity is shown in Figure 7.

According to [28], clients with highly similar models (values close to 1) may
inadvertently leak information about their individual data distributions. Such
similarity could lead to privacy risks, including membership inference
attacks. On the other hand, clients with diverse models (values closer to 0)
enhance privacy by introducing aggregated noise and making it harder for
adversaries to pinpoint specific contributions. In our case, the similarity
between the clients’ intermediate models is very low, which indicates the
advantage of our approach for privacy preservation. In addition to this, we
still need to emphasize that during the transmission stage of our models, only
Global-LiGO is transmitted, and it is very difficult for an attacker to access
the parameters of our intermediate or small models as well as the training
data of each client through traditional attack measures.

Figure 7: The cosine similarity between intermediate models in different
clients in agnews dataset, non-IID and 10-clients setting.

##  Appendix D Limitations

Despite the innovative approach and promising results of Fed-Grow with Dual-
LiGO, we acknowledge certain limitations that warrant further investigation.

In the spirit of continuous improvement, we note that the current iteration of
Fed-Grow shows room for enhancement in scenarios involving 20-client agnews
datasets with non-IID settings (can be found in Table 1). This observation
opens up opportunities to deepen our understanding of data diversity’s impact
on model performance. Our forthcoming research will be dedicated to exploring
these dynamics further, with the aim of fortifying the robustness of our
framework across varied client environments.

The journey of a global large model to its full potential includes a pivotal
step of client-specific fine-tuning. This process is key to unlocking a
model’s robust performance tailored to individual datasets. Recognizing that
computational resources are a valuable commodity, we are motivated to explore
more efficient fine-tuning methodologies. Our future work is poised to focus
on this endeavor, aspiring to make the fine-tuning process as seamless as
possible for all clients, regardless of their resource availability.


eq

$\displaystyle\begin{split}\BODY\end{split}$ (1)

# Heavy QCD axion dark matter from avoided level crossing

David Cyncynates<EMAIL_ADDRESS>Department of Physics, University of
Washington, Seattle, WA 98195, U.S.A.  Jedidiah O. Thompson
<EMAIL_ADDRESS>Stanford Institute for Theoretical Physics, Stanford
University, Stanford, CA 94305, U.S.A.

###### Abstract

The QCD axion offers a natural resolution to the strong CP problem and
provides a compelling dark matter candidate. If the QCD axion constitutes all
the dark matter, the simplest models pick out a narrow range of masses around
$100\,\mu{\rm eV}$. We point out a natural production mechanism for QCD axion
dark matter at masses up to existing astrophysical bounds
($\mathcal{O}(20\,\mathrm{meV})$ for the most minimal models and
$\mathcal{O}(1\,\mathrm{eV})$ for nucleophobic models). If the QCD axion mixes
with a sterile axion, the relative temperature dependence of their potentials
can lead to an avoided level crossing of their mass eigenstates. This leads to
a near-total transfer of energy density from the sterile axion to the QCD
axion, resulting in a late-time QCD axion abundance sufficient to make up all
of present-day dark matter. Our result provides additional theoretical
motivation for several direct detection experiments that will probe this part
of parameter space in the near future.

##  I Introduction

The QCD axion is one of the best-motivated candidates for physics Beyond the
Standard Model (BSM). Although it was originally proposed as a solution to the
strong-CP problem [1, 2, 3], it was quickly realized that such a new field
could also have cosmological consequences [4, 5, 6, 7]. In particular, it is
an excellent candidate for dark matter (DM), for which we have overwhelming
evidence from a number of sources [8, 9].

At low energies and in the absence of further model-building, the properties
of the QCD axion are determined (apart from some $\mathcal{O}(1)$ model
dependencies) entirely by one parameter: its zero-temperature mass $m_{a,0}$.
The axion has several couplings to the Standard Model (SM) whose strengths are
typically set by its decay constant $f_{a}\sim m_{\pi}f_{\pi}/m_{a,0}$ where
$m_{\pi}f_{\pi}\sim 200{\rm\,MeV}$. One particularly promising coupling to
target is the axion-photon coupling
$\mathcal{L}\supset-\frac{g_{a\gamma\gamma}}{4}\phi_{a}F_{\mu\nu}\tilde{F}^{\mu\nu}$,
where $\phi_{a}$ is the axion field and $F_{\mu\nu}$ is the SM photon field
strength. In minimal models, the constant $g_{a\gamma\gamma}$ is given by
$g_{a\gamma\gamma}=C_{a\gamma\gamma}\alpha_{\rm QED}/2\pi f_{a}$ where
$C_{a\gamma\gamma}$ is an $\mathcal{O}(1)$ model-dependent constant and
$\alpha_{\rm QED}$ is the fine-structure constant.

An axion making up the entirety of DM is ruled out for masses $m_{a,0}\gtrsim
20\,\mathrm{meV}$, since the axion coupling to SM nucleons would lead to
anomalous energy loss in neutron stars [10] and SN1987A [11]. In models where
the QCD axion couples only weakly to nucleons (so-called nucleophobic models
[12, 13]), the above axion-photon coupling still places a bound of
$m_{a,0}\lesssim 1\,\mathrm{eV}$ [14, 15]. At lower masses, however, there are
few phenomenological constraints on axion DM, and in fact there are many
existing and planned experiments that are probing regions here [16, 17, 18,
19, 20, 21, 22, 23, 24, 25, 26, 27, 28]. It is thus important to understand
the possible production mechanisms for axion DM, as they guide the most well-
motivated search targets. Since the axion arises as a pseudo-Nambu-Goldstone
boson associated with a new Peccei-Quinn (PQ) symmetry, these mechanisms split
into two categories depending on whether this symmetry is broken before or
after inflation ends.

If PQ symmetry is broken after inflation, then the axion takes random initial
values in each Hubble patch at the time of breaking. This stochastic initial
field evolves into a complicated network of axion strings and domain walls
that collapse and decay around the time of the QCD phase transition [29].
These dynamics can be simulated, and the simulations may be used to extract a
sharp prediction for the mass of a post-inflationary axion: $m_{a,0}\sim
40\div 180\,\mu{\rm eV}$ [30]. Although there is some modeling uncertainty
[31, 32], it seems unlikely that post-inflationary production could yield an
eV-scale mass.

If PQ symmetry is broken before the end of inflation on the other hand, then
the axion initial field value is effectively homogeneous and, depending on the
inflationary history, non-zero across the observed universe. This is the
misalignment mechanism; the initial value is known as the axion misalignment
angle $\Theta_{0}$, and it (along with the axion mass) determines the present-
day axion energy density. Because the QCD axion potential is periodic,
$\Theta_{0}$ is valued in the range $[-\pi,\pi)$. One minimal possibility is
therefore that $\Theta_{0}\sim\mathcal{O}(1)$, as would be the case if it were
selected by UV dynamics insensitive to the low-energy QCD potential. For
$|\Theta_{0}|\sim\pi/2$, the present-day DM abundance is produced for a QCD
axion with mass $m_{a,0}\sim 10\,\mu{\rm eV}$. For axion masses smaller than
this, the misalignment mechanism requires $|\Theta_{0}|\ll 1$, which can be
explained either by dynamic [33, 34] or anthropic [35, 36, 37] arguments.
However for much heavier axion masses $m_{a,0}\gg{\rm meV}$, it becomes
significantly more difficult to produce the proper DM abundance via
misalignment. QCD axions with masses $m_{a,0}\sim 1\,\mathrm{eV}$ for example
would require an initial misalignment angle tuned extremely close to $\pi$:
$\pi-|\Theta_{0}|\approx e^{-10^{3}}$ [38, 39]. Such a possibility is not only
aesthetically problematic, but also violates inflationary isocurvature
constraints [39].

On the other hand, this high-mass region of parameter space is experimentally
interesting, with several new experiments either probing or set to probe QCD
axion dark matter with masses $0.2{\,\rm meV}\lesssim m_{a,0}\lesssim 1{\,\rm
eV}$ [17, 19, 18]. It is thus important to understand what type of model can
produce heavy QCD axion DM and how complicated such a model must be.

The landscape of such models is far from vacant. There exist a handful of
models that rely on various dynamics of a UV partner field (often taken to be
the radial mode of the complex axion parent field) to generate energy density
that subsequently converts into axions, either via parametric resonance [40]
or intricate non-thermal dynamics [41, 42]. In the post-inflationary scenario,
there is also the possibility that the QCD axion has multiple nearly-
degenerate vacua, which can lead to the enhanced late-time axion energy
density necessary to be DM [43, 44, 45, 46, 47, 48]. In this letter, we will
describe a different type of model that is both simple and free of any tuning
problems. The dynamics we discuss are also quite independent of any new
couplings of the axion in the UV.

The key insight is as follows. It is possible (indeed even theoretically well-
motivated [49]) that there are one or more additional axions in the theory. We
will consider one such field $\phi_{S}$ with mass $m_{S}$ and decay constant
$f_{S}$, which we will call “sterile” as $f_{S}\gtrsim f_{a}$ in the parameter
space of interest. For such a field, $m_{S}$ and $f_{S}$ can be effectively
independent, and if it is sourced by misalignment, then the present-day energy
density is generically proportional to $f_{S}^{2}$. However it turns out that
this energy density can be easily transferred to the QCD axion via the
temperature-dependent nature of the QCD axion potential. The QCD axion mass is
extremely small at high temperatures but increases to its zero-temperature
mass $m_{a,0}$ as the universe cools below the QCD scale. It is thus possible
that the QCD and sterile axion masses cross each other, and if there is any
interaction between these two fields then the mass eigenstates can instead
undergo an avoided crossing, leading to an adiabatic transfer of energy from
$\phi_{S}$ to $\phi_{a}$.111A related phenomenon is pointed out in Ref. [50],
wherein the QCD axion transfers its energy to the sterile axion. The sterile
axion thus acts effectively as a battery that stores enough energy density for
the QCD axion to become the dark matter at late times.

##  II Dynamics

We consider the following model of the QCD axion $\phi_{a}$ interacting with a
sterile axion $\phi_{S}$ (which we UV complete in the subsequent section):

$\displaystyle{\mathcal{L}}$
$\displaystyle\supset\frac{1}{2}(\partial\phi_{a})^{2}+\frac{1}{2}(\partial\phi_{S})^{2}$
(2) $\displaystyle-
m_{a}^{2}(T)f_{a}^{2}\left[1-\cos\left(\frac{\phi_{a}}{f_{a}}+\frac{\phi_{S}}{f_{S}}\right)\right]$
$\displaystyle-
m_{S}^{2}f_{S}^{2}\left[1-\cos\left(\frac{\phi_{S}}{f_{S}}\right)\right]\,,$

where we approximate the temperature dependence by the simplified expression:

$\displaystyle m_{a}^{2}(T)=m_{a,0}^{2}\max\left\\{1,\left(\frac{T}{T_{\rm
QCD}}\right)^{-n}\right\\}\,,$ (3)

with $T_{\rm QCD}\approx 100{\rm MeV}$ and $n\approx 6.68$ in the dilute
instanton gas approximation [51]. The interesting dynamics that we will study
will occur when $m_{S}\ll m_{a,0}$ and $f_{S}\gg f_{a}$, and from here forward
we will work in this region of parameter space.

At leading order, we may approximate the two-axion potential by its quadratic
terms

$\displaystyle
V\approx\frac{1}{2}\left(\begin{array}[]{cc}\phi_{a}&\phi_{S}\end{array}\right)\left(\begin{array}[]{cc}m_{a}^{2}&\frac{f_{a}}{f_{S}}m_{a}^{2}\\\
\frac{f_{a}}{f_{S}}m_{a}^{2}&m_{S}^{2}+\frac{f_{a}^{2}}{f_{S}^{2}}m_{a}^{2}\\\
\end{array}\right)\left(\begin{array}[]{c}\phi_{a}\\\
\phi_{S}\end{array}\right)\,.$ (9)

The fields $\phi_{a},\phi_{S}$ are thus not propagation eigenstates. Instead,
we must rotate to a basis in which this mass matrix is diagonalized. As the
temperature of the universe drops and $m_{a}(T)$ evolves, the propagation
basis rotates, which leads to adiabatic energy density transfer between the
two fields.

If $f_{S}\gg f_{a}$, the mass matrix is nearly diagonal. At early times,
$m_{a}(T)\ll m_{S}$ and we can find the heavy and light mass eigenstates to be
$\phi_{H}\approx\phi_{S}$ and $\phi_{L}\approx\phi_{a}$ respectively. At late
times the temperature has dropped, leading to $m_{a}(T)\gg m_{S}$. Now the
heavy and light mass eigenstates are given by $\phi_{H}\approx\phi_{a}$ and
$\phi_{L}\approx\phi_{S}$ respectively. At some intermediate temperature, when
$m_{a}(T)\approx m_{S}$, the mass matrix is nearly the identity and the two
mass eigenvalues are nearly degenerate. However the off-diagonal elements
split this degeneracy and lead to an avoided crossing of the eigenvalues.
Provided the transition through this avoided crossing is adiabatic (meaning
slow compared to the oscillatory timescale of the two axion fields), all
energy density contained in the heavy propagation eigenstate will remain in
the heavy eigenstate. In other words, energy density will be smoothly
transferred from $\phi_{S}$ to $\phi_{a}$.

We will now check the necessary condition for this transition to be adiabatic,
and then compute the present-day energy density in the QCD axion after these
early-time dynamics. The avoided crossing occurs at a time when the mass
matrix is approximately the identity, so we define the crossing temperature
$T_{\times}$ and time $t_{\times}$ to occur when
$m_{a}^{2}=m_{S}^{2}+\frac{f_{a}^{2}}{f_{S}^{2}}m_{a}^{2}\approx m_{S}^{2}$.
The timescale over which the crossing happens is set by when the off-diagonal
terms are important. A parametric estimate is that it begins when
$m_{a}^{2}-m_{S}^{2}\approx\frac{f_{a}}{f_{S}}m_{a}$ and ends when
$m_{a}^{2}-m_{S}^{2}\approx-\frac{f_{a}}{f_{S}}m_{a}$. From this we can
calculate that the crossing lasts for a parametric duration $\Delta
t_{\times}$ given by:

$\Delta
t_{\times}\approx\frac{3}{n}\frac{f_{a}}{f_{S}}\sqrt{\frac{20}{\pi^{3}g_{\star}(T_{\times})}}\frac{m_{\rm
pl}}{T_{\rm QCD}^{2}}\left(\frac{m_{S}}{m_{a,0}}\right)^{4/n}\,,$ (10)

where $m_{\rm pl}=G_{N}^{-1/2}$ is the Planck mass. In order for the
transition to be adiabatic, we require:

$\Delta t_{\times}\gg m_{S}^{-1}\approx m_{a}(T_{\times})^{-1}\,,$ (11)

which can easily be satisfied so long as $f_{S}$ is not too large.

The equations of motion satisfied by the homogeneous axion field are identical
to the equations satisfied by the spatial axion perturbations, up to a common
diagonal gradient term [52]. An avoided crossing will therefore also occur for
each spatial fourier mode, causing the QCD axion to inherit the perturbations
of the sterile axion. So long as the sterile axion is initialized with
negligible isocurvature, the QCD axion will have purely adiabatic
perturbations.

Now provided the transition is adiabatic, nearly all of the energy density in
$\phi_{S}$ before the crossing will be transferred to $\phi_{a}$ after the
crossing. We can thus estimate the late-time QCD axion abundance as follows.
The initial energy density in the sterile axion field before it begins
oscillating is given by $\rho_{S}(H\gg
m_{S})\approx(1/2)m_{S}^{2}f_{S}^{2}\Theta_{0,S}^{2}$, where $\Theta_{0,S}$ is
the initial misalignment angle of $\phi_{S}$. At a time $H\sim m_{S}$, the
sterile axion field starts oscillating and this energy density begins
redshifting as $a^{-3}$ (where $a(t)$ is the scale factor). At level crossing,
this energy density is transferred to the QCD axion, but by construction this
must happen at a time when the QCD axion mass is still below its zero-
temperature mass. As $m_{a}(T)$ increases, the energy density stored in the
QCD axion field also increases, going as $a^{(n-6)/2}$. Finally, when the QCD
axion reaches its zero-temperature mass (i.e. when the universe temperature is
$T\sim T_{\rm QCD}$), this energy density again starts redshifting as $a^{-3}$
and does so until the present day. Putting all of this together, we obtain an
estimate for the present-day QCD axion energy abundance:

$\displaystyle\Omega_{a}\approx\frac{4\pi}{3}\frac{m_{a,0}m_{S}f_{S}^{2}\Theta_{0,S}^{2}}{m_{\mathrm{pl}}^{2}H_{0}^{2}}\frac{a^{3}(T_{\rm
osc})}{a^{3}(T_{0})}\,,$ (12)

where we have defined the oscillation temperature by $3H(T_{\rm osc})=m_{S}$.

Figure 1: (Upper panel) In pink and blue, we plot the energy density of the
mass eigenstate that is most strong coupled to the sterile and QCD instantons
respectively. Note that at crossing, the two states are roughly equally
coupled to the QCD instanton, but away from crossing this distinction is
robust. (Lower panel) In green and orange, we plot the heavy and light mass
eigenvalues respectively. The inset shows the avoided crossing in detail. In
both plots, temperature decreases (and time increases) to the right. Figure 2:
The axion-photon coupling $g_{a\gamma\gamma}$ versus the axion mass $m$. The
QCD axion line is highlighted in three colors, corresponding roughly to the
range of massess accessible to three different production mechanisms. In red
is pre-inflationary production assuming an initial misalignment angle in the
range $[0.1\pi,0.9\pi]$ [53, 38], in blue is post-inflationary production,
where the mass range is taken from Ref. [30], and in green are the higher
masses accessible through adiabatic transfer (this work). The dark gray
regions are excluded either by dark matter haloscopes [54, 55, 56, 57, 58, 59,
60, 61, 62, 63, 64, 65, 66, 67, 68, 69, 70, 71, 72, 73, 74, 75] or by
astrophysical probes [14, 15, 76]. The light-gray transparent regions are
prospective sensitivity curves for upcoming experiments [16, 17, 18, 19, 20,
21, 22, 23, 24, 25, 26, 27, 28]. The data used to make this plot is compiled
in Ref. [77].

It is instructive to compare this parametrically to the present-day abundance
expected for the minimal model of a QCD axion with $\mathcal{O}(1)$ initial
misalignment angle and zero-temperature mass $m_{a,0}$:

$\displaystyle\frac{\Omega_{a}^{\mathrm{(level\,cross)}}}{\Omega_{a}^{\mathrm{(minimal)}}}\sim\frac{f_{S}^{2}}{f_{a}^{2}}\left(\frac{T_{\rm
QCD}}{\sqrt{m_{\mathrm{pl}}m_{S}}}\right)\left(\frac{m_{\mathrm{pl}}m_{a,0}}{T_{\rm
QCD}^{2}}\right)^{\frac{2}{n+4}}\,,$ (13)

where we have dropped numerical coefficients, and the terms inside parentheses
are typically ${\mathcal{O}}({\rm few})$. Because the ratio $f_{S}/f_{a}$ can
be large, it is clear that an adiabatic transfer of energy from an initial
sterile field can provide a significant enhancement in the late-time abundance
of the QCD axion.

In Fig. 1, we show a representative example of these dynamics. We plot the
axion energy densities (upper panel) and mass eigenvalues (lower panel) as
functions of SM temperature, with temperature decreasing (time increasing)
from left to right. At high temperatures, the heavy state comprises mostly the
sterile axion, and the light state mostly the QCD axion. As the Standard Model
plasma cools and QCD axion potential turns on, the eigenvalues approach one
another and we observe an avoided crossing at $T_{\times}$. In this example,
$T_{\times}$ occurs long after both axions have started oscillating so that
the crossing takes place over many oscillations, therefore satisfying the
condition of Eq. 11. As a consequence, the energy initially associated with
the heavy state (and mostly with the sterile axion) remains with the heavy
state. After the crossing, however, the heavy state mostly comprises the QCD
axion, and its energy density increases as the universe cools due to the rapid
increase in its mass. This carries on until the universe cools below $T_{\rm
QCD}$ and the relic QCD axion matter fraction is frozen in. In this example,
parameters have been chosen so that the initial value of $f_{S}$ is precisely
right to make the final energy density in the QCD axion equal to the present-
day DM density.

We now explicitly check that the full range of QCD axion masses above the
typical misalignment range $m_{a,0}\geq 10\,\mu{\rm eV}$ are accessible
through this mechanism. By setting $\Omega_{a}$ equal to the present-day
observed DM abundance and taking a fiducial value of $\Theta_{0,S}=\pi/2$, we
can solve Eq. 12 for the necessary $f_{S}$. Plugging this into Eq. 11, we find
that the crossing will be adiabatic if:

$\displaystyle\frac{10^{6}}{n}\left(\frac{m_{S}}{\rm
eV}\right)^{\frac{3}{4}+\frac{4}{n}}\left(\frac{m_{a,0}}{\rm
eV}\right)^{-\left(\frac{1}{2}+\frac{4}{n}\right)}\gg 1\,.$ (14)

In addition, we require $f_{S}\gg f_{a}$, since otherwise the mass eigenstates
have a very different structure. Using the usual relation between $f_{a}$ and
$m_{a,0}$ as well as the value for $f_{S}$ necessitated by Eq. 12, we obtain
the requirement:

$\displaystyle 6\times 10^{5}\left(\frac{m_{a,0}}{\rm
eV}\right)^{1/2}\left(\frac{m_{S}}{\rm eV}\right)^{1/4}\gg 1\,.$ (15)

We must also require that a crossing actually happens and that it happens
while both axions have already started oscillating. This means we must require
$T_{\times}\ll T_{\rm osc}$ and $m_{S}\ll m_{a,0}$. Computing $T_{\times}$ and
$T_{\rm osc}$ this first requirement reads:

$\displaystyle 7\times 10^{-6}\left(\frac{m_{S}}{\rm
eV}\right)^{-\left(\frac{1}{2}+\frac{2}{n}\right)}\left(\frac{m_{a,0}}{\rm
eV}\right)^{\frac{2}{n}}<1\,.$ (16)

Finally, we must check that the value of $f_{S}$ required by Eq. 12 is not
already ruled out by direct detection. This is a weaker requirement than the
others, since it is possible that $\phi_{S}$ has no coupling to the SM, but we
conservatively assume the presence of at least a sterile axion-photon coupling
$\frac{g_{S\gamma\gamma}}{4}\phi_{S}F\tilde{F}$ with coupling strength
$g_{S\gamma\gamma}\sim\alpha_{\rm QED}/(2\pi f_{S})$. This implies that the
desired sterile axion would be ruled out unless the masses satisfy:

$\displaystyle 5\times 10^{-5}\left(\frac{m_{S}}{\rm
eV}\right)^{-\frac{1}{4}}\left(\frac{m_{a,0}}{\rm eV}\right)^{\frac{1}{2}}\ll
1\,.$ (17)

One can easily verify that these constraints may all be simultaneously
satisfied over the entire high-mass QCD axion mass range of interest here:
$10\,\mu{\rm eV}\leq m_{a,0}\leq 1\,{\rm eV}$.

Fig. 2 plots this available parameter space for QCD axions coupled to photons
through $\frac{g_{a\gamma\gamma}}{4}\phi_{a}\tilde{F}F$, with $F$ the
electromagnetic field strength. The colored area is the QCD axion band, with
the red region denoting the mass range accessible through misalignment with
$\phi_{a}(0)/f_{a}\in[0.1\pi,0.9\pi]$ [53, 38], the blue region representing
the expected range for post-inflationary production [30], and the green region
indicating those parts of parameter space accessible with the adiabatic
transfer described in this work. The dark gray regions represent excluded
parameter space either by axion haloscopes [54, 55, 56, 57, 58, 59, 60, 61,
62, 63, 64, 65, 66, 67, 68, 69, 70, 71, 72, 73, 74, 75] or by astrophysical
probes [14, 15, 76]. In transparent light-gray, we plot the expected reach of
some upcoming experiments [16, 17, 18, 19, 20, 21, 22, 23, 24, 25, 26, 27,
28]. In particular, we point out that some experiments at higher masses (for
example Refs. [17, 18, 19]) are probing regions of parameter space which are
generally not expected to be populated by the minimal formation mechanisms,
but are naturally produced by the mechanism described here.

##  III UV Completion

The dynamics described in the previous section can arise in a wide variety of
scenarios (see e.g. [78]), and in this section we provide one concrete
realization derived from the KSVZ mechanism [79, 80]. Let $q_{1}$ and $q_{\rm
mix}$ be new vector-like quarks charged under the Standard Model QCD gauge
group, and let $q_{2}$ be a vector-like quark charged under some new dark
confining gauge group. We now introduce two complex scalar fields
$\Phi_{1}=\rho_{1}e^{\text{i}\theta_{1}}$ and
$\Phi_{2}=\rho_{2}e^{\text{i}\theta_{2}}$, and suppose the Lagrangian is
invariant under the following pair of $U_{A}(1)$ symmetries

$\displaystyle\Phi_{1}\to\Phi_{1}e^{-\text{i}\alpha}\,,\hskip
28.45274pt\Phi_{2}\to\Phi_{2}e^{-\text{i}\beta}\,,$ (18) $\displaystyle
q_{1}\to e^{\text{i}\alpha\gamma^{5}/2}q_{1}\,,\hskip 11.38092ptq_{\rm mix}\to
e^{\text{i}\beta\gamma^{5}/2}q_{\rm mix}\,,\hskip 11.38092ptq_{2}\to
e^{\text{i}\beta\gamma^{5}/2}q_{2}\,,$

with $\alpha,\beta\in\mathbb{R}$. As a consequence, the structure of the
potential is limited to the form

$\displaystyle V$
$\displaystyle=\lambda_{1}\Phi_{1}\bar{q}_{1}q_{1}+\lambda_{\rm
mix}\Phi_{2}\bar{q}_{\rm mix}q_{\rm mix}+$
$\displaystyle+\lambda_{2}\Phi_{2}\bar{q}_{2}q_{2}+{\rm
h.c.}+V(|\Phi_{1}|^{2},|\Phi_{2}|^{2})\,.$ (19)

Upon rotating away the complex phases of $\Phi_{1}$ and $\Phi_{2}$ in an axial
rotation of the quarks, we are left with the following Lagrangian for the
axion-(dark) gluon interactions

$\displaystyle{\mathcal{L}}$
$\displaystyle\supset\frac{1}{32\pi^{2}}(\theta_{1}+\theta_{2}+\arg\det
M)G\tilde{G}$ (20) $\displaystyle+\frac{1}{32\pi^{2}}(\theta_{2}+\arg\det
M_{D})G_{D}\tilde{G}_{D}\,,$

where $M$ and $M_{D}$ are the (dark) quark mass matrices. One may absorb both
$\arg\det$’s into the choice of zero for $\theta_{1}$ and $\theta_{2}$. If the
dark gauge group has an instanton condensate similar to the SM QCD gauge
group, then the low-energy dynamics of this model will be the model of the
previous section. Provided the dark gauge group confines long before QCD, then
at the scales relevant for QCD it will lead to a temperature-independent
potential for the sterile axion, giving precisely the low-energy Lagrangian
shown in Eq. 2.

##  IV Discussion

We have shown that adiabatic level crossing between a sterile axion and the
QCD axion can lead to QCD axion dark matter at higher masses. Our result
provides motivation for several experiments that will probe this high-mass
range in the coming years, and expands the known mechanisms by which the QCD
axion may change its abundance. Other mechanisms that can produce heavy QCD
axion DM often modify large or small scale galactic structure [40, 41, 42, 43,
44, 45, 46, 47, 48]. In contrast, adiabatic level crossing relies only on
linear dynamics, and therefore leaves the matter power spectrum largely
unchanged (see e.g. Ref [52]). Observation of a heavy QCD axion without matter
power spectrum modifications is then suggestive of adiabatic transfer between
a sterile axion and the QCD axion.

This is not the first time such adiabatic transfer has been noticed. In
particular, previous work in Ref. [50] (also see Ref. [81, 82]) demonstrates a
similar effect, wherein the QCD axion may transfer its energy to some sterile
axion. The distinction between our mechanism and that of Ref. [50] is
particularly interesting: depending on whether the QCD axion couples to the
sterile axion through the QCD instanton or the dark instanton potential, the
energy flows either towards or away from the QCD axion respectively.222This
distinction comes down to the misalignment initial conditions implied by the
full non-linear potential. This demonstrates not only the ease with which the
QCD axion may change its abundance in a multi-axion theory, but also shows us
that we may learn something about the structure of the multi-axion potential
by measuring the abundance of the QCD axion.

###### Acknowledgements.

The authors would like to thank Masha Baryakhtar, Michael Fedderke, and
Zachary Weiner for helpful comments on the manuscript. D.C. is supported
through the Department of Physics and College of Arts and Science at the
University of Washington.

## References

  * Peccei and Quinn [1977] R. Peccei and H. R. Quinn, Phys. Rev. Lett. 38, 1440 (1977).
  * Weinberg [1978] S. Weinberg, Phys. Rev. Lett. 40, 223 (1978).
  * Wilczek [1978] F. Wilczek, Phys. Rev. Lett. 40, 279 (1978).
  * Preskill  _et al._ [1983] J. Preskill, M. B. Wise, and F. Wilczek, Phys. Lett. B 120, 127 (1983).
  * Abbott and Sikivie [1983] L. F. Abbott and P. Sikivie, Phys. Lett. B 120, 133 (1983).
  * Dine and Fischler [1983] M. Dine and W. Fischler, Phys. Lett. B 120, 137 (1983).
  * Duffy and van Bibber [2009] L. D. Duffy and K. van Bibber, New J. Phys. 11, 105008 (2009), arXiv:0904.3346 [hep-ph] .
  * Bertone  _et al._ [2005] G. Bertone, D. Hooper, and J. Silk, Phys. Rept. 405, 279 (2005), arXiv:hep-ph/0404175 .
  * Garrett and Duda [2011] K. Garrett and G. Duda, Adv. Astron. 2011, 968283 (2011), arXiv:1006.2483 [hep-ph] .
  * Buschmann  _et al._ [2022a] M. Buschmann, C. Dessert, J. W. Foster, A. J. Long, and B. R. Safdi, Phys. Rev. Lett. 128, 091102 (2022a), arXiv:2111.09892 [hep-ph] .
  * Carenza  _et al._ [2019] P. Carenza, T. Fischer, M. Giannotti, G. Guo, G. Martínez-Pinedo, and A. Mirizzi, JCAP 10, 016 (2019), [Erratum: JCAP 05, E01 (2020)], arXiv:1906.11844 [hep-ph] .
  * Di Luzio  _et al._ [2018] L. Di Luzio, F. Mescia, E. Nardi, P. Panci, and R. Ziegler, Phys. Rev. Lett. 120, 261803 (2018), arXiv:1712.04940 [hep-ph] .
  * Badziak and Harigaya [2023] M. Badziak and K. Harigaya, JHEP 06, 014 (2023), arXiv:2301.09647 [hep-ph] .
  * Ayala  _et al._ [2014] A. Ayala, I. Domínguez, M. Giannotti, A. Mirizzi, and O. Straniero, Phys. Rev. Lett. 113, 191302 (2014), arXiv:1406.6053 [astro-ph.SR] .
  * Dolan  _et al._ [2022] M. J. Dolan, F. J. Hiskens, and R. R. Volkas, JCAP 10, 096 (2022), arXiv:2207.03102 [hep-ph] .
  * Armengaud  _et al._ [2019] E. Armengaud _et al._ (IAXO), JCAP 06, 047 (2019), arXiv:1904.09155 [hep-ph] .
  * Baryakhtar  _et al._ [2018] M. Baryakhtar, J. Huang, and R. Lasenby, Phys. Rev. D 98, 035006 (2018), arXiv:1803.11455 [hep-ph] .
  * Schütte-Engel  _et al._ [2021] J. Schütte-Engel, D. J. E. Marsh, A. J. Millar, A. Sekine, F. Chadha-Day, S. Hoof, M. N. Ali, K.-C. Fong, E. Hardy, and L. Šmejkal, JCAP 08, 066 (2021), arXiv:2102.05366 [hep-ph] .
  * Liu  _et al._ [2022] J. Liu _et al._ (BREAD), Phys. Rev. Lett. 128, 131801 (2022), arXiv:2111.12103 [physics.ins-det] .
  * [20] “Desy, brass: Broadband radiometric axion searches,” http://www.iexp.uni-hamburg.de/groups/astroparticle/brass/brassweb.htm, accessed: 2023.
  * Aja  _et al._ [2022] B. Aja _et al._ , JCAP 11, 044 (2022), arXiv:2206.02980 [hep-ex] .
  * McAllister  _et al._ [2017] B. T. McAllister, G. Flower, J. Kruger, E. N. Ivanov, M. Goryachev, J. Bourhill, and M. E. Tobar, Phys. Dark Univ. 18, 67 (2017), arXiv:1706.00209 [physics.ins-det] .
  * Beurthey  _et al._ [2020] S. Beurthey _et al._ , (2020), arXiv:2003.10894 [physics.ins-det] .
  * Lawson  _et al._ [2019] M. Lawson, A. J. Millar, M. Pancaldi, E. Vitagliano, and F. Wilczek, Phys. Rev. Lett. 123, 141802 (2019), arXiv:1904.11872 [hep-ph] .
  * De Miguel and Hernández-Cabrera [2023] J. De Miguel and J. F. Hernández-Cabrera, (2023), arXiv:2303.03997 [hep-ph] .
  * Stern [2016] I. Stern, PoS ICHEP2016, 198 (2016), arXiv:1612.08296 [physics.ins-det] .
  * Brouwer  _et al._ [2022] L. Brouwer _et al._ (DMRadio), Phys. Rev. D 106, 103008 (2022), arXiv:2204.13781 [hep-ex] .
  * Berlin  _et al._ [2021] A. Berlin, R. T. D’Agnolo, S. A. R. Ellis, and K. Zhou, Phys. Rev. D 104, L111701 (2021), arXiv:2007.15656 [hep-ph] .
  * Davis [1986] R. L. Davis, Phys. Lett. B 180, 225 (1986).
  * Buschmann  _et al._ [2022b] M. Buschmann, J. W. Foster, A. Hook, A. Peterson, D. E. Willcox, W. Zhang, and B. R. Safdi, Nature Commun. 13, 1049 (2022b), arXiv:2108.05368 [hep-ph] .
  * Gorghetto  _et al._ [2021] M. Gorghetto, E. Hardy, and G. Villadoro, SciPost Phys. 10, 050 (2021), arXiv:2007.04990 [hep-ph] .
  * Gorghetto  _et al._ [2018] M. Gorghetto, E. Hardy, and G. Villadoro, JHEP 07, 151 (2018), arXiv:1806.04677 [hep-ph] .
  * Co  _et al._ [2019] R. T. Co, E. Gonzalez, and K. Harigaya, JHEP 05, 162 (2019), arXiv:1812.11186 [hep-ph] .
  * Papageorgiou  _et al._ [2023] A. Papageorgiou, P. Quílez, and K. Schmitz, JHEP 01, 169 (2023), arXiv:2206.01129 [hep-ph] .
  * Linde [1988] A. D. Linde, Phys. Lett. B 201, 437 (1988).
  * Tegmark  _et al._ [2006] M. Tegmark, A. Aguirre, M. Rees, and F. Wilczek, Phys. Rev. D 73, 023505 (2006), arXiv:astro-ph/0511774 .
  * Freivogel [2010] B. Freivogel, JCAP 03, 021 (2010), arXiv:0810.0703 [hep-th] .
  * Marsh [2016] D. J. E. Marsh, Phys. Rept. 643, 1 (2016), arXiv:1510.07633 [astro-ph.CO] .
  * Arvanitaki  _et al._ [2020] A. Arvanitaki, S. Dimopoulos, M. Galanis, L. Lehner, J. O. Thompson, and K. Van Tilburg, Phys. Rev. D 101, 083014 (2020), arXiv:1909.11665 [astro-ph.CO] .
  * Co  _et al._ [2018] R. T. Co, L. J. Hall, and K. Harigaya, Phys. Rev. Lett. 120, 211602 (2018), arXiv:1711.10486 [hep-ph] .
  * Harigaya and Leedom [2020] K. Harigaya and J. M. Leedom, JHEP 06, 034 (2020), arXiv:1910.04163 [hep-ph] .
  * Co  _et al._ [2020] R. T. Co, L. J. Hall, and K. Harigaya, Phys. Rev. Lett. 124, 251802 (2020), arXiv:1910.14152 [hep-ph] .
  * Kawasaki  _et al._ [2015] M. Kawasaki, K. Saikawa, and T. Sekiguchi, Phys. Rev. D 91, 065014 (2015), arXiv:1412.0789 [hep-ph] .
  * Hiramatsu  _et al._ [2011] T. Hiramatsu, M. Kawasaki, and K. Saikawa, JCAP 08, 030 (2011), arXiv:1012.4558 [astro-ph.CO] .
  * Hiramatsu  _et al._ [2013] T. Hiramatsu, M. Kawasaki, K. Saikawa, and T. Sekiguchi, JCAP 01, 001 (2013), arXiv:1207.3166 [hep-ph] .
  * Ringwald and Saikawa [2016] A. Ringwald and K. Saikawa, Phys. Rev. D 93, 085031 (2016), [Addendum: Phys.Rev.D 94, 049908 (2016)], arXiv:1512.06436 [hep-ph] .
  * Harigaya and Kawasaki [2018] K. Harigaya and M. Kawasaki, Phys. Lett. B 782, 1 (2018), arXiv:1802.00579 [hep-ph] .
  * Caputo and Reig [2019] A. Caputo and M. Reig, Phys. Rev. D 100, 063530 (2019), arXiv:1905.13116 [hep-ph] .
  * Arvanitaki  _et al._ [2010] A. Arvanitaki, S. Dimopoulos, S. Dubovsky, N. Kaloper, and J. March-Russell, Phys. Rev. D 81, 123530 (2010), arXiv:0905.4720 [hep-th] .
  * Ho  _et al._ [2018] S.-Y. Ho, K. Saikawa, and F. Takahashi, JCAP 10, 042 (2018), arXiv:1806.09551 [hep-ph] .
  * Wantz and Shellard [2010] O. Wantz and E. P. S. Shellard, Phys. Rev. D 82, 123508 (2010), arXiv:0910.1066 [astro-ph.CO] .
  * Cyncynates  _et al._ [2022] D. Cyncynates, T. Giurgica-Tiron, O. Simon, and J. O. Thompson, Phys. Rev. D 105, 055005 (2022), arXiv:2109.09755 [hep-ph] .
  * Grilli di Cortona  _et al._ [2016] G. Grilli di Cortona, E. Hardy, J. Pardo Vega, and G. Villadoro, JHEP 01, 034 (2016), arXiv:1511.02867 [hep-ph] .
  * Du  _et al._ [2018] N. Du _et al._ (ADMX), Phys. Rev. Lett. 120, 151301 (2018), arXiv:1804.05750 [hep-ex] .
  * Boutan  _et al._ [2018] C. Boutan _et al._ (ADMX), Phys. Rev. Lett. 121, 261302 (2018), arXiv:1901.00920 [hep-ex] .
  * Braine  _et al._ [2020] T. Braine _et al._ (ADMX), Phys. Rev. Lett. 124, 101303 (2020), arXiv:1910.08638 [hep-ex] .
  * Crisosto  _et al._ [2020] N. Crisosto, P. Sikivie, N. S. Sullivan, D. B. Tanner, J. Yang, and G. Rybka, Phys. Rev. Lett. 124, 241101 (2020), arXiv:1911.05772 [astro-ph.CO] .
  * Bartram  _et al._ [2021] C. Bartram _et al._ (ADMX), Phys. Rev. Lett. 127, 261803 (2021), arXiv:2110.06096 [hep-ex] .
  * Bartram  _et al._ [2023] C. Bartram _et al._ (ADMX), Rev. Sci. Instrum. 94, 2882903 (2023), arXiv:2110.10262 [hep-ex] .
  * Lee  _et al._ [2020] S. Lee, S. Ahn, J. Choi, B. R. Ko, and Y. K. Semertzidis, Phys. Rev. Lett. 124, 101802 (2020), arXiv:2001.05102 [hep-ex] .
  * Jeong  _et al._ [2020] J. Jeong, S. Youn, S. Bae, J. Kim, T. Seong, J. E. Kim, and Y. K. Semertzidis, Phys. Rev. Lett. 125, 221302 (2020), arXiv:2008.10141 [hep-ex] .
  * Kwon  _et al._ [2021] O. Kwon _et al._ (CAPP), Phys. Rev. Lett. 126, 191802 (2021), arXiv:2012.10764 [hep-ex] .
  * Lee  _et al._ [2022] Y. Lee, B. Yang, H. Yoon, M. Ahn, H. Park, B. Min, D. Kim, and J. Yoo, Phys. Rev. Lett. 128, 241805 (2022), arXiv:2206.08845 [hep-ex] .
  * Kim  _et al._ [2023] J. Kim _et al._ , Phys. Rev. Lett. 130, 091602 (2023), arXiv:2207.13597 [hep-ex] .
  * Yi  _et al._ [2023] A. K. Yi _et al._ , Phys. Rev. Lett. 130, 071002 (2023), arXiv:2210.10961 [hep-ex] .
  * Adair  _et al._ [2022] C. M. Adair _et al._ , Nature Commun. 13, 6180 (2022), arXiv:2211.02902 [hep-ex] .
  * De Panfilis  _et al._ [1987] S. De Panfilis, A. C. Melissinos, B. E. Moskowitz, J. T. Rogers, Y. K. Semertzidis, W. Wuensch, H. J. Halama, A. G. Prodell, W. B. Fowler, and F. A. Nezrick, Phys. Rev. Lett. 59, 839 (1987).
  * Hagmann  _et al._ [1990] C. Hagmann, P. Sikivie, N. S. Sullivan, and D. B. Tanner, Phys. Rev. D 42, 1297 (1990).
  * Zhong  _et al._ [2018] L. Zhong _et al._ (HAYSTAC), Phys. Rev. D 97, 092001 (2018), arXiv:1803.03690 [hep-ex] .
  * Backes  _et al._ [2021] K. M. Backes _et al._ (HAYSTAC), Nature 590, 238 (2021), arXiv:2008.01853 [quant-ph] .
  * Jewell  _et al._ [2023] M. J. Jewell _et al._ (HAYSTAC), Phys. Rev. D 107, 072007 (2023), arXiv:2301.09721 [hep-ex] .
  * Alesini  _et al._ [2019] D. Alesini _et al._ , Phys. Rev. D 99, 101101 (2019), arXiv:1903.06547 [physics.ins-det] .
  * Alesini  _et al._ [2021] D. Alesini _et al._ , Phys. Rev. D 103, 102004 (2021), arXiv:2012.09498 [hep-ex] .
  * Alesini  _et al._ [2022] D. Alesini _et al._ , Phys. Rev. D 106, 052007 (2022), arXiv:2208.12670 [hep-ex] .
  * Quiskamp  _et al._ [2022] A. P. Quiskamp, B. T. McAllister, P. Altin, E. N. Ivanov, M. Goryachev, and M. E. Tobar, Sci. Adv. 8, abq3765 (2022), arXiv:2203.12152 [hep-ex] .
  * Noordhuis  _et al._ [2022] D. Noordhuis, A. Prabhu, S. J. Witte, A. Y. Chen, F. Cruz, and C. Weniger, (2022), arXiv:2209.09917 [hep-ph] .
  * O’Hare [2020] C. O’Hare, “cajohare/axionlimits: Axionlimits,” https://cajohare.github.io/AxionLimits/ (2020).
  * Kivel  _et al._ [2022] A. Kivel, J. Laux, and F. Yu, JHEP 11, 088 (2022), arXiv:2207.08740 [hep-ph] .
  * Kim [1979] J. E. Kim, Phys. Rev. Lett. 43, 103 (1979).
  * Shifman  _et al._ [1980] M. A. Shifman, A. I. Vainshtein, and V. I. Zakharov, Nucl. Phys. B 166, 493 (1980).
  * Daido  _et al._ [2015] R. Daido, N. Kitajima, and F. Takahashi, Phys. Rev. D 92, 063512 (2015), arXiv:1505.07670 [hep-ph] .
  * Daido  _et al._ [2016] R. Daido, N. Kitajima, and F. Takahashi, Phys. Rev. D 93, 075027 (2016), arXiv:1510.06675 [hep-ph] .