# Surge Phenomenon in Optimal Learning Rate and Batch Size Scaling

Shuaipeng Li∗,1,†, Penghao Zhao∗,2, Hailin Zhang∗,2, Xingwu Sun1,3,
Hao Wu1, Dian Jiao1, Weiyan Wang1, Chengjun Liu1, Zheng Fang1,
Jinbao Xue1, Yangyu Tao1, Bin Cui2,†, Di Wang1,†
1 Tencent Hunyuan, 2 Peking University, 3 University of Macau

###### Abstract

In current deep learning tasks, Adam-style optimizers—such as Adam, Adagrad,
RMSprop, Adafactor, and Lion—have been widely used as alternatives to SGD-
style optimizers. These optimizers typically update model parameters using the
sign of gradients, resulting in more stable convergence curves. The learning
rate and the batch size are the most critical hyperparameters for optimizers,
which require careful tuning to enable effective convergence. Previous
research has shown that the optimal learning rate increases linearly (or
follows similar rules) with batch size for SGD-style optimizers. However, this
conclusion is not applicable to Adam-style optimizers. In this paper, we
elucidate the connection between optimal learning rates and batch sizes for
Adam-style optimizers through both theoretical analysis and extensive
experiments. First, we raise the scaling law between batch sizes and optimal
learning rates in the “sign of gradient” case, in which we prove that the
optimal learning rate first rises and then falls as the batch size increases.
Moreover, the peak value of the surge will gradually move toward the larger
batch size as training progresses. Second, we conduct experiments on various
CV and NLP tasks and verify the correctness of the scaling law.

††∗Equal contribution. †Corresponding author.

##  1 Introduction

Deep learning techniques, initiated by Stochastic Gradient Descent (SGD)
learning on large datasets, have significantly revolutionized various real-
world applications [1]. Over the past decade, numerous optimizers, such as
momentum [2], Adagrad [3], ADADELTA [4], RMSprop [5], Adam [6], Adafactor [7],
and Lion [8], have been introduced to stabilize the iterative learning process
and expedite convergence. Among them, the Adam optimizer is the most widely
used across various domains including Computer Vision (CV) [9, 10, 11],
Natural Language Processing (NLP) [12, 13, 14] and many others [15, 16]. It
retains the first and second moment information of parameters to facilitate
adaptive learning step size. Unlike SGD-style optimizers that use the raw
gradient to determine the learning direction and step size, Adam and its
variants (Adagrad, RMSprop, Lion, etc.) employ the sign of gradient for this
purpose, thereby ensuring greater robustness [17].

Beyond specific hyper-parameters in optimizer configurations, the batch size
and the learning rate are the most critical hyperparameters influencing
convergence. As the scale of training datasets in various workloads (e.g. CV
[18, 19], NLP [20, 14], and others [21, 22]) continues to grow, there is an
increasing demand for large batch size training across multiple data parallel
workers. However, large batch training presents great challenges for robust
training and meticulous tuning. The learning rate, which determines the actual
step size in each learning iteration, is highly dependent on the batch size
used. Prior research has explored methods to determine an optimal learning
rate according to the batch size in scenarios utilizing SGD optimizers,
including square root scaling [23], linear scaling [18, 24], and others [25].
Among these, the empirical model focusing on large batch training [25] yields
convincing results in both theoretical and empirical contexts, proposing the
following rule to depict the relationship between the optimal learning rate
and the batch size:

$\epsilon_{opt}(B)=\frac{\epsilon_{max}}{1+\frac{\mathcal{B}_{noise}}{B}}$ (1)

Figure 1: The relationship between the optimal learning rate and the batch
size is different between Adam and SGD. The orange line represents the
tendency of the optimal learning rate to converge to a non-zero value when the
batch size is large enough.

For Adam-style optimizers, though existing works also provide some
approximation [25, 24], they fail to capture the true scaling law of optimal
learning rates with batch sizes. As shown in Figure 1, we present a curve that
simulates the true optimal learning rate for the Adam optimizer, which does
not monotonically increase with batch size; instead, it first rises then
falls, resembling a surge in the sea. In this paper, we aim to elucidate the
connection between optimal learning rates and batch sizes for Adam-style
optimizers. By following the notation from the empirical model [25] and
conducting a more in-depth theoretical analysis, we discover that the
relationship between the optimal learning rate and the batch size in the above
parameter update formular satisfies:

$\epsilon_{opt}(B)=\frac{\epsilon_{max}}{\frac{1}{2}(\sqrt{\frac{\mathcal{B}_{noise}}{B}}+\sqrt{\frac{B}{\mathcal{B}_{noise}}})},$
(2)

which is different from SGD. Here the meaning of $\mathcal{B}_{noise}$ is
consistent with papers of scaling laws [25, 26], representing the trade-off
point between training speed and data efficiency. When the batch size is equal
to $\mathcal{B}_{noise}$, the optimal learning rate reaches a local maximum in
accordance with Eq 2, and then gradually converges to a non-zero value. We
also prove that the previous conclusions about training speed and data
efficiency are still valid for Adam-style optimizers, and the variable
$\mathcal{B}_{noise}$ gradually increases as the training progresses. It is
worth noting that when $B\ll\mathcal{B}_{noise}$, for SGD, the scaling law of
optimal learning rates with batch sizes transitions into linear scaling,
consistent with previous conclusions [18, 24]:

$\epsilon_{opt}(B)\approx\frac{\epsilon_{max}}{\mathcal{B}_{noise}}B;$ (3)

while for Adam, the relationship transitions into square root scaling,
aligning with previous approximations [25, 24]:

$\epsilon_{opt}(B)\approx\frac{2\epsilon_{max}}{\sqrt{\mathcal{B}_{noise}}}\sqrt{B}.$
(4)

In addition to theoretical analysis, our extensive empirical study on various
CV and NLP workloads further validate the conclusions. The true optimal
learning rate, across different Adam configurations, exhibits a clear downward
trend after reaching its peak value as the batch size increases. This behavior
contradicts previous research, but demonstrates the correctness and
generalizability of our theory. The experiments also reveal a gradual increase
in the variable $\mathcal{B}_{noise}$, corresponding to the peak optimal
learning rate, as training progresses.

##  2 Theorems

###  2.1 Batch Size and Optimal Learning Rate

In this section, we theoretically derive the optimal learning rate for a given
batch size. Initially, we introduce an approximation of Adam-style optimizers.
Upon reviewing the Adam optimizer and its variants, we can find that their
main difference from SGD lies in using the sign of the gradient for updates in
each iteration, rather than the gradient itself:

$\theta_{i+1}=\theta_{i}-\epsilon\cdot sign(G_{est}),$ (5)

where $\theta_{t}$ is the parameter at time $t$, $G_{est}$ is the gradient
estimated via mini-batch, and $\epsilon$ is the learning rate. As the batch
size increases, the expected value of the update amount tends to saturate. For
example, assuming that the mean value of the gradient is positive, when the
accumulated gradient of the mini-batch is positive, increasing the batch size
will have no contribution to the signed update amount. This is significantly
different from the behavior of SGD where the larger the batch size, the more
accurate the gradient estimate. In Appendix A, we provide a detailed
discussion on this approximation for the Adam optimizer. Next, we derive the
optimal learning rate that maximizes the loss improvement. And then we
establish a lemma that addresses the optimal learning rate given an estimated
mini-batch gradient:

######  Lemma 1.

Suppose that we are updating the parameter $\theta$ using the mini-batch
gradient $V$, with the true gradient being $G$ and the true Hessian being $H$.
Then the optimal learning rate that maximizes the decrease in loss is:

$\displaystyle\epsilon_{opt}\equiv argmax_{\epsilon}\ \mathbb{E}[\Delta
L]=\frac{G^{T}\mathbb{E}[V]}{tr[H\cdot
cov(V)]+\mathbb{E}[V]^{T}H\mathbb{E}[V]},$ (6)

and the corresponding loss improvement $\Delta L$ is:

$\Delta L_{opt}=\frac{G^{T}\mathbb{E}[V]}{2}\epsilon_{opt}.$ (7)

The proof is in Appendix B.

Now let’s consider the case where $V=sign(G_{est})$, and assuming that the
estimated gradient $G_{est}$ follows a Gaussian distribution. The Gaussian
distribution assumption is motivated by the following: if the minibatch size
is sufficiently large, we can invoke the Central Limit Theorem (CLT) and
approximate the distribution as Gaussian - a common assumption in previous
research [27, 28, 29, 30]. We have the following theorem:

######  Theorem 2.

Suppose the gradient of parameter $i$ for each sample follows a Gaussian
distribution with mean $\mu_{i}$ and variance $\sigma_{i}^{2}$, the expected
loss improvement is:

$\Delta
L_{opt}=\frac{1}{2}\frac{\sum_{i}\sum_{j}\mathcal{E}_{i}\mathcal{E}_{j}\mu_{i}\mu_{j}}{\sum_{i}(1-\mathcal{E}_{i}^{2})H_{i,i}+\sum_{i}\sum_{j}\mathcal{E}_{i}\mathcal{E}_{j}H_{i,j}},$
(8)

and the corresponding optimal learning rate is

$\displaystyle\epsilon_{opt}=\frac{\sum_{i}\mathcal{E}_{i}\mu_{i}}{\sum_{i}(1-\mathcal{E}_{i}^{2})H_{i,i}+\sum_{i}\sum_{j}\mathcal{E}_{i}\mathcal{E}_{j}H_{i,j}},$
(9)

where $\mathcal{E}_{i}$ is a function (derived from the Gauss error function)
with respect to the batch size $B$:

$\displaystyle\mathcal{E}_{i}(B)=\frac{2}{\sqrt{\pi}}\int_{0}^{\sqrt{\frac{B}{2}}\frac{\mu_{i}}{\sigma_{i}}}e^{-t^{2}}dt\approx\frac{\frac{\mu_{i}}{\sigma_{i}}}{\sqrt{\frac{\pi}{2B}+(\frac{\mu_{i}}{\sigma_{i}})^{2}}}.$
(10)

We prove the above theorem in Appendix C.

An important observation from the proof is that, not only is the covariance
matrix of $sign(G_{est})$ related to $B$, but its expected value also depends
on $B$. This implies that in Eq 6 the numerator is the first-order form of the
function about $B$, and the denominator is the second-order form of the
function about $B$:

$\begin{aligned} \epsilon(B)=\frac{\beta
f(B)}{f(B)^{2}+\gamma}=\frac{\beta}{f(B)+\frac{\gamma}{f(B)}}\end{aligned}.$
(11)

Therefore, the conclusion in the case of Adam cannot be derived by simply
following the form mentioned in [25]:

$\epsilon(B)\neq\frac{\epsilon_{*}}{(1+\frac{\mathcal{B}_{noise}}{B})^{\alpha}}.$
(12)

Then we aim to derive the specific expression for the optimal learning rate
with respect to the batch size through the following theorems.

######  Theorem 3.

When $B\ll\frac{\pi\sigma_{i}^{2}}{2\mu_{i}^{2}}$, the optimal learning rate
is a function with respect to batch size $B$:

$\begin{aligned}
\epsilon_{opt}(B)&\approx\frac{1}{\frac{1}{2}(\sqrt{\frac{\mathcal{B}_{noise}}{B}}+\sqrt{\frac{B}{\mathcal{B}_{noise}}})}\frac{\sqrt{\frac{\mathcal{B}_{noise}}{2\pi}}\sum_{i}\frac{\mu_{i}^{2}}{\sigma_{i}}}{\sum_{i}H_{i,i}}\leq\frac{\sqrt{\frac{\mathcal{B}_{noise}}{2\pi}}\sum_{i}\frac{\mu_{i}^{2}}{\sigma_{i}}}{\sum_{i}H_{i,i}}\end{aligned},$
(13)

where $\mathcal{B}_{noise}$ is a variable unrelated to batch size $B$:

$\begin{aligned}
\mathcal{B}_{noise}&=\frac{\pi\sum_{i}H_{i,i}}{2\sum_{i}\sum_{j}\begin{cases}\frac{\mu_{i}\mu_{j}}{\sigma_{i}\sigma_{j}}&i\neq
j\\\ 0&i=j\end{cases}H_{i,j}}\end{aligned}.$ (14)

Defining $B_{peak}$ as the batch size at which the optimal learning rate
reaches a peak value, it is obvious that:

$B_{peak}=\mathcal{B}_{noise}.$ (15)

The peak value is:

$\epsilon_{max}=\frac{\sqrt{\frac{\mathcal{B}_{noise}}{2\pi}}\sum_{i}\frac{\mu_{i}^{2}}{\sigma_{i}}}{\sum_{i}H_{i,i}}.$
(16)

We prove the theorem in Appendix D. From the theorem we can finally get Eq 2,
which implies that there is an interval where the batch size becomes larger
and the optimal learning rate needs to be reduced. Considering that
$\frac{\pi\sigma_{i}^{2}}{2\mu_{i}^{2}}$ is much larger than normal batch
sizes in research and industry (as shown in Figure 2), this theorem can cover
most of the scenarios. To make the conclusion more comprehensive, we also
derive the following theorem:

######  Theorem 4.

When $B\gg\frac{\pi\sigma_{i}^{2}}{2\mu_{i}^{2}}$, the optimal learning rate
becomes:

$\begin{aligned}
\epsilon_{opt}=\frac{\sum_{i}|\mu_{i}|}{\sum_{i}\sum_{j}sign(\mu_{i})sign(\mu_{j})H_{i,j}}\end{aligned}.$
(17)

We prove the theorem in Appendix E.

Therefore, when B increases infinitely, the optimal learning rate will
eventually converge to a non-zero value. If we make an (unrealistic)
assumption that $\frac{\mu_{i}}{\sigma_{i}}\approx sign(\mu_{i})$, we will
find that the lower bound of $\epsilon_{max}$ in Theorem 3 will become the one
in Theorem 4, which means that the local peak value of the optimal learning
rate is larger than the final convergence value. However, considering that the
variance of the gradient in the later stages of training is very small, which
makes the above conclusion $\frac{\mu_{i}}{\sigma_{i}}\approx sign(\mu_{i})$
difficult to establish, so the stable value in the later stages of training is
more likely to exceed the local maximum. We provide a reference curve in
Figure 1.

###  2.2 Data/Time Efficiency Trade-off

Following the empirical model for large-batch training [25], we also review
the trade-off between data and time efficiency during batch size selection. We
have the following theorem:

######  Theorem 5.

When $B\ll\frac{\pi\sigma_{i}^{2}}{2\mu_{i}^{2}}$, the derived loss
improvement with respect to the batch size is

$\Delta L_{opt}(B)=\frac{\Delta L_{max}}{1+\frac{\mathcal{B}_{noise}}{B}},$
(18)

where $\Delta L_{max}$ is defined as

$\begin{aligned} \Delta
L_{max}=\frac{\sum_{i}\sum_{j}\frac{\mu_{i}^{2}\mu_{j}^{2}}{\sigma_{i}\sigma_{j}}}{2\sum_{i}\sum_{j}\begin{cases}\frac{\mu_{i}\mu_{j}}{\sigma_{i}\sigma_{j}}&i\neq
j\\\ 0&i=j\end{cases}H_{i,j}}\end{aligned},$ (19)

We prove the theorem in Appendix F. This result aligns with the conclusion
drawn in the SGD situation [25], indicating that many related conclusions also
remain valid.

The relationship between training speed and data efficiency derived from the
SGD optimizer still holds (see the Eq 2.11 and the Appendix D in [25] for
details):

$(\frac{S}{S_{min}}-1)(\frac{E}{E_{min}}-1)=1.$ (20)

Here $S_{(min)}$ represents training speed, the actual (minimum) possible
number of steps taken to reach a specified model performance; and $E_{(min)}$
represents data efficiency, the actual (minimum) possible number of training
examples processed to reach that same level of performance. And because

$B_{peak}=\mathcal{B}_{noise}\approx\mathcal{B}_{crit}=\frac{E_{min}}{S_{min}}\approx\frac{B_{*}}{L^{\frac{1}{\alpha_{B}}}},$
(21)

is valid as referenced in the Eq 2.12 in [25] and Eq 1.4 in [26],
$\mathcal{B}_{noise}$ is not only the local maximum of the optimal learning
rate, but also the balance point between training speed and data efficiency.
Moreover, as the training progresses, the loss decreases and $B_{peak}$ will
gradually becomes larger.

###  2.3 Summary

In this section, we have drawn several conclusions from our theoretical
analysis, which are summarized as follows:

  1. 1.

As the batch size increases, the optimal learning rate demonstrates a
decreasing trend within a specified range (Eq 2).

  2. 2.

The batch size that corresponds to the local maximum optimal learning rate is
consistent with the balance point of training speed and data efficiency (Eq
21). As the training progresses and the loss decreases, $B_{peak}$ will
gradually becomes larger.

##  3 Experiments

In this section, we carry out a series of experiments to corroborate the
theoretical scaling law we proposed in Section 2 and detail the experimental
workloads and configurations in Section 3.1. The process for deriving the
estimated variables from our theory is elucidated in Section 3.2. We also
showcase and dissect the applicability of our scaling law across a variety of
workloads in Section 3.3.

###  3.1 Experimental Setup

Workloads. In our empirical study, we incorporate three open-source workloads
that are extensively utilized: (1) training a 5-layer CNN model on the
Fashion-MNIST [31], which is a typical CV test case to start with. It consists
of 60000 28x28 grayscale images in 10 classes; (2) training a ResNet-18 model
[32] on the Tiny-ImageNet dataset [33], which contains 100000 images of 200
classes (500 for each class) downsized to 64×64 colored images. In each epoch
we train the model with random 10k samples to reduce the overall complexity;
(3) training a Transformer model [12] (simplified DistilGPT2 [34]) on the
ELI5-Category dataset [35], which is a smaller but newer and categorized
version of the original ELI5 dataset [36]. It contains 10.5k complex, diverse
questions that require explanatory multi-sentence answers. These workloads are
popular in both academia and industry, covering typical deep learning tasks in
the domains of CV and NLP.

Batch sizes and learning rates. To showcase the optimal learning rate for each
batch size configuration, we leverage a grid-search-style experiments set.
Each point in the grid search corresponds to a certain round with the same
configuration but a different random number seed. The start point, stop point,
and the interval of different workloads are listed in Table 1.

Table 1: Grid search configurations. | Adam | Learning Rate | Batch Size |
---|---|---|---|---
Workload | $\beta_{1}$ | $\beta_{2}$ | Start | Stop | Step | Start | Stop | Step | Round
CNN | 0.9 | 0.999 | 1e-4 | 1e-3 | 1e-4 | 1 | 12 | 1 | 100
CNN | 0.9 | 0.999 | 1e-4 | 1e-3 | 1e-4 | 64 | 1164 | 100 | 100
DistilGPT2 | 0.9 | 0.999 | 1e-5 | 1.09e-3 | 1.2e-4 | 4 | 114 | 10 | 30
DistilGPT2 | 0.0 | 0.0 | 1e-5 | 5.5e-4 | 6e-5 | 4 | 114 | 10 | 30
ResNet18 | 0.0 | 0.0 | 1e-4 | 7.876e-4 | 7.65e-5 | 16 | 376 | 33 | 100

Hyper-parameters. Since we derive the theorems on Adam-style optimizers, we
conduct experiments using the Adam optimizer. We experiment on both the "sign
of gradient" configuration ($\beta_{1}=0$, $\beta_{2}=0$) and the default
hyper-parameters ($\beta_{1}=0.9$, $\beta_{2}=0.999$), as shown in Table 1.

Hardware environment. We execute each round of experiments utilizing an NVIDIA
A100 card. The training time of each round for the datasets are approximately
1 hour for Fashion-MNIST, 1.5 hours for TinyImageNet, and 2 hours for
ELI5-Category. Given our primary focus on the convergence process, the
specific hardware environment does not matter in our experiments. Both our
theoretical analysis and empirical study can be generalized to other hardware
settings.

###  3.2 Variable Estimation

We try to estimate the value of $\mathcal{B}_{noise}$ and the expectation of
$\epsilon_{max}$ through curve fitting. After using Eq 21 to simplify Eq 20
(see Appendix G for details), we can record the actual possible number of
steps taken $S$ and the actual possible number of training examples processed
$E$ to reach a specified level of performance corresponding to the optimal
learning rate of each batch size in the grid search results, and then perform
linear fitting to obtain the estimated value of $\mathcal{B}_{noise}$:

$\frac{1}{S}=-\mathcal{B}_{noise}\frac{1}{E}+\frac{1}{S_{min}}$ (22)

Subsequently, we use the optimal learning rate and batch size of the grid
search results to estimate the max optimal learning rate of Adam-style
$\mathbb{E}[\epsilon_{max}]_{Adam}$:

$\mathbb{E}[\epsilon_{max}]_{Adam}=\mathbb{E}[\frac{\epsilon_{opt}}{2}(\sqrt{\frac{\mathcal{B}_{noise}}{B}}+\sqrt{\frac{B}{\mathcal{B}_{noise}}})]$
(23)

and SGD-style $\mathbb{E}[\epsilon_{max}]_{SGD}$:

$\mathbb{E}[\epsilon_{max}]_{SGD}=\mathbb{E}[\epsilon_{opt}(1+\frac{\mathcal{B}_{noise}}{B})^{\alpha}]$
(24)

Previous research [25] represents the SGD optimizer and the Adam optimizer
using Eq 24 with $\alpha=1$ and $\alpha=0.5$, respectively. We include these
fitted curves as comparisons in the following section.

###  3.3 Results

Following Section 3.2, we first estimate the variables and fit the curves
using observations, then conduct grid-search-style experiments for learning
rates and batch sizes.

Figure 2, 3, 4 illustrate the experimental results of CNN-FashionMNIST,
ResNet18-TinyImageNet, and DistilGPT2-ELI5Category, respectively. Each figure
is divided into two parts: the right subfigure presents the estimation, while
the left subfigure depicts the grid-search results. In order to estimate the
variables, we train models from scratch using different learning rates and
batch sizes, then record the number of steps $S$ and examples $E$ in each
experiment to achieve an equivalent training loss. Using the recorded $S$ and
$E$, we fit the curve in the right subfigure and obtain the estimated
$\mathcal{B}_{noise}$. In the left subfigure, upon achieving the desired
training loss, all experiments continue to train the same number of steps. Any
subsequent decrease in training loss is represented through different colors,
as indicated in the color bar. For each batch size, we highlight the optimal
learning rate that results in the most significant reduction in training loss.
We also plot the batch size $\mathcal{B}_{noise}$ that corresponds to the peak
optimal learning rate, the fitted SGD curves with $\alpha=0.5$ and $\alpha=1$
as derived from previous research [25], and the fitted Adam curve as derived
from our theorems.

((a)) Statistical histogram of $\frac{\pi\sigma_{i}^{2}}{2\mu_{i}^{2}}$ ((b))
The situation of small batch size in Theorem 3 ((c)) The situation of large
batch size in Theorem 4

Figure 2: Batch size versus optimal learning rate within the context of CNN
trained on FashionMNIST

For the CNN-FashionMNIST workload, we train exactly 10 more step after
achieving the desired training loss. As shown in Figure 2 (a), the batch size
bound $\frac{\pi\sigma_{i}^{2}}{2\mu_{i}^{2}}$ for Theorem 3 is around 800 in
this task. Given the simplicity of the CNN-FashionMNIST workload, commonly-
used batch sizes are usually smaller than the batch size bound. We plot the
situations corresponding to Theorem 3 and Theorem 4 in Figure 2 (b) and (c),
respectively. In both cases, the trend predicted by our theory is consistent
with the actual optimal learning rate performance, showing a declining range
at small batch sizes and a saturation range at large batch sizes. For the
ResNet18-TinyImageNet workload, we train 50 more steps after achieving the
desired training loss. We plot the figures for Theorem 3 at different achieved
training losses, which represent the progress of training, as shown in Figure
3. The observed optimal learning rates primarily exhibit a downward trend
after the batch size exceeds the estimated $\mathcal{B}_{noise}$. Although the
SGD curve with $\alpha=0.5$, which is claimed by [25] to represent the Adam
optimizer, serves as a good approximation in certain cases, it fails to
capture the peak optimal learning rate as our Adam curve does. Comparing the
red dashed lines in different figures, we can see that the estimated
$\mathcal{B}_{noise}$ gradually increases as the training progresses (i.e.
training loss decreases), which corroborates the second conclusion in Section
2.3.

((a)) $L_{training}=5.213\pm 0.001$ ((b)) $L_{training}=5.146\pm 0.001$ ((c))
$L_{training}=5.113\pm 0.001$ ((d)) $L_{training}=4.863\pm 0.001$

((e)) $L_{training}=4.741\pm 0.001$ ((f)) $L_{training}=4.633\pm 0.001$

Figure 3: The relationship between batch sizes and optimal learning rates
within the context of ResNet-18 trained on TinyImageNet. The red dashed line
accurately predicts the peak value, and as the training loss decreases, the
peak value gradually shifts to the right.

((a)) $\beta_{1}=0.0,\beta_{2}=0.0$ ((b)) $\beta_{1}=0.9,\beta_{2}=0.999$

Figure 4: The relationship between batch sizes and optimal learning rates
within the context of DistilGPT2 trained on Eli5Category.

For the DistilGPT2-Eli5Category workload, we train 50 more steps after
achieving the desired training loss. As shown in Figure 4, we test on two
distinct Adam configurations for Theorem 3: the first with $\beta_{1}=0.0$,
$\beta_{2}=0.0$, and the second with $\beta_{1}=0.9$, $\beta_{2}=0.999$. In
both configurations, promising learning rates that lead to a substantial
decrease in loss are consistent with our Adam curve. It is worth noting that
another curve, SGD with $\alpha=0.5$ [25], also provides a suitable
approximation in this scenario. These experiments demonstrate that our
theorems can be generalized to different optimizer configurations, validating
the analysis in Appendix A.

##  4 Discussion

We have carried out empirical studies on representative workloads using the
Adam optimizer. In real-world applications, there are numerous different
learning workloads. Other factors, beyond the scope of this paper, may
influence the learning process - the specific optimizer used, weight decay,
gradient clipping, etc. While we assert that our theorem can be applied to
numerous practical scenarios, it may not fully encompass all situations
involving intricate training configurations.

As one of our conclusions points out, the variable $\mathcal{B}_{noise}$ will
gradually increases as the training progresses. It is natural to implement
adaptive learning rates (and batch sizes) if possible, to speed up the
training process. As mentioned in [25], using adaptive batch sizes and warmed-
up learning rates brings considerable benefits. Fully exploring the potential
of batch size and learning rate scheduling requires meticulous design, which
we leave as future work.

##  5 Related Work

Aiming to accelerate convergence, our work analyzes the scaling law of optimal
learning rates with respect to batch sizes for Adam-style optimizers. Numerous
related studies have been proposed to enhance the convergence of deep learning
tasks by investigating optimal learning rates, developing new optimizers, and
analyzing gradient noise.

Earlier works have proposed various scaling laws to tune learning rates for
SGD-style optimizers, including square root scaling [23], linear scaling [18,
24], and others [25]. They also obtained a scaling law for Adam-style
optimizers [25, 24] through approximation, revealing a square root-like
relationship where the optimal learning rate monotonically increases with the
batch size. However, as illustrated in Section 1 and 2, their analysis holds
only for small batch sizes, whereas the true scaling law exhibits greater
complexity, with the optimal learning rate reaching a peak value at a balanced
batch size.

There are many meticulously designed optimizers for various tasks and
scenarios. First-order optimizers dominate nowadays deep learning models,
including adaptive methods [3, 4, 5, 6, 7], sign-based methods [17, 8], layer-
wise methods (for large-batch training) [37, 38]. Second-order optimizers [39,
40, 41], though with stronger theoretical guarantees, are not efficient for
large-scale models due to quadratic complexity with respect to the number of
parameters. Despite the emergence of new optimizers, empirical evidence
confirms that Adam has remained the most widely used and effective optimizer
over the past decade.

Our analysis is inspired by the empirical model of large-batch training [25],
which predicts the useful batch size using the gradient noise scale. Gradient
noise can help with learning rate determination [42], batch size selection
[43, 44], and gaining deeper insights into the convergence process [45, 46,
47, 48].

##  6 Conclusion

In this paper, we established a scaling law between optimal learning rates and
batch sizes for Adam-style optimizers. We theoretically proved that the
optimal learning rate initially increases and then decreases as the batch size
grows, and that the peak value of the surge represents a trade-off point
between training speed and data efficiency. Through extensive experiments, we
validated our theory on diverse deep learning models and datasets.

## References

  * [1] Yann LeCun, Yoshua Bengio, and Geoffrey Hinton.  Deep learning.  nature, 521(7553):436–444, 2015.
  * [2] Ilya Sutskever, James Martens, George Dahl, and Geoffrey Hinton.  On the importance of initialization and momentum in deep learning.  In International conference on machine learning, pages 1139–1147. PMLR, 2013.
  * [3] John Duchi, Elad Hazan, and Yoram Singer.  Adaptive subgradient methods for online learning and stochastic optimization.  Journal of machine learning research, 12(7), 2011.
  * [4] Matthew D Zeiler.  Adadelta: an adaptive learning rate method.  arXiv preprint arXiv:1212.5701, 2012.
  * [5] Tijmen Tieleman and Geoffrey Hinton.  Rmsprop: Divide the gradient by a running average of its recent magnitude. coursera: Neural networks for machine learning.  COURSERA Neural Networks Mach. Learn, 17, 2012.
  * [6] Diederik P Kingma and Jimmy Ba.  Adam: A method for stochastic optimization.  arXiv preprint arXiv:1412.6980, 2014.
  * [7] Noam Shazeer and Mitchell Stern.  Adafactor: Adaptive learning rates with sublinear memory cost.  In International Conference on Machine Learning, pages 4596–4604. PMLR, 2018.
  * [8] Xiangning Chen, Chen Liang, Da Huang, Esteban Real, Kaiyuan Wang, Hieu Pham, Xuanyi Dong, Thang Luong, Cho-Jui Hsieh, Yifeng Lu, et al.  Symbolic discovery of optimization algorithms.  Advances in Neural Information Processing Systems, 36, 2024.
  * [9] Aaron Van Den Oord, Oriol Vinyals, et al.  Neural discrete representation learning.  Advances in neural information processing systems, 30, 2017.
  * [10] Alexey Dosovitskiy, Lucas Beyer, Alexander Kolesnikov, Dirk Weissenborn, Xiaohua Zhai, Thomas Unterthiner, Mostafa Dehghani, Matthias Minderer, Georg Heigold, Sylvain Gelly, et al.  An image is worth 16x16 words: Transformers for image recognition at scale.  arXiv preprint arXiv:2010.11929, 2020.
  * [11] René Ranftl, Alexey Bochkovskiy, and Vladlen Koltun.  Vision transformers for dense prediction.  In Proceedings of the IEEE/CVF international conference on computer vision, pages 12179–12188, 2021.
  * [12] Ashish Vaswani, Noam Shazeer, Niki Parmar, Jakob Uszkoreit, Llion Jones, Aidan N Gomez, Łukasz Kaiser, and Illia Polosukhin.  Attention is all you need.  Advances in neural information processing systems, 30, 2017.
  * [13] Jacob Devlin, Ming-Wei Chang, Kenton Lee, and Kristina Toutanova.  Bert: Pre-training of deep bidirectional transformers for language understanding.  arXiv preprint arXiv:1810.04805, 2018.
  * [14] Hugo Touvron, Thibaut Lavril, Gautier Izacard, Xavier Martinet, Marie-Anne Lachaux, Timothée Lacroix, Baptiste Rozière, Naman Goyal, Eric Hambro, Faisal Azhar, et al.  Llama: Open and efficient foundation language models.  arXiv preprint arXiv:2302.13971, 2023.
  * [15] John Schulman, Filip Wolski, Prafulla Dhariwal, Alec Radford, and Oleg Klimov.  Proximal policy optimization algorithms.  arXiv preprint arXiv:1707.06347, 2017.
  * [16] Hailin Zhang, Yujing Wang, Qi Chen, Ruiheng Chang, Ting Zhang, Ziming Miao, Yingyan Hou, Yang Ding, Xupeng Miao, Haonan Wang, et al.  Model-enhanced vector index.  Advances in Neural Information Processing Systems, 36, 2024.
  * [17] Jeremy Bernstein, Yu-Xiang Wang, Kamyar Azizzadenesheli, and Animashree Anandkumar.  signsgd: Compressed optimisation for non-convex problems.  In International Conference on Machine Learning, pages 560–569. PMLR, 2018.
  * [18] Priya Goyal, Piotr Dollár, Ross Girshick, Pieter Noordhuis, Lukasz Wesolowski, Aapo Kyrola, Andrew Tulloch, Yangqing Jia, and Kaiming He.  Accurate, large minibatch sgd: Training imagenet in 1 hour.  arXiv preprint arXiv:1706.02677, 2017.
  * [19] Alec Radford, Jong Wook Kim, Chris Hallacy, Aditya Ramesh, Gabriel Goh, Sandhini Agarwal, Girish Sastry, Amanda Askell, Pamela Mishkin, Jack Clark, et al.  Learning transferable visual models from natural language supervision.  In International conference on machine learning, pages 8748–8763. PMLR, 2021.
  * [20] Josh Achiam, Steven Adler, Sandhini Agarwal, Lama Ahmad, Ilge Akkaya, Florencia Leoni Aleman, Diogo Almeida, Janko Altenschmidt, Sam Altman, Shyamal Anadkat, et al.  Gpt-4 technical report.  arXiv preprint arXiv:2303.08774, 2023.
  * [21] Hailin Zhang, Penghao Zhao, Xupeng Miao, Yingxia Shao, Zirui Liu, Tong Yang, and Bin Cui.  Experimental analysis of large-scale learnable vector storage compression.  Proceedings of the VLDB Endowment, 17(4):808–822, 2023.
  * [22] Hailin Zhang, Zirui Liu, Boxuan Chen, Yikai Zhao, Tong Zhao, Tong Yang, and Bin Cui.  Cafe: Towards compact, adaptive, and fast embedding for large-scale recommendation models.  Proceedings of the ACM on Management of Data, 2(1):1–28, 2024.
  * [23] Alex Krizhevsky.  One weird trick for parallelizing convolutional neural networks.  arXiv preprint arXiv:1404.5997, 2014.
  * [24] Diego Granziol, Stefan Zohren, and Stephen Roberts.  Learning rates as a function of batch size: A random matrix theory approach to neural network training.  Journal of Machine Learning Research, 23(173):1–65, 2022.
  * [25] Sam McCandlish, Jared Kaplan, Dario Amodei, and OpenAI Dota Team.  An empirical model of large-batch training.  arXiv preprint arXiv:1812.06162, 2018.
  * [26] Jared Kaplan, Sam McCandlish, Tom Henighan, Tom B Brown, Benjamin Chess, Rewon Child, Scott Gray, Alec Radford, Jeffrey Wu, and Dario Amodei.  Scaling laws for neural language models.  arXiv preprint arXiv:2001.08361, 2020.
  * [27] Stephan Mandt, Matthew Hoffman, and David Blei.  A variational analysis of stochastic gradient algorithms.  In International conference on machine learning, pages 354–363. PMLR, 2016.
  * [28] Stanisław Jastrzębski, Zachary Kenton, Devansh Arpit, Nicolas Ballas, Asja Fischer, Yoshua Bengio, and Amos Storkey.  Three factors influencing minima in sgd.  arXiv preprint arXiv:1711.04623, 2017.
  * [29] Qianxiao Li, Cheng Tai, and E Weinan.  Stochastic modified equations and adaptive stochastic gradient algorithms.  In International Conference on Machine Learning, pages 2101–2110. PMLR, 2017.
  * [30] Zhanxing Zhu, Jingfeng Wu, Bing Yu, Lei Wu, and Jinwen Ma.  The anisotropic noise in stochastic gradient descent: Its behavior of escaping from sharp minima and regularization effects.  arXiv preprint arXiv:1803.00195, 2018.
  * [31] Han Xiao, Kashif Rasul, and Roland Vollgraf.  Fashion-mnist: a novel image dataset for benchmarking machine learning algorithms.  CoRR, abs/1708.07747, 2017.
  * [32] Kaiming He, Xiangyu Zhang, Shaoqing Ren, and Jian Sun.  Deep residual learning for image recognition.  In Proceedings of the IEEE conference on computer vision and pattern recognition, pages 770–778, 2016.
  * [33] Jia Deng, Wei Dong, Richard Socher, Li-Jia Li, Kai Li, and Li Fei-Fei.  Imagenet: A large-scale hierarchical image database.  In 2009 IEEE conference on computer vision and pattern recognition, pages 248–255. Ieee, 2009.
  * [34] Victor Sanh, Lysandre Debut, Julien Chaumond, and Thomas Wolf.  Distilbert, a distilled version of bert: smaller, faster, cheaper and lighter.  In NeurIPS $EMC^{2}$ Workshop, 2019.
  * [35] Jingsong Gao, Qingren Zhou, and Rui Qiu.  ELI5-Category: a categorized open-domain qa dataset.  2021\.
  * [36] Angela Fan, Yacine Jernite, Ethan Perez, David Grangier, Jason Weston, and Michael Auli.  Eli5: Long form question answering.  arXiv preprint arXiv:1907.09190, 2019.
  * [37] Yang You, Zhao Zhang, Cho-Jui Hsieh, James Demmel, and Kurt Keutzer.  Imagenet training in minutes.  In Proceedings of the 47th international conference on parallel processing, pages 1–10, 2018.
  * [38] Yang You, Jing Li, Sashank Reddi, Jonathan Hseu, Sanjiv Kumar, Srinadh Bhojanapalli, Xiaodan Song, James Demmel, Kurt Keutzer, and Cho-Jui Hsieh.  Large batch optimization for deep learning: Training bert in 76 minutes.  arXiv preprint arXiv:1904.00962, 2019.
  * [39] Zhewei Yao, Amir Gholami, Sheng Shen, Mustafa Mustafa, Kurt Keutzer, and Michael Mahoney.  Adahessian: An adaptive second order optimizer for machine learning.  In proceedings of the AAAI conference on artificial intelligence, volume 35, pages 10665–10673, 2021.
  * [40] Jimmy Ba, Roger Grosse, and James Martens.  Distributed second-order optimization using kronecker-factored approximations.  In International conference on learning representations, 2022.
  * [41] Hong Liu, Zhiyuan Li, David Hall, Percy Liang, and Tengyu Ma.  Sophia: A scalable stochastic second-order optimizer for language model pre-training.  arXiv preprint arXiv:2305.14342, 2023.
  * [42] Tom Schaul, Sixin Zhang, and Yann LeCun.  No more pesky learning rates.  In International conference on machine learning, pages 343–351. PMLR, 2013.
  * [43] Lukas Balles, Javier Romero, and Philipp Hennig.  Coupling adaptive batch sizes with learning rates.  arXiv preprint arXiv:1612.05086, 2016.
  * [44] Samuel L Smith and Quoc V Le.  A bayesian perspective on generalization and stochastic gradient descent.  arXiv preprint arXiv:1710.06451, 2017.
  * [45] Nitish Shirish Keskar, Dheevatsa Mudigere, Jorge Nocedal, Mikhail Smelyanskiy, and Ping Tak Peter Tang.  On large-batch training for deep learning: Generalization gap and sharp minima.  arXiv preprint arXiv:1609.04836, 2016.
  * [46] Ravid Shwartz-Ziv and Naftali Tishby.  Opening the black box of deep neural networks via information.  arXiv preprint arXiv:1703.00810, 2017.
  * [47] Umut Simsekli, Levent Sagun, and Mert Gurbuzbalaban.  A tail-index analysis of stochastic gradient noise in deep neural networks.  In International Conference on Machine Learning, pages 5827–5837. PMLR, 2019.
  * [48] Christopher J Shallue, Jaehoon Lee, Joseph Antognini, Jascha Sohl-Dickstein, Roy Frostig, and George E Dahl.  Measuring the effects of data parallelism on neural network training.  Journal of Machine Learning Research, 20(112):1–49, 2019.

##  Appendix A Parameter Update Amount in the Adam Optimizer

The update amount in the Adam optimizer consists of two parts, the first
moment:

$\displaystyle m_{t}=\beta_{1}m_{t-1}+(1-\beta_{1})G_{est,t}$ (25)
$\displaystyle\widehat{m_{t}}=\frac{m_{t}}{1-\beta_{1}^{t}}$

and the second moment:

$\displaystyle v_{t}=\beta_{2}v_{t-1}+(1-\beta_{2})G_{est,t}^{2}$ (26)
$\displaystyle\widehat{v_{t}}=\frac{v_{t}}{1-\beta_{2}^{t}}$

where $m_{0}=0$ and $v_{0}=0$. The final update amount is

$\displaystyle
V=\frac{\widehat{m_{t}}}{\sqrt{\widehat{v_{t}}}+\epsilon_{Adam}}=\frac{\frac{1-\beta_{1}}{1-\beta_{1}^{t}}\sum_{i}^{t}\beta_{1}^{t-i}G_{est,i}}{\sqrt{\frac{1-\beta_{2}}{1-\beta_{2}^{t}}\sum_{i}^{t}\beta_{2}^{t-i}G_{est,i}^{2}}+\epsilon_{Adam}}$
(27)

After ignoring the role of $\epsilon_{Adam}$, when $\beta_{1}\to 1$ and
$\beta_{2}\to 1$, Eq 27 transforms to

$\displaystyle
V=\frac{\frac{\sum_{i}^{t}G_{est,i}}{t}}{\sqrt{\frac{\sum_{i}^{t}G_{est,i}^{2}}{t}}}=\frac{\mathbb{E}_{t}[G_{est}]}{\sqrt{\mathbb{E}_{t}[G_{est}^{2}]}}=\frac{sign(\mathbb{E}_{t}[G_{est}])}{\sqrt{1+\frac{var_{t}(G_{est})}{\mathbb{E}_{t}[G_{est}]^{2}}}}$
(28)

The equation is obtained by using
$var_{t}(G_{est})=\mathbb{E}_{t}[G_{est}^{2}]-\mathbb{E}_{t}[G_{est}]^{2}$.
Note that the expected value $\mathbb{E}_{t}[G_{est}]$ is over the iteration
distribution, not over the data distribution. Obviously, when the variance of
$G_{est}$ is small, the update amount is approximately $sign(G_{est})$.

On the other hand, when $\beta_{1}\to 0$ and $\beta_{2}\to 0$, Eq 27 can be
simplified to

$\displaystyle V=\frac{G_{est}}{\sqrt{G_{est}^{2}}}=sign(G_{est})$ (29)

Therefore, we can approximate the parameter update amount of the Adam
optimizer as $sign(G_{est})$, without affecting the theoretical conclusion.

##  Appendix B Proof of Lemma 1

###### Proof.

We can approximate the true loss using a quadratic expansion in terms of
$\epsilon$ via Taylor expansion:

$L(\theta-\epsilon\cdot V)\approx L(\theta)-\epsilon
G^{T}V+\frac{1}{2}\epsilon^{2}V^{T}HV.$ (30)

Consider the expected value of loss improvement over a data distribution
$\rho(x)$ over data points $x$:

$\displaystyle\mathbb{E}[\Delta L]=\mathbb{E}[L(\theta)-L(\theta-\epsilon\cdot
V)]\approx\epsilon
G^{T}\mathbb{E}[V]-\frac{1}{2}\epsilon^{2}\mathbb{E}[V^{T}HV].$ (31)

By maximizing the expected loss improvement, we obtain the optimal learning
rate in Eq 6 and the optimal loss improvement in Eq 7. ∎

##  Appendix C Proof of Theorem 2

###### Proof.

Consider a random variable $x\sim\mathcal{N}(\mu,\sigma^{2})$, let
$y=\frac{x-\mu}{\sigma}$ and $z=-\frac{\mu}{\sigma}$, then

$\displaystyle\mathbb{E}[sign(x)]$
$\displaystyle=\int_{-\infty}^{\infty}sign(x)\frac{1}{\sigma\sqrt{2\pi}}e^{-\frac{(x-\mu)^{2}}{2\sigma^{2}}}dx$
(32)
$\displaystyle=\frac{1}{\sqrt{2\pi}}\int_{-\infty}^{\infty}sign(\mu+\sigma
y)e^{-\frac{y^{2}}{2}}dy$
$\displaystyle=\frac{1}{\sqrt{2\pi}}(\int_{z}^{\infty}e^{-\frac{y^{2}}{2}}dy-\int_{-\infty}^{z}e^{-\frac{y^{2}}{2}}dy)$
$\displaystyle=(1-\Phi(z))-\Phi(z)=erf(\frac{\mu}{\sqrt{2}\sigma})$

and the variance is

$\begin{aligned}
var(sign(x))=\mathbb{E}[sign(x)^{2}]-\mathbb{E}[sign(x)]^{2}=1-erf(\frac{\mu}{\sqrt{2}\sigma})^{2}\end{aligned},$
(33)

where $\Phi$ represents the cumulative distribution function of the standard
normal distribution, and $erf$ is the Gauss error function that is defined as
$erf(x)=\frac{2}{\sqrt{\pi}}\int_{0}^{x}e^{-t^{2}}dt$.

Given that the gradient $G_{x}$ of any data point $x$ in the data distribution
$\rho(x)$ relative to a certain parameter $\theta_{i}$ follows a Gaussian
distribution with mean $\mu_{i}$ and variance $\sigma_{i}^{2}$, then

$G_{est}(\theta_{i})=\frac{1}{B}\sum^{B}G_{x}(\theta_{i})\sim\mathcal{N}(\mu_{i},\frac{\sigma_{i}^{2}}{B}).$
(34)

Therefore, when $V=sign(G_{est})$, using Eq 32 and Eq 33 we can get the
expectation

$\begin{aligned} \mathbb{E}[V]=\begin{pmatrix}\vdots\\\
erf(\sqrt{\frac{B}{2}}\frac{\mu_{i}}{\sigma_{i}})\\\
\vdots\end{pmatrix}\end{aligned},$ (35)

and the covariance matrix

$\begin{aligned} cov(V)=\begin{pmatrix}\ddots&&0\\\
&1-erf(\sqrt{\frac{B}{2}}\frac{\mu_{i}}{\sigma_{i}})^{2}&\\\ 0&&\ddots\\\
\end{pmatrix}\end{aligned}.$ (36)

Given that the real gradient satisfies:

$G=\begin{pmatrix}\vdots\\\ \mu_{i}\\\ \vdots\end{pmatrix},$ (37)

and define $\mathcal{E}_{i}$ as a function with respect to the batch size $B$,
based on the Gauss error function:

$\displaystyle\mathcal{E}_{i}(B)=erf(\sqrt{\frac{B}{2}}\frac{\mu_{i}}{\sigma_{i}})=\frac{2}{\sqrt{\pi}}\int_{0}^{\sqrt{\frac{B}{2}}\frac{\mu_{i}}{\sigma_{i}}}e^{-t^{2}}dt,$
(38)

substituting the expectation and the variance of $V$ in Eq 6 and Eq 7 from
Lemma 1 using the above equations, we can get Eq 8 and Eq 9 respectively.

To simplify the subsequent computation, we approximate the function
$\mathcal{E}_{i}$ by other sigmoid-like analytical forms. Specifically, we
find that
$\mathcal{E}_{i}(B)\approx\frac{\frac{\mu_{i}}{\sigma_{i}}}{\sqrt{\frac{\pi}{2B}+(\frac{\mu_{i}}{\sigma_{i}})^{2}}}$,
which results in Eq 10.

∎

##  Appendix D Proof of Theorem 3

###### Proof.

When $B\ll\frac{\pi\sigma_{i}^{2}}{2\mu_{i}^{2}}$, Eq 10 reduces to:

$\mathcal{E}_{i}(B)\approx\sqrt{\frac{2B}{\pi}}\frac{\mu_{i}}{\sigma_{i}}.$
(39)

Substituting the function $\mathcal{E}_{i}$ in Eq 9 using the above equation,
and defining $\mathcal{B}_{noise}$ and $\epsilon_{max}$ as in Eq 14 and 16, we
can deduce the relationship between the optimal learning rate and the batch
size:

$\displaystyle\epsilon_{opt}(B)$
$\displaystyle=\frac{\sum_{i}\sqrt{\frac{2B}{\pi}}\frac{\mu_{i}}{\sigma_{i}}\mu_{i}}{\sum_{i}H_{i,i}+\frac{2B}{\pi}\sum_{i}\sum_{j}\begin{cases}\frac{\mu_{i}\mu_{j}}{\sigma_{i}\sigma_{j}}&i\neq
j\\\ 0&i=j\end{cases}H_{i,j}}$ (40)
$\displaystyle=\frac{\frac{\sum_{i}\sqrt{\frac{2B}{\pi}}\frac{\mu_{i}^{2}}{\sigma_{i}}}{\sum_{i}H_{i,i}}}{1+B\frac{2\sum_{i}\sum_{j}\begin{cases}\frac{\mu_{i}\mu_{j}}{\sigma_{i}\sigma_{j}}&i\neq
j\\\ 0&i=j\end{cases}H_{i,j}}{\pi\sum_{i}H_{i,i}}}$
$\displaystyle=\frac{\sqrt{B}}{1+\frac{B}{\mathcal{B}_{noise}}}\frac{\sqrt{\frac{2}{\pi}}\sum_{i}\frac{\mu_{i}^{2}}{\sigma_{i}}}{\sum_{i}H_{i,i}}$
$\displaystyle=\frac{\frac{\sqrt{\mathcal{B}_{noise}}}{2}}{\frac{1}{2}(\sqrt{\frac{\mathcal{B}_{noise}}{B}}+\sqrt{\frac{B}{\mathcal{B}_{noise}}})}\frac{\sqrt{\frac{2}{\pi}}\sum_{i}\frac{\mu_{i}^{2}}{\sigma_{i}}}{\sum_{i}H_{i,i}}$
$\displaystyle=\frac{\epsilon_{max}}{\frac{1}{2}(\sqrt{\frac{\mathcal{B}_{noise}}{B}}+\sqrt{\frac{B}{\mathcal{B}_{noise}}})}\leq\epsilon_{max}.$

It should be noted that since $\mathcal{B}_{noise}>0$ and $\epsilon_{max}>0$,
then both $\sum_{i}H_{i,i}$ and
$\sum_{i}\sum_{j}\begin{cases}\frac{\mu_{i}\mu_{j}}{\sigma_{i}\sigma_{j}}&i\neq
j\\\ 0&i=j\end{cases}H_{i,j}$ are greater than 0. Therefore, $\epsilon_{max}$
can be expressed as a form that does not contain $\mathcal{B}_{noise}$:

$\begin{aligned}
\epsilon_{max}&=\frac{\sqrt{\frac{\mathcal{B}_{noise}}{2\pi}}\sum_{i}\frac{\mu_{i}^{2}}{\sigma_{i}}}{\sum_{i}H_{i,i}}\\\
&=\frac{\sqrt{\sum_{i}H_{i,i}}}{2\sqrt{\sum_{i}\sum_{j}\begin{cases}\frac{\mu_{i}\mu_{j}}{\sigma_{i}\sigma_{j}}&i\neq
j\\\
0&i=j\end{cases}H_{i,j}}}\frac{\sum_{i}\frac{\mu_{i}^{2}}{\sigma_{i}}}{\sum_{i}H_{i,i}}\\\
&=\frac{\sum_{i}\frac{\mu_{i}^{2}}{\sigma_{i}}}{2\sqrt{\sum_{i}\sum_{j}\begin{cases}\frac{\mu_{i}\mu_{j}}{\sigma_{i}\sigma_{j}}&i\neq
j\\\ 0&i=j\end{cases}H_{i,j}}\cdot\sqrt{\sum_{i}H_{i,i}}}\\\
&\geq\frac{\sum_{i}\frac{\mu_{i}^{2}}{\sigma_{i}}}{\sum_{i}\sum_{j}\begin{cases}\frac{\mu_{i}\mu_{j}}{\sigma_{i}\sigma_{j}}&i\neq
j\\\ 1&i=j\end{cases}H_{i,j}}\\\ \end{aligned}.$ (41)

The last inequality is derived using the AM–GM inequality ($a^{2}+b^{2}\geq
2ab$).

∎

##  Appendix E Proof of Theorem 4

###### Proof.

When $B\gg\frac{\pi\sigma_{i}^{2}}{2\mu_{i}^{2}}$, Eq 10 converges to:

$\begin{aligned}
\mathcal{E}_{i}=sign(\frac{\mu_{i}}{\sigma_{i}})=sign(\mu_{i})\end{aligned}.$
(42)

Substituting the function $\mathcal{E}_{i}$ in Eq 9, we can obtain Eq 17. ∎

##  Appendix F Proof of Theorem 5

###### Proof.

When $B\ll\frac{\pi\sigma_{i}^{2}}{2\mu_{i}^{2}}$, we have the approximate
results in Eq 39 from Theorem 3. Substituting $\mathcal{E}_{i}$ and
$\mathcal{B}_{noise}$ using Eq 39 and 14, and defining $\Delta L_{max}$ as in
19, the optimal loss improvement in Eq 8 can be expressed as:

$\displaystyle\Delta L_{opt}(B)$
$\displaystyle=\frac{\cancel{\frac{1}{2}}\cdot\frac{\cancel{2}B}{\pi}\sum_{i}\sum_{j}\frac{\mu_{i}^{2}\mu_{j}^{2}}{\sigma_{i}\sigma_{j}}}{\sum_{i}H_{i,i}+\frac{2B}{\pi}\sum_{i}\sum_{j}\begin{cases}\frac{\mu_{i}\mu_{j}}{\sigma_{i}\sigma_{j}}&i\neq
j\\\ 0&i=j\end{cases}H_{i,j}}$ (43)
$\displaystyle=\frac{\sum_{i}\sum_{j}\frac{\mu_{i}^{2}\mu_{j}^{2}}{\sigma_{i}\sigma_{j}}}{\frac{\pi\sum_{i}H_{i,i}}{B}+2\sum_{i}\sum_{j}\begin{cases}\frac{\mu_{i}\mu_{j}}{\sigma_{i}\sigma_{j}}&i\neq
j\\\ 0&i=j\end{cases}H_{i,j}}$
$\displaystyle=\frac{1}{\frac{\mathcal{B}_{noise}}{B}+1}\frac{\sum_{i}\sum_{j}\frac{\mu_{i}^{2}\mu_{j}^{2}}{\sigma_{i}\sigma_{j}}}{2\sum_{i}\sum_{j}\begin{cases}\frac{\mu_{i}\mu_{j}}{\sigma_{i}\sigma_{j}}&i\neq
j\\\ 0&i=j\end{cases}H_{i,j}}$ $\displaystyle=\frac{\Delta
L_{max}}{\frac{\mathcal{B}_{noise}}{B}+1}\leq\Delta L_{max}$

Following the Appendix D in [25], this allows both the total number of steps
and data examples processed to still be written as

$\displaystyle S$ $\displaystyle=\int(1+\frac{\mathcal{B}_{noise}}{B})ds$ (44)
$\displaystyle E$ $\displaystyle=\int(\mathcal{B}_{noise}+B)ds$

Therefore the conclusion in [25]

$\displaystyle S_{min}$ $\displaystyle=\int ds$ (45) $\displaystyle E_{min}$
$\displaystyle=\int\mathcal{B}_{noise}ds$

and Eq 20, 21 still hold.

∎

##  Appendix G Variable Estimation for Data/Time Efficiency Relationship
Equation

When considering $S$, $E$, $S_{min}$ and $E_{min}>0$, Eq 20 can be simplified
to

$\displaystyle(\frac{S}{S_{min}}-1)(\frac{E}{E_{min}}-1)$ $\displaystyle=1$
(46) $\displaystyle SE-S_{min}E-SE_{min}+\cancel{S_{min}E_{min}}$
$\displaystyle=\cancel{S_{min}E_{min}}$ $\displaystyle S_{min}E+SE_{min}$
$\displaystyle=SE$ $\displaystyle\frac{S_{min}}{S}+\frac{E_{min}}{E}$
$\displaystyle=1$ $\displaystyle\frac{1}{S}$
$\displaystyle=-\frac{E_{min}}{S_{min}}\frac{1}{E}+\frac{1}{S_{min}}$

so a linear fit to the relationship between $\frac{1}{S}$ and $\frac{1}{E}$
can estimate $S_{min}$ and $E_{min}$.


# Constraining dark matter interactions mediated by a light scalar with white
dwarfs

Maura E. Ramirez-Quezada<EMAIL_ADDRESS>Department of
Physics, University of Tokyo, Bunkyo-ku, Tokyo 113–0033, Japan  Dual CP
Institute of High Energy Physics, C.P. 28045, Colima, México

###### Abstract

Observations of white dwarfs in dark matter-rich environments can provide
strong limits on the strength of dark matter interactions. Here we apply the
recently improved formalism of the dark matter capture rate in white dwarfs to
a general model in which dark matter interacts with the white dwarf ion
components via a light scalar mediator. We compute the dark matter capture
rate in the optically thin limit in a cold white dwarf from the globular
cluster Messier 4. We then estimate the threshold cross-section, which
significantly varies as a function of the light scalar mediator mass
$m_{\phi}$ when this has a mass below the dark matter mass
($m_{\phi}<m_{\chi}$) and becomes constant when $m_{\phi}>m_{\chi}$. We also
show that the bounds obtained from the dark matter capture in a white dwarf
from the globular cluster Messier 4 are complementary to direct detection
experiments and particularly strong in the sub-GeV regime.

##  I Introduction

The indirect evidence of the existence of dark matter (DM) in the universe is
a major milestone in physics. This elusive matter component may interact with
Standard Model (SM) particles in a weak but significant way. The leading
candidates are the weakly interacting massive particles (WIMPs) Bertone  _et
al._ (2005). Typically in direct detection (DD) experiments, the DM particles
are expected to interact with nucleons in the detector by exchanging a
marginal momentum. If the interaction mediator is a massive particle, it is
possible to assume a DM-nucleon point-like interaction. On the other hand,
when the interaction is mediated by a light scalar particle, this assumption
no longer holds. In this case, the mediator mass is comparable or smaller to
the momentum transfer, and the effective field theory (EFT) treatment breaks
down Fornengo  _et al._ (2011); Kaplinghat  _et al._ (2014a); Li  _et al._
(2015); Nobile  _et al._ (2015); Kahlhoefer  _et al._ (2017).

A promising way to search for DM interactions is to capture DM particles in
the core of white dwarfs in DM-rich environments Amaro-Seoane  _et al._
(2016); Panotopoulos and Lopes (2020); Bell  _et al._ (2021). In particular,
white dwarfs (WDs), due to their gravitational potential, are excellent
candidates to explore interactions of DM particles with masses in the Sub-GeV
mass region Bell  _et al._ (2021). The DM particles will fall into the WD
gravitational potential and interact with the stellar matter while traversing
the star. If the DM loses enough energy after scattering off the stellar
material, it will be captured and accumulated in the WD core. The accreted DM
within the WD core will thermalize and eventually annihilate into SM
particles. The capture, thermalization, and annihilation process will inject
energy into the WD and increase its temperature. This extra heating will
eventually generate an observable that we can use to set bounds on the
interaction strength of the DM particles Dasgupta  _et al._ (2019, 2020).

In principle, the capture process of DM in WD cores occurs via the scattering
of DM with either electrons or ions. In this work, we will apply the recently
improved capture rate formalism given in Ref. Bell  _et al._ (2021) to a
minimal model in which Dirac fermion DM interacts with the WD ion-components
via a light scalar mediator $\phi$ Balaji  _et al._ (2022); Kaplinghat  _et
al._ (2014b); Curtin  _et al._ (2014); Cotta  _et al._ (2014); Fornengo  _et
al._ (2011). Such particles are very well motivated since their existence have
important implications. For instance, light mediators can lead to long-range
attractive forces between DM particles and eventually enhance the DM
annihilation cross-sections at low temperatures via the mechanism of
Sommerfeld enhancement Hisano  _et al._ (2004, 2005); Sommerfeld (1931);
Arkani-Hamed  _et al._ (2009); Cassel (2010); Liu  _et al._ (2013).

By computing the capture rate of DM in WDs from the globular cluster Messier 4
(M4) we place limits on the DM-proton interactions in the Sub-GeV mass region.
We show that the bounds obtained by the DM capture are complementary to those
of DD experiments. We find that observations of WDs in M4 can set limits in
the sub-GeV region of the DM mass for a wide range of the scalar mediator
mass. However, it is important to emphasise that these bounds strongly depend
on whether there is DM in the globular cluster M4.

The paper is organized as follows. In Sec. II we introduce the DM-ion
scattering cross section and specify the role of the scalar light mediator. In
Sec. III we briefly discuss the capture rate in the optically thin limit and
argue why this is a good approximation in our parameters of interest. We
present and discuss our results in Sec. IV and conclude in Sec.V.

##  II DM-ion scattering cross section

DM particles are expected to have tiny interaction strengths with ordinary
matter in many models. For very heavy mediators, the DM particle interactions
with nucleons can be described by effective operators, which are parametrised
by the mass scale $\Lambda$ Fan  _et al._ (2010); Fitzpatrick  _et al._
(2013),

$D_{i}^{N}=\frac{c_{i}}{\Lambda^{2}}\bar{\chi}\Gamma_{i}\chi\bar{N}\Gamma^{\prime}_{i}N,$
(1)

where $c_{i}$ are the Wilson coefficients, $\Gamma_{i}$ are Lorentz-invariant
combinations of the Dirac matrices and $N=p,n$ for protons and neutrons
respectively. Here, we will consider Yukawa interactions with the DM particle,
which sets $\Gamma_{i},\Gamma_{i}^{\prime}=1$. Therefore, the interaction
corresponds to the operator $D_{1}^{N}$

$D_{1}^{N}=\frac{c_{1}}{\Lambda^{2}}\bar{\chi}\chi\bar{N}N,$ (2)

with

$c_{1}=\frac{\sqrt{2}m_{N}}{v}\left[\sum_{q=u,s,d}f_{T_{q}}^{N}+\frac{2}{9}f^{N}_{TG}\right],$
(3)

where $v=246\,\rm GeV$ is the electroweak (EW) vacuum expectation value of the
Higgs field, $m_{N}$ is the nucleon mass and $f_{T_{q}}$ and $f_{T_{G}}$ are
the hadronic elements Ellis  _et al._ (2018); Belanger  _et al._ (2014). The
expression in Eq. (3) only holds for the particular case when the couplings of
the mediator with quarks are proportional to their masses (Yukawa couplings).
This situation is realised if the mediator couples with the Standard Model
sector only via mixing with the Higgs boson; A more generic expression for
different types of interactions is given in, e.g., Eq.(C.6) in Ref. Fan  _et
al._ (2010).

Due to the very low energies involved in DM-nucleon scattering, such
interaction shall be described at the nucleus level by non-relativistic
operators $D_{i}^{NR}$. This is obtained by contracting the fermionic DM and
nucleon bilinears. For the bilinears in Eq. (2), the non-relativistic
expansion corresponds to Cirelli  _et al._ (2013),

$\bar{\chi}\chi\bar{N}N=4m_{\chi}m_{N}D_{1}^{NR},$ (4)

where $m_{\chi}$ is the DM mass and $D_{1}^{NR}=1$.

Here we will consider a light Dirac fermion DM candidate coupling to the SM
via a light scalar $\phi$ Kaplinghat  _et al._ (2014b, a). Since this mediator
is light the effective operator approach is modified by the simple replacement
$\Lambda^{2}\to(q^{2}+m_{\phi}^{2})/(g_{\chi}g_{SM})$, where $q$ is the
transferred momentum in the $t$-channel, $m_{\phi}$ is the light scalar
mediator mass and $g_{\chi,SM}$ are the couplings of the scalar to the DM and
the SM particles respectively. Therefore, for the Yukawa interaction, the DM-
nuclei amplitude is given as

$\mathcal{M}_{N}=4g_{\chi}g_{SM}c_{1}\frac{m_{\chi}m_{N}}{q^{2}+m_{\phi}^{2}}.$
(5)

From the EFT approach, we then notice that spin-averaged squared amplitude for
scattering off a target nucleus $T$ with mass $m_{T}$ is given by

$|\mathcal{M}_{T}|^{2}=\frac{m_{T}^{2}}{m_{N}^{2}}|\mathcal{M}_{N}|^{2}\langle
F(E_{R})^{2}\rangle,$ (6)

and $\langle F(E_{R})^{2}\rangle$ is the average of the Helm form factor Helm
(1956); Dūda  _et al._ (2007) and $E_{R}=q^{2}/2m_{T}$ is the recoil energy.
Specifically, we adopt the following form Lewin and Smith (1996),

$\langle F(E_{R})^{2}\rangle=\int_{0}^{E_{R}^{\rm
max}}\left(\frac{3j_{1}(qR_{1}))}{qR_{1}}\right)^{2}\frac{e^{-q^{2}s^{2}}}{E_{R}^{\rm
max}},$ (7)

here $j_{1}$ is the spherical Bessel function of the first kind, $R_{1}$ is
the effective nuclear radius, and $s$ is the nuclear skin thickness, see Ref.
Lewin and Smith (1996). $E_{R}^{\rm max}$ is the maximum recoil energy found
when the momentum transfer $q$ is maximised, $E_{R}^{\rm max}=q^{2\,\rm
max}/2m_{T}$. The differential scattering cross-section in the non-
relativistic approximation is given by

$\frac{d\sigma}{d\cos\theta}=\frac{1}{32\pi}\frac{|\mathcal{M}_{T}|^{2}}{(m_{\chi}+m_{T})^{2}}.$
(8)

In the centre-of-mass (cm) frame, we express the momentum transferred in terms
of the cm angle $\theta$ Busoni  _et al._ (2017)

$q=4\sin^{2}(\theta/2)v_{r}^{2}\frac{m_{\chi}^{2}m_{T}^{2}}{(m_{T}+m_{\chi})^{2}}.$
(9)

where $v_{r}$ is the relative velocity between the DM and the target.

Putting all of this together, the differential scattering cross-section for a
Yukawa DM-ion interaction is given as

$\frac{d\sigma}{d\cos\theta}=\frac{c_{1}^{2}}{2\pi}\frac{g_{\chi}^{2}g_{SM}^{2}\beta^{2}}{(4v_{r}^{2}\beta^{2}\sin^{2}(\theta/2)+m_{\phi}^{2})^{2}}\langle
F(E_{R})^{2}\rangle,$ (10)

where $\beta$ is the reduced mass of the system. Notice that if in Eq. (10) we
define $\alpha^{2}=g_{\chi}^{2}g_{SM}^{2}c_{1}^{2}/(8\pi)$ we recover the
expression given in Ref. Dasgupta  _et al._ (2020).

##  III Dark matter capture in white dwarfs

Figure 1: Threshold cross-section for $m_{\chi}=10\,\rm MeV$ (blue) and
$m_{\chi}=100\,\rm MeV$ (brown) as a function of the light mediator mass
$m_{\phi}$. The shaded area in both cases indicates the interactions in which
the capture process operates maximally.

The capture of DM in WDs is possible via the scattering process with electrons
and ions Bell  _et al._ (2021). Since in the general model studied here, DM
interacts with the nucleon via a scalar mediator Kaplinghat  _et al._ (2014b,
a), we will account only for the latter process. Therefore, the capture rate
of DM in WDs by ions in the optically thin limit is given by Bell  _et al._
(2021)

$C=\frac{\rho_{\chi}}{m_{\chi}}\int_{0}^{R\star}dr4\pi
r^{2}\int_{0}^{\infty}du_{\chi}\frac{w}{u_{\chi}}f_{\rm
MB}(u_{\chi})\Omega^{-}(w),$ (11)

where $R_{\star}$ is the WD radius, $\rho_{\chi}$ is the DM density,
$u_{\chi}$ is the DM velocity at infinity, and $w$ is the DM velocity after it
falls close to the WD with $w^{2}=u^{2}+v_{e}(r)^{2}$. $f_{\rm MB}$ is the
Maxwell-Boltzmann DM distribution function, which in the zero temperature
limit $T_{\star}\to 0$ is given by Busoni  _et al._ (2017); Bell  _et al._
(2021)

$f_{\rm
MB}(u_{\chi})=\frac{u_{\chi}}{v_{d}v_{\star}}\sqrt{\frac{3}{2\pi}}\left[e^{-\frac{3}{2v_{d}^{2}}\left(u_{\chi}-v_{\star}\right)^{2}}-e^{-\frac{3}{2v_{d}^{2}}\left(u_{\chi}+v_{\star}\right)^{2}}\right].$
(12)

where $v_{\star}$ is the WD velocity and $v_{d}$ is the velocity dispersion of
the DM.

Finally, $\Omega^{-}(w)$ is the interaction rate probability given as a
function of the differential scattering cross-section and the ion number
density. For ion targets in a WD in the zero temperature limit, it is given by
Busoni  _et al._ (2017)

$\displaystyle\Omega^{-}(w)$ $\displaystyle=$
$\displaystyle\frac{4\mu_{+}^{2}}{\mu
w}n_{T}(r)\int_{w\frac{|\mu_{-}|}{\mu_{+}}}^{v_{e}}dvv\dfrac{d\sigma}{d\cos\theta}.$
(13)

where $\mu=m_{\chi}/m_{T}$, $\mu_{\pm}=(\mu\pm 1)/2$. The capture process
happens when the DM velocity after scattering $v$ is below the escape velocity
$v_{e}$ of the WD.

The case where all DM traversing the star is captured, i.e. the geometric
limit, occurs when $\Omega^{-}(w)\to 1$. In this case, we can write the
capture rate as Bell  _et al._ (2021),

$\displaystyle C_{geom}=\frac{\pi
R_{\star}^{2}\rho_{\chi}}{3v_{\star}m_{\chi}}\Bigg{[}(3v_{e}^{2}(R_{\star})+$
$\displaystyle
3v_{\star}^{2}+v_{d}^{2})\mathrm{erf}\left(\sqrt{\frac{3}{2}}\frac{v_{\star}}{v_{d}}\right)$
$\displaystyle\left.+\sqrt{\frac{6}{\pi}}v_{\star}v_{d}e^{-\frac{3v_{\star}^{2}}{2v_{d}^{2}}}\right].$
(14)

This limit is a very good approximation for DM-cross sections above a
threshold cross-section $\sigma_{th}$. The threshold cross-section is where
the geometric and optically thin limits intersect.111For DM scattering cross-
sections of $\sigma\sim\sigma_{th}$ we shall introduce the star opacity as
discussed in Ref.Bell  _et al._ (2021). However, the DM-ion cross-sections we
will study are below this threshold, and it is safe to consider the optically
thin limit given in Eq. (11).

In Fig. 1, we show the threshold cross section for two different DM masses
$m_{\chi}=10\,\rm MeV$ (blue) and $m_{\chi}=100\,\rm MeV$ (red). The threshold
cross-section varies accordingly to the light mediator mass and the
$g_{\chi}g_{g_{SM}}$ parameter, which is not shown in the plot. When the light
mediator mass is large ($m_{\phi}\to\infty$), the threshold cross-section
becomes independent of $m_{\phi}$, and we recover the EFT operator approach.
This situation is reached once the light mediator mass reaches the DM mass,
and hence it happens first for the case of $m_{\chi}=10\,\rm MeV$ as shown in
the blue line in Fig. 1. The shaded area indicates that all cross-sections in
this region belong to the geometric limit, and DM is maximally captured in the
WD.

We must compute the capture rate for DM cross-sections below the threshold
cross-sections to obtain reliable bounds on DM-proton interaction.

##  IV Results and discussion

Figure 2: DM-proton interactions induced by scalar light mediator $\phi$ with
a mass of $m_{\phi}=1\,\rm MeV$ (left panel) and $m_{\phi}=10\,\rm MeV$ (right
panel). The bounds shown in green have been computed assuming a DM density of
$\rho_{\chi}=798\,\rm GeV\,cm^{-3}$ in the globular cluster M4 McCullough and
Fairbairn (2010). We show in red the latest constraints set by Pandax-II for
the two different light mediator masses for comparison Yang  _et al._ (2021).
The dashed-dotted line represents the EFT limit case of the DM-proton cross-
section.

To compute the DM capture rate, we have used the oldest WD observed in the
globular cluster M4 Bedin  _et al._ (2009) corresponding to a WD of
$M_{\star}=1.38\,\rm M_{\odot}$ and $R_{\star}=8.7\times 10^{2}\,\rm km$. If
DM is present in M4, the expected DM density is very large when compared to
the DM in the Solar vicinity, which enhances the capture rate. Using a
Navarro-Frenk-White profile to model the DM halo, the DM density in M4 is
expected to be as large as $\rho_{\chi}=798\,\rm GeVcm^{-3}$ McCullough and
Fairbairn (2010). The exact values of the star and DM dispersion velocities
for WDs in M4 are unknown. We assume the star velocity to be
$v_{\star}=20\,\rm km\,s^{-1}$ while the dispersion velocity is found to be
less than $v_{d}=8\,\rm km\,s^{-1}$ McCullough and Fairbairn (2010).

We consider a WD made entirely of Carbon-12. Using the Salpeter equation of
state Salpeter (1961); Hamada and Salpeter (1961) coupled with the
Tolman–Oppenheimer–Volkoff (TOV) equation, we model the inner structure of the
WD. We obtain the radial profiles for the target number density $n_{T}(r)$ and
the WD escape velocity $v_{e}(r)$. This allows us to compute the capture rate
in the optically thin limit given in Eq. (11). Nonetheless, we should not
exceed the threshold cross-section $\sigma_{th}$ to compute the DM-ion
interaction limits using the capture rate in the thin limit. Furthermore, for
the DM mass range studied here, the capture process is successful after a
single scattering Bell  _et al._ (2021).222This is true for DM masses below a
threshold DM mass which in the case of a WD of $M_{\star}=1.38\,\rm M_{\odot}$
the threshold mass is $m_{th}\sim 10^{4}$ GeV Bell  _et al._ (2021).

The DM capture and annihilation rates are already in equilibrium in the oldest
WDs Bertone and Fairbairn (2008). This means that the DM annihilation rate is
equal to the capture rate $C$ given in Eq. (11). Hence, the heating luminosity
observed at a distance is estimated to be,

$L_{\chi}=m_{\chi}C.$ (15)

We compare the WD observed luminosity $L_{\gamma}\sim 10^{31}\,\rm
GeV\,s^{-1}$ to the fraction luminosity due to the DM in the WD core. By
requiring that the luminosity $L_{\gamma}$ be larger than $L_{\chi}$, we
constrain the DM-proton cross-section.

In Fig. 2, we show the upper bounds for the DM-proton cross-section, induced
by a light mediator, given by the M4 carbon WD (green line). To compute these
bounds, we have assumed that all the observed luminosity comes from the DM
effects, i.e. $L_{\gamma}=L_{\chi}$.

In the left panel, we show the DM-proton cross-section bounds for a light
mediator of mass $m_{\phi}=1\,\rm MeV$ while in the right panel, the mass is
$m_{\phi}=10\,\rm MeV$. For comparison, we also show the bounds from the
latest results given by PandaX-II for the two light mediator masses (red) Yang
_et al._ (2021). The areas shaded in green, and red corresponds to the
excluded parameter space by WD in M4 and PandaX-II, respectively.333Here, we
have not included the updated and strongest limits given by LZ Aalbers  _et
al._ (2022) since it excludes the same parameter space regime, above
$m_{\chi}\sim\mathcal{O}(1)\,\rm GeV$, as PandaX-II Ren  _et al._ (2018); Yang
_et al._ (2021). We observe that the limit given by PandaX-II is stronger than
that provided by the M4 WD for the two possible mediator masses in the DM mass
region above a few GeV. The dashed grey line corresponds to the threshold
cross-section.

Additionally, we have checked that the couplings are always perturbative in
the region of interest. The product $g_{\chi}g_{SM}$ that fulfils
$L_{\gamma}=L_{\chi}$ remains small ($g_{\chi}g_{SM}\lesssim\mathcal{O}(1))$
as long as $m_{\phi}\lesssim m_{\chi}$. In dashed-dotted lines, we have also
added the WD bounds that would be given in the EFT limit of the DM-proton
cross-section. We observe that in both cases, the limits given by WDs using
this approach are overestimated and become a good approximation when
$m_{\chi}\lesssim m_{\phi}$. Particularly for the left panel with a light
mediator mass of $m_{\phi}=1$ MeV, the bounds given by our particular WD
completely surpass the DD bounds.

On the other hand, since the DM evaporation mass in WDs is of the order of
keV, it is possible to constrain the interactions for Sub-GeV DM Bell  _et
al._ (2021). This means that the bounds given by the WD are valid as long as
the DM mass is above this evaporation mass. Otherwise, the evaporation process
will reduce the number of accumulated DM particles in the WD core. Notice that
choices of scalar masses below 1 MeV require lower DM masses, and limits are
no longer reliable.

Here, we have shown that if DM is present in the core of M4, it is clear that
the bounds to DM interactions via a scalar mediator provided by WDs are
complementary to direct detection experiments for a wide range of scalar
mediator masses. If the DM density is a few $\rm GeV/cm^{3}$ as estimated in
Ref. Hooper  _et al._ (2010), instead of $\simeq 10^{2}$-$10^{3}\,\rm
GeV/cm^{3}$ the WD limits become several orders of magnitude weaker. Finally,
it is worth mentioning that such states are actively searched in collider
experiments, and constraints have been given for scalar and pseudoscalar
mediators above 10 GeV Aad  _et al._ (2021); ATL (2022). However, searching at
colliders might be complicated for very light mediators since these are long-
lived particles that might decay outside the detectors.

##  V Conclusion

In this work, we studied a general model for DM interactions with nuclei
induced by light mediators. We have discussed the DM capture in WDs and have
computed the threshold cross-section as a function of the light mediator mass
for two choices of DM mass. The threshold cross-section $\sigma_{th}$ varies
according to the scalar light mediator mass and coupling when
$m_{\phi}<m_{\chi}$. However, when the light mediator mass reaches the DM
mass, this dependence disappears, and the threshold cross section becomes
constant for all masses $m_{\phi}>m_{\chi}$.

Using WDs from the globular cluster M4, we set limits on these interactions
accounting for different light mediator masses $m_{\phi}=1\rm MeV$ and
$m_{\phi}=10\rm MeV$. We found that if DM is present in M4, the bounds set by
WDs on DM interactions with the nucleus via the exchange of a scalar light
mediator are complementary to those given by direct detection experiments.
Specifically, we can constrain interactions for DM masses in the sub-GeV
regime for a wide range of $m_{\phi}$.

##  VI Acknowledgements

The Author would like to thank Shyam Balaji, Koichi Hamaguchi, Natsumi Nagata
and Yongchao Zhang for helpful discussions and feedback on the draft. MRQ is
supported by the JSPS KAKENHI Grant Number 20H01897.

## References

  * Bertone  _et al._ (2005) G. Bertone, D. Hooper, and J. Silk, Phys. Rept. 405, 279 (2005), arXiv:hep-ph/0404175 .
  * Fornengo  _et al._ (2011) N. Fornengo, P. Panci, and M. Regis, Phys. Rev. D 84, 115002 (2011), arXiv:1108.4661 [hep-ph] .
  * Kaplinghat  _et al._ (2014a) M. Kaplinghat, S. Tulin, and H.-B. Yu, Phys. Rev. D 89, 035009 (2014a), arXiv:1310.7945 [hep-ph] .
  * Li  _et al._ (2015) T. Li, S. Miao, and Y.-F. Zhou, Journal of Cosmology and Astroparticle Physics 2015, 032 (2015).
  * Nobile  _et al._ (2015) E. D. Nobile, M. Kaplinghat, and H.-B. Yu, Journal of Cosmology and Astroparticle Physics 2015, 055 (2015).
  * Kahlhoefer  _et al._ (2017) F. Kahlhoefer, S. Kulkarni, and S. Wild, JCAP 11, 016 (2017), arXiv:1707.08571 [hep-ph] .
  * Amaro-Seoane  _et al._ (2016) P. Amaro-Seoane, J. Casanellas, R. Schödel, E. Davidson, and J. Cuadra, Mon. Not. Roy. Astron. Soc. 459, 695 (2016), arXiv:1512.00456 [astro-ph.CO] .
  * Panotopoulos and Lopes (2020) G. Panotopoulos and I. Lopes, Int. J. Mod. Phys. D 29, 2050058 (2020), arXiv:2005.11563 [hep-ph] .
  * Bell  _et al._ (2021) N. F. Bell, G. Busoni, M. E. Ramirez-Quezada, S. Robles, and M. Virgato, JCAP 10, 083 (2021), arXiv:2104.14367 [hep-ph] .
  * Dasgupta  _et al._ (2019) B. Dasgupta, A. Gupta, and A. Ray, JCAP 08, 018 (2019), arXiv:1906.04204 [hep-ph] .
  * Dasgupta  _et al._ (2020) B. Dasgupta, A. Gupta, and A. Ray, JCAP 10, 023 (2020), arXiv:2006.10773 [hep-ph] .
  * Balaji  _et al._ (2022) S. Balaji, P. S. B. Dev, J. Silk, and Y. Zhang, JCAP 12, 024 (2022), arXiv:2205.01669 [hep-ph] .
  * Kaplinghat  _et al._ (2014b) M. Kaplinghat, R. E. Keeley, T. Linden, and H.-B. Yu, Phys. Rev. Lett. 113, 021302 (2014b), arXiv:1311.6524 [astro-ph.CO] .
  * Curtin  _et al._ (2014) D. Curtin, Z. Surujon, and Y. Tsai, Phys. Lett. B 738, 477 (2014), arXiv:1312.2618 [hep-ph] .
  * Cotta  _et al._ (2014) R. C. Cotta, A. Rajaraman, T. M. P. Tait, and A. M. Wijangco, Phys. Rev. D 90, 013020 (2014), arXiv:1305.6609 [hep-ph] .
  * Hisano  _et al._ (2004) J. Hisano, S. Matsumoto, and M. M. Nojiri, Phys. Rev. Lett. 92, 031303 (2004), arXiv:hep-ph/0307216 .
  * Hisano  _et al._ (2005) J. Hisano, S. Matsumoto, M. M. Nojiri, and O. Saito, Phys. Rev. D 71, 063528 (2005), arXiv:hep-ph/0412403 .
  * Sommerfeld (1931) A. Sommerfeld, Annalen der Physik 403, 257 (1931).
  * Arkani-Hamed  _et al._ (2009) N. Arkani-Hamed, D. P. Finkbeiner, T. R. Slatyer, and N. Weiner, Phys. Rev. D 79, 015014 (2009), arXiv:0810.0713 [hep-ph] .
  * Cassel (2010) S. Cassel, J. Phys. G 37, 105009 (2010), arXiv:0903.5307 [hep-ph] .
  * Liu  _et al._ (2013) Z.-P. Liu, Y.-L. Wu, and Y.-F. Zhou, Phys. Rev. D 88, 096008 (2013), arXiv:1305.5438 [hep-ph] .
  * Fan  _et al._ (2010) J. Fan, M. Reece, and L.-T. Wang, JCAP 11, 042 (2010), arXiv:1008.1591 [hep-ph] .
  * Fitzpatrick  _et al._ (2013) A. L. Fitzpatrick, W. Haxton, E. Katz, N. Lubbers, and Y. Xu, JCAP 02, 004 (2013), arXiv:1203.3542 [hep-ph] .
  * Ellis  _et al._ (2018) J. Ellis, N. Nagata, and K. A. Olive, Eur. Phys. J. C 78, 569 (2018), arXiv:1805.09795 [hep-ph] .
  * Belanger  _et al._ (2014) G. Belanger, F. Boudjema, A. Pukhov, and A. Semenov, Comput. Phys. Commun. 185, 960 (2014), arXiv:1305.0237 [hep-ph] .
  * Cirelli  _et al._ (2013) M. Cirelli, E. Del Nobile, and P. Panci, JCAP 10, 019 (2013), arXiv:1307.5955 [hep-ph] .
  * Helm (1956) R. H. Helm, Phys. Rev. 104, 1466 (1956).
  * Dūda  _et al._ (2007) G. Dūda, A. Kemper, and P. Gondolo, Journal of Cosmology and Astroparticle Physics 2007, 012 (2007).
  * Lewin and Smith (1996) J. Lewin and P. Smith, Astroparticle Physics 6, 87 (1996).
  * Busoni  _et al._ (2017) G. Busoni, A. De Simone, P. Scott, and A. C. Vincent, JCAP 10, 037 (2017), arXiv:1703.07784 [hep-ph] .
  * McCullough and Fairbairn (2010) M. McCullough and M. Fairbairn, Phys. Rev. D 81, 083520 (2010), arXiv:1001.2737 [hep-ph] .
  * Yang  _et al._ (2021) J. Yang _et al._ (PandaX-II), Sci. China Phys. Mech. Astron. 64, 111062 (2021), arXiv:2104.14724 [hep-ex] .
  * Bedin  _et al._ (2009) L. R. Bedin, M. Salaris, G. Piotto, J. Anderson, I. R. King, and S. Cassisi, Astrophys. J. 697, 965 (2009), arXiv:0903.2839 [astro-ph.GA] .
  * Salpeter (1961) E. E. Salpeter, ApJ 134, 669 (1961).
  * Hamada and Salpeter (1961) T. Hamada and E. E. Salpeter, ApJ 134, 683 (1961).
  * Bertone and Fairbairn (2008) G. Bertone and M. Fairbairn, Phys. Rev. D 77, 043515 (2008), arXiv:0709.1485 [astro-ph] .
  * Aalbers  _et al._ (2022) J. Aalbers _et al._ (LZ), (2022), arXiv:2207.03764 [hep-ex] .
  * Ren  _et al._ (2018) X. Ren _et al._ (PandaX-II), Phys. Rev. Lett. 121, 021304 (2018), arXiv:1802.06912 [hep-ph] .
  * Hooper  _et al._ (2010) D. Hooper, D. Spolyar, A. Vallinotto, and N. Y. Gnedin, Phys. Rev. D 81, 103531 (2010), arXiv:1002.0005 [hep-ph] .
  * Aad  _et al._ (2021) G. Aad _et al._ (ATLAS), JHEP 04, 165 (2021), arXiv:2102.01444 [hep-ex] .
  * ATL (2022) (2022), arXiv:2211.05426 [hep-ex] .