# Is CLIP the main roadblock for fine-grained open-world perception?††thanks:
This work was partially supported by the following projects: SUN – Social and
hUman ceNtered XR (Horizon Europe R&I GA:101092612), FAIR – Future Artificial
Intelligence Research - Spoke 1 (EU NextGenerationEU PNRR M4C2 PE00000013),
ITSERR – ITalian Strengthening of the Esfri Ri Resilience (EU NextGenerationEU
CUP:B53C22001770006). MUCES – a MUltimedia platform for Content Enrichment and
Search in audiovisual archives (EU NextGenerationEU - PRIN 2022 PNRR
P2022BW7CW - CUP: B53D23026090001)

Lorenzo Bianchi12 , Fabio Carrara1 , Nicola Messina1 , Fabrizio Falchi1
0009-0005-3519-2991 0000-0001-5014-5089 0000-0003-3011-2487
0000-0001-6258-5313 1CNR-ISTI, Pisa, Italy 2 University of Pisa, Italy
Email<EMAIL_ADDRESS>
###### Abstract

Modern applications increasingly demand flexible computer vision models that
adapt to novel concepts not encountered during training. This necessity is
pivotal in emerging domains like extended reality, robotics, and autonomous
driving, which require the ability to respond to open-world stimuli. A key
ingredient is the ability to identify objects based on free-form textual
queries defined at inference time – a task known as _open-vocabulary object
detection_. Multimodal backbones like CLIP are the main enabling technology
for current open-world perception solutions. Despite performing well on
generic queries, recent studies highlighted limitations on the _fine-grained_
recognition capabilities in open-vocabulary settings – i.e., for
distinguishing subtle object features like color, shape, and material. In this
paper, we perform a detailed examination of these open-vocabulary object
recognition limitations to find the root cause. We evaluate the performance of
CLIP, the most commonly used vision-language backbone, against a fine-grained
object-matching benchmark, revealing interesting analogies between the
limitations of open-vocabulary object detectors and their backbones.
Experiments suggest that the lack of fine-grained understanding is caused by
the poor separability of object characteristics in the CLIP latent space.
Therefore, we try to understand whether fine-grained knowledge is present in
CLIP embeddings but not exploited at inference time due, for example, to the
unsuitability of the cosine similarity matching function, which may discard
important object characteristics. Our preliminary experiments show that simple
CLIP latent-space re-projections help separate fine-grained concepts, paving
the way towards the development of backbones inherently able to process fine-
grained details. The code for reproducing these experiments is available at
https://github.com/lorebianchi98/FG-CLIP.

###### Index Terms:

fine-grained understanding, open-vocabulary object detection, image-text
matching, evaluation study

##  I Introduction

Nowadays, pivotal technologies such as extended reality, autonomous driving,
and robotics necessitate more than adherence to closed-set assumptions; they
demand the ability to adapt to novel concepts not encountered during the
training phase. Open-vocabulary object detection (OVD) stands out as a
critical task for achieving this adaptability, as underlined by several open
challenges on egocentric data [9, 5]. It involves recognizing objects not
included in the training dataset, thus overcoming the limitations inherent in
traditional detectors confined to a predefined set of objects.

This flexibility is typically achieved by common-space multi-modal models.
These models embed images and texts in a shared latent space, thanks to the
contrastive pre-training performed on large datasets of image-text pairs
scraped from the web. This simplifies similarity calculations, which can be
achieved through an efficient dot product between the representations.
CLIP[26] stands out as the most widely utilized model in this category. Its
capabilities become crucial in open-vocabulary object detection, where models
typically perform the task by i) detecting regions of the image that are
likely to contain objects, ii) computing the similarity between the embedding
of the detected image region and that of a set of free-form texts defined at
test time, called vocabulary.

While open-vocabulary detectors excel in generalizing to new concepts not
encountered during training, recent studies indicate limitations in capturing
fine-grained properties of objects[2]. For instance, they may encounter
difficulties in distinguishing between a light brown dog and a dark brown one
(Figure 1). One potential explanation for these shortcomings is that CLIP
representations may exhibit bias towards category-level concepts while
overlooking attribute-level nuances [4]. To the best of our knowledge, it is
not well studied in the literature whether fine-grained properties are absent
in the latent space or if these characteristics exist, but trivial matching
methods (e.g., dot product, cosine similarity) are insufficient to extract
this information.

In this work, we assess the performance of CLIP, the most used backbone in
open-vocabulary object detectors, on a fine-grained benchmark to scrutinize
its capacity to accurately discern intricate properties of objects. The
findings reveal that the performance of CLIP in fine-grained understanding
mirrors that of an open-vocabulary object detector based on CLIP, indicating
shared challenges. This suggests that the fine-grained issues observed in the
open-vocabulary object detector may be attributed to the image-text alignment
carried out by CLIP rather than to detector localization failures.

Subsequently, by incorporating additional layers on top of frozen visual and
textual CLIP encoders and training them on a fine-grained dataset, we
illustrate the model’s proficiency in accurately assigning the relevant
attribute to the object. This shows that the original CLIP embeddings contain
fine-grained information that is ignored during the matching phase.

In summary, our paper contributes to the field in the following ways:

  * •

We comprehensively evaluate and analyze CLIP’s performance on a fine-grained
open-vocabulary object detection benchmark. This investigation sheds light on
the possibility that challenges faced by open-vocabulary object detectors may
be attributed to issues within the CLIP latent space.

  * •

We showcase the existence of fine-grained information within CLIP’s latent
space through the implementation of lightweight architectures trained on
frozen CLIP embeddings. This demonstration is substantiated by the model’s
ability to perform fine-grained matching successfully.

(a) Open-vocabulary object detection (OVD)

(b) Fine-Grained Open-Vocabulary object Detection (FG-OVD)

Figure 1: OVD (a) and FG-OVD (b): in the latter, fine-grained details about
the categories to detect are given as free-form text in the input vocabulary.

##  II Related Work

###  II-A Image-Text matching

In recent years, the focus of researchers on image-text matching increased.
The foundation of the shared space approach for cross-modal matching begins
with the exploration of hinge-based triplet ranking loss with hard-negative
mining. This was first attempted with GRU as text extractor [8], and later
with Transformer Encoder [15, 20, 18, 19, 25, 29, 32].

With the growing strength of Transformers also in vision tasks [6], many works
exploited an early-fusion approach, leveraging Transformers encoder to jointly
process images and texts from the very beginning of the proposed
architectures[31, 14, 16, 17, 30, 35]. These methods treat image-text matching
as a binary classification problem, where, given as input an image-text pair,
they train the Transformer architecture to predict the probability that the
text correctly describes the image. While achieving good performance, these
models cannot be used in many real-case scenarios since they are
computationally expensive at inference time, as they require processing every
image-text pair to obtain the score on the whole test set. Consequently, many
methods preferred to exploit a late-fusion approach, keeping the visual and
textual pipeline separated[20, 18, 19, 28, 32, 26, 11]. This allows separate
pipelines for producing the images and text representations, and the
similarity score can be computed in a second stage with a simple dot product.

Among these models, CLIP [26] stands out as one of the most widely used for
performing image-text matching. The majority of open-vocabulary object
detectors rely on the knowledge acquired by CLIP to embed object regions and
vocabulary entries in the same feature space [1, 10, 7, 22, 21, 33, 36, 37].
This approach circumvents the need to conduct inference for each pair of
detected image regions and vocabulary entries.

###  II-B Fine-grained understanding

Although CLIP shows strong performance in tasks such as classification and
coarse-grained retrieval, it has shortcomings in associating nuanced
properties between sentences and images [23, 24, 27, 36]. Yuksekgonul et al.
[34] suggest that contrastive pretraining does not optimize the model’s
understanding of the relationships between objects and their attributes.
Furthermore, standard retrieval benchmarks are considered inadequate for
assessing the compositional understanding of such models.

Krojer et al. [13] highlight that vision-language models tend to overlook
fine-grained visual information. Similarly, Chen et al. [4] show that
representations learned from contrastive pretraining in common-space
multimodal models are biased toward category-level concepts rather than
attributes, making attribute recognition difficult.

These limitations are even more pronounced in tasks where models usually rely
on CLIP latent space to work in an open-world environment, such as open-
vocabulary object detection. Some recent advanced benchmarks reveal the
weaknesses of these models [2, 3]. These benchmarks not only test the models’
ability to generalize to unseen objects during training but also assess their
ability to recognize object attributes. These benchmarks show that current
methods are far from achieving satisfactory results. Our work follows up on
those recent findings and investigates potential causes for the performance
gap in fine-grained settings.

##  III Method

Figure 2: FG-OVD Dataset for CLIP Matching. We leverage the Fine-Grained Open-
Vocabulary object Detection (FG-OVD) benchmark suite and training set to
investigate our two research questions Q1 and Q2. For each object, we extract
the corresponding bounding box crop and compute its visual encoding using
CLIP. Text embeddings are then generated for the assigned vocabulary entries
(composed by positive + negative captions) associated with the object.
Finally, we calculate the similarity and rank of the positive caption between
the image crop and the vocabulary entries. To address Q1 we use a cosine
similarity as model S, and the entire pipeline is used only during inference.
To address Q2, we choose model S from the solutions described in subsection
III-B and train it on the FG-OVD training set.

In this work, we aim to address two pivotal questions:
Q1. _Can the limitations observed in open-vocabulary object detection
regarding fine-grained understanding be traced back to deficiencies within the
CLIP latent space?_
Q2. If yes, _do these limitations arise from an absence of such information in
the latent space itself, or is it a consequence of the inadequacy of trivial
matching methods (e.g., dot product, cosine similarity) to extract this
nuanced information?_
In this section, we outline the methodologies employed to address these
inquiries.

###  III-A CLIP fine-grained evaluation

Our objective in this part is to address question Q1, aiming to discern
whether the failure of open-vocabulary object detectors can be attributed to
the image-text matching in the embedding space or to the localization phase.

To assess the fine-grained knowledge contained in the CLIP latent space and to
facilitate a comprehensive analysis in comparison to open-vocabulary object
detectors, we used the benchmark suite tailored for Fine-Grained Open-
Vocabulary Detection (FG-OVD) proposed in [2]. These benchmarks provide object
bounding boxes, each associated with a detailed natural language caption that
provides attributes about the object, referred to as a positive caption. In
addition, each positive caption is associated with a set of semantically
similar but subtly different negative captions.

For the purposes of matching rather than detection, we evaluate each object
individually. This adaptation involved retaining a vocabulary specific to each
object and cropping the associated bounding box, sparing the model from
performing region proposal and detection. Then, we compute the similarity
between the cropped image embedding and the captions embedding using the
cosine similarity. This evaluation pipeline is outlined in Figure 2.

To benchmark these results, we compare CLIP with OWL [22, 21], an open-
vocabulary object detector that relies on CLIP. The performance gap between
the two is only explained by the errors introduced by the region proposal and
object localization phases, thus providing an estimate of their contribution
to open-vocabulary detection performance.

###  III-B Latent Space Characteristics and Matching Approaches

This section provides the strategy to answer question Q2. Assuming that the
fine-grained knowledge is present within the CLIP latent space, we hypothesize
that the matching scheme used to compare the representations, i.e., the
typical cosine similarity, is insufficient to extract this specific
information. To explore this possibility, our strategy involves learning a
customized similarity function $S(\mathbf{v},\mathbf{t})$, which takes as
input the two embeddings $\mathbf{v}$ and $\mathbf{t}$ obtained from the
frozen visual and textual encoders, $\phi_{v}$ and $\phi_{t}$ respectively. By
forcing $S$ to recognize nuanced object properties based only on the embedded
information, we can state that successful results in this regard mean that the
embeddings inherently encode fine-grained knowledge. To this aim, we will use
two distinct datasets having similar image distribution but different
annotations. The first one comprises general image-text pairs, which can be
used to train and validate our model on standard coarse-grained category-
centric classification. Differently, the second dataset is dedicated to
training and evaluating the learned function $S$ for fine-grained
understanding and is organized in positive and negative captions for each
object as explained in Section III-A.

Therefore, the overall training strategy is composed of two steps. First, we
perform a warm-up phase in which we train $S$ on the coarse-grained image-text
pairing dataset.Then, we fine-tune $S$ using the fine-grained matching
dataset. The first warm-up phase works as a fine-tuning of the original CLIP
model, which repurposes the CLIP features to work well with our image
distribution and consequently initializes the $S$ function. This creates a
strong and reliable baseline that we can employ as a reference to track the
subsequent decline in coarse-grained performance after the fine-tuning step.

We perform the warm-up on the coarse-grained dataset using the hinge-based
triplet loss as a loss function. Namely, we optimize

$\mathcal{L}=\sum_{\begin{subarray}{c}i,j\in\mathcal{B}\\\ i\neq
j\end{subarray}}[\alpha+S(\mathbf{v}_{i},\mathbf{t}_{j})-S(\mathbf{v}_{i},\mathbf{t}_{i})]_{+}+[\alpha+S(\mathbf{v}_{j},\mathbf{t}_{i})-S(\mathbf{v}_{i},\mathbf{t}_{i})]_{+}\,,$
(1)

where $S$ is the similarity function, $\mathcal{B}=\\{1\dots B\\}$ is a batch,
$i$ is the index of an image $I_{i}$ described by the text $T_{i}$, while
$j\neq i$ is the index of a negative image $I_{j}$ and a negative text $T_{j}$
taken from the batch. $\phi_{v}$ and $\phi_{t}$ are respectively the visual
and textual CLIP encoders, and $\mathbf{v}_{i}=\phi_{v}(I_{i})$ and
$\mathbf{t}_{i}=\phi_{t}(T_{i})$ are the frozen encoded text and image,
respectively. $\alpha$ is the margin of separation.

We then perform training on the fine-grained dataset, where each image $I_{i}$
is associated with a vocabulary composed of a positive caption and a set of
$N$ similar but slightly different captions, called negative captions. Again,
we rely on the hinge-based triplet loss, but this time, we adapt it for the
fine-grained discrimination task: for each tuple (image, vocabulary), we take
as anchor the image, as positive the correct caption, and as negative the
incorrect ones. Formally,

$\mathcal{L}_{\text{FG}}=\sum_{i=1}^{B}\sum_{j=1}^{N}[\alpha+S(\mathbf{v}_{i},\mathbf{t}_{i,j}^{\text{neg}})-S(\mathbf{v}_{i},\mathbf{t}_{i}^{\text{pos}})]_{+}\,,$
(2)

where $B$ is the batch size, $\mathbf{v}_{i}$ is the encoding of the $i$-th
image, $\mathbf{t}_{i}^{\text{pos}}$ is the encoding of the positive caption
associated with the $i$-th image, $\mathbf{t}_{i,j}^{\text{neg}}$ is the
encoding of the $j$-th negative caption associated with image $i$, $\alpha$ is
the minimum separation margin that should hold between positive and negative
captions, and $[x]_{+}\equiv\max(x,0)$.

We evaluate different implementations of the matching function $S$:

####  III-B1 Baseline (CLIP matching function)

$S(\mathbf{v},\mathbf{t})=\cos(\mathbf{v},\mathbf{t})$ (3)

The vanilla cosine similarity represents the commonly used matching function
used in CLIP and open-vocabulary object detectors to match visual and textual
representations.

####  III-B2 Linear projection layer

$S(\mathbf{v},\mathbf{t})=\cos(W_{v}\mathbf{v}+b_{v},W_{t}\mathbf{t}+b_{t})$
(4)

We propose two linear projection layers that operate on top of the frozen
visual ($\mathbf{v}$) and textual ($\mathbf{t}$) features, aiming to project
the embeddings into a space of the same dimensionality. The final matching
function is kept as the cosine similarity between the transformed feature
vectors. We aim to explore the feasibility of linearly separating fine-grained
concepts (if they exist) embedded in the CLIP embeddings.

####  III-B3 Linear projection layer only above text encoder

$S(\mathbf{v},\mathbf{t})=\cos(\mathbf{v},W_{t}\mathbf{t}+b_{t})$ (5)

We introduce a linear projection layer solely above the text encoder while
keeping the visual embedding fixed. This setup forces the image latent space
to maintain coherence with the original CLIP space while repurposing only
textual representations.

####  III-B4 Linear projection layer only above visual encoder

$S(\mathbf{v},\mathbf{t})=\cos(W_{v}\mathbf{v}+b_{v},\mathbf{t})$ (6)

Similarly, we apply a linear projection layer solely above the visual encoder,
following the same principle as the previous scheme. By projecting only one
embedding modality at a time, we can explore any potential asymmetry in the
results and gain insights into the role of each modality in capturing fine-
grained information.

####  III-B5 MLPs layer

$S(\mathbf{v},\mathbf{t})=\cos(\text{MLP}_{v}(\mathbf{v}),\text{MLP}_{t}(\mathbf{t}))$
(7)

We incorporate two Multi-Layer Perceptrons (MLPs) with a non-linear activation
function to repurpose the embeddings before performing the cosine similarity.
This scheme aims to explore the effects of non-linearity on both coarse-
grained and fine-grained performance and to compare these results with the
effects observed with linear projection approaches.

####  III-B6 Attention layer

$S(\mathbf{v},\mathbf{t})=\sigma\left(\text{MHA}([\text{CLS},\mathbf{v},\mathbf{t}])^{(0,0)}\right)$
(8)

We build a multi-head attention layer above the two encoders to compute self-
attention utilizing the two embeddings along with a to-be-learned CLS token of
the same dimensionality. Then, after the attention layer, we apply the sigmoid
function to the first element of the CLS token output. The score of the
sigmoid represents the similarity between the two embeddings. This approach
explores a more complex and expressive non-linear alternative to the MLP that
exploits attention to automatically weight the contribution of image and text
features.

##  IV Experiments

###  IV-A Dataset and Metrics

Figure 3: CLIP vs. OWL in fine-grained understanding. We evaluate CLIP and OWL, configured as B/16 and L/14, against the Difficulty-based (first row) and Attribute-based (second row) FG-OVD benchmarks. The bar graph shows the Mean Rank of the positive label (lower is better), which represents the average position assigned by the model to the correct label within the overall vocabulary. Vocabulary lengths vary, with 3 for transparency, 8 for pattern, and 11 for other attributes. TABLE I: Coarse-grained and Fine-grained performance. We analyze the performance of the investigated similarity functions $S$ after a warm-up train on COCO and a subsequent fine-tuning on the fine-grained dataset FG-OVD. In the fine-tuned configuration (+FG-OVD rows), we denote the delta between these results and those obtained during the warm-up in parentheses. |  | COCO Retrieval
---|---|---
| FG-OVD | I$\to$T | T$\to$I
| Mean Rank $\downarrow$ | R@1 $\uparrow$ | R@5 $\uparrow$ | R@10 $\uparrow$ | R@1 $\uparrow$ | R@5 $\uparrow$ | R@10 $\uparrow$
CLIP B/16 | 2.98 | 41.5 | 65.9 | 76.2 | 22.6 | 44.1 | 54.9
Linear (both) | 3.78 | 48.0 | 76.6 | 85.4 | 37.2 | 65.6 | 76.6
+FG-OVD | 1.46 (-2.32) | 37.1 (-10.9) | 66.8 (-9.8) | 78.4 (-7.0) | 35.6 (-1.6) | 63.9 (-1.7) | 75.0 (-1.6)
Linear (visual only) | 3.53 | 45.8 | 74.2 | 83.7 | 35.4 | 64.2 | 75.3
+FG-OVD | 1.54 (-1.99) | 39.4 (-6.4) | 69.5 (-4.7) | 79.8 (-3.9) | 34.3 (-1.1) | 62.9 (-1.3) | 74.1 (-1.2)
Linear (text only) | 3.48 | 47.3 | 75.1 | 84.9 | 36.0 | 64.3 | 75.6
+FG-OVD | 1.57 (-1.91) | 41.1 (-6.2) | 70.4 (-4.7) | 80.6 (-4.3) | 34.7 (-1.3) | 63.2 (-1.1) | 74.6 (-1.0)
MLP | 3.49 | 45.9 | 75.5 | 84.6 | 36.5 | 64.9 | 76.2
+FG-OVD | 1.43 (-2.06) | 31.9 (-14.0) | 60.2 (-15.3) | 72.7 (-11.9) | 33.6 (-2.9) | 62.0 (-2.9) | 73.9 (-2.3)
MHA | 4.08 | 36.3 | 66.1 | 78.1 | 29.1 | 57.6 | 70.3
+FG-OVD | 1.54 (-2.54) | 22.3 (-14.0) | 48.3 (-17.8) | 61.2 (-16.9) | 22.6 (-6.5) | 49.2 (-8.4) | 62.0 (-8.3)

For the warm-up phase and to evaluate the coarse-grained capabilities of our
model, we selected the MS-COCO dataset as our image-text pair dataset. We
followed the partitioning introduced by Karpathy et al.[12], reserving 113,287
images for training, 5,000 for validation, and 5,000 for testing, each with
five captions. We evaluated retrieval performance on COCO using the Recall@$k$
metric to measure the ability of our model to retrieve relevant text or images
accurately. Specifically, Recall@$k$ assesses the proportion of queries that
successfully retrieve the correct item within the first $k$ results.

We used the Fine-Grained Open-Vocabulary Object Detection (FG-OVD) [2] suite
for fine-grained training and evaluation, adapting it for classification using
crops of object regions instead of detection. This suite associates each
object with a customized vocabulary consisting of a detailed sentence
describing the object and its attributes, called the positive caption, and a
set of negative captions subtly altering certain attributes (see Figure 2).
The benchmarks within the suite are categorized by difficulty level (Trivial,
Easy, Medium, and Hard), with the degree of change in the captions decreasing
as the selected benchmark becomes more difficult. For example, the Hard
benchmark indicates that only one attribute is changed in the negative labels.
In addition, the suite includes attribute-based benchmarks (Color, Material,
Pattern, and Transparency), where only attributes of a particular type are
changed, making it easier to evaluate model performance in each attribute
category. To evaluate on this benchmark, we measured the Mean Rank of the
positive label within the vocabulary when ranked by descending matching score.
As for the number of negative labels in the vocabulary $N$, we follow [2] and
choose 10 for Trivial, Easy, Medium, Hard, Color and Material, 7 for Pattern,
and 2 for Transparency. In subsection IV-C, we plot the Mean Rank for each
benchmark, while in subsection IV-D, we report the mean of the values obtained
from the eight benchmarks.

For training, we used the Hard training set with $N=10$ negatives.

###  IV-B Implementation details

For the warm-up phase on COCO, we train the matching function $S$ with the
Adam optimizer, using a learning rate of 5e-4 for 10 epochs and a triplet loss
margin $\alpha=0.2$. For fine-tuning on the FG-OVD, we use Adam with a reduced
learning rate (1e-5) and set the triplet loss separation margin $\alpha=0.05$.
Maintaining a deliberately low margin is critical, as higher values were
observed to disrupt the alignment established during the warm-up phase,
significantly reducing retrieval performance on COCO. The fine-tuning process
includes 10 epochs. Regarding the newly added layers, we set up the attention
layer with 64 heads, while the MLPs feature 2 layers with 512 neurons each and
the $tanh$ activation function. We train all the proposed architectures using
embeddings extracted from CLIP B/16.

###  IV-C Impact of object localization is marginal in FG-OVD

Figure 3 compares a CLIP-based open-vocabulary object detector with vanilla
CLIP applied to pre-segmented image regions on the FG-OVD benchmark suite. It
is important to note that the task of the detector is more challenging than
that of CLIP. Indeed, the detector must also localize the object rather than
solely classifying it. We make the comparison with OWL, an open vocabulary
object detector based on CLIP, presented in both its original [22] and second
[21] versions, with the same backbones (B/16 and L/14).

Looking at the results, the performance of CLIP mirrors the pattern shown by
the detectors. However, the overall performance of CLIP remains relatively
low. For example, in the Hard benchmark, the model ranks the correct caption
on average around 4th out of 11 possible captions. This highlights the
significant limitations of classification using the CLIP latent space and
points to the need for significant improvements in open-world fine-grained
classification.

Despite facing a more challenging task than CLIP, the detectors’ performance
is not significantly lower. Their results are consistent with CLIP’s patterns,
showing similar trends in both the Difficulty-based and Attribute-based
benchmarks, with better performance observed in the Color benchmark (an
attribute more commonly found in web-scraped images), while the Material and
Pattern benchmarks show lower performance. This suggests that the recent
challenges open vocabulary object detectors face in fine-grained understanding
are more related to classification within the shared image-text embedding
space than to the localization phase.

###  IV-D A linear projection is enough for fine-grained matching

The results presented in Table I illustrate the performance of the proposed
architecture after the warm-up on COCO and the subsequent fine-tuning on FG-
OVD.

Observing the results, the warm-up on COCO improves the performance of COCO
retrieval compared to the original CLIP, indicating increased specialization
in the characteristics of COCO captions and images. Conversely, performance on
the fine-grained benchmark is poor because COCO captions generally lack
detailed attributes about objects.

After fine-tuning on the fine-grained dataset, there’s a slight decrease in
retrieval performance on COCO. However, the key observation is the significant
decrease in the FG-OVD Mean Rank. This suggests that the newly added layers,
relying solely on information from the embeddings, effectively discriminate
the correct attributes for object mapping. This addresses our question Q2,
where we wondered if fine-grained knowledge was missing in the CLIP latent
space or if we were not using the right tools to extract it. These results
show that we can learn a more complex similarity matching between the
representations and that nuanced information is indeed present in CLIP
embeddings. In addition, the fact that these results can be achieved only with
a linear projection, demonstrate that this type of information can be linearly
separated in the embedding space.

Comparing the MLPs with the linear projection, it is evident that nonlinearity
does not provide any advantage in maintaining a favorable trade-off between
fine- and coarse-grained performance. Although the results on the fine-grained
benchmarks are slightly better, the retrieval performance on COCO worsens.

The results obtained with a linear projection applied to only one embedding
modality reveal interesting observations. During the warm-up phase, these
projections demonstrate superior fine-grained performance compared to models
with both embeddings repurposed. This phenomenon can be attributed to the fact
that only one projection needs to be learned, which is forced to maintain
coherence with the original CLIP latent space. Consequently, this leads to a
smaller fine-grained performance drop during the warm-up phase, which is
usually due to the limited fine-grained attribute information within the COCO
dataset. In addition, this adherence to the original embedding space results
in a minor coarse-grained performance degradation after fine-tuning compared
to other tested configurations. What is most compelling is that despite the
fine-grained results being slightly lower compared to the other matching
functions, there is a great improvement in the fine-grained results despite
the need to adhere to the CLIP latent space. This suggests that fine-grained
properties are not only present and linearly separable within the CLIP
embedding space but can even emerge with a simple linear adjustment of the
embedding of only one modality.

Experiments with the multi-head attention layer suggest that more complex and
expressive architecture can better adapt to the novel fine-grained task but
with a higher risk of overfitting the training data and disrupting the
original space.

##  V Conclusion

We studied the challenges confronting current state-of-the-art cross-modal
image-text models in achieving open-world understanding. We began our analysis
by examining the relationship between open-vocabulary object detectors and
their vision-language backbone, specifically focusing on CLIP. Our results
suggest that localization is marginal in the limitations observed in fine-
grained open-vocabulary object detection. This demonstrates that the primary
problem lies in the interaction between vision and language within the shared
latent space.

Furthermore, while fine-grained information exists within the CLIP latent
space, the representation is heavily biased towards coarse-grained concepts.
This bias causes similar concepts to be positioned too closely within the
latent space, making it difficult to detect nuanced differences using
traditional cosine similarity. Importantly, we demonstrated that fine-grained
information is still linearly separable within latent space despite this bias.

In the future, we aim to explore better pre-training strategies to construct
more balanced image-text representations that effectively incorporate fine-
and coarse-grained features. In addition, we will investigate alternative
matching functions capable of extracting fine-grained features within the CLIP
latent space without the need for task-specific datasets to learn this
function.

## References

  * [1] Relja Arandjelović, Alex Andonian, Arthur Mensch, Olivier J Hénaff, Jean-Baptiste Alayrac, and Andrew Zisserman.  Three ways to improve feature alignment for open vocabulary detection.  arXiv preprint arXiv:2303.13518, 2023.
  * [2] Lorenzo Bianchi, Fabio Carrara, Nicola Messina, Claudio Gennaro, and Fabrizio Falchi.  The devil is in the fine-grained details: Evaluating open-vocabulary object detectors for fine-grained understanding.  In [Accepted for publication in CVPR 2024], 2024.
  * [3] Maria A Bravo, Sudhanshu Mittal, Simon Ging, and Thomas Brox.  Open-vocabulary attribute detection.  In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 7041–7050, 2023.
  * [4] Keyan Chen, Xiaolong Jiang, Yao Hu, Xu Tang, Yan Gao, Jianqi Chen, and Weidi Xie.  Ovarnet: Towards open-vocabulary object attribute recognition.  In CVPR, 2023.
  * [5] Ahmad Darkhalil, Dandan Shan, Bin Zhu, Jian Ma, Amlan Kar, Richard Higgins, Sanja Fidler, David Fouhey, and Dima Damen.  Epic-kitchens visor benchmark: Video segmentations and object relations.  In Proceedings of the Neural Information Processing Systems (NeurIPS) Track on Datasets and Benchmarks, 2022.
  * [6] Alexey Dosovitskiy, Lucas Beyer, Alexander Kolesnikov, Dirk Weissenborn, Xiaohua Zhai, Thomas Unterthiner, Mostafa Dehghani, Matthias Minderer, Georg Heigold, Sylvain Gelly, Jakob Uszkoreit, and Neil Houlsby.  An image is worth 16x16 words: Transformers for image recognition at scale.  ICLR, 2021.
  * [7] Yu Du, Fangyun Wei, Zihe Zhang, Miaojing Shi, Yue Gao, and Guoqi Li.  Learning to prompt for open-vocabulary object detection with vision-language model.  pages 14084–14093, 2022.
  * [8] Fartash Faghri, David J Fleet, Jamie Ryan Kiros, and Sanja Fidler.  Vse++: Improving visual-semantic embeddings with hard negatives.  2018\.
  * [9] Kristen Grauman, Andrew Westbury, Eugene Byrne, Zachary Chavis, Antonino Furnari, Rohit Girdhar, Jackson Hamburger, Hao Jiang, Miao Liu, Xingyu Liu, et al.  Ego4d: Around the world in 3,000 hours of egocentric video.  In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 18995–19012, 2022.
  * [10] Xiuye Gu, Tsung-Yi Lin, Weicheng Kuo, and Yin Cui.  Open-vocabulary object detection via vision and language knowledge distillation.  2022\.
  * [11] Chao Jia, Yinfei Yang, Ye Xia, Yi-Ting Chen, Zarana Parekh, Hieu Pham, Quoc Le, Yun-Hsuan Sung, Zhen Li, and Tom Duerig.  Scaling up visual and vision-language representation learning with noisy text supervision.  In International conference on machine learning, pages 4904–4916. PMLR, 2021.
  * [12] Andrej Karpathy and Li Fei-Fei.  Deep visual-semantic alignments for generating image descriptions.  In Proceedings of the IEEE conference on computer vision and pattern recognition, pages 3128–3137, 2015.
  * [13] Benno Krojer, Vaibhav Adlakha, Vibhav Vineet, Yash Goyal, Edoardo Ponti, and Siva Reddy.  Image retrieval from contextual descriptions.  In Smaranda Muresan, Preslav Nakov, and Aline Villavicencio, editors, Proceedings of the 60th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers), pages 3426–3440, Dublin, Ireland, May 2022. Association for Computational Linguistics.
  * [14] Junnan Li, Ramprasaath Selvaraju, Akhilesh Gotmare, Shafiq Joty, Caiming Xiong, and Steven Chu Hong Hoi.  Align before fuse: Vision and language representation learning with momentum distillation.  Advances in neural information processing systems, 34:9694–9705, 2021.
  * [15] Kunpeng Li, Yulun Zhang, Kai Li, Yuanyuan Li, and Yun Fu.  Visual semantic reasoning for image-text matching.  In Proceedings of the IEEE/CVF international conference on computer vision, pages 4654–4662, 2019.
  * [16] Xiujun Li, Xi Yin, Chunyuan Li, Pengchuan Zhang, Xiaowei Hu, Lei Zhang, Lijuan Wang, Houdong Hu, Li Dong, Furu Wei, et al.  Oscar: Object-semantics aligned pre-training for vision-language tasks.  In Computer Vision–ECCV 2020: 16th European Conference, Glasgow, UK, August 23–28, 2020, Proceedings, Part XXX 16, pages 121–137. Springer, 2020.
  * [17] Jiasen Lu, Dhruv Batra, Devi Parikh, and Stefan Lee.  Vilbert: Pretraining task-agnostic visiolinguistic representations for vision-and-language tasks.  Advances in neural information processing systems, 32, 2019.
  * [18] Nicola Messina, Giuseppe Amato, Andrea Esuli, Fabrizio Falchi, Claudio Gennaro, and Stéphane Marchand-Maillet.  Fine-grained visual textual alignment for cross-modal retrieval using transformer encoders.  ACM Transactions on Multimedia Computing, Communications, and Applications (TOMM), 17(4):1–23, 2021.
  * [19] Nicola Messina, Fabrizio Falchi, Andrea Esuli, and Giuseppe Amato.  Transformer reasoning network for image-text matching and retrieval.  In 2020 25th International conference on pattern recognition (ICPR), pages 5222–5229. IEEE, 2021.
  * [20] Nicola Messina, Matteo Stefanini, Marcella Cornia, Lorenzo Baraldi, Fabrizio Falchi, Giuseppe Amato, and Rita Cucchiara.  Aladin: Distilling fine-grained alignment scores for efficient image-text matching and retrieval.  In International Conference on Content-based Multimedia Indexing, pages 64–70, 2022.
  * [21] Matthias Minderer, Alexey Gritsenko, and Neil Houlsby.  Scaling open-vocabulary object detection.  2023\.
  * [22] Matthias Minderer, Alexey Gritsenko, Austin Stone, Maxim Neumann, Dirk Weissenborn, Alexey Dosovitskiy, Aravindh Mahendran, Anurag Arnab, Mostafa Dehghani, Zhuoran Shen, et al.  Simple open-vocabulary object detection.  pages 728–755. Springer, 2022.
  * [23] Roni Paiss, Ariel Ephrat, Omer Tov, Shiran Zada, Inbar Mosseri, Michal Irani, and Tali Dekel.  Teaching clip to count to ten.  In Proceedings of the IEEE/CVF International Conference on Computer Vision, pages 3170–3180, 2023.
  * [24] Letitia Parcalabescu, Michele Cafagna, Lilitta Muradjan, Anette Frank, Iacer Calixto, and Albert Gatt.  Valse: A task-independent benchmark for vision and language models centered on linguistic phenomena.  arXiv preprint arXiv:2112.07566, 2021.
  * [25] Leigang Qu, Meng Liu, Da Cao, Liqiang Nie, and Qi Tian.  Context-aware multi-view summarization network for image-text matching.  In Proceedings of the 28th ACM International Conference on Multimedia, pages 1047–1055, 2020.
  * [26] Alec Radford, Jong Wook Kim, Chris Hallacy, Aditya Ramesh, Gabriel Goh, Sandhini Agarwal, Girish Sastry, Amanda Askell, Pamela Mishkin, Jack Clark, et al.  Learning transferable visual models from natural language supervision.  pages 8748–8763. PMLR, 2021.
  * [27] Kanchana Ranasinghe, Brandon McKinzie, Sachin Ravi, Yinfei Yang, Alexander Toshev, and Jonathon Shlens.  Perceptual grouping in contrastive vision-language models.  In Proceedings of the IEEE/CVF International Conference on Computer Vision, pages 5571–5584, 2023.
  * [28] Nikolaos Sarafianos, Xiang Xu, and Ioannis A Kakadiaris.  Adversarial representation learning for text-to-image matching.  In Proceedings of the IEEE/CVF international conference on computer vision, pages 5814–5824, 2019.
  * [29] Matteo Stefanini, Marcella Cornia, Lorenzo Baraldi, and Rita Cucchiara.  A novel attention-based aggregation function to combine vision and language.  In 2020 25th International Conference on Pattern Recognition (ICPR), pages 1212–1219. IEEE, 2021.
  * [30] Weijie Su, Xizhou Zhu, Yue Cao, Bin Li, Lewei Lu, Furu Wei, and Jifeng Dai.  Vl-bert: Pre-training of generic visual-linguistic representations.  In International Conference on Learning Representations, 2020.
  * [31] Wenhui Wang, Hangbo Bao, Li Dong, Johan Bjorck, Zhiliang Peng, Qiang Liu, Kriti Aggarwal, Owais Khan Mohammed, Saksham Singhal, Subhojit Som, and Furu Wei.  Image as a foreign language: BEiT pretraining for vision and vision-language tasks.  In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, 2023.
  * [32] Keyu Wen, Xiaodong Gu, and Qingrong Cheng.  Learning dual semantic relations with graph attention for image-text matching.  IEEE transactions on circuits and systems for video technology, 31(7):2866–2879, 2020.
  * [33] Xiaoshi Wu, Feng Zhu, Rui Zhao, and Hongsheng Li.  CORA: Adapting clip for open-vocabulary detection with region prompting and anchor pre-matching.  pages 7031–7040, 2023.
  * [34] Mert Yuksekgonul, Federico Bianchi, Pratyusha Kalluri, Dan Jurafsky, and James Zou.  When and why vision-language models behave like bags-of-words, and what to do about it?  In The Eleventh International Conference on Learning Representations, 2022.
  * [35] Pengchuan Zhang, Xiujun Li, Xiaowei Hu, Jianwei Yang, Lei Zhang, Lijuan Wang, Yejin Choi, and Jianfeng Gao.  Vinvl: Revisiting visual representations in vision-language models.  In Proceedings of the IEEE/CVF conference on computer vision and pattern recognition, pages 5579–5588, 2021.
  * [36] Yiwu Zhong, Jianwei Yang, Pengchuan Zhang, Chunyuan Li, Noel Codella, Liunian Harold Li, Luowei Zhou, Xiyang Dai, Lu Yuan, Yin Li, et al.  RegionCLIP: Region-based language-image pretraining.  pages 16793–16803, 2022.
  * [37] Xingyi Zhou, Rohit Girdhar, Armand Joulin, Philipp Krähenbühl, and Ishan Misra.  Detecting twenty-thousand classes using image-level supervision.  pages 350–368. Springer, 2022.


# Review on Axions

Andreas Ringwald1 1Deutsches Elektronen-Synchrotron DESY, Notkestr. 85, 22607
Hamburg, Germany<EMAIL_ADDRESS>
###### Abstract

This proceedings’ contribution explores the rationale behind the axion as a
resolution to the strong CP puzzle. It outlines various benchmark axion models
and examines their implications, focusing on two key aspects: (i) the axion’s
interactions with the Standard Model particles, and (ii) its potential role as
dark matter. Additionally, it provides an overview of the discovery prospects
associated with ongoing and future axion experiments. These efforts span from
laboratory-based endeavors aimed at directly producing and detecting axions to
searches for solar axions, and finally, to the direct detection of axion dark
matter.

##  1 The Axion

###  1.1 The strong CP puzzle

The axion emerges within extensions of the Standard Model (SM) that elegantly
address two fundamental questions simultaneously: i) the enigma of dark matter
(DM), and ii) the remarkable precision of time reversal ($T$) and parity ($P$)
invariance in strong interactions. Notably, experimental measurements of the
most sensitive observable of $T$ and $P$ violation in flavor-conserving
interactions – the electric dipole moment of the neutron (nEDM) – have thus
far only yielded an upper bound: $|d_{n}|<1.8\times 10^{-26}\,e\,{\rm cm}$
[1]. In contrast, from dimensional analysis, one expects an nEDM of
order111Here, $e$ is the unit of electric charge, and $m_{n}$ is the mass of
the neutron. $d_{n}\sim e/m_{n}\sim 10^{-13}\,e\,{\rm cm}$, if strong
interactions violate $T$ and $P$. Intriguingly, soon after the proposition of
Quantum Chromo-Dynamics (QCD) as the fundamental theory of strong
interactions, it was revealed that its most general Lagrangian encompasses a
term which violates both $T$ and $P$, and thus $CP$, where $C$ denotes charge
conjugation: the $\theta$-term,

${\mathcal{L}}_{\rm
QCD}\supset\overline{\theta}\,\frac{\alpha_{s}}{8\pi}G_{\mu\nu}^{b}\tilde{G}^{b,\mu\nu}\,.$
(1)

Here, $\alpha_{s}=g_{s}^{2}/(4\pi)$ is the strong fine-structure constant in
terms of the strong coupling $g_{s}$, $G_{\mu\nu}^{b}$
($\tilde{G}^{b,\mu\nu}$) is the (dual) gluonic field strength tensor, and
$\overline{\theta}\in[-\pi,\pi]$ is an angular parameter, which serves as a
measure of the strength of $T$ and $P$ violation in QCD. It induces an nEDM
[2] approximately of the order $d_{n}\sim\bar{\theta}\,e\,m_{*}/m_{n}^{2}\sim
10^{-16}\ \bar{\theta}\ e\,{\rm cm}$, where $m_{*}=m_{u}m_{d}/(m_{u}+m_{d})$
denotes the reduced mass of the $u$ and $d$ quarks. Consequently, the
experimental upper bound imposes a stringent constraint on the
$\overline{\theta}$-parameter: $|\overline{\theta}|\lesssim 10^{-10}$. The
unnatural small value of this quantitiy constitutes the strong $CP$ puzzle.

###  1.2 Minimal SM extension solving the strong CP puzzle

Kim [3], Shifman, Vainshtein, and Zakharov [4] (KSVZ) proposed a minimal
renormalizable extension of the SM that effectively tackles the strong $CP$
puzzle. This extension introduces a new complex scalar field $\sigma$, which
is a singlet under the SM, featuring a global axial Peccei-Quinn [5] (PQ)
$U(1)$ symmetry. This symmetry is assumed to be spontaneously broken at a
scale $v_{\rm PQ}$ much higher than the electroweak scale, as ensured by a
postulated Mexican hat like potential term in the Lagrangian,

$\mathcal{L}_{\rm
KSVZ}\;\supset\;-\lambda_{\sigma}\left(\left|\sigma\right|^{2}-{v_{\rm
PQ}^{2}}/{2}\right)^{\\!\\!2}.$

Additionally, a novel quark, denoted as $\mathcal{Q}$, is introduced, which
carries charge under the $U(1)_{\rm PQ}$ symmetry and interacts with the
complex PQ scalar $\sigma$ via a Yukawa coupling:

$\mathcal{L}_{\rm
KSVZ}\;\supset\;y\,\sigma\,\bar{\mathcal{Q}}_{L}{\mathcal{Q}}_{R}+{\rm h.c.}.$

Utilizing a parameterization of the complex field $\sigma$ in terms of its
modulus and phase, expressed as $\sigma(x)=(1/{\sqrt{2}})\left(v_{\rm
PQ}+\rho(x)\right){\rm e}^{ia(x)/v_{\rm PQ}}$, we can discern that this model
introduces three particles beyond the SM framework. Among them, two, denoted
as $\rho$ and $\mathcal{Q}$ , exhibit significant mass, approximately on the
scale of the PQ breaking scale.

Figure 1: Left: Triangle loop diagram giving rise to the axion coupling to
gauge fields. Right: Effective potential of the axion field.

Specifically, their masses are given by
$m_{\rho}=\sqrt{2\lambda_{\sigma}}v_{\rm PQ}$ and $m_{\mathcal{Q}}={y}v_{\rm
PQ}/{\sqrt{2}}$, respectively. Conversely, one particle, labeled as $a$,
remains massless, serving as the Nambu-Goldstone boson arising from the
breaking of the $U(1)_{\rm PQ}$ symmetry. By integrating out the heavy
particles $\rho$ and $\mathcal{Q}$, the dynamics of the massless field $a$ can
be described by an effective Lagrangian:

$\mathcal{L}_{\rm
eff}\supset\frac{1}{2}\partial^{\mu}a\,\partial_{\mu}a+\frac{\alpha_{s}}{8\pi}\frac{a}{f_{a}}G_{\mu\nu}^{a}\tilde{G}^{a\,\mu\nu}.$

The second term arises from the triangle loop diagram depicted in Fig. 1
(left), from which one can read-off the factor of proportionality
$\alpha_{s}/(8\pi f_{a})$, where $f_{a}=v_{\rm PQ}/N_{\mathcal{Q}}$ is the
axion’s decay constant, with $N_{\mathcal{Q}}$ the number of PQ-charged quark
flavors (one in the simplest scenario). Remarkably, this term assumes a form
analogous to the $\theta$-term (1) of QCD. Consequently, the
$\overline{\theta}$-parameter can be rendered insignificant through a simple
shift in the $a$ field: $a(x)+\overline{\theta}\,f_{a}\to a(x)$. Subsequently,
considering the mixing of the shifted $a$ field with the pion field, a non-
trivial effective potential emerges for the former, featuring an absolute
minimum at zero field value, as illustrated in Fig. 1 (right). Put simply, the
shifted $a$ field exhibits a vacuum expectation value of zero, $\langle
a\rangle=0$, effectively resolving the strong $CP$ puzzle [5]. The particle
[6, 7] associated with excitations of the shifted $a$ field has been termed
the “axion”. Its mass is determined by the second derivative of the axion
potential around the minimum, given by
$m_{a}={\sqrt{V^{\prime\prime}(0)}}\simeq\frac{\sqrt{z}}{1+z}\,\frac{m_{\pi}\,f_{\pi}}{f_{a}}\approx
6\ {\rm\mu eV}\left(\frac{10^{12}\,{\rm GeV}}{f_{a}}\right)$, where
$z=m_{u}/m_{d}\approx 1/2$, and $m_{\pi}$ and $f_{\pi}$ denote the mass and
decay constant of the neutral pion, respectively.

###  1.3 Electromagnetic coupling

Of paramount importance in both experimental and astrophysical [8] realms is
the axion’s interaction with electromagnetic fields, described by the term

$\mathcal{L}_{\rm
eff}\supset\frac{1}{4}g_{a\gamma}\,a\,F_{\mu\nu}\tilde{F}^{\mu\nu}\equiv
g_{a\gamma}\,a\,{\mathbf{E}}\cdot{\mathbf{B}}$ (2)

in its interaction Lagrangian, in terms of the (dual) electromagnetic field
strength tensor $F_{\mu\nu}^{b}$ ($\tilde{F}^{b,\mu\nu}$) and the electric and
magnetic fields, ${\mathbf{E}}$ and ${\mathbf{B}}$, respectively. Here, the
coupling parameter

$g_{a\gamma}\simeq\frac{\alpha}{2\pi
f_{\pi}}\frac{m_{a}}{m_{\pi}}\frac{1+z}{\sqrt{z}}\left({6\,q_{\mathcal{Q}}^{2}}{}-\frac{2}{3}\frac{4+z}{1+z}\right)$

is proportional to the axion mass and the electromagnetic fine-structure
constant, $\alpha=e^{2}/(4\pi)$. The first term within the brackets takes into
account a possible electric charge $q_{\mathcal{Q}}$, in units of $e$, of the
exotic quark $\mathcal{Q}$, while the second term originates from the
phenomenon of axion-pion mixing. In Fig. 2, the line labeled ‘KSVZ’ represents
the predicted values of $g_{a\gamma}$ for $q_{\mathcal{Q}}=0$. When the
electric charge $q_{\mathcal{Q}}$ is allowed to vary, it leads to a range of
predictions depicted by the yellow ‘band’, which is labeled as ‘Vanilla
Axions’ in Fig. 2 [10].

Figure 2: Electromagnetic coupling of the axion versus its mass (adapted from
Ref. [9]). Haloscope exclusion regions (filled blue) and projected
sensitivities (dashed blue lines) assume the axion to be 100% of the halo dark
matter.

The alternate yellow ‘band’ of predictions, denoted as ‘Monopole-philic
Axions’ in Fig. 2, emerges under the condition where the KSVZ exotic quark
carries a magnetic charge $g_{\mathcal{Q}}$, in units of the fundamental
magnetic charge $g_{0}$, rather than an electric charge $q_{\mathcal{Q}}$, in
units of the fundamental electric charge $e$, In this scenario, the model not
only resolves the strong CP puzzle but also addresses the challenge of charge
quantization, as originally envisaged by Dirac, Schwinger, and Zwanziger [11,
12, 13]. The triangular loop diagram depicted in the left panel of Fig. 1
induces an electromagnetic coupling $g_{am}$, which can be approximated as

$g_{am}\simeq\frac{\alpha_{m}}{2\pi
f_{\pi}}\frac{m_{a}}{m_{\pi}}\frac{1+z}{\sqrt{z}}6g_{\mathcal{Q}}^{2},$

where $\alpha_{m}$ denotes ${g_{0}^{2}}/{(4\pi)}$ [14, 15, 16, 17].222It’s
worth mentioning that the phenomenological viability of this non-standard
electromagnetic axion coupling has been recently challenged [18], and our
rebuttal can be found in an upcoming paper [19]. The principle of charge
quantization, represented by $eg_{0}=6\pi n$, where $n$ is an integer, leads
to a parametric amplification of the coupling ratio,

$\frac{g_{am}}{g_{a\gamma}}=\frac{9}{4}\alpha^{-2}\left(\frac{g_{\mathcal{Q}}}{q_{\mathcal{Q}}}\right)^{2}\sim
10^{5},$

as illustrated in Fig. 2. This enhanced ratio is not feasible within another
benchmark axion model proposed by Zhitnitsky [20] and Dine, Fischler, and
Srednicki [21]. In this so-called ‘DFSZ’ model, the axion couplings to the SM
gauge fields are generated by the SM quarks ($N_{\mathcal{Q}}=6$), none of
which carry magnetic charges. Consequently, the electromagnetic coupling in
this model falls within the lower boundary of the vanilla axion band depicted
in Fig. 2.

##  2 Axion Experiments Not Relying on Axion Dark Matter

###  2.1 Light-shining-through-a-wall searches

One powerful technique for producing and detecting axions in a controlled
laboratory environment, independent of specific astrophysical models, is the
“Light-Shining-through-a-Wall” (LSW) technique [22, 23].

Figure 3: LSW concept [24] (left) and magnet string of the ALPS experiment in
the HERA tunnel (right).

This method exploits the phenomenon that photons traveling along a transverse
magnetic field $B$ of length $L_{B}$ can undergo partial conversion into light
axions, and vice versa, see Fig. 3 (left). For light axions,
$m_{a}\ll(2\pi\omega/L_{B})^{1/2}$, where $\omega$ is the photon energy, the
probability of this conversion is approximately given by333Here, $g_{a\gamma}$
can be replaced by $g_{am}$ for the monopole-philic axion [16].

$P(\gamma\rightarrow
a)\simeq\frac{1}{4}\left(g_{a\gamma}BL_{B}\right)^{2}\simeq
P(a\rightarrow\gamma).$

By placing a light-tight barrier within the transverse magnetic field region,
the presence of axions can be detected by observing photons emerging behind
the barrier due to axion-photon conversion. The ALPS II experiment [25] at
DESY in Hamburg employs several advancements to enhance sensitivity compared
to previous LSW experiments [26]. It utilizes, as first proposed in Ref. [24],
two strings of recycled superconducting HERA dipole magnets – 12 before and 12
after the wall – within one of the straight sections of the HERA tunnel, see
Fig. 3 (right), opposed to just one magnet as in the preceding ALPS experiment
[27, 28]. Additionally, ALPS II incorporates an optical cavity on the after-
wall side to resonantly amplify the number of regenerated photons [29, 30,
31], contrasting with the use of only one optical cavity before the wall as in
ALPS. Data collection for ALPS II began in May 2023, with full sensitivity
expected to be achieved by 2025. It promises to be approximately 1000 times
more sensitive than previous LSW experiments such as ALPS and OSQAR [32],
thereby exploring previously uncharted parameter spaces, particularly for the
monopole-philic axions, see Fig. 2.

###  2.2 Helioscope searches

Another avenue for axion exploration is through the detection of solar axions
produced via the interaction of photons with the Coulomb field of nuclei in
the solar plasma, achievable with an axion helioscope [33]. This setup
involves a long dipole magnet pointed towards the Sun, where solar axions are
partially converted into photons and subsequently focused onto an X-ray
detector, see Fig. 4 (left).

Figure 4: Axion helioscope concept [34] (left) and CAD overview of the full
BabyIAXO assembly (right).

CAST at CERN has set an upper limit [35] on $g_{a\gamma}$ or $g_{am}$, which
closely approaches the limit derived from observations of stars in globular
clusters, see Fig. 2. The next-generation helioscope, BabyIAXO [36], see Fig.
4 (right), planned to be build in the HERA South Hall at DESY in Hamburg, is
designed to surpass the sensitivity of CAST by a factor of approximately four
in the same axion mass range, probing even the vanilla axion with masses
exceeding 10 meV, see Fig. 2. Data collection for BabyIAXO is anticipated to
commence around 2029.

##  3 Axion Dark Matter

Just a few years following their original proposition, it was discovered that
axions not only resolve the strong $CP$ puzzle but also emerge as promising
candidates for cold dark matter [37, 38, 39]. The prediction of the axion dark
matter abundance critically hinges on the cosmic history, particularly on
whether the PQ symmetry breaking occurs before or after the onset of the hot
Big Bang phase at time $t_{\rm hot}$. In the former scenario, known as the
‘pre-inflationary PQ symmetry breaking scenario,’ spatial gradients in the
axion field $\theta_{a}(x)=a(x)/f_{a}$ can be disregarded, and its cosmic
evolution can be fully described by a damped anharmonic oscillator. The
damping arises from Hubble friction, while the anharmonicity stems from the
specific form of the periodic axion potential, which, at temperatures
surpassing the QCD cross-over, is represented as
$V(\theta_{a},T)=\chi(T)(1-\cos\theta_{a})$, where $\chi(T)$ denotes the
topological susceptibility of QCD, which can be obtained from high-temperature
lattice QCD [40].

Figure 5: Top three orange bands: Theoretically predicted axion dark matter
mass ranges. Middle black band: Axion mass range excluded by astrophysics.
Green bands: Axion dark matter ranges probed by various haloscopes. Violet
band: Axion dark matter ranges probed by Nuclear Magnetic Resonance methods
sensitive to NEDM oscillations or the axion dark matter wind. Figure adapted
from Ref. [41].

The axion field remains fixed at its initial value, $\theta_{ai}\equiv
a(t_{\rm hot})/f_{a}$, until the Hubble expansion rate falls below the axion
mass $m_{a}(T)=\sqrt{\chi(T)}/f_{a}$. Subsequently, it oscillates around the
$CP$-conserving minimum, representing a coherent condensate of cold dark
matter. The predicted abundance of axion dark matter, stemming from this
mechanism called the ‘realignment mechanism,’ is estimated as [40]

$\Omega_{a}h^{2}\approx 0.12\,\left({6~{}\mu{\rm eV}\over
m_{a}}\right)^{1.165}\,\theta_{ai}^{2}\,.$

This prediction depends on both the axion mass and the initial value of the
axion field. Hence, in the pre-inflationary scenario, the requirement for
axion dark matter to avoid over-closure of the universe does not impose a
lower bound on the axion mass, as illustrated in Fig. 5. In contrast, in the
‘post-inflationary PQ symmetry breaking scenario,’ where $\theta_{ai}$ may
vary across different patches of the present universe, the realignment
mechanism contributes an average amount [40]

$\Delta\Omega_{a}h^{2}\approx 0.12\,\left(\frac{30~{}\mu{\rm
eV}}{m_{a}}\right)^{1.165}.$

Additionally, axion dark matter arises in this scenario also from the decay of
cosmic strings and domain walls, yet their predicted contribution is subject
to considerable uncertainties. Consequently, the plausible range of axion
masses capable of constituting 100% of the observed dark matter in post-
inflationary scenarios remains quite broad: $m_{a}\approx 26\ \mu{\rm eV}-0.5\
{\rm meV}$ for axion models with short-lived domain walls, such as the KSVZ
model ($N_{\rm DW}\equiv N_{\mathcal{Q}}=1$) [42, 43, 44, 45]. For models
featuring long-lived domain walls ($N_{\rm DW}>1$), like the DFSZ model with
PQ symmetry arising from an accidental discrete symmetry, the predicted mass
is notably higher [46, 47]: $m_{a}\gtrsim{\rm meV}$. Furthermore, substantial
density fluctuations in the initial state of the axion field in the post-
inflationary scenario lead to the formation of compact dark matter structures
called “miniclusters,” contributing to heightened theoretical uncertainty in
the local axion density for laboratory-based dark matter detection.

Figure 6: World map displaying current experiments searching for wavy dark
matter [9].

##  4 Axion Dark Matter Experiments

Axion dark matter experiments operate under the premise that the dark matter
halo surrounding the Milky Way consists predominantly of axions. The velocity
dispersion of these axions is determined by the galactic virial velocity,
which in turn implies a macroscopic de Broglie wavelength, denoted as
$\lambda_{\rm dB}=2\pi/(m_{a}v_{a})\simeq{\rm km}\,(\mu{\rm
eV}/m_{a})(10^{-3}\,c/v_{a})$. Consequently, axions within the halo of dark
matter behave akin to a spatially homogeneous and monochromatic classical
oscillating field, characterized by the expression
$a(t)\simeq\sqrt{2\rho_{a}}\cos(m_{a}t)/m_{a}$, thus falling into the category
referred to as ‘wavy dark matter’. In the context of presenting projected
limits on the coupling of axions within the halo of dark matter, it is
presumed that the axion energy density, denoted as $\rho_{a}$, is
approximately equal to the energy density of dark matter in the halo,
$\rho_{\rm DM}^{\rm halo}\approx 0.45\,{\rm GeV}\,{\rm cm}^{-3}$.

The current projections for the axion mass, potentially accounting for the
observed dark matter content, encompass a wide range of values spanning
multiple orders of magnitude, see Fig. 5. Consequently, investigating this
extensive parameter space necessitates the utilization of various experimental
techniques that may target different axion couplings.

An extensive array of axion experiments is currently underway worldwide, as
depicted in Fig. 6. Given the sheer number of endeavors, it’s evident that I
can only focus on a subset of these experiments in the remainder of this brief
overview.

###  4.1 Haloscopes

Numerous experiments investigating axion dark matter rely on the
electromagnetic coupling (2) and are thus referred to as ‘haloscopes’. These
haloscopes employ various experimental techniques tailored to different mass
ranges, as indicated by the shaded green regions in Fig. 5.

####  4.1.1 Microwave cavities

The original concept of the classic haloscope was introduced by Sikivie [33].
In this setup, a microwave cavity is positioned within a magnetic field
$B_{0}$, allowing the conversion of dark matter axions into photons. If the
axion’s mass aligns with the resonance frequency of the cavity, denoted as
$m_{a}=2\pi\nu_{\rm res}\sim 4\,\mu{\rm eV}\left(\frac{\nu_{\rm res}}{\rm
GHz}\right)$, the power output experiences amplification proportional to the
cavity’s quality factor $Q$ and volume $V$: $P_{\rm out}\sim
g_{a\gamma}^{2}\,\rho_{\rm a}\,B_{0}^{2}\,Q\,V$. Given that the exact mass of
the axion remains unknown, one has to scan the mass range of interest by
tuning the cavity’s resonance frequency. Currently, there exist numerous
microwave cavity haloscopes in operation, among which ADMX [48] and CAPP [49]
have achieved DFSZ sensitivity within certain mass ranges, as depicted in Fig.
2. Over the next decade, microwave cavity axion dark matter searches, such as
ADMX [50], BabyIAXO [51], and FLASH [52], will delve deeper into the vanilla
axion band within the range $0.5\,{\rm\mu eV}\lesssim m_{a}\lesssim
100\,{\rm\mu eV}$, as illustrated in Fig.2. Regrettably, these searches remain
insensitive to the non-standard coupling $g_{am}$ of the monopole-philic
axions [53].

Figure 7: Dish antenna axion haloscope concept [54] (left) and picture of
BRASS (right).

####  4.1.2 Dish antennas

Exploring higher masses can be achieved through a broadband search approach
facilitated by the dish antenna haloscope concept [54]. This method
capitalizes on the characteristic of oscillating axion dark matter, which,
when subjected to a background magnetic field $\mathbf{B}$, generates an
oscillating electric field component parallel to the magnetic field, denoted
as ${\mathbf{E}}_{a}(t)=-g_{a\gamma}{\mathbf{B}}a(t)$. Consequently, when a
metallic mirror is positioned in a magnetic field aligned parallel to its
surface, it emits an almost monochromatic electromagnetic wave perpendicular
to the mirror surface, as depicted in Fig. 7 (left), with a frequency
$\nu=m_{a}/(2\pi)$ and a cycle-averaged power per unit area of
${{\mathcal{P}}_{\gamma}}/{\mathcal{A}}=|\mathbf{E}_{a}|^{2}/2$. The BRASS
collaboration at the University of Hamburg is presently establishing a pilot
dish antenna haloscope utilizing a permanently magnetized conversion panel
[55], see Fig. 7 (right). Meanwhile, the BREAD collaboration at Fermilab
intends to utilize a cylindric parabolic conversion panel, enabling the
employment of a much stronger solenoidal magnetic field [56]. In their
ultimate configuration, these experiments have the potential to explore the
vanilla axion band region within the range $50\,{\rm\mu eV}\lesssim
m_{a}\lesssim{\rm meV}$ (BRASS) or to delve deeply into it for $20\,{\rm
meV}\lesssim m_{a}\lesssim 0.1\,{\rm eV}$ (BREAD), as illustrated in Fig. 2.

Figure 8: Boosted dish antenna aka as dielectric haloscope concept [57] (left)
and sketch of MADMAX (right).

An alternative method for detecting axion dark matter in the mass range above
approximately $50\,{\rm\mu eV}$ is based on the dielectric haloscope concept
[57], essentially an enhanced version of the dish antenna haloscope. This
setup comprises a mirror and a series of parallel, partially transparent
dielectric disks placed in front of it, all positioned within a magnetic field
parallel to their surfaces, with a receiver situated in a field-free region,
see Fig. 8 (left). Each disk functions as a flat dish antenna, emitting waves
that are reflected by and transmitted through the other disks before exiting.
With suitable separations between the disks, these waves coherently add up to
amplify the emitted power. This approach enables scanning over a range of
axion masses without requiring disks of varying thickness for each
measurement. Building upon this principle, the MADMAX collaboration aims to
install an adjustable multiple-disk system (booster) with an effective area of
approximately $\mathcal{A}\sim 1\,$m2 within a dipole magnet of around $\sim
9$ T at DESY in Hamburg [58], see Fig. 8 (right). While data collection in
Hamburg may commence around 2030, an earlier prototype magnet could facilitate
axion dark matter searches in mass and frequency ranges that have largely
remained unexplored. With an anticipated power amplification factor of
$\beta^{2}(\nu)\sim 10^{4}$ and equipped with a quantum-limited receiver, the
complete MADMAX experiment is expected to survey the $(40-400)\,\mu$eV mass
range with DFSZ sensitivity [59], as depicted in Fig. 2.

####  4.1.3 Plasma haloscopes

Another method suitable for exploring the mass range above $50\,{\rm\mu eV}$
is based on the plasma haloscope concept [60]. This technique capitalizes on
the phenomenon that the oscillating axion dark matter field induces plasmon
excitations in a magnetized plasma, expressed as ${\bf E}=-g_{a\gamma}{\bf
B}_{\rm
e}a\left(1-\frac{\omega_{p}^{2}}{\omega_{a}^{2}-i\omega_{a}\Gamma}\right)^{-1}$.
These excitations are resonantly amplified when the plasma frequency matches
the axion mass, $\omega_{p}=\omega_{a}\approx m_{a}$, albeit constrained by
losses ($\Gamma$). A plasma with a tunable plasma frequency in the GHz range
can be achieved through a wire array with variable interwire spacing, often
referred to as a “wire metamaterial.” The ALPHA collaboration intends to
construct a tunable cryogenic plasma haloscope along these lines at Yale.
Currently, a pathfinder experiment is under development, expected to begin
data collection in 2026. The full-scale ALPHA experiment is projected to delve
deeply into the vanilla axion band within the $(40-400)\,\mu$eV mass range, as
illustrated in Fig. 2.

Figure 9: LC circuit haloscope concept [66] (left) and picture of the
solenoidal magnet for WISPLC at the University of Hamburg (right).

####  4.1.4 Lumped elements

Around a decade ago, the concept of a lumped element axion haloscope emerged,
offering sensitivity in the sub-micro-eV mass range [61]. This approach is
rooted in the fact that, under the influence of a magnetic field $\mathbf{B}$,
the axion dark matter field triggers an oscillating effective displacement
current, denoted as ${\mathbf{j}}_{a}=-g_{a\gamma}{\mathbf{B}}\dot{a}$,
subsequently generating an oscillating magnetic field ${\mathbf{B}}_{a}$ such
that ${\mathbf{\nabla}}\times{\mathbf{B}}_{a}={\mathbf{j}}_{a}$. The induced
field can be transformed into an alternating current within a pickup loop,
resonantly amplified within a tunable LC circuit, and ultimately detected via
a SQUID, as depicted in Fig. 9 (left). Pilot experiments have already been
conducted along these lines, including ABRACADABRA [62, 63], ADMX SLIC [64],
and SHAFT [65], while the next generation is currently commencing with
projects like WISPLC [66] (see Fig. 9 (right)) and DMRadio [67]. The
overarching objective of the DMRadio collaboration is to investigate axion
dark matter within the neV to micro-eV range, aiming for DFSZ sensitivity, as
depicted in Fig. 2. Additionally, a variant of the lumped-element haloscope
utilizing a high-voltage capacitor instead of a magnetic field may also
explore the $g_{am}$ coupling of the monopole-philic axion [68, 69].

###  4.2 NMR Experiments

Lastly, we highlight an axion dark matter experiment that does not rely on the
electromagnetic coupling but instead leverages the axion’s least model-
dependent coupling: its interaction with the nucleon electric dipole moment
(NEDM) operator. This coupling is described by

$\mathcal{L}_{aN\gamma}=-\frac{i}{2}g_{aN\gamma}\,a\,\overline{\psi}N\sigma{\mu\nu}\gamma_{5}\psi_{N}F^{\mu\nu},$

where $g_{an\gamma}=-g_{ap\gamma}\approx 6\times
10^{-19}\left(\frac{m_{a}}{\rm neV}\right)\frac{1}{\rm GeV^{2}}$. The
oscillating axion field in the halo dark matter induces oscillations in NEDMs,
expressed as $d_{N}(t)=g_{aN\gamma}\,\sqrt{2\rho_{a}}\cos(m_{a}\,t)/m_{a}$
[70]. These induce precession of nuclear spins in a sample polarized with
nucleon spins in the presence of an electric field. The resulting transverse
magnetization can be detected by employing nuclear magnetic resonance (NMR)
techniques, which are most sensitive at low oscillation frequencies
corresponding to sub-neV axion masses, as indicated by the violet band in Fig.
5. CASPEr-electric [71] in Boston is developing such an NMR axion search
experiment [72]. Its ultimate aim is to probe axion dark matter with neV
masses, corresponding to a Peccei-Quinn (PQ) scale of approximately $10^{16}$
GeV, as predicted in Grand Unified models (see, e.g., Refs. [73, 74, 75]).

##  5 Conclusions

In conclusion, there is a significant global effort underway in the search for
axions, utilizing a variety of experimental techniques and exploring diverse
couplings. The development of numerous novel experimental methods underscores
the dynamic nature of this field, often stemming from close collaborations
between theorists focused on phenomenological aspects and experimentalists
driven by theoretical interest. As advancements continue to unfold, we
encourage staying engaged and keeping abreast of the latest developments in
this exciting area of research. Stay tuned for further updates!

## Acknowledgments

Special thanks to A. Sokolov for valuable comments on the draft. This work has
been partially funded by the Deutsche Forschungsgemeinschaft (DFG, German
Research Foundation) under Germany’s Excellence Strategy – EXC 2121 Quantum
Universe – 390833306 and under – 491245950. This article/publication is based
upon work from COST Action COSMIC WISPers CA21106, supported by COST (European
Cooperation in Science and Technology).

## References

  * [1] C. Abel et al., Phys. Rev. Lett. 124 (2020) no.8, 081803 [arXiv:2001.11966 [hep-ex]].
  * [2] R. J. Crewther et al., Phys. Lett. B 88 (1979), 123 [erratum: Phys. Lett. B 91 (1980), 487].
  * [3] J. E. Kim, Phys. Rev. Lett. 43, 103 (1979).
  * [4] M. A. Shifman, A. I. Vainshtein and V. I. Zakharov, Nucl. Phys. B 166, 493-506 (1980).
  * [5] R. D. Peccei and H. R. Quinn, Phys. Rev. Lett. 38, 1440-1443 (1977).
  * [6] S. Weinberg, Phys. Rev. Lett. 40, 223-226 (1978).
  * [7] F. Wilczek, Phys. Rev. Lett. 40, 279-282 (1978).
  * [8] A. Caputo and G. Raffelt, [arXiv:2401.13728 [hep-ph]].
  * [9] C. O’Hare, doi:10.5281/zenodo.3932430, https://github.com/cajohare/AxionLimits
  * [10] L. Di Luzio et al., Phys. Rev. Lett. 118, no.3, 031801 (2017) [arXiv:1610.07593 [hep-ph]].
  * [11] P. A. M. Dirac, Proc. Roy. Soc. Lond. A 133, no.821, 60-72 (1931).
  * [12] J. S. Schwinger, Phys. Rev. 144, 1087-1093 (1966).
  * [13] D. Zwanziger, Phys. Rev. D 3, 880 (1971).
  * [14] A. V. Sokolov and A. Ringwald, JHEP 06 (2021), 123 [arXiv:2104.02574 [hep-ph]].
  * [15] A. V. Sokolov and A. Ringwald, PoS EPS-HEP2021 (2022), 178 [arXiv:2109.08503].
  * [16] A. V. Sokolov and A. Ringwald, [arXiv:2205.02605 [hep-ph]].
  * [17] A. V. Sokolov and A. Ringwald, Annalen Phys. 2023, 202300112 [arXiv:2303.10170].
  * [18] B. Heidenreich, J. McNamara and M. Reece, JHEP 01 (2024), 120 [arXiv:2309.07951 [hep-ph]].
  * [19] A. V. Sokolov and A. Ringwald, [arXiv:24mm.nnnnn [hep-ph]].
  * [20] A. R. Zhitnitsky, Sov. J. Nucl. Phys. 31, 260 (1980).
  * [21] M. Dine, W. Fischler and M. Srednicki, Phys. Lett. B 104, 199-202 (1981).
  * [22] A. A. Anselm, Yad. Fiz. 42, 1480-1483 (1985).
  * [23] K. Van Bibber, et al. Phys. Rev. Lett. 59, 759-762 (1987).
  * [24] A. Ringwald, Phys. Lett. B 569 (2003), 51-56 [arXiv:hep-ph/0306106 [hep-ph]].
  * [25] R. Bähre et al., JINST 8 (2013), T09001 [arXiv:1302.5647 [physics.ins-det]].
  * [26] J. Redondo and A. Ringwald, Contemp. Phys. 52 (2011), 211-236 [arXiv:1011.3741 [hep-ph]].
  * [27] K. Ehret et al., Nucl. Instrum. Meth. A 612 (2009), 83-96 [arXiv:0905.4159 [physics.ins-det]].
  * [28] K. Ehret et al., Phys. Lett. B 689 (2010), 149-155 [arXiv:1004.1313 [hep-ex]].
  * [29] F. Hoogeveen and T. Ziegenhagen, Nucl. Phys. B 358, 3-26 (1991).
  * [30] Y. Fukuda et al., Prog. Cryst. Growth Charact. Mater. 33 (1996) no.1-3, 363-366.
  * [31] P. Sikivie et al., Phys. Rev. Lett. 98 (2007), 172002 [arXiv:hep-ph/0701198 [hep-ph]].
  * [32] R. Ballou et al. [OSQAR], Phys. Rev. D 92 (2015) 092002 [arXiv:1506.08082 [hep-ex]].
  * [33] P. Sikivie, Phys. Rev. Lett. 51 (1983), 1415-1417 [erratum: Phys. Rev. Lett. 52 (1984), 695]
  * [34] E. Armengaud et al., JINST 9 (2014), T05002 [arXiv:1401.3233 [physics.ins-det]].
  * [35] V. Anastassopoulos et al., Nature Phys. 13 (2017), 584-590 [arXiv:1705.02290 [hep-ex]].
  * [36] A. Abeln et al. [IAXO], JHEP 05 (2021), 137 [arXiv:2010.12076 [physics.ins-det]].
  * [37] J. Preskill, M. B. Wise and F. Wilczek, Phys. Lett. B 120, 127-132 (1983).
  * [38] L. F. Abbott and P. Sikivie, Phys. Lett. B 120, 133-136 (1983).
  * [39] M. Dine and W. Fischler, Phys. Lett. B 120, 137-141 (1983).
  * [40] S. Borsanyi et al., Nature 539 (2016) no.7627, 69-71 [arXiv:1606.07494 [hep-lat]].
  * [41] Y. K. Semertzidis and S. Youn, Sci. Adv. 8 (2022) abm9928 [arXiv:2104.14831 [hep-ph]].
  * [42] V. B. Klaer and G. D. Moore, JCAP 11 (2017), 049 [arXiv:1708.07521 [hep-ph]].
  * [43] M. Gorghetto et al., SciPost Phys. 10 (2021) no.2, 050 [arXiv:2007.04990 [hep-ph]].
  * [44] M. Buschmann et al., Nature Commun. 13 (2022) no.1, 1049 [arXiv:2108.05368 [hep-ph]].
  * [45] K. Saikawa, J. Redondo, A. Vaquero and M. Kaltschmidt, [arXiv:2401.17253 [hep-ph]].
  * [46] A. Ringwald and K. Saikawa, Phys. Rev. D 93 (2016) 085031 [arXiv:1512.06436 [hep-ph]].
  * [47] K. A. Beyer and S. Sarkar, SciPost Phys. 15 (2023) no.1, 003 [arXiv:2211.14635 [hep-ph]].
  * [48] T. Braine et al. [ADMX], Phys. Rev. Lett. 124 (2020) 101303 [arXiv:1910.08638 [hep-ex]].
  * [49] A. K. Yi et al., Phys. Rev. Lett. 130 (2023) 071002 [arXiv:2210.10961 [hep-ex]].
  * [50] I. Stern, PoS ICHEP2016 (2016), 198 [arXiv:1612.08296 [physics.ins-det]].
  * [51] S. Ahyoune et al., [arXiv:2306.17243 [physics.ins-det]].
  * [52] D. Alesini et al., Phys. Dark Univ. 42 (2023), 101370 [arXiv:2309.00351 [physics.ins-det]].
  * [53] M. E. Tobar et al., Annalen Phys. 2023, 2200594 [arXiv:2211.09637 [hep-ph]].
  * [54] D. Horns et al., JCAP 04 (2013), 016 [arXiv:1212.2970 [hep-ph]].
  * [55] F. Bajjali et al., JCAP 08 (2023), 077 [arXiv:2306.05934 [hep-ex]].
  * [56] J. Liu et al., Phys. Rev. Lett. 128 (2022) no.13, 131801 [arXiv:2111.12103 [physics.ins-det]].
  * [57] A. Caldwell et al., Phys. Rev. Lett. 118 (2017) 091801 [arXiv:1611.05865 [physics.ins-det]].
  * [58] P. Brun et al., Eur. Phys. J. C 79 (2019) no.3, 186 [arXiv:1901.07401 [physics.ins-det]].
  * [59] S. Beurthey et al., [arXiv:2003.10894 [physics.ins-det]].
  * [60] M. Lawson et al., Phys. Rev. Lett. 123 (2019) 141802 [arXiv:1904.11872 [hep-ph]].
  * [61] P. Sikivie et al., Phys. Rev. Lett. 112 (2014) 131301 [arXiv:1310.8545 [hep-ph]].
  * [62] J. L. Ouellet et al., Phys. Rev. Lett. 122 (2019) 121802 [arXiv:1810.12257 [hep-ex]].
  * [63] C. P. Salemi et al., Phys. Rev. Lett. 127 (2021) 081801 [arXiv:2102.06722 [hep-ex]].
  * [64] N. Crisosto et al., Phys. Rev. Lett. 124 (2020) 241101 [arXiv:1911.05772 [astro-ph.CO]].
  * [65] A. V. Gramolin et al., Nature Phys. 17 (2021) 79-84 [arXiv:2003.03348 [hep-ex]].
  * [66] Z. Zhang et al., Phys. Rev. D 106 (2022) 023003 [arXiv:2111.04541 [hep-ex]].
  * [67] L. Brouwer et al. [DMRadio], Phys. Rev. D 106 (2022) 103008 [arXiv:2204.13781 [hep-ex]].
  * [68] T. Li, R. J. Zhang and C. J. Dai, JHEP 03 (2023), 088 [arXiv:2211.06847 [hep-ph]].
  * [69] M. E. Tobar et al., Phys. Rev. D 108 (2023) 035024 [arXiv:2306.13320 [hep-ph]].
  * [70] P. W. Graham and S. Rajendran, Phys. Rev. D 88 (2013), 035023 [arXiv:1306.6088 [hep-ph]].
  * [71] D. Budker et al., Phys. Rev. X 4 (2014) no.2, 021030 [arXiv:1306.6089 [hep-ph]].
  * [72] D. Aybas et al., Phys. Rev. Lett. 126 (2021) no.14, 141802 [arXiv:2101.01241 [hep-ex]].
  * [73] A. Ernst, A. Ringwald and C. Tamarit, JHEP 02 (2018), 103 [arXiv:1801.04906 [hep-ph]].
  * [74] L. Di Luzio et al., Phys. Rev. D 98 (2018) 095011 [arXiv:1807.09769 [hep-ph]].
  * [75] P. Fileviez Pérez et al., JHEP 11 (2019), 093 [arXiv:1908.01772 [hep-ph]].