# Improvising the Learning of Neural Networks on Hyperspherical Manifold

Lalith Bharadwaj Baru
VNRVJIET
Hyderabad-90, T.S, India.
<EMAIL_ADDRESS>
&Sai Vardhan Kanumolu∗
VNRVJIET
Hyderabad-90, T.S, India.
<EMAIL_ADDRESS>
&Akshay Patel Shilhora∗
VNRVJIET
Hyderabad-90, T.S, India.
<EMAIL_ADDRESS>
Madhu G
VNRVJIET
Hyderabad-90, T.S, India.
<EMAIL_ADDRESS>
The authors contributed equally

###### Abstract

The impact of convolution neural networks (CNNs) in the supervised settings
provided tremendous increment in performance. The representations learned from
CNN’s operated on hyperspherical manifold led to insightful outcomes in face
recognition, face identification, and other supervised tasks. A broad range of
activation functions were developed with hypersphere intuition which performs
superior to softmax in euclidean space. The main motive of this research is to
provide insights. First, the stereographic projection is implied to transform
data from Euclidean space ($\mathbb{R}^{n}$) to hyperspherical manifold
($\mathbb{S}^{n}$) to analyze the performance of angular margin losses.
Secondly, proving theoretically and practically that decision boundaries
constructed on hypersphere using stereographic projection obliges the learning
of neural networks. Experiments have demonstrated that applying stereographic
projection on existing state-of-the-art angular margin objective functions
improved performance for standard image classification data sets
(CIFAR-10,100). Further, we ran our experiments on malaria-thin blood smear
images, resulting in effective outcomes. The code is publicly available at: :
https://github.com/barulalithb/stereo-angular-margin.

##  1 Introduction

The impact of deep learning has provided a notable surge in various domains
such as computer vision, language processing, speech processing, and graph
mining [10]. A wide range of neural networks is developed by captivating their
performance in deep learning. Specifically, convolution neural networks (CNNs)
have shown their impressive ability to extract invariances in many problems.
In computer vision, the CNN’s were implied to extract features that are class
specific, hierarchical, and other complex invariances [22]. Hence, these CNNs
are used as encoders which map high dimensional representations to lower
dimensions (feature sets).

Neural networks which were devised on hyperspherical manifold provided
significant performance compared to standard CNNs [13] [14]. A broad range of
objective functions was devised for solving face verification, face
recognition, and similar supervised tasks with hyperspherical intuition. Also,
this intuition is carried by the recent advances in self-supervision by
learning features contrastively. Further, stereographic projection (SP) on
hyperspheres is utilized as a pre-processing technique to understand
continuous rotational invariances.

It is observed that erstwhile research (which specifically operates on
hyperspherical manifold) assumes that their objective functions are operated
on the hypersphere. But in this work, we utilize a geometric transformation to
map the data points from the euclidean to hyperspherical decision region.
Further, we understand the behavior of angular margin function and standard
categorical cross-entropy (CCE) operated on this manifold.

###  1.1 Impact of angular margin losses

In recent years various losses have been stated for specific objectives.
Rather than learning separable features, the discriminative approach
encourages us to learn features selectively, thus increasing compactness in
intra-class and separability in inter-class features. In discriminative
learning, research has shown that cosine-margin for learned features has
significant improvement than euclidean- margin. Many angular-margin losses
have been implemented, specifically for the face recognition task, based on
the analogy that the learned features from a neural network interpret the
geometry as a hyperspherical manifold. Where a margin can control the decision
boundary. [12] has developed the motive of introducing angular constraints to
decision functions. Further, [11] has shown normalizing the weight vectors
will advance the [12] with no requirement of joint-supervision by a
straightforward implementation [18] [6] has improved the performance of
classifiers by introducing additive-margin loss function, thus tuning the
cosine space and angle space with the margin by rescaling their logits with a
fixed norm. [9] has overcome the problem of mini-batch learning by adding
queues to their framework containing embedding vectors learned from the neural
network with their corresponding identity-representative vector and these
vectors are updated throughout training. Although [9] has utilized the loss
function presented in [6], it adopted a new training procedure which has
proven significant improvement in the face recognition dataset.

###  1.2 Impact on contrastive representation learning

Recently, contrastive representation learning has advanced in deep
understanding to learn invariances both in supervised [8] and self-supervised
setting [4]. [19] produced two crucial factors in designing a contrastive
objective function: a) uniformity and b) alignment. These factors played a
vital role in developing a resilient contrastive objective function. While
learning representations contrastively, [8] developed a contrastive loss which
was superior to that of self-supervised method [4]. Further, [8] performed
unit sphere normalization to feature sets drawn from an encoder and resulted
in outstanding performance. Both the works focused on training dynamics on
unit hypersphere by designing a contrastive loss.

###  1.3 Stereographic projections

Closely related work, implementing stereographic projection, was done by [16],
[15],and [24]. First, [16] utilized stereographic projection as a pre-
processing technique to classify 4-bit parity. The projection transforms real-
valued patterns from one space to another by increasing the dimensionality and
providing superior performance on the two-spiral problem. Where [15], proposed
SphereGAN for image generation problem by utilizing stereographic projection
onto hypersphere to attain state-of-the-art results. [24] tackled with
continuity of rotation representations for better training dynamics of neural
networks. The insights of stereographic projections are implied while
understanding the n-dimensional groups with rotations. [21] have utilized
stereographic projection as a pre-processing technique to transform 3D objects
to 2D planar images.

The existing research, as mentioned, primarily focused on developing objective
functions which assume that data points lie in the hypersphere. Hence, an
appropriate map is not provided to shift the space and learn invariances from
neural networks. Accordingly, we solve this problem by providing a detailed
solution.

##  2 Uniqueness of Proposed Work

The uniqueness of our research is twofold.

  1. 1.

First, we imply stereographic projection to transform feature vectors from
euclidean space ($R^{n}$) to hyperspherical manifold ($S^{n}$) to analyze the
performance of angular margin losses.

  2. 2.

Second, proving theoretically and practically that decision boundaries
constructed on hypersphere using stereographic projection oblige the learning
of neural networks.

##  3 Stereographic projection on hyperspherical manifold

#### Definition

Stereographic projection is said to be a map
$\varphi:\mathbb{R}^{n}=\\{x\in\mathbb{R}^{n+1}:x_{n+1}=0\\}\to\mathbb{S}^{n}-\\{e_{n+1}\\}$
of $x$ $((n+1)^{th}$dimensional tuple$)$ in euclidean space onto a hypersphere
$(\mathbb{S}^{n})$ such that, the x meets a point on hypersphere by linearly
extending it till $e_{n+1}$.

Let $e_{i}$ be a vector in $\mathbb{R}^{n}$ where, $i^{th}$ coordinate is
unitary and else are null. Then $\\{e_{1},e_{2},...,e_{n}\\}$ is orthonormal
basis of $\mathbb{R}^{n}$ and also called as standard basis in
$\mathbb{R}^{n}$. So, $e_{n+1}$ is $(n+1)^{th}$ standard basis in
$\mathbb{R}^{n+1}$. So, $\varphi(x)$ is the stereographic projection of $x$,
in euclidean manifold, onto the hyperspherical manifold. This $\varphi(x)$ lie
on the line passing through x and is directed towards $e_{n+1}-x$. Hence,
$\varphi(x)$ can be written as,

$\varphi(x):=x+z*(e_{n+1}-x)$ (1)

A visual illustration of stereographic projection from
$\mathbb{R}^{1}\to\mathbb{S}^{1}$ and $\mathbb{R}^{2}\to\mathbb{S}^{2}$ is
provided for better intuitiveness in the Fig. 1. Further, there are two
underlying key points to be noted while applying stereographic projection are,

  * •

The euclidean space ($\mathbb{R}^{n}$) must contain the origin.

  * •

The radius of the hypersphere is unity.

Figure 1: Stereographic projections of a point on the real line and a plane
are projected onto a circle and a sphere.

As, $\varphi(x)$ passes through the unit sphere,

${\mid\varphi(x)\mid}^{2}:=1$ (2)

Substituting equation (1) in (2) results,

$z:=\frac{{\mid x\mid}^{2}-1}{{\mid x\mid}^{2}+1}$ (3)

Hence, the coordinates obtained after the stereographic projection $\varphi$
of $x$ ($x:(x_{1},x_{2},...,x_{n+1})$) onto hypersphere are,

$\varphi(x):=\bigg{(}\frac{2x_{1}}{{\mid x\mid}^{2}+1},\frac{2x_{2}}{{\mid
x\mid}^{2}+1},...,\frac{{\mid x\mid}^{2}-1}{{\mid x\mid}^{2}+1}\bigg{)}$ (4)

Algorithm 1 PyTorch-like pseudo code for stereographic projection

i/p: input; o/p: output; dim: dimensional ; concat: concatenation; sum:
summation; pow(.,2): Squaring the input

⬇

# This function considers (n)-dim i/p vector and returns (n+1)-dim o/p vector.

# in_vec: input vector

# norm: norm of in_vec vector

# a: first n dimensions of the projection

# b: final dimension of the projection

# out: stereographic projection

def streographic_preojection (in_vec):

norm = sum(pow(in_vec,2))

a = (2*in_vec)/(norm+1)

b = (norm-1)/(norm+1)

out = concat(a,b)

return out

##  4 Hypersphere as decision region

This section theoretically proves that the hypersphere can be used as a
decision region in feed-forward neural networks. To obtain the transformation
from euclidean space ($\mathbb{R}^{n}$) to hyperspherical manifold
($\mathbb{S}^{n}$), we imply the earlier described concept stereographic
projection.

######  Theorem 1.

A unit hyperspherical manifold $\mathbb{S}^{n}$ is always convex and
connected.

###### Proof.

First the convexity and next the connectedness are proven.

#### Convexity

The proof for the convexity of $\mathbb{S}^{n}$ is quite simple [5].

Suppose, $x,y\in\mathbb{S}^{n}$ and $\alpha\in[0,1]$.

Using triangle inequality [1],

$\mid\mid\alpha x+(1-\alpha)y\mid\mid\>\>\leq\>\>\mid\mid\alpha
x\mid\mid+\mid\mid(1-\alpha)y\mid\mid=\alpha+(1-\alpha)=1$

Hence, $\alpha x+(1-\alpha)y\in\mathbb{S}^{n}$ and thus $\mathbb{S}^{n}$ is
convex.

#### Connectedness

To prove this, $\mathbb{S}^{n}$ is equated as union of closed upper hemisphere
($\mathbb{S}_{+}^{n}$) and lower hemisphere ($\mathbb{S}_{-}^{n}$). Individual
hemispheres, $\mathbb{S}_{+}^{n}$ and $\mathbb{S}_{-}^{n}$ are homeomorphic to
closed unit disk ($\mathbb{D}^{n}$) in $\mathbb{R}^{n}$ and intersect
mutually.

First, let us consider, closed upper hemisphere
$\mathbb{S}_{+}^{n}:=\\{u\in\mathbb{R}^{n+1}:u_{n+1}\geq 0\\}$ and the unit
disk $\mathbb{D}^{n}:=\\{v\in\mathbb{R}^{n}:\>\>\mid\mid v\mid\mid\leq 1\\}$.
We claim that, $\mathbb{D}^{n}$ shows homeomorphism with $\mathbb{S}_{+}^{n}$.
The map $f_{+}:\mathbb{D}^{n}\to\mathbb{S}_{+}^{n}$ is clearly bijective and
continuous. Where,

$f_{+}(v):=\left(v_{1},v_{2},...,v_{n},\sqrt{1-\mid\mid v\mid\mid^{2}}\right)$

As, $f_{+}$ is a continuous map from a compact topological space to a definite
topological space with homeomorphism. So, in any of the case,
$\mathbb{S}_{+}^{n}$ is connected.

Similarly, it is concluded that, the closed lower hemisphere
$\mathbb{S}_{-}^{n}:=\\{u\in\mathbb{R}^{n+1}:u_{n+1}\leq 0\\}$ is an image of
$f_{-}:\mathbb{D}^{n}\to\mathbb{S}_{-}^{n}$ given by,

$f_{-}(v):=\left(v_{1},v_{2},...,v_{n},-\sqrt{1-\mid\mid
v\mid\mid^{2}}\right)$

The observation is clear that the intersection
$\mathbb{S}_{+}^{n}\cap\mathbb{S}_{-}^{n}:=\\{u\in\mathbb{R}^{n+1}:u_{n+1}=0\\}$
is a non-empty set. Hence, from theorem. 5 (Appendix. Appendix), we can
conclude that $\mathbb{S}^{n}$ is connected.

∎

#### Proposition 1.

A two-layered neural networks decision region possessing the property a)
convex and b) connected aids in decision making by assigning decision
boundaries [20]. As unit hypersphere possess these two properties, and it can
be used as a decision region.

###### Proof.

Theorem 1 gives sufficient justification for the above proposition 1. ∎

Thus, $\varphi(x)$ transformation applied at the final layer of the neural
networks shifts the decision regions in Euclidean space to the hyperspherical
manifold. As a note, erstwhile research assumed that data points lie in
hyperspherical manifold and devised objective functions. This work removed
this assumption and transformed the data onto a hyperspherical manifold. For a
better intuition, PyTorch-like pseudo-code is explained in Algorithm 1.

Figure 2: The above figure illustrates the t-sne embedding of the final
decision region (last layer of neural network). The parameters for t-sne
initialized are mentioned in detail. Learning rate $\leftarrow 200$;
perplexity $\leftarrow 50$; iterations $\leftarrow 1000$; angle $\leftarrow
6\times 10^{-1}$; initialized with PCA; metric as euclidean; method for
computation is chosen as barrens hut. The number of samples considered to
visualize both train and test is $10^{4}$. In the above illustration,
considering The train case without projection confirms a tiny cluster
including every (nearly) class, and in a test scenario, they tend to group
heavily. Whereas considering the case of applying stereographic projection,
there are fewer samples forming clusters during training, and there is less
affinity to form a cluster in the testing phase.

##  5 Experimentation

In this section, a brief introduction to all the angular margin loss functions
is provided, as these loss functions have an underlying assumption that the
data points lie in the hypersphere. But, with stereographic projection, the
data points shift their space. So, we analyze that whether stereographic
projections in practicality drive incredible performance for angular margin
losses. Subsequently, the hyperparameters constructed while training the
neural network and the training paradigm are discussed in detail.

###  5.1 Angular Margin Objectives

First, SphereFace has presented angular-margin Softmax loss(A-Softmax loss),
which defines a decision boundary with a controlled margin. The main task of
this loss function is to increase the posterior probability for the true label
by implying the multiplicative angular-margin ${m}$ between the learned
feature ${x_{i}}$ and the weight vector ${W_{i}}$ of the true label ${y_{i}}$.
As the margin increases for a more precise boundary, the constrained region
becomes smaller, and the learning task becomes more problematic. In the below
equation weight vector is normalized using ${L_{2}}$ normalization. For
general understanding, we replace ${cosine(.)}$ with ${\omega(.)}$ for
feasibility and the original equation is presented in [11].

$L_{SphereFace}=\frac{1}{N}\sum_{i^{N}}-\log\left(\frac{e^{||x_{i}||(\omega({m\theta_{y_{i},i})})}}{e^{||x_{i}||(\omega(m\theta_{y_{i},i}))}+\sum_{j\neq{y_{i}}}e^{||x_{i}||(\omega(\theta_{j,i}))}}\right)$
(5)

CosFace [18] has introduced additive cosine margin as a decision margin. They
have normalized the weight vector ${W_{i}}$ and the feature vector ${x_{i}}$
with ${L_{2}}$ normalization and additionally added a scaling factor ${s}$
which is the norm of the vector ${x}$ that is, ${s}={||x||}$. Instead of
tuning the angle space, it aims to build a smooth decision boundary and gives
reinforcement to the discriminative learning by tweaking the cosine space with
the help of extra margin ${m}$. For general understanding,

Table 1: The performance analysis of stereographic projection with ResNet as encoder for CIFAR-10 and 100 Data sets. The experiments were implemented five times for variant seeds (fixed batch size). Where the values are in the mean value, and the deviation is obtained. Data Augmentation is not implied while training any of the models. Datasets | Loss Methods | Accuracy(%)
---|---|---
Projection applied
Yes | NO
CIFAR-100 | CCE | 55.46$\pm$0.23 | 55.65$\pm$0.20
SphereFace | 59.68$\pm$0.15 | 57.25$\pm$0.34
CosFace | 60.71$\pm$0.40 | 57.84$\pm$0.41
ArcFace | 59.65$\pm$0.57 | 58.48$\pm$0.42
BroadFace | 61.23$\pm$0.28 | 60.29$\pm$0.47
CIFAR-10 | CCE | 80.92$\pm$0.41 | 81.34$\pm$0.15
SphereFace | 83.05$\pm$0.33 | 80.78$\pm$0.43
CosFace | 84.31$\pm$0.32 | 81.72$\pm$0.41
ArcFace | 84.47$\pm$0.34 | 81.83 $\pm$0.52
BroadFace | 85.59$\pm$0.24 | 81.95$\pm$0.53

$L_{CosFace}=\frac{1}{N}\ \sum_{i}^{N}\
-\log\left(\frac{e^{s(\omega{(\theta_{y_{i},i})-m)}}}{{e^{s(\omega{(\theta_{y_{i},i})-m})}+\sum_{j\neq{y_{i}}}e^{s({\omega{(\theta_{j,i}))}}}}}\right)$
(6)

ArcFace [6] has introduced additive angular-margin, which has prominent
improvement over multiplicative angular-margin and additive cosine-margin to
obtain a precise decision boundary. This margin is equal to the geodesic
distance of the normalized hypersphere [6] also realized the importance of
normalization of weight vector ${W_{i}}$ and the feature vector ${x_{i}}$ and
added a scaling factor similar to [18]. Below is the equation of arcface and
for better understanding, we replace ${\cos(.)}$ with ${\omega(.)}$ in the
original equation presented in the equations.

$L_{ArcFace}=\frac{1}{N}\ \sum_{i}^{N}\
-\log\left(\frac{e^{s(\omega{(\theta_{y_{i},i}+m)})}}{{e^{s(\omega{(\theta_{y_{i},i}+m}))}+\sum_{{j=1,j\neq{y_{i}}}}^{n}e^{s({\omega{(\theta_{j,i}))}}}}}\right)$
(7)

BroadFace [9] has adopted the loss function presented in [6] but with an
intuitive training procedure, it has achieved a notable improvement in the
face recognition task. For a given ${[b_{i}]}$ it supposedly stores the
feature embedding vectors in a queue along with their identity representative
vectors ${{W_{y}}_{i}}$. Utilizing this past feature embedding vectors present
in the queue, it computes loss for the ${{W_{y}}_{i}}$. So the neural network
has combined loss from the classifier and the embeddings. Embedding loss is
compensated to mitigate the risk from past embedding vectors arisen from
${b^{-}}$. To reduce the additional error, a compensation function is
implemented. Below is the equation for BroadFace, where ${X}$ is the sample,
${E}$ is embedding queue, ${b_{i}}$ is the embedding vector, and ${b_{j}^{*}}$
compensated embedding vector.

$L_{BroadFace}=\frac{1}{X\cup E}\left(\sum_{i\in X}l(b_{i})+\sum_{j\in
E}l(b_{j}^{*})\right)$ (8)

###  5.2 Training and Experimentation on CIFAR-10,100

In representation learning, the classification is achieved by extracting
representation from an encoder and performing non-linear mapping with feed-
forward neurons. Finally, using a logit function to classify the patterns. So,
we inject stereographic projection as a transformation function to project
latent feature representations onto the hypersphere. This mapping is applied
just before the classification.

During the evaluation, we considered ResNet as our standard baseline encoder
[7]. Considering all the layers would be redundant. Hence architecture pruning
is applied [3], [23]. The latent feature vector, $f_{v}$ obtained by passing
through the ResNet encoder and pruning the network by skipping two pooling
layers. Hence, a feature vector $f_{v}$ of shape $4\times 4\times 512$ is
obtained for standard input shape of $32\times 32\times 3$ (CIFAR-10,100
data). These feature sets were fed into a two-layered, fully connected dense
network with 512, 256 neurons, respectively. The intermediate activations are
typically activated with ReLU. Where $\varphi$ is stereographic projection and
utilizing it with angular margin loss functions at the final layer acts as
transformation and eventually acts as a decision boundary. To understand the
representations, t-sne [17] visualizations are provided for the best
performing model (BroadFace) for the CIFAR-10 model with and without
projection in Figure 2. The results obtained in Table 1 depict the performance
of stereographic projection on angular margin. The results are obtained
without any augmentation as our aim was to understand the performance with
standard data. Further, the experimentation is conducted five times with
random seeds, and the mean and standard deviations are reported111It should be
noted that training and testing samples for CIFAR-10 and 100 are $5\times
10^{4}$ and $10^{4}$ respectively. Hence, to plot t-SNE, $10^{4}$ random
training samples and complete testing samples were considered.

Table 2: The performance analysis of stereographic projection with ResNet as encoder for malaria data set. The experiments were implemented five times. Where the values obtained are the mean and the standard deviation of five variant executions. Data Augmentation is not implied while training any of the models. Datasets | Loss Methods | Accuracy(%)
---|---|---
Projection applied
Yes | NO
Malaria Data | CCE | 96.05$\pm$1.32 | 94.91$\pm$2.10
SphereFace | 96.43$\pm$2.71 | 95.48$\pm$1.49
BroadFace | 96.64$\pm$2.48 | 95.86$\pm$2.45

###  5.3 Training and Experimentation on Malaria Dataset

The dataset is acquired from the NIH repository222NIH repository (National
Library of Medicine) for malaria thin blood smears: Link and consist of two
classes: parasitized and uninfected. Each of the classes has 13,780 images of
varying sizes. The complete samples are reshaped into the identical shape of
$128\times 128\times 3$. Some of the samples from the dataset are visually
depicted in Figure 3. For evaluation, the training and testing samples were
partitioned into 70% and 30% of complete data, respectively.

Now, in a similar context, ResNet is chosen as encoder [7] and pruning is
performed very similarly to that of previous. Next, SGD is chosen as optimizer
(with the momentum of $0.92$) with a learning rate of $10^{-4}$ for angular
margin objectives. Whereas for CCE, the learning rate is set to $10^{-3}$. The
batch size is the same as that of the previous, and the evaluation has
proceeded until it reaches convergence. As per the results obtained from Table
1, we decided that CosFace and ArcFace are intermediate models, i.e., their
performance is higher than SphereFace and less than that of BroadFace. Hence,
to reduce redundancy, these executions are not considered for the malaria
dataset. As the data is sufficiently large for both classes, we did not
perform any augmentation. As previously mentioned, our goal is to understand
the performance of the model with pure data. The results obtained in Table 2
prove the performance of stereographic projection on angular margin objectives
and CCE.

Figure 3: Sample images of thin blood smear images infected with and without
malaria.

##  6 Disadvantages

#### Additional Tuning

The hyperparameters in angular margin objective functions such as scale factor
and margins are sophisticated to tune. For this, a consistent effort is
required to adjust these additional hyperparameters on variant data sets.

#### Slower Training

During experimentation, we observed that execution time is slightly higher for
the model with stereographic projection than that of the model without
projection. This eventually consumes more electricity, and it is hazardous to
the environment.

##  7 Conclusion

In this research, we justify theoretically and practically that stereographic
projection integrating with angular margin objective functions for image
classification problems improves the decision-making in neural networks.
Instead of assuming the space, it is required to shift the space from
euclidean to hypersphere to provide better performance for standard image
classification data as well as malaria thin blood smear images. In the future,
we aim to provide a versatile model that can enhance variant biomedical tasks’
performance.

## References

  * [1] Stephen Boyd, Stephen P Boyd, and Lieven Vandenberghe.  Convex optimization.  Cambridge university press, 2004.
  * [2] Victor Bryant.  Metric spaces: iteration and application.  Cambridge University Press, 1985.
  * [3] Shi Chen and Qi Zhao.  Shallowing deep networks: Layer-wise pruning based on feature representations.  IEEE transactions on pattern analysis and machine intelligence, 41(12):3048–3056, 2018.
  * [4] Ting Chen, Simon Kornblith, Mohammad Norouzi, and Geoffrey Hinton.  A simple framework for contrastive learning of visual representations.  In International conference on machine learning, pages 1597–1607. PMLR, 2020.
  * [5] Geir Dahl.  An introduction to convexity.  University of Oslo, Centre of Mathematics for Applications, Oslo, Norway, 2010.
  * [6] Jiankang Deng, Jia Guo, Niannan Xue, and Stefanos Zafeiriou.  Arcface: Additive angular margin loss for deep face recognition.  In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 4690–4699, 2019.
  * [7] Kaiming He, Xiangyu Zhang, Shaoqing Ren, and Jian Sun.  Deep residual learning for image recognition.  In Proceedings of the IEEE conference on computer vision and pattern recognition, pages 770–778, 2016.
  * [8] Prannay Khosla, Piotr Teterwak, Chen Wang, Aaron Sarna, Yonglong Tian, Phillip Isola, Aaron Maschinot, Ce Liu, and Dilip Krishnan.  Supervised contrastive learning.  In H. Larochelle, M. Ranzato, R. Hadsell, M. F. Balcan, and H. Lin, editors, Advances in Neural Information Processing Systems, volume 33, pages 18661–18673. Curran Associates, Inc., 2020.
  * [9] Yonghyun Kim, Wonpyo Park, and Jongju Shin.  Broadface: Looking at tens of thousands of people at once for face recognition.  In European Conference on Computer Vision, pages 536–552. Springer, 2020.
  * [10] Yann LeCun, Yoshua Bengio, and Geoffrey Hinton.  Deep learning.  nature, 521(7553):436–444, 2015.
  * [11] Weiyang Liu, Yandong Wen, Zhiding Yu, Ming Li, Bhiksha Raj, and Le Song.  Sphereface: Deep hypersphere embedding for face recognition.  In Proceedings of the IEEE conference on computer vision and pattern recognition, pages 212–220, 2017.
  * [12] Weiyang Liu, Yandong Wen, Zhiding Yu, and Meng Yang.  Large-margin softmax loss for convolutional neural networks.  In ICML, volume 2, page 7, 2016.
  * [13] Weiyang Liu, Yan-Ming Zhang, Xingguo Li, Zhiding Yu, Bo Dai, Tuo Zhao, and Le Song.  Deep hyperspherical learning.  In I. Guyon, U. V. Luxburg, S. Bengio, H. Wallach, R. Fergus, S. Vishwanathan, and R. Garnett, editors, Advances in Neural Information Processing Systems, volume 30. Curran Associates, Inc., 2017.
  * [14] Pascal Mettes, Elise van der Pol, and Cees Snoek.  Hyperspherical prototype networks.  Advances in Neural Information Processing Systems, 32:1487–1497, 2019.
  * [15] Sung Woo Park and Junseok Kwon.  Sphere generative adversarial network based on geometric moment matching.  In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 4292–4301, 2019.
  * [16] J Saffery and C Thornton.  Using stereographic projection as a preprocessing technique for upstart.  In IJCNN-91-Seattle International Joint Conference on Neural Networks, volume 2, pages 441–446. IEEE, 1991.
  * [17] Laurens van der Maaten and Geoffrey Hinton.  Visualizing data using t-sne.  Journal of Machine Learning Research, 9(86):2579–2605, 2008.
  * [18] Hao Wang, Yitong Wang, Zheng Zhou, Xing Ji, Dihong Gong, Jingchao Zhou, Zhifeng Li, and Wei Liu.  Cosface: Large margin cosine loss for deep face recognition.  In Proceedings of the IEEE conference on computer vision and pattern recognition, pages 5265–5274, 2018.
  * [19] Tongzhou Wang and Phillip Isola.  Understanding contrastive representation learning through alignment and uniformity on the hypersphere.  In International Conference on Machine Learning, pages 9929–9939. PMLR, 2020.
  * [20] Alexis Wieland and Russell Leighton.  Geometric analysis of neural network capabilities.  1987\.
  * [21] Mohsen Yavartanoo, Eu Young Kim, and Kyoung Mu Lee.  Spnet: Deep 3d object classification and retrieval using stereographic projection.  In Asian conference on computer vision, pages 691–706. Springer, 2018.
  * [22] Matthew D Zeiler and Rob Fergus.  Visualizing and understanding convolutional networks.  In European conference on computer vision, pages 818–833. Springer, 2014.
  * [23] Chenglong Zhao, Bingbing Ni, Jian Zhang, Qiwei Zhao, Wenjun Zhang, and Qi Tian.  Variational convolutional neural network pruning.  In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 2780–2789, 2019.
  * [24] Yi Zhou, Connelly Barnes, Jingwan Lu, Jimei Yang, and Hao Li.  On the continuity of rotation representations in neural networks.  In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 5745–5753, 2019.

## Appendix

In this section, we would like to introduce some basic theorems related to
connectedness of a topological space. These theorems provides an
interpretability of Theorem 1.

######  Theorem 2.

A set $X$ in a topological space $\mathcal{T}$ is disconnected if $X$ can be
written as intersection of two disjoint, open and non-empty sets. In other
cases $X$ is connected.

###### Proof.

The complete proof for this theorem is detailed and elucidated with examples
[2]. ∎

######  Theorem 3.

The topological space $\mathcal{T}$ is connected iff every continuous function
$f:\mathcal{T}\to\\{\pm 1\\}$ is constant.

$(or)$

Consider a set $X$, subset of topological space $\mathcal{T}$, is said to be
connected if and only if every continuous function $f:{X}\to\\{\pm 1\\}$ is
constant.

###### Proof.

Assume, $\mathcal{T}$ is not connected.

Therefore, $\exists\>\>X,Y\subset\mathcal{T}$ are two disjoint and proper non-
empty subsets in $\mathcal{T}$. These subsets in are both open and close in
$\mathcal{T}$ and $\mathcal{T}=X\cup Y$.

So, $f:\mathcal{T}\to\\{\pm 1\\}$ can be written as,

$\displaystyle f(u)=\left\\{\begin{array}[]{lll}+1&,if&u\in X\\\ -1&,if&u\in
Y\end{array}\right.$

Then, $f:\mathcal{T}\to\\{\pm 1\\}$ is a continuous non-constant function.
Hence, conversely, the theorem is proved. ∎

######  Theorem 4.

The topological space $\mathcal{T}$ has two connected subsets $X$, $Y$ such
that, $X\cap Y\neq\phi$. Then, $X\cup Y$ is connected,

###### Proof.

Consider, $z\in X\cap Y$ and $f:X\cap Y\to\\{\pm 1\\}$ be a continuous
function.

$\because X$ is connected, $f$ is constant on $X$.

So, $f(x)=f(z)\>\forall\>x\in X$.

Similarly, $f$ is constant on $Y$.

So, $f(y)=f(z)\>\forall\>y\in Y$.

From theorem. 4, it can be said that, any continuous function from $X\cup Y$
to $\\{\pm 1\\}$ is constant. Hence, $X\cup Y$ is connected. ∎


# Probing axion-like particles with RF cavities separated by thin barrier

Dmitry Salnikov<EMAIL_ADDRESS>Moscow State University, 119991
Moscow, Russia  Institute for Nuclear Research, 117312 Moscow, Russia  D. V.
Kirpichnikov<EMAIL_ADDRESS>Institute for Nuclear Research, 117312 Moscow,
Russia  Petr Satunin<EMAIL_ADDRESS>Moscow State University, 119991
Moscow, Russia  Institute for Nuclear Research, 117312 Moscow, Russia

###### Abstract

We address the Light-Shining-through thin Wall (LSthinW) laboratory setup to
estimate the axion-like particle (ALP) sensitivity of a two radio frequency
(RF) cavities immersed in static magnetic field. We show that the sufficiently
thin wall between cavities can lead the improved sensitivity of the purely
laboratory probes of ALP in the mass range $10^{-6}~{}\mbox{eV}\lesssim
m_{a}\lesssim 5\times 10^{-5}~{}\mbox{eV}$.

††preprint: INR-TH-2024-006

##  I Introduction.

Feebly-interacting particles (FIPs) Antel  _et al._ (2023) are hypotetical
light particles which extremely weak interact with the Standard Model (SM)
fields and may consist the dark matter component of the Universe or can be a
mediator between Dark Matter and SM sector. The well motivated FIPs are
photophilic: pseudoscalar axion-like-particle (ALP) Svrcek and Witten (2006);
Arvanitaki  _et al._ (2010) and vector dark photon (DP) Okun (1982); Holdom
(1986). Both of them can be tested with their interaction with Standard Model
photons.

A typical strategy for probing FIPs (ALPs and DPs) implies both their
production and detection in a laboratory, and usually called Light-Shining-
through-Wall (LSW) experiments Sikivie (1983); Anselm (1985); Van Bibber  _et
al._ (1987); Hoogeveen (1992); Salnikov  _et al._ (2021); Gao and Harnik
(2021). The LSW setups consist of two cavities separated by a non-transparent
wall for ordinary photons. The FIPs are produced in the first cavity by a
cavity electromagnetic mode (in case of DPs) or by an interaction of two
electromagnetic field components (in case of ALPs).

Generated FIPs can pass through the wall and convert back to photons in the
detection cavity. In the present paper, we focus on the purely laboratory LSW
setup for probing ALPs that consists of two radio frequency (RF) cavities
separated by thin barrier. We also show that regarding expected reach can be
comparable with the the typical LSW laser-based experimental limits Ehret  _et
al._ (2010); Ballou  _et al._ (2015). Typically, LSW RF cavity experiments,
such as CROWS Betz  _et al._ (2013), DarkSRF Romanenko  _et al._ (2023), imply
that the FIPs emitter and receiver cavities are divided by a distance that is
comparable to the typical size of the detector. This setup is mostly sensitive
to FIPs mass less than the emitter mode frequency since the heavier FIPs are
virtual and their density exponentially decrease with the distance between
cavities.

Recently the authors of Berlin  _et al._ (2023) propose an idea that a
sufficiently thin wall between cavities significantly improves the DP
sensitivity of in the region of the sufficiently large DP mass, such that it
can be considered as off-shell state (see also Salnikov  _et al._ (2024) for
another cavity configuration). In this paper, we address the thin wall idea
for probing ALPs in LSW experiments. In particular, we argue that employing
the sufficiently low driven emitter frequency $\omega\lesssim
1~{}\,\mbox{GHz}$ and thin barrier between cavities $d\ll 1/\omega$, one can
achieve a large density of the ALP field. As a result this leads to the
improved sensitivity in the off-shell ALP mas region $m_{a}\gtrsim\omega$, if
one compares it with the LSW purely laboratory CROWS Betz  _et al._ (2013)
experiment. We also show that the expected reach of suggested LSthinW setup is
complementary to the laser-based laboratory limits of ALPS-I Ehret  _et al._
(2010) and OSQAR Ballou  _et al._ (2015) experimental facilities.

This paper is organized as follows. In Sec. II we discuss the ALP
electrodynamics. In Sec. III we derive main formulas in order to estimate the
sensitivity of the LSthinW setup to probe ALPs. In Sec. IV we discuss the
numerical results for the expected sensitivity.

##  II The ALP electrodynamics

The Lagrangian describing coupling of ALPs and Standard model photons reads

$\mathcal{L}\supset-\frac{1}{4}F_{\mu\nu}F^{\mu\nu}+\frac{1}{2}\,\partial_{\mu}a\,\partial^{\mu}a-\frac{1}{2}\,m_{a}^{2}a^{2}-\frac{1}{4}\,g_{a\gamma\gamma}\,a\,F_{\mu\nu}\tilde{F}^{\mu\nu}\;,$
(1)

where $F_{\mu\nu}$ is the electromagnetic tensor and
$\tilde{F}^{\mu\nu}=\frac{1}{2}\epsilon^{\mu\nu\alpha\beta}F_{\alpha\beta}$ is
its dual, $a$ denotes the ALP field, and $g_{a\gamma\gamma}$ is dimensional
ALP-photon coupling.

The Lagrangian (1) implies that the equations for the ALP field and SM photon
in vacuum read respectively,

$\displaystyle(\partial_{\mu}\partial^{\mu}+m_{a}^{2})\,a=g_{a\gamma\gamma}(\vec{E}\cdot\vec{B})\;,$
(2)
$\displaystyle(\vec{\nabla}\cdot\vec{E})=\rho_{a}\;,\quad\quad[\vec{\nabla}\times\vec{B}]=\dot{\vec{E}}+\vec{j}_{a}\;,$
(3)

where the density of charge $\rho_{a}$ and current $\vec{j}_{a}$, induced by
the ALP, read respectively as follows

$\rho_{a}=-g_{a\gamma\gamma}(\vec{\nabla}a\cdot\vec{B})\;,\quad\vec{j}_{a}=g_{a\gamma\gamma}([\vec{\nabla}a\times\vec{E}]+\dot{a}\vec{B})\;.$
(4)

We note that Eq. (2) describes the ALP production by the typical combination
of electromagnetic fields $\propto(\vec{E}\cdot\vec{B})$ in its right hand
side. Contrary, both Eqs. (3) and (4) can be associated with the ALP
conversion in the strong magnetic field that results in the generation of a
resonantly enhanced signal EM mode in the detector.

Figure 1: Left panel: The typical scheme of LSthinW setup. Right panel: The
setup sensitivity to ALP parameter space for TM010 mode. The blue, green, and
red lines correspond to the barrier thickness of $d=10~{}\mbox{cm}$,
$d=1~{}\mbox{cm}$, and $d=1~{}\mbox{mm}$ respectively, the radius and length
of the cavities are chosen to be $R=0.8~{}\mbox{m}$ and $L=1~{}\mbox{m}$,
respectively. The purely laboratory bounds of CROWS Betz  _et al._ (2013),
OSQAR Ballou  _et al._ (2015), and ALPS-I Ehret  _et al._ (2010) are shown by
gray, magenta, and brown shaded region, respectively. The extraterrestrial
helioscope limit of CAST Anastassopoulos  _et al._ (2017) is shown by a solid
black line.

##  III Light-Shining-through thin Wall setup

In this section we discuss the following benchmark experimental setup
consisting of two RF cavities. We consider two equal cylindrical cavities of a
radius $R$ and length $L$, located coaxial end-by-end and separated by the
thin barrier of a length $d\ll L$, see Fig. 1, left panel. Both cavities are
immersed in the static magnetic field $B_{\rm ext}=3~{}\mbox{T}$ along the
cavities axis.

### The ALP emission:

the emitter cavity is pumped by TM010 electromagnetic mode
$\vec{E}_{{\text{em}}}(\vec{x})=\vec{e}_{z}E^{\text{em}}_{0}\mathcal{E}^{\text{em}}_{z}(\vec{x})e^{-i\omega
t}$, where $\omega=x_{01}/R$ is a driven frequency of the TM010 mode, $x_{01}$
being a first zero of the Bessel function $J_{0}(x_{01})=0$,
$E_{0}^{\text{em}}=3~{}\mbox{MV}/\mbox{m}=0.01~{}\mbox{T}$ is a magnitude of
the driven emitter electric field,
$\mathcal{E}^{\text{em}}_{z}(\vec{x})=\alpha J_{0}(\omega\rho)$ is a EM cavity
eigen-mode that is normalized as

$\int\limits_{V_{\text{em}}}d^{3}\vec{x}|\mathcal{E}^{\text{em}}_{z}(\vec{x})|^{2}=V_{\text{em}},$

that implies the normalization factor to be $\alpha=1/|J_{1}(\omega R)|$.

The Eq. (2) for produced ALP field implies the following solution,

$\displaystyle a(t,\vec{x})=g_{a\gamma\gamma}\,E^{\rm em}_{0}B_{\rm
ext}\\!\\!\\!\int\limits_{V_{\text{em}}}\\!\\!d^{3}\vec{x}^{\prime}\mathcal{E}^{\text{em}}_{z}(\vec{x}^{\prime})\dfrac{e^{ik_{a}|\vec{x}-\vec{x}^{\prime}|-i\omega
t}}{4\pi|\vec{x}-\vec{x}^{\prime}|},$ (5)

where $k_{a}=\sqrt{\omega^{2}-m_{a}^{2}}$ are typical momenta of the produced
ALPs ; the integration is performed over the emitter volume $V_{\rm em}$. One
replaces $ik_{a}$ with $-\varkappa_{a}=-\sqrt{m_{a}^{2}-\omega^{2}}$ in Eq.
(5) for $m_{a}\gtrsim\omega$.

### The ALP detection:

Now we discuss the signal induced by the ALP field in the resonant receiver
cavity which is a copy of the emitter cavity, so we would search for exactly
the same TM010 mode
$\mathcal{E}^{\text{em}}_{z}(\vec{x})=\mathcal{E}^{\text{rec}}_{z}(\vec{x})$
which is expected to be grown resonantly. In particular, the resonant
amplitude of the receiver is characterized by the term Gao and Harnik (2021);
Bogorad  _et al._ (2019)

$G=-\dfrac{Q_{\rm
rec}}{\omega_{s}}\cdot\dfrac{1}{V_{{\text{rec}}}}\int\limits_{V_{{\text{rec}}}}d^{3}x\,\mathcal{E}^{z*}_{\text{rec}}(\vec{x})j^{z}_{a}(\vec{x})\;,$
(6)

where $Q_{\rm rec}$ is a quality factor for the receiver eigenmode and $V_{\rm
rec}$ is the volume of the receiver cavity, $\omega_{s}\simeq\omega$ is a
frequency of the receiver signal eigenmode, and
$\mathcal{E}^{z}_{\text{rec}}(\vec{x})$ is a dimensionless signal eigenmode,
that reads $\mathcal{E}_{\text{rec}}^{z}(\vec{x})=\alpha J_{0}(\omega\rho)$.
The expression Eq. (6) reads

$G=-ig_{a\gamma\gamma}\frac{Q_{\text{rec}}B_{\rm ext}}{V_{\rm
rec}}\\!\\!\\!\int\limits_{V_{\rm
rec}}\\!\\!\\!d^{3}x\,\mathcal{E}^{z*}_{\text{rec}}(\vec{x})a(\vec{x})\;,$ (7)

where $B_{\rm ext}$ is a characteristic magnetic field of the detection cavity
and $a(\vec{x})$ is a stationary part of Eq. (5). The typical signal power
reads Salnikov  _et al._ (2024)

$P_{\rm signal}\\!=\\!\dfrac{\omega}{Q_{\rm rec}}\dfrac{1}{2}|G|^{2}V_{\rm
rec}.$ (8)

Now let us estimate the sensitivity numerically as maximum output in the
receiver cavity that is given by the Dicke radiometer equation, Dicke (1946),

$\mathrm{SNR}=\frac{P_{\mathrm{signal}}}{P_{\mathrm{noise}}}\cdot\sqrt{t\Delta\nu}\;,$
(9)

where $t$ is an integration time for a signal, $\Delta\nu$ is its bandwidth
and $P_{\rm noise}$ is a power of thermal noise which can be estimated as
$P_{\rm noise}\simeq T\Delta\nu$ in the limit $\omega\ll T$, where $T\simeq
1.5\,\mbox{K}$ is the typical temperature of the receiver. We consider the
narrowest possible bandwidth of a pump generator, which can be as small as
$\Delta\nu\simeq 1/t$ (see e. g. Refs. Bogorad  _et al._ (2019); Gao and
Harnik (2021) and references therein). The quality factor is chosen to be
$Q_{{\text{rec}}}\simeq 10^{5}$ throughout the paper. Moreover, in the
numerical estimate we conservatively set integration time to be $t\simeq
10^{6}~{}\mbox{s}\simeq 1~{}\mbox{year}$. We address the quantum description
of the signal in Appendix A.

Let $\vec{x}^{\prime}$ and $\vec{x}$ be the coordinates associated with the
emitter and receiver frame respectively. In addition, let $\vec{l}$ be the
vector linking the origins of the frames of both emitter and receiver for the
coaxial design of the experimental setup. In this case one has a simple link
between them $\vec{x}^{\prime}=\vec{l}+\vec{x}$. The vector linking the cavity
frames would be then $\vec{l}=(0,0,l_{z})$. For that notation the distance
between closest endcaps of the cavities is $d=l_{z}-L$.

In the following we rewrite $G$ via the dimensionless form-factor ${\cal G}$

$G={\cal G}\,{g_{a\gamma\gamma}^{2}E_{0}^{{\text{em}}}B_{\rm
ext}^{2}Q_{\text{rec}}V_{{\text{em}}}\omega},$

where the expression for ${\cal G}$ reads

${\cal G}\,=\\!\\!\\!\int\limits_{V_{\rm rec}}\\!\\!\\!\dfrac{d^{3}x}{V_{\rm
rec}}\\!\\!\int\limits_{V_{\rm em}}\\!\\!\dfrac{d^{3}x^{\prime}}{V_{\rm
em}}\,\mathcal{E}^{z*}_{\text{rec}}(\vec{x})\mathcal{E}^{z}_{\text{em}}(\vec{x}^{\prime})\dfrac{e^{ik_{a}|\vec{x}-\vec{x}^{\prime}-\vec{l}|}}{4\pi\omega|\vec{x}-\vec{x}^{\prime}-\vec{l}|}.$
(10)

Finally, by using Eqs. (8) and (9) we obtain the expected sensitivity,

$g_{a\gamma\gamma}\\!\\!=\\!\\!\left[\\!\dfrac{2T\,{\rm SNR}}{\omega^{3}Q_{\rm
rec}E^{2}_{0,{\rm em}}\\!B_{\rm ext}^{4}\\!V^{2}_{\rm em}\\!V_{\rm rec}{|\cal
G}|^{2}}\\!\right]^{\\!\\!\frac{1}{4}}\\!\\!\\!\\!\left(\\!\dfrac{\Delta\nu}{t}\\!\right)^{\\!\frac{1}{8}}\\!\\!\\!\\!,$
(11)

where the signal to noise ratio is chosen to be ${\rm SNR}\simeq 5$ for the
sensitivity estimate.

Remarkably, Eq. (10) can be rewritten in the following form Salnikov  _et al._
(2024)

$\widetilde{\mathcal{G}}=\frac{L^{2}\alpha^{2}\left[(\omega
R)J_{1}\left(\omega R\right)\right]^{2}}{\omega
V_{\text{rec}}V_{{\text{em}}}}\times\mathcal{I},$ (12)

where $\mathcal{I}$ can be expressed through the 2D integral

$\displaystyle\mathcal{I}$ $\displaystyle=$
$\displaystyle\\!\\!\int\limits_{0}^{\infty}\\!\\!dk_{\rho}k_{\rho}\frac{J_{0}^{2}(k_{\rho}R)}{\left(k_{\rho}^{2}-\omega^{2}\right)^{2}}$
(13) $\displaystyle\times$
$\displaystyle\\!\\!\\!\\!\\!\int\limits_{-\infty}^{\infty}\\!\\!\\!dk_{z}\frac{\sin^{2}\left(k_{z}L/2\right)e^{ik_{z}(d+L)}}{(k_{z}L/2)^{2}\left(k_{\rho}^{2}-\left[k_{A^{\prime}}^{2}-k_{z}^{2}+i\varepsilon\right]\right)}.$

For the case of $k_{a}^{2}\equiv\omega^{2}-m_{a}^{2}\gtrsim 0$, the
integration over $k_{z}$ can be split into a sum of two terms,
$\mathcal{I}=\mathcal{I}[k_{\rho}\gtrsim k_{a}]+\mathcal{I}[k_{\rho}\lesssim
k_{a}]$, where

$\displaystyle\mathcal{I}[k_{\rho}\lesssim k_{a}]\\!\\!$ $\displaystyle=\pi
ik_{a}\\!\\!\\!\\!\\!$
$\displaystyle\int\limits_{0}^{1}\\!\\!dx\frac{J_{0}^{2}\left(k_{a}R\sqrt{1-x^{2}}\right)}{\left(\omega^{2}-k_{a}^{2}(1-x^{2})\right)^{2}}$
(14) $\displaystyle\times\\!\\!\\!$
$\displaystyle\\!\\!\\!\\!\frac{\sin^{2}\left(xk_{a}L/2\right)}{\left(k_{a}xL\right/2)^{2}}\exp\left(ixk_{a}l_{z}\right)$

is a term for the typical integration region $k_{\rho}\lesssim k_{a}$ in Eq.
(13) for the variable replacement $k_{\rho}=k_{a}(1-x^{2})^{1/2}$, and

$\displaystyle\mathcal{I}[k_{\rho}\gtrsim k_{a}]$ $\displaystyle=$
$\displaystyle\pi
k_{a}\int\limits_{0}^{\infty}dx\frac{J_{0}^{2}\left(k_{a}R\sqrt{1+x^{2}}\right)}{\left(\omega^{2}-k_{a}^{2}(1+x^{2})\right)^{2}}$
(15) $\displaystyle\times$
$\displaystyle\frac{\sinh^{2}\left(xk_{a}L/2\right)}{\left(k_{a}xL\right/2)^{2}}\exp\left(-xk_{a}l_{z}\right)$

is a term that is associated with the integration range $k_{\rho}\gtrsim
k_{a}$ in Eq. (13), implying the variable redefinition
$k_{\rho}=k_{a}(1+x^{2})^{1/2}$ in the integrand. We note that both Eqs. (14)
and (15) imply the variable redefinition $k_{z}=xk_{a}$. Finally, for
$\varkappa_{a}^{2}\equiv m_{a}^{2}-\omega^{2}\gtrsim 0$ one has

$\displaystyle\mathcal{I}$ $\displaystyle=\pi\varkappa_{a}$
$\displaystyle\int\limits_{1}^{\infty}dx\frac{J_{0}^{2}\left(\varkappa_{a}R\sqrt{x^{2}-1}\right)}{\left(\omega^{2}-\varkappa_{a}^{2}(x^{2}-1)\right)^{2}}$
(16) $\displaystyle\times$
$\displaystyle\frac{\sinh^{2}\left(x\varkappa_{a}L/2\right)}{\left(\varkappa_{a}xL\right/2)^{2}}\exp\left(-x\varkappa_{a}l_{z}\right),$

where we use the integration variable replacement
$k_{\rho}=\varkappa_{a}(x^{2}-1)^{1/2}$.

##  IV Results and Discussion

In Fig. 1 (right panel) we show the setup sensitivity to ALP parameters
$g_{a\gamma\gamma}$ and $m_{a}$. The expected reaches are shown for the
barrier thickness in the range $1~{}\mbox{mm}\lesssim d\lesssim
10~{}\mbox{cm}$. For given benchmark design of the LSthinW setup ($R\simeq
80~{}\mbox{cm}$, $L\simeq 1~{}\mbox{m}$) the resonant enhancement of the ALP
sensitivity $g_{a\gamma\gamma}\lesssim 10^{-11}~{}\mbox{GeV}^{-1}$ can be
achieved for the typical masses $m_{a}\simeq\omega\simeq 6\times
10^{-6}~{}\mbox{eV}$. The expected reach for the sufficiently small ALP
masses, $m_{a}\ll\omega$, can be at the level of $g_{a\gamma\gamma}\lesssim
2\times 10^{-11}~{}\mbox{GeV}^{-1}$.

For the off-shell region of the ALP masses $m_{a}\gtrsim\omega$ one can
achieve the enhancement of the sensitivity due to the reducing the cavity
barrier thickness. In particular, Fig. 1 shows that the smaller the distance
between the cavities $d$ the smaller the slope of the expected reach curve.
The setup with the benchmark thickness of the wall at the level of $d\simeq
1~{}\mbox{mm}$ can rule out the typical masses of ALP below $m_{a}\lesssim
5\times 10^{-5}~{}\mbox{eV}$. Moreover, the regarding region is complementary
to the bounds of purely laboratory LWS setup, which are based on lasers.

In order to conclude this section we emphasize that one can employ the copper
based cavities with typical skin depth of the order of
$\delta\simeq~{}2\mu\mbox{m}$ for $\omega\simeq 1~{}\mbox{GHz}$. This
conservatively implies that the typical barrier thickness can be as large as
$d\gtrsim 30\delta\simeq 60~{}\mu\mbox{m}$, so that our benchmark value
$d\simeq 1~{}\mbox{mm}$ is a fairly reasonable wall thickness.

The possible extension of the present work can be addressed to a probing of
millicharged and axion-like particles in LSthinW setup by varying the cavity
design, magnetic field orientation and the specific choice of the pump mode of
the emitter, say TE011. We leave these tasks for future study.

The code for numerical calculations and drawing figures can be shared upon
request.

##  V Acknowledgements

We would like to thank Maxim Fitkevich, Dima Levkov, Aleksandr Panin, Leysan
Valeeva and Alexey Zhevlakov for fruitful discussions. This work is supported
by RSF grant no. 21-72-10151.

##  Appendix A Quantum description of the signal

Remarkably, that by employing Schwinger-Keldysh formalism Trunin (2023);
Kopchinskii and Satunin (2023) one can link the coherent transition
probability $P^{\rm coh}_{\gamma\to a\to\gamma}$ with the averaged typical
number of photon quanta in the receiver $\langle N_{{\rm
TM}_{010}}\rangle=\langle i|\hat{N}_{{\rm TM}_{010}}|i\rangle=\langle
i|\hat{a}^{\dagger}_{{\rm TM}_{010}}\hat{a}_{{\rm TM}_{010}}|i\rangle$, where
the initial state $|i\rangle$ implies the time evolution from $t_{i}$ to
$t_{f}$ and back to $t_{i}$

$\langle N_{{\rm TM}_{010}}\rangle=\int df^{\prime}\int
df^{\prime\prime}\langle i|\hat{S}^{\dagger}|f^{\prime}\rangle\langle
f^{\prime}|\hat{N}_{{\rm TM}_{010}}|f^{\prime\prime}\rangle\langle
f^{\prime\prime}|\hat{S}|i\rangle,$

where $\hat{S}$ is a S-matrix operator Kopchinskii and Satunin (2023) for ALP-
photon interaction, $|f^{\prime}\rangle$ and $|f^{\prime\prime}\rangle$
represent the full set of states in theory at the moment $t_{f}$. Only
$|f^{\prime}\rangle=|f^{\prime}\rangle$ states yield nonzero outcome, such
that $\langle N_{{\rm TM}_{010}}\rangle=P^{\rm coh}_{\gamma\to a\to\gamma}$,
implying a small number of mode quanta in a steady regime $\langle N_{{\rm
TM}_{010}}\rangle\lesssim 1$. In addition, we note that the straightforward
analytical calculation reveals a well known result Bogorad  _et al._ (2019)
for which the classical number of signal photons in the emitter $N_{\rm
TM_{010}}=Q_{{\text{rec}}}P_{{\text{sign}}}/\omega^{2}$ is $N_{\rm
TM_{010}}=\langle N_{{\rm TM}_{010}}\rangle$.

## References

  * Antel  _et al._ (2023) C. Antel _et al._ , “Feebly-interacting particles: FIPs 2022 Workshop Report,” Eur. Phys. J. C 83, 1122 (2023), arXiv:2305.01715 [hep-ph] .
  * Svrcek and Witten (2006) Peter Svrcek and Edward Witten, “Axions In String Theory,” JHEP 06, 051 (2006), arXiv:hep-th/0605206 .
  * Arvanitaki  _et al._ (2010) Asimina Arvanitaki, Savas Dimopoulos, Sergei Dubovsky, Nemanja Kaloper, and John March-Russell, “String Axiverse,” Phys. Rev. D 81, 123530 (2010), arXiv:0905.4720 [hep-th] .
  * Okun (1982) L. B. Okun, “LIMITS OF ELECTRODYNAMICS: PARAPHOTONS?” Sov. Phys. JETP 56, 502 (1982).
  * Holdom (1986) Bob Holdom, “Two U(1)’s and Epsilon Charge Shifts,” Phys. Lett. B 166, 196–198 (1986).
  * Sikivie (1983) P. Sikivie, “Experimental Tests of the Invisible Axion,” Phys. Rev. Lett. 51, 1415–1417 (1983), [Erratum: Phys.Rev.Lett. 52, 695 (1984)].
  * Anselm (1985) A. A. Anselm, “Arion $\leftrightarrow$ Photon Oscillations in a Steady Magnetic Field. (In Russian),” Yad. Fiz. 42, 1480–1483 (1985).
  * Van Bibber  _et al._ (1987) K. Van Bibber, N. R. Dagdeviren, S. E. Koonin, A. Kerman, and H. N. Nelson, “Proposed experiment to produce and detect light pseudoscalars,” Phys. Rev. Lett. 59, 759–762 (1987).
  * Hoogeveen (1992) F. Hoogeveen, “Terrestrial axion production and detection using RF cavities,” Phys. Lett. B 288, 195–200 (1992).
  * Salnikov  _et al._ (2021) Dmitry Salnikov, Petr Satunin, D. V. Kirpichnikov, and Maxim Fitkevich, “Examining axion-like particles with superconducting radio-frequency cavity,” JHEP 03, 143 (2021), arXiv:2011.12871 [hep-ph] .
  * Gao and Harnik (2021) Christina Gao and Roni Harnik, “Axion searches with two superconducting radio-frequency cavities,” JHEP 07, 053 (2021), arXiv:2011.01350 [hep-ph] .
  * Ehret  _et al._ (2010) Klaus Ehret _et al._ , “New ALPS Results on Hidden-Sector Lightweights,” Phys. Lett. B 689, 149–155 (2010), arXiv:1004.1313 [hep-ex] .
  * Ballou  _et al._ (2015) R. Ballou _et al._ (OSQAR), “New exclusion limits on scalar and pseudoscalar axionlike particles from light shining through a wall,” Phys. Rev. D 92, 092002 (2015), arXiv:1506.08082 [hep-ex] .
  * Betz  _et al._ (2013) M. Betz, F. Caspers, M. Gasior, M. Thumm, and S. W. Rieger, “First results of the CERN Resonant Weakly Interacting sub-eV Particle Search (CROWS),” Phys. Rev. D 88, 075014 (2013), arXiv:1310.8098 [physics.ins-det] .
  * Romanenko  _et al._ (2023) A. Romanenko _et al._ , “New Exclusion Limit for Dark Photons from an SRF Cavity-Based Search (Dark SRF),” (2023), arXiv:2301.11512 [hep-ex] .
  * Berlin  _et al._ (2023) Asher Berlin, Roni Harnik, and Ryan Janish, “Light Shining Through a Thin Wall: Evanescent Hidden Photon Detection,” (2023), arXiv:2303.00014 [hep-ph] .
  * Salnikov  _et al._ (2024) Dmitry Salnikov, Petr Satunin, Leysan Valeeva, and D. V. Kirpichnikov, “Light-shinning-through-thin-wall radio frequency cavities for probing dark photon,” (2024), arXiv:2402.09899 [hep-ph] .
  * Anastassopoulos  _et al._ (2017) V. Anastassopoulos _et al._ (CAST), “New CAST Limit on the Axion-Photon Interaction,” Nature Phys. 13, 584–590 (2017), arXiv:1705.02290 [hep-ex] .
  * Bogorad  _et al._ (2019) Zachary Bogorad, Anson Hook, Yonatan Kahn, and Yotam Soreq, “Probing Axionlike Particles and the Axiverse with Superconducting Radio-Frequency Cavities,” Phys. Rev. Lett. 123, 021801 (2019), arXiv:1902.01418 [hep-ph] .
  * Dicke (1946) R. H. Dicke, “The Measurement of Thermal Radiation at Microwave Frequencies,” Rev. Sci. Instrum. 17, 268–275 (1946).
  * Trunin (2023) Dmitrii A. Trunin, “Enhancement of particle creation in nonlinear resonant cavities,” Phys. Rev. D 107, 065004 (2023), arXiv:2209.10462 [hep-th] .
  * Kopchinskii and Satunin (2023) Ilia Kopchinskii and Petr Satunin, “Resonant generation of electromagnetic modes in nonlinear electrodynamics: Quantum perturbative approach,” (2023), arXiv:2304.10209 [quant-ph] .