Further author information: (Send correspondence to S. M. S.)
S. M. S.: E-mail<EMAIL_ADDRESS>Telephone: +1 304 435 7049
N. M. N. : E-mail<EMAIL_ADDRESS>Telephone: +1 304 293 4815
This paper is accepted in: SPIE Defense & Commercial Sensing 2023, Conference
12521, Automatic target recognition XXXIII, Orlando, Florida

# Deep Transductive Transfer Learning for Automatic Target Recognition

Shoaib M. Sami  LCSEE Dept., West Virginia University, Morgantown, WV, USA
Nasser M. Nasrabadi  LCSEE Dept., West Virginia University, Morgantown, WV,
USA  Raghuveer Rao  DEVCOM Army Research Laboratory, Adelphi, MD, USA

###### Abstract

One of the major obstacles in designing an automatic target recognition (ATR)
algorithm, is that there are often labeled images in one domain (i.e.,
infrared source domain) but no annotated images in the other target domains
(i.e., visible, SAR, LIDAR). Therefore, automatically annotating these images
is essential to build a robust classifier in the target domain based on the
labeled images of the source domain. Transductive transfer learning is an
effective way to adapt a network to a new target domain by utilizing a
pretrained ATR network in the source domain. We propose an unpaired
transductive transfer learning framework where a CycleGAN model and a well-
trained ATR classifier in the source domain are used to construct an ATR
classifier in the target domain without having any labeled data in the target
domain. We employ a CycleGAN model to transfer the mid-wave infrared (MWIR)
images to visible (VIS) domain images (or visible to MWIR domain). To train
the transductive CycleGAN, we optimize a cost function consisting of the
adversarial, identity, cycle-consistency, and categorical cross-entropy loss
for both the source and target classifiers. In this paper, we perform a
detailed experimental analysis on the challenging DSIAC ATR dataset. The
dataset consists of ten classes of vehicles at different poses and distances
ranging from 1-5 kilometers on both the MWIR and VIS domains. In our
experiment, we assume that the images in the VIS domain are the unlabeled
target dataset. We first detect and crop the vehicles from the raw images and
then project them into a common distance of 2 kilometers. Our proposed
transductive CycleGAN achieves 71.56% accuracy in classifying the visible
domain vehicles in the DSIAC ATR dataset.

###### keywords:

Automatic Target Recognition, Transductive Transfer Learning, CycleGAN, Domain
Adaptation.

##  1 INTRODUCTION

Intelligent automatic target recognition (ATR) algorithms are necessary for
advanced military applications where they can drastically reduce civilian
casualties. ATR algorithms are expected to automatically detect and classify
military vehicles[1] with very little human supervision. To build a robust ATR
algorithm, a large-scale target vehicle dataset is required. Generally, ATR
images are captured in the visible, mid-wave infrared (MWIR), long-wave
infrared (LWIR), or radar (i.e., SAR) domain. There are real-world scenarios
where there is well-labeled military vehicle data in one domain (source
domain), where an ATR classifier can easily be designed, but no labeled data
of the same vehicles are available in another domain (target domain) where an
ATR classifier is urgently needed. Also, annotating unlabeled data is costly,
time-consuming, and a complicated task. Therefore, constructing an ATR
classifier in the unlabeled target domain without labeling the data is a
challenging and essential task that can reduce the cost and complexity of the
ATR vehicle annotation and classifier design process.[2, 3, 4, 5]

A large number of algorithms have been proposed in the literature to annotate
unlabeled data. They can be broadly classified into two categories: self-
supervised and semi-supervised [6]. Transductive transfer learning [7, 8] is
one type of semi-supervised method to annotate unlabelled data in the target
domain using labeled information from the source domain. In the literature,
transductive transfer learning is widely used in different applications [9,
10]. However, to the best of our knowledge, transductive transfer learning-
based ATR vehicle annotation has not been exploited. Therefore, there is a
scope to annotate the ATR vehicles using a transductive transfer learning
approach.

In this paper, we propose a CycleGAN-based transductive transfer learning
approach with a CNN-based ATR classifier in the source and target domains.
CycleGAN [11], DRIT [12], and MUNIT [13] etc. are popular unpaired image-to-
image translation algorithms [14]. In this work, we use the CycleGAN network
because it is effective and widely used for domain translation tasks. The goal
of the proposed transductive transfer learning network is to construct a
target domain classifier with no labeled data. We analyze the performance of
both the source and the target domain classifiers on the DSIAC ATR dataset
[15]. The organization of this paper is as follows: we discuss the literature
review of our related work in Section 2. We also describe the methodology and
experiment details in Sections 3 and 4, respectively. The result, discussion,
and conclusion are elicited in Sections 5 and 6, respectively.

##  2 Related Works

###  2.1 Automatic Target Recognition

The automatic target recognition process can be divided into two major tasks:
(a) target vehicle detection and (b) target classification. Deep convolutional
neural networks such as Faster R-CNN [16], SSD [17], YOLO [18], etc., have
been used to detect different military vehicles in several benchmark ATR
datasets.[1]. Target classification algorithms can also be divided into two
major categories: i) feature-based and ii) model-based. The model-based ATR
algorithms perform better as compared to feature-based methods. The model-
based deep learning ATR classification algorithms are generally based on the
convolutional neural networks (CNNs) [1], generative adversarial networks
(GANs) [19], recurrent neural networks (RNN), long short-term memory (LSTM)
[20], or autoencoders [21]. Nasrabadi et al. [1] proposed a CNN-based network
to classify forward-looking infrared images in the LWIR Comanche dataset [22].
In [23] authors proposed Meta-UDA for detecting unlabelled mid-wave infrared
targets by using the labeled visible domain images in the DSIAC ATR dataset
[15]. Ding et al. [24] used different types of augmentation including
translation, speckle noise augmentation, and generating pose synthetic images
to classify SAR images using a CNN classifier. Furthermore, to classify the
noisy SAR images, Wang et al. [25] used a coupled network aggregating a
despeckling subnetwork and a CNN classifier.

The classical CNN classifiers need a large number of training samples; to
address this problem, few-shot learning is employed for the ATR vehicle
classification. Bi-LSTM-based prototypical few-shot learning [26], hybrid
inference [27], meta-learning [28] based few-shot learning is used to classify
SAR images in the MSTAR dataset. Also, autoencoder and supervised constraint
are used to classify the small samples in the ATR dataset [21]. Moreover, a
semi-supervised ATR recognition method is proposed to classify the SAR images
[19]. The authors [19] constructed a CNN-based ATR classifier by using
original target images as well as synthetic images generated by GAN.

###  2.2 Transductive Transfer Learning

Transductive transfer learning (TTL) is widely used in the literature for
classifying unlabelled data. For example, the authors [29] propose a cross-
domain aspect label propagation-based transductive learning method for opinion
mining and sentiment analysis. RNN and LSTM-based transductive learning method
is proposed for optical character recognition [30]. This approach considers
that both the OCR and the text contain common information in the high
dimensional space. The emotion recognition in the wild dataset is solved by
sparse transductive transfer linear discriminant analysis [31]. The
transductive deep transfer learning approach based on VGGFace16-Net is
proposed for cross-domain expression recognition; in this work, cross-entropy
loss of the source domain and regression loss of the target domain is
optimized [32]. TTL is used to detect zero-day attacks in intrusion detection
systems. This method uses cluster correspondence in the domain adaptation
using manifold alignment [33]. Supervised TTL is used for gesture
classification by using the electromyographic dataset [34]. Takagi-Sugeno-
Kang’s fuzzy logic systems based on transductive transfer learning are used
for detecting epilepsy using electroencephalogram signals [35, 36]. Genetic
programming and distributional correspondence indexing-based transductive
transfer learning is used for document and text classification [37, 38]. The
crop classification is essential for agriculture and food management, but
sometimes there is only a small amount of ground truth data. To solve this
issue, authors used a transductive transfer learning-based approach [10] for
crop classification. Deep Forest-based TTL is used in cross-domain transfer
learning and for accuracy measures into the MNIST, USPS, Amazon, DSLR, Webcam,
and Caltech-256 datasets [9].

##  3 Methodology

In this section, we will describe our proposed transductive CycleGAN. The
first generator of our CycleGAN network transforms the source domain images
into the target domain images. Then, the second CycleGAN generator transforms
the synthetic images back into the source domain images. Moreover, there are
dedicated source and target domain CNN-based ATR classifiers that
backpropagate the categorical cross-entropy loss to help to train the whole
TTL network. Our proposed method is depicted in Figure 1.

---
Figure 1:  Block diagram of the deep transductive transfer learning framework
for automatic target classification.

###  3.1 Generative Adversarial Network (GAN)

GAN [39] is a powerful tool for different types of image generation, image
translation, and super-resolution. A generator and a discriminator are the
building blocks of a GAN architecture. The GAN generates images from a noise
variable $z$; the generator function can be depicted as $G:z\to y$. The
generator $G(z;\theta_{g})$ is optimized for the $\theta_{g}$ parameter. On
the other hand, the discriminator function $D$ is designed to distinguish
between the real image distribution and the generated image distribution. In
practice, the discriminator is a simple binary classifier that classifies
whether the image is real (y) or fake (G(z)). In GAN, the generator and the
discriminator play with each other as a two-player mini-max game[39]. The
objective function of GAN can be expressed by $\mathcal{L}(G,D)$:

$\displaystyle\mathcal{L}(G,D)=\min_{G}\max_{D}\mathbb{E}_{x\sim
p_{\text{data}}(x)}[\log{D(x)}]+\mathbb{E}_{z\sim
p_{\text{z}}(z)}[1-\log{D(G(z))}].$ (1)

###  3.2 CycleGAN

In the unpaired image-to-image translation literature, CycleGAN [11] has been
successfully employed as a powerful domain-to-domain transformation tool. This
algorithm consists of two generators and two discriminators. The first
generator (G) translates the domain X images into the domain Y images. The
second generator (F) translates the domain Y images into the domain X images.
The combined adversarial loss for the CycleGAN’s generators G and F can be
expressed by $\mathcal{L}_{GAN}(G,F,D_{x},D_{y})$:

$\displaystyle\mathcal{L}_{GAN}(G,F,D_{x},D_{y})=\min_{G}\max_{D_{y}}\mathbb{E}_{y\sim
p_{\text{data}}(y)}[\log{D_{y}(y)}]+\mathbb{E}_{x\sim
p_{\text{data}}(x)}[1-\log{D_{y}(G(x))}]$ (2)
$\displaystyle+\min_{F}\max_{D_{x}}\mathbb{E}_{x\sim
p_{\text{data}}(x)}[\log{D_{x}(x)}]+\mathbb{E}_{y\sim
p_{\text{data}}(y)}[1-\log{D_{x}(F(y))}].$

This mapping does not always depict a purposeful way. So, the authors proposed
that the mapping should be a bijection. If there is no bijection mapping then
the generator (G) can map an image in an infinite way that follows a similar
distribution of domain Y. In the CycleGAN paper, the authors introduced the
cycle-consistency loss, which translates the domain X image to the domain Y
image then the domain Y translated image is converted back to the domain X
image by the second generator F. Mathematically; it can be described as a
generator $G:X\to Y$ and another generator $F:Y\to X$. Also, the generators G
and F should translate in the inverse domain, and mapping should be bijection
[11]. The Cycle consistency loss can be denoted as $\mathcal{L}_{Cycle}$ where
$\eta_{1}$ and $\eta_{2}$ are the hyper-parameters:

$\displaystyle\mathcal{L}_{Cycle}(G,F)=\eta_{1}*\mathbb{E}_{x\sim
p_{\text{data}}(x)}\left(\left\|F(G(x))-x\right\|_{1}\right)+\eta_{2}*\mathbb{E}_{y\sim
p_{\text{data}}(y)}\left(\left\|G(F(y))-y\right\|_{1}\right).$ (3)

---
Figure 2:  Block diagram of the CycleGAN’s generator and discriminator.[40]

Moreover, the CycleGAN generator has two vehicle identity mapping regularizers
that are employed as real images on the target domain and fed as input for the
source generator. This loss is called identity loss ($\mathcal{L}_{identity}$)
where $\eta_{3}$ and $\eta_{4}$ are the hyper-parameters:

$\displaystyle\mathcal{L}_{Identity}(G,F)=\eta_{3}*\mathbb{E}_{y\sim
p_{\text{data}}(y)}\left(\left\|G(y)-y\right\|_{1}\right)+\eta_{4}*\mathbb{E}_{x\sim
p_{\text{data}}(x)}\left(\left\|F(x)-x\right\|_{1}\right).$ (4)

The total CycleGAN loss ($\mathcal{L}_{CycleGAN}$) is the summation of the
adversarial loss, cycle consistency loss, and identity loss where
$\lambda_{a}$, $\lambda_{b}$, and $\lambda_{c}$ are the hyper-parameters:

$\displaystyle\mathcal{L}_{CycleGAN}(G,F,D_{x},D_{y})=\lambda_{a}*\mathcal{L}_{GAN}(G,F,D_{x},D_{y})+\lambda_{b}*\mathcal{L}_{Cycle}(G,F)+\lambda_{c}*\mathcal{L}_{Identity}(G,F).$
(5)

###  3.3 Source and Target Domain Classifiers

The transductive CycleGAN network shown in Figure 1, has source and target
domain ATR classifiers. In our experiment, the architecture of both
classifiers is ResNet-18 [41]. The skip connection in ResNet makes it possible
to build a deeper network and helps to backpropagate loss. In the proposed
method, we optimize the cross-entropy loss ($\mathcal{L}_{CE}$) of the
classifiers. In this equation, $y_{o,c}$ is the true probability and $p_{o,c}$
is the predicted probability of the target chip in the training data. Here,
the class number is denoted by c.

$\displaystyle\mathcal{L}_{CE}=-\sum_{c=1}^{10}y_{o,c}\log(p_{o,c}).$ (6)

###  3.4 Transductive CycleGAN

Our proposed transductive CycleGAN is depicted in Figure 1, which consists of
a CycleGAN and two residual network-based CNN classifiers. The architecture of
the transductive CycleGAN generators and discriminators is illustrated in
Figure 2. The transductive CycleGAN generator consists of fractionally-strided
convolutions, regular convolutions, and residual blocks.

In this experiment, we consider the MWIR and VIS domain ATR vehicle datasets
as the source and target domain data, respectively. Moreover, in this
experiment, the source domain classifier is assumed to be well-trained (called
the optimal MWIR classifier) and its weights are fixed. During the training,
the source domain classifier’s weights are initialized to the target domain
classifier’s weights which are gradually trained using the transductive
transfer learning process. The proposed network’s weights are obtained by
minimizing the adversarial loss, cycle consistency loss, identity loss of
CycleGAN, and categorical cross-entropy loss of the source and target domain
ATR classifiers. It should be be pointed out that the source domain classifier
loss is also backpropagated to the transductive network, but its weights are
not updated during the training of the transductive network. The total loss of
the transductive CycleGAN model is $\mathcal{L}_{Total}$, given by:

$\displaystyle\mathcal{L}_{Total}=\mathcal{L}_{CycleGAN}+\lambda_{CE}*\mathcal{L}_{CE-
Source}+\lambda_{CE}*\mathcal{L}_{CE-Target},$ (7)

where $\mathcal{L}_{CE-Source}$ and $\mathcal{L}_{CE-Target}$ are the cross-
entropy losses for the source and target domain classifiers, respectively.

##  4 Experiments

###  4.1 Dataset

We implemented our transductive transfer learning approach using the publicly
available DSIAC dataset [15]. This dataset is collected by the US Army Night
Vision and Electronic Sensors Directorate (NVESD). The DSIAC dataset contains
ten vehicle targets in the visible and MWIR domain. Among these classes, two
of them are civilian vehicles (‘Pickup’, ‘Sport vehicle’), one is artillery
piece (‘D20’), and the rest of the seven are military vehicles (‘2S3’,
‘BTR70’, ‘BRDM2’, ‘BMP2’, ‘MT-LB’, ‘T72’, ‘ZSU23’). The distance between
target vehicles and cameras is altered from 1 kilometer to 5 kilometers. The
interval of capturing distance is 500 meters. The dataset contains 189 video
sequences in the MWIR domain and 97 video sequences in the visible domain.
Each video sequence contains 1800 video frames. Typically, the image size in
the DSIAC dataset is 640x480 pixels. In this experiment, we detect and crop
the ATR vehicles from the DSIAC dataset by using the information from the
Meta-UDA [23]. The different distances of target vehicles are projected into
the canonical distance (2 kilometers) using bi-cubic interpolation. The final
target chip size is 68x68x3.

###  4.2 Training Details

Our transductive network consists of a CycleGAN and two CNN-based vehicle
classifiers. The learning rate of the CycleGAN network is set to 0.0002 for
the first 50 epochs; after that, the learning rate is reduced to 0.0001 for
the rest of the 50 epochs. For fast convergence of the network, we have
initialized the weights of the CycleGAN generator weight from the weights of
summer to winter translation on the Yosemite dataset [42]. The hyper-
parameters of the equations 3, 4, 5 are set to $\eta_{1}$ = $\eta_{2}$ = 10,
$\eta_{3}$ = $\eta_{4}$ = 5, and $\lambda_{a}$ = $\lambda_{b}$ = $\lambda_{c}$
= 1. For the initial 20 epochs of training, we set $\lambda_{CE}$ = 0.5, after
that it becomes 2.5 in equation 7. To optimize the weights of the generators
and discriminators, we used the Adam optimizer[43] with $\beta$1 = 0.5 and
$\beta$2 = 0.999. To stabilize the GAN training, we update the discriminator
loss five times less than the generator loss. For training the classifiers, we
also used the Adam optimizer with a learning rate of 0.0005 and $\beta$1 =
0.9, $\beta$2 = 0.999 for 40 epochs. We fine-tune the target domain classifier
for 10 epochs to get the result from 1%, 5%, and 10% labeled data in Figure 4.
This experiment was conducted on an NVIDIA RTX-8000 GPU using PyTorch
framework [44] with a batch size of 160.

##  5 Results and Discussion

We evaluate the source and target classifier’s performance on the DSIAC
dataset. The training, testing, and validation datasets are constructed by
randomly dividing the DSIAC dataset into 70:15:15 ratio, respectively. The
confusion matrix of the source domain classifier is illustrated in Figure
3(a). The confusion matrix provides the normalized performance of the
classifier. The average accuracy of the source domain classifier is 99.16%.
The performance of the source domain classifier with targets at different
distances is investigated in Table 1. From this table, we can conclude that
the classification performance of the source domain classifier is almost
consistent when capturing images at distances between 1 kilometer to 4
kilometers. However, the source classifier performance is degraded when the
target capturing distance is beyond 4 kilometers. Because long-distance target
chips are low in quality as compared to short-distance ones. We constructed a
target domain classifier in this experiment using the transductive CycleGAN
network. The confusion matrix of the target domain classifier is depicted in
Figure 3(b). The average accuracy of the target domain classifier is 71.6% for
the visible domain test set of the DSIAC dataset. The confusion matrix of the
target domain classifier exposes that the classification performances of the
‘MTLB’, ‘Sport Vehicle’, and ‘ZSU23-4’ are lower than other classes. Moreover,
we investigated the target domain classifier performance with a fraction of
the labeled data. The accuracy of the target domain classifier is 80.24%,
90.79%, and 94.86%, with 1%, 5%, and 10% of the labeled target domain dataset,
respectively. The confusion matrices of the visible domain classifier with 1%
and 10% labeled data are depicted in Figure 4. The synthetic images generated
by transductive CycleGAN are also illustrated in Figure 5. The Fréchet
Inception Distance (FID) [45] score of synthetic images is 216.109, which
elicits that the network is successful in generating high-quality synthetic
visible images.

Finally, the performance of the trained target domain classifier is moderate.
It may be the cause of transductive CycleGAN considers bijection mappings that
are too strict in condition [46]. Therefore, other unpaired domain translation
methods without using bijection mappings [47, 46] can be employed to
investigate the performance of optimal target domain classifier.

---
Figure 3:  Confusion matrices of (a) the source domain classifier and (b) the target domain classifier. Table 1: Performance of source domain classifier with different target distance. Target Distance (m) | Accuracy(%) | Number of Samples
---|---|---
1000 | 99.47 | 4,885
1500 | 99.72 | 4,713
2000 | 99.78 | 4,895
2500 | 99.54 | 4,830
3000 | 99.45 | 4,558
3500 | 99.76 | 4,193
4000 | 99.11 | 4,506
4500 | 97.83 | 4,196
5000 | 93.55 | 4,106
---
Figure 4:  Confusion matrices of (a) target domain classifier (1% labeled
data) and (b) target domain classifier (10% labeled data).
---
Figure 5:  Generated images by the proposed transductive CycleGAN framework.

##  6 Conclusion

In this paper, we propose a transductive transfer learning network consisting
of a CycleGAN and two CNN-based ATR classifiers (i.e., MWIR and VIS ATR
classifiers). The MWIR classifier is used to annotate the visible domain
images in the DSIAC ATR dataset. Our proposed Transductive CycleGAN framework
can be used to design an ATR classifier in the target domain without any
labeled data. It can also ameliorate the cumbersome and expensive manual ATR
annotation process. We analyze the performance of the source domain classifier
as well as the target domain classifier trained with no label data. The
performance of the trained target domain classifier is 71.6% which indicates
the effectiveness of our proposed method with no labeled data in the target
domain.

###### Acknowledgements.

This material is based upon work supported in part by the U. S. Army Research
Laboratory and the U. S. Army Research Office under contract number:
W911NF2210117.

## References

  * [1] Nasrabadi, N. M., “DeepTarget: An automatic target recognition using deep convolutional neural networks,” IEEE Transactions on Aerospace and Electronic Systems 55(6), 2687–2697 (2019).
  * [2] Chen, J., Du, L., He, H., and Guo, Y., “Convolutional factor analysis model with application to radar automatic target recognition,” Pattern Recognition 87, 140–156 (2019).
  * [3] El-Darymli, K., Gill, E. W., Mcguire, P., Power, D., and Moloney, C., “Automatic target recognition in synthetic aperture radar imagery: A state-of-the-art review,” IEEE access 4, 6014–6058 (2016).
  * [4] Topple, J. M. and Fawcett, J. A., “MiNet: Efficient deep learning automatic target recognition for small autonomous vehicles,” IEEE Geoscience and Remote Sensing Letters 18(6), 1014–1018 (2020).
  * [5] Dang, S., Cao, Z., Cui, Z., Pi, Y., and Liu, N., “Open set incremental learning for automatic target recognition,” IEEE Transactions on Geoscience and Remote Sensing 57(7), 4445–4456 (2019).
  * [6] Chen, T., Kornblith, S., Norouzi, M., and Hinton, G., “A simple framework for contrastive learning of visual representations,” in [International conference on machine learning ], 1597–1607, PMLR (2020).
  * [7] Shi, W., Gong, Y., Ding, C., Tao, Z. M., and Zheng, N., “Transductive semi-supervised deep learning using min-max features,” in [Proceedings of the European Conference on Computer Vision (ECCV) ], 299–315 (2018).
  * [8] Rohrbach, M., Ebert, S., and Schiele, B., “Transfer learning in a transductive setting,” Advances in neural information processing systems 26 (2013).
  * [9] Utkin, L. V. and Ryabinin, M. A., “A deep forest for transductive transfer learning by using a consensus measure,” Communications in computer and information science, 194–208, Springer International Publishing (2018).
  * [10] Luo, Y., Zhang, Z., Zhang, L., Han, J., Cao, J., and Zhang, J., “Developing high-resolution crop maps for major crops in the european union based on transductive transfer learning and limited ground data,” Remote Sensing 14, 1809 (Apr. 2022).
  * [11] Zhu, J.-Y., Park, T., Isola, P., and Efros, A. A., “Unpaired image-to-image translation using cycle-consistent adversarial networks,” in [Proceedings of the IEEE international conference on computer vision ], 2223–2232 (2017).
  * [12] Lee, H.-Y., Tseng, H.-Y., Mao, Q., Huang, J.-B., Lu, Y.-D., Singh, M., and Yang, M.-H., “DRIT++: Diverse image-to-image translation via disentangled representations,” International Journal of Computer Vision 128(10), 2402–2417 (2020).
  * [13] Huang, X., Liu, M.-Y., Belongie, S., and Kautz, J., “Multimodal unsupervised image-to-image translation,” in [Proceedings of the European conference on computer vision (ECCV) ], 172–189 (2018).
  * [14] Pang, Y., Lin, J., Qin, T., and Chen, Z., “Image-to-image translation: Methods and applications,” IEEE Transactions on Multimedia (2021).
  * [15] “Us army night vision and electronic sensors directorate (NVESD).”
  * [16] Girshick, R., Donahue, J., Darrell, T., and Malik, J., “Rich feature hierarchies for accurate object detection and semantic segmentation,” in [Proceedings of the IEEE conference on computer vision and pattern recognition ], 580–587 (2014).
  * [17] Liu, W., Anguelov, D., Erhan, D., Szegedy, C., Reed, S., Fu, C.-Y., and Berg, A. C., “SSD: Single shot multibox detector,” in [European conference on computer vision ], 21–37, Springer (2016).
  * [18] Redmon, J., Divvala, S., Girshick, R., and Farhadi, A., “You only look once: Unified, real-time object detection,” in [Proceedings of the IEEE conference on computer vision and pattern recognition ], 779–788 (2016).
  * [19] Zheng, C., Jiang, X., and Liu, X., “Semi-supervised SAR ATR via multi-discriminator generative adversarial network,” IEEE Sensors Journal 19(17), 7525–7533 (2019).
  * [20] Zhang, F., Hu, C., Yin, Q., Li, W., Li, H., and Hong, W., “SAR target recognition using the multi-aspect-aware bidirectional LSTM recurrent neural networks,” arXiv preprint arXiv:1707.09875 (2017).
  * [21] Deng, S., Du, L., Li, C., Ding, J., and Liu, H., “SAR automatic target recognition based on Euclidean distance restricted autoencoder,” IEEE Journal of Selected Topics in Applied Earth Observations and Remote Sensing 10(7), 3323–3333 (2017).
  * [22] Patel, V. M., Nasrabadi, N. M., and Chellappa, R., “Sparsity-motivated automatic target recognition,” Applied optics 50(10), 1425–1433 (2011).
  * [23] Vs, V., Poster, D., You, S., Hu, S., and Patel, V. M., “Meta-UDA: Unsupervised domain adaptive thermal object detection using meta-learning,” in [Proceedings of the IEEE/CVF Winter Conference on Applications of Computer Vision ], 1412–1423 (2022).
  * [24] Ding, J., Chen, B., Liu, H., and Huang, M., “Convolutional neural network with data augmentation for SAR target recognition,” IEEE Geoscience and remote sensing letters 13(3), 364–368 (2016).
  * [25] Wang, J., Zheng, T., Lei, P., and Bai, X., “Ground target classification in noisy SAR images using convolutional neural networks,” IEEE Journal of Selected Topics in Applied Earth Observations and Remote Sensing 11(11), 4180–4192 (2018).
  * [26] Wang, L., Bai, X., Xue, R., and Zhou, F., “Few-shot SAR automatic target recognition based on Conv-BiLSTM prototypical network,” Neurocomputing 443, 235–246 (2021).
  * [27] Wang, L., Bai, X., Gong, C., and Zhou, F., “Hybrid inference network for few-shot SAR automatic target recognition,” IEEE Transactions on Geoscience and Remote Sensing 59(11), 9257–9269 (2021).
  * [28] Fu, K., Zhang, T., Zhang, Y., Wang, Z., and Sun, X., “Few-shot SAR target classification via metalearning,” IEEE Transactions on Geoscience and Remote Sensing 60, 1–14 (2021).
  * [29] Marcacini, R. M., Rossi, R. G., Matsuno, I. P., and Rezende, S. O., “Cross-domain aspect extraction for sentiment analysis: A transductive learning approach,” Decis. Support Syst. 114, 70–80 (Oct. 2018).
  * [30] He, Y., Yuan, J., and Li, L., “Enhancing RNN based OCR by transductive transfer learning from text to images,” AAAI 32 (Apr. 2018).
  * [31] Zong, Y., Zheng, W., Huang, X., Yan, K., Yan, J., and Zhang, T., “Emotion recognition in the wild via sparse transductive transfer linear discriminant analysis,” Journal on Multimodal User Interfaces 10, 163–172 (June 2016).
  * [32] Yan, K., Zheng, W., Zhang, T., Zong, Y., Tang, C., Lu, C., and Cui, Z., “Cross-domain facial expression recognition based on transductive deep transfer learning,” IEEE Access 7, 108906–108915 (2019).
  * [33] Sameera, N. and Shashi, M., “Deep transductive transfer learning framework for zero-day attack detection,” ICT Express 6, 361–367 (Dec. 2020).
  * [34] Kobylarz, J., Bird, J. J., Faria, D. R., Ribeiro, E. P., and Ekárt, A., “Thumbs up, thumbs down: non-verbal human-robot interaction through real-time EMG classification via inductive and supervised transductive transfer learning,” J. Ambient Intell. Humaniz. Comput. 11, 6021–6031 (Dec. 2020).
  * [35] Deng, Z., Xu, P., Xie, L., Choi, K.-S., and Wang, S., “Transductive Joint-Knowledge-Transfer TSK FS for recognition of epileptic EEG signals,” IEEE Trans. Neural Syst. Rehabil. Eng. 26, 1481–1494 (Aug. 2018).
  * [36] Yang, C., Deng, Z., Choi, K.-S., and Wang, S., “Takagi–Sugeno–Kang transfer learning fuzzy logic system for the adaptive recognition of epileptic electroencephalogram signals,” IEEE Trans. Fuzzy Syst. 24, 1079–1094 (Oct. 2016).
  * [37] Fu, W., Xue, B., Zhang, M., and Gao, X., “Transductive transfer learning in genetic programming for document classification,” in [Simulated Evolution and Learning ], 556–568, Springer International Publishing (2017).
  * [38] Moreo, A., Esuli, A., and Sebastiani, F., “Lost in transduction: Transductive transfer learning in text classification,” ACM Trans. Knowl. Discov. Data 16, 1–21 (July 2021).
  * [39] Goodfellow, I., Pouget-Abadie, J., Mirza, M., Xu, B., Warde-Farley, D., Ozair, S., Courville, A., and Bengio, Y., “Generative adversarial networks,” Communications of the ACM 63(11), 139–144 (2020).
  * [40] Ali, M. B., Gu, I. Y.-H., Berger, M. S., Pallud, J., Southwell, D., Widhalm, G., Roux, A., Vecchio, T. G., and Jakola, A. S., “Domain mapping and deep learning from multiple MRI clinical datasets for prediction of molecular subtypes in low grade gliomas,” Brain sciences 10(7), 463 (2020).
  * [41] He, K., Zhang, X., Ren, S., and Sun, J., “Deep residual learning for image recognition,” in [Proceedings of the IEEE conference on computer vision and pattern recognition ], 770–778 (2016).
  * [42] “Cyclegan summer-winter image translation[pytorch],” (October 2020).
  * [43] Kingma, D. P. and Ba, J., “Adam: A method for stochastic optimization,” arXiv preprint arXiv:1412.6980 (2014).
  * [44] Paszke, A. e. a., “PyTorch: An imperative style, high-performance deep learning library,” in [Advances in Neural Information Processing Systems 32 ], 8024–8035, Curran Associates, Inc. (2019).
  * [45] Heusel, M., Ramsauer, H., Unterthiner, T., Nessler, B., and Hochreiter, S., “GANs trained by a two time-scale update rule converge to a local nash equilibrium,” Advances in neural information processing systems 30 (2017).
  * [46] Park, T., Efros, A. A., Zhang, R., and Zhu, J.-Y., “Contrastive learning for unpaired image-to-image translation,” in [European conference on computer vision ], 319–345, Springer (2020).
  * [47] Zheng, C., Cham, T.-J., and Cai, J., “The spatially-correlative loss for various image translation tasks,” in [Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition ], (2021).


ADMX Collaboration

# Dark Matter Axion Search Using a Josephson Traveling Wave Parametric
Amplifier

C.Bartram<EMAIL_ADDRESS>T. Braine  R. Cervantes  N. Crisosto  N. Du  G.
Leum  P. Mohapatra  T. Nitta  L. J Rosenberg  G. Rybka  J. Yang  University of
Washington, Seattle, Washington 98195, USA  John Clarke  I. Siddiqi
University of California, Berkeley, California 94720, USA  A. Agrawal  A. V.
Dixit  University of Chicago, Illinois 60637, USA  M. H. Awida  Fermi National
Accelerator Laboratory, Batavia, Illinois 60510, USA  A. S. Chou  Fermi
National Accelerator Laboratory, Batavia, Illinois 60510, USA  M. Hollister
Fermi National Accelerator Laboratory, Batavia, Illinois 60510, USA  S. Knirck
A. Sonnenschein  W. Wester  Fermi National Accelerator Laboratory, Batavia,
Illinois 60510, USA  J. R. Gleason  A. T. Hipp  S. Jois  P. Sikivie  N. S.
Sullivan  D. B. Tanner  University of Florida, Gainesville, Florida 32611, USA
E. Lentz  University of Göttingen, Göttingen 37077, Germany  R. Khatiwada
Illinois Institute of Technology, Chicago, Illinois 60616, USA  Fermi National
Accelerator Laboratory, Batavia, Illinois 60510, USA  G. Carosi  C. Cisneros
N. Robertson  N. Woollett  Lawrence Livermore National Laboratory, Livermore,
California 94550, USA  L. D. Duffy  Los Alamos National Laboratory, Los
Alamos, New Mexico 87545, USA  C. Boutan  M. Jones  B. H. LaRoque  N. S.
Oblath  M. S. Taubman  Pacific Northwest National Laboratory, Richland,
Washington 99354, USA  E. J. Daw  M. G. Perry  University of Sheffield,
Sheffield S10 2TN, UK  J. H. Buckley  C. Gaikwad  J. Hoffman  K. Murch
Washington University, St. Louis, Missouri 63130, USA  M. Goryachev  B. T.
McAllister  A. Quiskamp  C. Thomson  M. E. Tobar  University of Western
Australia, Perth, Western Australia 6009, Australia  V. Bolkhovsky  MIT
Lincoln Laboratory, 244 Wood Street, Lexington, MA 02421  G. Calusine  MIT
Lincoln Laboratory, 244 Wood Street, Lexington, MA 02421  W. Oliver  MIT
Lincoln Laboratory, 244 Wood Street, Lexington, MA 02421  Massachusetts
Institute of Technology, 77 Massachusetts Avenue, Cambridge, MA 02139  K.
Serniak  MIT Lincoln Laboratory, 244 Wood Street, Lexington, MA 02421

###### Abstract

We present a new exclusion bound of axion-like particle dark matter with
axion-photon couplings above $\mathrm{10^{-13}}$ $\mathrm{GeV^{-1}}$ over the
frequency range 4796.7–4799.5 MHz, corresponding to a narrow range of axion
masses centered around 19.84 $\mathrm{\SIUnitSymbolMicro}$eV. This measurement
represents the first implementation of a Josephson Traveling Wave Parametric
Amplifier (JTWPA) in a dark matter search. The JTWPA was operated in the
insert of the Axion Dark Matter eXperiment (ADMX) as part of an independent
receiver chain that was attached to a 0.588-liter cavity. The ability of the
JTWPA to deliver high gain over a wide (3 GHz) bandwidth has engendered
interest from those aiming to perform broadband axion searches, a longstanding
goal in this field.

††preprint: APS/123-QED

Thought to compose 85% Ade  _et al._ (2014) of the matter content of the
universe, dark matter appears necessary to explain a variety of observations,
from galactic rotation curves and gravitational lensing, to the anisotropies
of the cosmic microwave background and the properties of galaxy cluster
collisions Ade  _et al._ (2014); Rubin  _et al._ (1980); Kaiser and Squires
(1993); Robertson  _et al._ (2016). Despite the scope of these observations,
the nature of dark matter remains a mystery. One particularly promising
candidate is the axion, which has the unique ability not only to account for
all the dark matter Preskill  _et al._ (1983); Abbott and Sikivie (1983); Dine
and Fischler (1983), but also to solve the so-called strong-CP problem Peccei
and Quinn (1977a, b); Weinberg (1978); Wilczek (1978).

One attempt to render the axion visible is called the resonant cavity
haloscope, first proposed by Pierre Sikivie Sikivie (1985, 1983). The Axion
Dark Matter eXperiment (ADMX) is one such experiment that is unique in its
ability to detect Dine-Fischler-Srednicki-Zhitnitsky (DFSZ) axions Dine  _et
al._ (1981); Zhitnitskii (1980); Bartram  _et al._ (2021). Other haloscopes
have achieved sensitivity to the Kim-Schifman-Vainshtein-Zakharov (KSVZ) model
of the axion Kim (1979); Shifman  _et al._ (1980); Zhong  _et al._ (2018);
Brubaker  _et al._ (2017a, b); Hagmann  _et al._ (1998); Asztalos  _et al._
(2004, 2002). Of the two, the DFSZ axion is the more difficult to detect, with
signals an order of magnitude smaller than the KSVZ axion. Sensitivity to the
DFSZ axion is therefore a noteworthy feat. Such progress is attributable to
the advent of ultra-low-noise quantum amplifiers such as the Microstrip SQUID
Amplifier (MSA) Mück  _et al._ (1998) and the Josephson Parametric Amplifier
(JPA) Hatridge  _et al._ (2011); Boutin  _et al._ (2017). The subsequently
developed Josephson Traveling Wave Parametric Amplifier (JTWPA) Aumentado
(2020) provides power gain over a much wider bandwidth, which could enable
broadband axion searches.

The JTWPA consists of a lumped element transmission line, where Josephson
junctions are the non-linear inductive element. When a microwave signal
travels down the line, the non-linear inductance causes four-wave mixing. This
feature enables the JTWPA to provide broadband power gain Aumentado (2020).
Due to this feature, it is recognized that the JTWPA is generally well-suited
for axion searches that aim to cover a wider range of the axion parameter
space. Broadband experiments are ideal for axion searches involving transient
signals appearing off-resonance, and allow for the possibility of multimode
searches Arvanitaki  _et al._ (2020); Rosenberg (2015). Another advantage of
the JTWPA is that it does not require a circulator to separate incoming and
outgoing modes, enabling a compact receiver design. This helpful feature is
ideal for axion searches hoping to take advantage of any extra space in a
volume that is typically constrained by the diameter of a magnet bore.

The wide bandwidth of the JTWPA simplifies the amplifier optimization process.
A JTWPA requires only the adjustment of the pump frequency and power, whereas
a JPA requires the adjustment of the former plus a bias current to tune its
resonant frequency. A feature of the JTWPA is that its gain is not flat across
a wide range. Instead, there are oscillations in the gain by about 10 dB, with
peaks spaced roughly 38 MHz apart. This feature arises from imperfect
impedance matching to the Josephson Junctions in the transmission line that
constitutes the JTWPA. The exact spacing and height of the peaks is dependent
on the individual JTWPA. Adjusting the frequency of the JTWPA pump tone can
adjust the frequencies of these peaks, and therefore optimize the gain at a
particular frequency. A demonstration of this effect is shown in the plot of
the receiver gain as a function of frequency, shown in Fig. 2.

In an effort to integrate a JTWPA into an axion search, a prototype experiment
was operated within the ADMX magnet bore. The “sidecar” cavity is mounted on
top of the ADMX main cavity Bartram  _et al._ (2021), residing in a magnetic
field of about 3.81 T just above the main experiment’s superconducting magnet
Boutan (2017). Although it uses an independent receiver chain, its operation
is secondary to the main experiment. Data-taking is therefore subject to stop
when technical issues pertaining to the main experiment are encountered. The
0.588-$\ell$ sidecar cavity can be disassembled into two halves, thus allowing
a single tuning rod to be mounted inside with ease. Typical measurements of
the sidecar quality factor, or $Q$, were approximately 700. The quality factor
was lower than expected due to RF leakage attributable to poor alignment of
the two cavity halves. The cavity is coupled by an antenna to a receiver chain
that exists independently of the main experiment. At present, the current
iteration of the sidecar uses a JTWPA as its first-stage amplifier and a
heterostructure field-effect transistor (HFET) amplifier as its second-stage
amplifier SCH . A diagram of the receiver chain for the sidecar cavity can be
seen in Fig. 1. The receiver chain allows for transmission and reflection
measurements to determine the cavity frequency, quality factor, and antenna
coupling. The JTWPA is located on the output line, acting as the first stage
amplifier and conduit for power coming from the cavity. The JTWPA is operated
by providing a pump tone coming from a local oscillator in the warm
electronics space. The particular JTWPA used in this search was fabricated at
and acquired from Lincoln Labs O’Brien  _et al._ (2014). It is made from
niobium, with a critical temperature of 9.3 K. Data were acquired for about
two weeks with the sidecar cavity until the main experiment required a magnet
rampdown. Our first implementation tested only a narrow frequency range that
was accessible with the sidecar cavity due to a sticking tuning rod that
required repair after completion of the main experiments operation.
Nevertheless, this represents the first axion search with a JTWPA.

Figure 1: RF schematic of the sidecar cavity for this run. The JTWPA resides
between two circulators on the output line. These two circulators are used to
prevent power from reflecting back into the cavity. All these components are
mounted within a so-called quantum amplifier package that is thermally sunk
and bolted to the main ADMX cavity via a coldfinger. The package itself
resides in a low-field region that is maintained using a field cancellation
coil.

Throughout the course of data-taking, the JTWPA gain and signal-to-noise ratio
improvement (SNRI) were monitored and rebiased. Occasional rebiases were
necessary because of changing temperature conditions and mechanical vibrations
of the cavity and receiver chain. Therefore, the JTWPA pump frequency and the
local oscillator frequency were adjusted as the frequency shifted. A JTWPA
SNRI greater than 9 dB was maintained over the course of 2 weeks, as well as a
measured power gain between 12 and 17 dB over the same time frame, at the
frequency of the $TM_{\mathrm{010}}$ mode. This power gain was evident over
the range of several GHz. Figure 3 shows the gain as measured over a 3-GHz
range, with the characteristic peaks and troughs.

Figure 2: Reflection measurements, acquired by routing power from the cavity
bypass line through the output line, with two different pump frequencies
applied to the JTWPA, shown in solid (blue), and dashed (orange) curves. The
gain peaks are spaced roughly 38 MHz apart. The JTWPA was biased such that the
2-MHz power spectrum was centered on a region with maximum gain. Figure 3:
Wideband power gain of the JTWPA from 4-7 GHz, measured during data-taking
operations.

To perform an axion search with the sidecar cavity, it is necessary to
understand the system noise temperature, which can be written as

$T_{\mathrm{sys}}=\frac{T_{\mathrm{HFET}}}{{\epsilon}\,\mathrm{SNRI}}.$ (1)

Here, $T_{\mathrm{HFET}}$ is the noise temperature of the HFET and downstream
warm receiver components, $\epsilon{\textless}1$ is the transmission
efficiency between the JTWPA and the cavity, and $\mathrm{SNRI}$ is the
signal-to-noise ratio improvement when the JTWPA is turned on. The value of
$T_{\mathrm{HFET}}$ was measured during a period in which the experiment was
warmed from about 0.2 K to 0.6 K. The hot load, as shown in Fig. 1, was not
used due to an unreliable RF switch. In this scenario, the power measured off
cavity resonance is defined by

$P=G_{\text{HFET}}k_{\text{B}}b\left[T_{\text{attn}}\epsilon+T_{\text{cav}}(1-\epsilon)+T_{\text{HFET}}\right],$
(2)

where $T_{\mathrm{attn}}$ is the physical temperature from the final stage
attenuator, labelled as $\mathrm{A_{1}}$ in Fig. 1, on the sidecar bypass RF
line, $T_{\mathrm{cav}}$ is the physical temperature of the lines leading from
the cavity, and $b$ is the bandwidth.

Our measurement of the receiver noise was acquired during the period over
which the experiment was being warmed. Because the final stage attenuator is
thermally sunk to the main cavity along with the sidecar cavity, Eq. 2 then
reduces to

$P=G_{\text{HFET}}k_{\text{B}}b\left[T_{\text{attn}}+T_{\text{HFET}}\right].$
(3)

During the warm-up, power from the receiver was sampled and the temperature of
the attenuator was measured. A plot of the power as a function of the
attenuator temperature, and the associated fit to the data, can be seen in
Fig. 4. The fit parameter, $T_{\mathrm{HFET}}$, was measured to be 3.7$\pm$0.2
K at a frequency of 4.798 GHz.

The system noise was computed using Eq. 1, with this value for
$T_{\mathrm{HFET}}$. The SNRI was computed by smoothing and interpolating
between measurements of the JTWPA SNRI before and after sampling power from
the cavity to acquire a reasonable value over the time duration of data
acquisition.

We measured the attenuation of all cables between the cavity and the JTWPA in
liquid nitrogen. We then scaled the measured attenuation to 100 mK, the
temperature stage of the quantum amplifier package, using the temperature
scaling ratios described in the cable specifications sheet Key . The value of
$\epsilon$, which we measured to be 0.40$\pm$0.04 in linear units, was then
used in the computation of the system noise. This measurement includes
attenuation from the JTWPA, which has an associated insertion loss at 4.798
GHz of -3.0$\pm$0.3 dB Macklin  _et al._ (2015). Assuming the average JTWPA
SNRI of 11.25 dB, the system noise was measured to be 925$\pm$80 mK, with the
dominant uncertainty coming from the uncertainty in the attenuation from the
cavity to the JTWPA. This value is within the expected range of the system
noise using a JTWPA Simbierowicz  _et al._ (2021).

Figure 4: Mean digitized power of sidecar as a function of temperature of the
final stage attenuator for two frequencies off-resonance: 4.6 GHz (left) and
4.9 GHz (right). The actual temperature sensor for the $x$-axis was mounted to
the quantum amplifier package, which is thermally connected to the final stage
attenuator on the bypass line. The gain of the second-stage and downstream
electronics and their associated noise temperature were extracted from the
fit. Each data point consisted of an integration time of 10 s. Data are not
evenly spaced in temperature-space because of non-linear warming of the ADMX
insert. To compute this receiver noise temperature at the resonant frequency,
we interpolated between these two values of the receiver noise temperature.
The receiver noise temperature, along with the JTWPA SNRI, were used to
compute the system noise. The 1$\sigma$ error bounds come from taking the
square root of the diagonal of the covariance matrix.

Raw data from the sidecar digitizer consisted of 2-MHz wide power spectra with
20,000 bins, each 100 Hz wide. The first and last 5,000 bins were removed from
all spectra because the cavity Lorentzian shape was distorted due to intrinsic
distortions in the gain of the JTWPA. Bins at the edge of the Lorentzian are
least valuable to the data as they exist far from resonance. In a truly
broadband experiment that was not limited by the bandwidth of the cavity and
receiver electronics, these distortions could be removed, enabling a wide
frequency range to be probed. Individual power spectra from sidecar were
processed by first dividing by the measured warm electronics transfer function
and then applying a Padé approximant filter to remove the transfer function of
the cold receiver chain. To acquire the warm electronics transfer function, we
terminated the input of the warm receiver, then sampled the power from the
warm receiver. An example of the Padé approximant fit (accounting for the warm
receiver shape) to the raw spectrum can be seen in Fig. 5. Removing the
transfer functions of the warm and cold electronics resulted in power spectra
having the expected Gaussian noise distribution. See Fig. 6. A strong peak
between 4797.5 and 4797.75 MHz was determined to be due to radio-frequency
interference by examining scans far from cavity resonance and observing that
the peak occupied the same IF bins of the power spectrum regardless of the
cavity frequency. A mask was created to exclude these frequencies from the
final limit plot. The total bandwidth that was excluded from any individual
raw spectrum due to the mask corresponded to 120 kHz.

Figure 5: Padé approximant fit to an individual raw spectrum for sidecar. A
strong peak in the spectrum was determined to be interference coming from IF
electronics, as it appeared always in the same bins, even in power spectra
acquired far off cavity resonance. It was removed by masking bins 2800-4000,
explaining the gap in the data, above. Figure 6: Noise distribution (blue) and
Gaussian fit (red) of the data after removing the receiver shape for sidecar.
After background subtraction, the distribution of powers measured is well
represented by a Gaussian, as expected for white noise. The mean of the fit
was computed to be -0.007, with a sigma of 0.990.

Axion candidates were defined as any power excesses above 3$\sigma$
fluctuation above the average power level from the flattened spectrum. Two
candidates were flagged, but were not identified as axion-like due to lack of
persistence between scans. The systematic uncertainties are shown in Table 1.
The dominant systematic uncertainties come from the attenuation in the RF
cable that runs from the cavity to the JTWPA and the SNRI fit.

Source | Fractional uncertainty
---|---
$B^{2}VC_{\text{010}}$ | 0.04
$Q$ | 0.2
Antenna Coupling | 0.01
$T_{\mathrm{HFET}}/\epsilon$ | 0.11
SNRI measurement | 0.11
Total on power | 0.26
Table 1: Dominant sources of systematic uncertainty. The uncertainties were
added in quadrature to attain the uncertainty on the total axion power from
the cavity, shown in the bottom row. For the first entry, $B$ is the magnetic
field, $V$ is the volume, and $C_{\text{010}}$ is the form factor. The last
row shows the total uncertainty on the axion power from the cavity. Figure 7:
Limit plot from the sidecar data with a JTWPA, assuming an axion density of
0.45 GeV/cc, equal to the total dark matter density at the Earth. The two
lines represent DFSZ and KSVZ models.

A limit was set around an axion mass of 19.84 $\mathrm{\SIUnitSymbolMicro}$eV,
shown in Fig. 7, which excludes axion dark matter with an assumption that
axions constitute all of the dark matter, taken to have a density of 0.45
GeV/cc. Optimal filtering was used with the assumption of a Maxwell-Boltzmann
line shape Turner (1990). The shape of the excluded region shown in Fig. 7
arises from the fact that the frequency of the $\mathrm{TM_{010}}$ mode
shifted during the course of data-taking. These frequency shifts corresponded
to filling of the liquid helium reservoir, which is known to induce mechanical
vibrations in the insert. The mechanical vibrations cause small shifts in the
positions of the antenna and tuning rod. These vibrations resulted in slight
shifts in the resonant frequency of the cavity, despite the fact that the
cavity was not intentionally tuned during the period of this data acquisition.
The resonant frequency of the cavity drifted from about 4.7975 to 4.7990 GHz
throughout the course of data-taking.

We have demonstrated the sustained operation of a JTWPA for axion searches.
While the data analyzed in this run were acquired in about 2 weeks, the JTWPA
provided reasonable gain for a period of several months during engineering
runs prior to data acquisition. Furthermore, an incident occurred in which an
accidental increase in the magnet current led to a magnetic field of about
0.41 T in the vicinity of the JTWPA. As a result, the JTWPA was temporarily
disabled, but its operation was fully restored after the insert temperature
was raised above the critical temperature of niobium. We note that a similar
recovery is expected for both the MSA and JPA. Such resilience against
inadvertently applied fields bodes well for future axion experiments. This
work sets the stage for broadband axion searches, in which we take full
advantage of the ability of the JTWPA to operate with low noise temperature
and high gain over a wider bandwidth. Future searches that aim to use a JTWPA
should continue to improve the means by which we measure the system noise
temperature, and consider including an RF line to bypass the JTWPA. This
bypass would provide some ability to isolate the JTWPA and therefore study its
impacts on the receiver chain.

##  I Acknowledgements

This work was supported by the U.S. Department of Energy through Grants No.
DE-SC0009800, No. DESC0009723, No. DE-SC0010296, No. DE-SC0010280, No. DE-
SC0011665, No. DEFG02-97ER41029, No. DEFG02-96ER40956, No. DEAC52-07NA27344,
No. DEC03-76SF00098, and No. DE-SC0017987. Fermilab is a U.S. Department of
Energy, Office of Science, HEP User Facility. Fermilab is managed by Fermi
Research Alliance, LLC (FRA), acting under Contract No. DE-AC02-07CH11359.
Additional support was provided by the Heising-Simons Foundation and by the
Lawrence Livermore National Laboratory and Pacific Northwest National
Laboratory LDRD offices. LLNL Release No. LLNL-JRNL-825283. UWA was funded by
the ARC Centre of Excellence for Engineered Quantum Systems, CE170100009, and
Dark Matter Particle Physics, CE200100008. Ben McAllister is funded by the
Forrest Research Foundation. Tatsumi Nitta is supported by JSPS Overseas
Research Fellowships No. 202060305.

## References

  * Ade  _et al._ (2014) P. A. R. Ade _et al._ (Planck), Astron. Astrophys. 571, A16 (2014).
  * Rubin  _et al._ (1980) V. C. Rubin, J. Ford, W. K., and N. Thonnard, Astrophys. J. 238, 471 (1980).
  * Kaiser and Squires (1993) N. Kaiser and G. Squires, The Astrophysical Journal 404, 441 (1993).
  * Robertson  _et al._ (2016) A. Robertson, R. Massey, and V. Eke, Monthly Notices of the Royal Astronomical Society , stw2670 (2016).
  * Preskill  _et al._ (1983) J. Preskill, M. B. Wise, and F. Wilczek, Phys.Lett. B120, 127 (1983).
  * Abbott and Sikivie (1983) L. Abbott and P. Sikivie, Phys.Lett. B120, 133 (1983).
  * Dine and Fischler (1983) M. Dine and W. Fischler, Phys. Lett. B 120, 137 (1983).
  * Peccei and Quinn (1977a) R. D. Peccei and H. R. Quinn, Phys. Rev. Lett. 38, 1440 (1977a).
  * Peccei and Quinn (1977b) R. D. Peccei and H. R. Quinn, Phys. Rev. D 16, 1791 (1977b).
  * Weinberg (1978) S. Weinberg, Phys. Rev. Lett. 40, 223 (1978).
  * Wilczek (1978) F. Wilczek, Phys. Rev. Lett. 40, 279 (1978).
  * Sikivie (1985) P. Sikivie, Phys. Rev. D 32, 2988 (1985).
  * Sikivie (1983) P. Sikivie, Phys. Rev. Lett. 51, 1415 (1983).
  * Dine  _et al._ (1981) M. Dine, W. Fischler, and M. Srednicki, Phys.Lett. 104B, 199 (1981).
  * Zhitnitskii (1980) A. P. Zhitnitskii, Sov. J. Nucl. Phys. (Engl. Transl.); (United States) 31:2 (1980).
  * Bartram  _et al._ (2021) C. Bartram, T. Braine, R. Cervantes, N. Crisosto, N. Du, G. Leum, L. Rosenberg, G. Rybka, J. Yang, D. Bowring, _et al._ , Physical Review D 103, 032002 (2021).
  * Kim (1979) J. E. Kim, Phys. Rev. Lett. 43, 103 (1979).
  * Shifman  _et al._ (1980) M. A. Shifman, A. Vainshtein, and V. I. Zakharov, Nucl.Phys. B166, 493 (1980).
  * Zhong  _et al._ (2018) L. Zhong, S. Al Kenany, K. M. Backes, B. M. Brubaker, S. B. Cahn, G. Carosi, Y. V. Gurevich, W. F. Kindel, S. K. Lamoreaux, K. W. Lehnert, S. M. Lewis, M. Malnou, R. H. Maruyama, D. A. Palken, N. M. Rapidis, J. R. Root, M. Simanovskaia, T. M. Shokair, D. H. Speller, I. Urdinaran, and K. A. van Bibber, Phys. Rev. D 97, 092001 (2018).
  * Brubaker  _et al._ (2017a) B. M. Brubaker, L. Zhong, S. K. Lamoreaux, K. W. Lehnert, and K. A. van Bibber, Phys. Rev. D 96, 123008 (2017a).
  * Brubaker  _et al._ (2017b) B. M. Brubaker, L. Zhong, Y. V. Gurevich, S. B. Cahn, S. K. Lamoreaux, M. Simanovskaia, J. R. Root, S. M. Lewis, S. Al Kenany, K. M. Backes, I. Urdinaran, N. M. Rapidis, T. M. Shokair, K. A. van Bibber, D. A. Palken, M. Malnou, W. F. Kindel, M. A. Anil, K. W. Lehnert, and G. Carosi, Phys. Rev. Lett. 118, 061302 (2017b).
  * Hagmann  _et al._ (1998) C. Hagmann, D. Kinion, W. Stoeffl, K. van Bibber, E. Daw, H. Peng, L. J. Rosenberg, J. LaVeigne, P. Sikivie, N. S. Sullivan, D. B. Tanner, F. Nezrick, M. S. Turner, D. M. Moltz, J. Powell, and N. A. Golubev, Phys. Rev. Lett. 80, 2043 (1998).
  * Asztalos  _et al._ (2004) S. J. Asztalos, R. F. Bradley, L. Duffy, C. Hagmann, D. Kinion, D. M. Moltz, L. J. Rosenberg, P. Sikivie, W. Stoeffl, N. S. Sullivan, D. B. Tanner, K. van Bibber, and D. B. Yu, Phys. Rev. D 69, 011101(R) (2004).
  * Asztalos  _et al._ (2002) S. J. Asztalos, E. Daw, H. Peng, L. J. Rosenberg, D. B. Yu, C. Hagmann, D. Kinion, W. Stoeffl, K. van Bibber, J. LaVeigne, P. Sikivie, N. S. Sullivan, D. B. Tanner, F. Nezrick, and D. M. Moltz, The Astrophysical Journal Letters 571, L27 (2002).
  * Mück  _et al._ (1998) M. Mück, M.-O. André, J. Clarke, J. Gail, and C. Heiden, Applied Physics Letters 72, 2885 (1998).
  * Hatridge  _et al._ (2011) M. Hatridge, R. Vijay, D. H. Slichter, J. Clarke, and I. Siddiqi, Phys. Rev. B 83, 134501 (2011).
  * Boutin  _et al._ (2017) S. Boutin, D. M. Toyli, A. V. Venkatramani, A. W. Eddins, I. Siddiqi, and A. Blais, Phys. Rev. Applied 8, 054030 (2017).
  * Aumentado (2020) J. Aumentado, IEEE Microwave Magazine 21, 45 (2020).
  * Arvanitaki  _et al._ (2020) A. Arvanitaki, S. Dimopoulos, M. Galanis, L. Lehner, J. O. Thompson, and K. Van Tilburg, Physical Review D 101, 083014 (2020).
  * Rosenberg (2015) L. J. Rosenberg,  _Final Report for the ADMX Phase 2a Project at the University of Washington_ , Tech. Rep. (Univ. of Washington, Seattle, WA (United States), 2015).
  * Boutan (2017) C. Boutan,  _A Piezoelectrically Tuned RF-Cavity Search for Dark Matter Axions_ , Ph.D. thesis, University of Washington (2017).
  * (32) “Low noise factory hfet amplifier,” https://www.lownoisefactory.com/files/7015/7825/6000/LNF-LNC4_8C.pdf, accessed: 2021-09-23.
  * O’Brien  _et al._ (2014) K. O’Brien, C. Macklin, I. Siddiqi, and X. Zhang, Phys. Rev. Lett. 113, 157001 (2014).
  * (34) “Keycom,” https://www.keycom.co.jp/eproducts/upj/upj2/page.htm, accessed: 2021-09-26.
  * Macklin  _et al._ (2015) C. Macklin, K. O’brien, D. Hover, M. Schwartz, V. Bolkhovsky, X. Zhang, W. Oliver, and I. Siddiqi, Science 350, 307 (2015).
  * Simbierowicz  _et al._ (2021) S. Simbierowicz, V. Vesterinen, J. Milem, A. Lintunen, M. Oksanen, L. Roschier, L. Grönberg, J. Hassel, D. Gunnarsson, and R. E. Lake, Review of Scientific Instruments 92, 034708 (2021).
  * Turner (1990) M. S. Turner, Physical Review D 42, 3572 (1990).