# Genetic Instruct: Scaling up Synthetic Generation of
Coding Instructions for Large Language Models

Somshubra Majumdar*, Vahid Noroozi , Sean Narenthiran, Aleksander Ficek,
Jagadeesh Balam, Boris Ginsburg
NVIDIA
Santa Clara, CA 15213, USA
<EMAIL_ADDRESS>
Equal contribution

###### Abstract

Large Language Models (LLMs) rely on instruction samples for alignment, but
creating these datasets poses challenges, particularly in expert-dependent
tasks like coding, which can be cost-prohibitive. One approach to mitigate
these challenges is synthesizing data using another LLM. In this paper, we
introduce a scalable method for generating synthetic instructions to enhance
the code generation capability of LLMs. The proposed algorithm, Genetic-
Instruct, mimics evolutionary processes, utilizing self-instruction to create
numerous synthetic samples from a limited number of seeds. Genetic-Instruct is
designed for efficient scaling of the generation process. Fine-tuning multiple
coding LLMs with the synthetic samples demonstrates a significant improvement
in their code generation accuracy compared to the baselines.

spacing=nonfrench

Genetic Instruct: Scaling up Synthetic Generation of
Coding Instructions for Large Language Models

Somshubra Majumdar*, Vahid Noroozi ††thanks: Equal contribution, Sean
Narenthiran, Aleksander Ficek, Jagadeesh Balam, Boris Ginsburg NVIDIA Santa
Clara, CA 15213, USA<EMAIL_ADDRESS>

##  1 Introduction

Large Language Models (LLMs) have demonstrated significant advancements in
programming and coding problems and can be readily used as code assistant
models (Liang et al., 2024). To fully harness their capabilities, a process
known as alignment (Ouyang et al., 2022) is necessary, which relies on paired
instruction-solution examples to guide the model’s behavior and responses. One
challenge in alignment is creating diverse and complex instructions, which can
be costly, particularly in domains like coding that require specialized
expertise. One strategy to address the scarcity of instructional examples for
training is to generate synthetic instructions using another LLM. Research has
shown that synthetic instructions can be effective not only for coding
applications (Luo et al., 2024; Rozière et al., 2024; Yu et al., 2024) but
also for general tasks (Wang et al., 2023; Honovich et al., 2023; Xu et al.,
2024). This approach can be seen as a form of knowledge distillation or
transfer from a more capable LLM to another.

Figure 1: Genetic-Instruct Process. A small seed population is operated on by
an Instructor-LLM to perform crossover/mutation operations. A Coder-LLM then
generates an appropriate code solution, which is judged for correctness by a
Judge-LLM. Once a required population size is reached, the samples are de-
duplicated and form the final population.

In this paper, we introduce a scalable algorithm named Genetic-Instruct,
illustrated in Figure 1. Drawing inspiration from evolutionary algorithms,
Genetic-Instruct is designed to generate synthetic instructions for coding
problems. The algorithm begins with a small set of high-quality initial
instruction samples and employs some LLMs to generate instructions and their
corresponding code outputs through two evolutionary operations: crossover and
mutation. The crossover operation is adapted from self-instruct (Wang et al.,
2023), which prompts an LLM to generate new examples based on the context of a
few-shot examples. The LLM generates new samples using seed instructions as a
foundation, akin to a crossover operation. In the mutation operation, an LLM
is given an instruction and asked to evolve it into another instruction based
on predefined rules (Luo et al., 2024). Subsequently, another LLM proficient
in coding generates the output or solution to the instruction, typically
comprising coding solutions.

We also introduce a fitness function that employs an LLM to assess the
correctness and quality of the generated instruction and its solution. Samples
that pass these evaluations and checks are added to the population pool, and
the evolutionary process continues until the target population size is
reached. The entire pipeline is designed for efficient parallel execution with
multiple colonies of populations. We extend previous work by providing a
solution that enables existing operators with scalability as needed.

We generate a large dataset of synthetic coding instructions using the
proposed Genetic-Instruct algorithm, starting from a limited set of coding
problems. We then assessed the effectiveness of this dataset by training
multiple open-source software (OSS) LLMs. Our work is dedicated to enhancing
OSS models, and we strictly avoid using any closed-source data or models. The
coding models trained with our synthetic dataset were evaluated across a range
of coding benchmarks, showcasing the superior performance of our approach
compared to other algorithms designed for generating synthetic coding
instructions.

##  2 Previous Works

Synthetic data generation has emerged as a viable alternative to the time-
intensive and expensive process of collecting data from human interactions
with LLM systems. One notable method for automated data generation is Self-
Instruct (Wang et al., 2023), which employs a pre-trained base LLM to generate
instruction-output pairs from a small seed dataset. The resulting dataset is
then used to fine-tune the base LLM, resulting in an Instruction-tuned LLM.
This work is not focused on coding problems and generate instruction for
general tasks.

A limitation of Self-Instruct is that the synthesized samples are often simple
and lack diversity, failing to generate challenging instructions that require
additional steps to arrive at the solution. To address this, Xu et al. (2024)
introduced Evol-Instruct, an enhancement of Self-Instruct that integrates the
concept of mutation of instructions to create more complex tasks. By
introducing meta-instructions that increase the reasoning depth, add
constraints to solutions, or evolve instructions with similar concepts (in-
breadth evolution), Evol-Instruct produces a diverse set of complex
instructions that significantly improve the performance of the instruction-
tuned model. This concept was further adapted for the coding domain by Luo et
al. (2024) in a variant called WizardCoder, where models trained on these
datasets exhibit enhanced coding capabilities. Our work extends WizardCoder
mutation by supporting mutating multiple generations of the same sample and
contrarily open-sourcing our work.

While Self-Instruct and Evol-Instruct primarily focus on synthesizing code
generation-style instruction-solution pairs, Yu et al. (2024) introduced the
CodeOcean dataset, which adopts a novel approach to synthesizing complex tasks
for code models. The models trained on CodeOcean, referred to as WaveCoder,
are adept at code completion, documentation, fixing, and translation. Contrary
to previous methods that construct instructions based on in-context seed
instructions, the CodeOcean dataset leverages code snippets from existing
codebases as context. The LLM is then tasked with constructing a valid
instruction, one of the four tasks, followed by an appropriate solution,
leading to a more diverse and challenging set of instructions. Other works
such as PromptBreeder (Fernando et al., 2023) combine mutation and crossover
but at the prompt level and then operating on the provided instruction to
create a new solution. Our work mutates instructions by performing Wizard-
Coder (and potentially Wave-Coder) on the given instruction as opposed to
swapping a mutated prompt for a previous task prompt.

Holistically, our work does not replace previous methods but instead utilizes
them as operators in a pipeline to automatically construct novel data with
minimal human involvement and trivial amounts of seed data. This is due to
features of massive parallel scaling, operator neutrality and compound
mutations.

##  3 Genetic-Instruct

We introduce Genetic-Instruct, an algorithm inspired by the population-based
evolutionary approach of genetic algorithms. This algorithm employs two
primary evolutionary operations of mutation and crossover to evolve and
generate new generations from an initial population. The initial population,
termed Generation 0, comprises a limited set of high-quality seed
instructions. These seed instructions undergo a series of evolutionary
operations, mainly mutation, crossover and selection, to transform them into
new instructions. In this study, the operations of mutation, crossover, and
selection are executed by leveraging large language models (LLMs) and
enhancing their output with in-context learning (Brown et al., 2020).

We designate the LLM responsible for instruction generation as the Instructor-
LLM. Upon generating a new instruction, another LLM, referred to as the Coder-
LLM, is tasked with producing the code corresponding to this new instruction.
The Instructor-LLM and Coder-LLM can either be distinct models or the same
model but with different prompts. The newly generated instruction and its
associated code constitute a new coding instruction, which can be utilized for
training. However, there may be instances where the generated code does not
fully address the provided question, or the question itself may be poorly
formulated. To assess the quality of the new coding instruction, we employ
another LLM, termed the Judge-LLM, to evaluate the correctness of the
instruction and its code. If a sample passes this quality assessment, it is
added to the pool of instructions and may be selected as the seed instruction
for the next batch of synthesized samples.

The entire process is iterated multiple times to synthesize samples until the
desired population size is achieved. Subsequently, a deduplication process is
applied to minimize duplicate instructions in the current dataset. This
resulting population is then labeled as a generation, and the entire pipeline
can be repeated by considering this generation as the initial population for
the next generation. The complete pipeline is illustrated in Figure 1, and the
procedure for generating one generation is detailed in Algorithm 9.

###  3.1 Selection Process

Initially, from the instruction set of the initial population ($S$) (also
called the seed dataset), we randomly select a batch of instructions
($S^{\prime}$) with replacement. This batch is then subjected to either
mutation or crossover steps as detailed below, with the generations denoted by
$S^{\prime}_{gen}$. In subsequent iterations, we augment the initial
population with the generations from the previous iteration, such that
$S:=S\cup S^{\prime}_{gen}$. This process continues until a sufficient number
of samples have been synthesized for the current generation.

###  3.2 Mutation Operation

The mutation operation is inspired by an adaptation of the Evol-Instruct
algorithm, as devised by Xu et al. (2024), and further extended by WizardCoder
(Luo et al., 2024) to facilitate instruction generation for code models. Evol-
Instruct evolves an instruction into another using an LLM based on predefined
tasks. For a sample selected for mutation, we randomly choose one of the five
tasks defined and apply the mutation to generate a new instruction. We employ
the same five tasks introduced by Luo et al. (2024), with minor prompt
modifications to suit our Instructor-LLM. Details on the mutation prompts are
provided in Figure 2.

###  3.3 Crossover Operation

The crossover operation in Genetic-Instruct is influenced by concepts
introduced in Self-Instruct (Wang et al., 2023) and Unnatural Instructions
(Honovich et al., 2023). It combines multiple genes (instructions) and employs
the Instructor-LLM to generate new populations from the provided few-shot
example instructions. To enhance the efficiency of the crossover operation, we
provide multiple seed instructions and request the model to generate multiple
diverse new instructions based on the provided examples in a single
Instructor-LLM call. Simple post-processing, such as rejecting all samples
that don’t pass the Abstract Syntax Tree checks, is then applied to filter out
incorrect instructions. The prompt for the crossover operation is depicted in
Figure 3.

###  3.4 Code Generation

After the Instructor-LLM generates a batch of new instructions, they are
passed to the Coder-LLM to generate the corresponding code solutions. The
Coder-LLM should be proficient in coding tasks to ensure the generation of
high-quality solutions. However, some generated code may not be parseable or
compilable. Therefore, we filter out solutions whose code segments cannot be
parsed by the corresponding language’s parser/compiler. While determining the
correctness of code by execution is the ideal case, it is challenging due to
various factors, such as language constraints, missing dependencies, or having
to integrate the current solution into a much larger codebase that may not be
available in its entirety. The prompt used in this step is illustrated in
Figure 4.

###  3.5 Fitness Function

In genetic algorithms, once candidate solutions are produced, they are scored
using a fitness function in order to discard candidates that score lower
compared to others. Similarly, we employ a Judge-LLM to assign a binary score
indicating whether a candidate code solution meets the requirements of the
corresponding instruction. The Judge-LLM is provided with an instruction and
its code solution to determine the correctness of the instruction and its
corresponding solution. To enhance the Judge-LLM’s performance, we employ
techniques such as in-context learning with few-shot examples and Chain-of-
Thought (Wei et al., 2022) prompting to facilitate reasoning before making a
binary decision. The prompt for the Judge-LLM is depicted in Figure 5.

###  3.6 Scaling Up the Process

An advantage of genetic algorithms is their inherent capacity for
parallelization. When utilizing computationally intensive LLMs for sample
generation, it is crucial to exploit this parallel architecture. We
efficiently compute multiple colonies of populations in parallel processes or
compute nodes and synchronize them periodically. These colonies are evolved
and populated independently, starting from the same seed population. Upon
reaching the desired size, the colonies are merged into a single population,
where deduplication is performed to obtain a unified dataset, termed one
generation. We use only the instruction itself as the deduplication criterion
to eliminate redundancies across different colonies. Deduplication is
performed using the MinHash + LSH algorithm, implemented in text-dedup
library. This generation serves as the shared seed data for the next
generation, a process we refer to as inter-node parallelization.

Within each colony, using large batch sizes and selecting a single task for
each algorithm iteration leads to redundancy in the generated instructions,
and increases duplication. To address this, we employ sub-processes to compute
multiple sub-colonies within each colony, each possessing a small batch size
but capable of selecting diverse tasks. This approach maintains similar
computational and memory requirements as a single large batch of single-task
samples while enhancing the diversity of generated solutions, thereby reducing
duplication. This process is termed intra-node parallelization.

##  4 Experiments

In all experiments, the models are assessed using four benchmark datasets:
HumanEval (Chen et al., 2021), MBPP (Odena et al., 2021), HumanEval+, and
MBPP+ (Liu et al., 2023). The MBPP+ and HumanEval+ datasets from the EvalPlus
benchmark are extensions of the MBPP and HumanEval test sets, respectively.
These extensions are enriched with additional test cases to ensure the
correctness and accuracy of the generated code. All coding evaluations are
conducted using greedy decoding. In all reported results, an asterisk (*)
indicates that we computed the results ourselves, as the original works did
not report these numbers.

We use the Tiger-Leetcode collection111TigerResearch/tigerbot-kaggle-
leetcodesolutions-en-2k as the initial population in most of the experiments.
This collection serves as the seed dataset for the generation process and
contains interview-style coding questions and solutions.

In all the experiments, we employed the same generation model as Instructor-
LLM, Coder-LLM, and Judge-LLM. Since we are evaluating our approach solely on
Python coding benchmarks, we explicitly limit our generated solutions to
Python by requesting the models to generate only Python code. After the code
is generated by Coder-LLM, we verify its syntactical correctness using
Python’s ast package, regardless of its executability.

###  4.1 Experimental Settings

Table 1: Comparison of models fine-tuned with Genetic-Instruct with other
baselines. Cells marked with $\times$ indicate result could not be calculated
since the model is not available. The column labelled as OpenAI indicates
whether the synthetic data is generated with any of the OpenAI’s models like
ChatGPT. Best results for each base model are bolded. Cells marked with ✝ are
evaluated on the original 500 un-sanitized MBPP dataset and aren’t directly
comparable to the MBPP’s result calculated by EvalPlus.

Size | Base Model | Generation Algorithm | OpenAI | HumanEval | HumanEval+ | MBPP | MBPP+
---|---|---|---|---|---|---|---
47B | Mixtral-8x7B | - | N | 40.2 | 34.3 | 60.7 | 50.1
- | GPT-4-Turbo | - | Y | 85.4 | 81.7 | 83.5 | 70.7
7B | DeepSeek-Base | WaveCoder | Y | 64.0 | $\times$ | 62.8 ✝ | $\times$
7B | CodeLlama-Python | WaveCoder | Y | 48.1 | $\times$ | 47.2 | $\times$
7B | CodeLlama-Python | WizardCoder | Y | 55.5 | 45.9 | 59.9 | 49.1
34B | CodeLlama-Python | WizardCoder | Y | 73.2 | 64.6 | 73.2 | 59.9
7B | DeepSeek-Base | - | N | 47.4 | 39.5 | 60.6 | 52.1
Genetic-Instruct | N | 57.9 | 51.8 | 70.4 | 59.1
2-9 | CodeLlama-Python | - | N | 38.4 | 35.2 $*$ | 47.6 | 40.5 $*$
| CodeLlama-Instruct | N | 34.8 | 30.2 $*$ | 44.4 | 38.7 $*$
| Genetic-Instruct | N | 50.6 | 47.6 | 65.2 | 55.9
34B | CodeLlama-Python | - | N | 53.7 | 46.2 $*$ | 56.2 | 49.3 $*$
CodeLlama-Instruct | N | 41.5 | 33.7 $*$ | 57.0 | 50.5 $*$
Genetic-Instruct | N | 59.1 | 53.7 | 67.7 | 57.1
70B | CodeLlama-Python | - | N | 57.3 | 51.3 $*$ | 65.6 | 53.4 $*$
CodeLlama-Instruct | N | 67.8 |  58.3 $*$ | 62.2 | 51.8 $*$
Genetic-Instruct | N | 57.9 | 55.5 | 69.2 | 58.6

We trained the base models using the AdamW optimizer (Kingma and Ba, 2015)
with a learning rate of 5e-6 and zero weight decay for two epochs. The
learning rate scheduler employed is CosineAnnealing with 100 warmup steps. The
final model is obtained by averaging the last four checkpoints, which are
saved every 500 steps. All models are trained using tensor parallelism, and
BF16 precision is utilized to accelerate the training process.

For large throughput inference with large effective batch sizes, we employ
vLLM (Kwon et al., 2023) as our inference engine. Nucleus sampling (Holtzman
et al., 2020) is used for decoding, with a temperature of 0.95 for the
Instructor-LLM and a temperature of 1.0 for the Coder-LLM and Judge-LLM. To
enhance GPU utilization and expedite the generation process, we implement both
inter-node and intra-node parallelizations. Specifically, we use 20 colonies
for each generation in inter-node parallelization and 10 sub-colonies per
colony in intra-node parallelization. A maximum sequence length of 1024 is set
for all LLMs to optimize the generation speed and memory allocation.

For the mutation operation, a batch size of 100 per sub-colony is employed. In
the crossover operation, a batch size of 10 per sub-colony is used, as we find
that the model approximately generates 10 unique instructions on average per
sample, resulting in a similar number of generated samples per batch. We
randomly choose to perform either crossover or mutation on each batch. The
Instructor-LLM uses 3-shot samples for in-context learning and is instructed
to generate up to 20 new instructions.

The deduplication process is conducted using the MinHash algorithm from the
TextDedup library222https://github.com/ChenghaoMou/text-dedup with a 5-gram
overlap, 250 permutations, and a threshold of 0.7. The synthetic dataset
generated by our proposed algorithm, Genetic-Instruct, comprises approximately
1.3 million coding instructions, which are divided into different subsets for
evaluation. More details on the generated dataset can be found in Appendices F
and G.

Table 2: Comparing the effectiveness of different operations in the Genetic-
Instruct algorithm. Model CodeLlama-Python 7B has been used as the base model
in these experiments. Bolded cells indicate best score for a given dataset
size.

Population Size | Generation Algorithm | HumanEval | HumanEval+ | MBPP | MBPP+
---|---|---|---|---|---
- | CodeLlama-Python 7B | 38.4 | 35.2 $*$ | 47.6 | 40.5 $*$
| Cross-Over Only | 47.0 | 41.5 | 55.4 | 44.4
2K | Mutation Only | 40.2 | 34.8 | 60.4 | 47.1
| Genetic-Instruct | 46.3 | 41.5 | 56.4 | 45.4
| Cross-Over Only | 48.2 | 45.7 | 59.1 | 51.6
75K | Mutation Only | 43.9 | 39.0 | 63.2 | 51.9
| Genetic-Instruct | 49.4 | 45.7 | 62.7 | 52.9
| Cross-Over Only | 51.7 | 47.6 | 62.2 | 52.9
250K | Mutation Only | 47.6 | 42.1 | 62.7 | 52.1
| Genetic-Instruct | 51.8 | 47.6 | 63.9 | 54.6

Table 3: Ablation study on the effect of the generator model on data
generation quality. We did the evaluations on two base models: DeepSeek Coder
6.7B and 33B. The first three rows are the generation models used for data
synthesis.

Model | Dataset Size | HumanEval | HumanEval+ | MBPP | MBPP+
---|---|---|---|---|---
Mixtral 8x7B | - | 40.2 | 34.3 | 60.7 | 50.1
Mixtral 8x22B | - | 75.3 | 71.1 | 71.9 | 63.6
Llama 3 70B | - | 77.1 | 71.2 | 80.3 | 67.8
DeepSeek Coder 6.7B Base |  | 47.4 | 39.5 | 60.6 | 52.1
Mixtral 8x7B | 900K | 54.9 | 51.8 | 71.4 | 60.4
Mixtral 8x22B | 377K | 66.5 | 62.8 | 72.2 | 60.4
Mixtral 8x22B | 873K | 68.3 | 66.5 | 69.9 | 59.9
Mixtral 8x22B | 1.3M | 70.7 | 67.7 | 75.4 | 63.7
Llama 3 8B | 400K | 63.4 | 57.9 | 70.2 | 58.6
Llama 3 8B | 1.2M | 64.6 | 61.0 | 69.9 | 57.1
Llama 3 70B | 767K | 66.5 | 62.8 | 72.2 | 61.4
Llama 3 70B | 1.8M | 70.7 | 66.5 | 75.4 | 63.2
DeepSeek Coder 33B Base |  | 51.2 | 44.5 | 65.4 | 56.8
Llama 3 70B | 767K | 70.1 | 65.9 | 77.9 | 64.7
Llama 3 70B | 1.8M | 73.8 | 68.9 | 77.7 | 64.7

###  4.2 Performance Evaluation

In this section, we evaluate the effectiveness of our proposed approach to
generate synthetic SFT samples that can enhance the coding capabilities of
LLMs. We used CodeLLama-Python-7B (Rozière et al., 2024) and DeepSeek-
Coder-6.7B (Guo et al., 2024) as the base models and employed Mixtral 8X7B
(Jiang et al., 2024) as the Instructor-LLM, Coder-LLM, and Judge-LLM. The
results, along with the performance of other coding models, are reported in
Table 1. For cases where no generation algorithm is used, the results of the
base models are reported. All results are obtained using greedy decoding with
the pass@1 metric. The base models are fine-tuned with a subset of 405K
samples from all Genetic-Instruct’s generations, while the WizardCoder and
WaveCoder models are trained with datasets of 78K and 20K samples,
respectively.

The results show that training models with Genetic-Instruct significantly
improves performance on all benchmarks compared to the base models. This
demonstrates the effectiveness of the Genetic-Instruct approach in enhancing
the code generation capabilities of LLMs. The results for different sizes of
the CodeLlama-Python model indicate that as the size of the base model
increases, stronger performances can be achieved on most benchmarks. It should
also be noted that HumanEval+ and MBPP+ are more trustworthy benchmarks as
they are more robust to incorrect codes.

The results of GPT-4-Turbo and Mixtral 8x7B are also reported as references to
illustrate the performance gap between OpenAI’s model and Mixtral. We used the
Mixtral 8X7B as the generation model, which is a permissive model, whereas
other approaches like WaveCoder or WizardCoder have used OpenAI’s models,
which are not viable for the open-source community. To ensure a fair
comparison between Genetic-Instruct and other synthetic data generation
methods, we conducted more controlled experiments in Section 4.3.

It should be noted that any involvement of OpenAI’s models (Brown et al.,
2020) in data synthesis has a marked positive effect on the downstream model,
enabling substantial improvements in all models via data distillation. It can
be seen that for WaveCoder and WizardCoder, results on all benchmarks are
significantly improved compared to their base models. However, Genetic-
Instruct surpasses these 7B models on EvalPlus datasets, indicating that such
data distillation is not particularly robust. These results further suggest
that while the choice of generator LLM is crucial in improving the results of
downstream models, larger datasets can also provide more robust results.

An interesting observation of CodeLlama-70B’s results is its ability to
surpass the Mixtral-8x7B model scores on all benchmarks despite using the
relatively weaker Mixtral-8x7B models to generate data. The
CodeLlama-70B-Python model surpasses Mixtral-8x7B’s scores on all benchmarks
and shows further improvements after training on data generated by
Mixtral-8x7B. This indicates that relatively weaker models can generate
sufficiently diverse and challenging data, which, when applied to stronger
domain-specific models, can outperform both. Essentially, we observe that
improvements transfer from a relatively weaker model to a stronger model. This
goes against the general trend of data distillation, which is usually from a
stronger model to a weaker model. This result may enable an iterative self-
improvement loop, such that the model fine-tuned on Genetic-Instruct data can
be used to generate another dataset with higher quality for subsequent
generations of models, similar to Noisy Student Training proposed by Xie et
al. (2020).

###  4.3 Ablation Study

In this ablation study, we evaluate the effectiveness of the operations used
in Genetic-Instruct on the performance of the generated samples. We perform
three data generation ablations, labeled Crossover Only, Mutation Only, and
Genetic-Instruct, which represent data generated using the Genetic-Instruct
algorithm but with one task disabled (either crossover or mutation,
respectively). We generate 250K samples for each dataset, and then uniformly
sample without replacement subsets of 2K and 75K from this total dataset. We
then train a CodeLlama-Python-7B model on each dataset and evaluate their
performance. This ablation provides an objective comparison of Genetic-
Instruct generation with alternatives Self-Instruct (Wang et al., 2023) and
WizardCoder (Luo et al., 2024), which use crossover only and mutation only,
respectively. By controlling variables such as compute, amount of data,
generation model, and base model, we demonstrate a head-to-head comparison of
generation techniques.

As seen in Table 2, crossover provides substantial improvements over the base
model compared to mutation alone; however, both are necessary to achieve the
best results. At just 2K samples, there is no clear winner between the three
approaches, potentially due to the noisy sampling of 2K samples out of 250K,
but all approaches improve on the baseline. As the dataset size increases to
75K and 250K, the combination of both crossover and mutation surpasses either
separately, and this trend continues at larger data scales as measured on the
EvalPlus benchmark.

Table 4: The effect of the population size on the performance of the Genetic-
Instruct.

Dataset Size | HumanEval | HumanEval+ | MBPP | MBPP+
---|---|---|---|---
- | 38.4 | 35.2 $*$ | 47.6 | 40.5 $*$
2K | 46.3 | 41.5 | 56.4 | 45.4
75K | 49.4 | 45.7 | 62.7 | 52.9
250K | 51.8 | 47.6 | 63.9 | 54.6
405K | 50.6 | 47.6 | 65.2 | 55.9
900K | 51.2 | 48.2 | 64.9 | 56.4

###  4.4 Influence of the Generator Model on the Performance

A critical component of the Genetic-Instruct algorithm is the choice of the
Instructor-LLM, Coder-LLM, and Judge-LLM. A powerful Instructor-LLM can
generate more diverse instructions, a stronger Coder-LLM can produce code that
is more likely to be correct and free from syntactic errors, and an effective
Judge-LLM can better discern the quality of samples.

In Table 3, we utilized three different models (Mixtral 8x7B, Mixtral 8x22B,
and Llama-3 70B) to generate a large amount of data in two to three
generations and fine-tuned two base DeepSeek Coder models on them for
comparison. We found that the quality of the final model strongly correlates
with the quality of the data generated by it, and that different dataset sizes
can achieve similar scores when two sufficiently powerful models are compared.
An important observation is that while Mixtral 8x22B data shows strong
results, obtaining similar scores to Llama 3 70B generated data in fewer
samples, it required three generations of data to achieve similar scores to
two generations for Llama 3 70B, thereby requiring significant additional
compute.

Furthermore, when scaling up the dataset, the gap between the original and
EvalPlus benchmarks is substantially reduced, as seen in the CodeLlama-7B
experiments. Similarly, the results on CodeLlama-70B show a marginal
improvement on the original HumanEval dataset but a substantial improvement on
the EvalPlus counterpart. This indicates that despite saturation on the
original benchmarks, training on more synthetic data improves the robustness
of the models on extended benchmarks.

Table 5: Performance of the dataset generated by Genetic-Instruct with
different seed datasets. CodeLlama-Python 7B has been used as the base model
in the experiments.

Seed Dataset | Size | HumanEval | HumanEval+ | MBPP | MBPP+
---|---|---|---|---|---
No SFT | - | 38.4 | 35.2 $*$ | 47.6 | 40.5 $*$
Code Alpaca | 20K | 49.4 | 45.7 | 63.4 | 53.4
Tiger-LeetCode | 2K | 50.6 | 47.6 | 65.2 | 55.9

###  4.5 Effectiveness of Scaling up the Population

In this section, we examine the impact of the size of generated samples on
model performance. We generated 250K samples with Mixtral 8x7b and then
uniformly sampled without replacement subsets of 2K and 75K from it.
Subsequently, we trained a CodeLlama-Python 7B model on each dataset and
evaluated their performance. The results are shown in Table 4. As the dataset
size increases, performance generally improves, indicating that scaling
synthetic data generation with our proposed algorithm can be beneficial. This
trend is more evident in the MBPP+ and HumanEval+ datasets, showing that more
synthetic data generated by Genetic-Instruct can enhance the quality and
correctness of the codes the model generates.

###  4.6 Effect of the Initial Population

In this section, we evaluate the Genetic-Instruct algorithm on different
datasets as the initial population to demonstrate the effectiveness of the
proposed approach. We utilize CodeLlama-7B and two seed datasets: Tiger-
leetcode and Code Alpaca333sahil2801/CodeAlpaca-20k. We generated roughly 400K
samples with Genetic-Instruct using Code Alpaca as the seed set for comparison
against our Tiger-leetcode baseline. Code Alpaca is an instruction dataset of
around 20K samples created by ChatGPT 3.5. Table 5 shows the results of
different seed datasets when used with Genetic-Instruct.

We observe that our method is robust to the seed dataset, capable of
generating instruction data of similar quality for both seed sets, and
significantly surpasses the base model’s results. There is a modest gap
between the two seed datasets, which we attribute to the simplicity of
instructions in Code Alpaca. It is important to note that Genetic-Instruct
does not utilize the code solutions of the seed dataset when creating new
instructions via the Instructor-LLM. We conjecture that the relative
difficulty and complexity of Tiger-Leetcode’s questions lead to a more diverse
synthetic dataset.

Given the similar performance observed between the Code Alpaca seeds generated
by ChatGPT-3.5 and Tiger-leetcode, it appears that the success of the
approaches utilizing OpenAI’s models is primarily due to the model used for
generating instructions and codes, rather than the seeds themselves. This
observation underscores the importance of the quality of LLMs employed in the
generation process.

##  5 Conclusion

In this paper, we introduced Genetic-Instruct, a novel algorithm inspired by
evolutionary principles, designed to generate large-scale synthetic coding
instructions for large language models (LLMs). The algorithm utilizes two
evolutionary operations, crossover and mutation, applied iteratively to a
population pool to produce new instruction samples until the desired size is
reached. Genetic-Instruct is specifically engineered to support parallel
generation, making it a scalable solution for synthetic data creation. We
benchmarked our approach against several baseline methods, demonstrating that
Genetic-Instruct can generate instructions that lead to higher accuracy on
code generation benchmarks. We also showed that increasing the size of the
generated samples improves the performance of the final models. Furthermore,
we evaluated the algorithm’s performance across various initial populations
and base models, showcasing its robustness with respect to seed instructions
and base LLMs.

## References

  * Brown et al. (2020) Tom Brown, Benjamin Mann, Nick Ryder, Melanie Subbiah, Jared D Kaplan, Prafulla Dhariwal, Arvind Neelakantan, Pranav Shyam, Girish Sastry, Amanda Askell, Sandhini Agarwal, Ariel Herbert-Voss, Gretchen Krueger, Tom Henighan, Rewon Child, Aditya Ramesh, Daniel Ziegler, Jeffrey Wu, Clemens Winter, Chris Hesse, Mark Chen, Eric Sigler, Mateusz Litwin, Scott Gray, Benjamin Chess, Jack Clark, Christopher Berner, Sam McCandlish, Alec Radford, Ilya Sutskever, and Dario Amodei. 2020.  Language models are few-shot learners.  In _Advances in Neural Information Processing Systems_ , volume 33, pages 1877–1901. Curran Associates, Inc.
  * Chen et al. (2021) Mark Chen, Jerry Tworek, Heewoo Jun, Qiming Yuan, Henrique Ponde de Oliveira Pinto, Jared Kaplan, Harri Edwards, Yuri Burda, Nicholas Joseph, Greg Brockman, et al. 2021.  Evaluating large language models trained on code.  _arXiv preprint arXiv:2107.03374_.
  * Fernando et al. (2023) Chrisantha Fernando, Dylan Banarse, Henryk Michalewski, Simon Osindero, and Tim Rocktäschel. 2023.  Promptbreeder: Self-referential self-improvement via prompt evolution.  _Preprint_ , arXiv:2309.16797.
  * Guo et al. (2024) Daya Guo, Qihao Zhu, Dejian Yang, Zhenda Xie, Kai Dong, Wentao Zhang, Guanting Chen, Xiao Bi, Y. Wu, Y. K. Li, Fuli Luo, Yingfei Xiong, and Wenfeng Liang. 2024.  Deepseek-coder: When the large language model meets programming – the rise of code intelligence.  _Preprint_ , arXiv:2401.14196.
  * Holtzman et al. (2020) Ari Holtzman, Jan Buys, Li Du, Maxwell Forbes, and Yejin Choi. 2020.  The curious case of neural text degeneration.  In _International Conference on Learning Representations_.
  * Honovich et al. (2023) Or Honovich, Thomas Scialom, Omer Levy, and Timo Schick. 2023.  Unnatural instructions: Tuning language models with (almost) no human labor.  In _Proceedings of the 61st Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers)_ , pages 14409–14428, Toronto, Canada. Association for Computational Linguistics.
  * Jiang et al. (2024) Albert Q. Jiang, Alexandre Sablayrolles, Antoine Roux, Arthur Mensch, Blanche Savary, Chris Bamford, Devendra Singh Chaplot, Diego de las Casas, Emma Bou Hanna, Florian Bressand, Gianna Lengyel, Guillaume Bour, Guillaume Lample, Lélio Renard Lavaud, Lucile Saulnier, Marie-Anne Lachaux, Pierre Stock, Sandeep Subramanian, Sophia Yang, Szymon Antoniak, Teven Le Scao, Théophile Gervet, Thibaut Lavril, Thomas Wang, Timothée Lacroix, and William El Sayed. 2024.  Mixtral of experts.  _Preprint_ , arXiv:2401.04088.
  * Kingma and Ba (2015) Diederik Kingma and Jimmy Ba. 2015.  Adam: A method for stochastic optimization.  In _International Conference on Learning Representations (ICLR)_ , San Diega, CA, USA.
  * Kwon et al. (2023) Woosuk Kwon, Zhuohan Li, Siyuan Zhuang, Ying Sheng, Lianmin Zheng, Cody Hao Yu, Joseph Gonzalez, Hao Zhang, and Ion Stoica. 2023.  Efficient memory management for large language model serving with pagedattention.  In _Proceedings of the 29th Symposium on Operating Systems Principles_ , SOSP ’23, page 611–626, New York, NY, USA. Association for Computing Machinery.
  * Liang et al. (2024) Jenny T Liang, Chenyang Yang, and Brad A Myers. 2024.  A large-scale survey on the usability of ai programming assistants: Successes and challenges.  In _Proceedings of the 46th IEEE/ACM International Conference on Software Engineering_ , pages 1–13.
  * Liu et al. (2023) Jiawei Liu, Chunqiu Steven Xia, Yuyao Wang, and Lingming Zhang. 2023.  Is your code generated by chatGPT really correct? rigorous evaluation of large language models for code generation.  In _Thirty-seventh Conference on Neural Information Processing Systems_.
  * Luo et al. (2024) Ziyang Luo, Can Xu, Pu Zhao, Qingfeng Sun, Xiubo Geng, Wenxiang Hu, Chongyang Tao, Jing Ma, Qingwei Lin, and Daxin Jiang. 2024.  Wizardcoder: Empowering code large language models with evol-instruct.  In _The Twelfth International Conference on Learning Representations_.
  * Odena et al. (2021) Augustus Odena, Charles Sutton, David Martin Dohan, Ellen Jiang, Henryk Michalewski, Jacob Austin, Maarten Paul Bosma, Maxwell Nye, Michael Terry, and Quoc V. Le. 2021.  Program synthesis with large language models.  In _n/a_ , page n/a, n/a.  N/a.
  * Ouyang et al. (2022) Long Ouyang, Jeffrey Wu, Xu Jiang, Diogo Almeida, Carroll Wainwright, Pamela Mishkin, Chong Zhang, Sandhini Agarwal, Katarina Slama, Alex Ray, et al. 2022.  Training language models to follow instructions with human feedback.  _Advances in neural information processing systems_ , 35:27730–27744.
  * Rozière et al. (2024) Baptiste Rozière, Jonas Gehring, Fabian Gloeckle, Sten Sootla, Itai Gat, Xiaoqing Ellen Tan, Yossi Adi, Jingyu Liu, Romain Sauvestre, Tal Remez, Jérémy Rapin, Artyom Kozhevnikov, Ivan Evtimov, Joanna Bitton, Manish Bhatt, Cristian Canton Ferrer, Aaron Grattafiori, Wenhan Xiong, Alexandre Défossez, Jade Copet, Faisal Azhar, Hugo Touvron, Louis Martin, Nicolas Usunier, Thomas Scialom, and Gabriel Synnaeve. 2024.  Code llama: Open foundation models for code.  _Preprint_ , arXiv:2308.12950.
  * Wang et al. (2023) Yizhong Wang, Yeganeh Kordi, Swaroop Mishra, Alisa Liu, Noah A. Smith, Daniel Khashabi, and Hannaneh Hajishirzi. 2023.  Self-instruct: Aligning language models with self-generated instructions.  In _Proceedings of the 61st Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers)_ , pages 13484–13508, Toronto, Canada. Association for Computational Linguistics.
  * Wei et al. (2022) Jason Wei, Xuezhi Wang, Dale Schuurmans, Maarten Bosma, brian ichter, Fei Xia, Ed Chi, Quoc V Le, and Denny Zhou. 2022.  Chain-of-thought prompting elicits reasoning in large language models.  In _Advances in Neural Information Processing Systems_ , volume 35, pages 24824–24837. Curran Associates, Inc.
  * Xie et al. (2020) Qizhe Xie, Minh-Thang Luong, Eduard Hovy, and Quoc V Le. 2020.  Self-training with noisy student improves imagenet classification.  In _Proceedings of the IEEE/CVF conference on computer vision and pattern recognition_ , pages 10687–10698.
  * Xu et al. (2024) Can Xu, Qingfeng Sun, Kai Zheng, Xiubo Geng, Pu Zhao, Jiazhan Feng, Chongyang Tao, Qingwei Lin, and Daxin Jiang. 2024.  WizardLM: Empowering large pre-trained language models to follow complex instructions.  In _The Twelfth International Conference on Learning Representations_.
  * Yu et al. (2024) Zhaojian Yu, Xin Zhang, Ning Shang, Yangyu Huang, Can Xu, Yishujie Zhao, Wenxiang Hu, and Qiufeng Yin. 2024.  Wavecoder: Widespread and versatile enhanced instruction tuning with refined data generation.  _Preprint_ , arXiv:2312.14187.

##  Appendix A Mutation Prompts

Mutation Prompt Please increase the difficulty of the given programming test
question a bit. Do not provide any hints, solutions or outputs. Only one new
instruction is allowed. You can increase the difficulty using, but not limited
to, the following methods: {method} Original Instruction: {instruction} New
Instruction:

Operation: Constraint Rewrite the original instruction, adding new constraints
and requirements, with approximately 10 additional words.

Operation: Deepening Write the original instruction. Then, replace a commonly
used requirement in the programming task with a less common and more specific.

Operation: Erroneous Code Write the original instruction. Then provide a piece
of wrong python code as a reference solution to increase misdirection. Your
wrong reference solution should start with "Reference Solution (Wrong)",
marked in “‘ blocks. Finally, ask to write the correct solution for the
instruction. Do NOT provide the correct solution.

Operation: Reasoning Write the original instruction after the new instruction.
Then, if the original instruction can be solved with only a few logical steps,
please add more reasoning steps after the original instruction. Do NOT provide
any reason or explanation.

Operation: Task Complexity Write the original instruction after the new
instruction. Then propose higher time or space complexity requirements, but
please refrain from doing so frequently.

Figure 2: Prompt template for mutation operation

##  Appendix B Crossover Prompt

Cross-over Prompt You are asked to come up with a set of 20 diverse code
generation task instructions. These task instructions will be given to a GPT
model and we will evaluate the GPT model for completing the instructions. Here
are the requirements: 1\. Try not to repeat the verb for each instruction to
maximize diversity. 2\. The language used for the instruction also should be
diverse. For example, you should combine questions with imperative
instructions. 3\. The type of instructions should be diverse. The list should
include diverse types of programming tasks like open-ended generation,
classification, editing, optimization etc. 4\. A GPT language model should be
able to complete the instruction. 5\. The instructions should be in English.
6\. The instructions should at least 1 to 2 sentences long. Either an
imperative sentence or a question is permitted. 7\. You should generate an
appropriate input to the instruction. The input field should contain a
specific example provided for the instruction. It should involve realistic
data and should not contain simple placeholders. The input should provide
substantial content to make the instruction challenging but should ideally not
exceed 100 words. 8\. Not all instructions require input. For example, when a
instruction asks about some general information, "write a program to load a
file.", it is not necessary to provide a specific context. In this case, we
simply put "⟨noinput⟩" in the input field. 9\. The output should be an
appropriate response to the instruction and the input. 10\. All tasks should
be coding or programming-related. List of 20 tasks:

Few-Shot Examples ### 1\. Instruction: Convert a Binary Search Tree to a
sorted Circular Doubly-Linked List in place. You can think of the left and
right pointers as synonymous to the predecessor and successor pointers in a
doubly-linked list. For a circular doubly linked list, the predecessor of the
first element is the last element, and the successor of the last element is
the first element. We want to do the transformation in place. After the
transformation, the left pointer of the tree node should point to its
predecessor, and the right pointer should point to its successor. You should
return the pointer to the smallest element of the linked list. 1\. Input: root
= 4,2,5,1,3 ### 2\. Instruction: $\cdots$ $\cdots$ ### 4\. Instruction:

Figure 3: Prompt template for the cross-over operation with few-Shot in-
context learning

##  Appendix C Prompt Used for Generating Code with Coder-LLM

Python Code Generation Prompt You are an expert in Python coding. Using only
Python code, write the correct solution that answers the given coding problem.
{instruction} Answer: Figure 4: Prompt template for code Generation with
Coder-LLM

##  Appendix D Fitness Prompt for Judge-LLM

Fitness Prompt You are an expert python programmer. Below is a question and
code solution. Decide if the solution follows the below criteria and give a
final Yes/No, and place it in the ⟨judge⟩⟨/judge⟩ tags. Only look at the
function generated, not any examples/print statements etc. Just the core
logic. Please first briefly describe your reasoning (in less than 30 words),
and then write Decision: \\\boxed{Yes or No} in your last line. Criteria: 1\.
⟨llm-code⟩⟨/llm-code⟩ contains a code solution in any programming language.
2\. If the code was executed with the proper libraries imported and correct
inputs, it would execute without error. 3\. Given the question, the code
solution seems to answer the problem if it was to be used correctly. 4\. The
code solution provides an elegant solution to the problem and doesn’t seem
overly complicated. \n\n\n\n\n

Few-Shot Examples Question: {input} ⟨llm-code⟩ {code} ⟨/llm-code⟩ ⟨judge⟩
{reason} Score: \\\boxed{score}. ⟨/judge⟩ \n\n\n\n\n

Figure 5: Prompt template for code quality judgement with Judge-LLM

##  Appendix E Evaluation Prompt

Evaluation Prompt Template Below is an instruction that describes a task.
Write a response that appropriately completes the request. ### Instruction:
{input} ### Response: Figure 6: Prompt template for code evaluation on
HumanEval and MBPP

##  Appendix F Distribution of the Tasks

Figure 7:  Task distribution of the generated instructions on roughly 600K
samples of Genetic-Instruct out of 900K. Considering the domain of seed data,
we select keywords that correspond to coding challenges. It is not an
exhaustive list of the tasks and they can be overlapping.

##  Appendix G Length Distribution

Figure 8:  Length distribution of the generated instructions, and code outputs
on 900K samples of Genetic-Instruct. Instruction lengths are clipped to 300
words, while code outputs are clipped at 500 words for better visualization.

##  Appendix H Genetic-Instruct Algorithm

Genetic-Instruct Algorithm import numpy as np task_ids = 0: "cross-over", 1:
"mutation" task_prob = [0.5, 0.5] # 50 NUM_SAMPLES, NUM_COLONIES,
NUM_SUBCOLONY = 500_000, 20, 10 def genetic_instruct(seed_instructions:
List[Dict[str, str]]): """ Genetic Instruct for one generation, run on
multiple nodes """ max_samples = NUM_SAMPLES / NUM_COLONIES current_samples =
[] all_samples = [v for v in seed_instructions] while len(current_samples) <=
max_samples: # Parallelize across NUM_SUBCOLONY processes (sub-colonies)
futures = parallelize(genetic_instruct_one_iter, (all_samples,) for _in
range(NUM_SUBCOLONY) )) results = await_all(futures) # wait for all sub-
colonies to finish for result in results: current_samples.extend(result)
all_samples.extend(result) # Deduplicate across all colonies current_samples =
global_deduplicate_across_nodes(current_samples)
write_dataset(current_samples) def
genetic_instruct_one_iter(seed_instructions: List[Dict[str, str]]): """ One
iteration of the Genetic Instruct algorithm Inputs: seed_instructions: List of
(instruction, code) pairs from seed data + previous generations. Outputs: List
of (instruction, code) pairs. """ # (1) Task Selection task_id =
np.random.choice(task_ids.keys(), p=task_prob) # (2) Instruction Generation -
sample a batch of seed data prompts =
encode_inst_generation_prompts(seed_instructions, task_id) new_instructions =
llm.generate(prompts) # (3) Code Generation - generate code for new inst,
filter generations code_prompts = encode_codegen_prompts(new_instructions,
task_id) code_generations = llm.generate(code_prompts) new_inst_code =
filter_code_generations(new_instructions, code_generations) # (4) Code Quality
Judgement - judge inst-code pairs for correctness judge_prompts =
encode_judge_prompts(new_inst_code) code_judgements =
llm.generate(judge_prompts) final_samples = [new_inst for(new_inst, judgement)
in zip(new_instructions, code_judgements) if judgement == ’yes’] return
final_samples Figure 9: Algorithm of genetic-instruct as pseudo-code


# Microwave Technologies in Experiments for Detection of Dark Matter Axions

Jose R. Navarro-Madrid corresponding author<EMAIL_ADDRESS>Department
of Information Technologies and Communications, Universidad Politécnica de
Cartagena, 30202 - Cartagena, Spain  Jose María García-Barceló
<EMAIL_ADDRESS>Max-Planck-Institut für Physik (Werner-Heisenberg-
Institut), Boltzmannstr. 8, 85748 - Garching bei München, Germany  Alejandro
Díaz-Morcillo<EMAIL_ADDRESS>Department of Information Technologies
and Communications, Universidad Politécnica de Cartagena, 30202 - Cartagena,
Spain

Prepared for submission to IEEE Microwave Magazine

##  1 Introduction

What is our universe made of? This is one of the great questions that, at
present, has not been fully resolved. Thanks to advancements in technology
over the past century, scientists have gained access to previously
unattainable insights into the evolution of the universe and its composition,
estimating that the universe would be made up of only a $5$ $\%$ of ordinary
matter (the kind of matter that we can interact with). The rest is composed of
two other unknown elements: dark energy ($68$ $\%$) and dark matter (DM, $27$
$\%$). A distribution of the universe composition is depicted in Figure 1.
Astrophysical observations and studies carried out by F. Zwicky and others led
to the hypothesis of the existence of DM [1]. Its nature remains elusive, as
it does not interact significantly with electromagnetic (EM) waves and
therefore cannot be observed directly. Its presence is inferred through its
gravitational effects on the formation and evolution of large-scale cosmic
structures, such as galaxies and galaxy clusters.

Figure 1: Universe composition graph. The major part is composed of dark
energy which would be present throughout space and produces a pressure that
tends to accelerate the expansion of the universe. The $27$ $\%$ corresponds
to the dark matter, whose existence has been theorized by different
cosmological phenomena. Finally, the rest is conformed by ordinary matter.

First proposed to solve certain problems in quantum chromodynamics [2, 3], the
axion also has some characteristics that make it suitable for composing DM
among other candidates. Its mass would be extremely low, a property that makes
it virtually undetectable through EM and strong interactions. Moreover, during
the early stages of the universe, it is postulated that axions were produced
in abundance due to the breaking of the Peccei-Quinn symmetry [4, 5]. This
massive production in the cosmic past would contribute to the present density
of DM and explain its presence over time.

Axions are a subject of intense study, and efforts in the scientific community
to test their existence experimentally have been made since 1983, when P.
Sikivie proposed two experiments to detect axions from different sources [6].
These experiments take advantage of the interaction between the axion field
and the EM fields, where axions are converted into photons under a static
magnetic field. This physical phenomenon is the so-called inverse Primakoff
effect [7], and the expression that relates the frequency of the generated
photons and the axion mass is

$f\approx\frac{m_{a}c^{2}}{h},$ (1)

being $c$ the speed of light in vacuum and $h$ the Plank constant. Currently,
there are three types of axion experiments that make use of the inverse and
direct Primakoff effect, being haloscope experiments the only ones that can
detect DM axions. But the main obstacle to detecting the axion is that its
mass cannot be predicted from theory, and it must be experimentally determined
by sweeping a very broad energy range. Exhaustive reviews of axion searches
can be found in [8] and [9]. If the axion is detected, the nature of dark
matter will finally be unveiled.

———————————————————————

SIDEBAR 1: Axion detection experiments

Haloscopes: An axion haloscope is a physical device whose objective is to
detect and enhance EM signals induced by DM axions in our galactic halo.
Typically, they are placed in high magnetic field magnets working at cryogenic
temperatures (a scheme is shown in Figure 2 (a)). Due to the dependence of the
axion mass on frequency, the characteristics and dimensions of the haloscope
will depend on the mass range where the search is being performed, which will
involve the use of different technologies, such as microwave resonant
cavities, lumped LC resonators [10, 11], and broadband reflectors [12, 13].

Helioscopes: The aim of these experiments is to detect the abundant flux of
axions emitted from our sun based on the axion-photon conversion, using a
dipole magnet directed towards the sun (see scheme in Figure 2 (b)). When the
sun’s nuclear fission processes create X-rays that scatter off electrons and
protons in the presence of high electric fields, the sun’s core can produce
axions that are sent to our planet. Therefore, axion searches in helioscopes
use highly efficient photon detectors in the X-ray region. First measurements
with helioscopes were carried out at Brookhaven National Laboratory (BNL)
[14], then at the University of Tokyo [15] and lately at the CERN Axion Solar
Telescope (CAST) [16]. These experiments have searched different regions, and
the next new generation helioscope, the International Axion Observatory (IAXO)
[17], is expected to be a breakthrough in solar axion detection.

Light Shining through Walls (LSW): Axions could also be produced in the
laboratory from an artificial photon source, e.g. a laser, which introduces
photons into Fabry-Pérot cavities within a superconducting dipole magnet. The
axions that are produced in the first cavity propagate through an optical wall
to the second cavity. Then, they are transformed back into photons with their
original energy by a similar magnet system behind the wall, being collected by
an optical sensor. A scheme is shown in Figure 2 (c). Some LSW experiments are
the Brookhaven-Fermilab-Rochester-Trieste Collaboration [18], OSQAR [19],
ALPS-I [20] and ALPS-II [21].

(a)

(b)

(c)

Figure S2: Experiment scheme showing details of the detection principle for
(a) haloscopes, (b) helioscopes, and (c) LSWs.

———————————————————————

##  2 General concepts of the resonant haloscope experiment

Due to space limitations, within the possible configurations, in this article
we will focus on resonant cavity haloscopes that operate in the microwave
range. The primary goals of an effective axion detection apparatus involve the
maximization of power arising from the axion-photon coupling, augmentation of
the examined axion mass spectrum (and its scanning rate), and the optimization
of haloscope sensitivity. The detected power ($P_{d}$) depends on the inherent
axion characteristics and the experimental parameters of the cavity [22], and
it can be obtained in natural units as

$P_{d}=\kappa g_{a\gamma}^{2}\frac{\rho_{DM}}{m_{a}}B_{e}^{2}CVQ_{l}.$ (2)

Here, $\kappa$ represents the coupling to the external receiver ($\kappa=0.5$
denoting the critical coupling scenario), $g_{a\gamma}$ the axion-photon
coupling, $\rho_{DM}$ the dark matter density, $m_{a}$ the mass of the axion
particle, $B_{e}$ the magnetostatic field (provided by the employed magnet),
$C$ the form factor of the haloscope, $V$ the volume of the cavity, and
$Q_{l}$ its loaded quality factor. The form factor ($C$), quantifying the
alignment between the external magnetic field and the dynamic electric field
of the cavity induced by the axion-photon conversion, can be mathematically
expressed as

$C=\frac{|\int_{V}\vec{B_{e}}\cdot\vec{E}\>dV|^{2}}{\int_{V}|\vec{B_{e}}|^{2}dV\int_{V}\varepsilon_{r}|\vec{E}|^{2}\>dV},$
(3)

where $\varepsilon_{r}$ is the relative electrical permittivity within the
cavity. An indicator of the haloscope sensitivity lies in the detectable
axion-photon coupling, achievable for a specified signal-to-noise ratio $S/N$,
and can be obtained as

$g_{a\gamma}=\left(\frac{\frac{S}{N}k_{B}T_{sys}}{\kappa\rho_{DM}CVQ_{l}}\right)^{\frac{1}{2}}\frac{1}{B_{e}}\left(\frac{m_{a}^{3}}{Q_{a}\Delta
t}\right)^{\frac{1}{4}},$ (4)

where $k_{B}$ is the Boltzmann constant, $T_{sys}$ the system noise
temperature, $\Delta t$ the time window during which data is acquired, and
$Q_{a}$($\approx 10^{6}$) the axion quality factor, as detailed in [22].

The tuning of the resonant frequency within a haloscope stands as a critical
attribute, given the lack of knowledge of the axion mass value. The data
acquisition strategy involves systematically scanning through a defined mass
range across the spectrum, necessitating a procedure for frequency shifting.
The scanning rate $\frac{dm_{a}}{dt}$, commonly employed as a metric to gauge
haloscope performance, is derivable from eq. 4 as outlined in [22], obtaining

$\frac{dm_{a}}{dt}=Q_{a}Q_{l}\kappa^{2}g_{a\gamma}^{4}\left(\frac{\rho_{DM}}{m_{a}}\right)^{2}B_{e}^{4}C^{2}V^{2}\left(\frac{S}{N}k_{B}T_{sys}\right)^{-2}.$
(5)

Finally, the detection setup of the experiment is composed of common devices
in microwave engineering. A schematic of the readout chain is shown in Figure
3.

Figure 3: Schematic of a typical readout chain in haloscope experiments. Low
noise components are used in the cryogenic stage, while other devices like
filters and mixers are employed at room temperature.

———————————————————————

SIDEBAR 2: Cavity configuration depending on the magnet

The orientation of the magnetic field will determine the resonant mode to be
used for axion data acquisition since its electric field should be as parallel
as possible to the magnetostatic field for increasing the form factor ($C$).
Numerous haloscope experiments, such as ADMX [23], HAYSTAC [24] or CAPP [25],
traditionally employ ad hoc solenoid magnets that generate an axial magnetic
field. Consequently, the cavity configuration involves a cylindrical cavity
aligned parallel to the magnet bore axis, optimizing $C$ by facilitating the
excitation of a $\textrm{TM}_{010}$ mode characterised by an axial electric
field. This is the preferred type of magnet, and most of the current
experiments use it. A schematic description of the magnetic field arrangement
and the modes used in this type of magnet can be seen in Figure 4 (a). Other
structures with tall rectangular geometries could be used in solenoid magnets.
In this case, the chosen resonant mode is the $\textrm{TM}_{110}$, which
produces a vertical-polarized electric field [26].

In contrast with solenoid magnets, the less-used dipole magnet [16, 27]
produces a transverse magnetic field. For compatibility with such magnets, a
more suitable cavity shape is the rectangular configuration, where the
$\textrm{TE}_{101}$ mode features a vertically polarized electric field.
However, their geometrical counterpart, cylindrical cavities, can also be used
for this type of magnet, being the $\textrm{TE}_{111}$ mode the one whose
electric field is parallel to the dipole magnetic field [28] (see Figure 4
(b)).

(a)

(b)

Figure S4: Orientation of rectangular and cylindrical cavities in (a) solenoid
and (b) dipole magnets. External cylinders represent the magnet bores. Blue
lines show the external magnetic field distribution in the magnets, while red
lines depict the electric field of the chosen resonant modes in each cavity.

———————————————————————

##  3 Experimental objectives and associated technologies

Taking into account the extremely low interaction between axions and photons,
very small detection power is expected, in the order of $10^{-22}\sim
10^{-24}$ W, depending on the working frequency. This points out the necessity
of optimizing the operational parameters involved in the experiment. From eq.
5 we can identify those parameters imposed by nature ($g_{a\gamma}$, $m_{a}$,
$\rho_{DM}$), those that are given by the magnet ($B_{e}$) and partially by
the cryostat ($T_{sys}$), and those that can be optimized through different
microwave technologies (cavity volume, quality and form factors, coupling
between transmission line and cavity, and noise added by the read-out chain).
This gives us the following Figure of Merit to quantify the resonator
performance:

$\textrm{FoM}=Q_{0}V^{2}C^{2}.$ (6)

Since $C$ depends mainly on the operational cavity mode and the external
magnetic field, the parameters to be optimized in the cavity design are
$Q_{0}$ and $V$. In this section, we identify some techniques currently used
or proposed for improving these parameters.

###  3.1 Volume

When the magnet employed in the experiment is much bigger than the resonant
cavity, typically for high frequencies, it is possible to increase the volume
of the cavity in order to improve the sensitivity and, ultimately, the
resonator FoM. But if not made properly, the increase in the cavity size will
decrease its resonant frequency, moving away from our frequency target.
Different strategies have been proposed for increasing the volume without
impacting the resonant frequency.

Increasing the size of the cavity in that direction which does not affect the
resonant frequency is the simplest way to gain volume. Thus, when working with
solenoids and rectangular or cylindrical cavities, the $\textrm{TM}_{110}$ or
$\textrm{TM}_{010}$ mode, respectively, allows to increase the length of the
cavity ($z$-direction in Figure 4). Moreover, when one of the dimension with
EM field variation is much bigger than the other one, the resonant frequency
is set by the smallest dimension, and the large one can be increased with
negligible effect on the resonant frequency. This is the case, for instance,
of rectangular geometries with mode $\textrm{TE}_{101}$ in dipole magnets,
when the length ($d$) is much larger than the width ($a$), leading to

$f_{TE_{101}}=\frac{c}{2}\sqrt{\frac{1}{a^{2}}+\frac{1}{d^{2}}}\approx\frac{c}{2a}.$
(7)

The limit of this increase is given by the effect of mode clustering when the
cavity becomes very large electrically. This clustering hinders the
identification of the resonance and even can decrease the form factor around
the resonance peak since the energy from the photons is divided between the
axion mode and the closest neighbour mode, provided that that mode is excited
by the photon. A complete analysis of this limitation is presented in [26] for
rectangular cavities and [28] for cylindrical cavities.

An alternative or even complementary idea for increasing the volume is the use
of multiple identical cavities stacked within the magnet, whose signal is
coherently summed in a power combiner. In that way, the total detected power
is the sum of the power of each cavity, and, so, the effective volume is the
sum of individual volumes. This strategy has been proposed in [29] for $6$ GHz
and [30] for W-band frequencies. Fortunately, it is assumed from theoretical
models that the axion de Broglie wavelength, and so that of the photon
produced from it, is $\sim 100$ meters. Therefore, for experiments with a
small or medium size, we can consider that the photons produced in the
different cavities have the same phase. The main drawback here is that care
must be taken with the path from each cavity to the power combiner in order to
guarantee the same length and, therefore, the same phase shift in the
combiner.

In order to avoid this problem, it is possible to obtain a coherent addition
of signals using only one resonant device by means of the multicavity concept.
This is a well-known idea from cavity filter design [31] and it is adapted for
DM axion detection in [32, 22, 33]. Depending on the number of axes used to
stack the subcavities, the multicavity will be of type 1D, 2D, or 3D, as
Figure 5 depicts. The increase in volume can be enhanced when complementing
multicavity and large cavities, that is, a multicavity made up of large
cavities. Again, the limitation in volume comes from the clustering of cavity
modes and multicavity configurations. A complete analysis of these limits can
be found in [26] and [28] for rectangular and cylindrical multicavities,
respectively. There it is shown that it is possible to enhance the FoM with a
proper multicavity design up to $\sim 175$ times regarding a standard cavity.

(a)

(b)

(c)

(d)

Figure 5: Examples of rectangular multicavity haloscopes of (a) 1D, (b) 2D,
and (c) 3D type. (b) and (c) are taken and modified from [26]. (d) Electric
field pattern (vertical polarization) for the configuration mode
$\textrm{TE}^{+++++}_{101}$ which produces the highest form factor in the
multicavity of (a). The red fields denote positive levels, and the green
fields zero. Taken and modified from [26].

Another interesting concept proposed for increasing the volume is the
metamaterial or plasma haloscope, based on the axion-plasmon interaction,
which occurs when a plasma medium is present [34]. To create this medium,
haloscopes have been developed where the inner volume is filled with metallic
wires that create a medium with plasma characteristics, where the resonant
frequency depends on the distance between these wires independently of the
physical dimensions of the cavity (as is the case with conventional
haloscopes). This allows the search for axions at higher frequencies while
maintaining a large volume [35]. Figure 6 depicts this wire array cavity
concept.

Figure 6: Wire medium cavity scheme. The distance between wires ($a$) is
directly related to the resonant frequency of the operation mode, allowing the
volume to be increased while maintaining a high resonant frequency.

Although multicavity and metamaterial concepts are very useful for increasing
the volume, their main drawback is the complexity of the structure, not only
from the manufacturing point of view, especially when working at high
frequencies ($>30$ GHz), but mainly for the difficulty of implementing proper
tuning mechanisms within them.

###  3.2 Quality factor

The straightforward way to increase the cavity quality factor is by increasing
the electrical conductivity of the inner walls or any other metallic part
inside the cavity. Usually, haloscope cavities are made of high-purity copper
or copper-plated non-magnetic stainless steel. It is important to use non-
magnetic metals in order to avoid damage to the cavity in the case of quench
events in the magnet. An estimated conductivity of $2\times 10^{9}$ S/m for
copper at cryogenic temperature allows $Q_{0}$ in the order of $10^{5}$ for a
hundred of MHz or $10^{4}$ for a few or dozens of GHz.

In order to increase these values, superconductor materials must be used. Type
I superconductors have produced very high $Q_{0}$ ($\sim 10^{11}$) in cavities
for particle accelerators [36], but they cannot be used in the haloscope
experiment because of the strong magnetic field that permeates the cavity.
Fortunately, type II superconductors, based on rare earth oxides, are
magnetic-resilient and are able to achieve high, although more moderate,
$Q_{0}$ in the order of $10^{6}$, as reported in [37]. It is important to note
that in this case, slight losses due to gaps between the superconductor films
or seams/slits in the cavity junctions can spoil the quality factor
completely. Therefore, both new coating techniques and new cavity geometry and
manufacturing processes are of key importance to overcome this problem.

A completely different way to improve the quality factor is employed in [38]
by the use of low-loss dielectric inside the cavity and choosing a higher mode
as axion mode. Although a higher-order mode and the use of dielectric reduce
significantly the form factor (eq. 3), the resonant frequency increases, and
if there is enough room in the magnet bore, the volume of the cavity can be
increased to get the target frequency. In this way, the factor $C^{2}V^{2}$
does not change too much, and we get a higher $Q_{0}$ due to the fact that the
currents in the walls are smaller due to the dielectric confinement and the
high-order mode.

———————————————————————

SIDEBAR 3: Tuning systems

A complete haloscope experiment requires the exploration of a target frequency
range corresponding to an axion mass range. In a typical resonant haloscope
experiment, the objective is to be sensitive to the axions in a frequency
range (typically $1-10$ $\%$) around a central frequency. This frequency sweep
is achieved by means of a tuning mechanism that is able to change the resonant
frequency of the cavity, minimizing the impact on other important parameters
such as the form factor or the quality factor.

Mechanical tuning is widely used in haloscopes because of the greater ease of
changing the electric field distribution compared to electrical tuning.
However, it requires the use of auxiliary systems, such as motors or
piezoelectrics, that occupy space inside the magnet bore. In addition, these
devices must be compatible with working at cryogenic temperatures and under
strong magnetic fields. So far, the most commonly used mechanism is to move a
metallic cylinder inside the cavity [39, 24] from the edge to the centre,
changing the EM field distribution of the $TM_{010}$ and thus its resonant
frequency (see Figure 7 (a)). For different resonant modes, other techniques
have been proposed, such as the use of rotating plates inside the cavity for
$TE_{111}$ in [40] (see Figure 7 (b)). For rectangular cavities, the
displacement of one of the cavity walls allows to change one of its dimensions
depending on the EM mode of operation, affecting the resonant frequency (see
Figure 7 (c)) [30, 41]. These moving parts must incorporate mechanisms to
avoid radiation, such as the use of gaskets to improve metal contact or radio-
frequency chokes. Moreover, there is the possibility to perform tuning based
on ’vertical cut’, i.e. separating two halves of a cavity and thus varying its
width (see Figure 7 (d)). A study of this type of tuning can be found in [42].
In Figure 8, the obtained tuning range for a cavity based on rotating plates
and the FoM are shown. Minima in the FoM are due to mode crossings, and those
frequencies are considered as blind regions for axion detection in the
experiment.

(a)

(b)

(c)

(d)

Figure S7: Different tuning mechanical procedures in cavity haloscopes: (a)
metallic rod, (b) rotating plates, (c) sliding wall, and (d) vertical cut. In
(a) an off-centered angular movement is applied, in (b) a centered rotational
motion, and in (c) and (d) a longitudinal movement. Figure 8: Resonant
frequency (red line) and FoM (blue line) versus rotating plates angle from one
of the cavities described in [40].

In contrast to mechanical tuning, electrical tuning systems can provide a
design that is more resistant to mechanical failures (e.g., by omitting
movable components, which may pose a risk at cryogenic temperatures), which
reinforces and complements current methods. Moreover, scaling mechanical
systems at higher frequencies is challenging, due to the small dimensions of
the devices. Electrical tuning for haloscopes by using ferroelectric materials
has been studied in [43]. These materials allow their permittivity to be
changed by applying an external DC voltage, thus, changing the mode frequency.
Figure S9 shows an scheme of a cavity with ferroelectric films for tuning.
Along the same lines, another current research is tuning with a change in
permeability by means of ferromagnetic materials, allowing the frequency to be
controlled by the magnetic field of the magnet [44].

Figure S9: Transverse view of $E_{y}$ (vertical component of the E-field)
inside the resonant cavity (white) with ferroelectric elements (purple). The
red line denotes a positive value of the electric field, while the blue lines
show a negative value. The variation in the permittivity ($\varepsilon_{r}$)
shifts the 0-level electric field position, and hence it provides a frequency
change. Taken from [43].

Regardless of the tuning system, it will slightly modify the EM field
distribution in the cavity and, therefore, the coupling between the
transmission line and the cavity. Since the haloscope performance (eq. 5)
depends on this coupling, it is mandatory to have a movable probe in order to
keep the target coupling from one frequency point to the next. Paradoxically,
this target is not $\kappa=\frac{1}{2}$ (critical coupling), but
$\kappa=\frac{2}{3}$ (overcoupled) or even higher. A complete analysis of
optimal coupling can be found in [45]. For electrical coupling, a moving
monopole antenna is normally used, while a rotating loop is employed for
magnetic coupling [40]. For millimeter waves, where coaxial lines are avoided
due to their high losses, an iris on the waveguide connected to the cavity and
a moving pin are proposed in [30].

———————————————————————

###  3.3 Noise

Reducing the system noise is a key factor in improving the haloscope
performance, so working at cryogenic temperatures is mandatory, thus reducing
the thermal noise of the cavity and the first amplifier in the readout chain.

Several technologies can be used to amplify the low-expected signal. The most
commonly used devices are Low Noise Amplifiers, which introduce a small noise
to the signal, offering a typical system temperature of around $5-10$ K [22].
It is also possible to approach the Standard Quantum Limit (SQL) [46] by using
superconducting amplifiers, such as Superconducting Quantum Interference
Devices (SQUIDs). These detectors have been applied in [47], being able to
reduce the system noise temperature to $\sim 300$ mK. Other proposals that
make use of Josephson Parametric Amplifiers (JPAs) [48] or Josephson Traveling
Wave Parametric Amplifiers (JTWPAs) [49] can be seen in [50] and [51],
respectively.

Recent proposals significantly reduce the noise by using photon counting
devices, which detect individual photons and can be useful at lower
temperatures ($T\>\sim\>$ mK) and high cavity $Q_{0}$ values, allowing the
noise to go beyond the SQL. A complete performance analysis of single-photon
amplifiers compared to linear amplifiers can be found at [52]. A photon
counter based on a Superconducting Transmon Qubit [53] has been proposed as a
quantum detector for DM detection in [54], in this case for dark photons, not
axions, where the external magnetic field is not needed. Figure 10 depicts the
qubit-cavity system, where the qubit interacts with the storage cavity and
change its state if photons are detected. For higher frequencies, the use of
Kinetic Inductance Detectors (KIDs) [55] is described in [30]. These devices
are high-quality factor superconducting resonators that absorb photons,
producing a variation effect in the kinetic inductance of the resonator and
enabling the detection of extremely low-power signal. The major obstacle in
using quantum devices in DM axion detection is their incompatibility with
large magnetic fields. Therefore, the development of new magnet-resilient
quantum devices is a current research line.

Figure 10: Scheme of a qubit-cavity system for DM detection. The dark photon-
photon interaction takes place in the storage cavity and produces a change in
the state of the qubit. The readout cavity sends signals to the qubit to check
the current state. In the image, a zoom of the Josephson junction section is
shown.

##  4 Conclusions

In the last forty years, an accelerating race in the quest for the DM axion
has taken place within the particle physics community. As seen, the different
detection schemes make intensive use of microwave technologies, and the
success of the future detection of this particle depends on the improvement of
well-known techniques and the development of new ones. Microwave engineering
has the opportunity to contribute to solving the dark matter mystery. From the
review presented here, we can summarize a non-exhaustive list of research
lines where key contributions can come from:

\- Multicavity filters and metamaterial cavities for high-volume and high-Q
devices.

\- New electromagnetic mechanisms for resonator tuning, including
ferroelectric and ferromagnetic materials.

\- New magnetic field-resilient superconductor materials and new cavity
geometries for them to reach very high quality factors.

\- Single photon counters based on quantum devices in order to drastically
reduce the noise of the readout chain. Some examples are 3D transmon qubits.

Advances in these lines will increase the probability of detecting not only
the "invisible" axion but other particles such as the dark photon (another
candidate to make up the dark matter) or the graviton (gravitational waves),
since recently haloscope schemes have been proposed for both particle
detection [56, 57, 58].

## Acknowledgments

This work was performed within the RADES group. We thank our colleagues for
their support. It is part of the R$\&$D projects PID2019-108122GB-C33 and
PID2022-137268NBC53, funded by MICIU/AEI/10.13039/501100011033/ and by
“ERDF/EU”. JMGB thanks the European Research Council grant ERC-2018-StG-802836
(AxScale project), and the Lise Meitner program “In search of a new, light
physics” of the Max Planck society.

## References

  * [1] F. Zwicky  “The redshift of extragalactic nebulae”  In _Helvetica Physica Acta_ 6, 1933, pp. 110–127
  * [2] F. Wilczek  “Problem of Strong $P$ and $T$ Invariance in the Presence of Instantons”  In _Phys. Rev. Lett._ 40, 1978, pp. 279–282  DOI: 10.1103/PhysRevLett.40.279
  * [3] S. Weinberg  “A New Light Boson?”  In _Phys. Rev. Lett._ 40 American Physical Society, 1978, pp. 223–226  DOI: 10.1103/PhysRevLett.40.223
  * [4] R.. Peccei and H.. Quinn  “CP Conservation in the Presence of Instantons”  In _Phys. Rev. Lett._ 38, 1977, pp. 1440–1443  DOI: 10.1103/PhysRevLett.38.1440
  * [5] R.. Peccei and H.. Quinn  “Constraints Imposed by CP Conservation in the Presence of Instantons”  In _Phys. Rev. D_ 16, 1977, pp. 1791–1797  DOI: 10.1103/PhysRevD.16.1791
  * [6] P. Sikivie  “Experimental Tests of the "Invisible" Axion”  In _Phys. Rev. Lett._ 51 American Physical Society, 1983, pp. 1415–1417  DOI: 10.1103/PhysRevLett.51.1415
  * [7] H. Primakoff  “Photo-Production of Neutral Mesons in Nuclear Electric Fields and the Mean Life of the Neutral Meson”  In _Phys. Rev._ 81 American Physical Society, 1951, pp. 899–899  DOI: 10.1103/PhysRev.81.899
  * [8] Igor G. Irastorza and Javier Redondo  “New experimental approaches in the search for axion-like particles”  In _Progress in Particle and Nuclear Physics_ 102, 2018, pp. 89–159  DOI: https://doi.org/10.1016/j.ppnp.2018.05.003
  * [9] Luca Di Luzio, Maurizio Giannotti, Enrico Nardi and Luca Visinelli  “The landscape of QCD axion models” The landscape of QCD axion models  In _Physics Reports_ 870, 2020, pp. 1–117  DOI: https://doi.org/10.1016/j.physrep.2020.06.002
  * [10] N. Crisosto et al.  “ADMX SLIC: Results from a Superconducting LC Circuit Investigating Cold Axions”  In _Phys. Rev. Lett._ 124 American Physical Society, 2020, pp. 241101  DOI: 10.1103/PhysRevLett.124.241101
  * [11] Jonathan Ouellet et al.  “First Results from ABRACADABRA-10 cm: A Search for Sub-$\mu$eV Axion Dark Matter”  In _Physical Review Letters_ 122, 2019  DOI: 10.1103/PhysRevLett.122.121802
  * [12] Dieter Horns et al.  “Searching for WISPy Cold Dark Matter with a Dish Antenna”  In _JCAP_ 04, 2013, pp. 016  DOI: 10.1088/1475-7516/2013/04/016
  * [13] Jesse Liu et al.  “Broadband Solenoidal Haloscope for Terahertz Axion Detection”  In _Phys. Rev. Lett._ 128 American Physical Society, 2022, pp. 131801  DOI: 10.1103/PhysRevLett.128.131801
  * [14] D.. Lazarus et al.  “Search for solar axions”  In _Phys. Rev. Lett._ 69 American Physical Society, 1992, pp. 2333–2336  DOI: 10.1103/PhysRevLett.69.2333
  * [15] S. Moriyama et al.  “Direct search for solar axions by using strong magnetic field and X-ray detectors”  In _arXiv: High Energy Physics - Experiment_ , 1998  URL: https://api.semanticscholar.org/CorpusID:14425790
  * [16] V. Anastassopoulos  “New CAST Limit on the Axion-Photon Interaction”  In _Nature Phys._ 13, 2017, pp. 584–590  DOI: 10.1038/nphys4109
  * [17] E. Armengaud et al.  “Physics potential of the International Axion Observatory (IAXO)”  In _Journal of Cosmology and Astroparticle Physics_ 2019.06, 2019, pp. 047  DOI: 10.1088/1475-7516/2019/06/047
  * [18] G. Ruoso, R. Cameron and G. Cantatore  “Search for photon regeneration in a magnetic field”  In _Zeitschrift für Physik C Particles and Fields_ 56.4, 1992, pp. 505–508  DOI: 10.1007/BF01474722
  * [19] P. Pugnat et al.  “Results from the OSQAR photon-regeneration experiment: No light shining through a wall”  In _Physical Review D - PHYS REV D_ 78, 2008  DOI: 10.1103/PhysRevD.78.092003
  * [20] Klaus Ehret  “Resonant laser power build-up in ALPS: A ’Light-shining-through-walls’ experiment”  In _Nucl. Instrum. Meth. A_ 612, 2009, pp. 83–96  DOI: 10.1016/j.nima.2009.10.102
  * [21] Robin Bähre  “Any light particle search II —Technical Design Report”  In _JINST_ 8, 2013, pp. T09001  DOI: 10.1088/1748-0221/8/09/T09001
  * [22] Alejandro Díaz-Morcillo  “Design of New Resonant Haloscopes in the Search for the Dark Matter Axion: A Review of the First Steps in the RADES Collaboration”  In _Universe_ 8.1, 2021, pp. 5  DOI: 10.3390/universe8010005
  * [23] R Khatiwada, D Bowring, A S. Chou and Andrew Sonnenschein  “Axion Dark Matter eXperiment: Detailed design and operations”  In _Review of Scientific Instruments_ 92.12, 2021  DOI: 10.1063/5.0037857
  * [24] S. Al Kenany et al.  “Design and operational experience of a microwave cavity axion detector for the 20–100 $\mu$eV range”  In _Nuclear Instruments and Methods in Physics Research Section A: Accelerators, Spectrometers, Detectors and Associated Equipment_ 854, 2017, pp. 11–24  DOI: https://doi.org/10.1016/j.nima.2017.02.012
  * [25] Ohjoon Kwon et al.  “First Results from an Axion Haloscope at CAPP around $10.7\text{ }\text{ }\mu\mathrm{eV}$”  In _Phys. Rev. Lett._ 126 American Physical Society, 2021, pp. 191802  DOI: 10.1103/PhysRevLett.126.191802
  * [26] J.. García-Barceló et al.  “Methods and restrictions to increase the volume of resonant rectangular-section haloscopes for detecting dark matter axions”  In _Journal of High Energy Physics_ 2023.8, 2023, pp. 98  DOI: 10.1007/JHEP08(2023)098
  * [27] Andreas Abeln et al.  “Conceptual design of BabyIAXO, the intermediate stage towards the International Axion Observatory”  In _Journal of High Energy Physics_ 2021, 2021  DOI: 10.1007/JHEP05(2021)137
  * [28] J.. García-Barceló, A. Díaz-Morcillo and B. Gimeno  “Enhancing resonant circular-section haloscopes for dark matter axion detection: approaches and limitations in volume expansion”  In _JHEP_ 11, 2023, pp. 159  DOI: 10.1007/JHEP11(2023)159
  * [29] SungWoo Youn  “Multiple-Cavity Detectors for Axion Search”  In _12th Patras Workshop on Axions, WIMPs and WISPs_ , 2017, pp. 162–167  DOI: 10.3204/DESY-PROC-2009-03/Youn_SungWoo
  * [30] Beatriz Aja  “The Canfranc Axion Detection Experiment (CADEx): search for axions at 90 GHz with Kinetic Inductance Detectors”  In _JCAP_ 11, 2022, pp. 044  DOI: 10.1088/1475-7516/2022/11/044
  * [31] Richard J. Cameron, M. Chandra and Rafaat R. Mansour  “Microwave Filters for Communication Systems”  John Wiley & Sons, Ltd, 2018
  * [32] Alejandro Melcón et al.  “Axion Searches with Microwave Filters: the RADES project”  In _Journal of Cosmology and Astroparticle Physics_ 2018, 2018  DOI: 10.1088/1475-7516/2018/05/040
  * [33] Junu Jeong et al.  “Concept of multiple-cell cavity for axion dark matter search”  In _Physics Letters B_ 777, 2018, pp. 412–419  DOI: https://doi.org/10.1016/j.physletb.2017.12.066
  * [34] Matthew Lawson et al.  “Tunable Axion Plasma Haloscopes”  In _Phys. Rev. Lett._ 123 American Physical Society, 2019, pp. 141802  DOI: 10.1103/PhysRevLett.123.141802
  * [35] Rustam Balafendiev, Constantin Simovski, Alexander J. Millar and Pavel Belov  “Wire metamaterial filled metallic resonators”  In _Phys. Rev. B_ 106 American Physical Society, 2022, pp. 075106  DOI: 10.1103/PhysRevB.106.075106
  * [36] B. Aune et al.  “Superconducting TESLA cavities”  In _Phys. Rev. ST Accel. Beams_ 3 American Physical Society, 2000, pp. 092001  DOI: 10.1103/PhysRevSTAB.3.092001
  * [37] S. Posen et al.  “High-Quality-Factor Superconducting Cavities in Tesla-Scale Magnetic Fields for Dark-Matter Searches”  In _Phys. Rev. Appl._ 20 American Physical Society, 2023, pp. 034004  DOI: 10.1103/PhysRevApplied.20.034004
  * [38] D. Alesini et al.  “Realization of a high quality factor resonator with hollow dielectric cylinders for axion searches”  In _Nuclear Instruments and Methods in Physics Research Section A: Accelerators, Spectrometers, Detectors and Associated Equipment_ 985, 2021, pp. 164641  DOI: https://doi.org/10.1016/j.nima.2020.164641
  * [39] S.. Asztalos  “An Improved RF cavity search for halo axions”  In _Phys. Rev. D_ 69, 2004, pp. 011101  DOI: 10.1103/PhysRevD.69.011101
  * [40] Saiyd Ahyoune  “A Proposal for a Low-Frequency Axion Search in the 1-2 $\mu$eV Range and Below with the BabyIAXO Magnet”  In _Annalen Phys._ 535.12, 2023, pp. 2300326  DOI: 10.1002/andp.202300326
  * [41] Ben T. McAllister, Aaron P. Quiskamp and Michael E. Tobar  “Tunable rectangular resonant cavities for axion haloscopes”  In _Phys. Rev. D_ 109.1, 2024, pp. 015013  DOI: 10.1103/PhysRevD.109.015013
  * [42] Jessica Golm et al.  “Mechanical tuning of a rectangular axion haloscope operating around 8.4 GHz”, 2023  arXiv:2312.13109 [physics.ins-det]
  * [43] J.. García-Barceló et al.  “On the Development of New Tuning and Inter-Coupling Techniques Using Ferroelectric Materials in the Detection of Dark Matter Axions”  In _IEEE Access_ 11, 2023, pp. 30360–30372  DOI: 10.1109/ACCESS.2023.3260783
  * [44] Jose María García-Barceló  “Development of resonant cavity-based microwave filters for axion detection”, 2023
  * [45] Dongok Kim et al.  “Revisiting the detection rate for axion haloscopes”  In _Journal of Cosmology and Astroparticle Physics_ 2020, 2020, pp. 066–066  DOI: 10.1088/1475-7516/2020/03/066
  * [46] Vittorio Giovannetti, Seth Lloyd and Lorenzo Maccone  “Quantum-Enhanced Measurements: Beating the Standard Quantum Limit”  In _Science_ 306.5700, 2004, pp. 1330–1336  DOI: 10.1126/science.1104149
  * [47] C. Boutan  “Axion Dark Matter eXperiment: Run 1A analysis details”  In _Phys. Rev. D_ 109.1, 2024, pp. 012009  DOI: 10.1103/PhysRevD.109.012009
  * [48] José Aumentado  “Superconducting Parametric Amplifiers: The State of the Art in Josephson Parametric Amplifiers”  In _IEEE Microwave Magazine_ 21, 2020, pp. 45–59  URL: https://api.semanticscholar.org/CorpusID:220466449
  * [49] M. Sweeny and R. Mahler  “A travelling-wave parametric amplifier utilizing Josephson junctions”  In _IEEE Transactions on Magnetics_ 21.2, 1985, pp. 654–655  DOI: 10.1109/TMAG.1985.1063777
  * [50] C. Bartram et al.  “Axion dark matter experiment: Run 1B analysis details”  In _Phys. Rev. D_ 103 American Physical Society, 2021, pp. 032002  DOI: 10.1103/PhysRevD.103.032002
  * [51] C. Bartram et al.  “Dark matter axion search using a Josephson Traveling wave parametric amplifier”  In _Review of Scientific Instruments_ 94, 2023, pp. 044703  DOI: 10.1063/5.0122907
  * [52] S.. Lamoreaux, K.. Bibber, K.. Lehnert and G. Carosi  “Analysis of single-photon and linear amplifier detectors for microwave cavity dark matter axion searches”  In _Phys. Rev. D_ 88 American Physical Society, 2013, pp. 035020  DOI: 10.1103/PhysRevD.88.035020
  * [53] Thomas E. Roth, Ruichao Ma and Weng C. Chew  “The Transmon Qubit for Electromagnetics Engineers: An introduction”  In _IEEE Antennas and Propagation Magazine_ 65.2, 2023, pp. 8–20  DOI: 10.1109/MAP.2022.3176593
  * [54] Akash V. Dixit et al.  “Searching for Dark Matter with a Superconducting Qubit”  In _Phys. Rev. Lett._ 126 American Physical Society, 2021, pp. 141302  DOI: 10.1103/PhysRevLett.126.141302
  * [55] Peter Day et al.  “A broadband superconducting detector suitable for use in large arrays”  In _Nature_ 425, 2003, pp. 817–21  DOI: 10.1038/nature02037
  * [56] Andrea Caputo, Alexander J. Millar, Ciaran A.. O’Hare and Edoardo Vitagliano  “Dark photon limits: A handbook”  In _Phys. Rev. D_ 104 American Physical Society, 2021, pp. 095029  DOI: 10.1103/PhysRevD.104.095029
  * [57] Valerie Domcke, Camilo Garcia-Cely and Nicholas L. Rodd  “Novel Search for High-Frequency Gravitational Waves with Low-Mass Axion Haloscopes”  In _Phys. Rev. Lett._ 129.4, 2022, pp. 041101  DOI: 10.1103/PhysRevLett.129.041101
  * [58] Asher Berlin et al.  “Detecting high-frequency gravitational waves with microwave cavities”  In _Phys. Rev. D_ 105 American Physical Society, 2022, pp. 116011  DOI: 10.1103/PhysRevD.105.116011