# MAF: Multi-Aspect Feedback for Improving
Reasoning in Large Language Models

Deepak Nathani , David Wang , Liangming Pan , William Yang Wang
University of California, Santa Barbara, Santa Barbara, CA
{dnathani, d_wang, liangmingpan<EMAIL_ADDRESS>
###### Abstract

Language Models (LMs) have shown impressive performance in various natural
language tasks. However, when it comes to natural language reasoning, LMs
still face challenges such as hallucination, generating incorrect intermediate
reasoning steps, and making mathematical errors. Recent research has focused
on enhancing LMs through self-improvement using feedback. Nevertheless,
existing approaches relying on a single generic feedback source fail to
address the diverse error types found in LM-generated reasoning chains. In
this work, we propose Multi-Aspect Feedback, an iterative refinement framework
that integrates multiple feedback modules, including frozen LMs and external
tools, each focusing on a specific error category. Our experimental results
demonstrate the efficacy of our approach to addressing several errors in the
LM-generated reasoning chain and thus improving the overall performance of an
LM in several reasoning tasks. We see a relative improvement of up to 20% in
Mathematical Reasoning and up to 18% in Logical Entailment. We release our
source code, prompts, and data111Our source code can be found here to
accelerate future research.

##  1 Introduction

Recent research in Language Models has focused on augmenting them with
external tools (Schick et al., 2023; Paranjape et al., 2023), learning from
feedback (Ouyang et al., 2022; Akyurek et al., 2023) and iterative-refinement
(Madaan et al., 2023; Paul et al., 2023; Shinn et al., 2023). Iterative-
Refinement has been a necessary tool in human evolution and problem-solving.
Moreover, humans seek feedback from domain-specific knowledge sources. For
instance, an architect tasked with creating an environmentally friendly and
structurally sound building design will require targeted feedback from civil
engineers for structural integrity, and sustainability experts for eco-
friendly design principles. However, previous works on iterative refinement
overlook this requirement and collect generic feedback from multiple sources
or collect a single one from the Language Model itself.

Figure 1: Overview of the Multi-Aspect Feedback framework. We loop over steps
3 and 4 until we reach a final solution O’ or up to a set number of
iterations.

In this work, we first investigate whether the use of generic feedback is a
bottleneck in addressing the diverse range of errors present in the reasoning
chains generated by LMs. We posit that utilizing generic feedback for a broad
spectrum of errors may result in vague or non-actionable feedback for specific
errors. This is due to two main factors: (i) the inclusion of multiple error
categories within a single prompt, which not only increases the size of the
prompt but also poses a challenge to current models that have a limited
context length and struggle with long texts, and (ii) the model is burdened
with identifying a multitude of error types in a single instance, which
degrades the quality of the generated feedback.

To surmount these challenges, we introduce Multi-Aspect Feedback (MAF), a
novel general-purpose iterative refinement framework that employs a collection
of specialized feedback modules, including pre-trained LMs and external tools,
each tailored to address a specific error category. Our framework includes a
base model to generate an initial solution, multiple feedback modules, and a
refiner model to revise the solution using feedback. Contrary to previous
works, our feedback modules can use one of two refinement strategies: Eager
Refinement and Lazy Refinement section 3.4. When using the Eager-Refine mode,
the solution is revised immediately before moving on to the next feedback,
however, for Lazy Refinement, we first collect feedback from multiple feedback
sources and then refine the solution using this collective feedback.

In devising MAF, we first classified errors in LM-generated reasoning chains
based on Golovneva et al. (2022) and also identified some new error
categories. Subsequently, we decoupled our feedback modules such that each
module focuses on a single error category. This strategy not only elevates
performance but also allows one to leverage specialized tools tailored for
distinct error types, such as utilizing a code interpreter for syntax errors
in generated Python programs instead of relying on generic LM feedback.
Through a modular design, we ensure that each module of our framework is
replaceable depending on the task. We conduct extensive experimentation on a
variety of complex reasoning tasks such as Mathematical Reasoning (Shi et al.,
2023), Logical Reasoning (Dalvi et al., 2021), and Multi-Hop Question
Answering (Dua et al., 2019), and establish that MAF effectively addresses the
challenges previously discussed, and significantly enhances the performance of
Language Models in complex reasoning tasks. We compare our method against the
Base LMs and Self-Refine (Madaan et al., 2023), which also implements
iterative refinement. During our experiments, we find that Over-Refining
section 4 the solution can lead to worse performance, and thus we treat the
number of iterations as a hyperparameter. We also propose an Oracle Verifier
setting section 4, where we assume we have access to an "answer verifier" and
stop the refining process once we have reached the correct answer. To avoid
unfair comparisons, we test Self-Refine under this setting and find that MAF
provides actionable and specific feedback.

To summarize, the main contributions of this work are as follows:

  1. 1.

We propose a novel framework that decouples the feedback generation process
for different error categories. This allows us to use error-specific tools and
LMs. Moreover, the proposed framework is modular and all the modules are Plug-
and-Play. This allows for better flexibility on new tasks and the usage of
advanced models as they become available.

  2. 2.

Additionally, we propose two different types of refinement strategies, Eager
Refinement and Lazy Refinement. Eager Refinement allows immediate revision of
a solution by a feedback module to avoid conflicts, while Lazy Refinement
improves efficiency by performing revisions from multiple feedback modules
together.

  3. 3.

We show that our framework outperforms the base Language Models and similar
Iterative Refinement baselines on several complex reasoning tasks such as Math
reasoning (GSM-IC), Logical Reasoning (Entailment Bank), and Multi-hop
Question Answering (DROP).

##  2 Related Work

#### Chain-of-Thought Reasoning

There has been a plethora of research on prompting Large Language Models to
improve their reasoning capabilities. Wei et al. (2023) found that prompting
the models to generate a reasoning chain in few-shot setting before solving a
problem improves the performance by a huge margin. Further, Kojima et al.
(2022) found that in zero-shot setting prefixing the solution generation with
Let’s think step-by-step has the same effect as generating intermediate
reasoning chain and improves the performance. Following this work, Wang et al.
(2022) proposed sampling multiple outputs from the model and selecting the
final answer by majority voting. Zhou et al. (2023) also showed that
decomposing the original question into smaller sub-problems can give a
performance boost. Madaan et al. (2022) and Chen et al. (2022) showed that
models trained on code can be used to generate a program-of-thought similar to
a chain-of-thought, which enables the model to use a language interpreter for
translating mathematical calculations into code. In this work, we use LMs’ in-
context learning abilities to implement the LM-based feedback and refiner
modules.

#### Tool Augmented LLMs

However, even with these advanced prompting techniques, the LMs still fail for
problems that require external knowledge and are still prone to problems such
as hallucination and incorrect reasoning. To circumvent these issues, several
recent works have introduced the use of external tools such as calculator,
code interpreter, knowledge retrieval (Karpas et al., 2022; Chen et al., 2022;
Schick et al., 2023). Yang et al. (2023); Patil et al. (2023); Shen et al.
(2023) show that one can teach LLMs to use tools by generating API calls. Li
et al. (2023) also released a large dataset to enable research in the field of
augmented Language Models. Hao et al. (2023) propose to represent each tool as
a token and learn tool-specific embeddings to increase the robustness of these
language models for using external APIs. However, reasoning is an iterative
task and it is difficult to find a plausible answer for several problems in
one-shot even when these LLMs are augmented with tools. This has inspired some
recent works to use iterative refinement frameworks. In our work, we use
external tools for generating feedback for certain error categories such as
Programming Syntax Errors, Calculator for mathematical equations etc.

Task | Feedback Type | Type | ER
---|---|---|---
| Programming Syntax | Interpreter | ✓
| Variable Naming | OpenAI | ✓
Math | Redundancy | OpenAI | ✕
Reasoning | Commonsense | OpenAI | ✕
| Missing Step | OpenAI | ✕
Logical | Redundancy | OpenAI | ✕
Reasoning | Repetition | OpenAI | ✕
| Hallucination | OpenAI | ✕
| Redundancy | OpenAI | ✕
Question | Factuality | OpenAI | ✕
Answering | Commonsense | OpenAI | ✕
| Missing Step | OpenAI | ✕
Table 1: Feedback modules used for each task and their types. OpenAI type
modules use a LLM to provide feedback. Feedback Modules with Eager-Refine (ER)
enabled, refine the solution without waiting for feedback from the other
feedback modules. (section 3.4).

#### Learning from Feedback

Schick et al. (2022) took the first step towards iteratively fixing the
problems and introduced the idea of training multiple instances of the same
model to assist in different stages of problem-solving. It was introduced as a
collaborative editing framework. (Madaan et al., 2023) proposed to use the
same model for generating initial solution, feedback generation, and refiner.
Furthermore, recently there have been multiple works exploring the use of
natural language feedback to improve performance. Shinn et al. (2023) proposes
converting different types of feedback into natural language and storing the
feedback in a memory buffer to revisit later. Akyurek et al. (2023) train a
critique generator to maximize the end-task performance of a larger base model
using reinforcement learning. Paul et al. (2023) proposes a framework to fine-
tune LMs for generating intermediate reasoning steps while interacting with a
trained critique model to bridge the gap between small and large models.
Recent research (Wu et al., 2023) has also demonstrated the benefits of
incorporating decoupled fine-grained feedback for each error category by using
a set of fine-tuned reward models.

Despite these advancements, there is a lack of empirical evidence supporting
the effectiveness of decoupled multi-aspect feedback for iterative refinement.
Our work addresses this gap by demonstrating that a generic feedback module,
is insufficient to address the diverse range of potential errors in language
model responses. To overcome this limitation, we propose a suite of feedback
modules, each specifically targeting a particular error category and providing
detailed feedback to improve both reasoning and solution quality.

##  3 MAF: Multi-Aspect Feedback

Figure 2: Comparison of the feedback generated by MAF and Self-Refine.
Algorithm 1 MAF algorithm

1:Input x, model M, Refiner R, number of iterations $T$

2:$n$ Eager-refine $\\{s_{1},s_{2},...s_{n}\\}$ and $m$ Lazy-Refine
$\\{p_{1},p_{2},...p_{m}\\}$ Feedback Modules

3:Initialize output $y_{0}$ from M

4:while $i<T$ do

5: for $j\leftarrow 1$ to $n$ do $\triangleright$ Eager-refine Feedbacks

6: $fs_{j}\leftarrow generate\\_feedback(s_{j},y_{i})$ $\triangleright$
Generate Feedback

7: if $fs_{j}$ indicates revision is required then

8: $y_{i}\leftarrow revise(\textbf{{R}},fs_{j},y_{i})$ $\triangleright$ Revise
solution

9: $\textbf{{F}}\leftarrow""$ $\triangleright$ Initialize empty feedback

10: for $k\leftarrow 1$ to $m$ do $\triangleright$ Lazy-refine Feedbacks

11: $fp_{k}\leftarrow generate\\_feedback(p_{k},y_{i})$

12: if $fp_{j}$ indicates revision is required then

13: $\textbf{{F}}\leftarrow\textbf{{F}}+fp_{k}$

14: if F not empty then

15: $y_{i+1}\leftarrow revise(\textbf{{R}},fp_{k},y_{i})$

16: else

17: $y_{i+1}=y_{i}$

###  3.1 Overview

In this work, we present an iterative refinement framework with an explicit
focus on decoupling feedback generation for different error categories. This
approach allows us to systematically address different error types in the
generated solutions. Our proposed framework has three crucial components: a
base language model M that generates an initial solution O. A collection of n
feedback modules $\\{f_{0},f_{1},f_{2}...f_{n}\\}$, each focusing on a single
error category, collectively these modules generate a multi-aspect feedback F.
And a Refiner R that generates refined solution O’ based on initial solution O
and feedback F.

While there are other works that also use iterative refinement (Madaan et al.,
2023; Akyurek et al., 2023; Peng et al., 2023), to the best of our knowledge,
we are the first to explore the effect of decoupling different types of
feedbacks. Taking inspiration from ROSCOE (Golovneva et al., 2022), we
categorize feedback into ten distinct categories: arithmetic, programming
syntax, variable naming, missing step, coherency, redundancy, repetition,
hallucination, commonsense, and factuality. Definitions of these error
categories are provided in Appendix A. Moreover, a feedback module can be a
tool such as a code interpreter for syntax feedback, a calculator for
arithmetic feedback, a knowledge graph for factuality or commonsense feedback,
a Language Model, or even a fine-tuned model. It is important to note that the
feedback generation process is not limited to these categories and can be
extended to include other categories as well. The overall process is
illustrated in Figure 1 and Algorithm 1.

###  3.2 Initial Generation

We use a large model such as GPT3.5222https://openai.com/blog/chatgpt, GPT4
(OpenAI, 2023), to generate an initial solution. However, generating just one
solution isn’t ideal for the reasoning process since reasoning is an iterative
process. The solution often needs to be refined over time with every iteration
bringing us closer to the correct answer. We follow the same principle in this
work, where we initially produce a solution and then proceed to refine it
based on actionable feedback.

###  3.3 Feedback Modules

Feedback generation is an involved task, providing a comprehensive list of
common issues encountered in model outputs. The feedback generation is
accomplished through a variety of tools known as feedback modules, which may
include external tools, frozen LLMs, fine-tuned models, and scorers. These
modules are used to provide actionable feedback based on the initial solution.

Each type of feedback module is suited to address specific types of errors.
For instance, an external tool like a code interpreter would be ideal for
providing feedback on syntax errors in code, while a fine-tuned model could
provide more nuanced feedback on issues such as redundancy or hallucination.
This decoupled approach allows us to address errors in a more targeted manner,
improving the overall quality of the refined solution. Table 1 shows all the
feedback modules used in our work.

Furthermore, it’s worth noting that merely specifying the error categories in
a single prompt doesn’t yield satisfactory results. We see two main reasons
for this, firstly because the model is tasked with focusing on multiple errors
simultaneously and secondly, as the number of error categories increase, the
context length increases as well which results in high-quality feedback for
the first few error categories and a steep decline in quality for the rest.
While some of the previous works (Wu et al., 2023; Paul et al., 2023) have
explored using fine-grained feedback and achieved promising results.

Moreover, our feedback modules can choose between two refinement strategies:
Eager-Refine or Lazy-Refine. We discuss this distinction in more detail in the
next section.

###  3.4 Refinement

In this work, we reuse the initial model M as a Refiner. During the refining
phase, the refiner is given a solution and multi-aspect feedback and asked to
revise the solution according to feedback. We found that these large models
are proficient at the refining task, demonstrating a marked improvement in the
quality of the final solution compared to the initial output.

As mentioned in the previous section, we use two refinement strategies: Eager-
Refinement and Lazy-Refinement. The eager-Refinement approach is used for
feedback types that can cause conflicts during refinement. An example of such
as feedback module would be Variable Naming (VN), which is used to correct
variable names in the generated code. This module can cause conflicts with
others because when the other modules are referencing a variable that is
supposed to be changed according to the VN feedback and can render the program
inexecutable if refined incorrectly. Whereas, in the lazy refinement strategy
feedback from the multiple modules is concatenated together along with the
appropriate error categories to make a single multi-aspect feedback. This
collective feedback is then passed to the Refiner model in order to get a
revised solution. Another advantage of this approach is the increased
efficiency by refining once for multiple errors and the added flexibility.

However, we also found that the smaller open-source models like LLaMA (Touvron
et al., 2023), Alpaca (Taori et al., 2023) and
Vicuna333https://lmsys.org/blog/2023-03-30-vicuna/ often fail to adhere to the
feedback provided thus making the refinement process ineffective. These
smaller models also have smaller context lengths which make it difficult to
include all the feedback in the prompt. To address this issue, we use a
Selective Summarization approach. We only select parts of feedback that point
to a problem. Figure 2 shows the "summarized" feedback generated by our
approach. This simple selective summarization approach makes the feedback
succinct and also proves to be less distractive to the models during the
refining phase. This approach allows us to effectively combine all feedback
together and use models with smaller context lengths to some extent.

Model | EB | GSMIC | GSM8K | DROP
---|---|---|---|---
GPT3.5 | 56.1 | $76.2$ | $69.2$ | 72.3
$+$SR |  $53.5$ ↓2.6 |  $77.0$ ↑0.8 |  $69.2$ ↓0.0 | $62.0\makebox[0.0pt][l]{{\color[rgb]{1,0.30078125,0.30078125}\definecolor[named]{pgfstrokecolor}{rgb}{1,0.30078125,0.30078125}\textdownarrow 8.3}}$
$+$SR★ |  $54.5$ ↓1.6 |  $87.0$ ↑10.8 |  $\underline{77.4}$ ↑8.2 |  $\underline{77.5}$ ↑4.2
$+$MAF |  $54.5$ ↓1.6 |  77.4 ↑1.2 |  $69.8$ ↑0.6 |  ${66.2}$ ↓6.1
$+$MAF★ |  $\underline{60.4}$ ↑4.3 |  $\underline{91.4}$ ↑15.2 |  $73.4$ ↑4.2 |  $76.4$ ↑4.1
ChatGPT | $60.4$ | $72.0$ | $71.8$ | 70.7
$+$SR |  $65.8$ ↑5.4 |  $76.0$↑4.0 |  74.6 ↑2.8 |  $45.5$↓25.2
$+$SR★ |  $67.4$↑7.0 |  $78.0$↑6.0 |  $\underline{79.4}$ ↑7.6 |  $\underline{73.2}$↑2.5
$+$MAF |  68.4↑8.0 |  77.8↑5.8 |  $73.2$ ↑1.4 |  ${67.9}$↓2.8
$+$MAF★ |  $\underline{71.7}$ ↑11.3 |  $\underline{82.8}$↑10.8 |  $76.6$ ↑4.8 |  $72.7$↑2.0

Table 2: Experimental results for Entailment Bank (EB), GSMIC, GSM8K, DROP
dataset as described in section 4. SR represents Self-Refine Madaan et al.
(2023), and MAF represents our method. ★ represents the Oracle Verifier
setting (section 4.4). The best score for standard setting is in bold and
underlined for the Oracle Verifier setting.

Figure 3: Accuracy for baselines and MAF on GSM-IC using ChatGPT under
Standard and Oracle Verifier setting section 4.4.

##  4 Experiments

###  4.1 Datasets and Metrics

#### Mathematical Reasoning

The GSM8K dataset, presented by Cobbe et al. (2021), is a comprehensive
compilation of high-quality grade school math problems. GSM8K has proven to be
a great resource for testing LLMs on mathematical word problems, however, the
performance on this dataset has been saturated for a while.

To avoid that problem, we conduct experiments on a harder variant of this
dataset, GSM-IC (Grade-School Math with Irrelevant Context), that was
introduced by Shi et al. (2023). We run our main experiments on a randomly
sampled subset of 500 problems from GSM-IC and use % solve rate as the metric.

#### Logical Reasoning

EntailmentBank, as described by Dalvi et al. (2021), is a dataset that
contains multistep entailment trees. We use the validation set of Task 1
provided by the authors for our experimentation. For the metrics, we do not
use the automated metrics provided by the original work because these metrics
expect the trees to be similar in structure to gold trees. However, that is
not a fair comparison because we find that there exist multiple correct
entailment trees for a given hypothesis and information. Thus, we conduct a
human evaluation and ask humans to evaluate if the hypothesis can be entailed
from the predicted tree.

#### Question Answering

DROP (Dua et al., 2019) is a question-answering dataset. The Discrete
Reasoning Over the Content of Paragraphs (DROP) dataset is designed for
complex question-answering tasks that require multi-step reasoning over text
passages. It presents a valuable benchmark for our iterative refinement
framework, as the multi-step nature of the questions offers ample
opportunities for generating feedback and refining to guide the model toward a
correct solution. We use output parsing for answers and use % correct answers
as the final metric.

###  4.2 Baselines

In this work, we focus on comparing our method with the Base LMs using the
OpenAI API and a recently proposed iterative refinement framework, Self-Refine
(Madaan et al., 2023). To provide a fair comparison and avoid randomness in
the generated answer, we used Greedy Decoding for all our experiments.

We follow (Madaan et al., 2023) and use 8-shot Program of Thought (Chen et
al., 2022) for GSM-IC since the Python program written by the model acts as a
"calculator" for mathematical equations.

The input for Entailment Bank includes a hypothesis and the supporting text,
and we are limited by the context length of the current models. Hence, we use
4-shot prompting for this dataset. We use standard few-shot prompting (Brown
et al., 2020), because the Entailment Tree itself acts like a reasoning chain.

Similarly, the examples in the DROP dataset have a passage and an accompanying
question, so we use a 3-shot Chain of Thought Wei et al. (2023) prompting. We
also provide an instruction specifying that the model should select either a
number, date, or span from the passage to answer the question as shown in
Appendix C.

For Self-Refine, we use the prompts provided by the authors for GSM8K in their
work for GSM-IC and write our own prompts for DROP and Entailment Bank. Self-
Refine has three modules, initial generation, feedback, and refiner. We use
the same parameters and prompting strategy as the corresponding baseline for
the initial generation. The feedback module and refiner module are implemented
using the standard few-shot prompting with 3 in-context examples for all
datasets. Further details about the implementation of these baselines can be
found in Appendix C.

###  4.3 Implementation

We implement MAF (Multi-Aspect Feedback) following the basic structure defined
in section 3 and Algorithm 1. The iterative refinement process continues until
we reach the desired output quality or a task-specific stop criterion, up to a
maximum of 4 iterations. Our method includes an initial generation, refiner,
and feedback modules as shown in Table 1. The initial generation module uses
the same parameters and prompting strategies as the corresponding baseline.
Our Eager-Refine module uses 3-shot standard prompting and our Lazy-Refiner,
which includes feedback from multiple modules, uses a 2-shot standard
prompting approach. Our OpenAI-based feedback modules also use 3-shot standard
prompting following (Madaan et al., 2023) with an error-specific instruction.

To provide a fair comparison, we use the same prompts for Baselines and Self-
Refine wherever possible. We use Greedy Decoding for all our OpenAI-
based444Our experiments with GPT3.5 (Text-Davinci-003) and ChatGPT
(gpt-3.5-turbo) are conducted using OpenAI API between April-2023 and
August-2023. modules to avoid any randomness. As mentioned in section 3.3, we
use a selective summarization approach to fit all the lazy-refine feedbacks in
our refiner module. We use a basic rule-based strategy to summarize feedback.
Since our feedback modules are instructed to inspect each intermediate step,
they also include the steps with no mistakes in the generated feedback.
However, feedback on these lines is not useful as such because there is no
change in those steps. Thus we look for the steps with feedback "looks good"
and remove those steps. This simple approach helps us increase the efficiency
of our method by being able to include multiple feedbacks in our lazy-refiner.

###  4.4 Results

In this section, we discuss the performance of our method and compare it to
the baseline. We also discuss an ablation experiment studying the contribution
of each feedback module. The main results of this work are shown in Table 2.
We find that our method can outperforms the base LMs and Self-Refine on a
diverse set of reasoning datasets. For MAF, we report the results after $2$
iterations, and following Madaan et al. (2023) report the Self-Refine results
after $4$ iterations.

#### GPT3.5 vs ChatGPT

We found that GPT3.5 can generate better feedback if an error is present in
the solution, however often points out spurious errors even when they are not
present. On the other hand, ChatGPT is more conservative in its feedback
generation and often fails to detect the error even in an erroneous solution.
Because of this conflicting behavior from the two models, our method is able
to achieve similar performance using both models, even though ChatGPT is
generally considered to be a better model than GPT3.5.

#### Over-Refining

Due to the behavior described above, we face the problem of Over-Refining.
This means that once we have reached an optimal solution for a reasoning
problem, forcing the LM to refine it further deteriorates the performance and
the quality of the reasoning chain. As shown in Table 2 and Figure 3, Self-
Refine (Madaan et al., 2023) also suffers from this problem in the DROP
dataset. Moreover, all the other iterative refinement framework such as Self-
Refine uses a stop condition and stop the refining process if the generated
feedback does not point out any problem. We however cannot take advantage of
this stop condition because of the interplay between multiple feedback
modules. For example, let’s say our Missing Step module stops at iteration $k$
due to no missing steps at that point in time since the other feedback modules
continue to refine the solution and self-refinement is an imperfect process,
it might introduce a Missing Step error in further iterations.

Hence, we treat the number of iterations as a hyperparameter and find that $2$
iterations work best for our method.

#### Oracle Verifier

Since the main focus of this work is to improve the incorrect initial
generations from the model, we also evaluate the models under an Oracle
Verifier setting. In this setting, we assume access to an "Oracle Verifier"
which can judge the final answer generated by the model. If the final answer
is correct, we stop the refinement process for that test sample, otherwise, we
let the model continue refining the solution. It is important to note however
that the model is not privy to this verification, and hence can still stop
further refinement by not generating actionable feedback in the next
iteration. Under this setting, we report results for both Self-Refine and MAF
after 4 iterations. Even under this modified setting, we see that our method
can outperform Self-Refine because we generate diverse feedback and thus can
improve more solutions in the GSM-IC dataset.

###  4.5 Ablation

In this section, we perform two ablation studies. Firstly, we analyze the
impact of each feedback module, identifying those with the most significant
effect on performance. Secondly, we compare the outcomes of our two proposed
refinement strategies: Lazy vs Eager Refinement. The ablation studies are
conducted on the GSM-IC dataset. Due to resource constraints, we conduct the
following ablation studies on a smaller random subset of size 100.

Model | Standard | Oracle
---|---|---
ChatGPT | 73.0 | $-$
$+$MAF | 79.0 | 84.0
$-$Variable Naming (VN) | 79.0 ↓0.0 | 83.0 ↓1.0
$-$Redundancy (Red) | 70.0 ↓9.0 | 80.0 ↓4.0
$-$Commonsense (Com) | 73.0 ↓6.0 | 80.0 ↓4.0
$-$Missing Step (MS) | 79.0 ↓0.0 | 84.0 ↓0.0
MAF (VN, Red, Com) | 79.0 | 85.0
Table 3: This table shows the contribution of each feedback module in MAF for
the GSM-IC dataset. $-$ symbol in front of a module denotes the accuracy of
our method after removing that feedback module. MAF (VN, Red, Com) shows the
accuracy of our method using only the best-performing feedback modules.

#### Contribution of different Feedback Modules

We test the contribution of each feedback module by removing that module and
calculating the accuracy under standard and Oracle verifier settings. While
the standard setting highlights the potential negative impacts of some of the
modules, the oracle verifier setting highlights the absolute contribution of
each module. This highlights an important finding that using the error
categories is paramount to gaining performance. Results for this ablation
study can be found in Table 3.

We also calculate the accuracy of our method when using the three best-
performing modules as the only source of feedback. We found that this does
recover the performance of our method. Even though Variable Naming and Missing
Step modules do not affect MAFs performance by a huge margin, it still makes
our method more robust to a possible distribution shift. Moreover, the
Variable Naming module’s main contribution is not increasing the performance,
but rather to increase the readability of the code and not confuse users with
unclear names.

Note that we did not remove the Programming Syntax checker module as we use
Program of Thoughts which requires a Python interpreter.

#### Lazy vs Eager Refinement

Model | Standard | Oracle
---|---|---
GPT3.5 | 71.0 | $71.0$
$+$MAF |  $67.0$ ↓4.0 |  85.0 ↑14.0
$+$Only Lazy-Refine |  $66.0$ ↓5.0 |  $81.0$ ↑10.0
$+$Only Eager-Refine |  $66.0$ ↓5.0 |  85.0 ↑14.0
ChatGPT | $73.0$ | $73.0$
$+$MAF |  78.0 ↑5.0 |  81.0 ↑8.0
$+$Only Lazy-Refine |  $69.0$ ↓4.0 |  $80.0$ ↑7.0
$+$Only Eager-Refine |  $73.0$ ↓0.0 |  $80.0$ ↑7.0
Table 4: Accuracy on GSM-IC when using different refinement strategies under
Standard and Oracle settings. MAF shows the performance of our method when
using both Lazy and Eager refine in tandom. Only Lazy-Refine means all
feedback modules use Lazy-Refine and similarly Only Eager-Refine means all
modules use Eager-Refine strategy. The best score for each setting is in bold.

To illustrate the complementary nature of our two proposed refinement
strategies, we evaluate the performance of our iterative refinement framework
using all feedback modules in either Lazy or Eager mode. The results are
presented in Table 4, which showcases the performance of our framework under
different refinement settings. In this table, ’MAF’ corresponds to the results
obtained by combining both Lazy and Eager Refinement strategies, as defined in
Table 1, while ’Only Lazy/Eager-Refine’ displays the results of our framework
when utilizing only one type of refinement strategy.

The results clearly demonstrate that our framework, which leverages a
combination of eager and lazy refinement, consistently matches or outperforms
using either strategy in isolation, across both the standard and oracle
verifier settings.

Practical considerations also favor the use of a hybrid approach incorporating
both eager and lazy feedback. Relying solely on lazy feedback can lead to a
situation where multiple feedback categories are condensed into a single
prompt. Despite our feedback summarization technique, this can result in the
iteration prompt exceeding the context window limit of many widely available
models. Conversely, exclusively employing eager feedback may result in
rewriting the solution for each feedback module, leading to high token usage.
While using all modules in Eager-Refine mode can closely approach the
performance of ’MAF’ (as shown in Table 4), it is not scalable when dealing
with a large number of feedback modules.

##  5 Conclusion

In this work, we present Multi-Aspect Feedback (MAF), a novel iterative
refinement framework that decouples the feedback modules and takes advantage
of error-specific tools to generate feedback. We demonstrate the performance
of our framework on a set of diverse reasoning tasks and compare it with other
iterative refinement baselines.

Contrary to previous works, we found that Over-Refinement can be a problem in
iterative refinement frameworks since models are not certain if their own
answer is correct. Our work also draws focus on the necessity to devise better
feedback methods and call for augmenting Language Models with them. We hope
this work will inspire further research in this area and to this end, we make
all our code, data, and prompts available.

## Limitations

The main limitation of our approach is that the base models need to have
sufficient in-context learning abilities to process the feedback and refine
the solution. Even with in-context learning abilities, these models are not
perfect and thus can still make mistakes while refining the solution even when
correct feedback is given.

All the experiments conducted in this work use large powerful models provided
by OpenAI. We find that open-source LMs such as Vicuna, and Alpaca can
generate decent initial solutions but are not capable of refining their own
solutions. Thus, we leave the investigation of improving open-source models to
future work.

Another limitation inherent in our approach is the reliance on a fixed set of
Feedback Modules. Our method necessitates the pre-selection of feedback
modules before execution, which in turn demands human intervention to
determine the appropriate feedback categories for each new dataset or domain.
Future research could explore novel methods that can dynamically and
autonomously determine the most suitable feedback modules for specific
problems in real-time.

## Ethics Statement

The experiments in this work were performed with models that are not open-
sourced and are ever-changing. Moreover, these models are expensive to use,
and thus research using these models requires an enormous amount of funding.
The existing literature lacks details about the datasets that are being used
to train these huge models or the filtering mechanism that is being used to
clean the polluted corpora.

Furthermore, there is always a possibility for bad actors to use our method to
generate more toxic or harmful text. Our approach does not guard against this.

## Acknowledgements

This work was supported by the National Science Foundation award #2048122. The
views expressed are those of the authors and do not reflect the official
policy or position of the US government. Additionally, we thank our reviewers
for their detailed and useful feedback.

## References

  * Akyurek et al. (2023) Afra Feyza Akyurek, Ekin Akyürek, Aman Madaan, A. Kalyan, Peter Clark, D. Wijaya, and Niket Tandon. 2023.  Rl4f: Generating natural language feedback with reinforcement learning for repairing model outputs.  _ArXiv_ , abs/2305.08844.
  * Brown et al. (2020) Tom B. Brown, Benjamin Mann, Nick Ryder, Melanie Subbiah, Jared Kaplan, Prafulla Dhariwal, Arvind Neelakantan, Pranav Shyam, Girish Sastry, Amanda Askell, Sandhini Agarwal, Ariel Herbert-Voss, Gretchen Krueger, T. J. Henighan, Rewon Child, Aditya Ramesh, Daniel M. Ziegler, Jeff Wu, Clemens Winter, Christopher Hesse, Mark Chen, Eric Sigler, Mateusz Litwin, Scott Gray, Benjamin Chess, Jack Clark, Christopher Berner, Sam McCandlish, Alec Radford, Ilya Sutskever, and Dario Amodei. 2020.  Language models are few-shot learners.  _ArXiv_ , abs/2005.14165.
  * Chen et al. (2022) Wenhu Chen, Xueguang Ma, Xinyi Wang, and William W. Cohen. 2022.  Program of thoughts prompting: Disentangling computation from reasoning for numerical reasoning tasks.
  * Cobbe et al. (2021) Karl Cobbe, Vineet Kosaraju, Mohammad Bavarian, Mark Chen, Heewoo Jun, Lukasz Kaiser, Matthias Plappert, Jerry Tworek, Jacob Hilton, Reiichiro Nakano, Christopher Hesse, and John Schulman. 2021.  Training verifiers to solve math word problems.  _ArXiv_ , abs/2110.14168.
  * Dalvi et al. (2021) Bhavana Dalvi, Peter Jansen, Oyvind Tafjord, Zhengnan Xie, Hannah Smith, Leighanna Pipatanangkura, and Peter Clark. 2021.  Explaining answers with entailment trees.  In _Proceedings of the 2021 Conference on Empirical Methods in Natural Language Processing_ , pages 7358–7370, Online and Punta Cana, Dominican Republic. Association for Computational Linguistics.
  * Dua et al. (2019) Dheeru Dua, Yizhong Wang, Pradeep Dasigi, Gabriel Stanovsky, Sameer Singh, and Matt Gardner. 2019.  DROP: A reading comprehension benchmark requiring discrete reasoning over paragraphs.  In _Proceedings of the 2019 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, Volume 1 (Long and Short Papers)_ , pages 2368–2378, Minneapolis, Minnesota. Association for Computational Linguistics.
  * Golovneva et al. (2022) Olga Golovneva, Moya Chen, Spencer Poff, Martin Corredor, Luke Zettlemoyer, Maryam Fazel-Zarandi, and Asli Celikyilmaz. 2022.  Roscoe: A suite of metrics for scoring step-by-step reasoning.
  * Hao et al. (2023) Shibo Hao, Tianyang Liu, Zhen Wang, and Zhiting Hu. 2023.  Toolkengpt: Augmenting frozen language models with massive tools via tool embeddings.  _ArXiv_ , abs/2305.11554.
  * Karpas et al. (2022) Ehud Karpas, Omri Abend, Yonatan Belinkov, Barak Lenz, Opher Lieber, Nir Ratner, Yoav Shoham, Hofit Bata, Yoav Levine, Kevin Leyton-Brown, Dor Muhlgay, Noam Rozen, Erez Schwartz, Gal Shachaf, Shai Shalev-Shwartz, Amnon Shashua, and Moshe Tenenholtz. 2022.  Mrkl systems: A modular, neuro-symbolic architecture that combines large language models, external knowledge sources and discrete reasoning.
  * Kojima et al. (2022) Takeshi Kojima, Shixiang (Shane) Gu, Machel Reid, Yutaka Matsuo, and Yusuke Iwasawa. 2022.  Large language models are zero-shot reasoners.  In _Advances in Neural Information Processing Systems_ , volume 35, pages 22199–22213. Curran Associates, Inc.
  * Li et al. (2023) Minghao Li, Feifan Song, Bowen Yu, Haiyang Yu, Zhoujun Li, Fei Huang, and Yongbin Li. 2023.  Api-bank: A benchmark for tool-augmented llms.  _ArXiv_ , abs/2304.08244.
  * Madaan et al. (2023) Aman Madaan, Niket Tandon, Prakhar Gupta, Skyler Hallinan, Luyu Gao, Sarah Wiegreffe, Uri Alon, Nouha Dziri, Shrimai Prabhumoye, Yiming Yang, Sean Welleck, Bodhisattwa Prasad Majumder, Shashank Gupta, Amir Yazdanbakhsh, and Peter Clark. 2023.  Self-refine: Iterative refinement with self-feedback.
  * Madaan et al. (2022) Aman Madaan, Shuyan Zhou, Uri Alon, Yiming Yang, and Graham Neubig. 2022.  Language models of code are few-shot commonsense learners.  _ArXiv_ , abs/2210.07128.
  * OpenAI (2023) OpenAI. 2023.  Gpt-4 technical report.  _ArXiv_ , abs/2303.08774.
  * Ouyang et al. (2022) Long Ouyang, Jeff Wu, Xu Jiang, Diogo Almeida, Carroll L. Wainwright, Pamela Mishkin, Chong Zhang, Sandhini Agarwal, Katarina Slama, Alex Ray, John Schulman, Jacob Hilton, Fraser Kelton, Luke E. Miller, Maddie Simens, Amanda Askell, Peter Welinder, Paul Francis Christiano, Jan Leike, and Ryan J. Lowe. 2022\.  Training language models to follow instructions with human feedback.  _ArXiv_ , abs/2203.02155.
  * Paranjape et al. (2023) Bhargavi Paranjape, Scott Lundberg, Sameer Singh, Hannaneh Hajishirzi, Luke Zettlemoyer, and Marco Tulio Ribeiro. 2023.  Art: Automatic multi-step reasoning and tool-use for large language models.
  * Patil et al. (2023) Shishir G. Patil, Tianjun Zhang, Xin Wang, and Joseph E. Gonzalez. 2023.  Gorilla: Large language model connected with massive apis.  _ArXiv_ , abs/2305.15334.
  * Paul et al. (2023) Debjit Paul, Mete Ismayilzada, Maxime Peyrard, Beatriz Borges, Antoine Bosselut, Robert West, and Boi Faltings. 2023.  Refiner: Reasoning feedback on intermediate representations.  _ArXiv_ , abs/2304.01904.
  * Peng et al. (2023) Baolin Peng, Michel Galley, Pengcheng He, Hao Cheng, Yujia Xie, Yu Hu, Qiuyuan Huang, Lars Liden, Zhou Yu, Weizhu Chen, and Jianfeng Gao. 2023.  Check your facts and try again: Improving large language models with external knowledge and automated feedback.
  * Schick et al. (2023) Timo Schick, Jane Dwivedi-Yu, Roberto Dessì, Roberta Raileanu, Maria Lomeli, Luke Zettlemoyer, Nicola Cancedda, and Thomas Scialom. 2023.  Toolformer: Language models can teach themselves to use tools.
  * Schick et al. (2022) Timo Schick, Jane Dwivedi-Yu, Zhengbao Jiang, Fabio Petroni, Patrick Lewis, Gautier Izacard, Qingfei You, Christoforos Nalmpantis, Edouard Grave, and Sebastian Riedel. 2022.  Peer: A collaborative language model.  _ArXiv_ , abs/2208.11663.
  * Shen et al. (2023) Yongliang Shen, Kaitao Song, Xu Tan, Dong Sheng Li, Weiming Lu, and Yue Ting Zhuang. 2023.  Hugginggpt: Solving ai tasks with chatgpt and its friends in hugging face.  _ArXiv_ , abs/2303.17580.
  * Shi et al. (2023) Freda Shi, Xinyun Chen, Kanishka Misra, Nathan Scales, David Dohan, Ed Chi, Nathanael Schärli, and Denny Zhou. 2023.  Large language models can be easily distracted by irrelevant context.
  * Shinn et al. (2023) Noah Shinn, Federico Cassano, Beck Labash, Ashwin Gopinath, Karthik Narasimhan, and Shunyu Yao. 2023.  Reflexion: Language agents with verbal reinforcement learning.
  * Taori et al. (2023) Rohan Taori, Ishaan Gulrajani, Tianyi Zhang, Yann Dubois, Xuechen Li, Carlos Guestrin, Percy Liang, and Tatsunori B. Hashimoto. 2023.  Stanford alpaca: An instruction-following llama model.  https://github.com/tatsu-lab/stanford_alpaca.
  * Touvron et al. (2023) Hugo Touvron, Thibaut Lavril, Gautier Izacard, Xavier Martinet, Marie-Anne Lachaux, Timothée Lacroix, Baptiste Rozière, Naman Goyal, Eric Hambro, Faisal Azhar, Aur’elien Rodriguez, Armand Joulin, Edouard Grave, and Guillaume Lample. 2023.  Llama: Open and efficient foundation language models.  _ArXiv_ , abs/2302.13971.
  * Wang et al. (2022) Xuezhi Wang, Jason Wei, Dale Schuurmans, Quoc Le, Ed Huai hsin Chi, and Denny Zhou. 2022.  Self-consistency improves chain of thought reasoning in language models.  _ArXiv_ , abs/2203.11171.
  * Wei et al. (2023) Jason Wei, Xuezhi Wang, Dale Schuurmans, Maarten Bosma, Brian Ichter, Fei Xia, Ed Chi, Quoc Le, and Denny Zhou. 2023.  Chain-of-thought prompting elicits reasoning in large language models.
  * Wu et al. (2023) Zeqiu Wu, Yushi Hu, Weijia Shi, Nouha Dziri, Alane Suhr, Prithviraj Ammanabrolu, Noah A. Smith, Mari Ostendorf, and Hanna Hajishirzi. 2023.  Fine-grained human feedback gives better rewards for language model training.
  * Yang et al. (2023) Rui Yang, Lin Song, Yanwei Li, Sijie Zhao, Yixiao Ge, Xiu Li, and Ying Shan. 2023\.  Gpt4tools: Teaching large language model to use tools via self-instruction.  _ArXiv_ , abs/2305.18752.
  * Zhou et al. (2023) Denny Zhou, Nathanael Schärli, Le Hou, Jason Wei, Nathan Scales, Xuezhi Wang, Dale Schuurmans, Claire Cui, Olivier Bousquet, Quoc Le, and Ed Chi. 2023.  Least-to-most prompting enables complex reasoning in large language models.

##  Appendix A Error Categories

In this section we define all the error categories that are implemented in
this work. Table 5 shows the error categories introduced by Golovneva et al.
(2022).

###  A.1 Programming Syntax Feedback

Programming syntax feedback module is implemented as Python Interpreter. This
module aims to fix any syntax errors in the generated code. This particular
module benefits from a Eager-Refinement approach.

###  A.2 Variable Naming Feedback

Good variable names in a code can improve the readability of the code and
potentially can improve model’s own understanding of the program in further
iterations. Variable Naming Feedback is another module which benefits from an
eager-refine approach.

###  A.3 Redundancy Feedback

Redundant information is any information included in the reasoning that
doesn’t help answer the question. This additional information may distract the
model from correctly answering and should thus be removed.

###  A.4 Commonsense Feedback

Commonsense reasoning errors are errors about any relation or knowledge that
is should be known from general world such as "all ducks are birds".

###  A.5 Missing Steps

Missing steps errors are any gaps in reasoning or missing information that
prevent the reasoning chain from being correct. This also identifies the model
saying that the question is unanswerable as a missing step error because that
means additional reasoning steps are needed to answer the question from the
passage.

###  A.6 Factuality Feedback

Factuality errors are cases where the answer reasoning states infactual
information. This could be information that contradicts information given in
the passage or hallucinated.

###  A.7 Hallucination Feedback

LLMs are prone to hallucination, however, it has been shown that sampling from
a LLM multiple times and then selecting the majority answer can improve the
results. Thus Hallucination feedback aims to fix any hallucinated facts in the
initial generation. This module can be improved by using an external tool such
as a Knowledge Source instead of a LLM.

##  Appendix B Implementation Parameters

Table 7 provides parameters used for Self-Refine and Table 7 provides
parameters used for MAF.

##  Appendix C Few-Shot Prompt Examples

We add samples for all the prompts used in our work. Complete prompts can be
found in our source code. Note that for all feedback prompt examples, there is
at least one example with no errors, in which case the feedback will state
that there are no errors and the reasoning is correct. This helps decrease the
likelihood that the model identifies an error in a solution that is actually
correct.

Error Type |  Definition
---|---
Programming Syntax |  Syntax errors in code
Arithmetic |  Error in math calculations
Grammar |  Faulty, unconventional, or controversial grammar usage
Coherency |  Steps contradict each other or do not follow a cohesive story
Variable Naming |  Variable names in a program don’t give full information or are wrong
Repetition |  Step paraphrases information already mentioned in previous reasoning steps
Hallucination |  Information is not provided in the problem statement and is irrelevant or wrong
Commonsense |  Model lacks relations that should be known from the general world (e.g., "1 dozen = 12")
Factuality |  Information about an object (i.e. quantity, characteristics) or a named entity doesn’t match with the input context
Missing Step |  The content of the generated reasoning is incomplete and lacks the required information to produce the correct answer
Redundancy |  Explanation contains redundant information, which even though might be factual, is not required to answer the question
Table 5: Error types as defined in Golovneva et al. (2022). Each error
category is defined for a single step in the reasoning chain.

Dataset | Base LM | Feedback | Refiner
---|---|---|---
EB | 300 | 600 | 600
GSM-IC | 300 | 600 | 600
GSM8K | 300 | 600 | 600
DROP | 450 | 600 | 800
Table 6: Value of Maximum Number of Tokens parameter for Self-Refine on
various datasets.

Dataset | Base LM | Feedback | Refiner
---|---|---|---
EB | 300 | 600 | 600
GSM-IC | 300 | 600 | 600
GSM8K | 300 | 600 | 600
DROP | 450 | 600 | 800
Table 7: Value of Maximum Number of Tokens parameter for MAF on various
datasets.

Figure 4: Initialization prompt for Mathematical Reasoning Figure 5:
Commonsense Feedback for Mathematical Reasoning Figure 6: Missing Step
Feedback for Mathematical Reasoning Figure 7: Redundancy Feedback for
Mathematical Reasoning Figure 8: Variable Naming Feedback for Mathematical
Reasoning Figure 9: Our Iterative Refinement prompt for Mathematical Reasoning
Figure 10: Iterative Refinement prompt from Self-Refine for Mathematical
Reasoning Figure 11: Commonsense Feedback for Question Answering Figure 12:
Factuality Feedback for Question Answering Figure 13: Missing Step Feedback
for Question Answering Figure 14: Redundancy Feedback for Question Answering
Figure 15: Self-Refine style Iterative Refinement prompt for Question
Answering Figure 16: Initialization prompt for Question Answering


∎

corrauthor _correspondence_<EMAIL_ADDRESS>alsoMEPHI _also at:_ NRNU
MEPhI, Moscow, Russia nowDuke _now at:_ Duke University, Durham, NC USA
alsoLev _also at:_ Moscow Inst. of Physics and Technology, Russia alsoDubna
_also at:_ Dubna State University, Dubna, Russia alsoRM3 _also at:_ INFN Roma
Tre, Rome, Italy 11institutetext: INFN Laboratori Nazionali del Gran Sasso,
Assergi, Italy 22institutetext: INFN Laboratori Nazionali del Gran Sasso and
Gran Sasso Science Institute, Assergi, Italy 33institutetext: INFN Laboratori
Nazionali del Gran Sasso and Università degli Studi dell’Aquila, L’Aquila,
Italy 44institutetext: INFN Laboratori Nazionali del Sud, Catania, Italy
55institutetext: Institute of Physics, Jagiellonian University, Cracow, Poland
66institutetext: Institut für Kern- und Teilchenphysik, Technische Universität
Dresden, Dresden, Germany 77institutetext: Joint Institute for Nuclear
Research, Dubna, Russia 88institutetext: European Commission, JRC-Geel, Geel,
Belgium 99institutetext: Max-Planck-Institut für Kernphysik, Heidelberg,
Germany 1010institutetext: Department of Physics and Astronomy, University
College London, London, UK 1111institutetext: INFN Milano Bicocca, Milan,
Italy 1212institutetext: Dipartimento di Fisica, Università degli Studi di
Milano and INFN Milano, Milan, Italy 1313institutetext: Institute for Nuclear
Research of the Russian Academy of Sciences, Moscow, Russia 1414institutetext:
Institute for Theoretical and Experimental Physics, NRC “Kurchatov Institute”,
Moscow, Russia 1515institutetext: National Research Centre “Kurchatov
Institute”, Moscow, Russia 1616institutetext: Max-Planck-Institut für Physik,
Munich, Germany 1717institutetext: Physik Department, Technische Universität
München, Germany 1818institutetext: Dipartimento di Fisica e Astronomia,
Università degli Studi di Padova, Padua, Italy 1919institutetext: INFN Padova,
Padua, Italy 2020institutetext: Physikalisches Institut, Eberhard Karls
Universität Tübingen, Tübingen, Germany 2121institutetext: Physik-Institut,
Universität Zürich, Zurich, Switzerland

# An improved limit on the neutrinoless double-electron capture of 36Ar with
GERDA

The Gerda collaborationcorrauthor
M. AgostiniUCL  A. AlexanderUCL  G.R. AraujoUZH  A.M. BakalyarovKU  M.
BalataALNGS  I. BarabanovINRM  L. BaudisUZH  C. BauerHD  S.
BelogurovITEP,INRM,alsoMEPHI  A. BettiniPDUNI,PDINFN  L. BezrukovINRM  V.
BiancacciLNGSGSSI  E. BossioTUM  V. BotheHD  V. BrudaninJINR  R.
BrugneraPDUNI,PDINFN  A. CaldwellMPIP  C. CattadoriMIBINFN  A.
ChernogorovITEP,KU  T. ComellatoTUM  V. D’AndreaLNGSAQU,alsoRM3  E.V.
DemidovaITEP  N. Di MarcoLNGSGSSI  E. DoroshkevichINRM  F. FischerMPIP  M.
FominaJINR  A. GangapshevINRM,HD  A. GarfagniniPDUNI,PDINFN  C. GoochMPIP  P.
GrabmayrTUE  V. GurentsovINRM  K. GusevJINR,KU,TUM  J. HakenmüllerHD,nowDuke
S. HemmerPDINFN  W. HofmannHD  J. HuangUZH  M. HultGEEL  L.V.
InzhechikINRM,alsoLev  J. Janicskó CsáthyTUM  J. JochumTUE  M. JunkerALNGS  V.
KazalovINRM  Y. KermaïdicHD  H. KhushbakhtTUE  T. KihmHD  K. KilgusTUE  I.V.
KirpichnikovITEP  A. KlimenkoHD,JINR,alsoDubna  R. KneißlMPIP  K.T. KnöpfleHD
O. KochetovJINR  V.N. KornoukhovITEP,INRM,alsoMEPHI  M. KorošecTUM  P.
KrauseTUM  V.V. KuzminovINRM  M. LaubensteinALNGS  M. LindnerHD  I.
LippiPDINFN  A. LubashevskiyJINR  B. LubsandorzhievINRM  G. LutterGEEL  C.
MacolinoLNGSAQU  B. MajorovitsMPIP  W. ManeschgHD  L. ManzanillasMPIP  G.
MarshallUCL  M. MisiaszekCR  M. MorellaLNGSGSSI  Y. MüllerUZH  I.
NemchenokJINR,alsoDubna  L. PandolaCAT  K. PelczarGEEL  L. PertoldiTUM,PDINFN
P. PiseriMILUINFN  A. PulliaMILUINFN  C. RansomUZH  L. RauscherTUE  M.
RedchukPDINFN  S. RiboldiMILUINFN  N. RumyantsevaKU,JINR  C. SadaPDUNI,PDINFN
F. SalamidaLNGSAQU  S. SchönertTUM  J. SchreinerHD  M. SchüttHD  A-K.
SchützTUE  O. SchulzMPIP  M. SchwarzTUM  B. SchwingenheuerHD  O.
SelivanenkoINRM  E. ShevchikJINR  M. ShirchenkoJINR  L. ShtembariMPIP  H.
SimgenHD  A. SmolnikovHD,JINR  D. StukovKU  A.A. VasenkoITEP  A.
VeresnikovaINRM  C. VignoliALNGS  K. von SturmPDUNI,PDINFN  T. WesterDD  C.
WiesingerTUM  M. WojcikCR  E. YanovichINRM  B. ZatschlerDD  I. ZhitnikovJINR
S.V. ZhukovKU  D. ZinatulinaJINR  A. ZschockeTUE  A.J. ZsigmondMPIP  K.
ZuberDD  and G. ZuzelCR

###### Abstract

The GERmanium Detector Array (Gerda) experiment operated enriched high-purity
germanium detectors in a liquid argon cryostat, which contains 0.33% of 36Ar,
a candidate isotope for the two-neutrino double-electron capture (2$\nu$ECEC)
and therefore for the neutrinoless double-electron capture (0$\nu$ECEC). If
detected, this process would give evidence of lepton number violation and the
Majorana nature of neutrinos. In the radiative 0$\nu$ECEC of 36Ar, a
monochromatic photon is emitted with an energy of 429.88 keV, which may be
detected by the Gerda germanium detectors. We searched for the 36Ar 0$\nu$ECEC
with Gerda data, with a total live time of 4.34 yr (3.08 yr accumulated during
Gerda Phase II and 1.26 yr during Gerda Phase I). No signal was found and a
90% C.L. lower limit on the half-life of this process was established
$T_{1/2}>1.5\cdot 10^{22}$ yr.

††journal: Eur. Phys. J. C

##  1 Introduction

The simultaneous capture of two bound atomic electrons followed by the
emission of two neutrinos plus X-rays or Auger electrons, known as two-
neutrino double-electron capture (2$\nu$ECEC), is a nuclear process allowed in
the Standard Model. Compared to the two-neutrino double-beta
($2\nu\beta\beta$) decay, the simultaneous emission of two electrons and two
anti-neutrinos, 2$\nu$ECEC processes have lower probabilities due to the
smaller phase space, therefore experimentally, they are much more challenging
to observe. The first direct observation of 2$\nu$ECEC was made only in 2018
by the XENON1T experiment with 124Xe XenonCollaboration2019 . Previously,
indications of 2$\nu$ECEC were found in geochemical measurements with 130Ba
and 132Ba Meshik2001  and in a large proportional counter experiment with 78Kr
Ratk2017 .

The lepton number violating counterpart of 2$\nu$ECEC, the neutrinoless
double-electron capture (0$\nu$ECEC), in which no neutrinos are emitted, is
also predicted Winter1955 . This process must be accompanied by the emission
of at least another particle to ensure energy and momentum conservation.
Different modes can be considered in which 0$\nu$ECEC is associated with the
emission of different particles like $e^{+}e^{-}$ pairs, one or two photons,
or one internal conversion electron Doi1993 ; Blaum2020 . In analogy with the
neutrinoless double-beta ($0\nu\beta\beta$) decay, the 0$\nu$ECEC violates the
lepton number symmetry by two units and implies that neutrinos have a Majorana
mass component Georgi1981 . Although the sensitivity of 0$\nu$ECEC processes
to the Majorana neutrino mass is estimated to be many orders of magnitude
lower than that of the $0\nu\beta\beta$ decay, the interest in 0$\nu$ECEC is
theoretically motivated by the possibility of resonant enhancement when the
parent nucleus and an excited state of the daughter nucleus are energetically
degenerate Winter1955 ; Blaum2020 ; Georgi1981 ; Voloshin1982 ; Bernabeu1983 .
In this case, the half-life of 0$\nu$ECEC processes becomes comparable to that
of $0\nu\beta\beta$ decays. Experimental searches for 0$\nu$ECEC have been
performed by double-$\beta$ decay experiments, even though with less
sensitivity compared to the search for $0\nu\beta\beta$ decay Blaum2020 .

The GERmanium Detector Array (Gerda) experiment, whose main goal was to search
for the $0\nu\beta\beta$ decay of 76Ge GERDA:2012qwd ; GERDA:2020xhi ,
operated enriched high purity germanium detectors in a liquid argon (LAr)
cryostat, which naturally contains the 36Ar isotope with an isotopic abundance
of 0.33%. 36Ar can undergo 2$\nu$ECEC to the ground state of 36S Tretyak1995 .
The corresponding lepton number violating process, 0$\nu$ECEC, may occur via
the simplest radiative mode111Given the available energy of the process, the
internal conversion mode would also be allowed for 36Ar. Nevertheless, the
latter is strongly suppressed due to argon’s low atomic number and the
relatively high $\gamma$ energy Merle2009 .

$\text{${}^{36}$Ar}\rightarrow\,^{36}\text{S}+\gamma\;+\text{X}_{\text{K}}\;+\text{X}_{\text{L}}\;.$
(1)

The 36Ar nucleus captures one electron each from its K- and L-shells and turns
into 36S. Two X-rays are emitted, with energies E${}_{\text{K}}=2.47$ keV, and
E${}_{\text{L}}=0.23$ keV, corresponding to the capture of the electrons from
the K- and the L-shell, respectively. Given the available energy of the decay
Q${}_{\text{ECEC}}=(432.58\pm 0.19)$ keV Wang2012 , the corresponding energy
for the $\gamma$ ray is
E${}_{\gamma}=\text{Q}_{\text{ECEC}}-\text{E}_{\text{K}}-\text{E}_{\text{L}}=(429.88\pm
0.19)$ keV. Resonance enhancement of the process is not possible for 36Ar
Blaum2020 . In the light neutrino exchange scenario, assuming a Majorana mass
of 0.1 eV, the half-life of 36Ar 0$\nu$ECEC is predicted in the order of
$10^{40}$ yr, with calculations based on the quasiparticle random-phase
approximation (QRPA) Merle2009 . Experimental searches for 0$\nu$ECEC of 36Ar
have been performed since the early stages of the Gerda experiment
Chkvorets2008 . The most stringent limit to date on the 36Ar 0$\nu$ECEC half-
life is T${}_{1/2}>3.6\times 10^{21}$ yr (90% C.I.), established in Phase I of
the Gerda experiment Agostini2016 . More recently, this process has been
searched with the DEAP detector Dunford2018 , although with less sensitivity
than Gerda Phase I.

In this paper, we report on the search for the 429.88 keV $\gamma$ line from
the 36Ar 0$\nu$ECEC with the whole Gerda data, accumulated for a total live
time of 3.08 yr during Gerda Phase II and 1.26 yr during Gerda Phase I.

##  2 The GERDA experiment

The Gerda experiment was located at the Laboratori Nazionali del Gran Sasso
(LNGS) of INFN, in Italy GERDA:2012qwd ; GERDA:2017ihb ; Agostini:2019hzm ,
where a rock overburden of 3500 m water equivalent reduces the flux of cosmic
muons by six orders of magnitude GERDA:2012qwd . High-purity germanium (HPGe)
detectors, isotopically enriched in 76Ge, were operated inside a 64 m3 LAr
cryostat Knopfle:2022fso . In the second phase of the experiment, 10 coaxial
(including 3 detectors with natural isotopic abundance) and 30 Broad Energy
Germanium (BEGe) detectors were used GERDA:2017ihb . After an upgrade in May
2018, the three natural coaxial detectors were removed, and 5 additional
inverted coaxial (IC) detectors were installed GERDA:2020xhi . Detectors were
mounted on 7 strings, and each string was placed inside a nylon cylinder to
limit the collection of radioactive potassium ions on the detector surfaces
Lubashevskiy:2017lmf . The LAr volume around the detectors was instrumented
with a curtain of wavelength-shifting fibers connected to silicon photo-
multipliers (SiPM) and 16 cryogenic photo-multiplier tubes (PMTs) to detect
scintillation light in the LAr Janicsko-Csathy:2010uif ; GERDA:2017ihb .
During the upgrade, the geometrical coverage of the fibers was improved, more
SiPM channels were added, and their radiopurity increased GERDA:2020xhi . The
cryostat was surrounded by a water tank containing 590 m3 of pure water,
equipped with PMTs to detect the Cherenkov light of residual cosmic muons
reaching the detector site. The instrumented water tank formed, together with
scintillator panels on the top of the experiment, the muon veto system
Freund:2016fhz .

##  3 Data selection

The Gerda Phase II data taking started in December 2015; it was shortly
interrupted in the Summer of 2018 for the upgrade of the setup and lasted
until November 2019. The total collected data used to search for the 429.88
keV $\gamma$ line from the 0$\nu$ECEC of 36Ar corresponds to a live time of
3.08 yr, divided into 1.91 yr before the upgrade and 1.17 yr after the
upgrade. Due to the different detector properties, _e.g._ energy resolution
and efficiency, and the changes in the detector configuration during the
upgrade, data were split into 5 data sets, namely pre-upgrade enrBEGe, pre-
upgrade enrCoax, post-upgrade enrBEGe, post-upgrade enrCoax, and post-upgrade
enrIC. The natCoax detectors were excluded from the analysis since they have a
low duty factor due to their unstable operation in Gerda Phase II and made up
a minimal amount of the exposure.

Data have been processed following the procedures and digital signal
processing algorithms described in Agostini:2011mh . The energy of an event is
reconstructed using a zero-area-cusp filter GERDA:2015rik . Events must pass
several quality cuts based on the flatness of the baseline, polarity, and time
structure of the pulse to reject non-physical events. The acceptance
efficiency of physical events by quality cuts is larger than 99.9%
GERDA:2020xhi . Events preceded by a trigger in the muon-veto system within 10
$\mu$s are also discarded, with negligible induced dead time ($<$0.01%)
GERDA:2020xhi .

The experimental signature used to search for 36Ar 0$\nu$ECEC in the Gerda
data corresponds to the full energy deposition of the $\gamma$ ray in one
germanium detector. Neglecting the energy deposition of the two X-rays, no
coincident energy deposition is expected, neither in the other germanium
detectors nor the LAr. Consequently, the detector anti-coincidence cut and the
LAr veto cut were also applied. The energy of the two X-rays is low enough
that, even if they reached the germanium detector surface, they could not
penetrate the 1–2 mm dead layer and, therefore, not be detected by the
germanium detector. Nevertheless, since they deposit their energy in the LAr,
they could be seen by the LAr instrumentation and trigger the LAr veto. The
corresponding event would escape the data selection. This effect is considered
in the total detection efficiency, as will be explained in section 5. The LAr
veto cut reduces the background in the region of interest of this analysis by
a factor of $\sim 2$, as can be seen in Figure 1. In this energy region, 39Ar
$\beta$ decay dominates up to the endpoint at 565 keV, while $2\nu\beta\beta$
decay is the second dominant contribution. The Pulse Shape Discrimination
(PSD) cut, successfully employed in the search for $0\nu\beta\beta$ decay
Agostini2022 , is unsuitable for this analysis and, therefore, not used. In
fact, $\gamma$ rays mostly result in multiple separated energy depositions in
the germanium detector, i.e. multi-site events, in contrast to the single-site
events produced in the $0\nu\beta\beta$ decay. In addition, the performances
of the PSD cut at the energy of interest of this analysis are poorly known.
Consequently, part of the data excluded in the $0\nu\beta\beta$ decay analysis
from enrBEGe and enrIC data sets because of the PSD cut was instead included
here.

We combine the analysis of Gerda Phase II data with that of Gerda Phase I data
reported in Agostini2016 . The Gerda Phase I data taking started in November
2011 and lasted until May 2013. The total collected data used for searching
for 0$\nu$ECEC of 36Ar corresponded to a live time of 1.26 yr and was divided
into three data sets, namely enrCoax, enrBEGe, and natCoax. More details on
the data processing and selection of these three data sets can be found in
Agostini2016 . It has to be noticed that the instrumentation of the LAr volume
is a unique feature of Gerda Phase II and that no LAr veto cut was available
in Gerda Phase I.

Figure 1: Energy distribution of the low energy Gerda Phase II data before and
after LAr veto cut. The left part of the spectrum is dominated by the 39Ar
$\beta$ decay with an endpoint at 565 keV. On the right side, the
$2\nu\beta\beta$ decay dominates. Some known $\gamma$ lines are visible and
labeled. The orange dotted line indicates the energy at which the 36Ar
0$\nu$ECEC is expected, and the orange band indicates the energy region used
in the analysis.

##  4 Energy resolution and energy scale

The energy calibration of the Gerda germanium detectors was performed during
dedicated weekly calibration runs in which the germanium detectors were
exposed to three 228Th sources GERDA:2021pcs . All calibration data were
combined as detailed in GERDA:2021pcs  to determine the energy scale and
resolution throughout the experiment.

This work uses the effective resolution curves calculated for the five
analysis data sets Ransom:2021fcg . The resolution curves are evaluated at the
36Ar 0$\nu$ECEC $\gamma$ energy of 429.88 keV. The energy resolution in full
width at half maximum (FWHM) and their uncertainties are summarized in table
1. The uncertainty on the FWHM is calculated assuming the same relative
uncertainty as for the FWHM at the $Q_{\beta\beta}$ of the 76Ge
$0\nu\beta\beta$ decay ($Q_{\beta\beta}$ = 2039 keV). This was calculated in
GERDA:2021pcs  as exposure-weighted standard deviation. The picture might be
different at low energy, and the results obtained for the $0\nu\beta\beta$
decay peak at 2039 keV might not be valid for the 0$\nu$ECEC peak at 429.88
keV. In fact, the lowest energy peak used to determine the resolution curves
above is the 583 keV 208Tl peak, above the energy region of interest in this
analysis. To cross-check the energy resolution at the energy of interest, we
use the results of the special low-energy calibration performed at the end of
the Gerda data taking. This calibration run aimed to study the energy scale
and stability at low energy. The energy threshold was set to 100 keV (while it
was 400 keV during regular calibration runs), allowing to extend the energy
range in which the resolution curve is calculated to about 238 keV, the energy
of the first 212Pb $\gamma$ peak usable for the calibration. We use the peak
at 583 keV as a proxy for the 0$\nu$ECEC peak, being the closest in energy. We
should note that also the topology of the events for the two peaks is the
same. In both cases, it is a full energy deposition of the $\gamma$ energy in
one germanium detector, with the $\gamma$ ray starting in the surrounding of
the detector array. We calculate the residuals on the FWHM as the difference
between the FWHM extracted in the special low-energy calibration and the value
obtained evaluating the resolution curves above at 583 keV. The residuals for
each detector are shown in a histogram at the left-handed side of figure 2. We
find no systematic deviation of the FWHM at this energy compared to the
resolution curves. The RMS of the residuals is 0.049 keV, with only one
detector with a larger residual of -0.2 keV.222This is a enrCoax detector, so
the result is compatible with the larger FWHM uncertainty of the post-upgrade
enrCoax data set.

Figure 2: (Left) Distribution of the energy resolution (FWHM) residuals for
the 583 keV calibration peak. (Right) Distribution of the peak position
residuals for the same calibration peak. The mean and the RMS of the two
distributions are indicated.

The monitoring of the energy scale for the $0\nu\beta\beta$ decay search was
performed using the single escape peak of 208Tl at 2103 keV, which is
typically used as a proxy for the $0\nu\beta\beta$ decay peak at
$Q_{\beta\beta}$. The residuals between the peak position after energy
calibration and the nominal energy value were evaluated over time, giving a
mean energy bias of -0.07 keV with an average uncertainty of 0.17 keV
GERDA:2021pcs . To cross-check the energy bias at the energy of interest, we
use the results of the special low energy calibration run and the 583 keV peak
as a proxy for the 0$\nu$ECEC peak again. We calculate the residuals on the
peak position as the difference between the nominal energy value and the
energy value extracted from the special low-energy calibration. The residuals
for each detector are shown in a histogram at the right-handed side of figure
2. We find a mean energy bias of 0.03 keV with a RMS among detectors of 0.084
keV. This is below the estimated bias uncertainty of 0.17 keV for the
$0\nu\beta\beta$ decay peak at $Q_{\beta\beta}$. It should be noted that these
biases are well below the binning of 1 keV used in the analysis. The effect is
therefore expected to be marginal. In this work, we adopt a mean energy bias
of 0 keV with an uncertainty of 0.1 keV for all the five analysis data sets.

Table 1: Energy resolution (FWHM) and $\gamma$ detection efficiency (multiplied by the simulated mass of LAr) for the analysis data sets. The values for the Phase I data sets are taken from Agostini2016 . Data set | FWHM (keV) |  $\varepsilon_{\gamma}\cdot m_{LAr}$ (kg)
---|---|---
Phase II pre-upgrade |
enrBEGe | $2.2\pm 0.2$ | $2.04\pm 0.06$
enrCoax | $2.7\pm 0.2$ | $1.72\pm 0.06$
Phase II post-upgrade |
enrBEGe | $1.74\pm 0.09$ | $2.51\pm 0.07$
enrCoax | $3.1\pm 1.3$ | $1.29\pm 0.06$
enrIC | $1.72\pm 0.07$ | $0.841\pm 0.006$
Phase I |
enrCoax | $3.72\pm 0.05$ | $1.79\pm 0.18$
enrBEGe | $2.01\pm 0.10$ | $0.281\pm 0.018$
natCoax | $4.08\pm 0.20$ | $0.739\pm 0.073$

##  5 Detection efficiency

The $\gamma$ detection efficiency is defined as the probability that a 429.88
keV $\gamma$ ray entirely deposits its energy inside a single germanium
detector. This was determined via Monte Carlo simulations with the
Geant4-based MaGe framework Agostinelli2003 ; Boswell2011 . In total,
$10^{10}\;\gamma$ rays with an energy of 429.88 keV were generated in a
cylindrical volume of LAr, with a radius of 1.5 m and a height of 2.5 m,
around the detector array. This corresponds to a net volume of LAr, after
taking into account the volume occupied by the germanium detectors and
structural materials, of 17.657 m3. The corresponding LAr mass, given the LAr
density of 1385 kg/m3, is 24459 kg. The contribution from $\gamma$ rays
originating from outside this volume to the detection efficiency is
negligible, as shown in figure 3. The projected distribution of vertices from
which the simulated $\gamma$ rays originate is shown in blue for all the
events resulting in an energy deposition in the germanium detectors and black
for the events resulting in the deposition of the entire 429.88 keV $\gamma$
energy in one germanium detector.

Figure 3: Projected distribution of vertices from which the simulated $\gamma$
rays originate. $\gamma$ rays with an energy of 429.88 keV are simulated
uniformly in the cylindrical volume. Only those originating from the blue
vertices deposit some energy in the germanium detectors, while only those
originating from the back vertices deposit the entire energy in one germanium
detector, thus contributing to the $\gamma$ detection efficiency.

Only the last contribute to the $\gamma$ detection efficiency, defined for
each data set as the ratio between the number of events in which the full
energy is deposited in one germanium detector in the specific data set and the
number of initially simulated events. The number of simulated events is high
enough that the statistical uncertainties on these quantities are negligible.
Detector active volume and the status of each detector over the whole data
taking are considered in the simulation, as detailed in GERDA:2019cav . The
dominant systematic uncertainty on the $\gamma$ detection efficiency comes
from the detector active volume uncertainty. This is estimated by varying the
detector dead layer in the simulation by $\pm 1\,\sigma$, where $\sigma$ is
the dead layer uncertainty, and evaluating the impact on the efficiency.
Typical sizes of the detector dead layers are 1–2 mm known with a typical
uncertainty of 5–30 % GERDA:2019vry . The corresponding systematic uncertainty
on the $\gamma$ detection efficiency is 3% for enrBEGe detectors, 4% for
enrCoax detectors, and 1% for enrIC detectors. The $\gamma$ detection
efficiencies multiplied by the mass of LAr in the simulation volume, together
with their uncertainties, are summarized in table 1 for the different data
sets.

The two X-rays that are emitted in the process being searched for are
neglected in the simulations. As anticipated in section 3, their energy
deposition in LAr could trigger the LAr veto. To account for this possibility,
the survival probability of the two X-rays to the LAr veto cut is evaluated
and combined with the $\gamma$ detection efficiency. We use the Gerda photon
detection probability map developed in GERDA:2022hxs  to estimate the
probability $p(x,y,z)$ to detect scintillation light for each simulated event
starting at position $(x,y,z)$ and corresponding to a full $\gamma$ energy
deposition. From this probability, the number of photons $n$ produced by the
two X-rays of total energy $\text{E}_{\text{X-rays}}$ = (2.47 + 0.23) keV is
obtained:

$\displaystyle n$ $\displaystyle=\text{E}_{\text{X-rays}}\cdot
28.12\,\frac{\text{photons}}{\text{keV}}\cdot p(x,y,z)\;,$ (2)

where 28.12 is the number of photons produced for an energy deposition of 1
keV expected in the Gerda LAr GERDA:2022hxs . The probability P that the
corresponding event survives the LAr veto cut is the Poisson probability
$\mathcal{P}(0,n)$.333Where the probability mass function for a Poisson
variable is defined as
$\mathcal{P}(k,\lambda)=\frac{\lambda^{k}e^{-\lambda}}{k!}$. The mean survival
probability is obtained by averaging the survival probabilities of the events
corresponding to a full $\gamma$ energy deposition and results in
$\overline{P}=0.957$. Thus, the data selection discards almost 5% of the
events due to the X-rays depositing their energy in LAr. The calculation of
the survival probability assumes that the two X-rays deposit all the energy at
the exact point where the $\gamma$ ray is emitted. This assumption is
considered valid since the attenuation length for a 3 keV X-ray was estimated
to be about 42 $\mu$m XrayCoefficients , negligible compared to the $3\times
3\times 3$ mm3 binning of the photon detection probability map. The main
systematic uncertainty on the mean survival probability comes from the photon
detection probability map. The uncertainties on this probability map given in
GERDA:2022hxs  result in a 0.5% systematic uncertainty on the survival
probability. Finally, we should note that the photon detection probability map
assumes the pre-upgrade configuration of the LAr instrumentation GERDA:2022hxs
. This means the model does not include the inner fiber shroud installed
during the upgrade to improve the light detection efficiency near the
germanium detectors GERDA:2020xhi . Therefore, a customized LAr veto cut was
applied to select the post-upgrade data used in this work: the SiPM channels
corresponding to the inner fiber shroud are not considered to build the LAr
veto condition. This way, the X-rays survival probability obtained with the
pre-upgrade photon detection probability map is extended to the post-upgrade
data sets.

##  6 Analysis methods

The energy region used to set a limit on the half-life of 0$\nu$ECEC of 36Ar
is defined between 410 and 450 keV ($\pm$20 keV around the $\gamma$ energy of
429.88 keV, as indicated by the orange band in figure 1). Given the high
statistics in this energy region, data are used in a binned form, with a 1 keV
binning. It was checked that the binning choice did not impact the analysis
results. In this energy region, the dominant backgrounds are the $\beta$ decay
of 39Ar and the $2\nu\beta\beta$ decay of 76Ge. Subdominant contributions to
the background are, in order of importance, the 42K decays in LAr, the 40K,
214Pb, and 214Bi decays in structural materials. The sum of these
contributions in the analysis window can be approximated by a linear
distribution, as seen in figure 1. The signal is modeled with a Gaussian peak
centered at the $\gamma$ energy and with the width given by the detector
energy resolution ($\sigma$ = FWHM/2.355). Uncertainties on the energy scale
are parametrized by a shift of the signal peak $\delta$ compared to the
nominal energy.

A simultaneous fit is performed on the eight data sets listed in table 1 by
adopting the following binned likelihood:

$\mathcal{L}(T_{1/2},\vec{\theta})=\prod_{d}\prod_{i}\mathcal{P}(n_{di}|\mu_{di}(T_{1/2},\vec{\theta}_{d}))\times\text{Pull}(\vec{\theta}_{d})\,,$
(3)

where the number of events in each bin is Poisson distributed, and the
likelihood is given by the product of the Poisson probabilities $\mathcal{P}$
for all bins $i$ and data sets $d$. The likelihood depends on the half-life
$T_{1/2}$ of the investigated process, which is a common parameter among the
eight data sets and is the only parameter of interest, and on some nuisance
parameters $\vec{\theta}$ that are data set specific and affect both the
signal and background distributions. Gaussian pull terms
$\text{Pull}(\vec{\theta})$ are introduced in the likelihood to constrain some
of the nuisance parameters. Finally, $n_{di}$ denotes the number of observed
events in the data set $d$ and bin $i$, and $\mu_{di}$ is the expectation
value for the same data set and bin. The latter is given by the sum of the
signal and background in that bin: $\mu_{di}=b_{di}+s_{di}$. The number of
signal events $s_{di}$ is given by the integral of the signal distribution for
the data set $d$ in the bin $i$. This is a Gaussian distribution centered at
$E+\delta_{d}(E)$, where $E$ is the $\gamma$ energy of 429.88 keV and
$\delta_{d}(E)$ the energy bias for the data set $d$ calculated for the same
energy, and with the width given by the detector energy resolution
$\sigma_{d}(E)=\text{FWHM}(E)/2.355$ evaluated for the same data set and at
the same energy. The total number of signal events in a data set $d$ is
related to the half-life $T_{1/2}$ through the relation:

$S_{d}=\ln(2)\cdot N_{A}\cdot\frac{m_{LAr,d}}{M_{36}}\cdot f_{36}\cdot
t_{d}\cdot\varepsilon_{tot,d}\cdot\frac{1}{T_{1/2}}\,,$ (4)

where $N_{A}$ is the Avogadro constant, $M_{36}$ is the molar mass of argon
(35.968 g/mol), $m_{LAr,d}$ is the mass of LAr in the simulations from which
the $\gamma$ detection efficiencies are extracted (the product
$\varepsilon_{\gamma}\cdot m_{LAr}$ is given in table 1 for each analysis data
set), $f_{36}$ is the abundance of 36Ar in ultra-pure natural Argon (0.334%)
lee2006 , and $t_{d}$ is the live time of the experiment. The total efficiency
$\varepsilon_{tot,d}$ for the Phase II data sets is given by the product
$\varepsilon_{tot}=\varepsilon_{\gamma}\cdot\varepsilon_{X}\cdot\varepsilon_{LAr}$,
where $\varepsilon_{\gamma}$ is the $\gamma$ detection efficiency,
$\varepsilon_{X}$ the X-rays survival probability (both discussed in section
5), and $\varepsilon_{LAr}$ is the efficiency of the LAr veto cut. The latter
was estimated to be (97.7$\pm$0.1)% for the pre-upgrade data and
(98.2$\pm$0.1)% for the post-upgrade data GERDA:2020xhi . The total efficiency
of the Phase I data sets equals the $\gamma$ detection efficiency
$\varepsilon_{\gamma}$, because no LAr veto cut was available in Gerda Phase
I. Analogously, the number of background events $b_{di}$ is given by the
integral of the background distribution for the data set $d$ in the bin $i$.
The background distribution is a linear function that depends on two
parameters, the normalization and the slope, both data set-specific. We
verified that the first-order polynomial function describes the data in this
energy region well and that a second-order polynomial function does not fit
the data better. In modeling the background of Phase I data, an additional
Gaussian distribution is used to describe the full energy deposition of the
433.9 keV $\gamma$ ray from 108mAg, which lies in the energy region of the
analysis. Contamination from 108mAg was observed in the screening
measurements, and all the three expected $\gamma$ lines from 108mAg were
observed in Gerda Phase I data Agostini2016 ; GERDA:2015naf . The origin of
the 108mAg contamination in Gerda Phase I was found in the signal cables
GERDA:2015naf , which were exchanged in Gerda Phase II GERDA:2017ihb . In
addition, none of these $\gamma$ lines was observed in Gerda Phase II data
after the LAr veto cut. The decay of 108mAg proceeds through a cascade of
three equally probably $\gamma$ rays at energies of 433.9 keV, 614.3 keV, and
722.9 keV. Therefore, even if any 108mAg contamination were still present in
Gerda Phase II, the LAr veto cut would likely discard the corresponding
events. In total, the fit has 42 floating parameters, 22 describing the signal
peak ($\varepsilon_{d}$, $\delta_{d}$, $\sigma_{d}$)444While for the Phase II
data, the energy bias $\delta_{d}$ is assumed to be different among data sets,
for Phase I data only one parameter, common for the three data sets is
adopted, following the previous analysis Agostini2016 ., 10 for the linear
background of Phase II data sets, 6 for the linear background of Phase I data
sets, 3 parameters for the number of 108mAg events in Phase I data sets, plus
one common parameter to all data sets $T_{1/2}^{-1}$. The latter is
constrained to positive values. Gaussian pull terms in the likelihood given in
Eq. 3 constrain some of the nuisance parameters, namely the efficiency
$\varepsilon_{d}$, the energy bias $\delta_{d}$, and the energy resolution
$\sigma_{d}$ around their central value and uncertainty. All the other
nuisance parameters are free and unconstrained, and their uncertainties are
propagated into the result by profiling.

To set a lower limit on the half-life of the investigated process, we use a
modified frequentist approach, namely the $\text{CL}_{\text{s}}$ method
Read2002 . The latter was found to be a more appropriate choice in the case of
an experiment with low sensitivity or, in different words, a background-
dominated experiment Read2002 . Compared to a pure frequentist approach, the
$\text{CL}_{\text{s}}$ exclusion region does not assure the correct coverage
and often results in an over-coverage, thus a more conservative result. The
profile likelihood ratio test statistic is used for the $p\text{-value}$
calculation. Asymptotic distributions of the test statistic and the Asimov
data set are used Cowan2011 . The statistics in each bin is high enough for
this assumption to be valid.

##  7 Results

The best fit, defined as the minimum of the profiled likelihood ratio, yields
$T_{1/2}^{-1}$ = 0, i.e. we do not observe any signal events from 0$\nu$ECEC.
Data from the five Gerda Phase II analysis data sets and in the energy region
of the analysis are shown in figure 4 together with the best-fit model and the
residuals normalized to the expected statistical fluctuations of the bins.

Figure 4: Best fit of the combined Gerda data. The blue line shows the
combined best-fit model, corresponding to $T_{1/2}^{-1}$ = 0. The dashed
orange line indicates the energy at which a $\gamma$ line from 0$\nu$ECEC is
expected, and the orange peak displays the expected signal for a half-life
equal to the 90% C.L. lower limit $1.5\cdot 10^{22}$ yr. The pulls, i.e.
residuals normalized to the expected statistical fluctuations of the bins, are
shown in the bottom panels for each data set.

The 90% C.L. limit on the half-life is obtained by scanning the observed
$\text{CL}_{\text{s}}$ over different values of $T_{1/2}^{-1}$ and finding the
value for which $\text{CL}_{\text{s}}$ = 0.1. For Gerda Phase II data only,
this gives $T_{1/2}>1.3\cdot 10^{22}$ yr. The 90% C.L. sensitivity of the
Gerda Phase II experiment, i.e. the median expectation under the no signal
hypothesis, is obtained analogously by scanning the expected
$\text{CL}_{\text{s}}$ over different values of $T_{1/2}^{-1}$ and finding the
value for which $\text{CL}_{\text{s}}$ = 0.1. The latter gives
$T_{1/2}>8.0\cdot 10^{21}$ yr. The analysis of the combined Gerda Phase I and
Phase II data gives a 90% C.L. sensitivity of $T_{1/2}>8.6\cdot 10^{21}$ yr
and an observed lower limit of $T_{1/2}>1.5\cdot 10^{22}$ yr. Figure 5 shows
the scan of the observed and expected $\text{CL}_{\text{s}}$ over a range of
values of $T_{1/2}^{-1}$ obtained in the analysis of the combined Gerda Phase
I and Phase II data.

Figure 5: $\text{CL}_{\text{s}}$ as a function of the inverse of the half-life
obtained in the analysis of the combined Gerda Phase I and Phase II data. The
median of the $\text{CL}_{\text{s}}$ distribution for the Gerda experiment
under the no signal hypothesis and the observed $\text{CL}_{\text{s}}$ for the
Gerda data are shown by the continuous black line and the dashed line,
respectively. The spread of the $\text{CL}_{\text{s}}$ expected distribution,
given by the 68% and 95% probability intervals, is also shown by the colored
bands. The 90% C.L. limit (sensitivity) is given by the solid (dashed) black
line intersection with the dotted line, corresponding to a
$\text{CL}_{\text{s}}$ of 0.1.

Systematic uncertainties on the efficiency $\varepsilon_{d}$, the energy bias
$\delta_{d}$, and the energy resolution $\sigma_{d}$ are identified as primary
sources of systematic uncertainties and included in the likelihood through
nuisance parameters constrained by Gaussian pull terms as explained in section
6. Their overall effect on the limit derived in section 7 is estimated to be
2%. Potential systematic uncertainties related to the fit model, particularly
the background distribution, are also investigated. First, the assumption of a
linear distribution is compared to a more general second-order polynomial
distribution. This has a negligible impact on the result. The presence of
additional structures in the background is also investigated. As discussed in
section 6, a $\gamma$ line from 108mAg, very close to the expected signal
energy, is included in the background model of the Phase I data sets, as in
previous analysis Agostini2016 . A possible systematic uncertainty due to the
above $\gamma$ line in Phase II data is investigated by introducing it in the
background model. This would worsen our result of a 2%.

##  8 Conclusions

In this work, we searched for the 429.88 keV $\gamma$ line from the 36Ar
0$\nu$ECEC using the final total exposure of the Gerda Phase II experiment,
combined with the Gerda Phase I exposure. No signal was observed, and a lower
limit on the half-life of this process was derived, yielding $T_{1/2}>1.5\cdot
10^{22}$ yr (90% C.L.). This is the most stringent limit on the half-life of
the 36Ar 0$\nu$ECEC. This work shows that the potential of the Gerda
experiment in investigating physics beyond the Standard Model extends further
than the search for the $0\nu\beta\beta$ decay of 76Ge (see also GERDA:2020emj
; GERDA:2022ffe ). Even if the sensitivity is many orders of magnitude below
the theoretical expectation for this process, to our knowledge, the Gerda
experiment was, to date, the only experiment with the capability to search for
the 0$\nu$ECEC of 36Ar with competitive sensitivities. The Gerda sensitivity
is limited by the physical background from 39Ar $\beta$ and 76Ge
$2\nu\beta\beta$ decays in the energy region where the $\gamma$ peak is
expected, which is, for instance, orders of magnitude higher than the
background in the region of interest for the 76Ge $0\nu\beta\beta$ decay. An
additional limiting factor is the low detection efficiency since the $\gamma$
ray is emitted in the LAr and must be detected in one of the germanium
detectors. Only $\gamma$ rays emitted in the proximity of the detector array
contribute to the total efficiency as discussed in section 5 (See figure 3).

Among the planned future experiments, the Large Enriched Germanium Experiment
for Neutrinoless-$\beta\beta$ Decay (LEGEND) experiment can extend the search
for the 0$\nu$ECEC of 36Ar to higher sensitivity. In the first phase of the
project, LEGEND-200 will deploy about 200 kg of germanium detectors. This is
more than a factor of four compared to the Gerda detector mass and will imply
a higher detection efficiency to the $\gamma$ ray emitted in this process. On
the other hand, the background in the energy region where the $\gamma$ peak is
expected should be comparable to the Gerda background, largely dominated by
the 39Ar $\beta$ decay. Still, an improvement in the current sensitivity is
foreseen. LEGEND-1000 will deploy about 1 ton of germanium detectors, implying
an even higher detection efficiency to the $\gamma$ ray emitted in this
process. In addition, using underground Ar instead of atmospheric Ar is
intended. This is depleted of 39Ar, which is the main background contribution
in this search. A significant improvement in the sensitivity is therefore
expected. To our knowledge, no other planned experiment has competitive
sensitivity to LEGEND in the search for 0$\nu$ECEC of 36Ar.

###### Acknowledgements.

The Gerda experiment is supported financially by the German Federal Ministry
for Education and Research (BMBF), the German Research Foundation (DFG), the
Italian Istituto Nazionale di Fisica Nucleare (INFN), the Max Planck Society
(MPG), the Polish National Science Centre (NCN, grant number
UMO-2020/37/B/ST2/03905), the Polish Ministry of Science and Higher Education
(MNiSW, grant number DIR/WK/2018/08), the Russian Foundation for Basic
Research, and the Swiss National Science Foundation (SNF). This project has
received funding/support from the European Union’s Horizon 2020 research and
innovation programme under the Marie Sklodowska-Curie grant agreements No
690575 and No 674896. This work was supported by the Science and Technology
Facilities Council, part of the U.K. Research and Innovation (Grant No.
ST/T004169/1). The institutions acknowledge also internal financial support.
The Gerda collaboration thanks the directors and the staff of the LNGS for
their continuous strong support of the Gerda experiment.

This manuscript has associated data in a data repository. [Authors’ comment:
The data shown in Figs. 1 and 4 is available in ASCII format as Supplemental
Material supp-mat .]

## References

  * (1) E. Aprile et al. (XENON Collaboration), Nature 568(7753), 532 (2019).  DOI 10.1038/s41586-019-1124-4
  * (2) A.P. Meshik, C.M. Hohenberg, O.V. Pravdivtseva, Y.S. Kapusta, Phys. Rev. C 64, 035205 (2001).  DOI 10.1103/PhysRevC.64.035205
  * (3) S.S. Ratkevich, A.M. Gangapshev, Y.M. Gavrilyuk, F.F. Karpeshin, V.V. Kazalov, V.V. Kuzminov, S.I. Panasenko, M.B. Trzhaskovskaya, S.P. Yakimenko, Phys. Rev. C 96(6), 065502 (2017).  DOI 10.1103/PhysRevC.96.065502
  * (4) R.G. Winter, Phys. Rev. 100, 142 (1955).  DOI 10.1103/PhysRev.100.142
  * (5) M. Doi, T. Kotani, Progress of Theoretical Physics 89(1), 139 (1993).  DOI 10.1143/ptp/89.1.139
  * (6) K. Blaum, S. Eliseev, F.A. Danevich, V.I. Tretyak, S. Kovalenko, M.I. Krivoruchenko, Y.N. Novikov, J.T. Suhonen, Rev. Mod. Phys. 92, 045007 (2020).  DOI 10.1103/RevModPhys.92.045007
  * (7) H.M. Georgi, S.L. Glashow, S. Nussinov, Nucl. Phys. B 193, 297 (1981).  DOI 10.1016/0550-3213(81)90336-9
  * (8) R.A. Eramzhian, G. Mitselmakher, M.B. Voloshin, JETP Lett. 35, 656 (1982)
  * (9) J. Bernabeu, A. De Rujula, C. Jarlskog, Nucl. Phys. B 223, 15 (1983).  DOI 10.1016/0550-3213(83)90089-5
  * (10) K. H. Ackermann et al. (Gerda Collaboration), Eur. Phys. J. C 73(3), 2330 (2013).  DOI 10.1140/epjc/s10052-013-2330-0
  * (11) M. Agostini et al. (Gerda Collaboration), Phys. Rev. Lett. 125(25), 252502 (2020).  DOI 10.1103/PhysRevLett.125.252502
  * (12) V. Tretyak, Y. Zdesenko, Atom. Data Nucl. Data Tabl. 61, 43 (1995).  DOI 10.1016/S0092-640X(95)90011-X
  * (13) A. Merle, The Mysteries of Leptons: New Physics and unexplained Phenomena.  Dissertation (2009).  DOI 10.11588/heidok.00010180
  * (14) M. Wang, W.J. Huang, F.G. Kondev, G. Audi, S. Naimi, Chin. Phys. C 45(3), 030003 (2021).  DOI 10.1088/1674-1137/abddaf
  * (15) O. Chkvorets, Search for double beta decay with HPGe detectors at the Gran Sasso underground laboratory.  Dissertation (2008).  DOI 10.11588/heidok.00008572
  * (16) M. Agostini et al. (Gerda Collaboration), Eur. Phys. J. C 76(12), 652 (2016).  DOI 10.1140/epjc/s10052-016-4454-5
  * (17) M.G. Dunford, A Search for the Neutrinoles Double Electron Capture of 36Ar and a Measurement of the Specific Activity of 39Ar in Atmospheric Argon with the DEAP-3600 Detector.  Ph.D. thesis (2018).  DOI 10.22215/etd/2018-13483
  * (18) M. Agostini et al. (Gerda Collaboration), Eur. Phys. J. C 78(5), 388 (2018).  DOI 10.1140/epjc/s10052-018-5812-2
  * (19) M. Agostini et al. (Gerda Collaboration), Science 365, 1445 (2019).  DOI 10.1126/science.aav8613
  * (20) K.T. Knöpfle, B. Schwingenheuer, JINST 17(02), P02038 (2022).  DOI 10.1088/1748-0221/17/02/P02038
  * (21) A. Lubashevskiy, et al., Eur. Phys. J. C 78(1), 15 (2018).  DOI 10.1140/epjc/s10052-017-5499-9
  * (22) J. Janicskó-Csáthy, H.A. Khozani, X. Liu, B. Majorovits, A. Caldwell, Nucl. Instrum. Meth. A 654, 225 (2011).  DOI 10.1016/j.nima.2011.05.070
  * (23) K. Freund, et al., Eur. Phys. J. C 76(5), 298 (2016).  DOI 10.1140/epjc/s10052-016-4140-7
  * (24) M. Agostini, L. Pandola, P. Zavarise, J. Phys. Conf. Ser. 368, 012047 (2012).  DOI 10.1088/1742-6596/368/1/012047
  * (25) M. Agostini et al. (Gerda Collaboration), Eur. Phys. J. C 75(6), 255 (2015).  DOI 10.1140/epjc/s10052-015-3409-6
  * (26) M. Agostini et al. (Gerda Collaboration), Eur. Phys. J. C 82(4), 284 (2022).  DOI 10.1140/epjc/s10052-022-10163-w
  * (27) M. Agostini et al. (Gerda Collaboration), Eur. Phys. J. C 81(8), 682 (2021).  DOI 10.1140/epjc/s10052-021-09403-2
  * (28) C. Ransom, Energy Calibration for the GERDA and LEGEND-200 Experiments.  Ph.D. thesis (2021)
  * (29) S. Agostinelli et al. (GEANT4 Collaboration), Nucl. Instrum. Meth. A 506, 250 (2003).  DOI 10.1016/S0168-9002(03)01368-8
  * (30) M. Boswell, et al., IEEE Trans. Nucl. Sci. 58, 1212 (2011).  DOI 10.1109/TNS.2011.2144619
  * (31) M. Agostini et al. (Gerda Collaboration), JHEP 03, 139 (2020).  DOI 10.1007/JHEP03(2020)139
  * (32) M. Agostini et al. (Gerda Collaboration), Eur. Phys. J. C 79(11), 978 (2019).  DOI 10.1140/epjc/s10052-019-7353-8
  * (33) M. Agostini et al. (Gerda Collaboration), Eur. Phys. J. C 83(4), 319 (2023).  DOI 10.1140/epjc/s10052-023-11354-9
  * (34) J. Hubbell, S. Seltzer, X-Ray Mass Attenuation Coefficients — NIST.  URL https://www.nist.gov/pml/x-ray-mass-attenuation-coefficients
  * (35) J.Y. Lee, K. Marti, J.P. Severinghaus, K. Kawamura, H.S. Yoo, J.B. Lee, J.S. Kim, Geochimica et Cosmochimica Acta 70(17), 4507 (2006).  DOI 10.1016/j.gca.2006.06.1563
  * (36) M. Agostini et al. (Gerda Collaboration), J. Phys. G 42(11), 115201 (2015).  DOI 10.1088/0954-3899/42/11/115201
  * (37) A.L. Read, J. Phys. G 28, 2693 (2002).  DOI 10.1088/0954-3899/28/10/313
  * (38) G. Cowan, K. Cranmer, E. Gross, O. Vitells, Eur. Phys. J. C 71, 1554 (2011).  DOI 10.1140/epjc/s10052-011-1554-0.  [Erratum: Eur.Phys.J.C 73, 2501 (2013)]
  * (39) M. Agostini et al. (Gerda Collaboration), Phys. Rev. Lett. 125(1), 011801 (2020).  DOI 10.1103/PhysRevLett.125.011801.  [Erratum: Phys.Rev.Lett. 129, 089901 (2022)]
  * (40) M. Agostini et al. (Gerda Collaboration), JCAP 12, 012 (2022).  DOI 10.1088/1475-7516/2022/12/012
  * (41) See Supplemental Material at [URL] for the data shown in Figs. 1 and 4.