# GLiNER: Generalist Model for Named Entity Recognition using Bidirectional
Transformer

Urchade Zaratiana1,2, Nadi Tomeh2, Pierre Holat1,2, Thierry Charnois2
1 FI Group, 2 LIPN, CNRS UMR 7030, France
<EMAIL_ADDRESS>
https://github.com/urchade/GLiNER

###### Abstract

Named Entity Recognition (NER) is essential in various Natural Language
Processing (NLP) applications. Traditional NER models are effective but
limited to a set of predefined entity types. In contrast, Large Language
Models (LLMs) can extract arbitrary entities through natural language
instructions, offering greater flexibility. However, their size and cost,
particularly for those accessed via APIs like ChatGPT, make them impractical
in resource-limited scenarios. In this paper, we introduce a compact NER model
trained to identify any type of entity. Leveraging a bidirectional transformer
encoder, our model, GLiNER, facilitates parallel entity extraction, an
advantage over the slow sequential token generation of LLMs. Through
comprehensive testing, GLiNER demonstrate strong performance, outperforming
both ChatGPT and fine-tuned LLMs in zero-shot evaluations on various NER
benchmarks.

##  1 Introduction

Named Entity Recognition plays a crucial role in various real-world
applications, such as constructing knowledge graphs. Traditional NER models
are limited to a predefined set of entity types. Expanding the number of
entity types can be beneficial for many applications but may involve labeling
additional datasets. The emergence of Large Language Models, like GPT-3 (Brown
et al., 2020), has introduced a new era for open-type NER by enabling the
identification of any types of entity types only by natural language
instruction. This shift signifies a significant departure from the
inflexibility observed in traditional models. However, powerful LLMs typically
consist of billions of parameters and thus require substantial computing
resources. Although it is possible to access some LLMs via APIs (OpenAI,
2023), using them at scale can incur high costs.

Figure 1: BiLM for Open NER. Previous models, such as UniNER (Zhou et al.,
2023) (Fig. a), approach the task of open type NER by prompting LLMs with
natural language instructions (using a multi-turn dialogue style). Our
proposed GLiNER utilizes small bidirectional LMs and treats the NER task as
matching entity types with textual spans in a latent space. Figure 2: Model
architecture. GLiNER employs a BiLM and takes as input entity type prompts and
a sentence/text. Each entity is separated by a learned token [ENT]. The BiLM
outputs representations for each token. Entity embeddings are passed into a
FeedForward Network, while input word representations are passed into a span
representation layer to compute embeddings for each span. Finally, we compute
a matching score between entity representations and span representations
(using dot product and sigmoid activation). For instance, in the figure, the
span representation of (0, 1), corresponding to "Alain Farley," has a high
matching score with the entity embeddings of "Person".

Recently, researchers have explored the fine-tuning of open source language
models such as LLaMa (Touvron et al., 2023) for named entity recognition
tasks. Wang et al. (2023), for example, introduced InstructUIE, a fine-tuned
FlanT5-11B (Raffel et al., 2019; Chung et al., 2022) model on existing
information extraction datasets, achieving excellent performance in zero-shot
settings. Additionally, GoLLIE (Sainz et al., 2023) was introduced as an
extension of InstructUIE work by by finetuning a CodeLLaMa (Rozière et al.,
2023) using detailed annotation guidelines, resulting in significant
performance improvements. Another recent proposal by Zhou et al. (2023),
called UniversalNER, involves the fine-tuning of LLMs using diverse data from
various domains annotated with ChatGPT instead of relying on standard NER
datasets. Their approach enables the replication and even surpassing of the
original capability of ChatGPT when evaluated in zero-shot on several NER
datasets. While these works have achieved remarkable results, they present
certain limitations we seek to address: They use autoregressive language
models, which can be slow due to token-by-token generation; Moreover, they
employ large models with several billion parameters, limiting their deployment
in compute-limited scenarios. Furthurmore, as NER is treated as a text
generation problem, the generation of entities is done in several decoding
steps, and there is no way to perform the prediction of multiple entity types
in parallel.

In our work, we propose a model that addresses the above-mentioned problems.
Instead of relying on large autoregressive models, we utilize smaller-scale
Bidirectional Language Models (BiLM), such as BERT (Devlin et al., 2019) or
deBERTa (He et al., 2021). The core concept of our model involves treating the
task of Open NER as matching entity type embeddings to textual span
representations in latent space, rather than as a generation task. This
approach naturally solves the scalability issues of autoregressive models and
allows for bidirectional context processing, which enables richer
representations. When trained on the dataset released by (Zhou et al., 2023),
which comprises texts from numerous domains and thousands of entity types, our
model demonstrates impressive zero-shot performance. More specifilcally, it
outperforms both ChatGPT and fine-tuned LLMs on zero-shot NER datasets (Table
1). Our model’s robustness is further evident in its ability to handle
languages not even encountered during training. Specifically, our model
surpasses ChatGPT in 8 of 10 languages (Table 3) that were not included in its
training data.

##  2 Method

This section presents our model, GLiNER, which is train to extract any types
of entity using a Bidirectional Language Models. Our model has three main
components: i) a pre-trained textual encoder (a BiLM such as BERT), ii) a span
representation module which computes span embeddings from token embeddings,
iii) an entity representation module which computes entity embeddings that the
model seeks to extract. The goal is to have entity and span embeddings in the
same latent space to assess their compatibility (degree of matching). The
overall architecture of our model is depicted in Figure 2.

###  2.1 Architecture

#### Input format

The input to our model comprises a unified sequence combining entity types
(expressed in natural language) and the input text from which entities are to
be extracted. The input format is as follows:

[ENT] token represents a special token placed before each entity type and the
[SEP] token functions as a delimiter, separating the sequence of entity types
from the input text. They are initialized randomly at the start of training.

#### Token representation

The token encoder processes the unified input to compute interactions between
all tokens (both entity types and input text), producing contextualized
representations. Let
${\bm{p}}=\\{{\bm{p}}_{i}\\}_{0}^{M-1}\in\mathbb{R}^{M\times D}$ represent the
encoder’s output for each entity type, corresponding to all the [ENT] token
representations. Similarly,
${\bm{h}}=\\{{\bm{h}}_{i}\\}_{0}^{N-1}\in\mathbb{R}^{N\times D}$ denotes the
representation of each word in the input text. For words tokenized into
multiple subwords, we use the representation of the first subword, which is a
standard choice in the NER literature (Zaratiana et al., 2022).

#### Entity and Span Representation

In our model, we aim to encode entity types and span embeddings into a unified
latent space. The entity representation is computed by refining the initial
representation ${\bm{p}}$ using a two-layer feedforward network, resulting in
${\bm{q}}=\\{{\bm{q}}_{i}\\}_{0}^{M-1}\in\mathbb{R}^{M\times D}$. The
representation of a span starting at position $i$ and ending at position $j$
in the input text, ${\bm{S}}_{ij}\in\mathbb{R}^{D}$, is computed as:

${\bm{S}}_{ij}=\texttt{FFN}({\bm{h}}_{i}\otimes{\bm{h}}_{j})$ (1)

Here, FFN denotes a two-layer feedforward network, and $\otimes$ represents
the concatenation operation. In practice, The computation of all span
representations can be easily parallelized. Moreover, we set an upper bound to
the length (K=12) of the span in order to keep linear complexity, without
harming recall.

#### Entity Type and Span Matching

To evaluate whether a span $(i,j)$ corresponds to entity type $t$, we
calculate the following matching score:

$\phi(i,j,t)=\sigma({\bm{S}}_{ij}^{T}{\bm{q}}_{t})\in\mathbb{R}$ (2)

In this equation, $\sigma$ denotes a sigmoid activation function. As we train
with binary cross-entropy loss (see next sec. 2.2), $\phi(i,j,t)$ can be
interpreted as the probability of the span $(i,j)$ being of type $t$.

Model | Params | Movie | Restaurant | AI | Literature | Music | Politics | Science | Average
---|---|---|---|---|---|---|---|---|---
Vicuna-7B | 7B | 6.0 | 5.3 | 12.8 | 16.1 | 17.0 | 20.5 | 13.0 | 13.0
Vicuna-13B | 13B | 0.9 | 0.4 | 22.7 | 22.7 | 26.6 | 27.0 | 22.0 | 17.5
USM | 0.3B | 37.7 | 17.7 | 28.2 | 56.0 | 44.9 | 36.1 | 44.0 | 37.8
ChatGPT | – | 5.3 | 32.8 | 52.4 | 39.8 | 66.6 | 68.5 | 67.0 | 47.5
InstructUIE | 11B | 63.0 | 21.0 | 49.0 | 47.2 | 53.2 | 48.1 | 49.2 | 47.2
UniNER-7B | 7B | 42.4 | 31.7 | 53.6 | 59.3 | 67.0 | 60.9 | 61.1 | 53.7
UniNER-13B | 13B | 48.7 | 36.2 | 54.2 | 60.9 | 64.5 | 61.4 | 63.5 | 55.6
GoLLIE | 7B | 63.0 | 43.4 | 59.1 | 62.7 | 67.8 | 57.2 | 55.5 | 58.0
GLiNER-S | 50M | 46.9 | 33.3 | 50.7 | 60.0 | 60.9 | 61.5 | 55.6 | 52.7
GLiNER-M | 90M | 42.9 | 37.3 | 51.8 | 59.7 | 69.4 | 68.6 | 58.1 | 55.4
GLiNER-L | 0.3B | 57.2 | 42.9 | 57.2 | 64.4 | 69.6 | 72.6 | 62.6 | 60.9
Table 1: Zero-Shot Scores on Out-of-Domain NER Benchmark. We report the
performance of GLiNER with various DeBERTa-v3 (He et al., 2021) model sizes.
Results for Vicuna, ChatGPT, and UniNER are from Zhou et al. (2023); USM and
InstructUIE are from Wang et al. (2023); and GoLLIE is from Sainz et al.
(2023).

###  2.2 Training

During training, our objective is to optimize model parameters to enhance the
matching score for correct span-type pairs (positive pairs) and reduce it for
incorrect pairs (negative pairs). A span $(i,j)$ paired with an entity type
$t$ forms a positive pair ($s\in\mathcal{P}$) if the span is labeled with type
$t$ in the training data. Otherwise, it is a negative pair
($s\in\mathcal{N}$). The training loss for an individual example, comprising
spans $\mathcal{S}$ and entity types $\mathcal{T}$, is defined as:

$\displaystyle\mathcal{L}_{\text{BCE}}=-\sum_{s\in\mathcal{S}\times\mathcal{T}}$
$\displaystyle\mathbb{I}_{s\in\mathcal{P}}\log\phi(s)+$ (3)
$\displaystyle\mathbb{I}_{s\in\mathcal{N}}\log\left(1-\phi(s)\right)$

The variable $s$ represents a pair of span/entity type and $\mathbb{I}$ is an
indicator function, which returns 1 when the specified condition is true and 0
otherwise. This loss function corresponds to binary cross-entropy.

###  2.3 Decoding algorithm

In the decoding phase, we employ a greedy span section (Zaratiana et al.,
2022) that selects entity spans based on matching scores, to ensure
task/dataset specific constraints. This strategy is applied independently to
each sentence. Only, spans $(i,j)$ with matching scores $\phi(i,j,c)>0.5$ are
considered for selection.

#### Flat NER:

The algorithm chooses the highest-scoring non-overlapping span and continues
this process until all spans are evaluated.

#### Nested NER:

Similar to Flat NER, but the algorithm allows selection of fully nested spans
within other entities while still avoiding partial overlaps.

#### Algorithm Efficiency:

The decoding is implemented using a priority queue for spans, ensuring an
$O(n\log n)$ complexity, with $n$ being the number of candidate spans.

##  3 Experimental Setting

###  3.1 Training data

Our objective is to construct a versatile NER model capable of accurately
identifying a wide array of entity types across different textual domains. To
achieve this, it is essential that our training dataset encompasses a diverse
range of entity types. For this, we utilize the training data released by Zhou
et al. (2023), known as Pile-NER111https://huggingface.co/datasets/Universal-
NER/Pile-NER-type. This dataset is derived from the Pile corpus (Gao et al.,
2020), commonly used for pretraining large language models, and comprises text
from diverse sources. More specifically, Zhou et al. (2023) sampled 50,000
texts from the Pile data and employed ChatGPT to extract their associated
entity types. Notably, they did not specify the entity types to the LLMs,
aiming to extract a diverse range of entity types. They used the following
prompt:

Figure 3: Prompting ChatGPT for entity extraction. This prompt was used Zhou
et al. (2023) to construct the Pile-NER dataset.

Finally, after filtering bad outputs their datasets results in 44889 passages
containing in total 240k entity spans and 13k distinct entity types.

###  3.2 Hyperparameters

Our model, GLiNER, is trained on the Pile-NER dataset, which we described in
the previous section. We use the deBERTa-v3 (He et al., 2021) as our backbone
due to its proven empirical performance. All non-pretrained layers have a
width dimension of 768 and a dropout rate of 0.4. Regarding the training
process, we employ the AdamW optimizer (Loshchilov and Hutter, 2017), setting
a base learning rate of 1e-5 for pretrained layers (the transformer backbone)
and 5e-5 for non-pretrained layers (FFN layers, span representation). We
trained our models for a maximum of 30k steps, starting with a 10% warmup
phase, followed by a decay phase using a cosine scheduler. The Pile-NER
dataset natively contains only positive entities (i.e., entities that are
present in the sentence), and we found it useful to include negative entity
types during training. This is achieved by sampling random entities from other
examples in the same batch. In addition, we follow the strategies outlined in
Sainz et al. (2023) as a form of regularization, which includes shuffling
entity order and randomly dropping entities. Furthermore, we limit the number
of entity types to 25 per sentence during training. The larger variant of our
model, GLiNER-L, takes 5 hours to train on an A100 GPU.

###  3.3 Baselines

In our evaluation, we compare our model, GLiNER, with several recent models
designed for open-type NER. First, we examine chat models like ChatGPT and
Vicuna (Chiang et al., 2023), which utilize the prompting from Ye et al.
(2023); we report their results reported by Zhou et al. (2023). Next, we focus
on three recent Large Language Models (LLMs) that have been fine-tuned for
NER: InstructUIE (Wang et al., 2023), based on the FlanT5 11B model and fine-
tuned on various NER datasets; UniNER (Zhou et al., 2023), which employs a
LLaMa model fine-tuned on a dataset generated by ChatGPT; GoLLIE (Sainz et
al., 2023), fine-tuned to adhere to detailed annotation guidelines for
enhanced performance in unseen IE tasks, utilizing CodeLLama as its base
model. Finally, we include USM (Lou et al., 2023) in our comparison, which is
similar in size to ours but features a different architecture.

###  3.4 Evaluation

#### Datasets

We primarily evaluate our model in a zero-shot context on common NER
benchmarks, following previous works Wang et al. (2023); Zhou et al. (2023).
The first is the OOD NER Benchmark (Table 1), which comprises seven diverse
NER datasets from CrossNER and MIT. This benchmark is typically used for
evaluating out-of-domain generalization capabilities of NER models. The second
benchmark consists of 20 NER datasets (Table 2) from a wide range of domains,
including biomedical, news articles, and tweets. These datasets are commonly
used for training supervised NER models. Additionally, we evaluate our model
on multilingual NER datasets (Table 3) for further investigation. For this
purpose, we use the recently released Multiconer (Multilingual Complex NER)
(Malmasi et al., 2022), which contains data in 11 languages across various
domains.

#### Metric

We adopt the standard NER evaluation methodology, calculating F1-score based
on the exact match between predicted and actual entities.

Dataset | ChatGPT | UniNER-7B | GLiNER-L
---|---|---|---
ACE05 | 26.6 | 36.9 | 27.3
AnatEM | 30.7 | 25.1 | 33.3
bc2gm | 40.2 | 46.2 | 47.9
bc4chemd | 35.5 | 47.9 | 43.1
bc5cdr | 52.4 | 68.0 | 66.4
Broad Tweeter | 61.8 | 67.9 | 61.2
CoNLL03 | 52.5 | 72.2 | 64.6
FabNER | 15.3 | 24.8 | 23.6
FindVehicle | 10.5 | 22.2 | 41.9
GENIA | 41.6 | 54.1 | 55.5
HarveyNER | 11.6 | 18.2 | 22.7
MIT Movie | 5.3 | 42.4 | 57.2
MIT Restaurant | 32.8 | 31.7 | 42.9
MultiNERD | 58.1 | 59.3 | 59.7
ncbi | 42.1 | 60.4 | 61.9
OntoNotes | 29.7 | 27.8 | 32.2
PolyglotNER | 33.6 | 41.8 | 42.9
TweetNER7 | 40.1 | 42.7 | 41.4
WikiANN | 52.0 | 55.4 | 58.9
WikiNeural | 57.7 | 69.2 | 71.8
Average | 36.5 | 45.7 | 47.8
Table 2: Zero-shot performance on 20 NER datasets. Results of ChatGPT and
UniNER are reported from (Zhou et al., 2023).

##  4 Results

###  4.1 Zero-shot on English datasets

In this section, we discuss the performance of our model in a zero-shot
context, i.e., by only training on the Pile-NER dataset without further fine-
tuning on target datasets.

#### OOD NER Benchmark

We first evaluate our model on the OOD benchmark as reported in Table 1. We
compare three different sizes of our model (small, medium, and large) against
the baselines. The results demonstrate our model’s impressive capability,
irrespective of its size. For example, even our smallest model, with only 50M
parameters, outperforms general-purpose models such as ChatGPT and Vicuna. It
also shows better performance than the 11B InstructUIE, which has been
instruction-tuned for the NER task. Furthermore, when compared to UniNER,
which used the same training data as GLiNER, our medium-sized model (90M)
achieves comparable results to UniNER-13B (55 F1 for both), despite being 140
times smaller, while our largest version outperforms it by an average of 5
points. Our most best competitor, GoLLIE, which leads among the LLMs, achieves
better performance than most of our models but is still less effective than
GLiNER-L. Against USM, which has a comparable number of parameters to ours,
the performance is significantly lower, highlighting the superiority of our
architecture.

#### 20 NER Benchmark

Table 2 presents a comparison of our model against ChatGPT and UniNER across
20 diverse NER datasets. First, similar to the OOD benchmark, ChatGPT
significantly lags behind fine-tuned models for NER, trailing behind UniNER.
Furthermore, GLiNER achieves the highest performance on 13 of these datasets,
surpassing UniNER by an average of 2 points. This superior performance
underscores GLiNER’s robustness and adaptability across a broad spectrum of
domains. However, a notable observation is that GLiNER underperforms compared
to UniNER on tweet-based NER datasets. This highlights potential areas for
improvement in GLiNER’s ability to process informal, colloquial, or noisy
data, typical of social media content.

| Language | Sup. | ChatGPT | GLiNER
---|---|---|---|---
| En | Multi
Latin | German | 64.6 | 37.1 | 35.6 | 39.5
English | 62.7 | 37.2 | 42.4 | 41.7
Spanish | 58.7 | 34.7 | 38.7 | 42.1
Dutch | 62.6 | 35.7 | 35.6 | 38.9
Non-Latin | Bengali | 39.7 | 23.3 | 0.89 | 25.9
Persian | 52.3 | 25.9 | 14.9 | 30.2
Hindi | 47.8 | 27.3 | 11.3 | 27.8
Korean | 55.8 | 30.0 | 20.5 | 28.7
Russian | 59.7 | 27.4 | 30.3 | 33.3
Turkish | 46.8 | 31.9 | 22.0 | 30.0
Chinese | 53.1 | 18.8 | 6.59 | 24.3
Average | 54.9 | 29.9 | 23.6 | 32.9
Table 3: Zero-Shot Scores on Different Languages. The baseline, Sup., is an
XLM-R (Conneau et al., 2019) model fine-tuned on the training set of each
language separately, as reported by Malmasi et al. (2022). ChatGPT evaluation
is taken from Lai et al. (2023). GLiNER-En employs deBERTa-v3-Large, and Multi
uses mdeBERTa-v3-base.

###  4.2 Zero-Shot Multilingual Evaluation

In this section, we evaluate the performance of our model in a zero-shot
context on unseen languages to assess its generalizability. This evaluation
uses the Multiconel dataset, with results detailed in Table 3. Our model,
GLiNER, is presented in two variants: En, which employs deBERTa-v3-Large as
its backbone, and Multi, which utilizes a multilingual version of deBERTa-v3
(mdeBERTa-v3). Both versions were fine-tuned on the Pile-NER dataset. For
comparative purposes, we include results from ChatGPT and a supervised
baseline, the latter being fine-tuned on the training set of each dataset
using separate models.

#### Results

As expected, the supervised baseline demonstrated superior performance,
significantly outperforming the zero-shot models. Among these, GLiNER-Multi
showed the most promising results, surpassing ChatGPT in most languages, which
is noteworthy considering that the fine-tuning dataset, Pile-NER, consists
exclusively of English examples. Interestingly, its performance on Spanish
data slightly exceeded that in English. While GLiNER-En generally
underperformed compared to ChatGPT on average, it achieved competitive, and at
times superior, results in languages using the Latin script, such as Spanish
and German. However, its performance was markedly less competitive in non-
Latin languages, particularly in Bengali, where it scored only 0.89 in the F1
score.

Dataset | InstructUIE | UniNER-7B | GLiNER-L
---|---|---|---
w/o | w/ | w/ | w/o
ACE05 | 79.9 | 86.7 | 82.8 | 81.3
AnatEM | 88.5 | 88.5 | 88.9 | 88.4
bc2gm | 80.7 | 82.4 | 83.7 | 82.0
bc4chemd | 87.6 | 89.2 | 87.9 | 86.7
bc5cdr | 89.0 | 89.3 | 88.7 | 88.7
Broad Twitter | 80.3 | 81.2 | 82.5 | 82.7
CoNLL03 | 91.5 | 93.3 | 92.6 | 92.5
FabNER | 78.4 | 81.9 | 77.8 | 74.8
FindVehicle | 87.6 | 98.3 | 95.7 | 95.2
GENIA | 75.7 | 77.5 | 78.9 | 77.4
HarveyNER | 74.7 | 74.2 | 68.6 | 67.4
MIT Movie | 89.6 | 90.2 | 87.9 | 87.5
MIT Restaurant | 82.6 | 82.3 | 83.6 | 83.3
MultiNERD | 90.3 | 93.7 | 93.8 | 93.3
ncbi | 86.2 | 87.0 | 87.8 | 87.1
OntoNotes | 88.6 | 89.9 | 89.0 | 88.1
PolyglotNER | 53.3 | 65.7 | 61.5 | 60.6
TweetNER7 | 65.9 | 65.8 | 51.4 | 50.3
WikiANN | 64.5 | 84.9 | 83.7 | 82.8
wikiNeural | 88.3 | 93.3 | 91.3 | 91.4
Average | 81.2 | 84.8 | 82.9 | 82.1
Table 4: In-domain Supervised Finetuning. All the models are fine-tuned on the
mix of all training data of the benchmark. w/ indicates that the model was
trained on the Pile-NER dataset before finetuning.

###  4.3 In-domain Supervised tuning

In our work, we also perform in-domain supervised fine-tuning (on 20 NER
datasets) of our model to compare its capabilities against LLMs under this
setup. Specifically, we compare our model against InstructUIE and UniNER, both
of which have also been fine-tuned. The main difference is that UniNER has
been pre-trained on the Pile-NER dataset before fine-tuning.

#### Training Setup

For the supervised setting, we primarily adhere to the same experimental setup
as described in the main experiment (using deBERTa-v3 large), except for the
training dataset. Regarding the training data, we follow the approach of
InstructUIE: we randomly sample 10,000 data points for each dataset in the 20
NER benchmark. If a dataset does not contain 10,000 samples, we include all
available data. We implement two variants of our model: the first one
initializes the weights from our zero-shot model, which is a pretrained on the
Pile-NER dataset. The second variant is trained without the Pile-NER dataset,
same as InstructUIE.

#### Results

The results of our experiment are reported in Table 3. Firstly, we observe
that for the in-domain fine-tuning, our GLiNER model, pretrained on Pile-NER,
achieves slightly better results than the non-pretrained variant, with an
average difference of 0.8. Moreover, our pretrained GLiNER model outperforms
InstructUIE (with an average difference of 0.9) despite being fine-tuned on
the same dataset, whereas InstructUIE is significantly larger (approximately
30 times so). This demonstrates that our proposed architecture is indeed
competitive. However, our model falls behind UniNER by almost 3 points.
Nevertheless, our model still manages to achieve the best score in 7 out of 20
datasets.

Figure 4: Zero-shot performance for different backbones. It reports the avg.
results on 20 NER and OOD NER datasets

##  5 Further analysis and ablations

Here we conduct different set of experiments to better investigate our model.

###  5.1 Effect of Different Backbones

In our work, we primarily utilize the deBERTa-v3 model as our backbone due to
its strong empirical performance. However, we demonstrate here that our method
is adaptable to a wide range of BiLMs.

#### Setup

Specifically, we investigate the performance of our model using other popular
BiLMs, including BERT (Devlin et al., 2019), RoBERTa (Liu et al., 2019),
AlBERT (Lan et al., 2019), and ELECTRA (Clark et al., 2020). We also conducted
experiments with XLNet (Yang et al., 2019) but did not achieve acceptable
performance (achieving at most 3 F1 on the OOD benchmark) despite extensive
hyperparameter tuning. For a fair comparison, we employed the base size
(GLiNER-M) and tuned the learning rate for each model. We report the zero-shot
results on both the OOD benchmark and the 20 NER benchmark in Figure 4.

#### Result

The results of our experiment, as shown in the Figure 4, clearly demonstrate
the superiority of deBERTa-v3 over other pretrained BiLMs. It achieves the
highest performance on both benchmarks by a clear margin. ELECTRA and AlBERT
also show notable performance, albeit slightly lower, while BERT and RoBERTa
lag behind with similar scores. However, it should be noted that all of the
backbones we tested demonstrate strong performance compared to existing
models. More specifically, even BERT-base, which ranks among the lower
performers, achieves around 49 F1 on the OOD benchmark. This score is still 2
F1 points higher than the average for models like ChatGPT and InstructUIE.

###  5.2 Effect of Pretraining on In-domain Performance

In this section, we investigate the impact of pretraining on the Pile-NER
dataset for supervised in-domain training on the 20 NER datasets, across
various data sizes. The experiments range from 100 samples per dataset to
10,000 (full training setup). We use the same hyperparameters for all
configurations. The results are reported in Figure 5.

#### Results

As shown in the figure, models pretrained on Pile-NER consistently outperform
their counterparts that are only trained on supervised data, indicating
successful positive transfer. We further observe that the gain is larger when
supervised data is limited. For instance, the difference in performance is 5.6
when employing 100 samples per dataset, and the gap becomes smaller as the
size of the dataset increases.

Figure 5: Supervised performance across different dataset sizes. The evaluation is conducted on the 20 NER datasets (in table 4). Negative Samples | Prec | Rec | F1
---|---|---|---
0% | 49.3 | 58.1 | 53.3
50% | 62.3 | 59.7 | 60.9
75% | 61.1 | 56.5 | 58.6
Table 5: Effect of negative entity types sampling.

###  5.3 Ablations

#### Negative Entity Sampling

The original Pile-NER dataset, curated by Zhou et al. (2023), features
passages with positive entity instances, i.e., entities that are directly
present in the text. To better align training with real-world scenarios, where
some entity types might be absent, we implemented negative entity sampling as
mentioned in Section 3.2. In this study, we evaluate different sampling
ratios: 0% (only positive entities), 50%, and 75% negative entities. Table 5
shows that training with only positive entities results in lower precision but
higher recall, indicating that the model often makes false positive errors.
Conversely, using 75% negative entities increases precision but decreases
recall, as the abundance of negatives makes the model more cautious, leading
to missed correct entities. A 50% negative entity ratio proves to be the most
effective, providing a balanced approach.

#### Entity type dropping

In the experiment, we employed a strategy of randomly varying the number of
entity prompts during training. This approach aimed to expose the model to
different quantities of entity types in each training instance, thereby
increasing its adaptability to handle scenarios with varying numbers of
entities. The usage of this technique results in an average improvement of
over 1.4 points in out-of-domain evaluation, as shown in the Figure 5.

##  6 Related Works

#### Named Entity Recognition

NER is a well-established task in the field of NLP, with numerous
applications. Initially, NER models relied on rule-based system (Weischedel et
al., 1996) that were built using handcrafted algorithms and gazetteers
(Mikheev et al., 1999; Nadeau et al., 2006; Zamin and Oxley, 2011). However,
these models had limitations in terms of scalability and adaptability to new
domains or languages. To overcome these issues, machine learning approaches
have been proposed (Lafferty et al., 2001). In the early stages, NER tasks
were designed as sequence labeling (Huang et al., 2015; Lample et al., 2016;
Akbik et al., 2018) where the objective was to predict tagged sequences (e.g.,
BILOU tags (Ratinov and Roth, 2009)). Since then, several paradigm shifts have
occurred: span-based approaches treating NER as span classification (Sarawagi
and Cohen, 2004; Fu et al., 2021; Li et al., 2021; Zaratiana et al., 2023);
NER being treated as a question answering problem (Li et al., 2019); and even
as a generation task (Yan et al., 2021).

#### Zero-shot learning for NER

The advent of large-scale autoregressive models has recently transformed many
paradigms in NLP through natural language prompting (Min et al., 2022; Wei et
al., 2022; Qin et al., 2023). This is also the case for NER (Li et al., 2022;
Ashok and Lipton, 2023; Agrawal et al., 2022). Others have fine-tuned these
models for tasks to better align their capabilities with the requirements of
entity recognition (Cui et al., 2021; Zhou et al., 2023) or information
extraction in general (Lou et al., 2023; Wang et al., 2023; Sainz et al.,
2023; Lu et al., 2022). This is done through-instruction tuning (Mishra et
al., 2021; Wang et al., 2022; Longpre et al., 2023).

Figure 6: Randomly dropping entity types. We report the results with and
without negative entity sampling.

##  7 Conclusion

In this paper, we introduced GLiNER, a new method for identifying various
types of entities in text using bidirectional language models. Our model not
only outperforms state-of-the-art Large Language Models like ChatGPT in zero-
shot scenarios but also offers a more resource-efficient alternative, crucial
for environments with limited computing power. GLiNER is versatile, performing
well in multiple languages, including those it wasn’t trained on. In future
work, we aim to further improve GLiNER’s design for enhanced performance and
to better adapt it for low-resource languages.

## References

  * Agrawal et al. (2022) Monica Agrawal, Stefan Hegselmann, Hunter Lang, Yoon Kim, and David A. Sontag. 2022.  Large language models are few-shot clinical information extractors.  In _Conference on Empirical Methods in Natural Language Processing_.
  * Akbik et al. (2018) Alan Akbik, Duncan Blythe, and Roland Vollgraf. 2018.  Contextual string embeddings for sequence labeling.  In _Proceedings of the 27th International Conference on Computational Linguistics_ , pages 1638–1649, Santa Fe, New Mexico, USA. Association for Computational Linguistics.
  * Ashok and Lipton (2023) Dhananjay Ashok and Zachary Chase Lipton. 2023.  Promptner: Prompting for named entity recognition.  _ArXiv_ , abs/2305.15444.
  * Brown et al. (2020) Tom B. Brown, Benjamin Mann, Nick Ryder, Melanie Subbiah, Jared Kaplan, Prafulla Dhariwal, Arvind Neelakantan, Pranav Shyam, Girish Sastry, Amanda Askell, Sandhini Agarwal, Ariel Herbert-Voss, Gretchen Krueger, T. J. Henighan, Rewon Child, Aditya Ramesh, Daniel M. Ziegler, Jeff Wu, Clemens Winter, Christopher Hesse, Mark Chen, Eric Sigler, Mateusz Litwin, Scott Gray, Benjamin Chess, Jack Clark, Christopher Berner, Sam McCandlish, Alec Radford, Ilya Sutskever, and Dario Amodei. 2020.  Language models are few-shot learners.  _ArXiv_ , abs/2005.14165.
  * Chiang et al. (2023) Wei-Lin Chiang, Zhuohan Li, Zi Lin, Ying Sheng, Zhanghao Wu, Hao Zhang, Lianmin Zheng, Siyuan Zhuang, Yonghao Zhuang, Joseph E. Gonzalez, Ion Stoica, and Eric P. Xing. 2023.  Vicuna: An open-source chatbot impressing gpt-4 with 90%* chatgpt quality.
  * Chung et al. (2022) Hyung Won Chung, Le Hou, Shayne Longpre, Barret Zoph, Yi Tay, William Fedus, Yunxuan Li, Xuezhi Wang, Mostafa Dehghani, Siddhartha Brahma, Albert Webson, Shixiang Shane Gu, Zhuyun Dai, Mirac Suzgun, Xinyun Chen, Aakanksha Chowdhery, Alex Castro-Ros, Marie Pellat, Kevin Robinson, Dasha Valter, Sharan Narang, Gaurav Mishra, Adams Yu, Vincent Zhao, Yanping Huang, Andrew Dai, Hongkun Yu, Slav Petrov, Ed H. Chi, Jeff Dean, Jacob Devlin, Adam Roberts, Denny Zhou, Quoc V. Le, and Jason Wei. 2022.  Scaling instruction-finetuned language models.
  * Clark et al. (2020) Kevin Clark, Minh-Thang Luong, Quoc V. Le, and Christopher D. Manning. 2020.  Electra: Pre-training text encoders as discriminators rather than generators.
  * Conneau et al. (2019) Alexis Conneau, Kartikay Khandelwal, Naman Goyal, Vishrav Chaudhary, Guillaume Wenzek, Francisco Guzmán, Edouard Grave, Myle Ott, Luke Zettlemoyer, and Veselin Stoyanov. 2019.  Unsupervised cross-lingual representation learning at scale.  In _Annual Meeting of the Association for Computational Linguistics_.
  * Cui et al. (2021) Leyang Cui, Yu Wu, Jian Liu, Sen Yang, and Yue Zhang. 2021.  Template-based named entity recognition using bart.  In _Findings_.
  * Devlin et al. (2019) Jacob Devlin, Ming-Wei Chang, Kenton Lee, and Kristina Toutanova. 2019.  Bert: Pre-training of deep bidirectional transformers for language understanding.  In _North American Chapter of the Association for Computational Linguistics_.
  * Fu et al. (2021) Jinlan Fu, Xuanjing Huang, and Pengfei Liu. 2021.  SpanNER: Named entity re-/recognition as span prediction.  In _Proceedings of the 59th Annual Meeting of the Association for Computational Linguistics and the 11th International Joint Conference on Natural Language Processing (Volume 1: Long Papers)_ , pages 7183–7195, Online. Association for Computational Linguistics.
  * Gao et al. (2020) Leo Gao, Stella Rose Biderman, Sid Black, Laurence Golding, Travis Hoppe, Charles Foster, Jason Phang, Horace He, Anish Thite, Noa Nabeshima, Shawn Presser, and Connor Leahy. 2020.  The pile: An 800gb dataset of diverse text for language modeling.  _ArXiv_ , abs/2101.00027.
  * He et al. (2021) Pengcheng He, Jianfeng Gao, and Weizhu Chen. 2021.  Debertav3: Improving deberta using electra-style pre-training with gradient-disentangled embedding sharing.  _ArXiv_ , abs/2111.09543.
  * Huang et al. (2015) Zhiheng Huang, Wei Xu, and Kai Yu. 2015.  Bidirectional lstm-crf models for sequence tagging.
  * Lafferty et al. (2001) John D. Lafferty, Andrew McCallum, and Fernando Pereira. 2001.  Conditional random fields: Probabilistic models for segmenting and labeling sequence data.  In _International Conference on Machine Learning_.
  * Lai et al. (2023) Viet Dac Lai, Nghia Trung Ngo, Amir Pouran Ben Veyseh, Hieu Man, Franck Dernoncourt, Trung Bui, and Thien Huu Nguyen. 2023.  Chatgpt beyond english: Towards a comprehensive evaluation of large language models in multilingual learning.  _ArXiv_ , abs/2304.05613.
  * Lample et al. (2016) Guillaume Lample, Miguel Ballesteros, Sandeep Subramanian, Kazuya Kawakami, and Chris Dyer. 2016.  Neural architectures for named entity recognition.  In _North American Chapter of the Association for Computational Linguistics_.
  * Lan et al. (2019) Zhenzhong Lan, Mingda Chen, Sebastian Goodman, Kevin Gimpel, Piyush Sharma, and Radu Soricut. 2019.  Albert: A lite bert for self-supervised learning of language representations.  _ArXiv_ , abs/1909.11942.
  * Li et al. (2022) Dongfang Li, Baotian Hu, and Qingcai Chen. 2022.  Prompt-based text entailment for low-resource named entity recognition.  _ArXiv_ , abs/2211.03039.
  * Li et al. (2019) Xiaoya Li, Jingrong Feng, Yuxian Meng, Qinghong Han, Fei Wu, and Jiwei Li. 2019.  A unified mrc framework for named entity recognition.  _ArXiv_ , abs/1910.11476.
  * Li et al. (2021) Yangming Li, lemao liu, and Shuming Shi. 2021.  Empirical analysis of unlabeled entity problem in named entity recognition.  In _International Conference on Learning Representations_.
  * Liu et al. (2019) Yinhan Liu, Myle Ott, Naman Goyal, Jingfei Du, Mandar Joshi, Danqi Chen, Omer Levy, Mike Lewis, Luke Zettlemoyer, and Veselin Stoyanov. 2019.  Roberta: A robustly optimized bert pretraining approach.  _ArXiv_ , abs/1907.11692.
  * Longpre et al. (2023) S. Longpre, Le Hou, Tu Vu, Albert Webson, Hyung Won Chung, Yi Tay, Denny Zhou, Quoc V. Le, Barret Zoph, Jason Wei, and Adam Roberts. 2023.  The flan collection: Designing data and methods for effective instruction tuning.  In _International Conference on Machine Learning_.
  * Loshchilov and Hutter (2017) Ilya Loshchilov and Frank Hutter. 2017.  Decoupled weight decay regularization.  In _International Conference on Learning Representations_.
  * Lou et al. (2023) Jie Lou, Yaojie Lu, Dai Dai, Wei Jia, Hongyu Lin, Xianpei Han, Le Sun, and Hua Wu. 2023.  Universal information extraction as unified semantic matching.  In _AAAI Conference on Artificial Intelligence_.
  * Lu et al. (2022) Yaojie Lu, Qing Liu, Dai Dai, Xinyan Xiao, Hongyu Lin, Xianpei Han, Le Sun, and Hua Wu. 2022.  Unified structure generation for universal information extraction.  In _Annual Meeting of the Association for Computational Linguistics_.
  * Malmasi et al. (2022) Shervin Malmasi, Anjie Fang, Besnik Fetahu, Sudipta Kar, and Oleg Rokhlenko. 2022.  Multiconer: A large-scale multilingual dataset for complex named entity recognition.  In _International Conference on Computational Linguistics_.
  * Mikheev et al. (1999) Andrei Mikheev, Marc Moens, and Claire Grover. 1999.  Named entity recognition without gazetteers.  In _Conference of the European Chapter of the Association for Computational Linguistics_.
  * Min et al. (2022) Sewon Min, Xinxi Lyu, Ari Holtzman, Mikel Artetxe, Mike Lewis, Hannaneh Hajishirzi, and Luke Zettlemoyer. 2022.  Rethinking the role of demonstrations: What makes in-context learning work? _ArXiv_ , abs/2202.12837.
  * Mishra et al. (2021) Swaroop Mishra, Daniel Khashabi, Chitta Baral, and Hannaneh Hajishirzi. 2021.  Cross-task generalization via natural language crowdsourcing instructions.  In _Annual Meeting of the Association for Computational Linguistics_.
  * Nadeau et al. (2006) David Nadeau, Peter D. Turney, and Stan Matwin. 2006.  Unsupervised named-entity recognition: Generating gazetteers and resolving ambiguity.  In _Canadian Conference on AI_.
  * OpenAI (2023) OpenAI. 2023.  Gpt-4 technical report.
  * Qin et al. (2023) Chengwei Qin, Aston Zhang, Zhuosheng Zhang, Jiaao Chen, Michihiro Yasunaga, and Diyi Yang. 2023.  Is chatgpt a general-purpose natural language processing task solver? _ArXiv_ , abs/2302.06476.
  * Raffel et al. (2019) Colin Raffel, Noam M. Shazeer, Adam Roberts, Katherine Lee, Sharan Narang, Michael Matena, Yanqi Zhou, Wei Li, and Peter J. Liu. 2019.  Exploring the limits of transfer learning with a unified text-to-text transformer.  _J. Mach. Learn. Res._ , 21:140:1–140:67.
  * Ratinov and Roth (2009) Lev-Arie Ratinov and Dan Roth. 2009.  Design challenges and misconceptions in named entity recognition.  In _Conference on Computational Natural Language Learning_.
  * Rozière et al. (2023) Baptiste Rozière, Jonas Gehring, Fabian Gloeckle, Sten Sootla, Itai Gat, Xiaoqing Tan, Yossi Adi, Jingyu Liu, Tal Remez, Jérémy Rapin, Artyom Kozhevnikov, I. Evtimov, Joanna Bitton, Manish P Bhatt, Cristian Cantón Ferrer, Aaron Grattafiori, Wenhan Xiong, Alexandre D’efossez, Jade Copet, Faisal Azhar, Hugo Touvron, Louis Martin, Nicolas Usunier, Thomas Scialom, and Gabriel Synnaeve. 2023.  Code llama: Open foundation models for code.  _ArXiv_ , abs/2308.12950.
  * Sainz et al. (2023) Oscar Sainz, Iker García-Ferrero, Rodrigo Agerri, Oier Lopez de Lacalle, German Rigau, and Eneko Agirre. 2023.  Gollie: Annotation guidelines improve zero-shot information-extraction.  _ArXiv_ , abs/2310.03668.
  * Sarawagi and Cohen (2004) Sunita Sarawagi and William W. Cohen. 2004.  Semi-markov conditional random fields for information extraction.  In _Neural Information Processing Systems_.
  * Touvron et al. (2023) Hugo Touvron, Thibaut Lavril, Gautier Izacard, Xavier Martinet, Marie-Anne Lachaux, Timothée Lacroix, Baptiste Rozière, Naman Goyal, Eric Hambro, Faisal Azhar, Aurelien Rodriguez, Armand Joulin, Edouard Grave, and Guillaume Lample. 2023.  Llama: Open and efficient foundation language models.  _ArXiv_ , abs/2302.13971.
  * Wang et al. (2023) Xiao Wang, Wei Zhou, Can Zu, Han Xia, Tianze Chen, Yuan Zhang, Rui Zheng, Junjie Ye, Qi Zhang, Tao Gui, Jihua Kang, J. Yang, Siyuan Li, and Chunsai Du. 2023.  Instructuie: Multi-task instruction tuning for unified information extraction.  _ArXiv_ , abs/2304.08085.
  * Wang et al. (2022) Yizhong Wang, Swaroop Mishra, Pegah Alipoormolabashi, Yeganeh Kordi, Amirreza Mirzaei, Anjana Arunkumar, Arjun Ashok, Arut Selvan Dhanasekaran, Atharva Naik, David Stap, Eshaan Pathak, Giannis Karamanolakis, Haizhi Gary Lai, Ishan Purohit, Ishani Mondal, Jacob Anderson, Kirby Kuznia, Krima Doshi, Maitreya Patel, Kuntal Kumar Pal, M. Moradshahi, Mihir Parmar, Mirali Purohit, Neeraj Varshney, Phani Rohitha Kaza, Pulkit Verma, Ravsehaj Singh Puri, Rushang Karia, Shailaja Keyur Sampat, Savan Doshi, Siddharth Deepak Mishra, Sujan Reddy, Sumanta Patro, Tanay Dixit, Xudong Shen, Chitta Baral, Yejin Choi, Noah A. Smith, Hannaneh Hajishirzi, and Daniel Khashabi. 2022.  Super-naturalinstructions: Generalization via declarative instructions on 1600+ nlp tasks.  In _Conference on Empirical Methods in Natural Language Processing_.
  * Wei et al. (2022) Jason Wei, Xuezhi Wang, Dale Schuurmans, Maarten Bosma, Ed Huai hsin Chi, F. Xia, Quoc Le, and Denny Zhou. 2022.  Chain of thought prompting elicits reasoning in large language models.  _ArXiv_ , abs/2201.11903.
  * Weischedel et al. (1996) Ralph Weischedel, Sean Boisen, Daniel Bikel, Robert Bobrow, Michael Crystal, William Ferguson, Allan Wechsler, and The PLUM Research Group. 1996.  Progress in information extraction.  In _TIPSTER TEXT PROGRAM PHASE II: Proceedings of a Workshop held at Vienna, Virginia, May 6-8, 1996_ , pages 127–138, Vienna, Virginia, USA. Association for Computational Linguistics.
  * Yan et al. (2021) Hang Yan, Tao Gui, Junqi Dai, Qipeng Guo, Zheng Zhang, and Xipeng Qiu. 2021.  A unified generative framework for various ner subtasks.
  * Yang et al. (2019) Zhilin Yang, Zihang Dai, Yiming Yang, Jaime G. Carbonell, Ruslan Salakhutdinov, and Quoc V. Le. 2019.  Xlnet: Generalized autoregressive pretraining for language understanding.  In _Neural Information Processing Systems_.
  * Ye et al. (2023) Junjie Ye, Xuanting Chen, Nuo Xu, Can Zu, Zekai Shao, Shichun Liu, Yuhan Cui, Zeyang Zhou, Chao Gong, Yang Shen, Jie Zhou, Siming Chen, Tao Gui, Qi Zhang, and Xuanjing Huang. 2023.  A comprehensive capability analysis of gpt-3 and gpt-3.5 series models.  _ArXiv_ , abs/2303.10420.
  * Zamin and Oxley (2011) Norshuhani Zamin and Alan Oxley. 2011.  Building a corpus-derived gazetteer for named entity recognition.  In _International Conference on Software Engineering and Computer Systems_.
  * Zaratiana et al. (2023) Urchade Zaratiana, Nadi Tomeh, Niama Elkhbir, Pierre Holat, and Thierry Charnois. 2023.  Filtered semi-markov CRF.
  * Zaratiana et al. (2022) Urchade Zaratiana, Nadi Tomeh, Pierre Holat, and Thierry Charnois. 2022.  Named entity recognition as structured span prediction.  In _Proceedings of the Workshop on Unimodal and Multimodal Induction of Linguistic Structures (UM-IoS)_ , pages 1–10, Abu Dhabi, United Arab Emirates (Hybrid). Association for Computational Linguistics.
  * Zhou et al. (2023) Wenxuan Zhou, Sheng Zhang, Yu Gu, Muhao Chen, and Hoifung Poon. 2023.  Universalner: Targeted distillation from large language models for open named entity recognition.


††thanks: Present address: SLAC, Menlo Park, USA

# Bayesian Optimization of the Beam Injection Process into a Storage Ring

Chenran Xu (徐晨冉<EMAIL_ADDRESS>Tobias Boltz  Akira Mochihashi  Andrea
Santamaria Garcia  Marcel Schuh  Anke-Susanne Müller  Karlsruhe Institute of
Technology, Kaiserstraße 12, 76131 Karlsruhe, Germany

###### Abstract

We have evaluated the data-efficient Bayesian optimization method for the
specific task of injection tuning in a circular accelerator. In this paper, we
describe the implementation of this method at the Karlsruhe Research
Accelerator with up to nine tuning parameters, including the determination of
the associated hyperparameters. We show that the Bayesian optimization method
outperforms manual tuning and the commonly used Nelder-Mead optimization
algorithm both in simulation and experiment. The algorithm was also
successfully used to ease the commissioning phase after the installation of
new injection magnets and is regularly used during accelerator operations. We
demonstrate that the introduction of context variables that include intra-
bunch scattering effects, such as the Touschek effect, further improves the
control and robustness of the injection process.

##  I Introduction

Particle accelerators contribute to major discoveries in particle physics and
are used to generate synchrotron radiation for photon science applications
[1]. They consist of a large and varied number of components, from magnets to
steer, focus, or quickly redirect the particle beam, to radio frequency (RF)
sources to accelerate it. This leads to a large number of possible parameter
combinations for tuning, which is a complex and time consuming task. In
addition, the beam dynamics are influenced by many non-linearly correlated
parameters and are subject to physical phenomena such as the Touschek effect
[2]. Although advanced control systems are employed, the fine tuning of beam
properties is and will be a challenge, especially for future large-scale and
compact accelerators. While intervention is required for several accelerator
operation tasks, manual tuning may not find the global optimum, especially in
the shortest possible time, and does not adapt well to short-term and long-
term drifts of the accelerator condition due to the large number of correlated
parameters that influence it.

In such a case, computer algorithms can be introduced to assist the operator
and eventually automate the tuning process [3]. Metaheuristics such as the
evolutionary algorithm [4] and particle swarm optimization [5] need a large
number of evaluation steps to converge and are not suited for online tuning.
Gradient-based methods like robust conjugate direction search [6, 7] are
successfully used for online accelerator tuning, but they are prone to get
stuck in local optima and observation noise. Another promising local
optimization method is the model independent extremum seeking (ES) algorithm.
Despite also being sensitive to local optima, ES can be used as a feedback
control to track the optimum settings with respect to drifts of the
accelerator components. It has been successfully applied to tune time-varying
accelerator systems with a large number of parameters [8, 9]. As a method to
globally optimize an unknown function with expensive evaluations, Bayesian
Optimization (BO) is shown to perform well among other approaches [10]. With
observed data, BO builds a surrogate model of the unknown function using a
Gaussian process (GP) [11] and uses an acquisition function to guide the
search efficiently. Recently, BO has been successfully applied at the LCLS
[12, 13] and the SwissFEL [14] for free-electron laser performance tuning,
motivating the usage of Bayesian optimization for other accelerator tuning
tasks [15, 16].

The process of beam injection can be approximately viewed as a black-box
function optimization problem, which makes BO a well-suited method for the
injection tuning. To investigate this specific optimization problem, we use
the accelerator test facility Karlsruhe Research Accelerator (KARA), which is
a 110-meter storage ring and part of the KIT Light Source. It produces
synchrotron radiation for photon science users and accelerator physics
experiments at a top energy of $2.5\text{\,}\mathrm{G}\mathrm{e}\mathrm{V}$.
The electron storage ring is filled at
$500\text{\,}\mathrm{M}\mathrm{e}\mathrm{V}$ by a chain of pre-accelerators
with a repetition rate of $1\text{\,}\mathrm{H}\mathrm{z}$. The injection
tuning is performed manually, and initial studies have shown that BO is well-
suited to optimize this task [17]. Furthermore, it is known that the injection
condition may vary from one day to another and operators need to re-tune the
injection elements. This might result from a change in the environmental
conditions such as the ambient temperature or the quality of the vacuum ,
variables that are either non-controllable or can only be changed slowly. In
this article, besides showing the implementation of BO at a storage ring with
several examples and use cases, we also take into account the effect of the
environment by generalizing the standard BO method to contextual Bayesian
optimization (CBO) [18] to optimize the injection efficiency under the given
context.

##  II Bayesian Optimization

Bayesian optimization (BO) is a method designed to optimize a black-box
function $f$ in a sample-efficient way. The BO algorithm used in this paper is
shown in pseudo-code in Algorithm 1 [19]. It makes use of a statistical
surrogate model of the objective function $f$, generally built with a Gaussian
Process (GP), based on the belief that the objective function $f$ is drawn
from some prior probability distribution $p(A)$. After initialization and
observations $f(x)$, the posterior distribution $p(A|f(x))$ is built according
to the Bayes’ theorem

$p(A|f(x))\propto p(f(x)|A)p(A).$ (1)

The posterior distribution is further used to build an acquisition function
$\alpha(x)$, which determines the next point for evaluation.

Algorithm 1 Bayesian optimization algorithm

1:Define the prior for the GP.

2:Observe objective function $f$ on $n_{0}$ initial points to get initial data
set $D_{0}$.

3:for $t=1,2,...$ do

4: Build a GP using available data $D_{t-1}$.

5: Find the next evaluation point

6: $x_{t}=\operatorname{arg\,max}_{x}\alpha(x\;|\;D_{t-1})$.

7: Observe $y_{t}$ at point $x_{t}$.

8: Augment the dataset $D_{t}=D_{t-1}\cup{(x_{t},y_{t})}$.

9:end for

###  II.1 Gaussian Process

A Gaussian process (GP) is a generalization of the multivariate Gaussian
distribution and is characterized by its mean and covariance functions
$f(x)\sim GP(\mu(x),k(x,x^{{}^{\prime}}))$. In this study, we set the prior
mean function to be $\mu(x)=0$, which is commonly used when the shape of the
objective function is unknown. The covariance function $k(\cdot,\cdot)$, also
known as kernel, measures the similarity between data points. Based on the
assumption that $f$ is continuous, the data points $x$ that are close to each
other are expected to have similar output values to $f(x)$. In this study, we
use the widely used radial basis function (RBF) as the kernel function

$k_{\text{RBF}}(x,x^{{}^{\prime}})=\exp\left(-\frac{1}{2}\sum^{d}_{i=1}\left(\frac{x_{i}-x_{i}^{{}^{\prime}}}{l_{i}}\right)^{2}\right),$
(2)

where $l_{i}$ represents the length-scale of the i-th input dimension. It
roughly corresponds to the distance along one input axis, at which the two
data points become uncorrelated and the function values can change
significantly.

In order to emulate the noise present in the real observed signal, we
explicitly incorporate stochastic noise by adding Gaussian distributed noise
to the covariance function as diagonal terms. Thus, the kernel becomes

$k(x_{i},x_{j})=\sigma^{2}k_{\text{RBF}}(x_{i},x_{j})+\sigma_{\text{noise}}^{2}\delta_{ij}.$
(3)

The signal variance $\sigma^{2}$, the noise variance
$\sigma_{\text{noise}}^{2}$, and the length-scales $l_{i}$ are referred to as
the hyperparameters, which determine the behavior of the GP. Since no
dedicated optimization data is stored in the database, several measurements
are performed to estimate the hyperparameter settings.

Figure 1:  Example of a parameter scan measurement of the septum magnet
voltage, where all other accelerator tuning parameters are kept constant
during the scan. The GP hyperparameters $\\{l_{i},\sigma^{2}\\}$ are fitted to
the measured data points (crosses) via a log-likelihood fit. The solid line
depicts the GP posterior mean and the shaded region shows the 95% confidence
level.

First, the noise $\sigma_{\text{noise}}$ is determined by fixing the
accelerator settings and measuring the fluctuation of the objective function,
i.e. the injection efficiency, as defined in Eq. 7. By the assumption that the
noise mostly comes from the statistical fluctuation between different
injection shots, $\sigma_{\text{noise}}$ can be extracted as the standard
deviation of a Gaussian fit.

Secondly, the length-scales $l_{i}$ and the signal variance $\sigma^{2}$ are
estimated from one-dimensional parameter scans. Figure 1 shows the scan result
of the septum voltage, which corresponds to its magnetic field strength. We
extract the GP hyperparameters with the maximum likelihood fit.

###  II.2 Acquisition Function

With a GP model inferring the posterior distribution of the objective
function, an acquisition function $\alpha$ is built to determine the next
sample point to evaluate, so that the number of required physical observations
is effectively reduced. Concretely, the objective $f$ is sampled at
$\operatorname{arg\,max}_{x}\alpha(x|D)$ at each step, where $D$ is the
observed dataset. We use two different acquisition functions in this study:
the upper confidence bound (UCB) and the expected improvement (EI). UCB
explicitly controls the exploration-exploitation trade-off with a parameter
$\kappa$

$\alpha_{\text{UCB}}(x)=\mu(x)+\kappa\sigma(x),$ (4)

where $\mu(x)$ and $\sigma(x)$ are the GP posterior mean and standard
deviation. For high $\kappa$ values, the contribution of the variance term
becomes large and points with high uncertainty are sampled, which leads to
more exploration. On the contrary, small $\kappa$ values lead to more
exploitation of observed peaks, as they have a higher posterior mean.

An empirical choice of the parameter is $\kappa=2$, which corresponds to the
usual 95% confidence bound for Gaussian distributed values. Nevertheless,
$\kappa$ can also increase along with the evaluation steps to ensure that BO
converges to the global optimum [19, 20].

The expected improvement (EI) calculates the expected value of the improvement
of a proposed point $x$ over the best observed value $f_{\text{best}}$ [21]

$\begin{split}\alpha_{\text{EI}}(x)&=\mathbb{E}[\max(\mu(x)-(f_{\text{best}}+\xi),0)]\\\
&=(\mu(x)-(f_{\text{best}}+\xi))\Phi(Z)+\sigma(x)\phi(Z)\end{split}$ (5)

with the parameter $Z$ describing the normalized improvement

$\begin{split}Z&=\frac{\mu(x)-(f_{\text{best}}+\xi)}{\sigma(x)},\end{split}$
(6)

where $\phi$ is the probability distribution function and $\Phi$ is the
cumulative distribution function of the standard normal distribution. The
exploration-exploitation trade-off in EI is determined by a positive parameter
$\xi$. In general, higher $\xi$ values lead to more exploration. In this
study, we use the recommended value of $\xi=0.01$ according to Ref. [22],
which emphasizes exploitation.

##  III Implementation at a storage ring

In this section, we describe the implementation of the BO method to improve
the injection efficiency into a storage ring, namely the storage ring of the
KIT Light Source, the Karlsruhe Research Accelerator (KARA).

The electron storage ring is filled at
$500\text{\,}\mathrm{M}\mathrm{e}\mathrm{V}$ with a repetition rate of
$1\text{\,}\mathrm{H}\mathrm{z}$ by a chain of pre-accelerators, consisting of
an electron gun, a microtron, and a booster synchrotron. Further details can
be found in Ref. [23].

The storage ring employs a three-kicker injection scheme [24], as shown in
Fig. 2, to create a closed orbit bump. When the kicker magnets are powered
they steer the closed orbit away from the nominal trajectory and near the
septum magnet. Kicker magnets are fast pulsed (in the order of
$\mathrm{\SIUnitSymbolMicro}\mathrm{s}$) with a relatively low field strength,
so they are combined with septa magnets that provide a stronger field. The
septum magnet deflects the beam into the aperture of the storage ring and
provides a separation space between the circulating and injected beam. The
separation barrier (septum) is usually as thin as possible with minimum field
leakage. The injection bump orbit at KARA spans over one quadrant of the
storage ring. There are several sextupoles in the injection bump which add
non-linearities to the beam dynamics of the injection process. The stray field
of the septum magnet and the potential energy mismatch between the booster and
the storage ring also affect the settings required for a closed bump orbit.
Moreover, due to the lack of non-destructive beam diagnostics in the transfer
line except of one current monitor, the tuning is partly blind and relies
mostly on the operators’ experience.

In order to obtain a good injection rate, operators have to adjust multiple
parameters like the magnet currents and radio frequency (RF) parameters. The
manual tuning often ends up at a local optimum with lower injection rate,
which eventually reduces the beamtime availability. The tuning task after a
shutdown period can be especially difficult, since previous accelerator
settings can no longer be used and it costs valuable beamtime to recover the
expected injection performance. We show that the BO method outperforms the
simplex methods and achieves a sample-efficient optimization.

Figure 2:  In the injection scheme, electron bunches are injected from a
synchrotron via an injection line into the storage ring. The relative
positions of the magnets and their location are illustrated as they are placed
at the storage ring KARA at KIT. See Ref. [23] for further details. The tuning
parameters of the magnets and RF cavities, not shown in this figure, are used
as optimization parameters.

We selected 9 tuning parameters for the BO based on their influence on the
injection process and from prior knowledge of experienced operators. The
parameters are: the main radio frequency (RF), the injection septum magnet
strength, strength of one horizontal corrector magnet, and the strength and
timing of the three kicker magnets. The strength and timing of the three
kickers defines the shape of the injection bump orbit. The strength of the
septum magnet changes the angle of the beam injected into the storage ring.
The corrector magnet chosen here is located directly before the first kicker
magnet and controls the angle of the incoming beam to the injection bump.
Finally, the RF defines the orbit length and by changing it slightly the
closed-orbit condition is modified. The relative positions of the magnets
relevant for the injection optimization process are illustrated in Fig. 2.

Since only the storage ring parameters are considered, the objective function
is also chosen accordingly to decouple the effect of the pre-accelerators. The
injection efficiency is defined as the net injected current into the storage
ring (SR) normalized by the current measured at the booster synchrotron (S)
extraction point before the beam enters the injection line to KARA

$f_{\text{I-eff, exp.}}=\frac{\Delta
I_{\text{SR}}}{I_{\text{S}}}\frac{h_{\text{SR}}}{h_{\text{S}}},$ (7)

where ${h_{\text{SR}}}$ and ${h_{\text{S}}}$ are the harmonic numbers of the
storage ring and the booster synchrotron, respectively. The change of the
storage ring current $\Delta I_{\text{SR}}$ and the extracted bunch current
from the synchrotron $I_{\text{S}}$ are read out synchronously to calculate
the objective function. Whereas for the simulation study, $n_{i}$ electrons
extracted from the booster synchrotron and $n_{s}$ already stored electrons in
the storage ring are tracked. The objective function is then calculated from
the electrons that are successfully injected into the storage ring
$n_{i,\text{success}}$ and the stored electrons that are lost during the
injection process $n_{s,\text{lost}}$

$f_{\text{I-eff, sim.}}=\frac{n_{i,\text{success}}-n_{s,\text{lost}}}{n_{i}}.$
(8)

During a realistic beam injection process at a storage ring, the stored
current is usually much higher than the injected bunch current. Thus, a
detuned parameter setting causing a beam loss can result in a very large
negative objective function value and distort the landscape of the GP model.
Additionally, there could be outliers due to faulty readback values or
occasional beam losses. In order to mitigate these effects, the lower limit of
the injection efficiency is constrained to be $-1$, so that the GP model is
more robust to the fluctuation of readback values. This should not affect the
parameter region near the optima, where the stored beam is minimally disturbed
and the injection efficiency is always positive.

The BO package implemented in this study uses the Python interface pyepics
[25] for the communication with the control system of the accelerators, which
is based on the Experimental Physics and Industrial Control Systems (EPICS)
[26]. The software package GPy [27] is used for building the GP model and
scipy [28] functionalities are used for maximizing the acquisition functions.

###  III.1 Optimization Results in Simulation

Before deploying the algorithms at the accelerators, the BO is first tested on
a simulation. The injection model is built in Accelerator Toolbox for MATLAB
(AT) [29]. We used a simplified lattice without magnet errors based on an
existing MATLAB model for KARA [30, 31].

Figure 3:  Comparison of (a) a 2-dimensional grid scan (grid size 400) of the
injection efficiency with (b) the GP posterior mean predicted based on 20 BO
evaluation steps, shown in red crosses. The tuning parameters are the
strengths of the horizontal corrector magnet and the septum magnet, given in
deflection angles. The BO can approximate the simple structure of parameter
space in a small number of evaluation steps.

In the simulation, $n_{i}$ injected and $n_{s}$ stored electrons are generated
according to the particle distribution and tracked for the first 100 turns
after injection without collective effects. This proved to be sufficient, as
over 95% of the particle loss happens during the first 10 turns. The injection
efficiency is calculated according to Eq. 8.

The BO algorithm is first tested using two tuning parameters: the septum and
the corrector magnet strength. For comparison, a grid scan, shown in Fig. 3a,
is conducted with 400 observations in total. Fig. 3b shows the GP posterior
mean function after 20 evaluations. It can be seen that the BO already
converges to the maximum and is able to model the peak structure.

Figure 4:  Optimization results in an AT simulation model using UCB (green,
solid) and EI (blue, dashed) acquisition functions for the (a) 3- and (b)
6-dimensional problems. The lines depict the best evaluated injection
efficiency in each run, averaged over 10 independent runs. The shaded areas
are the one $\sigma$ spread. Within 50 and 100 steps for 3- and 6-dimensional
problems respectively, all the optimization runs converged to the optimum
setting. The required steps for convergence increases with the input
dimensions and both acquisition functions have a similar performance.

As a next step, the RF frequency is included as a third tuning parameter. For
the optimization, the BO randomly samples 5 points to initialize the GP model.
The results on the 3-dimensional problem using two different acquisition
functions are shown in Fig. 4a. It can be seen that both EI and UCB are able
to find the optimum in a small number of evaluation steps. Then, we include
the strength of the three kicker magnets as tuning parameters and the results
are shown in Fig. 4b. Although the number of required evaluations is larger,
BO can still efficiently solve the 6-dimensional problem.

Figure 5:  Example of a non-localized orbit defined by the magnet strengths
found by the BO algorithm (navy, solid), in comparison to the reference orbit
(grey, dashed) with design values, where the injection bump orbit is fully
localized. The orbit oscillation outside of the injection bump is not taken
into consideration by BO, as it causes no beam loss.

Nevertheless, BO does not take other physical properties into account, since
it only optimizes for the objective function $f_{\text{I-eff, sim.}}$ (cf.
Eq.8). For example, as can be seen in Fig. 5, the strength of the three kicker
magnets is often not matched perfectly, which leads to a non-localized
injection bump orbit. Although a large orbit oscillation of about
$3\text{\,}\mathrm{m}\mathrm{m}$ along the storage ring is visible, the
electrons can be stored and no beam loss is observed.

###  III.2 Beam Lifetime Correction

In simulation, the collective effects are neglected and the beam losses are
only due to detuned settings of the injection magnets, while in the actual
storage ring the electrons are lost over time due to other effects such as
scattering within the bunch and current dependent tune-shifts. In this study,
we focus on the Touschek effect, which is one of the main contributions to the
reduction of beam lifetime. Additionally, particles can be lost due to the
scattering with the residual gas, which depends on the vacuum quality. The two
effects can be combined in terms of the total lifetime $\tau$

$\frac{1}{\tau}=\frac{1}{\tau_{\text{T}}}+\frac{1}{\tau_{\text{other}}}=aI+b,$
(9)

where the Touschek lifetime $\tau_{\text{T}}$ is dominant and inversely
proportional to the current $I$, and other contributions
$\tau_{\text{other}}$, including beam-gas scattering, are approximately
constant.

The beam lifetime decreases for higher accumulated current and leads to a
decrease of the calculated injection efficiency, which is independent of the
performance of the BO algorithm. As a result, the objective function will
continue to decrease regardless of the sampling region, which is not taken
into account in our definition of the objective. It is observed that BO often
fails to optimize, if this effect is not properly dealt with.

This effect can be counteracted by including the beam current as a context
variable, which is discussed in section III.5. Alternatively, we explicitly
correct this effect for normal BO by measuring the lifetime related beam loss.
For the latter, we measured the beam lifetime at the injection optics and
fitted the Eq. 9 to it. The injection efficiencies are then corrected
accordingly in the next experiments.

###  III.3 Experimental Optimization Results

In the following section, we describe the results of the BO algorithms in a
real-world accelerator environment. We investigate its performance using the
EI and UCB acquisition functions introduced in Section II.2.

Figure 6:  Experimental optimization result of the injection efficiency using
BO with two different acquisition functions and Nelder-Mead algorithm (red,
dotted). UCB (green, solid) and EI (blue, dashed) have similar results and are
both able to optimize the injection efficiency within a reasonable number of
evaluations. For 9-dimensional problem, NM is clearly slower than BO.

We use the RBF kernel (Eq. 2) with the hyperparameters determined from the
results in Section II.1. For each test run, the machine was manually detuned
to a fixed initial setting, where the beam injection was still possible with a
very low injection efficiency. For the UCB acquisition function shown in Eq.
4, we chose the setting of $\kappa$ according to [20], which dynamically
changes the exploration-exploitation behavior

$\kappa=\sqrt{2\nu\log{(t^{d/2+2}\pi^{2}/(3\delta)}},$ (10)

with $\nu=1$ and $\delta=0.1$ being free parameters. The parameter $\kappa$
becomes larger with the number of evaluations $t$ and the algorithm focuses
more on the exploration.

The optimization was performed for a fixed number of steps: 50 steps for 3-
and 6-dimensional problems and 100 steps for 9-dimensional problem. This
corresponds to about 10-20 minutes for each optimization run, where most of
the time was spent on setting the parameter values and the evaluating the
injection efficiency. The computation time of the GP model and the acquisition
function is negligible. Additionally, it should be noted that we intentionally
decreased the total booster synchrotron current $I_{\text{S}}$ in the
following experiments to reduce the radiation dose. This is expected not to
influence the result, as the objective $f_{\text{I-eff, exp.}}$ defined in Eq.
7 is normalized with respect to $I_{\text{S}}$.

Figure 6 shows the averaged performance over 3 optimization runs for each
setting. For all the optimization runs, the injection efficiency clearly
increased from the detuned initial setting within the allowed evaluation
steps. The final achieved objective function value is, however, lower than the
result obtained in simulation studies, which is due to various reasons. First,
the objective function $f_{\text{I-eff, exp}}$ is normalized with respect to
the current measured in booster synchrotron $I_{\text{S}}$ right before
extraction. The beam loss in the injection line is not accounted for, which
needs to be mitigated by including additional magnets in BO. However this is
beyond the scope of this paper. Secondly, the current readout in the storage
ring and the synchrotron booster are calibrated independently, affecting their
ratio. Nevertheless, the performance of the BO can be compared and benchmarked
qualitatively. It can be seen that in the 3-dimensional case (Fig. 6a), the
UCB evaluations are more noisy than the other configurations, which is mainly
due to the setting of the trade-off parameter $\kappa$. However, for 6- and
9-dimensional problems, the parameter space is larger and more exploration
becomes indeed necessary. Therefore, the performance of both UCB and EI is
quite similar in those cases (Fig. 6 b,c). Due to its focus on exploration,
occasional unexpected beam loss is observed during the UCB runs. This can be
circumvented either by introducing a safety constraint, or setting $\kappa$ to
a lower value to focus more on exploitation.

For comparison, we perform the same optimization procedure using the Nelder-
Mead algorithm [32]. Nelder-Mead is a widely used numerical optimization
method and is considered as a standard benchmark in the accelerator community
[7]. The algorithm keeps track of $d+1$ evaluation points and uses these to
build a simplex. The Nelder-Mead method searches for the optimum via geometric
modifications of the stored simplex and is proven to converge relatively fast.
The results of the Nelder-Mead algorithm are also plotted in Fig. 6. For the
3- and 6-dimensional cases, Nelder-Mead achieves a similar performance to BO,
whereas for the 9 dimensional problem the optimization speed of Nelder-Mead is
clearly slower. The fact that Nelder-Mead scales not as well as BO is
expected, as the local search becomes less applicable for higher dimensional
spaces. The primary problem for BO to scale to higher dimensions is the
computation time of the GP model and maximizing the acquisition function.
However, these are still noncritical for $d<20$ [33]. It is worth mentioning
that we used randomly selected parameter settings to initialize the GP model,
which often leads to a poor performance in the first few steps, as can be seen
in Fig. 6 3-dimensional case. Alternatively, one can mitigate that by using
the initial steps of other algorithms, such as Nelder-Mead, or some historical
data to initialize the GP model. This could avoid the unstable phase of BO in
the beginning and subsequently reach quicker convergence.

Figure 7:  Convergence behavior of BO with UCB (green, triangle) and EI (blue,
square) acquisition functions and Nelder-Mead (red, circle) on the
9-dimensional problem showing the distance $\Delta x$ between two consecutive
sampled points in the scaled parameter space. Due to the noisy evaluation of
the objective function, Nelder-Mead optimizes mostly locally and often needs
to be restarted, whereas BO optimizes globally and doesn’t require a restart.

The distances $\Delta x$ between two consecutive sampled points of BO and
Nelder-Mead for the 9-dimensional problem are shown in Fig. 7. It is visible
that Nelder-Mead searches only locally. Since Nelder-Mead cannot handle the
noisy signal well, it easily gets stuck and continues to contract, which
eventually converges to a non-optimal setting. To prevent Nelder-Mead from
breaking down and force exploration of the parameter space, it is
automatically restarted when the step becomes too small. On the contrary, BO
is more robust against the observation noise by explicitly modeling it as a
hyperparameter. It can be seen that BO generally samples at a larger distance
and does not become trapped in local optima.

In the experiments we found that the timing of the the kicker magnets are
consistent with the previous set values. As visible in Fig. 6c, adding the
timing parameters only slows down the optimization and does not improve the
objective function further. Thus, we run the BO always with 6 parameters in
the following sections.

As mentioned in the Section III.1, the settings found by BO approach sometimes
result in a not closed injection bump orbit. We also observed this effect
during the experiments on KARA, despite the fact that the orbit oscillation is
partly mitigated by the radiation damping. Since the trajectory of the
electron beam is not centered through the quadrupole and sextupole magnets,
the resulting orbit is often different from the one usually obtained via
manual tuning and sometimes a slight tune shift is visible. Although this
effect does not affect the beam injection process for KARA, it might be
critical for other accelerators with more stringent constraints on orbit and
betatron tunes.

Figure 8:  Beam injection up to $100\text{\,}\mathrm{m}\mathrm{A}$ using BO
result (navy) compared to the manual tuning result (grey) obtained from an
operation day. The dashed lines are the accumulated current of BO and manual
tuning result. BO was roughly twice as fast as the manual tuning, greatly
reducing the required beam injection time.

Lastly, the optimal settings found by BO are compared to the result of one day
with dedicated manual tuning, taken a week prior to the BO experiment with a
comparable accelerator condition. In this experiment, the full booster current
is used in order to achieve higher injection rate. As shown in Fig. 8, BO took
$18\text{\,}\mathrm{m}\mathrm{i}\mathrm{n}$ and manual tuning took
$33\text{\,}\mathrm{m}\mathrm{i}\mathrm{n}$ for the injection of
$100\text{\,}\mathrm{m}\mathrm{A}$ current. BO clearly outperforms the manual
tuning and significantly reduces the required time for beam injection.

###  III.4 Application to the Commissioning of Injection Magnets

During a scheduled shutdown period, the power supplies of the septum and
kicker magnets were exchanged. Thus, it was necessary to find new parameter
settings during commissioning.

Figure 9:  Two runs with the highest (purple, solid) and lowest (yellow,
dashed) optimized injection efficiency achieved are plotted out of all five BO
runs with UCB acquisition using the new power supplies. The evolution of the
input parameters and the resulting objective function are shown over the
optimization steps, where the vertical limit is the total allowed optimization
range respectively. The bottom plot shows the measured injection functions
(markers) with the cumulative best results (lines). The dotted line depicts
the 90% quantile of the injection efficiency obtained by manual operation in a
month prior to the shutdown period. Even the BO run with worst performance
achieved a comparable result as the manual baseline.

Two steps are needed to apply BO and assist the commissioning of the new power
supplies: 1. The interface with the machine is updated, so that BO can control
the parameters of the new magnets. 2. Several parameter scans are conducted in
order to estimate the length scales and optimization ranges for the new
parameters. Once BO was reconfigured for the new machine status, it quickly
found new working points for the beam injection.

The BO started from a fixed non-optimal setting and was run multiple times
using 6 input parameters. Figure 9 compares two of these runs with the highest
and lowest achieved injection efficiency among all five runs. The evolution of
the parameters during the optimization are shown within the allowed tuning
range of each parameter respectively. The RF frequency is varied very little
and the set values are comparable for two runs, which is as expected. Since
the RF frequency directly depends on the ring circumference, it should only
change due to, for instance, the change of ambient temperature over the year.
The optimized septum strength is clearly higher than the starting point. This
is consistent with the expectation that the septum magnet strongly influences
the injection process, as it directly acts on the injected electron bunch. On
the other hand, the strength of kicker 2 barely changed. Since the kicker
magnet 2 is located closely after the septum magnet, it possibly implies that
the BO prefers to use only the septum magnet for beam deflection. The two runs
shown here have found different values for kicker 1 and 3, which lead to
different bump orbit shape and eventually different injection efficiency.

We also show the manual performance benchmark in Fig. 9, where the data is
collected in a month of synchrotron light operation prior to the shutdown
period with storage ring current up to $100\text{\,}\mathrm{m}\mathrm{A}$.
This value $f_{\text{manual}}=0.43$ is calculated as the 90% quantile of the
injection efficiencies obtained by manual tuning, so that the outliers due to
the fluctuation of readback values are excluded. All the BO runs reached or
exceeded the optimal settings found by manual tuning. This result shows that
BO is robust against the change of the accelerator’s condition and can indeed
reduce the commissioning time after even drastic changes to the accelerator
environment such as replacing elements to new model components.

###  III.5 Contextual Optimization

Lastly, we present the first results of including contextual parameters and
extending the BO algorithm to contextual Bayesian optimization (CBO) [18].

Figure 10:  Optimization results of BO (navy, solid) and contextual BO (cyan,
dashdot) on the 6-dimensional problem. Each line is the averaged injection
efficiencies from three runs. By using the stored current as a context
variable, CBO optimizes faster and obtains a consistently higher injection
efficiency than normal BO after about 10 steps.

In the above experiments we treat the beam injection problem as stationary,
which is only true as an approximation. In reality, the optimal injection
condition is affected by various context variables, which are ignored by the
normal BO. One example are collective effects, which become more relevant with
the accumulated current. For instance, a parameter setting with a larger
closed orbit is non-critical for low bunch current and could be considered
good by the normal BO. However, the same setting might cause more instability
and become non-optimal for a higher current. Therefore, we include the storage
ring current $I$ as a context variable by adding the current as an additional
dimension to the RBF kernel and the GP model. The acquisition function is then
maximized over the parameter space with the context dimension fixed. In this
case, we set the length-scale of the current to be
$l_{I}=$5\text{\,}\mathrm{m}\mathrm{A}$$ by empirical testing. We do not
correct the loss rate explicitly as in normal BO, and let the algorithm itself
recognize this effect. As before, we initialize CBO from a manually detuned
setting and use UCB acquisition. The number of evaluation steps is set to 200,
allowing the optimizer to achieve a higher current value. The normal BO is
performed with the same configurations. Figure 10 shows the results, where
each line is the mean objective averaged from three runs. The CBO results are
corrected afterwards for the comparison with BO results. We can see that CBO
can properly handle the current-dependent effect and optimizes efficiently. It
also shows a consistently better performance than BO. This is partly due to
the fact that the context variable provides an extra dimension for
exploration. To some extent, CBO is able to track the optimal setting along
the current dimension.

##  IV Summary and Outlook

In this article, we presented the implementation of Bayesian optimization
applied to the injection task at a storage ring. The algorithms were tested in
simulation and also in experiments, for which the accelerator test facility
and storage ring KARA at KIT was used. The hyperparameters were determined
from some simple scan measurements, since no dedicated archive data with
diverse parameter settings was available. We showed that BO can optimize the
injection efficiency in a small number of evaluations, which makes it suitable
for online optimization. We demonstrated experimentally that the BO method is
robust against noisy observations, outperforming the commonly used Nelder-Mead
method. We also present a real use case of BO for machine commissioning and
show that BO can be quickly reconfigured to find new parameter settings after
a change in the machine status. Lastly, we use the accumulated current as a
context variable for the optimization, which shows a better performance than
explicitly correcting for the Touschek effect. The final goal of this work is
to fully automate the injection tuning process, where the BO algorithm
implemented in this paper serves as a basic framework. It is now used by
operators routinely to assist the injection optimization task for the weekly
machine start-up. Nevertheless, the current version of BO could explore
excessively and lose the stored beam due to a poor setting of the kicker
magnets. To mitigate that, safety constraints can be included, so that the
exploration is restricted to safe parameter regions. With safety constraints
implemented, BO is expected to be also applicable for more use cases, or even
at other accelerators with a more stringent machine protection requirements.

The required computation time for the presented BO with 9 tuning parameters is
still negligible compared to the setting of new accelerator parameters. Thus
we expect that this framework can be effortlessly extended to a few tens of
tuning parameters by, for example, adding all magnets in the pre-accelerators.
Other context parameters, such as temperature and vacuum quality, can also be
incorporated.

Optimizing only for the highest injection efficiency sometimes results in an
unmatched setting of the kicker magnets where the injection bump is not
localized, which changes the closed orbit and can lead to a tune shift or a
lower beam lifetime. This can be mitigated by considering the beam orbit and
betatron tune as additional objective functions. Generalizing the single
objective BO to multi-objective optimization makes it possible to
simultaneously trade-off between multiple objectives, like the orbit deviation
and the injection rate, and find optimal parameter settings for user-specific
cases. Furthermore, the BO approach may find its application in other
accelerator control tasks. It is best-suited for quasi-static tuning tasks,
where the optimization condition is not drastically changed within the time
scale of the optimization and repeated evaluation of same settings will result
in comparable objective function values, e.g. tuning a linear accelerator or a
free-electron laser (FEL). Using BO in tandem with other tuning methods will
further improve its performance. For example, the Nelder-Mead can be used to
provide initial samples of the GP model, and the extremum seeking algorithm
can be used as a feedback system to track the optimum obtained by BO.

Whereas for solving dynamic problems for which the accelerator condition
continues to change during the optimization, we expect reinforcement learning
(RL) to be a promising alternative method [34, 35, 36, 37]. We believe that
combining the strength of BO and RL could automate the accelerator operation
to a great extent and achieve performance beyond manual-tuning.

## Acknowledgements

We thank E. Bründermann for reviewing and providing invaluable suggestions for
improving this paper. C. Xu acknowledges the support by the DFG-funded
Doctoral School ”Karlsruhe School of Elementary and Astroparticle Physics:
Science and Technology”.

## References

  * Bründermann  _et al._ [2012] E. Bründermann, H.-W. Hübers, M. Fitz, and G. Kimmitt,  _Terahertz techniques_ , Springer series in optical sciences, Vol. 151 (Springer Verlag, 2012) 54.01.01; LK 01.
  * Bernardini  _et al._ [1963] C. Bernardini, G. F. Corazza, G. Di Giugno, G. Ghigo, J. Haissinski, P. Marin, R. Querzoli, and B. Touschek, Lifetime and beam size in a storage ring, Phys. Rev. Lett. 10, 407 (1963).
  * Edelen  _et al._ [2018] A. Edelen, C. Mayes, D. Bowring, D. Ratner, A. Adelmann, R. Ischebeck, J. Snuverink, I. Agapov, R. Kammering, J. Edelen, I. Bazarov, G. Valentino, and J. Wenninger, Opportunities in Machine Learning for Particle Accelerators (2018), arXiv:1811.03172 .
  * Vikhar [2016] P. A. Vikhar, Evolutionary algorithms: A critical review and its future prospects, in  _2016 International Conference on Global Trends in Signal Processing, Information Computing and Communication (ICGTSPICC)_ (2016) pp. 261–265.
  * Kennedy and Eberhart [1995] J. Kennedy and R. Eberhart, Particle swarm optimization, in  _Proceedings of ICNN’95 - International Conference on Neural Networks_ , Vol. 4 (1995) pp. 1942–1948 vol.4.
  * Huang and Safranek [2015] X. Huang and J. Safranek, Online optimization of storage ring nonlinear beam dynamics, Phys. Rev. ST Accel. Beams 18, 084001 (2015).
  * Huang [2018] X. Huang, Robust simplex algorithm for online optimization, Phys. Rev. Accel. Beams 21, 10.1103/PhysRevAccelBeams.21.104601 (2018).
  * Scheinker  _et al._ [2018] A. Scheinker, X. Huang, and J. Wu, Minimization of betatron oscillations of electron beam injected into a time-varying lattice via extremum seeking, IEEE Transactions on Control Systems Technology 26, 336 (2018).
  * Scheinker  _et al._ [2019] A. Scheinker, D. Bohler, S. Tomin, R. Kammering, I. Zagorodnov, H. Schlarb, M. Scholz, B. Beutner, and W. Decking, Model-independent tuning for maximizing free electron laser pulse energy, Phys. Rev. Accel. Beams 22, 082802 (2019).
  * Jones [2001] D. R. Jones, A Taxonomy of Global Optimization Methods Based on Response Surfaces, J. Glob. Optim. 10.1023/A:1012771025575 (2001).
  * Rasmussen and Williams [2005] C. E. Rasmussen and C. K. I. Williams,  _Gaussian Processes for Machine Learning_ (The MIT Press, 2005).
  * McIntire  _et al._ [2016] M. McIntire, T. Cope, D. Ratner, and S. Ermon, Bayesian optimization of FEL performance at LCLS, in  _IPAC 2016 - Proc. 7th Int. Part. Accel. Conf._ (2016) pp. 2972–2975.
  * Duris  _et al._ [2020] J. Duris, D. Kennedy, A. Hanuka, J. Shtalenkova, A. Edelen, P. Baxevanis, A. Egger, T. Cope, M. McIntire, S. Ermon, and D. Ratner, Bayesian optimization of a free-electron laser, Phys. Rev. Lett. 124, 124801 (2020).
  * Kirschner  _et al._ [2019] J. Kirschner, A. Adelmann, N. Hiller, R. Ischebeck, A. Krause, M. Mutný, and M. Nonnenmacher, Bayesian Optimisation for Fast and Safe Parameter Tuning of SwissFEL, in  _39th International Free Electron Laser Conference_ (2019) p. THP061.
  * Hanuka  _et al._ [2019] A. Hanuka, J. Duris, J. Shtalenkova, D. Kennedy, A. Edelen, D. Ratner, and X. Huang, Online tuning and light source control using a physics-informed gaussian process (2019), arXiv:1911.01538 [physics.acc-ph] .
  * Roussel  _et al._ [2021] R. Roussel, A. Hanuka, and A. Edelen, Multiobjective bayesian optimization for online accelerator tuning, Phys. Rev. Accel. Beams 24, 062801 (2021).
  * Xu [2020] C. Xu,  _Bayesian Optimization of Injection Efficiency at KARA using Gaussian Processes_ , Master’s thesis, Karlsruhe Institute of Technology (2020).
  * Krause and Ong [2011] A. Krause and C. S. Ong, Contextual Gaussian process bandit optimization, in  _Adv. Neural Inf. Process. Syst. 24 25th Annu. Conf. Neural Inf. Process. Syst. 2011, NIPS 2011_ (2011).
  * Brochu  _et al._ [2010] E. Brochu, V. Cora, and N. Freitas, A tutorial on bayesian optimization of expensive cost functions, with application to active user modeling and hierarchical reinforcement learning, CoRR abs/1012.2599 (2010).
  * Srinivas  _et al._ [2010] N. Srinivas, A. Krause, S. Kakade, and M. Seeger, Gaussian process optimization in the bandit setting: No regret and experimental design, in  _ICML 2010 - Proceedings, 27th Int. Conf. Mach. Learn._ (2010) pp. 1015–1022, arXiv:0912.3995 .
  * Jones  _et al._ [1998] D. R. Jones, M. Schonlau, and W. J. Welch, Efficient Global Optimization of Expensive Black-Box Functions, J. Glob. Optim. 13, 455 (1998).
  * Lizotte [2008] D. J. Lizotte,  _Practical Bayesian Optimization_ , Ph.D. thesis, University of Alberta (2008).
  * Einfeld  _et al._ [1998] D. Einfeld, S. Hermle, E. Huttel, R. Rossmanith, and R. Walther, The injection scheme for the ANKA storage ring, in  _6th European Particle Accelerator Conference (EPAC 98)_ (1998) pp. 2135–2137.
  * Turner [1994] S. Turner, ed.,  _CAS-CERN Accelerator School: 5th general accelerator physics course, Jyvaskyla, Finland, 7-18 Sep 1992: Proceedings. 2 vol._, CERN Yellow Reports: School Proceedings (1994).
  * Newville  _et al._ [2019] M. Newville, K. Lauer, dchabot, T. A. Caswell, M. Gibbs, A. Péteut, S. Hartman, rokvintar, R. Clarken, B. Martins, D. Allan, T. Birke, P. R. Jemian, N. Claesson, J. Adelman, J. Dwyer, H. Slepicka, G. Brandl, E. Greenberg, D. Vine, and André, pyepics/pyepics 3.4.0 (2019).
  * Dalesio  _et al._ [1991] L. R. Dalesio, M. R. Kraimer, and a. J. Kozubal, EPICS Architecture, Proc. 1991 ICALEPCS (1991).
  * GPy [2012] GPy, GPy: A gaussian process framework in python, http://github.com/SheffieldML/GPy (since 2012).
  * Virtanen and SciPy 1.0 Contributors [2020] P. Virtanen and SciPy 1.0 Contributors, SciPy 1.0: Fundamental Algorithms for Scientific Computing in Python, Nature Methods 17, 261 (2020).
  * Terebilo [2001] A. Terebilo, Accelerator toolbox for MATLAB, in  _Workshop on Performance Issues at Synchrotron Light Sources_ (2001).
  * Portmann  _et al._ [2005] G. Portmann, J. Corbett, and A. Terebilo, An Accelerator control middle layer using Matlab, Conf. Proc. C 0505161, 4009 (2005).
  * Marsching  _et al._ [2011] S. Marsching, E. Huttel, M. Klein, and N. J. Smale, First Experience with the Matlab Middle Layer at ANKA, in  _ICALEPCS 2011_ (2011).
  * Nelder and Mead [1965] J. A. Nelder and R. Mead, A simplex method for function minimization, Comput. J. 7, 308 (1965).
  * Frazier [2018] P. I. Frazier, A tutorial on bayesian optimization (2018), arXiv:1807.02811 [stat.ML] .
  * St. John  _et al._ [2021] J. St. John, C. Herwig, D. Kafkes, J. Mitrevski, W. A. Pellico, G. N. Perdue, A. Quintero-Parra, B. A. Schupbach, K. Seiya, N. Tran, M. Schram, J. M. Duarte, Y. Huang, and R. Keller, Real-time artificial intelligence for accelerator control: A study at the fermilab booster, Phys. Rev. Accel. Beams 24, 104601 (2021).
  * Eichler  _et al._ [2021] A. Eichler, E. Bründermann, F. Burkart, J. Kaiser, W. Kuropka, A. Santamaria Garcia, O. Stein, and C. Xu, First Steps Toward an Autonomous Accelerator, a Common Project Between DESY and KIT, in  _Proc. IPAC’21_, International Particle Accelerator Conference No. 12 (JACoW Publishing, Geneva, Switzerland, 2021) pp. 2182–2185, https://doi.org/10.18429/JACoW-IPAC2021-TUPAB298.
  * Boltz  _et al._ [2020] T. Boltz, E. Bründermann, M. Caselle, A. Kopmann, W. Mexner, A.-S. Müller, and W. Wang, Accelerating Machine Learning for Machine Physics (an AMALEA-project at KIT), in  _Proc. ICALEPCS’19_, International Conference on Accelerator and Large Experimental Physics Control Systems No. 17 (JACoW Publishing, Geneva, Switzerland, 2020) pp. 781–788, https://doi.org/10.18429/JACoW-ICALEPCS2019-TUCPL06.
  * Wang  _et al._ [2021] W. Wang, M. Caselle, T. Boltz, E. Blomley, M. Brosi, T. Dritschler, A. Ebersoldt, A. Kopmann, A. Santamaria Garcia, P. Schreiber, E. Bründermann, M. Weber, A.-S. Müller, and Y. Fang, Accelerated deep reinforcement learning for fast feedback of beam dynamics at kara, IEEE Transactions on Nuclear Science 68, 1794 (2021).