# MCSE: Multimodal Contrastive Learning of Sentence Embeddings

Miaoran Zhang, Marius Mosbach, David Ifeoluwa Adelani,
Michael A. Hedderich, and Dietrich Klakow
Spoken Language Systems (LSV)
Saarland Informatics Campus, Saarland University, Germany
<EMAIL_ADDRESS>
<EMAIL_ADDRESS>

###### Abstract

Learning semantically meaningful sentence embeddings is an open problem in
natural language processing. In this work, we propose a sentence embedding
learning approach that exploits both visual and textual information via a
multimodal contrastive objective. Through experiments on a variety of semantic
textual similarity tasks, we demonstrate that our approach consistently
improves the performance across various datasets and pre-trained encoders. In
particular, combining a small amount of multimodal data with a large text-only
corpus, we improve the state-of-the-art average Spearman’s correlation by
$1.7\%$. By analyzing the properties of the textual embedding space, we show
that our model excels in aligning semantically similar sentences, providing an
explanation for its improved performance.

##  1 Introduction

Sentence embedding learning, i.e., encoding sentences into fixed-length
vectors that faithfully reflect the semantic relatedness among sentences, is a
fundamental challenge in natural language processing (NLP). Despite the
tremendous success of pre-trained language models (PLMs), such as BERT (Devlin
et al., 2019) and RoBERTa (Liu et al., 2019), it has been shown that the off-
the-shelf sentence embeddings of PLMs without fine-tuning are even inferior to
averaging Glove embeddings (Pennington et al., 2014) in terms of semantic
similarity measure (Reimers and Gurevych, 2019). Hence, recent research (Li et
al., 2020; Zhang et al., 2020; Su et al., 2021) focuses on adjusting the
original sentence embeddings derived from PLMs in an unsupervised manner. In
particular, there has been growing interest in adopting contrastive learning
objectives to achieve this goal (Carlsson et al., 2020; Kim et al., 2021; Gao
et al., 2021).

Although purely text-based models have led to impressive progress, it remains
an open question to what extent they capture the deeper notion of sentence
meaning beyond the statistical distribution of texts, which lies outside of
the text and is grounded in the real-world (Bender and Koller, 2020; Bisk et
al., 2020). As a central part of the human perceptual experience, vision has
been shown to be effective in grounding language models and improving
performance on various NLP tasks (Zhang et al., 2019; Bordes et al., 2019;
Zhao and Titov, 2020). We hypothesize that using vision as supplementary
semantic information can further promote sentence representation learning.

In this work, we propose MCSE, an approach for multimodal contrastive learning
of sentence embeddings. To exploit both visual and textual information, we
adopt the state-of-the-art contrastive sentence embedding framework SimCSE
(Gao et al., 2021) and extend it with a multimodal contrastive objective. In
addition to the textual objective in SimCSE that maximizes agreement between
positive sentence pairs, the multimodal objective maximizes agreement between
sentences and corresponding images in a shared space. We conduct extensive
experiments on standard Semantic Textual Similarity (STS) benchmarks and show
the effectiveness of MCSE across various datasets and pre-trained encoders. We
find that, using a small amount of multimodal data in addition to a text-only
corpus yields significant improvements on STS tasks. By analyzing the
alignment and uniformity properties of the embedding space (Wang and Isola,
2020), we show that MCSE better aligns the semantically similar sentences
while maintaining uniformity, providing an explanation for its superior
performance.111Our code and pre-trained models are publicly available at
https://github.com/uds-lsv/MCSE.

Figure 1: The overall architecture of MCSE. Compared to SimCSE, a new
multimodal objective is calculated in the grounded space. For each input
sentence, the positive instance is the paired image and the negative instances
are all other in-batch images.

##  2 Related Work

Sentence Representation Learning. Existing works for learning sentence
embeddings can be categorized into supervised (Conneau et al., 2017; Cer et
al., 2018; Reimers and Gurevych, 2019; Wieting et al., 2020) and unsupervised
approaches (Li et al., 2020; Carlsson et al., 2020; Su et al., 2021; Kim et
al., 2021; Gao et al., 2021; Liu et al., 2021; Yan et al., 2021). Supervised
approaches mostly utilize supervision from annotated natural language
inference data or parallel data. Unsupervised approaches are able to make use
of the intrinsic semantic information embedded in the natural language text
corpus by adjusting the training objective to STS tasks, thereby eliminating
the need for a costly annotation process. In particular, contrastive learning
objective (Carlsson et al., 2020; Kim et al., 2021; Gao et al., 2021; Liu et
al., 2021; Yan et al., 2021) regularizes the embedding space by pulling
positive (i.e., semantically similar) sentences closer and pushing apart
negatives, showcasing great effectiveness in capturing the semantic similarity
among sentences. Our approach adopts the contrastive learning framework and is
built on top of the current state-of-the-art approach (Gao et al., 2021),
further pushing the frontier of STS by leveraging multimodal semantic
information.

Visually Grounded Representation Learning. There are various works showing
that grounding NLP models to the visual world can improve textual
representation learning. Lazaridou et al. (2015) and Zablocki et al. (2018)
learn word embeddings by aligning words to the visual entity or visual
context. Kiela et al. (2018) ground sentence embeddings by predicting both
images and alternative captions related to the same image. Bordes et al.
(2019) enhance the Skip-Thought model (Kiros et al., 2015) by learning a
grounded space that preserves the structure of visual and textual spaces.
Recently, Tan and Bansal (2020) and Tang et al. (2021) train large scale
language models with multimodal supervision from scratch with the goal of
improving general language understanding. Different from the aforementioned
works, we focus on learning visually grounded sentence embeddings by fine-
tuning pre-trained models in a contrastive learning framework.

| Model | STS12 | STS13 | STS14 | STS15 | STS16 | STS-B | SICK-R | Avg.$\uparrow$
---|---|---|---|---|---|---|---|---|---
| BERT (first-last avg.) | 39.7 | 59.4 | 49.7 | 66.0 | 66.2 | 53.9 | 62.1 | 56.7
| RoBERTa (first-last avg.) | 40.9 | 58.7 | 49.1 | 65.6 | 61.5 | 58.6 | 61.6 | 56.6
wiki | SimCSE-BERT♢ | 68.4 | 82.4 | 74.4 | 80.9 | 78.6 | 76.9 | 72.2 | 76.3
SimCSE-RoBERTa♢ | 70.2 | 81.8 | 73.2 | 81.4 | 80.7 | 80.2 | 68.6 | 76.6
SimCSE-BERT | 67.8±1.6 | 80.0±2.1 | 72.5±1.7 | 80.1±0.8 | 77.6±0.8 | 76.5±0.8 | 70.1±0.9 | 74.9±1.1
SimCSE-RoBERTa | 68.7±1.0 | 82.0±0.5 | 74.0±1.0 | 82.1±0.4 | 81.1±0.4 | 80.6±0.3 | 69.2±0.2 | 76.8±0.5
wiki+flickr | SimCSE-BERT | 69.9±1.7 | 79.8±1.5 | 72.9±0.9 | 81.9±0.8 | 77.8±0.9 | 76.6±1.1 | 68.4±0.8 | 75.3±0.9
MCSE-BERT | 71.4±0.9 | 81.8${}^{*}_{\pm 1.3}$ | 74.8${}^{*}_{\pm 0.9}$ | 83.6±0.9 | 77.5±0.8 | 79.5${}^{*}_{\pm 0.5}$ | 72.6${}^{*}_{\pm 1.4}$ | 77.3${}^{*}_{\pm 0.5}$
SimCSE-RoBERTa | 69.5±0.9 | 81.6±0.5 | 74.1±0.6 | 82.4±0.3 | 80.9±0.5 | 79.9±0.3 | 67.3±0.5 | 76.5±0.4
MCSE-RoBERTa | 71.7${}^{*}_{\pm 0.2}$ | 82.7${}^{*}_{\pm 0.4}$ | 75.9${}^{*}_{\pm 0.3}$ | 84.0${}^{*}_{\pm 0.4}$ | 81.3±0.3 | 82.3${}^{*}_{\pm 0.5}$ | 70.3${}^{*}_{\pm 1.3}$ | 78.3${}^{*}_{\pm 0.1}$
wiki+coco | SimCSE-BERT | 69.1±1.0 | 80.4±0.9 | 72.7±0.7 | 81.1±0.3 | 78.2±0.9 | 73.9±0.6 | 66.6±1.2 | 74.6±0.2
MCSE-BERT | 71.2${}^{*}_{\pm 1.3}$ | 79.7±0.9 | 73.8±0.9 | 83.0${}^{*}_{\pm 0.4}$ | 77.8±0.9 | 78.5${}^{*}_{\pm 0.4}$ | 72.1${}^{*}_{\pm 1.4}$ | 76.6${}^{*}_{\pm 0.5}$
SimCSE-RoBERTa | 66.4±0.9 | 80.7±0.7 | 72.7±1.1 | 81.3±0.9 | 80.2±0.8 | 76.8±0.6 | 65.7±0.7 | 74.8±0.5
MCSE-RoBERTa | 70.2${}^{*}_{\pm 1.7}$ | 82.0${}^{*}_{\pm 0.7}$ | 75.5${}^{*}_{\pm 1.2}$ | 83.0${}^{*}_{\pm 0.6}$ | 81.5${}^{*}_{\pm 0.7}$ | 80.8${}^{*}_{\pm 1.0}$ | 69.9${}^{*}_{\pm 0.6}$ | 77.6${}^{*}_{\pm 0.8}$

$\ast$: difference between SimCSE and MCSE is significant at $\alpha=0.05$
according to an independent t-test.

Table 1: Performance comparison on STS tasks. STS-B: STS Benchmark, SICK-R:
SICK-Relatedness, Avg.: average across $7$ tasks. $\diamondsuit$ : single seed
results from Gao et al. (2021). All other results are from our implementation.
Models are trained with $5$ random seeds and we report the means and standard
deviations.

##  3 Method

To exploit both visual and textual information, we adopt SimCSE (Gao et al.,
2021) as the textual baseline and extend it with a multimodal contrastive
learning objective.

###  3.1 Background: Unsupervised SimCSE

Data augmentation plays a critical role in contrastive self-supervised
representation learning (Chen et al., 2020). The idea of unsupervised SimCSE
is to use dropout noise as a simple yet effective data augmentation strategy.
Given a collection of sentences $\\{x_{i}\\}_{i=1}^{m}$, we construct a
positive pair for each input $x_{i}$ by encoding it twice using different
dropout masks: ${\bm{h}}_{i}^{z}=g_{\phi}(f_{\theta}(x_{i},z))$ and
${\bm{h}}_{i}^{z^{\prime}}=g_{\phi}(f_{\theta}(x_{i},z^{\prime}))$, where $z$
and $z^{\prime}$ denote different dropout masks222The standard dropout masks
in Transformers are used., $f_{\theta}(\cdot)$ is a pre-trained language
encoder such as BERT, and $g_{\phi}(\cdot)$ is a projection head333There is a
MLP pooler layer over [CLS] in BERT’s implementation. Gao et al. (2021) use it
with re-initialization. on top of the [CLS] token. The training objective is:

$\ell_{i}^{\mathrm{S}}=-\log\frac{e^{\mathrm{sim}({\bm{h}}_{i}^{z_{i}},{\bm{h}}_{i}^{z_{i}^{\prime}})/\tau}}{\sum_{j=1}^{N}e^{\mathrm{sim}({\bm{h}}_{i}^{z_{i}},{\bm{h}}_{j}^{z_{j}^{\prime}})/\tau}}\;,$
(1)

where $N$ is the size of the mini-batch, $\tau$ is a temperature parameter and
$\mathrm{sim}({\bm{h}}_{1},{\bm{h}}_{2})$ is the cosine similarity
$\frac{{\bm{h}}_{1}^{T}{\bm{h}}_{2}}{\left\|{\bm{h}}_{1}\right\|\cdot\left\|{\bm{h}}_{2}\right\|}$.
After training, the [CLS] token outputs of the language encoder are taken as
the sentence embeddings.

###  3.2 Multimodal Contrastive Learning

Beyond the textual objective in SimCSE, we introduce a multimodal objective
within the contrastive learning framework. The overview of our MCSE model is
shown in Figure 1. Given a collection of sentence-image pairs
$D=\\{x_{i},y_{i}\\}_{i=1}^{m}$, firstly we map sentence $x_{i}$ and image
$y_{i}$ into a shared space:

${\bm{s}}_{i}^{z}=g_{\phi_{1}}(f_{\theta}(x_{i},z)),\,{\bm{v}}_{i}=g_{\phi_{2}}(f^{v}(y_{i}))\;,$
(2)

where $f^{v}(\cdot)$ is a pre-trained image encoder such as ResNet (He et al.,
2016), which is fixed during training. $g_{\phi_{1}}(\cdot)$ and
$g_{\phi_{2}}(\cdot)$ are distinct projection heads for text and image
modality respectively. To pull semantically close image-sentence pairs
together and push away non-related pairs, we define the multimodal contrastive
learning objective as:

$\ell_{i}^{\mathrm{M}}=-\sum_{z\in\\{z_{i},z_{i}^{\prime}\\}}\log\frac{e^{\mathrm{sim}({\bm{s}}_{i}^{z},{\bm{v}}_{i})/\tau^{\prime}}}{\sum_{j=1}^{N}e^{\mathrm{sim}({\bm{s}}_{i}^{z},{\bm{v}}_{j})/\tau^{\prime}}}\;,$
(3)

where $\tau^{\prime}$ is a temperature parameter. Let $\lambda$ denote the
trade-off hyperparameter between two objectives, we formulate the final loss
as:

$\ell_{i}=\ell_{i}^{S}+\lambda\ell_{i}^{M}\;.$ (4)

Our method further regularizes the sentence representation in a way that
aligns with the image representation in the grounded space.

##  4 Experiments

###  4.1 Setup

Dataset We use Flickr30k (Young et al., 2014) and MS-COCO (Lin et al., 2014)
as our multimodal datasets. Flickr30k contains $29,783$ training images and
MS-COCO contains $82,783$ training images. Each image is annotated with
multiple captions and we randomly sample only one caption to create image-
sentence pairs. Following Gao et al. (2021), we use Wiki1M as the text-only
corpus, which consists of $10^{6}$ sentences randomly drawn from English
Wikipedia.

Implementation Details We use BERTbase (Devlin et al., 2019) and RoBERTabase
(Liu et al., 2019) as language encoders and ResNet-50 (He et al., 2016) as the
image encoder. Distinct single-layer MLPs are applied as projection heads.
More details are provided in Appendix A.

Evaluation We evaluate the trained models on seven Semantic Textual Similarity
(STS) tasks: STS 2012-2016 (Agirre et al., 2012, 2013, 2014, 2015, 2016), STS
Benchmark (Cer et al., 2017) and SICK-Relatedness (Marelli et al., 2014). Each
of these datasets consists of a collection of sentence pairs and the goal is
to predict a similarity score for each sentence pair. Following Gao et al.
(2021), we report the Spearman’s correlation ($\times 100$) between gold
annotations and predicted scores in the “all” setting, i.e., for each task, we
concatenate all the subsets and report the overall Spearman’s correlation.

###  4.2 Main Results

#### Augmenting text-only corpus with small scale multimodal data yields
significant improvements.

To fully utilize different types of data resources, we conduct experiments
with a text-only corpus and multimodal data. SimCSE is trained on sentences
and captions only, while MCSE additionally computes the multimodal objective
for image-caption pairs. As shown in Table 1, averaging the off-the-shelf BERT
and RoBERTa embeddings444Following (Gao et al., 2021), we take the average of
the first and last layers, which is better than only using the last. yields
poor performance on STS tasks. SimCSE models significantly outperform the
average embeddings. MCSE models, which have access to auxiliary visual
information, further achieve noticeable improvements even if the amount of
multimodal data is relatively small. When MCSE is applied to the combination
of Wiki1M and Flickr30k, it improves the state-of-the-art result for BERT
($76.3\rightarrow 77.3$) and RoBERTa ($76.6\rightarrow 78.3$) by a decent
margin. Looking at performance on the individual tasks, we find that MCSE
models using BERT encoder perform worse on STS16. This can be attributed to
the domain discrepancy, where some subsets that are close to the training
distribution benefit more from visually grounding than others (see Appendix
B.1).

To further investigate the impact of different datasets, we train models
solely on multimodal data and report results in Table 2. We observe that,
without the large text-only corpus, the performances decrease considerably
compared to results in Table 1. Still, MCSE models consistently surpass SimCSE
models ($0.9$ – $3.8$ points improvement). Moreover, replacing the paired
images with shuffled images before training MCSE leads to $0.8$ – $5.0$ points
reduction in terms of average Spearman’s correlation, further validating the
efficacy of visual semantics. We also replace the ResNet encoder with CLIP
(Radford et al., 2021) and our results show that different image encoders lead
to similar results. Details are shown in Appendix B.2.

Model | Trained on
---|---
flickr | coco
SimCSE-BERT | 68.8±0.7 | 67.8±0.4
MCSE-BERT | 70.6${}^{*}_{\pm 0.5}$ | 71.6${}^{*}_{\pm 0.2}$
w/ shuffling | 67.9±0.6$\downarrow$ | 66.6±0.3$\downarrow$
SimCSE-RoBERTa | 72.9±0.3 | 72.8±0.3
MCSE-RoBERTa | 73.8${}^{*}_{\pm 0.2}$ | 74.3${}^{*}_{\pm 0.3}$
w/ shuffling | 73.0±0.4$\downarrow$ | 72.8±0.3$\downarrow$

$\ast$: difference between SimCSE and MCSE is significant.

Table 2:  Comparison of the average Spearman’s correlation on $7$ STS tasks
(Avg. column in Table 1). We report the means and standard deviations over $5$
seeds. Figure 2: The alignment-uniformity plot of models when using BERT
encoder. Colors of dots represent the average Spearman’s correlation. Figure
3: Performances of different data scales. The full set indicates 30K and 87K
samples for Flickr30k and MS-COCO respectively.

#### Grounding to the visual world improves alignment and maintains
uniformity.

To dissect the inner workings of MCSE, we use two quantifiable metrics
proposed in Wang and Isola (2020): alignment and uniformity, as measurements
of representation quality. Let $p_{\text{pos}}$ denote the positive pairs
distribution and $p_{\text{data}}$ denote the data distribution. The alignment
loss prefers encoders that assign similar features to semantically similar
instances (assuming features have been normalized):

$\mathcal{L}_{align}\triangleq\mathop{\mathbb{E}}_{(x,x^{+})~{}\sim~{}p_{\text{pos}}}\left\|f(x)-f(x^{+})\right\|^{2}_{2}\;.$
(5)

And the uniformity loss prefers a uniform distribution in the hypersphere:

$\mathcal{L}_{uniform}\triangleq\log\mathop{\mathbb{E}}_{x,y\stackrel{{\scriptstyle
i.i.d.}}{{\sim}}p_{\text{data}}}e^{-2\left\|f(x)-f(y)\right\|^{2}_{2}}\;.$ (6)

Gao et al. (2021) empirically showed that sentence embedding models with both
lower alignment and uniformity achieve better performance in general.
Similarly, we calculate the two losses on STS-B555We take STS-B pairs with a
score higher than $4.0$ as $p_{\text{pos}}$ and the full STS-B as
$p_{\text{data}}$. Since Gao et al. (2021) did not release the code for
calculating these two losses, the absolute values we obtained might be
different from theirs. We make sure our calculation across different models is
consistent. and results are presented in Figure 2. It shows that MCSE models
achieve better alignment scores compared to SimCSE while maintaining
uniformity. This analysis provides further support that visually grounding can
enhance sentence representation learning by improving the alignment property
of the textual embedding space.

###  4.3 Analysis

For brevity, we take BERT-based models trained merely on caption datasets and
investigate the impact of training data scales. More analysis results
(sentence retrieval, cross-modal retrieval) are provided in Appendix B.3. We
limit the number of training samples to $100$, $500$, $1000$, $5000$ and
$10000$, and compare their performance with the full set performance. In all
of these settings, we optimize the models for same number of training steps as
the full set setting. The results are shown in Figure 3. SimCSE achieves
better performance than MCSE with limited samples, while MCSE starts to
outperform SimCSE with the increasing data scale. We conjecture that this
phenomenon can be ascribed to the progressive training of weights in
multimodal projection heads.

##  5 Limitations

Despite showing performance improvements on STS benchmarks, MCSE has its
limitations as well. We take caption datasets as the source of multimodal
information, while these datasets are collected and curated with non-
negligible human efforts. It will have great practical value if we can
properly leverage noisy image-sentence pairs or even get rid of the explicit
alignments between images and sentences. Furthermore, we find that only
subsets from related domains can get significant improvements while others
suffer from distribution shifts. It is critical to mitigate domain gaps for
learning general-purpose sentence embeddings. In addition, the definition of
“semantic similarity” is highly task-dependent. Besides STS benchmarks, it is
worth exploring the performance gap between text-only models and multimodal
models on other benchmarks that can also assess the quality of sentence
representations.

##  6 Conclusion

In this paper, we propose MCSE, a novel approach for sentence embedding
learning that applies a multimodal contrastive objective to align sentences
and corresponding images in a grounded space. Experiments show that MCSE
consistently improves the performance on STS tasks. We also highlight the
superiority of our method by analyzing the alignment and uniformity properties
of the embedding space. The multimodal objective is generic and can be
potentially incorporated into other sentence embedding methods to boost their
performance.

## Acknowledgements

We thank Dingfan Chen, Fangzhou Zhai and Xiaoyu Shen for their helpful
comments on the paper draft. We would also like to thank the reviewers for
their valuable feedback. This work was funded by the Deutsche
Forschungsgemeinschaft (DFG, German Research Foundation) – project-id
232722074 – SFB 1102. This work was also partially funded by the EU Horizon
2020 project ROXANNE under grant number 833635 and COMPRISE grant number
3081705.

## References

  * Agirre et al. (2015) Eneko Agirre, Carmen Banea, Claire Cardie, Daniel Cer, Mona Diab, Aitor Gonzalez-Agirre, Weiwei Guo, Inigo Lopez-Gazpio, Montse Maritxalar, Rada Mihalcea, et al. 2015.  Semeval-2015 task 2: Semantic textual similarity, english, spanish and pilot on interpretability.  In _Proceedings of the 9th international workshop on semantic evaluation (SemEval 2015)_ , pages 252–263.
  * Agirre et al. (2014) Eneko Agirre, Carmen Banea, Claire Cardie, Daniel Cer, Mona Diab, Aitor Gonzalez-Agirre, Weiwei Guo, Rada Mihalcea, German Rigau, and Janyce Wiebe. 2014\.  Semeval-2014 task 10: Multilingual semantic textual similarity.  In _Proceedings of the 8th international workshop on semantic evaluation (SemEval 2014)_ , pages 81–91.
  * Agirre et al. (2016) Eneko Agirre, Carmen Banea, Daniel Cer, Mona Diab, Aitor Gonzalez Agirre, Rada Mihalcea, German Rigau Claramunt, and Janyce Wiebe. 2016.  Semeval-2016 task 1: Semantic textual similarity, monolingual and cross-lingual evaluation.  In _Proceedings of the 10th International Workshop on Semantic Evaluation (SemEval 2016)_ , pages 497–511.
  * Agirre et al. (2012) Eneko Agirre, Daniel Cer, Mona Diab, and Aitor Gonzalez-Agirre. 2012.  Semeval-2012 task 6: A pilot on semantic textual similarity.  In _* SEM 2012: The First Joint Conference on Lexical and Computational Semantics–Volume 1: Proceedings of the main conference and the shared task, and Volume 2: Proceedings of the Sixth International Workshop on Semantic Evaluation (SemEval 2012)_ , pages 385–393.
  * Agirre et al. (2013) Eneko Agirre, Daniel Cer, Mona Diab, Aitor Gonzalez-Agirre, and Weiwei Guo. 2013\.  * SEM 2013 shared task: Semantic textual similarity.  In _Second joint conference on lexical and computational semantics (* SEM), volume 1: proceedings of the Main conference and the shared task: semantic textual similarity_ , pages 32–43.
  * Bender and Koller (2020) Emily M. Bender and Alexander Koller. 2020.  Climbing towards NLU: On meaning, form, and understanding in the age of data.  In _Proceedings of the 58th Annual Meeting of the Association for Computational Linguistics (ACL)_ , pages 5185–5198.
  * Bisk et al. (2020) Yonatan Bisk, Ari Holtzman, Jesse Thomason, Jacob Andreas, Yoshua Bengio, Joyce Chai, Mirella Lapata, Angeliki Lazaridou, Jonathan May, Aleksandr Nisnevich, Nicolas Pinto, and Joseph Turian. 2020.  Experience grounds language.  In _Proceedings of the 2020 Conference on Empirical Methods in Natural Language Processing (EMNLP)_ , pages 8718–8735.
  * Bordes et al. (2019) Patrick Bordes, Eloi Zablocki, Laure Soulier, Benjamin Piwowarski, and Patrick Gallinari. 2019.  Incorporating visual semantics into sentence representations within a grounded space.  In _Proceedings of the 2019 Conference on Empirical Methods in Natural Language Processing and the 9th International Joint Conference on Natural Language Processing (EMNLP-IJCNLP)_ , pages 696–707.
  * Carlsson et al. (2020) Fredrik Carlsson, Amaru Cuba Gyllensten, Evangelia Gogoulou, Erik Ylipää Hellqvist, and Magnus Sahlgren. 2020.  Semantic re-tuning with contrastive tension.  In _International Conference on Learning Representations (ICLR)_.
  * Cer et al. (2017) Daniel Cer, Mona Diab, Eneko Agirre, Inigo Lopez-Gazpio, and Lucia Specia. 2017\.  Semeval-2017 task 1: Semantic textual similarity-multilingual and cross-lingual focused evaluation.  In _Proceedings of the 11th International Workshop on Semantic Evaluation (SemEval 2017)_ , pages 1–14.
  * Cer et al. (2018) Daniel Cer, Yinfei Yang, Sheng-yi Kong, Nan Hua, Nicole Limtiaco, Rhomni St. John, Noah Constant, Mario Guajardo-Cespedes, Steve Yuan, Chris Tar, Brian Strope, and Ray Kurzweil. 2018.  Universal sentence encoder for English.  In _Proceedings of the 2018 Conference on Empirical Methods in Natural Language Processing (EMNLP)_ , pages 169–174.
  * Chen et al. (2020) Ting Chen, Simon Kornblith, Mohammad Norouzi, and Geoffrey Hinton. 2020.  A simple framework for contrastive learning of visual representations.  In _Proceedings of the 37th International Conference on Machine Learning (ICML)_ , pages 1597–1607.
  * Conneau et al. (2017) Alexis Conneau, Douwe Kiela, Holger Schwenk, Loïc Barrault, and Antoine Bordes. 2017.  Supervised learning of universal sentence representations from natural language inference data.  In _Proceedings of the 2017 Conference on Empirical Methods in Natural Language Processing (EMNLP)_ , pages 670–680.
  * Devlin et al. (2019) Jacob Devlin, Ming-Wei Chang, Kenton Lee, and Kristina Toutanova. 2019.  BERT: Pre-training of deep bidirectional transformers for language understanding.  In _North American Chapter of the Association for Computational Linguistics: Human Language Technologies (NAACL-HIT)_ , pages 4171–4186.
  * Gao et al. (2021) Tianyu Gao, Xingcheng Yao, and Danqi Chen. 2021.  SimCSE: Simple contrastive learning of sentence embeddings.  In _Proceedings of the 2021 Conference on Empirical Methods in Natural Language Processing (EMNLP)_ , pages 6894–6910.
  * He et al. (2016) Kaiming He, Xiangyu Zhang, Shaoqing Ren, and Jian Sun. 2016.  Deep residual learning for image recognition.  In _Proceedings of the IEEE conference on computer vision and pattern recognition (CVPR)_ , pages 770–778.
  * Kiela et al. (2018) Douwe Kiela, Alexis Conneau, Allan Jabri, and Maximilian Nickel. 2018.  Learning visually grounded sentence representations.  In _Proceedings of the 2018 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies (NAACL-HIT)_ , pages 408–418.
  * Kim et al. (2021) Taeuk Kim, Kang Min Yoo, and Sang-goo Lee. 2021.  Self-guided contrastive learning for BERT sentence representations.  In _Proceedings of the 59th Annual Meeting of the Association for Computational Linguistics and the 11th International Joint Conference on Natural Language Processing (ACL-IJCNLP)_ , pages 2528–2540.
  * Kiros et al. (2015) Ryan Kiros, Yukun Zhu, Russ R Salakhutdinov, Richard Zemel, Raquel Urtasun, Antonio Torralba, and Sanja Fidler. 2015.  Skip-thought vectors.  In _Advances in neural information processing systems (NeurIPS)_ , pages 3294–3302.
  * Lazaridou et al. (2015) Angeliki Lazaridou, Marco Baroni, et al. 2015.  Combining language and vision with a multimodal skip-gram model.  In _Proceedings of the 2015 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies (NAACL-HLT)_ , pages 153–163.
  * Li et al. (2020) Bohan Li, Hao Zhou, Junxian He, Mingxuan Wang, Yiming Yang, and Lei Li. 2020.  On the sentence embeddings from pre-trained language models.  In _Proceedings of the 2020 Conference on Empirical Methods in Natural Language Processing (EMNLP)_ , pages 9119–9130.
  * Lin et al. (2014) Tsung-Yi Lin, Michael Maire, Serge Belongie, James Hays, Pietro Perona, Deva Ramanan, Piotr Dollár, and C. Lawrence Zitnick. 2014.  Microsoft coco: Common objects in context.  In _Proceedings of the European Conference on Computer Vision (ECCV)_ , pages 740–755.
  * Liu et al. (2021) Fangyu Liu, Ivan Vulić, Anna Korhonen, and Nigel Collier. 2021.  Fast, effective, and self-supervised: Transforming masked language models into universal lexical and sentence encoders.  In _Proceedings of the 2021 Conference on Empirical Methods in Natural Language Processing (EMNLP)_ , pages 1442–1459.
  * Liu et al. (2019) Yinhan Liu, Myle Ott, Naman Goyal, Jingfei Du, Mandar Joshi, Danqi Chen, Omer Levy, Mike Lewis, Luke Zettlemoyer, and Veselin Stoyanov. 2019.  Roberta: A robustly optimized bert pretraining approach.  _arXiv preprint arXiv:1907.11692_.
  * Marelli et al. (2014) Marco Marelli, Stefano Menini, Marco Baroni, Luisa Bentivogli, Raffaella Bernardi, Roberto Zamparelli, et al. 2014.  A sick cure for the evaluation of compositional distributional semantic models. In _International Conference on Language Resources and Evaluation (LREC)_ , pages 216–223.
  * Pennington et al. (2014) Jeffrey Pennington, Richard Socher, and Christopher D Manning. 2014.  Glove: Global vectors for word representation.  In _Proceedings of the 2014 conference on empirical methods in natural language processing (EMNLP)_ , pages 1532–1543.
  * Radford et al. (2021) Alec Radford, Jong Wook Kim, Chris Hallacy, Aditya Ramesh, Gabriel Goh, Sandhini Agarwal, Girish Sastry, Amanda Askell, Pamela Mishkin, Jack Clark, Gretchen Krueger, and Ilya Sutskever. 2021.  Learning transferable visual models from natural language supervision.  In _Proceedings of the 38th International Conference on Machine Learning (ICML)_ , pages 8748–8763.
  * Reimers and Gurevych (2019) Nils Reimers and Iryna Gurevych. 2019.  Sentence-BERT: Sentence embeddings using Siamese BERT-networks.  In _Proceedings of the 2019 Conference on Empirical Methods in Natural Language Processing and the 9th International Joint Conference on Natural Language Processing (EMNLP-IJCNLP)_ , pages 3982–3992.
  * Su et al. (2021) Jianlin Su, Jiarun Cao, Weijie Liu, and Yangyiwen Ou. 2021.  Whitening sentence representations for better semantics and faster retrieval.  _arXiv preprint arXiv:2103.15316_.
  * Tan and Bansal (2020) Hao Tan and Mohit Bansal. 2020.  Vokenization: Improving language understanding via contextualized, visually-grounded supervision.  In _Proceedings of the 2020 Conference on Empirical Methods in Natural Language Processing (EMNLP)_ , pages 2066–2080.
  * Tang et al. (2021) Zineng Tang, Jaemin Cho, Hao Tan, and Mohit Bansal. 2021.  Vidlankd: Improving language understanding via video-distilled knowledge transfer.  In _Advances in Neural Information Processing Systems (NeurIPS)_ , pages 24468–24481.
  * Wang and Isola (2020) Tongzhou Wang and Phillip Isola. 2020.  Understanding contrastive representation learning through alignment and uniformity on the hypersphere.  In _Proceedings of the 37th International Conference on Machine Learning (ICML)_ , pages 9929–9939.
  * Wieting et al. (2020) John Wieting, Graham Neubig, and Taylor Berg-Kirkpatrick. 2020.  A bilingual generative transformer for semantic sentence embedding.  In _Proceedings of the 2020 Conference on Empirical Methods in Natural Language Processing (EMNLP)_ , pages 1581–1594.
  * Wolf et al. (2020) Thomas Wolf, Lysandre Debut, Victor Sanh, Julien Chaumond, Clement Delangue, Anthony Moi, Pierric Cistac, Tim Rault, Rémi Louf, Morgan Funtowicz, Joe Davison, Sam Shleifer, Patrick von Platen, Clara Ma, Yacine Jernite, Julien Plu, Canwen Xu, Teven Le Scao, Sylvain Gugger, Mariama Drame, Quentin Lhoest, and Alexander M. Rush. 2020.  Transformers: State-of-the-art natural language processing.  In _Proceedings of the 2020 Conference on Empirical Methods in Natural Language Processing (EMNLP)_ , pages 38–45.
  * Yan et al. (2021) Yuanmeng Yan, Rumei Li, Sirui Wang, Fuzheng Zhang, Wei Wu, and Weiran Xu. 2021.  ConSERT: A contrastive framework for self-supervised sentence representation transfer.  In _Proceedings of the 59th Annual Meeting of the Association for Computational Linguistics and the 11th International Joint Conference on Natural Language Processing (EMNLP-IJCNLP)_ , pages 5065–5075.
  * Young et al. (2014) Peter Young, Alice Lai, Micah Hodosh, and Julia Hockenmaier. 2014.  From image descriptions to visual denotations: New similarity metrics for semantic inference over event descriptions.  _Transactions of the Association for Computational Linguistics (TACL)_ , pages 67–78.
  * Zablocki et al. (2018) Eloi Zablocki, Benjamin Piwowarski, Laure Soulier, and Patrick Gallinari. 2018.  Learning multi-modal word representation grounded in visual context.  In _Proceedings of the AAAI Conference on Artificial Intelligence_.
  * Zhang et al. (2020) Yan Zhang, Ruidan He, Zuozhu Liu, Kwan Hui Lim, and Lidong Bing. 2020.  An unsupervised sentence embedding method by mutual information maximization.  In _Proceedings of the 2020 Conference on Empirical Methods in Natural Language Processing (EMNLP)_ , pages 1601–1610.
  * Zhang et al. (2019) Zhuosheng Zhang, Kehai Chen, Rui Wang, Masao Utiyama, Eiichiro Sumita, Zuchao Li, and Hai Zhao. 2019.  Neural machine translation with universal visual representation.  In _International Conference on Learning Representations (ICLR)_.
  * Zhao and Titov (2020) Yanpeng Zhao and Ivan Titov. 2020.  Visually grounded compound pcfgs.  In _Proceedings of the 2020 Conference on Empirical Methods in Natural Language Processing (EMNLP)_ , pages 4369–4379.

##  Appendix A Implementation Details

Language Encoder Our implementation is based on the Hugging Face Transformers
library666https://github.com/huggingface/transformers (Wolf et al., 2020). We
start from the checkpoints of bert-base-uncased and roberta-base, and fine-
tune the pre-trained models using a contrastive objective function. We use the
768-dimensional [CLS] token outputs before the MLP pooler layer as sentence
embeddings for evaluation.

Image Encoder We use ResNet-50 and extract 2048-dimensional feature vectors at
the last layer. The image encoder is not fine-tuned.777In our preliminary
results, fine-tuning the image encoder does not have a significant impact on
the STS performance.

Projection Heads We use distinct projection heads for different modalities and
objectives. All of them are implemented by single-layer MLPs with Tanh
activation. We map sentence embeddings to a 768-dimensional space before
calculating the textual objective. We map both sentence embeddings and image
feature vectors to a 256-dimensional shared space, and normalize them before
calculating the multimodal objective.

Parameter Settings We explore $5$ training settings in the paper: {wiki,
wiki+flickr, wiki+coco, flickr, coco}. For wiki+flickr and wiki+coco, we
sample mini-batches from either Wiki1M or the caption dataset in proportion to
their data size. We adopt most of the parameter settings suggested by Gao et
al. (2021). Moreover, temperature parameters $\tau$ and $\tau^{\prime}$ are
set to $0.05$, and other hyperparameters are reported in Table 3. We use the
dev set of STS-B to tune the trade-off parameter $\lambda$ and ablation
studies are shown in Table 4. We evaluate models every $125$ training steps on
STS-B dev set and keep the best checkpoint for final evaluation.

settings: | wiki | wiki+flickr | wiki+coco | flickr | coco
---|---|---|---|---|---
BERT
learning rate | 3e-5
batch size | 64
$\lambda$ | – | 0.01 | 0.01 | 0.05 | 0.05
epochs | 3 | 3 | 3 | 6 | 3
RoBERTa
learning rate | 1e-5
batch size | 128
$\lambda$ | – | 0.01 | 0.01 | 0.01 | 0.01
epochs | 3 | 3 | 3 | 6 | 3

Table 3: The hyperparameters used for different training settings and pre-
trained encoders.

$\lambda$ | 0.001 | 0.01 | 0.05 | 0.1 | 0.5
---|---|---|---|---|---
MCSE-BERT | 78.38 | 79.95 | 80.41 | 80.35 | 80.01
MCSE-RoBERTa | 80.60 | 81.48 | 81.08 | 80.73 | 79.85

Table 4: STS-B performance of MCSE models trained on Flickr30k with different
trade-off parameters.

##  Appendix B More Results

###  B.1 Improvements on Different Subsets

To delve into the performance gap between MCSE-BERT and SimCSE-BERT, we
calculate the Spearman’s correlation for different subsets of each year’s STS
challenge separately. The improvements of MCSE over SimCSE are shown in Figure
4. In STS12, "MSRvid" subset achieves the largest improvement, which is a
corpus of video descriptions. "Image" subsets in STS14 and STS15 also get
considerable improvements. Meanwhile, the performance of "answers-students"
subset in STS15 drops extensively, and none of the subsets in STS16 get
noticeable improvement by MCSE. The results indicate that the subsets benefit
to different degrees from the visually grounding because of domain
discrepancy.

###  B.2 Ablation Study

#### CLIP as Image Encoder

We use CLIP (Radford et al., 2021) as an alternative image encoder. The
implementation is based on the Sentence Transformer
library888https://github.com/UKPLab/sentence-transformers (Reimers and
Gurevych, 2019) and we use the checkpoint clip-ViT-B-32 to extract
512-dimensional feature vectors. As shown in Table 7, different image encoders
lead to very similar results, thus we use ResNet as the default image encoder.

#### Combining Wiki1M, Flickr30k and MS-COCO

We adopt the same parameter setting as wiki+flickr and wiki+coco, and train
models on the combination of Wiki1M, Flickr30k, and MS-COCO. As shown in Table
5, MCSE models achieve $1.9$ point and $2.6$ point improvements when using
BERT and RoBERTa, respectively.

Model | Trained on
---|---
wiki+flickr+coco
SimCSE-BERT | 74.3±1.0
MCSE-BERT | 76.2±0.3
SimCSE-RoBERTa | 75.3±0.7
MCSE-RoBERTa | 77.9±0.6

Table 5:  Comparison of the average Spearman’s correlation of $7$ STS tasks.
We report the means and standard deviations over $5$ random seeds.

Model |  image  $\rightarrow$ text |  text  $\rightarrow$ image
---|---|---
R@1 | R@5 | R@1 | R@5
MCSE-BERTwiki+flickr | 16.7 | 43.5 | 22.5 | 50.4
MCSE-BERTflickr | 20.4 | 50.2 | 23.8 | 52.5
MCSE-BERTwiki+coco | 8.8 | 26.6 | 10.9 | 31.2
MCSE-BERTcoco | 8.2 | 25.2 | 9.0 | 27.1

Table 6: Multimodal retrieval results on Flickr30k test set (1k) and MS-COCO
minival set (5k).

(a) training data: wiki+flickr

(b) training data: wiki+coco

Figure 4: The Spearman’s correlation improvements over different subsets.

###  B.3 Analysis

#### Sentence Retrieval

We take BERT-based models trained on the Flickr30k train set (same seed) and
conduct a sentence retrieval experiment on Flickr30k test set. Given an input
sentence, the nearest neighbor will be retrieved based on cosine similarity.
Some retrieval examples are shown in Table 8. We observe that (1) SimCSE is
prone to retrieving sentences with similar syntax, while MCSE can retrieve
sentences that vary in syntax and share semantics. Examples: Q1, Q3, Q6. (2)
MCSE is better at recognizing similar event scenes and capturing the number of
entities. Examples: Q2, Q4, Q5.

#### Cross-Modal Retrieval

We take BERT-based models (same seed) and conduct cross-modal retrieval
experiments. We use the metric Recall@K, which is calculated based on if the
ground truth of the query image or caption appears in the top-K retrieved
captions or images. As results in Table 6 show, MCSE models also achieve a
decent level of retrieval performance as a by-product of multimodal
contrastive learning.

| Model | STS12 | STS13 | STS14 | STS15 | STS16 | STS-B | SICK-R | Avg.$\uparrow$
---|---|---|---|---|---|---|---|---|---
flickr | SimCSE-BERT | 62.1±0.5 | 73.8±0.9 | 64.2±0.6 | 74.2±0.8 | 74.8${}^{*}_{\pm 0.6}$ | 67.1±1.1 | 65.4±1.1 | 68.8±0.7
MCSE-ResNet-BERT | 63.6${}^{*}_{\pm 0.7}$ | 74.0±0.9 | 65.5±1.1 | 75.5±0.2 | 71.6±0.4 | 74.0±0.4 | 69.8±0.3 | 70.6±0.5
MCSE-CLIP-BERT | 63.1±0.7 | 73.9±1.0 | 65.8${}^{*}_{\pm 0.9}$ | 76.0${}^{*}_{\pm 0.7}$ | 70.7±0.3 | 74.9${}^{*}_{\pm 0.5}$ | 70.7${}^{*}_{\pm 0.3}$ | 70.7${}^{*}_{\pm 0.2}$
SimCSE-RoBERTa | 66.6±0.5 | 78.3±0.5 | 69.7±0.6 | 77.7±0.5 | 76.3${}^{*}_{\pm 0.5}$ | 75.8±0.3 | 66.2±0.4 | 72.9±0.3
MCSE-ResNet-RoBERTa | 67.6${}^{*}_{\pm 0.5}$ | 78.8±0.4 | 70.1±0.3 | 78.5±0.2 | 75.4±0.5 | 77.4${}^{*}_{\pm 0.3}$ | 68.6±0.3 | 73.8${}^{*}_{\pm 0.2}$
MCSE-CLIP-RoBERTa | 67.0±0.5 | 78.6±0.4 | 69.8±0.5 | 78.7${}^{*}_{\pm 0.8}$ | 74.9±0.5 | 77.4${}^{*}_{\pm 0.4}$ | 69.5${}^{*}_{\pm 0.5}$ | 73.7±0.2
coco | SimCSE-BERT | 59.3±0.9 | 73.0±1.2 | 62.7±0.6 | 74.7±0.7 | 74.4${}^{*}_{\pm 0.4}$ | 65.3±0.7 | 65.4±0.5 | 67.8±0.4
MCSE-ResNet-BERT | 64.9${}^{*}_{\pm 0.5}$ | 74.8${}^{*}_{\pm 0.9}$ | 68.1${}^{*}_{\pm 0.6}$ | 76.8${}^{*}_{\pm 0.6}$ | 72.7±0.8 | 74.5${}^{*}_{\pm 0.4}$ | 69.7±0.4 | 71.6${}^{*}_{\pm 0.2}$
MCSE-CLIP-BERT | 64.8±0.6 | 74.1±0.6 | 68.0±0.2 | 76.2±0.5 | 71.6±0.4 | 74.5${}^{*}_{\pm 0.3}$ | 70.3${}^{*}_{\pm 0.6}$ | 71.4±0.1
SimCSE-RoBERTa | 64.7±0.6 | 79.2±0.4 | 70.2±0.4 | 79.0±0.6 | 78.2±0.5 | 73.8±0.5 | 64.6±0.3 | 72.8±0.3
MCSE-ResNet-RoBERTa | 67.0${}^{*}_{\pm 0.8}$ | 79.4±0.4 | 70.9${}^{*}_{\pm 0.4}$ | 80.0${}^{*}_{\pm 0.4}$ | 77.8±0.5 | 76.9${}^{*}_{\pm 0.4}$ | 67.9±0.7 | 74.3${}^{*}_{\pm 0.3}$
MCSE-CLIP-RoBERTa | 66.0±1.0 | 79.0±0.7 | 70.6±0.6 | 80.0${}^{*}_{\pm 0.8}$ | 77.6±0.5 | 76.5±0.4 | 68.4${}^{*}_{\pm 0.8}$ | 74.0±0.2

$\ast$: difference between SimCSE and MCSE (ResNet/CLIP) is significant at
$\alpha=0.05$ according to an independent t-test.

Table 7: Performance comparison on STS tasks. STS-B: STS Benchmark, SICK-R:
SICK-Relatedness, Avg.: average across $7$ tasks. Models are trained with $5$
random seeds and we report means and standard deviations.

Model | Result
---|---
Query 1: A young girl is washing her teddy bear in the kitchen sink.
SimCSE: | A middle-aged woman is vacuuming her kitchen floor with a canister vac.
MCSE: | A young girl, blond and wearing a polka-dot shirt, washes a stuffed animal in a vanity sink.
Query 2: Three chefs , wearing white hats and black aprons , are preparing
food in a crowded kitchen.
SimCSE: | Numerous workers with blue shirts and white aprons are preparing fish for sale.
MCSE: | Three men are preparing food in a kitchen setting.
Query 3: A couple kisses in a shady walkway.
SimCSE: | A couple strolls down a path near benches and water.
MCSE: | Couple kissing outside on street.
Query 4: A man is standing on the streets taking photographs.
SimCSE: | People run a marathon on a city street with a crowd watching.
MCSE: | A guy wearing a white shirt is taking a picture.
Query 5: Two boys are playing in pool filled with sparkling blue water.
SimCSE: | A little girl is swimming under the crystal blue water.
MCSE: | Two children are swimming in a pool.
Query 6: An old man sitting on a bench staring at the ocean.
SimCSE: | A man sitting on a bench by the ocean.
MCSE: | An old man sits on a bench overlooking the water.

Table 8: Retrieved examples from Flickr30k test set.


# HL-LHC layout for fixed-target experiments in ALICE based on crystal-
assisted beam halo splitting

Marcin Patecki<EMAIL_ADDRESS>Warsaw University of Technology,
Faculty of Physics, ul. Koszykowa 75, 00-662 Warsaw, Poland.  Alex Fomin
Daniele Mirarchi  Stefano Redaelli  European Organization for Nuclear Research
(CERN), CH-1211 Geneva 23, Switzerland  Cynthia Hadjidakis  Université Paris-
Saclay, CNRS/IN2P3, IJCLab, 91405 Orsay, France  Francesca Galluccio  INFN
Sezione di Napoli, Complesso Universitario di Monte Sant’Angelo, Via Cintia,
80126 Napoli, Italy  Walter Scandale  Blackett Laboratory, Imperial College,
London SW7 2AZ, UK  INFN Sezione di Roma, Piazzale Aldo Moro 2, 00185 Rome,
Italy

###### Abstract

The Large Hadron Collider (LHC) at the European Organization for Nuclear
Research (CERN) is the world’s largest and most powerful particle accelerator
colliding beams of protons and lead ions at energies up to 7 Z TeV, Z is the
atomic number. ALICE is one of the detector experiments optimised for heavy-
ion collisions. A fixed-target experiment in ALICE is being considered to
collide a portion of the beam halo, split using a bent crystal inserted in the
transverse hierarchy of the LHC collimation system, with an internal target
placed a few meters upstream of the existing detector. This study is carried
out as a part of the Physics Beyond Collider effort at CERN. Fixed-target
collisions offer many physics opportunities related to hadronic matter and the
quark-gluon plasma to extend the research potential of the CERN accelerator
complex. Production of physics events depends on the particle flux on target.
The machine layout for the fixed-target experiment is developed to provide a
flux of particles on the target high enough to exploit the full capabilities
of the ALICE detector acquisition system. This paper summarises the fixed-
target layout consisting of the crystal assembly, the target and downstream
absorbers. We discuss the conceptual integration of these elements within the
LHC ring, the impact on ring losses, and expected performance in terms of
particle flux on target.

††preprint: APS/123-QED

##  I Introduction

Advancements in the knowledge of fundamental constituents of matter and their
interactions are usually driven by the development of experimental techniques
and facilities, with a significant role of particle accelerators. The Large
Hadron Collider (LHC) [1] at the European Organization for Nuclear Research
(CERN) is the world’s largest and most powerful particle accelerator colliding
opposite beams of protons (p) and lead ions (Pb), allowing for unprecedentedly
high centre-of-mass energies of up to 14 TeV and 5.5 TeV per nucleon,
respectively. An ALICE fixed-target (ALICE-FT) programme [2] is being
considered to extend the research potential of the LHC and the ALICE
experiment [3]. The setup of in-beam targets at the LHC is particularly
challenging because of the high-intensity frontier of LHC beams [4].

Fixed-target collisions in the LHC are designed to be operated simultaneously
with regular head-on collisions without jeopardising the LHC efficiency during
its main p-p physics programme. Several unique advantages are offered with the
fixed-target mode compared to the collider mode. With a high density of
targets, high yearly luminosities can be easily achieved, comparable with
luminosities delivered by the LHC (in the collider mode) and Tevatron [5]. In
terms of collision energy, the ALICE-FT layout would provide the most
energetic beam ever in the fixed-target mode with the centre of mass energy
per nucleon-nucleon of 115 GeV for proton beams and 72 GeV for lead ion beams
[5], in between the nominal Relativistic Heavy Ion Collider (RHIC) and Super
Proton Synchrotron (SPS) energies. Thanks to the boost between the colliding-
nucleon centre-of-mass system and the laboratory system, access to far
backward regions of rapidity is possible with the ALICE detector, allowing to
measure any probe even at far ranges of the backward phase space, being
utterly uncharted with head-on collisions [5]. Moreover, the possibility of
using various species of the target material extends the variety of physics
cases, especially allowing for unique neutron studies [5]. The physics
potential [5, 2] of such a fixed-target programme covers an intensive study of
strong interaction processes, quark and gluon distributions at high momentum
fraction (x), sea quark and heavy-quark content in the nucleon and nucleus and
the implication for cosmic ray physics. The hot medium created in ultra-
relativistic heavy-ion collisions offers novel quarkonium and heavy-quark
observables in the energy range between the Super Proton Synchrotron (SPS) and
the Relativistic Heavy Ion Collider (RHIC), where the QCD phase transition is
postulated.

A significant innovation of our proposal is to bring particles of high energy
collider to collisions with a fixed-target by splitting a part of the beam
halo using a bent crystal. Particles entering the crystal with a small impact
angle ($\rm\leq 2.4~{}\mu rad$ for silicon crystals and proton energy of 7 TeV
[6]) undergo the channeling process resulting in a trajectory deflection
equivalent to the geometric bending angle of the crystal body [7, 8]. Such a
setup enables an in-beam target at a safe distance from the circulating beam.
This type of advanced beam manipulation with bent crystals builds on the
experience accumulated in different accelerators (see for example [9, 10]),
and in particular, on the successful results achieved in the multi-TeV regime
for beam collimation at the LHC [11, 12, 13]. The halo-splitting technique
allows profiting from the circulating beam halo particles that are not
contributing to the luminosity production and are typically disposed of by the
collimation system.

The problem that we address is to design the machine layout that provides a
number of protons on the target high enough to exploit the full capabilities
of the ALICE detector acquisition system without affecting the LHC
availability for regular beam-beam collisions. Our proposal of the ALICE-FT
layout [14] follows general guidelines on technical feasibility and impact on
the LHC accelerator of potential fixed-target experiments provided by the LHC
Fixed Target Working Group of the CERN Physics Beyond Colliders forum [15, 4].
We also profit from the preliminary designs reported in [16, 17] and from the
design study of an analogous fixed target experiment at the LHC proposed to
measure electric and magnetic dipole moments of short-lived baryons [18].

In this paper, we summarise the ALICE-FT machine layout. We report on the
conceptual integration of its elements (crystal and target assemblies,
downstream absorbers), their impact on ring losses, and expected performance
in terms of particle flux on target. We also discuss a method of increasing
the flux of particles on the target by setting the crystal at the optimal
betatron phase by applying a local optics modification in the insertion
hosting the ALICE experiment (IR2). This method is independent of the crystal
location, allowing for a crystal installation in a place with good space
availability. Moreover, it allows to recover the maximum performance of the
system in case of changes in beam optics in the LHC.

##  II Machine configuration

A potential installation of the ALICE-FT setup will coincide with a major LHC
upgrade in terms of instantaneous luminosity, commonly referred to as the
High-Luminosity LHC (HL-LHC) [19], taking place in the Long Shutdown 3
(2026-2028), to make it ready for the LHC Run 4 starting in 2029. Some of the
expected beam parameters, having a direct impact on the ALICE-FT experiment
performance, are given in Table 1. One key beam parameter to be upgraded is
the total beam current that will increase nearly by a factor of two, up to
about 1.1 A, leading to about 0.7 GJ of total beam energy stored in the
machine.

Table 1: Some proton-beam parameters of the future HL-LHC beams important for the ALICE-FT experiment, referred to as standard in [19]. Colliding-beam energy | $\rm E$ | 7 TeV
---|---|---
Bunch population | $\rm N_{b}$ | $\rm 2.2\cdot 10^{11}$
Maximum number of bunches | $\rm n_{b}$ | 2760
Beam current | I | 1.09 A
Transverse normalised emittance | $\rm\varepsilon_{n}$ | 2.5 $\rm[\mu m]$
$\rm\beta^{*}$ at IP2 |  | 10 m
Beam crossing angle at IP2 |  | 200 $\rm\mu$rad

A highly efficient collimation system is therefore required in the LHC [20] to
protect its elements, especially the superconducting magnets, from impacts of
particles from the beam. The collimation system is organised in a precise
multi-stage transverse hierarchy (see Table 2) over two dedicated insertions
(IRs): IR3 for momentum cleaning and IR7 for betatron cleaning. Each
collimation insertion features a three-stage cleaning based on primary
collimators (TCPs), secondary collimators (TCSGs) and absorbers (TCLAs). In
addition, dedicated collimators are present in specific locations of the ring
to provide protection of sensitive equipment (e.g. TCTP for the inner
triplets), absorption of physics debris (TCL) and beam injection/dump
protection (TDI/TCDQ-TCSP). The collimation system undergoes an upgrade, as
described in [21], to make it compatible with HL-LHC requirements, but the
general working principle will remain the same. The system is designed to
sustain beam losses up to 1 MJ without damage and with no quench of
superconducting magnets.

Table 2: HL-LHC collimation settings expressed in units of RMS beam size ($\rm\sigma$), assuming a Gaussian beam distribution and transverse normalised emittance $\rm\varepsilon_{n}=2.5~{}\mu m$. Coll. family | IR | Settings ($\rm\sigma$)
---|---|---
TCP/TSCG/TCLA | 7 | 6.7/9.1/12.7
TCP/TSCG/TCLA | 3 | 17.7/21.3/23.7
TCTP | 1/2/5/8 | 10.4/43.8/10.4/17.7
TCL | 1/5 | 14.2
TCSP/TCDQ | 6 | 10.1/10.1

The halo-splitting scheme relies on placing a crystal into the transverse
hierarchy of the betatron collimation system, in between the primary and
secondary stage of IR7 collimators, such that the collimation system
efficiency is not affected. Placing the splitting crystal closer to the beam
than the primary collimators would not be possible without designing a
downstream system capable of withstanding the collimation design loss
scenarios. Retracting the crystal at larger amplitudes avoids this problem
while still allowing intercepting a significant fraction of the multi-stage
halo, as shown below. In this scheme, a fraction of secondary halo particles
redirected toward the target can be used for fixed-target collisions instead
of disposing them at the absorbers, in a safe manner. Note that, for this
scheme to work, the crystal does not need to be installed in IR7. In fact, the
halo splitting is done in the vicinity of the experiment, where the target
needs to be located.

##  III ALICE-FT layout

A general concept of the ALICE-FT layout is illustrated in Fig. 1. A fraction
of secondary halo particles are intercepted by the crystal and steered toward
the target. Collision products are registered by the ALICE detector, which can
handle in the order of $\rm 10^{7}$ protons on target per second [5]. Possible
losses originating either from the crystal or from the target are intercepted
by absorbers which would be installed downstream of the detector.

Figure 1: Working principle of the crystal-based fixed-target experiment
(right side of the graphics) being embedded into the multi-stage collimation
system (left side of the graphics). Graphics based on [18], mostly by D.
Mirarchi.

The ALICE-FT will act on B1 due to ALICE detector geometry. We assume the
layout to be operated with the optics envisioned for the HL-LHC (version 1.5
[22] at the moment), although it is not excluded that more optimized optics
could be envisaged if needed. Since ALICE operates at low luminosities in
proton runs, typically with offset levelling, this insertion does not demand
tight optics constraints like the high-luminosity insertions for the general-
purpose detectors [23]. Actually, a minor, local modification of the optics in
the IR2 region is proposed to enhance the system performance by optimising the
betatron oscillation phase at the location of the crystal. This will not
affect the rest of the machine.

Safe integration of the crystal into the hierarchy of the collimation system
is achieved by setting the crystal in the shadow of IR7 primary collimators.
As discussed in [18], the relative retraction of the crystal with respect to
the IR7 primary collimator should not be smaller than $\rm 0.5~{}\sigma$ (RMS
beam size), mostly to account for optics and orbit errors that, if not under
control, could cause the crystal to become primary collimator accidentally.
The retraction should be kept as low as possible to maximise the number of
protons impacting the target [4] as shown in Fig. 7. Similarly to [18], we
also assume that, for machine safety reasons, the distance from the deflected
beam to the aperture and the distance from the target to main beams should be
at least 4 mm. The system is to be installed in the vertical plane in order to
avoid issues related to the beam dump system operating in the horizontal
plane: this is the plane of the dump kickers and is subject to fast losses in
case of over-populate abort gap or in case of (unlikely) dump failures [24,
25]. In the case of a horizontal setup, larger aperture margins on settings
would be applied to lead to lower rates of impinging halo, which we prefer to
avoid.

The beams crossing scheme at IP2 is also in the vertical plane. Furthermore,
the main solenoid of the ALICE detector can be operated in two polarities
which affects the slope of both beams at IP2. We mark the negative slope of
the LHC beam 1 (B1) at IP2 as negX and the positive slope as posX. Given the
trajectory of the B1 for both crossing schemes (negX and posX), aperture
restrictions and space availability, the optimal longitudinal coordinate for
the crystal installation is 3259m (with 0 at IP1) as it allows for having just
one crystal serving both crossing schemes. The graphical illustration of the
proposed layout, which fulfils all the above requirements, is given in Fig. 2
and conditions inside the LHC tunnel are depicted in Fig. 3.

Figure 2: The proposed layout of the ALICE-FT experiment. Both beams (B1 and
B2) with their envelopes ($\rm 7.3~{}\sigma$) are given with solid lines for
both ALICE solenoid polarities (posX and negX). Deflected beams are given in
dashed blue lines. The machine aperture is given in solid black lines.
Vertical dashed lines mark the locations of crystals, target and IP2,
respectively. The location of absorbers is marked in the right bottom corner.
Figure 3: Conditions inside the LHC tunnel at the proposed location for an
installation of the crystal.

We consider a 16-mm long crystal made of silicon, with 110 bending planes and
a bending radius of 80 m, the same as a bending radius of crystals already
used in the LHC, following the parametric studies reported in [12] to ensure
an optimum crystal channelling performance at the LHC top energy while keeping
the nuclear interaction rate as low as possible. The crystal bending angle of
$\rm 200~{}\mu rad$ is chosen to serve both crossing schemes and to fit the
deflected beam within the available aperture. Any upstream location of the
crystal would require having two crystals with different bending angles,
depending on the crossing scheme.

The target assembly is planned to be installed nearly 5 m upstream from IP2
with a target made of either light or heavy material, e.g. carbon or tungsten
of about 5 mm in length. Details on target design studies can be found in
[26].

Four absorbers, of the same design as collimators already used in the LHC, are
proposed to be installed about 150 m downstream from the IP2. The first three
are proposed to be made of 1 m long carbon-fibre-carbon composite jaws, as the
present TCSGs in the LHC, while the last one is to be made of 1 m long
tungsten jaws, as the present TCLAs in the LHC; similarly as in [18]. The
difference is that in our study, we use a large opening of about 50
$\rm\sigma$ that still intercepts the channelled beam while being well in the
shadow of the entire collimation system. Such a choice is driven by minimising
the impact of these extra absorbers on the regular collimation system and
machine impedance instead of searching for minimum gaps that maintain the
collimation hierarchy. As will be shown later, we do not experience any
cleaning-related issues. The proposed setup of absorbers follows a
performance-oriented approach with the potential to reduce the number of
required absorbers, based on the energy deposition study to be done in the
future.

##  IV Expected performance

The MAD-X code [27] is used to manage the HL-LHC model, to prepare suitable
lattice and optics descriptions used as input to tracking studies, and to
calculate the trajectory of particles experiencing an angular kick equivalent
to the crystal bending angle. Detailed evaluation of the layout performance is
done using multi-turn particle tracking simulations in SixTrack [28] that
allows a simplectic, fully chromatic and 6D tracking along the magnetic
lattice of the LHC, including interactions with collimators and bent crystals,
and a detailed aperture model of the machine [29, 30]. In our simulations, we
use at least two million protons, initially distributed over a narrow ring of
radius $\rm r+dr$ slightly above 6.7 $\rm\sigma$ in the normalised transverse
vertical position-angle phase space (y,y’) which allows an estimation of the
number of protons impacting the collimation system (including the crystal and
the target of the ALICE-FT layout) as well as the density of protons lost per
metre in the aperture with a resolution of 10 cm along the entire ring
circumference.

The ALICE-FT experiment must be compatible with the standard physics programme
of the LHC, meaning that it cannot add any operational limitations, mostly
related to particle losses, which must stay within acceptable limits. This is
demonstrated in Fig. 4 where a loss map of the machine including the ALICE-FT
system does not contain any abnormal loss spikes comparing to the reference
loss map of the machine without the ALICE-FT system. The only new spikes
correspond to protons impacting the elements of the ALICE-FT setup.

Figure 4: Comparison of loss maps for the machine without (top) the ALICE-FT
system, with the ALICE-FT system without the optics optimisation (middle) and
with the ALICE-FT system with the optics optimisation (bottom). The local
cleaning inefficency (vertical axis) is a measure of the number of protons not
intercepted by the collimation system and impacting the machine aperture. The
simulation limit of 1 proton lost in the machine aperture corresponds to $\rm
5\cdot 10^{-7}m^{-1}$ in a 10 cm longitudinal bin. No abnormal loss spikes are
present in the loss maps.

The setup is developed to provide as many protons as possible impacting the
crystal ($\rm N_{PoC}$) in order to maximise, for a given channeling
efficiency, the number of protons on target. For a given crystal retraction
relative to primary collimators, the proton flux on the crystal strongly
depends on the betatron oscillation phase advance between the primary
collimators at IR7 and the IR2 crystal. With the nominal optics, the phase
advance is nearly least favourable, leading to a low $\rm N_{PoC}$. This can
be corrected by a minor, local modification of the IR2 optics, implemented by
changing strengths of IR2 quadrupole magnets labelled with natural numbers
from 4 to 10 (lower the number, closer the magnet to the IP2) on both sides of
the IP2. Quadrupoles upstream of the IP2 were constrained to shift the phase
advance at the crystal while keeping the IP2 optics parameters unchanged.
Quadrupoles downstream of the IP2 were constrained to recover the same optical
parameters, including the betatron phase, as in the nominal optics. The
maximum $\rm N_{PoC}$ was found for phase advance shifted by $\rm-65^{\circ}$,
as shown in Fig. 5.

Figure 5: The dependence of the fraction of the number of protons impacting
the crystal over the number of protons impacting the primary collimator (PoC)
on the betatron phase from the primary vertical collimator in IR7 (TCP.D) to
the IR2 crystal (CRY). Crystal retraction is $\rm 7.9~{}\sigma$. Statistical
errors are in the order of 1%, which makes the bars hardly visible.

The corresponding optical $\rm\beta_{y}$ function in IR2 is given in Fig. 6
and changes in strengths of quadrupoles are summarised in Table 3. Such a
modification of optics is well feasible to be implemented at the IR2 and does
not affect the rest of the machine, especially losses along the ring as shown
in Fig. 4.

Figure 6: Vertical $\rm\beta$ function and betatron phase for nominal (solid lines) and modified (dashed lines) optics. Position of the crystal is marked with a vertical purple line. Table 3: Normalised strengths of quadrupoles for nominal and modified optics. IR2 left and IR2 right stand for regions upstream and downstream from the IP2, respectively. | Quadrupole strength $\rm k_{1}$ [$\rm 10^{-3}~{}m^{-2}$]
---|---
Quadrupole | IR2 left | IR2 right
number | nominal | modified | nominal | modified
10 | -6.39 | -6.15 | 7.30 | 7.30
9 | 7.01 | 6.89 | -6.60 | -6.82
8 | -5.41 | -3.59 | 6.71 | 6.30
7 | 7.60 | 7.42 | -6.36 | -7.47
6 | -4.91 | -4.17 | 4.33 | 4.20
5 | 2.99 | 2.88 | -3.63 | -4.09
4 | -2.80 | -2.67 | 3.74 | 2.60

The number of protons hitting the target depends on the number of protons
hitting the crystal, their phase-space distribution at the crystal entry face,
parameters of the crystal and crystal position and orientation inside the beam
pipe. All these phenomena are treated by the simulation code we use. Protons
hitting the crystal emerge from the collimation system. Therefore, their
number and phase-space distribution are subject to a complex multi-turn
process which is also simulated. As a result, we obtain PoC and PoT being
fractions of protons hitting the crystal or the target (respectively) over all
protons intercepted by the collimation system. Three values of PoT resulting
from performed simulations and depending on the relative crystal retraction
are given in Fig. 7.

Figure 7: Fraction of the number of protons impacting the target over the
number of protons impacting the primary collimator (PoT) for some values of
crystal half-gap expressed as the number of beam $\rm sigma$. Limits of the
horizontal axis correspond to half-gaps of primary and secondary collimators
in IR7. Statistical errors are in the order of 1%, which makes the bars hardly
visible.

These scaling factors allow us to estimate the actual number of protons
hitting the crystal and (more interestingly) the target under realistic
conditions of the beam, where the most important factors are beam intensity
and lifetime. An exponential decrease of the beam intensity is assumed,
characterised by the time coefficient $\rm\tau$ interpreted as a beam
lifetime, which depends on beam parameters and machine state. A dominant
contribution to the total beam lifetime comes from the beam burn-off due to
collisions ($\rm\tau_{BO}$), while the number of protons on the target $\rm
N_{PoT}$ depends mostly on $\rm\tau_{coll}$ corresponding to beam core
depopulation towards tails that are intercepted by the collimation system.
Following the same assumptions as in [18], with $\rm I_{0}$ being the initial
beam intensity, time coefficients $\rm\tau_{BO}\approx 20h$ and
$\rm\tau_{coll}\approx 200h$, the number of protons impacting the target per
10 h long fill ($\rm T_{fill}$) in 2018 operation conditions can be estimated
as:

$\begin{split}\rm
N_{PoT}&=\frac{1}{2}PoT\int_{0}^{T_{fill}}\frac{I_{0}}{\tau_{coll}}exp\left(-\frac{t}{\tau_{BO}}\right)exp\left(-\frac{t}{\tau_{coll}}\right)dt\\\
&\approx 2.7\times 10^{10}.\end{split}$ (1)

This corresponds to an average flux of protons on target of $\rm 7.5\times
10^{5}$ p/s being roughly one order of magnitude away from the design goal of
$\rm 10^{7}$ p/s. These two numbers are summarised in Table 4. Please also
note that the intensity of HL-LHC beams will be about a factor of two larger,
which most probably will result in a larger proton flux on target, much closer
to the design goal. However, the final assessment can be established more
precisely by studying the lifetime and beam loss performance of HL-LHC beams.

Table 4: Expected proton flux on target based on 2018 operation conditions compared with expected capabilities of ALICE detector acquisition system. The first number may grow up to a factor of 2 for HL-LHC conditions due to two times larger initial beam intensity. | proton flux on target [p/s]
---|---
estimated for the proposed layout | $\rm 7.5\times 10^{5}$
ALICE acquisition system | $\rm 10^{7}$

##  V Conclusions and outlook

This paper summarises the layout for ALICE fixed-target experiments based on
crystal-assisted beam halo splitting. We have demonstrated that it can be
operated safely without affecting the availability of the LHC for regular
beam-beam collisions. The estimated proton flux on target is roughly one order
of magnitude away from the design goal of $\rm 10^{7}$, with a possible
improvement resulting from larger intensities of HL-LHC beams, allowing to
exploit of the full capabilities of the ALICE detector acquisition system.

##  VI Acknowledgements

We thank for the support provided by members of the ALICE Fixed-Target Project
meetings [31] and Physics Beyond Colliders Fixed-Target Working Group [32],
especially D. Kikoła, L. Massacrier and M. Ferro-Luzzi.

This research has received funding from the European Union’s Horizon 2020
research and innovation programme, project number: 101003442. This research
was also funded in part by the National Science Centre, Poland, project
number: 2021/43/D/ST2/02761.

## References

  * [1] O. S. Brüning _et al._ , _LHC Design Report_ , ser. CERN Yellow Reports: Monographs. Geneva: CERN, 2004\. http://cds.cern.ch/record/782076
  * [2] F. Galluccio _et al._ , “Physics opportunities for a fixed-target programme in the ALICE experiment”, Apr 2019. https://cds.cern.ch/record/2671944
  * [3] K. Aamodt _et al._ , “The ALICE experiment at the CERN LHC. A Large Ion Collider Experiment”, _JINST_ , vol. 3, p. S08002. 259 p, 2008, also published by CERN Geneva in 2010. https://cds.cern.ch/record/1129812
  * [4] C. Barschel _et al._ , “LHC fixed target experiments”, CERN, Geneva, Tech. Rep., Mar 2019. https://cds.cern.ch/record/2653780
  * [5] C. Hadjidakis _et al._ , “A fixed-target programme at the LHC: Physics case and projected performances for heavy-ion, hadron, spin and astroparticle studies”, _Physics Reports_ , vol. 911, pp. 1–83, 2021. https://www.sciencedirect.com/science/article/pii/S0370157321000405
  * [6] D. Mirarchi, “Crystal collimation for LHC”, 2015. https://cds.cern.ch/record/2036210
  * [7] V. M. Biryukov, Y. A. Chesnokov, and V. I. Kotov, _Crystal channeling and its application at high-energy accelerators_ , 1997.
  * [8] A. M. Taratin, “Particle channeling in a bent crystal”, _Physics of Particles and Nuclei_ , vol. 29, no. 5, pp. 437–462, Sep. 1998.
  * [9] W. Scandale and A. Taratin, “Channeling and volume reflection of high-energy charged particles in short bent crystals. crystal assisted collimation of the accelerator beam halo”, _Physics Reports_ , vol. 815, pp. 1–107, 2019.
  * [10] W. Scandale _et al._ , “Feasibility of crystal-assisted collimation in the cern accelerator complex”, _International Journal of Modern Physics A_ , vol. 37, no. 13, p. 2230004, 2022.
  * [11] W. Scandale _et al._ , “Observation of channeling for 6500 GeV/c protons in the crystal assisted collimation setup for LHC”, _Physics Letters B_ , vol. 758, pp. 129–133, 2016. https://www.sciencedirect.com/science/article/pii/S0370269316301514
  * [12] Mirarchi, D., Hall, G., Redaelli, S., and Scandale, W., “Design and implementation of a crystal collimation test stand at the Large Hadron Collider”, _Eur. Phys. J. C_ , vol. 77, no. 6, p. 424, 2017. https://doi.org/10.1140/epjc/s10052-017-4985-4
  * [13] Redaelli, S. _et al._ , “First observation of ion beam channeling in bent crystals at multi-TeV energies”, _Eur. Phys. J. C_ , vol. 81, no. 2, p. 142, 2021. https://doi.org/10.1140/epjc/s10052-021-08927-x
  * [14] M. Patecki, A. Fomin, D. Kikoła, D. Mirarchi, and S. Redaelli, “Status of Layout Studies for Fixed-Target Experiments in Alice Based on Crystal-Assisted Halo Splitting”, _Proceeding to the 64th ICFA Advanced Beam Dynamics Workshop, MOP26_. https://doi.org/10.18429/JACoW-HB2021-MOP26
  * [15] Kickoff meeting for the Physics Beyond Collider study, March 2016. https://indico.cern.ch/event/523655
  * [16] F. Galluccio, “UA9 proposal for beam splitting in IR2”, FTE@LHC and NLOAccess STRONG 2020 joint kick-off meeting, 8.11.2019. https://indico.cern.ch/event/853688/contributions/3620725/
  * [17] A. Fomin, “Updates on IP2 FT layouts”, 21st meeting of the PBC-FT working group, 16.12.2020. https://indico.cern.ch/event/981210/
  * [18] D. Mirarchi, A. Fomin, S. Redaelli, and W. Scandale, “Layouts for fixed-target experiments and dipole moment measurements of short-lived baryons using bent crystals at the LHC”, _The European Physical Journal C_ , vol. 80, 10 2020.
  * [19] G. Apollinari _et al._ , _High-Luminosity Large Hadron Collider (HL-LHC): Technical Design Report V. 0.1_ , ser. CERN Yellow Reports: Monographs. Geneva: CERN, 2017. https://cds.cern.ch/record/2284929
  * [20] R. W. Assmann _et al._ , “The final collimation system for the LHC”, p. 4 p, Jul 2006, revised version submitted on 2006-09-15 14:36:59. http://cds.cern.ch/record/972336
  * [21] S. Redaelli, R. Bruce, A. Lechner, and A. Mereghetti, “Chapter 5: Collimation system”, pp. 87–114, 2020. https://cds.cern.ch/record/2750434
  * [22] LHC optics repository. https://abpdata.web.cern.ch/abpdata/lhc_optics_web/www/hllhc15/
  * [23] O. Aberle _et al._ , _High-Luminosity Large Hadron Collider (HL-LHC): Technical design report_ , ser. CERN Yellow Reports: Monographs. Geneva: CERN, 2020. https://cds.cern.ch/record/2749422
  * [24] E. Carlier, “Safe Injection into the LHC”, 2003. https://cds.cern.ch/record/642470
  * [25] R. Filippini, E. Carlier, L. Ducimetière, B. Goddard, and J. Uythoven, “Reliability Analysis of the LHC Beam Dumping System”, 2005. https://cds.cern.ch/record/841092
  * [26] K. Pressard, “Solid target design for ALICE”, Joint workshop ”GDR-QCD/QCD@short distances and STRONG2020/PARTONS/FTE@LHC/NLOAccess”, 2.06.2021. https://indico.ijclab.in2p3.fr/event/7201/contributions/22534/
  * [27] H. Grote and F. Schmidt, “MAD-X: An Upgrade from MAD8”, no. CERN-AB-2003-024-ABP, p. 4 p, May 2003. http://cds.cern.ch/record/618496
  * [28] G. Ripken and F. Schmidt, “A symplectic six-dimensional thin-lens formalism for tracking”, CERN, Geneva, Tech. Rep. CERN-SL-95-12. CERN-SL-95-12-AP. DESY-95-063, Apr 1995. https://cds.cern.ch/record/281283
  * [29] _ICFA Mini-Workshop on Tracking for Collimation in Particle Accelerators: CERN, Geneva, Switzerland 30 Oct 2015. ICFA Mini-Workshop on Tracking for Collimation in Particle Accelerators_ , CERN. Geneva: CERN, Dec 2018. https://cds.cern.ch/record/2646800
  * [30] A. Mereghetti _et al._ , “Sixtrack-Fluka Active Coupling for the Upgrade of the SPS Scrapers”, _Conf. Proc._ , vol. C130512, p. WEPEA064, 2013. https://cds.cern.ch/record/1636191
  * [31] ALICE Fixed-Target Project meetings. https://indico.cern.ch/category/11595/
  * [32] Physics Beyond Colliders Working Group. https://indico.cern.ch/category/8815/