# Evaluating The Robustness of Self-Supervised Representations to
Background/Foreground Removal

Xavier F. Cadet
Imperial College London &Ranya Aloufi
Imperial College London
&Alain Miranville
Université de Poitiers
&Sara Ahmadi-Abhari
Imperial College London
&Hamed Haddadi
Imperial College London
Corresponding author<EMAIL_ADDRESS>
###### Abstract

Despite impressive empirical advances of SSL in solving various tasks, the
problem of understanding and characterizing SSL representations learned from
input data remains relatively under-explored. We provide a comparative
analysis of how the representations produced by SSL models differ when masking
parts of the input. Specifically, we considered state-of-the-art SSL
pretrained models, such as DINOv2, MAE, and SwaV, and analyzed changes at the
representation levels across 4 Image Classification datasets. First, we
generate variations of the datasets by applying foreground and background
segmentation. Then, we conduct statistical analysis using Canonical
Correlation Analysis (CCA) and Centered Kernel Alignment (CKA) to evaluate the
robustness of the representations learned in SSL models. Empirically, we show
that not all models lead to representations that separate foreground,
background, and complete images. Furthermore, we test different masking
strategies by occluding the center regions of the images to address cases
where foreground and background are difficult. For example, the DTD dataset
that focuses on texture rather specific objects.

##  1 Introduction

Self-supervised learning (SSL) extracts supervisory signals from the data
itself by exploiting its underlying structure [1, 2, 3, 4]. Recently, it has
been demonstrated that SSL methods such as SimCLR [5], VICReg [6], MAE [7],
and DINOv2 [8] can achieve superior performance in many downstream vision
tasks, including image classification, object detection, and semantic
segmentation. Self-supervised learning methods for visual representation
learning rely on the design of a good pretext task [9]. Examples include
autoencoders [10, 7], clustering [11, 12], instance-level discrimination [13,
14], and contrastive learning [15].

Predicting missing parts of the input is one of the standard tasks for SSL
pretraining [16, 17]. While this problem has limited the performance
improvement from SSL in vision, new SSL techniques, such as MaskCo [16] and
SimMIM [17], have begun to surpass accuracy records by using masked image
modeling approaches to advance the scaling performance of visual models. Such
observations provide insights into the inner-workings of SSL learning
performance. However, it remains unclear how SSL methods are robust in
learning representations where a portion of the input is masked or missing,
since most of the advances in SSL were developed using a highly curated
dataset, ImageNet [18, 19, 20, 21, 22]. To evaluate this, we list the
fundamental questions we aim to address. We provide tentative answers to these
questions through empirical studies.

Fundamental Questions

#### Q1.

How does the performance of self-supervised learning compare to supervised
learning when dealing with missing information?

#### Q2.

How similar are Self-Supervised representations of inputs with missing
information?

#### Q3.

How does the type of missing information affect the performance of self-
supervised learning algorithms?

###  1.1 Our Contributions

We consider linear evaluation protocol and similarities metrics to assess
robustness of features extracted by Self-Supervised models. We examine how
representations are affected by a lack of information in the inputs, based on
two main assumptions (i) Self-Supervised representations should be
transferable and (ii) Self-Supervised representations of disjoint views should
have low similarities. We provide empirical results based on multiple
experiments over various datasets.

  * •

First, we evaluate the robustness of representations extracted by SSL models
under different masking strategies.

  * •

Then, we investigate the similarities of the representations of images views
that have no intersections, and show that the State-Of-The-Art DINOv2 model
leads to representation similarities matching those from the baseline
Supervised model.

  * •

Finally, we show that depending on the Self-Supervised training strategies the
representations of masked input can be close to one another and separated by
the model.

##  2 Related Work

Self-supervised Learning. Self-supervised Learning has shown promising results
over the past years, closing the gap with Supervised Learning methods [5, 3,
19]. According to Balestriero et. al, in [4] SSL can be classified into four
broad families: the Deep Metric Learning family (e.g., SwAV [22] and SimCLR
[5]), the Self-Distillation family (e.g., DINOv2 [8]), the Canonical
Correlation Analysis family (e.g., VICReg [6]), and the Masked Image Modeling
family (e.g., MAE [7]). Self-supervised pretraining is commonly evaluated on
image classification [4], by training a linear classifier using pretrained
representations, but there are other evaluation methods like visual evaluation
[23]. Bordes et al. [23] suggest, for example, training a conditional
generative diffusion model using an SSL representation. Thus, one can gain
some insight into the representation by analyzing which information remains
constant across different generated samples and which information does not
remain constant. In contrast to other approaches, we are inspired by
literature [24, 25] that uses representational similarity analysis to compare
the robustness of supervised and SSL models to corrupted data inputs.

Masked Inputs and Representation Learning. Masked image modeling (MIM) has
recently become a popular self-supervised visual pretraining method due to its
impressive fine-tuning performance on a variety of downstream computer vision
tasks [26, 27, 28]. Beyond representation learning, masked image modeling is a
classical computer vision problem, named image inpainting. This problem has
been extensively studied in computer vision for a long time [29], aiming for
improving the inpainting quality and without connecting to self-supervised
representation learning. The iGPT, ViT, and BEiT research teams have recently
re-explored this approach with regard to modern vision transformers [30], and
they demonstrate a strong potential for representation learning by introducing
specific designs for some components. These include clustering pixels,
predicting mean color, and tokenizing using a block-wise masking strategy in
addition to dVAE networks [17]. We explore such techniques as a means of
constructing approximate views, based on multiple benchmark datasets, to
simulate the corruption that occurs in real-world scenarios.

Representation Similarities. Recent research has attempted to explain neural
networks’ behavior by comparing representations between layers and among
different trained models. We build upon previous studies investigating neural
network representations’ similarity [24] to evaluate the robustness of
different self-supervised learning methods when learning from masked or
missing data. In particular, we compare the learning performance of various
SSL methods that belong to different families using Canonical Correlation
Analysis (CCA) [31] and Centered Kernel Alignment (CKA) [32]. CCA is a
Multivariate Statistics technique that identifies and measures the
relationships between two sets of variables, or to find patterns that
represent the variation in the data. It constructs pairs of linear
combinations (called canonical variables) from the original variables that are
orthogonal within each set and maximally correlated between sets. In contrast,
CKA analyzes representational similarity matrices and is not invariant under
invertible linear transformations [25]. This enables CKA to identify
correspondences between representations extracted from networks trained from
different initializations.

##  3 Methodology

###  3.1 Preliminaries

In this study, we investigate how well self-supervised representations perform
with incomplete or corrupted data, assuming that these representations should
be adaptable and maintain satisfactory performance in real-world scenarios. To
test this, we evaluate the similarities between two self-supervised
representation vectors using Canonical Correlation Analysis (CCA) and Centered
Kernel Alignment (CKA) cross variations of datasets. Let’s denote the first
representation vector as $X=[x_{1},x_{2},...,x_{n}]$ and the second
representation vector as $Y=[y_{1},y_{2},...,y_{n}]$, where $n$ represents the
dimensionality of the vectors. We use different similarity metrics to quantify
the degree of similarity between $X$ and $Y$. By applying these statistical
approaches, we can quantitatively evaluate the similarities between self-
supervised representations to provide valuable insights into the alignment and
linear relationships between these representations.

###  3.2 Segmentation As Missing Information

Exploring segmentation as missing information in self-supervised learning has
the potential to improve the robustness and scalability of representation
learning. By inferring missing segmentations from visual cues, it is possible
to develop more comprehensive and holistic representations that encompass both
global context and local details. Furthermore, this approach could contribute
to the advancement of self-supervised learning techniques and contribute to
utilizing visual data more efficiently and effectively in a variety of
applications, such as object recognition, scene understanding, and image
synthesis. We leverage the Segment Anything Model (SAM) [33] to separate the
object associated to the target class in the dataset. For instance, if the
class to predict in the image is apple, we obtain the segmentation mask of the
apple and generate two variations of the image: the Foreground and the
Background images. These two variations share no pixels.

###  3.3 Region Masking as Missing Information

By considering region masking as missing information, self-supervised models
are encouraged to learn robust and discriminative representations that can
effectively handle unseen or partially masked regions in real-world scenarios.
By using this approach, the model is better able to capture the underlying
semantics and spatial relationships of objects without explicitly annotating
the regions. We generate Circular Masks, ensuring that the mask covers the
same distance in all direction from the center of the image. From these masks,
we generate two other variations of the image: the Center and Border versions
of the image. The two variations have no pixels in common.

Figure 1: Examples of the 5 variations explored. On the left we show the
Original, Foreground, and Background from top to bottom, on CIFAR-100,
Caltech101, FGVC Aircraft. On the right we show the Original, Center, Border
from top to bottom, CIFAR-100, DTD and DTD

##  4 Experimental Setup

Datasets. We consider 4 datasets. We consider the FGVC AirCraft [34],
Caltech-101 [35], CIFAR-100 [36], DTD [37]. This selection of dataset covers
different sample size, resolution. The DTD dataset provide a case where the
objective is focusing on the texture rather than a specific object.

Pretrained Models. We consider the default Supervised ResNet50 model from the
Pytorch [38] library. Only the Supervised architecture has access to labels
during the training phase. We obtain pretrained weights of ResNet50[39]
models. All these ResNet50 models are pretrained on ImageNet-1K. We obtain the
pretrained models, from different source, the majority of the weights come
from [40]. We selected the best performing models based on the linear
classification benchmark. Additionally we consider models based on the Vision
Transformers (ViT) [41].

Masking Strategies. We generate different versions of the datasets by applying
different masks. We consider an Image Masking Scheme with Circular Masks and
using Masks obtained from a Segmentation Tool. By construction, for the
Circular Masks the Center and Border images share no pixels. Similarly, the
Foreground an Background images share no pixels.

Feature Extraction. For each pretrained model, we freeze the weights and
replace the final layer by an identify function. For each dataset we resize
the images to $224$ pixels along the shorter side followed by center cropping
to obtain images of size $224\times 224$. We then normalize the images using
the mean and standard variation obtained on ImageNet-1K. Finally, we apply the
different masking strategies to obtain the 5 variants of the dataset. The
Original images, the Foreground only, Background only, Center only and Border
only.

Linear Evaluation. We processed to 5-fold cross validation using the feature
extracted using the different models.

##  5 Results

Table 1: Linear Evaluation on CIFAR-100 and Caltech101, reported results are based on average Balanced Accuracy over 5-folds. Dataset | cifar100
---|---
Variation | Original | Foreground | Background | Center | Border
supervised | $0.59_{(0.0)}$ | $0.54_{(0.01)}$ | $0.28_{(0.01)}$ | $0.52_{(0.0)}$ | $0.33_{(0.0)}$
swav | $0.65_{(0.0)}$ | $0.57_{(0.0)}$ | $0.3_{(0.0)}$ | $0.54_{(0.0)}$ | $0.33_{(0.01)}$
vicreg | $0.62_{(0.01)}$ | $0.57_{(0.0)}$ | $0.28_{(0.01)}$ | $0.54_{(0.0)}$ | $0.33_{(0.0)}$
dinov2_vitg14 | $0.86_{(0.0)}$ | $0.74_{(0.0)}$ | $0.31_{(0.01)}$ | $0.77_{(0.0)}$ | $0.44_{(0.01)}$
mae | $0.46_{(0.01)}$ | $0.36_{(0.01)}$ | $0.19_{(0.0)}$ | $0.32_{(0.01)}$ | $0.22_{(0.0)}$
Dataset | caltech101
Variation | Original | Foreground | Background | Center | Border
supervised | $0.93_{(0.01)}$ | $0.89_{(0.01)}$ | $0.75_{(0.01)}$ | $0.9_{(0.0)}$ | $0.62_{(0.01)}$
swav | $0.91_{(0.01)}$ | $0.87_{(0.02)}$ | $0.72_{(0.01)}$ | $0.88_{(0.01)}$ | $0.55_{(0.02)}$
vicreg | $0.92_{(0.01)}$ | $0.88_{(0.01)}$ | $0.73_{(0.01)}$ | $0.89_{(0.01)}$ | $0.56_{(0.02)}$
dinov2_vitg14 | $0.97_{(0.01)}$ | $0.96_{(0.01)}$ | $0.89_{(0.01)}$ | $0.98_{(0.0)}$ | $0.83_{(0.01)}$
mae | $0.69_{(0.03)}$ | $0.71_{(0.02)}$ | $0.37_{(0.02)}$ | $0.69_{(0.02)}$ | $0.25_{(0.02)}$

In the following we first analyze the linear evaluation performance achieved
using features extracted by the different models across dataset variants.
Then, we consider the representations similarities across variants using first
CCA, then CKA. Finally, we discuss the representation proximity using the
information from the nearest neighbors.

Figure 2: Matrices based on the Mean Squared CCA correlation $R^{2}_{CCA}$.
The first row is based on CIFAR-100, the second row on Caltech101. From left
to right, the matrices are associated with the representation extracted by the
Supervised, SwAV, VICReg, MAE and DINOv2 models. Original, Foreground,
Background, Center, Border.

Figure 3: Matrices based on the CKA scores. The first row is based on
CIFAR-100, the second row on Caltech101. From left to right, the matrices are
associated with Supervised, SwAVm VICReg, MAE and DINOv2. Original,
Foreground, Background, Center, Border.

Figure 4: The first row indicates how each variant neighboring points are
related to other variants. These observations are based on the representations
extracted on CIFAR-100. From left to right, we consider the Supervised, DINOv2
and MAE models. The second row depicts the UMAP obtained on $10\%$ of the
representations.

###  5.1 Linear Evaluation of the image variations

We report the results of Linear Evaluation on CIFAR-100 and Caltech-101 in
Table 1. Results on the other datasets can be found in the Supplementary
Material. All the models have consistently higher classification accuracy when
using the Original image rather than the Foreground, indicating that there is
additional contextual information that is extracted by the models. This
contextual information is then leveraged by the linear classifiers. DINOv2
obtains better performance when using the Center variant compare to the
Foreground variant. For the Foreground variant, there is no guarantee that the
entirety of the object will be located in the center of the image, even-though
most common datasets locate the object in the center of the image. For
example, in the Caltech-101 dataset, the images which have human faces are
target, are not necessarily centered and part of the faces are outside the
Center variant. Nonetheless, all the models except MAE reach higher balanced
accuracy on the Original image than on the Foreground, and the Center variant
leads to higher performance compared to the Foreground variant. That could
indicate that the Self-Supervised models tend to learn additional contextual
information from the background. When evaluated on only the Background version
of the dataset, all models except the MAE-based were able to achieve
relatively high balanced accuracy, above $70\%$ balanced accuracy. This could
indicate that the dataset lacks background diversity for the different classes
and that using only the background information is sufficient on this dataset.

Our findings, highlight that leveraging segmentation to generate different
variations of the datasets could help to improve the robustness of the models.

###  5.2 Representation Similarity using CCA

We wonder if the linear evaluation balanced accuracies give enough information
about the robustness of the representations learned by the Self-Supervised
models. We report the CCA matrices in figure 2 for CIFAR-100 and Caltech-101.
From CIFAR-100, we observe that across all models, the representations of the
Original images are most similar to the Foreground variants with similarity
scores ranging from $0.24$ for the MAE model, to the $0.4$ for the Supervised
model. This trend matches with the observations from the Linear Evaluations.
Furthermore, for both the Supervised and DINOv2 models, the Original and
Center variants of the datasets are the most similar followed by the Center to
Foreground representations. On the Caltech101, we observe that across all
models the most similar representations are those of the Original and Center
images. This could come from the smaller size of the objects of interest in
these datasets with respect to the size of the images. As such, the Original
and Foreground images would lead to less similar representations. This is
further supported by the higher level of similarities between Center and
Foreground as well as Border and Background.

We observe that DINOv2 follows similar learning patterns to the Supervised
model compared to other Self-supervised models. Nonetheless, the balanced
accuracies achieved using the representations extracted by DINOv2 are higher
than those obtained using the representations from the Supervised Model.

Our findings call for further research investing whether to design new pretext
tasks, or to focus on creating novel preprocessing steps to obtain curated
datasets.

###  5.3 Representation Similarity using CKA

We further compare the representations using the CKA, and provide the results
in Figure 3 We observe that the Supervised and DINOv2 models have higher
similarity scores between the Original and Center variants, followed by
Original and Foreground. On CIFAR-100 all the models have lower similarity
between the Center and Background representations. Which is to expect since
the objects are located in the center of the images. Furthermore, the MAE
model also shows really low similarity between the Background and Original,
and Border and Background. On the Caltech101 dataset, the DINOv2
representations lead to are mostly similar across all variants, with score all
above $0.64$. This also shows that while the CCA indicated that the Center and
Background representations were not that similar, the CKA indicates that for
DINOv2 the Center and Background representations are highly similar with a
score $0.77$. These corroborate the Linear Evaluation performance observed,
with the high balanced using DINOv2’s representations on only background
images. Given the pretrained task of the MAE model, it was expected that it
would be more robust to the masking strategies.

###  5.4 K-Neighbors in the representation space

We then ask: Are samples in the representations neighboring samples from the
same variants ? Through the K-Nearest Neighbors analysis, in Figure 4, we
observe that the Supervised and DINOv2 Models leads to Original
representations that are close to a mix of others. Indeed, only $40\%$ of the
Original and Foreground samples have variant-matching neighbors. Meanwhile, in
the MAE representation space the different variant are mostly matching their
neighbors. Indeed, all variants have more than $80\%$ of their closest
neighbors belonging to the similar variant.

We further visualize the representation spaces using UMAP. This highlights
that MAE leads to disjointed representation from different variants.
Interestingly, it isolated the Center and Border representations from the
other variants.

##  6 Discussion and Limitations

Understanding the robustness of self-supervised representations is crucial for
assessing their practicality and reliability in real-world applications. By
investigating the performance of these representations under varying degrees
of data incompleteness or corruption, we can identify potential limitations
and design strategies to enhance their robustness. We studied the effect of
Foreground / Background removal and input masking on the representations
extracted by Self-Supervised models. We investigated how masked input impacted
Self-Supervised models compared to Supervised models, and showed that the
DINOv2 model leads to representations similarity that match those of the
reference supervised model. Despite these similarities, the learned
representation lead to higher balanced accuracies scores.

Limitations: In this work we only considered segmentation and masking as means
of missing information. Further settings and strategies could be explored. Our
study could be extended to other tasks and diverse datasets.

Further analysis, should be conducted to determine whether developing novel
pretext tasks, or defining novel preprocessing steps to obtain curated
datasets would be most beneficial.

## Broader Impact

By examining the performance of self-supervised representations under masked
or missing inputs, we gain insights into their ability to handle real-world
scenarios where data may be corrupted, occluded, or partially available. Our
exploration contributes to the continuous development and improvement of self-
supervised learning techniques, allowing for the creation of more effective
and adaptable models in the face of imperfect or incomplete data.

## Acknowledgments and Disclosure of Funding

The authors would like to thank Sandra Siby for numerous comments, technical
questions, references, and invaluable suggestions for the presentation that
led to an improved text. Xavier F. Cadet is supported by UK Research and
Innovation [UKRI Centre for Doctoral Training in AI for Healthcare grant
number EP/S023283/1]. This research is supported by UK DRI/MRC Infrastructure
for a Trusted Research Environment to Support Cost-effective and Flexible
Processing of Healthcare Data grant and UK DRI Translational Machine
Intelligence Programme (UKDRI-7002) For the purpose of open access, the
authors have applied a Creative Commons Attribution (CC BY) license to any
Author Accepted Manuscript version arising.

## References

  * [1] Longlong Jing and Yingli Tian.  Self-supervised Visual Feature Learning with Deep Neural Networks: A Survey, February 2019.  arXiv:1902.06162 [cs].
  * [2] Xinlei Chen and Kaiming He.  Exploring Simple Siamese Representation Learning.  arXiv:2011.10566 [cs], November 2020.  arXiv: 2011.10566.
  * [3] Kaiming He, Haoqi Fan, Yuxin Wu, Saining Xie, and Ross Girshick.  Momentum Contrast for Unsupervised Visual Representation Learning, March 2020.  arXiv:1911.05722 [cs].
  * [4] Randall Balestriero, Mark Ibrahim, Vlad Sobal, Ari Morcos, Shashank Shekhar, Tom Goldstein, Florian Bordes, Adrien Bardes, Gregoire Mialon, Yuandong Tian, et al.  A cookbook of self-supervised learning.  arXiv preprint arXiv:2304.12210, 2023.
  * [5] Ting Chen, Simon Kornblith, Mohammad Norouzi, and Geoffrey Hinton.  A Simple Framework for Contrastive Learning of Visual Representations.  arXiv:2002.05709 [cs, stat], June 2020.  arXiv: 2002.05709.
  * [6] Adrien Bardes, Jean Ponce, and Yann Lecun.  Vicreg: Variance-invariance-covariance regularization for self-supervised learning.  In ICLR 2022-10th International Conference on Learning Representations, 2022.
  * [7] Kaiming He, Xinlei Chen, Saining Xie, Yanghao Li, Piotr Dollár, and Ross Girshick.  Masked autoencoders are scalable vision learners.  In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 16000–16009, 2022.
  * [8] Maxime Oquab, Timothée Darcet, Théo Moutakanni, Huy Vo, Marc Szafraniec, Vasil Khalidov, Pierre Fernandez, Daniel Haziza, Francisco Massa, Alaaeldin El-Nouby, et al.  Dinov2: Learning robust visual features without supervision.  arXiv preprint arXiv:2304.07193, 2023.
  * [9] Yanbei Chen, Massimiliano Mancini, Xiatian Zhu, and Zeynep Akata.  Semi-Supervised and Unsupervised Deep Visual Learning: A Survey.  IEEE Transactions on Pattern Analysis and Machine Intelligence, pages 1–23, 2022.
  * [10] Marc’Aurelio Ranzato, Fu Jie Huang, Y-Lan Boureau, and Yann LeCun.  Unsupervised learning of invariant feature hierarchies with applications to object recognition.  In 2007 IEEE conference on computer vision and pattern recognition, pages 1–8. IEEE, 2007.
  * [11] Adam Coates, Andrew Ng, and Honglak Lee.  An analysis of single-layer networks in unsupervised feature learning.  In Proceedings of the fourteenth international conference on artificial intelligence and statistics, pages 215–223. JMLR Workshop and Conference Proceedings, 2011.
  * [12] Mathilde Caron, Piotr Bojanowski, Armand Joulin, and Matthijs Douze.  Deep clustering for unsupervised learning of visual features.  In Proceedings of the European conference on computer vision (ECCV), pages 132–149, 2018.
  * [13] Zhirong Wu, Yuanjun Xiong, Stella Yu, and Dahua Lin.  Unsupervised Feature Learning via Non-Parametric Instance-level Discrimination, May 2018.  arXiv:1805.01978 [cs].
  * [14] Yucheng Zhao, Guangting Wang, Chong Luo, Wenjun Zeng, and Zheng-Jun Zha.  Self-Supervised Visual Representations Learning by Contrastive Mask Prediction, August 2021.  arXiv:2108.07954 [cs].
  * [15] Senthil Purushwalkam and Abhinav Gupta.  Demystifying Contrastive Self-Supervised Learning: Invariances, Augmentations and Dataset Biases, July 2020.  arXiv:2007.13916 [cs].
  * [16] Yucheng Zhao, Guangting Wang, Chong Luo, Wenjun Zeng, and Zheng-Jun Zha.  Self-supervised visual representations learning by contrastive mask prediction.  In Proceedings of the IEEE/CVF International Conference on Computer Vision, pages 10160–10169, 2021.
  * [17] Zhenda Xie, Zheng Zhang, Yue Cao, Yutong Lin, Jianmin Bao, Zhuliang Yao, Qi Dai, and Han Hu.  Simmim: A simple framework for masked image modeling.  In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 9653–9663, 2022.
  * [18] Jia Deng, Wei Dong, Richard Socher, Li-Jia Li, Kai Li, and Li Fei-Fei.  ImageNet: A Large-Scale Hierarchical Image Database.  page 8, 2009.
  * [19] Jean-Bastien Grill, Florian Strub, Florent Altché, Corentin Tallec, Pierre H. Richemond, Elena Buchatskaya, Carl Doersch, Bernardo Avila Pires, Zhaohan Daniel Guo, Mohammad Gheshlaghi Azar, Bilal Piot, Koray Kavukcuoglu, Rémi Munos, and Michal Valko.  Bootstrap your own latent: A new approach to self-supervised Learning.  arXiv:2006.07733 [cs, stat], September 2020.  arXiv: 2006.07733.
  * [20] Xiao Zhang and Michael Maire.  Self-Supervised Visual Representation Learning from Hierarchical Grouping.  In Advances in Neural Information Processing Systems, volume 33, pages 16579–16590. Curran Associates, Inc., 2020.
  * [21] Priya Goyal, Mathilde Caron, Benjamin Lefaudeux, Min Xu, Pengchao Wang, Vivek Pai, Mannat Singh, Vitaliy Liptchinsky, Ishan Misra, Armand Joulin, et al.  Self-supervised pretraining of visual features in the wild.  arXiv preprint arXiv:2103.01988, 2021.
  * [22] Mathilde Caron, Ishan Misra, Julien Mairal, Priya Goyal, Piotr Bojanowski, and Armand Joulin.  Unsupervised Learning of Visual Features by Contrasting Cluster Assignments.  arXiv:2006.09882 [cs], January 2021.  arXiv: 2006.09882.
  * [23] Florian Bordes, Samuel Lavoie, Randall Balestriero, Nicolas Ballas, and Pascal Vincent.  A surprisingly simple technique to control the pretraining bias for better transfer: Expand or narrow your representation.  arXiv preprint arXiv:2304.05369, 2023.
  * [24] Aarre Laakso and Garrison Cottrell.  Content and cluster analysis: assessing representational similarity in neural systems.  Philosophical psychology, 13(1):47–76, 2000.
  * [25] Simon Kornblith, Mohammad Norouzi, Honglak Lee, and Geoffrey Hinton.  Similarity of neural network representations revisited.  In International Conference on Machine Learning, pages 3519–3529. PMLR, 2019.
  * [26] Carl Doersch, Abhinav Gupta, and Alexei A Efros.  Unsupervised visual representation learning by context prediction.  In Proceedings of the IEEE international conference on computer vision, pages 1422–1430, 2015.
  * [27] Mark Chen, Alec Radford, Rewon Child, Jeffrey Wu, Heewoo Jun, David Luan, and Ilya Sutskever.  Generative pretraining from pixels.  In International conference on machine learning, pages 1691–1703. PMLR, 2020.
  * [28] Trieu H Trinh, Minh-Thang Luong, and Quoc V Le.  Selfie: Self-supervised pretraining for image embedding.  arXiv preprint arXiv:1906.02940, 2019.
  * [29] Deepak Pathak, Philipp Krahenbuhl, Jeff Donahue, Trevor Darrell, and Alexei A Efros.  Context encoders: Feature learning by inpainting.  In Proceedings of the IEEE conference on computer vision and pattern recognition, pages 2536–2544, 2016.
  * [30] Mathilde Caron, Hugo Touvron, Ishan Misra, Hervé Jégou, Julien Mairal, Piotr Bojanowski, and Armand Joulin.  Emerging properties in self-supervised vision transformers.  In Proceedings of the IEEE/CVF international conference on computer vision, pages 9650–9660, 2021.
  * [31] David R Hardoon, Sandor Szedmak, and John Shawe-Taylor.  Canonical correlation analysis: An overview with application to learning methods.  Neural computation, 16(12):2639–2664, 2004.
  * [32] Simon Kornblith, Mohammad Norouzi, Honglak Lee, and Geoffrey Hinton.  Similarity of Neural Network Representations Revisited.  In Proceedings of the 36th International Conference on Machine Learning, pages 3519–3529. PMLR, May 2019.  ISSN: 2640-3498.
  * [33] Alexander Kirillov, Eric Mintun, Nikhila Ravi, Hanzi Mao, Chloe Rolland, Laura Gustafson, Tete Xiao, Spencer Whitehead, Alexander C. Berg, Wan-Yen Lo, Piotr Dollár, and Ross Girshick.  Segment anything.  arXiv:2304.02643, 2023.
  * [34] Subhransu Maji, Esa Rahtu, Juho Kannala, Matthew Blaschko, and Andrea Vedaldi.  Fine-Grained Visual Classification of Aircraft, June 2013.  arXiv:1306.5151 [cs].
  * [35] Li Fei-Fei, R. Fergus, and P. Perona.  Learning Generative Visual Models from Few Training Examples: An Incremental Bayesian Approach Tested on 101 Object Categories.  In 2004 Conference on Computer Vision and Pattern Recognition Workshop, pages 178–178, June 2004.
  * [36] A. Krizhevsky.  Learning Multiple Layers of Features from Tiny Images, 2009\.
  * [37] Mircea Cimpoi, Subhransu Maji, Iasonas Kokkinos, Sammy Mohamed, and Andrea Vedaldi.  Describing Textures in the Wild.  In 2014 IEEE Conference on Computer Vision and Pattern Recognition, pages 3606–3613, Columbus, OH, USA, June 2014. IEEE.
  * [38] Eli Stevens, Luca Antiga, and Thomas Viehmann.  Deep learning with PyTorch.  Manning Publications Co, Shelter Island, NY, 2020.
  * [39] Kaiming He, Xiangyu Zhang, Shaoqing Ren, and Jian Sun.  Deep Residual Learning for Image Recognition.  arXiv:1512.03385 [cs], December 2015.  arXiv: 1512.03385.
  * [40] Priya Goyal, Quentin Duval, Jeremy Reizenstein, Matthew Leavitt, Min Xu, Benjamin Lefaudeux, Mannat Singh, Vinicius Reis, Mathilde Caron, Piotr Bojanowski, Armand Joulin, and Ishan Misra.  VISSL, 2021.
  * [41] Alexey Dosovitskiy, Lucas Beyer, Alexander Kolesnikov, Dirk Weissenborn, Xiaohua Zhai, Thomas Unterthiner, Mostafa Dehghani, Matthias Minderer, Georg Heigold, Sylvain Gelly, Jakob Uszkoreit, and Neil Houlsby.  An image is worth 16x16 words: Transformers for image recognition at scale.  In International Conference on Learning Representations, 2021.


Dipion and dikaon photoproduction in ultra-peripheral Pb-Pb collisions with
ALICE

Abdennacer Hamdi on behalf of the ALICE Collaboration

Nuclear Science Division, Lawrence Berkeley National Laboratory, Berkeley CA
94720 USA

> High energy photons originating from the electromagnetic field of
> ultrarelativistic lead nuclei can interact with the other lead ion. These
> reactions are studied in the ultra-peripheral heavy ion collisions to probe
> the physics of strong interactions. The analysis of dipion and dikaon
> photoproduction was carried out using the ALICE 2015 Pb–Pb data at center-
> of-mass energy $\sqrt{s}~{}=~{}5.02$ TeV, and 2017 Xe–Xe data at
> $\sqrt{s}~{}=~{}5.44$ TeV. We present the $\rho^{0}$ meson and direct
> $\pi^{+}$ $\pi^{-}$ measurements, as well as the prospects of studying
> exclusive $\rm{K}^{+}\rm{K}^{-}$ photoproduction, with sensitivity towards
> higher dikaon invariant mass above the $\phi(1020)$ threshold.

> PRESENTED AT
>
>
>
>
> DIS2023: XXX International Workshop on Deep-Inelastic Scattering and Related
> Subjects,
> Michigan State University, USA, 27-31 March 2023
>

##  1 Introduction

Photon-induced interactions can be produced abundantly in Ultra-Peripheral
Collisions (UPC) of relativistic heavy ions [1, 2, 3], with the very intense
photon beams reaching the energy frontiers at the LHC. At UPC the impact
parameter is greater then the sum of the two colliding nuclei radii, and due
to the short range nature of Quantum Chromo-Dynamics (QCD), the hadronic
interactions are suppressed.

The photons can fluctuate to a quark-antiquark dipole [4] and scatters off the
entire nucleus target coherently, or on some nucleon sites incoherently. The
vector meson is then produced with the same quantum number, $J^{PC}=1^{--}$,
as the photon, which then decays to a final state dihadrons or dileptons,
e.g.: dipions and dikaons. In addition to the vector meson production, the
photons can fluctuate directly to the dihadrons, leaving two undistinguishable
final states, which leads to an interference of the direct and resonant
production modes.

The photoproduction of vector mesons is expected to be sensitive to the gluon
distribution in nuclei [5, 6]. Allowing to study nuclear shadowing effects
[7], suppression of the nuclear form factor of nuclei with respect to the
superposition of its nucleons, both at low Bjorken-$x$ and low transferred
4-momentum $Q^{2}$.

The strong electromagnetic fields accompanying the colliding ions could give
rise to further photon exchanges between the nuclei. This leads to nuclear
fragmentation of one or both the excited ion targets, which is accompanied by
neutron emission at beam rapidities. In UPC, the photons could be emitted from
either nucleus, leading to a two-fold ambiguity on the photon energy [8]. The
measurement of vector mesons cross section in different neutron emission
classes at beam rapidities has been proposed as a tool to disentangle the two
photon energies [9, 10].

##  2 Experimental Setup and Event Selection

The ALICE detector, designed with wide kinematic coverage, is optimized to
study the high particle-multiplicity environment of ultra-relativistic heavy-
ion collisions, full detector description can be found in [11, 12]. For the
photoproduction of pion and kaon pairs in UPC events, few sub-detectors were
used. Mainly, the Inner Tracking System (ITS), where the pair tracks
traversing its silicon layers in a near back-to-back topology where selected.
The pions/kaons are identified with their energy loss
($\textrm{d}E/\textrm{d}x$) in the Time Projection Chamber (TPC), more details
on kaon identification is discussed in section 4. Both tracking systems are
placed in a large solenoid magnet of 0.5 T, and offer a pseudorapidity
coverage $\left|\eta\right|~{}<~{}0$.9 and full azimuth.

In addition, a dedicated trigger is used to select UPC events, involving
vetoes on any activity in forward V0A ($2.8<\eta<5.1$) and ADA
($4.7<\eta<6.3$) and backward V0C ($-3.7<\eta<-1.7$) and ADC
($-6.9<\eta<-4.9$) scintillators.

Two Zero Degree Calorimeters (ZDC) are used to detect neutrons from the
electromagnetic dissociation of the target nuclei. They are located at $\pm
112.5$ m from the interaction point and have a coverage of $|\eta|>8.8$.

##  3 Coherent ${\bf\rho^{0}}$ Photoproduction

The measurement of mass-dependent dipion photoproduction cross section, after
event selection, shows a clean $\rho^{0}$ peak in the pair invariant mass
distribution in Fig. 1. The coherent $\pi^{+}$ $\pi^{-}$ pairs were selected
in the transverse momentum range $p_{\rm T}$ $<$ 0.2 GeV/$c$ and rapidity
$|y|<$ 0.8. The $\rho^{0}$ resonance and direct $\pi^{+}$ $\pi^{-}$ are well
described by the mass-dependent width Breit–Wigner fit. A template for the
remaining small background contribution,
$\gamma\gamma\rightarrow\mu^{+}\mu^{-}$, is added to the fit. Similar to Pb–Pb
UPC, the exclusive $\rho^{0}$ cross section was also measured for the first
time in $\gamma\mathrm{Xe}\rightarrow\rho^{0}\mathrm{Xe}$ process, with
similar $\rho^{0}$ resonance shape observation in the dipion invariant mass.
The contribution of the direct $\pi^{+}$ $\pi^{-}$ to the $\rho^{0}$ resonance
production cross section ratio is found to be consistent for both $\gamma$Pb
and $\gamma$Xe with $0.57\pm 0.01$ (stat.) $\pm 0.02$ (syst.)
(GeV/$c^{2}$)-1/2 in Pb and $0.58\pm 0.04$ (stat.) $\pm 0.03$ (syst.)
(GeV/$c^{2}$)-1/2 in Xe.

Figure 1: Mass-dependent cross section of exclusive dipions in the coherent
pair range $p_{\rm T}$ $<$ 0.2 GeV/$c$, well described by a Breit–Wigner fit
for the $\rho^{0}$ and direct $\pi^{+}$ $\pi^{-}$, the fit includes background
from $\gamma\gamma\rightarrow\mu^{+}\mu^{-}$, figure taken from [13].

The rapidity-dependent $\rho^{0}$ photoproduction cross section in Pb–Pb and
Xe–Xe at UPC is compared to different model predictions (fig. 2). STARlight is
based on parameterized HERA data on exclusive photoproduction on proton, and a
Glauber-like eikonal formalism for nuclear targets [15]. The GKZ prediction is
based on the photon hadronic fluctuations interacting with the nuclei
according to the Gribov-Glauber model of nuclear shadowing [16]. Variations of
the GKZ prediction on the uncertainty of theory parameters are shown as upper
and lower limit of the model. The CCKT model is based on the colour dipole
model with the structure of the nucleon in the transverse plane described by
high gluonic density (hot-spot) [17, 18]. Two versions of the CCKT model are
included, one with the hot-spot and one without (marked ”nuclear”). The last
model is GMMNS, which uses the colour dipole approach with amplitudes obtained
from the IIM model [19], coupled to a Glauber prescription to go from the
nucleon to the nuclear target [20].

The models describe fairly well the Pb data within uncertainties, in
particular models based on nuclear shadowing (GKZ) and saturation (CCKT). For
the Xe data, STARlight and CCKT as well as the lower band of GMMNS predictions
are one standard deviation above data.

Figure 2: Rapidity-dependent cross section for dipion photoproduction in UPC
for (Left) Pb–Pb and (right) Xe–Xe, figures taken from [13, 14].

##  4 Dikaon photoproduction in UPCs

Similar to dipion photoproduction, dikaon final state may come from the
$\phi(1020)\rightarrow K^{+}K^{-}$, or direct production. The two processes
are undistinguishable and therefore interfere with each other. Since the kaons
are much heavier than pions, we expect a much smaller dikaon photoporoduction
cross section. Using current ALICE data, the dikaon production in UPC is
sensitive to much higher $K^{+}K^{-}$ invariant mass, above the $\phi(1020)$
mass threshold. In this high mass region, very limited studies are conducted
for dikaon photoproduction.

After the initial two good tracks selection, and unlike the dipion sample, the
dikaons are overpopulated with pions, with kaon:pion ratio $\sim$ 1 in 1000.
This is resolved by using the particles energy loss $\textrm{d}E/\textrm{d}x$
in TPC, where the number of standard deviations of the measured
$\textrm{d}E/\textrm{d}x$ away from Bethe–Bloch expectation for each particle
species is used, n$\sigma_{i}$ with $i=K,\pi,\mu,e$.

The contamination from pions and other lepton pairs are rejected by the
$\textrm{d}E/\textrm{d}x$ for each particle hypothesis,
$|n\sigma_{(\pi,\mu,e)}|$ $>$ 2\. Figure 3(a) shows the correlation between
the two tracks in the pairs in terms of n$\sigma_{K}$ compatible with kaons,
after $|n\sigma_{(\pi,\mu,e)}|$ $>$ 2\. A clear kaon pairs signal is visible
inside the green circle with a radius of 3$\sigma_{K}$ around the center
point, while the remnant background is estimated outside the signal region
with an equal area. The background region is estimated between two circles of
4$\sigma_{K}$ and 5$\sigma_{K}$ radius from the center, represented in the
region between the two dashed blue lines. The latter is used to estimate the
potential background inside the signal region. Figure 3(b) shows the dikaon
invariant mass for the signal (green) and background (blue) regions. A clean
dikaon signal with negligible background is observed in the
$\rm{K}^{+}\rm{K}^{-}$ mass range [1.1, 1.4 GeV/$c^{2}$ ], wile an increase in
the background level is seen at higher mass. The coherent dikaon
photoproduction cross section, with pair $p_{\rm T}$ $<$ 0.1 GeV/$c$ and
$|y|<$ 0.8, will be measured.

(a)

(b)

Figure 3:  (a) The TPC dE/dx in terms of the number of $\sigma$ away from the
kaon hypothesis for all events after cuts to reject pions, muons and
electrons, but without the cut $|n\sigma_{K}|$ $<$ 3\. (b) Dikaon invariant
mass for the signal (green) and background (blue) regions.

ACKNOWLEDGEMENTS

This work is supported in part by the U.S. Department of Energy, Office of
Science, Office of Nuclear Physics, under contract numbers DE-AC02-05CH11231.

## References

  * [1] C. A. Bertulani, S. R. Klein and J. Nystrand, Ann. Rev. Nucl. Part. Sci. 55 (2005), 271-310.
  * [2] S. Klein and P. Steinberg, Ann. Rev. Nucl. Part. Sci. 70 (2020), 323-354.
  * [3] J. G. Contreras and J. D. Tapia Takaki, Int. J. Mod. Phys. A 30 (2015), 1542012.
  * [4] T. H. Bauer, R. D. Spital, D. R. Yennie, and F. M. Pipkin, Rev. Mod.Phys. 50 (1978) 261.
  * [5] M. G. Ryskin, Z. Phys. C 57 (1993) 89.
  * [6] K. J. Eskola, H. Paukkunen and C. A. Salgado, JHEP 0904 (2009) 065.
  * [7] N. Armesto, J. Phys. G 32 (2006) R367-R394.
  * [8] S. R. Klein, J. Nystrand, Phys.Rev.Lett. 84 (2000) 2330-2333.
  * [9] A. J. Baltz, S. R. Klein, J. Nystrand, Phys. Rev. Lett. 89 (2002) 012301.
  * [10] V. Guzey, M. Strikman, M. Zhalov, Eur. Phys. J. C74 (7) (2014) 2942.
  * [11] B. B. Abelev et al. (ALICE), Int. J. Mod. Phys. A 29, 1430044 (2014).
  * [12] K. Aamodt et al. (ALICE), JINST 3, S08002 (2008).
  * [13] S. Acharya et al. [ALICE], J. High Energ. Phys. 2020, 35 (2020).
  * [14] S. Acharya et al. [ALICE], Phys. Lett. B 820 (2021) 136481.
  * [15] S. R. Klein, J. Nystrand, J. Seger, Y. Gorbunov, and J. Butterworth, Comput. Phys. Commun. 212 (2017) 258-268.
  * [16] V. Guzey, E. Kryshen, and M. Zhalov, Phys. Lett. B 782 (2018) 251-255.
  * [17] J. Cepila, J. G. Contreras, and J. D. Tapia Takaki, Phys. Lett. B 766 (2017) 186-191.
  * [18] J. Cepila, J. G. Contreras, M. Krelina, and J. D. Tapia Takaki, Nucl. Phys. B 934 (2018) 330-340.
  * [19] E. Iancu, K. Itakura, and S. Munier, Phys. Lett. B 590 (2004) 199-208.
  * [20] V. P. Gonçalves, M. V. T. Machado, B. Moreira, F. S. Navarra, and G. S. dos Santos, Phys. Rev. D 96 no. 9, (2017) 094027.