# Medical Image Retrieval Using Pretrained Embeddings

Farnaz Khun Jush  Bayer AG, Berlin, Germany
<EMAIL_ADDRESS>Tuan Truong  Bayer AG, Berlin, Germany
<EMAIL_ADDRESS>Steffen Vogler  Bayer AG, Berlin, Germany
<EMAIL_ADDRESS>Matthias Lenga  Bayer AG, Berlin, Germany
<EMAIL_ADDRESS>
###### Abstract

A wide range of imaging techniques and data formats available for medical
images make accurate retrieval from image databases challenging. Efficient
retrieval systems are crucial in advancing medical research, enabling large-
scale studies and innovative diagnostic tools. Thus, addressing the challenges
of medical image retrieval is essential for the continued enhancement of
healthcare and research. In this study, we evaluated the feasibility of
employing four state-of-the-art pretrained models for medical image retrieval
at modality, body region, and organ levels and compared the results of two
similarity indexing approaches. Since the employed networks take 2D images, we
analyzed the impacts of weighting and sampling strategies to incorporate 3D
information during retrieval of 3D volumes. We showed that medical image
retrieval is feasible using pretrained networks without any additional
training or fine-tuning steps. Using pretrained embeddings, we achieved a
recall of 1 for various tasks at modality, body region, and organ level.

###### Index Terms:

Content-based image retrieval, Medical imaging, Pretrained embeddings, Self-
supervised learning, Supervised learning

##  I Introduction

In the field of computer vision, researchers have been studying content-based
image retrieval (CBIR) for the past decades [1]. CBIR systems typically store
low-dimensional representations of images in a database and retrieve similar
images based on representation similarity. Traditionally, manual crafting of
distinctive features led to a semantic gap, where important image details were
lost due to low-dimensional feature design [2, 1]. Recent advances in deep
learning have shifted the focus toward machine-generated discriminative
feature spaces, to effectively bridge this semantic gap.

While natural image retrieval has seen extensive research, applying retrieval
frameworks to medical images, especially radiology images, remains
challenging. Previous efforts often relied on manually crafted or shallow
learning-based features for various medical image modalities, which are
insufficient for large-scale databases [3]. Deep learning-based approaches
show promise, but the lack of large-scale radiology datasets has been a hurdle
[4]. There are several pretrained models that are trained on huge publicly
available image datasets for various tasks, e.g., image segmentation and
classification. These datasets often consist of natural images e.g., ImageNet
[5] or COCO [6]. It was shown that in a supervised training approach domain-
specific pretraining gives rise to improved performance in downstream tasks
compared to the models trained on natural images [7]. Additionally, pretrained
models using annotated medical images demonstrated better interpretability
compared to the models trained using ImageNet, especially for smaller
radiology datasets [7].

On the other hand, studies showed that ImageNet pretrained features are in
principle transferable to medical imaging tasks [8] e.g., feature extractors
using knowledge distillation with vision transformer (ViT) [9] backbones like
DINO [10] yield richer embeddings on the downstream tasks than supervised
pretrained models [8].

To the best of our knowledge, no study has assessed the feasibility and
performance of utilizing pretrained embeddings, originally trained for non-
CBIR tasks, in the context of medical image retrieval. The contribution of
this study is as follows:

  * •

We evaluated the performance of using start-of-the-art self-supervised
pretrained models i.e., DINOv1 [10], DINOv2 [11], DreamSim [12] trained on
natural images versus a supervised pretrained model based on Swin Transformer
[13] and a ResNet50 model [14] trained on RadImageNet dataset presented in [7]
for 3D medical image retrieval tasks at the modality, body region, and organ
level.

  * •

We compared the results of two different vector database indexing approaches
(LSH and HNSW) for medical image retrieval in combination with different
embedding types.

  * •

Since all the used networks are trained on 2D images and medical images are
often 3D volumes, aggregating the information from various slides plays a
crucial role in the retrieval of 3D images. We propose a simple, yet effective
count-based scheme that allows aggregating 2D similarity information across
the 3D image volumes. Moreover, we compare the impact of weighting and
different sub-sampling techniques on the overall performance.

Figure 1: Overview of the proposed content-based 3D medical image retrieval
system and ablation experiments. Note: $s_{x}-v_{y}$ refers to slice (Sl.) $x$
of volume (Vol.) $y$.

Fig. 1 presents a schematic overview of the proposed method for medical image
retrieval.

##  II Methods

###  II-A Vector Database and Indexing

In the context of image search, the images and related metadata, e.g.
annotations, are typically stored and organized in a database. In contrast, a
query represents the user or system’s specific request for images, which can
generally be in the form of a reference image itself, a textual description,
or a combination of both. The primary goal of a query is to search the image
database for relevant matches based on criteria such as content or metadata.
In the scope of this work, the search process involves comparing a query image
to all images within the database to find the most similar image based on the
minimum distance of the embeddings. We do not rely on any metadata information
in any step.

A similarity search is used to compare data quickly and return relevant
results. As the database size grows, the challenges in accuracy and speed
increase, making naive approaches ineffective. To address this, vector
representations known as embeddings are created using statistical, machine
learning, or deep learning techniques. These embeddings are indexed for
efficient vector similarity search. This study compared two indexing
approaches: Locality Sensitive Hashing (LSH) [15] and Hierarchical Navigable
Small World (HNSW) [16], in the context of medical image retrieval. LSH hashes
data points to improve search efficiency, while HNSW organizes data into a
hierarchical graph structure for efficient navigation during search,
especially suitable for larger datasets with high dimensionality. We used the
Facebook AI Similarity Search (FAISS) package that offers state-of-the art
implementations of the datastructures mentioned above [17].

###  II-B Feature Extraction

The performance of four different slice embedding extractors is benchmarked
for the CBIR tasks. We use DINOv1 [10], DINOv2 [11] and DreamSim [12] which
were pretrained on ImageNet in a _self-supervised_ fashion. DINO was the first
model of this series published [10], and could learn efficient representations
from unlabeled data using self-distillation [10, 18]. DreamSim is based on
DINOv1 and is fine-tuned on synthetic images and a new perceptual metric that
aligns with human perception [12]. DINOv2 revisited existing approaches and
combined various techniques to scale the pretraining process in terms of both
data and model size [11] which outperformed DINOv1 in various tasks at the
time of its publication. Furthermore, a Swin Transformer model [13] and a
ResNet50 model [14] which are trained _supervised_ on the RadImageNet [7] is
used.

###  II-C Dataset

We designed a CBIR benchmark relying on the Medical Segmentation Decathlon
(MSD) challenge dataset which is publicly available [19]. The composition of
the original dataset is shown in the TABLE I. The query cases are sourced from
the MSD Challenge test split, while the train set serves as the database for
searching. The search is assessed on three different levels: _modality_ ,
_body regions_ , and _organs_. The modality category consists of CT or MR
scans (protocols are not included). Organ labels and body regions mapping are
shown in TABLE I. The organ level labels used for the evaluation were derived
from the associated MSD tasks. Thus, the organ level label of a case refers to
just one of the existing organs in the image.

TABLE I: MSD Challenge Dataset Organ | Modality | Cases (Train / Test) | Body region
---|---|---|---
Brain (BRATS) | MRI / 4D | 732 (466 / 266) | Head
Colon | CT / 3D | 190 (126 / 64) | Abdomen
Hepatic vessels | CT / 3D | 443 (303 / 140) | Abdomen
Hippocampus | MRI / 3D | 390 (260 / 130) | Head
Heart (La) | MRI / 3D | 30 (20 / 10) | Chest
Liver | CT / 3D | 201 (131 / 70) | Abdomen
Lung | CT / 3D | 95 (63 / 32) | Chest
Pancreas | CT / 3D | 420 (281 / 139) | Abdomen
Prostate | MRI / 4D | 48 (32 / 16) | Pelvis
Spleen | CT / 3D | 61 (41 / 20) | Abdomen

MSD challenge dataset as shown in TABLE I consists of 3D and 4D volumes (3D
volumes with multiple protocols). In the case of multiple protocols, first 3D
volumes are extracted, and then individual 2D slices of the corresponding 3D
volumes are utilized for embedding extraction. The input size for all the used
models is equal to $224\times 224$ pixels with image replication along the RGB
channel axis. For all the ViT-based models images are normalized to the
ImageNet mean and standard deviation of $(0.485,0.456,0.406)$ and
$(0.229,0.224,0.225)$, respectively. For the ResNet50 model, the images are
normalized to $0.5$ mean and $0.5$ standard deviation based on [7]. The total
size of our database is $439,782$ embeddings, while the final query set
comprises $237,883$ embeddings.

TABLE II: Recall for different body regions Abdomen, Chest, Head, and Pelvis. Comparison: different vector database indexes (Id.), sampling strategies, aggregation variants all, per Image-id level and slice wise (SW) |  Id. |  Embedding | # Random Samples | # Sampled every x mm | # every n slices |  all |  W |  SW
---|---|---|---|---|---|---|---|---
5 |  10 |  20 |  40 |  80 |  3 |  5 |  7 |  10 |  12 |  15 |  3 |  5 |  10
Abdomen |  HNSW |  DreamSim |  .993 |  .990 |  .995 |  .988 |  .997 |  .993 |  .993 |  .993 |  .993 |  .993 |  .995 |  .993 |  .995 |  .995 |  .993 |  1.0 |  .983
DINOv1 |  .857 |  .880 |  .880 |  .880 |  .995 |  1.0 |  1.0 |  1.0 |  .997 |  .997 |  1.0 |  1.0 |  1.0 |  .997 |  1.0 |  1.0 |  .988
DINOv2 |  .988 |  .990 |  .997 |  .993 |  .995 |  .993 |  .993 |  .993 |  .993 |  .993 |  .990 |  .993 |  .995 |  .995 |  .993 |  1.0 |  .979
ResNet50 |  .981 |  .988 |  .990 |  .988 |  .990 |  .990 |  .990 |  .990 |  .990 |  .990 |  .990 |  .990 |  .990 |  .993 |  .990 |  1.0 |  .976
SwinTrans. |  .983 |  .995 |  .993 |  .995 |  .993 |  .995 |  .993 |  .995 |  .995 |  .993 |  .997 |  .995 |  1.0 |  1.0 |  .997 |  .995 |  .987
LSH |  DreamSim |  .993 |  .995 |  .990 |  .993 |  .995 |  .995 |  .995 |  .993 |  .997 |  .997 |  .990 |  .993 |  .995 |  .995 |  .993 |  .997 |  .981
DINOv1 |  .995 |  1.0 |  .997 |  .995 |  1.0 |  1.0 |  1.0 |  1.0 |  1.0 |  1.0 |  1.0 |  1.0 |  1.0 |  1.0 |  1.0 |  1.0 |  .986
DINOv2 |  .988 |  .995 |  .993 |  .988 |  .993 |  .993 |  .993 |  .993 |  .993 |  .995 |  .995 |  .995 |  .995 |  .995 |  .993 |  1.0 |  .974
ResNet50 |  .983 |  .981 |  .983 |  .988 |  .988 |  .990 |  .990 |  .993 |  .997 |  .995 |  .995 |  .993 |  .997 |  .986 |  .990 |  .997 |  .979
SwinTrans. |  .997 |  .986 |  .995 |  .995 |  .997 |  .997 |  .995 |  .995 |  .995 |  .995 |  .993 |  .995 |  .997 |  .997 |  .997 |  .995 |  .985
Chest |  HNSW |  DreamSim |  .857 |  .857 |  .809 |  .857 |  .857 |  .904 |  .857 |  .833 |  .857 |  .857 |  .785 |  .880 |  .880 |  .833 |  .904 |  .595 |  .738
DINOv1 |  .857 |  .880 |  .880 |  .880 |  .880 |  .904 |  .904 |  .928 |  .928 |  .880 |  .857 |  .904 |  .904 |  .880 |  .904 |  .690 |  .804
DINOv2 |  .809 |  .833 |  .880 |  .857 |  .880 |  .857 |  .880 |  .857 |  .880 |  .833 |  .833 |  .857 |  .880 |  .880 |  .880 |  .619 |  .742
ResNet50 |  .880 |  .880 |  .880 |  .928 |  .928 |  .904 |  .928 |  .904 |  .928 |  .904 |  .904 |  .904 |  .952 |  .928 |  .904 |  .547 |  .744
SwinTrans. |  .880 |  .880 |  .952 |  .880 |  .928 |  .928 |  .952 |  .904 |  .992 |  .928 |  .952 |  .928 |  .904 |  .904 |  .928 |  .761 |  .822
LSH |  DreamSim |  .809 |  .880 |  .833 |  .809 |  .833 |  .857 |  .904 |  .883 |  .880 |  .880 |  .880 |  .880 |  .857 |  .857 |  .857 |  .547 |  .725
DINOv1 |  .809 |  .857 |  .880 |  .904 |  .880 |  .880 |  .880 |  .904 |  .857 |  .880 |  .857 |  .880 |  .904 |  .857 |  .904 |  .690 |  .768
DINOv2 |  .738 |  .833 |  .833 |  .833 |  .833 |  .857 |  .857 |  .880 |  .833 |  .833 |  .833 |  .833 |  .880 |  .833 |  .857 |  .5 |  .720
ResNet50 |  .857 |  .857 |  .857 |  .857 |  .904 |  .904 |  .976 |  .976 |  .928 |  .904 |  .928 |  .928 |  .976 |  .928 |  .976 |  .547 |  .741
|  SwinTrans. |  .880 |  .833 |  .928 |  .904 |  .857 |  .880 |  .857 |  .880 |  .833 |  .904 |  .904 |  .880 |  .904 |  .857 |  .880 |  .761 |  .803
Head |  HNSW |  DreamSim |  1.0 |  1.0 |  1.0 |  1.0 |  1.0 |  1.0 |  1.0 |  1.0 |  1.0 |  1.0 |  1.0 |  1.0 |  1.0 |  1.0 |  1.0 |  1.0 |  1.0
DINOv1 |  1.0 |  1.0 |  1.0 |  1.0 |  1.0 |  1.0 |  1.0 |  1.0 |  1.0 |  1.0 |  1.0 |  1.0 |  1.0 |  1.0 |  1.0 |  1.0 |  1.0
DINOv2 |  1.0 |  1.0 |  1.0 |  1.0 |  1.0 |  1.0 |  1.0 |  1.0 |  .997 |  1.0 |  1.0 |  1.0 |  1.0 |  .997 |  1.0 |  1.0 |  .99
ResNet50 |  1.0 |  1.0 |  1.0 |  1.0 |  1.0 |  1.0 |  1.0 |  1.0 |  1.0 |  1.0 |  1.0 |  1.0 |  1.0 |  1.0 |  1.0 |  1.0 |  1.0
SwinTrans. |  1.0 |  1.0 |  1.0 |  1.0 |  1.0 |  1.0 |  1.0 |  1.0 |  1.0 |  1.0 |  1.0 |  1.0 |  1.0 |  1.0 |  1.0 |  1.0 |  1.0
LSH |  DreamSim |  1.0 |  1.0 |  1.0 |  1.0 |  1.0 |  1.0 |  1.0 |  1.0 |  1.0 |  1.0 |  1.0 |  1.0 |  1.0 |  1.0 |  1.0 |  1.0 |  1.0
DINOv1 |  1.0 |  1.0 |  1.0 |  1.0 |  1.0 |  1.0 |  1.0 |  1.0 |  1.0 |  1.0 |  1.0 |  1.0 |  1.0 |  1.0 |  1.0 |  1.0 |  .99
DINOv2 |  1.0 |  1.0 |  1.0 |  1.0 |  1.0 |  1.0 |  1.0 |  1.0 |  1.0 |  1.0 |  1.0 |  1.0 |  1.0 |  1.0 |  1.0 |  1.0 |  .99
ResNet50 |  1.0 |  1.0 |  1.0 |  1.0 |  1.0 |  1.0 |  1.0 |  1.0 |  1.0 |  1.0 |  1.0 |  1.0 |  1.0 |  1.0 |  1.0 |  1.0 |  .999
SwinTrans. |  1.0 |  1.0 |  1.0 |  1.0 |  1.0 |  1.0 |  1.0 |  1.0 |  1.0 |  1.0 |  1.0 |  1.0 |  1.0 |  1.0 |  1.0 |  1.0 |  .999
Pelvis |  HNSW |  DreamSim |  1.0 |  1.0 |  1.0 |  1.0 |  1.0 |  1.0 |  1.0 |  1.0 |  1.0 |  1.0 |  1.0 |  1.0 |  1.0 |  1.0 |  1.0 |  .937 |  .977
DINOv1 |  .866 |  .75 |  .812 |  .769 |  .687 |  .812 |  .812 |  .812 |  .714 |  .875 |  .928 |  .875 |  1.0 |  1.0 |  .733 |  1.0 |  .846
DINOv2 |  1.0 |  1.0 |  1.0 |  1.0 |  1.0 |  1.0 |  1.0 |  1.0 |  1.0 |  1.0 |  1.0 |  1.0 |  1.0 |  1.0 |  1.0 |  1.0 |  .980
ResNet50 |  1.0 |  1.0 |  1.0 |  1.0 |  1.0 |  1.0 |  1.0 |  1.0 |  1.0 |  1.0 |  1.0 |  1.0 |  1.0 |  1.0 |  1.0 |  1.0 |  .979
SwinTrans. |  1.0 |  1.0 |  1.0 |  1.0 |  1.0 |  1.0 |  1.0 |  1.0 |  1.0 |  1.0 |  1.0 |  1.0 |  1.0 |  1.0 |  1.0 |  1.0 |  .984
LSH |  DreamSim |  1.0 |  1.0 |  1.0 |  1.0 |  1.0 |  1.0 |  1.0 |  1.0 |  1.0 |  1.0 |  1.0 |  1.0 |  1.0 |  1.0 |  1.0 |  1.0 |  .986
DINOv1 |  1.0 |  1.0 |  1.0 |  1.0 |  1.0 |  1.0 |  1.0 |  1.0 |  1.0 |  1.0 |  1.0 |  1.0 |  1.0 |  1.0 |  1.0 |  1.0 |  .986
DINOv2 |  1.0 |  1.0 |  1.0 |  1.0 |  1.0 |  1.0 |  1.0 |  1.0 |  1.0 |  1.0 |  1.0 |  1.0 |  1.0 |  1.0 |  1.0 |  1.0 |  .986
ResNet50 |  1.0 |  1.0 |  1.0 |  1.0 |  1.0 |  1.0 |  1.0 |  1.0 |  1.0 |  1.0 |  1.0 |  1.0 |  1.0 |  1.0 |  1.0 |  1.0 |  .986
SwinTrans. |  1.0 |  1.0 |  1.0 |  1.0 |  1.0 |  1.0 |  1.0 |  1.0 |  1.0 |  1.0 |  1.0 |  1.0 |  1.0 |  1.0 |  1.0 |  1.0 |  .986
Overall average |  .945 |  .951 |  .952 |  .953 |  .957 |  .963 |  .967 |  .966 |  .961 |  .962 |  .961 |  .965 |  .975 |  .967 |  .964 |  .89 |  .986

###  II-D Aggregation Strategies

The proposed CBIR system relies on pre-trained 2D image embeddings for 3D
image retrieval. Given a query case, first 2D slice embeddings are extracted.
We propose and benchmark different variants to aggregate 2D information to
volume level in order to find the best matching image volume.

##### All Slices

For every 2D slice of a query volume, the most similar slice in the database
is retrieved using embedding similarity, and the corresponding volume-ID is
stored in a hit table. The volume-ID with the most accumulated hits is then
returned as the most similar case in the database, see Fig. 1.

##### Weighted Slices

Since often the border slices of a volume carry less information compared to
center slices, a Gaussian weighting scheme can be used to increase the impact
of center slices and mitigate the impact of less informative slices. For each
slice of the query volume the most similar slice is retrieved from the
database, and the corresponding volume-ID is recorded. Then the Gaussian
weighting is applied to the matched volume-IDs (meaning, the volume-IDs that
are matched for center slices are repeated more than the further matches,
thus, their hit score increases). The final result is determined by selecting
the volume-ID with the highest weighted hit, see Fig. 1.

##### Sampled Slices

Volumetric medical images contain a high degree of spatial redundancy. Hence,
subsampling of transversal slices can improve the speed of the search without
impeding the retrieval performance. For all subsampling strategies, the final
result is again obtained based on maximum hit accumulation. The following
sampling schemes are compared:

  * •

Random subset: For each volume we randomly choose $5,10,20,40$, or $80$ slices
with a uniform distribution.

  * •

Equidistant: We sample a subset of equidistant transversal slices with a gap
size of $3,5,7,10,12$ or $15$ $mm$.

  * •

Fixed steps: the slices are sampled regardless of the spacing of the volume.
In this setup, every $3,5,$ and $10$ slices are sampled.

##  III Results and Discussions

Here we only report the recall rates, for the precision analysis please refer
to Appendix A. The performance analysis is conducted on modality, body region,
and organ level, i.e. a case is considered as a True Positive if modality,
body region, or organ label of the query and the most similar case match. As a
baseline measure, we report the recall of _slice-wise (SW)_ matching, i.e.
each slice from the query is treated as an individual instance and matched
against the most similar slice without aggregation to volume level and
performance is evaluated only on slice-wise level.

###  III-A Modality Level

On the modality level, for all possible combinations of the search index,
embedding, and aggregation scheme a recall of $1.0$ was obtained. The slice-
wise baseline has a recall of $0.999$ for all the tested variants. Here we
observed only a few mismatched black CT and MRI background slices.

###  III-B Body Region Level

The recall values on the body region level are shown in TABLE II for all
different combinations of search index, embedding, and aggregation scheme.

####  III-B1 All Slices

The Head and Pelvis regions of all the models and indexing approaches have a
perfect recall of $1.0$, except for the DINOv1 which shows a dropped
performance using HNSW indexing.

The retrieval of the Chest and Abdomen region is challenging due to the
presence of mixed slices of the Abdomen and Chest regions in the volumes. For
the retrieval of the Abdomen region DINOv1 has the perfect recall regardless
of the indexing. Swin Transformer has a slightly lower recall of $0.997$.
DreamSim and DINOv2 perform on par for the Abdomen with a recall of $0.993$.
For Chest, the best recall belongs to ResNet50 and LSH indexing with a recall
of $0.976$. Regardless of the indexing, DINOv1 is the second best with a
recall of $0.904$. DreamSim performs on par using HNSW but its performance
drops when LSH indexing is used.

####  III-B2 Weighted Slices

A perfect recall of 1.0 for the Head and Pelvis regions for all models (except
for DreamSim, HNSW) is achieved. The introduction of Gaussian weighting
significantly enhances the performance of all models in the Abdomen region,
but it comes at the cost of reduced recall in the Chest region. This trade-off
is due to the CT volumes often containing overlapping regions. Gaussian
weighting accentuates the center slices in Abdomen volumes, reducing the
chance of misclassification as Chest cases, even though lung-containing slices
are still present. However, for the Chest region, where the lung slices are
farther from the center slices, increasing the weight of the center slices
raises the likelihood of misclassification as Abdominal volumes.

####  III-B3 Sampled Slices

Except for DINOv1 and HNSW indexing, the sampling scenarios perform in most
cases on par with using all the slices which is expected due to the spatial
redundancy of the medical images. Especially for tasks that are distinct
enough, i.e., Head and Pelvis using even a few slices of the volumes can
result in perfect retrieval. Considering all the body regions, the DINOv1, LSH
indexing and sampling every $7$ $mm$ based on the spatial resolution has the
best overall performance with the recall of $1.0$ for Abodmen, Head, Pelvis,
and the recall of $0.904$ for the Chest region.

###  III-C Organ Level

In the interest of brevity, we provide the organ level analysis only for the
best method identified in the Section III-B, i.e. DINOv1 embeddings with LSH
search index and $7$mm sub-sampling. The associated confusion matrix on the
organ level is shown in Fig. 3. A comprehensive table can be found in Appendix
B. The recall values of this method are as follows: (Brain, Hippocampus,
Heart, Prostate: $1.0$), (Colon, $0.453$), (Hepatic vessel, $0.507$), (Liver,
$0.685$), (Lung, $0.875$), (Pancreas, $0.674$), and (Spleen, $0.1$). Fig. 3
shows that the organs belonging to the same body region, e.g. Abdomen are
mislabeled as another organ of the Abdomen which is in principle interpretable
because the organ level labels are derived from the MSD tasks and often all or
more than one of the abdominal organs are visible even on the single slice
which justifies for the low recall of the 5 abdomen organs. Similarly, CT
scans of abdominal and lung organs encompass more than just the specific organ
of focus; sometimes, abdominal slices are present in lung CTs, and vice versa,
leading to the occasional mislabeling of lung volumes as abdominal organs or
abdominal organs as lung volumes. This is considered as label noise that
should be taken into account when analyzing the results.

In order to better compare the performance of the different embedding types,
we aggregate all recall values obtained from varying the search index and the
slice aggregation scheme as discussed in Section II-D. Fig. 3 shows per
embedding type the median and the maximum recall of the aggregated recall
values separately for each organ. The detailed precision results at the organ
level were omitted because the values were very close and did not follow any
patterns, making their comparison uninformative.

Figure 2: Organ level confusion matrix of the best method based on body region
evaluation, colors represent the body regions: Head, Pelvis, Chest, and
Abdomen.

Figure 3: Organ level median (left) and maximum (right) of recall values
across all variants of search indices and the aggregation schemes.

##  IV Conclusion

We showed that without any retraining or fine-tuning pre-trained networks can
be used for medical retrieval tasks even if they are trained on non-medical
data for non-CBIR tasks. We could achieve perfect recall for modality and
multiple body regions. The proposed method uses embeddings from pretrained
models, computes per-slice embeddings only once, and combines the information
from different slices using various sampling and weighting schemes which
leverages the spatial redundancy in medical image volumes. The simple
weighting might not be suitable for some organs or body regions and based on
the information available on different target regions the weighting scheme
should be adjusted. However, the spatial redundancy of the medical images can
be leveraged by sampling the slices and therefore decreasing the size of the
search space. Simple heuristic rules, e.g., slice rejection based on intensity
values or height for specific organs can also be employed to improve the
method which can be considered in future studies. LSH and HNSW indexing
approaches perform on par on the tasks for different embeddings but since HNSW
is significantly faster we recommend using this method.

Using models trained on domain-specific medical images did not improve the
retrieval. But we hesitate to conclude any superiority. There are certain
limitations to point out. Firstly, the DINO-based models are in general larger
networks with more parameters. Secondly, the Swin Transformer and ResNet50 are
trained _supervised_ while the DINO-based models are all trained _self-
supervised_. Nevertheless, they perform on par. It is noteworthy that we used
MSD labels where for each volume only single organs are labeled, thus, the
organ-level performance does not include scenarios where multiple organs are
visible in the images which occurs frequently, especially in the Abdomen
region. Further studies should take this into account and more precise slice
labels should be derived from segmentation masks. A possible solution can be
using multiple labels per slice to compensate for the mentioned issue.

## Compliance with Ethical Standards

This research study was conducted retrospectively using human subject data
made available in open access by https://medicaldecathlon.com/. Ethical
approval was not required as confirmed by the license attached with the open-
access data.

## Acknowledgement

We thank Timothy Deyer and his RadImageNet team for providing the RadImagenet
pre-trained model weights for the Swin Transformer architecture.

## Conflicts of Interest

No funding was received for conducting this study. The authors have no
relevant financial or non-financial interests to disclose.

## References

  * [1] S. R. Dubey, “A decade survey of content based image retrieval using deep learning,” _IEEE Transactions on Circuits and Systems for Video Technology_ , vol. 32, no. 5, pp. 2687–2704, 2021.
  * [2] W. Wang, P. Jiao, H. Liu, X. Ma, and Z. Shang, “Two-stage content based image retrieval using sparse representation and feature fusion,” _Multimedia Tools and Applications_ , vol. 81, no. 12, pp. 16 621–16 644, 2022.
  * [3] N. F. Haq, M. Moradi, and Z. J. Wang, “A deep community based approach for large scale content based x-ray image retrieval,” _Medical Image Analysis_ , vol. 68, p. 101847, 2021.
  * [4] K. K. Wickstrøm, E. A. Østmo, K. Radiya, K. Ø. Mikalsen, M. C. Kampffmeyer, and R. Jenssen, “A clinically motivated self-supervised approach for content-based image retrieval of ct liver images,” _Computerized Medical Imaging and Graphics_ , vol. 107, p. 102239, 2023.
  * [5] O. Russakovsky, J. Deng, H. Su, J. Krause, S. Satheesh, S. Ma, Z. Huang, A. Karpathy, A. Khosla, M. Bernstein _et al._ , “Imagenet large scale visual recognition challenge,” _International journal of computer vision_ , vol. 115, pp. 211–252, 2015.
  * [6] T.-Y. Lin, M. Maire, S. Belongie, J. Hays, P. Perona, D. Ramanan, P. Dollár, and C. L. Zitnick, “Microsoft coco: Common objects in context,” in _Computer Vision–ECCV 2014: 13th European Conference, Zurich, Switzerland, September 6-12, 2014, Proceedings, Part V 13_. Springer, 2014, pp. 740–755.
  * [7] X. Mei, Z. Liu, P. M. Robson, B. Marinelli, M. Huang, A. Doshi, A. Jacobi, C. Cao, K. E. Link, T. Yang _et al._ , “Radimagenet: an open radiologic deep learning research dataset for effective transfer learning,” _Radiology: Artificial Intelligence_ , vol. 4, no. 5, p. e210315, 2022.
  * [8] T. Truong, S. Mohammadi, and M. Lenga, “How transferable are self-supervised features in medical image classification tasks?” in _Machine Learning for Health_. PMLR, 2021, pp. 54–74.
  * [9] A. Dosovitskiy, L. Beyer, A. Kolesnikov, D. Weissenborn, X. Zhai, T. Unterthiner, M. Dehghani, M. Minderer, G. Heigold, S. Gelly _et al._ , “An image is worth 16x16 words: Transformers for image recognition at scale,” _arXiv preprint arXiv:2010.11929_ , 2020.
  * [10] M. Caron, H. Touvron, I. Misra, H. Jégou, J. Mairal, P. Bojanowski, and A. Joulin, “Emerging properties in self-supervised vision transformers,” in _Proceedings of the IEEE/CVF international conference on computer vision_ , 2021, pp. 9650–9660.
  * [11] M. Oquab, T. Darcet, T. Moutakanni, H. Vo, M. Szafraniec, V. Khalidov, P. Fernandez, D. Haziza, F. Massa, A. El-Nouby _et al._ , “Dinov2: Learning robust visual features without supervision,” _arXiv preprint arXiv:2304.07193_ , 2023.
  * [12] S. Fu, N. Tamir, S. Sundaram, L. Chai, R. Zhang, T. Dekel, and P. Isola, “Dreamsim: Learning new dimensions of human visual similarity using synthetic data,” _arXiv preprint arXiv:2306.09344_ , 2023.
  * [13] Z. Liu, Y. Lin, Y. Cao, H. Hu, Y. Wei, Z. Zhang, S. Lin, and B. Guo, “Swin transformer: Hierarchical vision transformer using shifted windows,” in _Proceedings of the IEEE/CVF international conference on computer vision_ , 2021, pp. 10 012–10 022.
  * [14] K. He, X. Zhang, S. Ren, and J. Sun, “Deep residual learning for image recognition,” in _Proceedings of the IEEE conference on computer vision and pattern recognition_ , 2016, pp. 770–778.
  * [15] M. S. Charikar, “Similarity estimation techniques from rounding algorithms,” in _Proceedings of the thiry-fourth annual ACM symposium on Theory of computing_ , 2002, pp. 380–388.
  * [16] Y. A. Malkov and D. A. Yashunin, “Efficient and robust approximate nearest neighbor search using hierarchical navigable small world graphs,” _IEEE transactions on pattern analysis and machine intelligence_ , vol. 42, no. 4, pp. 824–836, 2018.
  * [17] J. Johnson, M. Douze, and H. Jégou, “Billion-scale similarity search with gpus,” _IEEE Transactions on Big Data_ , vol. 7, no. 3, pp. 535–547, 2019.
  * [18] K. He, H. Fan, Y. Wu, S. Xie, and R. Girshick, “Momentum contrast for unsupervised visual representation learning,” in _Proceedings of the IEEE/CVF conference on computer vision and pattern recognition_ , 2020, pp. 9729–9738.
  * [19] M. Antonelli, A. Reinke, S. Bakas, K. Farahani, A. Kopp-Schneider, B. A. Landman, G. Litjens, B. Menze, O. Ronneberger, R. M. Summers _et al._ , “The medical segmentation decathlon,” _Nature communications_ , vol. 13, no. 1, p. 4128, 2022.

## Appendix A

The precision values on the body region level are shown in TABLE III for all
different combinations of search index, embedding, and aggregation scheme.

###  IV-A Modality Level

Similar to the recall, on the modality level, for all possible combinations of
the search index, embedding, and aggregation scheme a precision of $1.0$ was
obtained.

###  IV-B Body Region Level

####  IV-B1 All Slices

The Head and Pelvis regions of all the models and indexing approaches have a
perfect precision of $1.0$, except for DINOv2 and LSH indexing which has a
slight drop to $0.99$. For the retrieval of the Abdomen region ResNet50 and
LSH indexing has the highest precision of $0.997$. Swin Transformer is the
second best with a precision of $0.993$. The DINO-based models have a slightly
lower precision in comparison ($[0.880-0.990]$). For Chest, however,
regardless of the indexing, DINOv1 performs the best with the precision of
$1.0$.

####  IV-B2 Weighted Slices

A perfect precision of $1.0$ for the Head and Pelvis regions for all models is
achieved. For the Abdomen region, the introduction of weighting decreased the
precision of all models except for DINOv1 which achieved a precision of $1.0$.
On the other hand, for the Chest region, the precision of all models except
for ResNet50 increased.

The increase and decrease of the precision values contrary to the drop of
recall values are in a small range and for all the body regions all the models
still have a precision higher than $0.9$.

####  IV-B3 Sampled Slices

Similar to the recall trends, subsampled slices in most cases perform on par
with using all the slices in terms of their precision.

TABLE III: Precision for different body regions Abdomen, Chest, Head, and Pelvis. Comparison: different vector database indexes (Id.), sampling strategies, aggregation variants all, per Image-id level and slice wise (SW) |  Id. |  Embedding | # Random Samples | # Sampled every x mm | # every n slices |  all |  W |  SW
---|---|---|---|---|---|---|---|---
5 |  10 |  20 |  40 |  80 |  3 |  5 |  7 |  10 |  12 |  15 |  3 |  5 |  10
Abdomen |  HNSW |  DreamSim |  .986 |  .986 |  .981 |  .986 |  .986 |  .990 |  .986 |  .983 |  .986 |  .986 |  .979 |  .990 |  .988 |  .984 |  .990 |  .960 |  .956
DINOv1 |  .986 |  .988 |  .988 |  .988 |  .988 |  .990 |  .990 |  .993 |  .993 |  .988 |  .986 |  .990 |  .990 |  .988 |  .990 |  .970 |  .967
DINOv2 |  .981 |  .983 |  .988 |  .986 |  .988 |  .986 |  .988 |  .986 |  .988 |  .983 |  .983 |  .986 |  .988 |  .988 |  .988 |  .964 |  ..956
ResNet50 |  .988 |  .988 |  .988 |  .993 |  .993 |  .990 |  .993 |  .990 |  .993 |  .990 |  .990 |  .990 |  .995 |  .993 |  .990 |  .972 |  .956
SwinTrans. |  .988 |  .988 |  .995 |  .988 |  .933 |  .993 |  .995 |  .990 |  .995 |  .993 |  .995 |  .993 |  .990 |  .990 |  .993 |  .977 |  .969
LSH |  DreamSim |  .981 |  .988 |  .983 |  .981 |  .984 |  .986 |  .990 |  .983 |  .988 |  .988 |  .988 |  .990 |  .986 |  .986 |  .986 |  .957 |  .954
DINOv1 |  .981 |  .986 |  .988 |  .990 |  .988 |  .988 |  .988 |  .990 |  .986 |  .988 |  .986 |  .988 |  .990 |  .986 |  .990 |  1.0 |  .961
DINOv2 |  .974 |  .984 |  .983 |  .983 |  .983 |  .923 |  .923 |  .925 |  .921 |  .945 |  .945 |  .984 |  .988 |  .984 |  .986 |  .953 |  .952
ResNet50 |  .986 |  .986 |  .986 |  .986 |  .990 |  .990 |  .997 |  .993 |  .993 |  .990 |  .993 |  .993 |  .997 |  .993 |  .997 |  .966 |  .956
SwinTrans. |  .988 |  .983 |  .993 |  .990 |  .986 |  .988 |  .986 |  .988 |  .984 |  .990 |  .990 |  .988 |  .990 |  .986 |  .988 |  .977 |  .966
Chest |  HNSW |  DreamSim |  .923 |  .9 |  .944 |  .878 |  .972 |  .926 |  .923 |  .921 |  .923 |  .923 |  .942 |  .990 |  .948 |  .945 |  .928 |  1.0 |  .882
DINOv1 |  .947 |  1.0 |  1.0 |  1.0 |  .948 |  .990 |  .993 |  .990 |  .993 |  .988 |  .986 |  .994 |  1.0 |  .973 |  1.0 |  1.0 |  .920
DINOv2 |  .871 |  .897 |  .973 |  .923 |  .948 |  .923 |  .925 |  .923 |  .925 |  .921 |  .897 |  .923 |  .948 |  .948 |  .925 |  1.0 |  .859
ResNet50 |  .840 |  .902 |  .925 |  .906 |  .928 |  .926 |  .928 |  .926 |  .928 |  .926 |  .926 |  .926 |  .930 |  .951 |  .926 |  .967 |  .859
SwinTrans. |  .940 |  .948 |  .930 |  .948 |  .928 |  .951 |  .930 |  .95 |  .952 |  .928 |  .975 |  .993 |  .990 |  .990 |  .975 |  .969 |  .884
LSH |  DreamSim |  .918 |  .948 |  .897 |  .918 |  .945 |  .947 |  .95 |  .921 |  .973 |  .973 |  .902 |  .925 |  .947 |  .947 |  .923 |  .958 |  .873
DINOv1 |  .944 |  1.0 |  .973 |  .95 |  1.0 |  1.0 |  1.0 |  1.0 |  1.0 |  1.0 |  1.0 |  1.0 |  1.0 |  1.0 |  1.0 |  1.0 |  .909
DINOv2 |  .861 |  .945 |  .921 |  .875 |  .921 |  .923 |  .923 |  .925 |  .921 |  .945 |  .945 |  .945 |  .948 |  .945 |  .923 |  1.0 |  .828
ResNet50 |  .851 |  .837 |  .857 |  .9 |  .883 |  .926 |  .931 |  .931 |  .975 |  .95 |  .975 |  .951 |  1.0 |  .886 |  .966 |  .931 |  .828
|  SwinTrans. |  .973 |  .853 |  .951 |  .95 |  .972 |  .973 |  .947 |  .948 |  .945 |  .95 |  .926 |  .948 |  .974 |  .972 |  .973 |  .969 |  .890
Head |  HNSW |  DreamSim |  1.0 |  1.0 |  1.0 |  1.0 |  1.0 |  .992 |  .992 |  .992 |  .985 |  .994 |  .992 |  1.0 |  1.0 |  1.0 |  1.0 |  1.0 |  .999
DINOv1 |  .992 |  .99 |  .992 |  .985 |  .987 |  1.0 |  .994 |  1.0 |  1.0 |  1.0 |  1.0 |  1.0 |  1.0 |  1.0 |  1.0 |  1.0 |  .999
DINOv2 |  1.0 |  1.0 |  1.0 |  1.0 |  1.0 |  1.0 |  1.0 |  1.0 |  .997 |  1.0 |  1.0 |  1.0 |  1.0 |  .997 |  1.0 |  1.0 |  .999
ResNet50 |  .997 |  .997 |  .997 |  .997 |  .997 |  .997 |  .997 |  .997 |  .997 |  .997 |  .994 |  .993 |  .997 |  .997 |  1.0 |  1.0 |  .999
SwinTrans. |  1.0 |  1.0 |  1.0 |  1.0 |  1.0 |  1.0 |  1.0 |  1.0 |  1.0 |  1.0 |  1.0 |  1.0 |  1.0 |  1.0 |  1.0 |  1.0 |  .999
LSH |  DreamSim |  1.0 |  1.0 |  1.0 |  1.0 |  1.0 |  1.0 |  1.0 |  1.0 |  1.0 |  1.0 |  1.0 |  1.0 |  1.0 |  1.0 |  1.0 |  1.0 |  .999
DINOv1 |  1.0 |  1.0 |  1.0 |  1.0 |  1.0 |  1.0 |  1.0 |  1.0 |  1.0 |  1.0 |  1.0 |  1.0 |  1.0 |  1.0 |  1.0 |  1.0 |  .999
DINOv2 |  1.0 |  1.0 |  1.0 |  1.0 |  1.0 |  1.0 |  1.0 |  1.0 |  1.0 |  1.0 |  1.0 |  1.0 |  1.0 |  1.0 |  .99 |  1.0 |  .999
ResNet50 |  .997 |  .997 |  .997 |  .997 |  .883 |  .997 |  .997 |  1.0 |  1.0 |  1.0 |  .997 |  .997 |  .997 |  .997 |  1.0 |  1.0 |  .997
SwinTrans. |  1.0 |  1.0 |  1.0 |  1.0 |  1.0 |  1.0 |  1.0 |  1.0 |  1.0 |  1.0 |  1.0 |  1.0 |  1.0 |  1.0 |  1.0 |  1.0 |  .999
Pelvis |  HNSW |  DreamSim |  1.0 |  1.0 |  1.0 |  1.0 |  1.0 |  1.0 |  1.0 |  1.0 |  1.0 |  1.0 |  1.0 |  1.0 |  1.0 |  1.0 |  1.0 |  1.0 |  1.0
DINOv1 |  .866 |  .75 |  .812 |  .769 |  .687 |  .812 |  .812 |  .812 |  .714 |  .875 |  .928 |  .875 |  1.0 |  1.0 |  1.0 |  1.0 |  .992
DINOv2 |  1.0 |  1.0 |  1.0 |  1.0 |  1.0 |  1.0 |  1.0 |  1.0 |  .941 |  1.0 |  1.0 |  1.0 |  1.0 |  .994 |  1.0 |  1.0 |  1.0
ResNet50 |  1.0 |  1.0 |  1.0 |  1.0 |  1.0 |  1.0 |  1.0 |  1.0 |  1.0 |  1.0 |  1.0 |  1.0 |  1.0 |  1.0 |  1.0 |  1.0 |  .996
SwinTrans. |  1.0 |  1.0 |  1.0 |  1.0 |  1.0 |  1.0 |  1.0 |  1.0 |  1.0 |  1.0 |  1.0 |  1.0 |  1.0 |  1.0 |  1.0 |  1.0 |  1.0
LSH |  DreamSim |  1.0 |  1.0 |  1.0 |  1.0 |  1.0 |  1.0 |  1.0 |  1.0 |  1.0 |  1.0 |  1.0 |  1.0 |  1.0 |  1.0 |  1.0 |  1.0 |  1.0
DINOv1 |  1.0 |  1.0 |  1.0 |  1.0 |  1.0 |  1.0 |  1.0 |  1.0 |  1.0 |  1.0 |  1.0 |  1.0 |  1.0 |  1.0 |  1.0 |  1.0 |  1.0
DINOv2 |  1.0 |  1.0 |  1.0 |  1.0 |  1.0 |  1.0 |  1.0 |  1.0 |  1.0 |  1.0 |  1.0 |  1.0 |  1.0 |  1.0 |  1.0 |  1.0 |  1.0
ResNet50 |  1.0 |  1.0 |  1.0 |  1.0 |  1.0 |  1.0 |  1.0 |  1.0 |  1.0 |  1.0 |  1.0 |  1.0 |  1.0 |  1.0 |  1.0 |  1.0 |  .952
SwinTrans. |  1.0 |  1.0 |  1.0 |  1.0 |  1.0 |  1.0 |  1.0 |  1.0 |  1.0 |  1.0 |  1.0 |  1.0 |  1.0 |  1.0 |  1.0 |  1.0 |  1.0
Overall average |  .968 |  .970 |  .976 |  .971 |  .970 |  .977 |  .976 |  .976 |  .974 |  .980 |  .980 |  .983 |  .989 |  .985 |  .985 |  .987 |  .956

## Appendix B

TABLE IV shows the detailed recall values at the organ level. The summarized
statistics were shown in Fig. 3.

TABLE IV: Recall for different organs. Comparison: different vector database indexes (Id.), sampling strategies, aggregation variants all, per Image-id level and slice wise (SW) |  Id. |  Embedding | # Random Samples | # Sampled every x mm | # every n slices |  all |  W |  SW
---|---|---|---|---|---|---|---|---
5 |  10 |  20 |  40 |  80 |  3 |  5 |  7 |  10 |  12 |  15 |  3 |  5 |  10
Brain |  HNSW |  DreamSim |  1.0 |  1.0 |  1.0 |  1.0 |  1.0 |  1.0 |  1.0 |  1.0 |  1.0 |  1.0 |  1.0 |  1.0 |  1.0 |  1.0 |  1.0 |  1.0 |  1.0
DINOv1 |  1.0 |  1.0 |  1.0 |  1.0 |  1.0 |  1.0 |  1.0 |  1.0 |  1.0 |  1.0 |  1.0 |  1.0 |  1.0 |  1.0 |  1.0 |  1.0 |  1.0
DINOv2 |  1.0 |  1.0 |  1.0 |  1.0 |  1.0 |  1.0 |  1.0 |  1.0 |  1.0 |  1.0 |  1.0 |  1.0 |  1.0 |  1.0 |  1.0 |  1.0 |  1.0
ResNet50 |  1.0 |  1.0 |  1.0 |  1.0 |  1.0 |  1.0 |  1.0 |  1.0 |  1.0 |  1.0 |  1.0 |  1.0 |  1.0 |  1.0 |  1.0 |  1.0 |  .999
SwinTrans. |  1.0 |  1.0 |  1.0 |  1.0 |  1.0 |  1.0 |  1.0 |  1.0 |  1.0 |  1.0 |  1.0 |  1.0 |  1.0 |  1.0 |  1.0 |  1.0 |  .999
LSH |  DreamSim |  1.0 |  1.0 |  1.0 |  1.0 |  1.0 |  1.0 |  1.0 |  1.0 |  1.0 |  1.0 |  1.0 |  1.0 |  1.0 |  1.0 |  1.0 |  1.0 |  1.0
DINOv1 |  1.0 |  1.0 |  1.0 |  1.0 |  1.0 |  1.0 |  1.0 |  1.0 |  1.0 |  1.0 |  1.0 |  1.0 |  1.0 |  1.0 |  1.0 |  1.0 |  0.99
DINOv2 |  1.0 |  1.0 |  1.0 |  1.0 |  1.0 |  1.0 |  1.0 |  1.0 |  1.0 |  1.0 |  1.0 |  1.0 |  1.0 |  1.0 |  1.0 |  1.0 |  0.99
ResNet50 |  1.0 |  1.0 |  1.0 |  1.0 |  1.0 |  1.0 |  1.0 |  1.0 |  1.0 |  1.0 |  1.0 |  1.0 |  1.0 |  1.0 |  1.0 |  1.0 |  .999
SwinTrans. |  1.0 |  1.0 |  1.0 |  1.0 |  1.0 |  1.0 |  1.0 |  1.0 |  1.0 |  1.0 |  1.0 |  1.0 |  1.0 |  1.0 |  1.0 |  1.0 |  1.0
Colon |  HNSW |  DreamSim |  .51 |  .42 |  .45 |  .49 |  .5 |  .5 |  .515 |  .515 |  .5 |  .515 |  .453 |  .453 |  .515 |  .484 |  .484 |  .704 |  .364
DINOv1 |  .5 |  .491 |  .531 |  .484 |  .5 |  .546 |  .546 |  .562 |  .546 |  .531 |  .562 |  .625 |  .5 |  .562 |  .564 |  .671 |  .424
DINOv2 |  .258 |  .421 |  .5 |  .5 |  .5 |  .542 |  .535 |  .507 |  .515 |  .531 |  .564 |  .531 |  .484 |  .564 |  .468 |  .590 |  .324
ResNet50 |  .370 |  .453 |  .419 |  .393 |  .475 |  .492 |  .492 |  .491 |  .492 |  .532 |  .476 |  .507 |  .523 |  .387 |  .484 |  .583 |  .294
SwinTrans. |  .327 |  .354 |  .373 |  .363 |  .430 |  .539 |  .539 |  .539 |  .451 |  .476 |  .507 |  .5 |  .571 |  .460 |  .53 |  .3 |  .332
LSH |  DreamSim |  .442 |  .421 |  .507 |  .421 |  .453 |  .437 |  .437 |  .437 |  .5 |  .515 |  .468 |  .484 |  .516 |  .546 |  .421 |  .754 |  .358
DINOv1 |  .606 |  .546 |  .468 |  .516 |  .5 |  .468 |  .453 |  .453 |  .453 |  .514 |  .5 |  .562 |  .451 |  .548 |  .468 |  .687 |  .396
DINOv2 |  .295 |  .359 |  .387 |  .5 |  .451 |  .483 |  .483 |  .438 |  .467 |  .451 |  .403 |  .578 |  .491 |  .419 |  .468 |  .612 |  .287
ResNet50 |  .265 |  .425 |  .453 |  .390 |  .437 |  .437 |  .437 |  .435 |  .515 |  .515 |  .468 |  .484 |  .571 |  .435 |  .437 |  .460 |  .304
SwinTrans. |  .326 |  .32 |  .466 |  .482 |  .442 |  .467 |  .451 |  .451 |  .532 |  .523 |  .523 |  .548 |  .539 |  .428 |  .453 |  .365 |  .316
Hepatic vessel |  HNSW |  DreamSim |  .421 |  .428 |  .5 |  .5 |  .51 |  .471 |  .471 |  .464 |  .471 |  .528 |  .528 |  .521 |  .457 |  .478 |  .5 |  .321 |  .422
DINOv1 |  .5 |  .621 |  .571 |  .557 |  .585 |  .592 |  .578 |  .578 |  .578 |  .571 |  .557 |  .592 |  .507 |  .578 |  .578 |  .471 |  .454
DINOv2 |  .492 |  .507 |  .557 |  .578 |  .492 |  .542 |  .535 |  .507 |  .514 |  .5 |  .442 |  .478 |  .435 |  .492 |  .542 |  .314 |  .421
ResNet50 |  .371 |  .414 |  .421 |  .435 |  .492 |  .471 |  .478 |  .457 |  .421 |  .385 |  .407 |  .414 |  .4 |  .364 |  .471 |  .292 |  .374
SwinTrans. |  .524 |  .57 |  .556 |  .566 |  .566 |  .407 |  .407 |  .392 |  .428 |  .407 |  .435 |  .421 |  .421 |  .464 |  .407 |  .589 |  .371
LSH |  DreamSim |  .442 |  .421 |  .507 |  .478 |  .507 |  .471 |  .492 |  .442 |  .485 |  .478 |  .528 |  .585 |  .478 |  .457 |  .514 |  .307 |  .423
DINOv1 |  .514 |  .507 |  .55 |  .55 |  .557 |  .535 |  .542 |  .55 |  .535 |  .514 |  .507 |  .571 |  .471 |  .542 |  .542 |  .464 |  .440
DINOv2 |  .464 |  .464 |  .507 |  .564 |  .542 |  .483 |  .483 |  .483 |  .467 |  .464 |  .507 |  .464 |  .457 |  .55 |  .507 |  .335 |  .418
ResNet50 |  .357 |  .4 |  .485 |  .45 |  .421 |  .442 |  .464 |  .414 |  .385 |  .357 |  .35 |  .407 |  .364 |  .414 |  .428 |  .242 |  .341
SwinTrans. |  .479 |  .512 |  .578 |  .539 |  .565 |  .414 |  .428 |  .4 |  .485 |  .485 |  .414 |  .428 |  .385 |  .442 |  .435 |  .606 |  .352
Hippocampus |  HNSW |  DreamSim |  1.0 |  1.0 |  1.0 |  1.0 |  1.0 |  1.0 |  1.0 |  1.0 |  1.0 |  1.0 |  1.0 |  1.0 |  1.0 |  1.0 |  1.0 |  1.0 |  1.0
DINOv1 |  .992 |  1.0 |  1.0 |  1.0 |  1.0 |  1.0 |  1.0 |  1.0 |  1.0 |  1.0 |  1.0 |  1.0 |  1.0 |  1.0 |  1.0 |  1.0 |  .998
DINOv2 |  1.0 |  1.0 |  1.0 |  1.0 |  1.0 |  1.0 |  1.0 |  1.0 |  .992 |  1.0 |  1.0 |  .992 |  1.0 |  1.0 |  1.0 |  1.0 |  .998
ResNet50 |  1.0 |  1.0 |  1.0 |  1.0 |  1.0 |  1.0 |  1.0 |  1.0 |  1.0 |  1.0 |  1.0 |  1.0 |  1.0 |  1.0 |  1.0 |  1.0 |  1.0
SwinTrans. |  1.0 |  1.0 |  1.0 |  1.0 |  1.0 |  1.0 |  1.0 |  1.0 |  1.0 |  1.0 |  1.0 |  1.0 |  1.0 |  1.0 |  1.0 |  1.0 |  1.0
LSH |  DreamSim |  1.0 |  1.0 |  1.0 |  1.0 |  1.0 |  1.0 |  1.0 |  1.0 |  1.0 |  1.0 |  1.0 |  1.0 |  1.0 |  1.0 |  1.0 |  1.0 |  1.0
DINOv1 |  1.0 |  1.0 |  1.0 |  1.0 |  1.0 |  1.0 |  1.0 |  1.0 |  1.0 |  1.0 |  1.0 |  1.0 |  1.0 |  1.0 |  1.0 |  1.0 |  1.0
DINOv2 |  1.0 |  1.0 |  1.0 |  1.0 |  1.0 |  1.0 |  1.0 |  1.0 |  1.0 |  1.0 |  1.0 |  1.0 |  1.0 |  1.0 |  1.0 |  1.0 |  1.0
ResNet50 |  1.0 |  1.0 |  1.0 |  1.0 |  1.0 |  1.0 |  1.0 |  1.0 |  1.0 |  1.0 |  1.0 |  1.0 |  1.0 |  1.0 |  1.0 |  1.0 |  1.0
|  SwinTrans. |  1.0 |  1.0 |  1.0 |  1.0 |  1.0 |  1.0 |  1.0 |  1.0 |  1.0 |  1.0 |  1.0 |  1.0 |  1.0 |  1.0 |  1.0 |  1.0 |  1.0
Heart |  HNSW |  DreamSim |  1.0 |  1.0 |  1.0 |  1.0 |  1.0 |  1.0 |  1.0 |  1.0 |  1.0 |  1.0 |  1.0 |  1.0 |  1.0 |  1.0 |  1.0 |  1.0 |  1.0
DINOv1 |  1.0 |  1.0 |  1.0 |  1.0 |  1.0 |  1.0 |  1.0 |  1.0 |  1.0 |  1.0 |  1.0 |  1.0 |  1.0 |  1.0 |  1.0 |  1.0 |  .993
DINOv2 |  1.0 |  1.0 |  1.0 |  1.0 |  1.0 |  1.0 |  1.0 |  1.0 |  1.0 |  1.0 |  1.0 |  1.0 |  1.0 |  1.0 |  1.0 |  1.0 |  1.0
ResNet50 |  1.0 |  1.0 |  1.0 |  1.0 |  1.0 |  1.0 |  1.0 |  1.0 |  1.0 |  1.0 |  1.0 |  1.0 |  1.0 |  1.0 |  1.0 |  .9 |  .985
SwinTrans. |  1.0 |  1.0 |  1.0 |  1.0 |  1.0 |  1.0 |  1.0 |  1.0 |  1.0 |  1.0 |  1.0 |  1.0 |  1.0 |  1.0 |  1.0 |  1.0 |  1.0
LSH |  DreamSim |  1.0 |  1.0 |  1.0 |  1.0 |  1.0 |  1.0 |  1.0 |  1.0 |  1.0 |  1.0 |  1.0 |  1.0 |  1.0 |  1.0 |  1.0 |  1.0 |  1.0
DINOv1 |  1.0 |  1.0 |  1.0 |  1.0 |  1.0 |  1.0 |  1.0 |  1.0 |  1.0 |  1.0 |  1.0 |  1.0 |  1.0 |  1.0 |  1.0 |  1.0 |  1.0
DINOv2 |  1.0 |  1.0 |  1.0 |  1.0 |  1.0 |  1.0 |  1.0 |  1.0 |  1.0 |  1.0 |  1.0 |  1.0 |  1.0 |  1.0 |  1.0 |  1.0 |  1.0
ResNet50 |  1.0 |  1.0 |  1.0 |  1.0 |  1.0 |  1.0 |  1.0 |  1.0 |  1.0 |  1.0 |  1.0 |  1.0 |  1.0 |  1.0 |  1.0 |  1.0 |  .978
SwinTrans. |  1.0 |  1.0 |  1.0 |  1.0 |  1.0 |  1.0 |  1.0 |  1.0 |  1.0 |  1.0 |  1.0 |  1.0 |  1.0 |  1.0 |  1.0 |  1.0 |  1.0
Liver |  HNSW |  DreamSim |  .671 |  .671 |  .614 |  .685 |  .714 |  .714 |  .714 |  .7 |  .728 |  .7 |  .771 |  .757 |  .685 |  .685 |  .728 |  .5 |  .658
DINOv1 |  .614 |  .628 |  .642 |  .742 |  .671 |  .7 |  .7 |  .685 |  .742 |  .714 |  .628 |  .685 |  .714 |  .671 |  .7 |  .471 |  .715
DINOv2 |  .614 |  .7 |  .685 |  .714 |  .728 |  .757 |  .742 |  .742 |  .728 |  .714 |  .757 |  .671 |  .728 |  .771 |  .742 |  .485 |  .674
ResNet50 |  .557 |  .585 |  .671 |  .7 |  .742 |  .757 |  .742 |  .785 |  .742 |  .714 |  .742 |  .714 |  .714 |  .771 |  .757 |  .557 |  .608
SwinTrans. |  .455 |  .560 |  .552 |  .556 |  .566 |  .757 |  .771 |  .771 |  .814 |  .757 |  .728 |  .742 |  .742 |  .685 |  .757 |  .380 |  .684
LSH |  DreamSim |  .628 |  .6 |  .685 |  .742 |  .685 |  .742 |  .757 |  .757 |  .742 |  .728 |  .742 |  .7 |  .714 |  .742 |  .742 |  .571 |  .661
DINOv1 |  .542 |  .714 |  .657 |  .7 |  .714 |  .714 |  .714 |  .685 |  .7 |  .685 |  .671 |  .7 |  .7 |  .685 |  .728 |  .514 |  .721
DINOv2 |  .614 |  .642 |  .771 |  .714 |  .757 |  .771 |  .757 |  .757 |  .728 |  .742 |  .671 |  .757 |  .8 |  .7 |  .771 |  .514 |  .677
ResNet50 |  .557 |  .614 |  .742 |  .742 |  .814 |  .8 |  .771 |  .814 |  .771 |  .771 |  .742 |  .742 |  .842 |  .771 |  .8 |  .585 |  .658
SwinTrans. |  .471 |  .536 |  .515 |  .563 |  .509 |  .8 |  .8 |  .771 |  .828 |  .842 |  .785 |  .8 |  .8 |  .7 |  .8 |  .381 |  .693
Lung |  HNSW |  DreamSim |  .812 |  .812 |  .75 |  .812 |  .812 |  .875 |  .812 |  .781 |  .8 |  .812 |  .718 |  .875 |  .843 |  .781 |  .875 |  .483 |  .700
DINOv1 |  .812 |  .843 |  .843 |  .843 |  .843 |  .875 |  .875 |  .906 |  .906 |  .843 |  .812 |  .875 |  .843 |  .875 |  .875 |  .612 |  .776
DINOv2 |  .75 |  .781 |  .843 |  .812 |  .843 |  .812 |  .843 |  .812 |  .843 |  .781 |  .781 |  .843 |  .843 |  .812 |  .843 |  .615 |  .705
ResNet50 |  .843 |  .843 |  .843 |  .906 |  .906 |  .875 |  .906 |  .875 |  .906 |  .875 |  .875 |  .875 |  .937 |  .906 |  .875 |  .625 |  .708
SwinTrans. |  .794 |  .931 |  .909 |  .931 |  .906 |  .906 |  .937 |  .875 |  .937 |  .906 |  .937 |  .906 |  .875 |  .875 |  .906 |  .956 |  .796
LSH |  DreamSim |  .75 |  .843 |  .781 |  .75 |  .781 |  .812 |  .875 |  .781 |  .843 |  .843 |  .843 |  .843 |  .812 |  .812 |  .812 |  .406 |  .685
DINOv1 |  .75 |  .838 |  .843 |  .875 |  .843 |  .843 |  .843 |  .875 |  .812 |  .843 |  .812 |  .875 |  .838 |  .843 |  .875 |  .612 |  .734
DINOv2 |  .656 |  .781 |  .781 |  .781 |  .781 |  .812 |  .812 |  .843 |  .781 |  .781 |  .781 |  .781 |  .843 |  .781 |  .812 |  .392 |  .679
ResNet50 |  .812 |  .812 |  .812 |  .812 |  .875 |  .875 |  .968 |  .968 |  .906 |  .875 |  .906 |  .906 |  .968 |  .906 |  .968 |  .566 |  .707
SwinTrans. |  .964 |  .806 |  .935 |  .933 |  .962 |  .834 |  .812 |  .843 |  .781 |  .875 |  .875 |  .843 |  .875 |  .812 |  .843 |  .916 |  .775
Pancreas |  HNSW |  DreamSim |  .647 |  .582 |  .611 |  .589 |  .604 |  .604 |  .597 |  .597 |  .580 |  .589 |  0625 |  .618 |  .633 |  .625 |  .604 |  .683 |  .582
DINOv1 |  .618 |  .604 |  .654 |  .633 |  .611 |  .654 |  .647 |  .647 |  .618 |  .625 |  .611 |  .611 |  .640 |  .633 |  .654 |  .741 |  .602
DINOv2 |  .604 |  .611 |  .589 |  .611 |  .561 |  .582 |  .589 |  .597 |  .625 |  .618 |  .539 |  .640 |  .611 |  .611 |  .582 |  .683 |  .557
ResNet50 |  .519 |  .489 |  .517 |  .611 |  .604 |  .575 |  .618 |  .625 |  .633 |  .618 |  .517 |  .589 |  .561 |  .496 |  .575 |  .496 |  .491
SwinTrans. |  .493 |  .556 |  .55 |  .592 |  .592 |  .503 |  .568 |  .561 |  .539 |  .568 |  .589 |  .532 |  .589 |  .604 |  .510 |  .492 |  .526
LSH |  DreamSim |  .517 |  .604 |  .568 |  .561 |  .561 |  .582 |  .561 |  .553 |  .539 |  .517 |  .633 |  .633 |  .611 |  .589 |  .546 |  .669 |  .553
DINOv1 |  .618 |  .589 |  .633 |  .683 |  .654 |  .640 |  .647 |  .647 |  .604 |  .615 |  .625 |  .654 |  .640 |  .654 |  .640 |  .726 |  .604
DINOv2 |  .525 |  .568 |  .611 |  .625 |  .597 |  .582 |  .589 |  .597 |  .618 |  .618 |  .625 |  .575 |  .654 |  .640 |  .582 |  .654 |  .555
ResNet50 |  .446 |  .424 |  .510 |  .525 |  .503 |  .478 |  .489 |  .489 |  .503 |  .503 |  .510 |  .582 |  .503 |  .453 |  .474 |  .460 |  .455
SwinTrans. |  .476 |  .578 |  .590 |  .551 |  .569 |  .510 |  .575 |  .575 |  .575 |  .575 |  .561 |  .525 |  .568 |  .568 |  .510 |  .530 |  .520
Prostate |  HNSW |  DreamSim |  1.0 |  1.0 |  1.0 |  1.0 |  1.0 |  1.0 |  1.0 |  1.0 |  1.0 |  1.0 |  1.0 |  1.0 |  1.0 |  1.0 |  1.0 |  .937 |  .977
DINOv1 |  .866 |  .75 |  .812 |  .769 |  .687 |  .812 |  .812 |  .812 |  .714 |  .875 |  .928 |  .875 |  1.0 |  1.0 |  .733 |  1.0 |  .846
DINOv2 |  1.0 |  1.0 |  1.0 |  1.0 |  1.0 |  1.0 |  1.0 |  1.0 |  1.0 |  1.0 |  1.0 |  1.0 |  1.0 |  1.0 |  1.0 |  1.0 |  .980
ResNet50 |  1.0 |  1.0 |  1.0 |  1.0 |  1.0 |  1.0 |  1.0 |  1.0 |  1.0 |  1.0 |  1.0 |  1.0 |  1.0 |  1.0 |  1.0 |  1.0 |  .979
SwinTrans. |  1.0 |  1.0 |  1.0 |  1.0 |  1.0 |  1.0 |  1.0 |  1.0 |  1.0 |  1.0 |  1.0 |  1.0 |  1.0 |  1.0 |  1.0 |  1.0 |
LSH |  DreamSim |  1.0 |  1.0 |  1.0 |  1.0 |  1.0 |  1.0 |  1.0 |  1.0 |  1.0 |  1.0 |  1.0 |  1.0 |  1.0 |  1.0 |  1.0 |  1.0 |  .986
DINOv1 |  1.0 |  1.0 |  1.0 |  1.0 |  1.0 |  1.0 |  1.0 |  1.0 |  1.0 |  1.0 |  1.0 |  1.0 |  1.0 |  1.0 |  1.0 |  1.0 |  .986
DINOv2 |  1.0 |  1.0 |  1.0 |  1.0 |  1.0 |  1.0 |  1.0 |  1.0 |  1.0 |  1.0 |  1.0 |  1.0 |  1.0 |  1.0 |  1.0 |  1.0 |  .986
ResNet50 |  1.0 |  1.0 |  1.0 |  1.0 |  1.0 |  1.0 |  1.0 |  1.0 |  1.0 |  1.0 |  1.0 |  1.0 |  1.0 |  1.0 |  1.0 |  1.0 |  .986
SwinTrans. |  1.0 |  1.0 |  1.0 |  1.0 |  1.0 |  1.0 |  1.0 |  1.0 |  1.0 |  1.0 |  1.0 |  1.0 |  1.0 |  1.0 |  1.0 |  1.0 |
Spleen |  HNSW |  DreamSim |  .1 |  0 |  .15 |  .05 |  .05 |  .1 |  .1 |  .1 |  .05 |  .05 |  .1 |  .1 |  .1 |  .1 |  .1 |  .1 |  .056
DINOv1 |  .1 |  .1 |  .05 |  .05 |  .05 |  .05 |  .05 |  .05 |  .15 |  .15 |  .1 |  .05 |  .05 |  .1 |  .05 |  .05 |  .047
DINOv2 |  .05 |  .05 |  .1 |  .05 |  .05 |  .05 |  .05 |  .05 |  .05 |  .05 |  .05 |  .15 |  .1 |  .05 |  .05 |  .05 |  .057
ResNet50 |  .05 |  .05 |  0 |  .05 |  .05 |  .05 |  .05 |  .05 |  .05 |  .05 |  .05 |  .05 |  0 |  .1 |  .05 |  .05 |  .049
SwinTrans. |  0 |  .060 |  .115 |  .035 |  .133 |  .1 |  .1 |  .1 |  .15 |  .15 |  .15 |  .15 |  .05 |  0 |  .1 |  0 |  .065
LSH |  DreamSim |  .05 |  0 |  .05 |  0 |  .1 |  .1 |  .1 |  .1 |  .1 |  .1 |  .05 |  .05 |  .05 |  .1 |  .15 |  .15 |  .057
DINOv1 |  .1 |  .05 |  0 |  .1 |  .1 |  .1 |  .1 |  .1 |  .05 |  .05 |  .15 |  0 |  .05 |  .15 |  .05 |  .05 |  .049
DINOv2 |  .05 |  0 |  .05 |  .05 |  .05 |  .05 |  .05 |  .05 |  0 |  0 |  .1 |  .05 |  .05 |  .1 |  0 |  0 |  .053
ResNet50 |  0 |  .15 |  .05 |  .05 |  0 |  0 |  0 |  0 |  .05 |  .05 |  .1 |  .1 |  .05 |  .1 |  0 |  .15 |  .049
SwinTrans. |  .083 |  .071 |  .105 |  .08 |  .103 |  .05 |  .05 |  .05 |  .1 |  .1 |  .1 |  .1 |  .1 |  .05 |  .05 |  .038 |  .653


# Modelling Solar Energetic Neutral Atoms from Solar Flares and CME-driven
Shocks

Gang Li Department of Space Science and CSPAR
University of Alabama in Huntsville
Huntsville, AL 35899, USA  Albert Y. Shih NASA Goddard Space Flight Center,
Greenbelt, MD 20771, USA  Robert C. Allen Johns Hopkins University Applied
Physics Laboratory, Laurel, MD, 20723, USA  George C. Ho Johns Hopkins
University Applied Physics Laboratory, Laurel, MD, 20723, USA  Christina M.S.
Cohen California Institute of Technology, Pasadena, CA, 91125, USA  Mihir
Desai Southwest Research Institute, San Antonio, TX, 78238, USA  Maher A.
Dayeh Southwest Research Institute, San Antonio, TX, 78238, USA  Glenn Mason
Johns Hopkins University Applied Physics Laboratory, Laurel, MD, 20723, USA

###### Abstract

We examine the production of energetic neutral atoms (ENAs) in solar flares
and CME-driven shocks and their subsequent propagation to 1 au. Time profiles
and fluence spectra of solar ENAs at 1 au are computed for two scenarios: 1)
ENAs are produced downstream at CME-driven shocks, and 2) ENAs are produced at
large-scale post-flare loops in solar flares. Both the time profiles and
fluence spectra for these two scenarios are vastly different. Our calculations
indicate that we can use solar ENAs as a new probe to examine the underlying
acceleration process of solar energetic particles (SEPs) and to differentiate
the two acceleration sites: large loops in solar flares and downstream of CME-
driven shocks, in large SEP events.

##  1 Introduction

Solar flares and coronal mass ejections (CMEs) are two of the most energetic
processes in the solar system. Efficient particle acceleration can occur in
both solar flares and at CME-driven shocks. Energetic protons accelerated at
either CME-driven shocks or solar flares can precipitate down to the Sun’s
surface or propagate into the interplanetary medium along open interplanetary
magnetic field (IMF) lines. During their propagation, they can interact with
ions and thermal neutral atoms in the solar atmosphere via charge exchange,
and produce energetic neutral hydrogen atoms. Once produced, energetic neutral
hydrogen atoms (hereafter referred as ENAs) do not feel solar magnetic field
and propagate along straight lines. They are subject to loss processes wherein
they lose the electron and become an energetic proton again. Because the
density of the solar wind drops quickly with the heliocentric distance, and
because the loss rate of ENAs is proportional to the solar wind density, ENAs
reaching 20 Rs suffer no further loss. Since the IMF does not affect the
propagation of ENA hydrogen, these ENAs therefore provide a powerful avenue in
probing the acceleration processes and plasma properties of the underlying
acceleration site.

Because the production cross section is small, the flux of ENAs at a distance
of 1 au from the Sun can be extremely small. To date, only a few observational
clues of ENAs accompanying SEP events were reported (Mewaldt et al., 2009;
Mason et al., 2021). Mewaldt et al. (2009) reported $1.6$ to $5$ MeV energetic
neutral atoms (ENAs) from STEREO-A/B observations. They inferred a power-law
spectrum of $dJ/dE\sim E^{-2.46}$ accompanying an X9-class solar flare and
suggested that these ENAs are produced via charge exchange of SEP protons with
O6+ ions. Following (Mewaldt et al., 2009), Wang et al. (2014) performed a
simulation and showed that sufficient counts of ENAs are expected for typical
gradual SEP events where particles are accelerated at CME-driven shocks. This
stimulated interests in observational effort. More recently Mason et al.
(2021) examined $18$ SEP events with SAMPEX, and found indirect, but
compelling evidence of solar ENAs near the geomagnetic equator at low
altitudes where the geomagnetic field filters out all charged SEPs. This new
insight also shed light on three previously reported puzzling $\sim$ MeV ion
intensity increases that were also observed near the equatorial regions about
$\sim 3$ hrs after the occurrence of the corresponding X-ray flares (Greenspan
et al., 1999). The discovery of ENAs by STEREO, and confirmation from SAMPEX,
shows that solar ENAs can be expected to accompany many large SEP events.

If ENAs can be detected in SEP events, one of the pressing questions would be
where do they originate. Are they accelerated at a rather confined
reconnection site at flares or at a broader shock front driven by CMEs? To
answer such a question, we examine ENA productions in two different scenarios:
CME-driven shock and large post flare loops in this work. A schematic of the
two acceleration sites are shown in Figure 1. Note that in many large SEP
events, CMEs and flares often occur together. However, the spatial extension
of the flare is much smaller than the CME. Ions can be efficiently accelerated
at both the flare site and the CME-driven shock front. In the case of CME-
driven shocks, protons and ions are accelerated at the shock front via the
first order Fermi acceleration mechanism. Once accelerated, they can escape
upstream propagating along IMF, or trapped downstream for an extended period
of time. They may precipitate down to the solar surface, causing, for example,
long duration gamma-ray events (Share et al., 2018; Jin et al., 2018). In the
case of flares, particles can be accelerated at the reconnection exhausts and
in solar flare loops (Petrosian, 2012; Ryan, 2000) by e.g. the second order
Fermi acceleration mechanism. Continued magnetic reconnection can lead to a
rising of the post-flare loops (West & Seaton, 2015). Accelerated particles
may be trapped in post-flare loops for very long period of time, serving as an
alternative candidate for the long-duration gamma ray events (Ryan, 2000; de
Nolfo et al., 2019).

We note that in large SEP events it is possible that flare accelerated
particles can be re-accelerated at the accompanying CME-driven shocks (Li &
Zank, 2005; Petrosian, 2012). Simulations by Li & Zank (2005) showed that
depending on if the observer is magnetically connected to the flare and/or the
shock surface, the characteristics of the ion time profiles differ and may
show two peaks as reported in (Cane et al., 2003). However, because the
presence of solar wind MHD turbulence can affect the propagation of charged
ions, the interpretation of 1-au ion observations is often complicated. ENAs,
with ballistic propagation, do not follow IMF and are not affected by the
solar wind MHD turbulence. Therefore, ENA observations with enough angular
resolution can clearly distinguish ENAs from flare sites and those from much
broader CME-driven shocks.

Figure 1: Schematic cartoon showing the two acceleration sites of ions in
large SEP events: CME-driven shocks and flares. Once produced, energetic ions
can propagate along open IMFs and be detected in-situ at 1 au. Near the
acceleration sites, the density of solar atmosphere is high enough so that
energetic ions can lead to the production of solar ENAs. The characteristics
of solar ENAs in both scenarios are examined in this work.

We examine solar ENA production from CME-dirven shocks in section 2 and from
solar flare loops in section 3. Production and loss processes of ENAs are
discussed in Appendix A.

##  2 ENAs from CME-driven shocks

In this section, we consider the observation of ENA particles generated at a
propagating CME-driven shock. The first ENA simulation was done by Wang et al.
(2014) who simulated a CME-driven shock from a side-on orientation and
suggested that the observed flux in (Mewaldt et al., 2009) is consistent with
ENA production at a CME-driven shock. More recently, following the work of
(Wang et al., 2014), Wang et al. (2022) examined a variety cases with
different CME speeds, open angles, and CME propagation directions. They also
examined the effect of solar wind density variation near the Sun on the
production of ENAs. These authors found similar results as Wang et al. (2014).

Here we reexamine the case considered in (Wang et al., 2014) and include
another two cases with different CME propagation directions to obtain an
estimate of ENA flux range at 1 au. Our treatment is similar to our previous
work (Wang et al., 2014) but with a few differences. As in (Wang et al.,
2014), we assume protons are accelerated at the shock and then distributed
uniformly downstream of the shock. This is based on the DSA mechanism and has
been adopted in our previous large SEP event simulations (Li et al., 2003,
2005, 2012b, 2021). Since the turbulence downstream of the shock is a lot
stronger than that upstream of the shock (see e.g. (Lee, 1983; Zank et al.,
2000; Li et al., 2003)), accelerated particles can be kept downstream of the
shock for a long period of time. In (Wang et al., 2014), we assumed there is
no leakage of accelerated particles from downstream of the shock. This was
mostly for simplicity since accelerated particles can precipitate back to the
sun along open field lines. Indeed, Jin et al. (2018) has explored the
possibility that the long duration gamma ray events are due to shock
acceleration protons. In such a scenario, accelerated protons downstream of
the shock can steadily precipitate to the solar surface. Therefore, in this
work, we include a decay of the accelerated protons downstream of the shock.
As an estimate of the decay time, we refer to Li et al. (2012a), who, from a
statistical study of twin-CME events, suggested that a decay time scale of the
turbulence in large SEP events is around $9$-$13$ hours. We use a decay time
$\tau=10$ hours in this work. We also set our inner boundary at $r=1.02R_{s}$,
which differs from that used in (Wang et al., 2014), $1.5R_{s}$. We further
improve the treatment of ENA propagation from downstream of the shock to the
observer. In (Wang et al., 2014), downstream medium was divided into shells
and ENAs produced in individual shells are assumed to propagate to the
observer all from the shell center. This is refined in our current work. We
now divide the downstream region of the shock into multiple parcels, as shown
in the left panel of Figure 2. ENAs are produced and followed in individual
parcels. Since ENAs in different parcels propagate to the observer along
different paths, our current treatment will lead to a more accurate survival
probability computation. Finally, a correction factor $cos(\theta)$ to the
flux expression, equation (4) in (Wang et al., 2014) is included, see equation
(A3).

Figure 2:  Left: Schematic plot showing the CME configuration for the base
case. Plasma downstream of the shock is divided into parcels. These parcels
are used to track the ENA production and propagation to the observer. The
observer is along the X axis at 1 au and the CME propagates along the Y axis.
Right: Another two cases, case II and case III are also considered. In case
II, the CME propagates toward the observer; and in case III, the CME
propagates $45^{\circ}$ off from the $+Y$ direction.

Figure 2 shows the configuration of the ENA production process for the CME
shock case. The left panel depicts the base case: the observer locates at 1 au
along the $X$ axis and the CME is propagating to the right along the $+Y$
direction, i.e., $\phi=90^{\circ}$ where $\phi$ is the angle between the sun-
observer line and the CME propagation direction. The plasma downstream of the
shock is divided into multiple parcels. ENA production is followed in these
parcels. ENAs produced in these parcels can propagate along straight lines to
the observer. These trajectories differ for different parcels, and lead to
different survival probabilities. Right panel of Figure 2 shows two other
cases with different CME propagation directions. In case II, the CME
propagates toward the observer with $\phi=0^{\circ}$. In case III, the CME
propagates $45^{\circ}$ off from the $+X$ and $+Y$ directions, i.e.
$\phi=45^{\circ}$. For our simulation, the shock has a constant speed of
$V_{sh}=1500$ km/s and a constant compression ratio of $s=3.5$. The open angle
of the shock is $60^{\circ}$ and the shock is followed up to $30R_{s}$. As in
the flare ENA case, we use the Leblanc model (Leblanc et al., 1998) to compute
the solar wind density.

Figure 3 plots the time profiles and the fluence of ENAs for the three cases
shown in Figure 2. The upper left, upper right, and lower left panels show the
time profiles for the base case, case II and case III, respectively. For all
three cases, ENAs of $11$ energies are considered. The three time profiles are
similar. Consider the base case (upper left panel). The x-axis is the time
after shock initiation, in unit of $10$ minutes; and the y-axis is the ENA
flux at the observer, in unit of #/(cm${}^{2}\cdot$ sec $\cdot$ keV). The
observer first see the $20$ MeV ENAs arriving $\sim 40$ minutes after the
shock initiation. The flux can reach $7*10^{-3}$ cm${}^{2}\cdot$ sec $\cdot$
keV. It then decreases, reflecting the fact that the density of energetic
protons decreases with time as the shock propagates out. As the ENA energy
become smaller, their first arrival times become later and the flux increases
with decreasing energy, till $E=0.75$ MeV. Below $E=0.75$ MeV the flux shows a
more plateau feature and drops slightly. This behaviour is due to the energy
dependence of the charge exchange cross sections that are responsible for the
ENA production. See Figure 6 in Appendix A. Comparing to the base case, cases
II and III are comparable and show larger fluxes than the base case. This is
easily understood from Figure 2 because the ENAs produced in these two cases
travel shorter distances and through less dense solar atmosphere to the
observer and consequently have larger survival probabilities. The lower right
panel of Figure 3 plots the fluence for these three cases. Note the relatively
plateau-like behavior below $E=0.75$ MeV, which is the consequence of the
energy dependence of the relevant charge exchange cross section. Above $1$
MeV, the ENA fluence spectrum shown here is comparable to that inferred in
Mewaldt et al. (2009). The general shape of the CME shock ENA fluence is
similar to the parent energetic ion spectrum which is a power law. This is in
stark contrast to the flare ENA case (see next section) where the ENA fluence
does not resemble the parent energetic ion spectra.

Figure 3: Upper left, upper right and lower left panels show time Profiles of
ENA hydrogens produced at CME-driven shocks for an observer at 1 au, for the
base case, case II and case III, respectively. Eleven energies are considered.
Shocks is followed up to $30R_{s}$. Lower Right panel is the fluence of the
ENA hydrogen for the time duration shown in the other three panels. Note the
bent-over at 1 MeV for the ENAs. The parent energetic protons has a power law
extended to $0.02$ MeV. The bent-over is due to the energy dependence of the
various charge cross sections shown in the Appendix A.

##  3 ENAs from solar flares

We examine ENA production by solar flares in this section. Both electrons and
ions are efficiently accelerated at solar flares, and the accelerated
electrons and ions lead to the emission of hard X-rays and gamma rays. It is
generally accepted that acceleration may occur at reconnection current sheets
and/or by turbulence in the flare loops. Observations of hard X-ray and gamma
rays suggest that the accelerated electron and ion spectra can be approximated
by a power law. Power law like spectra are supported by earlier theoretical
works by (Miller & Roberts, 1995; Petrosian, 2012), where ions are accelerated
in flare loops by MHD turbulence via second order Fermi acceleration. More
recent PIC simulations of ions in flare reconnection site also found a power
law spectrum (Zhang et al., 2021).

Figure 4: Cartoon showing the postflare loops in solar flares. Large scale and
high postflare loops are potential production site of solar ENAs. Upper image
is adopted from (West & Seaton, 2015).

Once accelerated, ions precipitate down to the solar surface along post-flare
loops. The density of a post-flare loop can be constrained by free–free
continuum emission for hot loops. In a recent work, Jejčič et al. (2018)
reported an electron density as high as $10^{13}$ cm-3, $10$ to $100$ higher
than that at typical flare loops. At a density of $\sim 10^{11}$ cm-3, ENAs
can be easily produced in these loops. Once produced, ENAs are not constrained
in the loops and can propagate in all directions. However, if the loops are
low, the solar atmosphere density in the surrounding environment can be too
dense to allow these ENAs to escape from the Sun. Therefore to observe flare
ENAs, the flare loops must be high. The height of flare loops can be estimated
from the looptop hard X-ray observations. A recent study of looptop hard X-ray
source of solar flares (Effenberger et al., 2017) showed that the height of a
typical flare ranges from $10$ to $50$ Mm. If ions are accelerated at and
below this height at flares, no ENAs can survive as they propagate out.
However, using the Sun Watcher with Active Pixels (SWAP) EUV imaging solar
telescope, West & Seaton (2015) examined an M2.2 flare which occurred on 2014
October 14 and found that the post-flare loops were long-lasting, and reached
a height of over $400$ Mm ($\geq 0.5\ {{R}_{\odot}}$) $\sim 48$ hours after
the eruption. West & Seaton (2015) argued that the giant arches in this event
are similar to ordinary post-flare loops and are the results of a long-lasting
($48$ hours) magnetic reconnection occurred along a large-scale current sheet
(Forbes & Lin, 2000). This continuous magnetic reconnection provides the
energy source to heat the loop and can accelerate particles. Besides magnetic
reconnection, turbulence inside the loop can also lead to stochastic
acceleration of ions (Ryan, 2000). We note that the magnetic reconnection at
the current sheet and the enhanced turbulence inside the large postflare loop
may be intimately related. In a recent work by Cheng et al. (2018), the
authors examined the 2017 09 10 flare and showed that a Kolmogorov-like
turbulence spectrum can develop in the current sheet above the flare loops.
Presence of such a turbulence implies that particles can be accelerated in the
turbulent current sheet in a similar way as in flare loops through a second
order Fermi acceleration process (Miller & Roberts, 1995; Petrosian, 2012).
The spatial extension of the current sheet is similar to the flare loops that
are beneath it (Cheng et al., 2018; French et al., 2019), but the density of
the current sheet is, however, smaller than the density in the postflare
loops. Indeed French et al. (2019) concludes that the density in the current
sheet is $\sim<10^{10}$/cm3. This is $100$ times smaller than the density
inferred in the lower flare loop as reported in (Jejčič et al., 2018), and is
$10$ times smaller than what we assume for the post flare loops,
$10^{11}$/cm3. The ENA production in these current sheets is therefore much
smaller than in postflare loops. So we only consider ENA producion in
postflare loops in this work. However, we remark that these turbulent current
sheets can be potential sites of ENA production, and if future ENA probes have
high enough sensitivities, it is possible to obtain direct observations of
these current sheets through ENA observations. A cartoon showing these post
flare loops are shown in Figure 4.

Continuous acceleration, as suggested by Ryan (2000), has been identified as a
possible scenario for the long duration gamma ray events de Nolfo et al.
(2019). Long duration gamma ray events are not uncommon. Recently Share et al.
(2018) examined $\sim 30$ long duration gamma ray events and found that the
energy spectral indices of $>300$ MeV proton producing gamma rays range from
$2.5$ to $6.5$, similar to typical flare events. In a recent study, de Nolfo
et al. (2019) compared the gamma-ray-producing proton numbers with the in-situ
SEP proton numbers in long duration gamma ray flares and found a poor
correlation. Their study supports the continuous acceleration in the post-
flare loop scenario, as suggested by Ryan (2000). We point out that the event
reported in (West & Seaton, 2015), despite having large post-flare loops, was
not a long duration gamma ray event. This is possible if particles are not
accelerated to high enough energies ($\sim 100$ MeV/nuc) to produce gamma
rays.

We now examine ENAs from post-flare loops. We model the post-flare loops as
semi-circle tubes. We assume that the loop has a height (radius) of $h(t)$,
which increases with time. We assume the starting height of the post flare
loop is $0.04R_{s}$ ($\sim 28$Mm), and a rising rate of $V_{r}=3$ km/s (West &
Seaton, 2015). This gives a height of $H=0.22$, $0.41$, and $0.60$ $R_{s}$
when $t=$ 12, 24, and 36 hours, respectively. The cross section of the tube
can be assumed to be a circle with a radius as $a$. One can take $a$ to be
$\sim 700$ km, which is comparable to the half width for a typical flare
ribbon. However, as we will see below, the ENA production depends on the total
number of accelerated protons and does not depend on the choice of $a$ and the
number of loops we consider.

We also assume a constant proton density inside the flare loop. By way of
example, we assume a loop density of $10^{11}$ cm-3. This is smaller than that
obtained in (Jejčič et al., 2018), but larger than the density at the solar
surface, which is $\sim 10^{9-10}$ cm-3. As a simplification, we assume the
acceleration process (Ryan, 2000) is time independent and the production rate
of energetic protons, $\alpha$, is a constant during the rising phase of the
post-flare loop. We denote the duration of the rising phase to be $T$, and the
total number of accelerated particle $N_{0}=\alpha T$. Once accelerated these
particles can precipitate to the solar surface. We model this as a loss
process with an energy-independent decay time $\tau$. The total number of
accelerated particles $N(t)$ in the loop is given by,

$\frac{dN(t)}{dt}=\frac{N_{0}}{T}\theta(T-t)-\frac{N(t)}{\tau}$ (1)

where $\theta(t)$ is the Heaviside function. The solution of equation (1) is,

$N(t)=N_{0}\frac{\tau}{T}\left[(1-e^{-t/\tau})*\theta(T-t)+(1-e^{-T/\tau})e^{-(t-T)/\tau}*\theta(t-T)\right].$
(2)

In equation (2), $N_{0}$ can be constrained from the following consideration.
In the long duration gamma ray events examined by (Share et al., 2018), the
authors inferred that accelerated particles at high energies ($>$300 MeV) in
the loops is about $0.01$ to $0.5$ of that of the accompanying SEP events,
presumably accelerated at the CME-driven shocks. Assuming this ratio is energy
independent, then one can estimate the range of $N_{0}$ from the CME-driven
shock case. Alternatively, one can estimate $N_{0}$ from an energy budget
point of view. In a study of the CME/Flare Energy Budget for two (Emslie et
al., 2005), and subsequently for $38$ large SEP events, Emslie et al. (2012)
found that the energy budget for $>1$ MeV flare ions can reach $\epsilon\sim
4*10^{31}$-$10^{32}$ erg, which can be comparable and even larger than those
observed in-situ. In this work, we estimate $N_{0}$ by assuming the total
energy for the accelerated particles ($>1$ MeV) is $\epsilon=10^{31}$ erg.
With a source spectrum of the accelerated protons given by,

$f(E,t)=\frac{N(t)(\gamma_{1}-1)}{E_{0}}\left[(\frac{E}{E_{0}})^{-\gamma_{1}}\theta(E_{b}-E)+(E_{b}/E_{0})^{-\gamma_{1}}(\frac{E}{E_{b}})^{-\gamma_{2}}\theta(E-E_{b})\right]\quad$
(3)

where $E_{0}$ is the injection energy, $E_{b}$ is the break energy,
$\gamma_{1}=2.5$ is the spectral index at energies below $E_{b}$ and
$\gamma_{2}=5.5$ is the spectral index at energies above $E_{b}$. This gives,

$N_{0}\approx(\frac{\gamma_{1}-2}{\gamma_{1}-1})\left(\frac{\epsilon
E_{0}^{1-\gamma_{1}}}{1-(E_{b})^{2-\gamma_{1}}}\right)$ (4)

For a choice of $E_{0}=0.02$ MeV, $E_{b}=30$ MeV, $\gamma_{1}=2.5$ and
$\gamma_{2}=5.5$, we find $N_{0}=9*10^{38}$. Equation (3), together with
equations (2) and (4) describe the energetic proton source, as a function of
time, for the ENAs inside the post-flare loop.

One can now compute the production of ENAs and obtain the time profiles and
fluence of ENAs as observed at 1 au. We consider three cases with $T=12$,
$24$, and $36$ hrs, corresponding to a final loop height of $H=0.22$, $0.41$,
and $0.60$ Rs, respectively. In all cases $\tau=3$ hr. We further assume that
the flare locates at $\phi=0$ degree, i.e. in a face-on situation. For other
viewing angles, the results are qualitatively similar.

Figure 5: Upper left, upper right, and lower left: time profiles of solar
flare ENAs for a loop with a final loop height $h=0.22R_{s}$, $0.41R_{s}$, and
$0.60R_{s}$, respectively. Lower Right: total ENA fluence for the three cases
considered. See text for details.

Figure 5 plots the time profiles and fluence of the flare ENAs. The upper
left, upper right and lower left panels are time profiles for the three
choices of the final flare loop heights. Seven energies are considered. These
are $1.0$, $1.5$, $2$, $5$, $10$, $15$, and $20$ MeVs. As can be seen from
these panels, high energy ENAs arrive earlier due to a short propagation time
from the Sun to 1 au. In all three panels, the peak of the time profiles occur
shortly after the loops reach the maximum height. The energy dependence of the
peak intensity (and the fluence, see the lower right panel) strongly depends
on the loop height. If the loop height is $0.22R_{s}$ (upper left panel), the
peak intensity of $T=2$ MeV ENAs is $5$ orders of magnitude smaller than that
of the $T=15$ MeV ENAs. Furthermore, there is no ENAs with $T<2$ MeV. In
comparison, when the loop height is $0.41$ or $0.6$ Rs, the peak intensity of
$T=2$ MeV ENAs is similar to that of the $T=15$ MeV ENAs. This energy
dependence can be also seen from the fluence plot shown in the lower right
panel of Figure 5. When the loop height is $0.60R_{s}$, the fluence has a
maximum $\sim 800$/cm2 at $T=2$ MeV, and at $T=20$ MeV, the fluence is about
$10$. When the loop height is $0.22R_{s}$, however, the fluence of 2 MeV ENAs
drop by a factor of $4*10^{8}$ to $\sim 2*10^{-6}$/cm2. In comparison, the
flunece of $20$ MeV ENA drops only by a factor of $50$, to $0.2$/cm2. This big
difference of ENA fluence at 1 au for different flare loop height is due to
efficient loss of ENA close to the Sun. Although plenty of ENAs are produced
in the flare loop, they can not escape the high density solar atmosphere if
the flare loop is not high enough. Note that during the eruption phase of
solar flares, the height of flare loops, as seen from X-ray imaging, is a lot
smaller than $0.22R_{s}$ (Effenberger et al., 2017), therefore we expect no
ENAs during the eruption phase of solar flares. However, large post flare
loops, as those reported in (West & Seaton, 2015), can reach $0.5R_{s}$. Our
calculations show that there will be clear ENA signals from such a flare. We
do note that the absolute amplitude and the shape of the ENA fluence depend on
the solar atmosphere density model as well as the relevant charge exchange
cross sections (see Appendix A). Nevertheless, because the ENA fluence, and in
particular, its energy dependence, sensitively depend on the flare loop
height, one can use the ENA fluence as a probe of the flare loop height. We
point out that these large post flare loops may not be common. Consequently,
flare ENAs may not be common either. Note that both the time profiles and the
fluence for flare ENAs shown in Figure 5 are vastly different from their
counterparts in shock accelerated ENAs shown in Figure 3. This suggests that
one can use ENA observations to discern if the parent energetic ions are
accelerated at CME-driven shock or at solar flares.

##  4 Conclusions

Understanding the underlying particle acceleration process in large SEP events
has been one of the central problems in heliophysics research. With only in-
situ observations of energetic ions, questions such as the relative roles of
magnetic reconnection in flares vs shock acceleration at CME shocks, and how
to discern the effects of acceleration from that of transport, can be very
hard to answer. In part, this is because our basic understanding of the near-
Sun conditions and the physical processes involved in the production of SEP
events is hampered by our inability to make direct measurements near the
acceleration sites and to remove the effects of transport. ENA observations
can significantly advance our understanding of SEP acceleration at its source
because ENAs do not interact with IMF and is not affected by the transport
effect.

In this paper, we examine the production of ENAs at CME-driven shock fronts
and in solar flares. We compute the time profiles and fluence of ENAs for
these two scenarios. Our calculations suggest that in large SEP events where
ions are efficiently accelerated at CME-driven shocks, ENAs are copiously
produced behind the shock. At 1 au the flux of these ENAs are at a level that
can be readily measured by a dedicated ENA detector. ENAs can also be produced
in flares where large scale and high postflare loops exist. The time profiles
and fluence of ENAs for these two scenarios differ considerably. This offers
us an opportunity to constrain the underlying particle acceleration process
via ENA observations. Our work also forms a theoretical basis for interpreting
future ENA observations.

This work is supported in part by NASA grants 80NSSC19K0075 and 80NSSC20K1783,
and NSF grant 2149771 at UAH. Work at SwRI is partially supported by NASA LWS
grants 80NSSC19K0079 and 80NSSC20K1815. And work at APL is partially supported
by NASA contract 80MSFC19F0002.

##  Appendix A Production and Loss of solar ENAs

Production: We examine the ENA production at solar flares and CME-driven
shocks in this work. The underlying ENA production process is the same for
both cases and is through charge exchange reactions. At time $t$ and location
${\bf r}$, the production rate of ENA is,

$A({\bf r},E,t)=\frac{dn}{dtdE}=\sum_{i}n_{i}\cdot\sigma_{i}\cdot v\cdot
f({\bf r},E)$ (A1)

Here $f({\bf r},E)$ is the distribution function of the accelerated proton
from either the CME-driven shock or the flare site; $E=\frac{1}{2}m_{p}v^{2}$
is the kinetic energy of the energetic proton and we consider non-relativistic
case; the sum is for all contributing charge exchange processes. For the case
of solar composition, the following three charge-exchange interactions are the
most relevant:

$p+O^{6+}\rightarrow H+O^{7+},\quad p+C^{4+}\rightarrow H+C^{5+}\quad
p+H\rightarrow H+p,$ (A2)

The abundance ratio of O6+/p is $\sim 10^{-3}$, and C4+/O6+ is $\sim 0.067$
(von Steiger et al., 2000). For neutral hydrogen, ionization by impact
collision and EUV balance the recombination and charge exchange collisions,
leading to a ratio of neutral H to proton to be $\sim 2.6*10^{-7}$ (D’Amicis
et al., 2007).

Figure 6: The relevant cross sections for ENA production and loss. Adopted
from (Wang et al., 2014, 2022). Dashed lines signal extrapolations.

The corresponding cross sections for the three charge-exchange interactions,
as a function of proton energy, are shown in Figure 6. These cross sections
were obtained from theoretical calculations (Gruntman et al., 2001; Yu Rang,
1992) and are subject to uncertainties. The energy range for these cross
sections are also limited. Following Wang et al. (2022), we have extended them
to a larger energy range whenever necessary. Note that as in (Wang et al.,
2014), we ignore charge exchanges by other ions (He+, N5+, etc) due to their
smaller abundances. Including these would marginally increase the ENA
production rate.

Propagation and Loss of ENAs: Once produced, solar hydrogen ENAs leave their
birth places along ballistic trajectory, subject to losses due to primarily
impact ionization and EUV ionization. The cross sections for the two most
important impact ionization processes are also shown in Figure 6. The
differential flux $J({\bf r},v{\hat{n}},t)$ (with unit of s-1 cm-2 keV-1), at
location ${\bf r}$, time $t$, and along the direction of ${\hat{n}}$, is given
by,

$\displaystyle J({\bf r},v{\hat{n}},t)=\int_{0}^{t}dt^{\prime}\int d^{3}{\bf
v^{\prime}}d^{3}{\bf r^{\prime}}\frac{A({\bf r^{\prime}},{\bf
v^{\prime}},t^{\prime})h({\bf r}-{\bf r^{\prime}},v)}{4\pi|{\bf r}-{\bf
r^{\prime}}|^{2}}\delta(v-v^{\prime})\delta(t-t^{\prime}-\frac{|{\bf r}-{\bf
r^{\prime}}|}{v})cos(\theta)$ (A3)

where $cos(\theta)=({\bf r}-{\bf r^{\prime}})\cdot\hat{\bf n}/|{\bf r}-{\bf
r^{\prime}}|$ and $h({\bf r}-{\bf r^{\prime}},v)$ is the survival probability
of the neutral hydrogen at location ${\bf r}$, produced at ${\bf r^{\prime}}$.
The survival probability $h({\bf r}-{\bf r^{\prime}},v)$ depends on the travel
history and its speed $v$ of the ENA hydrogen and is computed by (Wang et al.,
2014),

$h({\bf r}-{\bf r^{\prime}},v)=exp(-\int_{0}^{|{\bf r}-{\bf
r^{\prime}}|}\gamma({\bf r^{\prime}}){dl})$ (A4)

where the integration $dl$ is along the direction ${\bf r}-{\bf r^{\prime}}$
and $\gamma$ is the total loss rate. We consider three loss processes here:
electron impact ionization, proton impact ionization, and photo-ionization.
The loss rate for these processes are (Wang et al., 2014),

$\gamma_{eH}={\rho_{sw,e}({\bf
r})\sigma_{eH}},\quad\gamma_{pH}={\rho_{sw,p}({\bf
r})\sigma_{pH}},\quad\gamma_{\gamma
H}=4*10^{-3}(\frac{r_{s}}{r})^{2}\frac{1}{v}.$ (A5)

For both the flare ENAs and the CME-shock ENAs, the treatment of ENA
production and propagation/loss is the same. The difference between them is
the region of the energetic ion source. Comparing to the CME case, the post-
flare loops is more localized.

## References

  * Cane et al. (2003) Cane, H. V., von Rosenvinge, T. T., Cohen, C. M. S., & Mewaldt, R. A. 2003, Geophys. Res. Lett., 30, 8017, doi: 10.1029/2002GL016580
  * Cheng et al. (2018) Cheng, X., Li, Y., Wan, L. F., et al. 2018, ApJ, 866, 64, doi: 10.3847/1538-4357/aadd16
  * D’Amicis et al. (2007) D’Amicis, R., Orsini, S., Antonucci, E., et al. 2007, Journal of Geophysical Research (Space Physics), 112, A06110, doi: 10.1029/2006JA011969
  * de Nolfo et al. (2019) de Nolfo, G. A., Bruno, A., Ryan, J. M., et al. 2019, ApJ, 879, 90, doi: 10.3847/1538-4357/ab258f
  * Effenberger et al. (2017) Effenberger, F., Rubio da Costa, F., Oka, M., et al. 2017, ApJ, 835, 124, doi: 10.3847/1538-4357/835/2/124
  * Emslie et al. (2005) Emslie, A. G., Dennis, B. R., Holman, G. D., & Hudson, H. S. 2005, Journal of Geophysical Research (Space Physics), 110, A11103, doi: 10.1029/2005JA011305
  * Emslie et al. (2012) Emslie, A. G., Dennis, B. R., Shih, A. Y., et al. 2012, ApJ, 759, 71, doi: 10.1088/0004-637X/759/1/71
  * Forbes & Lin (2000) Forbes, T. G., & Lin, J. 2000, Journal of Atmospheric and Solar-Terrestrial Physics, 62, 1499, doi: 10.1016/S1364-6826(00)00083-3
  * French et al. (2019) French, R. J., Judge, P. G., Matthews, S. A., & van Driel-Gesztelyi, L. 2019, ApJ, 887, L34, doi: 10.3847/2041-8213/ab5d34
  * Greenspan et al. (1999) Greenspan, M. E., Mason, G. M., & Mazur, J. E. 1999, J. Geophys. Res., 104, 19911, doi: 10.1029/1999JA900225
  * Gruntman et al. (2001) Gruntman, M., Roelof, E. C., Mitchell, D. G., et al. 2001, J. Geophys. Res., 106, 15767, doi: 10.1029/2000JA000328
  * Jejčič et al. (2018) Jejčič, S., Kleint, L., & Heinzel, P. 2018, ApJ, 867, 134, doi: 10.3847/1538-4357/aae650
  * Jin et al. (2018) Jin, M., Petrosian, V., Liu, W., et al. 2018, The Astrophysical Journal, 867, 122, doi: 10.3847/1538-4357/aae1fd
  * Leblanc et al. (1998) Leblanc, Y., Dulk, G. A., & Bougeret, J.-L. 1998, Sol. Phys., 183, 165, doi: 10.1023/A:1005049730506
  * Lee (1983) Lee, M. A. 1983, J. Geophys. Res., 88, 6109, doi: 10.1029/JA088iA08p06109
  * Li et al. (2012a) Li, G., Moore, R., Mewaldt, R. A., Zhao, L., & Labrador, A. W. 2012a, Space Science Reviews, 171, 141, doi: 10.1007/s11214-011-9823-7
  * Li et al. (2012b) Li, G., Shalchi, A., Ao, X., Zank, G., & Verkhoglyadova, O. 2012b, Advances in Space Research, 49, 1067, doi: 10.1016/j.asr.2011.12.027
  * Li & Zank (2005) Li, G., & Zank, G. P. 2005, Geophysical Research Letters, 32, 1, doi: 10.1029/2004GL021250
  * Li et al. (2005) Li, G., Zank, G. P., & Rice, W. K. 2005, Journal of Geophysical Research: Space Physics, 110, A06104, doi: 10.1029/2004JA010600
  * Li et al. (2003) Li, G., Zank, G. P., & Rice, W. K. M. 2003, Journal of Geophysical Research: Space Physics, 108, 1, doi: 10.1029/2002JA009666
  * Li et al. (2021) Li, G., Jin, M., Ding, Z., et al. 2021, The Astrophysical Journal, 919, 146, doi: 10.3847/1538-4357/ac0db9
  * Mason et al. (2021) Mason, G. M., Greenspan, M. E., Kanekal, S. G., et al. 2021, ApJ, 923, 195, doi: 10.3847/1538-4357/ac2fa2
  * Mewaldt et al. (2009) Mewaldt, R. A., Leske, R. A., Stone, E. C., et al. 2009, ApJ, 693, L11, doi: 10.1088/0004-637X/693/1/L11
  * Miller & Roberts (1995) Miller, J. A., & Roberts, D. A. 1995, ApJ, 452, 912, doi: 10.1086/176359
  * Petrosian (2012) Petrosian, V. 2012, Space Sci. Rev., 173, 535, doi: 10.1007/s11214-012-9900-6
  * Ryan (2000) Ryan, J. M. 2000, Space Sci. Rev., 93, 581, doi: 10.1023/A:1026547513730
  * Share et al. (2018) Share, G. H., Murphy, R. J., White, S. M., et al. 2018, ApJ, 869, 182, doi: 10.3847/1538-4357/aaebf7
  * von Steiger et al. (2000) von Steiger, R., Schwadron, N. A., Fisk, L. A., et al. 2000, J. Geophys. Res., 105, 27217, doi: 10.1029/1999JA000358
  * Wang et al. (2014) Wang, L., Li, G., Shih, A. Y., Lin, R. P., & Wimmer-Schweingruber, R. F. 2014, ApJ, 793, L37, doi: 10.1088/2041-8205/793/2/L37
  * Wang et al. (2022) Wang, X. D., Klecker, B., Nicolaou, G., et al. 2022, Earth and Planetary Physics, 6, 42, doi: 10.26464/epp2022003
  * West & Seaton (2015) West, M. J., & Seaton, D. B. 2015, ApJ, 801, L6, doi: 10.1088/2041-8205/801/1/L6
  * Yu Rang (1992) Yu Rang, K. 1992, Journal of Physics B Atomic Molecular Physics, 25, 199, doi: 10.1088/0953-4075/25/1/023
  * Zank et al. (2000) Zank, G. P., Rice, W. K. M., & Wu, C. C. 2000, J. Geophys. Res., 105, 25079, doi: 10.1029/1999JA000455
  * Zhang et al. (2021) Zhang, Q., Guo, F., Daughton, W., Li, H., & Li, X. 2021, Phys. Rev. Lett., 127, 185101, doi: 10.1103/PhysRevLett.127.185101