# Rethinking Generalization Performance of Surgical Phase Recognition with
Expert-Generated Annotations

Seungbum Hong, Jiwon Lee, Bokyung Park, and Min-Kook Choi
hutom
Seoul, Republic of Korea
{qbration21, jiwon, bokyung<EMAIL_ADDRESS>
Woo Jin Hyung
hutom, Yonsei University College of Medicine
Seoul, Republic of Korea
<EMAIL_ADDRESS>
Ahmed A. Alwusaibie, Anwar H. Alfadhel, and SungHyun Park
Yonsei University College of Medicine
Seoul, Republic of Korea
<EMAIL_ADDRESS>
Corresponding author.

###### Abstract

As the area of application of deep neural networks expands to areas requiring
expertise, e.g., in medicine and law, more exquisite annotation processes for
expert knowledge training are required. In particular, it is difficult to
guarantee generalization performance in the clinical field in the case of
expert knowledge training where opinions may differ even among experts on
annotations. To raise the issue of the annotation generation process for
expertise training of CNNs, we verified the annotations for surgical phase
recognition of laparoscopic cholecystectomy and subtotal gastrectomy for
gastric cancer. We produce calibrated annotations for the seven phases of
cholecystectomy by analyzing the discrepancies of previously annotated labels
and by discussing the criteria of surgical phases. For gastrectomy for gastric
cancer has more complex twenty-one surgical phases, we generate consensus
annotation by the revision process with five specialists. By training the CNN-
based surgical phase recognition networks with revised annotations, we
achieved improved generalization performance over models trained with original
annotation under the same cross-validation settings. We showed that the
expertise data annotation pipeline for deep neural networks should be more
rigorous based on the type of problem to apply clinical field.

##  1 Introduction

Because algorithms using pattern recognition or machine learning are applied
for various tasks, problem definition and data acquisition are as important as
the model’s performance [1-4]. Training using deep neural networks, a trend in
machine learning, requires large amounts of annotated data to avoid
overfitting with a large number of parameters. Crowdsourcing-based tools, such
as Amazon Mechanical Turk, which requires simple training for annotators, can
be widely used to efficiently collect large amounts of data, especially for
visual tasks with high data redundancy and diverse variations between images
[5-9]. The process of securing the annotation database in the visual
recognition problem is essential for defining and training algorithms but is
considered separately from the model training process.

Figure 1 shows an example of the relationship between the annotation
difficulty and the visual recognition problem. In the example in Figure 1, if
the effort to annotate a single sample, e.g., image classification or object
detection, is relatively small and high generalization performance can be
easily achieved, the difficulty level of the annotation is relatively low.
However, even in the image classification problem, assuming that the
classifier classifies Malamute and Siberian Husky if the annotator lacks prior
knowledge of the dog’s breed, the difficulty of annotating may increase,
increasing the overall difficulty level of the generalization. It is important
to analyze the difficulty level of the annotation because the range of visual
recognition problems to be solved using deep learning is expanding from simple
recognition to problems requiring expertise. In particular, problems involving
medical diagnosis, aid, or surgery are often impossible to annotate based on
appearance only [10-17], e.g., through visual recognition of natural scenes.
Furthermore, the annotation of supervision for visual recognition that
includes this expertise is very expensive, so the inspection effort is also
very expensive.

Figure 1: Annotation difficulty of expert knowledge. Depending on the
annotation difficulty, thorough verification of the data collection and
annotation process is required. We exemplify the surgical phase recognition
problem for laparoscopic cholecystectomy and argue that a more difficult
problem requires more consideration in the annotation process.

In this paper, we argue that when deep neural networks train expert knowledge
such as surgical phase recognition which has a very high annotation
difficulty, one needs to consider the more precise annotation procedure than
the typical visual recognition problem. Our analysis shows that surgical phase
recognition can have very different generalization performance depending on
the type of supervision, which means that it can have a very large bias when
applying the training model in the clinical field. To do this, we compared the
generalization performances of the same model trained with revised annotations
obtained from different experts’ consensus processes. we tried two different
datasets generation pipeline for surgical phase recognition [17-23] of
laparoscopic cholecystectomy and robotic subtotal gastrectomy for gastric
cancer. To reconsider the accuracy of the existing annotations public dataset
for surgical phase recognition of laparoscopic cholecystectomy, we used 24
surgery videos from the Cholec80 [17] dataset and we matched the criteria of
two specialists and created two additional revised annotations. At the same
time, we devised a pipeline capable of generating consensus annotations for 40
cases of surgery by the Da Vinci system to agree with experts in gastrectomy
for gastric cancer, one of the most complex surgeries. To best our knowledge,
This is the first approach to automated surgical phase recognition of subtotal
gastrectomy for gastric cancer. By comparison of validation performance with
surgical phase recognition models based on Convolutional Neural Networks
(CNN), we confirmed the nontrivial differences in the test results of the
models trained under the same cross-validation conditions except the
annotation.

##  2 Background

Medical data annotation for visual recognition. Annotations for training
visual recognition models in natural scenes are primarily made to achieve
classification, detection, and segmentation of images and videos. Annotations
are sometimes performed to train algorithms for regression problems, such as
pose recognition and image registration. They are generally annotated at a
level that allows training by the public with a mild iterative training
process. For this reason, crowdsourcing platforms [5-9] that use web-based
annotation tools are widely used in visual recognition problems. As the
performance of visual recognition technology improves, various annotation
tools for training are being introduced in the field of medical imaging [10,
11]. In many cases, X-ray, magnetic resonance imaging, and computed tomography
images are used as annotation targets for automated diagnosis or diagnostic
assistance [12-15]. As the demand for data for training increases, web-based
annotation tools or methodologies, such as crowdsourcing, for medical imaging
are also being introduced [16]. Recently, automated visual recognition has
started to be used as a medical analysis method for surgical videos, and
recent datasets such as Cholec80 [17] have been widely used. Annotation for
surgical phase recognition with differences from other visual recognition
problems is very expensive because it requires expertise in a specific type of
surgery. Annotations made by different experts are likely to be different
because frame-level annotations must be performed for all surgical procedures,
and at the same time, surgical phases of a particular duration must be
determined by considering the continuous information on the temporal
dimension. In this study, we show that the generalization performance for
automated surgical phase recognition varies considerably with expert
annotation. This argues that more attention should be paid to the annotation
process considering its use in the clinical field.

Surgical phase recognition. With the development of visual recognition
technologies, various surgical phase recognition algorithms have been
developed to analyze the surgical process using surgical videos. Hierarchical
hidden Markov model with two levels to process temporal information of
features encoded by CNNs is developed by [17]. The features obtained from the
hierarchical Hidden Markov model were used to achieve high accuracy surgical
phase recognition for laparoscopic cholecystectomy using a support vector
machine classifier. In addition, an efficient training method from a small
number of surgery videos using a CNN-LSTM [27] structure with the same dataset
used in [17] has been proposed [18]. In [19], CNN-LSTM-based surgical phase
recognition was performed with the Cholec120 dataset, which was extended from
the Cholec80 dataset, using the pre-trained remaining surgery duration network
for self-supervised learning. In addition to cholecystectomy using the
Cholec80 dataset [20], surgical phase recognition for laparoscopic
sigmoidectomy [21] and laparoscopic hysterectomy [22] was performed. In [23],
CNN-based surgical phase recognition was performed using prostatectomy videos
for robotic surgery.

Saptiotemporal encoding for visual recognition. Various efforts have been made
to efficiently encode spatiotemporal feature in the visual recognition problem
[24, 25]. In the early days of the deep learning era, models were proposed to
utilize 2D-CNN feature and motion information together [26], and a 2D-CNN-LSTM
model including the consideration of the temporal domain of 2D-CNN features
was proposed [27]. Recent researches have been reported that a model that
simultaneously encodes spatiotemporal information in one network may perform
better than explicitly encoding separately processed motion information
[28-32]. Our goal is not to suggest a good spatiotemporal model, but to see
how the state-of-the-art performance models influence the results of inference
in training expertise under different annotations. For this purpose, the
results of surgical phase recognition were verified with models based on
2D-CNN-LSTM [27] and 3D-CNN architectures [29, 31].

##  3 Annotation verification

We used a surgical phase recognition dataset of laparoscopic cholecystectomy
to perform the annotation verification process. The original videos of the
laparoscopic cholecystectomy were a subset of the Cholec80 dataset [17]. We
selected 24 videos released for the training surgical phase recognition. The
revised annotations of the 24 videos were generated through expert consensus
with two specialists. In addition to laparoscopic cholecystectomy, we built an
annotation generation pipeline for gastrectomy for gastric cancer in 40 cases
to try to recognize more complex surgery111Please refer to supplementary
materials for more details of the refined surgical phase and surgical phase
for subtotal gastrectomy for gastric cancer.. The annotation and verification
process went through the processes shown in Figures 2 and 3, and a total of 5
specialists participated in this process.

Figure 2: Schematic of annotation verification process for surgical phase
recognition in laparoscopic cholecystectomy. With 24 surgical videos of
cholecystectomy, multiple expert opinions were processed to validate ground
truth annotations. We trained the CNN-based classification models with the
same architecture and initial weights as the original and revised annotations
and verified whether there is a change in inference performance. Figure 3:
Schematic of generating consensus annotation for surgical phase recognition in
robotic subtotal gastrectomy for gastric cancer. Since gastrectomy for gastric
cancer is one of the most complicated surgeries, we have established an
additional process for annotation generation. A total of 4 specialists
respectively annotate for each surgical video and generate a new label through
an AND operation between the annotations. For frames that remain blank after
the AND operation, another specialist writes the final label by colligating
the opinions of other annotations as an inspector.

Surgical phase recognition in laparoscopic cholecystectomy. Seven phases of
laparoscopic cholecystectomy surgery were defined in the Cholec80 dataset
[17]. The preparation, Calot triangle dissection, clipping and cutting,
gallbladder dissection, gallbladder packaging, cleaning and coagulation, and
gallbladder retraction were indicated with index 0, 1, 2, 3, 4, 5, and 6,
respectively. The annotation criteria were designed to target the multiclass
classification problem. Labels were annotated in a form in which all frames
were extracted from a video recording with a specific case of surgery.
Depending on the progress of the surgery, the surgical phase may have been
preceded by a higher index phase or returned to the previous phase, but
multiple labels for one frame were not allowed. Depending on the progress of
the surgery, the entire surgery length was distributed within approximately 30
min to 2 h. Currently, only one label is provided as the correct answer for
the classification dataset. We used the provided annotations from Cholec80 as
Annotation 1 to train the CNN-based surgical phase recognition model.

Figure 4: Comparison of annotations before and after expert consensus. (a)
Plot of the existing and revised annotations on the original video at the same
timestamp. (b) Result of re-plotting only the section where the change occurs
in a specific surgical phase in the video. Disagreements can be seen near the
surgical phase changes.

Surgical phase recognition in robotic subtotal gastrectomy. We divided the
gastrectomy surgical phases for gastric cancer into 27 phases, based on ARMES
[32]1. ARMES is a system that divides surgical processes into 20 separate
phases, from start to finish, to provide the surgeon with information on
gastrectomy for gastric cancer. We added one phase to the 21-phases surgical
procedure, as defined by ARMES, and composed 27 surgical steps. This included
the 21 surgical procedures as part of the actual gastrectomy and six non-
gastrectomy procedures. At this time, the six non-gastrectomy procedure types
include actions that are irrelevant to gastrectomy, such as camera cleaning,
adhesiolysis, and facility management, which may appear during gastrectomy.
Since such actions are unrelated to the gastrectomy and do not depend on a
specific surgical step; they may affect the training process of the surgical
phase recognition model. Therefore, these were included separately.

Annotation revision for laparoscopic cholecystectomy. We revised the
annotation of the phase of the laparoscopic cholecystectomy surgery through a
consensus process of three specialists. Figure 4 shows an example of the
agreed annotation of each surgical phase, which was updated based on the
original annotation. The consensus among specialists for the seven steps of
cholecystectomy was based on their consensus, and a separate consensus process
was performed on the points where changes in the surgical phase occurred or
where exceptions occurred. Finally, two types of annotation (Annotation 2 and
3) revised by two specialists were created from Annotation 1. Figure 4 (a)
shows the difference between the revised annotation and the original
annotation in the entire cholecystectomy, and (b) shows the difference between
the annotations according to the opinions of the experts in the section where
the surgical phase changed.

Annotation pipeline for robotic subtotal gastrectomy. For accurate labeling
for more complex surgery, we performed frame-level annotation for 40 cases
that were cross-validated by five specialists. For cross-validation, we
received annotated information according to each opinion from four
specialists, and this information was compared with the areas where the
opinions matched by AND operation. For labels containing blank generated by
the AND operation, another specialist annotates the label as an investigator,
resulting in the final consensus label. Robotic surgery for 40 cases of
gastric cancer proceeded subtotal gastrectomy and nineteen cases were recorded
on the da Vinci Si system, while twenty-one videos were recorded on the da
Vinci Xi. Figure 3 shows a schematic illustration of the annotation
verification process.

##  4 Experimental Results

To verify the difference in the model performance as the annotation changes,
we trained the 2D-CNN-LSTM model [27] and the two types of 3D-CNN based models
using the original and revised annotations for the Cholec80 and robotic
gastrectomy for gastric cancer222Additional results and analysis are included
in supplementary materials.. As the 3D-CNN model, we selected and trained the
latest-performance 3D-ResNet [28] and ECO [31] models. Test scenarios were
constructed according to the structural characteristics of the trained model,
and no pre- or post-processing of the input data and inference output was
performed to accurately analyze the effect of the label on the model training.
However, in the training and inference of each CNN model, basic data
augmentation was performed as part of the regularization at the input stage of
the architecture. All models have been implemented using the PyTorch v1.2.0
[33] and trained on NVIDIA V100.

The loss function for the final cross-entropy output in the recognition of all
surgical stages is

$L_{CE}=-\frac{1}{M}\sum^{M}_{i}\sum^{C}_{j}t_{i}\log(h(x^{i}_{j};\theta)),$
(1)

where $M$ is the size of the input minibatch, and $C$ is the total number of
surgical phases of a each surgery ($C=7$ for cholecystectomy, $C=27$ for
subtotal gastrectomy); $h(x^{i}_{j};\theta)$ is the softmax output for the
input sequence (or image) $x$ in each architecture with $\theta$ as the
training weight, and $t$ as the ground truth label for each $x^{i}_{j}$.

Training 2D-CNN-LSTM. ResNet34 [34] pre-trained with ImageNet was used as the
2D-CNN model. The input clip sequence for training received 16 frames at a
sampling rate of 1f/s and consisted of a total of 16 s of frame data; each
input clip sequence had an overlap ratio of 0.5 with the previous input clip.
ADAM was used as the optimizer; the initial learning rate was $1\times
10^{-4}$, and a scale of 0.1 was applied to the learning rate every 30 epochs.
A total of 100 epochs were trained with 256 mini-batches, and the input frame
was resized to 256$\times$256\. For data augmentation during training, spatial
random crop and left and right flips for multiple scales were applied.

Training 3D-ResNet. 3D-ResNet extends the residual learning structure [29]
designed for efficient training of CNNs into three dimensions by considering
time-level information simultaneously. We utilized a 34-layer 3D-ResNet with
pre-trained by the Kinetics dataset [35] to efficiently train spatiotemporal
information about the surgical phase in the surgery video. The input clip was
composed of 16 frames at 1f/s as in the case of the 2D-CNN-LSTM. SGD was used
as the optimizer with initial learning of 0.001, the momentum of 0.9, and
weight decay of $5\times 10^{-5}$; the learning scheduler was applied to
reduce the learning rate at the plateau. The mini-batch trained a total of 200
epochs at 128; the spatial size of the input sequence was resized to
256$\times$256 and finally processed to a training input through random
cropping to a size of 224$\times$224.

Training ECO. ECO has two structurally arranged networks to simultaneously
utilize the advantages of encoding the spatial information of the 2D CNNs and
the temporal information of the 3D CNNs. The 3D CNNs with a relatively large
number of parameters were applied in the final output stage of the network to
maximize the accuracy and efficiency of the inference speed [31]. We used pre-
trained BN-Inception [36] from the ImageNet [5] as the 2D CNN for training
ECO, and an 18-layer 3D ResNet network pre-trained with the Kinetics dataset
[35] as the 3D CNNs. SGD with Nesterov momentum was used as the optimizer, and
dropouts were applied to each fully connected layer. The initial learning rate
was set to 0.001, and the learning scheduler applied a scale of 0.1 when
verification errors are saturated every four epochs. We set the momentum to
0.9, weight decay to $5\times 10^{-4}$, and the size of the mini-batch to 32.
The input frame was resized to 224$\times$224 after applying a fixed corner
crop and horizontal flip to resized frame to 240$\times$320.

Table 1: AP for the inference of surgical phase recognition for a cross-
validation set of laparoscopic cholecystectomy for each annotation and model.

- |  Split 1 (Ann 1/2/3) |  Split 2 (Ann 1/2/3) |  Split 3 (Ann 1/2/3) |  Split 4 (An 1/2/3) |  Split 5 (Ann 1/2/3) |  Split 6 (Ann 1/2/3) |  mAP (Ann 1/2/3)
---|---|---|---|---|---|---|---
2D-CNN-LSTM | 65.8/66.1/62.3 | 59.2/60.4/58.5 | 51.1/37.8/51.59 | 51.7/48.1/57.1 | 70.9/70.3/68.0 | 67.8/72.0 /68.6 | 61.1/59.1/61.0
3D-ResNet | 67.8/73.2/72.9 | 49.9/61.3/60.6 | 52.3/51.1/57.1 | 52.5/53.1/46.3 |  73.5/68.7/72.9 | 66.7/65.1/62.9 | 60.5/61.5/67.6
ECO | 74.1/75.0/69.1 | 64.4/70.9/65.5 | 45.6/49.1/46.9 | 57.3/60.3/55.9 | 71.5/71.5/73.5 | 69.6/67.7/68.6 | 63.8/65.7/63.2

Table 2: Averaged AP for inference of the surgical phase recognition of
robotic gastrectomy for gastric cancer with each annotation and model.

2D-CNN-LSTM | Ann1 | Con-Ann1 | Ann2 | Con-Ann2 | Ann3 | Con-Ann.3 | Ann4 | Con-Ann4 | Con
---|---|---|---|---|---|---|---|---|---
Split1 | 65.07 | +2.49 | 63.23 | +4.33 | 67.38 | +0.18 | 64.77 | +2.79 | 67.56
Split2 | 59.32 | +4.81 | 62.21 | +1.92 | 60.21 | +3.92 | 55.72 | +8.41 | 64.13
Split3 | 68.59 | +3.05 | 68.8 | +2.84 | 67.59 | +4.05 | 68.57 | +3.07 | 71.64
3D-ResNet | Ann1 | Con-Ann1 | Ann2 | Con-Ann2 | Ann3 | Con-Ann.3 | Ann4 | Con-Ann4 | Con
Split1 | 68.31 | +1.98 | 67.52 | +2.77 | 67.54 | +2.75 | 65.34 | +4.95 | 70.29
Split2 | 66.84 | +3.11 | 64.68 | +5.27 | 68.29 | +1.66 | 62.46 | +7.49 | 69.95
Split3 | 72.4 | +4.79 | 72.25 | +4.94 | 74.31 | +2.88 | 74.16 | +3.03 | 77.19
ECO | Ann1 | Con-Ann1 | Ann2 | Con-Ann2 | Ann3 | Con-Ann.3 | Ann4 | Con-Ann4 | Con
Split1 | 64.97 | +4.69 | 63.89 | +5.77 | 66.39 | +3.27 | 67.13 | +2.53 | 69.66
Split2 | 60.46 | +4.76 | 61.85 | +3.37 | 61.26 | +3.96 | 60.5 | +4.72 | 65.22
Split3 | 71.45 | +4.1 | 70.73 | +4.82 | 72.79 | +2.76 | 72.28 | \+ 3.27 | 75.55

Testing. The inference scenario for obtaining the testing performance received
a sequence of the same size as that of the training process as input and
output the final confidence value for each surgical phase. In the 2D CNN-LSTM
and ECO, the spatial feature information encoded in the 2D-CNN was used as
input from the previous $(k-15)$ frames to the $k$th frame for inference on
the $k$th frame; in the 3DResNet, sequence information for 16 frames as input
was used. In the 2D-CNN of the 2D CNN-LSTM and ECO, the feature information
for the first 16 frame inputs was stored in a buffer, and only the feature
information for the $k$th frame was updated in the form of a queue. In the
3D-ResNet, it waited until the first 16 frames were stored in the input buffer
to perform inference. Then, in the form of a cue, the first and last frames of
the input sequence were replaced to perform inference. Finally, the current
surgical phase was determined as the class with the maximum confidence value,
and augmentation techniques other than the input image size resizing were not
applied in consideration of the real-time scenario [31].

Results of laparoscopic cholecystectomy. Table 1 shows the results of the
surgical phase recognition for each split of cholecystectomy. The cross-
validation set consists of 20 training surgical videos and 4 testing surgical
videos. All test performance was measured as the average precision (AP) of the
inference results. The test performance of two trained models with different
initial weights was averaged for a fair comparison. From the table, it can be
seen that the revised annotation (Annotations 2 and 3) shows improved
generalization performance compared to the original annotation in 3D-CNN-based
surgical phase recognition models. This seems to alleviate the bias of the
training information for the part where the classification of the surgical
phase is ambiguous (such as the section where the change in the surgical phase
occurs) in the process of agreeing with experts. Furthermore, the 3D-CNN-based
models show improved generalization performance in all experiments compared to
the 2D CNN-LSTM, and it is confirmed that simultaneously utilizing
spatiotemporal information in feature processing is effective for the
recognition of the surgical phase.

Results of robotic subtotal gastrectomy. We performed cross-validation by
dividing 40 cases of gastric cancer surgery into 35 cases of training data and
5 cases of evaluation data. In order to properly distribute the difficulty of
evaluation data, we constructed 3 types of split cross-validation sets by
considering the patient’s age, operation time, vertebral bleeding amount, and
BMI among 40 cases of surgical video. Table 2 shows the inference performance
for each evaluation split of the surgical phase recognition model trained from
the annotations performed by four specialists. At this time, Con denotes an
annotation in which the opinions of the investigator agreed upon. Con-Ann
denotes the difference between the evaluation performance of the model trained
with the annotations obtained from each annotator and the performance
evaluated by the consensus annotation.

In the case of the surgical phase recognition model trained with consensus
annotation, the highest performance was achieved in all models and evaluation
splits. In Figure 5, in all splits, the evaluation results varied from at
least $2\%$ to $5\%$ depending on the model. Figure 6 shows the visualization
of the inference results of a model trained with different annotations on the
same surgical video. The model trained from the annotations obtained through
consensus was able to confirm that the model showed more robust performance in
the section with frequent step movements in the middle of the operation, and
relatively stable inference results compared to the model trained on each
annotation.

Figure 5: Average performance gap of all splits between surgical phase
recognition model trained with individual and consensus annotation. Depending
on the model, the graph shows a performance gap from a minimum of $2\%$ to a
maximum of $5\%$. Figure 6: Visualization of inference output of same surgical
video with different annotations. The visualizations obtained from the model
trained by the four different types of annotation and the consensus annotation
are shown. bars are colored according to a total of 21 gastrectomy phases and
6 non-gastrectomy actions.

##  5 Conclusion

To rethink the generalization performance of the surgical phase recognition,
we performed verification on a training model trained with annotations
generated by experts. We performed cross-validation on the public Cholec80
dataset [17] and also tried additional validation to subtotal gastrectomy for
gastric cancer with multiple expertise annotations. It can be confirmed that
the difference in generalization error is probably very large under the same
conditions depending on how the annotation process is designed when the
difficulty of the annotation is extremely high. Through this study, it was
shown that recognition problems with high annotation difficulty, such as
surgical video or medical imaging analysis, have to undergo a more rigorous
annotation process considering the application of the clinical field. In the
future, we plan to conduct research by developing algorithms to secure
annotations that can improve generalization performance through model
uncertainty or evolutionary algorithms.

## References

[1] Jain, A. K. & Chandrasekaran, B. (1982) 39 Dimensionality and sample size
considerations in pattern recognition practice. Handbook of Statistics,
2:835-855.

[2] Raudys, S. J. & Jain, A. K. (1991) Small sample size effects in
statistical pattern recognition: Recommendations for practitioners. IEEE
Transactions on Pattern Analysis and Machine Intelligence, 13(3):252-264.

[3] Figueroa, R. L. Zeng-Treitler, Q. Kandula, S. & Ngo, L. H. (2012)
Predicting sample size required for classification performance. BMC Med.
Inform. Decis. Mak. 12(8).

[4] Beleites, C. Neugebauer, U. Bocklitz, T. Krafft, C. & Popp, J. (2013)
Sample size planning for classification models. Analytica Chimica Acta,
760(14): 25-33.

[5] Deng, J. Dong, W. Socher, R. Li, L. -J. Li, K. & Fei-Fei, L. (2009)
ImageNet: A large-scale hierarchical image database. Proc. of CVPR.

[6] Lin, T. -Y. Maire, M. Belongie, S. Hays, J. Perona, P. Ramanan, D. Dollar,
P. & Zitnick, C. L. (2014) Microsoft COCO: Common objects in context. Proc. of
ECCV.

[7] Vondrick, C. Patterson, D. & Ramanan, D. (2012) Efficiently scaling up
crowdsourced video annotation. International Journal of Computer Vision
101(1):184-204.

[8] Heilbron F. C. & Niebles, J. C. (2014) Collecting and annotating human
activities in web videos. Proc. of ICMR.

[9] Damen, D. Doughty, H. Farinella, G. M. Fidler, S. Furnari, A. Kazakos, E.
Moltisanti, D. Munro, J. Perrett, T. Price, W. & Wray, M. (2018) Scaling
egocentric vision: The EPIC-KITCHENS dataset. Proc. of ECCV.

[10] Grunberg, K. Jimenez-del-Toro, O. Jakab, A. Langs, G. Fernandez, T. S.
Winterstein, M. Weber, M. -A. & Krenn, M. Annotating medical image data.
Cloud-Based Benchmarking of Medical Image Analysis 45-67.

[11] Pelka, O. Nensa, F. & Friedrich, C. M. (2018) Annotation of enhanced
radiographs for medical image retrieval with deep convolutional neural
networks. PLoS ONE 13(11).

[12] Seiferta, S. Kelma, M. Moellerb, M. Mukherjeec, S. Cavallarod, A. Hubera,
M. & Comaniciu, D. (2010) Semantic annotation of medical images., Proc. of
SPIE Medical Imaging.

[13] Lundervold, A. S. & Lundervold, A. (2018) An overview of deep learning in
medical imaging focusing on MRI. Zeitschrift für Medizinische Physik
29(2):102-127.

[14] Philbrick, K. A. Weston, A. D. Akkus, Z. Kline, T. L. Korfiatis, P.
Sakinis, T. Kostandy, P. Boonrod, A. Zeinoddini, A. Takahashi, N. & Erickson,
B. J. (2019) RIL-contour: A medical imaging dataset annotation tool for and
with deep learning. Journal of Digital Imaging 32(4):571-581.

[15] Kohli, M. D. Summers, R. M. & Geis, J. R. (2017) Medical image data and
datasets in the era of machine learning-whitepaper from the 2016 C-MIMI
meeting dataset session. Journal of Digital Imaging 30(4):392-399.

[16] Ørting, S. Doyle, A. van Hilten, A. Hirth, M. Inel, O. Madan, C. R.
Mavridis, P. Spiers, H. & Cheplygina, V. (2020) A survey of crowdsourcing in
medical image analysis. Human Computation 7(1):1-26.

[17] Twinanda, A. P. Shehata, S. Mutter, D. Marescaux, J. de Mathelin, M. &
Padoy, N. (2017) EndoNet: A deep architecture for recognition tasks on
laparoscopic Videos. IEEE Transactions on Medical Imaging 36(1):86-97.

[18] Yu, T. Mutter, D. Marescaux, J. & Padoy, N. (2018) Learning from a tiny
dataset of manual annotations: a teacher/student approach for surgical phase
recognition. Proc. of IPCAI.

[19] Yengera, G. Mutter, D. Marescaux, J. & Padoy, N. (2018) Less is more:
Surgical phase recognition with less annotations through self-supervised pre-
training of CNN-LSTM networks. arXiv:1805.08569.

[20] Loukas, C. (2019) Surgical phase recognition of short video shots based
on temporal modeling of deep features. Proc. of BIOSTEC.

[21] Kitaguchi, D. Takeshita, N. Matsuzaki, H. Takano, H. Owada, Y. Enomoto,
T. Oda, T. Miura, H. Yamanashi, T. Watanabe, M. Sato, D. Sugomori, Y. Hara, S.
& Ito, M. (2020) Real-time automatic surgical phase recognition in
laparoscopic sigmoidectomy using the convolutional neural network-based deep
learning approach. Surgical Endoscopy 34(11):4924-4931.

[22] Meeuwsen, F. C. van Luyn, F. Blikkendaal, M. D. Jansen, F. W. & van den
Dobbelsteen, J. J. (2019) Surgical phase modeling in minimal invasive surgery.
Surgical Endoscopy 33(5):1426-1432.

[23] Zia, A. Hung, A. Essa, I. & Jarc, A. (2018) Surgical activity recognition
in robot-assisted radical prostatectomy using deep learning. Proc. of MICCAI.

[24] Kong, Y. & Fu, Y. (2018) Human action recognition and prediction. CoRR
abs/1806.11230.

[25] Bhoi, A. (2019) Spatio-temporal action recognition: A survey. CoRR
abs/1901.09403.

[26] Simonyan, K. & Zisserman, A. (2014) Two-stream convolutional networks for
action recognition in videos. In Proc. of NIPS.

[27] Donahue, J. Hendricks, L. A. Rohrbach, M. Venugopalan S. Guadarrama, S.
Saenko, K. & Darrell, T. (2015) Long-term recurrent convolutional networks for
visual recognition and description. Proc. of CVPR.

[28] Tran, D. Wang, H. Torresani, L. Ray, J. LeCun, Y. & Paluri, M. (2018) A
closer look at spatiotemporal convolutions for action recognition, Proc. of
CVPR.

[29] Hara, K. Kataoka, H. & Satoh, Y. (2018) Can spatiotemporal 3D CNNs
retrace the history of 2D CNNs and ImageNet? Proc. of CVPR.

[30] Feichtenhofer, C. Fan, H. Malik, J. & He, K. (2019) SlowFast networks for
video recognition. Proc. of ICCV.

[31] Zolfaghari, M. Singh, K. & Brox, T. (2018) ECO: Efficient Convolutional
Network for Online Video Understanding. Proc. of ECCV.

[32] Lee, J. H. Tanaka, E. Woo, Y. Ali, G. Son, T. Kim, H., -I. & Hyung, W.,
J. (2017) Advanced real-time multi-display educational system (ARMES): An
innovative real-time audiovisual mentoring tool for complex robotic surgery.
Journal of Surgical Oncology 116(7):894-897.

[33] Paszke, A. Gross, S. Massa, F. Lerer, A. Bradbury, J. Chanan, G. Killeen,
T. Lin, Z. Gimelshein, N. Antiga, L. Desmaison, A. Kopf, A. Yang, E. DeVito,
Z. Raison, M. Tejani, A. Chilamkurthy, S. Steiner, B. Fang, L. Bai, J. &
Chintala. S. (2019) PyTorch: An imperative style, high-performance deep
learning library. Proc. of NeurIPS.

[34] He, K. Zhang, X. Ren, S. & Sun, J. (2016) Deep residual learning for
image recognition. Proc. of CVPR.

[35] Kay, W. Carreira, J. Simonyan, K. Zhang, B. Hillier, C. Vijayanarasimhan,
S. Viola, F. Green, T. Back, T. Natsev, P. Suleyman, M. & Zisserman, A. (2017)
The Kinetics human action video dataset. CoRR abs/1705.06950.

[36] Szegedy, C. Ioffe, S. Vanhoucke, V. & Alemi A. (2017) Inception-v4,
Inception-ResNet and the impact of residual connections on learning. Proc. of
AAAI.

## Supplementary Materials

## A. Network Architecture

Figure 7 shows schematics of model architectures of 2D-CNN-LSTM [27],
3D-ResNet [29], and ECO [31] models. Test scenarios were constructed according
to the structural characteristics of the trained model, and no pre- or post-
processing of the input data and inference output was performed to accurately
analyze the effect of the label on the model training. However, in the
training and inference of each CNN model, basic data augmentation was
performed as part of the regularization at the input stage of the
architecture.

Annotation revision for laparoscopic cholecystectomy. We revised the
annotation of the phase of the laparoscopic cholecystectomy surgery through a
consensus process of three specialists. The consensus among specialists for
the seven steps of cholecystectomy was based on their consensus, and a
separate consensus process was performed on the points where changes in the
surgical phase occurred or where exceptions occurred. Finally, two types of
annotation (Annotation 2 and 3) revised by two specialists were created from
Annotation 1. Table 3 shows the details of each redefined surgical phase.

Annotation pipeline for robotic subtotal gastrectomy. For accurate labeling
for more complex surgery, we performed frame-level annotation for 40 cases
that were cross-validated by three specialists. For cross-validation, we
received annotated information according to each opinion from four
specialists, and this information was compared with the areas where the
opinions matched by AND operation. For labels containing blank generated by
the AND operation, another specialist annotates the label as an investigator,
resulting in the final consensus label. Robotic surgery for 40 cases of
gastric cancer proceeded subtotal gastrectomy and nineteen cases were recorded
on the da Vinci Si system, while twenty-one videos were recorded on the da
Vinci Xi. Table 5 shows the details of each surgical phase of subtotal
gastrectomy.

Figure 7: Schematic illustration of each network architecture utilized for
surgical phase recognition. 2D-CNN-LSTM encodes information in the spatial
domain with 2D-CNN only, and then considers the information together in the
time domain, and the other two models simultaneously consider spatiotemporal
information through the 3D convolution layer. Table 3: Surgical phase of
cholecystectomy redefined through expert consensus.

Phase ID | Definition
---|---
preparation (0) |  Time period from the start of the surgery until the first laparoscopic tool appears.
calot triangle dissection (1) |  Time period from when the first laparoscopic tool appears until the first clip applier appears.
clipping and cutting (2) |  Time period from when the first clip applier appears until the cystic.a & duct is finished (The end point is until the next tool touches the organization).
gallbladder dissection (3) | Time period from when the tool touches tissue until the specimen bag appears.
gallbladder packaging (4) |  Time period from when the specimen bag appears until it is closed (or until a suction tip or other aid tool appears).
cleaning and coagulation (5) |  Time period from when the specimen bag is closed until the tool touches the bag strap (or until the suction tip is withdrawn or the trocar appears on the screen).
gallbladder retraction (6) |  Time period from the first touch of the bag strap until the specimen bag is withdrawn.

Table 4: Defining the surgical stage of subtotal gastrectomy for gastric
cancer based on ARMES. IDs 1 to 21 include surgical steps related to
gastrectomy, and 22 to 27 refer to non-gastrectomy actions not included in the
subtotal gastrectomy process.

ID | Surgical phase | ID | Surgical phase
---|---|---|---
1 | Trocar insertion | 15 |  Dissection of LN station 7 and ligation of left gastric artery
2 | Docking | 16 | Dissection of LN station 11p
3 |  Division of less omentum up to the right side of the esophagus | 17 |  Clearance of soft tissue along the lesser curvature
4 | Liver retraction | 18 | Gastric transection
5 | Partial (or total) omentectomy | 19 | Harvesting resected specimen into Endo bag
6 | Ligation of left gastroepiploic vessels | 20 | Anastomosis
7 |  Clearance of soft tissues along the greater curvature | 21 | Retrieval of specimen
8 | Ligation of Right gastroepiploic vein | 22 | Adhesiolysis
9 | Ligation of Right Gastroepiploic Artery | 23 | Housekeeping
10 | Creation of window for duodenal transection | 24 | Clean camera
11 | Duodenal transection | 25 | Junk
12 | Ligation of right gastric artery | 26 | Other procedure
13 | Dissection of LN stations 12a | 27 | Unexpected surgical events
14 | Dissection of LN station 8 and 9 | - | -

##  6 B. Additional Results

Table 5 shows per phase recognition performance. As seen in Table 3, 3D-CNN
based model outperforms every phase in laparoscopic cholecystectomy by 2D-CNN-
LSTM model, and 3D-ResNet achieved the highest performance in 12 phases for
subtotal gastrectomy.

Additional statistics for laparoscopic cholecystectomy. Figure 8, 9, and 10
show the confusion matrix for each surgical video in split 1 for each training
model trained with different annotations. These figures shows the relationship
between classes that are difficult to accurately classify. Figures 11, 12, and
13 visualize the inferred surgical phase on the time axis versus the ground
truth label for the surgical phase. In all graphs, the ground truth label is
orange and the inferred output is blue. Models trained at different
annotations can be observed on the time base to indicate which inference
output. In the case of the surgical phase recognition model trained with
consensus annotation, the highest performance was achieved in all models and
evaluation splits.

Additional statistics for robotic subtotal gastrectomy. Figure 14, and 15 show
the confusion matrix for two example cases in split 1 for each training model
trained with consensus annotations, respectively. Figure 16, and 17 show the
mean AP for each split and surgical phase and the deviation value of each mean
AP in laparoscopic cholecystectomy and robotic gastrectomy, respectively. The
graph in Figure 5 and 6 show the sensitivity of generalization to each model.

Table 5: Generalization AP for each surgical phase for laparoscopic
cholecystectomy and robotic gastrectomy.

Laparoscopic cholecystectomy
---
Phase ID | 0 | 1 | 2 | 3 | 4 | 5 | 6
2D-CNN-LSTM | 71.8 | 74.1 | 36.5 | 44.7 | 56.8 | 57.1 | 37.7
3D-ResNet | 78.7 | 76.7 | 30.6 | 37.6 | 60.1 | 53.7 | 23.4
ECO | 75.8 | 72.0 | 42.3 | 57.8 | 69.2 | 59.4 | 44.9
Robotic gastrectomy for gastric cancer
Phase ID | 1 | 2 | 3 | 4 | 5 | 6 | 7 | 8 | 9 | 10 | 11 | 12 | 13 | 14
2D-CNN-LSTM | 64.0 | 58.5 | 31.2 | 81.2 | 74.1 | 50.2 | 37.9 | 62.4 | 60.9 | 76.3 | 47.2 | 39.3 | 0.3 | 23.3
3D-ResNet | 60.6 | 69.7 | 40.5 | 72.8 | 76.1 | 55.9 | 57.8 | 53.7 | 62.5 | 76.4 | 52.1 | 22.7 | 0.4 | 62.1
ECO | 70.3 | 66.8 | 32.2 | 76.6 | 72.7 | 32.9 | 57.4 | 54.5 | 72.7 | 82.8 | 54.0 | 26.4 | 3.3 | 55.8
Phase ID | 15 | 16 | 17 | 18 | 19 | 20 | 21 | 22 | 23 | 24 | 25 | 26 | 27 | -
2D-CNN-LSTM | 49.7 | 38.5 | 74.1 | 72.5 | 53.8 | 75.6 | 26.9 | 13.2 | 76.1 | 59.6 | 33.1 | 0.3 | 0.0 | -
3D-ResNet | 31.9 | 31.2 | 67.4 | 90.1 | 70.7 | 51.4 | 53.4 | 30.8 | 49.0 | 27.8 | 71.3 | 5.7 | 3.8 | -
ECO | 32.0 | 23.2 | 76.2 | 91.0 | 54.9 | 62.0 | 10.6 | 25.4 | 88.6 | 30.0 | 64.0 | 4.0 | 0.0 | -

Figure 8: Confusion matrix for inference output from 2D-CNN-LSTM trained with
difference annotations. The unit of each cell means one input clip in each
surgical phase. Figure 9: Confusion matrix for inference output from 3D-CNN
trained with difference annotations. The unit of each cell means one input
clip in each surgical phase. Figure 10: Confusion matrix for inference output
from ECO trained with difference annotations. The unit of each cell means one
input clip in each surgical phase. Figure 11: Visualization on the time axis
for inference output from 2D-CNN-LSTM. In all graphs, orange is the ground
truth label and blue is the inference output. Figure 12: Visualization on the
time axis for inference output from 3D-CNN. In all graphs, orange is the
ground truth label and blue is the inference output. Figure 13: Visualization
on the time axis for inference output from ECO. In all graphs, orange is the
ground truth label and blue is the inference output. Figure 14: Confusion
matrix for inference output from difference models in robotic subtotal
gastrectomy. The unit of each cell means one input clip in each surgical
phase. Figure 15: Visualization on the time axis for inference output for
robotic subtotal gastrectomy. In all graphs, orange is the ground truth label
and blue is the inference output. Figure 16: Deviation of mean AP by model in
each split and surgical phase of laparoscopic cholecystectomy. Dark bars
indicate total AP and light bars indicate deviation from Total AP. Figure 17:
Deviation of mean AP by model in each split and surgical phase of robotic
subtotal gastrectomy. Dark bars indicate Total AP and light bars indicate
deviation from total AP.


# MUSE-ALMA Haloes X: The stellar masses of gas-rich absorbing galaxies

Ramona Augustin,1,2 Céline Péroux,3,4 Arjun Karki,5 Varsha Kulkarni,5 Simon
Weng,3,6,7,8 A. Hamanowicz,1 M. Hayes,9 J. C. Howk,10 G. G. Kacprzak,11,7 A.
Klitsch,12 M. A. Zwaan,3 A. Fox,13,14 A. Biggs,3 A. Y. Fresco,15 S. Kassin,1
H. Kuntschner3
1 Space Telescope Science Institute, 3700 San Martin Drive, Baltimore, MD
21218, USA
2 Leibniz-Institut für Astrophysik Potsdam (AIP), An der Sternwarte 16, 14482
Potsdam, Germany
3 European Southern Observatory (ESO), Karl-Schwarzschild-Str. 2, 85748
Garching bei München, Germany
4 Aix Marseille Université, CNRS, LAM (Laboratoire d’Astrophysique de
Marseille) UMR 7326, 13388, Marseille, France
5 Department of Physics and Astronomy, University of South Carolina, Columbia,
SC 29208, USA
6 Sydney Institute for Astronomy, School of Physics A28, University of Sydney,
NSW 2006, Australia
7 ARC Centre of Excellence for All Sky Astrophysics in 3 Dimensions (ASTRO 3D)
8 ATNF, CSIRO Space and Astronomy, PO Box 76, Epping, NSW 1710, Australia
9 Stockholm University, Department of Astronomy and Oskar Klein Centre for
Cosmoparticle Physics, AlbaNova University Centre, SE-10691, Stockholm, Sweden
10 Department of Physics, University of Notre Dame, Notre Dame, Indiana 46556,
USA
11 Centre for Astrophysics and Supercomputing, Swinburne University of
Technology, Hawthorn, Victoria 3122, Australia
12 DARK, Niels Bohr Institute, University of Copenhagen, Jagtvej 128, 2200
Copenhagen, Denmark
13AURA for ESA, Space Telescope Science Institute, 3700 San Martin Drive,
Baltimore, MD 21218
14 Department of Physics & Astronomy, Johns Hopkins University, 3400 N.
Charles Street, Baltimore, MD 21218, USA
15 Max-Planck-Institut für Extraterrestrische Physik (MPE), Giessenbachstrasse
1, D–85748 Garching, Germany
E-mail<EMAIL_ADDRESS>
(Accepted XXX. Received YYY; in original form ZZZ)

###### Abstract

The physical processes by which gas is accreted onto galaxies, transformed
into stars and then expelled from galaxies are of paramount importance to
galaxy evolution studies. Observationally constraining each of these baryonic
components in the same systems however, is challenging. Furthermore,
simulations indicate that the stellar mass of galaxies is a key factor
influencing CGM properties. Indeed, absorption lines detected against
background quasars offer the most compelling way to study the cold gas in the
circumgalactic medium (CGM). The MUSE-ALMA Haloes survey is composed of quasar
fields covered with VLT/MUSE observations, comprising 32 H i absorbers at 0.2
$<$ $z$ $<$ 1.4 and 79 associated galaxies, with available or upcoming
molecular gas measurements from ALMA. We use a dedicated 40-orbit HST UVIS and
IR WFC3 broad-band imaging campaign to characterise the stellar content of
these galaxies. By fitting their spectral energy distribution, we establish
they probe a wide range of stellar masses: 8.1 $<$ log($M_{*}$/M⊙) $<$ 12.4.
Given their star-formation rates, most of these objects lie on the main
sequence of galaxies. We also confirm a previously reported anti-correlation
between the stellar masses and CGM hydrogen column density N(H i), indicating
an evolutionary trend where higher mass galaxies are less likely to host large
amounts of H i gas in their immediate vicinity up to 120 kpc. Together with
other studies from the MUSE-ALMA Haloes survey, these data provide stellar
masses of absorber hosts, a key component of galaxy formation and evolution,
and observational constraints on the relation between galaxies and their
surrounding medium.

###### keywords:

galaxies: evolution – galaxies: stellar content – quasars: absorption lines

††pubyear: 2015††pagerange: MUSE-ALMA Haloes X: The stellar masses of gas-rich
absorbing galaxies–A

##  1 Introduction

One of the key questions in galaxy evolution is how galaxies interact and
connect to their immediate environment, called the circumgalactic medium (CGM,
Tumlinson et al., 2017). Once stars are formed, galaxies expel ionising
photons and heavy elements formed in stars and supernovae into their
surrounding environment through galactic winds (Pettini et al., 2008; Shull et
al., 2014). Any gas flows into and out of the galaxy traverse through this
extended gas halo and therefore the CGM is closely connected to the galaxy’s
evolution (Muratov et al., 2015; Fox & Davé, 2017). A detailed investigation
of gas inflows and outflows is of paramount importance for understanding these
processes. Since gas, stars, and metals are intimately connected, gas flows
affect the history of star formation and chemical enrichment in galaxies.
Therefore the study of the multi-phase CGM (extending over hundreds of kpc
around galaxies - Shull et al. 2014) is crucial for understanding the star
formation and galaxy evolution as a whole.

However, determining what drives the physical processes at play in the CGM
still remains a complex problem in galaxy evolution, in large part due to the
lack of significant observational constraints. Direct observations of this gas
halo are challenging as the gas is extended and diffuse and the expected
emission from the CGM is faint and mostly below current instrument detection
limits (Augustin et al., 2019; Corlies et al., 2020).

In order to effectively explore the diffuse gas in the Universe, absorption-
line spectroscopy proves to be a valuable method, as its detection sensitivity
remains unaffected by redshift. One particularly compelling approach is to
detect absorption lines in the spectra of bright background quasars, which
allows for a comprehensive study of the distribution, chemical properties, and
kinematics of CGM gas (e.g. Prochaska et al. 2005; Tumlinson et al. 2013).

Typically, strong (log [$N$(H I)/cm-2]$\geq$18) absorbers observed in the
lines-of-sight to high redshift quasars serve as suitable targets to be
followed up with additional imaging and spectroscopy to determine its relation
to galaxies. A comprehensive understanding of the baryon cycle can be achieved
by combining quantitative data on stellar properties with the already
established gaseous measurements. A significant number of studies have
successfully identified hosts to so-called DLAs (Damped Lyman-$\rm\alpha$
systems) and sub-DLAs (Péroux et al., 2003), which are the strongest H i
absorbers (Augustin et al., 2018; Rhodin et al., 2018; Krogager et al., 2017).
Due to their high column density these absorbers host a significant amount of
neutral gas (Péroux & Howk, 2020) and are therefore believed to be closely
connected to their host galaxies. This makes them ideal targets to study the
relation between galaxies and their CGM.

Recently, 3D Integral Field Spectroscopy (IFS), which produces data cubes
where each pixel on the image has a spectrum, has provided a novel technique
to examine the gas in absorption against background sources whose lines of
sight pass through a galaxy’s CGM. Building on early efforts with Near-
Infrared (NIR) Integral Field Spectrograph (IFS) SINFONI (Spectrograph for
INtegral Field Observations in the Near Infrared) on the Very Large Telescope
(VLT) (Bouché et al., 2007; Péroux et al., 2011, 2013; Péroux et al., 2016;
Augustin et al., 2018), the potential of this technique for studying the CGM
with the optical IFU VLT/MUSE (Multi Unit Spectroscopic Explorer) has been
demonstrated (Schroetter et al., 2016; Bouché et al., 2016; Zabl et al., 2019;
Fumagalli et al., 2016; Muzahid et al., 2020; Dutta et al., 2021; Berg et al.,
2022). Using this technique, a novel survey has been designed over the past
years, combining IFU data from VLT/MUSE with ALMA and HST data of 19 quasar
fields, labeled MUSE-ALMA haloes (Hamanowicz et al., 2020; Péroux et al.,
2022; Weng et al., 2023a; Karki et al., 2023). Within these fields, 79
galaxies were identified to be associated within 500 km/s of strong quasar
absorbers at $z\sim$ 1 (Péroux et al., 2022). By probing the neutral,
molecular and ionised gas around these galaxies, the physical properties of
the CGM have been characterised from the combination of abundance
determination and kinematics through Voigt profile fitting of the absorber and
emission-line and stellar continuum luminosities of the host galaxies
(Augustin et al., 2018; Klitsch et al., 2018; Péroux et al., 2019; Hamanowicz
et al., 2020).

The goal of the present work is to compute the stellar mass of galaxies
related to gas-rich absorbers and understand what population of galaxies is
traced by those absorbers. Particularly, we want to revisit the previously
identified stellar mass - H i column density anticorrelation (Augustin et al.,
2018) and fit its slope. Additionally, we determine whether the galaxies that
are found as absorber hosts have physical properties which differ from the
general population of galaxies, whether they are more star forming or quenched
than the general galaxy population or whether they are following the same
scaling relations as the general galaxy population. Particularly, Péroux et
al. (2020) have shown that the metal distribution around galaxies is
anisotropic. Their findings indicate that the metallicity trend with azimuthal
angle is strongly dependent on the stellar mass of the host, indicating that
the stellar mass is a critical component to the structure and evolution of
galactic haloes. In this work we determine the stellar masses of galaxies
associated with absorbers, providing essential information on these galaxies
to better understand their relation to the gas seen in absorption.

The manuscript is organized as follows: Section 2 presents the observations
used in this study. Section 3 details the process of fitting Spectral Energy
Distribution to the data, while Section 4 relates the stellar properties of
these galaxies with their CGM characteristics. We summarize and conclude in
Section 5. Here, we adopt an $H_{0}$ = 67.74 km s-1 Mpc-1, $\Omega_{M}$ =
0.3089, and $\Omega_{\Lambda}$ = 0.6911 cosmology.

##  2 MUSE-ALMA Haloes Observations

###  2.1 Survey overview and relevant data

Integral Field Units (IFUs) have opened a new era in establishing the relation
between absorption and emission. The optical IFU VLT/MUSE (Bacon et al., 2010)
has proven to be a true game-changer in the field. The MUSE-ALMA Haloes survey
probes the multi-phase CGM gas of intermediate redshift galaxies. The main
goal of the survey is to reveal and understand the physical processes
responsible for the transformation of baryons in galaxies (Péroux et al.,
2022). The survey is based on a unique selection of known quasar absorbers
with measured H i column density log [$N$(H I)/cm-2]$\geq$18, from HST UV
spectroscopy with resolutions of $R$=20,000–30,000. Weng et al. (2023a) have
measured metal lines from optical spectroscopy and analysed the physical and
emission-line properties of 79 galaxies associated with 32 H i absorbers at
redshift $0.20\lesssim z\lesssim 1.4$. These associated galaxies are selected
to have velocities $\leq 500$ $\rm kms^{-1}$ relative to the absorber
redshift. They are found at impact parameters of $\sim$5.7-100 kpc at the
lower redshifts and up to 270 kpc at the higher redshifts, where the upper
cut-off reflects the field-of-view of MUSE. The star-formation rates (SFRs) of
associated galaxies are measured using the H$\alpha$ emission line when
available. For sources at redshift $z$ $\geq$ 0.4, where H$\alpha$ is not
observable with MUSE, Weng et al. (2023a) estimate the SFR using the [O ii]
luminosity. For galaxies with dust corrections available, they also provide a
dust-corrected SFR. 3-$\sigma$ SFR limits are calculated for non-detections.
In the resulting sample, there are both passive galaxies without detectable
emission lines, and star-forming galaxies with SFRs up to 15 $\rm
M_{\odot}yr^{-1}$.

###  2.2 HST Broad-band Imaging Data

MUSE-ALMA Haloes also includes broad-band imaging of all but one field in the
sample. This includes HST WFC3 imaging data in selected near-UV and optical
filters, combined with archival Ultra-Violet, optical and Infra-Red WFPC2 and
WFC3 imaging data to investigate the immediate surroundings of the quasar
absorbers (Péroux et al., 2022). The filters for the observations are
carefully designed to surround the 4000 Å break at each absorber’s redshift.
For the majority (16/19) of the fields images in three broad-band filters are
available. Using additional data available in the HST archive, we cover a
total of four filters for some of the targets. The MUSE IFU observations of
the quasar fields are used to design the HST observations. We identified [OII]
emission from galaxies at the redshift of the known absorbers as potential
host galaxies and determined their location with respect to the quasar. The
locations of these galaxies are used to optimize the HST pointings such that
the diffraction spikes from the HST optics would not spatially overlap with
the galaxies, while simultaneously keeping the whole MUSE FOV within the HST
field of view. We then performed a careful quasar PSF subtraction, following
Augustin et al. (2018), and photometry measurements (as described in detail in
Péroux et al. 2022; Karki et al. 2023). The Astropy package Photutils is used
on the processed HST broad-band images in each filter for each field to search
for all objects and to perform photometry of the detected objects. The details
are described in Karki et al. (2023). In cases where an object detected in one
filter was not detected in another filter, a 3-$\sigma$ magnitude limit was
calculated for the filter with the non-detection by measuring the 3-$\sigma$
noise level within the same aperture as for the detection. The resulting
magnitudes are publicly released with Péroux et al. (2022) and are used for
the SED fitting described below.

##  3 Spectral Energy Distribution fitting

Building on the successes of earlier multi-band approaches (e.g. Christensen
et al. 2014; Augustin et al. 2018), we perform Spectral Energy Distribution
(SED) fits of the galaxies associated with the absorbers. The wavelength
coverage and SNR of the continuum emission of the galaxies in question in the
VLT/MUSE observations alone does not provide sufficient data for state-of-the-
art SED algorithms to converge for the redshifts we are probing. Instead, the
new HST data cover wavelengths around the 4000 Å break and are designed to
detect the stellar continuum of absorber counterparts in order to probe the
stellar population of these galaxies. The coverage of this break significantly
improves the accuracy of the stellar mass estimates, so we ensure that we
obtain observations on both sides of the break. Additionally, the strength of
the break directly affects the ability to measure the stellar masses
effectively. We note that, highly star-forming galaxies have a less defined
break than quenched galaxies. Therefore the masses of quenched galaxies are
determined more robustly by the fitting procedure.

We use the Le Phare (Arnouts et al., 1999; Ilbert et al., 2006) SED fitting
code to determine the physical properties of the observed galaxy sample. We
assume the Bruzual & Charlot (2003) SED library, a Calzetti et al. (1994) dust
extinction law and a Chabrier (2003) initial mass function. We focus here on
the galaxies whose redshifts have been measured with VLT/MUSE spectroscopy and
are known to be within 500 $\rm kms^{-1}$ from the strong H i absorbers. The
resulting sample comprises 79 galaxies for 31 absorbers (see Table 1). To
perform the SED fitting, we use the reliably determined magnitudes based on
the high spatial resolution HST imaging. Some of the faint objects are
detected in VLT/MUSE through their line-emission signatures, but not in
continuum in the broad-band HST images, particularly in the bluest filters. In
cases where galaxies had no detection in any HST filter, we make use of the
photometry catalogs from the Dark Energy Legacy Survey (DECaLS, Dey et al.
2019) to perform the SED fitting. These systems are labeled as "LEGACY" in the
last column of table 1. For galaxies with detections in one of the three
bands, the SED fitting provides less robust stellar mass estimates, dubbed
"probable". For illustration, we show in the Appendix examples of SED fits for
a robust and a probable system (Fig. 4). These estimates are included in our
sample for completeness, while we highlight the more reliable stellar mass
measurements. We therefore focus here on the 34 galaxies with the most robust
fits.

Figure 1: Distribution of stellar masses of absorber host galaxies: We find a
large spread of stellar masses for absorber-associated galaxies, with
log[$M_{*}$/M⊙]= 7.8 – 12.4.

Figure 1 displays the distribution of stellar masses resulting from the SED
fits. The grey histogram indicates the possible $M_{*}$ determinations where
fewer than two photometry measurements are available while the red histogram
shows the more robust determinations where 2-4 photometry measurements are
available. The derived stellar masses cover a broad range from log
[$M_{*}$/M⊙]= 7.8 – 12.4, which illustrates the diversity of these systems. We
see that the galaxies with stellar masses above log[$M_{*}$/M⊙]= 10 have their
SED fit typically robustly determined while the lower mass galaxies often have
non-detections in the bluer filters, limiting their stellar mass determination
and potentially biasing our detection numbers and robust mass determinations
towards higher masses.

We additionally run the Le Phare SED fitting code on all galaxies in the HST
fields regardless of whether or not they have spectroscopic redshifts, in
order to assess the potential of determining photometric redshifts with SED
fitting. We derive uncertainties of the order $\delta z\geq$0.5 in redshift,
so we deem the results for such objects not robust enough to identify further
absorber hosts beyond the ones detected through spectroscopy in MUSE cubes.

##  4 stellar properties of absorber hosts

Figure 2:  Physical properties of the galaxies associated with gas-rich
absorbers. In all these figures we denote robust stellar mass measurements in
red and probable measurements in grey. The closest impact parameter galaxy of
any system is marked with a diamond instead of a circle. Upper left: The
redshift distribution of the determined stellar masses. We probe a range of
stellar masses within 0.2$\leq$z$\leq$1.2, although there is a slight trend of
finding the lower mass systems preferentially at lower redshifts. Upper right:
The impact parameter range probed by the absorber hosts. We find no
correlation between stellar masses and impact parameters. We consider the
galaxy closest to the quasar sightline to be the main absorber host (diamond).
The impact parameter range for these is up to $\sim$ 120 kpc. Bottom left: We
show how the absorber associated galaxies in our sample fall into the stellar
mass - SFR plane in comparison to the main sequence at a matching redshift
range (Belfiore et al., 2018). We establish that most of these galaxies are
normal star-forming galaxies and follow the main sequence. Some SFR
measurements are found below the main sequence and could indicate quenching,
particularly at the higher mass end. Bottom right: Same as the Figure on the
left, but showing the specific SFR (SFR over stellar mass) instead of absolute
SFR, hightlighing the deviation of the higher mass galaxies from the star
forming main sequence.

Figure 3:  Left: The relation between H i column density of the absorber
versus the stellar masses of galaxies associated with those absorbers. The
literature data are taken from Augustin et al. (2018); Rhodin et al. (2018)
and Christensen et al. (2014). We find and quantify for the first time the
anticorrelation between H i absorption strength and stellar mass of the main
host galaxy (dotted line). The fit was performed on the stars and red diamond
data points in the plot. Right: Summary of all measured quantities in this
work. We show the impact parameter scaled with virial radius of each system
versus the neutral gas H i column density. The color of each circle indicates
the stellar mass of that object and the size of the circle is proportional to
the star formation rate of this galaxy. We find that the galaxies associated
with high column density H i absorbers span a range of SFR and stellar mass.
As explored in the panel on the left, the highest column density systems seem
to be hosted in groups of galaxies with smaller stellar masses, while the
higher stellar mass objects are found at lower column densities.
Simultaneously, at the higher column density end the host galaxies show
relatively low SFRs compared to the galaxies found at the lower column density
end of our sample.

###  4.1 Dependence of SFR, redshift and impact parameter on M∗

First we investigate how the determined stellar masses depend on other
properties of the galaxy that are connected to its evolution.

In the upper left panel of Figure 2 we show the dependency of the determined
stellar mass on the redshift of the galaxy. Given that galaxies grow and
evolve over time we expect to find generally more massive galaxies at lower
redshifts and less massive galaxies at higher redshifts. However, at the same
time, due to surface brightness dimming, we expect to find preferentially
bright and therefore massive galaxies at higher redshift and a range over all
stellar masses at lower redshifts. What we find in our sample is a mix of
those effects. Overall we find a large scatter of stellar masses at all
redshifts, however we do notice a lack of lower mass systems at redshifts
higher than $z$ $\geq$ 0.7.

In the upper right panel of Figure 2, we investigate the dependency of the
associated host stellar mass on the impact parameter to the absorber. We find
no correlation between these two quantities, indicating that galaxies of any
mass can be found at any distance to a known absorber. The diamonds indicate
the sample of galaxies with lowest impact parameter for each absorption
system, which are thought to be the main absorber hosts sample. Here we see
that the maximum impact parameter we probe with this sample of main absorber
hosts is $\sim$120 kpc. Following Behroozi et al. (2010) and Read et al.
(2017), we approximate the halo mass of the galaxies in our sample as $\rm
M_{halo}\approx 100\times M_{\star}$. The dashed line in Figure 2 shows the
resulting virial radius, assuming a standard Navarro–Frenk–White (NFW) profile
(Navarro et al., 1996). We find that particularly at the lower mass end, a
significant fraction of galaxies associated with the absorber lie outside the
virial radius. As in any absorber-host-connection study, there is a
possibility that the closest galaxy contributing most the absorbing gas lies
directly close to the projected position of the background quasar and/or is
too faint to be detected. Weng et al. (2024) have recently quantifies this
effect using hydrodynamical cosmological TNG50 simulations. Therefore, we
cannot fully exclude the possibility of missing faint galaxies close to the
quasar sightline.

In the bottom row (left and right panels) of Figure 2, we show the
measurements for stellar mass from SED fitting and SFR (left) and specific SFR
(right) from emission lines in MUSE (H$\rm\alpha$ and [OII], Weng et al.,
2023a) compared to the expected main-sequence relation at $z$=0.5 (Belfiore et
al., 2018). We find that on average our sample of galaxies follows the main
sequence, consistent with results of previous studies (e.g., Kulkarni et al.
2022). However, at higher masses the galaxies fall slightly below the main
sequence. We recall that a fraction (17 systems) of the SFR measurements is
dust corrected, specifically those with H$\rm\alpha$ and H$\rm\beta$
measurements, corresponding to systems with $z$ $<$ 0.4 (see Weng et al.,
2023a). Therefore estimates of SFR in the low-resdhift galaxies represent a
lower limit on the total SFR. In particular, if the highest mass galaxies with
log([$M_{*}$/[M⊙])$>$ 11 were dust corrected, they might then well follow the
main sequence. We furthermore note that the combined sample of H$\rm\alpha$
and [OII] determined SFRs may introduce a larger scatter in the SFR
distributions than a more homogeneous sample would. We note that the main
sequence should also flatten at the higher mass end of the mass distribution
(Tomczak et al., 2016). Therefore we conclude that the sample of galaxies is
representative of normal star-forming galaxies. Langan et al. (2023) and Guha
& Srianand (2023) find a similar result for Mg ii absorption selected systems,
indicating that they follow the main sequence. Together with our findings,
this indicates that absorber host galaxies follow the trends of the typical
galaxies population.

###  4.2 Correlation between M∗ and CGM atomic gas content

One of the early findings of relating the H i absorption properties with the
host galaxy properties was an apparent anti-correlation between stellar mass
of the host galaxy and the absorption H i column density (Augustin et al.,
2018). Such an anti-correlation seems counter-intuititive and may be caused by
additional factors such as measuring the impact parameter between the galaxy
and the absorber, star formation rate of the host, or the environment of the
galaxy. While previous studies were characterising somewhat limited sample
sizes, we make use here of the currently complete MUSE-ALMA Haloes survey
sample, covering quasar fields with 32 known absorbers and 79 associated
galaxies with those absorbers, out of which we have robust masses for 34
galaxies. We are therefore testing this relation over a larger redshift and
column density range and in more environments than previously possible.

In the left panel of Figure 3 we show for our sample the relation between the
stellar mass of the host and the detected H i column density in its CGM. While
previous studies that found this anti-correlation (Augustin et al., 2018;
Rhodin et al., 2018) considered objects where each absorber was associated
with only one - typically the closest - galaxy, here we are considering more
complex systems, including groups of galaxies associated with a strong H i
absorber. The reason for the single detections in the literature data,
particularly the ones using SINFONI (Augustin et al., 2018) is the somewhat
limited field-of-view around the quasar. To simplify and homogenize this
investigation and make it comparable to previous studies we consider here the
galaxy at the closest impact parameter, marked with a diamond, to be the main
absorber host galaxy. The reason behind choosing the closest galaxy in impact
parameter is the assumption that the physically closest galaxy to the gas we
are probing in absorption will have the largest contribution to the absorption
signal of all the galaxies within a given group (Weng24).

Now, we investigate this previously identified anti-correlation between
stellar mass and circumgalactic H i column density for the larger sample. We
also add the data points from Christensen et al. (2014) for comparison and
note that all of these previous DLA host galaxy studies, have a larger average
and greater spread in redshift, whereas our sample is at a slightly lower
redshift on average.

Assuming an anti-correlation power law between stellar mass and H i column
density, we fit the relation. We note that the upper limits were not
considered in the fit. The data points used for the fit are marked with black
dots in the left panel of Figure 3. For the fit we consider robust $M_{*}$
measurements of the closest impact parameter galaxies (red data points with
diamond) as well as the literature samples from Christensen et al. (2014),
Augustin et al. (2018) and Rhodin et al. (2018). These literature samples
consist of single counterparts detections of DLAs and subDLAs and showed a
tentative trend for this anti-correlation, which we now have enough data
points to actually measure. Our choice of considering the closest galaxy in
line of sight to the quasar homogenizes the chosen sample for the fit of the
anticorrelation. We perform a Pearson correlation test and find a value of
$-$0.57 thus confirming the anticorrelation. Interestingly, we note that the
trend is driven by the literature subsample. When focussing on the new
measurements from this work, we find a Pearson coefficient of $-$0.36,
indicating little anticorrelation. We fit a simple power-law to the relation
in logarithmic space:

$\rm log(\textit{N}_{HI}/[cm^{-2}])=-0.51(\pm 0.1)\times
log(\textit{M}_{*}/[M_{\odot}])+25.0(\pm 1.2)$ (1)

The lack of high-mass high-$\rm\textit{N}_{HI}$ systems could be due to an
observational bias or selection effect where high-$\rm\textit{N}_{HI}$, high
stellar mass systems are artificially excluded. Indeed, a
high-$\rm\textit{N}_{HI}$, high stellar mass system could arise in the center
of a massive, potentially dusty galaxy, causing a reddening of the background
quasar, which we would miss from typical quasar optical selection (Vladilo &
Péroux, 2005).

However, assuming this anti-correlation, or rather, upper envelope in detected
$\rm\textit{N}_{HI}$ \- stellar mass values has a physical origin, it holds
information on the composition and evolution of galactic haloes. The lack of
high-mass high-$\rm\textit{N}_{HI}$ systems can intuitively be explained by
the ambient temperature of higher mass galaxies being typically higher than in
lower mass galaxies, due to stronger feedback heating up the halo (see e.g.
Suresh et al. 2017). This hotter gas may prevent the survival of cooler gas
clouds hosting HI, at least within the inner $\sim$ 120 kpc as probed by our
sample of closest impact parameter hosts. Likewise, the higher mass galaxies
have undergone a rapid star forming process and already used up their cool H i
gas reservoir, whereas lower mass galaxies take longer to use their gas
supplies and still maintain more of it in their CGM. As we probe lower column
density systems, the spread in stellar mass of associated galaxies increases,
with the mean shifting towards higher masses. This means that we can find
lower column density systems around a variety of galaxies but preferentially
around higher mass systems as compared to the high $\rm\textit{N}_{HI}$
absorbers. In a lower column density regime, Berg et al. (2022) find a scatter
of stellar masses log([$M_{*}$/[M⊙])$\sim$ 8.5 – 12 for absorber systems with
$N_{\text{H\,{i}}}$ $<$ $10^{17}$ $\rm cm^{-2}$ without any clear trend.
Similarly Chen et al. (2019) find column densities of $\sim$ $10^{18}$ $\rm
cm^{-2}$ around galaxies with log([$M_{*}$/[M⊙])$\sim$ 7.5 – 11.5. A reason
for this phenomenon could be that more massive galaxies reside in larger
haloes, providing a larger cross section for low column density absorption
systems than lower mass galaxies. Our results are also in line with Khare et
al. (2007) and Kulkarni et al. (2010) who argue that sub-DLAs hosts are more
massive than DLA hosts. Finally, Dutta et al. (2023) recently found that
Ly$\rm\alpha$ rest-frame equivalent widths show a peak at
log([$M_{*}$/[M⊙])$\sim$ 9 and decline at both high and low masses. They
attributed this trend to high virial temperature and efficient feedback in
high mass halos. Since most of the galaxies in the present study have
log([$M_{*}$/[M⊙])$>$ 9, and while we considered column densities rather than
equivalent widths, our observed anti-correlation is aligned with their
results.

Taken together, these findings suggest an evolutionary trend of the CGM
composition with stellar mass where lower mass systems host haloes that are
abundant with cool and dense H i gas and higher mass systems are more depleted
of cool gas. However, even the more massive galaxies are fairly normally star
forming, given the depletion of fuel in their immediate surroundings. More
detailed studies of the gas content around galaxies and the galaxy properties
such as stellar mass and star-formation rate but also metallicity, can measure
the accretion timescales. These timescales describe the gas flows from the
halo onto the disk and their transformation into stars versus the timescales
for heating up the halo through feedback from a growing galaxy and depleting
the gas fuel.

##  5 Conclusions

In this work, we have performed stellar mass measurements of 79 0.2$<$z$<$1.4
galaxies known to have velocities within 500 $\rm kms^{-1}$ from a strong N(H
i) absorber. To this end, we have used Le Phare SED fitting on multi-broad-
band HST imaging of the fields to robustly determine the stellar masses of the
subset of 34 galaxies that were detected in at least two imaging bands.

Our main results are:

  * •

We find that galaxies associated with high H i column density absorbers span a
large range in stellar masses of log([$M_{*}$/[M⊙])= 8.1 – 12.4 at
0.2$<$z$<$1.4. There is no siginificant trend with redshift or impact
parameter. All these galaxies are normal star-forming galaxies following the
expected main-sequence relation.

  * •

A previously tentative detection of an anti-correlation between the stellar
mass and CGM H i column density has been confirmed by the larger sample of
absorber-host pairs, indicating an evolutionary trend of cool gas depletion in
the CGM with stellar mass.

Ultimately, connecting the CGM gas properties of galaxies with their stellar
content provides fresh clues on the baryon cycle, a key component of galaxy
formation and evolution. In particular, knowledge of the stellar masses of
galaxies allows to perform studies of the azimuthal metallicity distribution
(Weng et al., 2023b). Further studies of the detailed chemical composition of
absorber host galaxies may reveal the timescales involved in gas accretion and
recycling. Cosmological simulations are now challenged to resolve both the
cool gas in the CGM as well as the properties of the host galaxies in order to
provide physical ground to our findings and make predictions to the galactic
baryon cycle that can be tested by such observations.

## Acknowledgements

The authors would like to thank Dylan Nelson, Max Pettini and Jason Tumlinson
for useful discussions and constructive feedback on this work. This research
was supported by the International Space Science Institute (ISSI,
https://www.issibern.ch/) in Bern, through ISSI International Team project
#564 (The Cosmic Baryon Cycle from Space). RA acknowledges financial support
from the STScI Director’s Discretionary Research Fund (DDRF) and funding by
the European Research Council through ERC-AdG SPECMAP-CGM, GA 101020943. AK
and VPK acknowledge support from a grant from the Space Telescope Science
Institute for GO program 15939 (PI: Péroux), and additional partial support
from US National Science Foundation grant AST/2007538 and NASA grant
80NSSC20K0887 (PI: Kulkarni). SW acknowledge the financial support of the
Australian Research Council through grant CE170100013 (ASTRO3D). GGK
acknowledges the support of the Australian Research Council through the
Discovery Project DP170103470. A.K. gratefully acknowledges support from the
Independent Research Fund Denmark via grant number DFF 8021-00130.

In this work we used the following python packages: Numpy (Harris et al.,
2020); Matplotlib (Hunter, 2007); pandas (Wes McKinney, 2010; pandas
development team, 2020); AstroPy (The Astropy Collaboration et al., 2018);
Halotools (Hearin et al., 2017)

## Data Availability

All data presented in this work is publicly available or available upon
request.

## References

  * Arnouts et al. (1999) Arnouts S., Cristiani S., Moscardini L., Matarrese S., Lucchin F., Fontana A., Giallongo E., 1999, MNRAS, 310, 540
  * Augustin et al. (2018) Augustin R., et al., 2018, MNRAS, 478, 3120
  * Augustin et al. (2019) Augustin R., et al., 2019, MNRAS, 489, 2417
  * Bacon et al. (2010) Bacon R., et al., 2010, in McLean I. S., Ramsay S. K., Takami H., eds, Society of Photo-Optical Instrumentation Engineers (SPIE) Conference Series Vol. 7735, Ground-based and Airborne Instrumentation for Astronomy III. p. 773508, doi:10.1117/12.856027
  * Behroozi et al. (2010) Behroozi P. S., Conroy C., Wechsler R. H., 2010, ApJ, 717, 379
  * Belfiore et al. (2018) Belfiore F., et al., 2018, Monthly Notices of the Royal Astronomical Society, 477, 3014
  * Berg et al. (2022) Berg M. A., et al., 2022, arXiv e-prints, p. arXiv:2204.13229
  * Bouché et al. (2007) Bouché N., Murphy M. T., Péroux C., Davies R., Eisenhauer F., Förster Schreiber N. M., Tacconi L., 2007, ApJ, 669, L5
  * Bouché et al. (2016) Bouché N., et al., 2016, ApJ, 820, 121
  * Bruzual & Charlot (2003) Bruzual G., Charlot S., 2003, MNRAS, 344, 1000
  * Calzetti et al. (1994) Calzetti D., Kinney A. L., Storchi-Bergmann T., 1994, ApJ, 429, 582
  * Chabrier (2003) Chabrier G., 2003, PASP, 115, 763
  * Chen et al. (2019) Chen H.-W., et al., 2019, MNRAS, 484, 431
  * Christensen et al. (2014) Christensen L., Møller P., Fynbo J. P. U., Zafar T., 2014, MNRAS, 445, 225
  * Corlies et al. (2020) Corlies L., Peeples M. S., Tumlinson J., O’Shea B. W., Lehner N., Howk J. C., O’Meara J. M., Smith B. D., 2020, ApJ, 896, 125
  * Dey et al. (2019) Dey A., et al., 2019, AJ, 157, 168
  * Dutta et al. (2021) Dutta R., et al., 2021, MNRAS, 508, 4573
  * Dutta et al. (2023) Dutta S., Muzahid S., Schaye J., Mishra S., Chen H.-W., Johnson S., Wisotzki L., Cantalupo S., 2023, arXiv e-prints, p. arXiv:2303.16933
  * Fox & Davé (2017) Fox A., Davé R., eds, 2017, Gas Accretion onto Galaxies Astrophysics and Space Science Library Vol. 430, doi:10.1007/978-3-319-52512-9.
  * Fumagalli et al. (2016) Fumagalli M., Cantalupo S., Dekel A., Morris S. L., O’Meara J. M., Prochaska J. X., Theuns T., 2016, MNRAS, 462, 1978
  * Guha & Srianand (2023) Guha L. K., Srianand R., 2023, MNRAS, 519, 3319
  * Hamanowicz et al. (2020) Hamanowicz A., et al., 2020, MNRAS, 492, 2347
  * Harris et al. (2020) Harris C. R., et al., 2020, Nature, 585, 357
  * Hearin et al. (2017) Hearin A. P., et al., 2017, AJ, 154, 190
  * Hunter (2007) Hunter J. D., 2007, Computing in Science & Engineering, 9, 90
  * Ilbert et al. (2006) Ilbert O., et al., 2006, A&A, 457, 841
  * Karki et al. (2023) Karki A., et al., 2023, MNRAS, 524, 5524
  * Khare et al. (2007) Khare P., Kulkarni V. P., Péroux C., York D. G., Lauroesch J. T., Meiring J. D., 2007, A&A, 464, 487
  * Klitsch et al. (2018) Klitsch A., Péroux C., Zwaan M. A., Smail I., Oteo I., Biggs A. D., Popping G., Swinbank A. M., 2018, MNRAS, 475, 492
  * Krogager et al. (2017) Krogager J. K., Møller P., Fynbo J. P. U., Noterdaeme P., 2017, MNRAS, 469, 2959
  * Kulkarni et al. (2010) Kulkarni V. P., Khare P., Som D., Meiring J., York D. G., Péroux C., Lauroesch J. T., 2010, New Astron., 15, 735
  * Kulkarni et al. (2022) Kulkarni V. P., Bowen D. V., Straka L. A., York D. G., Gupta N., Noterdaeme P., Srianand R., 2022, ApJ, 929, 150
  * Langan et al. (2023) Langan I., et al., 2023, MNRAS, 521, 546
  * Muratov et al. (2015) Muratov A. L., Kereš D., Faucher-Giguère C.-A., Hopkins P. F., Quataert E., Murray N., 2015, MNRAS, 454, 2691
  * Muzahid et al. (2020) Muzahid S., et al., 2020, MNRAS, 496, 1013
  * Navarro et al. (1996) Navarro J. F., Frenk C. S., White S. D. M., 1996, ApJ, 462, 563
  * Péroux & Howk (2020) Péroux C., Howk J. C., 2020, ARA&A, 58, 363
  * Péroux et al. (2003) Péroux C., McMahon R. G., Storrie-Lombardi L. J., Irwin M. J., 2003, MNRAS, 346, 1103
  * Péroux et al. (2011) Péroux C., Bouché N., Kulkarni V. P., York D. G., Vladilo G., 2011, MNRAS, 410, 2237
  * Péroux et al. (2013) Péroux C., Bouché N., Kulkarni V. P., York D. G., 2013, MNRAS, 436, 2650
  * Péroux et al. (2016) Péroux C., et al., 2016, MNRAS, 457, 903
  * Péroux et al. (2019) Péroux C., et al., 2019, MNRAS, 485, 1595
  * Péroux et al. (2020) Péroux C., Nelson D., van de Voort F., Pillepich A., Marinacci F., Vogelsberger M., Hernquist L., 2020, MNRAS, 499, 2462
  * Péroux et al. (2022) Péroux C., et al., 2022, MNRAS, 516, 5618
  * Pettini et al. (2008) Pettini M., Zych B. J., Steidel C. C., Chaffee F. H., 2008, MNRAS, 385, 2011
  * Prochaska et al. (2005) Prochaska J. X., Herbert-Fort S., Wolfe A. M., 2005, ApJ, 635, 123
  * Read et al. (2017) Read J. I., Iorio G., Agertz O., Fraternali F., 2017, MNRAS, 467, 2019
  * Rhodin et al. (2018) Rhodin N. H. P., Christensen L., Møller P., Zafar T., Fynbo J. P. U., 2018, A&A, 618, A129
  * Schroetter et al. (2016) Schroetter I., et al., 2016, ApJ, 833, 39
  * Shull et al. (2014) Shull J. M., Danforth C. W., Tilton E. M., 2014, ApJ, 796, 49
  * Suresh et al. (2017) Suresh J., Rubin K. H. R., Kannan R., Werk J. K., Hernquist L., Vogelsberger M., 2017, MNRAS, 465, 2966
  * The Astropy Collaboration et al. (2018) The Astropy Collaboration et al., 2018, AJ, 156, 123
  * Tomczak et al. (2016) Tomczak A. R., et al., 2016, ApJ, 817, 118
  * Tumlinson et al. (2013) Tumlinson J., et al., 2013, ApJ, 777, 59
  * Tumlinson et al. (2017) Tumlinson J., Peeples M. S., Werk J. K., 2017, ARA&A, 55, 389
  * Vladilo & Péroux (2005) Vladilo G., Péroux C., 2005, A&A, 444, 461
  * Weng et al. (2023a) Weng S., et al., 2023a, MNRAS, 523, 676
  * Weng et al. (2023b) Weng S., et al., 2023b, MNRAS, 523, 676
  * Weng et al. (2024) Weng S., Péroux C., Ramesh R., Nelson D., Sadler E. M., Zwaan M., Bollo V., Casavecchia B., 2024, MNRAS, 527, 3494
  * Wes McKinney (2010) Wes McKinney 2010, in Stéfan van der Walt Jarrod Millman eds, Proceedings of the 9th Python in Science Conference. pp 56 – 61, doi:10.25080/Majora-92bf1922-00a
  * Zabl et al. (2019) Zabl J., et al., 2019, MNRAS, 485, 1961
  * pandas development team (2020) pandas development team T., 2020, pandas-dev/pandas: Pandas, doi:10.5281/zenodo.3509134, https://doi.org/10.5281/zenodo.3509134

ID | Filters | Mass | Notes | ID | Filters | Mass | Notes
---|---|---|---|---|---|---|---
Q0138m0005_14 | 3 | 9.8${}_{-0.2}^{+0.1}$ | HST | Q1130m1449_68 | 1 | 7.8${}_{-0.4}^{+0.4}$ | HST
Q0152m2001_4 | 2 | 11.5${}_{-0.1}^{+0.1}$ | HST | Q1130m1449_76 | 0 | N/A | HST
Q0152m2001_5 | 2 | 11.3${}_{-0.1}^{+0.1}$ | HST | Q1211p1030_7 | 3 | 8.8${}_{-0.1}^{+0.1}$ | HST
Q0152m2001_7 | 2 | 11.0${}_{-0.1}^{+0.1}$ | HST | Q1211p1030_9 | 2 | 9.7${}_{-0.4}^{+0.4}$ | HST
Q0152m2001_12 | 2 | 11.8${}_{-0.4}^{+0.2}$ | HST | Q1211p1030_13 | 3 | 8.7${}_{-0.1}^{+0.1}$ | LEGACY
Q0152m2001_13 | 2 | 10.5${}_{-0.2}^{+0.2}$ | HST | Q1211p1030_16 | 2 | 10.5${}_{-0.5}^{+0.5}$ | HST
Q0152m2001_14 | 3 | 10.5${}_{-0.1}^{+0.1}$ | LEGACY | Q1211p1030_17 | 3 | 11.4${}_{-0.4}^{+0.3}$ | LEGACY
Q0152m2001_62 | 0 | N/A | HST | Q1211p1030_38 | 1 | 8.9${}_{-0.7}^{+0.7}$ | HST
Q0152p0023_7 | 2 | 10.1${}_{-0.1}^{+0.2}$ | HST | Q1211p1030_48 | 0 | N/A | HST
Q0152p0023_13 | 2 | 9.0${}_{-0.1}^{+0.2}$ | HST | Q1211p1030_57 | 0 | N/A | HST
Q0152p0023_20 | 2 | 8.1${}_{-0.1}^{+0.1}$ | HST | Q1211p1030_58 | 0 | N/A | HST
Q0152p0023_23 | 2 | 8.4${}_{-0.1}^{+0.1}$ | HST | Q1229m021_5 | 3 | 9.5${}_{-0.2}^{+0.3}$ | LEGACY
Q0152p0023_44 | 1 | 8.0${}_{-0.6}^{+0.6}$ | HST | Q1229m021_6 | 0 | N/A | HST
Q0420m0127_8 | 3 | 10.3${}_{-0.4}^{+0.2}$ | LEGACY | Q1229m021_8 | 1 | 9.8${}_{-0.7}^{+0.7}$ | HST
Q0420m0127_12 | 3 | 10.8${}_{-0.1}^{+0.1}$ | LEGACY | Q1229m021_10 | 1 | 9.6${}_{-0.7}^{+0.7}$ | HST
Q0420m0127_13 | 1 | 9.7${}_{-0.9}^{+0.9}$ | HST | Q1229m021_13 | 1 | 9.7${}_{-0.7}^{+0.7}$ | HST
Q0420m0127_30 | 1 | 9.1${}_{-0.9}^{+0.9}$ | HST | Q1229m021_29 | 0 | N/A | HST
Q0454m220_4 | 2 | 12.4${}_{-0.1}^{+0.1}$ | HST | Q1229m021_39 | 1 | 8.5${}_{-0.6}^{+0.7}$ | HST
Q0454m220_69 | 3 | 10.8${}_{-0.1}^{+0.1}$ | LEGACY | Q1229m021_40 | 1 | 9.7${}_{-0.7}^{+0.7}$ | HST
Q0454p039_15 | 0 | N/A | HST | Q1229m021_41 | 0 | N/A | HST
Q0454p039_57 | 0 | N/A | HST | Q1342m0035_4 | 3 | 10.1${}_{-0.1}^{+0.1}$ | HST
Q0454p039_65 | 2 | 9.9${}_{-0.8}^{+0.8}$ | HST | Q1342m0035_9 | 2 | 10.9${}_{-0.1}^{+0.2}$ | HST
Q1110p0048_6 | 3 | 9.1${}_{-0.1}^{+0.1}$ | HST | Q1345m0023_13 | 1 | 10.1${}_{-0.6}^{+0.6}$ | HST
Q1110p0048_15 | 1 | 8.8${}_{-0.6}^{+0.6}$ | HST | Q1345m0023_40 | 1 | 8.7${}_{-0.6}^{+0.6}$ | HST
Q1110p0048_44 | 3 | 9.2${}_{-0.3}^{+0.3}$ | HST | Q1431m0050_10 | 3 | 9.8${}_{-0.1}^{+0.1}$ | HST
Q1130m1449_4 | 3 | 10.9${}_{-0.1}^{+0.1}$ | HST | Q1431m0050_26 | 2 | 10.6${}_{-0.3}^{+0.2}$ | HST
Q1130m1449_5 | 3 | N/A | HST | Q1431m0050_68 | 0 | N/A | HST
Q1130m1449_6 | 4 | 10.9${}_{-0.1}^{+0.1}$ | HST | Q1431m0050_73 | 1 | 8.9${}_{-0.6}^{+0.6}$ | HST
Q1130m1449_8 | 3 | 10.5${}_{-0.1}^{+0.1}$ | HST | Q1515p0410_4 | 3 | 11.0${}_{-0.2}^{+0.1}$ | HST
Q1130m1449_9 | 4 | 9.9${}_{-0.2}^{+0.2}$ | HST | Q1515p0410_9 | 2 | 10.2${}_{-0.1}^{+0.2}$ | HST
Q1130m1449_13 | 3 | 9.3${}_{-0.2}^{+0.3}$ | HST | Q1515p0410_11 | 2 | 10.1${}_{-0.1}^{+0.2}$ | HST
Q1130m1449_16 | 1 | 9.2${}_{-0.4}^{+0.4}$ | HST | Q1515p0410_13 | 2 | 10.2${}_{-0.1}^{+0.2}$ | HST
Q1130m1449_17 | 3 | 8.9${}_{-0.2}^{+0.2}$ | HST | Q1515p0410_42 | 1 | 9.1${}_{-0.6}^{+0.6}$ | HST
Q1130m1449_18 | 3 | 8.6${}_{-0.3}^{+0.2}$ | HST | Q1515p0410_54 | 0 | N/A | HST
Q1130m1449_24 | 1 | 8.8${}_{-0.4}^{+0.4}$ | HST | Q1554m203_51 | 1 | 9.4${}_{-0.5}^{+0.5}$ | HST
Q1130m1449_31 | 0 | N/A | HST | Q2131m1207_5 | 2 | 10.1${}_{-0.1}^{+0.2}$ | HST
Q1130m1449_36 | 0 | N/A | HST | Q2131m1207_26 | 0 | N/A | HST
Q1130m1449_43 | 0 | N/A | HST | Q2131m1207_34 | 0 | N/A | HST
Q1130m1449_50 | 2 | 8.2${}_{-0.3}^{+0.3}$ | HST | Q2131m1207_43 | 0 | N/A | HST
Q1130m1449_56 | 1 | 7.9${}_{-0.4}^{+0.4}$ | HST |  |  |  |
Table 1: Stellar masses for 79 absorber host galaxies. The values are labelled
"robust" if at least 2 detections in different HST filters were available for
the SED fit. For the purpose of SED fit, photometry from the Dark Energy
Camera Legacy Survey (DECaLS,https://www.legacysurvey.org/, Dey et al., 2019)
is used for some objects. The "probable" values are based one photometry
estimate or upper limits. These latter measurements are mainly present in the
lower mass end of the distribution.

##  Appendix A Spectral Energy Distribution Fit Resulting Spectra

Figure 4 displays two randomly chosen examples of Spectral Energy Distribution
(SED) fits performed with Le Phare algorithm (Arnouts et al., 1999; Ilbert et
al., 2006).

Figure 4: The red points display the AB magnitudes measurements from the HST
observations. The blue line represents the best fit galaxy spectrum derived
from Le Phare algorithm. Left: An example of a fit based on detections of the
galaxy in three HST bands and resulting in a fit providing a robust estimate
of the stellar mass of the object. Right: An example of a fit based on a
single magnitude measurement and providing a probable estimate of the stellar
mass of that galaxy.