# Structure of 13Be probed via quasi-free scattering

A. Corsi<EMAIL_ADDRESS>Y. Kubota  J. Casal  M. Gómez-Ramos  A. M. Moro  G.
Authelet  H. Baba  C. Caesar  D. Calvet  A. Delbart  M. Dozono  J. Feng  F.
Flavigny  J.-M. Gheller  J. Gibelin  A. Giganon  A. Gillibert  K. Hasegawa  T.
Isobe  Y. Kanaya  S. Kawakami  D. Kim  Y. Kiyokawa  M. Kobayashi  N. Kobayashi
T. Kobayashi  Y. Kondo  Z. Korkulu  S. Koyama  V. Lapoux  Y. Maeda  F. M.
Marqués  T. Motobayashi  T. Miyazaki  T. Nakamura  N. Nakatsuka  Y. Nishio  A.
Obertelli  A. Ohkura  N. A. Orr  S. Ota  H. Otsu  T. Ozaki  V. Panin  S.
Paschalis  E. C. Pollacco  S. Reichert  J.-Y. Rousse  A. T. Saito  S.
Sakaguchi  M. Sako  C. Santamaria  M. Sasano  H. Sato  M. Shikata  Y. Shimizu
Y. Shindo  L. Stuhl  T. Sumikama  Y.L. Sun  M. Tabata  Y. Togano  J. Tsubota
T. Uesaka  Z. H. Yang  J. Yasuda  K. Yoneda  J. Zenihiro  Département de
Physique Nucléaire, IRFU, CEA, Université Paris-Saclay, F-91191 Gif-sur-
Yvette, France  RIKEN Nishina Center, Hirosawa 2-1, Wako, Saitama 351-0198,
Japan  Département d’électronique des Détecteurs et d’Informatique pour la
Physique, IRFU, CEA, Université Paris-Saclay, F-91191 Gif-sur-Yvette, France
Département des Accélérateurs, de Cryogénie et de Magnétisme, IRFU, CEA,
Université Paris-Saclay, F-91191 Gif-sur-Yvette, France  Département
d’Ingénierie des Systèmes, IRFU, CEA, Université Paris-Saclay, F-91191 Gif-
sur-Yvette, France  Center for Nuclear Study, University of Tokyo, Hongo
7-3-1, Bunkyo, Tokyo 113-0033, Japan  European Centre for Theoretical Studies
in Nuclear Physics and Related Areas (ECT∗), Villa Tambosi, Strada delle
Tabarelle 286, I-38123 Trento, Italy  Dipartimento di Fisica e Astronomia “G.
Galilei” and INFN - Sezione di Padova, Via Marzolo 8, I-35131 Padova, Italy
Departamento de Física Atómica, Molecular y Nuclear, Facultad de Física,
Universidad de Sevilla, Apartado 1065, E-41080 Sevilla, Spain  Department of
Physics, Institut für Kernphysik, Technische Universität Darmstadt, 64289
Darmstadt, Germany  Department of Physics, Peking University, China  Institut
de Physique Nucleaire Orsay, IN2P3-CNRS, F-91406 Orsay Cedex, France  LPC
Caen, ENSICAEN, Universite de Caen, CNRS/IN2P3, F-14050 Caen, France
Department of Physics, Tohoku University, Aramaki Aza-Aoba 6-3, Aoba, Sendai,
Miyagi 980-8578, Japan  Department of Applied Physics, University of Miyazaki,
Gakuen-Kibanadai-Nishi 1-1, Miyazaki 889-2192, Japan  Department of Physics,
Ehwa Womans University  Department of Physics, University of Tokyo, Hongo
7-3-1, Bunkyo, Tokyo 113-0033, Japan  Department of Physics, Tokyo Institute
of Technology, 2-12-1 O-Okayama, Meguro, Tokyo 152-8551, Japan  Department of
Physics, Kyoto University, Kitashirakawa, Sakyo, Kyoto 606-8502, Japan
Department of Physics, Kyushu University, Nishi, Fukuoka 819-0395, Japan
Department of Physics, Technische Universitat Munchen, Germany

###### Abstract

We present an investigation of the structure of 13Be obtained via a
kinematically complete measurement of the $(p,pn)$ reaction in inverse
kinematics at 265 MeV/nucleon. The relative energy spectrum of 13Be is
compared to Transfer-to-the-Continuum calculations which use as structure
inputs the overlaps of the 14Be ground-state wave function, computed in a
three-body model, with the unbound states of the 13Be residual nucleus. The
key role of neutron $p$-wave orbital in the interpretation of the low-
relative-energy part of the spectrum is discussed.

###### keywords:

quasi-free scattering , Borromean nuclei , three-body model , resonances

††journal: Physics Letters B

##  1 Introduction

Light nuclei at the neutron dripline display exotic properties connected to
the large spatial distribution of weakly bound valence neutrons, such as halo
and clustering. A special class of halo nuclei are two-neutron halo nuclei
like 6He, 11Li and 14Be, known as Borromean nuclei, since they can be
described as a three-body system without any bound two-body subsystem. They
provide a good environment to study dineutron correlations which are expected
to be a key element in their stabilization [1]. Different methods have been
advocated and used to probe the properties of Borromean nuclei, such as
Coulomb dissociation [2, 3], dineutron decay [4], and quasi-free scattering
reactions [5]. As mentioned, the subsystems constituted of one neutron and the
remaining fragment (5He, 10Li, 13Be) are unbound and their continuum exhibit a
resonant structure.

In this paper we present a study of the spectroscopy of the unbound 13Be
nucleus obtained by measuring the invariant mass of the 12Be-neutron system
resulting from the decay of the 13Be system produced via the quasi-free
scattering reaction 14Be($p$,$pn$). The unbound nature of 13Be was suggested
more than 30 years ago [6, 7], and confirmed in 1973 [8]. Several experiments
have attempted to study the spectroscopy of 13Be both via missing mass and
invariant mass technique using charge exchange [9], fragmentation [10], proton
removal from 14B [11, 12, 13] and neutron removal from 14Be [14, 15, 16, 17].
The missing mass technique offers the advantage of yielding the absolute
energy above the one-neutron decay threshold, and typically allows to explore
a larger range in excitation energy above the two-neutron separation
threshold. Using this method, resonances at $\sim$2, 5, 7, and 10 MeV were
observed in Ref. [18], and at 1.22(10), 2.10(16), 4.14(12), 5.09(14), and
7.0(2) MeV in [19]. Invariant mass spectra from different experiments display
a peak at about 0.5 MeV above the 12Be+neutron threshold, and a broader
structure around 2 MeV. As already discussed in Ref. [20], the spectral shape
strongly differs depending on the production mechanism, namely if 13Be is
produced starting from 14Be [14, 15, 16, 17] or 14B [11, 12, 13]. Even
limiting ourselves to the first case, the mechanism adopted in this work,
different interpretations of the relative energy spectrum have been provided.
Ref. [15] interprets the low lying peak as a 1/2- ($\ell$=1) intruder state
that appears due to the quenching of the $N$=8 spin-orbit shell gap, and the
structure around 2 MeV as a 5/2+ ($\ell$=2) state. This interpretation is
based on the analysis of the transverse momentum distribution using $s$, $p$
and $d$ waves, corroborated by shell-model calculations, and is in agreement
with predictions by [21]. Ref. [17] makes a synthesis of existing experimental
results, with special emphasis on those obtained from proton-induced one-
neutron removal [15]. Nevertheless, the analysis of the transverse momentum
distribution performed in Ref. [17] yields quite different conclusions with
respect to Ref. [15]: a much stronger $d$-wave component (dominant above 2.6
MeV), and a dominance of $s$-wave (80(10)%) around 0.5 MeV, instead of
$p$-wave.

This diversity and sometimes inconsistency in the positions and spin
assignment of the states of 13Be indicate that the standard fitting procedures
used for the analysis of these spectra may be lacking some constraints on the
possible structures due to the complexity of 13Be spectrum. As such, in this
work we study the 14Be$(p,pn)$ reaction using a novel method, proposed in
[22], which uses consistent two- and three-body models for 14Be and 13Be and
is able to provide predictions for the positions and weights of the structures
of the spectrum, thus reducing the ambiguities in the analysis.

Part of the complexity of the 13Be continuum spectrum stems from the
admixtures of single-particle structures with core-excited components. In
fact, core excitation has been postulated as a key element to understand the
formation of Borromean systems [23, 24], but it is very difficult to pin down.
The level scheme of 12Be is well established. A strong excitation of the
12Be(2+) state in inelastic proton scattering, consistent with a strong
quadrupole deformation, provided a first evidence of $N=8$ shell gap quenching
in 12Be [25]. Furthermore, neutron removal from 12Be revealed that the last
neutrons have a significant ($2s_{1/2}$)2 +($1d_{5/2}$)2 configuration and
that there is only of order 30% of the $(1p_{1/2})^{2}$ closed shell component
[26]. In this experiment we were able to measure with high statistics the
possible 12Be(2+, 1-) core excited component that decays via gamma rays.

The experiment and the results on the spectroscopy of 13Be are presented in
Sec. 2, while their interpretation follows in Sec. 3 after a brief description
of the theoretical framework.

##  2  Experiment

The experiment was performed at the Radioactive Isotope Beam Factory operated
by the RIKEN Nishina Center and the Center for Nuclear Study (CNS) of the
University of Tokyo. Secondary beams were produced and separated by the
BigRIPS fragment separator [27], using projectile fragmentation of a 48Ca
primary beam at 345 MeV/nucleon with a typical intensity of 400 particle nA on
a Be target. Fragmentation products were detected and identified by using
plastic scintillators and multi-wire drift chambers (MWDCs) positioned along
the BigRIPS line. The main components of the cocktail beams were 11Li, 14Be,
and 17B (80%, 12%, and 8%, respectively) and impinged on the secondary target
with an average energy of 246, 265 and 277 MeV/nucleon, respectively. The
secondary target was a 15-cm thick liquid hydrogen target surrounded by a Time
Projection Chamber (TPC) of the MINOS device [28]. The TPC in conjunction with
the beam tracking MWDC detectors was acting as vertex tracker, allowing to
improve invariant-mass and gamma spectroscopy resolution. The combination of
the thick MINOS target and the intense secondary beams from RIBF ($\sim$ 105
pps) was a key ingredient to obtain enough statistics for a kinematically
complete measurement. A rather complex detection system (Fig. 1) was deployed
to perform an exclusive measurement. The knocked-out neutron was measured by
the WINDS array of plastic scintillators [29]. Its kinetic energy was deduced
with the time of flight technique. The recoil proton was tracked first in the
TPC, then in a MWDC, and subsequently traversed an array of plastic
scintillators allowing the measurement of its kinetic energy via the time of
flight technique. The ensemble of MWDC and plastic scintillators wall is
called the recoil proton (RP) detector hereafter. The identification and
momentum analysis of the heavy charged fragment was achieved via the
combination of tracking in the SAMURAI [30] dipole magnet via a set of MWDC
placed before and after the magnet, and the energy loss and time of flight
measurement in an array of plastic scintillators placed at the focal plane of
SAMURAI. Their momentum could then be deduced from the measurement of their
trajectory. The dipole gap was kept under vacuum using a chamber equipped with
thin exit windows [31] so as to reduce to a minimum the amount of material
encountered by both the fragments and neutrons. The decay neutron was detected
in the two walls of plastic scintillators of the NEBULA array [32]. The
efficiency of the NEBULA array for the detection of one neutron is $\sim$
35$\%$ at the beam energy of this experiment. The WINDS and recoil proton
detector covered angles between 20∘ and 60∘, and 30∘ to 65∘, respectively,
allowing the selection of high-momentum-transfer events corresponding to
quasi-free scattering [33, 34, 3].

Figure 1:  Scheme of the experimental setup.

In this process, the dominant mechanism for the knockout reaction is a single
interaction between the incident particle and the struck nucleon which yields
a kinematics for the $(p,pn)$ reaction very close to the one of free
scattering. The angular correlation between the scattered neutron and the
recoil proton is shown in Fig. 2 (left). The opening angle in the laboratory
frame corresponds to 85∘, close to 86∘ as predicted using the QFS kinematics
simulation of Ref. [35]. This confirms that the high-momentum transfer ($>$ 1
fm-1) events selected by the detection system correspond to QFS. Invariant-
mass resolution and efficiency have been determined via a GEANT4 simulation
[36] with the code used in [32]. The resolution follows a
FWHM=$0.587\sqrt{E_{r}}$ MeV law and is shown in Fig. 2 (right). The
efficiency is also shown in Fig. 2 (right) and is estimated assuming 100%
transmission of the beam and the fragment. The transmission (including
tracking detectors efficiency and loss of beam and fragment in the thick MINOS
target) is evaluated separately from experimental data taking the average of
the values obtained for 12Be and 14Be beam transmission (65.0(1)% and
62.9(2)%, respectively), and corresponds to 64(1)%.

12Be core excited states that decay via gamma emission (2+, 1-) were
identified using a reduced version of the DALI2 gamma array consisting of 68
crystals, partially covering angles between 34∘ and 115∘ and arranged in order
to avoid interference with $(p,pn)$ measurement. Photopeak efficiency of this
reduced version of DALI2 (called DALI2 hereafter) was 8.9(5)% and 7.0(4)% at
2.1 and 2.7 MeV, respectively. This experiment was not sensitive to
12Be(0${}^{+}_{2}$) core excited state as this state is isomeric and its
gamma-decay lifetime (1.87(15) $\mu$s) is too long for in-flight detection
[37].

Figure 2:  Left: angular correlation in the laboratory frame of proton and
neutron in 14Be$(p,pn)$ reaction. The red line correspond to the kinematics
calculated with the QFS code of Ref. [35]. Right: efficiency (red line) and
resolution (blue line) for 13Be invariant mass measurement with NEBULA and
SAMURAI.

The invariant mass spectrum of 13Be is shown in Fig. 3 (left). The absolute
cross section is determined taking into account the efficiency for invariant
mass measurement and the fragment transmission. The error bars take into
account the uncertainty on the transmission (1%), on the neutron detection
efficiency (2.5%) and the statistical uncertainty on the number of beam and
fragment particles. The spectrum is characterized by a prominent peak with
maximum at $\sim$ 0.48 MeV and a broader structure, peaked at $\sim$ 2.3 MeV,
extending from $\sim$1 MeV to $\sim$5 MeV. The contribution corresponding to
12Be(2+) and 12Be(1-) core excited states has been fixed via coincidences with
2.1 and 2.7 MeV gamma transitions, respectively, and is shown for comparison
after correcting for gamma-detection efficiency. The uncertainty on gamma
detection efficiency (6%) has been added to the error bars.

The gamma spectrum of 12Be is shown in Fig. 3 (right). The 2.1(0.1) and
2.7(0.4) MeV transitions are consistent with the known transitions deexciting
the 2+ and 1- excited states of 12Be to its ground state. We note that the
same gamma transitions were observed in Ref. [15], though with very limited
statistics, while [13] observed only the 2.1 MeV transition. As can be better
seen in the inset of Fig. 3 (left), the 2.1 MeV one is observed in coincidence
with a structure peaking at $\sim$0 and $\sim$3 MeV in the relative energy
spectrum, as in [13]. The 2.7 MeV one is observed in coincidence with a
structure at $\sim$3 MeV. The contribution from the Compton events associated
to the 2.7 MeV transition summing up to the 2.1 MeV transition has been
estimated via the simulation and subtracted from the cross section.

Figure 3:  Left: relative energy spectrum of 13Be and contributions from core
excited components. The inset shows the spectrum in logarithmic scale. Right:
gamma spectrum of 12Be. The two transitions are reproduced by the sum of an
exponential background and the response functions (dashed curves) of DALI2 to
a transition at 2.1 MeV and 2.7 MeV, obtained via a GEANT4 simulation.

Based on this, we built a partial level scheme presented in Fig. 4. Only the
levels that can be clearly deduced from the present data are shown. The 2.3
MeV peak observed in the relative energy spectrum likely corresponds to the
well-accepted 5/2+ state in 13Be, whose tail may be responsible for the
$\sim$0 MeV transition in coincidence with the 2+ state in 12Be (as discussed,
for instance, in Ref. [17]). This, together with the spin-parity assignment of
the lowest level at 0.48 MeV, will be further discussed in Sec. 3, where we
also analyze the information from the corresponding transverse momentum
distributions.

Figure 4:  Partial level scheme based on the observed neutron-12Be relative
energy spectrum and gamma-neutron-12Be coincidences. Transitions in the
relative energy spectrum are represented by lines (black for transitions to
the ground state of 12Be, blue and green for transitions populating 12Be(2+)
and 12Be(1-), respectively). Gamma transitions are represented by the red wavy
arrows. Energies are given in MeV.

##  3 Theoretical analysis

###  3.1 Three-body calculations

In order to better understand the experimental results, we have performed
structure calculations for 14Be using a three-body model
$(^{12}\text{Be}+n+n)$ within the hyperspherical formalism [38, 39, 40].
Details on how the wave function is built can be found, for instance, in Ref.
[41] and references therein. This consists in diagonalizing the Hamiltonian in
an analytical transformed harmonic oscillator (THO) basis for three-body
systems. The method has been previously applied with great success to describe
direct reactions induced by three-body projectiles [42, 43, 22, 44].

Three-body calculations require, as input, the binary interactions between all
constituents. For the $n$-$n$ potential, we employ the GPT tensor interaction
[45]. This potential, although simpler than the more sophisticated AV18 [46],
CD-Bonn [47] or Reid93 [48] interactions, reproduces $NN$ observables up to
300 MeV, so it is suitable for three-body structure calculations. In order to
get a realistic description of 14Be, the 12Be-$n$ interactions needs to
reproduce the properties of the unbound binary subsystem 13Be. From previous
fragmentation and knockout experiments, it is mostly accepted that 13Be
exhibits a low-lying $s$-wave state and a $d$-wave resonance around 2 MeV
relative energy [10, 11, 14]. There are, however, large discrepancies in the
interpretation of the 13Be spectrum from different experimental works [17,
13], many of which are associated with the long-debated existence of a low-
lying $p$-wave resonance and the contribution from excited 12Be components
[15]. For this reason, we make use of different core-neutron potentials to
study the sensitivity of the structure and reactions observables to the
properties of 13Be.

In order to include some excited-core components in the description of 14Be,
we parametrize the 12Be-$n$ interaction with a deformed Woods-Saxon potential
with $l$-dependent central and spin-orbit terms. Following Ref. [49], we
introduce an effective quadrupole deformation parameter of $\beta_{2}=0.8$,
and the $0^{+}$ ground state and the first $2^{+}$ excited state in 12Be are
coupled by means of a simple rotational model [39]. In this scheme, no other
excited states of 12Be are included. Three-body calculations including also
the 1- state in a consistent way are not available. As shown in Ref. [49],
despite the deformation the $s$-wave interaction still gives rise to a 1/2+
virtual state in 13Be. The potential parameters $V_{c}^{(0,2)}$ and $V_{ls}$
are adjusted to fix the scattering length of this virtual state and to provide
a 5/2+ resonance just below the 2+ excitation threshold in 12Be, i.e. at 2.11
MeV [50]. Note that, in this scheme, the 5/2+ state may decay via $d_{5/2}$
neutrons to the ground state of 12Be, but also via $s_{1/2}$ to the 2+ excited
state, given its finite width. For simplicity, we start with a shallow
$V_{c}^{(1)}$ potential, so no negative-parity resonances appear. This
potential, labeled P1, produces a $1/2^{+}$ virtual state characterized by a
large scattering length. Details are given in Table 1. In addition, the
calculations include a phenomenological three-body force to correct the
position of the $0^{+}$ ground state to the experimental two-neutron
separation energy of 14Be, i.e. $S_{2n}=1.27$ MeV [51], which is kept fixed.
Some properties of the resulting 14Be ground state are also given in Table 1.
We remind the reader that although the energy of a virtual state is strictly
negative, it is customary to define a nominal positive energy as
$E_{s}=\hbar^{2}/(2\mu a^{2})$ (with $a$ indicating the scattering length, see
e.g. Chap. 2 of Ref. [52]) to quantify the proximity of the virtual-state pole
to the threshold.
Given the open debate about the presence of a low-lying $p$-wave resonance in
13Be, we consider another potential labeled P2. In this case, we increase the
$p$-wave potential depth to produce a 1/2- resonance around the maximum of the
12Be-$n$ relative-energy distribution, while keeping a small scattering length
for the $s$-wave state and the same $d$-wave resonance as with P1. With the
adopted parameters, the scattering length of the 1/2+ state is -9.2 fm, which
corresponds to a nominal energy of 0.265 MeV. The computed energy and width of
the 1/2- (5/2+) resonance are 0.46 (1.96) and 0.40 (0.27) MeV. The resulting
14Be properties are also listed in Table 1. It is worth noting that the P2
model produces a strong mixing between different-parity states, as shown in
Table 1. This gives rise to a marked dineutron character of the 14Be wave
function, as opposed to potential P1 and in accord with the discussion in Ref.
[53].

In the next section, we will study the sensitivity of the $(p,pn)$ cross
section to the structure properties of 13,14Be. For this purpose, we consider
variations of potential P2 in which the $1/2^{-}$ and $5/2^{+}$ resonances are
placed at different energies. These variations are labeled P3-5 and their
details are also presented in Table 1.

| $a$ | $E(5/2^{+})$ | $\Gamma(5/2^{+})$ | $E(1/2^{-})$ | $\Gamma(1/2^{-})$ | $s$ | $p$ | $d$ | $2^{+}$
---|---|---|---|---|---|---|---|---|---
P1 | $-40.1$ | 1.96 | 0.27 | - | - | 0.59 | 0.13 | 0.26 | 0.34
P2 | $-9.2$ | 1.96 | 0.27 | 0.46 | 0.40 | 0.19 | 0.62 | 0.18 | 0.22
P3 | $-9.2$ | 1.53 | 0.28 | 0.46 | 0.40 | 0.20 | 0.53 | 0.25 | 0.25
P4 | $-9.2$ | 2.10 | 0.19 | 0.46 | 0.40 | 0.17 | 0.68 | 0.13 | 0.19
P5 | $-9.2$ | 1.96 | 0.27 | 0.62 | 0.60 | 0.24 | 0.53 | 0.22 | 0.24
Table 1:  Scattering length $a$ (in fm) of the 1/2+ virtual state in 13Be and
energies and widths of the 5/2+ and 1/2- resonances (in MeV) using the
different core-neutron potentials P1-5. On the right, the resulting properties
of the 14Be ground state: partial wave content for $L=0,1,2$ neutrons, and
weight of the 2+ core-excited components. The two-neutron separation energy in
14Be is fixed to the experimental value of 1.27 MeV [51].

###  3.2 Reaction calculations

We have compared the present $(p,pn)$ data with reaction calculations based on
the so-called Transfer to the Continuum (TC) framework [54], which was
recently extended to describe processes induced by Borromean projectiles [22].
In this method, the differential cross section for the $(p,pn)$ reaction is
obtained from the prior-form transition amplitude leading to the three-body
continuum states for $p+n+^{13}$Be. The latter are expanded in a basis of
$p$-$n$ states, which are conveniently discretized using a binning procedure
akin to that employed in the continuum-discretized coupled-channels (CDCC)
method [55]. Good convergence of the calculated observables (within 10%) was
attained using a set of energy bins with a width $\Delta\epsilon_{pn}=15$ MeV,
a maximum $p-n$ relative energy of $\epsilon_{pn}=210$ MeV and a maximum
angular momentum and parity $j^{\pi}\leq 3^{\pm}$.

The model considers a spectator/participant scenario, in which the incident
proton is assumed to remove one of the valence neutrons without modifying the
state of the remaining 13Be (${}^{12}{\rm Be}+n$) subsystem. This is
consistent with QFS conditions. Under this assumption, the 14Be structure
enters through the overlap functions between the initial state (the 14Be g.s.)
and the final 13Be states so that the cross section for different
configurations of 13Be (defined by their energy $E_{n-^{12}{\rm Be}}$ and
angular momentum and parity $J_{T}^{\pi}$) can be computed independently.

Important ingredients are also the proton-neutron and nucleon-13,14Be
interactions. For the former, following previous applications of the method
[54, 22], we adopt the Reid93 parametrization [48], which provides an accurate
description of the proton-neutron cross sections and phase-shifts up to 350
MeV. For proton-14Be and proton/neutron-13Be interactions, which take into
account the distortion and absorption of the incoming proton and of the
outgoing nucleons, we employ the optical potentials computed from the Dirac
phenomenological parametrization [56, 57].

Figure 5: 13Be relative-energy spectrum using the (a) P1 and (b) P2
potentials. Results are shown after convoluting the theoretical lineshapes
with the experimental resolution function. The inset shows the relative energy
spectrum measured in coincidence with the 12Be(2+) decay transition, compared
to the calculated core-excited component. See text for details.

The ${}^{13}\text{Be}={{}^{12}}\text{Be}+n$ relative-energy spectrum obtained
by using the P1 core-neutron potential and convoluting with the experimental
resolution from the simulation (determined as explained in Sec. 2) is shown in
Fig. 5a. Note that these reaction calculations provide absolute cross
sections, so no fitting or scaling is carried out. The different contributions
are labeled $J_{T}[L_{J}\otimes I]$, where $J_{T}$ is the total 13Be angular
momentum resulting from coupling the single-particle configuration $L_{J}$
with the spin $I$ of 12Be. Trivially, since the spin of 14Be is $0^{+}$,
$J_{T}$ equals $j_{2}=[l_{2}\otimes s_{2}]$, the angular momentum of the
removed nucleon. In this figure, only the leading components are shown,
together with the total cross section. In this way, the contributions from
$I=0$ and $I=2$ can be separated. The cross section using P1 overestimates the
experimental data at low relative energies due to the large scattering length
of the $s$-wave virtual state. Moreover, the maximum of the cross section
around 0.5 MeV is not reproduced. We have checked that variations of the
position of the 1/2+ virtual state do not improve the agreement.

The results using the P2 potential are shown in Fig. 5b. Again, for clarity
only the leading terms are presented. Notice that, although the width of the
5/2+ state in the present model is smaller than that of the 1/2- (see Table
1), its contribution to the relative-energy spectrum becomes broader due to
the energy resolution. Note also that the contribution of the $2^{+}$ state is
small in spite of the significant core-excited component in these models. This
is because the nominal energy of the 5/2+ state, which collects most of the
core-excitation weight in the 14Be wave function, lays below the 12Be($2^{+}$)
excitation threshold. The small $5/2^{+}[s_{1/2}\otimes 2^{+}]$ contribution
is consistent with the analysis from gamma coincidences. The comparison
between this component and the experimental data in coincidence with the
12Be(2+) decay transition is presented in the inset of Fig. 5b, where we can
see that the calculations describe the data at low energies quite well. The
disagreement at energies above $\sim 1.5$ MeV might indicate that the present
three-body calculations are missing some high-lying state which can also decay
via 12Be($2^{+}$), as suggested in Fig. 4. As for the full spectrum, Fig. 5b
shows that the low-energy peak can be described reasonably well by the 1/2-
resonance using the P2 potential. The theoretical distribution is somewhat
broader than the experimental data, so the calculation overestimates the
measurements between $\sim$0.6-1 MeV. This might be an indication that the
$p$-wave content obtained with the P2 potential is perhaps too large. In
addition to the underestimation at large relative energies, this suggests that
there might be missing components in the wave-function expansion, in
particular those coming from the decay of other states in 13Be via
12Be($2^{+}$), or the coupling to other excited states of 12Be. Calculations
including these features are not available yet.

To test the sensitivity of the relative-energy spectrum to the specific
features included in the potential, in Fig. 6 we compare the P2 calculations
with three additional models. As shown in Table 1, P3 and P4 are obtained by
placing the 5/2+ resonance of 13Be at lower or higher energies, respectively,
while P5 involves a variation on the position of the 1/2- state. Note that a
difference in the position of the relevant levels changes also the weight of
the different components in the ground state of 14Be. It is shown that the
best agreement up to 2 MeV relative energy is achieved using the potential P2.

Figure 6: Results for the 13Be relative-energy spectrum with potentials P2-5.
Only the total calculations are presented. Figure 7: Experimental transverse
momentum distributions at 0-0.2, 0.4-0.5 and 1.8-2.2 MeV relative energy with
P2 potential. The solid black line is the total TC result, convoluted with the
experimental resolution and globally rescaled through a $\chi^{2}$ fit. Dashed
lines are the contributions corresponding to removal of a neutron from a $s$\-
(red), $p$\- (blue) or $d$-wave (green).

The 13Be structure can be further studied from the transverse momentum
distributions of the knocked-out neutron. The comparison between the present
calculations, using potential P2, and the experimental momentum distributions
is presented in Fig. 7, for three different relative-energy bins: 0-0.2,
0.4-0.5 and 1.8-2.2 MeV. Calculations have been convoluted with the
experimental resolution of $\sim$ 39 MeV/c (FWHM) obtained from the direct
beam measurement. The contribution from momentum resolution of the neutron was
checked via the simulation and found to be negligible. The overall
normalization of the total theoretical distribution with respect to the data
has been adjusted to obtain the best $\chi^{2}$ fit. Individual contributions
from the different orbital angular momenta of the knocked-out neutron are also
presented. The relative weights of these contributions are fixed by the
structure and reaction calculations and not via $\chi^{2}$ fit. Note that, due
to the non-zero spin of the 12Be core in its excited components, the orbital
angular momentum $l_{2}$ of the removed neutron is not necessarily the same as
the one of the valence neutron in 13Be. Overall, it is found that the width of
the momentum distributions is well reproduced. In particular, we can describe
the data at 0.4-0.5 MeV with a dominant $p$-wave contribution. In Ref. [17],
an $s$-wave resonance (or a combination of two overlapping $s$-wave
resonances) was proposed to explain the peak in the 13Be spectrum.

To test the assumption of the $s$-wave resonance suggested in Ref. [17], in
the top panel of Fig. 8 we have performed a $\chi^{2}$ fit of the momentum
distribution for the 0.4-0.5 energy bin retaining only the $s$-wave
contribution in our calculations with the P2 potential. In this case, the
resulting width is clearly too small. A similar conclusion is achieved if the
analysis is performed over the data from Ref. [17], which is shown in the
bottom panel. In this latter case, our full calculation with a dominant
$p$-wave component reproduces very well the experimental momentum
distribution, whereas the pure $s$-wave assumption gives again a too narrow
distribution. Our analysis shows that the peak observed in the 13Be relative-
energy spectrum at $E_{r}\sim 0.5$ MeV, populated in the 14Be($p,pn)$
reaction, is most likely dominated by the $p$-wave contribution. This
assignment is in disagreement with that of Ref. [17]. We believe this
discrepancy can be understood as due to the inherent uncertainties in the
$\chi^{2}$ procedure used in Ref. [17] to assign the orbital momentum, as well
as the differences in structure and reaction model.

Figure 8: Transverse momentum distribution at 0.4-0.5 MeV relative energy. The
top and bottom panels are for the present experiment and from the GSI data at
304 MeV/nucleon with resolution $\sim$72 MeV/c (FWHM) [17], respectively. The
black solid line is the total P2 result, while the red dashed line corresponds
to a $\chi^{2}$ fit assuming a pure $s$-wave distribution.

##  4  Conclusions

We have presented a high statistics measurement of the spectroscopy of 13Be
via invariant mass, including the measurement of 12Be core excited states
which decay via gamma rays. We clearly observed for the first time the
contribution of both 12Be(2+, 1-) states in 13Be populated via $(p,pn)$
reaction. Their contribution to the 13Be relative-energy spectrum is small. A
still missing information is the contribution of the isomeric
12Be(0${}^{+}_{2}$) core-excited state, that will demand a dedicated
measurement. A key and novel aspect of our analysis consists in calculating,
for the first time, the relative energy cross section and momentum
distribution using a well-founded reaction framework and a realistic three-
body model of 14Be that incorporates 12Be(2+) excitations, thus avoiding the
common procedure of extracting individual angular momentum components from a
fit, a technique that becomes more ambiguous in the case of complex spectra
with overlapping structures as the one of 13Be. This analysis permitted to pin
down the dominant $\ell=1$ contribution of the resonant peak observed in the
low-lying spectrum, in agreement with [15, 13] and at variance with the
conclusions of Refs. [16, 17], which assigned a dominant $\ell=0$ to this
peak.

Additional observables, such as the distribution of the opening angle between
the momenta of 13Be and the removed neutron in the final state, may help to
shed light on the structure of 14Be and hence 13Be. The interpretation of such
observables, as extracted from the present experiment, will be the subject of
an upcoming publication. An improvement of the three-body model, e.g. the
inclusion of other 12Be excited state, proper antisymmetrization of valence
neutrons and a better treatment of Pauli principle, will increase the
capability of theory to capture the features of the experimental spectrum.

## Acknowledgements

This work has been supported by the European Research Council through the ERC
Starting Grant No. MINOS-258567. A.C. acknowledges Y. Kikuchi and K. Ogata for
fruitful discussions, and N.Paul for careful proofreading. M.G.R., A.M.M. and
J.C. are partially supported by the Spanish Ministerio de Ciencia, Innovación
y Universidades and FEDER funds (projects FIS2014-53448-C2-1-P and
FIS2017-88410-P) and by the European Union’s Horizon 2020 research and
innovation program under grant agreement No. 654002. J.G., F.M.M. and N.A.O.
acknowledge partial support from the Franco-Japanese LIA-International
Associated Laboratory for Nuclear Structure Problems as well as the French
ANR14-CE33-0022-02 EXPAND.

## References

## References

  * [1] H. Sagawa, K. Hagino, Theoretical models for exotic nuclei, The European Physical Journal A 51 (8) (2015) 102.  doi:10.1140/epja/i2015-15102-4.
  * [2] T. Nakamura, et al., Phys. Rev. Lett. 96 (2006) 252502.  doi:10.1103/PhysRevLett.96.252502.
  * [3] T. Aumann, C. A. Bertulani, J. Ryckebusch, Phys. Rev. C 88 (2013) 1.  doi:10.1103/PhysRevC.88.064610.
  * [4] K. Hagino, H. Sagawa, Few-Body Systems 57 (2016) 185.  doi:10.1007/s00601-015-1027-3.
  * [5] Y. Kikuchi, K. Ogata, Y. Kubota, M. Sasano, T. Uesaka, Progr. Theor. Exp. Phys. 2016 (2016) 103D03.  doi:10.1093/ptep/ptw148.
  * [6] A. M. Poskanzer, S. W. Cosper, E. K. Hyde, J. Cerny, Phys. Rev. Lett. 17 (1966) 1271\.  doi:10.1103/PhysRevLett.17.1271.
  * [7] A. G. Artukh, et al., Phys. Lett. B 33 (1970) 407.  doi:10.1016/0370-2693(70)90616-7.
  * [8] J. D. Bowman, A. M. Poskanzer, R. G. Korteling, G. W. Butler, Phys. Rev. Lett. 31 (1973) 614.  doi:10.1103/PhysRevLett.31.614.
  * [9] B. R. Marks, et al., Phys. Rev. C 92 (2015) 054320.  doi:10.1103/PhysRevC.92.054320.
  * [10] M. Thoennessen, S. Yokoyama, P. G. Hansen, Phys. Rev. C 63 (2000) 014308.  doi:10.1103/PhysRevC.63.014308.
  * [11] J. L. Lecouey, Few-Body Systems 34 (2004) 21.  doi:10.1007/s00601-004-0029-3.
  * [12] G. Randisi, et al., Phys. Rev. C 89 (2014) 034320.  doi:10.1103/PhysRevC.89.034320.
  * [13] G. Ribeiro, et al., Phys. Rev. C 98 (2018) 024603.  doi:10.1103/PhysRevC.98.024603.
  * [14] H. Simon, et al., Nucl. Phys. A 791 (2007) 267.  doi:10.1016/j.nuclphysa.2007.04.021.
  * [15] Y. Kondo, et al., Phys. Lett. B 690 (2010) 245.  doi:10.1016/j.physletb.2010.05.031.
  * [16] Y. Aksyutina, et al., Phys. Lett. B 718 (2013) 1309.  doi:10.1016/j.physletb.2012.12.028.
  * [17] Y. Aksyutina, et al., Phys. Rev. C 87 (2013) 064316.  doi:10.1103/PhysRevC.87.064316.
  * [18] A. Korsheninnikov, et al., Phys. Lett. B 343 (1995) 53.  doi:10.1016/0370-2693(94)01435-F.
  * [19] A. Belozyorov, et al., Nucl. Phys. A 636 (1998) 419.  doi:10.1016/S0375-9474(98)00217-6.
  * [20] T. Nakamura, H. Sakurai, H. Watanabe, Prog. Part. Nucl. Phys. 97 (2017) 53.  doi:10.1016/j.ppnp.2017.05.001.
  * [21] G. Blanchon, A. Bonaccorso, D. Brink, A. García-Camacho, N. V. Mau, Nucl. Phys. A 784 (2007) 49.  doi:10.1016/j.nuclphysa.2006.11.033.
  * [22] M. Gómez-Ramos, J. Casal, A. M. Moro, Phys. Lett. B 772 (2017) 115.  doi:10.1016/j.physletb.2017.06.023.
  * [23] I. Tanihata, et al., Phys. Rev. Lett. 100 (2008) 1.  doi:10.1103/PhysRevLett.100.192502.
  * [24] G. Potel, F. Barranco, E. Vigezzi, R. A. Broglia, Phys. Rev. Lett. 105 (2010) 9\.  doi:10.1103/PhysRevLett.105.172502.
  * [25] H. Iwasaki, et al., Phys. Lett. B 491 (2000) 8.  doi:10.1016/S0370-2693(00)01017-0.
  * [26] S. D. Pain, et al., Phys. Rev. Lett. 96 (2006) 032502.  doi:10.1103/PhysRevLett.96.032502.
  * [27] N. Fukuda, T. Kubo, T. Ohnishi, N. Inabe, H. Takeda, D. Kameda, H. Suzuki, Nucl. Instr. Meth. B 317 (2013) 323 – 332.  doi:https://doi.org/10.1016/j.nimb.2013.08.048.
  * [28] A. Obertelli, et al., Eur. Phys. J. A 50 (2014) 8.  doi:10.1140/epja/i2014-14008-y.
  * [29] K. Yako, et al., RIKEN Accel. Prog. Rep. 45 (2012) 137.
  * [30] T. Kobayashi, et al., Nucl. Instrum. Methods Phys. Res. B 317 (2013) 294.  doi:10.1016/j.nimb.2013.05.089.
  * [31] Y. Shimizu, et al., Nucl. Instr. Meth. B 317 (2013) 739.  doi:10.1016/j.nimb.2013.08.051.
  * [32] T. Nakamura, Y. Kondo, Nucl. Instrum. Methods Phys. Res. B 376 (2016) 156.  doi:10.1016/j.nimb.2016.01.003.
  * [33] G. Jacob, T. A. J. Maris, Rev. Mod. Phys. 38 (1966) 121.  doi:10.1103/RevModPhys.38.121.
  * [34] G. Jacob, T. A. J. Maris, Rev. Mod. Phys. 45 (1973) 6.  doi:10.1103/RevModPhys.45.6.
  * [35] V. Panin, et al., Phys. Lett. B 753 (2016) 204.  doi:10.1016/j.physletb.2015.11.082.
  * [36] S. Agostinelli, et al., Geant4: a simulation toolkit, Nucl. Instr. Meth. A 506 (3) (2003) 250.  doi:https://doi.org/10.1016/S0168-9002(03)01368-8.
  * [37] S.Shimoura, et al., Phys. Lett. B 654 (2007) 87.  doi:10.1016/j.physletb.2007.08.053.
  * [38] P. Descouvemont, C. Daniel, D. Baye, Phys. Rev. C 67 (2003) 044309.  doi:10.1103/PhysRevC.67.044309.
  * [39] I. Thompson, F. Nunes, B. Danilin, Comput. Phys. Commun. 161 (2004) 87.  doi:10.1016/j.cpc.2004.03.007.
  * [40] M. Rodríguez-Gallardo, J. M. Arias, J. Gómez-Camacho, A. M. Moro, I. J. Thompson, J. A. Tostevin, Phys. Rev. C 72 (2005) 024007.  doi:10.1103/PhysRevC.72.024007.
  * [41] J. Casal, E. Garrido, R. de Diego, J. M. Arias, M. Rodríguez-Gallardo, Phys. Rev. C 94 (2016) 054622.  doi:10.1103/PhysRevC.94.054622.
  * [42] J. Casal, M. Rodríguez-Gallardo, J. M. Arias, Phys. Rev. C 92 (2015) 054611\.  doi:10.1103/PhysRevC.92.054611.
  * [43] J. Casal, M. Gómez-Ramos, A. M. Moro, Phys. Lett. B 767 (2017) 307.  doi:10.1016/j.physletb.2017.02.017.
  * [44] A. Arazi, et al., Phys. Rev. C 97 (2018) 044609.  doi:10.1103/PhysRevC.97.044609.
  * [45] D. Gogny, P. Pires, R. D. Tourreil, Phys. Lett. B 32 (1970) 591.  doi:10.1016/0370-2693(70)90552-6.
  * [46] R. B. Wiringa, V. G. J. Stoks, R. Schiavilla, Phys. Rev. C 51 (1995) 38.  doi:10.1103/PhysRevC.51.38.
  * [47] R. Machleidt, Phys. Rev. C 63 (2001) 024001.  doi:10.1103/PhysRevC.63.024001.
  * [48] V. G. J. Stoks, R. A. M. Klomp, C. P. F. Terheggen, J. J. de Swart, Phys. Rev. C 49 (1994) 2950.  doi:10.1103/PhysRevC.49.2950.
  * [49] T. Tarutina, I. Thompson, J. Tostevin, Nucl. Phys. A 733 (2004) 53.  doi:10.1016/j.nuclphysa.2003.12.003.
  * [50] J. H. Kelley, J. E. Purcell, C. G. Sheu, Nucl. Phys. A 968 (2017) 71.  doi:10.1016/j.nuclphysa.2017.07.015.
  * [51] M. Wang, G. Audi, F. G. Kondev, W. Huang, S. Naimi, X. Xu, Chin. Phys. C 41 (2017) 030003.  doi:10.1088/1674-1137/41/3/030003.
  * [52] J. M. Blatt, V. F. Weisskopf, Two-Body Problems at Low Energies, Springer New York, New York, 1979.  doi:10.1007/978-1-4612-9959-2_2.
  * [53] F. Catara, A. Insolia, E. Maglione, A. Vitturi, Phys. Rev. C 29 (1984) 1091.  doi:10.1103/PhysRevC.29.1091.
  * [54] A. M. Moro, Phys. Rev. C 92 (2015) 044605.  doi:10.1103/PhysRevC.92.044605.
  * [55] N. Austern, Y. Iseri, M. Kamimura, M. Kawai, G. Rawitscher, M. Yahiro, Phys. Rep. 154 (1987) 125.  doi:10.1016/0370-1573(87)90094-9.
  * [56] S. Hama, B. C. Clark, E. D. Cooper, H. S. Sherif, R. L. Mercer, Phys. Rev. C 41 (1990) 2737.  doi:10.1103/PhysRevC.41.2737.
  * [57] E. D. Cooper, S. Hama, B. C. Clark, R. L. Mercer, Phys. Rev. C 47 (1993) 297.  doi:10.1103/PhysRevC.47.297.


*algorithm

# On the Out of Distribution Robustness of
Foundation Models in Medical Image Segmentation

Duy M. H. Nguyen∗ 1,2,3, Tan N. Pham∗ 3,4, Nghiem T. Diep3, Nghi Phan3,
Quang Pham5, Vinh Tong1,2, Binh T. Nguyen4, Ngan Le6, Nhat Ho7, Pengtao
Xie8,9,
Daniel Sonntag3,10, Mathias Niepert1,2

1University of Stuttgart, 2IMPRS for Intelligent Systems,
3German Research Center for Artificial Intelligence, 4University of Science -
VNUHCM,
5Singapore Management University, 6University of Arkansas, 7University of
Texas at Austin,
8University of California San Diego, 9MBZUAI, 10Oldenburg University

###### Abstract

Constructing a robust model that can effectively generalize to test samples
under distribution shifts remains a significant challenge in the field of
medical imaging. The foundational models for vision and language, pre-trained
on extensive sets of natural image and text data, have emerged as a promising
approach. It showcases impressive learning abilities across different tasks
with the need for only a limited amount of annotated samples. While numerous
techniques have focused on developing better fine-tuning strategies to adapt
these models for specific domains, we instead examine their robustness to
domain shifts in the medical image segmentation task. To this end, we compare
the generalization performance to unseen domains of various pre-trained models
after being fine-tuned on the same in-distribution dataset and show that
foundation-based models enjoy better robustness than other architectures. From
here, we further developed a new Bayesian uncertainty estimation for frozen
models and used them as an indicator to characterize the model’s performance
on out-of-distribution (OOD) data, proving particularly beneficial for real-
world applications. Our experiments not only reveal the limitations of current
indicators like accuracy on the line or agreement on the line commonly used in
natural image applications but also emphasize the promise of the introduced
Bayesian uncertainty. Specifically, lower uncertainty predictions usually tend
to higher out-of-distribution (OOD) performance.

††∗Co-equal contribution. Corresponding email<EMAIL_ADDRESS>
##  1 Introduction

Recent years have witnessed tremendous success of foundation models [1, 2, 3],
which have greatly impacted research in many domains, ranging from
understanding language [4], to vision [5]. Such models are pre-trained on
massive datasets and have shown encouraging capabilities in performing many
tasks, even when only fine-tuned on a small number of samples [6]. Since then,
foundation models have presented unprecedented opportunities for researchers
to explore more challenging and impactful problems. Among these, medical image
understanding [7] has been an attractive topic due to its potential impacts on
our society. However, because of the intrinsic differences between medical and
natural images, directly applying models pre-trained on natural images to the
medical domain may lead to unsatisfactory results [8, 9, 10]. Thus, it is
imperative to investigate the transferability and robustness of foundation
models to unlock their full potential for real-world medical applications.

In this work, we investigate the ability to generalize to unseen distributions
of various deep learning models, especially large foundation models, in the
medical image segmentation task. To this end, we first consider several
popular architectures based on UNet and MedSAM. UNet and its derivatives,
including UNet++ [11] or TransUNet [12], have conventionally served as
prevalent approaches in medical image segmentation. In contrast, MedSAM [13]
is a recent method that focuses on fine-tuning the Segment Anything Model
(SAM) [6]), which is one of the best publicly available models for generic
segmentation [14], on a specific medical dataset. We comprehensively evaluate
such models on various medical image segmentation tasks by training them on
the source domain and then performing evaluations on target domains, which
come from different distributions as illustrated in Figure 1. Across all
datasets, the experiments showed that fine-tuned foundation models such as
MedSAM offer better generalization to unseen domains than traditional models
pre-trained on ImageNet. These results demonstrate the promising capabilities
of foundation models for real-world deployment.

For a reliable real-world deployment, however, one needs to estimate the
model’s OOD performance without actually training the model on the target
domain. For example, consider a collection of pre-trained models that can
segment liver images; given a hospital in a different location, one would like
to select a model that will yield the best segmentation results for the
patients at this location without training all the models in the collection.
However, the lack of labeled samples in the target domain [15, 16]
necessitates an effective strategy to model the OOD performance using only
unlabeled data. To investigate this problem, we first consider several popular
indicators that have shown promising results with natural images, such as the
in-domain (ID) performance [15] or the agreement with another network [17].
Interestingly, our findings show that none of such indicators are applicable
in the medical image segmentation tasks as they do not hold linear
correlations to the OOD performance as expected (Figure 4). This motivates us
to introduce a tailored Bayesian uncertainty estimator designed specifically
for segmentation tasks, aiming to provide a more dependable indicator for
predicting out-of-distribution (OOD) performance. Our experimental results
indicate that higher uncertainty in the model’s predictions consistently
reflects lower OOD performance. In summary, we shed light on the challenges
associated with accurately estimating OOD performance in medical image
segmentation tasks, underscore the limitations of conventional indicators
applied in natural image contexts, and demonstrate the promising results
achieved through the proposed Bayesian uncertainty as a surrogate estimator.

Figure 1: Illustration of domain shift in medical segmentation. From left to
right are pairs of liver (a), prostate (b), and diabetic retina images (c)
selected from different datasets.

##  2 Methodology

###  2.1 Notations and Settings

We denote
$D=\\{(\bm{x}_{i},\bm{y}_{i})\\}_{i=1}^{N}\overset{\text{i.i.d}}{\sim}S$ as a
labeled training set of $N$ samples which are independently and identically
distributed (i.i.d) sampled from a source domain $S$. Here,
$(\bm{x}_{i},\bm{y}_{i})$ represents a pair consisting of an image (flattened
to become a vector) $\bm{x}_{i}\in\mathbb{R}^{m}$ and its label
$\bm{y}_{i}\in\mathbb{R}^{m}$. We introduce a deep network denoted as
$\bm{\Phi}(.;\theta):\mathbb{R}^{m}\rightarrow\mathbb{R}^{m}$, parameterized
by $\theta$, which maps the images from the set $\bm{x}_{i}$ to the desired
outputs $\bm{y}_{i}$. The primary objective of our learning process is to
train a model $\bm{\Phi}$ using the training dataset $D$ where $\theta$ is
possibly initialized from foundational models (SAM) or pre-trained models like
ImageNet, which are trained on large amounts of natural images. The trained
model will then be evaluated for accuracy when applied to samples from a
different target domain $T$. In our setup, the source and target domains, $S$
and $T$, respectively, are chosen from datasets that pertain to the same
organ. However, these datasets experience domain shift issues due to
variations in scanner devices or acquisition conditions (Figure 1).

Figure 2: Predicting OOD performance using uncertainties for a pool of frozen
models. Each trained network $\bm{\Phi}_{k}(.;\theta^{*})$ on a source domain
is paired with an independent network $\bm{\delta}(.;\gamma)$ which takes both
input images $\bm{x}$ and output
$\hat{\bm{y}}=\bm{\Phi}_{k}(\bm{x};\theta^{*})$ and produces uncertainty for
each prediction $\hat{\bm{y}}$ of $\bm{\Phi}_{k}(.;\theta^{*})$ in a target
domain. The uncertainty values are extracted to estimate a correlation to OOD
performance in the target domain for each model.

###  2.2 Prompt-based Segmentation

We examine the generalization capabilities of SAM for a prompt-based
segmentation scenario. We follow the MedSAM method [9] to freeze the image and
prompt encoder layers; only mask decoder layers are learned during the fine-
tuning phase. The prompt layers use box-based prompts generated by rectangles
covering ground-truth regions perturbed with random offsets. We then train it
with training data in the single source domain $S$ and take the trained
network to predict the test set of the target domain $T$. Implementation
details are presented in Section A.1 Appendix.

###  2.3 Performance Prediction on Out-of-Distribution under Uncertainty
Perspective

Our research proposes a new perspective to explore the relationship between
in-distribution (ID) and OOD performance by quantifying the model’s
uncertainty in the target domains. Unlike previous research [15, 16, 17],
which mainly focused on image classification tasks and proposed indicators to
correlate ID and OOD linearly, our settings in medical image segmentation
tasks highlight a crucial difference: none of the previously suggested
indicators are applicable in our domain, as demonstrated in Figure 4, where
relatively modest Pearson correlation coefficients are observed. This
underscores the need for a different approach specific to medical image
analysis to capture such complex relationships. Our focus delves into these
challenges and provides a proof of concept, indicating that leveraging
Bayesian techniques for uncertainty estimation in frozen models can
effectively forecast model performance in out-of-distribution scenarios,
eliminating the need for labeled samples.

####  Formulation:

Given a trained model $\bm{\Phi}(,;\theta^{*})$, for an input $\bm{x}_{i}\in
T$, our goal is to model a distribution of
$\mathcal{P}(\tilde{\bm{y}}|\bm{x}_{i})$ rather than a point estimate
$\hat{\bm{y}}_{i}=\bm{\Phi}(\bm{x}_{i};\theta^{*})$. While Bayesian deep
learning-based approaches [18] can provide such uncertainty estimations, they
require predefined architectures and have to train models from scratch [19,
20]. However, in our context, accessing complete training data during the pre-
trained stage is not feasible, as seen in foundation models, and training
costs can be prohibitively high. To tackle this challenge, we turn to the
latest post-hoc techniques [18, 21, 22, 23, 24, 25] for estimating uncertainty
in tasks such as image translation, image enhancement, and depth prediction in
self-driving cars. This technique has recently been adapted to active learning
research [23] and Vision-Language Models [25]. We tailor these methods for
medical segmentation, where regions of interest are often small and surrounded
by similar tissue structures. We depict our method in Figure 2.

In particular, we construct an probabilistic model $\bm{\delta}(.;\gamma)$ to
estimate uncertainty for the frozen model $\bm{\Phi}(.;\theta)$ by producing
an independent multivariate Gaussian distribution
$\mathcal{P}\left(\tilde{\bm{y}};\\{\bm{\mu}_{i},\mathrm{diag}(\bm{\sigma}_{i}^{2})\\}\right)$
parameterized by
$\\{\bm{\mu}_{i},\bm{\sigma}_{i}^{2}\\}=\bm{\delta}(\\{\hat{\bm{y}}_{i},\bm{x}_{i}\\};\gamma)$
for every input-output pair $\\{\bm{x}_{i},\hat{\bm{y}}_{i}\\}$. This
distribution conveys information about the possible values of the
reconstructed output $\tilde{\bm{y}}_{i}$ and the uncertainty
$\bm{\sigma}^{2}_{i}$ of the prediction. We seek to optimize the uncertainty
estimator to maximize the data likelihood:

$\displaystyle\gamma^{*}$
$\displaystyle=\operatorname*{arg\,max}\limits_{\gamma}\prod_{i=1}^{N}\,\mathcal{P}\left({\bm{y}_{i}};\\{\bm{\mu}_{i},\mathrm{diag}(\bm{\sigma}_{i}^{-2})\\}\right)$
(1)
$\displaystyle=\operatorname*{arg\,max}\limits_{\gamma}\prod_{i=1}^{N}\,\frac{1}{(2\pi)^{m/2}\left(\prod_{j=1}^{m}\sigma_{ij}^{2}\right)^{\frac{1}{2}}}\exp\left(-\frac{1}{2}({\bm{y}_{i}}-\bm{\mu}_{i})^{\top}\mathrm{diag}(\bm{\sigma}_{i}^{-2})(\bm{y}_{i}-\bm{\mu}_{i})\right)$
(2)
$\displaystyle=\operatorname*{arg\,max}\limits_{\gamma}\prod_{i=1}^{N}\,\prod_{j=1}^{m}\frac{1}{(2\pi)^{1/2}\sigma_{ij}}\,\exp\left(-\frac{1}{2\sigma_{ij}^{2}}({\bm{y}_{ij}}-\bm{\mu}_{ij})^{2}\right)$
(3)
$\displaystyle=\operatorname*{arg\,min}\limits_{\gamma}\sum_{i=1}^{N}\,\sum_{j=1}^{m}\,\frac{({\bm{y}_{ij}}-\bm{\mu}_{ij})^{2}}{2\sigma_{ij}^{2}}+\frac{\log{\sigma_{ij}^{2}}}{2}.$
(4)

In order to reconstruct output of $\bm{\Phi}(,;\theta)$ from
$\bm{\delta}(.;\gamma)$, we can extend Eq.(4) to:

$\displaystyle\gamma^{*}=\operatorname*{arg\,min}\limits_{\gamma}\,\sum_{i=1}^{N}\,\sum_{j=1}^{m}\,\frac{({\bm{y}_{ij}}-\bm{\mu}_{ij})^{2}}{2\sigma_{ij}^{2}}+\frac{\log{\sigma_{ij}^{2}}}{2}+\lambda\|\bm{\mu}_{i}-\hat{\bm{y}}_{i}\|^{2}$
(5)

where $\lambda$ indicates the hyper-parameter controlling the trade-off
between maximum likelihood and reconstructed output contributions. After
optimizing Eq.(5) on samples of source domain $S$, we then can compute
uncertainty for predictions
$\hat{\bm{y}}_{t}=\bm{\Phi}(\bm{x}_{t};\theta^{*})$ with $\bm{x}_{t}\in T$ by:

$\displaystyle\mathrm{Uncertainty}(\hat{\bm{y}}_{t})=\bm{\sigma}_{t}^{2}\in\mathbb{R}^{m},\;\text{where
}\\{\bm{\mu}_{t},\,\bm{\sigma}_{t}^{2}\\}=\bm{\delta}(\\{\hat{\bm{y}}_{t},\bm{x}_{t}\\};\gamma^{*}).$
(6)

####  Generalized Uncertainty with Heavy-Tailed Distribution:

Note that Eq. (3) can be seen as a product of univariate Gaussians
$\mathcal{N}(\bm{\mu}_{ij},\sigma_{ij}^{2})$ modeling distributions at the
per-pixel level. Therefore, one can extend them to generalized Gaussian
distributions [26] $\mathcal{GGD}(\bm{\mu}_{ij},\alpha_{ij},\,\beta_{ij})$
with scale $\alpha_{ij}$ and shape $\beta_{ij}$ coefficients used to model
heavy-tailed distribution at pixels which usually occur due to the presence of
noises or artifacts [22, 24, 25]. The uncertainty for the prediction of
$\bm{x}_{t}\in T$ in this case is computed as:

$\displaystyle\\{\bm{\mu}_{t},\,\bm{\beta}_{t},\bm{\alpha}_{t}\\}=\bm{\delta}(\\{\hat{\bm{y}}_{t},\bm{x}_{t}\\};\gamma^{*});\
\bm{\beta}_{t}=\left[\beta_{t1},...,\beta_{tm}\right];\bm{\alpha}_{t}=\left[\alpha_{t1},...,\alpha_{tm}\right]$
(7)
$\displaystyle\mathrm{Uncertainty}(\hat{\bm{y}}_{t})=\left[\frac{\alpha_{t1}^{2}\Gamma\left(3\beta_{t1}^{-1}\right)}{\Gamma\,\left(\beta_{t1}^{-1}\right)},...,\frac{\alpha_{km}^{2}\Gamma\left(3\beta_{tm}^{-1}\right)}{\Gamma\,\left(\beta_{tm}^{-1}\right)}\right]^{\top}\in\mathbb{R}^{m}.$
(8)

where $\Gamma(z)=\bigintsss_{\,0}^{\mathrm{inf}}x^{z-1}\exp(-x)dx,\ \forall
z>0$ is the Gamma function.

###  2.4 Comparison to Existing Models for Post-hoc Uncertainty Quantification

Our formulation is similar to that of existing approaches [22, 24, 25] in that
we also estimate the uncertainty of the frozen model $\bm{\Phi}(.;\theta^{*})$
by training an auxiliary network $\bm{\delta}(.;\gamma)$ with the same
objective. However, instead of conditioning only on the output
$\hat{\bm{y}}_{i}=\bm{\Phi}(\bm{x}_{i};\theta^{*})$, i.e.,
$\mathrm{Uncertainty}(\hat{\bm{y}}_{i})\sim\bm{\delta}(\hat{\bm{y}}_{i};\gamma^{*})$,
we propose a model for uncertainty quantification $\bm{\delta}(.;\gamma)$
driven by both $\hat{\bm{y}}_{i}$ and the original image input $\bm{x}_{i}$
for the medical segmentation by
$\mathrm{Uncertainty}(\hat{\bm{y}}_{i})\sim\bm{\delta}(\\{\hat{\bm{y}}_{i},\bm{x}_{i}\\};\gamma^{*})$.
This is motivated by the following observations.

First, in segmentation settings, the outputs $\hat{\bm{y}}_{i}$ produced by
the frozen models are simply binary masks and, therefore, there is no
surrounding context for $\bm{\delta}(.;\gamma)$ to learn the maximum
likelihood conditions as in Eq.(4). In other words, the model tends to
reconstruct only the output of frozen models $\bm{\Phi}(,;\theta^{*})$ by
minimizing $\|\bm{\mu}_{i}-\hat{\bm{y}}_{i}\|^{2}$ while tending to over-fit
the remaining components in Eq.(5). This is essentially different from other
methods [22, 25, 24] designed for image enhancement, image translation, or
depth estimation, wherein outputs $\hat{\bm{y}}_{i}$ are continuous signals
and have high mutual information with input images $\bm{x}_{i}$.

Second, under a probabilistic view, we have:

$\mathcal{P}\left(\mathrm{Uncertainty}(\hat{\bm{y}}),\hat{\bm{y}}\,|\,\bm{x}\right)=\mathcal{P}\left(\mathrm{Uncertainty}(\hat{\bm{y}})\,|\,\hat{\bm{y}},\bm{x}\right)\cdot\mathcal{P}\left(\hat{\bm{y}}\,|\,\bm{x}\right).$
(9)

Both our model $\bm{\delta}(;\gamma)$ and existing methods estimate
$\mathcal{P}\left(\hat{\bm{y}}|\bm{x}\right)$ by reconstructing the output of
the frozen model $\hat{\bm{y}}=\bm{\Phi}(\bm{x}|\theta^{*})$ trained during
the pre-training step. However, the remaining factor
$\mathcal{P}\left(\mathrm{Uncertainty}(\hat{\bm{y}})\,|\,\hat{\bm{y}},\bm{x}\right)$
is learnt by utilizing pairs of $\\{\hat{\bm{y}},\bm{x}\\}$ in
$\bm{\delta}(;\gamma)$ while other methods approximate
$\mathcal{P}\left(\mathrm{Uncertainty}(\hat{\bm{y}})\,|\,\hat{\bm{y}},\bm{x}\right)\approx\mathcal{P}\left(\mathrm{Uncertainty}(\hat{\bm{y}})\,|\,\hat{\bm{y}}\right)$.
This approximation fails, however, if $\hat{\bm{y}}$ does not contain
sufficient information about the input $\bm{x}$ as is the case in the
segmentation settings.

Finally, instead of computing averaging uncertainty values as prior works, we
utilize Otsu’s parameter-free thresholding algorithm [27] on the uncertainty
matrix. This identifies pixel groups with the highest uncertainty, allowing us
to measure their areas. Our approach, particularly effective for small regions
of interest in segmentation tasks, exhibits a stronger correlation to OOD
scenarios.

Table 1: Performance comparison in domain-shift of prostate segmentation.
Results are reported in average 3D IoU score with three training times. The
BMC and BIDMC datasets are selected as source domains, and the remaining
datasets are used as target domains. Arrows $\uparrow$ and $\downarrow$
indicate the increase/drop performance between target and source domains.

|  | UNet (R50) | Unet ++ (R50) | Unet (Eff. Net) | UNet ++ (Eff. Net) | TransUNet | MedSAM
---|---|---|---|---|---|---|---
Same- Dom. | BMC | 76.77 $\pm$ 0.47 | 75.99 $\pm$ 1.21 | 77.86 $\pm$ 1.12 | 77.63 $\pm$ 1.00 | 65.61 $\pm$ 2.72 | 91.38 $\pm$ 0.69
RUNMC | 77.92 $\pm$ 1.39 | 78.62 $\pm$ 0.24 | 79.33 $\pm$ 0.92 | 79.79 $\pm$ 0.36 | 66.17 $\pm$ 1.64 | 83.75 $\pm$ 0.87
BIDMC | 71.44 $\pm$ 1.28 | 73.12 $\pm$ 0.41 | 75.55 $\pm$ 0.66 | 74.42 $\pm$ 0.45 | 49.30 $\pm$ 8.22 | 85.04 $\pm$ 1.80
HK | 71.34 $\pm$ 0.85 | 70.76 $\pm$ 1.48 | 70.45 $\pm$ 1.43 | 69.69 $\pm$ 3.50 | 44.58 $\pm$ 4.49 | 82.36 $\pm$ 1.78
Cross-Dom. $S\rightarrow T$ | BMC $\rightarrow$ RUNMC | 63.37 $\pm$ 1.85 | 62.74 $\pm$ 2.67 | 58.29 $\pm$ 5.09 | 56.34 $\pm$ 5.43 | 21.51 $\pm$ 2.28 | 85.23 $\pm$ 1.19
($\downarrow$ 13.4) | ($\downarrow$ 13.3) | ($\downarrow$ 19.6) | ($\downarrow$ 21.3) | ($\downarrow$ 44.1) | ($\downarrow$ 6.1)
BMC $\rightarrow$ BIDMC | 51.22 $\pm$ 16.52 | 55.18 $\pm$ 3.78 | 46.35 $\pm$ 9.48 | 43.21 $\pm$ 7.95 | 4.67 $\pm$ 0.47 | 84.73 $\pm$ 1.05
($\downarrow$ 25.5) | ($\downarrow$ 20.8) | ($\downarrow$ 31.5) | ($\downarrow$ 34.4) | ($\downarrow$ 60.9) | ($\downarrow$ 6.6)
BMC $\rightarrow$ HK | 56.11 $\pm$ 4.93 | 55.34 $\pm$ 6.01 | 34.61 $\pm$ 6.66 | 36.11 $\pm$ 5.29 | 13.11 $\pm$ 4.66 | 83.40 $\pm$ 0.44
($\downarrow$ 20.7) | ($\downarrow$ 20.7) | ($\downarrow$ 43.3) | ($\downarrow$ 41.5) | ($\downarrow$ 52.5) | ($\downarrow$ 8.0)
BIDMC $\rightarrow$ BMC | 27.07 $\pm$ 1.67 | 24.86 $\pm$ 9.09 | 45.33 $\pm$ 6.39 | 42.41 $\pm$ 7.27 | 6.40 $\pm$ 1.33 | 90.77 $\pm$ 0.69
($\downarrow$ 44.4) | ($\downarrow$ 48.3) | ($\downarrow$ 30.2) | ($\downarrow$ 32.0) | ($\downarrow$ 42.9) | ($\uparrow$ 5.7)
BIDMC $\rightarrow$ RUNMC | 4.41 $\pm$ 0.69 | 6.37 $\pm$ 1.41 | 20.81 $\pm$ 9.45 | 18.12 $\pm$ 11.04 | 4.19 $\pm$ 0.54 | 81.27 $\pm$ 1.05
($\downarrow$ 67.0) | ($\downarrow$ 66.8) | ($\downarrow$ 54.7) | ($\downarrow$ 56.3) | ($\downarrow$ 45.1) | ($\downarrow$ 3.8)
BIDMC $\rightarrow$ HK | 52.83 $\pm$ 2.62 | 50.16 $\pm$ 2.09 | 47.59 $\pm$ 1.52 | 48.23 $\pm$ 4.30 | 24.64 $\pm$ 4.94 | 81.19 $\pm$ 0.92
($\downarrow$ 18.6) | ($\downarrow$ 23.0) | ($\downarrow$ 28.0) | ($\downarrow$ 26.2) | ($\downarrow$ 24.7) | ($\downarrow$ 3.9)

##  3 Experiment Results

###  3.1 Datasets and Implementations

We experiment on three segmentation tasks, including Diabetic Retinopathy (DR)
grading-related lesion segmentation in 2D fundus images, liver structure
segmentation from 3D CT scans, and prostate segmentation from 3D MRI data.
Table 5 in the Appendix provides details about each task, including
information about the source and target domains, aiming to explore challenges
related to domain shifts. We use the default training, testing split in each
dataset if available; otherwise, we randomly select $80\%$ total samples for
training and $20\%$ remaining for testing. On 3D segmentation problems, we
reformulate them as independent 2D slice segmentation and merge results to a
single 3D volume to compare with ground truths.

The large vision model SAM [6] is bench-marked by utilizing the MedSAM method
[13] to fine-tune on a specific medical downstream task. It is important to
emphasize that the SAM model was not previously pre-trained with extensive
medical images, aligning with the approach taken by pre-trained ImageNet
models [28]. We also compare SAM against popular supervised methods such as
TransUNet with ViT-16 backbone [29], U-Net [30] and U-Net++ [11] with ResNet50
(R50) [31], and Efficient-Net b0 (Eff.Net) [32]. All weights are initialized
from ImageNet [28]. Details for uncertainty network
$\bm{\delta}(\\{\mathbb{\hat{y}},\mathbb{x}\\};\gamma^{*})$ are presented in
Appendix.

###  3.2 Quantitative Performance on Cross-domain

We report the performance of various model architectures on different medical
modalities in Tables 1, 2, and 3. Each model was initially trained and
evaluated within the same domain. Subsequently, training is conducted on the
source domain $S$, and evaluation is performed on the target domain $T$,
adhering to the cross-domain $S\rightarrow T$ setting. Across all scenarios,
MedSAM consistently demonstrates superior in- and out-of-domain performances,
significantly surpassing other models. This noteworthy achievement highlights
MedSAM’s robust OOD generalization capabilities, proven effective across both
balanced (prostate modality) and imbalanced (diabetic retinopathy lesion and
liver modalities) datasets. Further detailed analysis is available in the
Appendix. To our latest knowledge, we first examine the robustness of the SAM
model under domain shift in medical segmentation tasks, given models are fine-
tuned on medical source domains [33, 34].

Table 2: Performance comparison in domain-shift of DR lesion segmentation.
Results are reported in average 2D Dice score with three training times.
Arrows $\uparrow$ and $\downarrow$ indicate the increase/drop performance.

| Same-Domain | Cross-Domain $S\rightarrow T$
---|---|---
| IDRID | FGADR | IDRID $\rightarrow$ FGADR | FGADR $\rightarrow$ IDRID
UNet (R50) | 35.72 $\pm$ 1.35 | 49.46 $\pm$ 1.07 | 13.51 $\pm$ 5.56 ($\downarrow$ 22.21) | 31.76 $\pm$ 1.61 ($\downarrow$ 17.7)
Unet ++ (R50) | 32.36 $\pm$ 3.70 | 48.73 $\pm$ 1.42 | 10.42 $\pm$ 2.19 ($\downarrow$ 21.9) | 30.60 $\pm$ 1.31 ($\downarrow$ 18.1)
Unet (Eff.Net) | 34.18 $\pm$ 2.00 | 48.78 $\pm$ 0.59 | 12.72 $\pm$ 1.24 ($\downarrow$ 21.5) | 33.61 $\pm$ 1.29 ($\downarrow$ 15.2)
UNet ++ (Eff.Net) | 36.70 $\pm$ 1.61 | 49.88 $\pm$ 0.85 | 20.51 $\pm$ 1.41 ($\downarrow$ 16.2) | 32.49 $\pm$ 2.08 ($\downarrow$ 17.4)
TransUNet | 15.28 $\pm$ 1.62 | 46.59 $\pm$ 0.80 | 11.58 $\pm$ 11.04 ($\downarrow$ 3.7) | 22.52 $\pm$ 2.39 ($\downarrow$ 24.1)
MedSAM | 37.76 $\pm$ 1.37 | 58.49 $\pm$ 0.29 | 44.63 $\pm$ 1.42<EMAIL_ADDRESS>| 39.24 $\pm$ 0.62 ($\downarrow$ 19.25)

Table 3: Performance comparison in domain-shift of liver segmentation. Results
are computed by average 2D Dice score in three training times. Arrows
$\uparrow$ and $\downarrow$ indicate the increase/drop performance.

| Same-Domain |  Cross-Domain $S\rightarrow T$
---|---|---
| FLARE | LiTS | FLARE $\rightarrow$ LiTS | LiTS $\rightarrow$ FLARE
UNet (R50) | 94.35 $\pm$ 1.16 | 95.69 $\pm$ 0.09 | 82.28 $\pm$ 4.41 ($\downarrow$ 12.1) | 95.57 $\pm$ 0.39 ($\downarrow$ 0.1)
Unet ++ (R50) | 96.08 $\pm$ 0.40 | 95.84 $\pm$ 0.12 | 72.51 $\pm$ 5.97 ($\downarrow$ 23.6) | 95.41 $\pm$ 0.45 ($\downarrow$ 0.4)
Unet (Eff. Net) | 95.11 $\pm$ 0.18 | 95.86 $\pm$ 0.23 | 68.91 $\pm$ 7.06 ($\downarrow$ 26.2) | 95.04 $\pm$ 0.63 ($\downarrow$ 0.8)
UNet ++ (Eff. Net) | 95.23 $\pm$ 0.98 | 95.57 $\pm$ 0.40 | 67.32 $\pm$ 3.27 ($\downarrow$ 27.91) | 95.0 $\pm$ 0.95 ($\downarrow$ 0.6)
TransUNet | 92.01 $\pm$ 0.78 | 92.22 $\pm$ 3.22 | 61.69 $\pm$ 1.17 ($\downarrow$ 30.3) | 93.2 $\pm$ 1.68 ($\uparrow$ 1.0)
MedSAM | 92.47 $\pm$ 0.02 | 97.80 $\pm$ 0.01 | 97.53 $\pm$ 0.03 ($\uparrow$ 5.1) | 92.17 $\pm$ 0.24 ($\downarrow$ 5.6)

###  3.3 Estimating Out-of-Distribution Performance via Uncertainty
Perspective

We now investigate different indicators to characterize the models’ OOD
properties. We consider both popular indicators used in the natural image
domains, such as the ID performance [15], the agreement [17], and our proposed
Bayesian uncertainty.

Figure 3 illustrates a strong correlation between uncertainty and OOD
performance in three settings of prostate segmentation (Table 1), underscoring
that elevated uncertainty values tend to align with diminished OOD
performance. Additionally, when comparing different models on the same
dataset, the OOD performance gaps between these models are found to be
correlated with the gaps in their uncertainties. However, it is important to
highlight that uncertainty only approximated the true error in this context.
As a result, even when two models exhibit similar OOD performances, there may
still be some subtle differences in their uncertainties, reflecting nuanced
variations in their predictive capabilities. This phenomenon presents an
interesting future research direction for accurately estimating a model OOD
performance.

Figure 3: Visualization of uncertainty and OOD performance on three cross-
domain experiments in prostate segmentation. Uncertainty exhibits a strong
correlation with OOD performance, indicating that higher uncertainty values
tend to correspond to lower OOD performance.

####  Comparing with Other Correlations:

Figure 4 reports the correlations between the OOD performance and the ID
performance or the ID agreement when using BMC as the source domain and BIDMC
as the target domain. By varying the training configurations, such as learning
rates, epochs, etc., we plot the OOD performance against the ID indicator for
each model. Afterward, we compute Pearson correlation coefficients between the
OOD performance and ID indicators. This analysis aims to substantiate whether
linear correlations, as proposed in previous studies [15, 17], are indeed
observed. Nevertheless, none of these indicators proves effective in
adequately characterizing the OOD performance, as they only yield relatively
low Pearson correlation coefficients. Consequently, we can infer that the
currently employed natural image indicators are not well-suited for our
specific medical image segmentation task.

Figure 4: OOD performance/agreement versus ID performance/agreement on BMC and
HK datasets (left) and on BMC and BIDMC datasets (right). Table 4: Comparison
among uncertainty approaches
measured by mean squared error. Smaller is better.

Method |  BMC $\rightarrow$ RUNMC |  BMC $\rightarrow$ BIDMC |  BMC $\rightarrow$ HK
---|---|---|---
Our | 0.012 | 0.011 | 0.018
BayesCap | 0.015 | 0.017 | 0.021
TTDAp | 0.247 | 0.326 | 0.387
TTDAc | 0.212 | 0.248 | 0.291
DropOut | 0.197 | 0.227 | 0.271

####  Comparing with Uncertainty Baselines:

We furthermore compare our approach against two different baseline groups,
including Bayesian identity mapping [22] and perturbing the original images to
gauge the uncertain level for the SAM model. The former group, known as
BayesCap [22], calibrates the uncertainties of a pre-trained model without the
need to train itself on a large-scale dataset. The latter is based on test-
time modification techniques [35, 36, 37]. Herein, we employ two types of
modifications: test-time data augmentation (TTDA) [35, 36] and dropout [38]
before the final prediction [39, 40]. Two classes of data augmentation are
adapted for baselines, including pixel-wise noise (TTDAp) and color jittering
(TTDAc). Table 4 illustrates various methods’ performance in three cross-
domain prostate segmentation settings. The results are presented as mean
square errors between the uncertainty values, estimated using the Otsu
thresholding algorithm, and the actual errors of the model. Notably, our
approach consistently outperforms others, attaching the top records across all
three Out-of-Domain (OOD) settings.

##  4 Conclusion

In this study, we first conducted a systematic investigation into the
performance of the foundation model SAM under domain-shift segmentation tasks.
Our findings reveal that this foundation model demonstrates better
generalization capabilities than other methods initialized from pre-trained
ImageNet. Additionally, we show the correlation between model uncertainties
and their out-of-distribution performance; thereby, lower uncertainty
predictions tend to reflect a higher performance in OOD. To achieve this, we
constructed a post-hoc estimation approach tailored for pre-trained
deterministic models. These models have demonstrated the ability to
consistently generate well-calibrated uncertainty estimates across various
segmentation scenarios, proving their utility in aiding experts during the
decision-making process. One limitation of our algorithm is that the
uncertainty can only approximate the true errors; therefore, it may not be
suitable to compare models whose performances have small margins. In future
work, we plan to improve this issue as well as explore the proposed method in
a broader range of foundation methods with diverse settings to gain better
insights into the behaviors of the proposed algorithms.

## Acknowledgements

This research has been supported by the pAItient project (BMG, 2520DAT0P2),
Ophthalmo-AI project (BMBF, 16SV8639), and Endowed Chair of Applied Artificial
Intelligence, Oldenburg University, Germany. Tan Ngoc Pham would like to thank
the Vingroup Innovation Foundation (VINIF) for the Master’s training
scholarship program. The authors thank the International Max Planck Research
School for Intelligent Systems (IMPRS-IS) for supporting Duy M. H. Nguyen and
Vinh Tong. Mathias Niepert acknowledges funding by Deutsche
Forschungsgemeinschaft (DFG, German Research Foundation) under Germany’s
Excellence Strategy - EXC and support by the Stuttgart Center for Simulation
Science (SimTech).

## References

  * Bommasani et al. [2021] Rishi Bommasani, Drew A Hudson, Ehsan Adeli, Russ Altman, Simran Arora, Sydney von Arx, Michael S Bernstein, Jeannette Bohg, Antoine Bosselut, Emma Brunskill, et al.  On the opportunities and risks of foundation models.  _arXiv preprint arXiv:2108.07258_ , 2021.
  * Floridi and Chiriatti [2020] Luciano Floridi and Massimo Chiriatti.  Gpt-3: Its nature, scope, limits, and consequences.  _Minds and Machines_ , 30:681–694, 2020.
  * Devlin et al. [2018] Jacob Devlin, Ming-Wei Chang, Kenton Lee, and Kristina Toutanova.  Bert: Pre-training of deep bidirectional transformers for language understanding.  _arXiv preprint arXiv:1810.04805_ , 2018.
  * Touvron et al. [2023] Hugo Touvron, Thibaut Lavril, Gautier Izacard, Xavier Martinet, Marie-Anne Lachaux, Timothée Lacroix, Baptiste Rozière, Naman Goyal, Eric Hambro, Faisal Azhar, et al.  Llama: Open and efficient foundation language models.  _arXiv preprint arXiv:2302.13971_ , 2023.
  * Oquab et al. [2023] Maxime Oquab, Timothée Darcet, Théo Moutakanni, Huy Vo, Marc Szafraniec, Vasil Khalidov, Pierre Fernandez, Daniel Haziza, Francisco Massa, Alaaeldin El-Nouby, et al.  Dinov2: Learning robust visual features without supervision.  _arXiv preprint arXiv:2304.07193_ , 2023.
  * Kirillov et al. [2023] Alexander Kirillov, Eric Mintun, Nikhila Ravi, Hanzi Mao, Chloe Rolland, Laura Gustafson, Tete Xiao, Spencer Whitehead, Alexander C Berg, Wan-Yen Lo, et al.  Segment anything.  _arXiv preprint arXiv:2304.02643_ , 2023.
  * Willemink et al. [2022] Martin J Willemink, Holger R Roth, and Veit Sandfort.  Toward foundational deep learning models for medical imaging in the new era of transformer networks.  _Radiology: Artificial Intelligence_ , 4(6):e210284, 2022.
  * Nguyen et al. [2022] Duy MH Nguyen, Thu T Nguyen, Huong Vu, Quang Pham, Manh-Duy Nguyen, Binh T Nguyen, and Daniel Sonntag.  Tatl: task agnostic transfer learning for skin attributes detection.  _Medical Image Analysis_ , 78:102359, 2022.
  * Ma and Wang [2023] Jun Ma and Bo Wang.  Segment anything in medical images.  _arXiv preprint arXiv:2304.12306_ , 2023.
  * Shi et al. [2023] Peilun Shi, Jianing Qiu, Sai Mu Dalike Abaxi, Hao Wei, Frank P-W Lo, and Wu Yuan.  Generalist vision foundation models for medical imaging: A case study of segment anything model on zero-shot medical segmentation.  _arXiv preprint arXiv:2304.12637_ , 2023.
  * Zhou et al. [2018] Zongwei Zhou, Md Mahfuzur Rahman Siddiquee, Nima Tajbakhsh, and Jianming Liang.  Unet++: A nested u-net architecture for medical image segmentation.  In _Deep Learning in Medical Image Analysis and Multimodal Learning for Clinical Decision Support: 4th International Workshop, DLMIA 2018, and 8th International Workshop, ML-CDS 2018, Held in Conjunction with MICCAI 2018, Granada, Spain, September 20, 2018, Proceedings 4_ , pages 3–11. Springer, 2018.
  * Chen et al. [2021] Jieneng Chen, Yongyi Lu, Qihang Yu, Xiangde Luo, Ehsan Adeli, Yan Wang, Le Lu, Alan L Yuille, and Yuyin Zhou.  Transunet: Transformers make strong encoders for medical image segmentation.  _arXiv preprint arXiv:2102.04306_ , 2021.
  * Huang et al. [2023] Yuhao Huang, Xin Yang, Lian Liu, Han Zhou, Ao Chang, Xinrui Zhou, Rusi Chen, Junxuan Yu, Jiongquan Chen, Chaoyu Chen, et al.  Segment anything model for medical images?  _arXiv preprint arXiv:2304.14660_ , 2023.
  * Minaee et al. [2021] Shervin Minaee, Yuri Boykov, Fatih Porikli, Antonio Plaza, Nasser Kehtarnavaz, and Demetri Terzopoulos.  Image segmentation using deep learning: A survey.  _IEEE transactions on pattern analysis and machine intelligence_ , 44(7):3523–3542, 2021.
  * Miller et al. [2021] John P Miller, Rohan Taori, Aditi Raghunathan, Shiori Sagawa, Pang Wei Koh, Vaishaal Shankar, Percy Liang, Yair Carmon, and Ludwig Schmidt.  Accuracy on the line: on the strong correlation between out-of-distribution and in-distribution generalization.  In _International Conference on Machine Learning_ , pages 7721–7735. PMLR, 2021.
  * Deng et al. [2022] Weijian Deng, Stephen Gould, and Liang Zheng.  On the strong correlation between model invariance and generalization.  _Advances in Neural Information Processing Systems_ , 35:28052–28067, 2022.
  * Baek et al. [2022] Christina Baek, Yiding Jiang, Aditi Raghunathan, and J Zico Kolter.  Agreement-on-the-line: Predicting the performance of neural networks under distribution shift.  _Advances in Neural Information Processing Systems_ , 35:19274–19289, 2022.
  * Daxberger et al. [2021] Erik Daxberger, Agustinus Kristiadi, Alexander Immer, Runa Eschenhagen, Matthias Bauer, and Philipp Hennig.  Laplace redux-effortless bayesian deep learning.  _Advances in Neural Information Processing Systems_ , 34:20089–20103, 2021.
  * Godard et al. [2017] Clément Godard, Oisin Mac Aodha, and Gabriel J Brostow.  Unsupervised monocular depth estimation with left-right consistency.  In _Proceedings of the IEEE conference on computer vision and pattern recognition_ , pages 270–279, 2017.
  * Dosovitskiy et al. [2015] Alexey Dosovitskiy, Philipp Fischer, Eddy Ilg, Philip Hausser, Caner Hazirbas, Vladimir Golkov, Patrick Van Der Smagt, Daniel Cremers, and Thomas Brox.  Flownet: Learning optical flow with convolutional networks.  In _Proceedings of the IEEE international conference on computer vision_ , pages 2758–2766, 2015.
  * Eschenhagen et al. [2021] Runa Eschenhagen, Erik Daxberger, Philipp Hennig, and Agustinus Kristiadi.  Mixtures of laplace approximations for improved post-hoc uncertainty in deep learning.  _arXiv preprint arXiv:2111.03577_ , 2021.
  * Upadhyay et al. [2022] Uddeshya Upadhyay, Shyamgopal Karthik, Yanbei Chen, Massimiliano Mancini, and Zeynep Akata.  Bayescap: Bayesian identity cap for calibrated uncertainty in frozen neural networks.  In _European Conference on Computer Vision_ , pages 299–317. Springer, 2022.
  * Rangnekar et al. [2023] Vikrant Rangnekar, Uddeshya Upadhyay, Zeynep Akata, and Biplab Banerjee.  Usim-dal: Uncertainty-aware statistical image modeling-based dense active learning for super-resolution.  _arXiv preprint arXiv:2305.17520_ , 2023.
  * Upadhyay et al. [2023a] Uddeshya Upadhyay, Sairam Bade, Arjun Puranik, Shahir Asfahan, Melwin Babu, Francisco Lopez-Jimenez, Samuel Asirvatham, Ashim Prasad, Ajit Rajasekharan, Samir Awasthi, et al.  Hypuc: Hyperfine uncertainty calibration with gradient-boosted corrections for reliable regression on imbalanced electrocardiograms.  _Transactions on Machine Learning Research_ , 2023a.
  * Upadhyay et al. [2023b] Uddeshya Upadhyay, Shyamgopal Karthik, Massimiliano Mancini, and Zeynep Akata.  Probvlm: Probabilistic adapter for frozen vison-language models.  In _Proceedings of the IEEE/CVF International Conference on Computer Vision_ , pages 1899–1910, 2023b.
  * Dytso et al. [2018] Alex Dytso, Ronit Bustin, H Vincent Poor, and Shlomo Shamai.  Analytical properties of generalized gaussian distributions.  _Journal of Statistical Distributions and Applications_ , 5(1):1–40, 2018.
  * Otsu [1979] Nobuyuki Otsu.  A threshold selection method from gray-level histograms.  _IEEE transactions on systems, man, and cybernetics_ , 9(1):62–66, 1979.
  * Deng et al. [2009] Jia Deng, Wei Dong, Richard Socher, Li-Jia Li, Kai Li, and Li Fei-Fei.  Imagenet: A large-scale hierarchical image database.  In _2009 IEEE conference on computer vision and pattern recognition_ , pages 248–255. Ieee, 2009.
  * Dosovitskiy et al. [2021] Alexey Dosovitskiy, Lucas Beyer, Alexander Kolesnikov, Dirk Weissenborn, Xiaohua Zhai, Thomas Unterthiner, Mostafa Dehghani, Matthias Minderer, Georg Heigold, Sylvain Gelly, et al.  An image is worth 16x16 words: Transformers for image recognition at scale.  In _International Conference on Learning Representatio_ , 2021.
  * Ronneberger et al. [2015] Olaf Ronneberger, Philipp Fischer, and Thomas Brox.  U-net: Convolutional networks for biomedical image segmentation.  In _Medical Image Computing and Computer-Assisted Intervention–MICCAI 2015: 18th International Conference, Munich, Germany, October 5-9, 2015, Proceedings, Part III 18_ , pages 234–241. Springer, 2015.
  * He et al. [2016] Kaiming He, Xiangyu Zhang, Shaoqing Ren, and Jian Sun.  Deep residual learning for image recognition.  In _Proceedings of the IEEE conference on computer vision and pattern recognition_ , pages 770–778, 2016.
  * Tan and Le [2019] Mingxing Tan and Quoc Le.  Efficientnet: Rethinking model scaling for convolutional neural networks.  In _International conference on machine learning_ , pages 6105–6114. PMLR, 2019.
  * Mazurowski et al. [2023] Maciej A Mazurowski, Haoyu Dong, Hanxue Gu, Jichen Yang, Nicholas Konz, and Yixin Zhang.  Segment anything model for medical image analysis: an experimental study.  _Medical Image Analysis_ , 89:102918, 2023.
  * Qiao et al. [2023] Yu Qiao, Chaoning Zhang, Taegoo Kang, Donghun Kim, Shehbaz Tariq, Chenshuang Zhang, and Choong Seon Hong.  Robustness of sam: Segment anything under corruptions and beyond.  _arXiv preprint arXiv:2306.07713_ , 2023.
  * Ayhan and Berens [2022] Murat Seckin Ayhan and Philipp Berens.  Test-time data augmentation for estimation of heteroscedastic aleatoric uncertainty in deep neural networks.  In _Medical Imaging with Deep Learning_ , 2022.
  * Wang et al. [2022] Guotai Wang, Wenqi Li, Michael Aertsen, Jan Deprest, Sebastien Ourselin, and Tom Vercauteren.  Test-time augmentation with uncertainty estimation for deep learning-based medical image segmentation.  2022\.
  * Wang et al. [2019] Guotai Wang, Wenqi Li, Michael Aertsen, Jan Deprest, Sébastien Ourselin, and Tom Vercauteren.  Aleatoric uncertainty estimation with test-time augmentation for medical image segmentation with convolutional neural networks.  _Neurocomputing_ , 338:34–45, 2019.
  * Srivastava et al. [2014] Nitish Srivastava, Geoffrey Hinton, Alex Krizhevsky, Ilya Sutskever, and Ruslan Salakhutdinov.  Dropout: a simple way to prevent neural networks from overfitting.  _The journal of machine learning research_ , 15(1):1929–1958, 2014.
  * Kendall et al. [2015] Alex Kendall, Vijay Badrinarayanan, and Roberto Cipolla.  Bayesian segnet: Model uncertainty in deep convolutional encoder-decoder architectures for scene understanding.  _arXiv preprint arXiv:1511.02680_ , 2015.
  * Gal and Ghahramani [2016] Yarin Gal and Zoubin Ghahramani.  Dropout as a bayesian approximation: Representing model uncertainty in deep learning.  In _international conference on machine learning_ , pages 1050–1059. PMLR, 2016.
  * Vaswani et al. [2017] Ashish Vaswani, Noam Shazeer, Niki Parmar, Jakob Uszkoreit, Llion Jones, Aidan N Gomez, Łukasz Kaiser, and Illia Polosukhin.  Attention is all you need.  _Advances in neural information processing systems_ , 30, 2017.
  * Brown et al. [2020] Tom Brown, Benjamin Mann, Nick Ryder, Melanie Subbiah, Jared D Kaplan, Prafulla Dhariwal, Arvind Neelakantan, Pranav Shyam, Girish Sastry, Amanda Askell, et al.  Language models are few-shot learners.  _Advances in neural information processing systems_ , 33:1877–1901, 2020.
  * He et al. [2022] Kaiming He, Xinlei Chen, Saining Xie, Yanghao Li, Piotr Dollár, and Ross Girshick.  Masked autoencoders are scalable vision learners.  In _Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition_ , pages 16000–16009, 2022.
  * Tancik et al. [2020] Matthew Tancik, Pratul Srinivasan, Ben Mildenhall, Sara Fridovich-Keil, Nithin Raghavan, Utkarsh Singhal, Ravi Ramamoorthi, Jonathan Barron, and Ren Ng.  Fourier features let networks learn high frequency functions in low dimensional domains.  _Advances in Neural Information Processing Systems_ , 33:7537–7547, 2020.
  * Radford et al. [2021] Alec Radford, Jong Wook Kim, Chris Hallacy, Aditya Ramesh, Gabriel Goh, Sandhini Agarwal, Girish Sastry, Amanda Askell, Pamela Mishkin, Jack Clark, et al.  Learning transferable visual models from natural language supervision.  In _International conference on machine learning_ , pages 8748–8763. PMLR, 2021.
  * Milletari et al. [2016] Fausto Milletari, Nassir Navab, and Seyed-Ahmad Ahmadi.  V-net: Fully convolutional neural networks for volumetric medical image segmentation.  In _2016 fourth international conference on 3D vision (3DV)_ , pages 565–571. Ieee, 2016.
  * Lin et al. [2017] Tsung-Yi Lin, Priya Goyal, Ross Girshick, Kaiming He, and Piotr Dollár.  Focal loss for dense object detection.  In _Proceedings of the IEEE international conference on computer vision_ , pages 2980–2988, 2017.
  * Kingma and Ba [2014] Diederik P Kingma and Jimmy Ba.  Adam: A method for stochastic optimization.  _arXiv preprint arXiv:1412.6980_ , 2014.
  * Loshchilov and Hutter [2016] Ilya Loshchilov and Frank Hutter.  Sgdr: Stochastic gradient descent with warm restarts.  _arXiv preprint arXiv:1608.03983_ , 2016.
  * Zhou et al. [2020] Yi Zhou, Boyang Wang, Lei Huang, Shanshan Cui, and Ling Shao.  A benchmark for studying diabetic retinopathy: segmentation, grading, and transferability.  _IEEE Transactions on Medical Imaging_ , 40(3):818–828, 2020.
  * Porwal et al. [2018] Prasanna Porwal, Samiksha Pachade, Ravi Kamble, Manesh Kokare, Girish Deshmukh, Vivek Sahasrabuddhe, and Fabrice Meriaudeau.  Indian diabetic retinopathy image dataset (idrid): a database for diabetic retinopathy screening research.  _Data_ , 3(3):25, 2018.
  * Simpson et al. [2019] Amber L Simpson, Michela Antonelli, Spyridon Bakas, Michel Bilello, Keyvan Farahani, Bram Van Ginneken, Annette Kopp-Schneider, Bennett A Landman, Geert Litjens, Bjoern Menze, et al.  A large annotated medical image dataset for the development and evaluation of segmentation algorithms.  _arXiv preprint arXiv:1902.09063_ , 2019.
  * Bilic et al. [2023] Patrick Bilic, Patrick Christ, Hongwei Bran Li, Eugene Vorontsov, Avi Ben-Cohen, Georgios Kaissis, Adi Szeskin, Colin Jacobs, Gabriel Efrain Humpire Mamani, Gabriel Chartrand, et al.  The liver tumor segmentation benchmark (lits).  _Medical Image Analysis_ , 84:102680, 2023.
  * Liu et al. [2020] Quande Liu, Qi Dou, Lequan Yu, and Pheng Ann Heng.  Ms-net: multi-site network for improving prostate segmentation with heterogeneous mri data.  _IEEE transactions on medical imaging_ , 39(9):2713–2724, 2020.

Supplementary Material for “On the Out of Distribution Robustness of
Foundation Models in Medical Image Segmentation"

##  Appendix A Background

###  A.1 SAM Architecture

SAM [6] was introduced by Meta AI to improve segmentation performance across a
wide range of images. SAM utilizes a transformer-based architecture [41],
which has shown impressive achievement in both natural language processing
[42] and computer vision [29]. In general, SAM is a Vision Transformer
(ViT)-based model, which consists of three components: an image encoder, a
prompt encoder, and a mask decoder. The image encoder is based on ViT [29],
which is pre-trained with masked auto-encoder (MAE) [43]. Two sets of prompts
can be considered, including sparse prompts (points, boxes, text) and dense
prompts (masks). To represent the first two prompts, points, and boxes, SAM
employs positional encoding [44] combined with learned embeddings. The text
prompt is encoded by the pre-trained text-encoder CLIP [45]. The mask prompt
has the same spatial resolution as the input image, and it is encoded using
convolutions and summed element-wise with the image embedding. The mask
decoder uses a lightweight network consisting of two transformer layers and a
dynamic mask prediction head with an Intersection-over-Union (IoU) score
regression head. SAM is trained in a supervised learning manner on a large-
scale SA-1B dataset with over 1 billion masks from 11 million natural images
using a linear combination of Dice loss [46] and Focal loss [47].

###  A.2 Implementation Details

We use MedSAM to fine-tune the SAM model on a specific medical downstream
task. The Adam optimizer is utilized for training networks with a combination
of Dice loss and Cross-entropy. The learning rates for problems are selected
from a set of $\\{1e\mathrm{-}4,3e\mathrm{-}4,5e\mathrm{-}4,1e\mathrm{-}5\\}$
depend on validation performance.

To gauge the extent of uncertainty arising from the foundational models, we
utilize a deep architecture named
$\bm{\delta}(\\{\mathbb{\hat{y}},\mathbb{x}\\};\gamma^{*})$, inspired by the
ResNet family [31] with 18 convolutional layers. In our experiment, we use
Eqs. (7),(8) for training with $\bm{\beta}_{t}$ fixed as
$\textbf{2}_{\textbf{m}}$. The initial input for the model $\bm{\delta}$ is a
joint feature vector constructed by concatenating the original image
$\mathbb{x}$ with the prediction mask $\hat{\mathbb{y}}$ after they have
passed through two different convolutional layers. Additionally, the output of
the model $\bm{\delta}$ is also combined with the aforementioned joint feature
vector. These combined features then undergo processing through distinct
convolutional operators, ultimately producing parameters representing
uncertainty distributions as described in Eq.(6). In the end, the uncertainty
model $\bm{\delta}$ is trained with Adam optimizer [48] with the learning rate
of $1e^{-4}$ in 50 epochs and cosine annealing strategy [49] for the learning
rate warm-up.

##  Appendix B Datasets

Table 5 overviews used dataset in our experiment along with the modality and
image types. Each task has at least two datasets, one for a source domain and
the remaining for a target domain.

Table 5: Overview datasets used in our experiment.

Task | Objects | Datasets | Modality | # Images
---|---|---|---|---
DR Lesion Segmentation | HE, SE, EX, MA | FGADR [50] | 2D Fundus | 1842
IDRiD [51] | 2D Fundus | 81
Liver Segmentation | Liver | FLARE [52] | 3D CT | 50
LiTS [53] | 3D CT | 130
Prostate Segmentation | Prostate | BMC [54] | 3D MRI | 30
BIDMC [54] | 3D MRI | 12
RUNMC [54] | 3D MRI | 30
HK [54] | 3D MRI | 12

##  Appendix C Experiment Details & Analyses

###  C.1 Further Analysis on Cross-domain Performance

The performance comparisons are presented in Tables 1, 2, and 3 for three
different medical modalities: Prostate, DR, and Liver, respectively. For each
dataset, we report the performance on both Unet-based models (including Unet
(R50), Unet++ (R50), Unet (Eff.Net), Unet++ (Eff.Net)), and SAM-based model
(i.e., MedSAM). Each model was initially trained and tested within the same
domain. It is then trained on source domain $S$ and tested on target domain
$T$ under the cross-domain $S\rightarrow T$ settings.

Table 1 demonstrates that MedSAM not only outperforms other models but also
exhibits superior generalization with minimal performance gaps between the
same domain and cross-domain.

Four datasets (BMC, BIDMC, RUNMC, and HK) on Prostate modality are well-
balanced in terms of the number of images. Thus, the domain shift has been
clearly observed, as shown in Table 1. On the other hand, the DR Lesion
datasets (FGADR and IDRiD) and Liver datasets (FLARE and LiTS) suffer from
heavy class imbalance, as indicated in Table 5. Notably, the Liver datasets
consist of 3D CT Scans processed slide-by-slide, with FLARE having 3,080
slides and LiTS having 25,660 slides. In Table 2, MedSAM achieves state-of-
the-art performance with small domain gaps. Due to the large imbalance between
LiTS (25,660 slides) and FLARE (3,080 slides), models trained on LiTS, whether
Unet-based or SAM-based, demonstrate better generalization compared to those
trained on FLARE, as demonstrated in Table 3, except MedSAM. While MedSAM
shows a great improvement on small datasets such as FLARE (i.e., FLARE
$\rightarrow$ LiTS), its generalization is dropped on large dataset LiTS.

###  C.2 Qualitative Results

(a)

(b)

Figure 5: A visual demonstration on MedSAM’s performance in the same- and
cross-domain. The two on the left depict the true mask and MedSAM’s prediction
mask in the same domain (FLARE). The other two display the true mask and
MedSAM’s prediction mask in an out-of-domain scenario (FLARE $\rightarrow$
LiTs). Best views in color with zoom. Figure 6: A qualitative comparison among
various foundation models. Respectively, from left to right, the top row
demonstrates the original image, the true mask for that image, MedSAM’s
prediction mask, the reconstructed mask from the uncertainty model, and the
error map between the true mask and MedSAM’s prediction mask. The bottom row
ranks the uncertain levels of different foundation models from the most
uncertainties (leftmost) to the fewest uncertainties (rightmost), whereas the
fewer, the better. Following this, the prediction mask derived from MedSAM has
the fewest uncertainties.

In Figure 5, we present a selection of predictions generated by the MedSAM
model in both same and cross-domain settings, showcasing the segmentation
results for lung, prostate, and soft exudates. Given the prompt-based nature
of MedSam, we simulate bounding boxes-based prompts as [9]. Our observations
indicate that MedSAM is capable of producing satisfactory masks for prostate
structures and DR lesions (shown in the second and bottom rows) despite the
challenging conditions of small object sizes and domain shifts. However, when
applied to the LiTS dataset, MedSAM tends to exhibit tendencies of over-
segmenting boundaries or missing structures at the bends of objects.

We offer illustrative examples of uncertainty estimations generated by our
algorithm, showcased in Figure 6, across various trained models. It can be
seen the MedSam model has the highest relevant uncertainty regions compared to
the Error map among models.