112 722022

The OGLE Collection of Variable Stars.
Over 2600 $\delta$ Scuti Stars in the Small Magellanic Cloud***Based on
observations obtained with the 1.3-m Warsaw telescope at the Las Campanas
Observatory of the Carnegie Institution for Science. I. S o s z y ń s k i1, A.
U d a l s k i1, J. S k o w r o n1, P. P i e t r u k o w i c z1,
M. K. S z y m a ń s k i1, R. P o l e s k i1, D. M. S k o w r o n1, S. K o z ł
o w s k i1,
P. M r ó z1, P. I w a n e k1, M. W r o n a1, K. U l a c z y k2,1, K. R y b i c
k i3.1,
and M. G r o m a d z k i1 1Astronomical Observatory, University of Warsaw, Al.
Ujazdowskie 4, 00-478 Warszawa, Poland
2Department of Physics, University of Warwick, Gibbet Hill Road, Coventry, CV4
7AL, UK
3Department of Particle Physics and Astrophysics, Weizmann Institute of
Science, Rehovot 76100, Israel

We present the first-ever collection of $\delta$ Scuti stars found over the
entire area of the Small Magellanic Cloud (SMC). The sample consists of 2810
variables of which over 2600 objects belong to the SMC while the remaining
stars are most likely members of the Milky Way’s halo. The sample has been
divided into 2733 singlemode and 77 multimode pulsators. We provide
observational parameters (pulsation periods, mean magnitudes, amplitudes,
Fourier coefficients) of all $\delta$ Sct stars and the long-term I\- and
V-band time-series photometric measurements collected during the fourth phase
of the Optical Gravitational Lensing Experiment (OGLE-IV).Stars: variables:
delta Scuti – Stars: oscillations – Magellanic Clouds – Catalogs

##  1 Introduction

$\delta$ Scuti variables are intermediate-mass stars pulsating in low-order
radial or non-radial modes driven by the $\kappa$ mechanism operating in the
He II ionization zone (Chevalier 1971). This class includes young and
intermediate-age stars on the pre-main sequence, main sequence, and post-main
sequence, as well as stars belonging to the old population that are probably
merged binaries (Breger 2000). The latter group is referred to as SX Phoenicis
stars. The pulsation periods of $\delta$ Sct variables are shorter than 0.3 d,
while the amplitudes reach 1 mag in the V band, although most stars of this
type exhibit much smaller amplitudes, down to the sub-millimagnitude level
(Balona and Dziembowski 2011).

$\delta$ Sct stars, like other classical pulsators, follow period–luminosity
(PL) and period–luminosity–color (PLC) relations. The Large and Small
Magellanic Cloud (LMC and SMC) play a crucial role in studying the PL
relations obeyed by various types of pulsating stars (Leavitt and Pickering
1912, Glass and Lloyd Evans 1981, Madore 1982, Udalski et al.1999, Muraveva et
al.2015). The PL relations for $\delta$ Sct stars were also investigated based
on variables detected in the Magellanic Clouds (Poleski et al.2010, McNamara
2011, Martínez-Vázquez 2022), but these studies were limited by a small number
of known $\delta$ Sct stars in the Clouds, especially in the SMC.

The first $\delta$ Sct variables in the SMC were discovered as a by-product of
a search for RR Lyr stars in the OGLE-II photometric database. Soszyński et
al.(2002) published a list of 19 short-period variables (with periods in the
range 0.16–0.57 d) suggesting that most of them are $\delta$ Sct stars. In the
following years, about half of these stars were re-classified as RR Lyr stars
or Cepheids (see Section 4 for the definition of the boundary between
classical Cepheids and $\delta$ Sct stars), but eight of them turned out to be
bona fide $\delta$ Sct variables and are included in the present work. One
more $\delta$ Sct star from the OGLE project was reported by Pietrukowicz
(2018). The second, and so far the last, sample of $\delta$ Sct stars in the
SMC was published by Martínez-Vázquez et al.(2021). They used deep photometry
of the SMC globular cluster NGC 419 obtained by the Gemini South telescope to
detect 54 $\delta$ Sct stars of which six objects are probable cluster members
and 48 are field variables. The Gaia DR3 catalog of main-sequence oscillators
(Gaia Collaboration et al.2022) contains over 50 $\delta$ Sct stars in the
region of the sky occupied by the SMC, but all these pulsators belong to the
Milky Way halo and are located in the foreground of the SMC.

Thus, only 63 $\delta$ Sct stars in the SMC have been known until now. With
this paper, we release the first catalog of $\delta$ Sct variables found over
the entire area of the SMC. Our sample consists of 2810 pulsators of which
over 2600 objects are likely members of the SMC. This is part of the OGLE
Collection of Variable Stars (OCVS), currently containing over a million
manually classified variable stars in the Milky Way and Magellanic Clouds. In
the next paper in this series, we will present the OGLE collection of over
14,000 $\delta$ Sct pulsators in the LMC.

This work is structured as follows. In Section 2, we present a summary of the
OGLE survey of the Magellanic Clouds. Section 3 is devoted to a brief review
of the variable star selection and classification procedures. In Section 4, we
discuss the criteria that can be used to distinguish between $\delta$ Sct
stars and classical Cepheids. The OGLE collection of $\delta$ Sct variables in
the SMC is described in Section 5. The completeness of our catalog is
discussed in Section 6. We close the paper with a summary in Section 7.

##  2 OGLE Observations of the Magellanic System

The OGLE survey uses a dedicated 1.3-m Warsaw Telescope equipped with a large
format mosaic CCD camera with a field of view of 1.4 square degrees and a
pixel scale of 0.26 arcsec. The telescope is located at Las Campanas
Observatory, Chile, operated by the Carnegie Institution for Science. Our
collection of $\delta$ Sct stars is based on photometric data from the fourth
phase of the OGLE project (OGLE-IV, Udalski et al.2015), which has been
operating since 2010 until today. In this work, we use the observations
obtained before March 2020, when the Warsaw Telescope was temporarily closed
due to the COVID-19 pandemic.

The OGLE observations are carried out in the I and V photometric bands,
closely resembling those from the Cousins-Johnson standard systems. From
several dozen to over 1000 observations per star (typically 600-700) have been
acquired in the I-band and from several to over 300 (typically 40-60) data
points have been gathered in the V-band light curves. The uniform exposure
time of 150 s resulted in the saturation limit of the OGLE photometry at the
level of about $I\approx 13$ mag and the sensitivity limit at about $I\approx
21.5$ mag.

The total sky area monitored by the OGLE survey in the region of the
Magellanic System is 765 square degrees. The OGLE footprint covers both
galaxies together with the Magellanic Bridge and vast regions on their
periphery. For the purposes of this work, we adopted the celestial meridian of
28 as an arbitrary on-sky boundary between the LMC and SMC regions. The same
value was selected to separate the LMC and SMC RR Lyr stars in the OCVS
(Soszyński et al.2016). In the SMC area of about 280 square degrees, OGLE
regularly observes about 14 million stars belonging to both the SMC and the
halo of the Milky Way.

##  3 Selection and Classification of $\delta$ Sct Stars

We began our search for $\delta$ Sct variables in the SMC with a massive
period analysis for all 14 million I-band light curves stored in the OGLE-IV
databases. We used the Fnpeaks
code†††http://helas.astro.uni.wroc.pl/deliverables.php?lang=en&active=fnpeaks
to span the frequency domain from 0 to 100 cycles per day with a step of
$5\cdot 10^{-5}$ cycles per day. For each light curve, the period with the
highest signal-to-noise ratio was considered to be dominant. Then, this
dominant frequency and its harmonics were subtracted from the light curve and
the period search was repeated on the residual data.

OGLE-IV I-band (red) and V-band (blue) light curves of nine example $\delta$
Sct stars in the SMC.

In the next step of the procedure, we used both I\- and V-band OGLE-IV
photometric data to select and classify $\delta$ Sct stars in the SMC. The
much smaller number of the OGLE observations in the V-band was compensated by
the lower noise of the V-band light curves compared to the I-band ones. Fig. 1
shows the I\- and V-band light curves of nine example $\delta$ Sct variables
arranged in descending pulsation periods. Note that the shortest-period (and
therefore the faintest) stars have a significant scatter of the data points in
the I-band, while the V-band light curves of these objects are much less
dispersed. However, every $\delta$ Sct variable identified based on the V-band
data was later confirmed with the I-band observations, although in some cases
the I-band light curves are very noisy.

Our variability classification was primarily based on the shapes of the I\-
and V-band light curves. For single-periodic variables, we filtered out stars
with sinusoidal light curves and obvious eclipsing binaries. For multi-
periodic objects, we took into account the characteristic period ratios of
multimode pulsators.

$(V-I)_{0}$ vs. $I_{0}$ color–magnitude diagram for $\delta$ Sct stars in the
SMC (blue points). The background gray points show stars from the subfield
SMC719.11.

In the final stage of our classification procedure, we examined the positions
of the candidate pulsation stars in the color–magnitude diagram depicted in
Fig. 2, where the I-band magnitudes and $V-I$ color index were corrected for
the interstellar extinction using the most accurate reddening maps of the
Magellanic Clouds recently published by Skowron et al.(2021). Most of the
stars with the dereddened color index $(V-I)_{0}<0.2$ mag or $(V-I)_{0}>0.7$
mag, outside the $\delta$ Sct instability strip, were removed from the list.
However, as can be seen in Fig. 2, we decided to keep some too-blue or too-red
stars on the list, because their light curve morphology suggested that these
objects could be genuine $\delta$ Sct variables blended with nearby unresolved
stars. Nonetheless, the objects with atypical colors should be treated with
caution as uncertain candidates for $\delta$ Sct stars.

Finally, our collection of $\delta$ Sct stars detected toward the SMC consists
of 2810 objects. In addition, the OCVS has been enriched with nine new
classical Cepheids and 123 RR Lyr stars identified as a by-product of the
present search for $\delta$ Sct stars or a result of cross-matching the OGLE
database with the Gaia DR3 catalog of Cepheids (Ripepi et al.2022) and RR Lyr
stars (Clementini et al.2022). A more detailed comparison of the Gaia DR3 and
OGLE catalogs in the area of the Magellanic System will be discussed in the
forthcoming paper presenting the OGLE collection of $\delta$ Sct stars in the
LMC (Soszyński et al.in prep.).

Period–luminosity (upper panel) and period–Wesenheit index (lower panel)
diagrams for classical pulsators in the SMC. Green points show classical
Cepheids, violet points – anomalous Cepheids, red points – type II Cepheids,
orange points – RR Lyr stars, and blue points – $\delta$ Sct stars. Lighter
colors indicate stars oscillating in higher modes. Wesenheit index is a
reddening-free function defined as $W_{I}=I-1.55(V-I)$ (Madore 1982).

##  4 Boundary Between $\delta$ Sct Stars and Classical Cepheids

Population I $\delta$ Sct stars and classical Cepheids occupy common sequences
in the PL (Fig. 3), color–magnitude, period–radius (Fernie 1992), Petersen
(Poleski et al.2010) and other diagrams. In fact, it is a matter of convention
what value of the pulsation period is adopted to separate both types of
variable stars. Various authors proposed different maximum periods of $\delta$
Sct stars. For example, in the General Catalogue of Variable Stars (Samus’ et
al.2017) it is 0.2 d, Breger (2000) defined $\delta$ Sct variables as
pulsators with periods shorter than 0.25 d, Rodríguez et al.(2000) adopted 0.3
d, Handler (2009) – 0.33 d (8 hr), while Catelan and Smith (2015) – 0.42 d.

One might ask if it is possible to establish a “natural” boundary period
separating $\delta$ Sct stars from classical Cepheids. Both types of variables
may pulsate, among others, in the fundamental mode (F), first-overtone (1O),
second-overtone (2O), or in the mix of these radial modes. All but one of
these combinations form continuity in the vicinity of the Cepheid–$\delta$ Sct
border. The exception is a class of single-mode F pulsators that show a
natural gap between $\delta$ Sct stars and classical Cepheids. OGLE-SMC-
CEP-1899 ($P_{\mathrm{F}}=0.842$ d; Soszyński et al.2010) is a classical
Cepheids oscillating solely in the F mode with the shortest known period,
while OGLE-LMC-DSCT-0617 ($P_{\mathrm{F}}=0.299$ d; Poleski et al.2010) is the
longest-period single-mode $\delta$ Sct star. Currently, no known Population I
single-mode F-mode star from the Cepheid instability strip pulsates with a
period in the range of 0.3–0.8 d‡‡‡Of course, the Population II pulsators,
like RR Lyr stars or anomalous Cepheids, may have periods in this range, but
these stars can be isolated using other properties, in the PL diagram, they
lie below the classical Cepheid–$\delta$ Sct ridge.. For this reason, all
Population I radially oscillating stars (both single- and multimode pulsators)
with an F-mode period below 0.3 d are classified in our collection as $\delta$
Sct variables. Consequently, the 1O boundary period between $\delta$ Sct stars
and Cepheids was fixed at $P_{\mathrm{1O}}=0.23$ d (corresponding to the
period ratio $P_{\mathrm{1O}}/P_{\mathrm{F}}$ of about 0.77).

With the strict definition of different types of pulsating stars, we could
extend the OCVS by nine short-period classical Cepheids in the SMC. One F/1O
double-mode and eight 1O single-mode pulsators have 1O periods in the range
0.23–0.9 d. The OCVS (Soszyński et al.2019) currently contains 4954 carefully
selected classical Cepheids in the SMC.

##  5 The OGLE Collection of $\delta$ Sct Stars in the SMC

Our collection of 2810 $\delta$ Sct variables in the SMC is available at the
OGLE Internet Archive:

https://ogle.astrouw.edu.pl $\rightarrow$ OGLE Collection of Variable Stars
https://www.astrouw.edu.pl/ogle/ogle4/OCVS/smc/dsct/

The sample was divided into 2733 singlemode and 77 multimode pulsators. The
latter group contains objects with well-defined secondary or tertiary periods
that may be associated with additional radial or non-radial pulsation modes.
At least 160 stars in our sample are members of our Galaxy – these are likely
SX Phe variables in the halo of the Milky Way located in front of the SMC.
Fig. 4 shows on-sky maps of $\delta$ Sct stars from our collection. The left
panel presents the spatial distribution of probable members of the SMC, while
the right panel shows the positions of the putative Milky Way SX Phe stars.
The line separating these two populations was arbitrarily set at 1.5 mag above
the mean PL relation for the SMC $\delta$ Sct variables (Fig. 3). More
sophisticated methods would probably divide the Galactic and SMC populations
more efficiently, however, one should remember that this separation cannot be
done unequivocally because the Magellanic Clouds are submerged in the Milky
Way halo – old stellar populations in both galaxies smoothly transition into
each other.

The full list of $\delta$ Sct stars with their equatorial coordinates
(J2000.0), classification (singlemode, multimode), internal designations in
the OGLE-IV, OGLE-III, and OGLE-II databases (if available), and the cross-
matches with the International Variable Star Index (VSX; Watson et al.2006)
are provided in the ident.dat file. In total, we identified only 11 common
stars between our collection and the VSX catalog – all of them are
unquestionable members of the Milky Way.

The file dsct.dat contains observation parameters of the stars: their
intensity-averaged mean magnitudes in the I\- and V-bands, up to three
pulsation periods derived with the Tatry code (Schwarzenberg-Czerny 1996),
epochs of the maximum light, I-band peak-to-peak amplitudes, and Fourier
coefficients. The periods are very precisely determined thanks to the long
timespan of the OGLE-IV light curves, however, it cannot be ruled out that in
the case of some the most noisy light curves our approach produced daily
aliases of the true periods.

The OGLE-IV time-series photometry in the I\- and V-bands are stored in the
directory phot/. Obvious outlying points (deviating by more than $\pm 4$ sigma
from the fitted model) were removed from the light curves, but it was verified
beforehand whether these points are due to, for example, additional eclipses.
We did not find any $\delta$ Sct stars with eclipses in the SMC. Finding
charts can be found in the directory fcharts/. These are
$60^{\prime\prime}\times 60^{\prime\prime}$ subframes of the I-band reference
images, oriented with North at the top and East to the left.

On-sky distributions of $\delta$ Sct stars toward the SMC. Left panel shows
positions of over 2600 probable members of the SMC. Right panel shows $\delta$
Sct variables that likely belong to the Milky Way halo. The gray area shows
the OGLE footprint in the Magellanic System region.

##  6 Completeness of the Catalog

The completeness of our collection is strongly limited by the brightness,
amplitudes, and light curve shapes of $\delta$ Sct stars. Figs. 2 and 3 show
that the number of variables in our sample drops sharply for mean magnitudes
$I>21.2$ mag due to the sensitivity limit of the OGLE photometry.

We cross-matched our collection with the list of 54 $\delta$ Sct variables
discovered with the Gemini South telescope in the field surrounding the SMC
globular cluster NGC 419 (Martínez-Vázquez et al.2021). We found only one
common object in both catalogs – V46 = OGLE-SMC-DSCT-2067 – which is the
brightest $\delta$ Sct star detected by Martínez-Vázquez et al.(2021). The
remaining $\delta$ Sct variables identified with the 8.1-meter Gemini
telescope are below the detection limit of the 1.3-meter Warsaw telescope used
by the OGLE survey.

Luminosity–amplitude diagram for $\delta$ Sct stars in the SMC.

The vast majority of $\delta$ Sct variables are small-amplitude pulsators
(Breger 2000, Balona and Dziembowski 2011), so our collection must be the tip
of the iceberg because we had the opportunity to discover only the high-
amplitude $\delta$ Sct stars in the SMC. Fig. 5 shows the I-band peak-to-peak
amplitudes plotted against the mean I-band magnitudes of our sample. It is
obvious that the amplitude detection limits in the OGLE data are strongly
correlated with the brightness of the pulsating candidates: for stars with the
luminosities of about $I=19$ mag, the smallest detectable amplitudes are of
the order of 0.04 mag, for $I=20$ mag stars, the amplitude detection limit
rises to 0.1 mag, and for the faintest stars in our collection ($I=21$ mag),
the light curve amplitude should be larger than about 0.2 mag to be recognized
as a $\delta$ Sct variable. On the other hand, our sample of the Galactic
$\delta$ Sct stars ($I<18.5$ mag) should be much more complete because we
could detect variables with amplitudes as small as 0.01 mag in the I-band.

We used the $\delta$ Sct stars with double entries in the OGLE-IV database to
estimate the completeness of our collection within the brightness and
amplitude detection limits. These objects are located in the overlapping parts
of the adjacent OGLE fields, so we had the opportunity to identify them twice
during the selection and classification process. We found that 311 out of 2810
$\delta$ Sct stars included in the final version of our collection have their
duplicates in the OGLE-IV database, so we had a chance to detect 622
counterparts. In fact, 449 objects from this list were independently
classified as $\delta$ Sct variables which correspond to the catalog
completeness slightly above 60%. However, if we consider only stars brighter
than $I=19$ mag, the formal completeness of our collection increases to about
85%. Nonetheless, one should remember that the bulk of $\delta$ Sct stars in
the SMC have luminosities and amplitudes below the detection limits of the
OGLE project.

##  7 Summary

We presented the first-ever catalog of $\delta$ Sct stars found in the entire
area of the SMC. The number of known $\delta$ Sct variables in this galaxy has
increased from about 60 to over 2600. Additionally, our collection contains at
least 160 Galactic SX Phe stars located in the foreground of the SMC. The most
obvious application of our collection includes the examination of the PL and
PLC relations obeyed by $\delta$ Sct stars in the metal-poor environment of
the SMC compared to the more metal-rich variables in the LMC and Milky Way.
The three-dimensional distribution of $\delta$ Sct variables can be analyzed
to better understand the structure of the SMC. Moreover, the long-term
photometric time series produced by the OGLE project can be used to study the
stability of stellar pulsations over long time scales.

This work demonstrates the great versatility of OGLE photometric data. The
OCVS contains variable stars with periods ranging from minutes to years,
luminosities from 12 mag to over 22 mag in the V-band, and amplitudes from
millimagnitudes to several magnitudes. We expect the next breakthrough in the
field of variable stars in the Magellanic Clouds to come after the launch of
the LSST project on the Rubin Observatory (Hambleton et al.2022).

This work has been supported by the National Science Centre, Poland, grant no.
2022/45/B/ST9/00243. MG is supported by the EU Horizon 2020 research and
innovation programme under grant agreement no. 101004719. This research has
made use of the International Variable Star Index (VSX) database, operated at
AAVSO, Cambridge, Massachusetts, USA.

## References

  * Balona, L.A., and Dziembowski, W.A.2011417591 2011MNRAS.417..591B Breger M.2000 in Breger M., Montgomery M., eds, ASP Conf. Ser. Vol. 210, Delta Scuti and Related Stars. Astron. Soc. Pac., San Francisco, p. 3 Catelan, M., and Smith, H.A.2015 Pulsating Stars (New York: Wiley) Chevalier, C.1971Å1424 Clementini, G., Ripepi, V., Garofalo, A., et al.2022 arXiv:2206.06278 Fernie, J.D.19921031647 Gaia Collaboration, De Ridder, J., Ripepi, V., et al.2022 arXiv e-prints, arXiv:2206.06075 Glass, I.S., and Lloyd Evans, T.1981Nature291303 Hambleton, K.M., Bianco, F.B., Street, R., et al..2022 arXiv:2208.04499 Handler, G.2009 in AIP Conf. Ser. 1170, Stellar Pulsation: Challenges for Theory and Observation, ed. J. A. Guzik & P. A. Bradley (Melville, NY: AIP), 403 Leavitt, H.S., and Pickering, E.C.1912 Harvard Coll. Obs. Circ.1731 Madore, B.F.1982253575 Martínez-Vázquez, C.E., Salinas, R., and Vivas, A.K.2021161120 Martínez-Vázquez, C.E., Salinas, R., Vivas, A.K., and Catelan, M.2022940L25 McNamara, D.H.2011142110 Muraveva, T., Palmer, M., Clementini, G., et al.2015807127 Pietrukowicz, P.2018 in The RR Lyrae 2017 Conference. Revival of the Classical Pulsators: from Galactic Structure to Stellar Interior Diagnostics., Vol. 6., ed. R. Smolec, K. Kinemuchi, & R.I. Anderson (Poland: Proc. Polish Astronomical Soc.), 258 Poleski, R., Soszyński, I., Udalski, A., et al.2010601 Ripepi, V., Clementini, G., Molinaro, R., et al.2022 arXiv:2206.06212 Rodríguez, E., López-González, M.J., and López de Coca, P.2000144469 Samus’, N.N., Kazarovets, E.V., Durlevich, O.V., Kireeva, N.N., and Pastukhova, E.N.2017Astronomy Reports6180 Schwarzenberg-Czerny, A.1996460L107 Skowron, D.M., Skowron, J., Udalski, A., et al.202125223 Soszyński, I., Udalski, A., Szymański, M., et al.200252369 Soszyński, I., Poleski, R., Udalski, A., et al.20106017 Soszyński, I., Udalski, A., Szymański, M.K., et al.201666131 Soszyński, I., Udalski, A., Szymański, M.K., et al.20196987 Udalski, A., Szymański, M., Kubiak, M., Pietrzyński, G., Soszyński, I., Woźniak, P., and Żebruń, K.199949201 Udalski, A., Szymański, M.K., and Szymański, G.2015651 Watson, C.L., Henden, A.A., and Price, A.2006Soc. Astron. Sci. Annu. Symp.2547


# Dynamic X-Ray Vision in Mixed Reality

Hung-Jui Guo  0000-0003-2233-846X<EMAIL_ADDRESS>The University of
Texas at Dallas800 W Campbell RdRichardsonTexasUSA75080 ,  Jonathan Z. Bakdash
0000-0002-1409-4779<EMAIL_ADDRESS>,  Laura R. Marusich
0000-0002-3524-6110<EMAIL_ADDRESS>U.S. DEVCOM Army Research
Laboratory800 W Campbell RdTexasUSA75080 and  Balakrishnan Prabhakaran
0000-0003-0385-8662<EMAIL_ADDRESS>The University of Texas at
Dallas800 W Campbell RdRichardsonTexasUSA75080

(2018)

###### Abstract.

X-ray vision, a technique that allows users to see through walls and other
obstacles, is a popular technique for Augmented Reality (AR) and Mixed Reality
(MR). In this paper, we demonstrate a dynamic X-ray vision window that is
rendered in real-time based on the user’s current position and changes with
movement in the physical environment. Moreover, the location and transparency
of the window are also dynamically rendered based on the user’s eye gaze. We
build this X-ray vision window for a current state-of-the-art MR Head-Mounted
Device (HMD) – HoloLens 2 (Microsoft, 2015) by integrating several different
features: scene understanding, eye tracking, and clipping primitive.

X-ray vision, HoloLens, Mixed Reality

††copyright: acmcopyright††journalyear: 2018††doi: XXXXXXX.XXXXXXX††journal:
POMACS††journalvolume: 37††journalnumber: 4††article: 111††publicationmonth:
8††ccs: Human-centered computing Mixed / augmented reality Figure 1. (a) Side
view of the created collaborative scene understanding generated augmented
reality environment based on an indoor room, (b) Dynamic X-ray vision window
used by a user to navigate the indoor environment

##  1\. Introduction

Generally, in a real environment it is almost impossible for a user to obtain
information occluded by environmental objects, such as a chair behind a wall,
when the user has no prior understanding of the environment. Even when
advanced information about the structure of the environment is available and
presented with a tool such as a minimap, physical occlusions still challenge
users’ ability to obtain direct visual information about objects and the
spatial layout of the beyond-line-of-sight environment.

A virtual X-ray vision window allows a user to directly view a target object
through a physical occlusion at a fixed point, enabling quick and accurate
judgments about the occluded target object while not sacrificing knowledge
about the occluding surrounding environment. By providing information
otherwise unavailable in the real-world environment, X-ray vision has
potential law enforcement and military applications (e.g., knowledge of the
locations of beyond-line-of-sight threats and civilians prior to entering a
room) (Phillips et al., 2021). Examples of previous implementations of X-ray
vision systems include robot with a stereo depth camera (Phillips et al.,
2021) and image-based rendering (Avery et al., 2009). In both of these
implementations, the X-ray window is a flat image aligned with a physical
surface.

In contrast to previous approaches, virtual content in our X-ray window is
rendered dynamically using scene understanding. That is, the virtual window
updates in real-time based upon the user’s position, movement, and eye-gaze in
the physical environment. Consequently, our X-ray vision system mimics a
physical window. It provides non-pictorial depth cues such as motion parallax
(changes in the perspectives of stationary objects with self-motion) and
binocular disparity (slightly different visual images in each eye due to their
horizontal separation) typically available in real-world environments (Cutting
and Vishton, 1995). This allows users to move and still naturally see through
physical occlusions (such as real walls, created by the scene understanding
observer) as if looking through a window in the real-world.

##  2\. Dynamic X-Ray Vision Window

The dynamic X-ray vision window is built in the HoloLens 2 device (Microsoft,
2015), a Head-Mounted Device (HMD) that includes the Mixed Reality (MR)
environment. This system incorporated the virtual environment created by
HoloLens 2 Scene Understanding feature pre-built in the Mixed Reality ToolKit
(MRTK) (https://docs.microsoft.com/en-us/windows/mixed-reality/mrtk-
unity/mrtk2/). We use the long-throw depth camera to observe the surrounding
real-world environment; we call this observed environment a collaborative
scene understanding generated augmented reality environment, see Figure 1 (a).

In addition, to further help users see through the occluding real-world
surface from a distance, the X-ray vision window uses the clipping primitive
feature created by the MRTK. This makes the contact area of the touched scene
objects partially transparent, as shown in Figure 1 (b). Moreover, we also use
the gaze-tracking feature in HoloLens 2 to obtain real-time eye
direction/movements and its intersection with scene objects to determine the
location of the X-ray vision window.

Using a dynamic window for X-ray vision based on user position/movement and
eye-gaze is an advancement over image based X-ray vision. This X-ray vision
has the potential to provide benefits over prior implementations. However, the
usability and human performance with our system needs to be evaluated. For
example, it is possible users may find that interacting with the X-ray vision
window is unfamiliar, uncomfortable, or even distracting. Using previous
behavioral research on X-ray vision in AR/MR as a reference (see (Livingston
et al., 2013) for a summary), we propose a user evaluation of the dynamic
X-ray vision window demonstrated here.

###  2.1. Proposed User Study Design

Among previous evaluations of X-ray vision in AR/MR, the most common method is
to assess how an X-ray vision window may affect participants’ depth perception
of physical/virtual objects ((Phillips et al., 2021; Zollmann et al., 2014)).
Other methods include testing whether the participants can understand
occlusion layers (Kitajima et al., 2015), or asking the participants to
acquire target objects under different X-ray vision settings (Sandor et al.,
2010).

According to (Livingston et al., 2013), most previous X-ray vision studies are
conducted while participants are stationary. In our proposed study, We will
incorporate participants’ physical movement, allowing them to explore the
environment, while dynamically updating virtual information in real-time. This
feature allows us to explore the benefit of our X-ray vision window system and
design a more realistic user study with visual depth cues that are not present
when stationary (e.g., motion parallax).

Therefore, we plan to build a game-like user study to explore how a dynamic
X-ray vision window (as compared to other forms of X-ray vision or other types
of advanced spatial knowledge) may benefit human performance, measured by
physical movement direction and/or reaction time. The study will include
target objects placed behind occluders in a physical environment (such as an
indoor room), and will require participants to move in the environment and
acquire target objects with and without the X-ray vision window.

##  3\. Conclusion and Future Work

In this paper, we present a new dynamic X-ray vision window built in HoloLens
2, which can render beyond-line-of-sight content in real-time based on user
position and movement with eye-gaze direction to potentially improve
usability. In the future, we will evaluate and improve this proposed system
according to the user study design described in the previous section and make
this system more realistic and close to the physical world by attaching
textures to virtual surfaces.

## Acknowledgments

This research was sponsored by the DEVCOM U.S. Army Research Laboratory under
Cooperative Agreement Number W911NF-21-2-0145 to B.P.
The views and conclusions contained in this document are those of the authors
and should not be interpreted as representing the official policies, either
expressed or implied, of the DEVCOM Army Research Laboratory or the U.S.
Government. The U.S. Government is authorized to reproduce and distribute
reprints for Government purposes notwithstanding any copyright notation.

## References

  * (1)
  * Avery et al. (2009) Benjamin Avery, Christian Sandor, and Bruce H. Thomas. 2009\.  Improving Spatial Perception for Augmented Reality X-Ray Vision. In _2009 IEEE Virtual Reality Conference_. 79–82.  https://doi.org/10.1109/VR.2009.4811002
  * Cutting and Vishton (1995) James E Cutting and Peter M Vishton. 1995.  Perceiving layout and knowing distances: The integration, relative potency, and contextual use of different information about depth.  In _Perception of space and motion_. Elsevier, 69–117.
  * Kitajima et al. (2015) Yuki Kitajima, Sei Ikeda, and Kosuke Sato. 2015.  [POSTER] Vergence-Based AR X-ray Vision. In _2015 IEEE International Symposium on Mixed and Augmented Reality_. 188–189.  https://doi.org/10.1109/ISMAR.2015.58
  * Livingston et al. (2013) Mark A. Livingston, Arindam Dey, Christian Sandor, and Bruce H. Thomas. 2013.  _Pursuit of “X-Ray Vision” for Augmented Reality_.  Springer New York, New York, NY, 67–107.  https://doi.org/10.1007/978-1-4614-4205-9_4
  * Microsoft (2015) Microsoft. 2015\.  HoloLens.  https://www.microsoft.com/en-us/hololens.
  * Phillips et al. (2021) Nate Phillips, Farzana Alam Khan, Brady Kruse, Cindy Bethel, and J. Edward Swan. 2021\.  An X-Ray Vision System for Situation Awareness in Action Space. In _2021 IEEE Conference on Virtual Reality and 3D User Interfaces Abstracts and Workshops (VRW)_. 593–594.  https://doi.org/10.1109/VRW52623.2021.00179
  * Sandor et al. (2010) Christian Sandor, Andrew Cunningham, Arindam Dey, and Ville-Veikko Mattila. 2010.  An Augmented Reality X-Ray system based on visual saliency. In _2010 IEEE International Symposium on Mixed and Augmented Reality_. 27–36.  https://doi.org/10.1109/ISMAR.2010.5643547
  * Zollmann et al. (2014) Stefanie Zollmann, Raphael Grasset, Gerhard Reitmayr, and Tobias Langlotz. 2014.  Image-Based X-Ray Visualization Techniques for Spatial Understanding in Outdoor Augmented Reality. In _Proceedings of the 26th Australian Computer-Human Interaction Conference on Designing Futures: The Future of Design_ (Sydney, New South Wales, Australia) _(OzCHI ’14)_. Association for Computing Machinery, New York, NY, USA, 194–203.  https://doi.org/10.1145/2686612.2686642


# TokenSHAP: Interpreting Large Language Models with Monte Carlo Shapley Value
Estimation

Miriam Horovicz
Tel Aviv, Israel
NI
<EMAIL_ADDRESS>Roni Goldshmidt
Tel Aviv, Israel
Nexar
<EMAIL_ADDRESS>
###### Abstract

As large language models (LLMs) become increasingly prevalent in critical
applications, the need for interpretable AI has grown. We introduce TokenSHAP,
a novel method for interpreting LLMs by attributing importance to individual
tokens or substrings within input prompts. This approach adapts Shapley values
from cooperative game theory to natural language processing, offering a
rigorous framework for understanding how different parts of an input
contribute to a model’s response. TokenSHAP leverages Monte Carlo sampling for
computational efficiency, providing interpretable, quantitative measures of
token importance. We demonstrate its efficacy across diverse prompts and LLM
architectures, showing consistent improvements over existing baselines in
alignment with human judgments, faithfulness to model behavior, and
consistency.

Key contributions include:

  * •

A theoretical framework extending Shapley values to variable-length text LLM
inputs.

  * •

An efficient Monte Carlo sampling approach tailored for language models.

  * •

Comprehensive evaluation across various prompts and model types.

  * •

Capability to effortlessly visualize the insights.

Our method’s ability to capture nuanced interactions between tokens provides
valuable insights into LLM behavior, enhancing model transparency, improving
prompt engineering, and aiding in the development of more reliable AI systems.
TokenSHAP represents a significant step towards the necessary interpretability
for responsible AI deployment, contributing to the broader goal of creating
more transparent, accountable, and trustworthy AI systems.

Figure 1: Flowchart of the TokenSHAP algorithm illustrating the process of
Shapley value estimation for token importance in large language models by
accepting parts of the text to the players and a cosine similarity measure to
the base prompt as a gain.

##  1 Introduction

Large language models (LLMs) have greatly advanced natural language
processing, delivering near or at human-level performance on many tasks.
However, their ”black box” nature poses interpretability challenges, crucial
for applications in fields like healthcare and legal analysis, where
understanding AI decision-making is vital.

This paper introduces TokenSHAP, a method enhancing LLM interpretability by
adapting Shapley values from game theory. TokenSHAP treats input tokens as
players, assessing their contribution to model outputs. This allows for a
deeper understanding of how LLMs process information, crucial for improving
model transparency and reliability.

We propose a Monte Carlo sampling method for practical Shapley value
estimation, accommodating the variable lengths and contextual nature of
language inputs. Our evaluations across different prompts and models confirm
TokenSHAP’s versatility and effectiveness in revealing LLM decision-making
processes. This breakthrough aids the development of more accountable AI
systems, ensuring their responsible use as they become more integrated into
critical applications.

##  2 Related Work

###  2.1 Interpretability in Machine Learning

Interpretability in machine learning has gained significant attention as
models become increasingly complex. Methods for explaining AI systems can be
broadly categorized into two approaches: black box methods and white box
methods [1].

Black box methods, such as LIME [2] and SHAP [3], have emerged as popular
approaches for explaining predictions across various ML models without
requiring access to the model’s internal architecture or parameters. LIME
provides local approximations of model behavior by perturbing input data,
while SHAP unifies several feature attribution methods under the Shapley value
framework. These methods are particularly valuable when working with
proprietary or complex models where internal access is limited or impractical
[4].

White box methods, on the other hand, require knowledge of and access to the
model’s internal structure. These include techniques like gradient-based
saliency maps [5] and layer-wise relevance propagation (LRP) [6]. While these
methods can provide more detailed insights into the model’s decision-making
process, they are limited to scenarios where the model architecture is fully
accessible and understood [7].

Recent advancements include counterfactual explanations [8], which explore how
altering inputs changes model predictions. While these methods offer valuable
insights for tabular and image data, they face challenges when applied to the
sequential and contextual nature of language, highlighting the need for
specialized NLP interpretability techniques [9].

###  2.2 Interpretability in Natural Language Processing

In the NLP domain, attention visualization techniques [10] have gained
popularity, offering insights into which parts of the input a model focuses
on. However, these visualizations often lack quantitative rigor. More
sophisticated methods like Integrated Gradients [11] and Layer-wise Relevance
Propagation (LRP) [6] provide continuous importance scores for input tokens
but can struggle with gradient saturation and non-linearity in deep models.

Probing tasks [12] have also been employed to examine the representations
learned by language models, revealing the types of linguistic information
encoded at different layers. However, these methods do not directly interpret
how inputs lead to specific outputs in inference tasks.

###  2.3 Shapley Values in Machine Learning and NLP

Shapley values, originating from cooperative game theory, have emerged as a
powerful tool for feature importance estimation in machine learning. Lundberg
and Lee’s SHAP method [3] unified several feature attribution techniques under
the Shapley value framework, ensuring consistency and local accuracy. However,
the computational intensity of exact Shapley value calculation has led to
approximations like KernelSHAP and TreeSHAP, which are primarily designed for
fixed-length feature vectors.

Applying Shapley values to NLP tasks presents unique challenges due to the
combinatorial explosion of possible token subsets in variable-length text.
Recent work by Sundararajan et al. [11] introduced TracIn to track the
influence of training data points on predictions, but it doesn’t provide
granular token-level insights for individual predictions.

##  3 Methodology

###  3.1 TokenSHAP Overview

TokenSHAP attributes importance to individual tokens or substrings in an input
prompt by estimating their Shapley values. The Shapley value for a token
represents its average marginal contribution to the model’s output across all
possible combinations of tokens. This approach provides a rigorous framework
for understanding how each part of the input influences the final response of
large language models (LLMs).

###  3.2 Tokenization and Sampling

Given an input prompt $x=(x_{1},...,x_{n})$, where $x_{i}$ represents
individual tokens or substrings, we consider all possible subsets $S\subseteq
N$, where $N=\\{1,...,n\\}$. The exponential number of subsets ($2^{n}$) makes
exact computation impractical, so we employ Monte Carlo sampling to estimate
Shapley values efficiently. This sampling approach balances the need for
computational feasibility with the accuracy of Shapley value estimations.

###  3.3 Monte Carlo Shapley Estimation

We adapt the Monte Carlo sampling approach to the context of text inputs. For
each token $x_{i}$, we estimate its Shapley value $\phi_{i}$ as follows:

  1. 1.

Generate a set of combinations that includes:

    1. (a)

All combinations where $x_{i}$ is the only token removed (essential
combinations)

    2. (b)

A random sample of other combinations based on a specified sampling ratio

  2. 2.

For each combination:

    1. (a)

Generate the model’s response

    2. (b)

Calculate the cosine similarity between this response and the full prompt
response

  3. 3.

Compute the average similarity for combinations with and without $x_{i}$

  4. 4.

Calculate $\phi_{i}$ as the difference between these averages

This Monte Carlo estimation approach ensures a balance between computational
efficiency and estimation accuracy. The use of essential combinations
alongside random samples provides a robust basis for estimating Shapley
values, even with a relatively small number of samples.

###  3.4 Value Function

We define the value function $v(S)$ as the cosine similarity between the TF-
IDF vectors of the model’s response to the subset $S$ and the response to the
full prompt. Formally:

$v(S)=\text{cosine\\_similarity}(\text{TF-IDF}(r(S)),\text{TF-IDF}(r(N)))$ (1)

where $r(S)$ is the model’s response to the subset $S$, and $r(N)$ is the
response to the full prompt. This formulation allows us to measure how closely
the response to a subset resembles the response to the entire input, providing
a quantitative basis for attributing importance to individual tokens.

###  3.5 Model Interaction

For each sampled subset $S$, we query the LLM to generate a response. The
prompt for a subset is constructed by concatenating the tokens or substrings
corresponding to the indices in $S$. This step ensures that the model’s
behavior is consistently evaluated across varying subsets of the input.

###  3.6 Shapley Value Computation

The estimated Shapley value for token $x_{i}$ is computed as:

$\displaystyle\phi_{i}=$ $\displaystyle\text{(average similarity of
combinations including }x_{i}\text{)}$ $\displaystyle-\text{(average
similarity of combinations excluding }x_{i}\text{)}$ (2)

This difference in average similarities provides a measure of the token’s
importance to the model’s output. The final Shapley values are normalized to
ensure comparability across different inputs and models.

Algorithm 1 TokenSHAP

1:Input prompt $x$, model name, sampling ratio $r$, tokenizer/splitter

2:Shapley values $\phi_{i}$ for each token $x_{i}$

3:Tokenize or split $x$ into $n$ tokens $(x_{1},\ldots,x_{n})$

4:Calculate baseline response $b$ for full prompt $x$

5:Initialize essential combinations $E\leftarrow\\{\\}$

6:for $i=1$ to $n$ do

7: $E\leftarrow E\cup{(x_{1},\ldots,x_{i-1},x_{i+1},\ldots,x_{n})}$

8:end for

9:$N\leftarrow\min(n,\lfloor(2^{n}-1)\cdot r\rfloor)$ $\triangleright$ Number
of sampled combinations

10:if $N<n$ then

11: $C\leftarrow E$ $\triangleright$ Use only first-order samples

12:else

13: $S\leftarrow$ Random sample of $N-n$ combinations excluding $E$

14: $C\leftarrow E\cup S$ $\triangleright$ All combinations to process

15:end if

16:for each combination $c$ in $C$ do

17: Get model response $R_{c}$ for $c$

18: Calculate cosine similarity $\text{sim}(b,R_{c})$

19:end for

20:for $i=1$ to $n$ do

21: $\text{with}_{i}\leftarrow$ average similarity of combinations including
$x_{i}$

22: $\text{without}_{i}\leftarrow$ average similarity of combinations
excluding $x_{i}$

23: $\phi_{i}\leftarrow\text{with}_{i}-\text{without}_{i}$

24:end for

25:Normalize ${\phi_{1},\ldots,\phi_{n}}$

26:return ${\phi_{1},\ldots,\phi_{n}}$

###  3.7 Visualization

We present the results using a color-coded visualization of the input text.
The color intensity represents the magnitude of the Shapley value for each
token or substring, with a diverging color map (e.g., coolwarm) to distinguish
positive and negative values. This visualization aids in intuitively
understanding the model’s decision-making process by highlighting the most
influential parts of the input.

Figure 2: A graph that shows the visualization of the prompt in blue-red
colors.

By providing a clear and quantitative analysis of token importance, TokenSHAP
enhances the interpretability of LLMs, offering insights that are critical for
improving model transparency, trustworthiness, and overall performance.

##  4 Experiments

###  4.1 Injection of Random Words and Method Comparison

This experiment evaluates the ability of different interpretability methods to
accurately assign low importance to randomly injected words within prompts.
The goal is to test each method’s sensitivity and precision in identifying
extraneous words that should not significantly impact model decisions.

####  4.1.1 Experimental Design

We selected random prompts from the alpaca dataset and injected each with
random words at random places. We examined the performance of the following
explainability methods in assigning low importance to those random words:

  1. 1.

Random: This method uses a random baseline, assigning random importance to
each token.

  2. 2.

Prompt Engineer: This method uses relevant prompts to derive the tokens’
importance from an LLM model. Llama3 was used with few-shot in context
learning.

  3. 3.

TokenSHAP: Utilizes Shapley values to quantify the impact of each token within
a prompt on the model’s output, effectively identifying tokens with low
importance.

####  4.1.2 Results and Evaluation

This section details the performance of each interpretability method when
applied to both regular and injected prompts. Effective methods are expected
to demonstrate the ability to discern between ’real’ and injected words by
assigning significantly lower importance to the latter.

##### Statistical Analysis

The analysis focused on comparing the average importace values and standard
deviations for ’real’ words against those for injected words. Effective
discrimination by a method would manifest as a substantial difference in these
metrics, with lower values for injected words indicating better performance.

##### Results Summary

Table 1 presents the differences in mean importance values and standard
deviations between non-injected and injected words for each evaluated method.
Notably, a method performing well should show a larger mean difference and a
controlled standard deviation.

Method |  $\Delta$ Mean Importance |  $\Delta$ Std Dev
---|---|---
Random | 0.017 | -0.017
Prompt Engineer | 0.019 | -0.001
TokenSHAP | 0.033 | 0.011
Table 1: Differential importance values between non-injected and injected
words across methods

####  4.1.3 Visual Analysis

Boxplots were generated to visually depict the distribution of importance
values for each method, contrasting injected versus non-injected words. These
plots underscore the quantitative findings and highlight how each method
manages the variance and central tendency of importance values across
conditions.

Figure 3: Box plot showing the distribution of importance values for the
Random Baseline method. Figure 4: Box plot showing the distribution of
importance values for the Prompt Engineering method. Figure 5: Box plot
showing the distribution of importance values for TokenSHAP.

####  4.1.4 Discussion

As anticipated, the Random method performed the poorest, showing minimal
differentiation between real and injected words. Prompt Engineering
demonstrated slight improvement but remained limited in discriminative power.
In contrast, TokenSHAP significantly excelled, effectively distinguishing
between relevant and irrelevant tokens with its realistic and lower SHAP
values for injected words, thus proving to be the most reliable method for
ensuring model interpretability and transparency.

###  4.2 Monte Carlo Shapley Value Approximation

####  4.2.1 Experimental Setup

To assess the effectiveness of Monte Carlo sampling in approximating Shapley
values under diverse conditions, we designed an experiment comparing different
sampling ratios, both with and without the inclusion of first-order omission
conditions. The first-order omission condition entails always including
subgroups that omit exactly one token, offering a consistent baseline for
comparison. This condition was tested alongside scenarios where it was
entirely excluded, allowing us to explore the impact of this methodological
choice on the approximation accuracy.

####  4.2.2 Methodology

The experiment involved calculating the cosine similarity between true Shapley
values and those approximated by the Monte Carlo method across various
sampling ratios. These ratios ranged from 1.0 (full sampling) down to 0.0. The
true Shapley values were computed comprehensively, and then the similarity to
these values was measured by comparing the results from the Monte Carlo
approximations to the original Shapley value vector through cosine similarity.
This metric provides a clear measure of how closely the approximations match
the true values, highlighting the accuracy of the sampling method.

####  4.2.3 Results and Analysis

Figure 6: Change in average similarity between true Shapley values and their
approximations under different sampling ratios, with and without the condition
of first-order omission.

Figure 6 presents the results, demonstrating significant differences in
approximation accuracy depending on the presence of the first-order omission
condition. The sampling ratio plays a crucial role in determining the number
of combinations considered beyond the essential first-order samples. It’s
important to note:

  * •

A sampling ratio of 0.0 means only the essential first-order combinations are
used, with no additional combinations.

  * •

At low sampling ratios (e.g., 0.1), the number of additional combinations is
often negligible, especially for smaller input sizes, effectively yielding
results similar to the 0.0 ratio.

  * •

As the sampling ratio increases (e.g., 0.2), a small number of additional
combinations are included. However, this limited increase may introduce some
noise without significantly improving accuracy.

  * •

With the first-order omission condition present, even at lower sampling ratios
(0.1 and 0.0), the similarity between estimated and true Shapley values
remains relatively stable or shows a slight increase.

  * •

Starting from a sampling ratio of 0.4 and higher, the increased number of
combinations begins to reduce noise, and the accuracy of the approximations
exceeds those observed at lower ratios.

####  4.2.4 Implications

These findings underscore the importance of including first-order omissions in
Monte Carlo sampling to maintain robustness and reliability in Shapley value
approximations. This approach validates the Monte Carlo sampler’s capability
to accurately estimate Shapley values, highlighting its utility in practical
applications where computational efficiency is critical. The results also
illustrate a delicate balance between computational demands and the fidelity
of approximations, emphasizing the necessity for strategically designed
sampling techniques in deploying TokenSHAP to interpret large language models.

##  5 Discussion

###  5.1 Interpretability Insights

TokenSHAP offers several advantages for interpreting LLM outputs:

  1. 1.

Quantitative Measure: It provides a rigorous, quantitative assessment of token
importance, utilizing the Shapley value framework to quantify the contribution
of each token to the model’s output in a consistent and objective manner.

  2. 2.

Context-awareness: The method captures the interdependence between tokens,
reflecting how the model processes the entire input. This contextual
sensitivity is essential for accurately interpreting the sophisticated
dynamics of LLMs.

  3. 3.

Model-agnostic: TokenSHAP can be applied to any LLM without requiring access
to its internal architecture, making it a versatile tool for users working
with proprietary or black-box models. This positions TokenSHAP as a powerful
black box method in the landscape of explainable AI, contrasting with white
box methods that require detailed knowledge of model internals.

  4. 4.

Granularity: The approach allows for analysis at both token and substring
levels, offering significant flexibility and enabling detailed exploration of
how linguistic constructs larger than single tokens influence the model’s
decisions.

As a black box method, TokenSHAP complements existing white box approaches in
the field of explainable AI. While white box methods like gradient-based
saliency maps offer insights tied directly to model parameters, TokenSHAP’s
model-agnostic nature allows it to be applied more broadly, including to
proprietary models or in situations where internal model access is restricted.
This versatility makes TokenSHAP particularly valuable in real-world scenarios
where model transparency may be limited due to commercial or security
considerations.

###  5.2 Limitations

  1. 1.

Computational Cost: Despite the efficiency gains from Monte Carlo sampling,
TokenSHAP remains more computationally intensive than simpler interpretability
methods, due to the need for repeated model evaluations.

  2. 2.

Sensitivity to Sampling: The stochastic nature of Monte Carlo sampling
introduces variability in the importance scores, which may slightly vary
between runs, affecting reproducibility in sensitive applications.

  3. 3.

Assumption of Additivity: The theoretical foundation of Shapley values assumes
that contributions from individual tokens can be additively combined, which
may not always be accurate in cases where complex interactions and non-linear
dynamics dominate.

###  5.3 Future Work

  1. 1.

Exploring Alternative Value Functions: Future research could include
developing more sophisticated value functions that better capture nuanced
aspects of semantic similarity and contextual alignment. Usage of LLM can also
be considered for this task.

  2. 2.

Investigating Shapley Value Stability: Further studies are needed to assess
the stability of Shapley values across different LLM architectures and input
sizes, to understand their robustness and generalizability.

  3. 3.

Developing Interactive Tools: There is a substantial opportunity to create
interactive, user-friendly tools that allow practitioners to dynamically
explore token importance, enhancing the accessibility and practical utility of
TokenSHAP.

  4. 4.

Extending to Multi-turn Conversations: Applying TokenSHAP to multi-turn
conversational contexts could provide insights into how contextual
understanding evolves in dialogue systems.

  5. 5.

Bias Analysis: Utilizing TokenSHAP for systematic identification and analysis
of potential biases in LLM outputs could contribute to the development of more
ethical and fair AI systems.

##  6 Conclusion

TokenSHAP offers a significant advancement in the interpretability of large
language models (LLMs) by adapting Shapley values to natural language
processing and employing Monte Carlo estimation for feasibility. This approach
overcomes the challenges of variable input lengths and contextual
dependencies, offering a scalable solution for complex language models.

Key achievements include:

  * •

A novel framework that extends Shapley values to natural language, providing a
rigorous, theoretically grounded method for interpreting token importance.

  * •

An efficient Monte Carlo sampling method that enhances the computational
feasibility of applying TokenSHAP to large-scale models.

  * •

Superior performance over existing methods in terms of alignment with human
judgments, model behavior faithfulness, and consistency.

  * •

Detailed insights into LLM behavior, revealing how models process and
prioritize input components.

Our method’s capacity to capture detailed token interactions enhances model
transparency and aids in debugging, bias mitigation, and regulatory
compliance, which is essential as LLMs are increasingly deployed in critical
domains.

Future research will explore sophisticated value functions, the stability of
Shapley values across models, and the extension of TokenSHAP to conversational
AI. Developing interactive tools based on TokenSHAP could also enhance its
accessibility and practical utility for practitioners.

TokenSHAP represents a vital step towards making AI systems not only powerful
but also transparent and accountable, ensuring their responsible development
and deployment in transformative applications.

## References

  * [1] Guidotti, R., Monreale, A., Ruggieri, S., Turini, F., Giannotti, F., & Pedreschi, D. (2018). A survey of methods for explaining black box models. ACM computing surveys (CSUR), 51(5), 1–42.
  * [2] Ribeiro, M. T., Singh, S., & Guestrin, C. (2016). Why should I trust you? Explaining the predictions of any classifier. In Proceedings of the 22nd ACM SIGKDD international conference on knowledge discovery and data mining (pp. 1135–1144).
  * [3] Lundberg, S. M., & Lee, S. I. (2017). A unified approach to interpreting model predictions. In Advances in neural information processing systems (pp. 4765–4774).
  * [4] Molnar, C. (2020). Interpretable machine learning. Lulu. com.
  * [5] Simonyan, K., Vedaldi, A., & Zisserman, A. (2013). Deep inside convolutional networks: Visualising image classification models and saliency maps. In Workshop at International Conference on Learning Representations.
  * [6] Bach, S., Binder, A., Montavon, G., Klauschen, F., Müller, K. R., & Samek, W. (2015). On pixel-wise explanations for non-linear classifier decisions by layer-wise relevance propagation. PloS one, 10(7), e0130140.
  * [7] Gilpin, L. H., Bau, D., Yuan, B. Z., Bajwa, A., Specter, M., & Kagal, L. (2018). Explaining explanations: An overview of interpretability of machine learning. In 2018 IEEE 5th International Conference on data science and advanced analytics (DSAA) (pp. 80–89). IEEE.
  * [8] Wachter, S., Mittelstadt, B., & Russell, C. (2017). Counterfactual explanations without opening the black box: Automated decisions and the GDPR. Harvard Journal of Law & Technology, 31, 841.
  * [9] Danilevsky, M., Qian, K., Aharonov, R., Katsis, Y., Kawas, B., & Sen, P. (2020). A survey of the state of explainable AI for natural language processing. arXiv preprint arXiv:2010.00711.
  * [10] Vig, J. (2019). A multiscale visualization of attention in the transformer model. In Proceedings of the 57th Annual Meeting of the Association for Computational Linguistics: System Demonstrations (pp. 37–42).
  * [11] Sundararajan, M., Taly, A., & Yan, Q. (2017). Axiomatic attribution for deep networks. In Proceedings of the 34th International Conference on Machine Learning (pp. 3319–3328).
  * [12] Tenney, I., Das, D., & Pavlick, E. (2019). BERT rediscovers the classical NLP pipeline. In Proceedings of the 57th Annual Meeting of the Association for Computational Linguistics (pp. 4593–4601).
  * [13] Castro, J., Gómez, D., & Tejada, J. (2009). Polynomial calculation of the Shapley value based on sampling. Computers & Operations Research, 36(5), 1726–1730.


[a]David Mason

# The deconfinement phase transition in $\SpN$ gauge theories and the density
of states method

Biagio Lucini  Maurizio Piai  Enrico Rinaldi  Davide Vadacchino

###### Abstract

First-order phase transitions in the early universe might produce a detectable
background of gravitational waves. As these phase transitions can be generated
by new physics, it is important to quantify these effects. Many pure Yang-
Mills gauge theories are known to undergo first-order deconfinement phase
transitions, with properties that can be studied with lattice simulations.
Despite the recent surge of interest in $\SpN$ gauge theories as a candidate
for models of physics beyond the standard model, studies of these theories at
finite temperature are still very limited. In this contribution we will
present preliminary results of an ongoing numerical investigation of the
thermodynamic properties of the deconfinement phase transition in $\SpF$ Yang-
Mills theory, using the linear logarithmic relaxation algorithm. This method
enables us to obtain a highly accurate determination of the density of states,
allowing for a precise reconstruction of thermodynamic observables. In
particular, it gives access to otherwise difficult to determine quantities
such as the free energy of the system, even along metastable and unstable
branches, hence providing an additional direct observable to study the
dynamics of the phase transition.

##  1 Introduction

New strongly interacting sectors based on symplectic gauge theories have
properties which are phenomenologically interesting to physics beyond the
standard model. For a recent review on this see Ref. [1] and references
therein. Due to the non-perturbative nature of these theories, the lattice is
the natural first-principles method to study their properties.

If, as is the case for deconfinement in $\SpN$ $(N>1)$, these theories undergo
first-order phase transitions in the early universe, they can leave imprint
through the generation of long wavelength gravitational waves, opening
gravitational wave astronomy as a promising route to constrain new physics,
see for instance discussion from Ref. [2] and references therein. The recent
announced detection of a gravitational wave background from NANOGrav would be
consistent with this type of signal [3]. However, to constrain new physics,
accurate estimations of the expected power spectrum from theory is required.
The lattice literature on $\SpN$ gauge theories at finite temperature is quite
limited [4, 5]. We present the initial results of a project which aims to
accurately calculate the thermodynamic properties of the deconfinement phase
transition in $\SpN$ gauge theories through the use of a novel lattice method,
the logarithmic linear relaxation (LLR) method [6, 7, 8, 9, 10, 11, 12, 13,
14, 15, 16].

Two important quantities of interest are latent heat and surface tension. In
the critical region the system will exhibit co-existing phases separated by a
potential barrier. The probability of tunnelling through the barrier is
related to the surface tension and the latent heat is equal to the energy
difference between the phases. Through continuum and infinite volume
extrapolations of observables of the lattice system, these quantities can be
computed in the continuum theory.

Sec. 2 presents the lattice setup and details of the LLR algorithm. In Sec. 3,
we report the thermodynamic observables for a single lattice size,
$N_{t}\times N_{s}^{3}=4\times 20^{3}$, for $\SpF$ pure gauge theory, and we
discuss and compare our results to those of SU(3) pure gauge theory [6] — see
also the comprehensive discussion in Ref. [17].

##  2 Lattice setup and the LLR method

We use an $N_{t}\times N_{s}^{3}$ hypercubic lattice in Euclidean spacetime,
with periodic boundary conditions, spacing $a$ and volume
$\tilde{V}=a^{4}N_{t}N_{s}^{3}$. When $N_{t}<N_{s}$, the temperature of this
system is given by the inverse of the temporal extent $T=1/aN_{t}$. We use the
standard Wilson action, defined by

$S[U]\equiv\frac{6\tilde{V}}{a^{4}}(1-u_{p}[U]),$ (1)

where $U$ refers to the lattice configuration and $u_{p}$ is the average
plaquette. The partition function of this system at coupling $\beta$ is given
by

$Z_{\beta}\equiv\int[DU_{\mu}]e^{-\beta S[U]}.$ (2)

To study the thermodynamic properties across the phase transition, the
temperature is varied by keeping the number of sites fixed and altering the
lattice spacing through the coupling, $\beta(a)$.

The lattice configurations, $U$, consist of link variables
$U=\\{U_{\mu}(n_{t},\vec{n_{s}})\\}$, where $U_{\mu}\in\SpN\subset$SU(2N) ,
obey the condition $U_{\mu}\Omega(U_{\mu})^{T}=\Omega$. The symplectic matrix,
$\Omega$, is defined as

$\Omega=\begin{pmatrix}0&\mathds{1}_{N\times N}\\\ -\mathds{1}_{N\times
N}&0\end{pmatrix}.$ (3)

The centre of the group $\SpN$ is $\mathbb{Z}_{2}$, for all finite $N$. On the
lattice, the deconfinement phase transition is associated with the spontaneous
breaking of the centre symmetry. The corresponding order parameter is the
average Polyakov loop, defined as

$\left\langle
l_{p}\right\rangle_{\beta}\equiv\left\langle\frac{1}{2NN_{s}^{3}}\sum_{\vec{n_{s}}}\textrm{Tr}\left(\prod_{n_{t}=0}^{N_{t}-1}U_{0}(n_{t},\vec{n_{s}})\right)\right\rangle_{\beta}\
\ \begin{cases}=0\text{ confined phase}\\\ \neq 0\text{ deconfined
phase}\end{cases}\ .$ (4)

For $N>1$, $\SpN$ gauge theories have first-order deconfinement phase
transitions. The co-existence of phases in the proximity of first-order
transitions leads to metastable dynamics which is problematic when using Monte
Carlo importance sampling methods. To accurately sample the phase space, the
system must tunnel between the vacua many times. For small lattices, in
general this is not a problem as the potential barrier is small. However, as
the lattice volume increases, the potential barrier grows and more
configurations are required to obtain accurate results.

Following the results presented in Ref. [6], we employ the logarithmic linear
relaxation method to overcome the metastability problems around the
deconfinement phase transition. The goal of the LLR method is to accurately
estimate the density of states,

$\rho(E)\equiv\int[DU]\delta(S[U]-E),$ (5)

as a piecewise log linear function, with the total energy range relevant to
the processes of interest broken down into $2N_{I}-1$ intervals,

$\ln\rho(E)\approx a_{n}(E-E_{n})+c_{n}.$ (6)

where $n=1,...,2N_{I}-1$, denotes the energy interval that the expression is
valid for, $E_{n}-\Delta_{E}/4\leq E\leq E_{n}+\Delta_{E}/4$. The $c_{n}$ term
is set by the continuity of $\rho$ at the boundary of the intervals,
$c_{n}=a_{1}\Delta_{E}/4+(\Delta_{E}/2)\sum_{k=2}^{n-1}a_{k}+\Delta_{E}a_{n}/4$.
We take $E_{1}\neq 0$, therefore $c_{1}$ is arbitrary and we set it to 0.

In each interval we calculate the coefficient, $a_{n}$, by solving the
equation $\langle\langle E-E_{n}\rangle\rangle_{n}(a_{n})=\langle\langle
u_{p}-(u_{p})_{n}\rangle\rangle_{n}(a_{n})=0$, where
$(u_{p})_{n}=1-E_{n}a^{4}/6\tilde{V}$. The double angle bracket
$\langle\langle...\rangle\rangle_{n}(a_{n})$ denotes the expectation value for
configurations restricted to the $n^{th}$ energy interval, for a coupling
$a_{n}$. In this work, these expectation values are computed through a
modified heat bath algorithm, as discussed in Ref. [6].

We solve these equations, to find the set of $a_{n}$ values, through a
combination of Newton-Raphson and Robbins-Monro iterations, Eq. (12) and (14)
of Ref. [6] respectively. In the limit of infinite Robbins-Monro iterations,
the exact solution would be found, however only a finite number of iterations
are possible, introducing a truncation error. The truncation error is
estimated by repeating the determination of the $a_{n}$ values and
bootstrapping the results.

Once $\\{a_{n}\\}_{n=1}^{2N_{I}-1}$ has been found, the density of states can
be reconstructed. This is used to compute the plaquette, or equivalently
energy, distribution of the system at coupling $\beta$, through the relation

$P_{\beta}(u_{p})=\frac{1}{Z_{\beta}}\rho(E)e^{-\beta
E}|_{E=6\tilde{V}(1-u_{p})/a^{4}},\quad Z_{\beta}=\int dE\rho(E)e^{-\beta E}.$
(7)

Expectation values can then be reconstructed avoiding the metastability
problem.

This work focuses on $\SpF$ pure gauge theory on lattice with size $4\times
20^{3}$. The energy range is split into 48 intervals, from $(u_{p})_{1}=0.58$
to $(u_{p})_{48}=0.565$, and interval size
$\Delta_{E}a^{4}/6\tilde{V}=0.0006$. The initial estimates for $a_{n}$ were
based on standard importance sampling results. This guess is improved through
by 10 Newton-Raphson iterations followed by 300 Robbins-Monro iterations. This
process has been repeated 20 times to estimate truncation errors. The final
$a_{n}$ values and their dependence on the centre of the interval
$(u_{p})_{n}$ is shown in blue in Fig. 1. In black is the importance sampling
results for $\langle u_{p}\rangle_{\beta}$ against $\beta$. The initial
guesses for $a_{n}$ is shown in red. The non-invertible behaviour in the
function of $a_{n}(u_{p})$, that is characteristic of a first-order
transition, emerges dynamically.

Figure 1: The values of $a_{n}$ against $(u_{p})_{n}=1-E_{n}a^{4}/6\tilde{V}$,
the plaquette values corresponding to the centre of each energy interval, for
$\SpF$ pure gauge theory on a lattice of size $4\times 20^{3}$. We use 48
intervals between $(u_{p})_{1}=0.58$ and $(u_{p})_{48}=0.565$, with
$\Delta_{E}a^{4}/6\tilde{V}=0.0006$. The solid blue line shows the final
values of $a_{n}$ after 10 Newton-Raphson iterations followed by 300 Robbins-
Monro iterations, with errors calculated from 20 repeats. The black crosses
show the measured vacuum expectation value of the average plaquette against
$\beta$, calculated from 500,000 configurations generated using conventional
importance sampling methods, for $\beta=7.32,7.33,7.34$ and $7.35$. The red
dashed line shows the initial $a_{n}$ values, constructed by fitting a cubic
polynomial to the importance sampling results.

##  3 Thermodynamic observables

The main thermodynamic properties we are interested in studying in this work
are the latent heat and the surface tension. These observables can be related,
through infinite volume and continuum extrapolations, to properties of the
plaquette distribution at the critical point, $\beta_{c}$, at which the two
coexisting phases are equally probable — the plaquette distribution has a
double Gaussian structure with two peaks of equal height. The difference in
the value of plaquette between the two peaks, $\Delta\langle
u_{p}\rangle_{\beta_{c}}$, can be used to compute the latent heat, Eq. (33) of
Ref. [18]. The logarithm of the ratio of the height of the central (unstable)
minima, $P_{min}$, and the degenerate maxima of the plaquette distribution,
$P_{max}$, $-\ln(P_{min}/P_{max})$, can be related to the surface tension, see
Eq. (48) of Ref. [18]. Here we compute these lattice quantities at a single
lattice size.

The critical coupling can be found by fitting a double Gaussian distribution
to the plaquette probability distribution, Eq. (8), and tuning the coupling
until the difference in height between the two maxima of the distribution is
below a certain threshold. At this point $\Delta\langle
u_{p}\rangle_{\beta_{c}}$ and $-\ln(P_{min}/P_{max})$ can be read directly
from the distribution, as shown in Fig. 2(a), with the results presented in
Tab. 1.

The same quantities can also be determined directly from the free energy of
the micro-states, $F$, we define as

$F(t)\equiv E-ts,\quad s\equiv\ln\rho\quad t\equiv\frac{\partial E}{\partial
s}\equiv\frac{1}{a_{n}},\quad f(t)\equiv\frac{a^{4}}{\tilde{V}}(F(t)+\Sigma
t),$ (8)

where $s$ is the entropy and $t$ is the temperature. $f(t)$ is defined to
remove the dependence of the arbitrary $c_{1}$ term in Eq. (6) and make the
observed swallow tail structure more clear. The additive constant term
$\Sigma$, is equal to the average entropy.

From the plot of $f$ against $t$, Fig. 2(b), we can determine the critical
point, the plaquette jump and $-ln(P_{min}/P_{max})$. The critical point is
the point at which the two metastable branches (solid blue lines) cross (cyan
dashed vertical line), with $\beta_{c}=1/t_{c}$. The plaquette jump is
determined by the difference between the plaquette values in the metastable
regions at which $a_{n}=\beta_{c}$ in the plot of $a_{n}$ against $u_{p}$, see
the inset of Fig. 2(b). Note we use a linear interpolation between the points.
The free energy is related to the extrema of the energy distribution, through

$e^{-\frac{F(t)}{t}}=Z_{\beta}P_{\beta}(E)|_{\beta=1/t,E=F(t)+ts}.$ (9)

The change in the free energy between the metastable and unstable branches at
the critical point, $\Delta F(t_{c})=\tilde{V}\Delta f(t_{c})/a^{4}$, can be
related to the logarithmic term we calculated previously as
$-\ln(P_{min}/P_{max})=\tilde{V}\Delta f(t_{c})/a^{4}t_{c}$.

In Tab. 1 the results from the probability distribution and from the free
energy are presented, for $\SpF$, for a single lattice size. They are in good
agreement, demonstrating that the methods calculate equivalent values.
Additionally, included in this table are the results for SU(3) for the same
lattice size, calculated from the free energy using LLR results presented
previously in Ref. [6]. Direct comparison of the results, suggest that SU(3)
has a stronger first-order phase transition as both the plaquette jump is
larger and the change in the free energy is larger. Therefore making jump in
energy larger and the probability of tunnelling through the barrier lower.
However, these quantities are not physical. Without taking the infinite volume
and continuum limits, to gain access to the physical quantities of the
continuum theory — surface tension and latent heat— we cannot reach any
definite conclusions on their relative value.

(a)

(b)

Figure 2: The reconstructed plaquette distribution at the critical point (left) and the free-energy (right) for $\SpF$ pure gauge theory on a lattice of size $4\times 20^{3}$, found using the LLR method with 48 intervals between plaquette values of $(u_{p})_{1}=0.58$ and $(u_{p})_{48}=0.565$, with $\Delta_{E}a^{4}/6\tilde{V}=0.0006$. The critical coupling in the left plot was found by tuning $\beta$ until the two peaks of the distribution have equal height. The plaquette values corresponding to the peaks of the distribution are shown by the green dashed line. The height of the maxima, $P_{max}$, and minima, $P_{min}$, are shown by the orange dashed line. On the right panel the red, blue and black lines show the unstable, metastable and stable regions, respectively. The points in the inset match those of the main plot, showing the corresponding values of $a_{n}$ and $u_{p}$. The critical coupling is shown by the dashed cyan line, the orange dashed line shows the free energy values when $t=t_{c}$ and the green dashed line on the inset show the plaquette values when $a_{n}=1/t_{c}$.  Table 1: The values of the critical coupling, $\beta_{c}$, the plaquette jump, $\Delta\langle u_{p}\rangle_{\beta_{c}}$, and the change in the free energy divided by the critical temperature, $\Delta F(t_{c})/t_{c}$, are shown for SU(3) and $\SpF$ pure gauge theories on $4\times 20^{3}$ lattices. The $\SpF$ results were found using the LLR method with 48 intervals between plaquette values of $(u_{p})_{1}=0.58$ and $(u_{p})_{48}=0.565$, with $\Delta_{E}a^{4}/6\tilde{V}=0.0006$. The SU(3) values were computed using the results of previous work presented in Ref. [6] with $\Delta_{E}a^{4}/6\tilde{V}=0.0007$. The second column notes whether the value was calculated from the free energy,$F(t)$, or from the plaquette distribution, $P_{\beta_{c}}(u_{p})$.  |  | $\beta_{c}=1/t_{c}$ | $\Delta\langle u_{p}\rangle_{\beta_{c}}$ | $\Delta F(t_{c})/t_{c}=-\ln(P_{min}/P_{max})$
---|---|---|---|---
SU(3) | $F(t)$ | $5.69189(4)$ | $0.00257(3)$ | $0.0919(83)$
$\SpF$ | $P_{\beta_{c}}(u_{p})$ | $7.34009(3)$ | $0.00203(2)$ | $0.0714(43)$
$\SpF$ | $F(t)$ | $7.34010(3)$ | $0.00205(3)$ | $0.0708(48)$

##  4 Conclusion

We presented initial results for the thermodynamic properties of the
deconfinement phase transition in $\SpN$ gauge theories using the LLR method.
Building on the methodology presented in Ref. [6], we calculated the critical
coupling, the plaquette jump and the change in the free energy at this
coupling for $\SpF$ on a single lattice size $4\times 20^{3}$, presented in
Tab. 1. These quantities have been calculated by reconstructing the energy
distribution at the critical point, Fig. 2(a), and separately from the free
energy of the micro-states, Fig. 2(b). Both methods are in good agreement.
These results were compared with results for SU(3) at the same lattice size,
although without continuum and infinite volume extrapolations these
comparisons are not physical. In future work, we aim to extend our analysis to
additional lattice sizes to accurately compute the physical thermodynamic
observables, the latent heat and the surface tension.

## Acknowledgments

The work of D. V. is partly supported by the Simons Foundation under the
program “Targeted Grants to Institutes” awarded to the Hamilton Mathematics
Institute. The work of D. M. is supported by a studentship awarded by the Data
Intensive Centre for Doctoral Training, which is funded by the STFC grant
ST/P006779/1. B. L. and M. P. received funding from the European Research
Council (ERC) under the European Union’s Horizon 2020 research and innovation
program under Grant Agreement No. 813942, and by STFC under Consolidated
Grants No. ST/P00055X/1, ST/T000813/1, and ST/X000648/1. The work of B. L. is
further supported in part by the Royal Society Wolfson Research Merit Award
WM170010 and by the Leverhulme Trust Research Fellowship No. RF-2020-4619.
Numerical simulations have been performed on the Swansea SUNBIRD cluster (part
of the Supercomputing Wales project) and AccelerateAI A100 GPU system, on the
DiRAC Data Intensive service at Leicester, and on the DiRAC Extreme Scaling
service at the University of Edinburgh. The Swansea SUNBIRD system and
AccelerateAI are part funded by the European Regional Development Fund (ERDF)
via Welsh Government. The DiRAC Data Intensive service at Leicester is
operated by the University of Leicester IT Services, and the DiRAC Extreme
Scaling service is operated by the Edinburgh Parallel Computing Centre, they
form part of the STFC DiRAC HPC Facility (www.dirac.ac.uk). The DiRAC Data
Intensive service equipment at Leicester was funded by BEIS capital funding
via STFC capital grants ST/K000373/1 and ST/R002363/1 and STFC DiRAC
Operations grant ST/R001014/1. The DiRAC Extreme Scaling service was funded by
BEIS capital funding via STFC capital grant ST/R00238X/1 and STFC DiRAC
Operations grant ST/R001006/1. DiRAC is part of the National e-Infrastructure.

Open Access Statement -  For the purpose of open access, the authors have
applied a Creative Commons Attribution (CC BY) licence to any Author Accepted
Manuscript version arising.

Research Data Access Statement—The results for $\SpF$ are based on preliminary
analysis. Further analysis and the data generated for this manuscript will be
released together with an upcoming publication. The simulation code and data
generated for the $SU(3)$ results can be found in data releases Ref. [19] and
Ref. [20] respectively.

## References

  * [1] E. Bennett, J. Holligan, D.K. Hong, H. Hsiao, J.-W. Lee, C.J.D. Lin et al., _Sp(2N) Lattice Gauge Theories and Extensions of the Standard Model of Particle Physics_ , _Universe_ 9 (2023) 236 [2304.01070].
  * [2] R. Pasechnik, M. Reichert, F. Sannino and Z.-W. Wang, _Gravitational Waves from Composite Dark Sectors_ , 2309.16755.
  * [3] NANOGrav collaboration, _The NANOGrav 15 yr Data Set: Search for Signals from New Physics_ , _Astrophys. J. Lett._ 951 (2023) L11 [2306.16219].
  * [4] K. Holland, M. Pepe and U.J. Wiese, _The Deconfinement phase transition of Sp(2) and Sp(3) Yang-Mills theories in (2+1)-dimensions and (3+1)-dimensions_ , _Nucl. Phys. B_ 694 (2004) 35 [hep-lat/0312022].
  * [5] M. Bruno, N. Forzano, M. Panero and A. Smecca, _Dark matter from a symplectic glueball model_ , _in preparation_ (2023) .
  * [6] B. Lucini, D. Mason, M. Piai, E. Rinaldi and D. Vadacchino, _First-order phase transitions in Yang-Mills theories and the density of state method_ , 2305.07463.
  * [7] D. Mason, B. Lucini, M. Piai, E. Rinaldi and D. Vadacchino, _The density of states method in Yang-Mills theories and first order phase transitions_ , _EPJ Web Conf._ 274 (2022) 08007 [2211.10373].
  * [8] D. Mason, B. Lucini, M. Piai, E. Rinaldi and D. Vadacchino, _The density of state method for first-order phase transitions in Yang-Mills theories_ , _PoS_ LATTICE2022 (2023) 216 [2212.01074].
  * [9] K. Langfeld, B. Lucini and A. Rago, _The density of states in gauge theories_ , _Phys. Rev. Lett._ 109 (2012) 111601 [1204.3243].
  * [10] K. Langfeld and J.M. Pawlowski, _Two-color QCD with heavy quarks at finite densities_ , _Phys. Rev. D_ 88 (2013) 071502 [1307.0455].
  * [11] K. Langfeld, B. Lucini, R. Pellegrini and A. Rago, _An efficient algorithm for numerical computations of continuous densities of states_ , _Eur. Phys. J. C_ 76 (2016) 306 [1509.08391].
  * [12] G. Cossu, D. Lancaster, B. Lucini, R. Pellegrini and A. Rago, _Ergodic sampling of the topological charge using the density of states_ , _Eur. Phys. J. C_ 81 (2021) 375 [2102.03630].
  * [13] F. Springer and D. Schaich, _Density of states for gravitational waves_ , _PoS_ LATTICE2021 (2022) 043 [2112.11868].
  * [14] F. Springer and D. Schaich, _Progress applying density of states for gravitational waves_ , _EPJ Web Conf._ 274 (2022) 08008 [2212.09199].
  * [15] Lattice Strong Dynamics (LSD) collaboration, _Advances in using density of states for large-N Yang–Mills_ , _PoS_ LATTICE2022 (2023) 223 [2303.01149].
  * [16] B. Lucini, W. Fall and K. Langfeld, _Overcoming strong metastabilities with the LLR method_ , _PoS_ LATTICE2016 (2016) 275 [1611.00019].
  * [17] S. Borsanyi, K. R., Z. Fodor, D.A. Godzieba, P. Parotto and D. Sexty, _Precision study of the continuum SU(3) Yang-Mills theory: How to use parallel tempering to improve on supercritical slowing down for first order phase transitions_ , _Phys. Rev. D_ 105 (2022) 074513 [2202.05234].
  * [18] B. Lucini, M. Teper and U. Wenger, _Properties of the deconfining phase transition in SU(N) gauge theories_ , _JHEP_ 02 (2005) 033 [hep-lat/0502003].
  * [19] B. Lucini, D. Mason, M. Piai, E. Rinaldi and D. Vadacchino, _First-order phase transitions in Yang-Mills theories and the density of state method — HiRep LLR Code v1.0.0_ , 2023.  10.5281/zenodo.8134756.
  * [20] B. Lucini, D. Mason, M. Piai, E. Rinaldi and D. Vadacchino, _First-order phase transitions in Yang-Mills theories and the density of state method—data and analysis code release_ , 2023.  10.5281/zenodo.8124749.