###### Abstract

We report on the development of scintillating bolometers based on lithium
molybdate crystals containing molybdenum depleted in the double-$\beta$ active
isotope 100Mo (Li2${}^{100\textnormal{depl}}$MoO4). We used two
Li2${}^{100\textnormal{depl}}$MoO4 cubic samples, 45 mm side and 0.28 kg each,
produced following purification and crystallization protocols developed for
double-$\beta$ search experiments with 100Mo-enriched Li2MoO4 crystals.
Bolometric Ge detectors were utilized to register scintillation photons
emitted by the Li2${}^{100\textnormal{depl}}$MoO4 crystal scintillators. The
measurements were performed in the CROSS cryogenic set-up at the Canfranc
underground laboratory (Spain). We observed that the
Li2${}^{100\textnormal{depl}}$MoO4 scintillating bolometers are characterized
by excellent spectrometric performance ($\sim$3–6 keV FWHM at 0.24–2.6 MeV
$\gamma$’s), moderate scintillation signal ($\sim$0.3–0.6 keV/MeV depending on
light collection conditions) and high radiopurity (228Th and 226Ra activities
are below a few $\mu$Bq/kg), comparable to the best reported results of low-
temperature detectors based on Li2MoO4 with natural or 100Mo-enriched
molybdenum content. Prospects of Li2${}^{100\textnormal{depl}}$MoO4 bolometers
for use in rare-event search experiments are briefly discussed.

###### keywords:

Cryogenic detector; Bolometer; Crystal scintillator; Lithium molybdate;
Molybdenum depleted in 100Mo; Rare events

xx 1 5 Received: date; Accepted: date; Published: date
Li2${}^{100\textnormal{depl}}$MoO4 Scintillating Bolometers for Rare-Event
Search Experiments I.C. Bandac 1, A.S. Barabash 2, L. Bergé 3, Yu.A. Borovlev
4, J.M. Calvo-Mozota 1,5, P. Carniti 6, M. Chapellier 3, I. Dafinei 7, F.A.
Danevich 8,9, L. Dumoulin 3, F. Ferri 10, A. Giuliani 3, C. Gotti 6, Ph. Gras
10, V.D. Grigorieva 4, A. Ianni 11, H. Khalife 10, V.V. Kobychev 8, S.I.
Konovalov 2, P. Loaiza 3, M. Madhukuttan 3, E.P. Makarov 4, P. de Marcillac 3,
S. Marnieros 3, C.A. Marrache-Kikuchi 3, M. Martinez 12, C. Nones 10, E.
Olivieri 3, A. Ortiz de Solórzano 12, G. Pessina 6, D.V. Poda 3*, Th. Redon 3,
J.A. Scarpaci 3, V.N. Shlegel 4, V.I. Tretyak 8,11, V.I. Umatov 2, M.M.
Zarytskyy 8, and A. Zolotarova 10 Denys Poda and CROSS collaboration
Correspondence<EMAIL_ADDRESS>
##  1 Introduction

Crystal scintillators are widely used in searches for rare-event processes
(like two-$\nu$ and $\nu$-less double-$\beta$ decays, rare $\alpha$ and
$\beta$ decays, dark matter particles), particularly using technologies of
low-temperature detectors Pirro and Mauskopf (2017); Bellini (2018); Biassoni
and Cremonesi (2020); Poda (2021); Zolotarova (2021). Among them, molybdenum
containing crystals represent a long-standing interest for double-$\beta$
decay searches Pirro and Mauskopf (2017); Bellini (2018); Biassoni and
Cremonesi (2020); Poda (2021); Zolotarova (2021), mainly in isotope 100Mo
(possible also for 92Mo and 98Mo Tretyak and Zdesenko (2002)); this isotope is
also promising for solar and supernova neutrino detection Ejiri et al. (2000,
2002); Ejiri and Elliott (2014, 2017). Different compounds with natural and a
few ones with 100Mo-enriched molybdenum content have been developed and
investigated as low-temperature detectors with simultaneous heat and
scintillation detection, i.e. scintillating bolometers Pirro and Mauskopf
(2017); Bellini (2018); Biassoni and Cremonesi (2020); Poda (2021); Zolotarova
(2021). Lithium molybdate (Li2MoO4) is found to be one of the most promising
Mo-containing scintillators for such applications Cardani et al. (2013);
Bekker et al. (2016); Armengaud et al. (2017). Natural and 100Mo-enriched
Li2MoO4 (Li2100MoO4) have been developed within the LUMINEU project and used
in searches for 100Mo double-$\beta$ decay Armengaud et al. (2017); Grigorieva
et al. (2017); Poda and LUMINEU, EDELWEISS, CUPID-0/Mo collaborations (2017);
Armengaud et al. (2020a). The LUMINEU technology of scintillating bolometers
has been adopted for CUPID-Mo double-$\beta$ experiment Armengaud et al.
(2020b, 2021); Augier et al. (2022, 2023). Such detector material is also
selected for CUPID Armstrong et al. (2019); Armatol et al. (2021a, b); Alfonso
et al. (2022, 2023) and CROSS Bandac et al. (2020, 2021); Armatol et al.
(2021b); Alfonso et al. (2023) projects, as well as it represents a great
interest for the AMoRE experiment Alenkov et al. (2015); Kim et al. (2020a, b,
2022a, 2022b).

Lithium is also an element with a long-standing interest for rare-event search
experiments. The dominant isotope, i.e. 7Li (92% in natural lithium), is a
light nucleus with non-zero spin, thus a viable candidate to probe spin-
dependent dark matter interaction Bednyakov and Šimkovic (2005); Abdelhameed
et al. (2019). Moreover, 7Li is a good target to search for solar axions, via
a resonant absorption of an axion by 7Li and its subsequent $\gamma$
deexcitation Krčmar et al. (2001); Belli et al. (2008); Barinova et al.
(2010); Belli et al. (2012); Cardani et al. (2013). Last but not least,
presence of 6Li (8% in natural lithium) allows neutron detection via a 6Li(n,
t)$\alpha$ reaction, characterized by a high cross-section to thermal
neutrons; also, enrichment in 6Li (up to 95%) is feasible and it can enhance
the detection efficiency. Thus, Li-containing detectors are of special
interest for neutron flux monitoring in rare-event search experiments Martínez
et al. (2012); Coron et al. (2016).

In this work we present development and investigation of scintillating
bolometers based on a new type of the Li2MoO4 compound, produced from
molybdenum depleted in 100Mo (Li2${}^{100\textnormal{depl}}$MoO4). The low
concentration of the double-$\beta$ active isotope 100Mo significantly reduces
the related background counting rate, which is on the level of 10 mBq/kg in a
Li2100MoO4 crystal (an order of magnitude lower for the natural one), and it
can be a dominant internal background of this material. Therefore,
Li2${}^{100\textnormal{depl}}$MoO4 looks more suitable than natural or 100Mo-
enriched Li2MoO4 crystals for dark matter and axion search experiments with
7Li, for double-$\beta$ decay searches in 92Mo and 98Mo, as well as for 6Li-
based neutron detection. Moreover, Li2${}^{100\textnormal{depl}}$MoO4 can be
used in bolometric experiments to search for double-$\beta$ decay in 100Mo, as
a complementary detector for better understanding a background model.
Therefore, a goal of this work is to study prospects of
Li2${}^{100\textnormal{depl}}$MoO4 low-temperature detectors for rare-event
search applications.

##  2 Development and test of Li2${}^{100\textnormal{depl}}$MoO4 scintillating
bolometers

###  2.1 Crystals production and construction of detectors

In this study we used two Li2${}^{100\textnormal{depl}}$MoO4 scintillation
crystals with a size of 45$\times$45$\times$45 mm and a mass of around 0.28 kg
each, identical to Li2100MoO4 crystal produced for the CROSS experiment Bandac
et al. (2020). The samples were cut from the same crystal boule grown by the
low-thermal-gradient Czochralski technique, as detailed in Grigorieva et al.
(2020). A 4N purity lithium carbonate, selected for the LUMINEU and CUPID-Mo
crystals production Grigorieva et al. (2017); Armengaud et al. (2017, 2020b),
and molybdenum oxide depleted in 100Mo ($\sim$0.01% of 100Mo; i.e. 1000 times
lower than in natural Mo) have been used as starting materials. The
${}^{100\textnormal{depl}}$MoO3 material purification, the solid-phase
synthesis of the Li2${}^{100\textnormal{depl}}$MoO4 compound, and the crystal
growth in a Pt crucible using the low-thermal-gradient Czochralski method
(double crystalization approach) were realized following protocols of Mo-
containing crystals production developed by LUMINEU Bergé et al. (2014);
Grigorieva et al. (2017); Armengaud et al. (2017) and adopted by CUPID-Mo
Armengaud et al. (2020b). A large crystal boule (the cylindrical part is
around $\oslash$60$\times$100 mm) has been grown with a crystal yield of about
80% from the initial charge Grigorieva et al. (2020) and two twin cubic
samples (with a few mm chamfers) were produced.

The assembly of detectors have been carried out in an ISO class 4 clean room
of the IJClab (Orsay, France). The first Li2${}^{100\textnormal{depl}}$MoO4
sample (LMO-depl-1), corresponding to the upper part of the boule, was mounted
inside a Cu housing using polytetrafluoroethylene (PTFE) supporting elements
and Cu screws, as shown in Figure 1 (Left). The holder design is rather
similar to one used in bolometric measurements with the same-size Li2100MoO4
Armatol et al. (2021b) and slightly larger TeO2 Bergé et al. (2018) crystals.
The second Li2${}^{100\textnormal{depl}}$MoO4 sample (LMO-depl-2),
corresponding to the bottom part of the boule, was assembled using Cu frames
and columns, and PTFE pieces, as seen in Figure 1 (Middle). This sample was a
part of a twelve-crystal array, in which all the other crystals are Li2100MoO4
Alfonso et al. (2023). The lateral side of the LMO-depl-2 crystal was
surrounded by a reflective film (Vikuiti™) to improve the light collection,
while the Cu housing of the LMO-depl-1 detector serves as a reflective cavity,
however the aperture reduces a direct sight of the light detector. Thus, the
light collection is non-optimal both due to a poor reflectivity of copper and
low scintillation photons collection efficiency on the light detector(s).

Figure 1: Photographs of Li2${}^{100\textnormal{depl}}$MoO4 low-temperature
detectors LMO-depl-1 (Left) and LMO-depl-2 (Middle). Both crystals have two
epoxy-glued sensors; the left one is an NTD Ge thermistor, while the right one
is a P-doped Si heater. Each scintillator was accompanied by a circular
bolometric Ge light detector, as one seen in transparent of the crystal on the
middle panel. An additional square-shaped Ge light detector (Right) was used
for the LMO-depl-2 sample. All light detectors were instrumented with an NTD
Ge sensor.

To detect particle interactions, each Li2${}^{100\textnormal{depl}}$MoO4
crystal was instrumented with a Neutron Transmutation Doped Ge Haller (1994)
thermistor (NTD). A sensor with a size of 3$\times$3$\times$1 mm was epoxy
glued onto the crystal’s surface using 6 spots of a bi-component glue
(Araldite® Rapid). The temperature-dependent resistance of NTDs can be
approximated as $R(T)=R_{0}\cdot e^{(T_{0}/T)^{0.5}}$ with the parameters
$R_{0}$ $\sim$ 1 $\Omega$ and $T_{0}$ $\sim$ 3.7 K. The NTD sensors with
similar irradiation parameters have been used in the CUPID-Mo experiment
Armengaud et al. (2020b), and CUPID related R&D tests Armatol et al. (2021a,
b); Alfonso et al. (2022, 2023). Also, a P-doped Si heater Andreotti et al.
(2012) was glued on each crystal with a veil of the epoxy glue. This heating
element is exploited to inject by Jules effect power, which can be used e.g.
for the stabilization of the thermal gain Alessandrello et al. (1998),
optimisation of the detector working point, heating of a device if necessary
(as in CUPID-0 Azzolini et al. (2018)), pile-up simulations Armatol et al.
(2021). To provide electrical contacts, the NTDs and heaters were wire-bonded
using $\oslash$25-$\mu$m Au wires.

To allow for the detection of Li2${}^{100\textnormal{depl}}$MoO4
scintillation, we accompanied crystals with bolometric detectors based on
electronic-grade purity Ge wafers, supplied by Umicore (Belgium) Umi . Two of
them have a circular shape (with a size of $\oslash$45$\times$0.18 mm each),
while the third device is square-shaped (45$\times$45$\times$0.30 mm). All the
Ge disks are coated on both sides with a 70 nm SiO layer aiming at reducing
the light reflection Mancuso et al. (2014); Azzolini et al. (2018); Armengaud
et al. (2020b). Smaller NTD sensors (3$\times$1$\times$1 mm or
3$\times$0.7$\times$1 mm) are attached to the Ge wafers with a veil of the
epoxy glue. The Ge disks are PTFE-clamped in the Cu structure. A single
circular light detector (LD-1-c) has been coupled to the LMO-depl-1 crystal,
while both circular (LD-2-c) and square-shaped (LD-2-s) bolometric
photodetectors viewed the LMO-depl-2 crystal. The mounted Ge light detectors
are shown in Figure 1 (Middle and Right).

###  2.2 Operation at Canfranc underground laboratory

The Li2${}^{100\textnormal{depl}}$MoO4 scintillating bolometers were operated
in the CROSS cryogenic set-up (C2U) Olivieri and CROSS collaboration (2020);
Armatol et al. (2021b) at the Canfranc underground laboratory (LSC, Spain),
providing a substantial reduction of the cosmic muon flux thanks to the rock
overburden Trzaska et al. (2019). The detectors were assembled as parts of
scintillating bolometer arrays and installed inside the cryostat, as
illustrated in Figure 2. The facility exploits the HEXA-DRY dilution fridge by
CryoConcept (France), which is equipped by the Ultra-Quiet Technology™ UQT  to
decouple a pulse tube (Cryomech PT415) from the dilution unit, thus reducing
vibrations Olivieri et al. (2017). To further improve noise conditions, the
detector arrays were spring-suspended from the cold plate of the cryostat. The
detector volume inside the cryostat is shielded on the top by a 13 cm thick
disk made of interleaved lead and copper (partially seen in Figure 2), while
the outer vacuum chamber is surrounded by a 25 cm thick layer of low-
radioactivity lead. In addition, a deradonized air flow around the cryostat
had been supplied during the whole time of the experiment with the LMO-depl-2
detector.

Figure 2: Detector configurations in the C2U cryogenic runs at the LSC, in
which Li2${}^{100\textnormal{depl}}$MoO4 scintillating bolometers LMO-depl-1
(Left) and LMO-depl-2 (Right) were operated. Other scintillating bolometers
are based on Li2MoO4 crystals with natural (CROSS Khalife (2021) and CLYMENE
Velázquez et al. (2017); Stelian et al. (2020) R&D) and 100Mo-enriched (joint
CROSS and CUPID R&D Armatol et al. (2021b); Alfonso et al. (2023)) molybdenum
content, as well as on natural and 116Cd-enriched CdWO4 crystals Olivieri and
CROSS collaboration (2020); Zolotarova and CROSS collaboration (2020); Helis
et al. (2020); Helis (2021).

After reaching the base temperature of the cryostat ($\sim$10 mK), we
regulated the detector plate temperature at 18 and then 12 mK for measurements
with the LMO-depl-1 bolometer, and at 14 mK for the LMO-depl-2 operation. The
control and readout of bolometers was done with the help of a low-noise room-
temperature DC front-end electronics, restyled from the Cuoricino experiment
Arnaboldi et al. (2002). The data acquisition (DAQ) system was composed of two
12-channel boards with integrated 24-bit ADC and a programmable 6-pole Bessel-
Thomson anti-aliasing filter (the cut-off frequency of the low-pass filter was
set at 300 Hz) Carniti et al. (2020, 2023).

In order to find an optimal working point of the detectors, representing the
best signal-to-noise ratio, we spanned the bolometric response with respect to
the current across an NTD Novati et al. (2019). We used heaters to inject
thermal pulses to both Li2${}^{100\textnormal{depl}}$MoO4 bolometers, while
LED generated photons, transmitted from a room-temperature LED (the emission
maximum is at $\sim$880 nm) through an optic fibre, were exploited for the
light detectors. The heater/LED signal injection was performed with the help
of a wave-function generator (Keysight 33500B). As a result of the
optimization, we polarized NTDs of the detectors at a few nA current, reducing
NTD resistances from hundreds M$\Omega$ (at low power) to a few M$\Omega$ (at
the working point).

For each operational temperature we performed measurements with a removable
232Th $\gamma$ source, made of a thoriated tungsten wire, and without the
source (data are referred to calibration and background respectively). Even if
the $\gamma$ source is primarily conceived for calibration of the
Li2${}^{100\textnormal{depl}}$MoO4 bolometers, we also used it to evaluate the
energy scale of the light detectors similar to Bergé et al. (2018); Armengaud
et al. (2020b).

The continuous data of each channel were acquired with a sampling rate of 2
kS/s and stored on a disk for the offline analysis. We processed data with the
help of a MATLAB-based analysis tool Mancuso (2016), which implements the
signal processing using the Gatti-Manfredi optimum filter Gatti and Manfredi
(1986) to maximize the signal-to-noise ratio. In order to apply the filter, we
used data-based information about the signal shape (represented by an average
signal of tens of high-energy events) and the measured noise (represented by
10000 waveforms with no signal). The data were triggered with a threshold
corresponding to 5$\sigma$ of the filtered noise. For each triggered signal,
we collected an information about its amplitudes (i.e. energy) and several
pulse-shape parameters. The results of detectors characterization are
presented in the next section.

##  3 Characterization of Li2${}^{100\textnormal{depl}}$MoO4 scintillating
bolometers

###  3.1 Performance of detectors

At first, we investigated the recorded bolometric signals in terms of time
constants of the signal shape. The rising part of a signal is commonly
characterized by the rise time parameter, which is computed as the time
required by the signal to increase from 10% to 90% of its amplitude. The
descending part is described by the decay time, defined here as the time
required to drop from 90% to 30% of signal amplitude. The rise and decay time
parameters of the operated low-temperature detectors are summarized in Table
1. We find that the Li2${}^{100\textnormal{depl}}$MoO4 bolometers have signals
with the rise time of $\sim$20 ms and the decay time $\sim$100 ms. These time
constants are similar to the values reported for NTD-instrumented low-
temperature detectors based on similar-size Li2MoO4 crystals produced from
molybdenum with the natural isotopic abundance and from enriched in 100Mo
Bekker et al. (2016); Armengaud et al. (2017, 2020b); Armatol et al. (2021a,
b); Alfonso et al. (2023). The Ge light detectors, being gram-scale bolometric
devices equipped with smaller NTDs (i.e. reduced heat capacity compared to the
Li2${}^{100\textnormal{depl}}$MoO4 bolometers), have an order of magnitude
faster response, typical for such devices Beeman et al. (2013); Armengaud et
al. (2017); Azzolini et al. (2018); Armengaud et al. (2020b); Armatol et al.
(2021a, b); Alfonso et al. (2023).

Table 1: Performance of Li2${}^{100\textnormal{depl}}$MoO4 scintillating bolometers and Ge light detectors. We report the detector plate temperature, the NTD resistance, the signal rise and decay time parameters, the detector sensitivity, the baseline noise resolution, and the energy resolution at a given energy.  Bolometer | Tempe- | Resistance | Rise | Decay | Sensitivity |  FWHMNoise | FWHM (keV)
---|---|---|---|---|---|---|---
| rature | of NTD | time | time | (nV/keV) | (keV) | at energy (keV)
| (mK) |  (M$\Omega$) | (ms) | (ms) |  |  |
LMO-depl-1 | 18 | 2.4 | 16 | 112 | 17 | 3.66(3) | 5.9(10) at 1765
| 12 | 6.5 | 20 | 115 | 37 | 2.18(3) | 5.8(3) at 2615
LMO-depl-2 | 14 | 3.0 | 16 | 97 | 29 | 3.80(3) | 6.8(3) at 2615
LD-1-c | 18 | 1.6 | 1.5 | 9.0 | 1200 | 0.097(1) | 0.174(4) at 5.9
| 12 | 2.6 | 1.6 | 10.5 | 1380 | 0.100(1) | 0.146(2) at 5.9
LD-2-s | 14 | 0.47 | 1.6 | 5.2 | 380 | 0.343(5) | 0.94(6) at 17.5
LD-2-c | 14 | 4.4 | 2.0 | 7.8 | 2200 | 0.059(1) | 0.90(6) at 17.5

Figure 3: (Left) Energy spectrum of a 232Th source measured with the
Li2${}^{100\textnormal{depl}}$MoO4 (LMO-depl-1) bolometer, operated at 12 mK
over 125 h. The most intense $\gamma$-ray peaks observed in the spectrum are
labeled by their origin; D.E. and S.E. mean double and single escape peaks,
respectively. A fit to the 2615 keV peak of 208Tl is shown in the inset; the
energy resolution is 5.8(3) keV FWHM. (Right) The energy dependence of the
Li2${}^{100\textnormal{depl}}$MoO4 (LMO-depl-1) bolometer energy resolution in
calibration (red, 125 h) and background (blue, 1109 h) data acquired at 12 mK.
The fit is shown by a dotted line.

Then, using an amplitude distribution of events recorded in the calibration
runs, we calibrated the energy scale of bolometers and evaluated their
sensitivity, expressed as a voltage amplitude per unit of a deposited energy
(e.g. nV/keV), as well as their energy resolution in the limit of zero
amplitude (baseline noise) and at a mono-energetic radiation. In order to
calibrate the Li2${}^{100\textnormal{depl}}$MoO4 bolometers, we used the most
intense $\gamma$ quanta emitted by the 232Th source in the energy interval of
0.2–2.6 MeV, as illustrated in Figure 3 (Left). In the background data we also
relied on the presence of $\gamma$ peaks from environmental radioactivity
(mainly $\gamma$-active daughters of radon, 214Pb and 214Bi, examples are
given below). The sensitivities of the Li2${}^{100\textnormal{depl}}$MoO4
bolometers were measured as $\sim$20–40 nV/keV (see in Table 1); the LMO-
depl-1 signal increases by a factor 2 at a colder temperature of the heat
sink. Taking into account that the chosen working points are characterised by
relatively high NTD currents, the achieved sensitivities are not exceptional
among Li2MoO4-based bolometers (the highest reported values are $\sim$100–150
nV/keV) Bekker et al. (2016); Armengaud et al. (2017, 2020b); Armatol et al.
(2021a, b); Alfonso et al. (2023). Also, the baseline noise is found to be
rather low, $\sim$2–4 keV FWHM (Table 1), and similar to early reported
performance of NTD-instrumented Li2MoO4 bolometers (the best performing
detectors have $\sim$1 keV FWHM noise) Bekker et al. (2016); Armengaud et al.
(2017, 2020b); Armatol et al. (2021a, b); Alfonso et al. (2023). To further
improve the baseline noise (e.g. for dark matter search applications), one can
reduce the absorber’s volume (i.e. heat capacity) and/or use an advanced
performance phonon sensor technology Abdelhameed et al. (2019). Despite of not
extraordinary noise resolution, both Li2${}^{100\textnormal{depl}}$MoO4
bolometers show a comparatively high energy resolution as exampled in Figure 3
(Right). As it is also seen in Table 1, the energy resolution for high-energy
$\gamma$ quanta (1.8 and 2.6 MeV) is only a factor 2–3 worse than the
resolution at zero energy, which is a good feature of Li2MoO4 bolometers
Armengaud et al. (2017). Consequently, the Li2${}^{100\textnormal{depl}}$MoO4
energy resolution at the 2615 keV MeV $\gamma$’s, listed in Table 1 and
illustrated in Figure 3 (Left, Inset), is among the best reported for Li2MoO4
low-temperature detectors Armengaud et al. (2017, 2020b); Armatol et al.
(2021a, b); Alfonso et al. (2022, 2023).

Figure 4: Energy spectra of X-rays from a close 55Fe X-ray source (Left) and
Cu / Mo X-rays induced by the 232Th $\gamma$-ray source (Right), measured by
the LD-1-c (1109 h of background data) and LD-2-c (266 h, calibration)
bolometers respectively. Fits to the spectra with two Gaussians and a linear
background component are shown by solid red lines.

Aiming a permanent calibration during measurements, the LD-1-c was supplied by
an 55Fe X-ray source, which irradiated a Ge side opposite to the LMO-depl-1
crystal. This source emits a doublet of Mn Kα and Kβ X-rays with energies of
5.9 and 6.5 keV, and intensities 25% and 3%, respectively; an example of the
energy spectrum is presented in Figure 4 (Left). To overcome the absence of
permanent X-ray sources in the assembly of the LMO-depl-2 scintillating
bolometer, we irradiated this detector with the 232Th $\gamma$ source to
induce X-ray fluorescence of the materials close to the light detectors, i.e.
in the Cu structure and in the crystal. An illustration of the resulting
spectrum is shown in Figure 4 (Right). So, knowing the energy scale, we
observed good sensitivity of two light detectors (1.2–2.2 $\mu$V/keV), while
the third device had a reduced value (0.4 $\mu$V/keV) due to stronger
polarization of NTD, as exhibited by lower NTD resistance (see Table 1). This
performance is typical for such type of bolometric detectors with NTD
thermistors; a further gain is also feasible by reducing heat capacity of the
sensor / absorber (e.g. see Armengaud et al. (2017) and references therein)
and/or upgrading with Neganov-Trofimov-Luke-effect-based signal amplification
Novati et al. (2019). The less sensitive light detector had comparatively
modest noise resolution (about 300 eV FWHM), while the other two detectors
demonstrated a rather low noise, 60–100 eV FWHM (e.g. see in Poda (2021)). The
resolution of the 6 keV X-ray peak is found to be close to the baseline noise
value, while a more broader 17 keV Mo X-ray peak has been detected by both
light detectors irrespective of 5 times difference in the baseline noise. This
effect can be explained by a position-dependent response of such thin
bolometers Armengaud et al. (2017).

###  3.2 Scintillation detection and particle identification

A combination of both heat and scintillation channels of a scintillating
bolometer can provide particle identification, exploiting the dependence of
the light output on the energy loss mechanism (i.e. particle type) Pirro et
al. (2006); Poda (2021). In order to find coincidences between signals in the
Li2${}^{100\textnormal{depl}}$MoO4 bolometers and in the associated light
detectors, the latter channels were processed using trigger positions of
Li2${}^{100\textnormal{depl}}$MoO4 events and accounting for a difference in
time response (see Table 1), similarly to Piperno et al. (2011). It is
convenient to present such data by normalizing the light detector signal on
the corresponding heat energy release, the so-called light-to-heat parameter
(L/H) in units of keV/MeV. The dependence of the L/H parameter on energy and
type of particles impinged the Li2${}^{100\textnormal{depl}}$MoO4 detectors is
illustrated in Figure 5.

Figure 5: Scintillation (light-to-heat parameter) versus heat energy release
measured by the Li2${}^{100\textnormal{depl}}$MoO4 scintillating bolometers.
Top panel, left shows a sum of calibration (125 h) and background (1109 h)
data of the LMO-depl-1. The LMO-depl-2 events detected in background (top,
right; 1536 h) and calibration (bottom; 111 h) measurements in coincidences
with the bolometric photodetectors LD-2-c and LD-2-s, respectively.

Several populations of events are clearly seen in each data presented in
Figure 5. The most dominant one, distributed mainly below 3 MeV, is originated
by $\gamma$($\beta$) particles. We selected $\gamma(\beta)$’s with energies
above 2 MeV to evaluate the corresponding light-to-heat parameter (L/Hγ(β)),
reported in Table 2. The lowest L/Hγ(β) value ($\sim$0.3 keV/MeV) is obtained
for the LMO-depl-1 and it is expected due to sub-optimal scintillation light
collection conditions (e.g. the aperture between the crystal and the
photodetector, the Cu surrounding instead of a reflective foil, smaller light-
detector area). Indeed, the twin detector (LMO-depl-2), being surrounded by
the reflective foil and coupled to the same-size photodetector, detected about
30% more scintillation energy ($\sim$0.4 keV/MeV), while the square-shaped
light detector allowed almost to double the scintillation signal ($\sim$0.6
keV/MeV). At higher energies, above $\sim$3 MeV, we see populations of events,
which are characterised by scintillation reduced to $\sim$20% compared to
$\gamma$($\beta$)’s, as seen in Table 2. These events are originated by
$\alpha$’s from either U/Th traces of detector bulk / surface contamination or
a U source and to $\alpha$+t particles, products of neutron capture on 6Li.
Despite the different light collection conditions the characteristics of the
Li2100deplMoO4 scintillating bolometers, $L/H_{\gamma(\beta)}$ and
$QF_{\alpha}$ listed in Table 2, are similar to the ones reported for
detectors based on Li2MoO4 crystals produced from molybdenum with the natural
isotopic composition and enriched in 100Mo Armengaud et al. (2017, 2020b);
Poda and CUPID-Mo collaboration (2020); Armatol et al. (2021a, b); Alfonso et
al. (2022).

Table 2: Results on scintillation detection with Li2${}^{100\textnormal{depl}}$MoO4 scintillating bolometers. We report the light-to-heat ratios for $\gamma$($\beta$) events (L/Hγ(β)) and the quenching factors for scintillation induced by $\alpha$’s of 210Po (QFα).  Crystal | Photodetector |  L/Hγ(β) |  QFα
---|---|---|---
|  | (keV/MeV) |
LMO-depl-1 | LD-1-c | 0.33(3) | 0.21(4)
LMO-depl-2 | LD-2-c | 0.44(3) | 0.19(4)
| LD-2-s | 0.59(9) |

###  3.3 Radiopurity of Li2${}^{100\textnormal{depl}}$MoO4 crystals

Thanks to efficient particle identification (Figure 5) and comparatively long
background measurements, we can investigate radiopurity of the
Li2${}^{100\textnormal{depl}}$MoO4 crystals with a high sensitivity to
$\alpha$-active radionuclides of the U/Th decay chains. With this aim, we
selected $\alpha$ particles from the data of both detectors and re-calibrated
spectra to alpha-energy scale for analysis of $\alpha$ contaminants, the
resulting data are shown in Figure 6.

Figure 6: Energy spectra of $\alpha$ events detected by
Li2${}^{100\textnormal{depl}}$MoO4 scintillating bolometers made of crystals
LMO-depl-1 (Left; 1109 h of measurements) and LMO-depl-2 (Right; 1528 h),
operated underground in the C2U facility.

It is seen in Figure 6 that the $\alpha$ spectra of the
Li2${}^{100\textnormal{depl}}$MoO4 crystals are rather similar with the
exception of the energy region below $\sim$4.7 MeV, which is populated by
$\alpha$ particles from a 238U/234U source used in the set-up nearby the LMO-
depl-2 detector. The spectra contain only two peak-like structures, which is a
clear indication of a high internal radiopurity. Moreover, the first peak at
$\sim$4.8 MeV is originated by the detection of products ($\alpha$ plus
triton) of thermal neutron captures by 6Li; this peak is detected by both
bolometers with a similar rate of 1.8(2) counts/day. Furthermore, a doublet of
5.3 and 5.4 MeV peaks is a summed contribution of detector surface and crystal
bulk contaminations by 210Po; in the later case $\alpha$ particle and 206Pb
nuclear recoil (taking away 0.1 MeV energy) were detected. The activity of
210Po in both crystals is the same $\sim$35 $\mu$Bq/kg; it is originated by a
residual 210Pb contamination, typical for scintillators, particularly for Mo-
containing compounds Armengaud et al. (2015, 2017, 2020b); Danevich (2017);
Danevich and Tretyak (2018).

We found no clear evidence of other $\alpha$-active radionuclides from U/Th
chains in the $\alpha$ spectra of the Li2${}^{100\textnormal{depl}}$MoO4
detectors and thus we set upper limits on their activities in the crystal
bulk, using the Feldman-Cousins approach Feldman and Cousins (1998). As a
signal, we took all events in the region $\pm$25 keV around a $Q$-value of a
radionuclide searched for, while background estimate is done in the neighbour
energy interval out of the $Q$-values of U/Th $\alpha$’s, as e.g. detailed in
Armengaud et al. (2015). The results of the study of
Li2${}^{100\textnormal{depl}}$MoO4 crystals radiopurity are summarized in
Table 3. The limits on U/Th activity are obtained on the level of a few
$\mu$Bq/kg; therefore radiopurity of the two
Li2${}^{100\textnormal{depl}}$MoO4 samples is similar to the purity level of
Li2100MoO4 crystals produced for LUMINEU Armengaud et al. (2017); Poda and
LUMINEU, EDELWEISS, CUPID-0/Mo collaborations (2017), CUPID-Mo Armengaud et
al. (2020b); Poda and CUPID-Mo collaboration (2020) and CROSS Armatol et al.
(2021b); Alfonso et al. (2023) experiments, thanks to the same purification
and crystallization protocols applied.

Table 3: Radioactive contamination of the Li2${}^{100\textnormal{depl}}$MoO4 crystals by $\alpha$-active radionuclides from 238U/232Th families (their $Q$-values are listed in keV). The uncertainties are given at 68% C.L., while the limits are set at 90% C.L. Crystal |  Activity ($\mu$Bq/kg)
---|---
|  232Th |  228Th |  238U |  234U |  226Ra |  210Po
| [4082] | [5520] | [4270] | [4858] | [4871] | [5407]
LMO-depl-1 | < 2 | < 2 | < 2 | < 5 | < 7 | 35(6)
LMO-depl-2 |  | < 2 |  |  | < 4 | 36(5)

###  3.4 Background reconstruction capability of
Li2${}^{100\textnormal{depl}}$MoO4 bolometers

As mentioned in Section 1, 100Mo represents a great interest for
double-$\beta$ decay studies. However, the two-$\nu$ double-$\beta$ decay of
100Mo —characterized by a relatively “fast” half-life ($\sim$7$\times$1018 yr
Barabash (2020); Armengaud et al. (2020a))— is an important source of
background in $\nu$-less double-$\beta$ decay search experiments. Indeed, it
generates a 10 mHz rate in a 1 kg 100Mo-enriched lithium molybdate crystal
being a dominant background component in a wide energy interval Armengaud et
al. (2017); Augier et al. (2022, 2023). But even crystals with natural Mo
content have not negligible internal activity of 100Mo ($\sim$1 mBq/kg). The
impact of 100Mo radioactivity can be seen in Figure 7, where the energy
spectra accumulated with 100Mo-enriched/depleted Li2MoO4 bolometers in a
common measurement at the C2U facility are shown. It is worth noting that the
background of the Li2100MoO4 bolometer was spoiled by a $\beta$-component of
the used external $\alpha$ source with a notable activity, around an order of
magnitude higher than that of the 100Mo two-$\nu$ double-$\beta$ decay, as
evident in Figure 7. A clear $\gamma$ background reduction exhibited by the
LMO-depl-2 bolometer, in comparison to the LMO-depl-1 data, was achieved
thanks to using a deradonized air flow around the cryostat shielding. Both
Li2${}^{100\textnormal{depl}}$MoO4 detectors have similar high background
below 0.7 MeV, which is explained by a 210Bi activity induced by the 210Pb
contamination of the lead shield. Also, the residual $\gamma$($\beta$)
activity inside the experimental volume of the set-up, detected by both
Li2${}^{100\textnormal{depl}}$MoO4 bolometers above $\sim$1 MeV is higher than
e.g. in CUPID-Mo Augier et al. (2022, 2023), CUPID-0 Azzolini et al. (2019,
2022), and CUORE Adams et al. (2021, 2022). Thus, the difference between the
100Mo two-$\nu$ double-$\beta$ decay distribution and the background data of
the Li2${}^{100\textnormal{depl}}$MoO4 detectors, acquired in not fully
optimized background conditions, seen in Figure 7 is not remarkable as it
might be in a better shielded set-up.

Figure 7: Energy spectra of $\gamma$($\beta$) events detected by bolometers
based on 0.28 kg lithium molybdate crystals produced from molybdenum either
depleted in 100Mo (pink, LMO-depl-1, and black, LMO-depl-2) or enriched in
100Mo (blue, LMO-enr data from Armatol et al. (2021b)), and operated in the
C2U set-up at the Canfranc underground laboratory. The detectors LMO-depl-1
and LMO-enr were running together (see Figure 2), while the LMO-depl-2
detector was measured in the next cryogenic run, in which the set-up was
flushed with deradonized air. It is worth noting the LMO-enr bolometer was
irradiated by a close 238U/234U $\alpha$ source, which emits $\beta$ particles
too, 234Th ($Q_{\beta}$ = 0.27 MeV) and 234mPa ($Q_{\beta}$ = 2.2 MeV). Thus,
the difference between the acquired spectra is mainly explained by both the
100Mo two-$\nu$ double-$\beta$ activity ($\sim$3 mHz, $Q_{\beta\beta}$ = 3.0
MeV) and the $\alpha$-source-induced $\beta$ background (about 20 mHz of 234Th
and 234mPa) of the Li2100MoO4 detector. A Monte Carlo distribution of the
100Mo two-$\nu$ double-$\beta$ decay events (green) Armengaud et al. (2020a)
is shown for comparison.

It is evident on Figure 7 that the Li2${}^{100\textnormal{depl}}$MoO4
bolometer allows significantly improved reconstruction of $\gamma$ background,
including low-intensity contributions, compared to the Li2100MoO4 detector
having a dominant double-$\beta$ (and $\beta$) decay events continuum.
Therefore, Li2${}^{100\textnormal{depl}}$MoO4 low-temperature detectors with
high spectrometric performance and high radiopurity can provide a
complementary information about a background model in double-$\beta$ decay
searches with Li2100MoO4 bolometers. A similar example is the CUPID-0
experiment with 2 natural and 24 82Se-enriched zinc selenide bolometers
Azzolini et al. (2018)), however the detectors from selenium with the natural
isotopic concentration were not included to the background model analysis
Azzolini et al. (2019). Moreover, a combination of
Li2${}^{100\textnormal{depl}}$MoO4 and Li2100MoO4 bolometers of similar
performance and purity would allow extracting a half-life and a spectral shape
of the 100Mo two-$\nu$ double-$\beta$ decay. A similar approach has been used
in several double-$\beta$ searches, e.g. with 40Ca-/48Ca-enriched calcium
fluoride scintillation detectors der Mateosian and Goldhaber (1966), a 78Kr-
enriched/depleted gas filled proportional counter Gavrilyuk et al. (2013), and
a 136Xe-enriched/depleted xenon gas based time-projection chamber Novella et
al. (2022). However, the enrichment in 48Ca, present in natural calcium at a
$\sim$0.2% level, is not yet available in large quantity as for 100Mo, 78Kr
and 136Xe Giuliani and Poves (2012). Moreover, the very long half-lives of
78Kr (1022 yr) and 136Xe (1021 yr) require a huge exposure to collect a
reasonably high statistics of double-$\beta$ decay events, while more than two
orders of magnitude faster rate of 100Mo is an advantage for such study.

An order of magnitude lower background of the
Li2${}^{100\textnormal{depl}}$MoO4 bolometer at $\sim$0.5 MeV, compared to the
Li2100MoO4 one, would provide a higher sensitivity to a peak expected at 478
keV in case of a resonant absorption of solar axions to the first excited
state of 7Li and its subsequent $\gamma$ deexcitation Belli et al. (2012);
Cardani et al. (2013). At lower energies, the contribution of 100Mo two-$\nu$
double-$\beta$ decay becomes negligible Armatol et al. (2021b), making no
preference on 100Mo content in Li2MoO4 crystals for a search of spin-dependent
dark matter interaction on 7Li. A the same time the presence of 6Li allows
neutron detection (illustrated above in Figures 5 and 6), which can be
exploited for neutron flux monitoring, particularly relevant for dark matter
search applications.

##  4 Conclusions

In the present study we demonstrate that scintillating bolometers made of
lithium molybdate crystals produced from molybdenum depleted in 100Mo
(Li2${}^{100\textnormal{depl}}$MoO4) show high performance, comparable to the
devices based on crystals from molybdenum of the natural isotopic composition
or enriched in 100Mo. Thanks to the strict purification and crystallization
protocols, developed and already applied to high-sensitivity searches for
100Mo $\nu$-less double-$\beta$ decay, radiopurity of
Li2${}^{100\textnormal{depl}}$MoO4 crystals is rather high and compatible to
100Mo-enriched crystals of the same production line. Thus, given an
availability of molybdenum depleted in 100Mo (a by product of industrial
enrichment in 100Mo) for Li2${}^{100\textnormal{depl}}$MoO4 crystals
production, as well as high spectrometric performance, efficient
scintillation-assisted particle identification capability and high material
radiopurity, Li2${}^{100\textnormal{depl}}$MoO4 scintillating bolometers show
a high potential for applications in rare-event search experiments. In
particular such detectors represent a great interest for studies of 100Mo two-
neutrino double-$\beta$ decay, to searches for 7Li axions and spin-depend
interactions of dark matter particles on 7Li, as well as for 6Li-based neutron
spectroscopy and $\gamma$($\beta$) background control measurements in low-
temperature low-background experiments.

“This research received no external funding”

###### Acknowledgements.

This work is supported by the European Commission (Project CROSS, Grant No.
ERC-2016-ADG, ID 742345). This work was also supported by the National
Research Foundation of Ukraine under Grant No. 2020.02/0011 and by the
National Academy of Sciences of Ukraine in the framework of the project
“Development of bolometric experiments for the search for double beta decay”,
the grant number 0121U111684. Y.A.B., V.D.G., E.P.M., and V.N.S. were
supported by the Ministry of Science and Higher Education of the Russian
Federation N121031700314-5. Russian and Ukrainian scientists have given and
give crucial contributions to CROSS. For this reason, the CROSS collaboration
is particularly sensitive to the current situation in Ukraine. The position of
the collaboration leadership on this matter, approved by majority, is
expressed at https://a2c.ijclab.in2p3.fr/en/a2c-home-en/assd-home-en/assd-
cross/. Majority of the work described here was completed before February 24,
2022. “The authors declare no conflict of interest.” no References

## References

  * Pirro and Mauskopf (2017) Pirro, S.; Mauskopf, P.  Advances in Bolometer Technology for Fundamental Physics.  Annu. Rev. Nucl. Part. Sci. 2017, 67, 161.
  * Bellini (2018) Bellini, F.  Potentialities of the future technical improvements in the search of rare nuclear decays by bolometers.  Int. J. Mod. Phys. A 2018, 33, 1843003.
  * Biassoni and Cremonesi (2020) Biassoni, M.; Cremonesi, O.  Search for neutrino-less double beta decay with thermal detectors.  Prog. Part. Nucl. Phys. 2020, 114, 103803.
  * Poda (2021) Poda, D.  Scintillation in Low-Temperature Particle Detectors.  Physics 2021, 3, 473–535.
  * Zolotarova (2021) Zolotarova, A.  Bolometric Double Beta Decay Experiments: Review and Prospects.  Symmetry 2021, 13, 2255.
  * Tretyak and Zdesenko (2002) Tretyak, V.I.; Zdesenko, Y.G.  Tables of double beta decay data — an update.  At. Data Nucl. Data Tables 2002, 80, 83–116.
  * Ejiri et al. (2000) Ejiri, H.; Engel, J.; Hazama, R.; Krastev, P.; Kudomi, N.; Robertson, R.  Spectroscopy of Double-Beta and Inverse-Beta Decays from 100Mo for Neutrinos.  Phys. Rev. Lett. 2000, 85, 2917.
  * Ejiri et al. (2002) Ejiri, H.; Engel, J.; Kudomi, N.  Supernova-neutrino studies with 100Mo.  Phys. Lett. B 2002, 530, 27–32.
  * Ejiri and Elliott (2014) Ejiri, H.; Elliott, S.R.  Charged current neutrino cross section for solar neutrinos, and background to $\beta\beta$($0\nu$) experiments.  Phys. Rev. C 2014, 89, 055501.
  * Ejiri and Elliott (2017) Ejiri, H.; Elliott, S.R.  Solar neutrino interactions with the double-$\beta$ decay nuclei 82Se, 100Mo, and 150Nd.  Phys. Rev. C 2017, 95, 055501.
  * Cardani et al. (2013) Cardani, L.; Casali, N.; Nagorny, S.; Pattavina, L.; Piperno, G.; Barinova, O.; Beeman, J.; Bellini, F.; Danevich, F.; Di Domizio, S.; others.  Development of a Li2MoO4 scintillating bolometer for low background physics.  J. Instrum. 2013, 8, P10002.
  * Bekker et al. (2016) Bekker, T.; Coron, N.; Danevich, F.; Degoda, V.Y.; Giuliani, A.; Grigorieva, V.; Ivannikova, N.; Mancuso, M.; De Marcillac, P.; Moroz, I.; others.  Aboveground test of an advanced Li2MoO4 scintillating bolometer to search for neutrinoless double beta decay of 100Mo.  Astropart. Phys. 2016, 72, 38.
  * Armengaud et al. (2017) Armengaud, E.; Augier, C.; Barabash, A.; Beeman, J.; Bekker, T.; Bellini, F.; Benoît, A.; Bergé, L.; Bergmann, T.; Billard, J.; others.  Development of 100Mo-containing scintillating bolometers for a high-sensitivity neutrinoless double-beta decay search.  Eur. Phys. J. C 2017, 77, 785.
  * Grigorieva et al. (2017) Grigorieva, V.D.; Shlegel, V.; Bekker, T.; Ivannikova, N.; Giuliani, A.; de Marcillac, P.; Marnieros, S.; Novati, V.; Olivieri, E.; Poda, D.; Nones, C.; Zolotarova, A.; Danevich, F.  Li2MoO4 crystals grown by low thermal gradient Czochralski technique.  J. Mat. Sci. Eng. B 2017, 7, 63.
  * Poda and LUMINEU, EDELWEISS, CUPID-0/Mo collaborations (2017) Poda, D.V.; LUMINEU, EDELWEISS, CUPID-0/Mo collaborations.  100Mo-enriched Li2MoO4 scintillating bolometers for $0\nu 2\beta$ decay search: from LUMINEU to CUPID-0/Mo projects.  AIP Conf. Proc. 2017, 1894, 020017.
  * Armengaud et al. (2020a) Armengaud, E.; Augier, C.; Barabash, A.; Bellini, F.; Benato, G.; Benoît, A.; Beretta, M.; Bergé, L.; Billard, J.; Borovlev, Y.A.; others.  Precise measurement of $2\nu\beta\beta$ decay of 100Mo with the CUPID-Mo detection technology.  Eur. Phys. J. C 2020, 80, 674.
  * Armengaud et al. (2020b) Armengaud, E.; Augier, C.; Barabash, A.; Bellini, F.; Benato, G.; Benoît, A.; Beretta, M.; Bergé, L.; Billard, J.; Borovlev, Y.A.; others.  The CUPID-Mo experiment for neutrinoless double-beta decay: performance and prospects.  Eur. Phys. J. C 2020, 80, 44.
  * Armengaud et al. (2021) Armengaud, E.; Augier, C.; Barabash, A.; Bellini, F.; Benato, G.; Benoit, A.; Beretta, M.; Bergé, L.; Billard, J.; Borovlev, Y.A.; others.  New Limit for Neutrinoless Double-Beta Decay of 100Mo from the CUPID-Mo Experiment.  Phys. Rev. Lett. 2021, 126, 181802.
  * Augier et al. (2022) Augier, C.; Barabash, A.; Bellini, F.; Benato, G.; Beretta, M.; Bergé, L.; Billard, J.; Borovlev, Y.; Cardani, L.; Casali, N.; Cazes, A.; others.  Final results on the $0\nu\beta\beta$ decay half-life limit of 100Mo from the CUPID-Mo experiment.  Eur. Phys. J. C 2022, 82, 1033.
  * Augier et al. (2023) Augier, C.; Barabash, A.; Bellini, F.; Benato, G.; Beretta, M.; Bergé, L.; Billard, J.; Borovlev, Y.A.; Cardani, L.; Casali, N.; others.  New measurement of double-$\beta$ decays of 100Mo to excited states of 100Ru with the CUPID-Mo experiment.  Phys. Rev. C 2023, 107, 025503.
  * Armstrong et al. (2019) Armstrong, W.; Chang, C.; Hafidi, K.; Lisovenko, M.; Novosad, V.; Pearson, J.; Polakovic, T.; Wang, G.; Yefremenko, V.; Zhang, J.; others.  CUPID pre-CDR.  arXiv 2019, [arXiv:physics.ins-det/1907.09376].
  * Armatol et al. (2021a) Armatol, A.; Armengaud, E.; Armstrong, W.; Augier, C.; Avignone, F.; Azzolini, O.; Barabash, A.; Bari, G.; Barresi, A.; Baudin, D.; others.  Characterization of cubic Li2100MoO4 crystals for the CUPID experiment.  Eur. Phys. J. C 2021, 81, 104.
  * Armatol et al. (2021b) Armatol, A.; Armengaud, E.; Armstrong, W.; Augier, C.; Avignone III, F.; Azzolini, O.; Bandac, I.; Barabash, A.; Bari, G.; Barresi, A.; others.  A CUPID Li2100MoO4 scintillating bolometer tested in the CROSS underground facility.  JINST 2021, 16, P02037.
  * Alfonso et al. (2022) Alfonso, K.; Armatol, A.; Augier, C.; Avignone III, F.; Azzolini, O.; Balata, M.; Barabash, A.; Bari, G.; Barresi, A.; Baudin, D.; others.  Optimization of the first CUPID detector module.  Eur. Phys. J. C 2022, 82, 810.
  * Alfonso et al. (2023) Alfonso, K.; Armatol, A.; Augier, C.; Avignone III, F.; Azzolini, O.; Balata, M.; Bandac, I.; Barabash, A.; Bari, G.; Barresi, A.; others.  Twelve-crystal prototype of Li2MoO4 scintillating bolometers for CUPID and CROSS experiments.  Submitted to JINST 2023, [arXiv:physics.ins-det/2304.04611].
  * Bandac et al. (2020) Bandac, I.; Barabash, A.; Bergé, L.; Brière, M.; Bourgeois, C.; Carniti, P.; Chapellier, M.; de Combarieu, M.; Dafinei, I.; Danevich, F.; others.  The 0$\nu$2$\beta$-decay CROSS experiment: preliminary results and prospects.  JHEP 2020, 01, 018.
  * Bandac et al. (2021) Bandac, I.; Barabash, A.; Bergé, L.; Bourgeois, C.; Calvo-Mozota, J.; Carniti, P.; Chapellier, M.; deCombarieu, M.; Dafinei, I.; Danevich, F.; others.  Phonon-mediated crystal detectors with metallic film coating capable of rejecting $\alpha$ and $\beta$ events induced by surface radioactivity.  Appl. Phys. Lett. 2021, 118, 184105.
  * Alenkov et al. (2015) Alenkov, V.; Aryal, P.; Beyer, J.; Boiko, R.; Boonin, K.; Buzanov, O.; Chanthima, N.; Chernyak, M.; Choi, J.; Choi, S.; others.  Technical Design Report for the AMoRE $0\nu\beta\beta$ Decay Search Experiment.  arXiv 2015, [arXiv:physics.ins-det/1512.05957].
  * Kim et al. (2020a) Kim, H.; Jeon, J.; Kim, I.; Kim, S.; Kim, H.; Kim, Y.; Kwon, D.; Lee, M.; So, J.  Compact phonon-scintillation detection system for rare event searches at low temperatures.  Nucl. Instrum. Methods Phys. Res. A 2020, 954, 162107.
  * Kim et al. (2020b) Kim, H.; Kim, H.; Kim, I.; Kim, S.R.; Kim, Y.; Kim, Y.H.; Kwon, D.; Jeon, J.A.; Lee, M.H.; Lee, M.; others.  Li2MoO4 Phonon–Scintillation Detection Systems with MMC Readout.  J. Low Temp. Phys. 2020, 199, 1082.
  * Kim et al. (2022a) Kim, W.; Kim, S.; Sharma, B.; Jeon, J.; Kim, H.; Kim, S.; Kim, S.; Kim, Y.; Kim, Y.; Lee, H.; others.  Test Measurements of an MMC-Based 516-g Lithium Molybdate Crystal Detector for the AMoRE-II Experiment.  J. Low Temp. Phys. 2022, 209, 299–307.
  * Kim et al. (2022b) Kim, H.; Ha, D.; Jeon, E.; Jeon, J.; Jo, H.; Kang, C.; Kang, W.; Kim, H.; Kim, S.; Kim, S.; others.  Status and Performance of the AMoRE-I Experiment on Neutrinoless Double Beta Decay.  J. Low Temp. Phys. 2022, 209, 962–970.
  * Bednyakov and Šimkovic (2005) Bednyakov, V.A.; Šimkovic, F.  Nuclear Spin Structure in Dark Matter Search: The Finite Momentum Transfer Limit.  Phys. Part. Nuclei 2005, 37, S106–S128.
  * Abdelhameed et al. (2019) Abdelhameed, A.; Angloher, G.; Bauer, P.; Bento, A.; Bertoldo, E.; Bucci, C.; Canonica, L.; D’Addabbo, A.; Defay, X.; Di Lorenzo, S.; others.  First results on sub-GeV spin-dependent dark matter interactions with 7Li.  Eur. Phys. J. C 2019, 79, 630.
  * Krčmar et al. (2001) Krčmar, M.; Krečak, Z.; Ljubičič, A.; Stipčević, M.; Bradley, D.A.  Search for solar axions using ${}^{7}\mathrm{Li}$.  Phys. Rev. D 2001, 64, 115016.
  * Belli et al. (2008) Belli, P.; Bernabei, R.; Cerulli, R.; Danevich, F.; d’Angelo, A.; Goriletsky, V.; Grinyov, B.; Incicchitti, A.; Kobychev, V.; Laubenstein, M.; others.  7Li solar axions: Preliminary results and feasibility studies.  Nucl. Phys. A 2008, 806, 388–397.
  * Barinova et al. (2010) Barinova, O.; Danevich, F.; Degoda, V.Y.; Kirsanova, S.; Kudovbenko, V.; Pirro, S.; Tretyak, V.  First test of Li2MoO4 crystal as a cryogenic scintillating bolometer.  Nucl. Instrum. Methods Phys. Res. A 2010, 613, 54–57.
  * Belli et al. (2012) Belli, P.; Bernabei, R.; Cappella, F.; Cerulli, R.; Danevich, F.; Incicchitti, A.; Kobychev, V.; Laubenstein, M.; Polischuk, O.; Tretyak, V.; others.  Search for 7Li solar axions using resonant absorption in LiF crystal: Final results.  Phys. Lett. B 2012, 711, 41–45.
  * Martínez et al. (2012) Martínez, M.; Coron, N.; Ginestra, C.; Gironnet, J.; Gressier, V.; Leblanc, J.; de Marcillac, P.; Redon, T.; Di Stefano, P.; Torres, L.; others.  Scintillating bolometers for fast neutron spectroscopy in rare events searches.  J. Phys.: Conf. Ser. 2012, 375, 012025.
  * Coron et al. (2016) Coron, N.; Cuesta, C.; García, E.; Ginestra, C.; Gironnet, J.; de Marcillac, P.; Martínez, M.; Ortigoza, Y.; de Solórzano, A.O.; Puimedón, J.; others.  Neutron Spectrometry With Scintillating Bolometers of LiF and Sapphire.  IEEE Trans. Nucl. Sci. 2016, 63, 1967–1975.
  * Grigorieva et al. (2020) Grigorieva, V.D.; Shlegel, V.; Borovlev, Y.; Bekker, T.; Barabash, A.; Konovalov, S.; Umatov, V.; Borovkov, V.; Meshkov, O.  Li2${}^{100\text{depl}}$MoO4 crystals grown by low-thermal-gradient Czochralski technique.  J. Cryst. Growth 2020, 552, 125913.
  * Bergé et al. (2014) Bergé, L.; Boiko, R.; Chapellier, M.; Chernyak, D.; Coron, N.; Danevich, F.; Decourt, R.; Degoda, V.Y.; Devoyon, L.; Drillien, A.; others.  Purification of molybdenum, growth and characterization of medium volume ZnMoO4 crystals for the LUMINEU program.  JINST 2014, 9, P06004.
  * Bergé et al. (2018) Bergé, L.; Chapellier, M.; De Combarieu, M.; Dumoulin, L.; Giuliani, A.; Gros, M.; De Marcillac, P.; Marnieros, S.; Nones, C.; Novati, V.; others.  Complete event-by-event $\alpha$/$\gamma$($\beta$) separation in a full-size TeO2 CUORE bolometer by Neganov-Luke-magnified light detection.  Phys. Rev. C 2018, 97, 032501.
  * Haller (1994) Haller, E.E.  Advanced far-infrared detectors.  Infrared Phys. Techn. 1994, 35, 127.
  * Andreotti et al. (2012) Andreotti, E.; Brofferio, C.; Foggetta, L.; Giuliani, A.; Margesin, B.; Nones, C.; Pedretti, M.; Rusconi, C.; Salvioni, C.; Tenconi, M.  Production, characterization and selection of the heating elements for the response stabilization of the CUORE bolometers.  Nucl. Instrum. Meth. A 2012, 664, 161.
  * Alessandrello et al. (1998) Alessandrello, A.; Brofferio, C.; Bucci, C.; Cremonesi, O.; Giuliani, A.; Margesin, B.; Nucciotti, A.; Pavan, M.; Pessina, G.; Previtali, E.; others.  Methods for response stabilization in bolometers for rare decays.  J. Cryst. Growth 1998, 412, 454.
  * Azzolini et al. (2018) Azzolini, O.; Barrera, M.; Beeman, J.; Bellini, F.; Beretta, M.; Biassoni, M.; Brofferio, C.; Bucci, C.; Canonica, L.; Capelli, S.; others.  CUPID-0: the first array of enriched scintillating bolometers for $0\nu\beta\beta$ decay investigations.  Eur. Phys. J. C 2018, 78, 428.
  * Armatol et al. (2021) Armatol, A.; Armengaud, E.; Armstrong, W.; Augier, C.; Avignone III, F.; Azzolini, O.; Barabash, A.; Bari, G.; Barresi, A.; Baudin, D.; others.  Novel technique for the study of pileup events in cryogenic bolometers.  Phys. Rev. C 2021, 104, 015501.
  * (49) Umicore Germanium substrates.  https://eom.umicore.com/en/germanium-solutions/products/germanium-substrates/.  Accessed: 2023-04-25.
  * Mancuso et al. (2014) Mancuso, M.; Beeman, J.; Giuliani, A.; Dumoulin, L.; Olivieri, E.; Pessina, G.; Plantevin, O.; Rusconi, C.; Tenconi, M.  An experimental study of antireflective coatings in Ge light detectors for scintillating bolometers.  EPJ Web Conf. 2014, 65, 04003.
  * Olivieri and CROSS collaboration (2020) Olivieri, E.; CROSS collaboration.  The new CROSS Cryogenic Underground (C2U) facility: an overview.  XXIX International (online) Conference on Neutrino Physics and Astrophysics (Neutrino 2020), June 22 – July 02, 2020, 2020.  Poster presented at the XXIX International (online) Conference on Neutrino Physics and Astrophysics (Neutrino 2020), June 22 – July 02, 2020.
  * Trzaska et al. (2019) Trzaska, W.H.; Slupecki, M.; Bandac, I.; Bayo, A.; Bettini, A.; Bezrukov, L.; Enqvist, T.; Fazliakhmetov, A.; Ianni, A.; Inzhechik, L.; others.  Cosmic-ray muon flux at Canfranc Underground Laboratory.  Eur. Phys. J. C 2019, 79, 721.
  * (53) The Ultra-Quiet Technology.  https://cryoconcept.com/product/the-ultra-quiet-technology/.  Accessed: 2023-04-25.
  * Olivieri et al. (2017) Olivieri, E.; Billard, J.; De Jesus, M.; Juillard, A.; Leder, A.  Vibrations on pulse tube based Dry Dilution Refrigerators for low noise measurements.  Nucl. Instrum. Meth. A 2017, 858, 73.
  * Khalife (2021) Khalife, H.  CROSS and CUPID-Mo : future strategies and new results in bolometric search for $0\nu\beta\beta$.  PhD thesis, Université Paris-Saclay, Orsay, France, 2021.
  * Velázquez et al. (2017) Velázquez, M.; Veber, P.; Moutatouia, M.; De Marcillac, P.; Giuliani, A.; Loaiza, P.; Denux, D.; Decourt, R.; El Hafid, H.; Laubenstein, M.; others.  Exploratory growth in the Li2MoO4-MoO3 system for the next crystal generation of heat-scintillation cryogenic bolometers.  Solid State Sci. 2017, 65, 41.
  * Stelian et al. (2020) Stelian, C.; Velazquez, M.; Veber, P.; Ahmine, A.; Duffar, T.; de Marcillac, P.; Giuliani, A.; Poda, D.; Marnieros, S.; Nones, C.; others.  Experimental and numerical investigations of the Czochralski growth of Li2MoO4 crystals for heat-scintillation cryogenic bolometers.  J. Cryst. Growth 2020, 531, 125385.
  * Zolotarova and CROSS collaboration (2020) Zolotarova, A.; CROSS collaboration.  The CROSS experiment: search for 0$\nu$2$\beta$ decay with surface sensitive bolometers.  J. Phys.: Conf. Ser. 2020, 1468, 012147.
  * Helis et al. (2020) Helis, D.L.; Bandac, I.; Barabash, A.; Billard, J.; Chapellier, M.; de Combarieu, M.; Danevich, F.; Dumoulin, L.; Gascon, J.; Giuliani, A.; Kasperovych, D.; others.  Neutrinoless double-beta decay searches with enriched 116CdWO4 scintillating bolometers.  J. Low Temp. Phys. 2020, 199, 467.
  * Helis (2021) Helis, D.  Searching for neutrinoless double-beta decay with scintillating bolometers.  PhD thesis, Université Paris-Saclay, Orsay, France, 2021.
  * Arnaboldi et al. (2002) Arnaboldi, C.; Bucci, C.; Campbell, J.; Capelli, S.; Nucciotti, A.; Pavan, M.; Pessina, G.; Pirro, S.; Previtali, E.; Rosenfeld, C.; others.  The programmable front-end system for CUORICINO, an array of large-mass bolometers.  IEEE Trans. Nucl. Sci. 2002, 49, 2440–2447.
  * Carniti et al. (2020) Carniti, P.; Gotti, C.; Pessina, G.  High-Resolution Digitization System for the CROSS Experiment.  J. Low Temp. Phys. 2020, 199, 833.
  * Carniti et al. (2023) Carniti, P.; Gotti, C.; Pessina, G.  High resolution filtering and digitization system for cryogenic bolometric detectors.  Nucl. Instrum. Meth. A 2023, 1045, 167658.
  * Novati et al. (2019) Novati, V.; Bergé, L.; Dumoulin, L.; Giuliani, A.; Mancuso, M.; de Marcillac, P.; Marnieros, S.; Olivieri, E.; Poda, D.; Tenconi, M.; others.  Charge-to-heat transducers exploiting the Neganov-Trofimov-Luke effect for light detection in rare-event searches.  Nucl. Instrum. Methods Phys. Res. A 2019, 940, 320–327.
  * Mancuso (2016) Mancuso, M.  Development and optimization of scintillating bolometers and innovative light detectors for the search for neutrinoless double beta decay.  PhD thesis, Université Paris-Sud, Orsay, France, 2016.
  * Gatti and Manfredi (1986) Gatti, E.; Manfredi, P.  Processing the signals from solid-state detectors in elementary-particle physics.  Riv. Nuovo Cim. 1986, 9, 1.
  * Beeman et al. (2013) Beeman, J.; Bellini, F.; Casali, N.; Cardani, L.; Dafinei, I.; Di Domizio, S.; Ferroni, F.; Gironi, L.; Nagorny, S.; Orio, F.; others.  Characterization of bolometric Light Detectors for rare event searches.  JINST 2013, 8, P07021.
  * Pirro et al. (2006) Pirro, S.; Beeman, J.; Capelli, S.; Pavan, M.; Previtali, E.; Gorla, P.  Scintillating double beta decay bolometers.  Phys. Atom. Nucl. 2006, 69, 2109.
  * Piperno et al. (2011) Piperno, G.; Pirro, S.; Vignati, M.  Optimizing the energy threshold of light detectors coupled to luminescent bolometers.  JINST 2011, 6, P10005.
  * Poda and CUPID-Mo collaboration (2020) Poda, D.V.; CUPID-Mo collaboration.  Performance of the CUPID-Mo double-beta decay bolometric experiment.  XXIX International (online) Conference on Neutrino Physics and Astrophysics (Neutrino 2020), June 22 – July 02, 2020, 2020.  Poster presented at the XXIX International (online) Conference on Neutrino Physics and Astrophysics (Neutrino 2020), June 22 – July 02, 2020.
  * Armengaud et al. (2015) Armengaud, E.; others.  Development and underground test of radiopure ZnMoO4 scintillating bolometers for the LUMINEU $0\nu 2\beta$ project.  JINST 2015, 10, P05007.
  * Danevich (2017) Danevich, F.A.  Radiopure tungstate and molybdate crystal scintillators for double beta decay experiments.  Int. J. Mod. Phys. A 2017, 32, 1743008.
  * Danevich and Tretyak (2018) Danevich, F.A.; Tretyak, V.I.  Radioactive contamination of scintillators.  Int. J. Mod. Phys. A 2018, 33, 1843007.
  * Feldman and Cousins (1998) Feldman, G.J.; Cousins, R.D.  Unified approach to the classical statistical analysis of small signals.  Phys. Rev. D 1998, 3873, 57.
  * Barabash (2020) Barabash, A.S.  Precise Half-Life Values for Two-Neutrino Double-$\beta$ Decay: 2020 Review.  Universe 2020, 6, 159.
  * Augier et al. (2023) Augier, C.; Barabash, A.; Bellini, F.; Benato, G.; Beretta, M.; Bergé, L.; Billard, J.; Borovlev, Y.; Cardani, L.; Casali, N.; Cazes, A.; others.  The background model of the CUPID-Mo $0\nu\beta\beta$ experiment.  Eur. Phys. J. C (to be submitted) 2023.
  * Azzolini et al. (2019) Azzolini, O.; Beeman, J.; Bellini, F.; Beretta, M.; Biassoni, M.; Brofferio, C.; Bucci, C.; Capelli, S.; Cardani, L.; Carniti, P.; others.  Background model of the CUPID-0 experiment.  Eur. Phys. J. C 2019, 79, 583.
  * Azzolini et al. (2022) Azzolini, O.; Beeman, J.; Bellini, F.; Beretta, M.; Biassoni, M.; Brofferio, C.; Bucci, C.; Capelli, S.; Caracciolo, V.; Cardani, L.; others.  Final Result on the Neutrinoless Double Beta Decay of ${}^{82}\mathrm{Se}$ with CUPID-0.  Phys. Rev. Lett. 2022, 129, 111801.
  * Adams et al. (2021) Adams, D.; Alduino, C.; Alfonso, K.; Avignone III, F.; Azzolini, O.; Bari, G.; Bellini, F.; Benato, G.; Biassoni, M.; Branca, A.; others.  Measurement of the $2\nu\beta\beta$ Decay Half-Life of 130Te with CUORE.  Phys. Rev. Lett. 2021, 126, 171801.
  * Adams et al. (2022) Adams, D.; Alduino, C.; Alfonso, K.; Avignone III, F.; Azzolini, O.; Bari, G.; Bellini, F.; Benato, G.; Beretta, M.; Biassoni, M.; others.  Search for Majorana neutrinos exploiting millikelvin cryogenics with CUORE.  Nature 2022, 604, 53.
  * der Mateosian and Goldhaber (1966) der Mateosian, E.; Goldhaber, M.  Limits for Lepton-Conserving and Lepton-Nonconserving Double Beta Decay in Ca48.  Phys. Rev. 1966, 146, 810–815.
  * Gavrilyuk et al. (2013) Gavrilyuk, Y.M.; Gangapshev, A.M.; Kazalov, V.V.; Kuzminov, V.V.; Panasenko, S.I.; Ratkevich, S.S.  Indications of $2\nu 2K$ capture in 78Kr.  Phys. Rev. C 2013, 87, 035501.
  * Novella et al. (2022) Novella, P.; Sorel, M.; Usón, A.; Adams, C.; Almazán, H.; Álvarez, V.; Aparicio, B.; Aranburu, A.; Arazi, L.; Arnquist, I.; others.  Measurement of the 136Xe two-neutrino double-$\beta$-decay half-life via direct background subtraction in NEXT.  Phys. Rev. C 2022, 105, 055501.
  * Giuliani and Poves (2012) Giuliani, A.; Poves, A.  Neutrinoless Double-Beta Decay.  Adv. High Energy Phys. 2012, 2012, 857016.


:
LEAVE UNSET 2021 LEAVE UNSET LEAVE UNSET Machine Learning for Health (ML4H)
2021

# Adding more data does not always help: A study in medical conversation
summarization with PEGASUS

Varun Nair<EMAIL_ADDRESS>
Namit<EMAIL_ADDRESS>
Xavier Amatriain<EMAIL_ADDRESS>
Ilya Valmianski<EMAIL_ADDRESS>
Anitha Kannan<EMAIL_ADDRESS>
Curai Duke University. Work done while author was a research intern at Curai.

###### Abstract

Medical conversation summarization is integral in capturing information
gathered during interactions between patients and physicians. Summarized
conversations are used to facilitate patient hand-offs between physicians, and
as part of providing care in the future. Summaries, however, can be time-
consuming to produce and require domain expertise. Modern pre-trained NLP
models such as PEGASUS have emerged as capable alternatives to human
summarization, reaching state-of-the-art performance on many summarization
benchmarks. However, many downstream tasks still require at least moderately
sized datasets to achieve satisfactory performance. In this work we (1)
explore the effect of dataset size on transfer learning medical conversation
summarization using PEGASUS and (2) evaluate various iterative labeling
strategies in the low-data regime, following their success in the
classification setting. We find that model performance saturates with increase
in dataset size and that the various active-learning strategies evaluated all
show equivalent performance consistent with simple dataset size increase. We
also find that naive iterative pseudo-labeling is on-par or slightly worse
than no pseudo-labeling. Our work sheds light on the successes and challenges
of translating low-data regime techniques in classification to medical
conversation summarization and helps guides future work in this space.
Relevant code available at https://github.com/curai/curai-
research/tree/main/medical-summarization-ML4H-2021.

##  1 Introduction

Medical conversation summarization can help medical providers to keep a record
of patient encounters and also provide the necessary context of a patient’s
medical history during patient hand-offs between providers. However, creating
these summaries presents a significant clerical load on medical providers,
which can lead to burnout (Shanafelt et al., 2016).

To tackle this problem, we evaluate the use of transfer learning with
abstractive summarization models, such as PEGASUS (Zhang et al., 2019), to
fine tune on medical conversation summarization. Availability of pre-trained
models has led to tremendous progress in multiple other domains (Yadav et al.,
2021; Dai et al., 2021; Xu et al., 2021), with fine-tuning being the main
strategy to derive task specificity. Notably, many tasks still require at
least moderately sized datasets to capture all task-specific regularities. In
the medical domain especially, obtaining labels is resource-intensive and
dependent on domain expertise.

Much of the recent work in label-scarce and low-data regime domains has
focused on either effectively collecting informative labeled data or
leveraging unlabeled data (Mindermann et al., 2021; Du et al., 2020; Chen et
al., 2020). The goal is to label enough informative samples such that the
overall need for expert-labels can be alleviated while still achieving
satisfactory performance. This can be done via active learning, in which an
acquisition function is used to iteratively inform the labeling process and
select the most “useful” points to label. An alternate approach that has
regained popularity is self-training or pseudo-labeling, in which the model’s
predictions are used as ground truth in subsequent re-trained iterations of a
model. In particular, Du et al. (2020) show that self-training with pseudo-
labeling can improve performance on text classification benchmarks without the
need for in-domain unlabeled data.

Given the context of the success of active learning and pseudo-labeling for
low-data regime classification tasks, we are interested in the following:

  * •

Will these strategies translate from the classification setting to the
generative task of medical conversation summarization?

  * •

Is there (empirical) consensus on how we can most effectively utilize our
label budget to maximize summarization performance with these strategies?

In this work, we explore these questions by applying batched active-learning
to the task of medical conversation summarization in the low-data regime,
implemented by leveraging model confidence on unlabeled samples as signal for
expert-labeling and pseudo-labeling. Following our exploration, the
contributions of this work are as follows:

  * •

We find that among the strategies for selecting labels to be expert-labeled,
no one strategy emerges as the best. Furthermore, performance on our
conversation-summary pair dataset saturates as the number of expert labels
increase, while still falling below the theoretical maximum metric values (see
§ 2.3).

  * •

We find that naive pseudo-labeling is on-par or slightly worse than using no
pseudo-labeling, often providing the biggest increase following the first
iteration of self-training.

##  2 Experimental Setup

###  2.1 Approaches Considered

Medical conversation summarization suffers not only from lack of annotated
data but also from a high cost of annotating additional examples (Chintagunta
et al., 2021; Joshi et al., 2020). We investigate111Code at
https://github.com/curai/curai-research/tree/main/medical-summarization-
ML4H-2021 how to collect more labeled examples to improve PEGASUS (Zhang et
al., 2019), a state-of-the-art abstractive summarization model, on the task of
medical conversation summarization while starting with only a small amount of
human labeled data.

Algorithm 1 describes our overall strategy to add labeled data following each
iteration $i$ of self-training. We experiment with two ways to add labeled
points to our training set.

  1. 1.

Expert/Human labeling (HL): Have medical-expert summarized conversations added
to the labeled set, given that PEGASUS is not likely to do well on these
samples.

  2. 2.

Pseudo-labeling (PL): Have model-generated summaries added to the labeled set,
given that PEGASUS is likely to do well on them.

We allow the strategies for pseudo-labeling (PL) and expert-labeling (HL) to
return empty sets. This allows us to experiment with pseudo-labeling and
expert-labeling strategies in isolation as well as in tandem. For both pseudo-
labeling and human labeling, we experiment with a budget of zero and 1% of the
size of $U_{0}$.

Algorithm 1 Iterative labeling strategy

1:

2:labeled set $\mathcal{L}_{0}$

3:unlabeled set $U_{0}$

4:pretrained summarization model PEGASUS $M_{0}$

5:labeling budgets $b_{P}$ for pseudo-labeling

6:labeling budgets $b_{E}$ for expert-labeling,

7:total number of iterations for self-training $n$

8:

9:for $i\leftarrow 1,\cdots,n$ do

10: $M_{i}\leftarrow$ Train $M_{0}$ on $\mathcal{L}_{i-1}$

11: $s_{i}\leftarrow\text{log likelihood of }M_{i}\text{ on }U_{i-1}$

12: $P_{i}\leftarrow\text{Pseudo-label
$\textsf{PL}(s_{i},U_{i-1},b_{P})$}\text{ using }M_{i}$

13: $E_{i}\leftarrow\text{Expert-label $\textsf{HL}(s_{i},U_{i-1},b_{E})$}$

14: $\mathcal{L}_{i}\leftarrow\mathcal{L}_{i-1}+P_{i}+E_{i}$

15: $U_{i}\leftarrow U_{i-1}-P_{i}-E_{i}$

16:return $M_{n}$

We derive our confidence score of how well PEGASUS can generate a summary for
a given data point by using the log-likelihood of the generated summary as a
proxy. The log-likelihood of class predictions has been used in prior work for
classification tasks in a similar manner (Xie et al., 2019; Sohn et al.,
2020).

###  2.2 Experiment Configurations

We examine performance characteristics of PEGASUS fine tuned on datasets
acquired with a large number of experimental configurations. For each
combination described in Table 1, we ran three additional iterations when
either human or pseudo-labeling strategy was not `None`, for a total of 264
experiments (some iteration 0 experiments are equivalent and thus were not
performed, since some conditions only affect later iterations of self-
training). For experiments differentiated only by the dropout value, 0.1
(default) and 0.5 (large regularization), the reported result is the best of
the two.

Condition | Variants
---|---
Dropout | 0.1
| 0.5
Pseudo-labeling strategy (PL) | None
| Top 1%
Human-labeling strategy (HL) | None
| Bottom 1%
| Middle 1%
| Random 1%
Starting number of samples ($|\mathcal{L}_{0}|$) | 100
| 250
| 500
| 750
| 1000
| 1250
Table 1: Overview of experimental conditions as detailed in subsection 2.2.

###  2.3 Training Details and Metrics

We replicate the experimental settings with small modifications222See Appendix
Appendix A for further training details. for PEGASUS from Chintagunta et al.
(2021) by importing the pre-trained PEGASUS model on the CNN/DailyMail
summarization dataset from
HuggingFace.333https://huggingface.co/transformers/model_doc/pegasus.html We
apply these experimental settings uniformly across all iterations of self-
training as well as all baselines plotted on Figure 1.

Performance of our model was measured using the same metrics defined in (Joshi
et al., 2020; Chintagunta et al., 2021) viz. concept F1, affirmation F1 and
rouge-L F1. We also computed the theoretical maximum of these metrics on our
test set. The maximum value for concept F1 and affirmation F1 is computed by
assuming that the predicted summary is the same as the ground truth label,
i.e. we predict both the concepts and their affirmations correctly for each
conversation.444We provide further explanation of these metrics and the
theoretical maximum in Appendix Appendix B.

##  3 Results

Figure 1: (a) Concept, (b) Affirmation, and (c) Rouge F1 scores as a function
of number of training points and sampling strategy. Shown are two trend-lines
and their standard deviations for models trained with and without pseudo-
labeled data (“PL from top 1%” and “No PL”, respectively). The horizontal
lines represent performance for: theoretical maximum, dataset of 6400 human
labels, and combined dataset of 6400 human labels and 25600 GPT-3 derived
labels (Chintagunta et al., 2021). Note: this paper does not study the role of
GPT-3 as a function of training set size, which we leave as future work.

We use a human expert labeled dataset from Chintagunta et al. (2021) to study
the effect of dataset size and iterative labeling strategies (Table 1). We
present two main findings: (1) performance saturation with dataset size (2)
effect of iterative labeling strategies in the low data regime.

###  3.1 Performance saturation with dataset size

Our main finding is the saturation of model performance as the number of
dataset samples increase. An overview of the results can be seen in
fig:saturation.

  1. 1.

Setting $\textsf{PL}=\textsf{HL}=\textsf{None}$ and $n=1$, we find model
performance saturates quickly. Specifically, we find the model with
$|\mathcal{L}_{0}|=1250$ achieves similar performance to
$|\mathcal{L}_{0}|=6400$, with concept F1 scores of 0.41 vs 0.43 respectively.

  2. 2.

We also find the
$(n=2,|\mathcal{L}_{0}|=1000,|\mathcal{L}_{2}|=1159,\textsf{PL}=\textsf{Top
1\%},\textsf{HL}=\textsf{None})$ model beats the
$(n=1,|\mathcal{L}_{0}|=6400,\textsf{PL}=\textsf{HL}=\textsf{None})$ model,
with concept F1 scores 0.44 vs 0.43 respectively.

Furthermore, we compare our results with Chintagunta et al. (2021), with whom
we share the test-set. We find that our best low-data models achieve similar
results to the baselines with significantly more human-labeled points and to
the baseline with both human-labeled and GPT-3 labeled points. Notably,
Chintagunta et al. (2021) saw continued improvement with additional data while
we do not, which we suspect to be due to insufficient optimization of training
hyper-parameters (see 2.3 for a comparison of hyper-parameters). Notably,
however, we did not study the utility of GPT-3 generated labels in a low-data
regime and leave this to future work.

We also observe the saturation behavior by comparing our achieved metrics with
the maximum theoretically achievable values. We find that performance is often
significantly below maximum values (e.g. concept F1 of 0.44 vs 0.63,
affirmation F1 of 0.31 vs 0.45, and rouge F1 of 0.62 vs 1.0 respectively). We
suspect that the early saturation below the maximum theoretical values reflect
the limits of PEGASUS on the medical conversation summarization task. An
interesting future work would include evaluating the saturation behavior of
other pre-trained abstractive summarization models.

###  3.2 Effect of iterative labeling strategies in low data regime

In the low data regime we examined several iterative labeling strategies
consisting of adding additional human and pseudo-labeled examples (see Table
1).

From fig:saturation, we find that for a given number of training points,
$\textsf{PL}=\textsf{None}$ experiments have larger mean value than
$\textsf{PL}=\textsf{Top 1\%}$ experiments for every metric for most values of
$|\mathcal{L}_{0}|$ and $i$. However, these results are only statistically
significant for affirmation F1 scores with $\approx 500$ training points.

Overall, the performance improved with additional human labeled data
consistently with the number of points added. A more detailed view of the
effect of human-labeling can be seen from fig:sampling_strat. For our studies
of up to three additional iterations, we did not find that the performance is
impacted by the choice of human-labeling strategies HL, as Top 1%, Bottom 1%
and Random 1% produced similar results. We did find that in the low data
regime, human labels are necessary. In fig:sampling_strat looking at
$\textsf{HL}=\textsf{None}$ strategy, we find that pseudo-labels are
ineffective at improving test performance when $|\mathcal{L}_{0}|=100$.

##  4 Discussion

We extended active-learning and pseudo-labeling techniques to the generative
setting following recent success in the classification setting. We found that
for PEGASUS, a SOTA summarization model, there is a rapid performance
saturation at 1000 examples when fine tuned for medical conversation
summarization. Beyond a small number of examples, providing additional labels
did not improve performance. Even with few labels, we found that the choice of
active labeling strategy had little effect on the performance, rather, the
number of samples ended up being paramount.

Compared to previous results, we also found performance saturation to occur at
a lower sample count, due to hyper-parameter optimization done for fine-tuning
on a smaller dataset. This provides an important lesson on ensuring hyper-
parameters are well tuned before requesting more data.

Through the lens of the bias-variance trade off, we believe PEGASUS represents
a high bias model and this potentially allows it to generalize across domains.
However, it cannot fully adapt to hyper-specialized regularities, as we
struggled to over-fit to the train set. This explains why PEGASUS quickly
saturates on medical dialogue data and why all sampling strategies are
somewhat equivalent, since low confidence examples contain medical jargon not
present during PEGASUS pre-training and tokenizer initialization.
Understanding how such text generation models fine tune to fields with
significant field-specific jargon presents an important area for future
exploration.

## References

  * Chen et al. (2020) Jiaao Chen, Zichao Yang, and Diyi Yang.  Mixtext: Linguistically-informed interpolation of hidden space for semi-supervised text classification.  _CoRR_ , abs/2004.12239, 2020.  URL https://arxiv.org/abs/2004.12239.
  * Chintagunta et al. (2021) Bharath Chintagunta, Namit Katariya, Xavier Amatriain, and Anitha Kannan.  Medically aware GPT-3 as a data generator for medical dialogue summarization.  In _Proceedings of the Second Workshop on Natural Language Processing for Medical Conversations_ , pages 66–76, Online, June 2021. Association for Computational Linguistics.  10.18653/v1/2021.nlpmc-1.9.  URL https://aclanthology.org/2021.nlpmc-1.9.
  * Dai et al. (2021) Songtai Dai, Quan Wang, Yajuan Lyu, and Yong Zhu.  Bdkg at mediqa 2021: System report for the radiology report summarization task.  In _BIONLP_ , 2021.
  * Du et al. (2020) Jingfei Du, Edouard Grave, Beliz Gunel, Vishrav Chaudhary, Onur Celebi, Michael Auli, Ves Stoyanov, and Alexis Conneau.  Self-training improves pre-training for natural language understanding.  _CoRR_ , abs/2010.02194, 2020.  URL https://arxiv.org/abs/2010.02194.
  * Harkema et al. (2009) Henk Harkema, John N. Dowling, Tyler Thornblade, and Wendy W. Chapman.  Context: An algorithm for determining negation, experiencer, and temporal status from clinical reports.  _Journal of Biomedical Informatics_ , 42(5):839 – 851, 2009.  Biomedical Natural Language Processing.
  * Joshi et al. (2020) Anirudh Joshi, Namit Katariya, Xavier Amatriain, and Anitha Kannan.  Dr. summarize: Global summarization of medical dialogue by exploiting local structures.  In _Empirical Methods in Natural Language Processing (EMNLP) Findings_ , 2020.
  * Lin (2004) Chin-Yew Lin.  ROUGE: A package for automatic evaluation of summaries.  In _Text Summarization Branches Out_ , pages 74–81, Barcelona, Spain, July 2004. Association for Computational Linguistics.  URL https://www.aclweb.org/anthology/W04-1013.
  * Mindermann et al. (2021) Sören Mindermann, Muhammed Razzak, Winnie Xu, Andreas Kirsch, Mrinank Sharma, Adrien Morisot, Aidan N. Gomez, Sebastian Farquhar, Jan Brauner, and Yarin Gal.  Prioritized training on points that are learnable, worth learning, and not yet learned.  2021\.
  * Shanafelt et al. (2016) Tait D. Shanafelt, Lotte N. Dyrbye, Christine Sinsky, Omar Hasan, Daniel Satele, Jeff Sloan, and Colin P. West.  Relationship between clerical burden and characteristics of the electronic environment with physician burnout and professional satisfaction.  _Mayo Clinic Proceedings_ , 91(7):836–848, 2016\.  ISSN 0025-6196.  https://doi.org/10.1016/j.mayocp.2016.05.007.  URL https://www.sciencedirect.com/science/article/pii/S0025619616302154.
  * Sohn et al. (2020) Kihyuk Sohn, David Berthelot, Chun-Liang Li, Zizhao Zhang, Nicholas Carlini, Ekin D. Cubuk, Alex Kurakin, Han Zhang, and Colin Raffel.  Fixmatch: Simplifying semi-supervised learning with consistency and confidence.  _CoRR_ , abs/2001.07685, 2020.  URL https://arxiv.org/abs/2001.07685.
  * Xie et al. (2019) Qizhe Xie, Zihang Dai, Eduard H. Hovy, Minh-Thang Luong, and Quoc V. Le.  Unsupervised data augmentation.  _CoRR_ , abs/1904.12848, 2019.  URL http://arxiv.org/abs/1904.12848.
  * Xu et al. (2021) Liwen Xu, Yan Zhang, Lei Hong, Yi Cai, and Szui Sung.  Chichealth @ mediqa 2021: Exploring the limits of pre-trained seq2seq models for medical summarization.  In _BIONLP_ , 2021.
  * Yadav et al. (2021) Shweta Yadav, Deepak Gupta, Asma Ben Abacha, and Dina Demner-Fushman.  Question-aware transformer models for consumer health question summarization.  _CoRR_ , abs/2106.00219, 2021.  URL https://arxiv.org/abs/2106.00219.
  * Zhang et al. (2019) Jingqing Zhang, Yao Zhao, Mohammad Saleh, and Peter J. Liu.  PEGASUS: pre-training with extracted gap-sentences for abstractive summarization.  _CoRR_ , abs/1912.08777, 2019.  URL http://arxiv.org/abs/1912.08777.

##  Appendix A Training Details

We follow most experimental settings from Chintagunta et al. (2021), but make
modifications to the length of training to 6 epochs and increase the effective
batch size to 128 for most experiments.555Experiments with fewer than 128
samples have an adjusted effective batch size that is 2-4x smaller to allow
for adequate training. These modifications were made for better optimization
in the lower-data regime, compared to the 6400+ samples used in experiments
from Chintagunta et al. (2021). We note again that these experimental settings
are applied uniformly across all iterations of self-training as well as all
baselines plotted on Figure 1.

##  Appendix B Metrics

We measure model performance on standard metrics of ROUGE (Lin, 2004) 666We
use the following package with default configuration:
https://github.com/google-research/google-research/tree/master/rouge as well
as measure a model’s effectiveness in capturing the medical concepts that are
of importance, and their negations Joshi et al. (2020).

Medical Concept Coverage: The concept coverage set of metrics captures the
coverage of medical terms in the model’s output summary with respect to the
ground truth. In particular, let $\mathcal{C}$ be the set of medical concepts
in the reference summary and $\hat{\mathcal{C}}$ be the set of concepts in the
summary output by the model. Then:

$\textrm{Concept
recall}=\frac{\sum_{n=1}^{N}|\hat{\mathcal{C}}^{(n)}\cap\mathcal{C}^{(n)}|}{\sum_{n=1}^{N}|\mathcal{C}^{(n)}|}$
$\textrm{Concept
precision}=\frac{\sum_{n=1}^{N}|\hat{\mathcal{C}}^{(n)}\cap\mathcal{C}^{(n)}|}{\sum_{n=1}^{N}|\hat{\mathcal{C}}^{(n)}|}$

.

We use these to compute a concept F1777Note if there are no concepts detected
in the snippet and summary by the entity extractor, then a conservative F1
score of 0 is given for that example. and use an in-house medical entity
extractor to extract medical concepts in the summary. Medical concepts in the
decoded summary that weren’t present in the original conversation would be
false positives and vice versa for false negatives.

Affirmation Correctness: To measure the effectiveness of the model to identify
the affirmative (alternatively, negated) status of medical concepts, we use
Negex (Harkema et al., 2009) to determine negated concepts. Of the concepts
present in the decoded summary, we evaluate precision and recall on whether
the affirmations and negations were accurate for the decoded concepts and
compute an affirmation F1.${}^{\ref{footnote_f1}}$

Theoretical Maximum Derivation: The maximum score of the concept and
affirmation metrics is not 1.0 because, as in Chintagunta et al. (2021), a
conservative concept as well as affirmation F1 score of 0 is given for an
example if there are no concepts detected in the conversation. As our
affirmation tagger is not perfect, it may also ignore/miss certain concepts
leading to the affirmation F1 limit not being identical to the concept F1
limit.

##  Appendix C Detailed view into sampling strategy experiments

Figure C.1: Concept F1 scores for each starting number of examples broken down
by human-label sampling strategy and labeling iteration number. Strategy
”None” corresponds to no additional human-labeling (only pseudo-labeling). The
maximum value of the axis corresponds to the value achieved by training on all
6400 human labels.