#  _XMM-Newton_ and INTEGRAL observations of the bright GRB 230307A: vanishing
of the local absorption and limits on the dust in the Magellanic Bridge

Sandro Mereghetti Istituto Nazionale di Astrofisica, Istituto di Astrofisica
Spaziale e Fisica Cosmica di Milano, via A. Corti 12, 20133 Milano, Italy
Michela Rigoselli Istituto Nazionale di Astrofisica, Istituto di Astrofisica
Spaziale e Fisica Cosmica di Milano, via A. Corti 12, 20133 Milano, Italy
Ruben Salvaterra Istituto Nazionale di Astrofisica, Istituto di Astrofisica
Spaziale e Fisica Cosmica di Milano, via A. Corti 12, 20133 Milano, Italy
Andrea Tiengo Scuola Universitaria Superiore IUSS Pavia, Piazza della Vittoria
15, 27100 Pavia, Italy  Istituto Nazionale di Astrofisica, Istituto di
Astrofisica Spaziale e Fisica Cosmica di Milano, via A. Corti 12, 20133
Milano, Italy  Dominik P. Pacholski Istituto Nazionale di Astrofisica,
Istituto di Astrofisica Spaziale e Fisica Cosmica di Milano, via A. Corti 12,
20133 Milano, Italy  Dipartimento di Fisica G. Occhialini, Università degli
Studi di Milano Bicocca, Piazza della Scienza 3, 20126 Milano, Italy

(Received 2023 July 25; Revised 2023 August 16; Accepted 2023 September 9)

###### Abstract

GRB 230307A is the second brightest gamma ray burst detected in more than 50
years of observations and is located in the direction of the Magellanic
Bridge. Despite its long duration, it is most likely the result of the compact
merger of a binary ejected from a galaxy in the local universe (redshift
$z=0.065$). Our _XMM-Newton_ observation of its afterglow at 4.5 days shows a
power-law spectrum with photon index $\Gamma=1.73\pm 0.10$, unabsorbed flux
$F_{0.3-10\,\rm keV}=(8.8\pm 0.5)\times 10^{-14}$ erg cm-2 s-1 and no
absorption in excess of that produced in our Galaxy and in the Magellanic
Bridge. We derive a limit of $N_{\rm H}^{\rm HOST}<5\times 10^{20}$ cm-2 on
the absorption at the GRB redshift, which is a factor $\sim\,$5 below the
value measured during the prompt phase. We searched for the presence of dust
scattering rings with negative results and set an upper limit of the order of
$A_{V}<0.05$ on the absorption from dust in the Magellanic Bridge.

Gamma-ray bursts(629) — Interstellar dust(836) — Magellanic Clouds(990)

††facilities: _XMM-Newton_ , INTEGRAL††software: SAS (v19.1.0; Gabriel et al.
2004), HEASoft package (v.6.31; Nasa High Energy Astrophysics Science Archive
Research Center (2014) Heasarc), Xspec (v12.13.0c; Arnaud 1996), xscat
(v1.0.0; Smith et al. 2016), dustmaps (v1.0.4; Green 2018).

##  1 Introduction

Not many years after their discovery it was recognized that gamma-ray bursts
(GRBs) can be divided into two groups, based mostly on their duration and, to
a lesser extent, spectral hardness (Norris et al., 1984; Kouveliotou et al.,
1993). It was later realized that this phenomenological distinction is related
to different formation mechanisms. GRBs of the “long” class have been firmly
associated with type Ib/c core-collapse supernovae of massive stars (e.g.
Galama et al., 1998; Hjorth et al., 2003; Stanek et al., 2003). The
progenitors of the “short” GRBs remained elusive for a longer time, even
though many hints consistently pointed to a compact binary merger progenitor
(e.g. Narayan et al., 1992; Mochkovitch et al., 1993; Nakar, 2007). This was
finally confirmed by the association of the short GRB 170817A with the
gravitational wave signal GW 170817 produced by a binary neutron star merger
(Abbott et al., 2017a, b; Goldstein et al., 2017; Savchenko et al., 2017a).

However, several results obtained in the last few years indicate that a
classification based only on duration does not always reflect the intrinsic
properties related to the GRB origin. For example, the short GRB 090426
resembled long bursts for what concerns its spectrum, energetics, and
afterglow properties (Antonelli et al., 2009; Nicuesa Guelbenzu et al., 2011).
Evidence for a supernova, as usually detected in nearby collapsar-like events,
was found for GRB 040924 and GRB 200826A (Wiersema et al., 2008; Ahumada et
al., 2021; Rossi et al., 2022), which, although not extremely short, had rest-
frame durations below 2 s. On the other hand, deep observations of some long
GRBs at low redshift failed to detect supernovae down to stringent limits
(Fynbo et al., 2006), while kilonova signatures calling for a compact object
merger origin have been associated with bursts of long duration, such as GRB
060614 (Yang et al., 2015) and, more recently, GRB 211221A (Rastinejad et al.,
2022; Troja et al., 2022; Yang et al., 2022). A classification in terms of
Type I/II, respectively for mergers and collapsars, seems thus more convenient
(Zhang, 2006).

Figure 1: Light curve of GRB 230307A at energies above 75 keV obtained with
the SPI/ACS detector. The bin size is 50 ms. The inset shows a zoom of the
initial part.

A recent example of a GRB with long duration ($\sim$150 s), but most likely
originating from a compact binary merger is provided by GRB 230307A (Levan et
al., 2023a; Dichiara et al., 2023; Sun et al., 2023; Wang et al., 2023). The
evidence for a Type I classification comes from the James Webb Space Telescope
discovery of an associated kilonova (Levan et al., 2023a; Yang et al., 2023),
similar to that seen for the gravitational wave merger GW 170817 (Abbott et
al., 2017a). Most likely, the binary merger was ejected from a galaxy seen at
an angular offset of 30′′ from GRB 230307A. Given the galaxy’s redshift of
$z=0.065$, this corresponds to a large, but still plausible, projected
distance of about 40 kpc. The alternative association with a galaxy at
redshift $z=3.87$ (Levan et al., 2023b) is unlikely because it would imply an
unprecedented isotropic energy as large as $10^{56}$ erg and, at such a high
redshift, the observed rapidly varying kilonova-like emission corresponds to
rest-frame rise and decay times too short for a supernova of this luminosity.
Furthermore, the spectroscopic detection of an emission line at 2.15 $\mu$m,
interpreted as due to tellurium, makes the low redshift type I GRB hypothesis
very strong (Levan et al., 2023a).

GRB 230307A immediately attracted much interest owing to its very large
fluence of a few $10^{-3}$ erg cm-2 (Dalessi & Fermi GBM Team, 2023; Dalessi
et al., 2023), surpassed only by that of the exceptional GRB
221009A111Although the occurrence within six months of the two brightest GRBs
ever observed seems at first remarkable, the time distribution of the complete
sample of 17 GRBs with fluence above $10^{-3}$ erg cm-2 detected since 1971
(Table 2 of Burns et al. 2023b) is fully consistent with a constant rate.
(Xiong et al., 2023; Burns et al., 2023a; Svinkin et al., 2023). GRB 230307A
is also interesting because it occurred in the direction of the Magellanic
Bridge, a structure consisting mainly of HI which connects the Small and Large
Magellanic Clouds and was probably produced by the tidal interaction between
these two galaxies (see, e.g., Gaia Collaboration et al., 2021, and references
therein). The occurrence of a bright GRB in this direction offers the
possibility to constrain the poorly known properties of the interstellar dust
in the Bridge through the analysis of X-ray scattered radiation, as done for
Galactic dust clouds with other GRBs (Vaughan et al., 2004; Tiengo &
Mereghetti, 2006; Vaughan et al., 2006; Vianello et al., 2007; Pintore et al.,
2017; Tiengo et al., 2023).

Here we report the results of an _XMM-Newton_ observation of GRB 230307A
carried out about 5 days after the burst occurrence (Section 2.2),
complemented by data of the prompt emission in the hard X-ray range obtained
with the INTEGRAL satellite (Section 2.1). In Section 2.3 we use the X-ray
data to look for the presence of dust scattering rings and to derive
constraints on the amount of dust in the Magellanic Bridge.

##  2 Data analysis and results

###  2.1 Prompt gamma-ray emission with INTEGRAL

GRB 230307A was detected by the Anti-Coincidence Shield (ACS) of the SPI
instrument on board the INTEGRAL satellite. The ACS, besides serving as an
active shield for the germanium detectors of SPI, acts as a nearly omni-
directional detector with high sensitivity at energies above $\sim\,$75 keV
(von Kienlin et al., 2003). It provides data with fixed time resolution of 50
ms in a single energy channel and without directional information. The burst
occurred at an angle of 75∘ with respect to the INTEGRAL pointing direction,
and at an azimuthal angle for which the ACS is not obstructed by other
instruments on board the satellite. Therefore, the ACS provided an optimal
response for the direction of GRB 230307A (see, e.g., Savchenko et al.,
2017b).

The ACS light curve of GRB 230307A is presented in Figure 1. The burst showed
remarkable variability on short time scales, within an envelope with a fast-
rise and a slower decay. The $T_{90}$ duration is of 28.6 s, but significant
emission is visible up to 130 s after $T_{0}=$ 15:44:07 UT. The burst starts
with a $\sim\,$0.2 s long precursor, which contains $\sim\,$0.4% of the total
fluence. This is followed by a multipeaked pulse lasting about 17 s, until a
dip at $\sim T_{0}+18$ s. The temporal decay after $T_{0}+20$ s can be
approximated by a power law with index $\alpha=-3.467\pm 0.007$. The total
fluence, from $T_{0}$ to $T_{0}+130$ s, is $(1.5239\pm 0.0008)\times 10^{7}$
ACS counts, which, adopting the average conversion factor of 1 ACS count $\sim
10^{-10}$ erg cm-2 (Viganò & Mereghetti, 2009), corresponds to $\sim
1.52\times 10^{-3}$ erg cm-2 in the 75–1000 keV range.

###  2.2 X-ray afterglow with _XMM-Newton_

A 50 ks long Target of Opportunity observation of GRB 230307A was carried out
with the _XMM-Newton_ satellite, starting on 2023 March 12 at 02:16:54 UT,
about 4.5 days after the GRB trigger. The EPIC-pn (Strüder et al., 2001) and
the two EPIC-MOS (Turner et al., 2001) cameras were operated in Full window
mode, with the thin optical-blocking filter. After processing the data with
the SAS 19.1.3 (Gabriel et al., 2004) and the most recent calibration files,
we selected the EPIC events with standard filtering expressions and removing
time intervals of high background. This resulted in net exposure times of 37.2
ks (pn) and 41.5 ks (MOS). In addition to the GRB afterglow, several sources
were detected (see Appendix A).

The afterglow spectra were extracted from a circular region of 30′′ radius,
centered at coordinates R.A.$=04^{\rm h}03^{\rm m}26^{\rm s}\\!\\!.24$,
Dec.$=-75^{\circ}22^{\prime}43^{\prime\prime}\\!\\!\\!.\,8$, while those of
the background from a nearby circle of 40′′ radius. The spectra of the three
cameras were rebinned with a minimum of 30 counts per bin and then fitted
simultaneously using the xspec software (version 12.13.0c). For the absorption
model we used cross sections and abundances from Wilms et al. (2000). Errors
on the spectral parameters are given at 1$\sigma$ confidence level.

The spectra are well described ($\chi^{2}/dof=29.14/49$, null hypothesis
probability (nhp) = 0.99) by an absorbed power law with photon index
$\Gamma=1.73\pm 0.10$, column density $N_{\rm H}=(9\pm 2)\times 10^{20}$ cm-2,
and unabsorbed flux $F_{0.3-10\,\rm keV}=(8.8\pm 0.5)\times 10^{-14}$ erg cm-2
s-1 (see Figure 2).

The derived absorption is consistent with the local (Galactic plus Magellanic
Clouds) column density in this direction, $N_{\rm H}^{\rm LOC}=9.4\times
10^{20}$ cm-2 (HI4PI Collaboration et al., 2016). Therefore, in order to
constrain possible absorption in the GRB host, we fixed the column density to
$N_{\rm H}^{\rm LOC}$ and added to the model a redshifted absorption component
($N_{\rm H}^{\rm HOST}$). We found that, at the GRB redshift $z=0.065$, the
host absorption is $N_{\rm H}^{\rm HOST}<5\times 10^{20}$ cm-2 ($2\sigma$
upper limit, see Figure 3).

Figure 2: Top panel: EPIC-pn (black), -MOS1 (red) and -MOS2 (green) spectra of
the afterglow of GRB 230307A fitted with an absorbed power law. The blue
dashed line indicates, for comparison, the spectrum expected with the
intrinsic absorption $N_{\rm H}^{\rm HOST}=2.73\times 10^{21}$ cm-2 measured
in the prompt emission. Bottom panel: residuals of the best fit in units of
$\sigma$. Figure 3: Confidence contours (1, 2 and 3$\sigma$) of photon index
$\Gamma$ and absorption at the GRB redshift $N_{\rm H}^{\rm HOST}$. Figure 4:
Pseudo-distance distribution extracted from EPIC-pn and -MOS events in the
0.5–4 keV energy range. The red line represents the best power-law fit to the
data, while the vertical dashed lines indicate the relevant distance range for
the analysis of interstellar dust in the Magellanic Bridge. Figure 5: X-ray
flux between 0.3–10 keV of GRB 230307A (black histogram: LEIA) and its
afterglow (blue: _Swift_ /XRT; red: _XMM-Newton_ /EPIC; green: _Chandra_
/ACIS-S). The light curve of the afterglow has been fitted with a power law in
the range $T_{0}+[1.3,25.3]$ d (dashed line).

###  2.3 Limits on dust in the Magellanic Bridge

We searched for the presence of dust scattered events using the method based
on “pseudo-distances” (Tiengo & Mereghetti, 2006), which provides a higher
sensitivity for expanding rings compared to a simple analysis of time-
integrated images. Briefly, this method exploits the fact that for a source at
distance much greater than that of the dust, X-ray photons detected at an
angle $\Theta$ from the source direction arrive with a time delay
$t=(D/2c)\Theta^{2}$, where $D$ is the distance of the dust. A pseudo-distance
$D_{i}$ can be computed for each detected count, based on its coordinates
($x_{i}$, $y_{i}$) and arrival time $t_{i}$:

$D_{i}=\frac{2c\,(t_{i}-t_{B})}{(x_{i}-x_{B})^{2}+(y_{i}-y_{B})^{2}}$ (1)

where $x_{B}$, $y_{B}$ and $t_{B}$ are the coordinates and time of the GRB. An
expanding X-ray ring produced by dust at distance $D$ appears as a Lorentzian
peak in the $D_{i}$ distribution.

To derive the $D_{i}$ distribution we selected the _XMM-Newton_ /EPIC events
with energy in the range 0.5–4 keV, flag$\,=\,$0, pattern$\,\leq\,$4 (pn) and
$\leq\,$12 (MOS), and we removed several point sources (see Appendix A) by
excluding circular regions of radius $\sim$ 20′′. The resulting empty regions,
as well as the dead areas between CCD gaps, were filled with uniformly-
distributed events, in order to avoid artifacts and spurious peaks in the
$D_{i}$ distribution.

Most of the Galactic dust in this direction is concentrated in two layers at
distances of 225 pc and 375 pc (Lallement et al., 2022), but, at the time of
our observation, possible scattering rings produced by these layers had
already expanded to angular radii of $\sim 19^{\prime}$ and $\sim
14.5^{\prime}$, outside the EPIC field of view. On the other hand, X-ray
scattered by dust in the Magellanic Bridge can be detected because radii in
the range $\sim 1-2^{\prime}$ are expected in this case (see Figure 7).

As it can be seen in Figure 4, the distribution of the $D_{i}$ values does not
show any prominent peak in the range of relevant distances. In fact it is well
fit by a single power law with slope $-1.98\pm 0.02$ ($\chi^{2}$/dof $=1.05$,
nhp = 0.35), as expected for uniformly distributed background counts and
unscattered X-ray photons.

Figure 6: Broad band fluence spectra for three GRBs with detections or limits
on the dust scattering. The spectra for GRB 230307A (red) are from Sun et al.
(2023). For GRB 221009A (black) we indicate the best estimate (solid) and the
maximum and minimum values (dashed) for the soft X-ray fluence derived from
the analysis of dust scattering rings (Tiengo et al., 2023), while the gamma-
ray spectrum is from Frederiks et al. (2023). For GRB 031203 (blue) we
indicate two estimates of the soft X-ray fluence from dust scattering analysis
by Tiengo & Mereghetti (2006) (lowest) and Watson et al. (2006) (highest),
while the gamma-ray spectrum is from Vianello et al. (2009).

To derive upper limits on the scattered X-ray flux we carried out Monte Carlo
simulations in which we added to the observed $D_{i}$ histogram $N_{s}$
photons distributed according to Lorentzian profiles expected for different
distances of the dust. In principle, the Lorentzian width $\Gamma$ depends on
the thickness $\delta D$ of the dust layer, but, in the case at hand, it is
dominated by the finite angular resolution of the instrument
($\delta\Theta_{\rm FWHM}\sim 5^{\prime\prime}$), which results in
$\Gamma=2D\,|\delta\Theta_{\rm FWHM}/\Theta|$. At the mean time of the
observation (4.7 d after the burst) this gives the value $\Gamma\sim
0.017D_{\rm kpc}^{3/2}$ kpc, which we used in our simulations. These were
carried out for two representative values, 50 and 70 kpc, which bracket
possible distances for dust structures in the Magellanic Bridge (see, e.g.
Jacyszyn-Dobrzeniecka et al., 2016). We found 95% confidence level upper
limits of $N_{s}=245$ EPIC counts for $D=50$ kpc ($\Gamma=6$ kpc) and
$N_{s}=260$ EPIC counts for $D=70$ kpc ($\Gamma=10$ kpc).

##  3 Discussion

The X-ray flux (0.3–10 keV) of our _XMM-Newton_ observation is plotted in
Figure 5, together with those from other available measurements of the X-ray
afterglow. These are the _Swift_ /XRT observations carried out at $T_{0}+1.3$
d (Burrows et al., 2023)222See also
https://www.swift.ac.uk/xrt_products/TILED_GRB00110/ Source2/curve and
https://www.swift.ac.uk/xrt_curves/00021537/ and $T_{0}+4.7$ d, and the
_Chandra_ observation at $T_{0}+25.3$ d (Rouco Escorial et al., 2023). We also
show in Figure 5 the X-ray light curve of the prompt emission measured with
LEIA in the 0.5–4 keV (Sun et al., 2023), opportunely rescaled in the 0.3–10
keV range according to their spectrum (see below). The temporal evolution of
the afterglow is well fitted by a power law of index $\alpha=-1.05\pm 0.13$
and normalization $n=(4.4\pm 0.9)\times 10^{-13}$ erg cm-2 s-1 at $T_{0}+1$ d.

The upper limit we derived at $T_{0}+4.5$ d on the host absorption, $N_{\rm
H}^{\rm HOST}<5\times 10^{20}$ cm-2, is significantly smaller than the value
observed during the prompt emission phase, $N_{\rm H}^{\rm PROMPT}=2.73\times
10^{21}$ cm-2 (Sun et al., 2023). Evidence of X-ray column density variations
has been reported so far only for a handful of GRBs (Starling et al., 2005;
Campana et al., 2007, 2021; Grupe et al., 2007). Indeed, a systematic analysis
of 199 GRBs showed that only seven bursts display signs of a decrease in the
intrinsic $N_{\rm H}$ (Valan et al., 2023), although the observed variation
could also be explained in terms of spectral evolution of the afterglow
emission. Variations in the X-ray absorption are predicted due to the
ionization of the surrounding medium by the GRB radiation (Perna & Loeb, 1998;
Lazzati & Perna, 2002; Perna & Lazzati, 2002). However, it is unlikely that
this scenario applies to the case at hand: at more than 40 kpc from its host
galaxy, GRB 230307A probably exploded in a low density environment, as also
suggested by the relative faintness of its afterglow (Yang et al., 2023; Levan
et al., 2023a). The absorption measured with LEIA during the prompt phase was
most likely caused by very local material, possibly ejected during the last
phases preceding the merger. The ejecta responsible for the absorption might
have moved out of the line of sight by the time of the _XMM-Newton_
observation. Another possibility is that the afterglow was produced at a large
distance, not affected by material close to the central engine. A fully self-
consistent model of mass ejection in short GRBs would be a significant step
forward, but unfortunately this has not been developed yet. In this respect,
more measurements of the X-ray absorption and its evolution during the initial
phases of the burst would be very useful.

We searched for X-ray scattering rings caused by dust in the Magellanic
Bridge, with negative results. Assuming a model for the scattering cross
section and knowing the X-ray fluence and spectrum of GRB 230307A, the upper
limits on the number of scattered photons derived in Section 2.3 can be
translated into limits on the amount of dust. We adopt the analytical dust
model of Draine (2003) and the results obtained with the LEIA experiment (Sun
et al., 2023) on the X-ray prompt emission (a power-law spectrum with photon
index $\Gamma=1.67$, $N_{\rm H}^{\rm LOC}=9.41\times 10^{20}$ cm-2, $N_{\rm
H}^{\rm HOST}=2.73\times 10^{21}$ cm-2 and fluence $F_{\rm 0.5-4\,keV}$ =
$2.27\times 10^{-5}$ erg cm-2). The resulting limits of $A_{V}<0.040-0.056$,
for dust at distances in the 50–70 kpc range, show the potentiality of this
method to constrain the dust properties in the Magellanic Bridge. However, we
note that the results depend on the chosen dust model. As an example, adopting
the BARE-GR-B model of Zubko et al. (2004), which provided the best fit in
other observations of dust scattering X-ray halos/rings (e.g., Smith et al.
2006; Tiengo et al. 2010, 2023), we can limit the quantity of dust in this
part of the Magellanic Bridge to $A_{V}<0.25-0.32$. Even so, these limits are
more constraining than the total reddening $E(B-V)=0.08\pm 0.08$ reported in
the Chen et al. (2022) map, which in this direction has a spatial resolution
of 55′ and estimates the same reddening value also for foreground Galactic
stars.

##  4 Conclusions

We have observed GRB 230307A with _XMM-Newton_ in order to study its X-ray
afterglow about five days after the burst and to search for dust scattering
features, exploiting its very high brightness and fortuitous location close to
the Large Magellanic Cloud.

The good counting statistics and wide energy range of the EPIC spectra allowed
us to reveal a significant decrease in the absorption, which passed from
$2.73\times 10^{21}$ cm-2 in the prompt emission phase to less than $5\times
10^{20}$ cm-2 during the afterglow at $\sim$4.5 days after the burst. This
large variation suggests that tne absorption in the prompt emission was caused
by ejecta very close to the central engine, which moved out of the line of
sight by the time of the _XMM-Newton_ observation.

The study of dust scattered X-rays from GRBs with sensitive telescopes offers
great potentialities. As it is schematically shown in Figure 6, in at least
two cases, GRB 031203A (Watson et al., 2006; Tiengo & Mereghetti, 2006) and
GRB 221009A (Tiengo et al., 2023), these analysis provided evidence for a
bright soft X-ray excess that could not be observed directly. Conversely, the
prompt soft X-ray emission of GRB 230307A was directly observed and we could
exploit the resulting information to set limits on the amount of dust in the
Magellanic Bridge. Although the limits are not very deep in this particular
case, our results indicate that more constraining observations are within
reach of the current instrumentation. For example, an earlier _XMM-Newton_
observation of GRB 230307A, performed when the X-rays potentially scattered by
nearby dust clouds were still within the telescope field of view, would have
allowed to disentangle the effects of dust in our Galaxy and in the Magellanic
Bridge.

##  5 Acknowledgments

The scientific results reported in this article are based on observations
obtained with _XMM-Newton_ and INTEGRAL, ESA science missions with instruments
and contributions directly funded by ESA Member States, NASA and Russia. We
acknowledge financial support from the Italian Ministry for University and
Research, through grants 2017LJ39LM (UNIAM) and INAF Large Program for
Fundamental Research 2022. We thank the _XMM-Newton_ Mission Scientist N.
Schartel for granting the ToO observation and an anonymous referee for his/her
useful comments.

##  Appendix A X-ray sources in the GRB 230307A field

We reduced and filtered the EPIC data using standard procedures as described
in Section 2.2 and created pn+MOS images in three energy bands: 0.5–2 keV,
2–12 keV, and 0.5–12 keV. These images were used to perform source detection
using the SAS tool _edetect_chain_. The resulting list of sources was
inspected visually to remove possible spurious detections and cross-correlated
with optical, infrared and radio catalogs to search for possible
identifications.

The properties of the most interesting sources (i.e. the brightest ones and
those with a plausible identification) are reported in Table 1 and are shown
in Figure 7. The EPIC-pn count rates and fluxes, derived assuming a power-law
model with photon index $\Gamma=1.7$ and a column density $N_{\rm H}^{\rm
LOC}=9.4\times 10^{20}$ cm-2 (HI4PI Collaboration et al., 2016), refer to the
0.5–2 keV range. The X-ray to optical flux ratio has been computed as

$\log(f_{X}/f_{\rm opt})=\log f_{X}+\frac{m}{2.5}+5.37,$ (A1)

where $f_{X}$ is the flux in the 0.5–12 keV range and $m$ is magnitude in the
red band. When no optical counterpart was present in the error region, a
limiting magnitude $m=21$ was assumed.

Eight sources are identified with foreground stars on the basis of their
$f_{X}/f_{\rm opt}$ and positional coincidence with objects of the GAIA and
USNO-B1.0 catalogs. Several sources have been classified as quasars based on
cross-correlation with QSO candidates catalogs (Guo et al., 2018; Liao et al.,
2019).

Figure 7: EPIC-pn image in the 0.3–10 keV energy range of the field of GRB
230307A. The yellow circles show the expected position of scattering rings
produced by dust in the Magellanic Bridge, for distances of 50 kpc (outermost
circle) and 70 kpc (innermost circle). The sources listed in Table 1 are
indicated by the small circles.

Table 1: The brightest or classified detected sources with X-ray and optical parameters Source | R.A. | Dec. | Error | Rate | Flux ($10^{-14}$ | Class | Rmag | X/O ratio | Counterpart Name
---|---|---|---|---|---|---|---|---|---
number |  |  | (arcsec) | ($10^{-3}$ cts s-1) | erg cm-2 s-1) |  | mag | (log10) |
1 | 04:05:10.43 | -75:15:21.14 | 0.3 | $25.80\pm 1.38$ | $3.54^{+0.32^{*}}_{-0.35}$ | QSO | 18.8 | $0.14\pm 0.02$ | USNO-B1.0 0147-0023343
2 | 04:04:42.37 | -75:33:28.00 | 0.4 | $14.31\pm 1.13$ | $2.18^{+0.28^{*}}_{-0.33}$ | Unc. | 20.5 | $0.61\pm 0.03$ | USNO-B1.0 0144-0021053
3 | 04:03:46.24 | -75:32:37.90 | 2.2 | $9.59\pm 1.32$ | $1.99\pm 0.27$ | Unc. | … | $>0.59$ | … …
4 | 04:03:12.50 | -75:26:13.62 | 0.4 | $9.41\pm 0.63$ | $1.96\pm 0.13$ | star | 14.2 | $-2.33\pm 0.04$ | Gaia DR3 4628749515476145792
5 | 04:06:27.49 | -75:26:19.46 | 0.7 | $11.24\pm 1.14$ | $1.78^{+0.27^{*}}_{-0.31}$ | Unc. | 19.6 | $0.21\pm 0.04$ | USNO-B1.0 0145-0021710
6 | 04:06:12.80 | -75:17:58.48 | 0.6 | $7.19\pm 0.88$ | $1.49\pm 0.18$ | QSO | 18.9 | $-0.39\pm 0.06$ | USNO-B1.0 0147-0023428
7 | 04:00:27.96 | -75:23:53.16 | 0.6 | $6.82\pm 0.74$ | $1.42\pm 0.15$ | Unc. | 19.3 | $-0.32\pm 0.06$ | USNO-B1.0 0146-0022065
8 | 04:02:56.71 | -75:26:55.67 | 0.4 | $8.53\pm 0.61$ | $1.25^{+0.14^{*}}_{-0.16}$ | Unc. | 20.7 | $0.31\pm 0.04$ | USNO-B1.0 0145-0021402
9 | 04:00:21.17 | -75:27:13.32 | 0.9 | $5.77\pm 0.92$ | $1.20\pm 0.19$ | star | 15.7 | $-1.88\pm 0.09$ | Gaia DR3 4628795798043612800
10 | 04:02:20.38 | -75:23:27.74 | 0.5 | $5.69\pm 0.50$ | $1.18\pm 0.10$ | Unc. | … | $>0.32$ | … …
11 | 04:03:56.23 | -75:21:25.54 | 0.4 | $8.10\pm 0.59$ | $1.15^{+0.15^{*}}_{-0.17}$ | QSO | 19.5 | $-0.18\pm 0.03$ | USNO-B1.0 0146-0022388
12 | 04:03:24.89 | -75:10:54.67 | 0.9 | $5.09\pm 0.72$ | $1.06\pm 0.15$ | Unc. | 19.2 | $-0.45\pm 0.07$ | USNO-B1.0 0148-0024169
13 | 04:04:08.92 | -75:22:26.51 | 0.6 | $4.85\pm 1.19$ | $1.01\pm 0.25$ | Unc. | … | $>0.22$ | … …
14 | 04:01:34.12 | -75:13:57.52 | 0.6 | $4.80\pm 0.65$ | $1.00\pm 0.13$ | Unc. | 20.1 | $-0.06\pm 0.06$ | Gaia DR3 4628825240041806080
15 | 04:04:23.92 | -75:24:10.33 | 0.5 | $6.52\pm 0.56$ | $0.98^{+0.15^{*}}_{-0.17}$ | QSO | 19.9 | $-0.10\pm 0.04$ | Gaia DR3 4628772532202907008
16 | 04:03:17.55 | -75:15:42.63 | 0.7 | $4.33\pm 0.54$ | $0.90\pm 0.11$ | Unc. | 20.5 | $-0.05\pm 0.06$ | Gaia DR3 4628821460470380928
17 | 04:02:16.49 | -75:21:34.49 | 0.5 | $4.23\pm 0.45$ | $0.88\pm 0.09$ | Unc. | … | $>0.10$ | … …
18 | 04:01:26.12 | -75:12:38.80 | 0.5 | $5.87\pm 0.78$ | $0.88^{+0.19^{*}}_{-0.24}$ | QSO | 19.2 | $-0.27\pm 0.06$ | Gaia DR3 4628825313058849024
19 | 04:03:23.21 | -75:28:12.74 | 0.6 | $6.25\pm 0.80$ | $0.84^{+0.14^{*}}_{-0.17}$ | Unc. | 19.7 | $-0.19\pm 0.06$ | USNO-B1.0 0145-0021446
20 | 04:05:44.22 | -75:29:58.64 | 0.9 | $3.98\pm 0.72$ | $0.83\pm 0.15$ | QSO | 18.2 | $-0.91\pm 0.08$ | Gaia DR3 4628768099796605056
21 | 04:00:45.39 | -75:17:11.12 | 0.9 | $3.66\pm 0.68$ | $0.76\pm 0.14$ | Unc. | … | $>0.11$ | … …
22 | 04:05:49.51 | -75:19:17.97 | 1.1 | $3.58\pm 0.60$ | $0.74\pm 0.12$ | star | 11.0 | $-3.94\pm 0.09$ | Gaia DR3 4628771776288001536
23 | 04:03:55.22 | -75:24:14.16 | 0.5 | $5.45\pm 0.50$ | $0.73^{+0.12^{*}}_{-0.14}$ | Unc. | 20.5 | $0.01\pm 0.04$ | USNO-B1.0 0145-0021502
24 | 04:02:58.65 | -75:33:57.98 | 1.6 | $3.41\pm 0.64$ | $0.71\pm 0.13$ | star | 10.4 | $-4.56\pm 0.19$ | Gaia DR3 4628746251297682816
25 | 04:03:34.46 | -75:28:32.39 | 0.6 | $3.35\pm 0.43$ | $0.70\pm 0.09$ | Unc. | … | $>0.08$ | … …
26 | 04:03:39.39 | -75:26:05.71 | 0.7 | $3.24\pm 0.80$ | $0.67\pm 0.17$ | Unc. | … | $>-0.32$ | … …
27 | 04:04:33.81 | -75:26:58.40 | 0.8 | $3.23\pm 0.43$ | $0.67\pm 0.09$ | Unc. | … | $>0.07$ | … …
28 | 04:01:49.74 | -75:26:22.71 | 0.8 | $3.20\pm 0.43$ | $0.66\pm 0.09$ | QSO | 20.2 | $-0.23\pm 0.06$ | Gaia DR3 4628795179565576832
29 | 04:03:13.81 | -75:31:45.63 | 1.1 | $3.19\pm 0.51$ | $0.66\pm 0.11$ | Unc. | … | $>0.04$ | … …
30 | 04:03:18.83 | -75:31:14.98 | 1.1 | $3.14\pm 0.58$ | $0.65\pm 0.12$ | Unc. | … | $>0.17$ | … …
31 | 04:02:09.33 | -75:28:11.85 | 1.2 | $3.06\pm 1.20$ | $0.64\pm 0.25$ | Unc. | … | $>0.13$ | … …
32 | 04:02:49.05 | -75:24:19.52 | 0.5 | $4.38\pm 0.43$ | $0.63^{+0.10^{*}}_{-0.12}$ | Unc. | … | $>0.14$ | … …
33 | 04:03:46.01 | -75:27:59.01 | 0.8 | $2.98\pm 0.46$ | $0.62\pm 0.10$ | star | 15.9 | $-2.10\pm 0.08$ | USNO-B1.0 0145-0021480
34 | 04:00:42.97 | -75:23:54.06 | 1.0 | $2.93\pm 0.52$ | $0.61\pm 0.11$ | Unc. | … | $>-0.01$ | … …
35 | 04:01:35.81 | -75:30:20.66 | 0.8 | $2.67\pm 0.50$ | $0.55\pm 0.10$ | Unc. | … | $>0.06$ | … …
36 | 04:05:34.19 | -75:23:39.68 | 1.0 | $2.55\pm 0.49$ | $0.53\pm 0.10$ | Unc. | … | $>0.02$ | … …
37 | 04:02:22.90 | -75:21:54.58 | 0.8 | $2.35\pm 0.69$ | $0.49\pm 0.14$ | QSO | 18.0 | $-1.49\pm 0.24$ | USNO-B1.0 0146-0022230
38 | 04:01:52.33 | -75:32:03.49 | 1.6 | $2.26\pm 0.48$ | $0.47\pm 0.10$ | Unc. | … | $>-0.15$ | … …
39 | 04:01:54.32 | -75:21:17.31 | 0.8 | $2.16\pm 0.36$ | $0.45\pm 0.07$ | Unc. | … | $>-0.12$ | … …
40 | 04:01:20.27 | -75:14:59.55 | 1.3 | $2.15\pm 0.48$ | $0.45\pm 0.10$ | Unc. | 19.8 | $-0.58\pm 0.12$ | USNO-B1.0 0147-0022964
41 | 04:00:17.79 | -75:23:58.63 | 1.9 | $2.11\pm 0.50$ | $0.44\pm 0.10$ | star | 12.3 | $-3.67\pm 0.13$ | Gaia DR3 4628796038561772928
42 | 04:03:35.41 | -75:18:00.50 | 0.8 | $2.10\pm 0.35$ | $0.44\pm 0.07$ | Unc. | … | $>-0.32$ | … …
43 | 04:06:10.78 | -75:21:28.39 | 0.9 | $2.08\pm 0.63$ | $0.43\pm 0.13$ | QSO | 20.1 | $-0.41\pm 0.13$ | USNO-B1.0 0146-0022585
44 | 04:04:14.29 | -75:19:28.19 | 1.0 | $2.08\pm 0.59$ | $0.43\pm 0.12$ | Unc. | … | $>-0.25$ | … …
45 | 04:05:43.35 | -75:25:55.43 | 1.5 | $1.92\pm 0.47$ | $0.40\pm 0.10$ | Unc. | … | $>-0.23$ | … …
46 | 04:03:21.87 | -75:24:59.60 | 0.9 | $1.92\pm 0.32$ | $0.40\pm 0.07$ | Unc. | … | $>-0.31$ | … …
47 | 04:02:49.10 | -75:28:45.49 | 1.1 | $1.88\pm 0.34$ | $0.39\pm 0.07$ | Unc. | … | $>-0.19$ | … …
48 | 04:04:01.56 | -75:27:03.37 | 0.8 | $1.75\pm 0.33$ | $0.36\pm 0.07$ | Unc. | … | $>-0.29$ | … …
49 | 04:04:27.08 | -75:32:50.58 | 1.0 | $1.74\pm 0.46$ | $0.36\pm 0.10$ | QSO | 20.0 | $-0.63\pm 0.15$ | Gaia DR3 4628745014347539968
50 | 04:03:48.28 | -75:25:31.90 | 0.9 | $1.62\pm 0.34$ | $0.34\pm 0.07$ | Unc. | … | $>-0.28$ | … …
51 | 04:03:07.43 | -75:27:48.46 | 1.0 | $1.43\pm 0.31$ | $0.30\pm 0.06$ | Unc. | … | $>-0.50$ | … …
52 | 04:04:13.72 | -75:25:15.45 | 1.4 | $1.23\pm 0.29$ | $0.26\pm 0.06$ | star | 18.5 | $-1.46\pm 0.12$ | USNO-B1.0 0145-0021522
53 | 04:04:02.71 | -75:17:25.46 | 1.2 | $1.05\pm 0.29$ | $0.22\pm 0.06$ | star | 19.5 | $-1.05\pm 0.13$ | USNO-B1.0 0147-0023232

## References

  * Abbott et al. (2017a) Abbott, B. P., Abbott, R., Abbott, T. D., et al. 2017a, ApJ, 848, L12, doi: 10.3847/2041-8213/aa91c9
  * Abbott et al. (2017b) —. 2017b, ApJ, 848, L13, doi: 10.3847/2041-8213/aa920c
  * Ahumada et al. (2021) Ahumada, T., Singer, L. P., Anand, S., et al. 2021, Nature Astronomy, 5, 917, doi: 10.1038/s41550-021-01428-7
  * Antonelli et al. (2009) Antonelli, L. A., D’Avanzo, P., Perna, R., et al. 2009, A&A, 507, L45, doi: 10.1051/0004-6361/200913062
  * Arnaud (1996) Arnaud, K. A. 1996, in Astronomical Society of the Pacific Conference Series, Vol. 101, Astronomical Data Analysis Software and Systems V, ed. G. H. Jacoby & J. Barnes, 17
  * Burns et al. (2023a) Burns, E., Goldstein, A., Lesage, S., Dalessi, S., & Fermi-GBM Team. 2023a, GRB Coordinates Network, 33414, 1
  * Burns et al. (2023b) Burns, E., Svinkin, D., Fenimore, E., et al. 2023b, ApJ, 946, L31, doi: 10.3847/2041-8213/acc39c
  * Burrows et al. (2023) Burrows, D. N., Gropp, J. D., Osborne, J. P., et al. 2023, GRB Coordinates Network, 33465, 1
  * Campana et al. (2021) Campana, S., Lazzati, D., Perna, R., Grazia Bernardini, M., & Nava, L. 2021, A&A, 649, A135, doi: 10.1051/0004-6361/202140439
  * Campana et al. (2007) Campana, S., Lazzati, D., Ripamonti, E., et al. 2007, ApJ, 654, L17, doi: 10.1086/510719
  * Chen et al. (2022) Chen, B. Q., Guo, H. L., Gao, J., et al. 2022, MNRAS, 511, 1317, doi: 10.1093/mnras/stac072
  * Dalessi & Fermi GBM Team (2023) Dalessi, S., & Fermi GBM Team. 2023, GRB Coordinates Network, 33407, 1
  * Dalessi et al. (2023) Dalessi, S., Roberts, O. J., Meegan, C., & Fermi GBM Team. 2023, GRB Coordinates Network, 33411, 1
  * Dichiara et al. (2023) Dichiara, S., Tsang, D., Troja, E., et al. 2023, arXiv e-prints, arXiv:2307.02996, doi: 10.48550/arXiv.2307.02996
  * Draine (2003) Draine, B. T. 2003, ApJ, 598, 1026, doi: 10.1086/379123
  * Frederiks et al. (2023) Frederiks, D., Svinkin, D., Lysenko, A. L., et al. 2023, ApJ, 949, L7, doi: 10.3847/2041-8213/acd1eb
  * Fynbo et al. (2006) Fynbo, J. P. U., Watson, D., Thöne, C. C., et al. 2006, Nature, 444, 1047, doi: 10.1038/nature05375
  * Gabriel et al. (2004) Gabriel, C., Denby, M., Fyfe, D. J., et al. 2004, in ASP Conf. Ser. (San Francisco, CA: ASP), Vol. 314, Astronomical Data Analysis Software and Systems (ADASS) XIII, ed. F. Ochsenbein, M. G. Allen, & D. Egret, 759
  * Gaia Collaboration et al. (2021) Gaia Collaboration, Luri, X., Chemin, L., et al. 2021, A&A, 649, A7, doi: 10.1051/0004-6361/202039588
  * Galama et al. (1998) Galama, T. J., Vreeswijk, P. M., van Paradijs, J., et al. 1998, Nature, 395, 670, doi: 10.1038/27150
  * Goldstein et al. (2017) Goldstein, A., Veres, P., Burns, E., et al. 2017, ApJ, 848, L14, doi: 10.3847/2041-8213/aa8f41
  * Green (2018) Green, G. 2018, The Journal of Open Source Software, 3, 695, doi: 10.21105/joss.00695
  * Grupe et al. (2007) Grupe, D., Gronwall, C., Wang, X.-Y., et al. 2007, ApJ, 662, 443, doi: 10.1086/517868
  * Guo et al. (2018) Guo, S., Qi, Z., Liao, S., et al. 2018, A&A, 618, A144, doi: 10.1051/0004-6361/201833135
  * HI4PI Collaboration et al. (2016) HI4PI Collaboration, Ben Bekhti, N., Flöer, L., et al. 2016, A&A, 594, A116, doi: 10.1051/0004-6361/201629178
  * Hjorth et al. (2003) Hjorth, J., Sollerman, J., Møller, P., et al. 2003, Nature, 423, 847, doi: 10.1038/nature01750
  * Jacyszyn-Dobrzeniecka et al. (2016) Jacyszyn-Dobrzeniecka, A. M., Skowron, D. M., Mróz, P., et al. 2016, Acta Astron., 66, 149, doi: 10.48550/arXiv.1602.09141
  * Kouveliotou et al. (1993) Kouveliotou, C., Meegan, C. A., Fishman, G. J., et al. 1993, ApJ, 413, L101, doi: 10.1086/186969
  * Lallement et al. (2022) Lallement, R., Vergely, J. L., Babusiaux, C., & Cox, N. L. J. 2022, A&A, 661, A147, doi: 10.1051/0004-6361/202142846
  * Lazzati & Perna (2002) Lazzati, D., & Perna, R. 2002, MNRAS, 330, 383, doi: 10.1046/j.1365-8711.2002.05064.x
  * Levan et al. (2023a) Levan, A., Gompertz, B. P., Salafia, O. S., et al. 2023a, arXiv e-prints, arXiv:2307.02098, doi: 10.48550/arXiv.2307.02098
  * Levan et al. (2023b) Levan, A. J., Watson, D., Hjorth, J., et al. 2023b, GRB Coordinates Network, 33580, 1
  * Liao et al. (2019) Liao, S.-L., Qi, Z.-X., Guo, S.-F., & Cao, Z.-H. 2019, Research in Astronomy and Astrophysics, 19, 029, doi: 10.1088/1674-4527/19/2/29
  * Mochkovitch et al. (1993) Mochkovitch, R., Hernanz, M., Isern, J., & Martin, X. 1993, Nature, 361, 236, doi: 10.1038/361236a0
  * Nakar (2007) Nakar, E. 2007, Phys. Rep., 442, 166, doi: 10.1016/j.physrep.2007.02.005
  * Narayan et al. (1992) Narayan, R., Paczynski, B., & Piran, T. 1992, ApJ, 395, L83, doi: 10.1086/186493
  * Nasa High Energy Astrophysics Science Archive Research Center (2014) (Heasarc) Nasa High Energy Astrophysics Science Archive Research Center (Heasarc). 2014, HEAsoft: Unified Release of FTOOLS and XANADU, Astrophysics Source Code Library, record ascl:1408.004.  http://ascl.net/1408.004
  * Nicuesa Guelbenzu et al. (2011) Nicuesa Guelbenzu, A., Klose, S., Rossi, A., et al. 2011, A&A, 531, L6, doi: 10.1051/0004-6361/201116657
  * Norris et al. (1984) Norris, J. P., Cline, T. L., Desai, U. D., & Teegarden, B. J. 1984, Nature, 308, 434, doi: 10.1038/308434a0
  * Perna & Lazzati (2002) Perna, R., & Lazzati, D. 2002, ApJ, 580, 261, doi: 10.1086/343081
  * Perna & Loeb (1998) Perna, R., & Loeb, A. 1998, ApJ, 501, 467, doi: 10.1086/305865
  * Pintore et al. (2017) Pintore, F., Tiengo, A., Mereghetti, S., et al. 2017, MNRAS, 472, 1465, doi: 10.1093/mnras/stx2062
  * Rastinejad et al. (2022) Rastinejad, J. C., Gompertz, B. P., Levan, A. J., et al. 2022, Nature, 612, 223, doi: 10.1038/s41586-022-05390-w
  * Rossi et al. (2022) Rossi, A., Rothberg, B., Palazzi, E., et al. 2022, ApJ, 932, 1, doi: 10.3847/1538-4357/ac60a2
  * Rouco Escorial et al. (2023) Rouco Escorial, A., Gompertz, B., Fong, W., et al. 2023, GRB Coordinates Network, 33558, 1
  * Savchenko et al. (2017a) Savchenko, V., Ferrigno, C., Kuulkers, E., et al. 2017a, ApJ, 848, L15, doi: 10.3847/2041-8213/aa8f94
  * Savchenko et al. (2017b) Savchenko, V., Bazzano, A., Bozzo, E., et al. 2017b, A&A, 603, A46, doi: 10.1051/0004-6361/201730572
  * Smith et al. (2006) Smith, R. K., Dame, T. M., Costantini, E., & Predehl, P. 2006, ApJ, 648, 452, doi: 10.1086/505650
  * Smith et al. (2016) Smith, R. K., Valencic, L. A., & Corrales, L. 2016, ApJ, 818, 143, doi: 10.3847/0004-637X/818/2/143
  * Stanek et al. (2003) Stanek, K. Z., Matheson, T., Garnavich, P. M., et al. 2003, ApJ, 591, L17, doi: 10.1086/376976
  * Starling et al. (2005) Starling, R. L. C., Vreeswijk, P. M., Ellison, S. L., et al. 2005, A&A, 442, L21, doi: 10.1051/0004-6361:200500181
  * Strüder et al. (2001) Strüder, L., Briel, U., Dennerl, K., et al. 2001, A&A, 365, L18, doi: 10.1051/0004-6361:20000066
  * Sun et al. (2023) Sun, H., Wang, C. W., Yang, J., et al. 2023, arXiv e-prints, arXiv:2307.05689, doi: 10.48550/arXiv.2307.05689
  * Svinkin et al. (2023) Svinkin, D., Frederiks, D., Ulanov, M., et al. 2023, GRB Coordinates Network, 33427, 1
  * Tiengo & Mereghetti (2006) Tiengo, A., & Mereghetti, S. 2006, A&A, 449, 203, doi: 10.1051/0004-6361:20054162
  * Tiengo et al. (2010) Tiengo, A., Vianello, G., Esposito, P., et al. 2010, ApJ, 710, 227, doi: 10.1088/0004-637X/710/1/227
  * Tiengo et al. (2023) Tiengo, A., Pintore, F., Vaia, B., et al. 2023, ApJ, 946, L30, doi: 10.3847/2041-8213/acc1dc
  * Troja et al. (2022) Troja, E., Fryer, C. L., O’Connor, B., et al. 2022, Nature, 612, 228, doi: 10.1038/s41586-022-05327-3
  * Turner et al. (2001) Turner, M. J. L., Abbey, A., Arnaud, M., et al. 2001, A&A, 365, L27
  * Valan et al. (2023) Valan, V., Larsson, J., & Ahlgren, B. 2023, ApJ, 944, 73, doi: 10.3847/1538-4357/acafe4
  * Vaughan et al. (2004) Vaughan, S., Willingale, R., O’Brien, P. T., et al. 2004, ApJ, 603, L5, doi: 10.1086/382785
  * Vaughan et al. (2006) Vaughan, S., Willingale, R., Romano, P., et al. 2006, ApJ, 639, 323, doi: 10.1086/499353
  * Vianello et al. (2009) Vianello, G., Götz, D., & Mereghetti, S. 2009, A&A, 495, 1005, doi: 10.1051/0004-6361:200811251
  * Vianello et al. (2007) Vianello, G., Tiengo, A., & Mereghetti, S. 2007, A&A, 473, 423, doi: 10.1051/0004-6361:20077968
  * Viganò & Mereghetti (2009) Viganò, D., & Mereghetti, S. 2009, in The Extreme Sky: Sampling the Universe above 10 keV, 49, doi: 10.22323/1.096.0049
  * von Kienlin et al. (2003) von Kienlin, A., Beckmann, V., Rau, A., et al. 2003, A&A, 411, L299, doi: 10.1051/0004-6361:20031231
  * Wang et al. (2023) Wang, Y., Xia, Z.-Q., Zheng, T.-C., Ren, J., & Fan, Y.-Z. 2023, arXiv e-prints, arXiv:2303.11083, doi: 10.48550/arXiv.2303.11083
  * Watson et al. (2006) Watson, D., Vaughan, S. A., Willingale, R., et al. 2006, ApJ, 636, 967, doi: 10.1086/498089
  * Wiersema et al. (2008) Wiersema, K., van der Horst, A. J., Kann, D. A., et al. 2008, A&A, 481, 319, doi: 10.1051/0004-6361:20078050
  * Wilms et al. (2000) Wilms, J., Allen, A., & McCray, R. 2000, ApJ, 542, 914, doi: 10.1086/317016
  * Xiong et al. (2023) Xiong, S., Wang, C., Huang, Y., & Gecam Team. 2023, GRB Coordinates Network, 33406, 1
  * Yang et al. (2015) Yang, B., Jin, Z.-P., Li, X., et al. 2015, Nature Communications, 6, 7323, doi: 10.1038/ncomms8323
  * Yang et al. (2022) Yang, J., Ai, S., Zhang, B.-B., et al. 2022, Nature, 612, 232, doi: 10.1038/s41586-022-05403-8
  * Yang et al. (2023) Yang, Y.-H., Troja, E., O’Connor, B., et al. 2023, arXiv e-prints, arXiv:2308.00638, doi: 10.48550/arXiv.2308.00638
  * Zhang (2006) Zhang, B. 2006, Nature, 444, 1010, doi: 10.1038/4441010a
  * Zubko et al. (2004) Zubko, V., Dwek, E., & Arendt, R. G. 2004, ApJS, 152, 211, doi: 10.1086/382351


# A BERT-based Distractor Generation Scheme with Multi-tasking and Negative
Answer Training Strategies

Ho-Lam Chung1, Ying-Hong Chan2, Yao-Chung Fan3
Department of Computer Science and Engineering
National Chung Hsing University,
Taichung, Taiwan
<EMAIL_ADDRESS>
<EMAIL_ADDRESS>
<EMAIL_ADDRESS>

###### Abstract

In this paper, we investigate the following two limitations for the existing
distractor generation (DG) methods. First, the quality of the existing DG
methods are still far from practical use. There are still room for DG quality
improvement. Second, the existing DG designs are mainly for single distractor
generation. However, for practical MCQ preparation, multiple distractors are
desired. Aiming at these goals, in this paper, we present a new distractor
generation scheme with multi-tasking and negative answer training strategies
for effectively generating multiple distractors. The experimental results show
that (1) our model advances the state-of-the-art result from 28.65 to 39.81
(BLEU 1 score) and (2) the generated multiple distractors are diverse and
shows strong distracting power for multiple choice question.

##  1 Introduction

Given a passage, a question, and an answer phrase, the goal of distractor
generation (DG) is to generate context-related wrong options (i.e.,
distractor) for multiple-choice questions (MCQ). Pioneering research Gao et
al. (2019); Yeung et al. (2019); Zhou et al. (2019) have demonstrated the
feasibility of generating distractors based on deep learning techniques.

Example 1
---
Context Omitted. (See Appendix)
Question
$\cdot$ Why did Mr.King want to send Henry away?
Answer
|  $\cdot$ Because Henry was too lazy.
---
Gen. Distractors
$\cdot d_{1}:$ Because Henry didn’t want to go.
$\cdot d_{2}:$ Because Henry didn’t want to go to the bookstore.
Example 2
Context Omitted. (See Appendix)
Question
$\cdot$ Which of the following women would look most attractive?
Answer
$\cdot$ A short red-haired woman who wears a purple hat.
Gen. Distractors
$\cdot d_{1}:$ A young woman who wears a white hat.
|  $\cdot d_{2}:$ A woman who wears a white hat.
---

Table 1: Two examples for showing the issue of generating multiple distractors
by a simple beam search: Note that the generated distractors (i.e., $d_{1}$
and $d_{2}$) are the same statements with only slight word usage difference.
Such results lower the distracting power for MCQ preparation.

While significant advances for DG were reported in the literature, we find
that the existing DG results are still far from practical use. In this paper,
we investigate the following two issues for distractor generation: (1) DG
quality improvement and (2) Multiple distractor generation.

DG Quality Improvement There is still room to be improved for high-quality
distractor generation. By manually examining the DG results generated by the
existing method, we find that the results are still far from ideal for
practical use. Thus, one goal of our research is to improve the DG quality
further.

For the quality issues, in this paper, we explore BERT model’s employment for
performance improvement. As known, employing transformer-based language models
has shown to be useful for improving NLP tasks. Thus, we investigate the BERT
model’s application for DG and report our design in this paper.

Multiple Distractor Generation The existing DG methods mainly focus on single
distractor generation. However, for practical MCQ preparation, multiple
distractors are desired. For more than one distractor, the existing practice
is to keep multiple results given by a beam search strategy. However, we find
that in many cases, the generated distractors are all referred to the same
concept/thing. In fact, the generated distractors are all from the same latent
representation, which brings concerns that they might be semantically similar.
In Table 1, we show two DG examples for this problems. In the illustrated
examples, one can observe that the generated distractors are the same
statements with only a slight word usage difference. Such results lower the
distracting power for MCQ preparation.

For this limitation, we propose to view multiple distractor
generation/selection problems as a coverage problem, rather than individually
selecting top-k distractors based on prediction probability. In other words,
we propose to choose a distractor set, which maximizes the difficulty of
multiple-choice questions, rather than individually picking results with the
highest probability but with similar semantic.

The contributions of this paper are (1) a new DG model based on the BERT model
employment. The experiment evaluation with benchmarking datasets shows that
our model outperforms the existing best models Zhou et al. (2019) and pushes
the state-of-the-art result from 28.65 to 39.81 (BLEU 1 score). (2) An
investigation to employ the use of multiple-choice question answering task to
evaluate the DG performance. (3) An investigation for considering the multiple
distractors generation problem as a coverage problem. The experiment result
demonstrates that the generated multiple distractors are diverse and show
strong distracting power for multiple-choice questions.

The rest of this paper is organized as follows. In Section 2, we introduce our
model design for a single distractor generation. In Section 3, we introduce to
our multiple distractor schemes and the incorporation of the question-answer
models for distractor selection. In Section 4, we report the result of
performance analysis. In Section 5, we review the literature related to this
work. Finally, Section 6 concludes our study and discusses future works.

##  2 BERT Distractor Generation

###  2.1 BERT Model Review

The BERT model and its family Liu et al. (2019); Lan et al. (2019) are
composed of a stack of multi-layer bidirectional Transformer encoders. The
input to a BERT model is a sequence of tokens. For a given token, its input
representation to the BERT model is first constructed by summing the
corresponding token, segment, and position embeddings. After the input
representation, the input embeddings travel through the pre-trained/fine-tuned
BERT for task learning and prediction. In general, BERT can be employed in
two-level language modeling tasks: sequence-level classification and token-
level prediction tasks. For the tasks, there are three special tokens, [C],
[S], and [M]. The embedding of the [C] token is designed to be used as the
aggregate sequence representation for classification tasks. The [S] is
designed to distinguish different sentences of a token sequence (to
provide/signal information from multiple sentences, as the input token
sequence can be a pack of multiple sentences). On the other hand, the [M]
token is designed to be used in token-level prediction (e.g., predicting a
masked token based on context words or predicting the starting/ending
probabilities for span-based tasks such as QA tasks).

As reported in Chan and Fan (2019); Dong et al. (2019), BERT essentially is an
auto-encoder language modeling design, which aims to reconstruct the original
data from corrupted inputs. If BERT is asked to predict a sequence of
consecutive masked tokens, it often produces incoherent and ramble results.
For example, when using BERT to predict three consecutive [M][M][M] masked
tokens, the same prediction result for the tokens are often observed. This is
because the context (the information for predicting the tokens) for the masked
tokens are nearly the same except for the position embedding, making the
generated sentences incoherent. Thus, we take into consideration the previous
decoded results for decoding the next distractor token, as will be introduced
in the next subsection.

Iter. | Input Sequence | Predict
---|---|---
1 | [C] $C$ [S][M] | Because
2 |  [C] $C$ [S] Because [M] | Henry
3 |  [C] $C$ [S] Because Henry [M] | didn’t
4 |  [C] $C$ [S] Because Henry didn’t [M] | want
5 |  [C] $C$ [S] Because Henry didn’t want [M] | to
6 |  [C] $C$ [S] Because Henry didn’t want to [M] | go
7 |  [C] $C$ [S] Because Henry didn’t want to go [M] | .
8 |  [C] $C$ [S] Because Henry didn’t want to go.[M] | [S]
Table 2: A Running Example for the BDG scheme

###  2.2 BERT-based Distractor Generation (BDG)

In a distractor generation scenario, there are three given inputs: (1) a
paragraph $P$, (2) an answer $A$, and (3) a question $Q$. For ease of
discussion, let $C$ (referred to as a context sequence) denote the sequence of
tokens given by concatenating $P$, $Q$, and $A$.

Our BDG model generates distractor tokens in an auto-regressive manner.
Specifically, the BDG model predicts a token at a time based on (1) the given
context sequence $C$ and (2) the previously predicted distractor tokens. The
BDG model takes multiple iterations to generate a distractor. In Table 2, we
show a running example of the BDG model. Note that our model predicts a token
based on $C$ and the previously generated tokens at each iteration. For
example, at Iteration 1, we generate ”Because” based on $C$. At Iteration 2,
we generate ”Henry” based on $C$ and ”Because” tokens, and Iteration 3, we
generate ”didn’t” based on $C$, ”Because”, and ”Henry”. The generation
terminates when [S] is predicted. In this example, ”Because Henry didn’t want
to go.” is the final generated result.

Specifically, the input sequence $X_{i}$ at Iteration $i$ to BERT is

$\begin{split}X_{i}=(\mathrm{\texttt{[C]}},C,\mathrm{\texttt{[S]}},\hat{d_{1}},...,\hat{d_{i}},\mathrm{\texttt{[M]}})\end{split}$

Let $\mathbf{h}_{\texttt{[M]}}\in\mathbb{R}^{h}$ denote the hidden
representation of [M] of $X_{i}$ returned by BERT transformer stacks. The
prediction of $\hat{d_{i}}$ is given by a linear layer transformation
$\mathbf{W}_{\texttt{DG}}\in\mathbb{R}^{h\times|V|}$ and a softmax activation
to all vocabulary dimension as follows.

$p(w|X_{i})=softmax(\mathbf{h}_{\texttt{[M]}}\cdot\mathbf{W}_{\texttt{DG}}+\mathbf{b}_{\texttt{DG}})$
$\hat{d_{i+1}}=\mathrm{argmax}_{w}{Pr(w|X_{i})}$

Subsequently, the newly generated token $\hat{d_{i}}$ is appended into
$X_{i+1}$ and the distractor generation process is repeated based on the new
$X_{i+1}$ until [S] is predicted. Our loss function is as follows.

$\underset{\theta}{\text{minimize}}-\sum_{\forall(C,D)}\sum_{i=0}^{|D|}(\text{log}_{2}p(d_{i+1}|{C,d_{1:i};\theta}))$

###  2.3 Multi-task with Parallel MLM

From the experiment results (will be presented in the later section), we see
the BDG model advances the state-of-the-art result Zhou et al. (2019) from
28.65 to 35.30 (BLEU 1 score). While the token-level evaluation result looks
promising, we find that generation results still have room to be improved.

For performance improvement, we first propose to jointly train BDG and a
parallel MLM (P-MLM) architecture for distractor generation to enhance the
quality of BDG. The P-MLM scheme for generating distractors is structured as
follows.

For a given context $C$, the input sequence $X$ to P-MLM model is formulated
as

$\begin{split}X=(\mathrm{\texttt{[C]}},C,\mathrm{\texttt{[S]}},\mathrm{[\texttt{M}]_{d_{1}}},\mathrm{[\texttt{M}]_{d_{2}}},...,\mathrm{[\texttt{M}]_{d_{|D|}}})\end{split}$

Let $\mathbf{h}_{\mathrm{\texttt{[M]}_{d_{i}}}}\in\mathbb{R}^{h}$ denote the
hidden representation of $\mathrm{\texttt{[M]}_{d_{i}}}$ of $X$ returned by
BERT transformer stacks. The prediction of $\hat{q_{i}}$ is given by a linear
layer transformation $\mathbf{W}_{\texttt{P-MLM}}\in\mathbb{R}^{h\times|V|}$
and applying a softmax activation to all vocabulary dimension as follows.

$p(w|X)=softmax(\mathbf{h}_{\mathrm{[\texttt{M}]_{d_{i}}}}\cdot\mathbf{W}_{\texttt{P-MLM}}+\mathbf{b}_{\texttt{P-MLM}})$
$\hat{d_{i}}=\mathrm{argmax}_{w}{Pr(w|X)}$

The loss function for P-MLM is

$\underset{\theta}{\text{minimize}}-\sum_{\forall(C,D)}\phi_{{}_{\texttt{P-MLM}}}(C,D)$
$\phi_{{}_{\texttt{P-MLM}}}(C,D)=\sum_{\forall
d_{i}}\text{log}_{2}p(d_{i}|{C,\mathrm{[\texttt{M}]}_{d_{i}};\theta})$

We propose to jointly train P-MLM and BDG by the following multi-tasking loss
function. Note that $\gamma$ is a hyper-parameter controlling the weighting
between the two tasks. See also the effect of the $\gamma$ value in Subsection
4.6.

$\underset{\theta}{\text{minimize}}-\sum_{\forall(C,D)}[\phi_{{}_{\texttt{BDG}}}(C,D)+\gamma\cdot\phi_{{}_{\texttt{P-MLM}}}(C,D)],$
$\phi_{{}_{\texttt{BDG}}}(C,D)=\sum_{i=0}^{|D|}(\text{log}_{2}p(d_{i+1}|{C,d_{1:i};\theta}))$

The multi-task design is motivated by the following observations. First, as
mentioned, we target at learning distractor generation from real reading
comprehension examination (RACE-like MCQ), and we find that many questions in
the RACE dataset are summary-oriented; many questions are about ”what is the
best title for this passage?” or ”what is this passage about?” Such questions
require the model to have the capability of passage semantic summarization.
While the original BDG scheme design successfully generates fluent question
sentences, we find that it may over-fit in sentence writing and under-fit in
learning the passage semantic understanding capability. Note that the
sequential-MLM design (BDG) essentially is a one-by-one masked token
prediction architecture. Such a method may over-focus on the guess of a single
token and ignore the overall semantic understanding. Thus, we propose to
incorporate the multi-task learning setting to prevent the potential over-
fitting problem. From the experiments, we find the multi-task learning setting
indeed improves the quality of distractor generation.

###  2.4 Answer Negative Regularization

In addition to the multi-task design, from the DG result examination, we find
another observation that in many cases, there is an answer copying problem;
the generated distractors are similar to the given answers. To better see this
phenomenon, we experiment to count such cases. In the following table, we show
the number of cases that the generated distractor $\hat{D}$ has a token-level
similarity score greater than $0.95$ with respect to the answer $A$. We also
show the cases for the gold distractors (the human-invented distractors from
the RACE dataset). By comparison in Table 3, there is a significant gap
between the human-invented distractors and the model generated ones.

| P.M. | Gold
---|---|---
# of cases on BLEU 1 | 57 | 12
# of cases on BLEU 2 | 55 | 4
# of cases on BLEU 3 | 48 | 0
# of cases on BLEU 4 | 35 | 0
# of cases on ROUGE-L | 55 | 1
Table 3: Answer Copying Problem on P.M.

Motivated by the answer copying problem, we propose to incorporate a loss
(referred to as answer negative loss) to discourage predicting tokens in $A$
when predicting $\hat{d_{i}}$. With the answer negative loss, our loss
function for BDG is as follows.

$\begin{split}&\underset{\theta}{\text{minimize}}-\sum_{\forall(C,D)}(\phi_{{}_{\texttt{AN}}}(C,D)+\gamma\cdot\phi_{{}_{\texttt{P-MLM}}}(C,D)),\\\
&\begin{split}\phi_{{}_{\texttt{AN}}}=&\sum_{i=0}^{|D|}(\text{log}_{2}p(d_{i+1}|{C,d_{1:i};\theta})+\\\
&\sum_{\forall a_{j}\in
A}\text{log}_{2}(1-p(a_{j}|{C,\mathrm{[\texttt{M}]}_{a_{j}};\theta}))\end{split}\end{split}$
(1)

The design of answer negative loss is motivated by that we expect to regulate
the generated distractor $\hat{D}$ to use words different from $A$.

The overall architecture for training our BDG model is shown in Figure 1. The
core structure for our distractor generation is mainly based on the sequential
recurrent MLM decoding mechanism. That is, during the the testing stage, we
use the results from the sequential recurrent MLM decoding part. However,
during the training stage, we incorporate the parallel MLM decoding mechanism
by jointly considering answer negative regularization and sentence-level
distractor loss, as shown in the right-part of the architecture in Figure 1.

Figure 1: The Multi-tasking Architecture

##  3 Multiple Distractor Generation

###  3.1 Selecting Distractors by Entropy Maximization

As mentioned, another point that can be improved for DG is that the existing
methods mainly focus on single distractor generation. For having more than one
distractor, the existing practices are to select the results on different beam
search paths as multiple options for distractor generation, which lowers the
power of distracting a reader for MCQ preparation.

Our viewpoint is to select a distractor set (by considering semantic
diversity) rather than individually selecting top-k distractors based on
prediction probability.

Based on this view, we propose to incorporate a multi-choice reading
comprehension (MRC) model for ranking/selecting distractor sets. First, let
$\mathbb{M}_{\texttt{MRC}}$ be a MRC model. Note that
$\mathbb{M}_{\texttt{MRC}}$ takes a passage $P$, a question $Q$, and a set of
options (including an answer $A$ and distractors $D_{1},D_{2},...,D_{n}$) as
input and outputs [$p_{A},p_{D_{1}},...,p_{D_{n}}$] as the answer
probabilities of the options. $\mathbb{M}_{\texttt{MRC}}$ is trained by
maximizing the answer probability $p_{A}$ while minimizing the probabilities
[$p_{D_{1}},...,p_{D_{n}}$].

With $\mathbb{M}_{\texttt{MRC}}$, our idea is as follows. First, let
$\mathbb{DG}_{\texttt{BDG}}$ be a BDG model for distractor generation. Also,
let $\hat{D}=\\{\hat{d}_{1},\hat{d}_{2},...,\hat{d}_{n}\\}$ be the set of
generated distractors by the BDG model. In a common MCQ setting, there are
four options (one answer $A$ and three distractors $d_{i},d_{j},d_{k}$) for
each question. Our idea is to enumerate all possible triples from
$\\{\hat{d}_{1},\hat{d}_{2},...,\hat{d}_{n}\\}$. That is, we have a triple set

$\\{(d_{i},d_{j},d_{k})|i\neq j\neq k,d_{i},d_{j},d_{k}\in\hat{D}\\}$

For a given passage $P$, question $Q$, and answer $A$, our goal is to find a
triple ($d_{i},d_{j},d_{k}$) to form an option set $O$ (i.e.,
{$d_{i},d_{j},d_{k}$, $A$} ) that maximizes the following entropy function.

$\underset{}{\text{maximize}}-\sum_{\forall o_{i}\in
O}p_{o_{i}}log_{2}p_{o_{i}}$ (2)

###  3.2 BDG-EM

The idea of selecting distractors by entropy maximization can be further
generalized by employing multiple DG models. For having multiple DG models,
our idea is to leverage the variants of the BDG model (i.e., models
with/without answer negative regularization or with/without both answer
negative regularization and P-MLM multi-task training). Let
$\hat{D},\hat{D}_{\texttt{PM}}$, and $\hat{D}_{\texttt{PM+AN}}$ be the BDG
model without both answer negative regularization and P-MLM multi-task
training, the BDG model without answer negative regularization, and the full
BDG model. That is, we have a triple set as follows.

$\\{(d_{i},d_{j},d_{k})|d_{i}\in\hat{D},d_{j}\in\hat{D}_{\texttt{PM}},d_{k}\in\hat{D}_{\texttt{PM+AN}}\\}$

With the triple set, the set that maximizes Eq. (2) is selected as final
distractors.

##  4 Performance Evaluation

###  4.1 Experimental Settings

Datasets We follow the setting Gao et al. (2019) to evaluate our framework
with the RACE Lai et al. (2017) dataset. RACE contains 27,933 articles with
97,687 questions from English examinations of Chinese students from grade 7 to
12. We use data split setting from Gao et al. (2019). Table 4 reports the
statistics for the test data set. All sentences are tokenized by the WordPiece
tokenizer Wu et al. (2016).

Implementation Details Our models are implemented based on huggingface
transformers framework Wolf et al. (2019). All experiments are based on bert-
base-cased model. For optimization in the training, we use AdamW as the
optimizer and the initial learning rate 5e-5 for all baselines and our model.
The maximum number of epoch is set to 6 with a batch size of 30 on two RTX
Titan GPUs. We also make our code and model available at
https://github.com/voidful/BDG

###  4.2 Compared Methods

In the experiments, we mainly compare the following distractor generation
methods.

  * •

CO-Att. We compare with the state-of-the-art method reported in Zhou et al.
(2019). The model is based on LSTM augmented by co-attention mechanism.

  * •

DS-Att. We also compare with the method based on LSTM augmented by dynamic and
static attention designed reported in Gao et al. (2019). This method is served
as a baseline for distractor generation based on seq2seq RNN architectures.

  * •

GPT We also experiment with a model based on GPT Radford et al. (2018) to
learn the distractor generation. This scheme can be served as a baseline based
on transformer-based pre-trained model.

  * •

BDG The scheme without the answer negative technique and parallel masked-LM
multi-task training.

  * •

BDG${}_{\texttt{PM}}$ The BDG scheme with the parallel masked-LM multi-task
training ($\gamma=1$).

  * •

BDG${}_{\texttt{AN+PM}}$ The BDG scheme with both techniques ($\gamma=1$).

Train samples | 96501
---|---
Test samples | 12284
Avg.article length | 335.6
Avg.distractor length | 8.6
Avg.question length | 10.0
Avg.answer length | 8.3
Avg.distractor number | 2.1
Table 4: Training Data Statistics | BLEU 1 | BLEU 2 | BLEU 3 | BLEU 4 | ROUGE L
---|---|---|---|---|---
BDG${}_{\texttt{AN+PM}}$ | 39.52 | 24.29 | 17.28 | 13.28 | 33.40
BDG${}_{\texttt{PM}}$ | 39.81 | 24.81 | 17.66 | 13.56 | 34.01
BDG | 35.30 | 20.65 | 13.66 | 9.53 | 31.11
GPT | 36.49 | 20.75 | 13.31 | 9.31 | 31.59
DS-Att. | 27.32 | 14.69 | 9.29 | 6.47 | 15.12
CO-Att. | 28.65 | 15.15 | 9.77 | 7.01 | 15.39
Table 5: Performance Comparison on Token Scores | BDG${}_{\texttt{AN+PM}}$ | BDG${}_{\texttt{PM}}$ | BDG | GPT | Gold | Random
---|---|---|---|---|---|---
BLEU 1 | 43 | 57 | 115 | 124 | 12 | 0
BLEU 2 | 40 | 55 | 115 | 121 | 4 | 0
BLEU 3 | 37 | 48 | 109 | 109 | 0 | 0
BLEU 4 | 30 | 35 | 97 | 88 | 0 | 0
ROUGE-L | 42 | 55 | 122 | 123 | 1 | 0
Table 6: The Effect on Mitigating Answer Copying Problem

###  4.3 Token Score Comparison

We employ BLEU score Papineni et al. (2002) and ROUGE (L) Lin (2004) scores to
evaluate the performance of the compared methods. The BLEU scores evaluate
average n-gram precision on a set of reference sentences, with a penalty for
overly long sentences. The ROUGE (L) measure is the recall of longest common
sub-sequences.

The comparison results are summarized in Table 5. There are three observations
to note. First, one can see that our models significantly outperform the
existing methods (i.e., DS-Att. and CO-Att.). Our best performing model
advances the state-of-the-art result from 28.65 to 39.81 (BLEU 1 score).
Second, as shown, the methods based on transformer models outperform the RNN-
based models. This result again demonstrates the effectiveness of the
employment of pre-trained transformer model to the downstream tasks. Third,
one may notice that our models based on BERT outperforms the GPT-based model.
We believe the reason is that the distractors in the RACE data set is mostly a
summary type sentence that requires semantic understanding. The GPT-based
model may over-focus on sentence writing, and fail to capture the whole
context to generate summary-type sentences, and therefore obtain lower scores.

We also provide experiment results to observe the effectiveness on reducing
the answer copying problem discussed in Subsection 2. In Table 6, we show the
number of cases that the generated distractor $\hat{D}$ has a token-level
similarity score greater than $0.95$ with respect to the context answer $A$.
From the experiment result, we see that there are significant improvement made
by the BDG schemes.

###  4.4 MCQ Model Accuracy Comparison

In this set of experiment, we evaluate the DG quality by the RACE reading
comprehension task Lai et al. (2017). Our idea is that a poorly generated DG
result will reduce the difficulty of a MCQ task. Thus, we propose to
incorporate a MCQ answering model (also trained by the RACE dataset) to
evaluate the accuracy of a multiple-choice question with the distractors
generated by the compared model. Specifically, given $C$, $Q$, and $A$, we
generate three distractors $D_{1}$, $D_{2}$, and $D_{3}$, and then submit the
multiple-choice question to the RACE model. Randomly generated results will be
the easiest task to solve, and the best generated results will bring
challenges to the MCQ model. Therefore, we use the accuracy of the model as a
metric. The higher the accuracy, the worse the generation quality.

The training details of the RACE model is as follows. We use PyTorch
TransformersWolf et al. (2019) and the roberta-base-openai-detector fine-tuned
by OpenAI Solaiman et al. (2019) with max 512 tokens to implement the model.
AdamW with a Learning rate = 1e-5 is used for fine-tuning. The model is
trained for 10 epoch on 2 GPUs (V100) with gradient accumulation per two
steps, which makes the batch size approximately equal to 18. Model checkpoints
are saved and evaluated on the validation set every 5,000 steps. We select the
top checkpoint based on evaluation loss on the validations set. The RACE
dataset includes middle and high dataset. The total number of passages and
questions is 27,933 and 97,687 respectively. Middle dataset averages about 250
words per passage while the High dataset averages 350 words per passage.

| Accuracy
---|---
Random Selected Distractors | 88.10%
Gold Distractor | 78.00%
GPT | 78.07%
BDG | 73.96%
BDG${}_{\texttt{PM}}$ | 74.34%
BDG${}_{\texttt{AN+PM}}$ | 74.05%
BDG${}_{\texttt{EM}}$ | 69.44%
Table 7: Comparison by MCQ Accuracy

In this set of experiment, we compare BDG, BDG${}_{\texttt{PM}}$,
BDG${}_{\texttt{AN+PM}}$, the BDG model with entropy maximization (called
BDG${}_{\texttt{EM}}$) (introduced in Subsection 3.2) by setting the beam
search size to 3, and the BDG model ensemble introduced in Subsection 3.2. In
addition, we also experiment with the GPT, a scheme that takes randomly
selected distractors from the data as the DG result, and the scheme uses the
gold distractors. The results of the compared methods are summarized in Table
7.

We have the following findings to note about the results shown in Table 7.
First, as expected, the method with randomly selected distractors makes the
MCQA model has the highest accuracy, as the randomly selected distractors
obviously lower the difficulty of MCQ task. Second, all our models outperform
the MCQ with the gold distractors, showing the effectiveness of the proposed
models. Third, as expected, our BDG${}_{\texttt{EM}}$ provides the best
performing result on this metric.

###  4.5 Qualitative Examination by Case Study

In this subsection, we present showcases to see the improvement on multiple
distractor generation scenario. We use the same examples introduced in Section
1 for comparison. First, as mentioned, the naive employment of beam search
strategy produces similar DG results. As shown in the examples, the
distractors generated by BDG are about the same concept. However, as shown in
Table 8, we see the BDG${}_{\texttt{EM}}$ produce more diverse distractors
with respect to each other. The results demonstrate the effectiveness of our
BDG${}_{\texttt{EM}}$ scheme for generating multiple distractors for MCQ
preparation.

Example 1
---
Context Omitted. (See Appendix)
Question
$\cdot$ Why did Mr.King want to send Henry away?
Answer
|  $\cdot$ Because Henry was too lazy.
---
BDG
$\cdot d_{1}:$ Because Henry didn’t want to go.
$\cdot d_{2}:$ Because Henry didn’t want to go to the bookstore.
$\cdot d_{3}:$ Because Henry didn’t want to go out.
BDGEM
$\cdot d_{1}:$ Because Henry didn’t want to go.
$\cdot d_{2}:$ Because Henry wanted to be rich.
$\cdot d_{3}:$ Because Henry wanted to be a clever man.
Example 2
Context Omitted. (See Appendix)
Question
$\cdot$ Which of the following women would look most attractive?
Answer
$\cdot$ A short red-haired woman who wears a purple hat.
BDG
$\cdot d_{1}:$ A young woman who wears a white hat.
|  $\cdot d_{2}:$ A woman who wears a white hat .
---
BDGEM
$\cdot d_{1}:$ A short black woman with big, round faces.
$\cdot d_{2}:$ A young woman who doesn’t like a white hat.
$\cdot d_{3}:$ A little woman who wears a pink hat.

Table 8: Qualitative Examination by Case Study

| BLEU 1 | BLEU 2 | BLEU 3 | BLEU 4 | ROUGE L
---|---|---|---|---|---
PM($\gamma$=1) | 36.97 | 22.07 | 14.82 | 10.50 | 32.64
PM($\gamma$=2) | 38.45 | 23.21 | 15.81 | 11.36 | 33.18
PM($\gamma$=3) | 39.23 | 24.27 | 17.04 | 12.78 | 33.82
PM($\gamma$=4) | 39.22 | 24.24 | 17.08 | 12.95 | 34.05
PM($\gamma$=5) | 39.74 | 24.50 | 17.29 | 13.09 | 34.11
PM($\gamma$=6) | 39.81 | 24.81 | 17.66 | 13.56 | 34.01
PM($\gamma$=7) | 39.37 | 24.13 | 17.09 | 13.07 | 33.45
AN+PM($\gamma$=1) | 37.49 | 22.08 | 13.73 | 10.44 | 32.40
AN+PM($\gamma$=2) | 38.25 | 22.81 | 15.33 | 10.91 | 32.99
AN+PM($\gamma$=3) | 38.71 | 23.54 | 16.26 | 12.04 | 33.82
AN+PM($\gamma$=4) | 38.84 | 23.70 | 16.57 | 12.46 | 33.53
AN+PM($\gamma$=5) | 39.19 | 23.97 | 16.96 | 12.92 | 33.67
AN+PM($\gamma$=6) | 39.58 | 24.23 | 17.11 | 13.11 | 33.38
AN+PM($\gamma$=7) | 39.52 | 24.29 | 17.28 | 13.28 | 33.40

Table 9: Performance Comparison on Token Scores with Different $\gamma$
Settings

###  4.6 Parameter Study on $\gamma$

In this subsection, we examine the effects on varying the values of the
parameter $\gamma$. In Table 9, we show the results. From the result, we can
see that the best setting for $\gamma$ is 6, and for BDG trained by answer
negative and parallel-MLM, the best setting for $\gamma$ is 7.

##  5 Related Work

The DG research can be categorized from different perspectives. First, for DG
task type, there are two main task categories for DG: cloze-style distractor
generation and reading comprehension (RC) distractor generation. In cloze-
style DG task, it is viewed as a word filling problem. In general, the first
step is to extract distractor candidates from context or some knowledge base,
and then the next step is to rank the extracted distractors as a final result.
Along this direction, the models are mainly based on similarity heuristic
Sumita et al. (2005); Mitkov et al. (2006); Guo et al. (2016); Ren and Zhu
(2020) or supervised machine learning way Liang et al. (2018); Yeung et al.
(2019). The distractors generated for cloze-style DG are mainly word/phrase
level. On the other hand, the RC-type QG focuses on generating sentence-level
distractors for reading comprehension level testing, such as summarizing
article or understanding author opinion Gao et al. (2019); Zhou et al. (2019).
For the sentence-level distractors, neural models are commonly employed as it
is difficult to generate a semantic rich and fluent distractor from question,
content, and answer. In this paper, we also focus on generative sentence-level
DG for RC task. However, as mentioned in the introduction, we find the
existing DG results are still far from human level. The best SOTA result (in
terms of BLEU 1 score) is 29, which is far from the ideal result for practical
use. Aiming at this point, we explore the employment of transformer-based pre-
trained models for performance improvement. For clarity of comparison, we
summarize the existing studies on distractor generation in Table 10.

| Distractor Level | Answer Type | Method Type | Model
---|---|---|---|---
| Word/phrase | Sentence | Cloze | R.C. | Extractive | Generative | Type
Gao et al. 2019 | Y | Y |  | Y |  | Y | RNN
Zhou et al. 2019 | Y | Y |  | Y |  | Y | RNN
Araki et al. 2016 | Y |  | Y |  | Y |  | Non-neural model
Welbl et al. 2017 | Y |  |  | Y | Y |  | Random forests
Guo et al. 2016 | Y |  | Y |  | Y |  | Word2Vec
Kumar et al. 2015 | Y | Y | Y |  | Y |  | SVM
Liang et al. 2017 | Y |  | Y |  |  | Y | GAN
Liang et al. 2018 | Y | Y |  | Y | Y |  | Non-neural model

Table 10: An Overview of the Existing DG works

##  6 Conclusion

We present a state-of-the-art neural model based on a pre-trained transformer-
based model for DG. We introduce two techniques, Answer Negative
Regularization and Multi-task with Parallel MLM, to boost the DG performance.
In addition, we also introduce BDG ensemble with an entropy maximization
mechanism to enhance the DG quality by leveraging a reading comprehension
model. By experimental evaluation, our models outperform the existing best
performing models and advances the state-of-the-art result to 39.81 (BLEU 1
score).

## Acknowledgement

This work was supported by the Ministry of Science and Technology, Taiwan,
under projects No. 109-2221-E-005-058-MY3 and 107-2221-E-005-064-MY2

## References

  * Araki et al. (2016) Jun Araki, Dheeraj Rajagopal, Sreecharan Sankaranarayanan, Susan Holm, Yukari Yamakawa, and Teruko Mitamura. 2016.  Generating questions and multiple-choice answers using semantic analysis of texts.  In _Proceedings of COLING 2016, the 26th International Conference on Computational Linguistics: Technical Papers_ , pages 1125–1136.
  * Chan and Fan (2019) Ying-Hong Chan and Yao-Chung Fan. 2019.  A recurrent bert-based model for question generation.  In _Proceedings of the 2nd Workshop on Machine Reading for Question Answering_ , pages 154–162.
  * Dong et al. (2019) Li Dong, Nan Yang, Wenhui Wang, Furu Wei, Xiaodong Liu, Yu Wang, Jianfeng Gao, Ming Zhou, and Hsiao-Wuen Hon. 2019.  Unified language model pre-training for natural language understanding and generation.  In _Advances in Neural Information Processing Systems_ , pages 13042–13054.
  * Gao et al. (2019) Yifan Gao, Lidong Bing, Piji Li, Irwin King, and Michael R Lyu. 2019.  Generating distractors for reading comprehension questions from real examinations.  In _Proceedings of the AAAI Conference on Artificial Intelligence_ , volume 33, pages 6423–6430.
  * Guo et al. (2016) Qi Guo, Chinmay Kulkarni, Aniket Kittur, Jeffrey P Bigham, and Emma Brunskill. 2016\.  Questimator: Generating knowledge assessments for arbitrary topics.  In _Proceedings of the Twenty-Fifth International Joint Conference on Artificial Intelligence (IJCAI’16). AAAI Press_.
  * Kumar et al. (2015) Girish Kumar, Rafael E Banchs, and Luis Fernando D’Haro. 2015.  Revup: Automatic gap-fill question generation from educational texts.  In _Proceedings of the Tenth Workshop on Innovative Use of NLP for Building Educational Applications_ , pages 154–161.
  * Lai et al. (2017) Guokun Lai, Qizhe Xie, Hanxiao Liu, Yiming Yang, and Eduard Hovy. 2017.  Race: Large-scale reading comprehension dataset from examinations.  _arXiv preprint arXiv:1704.04683_.
  * Lan et al. (2019) Zhenzhong Lan, Mingda Chen, Sebastian Goodman, Kevin Gimpel, Piyush Sharma, and Radu Soricut. 2019.  Albert: A lite bert for self-supervised learning of language representations.  _arXiv preprint arXiv:1909.11942_.
  * Liang et al. (2018) Chen Liang, Xiao Yang, Neisarg Dave, Drew Wham, Bart Pursel, and C Lee Giles. 2018\.  Distractor generation for multiple choice questions using learning to rank.  In _Proceedings of the thirteenth workshop on innovative use of NLP for building educational applications_ , pages 284–290.
  * Liang et al. (2017) Chen Liang, Xiao Yang, Drew Wham, Bart Pursel, Rebecca Passonneaur, and C Lee Giles. 2017.  Distractor generation with generative adversarial nets for automatically creating fill-in-the-blank questions.  In _Proceedings of the Knowledge Capture Conference_ , pages 1–4.
  * Lin (2004) Chin-Yew Lin. 2004.  Rouge: A package for automatic evaluation of summaries.  In _Text summarization branches out_ , pages 74–81.
  * Liu et al. (2019) Yinhan Liu, Myle Ott, Naman Goyal, Jingfei Du, Mandar Joshi, Danqi Chen, Omer Levy, Mike Lewis, Luke Zettlemoyer, and Veselin Stoyanov. 2019.  Roberta: A robustly optimized bert pretraining approach.
  * Mitkov et al. (2006) Ruslan Mitkov, Ha Le An, and Nikiforos Karamanis. 2006.  A computer-aided environment for generating multiple-choice test items.  _Natural language engineering_ , 12(2):177–194.
  * Papineni et al. (2002) Kishore Papineni, Salim Roukos, Todd Ward, and Wei-Jing Zhu. 2002.  Bleu: a method for automatic evaluation of machine translation.  In _Proceedings of the 40th annual meeting on association for computational linguistics_ , pages 311–318. Association for Computational Linguistics.
  * Radford et al. (2018) Alec Radford, Karthik Narasimhan, Tim Salimans, and Ilya Sutskever. 2018.  Improving language understanding by generative pre-training.  _URL https://s3-us-west-2. amazonaws. com/openai-assets/research-covers/languageunsupervised/language understanding paper. pdf_.
  * Ren and Zhu (2020) Siyu Ren and Kenny Q Zhu. 2020.  Knowledge-driven distractor generation for cloze-style multiple choice questions.  _arXiv preprint arXiv:2004.09853_.
  * Solaiman et al. (2019) Irene Solaiman, Miles Brundage, Jack Clark, Amanda Askell, Ariel Herbert-Voss, Jeff Wu, Alec Radford, and Jasmine Wang. 2019.  Release strategies and the social impacts of language models.  _arXiv preprint arXiv:1908.09203_.
  * Sumita et al. (2005) Eiichiro Sumita, Fumiaki Sugaya, and Seiichi Yamamoto. 2005.  Measuring non-native speakers’ proficiency of english by using a test with automatically-generated fill-in-the-blank questions.  In _Proceedings of the second workshop on Building Educational Applications Using NLP_ , pages 61–68.
  * Welbl et al. (2017) Johannes Welbl, Nelson F Liu, and Matt Gardner. 2017.  Crowdsourcing multiple choice science questions.  _arXiv preprint arXiv:1707.06209_.
  * Wolf et al. (2019) Thomas Wolf, Lysandre Debut, Victor Sanh, Julien Chaumond, Clement Delangue, Anthony Moi, Pierric Cistac, Tim Rault, R’emi Louf, Morgan Funtowicz, and Jamie Brew. 2019.  Huggingface’s transformers: State-of-the-art natural language processing.  _ArXiv_ , abs/1910.03771.
  * Wu et al. (2016) Yonghui Wu, Mike Schuster, Zhifeng Chen, Quoc V Le, Mohammad Norouzi, Wolfgang Macherey, Maxim Krikun, Yuan Cao, Qin Gao, Klaus Macherey, et al. 2016.  Google’s neural machine translation system: Bridging the gap between human and machine translation.  _arXiv preprint arXiv:1609.08144_.
  * Yeung et al. (2019) Chak Yan Yeung, John SY Lee, and Benjamin K Tsou. 2019.  Difficulty-aware distractor generation for gap-fill items.  In _Proceedings of the The 17th Annual Workshop of the Australasian Language Technology Association_ , pages 159–164.
  * Zhou et al. (2019) Xiaorui Zhou, Senlin Luo, and Yunfang Wu. 2019.  Co-attention hierarchical network: Generating coherent long distractors for reading comprehension.

Appendix
Content The building is shaking. A woman with a baby in her arms is trying to
open the door, but fails. Finding no way, she rushes into her bedroom and
there they survive the earthquake. In a factory building, as the workshop
floor swings under the terrible shaking, workers run for safety. Some hide
under the machines and survive, but others who try to run outside are killed
by the falling ceilings. These scenes, played by actors and actresses, are
from a film of science education Making a Split Second Decision shown in 1998
on China Central TV in memory of Tangshan Earthquake. By studying actual cases
in the earthquake areas and scientific experiments, experts find that
buildings remain untouched for the first 12 seconds of an earthquake. In this
short time, one has the best chance of surviving an earthquake by staying near
the inside walls, in bedrooms and under beds, experts concluded in the film.
”Earthquakes seem to catch the lives of those who run,” said many survivors in
the earthquake areas, describing how their friends were killed on the doorways
or along the stair steps as they tried to get out of the building. Their
advice was proved in the film, ”Take a hiding-place where you are rather than
run, unless you are sure you can reach a safe open place in ten seconds.”
Question The workers who try to run outside the building die because? Answer
They don’t have enough time to run outside. Distractor They don’t know how to
get out of the building.

Table 11: BDG showcase

Content |  Henry found work in a bookstore after he finished middle school. He wouldn’t do anything but wanted to get rich. Mr.King thought he was too lazy and was going to send him away. Henry was afraid and had to work hard. It was a cold morning. It was snowing and there was thin ice on the streets. Few people went to buy the books and the young man had nothing to do. He hated to read, so he watched the traffic. Suddenly he saw a bag fall off a truck and it landed by the other side of the street. Ït must be full of expensive things.Ḧenry said to himself. Ï have to get it, or others will take it away.Ḧe went out of the shop and ran across the street. A driver saw him and began to whistle, but he didn’t hear it and went on running. The man drove aside, hit a big tree and was hurt in the accident. Two weeks later Henry was taken to court. A judge asked if he heard the whistle when he was running across the street. He said that something was wrong with his ears and he could hear nothing. ”But you’ve heard me this time.” said the judge. ”Oh , I’m sorry. Now I can hear with one ear.” ”Cover the ear with your hand and listen to me with your deaf one. Well, can you hear me ?” ” No, I can’t, Sir.”
---|---
Question |  Why did Mr.King want to send Henry away?
Answer |  Because Henry was too lazy.
BDG |  Because Henry didn’t want to go.
|  Because Henry didn’t want to go out.
Because Henry didn’t want to go to the bookstore.
BDG ensemble |  Because Henry didn’t want to go.
|  Because Henry wanted to be rich.
Because Henry wanted to be a clever man.

Table 12: Context for Example 1

Content |  Most of the time, people wear hats to protect themselves from weather conditions . Hats are also worn to show politeness and as signs of social position. But nowadays, hats, especially women’s hats, are much more than that. More exactly, hats have changed into fashion and style symbols by many movie stars. What’s more, people now consider many different features when choosing even a simple hat. Many designers point out that, when choosing the right hat, it’s important to consider the color of your skin as well as your hair, your height, and the shape of your face. First of all, the color of the hat should match the color of your skin and hair. For instance, black hats should be avoided if you are dark skinned. If a purple hat is placed on top of red hair, one will look as attractive as a summer flower. Second, the height of the hat is also an important point. Tall women should not go for hats with tall crowns, just as short women should choose hats with upturned brims to give the look of height. Third, and most importantly, the shape of the face decides the kind of hat one should pick. A small, gentle hat that fits the head looks good on a small face. However, women with big, round faces should choose a different style. As the saying goes, F̈ine feathers make fine birds.Ä good hat can not only help your dress but also support your features, so why not choose the best possible one next time you want to be in public?
---|---
Question |  According to the article, which of the following women would look most attractive?
Answer |  A short red-haired woman who wears a purple hat.
BDG |  A young woman who wears a white hat.
|  A young woman who doesn’t like a white hat.
A woman who wears a white hat.
BDG ensemble |  A short black woman with big, round faces.
|  A young woman who doesn’t like a white hat.
A little woman who wears a pink hat.

Table 13: Context for Example 2

Content |  Memory, they say, is a matter of practice and exercise. If you have the wish and really made a conscious effort, then you can quite easily improve your ability to remember things. But even if you are successful, there are times when your memory seems to play tricks on you. Sometimes you remember things that really did not happen. One morning last week, for example, I got up and found that I had left the front door unlocked all night, yet I clearly remember locking it carefully the night before. Memory ”trick” work the other way as well. Once in a while you remember not doing something, and then find out that you did. One day last month, for example, I was sitting in a barber shop waiting for my turn to get a haircut, and suddenly I realized that I had got a haircut two days before at the barber shop across the street from my office. We always seem to find something funny and amusing in incidents caused by people’s forgetfulness or absent-mindedness. Stories about absent-minded professors have been told for years, and we never got tired of hearing new ones. Unfortunately, however, absent-mindedness is not always funny. There are times when ”trick” of our memory can cause us great trouble.
---|---
Question |  Which of the following statements is true according to the passage ?
Answer |  One night the writer forgot to lock the front door.
BDG |  The writer couldn’t find a hair cut in the barber shop.
|  The writer couldn’t find a hair cut in the shop.
BDG ensemble |  The writer didn’t want to open the front door.
|  The writer couldn’t find the reason why he left the front door.

Table 14: Yet another example for BDG multiple distractor generation