#  GW190425: Pan-STARRS and ATLAS coverage of the skymap and limits on optical
emission associated with FRB190425

S. J. Smartt1,2, M. Nicholl2, S. Srivastav2, M. E. Huber3, K. C. Chambers3, K.
W. Smith2, D. R. Young2, M. D. Fulton2, J. L. Tonry3, C. W. Stubbs4, L.
Denneau3, A. J. Cooper1, A. Aamer2, J. P. Anderson5,6, A. Andersson1, J.
Bulger3, T.-W Chen7, P. Clark8, T. de Boer3, H. Gao3, J. H. Gillanders1, A.
Lawrence3, C. C. Lin3, T. B. Lowe3, E. A. Magnier3, P. Minguez3,T. Moore2, A.
Rest9,10, L. Shingles11, R. Siverd3, I. A. Smith12, B. Stalder13, H. F.
Stevance1 R. Wainscoat3, R. Williams14
1Astrophysics sub-Department, Department of Physics, University of Oxford,
Keble Road, Oxford, OX1 3RH, UK
2Astrophysics Research Centre, School of Mathematics and Physics, Queen’s
University Belfast, BT7 1NN, UK
3Institute for Astronomy, University of Hawai’i, 2680 Woodlawn Drive,
Honolulu, HI 96822, USA
4Department of Physics and Department of Astronomy, Harvard University,
Cambridge, MA 02138, USA
5European Southern Observatory, Alonso de Córdova 3107, Casilla 19, Santiago,
Chile
6Millennium Institute of Astrophysics (MAS), Nuncio Monseñor Sótero Sanz 100,
Off. 104, Providencia, Santiago, Chile
7 Graduate Institute of Astronomy, National Central University, 300 Jhongda
Road, 32001 Jhongli, Taiwan
8Institute of Cosmology and Gravitation, University of Portsmouth PO1 3FX,
Portsmouth, UK
9Space Telescope Science Institute, 3700 San Martin Drive, Baltimore, MD
21218, USA
10Department of Physics and Astronomy, Johns Hopkins University, Baltimore, MD
21218, USA
11GSI Helmholtzzentrum für Schwerionenforschung, Planckstrasse 1, D-64291
Darmstadt, Germany
12 Institute for Astronomy, University of Hawai’i, 34 Ohia Ku St., Pukalani,
HI 96768-8288, USA
13Vera C. Rubin Observatory Project Office, 950 N Cherry Ave, Tucson, AZ
95719, USA
14 Institute for Astronomy, University of Edinburgh, Royal Observatory,
Blackford Hill, EH9 3HJ, UK
<EMAIL_ADDRESS>0000-0002-8229-1731 0000-0002-2555-3192
0000-0003-4524-6883 0000-0003-1059-9603 0000-0001-6965-7789
0000-0001-9535-3199 0000-0002-1229-2499 0000-0003-1916-0664
0000-0003-2858-9657 0000-0003-0347-1724 0000-0002-4033-3139
0000-0003-0227-3451 0000-0003-2734-1895 0000-0002-1066-6098
0000-0002-6576-7400 0000-0003-1015-5367 0000-0002-8094-6108
0000-0002-4474-4963 0000-0002-7272-5129 0000-0002-7965-2815
0000-0001-8385-3727 0000-0002-4410-5387 0000-0002-5738-1612
0000-0001-5016-3359 0000-0001-8605-5608 0000-0003-0973-4900
0000-0002-1341-0952

(Submitted 20th September 2023)

###### Abstract

GW190425 is the second of only two binary neutron star (BNS) merger events to
be significantly detected by the LIGO-Virgo-Kagra gravitational wave
detectors. With a detection only in LIGO Livingston, the skymap containing the
source was large and no plausible electromagnetic counterpart was found in
real time searching in 2019. Here we summarise our ATLAS and Pan-STARRS wide-
field optical coverage of the skymap beginning within 1 hour and 3 hours
respectively of the GW190425 merger time. More recently, a potential
coincidence between GW190425 and a fast radio burst FRB 190425 has been
suggested, given their spatial and temporal coincidence. The smaller sky
localisation area of FRB 190425 and its dispersion measure have led to the
identification of a likely host galaxy, UGC 10667 at a distance of $141\pm 10$
Mpc. Our optical imaging covered the galaxy 6.0 hrs after GW190425 was
detected and 3.5 hrs after the FRB 190425. No optical emission was detected
and further imaging at +1.2 and +13.2 days also revealed no emission. If the
FRB 190425 and GW190425 association were real, we highlight our limits on
kilonova emission from a BNS merger in UGC 10667. The model for producing FRB
190425 from a BNS merger involves a supramassive magnetised neutron star
spinning down by dipole emission on the timescale of hours. We show that
magnetar enhanced kilonova emission is ruled out by optical upper limits. The
lack of detected optical emission from a kilonova in UGC 10667 disfavours, but
does not disprove, the FRB-GW link for this source.

###### keywords:

gravitaional waves: GW190425 – fast radio bursts: FRB190425 – kilonovae –
surveys

††pubyear: 2023††pagerange:  GW190425: Pan-STARRS and ATLAS coverage of the
skymap and limits on optical emission associated with FRB190425– GW190425:
Pan-STARRS and ATLAS coverage of the skymap and limits on optical emission
associated with FRB190425

##  1 Introduction

The historic gravitational wave event GW170817 resulting from a binary neutron
star (BNS) merger (Abbott et al., 2017a) produced a short gamma-ray burst
(GRB170817A; Abbott et al., 2017c) and a rapidly evolving optical and infrared
transient (AT2017gfo; Abbott et al., 2017b). The relatively small sky
localisation map (31 square degrees), inferred from the strong signals in the
two LIGO detectors and an upper limit in Virgo (Abbott et al., 2017a), allowed
the rapid identification of an optical counterpart. This was achieved 11 hours
after the BNS merger, during the first night of observing the sky map from
Chile (Arcavi et al., 2017; Coulter et al., 2017; Lipunov et al., 2017;
Soares-Santos et al., 2017; Tanvir et al., 2017; Valenti et al., 2017). Global
monitoring followed, with the spectra from Chilean and South African
observatories showing an unprecedented evolution within the firts 24hrs and
confirming that this was the signature of a unique transient with no known
counterpart (Chornock et al., 2017; McCully et al., 2017; Nicholl et al.,
2017; Pian et al., 2017; Shappee et al., 2017; Smartt et al., 2017). The
lightcurve monitoring showed AT2017gfo faded rapidly with the flux emission
shifting to near infra-red and possibly even beyond (Andreoni et al., 2017;
Cowperthwaite et al., 2017; Drout et al., 2017; Evans et al., 2017; Kasliwal
et al., 2017; Kilpatrick et al., 2017; Tanvir et al., 2017; Troja et al.,
2017; Utsumi et al., 2017). The transient was detected in the x-ray and radio
a few days after the merger (Alexander et al., 2017; Haggard et al., 2017;
Hallinan et al., 2017; Margutti et al., 2017; Troja et al., 2017), providing
constraints on the jet physics giving rise to the short GRB.

GW170817 was discovered towards the end of the LIGO-Virgo collaboration’s
second observing run (O2) and at the time of writing (3 months into O4) only
one further BNS merger has been significantly detected. GW190425 was observed
in only one LIGO detector (Abbott et al., 2020) close to the start of the 3rd
Observing run (O3). With a signal only from Livingston, the sky localisation
map was very large and half of the high probability region was in the day time
sky. No electromagnetic counterpart was discovered at the time at any
wavelength (Coughlin et al., 2019; Smith et al., 2019b; Boersma et al., 2021;
Paterson et al., 2021) which was not a major surprise given the large sky map,
the inferred distance $D_{L}$=$159^{+69}_{-72}$ Mpc (from the GW analysis of
Abbott et al., 2020) and solar conjunction.

Fast radio bursts (FRBs) are extra-galactic millisecond-duration bursts of
unknown origin. The large all-sky rate of FRBs appears inconsistent with a
single compact object merger origin of all sources (Ravi, 2019). However, some
models suggest mergers may be responsible for a subset of FRBs, powered by
either pre-merger magnetic interaction (e.g. Totani 2013) or the merger
remnant (e.g. Falcke & Rezzolla 2014). These models can be effectively tested
by performing prompt FRB searches on localization regions of gravitational
wave events and short gamma-ray bursts, or through post-FRB follow-up
observations of nearby sources in search of a kilonova or radio afterglow
(Cooper et al., 2023).

A search for spatial and temporal coincidences of gravitational wave events
and FRBs by Moroianu et al. (2023) proposed a link between FRB 190425 and
GW190425. They searched for GW-FRB coincidences with the Canadian Hydrogen
Intensity Mapping Experiment FRB catalogue (CHIME/FRB) (CHIME/FRB
Collaboration et al., 2021) and the O3 GWTC-2 catalogue (Abbott et al., 2021).
They used a time window of 26hrs, from 2hrs before the GW event and up to
24hrs after. The CHIME sky localisation region is typically of order
arcminutes in diameter, much smaller than that of GW sources (tens to hundreds
of square degrees depending on the number of detectors retrieving a signal).

FRB 190425 was detected at 58598.44899 (2019 Apr 25 10:46:33UT; CHIME/FRB
Collaboration et al., 2021), which was 2.5 hrs after GW190425 (Moroianu et
al., 2023). Moroianu et al. (2023) report that its estimated sky position of
RA$=255.72\pm 0.14^{\circ}$ DEC$=21.52\pm 0.18^{\circ}$ places it within the
66.7 per cent probability countour of the final, most reliable sky map from
GWTC-2 (Abbott et al., 2021). In addition, the dispersion measure (DM) of FRB
190425 provides an upper limit to the redshift of $z<0.0394$ corresponding to
a luminosity distance111We assume a standard flat cosmology with
$H_{0}=68$$\rm{km}\,s^{-1}$ Mpc-1 from the Planck Collaboration et al. (2016)
as adopted in Abbott et al. (2020). of $D_{\rm L}<179$ Mpc. The FRB and GW
signals were therefore coincident in their sky positions, distances and time
(given the definition of coincidences described). Moroianu et al. (2023)
propose that there is only one catalogued galaxy (in NED, the NASA/IPAC
Extragalactic Database) within the CHIME error ellipse that has a measured
spectroscopic redshift placing it within the upper limit measured from the DM
of $z<0.0394$. This is UGC 10667, at a redshift of $0.031224\pm 0.000011$
(Albareti et al., 2017, from SDSS Data Release 13). We adopt a foreground
extinction toward UGC 10667 of $E(B-V)$=0.066, corresponding to $A_{g}$=0.247,
$A_{r}$=0.171, $A_{i}$=0.127, (Schlafly & Finkbeiner, 2011).

Panther et al. (2023) further investigated the plausibility of UGC 10667 being
the host of FRB190425. Both Panther et al. (2023) and Moroianu et al. (2023)
selected UGC 10667 as the most probable host due to its location within the
68% localization uncertainty of FRB 190425 (from CHIME’s ellipse, of
dimensions $0.1\times 0.2$ deg). The redshift corresponds to a Hubble flow
distance of $D_{L}=$$141\pm 10$ Mpc (corrected for Virgo infall, from NED),
consistent with the LIGO-Virgo distance constraint $D_{L}$=$159^{+69}_{-72}$
Mpc (Abbott et al., 2020) and the upper limit to the FRB DM $D_{\rm L}<179$
Mpc. Panther et al. (2023) find that UGC10667 is a spiral galaxy with a modest
star formation rate and luminosity dominated by an old stellar population.
They also searched for transient radio emission in and around the galaxy at
2.5 yrs post-burst that could be associated with either the FRB 190425 or
GW190425 and found no convincing radio transient emission in VLA 6 Ghz data
taken in September and October 2021.

Moroianu et al. (2023) and Panther et al. (2023) highlight that FRB 190425 had
a number of notable properties; one of lowest DM non-repeating events in the
CHIME FRB Catalogue 1, a high flux for those with low DMs, a short pulse
duration and a flat spectrum. All of this led Moroianu et al. (2023) to
suggest that the co-production of GW190425 and FRB 190425 could be explained
by the theory of Zhang (2014). In this scenario the BNS merger produces a
supramassive neutron star, which is highly magnetised. The compact object has
a short rotation period and loses angular momentum as it spins down and
collapses to a black hole. The FRB is created as the magentosphere is ejected
(Falcke & Rezzolla, 2014), through the so-called “blitzar” mechansim. The
supramassive neutron star must survive for 2.5hrs, the time between merger and
the FRB. While the data and theory of association are intriguing, Bhardwaj et
al. (2023) have cautioned against assuming physical association from analysis
of the GW signal and constraints on the ejecta mass for the 400MHz radio
signal to propagate.

In this paper we present a summary of the ATLAS and Pan-STARRS wide-field
optical coverage of the GW190425 skymap starting 0.8 hrs and 1.36 hrs
respectively after the BNS merger. We present images of the proposed most
likely host of FRB 190425 (UGC 10667) taken over the first few nights, finding
no optical transient emission. We also report publicly available Zwicky
Transient Facility data (Bellm et al., 2019) of the host. We discuss the
plausibility of the GW-FRB link assuming that the host galaxy is indeed UGC
10667 and the BNS produced a kilonova through mass ejection.

Figure 1: The Pan-STARRS1 coverage over the first three nights of observing
starting 1.36 hrs hrs after the detection of GW190425. We used the
bayestar.fits skymap to define our pointings at the time, but the map above
shows the final GWTC-2 skymap as released on GraceDB. The cumulative
probability covered at after one night was 23.8 per cent, which was
incremented to 24.9 per cent after three nights.  Figure 2: The ATLAS coverage
over the first three nights of observing starting 0.8 hrs hrs after the
detection of GW190425. We used the bayestar.fits skymap to define our
pointings at the time, but the map above shows the final GWTC-2 skymap as
released on GraceDB. The cumulative probability covered at this stage was 41.6
per cent.

Figure 3: Illustration of the limiting magnitudes of Pan-STARRS1 (top) and
ATLAS imaging (bottom) of images that fell within the 90 per cent probability
of the sky map gw190425z$\\_$skymap.multiorder.fits. The left hand side of the
axis is set to the GW detection time 58598.34589. Each Pan-STARRS1 point
represents a single skycell of the processed GPC1 frame with exposure times of
45 sec, and the ATLAS points represent a single full-frame ATLAS camera
footprint of 30 seconds.  Figure 4: The Pan-STARRS1 images and difference
images of the galaxy UGC 10667 from MJD = 58598.5957604. This $i_{\rm
P1}$-band image was taken 6.0 hrs after GW190425 merger time and 3.5 hrs hrs
after FRB190425. As discussed in Section 2.1, the excess flux at the core of
the galaxy is either a difference image residual or low level AGN activity and
there is no evidence of a transient source to typical depths of $i_{\rm
P1}$$>21.6$ (north up and east left).

##  2 Observations and data

###  2.1 Pan-STARRS1 observations and data

The Pan-STARRS (PS) is a dual 1.8 m telescope system (Pan-STARRS1 and Pan-
STARRS2) each equipped with a 1.4 Gigapixel camera located at the summit of
Haleakala on the Hawaiian island of Maui (Chambers et al., 2016). The data for
this paper were all taken with the Pan-STARRS1 (PS1) telescope and camera. The
0$\aas@@fstack{\prime\prime}$26 arcsec pixels give a focal plane of 3.0 degree
diameter, that corresponds to a field-of-view area of 7.06 sq deg. It is
equipped with a filter system, denoted as $grizy_{\rm P1}$ as described in
Tonry et al. (2012). The Pan-STARRS1 Science Consortium 3$\pi$ Survey produced
$grizy_{P1}$ images of the whole sky north of $\delta=-30\mbox{${}^{\circ}$}$
(Chambers et al., 2016). We also have proprietary $i_{P1}$ data between
$-40\mbox{${}^{\circ}$}<\delta<-30\mbox{${}^{\circ}$}$. These data provide
reference images for immediate sky subtraction. Images from Pan-STARRS1 are
processed immediately with the Image Processing Pipeline (Magnier et al.,
2020a; Waters et al., 2020).

The individual exposure frames (called warps) are astrometrically and
photometrically calibrated (Magnier et al., 2020c). The 60 CCDs in GPC1 are
processed individually and warped onto a fixed tessellation of skycells as
described in Chambers et al. (2016), each of which is typically 24$\times$24′.
Overlapping exposures can be co-added together (on the skycell tessellation)
with median clipping applied (to produce nightly stacks). The Pan-STARRS1
3$\pi$ reference sky images are subtracted from both the warps and the stacks
(Waters et al., 2020) and photometry carried out on the resulting difference
image (Magnier et al., 2020b). These individual detections are ingested into
the Pan-STARRS Transient Server database at Queen’s University Belfast and
assimilated into distinct objects with a time variable history. A series of
quality filters are applied using the IPP image attributes and known asteroids
and variable stars are removed. The objects remaining are cross-matched with
all catalogued galaxies, AGN, CVs and historical transients (Smartt et al.,
2016a) and simultaneously a machine learning algorithm is applied to image
pixel stamps at each transient position (Wright et al., 2015). This reduces
the bogus detections to a manageable number for human scanning. During the
first three LIGO-Virgo observing runs we had a programme in place to cover the
LIGO-Virgo skymaps for optical/near-infrared counterpart searches (Smartt et
al., 2016a, b) and can typically cover 500-1000 square degrees per night
multiple times with one Pan-STARRS telescope. At the time of GW190425, we were
using Pan-STARRS1 (PS1) as the primary search facility for optical
counterparts to GW sources.

GW190425 was discovered at MJD=58598.34589 or 2019-04-25 08:18:05 UT (data
from LIGO Scientific Collaboration, & the Virgo Collaboration, 2019b), and
announced publicly with an initial localisation skymap in a Gamma-ray
coordination network (GCN) circular 95 minutes later at MJD=58598.4118 (LIGO
Scientific Collaboration, & the Virgo Collaboration, 2019a). We began
observing the initial LIGO-Virgo bayestar.fits skymap (Singer & Price, 2016)
with PS1 beginning at MJD=58598.40265, or 1.36 hrs hours after the BNS merger
time. A series of dithered and overlapping 45 sec exposures were taken in the
$i$-band over a period of 2hrs. The images were typically taken in a set of
$4\times 45$ sec, separated across 1hr or so to identify and remove moving
objects. The images can either be processed individually or co-added to create
a single stacked image for each Pan-STARRS skycell (Chambers et al., 2016).
Standard processing as described above was immediately carried out on all the
individual 45 sec exposurs and these had typical limiting magnitudes of
$i_{\rm P1}$$>21.3\pm 0.3$ AB. These values represent the median of the
3.5$\sigma$ upper limits of the processed skycells and the standard deviation
of the sample. While we used the bayestar.fits skymap to define our pointings
at the time of the event, an updated map is available from the GWTC-2 release
(Abbott et al., 2021) and all probability sky coverage in this paper is with
respect to that map
(gw190425z$\\_$skymap.multiorder.fits)222gw190425z$\\_$skymap.multiorder.fits
is available on https://gracedb.ligo.org/superevents/S190425z/view/.

On this first night we covered 1266 square degrees of the GWTC-2 skymap, which
corresponds to a cumulative probability coverage of 23.8 per cent. This was
incremented over the first 3 days to cover 1374 square degrees (24.9 per
cent). At the time, we released twenty five transients detected in the skymap
(Smith et al., 2019b, c) but none of these emerged as a compelling candidate
for an optical counterpart of GW190425 lying in a host galaxy within the GW
constrained redshift range. A number of candidates were followed up, and
PS19qp showed a red continuum (Nicholl et al., 2019b; Jonker et al., 2019) but
it was subsequently classified as a type Ic supernova suffering significant
extinction (Morokuma et al., 2019; Dimitriadis et al., 2019; McCully et al.,
2019). Figure 1 shows the Pan-STARRS1 skymap coverage over the first three
days.

In the context of FRB190425 and its most likely host galaxy UGC 10667
(RA=255.662479, DEC=21.576746), we covered the position of this galaxy with
PS1 within a few hours of both the FRB and the GW signals. A single 45 sec
$i_{\rm P1}$-band image was taken at 58598.5957604, which is 6.0 hrs after
GW190425 merger time. The image is of good quality, taken at an airmass of
1.04 and with image FWHM of 0.97″. The image is shown in Figure 4 along with
the PS1 3$\pi$ reference image (Chambers et al., 2016) and the difference
image created from subtracting the latter from the target image. There is no
transient source visible in the PS1 images either in or around UGC 10667.
There is a residual at the core of the galaxy which is almost certainly an
image subtraction artefact and this is visible in historic monitoring of this
sky region. We estimate a 3.5$\sigma$ limit of in the skycell of the image
containing UGC 10667 to be $i>21.6$. The limit is estimated using the method
described in Magnier et al. (2020b), in which the the flux and variance images
are smoothed with a circularly symmetric Gaussian kernel and a significance
image in signal-to-noise units is generated. A manual check of the background
sky noise in a PSF aperture and locating the faintest sources deteted and
visible in the image yields a detection limit of between $i_{\rm P1}$
$>21.5\,{\rm to}\,21.8$, corroborating the PS1 processing method. The circular
radius around the core of UGC 10667 which contains good and complete pixel
data is 32$\aas@@fstack{\prime\prime}$0, or a projected galactocentric
distance in the sky plane of the galaxy of $R_{\rm g}=20.5$ kpc. Therefore we
can say with reasonable confidence that there is no optical transient within
20.5 kpc of UGC 10667 to a limiting mag of $i_{\rm P1}$$>21.6$, at 6hrs after
GW190425 merger time. We revisited this sky region +13.2 days and +35.2 days
later during routine PS1 sky survey operations. A quad of images was taken on
each occasion ($4\times 45$ sec), in the $w_{\rm P1}$ filter and these were
combined to create a nightly stack (a 180 sec exposure). No transient source
was detected in the difference images to $w_{\rm P1}$$>23.5$ and $w_{\rm
P1}$$>22.6$ on either night respectively. A summary of the limiting magnitudes
is listed in Table 1. Although a projected offset of 20 kpc would enclose most
short GRBs (Fong et al., 2022) and the few candidate kilonovae known, the
recent GRB230307A and its associated kilonova was observed at an offset of 40
kpc from its likely host (Levan et al., 2023; Yang et al., 2023; Gillanders et
al., 2023). The ATLAS data described in Section 2.2 do not suffer from this
pixel chip gap issue.

Panther et al. (2023) highlight six other galaxies that they estimate had a
non-zero probability of being the candidate host of FRB 190425, in their
methodology. The probabilities of any of them being the host ranged from 1 to
3.3 per cent (Table 1 in Panther et al., 2023). We covered all apart from
WISEAJ170930.73+213633.8 with PS1 imaging on the first night and no positive
and significant transient sources were detected in the difference images. No
automated detections were found and all images were inspected visually.
WISEAJ170930.73+213633.8 (probability of being the host of FRB 190425, $P_{\rm
PATH}=0.0311$) fell on a chip gap and no definitive conclusion on transient
emission can be drawn.

###  2.2 ATLAS observations and data

At the time of writing, ATLAS is operating as a four telescope survey system
with identical units in Haleakala and Mauna Loa (in Hawaii), El Sauce (Chile)
and Sutherland (South Africa). However during the O3 observing run, the two
operational telescopes were the northern units. As described in Tonry et al.
(2018), each ATLAS unit is a "Wright Schmidt" type telescope with a 0.65m
primary and a Schmidt corrector providing a 0.5m clear aperture. The detectors
are STA-1600 CCDs, which are arrays of 10560$\times$10560 9$\mu$m pixels. The
pixel scale of 1$\aas@@fstack{\prime\prime}$89 gives a field-of-view of 28.9
square degrees for each camera. In normal survey mode in 2019 we were
typically covering the sky north of $\delta>-45^{\circ}$ every 2 nights.
During the O2 and O3 observing runs we frequently adjusted the ATLAS survey
schedule to promptly cover GW maps, with no loss to the primary near-Earth
asteroid mission. We discovered the fast transient ATLAS17aeu which turned out
to be a GRB afterglow, coincidentally in the skymap of the binary black hole
merger GW170104 (Stalder et al., 2017)

After the GW190425 alert, we scheduled sequences of 30 sec images in the ATLAS
$o$-band, and at each pointing position a sequence of quads (4$\times$30 sec)
was taken. A summary of our observations was posted by McBrien et al. (2019).
The images were processed with the ATLAS pipeline and reference images
subtracted from each one Tonry et al. (2018). Transient candidates were run
through our standard filtering procedures within the ATLAS Transient Science
server (Smith et al., 2020). After quality control filters and real-bogus
labelling with machine learning algorithms, candidates were spatially cross-
matched with known minor planets, and star, galaxy, AGN and multi-wavelength
catalogues. We began observing the northern part of the skymap within the
first hour of the preliminary notice. During the first night, ATLAS covered
2799 sq. degrees of the 90 per cent credible region of the GWTC-2 skymap and
covered a sky region totalling of 32.7 per cent of the probability area. By
the third night of observing this was incremented to 4560 sq degrees and 41.6
per cent. The ATLAS coverage of the skymap is presented in Fig. 2 and the
5$\sigma$ limiting magnitudes of the individual exposures are illustrated in
Fig. 3. The median and standard deviation of the limiting magnitudes are
$o>19.2\pm 0.3$.

In McBrien et al. (2019), we flagged 25 transients but all were either already
known, or we detected previous flux in our own forced photometry in images
before the merger. No further convincing counterpart candidates were found
brighter than $o\sim 19.5$ which were plausibly associated with a galaxy
within 100-200 Mpc (i.e. less than 50 kpc separation). We reported 5 marginal
candidates and noted they required independent confirmation (McBrien et al.,
2019), but all five were not recovered by other surveys and therefore were
likely noise artefacts (e.g. Nicholl et al., 2019a).

In the context of FRB190425 and its most likely host galaxy UGC 10667, we
covered the position of this galaxy with ATLAS within 6.9 hours after the GW
signal. A single 30 sec exposure covered the coordinates of UGC 10667 (the
quad was not completed at this sky position) and no transient flux is observed
to a limiting magnitude of $o>18.2$ (this was one of the poorer images on the
night of 58598).

ATLAS also covered this position at +1.23 days after GW190425 and this time
the $4\times 30$ $o$-quad was completed in good conditions. The 4 separate
difference image frames were co-added and no transient is visible within
several arc minutes of UGC 10667 to a 3.5$\sigma$ limiting magnitude of
$o>20.6$. A summary of epoch and observations is given in Table 1.

###  2.3 Zwicky Transient Facility observations and public data of UGC 10667

The Zwicky Transient Facility (ZTF) observed the skymap in some of its public
survey mode as described in (Coughlin et al., 2019). No transient source is
found within 30 arcseconds of UGC 10667 in the public stream ingested by the
Lasair broker333https://lasair-ztf.lsst.ac.uk (Smith et al., 2019a). The ZTF
survey (Bellm et al., 2019) allows forced photometry to be run at any position
in the public data and the images to be requested (Masci et al., 2023). We
forced photometry at five positions at and around UGC 10667 on 3 nights after
the time of GW190425 (58598.414 to 58600.393) and inspected the difference
images. No source was detected apart from what appears to be a difference
image residual at the core of UGC 10667 on 58599.397, similar to the PS1
residual in Figure 4. The 3.5$\sigma$ limits are listed in Table 1.

Table 1: The 3.5$\sigma$ limits of the Pan-STARRS1, ATLAS and ZTF images of UGC 10667 around the time of GW190425. The epoch refers to the time of the image compared to the merger time of GW190425, in units of days. MJD | Epoch | Telescope | ExpTime | Filter | limit
---|---|---|---|---|---
58598.414317 | +0.06775 | ZTF | 30s | $g$ | $>20.6$
58598.445932 | +0.09936 | ZTF | 60s | $r$ | $>21.0$
58598.595760 | +0.24919 | PS1 | 45s | $i_{\rm P1}$ | $>21.6$
58598.635555 | +0.28899 | ATLAS | 30s | $o$ | $>18.2$
58599.254896 | +0.90833 | ZTF | 90s | $g$ | $>21.3$
58599.396968 | +1.05040 | ZTF | 90s | $r$ | $>20.4$
58599.576363 | +1.22979 | ATLAS | 120s | $o$ | $>20.6$
58600.392824 | +2.04625 | ZTF | 30s | $g$ | $>21.1$
58611.567044 | +13.22047 | PS1 | 180s | $w_{\rm P1}$ | $>23.4$
58633.524370 | +35.17780 | PS1 | 180s | $w_{\rm P1}$ | $>22.6$
Figure 5: The upper limits measured by PS1, ATLAS and ZTF in images taken of
UGC 10667 are plotted as inverted triangles. The $w_{\rm P1}$ filter is very
close to $r$-band and are plotted in the same colour. Three models of kilonova
emission are plotted. The dotted and dashed lines are radioactively powered
kilonova emission of GW170817 (AT2017gfo) and the fiducial model of ejecta
mass of GW190425 estimated by Nicholl et al. (2021), see Section 3.1. A more
luminous model with emission enhanced by a magnetar as described in Section
3.2 is the solid line with model uncertainty regions. The models are
calculated assuming a distance to UGC 10667 or UGC 10667 and the Milky Way
foreground extinction as discussed in the text.

##  3 Constraints on kilonova emission

The general constraints on kilonova emission across the sky area covered
jointly by Pan-STARRS and ATLAS are inconclusive, given that we covered 24.9
percent ($i_{\rm P1}$$>21.3\pm 0.3$) and 41.2 percent ($o\gtrsim 19.0$)
integrated probability respectively. At the estimated distance of GW190425 of
$D_{\rm L}=$$159^{+69}_{-72}$ Mpc (Abbott et al., 2020), the PS1 data
correspond to absolute magnitudes limits of $M_{i}\gtrsim-14.7^{-0.9}_{+1.3}$,
assuming negligible extinction and combining the standard deviation of the
limits with the distance uncertainty in quadrature. For ATLAS the obsevational
constraints of ($o>19.2\pm 0.3$) corresponds to absolute magnitudes of
$M_{o}\gtrsim-16.8^{-0.9}_{+1.3}$.

The Zwicky Transient Facility covered 46 per cent of the initial skymap and 21
per cent of the final skymap to magnitudes $g,r\sim 21$ (Coughlin et al.,
2019). At the distance of $D_{\rm L}=$$141\pm 10$ Mpc there are plausible
models of kilonova emission (calculated with varying ejecta masses and
electron fractions) that would go undetected at the limits of Pan-STARRS, ZTF
and ATLAS (e.g. Bulla, 2019; Nicholl et al., 2021).

Other searches for counterparts were similar to, or less constraining than,
the Pan-STARRS+ATLAS+ZTF combination in their coverage of the skymap (e.g.
Hosseinzadeh et al., 2019; Lundquist et al., 2019; Antier et al., 2020;
Gompertz et al., 2020). There is little quantitative and meaningful limits
that can be placed on the emission of a kilonova from this single event given
the observing constraints.

###  3.1 Constraints on kilonova emission specifically in UGC 10667

We can directly and quantitatively assess the plausibility of optical emission
from GW190425 if FRB190425 is associated with the GW emission and if UGC 10667
is the host galaxy as proposed by Moroianu et al. (2023) and Panther et al.
(2023).

To assess the significance of our non-detections of any optical emission from
UGC 10667, we compare to a range of representative kilonova light curve models
generated using mosfit (Guillochon et al., 2018; Villar et al., 2017; Nicholl
et al., 2021); these are shown in Figure 5. Models are calculated in the ATLAS
and Pan-STARRS filters and at the distance of UGC 10667, adopting also the
foreground reddening from NED. The simplest comparison is with the well-
sampled, nearby kilonova AT2017gfo (from GW 170817). For this we use the best
fitting parameters from Nicholl et al. (2021) changing only the distance and
extinction. The reader is referred to Nicholl et al. (2021) for details of the
model assumptions. The Pan-STARRS 6.0 hrs limit and the ATLAS limit at +1.22
days both disfavour kilonova emission similar to that predicted for the
GW170817 model (which matches the AT2017gfo data well).

The GW190425 signal favoured a more massive merger than GW170817, and indeed
more massive than any known Galactic NS binary, with a chirp mass
$\mathcal{M}=1.44$ M⊙ (total mass $\approx 3.4$ M⊙). This suggests a kilonova
model calculated specifically for GW190425 may be more appropriate. We use a
BNS-informed model from Nicholl et al. (2021), with a narrow Gaussian prior on
$\mathcal{M}$ and a flat prior on the mass ratio $0.8\leq q\leq 1$. We also
marginalise over uncertainties in the fraction of the remnant disk ejected
($0.1\leq\epsilon_{\rm disk}\leq 0.5$), the fraction of lanthanide-poor ejecta
from dynamical, rather than magnetic, processes ($0.5\leq\alpha\leq 1$), and
the fraction of polar ejecta heated by a gamma-ray burst jet ($0\leq\zeta_{\rm
sh}\leq 0.5$). The median light curve for GW190425 is $\approx 0.7-1$
magnitude fainter at peak than GW170817 (Nicholl et al., 2021), though the
uncertainties in parameters unconstrained by the GW signal result in a 90%
credible range spanning roughly $\pm 1$ mag around the median. The early PS1
data point rules out the median model and excludes $\approx 75\%$ of our model
realisations. However, we note that since the models include several
parameters without physically informed priors, this is not equivalent to
ruling out a kilonova at 75% confidence.

For these simplest GW170817 and GW190425 models, we have assumed a maximum
stable NS mass $M_{\rm TOV}=2.17$ M⊙ (Margalit & Metzger, 2017; Nicholl et
al., 2021). However, the time delay between the GW and FRB signals favours a
substantially larger $M_{\rm TOV}$ (Moroianu et al., 2023). A remnant NS in
uniform rotation near break-up velocity is stable against collapse if its
gravitational mass $M_{\rm rem}>1.2M_{\rm TOV}$. Thus to avoid prompt collapse
of the rather massive remnant $M_{\rm rem}\approx 3.2$ M⊙ (Abbott et al.,
2020), an association between the GW and FRB signals requires $M_{\rm
TOV}>2.6$ M⊙ (Moroianu et al., 2023). The authors also note that if the FRB
results from the collapse of the remnant to a black hole (i.e., the remnant is
not unstable indefinitely), we also have $M_{\rm TOV}<3.1$ M⊙. Marginalising
over this uncertainty in $M_{\rm TOV}$, we find a light curve that looks
essentially unchanged during the first $\sim 1$ day (compared to the fiducial
set of models with $M_{\rm TOV}$ set at $2.17$M⊙) but is brighter by $1-2$
magnitudes during the next $\sim$ week. This is the phase when the
intermediate opacity disk wind ejecta are expected to dominate the observed
emission, and the increase in luminosity with $M_{\rm TOV}$ results from the
more massive disk wind from a longer-lived remnant. These are plotted in
Figure 5, but the data we have are not constraining at the epochs that each
set of models diverge.

###  3.2 Magnetar spin-down emission

If the link between GW190425 and FRB190425 were veracious and the physical
picture is a supramassive (rotationally-supported) NS remnant subsequently
collapsing into a black hole after 2.5 hours, the remnant must lose its
rotational energy on this timescale. Merger remnants are expected to be
rotating near break-up (Radice et al., 2018), with $P\simeq 0.7$ ms. For
remnants with $M_{\rm TOV}<M_{\rm rem}<1.2M_{\rm TOV}$, the NS initially
survives due to centrifugal support, and collapses once this is lost. Spin-
down can occur through GW emission if the remnant has a quadrupole moment.
However, we may expect that rotational energy loss is dominated by magnetic
spin-down, particularly since the merger product is expected to have a strong
magnetic field exceeding $10^{15}-10^{16}$ G (Price & Rosswog 2006; Zrake &
MacFadyen 2013; Kiuchi et al. 2023 – and assuming the field is dominated by an
ordered dipole; Dall’Osso et al. 2009). Spinning down through dipole emission
on a timescale of 2.5 hours requires only a modest $B\sim{\rm few}\times
10^{14}$ G (Moroianu et al., 2023).

The rotational energy extracted from the remnant can greatly enhance the
kilonova luminosity (e.g. Yu et al., 2013; Gao et al., 2015; Fong et al.,
2021; Sarin et al., 2022). Metzger (2019) provide an analytic model for the
luminosity resulting from dipole spin-down in a NS merger remnant, and show
that it can boost the optical emission very significantly, by up to $\approx
4$ magnitudes. We refer the reader to that work for details, but in brief this
model takes into account the typical dipole spin-down formula for the
evolution of the spin period and magnetar luminosity often applied to
supernova remnants (e.g. Kasen & Bildsten, 2010; Woosley, 2010), and modifies
it by a thermalisation efficiency (close to unity at early times) and the
energy removed by electron-positron pair-creation at late times (Metzger &
Piro, 2014; Kasen et al., 2016). The input luminosity goes to zero as soon as
the remnant collapses.

We have created a mosfit module to calculate magnetar-powered kilonova light
curves using this framework. We assume a thermalisation efficiency
$\epsilon_{\rm th}=1$, an albedo of 0.5 for the pair cascade and a pair
multiplicity of 0.1 (as formulated in Metzger, 2019). The resulting energy
injected by the magnetar as it spins down is converted to an output optical
luminosity using the usual Arnett (1982) model employed by mosfit. We have
verified that this produces light curves in very good agreement with Metzger
(2019). We fix the initial spin period at 0.7 ms (i.e. maximal spin), as
expected from simulations. As the magnetar spins down, we compute the
rotational energy at the time of collapse to a black hole following Margalit &
Metzger (2017). The energy available to power the transient is the difference
between the initial rotational energy and that at collapse.

In the case of FRB190425, we fix $B=1.8\times 10^{14}$ G to give the
appropriate time to collapse (Moroianu et al., 2023), resulting in no
additional free parameters compared to the radioactive kilonova models. We
marginalise over the chirp mass, mass ratio and ejecta parameters with the
same priors as before. This produces a luminous light curve, at all times
brighter than the other models, and peaking later at $\sim 19$ mag around 5-7
days after merger. The median model is strongly disfavoured by our Pan-STARRS
and ATLAS observations of UGC 10667. Furthermore, in this case the credible
range of the models does not overlap with our observational limits.

We return to the quesition of what this can tell us about the plausibility of
the GW190425 and FRB190425 connection. If the FRB-GW connection were true, and
if the merger did occur in UGC 10667 then we can exclude, with high
confidence, that the merger produced a supramassive NS, spinning down by
dipole emission on a timescale of hours. The working model to produce
FRB190425 from a BNS merger, as proposed by Moroianu et al. (2023) is that of
a supramassive neutron star that is highly magnetized (Zhang, 2014; Falcke &
Rezzolla, 2014). Hence the lack of detected optical emission disfavours but
does not disprove, the FRB-GW link. Were such an FRB producing remnant formed,
then we have shown it would likely have produced detectable optical emission.
Bhardwaj et al. (2023) propose that the FRB-GW association is unlikely as they
find that a very low ejecta mass is required in order for the 400 MHz fluz to
propogate through the material ejected in the merger and that the viewing
angle requriements from the FRB and GW data are inconsistent.

##  4 Conclusions

We promptly observed the LIGO-Virgo skymap of the BNS merger event GW190425
with Pan-STARRS and ATLAS beginning several hours after the merger event. With
Pan-STARRS1 we managed to cover a total integrated probability area of 24.9
per cent (to limiting magnitudes of $i_{\rm P1}$$>21.3\pm 0.3$) over the first
three days and with ATLAS we covered 41.2 per cent ($o>19.2\pm 0.3$). These
correspond to absolute magnitudes of $M_{i}\gtrsim-14.7^{-0.9}_{+1.3}$ and
$M_{o}\gtrsim-16.8^{-0.9}_{+1.3}$. (assuming negligible extinction) with the
errors dominated by the uncertainy in the distance to GW190425. The physical
limits on an electromagnetic counterpart to GW190425 are not strong, given
that approximately half the sky map was unobservable due to solar conjunction
- a problem that affected all wide-field searches for optical counterparts.
However they do show the joint capability of the ATLAS and Pan-STARRS systems
for gravitational wave followup, particularly as ATLAS is now a 4 unit system
(and all-sky) and Pan-STARRS now is a twin facility on Haleakala.

A recent proposed connection between GW190425 and the fast radio burst FRB
190425 has emerged with a temporal and spatial coincidence found by Moroianu
et al. (2023). If this association were to be physically true then it implies
that a supramassive, rapidly rotating and magnetised neutron star was formed
for at least a few hours after BNS merger (the GW and FRB signals were
separated by 2.5hrs). FRB 190425 has been pinpointed to a most probable host
galaxy, UGC 10667 which is at a compatible redshift with the distance to
GW190425 (Moroianu et al., 2023; Panther et al., 2023). With Pan-STARRS and
ATLAS, we observed this host galaxy within a few hours of the FRB and GW
signals. No optical emission was found. We calculated samples of kilonova
lightcurves with ejecta masses and radioactive heating based on the data from
AT2017gfo and the physical parameters inferred from the GW data of GW190425
(Nicholl et al., 2021). The Pan-STARRS limiting magnitude of $i_{\rm P1}>21.6$
at +0.25 days after GW190425 merger time, precludes an AT2017gfo type of
kilonova and marginally disfavours a fiducial kilonova model based on the
GW190425 data.

The magnetised, rotating neutron star required to explain the FRB emission has
a magnetic field of $B\sim 1.8\times 10^{14}$ G which would result in an
enhancement of the kilonova luminosity by magnetar powering. We calculate such
models by fixing the magnetic field to that required by the FRB and
marginalising over chirp mass, mass ratio and ejecta parameters. The rather
luminous optical lightcurves are all ruled out by the limits from Pan-STARRS1
and ATLAS within the first +1.2 days from merger. This excludes a supramassive
NS, spinning down by dipole emission on a timescale of hours. The lack of
detected optical emission disfavours, but does not disprove, the FRB-GW link.
If such an FRB-GW link were proven in the future (Moroianu et al., 2023) then
the FRB sky localisation and potential for immediate identification of a host
galaxy (Panther et al., 2023) would be an extremely promising route to advance
multi-messenger astronomy and further such coincidences should be searched
for.

## Acknowledgements

Pan-STARRS is primarily funded to search for near earth asteroids through NASA
grants NASA Grants NNX08AR22G, NNX14AM74G. The Pan-STARRS science products for
LIGO-Virgo follow-up made possible through the contributions of the University
of Hawaii Institute for Astronomy and the Queen’s University Belfast. The Pan-
STARRS1 Sky Surveys have been made possible through contributions by the
Institute for Astronomy, the University of Hawaii, the Pan-STARRS Project
Office, the Max Planck Society and its participating institutes, the Max
Planck Institute for Astronomy, Heidelberg and the Max Planck Institute for
Extraterrestrial Physics, Garching, The Johns Hopkins University, Durham
University, the University of Edinburgh, the Queen’s University Belfast, the
Harvard-Smithsonian Center for Astrophysics, the Las Cumbres Observatory
Global Telescope Network Incorporated, the National Central University of
Taiwan, the Space Telescope Science Institute, and the National Aeronautics
and Space Administration under Grant No. NNX08AR22G issued through the
Planetary Science Division of the NASA Science Mission Directorate, the
National Science Foundation Grant No. AST-1238877, the University of Maryland,
Eotvos Lorand University (ELTE), and the Los Alamos National Laboratory.

ATLAS is primarily funded to search for near earth asteroids through NASA
grants NN12AR55G, 80NSSC18K0284, and 80NSSC18K1575; byproducts of the NEO
search include images and catalogs from the survey area. The ATLAS science
products have been made possible through the contributions of the University
of Hawaii Institute for Astronomy, the Queen’s University Belfast, the Space
Telescope Science Institute, and the South African Astronomical Observatory.

Lasair is supported by the UKRI Science and Technology Facilities Council and
is a collaboration between the University of Edinburgh (grant ST/N002512/1)
and Queen’s University Belfast (grant ST/N002520/1) within the LSST:UK Science
Consortium. ZTF is supported by National Science Foundation grant AST-1440341
and a collaboration including Caltech, IPAC, the Weizmann Institute for
Science, the Oskar Klein Center at Stockholm University, the University of
Maryland, the University of Washington, Deutsches Elektronen-Synchrotron and
Humboldt University, Los Alamos National Laboratories, the TANGO Consortium of
Taiwan, the University of Wisconsin at Milwaukee, and Lawrence Berkeley
National Laboratories. Operations are conducted by COO, IPAC, and UW. This
research has made use of “Aladin sky atlas’‘ developed at CDS, Strasbourg
Observatory, France.

SJS, SS, SAS, LS, KS, DY acknowledge UKRI STFC grants ST/X006506/1,
ST/T000198/1, ST/S006109/1 and ST/X001253/1. MN, SS, AA and XS are supported
by the European Research Council (ERC) under the European Union’s Horizon 2020
research and innovation programme (grant agreement No. 948381). MN also
acknowledges UK Space Agency Grant No. ST/Y000692/1. TWC acknowledges the
Yushan Young Fellow Program by the Ministry of Education, Taiwan for the
financial support. LJS acknowledges support by the European Research Council
(ERC) under the European Union’s Horizon 2020 research and innovation program
(ERC Advanced Grant KILONOVA No. 885281) and support by Deutsche
Forschungsgemeinschaft (DFG, German Research Foundation) - Project-ID
279384907 - SFB 1245 and MA 4248/3-1. This work was funded by ANID, Millennium
Science Initiative, ICN12_009 CWS is grateful for support from Harvard
University. This research has made use of the NASA/IPAC Extragalactic Database
(NED), which is operated by the Jet Propulsion Laboratory, California
Institute of Technology, under contract with the National Aeronautics and
Space Administration. For the purpose of open access, a Creative Commons
Attribution (CC BY 4.0) licence will apply to any Author Accepted Manuscript
version arising.

## Data Availability

The data table for the ATLAS and Pan-SATRRS exposures (to produce Figures 1
and 2), the pixel images for Figure 3 will be made available when accepted
either on the MNRAS website or on Research Data Oxford. The code for the
models to produce Figure 5 is available on the MOSFiT
https://github.com/guillochon/MOSFiT.

## References

  * Abbott et al. (2017a) Abbott B. P., et al., 2017a, Phys. Rev. Lett., 119, 161101
  * Abbott et al. (2017b) Abbott B. P., et al., 2017b, ApJ, 848, L12
  * Abbott et al. (2017c) Abbott B. P., et al., 2017c, ApJ, 848, L13
  * Abbott et al. (2020) Abbott B. P., et al., 2020, ApJ, 892, L3
  * Abbott et al. (2021) Abbott R., et al., 2021, Physical Review X, 11, 021053
  * Albareti et al. (2017) Albareti F. D., et al., 2017, ApJS, 233, 25
  * Alexander et al. (2017) Alexander K. D., et al., 2017, ApJ, 848, L21
  * Andreoni et al. (2017) Andreoni I., et al., 2017, Publ. Astron. Soc. Australia, 34, e069
  * Antier et al. (2020) Antier S., et al., 2020, MNRAS, 497, 5518
  * Arcavi et al. (2017) Arcavi I., et al., 2017, Nature, 551, 64
  * Arnett (1982) Arnett W. D., 1982, ApJ, 253, 785
  * Bellm et al. (2019) Bellm E. C., et al., 2019, PASP, 131, 018002
  * Bhardwaj et al. (2023) Bhardwaj M., Palmese A., Magaña Hernandez I., D’Emilio V., Morisaki S., 2023, arXiv e-prints, p. arXiv:2306.00948
  * Boersma et al. (2021) Boersma O. M., et al., 2021, A&A, 650, A131
  * Bulla (2019) Bulla M., 2019, MNRAS, 489, 5037
  * CHIME/FRB Collaboration et al. (2021) CHIME/FRB Collaboration et al., 2021, ApJS, 257, 59
  * Chambers et al. (2016) Chambers K. C., et al., 2016, preprint,  (arXiv:1612.05560)
  * Chornock et al. (2017) Chornock R., et al., 2017, ApJ, 848, L19
  * Cooper et al. (2023) Cooper A. J., Gupta O., Wadiasingh Z., Wijers R. A. M. J., Boersma O. M., Andreoni I., Rowlinson A., Gourdji K., 2023, MNRAS, 519, 3923
  * Coughlin et al. (2019) Coughlin M. W., et al., 2019, ApJ, 885, L19
  * Coulter et al. (2017) Coulter D. A., et al., 2017, Science, 358, 1556
  * Cowperthwaite et al. (2017) Cowperthwaite P. S., et al., 2017, ApJ, 848, L17
  * Dall’Osso et al. (2009) Dall’Osso S., Shore S. N., Stella L., 2009, MNRAS, 398, 1869
  * Dimitriadis et al. (2019) Dimitriadis G., et al., 2019, GRB Coordinates Network, 24358, 1
  * Drout et al. (2017) Drout M. R., et al., 2017, Science, 358, 1570
  * Evans et al. (2017) Evans P. A., et al., 2017, Science, 358, 1565
  * Falcke & Rezzolla (2014) Falcke H., Rezzolla L., 2014, A&A, 562, A137
  * Fong et al. (2021) Fong W., et al., 2021, ApJ, 906, 127
  * Fong et al. (2022) Fong W.-f., et al., 2022, ApJ, 940, 56
  * Gao et al. (2015) Gao H., Ding X., Wu X.-F., Dai Z.-G., Zhang B., 2015, ApJ, 807, 163
  * Gillanders et al. (2023) Gillanders J. H., et al., 2023, arXiv e-prints, p. arXiv:2308.00633
  * Gompertz et al. (2020) Gompertz B. P., et al., 2020, MNRAS, 497, 726
  * Guillochon et al. (2018) Guillochon J., Nicholl M., Villar V. A., Mockler B., Narayan G., Mandel K. S., Berger E., Williams P. K. G., 2018, ApJS, 236, 6
  * Haggard et al. (2017) Haggard D., Nynka M., Ruan J. J., Kalogera V., Cenko S. B., Evans P., Kennea J. A., 2017, ApJ, 848, L25
  * Hallinan et al. (2017) Hallinan G., et al., 2017, Science, 358, 1579
  * Hosseinzadeh et al. (2019) Hosseinzadeh G., et al., 2019, ApJ, 880, L4
  * Jonker et al. (2019) Jonker P., et al., 2019, GRB Coordinates Network, 24221, 1
  * Kasen & Bildsten (2010) Kasen D., Bildsten L., 2010, ApJ, 717, 245
  * Kasen et al. (2016) Kasen D., Metzger B. D., Bildsten L., 2016, ApJ, 821, 36
  * Kasliwal et al. (2017) Kasliwal M. M., et al., 2017, Science, 358, 1559
  * Kilpatrick et al. (2017) Kilpatrick C. D., et al., 2017, preprint,  (arXiv:1710.05434)
  * Kiuchi et al. (2023) Kiuchi K., Reboul-Salze A., Shibata M., Sekiguchi Y., 2023, arXiv e-prints, p. arXiv:2306.15721
  * LIGO Scientific Collaboration, & the Virgo Collaboration (2019a) LIGO Scientific Collaboration, the Virgo Collaboration 2019a, GRB Coordinates Network, 24168
  * LIGO Scientific Collaboration, & the Virgo Collaboration (2019b) LIGO Scientific Collaboration, the Virgo Collaboration 2019b, GW190425 Data Set, version 1, doi:10.7935/ggb8-1v94
  * Levan et al. (2023) Levan A., et al., 2023, arXiv e-prints, p. arXiv:2307.02098
  * Lipunov et al. (2017) Lipunov V. M., et al., 2017, ApJ, 850, L1
  * Lundquist et al. (2019) Lundquist M. J., et al., 2019, ApJ, 881, L26
  * Magnier et al. (2020a) Magnier E. A., et al., 2020a, ApJS, 251, 3
  * Magnier et al. (2020b) Magnier E. A., et al., 2020b, ApJS, 251, 5
  * Magnier et al. (2020c) Magnier E. A., et al., 2020c, ApJS, 251, 6
  * Margalit & Metzger (2017) Margalit B., Metzger B. D., 2017, ApJ, 850, L19
  * Margutti et al. (2017) Margutti R., et al., 2017, ApJ, 848, L20
  * Masci et al. (2023) Masci F. J., et al., 2023, arXiv e-prints, p. arXiv:2305.16279
  * McBrien et al. (2019) McBrien O., et al., 2019, GRB Coordinates Network, 24197, 1
  * McCully et al. (2017) McCully C., et al., 2017, ApJ, 848, L32
  * McCully et al. (2019) McCully C., et al., 2019, GRB Coordinates Network, 24295, 1
  * Metzger (2019) Metzger B. D., 2019, Living Reviews in Relativity, 23, 1
  * Metzger & Piro (2014) Metzger B. D., Piro A. L., 2014, MNRAS, 439, 3916
  * Moroianu et al. (2023) Moroianu A., Wen L., James C. W., Ai S., Kovalam M., Panther F. H., Zhang B., 2023, Nature Astronomy, 7, 579
  * Morokuma et al. (2019) Morokuma T., et al., 2019, GRB Coordinates Network, 24230, 1
  * Nicholl et al. (2017) Nicholl M., et al., 2017, ApJ, 848, L18
  * Nicholl et al. (2019a) Nicholl M., et al., 2019a, GRB Coordinates Network, 24211, 1
  * Nicholl et al. (2019b) Nicholl M., et al., 2019b, GRB Coordinates Network, 24217, 1
  * Nicholl et al. (2021) Nicholl M., Margalit B., Schmidt P., Smith G. P., Ridley E. J., Nuttall J., 2021, MNRAS, 505, 3016
  * Panther et al. (2023) Panther F. H., et al., 2023, MNRAS, 519, 2235
  * Paterson et al. (2021) Paterson K., et al., 2021, ApJ, 912, 128
  * Pian et al. (2017) Pian E., et al., 2017, Nature, 551, 67
  * Planck Collaboration et al. (2016) Planck Collaboration et al., 2016, A&A, 594, A13
  * Price & Rosswog (2006) Price D. J., Rosswog S., 2006, Science, 312, 719
  * Radice et al. (2018) Radice D., Perego A., Bernuzzi S., Zhang B., 2018, MNRAS, 481, 3670
  * Ravi (2019) Ravi V., 2019, MNRAS, 482, 1966
  * Sarin et al. (2022) Sarin N., Omand C. M. B., Margalit B., Jones D. I., 2022, MNRAS, 516, 4949
  * Schlafly & Finkbeiner (2011) Schlafly E. F., Finkbeiner D. P., 2011, ApJ, 737, 103
  * Shappee et al. (2017) Shappee B. J., et al., 2017, Science, 358, 1574
  * Singer & Price (2016) Singer L. P., Price L. R., 2016, Phys. Rev. D, 93, 024013
  * Smartt et al. (2016a) Smartt S. J., et al., 2016a, MNRAS, 462, 4094
  * Smartt et al. (2016b) Smartt S. J., et al., 2016b, ApJ, 827, L40
  * Smartt et al. (2017) Smartt S. J., et al., 2017, Nature, 551, 75
  * Smith et al. (2019a) Smith K. W., et al., 2019a, Research Notes of the American Astronomical Society, 3, 26
  * Smith et al. (2019b) Smith K. W., et al., 2019b, GRB Coordinates Network, 24210, 1
  * Smith et al. (2019c) Smith K. W., et al., 2019c, GRB Coordinates Network, 24262, 1
  * Smith et al. (2020) Smith K. W., et al., 2020, PASP, 132, 085002
  * Soares-Santos et al. (2017) Soares-Santos M., et al., 2017, ApJ, 848, L16
  * Stalder et al. (2017) Stalder B., et al., 2017, ApJ, 850, 149
  * Tanvir et al. (2017) Tanvir N. R., et al., 2017, ApJ, 848, L27
  * Tonry et al. (2012) Tonry J. L., et al., 2012, ApJ, 745, 42
  * Tonry et al. (2018) Tonry J. L., et al., 2018, Publications of the Astronomical Society of the Pacific, 130, 064505
  * Totani (2013) Totani T., 2013, PASJ, 65, L12
  * Troja et al. (2017) Troja E., et al., 2017, Nature, 551, 71
  * Utsumi et al. (2017) Utsumi Y., et al., 2017, PASJ, 69, 101
  * Valenti et al. (2017) Valenti S., et al., 2017, ApJ, 848, L24
  * Villar et al. (2017) Villar V. A., et al., 2017, ApJ, 851, L21
  * Waters et al. (2020) Waters C. Z., et al., 2020, ApJS, 251, 4
  * Woosley (2010) Woosley S. E., 2010, ApJ, 719, L204
  * Wright et al. (2015) Wright D. E., et al., 2015, MNRAS, 449, 451
  * Yang et al. (2023) Yang Y.-H., et al., 2023, arXiv e-prints, p. arXiv:2308.00638
  * Yu et al. (2013) Yu Y.-W., Zhang B., Gao H., 2013, ApJ, 776, L40
  * Zhang (2014) Zhang B., 2014, ApJ, 780, L21
  * Zrake & MacFadyen (2013) Zrake J., MacFadyen A. I., 2013, ApJ, 769, L29


maggic (n=40) | $67.50_{(0.04)}$ | $58.26_{(0.55)}$ | $59.29_{(0.50)}$ | $\textbf{60.77}_{(0.36)}$ | $57.50_{(0.69)}$ | $59.15_{(0.48)}$ | $55.00_{(1.27)}$ | $56.78_{(1.17)}$ | $55.24_{(0.63)}$ | $56.94_{(0.50)}$ | $48.81_{(0.73)}$ | $56.64_{(0.63)}$ | $54.68_{(0.53)}$ | $58.58_{(0.54)}$ | $57.40_{(0.57)}$ | $57.44_{(0.67)}$ | $55.04_{(1.14)}$ | $57.33_{(1.06)}$
seer (n=40) | $87.92_{(0.08)}$ | $82.93_{(0.55)}$ | $84.29_{(0.39)}$ | $\textbf{84.93}_{(0.46)}$ | $83.46_{(1.20)}$ | $84.44_{(0.69)}$ | $80.05_{(0.93)}$ | $83.67_{(0.51)}$ | $82.59_{(1.48)}$ | $81.37_{(1.11)}$ | $54.93_{(2.06)}$ | $81.11_{(1.24)}$ | $79.88_{(0.53)}$ | $84.36_{(0.62)}$ | $80.79_{(0.67)}$ | $82.21_{(0.67)}$ | $78.69_{(2.32)}$ | $83.62_{(1.06)}$
compas (n=40) | $67.51_{(0.03)}$ | $62.34_{(0.79)}$ | $67.57_{(0.40)}$ | $\textbf{67.85}_{(0.37)}$ | $61.34_{(1.67)}$ | $62.84_{(1.21)}$ | $56.29_{(1.96)}$ | $61.02_{(1.69)}$ | $58.85_{(1.28)}$ | $60.11_{(1.22)}$ | $58.88_{(1.04)}$ | $64.37_{(0.91)}$ | $58.61_{(1.35)}$ | $63.54_{(1.11)}$ | $60.83_{(1.19)}$ | $60.95_{(1.22)}$ | $55.94_{(1.50)}$ | $61.04_{(1.56)}$
adult (n=40) | $84.17_{(0.10)}$ | $79.44_{(1.03)}$ | $48.31_{(3.58)}$ | $73.82_{(2.19)}$ | $49.21_{(1.78)}$ | $74.27_{(1.49)}$ | $71.82_{(1.22)}$ | $79.11_{(0.71)}$ | $71.51_{(1.85)}$ | $77.99_{(0.65)}$ | $66.77_{(3.32)}$ | $78.81_{(1.36)}$ | $71.13_{(2.27)}$ | $79.71_{(1.02)}$ | $77.90_{(1.07)}$ | $78.84_{(1.19)}$ | $72.58_{(1.34)}$ | $\textbf{80.02}_{(0.76)}$
drug (n=40) | $77.81_{(0.55)}$ | $71.86_{(1.07)}$ | $74.30_{(0.59)}$ | $\textbf{75.79}_{(0.39)}$ | $71.33_{(0.88)}$ | $72.76_{(0.97)}$ | $69.46_{(2.23)}$ | $72.74_{(1.70)}$ | $71.08_{(1.72)}$ | $73.07_{(1.03)}$ | $64.89_{(1.39)}$ | $73.64_{(0.87)}$ | $62.51_{(3.03)}$ | $70.97_{(1.91)}$ | $69.23_{(1.68)}$ | $69.78_{(1.46)}$ | $65.22_{(1.37)}$ | $70.30_{(1.04)}$
covid (n=100) | $74.41_{(0.11)}$ | $71.57_{(0.48)}$ | $73.77_{(0.27)}$ | $\textbf{74.71}_{(0.34)}$ | $70.71_{(0.46)}$ | $72.76_{(0.44)}$ | $69.05_{(0.96)}$ | $72.13_{(0.66)}$ | $71.60_{(0.59)}$ | $73.22_{(0.46)}$ | $63.52_{(1.29)}$ | $72.04_{(0.57)}$ | $64.25_{(1.51)}$ | $72.64_{(0.64)}$ | $70.08_{(0.67)}$ | $70.78_{(0.59)}$ | $69.05_{(0.48)}$ | $71.96_{(0.49)}$
cutract (n=100) | $72.23_{(0.65)}$ | $70.96_{(0.68)}$ | $70.20_{(0.45)}$ | $\textbf{72.51}_{(0.55)}$ | $69.97_{(0.97)}$ | $71.94_{(0.89)}$ | $67.94_{(1.01)}$ | $72.42_{(0.66)}$ | $70.53_{(1.51)}$ | $71.98_{(1.39)}$ | $55.72_{(2.04)}$ | $69.14_{(0.93)}$ | $67.59_{(0.71)}$ | $72.42_{(0.54)}$ | $68.79_{(0.83)}$ | $69.68_{(0.76)}$ | $66.89_{(1.03)}$ | $71.52_{(0.70)}$
maggic (n=100) | $67.50_{(0.04)}$ | $59.65_{(0.50)}$ | $58.98_{(0.23)}$ | $\textbf{61.32}_{(0.42)}$ | $55.71_{(0.83)}$ | $58.90_{(0.72)}$ | $57.20_{(0.91)}$ | $59.34_{(0.64)}$ | $57.26_{(0.50)}$ | $58.28_{(0.46)}$ | $49.54_{(0.71)}$ | $57.91_{(0.74)}$ | $56.36_{(0.54)}$ | $60.11_{(0.54)}$ | $58.89_{(0.51)}$ | $58.99_{(0.42)}$ | $56.17_{(0.68)}$ | $58.86_{(0.69)}$
seer (n=100) | $87.92_{(0.08)}$ | $83.95_{(0.32)}$ | $84.45_{(0.38)}$ | $\textbf{85.37}_{(0.47)}$ | $83.92_{(0.81)}$ | $85.08_{(0.32)}$ | $81.60_{(0.73)}$ | $85.14_{(0.36)}$ | $83.04_{(0.78)}$ | $84.83_{(0.49)}$ | $70.32_{(2.52)}$ | $83.83_{(0.39)}$ | $81.16_{(1.03)}$ | $85.03_{(0.39)}$ | $81.82_{(0.45)}$ | $82.49_{(0.46)}$ | $78.88_{(0.71)}$ | $84.50_{(0.44)}$
compas (n=100) | $67.51_{(0.03)}$ | $62.56_{(0.72)}$ | $68.02_{(0.29)}$ | $\textbf{68.19}_{(0.37)}$ | $60.10_{(1.60)}$ | $62.47_{(1.11)}$ | $60.01_{(1.16)}$ | $63.73_{(0.99)}$ | $58.32_{(0.91)}$ | $61.34_{(0.94)}$ | $59.97_{(0.82)}$ | $64.19_{(0.83)}$ | $60.02_{(0.63)}$ | $64.04_{(0.60)}$ | $61.44_{(0.84)}$ | $61.73_{(0.81)}$ | $59.97_{(1.00)}$ | $62.82_{(1.05)}$
adult (n=100) | $84.17_{(0.10)}$ | $81.24_{(0.48)}$ | $46.09_{(1.86)}$ | $74.57_{(1.74)}$ | $47.56_{(3.43)}$ | $73.97_{(1.57)}$ | $74.29_{(1.23)}$ | $80.45_{(0.81)}$ | $75.93_{(1.45)}$ | $78.22_{(1.22)}$ | $77.09_{(0.74)}$ | $\textbf{81.66}_{(0.53)}$ | $70.70_{(}{1.02)}$ | $81.04_{(0.37)}$ | $80.56_{(0.53)}$ | $81.10_{(0.44)}$ | $74.04_{(1.44)}$ | $80.23_{(0.98)}$
drug (n=100) | $77.81_{(0.55)}$ | $73.58_{(0.72)}$ | $76.24_{(0.64)}$ | $\textbf{76.74}_{(0.51)}$ | $69.46_{(2.43)}$ | $71.05_{(2.30)}$ | $68.19_{(2.06)}$ | $73.28_{(1.41)}$ | $72.43_{(0.96)}$ | $73.79_{(0.65)}$ | $67.26_{(1.22)}$ | $75.28_{(0.46)}$ | $62.67_{(2.23)}$ | $73.12_{(0.97)}$ | $70.90_{(1.08)}$ | $71.53_{(1.13)}$ | $68.22_{(1.60)}$ | $73.59_{(0.97)}$
covid (n=200) | $74.41_{(0.11)}$ | $72.33_{(0.52)}$ | $73.40_{(0.25)}$ | $\textbf{74.62}_{(0.13)}$ | $70.70_{(0.73)}$ | $73.12_{(0.48)}$ | $71.07_{(0.35)}$ | $73.89_{(0.40)}$ | $72.47_{(0.50)}$ | $74.44_{(0.44)}$ | $65.55_{(0.67)}$ | $73.07_{(0.38)}$ | $65.04_{(0.69)}$ | $72.90_{(0.37)}$ | $71.68_{(0.49)}$ | $71.87_{(0.49)}$ | $67.89_{(0.56)}$ | $72.38_{(0.42)}$
cutract (n=200) | $72.23_{(0.65)}$ | $71.75_{(0.71)}$ | $71.39_{(0.76)}$ | $73.01_{(0.85)}$ | $70.28_{(0.67)}$ | $72.39_{(0.84)}$ | $69.28_{(0.55)}$ | $72.41_{(0.65)}$ | $71.83_{(0.65)}$ | $\textbf{74.03}_{(0.69)}$ | $66.66_{(0.94)}$ | $72.49_{(0.71)}$ | $68.77_{(0.71)}$ | $73.16_{(0.68)}$ | $70.23_{(0.83)}$ | $70.80_{(0.79)}$ | $66.61_{(0.76)}$ | $71.87_{(0.71)}$
maggic (n=200) | $67.50_{(0.04)}$ | $61.39_{(0.44)}$ | $58.92_{(0.47)}$ | $\textbf{61.41}_{(0.39)}$ | $57.33_{(0.51)}$ | $60.16_{(0.41)}$ | $58.48_{(0.59)}$ | $61.33_{(0.57)}$ | $56.26_{(0.82)}$ | $57.20_{(0.91)}$ | $50.74_{(0.74)}$ | $59.60_{(0.55)}$ | $55.95_{(0.72)}$ | $60.75_{(0.39)}$ | $60.73_{(0.44)}$ | $60.78_{(0.49)}$ | $57.18_{(0.56)}$ | $60.23_{(0.44)}$
seer (n=200) | $87.92_{(0.08)}$ | $84.63_{(0.35)}$ | $84.39_{(0.19)}$ | $85.56_{(0.30)}$ | $83.48_{(0.41)}$ | $84.80_{(0.34)}$ | $82.04_{(0.70)}$ | $85.34_{(0.36)}$ | $84.39_{(0.44)}$ | $\textbf{86.57}_{(0.27)}$ | $82.15_{(0.43)}$ | $86.03_{(0.23)}$ | $77.73_{(1.65)}$ | $85.19_{(0.22)}$ | $83.38_{(0.32)}$ | $84.15_{(0.42)}$ | $79.71_{(0.58)}$ | $85.26_{(0.25)}$
compas (n=200) | $67.51_{(0.03)}$ | $63.27_{(0.60)}$ | $67.02_{(0.53)}$ | $\textbf{68.15}_{(0.40)}$ | $60.48_{(1.86)}$ | $63.39_{(1.28)}$ | $60.58_{(0.83)}$ | $64.32_{(0.78)}$ | $60.60_{(0.90)}$ | $63.52_{(1.04)}$ | $61.11_{(0.63)}$ | $65.08_{(0.53)}$ | $56.58_{(1.03)}$ | $63.60_{(0.83)}$ | $61.99_{(0.64)}$ | $62.80_{(0.61)}$ | $60.15_{(1.14)}$ | $63.99_{(0.71)}$
adult (n=200) | $84.17_{(0.10)}$ | $82.12_{(0.41)}$ | $40.96_{(2.47)}$ | $75.84_{(1.43)}$ | $49.89_{(3.22)}$ | $76.11_{(1.43)}$ | $78.18_{(0.26)}$ | $82.32_{(0.29)}$ | $81.66_{(0.19)}$ | $83.17_{(0.20)}$ | $80.06_{(0.53)}$ | $\textbf{83.32}_{(0.35)}$ | $74.31_{(0.81)}$ | $82.64_{(0.33)}$ | $82.26_{(0.38)}$ | $82.39_{(0.33)}$ | $75.21_{(0.69)}$ | $82.02_{(0.25)}$
drug (n=200) | $77.81_{(0.55)}$ | $76.10_{(0.45)}$ | $75.58_{(0.55)}$ | $76.06_{(0.42)}$ | $70.66_{(1.56)}$ | $72.81_{(1.18)}$ | $71.31_{(1.17)}$ | $75.98_{(0.76)}$ | $69.61_{(2.76)}$ | $71.79_{(1.99)}$ | $72.35_{(0.61)}$ | $\textbf{77.41}_{(0.56)}$ | $65.25_{(2.47)}$ | $75.26_{(0.77)}$ | $74.38_{(0.50)}$ | $74.78_{(0.54)}$ | $68.39_{(1.26)}$ | $74.33_{(0.61)}$


# A Systematic Comparison of Architectures
for Document-Level Sentiment Classification

Jeremy Barnes, Vinit Ravishankar, Lilja Øvrelid, and Erik Velldal
University of Oslo
Department of Informatics
<EMAIL_ADDRESS>
###### Abstract

Documents are composed of smaller pieces – paragraphs, sentences, and tokens –
that have complex relationships between one another. Sentiment classification
models that take into account the structure inherent in these documents have a
theoretical advantage over those that do not. At the same time, transfer
learning models have shown promise for document classification. However, these
two paradigms have not been systematically compared and it is not clear under
which circumstances one approach is better than the other. In this work we
empirically compare flat, hierarchical, and transfer learning models for
document-level sentiment classification in five languages. We show that the
choice of model depends on data attributes and that hierarchical models
perform similar to transfer learning, even in low-resource settings.

##  1 Introduction

The inherent structure found in documents – paragraphs, sentences, and tokens
– and their inter-dependence is vital to document-level sentiment, as
rhetorical devices and anaphora relationships disperse the sentiment signal
across the various sub-components Yang and Cardie (2014). This also means that
not all sub-components contribute equally towards identifying the overall
polarity of a document Yu and Hatzivassiloglou (2003); Pang and Lee (2004) and
models that are able to take these relationships into account should
theoretically perform better.

Recently, two divergent research directions have shown promise for document-
classification: on the one hand, _transfer learning_ Peters et al. (2018);
Howard and Ruder (2018); Devlin et al. (2019) and on the other hand
_hierarchical modeling_ Xiao and Cho (2016); Conneau et al. (2017); Yang et
al. (2016). Transfer learning (in its current form) attempts to take advantage
of large amounts of unlabeled text in order to improve contextualized
representations of tokens, while ignoring the structure of documents.
Hierarchical models, on the other hand, attempt to take document structure
into account by first building up representations for sentences and then
aggregating them to create document representations.

While the two approaches are complementary in the sense that one could use
pretrained LMs for transfer-learning also for hierarchical models, we here
focus on isolating their relative strengths and weaknesses. In this paper we
empirically show that methods which explicitly incorporate the structure of
documents outperform those that do not and further examine the influence of
data characteristics such as document length and size of training data on the
choice of architecture. Finally, we release the code to reproduce the results
from our study. 111https://github.com/jerbarnes/hier_vs_transfer

##  2 Background and related work

Document-level sentiment classification is a fundamental task in natural
language processing and has a long tradition. Although there are document
representation approaches which are more linguistically motivated, such as
Rhetorical Structure Theory Mann and Thompson (2009), or centering Grosz and
Sidner (1986); Grosz et al. (1995) these are not currently competitive with
state-of-the-art approaches. In this section, we will review two current
paradigms towards improving document-level classification: _hierarchical
models_ and _transfer learning_.

#### Hierarchical models

Hierarchical approaches to document classification aim to model the
relationship between sub-components in a document by encoding first tokens,
then sentences, and aggregating their representations in some way to create a
full document representation which can be used for classification. The first
work on hierarchical models for text classification was based on Cnns, either
by stacking Cnn layers Zhang et al. (2015); Conneau et al. (2017) or by using
a single Rnn to aggregate the output of the convolutional layers Xiao and Cho
(2016). The performance of these models depends largely on the characteristics
of the data, e.g., number of classes or dataset size, as these authors have
conflicting findings on how many layers are optimal. Hierarchical models can
also be based solely on Rnns. Yang et al. (2016) propose a hierarchical model
that uses an attention mechanism Bahdanau et al. (2015) over both sentence-
and document-level GRU-based sequence encoders in order to attend to the most
salient information, given the task. This model has shown promise for
sentiment analysis and topic classification Yang et al. (2016), as well as
classification of social media texts for e-health Ive et al. (2018).

#### Transfer learning

Transfer learning approaches, on the other hand, attempt to improve
contextualized word representations, specifically by pretraining with a
language modelling objective Peters et al. (2018); Devlin et al. (2019); Chang
et al. (2019); Wang et al. (2019). Howard and Ruder (2018) pretrain a state-
of-the-art LM Merity et al. (2018) and introduce a number of improvements to
the fine-tuning procedures. They demonstrate that this approach is able to
make better use of later supervision. These approaches have shown promise for
several document classification tasks, thanks largely to the availability of
unannotated text and the size of the models used. However, to the best of our
knowledge, these models have not been tested extensively on large documents.

##  3 Data

We perform experiments on document-level sentiment datasets in five languages:
English, French, German, Japanese, and Norwegian. For the first four, we use
the Amazon Customer Reviews datasets, a 5-class sentiment dataset with labels
$L\in\\{1,2,3,4,5\\}$ stars.222https://s3.amazonaws.com/amazon-reviews-
pds/readme.html Although the full corpora are much larger, due to
preprocessing requirements and in the interest of having similar sized data
for all languages, we create a subcorpus $D$ by sampling 50,000 documents for
each language without regarding domain, finally splitting these into
test/dev/train splits of 35,000/5,000/10,000 documents. Each document is
sentence split and tokenized using UDPipe Straka and Straková (2017) and
stored in CoNLL-U format. For Norwegian, we use the NoReC corpus 2.0, which is
a 6-class task with labels $L\in\\{1,2,3,4,5,6\\}$ ratings. It differs from
version 1.0 Velldal et al. (2018) in that it has more training examples. Table
1 shows the statistics for each dataset.

| $|D|$ | $|L|$ | T. | S. | T. / S. | $|V|$
---|---|---|---|---|---|---
Fr | 50k | 5 | 81 | 6.3 | 12.9 | 100k
De | 50k | 5 | 77 | 3.9 | 20.1 | 156k
En | 50k | 5 | 114 | 8.0 | 14.3 | 109k
Ja | 50k | 5 | 365 | 13.2 | 27.7 | 251k
No | 43k | 6 | 463 | 27.8 | 16.7 | 564k

Table 1: Statistics ($|D|$ = number of documents, $|L|$ = number of labels, T.
= average number of tokens per document, S. = average number of sentences per
document, T./S. = average sentence length in tokens, $|V|$ = vocabulary size)
for sentiment datasets.

##  4 Experimental Setup

The main research questions we seek to address in this section are: for
document-level sentiment classification, are there systematic performance
differences between flat, hierarchical, and LM-pretrained models, and do any
of these approaches offer consistent improvements? Further, we investigate how
performance is affected by several relevant data characteristics. Besides
testing on the original data, we also test on documents where the order of the
sentences is shuffled (_shuffled_), to determine sensitivity to sentence
order.

###  4.1 Models

We start by briefly summarizing the architectures. For all flat and
hierarchical models, we use 200-dimensional randomly initialized embeddings
that are updated during training, implemented in AllenNLP Gardner et al.
(2017). The flat and hierarchical models use randomly initialized word
(sentence) embeddings that are updated during training. Unless stated
otherwise, all models are trained for 50 epochs with a patience of 5 with
Adam, and the best model is chosen using accuracy on the dev set.

####  Bow:

We train a linear SVM implemented in sklearn Pedregosa et al. (2011) on bag-
of-words vectors learned from the train set and tune the $C$ parameter on the
development set.

####  Cnn:

Cnns are known to be strong baselines for document-level classification Kim
(2014). We implement a Cnn with filter sizes $F\in\\{2,3,4,5\\}$, with 50
filters per size and max pooling before a fully connected layer with a hidden
size of 50.

####  An:

We use bidirectional Gated Recurrent Units (Cho et al., 2014, GRU) with dot
product attention as a document encoder and a hidden size of 50.

#### Hierarchical Cnn (Hcnn):

The Hierarchical Cnn uses filters $F_{sent}\in\\{2,3,4,5\\}$ and
$F_{doc}\in\\{2,3\\}$, with 50 filters per size and max pooling before a fully
connected layer to create sentence and document-level representations
respectively. The output size for both layers is 50 dimensions.

#### Hierarchical Attention Network (Han):

Hierarchical Attention Networks Yang et al. (2016) have shown promise for
document-level tasks. We use Gated Recurrent Units (Cho et al., 2014, GRUs) as
our encoders, with dot product attention.

#### Universal Language Model Fine-Tuning (ULMFiT):

We use the AWD-LSTM architecture Merity et al. (2018) and pretrain on
Wikipedia data (or Common Crawl in the case of Norwegian) taken from the CONLL
2017 shared task Zeman et al. (2017). The data was sentence and word tokenized
using UDPipe Straka and Straková (2017) and we perform no further
preprocessing steps. We use between 14 and 18.7 million tokens (for No and Ja
respectively) to pretrain the language model and choose the best model after
pretraining for 100 epochs as determined by perplexity on the development set.

We then fine-tune the language models on the target domain, using slanted
triangular learning rate schedule and finally fine-tune the models to the
sentiment task using discriminative training proposed in Howard and Ruder
(2018). All experiments were performed using fastai Howard et al. (2018).

#### Multilingual BERT (mBert):

The multi-lingual BERT model is a transformer model Vaswani et al. (2017)
which has been pretrained on masked-language and next sentence prediction
tasks Devlin et al. (2019). We use the cased model which was trained on
Wikipedia dumps from 104 languages. We use the [CLS] token for prediction and
finetune the model for 20 epochs, and test the model with the best performance
on the dev set.

|  | En | No | Fr | De | Ja | Avg.
---|---|---|---|---|---|---|---
Flat | Bow | 71.1 | 55.6 | 63.3 | 73.2 | 60.6 | 64.2
Cnn | 68.1 | 52.8 | 63.4 | 72.4 | 61.1 | 63.6
_shuffled_ | 6.0 | 6.1 | 9.3 | 11.8 | 8.2 | 8.3
An | 71.9 | 58.1 | 63.9 | 74.3 | 62.1 | 66.1
_shuffled_ | 0.1 | 1.8 | 0.1 | 0.0 | 0.6 | 0.5
Hier. | Hcnn | 68.3 | 55.7 | 61.9 | 71.2 | 61.0 | 63.6
_shuffled_ | 0.3 | 0.1 | 0.2 | 0.1 | 0.2 | 0.0
Han | 72.1 | 61.4† | 63.2 | 73.2 | 61.9 | 66.4
_shuffled_ | 0.6 | 1.9 | 0.4 | 0.0 | 0.3 | 0.7
Transfer | ULMFiT | 69.4 | 55.7 | 60.8 | 69.3 | 63.7† | 63.8
_shuffled_ | 0.1 | 15.8 | 0.3 | 0.0 | 0.4 | 3.3
mBERT | 73.4† | 54.4 | 65.6† | 74.6 | 56.3 | 64.9
_shuffled_ | 0.7 | 8.0 | 0.5 | 0.8 | 0.4 | 2.0

Table 2: Accuracy of flat (Bow, An, Cnn), hierarchical (Hcnn, Han), and
transfer (ULMFiT, mBert) models on document-level sentiment datasets
($\dagger$ denotes statistically significant difference (p $<0.01$) when
compared to the second best model. Bold numbers are the best model for each
dataset, red numbers indicate poorer performance (in percentage points) when
the sentence order is shuffled, while blue numbers indicate better results
when shuffled.

###  4.2 Results

Table 2 shows the accuracy ($\text{F}_{1}$ results are similar) of the seven
models for all five languages and their average, as well as statistical
significance333We calculate statistical significance with approximate
randomization testing Yeh (2000) with 10000 runs.). The Bow model performs
well across all experiments, achieving an average 64.2 accuracy, and ties Han
on the German dataset (73.2). The Cnn performs worse than the Bow across all
experiments except Japanese (an average loss of 1.2 percentage points (pp)).
An has the second best overall performance (66.4).

Regarding the hierarchical models, the Hcnn performs on par with the flat Cnn
(avg. 63.6), while the Han model is the best on Norwegian (61.4) and the best
overall (avg. 66.4). This seems to indicate that while it is useful to
explicitly model hierarchical structure using the Han model, the Hcnn is not
as well suited to the task.

ULMFiT performs better than Bow on Norwegian and Japanese (0.1 / 3.1 pp), but
0.4 pp worse overall. mBert is the best model on English, French, and German
(73.4, 65.6, 74.6 respectively), but performs poorly on Norwegian (54.4) and
Japanese (56.3).

Shuffling the test data hurt 22 of 30 experiments and had the highest impact
on the Norwegian and Japanese data – an average drop of 5.6 pp – both of which
have the longest documents. The Cnn seems most sensitive to sentence order,
losing 8.3 pp overall, followed by ULMFiT 3.3 and mBert 2.0. Surprisingly, the
hierarchical models are more robust to changes in sentence order.

##  5 Analysis

In this section we analyze what effects the number of training examples and
document length has on the classifiers.

###  5.1 Simulated low-resource settings

In this section we compare transfer and hierarchical models in a simulated
low-resource setting, where there are only a few training examples. _A priori_
, one might expect that transfer learning should perform better, given the use
of additional unlabeled data.

We compute learning curves by training models on NoReC (same results on the
other data) using between 64 to 30,000 labeled documents. Development and test
data are kept the same. Figure 1 indicates that the models have the nearly
same relative ranking with as few as 64 training examples as they do with
30,000, with the exception of mBert. Surprisingly, even for low data
scenarios, hierarchical modeling is as beneficial as transfer learning.

Figure 1: Accuracy on Norwegian test set with increasing number of training
examples.

###  5.2 Effect of document length

Although most documents in the datasets are multi-sentence, they are not all
of the same length. Therefore, we ask ourselves: _do any of these models
perform significantly better than other models on shorter/longer documents?_

Figure 2 shows the accuracy of Han and mBert across the five languages on test
documents, where the x-axis denotes document lengths (from 1 to 50), keeping
those lengths that have more than 25 examples in order to avoid spurious
results. Although mBert performs better on short documents, Han performs much
better than mBert on longer documents ($|d|>10$), with Pearson ranked
correlation of 0.41 (p $<$ 0.01). This trend also holds when comparing Han and
An. Hierarchical modeling, therefore, increases robustness to document length.

Figure 2: Han outperforms mBert across different document lengths as
determined by number of sentences per document. The scatter plot shows the
accuracy on individual sentence counts (each color represents a dataset, and
the markers represent the two models) while the bar plots show the mean
accuracy of Han and mBert respectively on 5 sentence-length bins.

##  6 Conclusion

We have compared flat, hierarchical, and transfer learning models for
document-level sentiment classification for five different languages and have
shown that the best model depends on characteristics of your data. We also
found that hierarchical models perform similar to transfer learning approaches
even in low-resource scenarios, contrary to expectation.

## Acknowledgements

This work has been carried out as part of the SANT project (Sentiment Analysis
for Norwegian Text), funded by the Research Council of Norway (grant number
270908).

## References

  * Bahdanau et al. (2015) Dzimitry Bahdanau, Kyunghyun Cho, and Yoshua Bengio. 2015.  Neural machine translation by jointly learning to align and translate.  In _Proceedings of the Third International Conference on Learning Representations (ICLR)_.
  * Chang et al. (2019) Ming-Wei Chang, Kristina Toutanova, Kenton Lee, and Jacob Devlin. 2019.  Language model pre-training for hierarchical document representations.  _CoRR_ , abs/1901.09128.
  * Cho et al. (2014) Kyunghyun Cho, Bart van Merriënboer, Caglar Gulcehre, Dzmitry Bahdanau, Fethi Bougares, Holger Schwenk, and Yoshua Bengio. 2014.  Learning phrase representations using RNN encoder–decoder for statistical machine translation.  In _Proceedings of the 2014 Conference on Empirical Methods in Natural Language Processing (EMNLP)_ , pages 1724–1734, Doha, Qatar.
  * Conneau et al. (2017) Alexis Conneau, Holger Schwenk, Loïc Barrault, and Yann Lecun. 2017.  Very deep convolutional networks for text classification.  In _Proceedings of the 15th Conference of the European Chapter of the Association for Computational Linguistics_ , pages 1107–1116, Valencia, Spain.
  * Devlin et al. (2019) Jacob Devlin, Ming-Wei Chang, Kenton Lee, and Kristina Toutanova. 2019.  BERT: Pre-training of deep bidirectional transformers for language understanding.  In _Proceedings of the 2019 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, Volume 1 (Long and Short Papers)_ , pages 4171–4186, Minneapolis, Minnesota. Association for Computational Linguistics.
  * Gardner et al. (2017) Matt Gardner, Joel Grus, Mark Neumann, Oyvind Tafjord, Pradeep Dasigi, Nelson F. Liu, Matthew Peters, Michael Schmitz, and Luke S. Zettlemoyer. 2017\.  Allennlp: A deep semantic natural language processing platform.
  * Grosz et al. (1995) Barbara J. Grosz, Aravind K. Joshi, and Scott Weinstein. 1995.  Centering: A framework for modeling the local coherence of discourse.  _Computational Linguistics_ , 21(2):203–225.
  * Grosz and Sidner (1986) Barbara J. Grosz and Candace L. Sidner. 1986.  Attention, intentions, and the structure of discourse.  _Computational Linguistics_ , 12(3):175–204.
  * Howard and Ruder (2018) Jeremy Howard and Sebastian Ruder. 2018.  Universal language model fine-tuning for text classification.  In _Proceedings of the 56th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers)_ , pages 328–339, Melbourne, Australia. Association for Computational Linguistics.
  * Howard et al. (2018) Jeremy Howard et al. 2018.  fastai.  https://github.com/fastai/fastai.
  * Ive et al. (2018) Julia Ive, George Gkotsis, Rina Dutta, Robert Stewart, and Sumithra Velupillai. 2018\.  Hierarchical neural model with attention mechanisms for the classification of social media text related to mental health.  In _Proceedings of the Fifth Workshop on Computational Linguistics and Clinical Psychology: From Keyboard to Clinic_ , pages 69–77, New Orleans, LA. Association for Computational Linguistics.
  * Kim (2014) Yoon Kim. 2014.  Convolutional Neural Networks for Sentence Classification.  In _Proceedings of the 2014 Conference on Empirical Methods in Natural Language Processing_ , pages 1746–1751, Doha, Qatar.
  * Mann and Thompson (2009) William C. Mann and Sandra A. Thompson. 2009.  Rhetorical structure theory: Toward a functional theory of text organization.  _Text - Interdisciplinary Journal for the Study of Discourse_ , 8(3):243–281.
  * Merity et al. (2018) Stephen Merity, Nitish Shirish Keskar, and Richard Socher. 2018.  Regularizing and optimizing LSTM language models.  In _International Conference on Learning Representations_.
  * Pang and Lee (2004) Bo Pang and Lillian Lee. 2004.  A sentimental education: Sentiment analysis using subjectivity summarization based on minimum cuts.  In _Proceedings of the 42nd Annual Meeting of the Association for Computational Linguistics (ACL-04)_ , pages 271–278, Barcelona, Spain.
  * Pedregosa et al. (2011) F. Pedregosa, G. Varoquaux, A. Gramfort, V. Michel, B. Thirion, O. Grisel, M. Blondel, P. Prettenhofer, R. Weiss, V. Dubourg, J. Vanderplas, A. Passos, D. Cournapeau, M. Brucher, M. Perrot, and E. Duchesnay. 2011.  Scikit-learn: Machine learning in Python.  _Journal of Machine Learning Research_ , 12:2825–2830.
  * Peters et al. (2018) Matthew Peters, Mark Neumann, Mohit Iyyer, Matt Gardner, Christopher Clark, Kenton Lee, and Luke Zettlemoyer. 2018.  Deep contextualized word representations.  In _Proceedings of the 2018 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, Volume 1 (Long Papers)_ , pages 2227–2237, New Orleans, Louisiana. Association for Computational Linguistics.
  * Straka and Straková (2017) Milan Straka and Jana Straková. 2017.  Tokenizing, POS tagging, lemmatizing and parsing UD 2.0 with UDPipe.  In _Proceedings of the CoNLL 2017 Shared Task: Multilingual Parsing from Raw Text to Universal Dependencies_ , Vancouver, Canada.
  * Vaswani et al. (2017) Ashish Vaswani, Noam Shazeer, Niki Parmar, Jakob Uszkoreit, Llion Jones, Aidan N Gomez, Ł ukasz Kaiser, and Illia Polosukhin. 2017.  Attention is all you need.  In I. Guyon, U. V. Luxburg, S. Bengio, H. Wallach, R. Fergus, S. Vishwanathan, and R. Garnett, editors, _Advances in Neural Information Processing Systems 30_ , pages 5998–6008. Curran Associates, Inc.
  * Velldal et al. (2018) Erik Velldal, Lilja Øvrelid, Cathrine Stadsnes Eivind Alexander Bergem, Samia Touileb, and Fredrik Jørgensen. 2018.  NoReC: The Norwegian Review Corpus.  In _Proceedings of the 11th edition of the Language Resources and Evaluation Conference_ , pages 4186–4191, Miyazaki, Japan.
  * Wang et al. (2019) Alex Wang, Jan Hula, Patrick Xia, Raghavendra Pappagari, R. Thomas McCoy, Roma Patel, Najoung Kim, Ian Tenney, Yinghui Huang, Katherin Yu, Shuning Jin, Berlin Chen, Benjamin Van Durme, Edouard Grave, Ellie Pavlick, and Samuel R. Bowman. 2019.  Can you tell me how to get past sesame street? sentence-level pretraining beyond language modeling.  In _Proceedings of the 57th Annual Meeting of the Association for Computational Linguistics_ , pages 4465–4476, Florence, Italy. Association for Computational Linguistics.
  * Xiao and Cho (2016) Yijun Xiao and Kyunghyun Cho. 2016.  Efficient character-level document classification by combining convolution and recurrent layers.  _CoRR_ , abs/1602.00367.
  * Yang and Cardie (2014) Bishan Yang and Claire Cardie. 2014.  Context-aware learning for sentence-level sentiment analysis with posterior regularization.  In _Proceedings of the 52nd Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers)_ , pages 325–335, Baltimore, Maryland. Association for Computational Linguistics.
  * Yang et al. (2016) Zichao Yang, Diyi Yang, Chris Dyer, Xiaodong He, Alex Smola, and Eduard Hovy. 2016\.  Hierarchical attention networks for document classification.  In _Proceedings of the 2016 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies_ , pages 1480–1489, San Diego, California. Association for Computational Linguistics.
  * Yeh (2000) Alexander Yeh. 2000.  More accurate tests for the statistical significance of result differences.  In _COLING 2000 Volume 2: The 18th International Conference on Computational Linguistics_.
  * Yu and Hatzivassiloglou (2003) Hong Yu and Vasileios Hatzivassiloglou. 2003.  Towards answering opinion questions: Separating facts from opinions and identifying the polarity of opinion sentences.  In _Proceedings of the 2003 Conference on Empirical Methods in Natural Language Processing_ , pages 129–136.
  * Zeman et al. (2017) Daniel Zeman, Martin Popel, Milan Straka, Jan Hajič, Joakim Nivre, Filip Ginter, Juhani Luotolahti, Sampo Pyysalo, Slav Petrov, Martin Potthast, Francis Tyers, Elena Badmaeva, Memduh Gokirmak, Anna Nedoluzhko, Silvie Cinková, Jan Hajič jr., Jaroslava Hlaváčová, Václava Kettnerová, Zdeňka Urešová, Jenna Kanerva, Stina Ojala, Anna Missilä, Christopher D. Manning, Sebastian Schuster, Siva Reddy, Dima Taji, Nizar Habash, Herman Leung, Marie-Catherine de Marneffe, Manuela Sanguinetti, Maria Simi, Hiroshi Kanayama, Valeria de Paiva, Kira Droganova, Héctor Martínez Alonso, Çağrı Çöltekin, Umut Sulubacak, Hans Uszkoreit, Vivien Macketanz, Aljoscha Burchardt, Kim Harris, Katrin Marheinecke, Georg Rehm, Tolga Kayadelen, Mohammed Attia, Ali Elkahky, Zhuoran Yu, Emily Pitler, Saran Lertpradit, Michael Mandl, Jesse Kirchner, Hector Fernandez Alcalde, Jana Strnadová, Esha Banerjee, Ruli Manurung, Antonio Stella, Atsuko Shimada, Sookyoung Kwak, Gustavo Mendonça, Tatiana Lando, Rattima Nitisaroj, and Josie Li. 2017.  CoNLL 2017 shared task: Multilingual parsing from raw text to universal dependencies.  In _Proceedings of the CoNLL 2017 Shared Task: Multilingual Parsing from Raw Text to Universal Dependencies_ , pages 1–19, Vancouver, Canada.
  * Zhang et al. (2015) Xiang Zhang, Junbo Zhao, and Yann LeCun. 2015.  Character-level convolutional networks for text classification.  In C. Cortes, N. D. Lawrence, D. D. Lee, M. Sugiyama, and R. Garnett, editors, _Advances in Neural Information Processing Systems 28_ , pages 649–657. Curran Associates, Inc.