Enabling Science from the Rubin Alert Stream with Lasair–A

# Enabling Science from the Rubin Alert Stream with Lasair

Roy D. Williams  1 E-mail<EMAIL_ADDRESS>Gareth P. Francis1 Andy Lawrence1
Terence M. Sloan1 Stephen J. Smartt2 Ken W. Smith3 David R. Young3
1Royal Observatory  University of Edinburgh  Blackford Hill  Edinburgh EH9 3HJ
UK
2Department of Physics  University of Oxford  Keble Road  Oxford  OX1 3RH  UK
3Astrophysics Research Centre  School of Mathematics and Physics  Queen’s
University Belfast  BT7 1NN  UK

(Accepted XXX. Received YYY; in original form ZZZ; 2024)

###### Abstract

Lasair is the UK Community Broker for transient alerts from the Legacy Survey
of Space and Time (LSST) from the Vera C. Rubin Observatory. We explain the
system’s capabilities, how users can achieve their scientific goals, and how
Lasair is implemented. Lasair offers users a kit of parts that they can use to
build filters to concentrate their desired alerts. The kit has novel
lightcurve features, sky context, watchlists of special sky objects and
regions of the sky, dynamic crossmatching with catalogues of known
astronomical sources, and classifications and annotations from other users and
partner projects. These resources can be shared with other users, copied, and
modified. Lasair offers real-time machine-to-machine notifications of filtered
transient alerts. Even though the Rubin Observatory is not yet complete,
Lasair is a mature system: it has been processing and serving data from the
similarly formatted stream of the Zwicky Transient Facility (ZTF) alerts.

###### keywords:

Software – Rubin – Lasair – transient

##  1 Introduction

The Legacy Survey of Space and Time – LSST; (Ivezic et al., 2019), using the
Charles Simonyi Telescope at the Vera Rubin Observatory, will be a ten-year
survey of the changing sky expected to begin operations in early 2025. Of the
$\sim$10 million alerts expected each night, most will be the flickering of
active and variable objects and wandering of solar system objects. There will
also be a few thousand nightly detections of supernovae and active galaxies.

Lasair is designed to filter the massive stream of transients in different
ways for different kinds of science, add value to the LSST alerts, and store
alerts with their added value. It is available to users as a
website111https://lasair.lsst.ac.uk and an API, as well as real-time push
notifications that can communicate machine to machine.

The broker is built to process transient alerts rapidly and to make the key
decision: is this an object I want to follow up? LSST alerts will come at a
very high rate, and Lasair takes advantage of the design of the distribution
system, which issues events in rich alert packets to enable standalone
classification. Incoming alerts are judged only on that rich alert packet,
without database interaction, leading to a fast and scalable ingestion system.
Lasair has been available for some years (Smith & Williams, 2019), as an LSST
prototype , delivering alerts from the Zwicky Transient Facility (Bellm,
2018).

##  2 Lasair Concepts

The philosophy of Lasair is to be a platform for science rather than producing
the science itself. Lasair doesn’t focus exclusively on classifying; rather,
it allows scientists to make their own filters and classifiers from numerical
features and attributes. Suppose, for example, a classifier is built to pick
out a specific type of supernova. The advantage of Lasair’s approach, with
user-made features and filters, is that it may be possible to differentiate
supernova subtypes, based on a user-defined filter, whereas pre-built
classifiers typically lump them together.

Lasair users combine attributes from several rich data tables to build a
filter, or they can copy a public filter built by another user and modify it.
These rich tables are described below. Filters can be run on-demand on the
database of past alerts or in real time, with the results fed rapidly into a
machine-readable notification so that a user’s own machine can take further
action.

Lasair adds to the lightcurve features already computed by Rubin for
periodic/stochastic sources, with Lasair features being directed to finding
new explosive transients.

Lasair filters can utilise the powerful Sherlock system (Young, 2023b): a
contextual classifier for astronomical transients that crossmatches a
transient’s sky-location against an extensive library of astronomical
catalogues and, based on matched data, attempts to categorise the transient
into one of seven classes; variable star VS, cataclysmic variable CV, bright
star BS, nuclear transient NT, supernova SN, active galactic nucleus AGN and
orphan (if the transient fails to be matched against any catalogued source).
Lasair also has a continuously updated copy of the Transient Name Server
(TNS)222https://www.wis-tns.org/ database (Gal-Yam, 2021); the IAU reporting
system for supernovae and other transient sources.

Users can upload their own selection of interesting objects as a watchlist,
which can become a filter of those alerts which are crossmatched with the
watchlist. A related resource is the watchmap to find all alerts that fall
into a specific region of the sky.

Lasair encourages users to build annotator systems as a way to contribute
further value to the information portfolio of an event. Lasair has agreements
with other brokers to contribute their classifications as annotations. A user
can set up a machine to receive, compute, and contribute information about
alerts: an example is NEEDLE (Sheng, 2023), which collects pixels and
lightcurves to find rare supernovae and tidal disruption events.

All these resources can be used together in Lasair filters: Lightcurve
features, Sherlock, TNS crossmatching, watchlists, watchmaps, and annotations.
User-contributed resources can be shared with others, copied and modified, or
kept private. Lasair has extensive documentation as text, python notebooks,
and how-to videos 333https://lasair.readthedocs.io/.

Lasair filters return an initial selection of objects, based on the rich
value-added data content, and users can then manually scan the results with
the Lasair Marshall Notebook (Lasair-Team, 2023a), or run their own local code
on the lightcurve and all the other data about each object. SQL filters can be
escalated from static (run on command) to streaming filters, that run whenever
new alerts arrive and push the results via email or Kafka. Another approach
could use the Lasair API to download results into a user’s own marshall
system.

###  2.1 Scientific goals of Lasair

Lasair and its partners will provide combined access to the alerts, annual
LSST data releases, and external data sources. The joint system will provide a
flexible platform that creative users can adapt to their own ends. The science
themes are below.

####  2.1.1 Extragalactic Transients

Luminous transients outside our own galaxy include supernovae, kilonovae,
tidal disruption events, AGN flare activity, nuclear transients of unknown
origin, gamma-ray burst afterglows, stellar mergers, and compact object
mergers. All this science requires lightcurves, links to galaxy and redshift
catalogues, precise astrometric cross-matching, correlation with high energy
information, and multiwavelength cross-matching.

####  2.1.2 Multi-Messenger Astronomy

Lasair responds rapidly to alerts from NASA-GCN (Barthelmy & Racusin, 2023)
about gravitational waves, and is able to set up a watchmap (see below) for
the sky area. Lasair will collect LSST alerts in that area and allow users to
build filters to find the possible optical counterpart. In the future this
capability will be extended to gamma-ray bursts, neutrinos, etc.

####  2.1.3 Massive Samples of Supernovae

Lasair will link all transients to a list of likely host galaxies together
with their photometric and spectroscopic redshifts. See TiDES, 4MOST, and SoXS
in the “Partners” section (2.3) below.

####  2.1.4 AGN, TDE and long-lived transients

Lasair will allow users to select known AGN, upload their own AGN catalogues,
and select flaring events in both active and passive galaxies. This will
support the science of tidal disruption events, changing-look quasars, AGN
flares, microlensing of background QSOs by foreground galaxies, and unusual
long-lived nuclear transients.

####  2.1.5 Stellar Transients

Most science for variables (typically recurrent and periodic signals) will be
achieved with the annual data releases. However, there is a great opportunity
in combining alerts with the data releases. Users can discover outbursts or
large amplitude variability through the alerts, which link to the data
releases and full multi-year lightcurves. Lasair will provide streams of
objects matched to known stars and trigger on a particular magnitude
variability index.

###  2.2 Objects and Sources

Lasair handles objects and sources delivered by the Rubin Observatory. A
source is a detection by the telescope of an object, a collection of pixels on
the telescope’s light-collection device, which is significantly brighter than
it was in the reference imagery taken at the beginning of the survey. A source
is detected with a specific narrowband optical filter: LSST uses filters
u,g,r,i,z,y and ZTF uses g,r.

The brightness of a source in a transient survey is different from that source
in a reference sky acquired in the past. Difference flux can be positive or
negative, but when expressed as magnitudes, the measurement has two parts: an
absolute value converted to magnitudes and a flag to indicate a positive or
negative difference. Note that if nothing was detected in the reference sky,
then the difference magnitude is the same as the apparent magnitude.

To detect transients, the LSST will trigger on significant (5$\sigma$) sources
which are a positive excess with a point-spread-function shape measured on the
difference image444Negative sources with point-spread-function shapes will
also be catalogued for variable object selection. The 5$\sigma$ detection
limit is with respect to the noise in the difference image after the reference
image has been subtracted. When such a detection is made, a data packet called
diaSource captures the result of the source detection. The Rubin observatory
searches for previous transients close to that sky location, and from these,
it builds a collective data packet called an diaObject. If the transient is
associated with a fixed object in the sky, the diaObject and its diaSources
define the lightcurve of that object. The Lasair project is primarily devoted
to these fixed sources. For more information on LSST data products, see (Jurić
et al., 2023).

However, if the source detection is from a moving solar system object, there
will be no previous detection at that sky position. In a further processing
step at the Rubin Observatory, the locations of sources are crossmatched with
all known solar system objects. If a match is found, additional data packets
are added to the alert, including ssObject with properties of the solar-system
object and its orbit.

Thus the alerts are organised as a central data packet and a number of
associated packets. For astronomical transients fixed in the sky, the central
packet is the diaObject and the light curve is defined by the diaSources, as
well as forced photometry and non-detections. For moving objects, there is
primarily the ssObject and ssSources. Lasair is a platform for science
involving fixed objects but it also collaborates with the “Adler” project, a
platform for solar system objects – more information in the section “Partners”
below.

###  2.3 Partners

####  2.3.1 UK Data Access Centre

The UK will build and operate an Independent Data Access Centre (IDAC)
(Beckett & Mann, 2023), sized to serve 20% of the Rubin community. Like the
two Rubin-operated DACs, in the US and Chile, this will be a “Full IDAC”,
serving all the Data Release products. Lasair will work hand-in-hand with the
UK IDAC, giving longer-term lightcurves and images to back up discovery from
the alert stream, facilitating joint data mining projects, and allowing Lasair
to serve proprietary data to those of its users who are Rubin Data Rights
Holders (RubinObservatory, 2022).

####  2.3.2 TiDES and 4MOST

Lasair works closely with the two major ESO projects that will provide tens of
thousands of spectra for LSST supernovae. We will coordinate supernova
discoveries in Lasair with spectra from the 4MOST multi-fibre spectrometer on
the ESO VISTA telescope. We expect to select 35,000 live transients for
spectra and obtain spectra of 70,000 host galaxies in the TiDES (Time Domain
Extragalactic Survey) (Swann et al., 2019). This will provide the largest
cosmological sample of SN Ia, together with a massive statistical sample to
understand supernova explosion physics across a range of redshifts and host
galaxy masses and metallicities. Lasair will provide both (reproducible)
selection and extract the scientific content (type, phase, redshift etc) to
re-ingest into the broker for further user exploitation.

####  2.3.3 SoXS

Lasair also works closely with the UK team responsible for building the
science software infrastructure for SoXS (Schipani et al., 2018); a single-
shot 0.35-2 $\mu$m spectrometer on ESO’s New Technology Telescope. ESO is
fully dedicating the NTT to time domain science, with the schedule being run
by the SoXS consortium. We will enable the SoXS marshall to interface with
Lasair to select LSST transients for classification and then re-ingest the
information and public data for all users to access.

####  2.3.4 PESSTO

Lasair works with the Public European Southern Observatory Spectroscopic
Survey of Transient Objects (PESSTO) (Smartt, 2015). It is a public
spectroscopic survey that began in 2012, classifying transients from publicly
available sources and wide-field surveys, and selecting science targets for
detailed spectroscopic and photometric follow-up.

####  2.3.5 Citizen Science

Citizen science builds unique and authentic research experiences for the
public that directly engage individuals with little or no scientific training
or background, thus lowering the barrier for the public to contribute directly
to scientific investigations. Lasair already has a strong collaboration with
Zooniverse.org, having established a project (Smith et al., 2022) to hunt for
rare superluminous supernovae. This will be broadened, and other projects
added, in the LSST era.

####  2.3.6 Solar System Science with Adler

The Legacy Survey of Space and Time (LSST) will discover 6 million new Solar
System small bodies, including the comet to which ESA’s Comet Interceptor
mission will target (Schwamb, 2019). With each planetesimal receiving hundreds
of observations across six filters, LSST will radically transform the view of
our planetary system and usher in a revolution for time-domain Solar System
science. The Adler system will work closely with Lasair, ingesting the Rubin
Solar System alerts and identifying potentially active sources and unexpected
photometric behaviour.

####  2.3.7 Other Rubin Alert Brokers

The LSST alert stream is expected to be high bandwidth (0.2 - 5 Gb/sec)(Graham
et al., 2020), so it would be impractical to host large numbers of readers,
each taking the whole stream. The Rubin Observatory instead selected seven
“Community Brokers” through a proposal process, each providing its own vision
of how such a broker should work. In addition to Lasair, they are Alerce
(Förster & Cabrera-Vives, 2023), ANTARES (Matheson, 2021), AMPEL (Nordin,
2019), Babamul (Graham, 2023), FINK (Ishida et al., 2023), and Pitt-Google
(Raen et al., 2023).

###  2.4 What Lasair is not

The “rich data packet” that comes with each alert means that lightcurve
features and subsequent filtering are made only on that packet, which has only
a year of past data about each object (only a month for the ZTF prototype).
Lightcurve features and thus decisions are not based on the full multi-year
light curve, even though the Lasair databases include these. Rather, decisions
are based on these shorter light curves. Even though the lightcurve features
are based on the short lightcurve, the web page and API deliver all of it. In
the case of ZTF, Lasair lightcurves go back 6 years, and these can be
effectively mined – see section 3.5 below.

The fact that a feature is not built into Lasair doesn’t mean it can’t be
done; rather, the user has to do some of the work themselves through
annotation—and we try to provide a good set of example notebooks to act as
starting points for some of these things. It comes down to the distinction
between doing science and providing a toolkit for people to do science with.

##  3 Lasair Functionality

Lasair provides a portfolio of data for each object (diaObject), and a means
to build a filter that selects which objects the user finds interesting. In
addition to that provided by the Rubin Observatory pipeline, Lasair adds novel
lightcurve features, rich sky context via Sherlock, watchlists and watchmaps,
TNS crossmatch, and annotations from other users and brokers. All this can be
combined into the WHERE clause of an SQL statement that is the heart of a
Lasair filter.

The components described below are like LEGO bricks that can be used to build
a bespoke filter that concentrates the alerts a scientist is searching for.
Each component is expressed as a database table whose attributes can be used
to create the filter. Filters can be public and, therefore, shared, copied and
modified; the same is true of the other resources described below.

In addition to making filters, users can create watchlists of their special
objects and watchmaps of sky areas or build their own classifier running on
their own machine. They can then push the results back to Lasair as
annotations.

###  3.1 Sky Context from Sherlock

Sherlock finds known astronomical objects at the sky location of an alert.
Published astronomical catalogues are carefully curated collections of stars,
galaxies, variable stars, cataclysmic variables, active galactic nuclei, etc.
When a transient object is detected, an astronomer will want to know if the
transient is associated with a previously known source and, if so, what kind
of source it is. If astronomers are searching for extra-galactic explosive
events such as supernovae, they are typically interested in transients
associated with a galaxy.

Sherlock consists of a Python package and a curated library of astronomical
catalogues (in a MariaDB database) and provides a rapid and reliable spatial
cross-match service for any astrophysical variable or transient. It associates
a transient’s position against its library of astronomical catalogues and uses
an intelligent ranking algorithm to assign a classification. At its most
basic, Sherlock can triage transients into stars, AGN, and supernova-like
objects. It has been used for many years as part of the ATLAS, Pan-STARRS, and
Lasair-ZTF decision-making.

One of the main purposes of Sherlock within Lasair is to identify known
variable stars, since they will make up a majority of LSST alerts, and to
associate candidate extragalactic sources with potential host galaxies.
Sherlock’s library of catalogues contains datasets from many all-sky surveys
such as:

  * •

Gaia DR2 (Gaia-collaboration, 2023)

  * •

Pan-STARRS1 Science Consortium surveys (Chambers, 2019) and the catalogue of
probabilistic classifications of unresolved point sources by (Tachibana &
Miller, 2018) based on the Pan-STARRS1 survey data.

  * •

The SDSS DR12 PhotoObjAll and SDSS DR12 SpecObjAll Tables (Alam, 2015). The
PhotoObjAll table contains a photometry-based measurement of the star-galaxy
separation, and the SpecObjAll table contains a spectroscopic redshift for
each source.

  * •

Guide Star Catalogue v2.3 (Lasker et al., 2008).

  * •

2MASS point- and extended-source catalogues (Skrutskie, 2006).

Sherlock also employs many smaller source-specific catalogues such as

  * •

Million Quasars Catalog v5.2 (Flesch, 2023),

  * •

Veron-Cetty AGN Catalogue v13 (Véron-Cetty & Véron, 2010),

  * •

Downes Catalog of CVs (Downes et al., 2001),

  * •

Ritter Cataclysmic Binaries Catalog v7.21 (Ritter & Kolb, 2003).

For spectroscopic redshifts and/or non-redshift based distance measurements
Sherlock uses:

  * •

LASr-GPS is a 100Mpc volume-limited galaxy catalogue (Asmus et al., 2020) and
the

  * •

NED-D Galaxy Catalogue v17.1.2. A Master List of Redshift-Independent
Extragalactic Distances (Steer et al., 2017)

  * •

A dynamic query of the NASA/IPAC Extragalactic Database via the neddy software
package (Young, 2023a).

  * •

SDSS DR12 SpecObjAll table (Alam, 2015).

At a base level of matching, Sherlock distinguishes between transient objects
synonymous with (the same as, or very closely linked to) and those it deems as
merely associated with the catalogued source. The resulting classifications
are tagged as synonyms and associations, with synonyms providing intrinsically
more secure transient nature predictions than associations. Depending on the
underpinning characteristics of the source, there are 7 types of predicted-
nature classifications that Sherlock will assign to a transient:

  * •

VS: Variable Star, if the transient lies within the synonym radius of a
catalogued point-source,

  * •

CV: Cataclysmic Variable, if the transient lies within the synonym radius of a
catalogued CV,

  * •

BS: Bright Star, if the transient is not matched against the synonym radius of
a star but is associated within the magnitude-dependent association radius,

  * •

AGN: Active Galactic Nucleus, if the transient falls within the synonym radius
of catalogued AGN or QSO.

  * •

NT: Nuclear Transient, if the transient falls within the synonym radius of the
core of a resolved galaxy,

  * •

SN: Supernova, if the transient is not classified as an NT but is found within
the magnitude-, morphology- or distance-dependant association radius of a
galaxy, or

  * •

ORPHAN: if the transient fails to be matched against any catalogued source.

For Lasair-ZTF the synonym radius is set at 1.5 arcsec, which we find is a
good match to the systems PSF and astrometric accuracy. This is the
crossmatch-radius used to assign predictions of VS, CV, AGN and NT. The
process of attempting to associate a transient with a catalogued galaxy is
relatively nuanced compared with other crossmatches as there are often a
variety of data assigned to the galaxy that help to more reliably inform the
decision to associate the transient with the galaxy or not.

Once each transient has a set of independently crossmatched synonyms and
associations, Sherlock selects what it thinks is the most likely
classification. The last step is to calculate value-added parameters for the
transients, such as absolute peak magnitude if a redshift or distance can be
assigned from a matched catalogued source. The predicted nature of each
transient is presented to the user along with the lightcurve and other
information.

As part of the Lasair project, there is public access to the integrated
Sherlock code and database information through the Lasair API (Lasair-Team,
2023a). We plan further additions to the Sherlock database with the Legacy
Survey catalogues and eventually LSST data releases.

###  3.2 Transient Name Server

An astronomer who is interested in an explosive event wants to know if it has
already been seen and registered by a different survey. The Transient Name
Server (TNS)(Gal-Yam, 2021) is the official IAU mechanism for reporting new
astronomical transients, with a focus on extragalactic explosive transients.
Once spectroscopically confirmed, new supernova discoveries are officially
designated an SN name. Lasair keeps a cache of the database, updated every few
hours, which can be used in Lasair filters.

###  3.3 Watchlists and Watchmaps

Many astronomers study a specific set of existing objects and are interested
in transients from those – for example, active galaxies, galaxy clusters, or
star formation regions. A watchlist is a set of named points in the sky,
together with a radius in arcseconds – which can be the same for all sources,
or different for each - that is, a set of named cones. It is assumed to be a
list of “interesting” sources so that any transient that falls within the
radius of one of the sources might indicate the activity of that source. Each
user of the Lasair system can have one or more watchlists, and can make a
filter to be alerted when a transient is coincident with a watchlist source.
An “Active” watchlist is one that is run every day so that it is up to date
with the latest objects.

To be specific, suppose we are interested in the 42 objects in the catalogue
of BL Lac candidates for TeV observations (Massaro et al., 2013), which can be
found in the Vizier library of catalogues. The user can make their watchlist
“public” so other Lasair users can see it and use it in queries, and can make
the watchlist “active”, meaning that the crossmatch is kept up to date with
incoming alerts, and active filters respond immediately. Full details are in
(Lasair-Team, 2023a).

A watchmap is a specification of an area of the sky, that can be used as part
of a Lasair filter. The crucial purpose here is for gravitational wave events
and other multimessenger transients, where the sky location is actually a
probability distribution. However, the MOC doesn’t embody the idea of a
probability distribution – it’s just an area of the sky – so the watchmap
might cover the 90-percentile. A watchmap might also be the footprint of
another survey or a large sky area like the Orion Nebula. If the watchmap has
been set to “active”, then all alerts ingested to Lasair are tested against
it, and tagged if inside the watchmap. A filter can then be built that selects
only alerts inside the watchmap. Watchmaps are created by building and
uploading a MOC (Multi-Order Coverage) file (Fernique et al., 2014).

###  3.4 Annotations

Lasair allows users to add information to the database, that can then be used
as part of a query by another user. An annotation is a structured packet of
extra information about a given object, that is stored in the annotations
table in the SQL database. This could be the result of running a machine-
learning algorithm on the lightcurve, the classification created by another
broker, or data from a follow-up observation on the object, for example, a
link to a spectrum. Users that put annotations into the Lasair database are
validated, and administrators then make it possible. That user will run a
method in the Lasair API – from their own machine – that pushes the
annotation: all this can be automated, meaning the annotation may arrive
within minutes of the observation that triggers it.

Each annotation is associated with a specific Lasair object, and may contain:

  * •

objectId: the Lasair object being annotated

  * •

topic: the name of the annotator that produced this annotation

  * •

classification: a short string drawn from a fixed vocabulary, eg “kilonova”.

  * •

explanation: a natural language explanation of the classification, eg
“probable kilonova but could also be supernova”

  * •

classjson: the annotation information expressed as a JSON dictionary

  * •

url: a URL where more information can be obtained, for example a spectrum of
the object obtained by follow-up.

The NEEDLE project (Sheng, 2023) is an example of an annotator, designed to
find superluminous supernovae in dwarf galaxies, and tidal disruption events
occurring in the centres of nucleated galaxies. There is a “pre-filter” that
selects based on the Sherlock classification and the current magnitude versus
mean magnitude; results of this filter are pushed via Kafka to a separate
analysis machine, which pulls the lightcurve and cutout images for analysis by
a neural-net system. Promising candidates are pushed back to Lasair as an
annotation, with the classjson attribute set to something like:

    {"SN": "0.799", "SLSN-I": "0.103", "TDE": "0.099"}

These returned attributes can then be used as part of a filter in the usual
way; there is special syntax to refer to the parts of the classjson in the SQL
language.

Another annotator is Fastfinder that models lightcurves of new transients to
find rapid brightening from kilonovae. It classifies alerts as SLOW, FAST, and
SN so that the latter could be picked out with a clause
fastfinder.classification="SN" in the WHERE part of the filter. As explained
above, the classjson can hold complex information, and querying is more
sophisticated; for more information see the Lasair documentation (Lasair-Team,
2023a). Annotations can be pushed to the Lasair database using the Lasair
client, however the user must be authenticated to do so. Lasair staff are
happy to receive a request to create an annotator, and the successful user’s
topic name and API key will allow them to upload annotations.

Because annotation is a process of inserting data into the Lasair database,
the Lasair Team needs to know who is doing it. Therefore, running an annotator
starts with asking the team to set it up, and upload of annotations can only
be done by the person responsible.

####  3.4.1 Fast Annotations

Some special annotations can be upgraded to “fast” annotations. This means
that as soon as the annotation is uploaded, all user filters that involve that
annotator are rerun immediately. Suppose, for example, a user builds a filter
based on the NEEDLE annotator, the criterion being that the SLSN-I probability
for that object is more than 0.7. If NEEDLE is a fast annotator, then that
user’s filter will run as soon as the annotation arrives, with an immediate
email or Kafka result. Otherwise, there will be a delay, until another
detection of that object occurs, which may be days later.

####  3.4.2 Alerce and Fink

Lasair is one of seven Community Brokers for the Rubin alerts and has arranged
with Alerce (Förster & Cabrera-Vives, 2023) and Fink (Ishida et al., 2023) to
consume their outgoing Kafka streams and ingest the results of their
classifiers as annotations. Currently, these include the Alerce Stamp
Classifier and Light Curve Classifier, as well as some of the many Fink
classifiers, which include Kilonova, TDE, Microlensing, and subtypes of
supernovae.

###  3.5 Mining the Lasair Database – an example

Here we provide a science example of how the Lasair database can be mined.
There is a ‘pre-filter’, then all the resulting lightcurves are fetched by API
and cuts applied, until there are few enough to look by eye. Our example is
based on the transient known as AT2021lwx, which released $1.5\times 10^{53}$
erg of energy over three years, more than any other optical transient. It was
a transient from the ZTF survey but sat in the Lasair database until it was
found (Wiseman, 2023). AT2021lwx was extragalactic, and had a long, smooth
decline in brightness over three years. Below we sketch a way to find other
such events in the Lasair database.

For this search, there were two Lasair filters: one for transients in a
catalogued galaxy, and another for those where the host galaxy is too faint to
have been catalogued. The first query finds alerts with Sherlock
classifications SN, NT, or AGN; the second finds Sherlock ORPHANs away from
the Milky Way [abs(galactic latitude) < 10]. Each of the resulting 59,000 ZTF
lightcurves is fetched from Lasair and features computed for both g and r
filters: being a linear fit to the lightcurve after the peak brightness. If
the variance is low about this line, and the rate of fading is slow, then we
have a candidate. The result of this specific case produced 58 candidates,
still under investigation (Wiseman, in prep.)

A data-mining project of this kind requires a lot of API calls, therefore
requesting that the Lasair team remove the default API throttling – which is
only 20 API calls allowed in any given hour.

###  3.6 Realtime Output

Alerts can be visualised on the Lasair web interface, or the underlying data
fetched using the Lasair API. But there is another way to get Lasair data,
through push notification, where the Lasair system pushes data as soon as it
is available. The push can be a human-readable email, or via a machine-
readable Kafka stream. Such a stream is designed to be consumed by machines,
not people; it may be that an analysis program sits waiting, and springs into
action as soon as the Kafka alert arrives. It could compute a lightcurve
classification, it could drive a followup telescope, perhaps followed by an
annotation. The consumer of a Kafka stream need not run continuously as alerts
can build up until they are summoned, perhaps by the Lasair Marshall Notebook
(see section 4.5). For more information about Lasair’s Kafka output, see
section 4.4.

##  4 Lasair in Practice

###  4.1 Making a Filter

Figure 1: Screenshot of the filter builder showing the user inputting the
SELECT and WHERE parts of the filter.

Lasair is built around “filters” of the alert stream. Users create a filter
with SQL clauses, based on the attributes of the object associated with the
alert; its lightcurve, sky context, etc. First, the user makes a filter and
runs it on previous alerts, and then they can save the filter. The user can
convert a filter to an “active” filter, so that results are sent to email or
to their own machine as soon as they are available.

In Figure 1 is a screenshot of an example filter for bright stars that have
had recent alerts. The SELECT clause can be typed (with the assistance of
auto-complete and schema browser) as:

    objects.objectId, objects.gmag,
    jdnow()-objects.jdmax AS since

and the WHERE clause similarly as:

    objects.gmag < 19 ORDER BY objects.gmag

In English, the filter is: select the object ID, g-magnitude, and time in days
– call it since – since the latest detection, and we only want to see those
objects brighter than 19th magnitude, sorted with the brightest first. There
is also a FROM section listing the tables being joined to make the query; this
list is made automatically from the content of the SELECT and WHERE clauses.

Clicking “Run Filter” gives the brightest from years of stored alerts, as
expected. If, however, the filter is saved, there is a popup form asking for
name, description, etc, as well as if it should be public and if it should be
“active”. The active filter is one of Lasair’s crucial and powerful ideas: it
means that all incoming alerts are passed through the filter automatically, in
near-real-time, and what passes the filter is pushed to the user, by email or
by Kafka, which can also drive a Slack workspace or other alert mechanism.
Filters run the same way whether the filter operation is on-demand by
click/API, or whether it runs in near-real-time. However there is one
difference: the above filter run on-demand returns the brightest alerts first
(because of the ORDER BY clause), but by the nature of a streaming filter, it
can only return all its results in time order.

Figure 2: Screenshot of the filter builder showing the user inputting the
SELECT and WHERE parts of the filter.

To get alerts coincident with a watchlist, Figure 2 shows how all that is
needed is choosing the watchlist in the pull-down list. The attributes arcsec
and name are then available for the name of the coincident object from the
watchlist, and its distance in arcsec from the alert.

There will be many lightcurve features from the LSST data pipeline, packaged
with each alert (Bellm, 2023). Some ideas for adding value to these, now in
development, are described in Appendix A.

###  4.2 Learning

Lasair has full documentation (Lasair-Team, 2023a), explaining the basic
concepts of Lasair, as well as detailed how-to tutorials. There are also
videos showing how to get a user account, how to make watchlist and watchmap,
and other topics. On the Lasair webpage itself are numerous tooltips – short
popup explanations when the mouse hovers – each of which has a link to more
detail in the main documentation. There are Jupyter notebooks in the
documentation, showing simple and complex queries through the API, the API
interface to Sherlock, how to utilise Kafka streams and other topics.

Figure 3: Screenshot from the Lasair Marshall Notebook

###  4.3 The Lasair Client

There is a Python software package called lasair (Lasair-Team, 2021) which can
be installed with pip. It can be used to query the database and Sherlock, to
create an annotation, or to consume a Kafka stream. Full details are in the
Lasair documentation (Lasair-Team, 2023a).

When a filter has been set to “active” and Kafka output, a topic name will be
displayed on the web page of that specific filter. It is this that is used to
select the output stream from that filter. Users need to understand the
concepts of topic and groupId, perhaps by watching the relevant videos in the
documentation. The code then creates a consumer object and calls the poll
method to get the messages. Full details are in the Lasair documentation
(Lasair-Team, 2023a).

###  4.4 Real-time Notification

As described above, the Lasair broker sends notifications when an active
filter sees an interesting alert. This can be done by email, which is suitable
for low-volume filters and human attention. Notifications can also be sent via
Kafka, an open-source distributed event streaming platform used throughout
industry for high-performance data pipelines, streaming analytics, data
integration, and mission-critical applications (see kafka.apache.org). Indeed,
the data ingestion from Rubin and the Lasair processing pipeline are both
based on Kafka. Other methods of notification are then easily implemented
(e.g. Slack).

###  4.5 Marshall Notebook

The purpose of a transient survey is to find the needle in the haystack: the
scientifically interesting alert among millions of others. Lasair users can do
this in two parts: first an automated filter that runs as the alerts are
ingested, then a human looking at the results, with an application known as a
“marshall”. The scientist looks at a batch, flagging some as “favourites”, and
some as “don’t show me this object again” (veto), and checking full objects
information for some. Then the next batch, until all have been seen. The next
night there will be a fresh batch to be checked. The Lasair project has built
a “marshall”, implemented as a Jupyter notebook. The alert in Figure 3 is a
supernova in the centre of the Science image – near the centre of the spiral
NGC 1086.

At the top are the attributes selected by the filter, then lightcurves, the
ZTF reference and latest image, and the colour image from Pan-STARRS. The SN
II is already registered in TNS, and has a link. There are two checkboxes and
a place for a comment. Ticking “veto” means the object will not be seen again
(like the object at the bottom of the image), and ticking “fave” means it will
be emphasised next time. If the notebook is run again, a new set of results is
shown, until all have been seen. Using the notebook for eyeballing is much
easier than having Lasair send email notifications. Further instructions can
be found in the Lasair documentation (Lasair-Team, 2023a). This Jupyter
notebook is meant for individual users, but we expect that other projects
(e.g. SoXS) will build their own marshall system, as the PTF, ZTF, and PESSTO
projects have done, connecting through Kafka and the Lasair API.

###  4.6 Kinetic Display

Figure 4: The do-it-yourself realtime data display for Lasair’s public Kafka

This display shows the status of the Lasair data flow, as shown in Figure 4.
The total cost is about £100. A 32x32 RGB matrix is powered by a Raspberry Pi,
and it can be mounted in a box frame to make a wall-mounted display. The RPi
needs to be on a Wi-Fi network. The code for the display and full instructions
can be found in (Williams, 2022).

Each alert that arrives has RA and Dec in the sky: RA is 0 to 360 from left to
right, and Dec is -25 to 90 from bottom to top. Underlying the colours of the
display is a matrix of bits. When a batch of alerts has come in, the matrix is
replaced with the new matrix with bits switched on based on the positions of
the alerts of the batch. Simultaneously, Conway’s Game of Life (Conway, 1970)
cellular automata runs continuously on the matrix, leading to patterns that
evolve and change rapidly after the batch of alerts is received. The actual
colour display is an RGB matrix derived from the history of the binary matrix,
with R, G, and B fading at different rates.

The display can also be shown on a computer monitor, without buying extra
hardware, by installing the OPenCV (OpenCV-Team, 2022) Python package.

##  5 Technical Implementation

In this section, we give a brief overview of the technical implementation of
the Lasair system. The overall architecture is illustrated in Fig. 5. Given
that Rubin will produce ten million alerts per night, the processing system
must be carefully architected and well-resourced to keep up. We don’t want
Lasair falling hours behind, still processing a night’s alerts well into the
next day, or even worse, not being finished when the firehose starts up the
next night.

Lasair runs on an Openstack cloud “Somerville” at the University of Edinburgh
Advanced Computing Facility, which is part of “IRIS” – the UK academic cloud
555IRIS: Digital Research Infrastructure for UK Science,
https://www.iris.ac.uk/. It shares that cloud with the Data Access Centre
described in section 2.3.1.

Lasair ingests data with a pipeline of scalable clusters, as shown in Figure
5: Kafka, ingest, Sherlock, filter. Each cluster does a different job, some
more compute/data intensive than others. It is difficult to know a priori how
much resource should be allocated to each cluster, so our design gives
flexibility: each cluster can be grown or reduced according to need. Also,
there are persistent data stores (Cassandra, MariaDB via a Galera cluster);
again, each is a resilient cluster architecture that can be grown or reduced
according to need. The diagram shows the concept: data enters the Kafka system
on the left and progresses to the right. The Kafka cluster (grey) consumes and
caches data from Rubin and from the other pipeline clusters; the ingest
cluster (yellow) splits and redirects, then puts data back into the Kafka
communication bus; the Sherlock cluster (red) consumes and produces Kafka; the
filter cluster (blue) consumes the result and writes them to the Galera
database cluster and public Kafka. We also include the web and annotator nodes
in this picture (bottom and right), as well as the mining nodes, although they
are not part of the data ingestion pipeline.

The web server supports users by delivering web pages and responding to API
requests. The annotator nodes may be far from the Lasair computing centre and
controlled by Lasair users, but they are in this picture because just like the
others, they push data into the data storage and may consume from public
Kafka.

Figure 5: The architecture of the Lasair system. Scalable homogeneous clusters
are connected by Kafka to form a pipeline

The Kafka system is represented by the green nodes in Figure 5 as well as the
grey arrow at the top. It is responsible for reading and caching the alert
packets from Rubin, as well as sending data to the compute clusters and
receiving their resulting packets.

The Ingest nodes read the original alerts from the Kafka system and put the
cutout images in a dedicated database. The recent lightcurve is then sent to
the NoSQL (Cassandra) database. The alert without cutouts is reformatted as
JSON since there is no binary content, and then it is pushed into the Kafka
system.

Each Sherlock node has its own database of 5 Tbytes of astronomical
catalogues. If necessary, this can be replicated to provide a higher
throughput.

Each filter node computes features of the 12-month LSST light curve that comes
with the alert, as well as matching the alert against user-made watchlists and
watchmaps. Records are written to a local SQL database onboard the node for
the object and features, the Sherlock data, the watchlist and watchmaps tags.
Other tables have already been copied into the local database from the main
SQL database. After a batch of perhaps 40,000 alerts is ingested into the
local database, it can now execute the user-made queries and push out results
via the public Kafka system – or via email if the user has chosen this option.

The tables in the local database are then pushed to the main SQL database and
replace any earlier information where an object is already known. Once a batch
is finished, the local database tables are truncated and a new batch started.

There are also “twilight services” to prepare the ingestion system for the
night: caching annotations, watchlists, and watchmaps for the filter nodes,
the latest version of the TNS database, and others.

###  5.1 Scalability and Efficiency

The Lasair project will store all the alert information from Rubin in a
Cassandra database (although the more recent cutouts may only be stored
there). Each Rubin alert is issued because of a single detection that is
$5\sigma$ significance above the noise in the difference image (target image
minus the reference sky), but it is accompanied by a host of additional data
to allow judgement to be made. In principle, this extra data is not needed
since it is already available in the Cassandra database; however, this
redundancy allows the alert processing to be handled by many independent
nodes, with no need to communicate with each other or read from the databases.
Thus during data ingestion, there is no reading from the databases at all.
When writing data, these systems can work asynchronously without blocking, an
“eventual consistency” paradigm that allows maximum write speed. Any processes
that require consultation with the database are then left to the next day,
when the only usage of the databases is much lighter: only the web/API
customers.

There is a multi-node relational database behind the web server and API,
providing users with the richness of the SQL language. The much large
Cassandra database is indexed on diaObjectId and sky position, so it is very
much a key-value retrieval system rather than a query-based relational
database.

###  5.2 Webserver

The Lasair webserver is built in Python with Django to separate computation
from presentation. The Twitter Bootstrap v5.2 framework is used alongside
Plotly for lightcurve display. AladinLite (Boch & Baumann, 2023) shows sky
context in numerous wavebands, and the JS9 package is used to display image
stamps. A Grafana dashboard shows the status of all the nodes, with live
traces of the pipeline’s status in terms of alerts per minute.

###  5.3 Self-protection

The Lasair Team has worked to ensure flexible and useful filtering of alerts
for users and has chosen the SQL language for this, as it is well-known now to
astronomers who are data specialists. However, Lasair guards against mistakes
and attacks through SQL. All queries are run in a user account with read-only
access, and query strings are carefully vetted before being run against the
database. User-made queries have a time-out of 10 seconds when running in
streaming mode against a batch of alerts, so they can’t hold up the entire
system if they are too resource-intensive.

The filter nodes compare each alert against active watchlists and watchmaps
and run each batch against active user queries. Each of these active resources
slows the ingestion system. Therefore, Lasair has a 6-month expiry for each,
and the user must then go to the website and renew its activity.

###  5.4 Deployment

Lasair has a flexible system for deploying an entire system on the Openstack
cloud, using Ansible and Terraform, with a specification of the numbers of
nodes to be used for each functional cluster, their memory and local disk,
spinning disk or SSD, how they are to be located on hypervisors, and other
hardware considerations. This makes it easy to have extra systems running the
entire Lasair stack: for example a development system or a system for
measuring performance.

##  6 Summary and Conclusion

Lasair has been operating for years with the ZTF alert stream, has several
hundred user accounts, and is ready for the greater challenge of LSST, with
about 30 times the alert rate. A user of Lasair does not simply take a ready-
made sub-stream of classified alerts; rather, they combine different elements
from a kit. The kit has the Sherlock association engine to find what is
already published about a specific sky location; it has watchlists, so a user
can ask for alerts associated with a set of astrophysical sources; it has
watchmaps to restrict attention to arbitrary sky regions; it has sophisticated
features to characterise the lightcurve. Lasair allows users to do deep
analysis of objects on their own machine and annotate Lasair with their
computed results and classifications. These different resources are then
combined, using the SQL language, to create a filter, and the filter can run
from a web-click, by API, or run in real-time streaming mode as alerts come
in, so no time is lost, with machine-readable output. A user can publically
share (or not) their watchlists, watchmaps, annotations, and filters. Lasair
is being upgraded to tackle gravitational-wave events, thus allowing users to
pick the LSST alert most likely to be associated with the counterpart.

## Acknowledgements

Lasair is currently supported by the UKRI Science and Technology Facilities
Council and is a collaboration between the University of Edinburgh, Queen’s
University Belfast, and the University of Oxford (grants ST/X001334/1 and
ST/X001253/1) within the LSST:UK Science Consortium. Lasair is hosted by the
STFC IRIS academic cloud. Lasair relies on the ZTF survey, supported by the
National Science Foundation under Grant No. AST-2034437 and a collaboration
including Caltech, IPAC, the Weizmann Institute for Science, the Oskar Klein
Center at Stockholm University, the University of Maryland, Deutsches
Elektronen-Synchrotron and Humboldt University, the TANGO Consortium of
Taiwan, the University of Wisconsin at Milwaukee, Trinity College Dublin,
Lawrence Livermore National Laboratories, and IN2P3, France. Operations are
conducted by COO, IPAC, and UW. The ZTF forced-photometry service was funded
under the Heising-Simons Foundation grant 12540303. This research has made use
of the NASA/IPAC Extragalactic Database (NED), which is funded by the National
Aeronautics and Space Administration and operated by the California Institute
of Technology.

## Code and Data Availability

The Lasair software is available at (Lasair-Team, 2023b). ZTF data are
available at (ZTF-collaboration, 2018). LSST data rights are described at
(RubinObservatory, 2022).

## References

  * Alam (2015) Alam, S. o., 2015.  The Eleventh and Twelfth Data Releases of the Sloan Digital Sky Survey: Final Data from SDSS-III, ApJS, 219(1), 12.
  * Asmus et al. (2020) Asmus, D. et al., 2020.  Local AGN survey (LASr): I. Galaxy sample, infrared colour selection, and predictions for AGN within 100 Mpc, Monthly Notices of the Royal Astronomical Society, 494(2), 1784–1816.
  * Barthelmy & Racusin (2023) Barthelmy, S. & Racusin, J., 2023.  Gcn: General coordinates network, https://gcn.nasa.gov/missions/lvk.
  * Bazin (2011) Bazin, G. o., 2011.  Photometric selection of type ia supernovae in the supernova legacy survey, A&A, 534, A43.
  * Beckett & Mann (2023) Beckett, G. & Mann, R., 2023.  Operational plan for uk data facility, https://lsst-uk.atlassian.net/wiki/download/attachments/1146928/LUSC-B-46-D2.1.3_lusc_operational_plan.pdf.
  * Bellm (2023) Bellm, E. C., 2023.  Review of timeseries features, https://dmtn-118.lsst.io/.
  * Bellm (2018) Bellm, E. o., 2018.  The zwicky transient facility: System overview, performance, and first results, Publications of the Astronomical Society of the Pacific, 131(995), 018002.
  * Boch & Baumann (2023) Boch, T. & Baumann, M., 2023.  Aladinlite, https://aladin.cds.unistra.fr/AladinLite/doc/.
  * Chambers (2019) Chambers, K. C. o., 2019.  The pan-starrs1 surveys, https://arxiv.org/abs/1612.05560.
  * Conway (1970) Conway, J. H., 1970.  Conway’s game of life, https://en.wikipedia.org/wiki/Conway%27s_Game_of_Life.
  * Downes et al. (2001) Downes, R. A. et al., 2001.  A catalog and atlas of cataclysmic variables: The living edition, Publications of the Astronomical Society of the Pacific, 113(764).
  * Fernique et al. (2014) Fernique, P., Boch, T., Donaldson, T., Durand, D., O’Mullane, W., Reinecke, M., & Taylor, M., 2014.  MOC - HEALPix Multi-Order Coverage map Version 1.0, IVOA Recommendation 02 June 2014.
  * Flesch (2023) Flesch, E. W., 2023.  The Million Quasars (Milliquas) catalogue, https://arxiv.org/abs/2308.01505.
  * Förster & Cabrera-Vives (2023) Förster, F. & Cabrera-Vives, G. o., 2023.  Alerce community broker, https://alerce.science/.
  * Gaia-collaboration (2023) Gaia-collaboration, 2023.  The gaia survey, https://www.cosmos.esa.int/web/gaia.
  * Gal-Yam (2021) Gal-Yam, A., 2021.  The TNS alert system, in American Astronomical Society Meeting Abstracts, vol. 53 of American Astronomical Society Meeting Abstracts, p. 423.05.
  * Graham et al. (2020) Graham, M. et al., 2020.  Lsst key numbers, https://dmtn-102.lsst.io/.
  * Graham (2023) Graham, M. o., 2023.  Babamul community broker, https://docs.babamul.dev/.
  * Ishida et al. (2023) Ishida, E. O., Möller, A., & Peleton, J. o., 2023.  Fink community broker, https://fink-broker.org/.
  * Ivezic et al. (2019) Ivezic, Z. et al., 2019.  Lsst: From science drivers to reference design and anticipated data products, The Astrophysical Journal, 873(2), 111.
  * Jurić et al. (2023) Jurić, M. et al., 2023.  Data products definition document.
  * Lasair-Team (2021) Lasair-Team, 2021.  Lasair client software, https://pypi.org/project/lasair/.
  * Lasair-Team (2023a) Lasair-Team, 2023a.  Lasair documentation, https://lasair.readthedocs.io/.
  * Lasair-Team (2023b) Lasair-Team, 2023b.  Lasair software repository, https://github.com/lsst-uk/lasair-lsst/.
  * Lasker et al. (2008) Lasker, B. M. et al., 2008.  The second-generation guide star catalog: Description and properties, The Astronomical Journal, 136(2), 735.
  * Massaro et al. (2013) Massaro, F. et al., 2013.  Bl lac candidates for tev observations, The Astrophysical Journal Supplement Series, 207(1), 16.
  * Matheson (2021) Matheson, T. o., 2021.  The antares astronomical time-domain event broker, The Astronomical Journal, 161(3), 107.
  * Nordin (2019) Nordin, J. o., 2019.  631(3), 107.
  * OpenCV-Team (2022) OpenCV-Team, 2022.  Opencv software, https://pypi.org/project/opencv-python/.
  * Raen et al. (2023) Raen, T. et al., 2023.  Pitt-google community broker, https://pitt-broker.readthedocs.io/en/latest/.
  * Ritter & Kolb (2003) Ritter, H. & Kolb, U., 2003.  Catalogue of cataclysmic binaries, low-mass X-ray binaries and related objects (Seventh edition), A&A, 404, 301–303.
  * RubinObservatory (2022) RubinObservatory, 2022.  Lsst data rights, https://www.lsst.org/content/data-rights.
  * Russeil et al. (2024) Russeil, E. et al., 2024.  Rainbow: A colorful approach to multipassband light-curve estimation, A&A, 683, A251.
  * Schipani et al. (2018) Schipani, P. et al., 2018.  SOXS: a wide band spectrograph to follow up transients, in Ground-based and Airborne Instrumentation for Astronomy VII, vol. 10702 of Society of Photo-Optical Instrumentation Engineers (SPIE) Conference Series, p. 107020F.
  * Schwamb (2019) Schwamb, M. o., 2019.  A software roadmap for solar system science with the large synoptic survey telescope, Research Notes of the AAS, 3(3), 51.
  * Sheng (2023) Sheng, X. o., 2023.  Neural engine for discovering luminous events (needle): identifying rare transient candidates in real time from host galaxy images, https://arxiv.org/abs/2312.04968.
  * Skrutskie (2006) Skrutskie, M. F. o., 2006.  The two micron all sky survey (2mass), AJ, 131(2), 1163–1183.
  * Smartt (2015) Smartt, S. J. o., 2015.  PESSTO: survey description and products from the first data release by the Public ESO Spectroscopic Survey of Transient Objects, A&A, 579, A40.
  * Smith & Williams (2019) Smith, K. W. & Williams, R. D. o., 2019.  Lasair: The transient alert broker for lsst:uk, Research Notes AAS, 3, 26.
  * Smith et al. (2022) Smith, K. W., O’Donnell, J., et al., 2022.  Superluminous supernovae, https://www.zooniverse.org/projects/mrniaboc/superluminous-supernovae/.
  * Steer et al. (2017) Steer, I., Madore, B. F., Mazzarella, J. M., Schmitz, M., Corwin, H. G., Chan, B. H. P., Ebert, R., Helou, G., Baker, K., Chen, X., Frayer, C., Jacobson, J., Lo, T., Ogle, P., Pevunova, O., & Terek, S., 2017.  Redshift-independent Distances in the NASA/IPAC Extragalactic Database: Methodology, Content, and Use of NED-D, AJ, 153(1), 37.
  * Swann et al. (2019) Swann, E., Sullivan, M., Carrick, J., Hoenig, S., Hook, I., Kotak, R., Maguire, K., McMahon, R., Nichol, R., & Smartt, S., 2019.  4MOST Consortium Survey 10: The Time-Domain Extragalactic Survey (TiDES), The Messenger, 175, 58–61.
  * Tachibana & Miller (2018) Tachibana, Y. & Miller, A. A., 2018.  A morphological classification model to identify unresolved panstarrs1 sources: Application in the ztf real-time pipeline, Publications of the Astronomical Society of the Pacific, 130(994).
  * Véron-Cetty & Véron (2010) Véron-Cetty, M. P. & Véron, P., 2010.  A catalogue of quasars and active nuclei: 13th edition, A&A, 518, A10.
  * Williams (2022) Williams, R. D., 2022.  Lasair data monitoring rgb matrix, https://github.com/RoyWilliams/RGBMatrix.
  * Williams (2023) Williams, R. D., 2023.  Fitting early light curves of explosive astrophysics, https://roywilliams.github.io/papers/Bazin_Exp_Blackbody.pdf.
  * Wiseman (2023) Wiseman, P. o., 2023.  Multiwavelength observations of the extraordinary accretion event AT2021lwx, Monthly Notices of the Royal Astronomical Society, 522(3), 3992–4002.
  * Young (2023a) Young, D. R., 2023a.  neddy. Query the Nasa Extra-Galactic from the command-line, https://zenodo.org/doi/10.5281/zenodo.8037632.
  * Young (2023b) Young, D. R., 2023b.  Sherlock. Contextual classification of astronomical transient sources, https://zenodo.org/doi/10.5281/zenodo.8038057.
  * ZTF-collaboration (2018) ZTF-collaboration, 2018.  Ztf alert archive, https://www.ztf.caltech.edu/ztf-alert-stream.html.

##  Appendix A Lightcurve Features

In addition to other object properties, lightcurve (photometric) features are
computed on each object, and users can build filters based on these. The Rubin
observatory will precompute many features (Bellm, 2023): a set of features
that comprehensively covers long-lived periodic and stochastic lightcurves.
But some of the added value of Lasair is extra lightcurve features tailored to
explosive events – emphasising very recent behaviour – either a long-term
lightcurve that brightens significantly, or a new transient that has appeared
recently.

Non-parametric features have no underlying model; rather, they are statistics,
such as mean, median, max flux, moving averages, rate of change of flux, and
so on. Lasair includes these and also has a feature fluxJump which crudely
looks for brightening: first take all fluxes together and compute the standard
deviation of this early flux, then compute how many standard deviations
different is the current flux. If a user wants to know about sudden
brightening, this a first criterion.

Figure 6: Early light curve simulated in six filters, fitted with both
exponential (left) and Bazin (right) for the time dimension and a constant-
temperature black-body in the wavelength dimension. In this case, the best-fit
criterion chooses the Bazin fit.

Parametric features imply fitting a model to the data. In the time dimension
such fits could include generic models like linear, polynomial, or
exponential, as well as more physically motivated models such as microlensing,
eclipsing binary, or Bazin fits (Bazin, 2011). Many proposed algorithms work
with a monochromatic lightcurve, or use the word “colour” to imply there are
only two filters whose fluxes can be subtracted. But LSST provides six
filters, so perhaps we should not simply build separate features for each
filter, but think of the “time-wavelength surface”. Lasair utilises this 2D
space to build a simultaneous black-body fit with either a Bazin curve or a
plain exponential in time, as in (Williams, 2023). Figure 6 shows two fits to
some simulated six-filter fluxes, on the left is a product of exponential rise
in time and black-body in wavelength, on the right is Bazin in time with
black-body. The diamond marks the discovery epoch. A goodness-of-fit criterion
picks the best fit as the right-hand plot. This two-dimensional approach to
fitting has been used later in other work (Russeil et al., 2024).