title= TriggerCit: Early Flood Alerting using Twitter and Geolocation - a
comparison with alternative sources , short title=TriggerCit: Early Flood
Alerting, author= short name = C. Bono, full name=Carlo Bono and Barbara
Pernici††thanks: corresponding authors, affiliation= Politecnico di Milano -
DEIB
<EMAIL_ADDRESS>, , author= full name= Jose Luis Fernandez-Marquez and
Amudha Ravi Shankar, affiliation= University of Geneva , , author= short name
= M. O. Mülâyim, full name= Mehmet Oğuz Mülâyim, affiliation= Artificial
Intelligence Research Institute (IIIA-CSIC) , , author= full name= Edoardo
Nemni, affiliation= United Nations Satellite Centre (UNOSAT), United Nations
Institute for Training and Research (UNITAR) , ,

###### Abstract

Rapid impact assessment in the immediate aftermath of a natural disaster is
essential to provide adequate information to international organisations,
local authorities, and first responders. Social media can support emergency
response with evidence-based content posted by citizens and organisations
during ongoing events. In the paper, we propose TriggerCit: an early flood
alerting tool with a multilanguage approach focused on timeliness and
geolocation. The paper focuses on assessing the reliability of the approach as
a triggering system, comparing it with alternative sources for alerts, and
evaluating the quality and amount of complementary information gathered.
Geolocated visual evidence extracted from Twitter by TriggerCit was analysed
in two case studies on floods in Thailand and Nepal in 2021.

###### keywords:

Social Media, Disaster management, Early Alerting

##  1 Introduction

The increasing use of mobile phones and social media has transformed the way
people witness and report climate and weather related disasters. Social media
has been demonstrated to be a worthy source of data to provide situational
awareness in the course of a disaster event. Effective response to natural
hazards, such as floods, can reduce loss of life and mitigate structural
damages. Access to timely and accurate data is essential to the disaster
response. The effectiveness of the humanitarian response is outside the scope
of this work. Instead, we focus on the mechanism of early alert using social
media, under the assumption that supporting the activation of the disaster
response protocol with pertinent and geolocated evidence is beneficial for the
affected community.

Remote sensing data such as satellite images have also proven to be a valuable
source of information to support humanitarian relief and disaster response.
Advances in Artificial Intelligence (AI) provide a way of extracting relevant
information from satellite imagery and thus contribute to speeding up the
damage assessment. Even though computer vision techniques can process large
amounts of data in a short time, the triggering of such systems is often tied
to manual or semi-automated procedures.

To leverage an alternative and abundant source - social media - this paper
proposes a methodology for early flood alerting using Twitter, providing a
systematic approach to extract near-real-time and geolocated evidence about
onsets of flood events. The goal of this paper is to analyse how this
information correlates with existing alert mechanisms.

The paper is structured as follows: the next section presents existing
disaster alert and trigger mechanisms, and the current use of social media as
an early alerting system. In the subsequent section, the TriggerCit
methodology for extracting information from social media posts is illustrated.
Then, the TriggerCit methodology is applied to two different case studies for
flood events in Thailand and Nepal and the results are discussed.

##  2 Related work

###  2.1 Existing disaster alert and trigger mechanisms

Multi-hazard early warning systems and disaster alert mechanisms are a
significant component of disaster risk reduction. They help prevent loss of
life, reduce the impact of natural hazards, and assist responders in disaster
handling. In this section, we will discuss two key state-of-the-art alert
mechanisms.

_The Global Disaster Alert and Coordination System_ 111https://www.gdacs.org
(GDACS), started in 2004, is a joint initiative of the United Nations Office
for the Coordination of Humanitarian Affairs (UN
OCHA)222https://www.unocha.org and the European Commission (EC). It is
designed to alert the international community during sudden-onset disasters
that might require international assistance. GDACS serves to consolidate and
improve the dissemination of disaster-related information and coordination in
the first phase of a disaster. The GDACS website consists of different tools,
which have been incorporated into disaster response plans by several
governments and organisations. The GDACS Disaster Alerts are auto-generated
based on algorithms and available data to alert oncoming disasters. The system
calculates alerts based on specific risk indicators, like the severity of the
disaster, the affected population, and the country’s vulnerability.
Subscribers are notified via Email and SMS ([8]). GDACS information is openly
accessible. For the first phase of a disaster, governments and disaster
response organisations rely on GDACS alerts and automatic impact estimations
to plan international assistance. The Virtual On-Site Operations Coordination
Center (Virtual OSOCC)333https://vosocc.unocha.org facilitates information
exchange and coordination.

_The Global Flood Awareness System_ 444https://www.globalfloods.eu (GloFAS) is
a component of the Copernicus Emergency Management Service (CEMS) that
delivers global hydrological forecasts to registered users. GloFAS combines
information from the satellites and forecast models to produce GloFAS
forecasts, Seasonal forecasts, and Impact Forecasts ([8, 22]). The GloFAS
Forecasts are available free of cost to all, with registered users from Hydro-
Meteorological services, academia, and the humanitarian sector ([5]).

###  2.2 Social media as early warning and alerting systems

Several social media sources have been considered in the literature, the main
one being Twitter555https://twitter.com as it provides public application
programming interfaces (APIs) to search posts and related metadata. Several
research works have been published on tweets analysis. Here we focus on tweet
analysis for early event detection. One of the first papers on the topic is
[16], in which the focus is on the timeliness of the extracted information, in
particular in the case of earthquakes. Like in other papers analysing social
media, the post geolocation is extracted from Twitter metadata. As studied in
the literature (e.g., [17]), this is usually available only in a small
percentage of the posts. Recent papers focused on analysing in parallel
several sources of information for deriving awareness on emergency events. In
[19], the impact of extensive floods in Japan has been assessed using
hydrogeological information and tweets for detecting event outbreaks and
phases in the affected areas.

The importance of images and geolocation via text has been recently emphasised
to overcome the limitations of native Twitter geolocations. There are several
advantages of having precisely geolocated tweets. [3] show how a very limited
number of tweets, combined with elevation profiles, can give a precise map of
flooded areas; also [18] emphasise the integration of tweet data and hydraulic
modeling for mapping flooded areas. [21] uses text to geolocate posts and to
classify phases of events automatically. The geolocation steps are based on
Named-Entity Recognition (NER) and gazetteers to identify locations. However,
the focus is mainly on locations in the US and the English language.

A more comprehensive approach to social media posts geolocation has been
proposed in the CIME system ([17]). The NER is based on Polyglot ([14]), which
supports more than 40 languages. CIME disambiguates candidate locations using
Nominatim666https://nominatim.openstreetmap.org, a search engine for
OpenStreetMap data. Disambiguation considers candidate locations, their
distances, and their ranks in the OpenStreetMap administration level
hierarchy.

Post geolocation can achieve a significant, although variable, level of
precision. CIME was exploited in the E2mC project ([4]) to support rapid
mapping activities within Copernicus Emergency Management Services
(EMS)777https://emergency.copernicus.eu. An important issue that emerged is
the high number of duplicate and non-relevant images. A systematic approach to
image filtering for geolocated tweets has been proposed in [10], developing
the VisualCit image processing toolkit, aimed at extracting indicators about
emergency events. The toolkit allows image filtering based on contents, such
as similarity, non-photographic content, presence of specific objects, and
event-specific classifiers (including floods, see [15]). Other authors have
proposed to create event-specific classifiers (e.g., [6]) automatically,
without focusing on locations and systematic data preprocessing.

Another important issue is determining the severity of events using tweets as
a source of citizen knowledge. A combination of sentiment analysis and
frequency of impact-related keywords has been used in [7].

The availability of a representative set of search keywords is paramount for
leveraging social media data. The goal is to get sufficient coverage of the
event, both in the initial phase and as the event unfolds, as shown in [12].
In several papers, general purpose keywords are used. A systematic approach
has been developed in E2mC based on data mining techniques for multilingual
keyword extraction ([9]).

In the present paper we propose TriggerCit, a systematic methodology to
extract flood alerts from Twitter posts. We focus on detecting the occurrence
of flood events within the first 48 hours of the event, and describing them at
the regional administrative level, providing image evidence with detailed
locations. The goal is to select localised, relevant information from social
media in a short time. The approach leverages VisualCit processing and CIME
geolocation. The results are compared, over two case studies, with data from
the event alert systems described in 2.1.

##  3 Methodology

The proposed approach, shown in Fig. 1, is implemented as an analytics
pipeline aimed at detecting flood events and extracting high-quality event-
related information. The components are adapted to Twitter. However, most of
the concepts can be generalised to other compatible social media.

Figure 1: TriggerCit methodology for identifying occurrences of flood events
and related locations.

The main steps of TriggerCit are illustrated in the following.

  1. 1.

_Word dictionaries for social sensing_
A _seed dictionary_ , which consists of a curated dictionary of words, is
built for each language and event of interest. Ideally, this dictionary should
be rich enough to account event variability, yet specific enough to match
relevant messages only. Language-dependent dictionaries should be curated by
expert native speakers. Moreover, the same event could be described in
different languages, since multiple languages could be spoken in affected
regions. In the current experiments, we assume that a small general-purpose
set of terms is available, as we provide novel techniques for augmenting the
event-related set of words (see Step 7).

  2. 2.

_Word count monitoring for event detection_
With proper linguistic resources in place, a monitoring system for disaster-
related terms can be implemented. Dictionaries can be used to build one or
more queries towards this goal. We chose to build a query as the logical OR of
the words in the dictionary. Alternatively, per-word time series can be used.
Getting the time series of the number of matched posts is straightforward with
Twitter APIs. The main advantage of querying counts is that no actual tweet
content data is exchanged, so almost no computation is involved at this stage.
Moreover, no tweet quota is consumed, which could be critical if the number of
queries to monitor is high (e.g., many countries/languages). Rate limits are
compatible with near-real-time monitoring. Geographical boundaries can be
specified: however, this will limit the results to the few tweets that are
natively geotagged in Twitter, which is usually a limiting factor as discussed
in [17] and other works.

  3. 3.

_Triggering_
Using the counts time series, an event triggering system can be designed.
Ideally, the component should be a supervised classifier, using one or more
validated datasets such as historical GDACS alerts for the event starts (and
ends) as reference data. A more straightforward mechanism triggers an event if
the tweet count growth exceeds some threshold.

  4. 4.

_Get tweet contents_
If an event is likely occurring, we selectively download tweet data for the
time interval of the event using the Twitter search endpoint, excluding
retweets. Full-archive search API is available for research purposes. The
recent search API is also available, which is accessible to the general public
and limited to the last 7 days.

  5. 5.

_Extract relevant content with VisualCit_
We then concentrate on tweets with images, applying a VisualCit semantic
filtering pipeline to extract relevant contents. The tasks in the pipeline are
ordered “quickest first” as a heuristic to get a faster overall processing
time. The proposed pipeline consists of the following tasks:

    1. (a)

Remove duplicated images and similar images (e.g., rescaled versions, modified
versions).

    2. (b)

Remove non-photos, such as drawings, screenshots and computer-generated
images.

    3. (c)

Remove not-safe-for-work images.

The confidence thresholds for the single tasks are chosen through empirical
validation, to achieve a reasonable performance without substantially removing
useful data.

  6. 6.

_Use CIME for geolocation_
The CIME geolocation algorithm described in [17] is then applied to associate
tweets with geographical entities, whenever possible. Locations in unrelated
countries are filtered out using national administrative boundaries.

  7. 7.

_Use pipeline output as feedback for input_
Having a selected set of relevant items could be useful to leverage the
significant, unfiltered mass of tweets without images. A straightforward
approach could be extracting tweets related to the ones in the VisualCit
output, searching in the broader set of text-only tweets retrieved in Step 4.
Textual frequency analysis may help us to identify keywords that were not
explicitly listed in the original seed dictionary. New keywords can then be
posed as a query to the corpus of text-only tweets. A simple measure like the
TF-IDF score (possibly combined with the widespread cosine similarity) can
rank individual tweets concerning this query. Selected text-only tweets with a
score higher than some threshold (henceforth referred to as _promising text-
only tweets_) can then be fed to CIME for geolocation, thus expanding the
real-time geographical description of the event. In next section, focusing on
the use case in Thailand, we show how this approach can effectively augment
the map of relevant tweets. This approach could be further extended in order
to refine the input space by recovering information with additional queries in
Step 4.

  8. 8.

_Visualization_
The process output consists of a representative set of social media posts,
used to deliver thematic maps providing geolocated evidence to stakeholders.

##  4 Case studies

In this section, we apply the proposed methodology to two flood events that
occurred in 2021 in Thailand and Nepal. The two events were selected since the
United Nations Satellite Centre (UNOSAT)888https://unitar.org/sustainable-
development-goals/united-nations-satellite-centre-UNOSAT supported both
activations in Nepal999https://unitar.org/maps/countries/70 and
Thailand101010https://unitar.org/maps/countries/100 with satellite-derived
maps and AI-based flood detection analysis, updating operational dashboards in
near-real-time111111https://unitar.org/about/news-stories/news/unosat-flood-
ai-dashboards-nepal-creation-one-stop-shop-real-time-evidence-based-decision-
making and providing daily updates.

###  4.1 Thailand Flood Event - September-October 2021

####  4.1.1 Event description

In late September and October 2021, tropical storm Dianmu inundated regions in
Vietnam, Laos, Cambodia, and Thailand. According to the UNOSAT Thailand flood
monitoring dashboard121212https://unosat-
geodrr.cern.ch/portal/apps/opsdashboard/index.html#/4f878691713a40f3b8ef3140e63c9f6d,
around 14.3 square kilometers of the country were inundated between September
27 and October 27, 2021. An estimated 1.4 million people were affected by the
flooding, with 32 of 76 provinces affected by heavy rains for nearly a month.
The statistics mentioned above were computed in the aftermath of the flood
events and should be considered as a preliminary analysis since they have not
been validated yet in the field.

####  4.1.2 Data description

We queried Twitter with the logical OR of the three most frequent terms in the
dictionary (see Fig. 2(a)) of Thai terms correlated to flood events and
provided by the UNOSAT office in Bangkok. The event onset is clear using the
Thai dictionary, but not with the equivalent English dictionary (see Fig.
2(b)).

(a) Thai entries

(b) Thai / English

Figure 2: Daily tweet counts for Thai dictionary entries

We concentrated the analysis on the 48 hours between September 26, 00:00 UTC
and September 28, 00:00 UTC, since there is marked evidence of the event
onset. Over this period, more than 4 million tweets match the seed query.

Data | Count
---|---
All tweets, September 26-27 | 4’145’447
No retweets | 66’868
Containing images | 6’292
Native Twitter locations | 227
Overall images | 8’774
Passed VisualCit filters | 3’056
Places geolocated by CIME | 1’671
Table 1: Dataset cardinality through processing, Thailand case study Level | Admin level | Count
---|---|---
Province | 4 | 7
District | 6 | 8
Municipality or subdistrict | 8 | 176
Village or community | 10 | 9
Other points | 15 | 1’265
Table 2: Administrative level counts for CIME geolocations

Processing output
We first focused on processing the 8’774 images available131313The datasets
for this paper will be made available on Zenodo. with VisualCit (see Fig. 4
for an example of the result). The VisualCit pipeline is configured to remove
duplicates, non-photos, and not-safe-for-work contents. The average processing
time for each item, including network overheads, is 171ms (roughly 20’000
elements/hour) on a single server.

(a) Native Twitter geolocation

(b) VisualCit geolocation

(c) VisualCit geolocation with extended tweet set

(d) Affected people

Figure 3: Geolocations / inhabitants ratio by region (a) Twitter native
geolocations, (b) Twitter native + CIME geolocated, (c) Twitter native + CIME
geolocated from extended dataset (with images + promising text-only tweets),
and (d) Number of affected persons by region at September, 28th (source:
ReliefWeb)

The resulting dataset is then processed with the CIME algorithm in order to
extract locations. The distribution of the places extracted by CIME does not
simply replicate the distribution of the native geotags, as it can be seen
comparing Fig. 3(a) and Fig. 3(b), and allows a clearer description of
affected areas. In our experimental setup, to the 206 locations natively
present in the tweets and located in Thailand, we can add 1’465 locations that
are within Thailand state borders. The average processing time for each item
with CIME is 224ms141414With no parallelism. The correlation between the
geolocated data and the affected areas, as described by official
sources151515https://reliefweb.int/sites/reliefweb.int/files/resources/FlashUpdate_02_28Sep2021-TC-
DIANMU-THLVNMLAO.pdf, can be appreciated in Fig. 3. Population estimates by
province used for data normalisation were extracted from
Wikipedia161616https://en.wikipedia.org/wiki/Provinces_of_Thailand. Mainly,
specific locations or entities with sub-district administrative level are
extracted (see Table 2171717OpenStreetMap administrative levels for Thailand
https://wiki.openstreetmap.org/wiki/Tag:boundary=administrative.). Data
cardinalities through processing steps are described in Tab. 1.

Fig. 3(c) shows how we can further extend location analysis by exploiting the
text-only tweets as described in Step 7 of the TriggerCit methodology. A word
frequency analysis of the VisualCit output provided us with two new keywords
(water and house). We posed these two together with the original seeds as a
query to the text-only corpus and measured the cumulative TF-IDF score of each
tweet. We regarded the 6’021 tweets within the 0.9 percentile of the score as
_promising_ and fed them to CIME for geolocation. 661 of these tweets were
successfully geolocated within Thailand by CIME.

(a) Removed items

(b) Kept items

Figure 4: Example images for (a) items being removed by the pipeline, and (b)
items being kept

####  4.1.3 Assessment of the results

TriggerCit outputs an alert on September 27 due to the increase of tweets
compared to the days before considered as a baseline. It also returns a
weighted set of administrative levels where the alert is coming from, which
partially correlates with the number of affected people and gives a
preliminary indication of the affected regions.

###  4.2 Nepal Flood Event - June-July 2021

####  4.2.1 Event description

Between June and October 2021, 673 people have lost their lives in Nepal due
to floods and landslides caused by seasonal and unseasonal heavy rain and
severe
weather181818https://reliefweb.int/disaster/fl-2021-000134-npl#overview. Among
the events at the beginning of the monsoon season, heavy rain triggered
multiple landslides and flooding on the Melamchi river, in the Sindhupalchok
district, resulting in more than 20
deaths191919https://en.wikipedia.org/wiki/2021_Melamchi_flood. Damage and
casualties followed since late June and the beginning of July, marking the
onset of the disaster event. The case study focuses on evidence from tweets
gathered in this early stage.

####  4.2.2 Data description

Using a pre-defined dictionary of Nepali words generically related to flood
events, the query consisted of the logical OR of the terms (see Fig. 5). As in
the case of Thailand floods, it is difficult to extract evidence with English
seed keywords (see Fig. 5(c)).

(a) Nepali entries in June

(b) Nepali entries in July

(c) Nepali / English

Figure 5: Daily tweet counts for Nepali dictionary entries

The most evident event using our dictionary happened in the middle of June, as
seen in Fig. 5(a), which coincides with the event dates reported in the news.
Over June 16-17, 6’639 tweets matched the query. It is relevant that, among
the few posts with native Twitter geolocations, none contained information
specific to this event.

We replicated the analysis for the first two days of July, since in these days
several incidents with casualties happened in different places of
Nepal202020https://floodlist.com/asia/nepal-floods-landslides-july-2021. In
this case, the time series of tweets matching our query show an increasing
trend, albeit less evident than in the previous case.

Data | June 16-17 | July 1-2
---|---|---
All tweets | 6’639 | 1’225
No retweets | 2’807 | 594
Containing images | 261 | 63
Native Twitter locations | 8 | 10
Overall images | 391 | 80
Passed through VisualCit | 218 | 55
Places geolocated with CIME | 51 | 10
Table 3: Data cardinality in Nepal case study for June 16-17 and July 1-2

####  4.2.3 Processing output

The experimental setup is equivalent to the one used for the Thailand case
study. Data cardinalities through processing are reported in Tab. 3. Since the
pipeline did not support Nepali natively, we processed Nepali texts with a
publicly available NER tool for Nepali212121https://github.com/oya163/nepali-
ner, described in [20]. The processed posts are then fed to a modified CIME
algorithm for location extraction.

For data in June 16-17, among the 51 locations extracted, 21 refer to the
Bagmati province, in which the event is taking place, and 19 explicitly refer
to Sindhupalchok district (as Fig. 6 shows). Another 11 generically refer to
Nepal, so they are ignored. It is interesting to note that, compared to the
GloFAS Rapid Impact Assessment map, there is a complementarity regarding this
event, since there are no reporting points for the Sindhupalchok district.

For data in July 1-2, an overview of the processed data is reported in Table
3. The amount of evidence gathered is smaller than the previous time frame,
with only 6 useful locations extracted after running the images through the
pipeline (4 out of 10 are generic mentions of Nepal), and 14 in the overall
set of tweets with images. Nevertheless, it is worth comparing the results
with different sources of information to understand how to complement
available information, which is not always concordant and could suffer from
data scarcity. In Fig. 7 we gathered figures for different signals on July 2,
coming from UNOSAT, GloFAS, TriggerCit, and reports from the Nepal Disaster
Risk Reduction portal (NDRR)222222Events of flood, heavy rain and landslides
reported from http://drrportal.gov.np/ for July 1-2. The relevance of the
locations extracted with our approach is partial, but significant in some
cases. The limited amount of data is possibly related to the specificity of
our queries, which are focused on flood terms, while 50 out of 59 incidents
reported by NDRR have a “Heavy Rainfall” or “Landslides” label. The impact of
a tailored dictionary should be further investigated.

(a) Nepali tweets geolocated with CIME - June 16-17, 2021

(b) GloFAS reporting points and Rapid Impact Assessment map - June 17, 2021

Figure 6: Nepali case study - June 16-17, 2021

(a) UNOSAT Flood AI Monitoring Dashboard - July 1-2, 2021

(b) GloFAS reporting points and Rapid Impact Assessment map - July 2, 2021

(c) Geolocated Nepali tweets by district - July 1-2, 2021

(d) Nepal Disaster Risk Reduction portal incident count by district - July
1-2, 2021

Figure 7: Nepali case study - July 1-2, 2021

####  4.2.4 Assessment of the results

TriggerCit outputs an alert on June 16 due to the increase of tweets compared
to the days before considered as baseline (Fig. 6). This trigger is related to
a specific local event. Heavy rainfall caused a number of floods and
landslides, across different regions of Nepal during the following weeks.
Comparing the two periods, it appears that the peaks of tweets are more likely
for sudden unexpected events, rather than for an expected event which is
developing over time with many local events over an extended area. The
corresponding maps and reference maps are shown in Fig. 7. The ability of
TriggerCit of keeping track of these events is still under evaluation.

##  5 Comparison with alternative sources

UNOSAT Rapid Mapping Service232323https://www.unitar.org/maps/unosat-rapid-
mapping-service provides satellite image analysis during humanitarian
emergencies, both natural disasters, and conflict situations. With a 24/7
year-round availability to process requests, a team of experienced analysts
ensures timely delivery of satellite imagery-derived maps, reports, and data
ready for direct inclusion in Geographic Information Systems (GIS) according
to needs. The service can be activated by UN offices and agencies Government
agencies, the Red Cross and Red Crescent Movement (ICRC and IFRC),
international and regional organizations, and Humanitarian Non-Governmental
Organizations (NGOs). As shown in Table 4, UNOSAT was activated on September
28, 2021 for the flood event in Thailand and on June 30, 2021 for the flood in
Nepal. The activation data refers to the day when UNOSAT received the request
to support the flood events. We were not able to find any flood activation
from Copernicus
EMS242424https://emergency.copernicus.eu/mapping/#zoom=2&lat=27.6533&lon=-25.0083&layers=0BT00
in Thailand and Nepal respectively.

GDACS reported floods on September 27, 2021 for
Thailand252525https://www.gdacs.org/report.aspx?eventtype=FL&eventid=1101113,
and on June 28, 2021 for
Nepal262626https://www.gdacs.org/report.aspx?eventid=1100949&episodeid=1&eventtype=FL
both with a green GDACS
score272727https://www.gdacs.org/Knowledge/models_FL.aspx of 0.5. The event on
June 16, 2021 was not recorded in GDACS following floods reported on the GDACS
website282828https://www.gdacs.org/flooddetection/currentfloods.aspx. We found
the events reported by GloFAS using the GloFAS Map
Viewer292929https://www.globalfloods.eu/glofas-forecasting/ on September, 24
in Thailand with a yellow and red score of Rapid Impact Assessment and on June
28, 2021 in Nepal with a orange score of Rapid Impact Assessment. FloodList
also reported news of flood in
Thailand303030https://floodlist.com/asia/thailand-tropical-storm-dianmu-
floods-september-2021 on September 27, 2021, and in Nepal on July 4,
2021313131https://floodlist.com/asia/nepal-floods-landslides-july-2021 and on
June 16, 2021323232https://floodlist.com/asia/bhutan-and-nepal-flash-floods-
leave-at-least-10-dead-many-missing.

In the two scenarios, TriggerCit respectively returned a large scale and a
local scale alert, both in a timely manner and accompanied by a valid
geographical description, while providing information complementary to
existing disaster alert mechanisms (e.g., images).

Dates / Countries | Thailand | Nepal
---|---|---
UNOSAT activation | 28/09/2021 | 30/06/2021
Copernicus EMS activation | None | None
GDACS Disaster Alerts | 27/09/2021 (Green Alert) | 28/06/2021 (Green Alert)
GloFAS | 24/09/2021 | 28/06/2021
FloodList reported news | 27/09/2021 | 04/07/2021
TriggerCit | 26/09/2021 | 02/07/2021
Table 4: Timeline of alerts for the Thailand and Nepal activations

##  6 Limitations and Future Work

Linguistic limitations while mining multi-lingual tweets are mentioned in most
of the works discussed in the state-of-the-art. The task is usually easier if
the tweets are in English, given the quality of linguistic resources and the
number of posts available. The use of native languages for communication has
always posed a significant challenge to social media mining. In this paper, we
propose a multi-language and extensible approach towards a solution to this
problem. Further work is needed to develop fast and reliable seed dictionaries
for social media monitoring with TriggerCit. For this purpose, several
directions have to be investigated, leveraging on local resources, including
local administrations and NGOs, and citizens. They can provide information
about the nature of repeating local events, such as monsoons, and their
consequences, to be mapped to keywords. Citizen scientists can also support in
the evaluation of extracted images with crowdsourcing, as it has been proposed
in the Crowd4EMS project ([13]), as well as of the resulting keywords of the
TriggerCit methodology (see Step 7). A software library could then be used to
automate the aggregation of annotations (e.g., [2]).

As discussed in the paper, Twitter has different usage rates in different
countries. Alternatives to Twitter include other social media like Facebook,
Telegram, and WhatsApp. These are used to facilitate communication between
volunteers and emergency services, enable sharing of information within a
community, providing swift updates on emergency situations in real-time.
However, even though such platforms contain a wealth of information, access to
data is usually restricted (e.g., for privacy concerns). On the other hand,
systems like Facebook Disaster Maps provide aggregated insights for crisis
response and recovery and its use should be further investigated. In addition,
as some tweets contain URLs pointing to other contents, such as web pages
(e.g., newspapers) or posts in other social networks, these contents can also
be crawled in order to obtain more information about the event. Moreover, the
information extracted from one social media could be leveraged to crawl other
platforms iteratively, as in [1].

Further directions to be explored in order to enhance TriggerCit capabilities
include:

  * •

Training a supervised learning classifier for triggering
We are still in the phase of collecting high-quality time series data related
to a representative number of flood events. Once a large dataset is available,
we plan to experiment with supervised learning approaches, in order to
automate the triggering decision and validate it against a variety of events
(e.g., different scales and areas). Such a classifier should also include
contextual data about the event, information about flooded areas (such as the
ones automatically classified in [11]), and about the readiness of the areas
in terms of preparation measures for recurring flood events.

  * •

Automating dictionary management
Our system needs high-quality dictionaries for each language. Part of the
future work will focus on automatically deriving event-specific keywords, both
in general and adapted to a particular event. This automation would enhance
the sensing phase, in which the system is monitoring social media for
potential triggers. On the other hand, online adaptation to a specific event
could increase the recall of the process.

##  7 Concluding Remarks

The paper proposed a multilingual methodology to provide early flood alerts
with associated images and locations. The results in two case studies in Asia
in 2021 show that the alert is consistent in the Thailand case and that a
timely evidence could have been provided for the Nepal floods. Based on this
analysis, TriggerCit could be deployed at UNOSAT into a cloud centralised
service at CERN built on Kubeflow, a machine learning platform running on
Kubernetes, allowing the integration with automatic flood detection from
satellite data.

##### Acknowledgements

The work at Politecnico di Milano, University of Geneva and IIIA-CSIC was
funded by the European Commission H2020 Project Crowd4SDG, #872944.

The work at the United Nations Satellite Centre (UNOSAT) is part of the
operational mapping service funded by the Norwegian Ministry of Foreign
Affairs.

## References

  * [1] Andrea Autelitano, Barbara Pernici and Gabriele Scalia  “Spatio-temporal mining of keywords for social media cross-social crawling of emergency events”  In _GeoInformatica_ 23.3, 2019, pp. 425–447  DOI: 10.1007/s10707-019-00354-1
  * [2] Jesus Cerquides and Mehmet Oğuz Mülâyim  “crowdnalysis: A software library to help analyze crowdsourcing results”  Zenodo, https://doi.org/10.5281/zenodo.5898579, 2022  DOI: 10.5281/zenodo.5898579
  * [3] Joachim Fohringer, Doris Dransch, Heidi Kreibich and Kai Schröter  “Social media as an information source for rapid flood inundation mapping”  In _Natural Hazards and Earth System Sciences_ 15.12 Copernicus GmbH, 2015, pp. 2725–2738
  * [4] Clemens Havas, Bernd Resch, Chiara Francalanci, Barbara Pernici, Gabriele Scalia, Jose Luis Fernandez-Marquez, Tim Van Achte, Gunter Zeug, Maria Rosa Rosy Mondardini and Domenico Grandoni  “E2mC: Improving emergency management service practice through social media and crowdsourcing analysis in near real time”  In _Sensors_ 17.12 Multidisciplinary Digital Publishing Institute, 2017, pp. 2766
  * [5] “HydroConference — WMO”, https://hydroconference.wmo.int/en/partners/products/global-flood-awareness-system-glofas, Accessed on 01/21/2022
  * [6] Muhammad Imran, Ferda Ofli, Doina Caragea and Antonio Torralba  “Using AI and Social Media Multimodal Content for Disaster Response and Management: Opportunities, Challenges, and Future Directions”  In _Information Processing & Management_ 57.5 Elsevier, 2020, pp. 102261
  * [7] Nayomi Kankanamge, Tan Yigitcanlar, Ashantha Goonetilleke and Md Kamruzzaman  “Determining disaster severity through social media analysis: Testing the methodology with South East Queensland Flood tweets”  In _International journal of disaster risk reduction_ 42 Elsevier, 2020, pp. 101360
  * [8] Dimitrios Kavallieros, George Leventakis, Stefanos Malliaros, Ioannis Daniilidis and Vasileios Grizis  “PPDR Information Systems-A Current Status Review Report.”  In _ISCRAM_ , 2015
  * [9] Birgit Kirsch, Sven Giesselbach, David Knodt and Stefan Rüping  “Robust End-User-Driven Social Media Monitoring for Law Enforcement and Emergency Monitoring”  In _Community-Oriented Policing and Technological Innovations_ Springer, Cham, 2018, pp. 29
  * [10] Virginia Negri, Dario Scuratti, Stefano Agresti, Donya Rooein, Gabriele Scalia, Amudha Ravi Shankar, Jose Luis Fernandez-Marquez, Mark James Carman and Barbara Pernici  “Image-Based Social Sensing: Combining AI and the Crowd to Mine Policy-Adherence Indicators from Twitter”  In _43rd IEEE/ACM International Conference on Software Engineering: Software Engineering in Society, ICSE (SEIS) 2021, Madrid, Spain, May 25-28, 2021_ IEEE, 2021, pp. 92–101  DOI: 10.1109/ICSE-SEIS52602.2021.00019
  * [11] Edoardo Nemni, Joseph Bullock, Samir Belabbes and Lars Bromley  “Fully Convolutional Neural Network for Rapid Flood Segmentation in Synthetic Aperture Radar Imagery”  In _Remote Sensing_ 12.16, 2020  DOI: 10.3390/rs12162532
  * [12] Alexandra Olteanu, Carlos Castillo, Fernando Diaz and Sarah Vieweg  “CrisisLex: A Lexicon for Collecting and Filtering Microblogged Communications in Crises”  In _Proceedings of the International AAAI Conference on Web and Social Media_ 8.1, 2014, pp. 376–385  URL: https://ojs.aaai.org/index.php/ICWSM/article/view/14538
  * [13] Amudha Ravi Shankar, Jose Luis Fernandez-Marquez, Barbara Pernici, Gabriele Scalia, Maria Rosa Mondardini and Giovanna Di Marzo Serugendo  “Crowd4Ems: A crowdsourcing platform for gathering and geolocating social media content in disaster response”  In _International Archives of the Photogrammetry, Remote Sensing and Spatial Information Sciences_ 42 Copernicus, 2019, pp. 331–340
  * [14] Rami Al-Rfou, Vivek Kulkarni, Bryan Perozzi and Steven Skiena  “Polyglot-NER: Massive multilingual named entity recognition”  In _Proceedings of the 2015 SIAM International Conference on Data Mining_ , 2015, pp. 586–594  SIAM
  * [15] Olimpia Rivera, Juan Felipe Calderon, Paul Planchon and Barbara Pernici  “Evaluating the impact of floods on gender equality from social media evidence”  In _2nd AISWN International Research Workshop on Women IS and Grand Challenges @ICIS_ , 2021
  * [16] Takeshi Sakaki, Makoto Okazaki and Yutaka Matsuo  “Tweet Analysis for Real-Time Event Detection and Earthquake Reporting System Development”  In _IEEE Trans. Knowl. Data Eng._ 25.4, 2013, pp. 919–931  DOI: 10.1109/TKDE.2012.29
  * [17] Gabriele Scalia, Chiara Francalanci and Barbara Pernici  “CIME: Context-aware geolocation of emergency-related posts”  In _GeoInformatica_ Springer, 2021, pp. 1–33
  * [18] Vincenzo Scotti, Mario Giannini and Francesco Cioffi  “Enhanced flood mapping using synthetic aperture radar (SAR) images, hydraulic modelling, and social media: A case study of Hurricane Harvey (Houston, TX)”  In _Journal of Flood Risk Management_ 13.4 Wiley Online Library, 2020, pp. e12647
  * [19] Kikuko Shoyama, Qinglin Cui, Makoto Hanashimaa, Hiroaki Sano and Yuichiro Usuda  “Emergency flood detection using multiple information sources: Integrated analysis of natural hazard monitoring and social media data”  In _Science of the Total Environment_ 767.144371, 2021, pp. 1–11
  * [20] O.. Singh, A. Padia and A. Joshi  “Named Entity Recognition for Nepali Language”  In _2019 IEEE 5th International Conference on Collaboration and Internet Computing (CIC)_ , 2019, pp. 184–190  DOI: 10.1109/CIC48465.2019.00031
  * [21] Ruo-Qian Wang, Yingjie Hu, Zikai Zhou and Kevin Yang  “Tracking Flooding Phase Transitions and Establishing a Passive Hotline With AI-Enabled Social Media Data”  In _IEEE Access_ 8, 2020, pp. 103395–103404  DOI: 10.1109/ACCESS.2020.2994187
  * [22] Zuzanna Zajac, Beatriz Revilla-Romero, Feyera Aga Hirpa, Victor Ntegeka, Peter Salamon, Jutta Thielen, Peter Burek and Hylke Beck  “Recent advances in the Global Flood Awareness System (GloFAS)”  In _EGU General Assembly Conference Abstracts_ , 2014, pp. 12182


# Supervised Radio Frequency Interference Detection with SNNs ††thanks: This
work was supported by a Westpac Future Leaders Scholarship, an Australian
Government Research Training Program Fees Offset and an Australian Government
Research Training Program Stipend.

Nicholas J. Pritchard  International Centre for Radio Astronomy Research
University of Western Australia
Perth, Australia
0000-0003-0587-2310  Andreas Wicenec  International Centre for Radio Astronomy
Research
University of Western Australia
Perth, Australia
0000-0002-1774-5653  Mohammed Bennamoun  School of Physics, Mathematics and
Computing
University of Western Australia
Perth, Australia
0000-0002-6603-3257  Richard Dodson  International Centre for Radio Astronomy
Research
University of Western Australia
Perth, Australia
0000-0003-0392-3604

###### Abstract

Radio Frequency Interference (RFI) poses a significant challenge in radio
astronomy, arising from terrestrial and celestial sources, disrupting
observations conducted by radio telescopes. Addressing RFI involves intricate
heuristic algorithms, manual examination, and, increasingly, machine learning
methods. Given the dynamic and temporal nature of radio astronomy
observations, Spiking Neural Networks (SNNs) emerge as a promising approach.
In this study, we cast RFI detection as a supervised multi-variate time-series
segmentation problem. Notably, our investigation explores the encoding of
radio astronomy visibility data for SNN inference, considering six encoding
schemes: rate, latency, delta-modulation, and three variations of the step-
forward algorithm. We train a small two-layer fully connected SNN on simulated
data derived from the Hydrogen Epoch of Reionization Array (HERA) telescope
and perform extensive hyper-parameter optimization. Results reveal that
latency encoding exhibits superior performance, achieving a per-pixel accuracy
of 98.8% and an f1-score of 0.761. Remarkably, these metrics approach those of
contemporary RFI detection algorithms, notwithstanding the simplicity and
compactness of our proposed network architecture. This study underscores the
potential of RFI detection as a benchmark problem for SNN researchers,
emphasizing the efficacy of SNNs in addressing complex time-series
segmentation tasks in radio astronomy.

###### Index Terms:

radio astronomy, spiking neural networks, neuromorphic computing, supervised
learning, multi-variate time-series segmentation

##  I Introduction

Radio astronomy is a data-intensive science involving building and operating
sophisticated observatories and high-performance computing facilities. Since
it is a fundamentally observational science, removing unwanted radio frequency
interference (RFI) from artificial technology on Earth and in Space is crucial
to an observatory’s scientific capability. Detecting RFI is a challenging task
that increasingly requires adaptive data-driven methods in an era of
increasing radio telescope sensitivity and the presence of terrestrial and
satellite communication [1]. State-of-the-art RFI detection methods are either
domain-specific traditional algorithms, e.g., [2] or CNNs, e.g., [3]; both
families of methods treat RFI detection as a boolean segmentation task on two-
dimensional spectrograms (‘visibility data’ or ‘visibilities’). Traditional
methods often require expert intervention, which becomes untenable as radio
telescopes become larger, more sensitive, and produce more observations.
Inspired by SNNs’ performance in audio-based processing tasks, we formulate
RFI detection as a supervised time-series segmentation problem to explore how
to encode visibility data for SNNs without prior offline analysis and exploit
the time-varying nature of radio astronomy observations. We present this task
as a challenging benchmark problem ideal for SNNs and neuromorphic computing.

This work provides 1) a robust formulation of RFI detection as a time-series
segmentation task that is appropriate for SNNs, 2) an investigation into
encoding visibility data with latency, rate, delta-modulation, and three
distinct variations on step-forward encodings for time-series segmentation,
and 3) results for small SNNs trained and optimized with each encoding method
on an RFI detection dataset. The findings indicate that latency encoding is
the most efficient encoding method. This work establishes a baseline level of
accuracy for SNNs trained for RFI detection without ANN conversion,
introducing RFI detection as a challenging benchmark problem that the
neuromorphic community should consider. Thus, this research work is an
essential contribution to RFI detection and demonstrates the potential of SNNs
for solving critical problems in radio astronomy.

##  II Related Work

Radio astronomy relies on spectrotemporal data processing, and while it is
known that radio astronomy data is, in principle, a good match for SNNs and
neuromorphic methods [4], there exist only a few attempts to apply SNNs
anywhere in radio astronomy [5]. Only one work applies SNNs to RFI detection
[6].

RFI detection is a ubiquitous problem for radio telescopes, and as such,
several methods exist to combat RFI contamination. In the field, detecting RFI
is called ‘flagging’ as the output of a detection scheme is an array of
boolean flags that can mask out contaminated pixels from downstream processing
tasks. Traditional methods to flag RFI build on variations of a cumulative sum
algorithm, looking for regions of increased power in a spectrogram combined
with domain and situation-specific additional filters and heuristics [2].
While operational and efficient, these methods require constant adjustment,
expert opinion, and, in the most challenging cases, resort to human labeling.

Applications of neural networks to RFI detection aim to ease the burden of
manual labor. They are primarily built around UNet-like CNNs [3, 7],
unsupervised nearest-neighbor methods [8], or auto-encoder-based anomaly
detection routines based on finding nearest neighbors in latent space [9]. See
[10] for a recent comparison of deep learning approaches for RFI detection.
The only other application of SNNs to RFI detection employs ANN2SNN conversion
on a pre-trained autoencoder to simplify the downstream RFI detection task
[6].

This article focuses on a simulated dataset of observations from the Hydrogen
Epoch of Reionization Array (HERA) in South Africa [11]. HERA’s primary goal
is to investigate the earliest periods of formation activity in the universe.
This simulated dataset helps characterize the type of RFI HERA will be subject
to and has been benchmarked against traditional flagging algorithms, ANNs [9,
10], and a converted SNN [6], therefore making this dataset an excellent
initial target to investigate applying from-scratch trained SNNs to RFI
detection.

Numerous well-known methods exist to encode continuous-valued time-series
data, including well-known rate, latency, and delta-modulation methods [12].
Encoding speech data, in particular, is greatly relevant to radio astronomy
data since methods such as Speech2Spikes [13] rely on first transforming audio
recordings into spectrograms, a step not needed for already spectrotemporal
visibility data. In this work, we adapt the step-forward encoding algorithm to
facilitate the direct readout of RFI flags by adding an additional small
temporal element to the encoding method. This allows the output of the SNNs to
be read as an RFI mask directly.

##  III Methods

Due to the lack of preexisting information on detecting RFI with SNNs, we
formulate the problem and define the encoding methods investigated. Our
principle contribution is formulating this problem so that the time axis
included in the original data is unrolled into the SNN execution, translating
what is typically considered a two-dimensional image segmentation problem into
a multi-variate time-series segmentation problem. This reformulation allows us
to interpret the output spike trains of an SNN as an RFI mask with minimal
post-processing. This section outlines our definition of RFI detection with
SNNs, the parameters of our network design, and our approach toward hyper-
parameter optimization.

###  III-A RFI Detection Formulation

In radio astronomy, specialized hardware correlates raw signals from antennae,
transforming the data from voltage information into complex-valued
‘visibilities’ $V(\upsilon,T,b)$, which vary in frequency, time, and baseline
(pair of antennae). At this stage in the pipeline, RFI detection requires
producing a boolean mask of ‘flags’ $G(\upsilon,T,b)$ that vary in the same
parameters but are binary-valued. Previous supervised approaches to RFI
detection formulate the problem as

$\mathcal{L}_{sup}=min_{\theta_{n}}\mathcal{H}(m_{\theta_{n}}(V(\upsilon,T,b)),G(\upsilon,T,b)),$
(1)

where $\theta_{n}$ are the parameters of some classifier $m$ and $\mathcal{H}$
is an entropy-based similarity measure [9]. We exploit the time-varying nature
of this information and, therefore, include additional spike encoding and
decoding steps and push the similarity measure inwards, operating on each
element of the resulting time-series.

$\mathcal{L}_{sup}=min_{\theta_{n}}(\Sigma_{t}^{T}\mathcal{H}(m_{\theta_{n}}(E(V(\upsilon,t,b))),F(G(\upsilon,t,b))))$
(2)

where $E$ is an input encoding function and $F$ is an output encoding
function. The following six subsections describe different implementations of
$E$ and $F$ and their accompanying comparison function, $\mathcal{H}$. Figure
1 contains a raster plot for each encoding method presented with an exposure
time of four where applicable, and the associated original spectrogram and
mask in Figure 1(g).

###  III-B Encoding Methods

####  III-B1 Latency

Our latency encoding scheme is relatively straightforward. For a given
exposure $E$, input pixel intensities are mapped inversely linearly from 0 to
$E$, meaning high-intensity pixels spike almost immediately and low-intensity
pixels progressively later. For output decoding, we interpret any spikes
before the last exposure step as an RFI mask and no spikes or a spike in the
final exposure as background. The following equation maps supervised masks:

$G(\upsilon,t,b))=\begin{cases}0&b=1\\\ E&otherwise\end{cases}$ (3)

The comparison function, $\mathcal{H}$, is the mean square spike time governed
by the following function:

$\mathcal{H}_{latency}=\Sigma_{e}^{E}\Sigma_{\upsilon}^{\Upsilon}(y_{\upsilon,e}-f_{\upsilon,e})^{2}.$
(4)

Figure 1(a) contains an example input raster plot.

####  III-B2 Rate

Rate encoding is well-understood and widely used when processing image data in
SNNs. For a given exposure $E$, we interpret pixel values as firing
probabilities. We map supervised masks to firing rates of 0.8 for pixels
labeled as containing RFI and 0.2 for pixels that do not. For output decoding,
we interpret outputs with a firing rate greater than 0.75 as an RFI flag.

To map rate-encoding into a time-series segmentation problem, we treat each
time-step in the original signal as a classification problem. The comparison
function, $\mathcal{H}$, is an adjusted mean square error spike count loss,
adapted from the implementation in snnTorch [14] to handle time segments with
no positive labels present, and is governed by the following equation:

$\mathcal{H}_{rate}=\Sigma_{e}^{E}(y_{\upsilon,e}-f_{\upsilon,e})^{2}.$ (5)

Figure 1(b) contains an example input raster plot.

####  III-B3 Delta-Modulation

Delta-modulation encoding encodes inputs only on relative change. Our method
utilizes the delta-modulation encoding method in snnTorch [14], using both
positive and negative polarity spikes. Output decoding is a more complex
procedure. Encoding regions of an RFI mask that can span as little as a single
pixel requires spike polarity for ‘on’ and ‘off’ spikes. However, in our
networks, neurons only emit positive spikes. Therefore, we double the
frequency width of the output layer and move all negative polarity spikes to
the lower region as positive spikes.

The comparison function, $\mathcal{H}$, is the Huber loss function, which
combines L1 and MSE loss functions, computing the squared MSE term if the
error falls below a threshold and L1 loss otherwise. We employ this loss
function to handle the sparsity of the output spike trains. The Huber loss as
acting on spike-trains is:

$\mathcal{H}_{delta}=\begin{cases}\frac{1}{2}(y-f)^{2}&|y-f|\leq\delta\\\
\delta(|y-f|-\frac{1}{2}\delta)&otherwise\end{cases}$ (6)

####  III-B4 Step-Forward

Visibility data is spectrotemporal, so we tested encoding methods based on
encoding audio data, the Step-Forward algorithm [13]. This approach takes the
step-wise difference between the signal at each time-step and a running
cumulative sum, emitting a positive or negative spike when the difference
crosses a threshold (in our case, $0.1$) upwards or downwards. Additionally, a
cumulative sum of these differences is appended to the end of the frequency
channels, doubling the width of the input. Output decoding and supervised mask
encoding, covered in Equation 3, are handled identically to latency encoding.
The comparison function is also identical to latency encoding, covered in
Equation 4. We provide three methods to adapt the original step-forward
algorithm to the time-series segmentation problem outlined below; each method
handles the required exposure parameter.

##### First

This exposure mode presents spikes to the network only on the first exposure
step for each time-step, with silence following. Figure 1(d) contains an
example raster plot for this exposure method.

##### Direct

This exposure mode presents spikes to the network at every exposure step for
each time-step. Figure 1(e) contains an example raster plot for this exposure
method.

##### Latency

This exposure mode presents spikes to the network latency encoded at exposure
time $0$ where a spike is present and exposure time $E$ otherwise. Figure 1(f)
contains an example raster plot for this exposure method.

(a) Latency encoded example.

(b) Rate encoded example.

(c) Delta encoded example.

(d) Step-Forward encoded example with ‘first’ exposure mode.

(e) Step-Forward encoded example with ‘direct’ exposure mode.

(f) Step-Forward encoded example with ‘latency’ exposure mode.

(g) Original input spectrogram and associated RFI mask.

Figure 1: Example spike raster plots for all explored encodings, original
spectrogram, and associated RFI mask.

###  III-C Training Strategy

We train a simple feedforward SNN on a dataset from the Hydrogen Epoch of
Reionization Array (HERA) simulator [11]. This dataset, described fully by
Mesarcik et al. [9], comprises 420 training spectrograms, 140 test
spectrograms, and 2.76% RFI contamination by pixel count. Each spectrogram
covers a 30-minute simulation integrating every 3.52 seconds (producing 512
time-steps) and 512 frequency channels from 105MHz to 195MHz. The RFI is
synthetic but modeled from satellite communications (localized in frequency,
covering all time), lightning (covering all frequencies, localized in time),
ground communication (localized in frequency, partially localized in time),
and impulse blips (localized in frequency and time). After encoding into
spikes, we split each spectrogram further into $32\times 32$ sub-spectrogram
patches, allowing for a smaller test model and less intensive training
requirements. We chose to train a small two-layer fully connected network
outlined in Table I to focus on the effectiveness of each encoding method
specifically.

TABLE I: Spiking Neural Network Architecture, (*) indicates value for step-forward encodings and (†) for delta-modulation. Layer | Input | Output
---|---|---
Dense 1 | 32 (64*) | 128
Dense 2 | 128 | 32 (64*†)

We trained our SNNs using surrogate-gradient backpropagation through time
(BPTT) with snnTorch [14] using ATan as the surrogate gradient, combined with
lightning111https://lightning.ai/ for data-driven parallelism when training.
During training, we reset the internal network state after processing each
sub-spectrogram, allowing temporal information to propagate through several
time-steps (in the original spectrogram). We trained all models with Adam as
the optimizer and utilized automatic learning rate scheduling with early
stopping. All code is available online [15].

###  III-D Hyper-Parameter Tuning

We employ Optuna [16] for hyper-parameter optimization, specifically the tree-
structured Parzan estimation algorithm, to simultaneously conduct multi-
variate optimization for AUROC, AUPRC, MSE, and F1-Score. Table II contains
the bounds of each variable included in the optimization for all encoding
methods. We conduct 100 trials for each encoding method.

TABLE II: Parameter ranges for attributes included in hyper-parameter searches. Beta defines the excitability of LiF neurons within the network, and exposure is a parameter for the encoding method used in all but the delta-modulation method. Attribute | Parameter Range
---|---
Batch Size | 16 - 128
Epochs | 5 - 100
Beta | 0.5 - 0.99
Exposure | 1 - 64

For each encoding method, we take the trials with the best score in any metric
into a final set and subsequently compare these trials between all encoding
techniques to determine the overall best encoding method and set of
parameters. We discuss the result of this process in Section IV as they
comprise an integral part of our investigation. Finally, we perform fifty
repeat trials for the best parameters in each encoding method to confirm their
performance.

##  IV Results

###  IV-A Encoding Comparison

We present the results of our hyper-parameter searches in Table III. No prior
investigation has been conducted into encoding radio astronomy visibility data
into SNNs for RFI detection, and as such, these results are of interest. For
each method, we present the number of trials conducted, the best candidate set
(an artifact of performing a multi-variate optimization), and the best trials
whose performance in at least one metric (accuracy, MSE, AUROC, AUPRC, or
F1-Score) is best. We ran fifty trials for the best parameters for each
encoding method and present the results in Table IV.

TABLE III: Hyper-Parameter search using Optuna multi-variate optimization. The best trial for each method is bolded, and the best SNN encoding method is underlined. A standard ANN narrowly produced the highest overall performing hyper-parameters in a single trial, producing top AUPRC and F1-Scores. Encoding Method | Accuracy | MSE | AUROC | AUPRC | F1-Score | Batch Size | Epochs | Beta | Exposure
---|---|---|---|---|---|---|---|---|---
Delta-Modulation |  |  |  |  |  |  |  |  |
| 0.983 | 0.017 | 0.846 | 0.697 | 0.676 | 48 | 80 | 0.504 | -
| 0.981 | 0.019 | 0.849 | 0.668 | 0.620 | 19 | 37 | 0.555 | -
| 0.978 | 0.022 | 0.848 | 0.655 | 0.632 | 62 | 59 | 0.564 | -
Latency |  |  |  |  |  |  |  |  |
| 0.967 | 0.07 | 0.846 | 0.674 | 0.668 | 36 | 44 | 0.727 | 6
| 0.958 | 0.04 | 0.640 | 0.296 | 0.286 | 127 | 29 | 0.720 | 2
| 0.960 | 0.230 | 0.862 | 0.650 | 0.633 | 27 | 34 | 0.842 | 13
Step-Forward-Direct |  |  |  |  |  |  |  |  |
| 0.988 | 0.008 | 0.938 | 0.789 | 0.768 | 31 | 43 | 0.920 | 50
| 0.988 | 0.001 | 0.929 | 0.740 | 0.697 | 27 | 14 | 0.630 | 17
| 0.983 | 0.000 | 0.947 | 0.674 | 0.575 | 42 | 77 | 0.836 | 9
| 0.990 | 0.001 | 0.796 | 0.694 | 0.645 | 18 | 42 | 0.980 | 16
Step-Forward-First |  |  |  |  |  |  |  |  |
| 0.990 | 0.005 | 0.892 | 0.857 | 0.844 | 79 | 98 | 0.949 | 41
| 0.989 | 0.001 | 0.978 | 0.773 | 0.747 | 22 | 38 | 0.984 | 41
| 0.995 | 0.005 | 0.914 | 0.730 | 0.686 | 19 | 60 | 0.666 | 2
| 0.982 | 0.060 | 0.939 | 0.667 | 0.566 | 30 | 14 | 0.960 | 5
Step-Forward-Latency |  |  |  |  |  |  |  |  |
| 0.966 | 0.004 | 0.900 | 0.757 | 0.731 | 54 | 83 | 0.921 | 22
| 0.966 | 0.002 | 0.0875 | 0.719 | 0.697 | 112 | 40 | 0.858 | 20
| 0.972 | 0.028 | 0.500 | 0.514 | 0.052 | 106 | 96 | 0.657 | 1
| 0.958 | 0.043 | 0.907 | 0.616 | 0.478 | 18 | 79 | 0.579 | 3
| 0.968 | 0.002 | 0.897 | 0.759 | 0.727 | 40 | 93 | 0.935 | 18
Rate |  |  |  |  |  |  |  |  |
| 0.972 | 0.08 | 0.500 | 0.514 | 0.052 | 107 | 50 | 0.599 | 1
| 0.917 | 0.083 | 0.511 | 0.070 | 0.056 | 114 | 36 | 0.781 | 4
| 0.028 | 0.972 | 0.500 | 0.514 | 0.052 | 119 | 80 | 0.732 | 32
ANN |  |  |  |  |  |  |  |  |
| 0.989 | 0.011 | 0.901 | 0.870 | 0.845 | 44 | 29 | - | -
| 0.978 | 0.022 | 0.956 | 0.782 | 0.711 | 26 | 56 | - | -
| 0.982 | 0.018 | 0.953 | 0.827 | 0.793 | 49 | 75 | - | -
TABLE IV: Encoding method performance using final hyper-parameters. Each metric is listed as mean and standard deviation. The best scores are bolded. Latency encoding offers superior performance in all metrics and with less variability than all but rate-encoded methods. ANN performance exhibits the highest overall variability. Encoding Method | Accuracy | MSE | AUROC | AUPRC | F1 | Num Trials
---|---|---|---|---|---|---
Delta | 0.963 | 0.037 | 0.037 | 0.037 | 0.840 | 0.013 | 0.630 | 0.075 | 0.571 | 0.126 | 50
Latency | 0.988 | 0.004 | 0.012 | 0.004 | 0.929 | 0.004 | 0.785 | 0.035 | 0.761 | 0.050 | 50
Step-Forward-Direct | 0.976 | 0.015 | 0.024 | 0.015 | 0.856 | 0.022 | 0.681 | 0.071 | 0.643 | 0.105 | 50
Step-Forward-First | 0.975 | 0.015 | 0.025 | 0.015 | 0.859 | 0.021 | 0.679 | 0.074 | 0.637 | 0.109 | 50
Step-Forward-Latency | 0.972 | 0.015 | 0.028 | 0.015 | 0.863 | 0.021 | 0.662 | 0.073 | 0.611 | 0.108 | 50
Rate | 0.943 | 0.009 | 0.057 | 0.009 | 0.513 | 0.003 | 0.066 | 0.006 | 0.059 | 0.002 | 50
ANN | 0.816 | 0.181 | 0.184 | 0.181 | 0.846 | 0.081 | 0.595 | 0.110 | 0.368 | 0.223 | 50

The best SNN encoding scheme based on the hyper-parameter search is the step-
forward-first method, with an accuracy of 99%, near-zero MSE of $0.005$, AUROC
of $0.89$, AUPRC of $0.857$, and F1-Score of $0.844$. The step-forward-direct
and step-forward-latency methods also perform strongly. Combining the delta-
modulated changes and sustained value helps encode visibility data, which is
similarly effective to the speech2spikes encoding method, albeit demonstrated
on a classification task. Latency and Delta-modulation encoding perform
similarly well to each other. Rate encoding is deeply non-competitive, with an
accuracy of 15% and an F1-score of $0.064$. We conducted a hyper-parameter
search and evaluation with an ANN of the identical two-layer architecture of
the SNNs. We found that the single-trial performance in the hyper-parameter
search narrowly outperformed all SNN methods by providing top AUPRC and
F1-Scores.

The repeat trials, however, tell a different story. Latency encoding
outperforms all other methods in terms of raw performance and lower
variability. We conclude that latency encoding is the overall best-performing
SNN encoding method for this task. Figure 2 contains example output RFI masks
for a latency-encoded model. The trained SNN manages to detect major RFI
features but, unsurprisingly, produces some misclassifications on individual
pixels. Moreover, ANN performance is surprisingly much more variable and
performs worse on average than all SNN methods except rate-encoding.

(a) Original spectrogram.

(b) Latency-based inference.

(c) Ground-truth RFI mask.

Figure 2: Example HERA Spectrogram, the output of a latency-encoded trained
SNN, and the original RFI mask.

##  V Discussion

The current state-of-the-art performance for any RFI detection method on this
HERA dataset achieves an AUROC of $0.983$, AUPRC of $0.940$, and F1-Score of
$0.939$ [9, 6]. The best performance of any encoding method tested achieves an
AUROC of $0.929$, AUPRC of $0.785$, and F1-score of $0.761$. Our method is
surprisingly competitive with other methods tested on this dataset, which is
promising given the simplicity and size of the SNN tested in this work.

Re-formulating the task of RFI detection from an image-domain semantic
segmentation problem to a time-series segmentation problem permits the use of
simple SNN networks as the input width of the network only includes frequency
channels, leaving the spiking neurons to handle the temporal information
instead of including two dimensions of information as is typical when treating
the problem as an image-segmentation task. Our hyper-parameter optimization
shows that the encoding methods that maximally leverage temporal information,
in this case, latency and step-forward encoding, achieve the best overall
performance under this formulation. We additionally show that rate-encoding,
typically employed when converting static image data to a spike-coded modality
or ANN2SNN conversion, performs poorly in this formulation. In combination
with the ANN model’s poor and highly variable performance, it is possible that
the problem of RFI detection, as formulated as a time-series segmentation
problem, may be particularly well suited to SNN evaluation.

###  V-A Limitations

While we endeavor to cover various encoding techniques, our investigation is
not exhaustive. Moreover, training for small segments of a spectrogram,
limited in frequency and time, may limit the accuracy possible in this
formulation of RFI detection. Ideally, such an RFI scheme would operate on a
continual streamed input of visibility information for an entire observation.
The HERA dataset tested is synthetic, and we know most methods will likely
overfit this simulated data. Therefore, performing performance testing on real
astronomical observations is desirable to gain a more realistic understanding
of our method’s performance. Finally, a supervised approach requires high-
quality supervised labels, which are difficult to find in practice [9].

###  V-B Future Direction

This work implements a method for RFI detection that achieves excellent
performance, considering the small size and simplicity of the SNNs tested.
However, a performance gap exists between the state-of-the-art performance and
the dataset tested today. An obvious extension to this work includes expanding
the network architectures. Expansions to network architectures include larger,
fully connected networks, adding recurrence relationships, and incorporating
some spectral information through convolution. Moreover, conducting network
architecture searches and evolutionary optimization may reveal interesting
patterns. Testing the performance of larger SNNs trained on patches of
spectrograms encoded in two dimensions, based on the original RFI detection
formulation in Equation 1 will also determine if there is any efficiency
benefit to our time-series formulation specifically. Expanding the datasets
tested to include real observational data is a second obvious extension. It
would also be interesting to test other learning rules, such as supervised
membrane-based backpropagation, online unsupervised bio-inspired learning
rules, or reservoir computing. Finally, mapping an SNN-based RFI detection
approach to available neuromorphic hardware would test a possible operational
improvement in energy usage or computational performance.

###  V-C Conclusion

We formulate RFI detection as a time-series segmentation problem suitable for
direct SNN inference and evaluate the accuracy of six encoding methods. We
find latency encoding the most performant on the synthetic HERA dataset tested
here. Although there is a gap to the state-of-the-art non-SNN methods for RFI
detection, the small size and simplicity of the SNN networks tested here
strongly indicate that SNNs have a future in RFI detection both as a potential
solution to RFI detection in contemporary radio telescopes and as a data-
intensive benchmark for neuromorphic computing platforms.

## References

  * [1] “Report of the Committee on the Peaceful Uses of Outer Space,” United Nations Office for Outer Space Affairs, Tech. Rep. 65, Aug. 2022. [Online]. Available: https://www.unoosa.org/oosa/oosadoc/data/documents/2022/a/a7720_0.html
  * [2] A. Offringa, “AOFlagger: RFI Software,” _Astrophysics Source Code Library_ , pp. ascl–1010, 2010.
  * [3] J. Akeret, C. Chang, A. Lucchi, and A. Refregier, “Radio frequency interference mitigation using deep convolutional neural networks,” _Astronomy and Computing_ , vol. 18, pp. 35–39, Jan. 2017. [Online]. Available: https://www.sciencedirect.com/science/article/pii/S2213133716301056
  * [4] N. Kasabov, N. M. Scott, E. Tu, S. Marks, N. Sengupta, E. Capecci, M. Othman, M. G. Doborjeh, N. Murli, R. Hartono, J. I. Espinosa-Ramos, L. Zhou, F. B. Alvi, G. Wang, D. Taylor, V. Feigin, S. Gulyaev, M. Mahmoud, Z.-G. Hou, and J. Yang, “Evolving spatio-temporal data machines based on the NeuCube neuromorphic framework: Design methodology and selected applications,” _Neural Networks_ , vol. 78, pp. 1–14, Jun. 2016. [Online]. Available: https://www.sciencedirect.com/science/article/pii/S0893608015001860
  * [5] N. M. Scott, “Evolving Spiking Neural Networks for Spatio- and Spectro- Temporal Data Analysis: Models, Implementations, Applications,” Ph.D. dissertation, Auckland University of Technology, 2015. [Online]. Available: https://hdl.handle.net/10292/10601
  * [6] N. J. Pritchard, A. Wicenec, M. Bennamoun, and R. Dodson, “RFI Detection with Spiking Neural Networks,” _Publications of the Astronomical Society of Australia_ , pp. 1–11, Apr. 2024. [Online]. Available: https://www.cambridge.org/core/journals/publications-of-the-astronomical-society-of-australia/article/rfi-detection-with-spiking-neural-networks/994EA6FDFA18B7799D7F0552264111E6
  * [7] Z. Yang, C. Yu, J. Xiao, and B. Zhang, “Deep residual detection of radio frequency interference for FAST,” _Monthly Notices of the Royal Astronomical Society_ , vol. 492, no. 1, pp. 1421–1431, Feb. 2020. [Online]. Available: https://doi.org/10.1093/mnras/stz3521
  * [8] C. J. Wolfaardt, “Machine learning approach to radio frequency interference(RFI) classification in Radio Astronomy,” Thesis, Stellenbosch : Stellenbosch University, Mar. 2016, accepted: 2016-03-09T14:22:12Z. [Online]. Available: https://scholar.sun.ac.za:443/handle/10019.1/98464
  * [9] M. Mesarcik, A.-J. Boonstra, E. Ranguelova, and R. V. van Nieuwpoort, “Learning to detect radio frequency interference in radio astronomy without seeing it,” _Monthly Notices of the Royal Astronomical Society_ , vol. 516, no. 4, pp. 5367–5378, Nov. 2022. [Online]. Available: https://doi.org/10.1093/mnras/stac2503
  * [10] C. D. Du Toit, T. L. Grobler, and D. J. Ludick, “A comparison framework for deep learning RFI detection algorithms,” _Monthly Notices of the Royal Astronomical Society_ , vol. 530, no. 1, pp. 613–629, May 2024. [Online]. Available: https://doi.org/10.1093/mnras/stae892
  * [11] M. Mesarcik, A.-J. Boonstra, R. van Nieuwpoort, and Ranguelova, “Learning to detect RFI in radio astronomy without seeing it,” Jun. 2022. [Online]. Available: https://zenodo.org/records/6724065
  * [12] Z. Yi, J. Lian, Q. Liu, H. Zhu, D. Liang, and J. Liu, “Learning Rules in Spiking Neural Networks: A Survey,” _Neurocomputing_ , Feb. 2023. [Online]. Available: https://www.sciencedirect.com/science/article/pii/S0925231223001662
  * [13] K. M. Stewart, T. Shea, N. Pacik-Nelson, E. Gallo, and A. Danielescu, “Speech2Spikes: Efficient Audio Encoding Pipeline for Real-time Neuromorphic Systems,” in _Proceedings of the 2023 Annual Neuro-Inspired Computational Elements Conference_ , ser. NICE ’23. New York, NY, USA: Association for Computing Machinery, Apr. 2023, pp. 71–78. [Online]. Available: https://dl.acm.org/doi/10.1145/3584954.3584995
  * [14] J. K. Eshraghian, M. Ward, E. Neftci, X. Wang, G. Lenz, G. Dwivedi, M. Bennamoun, D. S. Jeong, and W. D. Lu, “Training spiking neural networks using lessons from deep learning,” _Proceedings of the IEEE_ , vol. 111, no. 9, pp. 1016–1054, 2023.
  * [15] N. Pritchard, “pritchardn/SNN-RFI-SUPER: Initial Release,” Apr. 2024. [Online]. Available: https://zenodo.org/records/10929492
  * [16] T. Akiba, S. Sano, T. Yanase, T. Ohta, and M. Koyama, “Optuna: A Next-generation Hyperparameter Optimization Framework,” in _Proceedings of the 25th ACM SIGKDD International Conference on Knowledge Discovery and Data Mining_ , 2019.