[type=editor, auid=000,bioid=1, ] [1] [1] Conceptualization of this study,
Methodology, Software 1]organization=Monash University, city=Melbourne,
country=Australia

[type=editor, auid=000,bioid=1, ] [2] [2] Conceptualization of this study,
Methodology, Software 2]organization=Deakin University, city=Melbourne,
country=Australia

[type=editor, auid=000,bioid=1, ] [3] [3] Conceptualization of this study,
Methodology, Software 3]organization=RMIT University, city=Melbourne,
country=Australia

[type=editor, auid=000,bioid=1, ] [4] [1] Conceptualization of this study,
Methodology

[cor1]Corresponding author [cor2]Principal corresponding author

# A Comprehensive Study of Disaster Support Mobile Apps

Muhamad Syukron<EMAIL_ADDRESS>[  Anuradha Madugalla
<EMAIL_ADDRESS>[  Mojtaba Shahin<EMAIL_ADDRESS>[  John Grundy<EMAIL_ADDRESS>
###### Abstract

Context: Disasters are a common global occurrence with climate change leading
to increase both their frequency and intensity. To reduce the impact of these
disasters on lives and livelihoods it is important to provide accurate
warnings and information about recovery and mitigation. Today most emergency
management agencies deliver this information via mobile apps.
Objective: There are a large collection of disaster mobile apps available
across the globe. But a detailed study is not yet conducted on these apps and
their reviews to understand their key features and user feedback. In this
paper we present a comprehensive analysis to address this research gap.
Method: We conducted a detailed analysis of 45 disaster apps and 28,161
reviews on these apps. We manually analysed the features of these 45 apps and
for review analysis employed topic modelling and sentiment analysis
techniques.
Results: We identified 13 key features in these apps and categorised them in
to the 4 stages of disaster life cycle. Our analysis revealed 22 topics with
highest discussions being on apps alert functionality, app satisfaction and
use of maps. Sentiment analysis of reviews showed that while 22% of users
provided positive feedback, 9.5% were negative and 6.8% were neutral. It also
showed that signup/signin issues, network issues and app configuration issues
were the most frustrating to users. These impacted user safety as these
prevented them from accessing the app when it mattered most.
Conclusions: We provide a set of practical recommendations for future disaster
app developers. Our findings will help emergency agencies develop better
disaster apps by ensuring key features are supported in their apps, by
understanding commonly discussed user issues. This will help to improve the
disaster app eco-system and lead to more user friendly and supportive disaster
support apps in the future.

###### keywords:

mobile apps, disaster apps, crisis management, user reviews, analysis,
recommendations

##  1 Introduction

Natural disasters – such as floods, heatwaves, earthquakes, and bushfires –
have become increasingly common. Driven by climate change, these disasters are
affecting regions that previously experienced minimal impact, such as the
floods in Oman in April 2024 [1]. These impacts are anticipated to only
escalate in the future. Such disasters lead to loss of lives and livelihoods,
disrupt education, harm health and hinder economic growth.

However, many communities can reduce the impact of these disasters by
improving their resilience with better preparation via information on better
disaster management strategies before, during and after disasters.
Traditionally, such information was communicated via broadcast media such as
television, radios or via words of mouth. However, with advances in
technology, software has started taking a prominent place in providing such
information. With 85% of the today’s global population being mobile phone
users, increasingly this information is provided via mobile apps [2, 3].

Disaster apps are more and more considered to be critical support applications
due to their ability to impact life and death situations [4, 5]. Currently
most disaster apps are built and maintained by emergency agencies. These apps
tend to have a localized focus on a specific country or a state. This is due
to the inherently localised nature of most disasters and the need to
incorporate localised tacit knowledge such as evacuation routes, into these
apps. Therefore most local emergency agencies seek to develop their own
version of a disaster app version. This has led to large number of disaster
apps available in the Google Play Store and Apple Store. However, several
issues have been reported in these apps, such as complicated interfaces,
missed alerts and overwhelming information load [6, 4, 3]. In acute situations
such as disaster led emergencies, even seemingly minor issues can become
critical concerns that can lead to a decision between life and death.

Consider an example of a poorly designed disaster app. Firstly, due to bugs in
the app, it may issue delayed bush fire alerts or miss alerts completely,
which can lead to loss of lives and livelihoods. Secondly, during cascading
disasters such as floods followed by landslides, due to poor color choices for
each disaster, it may be difficult to locate impacted areas due to burst of
colors in the map interface. Thirdly, due to issues in privacy and security
the app may get hacked which can lead to issuing incorrect alerts that leads
to massive community panics. Finally, it may lack critical advice on escape
from and key support after the disaster, leading to lack of key information
for users. Therefore, it is important to develop these apps methodically and
by following good software engineering principles, ensuring key disaster
support features are built and to high quality. When developing these apps,
rather than re-inventing the wheel and each local emergency agency developing
a disaster app from scratch, they can benefit by referring to existing apps.
However, to the best of authors knowledge, a study analysing the wide range of
disaster app features and their reviews on both Google Play Store and the App
Store is currently lacking. In this study we aim to address this research gap.

In this paper, we present a detailed analysis of all disaster apps that are
available on Google Play Store and App Store and provide a list of commonly
used features across all apps. We hope this will provide emergency agencies a
pool of features to choose for their own app. We also analyse user reviews for
these apps via topic modelling techniques to identify topics that are commonly
discussed by users. We then conduct a sentiment analysis on these reviews to
understand common preferences and pain points of the users. This analysis will
help agencies to understand what aspects of the app should receive more
attention and what is important for users.

The rest of this paper is structured as follows: Section 2 describes related
work, Section 3 outlines the research design and Section 4 presents the
results of our analysis. Section 5 presents an overall discussion and
implications of our findings in Section 6, threats to validity and in Section
7, a summary with key future directions.

##  2 Related Work

###  2.1 Mobile Applications for Disaster Management

Disaster apps have a significant diversity in the type of information they
provide and their intended target groups. The type of information vary based
on type of disaster and the stage of disaster life cycle they focus on [5].
The type of disaster can be a single disaster such as floods or may present
information about multiple disasters in which case they are known as multi-
hazard apps. The stages of a disaster life cycle are four fold according to
disaster management literature. These are ’Preparation’, ’Response’,
’Recovery’ and ’Mitigation’ [7]. The disaster apps can focus on one or many of
these stages. The _preparation stage_ involves information such as capacity
building, monitoring and providing early warnings; the _response stage_
involves information for evacuation, search and rescue; the _recovery stage_
involves damage assessment and reconstruction; and lastly the _mitigation
stage_ involves information such as risk analysis and risk appraisals [8]. The
target groups can be the global community for international apps and local
communities such as state, regional communities for local apps.

Disaster apps are broadly classified into two categories: general purpose apps
and built for purpose apps. General purpose apps include three sub-categories
One-to-one, One-to-many and Many-to-many [9]. One-to-one refers to messaging
apps such as Whatsapp and Wechat which are used by a person to send personal
message to another. One-to-many refers to news apps are maintained by news
agencies such as CNN and while publishing other news, they also publish
disaster news to the public. Many-to-many include social media apps such as
Twitter and Facebook. Out of these, people tend to favour familiar platforms
such as social media which they have frequently used before the disaster
occurrence [10].

However, disaster management authorities have concerns in promoting the use of
general-purpose platforms for emergency situations, as many issues arise such
as privacy, information quantity, and content quality [9]. Built for purpose
apps are usually maintained by emergency agencies and will focus on functions
such as Alert and information dissemination: disseminating authorised
information before and during disasters; and Information collation: gathering,
filtering and analysing data to build situation awareness. These apps are
usually one-way; commonly originating from the authorities to the public.
Examples of such apps are the American Red Cross Apps and the Federal
Emergency Management Agency (FEMA) App, both of which were developed by their
respective agencies.

###  2.2 App Review Analysis

Given that app reviews include rich information, a growing number of
researchers and practitioners have investigated app reviews to extract
actionable insights for different software engineering tasks [11, 12]. App
reviews have been mainly analyzed to support requirements engineering tasks.
For example, the studies [13, 14] developed automated approaches to identify
new features requested by users in mobile app reviews. Some have extracted
non-functional requirements from app reviews (e.g., [15, 16]. Several studies
have focused on identifying issues (e.g., UI issues, crashes, bugs) that users
face when working with apps by mining user reviews (e.g., [17, 18, 19, 20].
Another line of research attempted to (automatically) detect fake reviews and
non-informative reviews (e.g., [21, 22]). Another important research theme is
the sentiment analysis of app reviews to detect users’ emotions about the app
(e.g., [23, 24]). Finally, some research has recently focused on manually or
automatically identifying and classifying human, social, and ethical factors
(e.g., human and ethical values [25, 26, 27, 28] and privacy [29]) from app
reviews.

###  2.3 Disaster Mobile App Analysis

Researchers have sought to explore disaster mobile apps in a limited number of
studies. Tan et al. [9] systematically reviewed crisis informatics literature,
including disaster apps. But these focused on apps discussed in academic
literature, as we found in our analysis; not all of them were available to the
public as some were still in the proof of concept level [9]. Corcuera et al.
[30] also looked into disaster apps. Still, their literature review was
limited to apps in the Google Play store, and they sought only to explore the
functionalities of these apps and evaluate their effectiveness for disaster
risk reduction [30]. They did not explore the user reviews that are available
in these apps for insights into user perception of features and app
performance.

Studies analyzing app reviews, described in Section 2.2, typically do not
cover disaster apps. This is mainly because these studies choose the reviews
from the most popular apps (e.g., Instagram and TikTok apps) and app
categories (e.g., health, social, communication) or randomly select reviews in
Google Play Store and App Store. The popularity of an app is usually defined
by the number of downloads, which excludes disaster apps from consideration.
This exclusion also applies to other less frequently downloaded but important
app categories, such as depression apps [28] and sports apps [31]. However, as
explained earlier, disaster apps are used in critical situations where even a
minor issue can make a difference between life and death. Therefore, a
detailed study of disaster apps available to the public today is long overdue.

Figure 1: Overview of our study approach

##  3 Research Design

In this study we aim to answer three key research questions. We provide these
questions and our rationale for choosing them below.

###  3.1 Research Questions

RQ1. What are the key characteristics of disaster apps?

Rationale. We want to systematically identify, examine, and evaluate the
features of disaster apps. This investigation involves a detailed review of
the app functionalities, the types of natural disasters they address, and
their geographic coverage. Subsequently, we analyze the apps’ popularity and
quality based on metrics such as the number of downloads and user ratings and
explore their monetization strategies. This analysis will help emergency
agencies and their app developers to understand which features should be
included in disaster apps.

RQ2. What do users discuss in disaster app reviews?

Rationale. We want to explore the perceptions and opinions of disaster app
users that are expressed through the reviews of these mobile apps. By
employing topic modeling techniques, this RQ extracts and categorizes the
primary topics discussed in user reviews. This analysis, covering 28,161
reviews across 45 mobile apps, identified 22 distinct topics.

RQ3. What are the common challenges in using disaster apps as reported by
their users?

Rationale. We want to develop deeper understanding of key grievances and
dissatisfaction among users of disaster mobile apps. This was analysed with
the ’user ratings’ provided for each app, and via a sentiment analysis
performed on the reviews.

###  3.2 Data Collection

Figure 1 summarizes our research methodology. This section details the steps
involved in identifying, gathering, and preparing the data to answer our RQs.
The specific methodologies employed to address each research question are
outlined in Section 4.

####  3.2.1 A Dataset of Disaster Mobile Apps

To answer RQ1, we needed a collection of disaster apps. Currently, there is no
available dataset of such apps. We adopted the following steps to identify
disaster apps.

Step 1: App Collection This step involved locating apps for disasters. We used
three approaches to achieve this.

Approach 1: Literature-based app collection: We first extracted a list of
mobile apps used for disaster management via the systematic review of Tan et
al., which identified 29 mobile apps [9]. We extracted 5 apps from the Apple
Store and Google Play Store. The rest of the apps discussed in this paper were
still in the research stage and, hence, were not published at the Apple Store
or Google Play Store. They thus lack user reviews, finalised features and real
end users.

Approach 2: Keywords-based app collection. Due to the limited apps that were
available via the literature-based app collection approach, we searched for
relevant apps in both Google and Apple app stores in different parts of the
world. We identified disaster apps using disaster-specific keywords [32], such
as ‘flood’, ‘earthquake’, and ‘tornado’. These keywords were used to search in
the titles and descriptions of apps in the Play Store and Apple Store. This
process led to an identification of 38 apps tailored to addressing natural
disaster situations.

Approach 3: Country-based app collection. Despite conducting searches with
disaster-specific keywords, not all potential apps were discovered. This issue
arose because some apps which contain information on natural disasters, do not
include explicitly disaster-related terms in their names, tags, or
descriptions. For instance, ‘VicEmergency’, an app developed by an Australian
state government, provides a variety of warnings, including those for natural
disasters. However, its name and tags lack any direct reference to ‘disaster’
or specific disaster types. The description of ‘VicEmergency’ merely states it
as the official app for emergency warnings and information in Victoria, with
tags limited to ‘News and Emergency’. Consequently, such apps did not appear
in search results for disaster-related queries. Additionally, apps developed
in non-English speaking countries often have names in their native languages,
making them hard to find through English keyword searches, such as the German
government’s app ‘NINA - Die Warn-App des BBK’.

To address this challenge we adopted search approach 3 that involved
conducting a google search for potential disaster support apps in each
country. We used the query “Emergency apps in [Country]” as emergency apps
often included functionalities related to natural disasters. We did not use
“Disaster apps in [Country]” as a query because this would be overly
restrictive and may not yield a complete app list. Some of the disaster apps
obtained through this method include ‘Yurekuru Call’ and ‘Safety Tips’. These
apps were found in a blog written by Robin Lewis [33], which was discovered
using the query “Emergency apps in Japan.” This approach was repeated for all
196 countries worldwide [34].

With approach 3, we found 73 disaster-related apps developed by government and
non-government entities, available on the Google Play Store and Apple App
Store. However, this was much less than what we expected for an overall of 200
countries in the world. While it was understandable that some developing
countries may not have an app due to resource limitations, we did not feel
that this justified all the missing countries. Therefore, to understand why
many countries haven’t developed specific disaster apps, we visited official
state government websites to find how they provided disaster information to
their citizens. Our research showed that each country had various methods of
disseminating disaster information, many often without relying on mobile apps.
For instance, Austria utilizes sirens
111https://www.bmi.gv.at/204_english/skkm/warning.aspx, Canada informs their
citizens via television and radio 222https://www.alertready.ca, Saudi Arabia
sends notifications directly to personal devices
333https://998.gov.sa/English/Pages/Automatic-Early-Warning.aspx, India
employs SMS 444https://sachet.ndma.gov.in, and Belgium opts for email
555https://www.be-alert.be/en/how-does-be-alert-work. These are only one of
the methods that are used by these countries as most countries prefer to
employ multiple methods for disseminating disaster information.

Step 2: Removing Duplicates. Figure 1 shows that 116 apps were obtained from
our three app collection methods. This included 5 apps from the literature
review, 38 apps from keyword-based searches, and 73 apps from country-based
searches (See Figure 1). However, there were overlaps in apps that were
collected from some methods. For instance, the ‘FEMA’ app, designed for
disaster preparedness in the USA, was identified across all three methods.
Similarly, the literature-based review and the keyword search found the
‘Disaster Alert’ app, which provides notifications for disasters in the
Pacific region. Once these duplicates were removed we had a collection of 99
apps.

Step 3: Removing False Positives. From an initial collection of 99 apps, a
filtering process was undertaken to remove false positive apps [31]. False
positives refer to apps that do not have features pertinent to natural
disasters. Such general-purpose emergency apps tended to focus on providing
services to connect with police or fire, provide assistance during medical
emergencies, traffic incidents, and other non-disaster-related emergencies.
Most of these apps were added to our data set as a result of our Approach 3 in
data collection. To assess the 99 apps accurately, we examined their
descriptions and downloaded them to directly inspect the available features.
This evaluation identified 45 apps that have natural disaster-related
functionalities as shown in Figure 1. The list of apps can be found in
Appendix Tables 10, 11and 12.

####  3.2.2 A Dataset of Disaster App Reviews

Collecting Reviews: To answer RQ2 and RQ3, we needed user reviews of the 45
disaster apps. To gather user reviews of 45 disaster apps from the Google Play
Store and App Store, we utilized an app-store-scraper
666https://github.com/cowboy-bebug/app-store-scraper and google-play-scraper
777https://github.com/JoMingyu/google-play-scraper. These tools require the
app’s ID, name, and the specific country of the store. The app’s name is
visible upon selection, whereas its ID and the relevant country code are
identifiable through its URL. We collected 37,272 user reviews from 45
disaster apps.

Translation: The collected reviews encompassed a diverse linguistic range,
particularly for apps from non-English speaking countries. This included apps
such as ‘DEC112 2.0’, which has reviews in German, ‘Zachranka’ in Czech, and
‘Yurekuru Call’ in Japanese. Of 45 apps, 9 had reviews in languages other than
English, totaling 2,793 reviews requiring translation. To address this, we
used a Zero-Shot Translation model 888https://huggingface.co/Helsinki-NLP to
translate non-English reviews into English reviews.

Exclusions: The next step was to exclude apps without user reviews and those
with brief reviews comprising fewer than four words. The reason behind this
exclusion is twofold: firstly, the absence of user reviews renders it
impossible to address RQ2 and RQ3, which rely on user reviews. Secondly, short
reviews, offering minimal commentary such as ‘good’, ‘not accurate’, and
‘works well’, provide insufficient information for meaningful analysis.
Additionally, reviews with no rating were eliminated from the dataset, as they
could not be used in the sentiment analysis stage. This filtering process
reduced our app review dataset to comprising 28,161 reviews, laying a
foundation for data analysis for RQ2 and RQ3.

##  4 Results

###  4.1 RQ1: Key features of disaster apps

####  4.1.1 Approach

In this RQ, we examine the features available within disaster support apps,
the types of natural disasters they address, and their geographic coverage.
Subsequently, we analyze the apps’ popularity and quality based on metrics
such as the number of downloads and user ratings and explore their
monetization strategies.

Analysis 1- App Features:  To identify the available app features, we employed
a three fold approach: reading the app descriptions, examining app image
previews, and downloading the app for direct inspection. Some apps offered
complete feature descriptions. Hence, we read such descriptions to identify
and extract the app features. However, if an app lacked a clear description,
we referred to the app image previews, typically a set of screenshots of the
app contents in use displayed by the developer on the app’s page in the Play
Store or App Store. If the feature images were incomplete, we downloaded the
app and examined its features directly. After collecting data on each app’s
features, we grouped them into three types of features based on four stages of
the disaster life cycle: preparation, response, recovery and mitigation [7].

Analysis 2- Disaster Type Coverage: Using the same three fold approach
described in the app feature analysis stage above, we also determined the
types of disasters the app covers. This identified two types of apps: multi-
hazard apps and single hazard apps. The single hazard app included four types
of disasters: earthquake, wind, flood and fire.

Analysis 3- Geographic Coverage: To determine the area coverage of the app, we
used the same three fold approach as before. Then, based on the area coverage,
we grouped apps into two main categories: international and local. The local
included three sub-categories: region-specific, national and territorial.

Analysis 4- Popularity and Quality Analysis: The popularity of an app can be
gauged by its number of downloads, while user ratings offer insights into its
perceived quality. These data can be obtained from the homepage of each app on
the Play Store and App Store. For example, Figure 2 shows that the ‘Hurricane
Tracker Pro’ app has been downloaded 5K+ and has a rate of 4.7.

Figure 2: App pricing scheme, number of downloads, and rating

Analysis 5- Monetization Strategies: Not all disaster support apps are made
freely available by government agencies. To gather data about how app
developers may be supported we examined the homepage of each app. The home
page indicated whether the app contained advertising or offers in-app
purchases (payments within the app to access certain features or remove
advertisements). Additionally, on the install button, there was a price if the
app is paid, and is labeled as ‘free’ if it is available without charge. Thus,
each app can be completely free, or there can be a combination of profit-
making methods. For example, as shown in Figure 2, users need to pay $2.99 to
download the ‘Hurricane Tracker Pro’ app. This app also offers a subscription
feature. Another example is the ‘Wind Map Hurricane Tracker 3D’ app in Figure
2, which is free to download but contains advertising and offers subscriptions
within the app.

####  4.1.2 Results

Analysis 1 – App Features: Each disaster support app provides one or more
features related to natural disasters. The following is an explanation of the
key disaster app features we categorized according to the stages of a disaster
event: pre-disaster, disaster, and post-disaster. To view the number of apps
with each feature type, refer to Table 1.

Table 1: App feature categorisation and number of apps Category | Feature | No. of Apps
---|---|---
Preparation | Early Warning Alerts | 10
Disaster Forecasting | 7
Prep Tips/Tutorials | 8
Response | Real Time Alerts | 30
Disaster Maps | 21
Real-Time Tracking | 6
Emergency Reporting | 1
Recovery | Disaster Updates | 26
News/Bulletins | 8
Share Experiences | 3
Safety Confirmation | 3
Mitigation | Help/Shelter Info | 7
Recommendation Actions | 2

  * (a)

(b)

(c)

Figure 3: Features of Preparation Stage

  * •

Features of Preparation Stage:  The user has not yet experienced a disaster
during this stage, so the features here are geared towards preparation.

    1. 1.

_Early Warning Alerts:_ These enable users to immediately locate a safe area
and take action to protect themselves before a disaster occurs. Apps offering
this feature are typically from Japan, especially those related to
earthquakes. A good example is from ‘Yukureku Call’ app (see Figure
LABEL:fig_ew).

    2. 2.

_Disaster Forecasting:_ This feature supports aspects of disaster and weather
forecasting, which lasts longer than an early warning alert. While early
warning alerts can be sent moments before a disaster, forecasting can be
accessed anytime, even if a disaster does not occur. This feature is widely
available in apps that monitor wind movements. For example, as seen in the
‘Seastorm Hurricane Tracker’ app (see the Figure LABEL:fig_forecast).

    3. 3.

_Preparation Tips/Tutorials:_ This includes education and training material to
help prepare for disasters. It can include tips and tutorials for self-rescue
and recovery steps. Such an example is shown as advice on earthquake response
protocols from the ‘Hazard Red Cross’ app (see Figure LABEL:fig_tips).

(a)

(b)

(c)

(d)

Figure 4: Features of Response Stage

  * •

Features of Response Stage: When a disaster occurs, several features can be
utilized.

    1. 1.

_Real Time Alerts:_ This alert appears as some form of pop-up, informing the
user of a disaster happening in the area according to their radius settings or
specified area of interest. This feature differs from an early warning alert
because notifications can arrive during a disaster, not beforehand. For
instance, if a user sets a threshold for earthquakes measuring 3 on the
Richter scale and above, they will receive a notification when such an event
occurs.Such an example is provided in ‘QuakeFeed Earthquake Tracker’ app (see
Figure LABEL:fig_at_disaster_1).

    2. 2.

_Disaster Maps:_ Such a feature aims to provide some form of geographic-based
information to users regarding a disaster occurring at a specific location or
coverage area, presented in the form of an interactive map. Such an example
can be seen in ‘Hazards Near Me NSW’ app in Figure LABEL:fig_at_disaster_2).

    3. 3.

_Real-Time Tracking:_ This feature provides live updates on ongoing disaster
events or wind conditions, as it is available for wind-related disasters such
as hurricanes, cyclones, and typhoons. Therefore, the app will display
animated representations of wind-related disasters, which typically occur in
the ocean. An example can be seen in ‘Wind Map Hurricane Tracker 3D’ app (see
Figure LABEL:fig_at_disaster_3).

    4. 4.

_Emergency Reporting:_ This feature aims to facilitate emergency calls or
reporting when a disaster strikes. This feature was only available on ‘SES
Assistance QLD’ app in our dataset (see Figure LABEL:fig_at_disaster_4).

(a)

(b)

(c)

(d)

Figure 5: Features of Recovery Stage

  * •

Features of Recovery Stage: The key features available in the post-disaster
stage.

    1. 1.

_Disaster Updates:_ Information regarding disaster events can be presented in
the form of short updates. This usually consist of a list format, such as a
list of all the earthquakes in the past month. In such a feature, users can
view disaster statistics, such as the magnitude of the disaster, the depth
from the ocean, and the distance to the nearest city. A good example is that
used in ‘Earthquake Alert!’ app (see Figure LABEL:fig_rec_disaster_1)

    2. 2.

_News/Bulletins :_ Information regarding disaster events can also be presented
as comprehensive news bulletins. These provide post-disaster information with
comprehensive analysis and reporting, such as the number of people injured,
the count of collapsed buildings, and the economic losses incurred in the
areas affected by the earthquake. News bulletins tend to be longer than short
disaster updates. One app that provides this feature is ‘Disaster Alert’ app
(see Figure LABEL:fig_rec_disaster_2).

    3. 3.

_Share Experiences:_ This feature can be utilized to notify others when we
feel a disaster. For example, in the ‘Volcanoes & Earthquakes’ app, there is
an ‘I Felt It’ button that appears when the app launches a disaster
notification (see Figure LABEL:fig_rec_disaster_3).

    4. 4.

_Safety Confirmation:_ Some apps offer safety confirmation features, such as
the ‘National Evacuation Center Guide’ app. This app allows users to confirm
their safety to friends or family through contact tracing, location sharing,
and short messages within a safety confirmation group in the app (see Figure
LABEL:fig_rec_disaster_4).

(a)

(b)

Figure 6: Features of Mitigation Stage

  * •

Features of Mitigation Stage: This stage is about taking steps to prevent the
impact of future disasters.

    1. 1.

_Help/Shelter Info:_ In preparation for the next disaster, users can search
for the nearest shelter using some apps. This will help them prepare
evacuation routes and evacuation modes in preparation for a future disaster.
Such an example if found in ‘Emergency: Severe Weather App’ (see Figure
LABEL:fig_mit_disaster_1).

    2. 2.

_Recommendation Actions:_ This feature provides actions to take based on the
post-disaster situation. Such recommendations for such actions can be found in
’NINA - Die Warn-App des BBK’ (see Figure LABEL:fig_mit_disaster_2). The
following is a recommendation made by this app for the user, which is
translated from German to English.

> “Disaster: Due to a fire in a warehouse, there is heavy smoke development
> Recommended action: Please immediately close windows and doors. Turn off
> ventilation and air conditioning systems.”

Analysis 2 – Disaster Type Coverage: Based on the types of disasters covered,
the apps can be divided into two: multi-hazard apps and single hazard apps.
The _single hazard app_ category (24 apps) includes four types of disasters:
Earthquake, Wind-Related Disaster, Flood, and Fire-Related Disaster as shown
in Table 3. _Multi-hazard apps_ (27 apps) cover different types of disasters.
For example, ‘Disaster Alert’ app covers 18 types of disasters, ranging from
earthquakes and floods to wildfires. Earthquake apps (15 apps), like
‘Earthquakes Today’, exclusively cover earthquakes, while others, such as
‘Safety Tips’, also encompass volcanic activity and tsunamis. Wind disaster
apps (8 apps) primarily address three kinds of wind events: hurricanes,
cyclones, and typhoons. The last two single types of disasters were floods (4
apps) and fires (1 app), with the latter covering bushfires, forest fires, or
other fire-related disasters (refer to Table 3 to view the number of apps in
each category). The detailed lists of these apps can be found in Appendix
Tables 10 and11 for Playstore and Table 12 for AppStore.

Table 2: Apps by Disaster Type

Type of Disaster | Number of Apps
---|---
Multi-Hazard |  22
Earthquakes |  12
Wind-Related Disaster |  8
Floods |  3
Fire-Related Disaster |  1
|

Analysis 3 – Geographic Coverage: Geographic locations supported in disaster
apps can be categorised as international and local. The local apps can have
sub-categories of region-specific, national and territorial. Intentional apps
(22 apps) can be accessed from any location and cover all countries. Different
local apps can be accessed from only those geographical areas. Region-specific
apps (6 apps), cover regions such as the USA and Canada or the USA and Europe
(see Appendix Table A Comprehensive Study of Disaster Support Mobile Apps).
Some regions may also extend to various basins or oceans, particularly for
apps focusing on wind-related disasters. National apps (20 apps) exclusively
target a single country. Lastly, territorial apps (8 apps), such as
‘VicEmergency’, exclusively cover disasters within a specific sub-national
area, such as Victoria, Australia, while ‘SD Emergency’ focuses solely on
disasters in San Diego, California, USA. The detailed lists of these apps
along with their geographic coverage can be found in Appendix Tables 10 and11
for Playstore and Table 12 for AppStore.

Table 3: Apps by Geographic Coverage

Geographic Coverage | Number of Apps
---|---
International |  19
Region-Specific |  6
National |  19
Territorial |  5
|

Analysis 4 – Popularity and Quality: On the Google Play Store, the rating of
an app is given in a numerical range from 1 to 5, and the number of downloads
is expressed within specific ranges, such as 10K+, 500K+, 1M+, and 5M+.
Meanwhile, on the App Store, the app rating follows the same scale, ranging
from 1 to 5. However, the App Store provides precise figures without rounding
to the nearest thousand or million for the number of downloads. The specific
number of downloads for each of the apps we explored can be found in Appendix
Tables 10, 11 and 12.

Table 4: Comparison of Total Downloads and App Ratings on Play Store and App Store Variable | Statistics | Play Store | App Store
---|---|---|---
Rating | Mean | 3.88 | 3.68
Median | 4.1 | 3.75
Min | 2.1 | 1.7
Max | 5 | 4.8
Downloads | Mean | 559K+ | 840
Median | 100K+ | 435
Min | 5K+ | 6
Max | 5M+ | 6,400

Of the 45 apps in this study, some of are published in both the Play Store and
App Store. The number of disaster-related apps in the Play Store is 41, while
in the App Store, there are 16. On average and median (see Table 4), the
rating value in the Play Store surpasses that of the App Store. The average
and median ratings on the Play Store are 3.88 and 4.1, respectively, whereas
on the App Store, they stand at 3.68 and 3.75. The app, ‘BD999’, with the
highest rating, is also on the Play Store, which received a perfect rating of
5. Meanwhile, the app, ‘SES Assistance QLD’, with the lowest rating, is on the
App Store, which received a rating of 1.7.

The number of downloads for all disaster-related apps in the Play Store
exceeded 22 million, while in the App Store, it is over 13,000. Statistically,
the average and median number of downloads on the Play Store are over 559,000
and 100,000, respectively, while on the App Store, they are 840 and 435.
What’s interesting here is that the app with the lowest number of downloads on
the Play Store has almost the same value as the one with the most on the App
Store. However, the significant difference in the number of downloads in these
two stores is reasonable, considering the higher percentage of Android users
compared to iOS, with Android users accounting for 70.11% and iOS users for
29.19% [35]. The app ‘NINA - Die Warn-App des BBK’ has the most downloads is
in the Play Store with more than 5 million downloads. The app ‘SES Assistance
QLD’ has the fewest downloads is in the App Store, only downloaded 6 times.

Analysis 5 – Monetization Strategies: Developers can earn money from the apps
they create through ads, in-app purchases, and/or purchases made during the
download process. More than half (24 apps) of disaster apps were free, while
the other half used a combination of pricing strategies. As shown in Table 5,
10 apps on the Play Store generate revenue by incorporating ads within their
apps, offering premium feature packages for purchase, or providing options for
ad removal, either individually or in combination. Additionally, there are 3
apps that require users to pay when downloading them. Meanwhile, the sole
method for generating revenue in the App Store is offering in-app purchase
services. The monetization strategy for each of the apps we explored can be
found in Appendix Tables 10, 11 and 12.

Table 5: Pricing Schema of Natural Disaster App Pricing Schema | Play | App
---|---|---
| Store | Store
Free | 24 | 12
Free with Ads | 5 | 0
Free with In-App Purchase | 2 | 2
Free with Ads and In-App Purchase | 8 | 0
Paid | 2 | 0
Paid with In-App Purchase | 1 | 0
|  |

###  4.2 RQ2: User Discussions in App Reviews

####  4.2.1 Approach

To understand the topics discussed by users of disaster apps, we employed
BERTopic [36], a topic modeling technique, on our dataset of 28,161 reviews.
We then performed manual cluster merging and naming, followed by cluster
refinement to remove outliers. The process of topic modeling can be seen in
Figure 7, and the details of the approach are explained below.

Figure 7: Topic Modelling Process

BERTopic: In this research, BERTopic was utilized for topic modeling. BERTopic
converts documents into vectors using the Sentence-BERT (SBERT) framework.
SBERT is utilized for document embedding because it can retain semantic
information. BERTopic provides two options for sentence embedding: all-
MiniLM-L6-v2 for English sentences and paraphrase-multilingual-MiniLM-L12-v2
for multilingual sentences [37]. The first option was chosen because we
translated all non-English reviews into English. Subsequently, the
dimensionality of the embedding vector is reduced using UMAP (Uniform Manifold
Approximation and Projection) [38].

UMAP was selected for dimension reduction in BERTopic for its superior
embedding stability and significantly faster runtime compared to other methods
such as T-SNE [39]. Once reduced, the embedding vector is clustered using
HDBSCAN (Hierarchical density-based clustering) [40]. HDBSCAN was chosen
because this model can prevent unrelated documents (outliers) from being
assigned to any cluster and is more robust in parameter selection. HDBSCAN has
parameters that need to be specified, among which is min_topic_size [37]. This
parameter ensures that each cluster has at least min_topic_size members. If
there are numerous points in close proximity but fewer than the
min_topic_size, these points will either join another cluster or become
outliers. In this study, the min_topic_size used in this clustering was 100.

Cluster Merging and Naming: After the model training is complete, BERTopic
will provide results in the form of topic IDs, topic names, keywords, and the
number of members in each topic. To ensure the accuracy of the topic/cluster
naming, a manual analysis is conducted to rename each topic. This analysis
involves examining the keywords of each topic, as well as checking 50 data
points that are closest in distance to the center of each cluster. After all
clusters are appropriately named, similar clusters are merged, for example,
clusters named “app notification” and “app alert” are considered the same
topic but differ only in terminology.

Cluster Refinement: In the topic modeling process, each identified topic is
refined by removing reviews that don’t align well with the topic’s main theme.
This involves using BERTopic to re-cluster the reviews within a topic into two
groups: outliers and non-outliers. Reviews classified as outliers are
discarded, ensuring that the remaining, relevant reviews are retained and
become the final result of the topic modeling process. If a topic contains no
outliers, the quantity of reviews remains unchanged.

####  4.2.2 Results

Our topic modeling process using BERTopic resulted in 20 topics. These topics
are sorted based on the number of reviews per topic in Table 6. Each topic has
a minimum of 50 reviews, as specified by the min_topic_size parameter in the
BERTopic model. In the following section, we will briefly discuss each topic
along with examples.

Table 6: Topic Extracted from User Reviews Topic ID |  Topic |  Keywords |  Total |  Review Length
---|---|---|---|---
1 |  Alert Functionality |  alert, notification, warning, report |  3,129 |  17
2 |  App Satisfaction |  good, great, nice, love, like |  2,194 |  10
3 |  Map Usage |  map, satellite, view, landscape |  567 |  16
4 |  Monetary Transactions and Ads |  money, cost, refund, ads, price |  542 |  20
5 |  Location Functionality |  location, GPS, enabled |  536 |  22
6 |  Disaster Updates |  know, informed, world |  499 |  22
7 |  App Crashes |  crash, wont, open, closes |  491 |  17
8 |  Alert Sound |  beep, sounds, loud, alarm, siren |  462 |  20
9 |  App Dissatisfaction |  disappointing, unacceptable, uninstall, useless |  457 |  22
10 |  Slow Performance |  heavy, slow, unresponsive, lag, sluggish |  363 |  21
11 |  Bug Fixes |  please, fix, repair, problem |  285 |  25
12 |  App Configuration |  configuration, settings, change, setup |  246 |  17
13 |  User Interface |  hard, see, font, small, interface |  227 |  18
14 |  User Safety |  safe, security, family, stay |  214 |  22
15 |  Sign Up and Sign In Problems |  register, failed, password, email |  155 |  22
16 |  Network Connectivity |  network, error, connect, server |  140 |  20
17 |  Watch Zone Functionality |  watch, zone, warnings, notified |  139 |  27
18 |  Battery Drainage |  battery, consumption, drain, energy |  136 |  19
19 |  Reliability |  reliability, inaccurate, inconsistent |  105 |  18
20 |  Language Preferences |  translation, German, English, speak |  70 |  21
|  |  |  |

####  4.2.3 Identified Topics

1) Alert Functionality: The most common topic discussed by users is the alert
functionality. This includes 3,129 reviews (see Table 6). Users mainly express
their satisfaction with the alert functionality because they are informed
quickly during disasters, as exemplified in the review “I absolutely love this
app. It’s accurate and informative. I check it daily for New Zealand
earthquake and volcanic activity, and I receive alerts/notifications. News
comes through almost instantly when earthquakes occur”

However, there are some reviews in which users were disappointed with the
alert functionality. One user complained about the late delivery of alerts “…
I didn’t receive timely warnings. Alerts can be sent between 10 minutes to an
hour late after an earthquake occurs!”

Another common complaint is some users not receiving alerts when disasters
occur. One review related to this issue is “… This week has been full of
severe thunderstorms, flood warnings, and tornado warnings, yet I have not
received any notifications or alerts from this app…”

2) App Satisfaction: The second most discussed topic is app satisfaction, with
2,194 reviews, where users indicate they are generally happy with the app (see
Table 6). The majority of reviews in this topic are brief expressions of
satisfaction. The median word count for reviews in this topic is only 10, the
lowest among all topics. An example of a short review is “Great app, easy to
use, and provides good information.” However, there are also longer reviews
where users explain their experience using the app, such as “I really
appreciated how up-to-date the app is with fire information! The other day
when I was driving on back roads in country Victoria, I saw smoke and checked
the app, and realized I was heading straight for a quickly growing new grass
fire.”

3) Map Usage: The third most-discussed topic is regarding map usage, with 567
reviews (see Table 6. Although only 21 out of 45 disaster apps had a map
feature (see Table 1), this topic has many reviews, many of them negative. The
common complaints are level of details on the map, clumping of alerts when
zooming and issues in map functionality and usability, such as “… The UI is
incredibly clunky, maps are extremely slow to load, and warning icons
disappear when moving across the map…”

However, some users were satisfied with the map feature in the app. They are
happy about aspects such as accuracy of the maps, fastness in loading, ease of
reading and usage. This is shown in the review “… The map feature works very
well, and zooming in provides great detail. Positioning the map for a
particular global view also works very well…”

4) Monetary Transactions and Ads: This topic contains customer reviews
regarding subscriptions, payments, and advertising. Some users give positive
ratings and comments because the app is free and does not have ads, such as
“Brilliant. This is actual free app… It’s the first 5-star app for me… Plus,
there are no ads!”

However, many other users complain about the disruptive ads, for example “…
with the latest update, you don’t just get banner ads,but you also get forced
full-page pop-up ads, which are incredibly obtrusive and annoying.”  Other
users also complained about dissatisfaction after purchasing the app or
upgrading to the pro version, such as “I paid for the Pro version, and there’s
nothing special about it. The app doesn’t warn you right away; it has like a
10-15 minute wait time before it pops up. Living in Alaska, this app is
worthless to me.”

5) Location Functionality: This topic contains user reviews related to
location functionality, such as satisfaction regarding the ability to add
their preferred location to be monitored. This is shown in review “I have
friends and family scattered all over the country. The app allows me to add
the places where they live and provides alerts if there are hazards such as
wildfire weather, earthquakes, or hurricanes approaching. It’s easy to
customize the alerts I want and to add new places. It’s also great that it
will monitor wherever I am.”

Several other reviews mention user dissatisfaction due to inability to add
locations, location errors, or inability to find locations. One such review is
“The app will not allow me to add a home location or turn on live monitoring,
rendering the entire app useless. My location permissions are set to ’always’,
and the app can detect my location via GPS. However, saving never works and
instead always produces an error message.”

6) Disaster Updates: This topic contains user reviews from individuals who are
pleased to stay informed about disasters occurring in various parts of the
world. This topic only contained short reviews. The median word count for user
reviews in this topic is the second lowest, with only 17 words (See Table 6).
Some examples are “Great to be able to look all over the world and see what is
happening!”, “…The information it contains offers a full view of what earth is
doing. Checking out this app is like watching the earth breath and grow or gag
and stumble…”. and “… Since I have friends all around the world, I like
knowing where the earth is quaking. At times, it is shocking to know how many
tremors and earthquakes happen everyday, but I would rather know, than not.”

7) App Crashes: This topic contains user reviews related to the app being
unusable due to crashes. Almost all reviews are negative. App crashes can
occur in different stages of using the app. One instance is when users attempt
to open the application “… lately, either it won’t open at all, or it closes
after a couple of seconds…”. It can also happen in other stages of using the
app such as “… Sometimes it crashes when I am in the settings, and I have to
restart it…”

8) Alert Sound. This topic discusses issues in the level of alert sounds.
Sometimes the issue is with the loudness of the alert sound “… but the
notification sound is loud and annoying. I turned down my notification volume
on my phone, which made other apps quieter, not what I wanted. However, within
2 days, the volume for this app was so high-pitched that it hurts your ears,
and the volume on other apps was still low.”. Some users had complaints about
the absence of sound, causing users to be unaware if a disaster occurs “A
tornado alert was issued by this app, but there are no sounds at all! A wind
chime would be more useful than this app. An audible alarm that alerts you
should be mandatory on all weather apps.”

9) App Dissatisfaction. This topic discusses user disappointment with the
application, with all reviews negative. Disappointment typically contains
issues such as lack of developer feedback, issues in functionalities like
logging in. The following review shows a combination of these issues plus a
complaint about not receiving benefits after upgrading to the pro version. “I
had trouble with the free version not staying logged in, so I bought the
subscription hoping it would stay connected so I could receive emergency
alerts. But it made no difference; the app still won’t stay logged in. It’s
useless as an Emergency Notification app. I tried emailing for assistance
twice, but each time, I received a reply stating that the mailbox was too full
for delivery. It must be full of other frustrated users who are getting no
help from the developers. VERY DISAPPOINTED!”

10) Slow Performance: This topic contains user reviews related to slow app
performance experiences. It can involve slow loading features or slowness when
using the features. Example reviews include “Worst app. If you want to view
the map and look at details, you have to wait 10 minutes. Super slow.” and “…
it’s very slow trying to move around the app, especially when trying to make a
watch list…”

11) Bug Fixes: This topic contains customer reviews regarding bug fixing,
including both request to fix bugs and appreciation about bugs that have been
resolved. Example of a request to fix bugs is “I have not received any
notifications from this app for months! I have logged out, uninstalled, and
reinstalled multiple times, but still nothing… Please fix it!”. Example of an
appreciation about a bug fix is “Thank you very much for fixing the app! It’s
the best one for looking up earthquakes and has a great interface and
filters!”

12) App Configuration: This topic contains user reviews related to app
configuration. Users may express satisfaction with the flexible configuration,
or they may encounter problems with the application configuration. Satisfied
customer review can look like this “… a fantastic tool. It offers many
settings for configurations tailored to specific needs.” Problems in
configuration can be “It won’t open settings when I click, and I can’t adjust
which warnings or emergencies I want alerts for.”

13) User Interface: This topic contains user reviews related to the app’s
interface. Many users may express satisfaction with the UI such as “Great
clean new look. Looking forward to future updates, more new features, and
improved design.”. They also highlight problematic issues in UI aspects, such
as color and size. One review particularly from a user with vision issues
stated “The color combination of black with dark blue is terrible. I would
have given it five stars if I could actually read some of the menu labels at
the bottom of the screen. My eyesight is limited to one eye, with about 65 to
70% vision out of that eye. I need good contrast to be able to see clearly.”

14) User Safety: This topic contains reviews from users who are happy because
they feel safe and helped by this application. This topic often contains
lengthy reviews as users share their experiences using this app. The median
word count per user review in this topic is 22, making it one of the third
highest among other topics (See Table 6). An example reviews are “… puts a lot
of safety ideas for different emergencies in one place. I really like the
monitoring feature, with family scattered or traveling a lot” and “Last night
we had an earthquake 40km away had this app on my phone unopened for many
months but still got a notification seconds before we felt it. It allowed us
to immediately recognise what was happening and start heading to a safe area.
Quite impressed!”

15) Sign Up and Sign In Problems: This topic contains user reviews complaining
about not being able to register or problems during the login process, with
almost all reviews negative. Examples are “Keeps saying registration failed…
Already missed a fire near me due to this new app. Please fix.” and “It won’t
even let me log into the app at all. It keeps saying my login credentials are
incorrect, but when I use the same login info on the web browser, it logs
right in. I uninstalled it and won’t be reinstalling it!!”

16) Network Connectivity: This topic contains user complaints regarding the
inability to use the app due to connectivity issues, almost all reviews
negative. For example, as shown in “Keeps telling me no internet connection
detected. Yet I can do everything else and use all other apps without any
problems.” Another example of a problem is a user failing to connect to the
app server “Many times I get an error message about not being able to connect
to the server…”. Some users requested the offline mode for the app as shown in
review “Great app, although it’s pretty limited without power or internet
connectivity. It would be better if it could be mirrored locally for offline
use.”

17) Watch Zone Functionality: This topic contains the longest reviews compared
to other topics, with the median word count being 27 (see Table 6). This topic
contains complaints about the ‘watch zone’ not being able to be set such as
“Almost useless. You cannot create watch zones. It just says ’unable to create
watch zones’ over and over…”. Besides that, this topic also contains
complaints about the watch zone not working properly as shown in review “It
gives me notifications every time there’s a thunderstorm anywhere in the
state, but when there was actually a fire in my watch zone, it didn’t come
up.”

18) Battery Drainage: This topic is quite straightforward, covering the issue
of the app consuming a lot of battery, with almost all reviews highly
negative. “… the app was using a huge amount of battery and data in the
background. Thirty-one percent battery drain was attributed to this app, along
with 337MB of data in only a few days of background activity.”

19) Reliability: This topic contains customer reviews regarding whether users
find the app reliable or not. Example positive perceptions look like e.g. “The
app is very reliable and trustworthy. I depend on it as I live in a quake-
prone area, and the information that this app provides is very important to me
and my family.” A negative perception is reflected in the review “Unreliable
for personal safety and potentially life-threatening during the recent 2020
fires for many people. I don’t know if the networks were jammed or if the app
was broken, but we had to rely on looking out a window for a true assessment
of whether there are fires near me or not.”

20) Language Preferences: This topic contains customer reviews related to
language issues. Many users wish for the app to be available in a language
they understand or are fluent, as reflected in the review “… The only
criticism I see right now is the lack of supported languages. I think an
English version should at least be provided. Other languages could be added by
volunteers, for example, through open-source contributions or via a "donate a
translation" button in the app.” Some users suggested alternatives for in app
translation : “Could you make it possible to have the English translation? Or
at least make it possible to copy the text so that I can translate it.”

####  4.2.4 Summary

Comparison of Disaster App Topics with other Apps:  From the topics generated
by our model, it is evident that there are two special features exclusive to
disaster-related applications which are not found in other types of apps.
These include Alert and Watch zone functionalities.
1) Alert Feature:  This is considered one of the most important features in
disaster-related applications because they enable users to take immediate
action to ensure their safety before or when a disaster strikes.
2) The Watch Zone Feature:  This allows users to set a specific radius for
receiving alerts, ensuring they are not overwhelmed by unnecessary
notifications and are only informed about disasters near their location.
Additionally, the watch zone can be customized to monitor specific locations,
not just those within a certain radius. This feature is particularly useful
for individuals with family or friends living in particular areas and wishing
to ensure their safety.

Distribution of Topics Across Reviews: An analysis of Table 6 reveals that the
topics of alert functionality and app satisfaction, with 3,129 and 2,194
reviews, significantly outnumber the mentions of other topics. These two
topics combined constitute 48.6% of the total reviews, nearly half of the
contributions across all topics. This substantial percentage highlights the
prominence of app satisfaction, suggesting that at least 20% of the users
express satisfaction and commend the app’s performance. In contrast, the topic
of alert functionality presents a more complex picture. It is challenging to
draw definitive conclusions about user satisfaction from this topic alone.
Complaints may arise from various issues, such as delayed alerts, failure to
receive alerts, excessive alerts, or reliability concerns. Conversely,
satisfaction could stem from timely alerts that effectively precede disasters
or alerts that deliver crucial information. Hence, we defined a RQ3 to better
understand the users’ positive or negative sentiments towards these topics.

###  4.3 RQ3: Challenges in Disaster Apps

####  4.3.1 Approach

Figure 8: Sentiment Analysis Process

Sentiment Analysis: This RQ examines which topics have the highest percentage
of negative reviews. This information can help developers identify dominant
user complaints and prioritize their resolution. Ratings serve as an indicator
to assess user satisfaction with the app. However, there is potential for
Text-Rating-Inconsistency (TRI) to arise because some users may give ratings
carelessly [41]. For example, they may give a positive rating immediately
after downloading the app without trying it. Another example is a user who
provides numerous complaints but assigns a middle rating (a rating of 3).
Therefore, relying solely on one indicator may not be robust to ensure that
the review aligns with the rating.

We use VADER (A Parsimonious Rule-based Model for Sentiment Analysis of Social
Media Text) [42] to predict sentiment in user reviews. The process of
sentiment analysis is shown in Figure 8. The outcome of VADER includes four
types of sentiment scores: negative scores, neutral scores, positive scores,
and compound scores. VADER considers that each sentence may contain both
negative and positive elements. For example, consider the review “Currently, I
like the app very much, but you should be able to change the language within
the app.” This review contains a positive element: “I like the app very much”
but also a negative element: “you should be able to change the language”.

VADER provides a combined score of these elements, known as a compound score,
in addition to the individual sentiment scores. This compound score, ranging
from -1 to 1, will be used as the result of sentiment analysis in this study.
We will classify sentiment as positive (score >= 0.05), neutral (0.05 > score
> -0.05), or negative (score <= -0.05), following similar approaches used in
prior work [31].

Distribution of review sentiment: Our analysis divides sentiment into three
categories: positive, neutral, and negative. We observe the number of reviews
for each sentiment and their corresponding percentages.

Percentage of negative reviews for each topic: In this analysis, we examine
the percentage of negative reviews for each topic based on sentiment from
VADER or rating (with a value of less than 2). Next, we identify which topics
have the highest percentage of negative reviews in terms of rating and
sentiment analysis, enabling developers to prioritize these topics for
improvement.

####  4.3.2 Results

Table 7: Sentiment Analysis by User Rating and Sentiment Score

Sentiment | Number of Reviews [%]
---|---
User Rating |
Negative |  3834 [13.7%]
Neutral |  1911 [4.4%]
Positive |  6383 [21%]
Sentiment Score |
Negative |  3834 [9.5%]
Neutral |  1233 [6.8%]
Positive |  5890 [22.8%]
|

Distribution of review sentiment: Table 7 shows the distribution of sentiment
analysis across our user review dataset. The percentage of positive sentiment
from the user ratings is 21%, and from the sentiment score is 22.8%. The
neutral percentage in the user rating is 4.4%, slightly lower than the
sentiment score, which is 6.8%. The negative percentage of users is 13.7%,
while for the sentiment score, it is 9.5%. From this, it can be seen that the
model assumes there are some negative user reviews, but they are actually in
the positive and neutral categories, accounting for around 1% and 3%,
respectively. However, in general, the distribution of sentiment is the same.

Table 8: The percentage of negative reviews and the average score is based on sentiment analysis and user ratings. Topic |  Percentage
Negative
Sentiment
Review |  Average
Converted
Sentiment
Score |  Percentage
Negative
Rating
Score |  Average
Rating
Score
---|---|---|---|---
App Dissatisfaction |  56.67% |  2.70 |  77.46% |  1.85
Sign Up and Sign In Problems |  40.00% |  2.94 |  80.65% |  1.65
Network Connectivity |  37.14% |  2.99 |  62.86% |  2.28
App Configuration |  32.11% |  3.18 |  42.28% |  2.89
Watch Zone Functionality |  31.65% |  3.23 |  53.24% |  2.53
Slow Performance |  31.13% |  3.27 |  44.90% |  2.86
User Safety |  30.37% |  3.55 |  30.84% |  3.71
Alert Sound |  30.09% |  3.33 |  29.00% |  3.47
Monetary Transactions and Ads |  29.52% |  3.40 |  45.39% |  3.01
Battery Drainage |  28.68% |  3.25 |  57.35% |  2.47
Location Functionality |  26.12% |  3.34 |  42.35% |  2.86
Language Preferences |  25.71% |  3.37 |  44.29% |  2.89
Reliability |  25.71% |  3.33 |  44.76% |  3.17
Bug Fixes |  24.91% |  3.59 |  45.61% |  2.85
App Crashes |  24.44% |  3.24 |  68.02% |  2.12
Alert Functionality |  21.96% |  3.47 |  30.17% |  3.48
App Satisfaction |  18.87% |  3.60 |  21.01% |  3.97
Map Usage |  18.52% |  3.64 |  29.63% |  3.48
User Interface |  17.62% |  3.75 |  17.18% |  3.81
Keep Updates |  5.81% |  3.86 |  4.21% |  4.66
|  |  |  |

Highest Percentage of Negative Reviews Topics:Table 8 shows the percentage of
negative reviews generated from sentiment analysis. Not surprisingly, the
topic with the highest percentage is app dissatisfaction. Additionally, it
ranks second highest based on user ratings, which is expected given the
topic’s nature involving user complaints, either regarding specific features
or more general aspects. The next topics with the highest negative reviews
based on sentiment analysis are Sign Up and Sign In Problems at 40%, Network
Connectivity at 37.14%, and App Configuration at 32.11%. Meanwhile, for user
ratings, the highest percentage of negative reviews were found in the topics
of Sign Up and Sign In Problems at 80.65%, App Crashes at 68.02%, and Network
Connectivity at 62.86%. In terms of average converted sentiment score and
average rating, these topics generally have a value below 3, indicating
dissatisfaction among users.

##  5 Recommendations

Expanding the Scope of Disaster Management Features in Mobile Applications: In
disaster management applications, the focus often lies on functionalities that
are active during or immediately after a disaster, such as real-time alerts,
disaster maps, and updates. While these are undoubtedly crucial, there is a
significant opportunity to expand the feature set to better prepare and
support users throughout the entire disaster lifecycle. For example, early
warning alerts, which are present in only 22% of the surveyed applications,
can provide critical advance notice to users, allowing them to take necessary
precautions and potentially save lives.

Moreover, the recovery and mitigation phases are equally important, yet often
underrepresented in current applications. Features like shelter information,
safety confirmations, and recovery actions are vital for helping individuals
locate assistance, confirm the safety of friends and family, and understand
the steps they can take next. Enhancing the availability of these features
could significantly improve the efficacy of disaster response and recovery
efforts, addressing gaps that currently leave users under prepared or
insufficiently supported after a disaster strikes. By integrating a more
comprehensive set of features that span the pre and post-disaster phases,
developers can create more robust applications that cater to a wider range of
user needs and scenarios.

Internationalisation: As shown in Table 3, among the various apps categorized
based on their geographic coverage—19 international apps, 6 region-specific
apps, 19 national apps, and 6 territorial apps—only the international, region-
specific, and territorial apps consistently support the English language.
Among the national apps from non-English speaking countries, such as ‘Anhaar’
from Mongolia, ‘Emergency Ready’ App from South Korea, ‘BD 999’ from
Bangladesh, ‘Alertswiss’ from Switzerland, ‘NINA - Die Warn-App des BBK’ from
Germany, and several from Japan including ‘NERV Disaster Prevention’, ‘Safety
Tips’, ‘National Evacuation Center Guide’, and ‘Yurekuru Call’, only ‘Safety
Tips’, ‘NERV Disaster Prevention’, and ‘Emergency Ready’ App offer English
language support.

This limited availability poses significant usability challenges for non-
native speakers. For example, some users have expressed frustration with the
German app ‘NINA - Die Warn-App des BBK’ for not offering an English version,
highlighting the app’s importance and the expectation for broader language
support. Similar concerns have been raised about ‘Alertswiss’, where some
users have noted the absence of support for all official languages and
limitations in using external tools for translation. Another point of
contention is the quality of translations, with some users describing them as
inadequate.

These examples underline the importance of incorporating at least one
universally understood language, like English, in disaster management apps.
This feature is essential for ensuring the usability of the app especially for
visitors and temporary residents. Providing quality translations or,
alternatively, enabling features that assist with language translation, like
text copying, can significantly enhance the functionality and user experience
of these apps.

Monetization Techniques and their Impact: Disaster apps play a crucial role in
preparing communities for emergencies and enabling timely evacuations.
However, the implementation of certain pricing schemes or the presence of
advertisements can hinder their effectiveness. In particular, displaying ads
during an ongoing disaster can significantly impact the user experience. Ads
can be distracting or cause additional stress, potentially leading to delays
in accessing crucial information, which in turn, can jeopardize user safety.
It is advisable for developers to consider disabling ads during critical
situations to ensure that users can react quickly.

Regarding app monetization, from the user reviews, it is known that while some
users are willing to endure few ads for a free service, others prefer a paid
model without getting ads at all. To address these divergent preferences and
needs, developers should consider flexible pricing models. One effective
strategy could be offering a basic free version with non-intrusive ads and an
optional subscription-based model that removes ads and offers enhanced
features. This approach allows users to choose the version that best fits
their needs and financial situation. Moreover, developers should ensure that
essential disaster-related functionalities remain accessible to all users,
regardless of the version they are using. By providing options for
customization and control, developers can create a more user-friendly app that
accommodates diverse user preferences while still generating revenue.

Addressing Topics with the Highest Percentage of Negative Reviews: From
Section 4.2.2, it is apparent that the topics with the highest percentage of
negative reviews, as determined by user ratings and sentiment analysis, are
Sign Up and Sign In Problems, Network Connectivity, App Crashes, and App
Configuration. Except for app configuration, the other three topics pose
significant issues for users, as they prevent them from even opening the app.

Sign Up and Sign In Issues: Common problems in this category include system
bugs that prevent users from registering or logging in even with correct
credentials. Developers should address these bugs promptly and also provide
seamless integration between web and mobile platforms to allow users to access
their accounts across different devices. Enhancing user convenience by
integrating sign-in options with Google, Facebook, or other social media
accounts can also improve user experience and accessibility.

Network Connectivity: Users often face challenges with internet connectivity
and server access, especially in areas with low bandwidth. Developers should
optimize their apps to handle low-bandwidth situations effectively and
consider server scalability to manage high traffic, particularly during peak
times such as disasters. Implementing features that allow the app to function
offline can be invaluable, especially since disasters can disrupt normal
communication infrastructures.

App Crashes: Crashes are a significant barrier to app usability and can result
from various factors, including poor memory management and incomplete testing.
To address memory management issues, developers should optimize their code to
efficiently use resources and regularly profile the app to identify and fix
memory leaks. For testing, implementing a comprehensive testing protocol that
includes unit, integration, and stress tests will help ensure that the app is
robust and can handle real-world operating conditions without failure.

App Configuration: While users appreciate apps that offer customizable
settings, frustrations arise when these configurations do not save correctly.
To enhance user satisfaction, developers should ensure that the app reliably
saves user settings and consider adding more flexible options to cater to
individual preferences. Adding a help feature to guide users through setup and
configuration processes can also improve the overall user experience.

Better Support Diverse App End Users and Platforms: A number of user feedback
indicated poor support for diverse users of the disaster support apps. This
included language aspects, as discussed above, as well as challenges with
colors e.g.for colorblind users, size and complexity of screens, need to run
apps on lower-spec devices e.g. more limited battery life, network
communications, older handsets, etc. Disaster apps need to be better designed
for use on a wide variety of handsets, operating system versions, low resource
capabilities, low network bandwidth and capacity, and diverse end users of the
app. They need careful testing with diverse users and platforms, including
under diaster-induced situations e.g. stressed users, network interruptions,
high loading of servers etc.

##  6 Threats to Validity

###  6.1 Internal Validity

App Filtering in Dataset: Constructing the dataset of disaster apps posed
specific challenges that could threaten the validity of our study. The process
of filtering risked excluding apps that contain disaster-related features, as
they could be wrongly identified as false positives. To mitigate these
threats, we implemented multiple verification layers by reading app
descriptions, analyzing screenshots from Play Store and App Store, and
directly downloading and evaluating the apps. Moreover, all three authors
engaged in regular discussions to refine the selection of apps, ensuring that
our dataset accurately represents relevant disaster management tools.

Cluster Naming: In our use of BERTopic for topic analysis, the algorithm
automatically generates names for each cluster, which might not accurately
reflect the core themes of the topics. To address this, we manually checked
and, if necessary, renamed the clusters based on the evaluation of keywords
and analysis of the k-closest reviews (where k=100 for this study) to their
centroids. This manual intervention introduces a degree of subjectivity, which
could affect the study’s validity. To counter this, we involved multiple
reviewers in the process; three authors participated in reviews and
discussions, and any disagreements were resolved using the negotiated
agreement method [43], widely acknowledged in qualitative research.

###  6.2 Construct Validity

Methods: In this study, our analysis relied on a single clustering technique,
BERTopic, and one method for sentiment analysis, a rule-based model called
VADER. Relying solely on these methods might limit the generalizability of our
findings, as different techniques could potentially yield varying results.

Fine-tuning the Models: The configuration and fine-tuning of BERTopic might
have also influenced the results. The hyper-parameters such as the sentence
embedding method, distance metric formula, and minimum topic size can
significantly affect what the model identifies as a topic. Similarly, the
configuration of the VADER model, particularly the sentiment thresholds set
for classifying texts as positive (score >= 0.05), neutral (0.05 > score >
-0.05), or negative (score <= -0.05), might impact the sentiment analysis.
Since VADER is rule-based, there are no hyper-parameters to tune, but the
thresholds chosen are crucial in determining the sentiment classification.

These factors suggest a need for caution in interpreting the results. Future
studies could benefit from incorporating multiple clustering and sentiment
analysis techniques to validate the findings. For sentiment analysis,
comparisons between rule-based methods like VADER and machine learning
techniques could be particularly illuminating. However, the application of
machine learning techniques would require access to well-labeled data, which
is essential for training and testing the models accurately.

###  6.3 External Validity

App Dataset: Our compilation of disaster-related apps utilized literature-
based, keyword-based, and country-based methods. Despite these comprehensive
strategies, some relevant apps may not have been included due to inaccessible
literature, incomplete keywords, or inadequate country-specific queries. These
limitations might affect the comprehensiveness and generalizability of our
findings, potentially omitting key apps used globally for disaster management
and response.

Review Scraping: Data scraping was performed using both Play Store and App
Store scrapers, employing batch scraping strategies to manage access
limitations imposed by Google and Apple. We paused for 5 seconds after every
500 reviews, but despite these precautions, some reviews may still have been
missed if access was denied due to scraping limits.

Translation:  Due to limited resources, our translation efforts are focused
solely on reviews from apps developed in non-English-speaking countries. While
we acknowledge that apps from English-speaking regions can also provide
reviews in various languages, we do not translate these in this research. This
decision may impact the depth and breadth of our understanding of user
feedback from different linguistic backgrounds.

###  6.4 Reliability

To enhance the reliability of our study and ensure reproducibility for future
researchers, we have compiled a comprehensive replication package openly
available on GitHub 999github.com/Syukrondzeko/natural-disaster-app-review-
analysis, which includes the dataset used in our analysis and the source code
of clustering and sentiment analysis models utilized. This repository is
designed to facilitate replication of our study, providing step-by-step
guidelines on how to use it, thus minimizing the risk of obtaining different
results.

##  7 Conclusion

Disaster apps represent a group of apps that are critical for human survival.
In this study we conducted a detailed analysis of 45 apps along with their
reviews. The analysis of the app features resulted in creating a list of
features allocated to four stages of disaster life cycle: preparation,
response, recovery and mitigation. This pool of features will help emergency
agencies and future app developers to get an overview of the features that can
be included in disaster apps. The topics we have identified in RQ2 and the
users reactions to these features from RQ3, will help future developers
understand common issues that exist in disaster apps and take precautionary
steps to prevent them in their apps. These RQ results also show which features
were most preferred by users and by incorporating these features the future
app developers can increase their user satisfaction. The recommendation we
have provided can be applied for any crisis management app such as SOS apps,
Cyber security apps and even health emergency apps.

##  8 Acknowledgement

Syukron, Madugalla and Grundy were supported by ARC Laureate Fellowship
FL190100035.

## References

  * [1] W. W. Attribution, Heavy precipitation hitting vulnerable communities in the uae and oman becoming an increasing threat as the climate warms – world weather attribution, https://www.worldweatherattribution.org/heavy-precipitation-hitting-vulnerable-communities-in-the-uae-and-oman-becoming-an-increasing-threat-as-the-climate-warms/, (Accessed on 07/10/2024).
  * [2] E. Topics, How many people own smartphones? (2024-2029), https://explodingtopics.com/blog/smartphone-stats, (Accessed on 03/19/2024) (2023).
  * [3] M. L. Tan, R. Prasanna, K. Stock, E. E. Doyle, G. Leonard, D. Johnston, Usability factors influencing the continuance intention of disaster apps: A mixed-methods study, International Journal of Disaster Risk Reduction 50 (2020) 101874.
  * [4] M. L. Tan, R. Prasanna, K. Stock, E. E. Doyle, G. Leonard, D. Johnston, Understanding end-users’ perspectives: Towards developing usability guidelines for disaster apps, Progress in Disaster Science 7 (2020) 100118.
  * [5] I. Dallo, M. Marti, Why should i use a multi-hazard app? assessing the public’s information needs and app feature preferences in a participatory process, International journal of disaster risk reduction 57 (2021) 102197.
  * [6] Cfa app glitch: New vic emergency app provides incorrect fire ban information - abc news, https://www.abc.net.au/news/2016-11-22/cfa-app-glitch-vic-emergency-app-provides-incorrect-fire-ban/8042572, (Accessed on 05/13/2024).
  * [7] U. F. E. M. Agency, 04 unit 4.doc, https://training.fema.gov/emiweb/downloads/is111_unit%204.pdf, (Accessed on 05/14/2024).
  * [8] K. Poser, D. Dransch, Volunteered geographic information for disaster management with application to rapid flood damage estimation, Geomatica 64 (1) (2010) 89–98.
  * [9] M. L. Tan, R. Prasanna, K. Stock, E. Hudson-Doyle, G. Leonard, D. Johnston, Mobile applications in crisis informatics literature: A systematic review, International Journal of Disaster Risk Reduction 24 (2017) 297–311.  doi:10.1016/j.ijdrr.2017.06.009.
  * [10] G. D. Haddow, K. S. Haddow, Chapter four - disaster coverage past and present, in: G. D. Haddow, K. S. Haddow (Eds.), Disaster Communications in a Changing Media World (Second Edition), second edition Edition, Butterworth-Heinemann, 2014, pp. 53–70.
  * [11] J. Dąbrowski, E. Letier, A. Perini, A. Susi, Analysing app reviews for software engineering: a systematic literature review, Empirical Software Engineering 27 (2) (2022) 43.
  * [12] N. Genc-Nayebi, A. Abran, A systematic literature review: Opinion mining studies from mobile app store user reviews, Journal of Systems and Software 125 (2017) 207–219.
  * [13] T. Johann, C. Stanik, W. Maalej, et al., Safe: A simple approach for feature extraction from app descriptions and app reviews, in: 2017 IEEE 25th international requirements engineering conference (RE), IEEE, 2017, pp. 21–30.
  * [14] A. F. de Araújo, R. M. Marcacini, Re-bert: automatic extraction of software requirements from app reviews using bert language model, in: Proceedings of the 36th annual ACM symposium on applied computing, 2021, pp. 1321–1327.
  * [15] N. Jha, A. Mahmoud, Mining non-functional requirements from app store reviews, Empirical Software Engineering 24 (2019) 3659–3695.
  * [16] M. Lu, P. Liang, Automatic classification of non-functional requirements from augmented app user reviews, in: Proceedings of the 21st International Conference on Evaluation and Assessment in Software Engineering, 2017, pp. 344–353.
  * [17] Y. Man, C. Gao, M. R. Lyu, J. Jiang, Experience report: Understanding cross-platform app issues from user reviews, in: 2016 IEEE 27th international symposium on software reliability engineering (ISSRE), IEEE, 2016, pp. 138–149.
  * [18] Q. Chen, C. Chen, S. Hassan, Z. Xing, X. Xia, A. E. Hassan, How should i improve the ui of my app? a study of user reviews of popular apps in the google play, ACM Transactions on Software Engineering and Methodology (TOSEM) 30 (3) (2021) 1–38.
  * [19] W. Maalej, H. Nabil, Bug report, feature request, or simply praise? on automatically classifying app reviews, in: 2015 IEEE 23rd international requirements engineering conference (RE), IEEE, 2015, pp. 116–125.
  * [20] S. McIlroy, N. Ali, H. Khalid, A. E. Hassan, Analyzing and automatically labelling the types of user issues that are raised in mobile app reviews, Empirical Software Engineering 21 (2016) 1067–1106.
  * [21] D. Martens, W. Maalej, Towards understanding and detecting fake reviews in app stores, Empirical Software Engineering 24 (6) (2019) 3316–3355.
  * [22] E. Noei, F. Zhang, Y. Zou, Too many user-reviews! what should app developers look at first?, IEEE Transactions on Software Engineering 47 (2) (2019) 367–378.
  * [23] D. Martens, T. Johann, On the emotion of users in app reviews, in: 2017 IEEE/ACM 2nd International workshop on emotion awareness in software engineering (SEmotion), IEEE, 2017, pp. 8–14.
  * [24] W. Luiz, F. Viegas, R. Alencar, F. Mourão, T. Salles, D. Carvalho, M. A. Gonçalves, L. Rocha, A feature-oriented sentiment rating for mobile app reviews, in: Proceedings of the 2018 world wide web conference, 2018, pp. 1909–1918.
  * [25] M. Shahin, M. Zahedi, H. Khalajzadeh, A. R. Nasab, A study of gender discussions in mobile apps, in: 2023 IEEE/ACM 20th International Conference on Mining Software Repositories (MSR), IEEE, 2023, pp. 598–610.
  * [26] H. O. Obie, H. Du, K. Madampe, M. Shahin, I. Ilekura, J. Grundy, L. Li, J. Whittle, B. Turhan, H. Khalajzadeh, Automated detection, categorisation and developers’ experience with the violations of honesty in mobile apps, Empirical Software Engineering 28 (6) (2023) 134.
  * [27] R. A. Shams, W. Hussain, G. Oliver, A. Nurwidyantoro, H. Perera, J. Whittle, Society-oriented applications development: Investigating users’ values from bangladeshi agriculture mobile applications, in: Proceedings of the ACM/IEEE 42nd International Conference on Software Engineering: Software Engineering in Society, 2020, pp. 53–62.
  * [28] D. Bowie-DaBreo, C. Sas, H. Iles-Smith, S. Sünram-Lea, User perspectives and ethical experiences of apps for depression: a qualitative analysis of user reviews, in: Proceedings of the 2022 CHI Conference on Human Factors in Computing Systems, 2022, pp. 1–24.
  * [29] P. Nema, P. Anthonysamy, N. Taft, S. T. Peddinti, Analyzing user perspectives on mobile app privacy at scale, in: Proceedings of the 44th International Conference on Software Engineering, 2022, pp. 112–124.
  * [30] L. Navarro De Corcuera, M. D. M. Barbero-Barrera, A. Campos Hidalgo, J. Recio Martínez, Assessment of the adequacy of mobile applications for disaster reduction 24 (5) 6197–6223.
  * [31] B. Chembakottu, H. Li, F. Khomh, A large-scale exploratory study of android sports apps in the google play store, Information and Software Technology 164 (2023) 107321.  doi:10.1016/j.infsof.2023.107321.
  * [32] L. Navarro de Corcuera, M. d. M. Barbero-Barrera, A. Campos Hidalgo, J. Recio Martínez, Assessment of the adequacy of mobile applications for disaster reduction, Environment, Development and Sustainability 24 (5) (2021) 6197–6223.  doi:10.1007/s10668-021-01697-2.
  * [33] R. Lewis, 8 free apps that could save your life in japan, medium (Sep 2018).
URL https://medium.com/@robintlewis/8-free-apps-that-could-save-your-life-in-
japan-8a8b6b61b955

  * [34] E. of Encyclopaedia Britannica, List of countries (November 11 2019).
URL https://www.britannica.com/topic/list-of-countries-1993160

  * [35] A. Sherif, Global market share held by mobile operating systems from 2009 to 2021, accessed: March 18, 2024 (2024).
URL https://www.statista.com/statistics/272698/global-market-share-held-by-
mobile-operating-systems-since-2009/

  * [36] M. Grootendorst, Bertopic: Neural topic modeling with a class-based tf-idf procedure (2022).  arXiv:2203.05794.
  * [37] M. P. Grootendorst, Parameter tuning - bertopic, GitHub.io (2023).
URL
https://maartengr.github.io/BERTopic/getting_started/parameter%20tuning/parametertuning.html#n_components

  * [38] L. McInnes, J. Healy, J. Melville, Umap: Uniform manifold approximation and projection for dimension reduction (2020).  arXiv:1802.03426.
  * [39] L. van der Maaten, G. Hinton, Visualizing data using t-sne, Journal of Machine Learning Research 9 (86) (2008) 2579–2605.
URL http://jmlr.org/papers/v9/vandermaaten08a.html

  * [40] L. McInnes, J. Healy, J. Melville, hdbscan: Hierarchical density based clustering, Journal of Open Source Software 2 (11) (2017) 205.  doi:10.21105/joss.00205.
  * [41] B. Fu, J. Lin, L. Li, C. Faloutsos, J. Hong, N. Sadeh, Why people hate your app: making sense of user feedback in a mobile app store, in: Proceedings of the 19th ACM SIGKDD International Conference on Knowledge Discovery and Data Mining, KDD ’13, Association for Computing Machinery, New York, NY, USA, 2013, p. 1276–1284.  doi:10.1145/2487575.2488202.
URL https://doi.org/10.1145/2487575.2488202

  * [42] C. Hutto, E. Gilbert, Vader: A parsimonious rule-based model for sentiment analysis of social media text, in: Proceedings of the International AAAI Conference on Web and Social Media, Vol. 8, 2014, pp. 216–225.
URL https://doi.org/10.1609/icwsm.v8i1.14550

  * [43] J. L. Campbell, C. Quincy, J. Osserman, O. K. Pedersen, Coding in-depth semistructured interviews: Problems of unitization and intercoder reliability and agreement, Sociological methods & research 42 (3) (2013) 294–320.

Table 9: Region-Specific Disaster Apps

App Name | Land Area | Oceanic and Basin
---|---|---
(lr)2-4 (l)5-9 |  USA |  Canada |  Europe |  Atlantic Ocean |  Indian Ocean |  Pacific Ocean |  Central Pacific Basin |  Eastern Pacific Basin
CodeRED Mobile Alert | x | x |  |  |  |  |  |
FloodAlert Waterlevel Alerts | x |  | x |  |  |  |  |
Global Storms |  |  |  | x | x | x |  |
Hurricane Tracker |  |  |  |  |  |  | x | x
SeaStorm Hurricane Tracker |  |  |  | x |  |  | x | x
Tropical Hurricane Tracker |  |  |  | x |  |  | x | x
|  |  |  |  |  |  |  |

Table 10: Overview of Disaster-Related Mobile Applications from Play Store Part 1 App Name |  Downloads |  Rating |  Pricing |  Geographic Coverage |  Disaster Type
---|---|---|---|---|---
Alert SA |  100K+ |  3.5 |  Free |  Local |  Multi-Disaster
Alert2Me - Emergency Alerts |  10K+ |  3.5 |  Free |  Global |  Multi-Disaster
Alertswiss |  500K+ |  4.1 |  Free |  National |  Multi-Disaster
Anhaar |  10K+ |  4.7 |  Free |  National |  Multi-Disaster
BD 999 |  10K+ |  5.0 |  Free |  National |  Multi-Disaster
CodeRED Mobile Alert |  500K+ |  2.8 |  Free |  Regional |  Multi-Disaster
Disaster Alert |  500K+ |  4.1 |  Free |  Global |  Multi-Disaster
Earthquake |  100K+ |  4.1 |  Free with Ads |  Global |  Earthquake
Earthquake Alert! |  1M+ |  4.6 |  Free |  Global |  Earthquake
Earthquake + Alerts Map & Info |  100K+ |  4.1 |  Free with Ads and In-App Purchase |  Global |  Earthquake
Earthquakes Today |  100K+ |  4.4 |  Free with Ads and In-App Purchase |  Global |  Earthquake
Earthquakes Tracker |  500K+ |  4.5 |  Free with Ads and In-App Purchase |  Global |  Earthquake
Emergency |  10K+ |  4.2 |  Free |  National |  Multi-Disaster
Emergency Ready App |  10K+ |  3.5 |  Free |  National |  Multi-Disaster
Emergency: Severe Weather App |  100K+ |  3.7 |  Free |  National |  Multi-Disaster
FEMA |  1M+ |  3.8 |  Free |  National |  Multi-Disaster
FloodAlert Waterlevel Alerts |  100K+ |  4.2 |  Free with In-App Purchase |  Regional |  Flood
Floods Near Me NSW |  50K+ |  2.1 |  Free |  Local |  Flood
Global Storms |  1M+ |  4.4 |  Free with Ads and In-App Purchase |  Regional |  Wind-Related Disaster
Hazards Near Me NSW |  1M+ |  2.5 |  Free |  Local |  Multi-Disaster
Hazards Red Cross |  50K+ |  3.1 |  Free |  National |  Multi-Disaster
Hurricane & Typhoon Track |  100K+ |  3.8 |  Free with Ads |  Global |  Wind-Related Disaster
Hurricane Tracker |  10K+ |  4.3 |  Paid with In-App Purchase |  Regional |  Wind-Related Disaster
My Alert |  50K+ |  3.5 |  Free |  Global |  Multi-Disaster
My Earthquake Alerts |  5M+ |  4.7 |  Free with Ads |  Global |  Earthquake
My Hurricane Tracker |  500K+ |  4.7 |  Free with Ads |  Global |  Wind-Related Disaster
My Hurricane Tracker Pro |  5K+ |  4.7 |  Paid |  Global |  Wind-Related Disaster
NINA - Die Warn-App des BBK |  5M+ |  2.6 |  Free |  National |  Multi-Disaster
NERV Disaster Prevention |  1M+ |  4.6 |  Free with In-App Purchase |  National |  Multi-Disaster
Natural Disaster Monitor |  50K+ |  3.5 |  Free with Ads |  Global |  Multi-Disaster
National evacuation center guide |  100K+ |  3.3 |  Free |  National |  Multi-Disaster
|  |  |  |  |

Table 11: Overview of Disaster-Related Mobile Applications from Play Store Part 2 App Name |  Downloads |  Rating |  Pricing |  Geographic Coverage |  Disaster Type
---|---|---|---|---|---
SD Emergency |  50K+ |  3.5 |  Free |  Local |  Multi-Disaster
SES Assistance QLD |  10K+ |  3.1 |  Free |  Local |  Flood
Safety tips |  100K+ |  2.3 |  Free |  National |  Earthquake
SeaStorm Hurricane Tracker |  10K+ |  4.1 |  Free |  Regional |  Wind-Related Disaster
Tropical Hurricane Tracker |  100K+ |  4.9 |  Free with Ads and In-App Purchase |  Regional |  Wind-Related Disaster
VicEmergency |  1M+ |  3.3 |  Free |  Local |  Multi-Disaster
Volcanoes & Earthquakes |  1M+ |  4.4 |  Free with Ads and In-App Purchase |  Global |  Earthquake
Wind Map Hurricane Tracker 3D |  1M+ |  4.3 |  Free with Ads and In-App Purchase |  Global |  Wind-Related Disaster
Yurekuru Call |  1M+ |  4.4 |  Free with Ads and In-App Purchase |  National |  Earthquake
|  |  |  |  |

Table 12: Overview of Disaster-Related Mobile Applications from App Store App Name |  Downloads |  Rating |  Pricing |  Geographic Coverage |  Disaster Type
---|---|---|---|---|---
Alert2Me - Emergency Alerts |  57 |  3.2 |  Free |  Global |  Multi-Disaster
Alert SA |  129 |  2.3 |  Free |  Local |  Multi-Disaster
Alertswiss |  522 |  3.7 |  Free |  National |  Multi-Disaster
Disaster Alert |  26 |  3.8 |  Free |  Global |  Multi-Disaster
Earthquake |  1.7K |  4.8 |  Free with In-App Purchased |  Global |  Earthquake
Earthquake + Alerts Map & Info |  790 |  4.6 |  Free with In-App Purchased |  Global |  Earthquake
Emergency Ready App |  15 |  2.8 |  Free |  National |  Multi-Disaster
GeoNet |  465 |  4.8 |  Free |  National |  Earthquake
Hazards Near Me NSW |  1.1K+ |  3.3 |  Free |  Local |  Multi-Disaster
LastQuake |  104 |  4.7 |  Free |  Global |  Earthquake
My Earthquake Alerts |  405 |  4.6 |  Free |  Global |  Earthquake
QuakeFeed Earthquake Tracker |  1.1K |  4.7 |  Free with In-App Purchase |  Global |  Earthquake
Safety tips |  13 |  2.3 |  Free |  National |  Earthquake
SES Assistance QLD |  6 |  1.7 |  Free |  National |  Flood
VIC Fires |  6.4K |  4.7 |  Paid |  National |  Fire
VicEmergency |  614 |  2.9 |  Free |  National |  Multi-Disaster
|  |  |  |  |