# AI Product Security: A Primer for Developers

Ebenezer R.H.P. Isaac  0000-0003-0830-8862 AI Technical Product Manager,
Global AI Accelerator (GAIA)EricssonChennaiIndia<EMAIL_ADDRESS>and  Jim Reno  Distinguished Engineer, GAIAEricssonSanta ClaraUSA
<EMAIL_ADDRESS>
###### Abstract.

Not too long ago, AI security used to mean the research and practice of how AI
can empower cybersecurity, that is, AI for security. Ever since Ian Goodfellow
and his team popularized adversarial attacks on machine learning, security for
AI became an important concern and also part of AI security. It is imperative
to understand the threats to machine learning products and avoid common
pitfalls in AI product development. This article is addressed to developers,
designers, managers and researchers of AI software products.

artificial neural networks, adversarial attacks, trustworthy AI, MLOps

††ccs: Computing methodologies Machine learning††ccs: Computing methodologies
Artificial intelligence††ccs: Security and privacy††ccs: Software and its
engineering Software creation and management

##  1\. Introduction

Trustworthy AI is being explored by a number of jurisdictions around the
world. One example is the Ethics Guidelines for Trustworthy AI, from the High-
Level Expert Group on AI set up by the European Commission. According to the
EC guidelines, trustworthy AI should be lawful, ethical and robust
(Commission, 2019). The security of AI models is essential to addressing many
of its requirement areas, which are becoming codified into laws and
regulations, e.g., the EU AI Act (Comission, 2021). As we continue to develop
and rely on AI, we must prioritize security and work to address the challenges
of AI safety.

The market for AI startups has exploded in recent years, with many companies
working on new and innovative applications. Expertise in security is not a
given among all those working in AI, which makes it essential to have a
dedicated focus on it to ensure safe and secure AI systems.

The other day we came across this article titled “Computer security checklist
for non-security technology professionals.” (Garrison and Posey, 2006) It is a
short checklist grouped into three main activities: (1) Perform risk analysis,
(2) Conduct vulnerability assessments, and (3) Education, procedures and
policy. Though published in 2006, these practices are still relevant today. AI
Security, in today’s world, goes a bit beyond that. From understanding the
specific threats in machine learning (ML) to the crucial role of generic
software product security, this article will take you on a journey through the
ever-evolving landscape of AI security.

##  2\. AI-Specific Threats in ML

To address the complex challenges of AI security, we need a holistic approach
that covers all the aspects of the AI system, from data collection and
preprocessing to deployment and monitoring. The taxonomy of ML product attacks
can be divided into two surfaces as shown in Fig 1.

Figure 1. Taxonomy of threats of an ML product. Figure adapted from Hyrum
Anderson (Anderson, 2021)

A flow diagram highlighting various threats of an ML system in production

AI-specific attacks appear in the ML attack surface. These attacks are also
collectively known as adversarial attacks.

  * •

Poisoning: modifying a benign training dataset

  * •

Evasion: give a malicious input to get an unexpected output

  * •

Oracle: stealing information by probing the model

  * •

Adversarial reprogramming: use the model for a task it is not intended to do

The traditional attack surface exploits vulnerabilities that can be found in
any software product, regardless of its specific purpose or application.

  * •

Unauthorized access to data or the model can affect confidentiality,
integrity, and availability of the system.

  * •

Malicious feedback can negatively influence a model’s development which may
limit its ability to perform as expected.

Let us have a closer look at these attacks with examples and possible
countermeasures.

###  2.1. Poisoning

Tampering with the training dataset by adding, deleting, changing, or
reordering its contents can lead to erroneous learning and, ultimately, a
model that generates incorrect inferences. Poisoning can violate one or more
of availability, integrity, and confidentiality/privacy (Wang et al., 2022).

In some cases, poisoning means adding malicious samples to the dataset,
affecting the learned concept to benefit the attacker. An attacker might
poison the training dataset of a malware scanner to misclassify malware as
benign code. A facial recognition system might be tricked by introducing
images of one face labeled with a different identity. This false mapping might
used for identify theft or to warrant unauthorized access, e.g., unlocking a
phone or restricted areas.

In a classification model, an attacker can inject multiple copies of seemingly
benign samples into the training dataset. If more of one class is supplied,
then the integrity of the output is affected by changing the convergence – the
point where the model is said to have learned its target concept. Injecting an
abnormally high volume might increase training computational requirements,
stall the pipeline, and reduce system availability.

Consider a linear regression model which forecasts financial information. If
the attacker can poison the training data, the subsequently corrupted model
may give him some advantage. For example, he might affect the predicted price
of a stock in order to facilitate securities fraud.

Risk of poisoning is greater in products where there is limited control on
data collection, such as crowd-sourcing or federated learning. The following
steps may help mitigate the risks.

  * •

Assess the trustworthiness of participants. Ensure that only verified members
can provide inputs to the dataset (Papadopoulos et al., 2021). Verification
can involve authorization of members contributing to the dataset and
authenticity of messages before they are used to train an ML model.

  * •

Assess load prior to training and preprocessing to ensure the pipeline won’t
stall. Include exceptions when the data exceeds the expected distribution.
E.g., depending on the use case, a subset can be preferred over the entirety
of the input data for training.

  * •

Assess class-wise distribution to detect data drifts. Outliers, particularly
if they consistently come from a small subset of participants, could indicate
an attack. Depending on the use case, this method may not distinguish natural
drift from a poisoning attack. Nevertheless, it can alert the model operator
to initiate an investigation.

  * •

When crowd-sourcing, use randomly selected subsets taken from a very large
community for the training and test datasets. This approach increases the
amount of work the attacker must do in order to have enough poisoned data
points included in the model.

It is worth noting that these methods are not a panacea and need to be
combined with other techniques like data validation, data sanitizing and model
validation to protect against poisoning.

###  2.2. Evasion

Evasion attacks manipulate input data in a way that causes a model to make
incorrect decisions. With slight, ideally unnoticeable, input modifications,
the attacker may be able to cause an output that is in her favor. The modified
sample is called an adversarial example. Vulnerabilities to evasion attacks
are common in image classification systems (usually neural networks).
Goodfellow et al. (Goodfellow et al., 2015) show an image of a panda perturbed
to make GoogLeNet misclassify it as a gibbon with 99.3% confidence. Both
images before and after perturbation look identical to the human eye. The
perturbation is done only on the sample fed to the model, without altering
model parameters/weights.

An evasion attack is a direct violation of integrity, but can also violate
confidentiality (if the model is used to authorize access) or availability
(e.g., a virus that bypasses malware checks by warping its signature). A
detailed outlook on evasion attacks has been studied in reference (Sagar et
al., 2020).

Specific risk mitigation for evasion attacks is influenced by the application
that uses the ML model, such as malware detection, phishing detection,
internet of things (IoT), smart grids, etc. Nevertheless, studying the threat
surface can help to identify possible controls to reduce this risk. Start by
answering the following questions:

  * •

Where in the pipeline can a possible perturbation occur?

  * •

What is the data source for the example used for inference and validation?

  * •

Is the pathway from the source to the inference engine hardened?

For instance, in an image classification scenario, perturbations can occur in
the image samples. In face detection, the data source is the camera. Ensure
that the data flow from camera $\rightarrow$ storage $\rightarrow$ inference
is secure. Where you have minimal control over the data source, you may apply
certain transformations on the data (Yuan et al., 2019). Defensive
distillation is a way to strengthen a learning algorithm against potential
adversarial attacks by adding more adaptability to its classification process.
It involves training one model to predict the output probabilities of another
model that was trained on a previous, standard dataset, focusing on overall
accuracy (Papernot et al., 2016). However, defensive distillation is not
robust against poisoning, so adequate protections against poisoning should
also be in place.

###  2.3. Oracle

An oracle attack in ML, or model extraction/stealing attack (Jagielski et al.,
2020), involves an attacker attempting to extract the internal parameters or
architecture of a model by querying it to infer its decision boundaries. The
goal is to recreate a copy of the model and potentially use it for malicious
purposes, such as stealing sensitive information or intellectual property.
This type of attack can happen when the attacker has access to the model’s
predictions, but not the training data or the model’s parameters. The attacker
can construct a specific set of queries to the model to infer the underlying
logic or data. The information gained from an oracle attack can also be used
to facilitate other types of attacks. Two interesting types of oracle attacks
are membership inference attacks and model inversion attacks.

In a membership inference attack (Carlini et al., 2022), an attacker aims to
determine whether a specific individual’s data was used to train a machine
learning model. The attacker makes use of the model’s predictions as an
”oracle”, sometimes correlated with publicly available data, to infer whether
the individual’s data was used in the training process. This can reveal
sensitive information, compromising the privacy of the individual. For
example, an attacker could use a model that has been trained on medical
records to infer whether a specific individual’s medical data was in the
training set. The attacker submits queries to the model with various
combinations of attributes of the individual, and observes the model’s
predictions. If the predictions match the individual’s known attributes, the
attacker can infer that the individual’s data was present. A case study of
this attack and its defenceis discussed in (McCarthy et al., 2023).

A model inversion attack (Fredrikson et al., 2015), on the other hand, is an
attack where an adversary tries to reverse engineer the training data that was
used to train a machine learning model. By doing this, the adversary can
uncover sensitive information from the training data, such as the identity of
people or the exact geographic locations that were used to train the model.

Oracle attacks exploit the availability of model. One simple defense is to
limit access to the model, similar to defending against denial of service
(DoS) attacks. The success of an oracle attack depends on the number of
queries the attacker can submit. Throttling query access can slow or prevent
an attack. Frequent model updates can also help, by changing the model before
an attacker can submit a sufficient number of queries. Note that when
performing transfer learning on a pretrained model that is already public
(such as ResNet50 trained on ImageNet (He et al., 2016)) or used by another
entity, then this vulnerability can still exist even though you curb the
availability of the model. Here the attacker can gain access to the base
pretrained model and have sufficient time and access to create a set of
curated adversarial examples, and compare them to the target system. This can
be done even if the attacker only has a few opportunities to test instances
and observe the output.

###  2.4. Adversarial Reprogramming

Untargeted adversarial attacks aim to affect the performance of a model
without a specific target output. Targeted attacks create an adversarial
perturbation to force a particular output for a given input. Adversarial
reprogramming, as demonstrated by Goodfellow and his team (Elsayed et al.,
2019), goes beyond this by considering the ability of an attacker to alter the
intended function of a machine learning model. They were able to redirect an
ImageNet object classification model to perform a separate counting and
classification task. This highlights the potential for malicious actors to
manipulate models for their own purposes ranging from the misuse or theft of
compute resources to secret message passing treating systems as spies.

Just like evasion and oracle, this attack does not require changing the model
weights. Once enough knowledge is gained from inputs and outputs of the
system, for instance through an oracle attack, an attacker can craft a program
to create an adversarial example. The model will map that example to the
output chosen by the attacker. The initial assumption for this attack is that
it requires a white-box knowledge of the model, i.e., the structure and
weights of the model are known to the attacker. However, it was proved later
that this attack can be done even with a black-box model wherein the weights
and structure of the network are unknown to the attacker (Tsai et al., 2020).

Protection of the model’s availability and parameters can help in controlling
the vulnerability of an adversarial reprogramming attack. The use of
adversarial training methods like defensive distillation do not eliminate its
vulnerability to reprogramming (although it might increase the compute
required). Other ways to reduce the risk of such an attack are regularly
updating the model to ensure that it remains resilient against new types of
attacks, and monitoring the model’s performance in real-world scenarios to
detect any unusual behavior.

###  2.5. Traditional Attacks

Data flows in the pipeline through multiple components from ingestion to
monitoring. An attacker can target the traffic flow to access confidential
information or even participate in a man-in-the-middle attack. A model also
undergoes various states as it passes through the pipeline. Once trained, it
holds the essence of the data in the form of weights. In production, the model
resides in a model store, and is served during inference. The model can be
attacked at any point in these state transitions. Some pipelines have feedback
mechanisms to support model retraining. Of course, if feedback from many
sources is used, then a few malicious values need not inhibit the training.
However, if the feedback flow itself is compromised, then an attacker can
inject/malform any data.

Some techniques under research for addressing the traditional attack surface
of an AI system include secure generation of the training set, training set
obfuscation, and securely acquiring the sample at inference (Khalid et al.,
2018). By focusing on the security of the overall product, not just the
machine learning component, organizations can protect their assets and
stakeholders. Hardening the ML pipeline can defend against traditional attacks
and possibly some AI-specific attacks. It also helps to maintain the overall
integrity and confidentiality of data, and to comply with various regulations
and industry standards. Security controls include (but are not limited to) the
following:

####  2.5.1. Constraining listening services and bindings

Binding a service connects it to a specific endpoint or location, such as a
network address or port. On a default setup of an end-to-end system, one Often
finds many accessible services. Even if these services require secure
credentials, it is a good practice to disable access to those that are not
required outside the product boundary. By restricting access to the services
and bindings, it becomes more difficult for an attacker to exploit
vulnerabilities within the system. Constraining listening services and
bindings can also help to improve performance and reduce the overall
complexity of the AI system.

####  2.5.2. Impose strict access control mechanisms

All software should have adequate authentication and authorization. The
principle of least privilege (PoLP) is a policy ensuring users are only
granted enough access to perform their tasks and no more (Steiner et al.,
2018). It ensures that only authorized individuals or entities can access or
change the system, reducing the risk of unauthorized access, data breaches and
other malicious activities. This is especially important for AI systems where
seemingly innocuous privileges (e.g., API query access) can enable certain
attacks (e.g., oracle or inference attacks).

####  2.5.3. Traffic and data protection

Traffic protection means measures implemented to secure incoming and outgoing
communication, usually at the IP level. Not all flows passing between product
components are equal. For instance, there should be separate flows for each
plane of traffic (Jones, 2004). These planes may include Operation and
Maintenance (O&M), data plane, control plane, and user plane traffic. The
traffic should also be secured by an up-to-date version of transport layer
security (TLS) 111As of the time of writing, 1.3 is the preferred version of
TLS (Rescorla, 2018). Data at rest should also be encrypted since a data leak
can expose sensitive information or help an attacker subvert or invert the
model. Zero-trust principles require that communication channels not only be
encrypted, but also authenticated in both directions. Data protection includes
the data pipeline all the way back to the source.

####  2.5.4. Periodic vulnerability analysis

Vulnerability assessment is the process of identifying, quantifying, and
prioritizing the vulnerabilities in a system. This can be done manually or
with automated tools such as grype (Anchore, 2023) and trivy (Security, 2023).
One key step of a vulnerability assessment is identifying Common
Vulnerabilities and Exposures (CVEs) (Corporation, 2023a) – a standardized way
of describing a vulnerability or security weakness. Each has a unique
identifier and includes information such as a brief description of the
vulnerability, the affected software or system, and its severity.
Vulnerabilities are found frequently, with several new ones identified every
day. Fixing them might seem like a never-ending process, but it is important
to prioritize on the most critical ones and address them before the product
hits the market, and then do periodic fixes. For development teams under tight
deadlines this means a careful tradeoff between time, cost and risk.

Full knowledge of vulnerability management in software product security
requires a course of its own. Fortunately there are resources to help.The
MITRE ATT&CK (Corporation, 2023b) provides a comprehensive understanding of
the tactics and techniques used by attackers. The NIST Cybersecurity Framework
(of Standards and Technology, 2023b) provides a structured approach to
managing their cybersecurity risks. The recently announced NIST AI Risk
Management Framework Playbook (of Standards and Technology, 2023a) builds on
the NIST Cybersecurity Framework and provides organizations with specific
guidance on how to manage the unique risks posed by AI systems. OWASP (Open
Web Application Security Project) keeps an up-to-date summary of the top 10
vulnerabilities that pertain to web applications (Project, 2021). Counterfit
(Microsoft, 2023) is another open-source tool released by Microsoft for
automating security testing of AI systems. It helps organizations perform AI
security risk assessments to ensure their algorithms are robust, reliable, and
trustworthy. By using these tools, organizations can develop a comprehensive
cybersecurity strategy that accounts for the complex threat landscape and
helps them effectively manage their risk.

####  2.5.5. Secure coding practices

Secure coding is a method of developing software to prevent the accidental
introduction of security vulnerabilities. It includes security code reviews,
security education, and use of automatic code analysis tools. One secure
coding framework is the Software Engineering Institute (SEI) Computer
Emergency Response Team (CERT) Coding Standard (Institute, 2016). CERT has
standards sets for C, C++, Android, Oracle Java, Perl. As of now, CERT does
not have a standard set for Python. Bandit (Authority, 2022) is a Python
linting tool that helps developers identify and prevent security-related code
issues in Python applications. Bandit checks the code against a set of
predefined security rules and generates a report indicating the potential
security issues.

####  2.5.6. Event logging

Logging provides a record of events and activities within the system. It can
provide evidence in the event of a security incident by showing what actions
were taken and when they occurred. In addition to security assessments, logs
can be used for a variety of purposes, including debugging, troubleshooting,
auditing, and monitoring. The system must be auditable: Ideally, every user-
initiated action should be traceable back to the user that initiated the
action. Logs that show a sequence of activities that lead to the source of the
action are called audit trails.

In AI systems, logs can include the queries made to the model (including the
parameters), the model version, and the inference data feed. Logging involves
tradeoffs. When every possible event is logged, the system is said to be more
secure, but more compute and storage is required. In ML, depending on the
querying frequency and the inference data size, the storage requirement for
logging may grow exponentially with respect to the number of interoperable
components. So, one should be mindful of the items logged and how long logs
should be kept. One possibility is to make such parameters configurable with
adequate documentation (such as a Security/Privacy User Guide). The user of
the AI product is then aware and can make an informed decision on what to be
logged. Also keep sensitive data out of logs, and control log access through
PoLP.

####  2.5.7. Ensure runtime environment security

Even if the model is developed with security principles in mind, if deployed
in an insecure environment, the model and its inputs and outputs can be
attacked. This facilitates targeting the system with the entire spectrum of
adversarial attacks. Hence, employ measures that harden the runtime
environment. The security measures mentioned in this traditional attack
section are equally applicable to the runtime environment. Other measures
include encrypting storage, monitoring interfaces, and periodic security patch
updates.

####  2.5.8. Secure the supply chain and development pipeline

Much of today’s AI code is open source, making it easy for vulnerabilities to
be identified and exploited. Patches come out periodically. Fixing those
vulnerabilities is a continuous process. Securing the software development
supply chain involves implementing measures to ensure the integrity and
security of the components and services used throughout the software
development life cycle. Supply chain security includes:

  * •

Verifying the authenticity of the components and services used, such as open-
source libraries, by checking their signatures and digital certificates.

  * •

Using a private repository or registry to store and manage internal
components, reducing the risk of using untrusted or malicious components.

  * •

Regularly monitoring and updating dependencies, including open-source
libraries.

  * •

Implementing security practices such as code signing, secure packaging, and
continuous integration and deployment (CI/CD) to prevent tampering and
maintain the integrity of the software. The CI/CD system itself should be
secured appropriately as a production system.

  * •

Training employees to recognize and avoid potential supply chain threats, and
implement security policies and procedures to reduce the risk of a successful
attack.

More detail on best practices for software supply chain security is provided
by the Cloud Native Computing Foundation (CNCF) (Group, 2021)

##  3\. Security Pitfalls in AI Product Development

Security is an important aspect of AI products, yet it is often overlooked
during development. Many AI developers focus primarily on building the
technology and achieving high performance, without considering the potential
security risks and vulnerabilities. This lack of security focus can result in
AI products that are vulnerable and may not protect the data and information
they process.

###  3.1. Lack of security expertise in your team

Security is likely to be the last thing on the mind of a Data Scientist,
Machine Learning Engineer, or AI Product Developer. Unless there is a
dedicated security expert in your team, you are bound to overlook some
security or privacy flaw that may creep into development. You cannot expect
the entire team to undergo the rigorous security training that is expected of
a security professional. Nevertheless, a minimal security training program
should exist for all members, and a security expert made available to help
with the details. For instance, a Data Scientist may not be expected to know
the current TLS standards to protect the data in transit between one component
to another.

###  3.2. Missing security requirements at the start of a project engagement

When security requirements come late in the project, it can create
complications for development and program management: Who takes ownership of
the activity? Who has the appropriate expertise? What overhead is added to the
project? Even if the project itself has little to do with security, an early
security discussion will avoid unpleasant surprises.

###  3.3. Assuming security aspects will be taken care of by another team.

Within a dedicated product development unit, the roles are often well-defined.
There might be a dedicated team to handle security-related issues. Program
management expects a representative from this team to participate in
requirements discussions to close security gaps. However, when working with
cross-functional teams, responsibilities aren’t always clear. It is vital in
such situations to set clear expectations as to who handles the security
function rather than assuming ”it will be taken care of by another team”. It
is best if every team member has knowledge and involvement in security, but at
a minimum, those responsible should be clearly understood and documented.

###  3.4. Delaying security and privacy compliance activity towards the end

Envision this scenario: your team has put in all their effort to polish a
product based on their priority functional requirements. The product then goes
through a security assessment. The report uncovers some critical
vulnerabilities, and component integration has introduced some traffic
threats. Your models aren’t encrypted and there are some packages that you
should not even use in production. In the worst case, this can lead to an
overhaul of the entire product design. Solution: shift-left security. This
practice involves bringing the best practices of security earlier in the
product design deveopment process - ideally from the very beginning - to avoid
such issues.

###  3.5. Using features without evaluating their sensitivity.

Let’s say you showcase a successful proof of concept (PoC), everyone gets
excited, and the sales team wants to show it to the customers as soon as
possible. In the assessment, you notice sensitive fields that shouldn;t be in
the model, such as gender, region, and age. Once you remove these fields and
retrain, model performance takes a hit – far below the promised value. The
product ultimately fails to meet its requirements. This might have been
averted by conducting a privacy impact assessment for all fields before they
are considered features (or used to derive features) for fitting the model.

This pitfall does not necessarily have to be ML model-related. In telecom,
subscriber data is highly sensitive. Examples include phone number (MSISDN)
and IMSI. When these items are not specifically needed by the model, they
should not be collected, or should be masked or removed from the training
data. If they are required, consider using privacy-preserving mechanisms such
as de-identification or anonymization. For instance, not all network operators
need to access this data when using a dashboard for anomaly detection of cell
traffic.

##  4\. Closing thought

Security of any kind is not just a checklist – it is a process. AI security is
no different in this regard. It is crucial that security be integrated into
the development process from the start and not as an afterthought. Work on the
following to help avoid the aforementioned pitfalls.

  * •

Understand the implications of AI security and address those gaps in a ML
pipeline

  * •

A minimal security training programme for non-security professionals

  * •

Discuss security requirements and responsibilities as early as possible;
preferably before the start of the project engagement

  * •

Shift-left security and security by design

  * •

Assess the privacy impact of your data before committing it to a model

  * •

Be wary of AI-specific attacks like poisoning, evasion and model/data access

  * •

Periodic security audits and testing against latest security vulnerabilities.

The goal of AI is not just to prove a possibility but to build a something
that provides value to us and our customers. That means that the ’mundane’
aspects are just as important as the data science.

###### Acknowledgements.

To Amit Sharma, Head of AI Hub India 2 at GAIA, Kalpana Angamuthu, Senior
Manager of Data Science at GAIA, and Michael Liljenstam, Principal Researcher
at Ericsson Research, for reviewing this work and sharing their valuable
suggestions.

## References

  * (1)
  * Anchore (2023) Anchore. 2023.  Grype.  Retrieved Jan 27, 2023 from https://github.com/anchore/grype
  * Anderson (2021) Hyrum Anderson. 2021\.  The Practical Divide between Adversarial $\\{$ML$\\}$ Research and Security Practice: A Red Team Perspective.  Retrieved Jan 24, 2023 from https://www.usenix.org/conference/enigma2021/presentation/anderson
  * Authority (2022) Python Code Quality Authority. 2022\.  Bandit – A security linter from PyCQA.  Retrieved Jan 30, 2023 from https://github.com/PyCQA/bandit
  * Carlini et al. (2022) Nicholas Carlini, Steve Chien, Milad Nasr, Shuang Song, Andreas Terzis, and Florian Tramer. 2022\.  Membership inference attacks from first principles. In _2022 IEEE Symposium on Security and Privacy (SP)_. IEEE, 1897–1914.
  * Comission (2021) European Comission. 2021\.  EU Artificial Intelligence Act.  Retrieved Feb 28, 2023 from https://artificialintelligenceact.eu
  * Commission (2019) European Commission. 2019\.  Ethics guidelines for trustworthy AI.  Retrieved Jan 24, 2023 from https://ec.europa.eu/futurium/en/ai-alliance-consultation.1.html
  * Corporation (2023a) The MITRE Corporation. 2023a.  CVE.  Retrieved Jan 27, 2023 from https://www.cve.org/
  * Corporation (2023b) The MITRE Corporation. 2023b.  MITRE ATT&CK.  Retrieved Jan 27, 2023 from https://attack.mitre.org/
  * Elsayed et al. (2019) Gamaleldin F Elsayed, Ian Goodfellow, and Jascha Sohl-Dickstein. 2019\.  Adversarial reprogramming of neural networks. In _International Conference on Learning Representations (ICLR)_.
  * Fredrikson et al. (2015) Matt Fredrikson, Somesh Jha, and Thomas Ristenpart. 2015\.  Model inversion attacks that exploit confidence information and basic countermeasures. In _Proceedings of the 22nd ACM SIGSAC conference on computer and communications security_. 1322–1333.
  * Garrison and Posey (2006) Chlotia P Garrison and Roderick B Posey. 2006.  Computer security checklist for non-security technology professionals.  _Journal of International Technology and Information Management_ 15, 3 (2006), 7.
  * Goodfellow et al. (2015) Ian J Goodfellow, Jonathon Shlens, and Christian Szegedy. 2015\.  Explaining and harnessing adversarial examples. In _International Conference on Learning Representations (ICLR)_.
  * Group (2021) CNCF Security Technical Advisory Group. 2021\.  Software Supply Chain Best Practices.
  * He et al. (2016) Kaiming He, Xiangyu Zhang, Shaoqing Ren, and Jian Sun. 2016\.  Deep residual learning for image recognition. In _Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition (CVPR)_. 770–778.
  * Institute (2016) Software Engineering Institute. 2016\.  SEI CERT Coding Standards.  Retrieved Jan 30, 2023 from https://wiki.sei.cmu.edu/confluence/display/seccode/SEI+CERT+Coding+Standards
  * Jagielski et al. (2020) Matthew Jagielski, Nicholas Carlini, David Berthelot, Alex Kurakin, and Nicolas Papernot. 2020\.  High accuracy and high fidelity extraction of neural networks. In _Proceedings of the 29th USENIX Conference on Security Symposium_. 1345–1362.
  * Jones (2004) G. Jones. 2004.  _Operational Security Requirements for Large Internet Service Provider (ISP) IP Network Infrastructure_.  RFC 3871. IETF.  https://www.rfc-editor.org/info/rfc3871
  * Khalid et al. (2018) Faiq Khalid, Muhammad Abdullah Hanif, Semeen Rehman, and Muhammad Shafique. 2018.  Security for machine learning-based systems: Attacks and challenges during training and inference. In _2018 International Conference on Frontiers of Information Technology (FIT)_. IEEE, 327–332.
  * McCarthy et al. (2023) Andrew McCarthy, Essam Ghadafi, Panagiotis Andriotis, and Phil Legg. 2023.  Defending against adversarial machine learning attacks using hierarchical learning: A case study on network traffic attack classification.  _Journal of Information Security and Applications_ 72 (2023), 103398\.
  * Microsoft (2023) Microsoft. 2023\.  Counterfit.  Retrieved Jan 30, 2023 from https://github.com/Azure/counterfit/
  * of Standards and Technology (2023a) National Institute of Standards and Technology. 2023a.  NIST AI Risk Management Framework Playbook.  Retrieved Jan 30, 2023 from https://pages.nist.gov/AIRMF/
  * of Standards and Technology (2023b) National Institute of Standards and Technology. 2023b.  NIST Cybersecurity Framework.  Retrieved Jan 30, 2023 from https://www.nist.gov/cyberframework
  * Papadopoulos et al. (2021) Pavlos Papadopoulos, Will Abramson, Adam J Hall, Nikolaos Pitropakis, and William J Buchanan. 2021\.  Privacy and trust redefined in federated machine learning.  _Machine Learning and Knowledge Extraction_ 3, 2 (2021), 333–356.
  * Papernot et al. (2016) Nicolas Papernot, Patrick McDaniel, Xi Wu, Somesh Jha, and Ananthram Swami. 2016.  Distillation as a defense to adversarial perturbations against deep neural networks. In _2016 IEEE symposium on security and privacy (SP)_. IEEE, 582–597.
  * Project (2021) Open Web Application Security Project. 2021\.  OWASP Top 10.  Retrieved Jan 24, 2023 from https://owasp.org/www-project-top-ten/
  * Rescorla (2018) E. Rescorla. 2018\.  _The Transport Layer Security (TLS) Protocol Version 1.3_.  RFC 8446. RFC Editor.
  * Sagar et al. (2020) Ramani Sagar, Rutvij Jhaveri, and Carlos Borrego. 2020\.  Applications in security and evasions in machine learning: a survey.  _Electronics_ 9, 1 (2020), 97.
  * Security (2023) Alpha Security. 2023\.  Trivy.  Retrieved Jan 27, 2023 from https://aquasecurity.github.io/trivy/
  * Steiner et al. (2018) Stuart Steiner, Daniel Conte de Leon, and Ananth A Jillepalli. 2018.  Hardening web applications using a least privilege DBMS access model. In _Proceedings of the Fifth Cybersecurity Symposium_. 1–6.
  * Tsai et al. (2020) Yun-Yun Tsai, Pin-Yu Chen, and Tsung-Yi Ho. 2020.  Transfer learning without knowing: Reprogramming black-box machine learning models with scarce data and limited resources. In _International Conference on Machine Learning_. PMLR, 9614–9624.
  * Wang et al. (2022) Chen Wang, Jian Chen, Yang Yang, Xiaoqiang Ma, and Jiangchuan Liu. 2022.  Poisoning attacks and countermeasures in intelligent networks: Status quo and prospects.  _Digital Communications and Networks_ 8, 2 (2022), 225–234.
  * Yuan et al. (2019) Xiaoyong Yuan, Pan He, Qile Zhu, and Xiaolin Li. 2019\.  Adversarial examples: Attacks and defenses for deep learning.  _IEEE Transactions on Neural Networks and Learning Systems_ 30, 9 (2019), 2805–2824.


# WATT-EffNet: A Lightweight and Accurate Model for Classifying Aerial
Disaster Images

Gao Yu Lee  Tanmoy Dam  Md Meftahul Ferdaus  Daniel Puiu Poenar  and Vu N.
Duong  Gao Yu Lee  Tanmoy Dam  Md Meftahul Ferdaus  Daniel Puiu Poenar and Vu
N. Duong Gao Yu Lee, Tanmoy Dam, Md Meftahul Ferdaus, and Vu N. Duong are with
the Air Traffic Management Research Institute (ATMRI), Nanyang Technological
University, Singapore. (email<EMAIL_ADDRESS><EMAIL_ADDRESS><EMAIL_ADDRESS>vu.duong@ntu.edu.sg)Gao Yu Lee and Daniel Puiu
Poenar are with the school of Electrical and Electronic Engineering (EEE),
Nanyang Technological University, Singapore. (email<EMAIL_ADDRESS><EMAIL_ADDRESS>
###### Abstract

Incorporating deep learning (DL) classification models into unmanned aerial
vehicles (UAVs) can significantly augment search-and-rescue operations and
disaster management efforts. In such critical situations, the UAV’s ability to
promptly comprehend the crisis and optimally utilize its limited power and
processing resources to narrow down search areas is crucial. Therefore,
developing an efficient and lightweight method for scene classification is of
utmost importance. However, current approaches tend to prioritize accuracy on
benchmark datasets at the expense of computational efficiency. To address this
shortcoming, we introduce the Wider ATTENTION EfficientNet (WATT-EffNet), a
novel method that achieves higher accuracy with a more lightweight
architecture compared to the baseline EfficientNet. The WATT-EffNet leverages
width-wise incremental feature modules and attention mechanisms over width-
wise features to ensure the network structure remains lightweight. We evaluate
our method on a UAV-based aerial disaster image classification dataset and
demonstrate that it outperforms the baseline by up to 15 times in terms of
classification accuracy and $38.3\%$ in terms of computing efficiency as
measured by Floating Point Operations per second (FLOPs). Additionally, we
conduct an ablation study to investigate the effect of varying the width of
WATT-EffNet on accuracy and computational efficiency. Our code is available at
https://github.com/TanmDL/WATT-EffNet.

###### Index Terms:

Convolutional Neural Networks (CNN), WATT-EffNet, Disaster Scene
classification, Unmanned Aerial Vehicles (UAVs)

##  I Introduction

Recent technological advancements in unmanned aerial vehicles (UAVs) have
significantly improved their capabilities for activities such as remote
sensing and visual geological surveying, leading to greater efficiency and
effectiveness [1]. In particular, UAVs play a critical role in search and
rescue operations, where they can monitor disaster areas for damage and locate
survivors. By classifying disaster scenes, UAVs can quickly determine the type
of disaster that has occurred and focus their search efforts. This is vital
given the limited power and memory resources of UAVs. As noted by [2], the
four primary types of disasters that can be found in aerial image databases
for emergency response applications are fires, floods, building collapses, and
traffic collisions. Fig. 1 displays representative samples of each image type.
The dataset used in this study consists of aerial images of various disaster
classes, including fires, floods, building collapses, and traffic collisions,
collected from multiple sources such as the internet and UAV platforms. To
simulate real-world scenarios as accurately as possible, the dataset also
contains a significant number of non-disaster images labeled as normal, as
highlighted in the same study.

The objective of this study is to create an efficient and lightweight deep
learning classifier that can swiftly detect and classify various events using
effective sensors and microprocessors. To this end, we have analyzed various
architectures, including MobileNet, SqueezeNet, ShuffleNet, and EfficientNet.
These models were designed with the objective of being lightweight by
incorporating a range of techniques, including depth-wise separable
convolutions, pointwise filters, group convolution, channel shuffling, and
strategic scaling of depth, width, and resolution. Specifically, MobileNet
models were created for mobile platform applications and utilized depth-wise
separable convolutions instead of traditional convolutions to reduce the
number of training parameters relative to conventional convolutional networks
with the same depth.

As an illustration, MobileNetV1 [3] was the first MobileNet architecture
proposed, requiring only 4.2 million training parameters. This is in contrast
to the VGG16 and GoogleNet architectures, which require 138 million and 6.8
million parameters, respectively [4]. MobileNetV2, a subsequent architecture
to MobileNetV1, introduced further modifications that significantly reduced
the number of training parameters required from 4.2 million to 3.4 million
[5]. Other instances include the SqueezeNet [6], which utilized 1$\times$1
pointwise filters rather than the 3$\times$3 filters in traditional
convolutional networks to reduce computational costs. The ShuffleNet [7]
introduced group convolution and adds a channel shuffling operation in the
depth-wise convolutional layer for efficient computation. Another model called
the EfficientNet [8] scales the depth, width and resolution of the network
architecture strategically to achieve both computational efficiency and
effectiveness. Despite the aforementioned modifications and reduced structural
complexity, the number of training parameters utilized in these lightweight
models still amounts to millions. Consequently, these models are not
particularly suitable for prolonged UAV operations due to their high FLOPs
demand on the on-board CPU [9].

Figure 1: Example of an image from each respective disaster classes in the
AIDER dataset. Figure 2: The algorithmic structure of our WATT-EffNet, as
shown on the left of the figure. Our modification to the MBConv block layer
using EfficientNet as the backbone is shown on the top right of the figure, as
highlighted by the blue dotted box. We also illustrate the original MBConv
block layer for comparison (red dotted box). The attention mechanism
architecture is illustrated in the dotted orange box on the bottom right of
the figure.

Unmanned aerial vehicles (UAVs) are critical for finding survivors in disaster
zones quickly and efficiently. However, the success of these missions heavily
relies on the computational efficiency and effectiveness of the UAVs’ on-board
classification model. To address this challenge, we propose a novel Wider
ATTENTION EfficientNet (WATT-EffNet) model that incorporates an attention
module across a wider EfficientNet network. Unlike traditional models, this
design emphasizes the width of the network rather than the depth.
Additionally, by utilizing attention, it reduces the computational cost of the
network by processing only key feature maps. The importance of increasing the
width of the network while reducing its depth is supported by [10], whose
findings consistently show gains in classification accuracy for widened
residual networks due to the increased representational power. Moreover, they
have shown that this increase can be achieved with only a marginal increase in
the number of training parameters, where a factor of 2 increase in the
network’s width with a depth reduction of 2.5 leads to a relatively smaller
increase in the number of training parameters. As far as our knowledge
extends, the amalgamation of a width-based architecture with an attention-
based mechanism for UAV disaster scene classification in an existing
EfficientNet model has not been attempted heretofore. In essence, the present
letter outlines several contributions, which can be summarized as follows:

  * •

We present WATT-EffNet, an architectural innovation that leverages the
principle of width as a foundation to augment the capabilities of the original
EfficientNet model. Through the integration of attention modules and a
reduction in overall complexity, this architecture aspires to surpass the
performance of the original EfficientNet by promoting both efficiency and
effectiveness.

  * •

WATT-EffNet architecture endeavors to strike a delicate balance between
computational efficiency and effectiveness, a critical consideration in the
context of classifying disaster scenes captured by UAVs, where mission success
probability must be optimized amidst limited resource constraints. This
approach deviates from the prevalent paradigm among state-of-the-art models,
which do not necessarily take such operational constraints into account.

  * •

The efficacy of our proposed WATT-EffNet architecture has been rigorously
evaluated through the utilization of a subset of the AIDER dataset, where the
results demonstrate that our model is capable of achieving superior $F_{1}$
scores, in contrast to the established baselines, while concurrently utilizing
a substantially lower number of FLOPs, thereby exemplifying its computational
efficiency.

##  II WATT-EffNet

###  II-A Width-varying feature module

The advent of deeper neural network architectures such as ResNet was a
response to the problem of vanishing gradients, an occurrence that plagues the
training of deep neural networks. The phenomenon of vanishing gradients is
characterized by the diminution of the magnitudes of gradients computed during
backpropagation, making it hard to make meaningful updates to the weights of
the network. This, in turn, can impede the successful training of deep neural
networks. In order to overcome the problem of vanishing gradients, the concept
of skip connections was introduced. These connections enable the gradients to
circumvent one or more layers and be directly propagated to the shallower
layers of the network, thus enabling the gradients to traverse the network
with greater ease and fluidity. This mechanism promotes the training of deep
neural networks by preserving the gradient magnitudes. For a given residual
unit $i$, its input $Y_{i-1}$, the learnt feature mapping $\phi_{i}(\cdot)$,
and the trainable parameters of the unit $\boldsymbol{\omega}_{i}$, the output
$Y_{i}$ of the residual unit can be defined recursively as

$Y_{i}=\phi_{i}(Y_{i},\boldsymbol{\omega}_{i})+Y_{i-1}.$ (1)

$\phi_{i}(\cdot)$ is often comprised of two to three stacked convolutional
stages, which, apart from the convolution layers, also comprised of the batch
normalization and a Rectifier Unit (ReLU) as an activation function. In each
of the convolutional layers, $\boldsymbol{\omega}_{i}$ comprised of the kernel
size $n$ $\times$ $m$ and the filter size $l$, and hence
$\boldsymbol{\omega}_{i}=\boldsymbol{\omega}_{i}(n,m,l)$, where the total
parameter in the layer is $nml$.

To enable a wider residual unit, a widening factor $k$ is introduced in which
any unit with $k>1$ are categorized as wide, with the original residual unit
categorized as $k$ = 1. Since we are utilizing the EfficientNet as the
backbone as mentioned in the introduction, the corresponding units modified is
the MBConv Block ($d$). As illustrated in the top right of Fig. 2, the
original MBConv block is composed of a 1$\times$1 convolutional layer followed
by a 5$\times$5 depth-wise convolution. The Squeeze and Excite (SE) [11] block
is then applied to improve feature representation by accounting for the inter-
dependencies across features from different channels. Finally, another
1$\times$1 convolutional layer is applied. Except for the SE block, batch
normalization and ReLU are incorporated in each layer of the MBConv block. In
our WATT-EffNet, we expand upon the width $k$ in the MBConv block so that for
each layer, $\boldsymbol{\omega}_{i}=\boldsymbol{\omega}_{i}(n,m,kl)$ (as
shown in Fig. 2) and hence for block $i$

$Y_{i}=\Phi_{i}(Y_{i},\boldsymbol{\omega}_{i}(n,m,kl))+Y_{i-1}$ (2)

where $\Phi_{i}(\cdot)$ is now the feature mapping associated with the MBConv
blocks. By applying (2) using one substitution steps, the forward pass for our
WATT-EffNet can be expanded as

$\begin{split}Y_{i+2}=\Phi_{i+2}(Y_{i+1},{\boldsymbol{\omega}}_{i+2}(n,m,kl))+Y_{i+1}\\\
=\Phi_{i+1}(Y_{i}+{\boldsymbol{\omega}}_{i+1}(n,m,kl))+\\\
\Phi_{i+2}(Y_{i}+\Phi_{i+1}(Y_{i}+{\boldsymbol{\omega}}_{i+1}(n,m,kl)),{\boldsymbol{\omega}}_{i+2}(n,m,kl)).\end{split}$
(3)

This forms the width-based MBConv block layer in each block of our WATT-EffNet
as illustrated by the light-blue box in the Fig. 2. The depth of the baseline
EfficientNet architecture has a total of 19 layers, including 17 MBConv blocks
[8]. We modified the number of MBConv blocks $d$ and $k$ in our approach such
that the number of training parameters did not exceed 1M. Therefore, the
possible combination of our WATT-EffNet architecture is represented as WATT-
EffNet-$d$-$k$. More details about the variation in our architecture are given
later in the experimental results section.

###  II-B Attention mechanism

The attention module comprises the channel attention and the spatial attention
modules, and their algorithmic structure is illustrated in the bottom right of
Fig. 2. For the channel attention, the process involved extracting and
squeezing the spatial dimension of the input feature map $\boldsymbol{\Phi}$,
which is extracted from the wider structure network in the previous
subsection, followed by parallel average and max pooling. The output feature
from each pooling are then fed into a shared Multi-Layer Perceptron (MLP)
network, with the resultant features finally merged using element-wise
summation. The mathematical representation can be described in the following
equation.

$\begin{split}\boldsymbol{M_{c}(\boldsymbol{\Phi})}&=\sigma(MLP(AvgPool(\boldsymbol{\Phi})+MLP(MaxPool(\boldsymbol{\Phi})))\\\
&=\sigma(\boldsymbol{\Omega_{1}}(\boldsymbol{\Omega_{0}}(\boldsymbol{{\Phi^{c}}_{avg}}))+\boldsymbol{W_{1}}(\boldsymbol{\Omega_{0}}(\boldsymbol{{\Phi^{c}}_{max}}))),\end{split}$
(4)

where $\sigma$ denotes the sigmoid activation function,
$\boldsymbol{\Omega_{0}}$, $\boldsymbol{\Omega_{1}}$ are the weights
associated with the MLPs, $\boldsymbol{M_{c}(\boldsymbol{\Phi})}$ is the
attention map associated with the channel attention, and
$\boldsymbol{{\Phi^{c}}_{avg}}$ and $\boldsymbol{{\Phi^{c}}_{max}}$ denotes
the feature maps obtained from the average and max pooling, respectively. For
the spatial attention, both average pooling and max pooling are applied once
again and concatenated along the channel axis. A convolutional layer of filter
size 7$\times$7 ($F^{7\times 7}$) is then applied to generate the spatial
attention map. Therefore, the spatial feature-map can be described in the
following form,

$\begin{split}\boldsymbol{M_{s}(\Phi)}&=\sigma(F^{7\times
7}([AvgPool(\boldsymbol{\Phi});MaxPool(\boldsymbol{\Phi})]))\\\
&=\sigma(F^{7\times
7}([\boldsymbol{{\Phi^{s}}_{avg}};\boldsymbol{{\Phi^{s}}_{max}}])),\end{split}$
(5)

where $\boldsymbol{M_{s}(\Phi)}$ is the attention map associated with the
spatial attention and $\boldsymbol{{\Phi^{s}}_{avg}}$ and
$\boldsymbol{{\Phi^{s}}_{max}}$ denotes the feature map obtained from the
average and max pooling respectively in this part of the attention module.
Finally, the overall attention mechanism is described as follows,

$\begin{split}\boldsymbol{\Phi^{\prime}}&=\boldsymbol{M_{c}(\Phi)}\otimes\boldsymbol{\Phi},\\\
\boldsymbol{\Phi^{\prime\prime}}&=\boldsymbol{M_{s}(\Phi^{\prime})}\otimes\boldsymbol{\Phi^{\prime}},\\\
\end{split}$ (6)

where $\otimes$ denotes the outer product between the relevant feature and
attention maps. This forms the attention block component of a WATT-EffNet
block as illustrated by the orange box in Fig. 2. A skip connection is also
performed before the next WATT-EffNet block. Therefore, the proposed
architecture can be repeated and used as a single modular block. This means
that the proposed design can be repeated multiple times to create a deeper
network, while still maintaining the benefits of the skip connections and
other features that have been discussed. This modular approach allows for
flexibility and scalability in the design of the network.

##  III Experimental Details

WATT-EffNet is evaluated against SOTA methods using a subset of the AIDER
dataset, which serves as a benchmark[2]. We compare methods that use minimal
training parameters for algorithmic efficiency. Therefore, we did not include
models like ResNet50 [12], VGG16, and Xception [13] which require more than 5
million training parameters. Instead, we include MobileNetV1 [3], MobileNetV2
[5], SqueezeNet [6], ShuffleNet [7], EfficientNet [8] and EmergencyNet [2].
The subset of the AIDER dataset used preserves class imbalance distributions.
The train-valid-test-split ratio is 4:1:2 as same in the AIDER experiment. The
original dataset has 700 images per disaster class and 5700 normal images,
while the subset has 6433 images, as illustrated in Table I.

TABLE I: List of training, validation and test image sets for each class in the subset of the AIDER dataset.  Class | Train | Valid | Test | Total per Class
---|---|---|---|---
Collapsed Building | 367 | 41 | 103 | 511
Fire | 249 | 63 | 209 | 521
Flood | 252 | 63 | 211 | 526
Traffic | 232 | 59 | 194 | 485
Normal | 2107 | 527 | 1756 | 4390
Total Per Set | 3207 | 753 | 2473 | 6433

All images were resized to 224$\times$224$\times$3\. To address class
imbalance, we applied under-sampling to the training and validation sets using
the RandomUnderSampler module from the imblearn library, as in the AIDER
experiment. All simulations were done using the Tensorflow Keras library in
Python on Google’s Colab Pro+ Tesla T4 GPUs and TPUs. All the experimental
simulations are evaluated (including our approach) with an epoch set to 300.
In the preprocessing stage, the intensity values of each pixel in the images
were divided by 255 to normalize the data. Additionally, a kernel regularizer
with a coefficient of 1e-4 was incorporated into every convolutional block
layer. Our approach employed both categorical cross-entropy and cosine loss as
the training loss, as suggested by [14], which argues that this combination is
effective for models that are trained from scratch and without any pre-
training on small data, which is the case for our dataset. Both losses were
given equal importance in the combined loss function. The optimization
algorithm used was Root-Mean-Square propagation (RMSprop) and was set to a
decay rate of 1e-6.

The development of our proposed WATT-EffNet model is predicated on the
utilization of the wider structure and attention mechanism principle, thus the
identification of the optimal width to depth ratio ($k$ and $d$ respectively)
is of paramount importance. This necessitates the examination of a plethora of
possible permutations and combinations of the width ($k$) and depth ($d$)
utilizing a range of values for $k=\\{2,3,4,5,6,7\\}$ and $d=\\{1,3,5\\}$. The
next section of this research will be dedicated to the exploration of the
aforementioned permutations and combinations through the utilization of the
AIDER dataset, in order to arrive at the optimal values for $k$ and $d$.

##  IV Results and Discussions

Table II illustrates that the proposed model has demonstrated a significant
improvement in performance in comparison to the baseline EfficientNet model as
well as the various benchmark methods. Specifically, the performance
improvement has been quantified as a $10.6\%$ increase in terms of $F_{1}$
score. Additionally, the proposed model has also shown a drastic reduction in
computational complexity as it requires 35 times fewer FLOPs when compared to
the experimental conditions. Furthermore, it is worth noting that although the
EmergencyNet model utilizes fewer parameters, the WATT-EffNet model’s wider
structure design allows for more efficient utilization of parameters,
resulting in lower FLOPs and overall computational complexity. The results of
these evaluations have revealed a substantial enhancement in performance as
compared to the EmergencyNet model with an improvement of more than $6.5\%$.
Furthermore, the proposed model also demonstrates a remarkable reduction in
computational complexity, as quantified by FLOPs, with a 8-fold decrease in
comparison to the EmergencyNet model.

TABLE II: $F_{1}$ scores, FLOPs, and training parameters for the existing SOTA
CNN-based approaches on the AIDER dataset∗. Bolded values denote the highest
$F_{1}$ score, lowest FLOPs and lowest training parameters.

SOTA Model |  $F_{1}(\%)$($\shortuparrow$) |  FLOPs ($\shortdownarrow$) |  Parameters ($\shortdownarrow$)
---|---|---|---
MobileNetV1 [3] | 80.2 | 972 | 3,233,861
MobileNetV2 [5] | 81.0 | 625 | 2,282,629
SqueezeNet [6] | 82.3 | 531 | 725,073
ShuffleNet [7] | 81.7 | 972 | 4,023,865
EfficientNet [8] | 80.0 | 774 | 3,499,453
EmergencyNet [2] | 83.1 | 185 | 94,420
WATT-EffNet-3-6 | 88.5 | 22 | 688,661

  * •

*All models are trained on the same environment.

We have examined the performance of each class through a confusion matrix and
a precision vs recall curve, which is illustrated in Fig. 3 and 4
respectively. Our analysis revealed that the normal class had the lowest
prediction percentage (81.6%) among all the classes due to the presence of
images that did not have an ideal view perspective and the key features only
occupied a small area of the image. On the other hand, the traffic incident
class had the highest prediction percentage (94.2%) among all the classes.
However, the $F_{1}$ values obtained in our work were lower than those
obtained in the EmergencyNet. This is because the methods utilized in our
context are mostly, if not all, CNN-based network, and such network usually
perform better with increasing amount of training data. Nevertheless, our
optimal lightweight algorithmic design is still capable of maximizing learning
even when using a limited dataset for training. Lastly, it is worth mentioning
that some works such as [15] and [16] have highlighted the significance of
incorporating a finer spatial resolution during classification and have
assessed its impact on the classification performance. This is crucial in
disaster scenarios, as they typically occur within a limited region in a
larger captured field of view. While this aspect was not the primary focus of
our research, it is an intriguing and critical direction that we plan to
explore in our future works.

Figure 3: The confusion matrix for our WATT-EffNet-3-6. Figure 4: The
precision vs recall curves for the predicted classes using our aforementioned
model against the original classes. Here, class 0-5 represents collapsed
building, fire, flood, traffic incident and the normal class respectively.
TABLE III: $F_{1}$ scores, FLOPs, training parameters for our WATT-EffNet
framework as a function of number of MBConv Blocks (first value in model
variant name) and widths (second value in model variant name) with respect to
the original EfficientNet framework∗.

Model Variant |  $F_{1}(\%)$($\shortuparrow$) |  FLOPs ($\shortdownarrow$) |  Parameters ($\shortdownarrow$)
---|---|---|---
WATT-EffNet-1-2 | 63.1$\pm$0.02 | 22 | 8,501
WATT-EffNet-1-3 | 64.2$\pm$0.07 | 22 | 13,693
WATT-EffNet-1-4 | 66.5$\pm$0.38 | 22 | 19,909
WATT-EffNet-1-5 | 68.2 $\pm$0.03 | 22 | 27,149
WATT-EffNet-1-6 | 70.4 $\pm$0.87 | 22 | 35,413
WATT-EffNet-1-7 | 72.7$\pm$0.05 | 22 | 44,701
WATT-EffNet-3-2 | 84.2$\pm$1.03 | 22 | 106,629
WATT-EffNet-3-3 | 83.3$\pm$1.08 | 22 | 205,673
WATT-EffNet-3-4 | 86.0$\pm$1.05 | 22 | 335,693
WATT-EffNet-3-5 | 85.2$\pm$0.98 | 22 | 496,689
WATT-EffNet-3-6 | 88.5$\pm$0.76 | 22 | 688,661
WATT-EffNet-3-7 | 87.3$\pm$0.72 | 22 | 911,609
WATT-EffNet-5-2 | 86.8$\pm$1.05 | 22 | 371,493
WATT-EffNet-5-3 | 86.4$\pm$0.93 | 22 | 720,233

  * •

*All models are trained on the same environment.

#### Ablation Studies

The WATT-EffNet model was developed with the goal of achieving high accuracy
while maintaining a lightweight architecture. To evaluate its performance, we
conducted an ablation study by varying the number of MBConv Blocks ($d$) and
width ($k$) of the model. Additionally, we extended the study to include
variations with and without the attention mechanism, as recorded in Table III
and IV respectively.

The results of various WATT-EffNet model variants considering attention
mechanism are presented in Table III and are quantified by the standard
deviation and mean $F_{1}$ scores. The first number in the variant name
indicates the number of MBConv blocks ($d$) used in the network, while the
second number represents the width multiplier ($k$). The $F_{1}$ score, FLOPs,
and number of parameters are reported for each variant. The best results were
obtained when $d$ was equal to 3 and $k$ was equal to 6, however it is
noteworthy that the results were similar or even better when using a smaller
value of $k=2$ with a smaller number of parameters than the SOTA models. It is
worth mentioning that we examined all the possible conditions until the
parameters did not exceed 1M.

Similarly, Table IV presents the results of various WATT-EffNet model variants
where the attention mechanism was discarded. The $F_{1}$ score, FLOPs, and
number of parameters are reported for each variant. The best results were
obtained when $d$ was equal to 3 and $k$ was equal to 6. The $F_{1}$ scores
obtained in Table IV are lower than those given in Table III. This indicates
that incorporating attention improves the classification performance by a
significant margin in some cases and by a small margin in others (e.g., WATT-
EffNet-3-3 displayed an improvement in $F_{1}$ scores by only around 0.2%
while WATT-EffNet-1-6 demonstrated an improvement by around 6%). It is worth
mentioning that the FLOPs used remains the same regardless of whether
attention is incorporated or not, which implies that attention modules can be
incorporated even for a given limited computational resource since no
additional costs are incurred to enhance the classification performance.

TABLE IV: $WATT-EffNet$ metrics without the attention mechanism∗.

Model Variant |  $F_{1}(\%)$($\shortuparrow$) |  FLOPs ($\shortdownarrow$) |  Parameters ($\shortdownarrow$)
---|---|---|---
WATT-EffNet-1-2 | 62.1$\pm$0.70 | 22 | 8,501
WATT-EffNet-1-3 | 63.2$\pm$0.25 | 22 | 13,693
WATT-EffNet-1-4 | 63.9$\pm$0.28 | 22 | 19,909
WATT-EffNet-1-5 | 65.9 $\pm$1.39 | 22 | 27,149
WATT-EffNet-1-6 | 66.4 $\pm$2.44 | 22 | 35,413
WATT-EffNet-1-7 | 69.1$\pm$0.91 | 22 | 44,701
WATT-EffNet-3-2 | 81.5$\pm$1.04 | 22 | 106,629
WATT-EffNet-3-3 | 83.1$\pm$1.22 | 22 | 205,673
WATT-EffNet-3-4 | 85.2$\pm$0.98 | 22 | 335,693
WATT-EffNet-3-5 | 85.0$\pm$0.92 | 22 | 496,689
WATT-EffNet-3-6 | 85.4$\pm$1.29 | 22 | 688,661
WATT-EffNet-3-7 | 85.2$\pm$0.46 | 22 | 911,609
WATT-EffNet-5-2 | 83.5$\pm$0.88 | 22 | 371,493
WATT-EffNet-5-3 | 84.3$\pm$0.89 | 22 | 720,233

  * •

*All models are trained on the same environment.

##  V Conclusions

The present study introduces a novel width-based EfficientNet architecture,
named Wider ATTENTION EfficientNet (WATT-EffNet), that incorporates the
convolutional block attention mechanism. This new network architecture enables
the creation of a shallower EfficientNet-based classifier, resulting in
reduced computational processing requirements, by selectively attending to
only the most informative features present on the feature map. The proposed
WATT-EffNet variants were trained and evaluated on a challenging subset of the
AIDER dataset, featuring a highly imbalanced distribution of non-disaster and
disaster images of different classes. The obtained results consistently
demonstrate the effectiveness and efficiency of the proposed architecture in
comparison to state-of-the-art methods. The significance of this work lies in
highlighting the role of width and attention in a CNN-based network in
enhancing both efficiency and classification performance, making it a
promising solution for visual-based UAV search-and-rescue operations. In
future works, the proposed WATT-EffNet architecture will be tested in real-
time UAV operations for validation, and alternative methods such as GAN-based
augmentation [17, 18] will be explored to further improve the model’s
performance.

## Acknowledgement

This research is supported by the Civil Aviation Authority of Singapore and
NTU under their collaboration in the Air Traffic Management Research
Institute. Any opinions, findings and conclusions or recommendations expressed
in this material are those of the author(s) and do not necessarily reflect the
views of the Civil Aviation Authority of Singapore.

## References

  * [1] A. Valsan, B. Parvathy, V. D. GH, R. Unnikrishnan, P. K. Reddy, and A. Vivek, “Unmanned aerial vehicle for search and rescue mission,” in _2020 4th International Conference on Trends in Electronics and Informatics (ICOEI)(48184)_. IEEE, 2020, pp. 684–687.
  * [2] C. Kyrkou and T. Theocharides, “Emergencynet: Efficient aerial image classification for drone-based emergency monitoring using atrous convolutional feature fusion,” _IEEE Journal of Selected Topics in Applied Earth Observations and Remote Sensing_ , vol. 13, pp. 1687–1699, 2020\.
  * [3] A. G. Howard, M. Zhu, B. Chen, D. Kalenichenko, W. Wang, T. Weyand, M. Andreetto, and H. Adam, “Mobilenets: Efficient convolutional neural networks for mobile vision applications,” _arXiv preprint arXiv:1704.04861_ , 2017.
  * [4] S. Liu and W. Deng, “Very deep convolutional neural network based image classification using small training sample size,” in _2015 3rd IAPR Asian conference on pattern recognition (ACPR)_. IEEE, 2015, pp. 730–734.
  * [5] M. Sandler, A. Howard, M. Zhu, A. Zhmoginov, and L.-C. Chen, “Mobilenetv2: Inverted residuals and linear bottlenecks,” in _Proceedings of the IEEE conference on computer vision and pattern recognition_ , 2018, pp. 4510–4520.
  * [6] F. Iandola, S. Han, M. Moskewicz, K. Ashraf, W. Dally, and K. Keutzer, “Squeezenet: Alexnet-level accuracy with 50$\times$ fewer parameters and¡ 0.5 mb model size. arxiv 2016,” _arXiv preprint arXiv:1602.07360_.
  * [7] N. Ma, X. Zhang, H.-T. Zheng, and J. Sun, “Shufflenet v2: Practical guidelines for efficient cnn architecture design,” in _Proceedings of the European conference on computer vision (ECCV)_ , 2018, pp. 116–131.
  * [8] M. Tan and Q. Le, “Efficientnet: Rethinking model scaling for convolutional neural networks,” in _International conference on machine learning_. PMLR, 2019, pp. 6105–6114.
  * [9] F. Yao, S. Wang, L. Ding, G. Zhong, L. B. Bullock, Z. Xu, and J. Dong, “Lightweight network learning with zero-shot neural architecture search for uav images,” _Knowledge-Based Systems_ , vol. 260, p. 110142, 2023.
  * [10] S. Zagoruyko and N. Komodakis, “Wide residual networks,” _arXiv preprint arXiv:1605.07146_ , 2016.
  * [11] H. Jie, S. Li, S. Gang, and S. Albanie, “Squeeze-and-excitation networks,” in _Proceedings of the IEEE conference on computer vision and pattern recognition_ , vol. 5, 2018.
  * [12] K. He, X. Zhang, S. Ren, and J. Sun, “Deep residual learning for image recognition,” in _Proceedings of the IEEE conference on computer vision and pattern recognition_ , 2016, pp. 770–778.
  * [13] F. Chollet, “Xception: Deep learning with depthwise separable convolutions,” in _Proceedings of the IEEE conference on computer vision and pattern recognition_ , 2017, pp. 1251–1258.
  * [14] B. Barz and J. Denzler, “Deep learning on small datasets without pre-training using cosine loss,” in _Proceedings of the IEEE/CVF Winter Conference on Applications of Computer Vision_ , 2020, pp. 1371–1380.
  * [15] D. He, Q. Shi, X. Liu, Y. Zhong, and L. Zhang, “Generating 2m fine-scale urban tree cover product over 34 metropolises in china based on deep context-aware sub-pixel mapping network,” _International Journal of Applied Earth Observation and Geoinformation_ , vol. 106, p. 102667, 2022.
  * [16] M. Mirik and R. J. Ansley, “Utility of satellite and aerial images for quantification of canopy cover and infilling rates of the invasive woody species honey mesquite (prosopis glandulosa) on rangeland,” _Remote Sensing_ , vol. 4, no. 7, pp. 1947–1962, 2012.
  * [17] T. Dam, S. G. Anavatti, and H. A. Abbass, “Mixture of spectral generative adversarial networks for imbalanced hyperspectral image classification,” _IEEE Geoscience and Remote Sensing Letters_ , 2020.
  * [18] T. Dam, M. M. Ferdaus, M. Pratama, S. G. Anavatti, S. Jayavelu, and H. Abbass, “Latent preserving generative adversarial network for imbalance classification,” in _2022 IEEE International Conference on Image Processing (ICIP)_. IEEE, 2022, pp. 3712–3716.