✓✓ ✗✗

# Adversarial Robustness in Unsupervised Machine Learning: A Systematic Review

Mathias Lundteigen Mohus<EMAIL_ADDRESS>0000-0002-1105-6515 and
Jingyue Li<EMAIL_ADDRESS>0000-0002-7958-391X NTNU Department of Computer
ScienceSem Sælandsvei 9TrondheimTrøndelagNorwayNO-7491

###### Abstract.

As the adoption of machine learning models increases, ensuring robust models
against adversarial attacks is increasingly important. With unsupervised
machine learning gaining more attention, ensuring it is robust against attacks
is vital. This paper conducts a systematic literature review on the robustness
of unsupervised learning, collecting 86 papers. Our results show that most
research focuses on privacy attacks, which have effective defenses; however,
many attacks lack effective and general defensive measures. Based on the
results, we formulate a model on the properties of an attack on unsupervised
learning, contributing to future research by providing a model to use.

unsupervised learning, machine learning, adversarial robustness, adversarial
attack, systematic literature review

††ccs: Computing methodologies Unsupervised learning††ccs: Security and
privacy Software security engineering

##  1\. Introduction

As technologies mature, hardware capabilities increase, and legislation is
introduced, adopting machine learning technologies in various applications
seems inevitable, with the technology generally divided into three types:
Supervised Learning (SL), Reinforcement Learning (RL), and Unsupervised
Learning. SL focuses on approaches where the training data is labeled, i.e., a
data sample maps to a correct value. A variety of cases use this, e.g., object
detection (Ali and Shah, 2005), image classification (Peña et al., 2014), and
voice recognition (Ali et al., 2021). RL is an approach where the model trains
in the working environment, rewarding or punishing the model based on its
performance. Use cases for this are bug detection in software (Rani et al.,
2023), computing resource optimization (Shi et al., 2020), and control of
cyber-physical systems (Liu et al., 2019c). UL performs training of its models
without guidance; pure data are used as the basis for training. Use cases for
this is data generation (Devaranjan et al., 2020), e.g., GANs;
encoding/decoding (Srivastava et al., 2015), e.g., autoencoders , and data
clustering (Chander and Vijaya, 2021).

By its lack of need for guided information, UL can enable more data to be
usable. In a world that generates a large amount of data (around 2.5
quintillion bytes each day (Dihuni, 2020)), methods for using unsupervised
data would be invaluable. However, where technology is used, adversaries
inevitably attempt to exploit vulnerabilities in these technologies. With the
increased use of ML in various fields, it is crucial to explore the
adversarial robustness of ML models, particularly the use of UL, as this field
has received less focus than supervised and reinforcement learning.

Liu et al. (2018) summarized attacks on machine learning models broadly, while
Guan et al. (2018) cover machine learning within cybersecurity. Zhang et al.
(2020) outline privacy vulnerabilities in machine learning, which could reveal
sensitive information used to train synthetic data generators in the context
of UL. At the same time, Martins et al. (2020)cover adversarial examples
relating to detection techniques, where the input to an ML model is perturbed
in a specific way in order to disrupt the functionality of the model.

In contrast with Liu et al. (2018), Guan et al. (2018) and Zhang et al.
(2020), where parts of UL adversarial robustness are covered, we aim to
conduct a Systematic Literature Review (SLR) on the topic of adversarial
robustness within the whole field of unsupervised machine learning. The papers
focus on answering the three research questions below:

  * •

RQ1: What types of attacks exist against UL-based systems, and what are the
common attributes of the attacks?

  * •

RQ2: What defensive countermeasures exist for attacks against UL-based
systems?

  * •

RQ3: What are the challenges to defending against possible attacks targeting
UL-based systems?

The SLR follows the guidelines outlined by Keele et al. (2007) and identified
86 papers relevant to adversarial robustness in unsupervised learning.
Thematic analysis was then used on these papers to summarize the papers
according to the RQs systematically.

The results of SLR reveal that:

  * •

GANs are mostly targeted by privacy attacks.

  * •

AutoEncoders are mostly targeted by adversarial examples.

  * •

Poisoning attacks are used against most types of UL.

  * •

Most attacks utilize distance metrics in limiting how much the attack alters
samples for the attack.

  * •

Several defensive methods that work in a supervised or reinforcement context
are not applicable in an unsupervised context. For example, fine pruning and
output activation clustering are not suitable defenses for generative models.

  * •

The adversary’s knowledge of the target system greatly affects the attack’s
performance.

  * •

There is a noticeable lack of defensive methods used in UL, except for
differential privacy.

The contributions of the study are:

  * •

We provide a complete and novel summary of the state-of-the-art adversarial
robustness of unsupervised learning relating to attributes of the attack,
target, and defense.

  * •

We have summarized and outlined the main challenges facing the adversarial
robustness of unsupervised learning.

  * •

We create a model of the attributes associated with attacks against
unsupervised machine learning to analyze and compare adversarial attacks
against UL.

The rest of the paper is organized as follows. Section 2 covers the basics of
unsupervised learning. Section 3 covers surveys related to the security and
robustness of machine learning. Section 4 explains our research method.
Sections 5 to 7 present the results. Section 8 discusses the results. Section
9 concludes and proposes future work.

##  2\. Background

Within UL, two types of technologies generally exist: discriminatory and
generative. Discriminatory covers models where data are input into the model,
producing a specific output based on its training. Examples of these are
clustering (Grira et al., 2004; Hahne et al., 2008), AutoEncoders (Bank et
al., 2020; Pinaya et al., 2020) and super-resolution networks (Anwar et al.,
2020).

  * •

Clustering refers to a broad family of methods that defines clusters of data
points based on certain heuristics. Unsupervised clustering finds these
clusters based on distance metrics between data points to find which data
points are mathematically close enough to belong to the same cluster.
Clustering is often used for data analysis in data-heavy fields, such as gene
information (Sumedha and Weigt, 2008).

  * •

AutoEncoders is a method for training a neural network to create an inner
representation of the training data, since the model trains to output the same
data as the input. In most cases, AutoEncoders force a dimensionality
reduction on the inner representation by limiting the number of hidden nodes
in the network, where the first part of the network training is to create the
encoding, and the last part of the network training is to decode the
representation into the original data. One application of AutoEncoders is
denoising data (Bank et al., 2020) where data is passed through the
AutoEncoder to remove noise.

  * •

Super-resolution networks are neural networks that work to increase the
dimensionality of the data, by training the model to output a higher-
dimensional piece of data, based on a lower-dimensional piece of data, e.g. a
low-resolution image. Super-resolution is often used in image technologies to
upscale the resolution of an image (Anwar et al., 2020).

Generative models cover models which do not directly use input data but are
used to create new data with properties tied to the properties of the training
set. Examples of this are Variational AutoEncoders (Kingma et al., 2019) and
Generative Adversarial Networks (Creswell et al., 2018; Wang et al., 2017;
Goodfellow et al., 2020).

  * •

Variational AutoEncoder is a method to use AutoEncoder to generate data by
forcing constraints on the inner representation of the AutoEncoder, in turn
making it possible to input random noise into the decoder part of the
AutoEncoder and producing an output that is statistically similar to the
training set used to train the AutoEncoder. This model is one of the main
components of generating deepfake images by supplanting some of the encoded
features of an image with another, generating an image with features from both
images Rocco (2021).

  * •

Generative adversarial networks consist of two models, the generator and the
discriminator. The generator is a neural network with random noise as input,
while the discriminator is a classifier trained to differentiate between the
output from the generator and the training data. The generator and the
discriminator are, in turn, trained against each other based on the
performance of the discriminator. GANs are used for various synthetic data
generation, e.g., medical data generation (Han et al., 2018).

##  3\. Related Work

As the field of machine learning security has matured, several papers have
been published that survey the state of the art within both broad and narrow
fields within ML security.

Liu et al. (2018) use the taxonomy from Fredrikson et al. (2015) to analyze 42
papers on attacks against machine learning models. The survey group the papers
based on 1) The targeted algorithm, with the types being: Naïve Bayes/Logistic
Regression/Decision Tree, Support Vector Machines (SVM), Principal Component
Analysis (PCA)/Least Absolute Shrinkage and Selection Operator (LASSO),
Clustering, and Deep Neural Network (DNN), and 2) The type of attack, with the
types being: Poisoning Attack, Evasion Attack, Impersonation Attack, and
Inversion Attack. Liu et al. (2018) also focus explicitly on comparing types
of Adversarial Examples attacks, comparing the techniques, advantages,
disadvantages, and formations of the adversarial examples. Additionally, Liu
et al. (2018) summarize defensive techniques for ML, outlining how its
surveyed papers assess the security of their ML models and which defensive
mechanisms are put in place: proactive or reactive. Defenses in training are
also discussed, such as Data Sanitation, Bootstrap Aggregating, Random
Subspace Method, and designing more robust algorithms. For defenses in the
testing/inferring phase, Liu et al. (2018) state that “countermeasures in the
testing/inferring phase mainly focus on the improvement of learning
algorithms’ robustness,” describing techniques like Game Theory, Adversarial
Training, Defensive Distillation, Dimension Reduction, as well as ensemble
methods. Lastly, Liu et al. (2018) also cover the privacy of data used in ML,
including Differential Privacy and Homomorphic Encryption.

Martins et al. (2020) perform a survey on the use of adversarial examples
targeted towards detection techniques, particularly malware (11 papers) and
intrusion detection (nine papers), surveyed from mainly IEEE Xplore, Springer,
arXiv, ScienceDirect, Research Gate, and Google, using the keywords “Intrusion
Detection,” “Malware Detection” and “Adversarial Machine Learning.” The paper
compares several features recurring in its literature: Attack Techniques,
Classifiers, Datasets, Metrics, and Results and Conclusions. Martins et al.
(2020) also cover defensive techniques against adversarial examples:
Adversarial Training, Gradient Masking, Defensive Distillation, Feature
Squeezing, Universal Perturbation Defense Method, and MagNet.

Guan et al. (2018) cover two related topics: ML’s impact on cybersecurity and
AI systems’ robustness, presenting 14 papers. The paper labels attributes
covering the Research Direction, whether AI for cybersecurity or security of
AI, points of innovation in the paper, and a summary of the idea for each
paper. The paper also synthesizes common threats and solutions in AI security,
such as Data Sanitation and Learning Algorithm Improvement being the solution
to Poisoning attacks, Adversarial Training and Output Smoothing being
solutions to Evasion and Impersonation Attacks, and Differential Privacy and
Homomorphic Encryption being the solution to Inversion Attacks.

Zhang et al. (2020) present a survey on privacy threats and mitigations within
ML. The paper outlines a taxonomy for attacks, covering the Stage (where the
attack happens), Goal, Knowledge, Capability, and Strategy of the attack. The
authors summarize privacy threats against models as Parameter/Function,
Hyperparameter, and Architecture, with defensive measures being Confidence
Perturbation and Malicious Query Behaviour Detection. Privacy threats against
data are summarized as Data Memorization attacks, Model Inversion Attacks, and
Membership Inference attacks, with defensive measures being Hierarchical Model
Stacking Techniques, Attribute Correlation with Adversarial Perturbations,
Meta-Classifier, Collaborative Learning, Homomorphic Encryption, and
Differential Privacy.

Table 1 outlines the contribution of the existing surveys. Only one survey
covers GAN (Zhang et al., 2020), two surveys cover clustering (Liu et al.,
2018; Martins et al., 2020), and none covers attacks on AutoEndoers. The
systematic description of attack goals is also only present in two papers (Liu
et al., 2018; Guan et al., 2018). There is also a need for systematic metrics
descriptions to determine attack success, with only one paper (Martins et al.,
2020) summarizing the metrics. Two papers cover the attacker’s knowledge,
while the other two somewhat systematically cover the knowledge. Additionally,
three papers cover attacks in training and production systems, while Martins
et al. (2020) do not cover attacks in training scenarios. Finally, all the
papers cover defensive measures against attacks.

Table 1. Comparison between the four surveys and this paper regarding categories covered and whether the survey is performed systematically. ✓ is when the paper covers the category, ✗ is when the paper does not cover the category, and * is when the category is partly covered by the paper Ref. |  Short Description |  Systematic |  Attack Goal |  Attack Knowledge

| Attack Focus |  Metrics | Target Types |  Defenses
---|---|---|---|---|---|---|---|---
Production |  Training |  GAN |  AutoEncoder
|  Clustering |  Classification |  Super-resolution
(Liu et al., 2018) |  Adversarial Examples attacks against Machine Learning |  ✗ |  ✓ |  * | ✓ |  ✓ |  ✗ | ✗ | ✗ |  ✓ |  ✗ |  ✗ |  ✓
(Martins et al., 2020) |  Adversarial Examples against detection techniques |  * |  ✗ |  ✓ | ✓ |  ✗ |  ✓ | ✗ | ✗ |  ✓ |  ✗ |  ✗ |  ✓
(Guan et al., 2018) |  Impact of ML on security, and robustness of AI systems |  ✗ |  ✓ |  * | ✓ |  ✓ |  ✗ | ✗ | ✗ |  ✗ |  ✗ |  ✗ |  ✓
(Zhang et al., 2020) |  Privacy threats and mitigations within ML |  ✗ |  ✗ |  ✓ | ✓ |  ✓ |  ✗ | ✓ | ✗ |  ✗ |  ✗ |  ✗ |  ✓
This work |  A systematic review on adversarial robustness of unsupervised machine learning |  ✓ |  ✓ |  ✓ | ✓ |  ✓ |  ✓ | ✓ | ✓ |  ✓ |  ✓ |  ✓ |  ✓

In summary, there is an evident lack of a systematic and comprehensive survey
focusing on attacks targeting all kinds of ULs and corresponding defenses.
Considering the potential importance of unsupervised learning in environments
with a large amount of unlabeled data, a survey to cover the adversarial
robustness of unsupervised machine learning would provide the industry with
insight into the shortcomings of UL technologies and academia with knowledge
on the state-of-the-art attacks and defenses that are covered in the field of
adversarial robustness within UL.

##  4\. Research Design and Implementation

###  4.1. Systematic Literature Review outline

In computer science, Keele et al. (2007) show one framework for conducting an
SLR within software engineering and related fields. This SLR uses the method
outlined in Keele et al. (2007) as a base, which includes 1) Creating the
search query based on inclusion criteria, 2) Filtering based on exclusion
criteria, and 3) Validating the quality of papers.

###  4.2. Creating the search query

Based on the research questions and research focus, we generate the search
query using a modified version of PICOC (Population, Intervention, Comparison,
Outcome, Context) from Petticrew and Roberts (2008), without the Comparison
section, as it was considered not applicable to the context of this SLR. While
the PIOC framework groupings guide the query’s creation, the specific
interpretation in Keele et al. (2007) do not map perfectly to the chosen
tokens, as described below.

  * •

Population: Relating to the security of unsupervised machine learning, the
population will cover general areas of technologies that are related to or
specifically UL.

  * •

Intervention: The intervention relates here to general descriptions of how an
attacker might exploit vulnerabilities in UL systems.

  * •

Outcome: Outcome relates to the desired data or information that a piece of
research produces through their papers, specifically finding the
vulnerabilities and defensive methods of UL methods.

  * •

Context: The context relates to the context in which the research is explored,
specifically which part the UL is considered, e.g., the technology versus
human error.

As Keele et al. (2007) do not outline a framework for evaluating the quality
of the search query, other than following the PICOC framework, while Denyer et
al. (2009) only state “it is often wise to spend significant time on
constructing the search strings, for this investment can improve the
efficiency of the search”, we formulate here a method for evaluating the
query. First, a list of 13 papers relating to security in UL was created,
based on an ad-hoc search, in addition to previous knowledge of specific
papers. We then search the IEEE Xplore, ACM Digital Library, arXiv, and Scopus
databases using the search query and note how many of the papers in the ad-hoc
list are present in the resulting list. While the ad-hoc nature of the list is
not ideal, we consider this method better than the alternative, i.e., not
having a baseline of comparison, as we know the papers in the ad-hoc list
should ideally be present in the resulting list. The second consideration
metric was the number of papers produced by the search query. For this, we
would only accept queries that produced between 1000 and 10 000 papers to
ensure enough relevant papers were included and ensure the following
filtration process would not take too long. The results of the query testing
are summarized in Table 2. Rounds 1 and 2 were good examples of casting a too-
narrow and too-wide net, highlighted by the matching percentage to the ad-hoc
list and the total number of papers being very low and very high,
respectively. Round 3 produced the correct number of papers but was considered
not precise enough, with a matching score of 15%. Round 4 also produced the
correct number of papers, increased the matching rate to the ad-hoc list, and
was therefore chosen as the query for the SLR.

Table 2. Evaluation of different queries Search query evaluation round | Peer-reviewed | Pre-published | Total | Match with ad-hoc list
---|---|---|---|---
1 | 529 | 39 | 568 | 8%
2 | 120 241 | 763 | 121 004 | 85%
3 | 4623 | 26 | 4649 | 15%
4 | 2647 | 57 | 2704 | 46%

The resulting query based on the PIOC categories produces the following search
query, modified to fit each database search engine:

(“Clustering” OR “cluster” OR “AutoEncoder” OR “GAN” OR “Generative
Adversarial Network” OR “Unsupervised Learning” OR “UL”) AND (“vulnerability”
OR “exploitation” OR “Poisoning” OR “extraction” OR “obfuscation” OR
“Adversarial attack” OR “cyber-attack” OR “cyber attack”) AND (“Increase
robustness” OR “defend against” OR “increase defence” OR “Mitigate attack” OR
“Attack success” OR “attack against” OR “attack vector”) AND (“model” OR
“models” OR “system” OR “systems”)

###  4.3. Paper filtering

The initial list of papers was generated using the query in IEEE Xplore, ACM
Digital Library, arXiv, and Scopus. Including unpublished papers in this
review follows from the recommendation from Denyer et al. (2009) “extensive
searches are conducted to include both published and unpublished studies.” The
initial list was imported into EndNote, automatically removing any duplicates.
We used the inclusion criterion for filtering the papers: the paper must cover
attacks against UL technologies. We show the exclusion criteria in Table 3. We
specify criteria E3 to 1) avoid including outdated research and 2) limit the
number of papers that would have to be filtered.

Table 3. The exclusion criteria for the systematic review Index |  Criteria
---|---
E1 |  The paper covers attack against general technologies which is not currently used in an unsupervised manner
E2 |  Not written in English
E3 |  Written before 2011
E4 |  Paper is not available
E5 |  Not relevant to UL

We performed several rounds of filtering. The first round focused on removing
irrelevant papers from the list. In this round, the papers were considered
only based on the title and evaluated on whether 1) It does not relate to
security, 2) It does not relate to UL, or 3) The title singularly refers to
supervised or reinforcement learning. The second round focused on further
limiting the number of irrelevant papers from the list. The evaluation
criteria remain the same as for filtration round 1. Still, we give the
evaluation for each paper more time, and we document each decision to filter
in or out. The third round limits the number of irrelevant papers even
further, where we use the same criteria as filtration rounds 1 and 2, use more
time for each evaluation and consider the paper’s abstract. Here, the
evaluator can also consider a paper in its entirety to confidently ascertain
whether it is relevant or not, according to the criteria referred to here as
round 3.5. The workflow for the filtration steps and quality assessment is
outlined in Figure 1.

Figure 1. Workflow for the filtration rounds and quality assessment

Outline of the steps taken to produce the final list of papers from the
initial results from the query. Filtration rounds 1, 2, 3, and 3.5, and
quality assessment, after which two rounds of forward and backward snowballing
produced another list of papers to be filtered.

###  4.4. Quality Assessment

To ensure the papers contributing to the SLR have a certain quality, Keele et
al. (2007) also outline the importance of quality assessment (QA). Our QA
consists of 11 questions based on the approach presented by Keele et al.
(2007), scoring 0, 0.5, or 1 point per question. For any paper scoring lower
than 5.5 points, we exclude the paper. Additionally, to ensure internal
validity, a random sample of papers will be analyzed using the Cohen Kappa
statistic (Warrens, 2015), measured between the primary researcher doing the
QA, and a separate researcher, to ensure the QA scores are accurately applied.
The questions used for the QA are in Appendix A.

###  4.5. Snowballing

After the initial round of search, filtration, and QA, we also perform two
backward and forward snowballing rounds to ensure we include as many relevant
papers as possible in this SLR. For the first round of snowballing, the final
list of papers after the QA is used as a base, and for the second round, we
use the list of papers after QA from the first round of snowballing. Backward
snowballing (BS) is performed by compiling a list of all the references used
in each paper. In contrast, forward snowballing (FS) is performed using
Semantic Scholar to find papers referencing each paper. After one round of
initial search and two rounds of forward and backward snowballing, we combine
the papers into a single list of papers to be analyzed.

###  4.6. Identified papers

For each round of the systematic literature review, the number of papers for
consideration is summarized in Table 4. After the snowballing steps, we
further reduced the list of 114 papers to 86 by finding duplicates, updated
versions, or published duplicates of unpublished papers, with the last forward
snowballing round performed in August 2022.

Table 4. Summary of the number of papers after each round of filtration and quality assurance. We base each snowballing round on the list of papers from the quality assurance in the previous round. |  Initial Round |  Forward Snowballing 1 |  Backward Snowballing 1 |  Forward Snowballing 2 |  Backward Snowballing 2 |  Sum
---|---|---|---|---|---|---
Initial List |  5825 |  996 |  365 |  683 |  1260 |  9129
Filtration 1 |  1002 |  651 |  160 |  95 |  171 |  2079
Filtration 2 |  478 |  330 |  75 |  53 |  36 |  972
Filtration 3 |  28 |  54 |  15 |  37 |  11 |  145
After Quality Assurance |  24 |  46 |  11 |  27 |  6 |  114

###  4.7. Data analysis and synthesis

Thematic analysis (TA) is a widely used method for analyzing qualitative data
(Braun and Clarke, 2006), as it “offers an accessible and theoretically
flexible approach to analyzing qualitative data.” Performing a TA consists of
coding sections of the qualitative data based on its content and categorizing
the codes into different themes, which in turn is used for describing and
comparing data. Two main ways of performing the TA are either by grounding it
in the content or using predetermined codes. In grounding the TA in the
content, the codes and themes emerge from the data itself instead of being
based on a theory of which codes should exist (Walker and Myrick, 2006). The
grounded codes can, in turn, be used for creating hypotheses or theories based
on the data. Using predetermined codes revolves around codes and themes
emerging from predetermined assumptions, hypotheses, and knowledge, on the
subject, and the codes and themes used within this for research based on these
preconceptions. Grounded and predetermined codes can be combined into a hybrid
process, with specific predetermined codes and themes complemented by grounded
codes to generate the finished set of codes and themes.

The workflow for our applied thematic analysis is as described in Figure 2.
The predetermined list of codes relating to the RQs, derived from our initial
understanding of UL and its security risks and defenses, are as follows:

  * •

Clustering

  * •

GAN

  * •

AutoEncoder

  * •

Training

  * •

Production

  * •

Model

  * •

Confidentiality

  * •

Privacy attacks

  * •

Integrity

  * •

Poisoning

  * •

Adversarial
Examples

  * •

Disruption

  * •

Privacy

  * •

Self-driving cars

  * •

Intrusion detection

Figure 2. Workflow for the thematic analysis of the produced papers

Outline of the stages in the thematic analysis 1) Creating framework codes, 2)
Grounded analysis on the initial list of papers, 3) Framework analysis on the
rest of papers using the grounded codes, 4) Summarize code groups and co-
occurrence.

For the analysis, we also split the coding into two sections, grounded and
predetermined coding. The grounded coding covers the initial set of papers and
is used to establish the primary set of grounded codes. Here, we code each
paper thoroughly and in-depth. The predetermined coding is performed on the
two rounds of snowballing papers and is used to cover the general content of
the papers using the established codes. For each paper, the codes relating to
the RQs are guaranteed to be coded in this process. While this produces less
detail because of the scope of the paper focusing on the big picture of
robustness in UL, we consider this a worthy tradeoff to speed up the process
of coding all the papers.

After the thematic analysis, we extract the data to answer each research
question using the relevant codes. Additionally, we also summarize the co-
occurrence of all the code types, i.e., attack types, target models, attack
goals, attack knowledge metrics, target types, and defenses. Based on a
collection of existing taxonomies and models found in (Liu et al., 2018;
Martins et al., 2020; Guan et al., 2018; Zhang et al., 2020) describing
adversarial attacks against machine learning, as well as this review, we
construct a general model of the attributes that are associated with the
attack in Figure 3. The model follows the seven coded groups from the thematic
analysis.

The model could outline the aspects of adversarial attacks that should be
considered in any research into adversarial attacks on ML. The replication
package of the study is publicly available at
https://github.com/mathialm/Adversarial-Robustness-in-Unsupervied-Machine-
Learning-A-Systematic-Review.

Figure 3. A 3-part model describing the relationship between the different
aspects of an adversarial attack

Of the final 86 papers for consideration, 39 are published as conference or
journal papers, and 47 are unpublished. It can be noted that even though
several of the unpublished papers also have published versions, only the
papers found through the SLR directly are included. The years of publication
for all the papers are found in Figure 5, and the number of papers published
in different journals and conferences are found in Figure 5. Most papers on
this topic were published around 2020, with relatively few papers before 2018.
As expected, the journals and conferences on this topic are mainly from the
databases selected to be directly used, i.e., ACM, and IEEE.

Figure 4. Number of papers published in a specific year, from 2013 to 2022

Figure 5. Number of papers in Journals and Conferences, with the long name
available in Appendix A

##  5\. Results of RQ1: attacks targeting UL-based systems

Below is a summary of the different kinds of attack methods, attack knowledge,
attack goals, metrics, and attack focus, for each of the UL types. A summary
of the relations between kinds of attacks and targets is found in Figure 7,
showing that most target types have a specific attack mainly associated with
it.

Figure 6. Sankey diagram on the number of papers covering different attack
methods against different target types, showing the degree to which papers in
this field cover different attacks against different technologies

Figure 7. Distribution of the different types of UL technologies found in the
review

The type of unsupervised machine learning technologies targeted by attacks is
summarized in Figure 7, showing that most of the target types covered consist
of GAN, autoencoder, or clustering.

In the following sections, the total percentage would, in some cases, be above
100%, meaning some papers would be covered in multiple groups and counted more
than once.

Details on target types can be found in Appendix B.1; attack types in Appendix
B.2; and metrics in Appendix B.3.

###  5.1. Attacks targeting AutoEncoder-based systems

We found 25 papers (29%) that cover AutoEncoder models in the SLR.

####  5.1.1. Attack Method:

Attacks against autoencoders include adversarial examples, inference and
extraction, and poisoning, which are covered by 60%, 16%, and 16% of the
papers on autoencoders, respectively, and one paper (Tantipongpipat et al.,
2021) cover the implementation of a differentially private autoencoder-GAN
hybrid generator, aimed to protect against privacy attacks generally.

Adversarial Examples can work at two points, i.e., perturbations of the actual
input (Kim et al., 2020; Tabacof et al., 2016; Gondim-Ribeiro et al., 2018;
Sun et al., 2021; Condessa and Kolter, 2020; Willetts et al., 2019; Cemgil et
al., 2019; Barrett et al., 2022; Camuto et al., 2021; Kos et al., 2018; Kuzina
et al., 2021, 2022; Sun et al., 2020), or on the latent code passed to the
decoder (Albaseer et al., 2020; Deng and Sang, 2020; Sadeghi and Larsson,
2019; Sun et al., 2021; Kos et al., 2018; Kuzina et al., 2021). The attacks
appear to rely on where an attacker could have reasonable access to the input
data. In contexts where the entire AutoEncoder is used as a single step, e.g.,
removing noise, attacks do not focus directly on the latent code but rather
construct an attack that tricks the encoder into creating different latent
codes, changing the output. Similarly, in contexts where the adversary has
direct access to the latent code, e.g., physical transmission or generation,
the focus is to create perturbations that trick the decoder.

Inference and extraction attacks are limited to membership inference (Chen et
al., 2020b; Bernau et al., 2022; Chandrasekaran et al., 2021; Hilprecht et
al., 2019), where the adversary aims to extract information on whether the
training used a specific sample.

Poisoning attacks can be placed into three categories, depending on how and
what the adversary does. Injection attacks (Sun et al., 2021; Nkashama et al.,
2022) aims to disrupt a model by introducing new samples into the training
set, manipulation attacks (Ding et al., 2019; Sun et al., 2021; Creswell et
al., 2017) changes existing samples, and logic corruption (Salem et al., 2020;
Sun et al., 2021) aims to introduce a backdoor into the model, which can later
be abused.

####  5.1.2. Attack Knowledge:

We separate an adversary’s knowledge into white box, grey box, and black box,
which are covered by 60%, 40%, and 28% of the papers on autoencoders,
respectively.

White box enable the attacker to craft their attack specifically to the model
to enable their objective. For certain attacks like membership inference (Chen
et al., 2020b; Sun et al., 2021), adversarial examples (Tabacof et al., 2016;
Gondim-Ribeiro et al., 2018; Condessa and Kolter, 2020; Cemgil et al., 2019;
Kos et al., 2018; Kuzina et al., 2021, 2022), poisoning (Creswell et al.,
2017; Nkashama et al., 2022; Salem et al., 2020), and PGD (Cemgil et al.,
2019), white box knowledge enables a more optimized attack. In a physical
system, knowledge of the AutoEncoder enables the adversary to perform a
general attack (Albaseer et al., 2020; Sadeghi and Larsson, 2019).
Additionally, white box attacks appear easier and more successful than
scenarios of less knowledge (Chen et al., 2020b; Albaseer et al., 2020; Kim et
al., 2020; Sadeghi and Larsson, 2019).

Grey box attacks mean the adversary has incomplete knowledge of the
AutoEncoder and has to make a sub-optimal attack. One such knowledge is the
mapping between input and output values, whether the input is the latent code
(Chen et al., 2020b; Sun et al., 2021; Camuto et al., 2021) or the actual data
(Ding et al., 2019; Sun et al., 2021; Willetts et al., 2019; Barrett et al.,
2022; Camuto et al., 2021; Sun et al., 2020). Other attacks, like adversarial
examples (Deng and Sang, 2020), create substitute networks from the available
information to craft universal perturbations, enabling attacks against any
AutoEncoder. For attacks like membership inference (Chandrasekaran et al.,
2021; Bernau et al., 2022), the adversary utilizes available training and test
data to infer whether the training used specific samples. Grey box attacks
also appear more successful than a black box and less than white box attacks
(Chen et al., 2020b).

Black box attacks rely on the known vulnerabilities of AutoEncoders and make
assumptions on the model and data to create the attack. For generative
AutoEncoders like VAE, the adversary would only have direct access to the
generated samples (Chen et al., 2020b; Sun et al., 2021; Kuzina et al., 2021).
In other systems like physical communication (Albaseer et al., 2020),
adversarial examples (Kim et al., 2020; Sadeghi and Larsson, 2019), and
membership inference (Hilprecht et al., 2019), the adversary uses an existing
model as a substitute for the actual model, which enables the adversary to
turn a black box attack into a kind of white box attack against the target
AutoEncoder.

One paper (Tantipongpipat et al., 2021) focuses on implementing differential
privacy, which is commonly used for membership inference attacks and would be
most helpful in the context of limited knowledge.

####  5.1.3. Attack Goal:

We separate an adversary’s goal into integrity, privacy, and availability,
which are covered by 68%, 20%, and 12% of the papers on autoencoders,
respectively.

Integrity attacks aim to introduce new functionality, change existing
functionality, or avoid the existing functionality of an AutoEncoder. Attacks
often focus on maximization of distortion of the model’s output, on all, or
subsets, of the input data (Gondim-Ribeiro et al., 2018; Sun et al., 2021).
For attacks like adversarial examples (Kim et al., 2020; Tabacof et al., 2016;
Condessa and Kolter, 2020; Creswell et al., 2017; Willetts et al., 2019;
Cemgil et al., 2019; Barrett et al., 2022; Camuto et al., 2021; Kos et al.,
2018; Kuzina et al., 2021, 2022; Sun et al., 2020), the adversary aim to force
a change in the latent representation of the autoencoder, in turn producing
different output from what the input data intended to produce. Similarly,
poisoning attacks focus on introducing unintended functionality into the
model, e.g., backdoors on specific input (Ding et al., 2019; Salem et al.,
2020), or prevent intended functionality of the model (Nkashama et al., 2022).

Privacy attacks have as a goal to extract some private information from the
autoencoder (Tantipongpipat et al., 2021). Membership inference (Chen et al.,
2020b; Hilprecht et al., 2019; Chandrasekaran et al., 2021; Bernau et al.,
2022) is the primary type of privacy attack against AutoEncoders, aiming to
determine if a specific sample was used in training by analyzing the output
from the model.

Availability attacks aim to prevent the intended functionality of the model by
performing the attack. In physical communication (Albaseer et al., 2020; Deng
and Sang, 2020; Sadeghi and Larsson, 2019), using adversarial examples causes
a jamming attack on the decoder, preventing the model from decoding the
encoded message.

####  5.1.4. Metrics to measure attack success:

We separate the metrics used by an adversary to determine attack success into
distance metrics, and true/false positive/negative based, which are covered by
60%, and 56% of the papers on autoencoders, respectively.

Distance metrics refer to using a mathematical formula comparing two or two
sets of samples to determine the amount of difference between them. Some
metrics, like Structured Similarity Index Measure (SSIM), Peak Signal-to-Noise
Ratio (PSNR), Mean Square Deviation (MSD), and Root Mean Square Deviation
(RMSD) are used for comparing whether there is a noticeable difference between
malicious and clean samples (Chen et al., 2020b; Tabacof et al., 2016;
Hilprecht et al., 2019; Gondim-Ribeiro et al., 2018; Ding et al., 2019;
Willetts et al., 2019; Kos et al., 2018; Kuzina et al., 2021, 2022; Salem et
al., 2020), with Evidence Lower BOund (ELBO) (Condessa and Kolter, 2020)
denoting the performance of the model in regards to the quality. Area Under
Distortion-Distortion Curve (AUDDC) is used to measure how well an attack can
influence the output of the model (Sun et al., 2020). Other metrics, like
$L_{2}$, are used as a threshold of perturbations in the input (Sun et al.,
2021; Creswell et al., 2017; Barrett et al., 2022; Kos et al., 2018).

True/False Positive/Negative based metrics are used to determine the rate of
success and failure by the attackers and determine how many measured instances
are successes (true positive and true negative) and how many are failures
(false positive and false negative). One way of using these metrics is to
measure the performance of the attacked system with and without the influence
of the attacker, e.g., detection systems (Nkashama et al., 2022), classifiers
using output from UL systems (Albaseer et al., 2020; Kim et al., 2020; Gondim-
Ribeiro et al., 2018; Kuzina et al., 2022), and physical communication
(Albaseer et al., 2020; Deng and Sang, 2020; Sadeghi and Larsson, 2019). In
adversarial examples, success rate is used for determining if the attack
works, both in a targeted and untargeted manner (Kos et al., 2018). For other
attacks like membership inference (Chen et al., 2020b; Hilprecht et al., 2019;
Chandrasekaran et al., 2021; Cemgil et al., 2019; Bernau et al., 2022), the
success is measured by how well their attack can differentiate if a sample was
used for training by the autoencoder.

Additionally, two papers do not contribute with metrics to determine attack
success, as they cover implementation of differential privacy (Tantipongpipat
et al., 2021), or an analysis on adversarial robustness to adversarial
examples (Camuto et al., 2021).

####  5.1.5. Attack focus:

We separate where the adversary focuses into production and training, which
are covered by 92%, and 12% of the papers on autoencoders, respectively.

Production models exist within their respective working environments, exposing
them somewhat to the outside world. Privacy attacks like membership inference
(Chen et al., 2020b; Hilprecht et al., 2019; Chandrasekaran et al., 2021;
Bernau et al., 2022) mainly only focus on production models, as the data
pursued are no longer directly used by the model. Defenses like differential
privacy (Tantipongpipat et al., 2021) also would only be applicable for
protecting a production model. Adversarial examples (Albaseer et al., 2020;
Deng and Sang, 2020; Kim et al., 2020; Sadeghi and Larsson, 2019; Tabacof et
al., 2016; Gondim-Ribeiro et al., 2018; Sun et al., 2021; Condessa and Kolter,
2020; Creswell et al., 2017; Willetts et al., 2019; Cemgil et al., 2019; Kos
et al., 2018; Kuzina et al., 2021, 2022; Sun et al., 2020) is another type of
attack mainly used on production models, where the adversary somehow has
direct access to the input to the model, and can introduce perturbations to
the input samples. Some papers cover generalized increases of autoencoder
robustness to perturbations (Barrett et al., 2022; Camuto et al., 2021), while
others cover uses of differential privacy to enable privacy guarantees on the
training data (Tantipongpipat et al., 2021; Bernau et al., 2022).

Training models are models in the context of training for a specific purpose.
Poisoning attacks (Ding et al., 2019; Creswell et al., 2017; Nkashama et al.,
2022; Salem et al., 2020) is the only attack targeting autoencoder in training
by modifying the training data itself to modify the training of the model in a
specific direction.

Summary of attacks on AutoEncoders: For autoencoders, the main source of
attacks stems from adversarial examples, being usable against both
discriminatory and generative models, often in a limited knowledge scenario in
order to disrupt the output from the model. There are also attacks focusing on
extracting information about the training data, which often are performed in
minimum knowledge scenarios. Poisoning attacks are also used to disrupt or
introduce new functionality to the model. An adversary targeting autoencoders
also often relies on distance metrics to determine the difference between
clean and malicious data in order to avoid simple detection techniques, while
true/false positive/negative metrics are used for determining how successful
the attack is.

###  5.2. Attacks targeting UL supported classification

We found 14 (16%) papers that cover UL-supported classification. However, not
every paper covered focuses on the adversarial robustness of the classifier
itself, and instead focuses on the use of classifiers to increase robustness
(Liu and Hsieh, 2019), or the use of classifiers as a baseline for determining
whether attacks were successful (Banerjee et al., 2018; Zhang et al., 2019;
Rajotte and Ng, 2020; Oleszkiewicz et al., 2018; Kunar et al., 2021; Bernau et
al., 2022; Chen et al., 2022; Gupta et al., 2020; Kos et al., 2018). In other
words, only four (5%) papers cover the adversarial robustness of classifiers
in conjunction with unsupervised learning.

####  5.2.1. Attack Method:

Attacks against UL supported classification include poisoning and adversarial
examples, which are covered by 75%, and 25% of the papers on UL supported
classification, respectively.

Poisoning attacks (Liu et al., 2019b; Franci et al., 2022) in the classifier’s
training are particularly effective in disrupting its performance. Similarly,
poisoning attacks are also effective at disrupting the performance of a
classifier based on domain adaption (Wu and He, 2021) by affecting the
training of the unsupervised model.

Adversarial examples against UL-supported classifiers means that the adversary
crafts perturbations, which affect the UL system to the degree that the
classification is also affected. An AutoEncoder is sometimes used before a
classifier to denoise the input before the classifier. Attacks on this often
consist of adversarial examples, constructed to disrupt the classifier
performance (Kim et al., 2020).

####  5.2.2. Attack Knowledge:

We separate an adversary’s knowledge into white box and black box, which are
covered by 75% and 75% of the papers on UL supported classification,
respectively.

White box attacks, in the use of AutoEncoder, corresponds to improved accuracy
under adversarial examples (Kim et al., 2020). In semi-supervised (Liu et al.,
2019b) and domain adaption (Wu and He, 2021) scenarios, poisoning with white-
box knowledge can significantly reduce the classifier performance.

Black box attacks against UL supported classification with an AutoEncoder has
a decreased performance with adversarial examples (Kim et al., 2020).
Additionally, semi-supervised (Liu et al., 2019b; Franci et al., 2022) and
black box poisoning attacks appear to be marginally worse than the white box,
but are still able to significantly reduce classifier performance.

####  5.2.3. Attack Goal:

An adversary’s goal against UL supported classification is integrity in 100%
of the papers.

Integrity attacks like poisoning against domain adaption (Wu and He, 2021) and
semi-supervised models (Liu et al., 2019b; Franci et al., 2022) can construct
poisoned samples that decrease the performance of the resulting classifier,
while adversarial examples (Kim et al., 2020) can manipulate the
classification results by perturbing the input.

####  5.2.4. Metrics to measure attack success:

We separate the metrics used by an adversary to determine attack success into
distance metrics, and true/false positive/negative based, which is covered by
50% and 75% of the papers on UL supported classification, respectively.

Distance metrics have several different ways of measurement. One measurement
of success is the degree to which the attacker can influence the results of
the classifier through an unsupervised model, like with semi-supervised (Liu
et al., 2019b), where the RMSE of the classification is used to determine
success. When joined with an autoencoder, classification accuracy is used to
determine the success of an attack (Kim et al., 2020). Another measurement is
the degree to which the attacker perturbs the input, as unrestricted
perturbation would be visible to statistical analysis or human evaluation. For
this, the $L_{2}$ norm (Liu et al., 2019b) can be used to determine
perturbations in a poisoning attack.

True/False Positive/Negative-based metrics, like how well the classifier can
function under attack, is one way of measuring attack success, e.g.,
adversarial examples (Kim et al., 2020) and poisoning (Franci et al., 2022; Wu
and He, 2021) both utilizing the accuracy of the classifier.

####  5.2.5. Attack focus:

Where the adversary focuses their attack is separated into training and
production, which are covered by 75% and 25% of the papers on UL supported
classification, respectively.

Training systems have adversaries that focus on disrupting classifier
functionality by performing a poisoning attack on semi-supervision (Liu et
al., 2019b; Franci et al., 2022) and domain adaption (Wu and He, 2021) models.

Production systems have adversaries where the focus is on disrupting the
functionality of a classifier in a real-world environment, specifically using
an autoencoder before the classifier itself (Kim et al., 2020), as it is
targeted by adversarial examples.

Summary of attacks on UL supported classification: Attacks on UL supported
classification are mainly adversarial examples, where the adversary perturbs
data to influence the UL part, in turn influencing the classification; and
poisoning, where the adversary introduces poisoned samples into the training
of domain adaption which influence the performance of the resulting
classifier. Attacks appear to be most successful in a high-knowledge scenario.
The measurements of success mainly focus on the classifier’s performance,
e.g., accuracy, with distance metrics used to measure how much the samples are
perturbed to avoid simple detection.

###  5.3. Attacks targeting clustering

We found ten papers (12%) covering unsupervised clustering.

####  5.3.1. Attack Method:

Attacks against clustering include poisoning, evasion, and adversarial
examples, which are covered by 60%, 50%, and 10% of the papers on clustering,
respectively.

Poisoning attacks work by inserting crafted samples that compromise the
cluster’s functionality. One such attack is the bridging attack (Biggio et
al., 2013, 2014a, 2014b; Sabottke et al., 2019), where the adversary crafts
samples that ”bridges” gaps between clusters, in turn altering the resulting
clusters. Other approaches involve manipulating nodes and edges of graph-based
clustering (Chen et al., 2017) to disrupt the system or to alter existing
samples from the training set according to a specified maximum total
perturbation (Cinà et al., 2022).

Evasion attacks consist of crafting samples with features overlapping with the
actual sample set to conceal the sample and avoid detection without
significantly altering the clustering functionality (Banerjee et al., 2018;
Biggio et al., 2013, 2014a, 2014b; Anindya and Kantarcioglu, 2018).

Adversarial examples work by optimizing the perturbation relating to the
maximization of the clustering results (Yang et al., 2020).

One paper (Rösner and Schmidt, 2018) covers the construction of a privacy
constraint enforced in training to prevent private sample information from
being inferred to protect against privacy attacks.

####  5.3.2. Attack Knowledge:

We separate an adversary’s knowledge into white box, grey box, and black box,
which are covered by 50%, 50%, and 10% of the papers on clustering,
respectively.

White box attacks can optimize poisoning (Biggio et al., 2013, 2014a, 2014b;
Sabottke et al., 2019; Chen et al., 2017) attacks, manipulate the resulting
cluster, and optimize obfuscation (Biggio et al., 2013) attacks by crafting
adversarial samples to hide samples in existing clusters.

Grey box means adversaries would have access to a limited data mapping with
its resulting cluster. Bridging attacks (Biggio et al., 2013) can be used to
gradually discover samples that disrupt the function of the resulting cluster,
or for evasion (Anindya and Kantarcioglu, 2018) and adversarial examples (Yang
et al., 2020), use the mapping between samples and resulting clusters to
optimize perturbation needed to avoid detection by the clustering. Some
methods of poisoning attacks (Cinà et al., 2022) rely on constructing a
general perturbation based on the mapping between samples and resulting
clusters by using a genetic algorithm to discover optimizations iteratively.
Other limited knowledge attacks rely on using approximate models based on
open-sourced data and performing white box attacks against these approximate
models to construct their attack (Chen et al., 2017).

Black box attacks include adversaries who can employ an evasion attack
(Banerjee et al., 2018) by slowly enforcing drift in the clustering towards a
specific point, which does not require any information of the model.

One paper (Rösner and Schmidt, 2018) enforces privacy constraints in training
and would likely be most applicable in limited knowledge scenarios.

####  5.3.3. Attack Goal:

We separate an adversary’s goal into integrity, privacy, and availability,
which are covered by 68%, 20%, and 12% of the papers on clustering,
respectively.

Availability attacks work by preventing or disrupting the general
functionality of the clustering. Poisoning attacks (Biggio et al., 2013,
2014a, 2014b; Cinà et al., 2022; Sabottke et al., 2019; Chen et al., 2017)
work by introducing specific samples to the training to influence the shape of
the resulting clusters, to the degree that prevents the clustering from
working as intended.

Integrity attacks focus on influencing the clustering results regarding
specific samples. Evasion attacks aim to hide specific samples in existing
clusters by overlapping their features with those of actual samples (Banerjee
et al., 2018; Biggio et al., 2013) or by slowly introducing modified samples
to drift the clustering in a specific direction (Anindya and Kantarcioglu,
2018). Adversarial examples (Yang et al., 2020) aim to modify samples in a way
that results in the misclustering of samples.

Privacy attacks cover theoretical constraints on clusters to ensure the
privacy of the samples used to train the model (Rösner and Schmidt, 2018).

####  5.3.4. Metrics to measure attack success:

We separate the metrics used by an adversary to determine attack success into
true/false positive/negative, distance metrics, number of clusters, and
objective function, which are covered by 50%, 40%, 40%, and 30% of the papers
on clustering, respectively.

True/false positive/negative metrics, like accuracy used in detection
(Banerjee et al., 2018; Anindya and Kantarcioglu, 2018) to determine its
performance against evasion attacks. In poisoning attacks, F-measure (Biggio
et al., 2014b) and success rate (Chen et al., 2017) are used for determining
the quality of the clusters produced under attack. In adversarial examples
(Yang et al., 2020), accuracy measures how well the clusters fit the labels.

Distance metrics are used in poisoning attacks (Biggio et al., 2013; Cinà et
al., 2022) to craft samples to avoid simple detection. Metrics like $L_{0}$,
$L_{2}$, and $L_{\infty}$ norms are used to limit the number of perturbations
added to the samples. In contrast, others use a feature modification threshold
(Anindya and Kantarcioglu, 2018) to limit the amount any single feature can be
modified. For adversarial examples, Normalized Mutual Information (NMI) is
used to compare actual and predicted cluster labels, and distortion measures
the perturbation of the sample (Yang et al., 2020).

Number of clusters is used to compare the performance of attacks and their
capability of reducing or increasing the number of clusters the model
produces. Poisoning attacks (Biggio et al., 2013, 2014a, 2014b; Sabottke et
al., 2019) use this metric to determine how well the attack works.

Objective function based metrics are used in (Biggio et al., 2013, 2014a,
2014b) and are crafted for the specific attack and model.

One paper (Rösner and Schmidt, 2018) does not describe concrete attacks and,
as such, does not outline any metrics for the success of an attack.

####  5.3.5. Attack focus:

We separate the adversary focus into production and training, which are
covered by 50%, and 50% of the papers on clustering, respectively.

Production systems are targeted by attacks like evasion (Banerjee et al.,
2018; Biggio et al., 2013; Anindya and Kantarcioglu, 2018) and adversarial
examples (Yang et al., 2020), where the aim is to reduce the functionality of
specific samples while keeping the general clustering functionality intact,
while other attacks like targeted noise injection and small community attacks
(Chen et al., 2017) aim to disrupt the functionality of existing graph-based
clusters. Additionally, papers, like (Rösner and Schmidt, 2018), aim to ensure
clusters’ privacy guarantees to prevent privacy attacks, such as membership
inference.

Training systems are targeted by poisoning attacks (Biggio et al., 2013,
2014a, 2014b; Cinà et al., 2022; Sabottke et al., 2019), modifying or
injecting malicious samples to the training set, which can significantly
reduce the functionality of the clustering systems.

Summary of attacks on clustering: Poisoning attacks on clustering involve
introducing samples to disrupt the cluster functionality, merge, or increase
the number of clusters found. Evasion attacks work by hiding specific samples
within clusters to avoid detection without altering the cluster’s
functionality. Most attacks on clustering rely on increased knowledge, as the
adversary has a hard time accomplishing their goals without knowing the data
samples used for training. True/False Positive/Negative metrics are used by
the adversary to determine how good the resulting cluster is, measuring how
well the cluster fits the labels. In contrast, distance metrics limit the
amount of perturbation a sample can have. The number of clusters is also used
to determine the success of the attacks, particularly when the attack’s goal
revolves around changing the cluster.

###  5.4. Attacks targeting GAN

We found 42 papers (48%) covering GAN models.

####  5.4.1. Attack Method:

Attacks against GAN include inference and extraction, poisoning, adversarial
examples, and evasion, which are covered by 33%, 14%, 10%, and 2% of the
papers on GAN, respectively. Additionally, 45% of the papers cover
implementations of privacy measures to prevent privacy attacks without
specifying concrete attacks. These include implementations of differential
privacy for GANs in order to facilitate sharing models and datasets without
risk of privacy violations (Xie et al., 2018; Zhang et al., 2018; Beaulieu-
Jones et al., 2019; Jordon et al., 2018; Xu et al., 2019; Chen et al., 2020a;
Zhang et al., 2021; Rajotte and Ng, 2020; Kossen et al., 2022; Tantipongpipat
et al., 2021; Fan, 2020; Long et al., 2021; Badu-Marfo et al., 2020; Chen et
al., 2022; Gupta et al., 2020; Liu et al., 2019a; Torfi et al., 2022),
context-dependent implementations of privacy guarantees to enable private
sharing (Oleszkiewicz et al., 2018), and a survey of privacy challenges in
GANs (Tinsley et al., 2021).

Inference and extraction attacks, like membership inference attacks, are
methods of attacks to determine whether specific samples were used during the
training of the GAN model (Frigerio et al., 2019; Lin et al., 2021). Methods
for this include using distance metrics between the sample and generated
samples (Chen et al., 2020b; Hilprecht et al., 2019; Mukherjee et al., 2021;
Sun et al., 2021; Hassanzadeh and Tillman, 2022), using discriminator
confidence (Hayes et al., 2017; Sun et al., 2021; Hassanzadeh and Tillman,
2022), or training a classifier (Lin et al., 2020; Sun et al., 2021; Kunar et
al., 2021; Zhou et al., 2021), in order to determine whether the training used
a sample. Other methods determine membership by controlling the latent code of
the generator and determining membership based on the reconstruction error of
the generator when trying to generate specific samples (Sum Liu et al., 2018).
Attribute inference is another way of extracting private attributes from GANs
by looking at publicly known features, collecting generated samples, and using
regression to predict the private features (Sun et al., 2021; Kunar et al.,
2021). Model extraction is a method for cloning an existing model based on the
generated samples of a target GAN, like accuracy extraction (Sun et al., 2021;
Hu and Pang, 2021), where the adversary measures the difference in the
distribution of the cloned and actual model; and fidelity extraction (Sun et
al., 2021; Hu and Pang, 2021) compares the distribution of the cloned model to
the actual training data set. The cloned model can subsequently be used as the
substitute model in a white-box membership inference attack (Hayes et al.,
2017; Hu et al., 2021).

Poisoning attacks is, among other, used against federated GAN training, where
the adversary can contribute to the global model as one or multiple
participant(s) and subsequently modify the information that it contributes,
and affect the performance of the global model (Zhang et al., 2019; Jin and
Li, 2022). Other methods modify the training data directly, either by
injecting new samples (Sun et al., 2021), modifying existing samples to
disrupt the model (Sun et al., 2021), or modifying samples to introduce new
by-products in the generated samples (Ding et al., 2019; Rawat et al., 2021;
Sun et al., 2021; Jin and Li, 2022; Salem et al., 2020).

Adversarial examples attack relies on access to the latent code used by the
generator and perturbing some input into it to generate a completely different
sample from what would have been generated without the perturbation.
Adversarial examples can be used during training for adversarial training (Liu
and Hsieh, 2019; Sajeeda and Hossain, 2022), increasing the robustness of the
model. Other papers focus on implementing changes to how the GAN works in
order to be theoretically robust against adversarial examples (Chrysos et al.,
2020). Additionally, metrics commonly used to determine sample quality, e.g.,
Fréchet Inception Distance (FID) and Inseption Score (IS), are also explored
and are determined to be insufficient in discovering adversarial examples
(Alfarra et al., 2022).

Evasion attacks focus on finding minimal perturbations on the latent codes,
which produce unsatisfactory samples, compared to what is produced by the
unperturbed latent code (Sun et al., 2021), in order to generate
unsatisfactory samples, but still following the statistical distribution of
the training samples.

####  5.4.2. Attack Knowledge:

We separate an adversary’s knowledge into white box, grey box, and black box,
which are covered by 26%, 26%, 26% of the papers on GAN, respectively. In
contrast, 45% of the papers cover differential privacy implementations (Xie et
al., 2018; Zhang et al., 2018; Beaulieu-Jones et al., 2019; Jordon et al.,
2018; Xu et al., 2019; Chen et al., 2020a; Zhang et al., 2021; Rajotte and Ng,
2020; Kossen et al., 2022; Tantipongpipat et al., 2021; Fan, 2020; Long et
al., 2021; Badu-Marfo et al., 2020; Chen et al., 2022; Gupta et al., 2020; Liu
et al., 2019a; Torfi et al., 2022), other privacy guarantees (Oleszkiewicz et
al., 2018), and general privacy challenges (Tinsley et al., 2021), and would
likely be applicable in scenarios of limited knowledge, where the adversary
does not have access to training data, i.e., black box or grey box.

White box attacks, like membership inference (Chen et al., 2020b; Hayes et
al., 2017; Hilprecht et al., 2019; Sun et al., 2021), fidelity extraction (Hu
and Pang, 2021), and poisoning (Ding et al., 2019; Sun et al., 2021; Salem et
al., 2020), enable full knowledge of the parameters of the models which allow
the attacker to fine-tune the attack towards the specific model. Additionally,
for membership inference attacks, full knowledge allows for querying the
discriminator to extract information on the training data (Frigerio et al.,
2019; Hayes et al., 2017; Mukherjee et al., 2021; Sun et al., 2021;
Hassanzadeh and Tillman, 2022; Hu and Pang, 2021; Sum Liu et al., 2018).
Against GANs, the access to internal information on the model allows for more
successful attacks compared to more restricted knowledge scenarios (Chen et
al., 2020b; Hayes et al., 2017; Hilprecht et al., 2019; Hu and Pang, 2021).

Grey box attacks, e.g., membership inference (Chen et al., 2020b), mean that
adversaries know the input-output mapping of the model, i.e., the latent code
and the generated samples, but not the internal parameters of the
discriminator or generator models. Other membership inference attacks (Hayes
et al., 2017; Sun et al., 2021; Zhou et al., 2021), as well as fidelity
extraction attacks (Sun et al., 2021; Hu and Pang, 2021), require limited
knowledge of the training and test data, representing potential public
databases that an adversary would have access to. In comparison, poisoning
(Zhang et al., 2019; Jin and Li, 2022) attacks on federated GAN allow for
knowing the training model’s structure but not the training data of the other
participants. Other poisoning (Rawat et al., 2021; Sun et al., 2021) attacks,
evasion (Sun et al., 2021) attacks, adversarial examples (Liu and Hsieh, 2019;
Chrysos et al., 2020; Sajeeda and Hossain, 2022; Alfarra et al., 2022), and
property inference attacks (Zhou et al., 2021) instead rely entirely on the
ability of the adversary to control the input to the model, i.e., training
data or latent code. In general, the more knowledge an adversary has, the
better performance of the attack (Chen et al., 2020b; Hu and Pang, 2021; Zhou
et al., 2021).

Black box attacks, like in the case of membership inference, mean that the
adversary only has access to the generated samples from the model, where
attacks rely on inferring if samples are part of the training based on
distance calculations between the sample and the generated set (Chen et al.,
2020b; Sun et al., 2021), or by training classifiers on the generated set (Lin
et al., 2020; Hu et al., 2021; Zhou et al., 2021). Other approaches for
membership inference (Kunar et al., 2021; Lin et al., 2021; Zhou et al., 2021;
Hayes et al., 2017; Hilprecht et al., 2019) and model extraction (Sun et al.,
2021; Hu and Pang, 2021) attacks rely on recreating a substitute model and
perform white-box attacks on this model, later used for an attack on the
original model.

####  5.4.3. Attack Goal:

We separate an adversary’s goal into privacy and integrity, which are covered
by 79%, and 24% of the papers on GAN, respectively.

Privacy attacks, like membership inference (Chen et al., 2020b; Lin et al.,
2020; Frigerio et al., 2019; Hayes et al., 2017; Hilprecht et al., 2019;
Mukherjee et al., 2021; Sun et al., 2021; Hassanzadeh and Tillman, 2022; Hu et
al., 2021; Kunar et al., 2021; Lin et al., 2021; Zhou et al., 2021; Sum Liu et
al., 2018), have a goal to determine whether the GAN model used a specific
sample for training, often measured by the success rate (Lin et al., 2020),
accuracy (Frigerio et al., 2019; Hayes et al., 2017; Hilprecht et al., 2019;
Mukherjee et al., 2021), or Area Under the Curve of the Receiver Operator
Characteristic curve (AUCROC) (Chen et al., 2020b). Other attacks like
attribute inference (Sun et al., 2021), and model inference (Sun et al., 2021;
Hu and Pang, 2021) attacks extract data distribution, for the former, and
model information, for the latter, using metrics like accuracy (Hu and Pang,
2021) to determine the success of the attack. Many papers also cover the
application of differential privacy (Xie et al., 2018; Zhang et al., 2018;
Beaulieu-Jones et al., 2019; Jordon et al., 2018; Xu et al., 2019; Chen et
al., 2020a; Zhang et al., 2021; Rajotte and Ng, 2020; Kossen et al., 2022;
Tantipongpipat et al., 2021; Fan, 2020; Long et al., 2021; Badu-Marfo et al.,
2020; Chen et al., 2022; Gupta et al., 2020; Liu et al., 2019a; Torfi et al.,
2022), or other privacy guarantees (Oleszkiewicz et al., 2018) in training to
ensure the privacy of the training samples, as a way to prevent private data
from being inferred from released datasets or models. One paper also covers
the privacy challenges in GANs (Tinsley et al., 2021).

Integrity attacks like poisoning (Zhang et al., 2019; Ding et al., 2019; Rawat
et al., 2021; Sun et al., 2021; Jin and Li, 2022; Salem et al., 2020) focus on
decreasing the quality of generated samples or altering the mapping between
latent code and generated samples, while evasion attacks (Sun et al., 2021)
and adversarial examples (Liu and Hsieh, 2019; Chrysos et al., 2020; Sajeeda
and Hossain, 2022) force the model to output unsatisfactory output with small
perturbations in the input.

####  5.4.4. Metrics to measure attack success:

We separate the metrics used by an adversary to determine attack success into
true/false positive/negative, and distance metrics based, which are covered by
38%, and 19% of the papers on GAN, respectively, with 45% of the papers
covering protection against general privacy attacks, like, differential
privacy (Xie et al., 2018; Zhang et al., 2018; Beaulieu-Jones et al., 2019;
Jordon et al., 2018; Xu et al., 2019; Chen et al., 2020a; Zhang et al., 2021;
Rajotte and Ng, 2020; Kossen et al., 2022; Tantipongpipat et al., 2021; Fan,
2020; Long et al., 2021; Badu-Marfo et al., 2020; Chen et al., 2022; Gupta et
al., 2020; Liu et al., 2019a; Torfi et al., 2022), context-dependent
implementations of privacy guarantees (Oleszkiewicz et al., 2018), and general
explorations of privacy challenges in GANs (Tinsley et al., 2021), and do not
outline any metrics for measuring attack success.

True/false positive/negative metrics in membership inference attacks measure
how well the adversary can determine if the query sample belongs to the
training set, measured by AUCROC (Chen et al., 2020b; Kunar et al., 2021; Zhou
et al., 2021; Sum Liu et al., 2018), Area Under Precision-Recall Curve (AUPRC)
(Hu et al., 2021; Kunar et al., 2021), F1-score (Kunar et al., 2021), accuracy
(Frigerio et al., 2019; Hayes et al., 2017; Hilprecht et al., 2019; Mukherjee
et al., 2021; Hassanzadeh and Tillman, 2022; Hu and Pang, 2021; Kunar et al.,
2021), precision (Hu et al., 2021), recall (Hu et al., 2021), true positive
(Frigerio et al., 2019; Lin et al., 2021; Sum Liu et al., 2018), false
positive (Frigerio et al., 2019; Lin et al., 2021; Sum Liu et al., 2018), and
success rate (Lin et al., 2020; Sun et al., 2021). For poisoning attacks,
accuracy (Zhang et al., 2019) is used to determine both if the poisoned task
and the main task of the model perform well. For adversarial examples,
accuracy of the discriminator determines how well the attack can influence the
generated samples (Liu and Hsieh, 2019).

Distance metrics in membership inference attacks, consists of reconstruction
distance (Sun et al., 2021) and $\epsilon$-ball distance (Sun et al., 2021),
which determine the distance between the query sample, and the set of
generated samples, to determine if the training used the sample. In poisoning
attacks, SSIM (Ding et al., 2019), PSNR (Ding et al., 2019), IS (Rawat et al.,
2021; Jin and Li, 2022), FID (Rawat et al., 2021; Jin and Li, 2022; Salem et
al., 2020), and Kernel Inception Distance (KID) (Jin and Li, 2022) are used
for evaluating the performance of the model in non-adversarial and adversarial
scenarios. With adversarial examples, SSIM and FID are used for evaluating the
performance of the generated samples under adversarial noise (Chrysos et al.,
2020). Additionally, distance metrics measure the perturbations applied to
data in evasion attacks (Sun et al., 2021) and adversarial examples (Sajeeda
and Hossain, 2022). Alfarra et al. (2022) evaluate the FID and IS metrics in
terms of the metrics’ ability to discover adversarial examples, which the
metrics are not able to.

####  5.4.5. Attack focus:

We separate the adversaries’ focus into production and training, which are
covered by 88%, and 12% of the papers on GAN, respectively.

Production systems under membership inference (Chen et al., 2020b; Lin et al.,
2020; Frigerio et al., 2019; Hayes et al., 2017; Hilprecht et al., 2019;
Mukherjee et al., 2021; Sun et al., 2021; Hassanzadeh and Tillman, 2022; Hu et
al., 2021; Kunar et al., 2021; Lin et al., 2021; Zhou et al., 2021; Sum Liu et
al., 2018), attribute inference, (Sun et al., 2021; Kunar et al., 2021) and
model extraction (Sun et al., 2021; Hu and Pang, 2021) attacks rely on using
the generated samples from the model to infer information.

Training systems under poisoning attacks are mainly attacked by altering the
training data of the model (Ding et al., 2019; Rawat et al., 2021; Salem et
al., 2020). In contrast, in federated GAN works by poisoning the global model
as a participant in the federated training (Zhang et al., 2019; Jin and Li,
2022).

Summary of attacks against GANs: The primary attacks against GANs revolve
around extracting information from the model based on the generated samples
they produce, including: membership inference, where the adversary tries to
find if the training used a specific sample, attribute inference to extract
the features of the training data, and model extraction, where one clone the
model, in turn giving the adversary access to optimize attacks on the cloned
model to attack the original model subsequently. Poisoning attacks are also
used, where the adversary crafts samples to be used in training, either to
decrease the utility of the generated models or to introduce backdoors into
the generated samples. Knowledge of the model is also essential, as more
knowledge enables optimized attacks. However, membership inference attacks are
made with only knowledge of the generated samples. Distance metrics are used
for determining the model’s performance by analyzing the clean and attacked
generated samples. In contrast, true/false positive/negative determines the
degree to which the attack was successful.

###  5.5. Super Resolution

We found three papers (3%) covering super-resolution models.

####  5.5.1. Attack Method:

Attacks against super resolution only include adversarial examples.

Adversarial example attacks perturb the input to the model to alter the result
significantly (Choi et al., 2019; Castillo et al., 2021; Yue et al., 2021).
One paper also produces general perturbations for super-resolution (Choi et
al., 2019), i.e., the perturbation is independent of the model’s training.

####  5.5.2. Attack Knowledge:

The adversary’s knowledge is only covered by white box attacks.

White-box attacks revolve around knowing a mapping between input and output,
and, with limited perturbations on the input, being able to produce large
perturbations on the output (Choi et al., 2019; Castillo et al., 2021; Yue et
al., 2021). For this attack, an adversary would require access to the model to
know the perturbed output. The perturbation is iteratively fine-tuned, which
also applies to the general perturbation (Choi et al., 2019).

####  5.5.3. Attack Goal:

Integrity attacks are the only goal covered for super-resolution.

Integrity attacks, like adversarial examples, have as a goal to insert
imperceptible perturbation on the input to the model, in turn causing an
extensive deterioration of the output from the model (Choi et al., 2019;
Castillo et al., 2021; Yue et al., 2021).

####  5.5.4. Metrics to measure attack success:

distance metrics are the only metrics an adversary uses to determine attack
success for super-resolution.

Distance metrics include the difference between perturbed and unperturbed
input and output measurements. The metrics used for this include PSNR (Choi et
al., 2019; Castillo et al., 2021; Yue et al., 2021) measuring noise, SSIM
(Castillo et al., 2021; Yue et al., 2021) measuring perceptual differences,
and Learned Perceptual Image Patch Similarity (LPIPS) (Castillo et al., 2021)
measuring the difference in activation in the model on two inputs.

####  5.5.5. Target Focus:

The adversary only focuses their attack on production systems.

Production systems under adversarial examples mean an adversary makes
perturbations to the input to the model (Choi et al., 2019; Castillo et al.,
2021; Yue et al., 2021), in order to cause large perturbations in the output.

Summary of attacks against super-resolution: Against super resolution,
adversarial examples is the only attack, focusing on disrupting the model’s
output by perturbing the input, requiring complete knowledge of the model to
perform the attack. The attack focuses mainly on using distance metrics to
measure the difference between clean and adversarial input and output.

##  6\. Results of RQ2: defenses methods against attacks

A summary of the percentage of papers with defenses used by different target
types is covered in Figure 9, which shows the proportion of defense types for
each target type, e.g., differential privacy contributes 33%, 50%, and 72%, to
the total number of defenses of AutoEncoder, classification, and GAN,
respectively. Details on the different defenses can be found in Appendix B.4.

Figure 8. Distribution of the different types of defenses found in the review

Figure 9. Sankey diagram showing the relative appearance between attack types
and defenses for autoencoders

###  6.1. Defenses against attacks targeting AutoEncoder

A summary of the relationship between the attack type and defensive measure
for autoencoders is shown in Figure 9.

Adversarial training consists of using adversaries’ strategy to generate
samples to be learned by the model, making the model more robust against the
specific attack used to generate the samples. This defense has certain
drawbacks, like reducing performance on clean samples (Kim et al., 2020;
Sadeghi and Larsson, 2019) or lacking generalizability on other perturbations
(Sadeghi and Larsson, 2019).

Smoothing, also referred to as gradient masking, works by preventing
adversaries from inferring information on the internal gradients of the model
(Kim et al., 2020; Sun et al., 2021; Willetts et al., 2019; Cemgil et al.,
2019; Barrett et al., 2022; Sun et al., 2020), providing noticeable robustness
increase against adversarial examples in white-box settings (Kim et al.,
2020), and marginal robustness increase in black-box settings (Kim et al.,
2020).

Robustness constraints work by enforcing a guarantee on the lower bounds of
when perturbations in the input would affect the output. In particular, this
is targeted as a general defense against adversarial examples (Condessa and
Kolter, 2020; Camuto et al., 2021).

Markov Chain Monte Carlo (MCMC) is employed on the latent code from the
encoder, where the variables are updated before being used by the decoder
(Kuzina et al., 2022). This method provides robustness against adversarial
examples and higher quality data than autoencoder under attack.

Differential privacy ensures that, under certain constraints, an adversary
cannot determine if a specific sample was used in training, providing slight
(Chen et al., 2020b; Chandrasekaran et al., 2021) to large (Bernau et al.,
2022) amount of privacy protection, depending on the privacy budget (Bernau et
al., 2022). Attacks like membership inference (Chen et al., 2020b; Sun et al.,
2021; Bernau et al., 2022), attribute inference (Sun et al., 2021), and model
extraction (Sun et al., 2021) can be mitigated by using this method.
Differential privacy is also used as a way of ensuring the privacy of
synthetic data (Tantipongpipat et al., 2021). However, this method requires
more data resources and causes a decrease in the quality of the data generated
(Chen et al., 2020b; Chandrasekaran et al., 2021; Tantipongpipat et al., 2021;
Bernau et al., 2022).

Privacy Constraints is another method for ensuring the privacy of training
data when releasing synthetic data (Chandrasekaran et al., 2021). Here though,
differential privacy is implemented with a causal model, where the utility of
the generated samples is higher than non-causal differential privacy
implementations for the same privacy budget.

Dropout works on neural networks by dropping random neurons during training to
prevent the model from overfitting the data. While the method is effective
against membership inference (Hilprecht et al., 2019; Sun et al., 2021), it
also results in a quality decrease of generated samples and requires more
computer resources (Hilprecht et al., 2019; Sun et al., 2021).

Weight normalization works by periodically re-parameterizing the neural
network weights. In turn, the re-parametrizing can increase the model’s
generalizability and provide some protection against membership inference
attacks (Sun et al., 2021).

Fine pruning works in neural networks by removing specific neurons that are
not improving the performance on clean samples. The method is not, however,
proven effective for autoencoders, as ”fine-pruning does not remove the by-
product task” introduced by poisoning attacks (Ding et al., 2019; Sun et al.,
2021). Additionally, fine pruning also decreases the quality of generated
samples and requires more computing resources (Ding et al., 2019; Sun et al.,
2021).

Activation Output Clustering works by analyzing a hidden layer of a neural
network to discover different distributions between clean and adversarial
input. The defense does not work for generative models. However, the strategy
struggles to differentiate between real and adversarial input (Ding et al.,
2019; Sun et al., 2021).

Lastly, several papers do not describe defensive measures against their
respective adversarial attack, like adversarial examples (Albaseer et al.,
2020; Deng and Sang, 2020; Tabacof et al., 2016; Gondim-Ribeiro et al., 2018;
Kos et al., 2018, 2018; Kuzina et al., 2021), and poisoning (Creswell et al.,
2017; Nkashama et al., 2022).

###  6.2. Defenses against attacks targeting classification

Adversarial training for defending UL-supported classification is done by
adversarially training an autoencoder and the classifier (Kim et al., 2020),
with adversarial robustness increasing or decreasing depending on the training
set. Kim et al. (2020) conclude that adversarial robustness using an
autoencoder and adversarial training provide the same effect, differing from
the robustness provided by gradient masking.

Relabeling entails calculating the influence a specific sample would have on
the training in case of poisoning (Franci et al., 2022), and re-labeling the
sample, in turn decreasing the performance of the poisoning attack.

###  6.3. Defenses against attacks targeting clustering

A summary of the relationship between the attack type and defensive measure
for clustering is shown in Figure 11.

Game theory is proposed as a general defensive strategy for clustering. The
general strategy involves the target employing defensive measures depending on
specified conditions (Banerjee et al., 2018) or modeling the attacker-target
system as an adversarial game to determine optimal defenses (Biggio et al.,
2013) for a variety of attacks.

Adversarial training is employed with adversarial strategies like adversarial
examples (Yang et al., 2020), noise injection (Chen et al., 2017), and small
community (Chen et al., 2017), which ensures the clustering trains to be more
robust against these specific attacks. The drawback of this method, however,
is the potential reduction of the overall accuracy of the models (Chen et al.,
2017), though some results indicate that adversarial training does improve the
performance (Yang et al., 2020).

Robustness enhancement constructs feature sets for the clustering consisting
of binaries for important features and numerical values for secondary features
(Sabottke et al., 2019), preventing adversarial attacks from merging clusters.

Privacy enhancement applies privacy constraints to the clustering process for
the anonymity of samples belonging to a specific cluster to be ensured (Rösner
and Schmidt, 2018).

Lastly, several papers do not describe defensive measures against adversarial
attacks, like poisoning (Biggio et al., 2014a, b; Cinà et al., 2022) and
evasion (Biggio et al., 2013; Anindya and Kantarcioglu, 2018).

Figure 10. Sankey diagram showing the relative appearance between attack types
and defenses for clustering

Figure 11. Sankey diagram showing the relative appearance between attack types
and defenses for GANs

###  6.4. Defenses against attacks targeting GAN

A summary of the relationship between the attack type and defensive measure
for GANs is shown in Figure 11.

Differential privacy is used as a defensive method in 27 papers. The research
in (Xie et al., 2018; Zhang et al., 2018; Beaulieu-Jones et al., 2019; Jordon
et al., 2018; Xu et al., 2019; Chen et al., 2020a; Zhang et al., 2021; Rajotte
and Ng, 2020; Kossen et al., 2022; Tantipongpipat et al., 2021; Fan, 2020;
Long et al., 2021; Badu-Marfo et al., 2020; Chen et al., 2022; Gupta et al.,
2020; Liu et al., 2019a; Tinsley et al., 2021; Torfi et al., 2022) all cover
theoretical applications of differential privacy. The defense is also applied
specifically to counter membership inference in (Chen et al., 2020b; Lin et
al., 2020; Frigerio et al., 2019; Hayes et al., 2017; Mukherjee et al., 2021;
Sun et al., 2021; Hu et al., 2021; Kunar et al., 2021; Lin et al., 2021),
providing slight (Kunar et al., 2021; Chen et al., 2020b; Hu et al., 2021) to
large (Frigerio et al., 2019; Mukherjee et al., 2021; Hu et al., 2021) privacy
protection, depending on the privacy budget (Hayes et al., 2017; Mukherjee et
al., 2021; Hu et al., 2021) and training set size (Frigerio et al., 2019).
Differential privacy can also cause quality decreases in generated samples
(Hayes et al., 2017; Mukherjee et al., 2021) and cause groups of data to
become over-represented by the generator (Mukherjee et al., 2021).

Privacy Enchancement are methods for ensuring the privacy of training samples
without implementing differential privacy. In (Hassanzadeh and Tillman, 2022),
the model trains to generate samples independent of the set of samples used
for training to protect the model from membership inference attacks. In
(Oleszkiewicz et al., 2018), the model trains the discriminator to determine
whether two samples belong to the same identity, which, through the
adversarial min-max game between the generator and the discriminator, forces
the generator to produce private samples.

Weight normalization recalculates the parameters of the weight in the model in
order to improve generalizability (Sun et al., 2021). While the method can
reduce the accuracy of memberhsip inference attacks (Hayes et al., 2017; Sun
et al., 2021), it also can result in instability between the discriminator and
the generator (Hayes et al., 2017; Sun et al., 2021).

Dropout works by removing random neurons in a neural network to prevent
overfitting by the model and might be viable for GANs, as it decreases the
accuracy of membership inference attacks (Hayes et al., 2017; Hilprecht et
al., 2019), but at the same time reduces the quality of the generated samples
(Hayes et al., 2017; Hilprecht et al., 2019; Sun et al., 2021).

Data perturbation is a method to perturb the output data produced by the model
in order to protect against model extraction attacks (Sun et al., 2021; Hu and
Pang, 2021), by adding different kinds of noise, e.g., Gaussian Filtering and
JPEG compression.

Adversarial training is used by Liu and Hsieh (2019), with an additional
classifier to increase the generator’s robustness and generalizability. In
contrast, Sajeeda and Hossain (2022) find other research covering adversarial
training for the discriminator and how adversarially trained GAN can be robust
against perturbations.

Robustness enhancement is a method for forcing the generator to generate
samples within the target space, in particular samples under adversarial and
non-adversarial noise (Chrysos et al., 2020), in order to avoid lousy quality
data from being generated under noise.

Fine pruning works by removing neurons in a neural network that does not
provide good quality generated samples but did not remove the introduced by-
products from poisoning attacks and decreased the performance of the model
(Sun et al., 2021).

Activation Output Clustering analyzes one of the layers of a neural network
and clusters the values from these, with the hope of differentiating between
actual and malicious data from a poisoning attack (Ding et al., 2019).

Inspection relies on analyzing the model parameters, i.e., static model
inspection (Rawat et al., 2021); analyzing the model behavior in action, i.e.,
dynamic model inspection (Rawat et al., 2021); or analyzing the output of the
model to compare with known adversarial data (Rawat et al., 2021). The
defensive measure appears unreliable in analyzing the output, and model
inspection methods require knowledge of the adversary to be effective.

Lastly, some papers cover adversarial attacks that are not covered by any
defensive measures, like membership inference (Zhou et al., 2021; Sum Liu et
al., 2018), poisoning (Zhang et al., 2019; Jin and Li, 2022; Salem et al.,
2020), and adversarial examples (Alfarra et al., 2022).

###  6.5. Defenses against attacks targeting super-resolution:

As shown in Section 5.5, all attacks on super-resolution consists of
adversarial examples.

Data pre-processing is one such defense, covered by Choi et al. (2019), where
the data size is reduced by one pixel and subsequently resized to the original
size, resulting in an increase of the measured PSNR on the adversarial input.

Ensamble methods covered by Choi et al. (2019) use a geometric self-ensemble
method, where the original images are geometrically transformed and passed
through the model, subsequently inversely transformed and combined to create
the final upscaled image. This method increases the adversarial inputs
measured by PSNR.

Adversarial training is performed by including adversarial examples in the re-
training of the super-resolution model, which increases model
generalizability, with the model performing better against adversarial
examples (Castillo et al., 2021; Yue et al., 2021).

##  7\. Results of RQ3: Challenges to secure UL-based systems

###  7.1. Challenges to secure systems using AutoEncoders

As autoencoders are applicable for several different purposes, there are also
different challenges depending on the use. For generative autoencoders,
privacy (Chen et al., 2020b; Hilprecht et al., 2019; Chandrasekaran et al.,
2021; Bernau et al., 2022) attack is a problem, particularly when an adversary
has access to the model parameters (Chen et al., 2020b). Though there are
defenses like differential privacy (Chen et al., 2020b; Tantipongpipat et al.,
2021; Bernau et al., 2022), fine pruning (Ding et al., 2019), activation
output clustering (Ding et al., 2019), the impact on generated samples is
notable. Autoencoder systems for encoding/decoding, or noise reduction, are
also vulnerable, particularly against adversarial examples in the input
(Tabacof et al., 2016; Mukherjee et al., 2021), or latent space (Albaseer et
al., 2020; Creswell et al., 2017). Defensive measures for these kinds of
attacks on autoencoders are also not very well researched (Sadeghi and
Larsson, 2019).

###  7.2. Challenges to secure UL supported classifications

We can divide the adversarial robustness of classification into two groups.
Firstly is the use of UL systems to provide adversarial robustness of the
classifier, e.g., using an autoencoder in front of the classifier (Kim et al.,
2020). Secondly is the exploration of adversarial robustness in systems
employing UL in tandem with classification, e.g., using clustering to separate
malicious and legitimate data for classification (Banerjee et al., 2018),
training classifiers on generated samples (Zhang et al., 2019; Rajotte and Ng,
2020; Oleszkiewicz et al., 2018; Kunar et al., 2021; Bernau et al., 2022; Chen
et al., 2022; Gupta et al., 2020), or employing a classifier in addition to
discriminator in a GAN (Liu and Hsieh, 2019). In this case, while there exist
several attacks, like poisoning (Liu et al., 2019b; Zhang et al., 2019; Franci
et al., 2022; Wu and He, 2021), defensive measures in the literature is mainly
theoretical, like game theory (Banerjee et al., 2018).

###  7.3. Challenges to secure clustering

While clustering as a concept relies on finding optimal groups of samples
based on some distance metric, how the distance is calculated impacts the
vulnerabilities and adversarial robustness of clustering. It is speculated
that the specific poisoning attacks by Biggio et al. (2013) might only apply
to single-linkage hierarchical clustering, while other clustering methods
would not. Additionally, known vulnerabilities of clustering are still not
explored enough in the literature, as most attacks focus on specific
clustering methods (Biggio et al., 2013, 2014a, 2014b; Rösner and Schmidt,
2018). Defending clustering methods is also not very well explored (Biggio et
al., 2013), with methods like adversarial simulation (Biggio et al., 2013) and
general privacy constraints (Rösner and Schmidt, 2018) left as future work in
the surveyed papers.

###  7.4. Challenges to secure GAN-based systems

Adversarial robustness for GAN models is covered by various papers, with many
limitations. One of the major limitations is the compromise between GAN
utility and the implementation of defensive measures. Differential privacy
provides privacy for training samples (Chen et al., 2020b; Lin et al., 2020;
Frigerio et al., 2019; Hayes et al., 2017; Xu et al., 2019; Sun et al., 2021;
Zhang et al., 2021; Rajotte and Ng, 2020; Kossen et al., 2022; Kunar et al.,
2021; Tantipongpipat et al., 2021; Fan, 2020; Long et al., 2021; Chen et al.,
2022; Gupta et al., 2020; Liu et al., 2019a; Torfi et al., 2022); and dropout
(Hayes et al., 2017; Hilprecht et al., 2019; Sun et al., 2021) and weight
normalization (Hayes et al., 2017; Sun et al., 2021) reduce the effectiveness
of membership inference attacks. Simultaneously, the defenses increase
computational cost (Chen et al., 2020b; Frigerio et al., 2019; Hayes et al.,
2017; Sun et al., 2021; Gupta et al., 2020), lower the quality of the
generated data (Chen et al., 2020b; Lin et al., 2020; Xie et al., 2018;
Frigerio et al., 2019; Hayes et al., 2017; Hilprecht et al., 2019; Xu et al.,
2019; Sun et al., 2021; Hassanzadeh and Tillman, 2022; Hu and Pang, 2021;
Kossen et al., 2022; Oleszkiewicz et al., 2018; Kunar et al., 2021;
Tantipongpipat et al., 2021; Fan, 2020; Gupta et al., 2020; Liu et al.,
2019a), lower the stability of the model (Xie et al., 2018; Zhang et al.,
2018; Xu et al., 2019; Sun et al., 2021; Fan, 2020; Alfarra et al., 2022; Chen
et al., 2022), and is not effective for other kinds of inference attacks (Lin
et al., 2020; Hu et al., 2021). When it comes to poisoning attacks, defensive
measures that work with discriminatory models, e.g., fine pruning and
activation output clustering, do not work to prevent or detect poisoning
attacks against generative models like GAN (Ding et al., 2019). Other
solutions like output inspection (Rawat et al., 2021) rely on knowledge of the
poisoning attack and would be impractical to implement outside particular
situations. Additionally, the notion of generalization (Chen et al., 2020b;
Frigerio et al., 2019; Hayes et al., 2017; Hilprecht et al., 2019; Mukherjee
et al., 2021; Sun et al., 2021; Hassanzadeh and Tillman, 2022; Hu et al.,
2021; Kossen et al., 2022; Lin et al., 2021; Sum Liu et al., 2018) and
diversity (Zhang et al., 2018; Hayes et al., 2017; Mukherjee et al., 2021;
Zhang et al., 2021; Hu et al., 2021; Sum Liu et al., 2018) in generated data
is discussed as underlying properties of GAN that influence the effectiveness
of attacks, e.g., when the GAN produces diverse but non-generalized samples,
the model is vulnerable to membership inference (Chen et al., 2020b; Frigerio
et al., 2019; Hayes et al., 2017; Hilprecht et al., 2019; Mukherjee et al.,
2021; Sun et al., 2021; Hassanzadeh and Tillman, 2022; Hu et al., 2021; Lin et
al., 2021; Sum Liu et al., 2018). In the literature, attacks like data
injection (Sun et al., 2021) and defenses like data augmentation (Ding et al.,
2019; Sun et al., 2021) are not explored but mentioned as potential future
work. No single defensive method can defend against several attacks
simultaneously (Sajeeda and Hossain, 2022).

###  7.5. Challenges to secure super-resolution-based systems

In the papers covering adversarial robustness of super-resolution models (Choi
et al., 2019; Castillo et al., 2021; Yue et al., 2021), the papers do not
cover the limitations of the work, other than ”more advanced defense methods
can be investigated in the future work” (Choi et al., 2019) about defenses
against adversarial examples attacks.

##  8\. Discussion

###  8.1. Comparison with related work

Liu et al. (2018), Guan et al. (2018), and Zhang et al. (2020) do not outline
their method of discovering the list of papers used for their analysis, and
Martins et al. (2020) only outline their search engines and keywords. This
study follows a more SLR systematic approach, and the papers covered by this
study are, therefore, more complete and representative of the state of the
field.

Table 5 compares the papers regarding the number of papers and which defenses
were covered. Several of the defenses are covered both by the other surveys
and this one, like adversarial training, smoothing/gradient masking, and
differential privacy. However, this paper does cover a multitude of papers
related to each defense, while most of the other papers cover only one or a
few papers on the same defensive measure. The exception is Zhang et al.
(2020), which covers differential privacy with several papers. The most
notable difference between this paper, and the other papers, is that this
paper has not found any defenses using homomorphic encryption, which can be
explainable by 1) this SLR did not craft its search well enough to include
homomorphic encryption, 2) unsupervised learning does not work well with
homomorphic encryption, or 3) homomorphic encryption has not yet been widely
used as a defensive measure for unsupervised learning. Doing a surface search
on Google Scholar on homomorphic encryption with unsupervised learning does
not reveal a large number of papers on this subject, which leads us to believe
that number three is the most likely answer as to why it does not show up in
this SLR. With this, research into defensive measures for UL using homomorphic
encryption might be a viable future endeavor. Other defensive measures that
are present in the other papers but not in ours, like reject on negative
impact, defensive distillation, universal perturbation defense method, do not
appear to be applicable in an unsupervised scenario, as they are only
applicable for classification. Lastly, defenses like transferability block,
MagNet, reverse poisoning, query distribution detection, hierarchical model
stacking, alternating training, defensive perturbation, and layer separation
appear on the surface to be applicable in an unsupervised scenario, and in
future research should be evaluated to find out whether or not they work for
UL.

Table 5. Comparison between surveys on adversarial robustness in machine learning. Cross (✗) shows that the column label is not applicable, Tick (✓) shows that the column label does apply, and Asterisk ($\ast$) shows that the column label partially applies. Ref. |  Short description |  Num. papers |  Defenses Covered
---|---|---|---
(Liu et al., 2018) |  Adversarial Examples attacks against Machine Learning |  42 |  Reject on Negative Impact (Nelson et al., 2009), Adversarial Training (Goodfellow et al., 2014), Defense Distillation (Papernot et al., 2016c; Carlini and Wagner, 2016), Ensemble method (Sengupta et al., 2018; Tramèr et al., 2017; Abbasi and Gagné, 2017), Differential Privacy (Dwork, 2006; Abadi et al., 2016; Kusner et al., 2015), Homomorphic Encryption (Damgård et al., 2012; Yao et al., 2017)
(Martins et al., 2020) |  Adversarial Examples against detection techniques |  20 |  Adversarial Training (Goodfellow et al., 2014; Sankaranarayanan et al., 2018), Gradient Masking (Duddu, 2018; Papernot et al., 2016b, 2017), Defensive Distillation (Papernot et al., 2016c), Feature Squeezing (Xu et al., 2017), Transferability Block (Hosseini et al., 2017), Universal Perturbation Defense Method (Akhtar et al., 2018), MagNet (Meng and Chen, 2017)
(Guan et al., 2018) |  Impact of ML on security, and robustness of AI systems |  14 |  Reject on Negative Impact (Nelson et al., 2009), Learning Algorithm Improvement (Liu et al., 2017b; Baracaldo et al., 2017), Adversarial Retraining, Output Smoothness (Gu and Rigazio, 2014), Defensive Distillation (Papernot et al., 2016c; Carlini and Wagner, 2016), Differential Privacy (Bittner et al., 2017), Homomorphic Encryption (Damgård et al., 2012)
(Zhang et al., 2020) |  Privacy threats and mitigations within ML |  45 |  Reverse Poisoning (Orekondy et al., 2020), Query Distribution Detection (Juuti et al., 2019), Hierarchical Model Stacking (Salem et al., 2018), Alternating Training (Nasr et al., 2018), Defensive Perturbation (Jia et al., 2019; Jia and Gong, 2018), Collaborative Training (Shokri and Shmatikov, 2015; McMahan et al., 2017), Homomorphic Encryption (Rivest et al., 1978; Gentry, 2009; Bansal et al., 2011; Zhang et al., 2015; Gilad-Bachrach et al., 2016; Chabanne et al., 2017; Liu et al., 2017a) , Differential Privacy (Dwork et al., 2006; Bae et al., 2018; Shokri and Shmatikov, 2015; Abadi et al., 2016; Phan et al., 2016, 2017; Papernot et al., 2016a) , Layer Separation (Osia et al., 2020; Chi et al., 2018)
This work |  A systematic review on adversarial robustness of unsupervised machine learning |  86 |  Adversarial training (Kim et al., 2020; Liu and Hsieh, 2019; Sadeghi and Larsson, 2019; Yang et al., 2020; Chen et al., 2017; Sajeeda and Hossain, 2022; Castillo et al., 2021; Yue et al., 2021), Differential Privacy (Chen et al., 2020b; Lin et al., 2020; Xie et al., 2018; Zhang et al., 2018; Beaulieu-Jones et al., 2019; Frigerio et al., 2019; Hayes et al., 2017; Jordon et al., 2018; Xu et al., 2019; Chen et al., 2020a; Mukherjee et al., 2021; Sun et al., 2021; Zhang et al., 2021; Hu et al., 2021; Chandrasekaran et al., 2021; Rajotte and Ng, 2020; Kossen et al., 2022; Kunar et al., 2021; Lin et al., 2021; Tantipongpipat et al., 2021; Bernau et al., 2022; Fan, 2020; Long et al., 2021; Badu-Marfo et al., 2020; Chen et al., 2022; Gupta et al., 2020; Liu et al., 2019a; Tinsley et al., 2021; Torfi et al., 2022), Dropout (Hayes et al., 2017; Hilprecht et al., 2019; Sun et al., 2021), Fine Pruning (Sun et al., 2021), Smoothing (Kim et al., 2020; Sun et al., 2021; Willetts et al., 2019; Cemgil et al., 2019; Barrett et al., 2022; Sun et al., 2020), Weight Normalization (Hayes et al., 2017; Sun et al., 2021), Robustness Constraints (Condessa and Kolter, 2020), Activation Output Clustering (Ding et al., 2019; Sun et al., 2021), MCMC (Kuzina et al., 2022), Relabeling (Franci et al., 2022), Game Theory (Banerjee et al., 2018), Robustness Enhancement (Sabottke et al., 2019), Privacy Enhancement (Rösner and Schmidt, 2018), Inspection (Rawat et al., 2021), Data pre-processing (Choi et al., 2019) , Ensamble (Choi et al., 2019)

###  8.2. Implications to academia

Our study gives a structured and comprehensive summary of the state-of-the-art
adversarial robustness of unsupervised learning and how this can be used for
further research. Attack knowledge is one of the core parts of the attack, and
while specific attacks like adversarial examples and membership inference can
work with minimal knowledge, all attacks perform noticeably better with an
increase in knowledge of the system. In this regard, our paper aims at
structuring and summarizing the performance of attacks and defenses depending
on the amount of knowledge, which could be used in future research to make
comparisons easier. Specific attacks also employ substitute models trained to
mimic the actual model and perform full-knowledge attacks on these, which can
transfer to the actual model. Future research should therefore consider a
broader range of knowledge scenarios in their analysis, as a black box attack
could be turned into a white box attack using such methods.

Another critical attack aspect is the attack goal. On the one hand, our paper
shows that most attacks focus on infringing on the privacy of the target,
which in turn motivates future research into finding methods for ensuring
privacy since this is the most popular attack vector. Additionally, we believe
our findings that differential privacy impact model performance should
motivate future research into providing privacy guarantees with less impact on
performance. On the other hand, our paper also shows that integrity attacks
are effective. However, their defenses are not given as much focus in the
research, which is something we wish to motivate further research in doing.

Future research would also benefit from this paper concerning the kinds of
attacks leveraged against specific technologies. We can, for example, see that
poisoning attacks target most technologies; or that there are no privacy
attacks on clustering, which future research could use as motivation for
studying these topics.

We also believe the summaries on metrics can enable future research with both
motivations for selecting specific metrics and categorizing which metrics are
used by which papers, making comparisons between papers easier.

Regarding defenses, we believe our paper provides future research with several
significant findings. We categorize how each defensive measure is used, for
which technology, and against which attack they are leveraged. We hope to
motivate future research to utilize this survey in finding defenses and make
for easier comparisons when researching defensive measures. We have also
presented how each defense performs in terms of effectiveness against specific
attacks and, where applicable, how they impact the model’s performance. In
turn, we hope to motivate future research into less performance-reducing
defenses. Lastly, while many papers cover complete knowledge scenarios to
defend against a worst-case attack, only focusing on one level of knowledge
might also be a problem, as defensive measures in different attacker knowledge
scenarios are not necessarily transferable. Therefore, as mentioned in regards
to attack knowledge, we believe future research should consider how the
defense is impacted when employed against attacks in different knowledge
scenarios.

One of the main challenges facing adversarially robust UL is the general need
for more focus on constructing defensive measures against attacks. Much
research focuses primarily on the vulnerability of a specific model without
formulating defenses. Additionally, when research does focus on defensive
measures, it is mostly against specific attacks. While exploring specific
defenses is good, there is a noticeable lack of more general defensive
measures, with the big exception of differential privacy, which has seen the
most focus out of any defense in UL. We, therefore, believe that future
research would benefit from focusing on more generalizable defenses.

We believe our model in Figure 3 of analyzing adversarial robustness, i.e.,
looking at target type, attack method, attack knowledge, attack goal, attack
focus, metrics, and defenses can be used in future research as a guide for
what is essential to cover in this kind of research, in order to structure
knowledge and make comparisons of results between papers easier. Future
research also needs to be performed in a manner that enables easy comparisons
between results by explicitly defining these different categories.

###  8.3. Implication to industry

Those in the industry who utilize any of the technologies presented, e.g.,
clustering for anomaly detection or synthetic data generation, can use the
results in this paper to better evaluate their use of this technology, e.g.,
in risk assessment. Additionally, since a significant part of this paper
explicitly outlines the state-of-the-art concerning defensive measures for
these technologies, this paper would be a good resource for finding and
implementing defensive measures to protect their UL models. Lastly, our focus
on finding vulnerabilities in these technologies could motivate the industry
to consider the security of their models and not just use unsupervised models
for their utility.

###  8.4. Threats to validity

We strictly followed the SLR guideline to ensure the method was unambiguous
and considered biases that could influence the results’ quality. However, we
could still identify some possible validity threats related to this study.

The researchers’ knowledge of adversarial robustness in unsupervised learning
would inevitably impact the search query. It would impact the specific words
and the decision for when the query is good enough. The researchers have tried
to mitigate this impact by 1) Documenting the specific papers that served as a
basis for what the search should produce and 2) The initial search query would
only be accepted if it produced between 1000 and 10 000 papers.

Another possible threat to validity is the research databases used for finding
papers. This study covered as many relevant databases as possible, including
IEEE Xplore, ACM Digital Library, arXiv, and Scopus.

When filtering the papers, for the first round of filtration, where papers
were excluded based on the title alone, the researchers had to decide to
exclude instead of include a paper explicitly. The reasoning is that deciding
to exclude requires more thought than an inclusive decision, making the
researcher less likely to exclude relevant papers. From the second round
onward, the researchers documented their reasoning for exclusion or inclusion
and produced documentation that could be scrutinized.

Regarding the quality assurance of the papers, there are also several possible
biases to consider. First, there is the bias in choosing which categories
contribute to the overall quality score. For this, we considered four
different papers (Keele et al., 2007; Kitchenham et al., 2015; Dybå and
Dingsøyr, 2008; Ivarsson and Gorschek, 2011) for constructing a list of
questions. Of these, (Keele et al., 2007) and (Kitchenham et al., 2015)
appeared most relevant. We compared these two and concluded that neither paper
was better than the other, ultimately producing the 11 questions for the QA by
excluding one irrelevant question from (Kitchenham et al., 2015). Secondly,
there is a bias in how the papers were graded based on these questions. To
combat this, after the QA of the remaining 28 papers in the initial round, we
conducted a kappa analysis on the scoring between the two authors to indicate
whether the scoring was externally valid. The two authors scored a random
sample of eight papers, and a weighted kappa score was calculated, producing a
weighted kappa score of 0.68.

##  9\. Conclusion and future work

The existing literature on adversarial robustness in machine learning needs
more coverage on unsupervised machine learning technologies. We have therefore
performed a systematic literature review to summarize adversarial attacks,
defenses, and research gaps related to unsupervised machine learning. From an
initial list of 9129 papers, we have identified a list of 86 primary studies.
We summarized the attack types, properties of targeted systems, metrics used
for attack success, and the impact of the attacks. For each target type, we
summarized and analyzed the papers in relation to 1) attack type, attack goal,
attack focus, attack knowledge, and metrics, 2) summarized and analyzed the
papers relating to defense, and 3) summarized and analyzed the current
challenges in research into adversarial robustness of UL.

One of the notable findings is that, except for differential privacy, the
field of unsupervised learning has a noticeable lack of focus on effective
defensive methods, as several papers also conclude that defensive methods used
in other fields of machine learning do not work in the context of unsupervised
learning. Additionally, despite the fact that unsupervised learning systems
are fairly diverse in function and form, the metrics used to determine whether
attacks are successful are primarily shared, and many adversaries use distance
metrics to determine the strength of the attack to avoid detection. Knowing
this, crafting more sophisticated detection methods that would not be affected
by distance-metric limitations might be worthwhile.

As mentioned in the discussion, further research into poisoning attacks is a
worthwhile direction due to both the applicability of poisoning attacks
against, seemingly, any unsupervised machine learning (and perhaps all types
of machine learning) and due to the amount of data that is used in UL, which
could contain poisoned samples since the data could be sourced from public
databases or data that anyone can contribute to.

## References

  * (1)
  * Abadi et al. (2016) Martin Abadi, Andy Chu, Ian Goodfellow, H Brendan McMahan, Ilya Mironov, Kunal Talwar, and Li Zhang. 2016.  Deep learning with differential privacy. In _Proceedings of the 2016 ACM SIGSAC conference on computer and communications security_. 308–318.
  * Abbasi and Gagné (2017) Mahdieh Abbasi and Christian Gagné. 2017.  Robustness to adversarial examples through an ensemble of specialists.  _arXiv preprint arXiv:1702.06856_ (2017).
  * Akhtar et al. (2018) Naveed Akhtar, Jian Liu, and Ajmal Mian. 2018.  Defense against universal adversarial perturbations. In _Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition_. 3389–3398.
  * Albaseer et al. (2020) Abdullatif Albaseer, Bekir Sait Ciftler, and Mohamed M Abdallah. 2020.  Performance evaluation of physical attacks against e2e autoencoder over rayleigh fading channel. In _2020 IEEE International Conference on Informatics, IoT, and Enabling Technologies (ICIoT)_. IEEE, 177–182.
  * Alfarra et al. (2022) Motasem Alfarra, Juan C Pérez, Anna Frühstück, Philip HS Torr, Peter Wonka, and Bernard Ghanem. 2022.  On the Robustness of Quality Measures for GANs.  _arXiv preprint arXiv:2201.13019_ (2022).
  * Ali et al. (2021) A Tahseen Ali, Hasanen S Abdullah, and Mohammad N Fadhil. 2021\.  Voice recognition system using machine learning techniques.  _Materials Today: Proceedings_ (2021), 1–7.
  * Ali and Shah (2005) Saad Ali and Mubarak Shah. 2005.  A supervised learning framework for generic object detection in images. In _Tenth IEEE International Conference on Computer Vision (ICCV’05) Volume 1_ , Vol. 2. IEEE, 1347–1354.
  * Anindya and Kantarcioglu (2018) Imrul Chowdhury Anindya and Murat Kantarcioglu. 2018.  Adversarial anomaly detection using centroid-based clustering. In _2018 IEEE International Conference on Information Reuse and Integration (IRI)_. IEEE, 1–8.
  * Anwar et al. (2020) Saeed Anwar, Salman Khan, and Nick Barnes. 2020.  A deep journey into super-resolution: A survey.  _ACM Computing Surveys (CSUR)_ 53, 3 (2020), 1–34.
  * Badu-Marfo et al. (2020) Godwin Badu-Marfo, Bilal Farooq, and Zachary Patterson. 2020\.  A Differentially Private Multi-Output Deep Generative Networks Approach For Activity Diary Synthesis.  _arXiv preprint arXiv:2012.14574_ (2020).
  * Bae et al. (2018) Ho Bae, Jaehee Jang, Dahuin Jung, Hyemi Jang, Heonseok Ha, Hyungyu Lee, and Sungroh Yoon. 2018.  Security and privacy issues in deep learning.  _arXiv preprint arXiv:1807.11655_ (2018).
  * Banerjee et al. (2018) Nikhil Banerjee, Thanassis Giannetsos, Emmanouil Panaousis, and Clive Cheong Took. 2018.  Unsupervised learning for trustworthy IoT. In _2018 IEEE international conference on fuzzy systems (FUZZ-IEEE)_. IEEE, 1–8.
  * Bank et al. (2020) Dor Bank, Noam Koenigstein, and Raja Giryes. 2020\.  Autoencoders.  _arXiv preprint arXiv:2003.05991_ (2020).
  * Bansal et al. (2011) Ankur Bansal, Tingting Chen, and Sheng Zhong. 2011.  Privacy preserving back-propagation neural network learning over arbitrarily partitioned data.  _Neural Computing and Applications_ 20 (2011), 143–150.
  * Baracaldo et al. (2017) Nathalie Baracaldo, Bryant Chen, Heiko Ludwig, and Jaehoon Amir Safavi. 2017.  Mitigating poisoning attacks on machine learning models: A data provenance based approach. In _Proceedings of the 10th ACM workshop on artificial intelligence and security_. 103–110.
  * Barrett et al. (2022) Ben Barrett, Alexander Camuto, Matthew Willetts, and Tom Rainforth. 2022.  Certifiably robust variational autoencoders. In _International Conference on Artificial Intelligence and Statistics_. PMLR, 3663–3683.
  * Beaulieu-Jones et al. (2019) Brett K Beaulieu-Jones, Zhiwei Steven Wu, Chris Williams, Ran Lee, Sanjeev P Bhavnani, James Brian Byrd, and Casey S Greene. 2019\.  Privacy-preserving generative deep neural networks support clinical data sharing.  _Circulation: Cardiovascular Quality and Outcomes_ 12, 7 (2019), e005122.
  * Bernau et al. (2022) Daniel Bernau, Jonas Robl, and Florian Kerschbaum. 2022\.  Assessing Differentially Private Variational Autoencoders under Membership Inference.  _arXiv preprint arXiv:2204.07877_ (2022).
  * Biggio et al. (2014a) Battista Biggio, Samuel Rota Bulò, Ignazio Pillai, Michele Mura, Eyasu Zemene Mequanint, Marcello Pelillo, and Fabio Roli. 2014a.  Poisoning complete-linkage hierarchical clustering. In _Joint IAPR International Workshops on Statistical Techniques in Pattern Recognition (SPR) and Structural and Syntactic Pattern Recognition (SSPR)_. Springer, 42–52.
  * Biggio et al. (2013) Battista Biggio, Ignazio Pillai, Samuel Rota Bulò, Davide Ariu, Marcello Pelillo, and Fabio Roli. 2013.  Is data clustering in adversarial settings secure?. In _Proceedings of the 2013 ACM workshop on Artificial intelligence and security_. 87–98.
  * Biggio et al. (2014b) Battista Biggio, Konrad Rieck, Davide Ariu, Christian Wressnegger, Igino Corona, Giorgio Giacinto, and Fabio Roli. 2014b.  Poisoning behavioral malware clustering. In _Proceedings of the 2014 workshop on artificial intelligent and security workshop_. 27–36.
  * Bittner et al. (2017) Daniel M Bittner, Anand D Sarwate, and Rebecca N Wright. 2017\.  Differentially Private Noisy Search with Applications to Anomaly Detection. In _Proceedings of the 10th ACM Workshop on Artificial Intelligence and Security_. 53–53.
  * Braun and Clarke (2006) Virginia Braun and Victoria Clarke. 2006.  Using thematic analysis in psychology.  _Qualitative research in psychology_ 3, 2 (2006), 77–101.
  * Camuto et al. (2021) Alexander Camuto, Matthew Willetts, Stephen Roberts, Chris Holmes, and Tom Rainforth. 2021.  Towards a theoretical understanding of the robustness of variational autoencoders. In _International Conference on Artificial Intelligence and Statistics_. PMLR, 3565–3573.
  * Carlini and Wagner (2016) Nicholas Carlini and David Wagner. 2016.  Defensive distillation is not robust to adversarial examples.  _arXiv preprint arXiv:1607.04311_ (2016).
  * Castillo et al. (2021) Angela Castillo, María Escobar, Juan C Pérez, Andrés Romero, Radu Timofte, Luc Van Gool, and Pablo Arbelaez. 2021\.  Generalized Real-World Super-Resolution through Adversarial Robustness. In _Proceedings of the IEEE/CVF International Conference on Computer Vision_. 1855–1865.
  * Cemgil et al. (2019) Taylan Cemgil, Sumedh Ghaisas, Krishnamurthy Dj Dvijotham, and Pushmeet Kohli. 2019.  Adversarially robust representations with smooth encoders. In _International Conference on Learning Representations_.
  * Chabanne et al. (2017) Hervé Chabanne, Amaury De Wargny, Jonathan Milgram, Constance Morel, and Emmanuel Prouff. 2017\.  Privacy-preserving classification on deep neural network.  _Cryptology ePrint Archive_ (2017).
  * Chander and Vijaya (2021) Satish Chander and P Vijaya. 2021.  Unsupervised learning methods for data clustering.  In _Artificial Intelligence in Data Mining_. Elsevier, 41–64.
  * Chandrasekaran et al. (2021) Varun Chandrasekaran, Darren Edge, Somesh Jha, Amit Sharma, Cheng Zhang, and Shruti Tople. 2021.  Causally Constrained Data Synthesis for Private Data Release.  _arXiv preprint arXiv:2105.13144_ (2021).
  * Chen et al. (2020a) Dingfan Chen, Tribhuvanesh Orekondy, and Mario Fritz. 2020a.  Gs-wgan: A gradient-sanitized approach for learning differentially private generators.  _Advances in Neural Information Processing Systems_ 33 (2020), 12673–12684.
  * Chen et al. (2020b) Dingfan Chen, Ning Yu, Yang Zhang, and Mario Fritz. 2020b.  Gan-leaks: A taxonomy of membership inference attacks against generative models. In _Proceedings of the 2020 ACM SIGSAC conference on computer and communications security_. 343–362.
  * Chen et al. (2022) Jia-Wei Chen, Chia-Mu Yu, Ching-Chia Kao, Tzai-Wei Pang, and Chun-Shien Lu. 2022.  DPGEN: Differentially Private Generative Energy-Guided Network for Natural Image Synthesis. In _Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition_. 8387–8396.
  * Chen et al. (2017) Yizheng Chen, Yacin Nadji, Athanasios Kountouras, Fabian Monrose, Roberto Perdisci, Manos Antonakakis, and Nikolaos Vasiloglou. 2017\.  Practical attacks against graph-based clustering. In _Proceedings of the 2017 ACM SIGSAC conference on computer and communications security_. 1125–1142.
  * Chi et al. (2018) Jianfeng Chi, Emmanuel Owusu, Xuwang Yin, Tong Yu, William Chan, Patrick Tague, and Yuan Tian. 2018.  Privacy partitioning: Protecting user data during the deep learning inference phase.  _arXiv preprint arXiv:1812.02863_ (2018).
  * Choi et al. (2019) Jun-Ho Choi, Huan Zhang, Jun-Hyuk Kim, Cho-Jui Hsieh, and Jong-Seok Lee. 2019.  Evaluating robustness of deep image super-resolution against adversarial attacks. In _Proceedings of the IEEE/CVF International Conference on Computer Vision_. 303–311.
  * Chrysos et al. (2020) Grigorios G Chrysos, Jean Kossaifi, and Stefanos Zafeiriou. 2020\.  Rocgan: Robust conditional gan.  _International Journal of Computer Vision_ 128, 10 (2020), 2665–2683.
  * Cinà et al. (2022) Antonio Emanuele Cinà, Alessandro Torcinovich, and Marcello Pelillo. 2022.  A black-box adversarial attack for poisoning clustering.  _Pattern Recognition_ 122 (2022), 108306.
  * Condessa and Kolter (2020) Filipe Condessa and Zico Kolter. 2020.  Provably robust deep generative models.  _arXiv preprint arXiv:2004.10608_ (2020).
  * Creswell et al. (2017) Antonia Creswell, Anil A Bharath, and Biswa Sengupta. 2017\.  Latentpoison-adversarial attacks on the latent space.  _arXiv preprint arXiv:1711.02879_ (2017).
  * Creswell et al. (2018) Antonia Creswell, Tom White, Vincent Dumoulin, Kai Arulkumaran, Biswa Sengupta, and Anil A Bharath. 2018.  Generative adversarial networks: An overview.  _IEEE signal processing magazine_ 35, 1 (2018), 53–65.
  * Damgård et al. (2012) Ivan Damgård, Valerio Pastro, Nigel Smart, and Sarah Zakarias. 2012.  Multiparty computation from somewhat homomorphic encryption. In _Advances in Cryptology–CRYPTO 2012: 32nd Annual Cryptology Conference, Santa Barbara, CA, USA, August 19-23, 2012. Proceedings_. Springer, 643–662.
  * Deng and Sang (2020) Zhixiang Deng and Qian Sang. 2020.  Harnessing the Adversarial Perturbation to Enhance Security in the Autoencoder-Based Communication System.  _Electronics_ 9, 2 (2020), 294.
  * Denyer et al. (2009) David Denyer, David Tranfield, D Buchanan, and A Bryman. 2009\.  The Sage handbook of organizational research methods.  _Reference and Research Book News_ 24, 3 (2009), 776\.
  * Devaranjan et al. (2020) Jeevan Devaranjan, Amlan Kar, and Sanja Fidler. 2020.  Meta-sim2: Unsupervised learning of scene structure for synthetic data generation. In _Computer Vision–ECCV 2020: 16th European Conference, Glasgow, UK, August 23–28, 2020, Proceedings, Part XVII 16_. Springer, 715–733.
  * Dihuni (2020) Dihuni. 2020.  _Every Day Big Data Statistics – 2.5 Quintillion Bytes of Data Created Daily_.  https://www.dihuni.com/2020/04/10/every-day-big-data-statistics-2-5-quintillion-bytes-of-data-created-daily/
  * Ding et al. (2019) Shaohua Ding, Yulong Tian, Fengyuan Xu, Qun Li, and Sheng Zhong. 2019.  Trojan attack on deep generative models in autonomous driving. In _International Conference on Security and Privacy in Communication Systems_. Springer, 299–318.
  * Duddu (2018) Vasisht Duddu. 2018\.  A survey of adversarial machine learning in cyber warfare.  _Defence Science Journal_ 68, 4 (2018), 356\.
  * Dwork (2006) Cynthia Dwork. 2006\.  Differential privacy. In _Automata, Languages and Programming: 33rd International Colloquium, ICALP 2006, Venice, Italy, July 10-14, 2006, Proceedings, Part II 33_. Springer, 1–12.
  * Dwork et al. (2006) Cynthia Dwork, Frank McSherry, Kobbi Nissim, and Adam Smith. 2006.  Calibrating noise to sensitivity in private data analysis. In _Theory of Cryptography: Third Theory of Cryptography Conference, TCC 2006, New York, NY, USA, March 4-7, 2006. Proceedings 3_. Springer, 265–284.
  * Dybå and Dingsøyr (2008) Tore Dybå and Torgeir Dingsøyr. 2008.  Strength of evidence in systematic reviews in software engineering. In _Proceedings of the Second ACM-IEEE international symposium on Empirical software engineering and measurement_. 178–187.
  * Fan (2020) Liyue Fan. 2020\.  A survey of differentially private generative adversarial networks. In _The AAAI Workshop on Privacy-Preserving Artificial Intelligence_. 8.
  * Franci et al. (2022) Adriano Franci, Maxime Cordy, Martin Gubri, Mike Papadakis, and Yves Le Traon. 2022.  Influence-Driven Data Poisoning in Graph-Based Semi-Supervised Classifiers. In _2022 IEEE/ACM 1st International Conference on AI Engineering–Software Engineering for AI (CAIN)_. IEEE, 77–87.
  * Fredrikson et al. (2015) Matt Fredrikson, Somesh Jha, and Thomas Ristenpart. 2015\.  Model inversion attacks that exploit confidence information and basic countermeasures. In _Proceedings of the 22nd ACM SIGSAC conference on computer and communications security_. 1322–1333.
  * Frigerio et al. (2019) Lorenzo Frigerio, Anderson Santana de Oliveira, Laurent Gomez, and Patrick Duverger. 2019.  Differentially private generative adversarial networks for time series, continuous, and discrete open data. In _IFIP International Conference on ICT Systems Security and Privacy Protection_. Springer, 151–164.
  * Gentry (2009) Craig Gentry. 2009\.  Fully homomorphic encryption using ideal lattices. In _Proceedings of the forty-first annual ACM symposium on Theory of computing_. 169–178.
  * Gilad-Bachrach et al. (2016) Ran Gilad-Bachrach, Nathan Dowlin, Kim Laine, Kristin Lauter, Michael Naehrig, and John Wernsing. 2016\.  Cryptonets: Applying neural networks to encrypted data with high throughput and accuracy. In _International conference on machine learning_. PMLR, 201–210.
  * Gondim-Ribeiro et al. (2018) George Gondim-Ribeiro, Pedro Tabacof, and Eduardo Valle. 2018\.  Adversarial attacks on variational autoencoders.  _arXiv preprint arXiv:1806.04646_ (2018).
  * Goodfellow et al. (2020) Ian Goodfellow, Jean Pouget-Abadie, Mehdi Mirza, Bing Xu, David Warde-Farley, Sherjil Ozair, Aaron Courville, and Yoshua Bengio. 2020\.  Generative adversarial networks.  _Commun. ACM_ 63, 11 (2020), 139–144.
  * Goodfellow et al. (2014) Ian J Goodfellow, Jonathon Shlens, and Christian Szegedy. 2014\.  Explaining and harnessing adversarial examples.  _arXiv preprint arXiv:1412.6572_ (2014).
  * Grira et al. (2004) Nizar Grira, Michel Crucianu, and Nozha Boujemaa. 2004\.  Unsupervised and semi-supervised clustering: a brief survey.  _A review of machine learning techniques for processing multimedia content_ 1, 2004 (2004), 9–16.
  * Gu and Rigazio (2014) Shixiang Gu and Luca Rigazio. 2014.  Towards deep neural network architectures robust to adversarial examples.  _arXiv preprint arXiv:1412.5068_ (2014).
  * Guan et al. (2018) Zhenyu Guan, Liangxu Bian, Tao Shang, and Jianwei Liu. 2018\.  When machine learning meets security issues: A survey. In _2018 IEEE international conference on intelligence and safety for robotics (ISR)_. IEEE, 158–165.
  * Gupta et al. (2020) Saurabh Gupta, Arun Balaji Buduru, and Ponnurangam Kumaraguru. 2020\.  imdpgan: Generating private and specific data with generative adversarial networks. In _2020 Second IEEE International Conference on Trust, Privacy and Security in Intelligent Systems and Applications (TPS-ISA)_. IEEE, 64–72.