# Bayesian evidence-driven likelihood selection for sky-averaged 21-cm signal
extraction

K.H. Scheutwinkel,1 W. Handley,1,2 and E. de Lera Acedo1,2
1Astrophysics Group, Cavendish Laboratory, J. J. Thomson Avenue, Cambridge CB3
0HE, UK
2Kavli Institute for Cosmology, Madingley Road, Cambridge CB3 0HA, UK

###### Abstract

We demonstrate that the Bayesian evidence can be used to find a good
approximation of the true likelihood function of a dataset, a goal of the
likelihood-free inference (LFI) paradigm. As a concrete example, we use
forward modelled sky-averaged 21-cm signal antenna temperature datasets where
we artificially inject noise structures of various physically motivated forms.
We find that the Gaussian likelihood performs poorly when the noise
distribution deviates from the Gaussian case e.g. heteroscedastic radiometric
or heavy-tailed noise. For these non-Gaussian noise structures, we show that
the generalised normal likelihood is on a similar Bayesian evidence scale with
comparable sky-averaged 21-cm signal recovery as the true likelihood function
of our injected noise. We therefore propose the generalised normal likelihood
function as a good approximation of the true likelihood function if the noise
structure is a priori unknown.

###### keywords:

dark ages, reionisation, first stars – methods: statistical – methods: data
analysis

##  1 Introduction

21-cm cosmology (Furlanetto et al., 2006) studies the hydrogen hyperfine
transition line with a characteristic free-space wavelength of $\lambda=21$
cm. When this spin-flip transition is emitted during Cosmic Dawn (CD) and
Epoch of Reionisation (EOR), the wavelength is redshifted to lower
frequencies, enabling us to probe these epochs through radio observatories.
Experimental approaches such as the sky-averaged experiment (Liu et al., 2013)
measure the contrast between integrated emission of the 21-cm signal and the
cosmic microwave background (CMB) across the whole sky. This contrast which is
known as the sky-averaged 21-cm signal has a characteristic shape (Pritchard &
Loeb, 2008) that can be probed to study and constrain the earliest epochs of
the universe.

The EDGES collaboration (Bowman et al., 2018a) were the first to report a deep
flattened absorption feature at CD with an amplitude of $500^{+500}_{-200}$ mK
centred at $78\pm 1$ MHz. However, these results are not compatible e.g. with
the astrophysical models of Cohen et al. (2017) which predicted a weaker
absorption signal with less than a half of the amplitude of what the EDGES
collaboration reported. Furthermore, the best-fitting profile reported by
EDGES is not a unique solution and an unmodelled sinusoidal systematic
structure is detected when changing the foreground model i.e. the number of
polynomials (Hills et al., 2018). These discrepancies caused a scientific
debate (Bowman et al., 2018b) that can only be resolved by an independent sky-
averaged experiment such as SARAS 2 and 3 (Singh et al., 2018; Singh et al.,
2021), LEDA (Price et al., 2018), REACH (de Lera Acedo, 2019) or many others.
Possible candidates to explain the EDGES signal, if of cosmological origin,
could involve new physics; an excessive cooling mechanism of the IGM through
dark matter particle interaction (Barkana et al., 2018; Barkana, 2018; Muñoz &
Loeb, 2018) or an enhanced radio background apart from the CMB (Jana et al.,
2019; Fialkov & Barkana, 2019; Mirocha & Furlanetto, 2019) with standard
stellar population models incapable to achieve this (Mittal & Kulkarni, 2022).
However, some argue that the claim of new physics is not justified as this
deep absorption feature could be a result of unmodelled systematic features
such as a ground plane artefact (Bradley et al., 2019), calibration issues
(Sims & Pober, 2020) or within the data analysis methods by changing the
polynomial foreground model to maximally smooth functions (Bevins et al.,
2021; Singh & Subrahmanyan, 2019). Additionally, recent data collected by
SARAS 3 (Singh et al., 2021) reject the best-fitting profile of EDGES with
95.3% confidence and exclude the astrophysical origin as the cause for this
deep flattened profile.

Adding these various model components into the simulations and comparing them
with the observations is the subject of current state-of-the-art sky-averaged
21-cm cosmological research. However, determining the statistical properties
of this often hierarchical generative model which potentially consists of many
latent hyperparameters poses a challenge for current Bayesian inference tasks.
This is due to the necessity of an explicit likelihood expression to recover
the parameters of our model i.e. solving the inverse problem of the sky-
averaged 21-cm simulations. Oversimplification of the noise structure e.g.
through a Gaussian approximation can engender inaccurate posterior inferences.

We investigate this problem of unknown noise structures, by generating antenna
temperature datasets with non-Gaussian or heavy-tailed noise and study its
influence on the sky-averaged 21-cm signal parameter inference by using
likelihood functions of various forms. We use a Bayesian evidence-driven
likelihood selection to decide which likelihood is preferred and the best
approximation of the (unknown) true likelihood of our dataset. This Bayesian
evidence-driven likelihood selection analysis can be embedded into the
Likelihood-Free Inference (LFI) paradigm Marin et al. (2011); Papamakarios &
Murray (2016); Cranmer et al. (2020); Papamakarios et al. (2021), where the
goal is to find a satisfactory approximation of the unknown true likelihood
function. Examples of various algorithmic LFI methods applied in cosmology can
be found in Alsing et al. (2018, 2019); Jeffrey et al. (2020).

In Section 2, we briefly explain the Bayesian Inference framework and its
parameter estimation and model comparison component. In Section 3, we describe
how we generate the sky-averaged 21-cm signal antenna temperature datasets
using a physically motivated forward model. In Section 4, we present various
candidates of likelihood functions to statistically model our (unknown)
datasets and in Section 5, we present our findings. Finally, we summarise our
work in Section 6.

##  2 Bayesian Inference and Nested Sampling

We use the Bayesian inference framework (Sivia & Skilling, 2006) to analyse a
dataset $D$, which is a powerful statistical tool based on Bayes Theorem of
conditioned probabilities:

$\mathcal{P}=\frac{\mathcal{L}\pi}{\mathcal{Z}},$ (1)

where $\mathcal{P}\equiv p(\theta|D,M)$ is the posterior distribution of the
parameters $\theta$ of model $M$ after the dataset $D$ has been observed. The
posterior is recovered by combining the likelihood $\mathcal{L}\equiv
p(D|\theta,M)$ of the dataset with the prior $\pi\equiv p(\theta|M)$ of the
parameters, a distribution containing prior assumptions before any data has
been observed. The product of these two distributions is normalised by
$\mathcal{Z}$ the marginalised likelihood or also known as the Bayesian
evidence.

With the help of algorithms such as Markov-Chain Monte Carlo (MCMC) methods
(MacKay, 2003) one is able to address the parameter estimation problem of
Bayesian inference to generate posterior samples $\theta^{*}$ from the prior.
However, MCMC methods avoid solving the generally computationally expensive
integration of the marginalised likelihood:

$\mathcal{Z}=\int\mathcal{L}(\theta)\pi(\theta)d\theta,$ (2)

which is an integration over all possible parameter values $\theta$ of the
model $M$. The solution of the integral is the Bayesian evidence
$\mathcal{Z}$, a key component for model comparison in Bayesian inference. As
the model comparison is essential for this analysis, we use a nested sampling-
based (Skilling, 2006) algorithm which numerically computes the Bayesian
evidence. We choose PolyChord (Handley et al., 2015b, a) as our nested
sampler, which utilises a slice sampling method to generate new proposal
candidates subject to an evolving likelihood constraint. Additionally,
PolyChord proves to be a more scalable solution than MultiNest (Feroz &
Hobson, 2008; Feroz et al., 2009) which is vital for high-dimensional
parameter spaces. Moreover, as PolyChord is a sampling-based method, we can
acquire posterior samples and, therefore, address the model comparison and
parameter estimation part of Bayesian inference simultaneously.

With the Bayesian evidence $p(D|M)$ we recover the model probabilities by
applying Bayes Theorem:

$p(M|D)=\frac{p(D|M)p(M)}{p(D)}.$ (3)

For two competing models $M_{1}$ and $M_{2}$ which are assumed a priori
equally likely $p(M_{1})=p(M_{2})$, we define the Bayes factor:

$\log\mathcal{K}=\log p(M_{1}|D)-\log p(M_{2}|D),$ (4)

which is a Bayesian evidence posterior ratio of both models given our
assumption. Hence, a positive Bayes factor indicates the preference of model
$M_{1}$ for the dataset $D$.

##  3 Forward Modelling of dataset

To generate the dataset $D$, we use a physically motivated forward model. The
forward model splits the dataset into a sum of three components:

$D\equiv T_{\mathrm{data}}=T_{\mathrm{fg}}+T_{21}+T_{\mathrm{noise}},$ (5)

where $T_{\mathrm{fg}}$ is the foreground component, $T_{21}$ the sky-averaged
21-cm signal and $T_{\mathrm{noise}}$ the contribution of the noise model. We
describe in the subsequent sections how we simulate each component.

###  3.1 Foreground simulation

For our simulated experimental setup, we use a conical log-spiral antenna beam
pattern (Dyson, 1965) which is reported to have the most accurate capabilities
to recover the sky-averaged 21-cm signal compared to five other antenna
designs (Anstey et al., 2021b). This setup is located at $\lambda=-30.71131$°
and $\phi=21.4476236$° at the Karoo Radio reserve in South Africa, where REACH
is currently being deployed. We choose a snapshot of the sky at ‘2019-10-01
00:00:00’ UTC, where the Milky Way centre is not at the zenith to mitigate
brighter foreground contributions.

To simulate the foreground we use the Global Sky Model (GSM) map of 2008 (De
Oliveira-Costa et al., 2008) at the 408 MHz frequency and downscale the map to
230 MHz to use an approximate representation of the sky in the 50-200 MHz
frequency band where the true sky-averaged 21-cm signal is expected.
Furthermore, 230 MHz is “far enough” that there is no contamination introduced
by the same signal, therefore removing potential issues when we add a
simulated absorption feature later on. We conduct the downscaling through the
spatially variational spectral index parameter:

$\beta(\Omega)=\frac{\log\left(\frac{T_{230}(\Omega)-T_{\mathrm{CMB}}}{T_{408}(\Omega)-T_{\mathrm{CMB}}}\right)}{\log\left(\frac{230}{408}\right)},$
(6)

with $T_{\mathrm{CMB}}=2.375$ K the CMB. We choose this spatially dependent
spectral index as it is physically more realistic than a constant spectral
index. Moreover, Anstey et al. (2021a) investigated the influence of a uniform
spectral index for the resulting sky-averaged 21-cm parameter inference and
concluded that a uniform index introduces spectral features which are
mimicking a sky-averaged 21-cm signal, hence, making the signal extraction
unnecessarily difficult or unsuccessful.

We use this spectral index $\beta(\Omega)$ to generate our foreground:

$T_{\mathrm{sim}}(\nu,\Omega)=(T_{230}-T_{\mathrm{CMB}})\left(\frac{\nu}{230}\right)^{-\beta(\Omega)}+T_{\mathrm{CMB}},$
(7)

which we convolve with the conical log spiral antenna beam pattern
$D_{\Omega}$:

$T_{\mathrm{fg}}(\nu)=\frac{1}{4\pi}\int_{\Omega}D(\Omega,\nu)T_{\mathrm{sim}}(\Omega,\nu)d\Omega$
(8)

to compute the resulting antenna temperature $T_{\mathrm{fg}}$.

###  3.2 Sky-averaged 21-cm signal

To simulate the sky-averaged 21-cm signal, we add a Gaussian absorption
profile which is a Gaussian approximation of the standard case of the
astrophysical models of Cohen et al. (2017):

$T_{\mathrm{21}}(\nu)=-A_{21}\exp\left(-\frac{1}{2}\left(\frac{\nu-
f_{0,21}}{\sigma_{21}}\right)^{2}\right),$ (9)

where $A_{21}$ is the amplitude of the absorption signal, $\sigma_{21}$ the
standard deviation, $f_{0,21}$ the central frequency and $\nu$ the frequency.
For our absorption feature, we set $A_{21}=155$ mK, $\sigma_{21}=15$ MHz and
$f_{0,21}=85$ MHz.

###  3.3 Antenna temperature noise

Figure 1: Comparison of standardised probability density functions.

The Bayesian pipeline developed by Anstey et al. (2021a) has demonstrated the
success of the sky-averaged 21-cm signal recovery through simulating antenna
temperature dataset with homoscedastic Gaussian noise. However, this noise
structure is not an accurate physically motivated choice, hence, we add
varying non-Gaussian noise models onto the antenna temperature
$T_{\mathrm{fg}}$ to study its influence on the sky-averaged 21-cm recovery.
As non-Gaussian noise can potentially arise through imperfections in a real
experimental setup or through complex unknown noise structures within a
physical process, we consider one of the following noise models for each
dataset which are also visualised in Figure (1).

  1. 1.

Gaussian noise:

$p(x;\mu,\sigma)=\frac{1}{\sqrt{2\pi\sigma^{2}}}\exp\left(-\frac{(x-\mu)^{2}}{2\sigma^{2}}\right)\,,$
(10)

with mean $\mu$ and standard deviation $\sigma$.

  2. 2.

Generalised normal noise:

$p(x;\mu,\alpha,\beta)=\frac{\beta}{2\alpha\Gamma(1/\beta)}\exp\left(-(|x-\mu|/\alpha)^{\beta}\right)\,,$
(11)

with mean $\mu$, shape parameter $\beta$ and scale parameter $\alpha$. If
$\beta=2$, this noise model recovers the Gaussian noise model. Furthermore,
the variance of the distribution can be expressed in terms of the shape and
scale parameter:
$\sigma^{2}=\frac{\alpha^{2}\Gamma(3/\beta)}{\Gamma(1/\beta)}$.

  3. 3.

The Student-t distribution:

$p(x;\nu,\hat{\mu},\hat{\sigma})=\frac{\Gamma(\frac{\nu+1}{2})}{\sqrt{\nu\pi}\hat{\sigma}\Gamma(\frac{\nu}{2})}\left(1+\frac{1}{\nu}\left(\frac{x-\hat{\mu}}{\hat{\sigma}}\right)^{2}\right)^{-\frac{\nu+1}{2}}\,,$
(12)

with $\nu$ degrees of freedom, the location parameter $\hat{\mu}$ and the
scale parameter $\hat{\sigma}$. This distribution arises after the antenna
temperature has been calibrated in a Bayesian way (Roque et al., 2021).

  4. 4.

The Cauchy distribution:

$p(x;x_{0},\gamma)=\frac{1}{\pi\gamma\left[1+\left(\frac{x-x_{0}}{\gamma}\right)^{2}\right]}\,,$
(13)

with the location parameter $x_{0}$ and the scale parameter $\gamma$. This
distribution has an undefined mean and variance, therefore, it is a heavy-tail
distribution simulating frequent outliers i.e. extreme noisy structures.

  5. 5.

The radiometric noise (Kraus et al., 1986), a physically motivated radiometer
model, which is defined as:

$\sigma_{\mathrm{radio}}(\nu)=\frac{\eta
T_{\mathrm{fg}}(\nu)+(1-\eta)T_{0}+T_{\mathrm{rec}}}{\sqrt{\Delta\nu\tau}}\,,$
(14)

where $T_{\mathrm{fg}}$ is the antenna temperature, $\eta$ the antenna
radiation efficiency, $T_{\mathrm{rec}}$ the antenna receiver temperature,
$\tau$ the integration time and $\Delta\nu$ the channel width. We model the
noise through a Gaussian distribution with heteroscedastic frequency-dependent
radiometric noise.

For each noise model, we centre the probabilistic distribution at the
mean/location parameter $\mu=T_{\mathrm{fg}}$, set the standard
deviation/scale parameter to $\sigma=0.025$ K and vary the shape parameter if
a shape parameter exists, otherwise we vary the scale parameter. For the
radiometric noise model, we assume a realistic choice of parameters with
$\eta=0.9$, $T_{0}=293.15$ K , $T_{\mathrm{rec}}=500$ K, $\Delta\nu=0.1$ MHz
and a variable integration time parameter $\tau$.

##  4 Bayesian modelling of dataset

After dataset creation $D$ with varying noise $T_{\mathrm{noise}}$, we define
two models:

$T_{M_{1}}=T_{\mathrm{fg}}+T_{21},$ (15)

the signal model with $T_{\mathrm{fg}}$ the foreground component, $T_{21}$ the
sky-averaged 21-cm signal component and $T_{\mathrm{noise}}$ the noise
contribution, and

$T_{M_{2}}=T_{\mathrm{fg}},$ (16)

the no-signal model, where we leave the sky-averaged 21-cm signal unmodelled.
In both models, the noise contribution is modelled through its likelihood
function.

The foreground component $T_{\mathrm{fg}}$ utilises the Bayesian foreground
modelling framework of Anstey et al. (2021a), where the sky is split into
$N_{\mathrm{reg}}=14$ regions with each region having its own uniform index to
model the spatially dependent spectral index of the whole sky. Hence, the
foreground parameterises through their spectral indices
$\theta_{\mathrm{fg}}=(\beta_{\mathrm{fg},1:N_{\mathrm{reg}}})$. Additionally,
we assume that the simulated experiment has a perfect ground plane which is a
simplification used to facilitate computation. The precise number of sky
regions $N_{\mathrm{reg}}$ is arbitrarily chosen, however, there is a tendency
that the simulated datasets with Gaussian noise prefer more than ten regions
and the Bayesian evidence plateaus with some fluctuations for higher number of
regions (Anstey et al., 2021a). One can treat this hyperparameter as an extra
dimension to “tune” the Bayesian evidence to find the optimum sky region
number, however, this would increase the number of parameter dimensions of the
cosmological model one has to track. Therefore, we do not progress with this
extra dimension as the inference is computationally expensive and it has no
significant effect on the sky-averaged 21-cm parameter inference as the
posterior distributions are seen to be uncorrelated later on. We note that for
the analysis of a real experimental dataset, this extra dimension has to be
taken into account.

For the sky-averaged 21-cm signal component, we parameterise the Gaussian
signal model of eq. (9) with $\theta_{21}=(f_{0,21},\sigma_{21},A_{21})$. For
the noise contribution, we have varying parameterisation
$\theta_{\mathrm{noise}}$ depending on the likelihood functions used.
Therefore, we have the following parameterisation for each model:
$\theta_{M_{1}}=(\theta_{\mathrm{fg}},\theta_{21},\theta_{\mathrm{noise}})$
and $\theta_{M_{2}}=(\theta_{\mathrm{fg}},\theta_{\mathrm{noise}})$.

###  4.1 Likelihood functions

For each cosmological model $M$, we construct the likelihood functions with:

$\log\mathcal{L}=\log p(T_{\mathrm{data}}|\theta_{M},M)=\sum_{\nu}\log
p(T_{\nu}|\theta_{M},M),$ (17)

which is a sum of the noise model components $p(T_{\nu}|\theta_{M},M)$ over
the frequencies $\nu$ with $\theta_{M}$ the parameters of the model $M$ and
$T_{\nu}$ the observed antenna temperature. Depending on the choice of the
likelihood function, we parameterise the likelihood in the following ways:

  * •

Gaussian: $\theta_{\mathrm{noise}}=\sigma_{\mathrm{noise}}$

  * •

Generalised normal:
$\theta_{\mathrm{noise}}=(\beta_{\mathrm{noise}},\sigma_{\mathrm{noise}})$

  * •

Cauchy: $\theta_{\mathrm{noise}}=\gamma$

  * •

Radiometric:
$\theta_{\mathrm{noise}}=(T_{\mathrm{rec}},\eta,\sigma_{\mathrm{radio}})$

For the radiometric likelihood, we define
$\sigma_{\mathrm{radio}}=1/\sqrt{\tau\Delta\nu}$ as the radiometric noise
level. Analogous to the radiometric noise, we model its likelihood function
through a Gaussian likelihood with the radiometric noise of eq. (14) inserted.
Moreover, we model the Student-t noise through the generalised normal
likelihood as they are similar in nature.

###  4.2 PolyChord initialisation

As PolyChord uses slice sampling within the nested sampling algorithm to draw
proposal candidates, PolyChord has hyperparameters for initialisation, see
Table 2. Additionally, we run PolyChord twice for each model $M$ and dataset
$D$. In the second run, we use the posterior sample mean and sample variance
of the first iteration to construct narrower prior ranges $\bar{\theta}^{*}\pm
5\sigma^{*}$ where we keep the same prior shape as our initial prior
distributions. This ensures posterior exploration around the mode, resulting
in a more tightly constrained set of high likelihood samples. This method is
adapted from and described in more detail in Anstey et al. (2021a).

We list the priors used with PolyChord and their ranges of all model
components and likelihoods in Table 1.

Parameter | Type | Range
---|---|---
$\beta_{1:N_{\mathrm{reg}}}$ | uniform | 2.458-3.146
$f_{0,21}$ | uniform | 50-200 MHz
$\sigma_{21}$ | uniform | 10-20 MHz
$A_{21}$ | uniform | 0-0.25 K
$\sigma_{noise}$ | log uniform |  $10^{-4}$-$10^{-1}$ K
$\beta_{noise}$ | uniform | 0-5
$\gamma$ | log uniform |  $10^{-4}$-$10^{-1}$ K
$T_{\mathrm{rec}}$ | log uniform | 100-1000 K
$\eta$ | uniform | 0.8-1
$\sigma_{radio}$ | log uniform |  $10^{-8}$-$10^{-1}$
Table 1: Prior choices for the foreground spectral indices $\beta_{1:N_{\mathrm{reg}}}$, sky-averaged 21-cm signal shape $f_{0,21}$, $\sigma_{21}$, $A_{21}$ and the varying noise parameters. Parameter | Settings
---|---
$n_{\mathrm{live}}$ | $N_{\mathrm{dim}}*25$
$n_{\mathrm{prior}}$ | $N_{\mathrm{dim}}*25$
$n_{\mathrm{fail}}$ | $N_{\mathrm{dim}}*25$
$n_{\mathrm{repeats}}$ | $N_{\mathrm{dim}}*5$
precision_criterion | 0.001
do_clustering | True
Table 2: PolyChord hyperparameters with $n_{\mathrm{live}}$ the number of live
points, $n_{\mathrm{prior}}$ the number of randomly drawn prior samples before
execution, $n_{\mathrm{fail}}$ the failed spawn criterion, and
$n_{\mathrm{repeats}}$ the number of slice sampling repeats which is
proportional to the model dimension $N_{\mathrm{dim}}$. The
precision_criterion is the termination criterion and do_clustering activates
the clustering algorithm within a nested sampling run.

##  5 Results

Figure 2: Bayesian evidence $\log\mathcal{Z}$ for various noise structures and
likelihood functions. We use each likelihood function twice for the signal
model $M_{1}$ and the no-signal model $M_{2}$. The radiometric noise case is
in logarithmic scale, the zoomed-in window is in linear scale. The errors are
in the order of $\sigma_{\log\mathcal{Z}}\approx 0.3$. Figure 3: Bayes factor
$\log K$ signal comparison for various noise structures. For each likelihood
function the signal model $M_{1}$ and the no-signal model $M_{2}$ are compared
to each other. The radiometric and Cauchy noise case is on a logarithmic
scale. The errors are in the order of $\sigma_{\log K}\approx 0.5$ Figure 4:
Bayes factor $\log K$ likelihood comparison for various noise structures. The
Gaussian likelihood is compared to the generalised normal likelihood and the
true likelihood of the noise structure for each model $M$. The radiometric
noise case is on a logarithmic scale. The errors are in the order of
$\sigma_{\log K}\approx 0.5$. Figure 5: Foreground subtracted residuals (red)
and sky-averaged 21-cm signal recovery (blue) for the Cauchy noise case
($\gamma=0.025$) (top) and radiometric noise case ($\tau=10^{4}$ s) (bottom)
when using varying likelihood functions for inference.

For each dataset $D$ we use PolyChord to numerically compute the Bayesian
evidence $\log\mathcal{Z}$ for the sky-averaged 21-cm signal model $M_{1}$ and
the no signal model $M_{2}$. We show the corresponding Bayesian evidences in
Figure (2) and the Bayes factors of the competing models in Figure (3).
Additionally, for each model $M$, we also vary its likelihood function and
present the Bayes factor likelihood comparison in Figure (4). In Figure (5),
we show exemplary sky-averaged 21-cm signal recoveries when using varying
likelihood functions for different noise structures.

###  5.1 Generalised normal noise

For the generalised normal noise datasets, we note that the signal model
$M_{1}$ has a higher Bayesian evidence than the no signal model $M_{2}$
irrespective of the shape parameter $\beta_{\mathrm{noise}}$ and their
likelihood functions. This indicates that the sky-averaged 21-cm signal model,
the true model of the dataset, is always preferred over the no-signal model.
However, we see a change in likelihood preference once we change the shape
parameter of the noise distribution.

When we decrease the shape parameter $\beta_{\mathrm{noise}}<2$, the noise
distribution turns to a heavy-tailed distribution, hence, allowing more
extreme noise realisations. For these cases, there is a clear preference for
the generalised normal likelihood over the Gaussian likelihood with the signal
model having the highest Bayesian evidence of all models considered. This is
due to the added flexibility of the generalised normal likelihood through its
shape parameter. Similar behaviour can be seen when increasing the shape
parameter i.e. concentrating most of the probability mass around the mean of
the noise distribution. This likelihood preference has an impact on the sky-
averaged 21-cm signal recovery which tends to be biased when using the
Gaussian likelihood i.e. the posterior sky-averaged 21-cm signal amplitude is
larger than expected (e.g. $\bar{T}^{*}_{21}\approx 185\pm 18$ mK for
$\beta=1$), whereas the generalised normal likelihood recovers the signal more
accurately (e.g. $\bar{T}^{*}_{21}\approx 170\pm 14$ mK for $\beta=1$).

For the no signal model $M_{2}$, there is a tendency that the Gaussian
likelihood is slightly more preferred than the generalised normal distribution
for heavy-tailed noise. This is due to the relatively weak sky-averaged 21-cm
signal within the dataset. The no-signal model can use the foreground model to
“fit away” the sky-averaged 21-cm signal and some of the extremer noise
structures, hence, not requiring the additional flexibility of the generalised
normal likelihood.

We confirm this by regenerating the dataset with a larger sky-averaged 21-cm
signal $A_{21}=500$ mK. For these datasets and for the no-signal model
$M_{2}$, the generalised normal likelihood is preferred over the Gaussian
likelihood, as seen in Table 3. For these cases, the foreground and Gaussian
likelihood function alone can not account for this larger structure, hence,
requiring the additional flexibility of the generalised normal likelihood. We
can see similar behaviour when we regenerate datasets where we increase the
scale of the non-Gaussian noise to higher values $\sigma_{\mathrm{noise}}=50$
mK. Here, the apparent preference of the Gaussian likelihood for the no-signal
model vanishes, as the Bayes Factor is around zero.

Change | Noise parameter | $M_{2},\mathrm{Gauss-GenNorm}$
---|---|---
in Dataset | in Dataset | $\log K$
$A_{21}=500$ mK | $\beta_{\mathrm{noise}}=1$ | $-2.90\pm 0.38$
$A_{21}=500$ mK | $\beta_{\mathrm{noise}}=1.2$ | $-3.00\pm 0.38$
$A_{21}=500$ mK | $\beta_{\mathrm{noise}}=1.5$ | $-5.11\pm 0.36$
$\sigma_{\mathrm{noise}}=50$ mK | $\beta_{\mathrm{noise}}=1$ | $-0.81\pm 0.51$
$\sigma_{\mathrm{noise}}=50$ mK | $\beta_{\mathrm{noise}}=1.2$ | $-0.50\pm 0.37$
$\sigma_{\mathrm{noise}}=50$ mK | $\beta_{\mathrm{noise}}=1.5$ | $+1.18\pm 0.38$
Table 3: The Bayes factor for the no-signal model $M_{2}$ when using the
Gaussian and generalised normal likelihood functions applied on a slightly
modified dataset i.e. stronger sky-averaged 21-cm signal or larger noise. The
noise parameter refers to the generalised normal noise distribution of the
dataset.

###  5.2 Student-t noise

For the Student-t noise datasets, we see a tendency in favour of the Gaussian
likelihood for the true signal model $M_{1}$ when the degree of freedom
parameter $\nu$ is increasing. This is due to the Student-t distributions
property to converge to a Gaussian distribution the more underlying samples
one has. Therefore, Occam’s razor penalise the generalised normal likelihood
for its shape parameter and selects the Gaussian likelihood with the true
signal model as the datasets best fitting choice. Moreover, the signal
recovery is for both likelihoods almost identical with no signs of bias in the
sky-averaged 21-cm signal posterior parameters as these Student-t noise
distributions are similar to the Gaussian distribution, therefore modelled
well with the likelihoods considered.

Consequently, when we decrease degree of freedom parameter $\nu$, the
generalised normal distribution will be preferred over the Gaussian likelihood
model. However, we note for these heavy-tailed noise cases $\nu\leq 5$ , the
no-signal model $M_{2}$ has a higher Bayesian evidence than the signal model
$M_{1}$. This indicates that the signal-to-noise ratio is too low to
confidently detect a sky-averaged 21-cm signal, therefore, preferring the
simpler no-signal model. In terms of likelihood preference, the no-signal
model has a tendency to prefer the simpler Gaussian likelihood function
similar to the generalised normal noise case. Hence, the foreground model is
used to “fit away” the sky-averaged 21-cm signal features and absorbing the
structure into the noise, therefore, not needing the extra flexibility offered
by the shape parameter of the generalised normal likelihood function.

###  5.3 Cauchy noise

For the Cauchy noise datasets, we conduct the inference with three different
likelihoods. We use the Gaussian, generalised normal and Cauchy likelihood
which is the true likelihood function of the dataset. Here, the Gaussian
likelihood has the lowest Bayesian evidence, irrespective of the scale
$\gamma$ and model we consider. Only when decreasing the scale parameter
$\gamma$ significantly, the Gaussian likelihoods performs gradually better.

In the signal model comparison, the true signal model $M_{1}$ is always
preferred. Furthermore, as expected, the Bayesian evidence is the highest for
the Cauchy likelihood signal model, as it is the true noise model of the
dataset. However, we note that the generalised normal likelihood performs
almost equally well on the absolute log-evidence scale i.e. in the likelihood
comparison, the Bayes factor is on a similar scale for the Cauchy and
generalised normal likelihood function. .

This Bayesian evidence difference between the likelihoods is noticed in the
sky-averaged 21-cm signal recovery which is shown in Figure (5). The
generalised normal and (true) Cauchy likelihood recovers the sky-averaged
21-cm signal well, however when using the Gaussian likelihood the sky-averaged
21-cm signal tends to have a noticeably larger amplitude i.e. the Gaussian
likelihood fails to recover the true signal parameters. Thus, the generalised
normal likelihood function can be considered as a good approximation of the
true Cauchy likelihood function for sky-averaged 21-cm signal antenna
temperature datasets containing Cauchy noise i.e. extreme noisy features.

###  5.4 Radiometric noise

For the radiometric noise datasets, we see a similar behaviour as the Cauchy
noise case. The Gaussian likelihood has always the lowest Bayesian evidence
for varying integration times $\tau$. For these datasets, the true radiometric
likelihood has the highest Bayesian evidence. Similarly as the Cauchy noise
datasets, the generalised normal likelihood performs reasonably well, hence,
accounting for heteroscedastic noise structures through its shape parameter.
This can be seen in the sky-averaged 21-cm signal recovery of Figure (5) where
the generalised normal and radiometric likelihood function recovers more
tightly constrained posterior parameters in contrast to the homoscedastic
Gaussian likelihood function that is more uncertain in the signal shape.

In the signal comparison, we see a change in model preference at integration
times of $\tau\sim 10^{3}$ s. For lower integration times, the signal-to-noise
ratio is too low to make a confident sky-averaged 21-cm signal detection,
therefore, preferring the no-signal model for these integration times.
Consequently, when we increase the integration time the radiometric noise got
sufficiently suppressed such that the signal model will be preferred.

However, with the Gaussian likelihood the signal model $M_{1}$ is preferred
again if one keeps lowering the integration time. This is due to the models
attempt to fit for a sky-averaged 21-cm signal imitated by the noise,
therefore, incorrectly recovering the sky-averaged 21-cm signal parameters. We
can mitigate this false sky-averaged 21-cm signal parameter recovery by
changing the central frequency $f_{0,21}$ prior range. However, we do not see
this behaviour for the generalised normal and radiometric likelihoods, as they
correctly determine that these are noise structures, whereas the relatively
constrained Gaussian likelihood failed to do so. Hence, we conclude that the
generalized normal likelihood is a better approximation of the radiometric
likelihood function if radiometric noise is present.

##  6 Conclusion

For Bayesian inference, the necessity for an explicit likelihood function
poses a challenging task for cosmological state-of-the-art data analysis.
Usually, it is easier to forward model a dataset for a physical process based
on some input parameters than to identify the often latent and hierarchical
statistical properties in each model component and solving the inverse
problem. Therefore, these complex noise structures can make the likelihood
intractable and a more simplified approximation such as a Gaussian likelihood
is frequently used. The task of finding an approximation of the true
likelihood function is known as likelihood-free inference (LFI) with often
complex and involved algorithms executed to achieve this. However, we
demonstrated that LFI can also be achieved through a simple Bayesian evidence-
driven analysis in which we show that the Gaussian likelihood function is not
always the best choice as an approximation of the true likelihood function of
the dataset.

We used a physically motivated forward model to generate antenna temperature
datasets where we injected noise of various complexity and scales. For each
dataset, we injected one of the following noise structures: Gaussian noise,
generalised normal noise, Student-t noise, Cauchy noise or radiometric noise.
As each of these noise distributions has a varying amount of free parameters,
we varied the shape parameter, if the distribution has a shape parameter,
otherwise we varied its scale parameter.

After dataset creation, we defined two models: the true signal model $M_{1}$
of the dataset containing the foreground, the sky-averaged 21-cm signal and
the noise contribution and the no signal model $M_{2}$ which does not contain
any sky-averaged 21-cm signal component.

We modelled the foreground by dividing the sky into $N_{\mathrm{reg}}=14$
regions of equal spectral index and the sky-averaged 21-cm signal through a
Gaussian-shaped absorption feature and the noise contribution through the
likelihood function. As we injected noise with various structures, we
investigated the following likelihood functions: the Gaussian, generalised
normal, Cauchy and the radiometric likelihood.

By providing priors for our model parameters, we used PolyChord, a nested
sampling-based algorithm, to numerically compute the Bayesian evidence of the
model and likelihood. Afterward, we selected the preferred likelihood through
the Bayesian evidence for each model and dataset.

We found that if the dataset contains Gaussian or Gaussian-like noise (e.g.
Student-t with high $\nu$), the Gaussian likelihood function has the highest
Bayesian evidence for the true signal model $M_{1}$ of the dataset. However,
if the noise is non-Gaussian/heavy-tailed e.g. Cauchy or radiometric noise,
the Gaussian likelihood has always a lower Bayesian evidence with biased
signal parameter recovery than the generalised normal and true likelihood
functions for the datasets studied. Consequently, for these noise structures,
the respective true likelihoods e.g. Cauchy likelihood for Cauchy noise or the
radiometric likelihood for radiometric noise has the highest Bayesian
evidence. However, the generalised normal likelihood is only marginally worse
on Bayesian evidence scale with comparable sky-averaged 21-cm signal recovery
than the true likelihood functions. This is due to the flexibility added by
the shape parameter of the generalised normal distribution to account for
extremely noisy features.

To summarise, this analysis demonstrated the influence of varying likelihoods
on sky-averaged 21-cm signal recovery and it showed that one can get biased
posterior parameter results such as amplitude enhancement or more uncertain
shape parameters by using the oversimplified Gaussian likelihood function for
inference. Hence, only if one is certain that the noise structure of the
antenna temperature dataset is Gaussian one should use the Gaussian likelihood
function for inference. Moreover, in the general case, if one is certain about
the precise noise structure, one should always use its true likelihood
function. However, when one is uncertain about the noise structure, which in
principle holds true for every complex physical process such as the
challenging task of sky-averaged 21-cm signal recovery, one should use the
generalised normal likelihood as a preferred first approximation for the true
unknown likelihood function and expand on it with further analysis. The
generalised normal likelihood offers the flexibility to account for more
complex noise structure and it generalises the Gaussian distribution by
including it as a special case. Hence, this form of preference of the
generalised normal likelihood selected by the Bayesian evidence is a simple
way of finding a good approximation of the true likelihood as a first step of
the likelihood-free inference paradigm applied to 21-cm cosmology.

## Acknowledgements

This work was performed using resources provided by the Cambridge Service for
Data Driven Discovery (CSD3) operated by the University of Cambridge Research
Computing Service, provided by Dell EMC and Intel using Tier-2 funding from
the Engineering and Physical Sciences Research Council (capital grant
EP/T022159/1), and DiRAC funding from the Science and Technology Facilities
Council. KHS would like to thank Dominic Anstey for providing the core
Bayesian REACH pipeline. WH is funded by a Gonville & Caius College Research
Fellowship and Royal Society University Research Fellowship. EdLA is funded by
the Science and Technologies Facilities Council Ernest Rutherford Fellowship.

## References

  * Alsing et al. (2018) Alsing J., Wandelt B., Feeney S., 2018, Monthly Notices of the Royal Astronomical Society, 477, 2874
  * Alsing et al. (2019) Alsing J., Charnock T., Feeney S., Wandelt B., 2019, Monthly Notices of the Royal Astronomical Society, 488, 4440
  * Anstey et al. (2021a) Anstey D., de Lera Acedo E., Handley W., 2021a, Monthly Notices of the Royal Astronomical Society, 506, 2041
  * Anstey et al. (2021b) Anstey D., Cumner J., de Lera Acedo E., Handley W., 2021b, Monthly Notices of the Royal Astronomical Society, 509, 4679
  * Barkana (2018) Barkana R., 2018, Nature, 555, 71
  * Barkana et al. (2018) Barkana R., Outmezguine N. J., Redigolo D., Volansky T., 2018, Physical Review D, 98, 103005
  * Bevins et al. (2021) Bevins H. T. J., Handley W. J., Fialkov A., de Lera Acedo E., Greenhill L. J., Price D. C., 2021, Monthly Notices of the Royal Astronomical Society, 502, 4405
  * Bowman et al. (2018a) Bowman J. D., Rogers A. E. E., Monsalve R. A., Mozdzen T. J., Mahesh N., 2018a, Nature, 555, 67
  * Bowman et al. (2018b) Bowman J. D., Rogers A. E. E., Monsalve R. A., Mozdzen T. J., Mahesh N., 2018b, Nature, 564, E35
  * Bradley et al. (2019) Bradley R. F., Tauscher K., Rapetti D., Burns J. O., 2019, The Astrophysical Journal, 874, 153
  * Cohen et al. (2017) Cohen A., Fialkov A., Barkana R., Lotem M., 2017, Monthly Notices of the Royal Astronomical Society, 472, 1915
  * Cranmer et al. (2020) Cranmer K., Brehmer J., Louppe G., 2020, Proceedings of the National Academy of Sciences, 117, 30055
  * De Oliveira-Costa et al. (2008) De Oliveira-Costa A., Tegmark M., Gaensler B. M., Jonas J., Landecker T. L., Reich P., 2008, Monthly Notices of the Royal Astronomical Society, 388, 247
  * Dyson (1965) Dyson J., 1965, IEEE Transactions on Antennas and Propagation, 13, 488
  * Feroz & Hobson (2008) Feroz F., Hobson M. P., 2008, Monthly Notices of the Royal Astronomical Society, 384, 449
  * Feroz et al. (2009) Feroz F., Hobson M. P., Bridges M., 2009, Monthly Notices of the Royal Astronomical Society, 398, 1601
  * Fialkov & Barkana (2019) Fialkov A., Barkana R., 2019, Monthly Notices of the Royal Astronomical Society, 486, 1763
  * Furlanetto et al. (2006) Furlanetto S., Oh S. P., Briggs F., 2006, Physics Reports, 433, 181
  * Handley et al. (2015a) Handley W. J., Hobson M. P., Lasenby A. N., 2015a, Monthly Notices of the Royal Astronomical Society: Letters, 450, L61
  * Handley et al. (2015b) Handley W. J., Hobson M. P., Lasenby A. N., 2015b, Monthly Notices of the Royal Astronomical Society, 453, 4385
  * Hills et al. (2018) Hills R., Kulkarni G., Meerburg P. D., Puchwein E., 2018, Nature, 564, E32
  * Jana et al. (2019) Jana R., Nath B. B., Biermann P. L., 2019, Monthly Notices of the Royal Astronomical Society, 483, 5329
  * Jeffrey et al. (2020) Jeffrey N., Alsing J., Lanusse F., 2020, Monthly Notices of the Royal Astronomical Society, 501, 954
  * Kraus et al. (1986) Kraus J. D., Tiuri M., Räisänen A. V., Carr T. D., 1986, Radio Astronomy. Cygnus-Quasar Books
  * Liu et al. (2013) Liu A., Pritchard J. R., Tegmark M., Loeb A., 2013, Physical Review D, 87, 043002
  * MacKay (2003) MacKay D. J. C., 2003, Information Theory, Inference, and Learning Algorithms
  * Marin et al. (2011) Marin J.-M., Pudlo P., Robert C. P., Ryder R., 2011, Technical report, Approximate Bayesian Computational methods, https://ui.adsabs.harvard.edu/abs/2011arXiv1101.0955M. https://ui.adsabs.harvard.edu/abs/2011arXiv1101.0955M
  * Mirocha & Furlanetto (2019) Mirocha J., Furlanetto S. R., 2019, Monthly Notices of the Royal Astronomical Society, 483, 1980
  * Mittal & Kulkarni (2022) Mittal S., Kulkarni G., 2022, arXiv:2203.07733 [astro-ph]
  * Muñoz & Loeb (2018) Muñoz J. B., Loeb A., 2018, Nature, 557, 684
  * Papamakarios & Murray (2016) Papamakarios G., Murray I., 2016, Technical report, Fast $\epsilon$-free Inference of Simulation Models with Bayesian Conditional Density Estimation, https://ui.adsabs.harvard.edu/abs/2016arXiv160506376P. https://ui.adsabs.harvard.edu/abs/2016arXiv160506376P
  * Papamakarios et al. (2021) Papamakarios G., Nalisnick E., Rezende D. J., Mohamed S., Lakshminarayanan B., 2021, Journal of Machine Learning Research, 22, 1
  * Price et al. (2018) Price D. C., et al., 2018, Monthly Notices of the Royal Astronomical Society, 478, 4193
  * Pritchard & Loeb (2008) Pritchard J. R., Loeb A., 2008, Physical Review D, 78, 103511
  * Roque et al. (2021) Roque I. L. V., Handley W. J., Razavi-Ghods N., 2021, Monthly Notices of the Royal Astronomical Society, 505, 2638
  * Sims & Pober (2020) Sims P. H., Pober J. C., 2020, Monthly Notices of the Royal Astronomical Society, 492, 22
  * Singh & Subrahmanyan (2019) Singh S., Subrahmanyan R., 2019, The Astrophysical Journal, 880, 26
  * Singh et al. (2018) Singh S., Subrahmanyan R., Shankar N. U., Rao M. S., Girish B. S., Raghunathan A., Somashekar R., Srivani K. S., 2018, Experimental Astronomy, 45, 269
  * Singh et al. (2021) Singh S., et al., 2021, arXiv:2112.06778 [astro-ph]
  * Sivia & Skilling (2006) Sivia D. S., Skilling J., 2006, Data analysis: a Bayesian tutorial, 2nd ed edn. Oxford science publications, Oxford University Press, Oxford, England
  * Skilling (2006) Skilling J., 2006, Bayesian Analysis, 1, 833
  * de Lera Acedo (2019) de Lera Acedo E., 2019, in 2019 International Conference on Electromagnetics in Advanced Applications (ICEAA). pp 0626–0629, doi:10.1109/ICEAA.2019.8879199


# Generative Adversarial Network-Driven Detection of Adversarial Tasks in
Mobile Crowdsensing

Zhiyan Chen and Burak Kantarci  The authors are with the School of Electrical
Engineering and Computer Science, University of Ottawa, Ottawa, ON, Canada.
emails: {zchen241<EMAIL_ADDRESS>
###### Abstract

Mobile Crowdsensing systems are vulnerable to various attacks as they build on
non-dedicated and ubiquitous properties. Machine learning (ML)-based
approaches are widely investigated to build attack detection systems and
ensure MCS systems security. However, adversaries that aim to clog the sensing
front-end and MCS back-end leverage intelligent techniques, which are
challenging for MCS platform and service providers to develop appropriate
detection frameworks against these attacks. Generative Adversarial Networks
(GANs) have been applied to generate synthetic samples, that are extremely
similar to the real ones, deceiving classifiers such that the synthetic
samples are indistinguishable from the originals. Previous works suggest that
GAN-based attacks exhibit more crucial devastation than empirically designed
attack samples, and result in low detection rate at the MCS platform. With
this in mind, this paper aims to detect intelligently designed illegitimate
sensing service requests by integrating a GAN-based model. To this end, we
propose a two-level cascading classifier that combines the GAN discriminator
with a binary classifier to prevent adversarial fake tasks. Through
simulations, we compare our results to a single-level binary classifier, and
the numeric results show that proposed approach raises Adversarial Attack
Detection Rate (AADR), from $0\%$ to $97.5\%$ by KNN/NB, from $45.9\%$ to
$100\%$ by Decision Tree. Meanwhile, with two-levels classifiers, Original
Attack Detection Rate (OADR) improves for the three binary classifiers, with
comparison, such as NB from $26.1\%$ to $61.5\%$.

###### Index Terms:

Mobile Crowdsensing, Internet of Things, Machine Learning, GAN, Adversarial
Detection.

##  I Introduction

Mobile Crowdsensing (MCS) enables individuals to provide sensing services via
embedded sensors that are built in various smart equipment such as mobile
phones, connected vehicles, dedicated sensors and wearables. As it is an
effective business model for data acquisition, integration of MCS-based
services with various applications have been explored such as smart
transportation systems, smart grids, public safety, waste management, smart
mobility and smart living [1]. On the other hand, open, ubiquitous and non-
dedicated deployment of sensing services in MCS systems remain vulnerabilities
against data poisoning, fake sensing service requests, malware, and
eavesdropping [2, 3, 4].

Sensing service requests that aim at clogging the sensing and computing
resources are known as fake task submission attacks [5]. When adversaries do
not possess sufficient resources to design advanced and intelligent models to
generate adversarial tasks, the features that characterize the fake tasks are
determined empirically and following certain distributions. We refer to these
type of fake task attacks as original fake tasks. As they do not rely on
intelligent design, the success of detecting these type of tasks is quite high
as shown in the previous studies [5].

Adversaries with high capacity generate fake tasks by leveraging adversarial
machine learning. We refer to this type of fake tasks as adversarial fake
tasks. Fig. 1 illustrates a snapshot of MCS platform that receives two types
of tasks: legitimate tasks by normal users, and fake tasks (i.e., fake sensing
service requests) by adversaries with either high or low capacity. Without
training the fake task classifier (to pass legitimate tasks) in the MCS
platform with adversarial samples, it is more challenging to detect the fake
tasks that are generated by adversarial machine learning models. In our
previous study [6], we presented the potential of the adversaries with high
capacity to generate fake tasks to clog the MCS system resources.
Specifically, adversaries with high capacity deploy Generative Adversarial
Network (GAN) [7] to create fake tasks via training the generator model and
discriminator model in GAN. In the previous study, the evaluation results show
$0$ detection accuracy for generated fake tasks by GAN. Therefore, adversarial
fake tasks (i.e., fake tasks generated through adversarial machine learning
models) result in higher damages (i.e., low detection rate) to MCS systems
when compared to fake tasks generated by adversaries with low capability.

Figure 1:  Adversarial fake tasks, original fake tasks and benign tasks are
submitted to MCS platform

In this paper, we extend our previous work in [6] and adopt the same approach
to generate adversarial fake tasks. However, we aim to boost the detection
performance for the adversarial fake tasks to ensure security of MCS systems.
We propose a cascaded framework that consists of two-level classification: (1)
level 1 includes a trained discriminator in GAN, (2) level 2 consists of a
legacy binary classifier (e.g., KNN, Decision Tree (DT), Naive Bayes (NB)).
The discriminator is responsible for recognizing adversarial attacks before
feeding to level 2 classifiers. Fig. 2 demonstrates empirically designed fake
task attacks in the test dataset (by adversaries with low capacity) and
adversarial fake task attacks (by adversaries with high capacity) are fed to
the discriminator before sending to the MCS server.

Figure 2: Discriminator and two types of adversary injections

Fig. 3 demonstrates the proposed system architecture. A GAN model is trained
to generate adversarial fake tasks. The adversarial fake tasks are combined
with the original tasks in test dataset and form the mixed dataset. Then, the
trained discriminator model decides whether a task is adversarial or original
since the discriminator has been trained to distinguish synthetic samples from
original samples during the GAN training procedure. Tasks predicted as real by
the discriminator are fed into level 2 classifiers to estimate whether they
are fake or legitimate. Numeric results show that Adversarial Attack Detection
Rate (AADR) under the cascaded model is improved up to $97.5\%$ when KNN/NB is
employed in the second step, and up to $100\%$ when a Decision Tree is
employed in the second step. Moreover, Original Attack Detection Rate (OADR)
increases dramatically, from $26.1\%$ to $61.5\%$ under NB.

The rest of the paper is organized as follows. Section II presents the state
of the art in machine learning-backed security for MCS systems. In Section 3,
we introduce GAN-based adversarial task detection methodology in detail.
Section III presents the numerical results and evaluations regarding the
presented adversarial model. Finally, the paper is concluded in Section IV.

##  II Related Work and Motivation

As tackled by the previous studies and reviewed in [2], MCS systems confront
several security vulnerabilities, such as data poisoning, fake sensing data
and user privacy information leaking. Various methods have been investigated
to secure MCS systems and prevent from the threats against MCS platforms and
participants. The study in [8] uses Rome as the setting/terrain to acquire the
data, and leverages a neural network-based solution to improve the quantity of
vehicular crowd-sensed data. Moreover, the work in [9] tackles data poisoning
attacks on MCS platforms and the authors present the impact of data poisoning
attacks on several machine learning classifiers when poisoning data are
injected by the adversaries that are equipped with capabilities to build and
run AI models. The study in [10] proposes a Lightweight Low Rank and False
Matrix Separation (LightLRFMS) scheme to accelerate the computation time to
detect false sensing data at significantly higher accuracy levels in
comparison to the other separation / decomposition techniques.

Generative Adversarial Networks (GANs) are used to generate synthetic samples
by training two neural networks that compete with each other, namely a
discriminator and a generator. Typically, a GAN is capable of generating
samples to pursue data augmentation the training dataset, which is known as
GAN-based oversampling. For instance, the study in [11] demonstrates a GAN-
based augmentation approach to tackle the challenge of limited attack samples
in intelligent transportation systems. On the basis of the research in [12]
and other similar studies in the area, it is known that GANs can generate
synthetic pictures which are not visually distinguishable from their natural
samples. GANs can be utilized in various other use cases. For instance, the
authors in [13] present a case study in predicting growth of glioma via a
innovative approach integrating Gp-GAN. However, adversaries could also use
GANs to deceive the machine learning classifiers such as those deployed in MCS
platforms. Without proper training with synthetic samples, traditional machine
learning models are not capable to distinguish the generated (i.e, also known
as adversarial, synthetic) samples from real ones. Therefore, an effective
method is required to identify the adversarial patterns alongside the attacks
that are not the output of generative models but still aim at attacking the
system.

As studied in [6], when adversaries deploy GANs to generate fake tasks
injected into an MCS platform that already runs ML-based fake task detection.
In the absence of adversarial training, the detection rate is $0$ for these
adversarial attacks. The synthetic fake tasks pose more crucial challenges to
the MCS platform than original fake tasks. With this in mind, this paper
builds on the work in [6], with the motivation of boosting the detection rate
of the adversarial fake task attacks to secure the MCS systems.

Figure 3: System of GAN-based adversarial tasks detection

We design a novel framework that implements a two-level cascaded classifier
architecture to predict the generated attack samples and original (empirically
designed) attack samples, and filter them before distributing them to MCS
participants. The first level is formed by the GAN discriminator whereas the
second level is a binary classifier. According to the system design in Fig.3,
the training dataset is utilized to train binary classifiers (e.g., KNN, NB
and DT) and the adversarial ML model, which is a GAN in this study. The
training dataset includes the fake tasks of the adversaries in the first
category, i.e, adversaries with low capacity (original fake tasks). GANs take
input samples of noise, and output adversarial (i.e., synthetic) samples [14].
An adversary with high capacity can design and train a GAN with its two neural
network modules that are in competition with each other, namely the generator
neural network and the discriminator neural network. The role of the generator
is to create synthetic samples as similar as the real ones whereas a
Discriminator designs to distinguish real samples from the synthetic ones. The
motive for the competition is to reduce the gap between generated samples for
Generator and increase detection accuracy for Discriminator [15]. On the basis
of this motivation, the Generator and Discriminator participate in a non-
cooperative game to reach a Nash Equilibrium [16].

Algorithm 1 Tasks in Mixed Dataset prediction procedure by Discriminator and
classifier

1:$OriginalDataset$ = {Fake Sensing Tasks by low-capacity adversaries}

2:$GANDataset$ = {Fake Sensing Tasks by high- capacity adversaries}

3:$MixedDataset$ = $OriginalDataset\cup\ GANDataset$

4:$Label(Task_{i})$ = 1, $\forall$($i|\ i\in MixedDataset$ $\wedge$ i:
Original Task)

5:$Label(Task_{i})$ = 0, $\forall$($i|\ i\in MixedDataset$ $\wedge$ i:
Synthetic Task)

6:begin

7:$Task_{i}\in MixedDataset$

8: $DP_{i}=Prediction\ of\ Discriminator\ for\ Task_{i}\ $

9: if $DP_{i}=1$ then $\triangleright$ $Task_{i}$ is original (not synthetic).
$\triangleright$ Either legitimate (1) or illegitimate (0)

10: Send $Task_{i}$ to binary classifier

11: $CP_{i}=Prediction\ of\ binary\ classifier\ for\ Task_{i}$

12: if $CP_{i}=1$ then
$\triangleright$ Classified as legitimate

13: Accept sensing task request, pursue recruit participants

14: else

15: Eliminate $Task_{i}$ By binary classifier

16: end if

17: else

18: Eliminate $Task_{i}$ by Discriminator

19: end if

20:end

The generated fake tasks are injected to the MCS platform as sensing service
requests. As the MCS platform is equipped with a Machine Learning (ML)-based
decision system, handling of the synthetic tasks is interpreted from the
standpoint of ML. In terms of ML, this means that the test dataset is injected
synthetic fake tasks so these samples mix with the fake tasks of the
adversaries with low capacity (i.e., original / empirically-designed fake
tasks). Samples in this mixed dataset have two clusters, including real tasks
in the test dataset (adversaries with low capacity, original fake tasks and
legitimate tasks) and synthetic tasks (adversaries with high capacity,
adversarial fake tasks). The Discriminator receives this mixed dataset, and it
is already trained during the fake task creation procedure through the
Generator of the GAN to distinguish synthetic from real samples so it can
classify the two clusters in the mixed dataset. The Discriminator runs an
estimation for the samples in the mixed dataset and filters out the samples
predicted as ’synthetic fake task’ before sending them to the level-2
classifier (e.g, KNN, NB, DT). In our previous work [6], we showed that the
ML-backed task detection module of the MCS platform is challenged by the
adversarial fake tasks of the generator so in this paper, the Discriminator is
utilized with motivation to improve the detection rate of the adversarial
(synthetic) fake tasks. The rationale for bridging these two levels of
prediction is as follows: It is possible that the Discriminator could predict
a synthetic task as real, which will still be sent to the ML classifiers for
prediction.

Following upon the prediction of the Discriminator, the sensing tasks that are
estimated as ’real’ are picked. An real instance can have two classes:
original (i.e., empirically designed) fake sensing tasks and legitimate
sensing tasks. As mentioned earlier, the Discriminator is trained to detect
and filter out the adversarial (GAN-generated) fake tasks. Thus, fake tasks
attacks of the high capacity adversaries generated through a generative model
are expected to be eliminated prior to the low capacity ML classifiers such as
KNN, NB, or DT. As a result, these classifiers are expected to detect and
eliminate the fake tasks submitted by the low capacity adversaries that set
the task features through empirical analysis. Indeed, this two-level ML-backed
fake sensing task detection system in the MCS platform aims to eliminate all
fake tasks to maintain the MCS services available without the platform being
clogged. Algorithm 1 describes the two-step prediction procedure for a sensing
task in the mixed MCS dataset through the Discriminator and binary
classifiers.

##  III Numerical Results and Analyses

This section presents the evaluation results and analyses regarding the impact
of defensive use of the two-stage fake task detection to filter out the
adversarial (submitted by high capacity adversaries) and original (submitted
by low capacity adversaries) fake tasks in the MCS platforms.

###  III-A Dataset introduction and Assumptions

The dataset used in this study is generated via a realistic MCS simulator,
namely CrowdSenSim [17]. We use realistic settings in Crowdsensim to simulate
an MCS platform. The simulation configuration is shown in Table I, and adopted
from [5]. Clarence-Rockland is set as the city to conduct crowdsensing
activities. A total of $14,484$ tasks have been generated under Crowdsensim
with a breakdown of $12,587$ legitimate and $1,897$ illegitimate (fake) tasks.
Train:test split of the dataset is set at 80%:20%. Legitimate and ”original”
fake tasks are generated based on the features/settings in the table.
Hereafter, we refer to the fake tasks generated by the adversaries with low
capacity (i.e., fake tasks that are empirically generated) as the original
fake tasks.

The rationale for the configuration of the fake task features in the
simulation environment is that fake tasks aim to drain more resources from
sensing service providing devices. Moreover, fake tasks seek longer sensing
duration and higher volume sensing data as the ultimate goal is to clog the
MCS sensing servers so the MCS platform and the sensing devices become
unavailable. The feature set is adopted from [5] to extract an MCS task with
the features listed as follows: {’ID’, ’latitude’, ’longitude’, ’day’, ’hour’,
’minute’, ’duration’, ’remaining time’, ’battery requirement (percentage)’,
’Coverage’, ’legitimacy’, ’GridNumber’, ’OnpeakHour’}.

Figure 4: One round training loss for GAN TABLE I:  CrowdSenSin simulation setup for low capacity adversaries (adopted from [5]). |  Fake Tasks |  Legitimate Tasks
---|---|---
Day | Uniformly distributed in [1, 6]
Hour (task) |  80% : 7am to 11am; 20%: 12pm to 5pm |  8%: 0pm to 5am; 92%: 6am to 23pm
Duration (task) |  70% in {40, 50, 60} [minutes]; 30% in {10, 20, 30} [minutes] |  Uniformly distributed over {10, 20, 30, 40, 50, 60} mins
Battery usage (%) |  80% in {7%-10%}; 20% in {1% -6%} |  Uniformly distributed in {1%-10%}
Recruitment Radius | Uniformly distributed in 30m to 100m
Movement Radius | [10m, 80m]
Number of tasks | 14,484

###  III-B Adversarial task detection performance by Discriminator

As illustrated in Fig 3, the GAN model is trained with the training dataset so
to have the adversaries with high capacity create fake sensing requests (i.e.,
tasks) via its Generator network. Discriminator and Generator are trained
separately in batches.

TABLE II: GAN model hyper-parameters. G:Generator, D:Discriminator, LR:Learning Rate, AF: Activation Function | LR | # of Neurons | AF | BatchSize | # of Epochs
---|---|---|---|---|---
G | 0.01 | 256/512/1024 | tanh | 15,20,25,32 | 2k,4k,8k
D | 0.01 | 512/256/256 | sigmoid | 15,20,25,32 | 2k,4k,8k

As introduced before, dataset is split into training and test sets with a
split ratio of 80%-20%. Training set contains $1,506$ fake sensing tasks and
$10,081$ legitimate tasks whereas the test set contains $391$ fake sensing
tasks and $2,506$ legitimate tasks. It takes $2,000$ epochs for GAN to train
itself. In order to demonstrate the reliability of test results, we repeat
each test 20 times and present the average below in the results. TABLE II
demonstrates hyperparameters for GAN model. Specifically, in the Generator
network, three layers are constructed with $256$, $512$ and $1024$ neurons
respectively. Meanwhile, the activation function is selected as ’tanh’. On the
other hand, in the Discriminator network, there are $512$, $256$, and $256$
neurons in each layer and the activation function is ’sigmoid’. Meanwhile,
different batch sizes are applied to train the GAN model such as 15, 20, 32,
and 64 in hyperparameter procedure. Finally, batch size of 32 is determined.
Fig.4 demonstrates one round training loss for GAN, which presents the loss
for every 10 epochs. It shows ”loss real” for training loss for the
Discriminator using real samples in the training dataset, ”loss fake” for the
training loss for the Discriminator using the generated samples by the
Generator, and finally ”loss GAN”. In Fig. 4, network convergence is observed
in $2000$ epochs. When training procedure completes, the Generator is used to
generate a total of $2,000$ fake sensing requests. These $2,000$ fake tasks
are designed by the adversaries with high capacity, and they are combined with
the original fake sensing tasks in the test dataset to form the mixed dataset.
It means there are two classes in the mixed dataset. We label real tasks in
test dataset as class $1$ for both legitimate and fake tasks, and synthetic
tasks as class $0$.

TABLE III: Mixed dataset sample distribution | Class 0 (Adversarial) | Class 1 (Real)
---|---|---
Num of Fake Task | 2000 | 391
Num of Legitimate Task | 0 | 2506
Total | 2000 | 2897

The mixed dataset contains $2,897$ real tasks in test dataset (e.g., $391$
original fake tasks and $2,506$ legitimate tasks) and $2,000$ adversarial fake
tasks, as shown in TABLE III. The discriminator pursues binary classification
to distinguish real tasks from adversarial tasks. TABLE IV presents the
prediction confusion matrix, which represents the average of 20-round results.
The Discriminator filters out samples that are predicted as adversaries,
including the estimated real as adversaries (False Negative (FN)) and
predicted as adversaries as adversaries(True Negative (TN)). On the other
hand, the samples are predicted as real, including the prediction of real as
real (True Positive (TP)) and the prediction of adversarial as real (False
Positive (FP)) samples. Column IsFilterOut in TABLE IV denotes whether these
samples would be removed or sent to the next-level classifier.

TABLE IV: Distribution of the Discriminator predictions. Adver. means Adversarial. Prediction | Task Type | Number of Tasks | IsFilterOut
---|---|---|---
Real as Real | Original fake | 175.1 | No
Legitimate | 2091.1 | No
Adver. as Real | Adversarial fake | 50.7 | No
Legitimate | N/A | No
Real as Adver. | Original fake | 215.9 | Yes
Legitimate | 414.9 | Yes
Adver. as Adver. | Adversarial fake | 1949.3 | Yes
Legitimate | N/A | Yes
TABLE V: Percentage of sensing tasks eliminated by the Discriminator due to predicting as adversaries | Num of prediction | Total | %
---|---|---|---
Original Fake | 215.9 | 391 | 55.2%
Legitimate | 414.9 | 2506 | 16.6%
Adversarial Fake | 1949.3 | 2000 | 97.5%

As seen in TABLE IV, the Discriminator predicts $50.7$ adversarial fake tasks
as real tasks, which are marked to be sent to the next level classifier
(KNN/NB/DT) to determine make a final decision of them being fake or
legitimate. Meanwhile, as seen in the FN breakdown, out of $630.8$ real tasks,
$215.9$ fake sensing requests and $414.9$ legitimate sensing requests are
predicted as synthetic ones, which are eliminated before being sent to the
next-level classifier. Table V illustrates the elimination percentage for real
tasks and adversarial fake tasks. The Discriminator detection rate for
synthetic samples is up to $97.5\%$, which means that most generated samples
by the adversaries with high capacity are prevented in the MCS systems.
Moreover, the Discriminator marks $55.2\%$ original fake tasks as synthetic
tasks and excludes them. However, $16.6\%$ ($414.9$) legitimate tasks are
estimated as synthetic ones and eradicated. Therefore, there is trade-off
between fake tasks detection and legitimate tasks completion.

TABLE VI: KNN, NB, and DT correct prediction of fake sensing tasks with/without Discriminator HasDiscriminator | FakeTaskFrom | KNN | NB | DT
---|---|---|---|---
No | Original | 299 | 102 | 391
Adversarial | 0 | 0 | 918.8
Yes | Original | 129.5 | 24.5 | 175.1
Adversarial | 0 | 0 | 50.7

###  III-C Adversarial task detection performance by the second level
classifiers

Following upon the Discriminator’s filtering, the remaining samples are fed to
the next-level (i.e., binary) classifiers, which are trained with the training
dataset. The distribution of the remaining samples is presented in Table IV
for the TP and FP prediction samples. There are $2,316.9$ tasks in total, with
$225.8$ fake tasks ($175.1$ original fake tasks and $50.7$ adversarial fake
tasks) and $2,091.1$ legitimate tasks.

Three classifiers are selected for the fake task detection in the MCS
platform, including Naive Bayes (NB), Decision Tree (DT), and K-Nearest
Neighbors (KNN) to predict whether a sensing task is legitimate or fake. In
order to present the effectiveness of the proposed architecture, we compare
the performance of the two-level architecture to the architecture without the
Discriminator, which stands for the case where the mixed dataset in TABLE III
is directly fed to the binary classification.

Three metrics, including Adversarial Attack Success Rate (AASR), Adversarial
Attack Detection Rate (AADR), and Original Attack Detection Rate (OADR), are
used to evaluate the system performance. AADR (1) represents the ability of a
defense system to prevent adversarial attacks, which is the ratio of detected
adversarial attacks to the total adversarial instances [18] .

$AADR=\frac{DA\\_DIS+DA\\_CLA}{TotalAdversarial}$ (1)

In Eq. 1, $DA\\_DIS$ denotes the number of detected adversarial attack samples
by the Discriminator, and $DA\\_CLA$ means the number of detected adversarial
attack samples by a classifier (e.g., KNN,NB,DT). Since our simulations
generate $2,000$ fake tasks by the Generator, the value of $TotalAdversarial$
is $2,000$.

AASR in represents the adversarial attack successful injection rate. Sum of
AADR and AASR ($AASR+AADR=1$) is $100\%$, which means an adversarial attack
either be detected or injected successfully to an MCS system.

Original Attack Detection Rate (OADR) shows the effectiveness for the
detection system in identifying the original fake sensing requests submitted
by adversaries with low capacity. The formulation shows in (2).

$OADR=\frac{DO\\_DIS+DO\\_CLA}{Total\\_Original\\_Attacks}$ (2)

In Eq. 2, $DO\\_DIS$ represents the number of detected original fake tasks by
the Discriminator and $DO\\_CLA$ denotes the number of detected original fake
tasks by a classifier (e.g., KNN,NB,DT). As introduced before, there are $391$
fake sensing tasks in the test dataset so $Total\\_Original\\_Attacks$ is
$391$.

TABLE VII: With/Without Discriminator results comparison for AASR, AADR, and OADR with/without Discriminator | HasDiscr | Contributed By | KNN | NB | DT
---|---|---|---|---|---
AASR | No | Classifier | 1.000 | 1.000 | 0.459
Yes | Discriminator | 0.025 | 0.025 | 0.025
Classifier | 0.025 | 0.025 | 0
Finally | 0.025 | 0.025 | 0
AADR | No | Classifier | 0.000 | 0.000 | 0.459
Yes | Discriminator | 0.975 | 0.975 | 0.975
Classifier | 0.975 | 0.975 | 1.000
Finally | 0.975 | 0.975 | 1.000
OADR | No | Classifier | 0.765 | 0.261 | 1.000
Yes | Discriminator | 0.552 | 0.552 | 0.552
Classifier | 0.331 | 0.063 | 0.448
Finally | 0.883 | 0.615 | 1.000

TABLE VI demonstrates the number of correct predictions for fake tasks by
classifiers (e.g., KNN, NB and DT), which contains original fake sensing
requests and fake sensing requests designed by high capacity adversaries. As
seen in the table, KNN and NB cannot detect any adversarial tasks regardless
of the presence of the Discriminator. DT can recognize adversarial fake tasks
under both scenarios (i.e., with or without the Discriminator). It is worth to
note that with the Discriminator, $50.7$ original fake tasks are identified
correctly by DT, which is much lower than $918.8$ without the Discriminator
case. This is because the Discriminator eliminates $215.9$ original fake tasks
as shown in Table IV. According to the prediction results by the Discriminator
and classifiers, as well as AASR, AADR, and OADR metrics (see Eq. 1 and 2), we
calculate these three metrics.

AASR, AADR, and OADR results are shown in TABLE VII. It is evident that the
architecture with the Discriminator results improves performance when compared
to the architecture without the Discriminator. For instance, KNN and NB
perform $100\%$ AASR without the Discriminator, that denotes KNN and NB have
no ability to prevent adversarial attacks. On the other hand, AASR is reduced
significantly to $2.5\%$ by using Discriminator in TABLE VII. Moreover, DT
identifies $2.5\%$ of the adversarial fake tasks, resulting in $0\%$ AASR
finally. Regarding to AADR, it can be calculated according to AASR in Eq. (1).
Moreover, a lower AASR leads to a higher AADR in TABLE VII. Original fake
tasks are filtered by the Discriminator firstly, and then by binary
classifiers so the finally OADR is the sum of OADR by the Discriminator and
OADR by the second-level classifier. DT results in the best detection
performance up $100\%$ accuracy for both with and without Discriminator cases.
The original fake task detection rate is improved, when with and without the
Discriminator architectures are compared. The improvement is from $76.5\%$ to
$88.3\%$ under KNN and from $26.1\%$ to $61.5\%$ under NB.

##  IV Conclusion and Open Issues

Adversaries with high capacity have the potential to integrate intelligent
techniques to design attacks on Mobile Crowdsensing (MCS) systems, which can
lead to more damage than normal attacks due to the low detection rate via
traditional ML-based classifiers. In this paper, we have adopted the previous
work to generate attack samples via GAN and we propose a novel two-level
framework to integrate the Discriminator of a Generative Adversarial Network
(GAN) and low-capacity binary classifiers. Binary classifiers (e.g., KNN,
Naive Bayes (NB), Decision Tree (DT)) are used to estimate whether a sensing
task is fake or legitimate. Numerical results illustrate that the proposed
approach boosts the Adversarial Attack Detection Rate (AADR) when compared to
the detection model without the Discriminator. Thus, detection performance of
KNN and NB increase from $0$ to $97.5\%$, respectively whereas DT raises from
$45.9\%$ to $100\%$. Thus, all generated adversarial fake sensing requests are
identified by the DT in the second level, and AASR is $0$. Meanwhile, the
proposed method increases the performance for the original fake sensing tasks,
with OADR boosting from $76.5\%$ to $88.3\%$ under KNN, $26.1\%$ to $61.5\%$
under NB. DT achieves $100\%$ in terms of OADR, which means all original fake
tasks are filtered out. DT achieves the best performance than NB and DT.

Indeed, maintaining the effectiveness of an MCS system while eliminating the
fake sensing requests by high capacity adversaries remains an open issue,
which will be tackled by our ongoing research agenda.

## References

  * [1] A. Capponi, C. Fiandrino, B. Kantarci, L. Foschini, D. Kliazovich, and P. Bouvry, “A survey on mobile crowdsensing systems: Challenges, solutions and opportunities,” IEEE Communications Surveys Tutorials, vol. 21, no. 3, pp. 2419–2465, 2019.
  * [2] Z. Chen, C. Fiandrino, and B. Kantarci, “On blockchain integration into mobile crowdsensing via smart embedded devices: A comprehensive survey,” Journal of Systems Architecture, p. 102011, 2021.
  * [3] Y. Liu, Z. Ma, X. Liu, S. Ma, S. Nepal, R. H. Deng, and K. Ren, “Boosting privately: Federated extreme gradient boosting for mobile crowdsensing,” in 2020 IEEE 40th International Conference on Distributed Computing Systems (ICDCS), pp. 1–11, 2020.
  * [4] A. Smahi, Q. Xia, H. Xia, N. Sulemana, A. A. Fateh, J. Gao, X. Du, and M. Guizani, “A blockchainized privacy-preserving support vector machine classification on mobile crowd sensed data,” Pervasive and Mobile Computing, vol. 66, p. 101195, 2020.
  * [5] Y. Zhang and B. Kantarci, “Ai-based security design of mobile crowdsensing systems: Review, challenges and case studies,” in IEEE International Conference on Service-Oriented System Engineering (SOSE), pp. 17–1709, 2019\.
  * [6] Z. Chen and B. Kantarci, “Adversarial machine learning-driven fake task anticipation in mobile crowdsensing systems,” in 2021 IEEE International Conference on Service-Oriented System Engineering (SOSE), pp. 57–63, 2021.
  * [7] A. Creswell, T. White, V. Dumoulin, K. Arulkumaran, B. Sengupta, and A. A. Bharath, “Generative adversarial networks: An overview,” IEEE Signal Processing Magazine, vol. 35, no. 1, pp. 53–65, 2018.
  * [8] X. Zhu, Y. Luo, A. Liu, W. Tang, and M. Z. A. Bhuiyan, “A deep learning-based mobile crowdsensing scheme by predicting vehicle mobility,” IEEE Transactions on Intelligent Transportation Systems, 2020.
  * [9] A. Prud’Homme and B. Kantarci, “Poisoning attack anticipation in mobile crowdsensing: A competitive learning-based study,” in Proceedings of the 3rd ACM Workshop on Wireless Security and Machine Learning, pp. 73–78, 2021\.
  * [10] X. Li, K. Xie, X. Wang, G. Xie, D. Xie, Z. Li, J. Wen, Z. Diao, and T. Wang, “Quick and accurate false data detection in mobile crowd sensing,” IEEE/ACM Transactions on Networking, vol. 28, no. 3, pp. 1339–1352, 2020.
  * [11] W. Bouzeraib, A. Ghenai, and N. Zeghib, “A multi-objective genetic gan oversampling: Application to intelligent transport anomaly detection$\backslash$,” in IEEE 22nd Intl. Conf. on High Performance Computing and Communications; IEEE 18th Intl. Conf. on Smart City; IEEE 6th Intl. Conf. on Data Science and Systems (HPCC/SmartCity/DSS), pp. 1142–1149, 2020.
  * [12] M. Barni, K. Kallas, E. Nowroozi, and B. Tondi, “Cnn detection of gan-generated face images based on cross-band co-occurrences analysis,” in 2020 IEEE International Workshop on Information Forensics and Security (WIFS), pp. 1–6, IEEE, 2020.
  * [13] A. Elazab, C. Wang, S. J. S. Gardezi, H. Bai, Q. Hu, T. Wang, C. Chang, and B. Lei, “Gp-gan: Brain tumor growth prediction using stacked 3d generative adversarial networks from longitudinal mr images,” Neural Networks, vol. 132, pp. 321–332, 2020.
  * [14] I. J. Goodfellow, J. Pouget-Abadie, M. Mirza, B. Xu, D. Warde-Farley, S. Ozair, A. Courville, and Y. Bengio, “Generative adversarial nets,” in Proceedings of the 27th International Conference on Neural Information Processing Systems - Volume 2, NIPS’14, (Cambridge, MA, USA), p. 2672–2680, MIT Press, 2014.
  * [15] K. Wang, C. Gou, Y. Duan, Y. Lin, X. Zheng, and F.-Y. Wang, “Generative adversarial networks: introduction and outlook,” IEEE/CAA Journal of Automatica Sinica, vol. 4, no. 4, pp. 588–598, 2017.
  * [16] T. Salimans, I. Goodfellow, W. Zaremba, V. Cheung, A. Radford, and X. Chen, “Improved techniques for training gans,” Advances in neural information processing systems, vol. 29, pp. 2234–2242, 2016.
  * [17] C. F. et al, “CrowdSenSim: a simulation platform for mobile crowdsensing in urban environments,” IEEE Access, vol. 5, pp. 3490–3503, 2017.
  * [18] A. Piplai, S. S. L. Chukkapalli, and A. Joshi, “Nattack! adversarial attacks to bypass a GAN based classifier trained to detect network intrusion,” in IEEE 6th Intl. Conference on Big Data Security on Cloud (BigDataSecurity), IEEE Intl Conference on High Performance and Smart Computing, (HPSC) and IEEE Intl Conference on Intelligent Data and Security (IDS), pp. 49–54, 2020.