# Adv-MIA: Efficient Membership Inference Attacks on DNNs using Adversarial
Perturbations

Hassan Ali1, Adnan Qayyum1, Ala Al-Fuqaha2, Junaid Qadir3 1IHSAN Lab,
Information Technology University, Lahore, Pakistan
<EMAIL_ADDRESS><EMAIL_ADDRESS>
2Hamad Bin Khalifa University, Qatar (aalfuqaha@hbku.edu.qa).
3Corresponding author (jqadir@qu.edu.qa). Qatar University, Doha, Qatar.

# Membership Inference Attacks on DNNs using Adversarial Perturbations

Hassan Ali1, Adnan Qayyum1, Ala Al-Fuqaha2, Junaid Qadir3 1IHSAN Lab,
Information Technology University, Lahore, Pakistan
<EMAIL_ADDRESS><EMAIL_ADDRESS>
2Hamad Bin Khalifa University, Qatar (aalfuqaha@hbku.edu.qa).
3Corresponding author (jqadir@qu.edu.qa). Qatar University, Doha, Qatar.

###### Abstract

Several membership inference (MI) attacks have been proposed to audit a target
DNN. Given a set of subjects, MI attacks tell which subjects the target DNN
has seen during training. This work focuses on the post-training MI attacks
emphasizing high confidence membership detection—True Positive Rates (TPR) at
low False Positive Rates (FPR). Current works in this category—likelihood
ratio attack (LiRA) and enhanced MI attack (EMIA)—only perform well on complex
datasets (e.g., CIFAR-10 and Imagenet) where the target DNN overfits its train
set, but perform poorly on simpler datasets (0% TPR by both attacks on
Fashion-MNIST, 2% and 0% TPR respectively by LiRA and EMIA on MNIST at 1%
FPR). To address this, firstly, we unify current MI attacks by presenting a
framework divided into three stages—preparation, indication and decision.
Secondly, we utilize the framework to propose two novel attacks: (1)
Adversarial Membership Inference Attack (AMIA) efficiently utilizes the
membership and the non-membership information of the subjects while
adversarially minimizing a novel loss function, achieving 6% TPR on both
Fashion-MNIST and MNIST datasets; and (2) Enhanced AMIA (E-AMIA) combines EMIA
and AMIA to achieve 8% and 4% TPRs on Fashion-MNIST and MNIST datasets
respectively, at 1% FPR. Thirdly, we introduce two novel augmented indicators
that positively leverage the loss information in the Gaussian neighborhood of
a subject. This improves TPR of all four attacks on average by 2.5% and 0.25%
respectively on Fashion-MNIST and MNIST datasets at 1% FPR. Finally, we
propose simple, yet novel, evaluation metric, the running TPR average (RTA) at
a given FPR, that better distinguishes different MI attacks in the low FPR
region. We also show that AMIA and E-AMIA are more transferable to the unknown
DNNs (other than the target DNN) and are more robust to DP-SGD training as
compared to LiRA and EMIA.

###### Index Terms:

Membership inference attacks, Deep Learning, Deep Neural Networks, Data
privacy

##  I Introduction

Deep Learning (DL) algorithms, particularly Deep Neural Networks (DNNs), are
now being used for decision-making in several sensitive domains, such as IoT
devices, healthcare [1], and purchase recommendations [2], where an
individual’s data privacy is the first and the foremost concern [3, 4, 5].
However, several previous works have shown that DNNs can leak sensitive
information about their training data. Privacy attacks [6, 7], most notably
membership inference (MI) attacks [2, 3, 4, 8, 9], exploit this leakage to
quantify the privacy of a target DNN. Given a trained target DNN and a subject
($x$:input to DNN, $y$:ground truth), an MI attack infers the training
membership of the subject—whether or not the target DNN has seen the subject
during training. This work focuses on the inference time MI attacks on the
target DNN with emphasis on high confidence membership detection—True Positive
Rates (TPR) at low False Positive Rates (FPR) (see Section V-A3 for
definitions).

Carlini et al. [3] note that most of the previous attacks are only good at
inferring the non-membership of a subject, and therefore, should instead be
called non-membership inference (non-MI) attacks. non-MI attacks are not
suitable for the threat setting where it is desired to infer the membership of
a subject with high confidence. Therefore, Carlini et al. [3] propose the
Likelihood Ratio Attack (LiRA) in two variations—online LiRA (n-LiRA) and
offline LiRA (f-LiRA)—along with a novel quantitative metric—TPR-FPR curve (in
log scale)—to outperform previous attacks in membership inference,
specifically in the low FPR region.

However, n-LiRA is computationally inefficient—given $k$ subjects, the
membership of which is to be inferred, n-LiRA requires training of $2kN$
shadow DNNs, where $N$ and $k$ are typically in hundreds [3]—making it almost
impossible for the future researchers to reproduce the attack, and for
practitioners to audit a target DNN in limited computational settings, which
is often the only feasible option. f-LiRA is computationally more
efficient—requiring $N$ shadow DNNs to be trained for $k$ subjects—but notably
less effective [3]. To address this, Ye et al. [4] propose Enhanced Membership
Inference Attack (EMIA) that uses soft labels from the target DNN to train the
shadow DNNs. EMIA performs best in the high FPR region and comparably to
n-LiRA in the low FPR region while being significantly computationally
efficient—requiring $N$ shadow DNNs to be trained for $k$ subjects.

Limitations and Challenges: Despite being effective on complex CIFAR-10,
CIFAR-100, and Imagenet datasets, both f-LiRA and EMIA do not perform well on
MNIST [4] and Fashion-MNIST datasets (Section V-C). One possible reason is
that the experimental setup of MI attacks only allows training the target DNN
on half of the training set that causes the DNN to overfit, yielding a large
train-test accuracy gap for complex datasets [3, 4]. The overfitted target DNN
leaks private information to even relatively weaker MI attacks. This is a
major limitation, as in practical scenarios, the target DNN is less likely to
overfit its training set. Previous works have also observed the overfitting of
the target DNN [3, 4, 10], but neither of them specifically relates it to MI
attacks under-performing on MNIST and Fashion-MNIST, while being effective on
complex datasets. We hypothesize two main reasons for this limitation detailed
below.

Both f-LiRA and EMIA only leverage the non-membership information of the
subject (how does the absence of the subject from the training set affect the
behavior of the target DNN?) while ignoring its membership information (how
does the inclusion of the subject in the training set affect the behavior of
the target DNN?) in order to gain the computational advantage. As noted by
Carlini et al. [3], the membership information plays a key role in inferring
the membership of the subjects that are hard-to-fit.

Both f-LiRA and EMIA use vanilla membership indicators that examine the loss
of the target DNN precisely over the subjects without considering the loss
landscape in the neighborhood (e.g., the Gaussian neighborhood) of the
subjects. However, local loss landscape around the subject might be helpful
for membership inference [11], highlighting the need for augmented membership
indicators. However, Carlini et al. [3] observe that simply ensembling the
loss in the Gaussian neighborhood of the subject does not improve MI attacks’
performance.

These limitations pose the following research challenges:

  1. 1.

How to utilize the membership information of the subjects while being
computationally efficient?

  2. 2.

How to positively leverage the loss information in the Gaussian neighborhood
of a given subject?

Findings and Contributions: In this paper, we first present a unified
framework based on the working of current MI attacks by dividing their
algorithms into three stages. The preparation stage prepares the variables
$\mathcal{V}$ that may include transformation functions or shadow DNNs used to
guide the subsequent stages. The indication stage defines a membership
indicator $\mathcal{I}$ that, guided by $\mathcal{V}$, indicates the
likelihood of a subject being the training set member. The decision stage
applies a threshold $\tau$ to $\mathcal{I}$ to finally decide whether the
subject is a member or not. For example, LiRA and EMIA respectively train
$2kN$ and $N$ shadow DNNs as $\mathcal{V}$, and use the likelihood ratio as
$\mathcal{I}$.

Building upon the proposed framework, we contribute towards a better
$\mathcal{V}$ by developing an Adversarial Membership Inference Attack (AMIA)
that utilizes both, the membership and the non-membership information of a
subject while being computationally efficient (only requiring $2N$ shadow DNNs
to be trained for $k$ subjects). Given a target DNN and a set of $k$ subjects,
AMIA first trains $2N$ shadow DNNs, where each shadow DNN is trained with
$k/2$ randomly selected subjects augmented with the original training set as a
batch for computational efficiency (Challenge 1), thereby, holding the (batch-
wise) membership information of these $k/2$ subjects and the non-membership
information of the remaining $k/2$ subjects. AMIA then adversarially computes
small magnitude perturbations to the input of each subject that maximizes the
difference between the loss of the member and the non-member shadow DNNs. This
lets AMIA exploit the loss landscape of the shadow DNNs in the local
neighborhood of the subject (Challenge 2) optimally for the membership
inference purpose. AMIA notably outperforms both f-LiRA and EMIA on Fashion-
MNIST and MNIST datasets and performs on par with f-LiRA (while outperforming
EMIA) on the CIFAR-10 dataset.

We also propose Enhanced AMIA (E-AMIA) that exploits soft labels of EMIA to
train the shadow DNNs of AMIA. E-AMIA shows comparable performance to AMIA on
Fashion-MNIST and MNIST datasets, while notably outperforming others on the
CIFAR-10 dataset.

We contribute towards a better $\mathcal{I}$ by extending the likelihood
ratio, a vanilla indicator proposed by Carlini et al. [3], to define two
augmented indicators that exploit the loss landscape in the Gaussian
neighborhood of a subject to infer its membership (Challenge 2). Our proposed
augmented indicators notably improve the performance of all the MI attacks
considered in this paper, i.e., the previously proposed attacks (f-LiRA and
EMIA) and the newly proposed attacks (AMIA and E-AMIA).

We also present a better evaluation metric of MI attacks in the low FPR
region. More specifically, we find that the running average of TPR (RTA) at a
given FPR (in log scale) better distinguishes between stronger and weaker
attacks in the low FPR region as compared to the TPR at a given FPR (in log
scale) proposed by Carlini et al. [3].

Finally, we study how well do the variables $\mathcal{V}$ prepared for $k$
subjects by an MI attack for the target DNN transfer to the unknown DNNs
trained on the same dataset. We find that EMIA is less transferable to the
unknown DNNs as compared to f-LiRA even in the high FPR region where EMIA
outperfroms others on the target DNN. This is because EMIA uses soft labels
generated by the target DNN which makes it more customized to the target DNN.
Interestingly, our proposed AMIA and E-AMIA, in addition to performing better
than LiRA and EMIA on the target DNN, transfer well to the unknown DNNs. We
attribute this to the transferability property of the adversarial
perturbations used by AMIA and E-AMIA.

Our findings and contributions are summarized below.

  1. 1.

We unify MI attacks in the literature under a framework divided into three
stages—preparation, indication, and decision stages (which are formalized in
Section III-B).

  2. 2.

We propose a compute-efficient methodology that leverages subject membership
information and adversarial perturbations to effectively distinguish members
from non-members on MNIST and Fashion-MNIST datasets.

  3. 3.

We propose improvements over the likelihood ratio indicator (used by both
f-LiRA and EMIA) by leveraging the local loss information in the Gaussian
neighborhood of the subject. More specifically, we perturb each subject with
the Gaussian noise and compute the likelihood ratio of each perturbed sample
separately, which notably improves the effectiveness of the attack.

  4. 4.

We study the transferability of MI attacks by using $\mathcal{V}$ prepared for
a target DNN to perform membership inference of the subjects on unknown DNNs.
Our results indicate that AMIA is the most transferable, respectively followed
by E-AMIA and f-LiRA (which give comparable performance) while EMIA is the
least transferable.

The source code will be made available at:
https://github.com/hassanalikhatim/AMIA.

##  II Related Work

Recent works have extensively highlighted several limitations of DNNs in real-
world safety-critical scenarios. For example, DNNs are vulnerable to
explainability attacks [12], adversarial attacks [13, 14], bias [15], data
imperfections [16] and privacy attacks [3, 5]. In this section, we present a
brief review of research on the MI attack (the most popular type of privacy
attacks) and their connection to the adversarial perturbations.

###  II-A Membership Inference Attacks on DNNs

MI attacks on DNNs can be mainly divided into two categories. Training time MI
attacks assume that an attacker is capable of influencing the DNN during
training, in addition to querying the DNN at test time [17, 18]. For example,
Tramer et al. [6] show that by carefully poisoning a DNN during training can
cause the DNN to leak greater membership information at test time. Training-
time MI attacks are most relevant to the frameworks such as federated
learning, where a DNN is trained on broad data collected through untrustworthy
sources. Inference time MI attacks assume that the attacker can only query the
DNN at test-time without influencing it during the training phase [19, 3, 4,
20, 21]. Therefore, inference time MI attacks are relatively more practical
because of the limited attacker capabilities. Inference time MI attacks can
further be divided into two categories. Attacks that estimate the behavior of
the DNNs on given subjects by training several shadow DNNs [2, 3, 4, 22] are
computationally expensive but notably stronger than the attacks that do not
train the shadow DNNs but rely on other membership indicator functions [23,
11].

Our work in this paper falls under the category of inference-time MI attacks
on black-box target DNN classifier by training shadow DNNs to estimate the
behavior of the target DNN. To the best of our knowledge, LiRA [3] and EMIA
[4] are two of the most popular and strongest MI attacks in this category.
Later works on MI attacks have either largely focused on specific scenarios
and applications (network pruning [24], multiple models [19], federated
learning [22], distillation [9]) or fall under a different category of
attacks. Here we briefly discuss both attacks and provide algorithmic details
in Section III after presenting a unified framework of MI attacks. LiRA has
two variations. Online LiRA trains thousands of member (subjects included in
the training set) and non-member (subjects not included in the training set)
shadow DNNs and learns the distribution of the $\phi$-processed confidences
(defined later in eq-(7)) of member and non-member shadow DNNs on the given
subjects. Online LiRA then computes the likelihood of the $\phi$-processed
confidences of the target DNN on the given subjects to fall within the member
or the non-member distribution learned previously. Finally, online LiRA uses
thresholding to label each subject as a member or a non-member. Offline LiRA
works the same as online LiRA but does not train the non-member shadow DNNs.
EMIA follows a similar algorithm as offline LiRA, but uses soft labels
generated by the target DNN to train the non-member shadow DNNs.

Countermeasures against MI attacks: Two of the most popular defenses against
MI attacks in the current literature are differentially private stochastic
gradient descent (DP-SGD) training and $L_{2}$ regularization [25]. We analyze
the effects of both of these approaches on the privacy leakage of the target
DNN.

###  II-B Adversarial perturbations in MI attacks

DNNs have been shown to be vulnerable to adversarial perturbations—small
magnitude perturbations carefully crafted through iterative optimization to
achieve a targeted goal. The goal of standard adversarial attacks is to change
the output of a target DNN $f$ to some target $t$. Formally,

$\delta=\underset{\delta}{\operatorname{argmin}}\ (f(x+\delta)=t)$ (1)

Several algorithms have been proposed to achieve the goal in eq-(1) under
numerous threat models resulting in a range of adversarial attacks [26, 27,
13, 28, 29] and countermeasures [30, 31, 32, 33].

To the best of our knowledge, only two recent works have leveraged the power
of adversarial perturbations to perform MI attacks on DNNs. Del et al. [23]
use the magnitude of adversarial perturbations computed for the target DNN to
estimate the likelihood of the membership. Jalalzai et al. [11] use the loss
values of the target DNN along the adversarial path computed using multiple
perturbation magnitudes to estimate the likelihood of the membership. However,
both works aim to compute a general threshold, not specifically customized to
the subjects (following Attack S methodology by Ye et al. [4]). Additionally,
both of these techniques compute adversarial perturbations over the target
DNN, assuming a white-box access to compute the gradients, and perform
membership inference based on how adversarial perturbations affect the
behavior of the target DNN on the subject. In contrast, we introduce a novel
objective function well-suited to the MI attacks framework (instead of
conventional adversarial attacks objective used by prior works [23, 11]) and
adversarially optimize the novel objective function over the shadow DNNs
(instead of over the target DNN), thereby using both the membership and the
non-membership information of the subject, and only use the output
probabilities of the target DNN to perform membership inference.

Figure 1: Illustrating and comparing the flow diagram of the preparation and
indication stages of EMIA [4] and AMIA. Unlike EMIA, which only trains non-
member shadow DNNs $f^{j}_{n}$, AMIA trains member shadow DNNs $f^{i,j}_{m}$,
along with the non-member shadow DNNs $f^{i,j}_{n}$ by including subject
batches in the training dataset, and optimizes a perturbation for each subject
that favors $f^{i,j}_{m}$ over $f^{i,j}_{n}$, where $j\in[0..N]$. $\Delta
x^{i}$, $f^{j}_{m}$, and $f^{j}_{n}$ constitute $\mathcal{V}$ of AMIA, which
is then used in the indication stage. Note: The flow diagram of offline LiRA
is the same as that of EMIA with the dashed lines excluded.

##  III Preliminaries

In this section, we first introduce the notations and problem setup, and then
present a framework typically followed by current MI attacks. We then briefly
explain the algorithm of f-LiRA, n-LiRA and EMIA based on the identified
framework.

###  III-A Notations and Problem Setup

We consider a DNN $f:\mathcal{X}\rightarrow[0,1]^{m}$ that, when given an
input $x\in\mathcal{X}$, outputs a vector $f(x)$ of length $m$ containing the
probabilities of the class $i$ at the $i$th index $f(x)[i]$. We use
$f_{t}\leftarrow\mathcal{T}(f,D_{t})$ to denote that $f$ is trained on some
training data $D_{t}=\\{(x^{i},y^{i})\\}_{i=0}^{S-1}$ of size $S$, yielding
$f_{t}$ as the trained DNN. $D_{t}$ is assumed to be randomly sampled from the
real-world data distribution $D_{r}$, denoted in the future as $D_{t}\sim
D_{r}\implies\forall(x,y)\in D_{t},(x,y)\sim D_{r}$.

Following [3], for any dataset (e.g. CIFAR-10), we take the available training
data as $D_{r}$ and randomly sample $D_{t}$ from $D_{r}$. The training phase
of the target DNN $f$ is detailed as follows:

  1. 1.

The available training data is assumed to be the real-world data $D_{r}$ of
size $|D_{r}|$.

  2. 2.

Randomly sample training dataset $D_{t}\sim D_{r}$ of size
$|D_{t}|=\frac{|D_{r}|}{2}$.

  3. 3.

Output $f_{t}\leftarrow\mathcal{T}(f,D_{t})$.

###  III-B General Membership Inference (MI) Attack Framework

Given query access to a target DNN $f_{t}\leftarrow\mathcal{T}(f,D_{t})$
trained on the dataset $D_{t}$, and a set of $k$ subjects
$D_{e}=\left\\{\left(x^{i},y^{i}\right)\right\\}_{i=0}^{k-1}$ the membership
of which is to be inferred, an MI attack $\mathcal{A}(\cdot)$ aims to find out
whether each of the given subjects is a member of $D_{t}$ or not. An MI attack
is based on a commonly-observed behavior of DNNs: for any $(x,y)\sim D_{r}$,
the loss of $f_{t}$ is typically smaller if $(x,y)\in D_{t}$, than the loss of
$f_{t}$ if $(x,y)\notin D_{t}$.

MI attacks assume that the attacker can collect samples from the real-world
data $D_{r}$, from which the training data $D_{t}$ is sampled. For
experimental simulation, MI attacks typically assume that the size of $D_{t}$
is half that of $D_{r}$, i.e. $|D_{t}|=\frac{|D_{r}|}{2}$). An MI attack
generally follows three stages—preparation, indication and decision.

  1. 1.

At the preparation stage, given $D_{e}$, an MI attack prepares the variables
$\mathcal{V}$ (e.g., training the models on the attacker dataset $D_{a}\sim
D_{r}$ sampled from the real-world data) that guide the quantification of
membership at the indication stage.

  2. 2.

At the indication stage, $\forall(x,y)\in D_{e}$ an MI attack uses an
indicator function $\mathcal{I}(f_{t},(x,y);\mathcal{V})$ guided by
$\mathcal{V}$ to compute a value that indicates the membership of the subject
$(x,y)\in D_{t}$.

The MI attack algorithm $\mathcal{A}$ can thus be characterized by
$\mathcal{V}$ and $\mathcal{I}$, such that

$\mathcal{A}(f_{t},(x,y))=\mathcal{I}(f_{t},(x,y),\mathcal{V})$ (2)

Because the objective of an MI attack is to differentiate between $(x,y)\in
D_{t}$ and $(x,y)\notin D_{t}$, any choice of $\mathcal{V}$ and $\mathcal{I}$
that satisfies the following condition can be used for membership inference.

$\mathcal{V},\mathcal{I},\ \text{such that},\\\ \underset{(x,y)\in
D_{t}}{\mathbb{E}}[\mathcal{I}(f_{t},(x,y),\mathcal{V})]>\underset{(x,y)\in
D_{r}\char 92\relax D_{t}}{\mathbb{E}}[\mathcal{I}(f_{t},(x,y),\mathcal{V})]$
(3)

  3. 3.

At the decision stage, a threshold $\tau$ is decided depending upon the
tolerable false positive rate such that $\mathcal{A}(f_{t},(x,y))\geq\tau$,
means $(x,y)\in D_{t}$, and $\mathcal{A}(f_{t},(x,y))<\tau$ means $(x,y)\notin
D_{t}$. Formally,

$b=\begin{cases}0,&\mathcal{A}(f_{t},(x,y))<\tau\\\
1,&\mathcal{A}(f_{t},(x,y))\geq\tau\end{cases}$ (4)

A number of algorithms $\mathcal{A}(\cdot)$ have been proposed to effectively
infer the membership of a data sample yielding several MI attacks with diverse
threat settings [2, 3, 4, 8, 9]. In this work, we consider two of the
strongest and the most recent MI attacks: the Likelihood Ratio Attack (LiRA)
proposed by Carlini et al. [3], and an Enhanced Membership Inference Attack
(EMIA) proposed by Ye et al. [4].

Algorithm 1 Offline LiRA (f-LiRA) Algorithm ([3])

1:

2:$f_{t}\leftarrow$ victim model trained on $D_{t}$

3:$D_{r}\leftarrow$ the real data distribution

4:$D_{e}\leftarrow\left\\{\left(x^{i},y^{i}\sim
D_{r}\right)\right\\}_{i=0}^{k+1}$ a set of target samples

5:

6:$\mathcal{I}\leftarrow$ membership inference indicator of each target in
$D_{e}$

7:

8:procedure $\phi$($p$)

9: return $\log\frac{p}{1-p}$

10:end procedure

11:// preparation stage

12:for $j=0..N-1$ do

13: $D^{j}_{a}\sim D_{r}$, such that $\forall(x_{a},y_{a})\in
D^{j}_{a},(x_{a},y_{a})\notin D_{e}$

14: $f^{j}_{n}\leftarrow\mathcal{T}\left(f,D^{j}_{a}\right)$

15:end for

16:$\mu^{i}_{n}\leftarrow\underset{j}{\mathbb{E}}\left[\phi(x^{i},y^{i}|f_{n}^{j})\right]$,
$\sigma^{i}_{n}\leftarrow\mathbb{S}\left[\phi(x^{i},y^{i}|f_{n}^{j})\right]$

17:$\mathcal{V}\leftarrow\\{\mu^{i}_{n},\sigma^{i}_{n}\\}$

18:// indication stage

19:$\mathcal{I}\leftarrow\\{\\}$

20:for $(x^{i},y^{i})$ in $D_{e}$ do

21: $LR_{f}(f_{t},(x^{i},y^{i}),\mathcal{V})\leftarrow 1-p(\
\phi(x^{i},y^{i}|f_{t})\ |\ \mathcal{N}(\mu^{i}_{n},\sigma^{i}_{n})\ )$

22: $\mathcal{I}\leftarrow\mathcal{I}\cup LR_{f}(f_{t},(x^{i},y^{i}))$

23:end for

24:return $\mathcal{I}$

Algorithm 2 Online LiRA (n-LiRA) Algorithm ([3]). Blue highlighted text
indicates modifications over f-LiRA.

1:

2:$f_{t}\leftarrow$ victim model trained on $D_{t}$

3:$D_{r}\leftarrow$ the real data distribution

4:$D_{e}\leftarrow\left\\{\left(x^{i},y^{i}\sim
D_{r}\right)\right\\}_{i=0}^{k+1}$ a set of target samples

5:

6:$\mathcal{I}\leftarrow$ membership inference indicator of each target in
$D_{e}$

7:// preparation stage

8:$\phi\leftarrow$ Algorithm 1

9:for $(x^{i},y^{i})\in D_{e}$ do

10: for $j=0..N-1$ do

11: $D^{j}_{a}\sim D_{r}$, such that $\forall(x_{a},y_{a})\in
D^{j}_{a},(x_{a},y_{a})\notin D_{e}$

12: $f^{j}_{n}\leftarrow\mathcal{T}\left(f,D^{j}_{a}\right)$,
$f^{i,j}_{m}\leftarrow\mathcal{T}\left(f,D^{j}_{a}\cup(x^{i},y^{i})\right)$

13: end for

14:
$\mu^{i}_{n}\leftarrow\underset{j}{\mathbb{E}}\left[\phi(x^{i},y^{i}|f_{n}^{j})\right],\sigma^{i}_{n}\leftarrow\mathbb{S}\left[\phi(x^{i},y^{i}|f_{n}^{j})\right]$

15:
$\mu^{i}_{m}\leftarrow\underset{j}{\mathbb{E}}\left[\phi(x^{i},y^{i}|f_{m}^{i,j})\right],\sigma^{i}_{m}\leftarrow\mathbb{S}\left[\phi(x^{i},y^{i}|f_{m}^{i,j})\right]$

16:
$\mathcal{V}\leftarrow\mathcal{V}\cup\\{\mu^{i}_{n},\sigma^{i}_{n},{\color[rgb]{0,0,1}\definecolor[named]{pgfstrokecolor}{rgb}{0,0,1}\mu^{i}_{m},\sigma^{i}_{m}}\\}$

17:end for

18:// indication stage

19:$\mathcal{I}\leftarrow\\{\\}$

20:for $(x^{i},y^{i})$ in $D_{e}$ do

21: $LR_{n}(f_{t},(x^{i},y^{i}),\mathcal{V})\leftarrow\cfrac{p(\
\phi(x^{i},y^{i}|f_{t})\ |\ \mathcal{N}(\mu^{i}_{m},\sigma^{i}_{m})\ )}{p(\
\phi(x^{i},y^{i}|f_{t})\ |\ \mathcal{N}(\mu^{i}_{n},\sigma^{i}_{n})\ )}$

22: $\mathcal{I}\leftarrow\mathcal{I}\cup
LR_{n}(f_{t},(x^{i},y^{i}),\mathcal{V})$

23:end for

24:return $\mathcal{I}$

Likelihood Ratio Attack (LiRA): LiRA may work in two different modes: offline
LiRA and online LiRA.

Offline LiRA (f-LiRA): Given $(x^{i},y^{i})\in D_{e}$, f-LiRA algorithm is
shown in Algorithm 1, and summarized below:

  1. 1.

Preparation (Steps 4 to 9):

$\forall j\in[0..N],\ f^{j}_{n}\leftarrow\mathcal{T}(D^{j}_{a}\sim D_{r}\char
92\relax D_{e})$ (5)

$\mathcal{V}:\
\mu^{i}_{n}=\underset{j}{\mathbb{E}}\left[\phi\left(x^{i},y^{i}|f^{j}_{n}\right)\right],\
\sigma^{i}_{n}=\mathbb{S}\left[\phi\left(x^{i},y^{i}|f^{j}_{n}\right)\right]$
(6)

where $\underset{j}{\mathbb{E}}$ is the expectation over $j$, $\mathbb{S}$ is
the standard deviation and $\phi(x^{i},y^{i}|f)$ is defined as follows,

$\phi\left(x^{i},y^{i}|f\right)=\log\frac{f(x^{i})[y^{i}]}{1-f(x^{i})[y^{i}]}$
(7)

  2. 2.

Indication (Steps 10 to 15): $\forall(x^{i},y^{i})\in D_{e},$

$\mathcal{I}:LR_{f}(f_{t},(x^{i},y^{i}),\mathcal{V})=\\\ 1-p(\
\phi(x^{i},y^{i}|f_{t})\ |\ \mathcal{N}(\mu^{i}_{n},\sigma^{i}_{n})\ )$ (8)

where $LR_{f}$ denotes the offline likelihood ratio and $p$ is the
probability.

  3. 3.

Decision Stage: The threshold $\tau$ is computed based on the tolerable false
positives.

Online LiRA (n-LiRA): Online LiRA (n-LiRA) customizes f-LiRA to each subject
in $D_{e}$, and is notably more effective than f-LiRA (Algorithm 2).

  1. 1.

Preparation:

$\displaystyle\forall j\in[0..N],\ D^{j}_{a}\sim D_{r}\char 92\relax D_{e},$
$\displaystyle f^{j}_{n}\leftarrow\mathcal{T}(D^{j}_{a})$ (9a)
$\displaystyle\forall(x^{i},y^{i})\in D_{e},\
f^{j}_{m}\leftarrow\mathcal{T}(D^{j}_{a}\cup(x^{i},y^{i}))$ (9b)

$\mathcal{V}:\
\mu^{i}_{n}\leftarrow\underset{j}{\mathbb{E}}\left[\phi(x^{i},y^{i}|f_{n}^{j})\right],\sigma^{i}_{n}\leftarrow\mathbb{S}\left[\phi(x^{i},y^{i}|f_{n}^{j})\right]\\\
\mu^{i}_{m}\leftarrow\underset{j}{\mathbb{E}}\left[\phi(x^{i},y^{i}|f_{m}^{i,j})\right],\sigma^{i}_{m}\leftarrow\mathbb{S}\left[\phi(x^{i},y^{i}|f_{n}^{i,j})\right]$
(10)

  2. 2.

Indication: $\forall(x^{i},y^{i})\in D_{e},$

$\mathcal{I}:LR_{n}(f_{t},(x^{i},y^{i}),\mathcal{V})=\\\ \cfrac{p(\
\phi(x^{i},y^{i}|f_{t})\ |\ \mathcal{N}(\mu^{i}_{m},\sigma^{i}_{m})\ )}{p(\
\phi(x^{i},y^{i}|f_{t})\ |\ \mathcal{N}(\mu^{i}_{n},\sigma^{i}_{n})\ )}$ (11)

where $LR_{n}$ denotes the online likelihood ratio.

  3. 3.

Decision: Finally, the threshold $\tau$ is computed based on the tolerable
false positives.

Algorithm 3 EMIA Algorithm ([4]). Blue highlighted text indicates
modifications over f-LiRA.

1:

2:$f_{t}\leftarrow$ victim model trained on $D_{t}$

3:$D_{r}\leftarrow$ the real data distribution

4:$D_{e}\leftarrow\left\\{\left(x^{i},y^{i}\sim
D_{r}\right)\right\\}_{i=0}^{k+1}$ a set of target samples

5:

6:$\mathcal{I}\leftarrow$ membership inference indicator of each target in
$D_{e}$

7:

8:// preparation stage

9:$\phi\leftarrow$ Algorithm 1

10:for $j=0..N-1$ do

11: $D^{(j)}_{a}\sim D_{r}$, s.t. $\forall(x_{a},y_{a})\in
D^{(j)}_{a},x_{a}\neq x$ and $y_{a}=f_{t}(x_{a})$

12: $f^{j}_{n}\leftarrow\mathcal{T}\left(f,D^{j}_{a}\right)$

13:end for

14:$\mu^{i}_{n}\leftarrow\underset{j}{\mathbb{E}}\left[\phi(x^{i},y^{i}|f_{n}^{j})\right]$,
$\sigma^{i}_{n}\leftarrow\mathbb{S}\left[\phi(x^{i},y^{i}|f_{n}^{j})\right]$

15:$\mathcal{V}\leftarrow\\{\mu^{i}_{n},\sigma^{i}_{n}\\}$

16:// indication stage

17:$\mathcal{I}\leftarrow\\{\\}$

18:for $(x^{i},y^{i})$ in $D_{e}$ do

19: $LR_{f}(f_{t},(x^{i},y^{i}),\mathcal{V})\leftarrow 1-p(\
\phi(x^{i},y^{i}|f_{t})\ |\ \mathcal{N}(\mu^{i}_{n},\sigma^{i}_{n})\ )$

20: $\mathcal{I}\leftarrow\mathcal{I}\cup LR_{f}(f_{t},(x^{i},y^{i}))$

21:end for

22:return $\mathcal{I}$

Enhanced Membership Inference Attack (EMIA): The working of EMIA is given in
Algorithm 3, and summarized below.

  1. 1.

Preparation (Steps 2 to 7):

$\forall j\in[0..N],\\\ D^{j}_{a}\sim D_{r}\char 92\relax D_{e}\ \text{s.t.}\
\forall(x_{a},y_{a})\in D^{j}_{a},\ y_{a}=f_{t}(x_{a}),\\\
f^{j}_{n}\leftarrow\mathcal{T}(D^{j}_{a})$ (12) $\mathcal{V}:\
\mu^{i}_{n}=\underset{j}{\mathbb{E}}\left[\phi\left(x^{i},y^{i}|f^{j}_{n}\right)\right],\
\sigma^{i}_{n}=\mathbb{S}\left[\phi\left(x^{i},y^{i}|f^{j}_{n}\right)\right]$
(13)

  2. 2.

The indication and decision stages of EMIA are the same as those of f-LiRA.

##  IV Proposed Membership Inference Attacks

In this section, we present our algorithm to prepare the variables for
membership inference with two novel improvements—firstly, our attack utilizes
both the membership and the non-membership information of the subjects at a
significantly higher computational efficiency compared to n-LiRA; secondly,
our attack optimally leverages the loss landscape around the subjects by
computing small magnitude input perturbations, adversarially optimized to
maximize the expected loss of non-member shadow DNNs and minimize that of the
member shadow DNNs on the subjects. We then present two novel augmented
indicators that examine the loss values in the Gaussian neighborhood of the
subjects, in addition to the loss over each subject, to infer the membership
of the subjects.

###  IV-A Adversarial Membership Inference Attack (AMIA)

Motivated by eq-(3), for each subject $(x^{i},y^{i})\in D_{e}$, we compute a
perturbation $\Delta x^{i}$ such that $(x^{i}+\Delta x^{i},y^{i})$ is more
vulnerable to MI attack as compared to $(x^{i},y^{i})$. More formally, we aim
to achieve the following,

$\begin{split}\mathcal{A}(f_{t},(x^{i}+\Delta
x^{i},y^{i}))\geq\mathcal{A}(f_{t},(x^{i},y^{i})),\ \text{if}\
(x^{i},y^{i})\in D_{t}\\\ \mathcal{A}(f_{t},(x^{i}+\Delta
x^{i},y^{i}))\leq\mathcal{A}(f_{t},(x^{i},y^{i})),\ \text{if}\
(x^{i},y^{i})\notin D_{t}\end{split}$ (14)

However, it is impossible to directly achieve eq-(14) because an MI attacker
does not have access to $D_{t}$. To address this, we leverage the
transferability property of adversarial perturbations. It has been shown that
the adversarial perturbation to an input computed to fool a deep learning
model $\mathcal{F}$ effectively transfers (fools) to other DNNs [14].
Additionally, using an ensemble of multiple DNNs as $\mathcal{F}$
significantly improves the transferability of the adversarial perturbations.
We leverage the transferability property to transfer the adversarial
perturbations computed over member and non-member shadow DNNs to $f_{t}$ as
detailed in Algorithm 4.

  1. 1.

Preparation Stage: Given a set of subjects $D_{e}$ for membership inference,
at every iteration $j\in[0..N-1]$, we first randomly sample a dataset
$D^{j}_{a}\sim D_{r}$ (Step 3) and randomly bisect $D_{e}$ into two distinct
subsets $D^{j}_{e1}$ and $D^{j}_{e2}$ (Step 4). We then train a pair of shadow
DNNs $f^{j}_{m1}$ and $f^{j}_{m2}$ on $D^{j}_{a}$ augmented with $D^{j}_{e1}$
and $D^{j}_{e2}$ respectively (Step 5). This lets us train the member and the
non-member shadow DNNs simultaneously—$\forall(x^{i},y^{i})\in D_{e}$, if
$(x^{i},y^{i})\in D^{j}_{e1}$, $f^{j}_{m1}$ is the member shadow DNN and
$f^{j}_{m2}$ is the non-member shadow DNN, and vice versa if $(x^{i},y^{i})\in
D^{j}_{e2}$. The process is repeated for $j\in[0..N-1]$.

To make the attack stronger, $\forall j\in[0..N-1],\forall(x^{i},y^{i})\in
D_{e}$, we first create lists of member shadow DNNs $f^{i,j}_{m}$ and non-
member shadow DNNs $f^{i,j}_{n}$ (Steps 10 to 13). We then adversarially learn
$\Delta x^{i}\in[-\epsilon,\epsilon]$, using the iterative Fast Gradient Sign
Method (i-FGSM) to minimize the expected loss of $(x^{i},y^{i})$ on the member
shadow DNNs, while maximizing the loss of $(x^{i},y^{i})$ on non-member shadow
DNNs (Step 16). Formally,

$g^{j}(\Delta x^{i})=\\\ \phi\left(x^{i}+\Delta
x^{i},y^{i}|f^{i,j}_{n}\right)-\phi\left(x^{i}+\Delta
x^{i},y^{i}|f^{i,j}_{m}\right)\\\ $ (15)

$\Delta x^{i}=\min_{\Delta
x^{i}\in[-\epsilon,\epsilon]}\quad\underset{j\in[0..N-1]}{\mathbb{E}}\left[g^{j}(\Delta
x^{i})\right]\\\ $ (16)

The prepared variables $\mathcal{V}$ are then,

$\mathcal{V}:\ \Delta x^{i},\\\
\mu^{i}_{n}=\underset{j}{\mathbb{E}}\left[\phi\left(x^{i}+\Delta
x^{i},y^{i}|f^{i,j}_{n}\right)\right],\\\
\sigma^{i}_{n}=\mathbb{S}\left[\phi\left(x^{i}+\Delta
x^{i},y^{i}|f^{i,j}_{n}\right)\right],\\\
\mu^{i}_{m}=\underset{j}{\mathbb{E}}\left[\phi\left(x^{i}+\Delta
x^{i},y^{i}|f^{i,j}_{m}\right)\right],\\\
\sigma^{i}_{m}=\mathbb{S}\left[\phi\left(x^{i}+\Delta
x^{i},y^{i}|f^{i,j}_{m}\right)\right]$ (17)

  2. 2.

Indication Stage: $\forall(x^{i},y^{i})\in D_{e}$, and corresponding
$i\in[0..|D_{e}|-1]$ AMIA computes the $\phi$-scaled confidences of non-member
and member shadow DNNs on the subject $(x,y)$ and compute $\mathcal{I}$ using
the online likelihood ratio $LR_{n}$ as follows,

$\mathcal{I}:LR_{n}(f_{t},(x^{i},y^{i}),\mathcal{V})=\\\ \cfrac{p(\
\phi(x^{i}+\Delta x^{i},y^{i}|f_{t})\ |\
\mathcal{N}(\mu^{i}_{m},\sigma^{i}_{m})\ )}{p(\ \phi(x^{i}+\Delta
x^{i},y^{i}|f_{t})\ |\ \mathcal{N}(\mu^{i}_{n},\sigma^{i}_{n})\ )}$ (18)

  3. 3.

Decision Stage: Finally, the threshold $\tau$ is computed based on the
tolerable false positives.

Algorithm 4 Proposed Adversarial MIA (AMIA) algorithm. Blue highlighted text
indicates modifications over f-LiRA.

1:

2:$f_{t}\leftarrow$ victim model trained on $D_{t}$

3:$D_{r}\leftarrow$ the real data distribution

4:$D_{e}\leftarrow\left\\{\left(x^{i},y^{i}\sim
D_{r}\right)\right\\}_{i=0}^{k+1}$ a set of target samples

5:

6:$\mathcal{I}\leftarrow$ membership inference indicator of each target in
$D_{e}$

7:// preparation stage

8:$\mathcal{V}\leftarrow\\{\\}$, $\phi\leftarrow$ Algorithm 1

9:for $j=0..N-1$ do

10: $D^{j}_{a}\sim D_{r}$, such that $\forall(x_{a},y_{a})\in
D^{j}_{a},(x_{a},y_{a})\neq(x,y)$.

11: $D^{j}_{e1}$, $D^{j}_{e2}\leftarrow$ random bisect $(D_{e})$

12: $f^{j}_{m1}\leftarrow\mathcal{T}\left(f,D^{j}_{a}\cup
D^{j}_{e1}\right),f^{j}_{m2}\leftarrow\mathcal{T}\left(f,D^{j}_{a}\cup
D^{j}_{e2}\right)$

13:end for

14:for $(x^{i},y^{i})\in D_{e}$ do

15: for $j\in[0..N-1]$ do

16: $\Delta x^{i}\leftarrow 0$

17: if $(x^{i},y^{i})\in D^{j}_{e1}$ then

18: $s_{j}\leftarrow-1$, $f^{i,j}_{m}\leftarrow f^{j}_{m1}$,
$f^{i,j}_{n}\leftarrow f^{j}_{m2}$

19: else if $(x^{i},y^{i})\in D^{j}_{e2}$ then

20: $s_{j}\leftarrow 1$, $f^{i,j}_{m}\leftarrow f^{j}_{m2}$,
$f^{i,j}_{n}\leftarrow f^{j}_{m1}$

21: end if

22: end for

23:

24: $\Delta x^{i}\leftarrow\underset{\Delta x^{i}}{\min}\
\underset{j}{\mathbb{E}}\left[\phi\left(x^{i}+\Delta
x^{i},y^{i}|f^{i,j}_{n}\right)-\phi\left(x^{i}+\Delta
x^{i},y^{i}|f^{i,j}_{m}\right)\right]$

25:
$\mu^{i}_{n}\leftarrow\underset{j}{\mathbb{E}}\left[\phi(x^{i},y^{i}|f_{n}^{i,j})\right],\sigma^{i}_{n}\leftarrow\mathbb{S}\left[\phi(x^{i},y^{i}|f_{n}^{i,j})\right]$

26:
$\mu^{i}_{m}\leftarrow\underset{j}{\mathbb{E}}\left[\phi(x^{i},y^{i}|f_{m}^{i,j})\right],\sigma^{i}_{m}\leftarrow\mathbb{S}\left[\phi(x^{i},y^{i}|f_{m}^{i,j})\right]$

27: $\mathcal{V}\cup\\{\Delta
x^{i},\mu^{i}_{n},\mu^{i}_{m},\sigma^{i}_{n},\sigma^{i}_{m}\\}$

28:end for

29:// indication stage

30:$\mathcal{I}\leftarrow\\{\\}$

31:for $(x^{i},y^{i})\in D_{e}$ do

32: $LR_{n}(f_{t},(x^{i},y^{i}),\mathcal{V})\leftarrow\cfrac{p(\
\phi(x^{i}+\Delta x^{i},y^{i}|f_{t})\ |\
\mathcal{N}(\mu^{i}_{m},\sigma^{i}_{m})\ )}{p(\ \phi(x^{i}+\Delta
x^{i},y^{i}|f_{t})\ |\ \mathcal{N}(\mu^{i}_{n},\sigma^{i}_{n})\ )}$

33: $\mathcal{I}\leftarrow\mathcal{I}\cup
LR_{n}(f_{t},(x^{i},y^{i}),\mathcal{V})$

34:end for

35:return $\mathcal{I}$

Enhanced Adversarial Membership Inference Attack.  It may be possible to
further increase the effectiveness of AMIA by reducing the data uncertainty
inspired by EMIA. To achieve this, we propose the enhanced adversarial
membership inference attack (E-AMIA). E-AMIA follows the same algorithm as
AMIA, except that it replaces the data sampling step 10 of Algorithm 4 with
the data sampling step 11 of Algorithm 3.

###  IV-B Augmented Membership Indicators

Both LiRA and EMIA use vanilla indicators that examine the loss of the target
DNN precisely upon the subject $(x^{i},y^{i})$. However, loss values in the
local neighborhood (e.g., the Gaussian neighborhood) of the subject might be
helpful for membership inference. In order to utilize the Gaussian
neighborhood information of a subject, we create a set of augmented inputs
$X^{i}$ by concatenating the original input $x$ with its Gaussian noise
augmented versions that are computed by adding to $x^{i}$, $p-1$ random
Gaussian noises $\bigcup_{l=1}^{p-1}n^{l}\sim\mathcal{N}(0,\sigma_{n})$ with
mean $0$ and standard deviation $\sigma_{n}$.

$X^{i}=\bigcup_{l=0}^{p-1}\left(x^{i}+n^{l}\sim\mathcal{N}(0,\sigma_{n})\right)$
(19)

where $n^{(0)}=0$. This lets us analyze the loss of $f_{t}$ at and around
$(x^{i},y^{i})$ allowing us to model local trends which are helpful in
membership inference (See Fig.).

####  IV-B1 Likelihood Ratio with Perturbation:

For any $(x^{i},y^{i})\in D_{e}$, follow the following steps:

  1. 1.

$\forall l\in[0..p-1]$, compute

$\begin{split}F_{n}^{l}=\left\\{\forall
j\in[0..N-1],\phi\left(x^{i}+n^{l},y|f^{j}_{n}\right)\right\\}\\\
F_{m}^{l}=\left\\{\forall
j\in[0..N-1],\phi\left(x^{i}+n^{l},y|f^{j}_{m}\right)\right\\}\end{split}$
(20)

  2. 2.

$\forall l\in[0..p-1]$, compute

$\begin{split}\mu_{n}^{l}=\operatorname{mean}\left(F_{n}^{l}\right),\sigma_{n}^{l}=\operatorname{std}\left(F_{n}^{l}\right)\\\
\mu_{m}^{l}=\operatorname{mean}\left(F_{m}^{l}\right),\sigma_{m}^{l}=\operatorname{std}\left(F_{m}^{l}\right)\end{split}$
(21)

  3. 3.

Finally, compute $LR_{p}(f_{t},(x^{i},y^{i}),\mathcal{V})$ the likelihood
ratio of $(x^{i},y^{i})$ to be a member of $D_{t}$ with perturbation defined
as the expected likelihood ratio of $(x^{i}+n^{l},y^{i})$ over $l$ for
$f_{t}$.

$LR_{p}(f_{t},(x,y),\mathcal{V})=\\\
\underset{l\in[0..p-1]}{\mathbb{E}}\left[LR_{n}\left(f_{t},\left(x^{i}+n^{l},y^{i}\right),F_{n}^{l},F_{m}^{l}\right)\right]$
(22)

####  IV-B2 Likelihood Ratio with the Optimized Perturbation:

For any $(x^{i},y^{i})\in D_{e}$, follow the following steps:

  1. 1.

Follow steps 1 and 2 of the likelihood ratio with perturbation.

  2. 2.

Set $O=\\{\\}$,

$\text{Repeat $z$ times:}\quad O:=O\cup\underset{l\in[0..p-1],l\notin
O}{\operatorname{argmax}}\left(F_{m}^{l}-F_{n}^{l}\right)$ (23)

$O$ is a set of $z$ integers denoting the indices $l\in[0..p-1]$ of the noise
that produce the maximum expected difference between the $\phi$-scaled
confidences of the member and the non-member shadow DNNs.

  3. 3.

Finally, compute $LR_{o}(f_{t},(x^{i},y^{i}))$ the likelihood ratio of
$(x^{i},y^{i})$ to be a member of $D_{t}$ with optimal perturbation defined as
the expected likelihood ratio of $(x+n^{l},y)$ over $l\in O$ for $f_{t}$.

$LR_{o}(f_{t},(x,y))=\underset{l\in
O}{\mathbb{E}}\left[LR_{n}\left(f_{t},\left(x^{i}+n^{l},y^{i}\right),F_{n}^{l},F_{m}^{l}\right)\right]$

(24)

Note that if $z=p$, $LR_{o}(\cdot)=LR_{n}(\cdot)$.

##  V Performance Evaluation

###  V-A Evaluation Methodology

Our evaluation methodology is similar to that used by the previous works [3,
4] as summarized below.

####  V-A1 Datasets

We evaluate MI attacks on three commonly used machine learning
datasets—CIFAR-10, MNIST and Fashion-MNIST. All our datasets are divided into
two sets: (1) the original training set of the data is assumed to be our real-
world data $D_{r}$, and (2) the original test set is denoted as $D_{s}$. The
training data $D_{t}\sim D_{r}$ for the target DNN is sampled from $D_{r}$.
For coherence with Carlini et al. [3], we use $|D_{t}|=|D_{r}|/2$, where
$|\cdot|$ denotes the size of the dataset. The size of the attacker data
$D_{a}\sim D_{r}$ is $|D_{a}|=|D_{t}|/2$ [3]. We construct the evaluation
dataset $D_{e}$ of size $|D_{e}|=k$, where each instance is sampled uniformly
either from $D_{t}$ or $D_{r}\char 92\relax D_{t}$. We use $k=200$ in our
experiments.

####  V-A2 Model architecture

Following Ye et al. [4], we chose a deep Convolutional Neural Network
(CNN)-based model as our target DNN. More specifically, we use the following
CNN: Input() - {Conv2D() - ReLU() - BatchNorm()}$\times 3$ \- MaxPool() -
Dropout(.) - Flatten() - Dense($\\#classes$) - softmax(). Unless stated
otherwise, we always train the CNN using $L_{2}$ regularization because it
makes the CNNs notably more robust to MI attacks [25]. We train $N=50$ non-
member shadow DNNs for f-LiRA and EMIA, and equal number of non-member and
member shadow DNNs for AMIA and E-AMIA for each experiment. We do not
implement n-LiRA due to its high computational overhead—n-LiRA requires
training 10000 shadow DNNs for a single experiment on each dataset in our
experimental setup.

Figure 2: Comparing the train and test accuracies of the CNN on the three
datasets considered in this paper.

####  V-A3 Evaluation Metrics

Here we first define the TPR-FPR curve typically used by prior works [3, 4,
22], and then present the running TPR average (RTA), a novel metric to test
the performance of MI attacks. Although AMIA and E-AMIA generally perform
better than f-LiRA and EMIA on both evaluation metrics, we believe that RTA
better distinguishes different attacks as compared to the TPR-FPR curve.

####  V-A4 TPR-FPR curve

Following previous works we evaluate the effectiveness of an MI attack by
computing its TPR at a given FPR in log scale (emphasizing low FPR region)
[3]. Given $\tau$, TPR is the ratio of the number of samples correctly marked
as members by $\mathcal{A}$, to the total number of member samples. Similarly,
the FPR is the total number of samples incorrectly marked by $\mathcal{A}$ as
members to the total number of non-member samples. Both TPR and FPR are define
below:

$\text{TPR}(\tau)=\frac{|\forall(x,y)\in D_{t}\cap D_{e},\ s.t.\
\mathcal{A}(f_{t},(x,y))>\tau|}{|\forall(x,y)\in D_{t}\cap D_{e}|}$ (25)
$\text{FPR}(\tau)=\frac{|\forall(x,y)\in\\{D_{r}\char 92\relax D_{t}\\}\cap
D_{e},\ s.t.\ \mathcal{A}(f_{t},(x,y))>\tau|}{|\forall(x,y)\in D^{\prime}\cap
D_{e}|}$ (26)

where, $|\cdot|$ denotes the total number of samples satisfying the condition.

####  V-A5 Running TPR Average (RTA)

We propose a new metric called the running TPR average (RTA) defined at a
given value $t\in[0,1]$ as the average value of TPR for FPR less than $t$.
Formally,

$\text{RTA}(t)=\underset{\tau\ \text{s.t.}\ \text{FPR}(\tau)\leq
t}{\mathbb{E}}[\text{TPR}(\tau)]$ (27)

Stated simply, RTA$(t)$ defines the success rate of membership detection rate
with more than $(1-t)\times 100\%$ confidence. We believe that RTA better
quantifies the member inference efficacy of an MI attack in the low FPR
region. Nevertheless, we show that our proposed attacks—AMIA and
E-AMIA—outperform the state-of-the-art on both of the evaluation metrics.

###  V-B Accuracy of the Models

On CIFAR-10 dataset, the CNN achieves an accuracy of 96.7% accuracy on the
training dataset $D_{t}$ with 64.6% accuracy on the test set $D_{s}$ as shown
in Fig. 2, which is the same as that achieved by Carlini et al. [3] using wide
ResNet. For F-MNIST and MNIST, Fig. 2 show that the CNN respectively achieves
98.2% and 99.9% accuracy on $D_{t}$, and 90.6% and 98.6% accuracy on $D_{s}$.
Like Carlini et al. [3], we achieve a significantly smaller accuracy on the
test set as compared to the state-of-the-art for CIFAR-10 and F-MNIST. This is
because the experimental setup of LiRA (and EMIA) further divides the original
training set $D_{r}$ into two equal halves, and only one of these halves
$D_{t}$ is used to train the CNN.

(a) CIFAR-10

(b) Fashion-MNIST

(c) MNIST

Figure 3: Comparing TPR of the state-of-the-art MI attacks (f-LiRA,$LR_{f}$
and EMIA,$LR_{f}$) with our proposed attacks (AMIA,$LR_{n}$ and
E-AMIA,$LR_{n}$) for the entire range of possible FPR in log scale to
emphasize low FPR region [3, 4]. Both AMIA,$LR_{n}$ and E-AMIA,$LR_{n}$
perform notably better than f-LiRA,$LR_{f}$ and EMIA,$LR_{f}$ in low FPR
region. (Settings: $\epsilon=0.02$, $N=50$).

(a) CIFAR-10

(b) Fashion-MNIST

(c) MNIST

Figure 4: Comparing the running TPR average (RTA) of the state-of-the-art MI
attacks (f-LiRA,$LR_{f}$ and EMIA,$LR_{f}$) with our proposed attacks
(AMIA,$LR_{n}$ and E-AMIA,$LR_{n}$) as $t$ increases logarithmically to
emphasize low FPR region [3, 4]. Both AMIA,$LR_{n}$ and E-AMIA,$LR_{n}$
perform notably better than f-LiRA,$LR_{f}$ and EMIA,$LR_{f}$ in low FPR
region. RTA more effectively distinguishes different attacks in the low FPR
region. (Settings: $\epsilon=0.02$, $N=50$).

###  V-C Evaluating MI Attacks

This section compares the state-of-the-art MI attacks—f-LiRA and EMIA—with the
newly proposed AMIA and E-AMIA. Throughout the section (and the following
sections), we denote an MI attack using the $\mathcal{V},\mathcal{I}$ format.
For example, f-LiRA,$LR_{f}$ denotes that f-LiRA algorithm is used to prepare
the variables and $LR_{f}$ is used as the indicator function.

####  V-C1 Comparing MI attacks

Comparison of f-LiRA,$LR_{f}$ [3], EMIA,$LR_{f}$ [4], AMIA,$LR_{n}$ (ours) and
E-AMIA,$LR_{n}$ (ours) respectively for the CIFAR-10, Fashion-MNIST and MNIST
datasets is provided in Fig. 3(a)-(c) based on TPR-FPR metric, and in Fig.
4(a)-(c) based on RTA metric. We note that f-LiRA,$LR_{f}$ is more effective
than EMIA,$LR_{f}$ in the low FPR region—Ye et al. [4] also observe that
Attack R (f-LiRA,$LR_{f}$ in this paper) performs better than Attack D
(EMIA,$LR_{f}$ in this paper) in the low FPR region on CIFAR-10 dataset. In
the high FPR region, EMIA,$LR_{f}$ typically outperforms including the
proposed AMIA and E-AMIA—in their work, Ye et al. [4] also observe that
EMIA,$LR_{f}$ is more effective in the high FPR region on CIFAR-10 dataset. We
also note that, although our general observations and conclusions from RTA and
TPR-FPR curve are consistent with each other, RTA is able to better
distinguish the effectiveness of different attacks and appears more stable as
compared to the TPR-FPR curve.

As we are mostly interested in membership detection with high confidence (low
FPR region) [3], our results show that AMIA,$LR_{n}$ and E-AMIA,$LR_{n}$
consistently outperform both f-LiRA,$LR_{f}$ and EMIA,$LR_{f}$ for low FPRs
(e.g., $\leq$ 3%) with a significant margin for all the three datasets. The
only exception is CIFAR-10 at FPR 1%, where f-LiRA,$LR_{f}$ performs slightly
better. For example, on the Fashion-MNIST dataset, f-LiRA,$LR_{f}$ and
EMIA,$LR_{f}$ exhibit 0% TPR, while AMIA,$LR_{n}$ and E-AMIA,$LR_{n}$
respectively achieve 6% and 8% TPR at 1% FPR. This is concerning as it implies
that around 8% of the training dataset members can be identified with 99%
confidence (1% FPR). We attribute this increased effectiveness to two reasons.
Firstly, AMIA,$LR_{n}$ is guided by both the membership and the non-membership
properties of the subject $(x^{i},y^{i})\in D_{e}$ in contrast to
EMIA,$LR_{f}$, which is only informed by the non-membership properties.
Secondly, AMIA,$LR_{f}$ leverages small adversarial perturbations (or
adversarial features) that have been carefully optimized to be indicative of
the membership of $(x,y)$.

TABLE I: Comparing RTA$(\tau)$ of f-LiRA, EMIA, AMIA and E-AMIA for multiple
indicator functions. $\ast$ denotes the newly proposed attacks and indicator
functions in this paper. Red highlighted values indicate instances where our
proposed attack or indicator functions perform worse than either LiRA or EMIA.

Dataset | Metric | f-LiRA | EMIA | AMIA $\ast$ | E-AMIA $\ast$
---|---|---|---|---|---
$LR_{f}$ |  $LR_{p}$$\ast$ | $LR_{f}$ |  $LR_{p}$$\ast$ | $LR_{f}$ |  $LR_{p}$$\ast$ |  $LR_{o}$$\ast$ | $LR_{f}$ |  $LR_{p}$$\ast$ |  $LR_{o}$$\ast$
CIFAR-10 | RTA(0) | 0.01 | 0.01 | 0.0 | 0.02 | 0.03 | 0.03 | 0.02 | 0.06 | 0.07 | 0.04
RTA(0.01) | 0.035 | 0.03 | 0.0 | 0.025 | 0.035 | 0.04 | 0.025 | 0.09 | 0.09 | 0.075
RTA(0.03) | 0.0575 | 0.0425 | 0.0 | 0.0325 | 0.0625 | 0.07 | 0.0575 | 0.1225 | 0.13 | 0.1325
RTA(0.05) | 0.07333 | 0.065 | 0.01667 | 0.05333 | 0.08167 | 0.08667 | 0.07667 | 0.145 | 0.15 | 0.155
RTA(1) | 0.5708 | 0.5546 | 0.6554 | 0.6483 | 0.5749 | 0.572 | 0.5703 | 0.5767 | 0.5845 | 0.586
Fashion-MNIST | RTA(0) | 0.0 | 0.0 | 0.0 | 0.01 | 0.01 | 0.01 | 0.0 | 0.01 | 0.02 | 0.02
RTA(0.01) | 0.0 | 0.0 | 0.0 | 0.045 | 0.035 | 0.04 | 0.035 | 0.045 | 0.055 | 0.04
RTA(0.03) | 0.0025 | 0.0075 | 0.0 | 0.0625 | 0.0525 | 0.0575 | 0.055 | 0.0625 | 0.0825 | 0.075
RTA(0.05) | 0.01333 | 0.015 | 0.02 | 0.07167 | 0.06167 | 0.065 | 0.06333 | 0.08167 | 0.09167 | 0.09
RTA(1) | 0.4852 | 0.5057 | 0.5262 | 0.5259 | 0.4857 | 0.486 | 0.4927 | 0.5225 | 0.5192 | 0.5154
MNIST | RTA(0) | 0.01 | 0.02 | 0.0 | 0.0 | 0.06 | 0.06 | 0.06 | 0.04 | 0.03 | 0.03
RTA(0.01) | 0.015 | 0.025 | 0.0 | 0.005 | 0.06 | 0.06 | 0.06 | 0.04 | 0.03 | 0.03
RTA(0.03) | 0.025 | 0.0275 | 0.0 | 0.0225 | 0.07 | 0.07 | 0.07 | 0.0475 | 0.0425 | 0.045
RTA(0.05) | 0.03167 | 0.03 | 0.02 | 0.04167 | 0.07333 | 0.07333 | 0.07333 | 0.06333 | 0.06333 | 0.06333
RTA(1) | 0.4877 | 0.4636 | 0.5353 | 0.5192 | 0.5368 | 0.5368 | 0.5377 | 0.5841 | 0.5857 | 0.5876

####  V-C2 Comparing indicator functions

We compare the performance of different indicator functions $\mathcal{I}$ with
f-LiRA, EMIA, AMIA and E-AMIA on the three datasets considered in this paper.
More specifically, for f-LiRA and EMIA, we compare two indicators $LR_{f}$ and
$LR_{p}$, and for AMIA, we compare three indicators $LR_{n}$, $LR_{p}$ and
$LR_{o}$, because $LR_{o}$ is only compatible with the algorithms that
leverage the membership information of $(x,y)$ (See eq-(24)) in Table I.

Carlini et al. [3] observe that Gaussian augmented indicator proposed by
Jayaraman et al. [34] does not improve the MI attack performance. On the
contrary, our proposed augmented indicators $LR_{p}$ and $LR_{o}$ typically
outperform vanilla indicators $LR_{f}$ and $LR_{n}$ for all the baseline
attacks, specifically on Fashion-MNIST and MNIST datasets. This is because our
novel augmented indicators compute the membership likelihood of each noisy
sample independently, and then ensemble the results as defined in eq-(22) and
(24). These observations are also consistent with our initial hypothesis that
analyzing local loss at and around $(x,y)$ will improve the effectiveness of
MI attacks.

Table I also validates our previous finding that AMIA outperforms both f-LiRA
and EMIA in the low FPR region, while EMIA is most effective for the high FPR
region (see AUC(1) in the table).

###  V-D Analyzing Hyperparameters

(a) CIFAR-10

(b) Fashion-MNIST

(c) MNIST

Figure 5: Analyzing the effect of varying $\epsilon$ (see eq-(16)) on the
performance of AMIA and E-AMIA for the three datasets considered in this
paper. AMIA and E-AMIA are most effective at $\epsilon=0.02$. (Settings:
$\sigma_{n}=0.02$, $N=50$, $\mathcal{I}$ in $LR_{n}$, $LR_{p}$ and $LR_{o}$).

####  V-D1 Effects of $\epsilon$ on the attack performance

Fig. 5(a)-(c) show the effects of varying $\epsilon$, the $l_{\infty}$ norm of
the adversarial perturbations, on the performance of AMIA on CIFAR-10,
Fashion-MNIST and MNIST datasets respectively. TPR of f-LiRA,$LR_{f}$ and
EMIA,$LR_{f}$ are also provided for comparison.

For all the three datasets and the indicator functions, we observe that the
performance of AMIA initially increases as $\epsilon$ is increased from 0.01
to 0.02. This is because a greater $l_{\infty}$ norm allows AMIA to analyze
and process more information about the local loss around $(x,y)$. TPRs of AMIA
reach a maximum value at $\epsilon=0.02$, and then start decreasing as
$\epsilon$ increases. This decrease in TPR is because a too large $\epsilon$
only sub-optimally captures the local loss trends around $(x,y)$.

(a) CIFAR-10

(b) Fashion-MNIST

(c) MNIST

Figure 6: Analyzing the effect of varying $\sigma_{n}$ (see eq-(22)) on the
performance of AMIA and E-AMIA for the three datasets considered in this
paper. AMIA and E-AMIA are most effective at $\epsilon=0.02$. (Settings:
$\epsilon=0.02$, $N=50$, $\mathcal{I}$ in $LR_{n}$, $LR_{p}$ and $LR_{o}$).

####  V-D2 Effects of $\sigma_{n}$ on the attack performance

Fig. 6(a)-(c) show the effects of varying $\sigma_{n}$, the standard deviation
of Gaussian noise in eq-(22), on the performance of AMIA on CIFAR-10, Fashion-
MNIST and MNIST datasets respectively. TPR of f-LiRA,$LR_{f}$ and
EMIA,$LR_{f}$ are also provided for comparison.

For all the three datasets and the indicator functions, we observe that the
performance of AMIA typically increases slightly as $\sigma_{n}$ is increased
from 0.01 to 0.02, and then starts decreasing as $\sigma_{n}$ increases. The
observed trend is interestingly similar to that observed when $\epsilon$ is
increased in Fig. 5. We attribute this to the same reasons as above—the
initial slight increase is caused by the greater $\sigma_{n}$ better capturing
the local loss around $(x,y)$, and a too high $\sigma_{n}$ sub-optimally
captures the local loss around $(x,y)$ causing a decrease in the running
average.

However, interestingly, the decrease in the RTA due to an increase in
$\sigma_{n}$ is significantly less drastic than that caused by an increase in
$\epsilon$. We hypothesize that because $\sigma_{n}$ is the standard deviation
of the Guassian noise, $n^{l}\sim\mathcal{N}(0,\sigma_{n})$ is not bound by
$l_{\infty}(\sigma_{n})$, unlike the adversarial noise which is bound by
$l_{\infty}(\epsilon)$. Therefore even if $\sigma_{n}$ is large, several noise
elements have notably smaller $l_{\infty}$ norm due to the Guassian
distribution. Therefore, a suboptimally large value of $\sigma_{n}$ better
captures local loss trends than the suboptimally large value of $\epsilon$.

(a) CIFAR-10

(b) Fashion-MNIST

(c) MNIST

Figure 7: Analyzing the effect of varying $N$ (see eq-(16)) on the performance
of AMIA and E-AMIA for the three datasets considered in this paper. AMIA and
E-AMIA are most effective at $\epsilon=0.02$. (Settings: $\epsilon=0.02$,
$\sigma_{n}=0.02$, $\mathcal{I}$ in $LR_{n}$, $LR_{p}$ and $LR_{o}$).

####  V-D3 Effects of $N$ on the attack performance

Fig. 7(a)-(c) show the effects of varying $N$, the number of shadow DNNs
trained by the attacker on the sampled datasets $D^{j}_{a}$, on the
performance of AMIA on CIFAR-10, Fashion-MNIST and MNIST datasets
respectively. TPR of f-LiRA,$LR_{f}$ and EMIA,$LR_{p}$ are also provided for
comparison. We observe that the effectiveness of the attacks increases as $N$
is increased. In MI attacks, shadow DNNs are used to by the attacker to study
the behavior of DNN on the subject given that the subject is or is not part of
its training set. Therefore, training a greater number of shadow DNNs enables
the attacker to more precisely approximate the expected behavior of the target
DNN, ultimately improving the attack performance.

It can be observed in the figure that AMIA outperforms EMIA,$LR_{f}$ on all
three considered datasets. On CIFAR-10 dataset, f-LiRA,$LR_{f}$ outperforms
AMIA with a significant margin, except at $N=50$ where AMIA performs slightly
better than f-LiRA,$LR_{f}$. We believe the reason for the increased
effectiveness of f-LiRA,$LR_{f}$ on CIFAR-10 is the overfitting of the target
CNN on the CIFAR-10 training set $D_{t}$. On Fashion-MNIST and MNIST datasets,
where the target CNN is more generalized over the test set, AMIA is typically
more effective than both f-LiRA,$LR_{f}$. For MNIST, $N=20$, f-LiRA,$LR_{f}$
shows surprisingly improved effectiveness—we regard it as an outlier, as we
were unable to achieve a similar effectiveness with f-LiRA,$LR_{f}$ when we
re-run the experiment.

##  VI Discussion

(a) CIFAR-10

(b) Fashion-MNIST

(c) MNIST

Figure 8: Comparing the RTA of f-LiRA,$LR_{f}$ and EMIA,$LR_{f}$ with
AMIA,$LR_{n}$ and E-AMIA,$LR_{n}$ as $t$ increases logarithmically when
$\mathcal{V}$ (prepared for the target DNN) is transferred to unknown CNNs.
Each is an average value computed over 10 unknown CNNs. AMIA,$LR_{n}$ gives
best performance overall, while both AMIA,$LR_{n}$ and E-AMIA,$LR_{n}$ perform
better than f-LiRA,$LR_{f}$ and EMIA,$LR_{f}$ in low FPR region for Fashion-
MNIST and MNIST datasets. (Settings: $\epsilon=0.02$, $N=50$.).

###  VI-A Transferability of MI Attacks

We study the transferability of MI attacks in Fig. 8(a)-(c) respectively for
CIFAR-10, Fashion-MNIST and MNIST datasets. We define transferability as the
ability of MI attack variables optimized for a target CNN to transfer to the
unknown CNNs trained on the same dataset that the variables have not been
optimized for. Fig. 8 reports RTA of different attacks averaged over 10
unknown CNNs. We observe that generally in the low FPR region AMIA,$LR_{f}$
most effectively transfers to unknown CNNs, with E-AMIA,$LR_{n}$ and
f-LiRA,$LR_{f}$ showing comparable performance. On the other hand,
EMIA,$LR_{f}$ is least transferable to unknown CNNs even in the high FPR
region where it performs best on the target CNN. This is because EMIA,$LR_{f}$
is more customized to the target DNN because of using soft labels. For the
CIFAR-10 dataset however, f-LiRA,$LR_{f}$ outshines both AMIA,$LR_{f}$ and
E-AMIA,$LR_{n}$ due to the overfitting of the unknown CNNs as discussed in
detail in Section V-D3.

Not surprisingly, the transferability of E-AMIA,$LR_{n}$ seems correlated with
the transferability of EMIA,$LR_{f}$. More specifically, when EMIA,$LR_{f}$
shows worse transferability than AMIA,$LR_{f}$—for example on CIFAR-10 and
Fashion-MNIST datasets in Fig. 8(a),(b)—E-AMIA,$LR_{n}$ consistently performs
slightly worse than AMIA,$LR_{f}$. Likewise, when EMIA,$LR_{f}$ shows better
transferability than AMIA,$LR_{f}$—for example on MNIST dataset in high TPR
region—E-AMIA,$LR_{n}$ performs slightly better than AMIA,$LR_{f}$. This is
simply because E-AMIA,$LR_{n}$ algorithm uses the data annotation approach
used by EMIA,$LR_{f}$ to modify AMIA,$LR_{f}$.

(a) Fashion-MNIST

(b) MNIST

Figure 9: Comparing the RTA of f-LiRA,$LR_{f}$ and EMIA,$LR_{f}$ with
AMIA,$LR_{n}$ and E-AMIA,$LR_{n}$ when $\mathcal{V}$ prepared for the target
DNN is transfered to the unknown CNNs (that $\mathcal{V}$ has not been
optimized for) trained with DP-SGD gradient updates as $t$ increases. Each is
an average value computed over 10 unknown CNNs. AMIA,$LR_{n}$ and
E-AMIA,$LR_{n}$ outperform both f-LiRA,$LR_{n}$ and EMIA,$LR_{n}$ with a
significant margin in low FPR region. (Settings: $\epsilon=0.02$, $N=50$).

###  VI-B Evaluating Differentially Private Training

Fig 9(a)-(b) compares the transferability of f-LiRA,$LR_{f}$ and EMIA,$LR_{f}$
with that of AMIA,$LR_{n}$ and E-AMIA,$LR_{n}$ to the unknown CNNs trained
with differential privacy algorithm (instead of with $L_{2}$ regularization as
in Fig. 8) for Fashion-MNIST and MNIST datasets respectively. Due to large
computational overhead, we refrain from optimizing new $\mathcal{V}$ directly
over DP-SGD CNNs and reuse $\mathcal{V}$ already prepared for the target CNN
in previous experiments. More specifically, each attack reuses $\mathcal{V}$,
prepared for the target CNN in previous experiments, to infer the membership
of given subjects for 10 unknown CNNs (that $\mathcal{V}$ has not been
optimized for) trained with $(e=1.56,\delta=10^{-5})$-DP-SGD gradient updates.

We make two key observations. (1) CNNs trained with DP-SGD are notably more
robust to MI attacks as compared to those trained with $L_{2}$ regularization
as illustrated by the reduced RTA in Fig. 9. This observation is consistent
with that of Choquette et al. [25] who note that only $L_{2}$ regularization
needs to be notably stronger in order to match the robustness of DP-SGD to
privacy attacks. (2) AMIA,$LR_{n}$ and E-AMIA,$LR_{n}$ are significantly more
effective against DP-SGD as compared to f-LiRA,$LR_{f}$ and EMIA,$LR_{f}$.
This is because (a) AMIA,$LR_{n}$ and E-AMIA,$LR_{n}$ are more transferable to
unknown CNNs as compared to f-LiRA,$LR_{f}$ and EMIA,$LR_{f}$, and (b)
AMIA,$LR_{n}$ and E-AMIA,$LR_{n}$ are significantly more effective against
well-performing DNNs that do not overfit their training data.

###  VI-C Limitations and Future Work

AMIA, similar to LiRA and EMIA, assumes access to the real data distribution
$D_{r}$, from which the training data $D_{t}$ is sampled. This assumption is
commonly shared by several standard MI attacks. Although it might be
challenging to meet this assumption in certain real-world scenarios, there
exist other (more complex) approaches, such as the model inversion and the
model stealing attacks, that can be used to estimate the training dataset
$D_{t}$, given that $f_{t}$ is vulnerable to MI attacks under the
aforementioned assumption. For example, an attacker may formulate the
optimization problem to maximize $\mathcal{A}(\cdot)$ over the subject
(similar to model inversion attacks [35, 36]). Therefore, MI attack is
believed to be a fundamental privacy attack, and understanding the
vulnerabilities of $f_{t}$ against MI attack is both critical and insightful
[3].

While our proposed attacks, AMIA and E-AMIA, to the best of our knowledge, are
currently the most computationally efficient methods for utilizing subject
membership information, they still require training multiple shadow DNNs. To
further enhance efficiency, we suggest incorporating universal adversarial
perturbations [37] that optimize the loss of member shadow DNNs and maximize
the loss of non-member shadow DNNs. Our future work aims to develop a
universal adversarial membership inference attack (U-AMIA) building upon this
approach.

##  VII Conclusions

We first present a unified framework of current MI attacks divided it into
three stages—preparation, indication and decision. We note several limitations
of Likelihood Ratio Attack (LiRA) and Enhanced Membership Inference Attack
(EMIA) at the preparation stage. Online LiRA is computationally highly
inefficient, while offline LiRA and EMIA sacrifice the membership information
of subjects for computational efficiency. This makes LiRA and EMIA perform
poorly on MNIST and Fashion-MNIST datasets where the target is well-trained,
unlike complex datasets (CIFAR-10, CIFAR-100, and Imagenet) where the target
DNN typically overfits its training set due to the experimental setup of MI
attacks. To address this, we propose Adversarial Membership Inference Attack
(AMIA) that efficiently utilizes the membership information by training member
shadow DNNs on subject batches. AMIA also optimally leverages the loss
landscape around subjects by computing adversarially minimizing a novel loss
function. We also propose the Enhanced AMIA (E-AMIA) that combines EMIA and
AMIA to further improve the attack effectiveness. We experiment with a range
of hyperparameters and observe that E-AMIA and AMIA notably outperform both
LiRA and EMIA in the low FPR region—e.g., LiRA and EMIA showed 0% TPR at 1%
FPR (99% confidence), while AMIA and E-AMIA showed 6% and 8% TPR on Fashion-
MNIST. We also study the transferability of each MI attack and show that AMIA
is most transferable followed by E-AMIA and LiRA, while EMIA is least
transferable.

## Acknowledgment

This publication was made possible by NPRP grant # [13S-0206-200273] from the
Qatar National Research Fund (a member of Qatar Foundation). The statements
made herein are solely the responsibility of the authors.

## References

  * [1] A. Qayyum, J. Qadir, M. Bilal, and A. Al-Fuqaha, “Secure and robust machine learning for healthcare: A survey,” _IEEE Reviews in Biomedical Engineering_ , vol. 14, pp. 156–180, 2020.
  * [2] R. Shokri, M. Stronati, C. Song, and V. Shmatikov, “Membership inference attacks against machine learning models,” in _2017 IEEE symposium on security and privacy (SP)_. IEEE, 2017, pp. 3–18.
  * [3] N. Carlini, S. Chien, M. Nasr, S. Song, A. Terzis, and F. Tramer, “Membership inference attacks from first principles,” in _2022 IEEE Symposium on Security and Privacy (SP)_. IEEE, 2022, pp. 1897–1914.
  * [4] J. Ye, A. Maddi, S. K. Murakonda, V. Bindschaedler, and R. Shokri, “Enhanced membership inference attacks against machine learning models,” in _Proceedings of the 2022 ACM SIGSAC Conference on Computer and Communications Security_ , 2022, pp. 3093–3106.
  * [5] N. Khalid, A. Qayyum, M. Bilal, A. Al-Fuqaha, and J. Qadir, “Privacy-preserving artificial intelligence in healthcare: Techniques and applications,” _Computers in Biology and Medicine_ , p. 106848, 2023.
  * [6] F. Tramèr, R. Shokri, A. San Joaquin, H. Le, M. Jagielski, S. Hong, and N. Carlini, “Truth serum: Poisoning machine learning models to reveal their secrets,” in _Proceedings of the 2022 ACM SIGSAC Conference on Computer and Communications Security_ , 2022, pp. 2779–2792.
  * [7] F. Boenisch, A. Dziedzic, R. Schuster, A. S. Shamsabadi, I. Shumailov, and N. Papernot, “Is federated learning a practical pet yet?” _arXiv preprint arXiv:2301.04017_ , 2023.
  * [8] T. Nguyen, P. Lai, K. Tran, N. Phan, and M. T. Thai, “Active membership inference attack under local differential privacy in federated learning,” _arXiv preprint arXiv:2302.12685_ , 2023.
  * [9] M. Jagielski, M. Nasr, C. Choquette-Choo, K. Lee, and N. Carlini, “Students parrot their teachers: Membership inference on model distillation,” _arXiv preprint arXiv:2303.03446_ , 2023.
  * [10] S. Yeom, I. Giacomelli, M. Fredrikson, and S. Jha, “Privacy risk in machine learning: Analyzing the connection to overfitting,” in _2018 IEEE 31st computer security foundations symposium (CSF)_. IEEE, 2018, pp. 268–282.
  * [11] H. Jalalzai, E. Kadoche, R. Leluc, and V. Plassier, “Membership inference attacks via adversarial examples,” _arXiv preprint arXiv:2207.13572_ , 2022\.
  * [12] H. Ali, M. S. Khan, A. Al-Fuqaha, and J. Qadir, “Tamp-x: Attacking explainable natural language classifiers through tampered activations,” _Computers & Security_, vol. 120, p. 102791, 2022.
  * [13] F. Khalid, H. Ali, M. A. Hanif, S. Rehman, R. Ahmed, and M. Shafique, “Fadec: A fast decision-based attack for adversarial machine learning,” in _2020 International Joint Conference on Neural Networks (IJCNN)_. IEEE, 2020, pp. 1–8.
  * [14] H. Ali, M. S. Khan, A. AlGhadhban, M. Alazmi, A. Alzamil, K. Al-Utaibi, and J. Qadir, “All your fake detector are belong to us: evaluating adversarial robustness of fake-news detectors under black-box settings,” _IEEE Access_ , vol. 9, pp. 81 678–81 692, 2021.
  * [15] M. A. Butt, A. Qayyum, H. Ali, A. Al-Fuqaha, and J. Qadir, “Towards secure private and trustworthy human-centric embedded machine learning: An emotion-aware facial recognition case study,” _Computers & Security_, vol. 125, p. 103058, 2023.
  * [16] S. Latif, A. Qayyum, M. Usama, J. Qadir, A. Zwitter, and M. Shahzad, “Caveat emptor: the risks of using big data for human development,” _Ieee technology and society magazine_ , vol. 38, no. 3, pp. 82–90, 2019.
  * [17] Y. Zhang, G. Bai, M. A. P. Chamikara, M. Ma, L. Shen, J. Wang, S. Nepal, M. Xue, L. Wang, and J. Liu, “Agrevader: Poisoning membership inference against byzantine-robust federated learning,” in _Proceedings of the ACM Web Conference 2023_ , 2023, pp. 2371–2382.
  * [18] Y. Chen, C. Shen, Y. Shen, C. Wang, and Y. Zhang, “Amplifying membership exposure via data poisoning,” _Advances in Neural Information Processing Systems_ , vol. 35, pp. 29 830–29 844, 2022.
  * [19] M. Jagielski, S. Wu, A. Oprea, J. Ullman, and R. Geambasu, “How to combine membership-inference attacks on multiple updated machine learning models,” _Proceedings on Privacy Enhancing Technologies_ , vol. 3, pp. 211–232, 2023\.
  * [20] J. Tan, D. LeJeune, B. Mason, H. Javadi, and R. G. Baraniuk, “A blessing of dimensionality in membership inference through regularization,” in _International Conference on Artificial Intelligence and Statistics_. PMLR, 2023, pp. 10 968–10 993.
  * [21] S. Rezaei, Z. Shafiq, and X. Liu, “Accuracy-privacy trade-off in deep ensemble: A membership inference perspective,” in _2023 IEEE Symposium on Security and Privacy (SP)_. IEEE Computer Society, 2022, pp. 1901–1918.
  * [22] Y. Liu, Z. Zhao, M. Backes, and Y. Zhang, “Membership inference attacks by exploiting loss trajectory,” in _Proceedings of the 2022 ACM SIGSAC Conference on Computer and Communications Security_ , 2022, pp. 2085–2098.
  * [23] G. Del Grosso, H. Jalalzai, G. Pichler, C. Palamidessi, and P. Piantanida, “Leveraging adversarial examples to quantify membership information leakage,” in _Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition_ , 2022, pp. 10 399–10 409.
  * [24] X. Yuan and L. Zhang, “Membership inference attacks and defenses in neural network pruning,” in _31st USENIX Security Symposium (USENIX Security 22)_ , 2022, pp. 4561–4578.
  * [25] C. A. Choquette-Choo, F. Tramer, N. Carlini, and N. Papernot, “Label-only membership inference attacks,” in _International conference on machine learning_. PMLR, 2021, pp. 1964–1974.
  * [26] N. Carlini and D. Wagner, “Towards evaluating the robustness of neural networks,” in _2017 ieee symposium on security and privacy (sp)_. Ieee, 2017, pp. 39–57.
  * [27] A. Madry, A. Makelov, L. Schmidt, D. Tsipras, and A. Vladu, “Towards deep learning models resistant to adversarial attacks,” in _International Conference on Learning Representations_ , 2018.
  * [28] F. Croce and M. Hein, “Reliable evaluation of adversarial robustness with an ensemble of diverse parameter-free attacks,” in _International conference on machine learning_. PMLR, 2020, pp. 2206–2216.
  * [29] H. Ali, M. A. Butt, F. Filali, A. Al-Fuqaha, and J. Qadir, “Consistent valid physically-realizable adversarial attack against crowd-flow prediction models,” _arXiv preprint arXiv:2303.02669_ , 2023.
  * [30] J. Cohen, E. Rosenfeld, and Z. Kolter, “Certified adversarial robustness via randomized smoothing,” in _international conference on machine learning_. PMLR, 2019, pp. 1310–1320.
  * [31] H. Ali, M. S. Khan, A. AlGhadhban, M. Alazmi, A. Alzamil, K. Al-utaibi, and J. Qadir, “Con-detect: Detecting adversarially perturbed natural language inputs to deep classifiers through holistic analysis,” _Computers & Security_, p. 103367, 2023.
  * [32] W. Nie, B. Guo, Y. Huang, C. Xiao, A. Vahdat, and A. Anandkumar, “Diffusion models for adversarial purification,” in _International Conference on Machine Learning_. PMLR, 2022, pp. 16 805–16 827.
  * [33] N. Carlini, F. Tramer, K. D. Dvijotham, L. Rice, M. Sun, and J. Z. Kolter, “(certified!!) adversarial robustness for free!” in _The Eleventh International Conference on Learning Representations_ , 2022.
  * [34] B. Jayaraman, L. Wang, K. Knipmeyer, Q. Gu, and D. Evans, “Revisiting membership inference under realistic assumptions,” _Proceedings on Privacy Enhancing Technologies_ , vol. 2021, no. 2, 2021.
  * [35] M. Fredrikson, S. Jha, and T. Ristenpart, “Model inversion attacks that exploit confidence information and basic countermeasures,” in _Proceedings of the 22nd ACM SIGSAC conference on computer and communications security_ , 2015, pp. 1322–1333.
  * [36] Y. Zhang, R. Jia, H. Pei, W. Wang, B. Li, and D. Song, “The secret revealer: Generative model-inversion attacks against deep neural networks,” in _Proceedings of the IEEE/CVF conference on computer vision and pattern recognition_ , 2020, pp. 253–261.
  * [37] S.-M. Moosavi-Dezfooli, A. Fawzi, O. Fawzi, and P. Frossard, “Universal adversarial perturbations,” in _Proceedings of the IEEE conference on computer vision and pattern recognition_ , 2017, pp. 1765–1773.