#  $\ell$-Leaks:Membership Inference Attacks with Logits

Shuhao Li  Yajie Wang
Yuan Tan
Yuanzhang Li

###### Abstract

Machine Learning (ML) has made unprecedented progress in the past several
decades. However, due to the memorability of the training data, ML is
susceptible to various attacks, especially Membership Inference Attacks
(MIAs), the objective of which is to infer the model’s training data. So far,
most of the membership inference attacks against ML classifiers leverage the
shadow model with the same structure as the target model. However, empirical
results show that these attacks can be easily mitigated if the shadow model is
not clear about the network structure of the target model.

In this paper, We present attacks based on black-box access to the target
model. We name our attack l-Leaks. The l-Leaks follows the intuition that if
an established shadow model is similar enough to the target model, then the
adversary can leverage the shadow model’s information to predict a target
sample’s membership.The logits of the trained target model contain valuable
sample knowledge. We build the shadow model by learning the logits of the
target model and making the shadow model more similar to the target model.
Then shadow model will have sufficient confidence in the member samples of the
target model. We also discuss the effect of the shadow model’s different
network structures to attack results. Experiments over different networks and
datasets demonstrate that both of our attacks achieve strong performance.

##  1 Introduction

Over the past decade, machine learning (ML) has witnessed tremendous progress,
enabling a wide range of applications, such as image recognition and machine
translation. However, recent work has demonstrated that ML, especially Deep
Learning(DL), is vulnerable to privacy leakage since they are shown to overfit
their training datasets. Adversaries can exploit this vulnerability with query
access to the trained models or the synthetic data outputs[1] [3] [5] [7] [18]
[23] [27], to infer membership of samples to the training datasets.

Figure 1: Construction of the Shadow model. Input $D_{shadow}$ into shadow
model and target model to obtain logits. As shown in Fig.2, the logits of the
target model contains valuable sample knowledge. Therefore, we designed MI-
Softmax to process the logits of the target model to retain the valuable
sample knowledge called MSLogits. The shadow model is constructed by reducing
the gap between the Outputs and MSLogits.The shadow model will be more similar
to the target model by learning the behavior of the target model.

This paper focuses on Membership Inference Attacks(MIA). Its purpose is to
determine whether a data sample is present in the original training set. For
example, suppose we use a tumor patient’s data in a hospital to train an
artificial intelligence system that predicts whether or not an undiagnosed
patient has a tumor. In that case, MIA will increase the risk of leaking
sensitive patient data. Besides, personal credit, daily physical location, and
other data are also vulnerable to MIA.

Membership inference attacks rely on the shadow model to construct an attack
model. The responsibility of the shadow model is to imitate the behavior of
the target model, so how to build the shadow model directly affects the
success rate of the final inference attack.

At present, there is no better way to build a shadow model. For example, in
the existing MIA attack, Shokri et al [30] emphasized that the shadow model
and the target model have the same network structure in the stage of building
the shadow model. It is easy to achieve when the model structure is known,
which loosens the shadow model’s construction restriction. The sampling attack
method of Rahimian et al. [24] also uses the same structure as the target
model for constructing the shadow model. The Label-Leaks method of Zheng Li et
al.[13] mentioned that When the adversary knows the main task of the target
model, it only needs to use a neural network similar to the target model to
act as a shadow model. This approach seems to increase the degree of
restriction on the shadow model, but there is no theoretical basis for
constructing the shadow model by guessing.

Assuming that the structure and parameters of the target model can’t be known,
we can’t set the same model structure as the target model or rely on guessing
to develop the shadow model. In that case, the membership inference attacks
can’t fully play the attack effect in the above paper.

Figure 2: For different samples, logits will contain valuable sample
knowledge. For (a), the output is ground truth $2$.For (b), logits will tell
us that the high probability of this picture is 2, and the small probability
is 3, hardly like other numbers. For (c), the high probability is 2, and the
small probability is 7. The information carried in logits will help the shadow
model learn the behavior of the target model.

In this paper, we propose a new inference attack method, termed l-Leaks. We
solve the problem of shadow-model construction. Our intuition is that the
logits of the target model provide more supervision signals than one-hot tags.

In machine learning, we often assume a potential functional relationship
between the input and the output. This function is unknown: learning a new
model is to approximate an unknown function from limited data. Suppose we want
the shadow model to be approximately equal to the target model, and the
functional of the target model is known. In that case, we can use a lot of
pseudo-data within the non-training set to train the shadow model, matching
the logits distribution of the shadow model to the real labels. Now, to make
the shadow model more similar to the target model without knowing the
structure of the target model network, we just need to make the shadow model
and the target model match the logits distribution under the same input.
Intuitively, the advantage is that the logits distribution of the trained
target model contains valuable sample knowledge. As shown in Fig.2, the actual
label can only tell us that Fig.2b is number 2, not 3, and not 7; but the
trained logits may tell us that it is most likely to be the number 2, and it
is unlikely to be 3, but it can never be 7. The behavior of logits is our
intuition to construct a shadow model.

Our ultimate goal is to make the distribution of the softmax output of the
shadow model and the target model sufficiently close. However, doing so
directly is problematic: in the general softmax function, the natural exponent
$e$ will widen the logit gap and normalize. The final distribution
approximates $argmax$, and its output is vectors that are close to one-hot.
One of the values is very large, and the others are tiny. In this case, the
knowledge of ”maybe 3, but never 7” mentioned earlier is very limited.

Compared with the hard output like one-hot, we want the output more smooth, so
we designed MI-Softmax(more details in Sec.4) to make the distribution of the
entire logits smoother. As shown in Fig.1, in the shadow model training
process, narrow the gap between the output and MSLogits distributions through
KL divergence. During the training process, the cross-entropy of the shadow
set label and its softmax distribution is added to the loss function to add
more supervision signals, which will significantly improve the previous shadow
model. It will improve the phenomenon that the shadow model is ”biased” by the
target model, and the shadow model will have a better-imitating effect on the
target model.

The change of l-Leaks lies in constructing the shadow model on a theoretical
basis instead of guessing or adopting the same structure as the target model.
l-Leaks has a strong performance in our experiments.

We summarize our contributions as below: first, we proposed l-Leaks method,
which solves the unfounded problem of the previous shadow-model construction;
second, in the process of constructing the shadow model, the concept of MI-
Softmax was proposed to solve the problem of model output; finally, Our method
was tested on different networks and 5 different datasets. We obtained a high
attack success rate. Our approach achieves strong performance in deep neural
network scenarios.

##  2 Related Work

###  2.1 MIAs on Classification Models

Membership inference attack refers to an attacker trying to determine whether
a piece of personal information exists in the training dataset of the target
model. When the training data contains sensitive information such as medical
data, the data owner does not want to reveal whether the personal data exists
in a specific training set. However, the membership inference attack revealed
this type of privacy.

Shokri et al. [30] conducted the pioneering work to propose the first MIA on
classification models. They invented a shadow training technique to train a
binary classifier-based attack model in a black-box setting. Salem et al. [28]
relax two main assumptions of the shadow training technique in [30], i.e.,
multiple shadow models and knowledge of the training data distribution. They
argue that the two assumptions are relatively strong, which heavily limit the
applicable scenarios of MIAs against ML models. They show that even with one
single shadow model, the attacker can achieve comparable attack performance
compared to using multiple shadow models. They also propose a data
transferring attack where a dataset used to train the shadow model is not
required to have the same distribution as the target model’s private training
dataset. Yeom et al. [33] also propose two metric-based MIAs, i.e., the
prediction correctness based MIA and the prediction loss based MIA. Compared
to binary classifier-based MIAs, metric-based MIAs are much simpler and cause
less computation cost. Long et al. [16] [17] investigate MIAs on ML models
which are not overfitted to their training data. They propose a generalized
MIA that can identify the membership of, particularly vulnerable records.

###  2.2 MIAs based metric

Li and Zhang [13] propose two label-only MIAs, i.e., a transfer-based MIA and
a perturbed-based MIA. The transfer-based MIA aims to construct a shadow model
to mimic the target model. The intuition is that if the shadow model is
similar enough to the target model, then the shadow model’s confidence scores
on an input record will indicate its membership. The perturbation-based attack
adds crafted noise to the target record to turn it into an adversarial
example. The intuition is that it is harder to perturb a member instance to a
different class than a non-member instance. Thus, the magnitude of the
perturbation can be used to distinguish members from non-members. Choquette et
al. [4] also propose two label-only MIAs, i.e., data augmentation-based MIA
and decision boundary distance-based MIA.

###  2.3 MIAs on Generative Models

Although most of the current MIA is carried out for classification models,
some MIA efforts are also for generative models.

Hayes et al. [6] introduce the first MIA on generative models in both black-
box and white-box settings.Hilprecht et al. [9] propose two MIAs on generative
models. One is a Monte Carlo integration attack designed for GANs in the
black-box setting, and the other is a reconstruction attack designed for VAEs
in the white-box setting. Liu et al. [15] proposed attack begins with
attacking a single target record and then extends to a set of records.

| Attack category
---
Attacks |  | Training data
---
details
| Target model
---
structure
Predicted label |  | Detail prediction model
---
(e.g. probabilities or logits)
Prior Works |  [10],[31],[29],[12],[25] |  $\surd$ or – |  $\surd$ or – | $\surd$ | $\surd$
l-Leaks | – | – | – | – | $\surd$
Table 1: Summarize the degree of understanding of different types of member
inference attacks on the target model, ”$\surd$”means that the attack requires
this information,”–” means that this information is not required. Figure 3: We
are training the attack model on the inputs and outputs of the shadow models.
For all records in$D_{shadow}^{train}$, we query the shadow model and obtain
the output.Then labeled 1 and added to the $A_{train}$.We also query the
shadow model with $D_{test}$ disjoint from $D_{shadow}^{train}$.The outputs on
this set are labeled 0 and also added to $A_{train}$. Having constructed a
dataset that reflects the black-box behavior of the shadow models on their
training and test datasets, we train a collection of $c_{target}$ attack
models, one per output class of the target model.

##  3 Preliminaries

In this section, we first define the membership inference attack in the
machine learning setting, second, describe the metric used, and third,
introduce the threat model used for l-Leaks.

###  3.1 MIA Against Machine Learning Models

In this part,we define the membership inference attack in the machine learning
setting. We mainly focus on classification tasks in machine learning, one of
the most common ML tasks. we use CNN to complete the image classification
task, and the selection of this network followed the setting of the previous
work [20] [21] [28] [30].A supervised ML classfication model aims to learn a
general rule that maps inputs to outputs from a labeled dataset[26].We
supposed $D_{train}={(x^{(n)},y^{(n)})}_{n=1}^{N}$ be a training dataset,$N$
is the number of sample,$x$ is a feature vector,$y$ is labels.

ML model is a function $f(\textbf{x};\Theta)$ ,input x and get output
$y=f(\textbf{x};\Theta)$.$\Theta$ is the parameter learned through the
training set.When $y$ is a discrete value, $f(\textbf{x};\Theta)$ is called
classification model.

In black-box attack,the attacker intelligently obtains the probability output
$p$ by input the data $x$ into the $f(\textbf{x};\Theta)$,then put the
obtained probability $p$ into the binary attack model. The attack model will
give $0/1$, $0$ means $x$ is not in the training dataset $D_{train}$ and $1$
means $x$ in $D_{train}$.Attack model can be defined as the following
function:

$\kappa=x,f\rightarrow{0,1}$

$\kappa$ means attack model, $x$ is the input data, $f$ is the target model.

###  3.2 Metrics Description

Our current work is oriented to the classification model in ML. We use
Accuracy as a measurement of the target model and shadow model, which is a
metric used in existing work to reflect the classification performance of the
target model; in addition to the general metrics for evaluating the target
model, existing work has also proposed or used many metrics for measuring
attack performance. What we use is AP[2] [9] [14] [19] [34], AR[22] [19] [34],
and F1-Score [8] [10] [32] [35] . A brief introduction to each standard.

AP is what fraction of records classified as members are indeed members of the
training dataset.

AR is what fraction of the training dataset’s members are correctly classified
as members.

Attack F1-score is the harmonic mean of Attack Precision and Attack Recall.

###  3.3 Threat Model Description

As shown in Tab.1, the adversary can only access the target model through the
black box. The information obtained by the attacker is limited to the training
data distribution and the black box query on the target model. When the
attacker queries the target classifier, only the logits of the input records
are obtained. Therefore, the adversary trained a shadow model $S$ to imitate
the behavior of the target model $f$, the data for training the shadow model
is $D_{shadow}^{train}$, relying on the $S$ to build a dataset for the attack
model($\kappa$) and then complete the attack.

The attack process is: first, input $D_{shadow}^{train}$ into $f$ to obtain
logits, after that, the obtained logits need to be processed, and the output
distribution can be made smoother through MI-Softmax; second, construct shadow
model $S$, combine the processed logits and the output of $S$ into a new loss.
Step 2 is critical in l-Leaks. It is used for $S$ to learn the behavior of
$f$; finally, as shown in Fig.3, make $S$ construct the training data of the
attack model $\kappa$, the training set of $\kappa$ is complete.

##  4 Methodology

To solve the problem of shadow model construction in MIA, we propose a model
learning construction method based on the target model logits, which achieve
the behavior of the shadow model imitating the target model. We provide a
theoretical basis for the construction of shadow training in the black box
state. Our proposed method aims at the classification task in machine
learning. The detailed description of the specific process is as follows.

###  4.1 Process logits to Construct Loss Function

To solve the problem of shadow model construction, we have deeply studied the
learning process of the machine learning model. For neural networks, when the
input information starts from the input layer (the input layer does not
participate in the calculation), the neurons in each layer calculate the
output of each neuron in the layer and pass it to the next layer until the
output layer calculates the output of the network, feedforward is only used to
calculate the output of the network, it does not adjust the parameters of the
network.

Backward propagation is used to adjust the network weights and thresholds
during training. The feedback mechanism is used to find the partial
derivative, and then the gradient descent is used to find the minimum value of
the cost function. This process requires supervised learning. So we use the
logits distribution of the model. After the transformation, it matches the
label. To construct the shadow model, we need to match the logits distribution
of the shadow model and the target model under a given input.

The advantage of constructing a shadow model by learning is that the target
model after model training contains valuable sample knowledge in its logits.
As shown in Fig.2, ground truth will tell us that Fig.2b is the number $2$;
however, logits will tell us that it may be the number $2$, which is unlikely
to be the number $3$ and cannot be the number $7$, so we use softmax as shown
in Eq.(1) to normalize the logits of the target and shadow models, then match
their distribution.

$P(z_{i})=\dfrac{e^{z_{i}}}{\sum_{k=1}^{n}{e^{z_{i}}}}$ (1)

In subsequent experiments, we found that in the softmax function of Eq.(1),
the natural exponent e first widens the gap between logits and then
normalizes. The final distribution approximates $argmax$, and its output is a
vector close to one-hot. One of the values is very large, and the others are
tiny. In this case, the knowledge we need will be weakened, so we hope that
after softmax normalizes logits, the information in the data can be retained.

We found that Boltzmann proposed the Softmax function in his foundational
statistical mechanic’s paper on Boltzmann distribution. The Boltzmann
distribution represents a process of simulated energy convergence to the
distribution, and its expression is as Eq.(2):

$F(state)=exp(-\frac{E}{kT})$ (2)

$E$ is the state energy, $k$ is the Boltzmann constant, and $T$ is the
thermodynamic temperature.

In statistics, the Boltzmann distribution gives the probability distribution
of a system in a specific state of energy and system temperature:

$p_{i}=\frac{exp(-\frac{z_{i}}{kT})}{\sum_{j=1}^{N}exp(-\frac{z_{j}}{kT})}$
(3)

Where $p_{i}$ is the probability of state $i$, $z_{i}$ is the state energy,
$k$ is Boltzmann’s constant, $T$ is the temperature of the system, and $M$ is
the number of states of the system.

We discussed the value of variable $T$, $T$ can take three states:

  1. 1.

When $T=1$,the formula represents the softmax function.

  2. 2.

When $T=0$,the probability of the best state is $1$, and the probability of
the other states is $0$, that is, the probability of the true category is $1$.

  3. 3.

When $T=+\infty$, it is similar to randomly selected states. That is, the
probability of each state appearing is equal. This is the so-called maximum
entropy state, which is the most chaotic state.

Finally, get a more generalized softmax function, we call it MI-Softmax, as
shown in Eq.(4):

$p_{i}=\frac{exp(\frac{z_{i}}{T})}{\sum_{j=1}^{N}exp(\frac{z_{j}}{T})}$ (4)

After adding T, we can keep more information after normalization. The
following describes the learning process.

###  4.2 Model Learning to Construct Shadow Model

We use KL divergence to reduce the gap between the two distributions, as in
the following Eq.(5).

$D_{KL}(p_{t},q_{s})=\sum_{i=1}^{N}p_{t}(x_{i})log(\frac{p_{t}(x_{i})}{q_{s}(x_{i})})$
(5)

We only need to minimize the above function. We use the above function to
derive an element $z_{i}$ in logits:

$\displaystyle\dfrac{\partial D}{\partial z_{i}}$
$\displaystyle=\dfrac{1}{T}(q_{i}-p_{i})$ (6)
$\displaystyle=\dfrac{1}{T}(\dfrac{\dfrac{exp(z_{i})}{T}}{\sum_{j}exp(\dfrac{z_{j}}{T})}-\dfrac{\dfrac{exp(v_{i})}{T}}{\sum_{j}exp(\dfrac{v_{j}}{T})})$

Assume that all logits are zero-averaged for each sample, that is
$\sum_{j}z_{j}=\sum_{j}v_{j}=0$, then there is:

$\displaystyle\dfrac{\partial D}{\partial z_{i}}$
$\displaystyle=\dfrac{1}{T}(\dfrac{1+\dfrac{exp(z_{i})}{T}}{N}-\dfrac{1+\dfrac{exp(v_{i})}{T}}{N})$
(7) $\displaystyle=\dfrac{1}{NT^{2}}(z_{i}-v_{i})$

Observing Eq.(7) finds that if $T$ is huge, logits are zero-averaged for all
samples, so multiplied by a $T^{2}$, it makes the gradient of the two terms of
the loss function roughly an order of magnitude.

If the shadow set is labeled data, we can add the label’s cross-entropy and
the shadow model’s softmax distribution to the loss function, as shown in
Eq.(8). Experiments show that the above operation will significantly improve
the performance of the shadow model because the training of the model adds
more supervise signals.

$\displaystyle L$ $\displaystyle=\alpha T^{2}L_{KL}(p_{t},q_{s})+\beta
L_{CE}(q_{s},label)$ (8) $\displaystyle=\alpha
T^{2}\sum_{i=1}^{N}p_{t}(x_{i})log(\frac{p_{t}(x_{i})}{q_{s}(x_{i})})+\beta\sum_{i=1}^{N}c_{j}log(q_{j}^{1})$

At this point, the construction of the shadow model is completed, and the
subsequent attack process is carried out according to Fig.3.

##  5 Experiment

In this section, first, we describe the experimental settings; second, define
the dataset used; finally, analyze and evaluate the result of attacks.

###  5.1 Evaluation Settings

The structure of the target model in the experiment uses LeNet-5. The shadow
model defaults to a two-layer Cov2d and three-layer fully connected neural
network. Finally, we will change the different structures of the shadow model
for comparison experiments.

We use Python 3.7 and the PyTorch framework to complete the construction of
the attack model. The experiment used is a PC, the GPU is RTX3090, and the
video memory is 24G.

###  5.2 Dataset Description

We used two types of data to evaluate our method, binary data and image data.
Binary data includes Purchases-100 and Texas-100. Image data includes MNIST,
CIFAR-10, and CIFAR-100.

Purchase100. For the Purchase100 dataset contains the shopping records of
thousands of online customers. We use a processed and simplified version of
this dataset. The dataset contains 600 different products, and each user has a
binary form indicating whether she has purchased each product (a total of
19,324 data records). We use 10, 000 randomly selected records from the
purchase dataset to train the target model as $D_{target}^{train}$. The rest
of the dataset contributes to the test set and the training sets of the shadow
models as $D_{shadow}^{train}$.

Texas100. For the Texas-100 dataset is based on the Hospital Discharge Data
public use files with information about inpatient stays in several health
facilities, released by the Texas Department of State Health Services from
2006 to 2009. The model task is to predict the patient’s main procedure based
on attributes other than secondary procedures. We focus on the 100 most
frequent procedures. The resulting dataset has 67,330 records and 6, 170
binary features. We use 20,000 randomly selected records to train the target
model as $D_{target}^{train}$, rest of the data as the training data of shadow
model as $D_{shadow}^{train}$.

MNIST. For the MNIST dataset, there are 60,000 training images and a total of
10,000 test images for researchers. We randomly selected 15,000 images in the
training set to train the target model as $D_{target}^{train}$, and the
remaining images were used to train the shadow model as $D_{shadow}^{train}$.

CIFAR. For the CIFAR-10 and CIFAR-100 datasets, [11], they both have 60,000
color images.CIFAR-10 is composed of 10 classes. Each class contains 5000
training pictures and 1000 test pictures; CIFAR-100 consists of 100 classes,
each class includes 500 training pictures and 100 test pictures. They follow
the same usage as MNIST.

Attack Set. The process of generating the attack training set $A_{train}$ is
shown in Fig.3,we put $D_{shadow}^{train}$ and $D_{test}$into the shadow model
to obtain the output probability $p_{shadow}^{train}$ and $p_{test}$, labeled
$(p_{shadow}^{train},1)$ and $(p_{test},0)$. $0$ means non-shadow model
training set. $1$ means it is the shadow model training set. So far, the
construction of the attack training set is complete.

###  5.3 Attack Result

Fig.4 describes the performance of l-Leaks. We observed that the performance
of our attack on the 5 datasets is relatively better than the attacks of Salem
et al.[28] and Zheng et al.[13] because our attack uses the logits of the
target model as input, the shadow model learns the behavior of the target
model, and they are more similar. The shadow model has more confidence in the
prediction of the member samples of the target model.

In the work of Salem et al.[28], the shadow model is trained using ground
truth labels from the shadow dataset instead of the output of the target
model, which shows that the training process is independent of the target
model. It also indicates that the closer the overfitting degree of the shadow
model and the target model is, the more influential the attack model can be
obtained. However, in real applications, the attacker cannot determine the
precise overfitting level of the target model, resulting in colossal attack
performance wasted. l-Leaks use the logits of the target model to modify the
structure of the shadow model and narrow the gap between the posterior
distributions of the two models, making the shadow model more similar to the
target model. As shown in Fig.4,we get an excellent attack effect.

((a))

((b))

((c))

Figure 4:  Comparison of our l-Leaks performance with the attack by Salem et
al and Zheng et al.The metrics we use are Attack precision(AP), Attack
Recall(AR), and Attack f1-score.The experimental results show that our method
has an excellent improvement on the 5 datasets.

###  5.4 Evaluation on Shadow Model Performance

A major difference between our attack and the previous work is the
construction of shadow models. As shown in Fig.5(a), we compared the accuracy
of our modified shadow model with the accuracy of the original shadow model.
Although in the beginning, there was a big gap between our model and the
previous model. After 30 epochs, the accuracy of the processed shadow model
and the original shadow is 98.5%, 98.7%, the target model is 99.1%, and there
is almost no difference between them, indicating that the shadow model
maintains the performance of the model after modification.

In Fig.5(b).we quantified the overfitting level of the model as the difference
in its prediction accuracy on the $D_{target}^{train}$ and $D_{test}$, similar
to previous works[28][33]. We find that the modified shadow model increases
with the increase of the overfitting level of the target model, showing a
nearly linear relationship. In contrast, the original shadow model has a
larger difference, showing the same confidence level for both the
$D_{target}^{train}$ and $D_{test}$. This experiment demonstrates that the
modified shadow model mimics the behavior of the target model and gets the
same performance as the target model on the $D_{target}^{train}$.

((a))

((b))

Figure 5: Shadow model performance comparison. Fig.5(a) shows the performance
of the modify shadow model, original shadow model, and the target model on the
test set. Fig.5(b) shows the relation between the overfitting level of the
target model (x-axis) and the performance of the shadow model (y-axis)
measured by the difference in prediction accuracy between the
$D_{target}^{train}$ and $D_{test}$.

###  5.5 Evaluation on Shadow Model Similarity

To prove that the shadow model has learned the behavior of the target model,
we use the previous work[13] method to verify the performance of the shadow
model to imitate the target model. If the shadow model $S$ and the target
model $f$ are sufficiently similar, the member state of the data sample in the
target model can be reflected in the shadow model. To quantify the prediction
confidence of the target sample on the shadow model $S$, we pay attention to
the loss of the target sample relative to the shadow model $S$. Note that the
target sample is never used to train the shadow model.

Fig.6 further shows the loss distributions of the target model members and
non-member samples(MNIST and CIFAR10) calculated on the shadow models (the
reconstructed shadow model and the unreconstructed shadow model). Although
neither member nor non-member samples are used to training the shadow model,
we still observe a specific difference between their losses.As shown in
Fig.6(a) and Fig.6(c), the difference in the l-Leaks shadow model is more
prominent.This shows that our intuition is correct, and the shadow model has
learned the behavior of the target model.

To further prove that the shadow model has learned the knowledge and behavior
of the target model, taking MNIST as an example. We try to delete all
instances of the number $2$ from $D_{shadow}^{train}$, so from the perspective
of the shadow model, $2$ is a mythical number that has never appeared before.
As shown in Tab.2, the shadow model only produced $176$ test errors in our
testing process, of which 112 were on the $1032$ $2^{\prime}s$ in the test
set. Most of the errors are caused by a low learning bias. If this deviation
increases by $4$, the shadow model will generate $89$ errors, 75 of which are
on $2^{\prime}s$. Our shadow model achieved a 88.7% correct rate with the
right bias in Test $2$, although it has never been seen before in training.

((a))

((b))

((c))

((d))

Figure 6: Measure the similarity between the shadow model and the target
model. We use MNIST and CIFAR-10 to measure. We are concerned about the loss
generated by the shadow model on the training set of the target model. If the
shadow model is sufficiently similar the target model, the corresponding loss
will distinguish between members and non-members. The experimental results
show that the shadow model under our setting has a remarkable degree of
discrimination between the member and non-member data of mnist and cifar10,
indicating the effectiveness and feasibility of the shadow model imitating the
target model.

Shadow Model | No 2 | Wrong | Right Bias | Wrong
---|---|---|---|---
l-Leaks | 82.5% |  112/1032 | 88.7% |  75/1032
Prior | 10.4% | 929/1032 | - | -

Table 2: Shadow model performance on unseen numbers. Our shadow model achieves
a prediction accuracy of 82.5% on unseen data because of the knowledge of the
target model on the number 2. The original shadow model is insensitive to
unseen numbers with only 10.4% accuracy.

###  5.6 Evaluation on MI-Softmax

Our method involves processing logits, and we compared the attack effects
before and after processing. Our comparative experiment focuses on three
scenarios: original shadow model; shadow model build with logits processed by
softmax; shadow model build logits processed by MI-Softmax. We tested our
attack effect on $5$ datasets.

As shown in Tab.3. Experimental results show that the impact of the original
shadow model is the worst. The effect of softmax processing is better than the
former. The shadow model processed by MI-Softmax has the best attack effect,
especially in the image classification task. We can see that the attack
success rate on the MNIST dataset has increased by 29%. In the Cifar10
dataset, it has risen by about 14%.

Dataset | General | Softmax | MI-Softmax | Compare
---|---|---|---|---
| $F_{1}S$ | $F_{1}S$ | $F_{1}S$ | $F_{1}S$
Purchase | 0.77 | 0.79 | 0.85 | +8%
Texas | 0.74 | 0.75 | 0.80 | +6%
MNIST | 0.62 | 0.65 | 0.91 | +29%
CIFAR10 | 0.80 | 0.81 | 0.94 | +14%
CIFAR100 | 0.83 | 0.85 | 0.95 | +12%

Table 3: MIA F1-Score under different construction conditions of the shadow
model. Test five different datasets under three cases. The comparison
experiment shows that the shadow model built by MI-Softmax has the highest
attack success rate, especially in image dataset MNIST, an increase of 29%.

###  5.7 Evaluation on Different Network Structures

Another advantage of l-Leaks is to build shadow models through learning
knowledge and the structure of the shadow model is not restricted. So we can
use different network structures to act as the shadow model.

This experiment section set the shadow model to three different structures:
the classic CNN network-VGG, our own locally implemented NN (CNN+FC), FC
(Fully Connect Network). We didn’t follow the previous work[13] that
continuously improved the parameter amount of the shadow model. Instead, we
reduce the shadow model parameters, and the final attack effect is still
strong. As shown in Tab.4, we tested on 5 datasets. As the model parameters
decrease, the accuracy of the attack falls within a tolerable range. Tolerable
means that we save calculation cost and time under the premise of reducing the
number of parameters. Our price is that the attack accuracy dropped by an
average of $2.2\%$ on the $5$ datasets.The reduced attack accuracy rate is
still higher than the previous work[28][13]. Therefore, our accuracy rate has
received a minor impact. At the same time, the calculation cost is reduced.
Compared with the previous work, our work is economical.

Summarizing the above experiments, our shadow model imitates the behavior of
the target model by learning the logits of the target model. Experiments prove
that the performance of the shadow model we constructed through learning is
excellent, and the effect is robustness.

NetWork | Purchase | Texas | MNIST | CIF10 | CIF100
---|---|---|---|---|---
| $F_{1}S$ | $F_{1}S$ | $F_{1}S$ | $F_{1}S$ | $F_{1}S$
VGG | 0.86 | 0.79 | 0.91 | 0.95 | 0.95
NN | 0.85 | 0.80 | 0.91 | 0.94 | 0.95
FC | 0.83 | 0.79 | 0.88 | 0.94 | 0.91
Compare | -3% | 0% | -3% | -1% | -4%

Table 4: MIA F1-score of shadow model under different network structures. In
the complex network VGG and the simple network FC, our attack performance is
reduced around 2%, which is acceptable. It is precise because our attack
relies on the shadow model’s ability to learn from the target model. The
shadow model we get is similar to the target model, so the shadow model has
greater confidence in the member data of the target model.

##  6 Conclusion and Future Work

###  6.1 Conclusion

In this paper, we systematically investigate the L-leaks membership inference
attack with logits to solve the problem of shadow model construction. We
propose an attack for constructing shadow models based on theory, namely
l-Leaks. For l-Leaks, she uses the target model as an oracle to inquire her
shadow dataset and get the output of the target model-logits. We proposed MI-
Softmax to process logits and modify the loss function of the shadow model to
reduce the logits distribution of the shadow model and target model, the
training of the shadow model based on it.

Our work demonstrates that our attack achieves strong performance, and a
theoretical basis can support the construction of the shadow model. In the
future, we can use the different knowledge of the target model to construct
the shadow model through various learning methods. We believe our work can
serve as a stepping stone for more advanced attacks in the future.

###  6.2 Future Work

The overfitting of the target ML model is the main factor for the success of
MIAs.The overfitting phenomenon of the ML model is usually caused by two
reasons, the high model complexity and the limited size of the training
dataset. Deep learning models learn effectively from big data and use many
epochs to repeat training on the same instance, so the model easily memorizes
the training instance. To the above problems, our next step is to design a
privacy detection framework for deep learning models, use MIA to evaluate the
degree of model leakage in actual production, and process the model’s training
data to reduce the degree of model overfitting. By MIA detecting and
processing over-fitting, the model’s performance and the degree of leakage of
the model can reach a dynamic equilibrium state during the training process.
Ultimately, prevent data leakage attacks fundamentally.

## References

  * [1] Nicholas Carlini, Chang Liu, Úlfar Erlingsson, Jernej Kos, and Dawn Song.  The secret sharer: Evaluating and testing unintended memorization in neural networks.  In 28th $\\{$USENIX$\\}$ Security Symposium ($\\{$USENIX$\\}$ Security 19), pages 267–284, 2019.
  * [2] Hanxiao Chen, Hongwei Li, Guishan Dong, Meng Hao, Guowen Xu, Xiaoming Huang, and Zhe Liu.  Practical membership inference attack against collaborative inference in industrial iot.  IEEE Transactions on Industrial Informatics, 2020.
  * [3] Min Chen, Zhikun Zhang, Tianhao Wang, Michael Backes, Mathias Humbert, and Yang Zhang.  When machine unlearning jeopardizes privacy.  arXiv preprint arXiv:2005.02205, 2020.
  * [4] Christopher A Choquette-Choo, Florian Tramer, Nicholas Carlini, and Nicolas Papernot.  Label-only membership inference attacks.  In International Conference on Machine Learning, pages 1964–1974. PMLR, 2021.
  * [5] Karan Ganju, Qi Wang, Wei Yang, Carl A Gunter, and Nikita Borisov.  Property inference attacks on fully connected neural networks using permutation invariant representations.  In Proceedings of the 2018 ACM SIGSAC conference on computer and communications security, pages 619–633, 2018.
  * [6] Jamie Hayes, Luca Melis, George Danezis, and Emiliano De Cristofaro.  Logan: Membership inference attacks against generative models.  In Proceedings on Privacy Enhancing Technologies (PoPETs), volume 2019, pages 133–152. De Gruyter, 2019.
  * [7] Xinlei He, Jinyuan Jia, Michael Backes, Neil Zhenqiang Gong, and Yang Zhang.  Stealing links from graph neural networks.  In 30th $\\{$USENIX$\\}$ Security Symposium ($\\{$USENIX$\\}$ Security 21), 2021.
  * [8] Yang He, Shadi Rahimian, Bernt Schiele, and Mario Fritz.  Segmentations-leak: Membership inference attacks and defenses in semantic image segmentation.  In European Conference on Computer Vision, pages 519–535. Springer, 2020.
  * [9] Benjamin Hilprecht, Martin Härterich, and Daniel Bernau.  Monte carlo and reconstruction membership inference attacks against generative models.  Proc. Priv. Enhancing Technol., 2019(4):232–249, 2019.
  * [10] Bo Hui, Yuchen Yang, Haolin Yuan, Philippe Burlina, Neil Zhenqiang Gong, and Yinzhi Cao.  Practical blind membership inference attack via differential comparisons.  arXiv preprint arXiv:2101.01341, 2021.
  * [11] Alex Krizhevsky, Geoffrey Hinton, et al.  Learning multiple layers of features from tiny images.  2009\.
  * [12] Klas Leino and Matt Fredrikson.  Stolen memories: Leveraging model memorization for calibrated white-box membership inference.  In 29th $\\{$USENIX$\\}$ Security Symposium ($\\{$USENIX$\\}$ Security 20), pages 1605–1622, 2020.
  * [13] Zheng Li and Yang Zhang.  Membership leakage in label-only exposures.  arXiv preprint arXiv:2007.15528, 2020.
  * [14] Hongbin Liu, Jinyuan Jia, Wenjie Qu, and Neil Zhenqiang Gong.  Encodermi: Membership inference against pre-trained encoders in contrastive learning.  arXiv preprint arXiv:2108.11023, 2021.
  * [15] Kin Sum Liu, Chaowei Xiao, Bo Li, and Jie Gao.  Performing co-membership attacks against deep generative models.  In 2019 IEEE International Conference on Data Mining (ICDM), pages 459–467. IEEE, 2019.
  * [16] Yunhui Long, Vincent Bindschaedler, Lei Wang, Diyue Bu, Xiaofeng Wang, Haixu Tang, Carl A Gunter, and Kai Chen.  Understanding membership inferences on well-generalized learning models.  arXiv preprint arXiv:1802.04889, 2018.
  * [17] Yunhui Long, Lei Wang, Diyue Bu, Vincent Bindschaedler, Xiaofeng Wang, Haixu Tang, Carl A Gunter, and Kai Chen.  A pragmatic approach to membership inferences on machine learning models.  In 2020 IEEE European Symposium on Security and Privacy (EuroS&P), pages 521–534. IEEE, 2020.
  * [18] Luca Melis, Congzheng Song, Emiliano De Cristofaro, and Vitaly Shmatikov.  Exploiting unintended feature leakage in collaborative learning.  In 2019 IEEE Symposium on Security and Privacy (SP), pages 691–706. IEEE, 2019.
  * [19] Yuantian Miao, Xue Minhui, Chao Chen, Lei Pan, Jun Zhang, Benjamin Zi Hao Zhao, Dali Kaafar, and Yang Xiang.  The audio auditor: user-level membership inference in internet of things voice services.  Proceedings on Privacy Enhancing Technologies, 2021:209–228, 2021\.
  * [20] Milad Nasr, Reza Shokri, and Amir Houmansadr.  Machine learning with membership privacy using adversarial regularization.  In Proceedings of the 2018 ACM SIGSAC Conference on Computer and Communications Security, pages 634–646, 2018.
  * [21] Milad Nasr, Reza Shokri, and Amir Houmansadr.  Comprehensive privacy analysis of deep learning: Passive and active white-box inference attacks against centralized and federated learning.  In 2019 IEEE symposium on security and privacy (SP), pages 739–753. IEEE, 2019.
  * [22] Milad Nasr, Shuang Song, Abhradeep Thakurta, Nicolas Papernot, and Nicholas Carlini.  Adversary instantiation: Lower bounds for differentially private machine learning.  arXiv preprint arXiv:2101.04535, 2021.
  * [23] Xudong Pan, Mi Zhang, Shouling Ji, and Min Yang.  Privacy risks of general-purpose language models.  In 2020 IEEE Symposium on Security and Privacy (SP), pages 1314–1331. IEEE, 2020.
  * [24] Shadi Rahimian, Tribhuvanesh Orekondy, and Mario Fritz.  Sampling attacks: Amplification of membership inference attacks by repeated queries.  arXiv preprint arXiv:2009.00395, 2020.
  * [25] Shahbaz Rezaei and Xin Liu.  On the difficulty of membership inference attacks.  In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 7892–7900, 2021.
  * [26] Stuart Russell and Peter Norvig.  Artificial intelligence: a modern approach.  2002\.
  * [27] Ahmed Salem, Apratim Bhattacharya, Michael Backes, Mario Fritz, and Yang Zhang.  Updates-leak: Data set inference and reconstruction attacks in online learning.  In 29th $\\{$USENIX$\\}$ Security Symposium ($\\{$USENIX$\\}$ Security 20), pages 1291–1308, 2020.
  * [28] Ahmed Salem, Yang Zhang, Mathias Humbert, Pascal Berrang, Mario Fritz, and Michael Backes.  Ml-leaks: Model and data independent membership inference attacks and defenses on machine learning models.  Network and Distributed System Security Symposium (NDSS)., 2019\.
  * [29] Reza Shokri, Martin Strobel, and Yair Zick.  On the privacy risks of model explanations.  In Proceedings of the 2021 AAAI/ACM Conference on AI, Ethics, and Society, pages 231–241, 2021.
  * [30] Reza Shokri, Marco Stronati, Congzheng Song, and Vitaly Shmatikov.  Membership inference attacks against machine learning models.  In 2017 IEEE Symposium on Security and Privacy (SP), pages 3–18. IEEE, 2017.
  * [31] Liwei Song and Prateek Mittal.  Systematic evaluation of privacy risks of machine learning models.  In 30th $\\{$USENIX$\\}$ Security Symposium ($\\{$USENIX$\\}$ Security 21), 2021.
  * [32] Bingzhe Wu, Chaochao Chen, Shiwan Zhao, Cen Chen, Yuan Yao, Guangyu Sun, Li Wang, Xiaolu Zhang, and Jun Zhou.  Characterizing membership privacy in stochastic gradient langevin dynamics.  In Proceedings of the AAAI Conference on Artificial Intelligence, volume 34, pages 6372–6379, 2020.
  * [33] Samuel Yeom, Irene Giacomelli, Matt Fredrikson, and Somesh Jha.  Privacy risk in machine learning: Analyzing the connection to overfitting.  In 2018 IEEE 31st Computer Security Foundations Symposium (CSF), pages 268–282. IEEE, 2018.
  * [34] Chiyuan Zhang, Samy Bengio, Moritz Hardt, Benjamin Recht, and Oriol Vinyals.  Understanding deep learning (still) requires rethinking generalization.  Communications of the ACM, 64(3):107–115, 2021.
  * [35] Jingwen Zhang, Jiale Zhang, Junjun Chen, and Shui Yu.  Gan enhanced membership inference: A passive local attack in federated learning.  In ICC 2020-2020 IEEE International Conference on Communications (ICC), pages 1–6. IEEE, 2020.