# Membership Inference Attacks on Diffusion Models
via Quantile Regression

Shuai Tang111 Shuai and Steven are the lead authors, and other authors are
ordered alphabetically<EMAIL_ADDRESS>Amazon AWS AI/ML  Zhiwei Steven Wu∗
Amazon AWS AI/ML  Carnegie Mellon University
Sergul Aydore  Amazon AWS AI/ML  Michael Kearns  Amazon AWS AI/ML  University
of Pennsylvania  Aaron Roth  Amazon AWS AI/ML  University of Pennsylvania

###### Abstract

Recently, diffusion models have become popular tools for image synthesis
because of their high-quality outputs. However, like other large-scale models,
they may leak private information about their training data. Here, we
demonstrate a privacy vulnerability of diffusion models through a _membership
inference (MI) attack_ , which aims to identify whether a target example
belongs to the training set when given the trained diffusion model. Our
proposed MI attack learns quantile regression models that predict (a quantile
of) the distribution of reconstruction loss on examples not used in training.
This allows us to define a granular hypothesis test for determining the
membership of a point in the training set, based on thresholding the
reconstruction loss of that point using a custom threshold tailored to the
example. We also provide a simple bootstrap technique that takes a majority
membership prediction over “a bag of weak attackers” which improves the
accuracy over individual quantile regression models. We show that our attack
outperforms the prior state-of-the-art attack while being substantially less
computationally expensive — prior attacks required training multiple “shadow
models” with the same architecture as the model under attack, whereas our
attack requires training only much smaller models.

##  1 Introduction

Diffusion models, based on generative neural networks, have gained attention
in the field of image generation [11, 21]. It has been shown that diffusion
models are remarkably capable of generating images that are higher-quality
than previous approaches such as GANs and VAEs, while also being more
scalable. However, as the size of these models has grown drastically over the
last decade, so has the privacy concern that these large-scale diffusion
models may reveal sensitive information about the dataset they are trained on.

One of the most popular classes of methods to evaluate the privacy risks of
machine learning (ML) models is _membership inference (MI) attacks_ (e.g.,
[12, 20, 14, 13, 18, 3]), in which an attacker aims to determine if a target
example belongs to the training dataset given the trained model. The success
of a MI attack falsifies privacy protections on the existence of any
individual, i.e. differential privacy guarantees [9]. MI can also be
disclosive if e.g. membership in the training data is determined based on a
sensitive attribute (as it would if e.g. the dataset consisted of medical
records for patients with a particular disease). In addition, MI attacks can
be a building block for other more sophisticated attacks such as _extraction
attacks_ on generative models. In general, a successful MI attack with
reasonable side information is a strong indicator of a privacy vulnerability.
Finally, when applied to differentially private algorithms [8], MI attacks can
serve as privacy auditing tools by providing lower bounds on the privacy
parameters, which in turn assess the tightness of the privacy analyses [13,
17] and help identify potential errors in the privacy proof or implementation
[23, 22].

A majority of the existing MI attacks focus on supervised learning [26, 25,
15, 18, 20, 3], and there has been significantly less development on MI
attacks against generative models (e.g., [10, 24, 4]). The goal of our work is
to develop strong MI attacks against state-of-the-art diffusion models.

Our work extends the quantile-regression-based attacks in [1] for supervised
learning to attacks for diffusion models. For a given trained diffusion model
parameterized by $\theta$, our attack first learns a quantile regression model
on public auxiliary data that predicts the $\alpha$-quantile $q_{\alpha}(z)$
of $\theta$’s reconstruction loss on each example $z$ (formally defined in
Definition 2.1). Then we indicate an example is a member of the training set
if its reconstruction loss is lower than its predicted $\alpha$-quantile. By
design, the attack has a false positive rate of $\alpha$: that is the
probability that it incorrectly declares a randomly selected point $z$ that
was not used in training to have been used in training is $\alpha$. We further
boost the attack performance of our approach through bagging aggregation over
small quantile regression models. We evaluate our attack on diffusion models
trained on image datasets, and demonstrate four major advantages:

  * •

Our quantile-regression-based attack obtains state-of-the-art accuracy on
several popular vision datasets. Even though our attacks leverage the same
reconstruction loss function considered in [7], their attack leverages the
same _marginal approach_ in [26] that applies a uniform threshold (that is,
the $\alpha$-quantile on the marginal distribution over the reconstruction
loss) across all examples. In comparison, our attack is _conditional_ since it
applies a finer-grained per-example threshold when performing membership
inference.

  * •

Compared to the prior state-of-the-art MI attacks against diffusion models
[19] also in the white-box setting, we achieve higher accuracy without
suffering their computational cost. Similar to the Likelihood Ratio Attack
(LiRA) attack proposed by [3], the Gradient-Subsample-Aggregate (GSA) attack
in [19] requires training multiple _shadow models_ , each of which is obtained
by running the training algorithm of the diffusion model under attack on a
randomly drawn dataset. While the accuracy of the MI attack improves as the
number of shadow models increases, their approach also becomes computationally
prohibitive. In comparison, our approach only requires learning _tiny_
quantile regression models.

  * •

Since our attack does not rely on shadow models, it also requires
significantly fewer details about the training algorithm of the diffusion
model under attack, such as hyperparameters and network architecture used in
training. In fact, our attack is effective even though the neural network for
the quantile regression model has significantly fewer parameters than the
attacked diffusion models.

  * •

While both our work and the prior work of [1] rely on quantile regression for
MI attacks, an important distinction in ours is the use of bootstrap
aggregation — _bagging_ — that takes an ensemble of tiny quantile regression
models, namely a “bag of weak attackers.” Bagging generally improves the
attack performance by reducing the variance of the individual models, each of
which can be viewed as a weak hypothesis test. The use of bagging immediately
enriches the space of MI attacks. Specifically, we use bagging techniques in
conjunction with small quantile regression models, which leads to a
substantial improvement in accuracy by introducing little computational
overhead (due to the small model size). In comparison, the MI attack in [1]
primarily leverages a parametric approach to fit a single quantile regression
model, and it falls short in performance compared to the shadow models
approach [3] on datasets such as CIFAR10.

##  2 Background and Preliminaries

We here present the objective of a Membership Inference (MI) attack along with
required side information, and briefly introduce diffusion models for context.

###  2.1 Membership inference attacks

Membership inference (MI) is a common privacy attack that attempts to predict
whether a given example was used to train a machine learning model (e.g., [12,
26, 20, 13, 15, 18, 3]). Our work focuses on performing MI attacks on
diffusion models.

Problem statement. Given a training dataset $\mathbf{Z}$ drawn from an
underlying distribution $P$, a diffusion model $\theta$ is trained on
$\mathbf{Z}$. The goal of a membership inference attack is to infer whether a
target example $z^{*}$ was included in the training set $\mathbf{Z}$ or not.

Adversary’s side information. Similar to almost all prior work on membership
inference [3, 7, 1, 20], we assume the adversary has access to some public
data drawn from $P$. In the standard terminology of MI, there are two types of
access to the algorithm’s output. In a _black-box_ attack, the adversary only
has access to the generated synthetic data. In a _white-box attack_ , the
adversary has access to the generative model $G$. In this work, we focus on
white-box attacks, and specifically we only need access to the parameters of
the trained diffusion model, without information regarding the training
algorithm.

###  2.2 Diffusion Models

Before describing our attack, it is helpful to briefly describe how diffusion
models work at a high level, following the notation of [11]. For a real image,
a diffusion model provides a stochastic path from the image to noise. A
diffusion model consists of two processes:

  1. 1.

a $T$-step diffusion process (denoted as $q$ below) that iteratively adds
Gaussian noise to an image, and

  2. 2.

a denoising process (denoted as $p_{\theta}$ below) that gradually
reconstructs the image from noise.

Let $z_{0}$ be the real image without noise and $z_{T}$ be the noisy image
with the largest amount of noise. The transitions of diffusion and denoising
are mathematically described as:

$\displaystyle
q({z_{t}|z_{t-1}})=\mathcal{N}(z_{t};\sqrt{1-\beta_{t}}z_{t-1},\beta_{t}I)$
(1) $\displaystyle
p_{\theta}(z_{t-1}|z_{t})=\mathcal{N}(z_{t-1};\mu_{\theta}(z_{t},t),\Sigma_{\theta}(z_{t},t))$
(2)

where $q_{z_{t}|z_{t-1}}$ is the probability distribution of the diffused
image $z_{t}$ given the previous image $z_{t-1}$, $p_{\theta}(z_{t-1}|z_{t})$
is the probability distribution of the denoised image $z_{t-1}$ given the
noisy image $z_{t}$, $\mu_{\theta}(\cdot)$ and $\Sigma_{\theta}(\cdot)$ are
the mean and covariance of the denoised image, respectively, as parameterized
by the model parameters $\theta$ , and $\beta_{t}$ is a noise schedule that
controls the amount of noise added at each step. Moreover, the marginal
distribution at any time step $t$ given the example $z_{0}$ can be written as

$\displaystyle q(z_{t}\mid
z_{0})=\mathcal{N}(z_{t};\sqrt{\overline{\alpha}_{t}}z_{0},(1-\overline{\alpha}_{t})I),$
(3)

where $\alpha_{t}=1-\beta_{t}$ and
$\overline{\alpha}_{t}=\prod_{s=1}^{t}\alpha_{s}$. We will work with the
following re-parameterization of $\mu_{\theta}$ with

$\displaystyle\mu_{\theta}(z_{t},t)=\frac{1}{\sqrt{\alpha_{t}}}\left(z_{t}-\frac{\beta_{t}}{1-\overline{\alpha}_{t}}\epsilon_{\theta}(z_{t},t)\right)$
(4)

where $\epsilon_{\theta}$ is a predictor (given by $\theta$) that predicts the
noise component given $z_{t}$.

Loss function. Many MI attacks proceed by identifying a loss function and
making membership inference by comparing the loss on the target example with a
threshold. Intuitively, if the loss is unusually low, then there is evidence
that the example was part of the training set. For supervised learning models,
MI attacks typically leverage the classification loss (e.g., the cross-entropy
loss). For diffusion models, existing work has proposed candidates of loss
functions that measure the reconstruction error at different time steps of the
diffusion process [5, 7]. We leveraged the $t$-error function defined in [7],
which has the compelling advantage that it is deterministic and avoids
repeated sampling from the diffusion process. Consider the following
deterministic approximation of the diffusion and denoising processes:

$\displaystyle z_{t+1}$
$\displaystyle=\phi_{\theta}(z_{t},t)=\sqrt{\overline{\alpha}_{t+1}}f_{\theta}(z_{t},t)+\sqrt{1-\overline{\alpha}_{t+1}}\epsilon_{\theta}(z_{t},t)$
(5) $\displaystyle z_{t-1}$
$\displaystyle=\psi_{\theta}(z_{t},t)=\sqrt{\overline{\alpha}_{t-1}}f_{\theta}(z_{t},t)+\sqrt{1-\overline{\alpha}_{t-1}}\epsilon_{\theta}(z_{t},t)$
(6)

where
$f_{\theta}(z_{t},t)=\frac{z_{t}-\sqrt{1-\overline{\alpha}_{t}}\epsilon_{\theta}(z_{t},t)}{\sqrt{\overline{\alpha}_{t}}}$
is the estimate of $z_{0}$ given the $z_{t}$ and the prediction
$\epsilon_{\theta}(z_{t},t)$. Then we could also define the deterministic
reverse result as

$\displaystyle\Phi(z_{0},t)=\phi_{\theta}(\cdots\phi_{\theta}(\phi_{\theta}(z_{0},0),1)\ldots,t-1)$
(7)

Now we can define the reconstruction loss function, termed as $t$-error, that
is used in our MI attack.

######  Definition 2.1.

($t$-error) For a given sample $z_{0}$ and the deterministic reverse result
$\tilde{z}_{t}=\Phi_{\theta}(z_{0},t)$ at time step $t$, the approximated
posterior estimation error at step $t$ is defined as $t$-error:

$\displaystyle\hat{\ell}_{t}({\theta,z_{0}})=||\psi_{\theta}(\phi_{\theta}(\tilde{z}_{t},t),t)-\tilde{z}_{t}||^{2},$
(8)

where $\hat{\ell}_{t}$ is estimated from the deterministic reverse function
that is used to approximate the sampling process in the Markov Chain, and
$\tilde{z}_{t}$ refers to the deterministic outcome. Intuitively, the
$t$-error function measures how much we change $\tilde{z}_{t}$ if we take one
step in the deterministic diffusion process $\phi_{\theta}$ and then rewind
back with one step of deterministic denoising $\psi_{\theta}$. While this loss
function is not what the training algorithm optimizes, it provides a
deterministic approximation to the loss function during training [7, 11].
Thus, smaller $t$-error values provides evidence that $z_{0}$ was used to
train the diffusion model $\theta$.

##  3 MI Attacks with Quantile Regression

We will now describe our new membership inference attacks. Under the setting
in Sec. 2.1, we assume that the attacker has access to a set of public
examples $D$ drawn from the underlying distribution $P$. Given the public
dataset $D$, a choice of $t$ for the $t$-error function, and the trained
diffusion model $\theta$, the attacker learns a quantile regressor
$q_{\alpha}$ such that $q_{\alpha}(z)$ predicts the $\alpha$-quantile of the
$t$-error $\hat{\ell}_{t}(\theta,z)$ for each example $z$ in $D$, where
$\alpha$ is a parameter that controls the false-positive rate. Then on any
target example $z^{*}$, the attacker declares the example is a member of the
training set if and only if the $t$-error $\hat{\ell}_{t}(\theta,z^{*})\leq
q_{\alpha}(z^{*})$. The formal description of the algorithm is in Algorithm 1.

By design, our attack has a false-positive rate of $\alpha$, which is the
probability that an attacker incorrectly declares a randomly selected point
$z$ that was not used in training to have been used in training is $\alpha$.
By varying the parameter $\alpha$, we can then trace the trade-off curves of
_true-positive rates_ at different false-positive rates.

Algorithm 1 Quantile Regression MI attacks for Diffusion Model

A set of auxiliary examples $D$ drawn from $P$, target example $z^{*}$,
trained diffusion model $\theta$, a choice of $t$ for $t$-error function.
Target false-positive rate $\alpha$.

for each $z\in D$ do

evaluate the score $\hat{\ell}_{t}(\theta,z)$

end for

Learn a quantile regression model $q_{\alpha}$ such that $q_{\alpha}(z)$
predicts the $\alpha$-quantile of the score $\hat{\ell}_{t}(\theta,z)$
conditioned on $z$.

return "IN" if $\hat{\ell}_{t}(\theta,z^{*})\leq q_{\alpha}(z^{*})$, otherwise
"NO"

Algorithm 2 Bag of Weak Attackers

A set of auxiliary examples $D$ drawn from $P$, target example $z^{*}$,
trained diffusion model $\theta$, a choice of $t$ for $t$-error function,
target false-positive rate $\alpha$, and the number of weak attackers $m$.

Initialize vote=$0$

for each $z\in D$ do

evaluate the score $\hat{\ell}_{t}(\theta,z)$

end for

for $i\in[m]$ do

Bootstrap sampling a dataset $D_{i}$ from $D$ (Sample with replacement, and
$|D_{i}|=|D|$)

Learn a $\alpha$-quantile regression model $q_{\alpha,i}$ on $D_{i}$ (such
that $q_{\alpha,i}(z)$ predicts the $\alpha$-quantile of the score
$\hat{\ell}_{t}(\theta,z)$ conditioned on $z$.)

$\text{vote}=\text{vote}+1$ if $\hat{\ell}_{t}(\theta,z^{*})\leq
q_{\alpha,i}(z^{*})$

end for

return "IN" if $\text{vote}\geq\frac{m}{2}$, otherwise "NO"

###  3.1 Quantile Regression Learner

A generic way to train a quantile regression model is to optimize pinball loss
over some function class $\mathcal{Q}$ (e.g., neural networks). Formally, for
any observed $t$-error $\hat{\ell}$ and quantile prediction from $q_{\alpha}$
at a target level $\alpha$, the pinball loss is defined as

$\displaystyle
L_{\alpha}(\hat{\ell},q_{\alpha})=(q_{\alpha}-\hat{\ell})(\mathbf{1}[\hat{\ell}\leq
q_{\alpha}]-\alpha)$ (9)

where $\mathbf{1}(\cdot)$ is an indicator function Then we can find a quantile
regression model $q_{\alpha}(\cdot)$ that minimizes the _pinball loss_ :

$\displaystyle\min_{q_{\alpha}\in\mathcal{Q}}\sum_{z\in
D}L_{\alpha}(\hat{\ell}_{t}(\theta,z),q_{\alpha}(z)),$ (10)

where $\mathcal{Q}$ is the class of quantile regression models. The pinball
loss is minimized by the function that predicts for each $z$ the target
$\alpha$-quantile of the $t$-error conditioned on $z$. However, prior work [1]
has found that pinball loss tends to be a difficult loss function to minimize.

Figure 1: Distrbutions of the (negative) log transformation of the $t$-error
on members and nonmembers of a dataset. It is clear that on each dataset,
members and nonmembers have slightly different marginal score distributions,
however, they are not drastically different from each other, which explains
why the marginal baselines are not optimal, and also motivates our approach
that conditions the score prediction on the input sample.

###  3.2 Bag of Weak Attackers

One of the challenges, as is also mentioned in [1], is that sometimes directly
optimizing pinball loss is difficult in practice since the gradient is either
$-\alpha$ or $1-\alpha$ depending on whether the quantile prediction is
smaller or larger than the target. Rather than tackling the optimization
challenge directly, we take advantage of the fact that the quantile regression
model here can be a much smaller neural network than the targeted diffusion
models, which enables us to train multiple models with minimal computational
overhead. In particular, we adopt the bagging approach [2] to improve the
generation of our attack by taking an ensemble of tiny models trained on
bootstrapped datasets. Since individual weak models may have relatively poor
attack performance, and bagging improves the performance, we call our approach
“bag of weak attackers”. After learning, each weak attacker makes a binary
decision on an input sample to decide whether this sample is used in training
the diffusion model or not, and we simply take the majority vote over all weak
attackers to obtain the final decision. The formal description of the
algorithm is in Alg. 2. It is worth noting that quantile regression models
only need to be trained once before attacking any sample, and can be used to
attack many samples since its computational cost is amortized over all samples
under attack.

In Section 4, we empirically demonstrate the performance improvement from
bagging, and the success of our attack using a bag of weak attackers, which
are efficient tiny models that are fast to learn.

##  4 Experimental Details

We demonstrate the effectiveness of our membership inference attack via
quantile regression on four de-noising diffusion probabilistic models [11]
(DDPMs) trained on CIFAR-10, CIFAR-100 [16], STL-10 [6] and Tiny-ImageNet,
respectively. On each dataset, data samples are split into two halves, and one
half is regarded as the private samples $\mathbf{Z}$ for training a DDPM. The
other half is then split into two sets, including one as the public samples
$D$ that are auxiliary information, and the other as the holdout set for
testing. On public samples, we train quantile regression models.

The base for our quantile regression model is a ResNet model, and it is
attached with multiple prediction heads, each of which predicts target
$\alpha$-quantile for a specific value of $\alpha$. Compared to the standard
ResNet-18 model for classifying CIFAR-10, we design each attacker to be much
smaller than the standard ResNet model. We present results with a varying
number of weak attackers, each with a varying number of parameters.

In our experiments, we use the same diffusion models trained on CIFAR10 and
CIFAR100 released by [7], and trained our own diffusion models — specifically
DDPM — on STL10 and Tiny-ImageNet using lower resolution, $32\times 32$, due
to the limit of computing resources. Each diffusion model was trained with 80k
steps, and it took around 2 days to finish training on a single V100 GPU card.
After obtaining these trained diffusion models, for membership inference
attacks, we use a fixed $t=50$ in the $t$-error function. [7] suggested that
the choice of $t$ does not influence the results drastically.

We adopt the same evaluation metric as prior work [1, 3]. Specifically, we are
interested in the True Positive Rates (TPRs) at very low False Positive Rates
(FPRs). Intuitively, a successful membership inference attack should identify
true members with high accuracy, and in the meantime, make few mistakes on
declaring nonmembers as members.

###  4.1 Comparison Partners

We mainly compare our MI attacks via quantile regression with two approaches.
The first one is a simple marginal baseline, which only looks at the error
distributions of members and nonmembers, respectively, and for a target FPR
value $\alpha$, it computes the quantile on $t$-errors of the public samples,
and then the performance of this marginal baseline is evaluated on the private
samples and the holdout set. It is clear that the marginal baseline only
produces a single threshold for a target FPR, and it does not condition on the
input images, whereas ours learns to predict the threshold for a given image,
thus each images has a different threshold for a target FPR. Fig. 1
illustrates the error distributions on individual datasets, and it is clear
that the error distribution of members is different from that of nonmembers,
but using a uniform (single) threshold for all samples would lead to poor
performance. Our attack produces, for a fixed $\alpha$-value, a sample-
conditioned threshold by learning quantile regression models, thus, for each
sample under attack, the threshold is unique, which empowers the membership
inference attack.

The other comparison partner is also a white-box attack using LiRA with
gradient information [19], namely GSA. The LiRA attack formulates MI as
hypothesis testing. Let $\theta$ denote the trained generative model, then the
two competing hypotheses are

$\displaystyle H_{0}:\theta\sim A(\mathbf{Z})\mid
z^{*}\not\in\mathbf{Z}\qquad\mbox{ and }\qquad H_{1}:\theta\sim
A(\mathbf{Z})\mid z^{*}\in\mathbf{Z}.$ (11)

These hypotheses correspond to whether or not the input dataset $\mathbf{Z}$
includes the target example $z^{*}$. Despite the simple formulation,
estimating the two distributions requires training shadow models using random
subsets from the same data domain, and each shadow model is a diffusion model,
which may take days to train.

Instead of looking at the distribution over the algorithm’s output, we instead
consider the distribution over examples with and without conditioning on the
fixed observed output $\theta$ returned by a training algorithm. Compared with
unconditioned hypothesis testing approaches, including LiRA, this construction
forces the test to be model-dependent. Formally, given the target point
$z^{*}$ and the trained generative model $\theta$, there are two hypotheses:

$\displaystyle H_{0}\colon z^{*}\sim P\mid A(\mathbf{Z})=\theta\qquad\mbox{
and }\qquad H_{1}\colon z^{*}\sim\mathbf{Z}\mid A(\mathbf{Z})=\theta$ (12)

That is, $H_{0}$ asserts that $z^{*}$ is a random example drawn from the
underlying distribution $P$, but $H_{1}$ asserts $z^{*}$ is a random example
drawn from a dataset $\mathbf{Z}$ that produces the trained model $\theta$. In
this view, MI is about distinguishing two distributions over examples instead
of over trained models. Distinguishing distributions over trained models, e.g.
LiRA, requires learning numerous shadow models using subsets of public data
from the same domain. By conditioning the test on the target model in the
above setup, MI attack then only needs to train a quantile regression model
that predicts the quantile of the model score distribution for each example,
rather than numerous shadow models. If the model score of the target example
$z^{*}$ is lower than its predicted $\alpha$-quantile, then the attack
indicates the example was used in the training set. By design, the attack aims
to achieve achieve a false-positive rate of $\alpha$ (e.g., 5%).

The advantages of our attack is that, firstly, our hypothesis testing setup
takes the condition on the target model, which makes our attack model-
specific; secondly, our attack only requires learning a bag of tiny models,
which is much more computationally efficient.

Regarding the quantile-regression-based attack in [1], which has a similar
concept as ours but for attacking classification models, the main difference
is that ours doesn’t require using the surrogate Gaussian likelihood objective
or hyperparameter optimization (HPO), which doesn’t have a fixed training time
for an attack, and ours only requires bagging over tiny neural networks
without HPO. We speculate that the bagging method over tiny networks works
well in our setting because of the choice of reconstruction error ($t$-error
in Definition 2.1).

Table 1: Performance of MI Attacks on CIFAR10 and CIFAR100. Each weak attack
is a neural network with only 5,666 parameters, and the number of weak
attackers is 7.

Dataset | CIFAR-10 | CIFAR-100
---|---|---
MI Attack | TPR @ 1% FPR | TPR @ 0.1% FPR | TPR @ 1% FPR | TPR @ 0.1% FPR
Bag of Weak Attackers | 99.94% | 99.86% | 99.89% | 99.75%
GSA1 (Shadow Models) | 99.70% | 82.90% | - | -
GSA2 (Shadow Models) | 97.88% | 58.57% | - | -
Marginal Baseline | 9.6% | 0.7% | 11.06% | 5.76%

Table 2: Performance of MI Attacks on Tiny-ImageNet and STL-10

Dataset | Tiny-ImageNet | STL-10
---|---|---
MI Attack | TPR @ 1% FPR | TPR @ 0.1% FPR | TPR @ 1% FPR | TPR @ 0.1% FPR
Bag of Weak Attackers | 99.99% | 99.98% | 99.98% | 99.98%
Marginal Baseline | 8% | 0.32% | 5.78% | 0.55%

###  4.2 Main Results

Numerical results are presented in Table 1 and Table 2, and they are averaged
across 10 random seeds. The weak attacker in our experiment is a small neural
network with only 5,666 parameters, and the number of weak attackers is 7. We
can see that our attack on CIFAR10, besides being much more efficient,
outperforms GSA attacks when we focus on lower TPR (0.1%). We also have
demonstrated the effectiveness of our attack on diffusion models trained on
other image datasets, including STL10 and Tiny-ImageNet in Table 1 and 2.
Besides the performance improvement over prior work, our algorithm is also
computationally efficient and requires no knowledge about the training
algorithm of the diffusion models.

###  4.3 Time Consumed in Preparing An Attack

Apart from simply using a uniform threshhold on the marginal distribution,
other white-box attack approaches including ours, involve learning machine
learning models. Efficient attack algorithms enable attackers to launch attack
more frequently on a machine learning system, and, on the bright side, it also
enables frequent privacy auditing for people who are maintaining these
systems.

Table 3: Time Consumed in Preparing An Attack on Tiny-ImageNet.

Preparation Steps |  | Computing Scores on Public Samples
---
using the Target Model
Learning Models
GSA (6 Shadow models) | 0 mins ( not required ) |  | 2 days (x 6 shadow models = 12 days)
---
Quantile (8 regression models ) | 8 mins |  | 4 mins (x 8 regression models = 32 mins )
---

LiRA attack on a target diffusion model trains shadow models, of which each is
a diffusion model of the same size. [19] proposed to extract gradient
information, and it introduces additional latency overhead on top of learning
shadow models. On contrary, our approach learns simple quantile regression
models. To demonstrate the efficiency of our algorithm in preparing the
attack, we estimated the clock time on a single V100 GPU card, and the rough
estimates are presented in Table 3.

###  4.4 Impact of Bagging

Since our algorithm doesn’t require knowledge of the training algorithm of the
target diffusion model, the quantile regression model technically can be an
arbitrary machine learning model. Therefore, with an inappropriately-chosen
model architecture for quantile regression, the attack performance at low FPR
may not be desirable. Our proposal to alleviate this issue to through bagging,
hence namely “bag of weak attackers”. We now present empirical ablation study
of the impact of bagging on various model sizes.

We select the base architecture for our quantile regression as a ResNet model,
however, we significantly reduce the number of channels in each layer to make
it much smaller in terms of the number of parameters and make it much more
efficient to train and conduct inference. A standard ResNet model with 18
layers for the classification task on CIFAR10 has around $4.4\times 10^{7}$
parameters, and in our experiments, the smallest ResNet for quantile
regression has only $5.6\times 10^{3}$ parameters. Our way of reducing the
number of parameters is through reducing the number of channels in each
convolution layer by a specific factor.

We consider three configuration of the number of channels in ResNet, which
then results in three base models with different number of parameters,
including $5.6\times 10^{3}$, $2.0\times 10^{4}$, and $8.0\times 10^{4}$. It
is worth noting that the number of parameters in a single diffusion model in
our experiments is $7.1\times 10^{8}$, and our attack models are significantly
smaller than the target model. We then vary the number of weak attackers from
1 to 7. Each configuration is run with 10 random seeds.

#### Improving Attack Performance.

Figure 2 presents results that indicate the effectiveness of our “bag of weak
attackers”. We can see that, for the smallest model, essentially the weakest
attacker, increasing the number of attackers improves the performance
drastically at low FPR, and for other two models, bagging improves in some
cases, but in general is not detrimental to the attack performance.

Figure 2: The Effect of Bagging. The bagging predictor takes the majority vote
over the decisions made by weak attackers, which in our case, are quantile
regressors. Due to the variance reduction aspect of bagging, we expect the
attack performance inceases as the number of weak attackers increases. We
present the effect of bagging at three different fixed FPR values, including
0.1%, 0.01%, and 0.001%, on four diffusion models, and we also present the
impact on three neural networks of different sizes. Overall, we can see that
the attack performance demonstrates a non-decreasing trend as the number of
weak attacker increases.

This is particularly interesting that the attacker first doesn’t need to know
the training algorithm of the diffusion model, which is required for LiRA
attacks, and even if the attacker chooses a small quantile regression model,
which might be a weak attacker, they can still obtain nearly perfect attack
performance through booststrap ensemble, namely, training a small number of
small quantile regression models.

#### Reducing Variance of Individual Predictors.

The performance improvement of bagging is mainly derived from variance
reduction by taking an ensemble over several predictors. In our case, we
directly take a majority vote over the decisions of individual weak attackers,
and Fig. 2 already shows that overall the attack performance improves as the
number of weak attackers increases. It would be interesting to show that the
performance improvement indeed comes from variance reduction.

We select the smallest model architecture in our experiments, which is the one
with only 5,666 parameters. For a fixed number of models in bagging, we
estimate the per-sample variance on the same holdout set by running our
algorithm several times, and we plot the Cumulative Distribution Function
(CDF) of the per-sample variance over the samples in the holdout set in Fig.
3. In general, as the number of models in bagging increases, the CDF shows a
steeper increase at the area where variance of predictors is small. Therefore,
more models in bagging lead to smaller variance.

Figure 3: Variance Reduction of Bagging. We here present CDF plots of
variances of bagging predictors with an increasing number of quantile
regression models on four datasets in our experiments. The x-axis indicates
the variance, and the y-axis indicates the cumulative count of samples used in
training the diffusion model. We can see that, as the number of models
increases in the bagging predictor, the CDF plot gradually becomes flatter,
which means that the predictions on more samples now have lower variance.
Thus, it shows that by, taking the majority vote of weak attackers, the
prediction variance decreases, which results in better performance.

##  5 Conclusion

We demonstrate the success of our attack based on quantile regression on
diffusion models. With side information, including auxiliary samples from the
data distribution, the results show the effectiveness of our membership
inference attack on four diffusion models trained on different datasets
respectively. Beside the performance, our attack is also computationally
efficient compared to prior approaches based on shadow models. Moreover, the
effectiveness and the efficiency of our algorithm indicates that diffusion
models are indeed extremely vulnerable to MI attacks, and extra care should be
taken when releasing a trained diffusion model. An exciting future direction
is to investigate whether we can extend our current approach to a black-box
attack setting, where we do not have direct access to the trained model
$\theta$.

## References

  * [1] M. A. Bertran, S. Tang, A. Roth, M. Kearns, J. Morgenstern, and S. Wu.  Scalable membership inference attacks via quantile regression.  pre-print, 2023.
  * [2] L. Breiman.  Bagging predictors.  Machine learning, 24:123–140, 1996.
  * [3] N. Carlini, S. Chien, M. Nasr, S. Song, A. Terzis, and F. Tramèr.  Membership inference attacks from first principles.  In 2022 IEEE Symposium on Security and Privacy (SP), pages 1897–1914, 2022.
  * [4] N. Carlini, J. Hayes, M. Nasr, M. Jagielski, V. Sehwag, F. Tramer, B. Balle, D. Ippolito, and E. Wallace.  Extracting training data from diffusion models.  In 32nd USENIX Security Symposium (USENIX Security 23), pages 5253–5270, 2023.
  * [5] N. Carlini, F. Tramer, E. Wallace, M. Jagielski, A. Herbert-Voss, K. Lee, A. Roberts, T. Brown, D. Song, U. Erlingsson, A. Oprea, and C. Raffel.  Extracting training data from large language models.  ArXiv, abs/2012.07805, 2020.
  * [6] A. Coates, A. Ng, and H. Lee.  An analysis of single-layer networks in unsupervised feature learning.  In Proceedings of the fourteenth international conference on artificial intelligence and statistics, pages 215–223. JMLR Workshop and Conference Proceedings, 2011.
  * [7] J. Duan, F. Kong, S. Wang, X. Shi, and K. Xu.  Are diffusion models vulnerable to membership inference attacks?  In A. Krause, E. Brunskill, K. Cho, B. Engelhardt, S. Sabato, and J. Scarlett, editors, Proceedings of the 40th International Conference on Machine Learning, volume 202 of Proceedings of Machine Learning Research, pages 8717–8730. PMLR, 23–29 Jul 2023.
  * [8] C. Dwork, F. McSherry, K. Nissim, and A. Smith.  Calibrating noise to sensitivity in private data analysis.  In Conference on Theory of Cryptography, TCC ’06, pages 265–284, New York, NY, USA, 2006.
  * [9] C. Dwork and A. Roth.  The algorithmic foundations of differential privacy.  Foundations and Trends in Theoretical Computer Science, 9(3-4):211–407, 2014.
  * [10] J. Hayes, L. Melis, G. Danezis, and E. De Cristofaro.  Logan: Membership inference attacks against generative models.  Proceedings on Privacy Enhancing Technologies, 2019:133–152, 01 2019.
  * [11] J. Ho, A. Jain, and P. Abbeel.  Denoising diffusion probabilistic models.  arXiv preprint arxiv:2006.11239, 2020.
  * [12] N. Homer, S. Szelinger, M. Redman, D. Duggan, W. Tembe, J. Muehling, J. V. Pearson, D. A. Stephan, S. F. Nelson, and D. W. Craig.  Resolving individuals contributing trace amounts of DNA to highly complex mixtures using high-density SNP genotyping microarrays.  PLoS genetics, 4(8):e1000167, 2008.
  * [13] M. Jagielski, J. Ullman, and A. Oprea.  Auditing differentially private machine learning: How private is private sgd?  In Advances in Neural Information Processing Systems, NeurIPS ’20, 2020.  https://arxiv.org/abs/2006.07709.
  * [14] B. Jayaraman and D. Evans.  Evaluating differentially private machine learning in practice.  In USENIX Security Symposium, 2019.
  * [15] B. Jayaraman, L. Wang, D. Evans, and Q. Gu.  Revisiting membership inference under realistic assumptions.  arXiv preprint arXiv:2005.10881, 2020.
  * [16] A. Krizhevsky.  Learning multiple layers of features from tiny images.  2009\.
  * [17] M. Nasr, J. Hayes, T. Steinke, B. Balle, F. Tramèr, M. Jagielski, N. Carlini, and A. Terzis.  Tight auditing of differentially private machine learning.  CoRR, abs/2302.07956, 2023.
  * [18] M. Nasr, S. Song, A. Thakurta, N. Papernot, and N. Carlini.  Adversary instantiation: Lower bounds for differentially private machine learning.  In IEEE Symposium on Security & Privacy, IEEE S&P ’21, 2021.  https://arxiv.org/abs/2101.04535.
  * [19] Y. Pang, T. Wang, X. Kang, M. Huai, and Y. Zhang.  White-box membership inference attacks against diffusion models.  arXiv preprint arXiv:2308.06405, 2023.
  * [20] R. Shokri, M. Stronati, C. Song, and V. Shmatikov.  Membership inference attacks against machine learning models.  In IEEE Symposium on Security and Privacy (S&P), Oakland, 2017\.
  * [21] Y. Song and S. Ermon.  Generative modeling by estimating gradients of the data distribution.  Advances in neural information processing systems, 32, 2019.
  * [22] T. Stadler, B. Oprisanu, and C. Troncoso.  Synthetic data - anonymisation groundhog day.  In K. R. B. Butler and K. Thomas, editors, 31st USENIX Security Symposium, USENIX Security 2022, Boston, MA, USA, August 10-12, 2022, pages 1451–1468. USENIX Association, 2022.
  * [23] F. Tramèr, A. Terzis, T. Steinke, S. Song, M. Jagielski, and N. Carlini.  Debugging differential privacy: A case study for privacy auditing.  CoRR, abs/2202.12219, 2022.
  * [24] B. van Breugel, H. Sun, Z. Qian, and M. van der Schaar.  Membership inference attacks against synthetic data through overfitting detection.  In F. J. R. Ruiz, J. G. Dy, and J. van de Meent, editors, International Conference on Artificial Intelligence and Statistics, 25-27 April 2023, Palau de Congressos, Valencia, Spain, volume 206 of Proceedings of Machine Learning Research, pages 3493–3514. PMLR, 2023.
  * [25] L. Wang, B. Jayaraman, D. Evans, and Q. Gu.  Efficient privacy-preserving nonconvex optimization.  arXiv preprint arXiv:1910.13659, 2019.
  * [26] S. Yeom, I. Giacomelli, M. Fredrikson, and S. Jha.  Privacy risk in machine learning: Analyzing the connection to overfitting.  In IEEE Computer Security Foundations Symposium, CSF ’18, pages 268–282, 2018.  https://arxiv.org/abs/1709.01604.