# Privacy Leakage on DNNs: A Survey of Model Inversion Attacks and Defenses

Author Name Affiliation<EMAIL_ADDRESS>Hao Fang1 Yixiang Qiu2 Hongyao Yu2
Wenbo Yu2 Jiawei Kong2 Baoli Chong2
Bin Chen2 Corresponding Author Xuan Wang2 Shu-Tao Xia1 1Tsinghua Shenzhen
International Graduate School, Tsinghua University
2School of Computer Science and Technology, Harbin Institute of Technology,
Shenzhen
<EMAIL_ADDRESS>{qiuyixiang, yuhongyao, yuwenbo, kongjiawei,
<EMAIL_ADDRESS><EMAIL_ADDRESS><EMAIL_ADDRESS><EMAIL_ADDRESS>
###### Abstract

Model Inversion (MI) attacks aim to disclose private information about the
training data by abusing access to the pre-trained models. These attacks
enable adversaries to reconstruct high-fidelity data that closely aligns with
the private training data, which has raised significant privacy concerns.
Despite the rapid advances in the field, we lack a comprehensive overview of
existing MI attacks and defenses. To fill this gap, this paper thoroughly
investigates this field and presents a holistic survey. Firstly, our work
briefly reviews the traditional MI on machine learning scenarios. We then
elaborately analyze and compare numerous recent attacks and defenses on Deep
Neural Networks (DNNs) across multiple modalities and learning tasks. Finally,
this paper discusses promising research directions and presents potential
solutions to open issues. To facilitate further study on MI, we have
implemented an available open-source toolbox on
GitHub111https://github.com/ffhibnese/Model-Inversion-Attack-ToolBox.

##  1 Introduction

With the unprecedented development of Deep Learning (DL), Deep Neural Networks
(DNNs) have been widely used in a variety of fields, including medical
research, financial analysis, and personalized recommendations. While many
practical applications of DNNs require training on privacy-sensitive datasets,
people may mistakenly expect that the private data is securely encoded in the
weights of the trained model. However, numerous studies have demonstrated that
a malicious attacker could utilize the pre-trained models to reveal critical
information about the training data. Among the different types of attacks,
Model Inversion (MI) stands out as a potent threat that has raised great
privacy concerns.

Fredrikson et al. (2014) first introduce MI attacks in the context of genomic
privacy and reveal that genomic markers could be recovered by maximizing
posterior probabilities for a given linear regression model. Subsequent
studies Fredrikson et al. (2015); Song et al. (2017); Yang et al. (2019)
further extend MI attacks to more Machine Learning (ML) models and more forms
of data with other optimizing algorithms such as gradient descent. Although
these aforementioned methods have exhibited outstanding performance in
traditional ML scenarios for shallow networks and simple data, the
effectiveness is significantly reduced when attacking deeper and wider DNNs
trained on more complex and high-dimensional data such as RGB images.

To address this challenge, recent studies have made great efforts and achieved
remarkable improvements in attacking image data. Generative Adversarial
Networks (GANs) Goodfellow (2016), first introduced to model inversion by
Zhang et al. (2020), are exploited as image priors by a majority of these
methods for better reconstruction results. Specifically, Zhang et al. (2020)
propose to train a GAN model with publicly available data to generate high-
fidelity images for target classes. Benefiting from the prior information
encoded in the trained GAN model, the reconstruction performance is
significantly improved. Furthermore, the studies of MI are extended to Natural
Language Processing (NLP) tasks Carlini et al. (2019), especially for the
privacy leakage risk of Large Language Models (LLMs), e.g., ChatGPT. Users can
generate text-based queries and interact with ChatGPT, and concerns arise
regarding the inadvertent exposure of sensitive information through the
model’s responses Nasr et al. (2023). Meanwhile, graph data used to train
Graph Neural Networks (GNNs) is also vulnerable to MI threats Zhang et al.
(2021). To protect training data from powerful MI attacks, a series of defense
strategies have been proposed to enhance the security of the trained models.
Most of these methods defend against MI attacks by adding perturbations to the
output predictions Yang et al. (2020), while others prefer to incorporate
well-designed mechanisms into the training process of the target model Wang et
al. (2021b).

Despite the privacy concerns in training data from different modalities, a
comprehensive overview summarizing their advances is currently not available.
Recent research Dibbo (2023) predominantly focus on the diverse taxonomies and
challenges of MI attacks for tabular and image data, or Jegorova et al. (2022)
briefly introduce several representative MI attacks in a subsection among
various types of privacy attacks in the inference stage. In contrast, we
present a systematic and elaborate review of the advanced MI attacks and
defenses on DNNs over multiple modalities and different learning tasks,
providing a thorough exploration of the landscape.

Figure 1: Threat model of GAN-based MI for image data.

##  2 Model Inversion Overview

###  2.1 Basic Concepts of Model Inversion

Model Inversion Fredrikson et al. (2014); Wang et al. (2021a) is a type of
attack that aims at inverting a given pre-trained model $f_{\theta}$ to
recover private training dataset $X$, such as images, texts, and graphs.
Different from the membership inference attack or the property inference
attack that only reveal partial information about the training data Jegorova
et al. (2022), MI enables an adversary to fully reconstruct private training
samples, which has raised growing concern.

###  2.2 MI across Different Modalities

To better understand the distinction between different modalities, we refine
the overall profile based on existing MI studies of the three modalities.
Next, we detail the MI attacks and defenses in the order of image (see Section
3), text (see Section 4), and graph (see Section 5).

##### Learning Tasks.

The victim models in computer vision and graph learning are primarily used for
classification tasks. On the other hand, most attacks on text data target the
language models of generative tasks. The varying learning tasks of victim
models further lead to different attack strategies.

##### Taxonomy of Attacks.

To better analyze and compare different reconstruction approaches, we
generally categorize these methods of three modalities into _white-box_ and
_black-box_ settings based on the attacker’s capability and knowledge.
Specifically, a white-box scenario implies that attackers have full access to
the weights and outputs of the target model. Conversely, in black-box
settings, only the predicted confidence probabilities or hard labels are
available.

Considering the distinctions of different modalities, we further provide a
more detailed summary or taxonomy of MI attacks for each modality. In visual
tasks, the MI attack is formulated as an optimization problem. We then
disassemble the optimization process and summarize the major characteristics
of representative attacks in Table 1. Due to the nature of text generative
models, various forms of attacks appear in the NLP realm. We characterize them
into three paradigms based on the key strategies to acquire reconstructed
sentences: Embedding Optimization, Token search, and Malicious Prompt Design.
For graph learning, researchers investigate the discrete structures of graph
data and propose powerful inversion techniques, which can be categorized as:
_Adjacency Optimization_ , _Inverse Mapping_ , and _Relationship Reasoning_.

##### Taxonomy of Defenses.

As MI attacks mainly leverage the rich information within the model output or
the correlation between the input and output of the victim model, the defenses
across different modalities primarily focus on improving security from these
two perspectives, and thus can be uniformly divided into two types: Model
Output Processing and Robust Model Training.

##  3 MI on Computer Vision

This section provides detailed introductions to MI attacks and defenses on
DNNs in the computer vision domain.

###  3.1 Private Image Recovery

We begin by elucidating the general threat model of MI on DNNs. To better
clarify the pivotal techniques of these methods, we decompose the attack
paradigm and analyze them across four perspectives: Generative Models, Data
Initialization, Attack Process and Additional Generic Techniques.

Due to the remarkable improvement facilitated by the generative priors,
generative models have been incorporated into the basic paradigm for
subsequent research of MI on DNNs Zhang et al. (2020); Wang et al. (2021a);
Yuan et al. (2023). Figure 1 outlines the threat model for a typical GAN-based
model inversion attack. Given the target image classifier $f_{\theta}$
parameterized with weights $\theta$ and the trained generator $G$, attackers
attempt to recover private images $\mathbf{x}^{*}$ from latent vectors
$\mathbf{z}$ initialized by Gaussian distribution:

$\displaystyle\mathbf{z}^{*}=\mathop{\arg\min}\limits_{\mathbf{z}}\mathcal{L}_{iden}(f_{\theta}(G(\mathbf{z})),c)+\lambda\mathcal{L}_{prior}(\mathbf{z};G),$
(1)

where $c$ is the target class, $\mathcal{L}_{iden}(\cdot,\cdot)$ denotes the
classification loss, $\lambda$ is a hyperparameter, and
$\mathcal{L}_{prior}(\cdot)$ is the prior knowledge regularization, e.g., the
discriminator loss. The first term measures the deviation from the private
data and the latter term serves as image priors to enhance the quality of the
reconstructed images. Before launching an attack, the adversary trains a
specialized generator or utilizes a pre-trained generator as prior information
about the target images. Then, they employ the generator to generate dummy
images and feed them into the target classifier. During the recovery process,
the attacker iteratively updates the latent vectors $\mathbf{z}$ by minimizing
the loss function in Eq. (1). Once obtain the optimal $\mathbf{z}^{*}$, the
reconstructed images that closely align with the private images can be
generated by $\mathbf{x}^{*}=G(\mathbf{z}^{*})$.

Method | Generative Models | Data Initialization | Optimization | Extra Features
---|---|---|---|---
Resolution | Preprocess | Optim-algorithm | Loss-fn | Search Space
White-Box Attacks
GMI Zhang et al. (2020) |  WGAN-GP | $64\times 64$ | ✗ | Momentum SGD | $\mathcal{L}_{CE}+\lambda\mathcal{L}_{Dis}$ |  $\mathcal{Z}$ space | -
KEDMI Chen et al. (2021) | Inversion-specific GAN† | $64\times 64$ | ✗ | Adam | $\mathcal{L}_{CE}+\lambda\mathcal{L}_{Dis}$ |  $\mathcal{Z}^{*}$ space |  Distributional recovery
VMI Wang et al. (2021a) |  DCGAN StyleGAN2-Ada |  $64\times 64$ $128\times 128$ | ✗ | Adam | $\mathcal{L}_{CE}+\lambda\mathcal{L}_{KL}$ |  $\mathcal{Z}$ space | Variational inference
SecretGen Yuan et al. (2022) | WGAN-GP | $64\times 64$ | ✔ | Momentum SGD | $\mathcal{L}_{CE}+\lambda\mathcal{L}_{Dis}$ |  $\mathcal{Z}$ space | -
Mirror An et al. (2022) | StyleGAN | $224\times 224$ | ✔ | Adam | $\mathcal{L}_{CE}$ |  $\mathcal{P}$ space |  $\mathbf{p}$ vector Clipping
PPA Struppek et al. (2022) | StyleGAN2-Ada | $224\times 224$ | ✔ | Adam | $\mathcal{L}_{Poincar\acute{e}}$ |  $\mathcal{W}$ space | Results selection
PLGMI Yuan et al. (2023) | Conditional GAN | $64\times 64$ | ✗ | Adam | $\mathcal{L}_{Max-margin}$ |  $\mathcal{Z}$ space |  Pseudo labels guidance
LOMMA Nguyen et al. (2023b) | $\sim$ | $\sim$ | $\sim$ | $\sim$ | $\mathcal{L}_{NLL}+\lambda\mathcal{L}_{Feature}$ | $\sim$ | Model augmentation
DMMIA Qi et al. (2023) | StyleGAN2-Ada | $224\times 224$ | ✔ | Adam | $\mathcal{L}_{CE}+\mathcal{L}_{Memory}$ | - |  Dynamic memory bank
Black-Box Attacks
SecretGen Yuan et al. (2022) | WGAN-GP | $64\times 64$ | ✔ | Momentum SGD | $\mathcal{L}_{Dis}$ |  $\mathcal{Z}$ space | -
BREPMI Kahla et al. (2022) | Inversion-specific GAN† | $64\times 64$ | ✔ | Gradient estimation | - |  $\mathcal{Z}$ space | -
Mirror An et al. (2022) | StyleGAN | $224\times 224$ | ✔ | Genetic algorithm | - |  $\mathcal{P}$ space |  $\mathbf{p}$ vector Clipping
C2FMI Ye et al. (2023) | StyleGAN2 | $128\times 128$ | ✔ | Genetic algorithm | - |  $\mathcal{W}$ space | Initial coarse optimization
RLBMI Han et al. (2023) | WGAN-GP | $64\times 64$ | ✗ | SAC | - |  $\mathcal{Z}$ space | Reinforcement learning
LOKT Nguyen et al. (2023a) | ACGAN | $128\times 128$ | ✗ | Adam | $\mathcal{L}_{Max-margin}$ |  $\mathcal{Z}$ space | Invert surrogate models
DiffusionMI Liu (2023) | Conditional Diffusion | $64\times 64$ | ✔ |  Optimization-free | - | - | Results selection

  * •

∗The latent vectors are sampled from a learnable Gaussian Distribution.

  * •

†This customized GAN structure is first proposed by KEDMI Chen et al. (2021).

  * •

∼LOMMA Nguyen et al. (2023a) is a plug-and-play technique that can perfectly
combine with existing model inversion attacks.

Table 1: Summary of existing MI attacks for visual tasks. Note that SecretGen
and Mirror consider both white-box and black-box scenarios.

####  3.1.1 Generative Models

Early studies directly searched image pixels to perform MI attacks, which were
limited to shallow networks and gray images. By leveraging powerful generative
models, subsequent studies successfully solved this issue and achieved
outstanding improvements on DNNs with RGB images. Specifically, Zhang et al.
(2020); Chen et al. (2021); Nguyen et al. (2023a) train a GAN model on a large
public dataset that has similar image structures to the private data. They
utilize different GAN models tailored to the MI scenario and design special
loss functions for generative adversarial training. Moreover, these methods
further strengthen attack effectiveness by leveraging the side information
from target models, e.g., employing the features extracted by the target model
to provide auxiliary knowledge to the generator Nguyen et al. (2023a).

However, constrained by the limited volume and poor quality of auxiliary
datasets for GAN training, the trained generator is only capable of
reconstructing low-resolution images at $64\times 64$. Additionally,
performing adversarial training incurs expensive computational overheads and
consumes a significant amount of time. One viable resolution is to utilize
pre-trained GANs learning from large public datasets An et al. (2022);
Struppek et al. (2022). Compared with GANs trained manually with low-quality
auxiliary datasets, these pre-trained generators typically have more intricate
structures and hold richer semantic information, enabling higher resolution
image reconstruction up to $224\times 224$. Liu (2023) substitute GANs with
the conditional diffusion model Ho and Salimans (2022). Specifically, they
leverage pseudo-labels classified by the target model as conditions to guide
the denoising process. This approach demonstrates notable improvements over
former GAN-based methods, particularly in terms of human perception distance
(e.g., LPIPS Zhang et al. (2018)).

####  3.1.2 Data Initialization

##### Resolution.

We summarize the image resolutions employed by different methods in Table 1.
Given that more image pixels imply more parameters to optimize, the difficulty
of MI attacks is highly determined by the resolution of recovered images.
Early MI methods resort to reconstructing low-resolution images, e.g., GMI
Zhang et al. (2020) and KEDMI Chen et al. (2021). To enable attacks on higher-
resolution images, subsequent studies introduce stronger GAN models (e.g.,
StyleGAN Karras et al. (2019)) and manage to recover images with a resolution
of $224\times 224$.

##### Preprocess.

While Zhang et al. (2020); Chen et al. (2021) sample randomly from certain
distributions, Yuan et al. (2022); Struppek et al. (2022) highlight that a
batch of appropriately initialized latent vectors tends to yield better
reconstruction results. Concretely, they opt for latent vectors whose
corresponding images can be correctly classified as the target labels by the
victim model. To further enhance the preprocessing, Ye et al. (2023) conduct
optimization to latents by minimizing the $L_{2}$ distance between the
features extracted from the dummy images and inverted from the given label.

####  3.1.3 Attack Process

##### White-Box Attacks.

As summarized in Table 1, white-box attacks generally apply Momentum SGD or
Adam optimizer to conduct the inversion. Most of them search the latent code
using the gradients computed from the Cross-Entropy (CE) loss. Considering
that CE loss suffers from the gradient vanishing problem, researchers propose
to employ Poincaré loss Struppek et al. (2022) or max-margin loss Yuan et al.
(2023) to mitigate this issue. Besides, Nguyen et al. (2023b) rethink the
optimization objective and emphasize that the fundamental goal of MI is to
reconstruct images that closely align with the target class, rather than
deviating from non-target classes. Since CE loss inherently combines both
objectives, they suggest bypassing the softmax function in the cross-entropy
loss and directly maximizing the confidence of the target class, i.e., the
Negative Log-Likelihood (NLL) loss.

As mentioned in Section 1, various regularizers are introduced as prior
knowledge to ensure the fidelity of the generated images. One representative
attempt is the use of the discrimination loss term $\mathcal{L}_{Dis}$.
However, Struppek et al. (2022) shows that $\mathcal{L}_{Dis}$ might degrade
the inversion performance by matching the generated images with the
distributions of the public data rather than the private ones. To avoid this
distribution shift, subsequent MI methods no longer use this loss term. Wang
et al. (2021a) formulate the MI attack as a variational inference process and
introduce KL-divergence as a regularization to better approximate the target
data distribution with a variational distribution $q(\mathbf{x})$. LOMMA
Nguyen et al. (2023b) fully use the penultimate layer representations in the
target model by adding an $\mathcal{L}_{Feature}$ term as a distance measure
between the features of reconstructed samples and public images. Based on a
dynamic memory bank, Qi et al. (2023) increase the diversity of inverted
images by designing a novel regularizer
$\mathcal{L}_{Memory}=\lambda_{1}\mathcal{L}_{imr}+\lambda_{2}\mathcal{L}_{idr}$,
where $\mathcal{L}_{imr}$ and $\mathcal{L}_{idr}$ denote the intra-class
multicentric representation (IMR) and the inter-class discriminative
representation (IDR) loss respectively. IMR provides multiple distinctive
features of intra-class images to prevent overfitting to specific image
features. IDR stores the historical knowledge for each class, encouraging the
reconstruction of images with more distinguishable characteristics.

Unlike these methods that focus on instance-level reconstruction, Chen et al.
(2021) aim to recover the private data distribution for a given label with a
learnable Gaussian distribution $\mathcal{N}(\mu,\sigma^{2})$. They adopt the
reparameterization trick to make the loss differentiable and further optimize
$\mu$ and $\sigma$. Attackers can then directly sample $z$ from the learned
distribution to generate sensitive images.

##### Black-Box Attacks.

In contrast to white-box settings, these attacks cannot access any information
within the target model. Consequently, the gradients can no longer be computed
using the back-propagation operation. Yuan et al. (2022) propose a
straightforward solution to this problem. They first sample numerous latent
vectors from random noise and select the ones that can produce images
predicted as the correct labels. The chosen vectors are then optimized
exclusively based on the discriminator loss. Han et al. (2023) introduce a
reinforcement learning-based MI attack, which leverages the confidence scores
to provide rewards and optimize the agent with the soft actor-critic (SAC)
algorithm.

Label-only scenario. BREPMI Kahla et al. (2022) employs gradient estimation to
tackle situations where only hard labels are available. By randomly sampling
$N$ unit vectors $\mathbf{u}$, the loss is calculated with sampled points on a
sphere at radius $r$ to estimate the gradients $\hat{g}_{\mathbf{z}}$ of
latent vectors $\mathbf{z}$:

$\hat{g}_{\mathbf{z}}=\frac{1}{N}\sum_{i=1}^{N}\Phi_{c}(\mathbf{z}+r\mathbf{u_{i}})\mathbf{u_{i}},$
(2)

where $\Phi_{c}(\cdot)$ denotes a function that equals zero if the generated
image is classified as the target class $c$, otherwise $-1$. During the
optimization, BREPMI gradually increases the value of $r$ to accurately
estimate the gradients. Nguyen et al. (2023a) transform this problem into a
white-box scenario by inverting multiple surrogate models instead of the
black-box victim model. This entails generating a substantial number of dummy
samples using an ACGAN trained with guidance from the victim model. Next,
multiple classifiers are trained on these dummy samples to serve as surrogates
for the victim model. Liu (2023) use an auxiliary dataset labeled with pseudo
labels to train a class-conditional diffusion model. The trained diffusion
model is later directly used to generate target images without any further
optimization.

Gradient-free optimizer. Orthogonal to the above methods, An et al. (2022); Ye
et al. (2023) explore gradient-free optimization techniques and perform
genetic algorithms that utilize the distance and direction information from
the current population to find the optimal latent vectors.

##### Search Space.

Most previous studies focus on finding the optimal vectors in the GAN’s latent
space $\mathcal{Z}$. However, An et al. (2022) show that as the resolution
increases, this optimization becomes under-constrained due to the large space
sparsity. Feature entanglement in $\mathcal{Z}$ space is another challenge
that further impedes the optimization process since images sharing similar
features may not correspond to analogous latent vectors. StyleGANs Karras et
al. (2019) transforms the latent space into the well-disentangled
$\mathcal{W}$ space through a mapping network
$G_{mapping}:\mathcal{Z}\rightarrow\mathcal{W}$. Therefore, searching the
$\mathcal{W}$ space alleviates these issues and has been adopted by a series
of following studies Struppek et al. (2022); Ye et al. (2023). Taking into
account the difficulty of modeling $\mathcal{W}$ distribution, An et al.
(2022) propose the concept of $\mathcal{P}$ space, the feature space located
before the final LeakyReLU function in the mapping network, to constraint the
final $\mathbf{w}$ vectors within the target distribution. Numerous vectors
are first sampled from $\mathcal{Z}$-space to compute the mean $\mu$ and
variance $\sigma$ of the activation values in $\mathcal{P}$-space. Given an
optimized $\mathbf{w}$ in $\mathcal{W}$-space, attackers first obtain
$\mathbf{p}$ by projecting $\mathbf{w}$ to $\mathcal{P}$-space. Then
$\mathbf{p}$ is clipped within the range $[\mu-\sigma,\mu+\sigma]$ and
projected back to the $\mathcal{W}$-space. This operation preserves the style
control capabilities of the $\mathcal{W}$ space while ensuring these controls
are within the desired distribution.

####  3.1.4 Additional Generic Techniques

Various innovative techniques have been explored and incorporated into
numerous studies. Next, we supplement a detailed review of several generic
mechanisms.

##### Pseudo Label Guidance.

Pseudo labels have been utilized to guide the training of both the generator
Yuan et al. (2023); Liu (2023) and the discriminator Chen et al. (2021).
Specifically, the target model is leveraged to reclassify public images with
labels from the private dataset. Since the target model is trained on the
private dataset, the use of pseudo labels exploits extra information within
the private training data, thus enhancing the capabilities of the generator or
discriminator trained on these auxiliary labeled images.

##### Augmentation.

Many studies have integrated various augmentation techniques into the MI
workflow to improve the attack effects. Yuan et al. (2022) employ sequential
cutout as a data augmentation for images to improve the initial latent vector
selection. Struppek et al. (2022); Liu (2023) process the reconstructed
results with image transformation and select results based on higher
confidence from the output of the target model. Furthermore, Yuan et al.
(2023) perform random augmentations on the generated images before feeding
them to the target model to provide more stable convergence to realistic
images during the GAN training. In addition to traditional data augmentations,
Nguyen et al. (2023b) present a novel approach called model augmentation. This
involves training several auxiliary models from the target model using model
distillation techniques. During the MI process, the adversary utilizes an
ensemble of the original target model and the trained auxiliary models to
calculate the loss function. By augmenting the surrogate models, this strategy
mitigates overfitting to the target model and encourages the recovered images
to capture more distinctive features of the private data.

##### Results Selection.

Struppek et al. (2022) note that DNNs often exhibit overconfidence in their
predictions, thus leading to attack results of low transferability. More
concretely, while some reconstructed images are labeled with high confidence
by the target model, they receive low scores when evaluated by another model.
To overcome this challenge, Struppek et al. (2022); Liu (2023) apply data
augmentations to the generated images before classifying them with the target
classifier. By selecting results with the highest confidence score after
augmentation, this approach achieves higher attack accuracy and better
transferability compared to the original ones.

###  3.2 Defending Image Data from MI Attacks

##### Model Output Processing.

Since MI attacks exploit the redundant information in the target model’s
output, a viable defense strategy is to reduce this redundancy by obscuring
the output confidence scores. Yang et al. (2020) train an autoencoder as a
purifier to process the confidence vector by reducing its dispersion. To
further enhance the autoencoder’s ability, they employ an adversarial learning
strategy that optimizes the purifier while jointly training additional
inversion models as adversaries. Wen et al. (2021) generate adversarial noise
to the confidence scores by maximizing the MI attack error, further leading to
confusing reconstructed images. They also preserve the main task accuracy via
a well-designed label modifier. Ye et al. (2022) design a Differential
Privacy-based approach, which divides the score vector into multiple sub-
ranges and applies an exponential mechanism to replace and normalize the
values in each sub-range. This mechanism is theoretically proven to ensure
differential privacy and improve the MI robustness of the target model.

##### Robust Model Training.

MID Wang et al. (2021b) emphasizes that MI attacks leverage the correlation
between the input $X$ and output $\hat{Y}$ of the target model $f_{\theta}$,
thereby proposing to penalize the mutual information between $X$ and $Y$
during training. The training objective can be expressed as:

$\min_{\theta}\mathbb{E}_{({x},y)\sim
p_{X,Y}({x},y)}\left[\mathcal{L}(f_{\theta}({x}),y)\right]+\lambda\mathcal{I}(X,\hat{Y}),$
(3)

where $y$ is the ground truth label, $\mathcal{L}(\cdot,\cdot)$ denotes the
loss for the main task, $\mathcal{I}(\cdot,\cdot)$ is the approximated mutual
information, and $\lambda$ is a coefficient balancing privacy and utility.

Nonetheless, MID meets the dilemma between privacy and model performance as
the second term conflicts with the main task loss. Peng et al. (2022) propose
a bilateral dependency optimization (BiDO) to solve this problem. Instead of
directly diminishing the correlation between the inputs and outputs, BiDO
minimizes the dependency between inputs and latent representations while
maximizing the dependency between latent representations and outputs. Gong et
al. (2023) mislead attackers from another perspective. They train a GAN and a
public classifier using a public dataset. Subsequently, the adversary performs
MI attacks on both the victim model and the public classifier to generate
private and public fake samples. The attacker then fine-tunes the victim model
with the generated fake samples, which minimizes the classification loss of
public samples while maximizing that of private ones, thereby deceiving the
attacker to invert samples that are significantly different from the private
images.

Titcombe et al. (2021) defend MI in split learning scenarios by adding
Laplacian noise to the intermediate representations before transmitting them
to the computing server. Along this line, Li et al. (2022) consider split
federated learning (SFL) and introduce a two-step framework that involves pre-
training a robust model $C^{*}$ against an adversarial inversion model and
then transferring $C^{*}$ to clients as initial models. Moreover, an attacker-
aware fine-tuning technique is employed during SFL training to enhance MI
resistance.

##  4 MI on Natural Language Processing

DNN models that process natural-language text also suffer from MI attacks. Due
to the unintended memorization of language models Carlini et al. (2019), an
adversary can invert the victim models to incur serious privacy leakage. This
has been amplified by the thrives of LLMs as these models are trained on
massive text corpora that often contain considerable privacy-sensitive
information.

###  4.1 Private Text Recovery

####  4.1.1 White-Box Attacks

These attacks take advantage of the full access to the victim model by
formulating the reconstruction as an optimization process that utilizes the
back-propagated gradients.

##### Embedding Optimization.

Parikh et al. (2022) attack the LSTM model $f_{\theta}$ by conducting a
discrete optimization, where each token $x_{i}$ of the sentence $s$ is
represented as a logit vector $z_{i}$ to get the embedding $E_{s}$ of the
dummy sentence $s$. Given the target label $y$, the vector
$z_{1},\ldots,z_{n}$ is repeatedly optimized with the gradients computed from
the cross-entropy loss $\mathcal{L}(f_{\theta}(E_{s}),y)$. Zhang et al.
(2023b) adopt the paradigm of prompt learning that freezes the parameter of
the victim GPT-Neo and tunes the pre-sentence soft prompt embedding $S$ by
forcing the model to output the target suffix $b$ for a given prefix $a$. The
learned prompt $S$ elicits the memorization of the target language model and
induces it to respond with private training data. Additionally, they introduce
a smoothing regularizer to make the loss distribution of the suffix sequence
smoother.

####  4.1.2 Black-Box Attacks

Since gradient back-propagation becomes impractical under black-box scenarios,
researchers have developed novel algorithms, namely the Token Search and
Malicious Prompt Design, to extract private sentences.

##### Token Search.

Given the sentence’s prefix tokens $a=x_{1},x_{2},\ldots,x_{i}$, attackers
attempt to search the most probable pathway for generating the remaining
portion of the sentence, i.e., $b=x_{i+1},x_{i+2},\ldots,x_{n}$. Based on the
search objective for the subsequent tokens, these methods are further divided
into three parts as follows.

Perplexity-metric Search. Carlini et al. (2019) use perplexity to measures the
probability of data sequences. Specifically, given a sequence $s=x_{1}\ldots
x_{n}$ and the target generative sequence model $f_{\theta}$ (i.e., LSTM,
qRNN), the perplexity $\mathrm{P}_{\theta}(s)$ can be expressed as:

$\mathrm{P}_{\theta}(s)=\exp\left(-\frac{1}{n}\sum_{i=1}^{n}\log
f_{\theta}\left(x_{i}|x_{1},\ldots,x_{i-1}\right)\right),$ (4)

where $\mathrm{P}_{\theta}(s)$ quantifies the “surprise” of a given sequence,
and a lower perplexity value indicates a higher probability that the language
model has seen the sequence. In this manner, the problem is transformed to
find the sequence with minimal perplexity. However, directly enumerating all
possible sequences and computing their perplexity leads to exponentially
growing search space. A series of mechanisms have been proposed to improve
search efficiency. Carlini et al. (2019) uses a modification of the Dijkstra
algorithm which efficiently reduces the search range by avoiding visiting
unnecessary sequence paths. The subsequent method Carlini et al. (2021) adopts
a greedy-based algorithm that only retains tokens with top-$k$ confidence
during each iteration of generating the next token. Moreover, they provide
several techniques for improved generated sentence diversity and enhanced
attack accuracy. Yu et al. (2023) adopt a look-ahead mechanism to improve the
quality of generated tokens. Instead of only depending on the next token’s
probability for the top-$k$ selection, they use the posterior probability of
the subsequent sequence to inform the generation of the next token $x_{t}$.

Confidence-metric Search. Considering the sentiment classification task,
Elmahdy et al. (2022) search for the next token that maximizes the confidence
score of target class $y$ as predicted by a fine-tuned BERT classifier. To
counteract the model’s bias towards high-frequency tokens, a regularizer is
introduced to penalize the frequent occurrence of certain tokens. However,
this method also faces the challenge of exponentially growing search space. To
tackle this, Elmahdy and Salem (2023) first use BERT with the original
generation head to generate candidate tokens with top-$k$ confidence, which
are further selected by maximizing the probability of label $y$ with the
classification head.

Difference-metric Search. Recently the pre-train and fine-tune paradigm has
gained growing popularity. Accordingly, several attacks appear to invert the
fine-tuning dataset by exploiting the difference between pre-trained model $M$
and fine-tuned model $M^{\prime}$. Considering the token sequence
$s={x_{1}\ldots x_{n}}$, Zanella-Béguelin et al. (2020) define the difference
score
$DS_{M}^{M^{\prime}}=\sum_{i=1}^{n}P_{M^{\prime}}(x_{i}|x_{<i})-P_{M}(x_{i}|x_{<i})$,
which measures the difference in the output probabilities between $M$ and
$M^{\prime}$. Intuitively, a larger difference score value indicates a higher
probability that the fine-tuned model $M^{\prime}$ has seen sequence $s$ while
the original model $M$ does not, i.e., $s$ belongs to the fine-tuning dataset.
By performing a beam search, they select next tokens with top-$k$ highest
differential scores and achieve a remarkable attack success rate.

##### Malicious Prompt Design.

LLMs have demonstrated powerful capabilities and great potential. However,
several studies have shown that some specially designed prompts can invert the
model to output sensitive training data. As Figure 2 depicts, feeding
malicious prompts into language models such as GPT-Neo can incur serious
privacy information leakage Huang et al. (2022). More surprisingly, Nasr et
al. (2023) suggest that data can also be extracted by simply asking ChatGPT to
repeat a word multiple times. One possible reason is that it causes the model
to “escape” its aligning training and revert to its original language modeling
objective.

Figure 2: An example of privacy leakage on LLMs.

###  4.2 Defending Text Data from MI Attacks

##### Model Output Processing.

Huang et al. (2022) propose that restricting the model output is an effective
defense approach. They suggest filtering the privacy-sensitive information
within the prediction, e.g., including a detection module to examine whether
the output text contains sensitive information and take appropriate measures
such as refusing to answer or masking the information for privacy guarantees.

##### Robust Model Training.

Anil et al. (2021) reveal that language models trained with variants of the
differential privacy-based optimizer exhibit improved MI resistance while at
the expense of performance degradation and increased computation. Kandpal et
al. (2022) find that training data de-duplication efficiently reduces the
memorization of the victim model, thereby improving MI robustness. Besides,
the memorization of language models essentially relates to the overfitting.
Therefore, regularization and early stopping are also alternative defense
strategies Ishihara (2023).

##  5 MI on Graph Learning

MI attacks on Graph Learning leverage Graph Neural Networks (GNNs) to
reconstruct the topology of the private graph data. As shown in Figure 3,
given the target GNN $f_{\theta}$ pre-trained on the private graph
$\mathcal{G}$, the adversary aims to infer the adjacency matrix
$\hat{\mathcal{A}}$, which is then converted into graph $\hat{\mathcal{G}}$ to
reconstruct the private training dataset.

Figure 3: Different types of MI attacks on graph data.

###  5.1 Private Graph Recovery

####  5.1.1 White-Box Attacks

Similarly, white-box attacks for graph data compute the gradients to optimize
the adjacency for privacy extraction.

##### Adjacency Optimization.

Denote the prediction vector or the embeddings output by the target GNN model
as $\mathcal{H}$, i.e., $\mathcal{H}=f_{\theta}(\mathcal{G})$. An adjacency
matrix $\hat{\mathcal{A}}_{0}$ is initialized to obtain the dummy graph
$\hat{\mathcal{G}}_{0}$ and the dummy output
$\hat{\mathcal{H}}_{0}=f_{\theta}(\hat{\mathcal{G}}_{0})$. By minimizing the
distance $\mathcal{L}_{rec}$ between $\hat{\mathcal{H}}_{0}$ and $\mathcal{H}$
with gradient descent, an attacker iteratively updates $\hat{\mathcal{A}}$ to
find the optimal solution. Based on this, Zhang et al. (2021) propose GraphMI
that uses a projected gradient module to tackle the discreteness of graph data
while introducing the feature smoothness term and the F-norm term to preserve
feature smoothness and sparsity. Zhou et al. (2023) propose to regard
$f_{\theta}$ as a Markov chain and optimize $\hat{\mathcal{A}}$ via a flexible
chain approximation, achieving excellent reconstructed results.

####  5.1.2 Black-Box Attacks

As shown in Figure 3, we categorize black-box attacks into two types: Inverse
Mapping and Relationship Reasoning.

##### Inverse Mapping.

Duddu et al. (2020) attempt to invert the original mapping $f_{\theta}$ and
construct an inverse mapping. They train an autoencoder on the auxiliary graph
dataset $\mathcal{G}_{aux}$ and utilize the decoder $\mathcal{F}_{dec}$ to map
$\mathcal{H}$ back to $\hat{\mathcal{G}}$, i.e.,
$\hat{\mathcal{G}}=\mathcal{F}_{dec}(\mathcal{H})$. However, the distribution
of intermediate features $\mathcal{H}_{int}$ within the autoencoder may differ
from the distribution of $\mathcal{H}$. Therefore, Zhang et al. (2022) further
query $f_{\theta}$ with $\mathcal{G}_{aux}$ to acquire $\mathcal{H}_{aux}$ and
fine-tune $\mathcal{F}_{dec}$ with $(\mathcal{H}_{aux},\mathcal{G}_{aux})$ for
enhanced reconstruction performance.

##### Relationship Reasoning.

These attacks reconstruct $\hat{\mathcal{G}}$ by reasoning about the relations
among pairs of nodes using the node attributes as auxiliary knowledge. He et
al. (2021) assume that two nodes $u$ and $v$ are linked if they share more
similar attributes or predictions. Furthermore, Wu et al. (2022) suggest that
if there is an edge between $u$ and $v$, the information of $u$ would be
propagated to $v$ during training. Thus, they presume $u$ and $v$ are linked
if changing the feature vector of $u$ can impact the prediction of $v$ to a
certain extent.

###  5.2 Defending Graph Data from MI Attacks

##### Model Output Processing.

He et al. (2021) restrict the GNN model to only output the $k$ largest
posterior probabilities to provide less information. Zhang et al. (2022)
propose adding Laplace noise to the model’s output $\mathcal{H}$ to mislead
the attacker. Since applying noise to $\mathcal{H}$ may result in utility
degradation, it is essential to choose a decent level of noise.

##### Robust Model Training.

Zhang et al. (2021) add Gaussian noise to the clipped gradients in each
training iteration to guarantee differential privacy (DP). However, subsequent
studies demonstrate that ensuring DP cannot effectively prevent MI attacks.
Instead, Zhang et al. (2023a) conceal real edges by pre-processing the
training graph, such as randomly flipping and rewriting. This operation
misleads the attacker into recovering dummy data that significantly differs
from the ground truth. To diminish the correlation between the model inputs
and outputs, Zhou et al. (2023) inject stochasticity into $\mathcal{A}$ and
regularize the mutual dependency among graph representations, adjacency, and
labels during training.

##  6 Conclusion and Research Directions

This survey presents a systematic review of recent advances in model inversion
on DNNs. We first comprehensively characterize existing MI methods for visual
tasks and compare them from different perspectives in detail. We also
investigate MI studies on discrete graph and text data and provide systematic
taxonomies. To further promote this field, we finally demonstrate several
unsolved challenges and suggest promising future directions.

##### Stronger Generative Prior for MI attacks.

In visual tasks, previous approaches leveraged various GANs as image priors
for reconstruction guidance. A notable contribution by Liu (2023) introduced
potent diffusion models into MI attacks. They trained a conditional diffusion
model Ho and Salimans (2022) to generate $64\times 64$ resolution images from
the target distribution. For addressing higher-resolution image recovery,
future research could explore utilizing pre-trained diffusion models with rich
image priors or alternative techniques to enhance generative model
utilization. This paradigm extension is also applicable to other modalities.

##### Certified Robustness against MI Attacks.

Although there have been intensive studies on MI robustness, current defense
approaches lack certified robustness guarantees. Certifiably robust approaches
for DNNs have been developed against adversarial attack Cohen et al. (2019)
and backdoor attack Wang et al. (2020). Hence, a solid theoretical framework
to provide certified guarantees in the context of MI attacks is indispensable.
Additionally, exploring ways to integrate user feedback and preferences into
the certification pipeline can enhance the practicality of ensuring certified
robustness in conversational AI models like ChatGPT.

##### Multi-Modal MI Attacks and Defenses.

Recently, advanced multi-modal models have shown sophisticated capabilities in
handling complex and diverse data. While current MI methods have made great
progress on unimodal models, there is a lack of research on multi-modal
foundation models, e.g., the CLIP Radford et al. (2021) and SAM Kirillov et
al. (2023). Exploring potential vulnerabilities during the cross-modal
inference process can be a promising and necessary direction for model
inversion attacks and defenses.

## References

  * An et al. [2022] Shengwei An, Guanhong Tao, Qiuling Xu, Yingqi Liu, Guangyu Shen, Yuan Yao, Jingwei Xu, and Xiangyu Zhang.  Mirror: Model inversion for deep learning network with high fidelity.  In NDSS, 2022.
  * Anil et al. [2021] Rohan Anil, Badih Ghazi, Vineet Gupta, Ravi Kumar, and Pasin Manurangsi.  Large-scale differentially private bert.  arXiv preprint arXiv:2108.01624, 2021.
  * Carlini et al. [2019] Nicholas Carlini, Chang Liu, Úlfar Erlingsson, Jernej Kos, and Dawn Song.  The secret sharer: Evaluating and testing unintended memorization in neural networks.  In USENIX Security, 2019.
  * Carlini et al. [2021] Nicholas Carlini, Florian Tramèr, Eric Wallace, Matthew Jagielski, Ariel Herbert-Voss, Katherine Lee, Adam Roberts, Tom Brown, Dawn Song, Úlfar Erlingsson, Alina Oprea, and Colin Raffel.  Extracting training data from large language models.  In USENIX Security, pages 2633–2650, August 2021.
  * Chen et al. [2021] Si Chen, Mostafa Kahla, Ruoxi Jia, and Guo-Jun Qi.  Knowledge-enriched distributional model inversion attacks.  In ICCV, 2021.
  * Cohen et al. [2019] Jeremy Cohen, Elan Rosenfeld, and Zico Kolter.  Certified adversarial robustness via randomized smoothing.  In ICML, 2019.
  * Dibbo [2023] Sayanton V. Dibbo.  Sok: Model inversion attack landscape: Taxonomy, challenges, and future roadmap.  In 2023 IEEE 36th Computer Security Foundations Symposium (CSF), pages 439–456, 2023.
  * Duddu et al. [2020] Vasisht Duddu, Antoine Boutet, and Virat Shejwalkar.  Quantifying privacy leakage in graph embedding.  In MobiQuitous, 2020.
  * Elmahdy and Salem [2023] Adel Elmahdy and Ahmed Salem.  Deconstructing classifiers: Towards a data reconstruction attack against text classification models.  arXiv preprint arXiv:2306.13789, 2023.
  * Elmahdy et al. [2022] Adel Elmahdy, Huseyin A Inan, and Robert Sim.  Privacy leakage in text classification: A data extraction approach.  arXiv preprint arXiv:2206.04591, 2022.
  * Fredrikson et al. [2014] Matthew Fredrikson, Eric Lantz, Somesh Jha, Simon Lin, David Page, and Thomas Ristenpart.  Privacy in pharmacogenetics: An $\\{$End-to-End$\\}$ case study of personalized warfarin dosing.  In USENIX Security, pages 17–32, 2014.
  * Fredrikson et al. [2015] Matt Fredrikson, Somesh Jha, and Thomas Ristenpart.  Model inversion attacks that exploit confidence information and basic countermeasures.  In CCS, pages 1322–1333, 2015.
  * Gong et al. [2023] Xueluan Gong, Ziyao Wang, Shuaike Li, Yanjiao Chen, and Qian Wang.  A gan-based defense framework against model inversion attacks.  IEEE TIFS, 2023.
  * Goodfellow [2016] Ian Goodfellow.  Nips 2016 tutorial: Generative adversarial networks.  arXiv preprint arXiv:1701.00160, 2016.
  * Han et al. [2023] Gyojin Han, Jaehyun Choi, Haeil Lee, and Junmo Kim.  Reinforcement learning-based black-box model inversion attacks.  In CVPR, 2023.
  * He et al. [2021] Xinlei He, Jinyuan Jia, Michael Backes, Neil Zhenqiang Gong, and Yang Zhang.  Stealing links from graph neural networks.  In USENIX Security, 2021.
  * Ho and Salimans [2022] Jonathan Ho and Tim Salimans.  Classifier-free diffusion guidance.  arXiv preprint arXiv:2207.12598, 2022.
  * Huang et al. [2022] Jie Huang, Hanyin Shao, and Kevin Chen-Chuan Chang.  Are large pre-trained language models leaking your personal information?  arXiv preprint arXiv:2205.12628, 2022.
  * Ishihara [2023] Shotaro Ishihara.  Training data extraction from pre-trained language models: A survey.  arXiv preprint arXiv:2305.16157, 2023.
  * Jegorova et al. [2022] Marija Jegorova, Chaitanya Kaul, Charlie Mayor, Alison Q O’Neil, Alexander Weir, Roderick Murray-Smith, and Sotirios A Tsaftaris.  Survey: Leakage and privacy at inference time.  IEEE TPAMI, 2022.
  * Kahla et al. [2022] Mostafa Kahla, Si Chen, Hoang Anh Just, and Ruoxi Jia.  Label-only model inversion attacks via boundary repulsion.  In CVPR, 2022.
  * Kandpal et al. [2022] Nikhil Kandpal, Eric Wallace, and Colin Raffel.  Deduplicating training data mitigates privacy risks in language models.  In ICML, 2022.
  * Karras et al. [2019] Tero Karras, Samuli Laine, and Timo Aila.  A style-based generator architecture for generative adversarial networks.  In CVPR, 2019.
  * Kirillov et al. [2023] Alexander Kirillov, Eric Mintun, Nikhila Ravi, Hanzi Mao, Chloe Rolland, Laura Gustafson, Tete Xiao, Spencer Whitehead, Alexander C Berg, Wan-Yen Lo, et al.  Segment anything.  arXiv preprint arXiv:2304.02643, 2023.
  * Li et al. [2022] Jingtao Li, Adnan Siraj Rakin, Xing Chen, Zhezhi He, Deliang Fan, and Chaitali Chakrabarti.  Ressfl: A resistance transfer framework for defending model inversion attack in split federated learning.  In CVPR, 2022.
  * Liu [2023] Rongke Liu.  Unstoppable attack: Label-only model inversion via conditional diffusion model.  arXiv preprint arXiv:2307.08424, 2023.
  * Nasr et al. [2023] Milad Nasr, Nicholas Carlini, Jonathan Hayase, Matthew Jagielski, A Feder Cooper, Daphne Ippolito, Christopher A Choquette-Choo, Eric Wallace, Florian Tramèr, and Katherine Lee.  Scalable extraction of training data from (production) language models.  arXiv preprint arXiv:2311.17035, 2023.
  * Nguyen et al. [2023a] Ngoc-Bao Nguyen, Keshigeyan Chandrasegaran, Milad Abdollahzadeh, and Ngai-Man Cheung.  Label-only model inversion attacks via knowledge transfer.  arXiv preprint arXiv:2310.19342, 2023.
  * Nguyen et al. [2023b] Ngoc-Bao Nguyen, Keshigeyan Chandrasegaran, Milad Abdollahzadeh, and Ngai-Man Cheung.  Re-thinking model inversion attacks against deep neural networks.  In CVPR, pages 16384–16393, 2023.
  * Parikh et al. [2022] Rahil Parikh, Christophe Dupuy, and Rahul Gupta.  Canary extraction in natural language understanding models.  arXiv preprint arXiv:2203.13920, 2022.
  * Peng et al. [2022] Xiong Peng, Feng Liu, Jingfeng Zhang, Long Lan, Junjie Ye, Tongliang Liu, and Bo Han.  Bilateral dependency optimization: Defending against model-inversion attacks.  In SIGKDD, 2022.
  * Qi et al. [2023] Gege Qi, YueFeng Chen, Xiaofeng Mao, Binyuan Hui, Xiaodan Li, Rong Zhang, and Hui Xue.  Model inversion attack via dynamic memory learning.  In ACM MM, pages 5614–5622, 2023.
  * Radford et al. [2021] Alec Radford, Jong Wook Kim, Chris Hallacy, Aditya Ramesh, Gabriel Goh, Sandhini Agarwal, Girish Sastry, Amanda Askell, Pamela Mishkin, Jack Clark, et al.  Learning transferable visual models from natural language supervision.  In ICML, 2021.
  * Song et al. [2017] Congzheng Song, Thomas Ristenpart, and Vitaly Shmatikov.  Machine learning models that remember too much.  In CCS, pages 587–601, 2017.
  * Struppek et al. [2022] Lukas Struppek, Dominik Hintersdorf, Antonio De Almeida Correira, Antonia Adler, and Kristian Kersting.  Plug & play attacks: Towards robust and flexible model inversion attacks.  In ICML, 2022.
  * Titcombe et al. [2021] Tom Titcombe, Adam J Hall, Pavlos Papadopoulos, and Daniele Romanini.  Practical defences against model inversion attacks for split neural networks.  arXiv preprint arXiv:2104.05743, 2021.
  * Wang et al. [2020] Binghui Wang, Xiaoyu Cao, Neil Zhenqiang Gong, et al.  On certifying robustness against backdoor attacks via randomized smoothing.  arXiv preprint arXiv:2002.11750, 2020.
  * Wang et al. [2021a] Kuan-Chieh Wang, Yan Fu, Ke Li, Ashish Khisti, Richard Zemel, and Alireza Makhzani.  Variational model inversion attacks.  In NeurIPS, 2021.
  * Wang et al. [2021b] Tianhao Wang, Yuheng Zhang, and Ruoxi Jia.  Improving robustness to model inversion attacks via mutual information regularization.  In AAAI, 2021.
  * Wen et al. [2021] Jing Wen, Siu-Ming Yiu, and Lucas CK Hui.  Defending against model inversion attack by adversarial examples.  In 2021 IEEE International Conference on Cyber Security and Resilience, 2021.
  * Wu et al. [2022] Fan Wu, Yunhui Long, Ce Zhang, and Bo Li.  Linkteller: Recovering private edges from graph neural networks via influence analysis.  In 2022 IEEE Symposium on Security and Privacy, 2022.
  * Yang et al. [2019] Ziqi Yang, Jiyi Zhang, Ee-Chien Chang, and Zhenkai Liang.  Neural network inversion in adversarial setting via background knowledge alignment.  In CCS, 2019.
  * Yang et al. [2020] Ziqi Yang, Bin Shao, Bohan Xuan, Ee-Chien Chang, and Fan Zhang.  Defending model inversion and membership inference attacks via prediction purification.  arXiv preprint arXiv:2005.03915, 2020.
  * Ye et al. [2022] Dayong Ye, Sheng Shen, Tianqing Zhu, Bo Liu, and Wanlei Zhou.  One parameter defense—defending against data inference attacks via differential privacy.  IEEE TIFS, 17:1466–1480, 2022.
  * Ye et al. [2023] Zipeng Ye, Wenjian Luo, Muhammad Luqman Naseem, Xiangkai Yang, Yuhui Shi, and Yan Jia.  C2fmi: Corse-to-fine black-box model inversion attack.  IEEE TDSC, 2023.
  * Yu et al. [2023] Weichen Yu, Tianyu Pang, Qian Liu, Chao Du, Bingyi Kang, Yan Huang, Min Lin, and Shuicheng Yan.  Bag of tricks for training data extraction from language models.  arXiv preprint arXiv:2302.04460, 2023.
  * Yuan et al. [2022] Zhuowen Yuan, Fan Wu, Yunhui Long, Chaowei Xiao, and Bo Li.  Secretgen: Privacy recovery on pre-trained models via distribution discrimination.  In ECCV, 2022.
  * Yuan et al. [2023] Xiaojian Yuan, Kejiang Chen, Jie Zhang, Weiming Zhang, Nenghai Yu, and Yang Zhang.  Pseudo label-guided model inversion attack via conditional generative adversarial network.  In AAAI, 2023.
  * Zanella-Béguelin et al. [2020] Santiago Zanella-Béguelin, Lukas Wutschitz, Shruti Tople, Victor Rühle, Andrew Paverd, Olga Ohrimenko, Boris Köpf, and Marc Brockschmidt.  Analyzing information leakage of updates to natural language models.  In CCS, 2020.
  * Zhang et al. [2018] Richard Zhang, Phillip Isola, Alexei A Efros, Eli Shechtman, and Oliver Wang.  The unreasonable effectiveness of deep features as a perceptual metric.  In CVPR, 2018.
  * Zhang et al. [2020] Yuheng Zhang, Ruoxi Jia, Hengzhi Pei, Wenxiao Wang, Bo Li, and Dawn Song.  The secret revealer: Generative model-inversion attacks against deep neural networks.  In CVPR, 2020.
  * Zhang et al. [2021] Zaixi Zhang, Qi Liu, Zhenya Huang, Hao Wang, Chengqiang Lu, Chuanren Liu, and Enhong Chen.  Graphmi: Extracting private graph data from graph neural networks.  In IJCAI, 2021.
  * Zhang et al. [2022] Zhikun Zhang, Min Chen, Michael Backes, Yun Shen, and Yang Zhang.  Inference attacks against graph neural networks.  In USENIX Security, 2022.
  * Zhang et al. [2023a] Zaixi Zhang, Qi Liu, Zhenya Huang, Hao Wang, Chee-Kong Lee, and Enhong Chen.  Model inversion attacks against graph neural networks.  IEEE TKDE, 35(9):8729–8741, 2023.
  * Zhang et al. [2023b] Zhexin Zhang, Jiaxin Wen, and Minlie Huang.  Ethicist: Targeted training data extraction through loss smoothed soft prompting and calibrated confidence estimation.  In The 61st Annual Meeting Of The Association For Computational Linguistics, 2023.
  * Zhou et al. [2023] Zhanke Zhou, Chenyu Zhou, Xuan Li, Jiangchao Yao, Quanming Yao, and Bo Han.  On strengthening and defending graph reconstruction attack with markov chain approximation.  In ICML, 2023.