# Label-Only Model Inversion Attacks via Boundary Repulsion

Mostafa Kahla
Virginia Tech
<EMAIL_ADDRESS>Si Chen
Virginia Tech
<EMAIL_ADDRESS>Hoang Anh Just
Virginia Tech
<EMAIL_ADDRESS>Ruoxi Jia
Virginia Tech
<EMAIL_ADDRESS>
###### Abstract

Recent studies show that the state-of-the-art deep neural networks are
vulnerable to model inversion attacks, in which access to a model is abused to
reconstruct private training data of any given target class. Existing attacks
rely on having access to either the complete target model (whitebox) or the
model’s soft-labels (blackbox). However, no prior work has been done in the
harder but more practical scenario, in which the attacker only has access to
the model’s predicted label, without a confidence measure. In this paper, we
introduce an algorithm, Boundary-Repelling Model Inversion (brep-MI), to
invert private training data using only the target model’s predicted labels.
The key idea of our algorithm is to evaluate the model’s predicted labels over
a sphere and then estimate the direction to reach the target class’s centroid.
Using the example of face recognition, we show that the images reconstructed
by brep-MI successfully reproduce the semantics of the private training data
for various datasets and target model architectures. We compare brep-MI with
the state-of-the-art whitebox and blackbox model inversion attacks and the
results show that despite assuming less knowledge about the target model,
brep-MI outperforms the blackbox attack and achieves comparable results to the
whitebox attack.

##  1 Introduction

Machine learning (ML) algorithms are often trained on private or sensitive
data, such as face images, medical records, and financial information.
Unfortunately, since ML models tend to memorize information about training
data, even when stored and processed securely, privacy information can still
be exposed through the access to the models [21]. Indeed, the prior study of
privacy attacks has demonstrated the possibility of exposing training data at
different granularities, ranging from “coarse-grained” information such as
determining whether a certain point participate in training [22, 15, 17, 11]
or whether a training dataset satisfies certain properties [10, 16], to more
“fine-grained” information such as reconstructing the raw data [8, 2, 25, 4].

In this paper, we focus on model inversion (MI) attacks, which goal is to
recreate training data or sensitive attributes given the access to the trained
model. MI attacks cause tremendous harm due to the “fine-grained” information
revealed by the attacks. For instance, MI attacks applied to personalized
medicine prediction models result in the leakage of individuals’ genomic
attributes [9]. Recent works show that MI attacks could even successfully
reconstruct high-dimensional data, such as images. For instance, [25, 4, 8,
24] demonstrated the possibility of recovering an image of a person from a
face recognition model given just their name.

Existing MI attacks have either assumed that the attacker has the complete
knowledge of the target model or assumed that the attack can query the model
and receive model’s output as confident scores. The former and the latter are
often referred to as the whitebox and the blackbox threat model, respectively.
The idea underlying existing whitebox MI attacks [25, 4] is to synthesize the
sensitive feature that achieves the maximum likelihood under the target model.
The synthesis is implemented as a gradient ascent algorithm. By contrast,
existing blackbox attacks [2, 20] are based on training an attack network that
predicts the sensitive feature from the input confidence scores. Despite the
exclusive focus on these two threat models, in practice, ML models are often
packed into a blackbox that only produces hard-labels when being queried. This
label-only threat model is more realistic as ML models deployed in user-facing
services need not expose raw confidence scores. However, the design of label-
only MI attacks is much more challenging than the whitebox or blackbox attacks
given the limited information accessible to the attacker.

Figure 1: Intuitive explanation of brep-MI. (A) Query the labels over a sphere
and estimate the direction on the sphere that can potentially lead to the
target label class. (B) Update the synthesized image according to the
estimated direction. Alternate between the estimation and update until the
sphere fits into the target class. (C) Increase the radius of the sphere. (D)
Repeat the steps above until the attack hits some query budget.

In this paper, we introduce, brep-MI, a general algorithm for MI attack in the
label-only setting, where the attacker can make queries to the target model
and obtain hard labels, instead of confidence scores. Similar to the main idea
of whitebox attacks, we still try to synthesize the most likelihood input for
the target class under the target model. However, in the label-only setting,
we cannot directly calculate the gradient information and leverage it to guide
the data synthesis. Our key insight to resolve this challenge is that a high-
likelihood region for a given class often lies at the center of the class and
is far away from any decision boundaries. Hence, we design an algorithm that
allows the synthesized image to iteratively move away from the decision
boundary, as illustrated in Figure 1. Specifically, we first query the labels
over a sphere and estimate the direction on the sphere that can potentially
lead to the target label class (A). We progressively move according to
estimated directions until the sphere fits into the target class (B). We then
increase the radius of the sphere (C) and repeat the steps above until the
attack hits some query budget (D). We theoretically prove that for linear
target models, the direction estimated from hard labels queried on the spheres
aligns with the gradient direction. We empirically show that brep-MI can also
lead to successful attacks against deep neural network-based target models. In
particular, the efficacy of the attack is even higher than the existing
blackbox attacks and comparable to the existing whitebox attacks.

Our contributions can be summarized as follows: (1) We propose the first
algorithm for label-only model inversion attacks. (2) We provide theoretical
justification for the algorithm in the linear target model case by proving the
updates used in our algorithm align with the gradient and also analyze the
error of alignment for nonlinear models. (3) We evaluate the attack on a range
of model architectures, datasets, and show that despite exploiting less
information about the target model, our attack outperforms the confidence-
based blackbox attack by a large margin and achieves comparable performance to
the state-of-the-art whitebox attack. Besides, we will release the data, code,
and models to facilitate future research.

##  2 Related Work

#### Model Inversion Attacks.

Model inversion attempts to reconstruct from partial up to full training
sample. Typically, MI attacks can be formalized as an optimization problem,
which goal is to find the sensitive feature value that achieves the highest
likelihood under the model been attacked. However, when the target model is a
deep neural network (DNN) or the private data lie in high-dimensional space,
such optimization problem becomes non-convex and directly solving it via
gradient descent may result in poor attack performance [8]; for example, when
attacking a face recognition model, the recovered images are blurry and do not
contain much private information. Recent work [25] proposes a GAN-based MI
attack method which is effective on DNNs. In particular, they learn a generic
prior from public data via GAN and solve the optimization problem over the
latent space rather than the unconstrained ambient space. However, their
attack method does not fully exploit private information contained in the
target model at the stage of training GAN. [4] significantly improves the
attack performance through a special design of GAN which can distill knowledge
from the target model; as a result, the generated images align better with the
private distribution. They further improve the performance by ensuring that
both the recovered image and its neighboring images have high likelihood.
While [25, 4] achieve success on attacking various models and datasets, their
attacks rely on whitebox access to the model. In many cases, the attacker can
only make prediction queries against a model, but not actually download the
model, which motivates the study of blackbox MI attacks. [24] analyzes the
blackbox setting and proposes an attack model which swaps the input and
prediction vector of the target model to perform model inversion. [2] proposes
to train a GAN and a surrogate model simultaneously, with the GAN generating
inputs that resemble private training data and the surrogate model mimicking
the target model’s behavior. All of the blackbox attacks make an assumption
that prediction confidences of the target model are revealed to the attacker.
However, it is more practical in real-world setting that an adversary, who
only makes queries to the model, can only obtain the hard labels, without
confidence scores. From this aspect, we aim to provide an effective MI attack
method that only requires access to the hard label, which we refer to as
label-only MI attacks.

#### Other Privacy Attacks.

Asides from MI, there are two other categories of privacy attacks that allow
adversaries to gain unauthorized information from the target model and its
data. In a membership inference attack, the attacker attempts to evaluate
whether a certain point is used in the target model’s training. This attack
technique was introduced by [22] who created multiple shadow models to
estimate the target model. [15, 17, 11] pointed out that the membership
inference attack exploits the overfitting of specific data points.
Interestingly, [6] performs a membership inference attack under same setting
as our brep-MI attack and notes that the viable defense against such an attack
is via differential privacy (DP). DP [7, 1] ensures that the trained model is
stable to the change of any single record in the training set. However, with
differential privacy, the target model’s test accuracy will significantly
degrade. Additionally, property inference attacks aim to infer from the
properties about the training dataset [10]. Compared to these attacks, MI is
arguably more challenging as the information it attempts to recover is higher
in resolution.

##  3 Threat Model

#### Attack goal.

In MI attacks, given the access to a target model
$f:[0,1]^{d}\rightarrow\mathbb{R}^{|C|}$ and any target class $c^{*}\in C$,
the attacker attempts to recover a representative point $x^{*}$ of the
training data from the class $c^{*}$; $d$ represents the dimension of the
model input; $C$ denotes the set of all class labels and $|C|$ is the size of
the label set. For example, an attack on the face recognition classifier would
try to recover the face image for a given identity based on the access to the
classifier.

#### Model knowledge.

The attacker’s knowledge about a target model can take different forms: (i)
Whitebox: complete access to all target model parameters; ii) Blackbox: access
to the confidence scores output by the target model; and iii) Label-only:
access to only the hard labels output by the model without the confidence
scores. Our paper will focus on the label-only setting. Specifically, given
the target network $f$, the attacker can query the target network at any input
$x$ and obtain the corresponding predicted label $\hat{y}(x)=\arg\max_{c\in
C}f_{c}(x)$.

#### Task Knowledge.

For the rest of the paper, we assume that the attacker has knowledge about the
task that the target model performs. This is a reasonable assumption, since
this information is available for existing online models, or can be inferred
from output labels.

#### Data Knowledge.

Since we assume that attackers know the task of the attacked model, it is
reasonable to assume that they can gain access to a public dataset from a
related distribution. For example, if attackers know that the target model is
trained to perform facial recognition, they can easily gather a public dataset
by leveraging the existing open-sourced datasets or crawling data from the
web. Throughout the paper, we assume that the public data and the private data
do not share any classes (e.g., identities) in common.

#### Target models.

Our approach neither makes assumption on the target model architecture, nor
requires the attacker to have any information about it. In other words, our
approach is model-agnostic. We will empirically show in Section 5 that our
brep-MI attack generalizes to a variety of models with different architectures
and sizes.

#### Target labels.

The attack can be _targeted_ , when the goal is to find $n$ input images that
maximize a set of $n$ _predefined_ labels, or _untargeted_ , when the goal is
to find $n$ input images that maximize a set of _any_ $n$ labels. The proposed
algorithm can apply to _both_ scenarios. In our evaluation, we will focus on
the more challenging scenario, where the attack is targeted for $n$ specific
labels.

##  4 Algorithm Design

In this section, we will present the design of our proposed algorithm brep-MI.
We will start by formulating the MI attack as an optimization problem. Then,
we describe an algorithm to estimate the gradient of the MI optimization
objective based only on predicted labels. We will rigorously characterize the
alignment between the estimate and the true gradient for the special case of
linear models and provide insights into the attack efficacy for deep,
nonlinear models.

###  4.1 Problem Formulation

Without loss of generality, we state the attack problem formulation for a
single target label and define $M_{c^{*}}:\mathbb{R}^{d}\rightarrow\mathbb{R}$
such that

$M_{c^{*}}(x)=f_{c^{*}}(x)-\max_{c\neq c^{*}}f_{c}(x),$ (1)

where $c^{*}$ is the target label. $M_{c^{*}}(x)$ represents the logit (or
confidence score) difference between the target class $c^{*}$ and the most
likely label in the rest of the classes. Note that when $x$ is predicted into
the target class (i.e., $c^{*}=\arg\max_{c\in C}f_{c}(x)$), $M_{c^{*}}(x)>0$.
Clearly, the most representative input for the target class $c^{*}$ should be
most distinguishable from all the other classes. Hence, we cast the MI problem
into an optimization problem that seeks for the input that achieves maximum
difference between the confidence for the target class and the highest
confidence for the other classes:

$\displaystyle\arg\max_{x\in[0,1]^{d}}M_{c^{*}}(x).$ (2)

However, for images, $x$ usually lies in a high-dimensional continuous data
space and optimizing over this space can easily get stuck in local minima that
do not correspond to any meaningful images. To resolve this issue, we leverage
the idea in [24, 25, 4] and optimize over a more semantically meaningful
latent space. This is done by using a public dataset to train GAN models and
then optimizing over the input to the GAN generator. Denote the publicly
trained generator by $G(z)$, where $z\in\mathbb{R}^{d^{\prime}}$ and
$d^{\prime}<d$. Now, the MI optimization problem can be updated to reflect the
change of optimizing $z$ rather than $x$ as follows:

$\arg\max_{z\in\mathbb{R}^{d}}M_{c^{*}}(G(z)).$ (3)

Unlike the whitebox setting, we cannot directly optimize $M_{c^{*}}(G(z))$
using gradients as we do not have access to the model parameters $f$.
Moreover, it is also not possible to apply zero-order optimization algorithms,
as they require access to the confidence scores output by the model.

###  4.2 brep-MI Algorithm

The intuition behind our algorithm is that the farther a point is from the
decision boundary of a class, the more representative this point becomes to
the class. Thus, the centroid of any class should be its good representative.
Inspired by this, we design an algorithm which tries to gradually move away
from the decision boundary. In a high level, our algorithm proceeds by first
sampling points over a sphere and then querying their labels. Intuitively, the
points that are not predicted into the target class represent the directions
that we want to move away from. Hence, we take an average over those points
and move in the direction opposite to the average. If all the points are
predicted into the target class, then we will increase the radius.

Let $\text{sign}(\cdot)$ be a function that returns $1$ if the input is
positive and $-1$, otherwise. We define
$\Phi_{c^{*}}:\mathbb{R}^{d}\rightarrow\\{-1,0\\}$:

$\displaystyle\small\Phi_{c^{*}}(z)$
$\displaystyle=\frac{\text{sign}(M_{c^{*}}(z))-1}{2}$ (4)
$\displaystyle=\left\\{\begin{array}[]{ll}&0,\text{ if
}c^{*}=\arg\underset{c\in C}{\max}f_{c}(G(z))\\\ &-1,\text{
otherwise.}\end{array}\right.$ (7)

Essentially, $\Phi_{c^{*}}(z)$ marks points that are not predicted into the
target class. Then, we define our gradient estimator as

$\widehat{M_{c^{*}}}(z,R)=\frac{1}{N}\sum_{n=1}^{N}\Phi_{c^{*}}(z+Ru_{n})u_{n},$
(8)

where $u_{n}$ is a uniformly random point sampled over a
$d^{\prime}$-dimensional sphere with radius $R$ and $N$ is the number of
points sampled on the sphere. Note that $\widehat{M_{c^{*}}}(z,R)$ can be
calculated in the label-only setting as it only requires the knowledge of
predicted label of the sampled points. We will then use
$\widehat{M_{c^{\prime}}}(z,R)$ to update $z$:

$z\leftarrow z+\alpha\widehat{M_{c^{*}}}(z,R),$ (9)

where $\alpha$ is the update step size. It can be either a fixed value or a
function of the current radius $R$. When all points sampled from the sphere of
the current radius are predicted into the target class, i.e.,
$\Phi_{c^{*}}(z+Ru_{n})=0$ for all $n=1,\ldots,N$, then we increase the radius
and alternate between estimating $\widehat{M_{c^{\prime}}}(z,R)$ using Eq. (8)
and performing update with Eq. (9) at the new radius.

The pseudo-code of brep-MI is provided in Algorithm 1. brep-MI starts with the
initial point correctly classified as the target class. To ensure this, images
are sampled from the GAN until a point belonging to the target class is
generated. Note that the initial point, although classified into the target
class, is almost _never_ a representative point for the target class (see more
examples in Fig. 3). The radius of the sphere is initialized to a reasonably
small value. Then, the algorithm will try to move away from the decision
boundary iteratively. At each iteration, we sample $N$ points on the sphere
with radius $R$ centered at the current point and query their labels from the
target model. If all the points are classified into the target class, the
radius will be enlarged; otherwise, we estimate
$\widehat{M_{c^{\prime}}}(z,R)$ using Eq. (8) and update $z$ according to Eq.
(9). Note that the update is reverted if the new point $z$ lies outside the
target class. In that case, we will resample the points on the sphere and
compute a new update. The algorithm will be halted when it is not possible to
find a larger sphere such that all the samples on that sphere fall into the
target class. The output of the algorithm is a point ($z^{*}$) with the
largest sphere that can fit into the target class. This indicates that the
point is the farthest from the boundary. We will use this point to evaluate
the attack.

input : Target model’s hard-label prediction $\hat{y}$ ; target class $c^{*}$,
number of samples $N$; number of maximum iterations $maxIters$; initial sphere
sampling radius $R_{0}$; radius multiplier $\gamma$; data point learning rate
$\alpha$

output : Representative sample $z^{*}$ for $c^{*}$.

ensure: A sample $z$ in the target class $c^{*}$ by repeatedly sampling from
the GAN’s latent space.

1 Set $R\leftarrow R_{0}$.

2 Set $iters\leftarrow 0$.

3 Set $points\leftarrow vector(N)$

4 while  _iters $<$ maxIters_ do

5 $points\leftarrow\text{random N points on a sphere r=R}$

// Check if all sampled points are in target class.

6 if  _ $points$ in $c^{*}$_ then

// Update radius and current best point

7 $R\leftarrow R\times\gamma$ .

8 $z^{*}\leftarrow z$ .

9 $iters\leftarrow 0$.

10

11 else

12 Compute $\widehat{M_{c^{*}}}(z,R)$ via Eq. (8)

13 $z_{\text{new}}\leftarrow$ the RHS of Eq. (9)

14 if  _if $\hat{y}(z_{\text{new}})=c^{*}$_ then

15 $z\leftarrow z_{\text{new}}$

16 end if

17

18 end if

19

20 end while

Algorithm 1 brep-MI Decision-Based Zero Order Optimization Algorithm.

###  4.3 Attack Justification

As our gradient estimator $\widehat{M_{c^{*}}}(z,R)$ repels non-target-class
points, intuitively, it points towards the direction that increases the target
class’ likelihood. We provide a theorem that characterizes the alignment
between the proposed estimator and the true gradient $\nabla
M_{c^{*}}\left(z\right)$ for special cases of linear classification models
(e.g., logistic regression).

######  Theorem 1.

Assume $f$ has a linear classification model. Let $z$ be an arbitrary point
within the target class, i.e. $M_{c^{*}}(z)>0$. Then, the cosine of the angle
between $\mathbb{E}[\widehat{M_{c^{*}}}(z,R)]$ and $\nabla
M_{c^{*}}\left(z\right)$ is bounded by

$\displaystyle\cos\angle\left(\mathbb{E}\left[\widehat{\nabla
M_{c^{*}}}\left(z,R\right)\right],\nabla M_{c^{*}}\left(z\right)\right)$ (10)
$\displaystyle\geq
1-\mathcal{O}\bigg{(}\frac{M_{c^{*}}\left(z\right)^{2}(d-1)^{2}}{\delta^{2}R^{2}\left\|\nabla
M_{c^{*}}\left(z\right)\right\|_{2}^{2}}\bigg{)}.$ (11)

Therefore, with increasing radius $R$,

$\lim_{R\rightarrow\infty}\cos\angle\left(\mathbb{E}\left[\widehat{M_{c^{*}}}\left(z,R\right)\right],\nabla
M_{c^{\star}}\left(z\right)\right)=1,$

which tells that the estimator is asymptotically unbiased for gradient
estimation.

The proof is provided in the Appendix 1.1. Theorem 2 shows that as long as $R$
is large enough, the gradient estimator aligns well with the actual gradient.

For the deep learning model with bounded nonlinearity, we can also derive the
bound for the cosine of the angle between the estimate and the true gradient:
$\cos\angle\left(\mathbb{E}\left[\widehat{\nabla
M_{c^{*}}}\left(z,R\right)\right],\nabla M_{c^{*}}\left(z\right)\right)\geq
1-\mathcal{O}\bigg{(}\frac{\left[M_{c^{*}}\left(z\right)^{2}+L^{2}\delta^{4}R^{4}+4M_{c}(z)\delta^{2}LR^{2}\right](d-1)^{2}}{\delta^{2}R^{2}\left\|\nabla
M_{c^{*}}\left(z\right)\right\|_{2}^{2}}\bigg{)}$, where $L$ characterizes the
level of nonlinearity. A more formal statement of the result will be provided
in Appendix. The result shows that with increasing $R$ the estimated gradient
will align with the true gradient. However, after a certain point of
inflection, increasing radius will only decrease the accuracy of the estimate.
This occurrence is actually well presented in our experiments in Section 5,
which shows the “sweet point” for reaching the radius R that maximizes the
success rate.

##  5 Evaluation

Our evaluation aims to answer the following questions: (1) Can brep-MI
successfully attack deep nets with different architectures and trained on
different datasets? (2) How many queries does brep-MI require to perform a
successful attack? (3) How does the distributional shift between private data
and public data affect the attack performance? (4) How sensitive is brep-MI to
the initialization and the sphere radius? In the main text, we will focus on a
canonical application–face recognition–as our attack target. We will leave
experiments on other applications to the Appendix.

###  5.1 Experimental Setup

#### Datasets.

We experiment on three different face recognition datasets: CelebA [14],
Facescrub [18], and Pubfig83 [19]. Similar to [25, 4, 24] we crop the images
of all datasets to the center and resize them to $64x64$. We split the
identities into the public domain (which we train our GAN on), and the private
domain (which we will train target models on). _There are no overlapping
identities between the public and the private domain._ This means that the
attacker has zero knowledge about the identities in the private domain. We
then perform the attack on the classifier that is trained on the private
domain. The details about each dataset are shown in Table 1. To study the
impact of a large distributional shift between private and public domain on
the attack performance, we use the FFHQ dataset [13] as our public domain to
train the GAN and the aforementioned three datasets as the private domains.

Dataset | #Images | #Total id | #Public id | #Private id | # Target id
---|---|---|---|---|---
CelebA | 202,599 | 10,177 | 9,177 | 1,000 | 300
Pubfig83 | 13,600 | 83 | 33 | 50 | 50
Facescrub | 106,863 | 530 | 330 | 200 | 200

Table 1:  Details on how we split datasets in evaluations into the public and
the private domains.

#### Target Models.

In addition to evaluating our attack on a range of datasets, we also evaluate
our attack on different models with a variety of architectures. To provide
consistent results with the previous work, we use the same model architectures
used in the state-of-the-art MI attack [4]: (1) face.evoLve adpated from [5];
(2) ResNet-152 adapted from [12]; and (3) VGG16 adapted from [23].

#### Evaluation Protocol.

For all evaluations with brep-MI, we perform a targeted attack as it is a more
challenging setting compared to untargeted attack. Following [25, 4], we use
the _attack accuracy_ to measure the attack performance. The attack accuracy
is based on an evaluation classifier, which predicts the identities for
reconstructed face images and is a proxy for human judge. Specifically, the
attack accuracy is calculated by the ratio of the number of reconstructed
images that are correctly classified into the corresponding target classes
over the total number of reconstructed images. As the evaluation classifier
reflects a human judge, it should have high performance. At the same time, it
should be different from the attacked target models to avoid some semantically
meaningless reconstructed images that overfits to the target models to be
considered as good reconstructions.

#### Hyperparameters.

We manually fine-tuned the hyperparameters of brep-MI in our evaluations. We
found empirically that the best initial radius $R_{0}$ is $2$, the radius
expansion coefficient $\gamma$ is 1.3, and the step size $\alpha_{t}$ =
min($R/3$, 3). We choose N, the number of sampled points on a sphere, to be 32
unless otherwise specified. maxIters is chosen to be 1000, i.e., brep-MI
terminates when more than 1000 iterations have passed for a certain $R$
without having all points on sphere to be classified as the target class.

#### Baselines.

Since this is the first work that provides a solution to label-only MI
attacks, we have no baselines to evaluate against. We opt to evaluate against
whitebox and blackbox attacks in which the attacker has a _greater_ advantage
in terms of additional knowledge about target models. To ensure fair
comparison, we apply all baselines over the same set of target identities for
each dataset and the same target models. Then we evaluate attack accuracy
against the same evaluation classifiers. Two of our baselines are white box
attacks, including Generative Model Inversion (GMI) [25], which is the first
MI attack algorithm against deep nets, and Knowledge-Enriched Distributional
Model Inversion attack (KED-MI) [4], which provides the currently state-of-
the-art performance for whitebox MI. The GAN models in GMI is set to be the
same as the GAN in our attack. KED-MI relies on the access to target model
parameters in training the GAN models. However, we cannot access such
information and train the same GAN in our setting. We also employ a blackbox
attack [24], referred to as the learning-based model inversion (LB-MI) as one
of our baselines. LB-MI builds an inversion model that learns to reconstruct
images from the soft-labels produced by the target model. To reconstruct the
most representative image for a given identity, we feed a one-hot encoding for
that identity at the input of the inversion model and receive the output.

###  5.2 Results

#### Performance on Different Datasets.

We compare brep-MI to whitebox and blackbox methods on the three different
face datasets. We use FaceNet64 as the target model across all datasets. For
each dataset, the GAN models are trained on its public identities, and target
models are trained on the private identities. Table 2 shows that our approach
considerably outperforms both the whitebox GMI attack and the blackbox attack
on all datasets. Further, our method surpasses the state-of-the-art whitebox
KED-MI attack on Pubfig83 and achieves a close attack accuracy on the CelebA
dataset. On the other hand, we fall behind by 15% on the Facescrub dataset. It
is worth noting that the outcome of this experiment implies that there is
still a considerable potential for development in the other threat models in
MI attacks, particularly blackbox attacks (which perform poorly with respect
to the other threat models). The reason why GMI performs poorly even with the
whitebox knowledge is that it optimizes the likelihood of only the synthesized
data point without considering the neighborhood of the point. Hence, it is
possible that optimization gets stuck in a sharp local maximum that does not
represent the class. On the other hand, both brep-MI and KED-MI explicitly
finds a neighborhood with high likelihood, which turns out to be crucial to
produce representative points and enhance attack performance. It is worth
noting that the blackbox attack, although leveraging more knowledge about
target model than our attack, consistently achieves the worst performance.
Compared to the other attacks, the blackbox attack utilizes a very different
idea for distilling knowledge from public datasets. It uses the public data to
train the inversion model whereas the other attacks all train GAN models on
the public data. The results suggest that GANs are more effective in
distilling public knowledge than an inversion model. So a potential way to
improve blackbox attack is to regularize the synthesized images via GAN.

Dataset | [Whitebox] | [Blackbox] | [Label-only]
---|---|---|---
GMI | KED-MI | LB-MI | brep-MI
CelebA | 32.00% | 82.00% | 1.67% | 75.67%
Pubfig83 | 24.00% | 62.00% | 2.00% | 66.00%
Facescrub | 19.00% | 48.00% | 0.50% | 35.68%

Table 2:  Attack performance comparison various datasets.

#### Performance on Different Models.

We also evaluate our attack on multiple different models trained on the same
dataset (CelebA). This experiment is intended to test whether our approach can
generalize to different model architectures. Table 3 shows that brep-MI indeed
continues to perform well on a variety of target model architectures. In
particular, brep-MI outperforms GMI and the blackbox attack by a substantial
margin for all model architectures. As we can see, the attack accuracy is
$2x-4x$ that of GMI attack, while the blackbox attack continues to have $<2\%$
accuracy. Additionally, our performance on all model architectures is
comparable to that of the state-of-the-art whitebox. Similar to other attacks,
our attack becomes more successful when the target model has higher predictive
power.

Model Archt. | [Whitebox] | [Blackbox] | [Label-only]
---|---|---|---
GMI | KED-MI | LB-MI | brep-MI
FaceNet64 | 32.00% | 82.00% | 1.67% | 75.67%
IR152 | 26.00% | 83.00% | 0.33% | 72.00%
VGG16 | 15.00% | 69.00% | 1.33% | 63.33%

Table 3:  Attack performance comparison on different model architectures
trained on the CelebA dataset.

#### Cross-Dataset Evaluation.

Public$\rightarrow$Private | [Whitebox] | [Blackbox] | [Label-only]
---|---|---|---
GMI | KED-MI | LB-MI | brep-MI
FFHQ$\rightarrow$CelebA | 9.00% | 48.33% | 0.67% | 46.00%
FFHQ$\rightarrow$Pubfig83 | 28.00% | 88.00% | 4.00% | 80.00%
FFHQ$\rightarrow$Facescrub | 12.00% | 60.00% | .015% | 39.20%

Table 4:  Performance comparison when there is a large distribution shift
between public and private data.

In prior experiments, we assumed that the attacker had access to public data
with low distributional shift with the private data. This is because both
public and private domains are derived from the same dataset. It is important
to consider a more pragmatic scenario, in which the attacker has access to
only public data that have a larger distributional shift. To investigate this
scenario, we perform an experiment in which we use the FFHQ dataset as public
data.

As shown in Table 4, the accuracy indeed decreases significantly for the
CelebA dataset when we utilize FFHQ as our public dataset. Interestingly, the
attack accuracy for Pubfig83 and Facescrub datasets has increased. The
rationale for this performance boost is that Pubfig83 and Facescrub datasets
have just 33 and 330 identities in their public distributions, respectively,
as shown in Table 1. This means that the GAN models trained on these datasets
would lack the ability to generalize and thus, produce bad results. Therefore,
the ability of GAN models to generalize to the large number of identities in
FFHQ compensate for the distributional shift and consequently, the results
improve. On the other hand, the CelebA dataset has a rather significant number
of public identities (9177 identities). Thus, the GAN is already capable of
generalizing across different identities, and the performance increase
associated with generalizing on a more varied dataset is insufficient to
compensate for the performance reduction associated with distributional shift.
The takeaway from this experiment is that having a large, diverse public data
for distilling a distributional prior is crucial to MI attack performance.

#### Limited Query Budget.

We investigate the performance of our attack at various query budgets. In
practice, some online models, such as Google’s cloud vision API, limit the
number of queries per minute, others may ban users if they identify an
unusually high volume of queries. Due to the fact that some attack scenarios
restrict the amount of queries that may be sent to the target model, it is
important to investigate the impact of this restriction on the attack
performance.

This restriction has not been addressed in prior works that conduct whitebox
MI attacks in the literature. This is because the attacker, by definition, has
complete access to the model parameters and can thus create an offline copy of
the model, and then proceed with the attack offline with unlimited queries.
However, for blackbox attacks in general (including label-only attack), the
user cannot copy model parameters to an offline model. As a result, the query
budget may become a constraint.

Fig.2 (a) demonstrates how brep-MI performs under different query budgets. We
see that the attack accuracy increases exponentially by increasing the query
budget. This is true until we hit some query budget, then attack accuracy
starts decreasing again. We will provide some insights on it in later in the
paper. For all datasets examined in this paper, recovering a representative
image to a private class requires from $10k$ to $16k$ queries to the model,
which is very reasonable.

The attacker should also be concerned when choosing the hyperparameter $N$
under limited query budgets. Choosing large $N$ would increase the number of
sampled points on sphere, and produce a better estimator for our update
direction. On the other hand, for a fixed query budget, increasing $N$ means
decreasing the number of possible iterations in the attack. We conducted
experiments to show this trade-off between spending queries to get better
gradient estimator per iteration vs using queries to apply more iterations.
Fig. 3 (b), (c), and (d) indicate that, for small query budgets, brep-MI
performs slightly better when spending query budget on increasing the number
of iterations, instead of increasing $N$. However, for sufficiently large
query budget, increasing $N$ produces better results.

Figure 2: Attack accuracy of brep-MI under different query budgets. (a)
compares different datasets. (b), (c), and (d) compare different sampling
strategies for CelebA, pubfig, and Facescrub respectively. Query Budget is
shown in a base-2 log for convenience.

#### Analyzing brep-MI.

A qualitative analysis for our brep-MI can be seen in Fig. 3. It is noticeable
that the first generated image at the beginning of the attack is not a good
representative for the target class. The progression of the image towards the
groundtruth images is clearly seen with the increase of $R$ .

Figure 3: brep-MI’s progression along each radius from the first random
initial point until the algorithm’s termination.

Below, we provide some quantitative analysis. Table 5 analyzes the
intermediate steps when attacking FaceNet64 model on CelebA dataset. We say
the attack reached a radius $R$ when it finds a center point, for which all
points sampled on a sphere with radius $R$ lie in the correct target class. We
report for each reached radius during the attack the following measurements:
(i) the percentage of the target identities that successfully reach it
(column: labels % ); (ii) the minimum, maximum, average number of iterations
required to reach it; and (iii) the attack success accuracy of the points that
reached it (column: success %).

As we can see in the ”labels %” column, brep-MI is able to increase $R$
multiple times for all target identities. In fact, all target identities had
their $R$ increased by brep-MI at least 5 times. This shows the effectiveness
of our algorithm to repel away from the boundary and get closer to the center
of the class (which is our goal).

Another interesting observation is that the bigger the radius is, the higher
the attack accuracy we get. This is true until reaching a certain radius size
then the accuracy starts dropping. As suggested by our theoretical analysis,
at high radii, the gradient estimator becomes erroneous; hence, following the
direction of gradient estimator will decrease the attack accuracy at those
radii. This observation is consistent for all our conducted experiments
regardless the model or dataset. Unfortunately, since the attacker does not
have any ground truth images of the target class (or an evaluator classifier),
it is not possible to decide what is the best radius that the algorithm should
stop at. Nevertheless, as seen in the table 5, the number of identities that
reached those radii is very low and their contribution to the final attack
accuracy is small. Additionally, our stopping criterion for the algorithm
empirically provides close results to the best radius. For this experiment
particularly, we were able to get 75.67% which is close to stopping at the
best radius.

Radius | labels % | min iters | max iters | avg iters | success %
---|---|---|---|---|---
2.00 | 100.00% | 0 | 191 | 37.29 | 23.00%
2.60 | 100.00% | 0 | 246 | 63.20 | 30.33%
3.38 | 100.00% | 0 | 374 | 103.43 | 45.00%
4.39 | 100.00% | 2 | 627 | 156.65 | 56.00%
5.71 | 100.00% | 28 | 947 | 230.64 | 63.67%
7.43 | 100.00% | 53 | 1721 | 336.38 | 71.67%
9.65 | 97.00% | 89 | 1899 | 502.60 | 77.66%
12.55 | 71.00% | 141 | 1909 | 746.60 | 80.28%
16.31 | 20.33% | 298 | 1823 | 939.90 | 70.49%
21.21 | 1.67% | 492 | 1875 | 1122.00 | 60.00%
27.57 | 0.67% | 660 | 728 | 694.00 | 50.00%
35.84 | 0.33% | 877 | 877 | 877.00 | 0.00%

Table 5:  Analysis on the intermediate steps of our algorithm. Figure 4:
Ablation study of initialization of brep-MI. “Init” is the initial sampled
point that we start our attack from. “Outp” is the final output of the attack.

#### Effect of Random Sampling

Due to the fact that brep-MI starts from an initial random point, we conducted
an experiment to show whether different initial points would affect the
algorithm outcome. We started three attacks with different random seeds on a
FaceNet64 model trained on the CelebA dataset. The accuracy is 75.67%, 76.33%,
and 75.67% respectively. This shows that the random initial point has little
effect quantitatively on our algorithm. Fig.4 demonstrates our quantitative
results, where we can observe that even under different initial points, the
output of the algorithm is close to the ground truth images.

##  6 Conclusion

We presented a novel algorithm to perform the first label-only MI attack.
Experiments have showed the effectiveness of our approach on different
datasets and model architectures. Interestingly, the approach provides
comparable results with the state-of-the-art whitebox attacks and outperforms
all the other baselines despite the fact that they make stronger assumption
about the attacker knowledge. As future work, the closeness of the results in
multiple experiments between the label-only attack and the state-of-the-art
whitebox attack indicates that there may still be room for improvement for
whitebox attack. Similarly, the blackbox baseline attack underperforms our
label-only attack by a huge margin although it can access more fine-grained
model output than our label-only attack. Theoretically, it should be an upper
bound for our performance.

## References

  * [1] Martin Abadi, Andy Chu, Ian Goodfellow, H Brendan McMahan, Ilya Mironov, Kunal Talwar, and Li Zhang.  Deep learning with differential privacy.  In Proceedings of the 2016 ACM SIGSAC conference on computer and communications security, pages 308–318, 2016.
  * [2] Ulrich Aïvodji, Sébastien Gambs, and Timon Ther.  Gamin: An adversarial approach to black-box model inversion.  arXiv preprint arXiv:1909.11835, 2019.
  * [3] Jianbo Chen, Michael I Jordan, and Martin J Wainwright.  Hopskipjumpattack: A query-efficient decision-based attack.  In 2020 ieee symposium on security and privacy (sp), pages 1277–1294. IEEE, 2020.
  * [4] Si Chen, Mostafa Kahla, Ruoxi Jia, and Guo-Jun Qi.  Knowledge-enriched distributional model inversion attacks, 2021.
  * [5] Yu Cheng, Jian Zhao, Zhecan Wang, Yan Xu, Karlekar Jayashree, Shengmei Shen, and Jiashi Feng.  Know you at one glance: A compact vector representation for low-shot learning.  In Proceedings of the IEEE International Conference on Computer Vision Workshops, pages 1924–1932, 2017.
  * [6] Christopher A Choquette-Choo, Florian Tramer, Nicholas Carlini, and Nicolas Papernot.  Label-only membership inference attacks.  In International Conference on Machine Learning, pages 1964–1974. PMLR, 2021.
  * [7] Cynthia Dwork, Aaron Roth, et al.  The algorithmic foundations of differential privacy.  Found. Trends Theor. Comput. Sci., 9(3-4):211–407, 2014.
  * [8] Matt Fredrikson, Somesh Jha, and Thomas Ristenpart.  Model inversion attacks that exploit confidence information and basic countermeasures.  In Proceedings of the 22nd ACM SIGSAC conference on computer and communications security, pages 1322–1333, 2015.
  * [9] Matthew Fredrikson, Eric Lantz, Somesh Jha, Simon Lin, David Page, and Thomas Ristenpart.  Privacy in pharmacogenetics: An end-to-end case study of personalized warfarin dosing.  In 23rd $\\{$USENIX$\\}$ Security Symposium ($\\{$USENIX$\\}$ Security 14), pages 17–32, 2014.
  * [10] Karan Ganju, Qi Wang, Wei Yang, Carl A Gunter, and Nikita Borisov.  Property inference attacks on fully connected neural networks using permutation invariant representations.  In Proceedings of the 2018 ACM SIGSAC conference on computer and communications security, pages 619–633, 2018.
  * [11] Jamie Hayes, Luca Melis, George Danezis, and Emiliano De Cristofaro.  Logan: Membership inference attacks against generative models.  In Proceedings on Privacy Enhancing Technologies (PoPETs), pages 133–152. De Gruyter, 2019.
  * [12] Kaiming He, Xiangyu Zhang, Shaoqing Ren, and Jian Sun.  Deep residual learning for image recognition.  In Proceedings of the IEEE conference on computer vision and pattern recognition, pages 770–778, 2016.
  * [13] Tero Karras, Samuli Laine, and Timo Aila.  A style-based generator architecture for generative adversarial networks.  In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 4401–4410, 2019.
  * [14] Ziwei Liu, Ping Luo, Xiaogang Wang, and Xiaoou Tang.  Deep learning face attributes in the wild.  In Proceedings of the IEEE international conference on computer vision, pages 3730–3738, 2015.
  * [15] Yunhui Long, Vincent Bindschaedler, Lei Wang, Diyue Bu, Xiaofeng Wang, Haixu Tang, Carl A Gunter, and Kai Chen.  Understanding membership inferences on well-generalized learning models.  arXiv preprint arXiv:1802.04889, 2018.
  * [16] Luca Melis, Congzheng Song, Emiliano De Cristofaro, and Vitaly Shmatikov.  Exploiting unintended feature leakage in collaborative learning.  In 2019 IEEE Symposium on Security and Privacy (SP), pages 691–706. IEEE, 2019.
  * [17] Milad Nasr, Reza Shokri, and Amir Houmansadr.  Comprehensive privacy analysis of deep learning: Passive and active white-box inference attacks against centralized and federated learning.  In 2019 IEEE symposium on security and privacy (SP), pages 739–753. IEEE, 2019.
  * [18] Hong-Wei Ng and Stefan Winkler.  A data-driven approach to cleaning large face datasets.  In 2014 IEEE international conference on image processing (ICIP), pages 343–347. IEEE, 2014.
  * [19] Nicolas Pinto, Zak Stone, Todd Zickler, and David Cox.  Scaling up biologically-inspired computer vision: A case study in unconstrained face recognition on facebook.  In CVPR 2011 WORKSHOPS, pages 35–42. IEEE, 2011.
  * [20] Anton Razzhigaev, Klim Kireev, Edgar Kaziakhmedov, Nurislam Tursynbek, and Aleksandr Petiushko.  Black-box face recovery from identity features.  In European Conference on Computer Vision, pages 462–475. Springer, 2020.
  * [21] Maria Rigaki and Sebastian Garcia.  A survey of privacy attacks in machine learning.  arXiv preprint arXiv:2007.07646, 2020.
  * [22] Reza Shokri, Marco Stronati, Congzheng Song, and Vitaly Shmatikov.  Membership inference attacks against machine learning models.  In 2017 IEEE Symposium on Security and Privacy (SP), pages 3–18. IEEE, 2017.
  * [23] Karen Simonyan and Andrew Zisserman.  Very deep convolutional networks for large-scale image recognition.  arXiv preprint arXiv:1409.1556, 2014.
  * [24] Ziqi Yang, Ee-Chien Chang, and Zhenkai Liang.  Adversarial neural network inversion via auxiliary knowledge alignment.  arXiv preprint arXiv:1902.08552, 2019.
  * [25] Yuheng Zhang, Ruoxi Jia, Hengzhi Pei, Wenxiao Wang, Bo Li, and Dawn Song.  The secret revealer: Generative model-inversion attacks against deep neural networks.  In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 253–261, 2020.

#### Appendix

##  Appendix A Bounding Estimated Gradient

The following proof technique was inspired by [3]. However, in our case the
major difference is that our initial point $z$ is an arbitrary point within
the target class instead of being near the decision boundary, the fact which
the authors [3] used to assume $M_{c^{*}}\left(z\right)=0$. Furthermore, our
goal is to move towards the centroid of a target class not to reach the
boundary, which is completely opposite direction than ours. In summary, by
exploiting the Taylor’s theorem and the Minkowski inequality for expectations,
we bounded the cosine angle between our gradient estimator and the true
gradient, which we present for two different attack settings.

###  A.1 Linear Case

######  Theorem 2.

Assume $f$ has a linear classification model. Let $z$ be an arbitrary point
within the target class, i.e. $M_{c^{*}}(z)>0$. Then, the cosine of the angle
between $\mathbb{E}[\widehat{M_{c^{*}}}(z,R)]$ and $\nabla
M_{c^{*}}\left(z\right)$ is bounded by

$\cos\angle\left(\mathbb{E}\left[\widehat{\nabla
M_{c^{*}}}\left(z,R\right)\right],\nabla M_{c^{*}}\left(z\right)\right)\\\
\geq
1-\mathcal{O}\bigg{(}\frac{M_{c^{*}}\left(z\right)^{2}(d-1)^{2}}{\delta^{2}R^{2}\left\|\nabla
M_{c^{*}}\left(z\right)\right\|_{2}^{2}}\bigg{)}.$ (12)

Therefore, with increasing radius $R$,

$\lim_{R\rightarrow\infty}\cos\angle\left(\mathbb{E}\left[\widehat{M_{c^{*}}}\left(z,R\right)\right],\nabla
M_{c^{\star}}\left(z\right)\right)=1,$ (13)

which tells that the estimator is asymptotically unbiased for gradient
estimation.

###### Proof.

Let $Ru$ be a random vector uniformly distributed on the sphere, where $R>0$
is a radius of the sphere. By Taylor’s theorem, for any $\delta\in(0,1)$, we
have that

$M_{c^{*}}\left(z+\delta Ru\right)=M_{c^{*}}\left(z\right)+\delta\nabla
M_{c^{*}}\left(z\right)^{T}Ru.$ (14)

Recall that

$M_{c^{*}}(z)>0$

and let $w:=\frac{M_{c^{*}}(z)}{\delta R}.$
For the case $\nabla M_{c^{*}}\left(z\right)^{T}u>w$, using Taylor series
expansion and the fact that

$\delta\nabla M_{c^{*}}(z)^{T}Ru>w\delta R=M_{c^{*}}(z)>0,$

we derive that $M_{c^{*}}\left(z+\delta Ru\right)>M_{c^{*}}(z)$.
Similarly, for the case $\nabla M_{c^{*}}\left(x_{t}\right)^{T}u<-w$, using
Taylor expansion and the fact that

$\delta\nabla M_{c^{*}}\left(z\right)^{T}Ru<-w\delta R=-M_{c^{*}}(z)<0,$

we have that $M_{c^{*}}\left(z+\delta Ru\right)<M_{c^{*}}(z)$.
Therefore, from these two cases, we arrive at

$\displaystyle\phi_{c^{*}}\left(z+\delta
Ru\right)=\left\\{\begin{array}[]{l}0\text{ if }\nabla
M_{c^{*}}\left(z\right)^{T}Ru>w\\\ -1\text{ if }\nabla
M_{c^{*}}\left(z\right)^{T}Ru<-w.\end{array}\right.$ (17)

We define $v_{1}=\nabla S\left(z\right)/\left\|\nabla
M_{c^{*}}\left(z\right)\right\|_{2},v_{2},\ldots,v_{d}$ by expanding the
vector $\nabla M_{c^{*}}\left(z\right)$ to orthogonal bases in
$\mathbb{R}^{d}$. Then, we can write a random vector
$Ru=\sum_{i=1}^{d}\beta_{i}v_{i}$, where $\beta$ is uniformly distributed on
the sphere of radius $R$. We construct an upper cap $E_{1}:=$ $\left\\{\nabla
M_{c^{*}}\left(z\right)^{T}Ru>w\right\\}$, the annulus
$E_{2}:=\left\\{\left|\nabla
M_{c^{*}}\left(z\right)^{T}Ru\right|<w.\right\\}$, and the lower cap
$E_{3}:=\left\\{\nabla M_{c^{*}}\left(z\right)^{T}Ru<-w\right\\}.$ Let
$p:=\mathbb{P}\left(E_{2}\right)$ be the probability of event $E_{2}$, then
$\mathbb{P}\left(E_{1}\right)=\mathbb{P}\left(E_{3}\right)=(1-p)/2.$ For any
$i\neq 1$ by symmetry:

$\displaystyle\mathbb{E}\left[\beta_{i}\mid
E_{1}\right]=\mathbb{E}\left[\beta_{i}\mid E_{3}\right]=0.$ (18)

Then, the expected value of the estimator becomes

$\displaystyle\begin{split}\mathbb{E}\left[\phi_{c^{*}}\left(z+\delta
Ru\right)u\right]=p\cdot\left(\mathbb{E}\left[\phi_{c^{*}}\left(z+\delta
Ru\right)u\mid E_{2}\right]\right)\\\
-\frac{1}{2}p\cdot\left(\mathbb{E}\left[\beta_{1}v_{1}\mid
E_{1}\right]+\mathbb{E}\left[-\beta_{1}v_{1}\mid E_{3}\right]\right)\\\
+\frac{1}{2}\mathbb{E}\left[\beta_{1}v_{1}\mid
E_{1}\right]+\frac{1}{2}\mathbb{E}\left[-\beta_{1}v_{1}\mid
E_{3}\right].\end{split}$ (19)

Now, we can bound the difference between
$\mathbb{E}\left[\left|\beta_{1}\right|v_{1}\right]=\frac{\mathbb{E}\left[\left|\beta_{1}\right|\right]}{\left\|\nabla
M_{c^{*}}\left(z\right)\right\|_{2}}\nabla M_{c^{*}}\left(z\right)$ and
$\mathbb{E}\left[\phi_{c^{*}}\left(z+\delta Ru\right)u\right]$ using Eq. 19:

$\displaystyle\begin{split}&\left\|\mathbb{E}\left[\phi_{c^{*}}\left(z+\delta
Ru\right)u\right]-\mathbb{E}\left[\left|\beta_{1}\right|v_{1}\right]\right\|_{2}\\\
&\leq\left\|p\cdot\left(\mathbb{E}\left[\phi_{c^{*}}\left(z+\delta
Ru\right)u\mid E_{2}\right]\right)\right.\\\
&-\left.\frac{1}{2}p\cdot\left(\mathbb{E}\left[\beta_{1}v_{1}\mid
E_{1}\right]+\mathbb{E}\left[-\beta_{1}v_{1}\mid
E_{3}\right]\right)\right\|_{2}\\\ &\leq
p\cdot(R+\frac{1}{2}R+\frac{1}{2}R)=2Rp.\end{split}$ (20)

In Ineq. LABEL:eq:expbound, we first substitute the LHS with
$\mathbb{E}\left[\left|\beta_{1}\right|v_{1}\right]=\frac{\mathbb{E}\left[\left|\beta_{1}\right|\right]}{\left\|\nabla
M_{c^{*}}\left(z\right)\right\|_{2}}\nabla M_{c^{*}}\left(z\right)$, then
square both sides, and lastly divide by
$\mathbb{E}\left[\left|\beta_{1}\right|\right]^{2}$ to derive the following

$\displaystyle\left\|\frac{\mathbb{E}\left[\phi_{c^{*}}\left(z+\delta
Ru\right)u\right]}{\mathbb{E}\left[\left|\beta_{1}\right|\right]}-\frac{\nabla
M_{c^{*}}\left(z\right)}{\left\|\nabla
M_{c^{*}}\left(z\right)\right\|_{2}}\right\|_{2}^{2}\leq\left(\frac{2Rp}{\mathbb{E}\left[\left|\beta_{1}\right|\right]}\right)^{2}.$
(21)

Using the property of the angle between two vectors, we obtain the cosine
inequality:

$\cos\angle\left(\mathbb{E}\left[\phi_{c^{*}}\left(z+\delta
Ru\right)u\right],\nabla M_{c^{*}}\left(z\right)\right)\geq
1-\frac{1}{2}\left(\frac{2Rp}{\mathbb{E}\left[\left|\beta_{1}\right|\right]}\right)^{2}.$
(22)

The probability $p$ can be bounded using the fact that
$\left\langle\frac{\nabla M_{c^{*}}\left(z\right)}{\left\|\nabla
M_{c^{*}}\left(z\right)\right\|_{2}},u\right\rangle^{2}$ is a Beta
distribution $\mathcal{B}\left(\frac{1}{2},\frac{d-1}{2}\right)$ :

$\displaystyle\begin{split}p&=\mathbb{P}\left(\left\langle\frac{\nabla
M_{c^{*}}\left(z\right)}{\left\|\nabla
M_{c^{*}}\left(z\right)\right\|_{2}},u\right\rangle^{2}\leq\frac{w^{2}}{\left\|\nabla
M_{c^{*}}\left(z\right)\right\|_{2}^{2}}\right)\\\
&\leq\frac{2w}{\mathcal{B}\left(\frac{1}{2},\frac{d-1}{2}\right)\left\|\nabla
M_{c^{*}}\left(z\right)\right\|_{2}}.\end{split}$ (23)

Substituting the Bound 23 into Eq. 22, we receive a bound for the cosine

$\displaystyle\begin{split}\cos\angle\left(\mathbb{E}\left[\phi_{c^{*}}\left(z+\delta
Ru\right)u\right],\nabla M_{c^{*}}\left(z\right)\right)\\\ \geq
1-\frac{8R^{2}w^{2}}{\left(\mathbb{E}\left[\left|\beta_{1}\right|\right]\right)^{2}\mathcal{B}\left(\frac{1}{2},\frac{d-1}{2}\right)^{2}\left\|\nabla
M_{c^{*}}\left(z\right)\right\|_{2}^{2}}\\\
=1-\frac{2M_{c^{*}}\left(z\right)^{2}R^{2}(d-1)^{2}}{R^{2}\delta^{2}R^{2}\left\|\nabla
M_{c^{*}}\left(z\right)\right\|_{2}^{2}}\\\
=1-\frac{2M_{c^{*}}\left(z\right)^{2}(d-1)^{2}}{\delta^{2}R^{2}\left\|\nabla
M_{c^{*}}\left(z\right)\right\|_{2}^{2}}\end{split}$ (24)

In addition, we have that

$\mathbb{E}\left[\widehat{\nabla
M_{c^{*}}}\left(z,R\right)\right]=\mathbb{E}\left[\phi_{c^{*}}\left(z+\delta
Ru\right)u\right],$ (25)

so plugging it to Ineq. 24, we have the final bound to be

$\displaystyle\begin{split}\cos\angle\left(\mathbb{E}\left[\widehat{\nabla
M_{c^{*}}}\left(z,R\right)\right],\nabla M_{c^{*}}\left(z\right)\right)\\\
\geq
1-\frac{9M_{c^{*}}\left(z\right)^{2}(d-1)^{2}}{2\delta^{2}R^{4}\left\|\nabla
M_{c^{*}}\left(z\right)\right\|_{2}^{2}}.\end{split}$ (26)

Therefore, we can observe that with increasing radius $R$, we will achieve

$\displaystyle\lim_{R\rightarrow\infty}\cos\angle\left(\mathbb{E}\left[\widehat{M_{c^{*}}}\left(z,R\right)\right],\nabla
M_{c^{\star}}\left(z\right)\right)=1,$ (27)

∎

###  A.2 Nonlinear Case

######  Theorem 3.

Let $z$ be an arbitrary point within the target class, i.e. $M_{c^{*}}(z)>0$.
Assume $f$ be a nonlinear classification model with a Lipschitz continuous
gradient in a neighborhood of $z$. Then, the cosine of the angle between
$\mathbb{E}[\widehat{M_{c^{*}}}(z,R)]$ and $\nabla M_{c^{*}}\left(z\right)$ is
bounded by

$\cos\angle\left(\mathbb{E}\left[\widehat{\nabla
M_{c^{*}}}\left(z,R\right)\right],\nabla M_{c^{*}}\left(z\right)\right)\\\
\geq
1-\frac{4M_{c^{*}}\left(z\right)^{2}+L^{2}\delta^{4}R^{4}+4M_{c^{*}}\left(z\right)^{2}L\delta^{2}R^{2}}{2\delta^{2}R^{2}(d-1)^{-2}\left\|\nabla
M_{c^{*}}\left(z\right)\right\|_{2}^{2}}.$ (28)

Therefore, with increasing radius $R$ up to the point of inflection $K$,

$\lim_{R\rightarrow
K}\cos\angle\left(\mathbb{E}\left[\widehat{M_{c^{*}}}\left(z,R\right)\right],\nabla
M_{c^{\star}}\left(z\right)\right)=1,$ (29)

which tells that the alignment between the proposed gradient estimator and the
true gradient $\nabla M_{c^{*}}\left(z\right)$ is increased with greater
radius $R$ until reaching a certain point K. After passing that point of
inflection, the alignment between them branches off.

###### Proof.

Let $Ru$ be a random vector uniformly distributed on the sphere, where $R>0$
is a radius of the sphere. By Taylor’s theorem, for any $\delta\in(0,1)$, we
have that

$\displaystyle\begin{split}&M_{c^{*}}\left(z+\delta Ru\right)\\\
&=M_{c^{*}}\left(z\right)+\delta\nabla
M_{c^{*}}\left(z\right)^{T}Ru+\frac{1}{2}\delta^{2}\nabla^{2}M_{c^{*}}\left(z\right)^{T}R^{2}u.\end{split}$
(30)

Since our function $f$ has a Lipschitz continuous gradient with Lipschitz
constant $L>0$, the following inequality holds

$\displaystyle\begin{split}\left\|\nabla^{2}M_{c^{*}}\left(z\right)u\right\|_{2}&\leq
L\\\ &\iff\\\
\left\|\frac{1}{2}\delta^{2}\nabla^{2}M_{c^{*}}\left(z\right)R^{2}u\right\|_{2}&\leq\frac{1}{2}L\delta^{2}R^{2}\end{split}$
(31)

Recall that

$M_{c^{*}}(z)>0$

and let $w:=\frac{M_{c^{*}}(z)}{\delta R}+\frac{1}{2}L\delta R.$
For the case $\nabla M_{c^{*}}\left(z\right)^{T}u>w$, using Taylor series
expansion and the fact that

$\displaystyle\begin{split}&\delta\nabla
M_{c^{*}}\left(z\right)^{T}Ru+\frac{1}{2}\delta^{2}\nabla^{2}M_{c^{*}}\left(z\right)^{T}R^{2}u\\\
&\geq\delta\nabla M_{c^{*}}\left(z\right)^{T}Ru-\frac{1}{2}L\delta^{2}R^{2}\\\
&>\delta\nabla
M_{c^{*}}\left(z\right)^{T}Ru-\frac{1}{2}L\delta^{2}R^{2}-M_{c^{*}}\left(z\right)\\\
&=\delta R\left(\nabla M_{c^{*}}\left(z\right)^{T}u-w\right)>0,\end{split}$
(32)

we derive that $M_{c^{*}}\left(z+\delta Ru\right)>M_{c^{*}}(z)$.
Similarly, for the case $\nabla M_{c^{*}}\left(x_{t}\right)^{T}u<-w$, using
Taylor expansion and the fact that

$\displaystyle\begin{split}\delta\nabla
M_{c^{*}}\left(z\right)^{T}Ru+\frac{1}{2}\delta^{2}\nabla^{2}M_{c^{*}}\left(z\right)^{T}R^{2}u\\\
\leq\delta\nabla M_{c^{*}}\left(z\right)^{T}Ru+\frac{1}{2}L\delta^{2}R^{2}\\\
<\delta\nabla
M_{c^{*}}\left(z\right)^{T}Ru+\frac{1}{2}L\delta^{2}R^{2}+M_{c^{*}}\left(z\right)\\\
=\delta R\left(\nabla M_{c^{*}}\left(z\right)^{T}u+w\right)<0,\end{split}$
(33)

we obtain that $M_{c^{*}}\left(z+\delta Ru\right)<M_{c^{*}}(z)$.
Therefore, from these two cases, we arrive at

$\displaystyle\phi_{c^{*}}\left(z+\delta
Ru\right)=\left\\{\begin{array}[]{l}0\text{ if }\nabla
M_{c^{*}}\left(z\right)^{T}Ru>w\\\ -1\text{ if }\nabla
M_{c^{*}}\left(z\right)^{T}Ru<-w.\end{array}\right.$ (36)

We define $v_{1}=\nabla S\left(z\right)/\left\|\nabla
M_{c^{*}}\left(z\right)\right\|_{2},v_{2},\ldots,v_{d}$ by expanding the
vector $\nabla M_{c^{*}}\left(z\right)$ to orthogonal bases in
$\mathbb{R}^{d}$. Then, we can write a random vector
$Ru=\sum_{i=1}^{d}\beta_{i}v_{i}$, where $\beta$ is uniformly distributed on
the sphere of radius $R$. We construct an upper cap $E_{1}:=$ $\left\\{\nabla
M_{c^{*}}\left(z\right)^{T}Ru>w\right\\}$, the annulus
$E_{2}:=\left\\{\left|\nabla
M_{c^{*}}\left(z\right)^{T}Ru\right|<w.\right\\}$, and the lower cap
$E_{3}:=\left\\{\nabla M_{c^{*}}\left(z\right)^{T}Ru<-w\right\\}.$ Let
$p:=\mathbb{P}\left(E_{2}\right)$ be the probability of event $E_{2}$, then
$\mathbb{P}\left(E_{1}\right)=\mathbb{P}\left(E_{3}\right)=(1-p)/2.$ For any
$i\neq 1$ by symmetry:

$\displaystyle\mathbb{E}\left[\beta_{i}\mid
E_{1}\right]=\mathbb{E}\left[\beta_{i}\mid E_{3}\right]=0.$ (37)

Then, the expected value of the estimator becomes

$\displaystyle\begin{split}\mathbb{E}\left[\phi_{c^{*}}\left(z+\delta
Ru\right)u\right]=p\cdot\left(\mathbb{E}\left[\phi_{c^{*}}\left(z+\delta
Ru\right)u\mid E_{2}\right]\right)\\\
-\frac{1}{2}p\cdot\left(\mathbb{E}\left[\beta_{1}v_{1}\mid
E_{1}\right]+\mathbb{E}\left[-\beta_{1}v_{1}\mid E_{3}\right]\right)\\\
+\frac{1}{2}\mathbb{E}\left[\beta_{1}v_{1}\mid
E_{1}\right]+\frac{1}{2}\mathbb{E}\left[-\beta_{1}v_{1}\mid
E_{3}\right].\end{split}$ (38)

Now, we can bound the difference between
$\mathbb{E}\left[\left|\beta_{1}\right|v_{1}\right]=\frac{\mathbb{E}\left[\left|\beta_{1}\right|\right]}{\left\|\nabla
M_{c^{*}}\left(z\right)\right\|_{2}}\nabla M_{c^{*}}\left(z\right)$ and
$\mathbb{E}\left[\phi_{c^{*}}\left(z+\delta Ru\right)u\right]$ using Eq. 38:

$\displaystyle\begin{split}&\left\|\mathbb{E}\left[\phi_{c^{*}}\left(z+\delta
Ru\right)u\right]-\mathbb{E}\left[\left|\beta_{1}\right|v_{1}\right]\right\|_{2}\\\
&\leq\left\|p\cdot\left(\mathbb{E}\left[\phi_{c^{*}}\left(z+\delta
Ru\right)u\mid E_{2}\right]\right)\right.\\\
&-\left.\frac{1}{2}p\cdot\left(\mathbb{E}\left[\beta_{1}v_{1}\mid
E_{1}\right]+\mathbb{E}\left[-\beta_{1}v_{1}\mid
E_{3}\right]\right)\right\|_{2}\\\ &\leq
p\cdot(R+\frac{1}{2}R+\frac{1}{2}R)=2Rp.\end{split}$ (39)

In Inequality LABEL:eq:expbound2, we first substitute the LHS with
$\mathbb{E}\left[\left|\beta_{1}\right|v_{1}\right]=\frac{\mathbb{E}\left[\left|\beta_{1}\right|\right]}{\left\|\nabla
M_{c^{*}}\left(z\right)\right\|_{2}}\nabla M_{c^{*}}\left(z\right)$, then
square both sides, and lastly divide by
$\mathbb{E}\left[\left|\beta_{1}\right|\right]^{2}$ to derive the following

$\displaystyle\left\|\frac{\mathbb{E}\left[\phi_{c^{*}}\left(z+\delta
Ru\right)u\right]}{\mathbb{E}\left[\left|\beta_{1}\right|\right]}-\frac{\nabla
M_{c^{*}}\left(z\right)}{\left\|\nabla
M_{c^{*}}\left(z\right)\right\|_{2}}\right\|_{2}^{2}\leq\left(\frac{2Rp}{\mathbb{E}\left[\left|\beta_{1}\right|\right]}\right)^{2}.$
(40)

Using the property of the angle between two vectors, we obtain the cosine
inequality:

$\displaystyle\begin{split}\cos\angle\left(\mathbb{E}\left[\phi_{c^{*}}\left(z+\delta
Ru\right)u\right],\nabla M_{c^{*}}\left(z\right)\right)\\\ \geq
1-\frac{1}{2}\left(\frac{2Rp}{\mathbb{E}\left[\left|\beta_{1}\right|\right]}\right)^{2}.\end{split}$
(41)

The probability $p$ can be bounded using the fact that
$\left\langle\frac{\nabla M_{c^{*}}\left(z\right)}{\left\|\nabla
M_{c^{*}}\left(z\right)\right\|_{2}},u\right\rangle^{2}$ is a Beta
distribution $\mathcal{B}\left(\frac{1}{2},\frac{d-1}{2}\right)$ :

$\displaystyle\begin{split}p&=\mathbb{P}\left(\left\langle\frac{\nabla
M_{c^{*}}\left(z\right)}{\left\|\nabla
M_{c^{*}}\left(z\right)\right\|_{2}},u\right\rangle^{2}\leq\frac{w^{2}}{\left\|\nabla
M_{c^{*}}\left(z\right)\right\|_{2}^{2}}\right)\\\
&\leq\frac{2w}{\mathcal{B}\left(\frac{1}{2},\frac{d-1}{2}\right)\left\|\nabla
M_{c^{*}}\left(z\right)\right\|_{2}}.\end{split}$ (42)

Substituting the Bound 42 into equality 41, we receive a bound for the cosine

$\displaystyle\begin{split}&\cos\angle\left(\mathbb{E}\left[\phi_{c^{*}}\left(z+\delta
Ru\right)u\right],\nabla M_{c^{*}}\left(z\right)\right)\\\ &\geq
1-\frac{8R^{2}w^{2}}{\left(\mathbb{E}\left[\left|\beta_{1}\right|\right]\right)^{2}\mathcal{B}\left(\frac{1}{2},\frac{d-1}{2}\right)^{2}\left\|\nabla
M_{c^{*}}\left(z\right)\right\|_{2}^{2}}\\\ &\geq
1-\frac{8M_{c^{*}}\left(z\right)^{2}+2L^{2}\delta^{4}R^{4}+8M_{c^{*}}\left(z\right)^{2}L\delta^{2}R^{2}}{\delta^{2}\left(\mathbb{E}\left[\left|\beta_{1}\right|\right]\right)^{2}\mathcal{B}\left(\frac{1}{2},\frac{d-1}{2}\right)^{2}\left\|\nabla
M_{c^{*}}\left(z\right)\right\|_{2}^{2}}\\\ &\geq
1-\frac{4M_{c^{*}}\left(z\right)^{2}+L^{2}\delta^{4}R^{4}+4M_{c^{*}}\left(z\right)^{2}L\delta^{2}R^{2}}{2\delta^{2}R^{2}(d-1)^{-2}\left\|\nabla
M_{c^{*}}\left(z\right)\right\|_{2}^{2}}.\end{split}$ (43)

In addition, we have that

$\mathbb{E}\left[\widehat{\nabla
M_{c^{*}}}\left(z,R\right)\right]=\mathbb{E}\left[\phi_{c^{*}}\left(z+\delta
Ru\right)u\right],$ (44)

so plugging it to inequality LABEL:eq:cosbound2, we have the final bound to be

$\displaystyle\begin{split}&\cos\angle\left(\mathbb{E}\left[\widehat{\nabla
M_{c^{*}}}\left(z,R\right)\right],\nabla M_{c^{*}}\left(z\right)\right)\\\
&\geq
1-\frac{4M_{c^{*}}\left(z\right)^{2}+L^{2}\delta^{4}R^{4}+4M_{c^{*}}\left(z\right)^{2}L\delta^{2}R^{2}}{2\delta^{2}R^{2}(d-1)^{-2}\left\|\nabla
M_{c^{*}}\left(z\right)\right\|_{2}^{2}}.\end{split}$ (45)

Therefore, we can observe that with increasing radius $R$, we will achieve

$\displaystyle\lim_{R\rightarrow
K}\cos\angle\left(\mathbb{E}\left[\widehat{M_{c^{*}}}\left(z,R\right)\right],\nabla
M_{c^{\star}}\left(z\right)\right)=1,$ (46)

∎

##  Appendix B Experiments

###  B.1 Attacking MNIST

We extend our experiments to include tasks other than facial recognition.
Particularly, we perform our attack on MNIST dataset. We use identical attack
model to the main experiments, in which the attacker only gets access to the
target model’s decision. We also assume that that the attacker has access to a
prior knowledge (public dataset). In our case, these are the digits from “5”
to “9”. The attacker’s goal is to infer information on the private dataset
(i.e., the digits from “0” to “4”). This is not trivial since the public
knowledge available to the attacker contains only 5 classes which makes it
harder for the attacker to generalize to other classes. Nevertheless, brep-MI
was able to successfully attack 4 out of the 5 private classes.

Similar to the main experiments, we train two models. The target model is a
network with 2 CNN layers followed by two linear layers and the evaluation
classifier has 3 CNN layers followed by two linear layers. As shown in Fig.5,
the initial image generated by GAN for class “0” resembles class “6”. This is
expected, since the attacker has only knowledge of classes from “5” to “9”.
The final output however, looks more similar to “0”. Similar behavior can be
seen when attacking class “4”. For classes “2” and “3”, the initial images
were noises, yet the we were still able to successfully attack class “3”.

Figure 5: Performing brep-MI on MNIST dataset. The output is shows for each
private label (digit).


††thanks: Correspond to<EMAIL_ADDRESS>These authors contributed
equally to this work ††thanks: Correspond to<EMAIL_ADDRESS>
# A van der Waals Interface Hosting Two Groups of Magnetic Skyrmions

Yingying Wu  Francis Bitter Magnet Laboratory and Plasma Science and Fusion
Center, Massachusetts Institute of Technology, Cambridge, MA 02139, USA  ,†
Brian Francisco  Department of Physics and Astronomy, University of
California, Riverside, CA 92521, USA  Wei Wang  Key Laboratory of Flexible
Electronics & Institute of Advanced Materials, Jiangsu National Synergetic
Innovation Center for Advanced Materials, Nanjing Tech University, Nanjing
211816, China  Yu Zhang  Caihua Wan  Xiufen Han  Institute of Physics, Chinese
Academy of Sciences, Beijing, 100190, China  Hang Chi  Francis Bitter Magnet
Laboratory and Plasma Science and Fusion Center, Massachusetts Institute of
Technology, Cambridge, MA 02139, USA  U.S. Army CCDC Army Research Laboratory,
Adelphi, MD 20783, USA  Yasen Hou  Alessandro Lodesani  Francis Bitter Magnet
Laboratory and Plasma Science and Fusion Center, Massachusetts Institute of
Technology, Cambridge, MA 02139, USA  Yong-tao Cui  Department of Physics and
Astronomy, University of California, Riverside, CA 92521, USA  Kang L. Wang
Department of Electrical and Computer Engineering, University of California,
Los Angeles, Los Angeles, CA 90095, USA.  Jagadeesh S. Moodera  Francis Bitter
Magnet Laboratory and Plasma Science and Fusion Center, Massachusetts
Institute of Technology, Cambridge, MA 02139, USA  Physics Department,
Massachusetts Institute of Technology, Cambridge, MA 02139, USA

###### Abstract

Multiple magnetic skyrmion phases add an additional degree of freedom for
skyrmion based ultrahigh-density spin memory devices. Extending the field to
two-dimensional van der Waals magnets is a rewarding challenge, where the
realizable degree of freedoms (e.g. thickness, twisting angle and electrical
gating) and high skyrmion density result in intriguing new properties and
enhanced functionality. We report a van der Waals interface, formed by two 2D
ferromagnets Cr2Ge2Te6 and Fe3GeTe2 with a Curie temperature of $\sim$ 65 K
and $\sim$ 205 K, respectively, hosting two groups of magnetic skyrmions. Two
sets of topological Hall effect are observed below 60 K when Cr2Ge2Te6 is
magnetically ordered. These two groups of skyrmions are directly imaged using
magnetic force microscopy. Interestingly, the magnetic skyrmions persist in
the heterostructure in the remanent state with zero applied magnetic field.
Our results are promising for the realization of skyrmionic devices based on
van der Waals heterostructures hosting multiple skyrmion phases.

Two dimensional (2D) magnetism was recently discovered in van der Waals (vdW)
ferromagnets1; 2; 3 and antiferromagnets4; 5; 6, providing unprecedented
opportunities for exploring magnetism, and towards spintronic applications in
the 2D limit7; 8. Among all the interface engineered heterostructures based on
vdW layered systems, magnetic proximity effect is integral to manipulating
spintronic9; 10; 11, superconducting12; 13; 14 and topological15; 16; 17
phenomena. Topological magnetic skrymions have been well studied due to their
nontrivial topology, which lead to many interesting fundamental and dynamical
properties18; 19; 20. These have been reported mostly for non-centrosymmteric
single crystals21; 22; 23, ultrathin epitaxial system24; 25, and magnetic
multilayers26; 27; 28; 29; 30. Recently Néel-type skyrmions were observed in a
vdW ferromagnet interfaced with an oxidized layer31 or a transition metal
dichalcogenide32 with a control of the skyrmion phase through tuning of the
ferromagnet thickness. Furthermore, with a variety of vdW magnets, skrymions
phase could be created in their new interfaces with unique properties.

Materials hosting multiple skyrmion phases add richness to the field, with an
additional degree of freedom in designing devices. Such a degree of freedom
adds value as a means to lower the error rate in racetrack memories with
skyrmions as information carriers33; 34. Multiple skyrmion phases have been
reported in FeGe grown on Si(111) at low temperatures35 and in Ir/Fe/Co/Pt
multilayers33. Different from these thin film systems, vdW heterostructures
possess well defined and ordered atomic layer range of thicknesses, atomically
sharp interfaces and easy electrical field control. Forming a heterostructure
of two vdW ferromagnets allows breaking of inversion symmetry at the
interface19, with skyrmions expected inside both these two ferromagnetic
layers in the context of strong spin orbit coupling. So far, there are no
direct observations of the multi groups of magnetic skyrmions at a vdW
interface.

In this work, we demonstrate a vdW interface, i.e. Fe3GeTe2/Cr2Ge2Te6
(FGT/CGT) interface, can host two groups of magnetic skyrmions. The initial
signature for this comes from the observation of the topological Hall effect
(THE). Below the Curie temperature $T_{\textrm{C}}$ ($\sim$ 65 K) of CGT, Hall
resistivity shows two sets of kinks, due to the magnetic skyrmion phase
formation on CGT and FGT sides. Above 60 K, there is only one set of kinks
seen in the Hall resistivity, likely arising from the skyrmion phase on the
FGT side. To ascertain and understand the origin of THE, magnetic force
microscopy was used to image the skyrmion lattice directly. These observations
could provide a new platform for designing a robust skyrmion-based memory and
computing architectures by coding the information in multiple groups of
skyrmions.

## Ferromagnetic Fe3GeTe2 and Cr2Ge2Te6 thin layers

CGT is a soft, 2D layered ferromagnetic crystal, with rhombohedral structure
and vdW stacking of hexagonal layers. As illustrated in Fig. 1a, a layer of Cr
atoms is sandwiched between two Ge-Te layers, forming a monolayer of CGT. Its
lattice parameters are $a$ = 6.826 Å and $c$ = 20.531 Å. Among the 2D magnets,
FGT is a ferromagnet with a strong perpendicular magnetic anisotropy. Bulk FGT
consists of weakly bonded Fe3Ge layers that alternate with two Te layers with
a space group P63mmc as shown in Fig. 1b with lattice parameters $a$ = 3.991 Å
and $c$ = 16.333 Å36. For both pristine CGT and FGT crystals, the inversion
symmetry is preserved inside the bulk. However, by forming an interface
between these two materials, the inversion symmetry is broken. With spin-orbit
coupling present, Dzyaloshinskii-Moriya interaction (DMI) is induced at the
interface as shown in Fig. 1c, both at FGT and CGT interfacial layers. The DMI
between two atomic spins ${\bf\it S}_{1}$ and ${\bf\it S}_{2}$ can be
expressed as: $H_{\textrm{DM}}=-{\bf\it D}_{12}\cdot{\bf\it
S}_{1}\times{\bf\it S}_{2}$.

The crystal structure of grown CGT crystal was analyzed by using X-ray
diffraction studies as depicted in Fig. 1d. Typical peaks corresponding to the
(003), (006) and (0012) planes in CGT were observed. These peaks indicate that
the layered CGT crystal with good quality is achieved. The $T_{\textrm{C}}$
for CGT was around 65 K, as seen from magnetization measurement using a
magnetometer(Fig. 1e). The ferromagnetic order in CGT develops at low
temperatures upon cooling down both without and with an out-of-plane magnetic
field. The magnetic hysteresis is shown in Fig. 1f, where the ferromagnetism
disappears above 65 K. Compared to CGT, FGT has a much higher $T_{\textrm{C}}$
of around 200 K, as shown in Fig. 1g.

## Two sets of THE signals

THE can arise from the accumulated Berry phase acquired by electrons in the
adiabatic limit as they encounter a skyrmion along their path. However, recent
experiments offered an alternative interpretation of the topological Hal
effect reported in topological insulator heterostructures; they attribute the
topological Hall signals to the overlapping of two anomalous Hall signals with
opposite signs. A similar reasoning was used to explain the topological Hall-
like signatures (one peak and one dip) in SrTiO3/SrRuO3/SrTiO337
heterostructures. In such a scenario, the competing anomalous Hall signals
contributing to topological Hall-like signatures originate either from the
coexisting surface and bulk magnetic phases in the magnetic topological
insulator system 38, or two interfaces in the heterostructures. As opposed to
previous systems, this current FGT/CGT bilayer system might facilitate the
demonstration of THE for three reasons: (1) only one interface is formed
between the two materials; (2) two sets of kinks, a signature for THE at both
sides of an interface, are observed when the temperature is lower than the
$T_{\textrm{C}}$ of CGT; and (3) skyrmions are imaged directly by magnetic
force microscopy (MFM).

In the transport measurements, CGT/FGT thin films were transferred onto
prepared Hall-bar bottom electrodes under a microscope (see Supplementary
Information Section A for device images) inside the glove box (N2 atmosphere),
and the resistivity was measured with lock-in technique. The CGT layers for
the transport measurement have a thickness between 30 nm and 40 nm, which is
relatively insulating compared to metallic FGT layers. Magnetic field was
applied perpendicular to the sample plane. As shown in Fig. 2a, 20-layer (20L)
FGT exhibits square hysteresis loop when the temperature is below 200 K,
demonstrating a strong perpendicular out-of-plane magnetic anisotropy. When
FGT thickness was reduced to 5L, interface coupling between FGT and CGT gave
rise to pronounced peaks and dips near the magnetic transition edge, the
signature of THE. Different from previously reported systems, two sets of THEs
(as indicated by blue and orange circles in Fig. 2b) are observed in this
heterostructure of two ferromagnets.

The topological Hall resistivity value is related to carrier density $n$, and
an effective field $B_{\textrm{eff}}$ generated by skyrmion lattice
($\rho_{\textrm{xy}}^{\textrm{THE}}\sim\frac{1}{ne}B_{\textrm{eff}}$)39. The
different magnitude of the topological Hall resistivity value indicates two
groups of skyrmions with different sizes. These two sets of THEs coexist when
the temperature is below the $T_{\textrm{C}}$ of CGT. One set indicated by the
orange circles vanishes at 60 K and above, is attributable to the
disappearance of ferromagnetic order in CGT with increasing temperature. It
may be noted that, this THE is only observed in CGT/5L FGT sample. The absence
of THE in this heterostructure with thicker FGT can be due to the high carrier
density. With larger carrier density in thicker FGT samples, the topological
Hall resistivity is dwarfed in comparison. For example, for 20L FGT at 20 K,
the estimated topological Hall resistivity contribution is less than
$10^{-3}\,\Omega$, assuming a skyrmion lattice with a size of 100 nm. This
resistivity value is more than two orders of magnitude smaller than the
anomalous Hall resistivity, and thus tends to be buried in the anomalous Hall
signals.

## Magnetic skyrmions directly imaged by MFM

To further examine the origin of THE arising at the FGT/CGT interface, we
performed low-temperature MFM measurements under applied magnetic fields. In
MFM, the magnetic stray field originating from nanoscale features on the
surface of a sample is detected by a scanning nanoscale magnetic tip (Fig.
3a). Note that in our topological Hall resistivity, two sets of kinks were
observed, attributable to magnetic skyrmions on FGT side and CGT side,
respectively (Fig. 3b). In our MFM measurement geometry, the MFM signal
characterizes the second order derivative of the out-of-plane component of the
stray magnetic field, i.e., d${}^{2}H_{z}/dz^{2}$. For samples with out-of-
plane magnetization, MFM responds both inside individual domains and at the
domain walls. Fig. 3c shows a series of MFM images taken on CGT/FGT device
(same device as in Fig. 2b) with applied perpendicular magnetic fields at 20
K. As the field increases from zero, the image shows a labyrinth domain
feature. Upon increasing the field to 160 mT, skyrmionic features started to
form as indicated by the yellow circles in Fig. 3c. At 200 mT, skyrmion
lattice with a size of $\sim$130 nm was directly imaged. This skyrmion lattice
disappears and the sample enters into a uniform single domain as the field is
increased to 250 mT (see Supplementary Information Section B for skyrmion
lattice at larger magnetic fields). This observed magnetic skyrmions are
expected to occur at the interface close to CGT side for three reasons: (1)
with CGT on top of FGT, MFM signals would be dominated by the magnetic fields
generated by the CGT layer, (2) the magnetic signal from 5L FGT under
multilayer CGT is too weak or screened to be observed, and (3) the formation
of magnetic skyrmion lattices exactly matches the magnetic field which lead to
the development of THE on CGT side (the dip at a positive magnetic field in
Fig. 2b and 3b).

With a reversed sample structure, i.e., FGT on top of CGT, MFM measurements
show domain images in FGT side in Fig. 3d. At 100 K, FGT shows similar
labyrinth domain feature at 25 mT but with a larger domain width, compared to
CGT case at 20 K. When the field was increasing, skyrmions started to appear,
as indicated by the yellow circles. The skyrmion size was estimated to be
$\sim$ 180 nm (100 K and 60 mT) and decreasing as the temperature went down.
Magnetic skyrmions lattice at a lower temperature can be found in
Supplementary Information Section C.

The skyrmion lattice in CGT/FGT develops with the magnetic field. When
sweeping from a negative field (-2 T) which is large enough to saturate CGT
magnetization at 20 K, the skyrmions develop at $\sim$ -50 mT (Fig. 4a).
Interestingly, the magnetic skyrmions are seen to persist when the applied
magnetic field is turned off, in the remanent state. Emergence of magnetic
skyrmion at 0 T magnetic field has been predicted to be a possible signature
of Moiré skyrmions40. The real reason and mechanism of zero-field magnetic
skyrmions is not clear in our case at present. One should also consider the
role of the exchange coupling between CGT and FGT, where FGT provides an
effective field at the interface. When the magnetic field was further
increased to 120 mT, skyrmion lattice emerged forming a hexagonal pattern.
Further increasing the magnetic field to 140 mT, the skyrmion images switched
to opposite contrast due to the flip of tip magnetization by the external
magnetic field.

The skyrmion size was extracted from THE and the skyrmion images by MFM
measurement (Fig. 4b). The skyrmion size for both FGT and CGT side decreases
with decreasing temperature. The error bars for data points in the MFM
measurement (dots and squares with dashed lines) are due to the variation of
skyrmion size under applied magnetic field. The skyrmion size extracted from
the THE (dots and squares with solid lines) offers an order of magnitude
estimate (details of the size extraction can be found in Supplementary
Information Section D), which is around one order of magnitude smaller than
that from MFM measurement.

For an interface formed between CGT and FGT, we can also estimate the DMI
energy from the strip domain width. The DMI in FGT is estimated to be,
$|D|=0.31\pm 0.13$ mJ m-2 (details can be found in Supplementary Information
Section E), a value similar to that reported in WTe2/FGT system10, a vdW
heterostructure hosting skyrmion lattice (difference for the skyrmions in the
heterostructure and skyrmion bubbles in the pristine centrosymmetric magnets
can be found in Supplementary Information Section F).

## Summary and outlook

We explored the ultrathin FGT/CGT heterostructures as a platform for hosting
small and tunable two groups of magnetic skyrmions. In this system, the
inversion symmetry breaking near the FGT/CGT interface gives rise to an
emergent DMI, thereby creating robust magnetic skyrmions. By harnessing the
interface between two vdW ferromagnets, we achieved magnetic skyrmions on two
sides of an interface.

The multi groups of skyrmions in FGT/CGT heterostructures allow versatility
when designing and fabricating skyrmion-based devices. Due to the atomically
thin nature of vdW materials and their easy control by electrical gating, one
can foresee manipulating the skyrmions on each side with dual electric gating
by which the skyrmion phase can be turned on or off. Furthermore, the twist
angle between FGT and CGT may be tuned, where a new functionality could be
added for manipulating skyrmions in atomically thin vdW heterostructures. With
an accurate determination of twist angle between FGT and CGT layers, magnetic
skyrmions from DMI or Moiré pattern may be differentiated. This versatile
advantage offers a fertile playground for exploring emergent phenomena arising
from hybrid interfaces, leading to magnetic skyrmions with additional
functionalities.

## Methods

### Growth of Fe3GeTe2 and Cr2Ge2Te6

High quality single crystals of FGT were grown by a typical chemical vapor
transport (CVT) method. The stoichiometric amounts of high purity elements
(99.999% Fe, 99.999% Ge and 99.999% Te from Alfa Aesar) along with 2 mg cm-3
iodines the transport agent were placed in a quartz ampoule and sealed under
vacuum. The ampoule was further placed in a horizontal 2-zone furnace over a
temperature gradient from 750∘C to 700∘C and kept at that condition for 2
weeks. The single crystals with a tabular shape in the $ab$-plane grew in the
low-temperature zone.

High quality CGT single crystals were synthesized from high purity elements
(99.99% Cr, 99.99% Ge and > 99.99% Te from Alfa Aesar). A mixture of materials
with an optimized ratio of Cr : Ge : Te equivalent to 10 : 13.5 : 76.5, was
sealed in evacuated quartz ampoules and heated at 1000∘C for one day, followed
by slow cooling to 450∘C for a period of 90 hours.

### MFM measurement

The MFM measurements were performed in a home-built low-temperature scanning
probe microscope using commercial MFM probes (Bruker MESPV2) with a spring
constant of 3 N m-1, a resonance frequency at $\sim$75 kHz, and a Co-Cr
magnetic coating. MFM images were taken in a constant height mode with the tip
scanning plane at $\sim$100 nm above the sample surface. The MFM signal, the
change in the resonance frequency, is measured by a Nanonis SPM Controller
using a phase-lock loop. The magnetic moment of the probe is nominally in the
order of $10^{-16}$ A m2.

### Magnetotransport measurement

Hall-bar bottom electrodes with dimension of $3.5\,\mu\textrm{m}\times
0.5\,\mu\textrm{m}$ were fabricated with e-beam lithography for the transport
measurements. Systematically altering experimental variables such as
temperature and magnetic field, in addition to multiple lock-in amplifiers and
sourcemeter enabled comprehensive and high-sensitivity transport measurements
in all the devices, using a commercial PPMS from Quantum Design (2 K, $\pm$
9T).

## Data availability Statement

The data that support the findings of this study are available from the
corresponding authors upon request.

## References

## References

  * (1) Gong, C. _et al._ Discovery of intrinsic ferromagnetism in two-dimensional van der Waals crystals.  _Nature_ 546, 265–269 (2017).
  * (2) Deng, Y. _et al._ Gate-tunable room-temperature ferromagnetism in two-dimensional Fe3GeTe2.  _Nature_ 563, 94–99 (2018).
  * (3) Fei, Z. _et al._ Two-dimensional itinerant ferromagnetism in atomically thin Fe3GeTe2.  _Nat. Mater._ 17, 778–782 (2018).
  * (4) Huang, B. _et al._ Layer-dependent ferromagnetism in a van der Waals crystal down to the monolayer limit.  _Nature_ 546, 270–273 (2017).
  * (5) Song, T. _et al._ Giant tunneling magnetoresistance in spin-filter van der Waals heterostructures.  _Science_ 360, 1214–1218 (2018).
  * (6) Deng, Y. _et al._ Quantum anomalous Hall effect in intrinsic magnetic topological insulator MnBi2Te4.  _Science_ 367, 895–900 (2020).
  * (7) Burch, K. S., Mandrus, D. & Park, J.-G.  Magnetism in two-dimensional van der Waals materials.  _Nature_ 563, 47–52 (2018).
  * (8) Avsar, A. _et al._ Colloquium: Spintronics in graphene and other two-dimensional materials.  _Rev. Mod. Phys._ 92, 021003 (2020).
  * (9) Tang, C., Zhang, Z., Lai, S., Tan, Q. & Gao, W.-b.  Magnetic proximity effect in graphene/CrBr3 van der Waals heterostructures.  _Adv. Mater._ 32, 1908498 (2020).
  * (10) Wu, Y. _et al._ Large exchange splitting in monolayer graphene magnetized by an antiferromagnet.  _Nat. Electron._ 3, 604–611 (2020).
  * (11) Dankert, A. & Dash, S. P.  Electrical gate control of spin current in van der Waals heterostructures at room temperature.  _Nat. Commun._ 8, 1–6 (2017).
  * (12) Kim, M. _et al._ Strong proximity Josephson coupling in vertically stacked NbSe2-graphene-NbSe2 van der Waals junctions.  _Nano Lett._ 17, 6125–6130 (2017).
  * (13) Wu, Y. _et al._ Induced Ising spin-orbit interaction in metallic thin films on monolayer WSe2.  _Phys. Rev. B_ 99, 121406 (2019).
  * (14) Lüpke, F. _et al._ Proximity-induced superconducting gap in the quantum spin Hall edge state of monolayer WTe2.  _Nat. Phys._ 16, 526–530 (2020).
  * (15) Kezilebieke, S. _et al._ Topological superconductivity in a van der Waals heterostructure.  _Nature_ 588, 424–428 (2020).
  * (16) Island, J. _et al._ Spin-orbit-driven band inversion in bilayer graphene by the van der Waals proximity effect.  _Nature_ 571, 85–89 (2019).
  * (17) Hellman, F. _et al._ Interface-induced phenomena in magnetism.  _Rev. Mod. Phys._ 89, 025006 (2017).
  * (18) Nagaosa, N. & Tokura, Y.  Topological properties and dynamics of magnetic skyrmions.  _Nat. Nanotech._ 8, 899–911 (2013).
  * (19) Fert, A., Cros, V. & Sampaio, J.  Skyrmions on the track.  _Nat. Nanotech._ 8, 152–156 (2013).
  * (20) Fert, A., Reyren, N. & Cros, V.  Magnetic skyrmions: advances in physics and potential applications.  _Nat. Rev. Mater._ 2, 1–15 (2017).
  * (21) Mühlbauer, S. _et al._ Skyrmion lattice in a chiral magnet.  _Science_ 323, 915–919 (2009).
  * (22) Yu, X. _et al._ Real-space observation of a two-dimensional skyrmion crystal.  _Nature_ 465, 901–904 (2010).
  * (23) Jonietz, F. _et al._ Spin transfer torques in MnSi at ultralow current densities.  _Science_ 330, 1648–1651 (2010).
  * (24) Wang, L. _et al._ Ferroelectrically tunable magnetic skyrmions in ultrathin oxide heterostructures.  _Nat. Mater._ 17, 1087–1094 (2018).
  * (25) Woo, S. _et al._ Observation of room-temperature magnetic skyrmions and their current-driven dynamics in ultrathin metallic ferromagnets.  _Nat. Mater._ 15, 501–506 (2016).
  * (26) Jiang, W. _et al._ Blowing magnetic skyrmion bubbles.  _Science_ 349, 283–286 (2015).
  * (27) Li, W. _et al._ Anatomy of skyrmionic textures in magnetic multilayers.  _Adv. Mater._ 31, 1807683 (2019).
  * (28) Zeissler, K. _et al._ Diameter-independent skyrmion Hall angle observed in chiral magnetic multilayers.  _Nat. Commun._ 11, 1–11 (2020).
  * (29) Wiesendanger, R.  Nanoscale magnetic skyrmions in metallic films and multilayers: a new twist for spintronics.  _Nat. Rev. Mater._ 1, 1–11 (2016).
  * (30) Maccariello, D. _et al._ Electrical detection of single magnetic skyrmions in metallic multilayers at room temperature.  _Nat. Nanotech._ 13, 233–237 (2018).
  * (31) Park, T.-E. _et al._ Néel-type skyrmions and their current-induced motion in van der Waals ferromagnet-based heterostructures.  _Phys. Rev. B_ 103, 104410 (2021).
  * (32) Wu, Y. _et al._ Néel-type skyrmion in WTe2/Fe3GeTe2 van der Waals heterostructure.  _Nat. Commun._ 11, 1–6 (2020).
  * (33) Mandru, A.-O. _et al._ Coexistence of distinct skyrmion phases observed in hybrid ferromagnetic/ferrimagnetic multilayers.  _Nat. Commun._ 11, 1–7 (2020).
  * (34) Suess, D., Vogler, C., Bruckner, F., Heistracher, P. & Abert, C.  A repulsive skyrmion chain as a guiding track for a racetrack memory.  _AIP Adv._ 8, 115301 (2018).
  * (35) Ahmed, A. S. _et al._ Chiral bobbers and skyrmions in epitaxial FeGe/Si (111) films.  _Phys. Rev. Mater._ 2, 041401 (2018).
  * (36) Yi, J. _et al._ Competing antiferromagnetism in a quasi-2D itinerant ferromagnet: Fe3GeTe2.  _2D Mater._ 4, 011005 (2016).
  * (37) Kimbell, G. _et al._ Two-channel anomalous Hall effect in SrRuO3.  _Phys. Rev. Mater._ 4, 054414 (2020).
  * (38) Fijalkowski, K. M. _et al._ Coexistence of surface and bulk ferromagnetism mimics skyrmion Hall effect in a topological insulator.  _Phys. Rev. X_ 10, 011012 (2020).
  * (39) Neubauer, A. _et al._ Topological Hall effect in the A phase of MnSi.  _Phys. Rev. Lett._ 102, 186602 (2009).
  * (40) Tong, Q., Liu, F., Xiao, J. & Yao, W.  Skyrmions in the Moiré of van der Waals 2D magnets.  _Nano Lett._ 18, 7194–7199 (2018).

## Acknowledgements:

The authors are grateful for discussion with Gen Yin from Georgetown
University. The transport measurements in this work are supported by the Army
Research Office (ARO) program under contract W911NF-15-1-10561, National
Science Foundation (NSF) with Award Nos. 1935362 and 1909416, CIQM-NSF
DMR-1231319, and ARO grant W911NF-20-2-0061. H. C. was sponsored by the Army
Research Laboratory under cooperative agreement no. W911NF-19-2-0015. We are
also grateful to the support from the NSF (DMR-1411085) and Department of
Energy, Office of Science under Award No. DE-SC0020221. MFM measurements are
supported by NSF DMR-2145735.

## Author Contributions

Y.W. conceived and designed the experiment. K.L.W. and J.S.M. supervised the
project. W.W. grew CGT and FGT bulk crystals. Y.W. performed the fabrication
and carried out the transport measurements. B.F. and Y.C. implemented the MFM
measurements. Y.Z., C.W. and X.H. prepared the bottom metal electrodes. H.C.,
Y.H. and A.L. helped with experimental setup. Y.W. and J.S.M. wrote the
manuscript with contributions from all the authors.

## Competing interests

The authors declare no competing interests.

Figure 1: Structure and magnetic properties in FGT and CGT crystals. a,
Crystal structure of CGT. Side view and top view for the rhombohedral
structure. b, Crystal structure of FGT. c, DMI is induced at the interface of
CGT and FGT. d, X-ray diffraction pattern of CGT indicating high quality
crystal. e, The temperature dependent magnetization profile of CGT under zero-
field-cool and field-cool (out-of-plane applied magnetic field $H$ = 10 Oe)
conditions, revealing a $T_{\textrm{C}}$ of $\sim$ 65 K. f, Magnetic
hysteresis profile of CGT at selected temperatures with applied magnetic field
$H\,//\,c$-axis displaying an anisotropy field of approximately 220 mT at 20
K. g, The temperature dependent magnetization profile of FGT under zero-field
cooling condition, indicating a $T_{\textrm{C}}$ of $\sim$ 205 K.  Figure 2:
Topological Hall effect in the Cr2Ge2Te6/Fe3GeTe2 heterostructure. Hall
resistivity (shifted for clarity) of a CGT/FGT heterostructure with different
FGT thickness of a, 20L, and b, 5L. A magnetic field was applied perpendicular
to the sample plane, i.e., along the $c$-axis. A square hysteresis loop
indicates the perpendicular magnetic anisotropy. Peaks and dips appear in
heterostructure with 5L FGT, signifying the existence of topological Hall
effect.  Figure 3: Skyrmions at both sides of Fe3GeTe2/Cr2Ge2Te6 interface
imaged by MFM. a, Schematic illustration of MFM measurements. b, Two sets of
THE signals observed for temperatures lower than 60 K. The blue circles
signifying THE on FGT side and orange circles signifying THE on CGT side. The
peak and dip features of THE resistivity indicating the skyrmions with
opposite polarity. c, Skyrmion lattice observed on CGT side with a magnetic
field of 200 mT and a temperature of 20 K. Scale bar: 1 $\mu$m. d, Skyrmions
on FGT side start to appear with a magnetic field of 50 mT at a temperature of
100 K. Scale bar: 1 $\mu$m.  Figure 4: Magnetic skyrmions showing up as a
function of magnetic field (even at zero field) and skyrmion size dependence
on the temperature. a, Skyrmion lattice on CGT side under various applied
magnetic fields at 20 K after subjecting to -2 T. Scale bar: 1 $\mu$m. b,
Skyrmion size extracted from topological Hall effect (an order-of-magnitude
estimate) and from magnetic force microscopy images. The date points with
dashed lines are from magnetic force microscopy measurements and the data
points with solid lines are from topological Hall effects.