/csteps/fill color=black /csteps/inner color=white

# Ownership Protection of Generative Adversarial Networks

Hailong Hu
SnT, University of Luxembourg  Jun Pang
FSTM & SnT, University of Luxembourg

###### Abstract

Generative adversarial networks (GANs) have shown remarkable success in image
synthesis, making GAN models themselves commercially valuable to legitimate
model owners. Therefore, it is critical to technically protect the
intellectual property of GANs. Prior works need to tamper with the training
set or training process, and they are not robust to emerging model extraction
attacks. In this paper, we propose a new ownership protection method based on
the common characteristics of a target model and its stolen models. Our method
can be directly applicable to all well-trained GANs as it does not require
retraining target models. Extensive experimental results show that our new
method can achieve the best protection performance, compared to the state-of-
the-art methods. Finally, we demonstrate the effectiveness of our method with
respect to the number of generations of model extraction attacks, the number
of generated samples, different datasets, as well as adaptive attacks.

##  1 Introduction

Generative adversarial networks (GANs), as one of the most successful
generative models, have already exerted revolutionary influences on many
application domains, such as image synthesis [10, 25, 17, 3, 28], image
editing [44, 31, 5, 36], and image translation [45, 37, 22, 14]. However,
building a well-trained state-of-the-art GAN model is not straightforward. It
usually requires the complicated and exhausting process of data collection,
expert-level knowledge in model architecture design, elaborate hyperparameter
tuning, and extensive computing resources. Therefore, a high-quality GAN model
is incredibly costly and should be regarded as the intellectual property of
the model owner.

As GAN models are valuable, this simultaneously incentivizes adversaries to
steal these models in various ways. On the one hand, adversaries can
physically steal a GAN model via malware infection or insider attacks [29]. An
insider attack can directly copy the victim model through those who are
authorized to access the full model. As a result, the stolen model is totally
the same as the victim model. On the other hand, adversaries can functionally
steal a GAN model via model extraction attacks [13]. This threat exists
because an increasing number of technology companies provide Machine Learning
as a Service (MLaaS) to their customers, such as Amazon AWS, Google Cloud, and
Microsoft Azure. A model extraction attack enables adversaries to obtain a
substitute model via exposed interfaces. As a consequence, the stolen model is
functionally similar to the victim model. In general, both physical stealing
attacks and model extraction attacks seriously jeopardize the intellectual
property of legitimate model owners. It is paramount to develop ownership
protection methods to safeguard the intellectual property of a GAN model.

Despite confronting these threats, research on ownership protection for GANs
is somehow much less explored. There is one work [26] that proposes a
watermark-based method for protecting ownership of GANs, but it needs to
utilize extra loss functions to retrain the model. More importantly, it only
considers physical stealing, and emerging model extraction attacks are taken
into consideration.

In this paper, we develop a new ownership protection method for GANs, which
can protect ownership on both physical stealing and model extraction attacks.
Our method claims the ownership of a GAN by leveraging common characteristics
of a target model and its stolen models. The rationale for our method is that
stolen models are derived from the target model while honest models are not.
Thus, these common characteristics can be leveraged to differentiate stolen
models from honest models. More specifically, we utilize generated samples
from GANs to build a discriminative classifier to learn these characteristics.
This is because the objective of a GAN is to learn the distribution of a
training dataset and the learned implicit distribution of a GAN can be
represented by these generated samples [2] (see Section 4.3).

We comprehensively evaluate our new method by comparing it with two state-of-
the-art works: watermark-based method (abbreviation as Ong method) [26] and
fingerprint-based method (abbreviation as Yu method) [40]. Here, note that
fingerprint-based methods are proposed for deepfake detection and attribution
[34, 39, 40, 41, 24]. In this work, we are the first to introduce them to the
ownership protection field, considering their common objective: fingerprints
can be used to infer whether a suspect sample is from the model. Extensive
experiments show that our method achieves the best performance in all
evaluations among these ownership protection methods. In contrast, the other
two methods cannot work in many cases, especially for model extraction attacks
(see Section 6). Furthermore, we analyze the protection performance by
visualizing the characteristics learned by our method (see Section 7.1). We
also show the stability of our method on the number of generations of model
extraction attacks, which is a new emerging threat in the generative domain.
In contrast, the protection performance of the Yu method presents a
significant linear decrease and the Ong method completely fails in providing
any protection (see Section 7.2). Our analysis with respect to the number of
generated samples and different datasets further shows the effectiveness of
our method. Finally, extensive evaluations under adaptive attacks also
demonstrate that our method is still effective and robust even if adversaries
obtain partial knowledge of our method (see Section 8).

In summary, we make the following contributions. (1) We propose a novel
ownership protection method GAN-Guards from a new perspective: detecting
ownership infringement by utilizing the common characteristics of a target
model and stolen models. (2) We evaluate our method on various attacks,
including emerging model extraction attacks. (3) Our method achieves a new
state-of-the-art performance in ownership protection for GANs compared to
prior works. The source code will be made public along with the final version
of the paper.

##  2 Related Work

Generative adversarial networks. Generative adversarial networks (GANs) have
undoubtedly achieved a series of successes in image generation [25, 16, 17,
18, 3], image manipulation [30, 31], and image super-resolution [19, 43]. The
seminal GAN introduced in 2014 [10] presents a promising result in image
synthesis, which significantly inspires more and more researchers to propose
various methods to further advance the performance of GANs. Karras et al. [16]
propose a progressive training strategy that enables a GAN to synthesize high-
resolution images. In addition to generating high-resolution images, Karras et
al. [17] further introduce neural style transfer structures into the
architecture of GANs and it empowers GANs to generate a variety of style
images. Takeru et al. [25] introduce spectral normalization to normalize the
weights of each layer to improve the quality of synthetic images. Overall,
each improvement in GANs requires talented researchers to devote tremendous
efforts. In this work, instead of further improving the performance of GANs,
we target at developing a technique to protect the intellectual property of
valuable GANs.

Ownership protection. There are numerous works aiming to protect ownership of
discriminative models [33, 42, 15, 4, 21, 20, 7, 6]. These works can be
generally classified into three groups: embedding watermarks into model
parameters [33], using predefined inputs as triggers [1] and utilizing unique
features of models [23, 4]. However, methods on discriminative models cannot
be applied to generative models since they are different machine learning
models.

There are only a few works on ownership protection of generative models. Ong
et al. [26] propose a protection framework for GANs by adding a novel
regularization term to the existing loss function. Some works on fingerprints
of GANs can be applied into this field. Yu et al. [40] propose to add
fingerprints into training data and then verify the fingerprints on GANs.
Additionally, Yu et al. [41] add a novel fingerprint embedding layer to
modulate the generation of fingerprinted images. However, these works do not
perform evaluations on emerging model extraction attacks. In this work, we
propose a novel protection method and thoroughly evaluate these works on
various attacks, including model extraction attacks.

##  3 Background

###  3.1 Generative Adversarial Networks

An unconditional GAN generally consists of a generator $G$ and a discriminator
$D$. In the training phase, the generator $G$ aims to generate fake data to
fool the discriminator $D$ while the discriminator $D$ attempts to distinguish
fake data from the generator from the real data from the training set. Once
finishing training, the generator $G$ can be utilized to generate data, given
latent codes. Gaussian distribution or uniform distribution is commonly used
to obtain latent codes. Mathematically, the generator of a GAN is a function
$G:\mathcal{Z}\rightarrow\mathcal{X}$ that maps a low dimensional latent space
$\mathcal{Z}\subseteq\mathbb{R}^{n}$ to a high dimension data space
$\mathcal{X}\subseteq\mathbb{R}^{m}$.

###  3.2 Paradigms of Ownership Protection

Current paradigms of ownership protection on GANs can be divided into two
classes: watermark-based methods and fingerprint-based methods. Watermark-
based methods initially are proposed in the work [26]. The key idea is that
model owners claim ownership of this GAN if special outputs can be obtained
from a suspect GAN. Generally, these special outputs (e.g. watermarked
generated samples) are obtained through special queries from a GAN. For
simplicity, we refer to this ownership protection method by the first author’s
name, i.e, Ong [26].

Fingerprint-based methods are initially proposed for deepfake detection and
attribution [40, 41]. In this work, we apply these methods to protect
ownership of GANs. This is because they share a common objective that
fingerprints can be used to infer whether a suspect sample is from their
model. Specifically, the key idea of fingerprint-based methods is that if the
fingerprint extracted from generated samples from a GAN is identical to the
true fingerprint, model owners can claim ownership of the GAN. To achieve this
goal, various methods are proposed, such as adding fingerprints on the
training set of a GAN [40], and designing new architectures of a GAN and loss
functions [41]. In this work, considering the flexibility and scalability of a
method, we choose the former method—Yu [40] to evaluate the performance in
ownership protection and make a comparison with our proposed method.

Note that, unlike these paradigms that require forcibly implanting
watermarks/fingerprints into target models and retraining target models, our
method provides a novel paradigm: the common characteristics of a target model
and its stolen models are exploited to claim ownership, motivated by the
emerging of model extraction attacks.

##  4 A New Ownership Protection Method

###  4.1 Threat Model

We assume that defenders, i.e. model providers who deploy an ownership
protection method on their target model, only have access to generated samples
from a suspect model deployed by the adversaries. Thus, the defenders make an
ownership infringement decision only based on these generated samples. This is
the most practical and strictest assumption for defenders.

###  4.2 Key Observations

The first key observation is that physical stealing and model extraction
attacks are two fundamental but different types of ownership infringement.
Physical stealing attacks refer that an adversary physically copies a model
$G_{\it sub}$ from the target model $G_{\it tar}$. Therefore, $G_{\it sub}$ is
totally the same as $G_{\it tar}$. Model extraction attacks [13] refer that an
adversary retrains a substitute model $G_{\it sub}$ on generated samples from
a target model $G_{\it tar}$. These generated samples can be obtained by an
adversary when model owners release generated samples or provide a querying
interface. Thus, $G_{\it sub}$ is functionally similar to $G_{\it tar}$.

The second key observation is that as stolen models (i.e. constructed by
physical stealing or model extraction attacks) are derived from the target
model but honest models are not, it is thus natural to assume that stolen
models and the target model share common characteristics which do not exist in
honest models. Therefore, we can learn and leverage such characteristics as an
evidence to differentiate stolen models from honest models.

Figure 1: Overview of our method. 1 A target model is trained on a dataset
$\mathfrak{D}_{\it tar}$. 2 A substitute model is constructed by model
extraction. 3 An independent model is trained on a dataset $\mathfrak{D}_{\it
ind}$ that has the same distribution as the dataset $\mathfrak{D}_{\it tar}$,
but it is disjoint with $\mathfrak{D}_{\it tar}$. 4 A classifier is trained to
discriminate between stolen models and honest models. 5 The classifier is used
for the verification of a suspect model. Here, the target model can also refer
to the physically stealing model. The defenders do not have any information
about a suspect model $G_{\it sus}$, except generated samples, in the
verification phase.

###  4.3 Ownership Protection Algorithm

Overview. In order to extract the characteristics for a target model, our
method proposes to learn these characteristics by training a binary classifier
on generated samples. Generated samples from model extraction and physical
stealing are labelled as positive while samples from honest models, i.e.
independently trained models, are labelled as negative. The reason why we use
generated samples is that a GAN model is to learn the distribution of a
training set. The learned distribution is implicit, which can be represented
through these generated samples [2]. This process is shown in the deployment
phase of Figure 1.

In practice, it is impossible for the defenders to consider all independently
trained models and all models constructed by model extraction. Therefore, our
method constructs positive and negative GAN models only by the limited
knowledge of the defenders: the architectures of the target model and the
training set. Specifically, the architectures of all models (i.e. $G_{\it
tar}$, $G_{\it sub}$, $G_{\it ind}$) in the deployment phase are the same. The
independent set $\mathfrak{D}_{\it ind}$ and the target set $\mathfrak{D}_{\it
tar}$ are from the same distribution but disjoint. $\mathfrak{D}_{\it ind}$
can be easily obtained, e.g. the defenders split a dataset into two parts: one
as the independent set and the other as the training set. We emphasize that
our method is practical as suspect models constructed by the adversaries can
be trained on any (unknown for the defenders) GAN architectures and datasets.
Our extensive experiments in Section 6.4, demonstrate our method can well
generalize beyond these unknown GANs and correctly recognize them.

Input: a target model: $G_{\it tar}$; an independent dataset
$\mathfrak{D}_{\it ind}$; $m$ samples $X_{\it sus}$ from a suspect model
$G_{\it sus}$.

Output: ownership decision: ${\it OwDecision}$

1 def  _buildProtection(_$G_{\it tar},\mathfrak{D}_{\it ind}$ _)_:

2

3 Sample $\tilde{n}$ samples $X_{\it gen}$ from $G_{\it tar}$;

4 $G_{\it sub}\leftarrow{\sf trainGAN}(X_{\it gen})$;

5 $G_{\it ind}\leftarrow{\sf trainGAN}(\mathfrak{D}_{\it ind})$;

6

7 Sample $n$ samples $X_{\it gen}$ from $G_{\it tar}$; $\triangleright$
labelling positive for physical stealing.

8 Sample $n$ samples $X_{\it sub}$ from $G_{\it sub}$; $\triangleright$
labelling positive for model extraction.

9 Sample $2n$ samples $X_{\it ind}$ from $G_{\it ind}$; $\triangleright$
labelling negative for the honest model.

10 ${\it Classifier}\leftarrow{\sf trainClassifier}(X_{\it gen},X_{\it
sub},X_{\it ind})$;

11

12 return ${\it Classifier}$

13

14 def  _performVerification(_${\it Classifier},X_{\it sus},\tau$ _)_:

15 Initialize prediction array $\it pred$ of length $m$ with $0$;

16 for  _$i=0$ to $m-1$_ do

17

18 ${{\it pred}[i]}\leftarrow{\it Classifier}(X_{\it sus}[i])$;
$\triangleright$ Prediction: 1 or 0.

19 ${\it ConfiSocre}={\sf sum({\it pred})}/{m}$ ;

20 $\triangleright$ Make a decision based on multiple samples.

21 if  _${\it ConfiSocre} >\tau$_ then ${\it OwDecision}=1$;

22 else ${\it OwDecision}=0$;

23

24 return ${\it OwDecision}$

25

Algorithm 1 The GAN-Guards Algorithm.

Algorithm. As illustrated in the function buildProtection of Algorithm 1,
specifically, given the generator of a target model $G_{\it tar}$ and an
independent dataset $\mathfrak{D}_{\it ind}$, we first construct a substitute
model $G_{\it sub}$ by extracting the target model $G_{\it tar}$. Next, we
train a GAN $G_{\it ind}$ on the independent dataset $\mathfrak{D}_{\it ind}$.
Samples from $G_{\it sub}$ and $G_{\it tar}$ are labeled as positive while
samples from $G_{\it ind}$ are labeled as negative. These samples are used for
training a classifier and in this work we choose ResNet50 [11] as the
classifier.

After obtaining the trained classifier, we start to perform the verification
of ownership. We first collect $m$ generated samples released by a suspect
model $G_{\it sus}$. These samples are fed into the classifier and $m$
predictions can be obtained. We calculate the percentage of these positive
predictions. If it is larger than a predefined threshold, the suspect model is
inferred as stealing from the target model. We also analyze how the number of
generated samples $m$ affects our performance in Section 7.3. This process is
also illustrated in the function performVerification of Algorithm 1.

##  5 Experiments

###  5.1 Datasets

We evaluate our method on two datasets: FFHQ [17] and Church [38]. They are
typically used in image generation. The FFHQ dataset is designed for human
face image synthesis and includes 70,000 images. The Church dataset is from
the LSUN dataset, which contains 126,277 outdoor church images.

All images are resized to $64\times 64$. For each dataset, we randomly split
the dataset into three disjoint equal parts and mark each part as the
corresponding dataset name plus ‘I’, ‘II’, and ‘III’, respectively, such as
FFHQ-I and FFHQ-II. Dataset I, i.e. $\mathfrak{D}_{\it tar}$, is used to train
a target GAN model. Dataset II is used to train a GAN and later the model
(i.e. Ind-a in Section 5.2) is used as negative to test the ownership
protection methods. Dataset III, i.e. $\mathfrak{D}_{\it ind}$, is used to
train a GAN model and later the model is used for building a classifier
together with the target model. Specifically, we set the size of each part of
FFHQ and Church as 20,000 and 40,000, respectively.

###  5.2 Suspect Models

We consider various suspect models. Positive suspect models are considered
ownership infringement and these models are derived from the target models via
physical stealing and model extraction, and obfuscation attacks, such as input
perturbation, output perturbation, overwriting, and fine-tuning attacks.
Negative suspect models are host models and they are built from independent
training. Here, we consider two types: Ind-a trained on dataset II with the
same architectures of target models, and Ind-b that is trained on dataset I,
with the same architectures of target models but uses different seeds, i.e.
different random initializations. In Section A.6 in Appendix, we detail the
implementation and report performance of these suspect models.

###  5.3 Metrics

We use FID [12] to measure the performance of a GAN. 50K generated samples
from a GAN and all training samples are used to compute the FID value.

In terms of protection performance, the Ong method [26] utilizes the SSIM [35]
score to measure the similarity between the groundtruth watermark and the
watermark extracted from a suspect model. If the SSIM score of an image is
higher than a threshold, the image is more likely from the target model. The
Yu method [40] calculates a bitwise accuracy between the groundtruth
fingerprint and an extracted fingerprint. Claiming ownership of a model based
on only one image is not robust enough. Therefore, we make a final decision by
computing a confidence score on multiple samples. Specifically, given $m$
samples and each sample gets an output $o\in\\{0,1\\}$ from a suspect method,
the confidence score that recognizes a suspect model as positive is computed
by: ${\it ConfidenceScore=\frac{\sum_{i=0}^{m-1}o_{i}}{m}}$. In this work, we
set threshold $\tau$ of all methods as 90% for consistency. Thus, a suspect
model is predicted as positive (stolen model) if $\tau\geq 90\%$. We fix the
number of samples $m$ as 1,000.

###  5.4 Experimental Setups

In terms of GANs, we use SNGAN [25], PGGAN [16] and StyleGAN [17]. These all
can achieve excellent performance in image synthesis. We use the official
implementation of each GAN to train GANs. For model extraction attacks,
considering a trade-off between attack cost and performance, we set the number
of generated samples as 50,000, which is also suggested by the work [13].

For our protection method, we use ResNet50 [11] pretrained on ImageNet [27]
dataset for our classifier. The SGD optimizer with a learning rate of 0.003 is
used and the number of epochs is fixed as 5. As shown in Algorithm 1, we use
the Gaussian prior distribution to obtain generated samples and the number of
samples $n$ is set as 100,000. Therefore, 400,000 samples in total are used
for training the classifier. For the Ong method [26] and the Yu method [40],
we adopt their official implementations with suggested hyperparameters.

##  6 Evaluation

In this section, we compare our method with two state-of-the-art methods: the
Ong method [26] and the Yu method [40], and both have been already discussed
in Section 3.2. We evaluate them from various perspectives, including model
utility, verification performance, robustness to obfuscations, and robustness
to more model extractions.

Table 1: Performance of target model SNGAN trained on FFHQ-I on different
methods. ($\downarrow$ is better).

Methods | Ong | Yu | Ours
---|---|---|---
FID$(\mathfrak{D},\tilde{G})\downarrow$ | 20.14 | 26.46 | 20.25

###  6.1 Model Utility

Table 1 shows the performance of the target model SNGAN trained on FFHQ-I with
different protection methods. The FID is computed by the original training set
$\mathfrak{D}$ and the protected GAN $\tilde{G}$. Overall, the watermark-based
method Ong and our method achieve similar outstanding performance, while the
fingerprint-based method Yu shows worse performance. This is because the Yu
method needs to add fingerprints into a training set, which is at the cost of
sacrificing model utility.

###  6.2 Verification Performance

Figure 2 presents verification performance on different ownership protection
methods. The red dashed line is the threshold $\tau$ of the confidence score.
A model is predicted as a stolen (positive) model if $\tau\geq 90\%$. PS
refers to models from physical stealing while ME refers to models from model
extraction. Note that here ME, Ind-a, Ind-b models used in the verification
phase are not the same models used in our deployment phase (detailed in
Section 5.2). Overall, our method can correctly differentiate all positive and
negative suspect models, achieving 100% accuracy. However, the Ong method and
the Yu method are unable to defend against model extraction attacks.
Additionally, the Ong and Yu methods mistakenly recognize the suspect model
Ind-b trained with different initializations as a stolen model. This is
because embedded watermarks or fingerprints cannot be changed only owing to
different initializations of a training process. Thus, their methods lead to
false alarms and hurt honest model providers.

Figure 2: Verification performance of all methods. The target model SNGAN is
trained on FFHQ-I. PS and ME are positive suspect models while Ind-a and Ind-b
are negative suspect models. Note that _green and orange bars in some cases
cannot be observed because their scores are 0%._

###  6.3 Robustness to Obfuscations

In order to evade verification, advanced adversaries may utilize obfuscation
techniques to obfuscate stolen models. In this work, we consider four types of
obfuscation techniques: input perturbation, output perturbation, overwriting
and fine-tuning. Input perturbation aims to modify the queries, i.e., latent
codes, to evade special queries. Here, we adopt random input perturbation.
That is, for any query, a target model resamples latent codes from Gaussian
distribution. For brevity, we rename it Inp. Output perturbation refers to
perturbing generated samples by various post-processing techniques. We use
four different output perturbations: additive Gaussian noise, Gaussian filter,
Gaussian blurring, and JPEG compression. We briefly rename them Oup-a, Oup-b,
Oup-c, and Oup-d, respectively. The magnitude of these perturbations is set as
0.01, 0.4, 0.5, and 0.85, respectively. Overwriting refers to encoding a
different watermark/fingerprint to overwrite the original
watermark/fingerprint. Our method does not rely on watermarks and
fingerprints, thus intrinsically eliminating the threat of this attack. In
this work, we consider wholly fine-tuning where we take the weights of the
stolen model as initialization and retrain a GAN model on a different dataset
FFHQ-II. Because these obfuscation operations can be added into physical
stealing (PS) or model extraction (ME), there are different combinations
between obfuscation operations and PS and ME. Here, we mark them as ’PS+‘ and
’ME+‘ corresponding obfuscation operations, such as PS+Inp. Implementations
are illustrated in Section A.1 in Appendix.

(a) Input perturbation.

(b) PS + output perturbation.

(c) ME + output perturbation.

Figure 3: Robustness to Obfuscations. Protection performance for target model
SNGAN trained on FFHQ-I. Again, green and orange bars in some cases cannot be
observed because their scores are 0% and cannot defend against these attacks.

Results. Figure 3 shows the protection performance under input and output
perturbation operations. Overall, our method can still remain 100% accuracy
against input perturbation and output perturbation attacks. In contrast, the
Ong method totally cannot resist the input perturbation attack, as shown in
Figure 3(a) and the Yu method cannot defend against additive Gaussian noise of
output perturbation attacks (PS+Oup-a and ME+Oup-a). Again, Figure 3(c) shows
that the Ong and Yu methods cannot defend against ME+Output perturbation. We
analyze the reasons for the Ong and Yu methods in Section A.2 in Appendix.

We perform the evaluation under the overwriting attack. We do not report
results for our method because our method does not rely on watermarks or
fingerprints. Overall, the Ong and Yu methods cannot defend against this type
of attack and both confidence scores are 0%. It indicates that the overwritten
watermarks and fingerprints make their methods unable to extract the expected
outputs. We summarize the results in Table 3 in Appendix.

We evaluate the protection performance under the fine-tuning attack.
Unfortunately, we observe that all methods are not robust to the fine-tuning
attack. We analyze that this is because the fine-tuned GAN model has learned a
different distribution in a new training set and neural networks suffer from
catastrophic forgetting [8, 9]. The former makes that our method recognizes
this model as an independent training model while the latter makes that the
Ong and Yu methods forget embedded watermarks and fingerprints. This also
inspires us to think about the ownership boundary of a GAN and develop more
powerful protection work in future. We summarize the results in Table 4 in
Appendix.

Figure 4: Protection performance under model extraction attacks with different
GANs as attack models. The target model SNGAN is trained on FFHQ-I.

###  6.4 Robustness to More Model Extraction

When mounting model extraction attacks, adversaries can utilize various
architectures of GANs to extract a target model. Figure 4 shows the results of
all methods in terms of robustness to model extraction attacks with different
GANs as attack models. The target model is SNGAN. We see that our method still
performs well, while the other two methods recognize them as honest models.
This shows our method can recognize models constructed by model extraction
attacks regardless of the GAN architectures of adversaries.

##  7 Analysis

###  7.1 Visualization of Characteristics

Figure 5 shows the T-SNE visualization of characteristics learned by our
method. We plot the T-SNE figure by using outputs from the penultimate layer
of the classifier and the dimension of the outputs is 2,048. We clearly see
that characteristics from stolen models including PS and ME are entangled
together and have a clear boundary with that from honest models.

Figure 5: T-SNE visualization of characteristics learned by our method for
stolen models and honest models.

###  7.2 Generations of Model Extraction Attacks

Theoretically, model extraction attacks on GANs can continue forever like the
process of biological heredity, as shown in Figure 6. Models produced during
this process, such as $G^{(i)}$, should be correctly identified by ownership
protection methods. This motivates us to investigate whether the protection
performance will decrease with the number of generations of model extraction
attacks. We emphasize this is our newly identified threat, which is not
discussed in the literature about GAN ownership protection.

Here, we fix the number of generated samples as 1,000 and the target model is
SNGAN trained on FFHQ-I. We mark the target model SNGAN as SNGAN(0), and the
first generation of model extraction is marked as SNGAN(1), which means an
adversary uses an attack model SNGAN to extract the target model SNGAN. We do
not show the performance of the Ong method because it cannot defend against
model extraction attacks.

As shown in Figure 7, we can clearly observe that with the increase in the
number of generations of model extraction, the Yu method becomes less and less
confident. It also indicates that the fingerprint is not robust and more and
more generated samples cannot extract the corresponding fingerprint. In
contrast, our method still remains almost 100% confident to verify ownership
of the target model.

Figure 6: Generations of model extraction attacks. Figure 7: Protection
performance with regard to the number of generations of model extraction
attacks.

###  7.3 Number of Generated Samples

Figure 8 presents the protection performance of our method under the different
numbers of generated samples. The target model SNGAN is trained on FFHQ-I. The
ground truth of PS and ME is positive while that of Ind-a and Ind-b is
negative. We can clearly see that the confidence scores gradually remain
stable after 1,000 generated samples on all suspect models. It also shows that
our protection method has advantages with respect to the _efficiency_ , i.e.
it requires as few as 1,000 samples.

###  7.4 Different Datasets

We now present the performance of our method on the Church dataset which is
widely used in scene synthesis. The detail of this dataset is also discussed
in Section 5.1. The target model is SNGAN trained on the Church-I dataset and
achieves 12.96 FID.

Overall, our method on the Church dataset can achieve the same exceptional
protection performance as that on the FFHQ dataset. Figure 9(a) shows
verification performance on our method. We can clearly observe that our method
can verify the positive suspect models as positive with high confidence. As
shown in Figure 9(b), Figure 9(c) and Figure 9(d), our method can still remain
100% accuracy under the input perturbation and output perturbation. Our method
on fine-tuning attacks shows it fails to recognize positive suspect models as
positive. We show details in Table 5 in Appendix, and we also analyze the
performance on different target models in Section A.4.2 in Appendix.

Figure 8: Protection performance with regard to the numbers of generated
samples.

(a) Verification performance.

(b) Input perturbation.

(c) PS + output perturbation.

(d) ME + output perturbation.

Figure 9: Protection performance on target model SNGAN trained on Church-I.

##  8 Adaptive Attacks

Although researchers studying defense techniques strongly advocates that a new
defense should be evaluated on adaptive attacks [32], prior works on ownership
protection on GANs do not adopt it. In this work, for the first time, we
present the performance of our method under adaptive attacks. That is, we
assume that adversaries have some knowledge of our protection method, and
design a series of specific attacks to evade our method.

Figure 10: Protection performance under the adaptive attack I. The target
model SNGAN is trained on FFHQ-I. Figure 11: Protection performance under the
adaptive attack II. The target model SNGAN is trained on FFHQ-I.

We discuss two types of adaptive attack scenarios. The main design principle
is that we assume that adversaries perceive our method which is based on the
common characteristics of a target model and its stolen models. Therefore, the
adversaries attempt to decrease the confidence score of our method by
sacrificing model utility (i.e. the quality of generated images).
Specifically, in adaptive attack I, adversaries choose an inferior performance
GAN from multiple snapshots of a GAN when mounting model extraction attacks.
In adaptive attack II, adversaries evade our verification by designing a
series of output perturbations by choosing the magnitude of the perturbation.

Figure 10 shows protection performance under the adaptive attack I. Here, we
choose an attack model SNGAN to extract the target model SNGAN trained on
FFHQ-I. We choose eight snapshots of SNGAN during model extraction attacks.
The performance of the attack model SNGAN, i.e. FID($p_{g},p_{\tilde{g}}$),
ranges from 22 to 2, as depicted in the red line. $p_{g}$ and $p_{\tilde{g}}$
are the implicit distribution of the target model and the attack model,
respectively. We can observe that confidence scores begin to decrease, then
increase and remain at 100%, with the decrease in FID of the attack model
SNGAN. In particular, the confidence scores of all snapshots are above 98%,
which indicates that our method can correctly recognize all snapshots as
stolen models.

Figure 11 shows protection performance under the adaptive attack II.
Considering the model utility, we design three strategies (strategy I,
strategy II and strategy III) based on different magnitudes of four types of
output perturbation. Table 2 summarizes the magnitudes of output perturbation
of each strategy. Note that we combine four types of output perturbation
instead of single output perturbation. In Figure 17 in Appendix, we visually
present images generated by each strategy and the quality of generated images
becomes much noisier and blurrier from strategy I to strategy III. Overall, we
can observe in Figure 11 that our method still can recognize all positive
suspect models, although the confidence score of each suspect model decreases
from strategy I to strategy III. In addition, although strategy III can lower
the confidence of our method, the model almost cannot be used due to the low
quality of generated images visually. In practice, the loss in model utility
can make the adversaries less competitive in market share, compared to
legitimate model owners.

Table 2: Magnitudes of output perturbation of the adaptive attack II. a:
Additive Gaussian Noise; b: Gaussian Filtering; c: Gaussian Blurring; d: JPEG
Compression.

Strategy | Output Perturbation
---|---
| a | b | c | d
I | 0.001 | 0.1 | 0.1 | 95
II | 0.005 | 0.2 | 0.3 | 90
III | 0.01 | 0.4 | 0.5 | 85

##  9 Conclusion

In this paper, we have proposed a novel method to protect GAN ownership by
leveraging the common characteristics of a target model and its stolen GANs.
Extensive experimental evaluations demonstrate that: (a) In terms of model
utility, our method can bring lossless fidelity, compared to models without
protection, because it does not modify well-trained target models. (b) In
terms of robustness, our method can achieve new state-of-the-art protection
performance, compared with watermark-based methods and fingerprint-based
methods. Furthermore, we have also shown that our method is still effective
under two types of carefully designed adaptive attacks. (c) In terms of
undetectability, our method is undetectable for adversaries because it builds
on a target model with normal training and does not rely on watermarks or
fingerprints. (d) In terms of efficiency, our method requires about 1,000
generated samples to confidently verify the ownership of a GAN. Finally, we
also have performed a fine-grained analysis of our method from various
aspects, such as visualizing learned characteristics, the stability of the
performance with regard to the number of generations of model extraction
attacks, the number of generated samples and different datasets.

Fine-tuning attacks remain a challenge for ownership protection on GANs. In
future, we plan to design more powerful methods to defend against these types
of attacks. In addition, applying our protection method to other domains, such
as table data synthesis and text generation, is also an interesting research
direction.

## Acknowledgments

This research was funded in whole by the Luxembourg National Research Fund
(FNR), grant reference 13550291.

## References

  * [1] Yossi Adi, Carsten Baum, Moustapha Cisse, Benny Pinkas, and Joseph Keshet.  Turning your weakness into a strength: Watermarking deep neural networks by backdooring.  In Proceedings of USENIX Security Symposium (USENIX Security), pages 1615–1631. USENIX Association, 2018.
  * [2] Yasaman Bahri, Jonathan Kadmon, Jeffrey Pennington, Sam S. Schoenholz, Jascha Sohl-Dickstein, and Surya Ganguli.  Statistical mechanics of deep learning.  Annual Review of Condensed Matter Physics, 11(1):501–528, 2020\.
  * [3] Andrew Brock, Jeff Donahue, and Karen Simonyan.  Large scale GAN training for high fidelity natural image synthesis.  In Proceedings of International Conference on Learning Representations (ICLR), 2019.
  * [4] Jialuo Chen, Jingyi Wang, Tinglan Peng, Youcheng Sun, Peng Cheng, Shouling Ji, Xingjun Ma, Bo Li, and Dawn Song.  Copy, right? a testing framework for copyright protection of deep learning models.  In Proceedings of IEEE Symposium on Security and Privacy (S&P), pages 1013–1030. IEEE, 2022.
  * [5] Anton Cherepkov, Andrey Voynov, and Artem Babenko.  Navigating the gan parameter space for semantic image editing.  In Proceedings of IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), pages 3671–3680. IEEE, 2021.
  * [6] Tianshuo Cong, Xinlei He, and Yang Zhang.  Sslguard: A watermarking scheme for self-supervised learning pre-trained encoders.  In Proceedings of ACM SIGSAC Conference on Computer and Communications Security (CCS), pages 579–593. ACM, 2022.
  * [7] Adam Dziedzic, Haonan Duan, Muhammad Ahmad Kaleem, Nikita Dhawan, Jonas Guan, Yannis Cattan, Franziska Boenisch, and Nicolas Papernot.  Dataset inference for self-supervised models.  In Proceedings of Annual Conference on Neural Information Processing Systems (NeurIPS). Curran Associates, Inc., 2022.
  * [8] Robert M French.  Catastrophic forgetting in connectionist networks.  Trends in Cognitive Sciences, 3(4):128–135, 1999.
  * [9] Ian Goodfellow, Mehdi Mirza, Da Xiao, Aaron Courville, and Yoshua Bengio.  An empirical investigation of catastrophic forgetting in gradient-based neural networks.  arXiv preprint arXiv:1312.6211, 2013.
  * [10] Ian Goodfellow, Jean Pouget-Abadie, Mehdi Mirza, Bing Xu, David Warde-Farley, Sherjil Ozair, Aaron Courville, and Yoshua Bengio.  Generative adversarial nets.  In Proceedings of Annual Conference on Neural Information Processing Systems (NeurIPS), pages 2672–2680. Curran Associates, Inc., 2014\.
  * [11] Kaiming He, Xiangyu Zhang, Shaoqing Ren, and Jian Sun.  Deep residual learning for image recognition.  In Proceedings of IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), pages 770–778. IEEE, 2016.
  * [12] Martin Heusel, Hubert Ramsauer, Thomas Unterthiner, Bernhard Nessler, and Sepp Hochreiter.  Gans trained by a two time-scale update rule converge to a local nash equilibrium.  In Proceedings of Annual Conference on Neural Information Processing Systems (NeurIPS), pages 6626–6637. Curran Associates, Inc., 2017\.
  * [13] Hailong Hu and Jun Pang.  Stealing machine learning models: Attacks and countermeasures for generative adversarial networks.  In Annual Computer Security Applications Conference (ACSAC), pages 1–16. ACM, 2021.
  * [14] Phillip Isola, Jun-Yan Zhu, Tinghui Zhou, and Alexei A Efros.  Image-to-image translation with conditional adversarial networks.  In Proceedings of IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), pages 1125–1134. IEEE, 2017.
  * [15] Hengrui Jia, Christopher A Choquette-Choo, Varun Chandrasekaran, and Nicolas Papernot.  Entangled watermarks as a defense against model extraction.  In Proceedings of USENIX Security Symposium (USENIX Security), pages 1937–1954. USENIX Association, 2021.
  * [16] Tero Karras, Timo Aila, Samuli Laine, and Jaakko Lehtinen.  Progressive growing of GANs for quality, stability, and variation.  In Proceedings of International Conference on Learning Representations (ICLR), 2018.
  * [17] Tero Karras, Samuli Laine, and Timo Aila.  A style-based generator architecture for generative adversarial networks.  In Proceedings of IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), pages 4401–4410. IEEE, 2019.
  * [18] Tero Karras, Samuli Laine, Miika Aittala, Janne Hellsten, Jaakko Lehtinen, and Timo Aila.  Analyzing and improving the image quality of stylegan.  In Proceedings of IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), pages 8110–8119. IEEE, 2020.
  * [19] Christian Ledig, Lucas Theis, Ferenc Huszár, Jose Caballero, Andrew Cunningham, Alejandro Acosta, Andrew Aitken, Alykhan Tejani, Johannes Totz, Zehan Wang, et al.  Photo-realistic single image super-resolution using a generative adversarial network.  In Proceedings of IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), pages 4681–4690, 2017.
  * [20] Zheng Li, Chengyu Hu, Yang Zhang, and Shanqing Guo.  How to prove your model belongs to you: A blind-watermark based framework to protect intellectual property of dnn.  In Annual Computer Security Applications Conference (ACSAC), pages 126–137. ACM, 2019.
  * [21] Nils Lukas, Yuxuan Zhang, and Florian Kerschbaum.  Deep neural network fingerprinting by conferrable adversarial examples.  In Proceedings of International Conference on Learning Representations (ICLR), 2021.
  * [22] Shuang Ma, Jianlong Fu, Chang Wen Chen, and Tao Mei.  Da-gan: Instance-level image translation by deep attention generative adversarial networks.  In Proceedings of IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), pages 5657–5666. IEEE, 2018.
  * [23] Pratyush Maini, Mohammad Yaghini, and Nicolas Papernot.  Dataset inference: Ownership resolution in machine learning.  In Proceedings of International Conference on Learning Representations (ICLR), 2020.
  * [24] Francesco Marra, Diego Gragnaniello, Luisa Verdoliva, and Giovanni Poggi.  Do gans leave artificial fingerprints?  In 2019 IEEE conference on multimedia information processing and retrieval (MIPR), pages 506–511. IEEE, 2019.
  * [25] Takeru Miyato, Toshiki Kataoka, Masanori Koyama, and Yuichi Yoshida.  Spectral normalization for generative adversarial networks.  In Proceedings of International Conference on Learning Representations (ICLR), 2018.
  * [26] Ding Sheng Ong, Chee Seng Chan, Kam Woh Ng, Lixin Fan, and Qiang Yang.  Protecting intellectual property of generative adversarial networks from ambiguity attacks.  In Proceedings of IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), pages 3630–3639. IEEE, 2021.
  * [27] Olga Russakovsky, Jia Deng, Hao Su, Jonathan Krause, Sanjeev Satheesh, Sean Ma, Zhiheng Huang, Andrej Karpathy, Aditya Khosla, Michael Bernstein, Alexander C. Berg, and Li Fei-Fei.  Imagenet large scale visual recognition challenge.  International Journal of Computer Vision, 115(3):211–252, 2015\.
  * [28] Axel Sauer, Katja Schwarz, and Andreas Geiger.  Stylegan-xl: Scaling stylegan to large diverse datasets.  In Proceedings ACM SIGGRAPH Conference (SIGGRAPH ’22), pages 1–10. ACM, 2022.
  * [29] E Eugene Schultz.  A framework for understanding and predicting insider attacks.  Computers & Security, 21(6):526–531, 2002.
  * [30] Yujun Shen, Jinjin Gu, Xiaoou Tang, and Bolei Zhou.  Interpreting the latent space of gans for semantic face editing.  In Proceedings of IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), pages 9243–9252. IEEE, 2020.
  * [31] Yujun Shen and Bolei Zhou.  Closed-form factorization of latent semantics in gans.  In Proceedings of IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), pages 1532–1540. IEEE, 2021.
  * [32] Florian Tramer, Nicholas Carlini, Wieland Brendel, and Aleksander Madry.  On adaptive attacks to adversarial example defenses.  In Proceedings of Annual Conference on Neural Information Processing Systems (NeurIPS), volume 33, pages 1633–1645. Curran Associates, Inc., 2020.
  * [33] Yusuke Uchida, Yuki Nagai, Shigeyuki Sakazawa, and Shin’ichi Satoh.  Embedding watermarks into deep neural networks.  In Proceedings of ACM International Conference on Multimedia Retrieval (ICMR), pages 269–277. ACM, 2017.
  * [34] Sheng-Yu Wang, Oliver Wang, Richard Zhang, Andrew Owens, and Alexei A Efros.  Cnn-generated images are surprisingly easy to spot… for now.  In Proceedings of IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), pages 8695–8704. IEEE, 2020.
  * [35] Zhou Wang, Alan C Bovik, Hamid R Sheikh, and Eero P Simoncelli.  Image quality assessment: from error visibility to structural similarity.  IEEE Transactions on Image Processing, 13(4):600–612, 2004.
  * [36] Weihao Xia, Yulun Zhang, Yujiu Yang, Jing-Hao Xue, Bolei Zhou, and Ming-Hsuan Yang.  Gan inversion: A survey.  IEEE Transactions on Pattern Analysis and Machine Intelligence, 2022\.
  * [37] Zili Yi, Hao Zhang, Ping Tan, and Minglun Gong.  Dualgan: Unsupervised dual learning for image-to-image translation.  In Proceedings of IEEE International Conference on Computer Vision (ICCV), pages 2849–2857. IEEE, 2017.
  * [38] Fisher Yu, Ari Seff, Yinda Zhang, Shuran Song, Thomas Funkhouser, and Jianxiong Xiao.  LSUN: Construction of a large-scale image dataset using deep learning with humans in the loop.  arXiv preprint arXiv:1506.03365, 2015.
  * [39] Ning Yu, Larry S Davis, and Mario Fritz.  Attributing fake images to gans: Learning and analyzing gan fingerprints.  In Proceedings of IEEE International Conference on Computer Vision (ICCV), pages 7556–7566. IEEE, 2019.
  * [40] Ning Yu, Vladislav Skripniuk, Sahar Abdelnabi, and Mario Fritz.  Artificial fingerprinting for generative models: Rooting deepfake attribution in training data.  In Proceedings of IEEE International Conference on Computer Vision (ICCV), pages 14448–14457. IEEE, 2021.
  * [41] Ning Yu, Vladislav Skripniuk, Dingfan Chen, Larry S. Davis, and Mario Fritz.  Responsible disclosure of generative models using scalable fingerprinting.  In Proceedings of International Conference on Learning Representations (ICLR), 2022.
  * [42] Jialong Zhang, Zhongshu Gu, Jiyong Jang, Hui Wu, Marc Ph Stoecklin, Heqing Huang, and Ian Molloy.  Protecting intellectual property of deep neural networks with watermarking.  In Proceedings of the Asia Conference on Computer and Communications Security (ASIACCS), pages 159–172. ACM, 2018.
  * [43] Wenlong Zhang, Yihao Liu, Chao Dong, and Yu Qiao.  Ranksrgan: Generative adversarial networks with ranker for image super-resolution.  In Proceedings of IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), pages 3096–3105. IEEE, 2019.
  * [44] Jiapeng Zhu, Yujun Shen, Deli Zhao, and Bolei Zhou.  In-domain gan inversion for real image editing.  In Proceedings of European conference on computer vision (ECCV), pages 592–608. Springer, 2020.
  * [45] Jun-Yan Zhu, Taesung Park, Phillip Isola, and Alexei A Efros.  Unpaired image-to-image translation using cycle-consistent adversarial networks.  In Proceedings of IEEE International Conference on Computer Vision (ICCV), pages 2223–2232. IEEE, 2017.

##  Appendix A Appendix

###  A.1 Details of Obfuscations

In this subsection, we introduce obfuscation operations, including input
perturbation, output perturbation, overwriting, and fine-tuning.

Input perturbation. Given a trained GAN model $G$, we can get a generated
sample $x_{g}$ from the GAN by a latent code $z$ which is drawn from prior
distribution $P$, i.e., $x_{g}=G(z),z\sim P$. Input perturbation aims to
modify the queries, i.e., latent codes. The reason why we consider this is
that some works verify the ownership by specific latent codes
$z^{\prime}=T(z)$. $T$ is a function to transform a normal latent code $z$ to
a specific latent code $z^{\prime}$. Therefore, an adversary can perturb
latent codes to evade this type of verification. In this work, we adopt random
input perturbation. Specifically, for any query, a target model resamples
latent codes from Gaussian distribution.

Output perturbation. In addition to perturbing latent codes, an adversary can
perturb generated samples. In this work, we consider the following four common
operations.

  1. a.

Random Noise. This operation adds noises into a generated sample $x_{g}$.
Common noises include Gaussian noise and Poisson noise. In this work, we
choose Gaussian noise and the mean $\mu$ and the variance $\sigma$ control the
strength of noises.

  2. b.

Filtering. This operation aims to enhance some characteristics of an image.
Common operations include mean filter, median filter and Gaussian filter. In
this work, we choose the Gaussian filter.

  3. c.

Blurring. This operation makes a generated sample $x_{g}$ less sharp by
convolution. Common blurring operations include box blurring and Gaussian
blurring. In this work, we choose Gaussian blurring.

  4. d.

Compression. This operation is to compress the size of an image without
significantly degrading the image quality. Common compression operations
include lossless compression JPEG and lossy compression JPG. In this work, we
choose JPEG compression.

Overwriting. This attack is to target a class of ownership protection methods
utilizing watermarks or fingerprints. An adversary can encode a different
watermark/fingerprint to overwrite the original watermark/fingerprint.
Ideally, an ownership protection method should still verify the ownership in
this case. In this work, our proposed method does not rely on watermarks and
fingerprints, thus intrinsically eliminating the threat of this attack.

Fine-tuning. After obtaining a substitute model, an adversary may further
fine-tune the model on their own dataset. Generally, an adversary can
partially or wholly fine-tune the model. Wholly fine-tuning refers that all
weights of the model are fine-tuned while partially fine-tuning refers that
the weights of some layers are frozen and the remaining are fine-tuned. In
this work, we consider wholly fine-tuning in which we take the weights of the
stolen model as initialization and retrain a GAN model on a new dataset FFHQ-
II.

###  A.2 Understanding In-depth

Figure 12 shows SSIM scores of watermarks under different output
perturbations. Specifically, Figure 12 (a) is the watermark used in the Ong
method. Figure 12 (b) - Figure 12 (e) show the watermark under different
output perturbations. We can observe that only if the Ong method can extract
watermarks, output perturbation attacks do not have a significant impact on
the final decision. This can explain the reason why the Ong Method on
PS+output perturbation can perform well, as shown in Figure 3(b).

However, model extraction attacks and their derivative attacks (i.e.
ME+obfuscations) make the Ong and Yu methods fail. This is because these
attacks severely undermine watermarks and fingerprints. It also indicates that
methods based on watermarks and fingerprints are not robust and too easily
perturbed.

Figure 12: Watermarks under different output perturbations. (a) is the
original watermark. From (b) to (e), the output perturbation operations are
Additive Gaussian Noise, Gaussian Filtering, Gaussian Blurring, and JPEG
Compression, respectively. The corresponding SSIM scores between (a) and each
output perturbation are 84.085%, 97.47%, 99.34%, 95.43%, respectively.

###  A.3 Additional Results in Section 6

####  A.3.1 Performance on overwriting attacks

Figure 13 shows a new watermark that is used for overwriting attacks for the
Ong method. The original watermark is shown in Figure 12 (a).

Figure 13: Watermarks used for overwriting attacks.

Table 3 presents protection performance under the overwriting attacks. We do
not show results for our method because our method does not rely on watermarks
or fingerprints. Here, we only choose the PS model obtained by physical
stealing because it can be perfectly recognized by both methods on physical
stealing attacks. Intuitively, the protection method cannot verify this
suspect PS model, which also means that other suspect models cannot be
correctly verified. We observe that the Ong and Yu methods cannot defend
against this type of attack.

Table 3: Protection performance on overwriting. The target model SNGAN is
trained on FFHQ-I. The suspect model PS is the model obtained by physical
stealing. Confi.: confidence; Pred.: prediction.

Types | Ong | Yu
---|---|---
| Confi. Score(%) | Pred. | Confi. Score(%) | Pred.
PS | 0.00 | 0 | 0.00 | 0

####  A.3.2 Performance on fine-tuning attacks

We adopt wholly fine-tuning where all weights of each model are fine-tuned on
FFHQ-II. Specifically, we take the weights of the stolen model as
initialization and retrain a GAN model on FFHQ-II.

Table 4 reports protection performance under the fine-tuning attack. We
observe that all protection methods cannot be robust against this attack. The
main reason is that fine-tuning enforces a GAN to learn the distribution of a
new dataset. Due to the catastrophic forgetting of a neural network,
previously embedded information, such as watermarks and fingerprints, cannot
be kept. Similarly, the fine-tuned GAN that has learned a new distribution is
no longer similar to the target model, which fails our method.

Table 4: Protection performance on fine-tuning. Target model SNGAN is trained
on FFHQ-I.

Types | Ong | Yu | Ours
---|---|---|---
|  | Confi.
---
Score(%)
Pred. |  | Confi.
---
Score(%)
Pred. |  | Confi.
---
Score(%)
Pred.
PS | 0.00 | 0 | 0.00 | 0 | 40.10 | 0
ME | 0.00 | 0 | 0.00 | 0 | 28.00 | 0

###  A.4 Additional Results in Section 7

####  A.4.1 Performance on different datasets

Table 5 shows the protection performance of our method on fine-tuning attacks.
we take the weights of the stolen model as initialization and retrain a GAN
model on Church-II. Our evaluation shows that it fails to recognize positive
suspect models as positive.

Figure 14 shows the results of our method in terms of robustness to different
model extractions. We observe that our method still performs perfectly
regardless of the types of attack models.

Table 5: Protection performance on fine-tuning. Target model SNGAN is trained
on Church-I.

Types | Confi. Score(%) | Pred.
---|---|---
PS | 50.50 | 0
ME | 46.40 | 0

Figure 14: Robustness to more model extraction attacks. Protection performance
under model extraction attacks with different GANs as attack models. Target
model SNGAN is trained on Church-I.

####  A.4.2 Performance on different target models

We also show the protection performance of our method on the target model
StyleGAN. The StyleGAN is trained on FFHQ-I and achieves 8.76 FID.

Overall, our method still has competitive protection performance on the target
model StyleGAN. Our method can achieve 100% accuracy on verification
performance for all suspect models, as shown in Figure 15(a). In terms of
obfuscations, 100% accuracy can be seen on input perturbation attacks and four
types of output perturbation attacks, as depicted in Figure 15(b), Figure
15(c) and Figure 15(d) respectively. In the face of fine-tuning attacks, we
can see in Table 6 that our method still fails to recognize these positive
suspect models. Figure 16 shows the results of our method in terms of
robustness to different model extractions. We observe that our method still
performs perfectly regardless of the types of attack models.

(a) Verification performance.

(b) Input perturbation.

(c) PS + output perturbation.

(d) ME + output perturbation.

Figure 15: Protection performance on target model StyleGAN trained on FFHQ-I.
Table 6: Protection performance on fine-tuning. Target model StyleGAN is
trained on FFHQ-I.

Types | Confi. Score(%) | Pred.
---|---|---
PS | 35.90 | 0
ME | 49.40 | 0

Figure 16: Robustness to more model extraction. Protection performance under
model extraction with different GANs as attack models. Target model StyleGAN
is trained on FFHQ-I.

###  A.5 Image Quality under Adaptive Attack II

Figure 17 shows the image quality of three strategies under adaptive attack
II. Adversaries need to trade off model utility, i.e. the quality of generated
images, and copyright infringement risks. Although a large magnitude output
perturbation can help adversaries evade ownership detection, it will lower
model utility. It also makes the stolen models less competitive, compared to
the target models.

(a) No perturbation.

(b) Strategy I.

(c) Strategy II.

(d) Strategy III.

Figure 17: Adaptive attack II. From (a) strategy I to (c) strategy III, the
magnitude of output perturbation gradually increases. The corresponding
magnitudes are shown in Table 2. The average SSIM score for strategy I,
strategy II, and strategy III is 92.20%, 82.97%, and 82.10%, respectively.

###  A.6 Suspect Models

####  A.6.1 Implementation details of suspect models

For positive suspect models, models from physical stealing (marked as PS) are
the same as target models. We use model extraction attacks proposed in the
work [13] to construct models from model extraction (marked as ME).
Specifically, given $m$ generated samples from a target model $G_{\it tar}$,
the adversaries retrain a model (also called the substitute model or attack
model) on $m$ generated samples. Attack models can use any architecture, such
as SNGAN, PGGAN, or StyleGAN. We study protection performance under model
extraction attacks with different GANs as attack models in Section 6.4.

For negative suspect models, Ind-a is trained with the same architectures of
target models but on dataset II. Ind-b is trained with the same architectures
and datasets of target models but uses different seeds, i.e. different random
initializations. Using different seeds means models trained on different
training environments and optimization processes. Theoretically, models
trained with different seeds should be different, because they do not derive
from model extraction and physical stealing, and they are honest models with
independent training. Thus, an ownership protection method should be able to
differentiate them. Here, setups for negative suspect models are very similar
to those for target models because we aim to test whether a protection method
hurt honest model providers in the strong assumption setting. This requires
that a protection method should be extremely robust.

Table 7: Performance of suspect positive models. The target model is SNGAN
trained on FFHQ-I. It is corresponding to Figure 2.

Types | Ong | Yu | Ours
---|---|---|---
| Attack Performance | Attack Performance | Attack Performance
| FID($p_{\tilde{g}},p_{r}$) | FID($p_{\tilde{g}},p_{g}$) | FID($p_{\tilde{g}},p_{r}$) | FID($p_{\tilde{g}},p_{g}$) | FID($p_{\tilde{g}},p_{r}$) | FID($p_{\tilde{g}},p_{g}$)
PS | 20.14 | 0.00 | 26.46 | 0.00 | 20.25 | 0.00
ME | 25.47 | 2.52 | 30.83 | 3.23 | 27.60 | 3.13

####  A.6.2 Performance of suspect models

In this section, we show the performance of suspect models. Overall, we choose
suspect models with the best performance in each setting. For positive suspect
models, that is these models which are obtained by physical stealing or model
extraction attacks, we use FID($p_{\tilde{g}},p_{r}$) and
FID($p_{\tilde{g}},p_{g}$) to demonstrate the performance. Here,
$p_{\tilde{g}}$ is the implicit distribution of a suspect model or an attack
model. $p_{r}$ is the implicit distribution of a training set. $p_{g}$ is the
implicit distribution of a target model. FID($p_{\tilde{g}},p_{g}$) represents
the similarity between an attack model and a target model, while
FID($p_{\tilde{g}},p_{r}$) represents the similarity between an attack model
and the training set of a target model. In the work [13], they are also called
fidelity and accuracy, respectively. Here, we explicitly use
FID($p_{\tilde{g}},p_{r}$) and FID($p_{\tilde{g}},p_{g}$) to report the
performance of suspect positive models. For negative suspect models, we use
FID($p_{r},p_{g}$) for evaluation.

Table 7 and Table 8 show positive and negative suspect models for target model
SNGAN trained on FFHQ. Table 9 shows the performance of suspect positive
models constructed by model extraction with different GANs. Table 10 and Table
11 show positive and negative suspect models for target model SNGAN trained on
Church. Table 12 and Table 13 show positive and negative suspect models for
target model StyleGAN trained on FFHQ.

Table 8: Performance of suspect negative models. The target model is SNGAN
trained on FFHQ-I. FID($p_{r},p_{g}$) is used for evaluation.

Types | Ong | Yu | Ours
---|---|---|---
Ind-a | 17.84 | 17.84 | 17.84
Ind-b | 20.63 | 29.13 | 17.15

Table 9: Performance of suspect positive models. The target model is SNGAN
trained on FFHQ-I. It is corresponding to Figure 4.

Attack Models | Ong | Yu
---|---|---
| Attack Performance | Attack Performance
| FID($p_{\tilde{g}},p_{r}$) | FID($p_{\tilde{g}},p_{g}$) | FID($p_{\tilde{g}},p_{r}$) | FID($p_{\tilde{g}},p_{g}$)
SNGAN | 25.47 | 2.52 | 30.83 | 3.23
PGGAN | 23.58 | 2.33 | 29.29 | 1.80
StyleGAN | 21.62 | 2.70 | 27.27 | 2.62
| Ours |  |
SNGAN | 27.60 | 3.13 |  |
PGGAN | 23.11 | 2.56 |  |
StyleGAN | 21.07 | 2.97 |  |

Table 10: Performance of suspect positive models. The target model is SNGAN
trained on Church-I. It is corresponding to Figure 9.

Types | Attack Performance
---|---
| FID($p_{\tilde{g}},p_{r}$) | FID($p_{\tilde{g}},p_{g}$)
PS | 12.96 | 0.00
ME | 19.17 | 3.64

Table 11: Performance of suspect negative models. The target model is SNGAN
trained on Church-I.

Types | FID($p_{r},p_{g}$)
---|---
Ind-a | 13.35
Ind-b | 14.56

Table 12: Performance of suspect positive models. The target model is StyleGAN
trained on FFHQ-I. It is corresponding to Figure 15.

Types | Attack Performance
---|---
| FID($p_{\tilde{g}},p_{r}$) | FID($p_{\tilde{g}},p_{g}$)
PS | 8.76 | 0.00
ME | 12.61 | 4.12

Table 13: Performance of suspect negative models. The target model is StyleGAN
trained on FFHQ-I.

Types | FID($p_{r},p_{g}$)
---|---
Ind-a | 17.84
Ind-b | 8.92