# MOVE: Effective and Harmless Ownership Verification via Embedded External
Features

Yiming Li, Linghui Zhu, Xiaojun Jia, Yang Bai, Yong Jiang, Shu-Tao Xia, and
Xiaochun Cao Yiming Li, Linghui Zhu, and Yang Bai are with Tsinghua Shenzhen
International Graduate School, Tsinghua University, Shenzhen, China (e-mail:
<EMAIL_ADDRESS>zlh20@mails.tsinghua.edu.cn),
<EMAIL_ADDRESS>Jia is with Institute of Information
Engineering, Chinese Academy of Sciences, Beijing, China (e-mail:
jiaxiaojun@iie.ac.cn).Yong Jiang, and Shu-Tao Xia are with Tsinghua Shenzhen
International Graduate School, Tsinghua University, and also with the Research
Center of Artificial Intelligence, Peng Cheng Laboratory, Shenzhen, China
(e-mail<EMAIL_ADDRESS>xiast@sz.tsinghua.edu.cn).Xiaochun Cao is
with School of Cyber Science and Technology, Sun Yat-sen University, Shenzhen,
China (email: caoxiaochun@mail.sysu.edu.cn).

###### Abstract

Currently, deep neural networks (DNNs) are widely adopted in different
applications. Despite its commercial values, training a well-performed DNN is
resource-consuming. Accordingly, the well-trained model is valuable
intellectual property for its owner. However, recent studies revealed the
threats of model stealing, where the adversaries can obtain a function-similar
copy of the victim model, even when they can only query the model. In this
paper, we propose an effective and harmless model ownership verification
(MOVE) to defend against different types of model stealing simultaneously,
without introducing new security risks. In general, we conduct the ownership
verification by verifying whether a suspicious model contains the knowledge of
defender-specified external features. Specifically, we embed the external
features by tempering a few training samples with style transfer. We then
train a meta-classifier to determine whether a model is stolen from the
victim. This approach is inspired by the understanding that the stolen models
should contain the knowledge of features learned by the victim model. In
particular, we develop our MOVE method under both white-box and black-box
settings to provide comprehensive model protection. Extensive experiments on
benchmark datasets verify the effectiveness of our method and its resistance
to potential adaptive attacks. The codes for reproducing the main experiments
of our method are available at https://github.com/THUYimingLi/MOVE.

###### Index Terms:

Model Stealing, Ownership Verification, Model Watermarking, Deep Intellectual
Property Protection, AI Security

##  1 Introduction

Deep learning, especially deep neural networks (DNNs), has been successfully
adopted in widespread applications for its high effectiveness and efficiency
[1, 2, 3]. In general, obtaining well-performed DNNs is usually expensive for
it requires well-designed architecture, a large number of high-quality
training samples, and many computational resources. Accordingly, these models
are the valuable intellectual properties of their owners.

However, recent studies [4, 5, 6] revealed that the adversaries can obtain a
function-similar copy model of the well-performed victim model to ‘steal’ it.
This attack is called _model stealing_. For example, the adversaries can copy
the victim model directly if they can access its source files; Even when the
victim model is deployed where the adversaries can only query the model, they
can still steal it based on its predictions ($i.e.$, labels or probabilities).
Since the stealing process is usually costless compared with obtaining a well-
trained victim model, model stealing poses a huge threat to the model owners.

Currently, there are also some methods to defend against model stealing. In
general, existing defenses can be roughly divided into two main categories,
including the _active defenses_ and _verification-based defenses_.
Specifically, active defenses intend to increase the costs ($e.g.$, query
times and accuracy decrease) of model stealing, while verification-based
defenses attempt to verify whether a suspicious model is stolen from the
victim model. For example, defenders can introduce randomness or perturbations
in the victim models [4, 7, 8] or watermark the victim model via (targeted)
backdoor attacks or data poisoning [9, 10, 11]. However, existing active
defenses may lead to poor performance of the victim model and could even be
bypassed by advanced adaptive attacks [10, 11, 12]; the verification-based
methods target only limited simple stealing scenarios ($e.g.$, direct copy or
fine-tuning) and have minor effects in defending against more complicated
model stealing. Besides, these methods also introduce some stealthy latent
_short-cuts_ ($e.g.$, hidden backdoors) in the victim model, which could be
maliciously used. It further hinders their applications. Accordingly, how to
defend against model stealing is still an important open question.

In this paper, we revisit the verification-based defenses against model
stealing, which examine whether a suspicious model has defender-specified
behaviors. If the model has such behaviors, the defense treats it as stolen
from the victim. We argue that a defense is practical if and only if it is
both effective and harmless. Specifically, effectiveness requires that it can
accurately identify whether the suspicious model is stolen from the victim, no
matter what model stealing is adopted; Harmlessness ensures that the model
watermarking brings no additional security risks, $i.e.$, the model trained
with the watermarked dataset should have similar prediction behaviors to the
one trained with the benign dataset. We first reveal that existing methods
fail to meet all these requirements and their reasons. Based on the analysis,
we propose to conduct model ownership verification via embedded external
features (MOVE), trying to fulfill both two requirements. Our MOVE defense
consists of three main steps, including 1) embedding external features, 2)
training ownership meta-classifier, and 3) ownership verification with
hypothesis-test. In general, the external features are different from those
contained in the original training set. Specifically, we embed external
features by tempering the images of a few training samples based on _style
transfer_. Since we only poison a few samples and do not change their labels,
the embedded features will not hinder the functionality of the victim model
and will not create a malicious hidden backdoor in the victim model. Besides,
we also train a _benign model_ based on the original training set. It is used
only for training the meta-classifier to determine whether a suspicious model
is stolen from the victim. In particular, we develop our MOVE method under
both white-box and black-box settings to provide comprehensive model
protection.

The main contribution of this work is five-fold: 1) We revisit the defenses
against model stealing from the aspect of ownership verification. 2) We reveal
the limitations of existing verification-based methods and their failure
reasons. 3) We propose a simple yet effective ownership verification method
under both white-box and black-box settings. 4) We verify the effectiveness of
our method on benchmark datasets under various types of attacks simultaneously
and discuss its resistance to potential adaptive attacks. 5) Our work could
provide a new angle about how to adopt the malicious ‘data poisoning’ for
positive purposes.

This paper is a journal extension of our conference paper [12]. Compared with
the preliminary conference version, we have made significant improvements and
extensions in this paper. The main differences are from four aspects: 1) We
rewrite the motivation of this paper to the significance and problems of
existing methods in Introduction to better clarify our significance. 2) We
generalize our MOVE from the white-box settings to the black-box settings in
Section 5 to enhance its abilities and widen its applications. 3) We analyze
the resistance of our MOVE to potential adaptive attacks and discuss its
relations with membership inference and backdoor attacks in Section 7.3 and
Section 7.4, respectively. 4) More results and analysis are added in Section
6.

The rest of this paper is organized as follows. We briefly review related
works, including model stealing and its defenses, in Section 2. After that, we
introduce the preliminaries and formulate the studied problem. In Section 4,
we reveal the limitations of existing verification-based defenses. We
introduce our MOVE under both white-box and black-box settings in Section 5.
We verify the effectiveness of our methods in Section 6 and conclude this
paper at the end. We hope that our paper can inspire a deeper understanding of
model ownership verification, to facilitate the intellectual property
protection of model owners.

##  2 Related Work

###  2.1 Model Stealing

In general, model stealing111In this paper, we focus on model stealing and its
defenses in image classification tasks. The attacks and defenses in other
tasks are out of the scope of this paper. We will discuss them in our future
work. aims to steal the intellectual property from a victim by obtaining a
function-similar copy of the victim model. Depending on the adversary’s access
level to the victim model, existing model stealing methods can be divided into
four main categories, as follows:

_1) Fully-Accessible Attacks ( $\mathcal{A}_{F}$): _ In this setting, the
adversaries can directly copy and deploy the victim model.

_2) Dataset-Accessible Attacks ( $\mathcal{A}_{D}$): _ The adversaries can
access the training dataset of the victim model, whereas they can only query
the victim model. The adversaries may obtain a stolen model by knowledge
distillation [13].

_3) Model-Accessible Attacks ( $\mathcal{A}_{M}$): _ The adversaries have
complete access to the victim model whereas having no training samples. This
attack may happen when the victim model is open-sourced. The adversaries may
directly fine-tune the victim model (with their own samples) or use the victim
model for data-free distillation in a zero-shot learning framework [14] to
obtain the stolen model.

_4) Query-Only Attacks ( $\mathcal{A}_{Q}$): _ This is the most threatening
type of model stealing where the adversaries can only query the victim model.
Depending on the feedback of the victim model, the query-only attacks can be
divided into two sub-categories, including the _label-query attacks_ [15, 16,
6] and the _logit-query attacks_ [4, 5]. In general, label-query attacks
adopted the victim model to annotate some substitute (unlabeled) samples,
based on which to train their substitute model. In the logit-query attacks,
the adversary usually obtains the function-similar substitute model by
minimizing the distance between its predicted logits and those generated by
the victim model.

###  2.2 Defenses against Model Stealing

####  2.2.1 Active Defenses

Currently, most of the existing methods against model stealing are active
defenses. In general, they intend to increase the costs ($e.g.$, query times
and accuracy decrease) of model stealing. For example, defenders may round the
probability vectors [4], introduce noise to the output vectors which will
result in a high loss in the processes of model stealing [7], or only return
the most confident label instead of the whole output vector [5]. However,
these defenses may significantly reduce the performance of victim models and
may even be bypassed by adaptive attacks [10, 11, 12].

####  2.2.2 Model Ownership Verification

In general, model ownership verification intends to verify whether a
suspicious model is stolen from the victim. Currently, existing methods can be
divided into two main categories, including _membership inference_ and
_backdoor watermarking_ , as follows:

_Verification via Membership Inference._ Membership inference [17, 18, 19]
aims to identify whether some particular samples are used to train a given
model. Intuitively, defenders can use it to verify whether the suspicious
model is trained on particular training samples used by the victim model to
conduct ownership verification. However, simply applying membership inference
for ownership verification is far less effective in defending against many
complicated model stealing ($e.g.$, model extraction) [11]. This is most
probably because the suspicious models obtained by these processes are
significantly different from the victim model, although they have similar
functions. Most recently, Maini _et al._ proposed dataset inference [11]
trying to defend against different types of model stealing simultaneously. Its
key idea is to identify whether a suspicious model contains the knowledge of
the inherent features that the victim model $V$ learned from the private
training set instead of simply particular samples. Specifically, let we
consider a $K$-classification problem. For each sample $(\bm{x},y)$, dataset
inference first generated its minimum distance $\bm{\delta}_{t}$ to each class
$t$ by

$\min_{\bm{\delta}_{t}}d(\bm{x},\bm{x}+\bm{\delta}_{t}),s.t.,V(\bm{x}+\bm{\delta}_{t})=t,$
(1)

where $d(\cdot)$ is a distance metric ($e.g.$, $\ell^{\infty}$ norm). It
defined the distance to each class ($i.e.$,
$\bm{\delta}=(\bm{\delta}_{1},\cdots,\bm{\delta}_{K})$) as the inherent
feature of sample $(\bm{x},y)$ $w.r.t.$ the victim model $V$. After that, the
defender will randomly select some samples inside (labeled as ‘+1’) or out-
side (labeled as ‘-1’) their private dataset and use the feature embedding
$\bm{\delta}$ to train a binary meta-classifier $C$, where
$C(\bm{\delta})\in[0,1]$ indicates the probability that the sample
$(\bm{x},y)$ is from the private set. In the verification stage, the defender
will select equal-sized sample vectors from private and public samples and
then calculates the inherent feature embedding $\bm{\delta}$ for each sample
vector $w.r.t.$ the suspicious model $S$. To verify whether $S$ is stolen from
$V$, the trained $C$ gives the confidence scores based on the inherent feature
embedding $\bm{\delta}$ of $S$. Besides, dataset inference adopted
_hypothesis-test_ based on the confidence scores of sample vectors to provide
a more confident verification. However, as shown in the following experiments
in Section 4.1, dataset inference is easy to make misjudgments, especially
when the training set of suspicious models has a similar distribution to that
of the victim model. The misjudgment is mostly because different models may
learn similar inherent features once their training sets have certain
distribution similarities.

_Verification via Backdoor Watermarking._ These methods [20, 9, 10] were
originally proposed to defend against fully-accessible attacks with or without
fine-tuning. However, we notice that they enjoy certain similarities to
dataset inference, since they both relied on the knowledge learned by the
victim model from the private dataset. Accordingly, they serve as potential
defenses against other types of model stealing, such as model-accessible and
query-only attacks. The main difference compared with dataset inference is
that they conducted model ownership based on defender-specified trigger-label
pairs that are out of the distribution of the original training set. For
example, backdoor watermarking first adopted backdoor attacks [21] to
watermark the model during the training process and then conducted the
ownership verification. In particular, a backdoor attack can be characterized
by three components, including 1) target class $y_{t}$, 2) trigger pattern
$\bm{t}$, and 3) pre-defined poisoned image generator $G(\cdot)$. Given the
benign training set $\mathcal{D}=\\{(\bm{x}_{i},y_{i})\\}_{i=1}^{N}$, the
backdoor adversary will randomly select $\gamma\%$ samples ($i.e.$,
$\mathcal{D}_{s}$) from $\mathcal{D}$ to generate their poisoned version
$\mathcal{D}_{p}=\\{(\bm{x}^{\prime},y_{t})|\bm{x}^{\prime}=G(\bm{x};\bm{t}),(\bm{x},y)\in\mathcal{D}_{s}\\}$.
Different backdoor attacks may assign different generator $G(\cdot)$. For
example,
$G(\bm{x};\bm{t})=(\bm{1}-\bm{\lambda})\otimes\bm{x}+\bm{\lambda}\otimes\bm{t}$
where $\bm{\lambda}\in\\{0,1\\}^{C\times W\times H}$ and $\otimes$ indicates
the element-wise product in the BadNets [22]; $G(\bm{x})=\bm{x}+\bm{t}$ in the
ISSBA [23]. After $\mathcal{D}_{p}$ was generated, $\mathcal{D}_{p}$ and
remaining benign samples
$\mathcal{D}_{b}\triangleq\mathcal{D}\backslash\mathcal{D}_{s}$ will be used
to train the (watermarked) model $f_{\theta}$ via

$\min_{\bm{\theta}}\sum_{(\bm{x},y)\in\mathcal{D}_{p}\cup\mathcal{D}_{b}}\mathcal{L}(f_{\bm{\theta}}(\bm{x}),y),$
(2)

where $\mathcal{L}(\cdot)$ is the loss function. In the verification stage,
the defender will examine the suspicious model in predicting $y_{t}$. If the
confidence scores of poisoned samples are significantly greater than those of
benign samples, the suspicious model is treated as watermarked and therefore
it is stolen from the victim. However, as shown in the following experiments
conducted in Section 4.2, these methods have minor effects in defending
against more complicated model stealing. Their failures are most probably
because the hidden backdoor is modified after the complicated stealing
process. Moreover, backdoor watermarking also introduce new security threats,
since it builds a stealthy latent connection between trigger pattern and
target label. The adversaries may use it to maliciously manipulate the
predictions of deployed victim models. This problem will also hinder their
utility in practice.

In conclusion, existing defenses still have vital limitations. How to defend
against model stealing is still an important open question and worth further
exploration.

##  3 Preliminaries

###  3.1 Technical Terms

Before we dive into technical details, we first present the definition of
commonly used technical terms, as follows:

  * •

_Victim Model_ : released model that could be stolen by the adversaries.

  * •

_Suspicious Model_ : model that is likely stolen from the victim model.

  * •

_Benign Dataset_ : unmodifed dataset.

  * •

_Benign Sample_ : unmodifed sample.

  * •

_Watermarked Dataset_ : dataset used for watermarking the (victim) model.

  * •

_Watermarked Sample_ : modified sample contained in the watermarked dataset.

  * •

_Poisoned Sample_ : modified sample used to create and activate the backdoor.

  * •

_Benign Accuracy_ : the accuracy of models in predicting benign testing
samples.

  * •

_Attack Success Rate_ : the accuracy of models in predicting poisoned testing
samples.

###  3.2 Problem Formulation

In this paper, we focus on defending against model stealing in image
classification tasks via model ownership verification. Specifically, given a
suspicious model $S$, the defender intends to identify whether it is stolen
from the victim model $V$. We argue that a model ownership verification is
promising in practice if and only if it satisfies both two requirements
simultaneously, as follows:

######  Definition 1 (Two Necessary Requirements).

  * •

_Effectiveness_ : The defense could accurately identify whether the suspicious
model is stolen from the victim no matter what model stealing is adopted.

  * •

_Harmlessness_ : The defense brings no additional security risks ($e.g.$,
backdoor), $i.e.$, the model trained with the watermarked dataset should have
similar prediction behaviors to the one trained with the benign dataset.

In general, effectiveness guarantees verification effects, while harmlessness
ensures the safety of the victim model.

###  3.3 Threat Model

In this paper, we consider both white-box and black-box settings of model
ownership verification, as follows:

_White-box Setting._ We assume that the defenders have complete access to the
suspicious model, $i.e.$, they can obtain its source files. However, the
defenders have no information about the stealing process. For example, they
have no information about the training samples, the training schedule, and the
adopted stealing method used by the adversaries. One may argue that only
black-box defenses are practical, since the adversary may refuse to provide
the suspicious model. However, white-box defenses are also practical. In our
opinion, the adoption of verification-based defenses (in a legal system)
requires an official institute for arbitration. Specifically, all commercial
models should be registered here, through the unique identification ($e.g.$,
MD5 code [24]) of their model’s source files. When this official institute is
established, its staff should take responsibility for the verification
process. For example, they can require the company to provide the model file
with the registered identification and then use our method (under the white-
box setting) for verification.

_Black-box Setting._ We assume that the defenders can only query and obtain
the predicted probabilities from the suspicious model, whereas cannot get
access to the model source files or intermediate results ($e.g.$, gradients)
and have no information about the stealing. This approach can be used as a
primary inspection of the suspicious model before applying the official
arbitration where white-box verifications may be adopted. In particular, we
did not consider the label-only black-box setting since the harmlessness
requires that no abnormal prediction behaviors (compared with the model
trained with a benign dataset) are introduced in the victim model. In other
words, it is impossible to accurately identify model stealing under the label-
only black-box setting.

##  4 Revisiting Verification-based Defenses

###  4.1 The Limitations of Dataset Inference

As we described in Section 2.2, dataset inference reached better performance
compared with membership inference by using inherent features instead of given
samples. In other words, it relied on the latent assumption that a benign
model will not learn features contained in the training set of the victim
model. However, different models may learn similar features from different
datasets, $i.e.$, this assumption does not hold and therefore leads to
misjudgments. In this section, we will illustrate this limitation.

_Settings._ We conduct the experiments on CIFAR-10 [25] dataset with VGG [26]
and ResNet [27]. To create two independent datasets that have similar
distribution, we randomly separate the original training set $\mathcal{D}$
into two disjoint subsets $\mathcal{D}_{l}$ and $\mathcal{D}_{r}$. We train
the VGG on $\mathcal{D}_{l}$ (dubbed VGG-$\mathcal{D}_{l}$) and the ResNet on
$\mathcal{D}_{r}$ (dubbed ResNet-$\mathcal{D}_{r}$), respectively. We also
train a VGG on a noisy verision of $\mathcal{D}_{l}$ ($i.e.$,
$\mathcal{D}_{l}^{\prime}$), where
$\mathcal{D}_{l}^{\prime}\triangleq\\{(\bm{x}^{\prime},y)|\bm{x}^{\prime}=\bm{x}+\mathcal{N}(0,16),(\bm{x},y)\in\mathcal{D}_{l}\\}$
(dubbed VGG-$\mathcal{D}_{l}^{\prime}$) for reference. In the verification
process, we verify whether the VGG-$\mathcal{D}_{l}$ and
VGG-$\mathcal{D}_{l}^{\prime}$ is stolen from ResNet-$\mathcal{D}_{r}$ and
whether the ResNet-$\mathcal{D}_{r}$ is stolen from VGG-$\mathcal{D}_{l}$ with
dataset inference [11]. As described in Section 2.2, we adopt the p-value as
the evaluation metric, following the setting of dataset inference. In
particular, the smaller the p-value, the more confident that dataset inference
believes that the suspicious model is stolen from the victim model.

_Results._ As shown in Table I, all models have promising performance even
with only half the numbers of original training samples. However, the p-value
is significantly smaller than 0.01 in all cases. In other words, dataset
inference is confident about the judgments that VGG-$\mathcal{D}_{l}$ and
VGG-$\mathcal{D}_{l}^{\prime}$ are stolen from ResNet-$\mathcal{D}_{r}$, and
ResNet-$\mathcal{D}_{r}$ is stolen from VGG-$\mathcal{D}_{l}$. However, none
of these models should be regarded as stolen from the victim since they adopt
completely different training samples and model structures. These results
reveal that _dataset inference could make misjudgments and therefore its
results are questionable_. Besides, the misjudgments may probably cause by the
distribution similarity among $\mathcal{D}_{l}$, $\mathcal{D}_{r}$, and
$\mathcal{D}_{l}^{\prime}$. The p-value of the VGG-$\mathcal{D}_{l}$ is lower
than that of the VGG-$\mathcal{D}_{l}^{\prime}$. This is probably because the
latent distribution of $\mathcal{D}_{l}^{\prime}$ is more different from that
of $\mathcal{D}_{r}$ (compared with that of $\mathcal{D}_{l}$) and therefore
models learn more different features.

TABLE I: The accuracy of victim models and p-value of verification processes.
In this experiment, dataset inference misjudges in all cases. The failed case
is marked in red.

| ResNet-$\mathcal{D}_{r}$ | VGG-$\mathcal{D}_{l}$ | VGG-$\mathcal{D}_{l}^{\prime}$
---|---|---|---
Accuracy | 88.0% | 87.7% | 85.0%
p-value | $10^{-7}$ | $10^{-5}$ | $10^{-4}$

###  4.2 The Limitations of Backdoor Watermarking

As described in Section 2.2, the verification via backdoor watermarking relies
on the latent assumption that the defender-specific trigger pattern matches
the backdoor embedded in stolen models. This assumption holds in its
originally discussed scenarios, since the suspicious model is the same as the
victim model. However, this assumption may not hold in more advanced model
stealing, since the backdoor may be changed or even removed during the
stealing process. Consequently, the backdoor-based watermarking may fail in
defending against model stealing. In this section, we verify this limitation.

_Settings._ We adopt the most representative and effective backdoor attack,
the BadNets [22], as an example for the discussion. The watermarked model will
then be stolen by the data-free distillation-based model stealing [14]. We
adopt _benign accuracy (BA)_ and _attack success rate (ASR)_ [21] to evaluate
the performance of the stolen model. The larger the ASR, the more likely the
stealing will be detected.

Figure 1: The main pipeline of our MOVE defense. In the first step, defenders
will modify some images via style transfer for embedding external features. In
the second step, defenders will train a meta-classifier to determine whether a
suspicious model is stolen from the victim based on gradients (under the
white-box setting) or prediction differences (under the black-box setting). In
the last step, defenders will conduct ownership verification via hypothesis-
test. TABLE II: The performance (%) of different models. The failed
verification case is marked in red.

Model Type $\rightarrow$ | Benign | Watermarked | Stolen
---|---|---|---
BA | 91.99 | 85.49 | 70.17
ASR | 0.01 | 100.00 | $3.84$

Figure 2: The adopted trigger pattern and synthesized ones obtained from the
watermarked and the stolen model. The trigger areas are indicated in the blue
box. (a) ground-truth trigger pattern; (b) pattern obtained from the
watermarked model; (c) pattern obtained from the stolen model.

_Results._ As shown in Table II, the ASR of the stolen model is only 3.84%,
which is significantly lower than that of the watermarked model. In other
words, _the defender-specified trigger no longer matches the hidden backdoor
contained in the stolen model_. As such, backdoor-based model ownership
verification will fail to detect this model stealing. To further discuss the
reason for this failure, we adopt the targeted universal adversarial attack
[28] to synthesize and visualize the potential trigger pattern of each model.
As shown in Figure 2, the trigger pattern recovered from the victim model is
similar to the ground-truth one. However, the recovered pattern from the
stolen model is significantly different from the ground-truth one. These
results provide a reasonable explanation for the failure of backdoor-based
watermarking.

In particular, backdoor watermarking will also _introduce additional security
risks_ to the victim model and therefore is harmful. Specifically, it builds a
stealthy latent connection between triggers and the target label. The
adversary may use it to maliciously manipulate predictions of the victim
model. This potential security risk will further hinder the adoption of
backdoor-based model verification in practice.

##  5 The Proposed Method

Based on the observations in Section 4, we propose a harmless model ownership
verification (MOVE) by embedding external features (instead of inherent
features) into victim model, without changing the label of watermarked
samples.

###  5.1 Overall Pipeline

In general, our MOVE defense consists of three main steps, including 1)
embedding external features, 2) training an ownership meta-classifier, and 3)
conducting ownership verification. In particular, we consider both white-box
and black-box settings. They have the same feature embedding process and
similar verification processes, while having different training manners of the
meta-classifier. The main pipeline of our MOVE is shown in Figure 1.

###  5.2 Embedding External Features

In this section, we describe how to embed external features. We first define
inherent and external features before reaching the technical details of the
embedding.

######  Definition 2 (Inherent and External Features).

A feature $f$ is called the inherent feature of dataset $\mathcal{D}$ if and
only if $\
\forall(\bm{x},y)\in\mathcal{X}\times\mathcal{Y},(\bm{x},y)\in\mathcal{D}\Rightarrow(\bm{x},y)\
\text{contains feature}f.$ Similarly, f is called the external feature of
dataset $\mathcal{D}$ if and only if $\
\forall(\bm{x},y)\in\mathcal{X}\times\mathcal{Y},(\bm{x},y)\ \text{contains
feature}\ f\Rightarrow(\bm{x},y)\notin\mathcal{D}.$

######  Example 1.

If an image is from the MNIST dataset, it is at least grayscale; If an image
is cartoon-type, it is not from the CIFAR-10 dataset for it contains only
natural images.

Although external features are well defined, how to construct them is still
difficult, since the learning dynamic of DNNs remains unclear and the concept
of features itself is complicated. However, at least we know that the _image
style_ can serve as a feature for the learning of DNNs in image-related tasks,
based on some recent studies [29, 30, 31]. As such, we can use _style
transfer_ [32, 33, 34] for embedding external features. People may also adopt
other methods for the embedding. It will be discussed in our future work.

Figure 3: Images involved in different defenses. (a) benign image; (b)
poisoned image in BadNets; (c) poisoned image in Gradient Matching; (d)
poisoned image in Entangled Watermarks; (e) style image; (f) transformed image
in our MOVE.

In particular, let $\mathcal{D}=\\{(\bm{x}_{i},y_{i})\\}_{i=1}^{N}$ denote the
benign training set, $\bm{x}_{s}$ is a defender-specified _style image_ , and
$T:\mathcal{X}\times\mathcal{X}\rightarrow\mathcal{X}$ is a (pre-trained)
style transformer. In this step, the defender first randomly selects
$\gamma\%$ (dubbed _transformation rate_) samples ($i.e.$, $\mathcal{D}_{s}$)
from $\mathcal{D}$ to generate their transformed version
$\mathcal{D}_{t}=\\{(\bm{x}^{\prime},y)|\bm{x}^{\prime}=T(\bm{x},\bm{x}_{s}),(\bm{x},y)\in\mathcal{D}_{s}\\}$.
The external features will be learned by the victim model $V_{\theta}$ during
the training process via

$\min_{\bm{\theta}}\sum_{(\bm{x},y)\in\mathcal{D}_{b}\cup\mathcal{D}_{t}}\mathcal{L}(V_{\bm{\theta}}(\bm{x}),y),$
(3)

where $\mathcal{D}_{b}\triangleq\mathcal{D}\backslash\mathcal{D}_{s}$ and
$\mathcal{L}(\cdot)$ is the loss function.

In this stage, how to select the style image is an important question.
Intuitively, it should be significantly different from those contained in the
original training set. In practice, defenders can simply adopt oil or sketch
paintings as the style image, since most of the images that need to be
protected are natural images. Defenders can also use other style images. we
will further discuss it in Section 7.1.

In particular, since we only modify a few samples and do not change their
labels, the embedding of external features will not hinder the functionality
of victim models or introduce new security risks ($e.g.$, hidden backdoors).

###  5.3 Training Ownership Meta-Classifier

Since there is no explicit expression of the embedded external features and
those features also have minor influences on the prediction, we need to train
an additional binary meta-classifier to determine whether the suspicious model
contains the knowledge of external features.

Under the white-box setting, we adopt the gradients of model weights as the
input to train the meta-classifier
$C_{\bm{w}}:\mathbb{R}^{|\bm{\theta}|}\rightarrow\\{-1,+1\\}$. In particular,
we assume that the victim model $V$ and the suspicious model $S$ have the same
model structure. This assumption can be easily satisfied since the defender
can retain a copy of the suspicious model on the training set of the victim
model if they have different structures. Once the suspicious model is
obtained, the defender will then train the benign version ($i.e.$, the $B$) of
the victim model on the benign training set $\mathcal{D}$. After that, we can
obtain the training set $\mathcal{D}_{c}$ of meta-classifier $C$ via

$\displaystyle\mathcal{D}_{c}=$
$\displaystyle\left\\{\left(g_{V}(\bm{x}^{\prime}),+1\right)|(\bm{x}^{\prime},y)\in\mathcal{D}_{t}\right\\}\cup$
(4)
$\displaystyle\left\\{\left(g_{B}(\bm{x}^{\prime}),-1\right)|(\bm{x}^{\prime},y)\in\mathcal{D}_{t}\right\\},$

where $\text{sgn}(\cdot)$ indicates the sign function [35],
$g_{V}(\bm{x}^{\prime})=\text{sgn}(\nabla_{\bm{\theta}}\mathcal{L}(V(\bm{x}^{\prime}),y))$,
and
$g_{B}(\bm{x}^{\prime})=\text{sgn}(\nabla_{\bm{\theta}}\mathcal{L}(B(\bm{x}^{\prime}),y))$.
In particular, we adopt its sign vector instead of the gradient itself to
highlight the influence of its direction.

Under the black-box setting, defenders can no longer obtain the model
gradients since they can not access model’s source files or intermediate
results. In this case, we adopt the difference between the predicted
probability vector of the transformed image and that of its benign version.
Specifically, let $V:\mathcal{X}\rightarrow[0,1]^{K}$ and
$B:\mathcal{X}\rightarrow[0,1]^{K}$ indicates the victim model and the benign
model, respectively. Assume that
$d_{V}(\bm{x},\bm{x}^{\prime})=V(\bm{x}^{\prime})-V(\bm{x})$ and
$d_{B}(\bm{x},\bm{x}^{\prime})=B(\bm{x}^{\prime})-B(\bm{x})$. In this case,
the training set $\mathcal{D}_{c}$ can be obtained via

$\displaystyle\mathcal{D}_{c}=$
$\displaystyle\left\\{\left(d_{V}(\bm{x},\bm{x}^{\prime}),+1\right)|(\bm{x}^{\prime},y)\in\mathcal{D}_{t}\right\\}\cup$
(5)
$\displaystyle\left\\{\left(d_{B}(\bm{x},\bm{x}^{\prime}),-1\right)|(\bm{x}^{\prime},y)\in\mathcal{D}_{t}\right\\},$

Different from MOVE under the white-box setting, we do not assume that the
victim model has the same model structure as the suspicious model under the
black-box setting, since only model predictions are needed in this case.

However, we found that directly using $\mathcal{D}_{c}$ defined in Eq.(5) is
not able to train a well-performed meta-classifier. This failure is mostly
because the probability differences contain significantly less information
than the model gradients. To alleviate this problem, we propose to introduce
data augmentations on the transformed image and concatenate their prediction
differences. Specifically, let $\bm{F}=\\{f_{aug}^{(i)}\\}_{i=1}^{N}$ denotes
$N$ given semantic and size preserving image transformations ($e.g.$,
flipping). The (augmented) training set is denoted as follows:

$\displaystyle\mathcal{D}_{c}^{aug}=$
$\displaystyle\left\\{\left(A_{V}(\bm{x},\bm{x}^{\prime}),+1\right)|(\bm{x}^{\prime},y)\in\mathcal{D}_{t}\right\\}\cup$
(6)
$\displaystyle\left\\{\left(A_{B}(\bm{x},\bm{x}^{\prime}),-1\right)|(\bm{x}^{\prime},y)\in\mathcal{D}_{t}\right\\},$

where

$A_{V}(\bm{x},\bm{x}^{\prime})=\text{CAT}\left(\left\\{d_{V}\left(\bm{x},f_{aug}^{(i)}(\bm{x}^{\prime})\right)\right\\}_{i=1}^{N}\right),$
(7)

$A_{B}(\bm{x},\bm{x}^{\prime})=\text{CAT}\left(\left\\{d_{B}\left(\bm{x},f_{aug}^{(i)}(\bm{x}^{\prime})\right)\right\\}_{i=1}^{N}\right),$
(8)

and ‘CAT’ is the concatenate function. Note that the dimensions of
$A_{V}(\bm{x},\bm{x}^{\prime})$ and $A_{V}(\bm{x},\bm{x}^{\prime})$ are both
$(1,K\times N)$. In this paper, we adopt five widespread transformations,
including 1) identical transformation, 2) horizontal flipping, 3) translation
towards the bottom right, 4) translation towards towards the right, and 5)
translation towards the bottom, for simplicity. We will discuss the potential
of using other transformations in our future work.

Once $\mathcal{D}_{c}$ is obtained, the meta-classifier $C_{\bm{w}}$ is
trained by

$\min_{\bm{w}}\sum_{(\bm{s},t)\in\mathcal{D}_{c}}\mathcal{L}(C_{\bm{w}}(\bm{s}),t).$
(9)

###  5.4 Ownership Verification with Hypothesis-Test

After training the meta-classifier $C$, the defenders can verify whether a
suspicious model is stolen from the victim simply by the result of meta-
classifier, based on a given transformed image $\bm{x}^{\prime}$. However, the
verification result may be sharply affected by the randomness of selecting
$\bm{x}^{\prime}$. In order to increase the verification confidence, we design
a hypothesis-test-based method, as follows:

######  Definition 3 (White-box Verification).

Let $\bm{X}^{\prime}$ is the variable of the transformed image, while
$\mu_{S}$ and $\mu_{B}$ denotes the posterior probability of event
$C(g_{S}(\bm{X}^{\prime}))=1$ and $C(g_{B}(\bm{X}^{\prime}))=1$, respectively.
Given a null hypothesis $H_{0}:\mu_{S}=\mu_{B}\ (H_{1}:\mu_{S}>\mu_{B})$, we
claim that the suspicious model $S$ is stolen from the victim if and only if
the $H_{0}$ is rejected.

######  Definition 4 (Black-box Verification).

Let $\bm{X}^{\prime}$ is the variable of the transformed image and $\bm{X}$
denotes the benign version of $\bm{X}^{\prime}$. Assume that $\mu_{S}$ and
$\mu_{B}$ indicates the posterior probability of event
$C\left(A_{S}(\bm{X},\bm{X}^{\prime})\right)=1$ and
$C\left(A_{B}(\bm{X},\bm{X}^{\prime})\right)=1$, respectively. Given a null
hypothesis $H_{0}:\mu_{S}=\mu_{B}\ (H_{1}:\mu_{S}>\mu_{B})$, we claim that the
suspicious model $S$ is stolen from the victim if and only if the $H_{0}$ is
rejected.

Specifically, we randomly sample $m$ different transformed images from
$\mathcal{D}_{t}$ to conduct the single-tailed pair-wise T-test [36] and
calculate its p-value. If the p-value is smaller than the significance level
$\alpha$, the null hypothesis $H_{0}$ is rejected. We also calculate the
_confidence score_ $\Delta\mu=\mu_{S}-\mu_{B}$ to represent the verification
confidence. The larger the $\Delta\mu$, the more confident the verification.

##  6 Main Experiments

###  6.1 Settings.

_Dataset and Model Selection._ We evaluate our defense on CIFAR-10 [25] and
ImageNet [37] dataset. CIFAR-10 contains 60,000 images (with size $3\times
32\times 32$) with 10 classes, including 50,000 training samples and 10,000
testing samples. ImageNet is a large-scale dataset and we use its subset
containing 20 random classes for simplicity and efficiency. Each class of the
subset contains 500 samples (with size $3\times 224\times 224$) for training
and 50 samples for testing. Following the settings of [11], we use the
WideResNet [38] and ResNet [27] as the victim model on CIFAR-10 and ImageNet,
respectively.

_Training Settings._ For the CIFAR-10 dataset, the training is conducted based
on the open-source codes222https://github.com/kuangliu/pytorch-cifar.
Specifically, both victim model and benign model are trained for 200 epochs
with SGD optimizer and an initial learning rate of 0.1, momentum of 0.9,
weight decay of 5 $\times 10^{-4}$, and batch size of 128. We decay the
learning rate with the cosine decay schedule [39] without a restart. We also
use data augmentation techniques including random crop and resize (with random
flip). For the ImageNet dataset, both the victim model and benign model are
trained for 200 epochs with SGD optimizer and an initial learning rate of
0.001, momentum of 0.9, weight decay of $1\times 10^{-4}$, and batch size of
32. The learning rate is decreased by a factor of 10 at epoch 150. All
training processes are performed on a single GeForce GTX 1080 Ti GPU.

TABLE III: Results of defenses on CIFAR-10 dataset under the white-box
setting. The best results among all defenses are indicated in boldface while
the failed verification cases are marked in red.

Model Stealing | BadNets |  | Gradient Matching |  | Entangled Watermarks |  | Dataset Inference |  | MOVE (Ours)
---|---|---|---|---|---|---|---|---|---
$\Delta\mu$ | p-value |  | $\Delta\mu$ | p-value |  | $\Delta\mu$ | p-value |  | $\Delta\mu$ | p-value |  | $\Delta\mu$ | p-value
$\mathcal{A}_{F}$ | Direct-copy | 0.91 | $10^{-12}$ |  | 0.88 | $10^{-12}$ |  | 0.99 | $\mathbf{10^{-35}}$ |  | - | $10^{-4}$ |  | 0.97 | $10^{-7}$
$\mathcal{A}_{D}$ | Distillation | $-10^{-3}$ | 0.32 |  | $10^{-7}$ | 0.20 |  | 0.01 | 0.33 |  | - | $10^{-4}$ |  | 0.53 | $\mathbf{10^{-7}}$
$\mathcal{A}_{M}$ | Zero-shot | $10^{-25}$ | 0.22 |  | $10^{-24}$ | 0.22 |  | $10^{-3}$ | $10^{-3}$ |  | - | $10^{-2}$ |  | 0.52 | $\mathbf{10^{-5}}$
Fine-tuning | $10^{-23}$ | 0.28 |  | $10^{-27}$ | 0.28 |  | 0.35 | 0.01 |  | - | $10^{-5}$ |  | 0.50 | $\mathbf{10^{-6}}$
$\mathcal{A}_{Q}$ | Label-query | $10^{-27}$ | 0.20 |  | $10^{-30}$ | 0.34 |  | $10^{-5}$ | 0.62 |  | - | $10^{-3}$ |  | 0.52 | $\mathbf{10^{-4}}$
Logit-query | $10^{-27}$ | 0.23 |  | $10^{-23}$ | 0.33 |  | $10^{-6}$ | 0.64 |  | - | $10^{-3}$ |  | 0.54 | $\mathbf{10^{-4}}$
Benign | Independent | $10^{-20}$ | 0.33 |  | $10^{-12}$ | 0.99 |  | $10^{-22}$ | 0.68 |  | - | $10^{-31}$ |  | 0.00 | 1.00

TABLE IV: Results of defenses on ImageNet dataset under the white-box setting.
The best results among all defenses are indicated in boldface while the failed
verification cases are marked in red.

Model Stealing | BadNets |  | Gradient Matching |  | Entangled Watermarks |  | Dataset Inference |  | MOVE (Ours)
---|---|---|---|---|---|---|---|---|---
$\Delta\mu$ | p-value |  | $\Delta\mu$ | p-value |  | $\Delta\mu$ | p-value |  | $\Delta\mu$ | p-value |  | $\Delta\mu$ | p-value
$\mathcal{A}_{F}$ | Direct-copy | 0.87 | $10^{-10}$ |  | 0.77 | $10^{-10}$ |  | 0.99 | $\mathbf{10^{-25}}$ |  | - | $10^{-6}$ |  | 0.90 | $10^{-5}$
$\mathcal{A}_{D}$ | Distillation | $10^{-4}$ | 0.43 |  | $10^{-12}$ | 0.43 |  | $10^{-6}$ | 0.19 |  | - | $10^{-3}$ |  | 0.61 | $\mathbf{10^{-5}}$
$\mathcal{A}_{M}$ | Zero-shot | $10^{-12}$ | 0.33 |  | $10^{-18}$ | 0.43 |  | $10^{-3}$ | 0.46 |  | - | $10^{-3}$ |  | 0.53 | $\mathbf{10^{-4}}$
Fine-tuning | $10^{-20}$ | 0.20 |  | $10^{-12}$ | 0.47 |  | 0.46 | 0.01 |  | - | $10^{-4}$ |  | 0.60 | $\mathbf{10^{-5}}$
$\mathcal{A}_{Q}$ | Label-query | $10^{-23}$ | 0.29 |  | $10^{-22}$ | 0.50 |  | $10^{-7}$ | 0.45 |  | - | $\bm{10^{-3}}$ |  | 0.55 | $\mathbf{10^{-3}}$
Logit-query | $10^{-23}$ | 0.38 |  | $10^{-12}$ | 0.22 |  | $10^{-6}$ | 0.36 |  | - | $10^{-3}$ |  | 0.55 | $\mathbf{10^{-4}}$
Benign | Independent | $10^{-24}$ | 0.38 |  | $10^{-23}$ | 0.78 |  | $\mathbf{10^{-30}}$ | 0.55 |  | - | $10^{-10}$ |  | $10^{-5}$ | 0.99

_Settings for Model Stealing._ Following the settings in [11], we conduct
model stealing attacks illustrated in Section 2.1 to evaluate the
effectiveness of different defenses. Besides, we also provide the results of
examining a suspicious model which is not stolen from the victim (dubbed
‘Independent’) for reference. Specifically, we implement the model
distillation (dubbed ‘Distillation’) [13] based on its open-sourced
codes333https://github.com/thaonguyen19/ModelDistillation-PyTorch. The stolen
model is trained with SGD optimizer and an initial learning rate of 0.1,
momentum of 0.9, and weight decay of $10^{-4}$; The zero-shot learning based
data-free distillation (dubbed ‘Zero-shot’) [14] is implemented based on the
open-source codes444https://github.com/VainF/Data-Free-Adversarial-
Distillation. The stealing process is performed for 200 epochs with SGD
optimizer and a learning rate of 0.1, momentum of 0.9, weight decay of
$5\times 10^{-4}$, and batch size of 256; For the fine-tuning, the adversaries
obtain stolen models by fine-tuning victim models on different datasets.
Following the settings in [11], we randomly select 500,000 samples from the
original TinyImages [40] as the substitute data to fine-tune the victim model
for experiments on CIFAR-10. For the ImageNet experiments, we randomly choose
samples with other 20 classes from the original ImageNet as the substitute
data. We fine-tune the victim model for 5 epochs to obtain the stolen model;
For the label-query attack (dubbed ’Label-query’), we train the stolen model
for 20 epochs with a substitute dataset labeled by the victim model; For the
logit-query attack (dubbed ’Logit-query’), we train the stolen model by
minimizing the KL-divergence between its outputs ($i.e.$, logits) and those of
the victim model for 20 epochs.

_Settings for Defenses._ We compare our defense with dataset inference [11]
and model watermarking [20] with BadNets [22], gradient matching [41], and
entangled watermarks [10]. We poison 10% training samples for all defenses.
Besides, we adopt a white square in the lower right corner as the trigger
pattern for BadNets and adopt an oil paint as the style image for our defense.
Other settings are the same as those used in their original paper. We
implement BadNets based on BackdoorBox [42] and other methods based on their
official open-sourced codes. An example of images ($e.g.$, poisoned images and
the style image) involved in different defenses is shown in Figure 3. In
particular, similar to our method, dataset inference uses different methods
under different settings, while other baseline defenses are designed under the
black-box setting. Since methods designed under the black-box setting can also
be used in the white-box scenarios, we also compare our MOVE with them under
the white-box setting.

_Evaluation Metric._ We use the confidence score $\Delta\mu$ and p-value as
the metric for our evaluation. Both of them are calculated based on the
hypothesis-test with 10 sampled images under the white-box setting and 100
images under the black-box setting. In particular, except for the independent
sources (which should not be regarded as stolen), the smaller the p-value and
the larger the $\Delta\mu$, the better the defense. For the independent ones,
the larger the p-value and the smaller the $\Delta\mu$, the better the method.
Among all defenses, the best result is indicated in boldface and the failed
verification cases are marked in red.

###  6.2 Results of MOVE under the White-box Setting

As shown in the Table III-IV, MOVE achieves the best performance in almost all
cases under the white-box setting. For example, the p-value of our method is
three orders of magnitude smaller than that of the dataset inference and six
orders of magnitude smaller than that of the model watermarking in defending
against the distillation-based model stealing on the CIFAR-10 dataset. The
only exceptions appear when defending against the fully-accessible attack. In
these cases, entangled watermarks based model watermarking has some
advantages. Nevertheless, our method can still easily make correct predictions
in these cases. In particular, our MOVE defense is the only method that can
effectively identify whether there is model stealing in all cases. Other
defenses either fail under many complicated attacks ($e.g.$, query-only
attacks) or misjudge when there is no stealing. Besides, our defense has minor
adverse effects on the performance of victim models. The accuracy of the model
trained on benign CIFAR-10 and its transformed version is 91.79% and 91.99%,
respectively; The accuracy of the model trained on benign ImageNet and its
transformed version is 82.40% and 80.40%, respectively. This is mainly because
we do not change the label of transformed images and therefore the
transformation can be treated as data augmentation, which is mostly harmless.

TABLE V: Results of defenses on CIFAR-10 dataset under the black-box setting.
The best results among all defenses are indicated in boldface while the failed
verification cases are marked in red.

Model Stealing | BadNets |  | Gradient Matching |  | Entangled Watermarks |  | Dataset Inference |  | MOVE (Ours)
---|---|---|---|---|---|---|---|---|---
$\Delta\mu$ | p-value |  | $\Delta\mu$ | p-value |  | $\Delta\mu$ | p-value |  | $\Delta\mu$ | p-value |  | $\Delta\mu$ | p-value
$\mathcal{A}_{F}$ | Direct-copy | 0.79 | $10^{-60}$ |  | 0.03 | $10^{-5}$ |  | 0.54 | $10^{-35}$ |  | - | $10^{-39}$ |  | 0.84 | $\mathbf{10^{-74}}$
$\mathcal{A}_{D}$ | Distillation | $-10^{-3}$ | 0.02 |  | 0.01 | 0.14 |  | 0.01 | $10^{-3}$ |  | - | 0.32 |  | 0.54 | $\mathbf{10^{-24}}$
$\mathcal{A}_{M}$ | Zero-shot | $-10^{-25}$ | 0.29 |  | $10^{-4}$ | $10^{-3}$ |  | $10^{-3}$ | 0.17 |  | - | $10^{-5}$ |  | 0.39 | $\mathbf{10^{-15}}$
Fine-tuning | $10^{-3}$ | 0.08 |  | $10^{-3}$ | 0.17 |  | $10^{-3}$ | 0.05 |  | - | $10^{-6}$ |  | 0.37 | $\mathbf{10^{-14}}$
$\mathcal{A}_{Q}$ | Label-query | $10^{-4}$ | 0.11 |  | 0.01 | 0.11 |  | $10^{-3}$ | 0.05 |  | - | $\mathbf{10^{-7}}$ |  | 0.07 | $10^{-3}$
Logit-query | $10^{-3}$ | 0.10 |  | $10^{-23}$ | 0.08 |  | $10^{-35}$ | 0.11 |  | - | $10^{-3}$ |  | 0.17 | $\mathbf{10^{-6}}$
Benign | Independent | $10^{-20}$ | 0.33 |  | $10^{-12}$ | 1.00 |  | 0.00 | 1.00 |  | - | $10^{-38}$ |  | $10^{-4}$ | 0.98

TABLE VI: Results of defenses on ImageNet dataset under the black-box setting.
The best results among all defenses are indicated in boldface while the failed
verification cases are marked in red.

Model Stealing | BadNets |  | Gradient Matching |  | Entangled Watermarks |  | Dataset Inference |  | MOVE (Ours)
---|---|---|---|---|---|---|---|---|---
$\Delta\mu$ | p-value |  | $\Delta\mu$ | p-value |  | $\Delta\mu$ | p-value |  | $\Delta\mu$ | p-value |  | $\Delta\mu$ | p-value
$\mathcal{A}_{F}$ | Direct-copy | 0.87 | $10^{-10}$ |  | 0.32 | $10^{-35}$ |  | 0.99 | $10^{-48}$ |  | - | $10^{-65}$ |  | 0.91 | $\mathbf{10^{-135}}$
$\mathcal{A}_{D}$ | Distillation | $10^{-4}$ | 0.09 |  | $10^{-4}$ | 0.31 |  | $10^{-5}$ | 0.06 |  | - | $10^{-50}$ |  | 0.84 | $\mathbf{10^{-84}}$
$\mathcal{A}_{M}$ | Zero-shot | 0.01 | $10^{-4}$ |  | $10^{-3}$ | 0.30 |  | $10^{-3}$ | 0.33 |  | - | $10^{-65}$ |  | 0.88 | $\mathbf{10^{-113}}$
Fine-tuning | 0.01 | $10^{-4}$ |  | $10^{-12}$ | 0.21 |  | 0.01 | 0.19 |  | - | $10^{-55}$ |  | 0.74 | $\mathbf{10^{-62}}$
$\mathcal{A}_{Q}$ | Label-query | $10{-3}$ | 0.26 |  | $10^{-3}$ | 0.09 |  | $10^{-3}$ | 0.27 |  | - | $\mathbf{10^{-50}}$ |  | 0.43 | $10^{-12}$
Logit-query | $10^{-3}$ | 0.11 |  | $10^{-12}$ | 0.15 |  | $10^{-3}$ | 0.24 |  | - | $\mathbf{10^{-54}}$ |  | 0.61 | $10^{-37}$
Benign | Independent | $10^{-24}$ | 0.38 |  | $10^{-5}$ | 0.28 |  | $\mathbf{10^{-30}}$ | 0.99 |  | - | $10^{-50}$ |  | $10^{-4}$ | 0.97

TABLE VII: Defending against multi-stage stealing.

Setting$\downarrow$ | Stage$\rightarrow$ | Stage 0 | Stage 1 | Stage 2
---|---|---|---|---
White-box | Method$\rightarrow$ | Direct-copy | Zero-shot | Zero-shot
p-value$\rightarrow$ | $10^{-7}$ | $10^{-5}$ | $10^{-4}$
Method$\rightarrow$ | Direct-copy | Logit-query | Zero-shot
p-value$\rightarrow$ | $10^{-7}$ | $10^{-4}$ | 0.01
Black-box | Method$\rightarrow$ | Direct-copy | Zero-shot | Zero-shot
p-value$\rightarrow$ | $10^{-74}$ | $10^{-15}$ | $10^{-5}$
Method$\rightarrow$ | Direct-copy | Logit-query | Zero-shot
p-value$\rightarrow$ | $10^{-74}$ | $10^{-6}$ | $10^{-3}$

###  6.3 Results of MOVE under the Black-box Setting

As shown in Table V-VI, our MOVE defense still reaches promising performance
under the black-box setting. For example, the p-value of our defense is over
twenty and sixty orders of magnitude smaller than that of the dataset
inference and entangled watermarks in defending against direct-copy on
CIFAR-10 and ImageNet, respectively. In those cases where we do not get the
best performance ($e.g.$, label-query and logit-query), our defense is usually
the second-best where it can still easily make correct predictions. In
particular, same as the scenarios under the white-box setting, our MOVE
defense is the only method that can effectively identify whether there is
stealing in all cases. Other defenses either fail under many complicated
stealing attacks or misjudge when there is no model stealing. These results
verify the effectiveness of our defense again.

###  6.4 Defending against Multi-Stage Model Stealing

In previous experiments, the stolen model is obtained by a single stealing. In
this section, we explore whether our method is still effective if there are
multiple stealing stages.

_Settings._ We discuss two types of multi-stage stealing on the CIFAR-10
dataset, including stealing with the same attack and model structure and
stealing with different attacks and model structures. In general, the first
one is the easiest multi-stage attack while the second one is the hardest.
Other settings are the same as those in Section 6.1.

_Results._ As shown in Table VII, the p-value increases with the increase of
the stage, which indicates that defending against multi-stage attacks is
difficult. Nevertheless, the p-value $\leq 0.01$ in all cases under both
white-box and black-box settings, $i.e.$, our method can identify the
existence of model stealing even after multiple stealing stages. Besides, the
p-value in defending the second type of multi-stage attack is significantly
larger than that of the first one, showing that the second task is harder.

##  7 Discussion

In this section, we further explore the mechanisms and properties of our MOVE.
Unless otherwise specified, all settings are the same as those in Section 6.

(a) MOVE under the White-box Setting (b) MOVE under the Black-box Setting

Figure 4: The effects of the transformation rate (%) and the number of sampled images of our MOVE on the CIFAR-10 dataset. TABLE VIII: The effectiveness of our defense with different style images on CIFAR-10 dataset. Method$\rightarrow$ | MOVE under the White-box Setting | MOVE under the Black-box Setting
---|---|---
Pattern$\rightarrow$ | Pattern (a) | Pattern (b) | Pattern (a) | Pattern (b)
Stealing Attack$\downarrow$, Metric$\rightarrow$ | $\Delta\mu$ | p-value | $\Delta\mu$ | p-value | $\Delta\mu$ | p-value | $\Delta\mu$ | p-value
Direct-copy | 0.98 | $10^{-10}$ | 0.98 | $10^{-12}$ | 0.92 | $10^{-126}$ | 0.93 | $10^{-112}$
Distillation | 0.72 | $10^{-8}$ | 0.63 | $10^{-7}$ | 0.71 | $10^{-41}$ | 0.91 | $10^{-100}$
Zero-shot | 0.74 | $10^{-8}$ | 0.67 | $10^{-7}$ | 0.67 | $10^{-37}$ | 0.65 | $10^{-33}$
Fine-tuning | 0.21 | $10^{-7}$ | 0.50 | $10^{-9}$ | 0.19 | $10^{-7}$ | 0.21 | $10^{-9}$
Label-query | 0.68 | $10^{-8}$ | 0.68 | $10^{-7}$ | 0.81 | $10^{-53}$ | 0.74 | $10^{-41}$
Logit-query | 0.62 | $10^{-6}$ | 0.73 | $10^{-7}$ | 0.6 | $10^{-28}$ | 0.23 | $10^{-9}$
Independent | 0.00 | 1.00 | $10^{-9}$ | 0.99 | $10^{-4}$ | 0.95 | $10^{-4}$ | 0.96

###  7.1 Effects of Key Hyper-parameters

In this part, we discuss the effects of hyper-parameters and components
involved in our method.

####  7.1.1 Effects of Transformation Rate

In general, the larger the transformation rate $\gamma$, the more training
samples are transformed during the training process of the victim model and
therefore the ‘stronger’ the external features. As shown in Figure 4, as we
expected, the p-value decreases with the increase of $\gamma$ in defending all
stealing methods under both white-box. In other words, increasing the
transformation rate can increase the performance of model verification.
However, under the black-box setting, the changes in p-Value are relatively
irregular. We speculate that this is probably because using prediction
differences to approximately learn external features under the black-box
setting has limited effects. Nevertheless, our method can successfully defend
against all stealing attacks in all cases (p-value $\ll 0.01$). Besides, we
need to emphasize that the increase of $\gamma$ may also lead to the accuracy
decrease of victim models. Defenders should specify this hyper-parameter based
on their specific requirements in practice.

Figure 5: The new style images adopted for the evaluation.

####  7.1.2 Effects of the Number of Sampled Images

Recall that our method needs to specify the number of sampled (transformed)
images ($i.e.$, the $m$) adopted in the hypothesis-based verification. In
general, the larger the $m$, the less the adverse effects of the randomness
involved in this process and therefore the more confident the verification.
This is probably the main reason why the p-value also decreases with the
increase of $m$, as shown in Figure 4.

####  7.1.3 Effects of Style Images

In this part, we examine whether the proposed defense is still effective if we
adopt other style images (as shown in Figure 5). As shown in Table VIII, the
p-value $\ll 0.01$ in all cases under attack, while it is nearly 1 when there
is no stealing, no matter under white-box or black-box setting. In other
words, our method remains effective in defending against different stealing
methods when different style images are used, although there will be some
fluctuations in the results. We will further explore how to optimize the
selection of style images in our future work.

TABLE IX: The effectiveness (p-value) of style transfer.

| CIFAR-10 |  | ImageNet
---|---|---|---
|  | Patch-based
---
Variant
Ours |  |  | Patch-based
---
Variant
Ours
Direct-copy | $\mathbf{10^{-74}}$ | $10^{-7}$ |  | 0.01 | $\mathbf{10^{-5}}$
Distillation | 0.17 | $\mathbf{10^{-7}}$ |  | 0.13 | $\mathbf{10^{-5}}$
Zero-shot | 0.01 | $\mathbf{10^{-5}}$ |  | $10^{-3}$ | $\mathbf{10^{-4}}$
Fine-tuning | $10^{-3}$ | $\mathbf{10^{-6}}$ |  | $10^{-3}$ | $\mathbf{10^{-5}}$
Label-query | $10^{-3}$ | $\mathbf{10^{-4}}$ |  | 0.02 | $\mathbf{10^{-3}}$
Logit-query | $10^{-3}$ | $\mathbf{10^{-4}}$ |  | 0.01 | $\mathbf{10^{-4}}$

###  7.2 The Ablation Study

There are three key components contained in our MOVE, including 1) embedding
external features with style transfer, 2) using the sign vector of gradients
instead of gradients themselves, and 3) using meta-classifier for
verification. In this section, we verify their effectiveness. For simplicity,
we use MOVE under the white-box setting for discussion.

TABLE X: The performance of our meta-classifier trained with different
features.

Dataset$\rightarrow$ | CIFAR-10 | ImageNet
---|---|---
Model Stealing$\downarrow$ | Gradient | Sign of Gradient (Ours) | Gradient | Sign of Gradient (Ours)
$\Delta\mu$ | p-value | $\Delta\mu$ | p-value | $\Delta\mu$ | p-value | $\Delta\mu$ | p-value
Direct-copy | 0.44 | $10^{-5}$ | $\bm{0.97}$ | $\bm{10^{-7}}$ | 0.15 | $10^{-4}$ | $\bm{0.90}$ | $\bm{10^{-5}}$
Distillation | 0.27 | 0.01 | $\bm{0.53}$ | $\bm{10^{-7}}$ | 0.15 | $10^{-4}$ | $\bm{0.61}$ | $\bm{10^{-5}}$
Zero-shot | 0.03 | $10^{-3}$ | $\bm{0.52}$ | $\bm{10^{-5}}$ | 0.12 | $10^{-3}$ | $\bm{0.53}$ | $\bm{10^{-4}}$
Fine-tuning | 0.04 | $10^{-5}$ | $\bm{0.50}$ | $\bm{10^{-6}}$ | 0.13 | $10^{-3}$ | $\bm{0.60}$ | $\bm{10^{-5}}$
Label-query | 0.08 | $10^{-3}$ | $\bm{0.52}$ | $\bm{10^{-4}}$ | 0.13 | $10^{-3}$ | $\bm{0.55}$ | $\bm{10^{-3}}$
Logit-query | 0.07 | $10^{-5}$ | $\bm{0.54}$ | $\bm{10^{-4}}$ | 0.12 | $10^{-3}$ | $\bm{0.55}$ | $\bm{10^{-4}}$
Independent | $\bm{0.00}$ | $\bm{1.00}$ | $\bm{0.00}$ | $\bm{1.00}$ | $\bm{10^{-10}}$ | $\bm{0.99}$ | $10^{-5}$ | $\bm{0.99}$

####  7.2.1 The Effectiveness of Style Transfer

To verify that the style watermark transfers better during the stealing
process, we compare our method with its variant which uses the white-square
patch (adopted in BadNets) to generate transformed images. As shown in Table
IX, our method is significantly better than its patch-based variant. It is
probably because DNNs are easier to learn the texture information [29] and the
style watermark is bigger than the patch one. This phenomenon partly explains
why our method works well.

TABLE XI: The effectiveness (p-value) of meta-classifier.

| CIFAR-10 |  | ImageNet
---|---|---|---
| w/o | w/ |  | w/o | w/
Direct-copy | $10^{-12}$ | $\mathbf{10^{-37}}$ |  | $10^{-10}$ | $\mathbf{10^{-11}}$
Distillation | 0.32 | $\mathbf{10^{-3}}$ |  | 0.43 | $\mathbf{10^{-3}}$
Zero-shot | 0.22 | $\mathbf{10^{-61}}$ |  | 0.33 | $\mathbf{10^{-3}}$
Fine-tuning | 0.28 | $\mathbf{10^{-5}}$ |  | 0.20 | $\mathbf{10^{-13}}$
Label-query | 0.20 | $\mathbf{10^{-50}}$ |  | 0.29 | $\mathbf{10^{-3}}$
Logit-query | 0.23 | $\mathbf{10^{-3}}$ |  | 0.38 | $\mathbf{10^{-3}}$

####  7.2.2 The Effectiveness of Sign Function

In this section, we compare our method with its variant which directly adopts
the gradients to train the meta-classifier. As shown in Table X, using the
sign of gradients is significantly better than using gradients directly. This
is probably because the ‘direction’ of gradients contains more information
compared with their ‘magnitude’. We will further explore it in the future.

####  7.2.3 The Effectiveness of Meta-Classifier

To verify that the meta-classifier is also useful, we compare the BadNets-
based model watermarking with its extension, which also uses the meta-
classifier (adopted in our MOVE defense) for ownership verification. In this
case, the victim model is the backdoored one and the transformed image is the
one containing backdoor triggers. As shown in Table XI, adopting meta-classier
significantly decrease the p-value in almost all cases, which verifies the
effectiveness of the meta-classifier. These results also partly explains the
effectiveness of our MOVE defense.

###  7.3 Resistance to Potential Adaptive Attacks

In this section, we discuss the resistance of our defense to potential
adaptive attacks. We take MOVE under the white-box setting as an example to
discuss for simplicity.

####  7.3.1 Resistance to Model Fine-tuning

In many cases, attackers have some benign local samples. As such, they may
first fine-tuning the stolen model before deployment. This adaptive attack
might be effective since the fine-tuning process may alleviate the effects of
transformed images due to the catastrophic forgetting [43] of DNNs.
Specifically, we adopt $p\%$ benign training samples (where
$p\in\\{0,5,10,15,20\\}$ is dubbed _sample rate_) to fine-tune the whole
stolen models generated by distillation, zero-shot, label-query, and logit-
query. We also adopt the p-value and $\Delta\mu$ to measure the defense
effectiveness.

As shown in Figure 6, the p-value increases while the $\Delta\mu$ decreases
with the increase of sample rate. These results indicate that model fine-
tuning has some benefits in reducing our defense effectiveness. However, our
defense can still successfully detect all stealing attacks (p-value $<0.01$
and $\Delta\mu\gg 0$) even the sample rate is set to $20\%$. In other words,
our defense is resistant to model fine-tuning.

(a) CIFAR-10 (b) ImageNet

Figure 6: Resistance to the adaptive attack based on model fine-tuning with
different sample rates.

Figure 7: Salience maps of BadNets and our MOVE. (a)-(b): salience maps of
poisoned images generated by BadNets. (c)-(d): salience maps of transformed
images generated by MOVE. (a)&(c): salience maps of images from CIFAR-10
dataset. (b)&(d): salience maps of images from ImageNet dataset.

####  7.3.2 Resistance to Saliency-based Backdoor Detections

These methods [44, 45, 46] use the saliency map to identify and remove
potential trigger regions. Specifically, they first generate the saliency map
of each sample and then calculate trigger regions based on the intersection of
all saliency maps. As shown in Figure 7, the Grad-CAM mainly focuses on the
trigger regions of images in BadNets while it mainly focuses on the object
outline towards images in our proposed defense. These results indicate that
our MOVE is resistant to saliency-based backdoor detections.

####  7.3.3 Resistance to STRIP

This method [47] detects and filters poisoned samples based on the prediction
randomness of samples generated by imposing various image patterns on the
suspicious image. The randomness is measured by the entropy of the average
prediction of those samples. _The higher the entropy, the harder an method for
STRIP to detect_. We compare the average entropy of all poisoned images in
BadNets and that of all transformed images in our defense. As shown in Table
XII, the entropy of our defense is significantly larger than that of the
BadNets-based method. These results show that our defense is resistant to
STRIP.

TABLE XII: Resistance (entropy) of BadNets-based model watermarking and our defense to STRIP.  CIFAR-10 | ImageNet
---|---
BadNets | Ours | BadNets | Ours
0.01 | 1.15 | 0.01 | 0.77

####  7.3.4 Resistance to Trigger Synthesis based Detections

These methods [48, 49, 50] detect poisoned images by reversing potential
triggers contained in given suspicious DNNs. They have a latent assumption
that the triggers should be sample-agnostic and the attack should be targeted.
However, our defense does not satisfy these assumptions since the
perturbations in our transformed images are sample-specific and we do not
modify the label of those images. As shown in Figure 8, synthesized triggers
of the BadNets-based method contain similar patterns to those used by
defenders ($i.e.$, white-square on the bottom right corner) or its flipped
version, whereas those of our method are meaningless. These results show that
our defense is also resistant to these detection methods.

Figure 8: Reversed triggers. First Row: triggers of BadNets-based defense.
Second Row: triggers of our method. First Column: triggers on CIFAR-10
dataset. Second Column: triggers on ImageNet dataset.

###  7.4 Relations with Related Methods

####  7.4.1 Relations with Membership Inference Attacks

Membership inference attacks [17, 18, 19] intend to identify whether a
particular sample is used to train given DNNs. Similar to these attacks, our
method also adopts some training samples for ownership verification. However,
our defense aims to analyze whether the suspicious model contains the
knowledge of external features rather than whether the model is trained on
those transformed samples. To verify it, we design additional experiments, as
follows:

_Settings._ For simplicity, we compare our method with its variant which
adopts testing instead of training samples to generate transformed images used
in ownership verification under the white-box setting. Except for that, other
settings are the same as those stated in Section 6.1.

_Results._ As shown in Table XIII, using testing images has similar effects to
that of using training images in generating transformed images for ownership
verification, although there are some small fluctuations. In other words, our
method indeed examines the information of embedded external features rather
than simply verifying whether the transformed images are used for training.
These results verify that our defense is fundamentally different from
membership inference attacks.

TABLE XIII: Results of ownership verification with transformed samples
generated by training and testing samples.

Dataset$\rightarrow$ | CIFAR-10 | ImageNet
---|---|---
Model Stealing$\downarrow$ | Training Set | Testing Set | Training Set | Testing Set
$\Delta\mu$ | p-value | $\Delta\mu$ | p-value | $\Delta\mu$ | p-value | $\Delta\mu$ | p-value
Direct-copy | 0.97 | $10^{-7}$ | 0.96 | $10^{-7}$ | 0.90 | $10^{-5}$ | 0.93 | $10^{-7}$
Distillation | 0.53 | $10^{-7}$ | 0.53 | $10^{-5}$ | 0.61 | $10^{-5}$ | 0.42 | $10^{-5}$
Zero-shot | 0.52 | $10^{-5}$ | 0.53 | $10^{-5}$ | 0.53 | $10^{-4}$ | 0.34 | $10^{-3}$
Fine-tuning | 0.50 | $10^{-6}$ | 0.47 | $10^{-6}$ | 0.60 | $10^{-5}$ | 0.72 | $10^{-5}$
Label-query | 0.52 | $10^{-4}$ | 0.52 | $10^{-4}$ | 0.55 | $10^{-3}$ | 0.40 | $10^{-3}$
Logit-query | 0.54 | $10^{-4}$ | 0.53 | $10^{-4}$ | 0.55 | $10^{-4}$ | 0.48 | $10^{-4}$
Independent | 0.00 | 1.00 | 0.00 | 1.00 | $10^{-5}$ | 0.99 | $10^{-9}$ | 0.99

####  7.4.2 Relations with Backdoor Attacks

Similar to that of (poisoning-based) backdoor attacks [51, 52, 53], our
defense embeds pre-defined distinctive behaviors into DNNs via modifying some
training samples. However, different from that of backdoor attacks, our method
neither changes the label of poisoned samples nor only selects training
samples with the specific category for poisoning. Accordingly, our defense
will not introduce any hidden backdoor into the trained victim model. In other
words, the dataset watermarking of our MOVE is also fundamentally different
from backdoor attacks. This is probably the main reason why most existing
backdoor defenses will have minor benefits in designing adaptive attacks
against our defense, as illustrated in Section 7.3.

##  8 Conclusion

In this paper, we revisited the defenses against model stealing from the
perspective of model ownership verification. We revealed that existing
defenses suffered from low effectiveness and may even introduced additional
security risks. Based on our analysis, we proposed a new effective and
harmless model ownership verification ($i.e.$, MOVE), which examined whether
the suspicious model contains the knowledge of defender-specified external
features. We embedded external features by modifying a few training samples
with style transfer without changing their label. In particular, we developed
our MOVE defense under both white-box and black-box settings to provide
comprehensive model protection. We evaluated our defense on both CIFAR-10 and
ImageNet datasets. The experiments verified that our method can defend against
various types of model stealing simultaneously while preserving high accuracy
in predicting benign samples.

## References

  * [1] Y. LeCun, Y. Bengio, and G. Hinton, “Deep learning,” _nature_ , vol. 521, no. 7553, pp. 436–444, 2015.
  * [2] R. Feng, S. Chen, X. Xie, G. Meng, S.-W. Lin, and Y. Liu, “A performance-sensitive malware detection system using deep learning on mobile devices,” _IEEE Transactions on Information Forensics and Security_ , vol. 16, pp. 1563–1578, 2020.
  * [3] S. Minaee, Y. Y. Boykov, F. Porikli, A. J. Plaza, N. Kehtarnavaz, and D. Terzopoulos, “Image segmentation using deep learning: A survey,” _IEEE Transactions on Pattern Analysis and Machine Intelligence_ , 2021.
  * [4] F. Tramèr, F. Zhang, A. Juels, M. K. Reiter, and T. Ristenpart, “Stealing machine learning models via prediction apis,” in _USENIX Security_ , 2016\.
  * [5] T. Orekondy, B. Schiele, and M. Fritz, “Knockoff nets: Stealing functionality of black-box models,” in _CVPR_ , 2019.
  * [6] V. Chandrasekaran, K. Chaudhuri, I. Giacomelli, S. Jha, and S. Yan, “Exploring connections between active learning and model extraction,” in _USENIX Security_ , 2020.
  * [7] T. Lee, B. Edwards, I. Molloy, and D. Su, “Defending against neural network model stealing attacks using deceptive perturbations,” in _IEEE S &P Workshop_, 2019.
  * [8] S. Kariyappa and M. K. Qureshi, “Defending against model stealing attacks with adaptive misinformation,” in _CVPR_ , 2020.
  * [9] Y. Li, Z. Zhang, J. Bai, B. Wu, Y. Jiang, and S.-T. Xia, “Open-sourced dataset protection via backdoor watermarking,” in _NeurIPS Workshop_ , 2020.
  * [10] H. Jia, C. A. Choquette-Choo, V. Chandrasekaran, and N. Papernot, “Entangled watermarks as a defense against model extraction,” in _USENIX Security_ , 2021.
  * [11] P. Maini, M. Yaghini, and N. Papernot, “Dataset inference: Ownership resolution in machine learning,” in _ICLR_ , 2021.
  * [12] Y. Li, L. Zhu, X. Jia, Y. Jiang, S.-T. Xia, and X. Cao, “Defending against model stealing via verifying embedded external features,” in _AAAI_ , 2022\.
  * [13] G. Hinton, O. Vinyals, and J. Dean, “Distilling the knowledge in a neural network,” in _NeurIPS Workshop_ , 2014.
  * [14] G. Fang, J. Song, C. Shen, X. Wang, D. Chen, and M. Song, “Data-free adversarial distillation,” _arXiv preprint arXiv:1912.11006_ , 2019.
  * [15] N. Papernot, P. McDaniel, I. Goodfellow, S. Jha, Z. B. Celik, and A. Swami, “Practical black-box attacks against machine learning,” in _AsiaCCS_ , 2017\.
  * [16] M. Jagielski, N. Carlini, D. Berthelot, A. Kurakin, and N. Papernot, “High accuracy and high fidelity extraction of neural networks,” in _USENIX Security_ , 2020.
  * [17] R. Shokri, M. Stronati, C. Song, and V. Shmatikov, “Membership inference attacks against machine learning models,” in _IEEE S &P_, 2017.
  * [18] K. Leino and M. Fredrikson, “Stolen memories: Leveraging model memorization for calibrated white-box membership inference,” in _USENIX Security_ , 2020\.
  * [19] B. Hui, Y. Yang, H. Yuan, P. Burlina, N. Z. Gong, and Y. Cao, “Practical blind membership inference attack via differential comparisons,” in _NDSS_ , 2021\.
  * [20] Y. Adi, C. Baum, M. Cisse, B. Pinkas, and J. Keshet, “Turning your weakness into a strength: Watermarking deep neural networks by backdooring,” in _USENIX Security_ , 2018.
  * [21] Y. Li, Y. Jiang, Z. Li, and S.-T. Xia, “Backdoor learning: A survey,” _arXiv preprint arXiv:2007.08745_ , 2020.
  * [22] T. Gu, K. Liu, B. Dolan-Gavitt, and S. Garg, “Badnets: Evaluating backdooring attacks on deep neural networks,” _IEEE Access_ , vol. 7, pp. 47 230–47 244, 2019.
  * [23] Y. Li, Y. Li, B. Wu, L. Li, R. He, and S. Lyu, “Invisible backdoor attack with sample-specific triggers,” in _ICCV_ , 2021.
  * [24] R. Rivest and S. Dusse, “The md5 message-digest algorithm,” 1992.
  * [25] A. Krizhevsky, G. Hinton _et al._ , “Learning multiple layers of features from tiny images,” Citeseer, Tech. Rep., 2009.
  * [26] K. Simonyan and A. Zisserman, “Very deep convolutional networks for large-scale image recognition,” in _ICLR_ , 2015.
  * [27] K. He, X. Zhang, S. Ren, and J. Sun, “Deep residual learning for image recognition,” in _CVPR_ , 2016.
  * [28] S.-M. Moosavi-Dezfooli, A. Fawzi, O. Fawzi, and P. Frossard, “Universal adversarial perturbations,” in _CVPR_ , 2017.
  * [29] R. Geirhos, P. Rubisch, C. Michaelis, M. Bethge, F. A. Wichmann, and W. Brendel, “Imagenet-trained cnns are biased towards texture; increasing shape bias improves accuracy and robustness,” in _ICLR_ , 2019.
  * [30] R. Duan, X. Ma, Y. Wang, J. Bailey, A. K. Qin, and Y. Yang, “Adversarial camouflage: Hiding physical-world attacks with natural styles,” in _CVPR_ , 2020.
  * [31] S. Cheng, Y. Liu, S. Ma, and X. Zhang, “Deep feature space trojan attack of neural networks by controlled detoxification,” in _AAAI_ , 2021.
  * [32] J. Johnson, A. Alahi, and L. Fei-Fei, “Perceptual losses for real-time style transfer and super-resolution,” in _ECCV_ , 2016.
  * [33] X. Huang and S. Belongie, “Arbitrary style transfer in real-time with adaptive instance normalization,” in _ICCV_ , 2017.
  * [34] X. Chen, Y. Zhang, Y. Wang, H. Shu, C. Xu, and C. Xu, “Optical flow distillation: Towards efficient and stable video style transfer,” in _ECCV_ , 2020.
  * [35] L. Sachs, _Applied statistics: a handbook of techniques_. Springer Science & Business Media, 2012.
  * [36] R. V. Hogg, J. McKean, and A. T. Craig, _Introduction to mathematical statistics_. Pearson Education, 2005.
  * [37] J. Deng, W. Dong, R. Socher, L.-J. Li, K. Li, and L. Fei-Fei, “Imagenet: A large-scale hierarchical image database,” in _CVPR_ , 2009.
  * [38] S. Zagoruyko and N. Komodakis, “Wide residual networks,” in _BMVC_ , 2016\.
  * [39] I. Loshchilov and F. Hutter, “Sgdr: Stochastic gradient descent with warm restarts,” in _ICLR_ , 2017.
  * [40] A. Birhane and V. U. Prabhu, “Large image datasets: A pyrrhic win for computer vision?” in _WACV_ , 2021.
  * [41] J. Geiping, L. Fowl, W. R. Huang, W. Czaja, G. Taylor, M. Moeller, and T. Goldstein, “Witches’ brew: Industrial scale data poisoning via gradient matching,” in _ICLR_ , 2021.
  * [42] Y. Li, M. Ya, Y. Bai, Y. Jiang, and S.-T. Xia, “BackdoorBox: A python toolbox for backdoor learning,” 2022.
  * [43] J. Kirkpatrick, R. Pascanu, N. Rabinowitz, J. Veness, G. Desjardins, A. A. Rusu, K. Milan, J. Quan, T. Ramalho, A. Grabska-Barwinska _et al._ , “Overcoming catastrophic forgetting in neural networks,” _Proceedings of the national academy of sciences_ , vol. 114, no. 13, pp. 3521–3526, 2017.
  * [44] X. Huang, M. Alzantot, and M. Srivastava, “Neuroninspect: Detecting backdoors in neural networks via output explanations,” _arXiv preprint arXiv:1911.07399_ , 2019.
  * [45] E. Chou, F. Tramer, and G. Pellegrino, “Sentinet: Detecting localized universal attack against deep learning systems,” in _IEEE S &P Workshop_, 2020.
  * [46] B. G. Doan, E. Abbasnejad, and D. C. Ranasinghe, “Februus: Input purification defense against trojan attacks on deep neural network systems,” in _ACSAC_ , 2020.
  * [47] Y. Gao, Y. Kim, B. G. Doan, Z. Zhang, G. Zhang, S. Nepal, D. Ranasinghe, and H. Kim, “Design and evaluation of a multi-domain trojan detection method on deep neural networks,” _IEEE Transactions on Dependable and Secure Computing_ , 2021.
  * [48] B. Wang, Y. Yao, S. Shan, H. Li, B. Viswanath, H. Zheng, and B. Y. Zhao, “Neural cleanse: Identifying and mitigating backdoor attacks in neural networks,” in _IEEE S &P_, 2019.
  * [49] Y. Dong, X. Yang, Z. Deng, T. Pang, Z. Xiao, H. Su, and J. Zhu, “Black-box detection of backdoor attacks with limited information and data,” in _ICCV_ , 2021.
  * [50] G. Shen, Y. Liu, G. Tao, S. An, Q. Xu, S. Cheng, S. Ma, and X. Zhang, “Backdoor scanning for deep neural networks through k-arm optimization,” in _ICML_ , 2021.
  * [51] S. Zhao, X. Ma, X. Zheng, J. Bailey, J. Chen, and Y.-G. Jiang, “Clean-label backdoor attacks on video recognition models,” in _CVPR_ , 2020.
  * [52] T. Zhai, Y. Li, Z. Zhang, B. Wu, Y. Jiang, and S.-T. Xia, “Backdoor attack against speaker verification,” in _ICASSP_ , 2021.
  * [53] Y. Li, H. Zhong, X. Ma, Y. Jiang, and S.-T. Xia, “Few-shot backdoor attacks on visual object tracking,” in _ICLR_ , 2022.

|  Yiming Li is currently a Ph.D. candidate in Computer Science and
Technology, Tsinghua University, China. Before that, he received his B.S.
degree in Mathematics and Applied Mathematics from Ningbo University, China,
in 2018. His research interests are in the domain of AI security, especially
backdoor learning, adversarial learning, and data privacy. His research has
been published in multiple top-tier conferences and journals, such as ICCV,
ECCV, ICLR, AAAI, IEEE TNNLS, PR Journal. He served as the senior program
committee member of AAAI, the program committee member of ICML, NeurIPS, ICLR,
etc., and the reviewer of IEEE TPAMI, IEEE TIFS, IEEE TDSC, etc.
---|---
|  Linghui Zhu received the B.S. degree in Computer Science and Technology
from Nankai University, Tianjin, China, in 2020. She is currently pursuing the
master degree in Tsinghua Shenzhen International Graduate School, Tsinghua
University. Her research interests are in the domain of data security and
federated learning.
---|---
|  Xiaojun Jia received his B.S. degree in Software Engineering from the China
University of Geosciences, China. He is now a Ph.D. student in the School of
Cyberspace Security, University of Chinese Academy of Sciences, China. His
research interests include computer vision, deep learning, and adversarial
machine learning. He is the author of referred journals and conferences in
CVPR, AAAI, ACM MM, etc.
---|---
|  Dr. Yang Bai received the B.S. degree in Electronic Information Science and
Technology and the Ph.D. degree in Computer Science and Technology from
Tsinghua University in 2017 and 2022, respectively. Her research interests
include Adversarial Machine Learning and Trustworthy AI. Her research has been
published in multiple top-tier conferences, including ICLR, NeurIPS, ECCV,
etc.
---|---
|  Dr. Yong Jiang received his M.S. and Ph.D. degrees in computer science from
Tsinghua University, China, in 1998 and 2002, respectively. Since 2002, he has
been with the Tsinghua Shenzhen International Graduate School of Tsinghua
University, Guangdong, China, where he is currently a full professor. His
research interests include computer vision, machine learning, Internet
architecture and its protocols, IP routing technology, etc. He has received
several best paper awards from top-tier conferences and his research has been
published in multiple top-tier journals and conferences, including IEEE ToC,
IEEE TMM, IEEE TSP, CVPR, ICLR, etc.
---|---
|  Dr. Shu-Tao Xia received the B.S. degree in mathematics and the Ph.D.
degree in applied mathematics from Nankai University, Tianjin, China, in 1992
and 1997, respectively. Since January 2004, he has been with the Tsinghua
Shenzhen International Graduate School of Tsinghua University, Guangdong,
China, where he is currently a full professor. His research interests include
coding and information theory, machine learning, and deep learning. His
research has been published in multiple top-tier journals and conferences,
including IEEE TPAMI, IEEE TIP, CVPR, ICLR, etc.
---|---
|  Dr. Xiaochun Cao received the B.S. and M.S. degrees in computer science
from Beihang University, China, and the Ph.D. degree in computer science from
the University of Central Florida, USA. He is with the School of Cyber Science
and Technology, Sun Yat-sen University, China, as a Full Professor and the
Dean. He has authored and co-authored multiple top-tier journal and conference
papers. He is on the Editorial Boards of the IEEE TIP, IEEE TMM, IEEE TCSVT.
He was the recipient of the Best Student Paper Award at the ICPR (2004, 2010).
He is the Fellow of the IET.
---|---


# Practical Applications of Unidimensional Optimization Using Octave.

Erick Clapton de Lima Silva
Centro de Ensino Superior do Seridó
Federal University of Rio Grande do Norte- UFRN
Caicó, Rio Grande do Norte
<EMAIL_ADDRESS>
&Francisco Marcio Barboza
Department of Computing and Technology
Federal University of Rio Grande do Norte- UFRN
Caicó, Rio Grande do Norte
<EMAIL_ADDRESS>
###### Abstract

This paper suggests integrating one-dimensional optimization methods to tackle
diverse problems, emphasizing their significance in resolving practical issues
and applying mathematical principles to real-world contexts. It focuses on
employing the Octave programming language, backed by specific examples, to
simplify the practical application of mathematical concepts and improve
problem-solving abilities. The research aims to assess the effect on students’
comprehension of one-dimensional optimization.

_K_ eywords Unidimensional Optimization $\cdot$ Octave $\cdot$ Real-world
Problems.

##  1 Introduction

Unidimensional optimization plays a crucial role in various domains, offering
an effective approach to enhancing processes and decision-making. This
mathematical discipline focuses on either maximizing or minimizing a single
objective function, proving indispensable for addressing a diverse array of
practical problems [1].

An integral aspect of unidimensional optimization is its ability to provide
optimal solutions in scenarios where complexity can be reduced to a single
dimension. By locating the extremes of a function, unidimensional optimization
enables us to identify values that either maximize benefits or minimize costs,
thereby exerting a positive influence across various aspects of daily life
[2].

This article explores two specific applications of unidimensional
optimization, demonstrating its practical significance: optimizing the
distance from the movie screen to enhance viewers’ visual experience and
calculating the maximum size of an L-shaped pipe to fit through a corridor.

By understanding the significance of these applications, we can discern how
unidimensional optimization not only enhances the efficiency and advancement
of systems but also contributes significantly to public well-being and health.
Throughout this article, we will meticulously examine each application,
analyzing the distinct challenges encountered in each domain and showcasing
how unidimensional optimization provides efficient solutions.

Optimization applications are predominantly pervasive in computer science and
applied mathematics, where they play a crucial role in addressing various
problems. However, optimization extends its reach across diverse fields of
knowledge, including engineering, natural sciences, and economics [1]. Despite
the varied disciplines, they share a common objective: leveraging optimization
for problem-solving in a more direct and precise manner.

The unidimensional optimization problem holds paramount importance in
practical optimization [3]. This significance arises not only from its
frequent occurrence in research but also from the fact that more complex
problems involving multiple variables can often be tackled through a sequence
of one-dimensional problems. In this article, we present some fundamental
numerical techniques for locating the minimum of a single-variable function.
We focus solely on minimization problems, with the understanding that
maximization concerns can be addressed similarly, as minimizing $f(x)$ is
equivalent to maximizing $-f(x)$.

##  2 Methodology

######  Definition 1.

(Function increasing and decreasing)
Be $f$ a function defined on an interval $I$, and let $x_{1}$ e $x_{2}$ any
two points in $I$.

  1. 1.

$f$ is increasing on $I$ if $x_{1}<x_{2}$ implies that $f(x_{1})<f(x_{2})$.

  2. 2.

$f$ is decreasing on $I$ if $x_{1}<x_{2}$ implies that $f(x_{1})>f(x_{2})$.

######  Theorem 1.

Be $f$ continuous on a closed interval $[a,b]$ and differentiable at $(a,b)$.

  1. 1.

Se $f(x)>0$ for all $x$ em $(a,b)$, then $f$ is increasing on $[a,b]$.

  2. 2.

Se $f(x)<0$ for all $x$ em $(a,b)$, then $f$ is decreasing on $[a,b]$.

###### Proof.

See [4]. ∎

######  Definition 2.

(Local Extrema)
Be $c$ belonging to the domain of a function $f$

  1. 1.

$f(c)$ is a local maximum of $f$ If there exists an open interval containing
$(a,b)$ including $c$, such that $f(x)\leq f(c)$ for any $x$ em $(a,b)$ that
belongs to the domain of $f$.

  2. 2.

$f(c)$ is a local minimum of $f$ if there exists an open interval $(a,b)$
containing $c$, such that $f(x)\geq f(c)$ for any $x$ em $(a,b)$ belonging to
the domain of $f$.

######  Definition 3.

(Critical Values)
If a function $f$ has an extreme value at a number $c$ in an open interval,
then either $f^{\prime}(c)=0$ or $f^{\prime}(c)$ does not exist. A number $c$
with this property is called a critical point of $f$.

######  Definition 4.

(First Derivative Test)
Suppose $f$ is continuous on an open interval containing a critical value $c$.

  1. 1.

If $f$ changes sign from positive to negative at $c$, then $f$ has a local
maximum value at $c$.

  2. 2.

If $f$ changes sign from negative to positive at $c$, then $f$ has a local
minimum value at $c$.

######  Definition 5.

(Second Derivative Test)
Suppose $f$ is twice different from $c$.

  1. 1.

Se $f(c)=0$ e $f^{\prime}(c)>0$, then $f$ has a relative minimum in em $x=c$.

  2. 2.

Se $f(c)=0$ e $f^{\prime}(c)<0$, then $f$ has a relative maximum in $x=c$.

  3. 3.

Se $f(c)=0$ e $f^{\prime}(c)=0$, then the test is inconclusive.

In many applications, there arises a necessity to ascertain the local minimum
or maximum of a function represented as $f(x)$. Definition 5 elucidates that
the value of $x$ may correspond to a local minimum or maximum, a determination
typically facilitated by finding the root of the function’s derivative. The
corresponding value of $y$ is then derived by substituting $x$ into the
function. In Octave, when seeking the minimum value of a one-dimensional
function within the interval $x_{1}\leqslant x\leqslant x_{2}$, one can
utilize the command fminbnd.

The fminbnd command in Octave serves to locate the minimum of a one-
dimensional function within a predefined interval. Employing an optimization
algorithm, it identifies the value of $x$ that minimizes the function $f(x)$
within the specified interval. The fundamental syntax of the command is as
follows:

[x, fval] = fminbnd (f, x1, x2)

where:

  * •

f is the function to be minimized.

  * •

x1 and x2 are the bounds of the interval over which the function is evaluated.

  * •

x is the value of $x$ that minimizes the function within the interval.

  * •

fval is the minimum value of the function found at x.

This command proves particularly handy when swiftly determining the minimum of
a function within a designated interval, eliminating the necessity of
computing derivatives or resorting to more intricate optimization techniques.

Moreover, the fminbnd command can be employed to pinpoint the maximum of a
function. This involves multiplying the function by $-1$ and subsequently
seeking the minimum [5].

###  2.1 Pipe Problem

A pipe of length $L$ and negligible diameter must be horizontally transported
around a corner from a corridor with a width of 3 feet to a corridor with a
width of 6 feet (see Figure 1). What is the maximum length that the pipe can
have?

Figure 1: Pipe, Source: [6]

Therefore, $L$ depends on the angle $\alpha$.

$L(\alpha)=L_{1}+L_{2}=\frac{3}{\sin(\alpha)}+\frac{6}{\cos(\alpha)}$

We conclude that the length of the longest pipeline is given by the absolute
minimum of $L(\alpha)$ in the interval $(0,\pi/2)$. The graph of $L(\alpha)$
shows that it has only one relative minimum; therefore, the absolute minimum
of $L(\alpha)$ in $(0,\pi/2)$ occurs at this relative minimum. The approximate
minimum of $L(\alpha)$ can be obtained using one of the numerical methods
described in this paper.

###  2.2 Cinema Problem

To achieve the best viewing experience while watching a movie, it is necessary
for a person to position themselves at a distance $x$ from the screen such
that the viewing angle $\theta$ is maximized. The problem at hand, taken from
[5], aims to determine the distance $x$ for which $\theta$ is maximum, as
illustrated in the configuration in Figure 2.

Figure 2: Geometric Modeling of the Problem. Source: [5]

##  3 Results and discussions

###  3.1 Piping Problem Solution

This question pertains to determining the maximum length of a pipe that can be
horizontally transported around a corner within a corridor. The initial
corridor is 3 feet wide, whereas the target corridor is 6 feet wide. To
address this problem, numerical methods, such as optimization techniques
utilizing the fminbnd command in Octave, can be employed. This method enables
the identification of the minimum of a function within a predefined interval,
which in this instance spans from $[0,90]$ degrees. To elaborate on this
question, we can reformulate the formula in several stages.

$L(\alpha)=L_{1}+L_{2}=\frac{3}{\sin(\alpha)}+\frac{6}{\cos(\alpha)}$

$L(\alpha)=L_{1}+L_{2}=3\csc(\alpha)+6\sec(\alpha)$

This code defines a function $f(x)=3csc(\alpha)+6\sec(\alpha)$, where it
calculates the minimum of this function in the interval $[0,90]$ using
fminbnd, It then prints the minimum and maximum values of the function at this
minimum. Finally, the minimum value of this function is maximized in Octave,
thus determining the maximum value of the pipe length to pass through the
corridor, represented as $L$. 2.

Figure 3: Graph of the equation. Source: [5]

Analyzing the function’s graph, based on the values we took for the width of
the L-shaped corridor, 3 feet and 6 feet respectively, when running the code,
we arrived at the value of approximately $64.403$. In this case, using the
method where the maximum of the function would be calculated as the minimum
with opposite sign, as explained earlier, $\max=-\min$, we were able to
calculate the maximum length that the metal beam can reach to pass through the
width of the corridor.

###  3.2 Cinema Problem Solution

To solve this problem using Octave, we can employ trigonometric calculations
to derive the expression for the viewing angle $\theta$ in terms of the
distance $x$. Subsequently, we aim to maximize this expression to determine
the distance $x$ that yields the maximum$\theta$.

According to Figure 2, the configuration is a right triangle where the
distance $x$ is the hypotenuse, and the other two sides are the height $h$
(distance between the screen and the eyes) and the base $d$ (width of the
screen).

Trigonometry tells us that:

$\cos(\theta)=\frac{x^{2}-\frac{5}{2}^{2}}{\left(x^{2}+\frac{4}{12}^{2}\right)+36}-\frac{2}{x^{2}+\frac{5}{2}^{2}+x^{2}+\frac{4}{12}^{2}}$

That is:

$\theta=\arccos\left(\dfrac{h}{d}\right)$

Therefore, we can write $h$ in terms of $x$ and $d$ using the Pythagorean
theorem:

$h=\sqrt{x^{2}-\left(\frac{d}{2}\right)^{2}}$

Substituting $h$ in the tangent equation, we obtain:

$\theta(x)=\arccos\left(\frac{\sqrt{x^{2}-\left(\frac{d}{2}\right)^{2}}}{d}\right)$

Assigning the values given in the problem, where by the Pythagorean theorem
the opposite leg, in this case denoted as $d$ with a value of 40 meters, and
additionally assigning a value to $x$ to calculate the value of $\theta$ where
the value would be 10 meters, upon running the code, we obtain a result of
1.57, which corresponds to the angle of inclination. Furthermore, the angle at
which the individual would be seated in the cinema can be calculated by
applying the corresponding values to where the person is fixed in the cinema,
thus determining the cinema’s inclination angle with respect to $\theta$.

##  4 Conclusion

Learning about optimization in issues involving maxima and minima of functions
is incredibly valuable, not only for the development of mathematical skills
but also for acquiring analytical and practical skills that are essential in
various areas of knowledge. Octave is an extremely effective tool that not
only facilitates teaching but also enhances the connection with the content,
simplifying the calculation of maxima and minima of a function. This work
represents a way to simplify the understanding of the topic of maxima and
minima, highlighting the importance of optimization and the use of Octave

## References

  * [1] Jorge Nocedal and Stephen J Wright.  Numerical Optimization.  2006\.
  * [2] David G Luenberger, Yinyu Ye, et al.  Linear and nonlinear programming, volume 2.  1984\.
  * [3] Gilbert Strang.  Introduction to applied mathematics.  1986\.
  * [4] Halsey Royden and Patrick Michael Fitzpatrick.  Real analysis.  2010\.
  * [5] Amos Gilat.  MATLAB: An introduction with Applications.  2004\.
  * [6] Abdelwahab Kharab and Ronald Guenther.  An introduction to numerical methods: a MATLAB® approach.  2018\.