# Leveraging Optimization for Adaptive Attacks on Image Watermarks

Nils Lukas, Abdulrahman Diaa, Lucas Fenaux, Florian Kerschbaum
University of Waterloo, Canada
{nlukas, abdulrahman.diaa, lucas.fenaux<EMAIL_ADDRESS>

###### Abstract

Untrustworthy users can misuse image generators to synthesize high-quality
deepfakes and engage in online spam or disinformation campaigns. Watermarking
deters misuse by marking generated content with a hidden message, enabling its
detection using a secret watermarking key. A core security property of
watermarking is robustness, which states that an attacker can only evade
detection by substantially degrading image quality. Assessing robustness
requires designing an adaptive attack for the specific watermarking algorithm.
A challenge when evaluating watermarking algorithms and their (adaptive)
attacks is to determine whether an adaptive attack is optimal, i.e., it is the
best possible attack. We solve this problem by defining an objective function
and then approach adaptive attacks as an optimization problem. The core idea
of our adaptive attacks is to replicate secret watermarking keys locally by
creating _surrogate keys_ that are differentiable and can be used to optimize
the attack’s parameters. We demonstrate for Stable Diffusion models that such
an attacker can break all five surveyed watermarking methods at negligible
degradation in image quality. These findings emphasize the need for more
rigorous robustness testing against adaptive, learnable attackers.

_K_ eywords Watermarking, Stable Diffusion, Robustness, Adaptive Attacks

##  1 Introduction

Deepfakes are images synthesized using deep image generators that can be
difficult to distinguish from real images. While deepfakes can serve many
beneficial purposes if used ethically, for example, in medical imaging (Akrout
et al., 2023) or education (Peres et al., 2023) they also have the potential
to be _misused_ and erode trust in digital media. Deepfakes have already been
used in disinformation campaigns (Boneh et al., 2019) and social engineering
attacks (Mirsky and Lee, 2021), highlighting the need for methods that control
the misuse of deep image generators.

Watermarking offers a solution to controlling misuse by embedding hidden
messages into all generated images that are later detectable using a secret
watermarking key. Images that are detected as deepfakes can be flagged by
social media platforms or news agencies, which can mitigate potential harm
(Grinbaum and Adomaitis, 2022). Providers of large image generators such as
Google have announced the deployment of their own watermarking methods (Gowal
and Kohli, 2023) to enable the detection of deepfakes and promote the ethical
use of their models.

A core security property of watermarking is _robustness_ , which states that
an attacker can evade detection only by substantially degrading the image’s
quality. While several watermarking methods have been proposed for image
generators (Wen et al., 2023; Zhao et al., 2023; Fernandez et al., 2023), none
of them are certifiably robust (Bansal et al., 2022) and instead, robustness
is tested empirically using a limited set of known attacks. Claimed security
properties of previous watermarking methods have been broken by novel attacks
(Lukas et al., 2022), and no comprehensive method exists to validate
robustness, which causes difficulty in trusting the deployment of watermarking
in practice.

We propose testing the robustness of watermarking by defining robustness using
objective function and approaching adaptive attacks as an optimization
problem. Adaptive attacks are specific to the watermarking algorithm used by
the defender but have no access to the secret watermarking key. Knowledge of
the watermarking algorithm enables the attacker to consider a range of
_surrogate_ keys similar to the defender’s key. This is also a challenge for
optimization since the attacker only has imperfect information about the
optimization problem. Adaptive attackers had previously been shown to break
the robustness of watermarking for image classifiers (Lukas et al., 2022), but
attacks had to be handcrafted against each watermarking method. Finding attack
parameters through an optimization process can be challenging when the
watermarking method is not easily optimizable, for instance, when it is not
differentiable. Our attacks leverage optimization by approximating watermark
verification through a differentiable process. We show that adaptive,
_learnable_ attackers, whose parameters can be optimized efficiently, can
evade watermark detection for 1 billion parameter Stable Diffusion models at a
negligible degradation in image quality.

Figure 1: An overview of our adaptive attack pipeline. The attacker prepares
their attack by generating a surrogate key and leveraging optimization to find
optimal attack parameters $\theta_{\mathcal{A}}$ (illustrated here as an
encoder $\mathcal{E}$ and decoder $\mathcal{D}$) for any message. Then, the
attacker generates watermarked images and applies a modification using their
optimized attack to evade detection. The attacker succeeds if the verification
procedure cannot detect the watermark in high-quality images.

##  2 Background

Latent Diffusion Models (LDMs) are state-of-the-art generative models for
image synthesis (Rombach et al., 2022). Compared to Diffusion Models (Sohl-
Dickstein et al., 2015), LDMs operate in a latent space using fixed, pre-
trained autoencoder consisting of an image encoder $\mathcal{E}$ and a decoder
$\mathcal{D}$. LDMs use a forward and reverse diffusion process across $T$
steps. In the forward pass, real data point $x_{0}$ is encoded into a latent
point $z_{0}=\mathcal{E}(x_{0})$ and is progressively corrupted into noise via
Gaussian perturbations. Specifically,

$q(z_{t}|z_{t-1})=\mathcal{N}\left(z_{t};\sqrt{1-\beta_{t}}z_{t-1},\beta_{t}\mathbf{I}\right),\quad
t\in\\{0,1,\ldots,T-1\\},$

where $\beta_{t}$ is the scheduled variance. In the reverse process, a neural
network $f_{\theta}$ guides the denoising, taking $z_{t}$ and time-step $t$ as
inputs to predict $z_{t-1}$ as $f_{\theta}(x_{t},t)$. The model is trained to
minimize the mean squared error between the predicted and actual $z_{t-1}$.
The outcome is a latent $\hat{z}_{0}$ resembling $z_{0}$ that can be decoded
into $\hat{x}_{0}=\mathcal{D}(z_{0})$. Synthesis in LDMs can be conditioned
with textual prompts.

###  2.1 Watermarking

Watermarking embeds a hidden signal into a medium, such as images, using a
secret watermarking key that is later extractable using the same secret key.
In the context of deep learning, watermarking can be characterized by the
medium used by the defender to verify the presence of the hidden signal.
White-box and black-box watermarking methods assume access to the model’s
parameters or query access via an API respectively, and have been used
primarily for Intellectual Property protection (Uchida et al., 2017).

_No-box_ watermarking (Lukas and Kerschbaum, 2023) assumes a more restrictive
setting where the defender only knows the generated content but does not know
the query used to generate the image. This type of watermarking has been used
to control misuse by having the ability to detect any image generated by the
provided image generator (Gowal and Kohli, 2023). Given a generator’s
parameters $\theta_{G}$, a no-box watermarking method defines the following
three procedures.

  * •

$\tau\leftarrow\textsc{KeyGen}(\theta_{G})$: A randomized function to generate
a watermarking key $\tau$.

  * •

$\theta_{G}^{*}\leftarrow\textsc{Embed}(\theta_{G},\tau,m)$: Given a generator
$\theta_{G}$, a watermarking key $\tau$ and a message $m$, return parameters
$\theta_{G}^{*}$ of a _watermarked_ generator that only generates watermarked
images.

  * •

$p\leftarrow\textsc{Verify}(x,\tau,m)$: This function (i) extracts a message
$m^{\prime}$ from $x$ using $\tau$ and (ii) returns the $p$-value to reject
the null hypothesis that $m$ and $m^{\prime}$ match by random chance.

A watermarking method is a set of algorithms that specify (KeyGen, Embed,
Verify). A watermark is a hidden signal in an image that can be mapped to a
message $m$ using a secret key $\tau$. In this paper, we denote the similarity
between two messages by their $L_{1}$-norm difference. We use more meaningful
similarity measures when $\mathcal{M}$ allows it, such as the Bit-Error-Rate
(BER) when the messages consist of bits. A watermark is _retained_ in an image
if the verification procedure returns $p<0.01$, following Wen et al. (2023).
Adi et al. (2018) specify the requirements for trustworthy watermarking, and
we focus on two properties: Effectiveness and robustness. Effectiveness states
that a watermarked generator has a high image quality while retaining the
watermark, and robustness means that a watermark is retained in an image
unless the image’s quality is substantially degraded. We refer to Lukas and
Kerschbaum (2023) for security games encoding effectiveness and robustness.

###  2.2 Watermarking for Image Generators

Several works propose no-box watermarking methods to prevent misuse for two
types of image generators: Generative Adversarial Networks (GANs) (Goodfellow
et al., 2020) and Latent Diffusion Models (LDMs) (Rombach et al., 2022). We
distinguish between _post-hoc_ watermarking methods that apply an
imperceptible modification to an image and _semantic_ watermarks that modify
the output distribution of an image generator and are truly “invisible" (Wen
et al., 2023).

For post-hoc watermarking, traditional methods hide messages using the
Discrete Wavelet Transform (DWT) and Discrete Wavelet Transform with Singular
Value Decomposition (DWT-SVD) (Cox et al., 2007) and are currently used for
Stable Diffusion111https://github.com/Stability-AI/stablediffusion. RivaGAN
(Zhang et al., 2019) watermarks by training a deep neural network
adversarially to stamp a pattern on an image. Yu et al. (2020, 2021) propose
two methods that modify the generator’s training procedure but require
expensive re-training from scratch. Lukas and Kerschbaum (2023) propose a
watermarking method for GANs that can be embedded into a pre-trained
generator. Zhao et al. (2023) propose a general method to watermark diffusion
models (WDM) that uses a method similar to Yu et al. (2020), which trains an
autoencoder to stamp a watermark on all training data before also re-training
the generator from scratch. Fernandez et al. (2023) pre-train an autoencoder
to encode hidden messages into the training data and embed the watermark by
fine-tuning the decoder $\mathcal{D}$ component of the LDM.

Wen et al. (2023) are the first to propose a semantic watermarking method for
LDMs they call Tree-Rings Watermarks (TRW). The idea is to mark the initial
noise $x_{T}$ with a detectable, tree-ring-like pattern $m$ in the frequency
domain before generating an image. During detection, they leverage the
property of LDM’s that the diffusion process is invertible, which allows
mapping an image back to its original noise. The verification extracts a
message $m^{\prime}$ by spectral analysis and tests whether the same tree-ring
patterns $m$ are retained in the frequency domain of the reconstructed noise.

Surveyed Watermarking Methods. In this paper, we evaluate the robustness of
five watermarking methods: TRW, WDM, DWT, DWT-SVD and RivaGAN. DWT, DWT-SVD
and RivaGAN are default choices when using StabilityAI’s Stable Diffusion
repository and WDM and TRW are two recently proposed methods for Stable
Diffusion models. However, WDM requires re-training a Stable Diffusion model
from scratch, which can require 150-1000 GPU days (Dhariwal and Nichol, 2021)
and is not replicable with limited resources. For this reason, instead of
using the autoencoder on the input data, we apply their autoencoder as a post-
processor after generating images.

##  3 Threat Model

We consider a provider capable of training large image generators who make
their generators accessible to many users via a black-box API, such as OpenAI
with DALL·E. Users can query the generator by including a textual prompt that
controls the content of the generated image. We consider an attack by an
untrustworthy user who wants to misuse the provided generator without
detection.

Provider’s Capabilities and Goals (Model Capabilities) The provider fully
controls image generation, including the ability to post-process generated
images. (Watermark Verification) In a no-box setting, the defender must verify
their watermark using a single generated image. The defender aims for an
effective watermark that preserves generator quality while preventing the
attacker from evading detection without significant image quality degradation.

Attacker’s Capabilities. (Model Capabilities) The user has black-box query
access to the provider’s watermarked model and also has white-box access to
less capable, open-source _surrogate_ generators, such as Stable Diffusion on
Huggingface. We assume the surrogate model’s image quality is inferior to the
provided model; otherwise, there would be no need to use the watermarked
model. Our attacker does not require access to image generators from other
providers, but, of course, such access may imply access to surrogate models as
our attack does require. (Data Access) The attacker has unrestricted access to
real-world image and caption data available online, such as LAION-5B
(Schuhmann et al., 2022). (Resources) Computational resources are limited,
preventing the attacker from training their own image generator from scratch.
(Queries) The provider charges the attacker per image query, limiting the
number of queries they can make. The attacker can generate images either
unconditionally or with textual prompts. (Adaptive) The attacker knows the
watermarking method but lacks access to the secret watermarking key $\tau$ and
chosen message $m$.

Attacker’s Goal. The attacker wants to use the provided, watermarked generator
to synthesize images (i) without a watermark that (ii) have a high quality. We
measure quality using a perceptual similarity function
$Q:\mathcal{X}\times\mathcal{X}\rightarrow\mathbb{R}$ between the generated,
watermarked image and a perturbed image after the attacker evades watermark
detection. We require that the defender verifies the presence of a watermark
correctly with a p-value of at least $p<0.01$, same as Wen et al. (2023).

##  4 Conceptual Approach

In this Section, we define robustness as an objective function and show that
we can optimize this function efficiently using stochastic gradient descent
when the watermark algorithm is known (i.e., in adaptive attacks) for all five
surveyed watermarking methods. Then, we propose _learnable_ attacks and find
optimal parameters through empirical loss minimization.

###  4.1 Robustness as an Objective Function

As described in Section 2, a watermarking method defines three procedures
(KeyGen, Embed, Verify). The provider generates a secret watermarking key
$\tau\leftarrow\textsc{KeyGen}(\theta_{G})$ that allows them to watermark
their generator so that all its generated images retain the watermark. To
embed a watermark, the provider chooses a message (we sample a message
$m\sim\mathcal{M}$ uniformly at random) and modifies their generator’s
parameters $\theta_{G}^{*}\leftarrow\textsc{Embed}(\theta_{G},\tau,m)$. Any
image generated by $\theta_{G}^{*}$ should retain the watermark. For any
$x\leftarrow\textsc{Generate}(\theta_{G}^{*})$ we call a watermark _effective_
if (i) the watermark is retained, i.e., $\textsc{Verify}(x,\tau,m)<0.01$ and
(ii) the watermarked images have a high perceptual quality. The attacker
generates images $x\leftarrow\textsc{Generate}(\theta_{G}^{*})$ and applies an
image-to-image transformation, $\mathcal{A}:\mathcal{X}\rightarrow\mathcal{X}$
with parameters $\theta_{\mathcal{A}}$ to evade watermark detection by
perturbing $\hat{x}\leftarrow\mathcal{A}(x)$. Finally, the defender verifies
the presence of their watermark in $\hat{x}$, as shown in Figure 1.

Let $W(\theta_{G},\tau^{\prime},m)=\textsc{Embed}(\theta_{G},\tau^{\prime},m)$
be the watermarked generator after embedding with key $\tau^{\prime}$ and
message $m$ and $G_{W}=\textsc{Generate}(W(\theta_{G},\tau^{\prime},m))$
denotes the generation of an image using the watermarked generator parameters.
For any high-quality $G_{W}$, the attacker’s objective becomes:

$\displaystyle\max_{\mathcal{\theta_{\mathcal{A}}}}\underset{\begin{subarray}{c}\tau^{\prime}\leftarrow\textsc{Keygen}(\theta_{G})\\\
m\in\mathcal{M}\end{subarray}}{\mathbb{E}}\left[\textsc{Verify}(\mathcal{A}(G_{W}),\tau^{\prime},m)+Q(\mathcal{A}(G_{W}),G_{W})\right]$
(1)

This objective seeks to maximize (i) the expectation of successful watermark
evasion over all potential watermarking keys $\tau^{\prime}$ and messages $m$
(since the attacker does not know which key-message pair was chosen) and (ii)
the perceptual similarity of the images before and after the attack. Note that
in this paper, we define image quality as the perceptual similarity to the
watermarked image before the attack. There are two obstacles for an attacker
to optimize this objective: (1) The attacker has imperfect information about
the optimization problem and must substitute the defender’s image generator
with a less capable, open-source surrogate generator. When KeyGen depends on
$\theta_{G}$, then the distribution of keys differs, and the attack’s
effectiveness must transfer to keys generated using $\theta_{G}$. (2) The
optimization problem might be hard to approximate, even when perfect
information is available, e.g., when the watermark verification procedure is
not differentiable.

###  4.2 Making Watermarking Keys Differentiable

We overcome the two aforementioned limitations by (1) giving the attacker
access to a similar (but less capable) surrogate generator $\hat{\theta}_{G}$,
enabling them to generate surrogate watermarking keys, and (2) by creating a
method $\textsc{GKeyGen}(\hat{\theta}_{G})$ that creates a surrogate
watermarking key $\theta_{K}$ through which we can backpropagate gradients. A
simple but computationally expensive method of creating differentiable keys
$\theta_{D}$ is using Algorithm 1 to train a watermark extraction neural
network with parameters $\theta_{D}$ to predict the message $m$ from an image.

Algorithm 1 GKeyGen: A Simple Method to Generate Differentiable Keys

1:Surrogate generator $\hat{\theta}_{G}$, Watermarking method (KeyGen, Embed,
Verify), $N$ steps

2:$\tau\leftarrow\textsc{Keygen}(\hat{\theta}_{G})$ $\triangleright$ The non-
differentiable surrogate key

3:for $j\leftarrow 1\textbf{ to }N$ do

4: $m\sim\mathcal{M}$ $\triangleright$ Sample a random message

5: $\hat{\theta_{G}^{*}}\leftarrow\textsc{Embed}(\hat{\theta}_{G},\tau,m)$
$\triangleright$ Embed the watermark

6: $x\leftarrow\textsc{Generate}(\hat{\theta_{G}^{*}})$

7: $m^{\prime}\leftarrow\textsc{Extract}(x;\theta_{D})$

8: $g_{\theta_{D}}\leftarrow\nabla_{\theta_{D}}||m-m^{\prime}||_{1}$
$\triangleright$ Compute gradients using distance between messages

9: $\theta_{D}\leftarrow\theta_{D}-\text{Adam}(\theta_{D},g_{\theta_{D}})$

10:return $\theta_{D}$ $\triangleright$ The differentiable surrogate key

Our adaptive attacker must invoke Algorithm 1 for the non-differentiable
watermarks DCT and DCT-SVD (Cox et al., 2007). Fortunately for the attacker,
the remaining three watermarking methods TRW (Wen et al., 2023), WDM (Zhao et
al., 2023) and RivaGAN Zhang et al. (2019) do not require invoking GKeyGen
because their watermarking keys are already differentiable, reducing
computational resources required for the attacker.

###  4.3 Leveraging Optimization against Watermarks

Equation 1 requires finding attack parameters $\theta_{\mathcal{A}}$ against
any watermarking key $\tau^{\prime}\leftarrow\textsc{KeyGen}(\theta_{G})$,
which can be computationally expensive if the attacker has to invocate GKeyGen
many times. We find empirically that generating many keys is unnecessary, and
the attacker can find effective attacks using only a single surrogate
watermarking key $\theta_{D}\leftarrow\textsc{GKeyGen}(\hat{\theta}_{G})$.

We propose two learnable attacks $\mathcal{A}_{1},\mathcal{A}_{2}$ whose
parameters $\theta_{\mathcal{A}_{1}},\theta_{\mathcal{A}_{2}}$ can be
optimized efficiently. The first attack, called _Adversarial Noising_ finds
adversarial examples given an image $x$ using the surrogate key as a reward
model. The second attack, called _Adversarial Compression_ , first fine-tunes
the parameters of a pre-trained autoencoder in a preparation stage and uses
the optimized parameters during an attack. The availability of a pre-trained
autoencoder is a realistic assumption if the attacker has access to a
surrogate Stable Diffusion generator, as the autoencoder is a detachable
component of any Stable Diffusion generator. Access to a surrogate generator
implies the availability of a pre-trained autoencoder at no additional cost in
computational resources for the attacker.

Algorithm 2 Adversarial Noising

1:surrogate $\hat{\theta}_{G}$, budget $\epsilon$, image $x$

2:$\theta_{\mathcal{A}}\leftarrow 0$ $\triangleright$ adversarial perturbation

3:$\theta_{D}\leftarrow\textsc{GKeyGen}(\hat{\theta}_{G})$

4:$m\leftarrow\textsc{Extract}(x;\theta_{D})$

5:for $j\leftarrow 1\textbf{ to }N$ do

6: $m^{\prime}\leftarrow\textsc{Extract}(x+\theta_{\mathcal{A}},\theta_{D})$

7:
$g_{\theta_{\mathcal{A}}}\leftarrow-\nabla_{\theta_{\mathcal{A}}}||m-m^{\prime}||_{1}$

8: $\theta_{\mathcal{A}}\leftarrow
P_{\epsilon}(\theta_{\mathcal{A}}-\text{Adam}(\theta_{\mathcal{A}},g_{\theta_{\mathcal{A}}}))$
return $x+\theta_{\mathcal{A}}$

Algorithm 3 Adversarial Compression

1:surrogate $\hat{\theta}_{G}$, strength $\alpha$, image $x$

2:$\theta_{\mathcal{A}}\leftarrow[\theta_{\mathcal{E}},\theta_{\mathcal{D}}]$
$\triangleright$ Compressor parameters

3:$\theta_{D}\leftarrow\textsc{GKeyGen}(\hat{\theta}_{G})$ $\triangleright$
surrogate key

4:for $j\leftarrow 1\textbf{ to }N$ do

5: $m\sim\mathcal{M}$

6:
$\hat{\theta}_{G}^{*}\leftarrow\textsc{Embed}(\hat{\theta}_{G},\theta_{D},m)$

7: $x\leftarrow\textsc{Generate}(\hat{\theta_{G}^{*}})$

8: $x^{\prime}\leftarrow\mathcal{D}(\mathcal{E}(x;\theta_{\mathcal{A}}))$
$\triangleright$ compression

9: $m^{\prime}\leftarrow\textsc{Extract}(x^{\prime},\theta_{D})$

10:
$g_{\theta_{\mathcal{A}}}\leftarrow\nabla_{\delta}(\mathcal{L}_{\text{LPIPS}}(x^{\prime},x)-\alpha||m-m^{\prime}||_{1})$

11:
$\theta_{\mathcal{A}}\leftarrow\theta_{\mathcal{A}}-\text{Adam}(\theta_{\mathcal{A}},g_{\theta_{\mathcal{A}}})$
return $\mathcal{D}(\mathcal{E}(x;\theta_{\mathcal{A}}))$

Adversarial Noising. 8 shows the pseudocode of our adversarial noising attack.
Given a surrogate generator $\hat{\theta}_{G}$, a budget
$\epsilon\in\mathbb{R}^{+}$ for the maximum allowed noise perturbation, and a
watermarked image $x$ generated using the provider’s watermarked model, the
attacker wants to compute a perturbation within an $\epsilon$-ball of the
$L_{\infty}$ norm that evades watermark detection. The attacker generates a
local surrogate watermarking key (line 2) and extracts a message $m$ from $x$
(line 3). Then, the attacker computes the adversarial perturbation by
maximizing the distance to the initially extracted message $m$ while clipping
the perturbation into an $\epsilon$-ball using $P_{\epsilon}$ (line 7).

Adversarial Compression. 11 shows the pseudocode of our adversarial
compression attack. After generating a surrogate watermarking key (line 2),
the attacker generates images containing a random message (lines 4-6) and uses
their encoder-decoder pair to compress the images (line 7). The attacker
iteratively updates their model’s parameters by (i) minimizing a quality loss,
which we set to the LPIPS metric (Zhang et al., 2018), and (ii) maximizing the
distance between the extracted and embedded messages (line 9). The output
$\theta_{\mathcal{A}}$ of the optimization loop between lines 3 and 10 only
needs to be run once, and the weights $\theta_{\mathcal{A}}$ can be re-used in
subsequent attacks.

We highlight that the attacker optimizes an approximation of Equation 1 since
they only have access to a surrogate generator $\hat{\theta}_{G}$, but not the
provider’s generator $\theta_{G}$. This may lead to a generalization gap of
the attack at inference time. Even if an attacker can find optimal attack
parameters $\theta_{\mathcal{A}}$ that optimizes Equation 1 using
$\hat{\theta}_{G}$, the attacker cannot test whether their attack remains
effective when the defender uses a different model $\theta_{G}$ to generate
watermarking keys.

##  5 Experiments

Image Generators. We experiment with Stable Diffusion, an open-source, state-
of-the-art latent diffusion model. The defender deploys a Stable
Diffusion-v2.0 model222https://huggingface.co/stabilityai/stable-
diffusion-2-base trained for 1.4m steps in total on a subset of LAION-5B
(Schuhmann et al., 2022). The attacker uses a less capable Stable
Diffusion-v1.1333https://huggingface.co/CompVis/stable-diffusion-v1-1
checkpoint, trained for 431k steps in total on LAION-2B and LAION-HD. All
experiments were conducted on NVIDIA A100 GPUs.

Images are generated using a DPM solver (Lu et al., 2022) with 20 inference
steps and a default guidance scale of 7.5. We create three different
watermarked generators for each surveyed watermarking method by randomly
sampling a watermarking key $\tau\leftarrow\textsc{KeyGen}(\theta_{G})$ and a
message $m\sim\mathcal{M}$, used to embed a watermark. Section A.1 contains
descriptions of the watermarking keys. All reported values represent the mean
value over three independently generated secret keys.

Quantitative Analysis. Similar to Wen et al. (2023), we report the True
Positive Rate when the False Positive Rate is fixed to 1%, called the
TPR@1%FPR. Section A.2 describes statistical tests used in the verification
procedure of each watermarking method to derive p-values. We report the
Fréchet Inception Distance (FID) (Heusel et al., 2017), which measures the
similarity between real and generated images. Additionally, we report the CLIP
score (Radford et al., 2021) that measures the similarity of a prompt to an
image. We generate 1k images to evaluate TPR@1%FPR and 5k images to evaluate
FID and CLIP score on the training dataset of MS-COCO-2017 (Lin et al., 2014).

###  5.1 Evaluating Robustness

Figure 2 shows a scatter plot of the effectiveness of our attacks against all
surveyed watermarking methods. We evaluate adaptive and non-adaptive attacks.
Similar to Wen et al. (2023), for the non-adpative attacks we use Blurring,
JPEG Compression, Cropping, Gaussian noise, Jittering, Quantization, and
Rotation, but find these attacks to be ineffective at removing the watermark.
Figure 2 highlights Pareto optimal attacks for pairs of (i) watermark
detection accuracies and (ii) perceptual distances. We find that only adaptive
attacks evade watermark detection and preserve image quality.

Table 1 summarizes the best attacks from Figure 2 with a lowest acceptable
detection accuracy of $10\%$. When multiple attacks achieve a detection
accuracy lower than $10\%$, we pick the attack with the lowest perceptual
distance to the watermarked image. We observe that adversarial compression is
an effective attack against all watermarking methods. TRW is also evaded by
adversarial compression, but adversarial noising at
$\epsilon=\nicefrac{{2}}{{255}}$ preserves a higher image quality.

Figure 2: The effectiveness of our attacks against all watermarks. We highlight the Pareto front for each watermarking method by dashed lines and indicate adaptive/non-adaptive attacks by colors.  | TRW | WDM | DWT | DWT-SVD | RivaGAN
---|---|---|---|---|---
Best Attack | Adv. Noising | Compression | Compression | Compression | Compression
Parameters | $\epsilon=\nicefrac{{2}}{{255}}$ | $r=1$ | $r=1$ | $r=1$ | $r=1$
LPIPS $\Downarrow$ | 3.2e-2 | 5.6e-2 | 7.7e-2 | 7.5e-2 | 7.3e-2
Accuracy $\Downarrow$ | 5.2% | 2.0% | 0.8% | 1.9% | 6.3%
Table 1: A summary of Pareto optimal attacks with detection accuracies less
than $10\%$. We list the attack’s name and parameters, the perceptual distance
before and after evasion, and the accuracy (TPR@1%FPR). $\epsilon$ is the
maximal perturbation in the $L_{\infty}$ norm and $r$ is the number of
compressions.

###  5.2 Image Quality after an Attack

Figure 3: A visual analysis of two adaptive attacks. The left image shows the
unwatermarked output, including a high-contrast cutout of the top left corner
of the image to visualize noise artifacts. On the right are images after
evasion with adversarial noising (top) and adversarial compression (bottom).

Figure 3 shows the perceptual quality after using our adaptive attacks. We
show a cutout of the top left image patch with high contrasts on the bottom
right to visualize noise artifacts potentially introduced by our attacks. We
observe that, unlike adversarial noising, the compression attack introduces no
new visible artifacts. Section A.3 displays more visualizations on the
perceptual impact of our attacks on the image quality.

| TRW | WDM | DWT | DWT-SVD | RivaGAN
---|---|---|---|---|---
| FID | CLIP | FID | CLIP | FID | CLIP | FID | CLIP | FID | CLIP
No WM | 23.32 | 31.76 | 23.48 | 31.77 | 23.48 | 31.77 | 23.48 | 31.77 | 23.48 | 31.77
WM | 24.19 | 31.78 | 23.43 | 31.72 | 23.16 | 32.11 | 23.10 | 32.15 | 22.96 | 31.84
A-Noise | 23.67 | 32.15 | N/A | N/A | 23.55 | 32.46 | 22.89 | 32.50 | N/A | N/A
A-Comp | 24.36 | 31.87 | 23.27 | 32.01 | 23.16 | 32.17 | 23.06 | 31.92 | 23.25 | 31.86
Table 2: Quality metrics before and after watermark evasion. FID$\Downarrow$
represents the Fréchet Inception Distance, and CLIP$\Uparrow$ represents the
CLIP score, computed on 5k images from MS-COCO-2017. N/A means the attack
could not evade watermark detection, and we do not report quality measures.

Table 2 shows the FID and CLIP score of the watermarked images and the images
after using adversarial noising and adversarial compression. We calculate the
quality using the best attack configuration from Figure 2 when the detection
accuracy is less than 10%. Adversarial Noising is ineffective at removing WDM
and RivaGAN for $\epsilon\leq\nicefrac{{10}}{{255}}$, which we denote by N/A
in Table 2.

###  5.3 Ablation Study

Figure 4 shows ablation studies for our adaptive attacks over the (i) maximum
perturbation budget and (ii) the number of compressions applied during the
attack. TRW and DWT-SVD are highly vulnerable to adversarial noising, whereas
RivaGAN and WDM are substantially more robust to these types of attacks. We
believe this is because keys generated by RivaGAN and WDM are sufficiently
randomized, which makes our attack (that uses only a single surrogate key)
less effective unless the surrogate key uses similar channels as the secret
key to hide the watermark.

Figure 4 shows that adversarial compression _without_ optimization of the
parameters $\theta_{\mathcal{A}}$ is ineffective at evading watermark
detection against all methods except DWT. After optimization, adversarial
compression evades detection from all watermarking methods with only a single
compression.

Figure 4: Ablation studies over (left) the maximum perturbation budget
$\epsilon$ in $L_{\infty}$ for adversarial noising and (right) the number of
adversarial compressions against each watermarking method. “No Optimizations"
means we did not optimize the parameters $\theta_{\mathcal{A}}$ of the attack.

##  6 Discussion & Related Work

Attack Scalability. The presented findings clearly indicate that even with a
less capable surrogate generator, an adaptive attacker can remove all surveyed
watermarks with minimal quality degradation. Our attackers generate a single
surrogate key and are able to evade watermark verification, which indicates a
design flaw since the key seems to have little impact. An interesting question
emerging from our study relates to the maximum difference between the
watermarked and surrogate generators for the attacks to remain effective. We
used two public checkpoints with the largest reported quality difference:
Stable Diffusion v1.1 and v2 for the attacker and defender respectively.

Types of Learnable Attacks. Measuring the robustness against different types
of learnable attacks is crucial in assessing the trustworthiness of
watermarking. We explored (i) Adversarial Examples, which rely solely on the
surrogate key, and (ii) Adversarial Compression, which additionally requires
the availability of a pre-trained autoencoder. We believe this requirement is
satisfied in practice, given that (i) training autoencoders is computationally
less demanding than training Stable Diffusion, and many pre-trained
autoencoders have already been made publicly available (Podell et al., 2023).
Although autoencoders enhance an attacker’s ability to modify images, our
study did not extend to other learnable attacks such as inpainting (Rombach et
al., 2022) or more potent image editing methods (Brooks et al., 2023) which
could further enhance an attack’s effectiveness.

Enhancing Robustness using Adaptive and Learnable Attacks. Relying on non-
adaptive attacks for evaluating a watermark’s robustness is inadequate as it
underestimates the attacker’s capabilities. To claim robustness, the defender
could (i) provide a certification of robustness (Bansal et al., 2022), or (ii)
showcase empirically that their watermark withstands strong attacks. The issue
is that we lack strong attackers. Although Lukas et al. (2022) demonstrated
that adaptive attackers can break watermarks for image classifiers, their
attacks were handcrafted and did not scale. Instead, we propose a better
method of empirically testing robustness by proposing adaptive _learnable_
attackers that require only the specification of a type of learnable attack,
followed by an optimization procedure to find parameters that minimize an
objective function. We believe that any watermarking method proposed in the
future should evaluate robustness using our attacks and expect that future
watermarking methods can enhance their robustness by incorporating our
attacks.

Limitations. Our attacks are based on the availability of the watermarking
algorithm and an open-source surrogate generator to replicate keys. While
providers like Stable Diffusion openly share their models, and replicas of
OpenAI’s DALL·E models are publicly available (Dayma et al., 2021), not all
providers release information about their models. To the best of our
knowledge, Google has not released their generators (Saharia et al., 2022),
but efforts to replicate are ongoing444https://github.com/lucidrains/imagen-
pytorch. Providers like Midjourney555www.midjourney.com/, who keep their image
generation algorithms undisclosed, prevent adaptive attackers altogether but
may be vulnerable to these attacks by anyone to whom this information is
released.

Ethical Considerations. The attacks we provide target academic systems and the
engineering efforts to attack real systems are substantial. We make it harder
by not releasing our code publicly. We will, however, release our code
including pre-trained checkpoints upon carefully considering each request.
Currently, there are no known security impacts of our attacks since users
cannot yet rely on the provider’s use of watermarking. The use of watermarking
is experimental and occurs at the provider’s own risk, and our research aims
to improve the trustworthiness of image watermarking by evaluating it more
comprehensively.

###  6.1 Related Work

Jiang et al. (2023) use adversarial noise to evade watermark detection, but
they require access to the secret watermarking key. Our attacks require no
access to the secret watermarking key. Peng et al. (2023); Cui et al. (2023)
propose black-box watermarking methods that protect the Intellectual Property
of Diffusion Models. We focus on no-box verifiable watermarking methods that
control misuse. Lukas and Kerschbaum (2023); Yu et al. (2020, 2021) propose
watermarking methods but only evaluate GANs. We focus on watermarking methods
for pre-trained Stable Diffusion models with much higher output diversity and
image quality (Dhariwal and Nichol, 2021).

##  7 Conclusion

We propose testing the robustness of watermarking through adaptive, learnable
attacks. Our empirical analysis shows that such attackers can evade watermark
detection at negligible degradation in image quality against all five surveyed
image watermarks. We encourage using these adaptive attacks to more
comprehensively test the robustness of watermarking methods in the future.

## References

  * Adi et al. [2018] Yossi Adi, Carsten Baum, Moustapha Cisse, Benny Pinkas, and Joseph Keshet.  Turning your weakness into a strength: Watermarking deep neural networks by backdooring.  In _27th USENIX Security Symposium (USENIX Security 18)_ , pages 1615–1631, 2018.
  * Akrout et al. [2023] Mohamed Akrout, Bálint Gyepesi, Péter Holló, Adrienn Poór, Blága Kincső, Stephen Solis, Katrina Cirone, Jeremy Kawahara, Dekker Slade, Latif Abid, et al.  Diffusion-based data augmentation for skin disease classification: Impact across original medical datasets to fully synthetic images.  _arXiv preprint arXiv:2301.04802_ , 2023.
  * Bansal et al. [2022] Arpit Bansal, Ping-yeh Chiang, Michael J Curry, Rajiv Jain, Curtis Wigington, Varun Manjunatha, John P Dickerson, and Tom Goldstein.  Certified neural network watermarks with randomized smoothing.  In _International Conference on Machine Learning_ , pages 1450–1465. PMLR, 2022.
  * Boneh et al. [2019] Dan Boneh, Andrew J Grotto, Patrick McDaniel, and Nicolas Papernot.  How relevant is the turing test in the age of sophisbots?  _IEEE Security & Privacy_, 17(6):64–71, 2019.
  * Brooks et al. [2023] Tim Brooks, Aleksander Holynski, and Alexei A Efros.  Instructpix2pix: Learning to follow image editing instructions.  In _Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition_ , pages 18392–18402, 2023.
  * Cox et al. [2007] Ingemar Cox, Matthew Miller, Jeffrey Bloom, Jessica Fridrich, and Ton Kalker.  _Digital watermarking and steganography_.  Morgan kaufmann, 2007.
  * Cui et al. [2023] Yingqian Cui, Jie Ren, Han Xu, Pengfei He, Hui Liu, Lichao Sun, and Jiliang Tang.  Diffusionshield: A watermark for copyright protection against generative diffusion models.  _arXiv preprint arXiv:2306.04642_ , 2023.
  * Dayma et al. [2021] Boris Dayma, Suraj Patil, Pedro Cuenca, Khalid Saifullah, Tanishq Abraham, Phúc Le Khac, Luke Melas, and Ritobrata Ghosh.  Dall e mini, 7 2021.  URL https://github.com/borisdayma/dalle-mini.
  * Dhariwal and Nichol [2021] Prafulla Dhariwal and Alexander Nichol.  Diffusion models beat gans on image synthesis.  _Advances in neural information processing systems_ , 34:8780–8794, 2021.
  * Fernandez et al. [2023] Pierre Fernandez, Guillaume Couairon, Hervé Jégou, Matthijs Douze, and Teddy Furon.  The stable signature: Rooting watermarks in latent diffusion models.  _arXiv preprint arXiv:2303.15435_ , 2023.
  * Goodfellow et al. [2020] Ian Goodfellow, Jean Pouget-Abadie, Mehdi Mirza, Bing Xu, David Warde-Farley, Sherjil Ozair, Aaron Courville, and Yoshua Bengio.  Generative adversarial networks.  _Communications of the ACM_ , 63(11):139–144, 2020.
  * Gowal and Kohli [2023] Sven Gowal and Pushmeet Kohli.  Identifying ai-generated images with synthid, 2023.  URL https://www.deepmind.com/blog/identifying-ai-generated-images-with-synthid.  Accessed: 2023-09-23.
  * Grinbaum and Adomaitis [2022] Alexei Grinbaum and Laurynas Adomaitis.  The ethical need for watermarks in machine-generated language.  _arXiv preprint arXiv:2209.03118_ , 2022.
  * Heusel et al. [2017] Martin Heusel, Hubert Ramsauer, Thomas Unterthiner, Bernhard Nessler, and Sepp Hochreiter.  Gans trained by a two time-scale update rule converge to a local nash equilibrium.  _Advances in neural information processing systems_ , 30, 2017.
  * Jiang et al. [2023] Zhengyuan Jiang, Jinghuai Zhang, and Neil Zhenqiang Gong.  Evading watermark based detection of ai-generated content.  _arXiv preprint arXiv:2305.03807_ , 2023.
  * Lin et al. [2014] Tsung-Yi Lin, Michael Maire, Serge Belongie, James Hays, Pietro Perona, Deva Ramanan, Piotr Dollár, and C Lawrence Zitnick.  Microsoft coco: Common objects in context.  In _Computer Vision–ECCV 2014: 13th European Conference, Zurich, Switzerland, September 6-12, 2014, Proceedings, Part V 13_ , pages 740–755. Springer, 2014.
  * Lu et al. [2022] Cheng Lu, Yuhao Zhou, Fan Bao, Jianfei Chen, Chongxuan Li, and Jun Zhu.  Dpm-solver++: Fast solver for guided sampling of diffusion probabilistic models.  _arXiv preprint arXiv:2211.01095_ , 2022.
  * Lukas and Kerschbaum [2023] Nils Lukas and Florian Kerschbaum.  Ptw: Pivotal tuning watermarking for pre-trained image generators.  _32nd USENIX Security Symposium_ , 2023.
  * Lukas et al. [2022] Nils Lukas, Edward Jiang, Xinda Li, and Florian Kerschbaum.  Sok: How robust is image classification deep neural network watermarking?  In _2022 IEEE Symposium on Security and Privacy (SP)_ , pages 787–804. IEEE, 2022.
  * Mirsky and Lee [2021] Yisroel Mirsky and Wenke Lee.  The creation and detection of deepfakes: A survey.  _ACM Computing Surveys (CSUR)_ , 54(1):1–41, 2021.
  * Peng et al. [2023] Sen Peng, Yufei Chen, Cong Wang, and Xiaohua Jia.  Protecting the intellectual property of diffusion models by the watermark diffusion process.  _arXiv preprint arXiv:2306.03436_ , 2023.
  * Peres et al. [2023] Renana Peres, Martin Schreier, David Schweidel, and Alina Sorescu.  On chatgpt and beyond: How generative artificial intelligence may affect research, teaching, and practice.  _International Journal of Research in Marketing_ , 2023.
  * Podell et al. [2023] Dustin Podell, Zion English, Kyle Lacey, Andreas Blattmann, Tim Dockhorn, Jonas Müller, Joe Penna, and Robin Rombach.  Sdxl: improving latent diffusion models for high-resolution image synthesis.  _arXiv preprint arXiv:2307.01952_ , 2023.
  * Radford et al. [2021] Alec Radford, Jong Wook Kim, Chris Hallacy, Aditya Ramesh, Gabriel Goh, Sandhini Agarwal, Girish Sastry, Amanda Askell, Pamela Mishkin, Jack Clark, et al.  Learning transferable visual models from natural language supervision.  In _International conference on machine learning_ , pages 8748–8763. PMLR, 2021.
  * Rombach et al. [2022] Robin Rombach, Andreas Blattmann, Dominik Lorenz, Patrick Esser, and Björn Ommer.  High-resolution image synthesis with latent diffusion models.  In _Proceedings of the IEEE/CVF conference on computer vision and pattern recognition_ , pages 10684–10695, 2022.
  * Saharia et al. [2022] Chitwan Saharia, William Chan, Saurabh Saxena, Lala Li, Jay Whang, Emily L Denton, Kamyar Ghasemipour, Raphael Gontijo Lopes, Burcu Karagol Ayan, Tim Salimans, et al.  Photorealistic text-to-image diffusion models with deep language understanding.  _Advances in Neural Information Processing Systems_ , 35:36479–36494, 2022.
  * Schuhmann et al. [2022] Christoph Schuhmann, Romain Beaumont, Richard Vencu, Cade Gordon, Ross Wightman, Mehdi Cherti, Theo Coombes, Aarush Katta, Clayton Mullis, Mitchell Wortsman, et al.  Laion-5b: An open large-scale dataset for training next generation image-text models.  _Advances in Neural Information Processing Systems_ , 35:25278–25294, 2022.
  * Sohl-Dickstein et al. [2015] Jascha Sohl-Dickstein, Eric Weiss, Niru Maheswaranathan, and Surya Ganguli.  Deep unsupervised learning using nonequilibrium thermodynamics.  In _International conference on machine learning_ , pages 2256–2265. PMLR, 2015.
  * Uchida et al. [2017] Yusuke Uchida, Yuki Nagai, Shigeyuki Sakazawa, and Shin’ichi Satoh.  Embedding watermarks into deep neural networks.  In _Proceedings of the 2017 ACM on international conference on multimedia retrieval_ , pages 269–277, 2017.
  * Wen et al. [2023] Yuxin Wen, John Kirchenbauer, Jonas Geiping, and Tom Goldstein.  Tree-ring watermarks: Fingerprints for diffusion images that are invisible and robust.  _arXiv preprint arXiv:2305.20030_ , 2023.
  * Yu et al. [2020] Ning Yu, Vladislav Skripniuk, Dingfan Chen, Larry Davis, and Mario Fritz.  Responsible disclosure of generative models using scalable fingerprinting.  _arXiv preprint arXiv:2012.08726_ , 2020.
  * Yu et al. [2021] Ning Yu, Vladislav Skripniuk, Sahar Abdelnabi, and Mario Fritz.  Artificial fingerprinting for generative models: Rooting deepfake attribution in training data.  In _Proceedings of the IEEE/CVF International conference on computer vision_ , pages 14448–14457, 2021.
  * Zhang et al. [2019] Kevin Alex Zhang, Lei Xu, Alfredo Cuesta-Infante, and Kalyan Veeramachaneni.  Robust invisible video watermarking with attention.  _arXiv preprint arXiv:1909.01285_ , 2019.
  * Zhang et al. [2018] Richard Zhang, Phillip Isola, Alexei A Efros, Eli Shechtman, and Oliver Wang.  The unreasonable effectiveness of deep features as a perceptual metric.  In _Proceedings of the IEEE conference on computer vision and pattern recognition_ , pages 586–595, 2018.
  * Zhao et al. [2023] Yunqing Zhao, Tianyu Pang, Chao Du, Xiao Yang, Ngai-Man Cheung, and Min Lin.  A recipe for watermarking diffusion models.  _arXiv preprint arXiv:2303.10137_ , 2023.

##  Appendix A Appendix

###  A.1 Parameters for Watermarking Methods

Tree Ring Watermark (TRW) [Wen et al., 2023]: We evaluate the Tree-
Ring${}_{\text{Rings}}$ method, which the authors state “delivers the best
average performance while offering the model owner the flexibility of multiple
different random keys". Using the author’s
implementation666https://github.com/YuxinWenRick/tree-ring-watermark, we
generate and verify watermarks using 20 inference steps, where we use no
knowledge of the prompt during verification and keep the remaining default
parameters chosen by the authors.

Watermark Diffusion Model (WDM) [Zhao et al., 2023]: As stated in the paper,
instead of stamping the model’s training data to embed a watermark, we apply
the pre-trained encoder to a generated image as a post-processing step. We
choose messages with $n=40$ bits and use the encoder architecture proposed by
[Yu et al., 2021], followed by a ResNet-50 decoder. Each call to
$\textsc{KeyGen}(\hat{\theta}_{G})$, where $\hat{\theta}_{G}$ is the surrogate
generator, trains a new autoencoder from scratch.

DWT, DWT-SVD [Cox et al., 2007] and RivaGAN [Zhang et al., 2019]. We use
32-bit messages and keep the default parameters set in the implementation used
by the Stable Diffusion models777https://github.com/ShieldMnt/invisible-
watermark.

###  A.2 Statistical Tests

Matching Bits. WDM [Zhao et al., 2023], DWT, DWT-SVD [Cox et al., 2007] and
RivaGAN [Zhang et al., 2019] encode messages $m\in\mathcal{M}$ by bits and our
goal is to verify whether message $m\in\mathcal{M}$ is present in
$x\in\mathcal{X}$ using key $\tau$. We extract $m^{\prime}$ from $x$ and want
to reject the following null hypothesis.

$H_{0}:m\text{ and }m^{\prime}\text{ match by random chance. }$

For a given pair of bit-strings of length $n$, if we denote the number of
matching bits as $k$, the expected number of matches by random chance follows
a binomial distribution with parameters $n$ and expected value $0.5$. The
p-value for observing at least $k$ matches is given by:

$p=1-\text{CDF}(k-1;n,0.5)$

Where CDF represents the cumulative distribution function of the binomial
distribution.

Matching Latents. TRW [Wen et al., 2023] leverages the forward diffusion
process of the diffusion model to reverse an image $x$ to its initial noise
representation $x_{T}$. This transformation is represented by
$m^{\prime}=\mathcal{F}(x_{T})$, where $\mathcal{F}$ denotes a Fourier
transform. The authors find that reversed real images and their
representations in the Fourier domain are expected to follow a Gaussian
distribution. The watermark verification process aims to reject the following
null hypothesis:

$H_{0}:y\text{ originates from a Gaussian distribution }N(0,\sigma^{2}_{IC})$

Here, $y$ is a subset of $m^{\prime}$ based on a watermarking mask chosen by
the provider, which determines the relevant coefficients. The test statistic,
$\eta$, denotes the normalized sum-of-squares difference between the original
embedded message $m$ and the extracted message $m^{\prime}$, which can be
complex-valued due to the Fourier transform. Specifically,

$\eta=\frac{1}{\sigma^{2}}\sum_{i}|m_{i}-m^{\prime}_{i}|^{2}$

And,

$p=\Pr\left(\chi^{2}_{|M|,\lambda}\leq\eta\mid
H_{0}\right)=\Phi_{\chi^{2}}(\eta)$

Where $\Phi_{\chi^{2}}$ represents the cumulative distribution function of the
noncentral $\chi^{2}$ distribution. We refer to Wen et al. [2023] for more
detailed descriptions of these statistical tests.

###  A.3 Qualitative of Watermarking Techniques

We refer to Figure 5 for examples of non-watermarked, watermarked, and
attacked images using the attacks summarized in Table 1.

Figure 5: Qualitative showcase of three kinds of images: non-watermarked,
watermarked with mentioned technique, and attacked images with the strongest
attack from Table 1. The p-values and text prompts are also provided.