(eccv) Package eccv Warning: Package ‘hyperref’ is loaded with option
‘pagebackref’, which is *not* recommended for camera-ready version

11institutetext: Arizona State University 22institutetext: Intel Labs
22email<EMAIL_ADDRESS>22email<EMAIL_ADDRESS>
# R.A.C.E. : Robust Adversarial Concept Erasure for Secure Text-to-Image
Diffusion Model

Changhoon Kim Kyle Min⋆ Yezhou Yang These authors contributed equally to this
work.11 0009-0000-5850-6483 22 0000-0002-3978-918X 11 0000-0003-0126-8976

###### Abstract

In the evolving landscape of text-to-image (T2I) diffusion models, the
remarkable capability to generate high-quality images from textual
descriptions faces challenges with the potential misuse of reproducing
sensitive content. To address this critical issue, we introduce Robust
Adversarial Concept Erase (RACE), a novel approach designed to mitigate these
risks by enhancing the robustness of concept erasure method for T2I models.
RACE utilizes a sophisticated adversarial training framework to identify and
mitigate adversarial text embeddings, significantly reducing the Attack
Success Rate (ASR). Impressively, RACE achieves a 30 percentage point
reduction in ASR for the “nudity” concept against the leading white-box attack
method. Our extensive evaluations demonstrate RACE’s effectiveness in
defending against both white-box and black-box attacks, marking a significant
advancement in protecting T2I diffusion models from generating inappropriate
or misleading imagery. This work underlines the essential need for proactive
defense measures in adapting to the rapidly advancing field of adversarial
challenges. Our code is publicly available:
https://github.com/chkimmmmm/R.A.C.E.

###### Keywords:

Concept Erasure Responsible Image Generative Models Secure T2I Diffusion
Models

##  1 Introduction

Figure 1:  Comparative demonstration of concept erasure, red teaming, and
robust erasure within T2I diffusion models. The ESD method [10] removes
targeted concepts from the original SD outputs, yet these concepts can be
reconstructed using UnlearnDiff [72]. Our proposed R.A.C.E. method showcases
enhanced robustness against such red teaming reconstruction efforts.

The field of text-to-image (T2I) diffusion models has garnered significant
attention for their ability to produce high-quality images that can be
adaptively generated from textual descriptions [46, 44]. This advancement is
predicated on the training of T2I models with extensive datasets, often
encompassing a range of content including copyrighted, explicit, and private
materials [52, 53, 54]. Consequently, these models possess the capacity to
inadvertently replicate protected images, potentially without user awareness
[53, 54, 2]. The misuse of T2I models by malicious actors for misinformation
or public opinion manipulation presents a significant concern [6, 34].

In response to the challenges posed by the malicious exploitation of
generative models, Stable Diffusion (SD)[46] has integrated a safety
checker[45] and advocates for the utilization of a watermarking module [71].
Despite these initiatives, the reliance on post-hoc interventions presents
limitations due to their potential for circumvention [22, 8]. Consequently,
the research community is pivoting towards formulating methodologies that
embed safety protocols directly within the image generation pipeline. This
approach ensures that content security is an intrinsic aspect of image
creation [60, 67, 66, 23, 37, 22, 8]. Although these methods are valuable for
identifying the source of content after an incident, their reactive nature
highlights the necessity for proactive strategies. This includes pioneering
techniques attempting early removal of sensitive content from T2I models [27,
10], thereby preempting the production of inappropriate or harmful material.

To address the challenge of removing undesirable content from T2I models, even
when users attempt to circumvent content restrictions, recent research has
focused on the development of concept erasure techniques within T2I diffusion
models [10, 27, 69, 13]. These techniques primarily aim to eliminate specific
concepts (e.g., “nudity”) by altering the text embeddings associated with
these concepts to neutral representations. Despite these efforts, there
remains a vulnerability wherein erased concepts can be reconstructed. This is
achieved by identifying text tokens that closely align with the visual
embeddings of the targeted concepts, thus enabling the regeneration of
prohibited content [4, 72, 55, 59, 64]. This issue is illustrated in Fig. 1,
demonstrating that even with concept erasure, T2I models can be manipulated
through prompt modification to regenerate the restricted content. This
underscores the imperative for a more robust concept erasure methodology that
can withstand such reconstruction attempts, ensuring the integrity of content
generation within T2I models.

Acknowledging the imperative for enhanced concept erasure methodologies within
T2I models, we pose a critical question: “Is it feasible to develop a concept
erasure approach that is resilient against reconstruction efforts?” In pursuit
of this, we introduce R.A.C.E. (Robust Adversarial Concept Erase), a novel
strategy aimed at bolstering the resilience of concept erasure techniques
against adversarial manipulations, as delineated in Algorithm 1. At the heart
of RACE lies an adversarial training framework, leveraging insights from the
effort of adversarial robustness [33]. Our method effectively identifies
adversarial text embeddings capable of reconstructing erased concepts and then
facilitates their integration into the T2I concept erasure workflow.

A pivotal aspect of RACE is its ability to efficiently uncover adversarial
text embeddings within a single time step of the diffusion process, an
approach elaborated in Section 3.2. This efficiency not only streamlines the
process of identifying adversarial examples but also facilitates the
integration of our adversarial attack mechanism into the concept erasure
workflow. To demonstrate the robustness of RACE, we have carried out an
extensive array of experiments. These experiments validate the effectiveness
of RACE in countering diverse red teaming strategies, with detailed results
presented in Section 4. Our empirical investigations underscore the capacity
of RACE to significantly enhance the robustness of T2I models against both
white-box and black-box attacks across a broad spectrum of target concepts,
including artistic, explicit, and object categories.

We summarize the three main contributions here:

  * $\bullet$

We are the first to present, to our knowledge, an adversarial training
approach specifically designed to fortify concept erasure methods against
prompt-based adversarial attacks without introducing additional modules.

  * $\bullet$

Our method, RACE, implements a computationally efficient adversarial attack
method that can be plugged into the concept erasing method.

  * $\bullet$

We show RACE significantly improves T2I models’ robustness against prompts
based on white/black box attacks.

##  2 Related Works

For a comprehensive overview of additional related works, please refer to the
supplementary material.

Text-to-Image Synthesis. The field of generative models has seen remarkable
advancements, notably extending their capabilities beyond generating
photorealistic images [21, 7] to include Text-to-Image (T2I) synthesis [36,
40, 49, 44, 46]. This progress has led to the development of fine-tuning
techniques that allow for the customization of T2I models to user-specific
needs [9, 48, 28, 39, 58], thereby enabling the creation of highly realistic
images that align closely with textual prompts. However, the potential for
misuse by malicious entities, using these models for purposes such as
spreading misinformation [6, 34], raises significant concerns. This
underscores the urgency of devising protective measures to mitigate the risk
of such exploitations.

Advanced Techniques in Concept Erasure for T2I Diffusion Models. Within the
realm of machine unlearning, concept erasure for T2I Diffusion models has
recently emerged as a critical area of research, focusing on the removal of
sensitive or copyrighted concepts from T2I models. Methods to achieve this
include guiding the image generation process or adjusting the model’s weights
to exclude these elements [51, 30, 35, 10, 27, 69, 13]. Notably, techniques by
Gandikota et al.[10] and Kumari et al.[27] involve mapping sensitive concepts
to null entities or benign equivalents by fine-tuning the weights of Stable
Diffusion (SD) [46] models, effectively preventing the generation of
undesirable content. Despite these advancements, red teaming methods have
exposed potential loopholes, indicating that erased concepts might be
regenerated through meticulously designed text prompts. Addressing this issue,
our work contributes an adversarial training strategy aimed at bolstering the
resilience of Stable Diffusion models against such text-prompt-based attacks
[4, 72, 59], thereby enhancing the security and integrity of the content
generation.

Robustness Evaluation via Red Teaming in T2I Models. While various safety
measures have been proposed to shield SD models from misuse, red teaming
strategies reveal vulnerabilities that still allow for circumvention. Research
has demonstrated that techniques like Textual Inversion [9] can be exploited
to regenerate content previously erased from SD models [41], prompting the
development of countermeasures aimed at safeguarding against such inversions
[63, 73]. In real-world applications, T2I services such as Midjourney
predominantly rely on user-provided text prompts, making them susceptible to
prompt-based red teaming attacks [4, 72, 55, 59, 64]. These methods employ
sophisticated prompt optimization techniques to restore images containing
erased content, with their efficacy contingent upon the level of model
access—categorized into white-box approaches, which utilize SD’s U-Net [47]
for prompt optimization [4, 72], and black-box strategies, where such access
is restricted [55, 59, 64]. Both approaches establish formidable benchmarks in
attack success rates, as detailed in Tab. 2. However, the landscape lacks
robust defense mechanisms against prompt-based red teaming, primarily due to
the prohibitive computational demands associated with identifying adversarial
prompts—a challenge that renders traditional adversarial training approaches
impractical. Addressing this gap, our work introduces a novel defense strategy
tailored to counteract prompt-based red teaming attacks, marking a significant
step forward in fortifying T2I diffusion models against adversarial threats.

##  3 Method

Algorithm 1 Robust Adversarial Concept Erasure: RACE Algorithm

Input: Diffusion Model $\Phi_{\theta}$, frozen diffusion model
$\Phi_{\theta*}$, scheduler $\mathcal{S}$, target concept $c$, training steps
$M$, adversarial steps $N$, perturbation limit $\epsilon$, attack step size
$\alpha$

for $i=0,\dots,M$ do

Sample noise $n\sim\mathcal{N}(0,1)$, timestep $t\sim\mathcal{U}(1,1000)$

Initialize $\delta\sim\mathcal{U}(-\epsilon,\epsilon)$

Denoise $z_{t}=\mathcal{S}(n,t,c)$

for $j=0,\dots,N$ do $\triangleright$ Perform targeted attack

$\delta=\delta+\alpha\cdot
sign(\nabla_{\delta}-L_{SD}(\Phi_{\theta},z_{t},t,c,\delta))$

Clamp $\delta$ within $[-\epsilon,\epsilon]$

end for

$\theta=\theta-\nabla_{\theta}L_{RACE}(\Phi_{\theta},\Phi_{\theta^{*}},z_{t},t,c,\delta)$

end for

return $\Phi_{\theta}$

Figure 2:  Single-Timestep Adversarial Attack Efficacy. This figure
illustrates the Attack Success Rate (ASR) across various timesteps, alongside
representative images. Notably, even when the adversarial attack is applied at
a singular timestep $t^{*}$, the perturbed text embedding $c+\delta_{t^{*}}$
successfully reproduces images containing the previously erased concept. For
method details, see Sec. 3.2.

Our methodology aims to expunge target concepts from T2I diffusion models
through an adversarial training framework [33]. Initially, in Section 3.1, we
establish the foundation by formally introducing the notations and the
rationale underpinning our approach. Following this, Section 3.2 details our
proposed adversarial attack, specifically designed for robust concept erasure,
and delineates its integration into the adversarial training regime.

###  3.1 Preliminaries

Stable Diffusion Models. Our method is built upon the Stable Diffusion Model
(SD)[46], which operates as the foundational architecture for our concept
erasure technique. The SD model is composed of two primary elements: firstly,
an image autoencoder that has been pre-trained on a diverse and extensive
image dataset[7]. Within this autoencoder, an encoder function
$\mathcal{E}(\cdot)$ transforms an input image $x$ into a latent
representation $z=\mathcal{E}(x)$. Conversely, a decoder function
$\mathcal{D}(\cdot)$ aims to reconstruct the input image from its latent form,
where $\mathcal{D}(z)=\hat{x}\approx x$.

The second element is a U-Net [47]-based diffusion model trained to craft
latent representations within the acquired latent space. This model
facilitates the conditioning on either class labels or text embeddings derived
from training data. Let us denote by $c=\mathcal{E}_{txt}(y)$ the textual
embedding encoded from a conditioning text prompt $y$, where
$\mathcal{E}_{txt}$ symbolizes the text encoder, such as CLIP [43]. Under
these constructs, the SD training objective is encapsulated by the loss
function:

$L_{SD}:=\mathbb{E}_{n\sim\mathcal{N}(0,1),z,c,t}\left[||n-\Phi_{\theta}(z_{t},t,c)||_{2}^{2}\right],$
(1)

where $t$ indexes the time step, $n$ represents a noise sample drawn from a
standard Gaussian distribution, $z_{t}$ is the perturbed version of $z$ up to
time step $t$, and $\Phi_{\theta}$ is the denoising network based on a U-Net
architecture. During inference, a random noise sample is procured from a
Gaussian distribution and denoised using $\Phi_{\theta}$ following a scheduler
$\mathcal{S}$, operating over a sequence of predetermined time steps $T$. The
resulting denoised latent, $z_{0}$, is then decoded to produce the final
image, $\hat{x}=\mathcal{D}(z_{0})$.

Erase and Reconstruction of Target Concept. The objective of concept erasure
is to remove specific concepts, such as “nudity”, from the latent space of a
pre-trained diffusion model. The Erased Stable Diffusion (ESD) approach [10]
introduces a method for excising these concepts from the latent
representations within the Stable Diffusion framework. The erasure loss is
formalized as follows:

$L_{erase}:=||\Phi_{\theta}(z_{t},t,c)-(\Phi_{\theta^{*}}(z_{t},t)-\eta(\Phi_{\theta^{*}}(z_{t},t,c)-\Phi_{\theta^{*}}(z_{t},t)))||_{2}^{2},$
(2)

where $\Phi_{\theta^{*}}$ represents the frozen denoising U-Net, and $\eta$
denotes the guidance scale associated with classifier-free guidance [17]. ESD
fine-tunes $L_{erase}$ with respect to $\theta$, guiding $\Phi_{\theta}$ to
produce outputs where the target concept is effectively nullified. Crucially,
this process does not necessitate additional datasets; it operates
successfully with only a concise textual description.

Conversely, red teaming efforts aim to counteract concept erasure by crafting
adversarial text prompts $\tilde{y}$ capable of resurrecting the erased
concept in the generated image. White-box methods [72, 4] engage in this
adversarial prompt optimization by leveraging the gradients of
$\Phi_{\theta}$. Meanwhile, black-box approaches [55, 59, 64] aim to achieve
comparable outcomes without reliance on the gradients of $\Phi_{\theta}$. Both
approaches pose significant computational demands, which presents challenges
for their integration into an adversarial training framework.

###  3.2 Adversarial Training on Concept Erased Diffusion Models

Motivation. The primary aim of T2I diffusion models is to generate high-
quality images conditioned on specific prompts. Intriguingly, the SD model’s
loss function, as depicted in Eq.(1), can also facilitate image classification
tasks [29]. This classification capability is derived by applying Bayes’
Theorem to the model’s predictions $p_{\theta}(x|c_{i})$ and the prior
distribution $p(c)$ across a set of conditions ${c_{i}}$, where each
$c_{i}=\mathcal{E}_{txt}(y_{i})$ represents a textual embedding of the prompt
$y_{i}$:

$p_{\theta}(c_{i}|x)=\frac{p(c_{i})p_{\theta}(x|c_{i})}{\sum_{j}p(c_{j})p_{\theta}(x|c_{j})}.$
(3)

Notably, the prior terms $p(c)$ can be disregarded when they are uniformly
distributed over the prompts ${c_{i}}$ (i.e., $p(c_{i})=\frac{1}{N}$). In the
context of diffusion models, directly computing $p_{\theta}(x|c_{i})$ is
computationally challenging, leading to the reliance on the computation of
$\log{p_{\theta}(x|c_{i})}$ and the utilization of the Evidence Lower Bound
(ELBO) for optimization purposes. Leveraging approximations introduced in
[16], we can approximate the posterior distribution over prompts ${c_{i}}$ as
follows:

$p_{\theta}(c_{i}|x)=\frac{exp\\{-\mathbb{E}_{z,n,t}\left[||n-\Phi_{\theta}(z_{t},t,c_{i})||^{2}\right]\\}}{\sum_{j}exp\\{-\mathbb{E}_{z,n,t}\left[||n-\Phi_{\theta}(z_{t},t,c_{j})||^{2}\right]\\}}.$
(4)

Building on this foundation, the Diffusion Classifier [29] proposes a method
for estimating the class of a given image $x$ by finding
$\operatorname{argmin}$ of the following expression:

$\operatorname*{argmin}_{\begin{subarray}{c}c\end{subarray}}\sum_{t=1}^{T}\left[||n-\Phi_{\theta}(z_{t},t,c)||^{2}\right],$
(5)

where $z_{t}=\mathcal{E}(x)$ represents the latent encoding of the image $x$
and the goal is to identify the label $c$ from the available set of classes
${c_{i}}$. A notable insight from the Diffusion Classifier is the feasibility
of classifying image $x$ even when computations are performed using a single
time step $t^{*}$.

Single-Timestep Adversarial Attacks on Erased T2I Models. Leveraging insights
from the Diffusion Classifier, we re-conceptualize the SD model’s loss
function in Eq.(1) as a classification mechanism. This perspective allows us
to view Textual Inversion (TI)[9] as a form of targeted adversarial attack,
where the objective is to optimize the conditioning text embedding $c$ to
regenerate the image $x$ [41, 72]. Notably, TI is computationally intensive as
it necessitates optimization across all time steps.

Prompted by these considerations, our investigation centers around a critical
inquiry: Can adversarial text embeddings be identified with just a single
timestep? We investigate whether adversarial text embeddings can be
effectively identified at a singular timestep $t^{*}$. Our approach is geared
towards nullifying the embedding of a target concept $c$ and its proximate
embeddings that might facilitate the regeneration of an erased concept image
$\tilde{x}$, such as an explicit image. To this end, we devise a targeted
adversarial attack to produce $\tilde{x}$ from a concept-erased diffusion
model $\Phi_{\theta}$ by introducing an adversarial perturbation $\delta$. The
perturbation $\delta$ is determined through the optimization:

$\operatorname*{argmin}_{\begin{subarray}{c}||\delta||_{\infty}\leq\epsilon\end{subarray}}||n-\Phi_{\theta}(\tilde{z_{t}},t,c+\delta)||_{2}^{2},$
(6)

where $\tilde{z}=\mathcal{E}(\tilde{x})$ denotes the latent representation of
image $\tilde{x}$, $\epsilon$ is a small number, and $c=\mathcal{E}_{txt}(y)$
encodes the textual embedding of the targeted concept, for instance,
$y=$“nudity”. The Projected Gradient Descent (PGD) algorithm [33] is employed
to address this optimization problem. Specifically, when selecting timestep
$t^{*}=500$ as the critical adversarial point, $\tilde{z_{t}}$ undergoes
denoising via $\Phi_{\theta}(\tilde{z_{t}},t,c)$ transitioning from timesteps
$t=1000$ to $t=500$. Subsequently, the targeted adversarial approach outlined
in Eq. (6) is executed to determine $\delta_{t^{*}}$ at $t=500$. In subsequent
denoising steps, $\tilde{z_{t}}$ is denoised considering the introduced
perturbation $c+\delta_{t^{*}}$.

Our method introduces a distinctive single-timestep adversarial attack,
contrasting with prior approaches that required optimization over a wide range
of timesteps [4, 72]. This approach enables the seamless incorporation of our
attack strategy into the adversarial training process specifically for T2I
concept erasure.

To assess the efficacy of our method, we execute tests on a $\Phi_{\theta}$
model trained for removing the “nudity” concept via the ESD method. Utilizing
142 nudity-centric prompts from the I2P dataset [51], we systematically select
timesteps $t^{*}$ at 100-step intervals within the $[1,1000]$ range and
compute the Attack Success Rate (ASR), as formulated in Eq.(8), for each
$t^{*}$. The results, depicted in Fig. 2, unveil a notable capability: the
reconstruction of the erased concept is feasible even with the attack confined
to a single timestep $t^{*}$. Interestingly, after $t^{*}=500$, we observe an
increasing trend in the ASR. The modified images resulting from the attack
visually convey the transition back to the erased concept and reflect the ASR
patterns. For additional insights into various concepts and their
corresponding ASR trends across different timesteps $t^{*}$, the supplementary
material offers further information.

Table 1: Performance comparison of $L_{erase}$ and $L_{RACE}$

$L_{erase}$ $L_{RACE}$ I2P [51] 0.08 0.05 FID [15] 33.12 25.16 CLIP-Score [14]
0.726 0.745

Adversarial Training for T2I Concept Erasure. Motivated by the findings from
our single-timestep adversarial attack experiments, we explore the potential
of such attacks to enhance the robustness of concept erasure in T2I models,
posing the question: Can adversarial attacks improve the resilience of concept
erasure mechanisms?

RACE distinguishes itself from existing approaches [10, 27] by aiming to
eliminate not only the targeted concept’s embedding but also its adjacent
embedding within the model’s latent space, which could otherwise lead to the
inadvertent generation of the erased concept by $\Phi_{\theta}$. We
incorporate our adversarial attack into the erasure loss function
($L_{erase}$), yielding an enhanced adversarial training loss:

$L_{RACE}:=||\Phi_{\theta}(z_{t},t,c+\delta)-(\Phi_{\theta^{*}}(z_{t},t)-\eta(\Phi_{\theta^{*}}(z_{t},t,c)-\Phi_{\theta^{*}}(z_{t},t)))||_{2}^{2}.$
(7)

This method is deliberate, substituting the concept embedding $c$ with
$c+\delta$ within the trainable parameters of $\Phi_{\theta}$. This precise
adjustment ensures enhanced fidelity of the generated images by mapping the
$\epsilon$-neighborhood of the concept embedding to its null representation.
Comparative metrics between the direct substitution in $L_{erase}$ and the
strategic use of $L_{RACE}$ are provided in Tab. 1. The latter approach
demonstrates promising reductions in ASR for “nudity” prompts within I2P
dataset [51] and improvements in image quality metrics, as assessed on the MS-
COCO [31]. The RACE methodology is comprehensively detailed in Algorithm 1. To
validate RACE’s effectiveness, we evaluate the ASR against both white-box and
black-box attacks, as elaborated in Sec. 4.

##  4 Experiments

###  4.1 Experimental Setting

Datasets. Our assessment of the RACE framework spans various domains,
including artistic styles, explicit concepts, and identifiable objects, in
line with established benchmarks [72, 4, 10]. To ensure a uniform image
generation process, we standardize key hyperparameters such as the scale of
classifier-free guidance and random seeds. Artistic style evaluations leverage
shared text prompts from ESD [10] and UnlearnDiff [72]. For explicit content,
we draw from the inappropriate image prompt benchmark [51], selecting a
diverse set of prompts encompassing 142 nudity, 98 illegal acts, and 101
violence instances. Objects are curated from a subset of Imagenette [19],
known for its distinct and recognizable classes, with text prompts synthesized
via ChatGPT [38] following the approach by Kumari et al. [27]. It’s crucial to
note that these datasets serve solely to gauge the effectiveness of
adversarial attacks; the adversarial training component of RACE does not
necessitate the use of these specific prompts.

Training Details. In validating RACE, we choose the Erased Stable Diffusion
(ESD) model [10] for its ability to erase a wide range of concepts, serving as
an ideal testbed to showcase RACE’s efficacy. We integrate RACE with ESD,
optimizing with attack parameters $\epsilon=0.1$ and $\alpha=\epsilon/4$. The
optimization of model parameters $\theta$ utilizes the Adam optimizer [26] at
a learning rate of $1e-5$, consistent with ESD. Additional details on attack
parameter selection are available in the supplementary material.

Red Teaming Methods. To rigorously test RACE’s robustness, we deploy a
comprehensive suite of adversarial attacks, spanning both white-box and black-
box approaches. Initial assessments utilize the I2P red teaming prompt dataset
[51]. In black-box scenarios, we employ PEZ [59], which crafts adversarial
prompts via CLIP [43]. In the white-box scenario, methods like P4D [4] and
UnlearnDiff [72] are used, which generate adversarial prompts by leveraging
gradients from the SD.

Evaluation. To gauge the robustness of RACE, we employ domain-specific
classifiers: a ViT-base model [61] pre-trained on ImageNet [5] and fine-tuned
on WikiArt [50] for artistic styles, Nudenet [1] for explicit content, and
ResNet-50 [12] trained on ImageNet for object removal. We measure robustness
using the Attack Success Rate (ASR):

$\text{ASR}=\frac{1}{N}\sum_{i=1}^{N}\mathbbm{1}\left(f\left(SD(\tilde{y_{i}})\right)=\tilde{y_{i}}\right),$
(8)

where $\tilde{y_{i}}$ is the adversarial prompt, $f$ is the classifier, and
$N$ is the number of prompts. Additionally, we assess image quality after
applying RACE by generating 5,000 images from the MS-COCO [31] test set,
computing the Frechet Inception Distance (FID) score [15] and the CLIP score
[14] to evaluate RACE’s impact on image fidelity while ensuring concept
erasure.

Table 2: Attack Success Rate (ASR) against white (○)/black (●) box attacks. We
conduct experiments on artistic style (Van Gogh), explicit concepts (nudity,
violence, and illegal acts), and objects (church, golf ball, and parachute).
“ESD/RACE-Concept” denotes the concept erased from the model. We also measure
the quality of T2I models. We can observe that RACE reduces ASR by over 30%
(-0.30 or below) in absolute value on Van Gogh, nudity, and church for
UnlearnDiff [72], which is the previous SOTA attack method.

Prompts PEZ [59] P4D [4] UnlearnDiff [72] CLIP-Score [14] FID [15] White/Black
Box ● ● ○ ○ - - ESD[10]-VanGogh 0.04 0.00 0.26 0.36 0.7997 19.16
ESD[10]-Nudity 0.14 0.08 0.75 0.80 0.7931 18.88 ESD[10]-Violence 0.27 0.13
0.84 0.79 0.7834 21.55 ESD[10]-Illegal 0.29 0.20 0.89 0.85 0.7854 21.50
ESD[10]-Church 0.16 0.00 0.58 0.68 0.7896 19.68 ESD[10]-GolfBall 0.04 0.00
0.16 0.16 0.7738 20.64 ESD[10]-Parachute 0.06 0.04 0.48 0.60 0.7865 19.72
RACE-VanGogh 0.00 (-0.04) 0.00 (-0.00) 0.00 (-0.26) 0.04 (-0.32) 0.8024 20.65
RACE-Nudity 0.05 (-0.09) 0.02 (-0.06) 0.49 (-0.26) 0.47 (-0.33) 0.7452 25.16
RACE-Violence 0.11 (-0.16) 0.08 (-0.05) 0.75 (-0.09) 0.68 (-0.11) 0.7374 28.71
RACE-Illegal 0.20 (-0.09) 0.13 (-0.07) 0.85 (-0.04) 0.80 (-0.05) 0.7591 24.87
RACE-Church 0.02 (-0.14) 0.00 (-0.00) 0.26 (-0.32) 0.38 (-0.30) 0.7730 23.92
RACE-GolfBall 0.00 (-0.04) 0.00 (-0.00) 0.10 (-0.06) 0.06 (-0.10) 0.7480 25.38
RACE-Parachute 0.02 (-0.04) 0.00 (-0.04) 0.24 (-0.24) 0.38 (-0.22) 0.7570
26.42

Figure 3: Although ESD significantly reduces the chance of generating images
with exposed body parts, state-of-the-art red teaming methods, such as
UnlearnDiff, can be used to bypass ESD’s defense and reconstruct explicit
content. RACE and its variant can effectively defend the malicious attempts to
reconstruct explicit content from the ESD model that erased the concept of
nudity.

###  4.2 Robust Concept Erase against Red Teaming

In our comprehensive analysis, RACE undergoes a series of red teaming
evaluations [4, 72, 51, 59], encompassing both white-box and black-box
techniques aimed at regenerating concepts targeted by RACE for erasure, as
depicted in Fig.1. The comparative analysis of ASR presented in Tab.2 spans
diverse conceptual domains, from “Van Gogh”-inspired artistry to explicit
content such as “nudity” and “violence”, extending to tangible objects like
“churches”, “golf balls”, and “parachutes”. Remarkably, RACE consistently
diminishes ASR, notably surpassing 30% for “Van Gogh” styles, “nudity”, and
“church” categories, particularly outperforming UnlearnDiff [72], the current
state-of-the-art in white-box adversarial methodologies. This marked decline
in ASR underscores RACE’s heightened robustness and delineates its capability
as a potent, computationally efficient defense mechanism for T2I diffusion
models against intricate adversarial attacks. Crucially, RACE’s methodological
advantage stems from its independence from external imagery or prompts,
diverging from traditional red teaming techniques reliant on such data for
prompt generation.

As demonstrated in Tab.2, RACE achieves a significant 33% reduction in ASR for
the “nudity” concept, underscoring its effectiveness. To further elucidate how
RACE enhances ASR, we analyze the specific categories or elements it targets
for removal. Utilizing Nudenet, we enumerate the body parts generated by
various models—original SD, ESD, UnlearnDiff, and RACE—when prompted with
nudity-related inputs from the I2P dataset. Illustrated in Fig.3, the analysis
reveals that while ESD enhances the original SD’s resilience to such prompts,
UnlearnDiff manages to bypass ESD’s defenses, reconstructing explicit content.
In contrast, RACE maintains its robustness even against the sophisticated
UnlearnDiff attacks, showcasing the advanced protective capabilities of our
approach in safeguarding against the regeneration of sensitive content.

One caveat to mention is that our experiments reveal a nuanced trade-off
between robustness and image quality. While artistic style erasures maintain
quality metrics, erasures of other concepts inadvertently degrade image
quality. This divergence could be attributed to methodological differences;
erasing artistic styles predominantly involves fine-tuning the cross-attention
layers of SD as per ESD guidelines, whereas erasing other concepts
necessitates adjustments in non-cross-attention layers [10]. Another plausible
explanation for the observed trade-off between robust concept erasure and
overall image quality could relate to the inherent complexity of
differentiating between closely related or overlapping concepts within the
model’s latent space. As RACE intensifies adversarial robustness, it may
inadvertently alter the delicate equilibrium within these conceptual overlaps,
leading to unintended modifications in adjacent, non-targeted conceptual
representations. This issue highlights the complex tension between precise
concept erasure and maintaining the model’s overall integrity against attacks.
To address the quality concerns arising from this trade-off, we explore a
potential strategy for improvement in Sec. 4.4, aiming to reduce the trade-off
between targeted erasure with high quality and the model’s defensive
robustness.

###  4.3 Disentanglement

Figure 4: RACE’s Disentanglement in Concept Erasure. This figure highlights
RACE’s precision in erasing specific concepts, as shown in diagonal images,
while preserving unrelated concepts, which is evident in off-diagonal images.
For reference, baseline images generated by the original Stable Diffusion (SD)
model are also presented.

Investigating RACE’s disentanglement performance, our study focuses on its
capability to precisely erase intended concepts without impacting other
elements. The evaluation spans both qualitative and quantitative measures. On
the qualitative front, we configure separate RACE-enhanced models to
specifically erase artistic imprints such as “Van Gogh”, “Thomas Kinkade”, and
“Kilian Eng” from the SD. The illustrative outcomes, showcased in Fig. 4,
underscore RACE’s erasure precision, ensuring that the excision of one
artistic style doesn’t lead to the collateral removal of others. This
meticulous erasure extends to discrete object concepts like “church”, “golf
ball”, and “parachute”, with the generated images further affirming the
method’s discernment.

Table 3: Accruacy of erased and non-erased classes in Imagenet [5]. We
evaluate the classification accuracy of ESD and RACE for the target erased
concepts and other non-target concepts.

Acc. Erased Acc. Others Erased Concept ESD RACE ESD RACE Church 0.16 0.02 0.57
0.53 Golf Ball 0.04 0.00 0.45 0.35 Parachute 0.06 0.02 0.57 0.41

Quantitatively, we generate 5,000 prompts with varied random seeds, such as
“an image of a [class name]”, to produce a diverse set of images, subsequently
evaluated using a pre-trained ResNet-50 classifier for top-1 accuracy. As
shown in Tab. 3, RACE showcases an improved capability for object concept
erasure compared to ESD. Despite this, there exists a slight decrement in
classification accuracy for non-target classes. This effect likely stems from
RACE’s method of targeting the $\epsilon$-neighborhood surrounding the
intended concept, potentially influencing proximate concepts. Nevertheless, we
can observe from Fig. 4 that RACE can precisely erase the target concept with
minimal visual impacts on other concepts.

###  4.4 Discussion

Table 4:  Ablation Studies for Performance Improvement. Using the “nudity”
concept, we evaluate the effectiveness of adding weight regularization and
close concept keywords. ASR and quality metrics are measured to assess
performance.

I2P [51] PEZ [59] P4D [4] UnleranDiff [72] CLIP-Score [14] FID [15] ESD 0.14
0.08 0.75 0.80 0.7931 18.88 RACE 0.05 0.02 0.49 0.47 0.7452 25.16 RACE+Reg.
0.07 0.02 0.60 0.62 0.7593 24.42 RACE+keywords 0.02 0.01 0.42 0.46 0.7201
30.97

Potential Strategy to Improve the Robustness-Quality Trade-off. Our findings
underscore the ability of RACE to significantly bolster the SD model’s defense
against prompt-based adversarial attacks across a variety of concepts.
However, as illustrated in Tab.2, enhancing robustness appears to inversely
impact image quality. To address this dichotomy, we test a refined version of
the RACE loss function incorporating a regularization term:

$L_{RACE+Reg.}:=L_{RACE}+\lambda||\theta-\theta^{*}||_{1},$ (9)

where $\theta$ and $\theta^{*}$ represent the parameters of the RACE and
original Stable Diffusion models, respectively, and $\lambda$ is the
regularization strength, set to $0.1$ in our experiment. This regularization
approach, as evidenced in Tab.4, helps to partially reconcile the robustness-
quality trade-off, enhancing image quality while maintaining improved
robustness over ESD.

Enhancing Concept Erasure. In pursuit of further reducing the ASR, we explore
strategies for more comprehensive concept erasure. Recognizing that a target
concept may manifest in various synonymous forms, we extend RACE’s erasure
scope to include semantically related concepts. Leveraging the CLIP text
encoder embedded within Stable Diffusion, we identify and subsequently erase
concepts closely related to the target, based on their proximity in the CLIP
embedding space. For instance, alongside “nudity”, we also target synonymous
concepts like “nude”, “nsfw”, and “bare”, identified as the top-3 semantically
similar terms. In Fig. 3, we can observe that our method equipped with this
expanded erasure strategy (denoted as RACE+keywords) is more effective in
defending the malicious attempts to bypass the ESD by further reducing the
number of exposed body parts. Tab. 4 also indicates that our strategy indeed
fortifies the model’s robustness against red teaming tactics. Nonetheless,
this broadened concept removal spectrum reaffirms the robustness-quality
trade-off, manifesting as a decrement in image quality. This aspect opens an
intriguing avenue for future enhancements to the RACE methodology, balancing
the twin objectives of robust concept erasure and preserved image fidelity.

##  5 Conclusion

In this work, we present RACE, a novel defense approach designed to protect
the Text-to-Image Stable Diffusion models from prompt-based red teaming
attacks. RACE effectively strengthens the model’s concept erasure capabilities
while maintaining computational efficiency, offering a valuable enhancement to
the current erasure framework and bolstering defenses against various
adversarial techniques. We also observe the robustness-quality trade-off and
discuss possible future directions to improve it. This initial contribution
lays the groundwork for further exploration, underscoring the critical
importance of developing sophisticated defenses in the rapidly evolving domain
of generative AI.
Acknowledgements. We would like to express our gratitude to Sangmin Jung for
assisting with the experiments. This work is partially supported by the
National Science Foundation under Grant No. 2038666 and No. 2101052. The views
and opinions of the authors expressed herein do not necessarily state or
reflect those of the funding agencies and employers.

## References

  * [1] Bedapudi, P.: Nudenet: Neural nets for nudity classification, detection and selective censoring (12 2019)
  * [2] Carlini, N., Hayes, J., Nasr, M., Jagielski, M., Sehwag, V., Tramèr, F., Balle, B., Ippolito, D., Wallace, E.: Extracting training data from diffusion models. ArXiv abs/2301.13188 (2023), https://api.semanticscholar.org/CorpusID:256389993
  * [3] Carlini, N., Wagner, D.A.: Towards evaluating the robustness of neural networks. 2017 IEEE Symposium on Security and Privacy (SP) pp. 39–57 (2016), https://api.semanticscholar.org/CorpusID:2893830
  * [4] Chin, Z.Y., Jiang, C.M., Huang, C.C., Chen, P.Y., Chiu, W.C.: Prompting4debugging: Red-teaming text-to-image diffusion models by finding problematic prompts. arXiv preprint arXiv:2309.06135 (2023)
  * [5] Deng, J., Dong, W., Socher, R., Li, L.J., Li, K., Fei-Fei, L.: Imagenet: A large-scale hierarchical image database. In: 2009 IEEE conference on computer vision and pattern recognition. pp. 248–255. Ieee (2009)
  * [6] Devlin, K., Cheetham, J.: Fake trump arrest photos: How to spot an ai-generated image. BBC News (March 2023), https://www.bbc.com/news/technology-68981525
  * [7] Esser, P., Rombach, R., Ommer, B.: Taming transformers for high-resolution image synthesis. In: Proceedings of the IEEE/CVF conference on computer vision and pattern recognition. pp. 12873–12883 (2021)
  * [8] Fernandez, P., Couairon, G., Jégou, H., Douze, M., Furon, T.: The stable signature: Rooting watermarks in latent diffusion models. arXiv preprint arXiv:2303.15435 (2023)
  * [9] Gal, R., Alaluf, Y., Atzmon, Y., Patashnik, O., Bermano, A.H., Chechik, G., Cohen-Or, D.: An image is worth one word: Personalizing text-to-image generation using textual inversion. ArXiv abs/2208.01618 (2022), https://api.semanticscholar.org/CorpusID:251253049
  * [10] Gandikota, R., Materzynska, J., Fiotto-Kaufman, J., Bau, D.: Erasing concepts from diffusion models. 2023 IEEE/CVF International Conference on Computer Vision (ICCV) pp. 2426–2436 (2023), https://api.semanticscholar.org/CorpusID:257495777
  * [11] Goodfellow, I.J., Shlens, J., Szegedy, C.: Explaining and harnessing adversarial examples. CoRR abs/1412.6572 (2014), https://api.semanticscholar.org/CorpusID:6706414
  * [12] He, K., Zhang, X., Ren, S., Sun, J.: Deep residual learning for image recognition. In: Proceedings of the IEEE conference on computer vision and pattern recognition. pp. 770–778 (2016)
  * [13] Heng, A., Soh, H.: Selective amnesia: A continual learning approach to forgetting in deep generative models. ArXiv abs/2305.10120 (2023), https://api.semanticscholar.org/CorpusID:258740988
  * [14] Hessel, J., Holtzman, A., Forbes, M., Bras, R.L., Choi, Y.: Clipscore: A reference-free evaluation metric for image captioning. arXiv preprint arXiv:2104.08718 (2021)
  * [15] Heusel, M., Ramsauer, H., Unterthiner, T., Nessler, B., Hochreiter, S.: Gans trained by a two time-scale update rule converge to a local nash equilibrium. In: Advances in Neural Information Processing Systems. pp. 6626–6637 (2017)
  * [16] Ho, J., Jain, A., Abbeel, P.: Denoising diffusion probabilistic models. Advances in neural information processing systems 33, 6840–6851 (2020)
  * [17] Ho, J., Salimans, T.: Classifier-free diffusion guidance. arXiv preprint arXiv:2207.12598 (2022)
  * [18] Hong, S., Lee, J., Woo, S.S.: All but one: Surgical concept erasing with model preservation in text-to-image diffusion models. ArXiv abs/2312.12807 (2023), https://api.semanticscholar.org/CorpusID:266374816
  * [19] Howard, J., Gugger, S.: fastai: A layered api for deep learning. Inf. 11, 108 (2020), https://api.semanticscholar.org/CorpusID:211082837
  * [20] Huang, C.P., Chang, K.P., Tsai, C.T., Lai, Y.H., Wang, Y.C.F.: Receler: Reliable concept erasing of text-to-image diffusion models via lightweight erasers. ArXiv abs/2311.17717 (2023), https://api.semanticscholar.org/CorpusID:265498506
  * [21] Karras, T., Laine, S., Aittala, M., Hellsten, J., Lehtinen, J., Aila, T.: Analyzing and improving the image quality of stylegan. In: Proceedings of the IEEE/CVF conference on computer vision and pattern recognition. pp. 8110–8119 (2020)
  * [22] Kim, C., Min, K., Patel, M., Cheng, S., Yang, Y.: Wouaf: Weight modulation for user attribution and fingerprinting in text-to-image diffusion models. arXiv preprint arXiv:2306.04744 (2023)
  * [23] Kim, C., Ren, Y., Yang, Y.: Decentralized attribution of generative models. In: International Conference on Learning Representations (2021)
  * [24] Kim, S., Jung, S., Kim, B., Choi, M., Shin, J., Lee, J.: Towards safe self-distillation of internet-scale text-to-image diffusion models. ArXiv abs/2307.05977 (2023), https://api.semanticscholar.org/CorpusID:259837117
  * [25] Kinfu, K.A., Vidal, R.: Analysis and extensions of adversarial training for video classification. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition. pp. 3416–3425 (2022)
  * [26] Kingma, D.P., Ba, J.: Adam: A method for stochastic optimization (2014), http://arxiv.org/abs/1412.6980, cite arxiv:1412.6980Comment: Published as a conference paper at the 3rd International Conference for Learning Representations, San Diego, 2015
  * [27] Kumari, N., Zhang, B., Wang, S.Y., Shechtman, E., Zhang, R., Zhu, J.Y.: Ablating concepts in text-to-image diffusion models. 2023 IEEE/CVF International Conference on Computer Vision (ICCV) pp. 22634–22645 (2023), https://api.semanticscholar.org/CorpusID:257687839
  * [28] Kumari, N., Zhang, B., Zhang, R., Shechtman, E., Zhu, J.Y.: Multi-concept customization of text-to-image diffusion. 2023 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) pp. 1931–1941 (2022), https://api.semanticscholar.org/CorpusID:254408780
  * [29] Li, A.C., Prabhudesai, M., Duggal, S., Brown, E., Pathak, D.: Your diffusion model is secretly a zero-shot classifier. In: Proceedings of the IEEE/CVF International Conference on Computer Vision (ICCV). pp. 2206–2217 (October 2023)
  * [30] Li, S., van de Weijer, J., Hu, T., Khan, F.S., Hou, Q., Wang, Y., Yang, J.: Get what you want, not what you don’t: Image content suppression for text-to-image diffusion models. ArXiv abs/2402.05375 (2024), https://api.semanticscholar.org/CorpusID:267547985
  * [31] Lin, T.Y., Maire, M., Belongie, S., Hays, J., Perona, P., Ramanan, D., Dollár, P., Zitnick, C.L.: Microsoft coco: Common objects in context. In: Computer Vision–ECCV 2014: 13th European Conference, Zurich, Switzerland, September 6-12, 2014, Proceedings, Part V 13. pp. 740–755. Springer (2014)
  * [32] Lyu, M., Yang, Y., Hong, H., Chen, H., Jin, X., He, Y., Xue, H., Han, J., Ding, G.: One-dimensional adapter to rule them all: Concepts, diffusion models and erasing applications. ArXiv abs/2312.16145 (2023), https://api.semanticscholar.org/CorpusID:266551849
  * [33] Madry, A., Makelov, A., Schmidt, L., Tsipras, D., Vladu, A.: Towards deep learning models resistant to adversarial attacks. arXiv preprint arXiv:1706.06083 (2017)
  * [34] Marcelo, P.: Fact focus: Fake image of pentagon explosion briefly sends jitters through stock market. Associated Press (May 2023), https://apnews.com/article/pentagon-explosion-fake-image-stock-market-jitters-78d5913603bdf9f17cc7a8c77a72e59b
  * [35] Ni, M., Wu, C., Wang, X., Yin, S.S., Wang, L., Liu, Z., Duan, N.: Ores: Open-vocabulary responsible visual synthesis. ArXiv abs/2308.13785 (2023), https://api.semanticscholar.org/CorpusID:261243073
  * [36] Nichol, A.Q., Dhariwal, P., Ramesh, A., Shyam, P., Mishkin, P., Mcgrew, B., Sutskever, I., Chen, M.: Glide: Towards photorealistic image generation and editing with text-guided diffusion models. In: International Conference on Machine Learning. pp. 16784–16804. PMLR (2022)
  * [37] Nie, G., Kim, C., Yang, Y., Ren, Y.: Attributing image generative models using latent fingerprints. arXiv preprint arXiv:2304.09752 (2023)
  * [38] OpenAI: Chatgpt. Online (2022), https://chat.openai.com/chat, accessed on February 24, 2024
  * [39] Patel, M., Jung, S., Baral, C., Yang, Y.: $\lambda$-eclipse: Multi-concept personalized text-to-image diffusion models by leveraging clip latent space. ArXiv abs/2402.05195 (2024), https://api.semanticscholar.org/CorpusID:267547418
  * [40] Patel, M., Kim, C.S., Cheng, S., Baral, C., Yang, Y.: Eclipse: A resource-efficient text-to-image prior for image generations. ArXiv abs/2312.04655 (2023), https://api.semanticscholar.org/CorpusID:266149498
  * [41] Pham, M., Marshall, K.O., Cohen, N., Mittal, G., Hegde, C.: Circumventing concept erasure methods for text-to-image generative models. In: The Twelfth International Conference on Learning Representations (2024), https://openreview.net/forum?id=ag3o2T51Ht
  * [42] Pinto, L., Davidson, J., Sukthankar, R., Gupta, A.: Robust adversarial reinforcement learning. In: International Conference on Machine Learning. pp. 2817–2826. PMLR (2017)
  * [43] Radford, A., Kim, J.W., Hallacy, C., Ramesh, A., Goh, G., Agarwal, S., Sastry, G., Askell, A., Mishkin, P., Clark, J., et al.: Learning transferable visual models from natural language supervision. In: International conference on machine learning. pp. 8748–8763. PMLR (2021)
  * [44] Ramesh, A., Dhariwal, P., Nichol, A., Chu, C., Chen, M.: Hierarchical text-conditional image generation with clip latents. arXiv preprint arXiv:2204.06125 (2022)
  * [45] Rando, J., Paleka, D., Lindner, D., Heim, L., Tramèr, F.: Red-teaming the stable diffusion safety filter. arXiv preprint arXiv:2210.04610 (2022)
  * [46] Rombach, R., Blattmann, A., Lorenz, D., Esser, P., Ommer, B.: High-resolution image synthesis with latent diffusion models. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR). pp. 10684–10695 (June 2022)
  * [47] Ronneberger, O., Fischer, P., Brox, T.: U-net: Convolutional networks for biomedical image segmentation. In: Medical Image Computing and Computer-Assisted Intervention–MICCAI 2015: 18th International Conference, Munich, Germany, October 5-9, 2015, Proceedings, Part III 18. pp. 234–241. Springer (2015)
  * [48] Ruiz, N., Li, Y., Jampani, V., Pritch, Y., Rubinstein, M., Aberman, K.: Dreambooth: Fine tuning text-to-image diffusion models for subject-driven generation. 2023 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) pp. 22500–22510 (2022), https://api.semanticscholar.org/CorpusID:251800180
  * [49] Saharia, C., Chan, W., Saxena, S., Li, L., Whang, J., Denton, E.L., Ghasemipour, K., Gontijo Lopes, R., Karagol Ayan, B., Salimans, T., et al.: Photorealistic text-to-image diffusion models with deep language understanding. Advances in Neural Information Processing Systems 35, 36479–36494 (2022)
  * [50] Saleh, B., Elgammal, A.: Large-scale classification of fine-art paintings: Learning the right metric on the right feature. ArXiv abs/1505.00855 (2015), https://api.semanticscholar.org/CorpusID:14168099
  * [51] Schramowski, P., Brack, M., Deiseroth, B., Kersting, K.: Safe latent diffusion: Mitigating inappropriate degeneration in diffusion models. 2023 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) pp. 22522–22531 (2022), https://api.semanticscholar.org/CorpusID:253420366
  * [52] Schuhmann, C., Beaumont, R., Vencu, R., Gordon, C., Wightman, R., Cherti, M., Coombes, T., Katta, A., Mullis, C., Wortsman, M., et al.: Laion-5b: An open large-scale dataset for training next generation image-text models. arXiv preprint arXiv:2210.08402 (2022)
  * [53] Somepalli, G., Singla, V., Goldblum, M., Geiping, J., Goldstein, T.: Diffusion art or digital forgery? investigating data replication in diffusion models. 2023 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) pp. 6048–6058 (2022), https://api.semanticscholar.org/CorpusID:254366634
  * [54] Somepalli, G., Singla, V., Goldblum, M., Geiping, J., Goldstein, T.: Understanding and mitigating copying in diffusion models. ArXiv abs/2305.20086 (2023), https://api.semanticscholar.org/CorpusID:258987384
  * [55] Tsai, Y.L., Hsu, C.Y., Xie, C., Lin, C.H., Chen, J.Y., Li, B., Chen, P.Y., Yu, C.M., ying Huang, C.: Ring-a-bell! how reliable are concept removal methods for diffusion models? ArXiv abs/2310.10012 (2023), https://api.semanticscholar.org/CorpusID:264146485
  * [56] Tsipras, D., Santurkar, S., Engstrom, L., Turner, A., Madry, A.: Robustness may be at odds with accuracy. arXiv: Machine Learning (2018), https://api.semanticscholar.org/CorpusID:52962648
  * [57] Wang, Z., Guo, H., Zhang, Z., Liu, W., Qin, Z., Ren, K.: Feature importance-aware transferable adversarial attacks. 2021 IEEE/CVF International Conference on Computer Vision (ICCV) pp. 7619–7628 (2021), https://api.semanticscholar.org/CorpusID:236493523
  * [58] Wei, Y., Zhang, Y., Ji, Z., Bai, J., Zhang, L., Zuo, W.: Elite: Encoding visual concepts into textual embeddings for customized text-to-image generation. 2023 IEEE/CVF International Conference on Computer Vision (ICCV) pp. 15897–15907 (2023), https://api.semanticscholar.org/CorpusID:257219968
  * [59] Wen, Y., Jain, N., Kirchenbauer, J., Goldblum, M., Geiping, J., Goldstein, T.: Hard prompts made easy: Gradient-based discrete optimization for prompt tuning and discovery. Advances in Neural Information Processing Systems 36 (2024)
  * [60] Wen, Y., Kirchenbauer, J., Geiping, J., Goldstein, T.: Tree-ring watermarks: Fingerprints for diffusion images that are invisible and robust. arXiv preprint arXiv:2305.20030 (2023)
  * [61] Wu, B., Xu, C., Dai, X., Wan, A., Zhang, P., Tomizuka, M., Keutzer, K., Vajda, P.: Visual transformers: Token-based image representation and processing for computer vision. ArXiv abs/2006.03677 (2020), https://api.semanticscholar.org/CorpusID:219531480
  * [62] Wu, X., Guo, W., Wei, H., Xing, X.: Adversarial policy training against deep reinforcement learning. In: 30th USENIX Security Symposium (USENIX Security 21). pp. 1883–1900 (2021)
  * [63] Wu, Y., Zhang, J., Kerschbaum, F., Zhang, T.: Backdooring textual inversion for concept censorship. ArXiv abs/2308.10718 (2023), https://api.semanticscholar.org/CorpusID:261049298
  * [64] Yang, Y., Gao, R., Wang, X., Xu, N., Xu, Q.: Mma-diffusion: Multimodal attack on diffusion models. ArXiv abs/2311.17516 (2023), https://api.semanticscholar.org/CorpusID:265498727
  * [65] Yoo, J.Y., Qi, Y.: Towards improving adversarial training of nlp models. arXiv preprint arXiv:2109.00544 (2021)
  * [66] Yu, N., Skripniuk, V., Abdelnabi, S., Fritz, M.: Artificial fingerprinting for generative models: Rooting deepfake attribution in training data. In: Proceedings of the IEEE/CVF International conference on computer vision. pp. 14448–14457 (2021)
  * [67] Yu, N., Skripniuk, V., Chen, D., Davis, L., Fritz, M.: Responsible disclosure of generative models using scalable fingerprinting. arXiv preprint arXiv:2012.08726 (2020)
  * [68] Yuan, Z., Zhang, J., Jia, Y., Tan, C., Xue, T., Shan, S.: Meta gradient adversarial attack. 2021 IEEE/CVF International Conference on Computer Vision (ICCV) pp. 7728–7737 (2021), https://api.semanticscholar.org/CorpusID:236956844
  * [69] Zhang, E., Wang, K., Xu, X., Wang, Z., Shi, H.: Forget-me-not: Learning to forget in text-to-image diffusion models. ArXiv abs/2303.17591 (2023), https://api.semanticscholar.org/CorpusID:257833863
  * [70] Zhang, H., Yu, Y., Jiao, J., Xing, E., Ghaoui, L.E., Jordan, M.: Theoretically principled trade-off between robustness and accuracy. In: Chaudhuri, K., Salakhutdinov, R. (eds.) Proceedings of the 36th International Conference on Machine Learning. Proceedings of Machine Learning Research, vol. 97, pp. 7472–7482. PMLR (09–15 Jun 2019), https://proceedings.mlr.press/v97/zhang19p.html
  * [71] Zhang, K.A., Xu, L., Cuesta-Infante, A., Veeramachaneni, K.: Robust invisible video watermarking with attention (2019)
  * [72] Zhang, Y., Jia, J., Chen, X., Chen, A., Zhang, Y., Liu, J., Ding, K., Liu, S.: To generate or not? safety-driven unlearned diffusion models are still easy to generate unsafe images … for now. ArXiv abs/2310.11868 (2023), https://api.semanticscholar.org/CorpusID:264289091
  * [73] Zheng, Y., Yeh, R.A.: Imma: Immunizing text-to-image models against malicious adaptation. ArXiv abs/2311.18815 (2023), https://api.semanticscholar.org/CorpusID:265506125

## Supplementary Material

##  Appendix 0.A Additional Related Works

###  0.A.1 Adversarial Training Approaches

Adversarial attacks [11, 57, 68, 3] craft perturbed inputs, known as
adversarial examples, that can mislead models into making erroneous
predictions. Adversarial training has emerged as a robust countermeasure,
demonstrating that models can be fortified by incorporating these adversarial
examples into the training process [33, 70]. This method involves an iterative
cycle of generating adversarial samples and utilizing them to update the
model’s parameters, thereby instilling resilience against such attacks.
Notably, studies like [70, 56] have highlighted a trade-off between standard
accuracy and robustness stemming from adversarial training, adding an
intriguing dimension to the field.

The idea of adversarial training has spurred its adoption across various
sectors to enhance model robustness [42, 25, 62, 65]. The text-to-image (T2I)
domain, for instance, has leveraged adversarial attacks to highlight its
susceptibility to meticulously crafted inputs [4, 72, 55, 59, 64]. Despite the
prevalence of adversarial attack strategies, there exists a notable scarcity
of adversarial training methodologies tailored for T2I models. Our work aims
to bridge this gap by introducing a comprehensive adversarial training
framework tailored for T2I models, as detailed in the main paper.

###  0.A.2 Additional Works in Concept Erase

Initial investigations into concept erasure demonstrate the capability to
modify representations in T2I models [10, 27]. Despite their groundbreaking
contributions, these studies also unveil limitations such as performance drops
when simultaneously erasing multiple concepts or inadvertently affecting
nearby concepts [20, 32, 24, 18]. In response, Huang et al.[20] and Lyu et
al.[32] introduce streamlined adapter layers with a loss function inspired by
ESD’s [10]. Kim et al.[24] and Hong et al.[18] also craft approaches inspired
by ESD’s foundational principles. While Huang et al.[20] investigate
adversarial training customized for their adapter layer, their methodology is
confined to this specific context and does not exhibit the versatility
inherent in our proposed approach.

Aligned with these developments, RACE is based on ESD, suggesting its
compatibility with these recent advances. Our reliance on the innovative
concept of single-timestep adversarial attacks presents a versatile solution
adaptable to future T2I model enhancements, regardless of their direct
association with ESD. This positions RACE as a significant contribution to
reinforcing T2I models.

##  Appendix 0.B Additional Training Details

For our experiments, we utilize pretrained concept-erased weights derived from
ESD [10], adhering to their configuration where the default $\eta$ value in
$L_{erase}$ is set to $\eta=1$. Consequently, we maintain this setting by also
assigning $\eta=1$ within $L_{RACE}$ for consistency.

We train models using $L_{RACE}$, allocating 3,000 iterations for style and
explicit concepts, and 2,000 iterations for object categories. These iteration
counts are tailored to the specific nature of each target concept. Similar to
the ESD framework, RACE does not require supplementary prompts or images for
concept erasure. Consistent with ESD’s methodology, a brief textual
description of the target concept suffices for its removal from the Stable
Diffusion model [46], underscoring the efficiency and simplicity of our
approach.

(a) (b)

Figure 5:  Additional Results of (a) Van Gogh and (b) church for Single-
Timestep Adversarial Attack Efficacy. It is observed that the perturbed text
embedding $c+\delta_{t^{*}}$ can reproduce images containing the previously
erased concept even when the adversarial attack is applied at a singular
timestep $t^{*}$.

##  Appendix 0.C Additional Analysis of Single-Timestep Adversarial Attack

In order to demonstrate the effectiveness of the proposed single-timestep
adversarial attack, we perform additional experiments on $\Phi_{\theta}$
models, each trained to erase the “Van Gogh” and “church” concepts via the ESD
method [10]. The results, as illustrated in Fig. 5, demonstrate that a single-
timestep adversarial attack can successfully reconstruct previously erased
concepts.

##  Appendix 0.D Extended Visual Results from RACE

In Figures Figs. 6, 7 and 8, we present a supplementary collection of images
to further demonstrate the capability of our method in erasing targeted
concepts. It is important to note that the set of images in Fig. 7 contains
explicit content.

Figure 6: Comparison Between Original SD and RACE-“Van Gogh” Figure 7: Comparison Between Original SD and RACE: For representations involving explicit or sensitive content, manual modifications have been applied (e.g., addition of censoring boxes or application of significant blurring) to ensure appropriateness for submission. Figure 8: Comparison Between Original SD and RACE. Table 5: Attack Success Rate (ASR) when varying $\epsilon$ and the number of adversarial attack steps. For each entry, we report three ASR values when $\epsilon=0.05,0.1$, and $0.2$ using a tuple enclosed in parentheses. | # of adversarial attack steps
---|---
|  10 |  20
RACE-VanGogh |  (0.88, 0.88, 0.88) |  (0.90, 0.92, 0.94)
RACE-Nudity |  (0.94, 0.96, 0.99) |  (0.94, 0.99, 0.97)
RACE-Violence |  (0.96, 0.98, 1.00) |  (0.95, 0.99, 0.99)
RACE-Illegal |  (0.92, 0.97, 1.00) |  (0.95, 0.97, 0.97)
RACE-Church |  (0.70, 0.86, 0.90) |  (0.74, 0.82, 0.96)
RACE-GolfBall |  (0.74, 0.82, 0.96) |  (0.32, 0.56, 0.74)
RACE-Parachute |  (0.60, 0.68, 0.88) |  (0.56, 0.76, 0.90)

##  Appendix 0.E Determining PGD Hyperparameters

Within the main paper, we settle on $\epsilon=0.1$ and designate 10 steps for
the adversarial attack. This section delves into the empirical analysis
underpinning this selection, particularly focusing on the balance between ASR
efficacy and hyperparameter tuning. As delineated in Tab. 5, an $\epsilon$
value of 0.2 marginally outperforms 0.1. However, as elaborated in the main
manuscript, an elevated attack intensity with $\epsilon=0.2$ may compromise
the equilibrium between adversarial robustness and the fidelity of generated
images. Holding $\epsilon$ at 0.1, we observe a minimal variance between 10
and 20 attack steps, reinforcing our decision to configure PGD parameters at
$\epsilon=0.1$ and 10 steps for an optimal trade-off.

Table 6: Comparison of ASR decrease and CLIP-Score for different keywords

Keyword Nudity Violence Illegal Act ASR Decrease -30% -11% -5% CLIP-Score [14]
0.658 0.533 0.526

##  Appendix 0.F Challenges in Erasing Violence and Illegal Act

The data presented in Table 2 of the main manuscript indicate that “violence”
and “illegal act” concepts exhibit less pronounced reductions in Attack
Success Rate (ASR) against UnlearnDiff white box attack [72]. Specifically,
“nudity” sees a 30% decrease in ASR, while “violence” and “illegal act”
experience reductions of 11% and 5%, respectively. To align with the
experimental framework of UnlearnDiff [72], we employed “violence” and
“illegal act” as keywords for concept removal. It is postulated that these
terms may not optimally represent the range of hazardous imagery producible by
Stable Diffusion (SD) [46] through the prompts in the I2P dataset [51]. To
examine this hypothesis, we calculate the CLIP-score [14] for explicit images
generated by a baseline SD model, $SD(\tilde{y})$, where $\tilde{y}$ refers
prompts from I2P and $c$ includes “nudity”, “violence”, and “illegal act”
(i.e. CLIP-score($SD(\tilde{y})$,$c$)). As illustrated in LABEL:tab:keywㅌord-
clipscore, a higher CLIP-score, indicating better keyword alignment with the
dangerous images, correlates with a more substantial decrease in ASR.
Conversely, keywords that poorly match the hazardous image content tend to
result in less significant ASR reductions. This underscores the critical
importance of selecting highly representative keywords for concept erasure in
T2I models, as their alignment with the generated content significantly
influences the effectiveness of concept erase.


*(0,0)[viewport=0 0 700 600]
*(0,602)[viewport=0 600 700 1600]

Comprehensive Summary

Creating Training Dataset

Semantic Representation

Convolutional Neural based Inference Learning

Self-Attention based Inference Learning

Missing 'biblatex' package
The bibliography requires the 'biblatex' package.

booktitleThe semantic web
titleDbpedia: A nucleus for a web of open data

journaltitlearXiv preprint arXiv:1803.01271
titleAn empirical evaluation of generic convolutional and recurrent networks for sequence modeling

Association for Computational Linguistics

booktitleProceedings of the ACL 2003 workshop on Multiword expressions: analysis, acquisition and treatment-Volume 18
titleAn empirical model of multiword expression decomposability

booktitleProceedings of the 24th ACM international conference on Multimedia
titleBidirectional long-short term memory for video description

booktitleProceedings of the 2008 ACM SIGMOD international conference on Management of data
titleFreebase: a collaboratively created graph database for structuring human knowledge

Association for Computational Linguistics

booktitleProceedings of the conference on human language technology and empirical methods in natural language processing
titleA shortest path dependency kernel for relation extraction

Association for Computational Linguistics

booktitleProceedings of the 23rd International Conference on Computational Linguistics
titleExploiting background knowledge for relation extraction

journaltitlearXiv preprint arXiv:1906.04341
titleWhat Does BERT Look At? An Analysis of BERT's Attention

Martin Hagan

titleNeural network design

journaltitlearXiv preprint arXiv:1810.04805
titleBert: Pre-training of deep bidirectional transformers for language understanding

booktitleProceedings of the 38th international ACM SIGIR conference on research and development in information retrieval
titleWord embedding based generalized language model for information retrieval

booktitle2017 IEEE International Conference on Big Data (Big Data)
titleImproving text classification with word embedding

journaltitlearXiv preprint arXiv:1603.06067
titleAdaptive joint learning of compositional and non-compositional phrase embeddings

MIT Press

journaltitleNeural computation
titleLong short-term memory

journaltitlearXiv preprint arXiv:1809.08037
titleUnderstanding convolutional neural networks for text classification

booktitleHuman Language Technologies 2007: The Conference of the North American Chapter of the Association for Computational Linguistics; Proceedings of the Main Conference
titleA systematic exploration of the feature space for relation extraction

journaltitlearXiv preprint arXiv:1412.1058
titleEffective use of word order for text categorization with convolutional neural networks

booktitleProceedings of the 2006 Conference on Empirical Methods in Natural Language Processing
titleProceedings of the 2006 Conference on Empirical Methods in Natural Language Processing

booktitleProceedings of the ACL Interactive Poster and Demonstration Sessions
titleCombining lexical, syntactic, and semantic features with maximum entropy models for information extraction

Max-Planck-Institut für Informatik

titleYago-a core of semantic knowledge

Association for Computational Linguistics

booktitleProceedings of the Workshop on Multiword Expressions: Identifying and Exploiting Underlying Properties
titleAutomatic identification of non-compositional multi-word expressions using latent semantic analysis

Seattle, Washington, USA

Association for Computational Linguistics

booktitleProceedings of the 2013 Conference on Empirical Methods in Natural Language Processing
titleDetecting Compositionality of Multi-Word Expressions using Nearest Neighbours in Vector Space Models

booktitleProceedings of the 2014 Conference on Empirical Methods in Natural Language Processing, EMNLP 2014, October 25-29, 2014, Doha, Qatar, A meeting of SIGDAT, a Special Interest Group of the ACL
titleConvolutional Neural Networks for Sentence Classification

booktitleAdvances in neural information processing systems
titleImagenet classification with deep convolutional neural networks

Amsterdam, The Netherlands, The Netherlands

Elsevier Science Publishers B. V.

journaltitleCogn. Syst. Res.
titleA Study of Metrics of Distance and Correlation Between Ranked Lists for Compositionality Detection
Compositionality detection,Metrics of distance and correlation

booktitleProceedings of the 38th International ACM SIGIR Conference on Research and Development in Information Retrieval
titleNon-compositional term dependence for information retrieval

booktitleInternational Conference on Advanced Data Mining and Applications
titleConvolution neural network for relation extraction

Association for Computational Linguistics

booktitleHuman Language Technologies: The 2010 Annual Conference of the North American Chapter of the Association for Computational Linguistics
titleAutomatic domain adaptation for parsing

booktitleAdvances in Neural Information Processing Systems
titleAre Sixteen Heads Really Better than One?

booktitleAdvances in neural information processing systems
titleDistributed representations of words and phrases and their compositionality

booktitleAdvances in neural information processing systems
titleSubsequence kernels for relation extraction

booktitleProceedings of the 52nd Annual Meeting of the Association for Computational Linguistics (Volume 2: Short Papers)
titleEmploying word representations and regularization for domain adaptation of relation extraction

booktitleIJCAI-99 workshop on machine learning for information filtering
titleUsing maximum entropy for text classification

Association for Computational Linguistics

booktitleProceedings of the ACL-02 Workshop on Automatic Summarization-Volume 4
titleUsing maximum entropy for sentence extraction

journaltitlearXiv preprint arXiv:1601.01356
titleFrom word embeddings to item recommendation

booktitleProceedings of the 40th annual meeting of the Association for Computational Linguistics
titleBLEU: a method for automatic evaluation of machine translation

booktitleProceedings of the 2014 conference on empirical methods in natural language processing (EMNLP)
titleGlove: Global vectors for word representation

Association for Computational Linguistics

booktitleProceedings of the 22nd International Conference on Computational Linguistics-Volume 1
titleExploiting constituent dependencies for tree kernel-based semantic relation extraction

booktitleProceedings of the 2015 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies
titleA word embedding approach to predicting the compositionality of multiword expressions

MIT press

titleLearning with kernels: support vector machines, regularization, optimization, and beyond

journaltitleIEEE transactions on Signal Processing
titleBidirectional recurrent neural networks

booktitle2019 IEEE International Conference on Big Data (Big Data)
titleThe performance of LSTM and BiLSTM in forecasting time series

booktitleThirtieth AAAI Conference on Artificial Intelligence
titleReturn of frustratingly easy domain adaptation

journaltitleNeural processing letters
titleLeast squares support vector machine classifiers

booktitleAdvances in neural information processing systems
titleAttention is all you need

Association for Computational Linguistics

booktitleProceedings of the conference on Human Language Technology and Empirical Methods in Natural Language Processing
titleMeasuring the relative compositionality of verb-noun (VN) collocations by integrating features

Association for Computational Linguistics

booktitleProceedings of the 57th Conference of the Association for Computational Linguistics, ACL 2019, Florence, Italy, July 28- August 2, 2019, Volume 1: Long Papers
titleAnalyzing Multi-Head Self-Attention: Specialized Heads Do the Heavy Lifting, the Rest Can Be Pruned

booktitleProceedings of the 2015 Conference on Empirical Methods in Natural Language Processing
titleLearning semantic composition to detect non-compositionality of multiword expressions

Association for Computational Linguistics

booktitleProceedings of the 21st International Conference on Computational Linguistics and the 44th annual meeting of the Association for Computational Linguistics
titleA composite kernel to extract relations between entities with both flat and structured features


* Cowley and Cowley (1965) E. Cowley and R. A. Cowley, Proceedings of the Royal Society of London. Series A. Mathematical and Physical Sciences 287, 259 (1965).
  * Cowley (1968b) R. A. Cowley, Reports on Progress in Physics 31, 123 (1968b).
  * Blumberg  _et al._ (2007) G. Blumberg, A. Mialitsin, B. Dennis, N. Zhigadlo, and J. Karpinski, Physica C: Superconductivity 456, 75 (2007).
  * Goncharov  _et al._ (2001) A. F. Goncharov, V. V. Struzhkin, E. Gregoryanz, J. Hu, R. J. Hemley, H.-k. Mao, G. Lapertot, S. Budko, and P. Canfield, Physical Review B 64, 100509 (2001).
  * Mialitsin  _et al._ (2007) A. Mialitsin, B. Dennis, N. Zhigadlo, J. Karpinski, and G. Blumberg, Physical Review B 75, 020509 (2007).
  * Renker  _et al._ (2003) B. Renker, H. Schober, P. Adelmann, P. Bohnen, D. Ernst, R. Heid, P. Schweiss, and T. Wolf, Journal of low temperature physics 131, 411 (2003).
  * Pickard  _et al._ (2020) C. J. Pickard, I. Errea, and M. I. Eremets, Annual Review of Condensed Matter Physics 11, 57 (2020).
  * Drozdov  _et al._ (2015) A. P. Drozdov, M. I. Eremets, I. A. Troyan, V. Ksenofontov, and S. I. Shylin, Nature 525, 73 (2015).
  * Snider  _et al._ (2021) E. Snider, N. Dasenbrock-Gammon, R. McBride, X. Wang, N. Meyers, K. V. Lawler, E. Zurek, A. Salamat, and R. P. Dias, Physical Review Letters 126, 117003 (2021).
  * Yesudhas  _et al._ (2021) S. Yesudhas, N. Yedukondalu, M. K. Jana, J. Zhang, J. Huang, B. Chen, H. Deng, R. Sereika, H. Xiao, S. Sinogeikin, C. Kenney-Benson, K. Biswas, J. B. Parise, Y. Ding, and H.-k. Mao, Inorganic Chemistry 60, 9320 (2021), pMID: 34152127\.
  * Ignatov  _et al._ (2012) A. Ignatov, A. Kumar, P. Lubik, R. Yuan, W. Guo, N. Wang, K. Rabe, and G. Blumberg, Physical Review B 86, 134107 (2012).
  * Lazarević  _et al._ (2011) N. Lazarević, H. Lei, C. Petrovic, Z. Popović, _et al._ , Physical Review B 84, 214305 (2011).
  * Shukla  _et al._ (2003) A. Shukla, M. Calandra, M. d’Astuto, M. Lazzeri, F. Mauri, C. Bellin, M. Krisch, J. Karpinski, S. Kazakov, J. Jun, _et al._ , Physical review letters 90, 095506 (2003).
  * Burkel (2000) E. Burkel, Reports on Progress in Physics 63, 171 (2000).
  * Krisch and Sette (2006) M. Krisch and F. Sette, in  _Light Scattering in Solid IX_ (Springer, 2006) pp. 317–370.
  * Baron (2009) A. Q. Baron, arXiv preprint arXiv:0910.5764 (2009).
  * Baron  _et al._ (2004) A. Baron, H. Uchiyama, Y. Tanaka, S. Tsutsui, D. Ishikawa, S. Lee, R. Heid, K.-P. Bohnen, S. Tajima, and T. Ishikawa, Physical review letters 92, 197004 (2004).
  * d’Astuto  _et al._ (2007) M. d’Astuto, M. Calandra, S. Reich, A. Shukla, M. Lazzeri, F. Mauri, J. Karpinski, N. Zhigadlo, A. Bossak, and M. Krisch, Physical Review B 75, 174508 (2007).
  * Novko (2018) D. Novko, Phys. Rev. B 98, 041112 (2018).
  * Novko  _et al._ (2020) D. Novko, F. Caruso, C. Draxl, and E. Cappelluti, Phys. Rev. Lett. 124, 077001 (2020).
  * Uchiyama  _et al._ (2004) H. Uchiyama, A. Baron, S. Tsutsui, Y. Tanaka, W.-Z. Hu, A. Yamamoto, S. Tajima, and Y. Endoh, Physical review letters 92, 197005 (2004).
  * Fukuda  _et al._ (2005) T. Fukuda, J. Mizuki, K. Ikeuchi, K. Yamada, A. Baron, and S. Tsutsui, Physical Review B 71, 060501 (2005).
  * Graf  _et al._ (2008) J. Graf, M. d’Astuto, C. Jozwiak, D. Garcia, N. L. Saini, M. Krisch, K. Ikeuchi, A. Baron, H. Eisaki, and A. Lanzara, Physical Review Letters 100, 227002 (2008).
  * Le Tacon  _et al._ (2014) M. Le Tacon, A. Bosak, S. Souliou, G. Dellea, T. Loew, R. Heid, K. Bohnen, G. Ghiringhelli, M. Krisch, and B. Keimer, Nature Physics 10, 52 (2014).
  * Fukuda  _et al._ (2008) T. Fukuda, A. Q. Baron, S.-i. Shamoto, M. Ishikado, H. Nakamura, M. Machida, H. Uchiyama, S. Tsutsui, A. Iyo, H. Kito, _et al._ , Journal of the Physical Society of Japan 77, 103715 (2008).
  * Kuroiwa  _et al._ (2008) S. Kuroiwa, A. Baron, T. Muranaka, R. Heid, K.-P. Bohnen, and J. Akimitsu, Physical Review B 77, 140503 (2008).
  * Li  _et al._ (2021) H. Li, T. Zhang, T. Yilmaz, Y. Pai, C. Marvinney, A. Said, Q. Yin, C. Gong, Z. Tu, E. Vescovo, _et al._ , Physical Review X 11, 031050 (2021).
  * Vig  _et al._ (2017) S. Vig, A. Kogar, M. Mitrano, A. Husain, L. Venema, M. Rak, V. Mishra, P. Johnson, G. Gu, E. Fradkin, _et al._ , SciPost Physics 3, 026 (2017).
  * Ibach and Mills (2013) H. Ibach and D. L. Mills,  _Electron energy loss spectroscopy and surface vibrations_ (Academic press, 2013).
  * Phelps  _et al._ (1994) R. Phelps, P. Akavoor, L. Kesmodel, A. Barr, J. Markert, J. Ma, R. Kelley, and M. Onellion, Physical Review B 50, 6526 (1994).
  * Mitrano  _et al._ (2018) M. Mitrano, A. Husain, S. Vig, A. Kogar, M. Rak, S. Rubeck, J. Schmalian, B. Uchoa, J. Schneeloch, R. Zhong, _et al._ , Proceedings of the National Academy of Sciences 115, 5392 (2018).
  * Huang  _et al._ (2021) E. W. Huang, K. Limtragool, C. Setty, A. A. Husain, M. Mitrano, P. Abbamonte, and P. W. Phillips, Physical Review B 103, 035121 (2021).
  * Setty  _et al._ (2018) C. Setty, B. Padhi, K. Limtragool, P. Abbamonte, A. A. Husain, M. Mitrano, and P. W. Phillips, arXiv preprint arXiv:1803.05439 (2018).
  * Sahadev  _et al._ (2012) N. Sahadev, D. N. Biswas, K. Srinivas, P. Manfrinetti, A. Palenzona, and K. Maiti, in  _AIP Conference Proceedings_ , Vol. 1447 (American Institute of Physics, 2012) pp. 841–842.
  * Wang  _et al._ (2017) Z. Wang, D. Walkup, P. Derry, T. Scaffidi, M. Rak, S. Vig, A. Kogar, I. Zeljkovic, A. Husain, L. H. Santos, _et al._ , Nature Physics 13, 799 (2017).
  * Zhang  _et al._ (2016) S. Zhang, J. Guan, X. Jia, B. Liu, W. Wang, F. Li, L. Wang, X. Ma, Q. Xue, J. Zhang, _et al._ , Physical Review B 94, 081116 (2016).
  * Gilvarry (1957) J. Gilvarry, Journal of applied physics 28, 1253 (1957).
  * Gilvarry (1956a) J. J. Gilvarry, Phys. Rev. 102, 331 (1956a).
  * Gilvarry (1956b) J. J. Gilvarry, Physical Review 102, 325 (1956b).
  * Grüneisen (1912) E. Grüneisen, Annalen der Physik 344, 257 (1912).
  * Cuffari and Bongiorno (2020b) D. Cuffari and A. Bongiorno, Phys. Rev. Lett. 124, 215501 (2020b).
  * Wang  _et al._ (2003) R. J. Wang, W. H. Wang, F. Y. Li, L. M. Wang, Y. Zhang, P. Wen, and J. F. Wang, Journal of Physics: Condensed Matter 15, 603 (2003).
  * Douglas and Xu (2021) J. F. Douglas and W.-S. Xu, Macromolecules 54, 3247 (2021).
  * Gilvarry (1955) J. J. Gilvarry, The Journal of Chemical Physics 23, 1925 (1955).
  * Macdonald (1966) J. R. Macdonald, Rev. Mod. Phys. 38, 669 (1966).
  * Anderson (1966) O. L. Anderson, Journal of Physics and Chemistry of Solids 27, 547 (1966).
  * Barker Jr (1967) R. Barker Jr, Journal of Applied Physics 38, 4234 (1967).
  * Sharma (1983) B. Sharma, Polymer 24, 314 (1983).
  * Anderson (2000) O. L. Anderson, Geophysical Journal International 143, 279 (2000).
  * de la Roza  _et al._ (2016) A. O. de la Roza, V. Luaña, and M. Flórez, in  _An Introduction to High-Pressure Science and Technology_ (CRC Press, 2016) pp. 25–72.
  * Yang  _et al._ (2022b) Z.-Y. Yang, Y.-J. Wang, and A. Zaccone, Phys. Rev. B 105, 014204 (2022b).
  * Waller and Froman (1952) I. Waller and P. Froman, Arkiv Fysik 4 (1952).
  * Elliott and Thorpe (1967) R. Elliott and M. Thorpe, Proceedings of the Physical Society (1958-1967) 91, 903 (1967).
  * Hudson (2006) B. S. Hudson, Vibrational spectroscopy 42, 25 (2006).
  * Axe and Shirane (1973) J. Axe and G. Shirane, Physical Review B 8, 1965 (1973).
  * Cowley and Woods (1971) R. Cowley and A. Woods, Canadian Journal of Physics 49, 177 (1971).
  * Reichardt  _et al._ (1994) W. Reichardt, L. Pintschovius, N. Pyka, P. Schweiss, A. Erb, P. Bourges, G. Collin, J. Rossat-Mignod, I. Henry, A. Ivanov, _et al._ , Journal of Superconductivity 7, 399 (1994).
  * Chou  _et al._ (1990) H. Chou, K. Yamada, J. Axe, S. Shapiro, G. Shirane, I. Tanaka, K. Yamane, and H. Kojima, Physical Review B 42, 4272 (1990).
  * Mittal  _et al._ (2009) R. Mittal, L. Pintschovius, D. Lamago, R. Heid, K. Bohnen, D. Reznik, S. Chaplot, Y. Su, N. Kumar, S. Dhar, _et al._ , Physical review letters 102, 217001 (2009).
  * Yamaura  _et al._ (2019) J.-i. Yamaura, H. Hiraka, S. Iimura, Y. Muraba, J. Bang, K. Ikeuchi, M. Nakamura, Y. Inamura, T. Honda, M. Hiraishi, _et al._ , Physical Review B 99, 220505 (2019).
  * Pintschovius  _et al._ (2008) L. Pintschovius, F. Weber, W. Reichardt, A. Kreyssig, R. Heid, D. Reznik, O. Stockert, and K. Hradil, Pramana 71, 687 (2008).
  * Weber  _et al._ (2014) F. Weber, L. Pintschovius, W. Reichardt, R. Heid, K.-P. Bohnen, A. Kreyssig, D. Reznik, and K. Hradil, Physical Review B 89, 104503 (2014).
  * Yildirim  _et al._ (2001) T. Yildirim, O. Gülseren, J. Lynn, C. Brown, T. Udovic, Q. Huang, N. Rogado, K. Regan, M. Hayward, J. Slusky, _et al._ , Physical review letters 87, 037001 (2001).
  * Muranaka  _et al._ (2002) T. Muranaka, T. Yokoo, M. Arai, E. Margiolaki, K. Brigatti, K. Prassides, O. Petrenko, and J. Akimitsu, journal of the physical society of japan 71, 338 (2002).
  * Cabrera  _et al._ (1969) N. Cabrera, V. Celli, and R. Manson, Physical Review Letters 22, 346 (1969).
  * Manson and Celli (1971) R. Manson and V. Celli, Surface Science 24, 495 (1971).
  * Benedek and Toennies (1994) G. Benedek and J. P. Toennies, Surface science 299, 587 (1994).
  * Sklyadneva  _et al._ (2011) I. Y. Sklyadneva, G. Benedek, E. V. Chulkov, P. M. Echenique, R. Heid, K.-P. Bohnen, and J. Toennies, Physical review letters 107, 095502 (2011).
  * Benedek  _et al._ (2014) G. Benedek, M. Bernasconi, K.-P. Bohnen, D. Campi, E. V. Chulkov, P. M. Echenique, R. Heid, I. Y. Sklyadneva, and J. P. Toennies, Physical Chemistry Chemical Physics 16, 7159 (2014).
  * Benedek  _et al._ (2020) G. Benedek, J. R. Manson, S. Miret-Artés, A. Ruckhofer, W. E. Ernst, A. Tamtögl, and J. P. Toennies, Condensed Matter 5, 79 (2020).
  * Gloria  _et al._ (2021) A. Gloria, C. A. Pablo, G. Manuela, C. Fabian, K. Chia-Nung, L. C. Shan, A. Politano, B. Giorgio, D. Farías, R. Miranda, _et al._ , NPJ 2D Materials and Applications 5 (2021).
  * Gester  _et al._ (1994) M. Gester, D. Kleinhesselink, P. Ruggerone, and J. Toennies, Physical Review B 49, 5777 (1994).
  * Zhu  _et al._ (2011) X. Zhu, L. Santos, R. Sankar, S. Chikara, C. Howard, F. Chou, C. Chamon, and M. El-Batanouny, Physical Review Letters 107, 186102 (2011).
  * Zhu  _et al._ (2012) X. Zhu, L. Santos, C. Howard, R. Sankar, F. Chou, C. Chamon, and M. El-Batanouny, Physical Review Letters 108, 185501 (2012).
  * Stenull  _et al._ (2016) O. Stenull, C. Kane, and T. Lubensky, Physical review letters 117, 068001 (2016).
  * Liu  _et al._ (2020) Y. Liu, X. Chen, and Y. Xu, Advanced Functional Materials 30, 1904784 (2020).
  * Di Miceli  _et al._ (2022) D. Di Miceli, C. Setty, and A. Zaccone, arXiv preprint arXiv:2203.03499 (2022).
  * Naidyuk and Yanson (2005) Y. G. Naidyuk and I. K. Yanson,  _Point-contact spectroscopy_ , Vol. 145 (Springer Science & Business Media, 2005).
  * Jansen  _et al._ (1980) A. G. M. Jansen, A. Van Gelder, and P. Wyder, Journal of Physics C: Solid State Physics 13, 6073 (1980).
  * Naidyuk and Yanson (2003) Y. G. Naidyuk and I. K. Yanson, arXiv e-prints , physics/0312016 (2003), arXiv:physics/0312016 [physics.pop-ph] .
  * Lee  _et al._ (2015) W.-C. Lee, W. K. Park, H. Z. Arham, L. H. Greene, and P. Phillips, Proceedings of the National Academy of Sciences 112, 651 (2015).
  * Yanson (1983) I. Yanson, Soviet Journal of Low Temperature Physics 9, 343 (1983).
  * Kulik (1992) I. Kulik, Sov. J. Low Temp 18, 450 (1992).
  * Naidyuk  _et al._ (2003) Y. G. Naidyuk, I. Yanson, O. Kvitnitskaya, S. Lee, and S. Tajima, Physical review letters 90, 197001 (2003).
  * Yanson and Naidyuk (2004) I. Yanson and Y. G. Naidyuk, Low Temperature Physics 30, 261 (2004).
  * Samuely  _et al._ (2003) P. Samuely, P. Szabo, J. Kacmarcik, T. Klein, and A. Jansen, Physica C: Superconductivity 385, 244 (2003).
  * Dahm  _et al._ (2009) T. Dahm, V. Hinkov, S. V. Borisenko, A. A. Kordyuk, V. B. Zabolotnyy, J. Fink, B. Büchner, D. J. Scalapino, W. Hanke, and B. Keimer, Nature Physics 5, 217 (2009).
  * Bloembergen  _et al._ (1948) N. Bloembergen, E. M. Purcell, and R. V. Pound, Physical review 73, 679 (1948).
  * Dioguardi  _et al._ (2013) A. Dioguardi, J. Crocker, A. Shockley, C. Lin, K. Shirer, D. Nisson, M. Lawson, P. Canfield, S. Bud?ko, S. Ran, _et al._ , Physical review letters 111, 207201 (2013).
  * Imai and Hirota (2018) T. Imai and K. Hirota, Journal of the Physical Society of Japan 87, 025004 (2018).
  * Hunt  _et al._ (2001) A. Hunt, P. Singer, A. Cederström, and T. Imai, Physical Review B 64, 134525 (2001).
  * Baek  _et al._ (2012) S.-H. Baek, T. Loew, V. Hinkov, C. Lin, B. Keimer, B. Büchner, and H.-J. Grafe, Physical Review B 86, 220504 (2012).
  * Wu  _et al._ (2013) T. Wu, H. Mayaffre, S. Krämer, M. Horvatić, C. Berthier, C. Lin, D. Haug, T. Loew, V. Hinkov, B. Keimer, _et al._ , Physical Review B 88, 014511 (2013).
  * Julien  _et al._ (1999) M.-H. Julien, F. Borsa, P. Carretta, M. Horvatić, C. Berthier, and C. Lin, Physical review letters 83, 604 (1999).
  * Mitrović  _et al._ (2008) V. Mitrović, M.-H. Julien, C. De Vaulx, M. Horvatić, C. Berthier, T. Suzuki, and K. Yamada, Physical Review B 78, 014504 (2008).
  * Sternlieb  _et al._ (1990) B. Sternlieb, G. Luke, Y. Uemura, T. Riseman, J. Brewer, P. Gehring, K. Yamada, Y. Hidaka, T. Murakami, T. Thurston, _et al._ , Physical Review B 41, 8866 (1990).
  * Rømer  _et al._ (2013) A. T. Rømer, J. Chang, N. B. Christensen, B. Andersen, K. Lefmann, L. Mähler, J. Gavilano, R. Gilardi, C. Niedermayer, H. M. Rønnow, _et al._ , Physical Review B 87, 144513 (2013).
  * Ryu  _et al._ (2015a) H. Ryu, K. Wang, M. Opacic, N. Lazarevic, J. Warren, Z. Popovic, E. S. Bozin, C. Petrovic, _et al._ , Physical Review B 92, 174522 (2015a).
  * Ryu  _et al._ (2015b) H. Ryu, M. Abeykoon, K. Wang, H. Lei, N. Lazarevic, J. Warren, E. Bozin, Z. Popovic, C. Petrovic, _et al._ , Physical Review B 91, 184503 (2015b).
  * Chou  _et al._ (1995) F. Chou, N. Belk, M. Kastner, R. Birgeneau, and A. Aharony, Physical review letters 75, 2204 (1995).
  * Wakimoto  _et al._ (2000) S. Wakimoto, S. Ueki, Y. Endoh, and K. Yamada, Physical Review B 62, 3547 (2000).
  * Grinenko  _et al._ (2012) V. Grinenko, M. Abdel-Hafiez, S. Aswartham, A. Wolter-Giraud, C. Hess, M. Kumar, S. Wurmehl, K. Nenkov, G. Fuchs, B. Holzapfel, _et al._ , arXiv preprint arXiv:1203.1585 (2012).
  * Yadav and Paulose (2010) C. Yadav and P. Paulose, Journal of Applied Physics 107, 083908 (2010).
  * Mydosh (2015) J. Mydosh, Reports on Progress in Physics 78, 052501 (2015).
  * Maradudin and Fein (1962) A. Maradudin and A. Fein, Physical Review 128, 2589 (1962).
  * Cowley (1965) R. Cowley, Journal de Physique 26, 659 (1965).
  * Cowley (1968c) R. A. Cowley, Reports on Progress in Physics 31, 123 (1968c).
  * Menéndez and Cardona (1984b) J. Menéndez and M. Cardona, Physical Review B 29, 2051 (1984b).
  * Singh and Singh (2023) A. Singh and N. P. Singh, Materials Today: Proceedings 79, 39 (2023).
  * Bianco and Errea (2023) R. Bianco and I. Errea, arXiv preprint arXiv:2303.02621 (2023).
  * Gonze and Vigneron (1989) X. Gonze and J.-P. Vigneron, Physical Review B 39, 13120 (1989).
  * Debernardi and Baroni (1994) A. Debernardi and S. Baroni, Solid state communications 91, 813 (1994).
  * Debernardi  _et al._ (1995) A. Debernardi, S. Baroni, and E. Molinari, Physical review letters 75, 1819 (1995).
  * Debernardi (1998) A. Debernardi, Physical Review B 57, 12847 (1998).
  * Debernardi  _et al._ (1999) A. Debernardi, C. Ulrich, K. Syassen, and M. Cardona, Physical Review B 59, 6774 (1999).
  * Tang and Fultz (2011) X. Tang and B. Fultz, Physical Review B 84, 054303 (2011).
  * Li  _et al._ (2013) W. Li, J. Carrete, and N. Mingo, Applied Physics Letters 103, 253103 (2013).
  * Girotto and Novko (2023) N. Girotto and D. Novko, Phys. Rev. B 107, 064310 (2023).
  * Souvatzis  _et al._ (2008) P. Souvatzis, O. Eriksson, M. I. Katsnelson, and S. P. Rudin, Phys. Rev. Lett. 100, 095901 (2008).
  * Car and Parrinello (1985) R. Car and M. Parrinello, Physical review letters 55, 2471 (1985).
  * Car and Parrinello (1988) R. Car and M. Parrinello, Physical review letters 60, 204 (1988).
  * Wang  _et al._ (1989a) C. Wang, C. T. Chan, and K. Ho, Physical Review B 39, 8586 (1989a).
  * Wang  _et al._ (1989b) C. Wang, C. T. Chan, and K. Ho, Physical Review B 40, 3390 (1989b).
  * Wang  _et al._ (1990) C. Wang, C. T. Chan, and K. Ho, Physical Review B 42, 11276 (1990).
  * Hellman  _et al._ (2011) O. Hellman, I. Abrikosov, and S. Simak, Physical Review B 84, 180301 (2011).
  * Paulatto  _et al._ (2015) L. Paulatto, I. Errea, M. Calandra, and F. Mauri, Physical Review B 91, 054304 (2015).
  * Errea  _et al._ (2015a) I. Errea, M. Calandra, C. J. Pickard, J. Nelson, R. J. Needs, Y. Li, H. Liu, Y. Zhang, Y. Ma, and F. Mauri, Physical Review Letters 114, 157004 (2015a).
  * Born and Hooton (1955) M. Born and D. J. Hooton, Zeitschrift für Physik 142, 201 (1955).
  * Hooton (1955) D. Hooton, The London, Edinburgh, and Dublin Philosophical Magazine and Journal of Science 46, 422 (1955).
  * Koehler (1966) T. R. Koehler, Physical Review Letters 17, 89 (1966).
  * Siciliano  _et al._ (2023) A. Siciliano, L. Monacelli, G. Caldarelli, and F. Mauri, Physical Review B 107, 174307 (2023).
  * Monacelli and Mauri (2021) L. Monacelli and F. Mauri, Physical Review B 103, 104305 (2021).
  * Errea  _et al._ (2013) I. Errea, M. Calandra, and F. Mauri, Phys. Rev. Lett. 111, 177002 (2013).
  * Errea  _et al._ (2014) I. Errea, M. Calandra, and F. Mauri, Phys. Rev. B 89, 064302 (2014).
  * Errea  _et al._ (2016) I. Errea, M. Calandra, C. J. Pickard, J. R. Nelson, R. J. Needs, Y. Li, H. Liu, Y. Zhang, Y. Ma, and F. Mauri, Nature 532, 81 (2016).
  * Tadano and Tsuneyuki (2018a) T. Tadano and S. Tsuneyuki, Journal of the Physical Society of Japan 87, 041015 (2018a).
  * Tadano and Tsuneyuki (2018b) T. Tadano and S. Tsuneyuki, Journal of the Physical Society of Japan 87, 041015 (2018b).
  * Tadano and Tsuneyuki (2015) T. Tadano and S. Tsuneyuki, Phys. Rev. B 92, 054301 (2015).
  * Klein and Horton (1972) M. L. Klein and G. K. Horton, Journal of Low Temperature Physics 9, 151 (1972).
  * Marsiglio and Carbotte (2008a) F. Marsiglio and J. Carbotte, in  _Superconductivity_ (Springer, 2008) pp. 73–162.
  * Caldeira and Leggett (1983) A. Caldeira and A. J. Leggett, Annals of physics 149, 374 (1983).
  * Weiss (2012) U. Weiss,  _Quantum Dissipative Systems_ , 4th ed. (WORLD SCIENTIFIC, 2012).
  * Kleinert (2018) H. Kleinert,  _Collective classical and quantum fields_ (World Scientific, Singapore, 2018).
  * Errea  _et al._ (2015b) I. Errea, M. Calandra, C. J. Pickard, J. Nelson, R. J. Needs, Y. Li, H. Liu, Y. Zhang, Y. Ma, and F. Mauri, Phys. Rev. Lett. 114, 157004 (2015b).
  * Baggioli  _et al._ (2020) M. Baggioli, C. Setty, and A. Zaccone, Phys. Rev. B 101, 214502 (2020).
  * Marsiglio and Carbotte (2008b) F. Marsiglio and J. Carbotte, in  _Superconductivity_ (Springer, 2008) pp. 73–162.
  * Allen and Dynes (1975) P. B. Allen and R. C. Dynes, Phys. Rev. B 12, 905 (1975).
  * Jiang  _et al._ (2023) C. Jiang, E. Beneduce, M. Baggioli, C. Setty, and A. Zaccone, Journal of Physics: Condensed Matter 35, 164003 (2023).
  * Lee and Ramakrishnan (1985) P. A. Lee and T. Ramakrishnan, Reviews of Modern Physics 57, 287 (1985).
  * Cordero  _et al._ (2001) F. Cordero, A. Paolone, R. Cantelli, and M. Ferretti, Physical Review B 64, 132501 (2001).
  * Julien (2003) M.-H. Julien, Physica B: Condensed Matter 329, 693 (2003).
  * Read  _et al._ (1995) N. Read, S. Sachdev, and J. Ye, Physical Review B 52, 384 (1995).
  * Bray and Moore (1985) A. Bray and M. Moore, Physical Review B 31, 631 (1985).
  * Miller and Huse (1993) J. Miller and D. A. Huse, Physical review letters 70, 3147 (1993).
  * Dalidovich and Phillips (1999) D. Dalidovich and P. Phillips, Physical Review B 59, 11925 (1999).
  * Bennemann and Ketterson (2008) K.-H. Bennemann and J. B. Ketterson,  _Superconductivity: Volume 1: Conventional and Unconventional Superconductors Volume 2: Novel Superconductors_ (Springer Science & Business Media, 2008).
  * Hirschfeld  _et al._ (2011) P. Hirschfeld, M. Korshunov, and I. Mazin, Reports on Progress in Physics 74, 124508 (2011).
  * Venkataraman (1979) G. Venkataraman, Bulletin of Materials Science 1, 129 (1979).
  * Xu (2013) Y. Xu,  _Ferroelectric materials and their applications_ (Elsevier, 2013).
  * Gonzalo (2006) J. A. Gonzalo,  _Effective field approach to phase transitions and some applications to ferroelectrics_ , Vol. 76 (World Scientific, 2006).
  * Cowley (2012) R. Cowley, Integrated Ferroelectrics 133, 109 (2012).
  * Setty  _et al._ (2022b) C. Setty, M. Baggioli, and A. Zaccone, Phys. Rev. B 105, L020506 (2022b).
  * Setty  _et al._ (2022c) C. Setty, M. Baggioli, and A. Zaccone, Phys. Rev. B 105, L020506 (2022c).
  * Edge  _et al._ (2015a) J. M. Edge, Y. Kedem, U. Aschauer, N. A. Spaldin, and A. V. Balatsky, Phys. Rev. Lett. 115, 247002 (2015a).
  * Koonce  _et al._ (1967) C. S. Koonce, M. L. Cohen, J. F. Schooley, W. R. Hosler, and E. R. Pfeiffer, Phys. Rev. 163, 380 (1967).
  * Kohn (1959) W. Kohn, Phys. Rev. Lett. 2, 393 (1959).
  * Peierls  _et al._ (1955) R. Peierls, R. Peierls, R. Peierls, and O. U. Press,  _Quantum Theory of Solids_ , International series of monographs on physics (Clarendon Press, 1955).
  * Blackburn  _et al._ (2011) S. Blackburn, M. Côté, S. G. Louie, and M. L. Cohen, Phys. Rev. B 84, 104506 (2011).
  * Ren  _et al._ (2021) S. Ren, H.-X. Zong, X.-F. Tao, Y.-H. Sun, B.-A. Sun, D.-Z. Xue, X.-D. Ding, and W.-H. Wang, Nature communications 12, 1 (2021).
  * Bergmann and Rainer (1973) G. Bergmann and D. Rainer, Zeitschrift für Physik 263, 59 (1973).
  * Leroux  _et al._ (2018) M. Leroux, V. Mishra, J. P. Ruff, H. Claus, M. P. Smylie, C. Opagiste, P. Rodière, A. Kayani, G. Gu, J. M. Tranquada, _et al._ , arXiv preprint arXiv:1808.05984 (2018).
  * Fernandes  _et al._ (2012) R. Fernandes, M. Vavilov, and A. Chubukov, Physical Review B 85, 140512 (2012).
  * Mishra and Hirschfeld (2016) V. Mishra and P. Hirschfeld, New Journal of Physics 18, 103001 (2016).
  * Ashcroft (1968) N. W. Ashcroft, Phys. Rev. Lett. 21, 1748 (1968).
  * Dasenbrock-Gammon  _et al._ (2023) N. Dasenbrock-Gammon, E. Snider, R. McBride, H. Pasan, D. Durkee, N. Khalvashi-Sutter, S. Munasinghe, S. E. Dissanayake, K. V. Lawler, A. Salamat, and R. P. Dias, Nature 615, 244 (2023).
  * Dogan  _et al._ (2022) M. Dogan, S. Oh, and M. L. Cohen, Journal of Physics: Condensed Matter 34, 15LT01 (2022).
  * Pickard and Needs (2007) C. J. Pickard and R. J. Needs, Nature Physics 3, 473 (2007).
  * Pickett (2022) W. E. Pickett, arXiv e-prints , arXiv:2204.05930 (2022), arXiv:2204.05930 [cond-mat.supr-con] .
  * Zurek and Bi (2019) E. Zurek and T. Bi, The Journal of Chemical Physics 150, 050901 (2019).
  * Setty  _et al._ (2021a) C. Setty, M. Baggioli, and A. Zaccone, Phys. Rev. B 103, 094519 (2021a).
  * Verma  _et al._ (2021) A. K. Verma, P. Modak, F. Schrodi, A. Aperis, and P. M. Oppeneer, Phys. Rev. B 103, 094505 (2021).
  * Hou  _et al._ (2021) P. Hou, F. Belli, R. Bianco, and I. Errea, Journal of Applied Physics 130, 175902 (2021), https://doi.org/10.1063/5.0063968 .
  * Rousseau and Bergara (2010) B. Rousseau and A. Bergara, Physical Review B 82, 104504 (2010).
  * Dangic  _et al._ (2023) D. Dangic, L. Monacelli, R. Bianco, F. Mauri, and I. Errea, arXiv e-prints , arXiv:2303.07962 (2023), arXiv:2303.07962 [cond-mat.supr-con] .
  * Poncé  _et al._ (2016) S. Poncé, E. R. Margine, C. Verdi, and F. Giustino, Computer Physics Communications 209, 116 (2016).
  * Setty  _et al._ (2022d) C. Setty, M. Baggioli, and A. Zaccone, Phys. Rev. B 106, 139902 (2022d).
  * Kunc  _et al._ (2003) K. Kunc, I. Loa, and K. Syassen, Phys. Rev. B 68, 094107 (2003).
  * Birch (1947) F. Birch, Phys. Rev. 71, 809 (1947).
  * Xu  _et al._ (2008) Y. Xu, J.-S. Wang, W. Duan, B.-L. Gu, and B. Li, Phys. Rev. B 78, 224303 (2008).
  * Chen  _et al._ (2021) W. Chen, D. V. Semenok, X. Huang, H. Shu, X. Li, D. Duan, T. Cui, and A. R. Oganov, arXiv e-prints , arXiv:2101.01315 (2021), arXiv:2101.01315 [cond-mat.supr-con] .
  * Brandt and Ginzburg (1969) N. B. Brandt and N. I. Ginzburg, Soviet Physics Uspekhi 12, 344 (1969).
  * Boughton  _et al._ (1970) R. Boughton, J. Olsen, and C. Palmy (Elsevier, 1970) pp. 163 – 203.
  * Lorenz and Chu (2005) B. Lorenz and C. Chu, “High pressure effects on superconductivity,” in  _Frontiers in Superconducting Materials_ , edited by A. V. Narlikar (Springer Berlin Heidelberg, Berlin, Heidelberg, 2005) pp. 459–497.
  * Hiroi  _et al._ (2011) Z. Hiroi, J.-i. Yamaura, and K. Hattori, Journal of the Physical Society of Japan 81, 011012 (2011).
  * Hattori and Tsunetsugu (2010) K. Hattori and H. Tsunetsugu, Physical Review B 81, 134503 (2010).
  * Müller and Burkard (1979) K. A. Müller and H. Burkard, Phys. Rev. B 19, 3593 (1979).
  * Appel (1969) J. Appel, Physical Review 180, 508 (1969).
  * Enderlein  _et al._ (2020) C. Enderlein, J. F. de Oliveira, D. A. Tompsett, E. B. Saitovitch, S. S. Saxena, G. G. Lonzarich, and S. E. Rowley, Nature Communications 11, 4852 (2020).
  * Edge  _et al._ (2015b) J. M. Edge, Y. Kedem, U. Aschauer, N. A. Spaldin, and A. V. Balatsky, Phys. Rev. Lett. 115, 247002 (2015b).
  * Franklin  _et al._ (2021) J. Franklin, B. Xu, D. Davino, A. Mahabir, A. V. Balatsky, U. Aschauer, and I. Sochnikov, Phys. Rev. B 103, 214511 (2021).
  * He  _et al._ (2020) X. He, D. Bansal, B. Winn, S. Chi, L. Boatner, and O. Delaire, Phys. Rev. Lett. 124, 145901 (2020).
  * Marsiglio (2020) F. Marsiglio, Annals of Physics 417, 168102 (2020), eliashberg theory at 60: Strong-coupling superconductivity and beyond.
  * Bloch  _et al._ (2008) I. Bloch, J. Dalibard, and W. Zwerger, Reviews of modern physics 80, 885 (2008).
  * Setty  _et al._ (2022e) C. Setty, J. Zhao, L. Fanfarillo, E. W. Huang, P. J. Hirschfeld, P. W. Phillips, and K. Yang, arXiv preprint arXiv:2209.10568 (2022e).
  * Setty  _et al._ (2021b) C. Setty, L. Fanfarillo, and P. Hirschfeld, arXiv preprint arXiv:2110.13138 (2021b).
  * Setty (2020) C. Setty, Physical Review B 101, 184506 (2020).
  * Setty (2021) C. Setty, Physical Review B 103, 014501 (2021).
  * Phillips  _et al._ (2020) P. W. Phillips, L. Yeo, and E. W. Huang, Nature Physics 16, 1175 (2020).
  * Fomin (2021) V. M. Fomin,  _Self-rolled Micro- and Nanoarchitectures_ (De Gruyter, Berlin, Boston, 2021).
  * Travaglino and Zaccone (2023) R. Travaglino and A. Zaccone, Journal of Applied Physics 133, 033901 (2023).
  * Zaccone (2023) A. Zaccone,  _Theory of Disordered Solids_ (Springer, Heidelberg, 2023).