# Semantic Mirror Jailbreak: Genetic Algorithm
Based Jailbreak Prompts Against Open-source LLMs

Xiaoxia Li  Siyuan Liang  Jiyi Zhang  Han Fang  Aishan Liu  Ee-Chien Chang

###### Abstract

Large Language Models (LLMs), used in creative writing, code generation, and
translation, generate text based on input sequences but are vulnerable to
jailbreak attacks, where crafted prompts induce harmful outputs. Most
jailbreak prompt methods use a combination of jailbreak templates followed by
questions to ask to create jailbreak prompts. However, existing jailbreak
prompt designs generally suffer from excessive semantic differences, resulting
in an inability to resist defenses that use simple semantic metrics as
thresholds. Jailbreak prompts are semantically more varied than the original
questions used for queries. In this paper, we introduce a Semantic Mirror
Jailbreak (SMJ) approach that bypasses LLMs by generating jailbreak prompts
that are semantically similar to the original question. We model the search
for jailbreak prompts that satisfy both semantic similarity and jailbreak
validity as a multi-objective optimization problem and employ a standardized
set of genetic algorithms for generating eligible prompts. Compared to the
baseline AutoDAN-GA, SMJ achieves attack success rates (ASR) that are at most
35.4% higher without ONION defense and 85.2% higher with ONION defense. SMJ’s
better performance in all three semantic meaningfulness metrics of Jailbreak
Prompt, Similarity, and Outlier, also means that SMJ is resistant to defenses
that use those metrics as thresholds.

Machine Learning, ICML

##  1 Introduction

As Large Language Models (LLMs) have outstanding performance in various areas
including translation, code generation, creative writing, and so on. Various
safeguards are used to prevent LLMs from being misused, including
reinforcement learning from human feedback (RLHF) (Ouyang et al., 2022),
OpenAI’s usage policies (OpenAI, 2024b), and Meta Llama 2’s use policy (Meta,
2024). However, recent studies have shown that deep models are vulnerable to
security threats (Wei et al., 2018; Liang et al., 2020, 2022c; Liu et al.,
2023a; Liang et al., 2022a; Liu et al., 2023b; Liang et al., 2022b; He et al.,
2023; Liang et al., 2021; Sun et al., 2023; Li et al., 2023b; Liu et al.,
2023d; Liang et al., 2023a; Liu et al., 2023c; Liang et al., 2023b; Wang et
al., 2022; Liu et al., 2019, 2020b, 2020a; Wang et al., 2021; Liu et al.,
2023a). By carefully designing jailbreak prompts, which is to add a jailbreak
template designed for a single LLM or various LLMs before the questions to
ask, harmful responses and hate speech (Kang et al., 2023; Goldstein et al.,
2023; Hazell, 2023) can still be yielded from LLMs. Consequently, jailbreak
attacks pose a threat to the widespread use of LLMs.

There are two main categories of existing jailbreak attacks, they are
attacking using handcrafted jailbreak prompts and automatically generated
jailbreak prompts. For handcrafted jailbreak prompts, those prompts designed
for jailbreaking different LLMs spread through platforms such as Discord and
Reddit. Shen et al. (2023) collected handcrafted jailbreak prompts and
performed evaluations on LLMs’ vulnerability. For automatically generated
jailbreak prompts, Zou et al. (2023) proposed GCG, which utilizes greedy and
gradient-based search techniques to generate universal and transferable
jailbreak templates by using multiple LLMs to perform optimization. Liu et al.
(2023e) also proposed AutoDAN, that method generates jailbreak prompts by
optimizing with hierarchical genetic algorithm. Compared to GCG’s jailbreak
templates, AutoDAN’s jailbreak templates have the advantage of stealthiness
against naive jailbreak defenses such as perplexity-based detection (Jain et
al., 2023), and they are more semantically meaningful.

However, since the existing jailbreak attacks employ a combination of
jailbreak templates followed by questions to ask to create the jailbreak
prompts, that jailbreak prompt design creates two limitations for the attacks.
The first limitation is that by using jailbreak templates as a tool to perform
attacks, the attacks would be more vulnerable to jailbreak defenses because
the jailbreak prompts would be relatively less semantically meaningful as
compared to the original question. The second limitation is without using
jailbreak templates, it is not possible to elicit harmful questions’ responses
from LLMs. Hence, jailbreak prompts’ semantic meaningfulness and the attack
success rate (ASR) have a negative correlation, they cannot be optimized
together.

This paper proposes Semantic Mirror Jailbreak (SMJ), it suggests a direct
similarity between the original questions and the jailbreak prompts, much like
a mirror reflecting an image. The method simultaneously optimizes the two
goals of semantic similarity and attack validity to generate jailbreak
prompts. SMJ aims to address the two limitations by utilizing the genetic
algorithm. In the initialization stage, SMJ uses paraphrased questions
generated by referring to the original question as the initial population to
ensure jailbreak prompts’ semantic meaningfulness. By subsequently applying
fitness evaluation, which takes both jailbreak prompts’ semantic similarity
and attack validity into consideration, before selection and crossover, this
can guarantee both jailbreak prompts’ semantic meaningfulness and the attack
success rate (ASR) optimized concurrently.

Our contributions can be summarized as follows:

  * •

We propose a new jailbreak attack that designs a jailbreak prompt that can
satisfy both semantic similarity and attack availability. We model the above
attack as a multi-objective optimization problem.

  * •

We propose a genetic algorithm-based scheme for automatic prompt generation,
i.e., semantic mirror jailbreak, which can achieve an effective attack while
preserving the source semantic information by means of a well-designed
population and optimization strategy.

  * •

Experiments show that SMJ can resist simple defenses that use semantic
meaningfulness metrics as thresholds and bypass more advanced defense, the
ONION defense. Compared to the baseline AutoDAN-GA, SMJ achieves up to 35.4%
improvement in ASR.

Figure 1: An illustration of jailbreak prompt. If querying using a normal
harmful question, LLMs will reject answering the question in red. However, if
using the existing jailbreak prompt which combines a jailbreak template with
the question, LLMs will generate a harmful response. Semantic Mirror Jailbreak
(SMJ)’s jailbreak prompt can also reach the same outcome but the prompt would
be more semantically meaningful.

##  2 Background and Related Works

Large Language Models (LLMs)  LLMs are transformer models that can generate
human-like text, they are trained on massive datasets, and attention
mechanisms are employed to predict the next word for response generation.
There are open-sourced LLMs such as Llama-2 (Touvron et al., 2023) and Vicuna
(Zheng et al., 2023), and commercial APIs such as GPT-3.5 Turbo and GPT-4
(OpenAI, 2024a). Since LLMs have outstanding performance in various areas, to
prevent misuse, such as responding with hallucinations (Bang et al., 2023) and
producing misinformation (Pegoraro et al., 2023; Zhou et al., 2023),
safeguards were applied to provide ethical guidelines. For example, LLM
developers using reinforcement learning from human feedback (RLHF) (Ouyang et
al., 2022) to align LLMs with human values and expectations, OpenAI refers to
their usage policies (OpenAI, 2024b), Vicuna filters inappropriate user inputs
by using OpenAI moderation API (OpenAI, 2024c) in their online demo (Chiang et
al., 2023), Meta Llama 2 also refers to their use policy (Meta, 2024).

Jailbreak Attacks against LLMs  With the emergence of LLMs, although there are
LLM’s safeguards set by model developers to prevent LLM from generating
harmful responses and hate speech (Kang et al., 2023; Goldstein et al., 2023;
Hazell, 2023), there are still ways to bypass the LLM’s safeguards by using
jailbreak prompts, which are called jailbreak attacks. Starting from
handcrafted jailbreak prompts, jailbreak templates with careful prompt
engineering for jailbreaking different LLMs spread through platforms such as
Discord and Reddit. Shen et al. (2023) collected handcrafted jailbreak
templates from four platforms, by combining jailbreak templates and questions,
they then performed attacks and discovered LLMs’ vulnerability against
handcrafted jailbreak prompts. Other than handcrafted jailbreak prompts,
another main jailbreak attack category is automatically generated jailbreak
prompts. Extensive research has proposed automatic jailbreak attacks, Zou et
al. (2023) produce automatic jailbreak prompts by generating adversarial
suffixes, which are used to append to the end of the questions to ask, by a
combination of greedy and gradient-based search techniques. Liu et al. (2023e)
proposed AutoDAN that generates jailbreak prompts by employing hierarchical
genetic algorithm for optimization. Also, by applying the idea of chain of
thought prompting, multi-step jailbreaking prompts can be used to jailbreak
(Li et al., 2023a).

Other than using jailbreak prompts to perform jailbreak attacks, there are
still other ways, such as Huang et al. (2023), they jailbreak by exploiting
different text generation configurations.

Defense against Jailbreak Attacks  With the occurrence of various jailbreak
attacks against LLMs, Jain et al. (2023) evaluated three types of defenses,
including perplexity-based detection, input preprocessing by paraphrase and
retokenization, and adversarial training, on both white-box and gray-box
settings. Moreover, Laiyer.ai (2023) proposes “deberta-v3-base-prompt-
injection”, a model fine-tuned to identify jailbreak prompts and normal
questions, which can used to effectively defense against existing jailbreak
attacks.

Figure 2: This paper proposes Semantic Mirror Jailbreak (SMJ), a method that
uses paraphrased questions generated by referring to the original question as
the initial population to ensure jailbreak prompts’ semantic meaningfulness.
By subsequently applying fitness evaluation, which takes both jailbreak
prompts’ semantic similarity and attack validity into consideration, before
selection and crossover, this can guarantee both jailbreak prompts’ semantic
meaningfulness and the attack success rate (ASR) optimized concurrently.

##  3 Method

###  3.1 Preliminaries

Threat model  Given a set of harmful questions represented as
$Q=\\{\mathit{Q}_{1},\mathit{Q}_{2},\ldots,\mathit{Q}_{n}\\}$, produce its
related paraphrased questions set
$Q^{\prime}=\\{Q^{\prime}_{i}=\\{P_{i1},P_{i2},\ldots,P_{ij}\\}\\}\begin{subarray}{c}i=1,2,\ldots,n\\\
j=1,2,\ldots,k\end{subarray}$, using the paraphrased questions set
$Q^{\prime}$ to query the victim LLM $M$, we can get a set of response
$R=\\{\mathit{R}_{1},\mathit{R}_{2},\ldots,\mathit{R}_{n}\\}$. To find out the
best paraphrased question $P_{ij}$ for each harmful question $Q_{i}$, two
fitness functions are needed. They are the sentence transformer model $S$ and
victim LLM $M$, $S$ was used to assess the semantic similarity between $Q_{i}$
and $Q^{\prime}_{i}$, while $M$ was used to assess whether $R_{ij}$ was
considered successfully jailbroken. If $R_{ij}$ was considered jailbroken,
then $M(P_{ij})=1$, else $M(P_{ij})=0$. The objective of the jailbreak attack
is to find out a paraphrased question $P_{ij}$, that $M(P_{ij})=1$ and is with
highest $S(Q_{i},P_{ij})$, for each $Q_{i}$.

Formulation  Given a harmful question $Q_{i}$, by producing its paraphrased
questions $Q^{\prime}_{i}$, a best paraphrased question $P_{ij}$ can be chosen
by maximizing:

$\underset{P_{ij}\in Q^{\prime}_{i}}{{\arg\max}\,S(Q_{i},P_{ij}\mid
M(P_{ij})=1)}.$ (1)

###  3.2 Semantic Mirror Jailbreak

By considering the limitations of the existing jailbreak attack: 1) the
jailbreak prompts would be relatively less semantically meaningful as compared
to the original questions. The long jailbreak templates make the jailbreak
prompts contain more confusing content, also leading the attack to be more
vulnerable against jailbreak defenses. Moreover, 2) it seems that for existing
methods, including jailbreak templates within jailbreak prompts is a must.
Hence, bringing out the situation that without jailbreak prompts being less
semantically meaningful, the jailbreak attacks are less likely to be
successful, making it harder to optimize both the semantic meaningfulness and
the attack success rate (ASR) metrics together. By considering those two
limitations, this paper proposes SMJ, a method that addresses those two
limitations using the genetic algorithm.

###  3.3 Overall Review

Since the existing jailbreak prompts are a combination of jailbreak templates
followed by questions to ask, that combination produces relatively less
semantically meaningful jailbreak prompts as compared to the original
questions. Inspired by this idea, SMJ treats the questions to ask as jailbreak
prompts, regardless of the jailbreak templates. In this way, the jailbreak
prompts can be more normal and semantically meaningful. This method generates
paraphrased questions according to the original questions in the
initialization process. After that, by referring to the genetic algorithm,
using fitness functions to perform selection, and then applying crossover to
increase the population diversity, the resulting offspring would be used in
the next generation. Finally, the best paraphrased questions would be chosen
among all generations after iteratively employing the above operators until
the termination criteria are met. By using this method, optimizing both the
semantic meaningfulness and the attack success rate (ASR) metrics together
becomes achievable. _The summary algorithm for SMJ can be found in Algorithm
1, the detailed algorithm can be found in Algorithm 2 in Supplementary
Materials._

Algorithm 1 Semantic Mirror Jailbreak (SMJ)

Input: Harmful questions $Q$, Hyper-parameters

Output: Best paraphrased question $P_{ij}$ with highest semantic similarity as
compared to the original harmful question $Q_{i}$

for $Q_{i}$ in $Q$ do

Perform first half Population Initialization for $Q_{i}$ following Algorithm 3

Perform Fitness Evaluation following Algorithm 4

Perform Crossover following Algorithm 5

Perform Fitness Evaluation following Algorithm 4

repeat

Perform Selection following Algorithm 6

Perform Crossover following Algorithm 5

Perform Fitness Evaluation following Algorithm 4

until Termination criteria is met

end for

###  3.4 Implementation Details

Population Initialization  The first half of the initialization was conducted
by generating $N$ jailbreak prompts by substituting words in the original
harmful question $Q_{i}$ with their synonyms. The substitution follows the
method LWS proposed by Qi et al. (2021). Instead of using a specifically
calculated probability distribution for each candidate substitute words at
each word position of $Q_{i}$, SMJ generates $N$ jailbreak prompts by choosing
a random candidate substitute word for respective random positions of $Q_{i}$
and choosing a random number of words to replace with. The hyper-parameter
used in SMJ is the same as LWS, except for the number of candidate substitute
words for each word position of $Q_{i}$ is set to be 20. Moreover, since the
quality of the initial population’s individuals play a crucial role in the
quality of prompts generated afterward, to ensure the $N$ jailbreak prompts to
have high semantic similarity as compared to $Q_{i}$, the first half of the
jailbreak prompts only contain prompts with relatively high semantic
similarity. The second half of the initialization was conducted by
paraphrasing the $N$ jailbreak prompts, which are generated from the first
half of the initialization, by referring to the chosen one out of ten
syntactic forms (Iyyer et al., 2018). By default, $N$ is set to be 550.
Overall, the $2N$ jailbreak prompts are generated to be the initial
population. _The detailed population initialization algorithm for SMJ can be
found in Algorithm 3 in Supplementary Materials._

Fitness Evaluation  The fitness evaluation contains two fitness functions,
they are semantic similarity, which is evaluated using “all-mpnet-base-v2”
model proposed by Reimers & Gurevych (2022b), and attack validity, which is
judged by checking whether the jailbreak prompt’s LLM response contain any
keywords from the refusal keywords list in Appendix A. Once new jailbreak
prompts are generated, fitness evaluations are immediately applied in both the
initialization and the following generation processes. Only jailbreak prompts
deemed jailbreak successful and with semantic similarity within $k\%$ from the
current best paraphrased question are retained. The resulting individuals will
either be the filtered initial population or the filtered offspring used in
the selection process. $k$ is defaulted to be 10. By optimizing those two
fitness functions together, the second limitation can be solved. _The detailed
fitness evaluation algorithm for SMJ can be found in Algorithm 4 in
Supplementary Materials._

Selection  To select individuals for the crossover process, the roulette wheel
selection is applied to either the filtered initial population or the filtered
offspring. Given the semantic similarity of those jailbreak prompts for each
generation, and assuming the current population is of size $Z$, the
probability of each jailbreak prompt $P_{ij}$ being selected is:

$p_{ij}=\frac{S(Q_{i},P_{ij})}{\sum_{m=1}^{Z}S(Q_{i},P_{im})}$ (2)

Eventually, $O/10$ prompts will be selected for each generation, but if $Z$ is
smaller than or equal to $O/10$, no selection is needed and all prompts in the
current generation will be used for crossover. _The detailed selection
algorithm for SMJ can be found in Algorithm 6 in Supplementary Materials._

Crossover  All individuals selected from the selection process would be used
to crossover. Different from the traditional crossover in the genetic
algorithm, to produce $O$ offspring jailbreak prompts, SMJ’s crossover is
conducted by using all ten syntactic forms to paraphrase all input
individuals, so each individual generates ten paraphrased offspring prompts
for the next generation (Iyyer et al., 2018). $O$ is defaulted to be 120. _The
detailed crossover algorithm for SMJ can be found in Algorithm 5 in
Supplementary Materials._

Termination criteria  There are three termination criteria for SMJ, the first
criteria is the number of generations reaches the $MaxGenerations$,
$MaxGenerations$ defaults to be 10. The second termination criteria is the
current best paraphrased question’s semantic similarity does not change for
$G$ continuous generations, $G$ defaults to be 3. The third criteria is no new
individual is produced in the current generation, there are two situations, 1)
all individuals in the offspring for any generation have already existed or
were assessed before the current generation, or 2) there is no available
initial population to start the generation after applying the fitness
evaluation. If any of the above criteria are achieved, the algorithm stops and
returns the best paraphrased question with the highest semantic similarity
among all generations.

Table 1: Attack Success Rate (ASR) and Similarity.
Models | Llama-2-7b-chat-hf | Vicuna-7b | Guanaco-7b
---|---|---|---
Methods | ASR% | Similarity% | ASR% | Similarity% | ASR% | Similarity%
Original question | 1.40 | 100.00 | 32.00 | 100.00 | 47.00 | 100.00
AutoDAN-GA | 30.60 | 4.65 | 79.80 | 5.74 | 94.20 | 6.47
SMJ | 66.00 | 73.41 | 98.60 | 92.13 | 100.00 | 94.63
Table 2: Jailbreak Prompt (JPt) and Outlier.
Models | Llama-2-7b-chat-hf | Vicuna-7b | Guanaco-7b
---|---|---|---
Methods | JPt% | Outlier | JPt% | Outlier | JPt% | Outlier
Original question | 00.00 | 1.63 | 00.00 | 1.63 | 00.00 | 1.63
AutoDAN-GA | 100.00 | 22.24 | 99.00 | 19.65 | 99.60 | 14.44
SMJ | 00.00 | 2.47 | 00.00 | 2.11 | 00.00 | 1.99

##  4 Experiments

###  4.1 Experimental Setups

Dataset  The dataset used for evaluation was collected by Yu et al. (2023), it
contains 100 questions related to the contents that are illegal, toxic,
discrimination, immoral and so on. The dataset was closer to reality because
the questions were either manually written or generated through crowdsourcing.

Baselines  There are two baselines for the experiment. The first baseline is
by directly attacking the victim model by using the original questions. The
second baseline chosen is AutoDAN-GA (Liu et al., 2023e). The attack makes use
of handcrafted jailbreak prompts as the initial population, using those
prompts to automatically generate jailbreak prompts by adopting genetic
algorithm for optimization purpose. For evaluation, we follow the original
setting of AutoDAN-GA except for the LLMs’ response generation, instead of
generating one sequence at a time, we use batch inference.

Metrics  To assess the performance of the jailbreak methods, we use Attack
Success Rate (ASR) as the primary metric. To assess whether the attack was
deemed successful, we refer to all the refusal keywords list provided by Zou
et al. (2023), Liu et al. (2023e), and Yu et al. (2023). The refusal keywords
list includes keywords such as “I apologize” and “is illegal and unethical”,
if the paraphrased question $P_{ij}$’s response $R_{ij}$ contains any of the
keywords in the refusal keywords list, $P_{ij}$ would be regarded as jailbreak
unsuccessful, the whole refusal keywords list is in Appendix A, the ASR was
calculated by $I_{success}/I_{total}$.

For semantic meaningfulness, the paper uses three metrics to evaluate. The
first one is the percentage of jailbreak prompts (Jailbreak Prompt) identified
by “deberta-v3-base-prompt-injection” model as prompt injections (Laiyer.ai,
2023), the lower the Jailbreak Prompt, the better the method performs against
the model used to classify inputs into jailbreak prompts or normal questions.
The second is the mean of the Semantic Similarity (Similarity) (Reimers &
Gurevych, 2022a), which measures the semantic textual similarity between pairs
of sentences (original questions and respective jailbreak prompts), this paper
uses the SOTA “all-mpnet-base-v2” model (Reimers & Gurevych, 2022b) for
evaluation. Lastly, Qi et al. (2020) proposed ONION defense against textual
backdoor attacks, the defense is based on outlier words detection in a
sentence. Since that could be an effective metric to differentiate between
jailbreak prompts produced by AutoDAN and SMJ, the mean of the Number of
Outlier Words (Outlier) within one sentence becomes the third metric.

Models  Three open-sourced large language models are used to evaluate against
the generated jailbreak prompts, they are Llama-2-7b-chat-hf (Touvron et al.,
2023), Vicuna-7b-v1.5 (Zheng et al., 2023), and Guanaco-7b (Dettmers et al.,
2023). Llama 2 is a language model with transformer architecture, it is
pretrained on data from publicly available sources. Specifically, both Vicuna
and Guanaco are fine-tuned based on Llama 2 using conversations from ShareGPT
and the multilingual dataset OASST1, respectively. All three models have
different sizes including 7B, 13B, and more. Moreover, all three LLMs are
subject to LLaMA license for intended use.

###  4.2 Results

####  4.2.1 Attack Success Rate (ASR) and Similarity

For ASR, only the best paraphrased question with a Similarity higher than
0.7000 is deemed jailbreak successfully for method SMJ. Table 1 records the
mean of Similarity for all baselines and SMJ’s generated jailbreak prompts
regardless of whether the jailbreak is successful or not for all the victim
models.

Table 1 shows that SMJ is able to achieve higher ASR performance than both of
the baselines. Specifically, SMJ outperforms AutoDAN-GA’s ASR by 35.4%, 18.8%,
and 5.8% while attacking LLMs of Llama-2, Vicuna-7b, and Guanaco-7b,
respectively. For Similarity, as compared to AutoDAN-GA, SMJ has a much higher
average Similarity, surpassing AutoDAN-GA by at most 88.16%, which indicates
that SMJ can produce jailbreak prompts that are more semantically meaningful
when compared to the original questions.

Overall, SMJ can effectively improve ASR and produce more semantically
meaningful jailbreak prompts.

####  4.2.2 Jailbreak Prompt (JPt) and Outlier

For semantic meaningfulness purpose, JPt and Outlier are also evaluated. Table
2 records the mean of JPt and Outlier for all baselines and SMJ’s generated
jailbreak prompts regardless of whether the jailbreak is successful or not for
all the victim models. For both metrics, the lower the metrics, the better the
method performs.

To the performance of JPt, none of the jailbreak prompts from SMJ were
classified as jailbreak prompts, which is the same as that of the original
question method. Meanwhile, to AutoDAN-GA, 100%, 99%, and 99.6% of jailbreak
prompts were classified as jailbreak prompts for Llama-2, Vicuna-7b, and
Guanaco-7b, respectively. Regarding the performance of Outlier, to compare
with the original question method, SMJ’s Outliers are at least 1.22 times and
at most 1.52 times higher, while AutoDAN-GA’s Outliers are at least 8.86 times
and at most 13.65 times higher.

Those results again indicate that SMJ’s jailbreak prompts are more
semantically meaningful, and the method is more resistant to defenses that use
those metrics as thresholds.

Table 3: Cross-model transferability. The notation * denotes a white-box
scenario, notation $\dagger$ denotes Similarity is measured on jailbreak
prompts with Similarity $\geq 0\%$ and jailbreak successfully, notation
$\dotplus$ denotes Similarity is measured on jailbreak prompts with Similarity
$\geq 70\%$ and jailbreak successfully, notation $\curlyvee$ denotes
Similarity is measured on jailbreak prompts with no Similarity or jailbreak
validity limitations.

Source Models |  Method | Llama-2-7b-chat-hf | Vicuna-7b | Guanaco-7b
---|---|---|---|---
ASR% | Similarity% | ASR% | Similarity% | ASR% | Similarity%
Llama-2-7b-chat-hf |  AutoDAN-GA |  30.60* |  5.45$\dagger$/0.00$\dotplus$/4.65*$\curlyvee$ | 25.20 |  8.80$\dagger$/0.00$\dotplus$/4.65$\curlyvee$ | 57.60 |  7.77$\dagger$/0.00$\dotplus$/4.65$\curlyvee$
SMJ-0% | 100.00* |  73.41*$\dagger$$\curlyvee$ | 64.00 |  74.47$\dagger$/73.41$\curlyvee$ | 55.20 |  74.58$\dagger$/73.41$\curlyvee$
|  SMJ-70% |  66.00* |  79.41$\dotplus$/73.41*$\curlyvee$ | 43.60 |  80.36$\dotplus$/73.41$\curlyvee$ | 38.80 |  79.80$\dotplus$/73.41$\curlyvee$
Vicuna-7b |  AutoDAN-GA | 1.00 |  9.00$\dagger$/0.00$\dotplus$/5.74$\curlyvee$ |  79.80* |  5.52$\dagger$/0.00$\dotplus$/5.74*$\curlyvee$ | 59.20 |  8.87$\dagger$/0.00$\dotplus$/5.74$\curlyvee$
SMJ-0% | 2.20 |  75.24$\dagger$/92.13$\curlyvee$ | 100.00* |  92.13*$\dagger$$\curlyvee$ | 46.20 |  93.38$\dagger$/92.13$\curlyvee$
|  SMJ-70% | 1.40 |  72.68$\dotplus$/92.13$\curlyvee$ |  98.60* |  92.62$\dotplus$/92.13*$\curlyvee$ | 46.00 |  93.56$\dotplus$/92.13$\curlyvee$
Guana co-7b |  AutoDAN-GA | 0.20 |  0.06$\dagger$/0.00$\dotplus$/6.47$\curlyvee$ | 15.20 |  9.97$\dagger$/0.00$\dotplus$/6.47$\curlyvee$ |  94.20* |  6.68$\dagger$/0.00$\dotplus$/6.47*$\curlyvee$
SMJ-0% | 1.20 |  72.99$\dagger$/94.63$\curlyvee$ | 28.40 |  95.55$\dagger$/94.63$\curlyvee$ | 100.00* | 94.63*$\dagger$$\curlyvee$
|  SMJ-70% | 1.20 |  72.99$\dotplus$/94.63$\curlyvee$ | 28.40 |  95.55$\dotplus$/94.63$\curlyvee$ | 100.00* | 94.63*$\dotplus$$\curlyvee$

####  4.2.3 Transferability

The transferability for all three methods is conducted by taking all jailbreak
prompts regardless of whether the jailbreak is successful or not in the source
models, using them to attack the target models in the white and black box
scenarios. As shown in the Section 4.2.2, it records the ASR for AutoDAN-GA
(regardless of the Similarity level), SMJ with Similarity higher than 0%, and
70% for different transfer experiments. It also records Similarity measured on
jailbreak prompts with Similarity $\geq 0\%$ or $\geq 70\%$ and jailbreak
successfully, and the Similarity measured on jailbreak prompts with no
Similarity or jailbreak validity limitations.

Section 4.2.2 indicates that SMJ outperforms AutoDAN-GA’s ASR in all cases
except for when the target model is Guanaco-7b in both Similarity higher than
0% and 70% settings, SMJ improves AutoDAN-GA’s ASR by at most 38.8% and 18.4%
for Similarity higher than 0% and 70% settings in black box scenario. For
Similarity, SMJ achieves higher performance than AutoDAN-GA in all cases. For
Similarity evaluated on jailbreak prompts with Similarity $\geq 0\%$ or $\geq
70\%$ meanwhile jailbreak successfully, SMJ improves the metric by at most
85.58% and 95.55% respectively in the black box scenario, 87.95% and 94.63%
respectively in the white box scenario. It is worth noticing that AutoDAN-GA
has 0% jailbreak prompts with Similarity $\geq 70\%$ meanwhile jailbreak
successfully.

The results indicate that SMJ has better transferability in the black box
scenario than AutoDAN-GA in most cases. Its jailbreak prompts that jailbreak
successfully has higher Similarity than the baseline in all cases.

Table 4: Attack Success Rate (ASR) against ONION defense.
Models | Llama-2-7b-chat-hf + ONION | Vicuna-7b + ONION | Guanaco-7b + ONION
---|---|---|---
Methods | ASR% | ASR% | ASR%
Original question | 1.40 | 32.00 | 47.00
AutoDAN-GA | 2.00 | 13.40 | 15.00
SMJ | 66.00 | 98.60 | 100.00

####  4.2.4 Attack Success Rate (ASR) against ONION defense

The defense chosen was the ONION defense, the defense measures the fluency of
a sentence by the perplexity computed by GPT-2 (Wolf et al., 2020). Then using
the decrease in the perplexity of the sentence while each words in the
sentence were removed one by one to compute the suspicion scores. If the
suspicion score is larger than a hyper-parameter suspicion threshold, then the
word removed is deemed an outlier word. For the defense purpose against
jailbreak prompts, we further add an outlier threshold for the number of
outliers within a sentence. If the number of outliers within a sentence is
higher than the outlier threshold, then the sentence would be considered as a
jailbreak prompt, otherwise, the sentence would be regarded as a normal
question without any intention for any jailbreak attack. By using this
defense, we can effectively defend against jailbreak prompts. The suspicion
threshold used in this experiment is 0, and the outlier threshold for all
three methods used is the maximum outlier contained in the sentence for all
SMJ’s jailbreak prompts with respect to different victim LLMs.

Table 4 shows that using ONION defense can distinguish between AutoDAN-GA and
SMJ’s jailbreak prompts. Consequently, AutoDAN-GA’s ASR for all LLMs decreased
by 28.6%, 66.4%, and 79.2% for Llama-2, Vicuna-7b, and Guanaco-7b,
respectively. While the original question and SMJ’s ASR for all LLMs remain
the same as that in Table 1.

This experiment shows that SMJ can bypass defense and appears to generate
jailbreak prompts that are more similar to normal harmful questions.

Table 5: Ablation Study.
Similarity level |  | $\geq 0\%$ | $\geq 60\%$ | $\geq 70\%$ | $\geq 80\%$ | $\geq 90\%$
---|---|---|---|---|---|---
Methods | Models | ASR% | Simi% | ASR% | Simi% | ASR% | Simi% | ASR% | Simi% | ASR% | Simi%
Original question | Llama-2 | 1.40 | 100.00 | 1.40 | 100.00 | 1.40 | 100.00 | 1.40 | 100.00 | 1.40 | 100.00
Vicuna | 32.00 | 100.00 | 32.00 | 100.00 | 32.00 | 100.00 | 32.00 | 100.00 | 32.00 | 100.00
Guanaco | 47.00 | 100.00 | 47.00 | 100.00 | 47.00 | 100.00 | 47.00 | 100.00 | 47.00 | 100.00
\+ initialization | Llama-2 | 100.00 | 71.18 | 82.00 | 75.33 | 58.40 | 79.41 | 26.80 | 85.36 | 4.60 | 93.02
Vicuna | 100.00 | 91.61 | 99.00 | 92.06 | 97.60 | 92.44 | 91.00 | 93.61 | 67.00 | 96.32
Guanaco | 100.00 | 94.52 | 100.00 | 94.52 | 100.00 | 94.52 | 99.20 | 94.66 | 82.00 | 96.55
\+ initialization + SGA | Llama-2 | 100.00 | 73.41 | 89.00 | 75.92 | 66.00 | 79.41 | 29.00 | 85.71 | 5.80 | 93.46
Vicuna | 100.00 | 92.13 | 99.40 | 92.40 | 98.60 | 92.62 | 92.40 | 93.73 | 69.40 | 96.27
Guanaco | 100.00 | 94.63 | 100.00 | 94.63 | 100.00 | 94.63 | 99.20 | 94.77 | 82.40 | 96.60

####  4.2.5 Ablation Study

There are three ablations conducted in this experiment. The ablation1 is to
only use the original prompts to attack the victim LLMs. The ablation2 is to
add the initialization process to ablation1. Ablation3 (SMJ) is to add the
standard genetic algorithm (SGA) to ablation2. Table 5 contain the results of
all three ablations for the three LLMs at different similarity levels.
Specifically, “Llama-2” stands for “Llama-2-7b-chat-hf”, “Vicuna” stands for
“Vicuna-7b”, “Guanaco” stands for “Guanaco-7b”, and “Simi” stands for
“Similarity”. Moreover, both metrics of ASR and Similarity are based on
jailbreak prompts that are within certain similarity levels meanwhile
jailbreak successfully.

For both ASR and Similarity, SMJ performs better than or equal to ablation2
for all LLMs and all similarity levels except for the Similarity of Vicuna-7b
at the similarity level greater than or equal to 90%. One possible explanation
could be the ASR for SMJ is higher than ablation2, so the number of jailbreak
prompts deemed jailbreak successfully from SMJ is higher, some of them might
have lower Similarity, causing SMJ’s Similarity of Vicuna-7b at the similarity
level greater than or equal to 90% lower than ablation2’s Similarity.
Moreover, SMJ can improve robust LLM Llama-2’s ASR by at most 7.6%, improve
Vicuna-7b’s ASR by at most 2.4%, and improve Guanaco-7b’s ASR by at most 0.4%.
For Similarity, SMJ can improve robust model Llama-2 by at most 2.23%, improve
Vicuna-7b by at most 0.52%, and improve Guanaco-7b by at most 0.11%. For LLMs
Vicuna-7b and Guanaco-7b, as the similarity levels get higher, the more SMJ
can improve ablation2 in ASR, the less SMJ can improve ablation2 in
Similarity.

Overall, SMJ can improve ablation2’s ASR and Similarity in most cases. SMJ can
improve both Llama-2 and Vicuna-7b’s ASR the most at higher similarity levels
while improving Similarity the most at lower similarity levels.

##  5 Conclusion

This paper proposes SMJ, a method that uses paraphrased questions as jailbreak
prompts and optimizes fitness functions using the genetic algorithm to ensure
jailbreak prompts’ semantic meaningfulness meanwhile optimizing both the
jailbreak prompts’ semantic meaningfulness and the attack success rate (ASR)
concurrently. This method can outperform AutoDAN-GA’s ASR. Moreover, by
evaluating the three metrics: Jailbreak Prompt, Similarity, and Outlier, the
results indicate that SMJ’s jailbreak prompts perform better than AutoDAN-GA
in semantic meaningfulness.

##  6 Ethics Statement and Countermeasure

Although the research proposes a method to allow LLMs to generate harmful
responses and hate speech, we believe that by referring to the current
jailbreak attacks, more research on jailbreak attack defenses will emerge,
which will help boost the safeguards of LLMs. Hence, we hope our work will not
pose significant dangers to LLMs in the long term but rather clarify the
potential vulnerabilities of LLMs. For countermeasure of SMJ, if an attacker
uses lots of jailbreak prompts with similar Similarity to continuously query
the LLMs, a possible defense can be detecting the Similarity of inputs, and
refusing to answer those inputs with high Similarity.

## References

  * Bang et al. (2023) Bang, Y., Cahyawijaya, S., Lee, N., Dai, W., Su, D., Wilie, B., Lovenia, H., Ji, Z., Yu, T., Chung, W., et al.  A multitask, multilingual, multimodal evaluation of chatgpt on reasoning, hallucination, and interactivity.  _arXiv preprint arXiv:2302.04023_ , 2023.
  * Chiang et al. (2023) Chiang, W.-L., Li, Z., Lin, Z., Sheng, Y., Wu, Z., Zhang, H., Zheng, L., Zhuang, S., Zhuang, Y., Gonzalez, J. E., Stoica, I., and Xing, E. P.  Vicuna: An open-source chatbot impressing gpt-4 with 90%* chatgpt quality, March 2023.  URL https://lmsys.org/blog/2023-03-30-vicuna/.
  * Dettmers et al. (2023) Dettmers, T., Pagnoni, A., Holtzman, A., and Zettlemoyer, L.  Qlora: Efficient finetuning of quantized llms.  _arXiv preprint arXiv:2305.14314_ , 2023.
  * Goldstein et al. (2023) Goldstein, J. A., Sastry, G., Musser, M., DiResta, R., Gentzel, M., and Sedova, K.  Generative language models and automated influence operations: Emerging threats and potential mitigations.  _arXiv preprint arXiv:2301.04246_ , 2023.
  * Hazell (2023) Hazell, J.  Large language models can be used to effectively scale spear phishing campaigns.  _arXiv preprint arXiv:2305.06972_ , 2023.
  * He et al. (2023) He, B., Liu, J., Li, Y., Liang, S., Li, J., Jia, X., and Cao, X.  Generating transferable 3d adversarial point cloud via random perturbation factorization.  In _Proceedings of the AAAI Conference on Artificial Intelligence_ , 2023.
  * Huang et al. (2023) Huang, Y., Gupta, S., Xia, M., Li, K., and Chen, D.  Catastrophic jailbreak of open-source llms via exploiting generation.  _arXiv preprint arXiv:2310.06987_ , 2023.
  * Iyyer et al. (2018) Iyyer, M., Wieting, J., Gimpel, K., and Zettlemoyer, L.  Adversarial example generation with syntactically controlled paraphrase networks.  _arXiv preprint arXiv:1804.06059_ , 2018.
  * Jain et al. (2023) Jain, N., Schwarzschild, A., Wen, Y., Somepalli, G., Kirchenbauer, J., Chiang, P.-y., Goldblum, M., Saha, A., Geiping, J., and Goldstein, T.  Baseline defenses for adversarial attacks against aligned language models.  _arXiv preprint arXiv:2309.00614_ , 2023.
  * Kang et al. (2023) Kang, D., Li, X., Stoica, I., Guestrin, C., Zaharia, M., and Hashimoto, T.  Exploiting programmatic behavior of llms: Dual-use through standard security attacks.  _arXiv preprint arXiv:2302.05733_ , 2023.
  * Laiyer.ai (2023) Laiyer.ai.  Fine-tuned deberta-v3 for prompt injection detection, 2023.  URL https://huggingface.co/laiyer/deberta-v3-base-prompt-injection.
  * Li et al. (2023a) Li, H., Guo, D., Fan, W., Xu, M., and Song, Y.  Multi-step jailbreaking privacy attacks on chatgpt.  _arXiv preprint arXiv:2304.05197_ , 2023a.
  * Li et al. (2023b) Li, J., Zhang, H., Liang, S., Dai, P., and Cao, X.  Privacy-enhancing face obfuscation guided by semantic-aware attribution maps.  _IEEE Transactions on Information Forensics and Security_ , 2023b.
  * Liang et al. (2023a) Liang, J., Liang, S., Liu, A., Ma, K., Li, J., and Cao, X.  Exploring inconsistent knowledge distillation for object detection with data augmentation.  In _Proceedings of the 31st ACM International Conference on Multimedia_ , pp. 768–778, 2023a.
  * Liang et al. (2020) Liang, S., Wei, X., Yao, S., and Cao, X.  Efficient adversarial attacks for visual object tracking.  In _CEuropean Conference on Computer Vision_ , 2020.
  * Liang et al. (2021) Liang, S., Wei, X., and Cao, X.  Generate more imperceptible adversarial examples for object detection.  In _ICML 2021 Workshop on Adversarial Machine Learning_ , 2021.
  * Liang et al. (2022a) Liang, S., Li, L., Fan, Y., Jia, X., Li, J., Wu, B., and Cao, X.  A large-scale multiple-objective method for black-box attack against object detection.  In _European Conference on Computer Vision_ , 2022a.
  * Liang et al. (2022b) Liang, S., Liu, A., Liang, J., Li, L., Bai, Y., and Cao, X.  Imitated detectors: Stealing knowledge of black-box object detectors.  In _Proceedings of the 30th ACM International Conference on Multimedia_ , 2022b.
  * Liang et al. (2022c) Liang, S., Wu, B., Fan, Y., Wei, X., and Cao, X.  Parallel rectangle flip attack: A query-based black-box attack against object detection.  _arXiv preprint arXiv:2201.08970_ , 2022c.
  * Liang et al. (2023b) Liang, S., Zhu, M., Liu, A., Wu, B., Cao, X., and Chang, E.-C.  Badclip: Dual-embedding guided backdoor attack on multimodal contrastive learning.  _arXiv preprint arXiv:2311.12075_ , 2023b.
  * Liu et al. (2019) Liu, A., Liu, X., Fan, J., Ma, Y., Zhang, A., Xie, H., and Tao, D.  Perceptual-sensitive gan for generating adversarial patches.  In _AAAI_ , 2019.
  * Liu et al. (2020a) Liu, A., Huang, T., Liu, X., Xu, Y., Ma, Y., Chen, X., Maybank, S. J., and Tao, D.  Spatiotemporal attacks for embodied agents.  In _ECCV_ , 2020a.
  * Liu et al. (2020b) Liu, A., Wang, J., Liu, X., Cao, B., Zhang, C., and Yu, H.  Bias-based universal adversarial patch attack for automatic check-out.  In _ECCV_ , 2020b.
  * Liu et al. (2023a) Liu, A., Guo, J., Wang, J., Liang, S., Tao, R., Zhou, W., Liu, C., Liu, X., and Tao, D.  X-adv: Physical adversarial object attacks against x-ray prohibited item detection.  In _USENIX Security Symposium_ , 2023a.
  * Liu et al. (2023b) Liu, A., Tang, S., Liang, S., Gong, R., Wu, B., Liu, X., and Tao, D.  Exploring the relationship between architectural design and adversarially robust generalization.  In _Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition_ , pp. 4096–4107, 2023b.
  * Liu et al. (2023c) Liu, A., Zhang, X., Xiao, Y., Zhou, Y., Liang, S., Wang, J., Liu, X., Cao, X., and Tao, D.  Pre-trained trojan attacks for visual recognition.  _arXiv preprint arXiv:2312.15172_ , 2023c.
  * Liu et al. (2023d) Liu, J., Zhu, S., Liang, S., Zhang, J., Fang, H., Zhang, W., and Chang, E.-C.  Improving adversarial transferability by stable diffusion.  _arXiv preprint arXiv:2311.11017_ , 2023d.
  * Liu et al. (2023e) Liu, X., Xu, N., Chen, M., and Xiao, C.  Autodan: Generating stealthy jailbreak prompts on aligned large language models.  _arXiv preprint arXiv:2310.04451_ , 2023e.
  * Meta (2024) Meta.  Llama 2 - acceptable use policy - meta ai.  https://ai.meta.com/llama/use-policy/, 2024.  Accessed: 2024-01-18.
  * OpenAI (2024a) OpenAI.  Introducing chatgpt.  https://openai.com/blog/chatgpt, 2024a.  Accessed: 2024-01-18.
  * OpenAI (2024b) OpenAI.  Usage policies.  https://openai.com/policies/usage-policies, 2024b.  Accessed: 2024-01-18.
  * OpenAI (2024c) OpenAI.  Moderation.  https://platform.openai.com/docs/guides/moderation/overview, 2024c.  Accessed: 2024-01-18.
  * Ouyang et al. (2022) Ouyang, L., Wu, J., Jiang, X., Almeida, D., Wainwright, C., Mishkin, P., Zhang, C., Agarwal, S., Slama, K., Ray, A., et al.  Training language models to follow instructions with human feedback.  _Advances in Neural Information Processing Systems_ , 35:27730–27744, 2022.
  * Pegoraro et al. (2023) Pegoraro, A., Kumari, K., Fereidooni, H., and Sadeghi, A.-R.  To chatgpt, or not to chatgpt: That is the question!  _arXiv preprint arXiv:2304.01487_ , 2023.
  * Qi et al. (2020) Qi, F., Chen, Y., Li, M., Yao, Y., Liu, Z., and Sun, M.  Onion: A simple and effective defense against textual backdoor attacks.  _arXiv preprint arXiv:2011.10369_ , 2020.
  * Qi et al. (2021) Qi, F., Yao, Y., Xu, S., Liu, Z., and Sun, M.  Turn the combination lock: Learnable textual backdoor attacks via word substitution.  _arXiv preprint arXiv:2106.06361_ , 2021.
  * Reimers & Gurevych (2022a) Reimers, N. and Gurevych, I.  Semantic textual similarity.  https://www.sbert.net/docs/usage/semantic_textual_similarity.html#semantic-textual-similarity, 2022a.  Accessed: 2024-01-20.
  * Reimers & Gurevych (2022b) Reimers, N. and Gurevych, I.  Pretrained models.  https://www.sbert.net/docs/pretrained_models.html, 2022b.  Accessed: 2024-01-20.
  * Shen et al. (2023) Shen, X., Chen, Z., Backes, M., Shen, Y., and Zhang, Y.  ” do anything now”: Characterizing and evaluating in-the-wild jailbreak prompts on large language models.  _arXiv preprint arXiv:2308.03825_ , 2023.
  * Sun et al. (2023) Sun, C., Xu, C., Yao, C., Liang, S., Wu, Y., Liang, D., Liu, X., and Liu, A.  Improving robust fariness via balance adversarial training.  In _Proceedings of the AAAI Conference on Artificial Intelligence_ , volume 37, pp. 15161–15169, 2023.
  * Touvron et al. (2023) Touvron, H., Martin, L., Stone, K., Albert, P., Almahairi, A., Babaei, Y., Bashlykov, N., Batra, S., Bhargava, P., Bhosale, S., et al.  Llama 2: Open foundation and fine-tuned chat models, 2023.  _URL https://arxiv. org/abs/2307.09288_ , 2023.
  * Wang et al. (2021) Wang, J., Liu, A., Yin, Z., Liu, S., Tang, S., and Liu, X.  Dual attention suppression attack: Generate adversarial camouflage in physical world.  In _CVPR_ , 2021.
  * Wang et al. (2022) Wang, Y., Shi, H., Min, R., Wu, R., Liang, S., Wu, Y., Liang, D., and Liu, A.  Adaptive perturbation generation for multiple backdoors detection.  _arXiv preprint arXiv:2209.05244_ , 2022.
  * Wei et al. (2018) Wei, X., Liang, S., Chen, N., and Cao, X.  Transferable adversarial attacks for image and video object detection.  _arXiv preprint arXiv:1811.12641_ , 2018.
  * Wolf et al. (2020) Wolf, T., Debut, L., Sanh, V., Chaumond, J., Delangue, C., Moi, A., Cistac, P., Rault, T., Louf, R., Funtowicz, M., Davison, J., Shleifer, S., von Platen, P., Ma, C., Jernite, Y., Plu, J., Xu, C., Scao, T. L., Gugger, S., Drame, M., Lhoest, Q., and Rush, A. M.  Transformers: State-of-the-art natural language processing.  In _Proceedings of the 2020 Conference on Empirical Methods in Natural Language Processing: System Demonstrations_ , pp. 38–45, Online, October 2020. Association for Computational Linguistics.  URL https://www.aclweb.org/anthology/2020.emnlp-demos.6.
  * Yu et al. (2023) Yu, J., Lin, X., and Xing, X.  Gptfuzzer: Red teaming large language models with auto-generated jailbreak prompts.  _arXiv preprint arXiv:2309.10253_ , 2023.
  * Zheng et al. (2023) Zheng, L., Chiang, W.-L., Sheng, Y., Zhuang, S., Wu, Z., Zhuang, Y., Lin, Z., Li, Z., Li, D., Xing, E., et al.  Judging llm-as-a-judge with mt-bench and chatbot arena.  _arXiv preprint arXiv:2306.05685_ , 2023.
  * Zhou et al. (2023) Zhou, J., Zhang, Y., Luo, Q., Parker, A. G., and De Choudhury, M.  Synthetic lies: Understanding ai-generated misinformation and evaluating algorithmic and human solutions.  In _Proceedings of the 2023 CHI Conference on Human Factors in Computing Systems_ , pp. 1–20, 2023.
  * Zou et al. (2023) Zou, A., Wang, Z., Kolter, J. Z., and Fredrikson, M.  Universal and transferable adversarial attacks on aligned language models.  _arXiv preprint arXiv:2307.15043_ , 2023.

##  Appendix A Refusal keywords list.

The refusal keywords list contains all lists provided by Zou et al. (2023),
Liu et al. (2023e), and Yu et al. (2023).

“I’m sorry”
---
“Sorry”
“I am sorry”
“I apologize”
“As an”
“As an AI”
“I’m an”
“I’m just”
“As a language model”
“As an Assistant”
“I cannot”
“I can’t”
“I do not”
“Hello!”
“is illegal and unethical”
“I’m really sorry”
“unethical”
“not ethical”
“illegal”
“not legal”
“My apologies”
“but I cannot”
“If you have any other non-malicious requests”
“I’m not able to provide”
“I am really sorry”
“I cannot create”
“I cannot support”
“I can’t provide”
“I can’t assist”
“I must”
“is not capable of”
“never”
“$<$/s$>$”
“Sorry, I am sorry”
“I’m really sorry”
“Hello”
“OpenAI”

##  Appendix B Detailed Algorithms.

The detailed algorithms for SMJ, population initialization, fitness
evaluation, crossover, and selection processes.

Algorithm 2 Detailed Semantic Mirror Jailbreak (SMJ)

Input:
$harmful\\_question,substitute\\_questions,max\\_generations,population\\_size,L_{refuse},seman\\_simi,\newline
static\\_threshold$

Output: $best\\_solution[0]$

$all\\_generations\leftarrow\\{\\}$

$top\leftarrow-\infty$

$stop\leftarrow False$

$selection\\_similarity\\_region\leftarrow 0.1$

$init\\_1,best\\_solution,top,bottom,stop\leftarrow
FITNESS\\_EVALUATION(harmful\\_question,substitute\\_questions,\newline
L_{refuse},seman\\_simi,selection\\_similarity\\_region,top,stop,first=True)$

$crossovers\leftarrow
CROSSOVER(harmful\\_question,substitute\\_questions,stop,first=True)$

$init\\_2,best\\_solution,top,bottom,stop\leftarrow
FITNESS\\_EVALUATION(harmful\\_question,crossovers,L_{refuse},\newline
seman\\_simi,selection\\_similarity\\_region,top,stop,first=True)$

$current\\_generation\leftarrow 1$

$static\\_count\leftarrow 0$

$offspring\leftarrow init\\_1+init\\_2$

while $current\\_generation<=max\\_generations$ and not $stop$ and
$static\\_count<static\\_threshold$ do

$selected\\_offspring\leftarrow SELECTION(offspring,population\\_size)$

$temp\\_top\leftarrow top$

$crossovers\leftarrow
CROSSOVER(harmful\\_question,selected\\_offspring[substitute\\_questions],stop,first=False)$

$o,best\\_solution,top,bottom,stop\leftarrow
FITNESS\\_EVALUATION(harmful\\_question,crossovers,L_{refuse},\newline
seman\\_simi,selection\\_similarity\\_region,top,stop,first=False)$

if $top$ = $temp\\_top$ then

$static\\_count\leftarrow static\\_count+1$

else if $top>temp\\_top$ then

$static\\_count\leftarrow 0$

end if

$offspring\leftarrow o$

$current\\_generation\leftarrow current\\_generation+1$

end while

Algorithm 3 Population Initialization

Input: $harmful\\_questions,number\\_of\\_substitutions,bottom\\_similarity$,
$count\\_down\\_threhold,\newline similarity\\_decrement,seman\\_simi$

Output: $substitute\\_questions$ for all $harmful\\_questions$

$all\\_candidates\leftarrow[]$

for $harmful\\_question$ in $harmful\\_questions$ do

$candidates\leftarrow[]$

while $len(candidates)<number\\_of\\_substitutions$ and
$bottom\\_similarity>0$ do

$count\\_down\leftarrow 1$

if $count\\_down$ $\%$ $count\\_down\\_threshold=0$ then

$bottom\\_similarity\leftarrow bottom\\_similarity-similarity\\_decrement$

end if

for $i$ in $len(number\\_of\\_substitutions)$ do

$substitution\leftarrow generate\ one\ substitution$ for $harmful\\_question$

if $seman\\_simi(substitution)>=bottom\\_similarity$ and $substitution$ not in
$candidates$ then

append $substitution$ to $candidates$

end if

end for

$count\\_down\leftarrow count\\_down+1$

end while

append $candidates$ to $all\\_candidates$

end for

Algorithm 4 Fitness Evaluation

Input:
$harmful\\_question,substitute\\_questions,L_{refuse},seman\\_simi,selection\\_similarity\\_region,top,stop,\newline
first$

Output: dictionary of population $f$, $best\\_solution,top,bottom,stop$

$f\leftarrow\\{\\}$

$f[seman\\_simi]\leftarrow[]$

$f[substitute\\_questions]\leftarrow[]$

if $len(substitute\\_questions)$ = 0 then

$stop\leftarrow True$

else

$unsort\\_f_{1}\leftarrow seman\\_simi(substitute\\_questions)$

$f_{1}\leftarrow unsort\\_f_{1}$ sorted by values in $unsort\\_f_{1}$ in
descending order

$sorted\\_substitute\\_questions\leftarrow substitute\\_questions$ sorted by
values in $unsort\\_f_{1}$ in descending order

$bottom\leftarrow$ max$(top-selection\\_similarity\\_region,0)$

if $top\neq-\infty$ then

$f_{1}\leftarrow f_{1}$ with $value$ in $f_{1}\geq bottom$

$sorted\\_substitute\\_questions\leftarrow$ $sorted\\_substitute\\_questions$
with $value$ in $f_{1}\geq bottom$

end if

for $substitute\\_question$ in $substitute\\_questions$ do

if model responses have no word in $L_{refuse}$ and
$seman\\_simi(substitute\\_question)\geq bottom$ then

append $seman\\_simi(substitute\\_question)$ to $f[seman\\_simi]$

append $substitute\\_question$ to $f[substitute\\_questions]$

if $len(f[seman\\_simi])$ = 1 and $f[seman\\_simi][0]>top$ then

$top\leftarrow f[seman\\_simi][0]$

$bottom\leftarrow$ max$(top-selection\\_similarity\\_region,0)$

$best\\_solution\leftarrow[substitute\\_question,seman\\_simi(substitute\\_question)]$

end if

else if $seman\\_simi(substitute\\_question)<bottom$ then

break

end if

if $len(f[substitute\\_questions])$ = 0 and not $First$ then

$stop\leftarrow True$

end if

end for

end if

Algorithm 5 Crossover

Input: $harmful\\_question,substitute\\_questions,stop,first$

Output: $crossovers$

$Q\leftarrow[]$

if not $stop$ then

$crossovers\leftarrow[]$

for $substitute\\_question$ in $substitute\\_questions$ do

if $first$ then

$crossover\leftarrow$ randomly choose one syntactic form to generate one
$crossover$ for $substitute\\_question$

append $crossover$ to $crossovers$

else

$crossover\leftarrow$ choose all ten syntactic forms to generate ten
$crossover$ for $substitute\\_question$

if $crossover$ not in $crossovers$ then

append $crossover$ to $crossovers$

end if

end if

end for

end if

Algorithm 6 Selection

Input: $offspring,population\\_size$

Output: $selected\\_offspring$

if $len(offspring)\leq population\\_size$/10 then

$offspring\leftarrow$ $offspring$

else

$total\\_score\leftarrow sum(offspring[seman\\_simi])$

$probability\\_distribution\leftarrow[score/total\\_score$ for $score$ in
$offspring[seman\\_simi]]$

select $offspring\\_indices$ of size $population\\_size$/10 according to
$probability\\_distribution$

$selected\\_offspring\leftarrow$ [$offspring[indice]$ for $indice$ in
$offspring\\_indices$]

end if