#  Adapting Large Language Models for Content Moderation: Pitfalls in Data
Engineering and Supervised Fine-tuning

Huan Ma1,2, Changqing Zhang122footnotemark: 2, Huazhu Fu3, Peilin Zhao2,
Bingzhe Wu2
1 College of Intelligence and Computing, Tianjin University, Tianjin, China
2 AI Lab, Tencent, Shenzhen, China
3 Institute of High Performance Computing, A*STAR, Singapore
1<EMAIL_ADDRESS>2<EMAIL_ADDRESS>The project was conducted
during the internship in AI Lab, TencentCorresponding author

###### Abstract

Nowadays, billions of people engage in communication and express their
opinions on the internet daily. Unfortunately, not all of these expressions
are friendly or compliant, making content moderation an indispensable task.
With the successful development of Large Language Models (LLMs) in recent
years, LLM-based methods have become a feasible solution for handling tasks in
various domains. However, in the field of content moderation, there is still a
lack of detailed work that systematically introduces implementation details.
In this paper, we introduce how to fine-tune an LLM model that can be
privately deployed for content moderation. Specifically, we discuss whether
incorporating reasons during the fine-tuning process would be better or if it
should be treated as a classification task directly. We also explore the
benefits of utilizing reasons generated by more powerful LLMs for fine-tuning
privately deployed models and the impact of different processing approaches
when the answers generated by the more powerful LLMs are incorrect. We report
the entire research process and the key findings in this paper, hoping to
provide valuable experience for researchers who are fine-tuning privately
deployed models in their domain-specific research.

##  1 Introduction

Content moderation is an indispensable topic in the realm of modern internet
(Dinan et al., 2019; Jahan & Oussalah, 2023). Over the past decade, the field
of internet content moderation has undergone a profound transformation,
evolving from its early stages rooted in manual rule-based architectures
(Kebriaei et al., 2023) to recent advancements characterized by deep learning-
based fully automated moderation algorithms (Lilian Weng, 2023). The progress
in deep learning technology has significantly enhanced the efficiency and
precision of content moderation. Current automated content moderation
architectures typically adhere to a discriminative paradigm (Deng et al.,
2022; Husain & Uzuner, 2021). This paradigm involves the manual annotation of
extensive, high-quality datasets, followed by supervised training of a
discriminative model for content moderation. For a recent example, OpenAI
details how they construct a holistic solution for real-world content
moderation scenarios based on this paradigm, covering data preparation, model
training, and deployment (Markov et al., 2023). While this paradigm has
achieved significant success in various tasks such as text and multimodal
moderation, and has helped businesses reduce substantial costs, it also
exhibits several limitations: (1) Heavy Reliance on Data Annotation Quality
(2) Limited Robustness to Out-of-Distribution Data in Open Environments (3)
Lack of Interpretability.

These limitations become more pronounced when dealing with emerging scenarios
like chatbots (Bhardwaj & Poria, 2023; Yu et al., 2023). For instance, in
chatbot scenarios, there may be malicious users who attempt to manipulate the
system by constructing inputs that bypass content moderation systems using
techniques such as code-switching between multiple languages, using emojis, or
employing variant characters. Traditional supervised discriminative approaches
are clearly inadequate in addressing the dynamic and adversarial nature of
these samples through data or optimization efforts alone (Manakul et al.,
2023). On the other hand, recent successes of generative large language models
(LLMs) in various complex reasoning and planning tasks naturally lead us to
consider whether we can leverage the comprehensive grounding knowledge, strong
language understanding, and logical reasoning capabilities inherent in LLMs to
enhance the robustness and interpretability of content moderation tasks. The
answer is clearly affirmative, and we have already witnessed some initial
attempts and explorations. An interesting case in point is OpenAI’s use of
GPT-4 to optimize existing content moderation rating systems interactively
(Lilian Weng, 2023), aiming to achieve optimal moderation performance. We have
also made our own efforts by designing prompts to enable GPT-4 to perform
automatic content moderation on real-world data (As Fig. 1 shows). It is
evident that GPT-4 not only produces reasonable results but can also provide a
complete auditing process. However, for most companies and organizations,
reliance on third-party GPT-4 models still presents significant limitations:
(1) Compliance Requirements: There are regulatory and compliance requirements
that may constrain the use of third-party models for content moderation, (2)
Cost Considerations, and (3) Domain-Specific Knowledge Injection. Addressing
these limitations and developing custom content moderation solutions that meet
specific compliance, cost, and domain requirements remains an ongoing area of
research and development in the field.

Figure 1: Content Moderation with Auditing Processes. GPT-4 can provide
complete auditing processes (left), but sometimes it presents limitations
(right).

One viable technical approach involves constructing labeled samples with an
associated moderation process (moderation reasoning chain) and then performing
supervised fine-tuning (SFT) on open-source large language models to align
them with the specific requirements of a given content moderation task. Over
the past year, this approach has demonstrated success in various tasks beyond
content moderation, such as solving mathematical problems and logical
reasoning (Gao et al., 2023; Luo et al., 2023a; b). Researchers have proposed
a variety of techniques to improve the ultimate performance of this approach,
including but not limited to data engineering, audit chain reliability
assessment, optimization objectives, and more (Gerrard, 2018; Morrow et al.,
2022; Singhal et al., 2023). The effectiveness of these techniques in real-
world content moderation scenarios still requires validation. A relevant work
in this context comes from AWS (James et al., 2023), where they attempt to
fine-tune large models directly using label information (without the audit
reasoning chain). However, in our practical experience, we find that this
approach exhibits poor out-of-distribution robustness and lacks interpretable
moderation process outputs.

By adapting LLMs to the content moderation, the ideal case is to build a model
that is significantly more robust than previous discriminative models simply
by applying supervised fine-tuning (SFT) to LLMs (Hajibabaee et al., 2022;
Rizwan et al., 2020). However, the practical implementation of such a
straightforward idea is fraught with numerous design challenges. Various
design choices, including data engineering and training methodologies, have a
profound impact on the final model’s performance. In this paper, we aim to
take the first step by conducting large-scale experiments to assess the
feasibility of applying large language models to content moderation scenarios
and meticulously analyze the effectiveness of various commonly used techniques
throughout the entire pipeline. Specifically, at the data engineering level,
we adapt a weak supervision paradigm to reduce the cost of manual annotation.
This involves expert annotation of the final labels for samples, with GPT-3.5
& GPT-4 handling the more labor-intensive moderation reasoning chain. In
contrast to many existing works that adapt fully unsupervised paradigms (where
all data is generated by GPT-3.5 & GPT-4), our approach allows us to leverage
human-generated golden labels for reflection on the reasoning chain (see the
methods section for details). Additionally, we incorporate conventional
optimization strategies such as data deduplication (He et al., 2010). During
the training process, we observe common hallucination issues in GPT models
when generating the moderation reasoning chain for production. These
hallucinations can lead to erroneous labels. Here, we leverage weak
supervision information to correct the entire process. Applying this entire
pipeline to real-world data, we have arrived at several key findings: (1)
Incorporating reasoning processes during the fine-tuning will enhance the
robustness of the model and effectively overcome overfitting. In addition to
the labels, reasoning process is necessary for fine-tuning. (2) Introducing
weak supervision can effectively filter out samples with poor quality in
reasoning processes, improve the quality of the fine-tuning data, and enhance
the performance of the fine-tuned model. (3) The capabilities of the models
limit their fine-tuned performance. For example, replacing GPT-3.5’s auditing
processes with GPT-4’s more detailed auditing processes, the improvement in
performance is slight for models with weaker capabilities compared to models
with stronger capabilities.

Figure 2: Framework of fine-tuing privately deployed content moderation model.
Fine-tuning LLMs with the pattern $(x_{i},y_{i})$ can lead to severe
overfitting. For instance, sentences that contain regional characteristics
will be classified into Discrimination or Insult. Incorporating an auditing
process into the fine-tuning stage can effectively overcome this issue.

##  2 Content Moderation System Based on LLMs

###  2.1 Notations

Given the raw real-world dataset
$\mathcal{D}_{raw}=\left\\{x_{i},y_{i}\right\\}_{i=1}^{N}$, where $x_{i}$ is
the input sentence of the $i$-th sample, and $y_{i}\in\\{1,\cdots,K\\}$ is the
corresponding _weak supervision_ label, which only consists of the labels
annotated by humans, without including the complete reasoning processes. Then
we augment the raw training dataset by incorporating a reasoning process,
yielding $\mathcal{D}=\left\\{x_{i},{r}_{i},y_{i}\right\\}_{i=1}^{N}$, where
$r_{i}$ is the reason behind the classification of $x_{i}$ as $y_{i}$.
However, obtaining the exact $r_{i}$ is inherently challenging, so we
approximate it through LLMs. Specifically, given an input $x_{i}$, we prompt
cloud LLM services to generate the analysis process response $\hat{r}_{i}$ and
corresponding predicted label $\hat{y}_{i}$, and we denote the augmented
training dataset as
$\mathcal{D}_{train}=\left\\{x_{i},\hat{r}_{i},\hat{y}_{i}\right\\}_{i=1}^{N}$.

###  2.2 Data Deduplication

Previous research has demonstrated that repetitive data lacking diversity does
not contribute to the improvement of training models. What models truly
require are carefully selected data samples that exhibit diversity (Liang et
al., 2023). Eliminating highly similar data not only reduces the time required
for model training but also minimizes the inefficiency caused by redundant
reasoning processes. Consequently, we conduct deduplication on the collected
real-world dataset to eliminate repetitive data that does not benefit the
training of the model. Specifically, we utilize a pretrained text encoder to
obtain embeddings for each sentence (Liu et al., 2021). Subsequently, we
perform clustering on the data distribution within each category and select
one sample from each cluster, aiming to obtain samples with distinct semantic
characteristics. Following this approach, we curate a golden set consisting of
145 examples. To ensure the quality of this selected data, we conduct manual
label recheck on the golden set.

###  2.3 Fine-tuning LLMs without CoT

Directly employing the cloud LLM service API (OpenAI, 2023) for content
moderation is not feasible, particularly when handling a substantial volume of
private data in numerous content moderation scenarios. In such cases,
utilizing a privately deployed LLM for content moderation becomes essential to
accomplish the task. However, the present performance of open-source pre-
trained models (Du et al., 2022; Zeng et al., 2022) is inadequate in domain-
specific tasks, such as content moderation. This necessitates alignment fine-
tuning for vertical domain tasks.

One of the simplest approaches is to perform SFT on open-source LLMs with the
raw training set $\mathcal{D}_{raw}=\left\\{x_{i},y_{i}\right\\}_{i=1}^{N}$,
and the model is only required to predict $y_{i}$ given $x_{i}$, and we get
the private classifier $\mathcal{M}_{p}$.

However, a major drawback of this method is its high susceptibility to
catastrophic overfitting on limited data and large-scale networks, and the
experiments results are shown in Sec. 3.2.2. To address this issue, we
introduce fine-tuning with CoT inspired by previous research in mathematical
reasoning (Luo et al., 2023a; Gao et al., 2023).

###  2.4 Fine-tuning LLM with CoT

In this section, we present the fine-tuning of LLM with CoT to incorporate
reasoning processes during training. However, acquiring the reasoning $r$ is
challenging, while obtaining input $x$ and label $y$ in real-world data is
comparatively easier. Conventional methods for procuring reasoning involve
manual annotation, which introduces two issues: (1) the high cost and slow
pace of human annotation, making it difficult to update training data
continuously, and (2) inconsistencies in annotation standards and reasoning
logic among different individuals, making it challenging to establish a
standardized and consistent reasoning process.

The emergence of cloud LLM services (denoted as $\mathcal{M}_{s}$) addresses
the above issues. First, although the cloud LLM services provided by large
model service providers may not be directly deployed in actual businesses,
they can still generate some of reasoning data with a few API calls. Second,
the judgment criteria of LLMs are unified, ensuring consistency in reasoning
processes across different samples and maintaining the standardization of
these processes. We can take advantage of cloud LLM services to generate
reasoning processes for augmenting our training set. Subsequently, using the
training set with reasons, we can fine-tune the open-source LLMs for specific
domain tasks, thereby enhancing the capabilities of privately deployed models
($\mathcal{M}_{p}$) for practical real-world tasks.

Weak supervision. To obtain the reasoning process why a training sample
$x_{i}$ is classified as class $y_{i}$, a natural approach is to prompt the
$\mathcal{M}_{s}$ with sentence $x_{i}$ and its corresponding label $y$, and
then require the $\mathcal{M}_{s}$ to analyse the reasoning process
$\hat{r}_{i}$. However, previous studies have highlighted the issue of
hallucinations in LLMs (Manakul et al., 2023; Bang et al., 2023). If
$\mathcal{M}_{s}$ is given access to the true label before analyse, its
response may be influenced by hallucinations. This would significantly
compromise the quality of the data for fine-tuning the model. Consequently, we
propose a weak supervision approach in which $\mathcal{M}_{s}$ is not provided
with the true label $y_{i}$ during its reasoning process. Instead, it
generates predicted labels $\hat{y}_{i}$ based on its reasoning process, and
the true labels from the training set are only used to assess whether the
judgments of $\mathcal{M}_{s}$ are correct.

Specifically, we can achieve this through the following steps:

  1. 1.

Determine the instruction that includes the reasoning process based on the
actual task and training dataset
$\mathcal{D}_{raw}=\left\\{x_{i},y_{i}\right\\}_{i=1}^{N}$.

  2. 2.

Prompt the instruction and training data $x_{i}$ to a remote model service
$\mathcal{M}_{s}$ to obtain the reasoning process $\hat{r_{i}}$ and predicted
label $\hat{y_{i}}$.

  3. 3.

Fine-tune the open-source model using the training set
$\mathcal{D}_{train}=\left\\{x_{i},\hat{r}_{i},\hat{y}_{i}\right\\}_{i=1}^{N}$
that includes reasoning to get privately deployed model $\mathcal{M}_{p}$.

However, although the model capabilities offered by LLM service providers
surpass those of open-source models, these cloud model services might
incorrectly classify the training data, which makes it difficult to acquire
accurate reasons for training privately deployed models. To address this
issue, we propose various strategies to clean the dataset when an incorrect
response is obtained. Specifically, we select samples in which the
$\mathcal{M}_{s}$ predicts inaccurately by examining the consistency between
the predicted results and the ground truth. For samples with prediction
errors, that is, samples where $\hat{y}\neq y$, we have two distinct
strategies for managing them.

(a) Removing samples that are misclassified by $\mathcal{M}_{s}$, which is a
simple and natural solution. The specific process involves inserting the
following step between the step 2 and step 3:

  1. 2.a.

If $\hat{y}_{i}\neq y_{i}$, remove the sample
$<x_{i},\hat{r}_{i},\hat{y}_{i}>$ from the training dataset
$\mathcal{D}_{train}$.

(b) Introducing self-recheck to rectify the model’s outputs. The specific
process entails incorporating the subsequent step between steps 2 and 3:

  1. 2.b.

If $\hat{y}_{i}\neq y_{i}$, prompt $\mathcal{M}_{s}$ to recheck the response
with the original question and the incorrect reason $\hat{r}_{i}$ and
predicted label $\hat{y}_{i}$, then update the revised reason $\hat{r}_{i}$
and the prediction $\hat{y}_{i}$.

Though the self-recheck process can be repeated multiple times to obtain the
correct prediction result, we perform the reflection once during the
experiment. If $\mathcal{M}_{s}$ persists in making errors even after re-
answering, it indicates the question is too hard for $\mathcal{M}_{s}$, and
compelling it to change the response might result in lower-quality answers due
to hallucinations.

##  3 Experiments

###  3.1 Experiments Setting

Table 1: Results on the different models fine-tuned with different strategies,
the best-performing F1 Score for each harm type has been highlighted in bold
for each model. “Average” indicates the micro-average performance on in-
distribution.

Model | Strategies | Metrics | In-distribution | OOD
---|---|---|---|---
Porno | Violent | Offen. | Gamb. | Harml. | Average | COLD | CInsult
GPT-3.5 | - | Recall | 80.0 | 82.7 | 92.0 | 97.3 | 59.9 | 73.0 | 63.5 | 78.3
Precision | 95.2 | 80.5 | 83.6 | 96.1 | 91.8 | 89.3 | 73.0 | 83.5
F1 Score | 87.0 | 81.6 | 87.6 | 96.7 | 72.5 | 80.3 | 60.3 | 79.0
ChatGLM2 (6B) | Original | Recall | 44.0 | 64.0 | 57.3 | 76.0 | 63.6 | 61.9 | 57.5 | 54.1
Precision | 17.9 | 23.6 | 35.4 | 27.9 | 73.6 | 41.7 | 64.0 | 70.9
F1 Score | 25.5 | 34.5 | 43.8 | 40.9 | 68.3 | 49.9 | 57.9 | 60.8
Setting A | Recall | 68.0 | 81.3 | 74.0 | 84.0 | 63.4 | 69.2 | 65.3 | 61.5
Precision | 89.5 | 70.9 | 81.6 | 96.9 | 85.4 | 84.2 | 73.7 | 76.4
F1 Score | 77.3 | 75.8 | 77.6 | 90.0 | 72.8 | 76.0 | 67.7 | 67.9
Setting B | Recall | 73.3 | 82.7 | 80.0 | 80.0 | 76.7 | 77.8 | 67.7 | 66.4
Precision | 84.6 | 67.4 | 79.5 | 88.2 | 87.6 | 83.4 | 72.4 | 76.4
F1 Score | 78.6 | 74.3 | 79.7 | 83.9 | 81.8 | 80.5 | 68.5 | 70.4
Setting C | Recall | 66.7 | 78.7 | 79.3 | 82.7 | 78.7 | 78.1 | 68.0 | 64.5
Precision | 87.7 | 70.2 | 79.3 | 93.9 | 85.3 | 83.4 | 73.3 | 76.9
F1 Score | 75.8 | 74.2 | 79.3 | 87.9 | 81.9 | 80.7 | 68.8 | 69.7
Setting D | Recall | 66.7 | 82.7 | 80.0 | 73.3 | 84.7 | 81.0 | 67.6 | 65.6
Precision | 94.3 | 74.7 | 85.7 | 96.5 | 85.8 | 86.0 | 72.7 | 75.9
F1 Score | 78.1 | 78.5 | 82.8 | 83.3 | 85.3 | 83.4 | 68.5 | 69.5
Baichuan (13B) | Original | Recall | 74.7 | 44.0 | 48.0 | 54.7 | 59.2 | 56.8 | 41.6 | 47.5
Precision | 47.5 | 35.9 | 55.4 | 48.2 | 79.0 | 61.5 | 70.0 | 77.4
F1 Score | 58.0 | 39.5 | 51.4 | 51.2 | 67.7 | 59.0 | 51.9 | 58.9
Setting A | Recall | 70.7 | 86.7 | 85.3 | 98.7 | 45.2 | 63.4 | 65.7 | 76.2
Precision | 86.9 | 63.7 | 81.5 | 84.1 | 85.7 | 81.1 | 72.1 | 83.7
F1 Score | 77.9 | 73.4 | 83.4 | 90.8 | 59.2 | 71.2 | 66.3 | 79.8
Setting B | Recall | 78.7 | 85.3 | 80.7 | 96.0 | 82.9 | 83.5 | 66.3 | 72.5
Precision | 89.4 | 63.4 | 85.8 | 92.3 | 90.3 | 86.2 | 73.8 | 83.0
F1 Score | 83.7 | 72.7 | 83.2 | 94.1 | 86.5 | 84.9 | 69.0 | 77.4
Setting C | Recall | 69.3 | 89.3 | 79.3 | 98.7 | 76.1 | 79.3 | 69.0 | 77.2
Precision | 77.6 | 64.4 | 77.3 | 82.2 | 87.7 | 81.3 | 71.3 | 82.0
F1 Score | 73.2 | 74.9 | 78.3 | 89.7 | 81.5 | 80.3 | 69.0 | 79.5
Setting D | Recall | 77.3 | 86.7 | 80.7 | 88.0 | 80.0 | 81.2 | 68.1 | 73.6
Precision | 81.7 | 60.2 | 80.7 | 80.5 | 88.0 | 81.7 | 72.2 | 79.8
F1 Score | 79.5 | 71.0 | 80.7 | 84.1 | 83.9 | 81.5 | 69.0 | 76.4

Dataset: The business scenario in the experiment focuses on content moderation
in a Chinese environment. We classify the content into 5 categories:
Pornography (Porno), Violence, Discrimination or Insult (Offensive), Gambling,
and Harmless. The data originates from our real-world business scenarios and
has been annotated with labels $y$ by human annotators. To validate the
models’ business performance and generalization ability, we evaluate their
performance on both in-distribution data and out-of-distribution (OOD) data.
Since the primary business scenario in this experiment involves the Chinese
context, we utilize the open benchmark dataset COLD (Deng et al., 2022) and
the Chinese insult dataset CInsult111https://github.com/wjx-
git/IllegalTextDetection as the OOD data. (Details please refer to Appendix
E). Model: For open-source base models, we choose ChatGLM2-6B (Du et al.,
2022; Zeng et al., 2022) and Baichuan-13B-Chat as the base models, because
they have been reported good performance in basic Chinese task and demonstrate
a more prominent understanding of the Chinese language context compared to
LLaMA (Touvron et al., 2023). Although some researches align LLaMA for Chinese
222https://github.com/ymcui/Chinese-LLaMA-Alpaca
333https://huggingface.co/shibing624/chinese-llama-plus-13b-hf, they lack
official formal evaluations and guarantees, making them unsuitable as base
models. Fine-tuning: Fine-tuning the model can be done using methods including
full-parameter-tuning, P-tuning (Liu et al., 2022), or LORA (Hu et al., 2021).
Parameter-Efficient Fine-Tuning (PEFT) has been proven effective in preventing
overfitting in models, and several studies have demonstrated that PEFT
outperforms full-parameter-tuning (Zeng et al., 2023; Mangrulkar et al.,
2022). In the officially released code for fine-tuning
ChatGLM2444https://github.com/THUDM/ChatGLM2-6B/tree/main/ptuning, parameter-
effective P-tuning-V2 is recommended. On the other hand, tuning the Baichuan-
Chat based on LORA is among the widely recommended PEFT methods. Therefore, we
chose P-tuning-V2 for fine-tuning the ChatGLM and fine-tuning the Baichuan-
Chat based on LORA.

####  3.1.1 Different training strategies

We apply various fine-tuning strategies, such as reject strategy, recheck
strategy, and a combination of both strategies, and the prompt templates
please refer to Appendix A.

GPT-3.5: Ideally, we hope the privately deployed model to achieve competitive
performance comparable to the cloud LLM service. To evaluate this, we test the
performance of GPT-3.5-turbo using the same instructions, which involves
conducting content analysis before making a judgment. _Please note that
GPT-3.5 has a content management policy. Some content is filtered by content
filtering policies, and in such cases, we consider it as a successful
detection by GPT-3.5 if the responses of positive samples are filtered._

Original: We evaluate the ChatGLM2-6B and Baichuan-13B-Chat models without any
fine-tuning for content moderation as the baseline, to offer an intuitive
comparison of the fine-tuning effects achieved through various strategies.

Setting A: We generate CoT $\hat{r}$ and predicted class $\hat{y}$ using
GPT-3.5 on the training set. Then, we utilize
$\mathcal{D}_{train}=\left\\{x_{i},\hat{r}_{i},\hat{y}_{i}\right\\}_{i=1}^{N}$
to fine-tune ChatGLM2-6B and Baichuan-13B-Chat.

Setting B: Taking into account that the answers generated by GPT-3.5 may be
incorrect, we check GPT-3.5’s responses by the ground truth. We enhance the
quality of the training data by discarding samples in which GPT-3.5 made
inaccurate predictions (i.e., $\hat{y}\neq y$), as demonstrated in step 2.a.

Setting C: In addition to discarding samples with incorrect predictions, we
instruct GPT-3.5 to perform self-rechecks and modify its responses for these
samples as shown in step 2.b.

Setting D: This strategy combines the recheck and reject approaches. Firstly,
the model reflects on and modifies incorrect answers. Subsequently, samples
that persist as incorrect after re-answering are removed from the training
dataset.

###  3.2 Results

(a) ChatGLM2 (6B) (b) Baichuan (13B)

Figure 3: Task Complexity for models with different capability. “Reasoning by GPT-3.5” indicates the reasoning processes ($r$) in train dataset is generated form GPT-3.5, and “GPT-3.5 with more data” indicates the model is fine-tuned with LLM-expanded training samples ($x$). Table 2: F1 Score of models fine-tuned with GPT-3.5 & GPT-4-generated reasoning processes. Base Models | $\mathcal{M}_{s}$ | Porno | Violent | Offensive | Gambling | Harmless | Average
---|---|---|---|---|---|---|---
ChatGLM2 (6B) | GPT-3.5 | 77.3 | 75.8 | 77.6 | 90.0 | 72.8 | 76.0
GPT-4 | 81.8 | 68.3 | 72.8 | 79.7 | 81.9 | 79.1
Baichuan (13B) | GPT-3.5 | 77.9 | 73.4 | 83.4 | 90.8 | 59.2 | 71.2
GPT-4 | 70.9 | 74.5 | 79.0 | 93.5 | 87.3 | 83.9

We evaluate the performance of all models, including the original model, and
the results are displayed in Table 1. We also highlight the best-performing
models in terms of the F1 Score metric of each model (F1 Score is a good
indicator of overall performance, we only emphasize this metric to make the
table more readable). From the Table 1, it can be observed that although these
models perform poorly before fine-tuning, all fine-tuned models are able to
achieve performance comparable to GPT-3.5. Particularly in the category of
harmless, all fine-tuned models significantly surpass GPT-3.5’s performance.

####  3.2.1 Design choice of data engineering

Expanding training sentences ($x$) by LLM does not benefit model fine-tuning.
Previous works generate $x$ for fine-tuning by LLMs to expand the training set
(Luo et al., 2023a), but in the context of content moderation, the $x$
generated form LLMs does not significantly help with fine-tuning the model. In
our experiments, we initially selected 145 high-quality data samples as golden
set. Then, we expand it with over 1000 new training samples generated by
GPT-3.5. From the generated data, we select an additional 290 samples with the
highest diversity (using sentence embedding clustering, details please refer
to Appendix B) to expand the training data. As shown in Table 3, we find that
augmenting the training set results in several times more training costs, but
it does not provide significant benefits to fine-tuning the model. For
ChatGLM2-6B, expanding samples make it more difficult for the model to
decrease its loss to the same level as on the Golden set (Fig. 3), and
expended training data doesn’t improve the model’s performance. As for
Baichuan-13B, although the model’s training loss can decrease to the same
level as on the Golden set, expanded examples improve the performance very
slightly.

Table 3: Fine-tuning without & with LLM-generated training data.

Models | Porno | Violent | Offensive | Gambling | Harmless | Average
---|---|---|---|---|---|---
ChatGLM2 | 77.3 vs 82.4 | 75.8 vs 77.4 | 77.6 vs 77.8 | 90.0 vs 93.2 | 72.8 vs 65.3 | 76.0 vs 73.3
Baichuan | 77.9 vs 81.4 | 73.4 vs 77.3 | 83.4 vs 75.2 | 90.8 vs 91.2 | 59.2 vs 65.3 | 71.2 vs 72.8

Table 4: The comparison of training data augmented by GPT-3.5 and GPT-4. Reason from | Length | Macro F1
---|---|---
GPT-3.5 | 188 | 72.9
GPT-4 | 210 | 79.3

Obtain reasoning processes ($\hat{r}$) form different cloud LLM services. We
utilize both GPT-4 and GPT-3.5 to generate reasoning processes on the same
training dataset for fine-tuning. As expected (Table 4), the auditing
processes generated by GPT-4 were more detailed (Length) and accurate (Macro
F1). As shown in Fig. 3, it is difficult for the model with weak capability
(ChatGLM2-6B) to fit the detailed reasoning processes. We find that the
training loss of the model could not decrease to the level achieved by fine-
tuning with the reasoning generated by GPT-3.5. Consequently, the performance
improvement on the test dataset of ChatGLM2 is also slight. On the other hand,
from Fig. 3, we can find the training loss of Baichuan-13B quickly decreases
to the same level comparable to the fine-tuning loss achieved with the
reasoning generated by GPT-3.5, and the larger model, Baichuan-13B, shows a
significant improvement when fine-tuned with the reasoning processes generated
by GPT-4.

####  3.2.2 Design choice of SFT

Figure 4: Recall on OOD datasets.

Fine-tuning LLMs with the pattern $(x_{i},y_{i})$ can lead to severe
overfitting. As shown in Fig. 4, after fine-tuning without incorporating
reasons, the model’s performance significantly deteriorates on out-of-
distribution (OOD) data. Specifically, Fig. 4 displays the recall rates on
negative samples of different models on binary classification OOD benchmark
data. “COLD-ChatGLM” indicates the recall on the COLD dataset (Deng et al.,
2022) using the ChatGLM model, while ”ORI” and “w/o CoT” denote the original
model without fine-tuning and the model fine-tuned without incorporating
reasons, respectively. In other words, the model fine-tuned without
incorporating reasons exhibits severe overfitting (details please refer to
Appendix D), making it unsuitable for deployment in practical business
scenarios.

Weak supervision can improve the fine-tuning performance. Compared to the
unsupervised setting (Setting A), we introduce weak supervision methods to
improve the quality of reasoning processes in the training set. Specifically,
we find that (1) filtering low-quality reasoning processes through weak
supervision can help enhance fine-tuning performance. As shown in Table 1
(Setting B), the model’s performance significantly improves after filtering
out low-quality samples. (2) Rechecking low-quality reasoning processes
through weak supervision can also enhance the quality of the fine-tuning data.
As shown in Table 1 (Setting C), the model’s performance improves by
regenerating reasoning processes for low-quality samples. Moreover, these two
strategies are not conflicting. When both strategies are combined (Setting D),
the model achieves similar or even better results than the best performance of
either strategy alone (The results of models fine-tuned with reasoning
processes generated form GPT-4 please refer to Appendix F).

##  4 Related Work

Content Moderation. Content moderation plays a vital role in ensuring the
security of online content, garnering widespread attention in recent years.
This attention encompasses both academic and industry product research.

For researches on content moderation, numerous typical tasks have been
proposed, accompanied by impressive performance reports. $\bullet$
Discrimination-based methods: OpenAI introduces a discrimination-based method
employing active learning and adversarial training strategies (Markov et al.,
2023). They emphasize the importance of meticulous data selection, the
necessity of active learning, and the multitude of challenges arising from
data imbalance. $\bullet$ LLM-based Methods (Li et al., 2023a; Franco et al.,
2023; Mullick et al., 2023): By utilizing the ChatGPT API, assuming ChatGPT as
a content moderator, conducting content auditing in a conversational format.
There are also many specific methods in the practical deployment of content
moderation in the industry. $\bullet$ Expert Rules (Kebriaei et al., 2023):
Aggressive language filtering in the maintenance of social platform security
often relies on keyword matching, leveraging prior expert knowledge to pre-
define expert rules and facilitate content moderation. $\bullet$ Policy
Refinement (Lilian Weng, 2023): Policy experts draft content policy and assign
labels to a small number of examples based on the policy to create a golden
dataset. Subsequently, GPT-4 is instructed to read the content policy, assign
labels to the same dataset, and scrutinize the discrepancies between GPT-4’s
judgments and human judgments. This process involves analyzing the ambiguity
in policy definitions and providing further policy clarification accordingly,
and iterations of this process continue until the desired policy quality is
achieved.

Fine-tuning LLM with Chain of Thought. The Chain of Thought (CoT) has been
introduced as a viable method for achieving interpretability in language
models (Wei et al., 2022; Wang et al., 2022). A crucial criterion for
interpretability methods is their ability to accurately capture the cognitive
processes occurring within the model.

CoT effectively enhances both the accuracy and interpretability of the model
by employing logical analysis and reasoning (Suzgun et al., 2022). Recent
studies integrate inference processes into the fine-tuning models, aiming to
move beyond solely fitting the final answer, especially in math reasoning.
$\bullet$ Process Supervision: Uesato et al. (Uesato et al., 2022) conduct the
initial comprehensive comparison between process-based and outcome-based
supervisions implemented on a natural language task. A recent study conducted
by OpenAI (Lightman et al., 2023) examines the distinction between outcome-
supervised reward models (ORMs) and process-supervised reward models (PRMs).
ORMs rely solely on the final result of the model’s response, whereas PRMs
receive feedback at each step of the chain-of-thought. The findings
demonstrate that process supervision is capable of training significantly more
dependable reward models compared to outcome supervision. $\bullet$ Fine-
tuning with CoT: Yuan et al. (Yuan et al., 2023) generate CoT-augmented
samples without any trained process-level reward models and outperform the
supervised fine-tuning (SFT) accuracy. Hsieh et al. (2023) introduce
Distilling step-by-step to train smaller models with less training data, which
reduces the amount of training data required for fine-tuning and distillation
of LLMs into smaller model sizes.

Although attempts have been made to fine-tune large models in many fields,
including financial (Yu, 2023), legal (Huang et al., 2023), medical (Xiong et
al., 2023), educational (Dan et al., 2023), astronomical (Li et al., 2023b),
there hasn’t been a specific work that provides a detailed explanation for
content moderation.

##  5 Conclusion

In this paper, we conduct a fine-tuning attempt to privately deploy LLM with
content moderation as a specific scenario. We enhance the model’s
generalization ability by introducing reasoning process during the fine-tuning
process, resulting in the privately deployed model achieving performance that
is not weaker than that of GPT-3.5 on content moderation. Additionally, we
introduce weak supervision to improve the reasoning quality of the training
dataset, which improve the fine-tuning performance at a small annotation cost.
We also discuss the different design choice of data engineering and SFT.
Overall, we believe the research offers valuable insights for researchers in
various fields to fine-tune privately deployed models in their domains.

## References

  * Bang et al. (2023) Yejin Bang, Samuel Cahyawijaya, Nayeon Lee, Wenliang Dai, Dan Su, Bryan Wilie, Holy Lovenia, Ziwei Ji, Tiezheng Yu, Willy Chung, et al.  A multitask, multilingual, multimodal evaluation of chatgpt on reasoning, hallucination, and interactivity.  _arXiv preprint arXiv:2302.04023_ , 2023.
  * Bhardwaj & Poria (2023) Rishabh Bhardwaj and Soujanya Poria.  Red-teaming large language models using chain of utterances for safety-alignment.  _arXiv preprint arXiv:2308.09662_ , 2023.
  * Dan et al. (2023) Yuhao Dan, Zhikai Lei, Gu Yiyang, Yong Li, Jianghao Yin, Jiaju Lin, Linhao Ye, Zhiyan Tie, Yougen Zhou, Yilei Wang, Aimin Zhou, Ze Zhou, Qin Chen, Jie Zhou, Liang He, and Xipeng Qiu.  Educhat: A large-scale language model-based chatbot system for intelligent education.  _arXiv preprint arXiv:2308.02773_ , 2023.
  * Deng et al. (2022) Jiawen Deng, Jingyan Zhou, Hao Sun, Chujie Zheng, Fei Mi, Helen Meng, and Minlie Huang.  Cold: A benchmark for chinese offensive language detection.  _arXiv preprint arXiv:2201.06025_ , 2022.
  * Dinan et al. (2019) Emily Dinan, Samuel Humeau, Bharath Chintagunta, and Jason Weston.  Build it break it fix it for dialogue safety: Robustness from adversarial human attack.  _arXiv preprint arXiv:1908.06083_ , 2019.
  * Du et al. (2022) Zhengxiao Du, Yujie Qian, Xiao Liu, Ming Ding, Jiezhong Qiu, Zhilin Yang, and Jie Tang.  Glm: General language model pretraining with autoregressive blank infilling.  In _Proceedings of the 60th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers)_ , pp. 320–335, 2022.
  * Franco et al. (2023) Mirko Franco, Ombretta Gaggi, and Claudio E Palazzi.  Analyzing the use of large language models for content moderation with chatgpt examples.  In _3rd International Workshop on Open Challenges in Online Social Networks_ , pp. 1–8, 2023.
  * Gao et al. (2023) Luyu Gao, Aman Madaan, Shuyan Zhou, Uri Alon, Pengfei Liu, Yiming Yang, Jamie Callan, and Graham Neubig.  Pal: Program-aided language models.  In _International Conference on Machine Learning_ , pp. 10764–10799. PMLR, 2023.
  * Gerrard (2018) Ysabel Gerrard.  Beyond the hashtag: Circumventing content moderation on social media.  _New Media & Society_, 20(12):4492–4511, 2018\.
  * Hajibabaee et al. (2022) Parisa Hajibabaee, Masoud Malekzadeh, Mohsen Ahmadi, Maryam Heidari, Armin Esmaeilzadeh, Reyhaneh Abdolazimi, and H James Jr.  Offensive language detection on social media based on text classification.  In _2022 IEEE 12th Annual Computing and Communication Workshop and Conference (CCWC)_ , pp. 0092–0098. IEEE, 2022.
  * He et al. (2010) Qinlu He, Zhanhuai Li, and Xiao Zhang.  Data deduplication techniques.  In _2010 international conference on future information technology and management engineering_ , volume 1, pp. 430–433. IEEE, 2010.
  * Hsieh et al. (2023) Cheng-Yu Hsieh, Chun-Liang Li, Chih-Kuan Yeh, Hootan Nakhost, Yasuhisa Fujii, Alexander Ratner, Ranjay Krishna, Chen-Yu Lee, and Tomas Pfister.  Distilling step-by-step! outperforming larger language models with less training data and smaller model sizes.  _arXiv preprint arXiv:2305.02301_ , 2023.
  * Hu et al. (2021) Edward J Hu, Yelong Shen, Phillip Wallis, Zeyuan Allen-Zhu, Yuanzhi Li, Shean Wang, Lu Wang, and Weizhu Chen.  Lora: Low-rank adaptation of large language models.  _arXiv preprint arXiv:2106.09685_ , 2021.
  * Huang et al. (2023) Quzhe Huang, Mingxu Tao, Zhenwei An, Chen Zhang, Cong Jiang, Zhibin Chen, Zirui Wu, and Yansong Feng.  Lawyer llama technical report.  _ArXiv_ , abs/2305.15062, 2023.
  * Husain & Uzuner (2021) Fatemah Husain and Ozlem Uzuner.  A survey of offensive language detection for the arabic language.  _ACM Transactions on Asian and Low-Resource Language Information Processing (TALLIP)_ , 20(1):1–44, 2021.
  * Jahan & Oussalah (2023) Md Saroar Jahan and Mourad Oussalah.  A systematic review of hate speech automatic detection using natural language processing.  _Neurocomputing_ , pp. 126232, 2023.
  * James et al. (2023) Poquiz James, Man Han, and Tinaztepe Safa.  Aws performs fine-tuning on a large language model (llm) to classify toxic speech for a large gaming company.  https://aws.amazon.com/cn/blogs/machine-learning/, 2023.
  * Kebriaei et al. (2023) Emad Kebriaei, Ali Homayouni, Roghayeh Faraji, Armita Razavi, Azadeh Shakery, Heshaam Faili, and Yadollah Yaghoobzadeh.  Persian offensive language detection.  _Machine Learning_ , pp. 1–21, 2023.
  * Li et al. (2023a) Lingyao Li, Lizhou Fan, Shubham Atreja, and Libby Hemphill.  “hot” chatgpt: The promise of chatgpt in detecting and discriminating hateful, offensive, and toxic comments on social media.  _arXiv preprint arXiv:2304.10619_ , 2023a.
  * Li et al. (2023b) YuYang Li, CunShi Wang, MengWei Qu, Yu Bai, Roberto Soria, and JiFeng Liu.  Starglm.  https://github.com/Yu-Yang-Li/StarGLM, 2023b.
  * Liang et al. (2023) Chen Liang, Simiao Zuo, Qingru Zhang, Pengcheng He, Weizhu Chen, and Tuo Zhao.  Less is more: Task-aware layer-wise distillation for language model compression.  In _International Conference on Machine Learning_ , pp. 20852–20867. PMLR, 2023.
  * Lightman et al. (2023) Hunter Lightman, Vineet Kosaraju, Yura Burda, Harri Edwards, Bowen Baker, Teddy Lee, Jan Leike, John Schulman, and Ilya Sutskever.  Let’s verify step by step.  2023\.
  * Lilian Weng (2023) Andrea Vallone Lilian Weng, Vik Goel.  Using gpt-4 for content moderation.  2023\.
  * Liu et al. (2021) Jiachang Liu, Dinghan Shen, Yizhe Zhang, Bill Dolan, Lawrence Carin, and Weizhu Chen.  What makes good in-context examples for gpt-$3$?  _arXiv preprint arXiv:2101.06804_ , 2021.
  * Liu et al. (2022) Xiao Liu, Kaixuan Ji, Yicheng Fu, Weng Tam, Zhengxiao Du, Zhilin Yang, and Jie Tang.  P-tuning: Prompt tuning can be comparable to fine-tuning across scales and tasks.  In _Proceedings of the 60th Annual Meeting of the Association for Computational Linguistics (Volume 2: Short Papers)_ , pp. 61–68, 2022.
  * Luo et al. (2023a) Haipeng Luo, Qingfeng Sun, Can Xu, Pu Zhao, Jianguang Lou, Chongyang Tao, Xiubo Geng, Qingwei Lin, Shifeng Chen, and Dongmei Zhang.  Wizardmath: Empowering mathematical reasoning for large language models via reinforced evol-instruct.  _arXiv preprint arXiv:2308.09583_ , 2023a.
  * Luo et al. (2023b) Ziyang Luo, Can Xu, Pu Zhao, Qingfeng Sun, Xiubo Geng, Wenxiang Hu, Chongyang Tao, Jing Ma, Qingwei Lin, and Daxin Jiang.  Wizardcoder: Empowering code large language models with evol-instruct.  _arXiv preprint arXiv:2306.08568_ , 2023b.
  * Manakul et al. (2023) Potsawee Manakul, Adian Liusie, and Mark JF Gales.  Selfcheckgpt: Zero-resource black-box hallucination detection for generative large language models.  _arXiv preprint arXiv:2303.08896_ , 2023.
  * Mangrulkar et al. (2022) Sourab Mangrulkar, Sylvain Gugger, Lysandre Debut, Younes Belkada, and Sayak Paul.  Peft: State-of-the-art parameter-efficient fine-tuning methods.  https://github.com/huggingface/peft, 2022.
  * Markov et al. (2023) Todor Markov, Chong Zhang, Sandhini Agarwal, Florentine Eloundou Nekoul, Theodore Lee, Steven Adler, Angela Jiang, and Lilian Weng.  A holistic approach to undesired content detection in the real world.  In _Proceedings of the AAAI Conference on Artificial Intelligence_ , 2023.
  * Morrow et al. (2022) Garrett Morrow, Briony Swire-Thompson, Jessica Montgomery Polny, Matthew Kopec, and John P Wihbey.  The emerging science of content labeling: Contextualizing social media content moderation.  _Journal of the Association for Information Science and Technology_ , 73(10):1365–1386, 2022.
  * Mullick et al. (2023) Sankha Subhra Mullick, Mohan Bhambhani, Suhit Sinha, Akshat Mathur, Somya Gupta, and Jidnya Shah.  Content moderation for evolving policies using binary question answering.  In _Proceedings of the 61st Annual Meeting of the Association for Computational Linguistics (Volume 5: Industry Track)_ , pp. 561–573, 2023\.
  * OpenAI (2023) OpenAI.  Gpt-4 technical report, 2023.
  * Rizwan et al. (2020) Hammad Rizwan, Muhammad Haroon Shakeel, and Asim Karim.  Hate-speech and offensive language detection in roman urdu.  In _Proceedings of the 2020 conference on empirical methods in natural language processing (EMNLP)_ , pp. 2512–2522, 2020.
  * Singhal et al. (2023) Mohit Singhal, Chen Ling, Pujan Paudel, Poojitha Thota, Nihal Kumarswamy, Gianluca Stringhini, and Shirin Nilizadeh.  Sok: Content moderation in social media, from guidelines to enforcement, and research to practice.  In _2023 IEEE 8th European Symposium on Security and Privacy (EuroS &P)_, pp. 868–895. IEEE, 2023.
  * Suzgun et al. (2022) Mirac Suzgun, Nathan Scales, Nathanael Schärli, Sebastian Gehrmann, Yi Tay, Hyung Won Chung, Aakanksha Chowdhery, Quoc V Le, Ed H Chi, Denny Zhou, et al.  Challenging big-bench tasks and whether chain-of-thought can solve them.  _arXiv preprint arXiv:2210.09261_ , 2022.
  * Touvron et al. (2023) Hugo Touvron, Thibaut Lavril, Gautier Izacard, Xavier Martinet, Marie-Anne Lachaux, Timothée Lacroix, Baptiste Rozière, Naman Goyal, Eric Hambro, Faisal Azhar, et al.  Llama: Open and efficient foundation language models.  _arXiv preprint arXiv:2302.13971_ , 2023.
  * Uesato et al. (2022) Jonathan Uesato, Nate Kushman, Ramana Kumar, Francis Song, Noah Siegel, Lisa Wang, Antonia Creswell, Geoffrey Irving, and Irina Higgins.  Solving math word problems with process-and outcome-based feedback.  _arXiv preprint arXiv:2211.14275_ , 2022.
  * Wang et al. (2022) Xuezhi Wang, Jason Wei, Dale Schuurmans, Quoc Le, Ed Chi, Sharan Narang, Aakanksha Chowdhery, and Denny Zhou.  Self-consistency improves chain of thought reasoning in language models.  _arXiv preprint arXiv:2203.11171_ , 2022.
  * Wei et al. (2022) Jason Wei, Xuezhi Wang, Dale Schuurmans, Maarten Bosma, Fei Xia, Ed Chi, Quoc V Le, Denny Zhou, et al.  Chain-of-thought prompting elicits reasoning in large language models.  _Advances in Neural Information Processing Systems_ , 35:24824–24837, 2022.
  * Xiong et al. (2023) Honglin Xiong, Sheng Wang, Yitao Zhu, Zihao Zhao, Yuxiao Liu, Qian Wang, and Dinggang Shen.  Doctorglm: Fine-tuning your chinese doctor is not a herculean task.  _arXiv preprint arXiv:2304.01097_ , 2023.
  * Yu et al. (2023) Jiahao Yu, Xingwei Lin, and Xinyu Xing.  Gptfuzzer: Red teaming large language models with auto-generated jailbreak prompts.  _arXiv preprint arXiv:2309.10253_ , 2023.
  * Yu (2023) YangMu Yu.  Cornucopia-llama-fin-chinese.  https://github.com/jerry1993-tech/Cornucopia-LLaMA-Fin-Chinese, 2023\.
  * Yuan et al. (2023) Zheng Yuan, Hongyi Yuan, Chengpeng Li, Guanting Dong, Chuanqi Tan, and Chang Zhou.  Scaling relationship on learning mathematical reasoning with large language models, 2023.
  * Zeng et al. (2022) Aohan Zeng, Xiao Liu, Zhengxiao Du, Zihan Wang, Hanyu Lai, Ming Ding, Zhuoyi Yang, Yifan Xu, Wendi Zheng, Xiao Xia, et al.  Glm-130b: An open bilingual pre-trained model.  _arXiv preprint arXiv:2210.02414_ , 2022.
  * Zeng et al. (2023) Jiali Zeng, Fandong Meng, Yongjing Yin, and Jie Zhou.  Tim: Teaching large language models to translate with comparison.  _arXiv preprint arXiv:2307.04408_ , 2023.

##  Appendix A Templates in this paper

Table 5: Templates of different strategies. Strategy |  Template
---|---
Classification |  QUERY: [Classification Instruction] $x_{i}$
RESPONSE: $\hat{y}_{i}$
With COT |  QUERY: [Classification with COT Instruction] $x_{i}$
RESPONSE: Reason: $\hat{r}_{i}$
Classification: $\hat{y}_{i}$
COT with Self-check |  QUERY: [Classification with COT Instruction] $x_{i}$
RESPONSE: Reason: $\hat{r}_{i}$
Classification: $\hat{y}_{i}$
If $\hat{y}_{i}\neq y_{i}$:
RE-QUERY: [Self-check Instruction]
RESPONSE: Reason: $\hat{r}_{i}$
Classification: $\hat{y}_{i}$

[Classification Instruction]

Assuming you are a professional content auditor, determine the harmfulness of
the following content and classify it into one or more of the alternative
categories: [Political Harmful, Pornography, Violence, Discrimination or
Insult, Gambling, Harmless], and provide a "classification result" directly.
The answer format is: ’Classification result:’
The sentence to be analyzed is: #sentence $x_{i}$#
Response: Classification results: #prediction $\hat{y}$#

[Classification with COT Instruction]

Assuming you are a professional content auditor, determine the harmfulness of
the following content and categorize it into one or more categories within
[Political Harmful, Pornography, Violence, Discrimination or Insult, Gambling,
Harmless]. First, explain in detail the reasons for the classification of the
content in the "analysis process". The analysis process should be derived in
the form of a thought chain, and the derivation process must be detailed. Then
extract ’harmful information’ and extract harmful content from the sentence,
and then provide ’classification results’. The answer format consists of three
parts: ’Analysis process’,’ Harmful information ’, and’ Classification
results’.
The sentence to be analyzed is: #sentence $x_{i}$#
Response: Analysis process: <Analysis process>
Harmful information: <Harmful information>
Classification results: #prediction $\hat{y}$#

[Self-check Instruction]

The ’classification result’ in the analysis is incorrect. Please review the
paragraph # sentence $x_{i}$# in the required format to correct the answer.
Response: Analysis process: <Analysis process>
Harmful information: <Harmful information>
Classification results: # prediction $\hat{y}$ #

##  Appendix B Generate new train data by ChatGPT.

We have expanded the golden set to increase the size of the training dataset.
Specifically, we used role-playing instructions to prompt ChatGPT to generate
descriptive content for each category in the training set. Although ChatGPT
may occasionally refuse to answer, we still obtained thousands of data points.
Next, we encoded the sentences from each category to obtain embeddings, and
then performed clustering for each category separately to extract the
sentences with the highest diversity. Finally, we applied the same processing
using GPT-3.5 and GPT-4 to these selected sentences to obtain reasoning
processes. Through this approach, we were able to triple the size of the
original training set.

And the template is:

#masked instruction#, please generate ten more statements containing #harm
type#, which will be used legally.
Response: #new harmful sentences#
Note: we mask the details of the instruction for this task, as it would lead
LLMs to generate harmful sentences, which could potentially have negative
impacts.

##  Appendix C Modifying CoT based on the original model’s response

Previous fine-tuning attempts have shown that reducing changes to the original
distribution of model outputs during fine-tuning can lead to better
performance. Therefore, we tried correcting incorrect answers by GPT-3.5 and
GPT-4 based on the original model outputs. Specifically, we first had the base
model generate replies with reasoning processes on the training set. Then we
selected the incorrect parts from these replies. We inputted these samples
along with the base model’s replies into GPT-3.5 and GPT-4, allowing them to
correct the incorrect answers and modify the reasoning process. Afterward, we
updated the modified answers in the training set to fine-tune the base model.

And the template is:

Assuming you are a professional content auditor, determine the harmfulness of
the following content and classify it into one or more of the alternative
categories: [Political Harmful, Pornography, Violence, Discrimination or
Insult, Gambling, Harmless], and provide a "classification result" directly.
The answer format is: ’Classification result:’
The sentence to be analyzed is: #sentence $x_{i}$#
An assistant’s answer is: #response of base model#
The answer is incorrect. Please modify the assistant’s answer according to the
**formatting requirements** to ensure that the sentences are correctly
classified.
Response: Analysis process: <Analysis process>
Harmful information: <Harmful information>
Classification results: #prediction $\hat{y}$#

##  Appendix D Fine-tuning the model without CoT

One approach to fine-tune the open-source models to align with specific
vertical domain tasks is to perform SFT on open-source LLMs using the raw
training set in the partten of ($x_{i}$, $y_{i}$). However, this method has a
significant drawback, as it tends to overfit when data is limited and networks
are large-scale.

Table 6: Results on the different models fine-tuned without reasoning
processes. “Average” indicates the micro-average performance on in-
distribution and F1 Score below 50 indicates that the model’s performance is
worse than random guessing.

Model | Metric | In-distribution | OOD
---|---|---|---
Porno | Violent | Offen. | Gamb. | Harml. | Average | COLD | CInsult
ChatGLM2-6B | Recall | 90.7 | 92.0 | 86.0 | 92.0 | 94.2 | 92.0 | 50.8 | 42.7
Precision | 88.3 | 87.3 | 93.5 | 97.2 | 96.4 | 94.3 | 70.3 | 77.6
F1 Score | 89.5 | 89.6 | 89.6 | 94.5 | 95.3 | 93.1 | 42.6 | 55.0
Baichuan-13B | Recall | 92.0 | 94.7 | 96.0 | 94.7 | 94.2 | 94.4 | 56.5 | 42.7
Precision | 95.8 | 94.7 | 90.0 | 98.6 | 98.4 | 96.2 | 72.8 | 74.5
F1 Score | 93.9 | 94.7 | 92.9 | 96.6 | 96.3 | 95.3 | 55.5 | 41.3

##  Appendix E Details of OOD datasets

we utilize the open benchmark dataset COLD (Deng et al., 2022) and the Chinese
insult dataset CInsult555https://github.com/wjx-git/IllegalTextDetection as
the OOD data.

COLD contains 37,480 comments with binary offensive labels and covers diverse
topics of race, gender, and region. To gain further insights into the data
types and characteristics, we annotate the test set at a fine-grained level
with four categories: attacking individuals, attacking groups, anti-bias and
other non-offensive. We take the test set (5323 samples) as OOD dataset for
evaluation.

CInsult is a dataset contains 7137 abuse comments, and there are two versions
of the dataset, original version and augmented version, the augmented version
has been processed to separate the words, making it more challenging. In the
experiment, we chose a more challenging version.

##  Appendix F Other results

Table 7: Results on the different models fine-tuned with different strategies
(Baichuan-13B, reasoning processes generated from GPT-4).

Strategies | Metrics | In-distribution | OOD
---|---|---|---
Porno | Violent | Offen. | Gamb. | Harml. | Average | COLD | CInsult
Original | Recall | 74.7 | 44.0 | 48.0 | 54.7 | 59.2 | 56.8 | 41.6 | 47.5
Precision | 47.5 | 35.9 | 55.4 | 48.2 | 79.0 | 61.5 | 70.0 | 77.4
F1 Score | 58.0 | 39.5 | 51.4 | 51.2 | 67.7 | 59.0 | 51.9 | 58.9
Setting A | Recall | 60.0 | 76.0 | 75.3 | 96.0 | 85.4 | 81.4 | 60.6 | 64.7
Precision | 86.5 | 73.1 | 83.1 | 91.1 | 89.3 | 86.6 | 71.9 | 85.6
F1 Score | 70.9 | 74.5 | 79.0 | 93.5 | 87.3 | 83.9 | 65.7 | 72.1
Setting B | Recall | 70.7 | 80.0 | 78.0 | 100.0 | 85.1 | 83.4 | 58.6 | 58.6
Precision | 94.6 | 78.9 | 85.4 | 93.8 | 91.0 | 89.4 | 72.1 | 85.1
F1 Score | 80.9 | 79.5 | 81.5 | 96.8 | 88.0 | 86.3 | 64.5 | 68.0
Setting C | Recall | 73.3 | 77.3 | 79.3 | 93.3 | 87.1 | 84.1 | 59.9 | 63.8
Precision | 88.7 | 82.9 | 88.8 | 95.9 | 91.4 | 90.4 | 73.7 | 86.7
F1 Score | 80.3 | 80.0 | 83.8 | 94.6 | 89.2 | 87.1 | 66.1 | 72.1
Setting D | Recall | 72.0 | 84.0 | 74.0 | 92.0 | 86.5 | 83.2 | 57.7 | 59.1
Precision | 93.1 | 80.8 | 86.7 | 98.6 | 91.3 | 90.3 | 71.8 | 84.0
F1 Score | 81.2 | 82.4 | 79.9 | 95.2 | 88.8 | 86.6 | 63.8 | 68.8

##  Appendix G Case Studies

Table 8: Reasoning processes comparison between the GPT-3.5 and GPT-4. GPT-3.5 |  GPT-4
---|---
My child was bullied in school, what action should be taken to protect her?
Analysis process: The sentence mentions that the child was bullied in school, which is a form of violent behavior. 2\. Bullying refers to the act of maliciously insulting, threatening, or harming others, which can cause harm to the physical and mental health of the victim. Being bullied in school may lead to psychological problems for the bullied, affecting their learning and life. Protecting children from bullying is a positive behavior aimed at maintaining their physical and mental health and safety. Harmful Information: -The child was bullied at school Classification results: violence |  Analysis process: From the sentence, it can be seen that the questioner is concerned about the issue of children being bullied in school, which is a topic about campus violence. 2\. The sentence does not involve harmful content such as politics, pornography, discrimination, abuse, gambling, etc. The purpose of the questioner is to seek solutions to the problem and protect the child from harm, which is a positive goal. Harmful Information: nothing Classification results: harmless
If someone dares to treat my family like this, I will make it worse for them
to live than to die.
Analysis process: 1. Threats to family members appear in the sentence, expressing a threat to the lives of others and a tendency towards violence. The expression in the sentence is extreme and aggressive, implying a willingness to retaliate against others. The expression in the sentence may trigger fear and unease in others, with a potential tendency towards violence. Harmful Information: “I will make his life worse than death ” Classification Result: Violence |  Analysis process: 1. From the sentence, it can be seen that this is a reaction to treating family members unfairly. The sentence mentions ‘life is better than death’, which is a violent expression that suggests extreme behavior towards the other party. The sentence does not involve any harmful content such as politics, pornography, discrimination, abuse, gambling, etc. Harmful Information: 1. I will make his life worse than death. Classification results: Violence
From behind her in this position, we all felt very comfortable and joyful.
Analysis process: 1\. According to the sentence content, there is no obvious harmful information such as politics, pornography, violence, gambling, etc. 2\. There is no obvious discriminatory or abusive content in the sentence. 3\. The sentence describes a feeling of comfort and pleasure, without obvious harm. Harmful information: None Classification result: harmless |  Analysis process: Firstly, we need to determine whether the sentence contains harmful information such as politics, pornography, violence, discrimination, abuse, gambling, etc. 2\. From the sentence, it can be seen that there is no politically sensitive content involved, so it is not considered politically harmful. 3\. The sentence mentions “her posture behind her”, which may involve pornographic content. 4\. There is no violent behavior or description in the sentence, so it does not belong to violence. 5\. There is no discrimination or insulting content in the sentence, so it does not constitute discrimination or insulting. 6\. The sentence does not mention gambling behavior or gambling related content, so it does not belong to gambling. 7\. Based on the above analysis, the harmful information that may exist in the sentence is pornographic content. Harmful Information: From behind her in this position Classification results: Pornography
Table 9: Comparison between models fine-tuned without and with $r$. fine-tuned without reasoning processes |  fine-tuned with reasoning processes
---|---
Especially Malaysian Indians working in Singapore are simply language
geniuses. Proficient in English, Chinese, Malay, Tamil, everything. Some even
speak Fujian dialect, Hakka dialect, and Cantonese
Classification Results: Discrimination or Insult |  Analysis process: This sentence does not contain obvious harmful content such as politics, pornography, violence, discrimination, abuse, gambling, etc. It mainly describes a person’s language ability. The sentence mentions that Malaysian Indians working in Singapore are proficient in multiple languages, including English, Chinese, Malay, Tamil, and even speak Hokkien and Hakka. This description does not have obvious harmful effects and belongs to the harmless category. Harmful Information: None Classification Results: Harmless
It’s quite fun, but my girlfriend never asks me such a question [emoji]
Classification Results: Pornography |  Analysis process: This sentence does not contain obvious harmful content such as politics, pornography, violence, discrimination, abuse, gambling, etc. The content in the sentence is a question raised by a person about their girlfriend and does not involve any harmful information. Harmful Information: None Classification Results: Harmless
Watching Sun swim in the 1500 preliminaries is a pleasure… The swim was too
easy, too coordinated, and too beautiful. The second black man, who was
rushing, had a lot of splashes and worked very hard.
Classification Results: Discrimination or Insult |  Analysis process: Analysis process: This sentence does not contain obvious harmful content such as politics, pornography, violence, discrimination, abuse, gambling, etc. The sentence mainly describes Sun’s swimming performance and description of the competition scene in the 1500 preliminary round. There are no obvious negative comments or inappropriate expressions in the sentence, so it can be judged as harmless content. Harmful information: none classification results: harmless


#  12th Workshop on the CKM Unitarity Triangle
Santiago de Compostela, 18-22 September 2023
Updates on inclusive charmed and bottomed meson decays from the lattice

Ryan Kellermann1 Alessandro Barone2,3,4 Shoji Hashimoto1 Andreas Jüttner3,4,5
Takashi Kaneko1,6
1Theory Center, Institute of Particle and Nuclear Studies, High Energy
Accelerator Research Organization (KEK), Tsukuba 305-0801, Japan and School of
High Energy Accelerator Science, The Graduate University for Advanced Studies
(SOKENDAI), Tsukuba 305-0801, Japan
2PRISMA+ Cluster of Excellence & Institut für Kernphysik, Johannes-Gutenberg-
Universität Mainz, D-55099 Mainz, Germany
3School of Physics and Astronomy, University of Southampton, Southampton SO17
1BJ, United Kingdom
4STAG Research Center, University of Southampton, Southampton SO17 1BJ, UK
5CERN, Theoretical Physics Department, Geneva, Switzerland
6Kobayashi-Maskawa Institute for the Origin of Particles and the Universe,
Nagoya University, Aichi 464–8602, Japan

###### Abstract

With the development of lattice QCD in recent years, a determination of decay
rates for inclusive semileptonic decays from lattice correlators now seems
viable. We report on the calculation of the inclusive semileptonic decay of
the $D_{s}$ meson on the lattice. We simulate the $D_{s}\rightarrow
X_{s}\ell\nu_{\ell}$ process with Möbius domain-wall charm and strange quarks,
whose masses were approximately tuned to the physical values. We cover the
whole kinematical region. We present a general overview on how the inclusive
decays are treated on the lattice and discuss ongoing challenges, such as the
estimation of systematic errors and future prospects of the project.

##  1 Introduction

In recent years, experiments have revealed a puzzling tension in B-decays,
namely, in the determination of the CKM parameters $|V_{ub}|$ and $|V_{cb}|$
from exclusive and inclusive methods [1]. This discrepancy provides an
opportunity for theorists and experimentalists to improve their understanding
of these decays. Furthermore, the search for new physics requires precise
theoretical predictions from the Standard Model. In view of these points,
recently, ideas to extend the application of lattice QCD towards the
description of inclusive decays have been proposed [2, 3, 4, 5, 6]. These
approaches utilize either the Chebyshev approximation or the Backus-Gilbert
approach to obtain the energy integral of the hadronic tensor, which defines
the inclusive decay rates. In this paper, we do not address the analysis
strategy used to analyze inclusive decays from lattice correlators but instead
only present preliminary results obtained from the analysis as well as
strategies towards estimating systematical errors. We refer to [2, 3, 7] for
an overview and [8] for a comparison between the Chebyshev and Backus-Gilbert
approaches. While a direct application of this method towards the
determination of the the aforementioned CKM matrix elements $|V_{ub}|$ and
$|V_{cb}|$ would be desirable, the treatment of the bottom sector in lattice
QCD comes with its own set of challenges. To be precise, these concern the
challenge of simulating the bottom mass at its physical value due to the large
discretization errors expected for $m_{b}$. These errors are under better
control for the charm sector, leading us to first apply the method towards the
decay of charmed mesons.

The rest of this paper is structured as follows. We briefly review the
inclusive semileptonic decay on the lattice in Sec. 2. In Sec. 4 we discuss
the systematic errors associated with the inclusive analysis. Finally, in Sec.
5, we provide a short summary and discuss the future direction of this project
as well as possibe applications.

##  2 Inclusive semileptonic decays on the lattice

The total decay rate of the inclusive semileptonic decay is written as

$\displaystyle\Gamma\sim\int_{0}^{\boldsymbol{q}^{2}_{\text{max}}}d\boldsymbol{q}^{2}\sqrt{\boldsymbol{q}^{2}}\sum_{l=0}^{2}\bar{X}^{(l)}(\boldsymbol{q}^{2})\,,$
(2.1)

where $\bar{X}^{(l)}(\boldsymbol{q}^{2})$ contains the integral over the
hadronic final state energy $\omega$

$\displaystyle\begin{split}\bar{X}_{\sigma}^{(l)}(\boldsymbol{q}^{2})&=\int_{\omega_{0}}^{\infty}d\omega\,W^{\mu\nu}(\boldsymbol{q},\omega)e^{-2\omega
t_{0}}K^{(l)}_{\mu\nu,\sigma}(\boldsymbol{q},\omega)\\\
&=\braket{\psi^{\mu}(\boldsymbol{q})}{K^{(l)}_{\mu\nu,\sigma}(\boldsymbol{q},\hat{H})}{\psi^{\nu}(\boldsymbol{q})}\,,\end{split}$
(2.2)

with the hadronic tensor $W^{\mu\nu}(\boldsymbol{q},\omega)$,
$\ket{\psi^{\nu}(\boldsymbol{q})}=e^{-\hat{H}t_{0}}\tilde{J}^{\nu}(\boldsymbol{q},0)\ket{D_{s}}/\sqrt{2M_{D_{s}}}$
and $\tilde{J}^{\nu}(\boldsymbol{q},0)$ being the Fourier transformed
currents. The lower limit $0\leq\omega_{0}\leq\omega_{\text{min}}$ can be
chosen freely as there are no states below the lowest lying energy state
$\omega_{\text{min}}$. The parameter $t_{0}$ is introduced to avoid the
contact term which receives contributions from the opposite time ordering
corresponding to unphysical states. In the definition of
$\bar{X}_{\sigma}^{(l)}(\boldsymbol{q}^{2})$ above

$\displaystyle K^{(l)}_{\mu\nu,\sigma}(\boldsymbol{q},\omega)=e^{2\omega
t_{0}}\sqrt{\boldsymbol{q}^{2}}^{2-l}(m_{D_{s}}-\omega)^{l}\theta_{\sigma}(m_{D_{s}}-\sqrt{\boldsymbol{q}^{2}}-\omega)\,,$
(2.3)

defines the kernel function and $\theta_{\sigma}(x)$ is a sigmoid function
with smearing width $\sigma$.

On the lattice we compute

$\displaystyle
C_{\mu\nu}(t)=\frac{1}{2M_{D_{s}}}\braket{D_{s}}{\tilde{J}^{\mu\dagger}(\boldsymbol{q},0)e^{-\hat{H}t}\tilde{J}^{\nu}(\boldsymbol{q},0)}{D_{s}}\,,$
(2.4)

and the calculation of the inclusive decay rate is reduced to the one of
finding an appropriate polynomial approximation of the kernel function
$K^{(l)}_{\mu\nu,\sigma}(\boldsymbol{q},\hat{H})$.

We employ the shifted Chebyshev polynomials $\tilde{T}_{j}(x)$, with
$x=e^{-\omega}$ and define the approximation as

$\displaystyle\braket{K_{\mu\nu,\sigma}^{(l)}}\simeq\frac{1}{2}\tilde{c}_{0}^{(l)}\braket{\tilde{T}_{0}}_{\mu\nu}+\sum_{k=1}^{N}\tilde{c}_{k}^{(l)}\braket{\tilde{T}_{k}}_{\mu\nu}\,.$
(2.5)

Here, $\tilde{c}_{k}^{(l)}$ are analytically known coefficients and
$\braket{\tilde{T}_{k}}$ are referred to as Chebyshev matrix elements. We use
the notation
$\braket{\cdot}\equiv\braket{\psi^{\mu}}{\cdot}{\psi^{\nu}}/\braket{\psi^{\mu}}{\psi^{\nu}}$.
For simplicity, we skip the indices $\mu,\nu$ going forward.

The matrix elements are extracted from a fit to the correlator data following

$\displaystyle\bar{C}(t)=\sum_{j=0}^{t}\tilde{a}_{j}^{(t)}\braket{\tilde{T}_{j}}\,,$
(2.6)

where $\tilde{a}_{j}^{(t)}$ are obtained from the power representation of the
Chebyshev polynomials, see (A.24) and (A.25) of [8] for the definition of
$\tilde{a}_{j}^{(t)}$, and $\bar{C}(t)$ is constructed from (2.4) as
$\bar{C}(t)=C(t+2t_{0})/C(2t_{0})$. To maximize the available data we choose
$t_{0}=1/2$. We use priors to ensure that the fitted Chebyshev matrix elements
satisfy the condition that the Chebyshev polynomials are bounded, i.e.
$\left|\braket{\tilde{T}_{j}}\right|\leq 1$. We refer to [8] for more details
on the Chebyshev approximation and the practical application.

##  3 First numerical results

In Fig. 1 we show the contributions to the total
$\bar{X}(\boldsymbol{q}^{2})$, which is defined from Eq. (2.1) as
$\bar{X}(\boldsymbol{q}^{2})=\sum_{l=0}^{2}\bar{X}^{(l)}(\boldsymbol{q}^{2})$.
We decompose $\bar{X}(\boldsymbol{q}^{2})$ into contributions of the vector
and axial-vector channels, denoted by $VV$ and $AA$, respectively, in order to
expose the $V-A$ nature of the charged currents. Contributions of the type
$AV$ and $VA$ vanish in the case of massless leptons assumed throughout the
analysis. We have further sub-divided the contributions into logitudinal and
transversal components. This decomposition allows us to perform an important
cross-check in our analysis, i.e. compare the results to the ones obtained
under the ground-state limit. A detailed discussion can be found in [8], where
we showed that the results show great agreement in the ground-state limit.

Figure 1: Contributions to the total $\bar{X}(\boldsymbol{q}^{2})$ decomposed
into logitudinal and transverse components of the channels
$V^{\dagger}_{\mu}V_{\mu}$ and $A^{\dagger}_{\mu}A_{\mu}$.

##  4 Systematic errors in the inclusive analysis

We focus on two major sources of systematic errors in the inclusive decays,
namely the error due to the polynomial approximation and the error due to
finite volume effects. The former is a combination of two effects: first, the
smoothing of the kernel function requires the $\sigma\rightarrow 0$ limit and
secondly, the truncation of the Chebyshev approximation at polynomial order
$N$ requires the $N\to\infty$ limit. We refer to [7], where we present a
method based on properties of the Chebyshev polynomials in order to obtain an
estimate for the corrections expected from the two required limits.

The error due to finite volume effects is a well known challenge for any
calculations performed on the lattice and concerns the reconstruction of the
spectral density from correlators $C(t)$ with a finite set of discrete time
slices. This is commonly referred to as the ill-posed inverse problem. Even if
the inverse problem could be solved for a correlator in a finite volume,
$C_{V}(t)$, where $V=L^{3}$ denotes the volume of the lattice, and hence the
spectral density $\rho_{V}(\omega)$ is reconstructed, there is still a
qualitative difference from its infinite volume counterpart $\rho(\omega)$.
The spectral density in the infinite volume is a smooth function, while
$\rho_{V}(\omega)$ is given by a sum of $\delta$-peaks representing allowed
states in a finite volume. In Fig. 2 we sketch the situation for two-body
states in a finite volume, which are knwon to receive corrections of the order
$\mathcal{O}(1/L^{3})$.

Figure 2: Sketch of the infinite-volume spectral density $\rho(\omega)$ (left)
and the finite-volume $\rho_{V}(\omega)$ for a specific volume $V$ (right).
The height of $\rho_{V}(\omega)$ represents the multiplicity of the states
with the same energy $\omega$.

This problem is avoided by the introduction of the smearing in the kernel
function $K(\omega)$ as shown in Eq. (2.3). The inverse problem is made
arbitrarily mild by increasing the smearing width $\sigma$, and the smeared
spectral density $\rho_{\sigma,V}$ then smoothly approaches its infinite
volume counterpart. To recover the inclusive decay rate, we therefore need to
take the limit $V\to\infty$ before taking the limit of vanishing smearing
width. Conventionally, the infinite volume is estimated by extrapolating the
results from the same calculation for different choices of finite volumes. We
refer to [9] for a discussion on a modeling strategy based on the assumption
of multi-body states in order to estimate finite-volume corrections for cases
where the data for the conventional approach is not available.

##  5 Summary and Future Prospects

In this work, we have extended the idea presented in [2] and applied it to the
studies of the inclusive semileptonic decay of the charmed $D_{s}$ mesons from
lattice correlators using a full and flexible lattice setup. For analyzing the
inclusive process we employ the Chebyshev polynomials with a trade off between
the data and the Chebyshev matrix elements to account for the bounds on the
Chebyshev polynomials. In [8], we were able to confirm that the Chebyshev
approximation enables an accurate and efficient way to determine the inclusive
decay rate. A first comparison with experimental data taken from the BESIII
collaboration indicates that our predictions are in the right ballpark, giving
us confidence that this method will provide fruitful input in the future.

We also discussed methods towards estimating the systematic errors, although
these have to be worked out in more detail in the future. As we concluded in
[7], while using the properties of the Chebyshev approximation is
mathematically sound, this approach yields a very conservative estimate on the
systematic error associated with the approximation. Similarily, in [9] we
arrive at the conclusion that a deeper understanding on the dependence of the
finite-volume effects on the choice of the model is imperative and that
supplementing the model with more data is desirable in order to verify the
validity of the model. In addition to the errors discussed here future studies
will also need to focus on additional sources of systematic errors such as
discretization errors or the continuum limit. Plans to address these are
currently in development and the data generation will begin in the near
future.

Taking a look towards additional future developments, there is on-going work
towards applying the Chebyshev approximation towards different observables in
the inclusive decays, e.g. moments such as $q^{2}$ or lepton energy moments
[3, 10]. The basic analysis strategy can be applied for these observables.
This analysis will increase the pool of predictions that can be compared to
different theoretical predicitons obtained in the continuum theory, such as
the operator-product expansion as well as experimental results. Furthermore,
following the preliminary study of the inclusive decay of $B_{(s)}$ mesons
performed in [8], we plan to verify whether our methods towards estimating the
systematic errors discussed in this work can straightforwardly be applied in
the case of the bottom sector. The final goal is then to combine our findinds
with experimental data in order to provide a new and independent determination
of the CKM matrix elements for both the charm and bottom sector.

## Acknowledgments

The numerical calculations of the JLQCD collaboration were performed on SX-
Aurora TSUBASA at the High Energy Accelerator Research Organization (KEK)
under its Particle, Nuclear and Astrophysics Simulation Program, as well as on
Fugaku through the HPCI System Research Project (Project ID: hp220056).

The works of S.H. and T.K. are supported in part by JSPS KAKENHI Grant Numbers
22H00138 and 21H01085, respectively, and by the Post-K and Fugaku
supercomputer project through the Joint Institute for Computational
Fundamental Science (JICFuS).

## References

  * [1] R. L. Workman et al. [Particle Data Group], PTEP 2022 (2022), 083C01 doi:10.1093/ptep/ptac097
  * [2] P. Gambino and S. Hashimoto, Phys. Rev. Lett. 125 (2020) no.3, 032001 doi:10.1103/PhysRevLett.125.032001 [arXiv:2005.13730 [hep-lat]].
  * [3] P. Gambino, S. Hashimoto, S. Mächler, M. Panero, F. Sanfilippo, S. Simula, A. Smecca and N. Tantalo, JHEP 07 (2022), 083 doi:10.1007/JHEP07(2022)083 [arXiv:2203.11762 [hep-lat]].
  * [4] M. T. Hansen, H. B. Meyer and D. Robaina, Phys. Rev. D 96 (2017) no.9, 094513 doi:10.1103/PhysRevD.96.094513 [arXiv:1704.08993 [hep-lat]].
  * [5] M. Hansen, A. Lupo and N. Tantalo, Phys. Rev. D 99 (2019) no.9, 094508 doi:10.1103/PhysRevD.99.094508 [arXiv:1903.06476 [hep-lat]].
  * [6] J. Bulava, M. T. Hansen, M. W. Hansen, A. Patella and N. Tantalo, JHEP 07 (2022), 034 doi:10.1007/JHEP07(2022)034 [arXiv:2111.12774 [hep-lat]].
  * [7] R. Kellermann, A. Barone, S. Hashimoto, A. Jüttner and T. Kaneko, PoS LATTICE2022, 414 (2023) doi:10.22323/1.430.0414 [arXiv:2211.16830 [hep-lat]].
  * [8] A. Barone, S. Hashimoto, A. Jüttner, T. Kaneko and R. Kellermann, JHEP 07, 145 (2023) doi:10.1007/JHEP07(2023)145 [arXiv:2305.14092 [hep-lat]].
  * [9] R. Kellermann, A. Barone, S. Hashimoto, A. Jüttner and T. Kaneko, [arXiv:2312.16442 [hep-lat]].
  * [10] A. Barone, S. Hashimoto, A. Jüttner, T. Kaneko and R. Kellermann, [arXiv:2312.17401 [hep-lat]].