# Fake News in Sheep’s Clothing: Robust Fake News Detection Against LLM-
Empowered Style Attacks

Jiaying Wu  National University of Singapore<EMAIL_ADDRESS>and  Bryan
Hooi  National University of Singapore<EMAIL_ADDRESS>
###### Abstract.

It is commonly perceived that online fake news and reliable news exhibit stark
differences in writing styles, such as the use of sensationalist versus
objective language. However, we emphasize that style-related features can also
be exploited for _style-based attacks_. Notably, the rise of powerful Large
Language Models (LLMs) has enabled malicious users to mimic the style of
trustworthy news outlets at minimal cost. Our analysis reveals that LLM-
camouflaged fake news content leads to substantial performance degradation of
state-of-the-art text-based detectors (up to 38% decrease in F1 Score), posing
a significant challenge for automated detection in online ecosystems. To
address this, we introduce SheepDog, a style-agnostic fake news detector
robust to news writing styles. SheepDog achieves this adaptability through
LLM-empowered news reframing, which customizes each article to match different
writing styles using style-oriented reframing prompts. By employing _style-
agnostic training_ , SheepDog enhances its resilience to stylistic variations
by maximizing prediction consistency across these diverse reframings.
Furthermore, SheepDog extracts _content-focused veracity attributions_ from
LLMs, where the news content is evaluated against a set of fact-checking
rationales. These attributions provide supplementary information and potential
interpretability that assist veracity prediction. On three benchmark datasets,
empirical results show that SheepDog consistently yields significant
improvements over competitive baselines and enhances robustness against LLM-
empowered style attacks.

Fake News; Large Language Models; Adversarial Robustness

††ccs: Information systems Data mining††ccs: Computing methodologies Natural
language processing

##  1\. Introduction

Figure 1. A motivating example of LLM-empowered style attacks on text-based
fake news detectors, where fake news is camouflaged with the style of reliable
news publishers.

Automated fake news detectors are central to safeguarding integrity and
harmony in the digital age (Nguyen et al., 2020; Shu et al., 2019; Wu and
Hooi, 2023; Zhou et al., 2020). Style-related signals embedded in news
articles serve as a crucial indicator of news veracity. For instance,
reputable news sources uphold journalistic integrity, emphasize accuracy and
fact-checking, and maintain a balanced tone (Bachmann et al., 2021). In stark
contrast, unreliable outlets often employ sensationalism, lack credible
sources, and may exhibit partisan biases (Higdon, 2020). Building upon this,
prior advances have extracted sentiment features (Ajao et al., 2019; Zhang et
al., 2021) to complement and enhance the detector, and highlighted the
significance of writing style between hyperpartisan news and well-balanced
mainstream reporting (Potthast et al., 2018).

While style-related features are invaluable for discerning news veracity, they
also offer a direct avenue for malicious users to conduct style-based attacks.
This problem is exacerbated by the advent of powerful Large Language Models
(LLMs) (Brown et al., 2020; OpenAI, 2022; Ouyang et al., 2022), whose
unprecedented capabilities for reasoning and generative tasks (Wei et al.,
2022; Zheng and Zhan, 2023) reduces the gap between machine-generated and
human-written news. Consequently, malicious actors now possess the capability
to mimic the style of reputable news sources, in an attempt to evade automated
detection. As shown in Figure 1, using a style-oriented prompt (i.e., “style
of The New York Times”), LLM-camouflaged fake news successfully bypasses a
RoBERTa (Liu et al., 2019)-based fake news detector.

The impact of style-based attacks on fake news detectors has received limited
attention, despite prior investigations into detector vulnerability related to
attacks regarding social engagement (Chen and Shu, 2023), word-level
perturbations (Koenders et al., 2021), and machine-generated malicious
comments (Le et al., 2020). To assess the robustness of text-based fake news
detectors, we introduce a series of style-based adversarial attacks,
specifically by tailoring news articles to adversarial writing styles (details
provided in Section 4.1). Our experiments, as summarized in Table 1, reveal a
significant performance degradation in state-of-the-art text-based detectors,
with some experiencing up to a 38% decline in F1 Score. These detectors, well-
adapted to real news from trustworthy sources and fake news from unreliable
sources, struggle to address the emerging real-world challenge where news
content is presented in various styles.

Motivated by the style-related vulnerability observed in state-of-the-art
text-based detectors, we present SheepDog — a robust, style-agnostic fake news
detector that effectively identifies deceptive content, even when concealed
within the LLM-empowered ”sheep’s clothing”. Built upon a pretrained language
model (LM) backbone, which can be fine-tuned to capture task-specific salient
features, and leveraging the strong zero-shot reasoning and generative
capabilities of LLMs, SheepDog capitalizes on the strengths of both (LLM:
versatile; fine-tuned LM: specialized).

Our key ideas for overcoming style attacks are: (1) style-agnostic training,
in which predictions across style-diverse news reframings are regulated to
ensure maximal consistency, and (2) content-focused veracity attributions, in
which textual insights on news veracity are extracted from an LLM to inform
veracity predictions.

The style-agnostic nature of SheepDog stems from its utilization of LLM-
empowered news reframings. To encompass a wide spectrum of real-world news
writing styles, we expand our training samples with style-diverse expressions,
and ensure coherence across different expressions to establish style
invariance. Specifically, through style-oriented prompts, we exploit the
impressive text generation capabilities of LLMs to generate reframings for
each news article, introducing a diversity of styles. Then, we maximize the
consistency in the model’s veracity predictions across the original article
and its various reframings. This strategy assists SheepDog to discount style-
related features, which enables it to capture style-agnostic signals from the
news content.

To train a content-focused detector that is robust against stylistic
variations, we further propose to distill the veracity-related knowledge from
LLMs, specifically via eliciting content-focused attributions from inference
APIs. We instruct the LLM to assess each article based on a defined set of
content-focused fact-checking rationales (for comprehensive details, refer to
Section 5.3). The resulting textual attributions serve to identify indicators
of fake news. By converting these attributions into precise pseudo-labels, we
introduce additional weak supervision that steers SheepDog towards learning a
robust, style-agnostic representation of news. Our approach leverages these
attribution-level predictions not only to facilitate style-agnostic training
but also to potentially offer explainability into the veracity of news
articles.

Overall, we make the following three contributions:

  * •

Empirical Finding: We present a novel finding on the style-related
vulnerability of state-of-the-art text-based fake news detectors to LLM-
empowered style attacks.

  * •

LLM-Empowered Style Robustness: We introduce SheepDog, a style-agnostic fake
news detector that achieves robustness through a combination of style-agnostic
training and content-focused veracity attributions.

  * •

Effectiveness: Through extensive experiments, we demonstrate that leveraging
LLM-empowered diverse news reframings significantly enhances the performance
of representative language models (LMs) across three benchmark datasets.

##  2\. Related Work

Fake News Detection. Automated fake news detection has been widely explored
using a wide range of neural architectures (Devlin et al., 2019; Liu et al.,
2019; Pelrine et al., 2021; Shu et al., 2019; Zhou et al., 2020). Apart from
extracting lexical (Rashkin et al., 2017) and sentiment features (Potthast et
al., 2018) within the news article text, many methods incorporate auxiliary
features to supplement veracity prediction, including user comments (Shu et
al., 2019), news environments (Sheng et al., 2022), knowledge bases (Cui et
al., 2020; Dun et al., 2021), temporal patterns from users (Ruchansky et al.,
2017), and social graphs (Lu and Li, 2020; Nguyen et al., 2020; Wu and Hooi,
2023). Recent studies also seek to address challenges including temporal shift
(Hu et al., 2023), entity bias (Zhu et al., 2022a) and domain shift (Nan et
al., 2021, 2022; Zhu et al., 2022b) in fake news detection scenarios. In this
work, we adopt a text-based perspective and focus on news article texts,
specifically on the stylistic variations in the news article under LLM-
empowered style attacks.

Adversarial Attack on Fake News Detectors.  Investigating the vulnerabilities
of fake news detectors is central to improving their real-world applicability.
Hence, existing efforts (He et al., 2021a; Horne et al., 2019; Koenders et
al., 2021; Lyu et al., 2023; Wang et al., 2023; Zhou et al., 2019) have
studied the impact of different attacks from multiple aspects, including
manipulation of social engagements (Lyu et al., 2023; Wang et al., 2023) and
user behavior (He et al., 2021a), fact distortion (Koenders et al., 2021),
subject-object exchange (Zhou et al., 2019), and blocking of data availability
(Horne et al., 2019). However, the impact of writing styles remains
underexplored. To bridge this gap, we investigate the robustness of text-based
detectors against LLM-empowered style attacks, and propose a style-agnostic
solution.

Large Language Models (LLMs). LLMs (OpenAI, 2022; Touvron et al., 2023;
OpenAI, 2023), with billions of parameters, have showcased remarkable
reasoning capabilities, even matching or surpassing human performance in some
scenarios (Wei et al., 2022; Zheng and Zhan, 2023). Benefiting from this
success, recent work extracts knowledge from LLMs to facilitate a wide range
of tasks, such as visual classification (Menon and Vondrick, 2023) and node
classification (He et al., 2023). On a related front, LLM-generated text
detection (Henrique et al., 2023; Muñoz-Ortiz et al., 2023) has also attracted
increasing attention. Recent investigations further explore LLM-generated
misinformation (Epstein et al., 2023; Huang et al., 2023; Zellers et al.,
2019; Chen and Shu, 2023), and leverage LLMs for program-guided fact-checking
(Pan et al., 2023). Although we also prompt an LLM to generate adversarial
articles, we aim for style-agnostic detection that yields robust predictions
across style-diverse news expressions. Moreover, existing work on LLM-
generated misinformation mainly focuses on analysis of LLM-induced impacts and
news article generation, which is orthogonal to our contributions. In this
work, we focus on improving the detector’s style robustness by eliciting news
reframings and content-focused veracity attributions from LLMs.

##  3\. Problem Definition

Let $\mathcal{D}$ be a news dataset consisting of $N$ questionable news
pieces, denoted as $p_{1},p_{2},\dots,p_{N}$. Among the news pieces,
$\mathcal{P}_{L}\subset\mathcal{D}$ is a set of labeled news articles. Each
news article in $\mathcal{P}_{L}$ is assigned a ground-truth veracity label
$y$. In line with prior work (Shu et al., 2019; Zhang et al., 2021; Zhou et
al., 2020), $y$ is a binary label that represents either real news or fake
news.

As we focus on style-related issues, we consider a text-based setting.
Formally, the problem can be defined as follows:

###### Problem 1 (Text-Based Fake News Detection).

Given a news dataset $\mathcal{D}$ with training labels $\mathcal{Y}_{L}$, the
goal is to predict the veracity labels of unlabeled news pieces
$\mathcal{P}_{U}=\mathcal{D}\setminus\mathcal{P}_{L}$.

##  4\. LLM-Empowered Style Attacks

In this section, we establish a series of LLM-empowered style attacks, and
conduct preliminary analysis to assess the robustness of state-of-the-art
text-based fake news detectors.

###  4.1. Attack Formulation

The impressive capabilities of LLMs (Brown et al., 2020; OpenAI, 2022; Ouyang
et al., 2022) have enabled malicious users to disguise fake news with
restyling prompts, resulting in camouflaged articles that closely resemble
reliable sources. Here, we explore a direct form of style-based attack
utilizing news publisher names (e.g., ”The New York Times”). These names
possess distinct styles that can be readily adopted by producers of fake news,
making them a likely occurrence in real-world scenarios.

To replicate adversarial situations where news pieces are restyled in relation
to various publishers, we manipulate the styles of both trustworthy and
unreliable news. Specifically, in our test data, we utlize an LLM to rephrase
real news in the style of tabloids, and fake news in the style of mainstream
sources. Our general prompt format is shown as follows:

Rewrite the following article using the style of [publisher name]: [news
article]

Based on publisher popularity, in the place of [publisher name], we select
“National Enquirer” to camouflage real news, and “The New York Times” to
camouflage fake news. These LLM-camouflaged test articles are then employed to
evaluate the resilience of a detector against style-based attacks, as
illustrated in Figure 1.

###  4.2. Style-Related Detector Vulnerability

Automated fake news detection becomes increasingly difficult with LLM-
empowered style attacks. In this subsection, we conduct preliminary analysis
on real-world news to evaluate the influence of writing styles on text-based
detectors. Our analysis is based on the FakeNewsNet (Shu et al., 2020)
benchmark (consisting of PolitiFact and GossipCop datasets) and the Labeled
Unreliable News (LUN) dataset (Rashkin et al., 2017). Further details
regarding these benchmark datasets can be found in Table 8 and Section 6.1.1.
Specifically, we seek to address the following question: To what extent can
text-based fake news detectors withstand LLM-empowered style attacks?

In Table 1, we examine $13$ representative text-based detectors under both
original and adversarial settings (detailed method descriptions are provided
in Section 6.1.2). These detectors employ diverse model architectures,
including Convolutional Neural Networks (CNNs) (Zhou et al., 2020), Recurrent
Neural Networks (RNNs), (Shu et al., 2019), Graph Neural Networks (GNNs)
applied to document graphs (Vaibhav et al., 2019), Transformer-based LMs
(Devlin et al., 2019; He et al., 2021b; Liu et al., 2019; Zhang et al., 2021;
Schick and Schütze, 2021; Hu et al., 2022a; Xie et al., 2020), and LLMs
(OpenAI, 2022; Ouyang et al., 2022; Touvron et al., 2023). We evaluate model
robustness against LLM-empowered style attack based on performance on the
adversarial test set formulated in Section 4.1. Our empirical results yield
the following insights:

###### Observation 1 (Style-Related Vulnerability of Fake News Detectors).

State-of-the-art text-based fake news detectors are susceptible to LLM-
empowered style attacks. This susceptibility results in substantial
performance degradation,with an F1 Score decline of up to 38.3% on the
adversarial test set.

###### Observation 2 (Insufficiency of LLMs as Zero-Shot Fake News Detectors).

LLMs, despite their impressive zero-shot capabilities as general-purpose
foundation models, exhibit inferior detection performance compared to text-
based fake news detectors and pre-trained LMs fine-tuned specifically for fake
news detection.

Our two findings reveal a critical challenge in achieving robustness again
stylistic variations. Ideally, an effective fake news detector should
prioritize content assessment over style. While the news content typically
remains stable, style can undergo rapid changes in the dynamic online
environment. Our observations, however, reveals a stark deviation from this
ideal. Detectors overly influenced by style may struggle to reliably
differentiate between real and fake news, and even powerful LLMs may prove
inadequate for the specific demands of fake news detection. The accessibility
for malicious users to manipulate style using LLMs further exacerbates this
issue, emphasizing the pressing need for a style-agnostic approach.

Table 1. Under LLM-empowered style attacks, existing text-based fake news
detectors suffer severe performance deterioration in terms of F1 Score (%).
Performance degradation is computed on adversarial test set (O: original; A:
adversarial).

Method | PolitiFact |  | GossipCop |  | LUN
---|---|---|---|---|---
O | A ($\downarrow$) |  | O | A ($\downarrow$) |  | O | A ($\downarrow$)
dEFEND\c (Shu et al., 2019) | 82.59 | 12.15 |  | 70.74 | 4.34 |  | 80.92 | 19.16
SAFE\v (Zhou et al., 2020) | 79.85 | 8.74 |  | 70.64 | 2.93 |  | 79.46 | 13.12
SentGCN (Vaibhav et al., 2019) | 80.77 | 13.82 |  | 69.29 | 5.59 |  | 79.66 | 16.65
BERT (Devlin et al., 2019) | 84.99 | 12.68 |  | 74.50 | 5.52 |  | 80.96 | 24.61
RoBERTa (Liu et al., 2019) | 87.40 | 11.23 |  | 74.05 | 3.05 |  | 82.12 | 29.65
DeBERTa (He et al., 2021b) | 86.30 | 11.73 |  | 73.80 | 2.85 |  | 83.67 | 30.34
UDA (Xie et al., 2020) | 87.74 | 10.14 |  | 74.22 | 4.54 |  | 82.94 | 20.71
DualEmo (Zhang et al., 2021) | 87.76 | 15.34 |  | 75.36 | 5.89 |  | 81.52 | 24.97
PET (Schick and Schütze, 2021) | 85.51 | 11.02 |  | 74.63 | 3.08 |  | 83.66 | 31.08
KPT (Hu et al., 2022a) | 87.70 | 13.26 |  | 74.23 | 2.63 |  | 84.06 | 31.83
GPT3.5 (OpenAI, 2022) | 69.61 | 27.48 |  | 56.30 | 16.71 |  | 79.97 | 20.34
InstructGPT (Ouyang et al., 2022) | 64.59 | 20.69 |  | 50.38 | 9.13 |  | 68.16 | 11.39
Llama2-13B (Touvron et al., 2023) | 63.15 | 29.91 |  | 53.54 | 27.75 |  | 70.97 | 38.33

Figure 2. Overview of our SheepDog style-agnostic fake news detector.

##  5\. Proposed Approach

Building upon our empirical findings regarding the style-related vulnerability
of fake news detectors, and the limitations of LLMs as zero-shot detectors, we
present SheepDog, a style-agnostic detector that reliably assesses news
veracity through the utilization of LLMs. As illustrated in Figure 2, the
robustness of SheepDog is obtained from two core objectives in a multi-task
learning paradigm: (1) style-agnostic training, which flexibly adapts a pre-
trained LM to the fake news classification objective, and ensures consistent
veracity predictions across a diverse array of news reframings; and (2)
content-focused veracity attributions, which extracts veracity-related
insights from LLMs to inform model predictions.

To enhance usability and composability, we design our method to be simple and
modular, allowing it to be used with any language model backbone.

###  5.1. LLM-Empowered News Reframing

In light of our Observation 1, it becomes evident that text-based fake news
detectors, when exclusively trained on style-consistent news articles, may
exhibit limited real-world adaptability. To overcome this limitation, our key
idea is to diversify the training data through a process we refer to as
reframing, which presents each news articles in various styles.

Our reframing strategy is driven by two sub-goals: (1) ensuring coherence in
generated texts, and (2) covering a diverse range of styles. LLMs inherently
exhibit both desirable properties. Hence, for each training article, we
generate a series of prompts, each consisting of the news article and a style-
oriented reframing instruction. The general prompt format is represented as
follows, with a detailed example provided in Table 9:

Rewrite the following article in a / an [specified] tone: [news article]

To generate news expressions that simulate both reliable and unreliable
sources, we establish a set of four general style-oriented adjectives for the
prompt: ”objective and professional” and ”neutral” to emulate reliable
sources, and ”emotionally triggering” and ”sensational” for unreliable ones.

During the training phase, for labeled news article $p\in\mathcal{P}_{L}$, we
randomly select one reliable-style reframing prompt and one unreliable-style
reframing prompt to generate diverse expressions. Through querying the LLM, we
obtain two corresponding reframings: one reliable-style reframing denoted as
$p_{R}$, and one unreliable-style reframing denoted as $p_{F}$.

###  5.2. Style-Agnostic Training

A robust fake news detector should possess the capability to discern the
veracity of news articles based on their content, without being influenced by
their stylistic attributes. To this end, we introduce a style alignment
objective that ensures close alignment among the veracity predictions of news
article $p$, its reliable-style reframing $p_{R}$, and its unreliable-style
reframing $p_{F}$. This objective is derived as follows.

Let $\mathcal{M}$ be a pre-trained Language Model (LM) such as RoBERTa (Liu et
al., 2019). Employing an LM as the backbone of the detector offers advantages,
as LMs can be readily fine-tuned for the fake news detection task, which
enables them to effectively extract salient task-specific features from the
news content. Through $\mathcal{M}$, based on $p$, $p_{R}$ and $p_{F}$, we
acquire article representations $\mathbf{h}\in\mathbb{R}^{d}$, and reframing
representations $\mathbf{h}_{R},\mathbf{h}_{F}\in\mathbb{R}^{d}$:

(1)
$\mathbf{h}_{p}=\mathcal{M}(p),\quad\mathbf{h}_{R}=\mathcal{M}(p_{R}),\quad\mathbf{h}_{F}=\mathcal{M}(p_{F}).$

Subsequently, we apply a Multi-Layer Perceptron (MLP) to these
representations, and obtain veracity predictions
$\tilde{\mathbf{y}},\tilde{\mathbf{y}}_{R},\tilde{\mathbf{y}}_{F}\in\mathbb{R}^{2}$
for $p$, $p_{R}$ and $p_{F}$:

(2)
$\tilde{\mathbf{y}}=\text{MLP}_{pred}(\mathbf{h}),\quad\tilde{\mathbf{y}}_{R}=\text{MLP}_{pred}(\mathbf{h}_{R}),\quad\tilde{\mathbf{y}}_{F}=\text{MLP}_{pred}(\mathbf{h}_{F}).$

Each of $\tilde{\mathbf{y}},\tilde{\mathbf{y}}_{R},\tilde{\mathbf{y}}_{F}$
contain two logits that correspond to the real and fake classes, respectively.

Despite differences in style, the fundamental news content remains consistent
across the original news article $p$ and its reframings $p_{R}$ and $p_{F}$.
Ideally, $p_{R}$ and $p_{F}$ should yield the same predicted label as $p$. To
facilitate this, we formulate the following style alignment loss defined as
follows:

(3)
$\mathcal{L}_{\text{style}}=\textsf{MEAN}\left(\mathcal{L}_{1}(\tilde{\mathbf{y}}_{R},\tilde{\mathbf{y}}),\;\mathcal{L}_{1}(\tilde{\mathbf{y}}_{F},\tilde{\mathbf{y}})\right),$

where $\mathcal{L}_{1}$ represents the The Kullback-Leibler (KL) divergence
loss.

While aligning the predictions of $p_{R}$ and $p_{F}$ with $p$, it is crucial
to ensure accurate predictions for $p$. Therefore, we also incorporate a fake
news detection loss:

(4) $\mathcal{L}_{\text{news}}=\mathcal{L}_{2}(\tilde{\mathbf{y}},y),$

where $\mathcal{L}_{2}$ represents the standard cross entropy (CE) loss.

###  5.3. Content-Focused Veracity Attributions

In addition to tuning the LM backbone with the style alignment objective for
robust predictions, to capture the veracity-related signals directly linked to
the news content, we further propose to integrate external knowledge and
reasoning to inform veracity predictions. General-purpose LLMs, known for
their remarkable zero-shot reasoning capabilities (Huang and Chang, 2023),
provides a promising solution.

Specifically, we generate content-focused veracity attributions by prompting
an LLM. Our prompt consists of the news article text and a predefined set
$\mathcal{C}$ of content-oriented fact-checking rationales (e.g., “lack of
credible sources” and “false or misleading information”; we present the
detailed prompt template and an example in Appendix B.3 and Table 10). Through
this prompt, we instruct the LLM to provide textual attributions indicative of
news veracity (i.e., identifying characteristics associated with fake news).
This process efficiently extracts the LLM’s reasoning capabilities and general
knowledge, presenting it in a text format. The resulting textual attributions,
in turn, enriches our model with additional veracity-informed knowledge. The
general prompt format is shown as follows:

Article: [news article] Question: [given a list of fact-checking rationales,
ask the LLM to identify content-related characteristics associated with fake
news]

Upon querying the LLM, we obtain a list of textual veracity attributions based
on the input article. These attributions are then transformed into
$|\mathcal{C}|$-dimensional pseudo-labels, where the attribution to each
rationale in $\mathcal{C}$ is represented by a distinct binary label. For a
given news article $p\in\mathcal{P}_{L}$, this process yields pseudo-labels
$\mathbf{s}\in\mathbb{R}^{|\mathcal{C}|}$, which provides supplementary
veracity-related information. Similarly, for reframings $p_{R}$ and $p_{F}$,
we obtain pseudo-labels
$\mathbf{s}_{R},\mathbf{s}_{F}\in\mathbb{R}^{|\mathcal{C}|}$, respectively.
Notably, as $\mathcal{C}$ is focused on indicators of fake news, the pseudo-
labels for real news and its reframings are uniformly set to all-zeros.

To incorporate the veracity-informed knowledge from these content-focused
veracity attributions into our detector, we introduce a multi-label
attribution prediction objective. As shown in Eq. 1, we learn news
representations $\mathbf{h},\mathbf{h}_{R},\mathbf{h}_{F}\in\mathbb{R}^{d}$
for news article $p$ and its reframings $p_{R}$ and $p_{F}$, respectively,
using a pre-trained LM $\mathcal{M}$. Then, the attribution-level prediction
scores
$\tilde{\mathbf{s}},\tilde{\mathbf{s}}_{R},\tilde{\mathbf{s}}_{F}\in\mathbb{R}^{|\mathcal{C}|}$
are computed through another MLP:

(5)
$\tilde{\mathbf{s}}=\text{MLP}_{attr}(\mathbf{h}),\quad\tilde{\mathbf{s}}_{R}=\text{MLP}_{attr}(\mathbf{h}_{R}),\quad\tilde{\mathbf{s}}_{F}=\text{MLP}_{attr}(\mathbf{h}_{F}).$

The veracity attribution loss is then defined as:

(6)
$\mathcal{L}_{\text{attr}}=\textsf{MEAN}\left(\mathcal{L}_{3}(\hat{\mathbf{s}},\;\mathbf{s}),\mathcal{L}_{3}(\hat{\mathbf{s}_{R}},\;\mathbf{s}_{R}),\mathcal{L}_{3}(\hat{\mathbf{s}_{F}},\;\mathbf{s}_{F})\right),$

where $\mathcal{L}_{3}$ represents the binary cross entropy (BCE) loss, and
$\hat{\mathbf{s}}$ denotes the sigmoid-transformed scores in
$\tilde{\mathbf{s}}$ corresponding to each rationale.

###  5.4. Final Objective Function of SheepDog

By enforcing consistency among style-diverse news reframings and exploiting
the content-focused attributions from the LLM, the final objective function of
SheepDog is defined as a linear combination of the the style alignment loss
(Eq. 3), the news classification loss (Eq. 4), and the veracity attribution
loss (Eq. 6):

(7)
$\mathcal{L}=\mathcal{L}_{\text{style}}+\mathcal{L}_{\text{news}}+\mathcal{L}_{\text{attr}}.$

Our proposed SheepDog detector is an end-to-end framework where we
simultaneously learn the style-agnostic news veracity predictor and the
content-focused attribution predictor.

Table 2. SheepDog significantly outperforms competitive baselines on four
adversarial test settings under LLM-empowered style attacks (formulated in
Section 4), in terms of F1 Score (%) . Bold (underlined) values indicate the
best overall (baseline) performance. Statistical significance over the most
competitive baselines computed using the Wilcoxon signed-rank test (Wilcoxon,
1945) is indicated with ∗ ($p<.01$). (G1: text-based fake news detectors; G2:
LMs fine-tuned to the fake news detection task; G3: LLMs)

| Method | PolitiFact |  | GossipCop |  | LUN
---|---|---|---|---|---|---
A | B | C | D |  | A | B | C | D |  | A | B | C | D
G1 | dEFEND\c | 70.44 | 69.77 | 73.67 | 72.98 |  | 66.40 | 66.55 | 68.93 | 69.07 |  | 61.76 | 62.28 | 72.95 | 72.50
SAFE\v | 71.11 | 70.80 | 75.55 | 75.24 |  | 67.71 | 67.05 | 68.31 | 67.65 |  | 66.34 | 67.08 | 72.40 | 73.16
SentGCN | 66.95 | 62.50 | 69.54 | 65.08 |  | 63.70 | 63.07 | 63.61 | 63.01 |  | 63.01 | 62.50 | 76.11 | 75.56
DualEmo | 72.42 | 71.23 | 77.07 | 75.80 |  | 69.47 | 68.50 | 71.69 | 70.71 |  | 56.55 | 54.78 | 68.53 | 66.80
G2 | BERT | 72.31 | 71.37 | 77.23 | 76.24 |  | 68.98 | 68.17 | 71.95 | 71.11 |  | 56.35 | 54.61 | 68.50 | 66.74
RoBERTa | 76.17 | 74.95 | 78.28 | 77.05 |  | 71.00 | 70.47 | 72.56 | 72.02 |  | 52.47 | 53.62 | 68.31 | 69.46
DeBERTa | 74.57 | 74.36 | 80.60 | 80.35 |  | 70.95 | 71.15 | 72.51 | 72.71 |  | 53.33 | 55.45 | 67.16 | 69.27
UDA | 77.60 | 75.57 | 79.21 | 77.17 |  | 69.68 | 69.33 | 72.16 | 71.80 |  | 62.23 | 61.80 | 68.25 | 67.80
PET | 74.49 | 70.75 | 75.49 | 71.76 |  | 71.55 | 70.85 | 73.74 | 73.02 |  | 52.58 | 53.30 | 63.71 | 64.33
KPT | 74.44 | 73.32 | 77.73 | 76.60 |  | 71.60 | 71.01 | 73.69 | 73.10 |  | 52.23 | 53.62 | 65.71 | 67.15
G3 | GPT3.5 | 42.13 | 43.44 | 56.61 | 58.17 |  | 39.59 | 38.67 | 48.44 | 47.38 |  | 59.63 | 61.24 | 65.74 | 67.43
InstructGPT | 43.90 | 43.90 | 54.21 | 54.21 |  | 41.25 | 40.18 | 44.26 | 43.12 |  | 56.77 | 57.15 | 58.93 | 59.32
Llama2-13B | 33.24 | 34.48 | 53.64 | 55.45 |  | 25.79 | 26.06 | 37.07 | 37.40 |  | 32.64 | 33.00 | 50.81 | 51.33
Ours | SheepDog | 80.99∗ | 79.89∗ | 82.36∗ | 81.24 |  | 74.45∗ | 74.38∗ | 75.95∗ | 75.88∗ |  | 85.63∗ | 86.06∗ | 87.89∗ | 88.32∗

##  6\. Experiments

In this section, we empirically evaluate SheepDog to investigate the following
six research questions:

  * •

Robustness Against Style Attacks (Section 6.2): How robust is SheepDog against
LLM-empowered style attacks?

  * •

Effectiveness on Unperturbed Articles (Section 6.3): How proficiently can
SheepDog identify fake news within the original unperturbed test articles?

  * •

Adaptability to Different LM Backbones (Section 6.4): How does SheepDog’s
performance vary with different representative LM backbones?

  * •

Ablation Study (Section 6.5): What are the respective impacts of style-
agnostic training and content-focused attributions on SheepDog’s overall
performance?

  * •

Stability Across Reframing Prompts (Section 6.6): Does SheepDog yield
consistent improvements across diverse settings of news reframing prompts?

  * •

Case Study (Section 6.7): How can we interpret SheepDog’s rationale for fake
news detection through its predictions on content-focused attributions?

###  6.1. Experimental Setup

####  6.1.1. Datasets

We evaluate our approach on three widely-used real-world benchmark datasets:
the FakeNewsNet public benchmark (Shu et al., 2020), which consists of the
PolitiFact and GossipCop datasets, and the Labeled Unreliable News (LUN)
dataset (Rashkin et al., 2017). PolitiFact and LUN center on political
discourse, while GossipCop focuses on celebrity gossip. PolitiFact and LUN
center on political discourse, while GossipCop focuses on celebrity gossip.

The dataset description and statistics are delegated to Appendix A and Table
8. We randomly select $80$% of the articles as the training data, and the
other $20$% as test data.

####  6.1.2. Baselines

We benchmark SheepDog against thirteen representative baseline methods, which
can be categorized as:

Text-based fake news detectors (G1) employ neural architectures tailored
specifically for the fake news detection task. dEFEND\c is a variant of dEFEND
(Shu et al., 2019) based on the news article text that adopts RNN-based
hierarchical co-attention. SAFE\v is a text-based variant of SAFE (Zhou et
al., 2020) that leverages a CNN-based architecture to learn semantic comments.
SentGCN (Vaibhav et al., 2019) encodes veracity-related sentence interaction
patterns within each article using a GNN, and DualEmo (Zhang et al., 2021)
incorporates dual emotion features from news publishers and news comments. As
our SheepDog approach does not involve user comments, we implement DualEmo on
a BERT-base (Devlin et al., 2019) backbone with publisher emotion features for
a fair comparison.

Fine-tuned LMs (G2) adapts pre-trained LMs to the fake news detection task,
and has proven effective in handling misinformation scenarios (Pelrine et al.,
2021). In addition to three widely-recognized LMs, namely BERT (Devlin et al.,
2019), RoBERTa (Liu et al., 2019), and DeBERTa (He et al., 2021b), we include
UDA (Xie et al., 2020), a representative BERT-based model that employs diverse
text augmentations to yield consistent model predictions against input noise.
We also select two methods under the popular prompting paradigm: PET (Schick
and Schütze, 2021), which converts textual inputs into cloze questions that
contain a task description; and KPT, (Hu et al., 2022b) which expands the
label word space with varied class-related tokens. For a fair comparison, as
our proposed approach does not involve unlabeled articles, we implement UDA
using consistency training on the supervised training data, and exclude the
self-training and PLM ensemble components for PET. All methods in this
category are implemented with base version LMs, in line with our approach.

LLMs (G3) generate textual veracity predictions in a zero-shot manner. We
select three representative LLMs as baselines: GPT3.5 (OpenAI, 2022) (version
name: gpt-3.5-turbo), InstructGPT (Ouyang et al., 2022) (version name:
gpt-3.5-turbo-instruct), and Llama2-13B (Touvron et al., 2023) (version name:
Llama-2-13b-chat-hf), with details delegated to Appendix B.1.

####  6.1.3. Implementation Details

We implement SheepDog and its variants based on PyTorch 1.10.0 with CUDA 11.1.
We utilize pretrained RoBERTa-base weights from HuggingFace Transformers
4.13.0 (Wolf et al., 2020). The LM backbone for SheepDog was configured with a
maximum sequence length of 512, a batch size of 4, and a learning rate of
$2\times 10^{-5}$. We prompt GPT3.5 (version name: gpt-3.5-turbo) to generate
news reframings, content-focused veracity attributions, and adversarial test
articles. Detailed prompting descriptions and examples are delegated to
Appendix B.2 and B.3. For SheepDog’s attribution prediction and veracity
prediction, we employ two MLPs, each with a single layer (we also implement a
variant with 2-layer MLPs in Section 6.5). Our model is fine-tuned for $5$
epochs. For the implementation of baseline methods, we adhere to the
architectures and hyperparameters recommended by their respective authors.

We evaluate model performance using Accuracy (%) and macro-F1 Score (%). For
all experiments except those involving LLMs, we report averaged metrics over
10 runs of each method to provide a comprehensive evaluation. In the case of
LLM zero-shot predictions, we employ greedy decoding and conduct each
experiment once.

###  6.2. Robustness Against Style Attacks

We establish a series of LLM-empowered style attacks to assess SheepDog’s
robustness. Following our prompt template formulated in Section 4.1, in the
place of [publisher name], we select “National Enquirer” and “The Sun” to
camouflage real news, and “CNN” and “The New York Times” for fake news,
according to publisher popularity. This yields $2\times 2=4$ distinct
adversarial test sets, labeled as A through D in Table 3. Note that we report
the results of SheepDog on adversarial set A in all other subsections.

Table 3. Notations and setup for the four style-based adversarial test sets in Section 6.2, denoted as A through D. [publisher name] | CNN | The New York Times
---|---|---
National Enquirer | A | B
The Sun | C | D

Table 2 compares the performance of SheepDog with competitive baselines across
adversarial test sets A through D under LLM-empowered style attacks. We can
observe that: (1) All baseline methods are highly susceptible to LLM-empowered
style attacks. This vulnerability suggests that existing methods may exhibit a
tendency towards over-fitting on style-related attributes, a consequence of
being trained on datasets where real and fake news exhibit distinct stylistic
features. (2) UDA, which leverages back-translation to generate diverse text
augmentations, consistently demonstrates higher robustness compared to its
BERT backbone. This suggests the efficacy of UDA’s augmentation strategy.
However, UDA still struggles to fully adapt to significant stylistic variances
in the input articles. This limitation may be attributed to the fact that
augmentations through back-translation alone may not provide sufficient
variance. (3) On the challenging adversarial test sets of LUN, CNN-based
SAFE\v and GNN-based SentGCN are more robust than LM-based baselines, which
suggests that LMs can be more prone to overfit to style-related features. (4)
SheepDog outperforms the most competitive baseline by significant margins.
Across the three benchmarks, this improvement averages to $2.59$%, $2.77$%,
and $15.70$% across the four adversarial test sets, in terms of F1 score. This
validates the effectiveness of SheepDog’s style-agnostic training and content-
focused veracity attributions against style attacks.

###  6.3. Effectiveness on Unperturbed Articles

A desirable fake news detector should achieve both robustness and
effectiveness. Our empirical results, detailed in Table 4, clearly demonstrate
that SheepDog excels in this regard. When tested on the original, unaltered
articles, SheepDog consistently matches (on PolitiFact and GossipCop) or
surpasses (on LUN) the performance of the most competitive baseline, in terms
of accuracy and F1 score. This showcases the framework’s proficiency in
accurately discerning unreliable news from reliable ones, and implies
SheepDog’s practical utility in real-world scenarios.

###  6.4. Adaptability to Different LM Backbones

To assess the flexibility of SheepDog, we evaluate the performance of SheepDog
combined with three representative LM backbones: RoBERTa, BERT and DeBERTa. As
demonstrated in Table 5, SheepDog substantially enhances the performance of
the corresponding LM backbones. This adaptability highlights SheepDog’s
robustness from style-agnostic training and content-focused attributions,
showcasing its potential for flexible deployment in diverse contexts where
different LMs may be preferred or more readily available. This finding further
reinforces the benefits of SheepDog’s design with a LM backbone, which
flexibly adapts it to task-specific needs.

Table 4. SheepDog achieves performance (%) that is either better or comparable
to competitive baselines on the unperturbed original test sets. Bold
(underlined) values indicate the best overall (baseline) performance, and ∗
indicates $p<.01$ using the Wilcoxon signed-rank test (Wilcoxon, 1945).

Method | PolitiFact |  | GossipCop |  | LUN
---|---|---|---|---|---
Acc. | F1 |  | Acc. | F1 |  | Acc. | F1
dEFEND\c | 82.67 | 82.59 |  | 70.85 | 70.74 |  | 81.33 | 80.92
SAFE\v | 79.89 | 79.85 |  | 70.71 | 70.64 |  | 79.93 | 79.46
SentGCN | 81.11 | 80.77 |  | 69.38 | 69.29 |  | 80.07 | 79.66
DualEmo | 87.78 | 87.76 |  | 75.51 | 75.36 |  | 81.78 | 81.52
BERT | 85.22 | 84.99 |  | 74.60 | 74.50 |  | 81.13 | 80.96
RoBERTa | 88.00 | 87.40 |  | 74.14 | 74.05 |  | 82.53 | 82.12
DeBERTa | 86.33 | 86.30 |  | 73.86 | 73.80 |  | 84.01 | 83.67
UDA | 87.77 | 87.74 |  | 74.28 | 74.22 |  | 83.02 | 82.94
PET | 85.56 | 85.51 |  | 74.75 | 74.63 |  | 84.00 | 83.66
KPT | 87.78 | 87.70 |  | 74.38 | 74.23 |  | 84.40 | 84.06
GPT3.5 | 71.11 | 69.61 |  | 61.49 | 56.30 |  | 80.67 | 79.97
InstructGPT | 67.78 | 64.59 |  | 58.33 | 50.38 |  | 70.87 | 68.16
Llama2-13B | 65.56 | 63.15 |  | 55.74 | 53.54 |  | 72.47 | 70.97
SheepDog | 88.44 | 88.39 |  | 75.77 | 75.75 |  | 93.05∗ | 93.04∗

Table 5. On different LM backbones, SheepDog demonstrates stable and significant improvements over the most competitive baseline (in F1 %). Statistical significance over the corresponding LM backbone is computed using the Wilcoxon signed-rank test (Wilcoxon, 1945) and indicated with ∗ ($p<.01$). Method | PolitiFact | GossipCop | LUN
---|---|---|---
RoBERTa | 76.17 | 71.00 | 52.47
SheepDog-RoBERTa | 80.99∗ | 74.45∗ | 85.63∗
BERT | 72.31 | 68.98 | 56.35
SheepDog-BERT | 81.37∗ | 73.54∗ | 80.36∗
DeBERTa | 74.57 | 70.95 | 53.33
SheepDog-DeBERTa | 81.10∗ | 73.89∗ | 82.58∗

###  6.5. Ablation Study

To gain deeper insights into the functioning of SheepDog and the role of its
different components, we conduct an ablation study to evaluate the respective
contributions of LLM-empowered news reframings and content-focused veracity
attributions. We compare SheepDog with the following three model variants:

  * •

SheepDog w/ 2-layer MLP, which employs 2-layer MLPs with hidden size of $64$
as attribution detector and veracity detector.

  * •

SheepDog-R, which excludes style-diverse news reframings and the style-
agnostic training component.

  * •

SheepDog-A, which omits content-focused veracity attributions and the
attribution prediction component.

Table 6 shows the performance of different SheepDog variants on adversarial
test sets C and D. The results suggest that: (1) SheepDog-R without news
reframings only yields slight improvements compared with a fine-tuned RoBERTa
model, which suggests the pivotal role of diverse reframings in SheepDog’s
robustness. (2) Removing content-focused veracity attributions (SheepDog-A)
results in inferior performance than SheepDog, which implies the effectiveness
of extracting content-related knowledge from LLMs. (3) SheepDog, utilizing one
single layer for veracity prediction and attribute predictions, slightly
outperforms the variant employing 2-layer MLPs. This suggests that the
expressiveness of LMs, harnessed through our proposed objectives, yields
article representations that contains rich indicators for both attributions
and veracity.

Table 6. Ablation of SheepDog demonstrates benefits of LLM-empowered news reframing (denoted as R) and content-focused veracity attributions (denoted as A) in F1 Score (%). Method | PolitiFact | GossipCop | LUN
---|---|---|---
SheepDog | 80.99 | 74.45 | 85.63
w/ 2-layer MLP | 79.83 | 74.03 | 84.75
\- R | 76.71 | 70.98 | 53.27
\- A | 80.73 | 73.74 | 84.83
RoBERTa | 76.17 | 71.00 | 52.47

###  6.6. Stability Across Reframing Prompts

As detailed in Section 5.1, we utilize an LLM to generate two types of
reframings for a given training article $p$: a reliable-style reframing
denoted as $p_{R}$, and an unreliable-style reframing denoted as $p_{F}$. To
assess SheepDog’s stability across different reframing prompts, we
systematically examine its performance using four diverse combinations of
prompts, denoted as R1 through R4.

Recall that we adopt the following template for news reframing:

Rewrite the following article in a / an [specified] tone: [$p$]

For R1 through R4, the specified tones are defined as:

  * •

R1: $p_{R}$ \- “objective and professional”; $p_{F}$ \- “emotionally
triggering”.

  * •

R2: $p_{R}$ \- “objective and professional”; $p_{F}$ \- “sensational”.

  * •

R3: $p_{R}$ \- “neutral”; $p_{F}$ \- “emotionally triggering”.

  * •

R4: $p_{R}$ \- “neutral”; $p_{F}$ \- “sensational”.

As shown in Table 7, SheepDog consistently demonstrates stable and significant
improvements over the most competitive baseline. This validates the
generalizability of our approach, suggesting its potential in effectively
combating deceptive online information in the ever-evolving digital landscape.
Notably, our SheepDog approach conducts sampling between two reliable-style
reframings and two unreliable-style reframings, leading to better versatility.

Table 7. Across different sets of news reframing prompts, SheepDog demonstrates stable and significant improvements over the most competitive baseline (in F1 %). (“Baseline”: performance of the most competitive baseline. Method | PolitiFact | GossipCop | LUN
---|---|---|---
Baseline | 77.60 | 71.60 | 66.34
SheepDog | 80.99 | 74.45 | 85.63
w/ R1 | 79.02 | 74.22 | 77.42
w/ R2 | 79.93 | 74.16 | 86.18
w/ R3 | 80.36 | 73.55 | 76.77
w/ R4 | 79.71 | 74.01 | 85.55
Figure 3. SheepDog yields robust predictions against LLM-camouflaged fake
news. Additionally, the top predicted veracity attribution points toward the
false and misleading information in the news content, providing
interpretability.

###  6.7. Case Study

To illustrate SheepDog’s potential for interpretability in real-world
scenarios, we conduct a case study based on a fake news sample from the LUN
test set. The baseline RoBERTa model correctly identifies the original article
as fake, but mistakenly categorizes two style-transformed adversarial articles
as real news under LLM-empowered style attacks. In contrast, our SheepDog
approach adeptly detects the original article as fake, as well as the two
adversarial counterparts. As shown in Figure 3, the article falsely asserts
the efficacy of cannabis oil in treating cancer, a claim clearly at odds with
established medical knowledge. By leveraging the softmax-converted
probabilities derived from the attribution-level prediction scores (refer to
Eq. 5), our SheepDog model effectively explains the article’s dubious nature
as “false or misleading information”. Combining content-focused veracity
attributions with style-agnostic training, SheepDog offers practitioners a
valuable tool for reliably discerning and interpreting news veracity.

##  7\. Conclusion and Future Work

In this paper, we address the critical aspect of style-related robustness in
fake news detection. Motivated by our empirical finding on the susceptibility
of state-of-the-art text-based detectors to LLM-empowered style attacks, we
introduce SheepDog, a style-agnostic fake news detector that emphasizes
content veracity over style. Leveraging the strengths of task-specific LM
backbones and versatile general-purpose LLMs, SheepDog adopts a multi-task
learning paradigm, which integrates style-agnostic training and content-
focused veracity attributions. Extensive experiments on three benchmark
datasets demonstrate SheepDog’s robustness and effectiveness across various
style-based adversarial settings, news reframing prompts, and representative
LM backbones. Moving forward, SheepDog lays a solid foundation for developing
more resilient and adaptable models in the ever-changing online landscape, and
demonstrates promising potential to be further extended to multi-modal
scenarios.

## References

  * (1)
  * Ajao et al. (2019) Oluwaseun Ajao, Deepayan Bhowmik, and Shahrzad Zargari. 2019.  Sentiment Aware Fake News Detection on Online Social Networks. In _ICASSP 2019 - 2019 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP)_. 2507–2511.
  * Bachmann et al. (2021) Philipp Bachmann, Mark Eisenegger, and Diana Ingenhoff. 2021.  Defining and Measuring News Media Quality: Comparing the Content Perspective and the Audience Perspective.  _The International Journal of Press/Politics_ 27 (03 2021).
  * Brown et al. (2020) Tom Brown, Benjamin Mann, Nick Ryder, Melanie Subbiah, Jared D Kaplan, Prafulla Dhariwal, Arvind Neelakantan, Pranav Shyam, Girish Sastry, Amanda Askell, Sandhini Agarwal, Ariel Herbert-Voss, Gretchen Krueger, Tom Henighan, Rewon Child, Aditya Ramesh, Daniel Ziegler, Jeffrey Wu, Clemens Winter, Chris Hesse, Mark Chen, Eric Sigler, Mateusz Litwin, Scott Gray, Benjamin Chess, Jack Clark, Christopher Berner, Sam McCandlish, Alec Radford, Ilya Sutskever, and Dario Amodei. 2020.  Language Models are Few-Shot Learners. In _Advances in Neural Information Processing Systems_ , Vol. 33. 1877–1901.
  * Chen and Shu (2023) Canyu Chen and Kai Shu. 2023.  Can LLM-Generated Misinformation Be Detected?  arXiv:2309.13788
  * Cui et al. (2020) Limeng Cui, Haeseung Seo, Maryam Tabar, Fenglong Ma, Suhang Wang, and Dongwon Lee. 2020.  DETERRENT: Knowledge Guided Graph Attention Network for Detecting Healthcare Misinformation. In _Proceedings of the 26th ACM SIGKDD International Conference on Knowledge Discovery & Data Mining_. 492–502.
  * Devlin et al. (2019) Jacob Devlin, Ming-Wei Chang, Kenton Lee, and Kristina Toutanova. 2019.  BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding. In _NAACL_. 4171–4186.
  * Dun et al. (2021) Yaqian Dun, Kefei Tu, Chen Chen, Chunyan Hou, and Xiaojie Yuan. 2021.  KAN: Knowledge-aware Attention Network for Fake News Detection.  _Proceedings of the AAAI Conference on Artificial Intelligence_ 35, 1 (2021), 81–89.
  * Epstein et al. (2023) Ziv Epstein, Mengying C Fang, Antonio A Arechar, and David G Rand. 2023.  What label should be applied to content produced by generative AI?  https://doi.org/10.31234/osf.io/v4mfz
  * He et al. (2021a) Bing He, Mustaque Ahamad, and Srijan Kumar. 2021a.  PETGEN: Personalized Text Generation Attack on Deep Sequence Embedding-Based Classification Models. In _Proceedings of the 27th ACM SIGKDD Conference on Knowledge Discovery & Data Mining_. 575–584.
  * He et al. (2021b) Pengcheng He, Xiaodong Liu, Jianfeng Gao, and Weizhu Chen. 2021b.  DeBERTa: Decoding-enhanced BERT with Disentangled Attention. In _International Conference on Learning Representations_.
  * He et al. (2023) Xiaoxin He, Xavier Bresson, Thomas Laurent, Adam Perold, Yann LeCun, and Bryan Hooi. 2023.  Harnessing Explanations: LLM-to-LM Interpreter for Enhanced Text-Attributed Graph Representation Learning.  arXiv:2305.19523 [cs.LG]
  * Henrique et al. (2023) Da Silva Gameiro Henrique, Andrei Kucharavy, and Rachid Guerraoui. 2023.  Stochastic Parrots Looking for Stochastic Parrots: LLMs are Easy to Fine-Tune and Hard to Detect with other LLMs.  arXiv:2304.08968 [cs.CL]
  * Higdon (2020) Nolan Higdon. 2020.  What is Fake News? A Foundational Question for Developing Effective Critical News Literacy Education.  _Democratic Communiqué_ 279 (03 2020).  Issue 1.
  * Horne et al. (2019) Benjamin D. Horne, Jeppe Nørregaard, and Sibel Adali. 2019.  Robust Fake News Detection Over Time and Attack.  _ACM Trans. Intell. Syst. Technol._ 11, 1, Article 7 (2019), 23 pages.
  * Hu et al. (2023) Beizhe Hu, Qiang Sheng, Juan Cao, Yongchun Zhu, Danding Wang, Zhengjia Wang, and Zhiwei Jin. 2023.  Learn over Past, Evolve for Future: Forecasting Temporal Trends for Fake News Detection. In _Proceedings of the 61st Annual Meeting of the Association for Computational Linguistics (Volume 5: Industry Track)_. 116–125.
  * Hu et al. (2022a) Shengding Hu, Ning Ding, Huadong Wang, Zhiyuan Liu, Jingang Wang, Juanzi Li, Wei Wu, and Maosong Sun. 2022a.  Knowledgeable Prompt-tuning: Incorporating Knowledge into Prompt Verbalizer for Text Classification. In _Proceedings of the 60th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers)_. 2225–2240.
  * Hu et al. (2022b) Shengding Hu, Ning Ding, Huadong Wang, Zhiyuan Liu, Jingang Wang, Juanzi Li, Wei Wu, and Maosong Sun. 2022b.  Knowledgeable Prompt-tuning: Incorporating Knowledge into Prompt Verbalizer for Text Classification. In _Proceedings of the 60th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers)_. 2225–2240.
  * Huang and Chang (2023) Jie Huang and Kevin Chen-Chuan Chang. 2023.  Towards Reasoning in Large Language Models: A Survey. In _Findings of the Association for Computational Linguistics: ACL 2023_. 1049–1065.
  * Huang et al. (2023) Kung-Hsiang Huang, Kathleen McKeown, Preslav Nakov, Yejin Choi, and Heng Ji. 2023.  Faking Fake News for Real Fake News Detection: Propaganda-Loaded Training Data Generation. In _Proceedings of the 61st Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers)_. 14571–14589.
  * Koenders et al. (2021) Camille Koenders, Johannes Filla, Nicolai Schneider, and Vinicius Woloszyn. 2021.  How Vulnerable Are Automatic Fake News Detection Methods to Adversarial Attacks?  arXiv:2107.07970 [cs.CL]
  * Le et al. (2020) Thai Le, Suhang Wang, and Dongwon Lee. 2020.  MALCOM: Generating Malicious Comments to Attack Neural Fake News Detection Models. In _2020 IEEE International Conference on Data Mining (ICDM)_. 282–291.
  * Liu et al. (2019) Yinhan Liu, Myle Ott, Naman Goyal, Jingfei Du, Mandar Joshi, Danqi Chen, Omer Levy, Mike Lewis, Luke Zettlemoyer, and Veselin Stoyanov. 2019.  RoBERTa: A Robustly Optimized BERT Pretraining Approach.  arXiv:1907.11692 [cs.CL]
  * Lu and Li (2020) Yi-Ju Lu and Cheng-Te Li. 2020.  GCAN: Graph-aware Co-Attention Networks for Explainable Fake News Detection on Social Media. In _Proceedings of the 58th Annual Meeting of the Association for Computational Linguistics_. 505–514.
  * Lyu et al. (2023) Yuefei Lyu, Xiaoyu Yang, Jiaxin Liu, Sihong Xie, Philip Yu, and Xi Zhang. 2023.  Interpretable and Effective Reinforcement Learning for Attacking against Graph-based Rumor Detection. In _2023 International Joint Conference on Neural Networks (IJCNN)_. 1–9.
  * Menon and Vondrick (2023) Sachit Menon and Carl Vondrick. 2023.  Visual Classification via Description from Large Language Models. In _The Eleventh International Conference on Learning Representations_.
  * Muñoz-Ortiz et al. (2023) Alberto Muñoz-Ortiz, Carlos Gómez-Rodríguez, and David Vilares. 2023.  Contrasting Linguistic Patterns in Human and LLM-Generated Text.  arXiv:2308.09067 [cs.CL]
  * Nan et al. (2021) Qiong Nan, Juan Cao, Yongchun Zhu, Yanyan Wang, and Jintao Li. 2021.  MDFEND: Multi-domain fake news detection. In _Proceedings of the 30th ACM International Conference on Information & Knowledge Management_. 3343–3347.
  * Nan et al. (2022) Qiong Nan, Danding Wang, Yongchun Zhu, Qiang Sheng, Yuhui Shi, Juan Cao, and Jintao Li. 2022.  Improving Fake News Detection of Influential Domain via Domain- and Instance-Level Transfer. In _Proceedings of the 29th International Conference on Computational Linguistics_. 2834–2848.
  * Nguyen et al. (2020) Van-Hoang Nguyen, Kazunari Sugiyama, Preslav Nakov, and Min-Yen Kan. 2020.  FANG: Leveraging Social Context for Fake News Detection Using Graph Representation. In _Proceedings of the 29th ACM International Conference on Information Knowledge Management_. 1165–1174.
  * OpenAI (2022) OpenAI. 2022.  ChatGPT: Optimizing language models for dialogue.
  * OpenAI (2023) OpenAI. 2023.  GPT-4 Technical Report.  arXiv:2303.08774 [cs.CL] https://arxiv.org/pdf/2303.08774.pdf
  * Ouyang et al. (2022) Long Ouyang, Jeffrey Wu, Xu Jiang, Diogo Almeida, Carroll Wainwright, Pamela Mishkin, Chong Zhang, Sandhini Agarwal, Katarina Slama, Alex Ray, John Schulman, Jacob Hilton, Fraser Kelton, Luke Miller, Maddie Simens, Amanda Askell, Peter Welinder, Paul F Christiano, Jan Leike, and Ryan Lowe. 2022.  Training language models to follow instructions with human feedback. In _Advances in Neural Information Processing Systems_ , Vol. 35. Curran Associates, Inc., 27730–27744.
  * Pan et al. (2023) Liangming Pan, Xiaobao Wu, Xinyuan Lu, Anh Tuan Luu, William Yang Wang, Min-Yen Kan, and Preslav Nakov. 2023.  Fact-Checking Complex Claims with Program-Guided Reasoning. In _Proceedings of the 61st Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers)_. 6981–7004.
  * Pelrine et al. (2021) Kellin Pelrine, Jacob Danovitch, and Reihaneh Rabbany. 2021.  The Surprising Performance of Simple Baselines for Misinformation Detection. In _Proceedings of the Web Conference 2021_. 3432–3441.
  * Potthast et al. (2018) Martin Potthast, Johannes Kiesel, Kevin Reinartz, Janek Bevendorff, and Benno Stein. 2018.  A Stylometric Inquiry into Hyperpartisan and Fake News. In _Proceedings of the 56th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers)_. 231–240.
  * Rashkin et al. (2017) Hannah Rashkin, Eunsol Choi, Jin Yea Jang, Svitlana Volkova, and Yejin Choi. 2017.  Truth of Varying Shades: Analyzing Language in Fake News and Political Fact-Checking. In _Proceedings of the 2017 Conference on Empirical Methods in Natural Language Processing_. Association for Computational Linguistics, 2931–2937.
  * Ruchansky et al. (2017) Natali Ruchansky, Sungyong Seo, and Yan Liu. 2017.  CSI: A Hybrid Deep Model for Fake News Detection. In _Proceedings of the 2017 ACM on Conference on Information and Knowledge Management_. 797–806.
  * Schick and Schütze (2021) Timo Schick and Hinrich Schütze. 2021.  Exploiting Cloze-Questions for Few-Shot Text Classification and Natural Language Inference. In _Proceedings of the 16th Conference of the European Chapter of the Association for Computational Linguistics: Main Volume_. 255–269.
  * Sheng et al. (2022) Qiang Sheng, Juan Cao, Xueyao Zhang, Rundong Li, Danding Wang, and Yongchun Zhu. 2022.  Zoom Out and Observe: News Environment Perception for Fake News Detection. In _Proceedings of the 60th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers)_. 4543–4556.
  * Shu et al. (2019) Kai Shu, Limeng Cui, Suhang Wang, Dongwon Lee, and Huan Liu. 2019.  DEFEND: Explainable Fake News Detection. In _Proceedings of the 25th ACM SIGKDD International Conference on Knowledge Discovery & Data Mining_. 395–405.
  * Shu et al. (2020) Kai Shu, Deepak Mahudeswaran, Suhang Wang, Dongwon Lee, and Huan Liu. 2020.  FakeNewsNet: A Data Repository with News Content, Social Context, and Spatiotemporal Information for Studying Fake News on Social Media.  _Big Data_ 8, 3 (2020), 171–188.
  * Touvron et al. (2023) Hugo Touvron, Louis Martin, Kevin Stone, Peter Albert, Amjad Almahairi, Yasmine Babaei, Nikolay Bashlykov, Soumya Batra, Prajjwal Bhargava, Shruti Bhosale, Dan Bikel, Lukas Blecher, Cristian Canton Ferrer, Moya Chen, Guillem Cucurull, David Esiobu, Jude Fernandes, Jeremy Fu, Wenyin Fu, Brian Fuller, Cynthia Gao, Vedanuj Goswami, Naman Goyal, Anthony Hartshorn, Saghar Hosseini, Rui Hou, Hakan Inan, Marcin Kardas, Viktor Kerkez, Madian Khabsa, Isabel Kloumann, Artem Korenev, Punit Singh Koura, Marie-Anne Lachaux, Thibaut Lavril, Jenya Lee, Diana Liskovich, Yinghai Lu, Yuning Mao, Xavier Martinet, Todor Mihaylov, Pushkar Mishra, Igor Molybog, Yixin Nie, Andrew Poulton, Jeremy Reizenstein, Rashi Rungta, Kalyan Saladi, Alan Schelten, Ruan Silva, Eric Michael Smith, Ranjan Subramanian, Xiaoqing Ellen Tan, Binh Tang, Ross Taylor, Adina Williams, Jian Xiang Kuan, Puxin Xu, Zheng Yan, Iliyan Zarov, Yuchen Zhang, Angela Fan, Melanie Kambadur, Sharan Narang, Aurelien Rodriguez, Robert Stojnic, Sergey Edunov, and Thomas Scialom. 2023.  Llama 2: Open Foundation and Fine-Tuned Chat Models.  arXiv:2307.09288 [cs.CL]
  * Vaibhav et al. (2019) Vaibhav Vaibhav, Raghuram Mandyam, and Eduard Hovy. 2019.  Do Sentence Interactions Matter? Leveraging Sentence Level Representations for Fake News Classification. In _Proceedings of the Thirteenth Workshop on Graph-Based Methods for Natural Language Processing (TextGraphs-13)_. 134–139.
  * Wang et al. (2023) Haoran Wang, Yingtong Dou, Canyu Chen, Lichao Sun, Philip S. Yu, and Kai Shu. 2023.  Attacking Fake News Detectors via Manipulating News Social Engagement. In _Proceedings of the ACM Web Conference 2023_. 3978–3986.
  * Wei et al. (2022) Jason Wei, Yi Tay, Rishi Bommasani, Colin Raffel, Barret Zoph, Sebastian Borgeaud, Dani Yogatama, Maarten Bosma, Denny Zhou, Donald Metzler, Ed H. Chi, Tatsunori Hashimoto, Oriol Vinyals, Percy Liang, Jeff Dean, and William Fedus. 2022.  Emergent Abilities of Large Language Models.  _Transactions on Machine Learning Research_ (2022).
  * Wilcoxon (1945) Frank Wilcoxon. 1945.  Individual Comparisons by Ranking Methods.  _Biometrics Bulletin_ 1 (1945), 80–83.
  * Wolf et al. (2020) Thomas Wolf, Lysandre Debut, Victor Sanh, Julien Chaumond, Clement Delangue, Anthony Moi, Pierric Cistac, Tim Rault, Rémi Louf, Morgan Funtowicz, et al. 2020\.  Transformers: State-of-the-Art Natural Language Processing. In _Proceedings of the 2020 Conference on Empirical Methods in Natural Language Processing: System Demonstrations_. 38–45.
  * Wu and Hooi (2023) Jiaying Wu and Bryan Hooi. 2023.  DECOR: Degree-Corrected Social Graph Refinement for Fake News Detection. In _Proceedings of the 29th ACM SIGKDD Conference on Knowledge Discovery and Data Mining_. 2582–2593.
  * Xie et al. (2020) Qizhe Xie, Zihang Dai, Eduard Hovy, Thang Luong, and Quoc Le. 2020.  Unsupervised Data Augmentation for Consistency Training. In _Advances in Neural Information Processing Systems_ , Vol. 33. 6256–6268.
  * Zellers et al. (2019) Rowan Zellers, Ari Holtzman, Hannah Rashkin, Yonatan Bisk, Ali Farhadi, Franziska Roesner, and Yejin Choi. 2019.  Defending Against Neural Fake News. In _Advances in Neural Information Processing Systems_ , H. Wallach, H. Larochelle, A. Beygelzimer, F. d'Alché-Buc, E. Fox, and R. Garnett (Eds.), Vol. 32. Curran Associates, Inc.  https://proceedings.neurips.cc/paper_files/paper/2019/file/3e9f0fc9b2f89e043bc6233994dfcf76-Paper.pdf
  * Zhang et al. (2021) Xueyao Zhang, Juan Cao, Xirong Li, Qiang Sheng, Lei Zhong, and Kai Shu. 2021.  Mining Dual Emotion for Fake News Detection. In _Proceedings of the Web Conference 2021_. 3465–3476.
  * Zheng and Zhan (2023) Haoyi Zheng and Huichun Zhan. 2023.  ChatGPT in Scientific Writing: A Cautionary Tale.  _The American Journal of Medicine_ 136, 8 (2023), 725–726.e6.
  * Zhou et al. (2020) Xinyi Zhou, Jindi Wu, and Reza Zafarani. 2020.  SAFE: Similarity-Aware Multi-modal Fake News Detection. In _Advances in Knowledge Discovery and Data Mining_. 354–367.
  * Zhou et al. (2019) Zhixuan Zhou, Huankang Guan, Meghana Bhat, and Justin Hsu. 2019.  Fake News Detection via NLP is Vulnerable to Adversarial Attacks. In _Proceedings of the 11th International Conference on Agents and Artificial Intelligence_.
  * Zhu et al. (2022a) Yongchun Zhu, Qiang Sheng, Juan Cao, Shuokai Li, Danding Wang, and Fuzhen Zhuang. 2022a.  Generalizing to the Future: Mitigating Entity Bias in Fake News Detection. In _Proceedings of the 45nd International ACM SIGIR Conference on Research and Development in Information Retrieval_. Association for Computing Machinery.
  * Zhu et al. (2022b) Yongchun Zhu, Qiang Sheng, Juan Cao, Qiong Nan, Kai Shu, Minghui Wu, Jindong Wang, and Fuzhen Zhuang. 2022b.  Memory-Guided Multi-View Multi-Domain Fake News Detection.  _IEEE Transactions on Knowledge and Data Engineering_ (2022).

##  Appendix A Dataset Description

In Section 6, we conduct extensive experiments on three widely-used real-world
benchmark datasets: the FakeNewsNet benchmark (Shu et al., 2020), which
consists of the PolitiFact and GossipCop datasets, and the Labeled Unreliable
News (LUN) dataset (Rashkin et al., 2017). PolitiFact and LUN contain news
articles about political discourse, while GossipCop focuses on celebrity
gossip.

In the case of the LUN dataset, which further classifies unreliable news into
three sub-categories: satire, hoax, and propaganda, we conduct binary
classification between reliable (real) and unreliable (fake) news, and ensure
an equal number of unreliable news from each of these fine-grained categories.
The detailed statistics for each dataset are shown as follows:

Table 8. Dataset statistics. Dataset | PolitiFact | GossipCop | LUN
---|---|---|---
# News Articles | 450 | 7,916 | 7,500
# Real News | 225 | 3,958 | 3,750
# Fake News | 225 | 3,958 | 3,750

##  Appendix B Details of LLM Prompting

###  B.1. Prompt Template for Baseline LLM Detectors

In section 6.1.2, we select three representative LLMs as baselines for fake
news detection: GPT3.5 (OpenAI, 2022) (version name: gpt-3.5-turbo),
InstructGPT (Ouyang et al., 2022) (version name: gpt-3.5-turbo-instruct), and
Llama2-13B (Touvron et al., 2023) (version name: Llama-2-13b-chat-hf).

For GPT-3.5 and InstructGPT, we utilize their APIs from OpenAI, and set the
temperature to $0$ for stable veracity predictions. For Llama2-13B, we utilize
the model weights from HuggingFace Transformers (Wolf et al., 2020) version
4.31.0, and set do_sample to False for greedy decoding. For all three LLMs, we
adopt the following prompt to conduct zero-shot fake news detection:

Question: Does the following contain real or fake news (or information)?
Answer in one word with either ‘Real’ or ‘Fake’, then explain why. [news
article] Answer: [starts with a predicted veracity label]

###  B.2. Prompt Template for Obtaining SheepDog’s News Reframings

Recall from Section 5.1 that we generate reliable-style reframings and
unreliable-style reframings with GPT3.5 for each labeled news article. Among
the four prompt templates presented below, we use the first two templates to
generate reliable-style reframings, and the other two to generate unreliable-
style reframings.

During reframing generation, we set the temperature to $0.7$, and set the
maximum number of response tokens to $512$. A detailed reframing example is
presented in Table 9.

Rewrite the following article in an objective and professional tone: [news
article]

Rewrite the following article in a neutral tone: [news article]

Rewrite the following article in an emotionally triggering tone: [news
article]

Rewrite the following article in a sensational tone: [news article]

###  B.3. Prompt Template for Obtaining SheepDog’s Veracity Attributions

In Section 5.3, we elicit veracity-related information based on the news
content from GPT3.5, specifically by prompting the LLM to assess the labeled
articles and their corresponding reframings based on a predefined set of
content-focused rationales. We utilize the following prompt template:

Article: [news article] Question: Which of the following problems does this
article have? Lack of credible sources, False or misleading information,
Biased opinion, Inconsistencies with reputable sources. If multiple options
apply, provide a comma-separated list ordered from most to least related.
Answer ”No problems” if none of the options apply.

In the process of prompting GPT3.5 for veracity attributions, we set the
temperature to $0$ to yield stable attributions, and set the maximum number of
response tokens to $300$. We demonstrate an example in Table 10.

Table 9. An example of GPT-3.5 generated reliable-style news reframing.

PROMPT: Rewrite the following article in an objective and professional tone:
BREAKING: Obamas Daughter BUSTED By The Cops For Doing This At the Democratic
National Convention, Michelle Obama tried to use her daughters as shining
examples of just how far America has come. But the fact is that the young
Obamas are pushing the limits as any teenager doesand probably even further.
According to new reports, Malia Obama was spotted at a party in Marthas
Vineyard recently. It was eventually broken up by police because of noise
complaints. The first daughter was immediately rushed from the party by Secret
Service before the cops arrived. The West Tisbury police declined to confirm
if Malia was there, citing national security. But the truth surfaced anyway.
This is just one incident in a string of wild behavior on Malias part. Just a
month ago the first daughter came under fire when photos surfaced of her
smoking marijuana at a concert. Malia is clearly using her gap year before
Harvard to party and experiment. Later, Malia and Barack were spotted
strolling around a pool together having a chat. Were sure he had a few choice
words for her. Clearly, a lack of respect for the law runs deep in this
family.
---
RESPONSE: Malia Obama’s Recent Activities Raise Concerns Recently, reports
have emerged regarding the behavior of Malia Obama, daughter of former
President Barack Obama and Michelle Obama. At the Democratic National
Convention, Michelle Obama showcased her daughters as symbols of progress in
America. However, it appears that the young Obamas, like many teenagers, are
testing boundaries and possibly going beyond them. According to sources, Malia
Obama was seen at a gathering in Marthas Vineyard that was eventually
disbanded by law enforcement due to noise complaints. The first daughter was
promptly escorted away from the scene by the Secret Service before the arrival
of the police. West Tisbury police have neither confirmed nor denied Malia’s
presence at the event, citing national security concerns. Nevertheless, the
truth regarding her attendance eventually surfaced. This incident is just one
in a series of instances where Malia’s behavior has attracted attention. Not
long ago, she faced criticism when photos emerged showing her allegedly
smoking marijuana at a concert. It appears that Malia is utilizing her gap
year before attending Harvard University as a period for socializing and
experimentation. Later on, both Malia and Barack Obama were observed engaging
in a conversation while walking near a pool. It can be assumed that the former
president had some words of advice or guidance for his daughter. It is evident
that a lack of regard for the law seems to be a recurring issue within this
family. In conclusion, recent events raise concerns about the behavior of
Malia Obama. These incidents, including her presence at a party disbanded by
the police and alleged marijuana use, suggest a period of exploration and a
certain disregard for legal boundaries.

Table 10. An example of content-focused veracity attributions extracted from
GPT-3.5.

PROMPT: Article: BREAKING: Obamas Daughter BUSTED By The Cops For Doing This
At the Democratic National Convention, Michelle Obama tried to use her
daughters as shining examples of just how far America has come. But the fact
is that the young Obamas are pushing the limits as any teenager doesand
probably even further. According to new reports, Malia Obama was spotted at a
party in Marthas Vineyard recently. It was eventually broken up by police
because of noise complaints. The first daughter was immediately rushed from
the party by Secret Service before the cops arrived. The West Tisbury police
declined to confirm if Malia was there, citing national security. But the
truth surfaced anyway. This is just one incident in a string of wild behavior
on Malias part. Just a month ago the first daughter came under fire when
photos surfaced of her smoking marijuana at a concert. Malia is clearly using
her gap year before Harvard to party and experiment. Later, Malia and Barack
were spotted strolling around a pool together having a chat. Were sure he had
a few choice words for her. Clearly, a lack of respect for the law runs deep
in this family.
---
Question: Which of the following problems does this article have? Lack of
credible sources, False or misleading information, Biased opinion,
Inconsistencies with reputable sources. If multiple options apply, provide a
comma-separated list ordered from most to least related. Answer ”No problems”
if none of the options apply.
RESPONSE: False or misleading information, Biased opinion.


# Music Score Expansion with Variable-Length Infilling

###### Abstract

In this paper, we investigate using the variable-length infilling (VLI) model
[1], which is originally proposed to infill missing segments, to “prolong”
existing musical segments at musical boundaries. Specifically, as a case
study, we expand 20 musical segments from 12 bars to 16 bars, and examine the
degree to which the VLI model preserves musical boundaries in the expanded
results using a few objective metrics, including the Register Histogram
Similarity we newly propose. The results show that the VLI model has the
potential to address the expansion task.

##  1 Introduction

Music score infilling is an instance of automatic symbolic music generation
tasks. Given two _disconnected_ musical segments, an infilling model aims to
“fill the gap” between them by generating novel content, as depicted in Figure
1. We refer to the two given segments as the _past context_ $C_{\text{past}}$
and the _future context_ $C_{\text{future}}$, respectively, and the generated
one as the _infilled segment_ $C_{\text{new}}$. We assume that these segments
are all represented by sequences of event tokens such as note-on and note-
duration [2].

Figure 1: An illustration of how the variable-length infilling model [1]
works, where the model is asked to generate 2-bar musical content (i.e., the
“context gap” is two bars). We do not explicitly consider key in this work.

Very recently, we proposed a variable-length infilling (VLI) model [1] that
accepts variable-length “context gaps” and generates variable-length musical
contents. Specifically, at inference time, the input needed by the VLI model
is composed of only the two token sequences $C_{\text{past}}$,
$C_{\text{future}}$, and _the number of bars_ between $C_{\text{past}}$ and
$C_{\text{future}}$ (i.e., the context gap). VLI decides on its own the number
of tokens to be filled to the designated gap between the given contexts.
Moreover, a single VLI model can deal with different context gaps (instead of
needing one model per context gap). In VLI, the context gap is expected to be
a non-zero and positive integer.

Interestingly, we later realize that VLI also holds the potential to address a
highly relevant, yet much less explored task, called music score expansion.
Given a sequence of music notes $\\{n_{1},n_{2},...,n_{L}\\}$, this task
entails increasing the length of the sequence by _inserting_ novel content at
one or multiple locations of the sequence. Without loss of generalizability,
we assume that we are about to insert content at only one location $p$, where
$1<p<L$. We can then use VLI to address this task by treating
$\\{n_{1},n_{2},...,n_{p}\\}$ as $C_{\text{past}}$,
$\\{n_{p+1},n_{p+2},...,n_{L}\\}$ as $C_{\text{future}}$, and setting the
desired context gap to a non-zero integer. In other words, while the
$C_{\text{past}}$ and $C_{\text{future}}$ are originally _connected_ with zero
gap in between, the idea here is to split them apart somewhere, create
artificial gap between then, and use VLI to fill the gap, to prolong the
sequence as a result.111We can also “shorten” a sequence by removing a sub-
sequence and use VLI to create a novel content that is shorter than the
removed sub-sequence to connect the gap smoothly; we leave this as a future
work.

In this paper, we present preliminary experiments exploring such a novel use
case of VLI, treating music score expansion as a sub-task of score infilling.

Figure 2: The result of subtracting the grooving pattern similarity of
($C_{\text{past}}$, $C_{\text{new}}$) from that of ($C_{\text{new}}$,
$C_{\text{future}}$).

##  2 Experiment

While there are many ways to choose the location $p$ for content insertion, in
our work, we utilize the VLI model to expand short-length musical segments at
musical boundaries. A musical boundary is viewed as the edge of two musical
groups [3], which is often accompanied by changes in rhythm, metre and pitch
patterns. It should be expected in general that there is still a musical
boundary after expansion. This is the focus of our experiment.

Specifically, we choose 20 pieces of 12-bar musical segments from the AILabs-
Pop1k7 dataset [2], which are MIDI transcriptions of polyphonic pop piano
performances. The detailed settings of the VLI model is identical to our
previous work [1]. For each segment, we use pre-trained VLI to insert 4-bar
novel content at a manually-chosen boundary which is located at the edge of
two _musical phrases_. MIDI files of all the original and expanded segments
can be found at the demo website.222https://tanchihpin0517.github.io/variable-
length-piano-expansion/

##  3 Evaluation

We use two metrics in our evaluation. We hypothesize that, if there is a
sudden change in _rhythm_ patterns at $p$ going from $C_{\text{past}}$ to
$C_{\text{future}}$, such a sudden change should be preserved within the
generated $C_{\text{new}}$. To quantify this, we use the grooving pattern
similarity ($\mathcal{GS}$) proposed in [4] to compute the similarity in
rhythm between $\\{C_{\text{past}},C_{\text{new}}\\}$ (denoted as
$\mathcal{GS}_{1}$) and between $\\{C_{\text{new}},C_{\text{future}}\\}$
($\mathcal{GS}_{2}$), respectively, and then compare the scores by subtracting
the former from the latter (i.e., $\mathcal{GS}_{2}-\mathcal{GS}_{1}$). A
positive subtraction result indicates that the rhythm pattern of
$C_{\text{new}}$ is closer to that of $C_{\text{future}}$, otherwise to
$C_{\text{past}}$. Besides, when there is a boundary, we expect the absolute
value of the subtraction result to be a large number, assuming that only
$\mathcal{GS}_{1}$ or $\mathcal{GS}_{2}$ would take a large value, but not
both.

Across boundaries, there will also be changes in _pitch_ patterns due to,
e.g., splitting a complete chord into arpeggios or shifting melody line to
another octave or key. To reflect these, we introduce a new metric called
register histogram similarity ($\mathcal{RHS}$). Given a sequence of notes, we
count the number of notes in each octave and construct a 7-dimensional
histogram $\vec{h}$ from C1 to C7. Given the histograms
$\vec{h}_{1},\vec{h}_{2}$ of two segments (e.g.,
$\\{C_{\text{new}},C_{\text{future}}\\}$), we evaluate their similarity by
their negative cross entropy (which lies in $(-\infty,0]$, the closer to zero
the more similar):

$\mathcal{RHS}(\vec{h}_{1},\vec{h}_{2})=\sum\nolimits_{i=0}^{7}h_{1,i}\,\log_{2}(h_{2,i})\,.$
(1)

We similarly quantify the degree of changes in pitch by computing the
$\mathcal{RHS}$ between $\\{C_{\text{past}},C_{\text{new}}\\}$ and between
$\\{C_{\text{new}},C_{\text{future}}\\}$, respectively, and then subtracting
them.

Figure 3: The result of subtracting the register histogram similarity of
($C_{\text{past}}$, $C_{\text{new}}$) from that of ($C_{\text{new}}$,
$C_{\text{future}}$)

Figures 2 and 3 show the result in $\mathcal{GS}$ and $\mathcal{RHS}$,
respectively. We observe that the VLI model tends to keep the boundary in two
ways: i) making $C_{\text{new}}$ an extended content of $C_{\text{past}}$ or
$C_{\text{future}}$ by imitating one of them, or ii) making $C_{\text{new}}$
an independent segment which plays the role of a bridge connecting
$C_{\text{past}}$ and $C_{\text{future}}$. The boundary is often preserved
well if the absolute subtraction result in $\mathcal{GS}$ or $\mathcal{RHS}$
is large, but not vice versa—informal listening shows that VLI is able to
preserve the boundary even when the absolute subtraction results in both
$\mathcal{GS}$ and $\mathcal{RHS}$ are low. We also find that VLI tends to
make $C_{\text{new}}$ more similar to $C_{\text{future}}$ than to
$C_{\text{past}}$, possibly because it is harder to extend a segment which has
already a proper sense of ending, which is often the case for
$C_{\text{past}}$ in pop music.

##  4 Conclusion

In this paper, we have shown that the VLI model performs well for music score
expansion in certain conditions. However, we currently use human-annotated
boundaries for generation tasks. Selecting a target place for expanding is
still an ongoing research topic. We aim to integrate music-structure analysis
into the model and make it capable to decide where to insert content.
Moreover, the metrics in our work are rather simplistic, describing the
properties of boundaries fairly roughly. Finding other proper metrics is also
one of our next steps.

## References

  * [1] C.-J. Chang _et al._ , “Variable-length music score infilling via XLNet and musically specialized positional encoding,” in _Proc. ISMIR_ , 2021.
  * [2] W.-Y. Hsiao _et al._ , “Compound Word Transformer: Learning to compose full-song music over dynamic directed hypergraphs,” in _Proc. AAAI_ , 2021\.
  * [3] F. Lerdahl and R. S. Jackendoff, _A Generative Theory of Tonal Music_. MIT Press, 1996.
  * [4] S.-L. Wu and Y.-H. Yang, “The Jazz Transformer on the front line: Exploring the shortcomings of AI-composed music through quantitative measures,” in _Proc. ISMIR_ , 2020.