# Prompt-and-Align: Prompt-Based Social Alignment for Few-Shot Fake News
Detection

Jiaying Wu  National University of Singapore<EMAIL_ADDRESS>,  Shen Li
National University of Singapore<EMAIL_ADDRESS>,  Ailin Deng  National
University of Singapore<EMAIL_ADDRESS>,  Miao Xiong  National University of
Singapore<EMAIL_ADDRESS>and  Bryan Hooi  National University of
Singapore<EMAIL_ADDRESS>
(2023)

###### Abstract.

Despite considerable advances in automated fake news detection, due to the
timely nature of news, it remains a critical open question how to effectively
predict the veracity of news articles based on limited fact-checks. Existing
approaches typically follow a “Train-from-Scratch” paradigm, which is
fundamentally bounded by the availability of large-scale annotated data. While
expressive pre-trained language models (PLMs) have been adapted in a “Pre-
Train-and-Fine-Tune” manner, the inconsistency between pre-training and
downstream objectives also requires costly task-specific supervision. In this
paper, we propose “Prompt-and-Align” ($\mathsf{P\&A}$), a novel prompt-based
paradigm for few-shot fake news detection that jointly leverages the pre-
trained knowledge in PLMs and the social context topology. Our approach
mitigates label scarcity by wrapping the news article in a task-related
textual prompt, which is then processed by the PLM to directly elicit task-
specific knowledge. To supplement the PLM with social context without inducing
additional training overheads, motivated by empirical observation on _user
veracity consistency_ (i.e., social users tend to consume news of the same
veracity type), we further construct a news proximity graph among news
articles to capture the veracity-consistent signals in shared readerships, and
align the prompting predictions along the graph edges in a confidence-informed
manner. Extensive experiments on three real-world benchmarks demonstrate that
$\mathsf{P\&A}$ sets new states-of-the-art for few-shot fake news detection
performance by significant margins. 111Data and code are available at:
https://github.com/jiayingwu19/Prompt-and-Align.

Fake News; Social Networks; Few-Shot Learning; Prompt

††journalyear: 2023††copyright: rightsretained††conference: Proceedings of the
32nd ACM International Conference on Information and Knowledge Management;
October 21–25, 2023; Birmingham, United Kingdom††booktitle: Proceedings of the
32nd ACM International Conference on Information and Knowledge Management
(CIKM ’23), October 21–25, 2023, Birmingham, United Kingdom††doi:
10.1145/3583780.3615015††isbn: 979-8-4007-0124-5/23/10††ccs: Information
systems Data mining††ccs: Computing methodologies Natural language
processing††ccs: Information systems Social networks

##  1\. Introduction

The proliferation of fake news online poses an imperative concern for human
cognition (Ecker et al., 2022; Roberts et al., 2021) and social development
(Pogue, 2017; T and Mathew, 2022). Given the timeliness trait of news stories
(Galtung and Ruge, 1965), it is crucial that automated fake news detection
applications enable accurate few-shot veracity predictions based on limited
related fact-checks.

Nevertheless, the success of existing approaches is usually contingent on
access to abundant fact-checked articles and auxiliary features, which is not
guaranteed in practice. Regardless of whether utilizing the news content (Zhou
et al., 2020; Shu et al., 2019) or the social context graph (Cui et al., 2020;
Nguyen et al., 2020), the majority of methods adopt a “Train-from-Scratch”
paradigm, where weight optimization is solely dependent on the supervised
training data. Consequently, these methods typically require large-scale
labeled news articles (Rashkin et al., 2017; Shu et al., 2018) and incorporate
auxiliary information that are laborious to retrieve, including stance
annotations (Nguyen et al., 2020), user history posts (Dou et al., 2021) and
knowledge bases (Cui et al., 2020; Dun et al., 2021; Hu et al., 2021). Under
label scarcity, these methods suffer from generalization issues. To address
this challenge, pre-trained language models (PLMs) (Devlin et al., 2019; Liu
et al., 2019) have been adapted to the task following a “Pre-Train-and-Fine-
Tune” paradigm (Cruz et al., 2020; Li and Zhou, 2020; Pelrine et al., 2021),
where a task-specific classification head is stacked upon a PLM. While methods
under this paradigm benefit from the pre-trained syntactic and semantic
knowledge, optimizing the auxiliary layers still requires abundant high-
quality annotations, and fine-tuning a PLM alongside a randomly initialized
task-specific architecture has been shown to distort the high-quality pre-
trained features and impair model robustness (Hendrycks et al., 2020; Kumar et
al., 2022). Under both paradigms, the real-world label scarcity of emerging
news events creates a fundamental hurdle for weight optimization, resulting in
substantial performance degradation.

Figure 1. Comparison of our proposed “Prompt-and-Align” ($\mathsf{P\&A}$) with
existing paradigms.

In this work, we develop “Prompt-and-Align” ($\mathsf{P\&A}$), a novel prompt-
based paradigm for few-shot fake news detection. Inspired by recent advances
in prompt-based learning that exploit PLMs as powerful few-shot learners in
various natural language processing tasks (Li and Liang, 2021; Petroni et al.,
2019; Qin and Eisner, 2021; Schick and Schütze, 2021a), we re-formulate fake
news detection as a task-oriented text completion problem embedded in a
natural language prompt. As illustrated in Figure 1, in contrast to existing
“Train-from-Scratch” and “Pre-Train-and-Fine-Tune” paradigms that incorporate
task-specific architectures, $\mathsf{P\&A}$ utilizes a textual prompt that
encodes task-related knowledge. Prompting establishes semantic relevance
between the task and the PLM (Liu et al., 2023), elicits the latent “built-in”
knowledge from PLMs for task-specific inference (Jiang et al., 2020), and
thereby effectively alleviates the label scarcity bottleneck.

On the basis of few-shot prompting, how can we further incorporate knowledge
from the social context? An intuitive solution would be to combine prompting
outputs with a Graph Neural Network (e.g., GCN (Kipf and Welling, 2017)) on
the social graph. However, this does not address the problem, as the
significant gap between the PLM pre-training objective and the downstream GNN
classification objective can impair model performance. Across a sparsely
labeled social graph, only a small portion of unlabeled nodes are involved in
message passing, which fundamentally limits the effectiveness of the GNN.

To leverage informative structural patterns from the social context without
inducing additional training overheads, we investigate the news consumption
preference of social media users, and make a key observation on user veracity
consistency, where users are consistently attracted to news articles of a
certain veracity type (i.e., real or fake). In other words, active social
users connect multiple news articles with veracity-consistent signals via user
engagements. Motivated by this empirical finding, we construct a news
proximity graph to connect news articles with shared readership. To fully
utilize the limited number of high-quality labels and high-confidence
predictions, we first enhance the prompting predictions with ground-truth
training data labels and soft labels generated via pseudo labeling (Lee,
2013), and then align the independent predictions over the veracity-consistent
graph edges, which further explicitly regularizes predictions of unlabeled
samples to alleviate label scarcity.

To summarize, our contributions are three-fold:

  * •

Empirical Finding: We present a novel finding, on how user veracity
consistency leads to shared veracity between news articles with large common
audiences.

  * •

Method: We propose “Prompt-and-Align” ($\mathsf{P\&A}$), a novel paradigm for
few-shot fake news detection that combines the benefits of both prompting and
veracity-guided social alignment.

  * •

Effectiveness: Extensive experiments show that $\mathsf{P\&A}$ significantly
enhances fake news detection accuracy by $8.93\%$, $11.01\%$, $6.36\%$ and
$5.74\%$ on average under training set sizes $16$, $32$, $64$, and $128$,
respectively.

##  2\. Related Work

Our work brings together two active lines of research.

###  2.1. Fake News Detection on Social Media

Deep learning models have shown impressive capacity for learning news
representations. Existing methods can be generally categorized into two
paradigms based on their training schemes: The dominant “Train-from-Scratch”
paradigm employs various neural architectures including Recurrent Neural
Networks (RNNs) (Ruchansky et al., 2017; Shu et al., 2019; Wu et al., 2019),
Convolutional Neural Networks (CNNs) (Wu and Hooi, 2022; Zhou et al., 2020)
and Graph Neural Networks (GNNs) (Vaibhav et al., 2019; Monti et al., 2019; Wu
and Hooi, 2023) to learn semantic and structural representations. To further
enhance model prediction, existing methods incur high costs in retrieving
various sources of auxiliary information including large-scale user response
(Ruchansky et al., 2017), entity descriptions from knowledge bases (Dun et
al., 2021; Hu et al., 2021), open-domain evidence (Sheng et al., 2021), news
producer description (Nguyen et al., 2020) and social media profiles (Dou et
al., 2021). Utilization of PLMs (Cruz et al., 2020; Li and Zhou, 2020; Pelrine
et al., 2021), specifically fine-tuning them on annotated news samples, opened
a new era of “Pre-Train-and-Fine-Tune” with enhanced generalizability.
However, existing methods typically incorporate task-specific architectures
alongside a PLM, which require abundant labeled samples to optimize. Although
initial progress has been made in meta-learning based multimodal fake news
detection (Wang et al., 2021) involving news articles and images, the
informative social context remains unconsidered. Hence, in this paper, we
study the task of few-shot fake news detection on social media, which aims to
effectively predict news veracity given a small number of annotated news
samples and minimal social context (i.e., the numeric IDs of related social
users).

###  2.2. Prompt-Based Learning

Recent years have realized PLMs (Brown et al., 2020; Devlin et al., 2019; Liu
et al., 2019) as a strong impetus for advancing natural language
understanding. Among PLM-based approaches, prompt-based learning has shown
promising potential under few-shot scenarios. As per the investigations of
(Petroni et al., 2019; Jiang et al., 2020), PLMs acquire abundant factual and
commonsense knowledge during their pre-training stage, which can be elicited
by re-formulating downstream tasks into text completion questions, either with
manually pre-defined templates (Schick and Schütze, 2021a; Tam et al., 2021)
or with patterns learned on a small training set (Li and Liang, 2021; Qin and
Eisner, 2021). Guided by this knowledge, prompt-based methods have achieved
impressive breakthroughs in a wide range of tasks including graph learning (He
et al., 2023), sentiment classification (Gao et al., 2021), natural language
inference (Schick and Schütze, 2021b), relation extraction (Chen et al.,
2022), and stance detection (Hardalov et al., 2022). Despite preliminary
investigation into knowledge-based prompting for fake news classification
(Jiang et al., 2022) (i.e., news documents augmented by auxiliary entity
descriptions from a knowledge base as the PLM input), existing efforts
overlook the informative social graph topology. On the related front of rumor
detection, recent work leverages prompt learning for zero-shot transfer
learning (Lin et al., 2022). However, (Lin et al., 2022) focuses on learning
language-agnostic contextual representations across different domains, which
is inherently orthogonal to our contributions. In this work, we propose a
novel “Prompt-and-Align” ($\mathsf{P\&A}$) few-shot paradigm that addresses
the generalization challenge by prompting the PLMs for task-related knowledge,
and further alleviates the label scarcity issue via incorporating news
readership patterns.

##  3\. Problem Definition

Let $\mathcal{D}$ be a news dataset containing $N$ samples. $\mathcal{D}$
consists of the following components: (1) a set of news articles $\mathcal{T}$
containing only textual information, with $|\mathcal{T}|=N$; (2) a set of
related social media users $\mathcal{U}$ who have reposted at least one
article in $\mathcal{T}$; and (3) a set of user engagements $\mathcal{R}$ in
terms of user reposts, where each engagement is formulated as
$\\{(u,T,r)|u\in\mathcal{U},T\in\mathcal{T}\\}$ (i.e., user $u$ has engaged in
spreading article $T$ via $r$ different reposts). Here, we assume minimal
access to user attributes, restricting $\mathcal{U}$ to only a set of unique
user IDs. For the few-shot setting, the article set $\mathcal{T}$ is split
into a small labeled training set $\mathcal{T}_{L}$ of size $n$ and a large
unlabeled test set $\mathcal{T}_{U}$. The articles in $\mathcal{T}_{L}$ are
annotated with the corresponding one-hot labels
$\mathcal{Y}_{L}=\\{y_{1},y_{2},\ldots,y_{n}\\}$, where
$y_{i}\in\mathbb{R}^{C}$. In line with prior work (Cui et al., 2020; Nguyen et
al., 2020; Shu et al., 2019), we formulate fake news detection as a binary
classification problem between the real and fake classes (i.e., $C=2$).

Based on the above-mentioned notations, we formulate our task as follows:

Few-Shot Fake News Detection on Social Media: Input: news dataset
$\mathcal{D}=\\{\mathcal{T},\mathcal{U},\mathcal{R}\\}$, training labels
$\mathcal{Y}_{L}$; Output: predicted labels $\mathcal{Y}_{U}$.

Figure 2. Overview of “Prompt-and-Align” ($\mathsf{P\&A}$) paradigm. Veracity
of the news article node with a question mark is determined by: (1) prompt-
based predictions; and (2) social alignment over user engagements. News
article nodes are connected by a news proximity graph (see Section 4.2), and
darker node colors denote higher confidence.

##  4\. Proposed Approach

In this section, we present our proposed “Prompt-and-Align” ($\mathsf{P\&A}$)
paradigm for few-shot fake news detection, overviewed in Figure 2.
$\mathsf{P\&A}$ consists of two major components, namely (1) a “Prompt”
component (Section 4.1) that elicits task-specific knowledge from the PLM to
predict news article veracity; and (2) an “Align” component motivated by our
empirical finding on user veracity consistency (Section 4.2), which leverages
the informative news readership patterns to enhance the prompting predictions
via confidence-informed alignment over the social graph (Section 4.3).

###  4.1. Prompting for Task-Specific Knowledge

(a) PolitiFact (b) GossipCop (c) FANG

Figure 3. Social users exhibit a clear preference for either spreading fake
news or real news. The fake news affinity score is formulated in Section 4.2.

The crucial factor in effectively connecting the PLM pre-training objective
(i.e., masked language modeling) with the downstream fake news detection
objective lies in identifying a shared template applicable to both tasks. To
this end, we first re-formulate fake news detection as a masked token
prediction problem.

Let $\mathcal{M}$ be a PLM with a vocabulary $\mathcal{V}$. Contrary to
standard classification methods, which assign news articles to a label logit
without any inherent meaning, prompting enables us to solve the fake news
detection problem as follows: Given a news article $T_{i}$, we firstly
construct a corresponding natural language prompt $\tilde{T}_{i}$ containing a
mask token:

(1) $\tilde{T}_{i}=\underline{\mathsf{[MASK]}}:T_{i}.$

Taking $\tilde{T}_{i}$ as input, $\mathcal{M}$ produces a score vector
$S(\tilde{T}_{i})\in\mathbb{R}^{|\mathcal{V}|}$ over $\mathcal{V}$ at the
masked position to complete the text. For token $v\in\mathcal{V}$, the
corresponding score logit $S_{v}(\tilde{T}_{i})$ is computed as:

(2) $S_{v}(\tilde{T}_{i})=\mathcal{M}(v|\tilde{T}_{i}).$

Here, the PLM is prompted to answer what is the most likely token that fits
the $\mathsf{[MASK]}$ in $\tilde{T}_{i}$.

Given $S(\tilde{T}_{i})$, we define the answer space $\mathcal{A}=${“news”,
“rumor”} as a small subset of the vocabulary containing permissible answers
for our fake news detection prompt. Then, we map $\mathcal{A}$ to the binary
labels following “news”$\rightarrow 0$ (real) and “rumor”$\rightarrow 1$
(fake), respectively.

To retrieve the conditional probability assigned to each answer token, we
convert $S(\tilde{T}_{i})$ into a conditional probability distribution
$P(\tilde{T}_{i})\in\mathbb{R}^{|\mathcal{V}|}$ over all vocabulary tokens via
the softmax function. For answer token $y\in\mathcal{A}$, we extract its
probability score $P_{y}(\tilde{T}_{i})$:

(3)
$P_{y}(\tilde{T}_{i})=\frac{\exp(S_{y}(\tilde{T}_{i}))}{\sum_{v\in\mathcal{V}}\exp(S_{v}(\tilde{T}_{i}))}.$

Intuitively, our prompt-based approach incorporates task-specific semantic
knowledge by prepending a task-related textual template to the input, and
queries the PLM for whether it associates an article’s text with news or
rumor, by asking it to fill in the masked token with either “news” or “rumor”.
This helps directly elicit the model’s task-specific knowledge, which
significantly reduces the amount of training data it needs to learn
effectively.

Training objective. The highest-scoring token in answer space $\mathcal{A}$
implies the PLM’s prediction of news veracity. Hence, given the PLM
$\mathcal{M}$ and our textual prompt $\tilde{T}_{i}$, the goal is to maximize
the probability score of the correct answer token $y_{+}$ and penalize the
incorrect token $y_{-}$. For instance, real news have $y_{+}$ as “news” and
$y_{-}$ as “rumor”. The cross-entropy (CE) loss over answer space
$\mathcal{A}$ is unsuitable here, as it only focuses on the two logits that
correspond to the two answer tokens in $\mathcal{A}$. Hence, it fails to
suppress the logits corresponding to non-answer tokens (e.g., task-irrelevant
tokens such as “coffee” can be assigned a large logit for filling in
$\mathsf{[MASK]}$), resulting in a suboptimal learned model. If we compute the
CE loss over the entire vocabulary, this loss does not distinguish between
$y_{-}$ and other non-answer tokens, and thus cannot specifically suppress
$y_{-}$. To address these issues, inspired by the decoupling label loss (Tam
et al., 2021), we adopt a loss function that simultaneously discourages non-
answer tokens and keeps a specific focus on the answer token probabilities
extracted from $P(\tilde{T}_{i})$ (see Eq. 3). The loss can be formulated with
the binary cross entropy (BCE) loss as:

(4)
$\mathcal{L}=\frac{1}{n}\sum_{i=1}^{n}\left(\mathsf{BCE}(P_{y_{+}}(\tilde{T}_{i}),1)+\mathsf{BCE}(P_{y_{-}}(\tilde{T}_{i}),0)\right),$

where $n$ is the number of annotated training samples.

Base prediction acquisition. To adapt the pre-trained weights to our fake news
detection problem, we tune the parameters of $\mathcal{M}$ by minimizing the
above loss function (Eq. 4) with the textual prompt, and retrieve a set of
“base prediction” scores w.r.t. the $C$ class label tokens (mapped from the
answer tokens) for all $N$ news articles in our dataset, denoted as
$P_{\mathcal{A}}(\mathcal{T})\in\mathbb{R}^{N\times C}$. As the scores are
extracted from a probabilistic distribution over all vocabulary tokens, we
perform a softmax function to focus on the distribution over our task-specific
answer space $\mathcal{A}$, formulated as a base prediction matrix
$\mathbf{P}\in\mathbb{R}^{N\times C}$:

(5) $\mathbf{P}=\mathsf{Softmax}(P_{\mathcal{A}}(\mathcal{T})).$

At this point, we have $N$ independent prompting predictions obtained by
probing the PLM with task-specific natural language prompts. Despite the rich
semantic knowledge encapsulated in the PLM, the PLM does not contain social
knowledge in terms of social graph topology. This motivates our innovations in
further integrating auxiliary social context knowledge, specifically via
distilling crowd wisdom from news readerships.

###  4.2. Veracity-Consistent Readership Modeling

Social context, usually in the form of graphs, constitutes a distinctive
property of the fake news detection problem (Shu et al., 2017). In this
subsection, we conduct preliminary analysis of real-world news engagements by
social media users, with the aim of investigating the veracity-related social
properties. Specifically, we explore the following question: are there any
connections between social users’ news engagements and news veracity?

To measure the social users’ news consumption preference in terms of news
veracity, we compute a “fake news affinity” (FNA) score for user
$u\in\mathcal{U}$, defined as:

###### Definition 1 (Fake News Affinity).

$FNA(u)=\frac{\text{\\# of fake news engagements by }u}{\text{\\# of news
engagements by }u}.$

Empirical observation. To capture patterns representative of user engagement
preference, we set a user engagement threshold $t_{u}=5$, and focus solely on
the active social users with at least $t_{u}$ engagements in spreading news
articles. In Figure 3, we visualize the FNA score distribution of active
social users with a histogram, and make the following observation:

###### Observation 1 (User Veracity Consistency).

Active social users with numerous news engagements tend to have FNA scores
either approaching $0$ (only engage in spreading real news) or $1$ (only
engage in spreading fake news).

Our observation echoes the confirmation bias theory (Nickerson, 1998) for news
consumption, where people tend to seek and retain information that reinforces
their prior beliefs. Specifically, in the fake news detection task,
confirmation bias can be manifested in terms of coordinated group behavior
(Sharma et al., 2019), specifically with the aim to manipulate opinions on
social platforms. For instance, social media has been shown to create opinion
polarization towards political events (Kubin and von Sikorski, 2021), and
users can coordinate boundaries to gain control over celebrity discussions on
gossip sites (McNealy and Mullis, 2019). Consequently, user attention tends to
become highly segregated on certain opinions, resulting in repeated
engagements in news articles of a similar veracity type.

As an active social user engages in spreading numerous news pieces, the set of
engaged news articles is connected with veracity-consistent signals.
Therefore, if two news articles are closely connected with a large shared
readership, the articles are highly likely to have the same veracity label.

News Proximity Graph. Guided by our empirical observation, we construct a news
proximity graph $\mathcal{G}$ that encodes the shared readerships between new
article nodes $\mathcal{T}$. Let
$\mathbf{A}_{\mathcal{T}}\in\mathbb{R}^{N\times N}$ be the adjacency matrix of
$\mathcal{G}$ that quantifies shared readerships, which we will next derive
from the set $\mathcal{R}$ of user repost records.

To focus on the active social users, we set a user engagement threshold
$t_{u}$ to remove the users with fewer than $t_{u}$ news reposts. From
$\mathcal{R}$, we construct a user engagement matrix
$\mathbf{B}\in\mathbb{R}^{|\mathcal{U}^{\prime}|\times N}$, where
$\mathcal{U}^{\prime}\subseteq\mathcal{U}$ filtered via user engagement
threshold $t_{u}$ denotes the active user set, and element $\mathbf{B}^{km}$
denotes the engagement intensity of user $u_{k}\in\mathcal{U}^{\prime}$
towards news article $T_{m}\in\mathcal{T}$. The value of $\mathbf{B}^{km}$ is
obtained from $r_{km}$ in the corresponding entry
$(u_{k},T_{m},r_{km})\in\mathcal{R}$ in the user engagement set.

On the basis of user engagement matrix $\mathbf{B}$, we formulate the news
proximity matrix $\mathbf{A}_{n}\in\mathbb{R}^{N\times N}$ as
$\mathbf{B}^{\top}\mathbf{B}$. Then, we conduct normalization on
$\mathbf{A}_{n}$ to derive:

(6)
$\mathbf{A}_{\mathcal{T}}=\mathbf{D}_{n}^{-\frac{1}{2}}\mathbf{A}_{n}\mathbf{D}_{n}^{-\frac{1}{2}}.$

###  4.3. Confidence-Informed Social Alignment

To inject social context into the prompting predictions, motivated by our
empirical finding on user veracity consistency (Section 4.2), we construct a
news proximity graph $\mathcal{G}$ to connect news article nodes with
veracity-consistent signals from shared readerships. However, training a task-
oriented GNN (e.g., GCN (Kipf and Welling, 2017)) on $\mathcal{G}$ is
inapplicable under label sparsity, as the involvement of unlabeled nodes is
heavily limited during message passing, resulting in poor model performance.
To fully exploit label knowledge and social graph structure without inducing
additional trainable modules, we propose a social alignment component that
combines the base predictions in a confidence-informed manner. In addition to
prompting, our graph-based alignment further bridges the gap between general
PLM pre-training and the downstream fake news detection task, which enhances
the effectiveness of our approach.

####  4.3.1. Label Knowledge Acquisition.

We first discuss how our alignment component acquires knowledge from the
relatively “cheap” base predictions $\mathbf{P}$ derived from task-specific
prompting. The key idea is that we leverage the ground truth training labels
and a small subset of unlabeled samples (i.e., news articles) with high
prediction confidence. To this end, we transform $\mathbf{P}$ into an enhanced
confidence matrix $\mathbf{H}\in\mathbb{R}^{N\times C}$. Specifically,
$\mathbf{H}$ consists of $\mathbf{H}_{L}\in\mathbb{R}^{n\times C}$ and
$\mathbf{H}_{U}\in\mathbb{R}^{(N-n)\times C}$, which respectively refer to the
enhanced confidence scores of labeled and unlabeled samples. $\mathbf{H}$ is
computed via the following two steps.

Incorporation of ground-truth knowledge. To fully exploit the high-quality
annotated training samples $\mathcal{T}_{L}$, we stack the one-hot training
data labels $\mathcal{Y}_{L}$ into a ground truth matrix
$\mathbf{Y}_{L}\in\mathbb{R}^{n\times C}$, and utilize the ground-truth class
probabilities to replace the base predictions
$\mathbf{P}_{L}\in\mathbb{R}^{n\times C}$ pertaining to the training data. The
corresponding rows in $\mathbf{H}$ is formulated as:

(7) $\mathbf{H}_{L}=\mathbf{Y}_{L}.$

Thresholded pseudo labeling. The prompting module of $\mathsf{P\&A}$ enables
access to abundant soft labels assigned by the PLM, presented in the form of
base predictions $\mathbf{P}_{U}$ for unlabeled data $\mathcal{T}_{U}$. To
fully utilize the soft labels, we devise a thresholded variant of the pseudo
labeling technique (Lee, 2013), denoted as $\mathsf{ThresholdedPL}(\cdot)$.
Given $\mathbf{P}_{U}$, we select a set of samples with high class probability
scores equal to or above the $t_{p}$-th percentile of all predicted confidence
scores in $\mathbf{P}_{U}$. Following this, we assign these high-confidence
samples with one-hot labels w.r.t. the class with maximum predicted
probability for each sample, while the remaining low-confidence samples are
kept unchanged. We augment $\mathbf{P}_{U}$ with the resultant pseudo labels
to obtain enhanced confidence scores $\mathbf{H}_{U}$:

(8) $\mathbf{H}_{U}=\mathsf{ThresholdedPL}(\mathbf{P}_{U}).$

####  4.3.2. Veracity-Guided Prediction Alignment

Knowledge-infused predictions in $\mathbf{H}$ facilitate effective utilization
of the limited high-quality ground truth labels and predictions. Hence, given
$\mathbf{H}$ and the veracity-consistent edges of news proximity graph
$\mathcal{G}$, we conduct veracity-guided social alignment to combine the
independent predictions in a confidence-informed manner.

Specifically, we encourage label smoothness over the graph structure by
aligning $\mathbf{H}$ via propagation over the $k$-hop neighborhood of each
news article node. From the propagated scores, we obtain the final aligned
predictions $\mathbf{\hat{Y}}$:

(9) $\mathbf{\hat{Y}}=A_{\mathcal{T}}^{k}\mathbf{H},$

among which the predicted class label of news article
$T_{m}\in\mathcal{T}_{U}$ is assigned by

(10)
$\hat{y}_{m}=\mathsf{argmax}_{j\in\\{1,\ldots,C\\}}\hat{\mathbf{Y}}_{mj}.$

Here, the obtained $\hat{y}_{m}$ aggregates social knowledge (specifically,
veracity signals) through our proposed social alignment component, which
thereby refines the base predictions derived from prompting.

Table 1. Dataset statistics. Dataset | PolitiFact | GossipCop | FANG
---|---|---|---
# News Articles | 482 | 1,736 | 664
# Real News | 241 | 868 | 332
# Fake News | 241 | 868 | 332
# User Engagements | 236,950 | 163,532 | 50,549
# Distinct Users | 148,023 | 79,474 | 37,165
Table 2. Performance comparison between $\mathsf{P\&A}$ and baselines in terms of fake news detection accuracy (%) at varying training set sizes ($n\in\\{16,32,64,128\\}$). Bold (underline) indicates the best overall (baseline) performance. ∗ denotes that improvements of $\mathsf{P\&A}$ over the most competitive baselines are significant at $p<0.01$ level under the Wilcoxon signed-rank test (Wilcoxon, 1945). Method | PolitiFact |  | GossipCop |  | FANG
---|---|---|---|---|---
16 | 32 | 64 | 128 |  | 16 | 32 | 64 | 128 |  | 16 | 32 | 64 | 128
dEFEND\c (Shu et al., 2019) | 51.78 | 54.62 | 61.06 | 66.34 |  | 50.43 | 50.35 | 51.48 | 52.66 |  | 50.19 | 50.85 | 50.91 | 54.60
SAFE\v (Zhou et al., 2020) | 51.71 | 54.52 | 61.91 | 68.36 |  | 51.35 | 51.20 | 53.39 | 57.40 |  | 51.90 | 53.01 | 55.53 | 58.35
SentGCN (Vaibhav et al., 2019) | 56.34 | 51.95 | 56.81 | 56.23 |  | 49.57 | 49.33 | 50.04 | 53.87 |  | 51.35 | 50.60 | 52.58 | 54.45
SentGAT (Vaibhav et al., 2019) | 56.09 | 52.99 | 55.64 | 58.36 |  | 49.54 | 50.11 | 50.52 | 54.65 |  | 51.04 | 51.59 | 54.09 | 56.34
FANG (Nguyen et al., 2020) | 57.87 | 60.45 | 68.72 | 77.56 |  | 51.73 | 54.10 | 63.93 | 69.65 |  | 53.32 | 56.22 | 58.53 | 60.49
GCNFN (Schick and Schütze, 2021a) | 55.37 | 61.86 | 70.06 | 84.46 |  | 52.18 | 53.85 | 62.29 | 71.89 |  | 52.43 | 56.08 | 60.12 | 61.85
BERT-FT (Devlin et al., 2019) | 61.27 | 67.48 | 73.52 | 77.43 |  | 52.45 | 52.74 | 55.02 | 59.28 |  | 53.28 | 54.71 | 57.02 | 58.41
RoBERTa-FT (Liu et al., 2019) | 54.80 | 61.16 | 78.95 | 81.44 |  | 52.54 | 54.17 | 54.09 | 61.35 |  | 51.47 | 54.56 | 56.92 | 60.92
PET (Schick and Schütze, 2021a) | 64.24 | 68.09 | 79.44 | 80.49 |  | 53.69 | 55.11 | 59.78 | 62.97 |  | 55.76 | 56.46 | 59.13 | 59.79
KPT (Hu et al., 2022) | 68.27 | 70.21 | 80.40 | 83.16 |  | 53.95 | 54.70 | 60.12 | 62.19 |  | 56.87 | 55.79 | 60.36 | 61.38
$\mathsf{P\&A}$ (ours) | 85.30∗ | 83.73∗ | 86.79∗ | 89.45∗ |  | 61.15∗ | 70.59∗ | 74.62∗ | 81.60∗ |  | 59.42∗ | 60.48∗ | 62.37∗ | 64.37∗

##  5\. Experiments

In this section, we empirically evaluate our “Prompt-and-Align”
($\mathsf{P\&A}$) paradigm to investigate the following five research
questions:

  * •

Few-Shot Performance (Section 5.2): How effective is $\mathsf{P\&A}$ in few-
shot fake news detection?

  * •

Ablation Study (Section 5.3): How do news content and social graph structure
contribute to the performance of $\mathsf{P\&A}$?

  * •

Parameter Sensitivity Analysis (Section 5.4): How does $\mathsf{P\&A}$ perform
under different alignment steps, pseudo labeling percentiles, and user
engagement thresholds?

  * •

Impact of Prompt Design (Section 5.5): How effective is $\mathsf{P\&A}$ across
different textual prompts?

  * •

Case Study (Section 5.6): How can we interpret the predictions of
$\mathsf{P\&A}$ via its intermediate results?

###  5.1. Experimental Setup

####  5.1.1. Datasets

We conduct evaluation on three real-world benchmark datasets commonly adopted
by existing work, namely the FakeNewsNet (Shu et al., 2018) public benchmark
consisting of the PolitiFact and GossipCop datasets, and the FANG (Nguyen et
al., 2020) dataset. All datasets contain news articles collected from leading
fact-checking websites and the related social user engagements (i.e., IDs of
repost users) retrieved from Twitter. The dataset statistics are shown in
Table 1.

We split the data into training and test sets by randomly sampling $n$ news
items as the training data (with $n\in\\{16,32,64,128\\}$), among which the
ratio of (fake news) / (real news) is set as $1:1$.

####  5.1.2. Baselines

We benchmark $\mathsf{P\&A}$ against ten representative baseline methods that
adopt the following paradigms:

“Train-from-Scratch” methods devise task-specific neural architectures for
fake news detection. dEFEND\c is a variant of the hierarchical attention
framework dEFEND (Shu et al., 2019) without user comments. SAFE\v is a variant
of SAFE (Zhou et al., 2020) without the visual component, which adopts a
TextCNN (Kim, 2014) based module to encode the news article texts. SentGCN and
SentGAT (Vaibhav et al., 2019) are graph-based approaches that respectively
employ the Graph Convolutional Network (GCN) (Kipf and Welling, 2017) and the
Graph Attention Network (GAT) (Veličković et al., 2018) to capture indicative
sentence interaction patterns. GCNFN (Monti et al., 2019) utilizes deep
geometric learning to model news dissemination patterns along with textual
node embedding features. FANG (Nguyen et al., 2020) constructs a heterogeneous
social graph with news articles, sources and social users, and adopts a
GraphSAGE (Hamilton et al., 2017) based framework to detect fake news. For a
fair comparison, social context based methods are implemented with the
components for encoding the news content, user-news relations and social user
identities.

PLM-based methods leverage the rich pre-trained knowledge in PLMs to mitigate
label scarcity. Among this category, “Pre-Train-and-Fine-Tune” methods BERT-FT
and RoBERTa-FT respectively combine BERT (Devlin et al., 2019) and RoBERTa
(Liu et al., 2019) models with a task-specific MLP to predict news veracity.
Prompt-tuning methods include PET (Schick and Schütze, 2021a), which provides
task descriptions to PLMs for supervised training via task-related cloze
questions and verbalizers; and KPT (Hu et al., 2022), which expands the label
word space with class-related tokens of varied granularities and perspectives.
For a fair comparison, we do not implement self-training and PLM ensemble for
PET. We utilize the base versions of BERT and RoBERTa for fine-tuning, and
adopt BERT-base as the backbone of prompt-tuning baselines, consistent with
the setup of our proposed approach.

As $\mathsf{P\&A}$ focuses on news content and social context, we do not
compare $\mathsf{P\&A}$ with knowledge-based approaches (Jiang et al., 2022;
Dun et al., 2021; Hu et al., 2021) that incorporate entity information from
external knowledge bases, which are orthogonal to our contributions.

####  5.1.3. Implementation Details

We implement $\mathsf{P\&A}$ and its variants based on PyTorch 1.8.0 with CUDA
11.1. We utilize pretrained BERT-base weights from HuggingFace Transformers
4.13.0 (Wolf et al., 2020). The maximum sequence length, batch size, and
learning rate are set to $512$, $16$, and $5\times 10^{-5}$ respectively,
consistent with (Devlin et al., 2019). We fine-tune the models for $3$ epochs
for $n\in\\{16,32\\}$, and $5$ epochs for $n\in\\{64,128\\}$. In the “Align”
module, we set the pseudo labeling threshold $t_{p}$ at the $95$-th percentile
among all test data predictions, and set the user engagement threshold $t_{u}$
to $5$. The number of alignment steps $k$ is set to $2$. For the baseline
methods, we follow the architectures and hyperparameter values suggested by
their respective authors. In all experiments, we report the average test
accuracy (%) across 20 different runs of each method.

(a) PolitiFact (b) GossipCop (c) FANG

Figure 4. Ablation study of $\mathsf{P\&A}$ at training set size $n$. “-TPL”
removes thresholded pseudo labeling (see Eq. 8), “-TPL -G” only implements the
“Prompt” component, and “BERT-FT” refers to the BERT model fine-tuned with
task-specific layers.

###  5.2. Few-Shot Detection Performance

Table 2 compares the performance of $\mathsf{P\&A}$ with competitive
baselines. We can observe that: (1) Among “Train-from-Scratch” methods, social
graph based methods (FANG and GCNFN) consistently outperform news content
based methods, which indicates the importance of incorporating social context
topology. (2) PLM-based methods outperform “Train-from-Scratch” methods in
numerous cases, showing that the informative pre-trained features in PLMs can
help to address the label scarcity issue. (3) Among PLM-based methods,
prompting methods (PET and KPT) consistently outperform the standard fine-
tuned PLMs, which justifies our analysis that prompting directly elicits task-
specific knowledge from PLMs. (4) $\mathsf{P\&A}$ substantially outperforms
the most competitive baseline. Given $n$ training samples
($n\in\\{16,32,64,128\\}$), $\mathsf{P\&A}$ respectively enhances few-shot
fake news detection accuracy by $8.93\%$, $11.01\%$, $6.36\%$ and $5.74\%$ on
average. The statistically significant performance gains validate the
effectiveness of eliciting task-specific knowledge with prompts and performing
confidence-informed prediction alignment across the news proximity graph.

###  5.3. Ablation Study

To investigate the contribution of $\mathsf{P\&A}$ components, we ablate two
important components of our approach (thresholded pseudo labeling and social
graph alignment) via the following variants:

  * •

$\mathsf{P\&A(-TPL)}$, which removes the thresholded pseudo labeling mechanism
on high-confidence samples (see Eq. 8).

  * •

$\mathsf{P\&A(-TPL-G)}$, which eliminates the social context (i.e., our news
proximity graph) and reduces to the “Prompt” component.

Note that when we only remove the graph-based social alignment steps (i.e.,
$\mathsf{(-G)}$), the results are the same as $\mathsf{(-TPL-G)}$, as
thresholded pseudo labeling does not invert any predicted labels.

Figure 4 shows the few-shot performance of different $\mathsf{P\&A}$ variants
on three datasets. We find that (1) Social alignment significantly contributes
to overall performance, which indicates that aligning confidence-enhanced
prompting predictions across our news proximity graph effectively utilizes
veracity-consistent signals in the social context. (2) Removing the
thresholded pseudo labeling mechanism leads to inferior performance than
$\mathsf{P\&A}$, which is expected as removing pseudo labels exacerbates label
sparsity on the social graph. (3) $\mathsf{P\&A(-TPL-G)}$ (i.e., our prompting
module) consistently outperforms BERT-FT, which is consistent with our
analysis that tuning the PLM via task-specific textual prompts helps to
directly elicit task-specific knowledge and alleviate label scarcity.

(a) $n=16$ (b) $n=32$ (c) $n=64$ (d) $n=128$

Figure 5. Parameter sensitivity analysis of $\mathsf{P\&A}$ on GossipCop under
different combinations of $t_{p}$ (pseudo labeling percentile) and $t_{u}$
(user engagement threshold) values. Lighter color represents higher accuracy
(z-axis, %).

###  5.4. Parameter Sensitivity Analysis

We explore the sensitivity of three important hyperparameters in
$\mathsf{P\&A}$: pseudo labeling percentile $t_{p}$ for leveraging high-
confidence predictions, user engagement threshold $t_{u}$ for filtering
inactive users, and alignment steps $k$ over the news proximity graph.

####  5.4.1. Effects of Threshold Values

To evaluate the impact of threshold values $t_{p}$ and $t_{u}$, we investigate
the performance of $\mathsf{P\&A}$ with $t_{p}\in\\{80,85,90,95\\}$ and
$t_{u}\in\\{2,3,5,10\\}$, and present the results in Figure 5. We observe
that: (1) The performance of $\mathsf{P\&A}$ remains relevantly smooth across
different variations of threshold values, with all combinations significantly
outperforming the best baselines. Our default threshold values of $t_{p}=95$
and $t_{u}=5$ consistently give satisfactory performance, which indicates that
our settings for the social alignment stage are practical for few-shot fake
news detection on social media. (2) When we gradually lower the pseudo
labeling threshold $t_{p}$, the performance of $\mathsf{P\&A}$ slightly
degrades. This is because $t_{p}$ is formulated as a percentile, with which
$\mathsf{P\&A}$ automatically determines the numeric threshold based on the
distribution of all prediction scores. Consequently, $\mathsf{P\&A}$ assigns
one-hot labels to more samples as $t_{p}$ decreases, whose inherent
uncertainty leads to error accumulation.

####  5.4.2. Effects of Graph Neighborhood Sizes

As validated by our ablation study in Section 5.3, the veracity-consistent
social context (i.e., news proximity) effectively enhances the prompting
predictions. To investigate the structural news readership patterns embedded
in the news proximity graph, we evaluate $\mathsf{P\&A}$ performance over
varying numbers of alignment steps (denoted as $k$ in Eq. 9), with
$k\in\\{1,2,3,4\\}$.

As shown in Table 3, we find that: (1) The performance of $\mathsf{P\&A}$
remains relevantly smooth across different variations of $k$, with all
combinations significantly outperforming the best baselines. Our proposed
$\mathsf{P\&A}$ adopts $k=2$ alignment steps across the news proximity graph,
which consistently yields satisfactory results. (2) In addition to 1-hop
neighborhood (i.e., between news articles with direct overlaps of
readerships), aggregating information from each article’s 2-hop and 3-hop
neighborhoods on the news proximity graph can boost the performance of
$\mathsf{P\&A}$, which indicates the effectiveness of higher-order social
context modeling. (3) Compared with incorporating 2-hop and 3-hop neighbors,
incorporating 4-hop neighborhoods leads to slight performance degradation.
Indeed, large neighborhood sizes might not be helpful for fake news detection,
as they contain intermingled veracity signals from the “fake” and “real”
classes, which obstructs prediction alignment.

Table 3. Accuracy (%) of $\mathsf{P\&A}$ over different numbers of alignment steps on the news proximity graph.  Dataset | $k$ | $n=16$ | $n=32$ | $n=64$ | $n=128$
---|---|---|---|---|---
PolitiFact | 1 | 83.15 | 82.25 | 87.03 | 90.11
2 | 85.30 | 83.73 | 86.79 | 89.45
3 | 85.44 | 83.18 | 86.38 | 88.81
4 | 85.34 | 82.44 | 85.68 | 88.32
FANG | 1 | 59.47 | 60.15 | 61.76 | 63.56
2 | 59.42 | 60.48 | 62.37 | 64.37
3 | 59.23 | 60.30 | 62.46 | 64.67
4 | 58.97 | 60.13 | 62.36 | 64.39
Figure 6. $\mathsf{P\&A}$ corrects the base predictions via confidence-aware
prediction alignment on the news proximity graph. This can be observed via the
logits assigned to the class labels throughout different alignment steps. The
node with a question mark represents a news article to be classified,
presenting an empirical illustration of how $\mathsf{P\&A}$ combines
information from (1) prompt-based predictions, (2) confidence-informed label
knowledge, and (3) social user engagements. For simplicity purposes, we only
visualize the edges with highest weights for each node (the general
illustration of $\mathsf{P\&A}$ is presented in Figure 2).

###  5.5. Impact of Prompt Design

To evaluate the effectiveness of the prompt-based paradigm utilized in
$\mathsf{P\&A}$, we explore the performance of $\mathsf{P\&A}$ across varying
prompt templates (denoted as P1-3, respectively). For a fair evaluation, the
mapping from prompting output to class labels is kept consistent across P1-3
as “news”$\rightarrow 0$ (real) and “rumor”$\rightarrow 1$ (fake). Note that
we report the results of $\mathsf{P\&A}$ using P1 in all other subsections.

  * •

P1: [MASK]: ¡news article input¿

  * •

P2: Contains [MASK]: ¡news article input¿

  * •

P3: Article with [MASK]: ¡news article input¿

From Table 4, we observe that (1) $\mathsf{P\&A}$ variants across multiple
templates consistently outperform the most competitive baseline, which
validates the effectiveness of prompting in eliciting pre-trained knowledge in
PLMs for downstream tasks. (2) With the increase of labeled samples, the
fluctuation across different prompt templates is reduced, indicating that the
need for a well-designed task-specific template is greatly eased with more
annotated data.

Table 4. $\mathsf{P\&A}$ accuracy (%) under different prompt templates. (B: best baseline acc. from Table 2; P1-3: varying prompts)  Dataset |  | $n=16$ | $n=32$ | $n=64$ | $n=128$
---|---|---|---|---|---
PolitiFact | B | 68.27 | 70.21 | 80.40 | 84.46
P1 | 85.30 | 83.73 | 86.79 | 89.45
P2 | 73.59 | 82.60 | 87.06 | 89.10
P3 | 78.57 | 81.81 | 86.98 | 89.66
FANG | B | 56.87 | 56.46 | 60.36 | 61.85
P1 | 59.42 | 60.48 | 62.37 | 64.37
P2 | 59.01 | 58.96 | 63.43 | 64.33
P3 | 59.19 | 61.03 | 62.47 | 64.66

###  5.6. Case Study

To further illustrate why our $\mathsf{P\&A}$ paradigm outperforms the most
competitive baselines, specifically regarding $\mathsf{P\&A}$’s effectiveness
of prediction alignment over the news proximity graph, we conduct a case study
involving a test sample from the PolitiFact dataset (Figure 6) that
demonstrates $\mathsf{P\&A}$’s capability of correcting base prediction
errors.

Given an evidently fake news article that claims “Donald Trump was pronounced
dead this morning” (denoted as $T_{3}$ in the Figure), the news content based
“Prompt” component of $\mathsf{P\&A}$ mistakenly assigned it with a
probability score of 0.596 for the “real” class, producing an incorrect base
prediction. However, in the news proximity graph, $T_{3}$ is closely connected
to multiple 1-hop and 2-hop neighbors ($T_{2},T_{4},T_{5},T_{6}$) with
predictions that favor the “fake” class at different confidence levels, in
line with our hypothesis that similar readership implies similar news
veracity. Among this neighborhood subgraph, the highest weighted edge connects
the misclassified $T_{3}$ with a ground truth one-hot label at $T_{2}$. The
edge weights of our news proximity graph are derived on the basis of the news
articles’ readership similarity; hence, $\mathsf{P\&A}$’s prediction alignment
across these edges facilitates prediction consistency in terms of veracity. As
we can see from the first alignment step that aggregates the prediction scores
from each node’s 1-hop neighbors, the high-confidence label knowledge from the
ground-truth node $T_{2}$ effectively propagated to $T_{3}$, correcting
$T_{3}$’s prediction scores to 0.148 for the “real” class and 0.199 for
“fake”. Besides this, the signals from other “fake” nodes
($T_{4},T_{5},T_{6}$) in the same subgraph are also reinforced via their
respective 1-hop neighbors. Specifically, consider $T_{4}$, whose base
prediction (i.e. class probabilities) only favors “fake” over “real” by a mere
0.02. Nevertheless, in the first alignment step, $T_{4}$ benefited from the
soft label knowledge from higher-confidence neighbors $T_{5}$ and $T_{6}$,
thereby gaining a more confident prediction of 0.151 for “real” and 0.245 for
“fake”. Similarly, in the second alignment step, the corrected “fake”
prediction of $T_{3}$ remains unchanged, and $T_{3}$’s veracity-related
signals are further enhanced, leading to improved results at the subsequent
final classification stage.

Our case study shows that despite the abundant pre-trained knowledge in PLMs,
base prediction errors can still arise due to the emergent nature of news
topics. This highlights the necessity of supplementing the base predictions
from PLMs with veracity-indicative evidence from the social context. Through
aligning the prompting predictions on a veracity-consistent news proximity
graph, $\mathsf{P\&A}$ effectively enhances fake news detection performance
under label scarcity, and offers potential explainability in terms of users’
news consumption preferences.

##  6\. Conclusion and Future Work

In this paper, we investigate the fake news detection problem under a
practical few-shot setting. We introduce “Prompt-and-Align” ($\mathsf{P\&A}$),
a novel prompt-based paradigm for few-shot fake news detection that jointly
leverages pre-trained knowledge from PLMs and the veracity-indicative news
readership patterns. $\mathsf{P\&A}$ alleviates the label scarcity bottleneck
by directly eliciting task-specific knowledge from a PLM via textual prompts,
and encodes the veracity-consistent user engagement patterns with a news
proximity graph. These innovations enable $\mathsf{P\&A}$ to fully exploit the
limited high-quality labels and predictions, specifically via aligning the
predictions over the news proximity graph in a confidence-informed manner.
Extensive experiments on three real-world benchmark datasets validate the
effectiveness of $\mathsf{P\&A}$ in terms of consistent performance gains
across varying hyperparameter settings and prompts. Our work demonstrates
promising potential for moving beyond the existing paradigms for fake news
detection, suggesting more focused research on generalizable prompt-based
learning frameworks and veracity-aware social context modeling schemes under
the few-shot scenario.

##  7\. Acknowledgements

This research was supported by NUS-NCS Joint Laboratory (A-0008542-00-00).

## References

  * (1)
  * Brown et al. (2020) Tom Brown, Benjamin Mann, Nick Ryder, Melanie Subbiah, Jared D Kaplan, Prafulla Dhariwal, Arvind Neelakantan, Pranav Shyam, Girish Sastry, Amanda Askell, et al. 2020\.  Language Models are Few-Shot Learners. In _Advances in Neural Information Processing Systems_ , Vol. 33. 1877–1901.
  * Chen et al. (2022) Xiang Chen, Ningyu Zhang, Xin Xie, Shumin Deng, Yunzhi Yao, Chuanqi Tan, Fei Huang, Luo Si, and Huajun Chen. 2022.  KnowPrompt: Knowledge-Aware Prompt-Tuning with Synergistic Optimization for Relation Extraction. In _Proceedings of the ACM Web Conference 2022_. 2778–2788.
  * Cruz et al. (2020) Jan Christian Blaise Cruz, Julianne Agatha Tan, and Charibeth Cheng. 2020.  Localization of Fake News Detection via Multitask Transfer Learning. In _Proceedings of the 12th Language Resources and Evaluation Conference_. 2596–2604.
  * Cui et al. (2020) Limeng Cui, Haeseung Seo, Maryam Tabar, Fenglong Ma, Suhang Wang, and Dongwon Lee. 2020\.  DETERRENT: Knowledge Guided Graph Attention Network for Detecting Healthcare Misinformation. In _Proceedings of the 26th ACM SIGKDD International Conference on Knowledge Discovery & Data Mining_. 492–502.
  * Devlin et al. (2019) Jacob Devlin, Ming-Wei Chang, Kenton Lee, and Kristina Toutanova. 2019.  BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding. In _Proceedings of the 2019 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, Volume 1 (Long and Short Papers)_. 4171–4186.
  * Dou et al. (2021) Yingtong Dou, Kai Shu, Congying Xia, Philip S. Yu, and Lichao Sun. 2021.  User Preference-Aware Fake News Detection. In _Proceedings of the 44th International ACM SIGIR Conference on Research and Development in Information Retrieval_. 2051–2055.
  * Dun et al. (2021) Yaqian Dun, Kefei Tu, Chen Chen, Chunyan Hou, and Xiaojie Yuan. 2021.  KAN: Knowledge-aware Attention Network for Fake News Detection.  _Proceedings of the AAAI Conference on Artificial Intelligence_ 35 (2021), 81–89.
  * Ecker et al. (2022) Ullrich Ecker, Stephan Lewandowsky, John Cook, Philipp Schmid, Lisa Fazio, Nadia Brashier, Panayiota Kendeou, Emily Vraga, and Michelle Amazeen. 2022.  The psychological drivers of misinformation belief and its resistance to correction.  _Nature Reviews Psychology_ 1 (01 2022), 13–29.
  * Galtung and Ruge (1965) Johan Galtung and Mari Holmboe Ruge. 1965.  The Structure of Foreign News: The Presentation of the Congo, Cuba and Cyprus Crises in Four Norwegian Newspapers.  _Journal of Peace Research_ 2, 1 (1965), 64–90.
  * Gao et al. (2021) Tianyu Gao, Adam Fisch, and Danqi Chen. 2021.  Making Pre-trained Language Models Better Few-shot Learners. In _Proceedings of the 59th Annual Meeting of the Association for Computational Linguistics and the 11th International Joint Conference on Natural Language Processing (Volume 1: Long Papers)_. 3816–3830.
  * Hamilton et al. (2017) William L. Hamilton, Rex Ying, and Jure Leskovec. 2017\.  Inductive Representation Learning on Large Graphs. In _NIPS_.
  * Hardalov et al. (2022) Momchil Hardalov, Arnav Arora, Preslav Nakov, and Isabelle Augenstein. 2022.  Few-Shot Cross-Lingual Stance Detection with Sentiment-Based Pre-Training.  _Proceedings of the AAAI Conference on Artificial Intelligence_ 36 (Feb 2022).
  * He et al. (2023) Xiaoxin He, Xavier Bresson, Thomas Laurent, and Bryan Hooi. 2023.  Explanations as Features: LLM-Based Features for Text-Attributed Graphs.  _arXiv preprint arXiv:2305.19523_ (2023).
  * Hendrycks et al. (2020) Dan Hendrycks, Xiaoyuan Liu, Eric Wallace, Adam Dziedzic, Rishabh Krishnan, and Dawn Song. 2020\.  Pretrained Transformers Improve Out-of-Distribution Robustness. In _Proceedings of the 58th Annual Meeting of the Association for Computational Linguistics_. Association for Computational Linguistics, Online, 2744–2751.
  * Hu et al. (2021) Linmei Hu, Tianchi Yang, Luhao Zhang, Wanjun Zhong, Duyu Tang, Chuan Shi, Nan Duan, and Ming Zhou. 2021\.  Compare to The Knowledge: Graph Neural Fake News Detection with External Knowledge. In _Proceedings of the 59th Annual Meeting of the Association for Computational Linguistics and the 11th International Joint Conference on Natural Language Processing (Volume 1: Long Papers)_. 754–763.
  * Hu et al. (2022) Shengding Hu, Ning Ding, Huadong Wang, Zhiyuan Liu, Jingang Wang, Juanzi Li, Wei Wu, and Maosong Sun. 2022\.  Knowledgeable Prompt-tuning: Incorporating Knowledge into Prompt Verbalizer for Text Classification. In _Proceedings of the 60th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers)_. 2225–2240.
  * Jiang et al. (2022) Gongyao Jiang, Shuang Liu, Yu Zhao, Yueheng Sun, and Meishan Zhang. 2022.  Fake news detection via knowledgeable prompt learning.  _Information Processing & Management_ 59, 5 (2022).
  * Jiang et al. (2020) Zhengbao Jiang, Frank F. Xu, Jun Araki, and Graham Neubig. 2020\.  How Can We Know What Language Models Know?  _Transactions of the Association for Computational Linguistics_ 8 (2020), 423–438.
  * Kim (2014) Yoon Kim. 2014.  Convolutional Neural Networks for Sentence Classification. In _Proceedings of the 2014 Conference on Empirical Methods in Natural Language Processing (EMNLP)_. 1746–1751.
  * Kipf and Welling (2017) Thomas N. Kipf and Max Welling. 2017.  Semi-Supervised Classification with Graph Convolutional Networks. In _International Conference on Learning Representations (ICLR)_.
  * Kubin and von Sikorski (2021) Emily Kubin and Christian von Sikorski. 2021.  The role of (social) media in political polarization: a systematic review.  _Annals of the International Communication Association_ 45, 3 (2021), 188–206.
  * Kumar et al. (2022) Ananya Kumar, Aditi Raghunathan, Robbie Matthew Jones, Tengyu Ma, and Percy Liang. 2022.  Fine-Tuning can Distort Pretrained Features and Underperform Out-of-Distribution. In _International Conference on Learning Representations_.
  * Lee (2013) Dong-Hyun Lee. 2013\.  Pseudo-Label : The Simple and Efficient Semi-Supervised Learning Method for Deep Neural Networks. In _Workshop on challenges in representation learning, ICML_.
  * Li and Zhou (2020) Qifei Li and Wangchunshu Zhou. 2020.  Connecting the Dots Between Fact Verification and Fake News Detection. In _Proceedings of the 28th International Conference on Computational Linguistics_. 1820–1825.
  * Li and Liang (2021) Xiang Lisa Li and Percy Liang. 2021.  Prefix-Tuning: Optimizing Continuous Prompts for Generation. In _Proceedings of the 59th Annual Meeting of the Association for Computational Linguistics and the 11th International Joint Conference on Natural Language Processing (Volume 1: Long Papers)_. 4582–4597.
  * Lin et al. (2022) Hongzhan Lin, Pengyao Yi, Jing Ma, Haiyun Jiang, Ziyang Luo, Shuming Shi, and Ruifang Liu. 2022.  Zero-Shot Rumor Detection with Propagation Structure via Prompt Learning.  https://arxiv.org/abs/2212.01117
  * Liu et al. (2023) Pengfei Liu, Weizhe Yuan, Jinlan Fu, Zhengbao Jiang, Hiroaki Hayashi, and Graham Neubig. 2023\.  Pre-Train, Prompt, and Predict: A Systematic Survey of Prompting Methods in Natural Language Processing.  _ACM Comput. Surv._ 55, 9, Article 195 (2023).
  * Liu et al. (2019) Yinhan Liu, Myle Ott, Naman Goyal, Jingfei Du, Mandar Joshi, Danqi Chen, Omer Levy, Mike Lewis, Luke Zettlemoyer, and Veselin Stoyanov. 2019.  RoBERTa: A Robustly Optimized BERT Pretraining Approach.  arXiv:1907.11692 [cs.CL]
  * McNealy and Mullis (2019) Jasmine McNealy and Michaela Devyn Mullis. 2019.  Tea and Turbulence: Communication Privacy Management Theory and Online Celebrity Gossip Forums.  _Comput. Hum. Behav._ 92, C (mar 2019), 110–118.  https://doi.org/10.1016/j.chb.2018.10.029
  * Monti et al. (2019) Federico Monti, Fabrizio Frasca, Davide Eynard, Damon Mannion, and Michael M. Bronstein. 2019\.  Fake News Detection on Social Media using Geometric Deep Learning.  _ArXiv_ abs/1902.06673 (2019).
  * Nguyen et al. (2020) Van-Hoang Nguyen, Kazunari Sugiyama, Preslav Nakov, and Min-Yen Kan. 2020.  FANG: Leveraging Social Context for Fake News Detection Using Graph Representation. In _Proceedings of the 29th ACM International Conference on Information Knowledge Management_ (Virtual Event, Ireland) _(CIKM ’20)_. 1165–1174.
  * Nickerson (1998) Raymond Nickerson. 1998\.  Confirmation Bias: A Ubiquitous Phenomenon in Many Guises.  _Review of General Psychology_ 2 (06 1998), 175–220.
  * Pelrine et al. (2021) Kellin Pelrine, Jacob Danovitch, and Reihaneh Rabbany. 2021\.  The Surprising Performance of Simple Baselines for Misinformation Detection. In _Proceedings of the Web Conference 2021_. 3432–3441.
  * Petroni et al. (2019) Fabio Petroni, Tim Rocktäschel, Sebastian Riedel, Patrick Lewis, Anton Bakhtin, Yuxiang Wu, and Alexander Miller. 2019\.  Language Models as Knowledge Bases?. In _Proceedings of the 2019 Conference on Empirical Methods in Natural Language Processing and the 9th International Joint Conference on Natural Language Processing (EMNLP-IJCNLP)_. 2463–2473.
  * Pogue (2017) David Pogue. 2017\.  How to Stamp Out Fake News.  _Scientific American_ 316 (01 2017), 24.
  * Qin and Eisner (2021) Guanghui Qin and Jason Eisner. 2021.  Learning How to Ask: Querying LMs with Mixtures of Soft Prompts. In _Proceedings of the 2021 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies_. 5203–5212.
  * Rashkin et al. (2017) Hannah Rashkin, Eunsol Choi, Jin Yea Jang, Svitlana Volkova, and Yejin Choi. 2017.  Truth of Varying Shades: Analyzing Language in Fake News and Political Fact-Checking. In _Proceedings of the 2017 Conference on Empirical Methods in Natural Language Processing_. 2931–2937.
  * Roberts et al. (2021) Kim Roberts, Katherine Wood, and Breanne Wylie. 2021\.  Children’s ability to edit their memories when learning about the environment from credible and noncredible websites.  _Cognitive Research: Principles and Implications_ 6 (12 2021).
  * Ruchansky et al. (2017) Natali Ruchansky, Sungyong Seo, and Yan Liu. 2017.  CSI: A Hybrid Deep Model for Fake News Detection. In _Proceedings of the 2017 ACM on Conference on Information and Knowledge Management_. 797–806.
  * Schick and Schütze (2021a) Timo Schick and Hinrich Schütze. 2021a.  Exploiting Cloze-Questions for Few-Shot Text Classification and Natural Language Inference. In _Proceedings of the 16th Conference of the European Chapter of the Association for Computational Linguistics: Main Volume_. 255–269.
  * Schick and Schütze (2021b) Timo Schick and Hinrich Schütze. 2021b.  It’s Not Just Size That Matters: Small Language Models Are Also Few-Shot Learners. In _Proceedings of the 2021 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies_. 2339–2352.
  * Sharma et al. (2019) Karishma Sharma, Feng Qian, He Jiang, Natali Ruchansky, Ming Zhang, and Yan Liu. 2019\.  Combating Fake News: A Survey on Identification and Mitigation Techniques.  _ACM Trans. Intell. Syst. Technol._ 10, 3, Article 21 (2019), 42 pages.
  * Sheng et al. (2021) Qiang Sheng, Xueyao Zhang, Juan Cao, and Lei Zhong. 2021\.  _Integrating Pattern- and Fact-Based Fake News Detection via Model Preference Learning_.  1640–1650.
  * Shu et al. (2019) Kai Shu, Limeng Cui, Suhang Wang, Dongwon Lee, and Huan Liu. 2019.  DEFEND: Explainable Fake News Detection. In _Proceedings of the 25th ACM SIGKDD International Conference on Knowledge Discovery & Data Mining_. 395–405.
  * Shu et al. (2018) Kai Shu, Deepak Mahudeswaran, Suhang Wang, Dongwon Lee, and Huan Liu. 2018.  FakeNewsNet: A Data Repository with News Content, Social Context and Dynamic Information for Studying Fake News on Social Media.  _arXiv preprint arXiv:1809.01286_ (2018).
  * Shu et al. (2017) Kai Shu, Amy Sliva, Suhang Wang, Jiliang Tang, and Huan Liu. 2017.  Fake News Detection on Social Media: A Data Mining Perspective.  _SIGKDD Explor. Newsl._ 19, 1 (2017), 22–36.
  * T and Mathew (2022) Sadiq T and Saji Mathew. 2022.  The disaster of misinformation: a review of research in social media.  _International Journal of Data Science and Analytics_ 13 (05 2022), 1–15.
  * Tam et al. (2021) Derek Tam, Rakesh R Menon, Mohit Bansal, Shashank Srivastava, and Colin Raffel. 2021\.  Improving and Simplifying Pattern Exploiting Training.  _Empirical Methods in Natural Language Processing (EMNLP)_.
  * Vaibhav et al. (2019) Vaibhav Vaibhav, Raghuram Mandyam, and Eduard Hovy. 2019\.  Do Sentence Interactions Matter? Leveraging Sentence Level Representations for Fake News Classification. In _Proceedings of the Thirteenth Workshop on Graph-Based Methods for Natural Language Processing (TextGraphs-13)_. Association for Computational Linguistics, Hong Kong, 134–139.
  * Veličković et al. (2018) Petar Veličković, Guillem Cucurull, Arantxa Casanova, Adriana Romero, Pietro Liò, and Yoshua Bengio. 2018.  Graph Attention Networks.  _International Conference on Learning Representations_ (2018).
  * Wang et al. (2021) Yaqing Wang, Fenglong Ma, Haoyu Wang, Kishlay Jha, and Jing Gao. 2021.  Multimodal Emergent Fake News Detection via Meta Neural Process Networks. In _Proceedings of the 27th ACM SIGKDD Conference on Knowledge Discovery & Data Mining_. 3708–3716.
  * Wilcoxon (1945) Frank Wilcoxon. 1945\.  Individual Comparisons by Ranking Methods.  _Biometrics Bulletin_ 1 (1945), 80–83.
  * Wolf et al. (2020) Thomas Wolf, Lysandre Debut, Victor Sanh, Julien Chaumond, Clement Delangue, Anthony Moi, Pierric Cistac, Tim Rault, Rémi Louf, Morgan Funtowicz, et al. 2020\.  Transformers: State-of-the-Art Natural Language Processing. In _Proceedings of the 2020 Conference on Empirical Methods in Natural Language Processing: System Demonstrations_. 38–45.
  * Wu and Hooi (2022) Jiaying Wu and Bryan Hooi. 2022.  Probing Spurious Correlations in Popular Event-Based Rumor Detection Benchmarks. In _Machine Learning and Knowledge Discovery in Databases - European Conference, ECML PKDD 2022_. 274–290.
  * Wu and Hooi (2023) Jiaying Wu and Bryan Hooi. 2023.  DECOR: Degree-Corrected Social Graph Refinement for Fake News Detection. In _Proceedings of the 29th ACM SIGKDD Conference on Knowledge Discovery and Data Mining_. 2582–2593.
  * Wu et al. (2019) Lianwei Wu, Yuan Rao, Haolin Jin, Ambreen Nazir, and Ling Sun. 2019.  Different Absorption from the Same Sharing: Sifted Multi-task Learning for Fake News Detection. In _Proceedings of the 2019 Conference on Empirical Methods in Natural Language Processing and the 9th International Joint Conference on Natural Language Processing (EMNLP-IJCNLP)_. 4644–4653.
  * Zhou et al. (2020) Xinyi Zhou, Jindi Wu, and Reza Zafarani. 2020.  SAFE: Similarity-Aware Multi-modal Fake News Detection. In _Advances in Knowledge Discovery and Data Mining_ , Hady W. Lauw, Raymond Chi-Wing Wong, Alexandros Ntoulas, Ee-Peng Lim, See-Kiong Ng, and Sinno Jialin Pan (Eds.). 354–367.