# Feed-Forward Neural Networks Need Inductive Bias to Learn Equality Relations

Tillman Weyde
Department of Computer Science
City, University of London
London, United Kigndom
<EMAIL_ADDRESS>
Radha Manisha Kopparti
Department of Computer Science
City, University of London
London, United Kingdom
<EMAIL_ADDRESS>

###### Abstract

Basic binary relations such as equality and inequality are fundamental to
relational data structures. Neural networks should learn such relations and
generalise to new unseen data. We show in this study, however, that this
generalisation fails with standard feed-forward networks on binary vectors.
Even when trained with maximal training data, standard networks do not
reliably detect equality.

We introduce differential rectifier (DR) units that we add to the network in
different configurations. The DR units create an inductive bias in the
networks, so that they do learn to generalise, even from small numbers of
examples and we have not found any negative effect of their inclusion in the
network. Given the fundamental nature of these relations, we hypothesize that
feed-forward neural network learning benefits from inductive bias in other
relations as well. Consequently, the further development of suitable inductive
biases will be beneficial to many tasks in relational learning with neural
networks.

##  1 Introduction

Basic relations such as equality are fundamental to relational data
structures. One goal of applying neural networks to relational data is that
the networks learn to infer these relational structure from data. Although
equality is typically not learned from data, equality or approximate equality
may be embedded as part of other tasks. The modelling of equality is clearly
in the hypothesis space of feed-forward neural networks (FFNNs) [1], but [2,
3] already highlighted that learning of identity relationships with neural
networks may not generalise to unseen data. Therefore, we see learning to
recognise equality as relevant from a theoretical and practical perspective.

In this study we test whether feed-forward networks learn equality as well as
a numeric comparison, thresholded digit sum, and digit reversal of pairs of
binary vectors and then generalise this to new data in different settings
regarding the task, the amount of data provided, and the depth of the network.
We find that the recognition of binary relations is not generalised reliably
by feed-forward networks.

To address this problem, we introduce an inductive bias with additional
predefined network structures, that we call differential rectifier (DR) units.
We find in our experiments that DR units induce reliable perfect
generalisation for equality and all other tasks except in digit reversal.

We see two questions that these results raise: First, which other relations
neural networks do not learn and what that means for more complex tasks.
Second, what kinds of inductive biases to design and how to implement them.

The remainder of this paper is organised as follows: Section 2 reviews related
literature, Section 3 introduces the task of learning vector equality and our
DR units for inductive bias. Section 4 presents the experimental results and
in Section 5 follow the conclusions of this paper.

##  2 Related work

In relational learning, equality is often not learned from the data, with the
exception of the work by [4] who learn to detect equality attributed of
objects from images. Learning equality could be interesting in the context of
constraint learning [5] to learn when equality constraints should be regarded
as satisfied. Another relevant area is rule learning and application, where
soft unification like in [6] could be replaced with a learnt model.

Since neural networks are currently by far the most popular machine learning
method, it seems of interest whether they can learn equality. There have been
a number of theoretical contributions showing that feed-forward networks are
universal approximators, most generally to our knowledge by [1]. Presumably
because of these results there was relatively little interest in the question
which functions neural networks can not learn. One of the few studies in this
direction was undertaken in [2] in 1999, where a recurrent neural network
failed to distinguish abstract patterns, based on equality relations between
sequence elements, although seven-month-old infants showed the ability to
distinguish them after a few minutes of exposure. This was followed by an
lively exchange on rule learning by neural networks and in human language
acquisition, where results by [7, 8, 9] could not be reproduced by [10, 11]
and [12] disputed claims by [11]. Other approaches, such as [13, 14, 15], use
different network architectures, problem formulations or evaluation methods.

A more specific problem of learning equality relations was posed in [3] by
showing that learning of equality on even numbers does not transfer to odd
numbers in binary representation. This relates to the input neuron for the
least significant bit not being set to $1$ during training. Recently, [16]
addressed this specific problem with different approaches as an example for
extrapolation and inductive biases for machine learning in natural language
processing. However, they did not address the general question of learning
equality with neural networks.

If standard neural networks do not generalise equality relations despite the
solution being in their hypothesis space, as we will show for FFNNs below,
then the question is how we can enable the learning of solutions that do
generalise. Inductive biases as a solution can be realised in a number of ways
and have been of increased interest recently [17, 18].

##  3 Equality relation learning

The studies listed above motivated the approach taken here to study a reduced
problem outside common contexts such as image analysis or cognitive modelling:
whether feed-forward neural networks trained with back-propagation generally
have the ability to learn equality relations and generalise to unseen data.

The general task is to learn the relation between pairs of binary vectors.
This leads to a binary classification of the pairs according to the equality
or otherwise of its element vectors. We use a standard FFNN as sketched in
Figure 1a). This network has $2n$ input neurons, where $n$ is the vector
dimensionality. The hidden layer has 10 neurons with ReLu activation. The
output layer has two neurons representing the two classes (equal/unequal),
which use softmax activation. The training uses the Adam optimiser[19] with
cross-entropy loss.

The data we train and test the network with is synthetically generated and we
vary the type and the distribution of the data in the experiments below. We
are interested in how many training examples are needed until the network
learns to correctly classify pairs of equal vs. unequal vectors. This network,
like the following ones have been implemented in Python using PyTorch
(http://pytorch.org).

a)  b)  c)

Figure 1: Network architectures: a) standard feed-forward network, DR
integration with b) early fusion and c) mid fusion. The DR units receive their
input in both cases from $vec\hskip 2.0pt1$ and $vec\hskip 2.0pt2$.

### Inductive bias creation with DR units

In our model, we use differential rectifier (DR) units that compare input
values by calculating the absolute difference: $f(x,y)=|x-y|$. We create one
DR unit for every vector dimension with weights from the inputs to the DR
units fixed at $1$, thus learning the suitable summation weights for the DRs
is sufficient for creating a generalisable equality detector.

We use two ways of integrating DR units into the neural networks: Early Fusion
and Mid Fusion. In Early Fusion, DR units are concatenated to input units 1b),
and in Mid Fusion they are added to the hidden layer 1c). In both cases the
existing input and hidden units are unchanged.

##  4 Experiments and Results

We performed different sets of experiments using binary vectors for estimating
vector equality in relation to vector dimensionality, data size and dataset
structure. We also use two additional tasks to test the effect of DR units in
different contexts.

### Effect of network architecture and vector dimensionality

We generate pairs of random binary vectors with dimensionality $n$ between 2
and 100 as shown in table 1. We use all the possible binary vectors to
generate equal pairs, i.e. $2^{n}$ pairs, for $n<10$ and a random selection of
1000 vectors otherwise. We also generate the same number of randomly selected
unequal vector pairs. Then we use stratified sampling to split the data 75:25
into train and test set. The network is then trained for 20 epochs, which led
to convergence in all cases. We run 10 simulations for each configuration. The
average results are shown in Table 1.

We see that the standard FFNNs never fully generalise, and in many cases
barely exceed chance level (50%). The early fusion model improves results, but
only reaches full performance for 100 dimensions. The Mid Fusion reaches
perfect test performance in all cases.

For the plain FFNN, it looks like there is a trend towards better performance
at higher dimensionality, but with the observed variation that may be
coincidental. We did not perform an exhaustive grid search over all
hyperparameters, but tested higher numbers of hidden layers (2,3), and larger
hidden layers (20,30 neurons) without observing a significant change in the
results.

Vector Dimensions | Plain FFNN | Early Fusion | Mid Fusion
---|---|---|---
n=2 | 52% | 82% | 100%
n=3 | 55% | 75% | 100%
n=5 | 37% | 67% | 100%
n=10 | 52% | 75% | 100%
n=30 | 65% | 75% | 100%
n=100 | 75% | 100% | 100%
Table 1: Accuracy of the different network types on pairs of vectors of
different dimensions. The joint train and test data covers all possible equal
vector pairs up to 1000, and a random selection where there are more. Only the
Mid Fusion architecture reaches reliable equality detection.

### Effect of training data size

We study here how much the performance depends on the training data size. For
this, we vary only the training data size and keep the test set and all other
parameters constant. We use training data sizes of 1% to 50% (in relation to
the totally available data as defined above) and the accuracy achieved in
various conditions is plotted in Figure 2. It is worth noting, that the Mid
Fusion network reaches 100% accuracy from 10% data size on while the FFNN
shows only small learning effects.

Figure 2: Accuracy of FFNN for 10 dimensional binary vectors after varying the
distributions of training data from (1%-50%) keeping the testing data fixed

### Effect of vector coverage

A possible hypothesis for the results of the FFNN is that the coverage of the
vectors in the training set plays a role. To share vectors in equal pairs
between training and test set would mean to train on the test data, but we
created a training set that contains all vectors that appear in the test set
in the unequal pairs for $n=10$. The results are shown in column a) of Table
2. We also created a training set where each vector appeared as above, but in
both position 1 and 2. The results are shown in column b) of table 2. The
results in both cases are similar to those without this additional coverage in
Table 1.

### Other classification tasks

We evaluate here whether the DR units have a negative effect on other learning
tasks (using $n=3$). We evaluated the networks on the classification by
comparing the two vectors in the pair as binary numbers with results shown in
column c) of Table 2. We also tested a task that is not a comparison of the
two vectors in the pair, by calculating the digit sum. We classify by checking
if the digit sum is $\geq 3$. In both c) and d) we see, that the performance
is actually not hindered but helped by the DR units. We finally tested the
task of recognising digit reversal (swapping least with most significant
bits), which DR units are not designed for, as they compare corresponding
digits. As we can see in column e), DR units do not deliver a perfect solution
here, but still lead to somewhat better results than a plain FFNN.

Type | a) | b) | c) | d) | e)
---|---|---|---|---|---
1) Plain FFNN | 50% | 52% | 75% | 77% | 50%
2) Early Fusion | 75% | 87% | 92% | 82% | 55%
3) Mid Fusion | 100% | 100% | 100% | 100% | 58%
Table 2: Test set accuracy of FFNN withour and with DR units for different
vector coverage (a,b, see text for details) and for classification by c)
numeric comparison ($\geq$), d) digit sum $\geq 3$, and e) inversion of
digits.

##  5 Conclusions

In this study we examined the learning behaviour of feed-forward neural
networks in vector equality detection and observed that the networks do not
generalise well to unseen data. We also had similar results in other tasks
like numeric inequality and sum of bits of binary vectors. We therefore
introduced a simple modification to the network with differential rectifier
(DR) units and noticed substantial improvements on unseen test data. This
improvement is largely independent of vector dimension, data size and other
parameters.

The question why standard FFNNs do not learn vector equality relations in a
generalisable way is a relevant one, and deserves further theoretical and
empirical study. It is also important to investigate the design of further
measures for creating and controlling inductive biases in neural network
learning, as we find that even relatively simple tasks like generalising
equality require them.

## References

  * Leshno et al. [1993] Moshe Leshno, Vladimir Ya Lin, Allan Pinkus, and Shimon Schocken.  Multilayer feedforward networks with a nonpolynomial activation function can approximate any function.  _Neural networks_ , 6(6):861–867, 1993.
  * Marcus et al. [1999] G. F. Marcus, S. Vijayan, S.B. Rao, and P.M. Vishton.  Rule learning by seven-month-old infants.  _Science, 283_ , 5398:77–80, 1999.
  * Marcus [2001] G. F. Marcus.  The algebraic mind: Integrating connectionism and cognitive science.  _Cambridge MIT Press_ , 2001.
  * Santoro et al. [2017] Adam Santoro, David Raposo, David G Barrett, Mateusz Malinowski, Razvan Pascanu, Peter Battaglia, and Tim Lillicrap.  A simple neural network module for relational reasoning.  In _Advances in neural information processing systems_ , pages 4967–4976, 2017.
  * De Raedt et al. [2018] Luc De Raedt, Andrea Passerini, and Stefano Teso.  Learning constraints from examples.  In _Proceedings in Thirty-Second AAAI Conference on Artificial Intelligence_ , 2018.
  * Campero et al. [2018] Andres Campero, Aldo Pareja, Tim Klinger, Josh Tenenbaum, and Sebastian Riedel.  Logical rule induction and theory learning using neural theorem proving, 2018.
  * Elman [1999] Jeffrey Elman.  Generalization, rules, and neural networks: A simulation of marcus et. al.  _https://crl.ucsd.edu/ elman/Papers/MVRVsimulation.html_ , 1999.
  * Altmann and Dienes [1999] Gerry Altmann and Zoltan Dienes.  Technical comment on rule learning by seven-month-old infants and neural networks.  _In Science_ , 284(5416)):875–875, 1999.
  * Shultz and Bale [2001] Thomas R. Shultz and Alan C. Bale.  Neural network simulation of infant familiarization to artificial sentences: Rule-like behavior without explicit rules and variables.  _Infancy, 2:4, 501-536, DOI: 10.1207/S15327078IN020407_ , 2001.
  * Vilcu and Hadley [2001] Marlus Vilcu and Robert F Hadley.  Generalization in simple recurrent networks.  In _Proceedings of the Annual Meeting of the Cognitive Science Society_ , volume 23, 2001.
  * Vilcu and Hadley [2005] Marius Vilcu and Robert F Hadley.  Two apparent ‘counterexamples’ to marcus: A closer look.  _Minds and Machines_ , 15(3-4):359–382, 2005\.
  * Shultz and Bale [2006] Thomas R Shultz and Alan C Bale.  Neural networks discover a near-identity relation to distinguish simple syntactic forms.  _Minds and Machines_ , 16(2):107–139, 2006.
  * Shastri and Chang [1999] Shastri and Chang.  A spatiotemporal connectionist model of algebraic rule-learning.  _International Computer Science Institute_ , pages TR–99–011, 1999\.
  * Dominey and Ramus [2000] P. Dominey and F. Ramus.  Neural network processing of natural language: Isensitivity to serial, temporal and abstract structure of language in the infant.  _Language and Cognitive Processes_ , pages 15(1),87–127, 2000.
  * Alhama and Zuidema [2016] Raquel G. Alhama and Willem Zuidema.  Pre-wiring and pre-training: What does a neural network need to learn truly general identity rules.  _CoCo at NIPS_ , 2016.
  * Mitchell et al. [2018] Jeff Mitchell, Pasquale Minervini, Pontus Stenetorp, and Sebastian Riedel.  Extrapolation in nlp.  _arXiv:1805.06648_ , 2018.
  * Hamrick et al. [2018] Jessica B Hamrick, Kelsey R Allen, Victor Bapst, Tina Zhu, Kevin R McKee, Joshua B Tenenbaum, and Peter W Battaglia.  Relational inductive bias for physical construction in humans and machines.  _arXiv preprint arXiv:1806.01203_ , 2018.
  * Snell et al. [2017] Jake Snell, Kevin Swersky, and Richard Zemel.  Prototypical networks for few-shot learning.  In _Advances in Neural Information Processing Systems_ , pages 4077–4087, 2017.
  * Kingma and Ba [2014] Diederik P. Kingma and Jimmy Ba.  Adam: A method for stochastic optimization.  _arXiv:1412.6980_ , 2014.


# Account credibility inference based on news-sharing networks

BT Bao Tran Truong  OMA Oliver Melbourne Allen  FM Filippo Menczer
Observatory on Social Media, Indiana University, Bloomington, US  Network
Science Institute, Northeastern University, Boston, US

###### Abstract

The spread of misinformation poses a threat to the social media ecosystem.
Effective countermeasures to mitigate this threat require that social media
platforms be able to accurately detect low-credibility accounts even before
the content they share can be classified as misinformation. Here we present
methods to infer account credibility from information diffusion patterns, in
particular leveraging two networks: the reshare network, capturing an
account’s trust in other accounts, and the bipartite account-source network,
capturing an account’s trust in media sources. We extend network centrality
measures and graph embedding techniques, systematically comparing these
algorithms on data from diverse contexts and social media platforms. We
demonstrate that both kinds of trust networks provide useful signals for
estimating account credibility. Some of the proposed methods yield high
accuracy, providing promising solutions to promote the dissemination of
reliable information in online communities. Two kinds of homophily emerge from
our results: accounts tend to have similar credibility if they reshare each
other’s content or share content from similar sources. Our methodology invites
further investigation into the relationship between accounts and news sources
to better characterize misinformation spreaders.

information spread,

social media,

misinformation,

network analysis,

credibility,

###### keywords:

Research

##  1 Introduction

Many people are now getting news from social media [1]. With just a click on
the “Share” button, anyone can be a broadcaster of news on these platforms.
Such a low barrier, combined with uneven journalistic standards in online
news, has facilitated the spread of misinformation in the information
ecosystem [2]. Such proliferation of misinformation poses grave threats to
democracy [3], the economy [4], and public health [5, 6, 7, 8].

Existing methods to curb misinformation focus on classifying either the
content or the account posting it. However, multiple challenges exist for
content-based methods, posing a need for methods to evaluate sources instead
of (or in addition to) the content itself. In particular, traditional fact-
checking methods involving human efforts to manually verify the accuracy of
claims cannot scale with the sheer volume and speed of information being
shared online. Automatic misinformation detection could potentially overcome
the problem of scale. But when they work, these methods rely on extensive
language features such as writing style, lexicon, and emotion [9]. Therefore
they need to be constantly retrained to reflect new knowledge and evolving
tactics employed by purveyors of false information. These challenges are
exacerbated by the rise of content created by generative AI. The availability
of open-source large language models (LLMs) brings down the cost of generating
content, creating opportunities for malicious actors to spread misleading
content and influence public opinion [10, 11]. Worse yet, LLMs’ ability to
mimic writing styles makes this type of content persuasive yet very
challenging to detect [12].

In this paper, we propose several methods to infer the credibility of news-
sharing accounts on social media to detect low-credibility accounts likely to
spread misinformation. The credibility of a news source or account may depend
on many factors, including reputation and adherence to factual reporting and
transparency. More precisely, we define _high-credibility news sources_ to be
those that meet objective journalistic criteria as assessed by third-party
fact-checking organizations [13, 14]. Following that, _credible accounts_ are
those who share or reshare high-credibility sources. Credibility indicators
provide useful signals for consumers to navigate the information landscape:
they not only help people seeking information outlets [15] but can also
decrease the propensity to share misinformation [16] by raising user
awareness. Knowing unreliable, influential accounts might also aid platforms
in mitigating their impact.

News-sharing decisions on social media depend on the actual content as well as
on trust in who shares it [15, 17] and what media outlet it originates from
[18]. When information sharing on social media is represented as a network
connecting accounts and/or news sources, we can apply network analysis methods
to infer node properties by propagating labels across links [19, 20, 21]. Most
existing network-based methods to detect low-credibility accounts [20] focus
solely on interactions between accounts. However, bipartite networks capturing
the reinforcing relationship between news outlets and consumers have been
shown to be effective for classifying misinformation content [22]. This merits
further examination of the interactions between accounts and sources to better
understand the characteristics of misinformation spreaders.

Social context is useful in detecting fake news [23]. Generalizing this
observation, we hypothesize that it is possible to infer the credibility of an
account by looking at either the sources or the other accounts they trust. The
fundamental assumption underlying such inference is the existence of
credibility homophily among misinformation spreaders. Homophily can be defined
based on different network relationships, such as following, resharing, co-
sharing, and so on. For example, adjacent nodes in the reshare network
(accounts that reshare each other) trust each other, so they should have
similar credibility. While trust in accounts and trust in sources are both
reasonable assumptions for the task, no other work has compared their
effectiveness in the same settings.

We explore three research questions:

  * Q1: Is there credibility homophily in the reshare network?

  * Q2: Is there credibility homophily in the co-share network?

  * Q3: Can we infer the credibility of accounts by leveraging such homophily and looking at neighbor nodes in the reshare or bipartite/co-share networks, i.e., by examining the trust relationships among accounts or between accounts and sources?

In this paper, we propose several network-based methods — including centrality
measures and graph embeddings — to infer the credibility of news-sharing
accounts on social media. We explore Q1 using the reshare network, where a
directed edge represents trust by one account in another. We explore Q2 using
the bipartite network where accounts are connected to the news sources they
share. The paper makes three contributions:

  1. 1.

We introduce several methods to measure the credibility of accounts by
leveraging different kinds of information-sharing networks.

  2. 2.

We introduce an evaluation framework to systematically estimate and compare
the accuracy of our algorithms using empirical data from multiple contexts and
social media platforms.

  3. 3.

We show that there are two kinds of homophily among information spreaders:
accounts tend to reshare content from individuals with similar credibility
(Q1) and to share content from the same sources as individuals with similar
credibility (Q2). These diffusion patterns explain the effectiveness of
network methods that estimate an account’s credibility using their trust in
other accounts or in sources (Q3).

After reviewing related work, we present our methodology, including the
definition of the credibility inference task and algorithms leveraging an
account’s trust in other accounts and in sources, respectively. We then
describe the experimental setup and discuss the evaluation results.

##  2 Related work

One approach to detect credible accounts uses heuristics such as the
assumption that online opinion leaders are credible [24]. This assumption is
violated by misinformation superspreaders [25, 8, 26] and leaves out
potentially credible ordinary users. Therefore this paper focuses on network-
based approaches that leverage accounts’ connectivity in addition to
heuristics.

PageRank [27] is a widely used centrality measure that assigns scores for
nodes in a directed network by simulating a diffusion process through the
network analogous to random surfing among web pages. Personalized PageRank
[28] incorporates prior knowledge about the importance of some nodes by
constraining the random surfer to stay close to those nodes. Methods extending
PageRank and Personalized PageRank have been applied to measure trust in peer-
to-peer (P2P) networks. EigenTrust was used to obtain global reputation values
for each user [29]; PowerTrust [30] and TrustRank [31, 32] were used to rank
search results. Different from existing work, one of our proposed methods
attempts to infer account credibility by finding the highest-ranking nodes in
a network where misinformation, rather than trust, propagates. Methods have
been introduced that similarly model the spread of “distrust” to measure
trustworthiness in P2P networks [33, 34, 35]. To our knowledge, no prior work
has explored distrust in social news-sharing networks.

Another well-known network centrality method is Hyperlink-Induced Topic Search
(HITS) [36]. This method ranks the web pages returned by a search query by
assuming that hubs are useful in leading a web surfer to authoritative pages.
Several algorithms extend HITS. Co-HITS [37] and Bipartite Graph Reinforcement
Model (BGRM) [38] incorporate pre-existing information about the relevance of
some web pages to constrain the final scores. BiRank [39] is a similar
extension developed for n-partite graphs. In the context of social media, HITS
has been used to find influential users [40] as well as high-quality content
[41]. One of our proposed methods extends HITS while maximizing prior
knowledge about accounts to produce accurate credibility scores. The intuition
is that misinformation sharers are hubs leading to unreliable news sources,
and vice versa.

Machine learning methods have been employed to classify social media accounts
based on their credibility. Previous studies have trained models on features
engineered from a user profile [42, 43] or the content they post [44, 45].
Graph embedding methods are popular means to obtain a compact representation
of nodes in a network. Since network structure information is preserved, nodes
with similar positions in the network have vectors that are close together in
the embedding space [46, 47, 48]. These methods have been applied to classify
rumor-spreading accounts in retweet networks, in combination with additional
features such as an account’s inferred believability [20], screen name,
profile description, and activity level [49]. We explore simpler methods that
only use vectors capturing an account’s position in the network in some of the
algorithms proposed here.

##  3 Methods

The task is formalized as follows: given (i) a set of posts with links to news
articles and (ii) credibility labels for a subset of accounts, assign
credibility scores to the unlabeled accounts.

An overview of the pipeline for mining social media for this task is presented
in Fig. 1. The pipeline consists of five steps:

Figure 1: Pipeline for mining trust patterns from information-sharing data:
trust in accounts is modeled by the reshare network (top); trust in sources
can be modeled by the bipartite account-source network (middle), or a
projection of it, the co-share networks (bottom). Colors represent credibility
labels: orange and purple for high- and low-credibility, respectively.

  1. 1.

Social media data is collected and cleaned. The data includes a set of known
labels and a list of interactions between accounts, for example, via tweets or
retweets.

  2. 2.

Different networks can be constructed to capture distinct potential signals.
The reshare network captures an account’s trust in other accounts. Some of our
algorithms use the trust network — the transpose of this reshare network. The
bipartite account-source network captures an account’s trust in sources. The
co-share network is obtained by projecting the bipartite network onto account
nodes.

  3. 3.

We label a subset of account nodes in the network. These labels are derived
from source credibility scores provided by a fact-checking organization, such
as NewsGuard111newsguardtech.com or Media Bias Fact
Check.222mediabiasfactcheck.com To obtain an account label, we calculate the
weighted mean of all the sources they shared and then apply a threshold to
this mean score, following the organization’s standard for the credibility
threshold.

  4. 4.

Each algorithm is applied to propagate or compute credibility scores for all
nodes.

  5. 5.

Lastly, unlabeled accounts are ranked by their credibility scores for
evaluation.

In the next subsections, we present several methods to infer account
credibility based on trust in accounts or sources. In each case, we evaluate
centrality-based methods and an embedding algorithm. Centrality-based methods
are less sophisticated but more efficient and interpretable.

###  3.1 Trust in accounts

We describe algorithms to calculate an account’s credibility by leveraging
their trust in other accounts. Trustworthiness can be inferred from the trust
network $G^{T}$, in which a link goes from $Alice\rightarrow Bob$ if Alice
follows or reshares Bob. In line with previous work, these actions can be
considered endorsements that signal trust by the sharing account in the
account being shared [20, 50, 51, 52].

The trust network is the transpose of the reshare network $G$, a weighted,
directed graph where the nodes represent accounts and edges correspond to
reshare interactions among accounts. Edges follow the direction of information
spread (from reshared to resharing account) and are weighted by the number of
reshares. Let $G_{ij}=n$ if $j$ retweets $i$ $n$ times.

Finally, we assume that some accounts have credibility labels. Let $H$ be a
set of nodes that are known to have high credibility and $L$ a set of nodes
known to have low credibility.

####  3.1.1 PageRank Trust

The PageRank family of algorithms can be used to calculate account
trustworthiness scores based on this signal. An account’s PageRank Trust score
is calculated iteratively using a weighted version of PageRank:

$\tau_{i}^{t+1}=(1-\alpha)\sum_{j}\frac{G^{T}_{ji}}{\sum_{\ell}G^{T}_{j\ell}}\tau_{j}^{t}+\frac{\alpha}{N}$
(1)

where $t$ is the iteration step, $\alpha$ is the teleportation factor. The
intuition of this method is that accounts with many incoming links are trusted
and therefore have high PageRank Trust scores, indicating high credibility.

####  3.1.2 Personalized PageRank Trust

Information about some high-credibility nodes may be available. To incorporate
this information, an account’s Personalized PageRank Trust is calculated as
follows:

$\tau_{i}^{t+1}=(1-\alpha)\sum_{j}\frac{G^{T}_{ji}}{\sum_{\ell}G^{T}_{j\ell}}\tau_{j}^{t}+\alpha\tau_{i}^{0}$
(2)

where $\tau_{i}^{0}$ is defined by:

$\tau^{0}_{i}=\begin{cases}\frac{1}{|H|}&\text{if }i\in H\\\
0&\text{otherwise.}\end{cases}$ (3)

Under this scheme, accounts that are trusted by credible accounts ($H$ list)
have higher Personalized PageRank Trust, indicating high credibility.

####  3.1.3 TrustRank

This method first creates a quality seed set, ideally to be evaluated by
experts, then uses this list to propagate scores. To apply the method in our
context, PageRank Trust (Eq. 1) is first used to select a set $S$ of seed
accounts with highest PageRank Trust scores, where $|S|$ is a parameter. These
seeds are labeled as “good” or “bad” depending on available credibility
labels:

$\tau_{i}^{0}=\begin{cases}1&\text{if }i\in H\cap S\\\ 0&\text{if }i\in L\cap
S\\\ \frac{1}{2}&\text{otherwise.}\end{cases}$ (4)

TrustRank scores are then calculated using Personalized PageRank Trust (Eq. 2)
with these $\tau^{0}_{i}$ values.

####  3.1.4 LoCred

The methods mentioned so far rely on the observation that good pages seldom
point to bad ones [31]. Decisions to circulate news online are not as
straightforward — bad content is designed to mislead and people could be
socially motivated. Therefore, we cannot assume that “good people seldom trust
bad ones,” or be sure that credible accounts do not reshare from low-
credibility accounts.

We propose a method that uses the diffusion of information to capture an
account’s credibility without the assumption that accounts have good judgment.
This method is performed on the reshare network. If a node reshares a lot, it
might be gullible. These accounts might not spread misinformation
intentionally but should be distrusted nonetheless. The Low Credibility
Account Estimation (LoCred) score is devised to capture such spreaders of
misinformation.

The LoCred score $s_{i}$ for each node $i$ is calculated iteratively until
convergence as a weighted version of personalized PageRank:

$s_{i}^{t+1}=(1-\alpha)\sum_{j}\frac{G_{ji}}{\sum_{\ell}G_{j\ell}}s_{j}^{t}+\alpha
s_{i}^{0}$ (5)

where $t$ is the iteration step, $\alpha$ is the teleportation factor, and the
initial value $s_{i}^{0}$ is defined as follows:

$s^{0}_{i}=\begin{cases}\frac{1}{|L|}&\text{if }i\in L\\\
0&\text{otherwise.}\end{cases}$ (6)

Accounts close to nodes in $L$ have higher LoCred scores, indicating low
credibility. Note that the initial values in Eqs. 3 and 6 are different. But
even if they were the same, the rankings based on Eqs. 2 and 5 would not
simply be the reverse of each other because the eigenvectors of a matrix and
those of its transpose are different for asymmetric matrices.

####  3.1.5 Reputation Scaling

Lastly, we introduce Reputation Scaling, a measure that balances the perceived
trustworthiness of an account and its spreading of misinformation. This method
builds on an application for distributed P2P systems [33].

The reputation of an account is “scaled” by combining both its LoCred and
Personalized PageRank Trust score. We calculate both of these scores, then
rank accounts based on a reputation score calculated as follows:

$r_{i}=\tau_{i}(1-s_{i})$ (7)

where $\tau_{i}$ is the trust of $i$ calculated with Eq.2 and $s_{i}$ is its
LoCred score calculated with Eq.5. An account with high reputation is
considered credible.

####  3.1.6 Node2vec on reshare network

We investigate whether an account’s network position provides predictive
signals about its credibility, i.e., accounts with similar network positions
are similarly credible. To this end, we use node2vec [46] to obtain embedding
vectors of accounts from the retweet network $G$. We use the typical values
for node2vec parameters. Specifically, vector dimension, window size, number
of walks, and walk length are 128, 10, 10, and 80, respectively. The
optimization is run for ten epochs. Account node2vec vectors are then used in
a $k$-Nearest Neighbors (KNN) classifier [53] with $k=10$ to rank unlabeled
accounts. We find the best results by performing a grid search over the random
walk bias parameters $p,q\in\\{0.25,0.50,1,2,4\\}$. We evaluate each parameter
combination using 5-fold cross-validation with 20% labeled data in each fold.

###  3.2 Trust in sources

We describe algorithms to calculate the credibility of accounts by leveraging
their trust in sources (websites or domains in our case). We hypothesize that
the mutually reinforcing relationship between news websites and news consumers
can be applied in our bipartite account-source network to infer account
credibility. Consider accounts as hubs and news domains as authorities: a
“good” account is one pointing to many “good” —authoritative, high-
credibility— websites; and vice versa, a good website is a node shared by many
good accounts [36].

Let us define the account-source network as a weighted bipartite graph where
nodes consist of two disjoint sets $U$ and $D$ that represent accounts and
sources, respectively. A weighted edge represents the number of times an
account shares links to a source. Formally, let $G$ be the adjacency matrix
such that $G_{ij}=n$ if account $i\in U$ shares links from source $j\in D$ $n$
times. Similar to previous algorithms, we assume that some accounts have
credibility labels.

####  3.2.1 HITS

HITS [36] updates account and source scores according to

$\displaystyle u_{i}^{t+1}$ $\displaystyle=\sum_{j\in D}A_{ij}d_{j}^{t}$ (8)
$\displaystyle d_{j}^{t+1}$ $\displaystyle=\sum_{i\in U}A_{ij}u_{i}^{t}$ (9)

respectively, where $A_{ij}=1\text{ if }G_{ij}>0$ and zero otherwise. The
scores are normalized after each step $t$.

####  3.2.2 Co-HITS

Information about some high-credibility sources may be available. Co-HITS
incorporates this information into the propagation to calculate node scores
[37]. Account and source scores are calculated iteratively according to the
strength of interactions between nodes from the two network partitions, until
convergence:

$\displaystyle u_{i}^{t+1}=\alpha
u_{i}^{0}+(1-\alpha)\sum_{j}\frac{G_{ij}d_{j}^{t}}{\sum_{\ell}{G_{i\ell}}}$
(10) $\displaystyle d_{j}^{t+1}=\beta
d^{0}+(1-\beta)\sum_{i}\frac{G_{ij}u_{i}^{t}}{\sum_{\ell}{G_{\ell j}}}$ (11)

respectively, where $\alpha$ and $\beta$ are the teleportation factors for
accounts and sources respectively, $d^{0}=1/|D|$ is the initial value for
sources, and $u_{i}^{0}$ is the initial value for account $i$, defined as:

$u^{0}_{i}=\begin{cases}0&\text{if }i\in H\\\ 1&\text{if }i\in L\\\
1/|U|&\text{otherwise.}\end{cases}$ (12)

These initial values are further normalized such that $\sum_{i\in
U}u^{0}_{i}=1$. The scores are normalized, so no further normalization is
necessary.

####  3.2.3 BGRM

BGRM [38] is similar to Co-HITS. The main difference is in the way they
normalize node scores at each iteration (see Table 1 in [39] for more
details).

####  3.2.4 BiRank

BiRank [39] is also similar to Co-HITS and BGRM, differing in the way node
scores are normalized at each iteration (see Table 1 in [39] for more
details).

####  3.2.5 CoCred

We propose Co-sharing network-based Credibility (CoCred), a measure that, like
Co-HITS, BGRM, and BiRank, incorporates existing knowledge about some
accounts’ credibility into the credibility estimation of other accounts. The
difference is that those algorithms apply the update rules to labeled
accounts, whereas CoCred preserves the known labels, only rescaling them in
the normalization step — we believe the accounts labels contain very strong
signals.

The CoCred score $u_{i}$ for each account $i$ and the corresponding score
$d_{j}$ for each source $j$ are updated at each timestep $t$ according to

$\displaystyle u_{i}^{t+1}$ $\displaystyle=\begin{cases}u_{i}^{0}&\text{if
}i\in H\cup L\\\ \alpha
u_{i}^{0}+(1-\alpha)\sum_{j}\frac{G_{ij}d_{j}^{t}}{\sum_{\ell}{G_{i\ell}}}&\text{otherwise}\end{cases}$
(13) $\displaystyle d_{j}^{t+1}$ $\displaystyle=\beta
d^{0}+(1-\beta)\sum_{i}\frac{G_{ij}u_{i}^{t}}{\sum_{\ell}{G_{\ell j}}}$ (14)

where $\alpha$ and $\beta$ are the teleportation factors for accounts and
sources, respectively; $d^{0}=1/|D|$ is the initial value for sources, and
$u_{i}^{0}$ is the initial value for account $i$, as defined in Eq. 12. Note
that Eq. 14 is the same as Eq. 11.

After each update, the scores are normalized so that $\sum_{i\in
U}u_{i}=\sum_{j\in D}d_{j}=1$. Our algorithm considers low-credibility
accounts as those sharing unreliable news sources and vice versa. Low-
credibility accounts have higher CoCred scores.

####  3.2.6 Node2vec on co-share network

The bipartite account-source network may also provide information about
similar news-sharing patterns between accounts. To explore this, we project
the bipartite network onto account nodes. Specifically, the co-share network
is obtained by connecting accounts if they share links to the same sources. It
is thus a weighted, undirected graph where the nodes are accounts having at
least one shared source in common with another account. Each account is
represented as a vector of shared domains. To mitigate the effect of popular
sources on the similarity among accounts, we use Term Frequency - Inverse
Document Frequency (TF-IDF) vector representations [54] where each dimension
is a news source. Edges correspond to co-share interactions and are weighted
by the cosine similarity between the TF-IDF vectors of the two connected
accounts. Account embedding vectors obtained from the co-share network may
reveal whether individuals who share information from the same sources have
similar credibility. We use node2vec to obtain account vectors and use them in
a KNN classifier with $k=10$ to rank unlabeled accounts. The evaluation of
this method uses the same parameter set and optimization procedure as that of
node2vec on the reshare network described above.

##  4 Evaluation

Our evaluation framework only requires a set of social media posts, and not
follower/friend data. We define the task of classifying low-credibility
accounts as follows: given (i) a set of social media posts with links to news
articles, and (ii) credibility labels for a subset of accounts, assign a
binary label — _credible_ or _not credible_ — to the unlabeled accounts. The
algorithms are evaluated on networks extracted from social media data. The
datasets, corresponding networks, experimental setup, and results are
described below.

###  4.1 Data

We consider three social media datasets.

  * •

The Twitter_Covid dataset includes tweets about COVID-19, collected with the
hashtags #coronavirus and #covid19 from 9–29 March 2020 [55].

  * •

The Twitter_Midterm dataset includes tweets containing hashtags and keywords
about the U.S. 2022 Midterm elections [56]. We use a subset of this
collection, from 8 October–18 November 2022.

  * •

The Facebook_Midterm dataset contains Facebook posts collected the same way
and spanning the same period as Twitter_Midterm [56].

We extract the data fields of interest from each social media post, including
the user ID, post ID, and domain names of the shared links. If a link is
shortened when shared on these platforms, a pre-processing step is performed
to extract the domain name from the expanded URL. Posts with domain names of
platforms such as amazon.com, yelp.com, youtube.com, etc. are not considered.
To limit noise, we further retain only accounts sharing at least five links
and posts containing domains that are shared at least five times across the
dataset. To assign scores to accounts, we start from source credibility
scores, which are domain ratings obtained from NewsGuard in April 2021 for the
Twitter_Covid and October 2022 for the Twitter_Midterm and Facebook_Midterm.
We then compute an account score as a weighted mean of the scores of the
sources they share. Note that not all accounts have a score as a result.
Finally, following NewsGuard’s rubric, we use a threshold of 60 for labeling
accounts with a score below/above 60 as low/high-credibility.

###  4.2 Network description

####  4.2.1 Reshare network

Since reshare information is not included in Facebook data, the reshare
network can only be constructed from the two Twitter333Because our data was
collected from Twitter before it was renamed ‘X,’ we continue to refer to the
platform by its original name. datasets out of the three datasets considered.
The statistics of these retweet networks are summarized in Table 1. The table
reports on network assortativity coefficients [57] using account credibility
scores. The high coefficients are indicative of credibility homophily
(answering Q1).

Table 1: Retweet network descriptions. Dataset | Nodes | Edges | Avg. degree | Credibility assortativity coeff.
---|---|---|---|---
Twitter_Covid | 322,208 | 382,499 | 1.2 | 0.83
Twitter_Midterm | 410,914 | 762,534 | 1.9 | 0.73

The homophily can also be observed in the core of the retweet network (Fig.
2), where low-credibility accounts tend to reshare from low-credibility
accounts, and vice versa.

Figure 2: Retweet network constructed from the Twitter_Covid dataset. To
visualize the network, we filter nodes and edges using k-core decomposition
[58] ($k=3$). Colors represent credibility score: orange and purple for high-
and low-credibility accounts, respectively, and gray for unlabeled accounts.
Node size represents in-strength, i.e., the number of retweets by the account.
Large purple nodes are known misinformation super-spreaders.

####  4.2.2 Bipartite network

Table 2: Bipartite network descriptions. Dataset | Accounts | Domains | Edges | Avg. degree
---|---|---|---|---
Twitter_Covid | 474,094 | 55,539 | 613,609 | 1.3
Twitter_Midterm | 126,445 | 13,415 | 1,190,390 | 9.4
Facebook_Midterm | 6,950 | 4,141 | 26,252 | 17.1

The statistics of the bipartite networks are summarized in Table 2. The core
of a bipartite network is visualized in Fig. 3.

Figure 3: Bipartite account-source network constructed from the Twitter_Covid.
For visualization, only the $k=4$ core of the network is shown. Node colors
represent account credibility scores: orange and purple for high- and low-
credibility accounts, respectively, and gray for unlabeled accounts. Source
nodes are in white. Node size represents the strength of an account node,
i.e., the number of posts with links by that account.

####  4.2.3 Co-share network

Table 3: Co-share network descriptions. Dataset | Nodes | Edges | Avg. degree | Credibility assortativity coeff.
---|---|---|---|---
Twitter_Covid | 67,657 | 434,963 | 1,441.7 | 0.5
Twitter_Midterm | 115,461 | 59,491,099 | 515.3 | 0.8
Facebook_Midterm | 3,488 | 59653 | 34.2 | 0.7

Descriptive statistics of the co-share networks are presented in Table 3.
There is an assortative mixing of credibility among accounts, suggesting the
existence of credibility homophily in the co-share networks (answering Q2).
The core backbone of the co-share network, shown in Fig. 4, illustrates this
other type of homophily: similarly credible accounts tend to share content
from similar sources.

Figure 4: Co-share network constructed from the Twitter_Covid. Given the
density of this network, we aggressively filter it for visualization. We first
keep the 10% of edges with the highest weights and the 10% of nodes in the
innermost core ($k=4$). Then we apply the multiscale backbone method [59]
(significance level 0.3) and retain the giant component of the backbone
network. Colors represent credibility score: orange and purple for high- and
low-credibility accounts, respectively; unlabeled accounts are in gray. Node
sizes represent core numbers: smaller nodes are in the periphery of the
network.

###  4.3 Experimental setup

We employ the area under the receiver operating characteristic curve (ROC_AUC)
and F1 score as metrics for low-credibility account classification. The
maximum value of ROC_AUC, one, indicates that all low-credibility accounts are
ranked before other accounts, whereas a value of 0.5 corresponds to random
ranking. F1 is the harmonic mean of precision and recall. An F1 value of one
means that the model correctly classifies all classes; zero means all samples
are wrongly classified. The F1 score requires converting the account
credibility scores provided by the algorithms into binary labels using a
threshold. We calculate precision and recall for a thousand threshold values
spanning the unit interval and select the optimal threshold that maximizes the
F1 score.

Since NewsGuard ratings are available for only a subset of domains shared in a
dataset, we assign a _label confidence score_ to an account by calculating the
fraction of known domains that they shared; this number is between zero and
one. The most stringent threshold is used in our evaluation — only accounts
with a confidence score of one are used as _known accounts_ to calculate
evaluation metrics.

The evaluation uses 5-fold cross-validation. We hold out a random subset of
20% of known accounts (the test set). For centrality-based methods on the
reshare and bipartite network, the initial scores of the remaining 80% of
accounts are set to their true scores, while the test set starts with default
scores at the beginning of propagation. Similarly, for classification using
embeddings, we train the classifier on the remaining 80% of accounts. Finally,
for both cases, predicted labels for accounts in the test set are checked
against the true labels. The reported metrics are averages of values across
five folds.

We use $\alpha=\beta=0.85$ for the teleportation factors in all network
centrality algorithms. The best node2vec parameters for the retweet networks
are $p=q=1.0$. For the co-share network, the best node2vec parameters are
$p=2.0$, $q=0.5$ for the Twitter_Covid dataset, $p=q=1.0$ for the
Twitter_Midterm dataset, and $p=0.25$, $q=4.0$ for the Facebook_Midterm
dataset.

###  4.4 Results

Evaluation results for the low-credibility account classification task are
summarized in Tables 4 (based on ROC_AUC) and 5 (based on F1 scores). The
trends in performance are similar for both metrics, therefore we focus on
ROC_AUC.

Table 4: Account classification ROC_AUC. Best results for each dataset are highlighted in darker shades. Sharing network | Method | Twitter_Covid | Twitter_Midterm | Facebook_Midterm
---|---|---|---|---
Reshare | node2vec | $0.910\pm 0.006$ | $0.885\pm 0.003$ | N/A
LoCred | $0.768\pm 0.004$ | $0.773\pm 0.044$
Rep. Scaling | $0.618\pm 0.008$ | $0.660\pm 0.003$
Trustrank | $0.534\pm 0.002$ | $0.517\pm 0.001$
PPR Trust | $0.534\pm 0.002$ | $0.517\pm 0.001$
PR Trust | $0.516\pm 0.002$ | $0.520\pm 0.002$
Bipartite/Co-share | CoCred | $0.802\pm 0.014$ | $0.831\pm 0.013$ | $0.715\pm 0.042$
node2vec | $0.654\pm 0.020$ | $0.873\pm 0.005$ | $0.829\pm 0.056$
Co-HITS | $0.699\pm 0.047$ | $0.733\pm 0.018$ | $0.707\pm 0.062$
HITS | $0.673\pm 0.003$ | $0.520\pm 0.006$ | $0.626\pm 0.043$
| BGRM | $0.559\pm 0.003$ | $0.513\pm 0.001$ | $0.614\pm 0.042$
| BiRank | $0.544\pm 0.003$ | $0.504\pm 0.008$ | $0.576\pm 0.031$

The results show that the trust relationships among accounts provide useful
information to effectively infer their credibility. In particular, the best-
performing method is node2vec on reshare networks, with a ROC_AUC score of
more than 0.88 across datasets. The next best-performing algorithms are LoCred
and Reputation Scaling; these algorithms consider reshares as information
diffusion channels without assuming good judgment by accounts. TrustRank,
Personalized PageRank Trust, and PageRank Trust, which assume that reshares
capture trust and hence reputation, perform only marginally better than random
at the task. These results reinforce the difference between web-surfing
behavior and news-sharing behavior on social media, where the assumption that
good accounts seldom trust bad ones might not hold.

The bipartite/co-share networks capturing account trust in sources also
provide effective information to estimate account credibility. The best-
performing algorithm in each dataset achieves a ROC_AUC score of more than 0.8
(CoCred in the Twitter_Covid dataset and node2vec in the Twitter_Midterm and
Facebook_Midterm datasets). Among the centrality-based algorithms on the
bipartite network, our proposed algorithm CoCred performs the best, followed
by Co-HITS and HITS; BGRM and BiRank perform close to random. CoCred’s high
ROC_AUC score confirms our intuition that the known account labels provide
strong signals and should be preserved in score propagation to produce
accurate ratings.

Table 5: Account classification F1. Best results for each dataset are highlighted in darker shades. Sharing network | Method | Twitter_Covid | Twitter_Midterm | Facebook_Midterm
---|---|---|---|---
Reshare | node2vec | $0.918\pm 0.003$ | $0.886\pm 0.003$ | N/A
LoCred | $0.786\pm 0.005$ | $0.757\pm 0.034$
Rep. Scaling | $0.576\pm 0.006$ | $0.514\pm 0.007$
Trustrank | $0.196\pm 0.004$ | $0.214\pm 0.002$
PPR Trust | $0.196\pm 0.004$ | $0.218\pm 0.002$
PR Trust | $0.176\pm 0.012$ | $0.251\pm 0.002$
Bipartite/Co-share | CoCred | $0.800\pm 0.022$ | $0.778\pm 0.014$ | $0.756\pm 0.032$
node2vec | $0.707\pm 0.024$ | $0.886\pm 0.008$ | $0.827\pm 0.029$
Co-HITS | $0.638\pm 0.016$ | $0.655\pm 0.019$ | $0.728\pm 0.051$
HITS | $0.738\pm 0.004$ | $0.199\pm 0.010$ | $0.628\pm 0.066$
| BGRM | $0.501\pm 0.002$ | $0.146\pm 0.002$ | $0.411\pm 0.121$
| BiRank | $0.477\pm 0.011$ | $0.163\pm 0.089$ | $0.434\pm 0.106$

In summary, account credibility can be estimated accurately using trust
signals from news-sharing networks. While all networks provide useful signals,
the reshare network is slightly more informative for the task than the
bipartite/co-share networks, based on the best-performing algorithm in each
dataset.

Fig. 5 shows node2vec embedding vectors of accounts, with their dimensionality
reduced using Principle Component Analysis (PCA) [60]. We observe that the
embeddings in the reshare network more effectively cluster accounts with
similar credibility, compared to the embeddings in the co-share network.

Figure 5: Account embedding vectors for the Twitter_Covid dataset. Node2vec
vectors representing accounts are projected onto a two-dimensional space using
PCA for (a) reshare and (b) co-share networks. Colors represent account
credibility scores: orange and purple for high- and low-credibility accounts
respectively.

##  5 Discussion

This paper suggests ways to mitigate misinformation on social media by
detecting low-credibility accounts likely to spread misinformation. To this
end, we introduce methods to infer account credibility based on two
information-sharing networks: a reshare network capturing account trust in
other accounts and a bipartite network reflecting account trust in information
sources. A systematic evaluation shows that the proposed methods are effective
in detecting low-credibility accounts. The most successful method utilizes
node2vec embeddings of accounts on reshare networks, with both ROC_AUC and F1
around 0.9. Our proposed network centrality algorithms, LoCred and CoCred,
consistently perform well across the three datasets considered, with ROC_AUC
and F1 both around 0.8. We find two kinds of credibility homophily in the
news-sharing networks, which help explain the effectiveness of these methods:
unreliable accounts tend to reshare content from one another (Q1), and share
content from similar sources (Q2). These results confirm our hypothesis that
the structure of the reshare or co-share network provides strong signals to
effectively infer account credibility (Q3). While the presence of credibility
homophily has also been observed in prior analyses of different Twitter data
[61], future work might consider whether these results generalize to
information-sharing networks derived from other social media platforms.

While our framework detects potential misinformation spreaders regardless of
intention, it can help to better characterize disinformation spreaders in
combination with other effective methods, such as bot and inauthentic
coordinated campaign detection [62, 63]. In particular, one of our proposed
methods, CoCred, provides source rankings simultaneously with account
rankings. These source rankings can be used to estimate the credibility of
emerging news media outlets. Such source credibility scores can be used as a
feature in misinformation classification, complementing extant content-based
methods to curb misinformation.

Our work has some limitations. First, we only label and investigate accounts
sharing sources with ratings by NewsGuard. Additional sources of
misinformation that are not labeled by NewsGuard, as well as false/misleading
claims in individual posts that do not include a link to some source, are
missed in our analysis. However, since the criteria used by NewsGuard to label
sources are independent from the methods we evaluate, there is no reason to
believe that our accuracy measurements are biased. Still, complementary
analyses using other sources of ground truth, such as post-level or account-
level annotations, could provide additional insights. Second, classification
results depend on the news-sharing data from which a network is derived. As
such, account credibility scores may vary across contexts. For example, users
have a higher tendency to share links to unconfirmed sources [64] when
information is scarce, diverging from their usual news-sharing behavior.
Accounts might be classified as unreliable in these situations, despite being
reliable in others. One way to address this concern might be to aggregate an
account’s credibility scores across diffusion networks from different topics.
Lastly, algorithms using the reshare network cannot be evaluated on the
Facebook platform, as the data does not provide information about reshares.
Furthermore, Twitter/X has recently removed free data access for researchers,
making this kind of analysis difficult to replicate in the future. Note,
however, that the proposed methods are platform-agnostic. The outlined
analyses can be applied to data from any platform that allows for the
construction of reshare or bipartite networks, such as Mastodon and Bluesky.
We urge all social media platforms to make data available to allow for
transparent analyses [65].

Overall, this work enriches the understanding of misinformation spreaders on
social media by investigating trust signals in different information-sharing
networks. The observed credibility homophily invites further exploration of
misinformation diffusion and spreader dynamics. The proposed methods can be
used to identify accounts of interest based on their estimated credibility by
considering only sharing metadata, before further scrutiny (or even without
any scrutiny) of the specific content they share. Platforms can readily apply
our proposed methods to enhance their moderation efforts. These insights are
crucial to social media platforms and policymakers in the debate on ways to
combat online misinformation.

##  6 Abbreviations

BGRM | Bipartite Graph Reinforcement Model
---|---
CoCred | Co-sharing network-based Credibility
HITS | Hyperlink-Induced Topic Search
KNN | k-Nearest Neighbors
LoCred | Low Credibility Account Estimation
P2P | peer-to-peer
PCA | Principle Component Analysis
PPR Trust | Personalized PageRank Trust
PR Trust | PageRank Trust
ROC_AUC | receiver operating characteristic curve
TF-IDF | Term Frequency - Inverse Document Frequency

## Declarations

### Ethics approval

This research is based on analyses of public social media data with minimal
risks to participants. This study has been granted exemption from review by
the Indiana University IRB (protocol 17036). The collection and release of the
dataset are in compliance with the platforms’ terms of service.

### Consent for publication

Not applicable.

### Availability of data and material

For reproducibility, we make the data and code available in a public
repository https://github.com/osome-iu/credibility-inference.

### Competing interests

The authors declare that they have no competing interests.

### Funding

This research is supported in part by the Knight Foundation, Craig Newmark
Philanthropies, DARPA (awards W911NF-17-C-0094 and HR001121C0169), and the
Luddy School of Informatics, Computing, and Engineering at Indiana University,
Bloomington.

### Authors’ contributions

FM and BT formulated the research, developed the methodology, and prepared the
manuscript. BT collected the data. BT and OA performed analyses. All authors
read and approved the final manuscript.

### Acknowledgements

We are grateful to Kai-Cheng Yang for help with the data collection and many
helpful comments. Ruj Akavipat and Pik-Mai Hui provided helpful discussions.
We also thank NewsGuard for licensing their source credibility scores.

## References

  * [1] Gottfried, J., Shearer, E.: News use across social media platforms 2016. pewresearch.org/journalism/2016/05/26/news-use-across-social-media-platforms-2016/. Pew Research Center (2016)
  * [2] Zarocostas, J.: How to fight an infodemic. The Lancet 395(10225), 676 (2020)
  * [3] Woolley, S.C., Howard, P.N.: Computational Propaganda: Political Parties, Politicians, and Political Manipulation on Social Media. Oxford University Press, ??? (2018)
  * [4] Fisher, M.: Syrian hackers claim AP hack that tipped stock market by $136 billion. Is it terrorism. washingtonpost.com/news/worldviews/wp/2013/04/23/syrian-hackers-claim-ap-hack-that-tipped-stock-market-by-136-billion-is-it-terrorism/ (2013)
  * [5] Tasnim, S., Hossain, M.M., Mazumder, H.: Impact of rumors and misinformation on covid-19 in social media. Journal of preventive medicine and public health 53(3), 171–174 (2020)
  * [6] Allington, D., Duffy, B., Wessely, S., Dhavan, N., Rubin, J.: Health-protective behaviour, social media usage and conspiracy belief during the covid-19 public health emergency. Psychological medicine 51(10), 1763–1769 (2021). doi:10.1017/S003329172000224X
  * [7] Pierri, F., Perry, B.L., DeVerna, M.R., Yang, K.-C., Flammini, A., Menczer, F., Bryden, J.: Online misinformation is linked to early covid-19 vaccination hesitancy and refusal. Scientific reports 12(1), 5966 (2022)
  * [8] Yang, K.-C., Pierri, F., Hui, P.-M., Axelrod, D., Torres-Lugo, C., Bryden, J., Menczer, F.: The covid-19 infodemic: Twitter versus facebook. Big Data & Society 8(1) (2021)
  * [9] Zhou, X., Jain, A., Phoha, V.V., Zafarani, R.: Fake news early detection: A theory-driven model. Digital Threats: Research and Practice 1(2), 1–25 (2020)
  * [10] Goldstein, J.A., Sastry, G., Musser, M., DiResta, R., Gentzel, M., Sedova, K.: Generative language models and automated influence operations: Emerging threats and potential mitigations. arXiv preprint arXiv:2301.04246 (2023)
  * [11] Menczer, F., Crandall, D., Ahn, Y.-Y., Kapadia, A.: Addressing the harms of AI-generated inauthentic content. Nature Machine Intelligence (2023). doi:10.1038/s42256-023-00690-w
  * [12] Kirchner, J.H., Ahmad, L., Aaronson, S., Leike, J.: New AI classifier for indicating AI-written text. openai.com/blog/new-ai-classifier-for-indicating-ai-written-text/. OpenAI (2023)
  * [13] Hovland, C.I., Weiss, W.: The influence of source credibility on communication effectiveness. Public opinion quarterly 15(4), 635–650 (1951)
  * [14] Westerman, D., Spence, P.R., Van Der Heide, B.: Social media as information source: Recency of updates and credibility of information. Journal of computer-mediated communication 19(2), 171–183 (2014)
  * [15] Turcotte, J., York, C., Irving, J., Scholl, R.M., Pingree, R.J.: News recommendations from social media opinion leaders: Effects on media trust and information seeking. Journal of computer-mediated communication 20(5), 520–535 (2015)
  * [16] Yaqub, W., Kakhidze, O., Brockman, M.L., Memon, N., Patil, S.: Effects of credibility indicators on social media news sharing intent. In: Proc. 2020 CHI Conf. on Human Factors in Computing Systems, pp. 1–14 (2020). https://doi.org/10.1145/3313831.3376213
  * [17] The Media Insight Project: “Who Shared It?”: How Americans Decide What News to Trust on Social Media. apnorc.org/projects/who-shared-it-how-americans-decide-what-news-to-trust-on-social-media/ (2017)
  * [18] Sterrett, D., Malato, D., Benz, J., Kantor, L., Tompson, T., Rosenstiel, T., Sonderman, J., Loker, K.: Who shared it?: Deciding what news to trust on social media. Digital journalism 7(6), 783–801 (2019)
  * [19] Mishra, A., Bhattacharya, A.: Finding the bias and prestige of nodes in networks based on trust scores. In: Proc. 20th Intl. Conf. on World Wide Web (WWW), pp. 567–576 (2011). doi:10.1145/1963405.1963485
  * [20] Rath, B., Gao, W., Ma, J., Srivastava, J.: Utilizing computational trust to identify rumor spreaders on twitter. Social Network Analysis and Mining 8(1), 1–16 (2018)
  * [21] Bild, D.R., Liu, Y., Dick, R.P., Mao, Z.M., Wallach, D.S.: Aggregate characterization of user behavior in twitter and analysis of the retweet graph. ACM Transactions on Internet Technology (TOIT) 15(1), 1–24 (2015)
  * [22] Shu, K., Bernard, H.R., Liu, H.: Studying fake news via network analysis: detection and mitigation. Emerging research challenges and opportunities in computational social network analysis and mining, 43–65 (2019)
  * [23] Shu, K., Wang, S., Liu, H.: Beyond news contents: The role of social context for fake news detection. In: Proceedings of the Twelfth ACM International Conference on Web Search and Data Mining, pp. 312–320 (2019)
  * [24] Al-Sharawneh, J., Sinnappan, S., Williams, M.-A.: Credibility-based twitter social network analysis. In: Proc. Asia-Pacific Web Conf., pp. 323–331 (2013)
  * [25] Grinberg, N., Joseph, K., Friedland, L., Swire-Thompson, B., Lazer, D.: Fake news on twitter during the 2016 u.s. presidential election. Science 363(6425), 374–378 (2019). doi:10.1126/science.aau2706
  * [26] DeVerna, M.R., Aiyappa, R., Pacheco, D., Bryden, J., Menczer, F.: Identification and characterization of misinformation superspreaders on social media. Preprint 2207.09524, arXiv (2022). doi:10.48550/ARXIV.2207.09524. https://doi.org/10.48550/ARXIV.2207.09524
  * [27] Page, L., Brin, S., Motwani, R., Winograd, T.: The pagerank citation ranking: Bringing order to the web. Technical report, Stanford InfoLab (1999)
  * [28] Haveliwala, T.H.: Topic-sensitive pagerank: A context-sensitive ranking algorithm for web search. IEEE transactions on knowledge and data engineering 15(4), 784–796 (2003)
  * [29] Kamvar, S.D., Schlosser, M.T., Garcia-Molina, H.: The eigentrust algorithm for reputation management in p2p networks. In: Proc. 12th Intl. Conf. on World Wide Web (WWW), pp. 640–651 (2003). doi:10.1145/775152.775242. https://doi.org/10.1145/775152.775242
  * [30] Zhou, R., Hwang, K.: Powertrust: A robust and scalable reputation system for trusted peer-to-peer computing. IEEE Transactions on Parallel and Distributed Systems 18(4), 460–473 (2007). doi:10.1109/TPDS.2007.1021
  * [31] Gyongyi, Z., Garcia-Molina, H., Pedersen, J.: Combating web spam with trustrank. In: Proc. 30th Intl. Conf. on Very Large Data Bases (VLDB) (2004). http://ilpubs.stanford.edu:8090/770/
  * [32] Wang, G., Wu, J.: Flowtrust: trust inference with network flows. Frontiers of Computer Science 5(2), 181 (2011). doi:10.1007/s11704-011-0323-4
  * [33] Akavipat, R.: Distrust reputation system for p2p information sharing. PhD thesis, Indiana University (2009). UMI Number: 3390252. https://proxyiub.uits.iu.edu/login?qurl=https%3A%2F%2Fsearch.proquest.com%2Fdocview%2F304900296%3Faccountid%3D11620
  * [34] Ortega, F.J., Troyano, J.A., Cruz, F.L., Vallejo, C.G., Enríquez, F.: Propagation of trust and distrust for the detection of trolls in a social network. Computer Networks 56(12), 2884–2895 (2012). doi:10.1016/j.comnet.2012.05.002
  * [35] Guha, R., Kumar, R., Raghavan, P., Tomkins, A.: Propagation of trust and distrust. In: Proc. 13th Intl. Conf. on World Wide Web (WWW), pp. 403–412 (2004). doi:10.1145/988672.988727. https://doi.org/10.1145/988672.988727
  * [36] Kleinberg, J.M.: Authoritative sources in a hyperlinked environment. Journal of the ACM 46(5), 604–632 (1999)
  * [37] Deng, H., Lyu, M.R., King, I.: A generalized co-hits algorithm and its application to bipartite graphs. In: Proc. 15th ACM SIGKDD Intl. Conf. Knowledge Discovery and Data Mining, pp. 239–248 (2009)
  * [38] Rui, X., Li, M., Li, Z., Ma, W.-Y., Yu, N.: Bipartite graph reinforcement model for web image annotation. In: Proc. 15th ACM Intl. Conf. on Multimedia, pp. 585–594 (2007)
  * [39] He, X., Gao, M., Kan, M.-Y., Wang, D.: Birank: Towards ranking on bipartite graphs. IEEE Transactions on Knowledge and Data Engineering 29(1), 57–71 (2016)
  * [40] Romero, D.M., Galuba, W., Asur, S., Huberman, B.A.: Influence and passivity in social media. In: Proc. Joint European Conf. on Machine Learning and Knowledge Discovery in Databases (ECML PKDD), pp. 18–33 (2011). doi:10.1007/978-3-642-23808-6_2. doi.org/10.1007/978-3-642-23808-6_2/
  * [41] Agichtein, E., Castillo, C., Donato, D., Gionis, A., Mishne, G.: Finding high-quality content in social media. In: Proc. 2008 Intl. Conf. on Web Search and Data Mining, pp. 183–194 (2008)
  * [42] Castillo, C., Mendoza, M., Poblete, B.: Predicting information credibility in time-sensitive social media. Internet Research 23(5), 560–588 (2013). doi:10.1108/IntR-05-2012-0095
  * [43] Gupta, A., Kumaraguru, P., Castillo, C., Meier, P.: Tweetcred: Real-time credibility assessment of content on twitter. In: Proc. Intl. Conf. on Social Informatics, pp. 228–243 (2014)
  * [44] Setiawan, E.B., Widyantoro, D.H., Surendro, K.: Measuring information credibility in social media using combination of user profile and message content dimensions. Intl. Journal of Electrical & Computer Engineering 10(4), 3537–3549 (2020). doi:10.11591/ijece.v10i4.pp3537-3549
  * [45] Barbier, G., Liu, H.: Information provenance in social media. In: Proc. Intl. Conf. on Social Computing, Behavioral-Cultural Modeling, and Prediction, pp. 276–283 (2011)
  * [46] Grover, A., Leskovec, J.: Node2vec: Scalable feature learning for networks. In: Proc. 22nd ACM SIGKDD Intl. Conf. Knowledge Discovery and Data Mining, pp. 855–864 (2016). doi:10.1145/2939672.2939754. https://doi-org.proxyiub.uits.iu.edu/10.1145/2939672.2939754
  * [47] Perozzi, B., Al-Rfou, R., Skiena, S.: Deepwalk: Online learning of social representations. In: Proc. 20th ACM SIGKDD Intl. Conf. Knowledge Discovery and Data Mining, pp. 701–710 (2014). doi:10.1145/2623330.2623732. https://doi-org.proxyiub.uits.iu.edu/10.1145/2623330.2623732
  * [48] Tang, J., Qu, M., Wang, M., Zhang, M., Yan, J., Mei, Q.: Line: Large-scale information network embedding. In: Proc. 24th Intl. Conf. on World Wide Web, pp. 1067–1077 (2015)
  * [49] Hamdi, T., Slimi, H., Bounhas, I., Slimani, Y.: A hybrid approach for fake news detection in twitter based on user features and graph embedding. In: Proc. Intl. Conf. on Distr. Comp. and Internet Technology, pp. 266–280 (2020)
  * [50] Roy, A., Sarkar, C., Srivastava, J., Huh, J.: Trustingness & trustworthiness: A pair of complementary trust measures in a social network. In: Proc. ACM/IEEE Intl. Conf. on Advances in Social Networks Analysis and Mining (ASONAM), pp. 549–554 (2016). doi:10.1109/ASONAM.2016.7752289
  * [51] Zhao, L., Hua, T., Lu, C.-T., Chen, I.-R.: A topic-focused trust model for twitter. Computer Communications 76, 1–11 (2016). doi:10.1016/j.comcom.2015.08.001
  * [52] Adali, S., Escriva, R., Goldberg, M., Hayvanovych, M., Magdon-Ismail, M., Szymanski, B., Wallace, W., Williams, G.: Measuring behavioral trust in social networks. In: Proc. IEEE Intl. Conf. on Intelligence and Security Informatics, pp. 150–152 (2010). doi:10.1109/ISI.2010.5484757
  * [53] Altman, N.S.: An introduction to kernel and nearest-neighbor nonparametric regression. The American Statistician 46(3), 175–185 (1992)
  * [54] Jones, K.S.: A statistical interpretation of term specificity and its application in retrieval. Journal of documentation (1972)
  * [55] Yang, K.-C., Torres-Lugo, C., Menczer, F.: Prevalence of low-credibility information on twitter during the covid-19 outbreak. In: Proc. ICWSM Intl. Workshop on Cyber Social Threats (CySoc) (2020). doi:10.36190/2020.16. https://doi.org/10.36190/2020.16
  * [56] Aiyappa, R., DeVerna, M.R., Pote, M., Truong, B.T., Zhao, W., Axelrod, D., Pessianzadeh, A., Kachwala, Z., Kim, M., Seckin, O.C., et al.: A multi-platform collection of social media posts about the 2022 us midterm elections. In: Proceedings of the International AAAI Conference on Web and Social Media, vol. 17, pp. 981–989 (2023)
  * [57] Newman, M.E.: Mixing patterns in networks. Physical review E 67(2), 026126 (2003)
  * [58] Alvarez-Hamelin, J.I., Dall’Asta, L., Barrat, A., Vespignani, A.: Large scale networks fingerprinting and visualization using the k-core decomposition. In: Advances in Neural Information Processing Systems, pp. 41–50 (2006)
  * [59] Serrano, M.Á., Boguná, M., Vespignani, A.: Extracting the multiscale backbone of complex weighted networks. Proc. National Academy of Sciences 106(16), 6483–6488 (2009)
  * [60] Labrín, C., Urdinez, F.: Principal component analysis. In: R for Political Data Science, pp. 375–393 (2020)
  * [61] Nikolov, D., Flammini, A., Menczer, F.: Right and left, partisanship predicts (asymmetric) vulnerability to misinformation. HKS Misinformation Review 1(7) (2021). doi:10.37016/mr-2020-55
  * [62] Yang, K.-C., Varol, O., Davis, C.A., Ferrara, E., Flammini, A., Menczer, F.: Arming the public with artificial intelligence to counter social bots. Human Behavior and Emerging Technologies 1(1), 48–61 (2019)
  * [63] Pacheco, D., Hui, P.-M., Torres-Lugo, C., Truong, B.T., Flammini, A., Menczer, F.: Uncovering coordinated networks on social media: Methods and case studies. Proc. Intl. AAAI Conf. on Web and Social Media (ICWSM) 15(1), 455–466 (2021)
  * [64] Oh, O., Kwon, K.H., Rao, H.R.: An exploration of social media in extreme events: Rumor theory and twitter during the haiti earthquake 2010. In: ICIS (2010)
  * [65] Pasquetto, I.V., Swire-Thompson, B., et al.: Tackling misinformation: What researchers could do with social media data. HKS Misinformation Review 1(8) (2020). doi:10.37016/mr-2020-49