The coercive logic of fake news

Alexander J. Stewart1,∗, Antonio A. Arechar2,3, David G. Rand3, Joshua B.
Plotkin4

1 School of Mathematics and Statistics, University of St Andrews, St Andrews,
UK
2 Center for Research and Teaching in Economics, CIDE, Aguascalientes, MX
3 Sloan School of Management, MIT, Cambridge, MA, USA
4 Department of Biology, University of Pennsylvania, Philadelphia, PA, USA
∗ E-mail<EMAIL_ADDRESS>
The spread of misinformation and “fake news” continues to be a major focus of
public concern. A great deal of research has examined who falls for
misinformation and why, and what can be done to make people more discerning
consumers of news. Comparatively little work, however, has considered
misinformation producers, and how their strategies interact with the
psychology of news consumers. Here we use game-theoretic models to study the
strategic interaction between news publishers and news readers. We show that
publishers who seek to spread misinformation can generate high engagement with
falsehoods by using strategies that mix true and false stories over time, in
such a way that they serve more false stories to more loyal readers. These
coercive strategies cause false stories to receive higher reader engagement
than true stories – even when readers strictly prefer truth over falsehood. In
contrast, publishers who seek to promote engagement with accurate information
will use strategies that generate more engagement with true stories than with
false stories. We confirm these predictions empirically by examining 1,000
headlines from 20 mainstream and 20 fake news sites, comparing Facebook
engagement data with 20,000 perceived accuracy ratings collected in a survey
experiment. We show that engagement is negatively correlated with perceived
accuracy among misinformation sites, but positively correlated with perceived
accuracy among mainstream sites. We then use our model to analyze the
conditions under which news sites seeking engagement will produce false
stories. We show that if a publisher incorrectly assumes that readers prefer
falsehoods, their resulting publication strategy can itself manufacture
greater engagement with false news – leading to a self-reinforcing cycle of
false news promotion. In order to effectively understand and combat
misinformation, it is not enough to study how people engage with news in
isolation - academics and policy makers must consider the strategic interplay
of those who produce and consume (mis)information.

False or misleading information is a fundamental problem for people trying to
form an accurate understanding of the world [1]. There has been widespread
concern in recent years about the spread of misinformation and “fake news” –
for example during Brexit and the 2016 US Presidential Election [1], the
COVID-19 pandemic [2, 3], and the 2020 US Presidential election [4] – among
policy makers, scientists, and the general public, leading to an explosion of
academic research on this topic.

Most prior work has focused on news consumers, asking empirical questions such
as: Who reads, shares, and believes fake news [5, 6, 7]? Why do people fall
for fake news [8, 9, 10]? What interventions can combat fake news [11, 12, 13,
14]? Although substantial progress has been made on these questions regarding
fake news consumers, relatively little work has considered the other side of
the misinformation problem: producers of fake news [15, 16]. The dynamics of
the media ecosystem are driven by a feedback loop between consumers and
producers [17, 18, 19]. Inferences about the revealed preferences of consumers
that fail to account for these dynamics are likely to be misleading. To fully
understand the spread of fake news, we must therefore examine the strategic
considerations that arise on both sides of the news market [17].

To this end, we develop a game-theoretic model for the dynamics of news
dissemination and engagement. Specifically, we study an asymmetric “fake news
game” played between news outlets, who can choose to publish true or false
information, and news consumers, who can choose whether or not to engage with
stories from a given outlet. We focus on the dissemination strategies of
outlets that try to generate engagement with either true or false stories, and
the news engagement strategies of readers who seek to consume true news but
operate under cognitive constraints.

We show that by the strategic dissemination of true and false stories in a
characteristic pattern, publishers seeking to spread fake news can
successfully induce a substantial level of engagement with misinformation
among truth-seeking readers who try to maximize their consumption of true
stories. In particular, our model predicts that publishers who seek to spread
fake news will adopt strategies that: i) include a mix of true and false
stories over time and ii) increase the rate of disseminating misinformation as
the level of engagement they receive from a reader increases. Building on the
game-theoretic concept of extortion strategies in repeated games [20, 21, 22,
23, 24, 25, 26], we call this type of behavior a “coercive” dissemination
strategy.

Our model predicts that, as a result of such coercive dissemination, outlets
seeking to spread fake news will induce their readers to have greater
engagement with false stories than with true stories, creating an apparent
preference for fake news among their readers; whereas outlets seeking to
spread accurate information will produce the opposite pattern. Importantly we
predict that this trend will hold even when all readers in fact value
accuracy, and false headlines are no more “attractive” (e.g. due to novelty,
emotional appeal, etc) than accurate headlines. We verify these predictions in
the empirical patterns of engagement observed among known fake news and
mainstream media outlets, using $20,000$ accuracy ratings for $1,000$ news
stories published across 40 outlets, along with Facebook engagement ratings
for each story from Crowdtangle [27].

Finally we show that revealed preference can be an unreliable approach to
determining readers’ intentions. News sites that assume readers prefer to
engage with false stories will tend to induce such an apparent preference,
even when none exists. This effect is exacerbated when readers are inattentive
to the accuracy of the news they consume, or when when news sources can target
stories at specific groups of readers. And so reducing the _production_ of
misinformation by news outlets who simply seek to maximize engagement may
require fixing incorrect assumptions about the preferences of their readers,
as well as the habits of news consumption among readers themselves.

The fake news game: We develop a theory of the production and consumption of
misinformation using a game-theoretic approach. We consider an asymmetric,
asynchronous, infinitely repeated coordination game, which we dub the “fake
news game”, in which a news transmitter (i.e. a news outlet or a platform
promoting news stories) chooses whether to transmit pieces of true or false
information (i.e. news stories) while a receiver (i.e. news consumer) chooses
whether or not to engage with each piece of transmitted news (Figure 1).

Figure 1: The fake news game. a) We developed the “fake news game” in which
transmitters choose whether to share true or false news stories and receivers
decide whether to engage with each story or not. We assume that receivers gain
an increase in utility $B$ from engaging with accurate information but suffer
a decrease in utility $C$ for engaging with misinformation. Thus the receivers
are incentivized to seek true news stories and avoid fake news stories (we
consider the alternate case in which receivers are incentivized to engage with
fake stories in the SI Section 2). We assume that a transmitter chooses each
subsequent story to be true or false based on the veracity of their previous
story and the level of engagement it received. The number of independent
receivers whose engagement a transmitter considers determines how precisely
the transmitter can target its stories. c) Receivers are assumed to update
their strategy using a myopic, noisy optimization process, which may in
general take account of the stories shared by multiple transmitters.

Receivers derive a utility benefit $B$ from engaging with true news stories,
and they suffer a utility loss $C$ from engaging with false stories (Figure
1); they derive no utility payoff if they do not engage. This payoff structure
reflects the preferences of news readers who prefer accurate information and
seek to avoid falsehoods; in the SI (Section 2.2 and Section 3.6), we provide
results for receivers who prefer falsehood or avoid accurate information.

Similarly, transmitters derive benefit $b_{t}$ or $b_{f}$ from engagement by a
receiver with a true or false story, and no payoff if the receiver does not
engage. We first consider two types of news transmitters: a misinformation
transmitter that seeks to promote fake news stories, so that $b_{t}=0$ and
$b_{f}>0$, and a mainstream transmitter that seeks to promote accurate
stories, so that $b_{t}>0$ and $b_{f}=0$. Later, we also consider a
“clickbait” transmitter who simply aims to maximize engagement regardless of
accuracy ($b_{t}=b_{f}>0$).

Transmitter strategies: We model the dynamics of news dissemination and
engagement as an optimization process, first specifying the strategy spaces
available to transmitters and receivers. A transmitter can choose to share
either a true or false story with a probability that depends on the level of
engagement their previous story generated. We assume that the transmitter
response function is linear, such that a transmitter strategy $\mathbf{r}$ has
the form

$\displaystyle r_{kt}$ $\displaystyle=$
$\displaystyle\alpha+\gamma\frac{k}{N}$ $\displaystyle r_{kf}$
$\displaystyle=$ $\displaystyle\beta+\theta\frac{k}{N}$ (1)

where $r_{kj}$ defines the probability of sharing a true story given that the
previous story was true or false ($j\in\\{t,f\\}$) and that $k$ out of $N$
targeted receivers chose to consume the story.

Receiver strategies: We assume that receivers can pay different levels, and
different types, of attention when deciding whether to engage with a news
story. They can attend to the likely veracity of the headline, and to their
recent experience of engagement with the source that transmitted the story. In
particular, a receiver engages with a story with probability $q^{l}_{ij}$
where

$q^{l}_{ij}=a_{0}\delta_{lt}+(1-a_{0})[(1-a_{1})p_{0}+a_{1}p_{ij}].$ (2)

The indices $i$ and $j$ describe the outcome of the previous news story shared
by the transmitter: index $j\in\\{t,f\\}$ indicates whether the transmitter’s
_previous_ story was true or false. The index $l\in\\{t,f\\}$ indicates
whether the transmitter’s _current_ story is true or false, and index
$i\in\\{c,n\\}$ indicates whether the receiver either consumed ($c$) or did
not consume ($n$) the previous story. With probability $a_{0}$ the receiver is
able to directly assess the veracity of the current story, where $\delta_{lt}$
is the Kronecker delta, equal to 1 if the newly shared story is true, and 0
otherwise. If the receiver is unable to assess the veracity of the current
story (e.g. due to lack of background knowledge, or failure to pay attention),
then with probability $a_{1}$ they make their decision based on their previous
experience of the news source, according to a behavioral strategy
$\mathbf{p}=\\{p_{ct},p_{cf},p_{nt},p_{nf}\\}$, where $p_{ct}$ is the
probability of engaging given that they chose to consume in the previous round
and the story was true, and so on. Finally, if the reader attends to neither
veracity nor past experience, they engage with the story with fixed
probability $p_{0}$.

Dynamics of reader engagement: We begin by analyzing the dynamics of the fake
news game for a pair of fixed transmitter and receiver strategies (see SI
Section 1). We show that a transmitter using a strategy of the type described
by Eq. 2 enforces a specific relationship between the proportion of
transmitted stories that are false, $v_{f}$, and the overall rate at which
receivers consume true and false stories, $v_{tc}$ and $v_{fc}$:

$v_{f}=\frac{1-\alpha}{1-\alpha+\beta}-\frac{\theta}{1-\alpha+\beta}v_{fc}-\frac{\gamma}{1-\alpha+\beta}v_{tc}$
(3)

Remarkably, this relationship between the false news transmission rate and the
receiver’s consumption of true and false news holds regardless of the
receiver’s strategy for engagement – and therefore regardless of the
receiver’s preference for consuming true versus false news. That is, the news
site can unilaterally shape the dynamics of consumer engagement by
constraining the way in which engagement and fake news transmission covary.
For example, a transmitter can seek to generate greater engagement with false
stories by ensuring that higher levels of receiver consumption always result
in a higher rate of the transmitter sharing false stories. This kind of
coercive strategy can be understood as a form of extortion [26, 22].
Critically, this means that false stories receiving more engagement than true
stories (e.g. Vosoughi _et al._ 2018 [28]) does not necessarily mean that
false stories are actually more attractive to readers. Instead, this pattern
could be entirely explained by transmitters using coercive strategies,

Having defined a strategy space for transmitters and receivers, and having
described how a transmitter can unilaterally constrain the dynamics of the
fake news game (Eq. 2), we now examine the dynamics of a receiver’s engagement
strategy in response to a coercive transmitter. We study a single receiver who
employs a noisy optimization process in an attempt to increase engagement with
true stories (that is, to increase payoff given truth-seeking preferences, see
Methods). We focus on a myopic optimization process in which the receiver
compares the payoff they gain under their current strategy to that under a
randomly selected alternate strategy, and tends to adopt the alternate
strategy if it will increase their payoff. The case of a single transmitter
and receiver corresponds to perfect microtargeting by the transmitter (i.e.
the transmitter is able to promote different types of news to specific
receivers in direct response to their engagement habits). However our results
also hold in the more realistic case where a transmitter is able to target
their stories only at the level of a group of receivers (see SI Section 3.3).
In the SI (Section 1.4) we also consider the case where receivers are
perfectly rational [29, 30], and the case where receiver behavior is the
product of social learning [31] (SI Section 3.2), where we find similar
results as for the myopic optimization process.

We simulated strategy optimization for inattentive receivers ($a_{0}=a_{1}=0$)
against $10^{8}$ randomly selected transmitter strategies
$\mathbf{r}=\\{\alpha,\beta,\gamma,\theta\\}$ (Eq. 3). We measured the average
rate at which the transmitter shares false stories with the receiver, $v_{f}$,
and the average rate at which the receiver engages with false stories
(probability of engaging per false story) and with true stories (probability
of engaging per true story), $v_{fc}/v_{f}$ and $v_{tc}/v_{t}$, at equilibrium
(see Methods).

We define the most successful misinformation transmission strategies as those
that i) shared false news more often than not, $v_{f}>0.5$ and ii) produced a
false news engagement rate within the top 10% of all transmitter strategies
considered (see Methods and SI Figure S2). We consider alternate definitions,
in which the minimum amount of fake news shared varies, in the SI (Figure S2).
Likewise, we define successful mainstream transmission strategies as those
that produce $v_{t}>0.5$ along with true news engagement within the top 10% of
all transmitter strategies considered. Note that mainstream transmitters, when
faced with inattentive and myopic readers whose incentives are unknown, may
also share a mix of true and false news stories as they attempt to induce
engagement, just as misinformation sites may share accurate stories to draw
readers in.

The most successful mainstream and misinformation dissemination strategies
induce characteristic, and opposite, patterns of engagement among readers in
our model (Figure 2). In particular, successful misinformation sites induce
higher reader engagement with each false news story, as well as greater
overall consumption of false news than true news (Table 1). This phenomenon
arises even though we assume that receivers strictly prefer true news over
false news, so there is no inherent appeal of false news stories under our
model.

To understand this phenomenon we inspected the strategies of successful
misinformation sites under our model. We find that all successful
misinformation sites indeed use coercive strategies that enforce a positive
correlation between engagement and false news transmission. A significant
proportion ($72\%$, see Methods) use a type of extortion strategy [22] that
enforces $v_{f}\geq v_{fc}+v_{tc}$, i.e. successful misinformation site
strategies tend to increase their false news output rapidly in response to
increased engagement by a given user. However, if engagement drops, they tend
to increase their output of true stories (to draw the user back in). As a
result, they “mash up” true and fake stories as engagement fluctuates over
time.

Figure 2: Successful news transmission strategies – To identify strategies
that are successful at promoting engagement with true news stories (mainstream
sites) or with fake news stories (misinformation sites), we randomly drew
$10^{8}$ transmitter strategies (Eq. 2) and allowed a single receiver,
incentivized to engage with true news, to optimize their engagement strategy
(see Methods) over the course of $10^{4}$ interactions. The receiver was
assumed to be inattentive ($a_{0}=a_{1}=0$) and optimization was assumed to be
noisy ($\sigma=1$). a) We identified the transmitter strategies that
successfully promote engagement with true news stories (mainstream sites,
blue), i.e those that produce a true news engagement rate within the 90th
percentile of the $10^{8}$ transmitter strategies considered, as well as
$v_{t}>0.5$. Similarly, we identified the transmitter strategies that
successfully promote engagement with fake news (misinformation sites, red),
i.e those that produce a fake news engagement rate within the 90th percentile,
as well as $v_{f}>0.5$. We plot the difference between engagement with true
and fake stories, $v_{tc}/v_{t}-v_{fc}/v_{f}$, for all mainstream and
misinformation sites. Mainstream site strategies induce engagement with
accurate stories while misinformation site strategies induce the opposite
effect. b) We also report the difference in overall consumption of true
stories, $v_{tc}$, and consumption of fake stories $v_{fc}$. In all cases
receiver mutations were local (see SI section 1.5) and we set receiver payoffs
$B=2$ and $C=1$. Results for other parameter choices are shown in the SI
Section 3.

The behavior of successful mainstream sites, that seek to generate engagement
with true stories, shows the opposite pattern from of successful
misinformation sites. All successful mainstream strategies enforce a negative
correlation between engagement and false news transmission.

Table 1: News consumption patterns. Summary of the engagement rates per story
and overall consumption rates for mainstream and misinformation site
transmission strategies shown in Figure 2.

A significant proportion ($56\%$, see Methods) use a strategy that enforces
$1-v_{f}\geq v_{fc}+v_{tc}$, i.e. successful mainstream site strategies tend
to decrease their output of false stories rapidly in response to increased
engagement, but may share more false stories when engagement is low (in a
misguided attempt to draw readers back in).

Empirical patterns of fake news engagement: Our analysis of the fake news game
makes four key predictions about the dynamics of engagement with
misinformation: i) misinformation sites that seek to spread fake news should
nonetheless also share accurate information to induce engagement and ii)
successful misinformation sites will produce higher engagement with false than
with true stories, even when consumers prefer truth to falsehood. In contrast
iii) mainstream sites that seek to spread accurate news stories may
nonetheless also share some misinformation in an attempt to induce engagement
and iv) successful mainstream sites will produce higher engagement with true
than with false stories.

We tested these theoretical predictions against empirical data on engagement
with news from publishers that prior work has determined to be either fake
news or mainstream outlets [32]. We sampled a total of $1,000$ articles from
40 outlets (20 fake news, 20 mainstream). We selected the 25 most recently
available news stories for each site from the Crowdtangle database (see
Methods), providing a snapshot of the output from each site (and avoiding
selecting on the dependent variable by only analyzing high-engagement articles
[33]). For each story, we retrieved Facebook engagement ratings from
Crowdtangle [27], and assessed perceived accuracy by recruiting $1,000$
American participants from Amazon Mechanical Turk and having them each rate
the accuracy of $20$ headlines (yielding a total of $20,000$ accuracy ratings)
(Figure 3).

Figure 3: Empirical patterns of news consumption and accuracy, for mainstream
and misinformation sites – We selected 20 mainstream and 20 fake news sites
identified in previous studies of misinformation [32]. Using Crowdtangle we
selected the most recent 25 news stories for which engagement data on Facebook
was available (see Methods). We then recruited American subjects from Amazon
Mechanical Turk to assess headline accuracy ($20$ ratings per headline). a)
Regression coefficient between accuracy and $\log_{10}$-engagement for
mainstream news sites (blue) and misinformation sites (red)). There is a
significant positive correlation between accuracy and engagement across
mainstream sites (Fisher’s combined test, $p=0.004$; Random effects meta-
analysis [34, 35], $p=0.005$, standardized average correlation coefficient –
blue line – $\mu=0.13$; See Methods and SI Section 3). However there is a
significant negative correlation between accuracy and engagement across
misinformation sites (Fisher’s combined test, $p=0.005$; Random effects meta-
analysis, $p=0.013$; standardized average correlation coefficient – red line –
$\mu=-0.11$; See Methods and SI Section 3). b) Regression of individual site
trust ratings [36] against standardized accuracy-engagement regression
coefficient (see Methods and SI Section 3). More trusted sites tend to produce
higher engagement with accurate information, while less trusted sites tend to
produce higher engagement with less accurate information ($p=0.0008$,
$r^{2}=0.25$). c) Distribution of accuracy ratings for mainstream sites (blue,
mean=$5.05$, SD=$0.56$) and mainstream sites (red, mean=$4.27$, SD=$0.62$). As
expected stories from misinformation sites are rated as significantly less
accurate than stories from mainstream sites ($p<0.001$, two-sample t-test).

Our data show that mainstream news sites, which we assume seek to promote
accurate information, tend to share headlines with higher accuracy ratings
than fake news sites ($p<0.001$, Figure 3c). Importantly, however, both
mainstream and fake news sites show wide variation in headline accuracy. These
observations are consistent with our theoretical predictions for transmitter
strategies that mash up stories with high and low accuracy, for both
mainstream and misinformation sites, as they vary their content to draw
readers in. Critically, our model predicts opposite correlations between
accuracy and engagement for the two types of transmitters (Figure 2). And
indeed we observe opposite correlations for mainstream versus fake news sites:
we find a significant negative correlation between engagement and perceived
accuracy across fake news sites ($p=0.005$), whereas we find a significant
positive correlation across mainstream sites ($p=0.004$). Taken together,
these empirical results confirm key predictions of our model.

We compare these empirical results to the predictions of previously proposed
theories for explaining engagement with misinformation [16, 28, 11] (SI
Section 2). We show that, in contrast to our model, none of these theories
predict the empirical pattern observed in Figure 3, in which more trustworthy
sources generate greater engagement with more accurate stories, while less
trustworthy sources generate the opposite pattern.

Coercion and revealed preference: Having empirically validated the key
predictions of our model, we now explore the dynamics that lead transmitters
to produce misinformation. So far we have focused on the behavior of receivers
optimizing in response to a fixed transmitter strategy. In reality both
transmitters and receivers update their behavior in response to one another.
In many cases transmitters may have no agenda other than generating engagement
with the stories they share – that is, they have no intrinsic desire to spread
misinformation or true information, but instead only to increase consumption
among their audience of receivers. However those consumption patterns in turn
depend on the strategy of the transmitter (Figure 2).

We explored the behaviors of transmitters and receivers when both players co-
optimize their strategies (Figure 4). We considered transmitters who seek only
to maximize engagement among receivers ($b_{f}=b_{t}$), but make different
assumptions about the type of news those receivers prefer. Transmitters who
assume receivers prefer fake news (e.g. because it is more novel) employ
strategies that seek to reinforce increased engagement by increasing the
amount of false stories they share in response (see SI Section 1). Similarly
transmitters who assume receivers prefer true news employ strategies that seek
to reinforce increased engagement by increasing the amount of true news they
share in response. Transmitters who make no assumption about receivers
preferences (and hence try out all possible strategies without bias) and
transmitters who assume receivers prefer true news, both produce greater
engagement with true than with false stories (Figure 4a). In contrast,
transmitters who assume receivers prefer fake news, produce greater engagement
with false than with true stories.

Figure 4: Self-reinforcing fake news We allowed both receivers and
transmitters to optimize under the process described in Figure 2. Here we
assume that transmitters seek only engagement with news, regardless of
veracity ($b_{f}=b_{t}=1$). But transmitters draw strategies that reflect
their assumptions about the receivers’ preferences (see Methods). a) Time
series of receiver engagement as receivers and transmitters co-optimize. We
initialized with a transmitter that always shares true news and a receiver
that never engages. We calculated the percentage difference in probability of
engagement with a true versus a fake story, averaged across $10^{4}$ replicate
simulations. When transmitters make no assumption (gray), or assume receivers
prefer true news (blue), true news receives greater engagement, which reflects
the underlying receiver preferences. When transmitters incorrectly assume
receivers prefer fake news (red), however, false stories receive greater
engagement – that is, the assumption that receivers prefer fake news is self-
reinforcing. b) We calculated the average engagement difference for different
levels of receiver attention to payoffs, $\sigma$, and for different
probabilities that receivers have prior knowledge about headline accuracy,
$a_{0}$. It is only when attention to accuracy and attention to payoffs are
low that fake news is self-reinforcing (red region). In all cases receiver
mutations were local (see SI Section 1.5) and receiver payoffs are set to
$B=2$ and $C=1$. Transmitter mutations were global. Receiver attention to
payoffs was set at $\sigma=1$, attention to accuracy at $a_{0}=0$ in panel a)
with memory $a_{1}=0$. Transmitter attention to payoffs was set at
$\sigma=100$.

As these results show, transmitters tend to elicit behavior among receivers
that reinforce their assumptions about receiver preferences. We explored more
broadly (Figure 4b) the conditions under which an assumed preference for fake
news is self-reinforcing in this way (i.e. leads receivers to engage more with
false stories than with true). We find that fake news is self-reinforcing when
attention to accuracy is low (i.e. when receivers either cannot assess or do
not pay attention to the accuracy of a headline) and when attention to payoffs
is low. This suggests that increasing reader attention to the news they are
consuming can reduce not only their own consumption of misinformation but also
the production of misinformation by transmitters who make faulty assumptions
about receiver preferences.

In the SI (Section 2.5) we also explore the impact of microtargetting and
receiver attention to accuracy. We find that when microtargeting is low, the
difference in engagement between false versus true stories declines. Thus the
ability to target news stories at specific receivers, either by news sites
directly or by social media algorithms, can exacerbate the apparent
attractiveness of false news stories (Figure S5). We also explored the impact
of three forms of receiver attention: i) attention to and/or prior knowledge
of story accuracy, $a_{0}$, ii) memory of past interactions when deciding to
engage with a news source, $a_{1}$, and iii) long-term attention to payoff,
parameterized as the amount of noise in the receiver optimization process,
$\sigma$ (see Methods and SI Section 2). We find that increasing prior
knowledge of or attention to headline accuracy reduces both the rate of
engagement with and the overall consumption of false stories under a myopic
optimization process (Figure S9). Similarly, increasing attention to payoffs
reduces engagement and consumption of false stories (Figure S9).

## Discussion

Understanding why people engage with fake news has become increasingly urgent,
as misinformation threatens to undermine democratic institutions and fuel
populist movements and polarization, as well as undermining public health
efforts and furthering the impact of the COVID-19 pandemic. Here, we shed new
light on misinformation by considering not just the consumers of fake news,
but also the producers. We develop a theoretical framework to study the
strategic interaction between readers and news sources. Our analysis makes two
key predictions for the dynamics of engagement with fake news. First, even a
misinformation site committed to pushing as much fake news as possible into
the information ecosystem should – when faced with readers who typically value
accuracy – publish a substantial amount of true news. Second, misinformation
site strategies that are successful in producing engagement among readers will
typically produce patterns of engagement in which the false stories receive
more engagement than true stories receive, even when false stories have no
greater inherent appeal over true stories. In contrast, mainstream sites that
seek to promote engagement with accurate stories tend to produce the opposite
engagement pattern. Both of these predictions are confirmed by our
experiments.

The dissemination strategies that induce these correlations are “coercive”,
since they effectively exert control over the behavior of receivers and push
them towards the transmitter’s preferred equilibrium, by increasing the amount
of accurate information they share when engagement trends in the wrong
direction. Strategies like these have been studied widely in other contexts,
such as the repeated Prisoner’s Dilemma [22, 23, 21, 24, 25] where an
“extortioner” can coerce their co-player into conceding a higher payoff to an
“extortioner” than they receive themselves. In psychological settings, these
types of behavioral strategies have been characterized as antisocial
reciprocity, where they are associated with pro-self social behavior [26].

Our analysis also helps to resolve an apparent contradiction in prior
empirical work. Prominent work has found that false stories get more
engagement than true stories, and argues that this occurs because false
stories are more novel (and thus more attractive to read) [28]. Yet other
studies find that true stories get as much or more engagement than false
stories [7, 8]. Our results suggest that these different patterns can be
explained by the different sources of news examined in the different studies.
Among claims that have been fact-checked by Snopes (largely coming from
misinformation sources) - as in [28] \- our model predicts that false news
gets more engagement. Conversely, this is not true when claims come from a
more balanced set of sources, as in [7, 8]. Of course, our findings do not
exclude the possibility that novelty may also drive engagement with fake news
[28], but our work shows that high levels of fake news engagement can arise
even in the absence of intrinsic novelty appeal or motivated reasoning, and
especially when transmitters can microtarget their content.

The idea that misinformation sites can successfully coerce readers into
engaging with false stories has important practical implications. First, our
findings suggest a new approach for identifying outlets that are seeking to
spread misinformation: examining the relationship between articles’ engagement
and perceived accuracy. Outlets from which particularly implausible articles
generate particularly high levels of engagement may be employing coercion
strategies to spread misinformation. Observing such a relationship could be
used as a signal that an outlet is intent on spreading misinformation, or
making faulty assumptions about the preferences of readers. Similarly, if such
a relationship is observed in the patterns of engagement with inaccurate
stories on a particular social media platform, this may suggest the platform
uses coercive strategies to promote false news stories to users. Second, the
tendency to share fake news can become self-reinforcing on the part of
transmitters: sites that seek to spread misinformation using coercion
strategies can create the incorrect impression that readers prefer falsehoods.
This pattern, if taken at face value, may then lead to increased false news
production by sites that do not have any particular agenda beyond simply
maximizing engagement – even when such sites would actually maximize
engagement by publishing true articles. Third, we find that this apparent
preference for false news can be reversed by encouraging readers to pay
attention to key aspects of their news consumption habits. This observation
reinforces the importance of (in)attention in combating misinformation.

In sum, we have demonstrated the importance of the feedback loop between news
publishers and readers. We have shown that focusing only on reader behavior
can lead to incorrect conclusions about readers’ preferences, while
considering publisher strategies can resolve apparent contradictions in the
literature, and can highlight new directions for combating misinformation.
News consumption does not occur in a vacuum, and to understand the dynamics of
misinformation it is essential to explore supply as well as demand.

## Methods

Game dynamics: We consider a transmitter strategy $\mathbf{r}$ as described by
Eq. 1 such that the probability of a transmitter sharing true news is
$r_{ki}$, where the last piece of news shared was consumed by $k$ receivers
and was of type $i$. The possible types of news are true ($t$) or fake ($f$),
so that $i\in\\{t,f\\}$, and in a population of $N$ receivers the amount of
consumption is $k\in[0,N]$. The transmitter strategy space we consider is thus
memory-1. When using a strategy of this type, a transmitter unilaterally
enforces a linear relationship between receiver consumption and the amount of
fake news shared by the transmitter (see Methods and SI Section 1). This
relationship is given by Eq. 3 and holds regardless of the receiver strategy
$\mathbf{q}$, i.e. it holds for other types of receiver strategy than given by
Eq. 2, including cases where the receiver strategy employs memory of their
full history of interactions with the transmitter (see SI Section 2).

To see this, encode the state of the repeated game in round $j$ as
$v_{ki}^{j}$ where $k$ is the number of receivers who consume in round $j$ and
$i$ is the type of news shared by the transmitter. The probability that the
game is in state $v_{ki}^{j+1}$ at round $j+1$ is then given by

$v_{kt}^{j+1}=\sum^{N}_{m=0}\sum_{l\in\\{t,f\\}}v_{ml}^{j}r_{ml}\sum_{A\in
F_{k}}\prod_{\alpha\in A}q_{\alpha}\prod_{\beta\in A^{C}}(1-q_{\beta})$ (4)

where $\text{Pr}(K=k)=\sum_{A\in F_{k}}\prod_{\alpha\in
A}q^{\alpha}\prod_{\beta\in A^{C}}(1-q^{\beta})$ is the Poisson Binomial
distribution describing the probability that $k$ receivers engage, where
$F_{k}$ encodes the possible subsets of $k$ integers that can be selected from
$N$, i.e. the possible combinations of receivers who can consume a piece of
news to produce overall consumption level $k$. The set $A$ is just a
particular such subset. Note that we allow each transmitter strategy
$q_{\alpha}$ and $q_{\beta}$ to be different and depend on the full history of
play of the receivers and transmitter i.e. we place no particular constraint
on the receiver strategy used. The master equation for $v_{kf}$ is defined in
the equivalent way with $(1-r_{ml})$ in place of $r_{ml}$ in Eq. 4.

At equilibrium the solution to Eq. 4 must satisfy
$v_{ki}^{j+1}=v_{ki}^{j}=v_{ki}$. We show in the SI (Section 1) that a
transmitter strategy $\mathbf{r}$ of the form Eq. 2 results in an equilibrium
solution satisfying Eq. 3. Since we assume an infinitely repeated game the
dynamics are guaranteed to converge to this equilibrium. Note that we assume
that receiver and transmitter actions are subject to a small execution error
at rate $\varepsilon$, which ensures that the Markov process described by Eq.
4 has no absorbing states, and a unique stationary distribution.

Coercive and extortionate strategies: We define a coercive strategy as a
strategy which assuredly increases the amount of news served to receivers of a
given type as the receivers increase their engagement. Thus a coercive fake
news strategy increases the rate of sharing fake news stories as receivers
increase their engagement. For this to be true, the coefficients
$\frac{\theta}{1-\alpha+\beta}$ and $\frac{\gamma}{1-\alpha+\beta}$ in Eq. 3
must be negative.

In order for a strategy to be viable Eq.2 must produce viable probabilities,
i.e. $r_{ki}\in[0,1]$. This in turn requires $0\leq\alpha\leq 1$,
$0\leq\beta\leq 1$, $-\alpha\leq\gamma\leq 1-\alpha$ and $-\beta\leq\theta\leq
1-\beta$. A viable coercive fake news strategy thus requires
$-\alpha\leq\gamma<0$ and $-\beta\leq\theta<0$. Equivalently, a coercive
mainstream news (i.e. true news) strategy requires $0<\gamma\leq 1-\alpha$ and
$0<\theta\leq 1-\beta$.

We can further define an extortionate fake news strategy as one that ensures
the amount of fake news shared always exceeds the amount of news consumed i.e.
$v_{f}>v_{fc}+v_{tc}$. For a fake news site an extortion strategy has the form
$\mathbf{r}_{-}^{*}=\big{\\{}\theta=-\beta,1-\alpha-\beta<\gamma\leq
0\big{\\}}$ (see SI Section 1). Conversely, an extortion strategy for a
mainstream site ensures that the amount of true news shared always exceeds the
amount of news consumed i.e. $v_{t}>v_{fc}+v_{tc}$. For a mainstream site an
extortion strategy has the form
$\mathbf{r}^{*}_{+}=\big{\\{}\gamma=1-\alpha,0\leq\theta<1-\alpha-\beta\big{\\}}$
.

Simulations: We performed simulations to determine the most successful
transmitter strategies for fake and mainstream sites. We selected a
transmitter strategy $\mathbf{r}=\\{\alpha,\beta,\gamma,\theta\\}$ (Eq. 2)
with each parameter drawn uniformly from the interval required to produce a
viable strategy. We initialized a receiver with a strategy that does not
consume any news from the transmitter, i.e. $\mathbf{q}=\\{0,0,0,0\\}$ (Eq. 1)
which was then allowed to update under a local optimization process (see SI
Section 1). For each set of receiver and transmitter strategies, we calculated
the stationary distribution for transmission and consumption of news stories
$\mathbf{v}=\\{v_{tc},v_{tn},v_{fc},v_{fn}\\}$ under the iterated fake news
game. For a single receiver the stationary distribution can be found
explicitly. For a larger population of receivers we simulated $10^{4}$ rounds
of the game, which allowed us to estimate the stationary distribution
numerically. In all simulations we assume that both transmitters and receivers
experience execution errors [37] with probability $\varepsilon=10^{-3}$.

We used the stationary distribution $\mathbf{v}$ to calculate receiver payoffs
at equilibrium. We then allowed receiver strategies to update under a local
optimization process (see SI Section 1.5 for further details). After a burn in
period of $10^{4}$ update events we measured the average engagement and
consumption rate of receivers over an additional $10^{4}$ update events.

Transmitters were identified as employing extortion strategies, $r_{+}^{*}$ or
$r_{-}^{*}$, if their strategy lay within a $\Delta$-neighborhood of a “true”
extortion strategy [24, 21], with $\Delta=0.05$. Significant over-
representation of extortion strategies among the most successful fake and
mainstream sites (i.e those strategies that successfully produce engagement
with fake and true stories respectively) was then determined by comparing the
prevalence of the strategy in our data to the null distribution, i.e. the rate
at which such strategies are randomly drawn. Significance level quoted is
$p<0.01$.

Experiments: We asked participants to evaluate the accuracy of 1,000 articles
across 40 mainstream and fake news sites, gathered from Crowdtangle. The study
was approved by MIT COUHES (protocol 1806400195).

Participants. From 22 to 29 November 2020, we recruited 1,000 participants
from Amazon Mechanical Turk who met the following three criteria: located in
the US, more than 100 studies completed on the platform, and more than 95 per
cent of them approved. The sample included 576 males and 424 females, with a
mean age of 38.49 years (min. 19; max. 96). Median completion time was 4
minutes and 55 seconds.

Materials. We used Crowdtangle to gather the 25 most recently available news
stories from 40 media outlets (i.e. 1,000 articles; 500 posted by 20 fake news
sites and 500 posted by 20 mainstream sites), along with the headline, lede,
date of publication, link and level of engagement. See Supplementary
Information Section 3 for additional details. We then used this data to
present the participants outlined above with 20 article headlines and ledes,
drawn randomly from within one of the two media outlet subsets, and asked them
to assess the accuracy of the information they were faced with. Specifically,
they were asked ”Do you think this story is true?”, to which they responded on
a seven-point scale from “Definitely NO” to “Definitely YES”. The study
concluded with seven demographic questions (age, gender, education, political
conservativeness on social and economic issues, political position, and
political preference) and a section to leave comments.

## Supplementary Information

In this supplement we present a formal analysis of the fake news game, along
with additional details of the modeling framework. We then compare our model
to three prior models of bias and misinformation in news production, followed
by robustness checks of our findings to alternate model formulations and
assumptions. Finally we present additional details of experiments and meta-
analysis.

## Formal analysis of the fake news game

The stage game payoff matrix for the fake news game is described in Table S1
for a single receiver and a single transmitter. We consider an infinitely
iterated version of this game, which in general is played by a single
transmitter and $N$ receivers, as described in the Methods section of the main
text. This iterated game describes the long term dynamics of a receiver being
served news stories by a transmitter.

Table S1:  Stage game payoff matrix

### Stationary distribution

The stationary distribution for the infinitely repeated game is encoded by the
vector of probabilities $\mathbf{v}$. Each entry $v_{ki}\in[0,1]$ describes
the stationary probability that the game is in state $(k,i)$, where
$k\in\\{0,1,2,\ldots N\\}$ gives the number of receivers who choose to consume
the piece of news shared by the transmitted, and $i\in\\{t,f\\}$ encodes the
type of news shared. The master equation for the Markov process associated
with the infinitely repeated game is given by Eq. 4 of the main text. At
equilibrium, for a true news item $i=t$, this equation has the form m

$v_{kt}=\sum^{N}_{m=0}\sum_{l\in\\{t,f\\}}v_{ml}r_{ml}\sum_{A\in
F_{k}}\prod_{\alpha\in A}q_{\alpha}\prod_{\beta\in A^{C}}(1-q_{\beta}).$ (5)

An analogous equation holds for $i=f$ by replacing $r_{ml}$ with $(1-r_{ml})$.

In order to solve for the stationary distribution of this process,
marginalized over all possible consumption values $k$, we first note that the
overall rate of sharing fake news by the transmitter is

$v_{f}=\sum_{k=0}^{N}v_{kf}$ (6)

and $v_{t}=1-v_{f}$ is the overall rate of production of true news by the
transmitter. Applying this definition to Eq. 5 we recover

$v_{t}=\sum^{N}_{m=0}\sum_{l\in\\{t,f\\}}v_{ml}r_{ml}$ (7)

since in summing over all possible engagement levels $k$, we sum over the
probabilities of all possible combinations of receiver consumption which
necessarily total 1. If we now replace $r_{ml}$ in Eq. 7 with the form given
in Eq. 2 we recover

$v_{t}=\sum^{N}_{m=0}v_{mt}\left(\alpha+\gamma\frac{m}{N}\right)+v_{mf}\left(\beta+\theta\frac{m}{N}\right).$
(8)

Now note that

$v_{tc}=\sum^{N}_{m=0}v_{mt}\frac{k}{N}$ (9)

is the overall rate of receivers consuming true news. And so Eq. 8 can be
written as

$v_{t}=\alpha v_{t}+\gamma v_{tc}+\beta v_{f}+\theta v_{fc}$ (10)

Replacing $v_{f}=1-v_{t}$ and rearranging gives us the relationship between
the proportion of transmitted stories that are false and the overall rate at
which receivers consume true and false stories, presented in Eq 3 of the main
text.

### Definition of coercion and extortion strategies

We have defined a coercive transmitter strategy as one that always serves more
news of a particular type to receivers who consume more news. Thus a coercive
fake news strategy serves more fake stories to receivers who consume more of
the transmitter’s stories. This is described as “coercive” because receivers
who seek to increase their benefit from true stories by engaging more, in fact
receive increasing amounts misinformation. In contrast, receivers who consume
little of the transmitter’s stories are targeted with more true stories and
thus are incentivized to engage more. The conditions for a strategy to be
coercive are $\gamma>0$ and $\theta>0$ (see Methods).

We define an extortion strategy as a particular type of coercive strategy that
additionally enforces $v_{f}>v_{c}$ (for fake news sites), where
$v_{c}=v_{tc}+v_{fc}$ is the overall rate of consumption by receivers.
Similarly, for mainstream sites extortion strategies must enforce
$v_{t}>v_{c}$ . The term “extortion” was previously introduced in reference to
analogous strategies studied in the Prisoner’s Dilemma [22, 24, 23, 21].
Extortion strategies in the fake news game constitute a strong form of
coercion. In particular the extortion strategies studied here ensure that the
rate of increase in fake (or true) news exceeds the rate of increase of
overall consumption by receivers.

### Necessary and sufficient conditions for extortion

The condition for a fake news site to be in extortioner is that $v_{f}>v_{c}$
for all receiver strategies. In order to find necessary and sufficient
conditions for a strategy to enforce such a relationship, first define the
following parameters [24, 20]

$\kappa=\frac{1-\alpha-(\gamma-\theta)/2}{1-\alpha+\beta+\theta}$

$\lambda=-\frac{(\gamma-\theta)/2}{1-\alpha+\beta+\theta}$

$\chi=-\frac{\gamma+\theta}{2(1-\alpha+\beta-(\gamma-\theta)/2)}$ (11)

where $|\chi|\leq 1$ is required to produce a viable strategy (see Methods).
Note that the denominator in Eq 11, namely
$(1-\alpha+\beta-(\gamma-\theta)/2)$, is required to be positive to produce a
viable strategy (see Methods) and thus any fake news promoting coercive
strategy necessarily has $\chi>0$.

Substituting Eq. 11 into Eq. 3 produces the following expression

$v_{f}-\kappa+\lambda(v_{tn}+v_{fc})=\chi\left[v_{c}-\kappa+\lambda(v_{tn}+v_{fc})\right]$
(12)

where $v_{tn}$ is the rate at which true news is shared but not consumed. Eq.
12 is convenient because it allows us to state the conditions for extortion as
i) $\chi>0$ (which is necessary for the strategy to be coercive) and ii)
$v_{c}\leq\kappa+\lambda(v_{tn}+v_{fc})$ for all possible receiver strategies
(which is necessary to ensure $v_{f}>v_{c}$, given that $|\chi|\leq 1$). If
these two conditions are met, the strategy is coercive and $v_{f}>v_{c}$ for
all receiver strategies, and so the strategy is an extortioner.

The first condition is met provided $\gamma+\theta<0$ (since $\gamma<1-\alpha$
is required to produce a viable strategy). To asses the second condition note
that ii) implies $v_{f}+v_{c}\leq 2\kappa-2\lambda(v_{tn}+v_{fc})$. Also note
that $v_{f}+v_{c}=1+v_{fc}-v_{tn}$ and so condition ii) is equivalent to

$1+v_{fc}-v_{tn}\leq 2\kappa-2\lambda(v_{tn}+v_{fc})$ (13)

All that remains is to inspect the limiting cases, where this conditions is
most stringent. A receiver who never consumes enforces $v_{fc}=0$ and Eq. 13
becomes $1-v_{tn}\leq 2\kappa-2\lambda v_{tn}$ which is guaranteed to hold
provided

$\kappa\geq 1/2$

and

$\kappa\geq\lambda$ (14a)

The other limiting case of a receiver who always consumes enforces $v_{tn}=0$
and generates conditions

$\kappa\geq 1/2$

and

$\kappa\geq 1+\lambda.$ (14b)

Eq. 14b is the more stringent of these two conditions and thus it provides
necessary and sufficient conditions for extortion. Substituting back from Eq.
11 these conditions become

$\theta=-\beta$

and

$\alpha+\beta+\gamma\leq 1.$ (15)

The equivalent condition for a mainstream transmitter is

$\gamma=1-\alpha$

and

$\alpha+\beta+\theta\leq 1.$ (16)

These are the conditions used to define extortion strategies discussed in the
main text.

### Nash equilibria in the fake news game

Our model assumes inattentive receivers who noisily optimization their
engagement strategies as they seek to consume true stories. Such a process
does not necessarily lead to optimal receivers behavior, or even to behaviors
predicted by standard solution concepts such as Nash equilibria. However, it
is useful to characterize Nash equilibria for the fake news game.

For a mainstream site, which seeks to promote accurate information, there is
no conflict between the incentives of transmitter and receiver, and so they
both should adopt a Nash equilibrium in which the receiver always engages and
the transmitter only shares true stories. For a fake news site, which seeks to
promote false news, there is a conflict between the incentives of transmitter
and receiver.

The Nash equilibria for the two-player fake news game can be easily
characterized using the Folk Theorem for infinitely iterated games [29]. The
minimax value for receivers in the fake news game is 0, which is achieved by
never consuming. If the receiver does consume, the transmitter can always hold
them to a lower payoff by transmitting fake news, assuming that the cost of
consuming fake news is positive, $C>0$. The minimax value for the transmitter
is also 0, since we assume the payoff to the transmitter of fake news being
consumed is $b_{f}\geq 0$ and also the payoff from true news being consumed is
$b_{t}\geq 0$, whereas the payoff when no news is consumed is 0, and so the
transmitter’s minimax profile is uniquely determined by a receiver strategy
that never consumes (Table S1). Thus any feasible payoff profile that provides
a positive payoff to both players is individually rational.

Figure S1: Feasible, individually rational payoffs for the fake news game –
The set of feasible payoffs (blue region) and the subset of individually
rational payoffs for both players (striped region) for a fake news game with
$b_{f}>b_{t}$ meaning that the transmitter prefers to generate consumption
with fake news. Individually rational payoffs for both players only arise when
both players receive a positive payoff. An equilibrium in which the receiver
always consumes and the transmitter always shares fake news, $(-C,b_{f})$ is
feasible but not individually rational.

Any individually rational, feasible payoff is a Nash equilibrium of the game
(Figure S1), and, since the game is assumed infinite, can always be enforced
by players employing a grim trigger strategy. That is, if the receiver
deviates from the prescribed course of play, the transmitter switches to only
sharing fake news. Any benefit gained by the receiver from deviating is lost
in the long run, since the best they can do is to cease consuming and receive
zero payoff in every subsequent round. Conversely, if the transmitter deviates
from the equilibrium, the receiver can switch to never consuming news, with
the same result that any benefit to the transmitter gained by deviating
eventually lost. As discussed above, the set of individually rational,
feasible payoffs (Figure S1) in the fake news game corresponds to cases in
which both transmitter and receiver have positive payoffs. Although a fake
news site receives payoffs $b_{t}=0$ and $b_{f}>0$, this nonetheless means
that they must share a mixture of both true and fake news, since the payoff
that results from only sharing fake news produces a negative payoff to the
receiver.

To explore this point further, consider a fake news site who simply transmits
true stories with fixed probability $r$ each round. The payoff to a receiver
who consumes news from the receiver at equilibrium rate $v_{c}$ is simply
$w_{t}=v_{c}(Br-C(1-r))$, which is positive provided $r>\frac{C}{B+C}$. If
this condition is met, the receiver’s payoff increases with consumption, and
so an equilibrium at which the receiver always consumes and the transmitter
shares true stories with probability $r>\frac{C}{B+C}$ can be enforced as a
Nash equilibrium of the fake news game (e.g. by employing a grim trigger, as
discussed above). This illustrates the need, in a general sense, for a fake
news site to “mash up” true and fake stories to produce engagement. Finally,
note that if $b_{t}>0$, $r=1$ and $v_{c}=1$ produces positive payoffs for both
players, and can also be enforced as a Nash equilibrium – i.e. mainstream news
sites do not need to mash up true and fake stories when faced with rational
receivers who value true news.

Thus our model predicts that misinformation sites should always share a
mixture of true and fake stories if their readers value accuracy, regardless
of whether those readers are rational (as per this Nash equilibrium
calculation), or lazy and irrational (as per our simulations above).

### Modelling local optimization

For the most part we do not assume that receivers in the fake news game are
perfectly rational, but rather update their behavior according to a local
optimization process [31]. Under this process a receiver temporarily adopts an
alternate strategy $j$ to the one they are currently employing, $i$, and
compares the payoff they receive with the new strategy to that received under
the old strategy. They then adopt the new strategy, and discard the old, with
probability $\pi_{i\to j}$ determined by a Fermi function.

$\pi_{i\to j}=\frac{1}{1+\exp[\sigma(w_{i}-w_{j})]}$ (17)

where $w_{i}$ is the payoff received under strategy $i$, and $\sigma$
determines the level of attention the player pays to their payoffs. If
$\sigma<1$ little attention is paid to payoffs and the optimization process is
very noisy, with receivers frequently adopting suboptimal consumption
strategies. If $\sigma\gg 1$ a high level of attention is paid to payoffs, and
only strategies that improve payoffs are adopted.

When choosing the alternate strategy we assume that $j$ is small perturbation
$\Delta_{\mu}\in[0,0.05]$ to the current strategy, such that each element of
the receiver strategy $\mathbf{p}=\\{p_{0},p_{ct},p_{cf},p_{nt},p_{nf}\\}$ is
increased or decrease by an independently drawn $\Delta_{\mu}$, with the
constraint that each element must remain a viable probability (i.e.
$p_{ij}\in[0,1]$).

### Modelling optimization through social learning

In addition to local optimization, receivers whose strategies are updated
through a process of cultural evolution via payoff-biased imitation [31].
Under this model, a pair of receivers, $i$ and $j$, are selected at random
from the population of receivers. Receiver $i$ then adopts the strategy of
receiver $j$ with probability

$\phi_{i\to j}=\frac{1}{1+\exp[\sigma(w_{i}-w_{j})]}$ (18)

In addition, mutations are introduced to the receiver population at rate
$\mu$, under which a player spontaneously adopts a news strategy drawn
uniformly from the space of possible receiver strategies. Note that Eq. 17 and
Eq. 18 are identical, but here $j$ is drawn from the strategies present in the
population, while in Eq. 17, $j$ is chosen as a perturbation to the current
strategy. As such, the process of copying other players’ strategies tends to
result in many different players using the same strategy, compared to the case
where each player independently optimizes their own strategy.

## Comparison to previous models

In this section we compare our model to three previously proposed theories of
misinformation engagement and/or production [16, 28, 11]. We ask whether the
theories presented in these papers can explain our empirical results (main
text Figure 3). That is, we ask whether our proposed theory is uniquely
predictive of the observed data. We find that, indeed, the other theories
cannot reproduce the observed patterns.

### Vosoughi et. al. 2018

Vosoughi et. al. [28] present empirical results showing that, among news
stories that were questionable enough to get fact-checked – by snopes.com,
politifact.com, factcheck.org, truthor-fiction.com, hoax-slayer.com, or
urbanlegends.about.com – false news spread further on social networks and
persisted longer. They also find that false news was more novel, in an
information-theoretic sense, than true news. The authors are careful to note
that “Although we cannot claim that novelty causes retweets or that novelty is
the only reason why false news is retweeted more often, we do find that false
news is more novel and that novel information is more likely to be retweeted.”

This suggests that readers derive benefit from engaging with misinformation
due to its novelty. However such an explanation, absent feedback between
receiver and transmitter sharing strategy, would predict greater engagement
with false stories among both fake and mainstream sites , which is not
consistent with our empirical findings (main text Figure 3).

### Pennycook et. al 2021

Pennycook et. al [11] investigate the role of inattention in shaping consumer
engagement with misinformation. The authors present a model in which receiver
utility is derived from a consumer’s preferences and the level of attention
they pay to a topic. Fitting the model parameters to experimental data they
find “the best-fit preference parameters indicate that participants value
accuracy as much as or more than partisanship. Thus, they would be unlikely to
share false but politically concordant content if they were attending to
accuracy and partisanship” suggesting a model in which consumers prefer true
news to false news, and occasionally share misinformation accidentally, due to
inattention. This account would therefore predict greater engagement with true
stories than false stories, regardless of whether to producer is a fake news
or mainstream site, which is inconsistent with our empirical findings (main
text Figure 3).

### Allcott and Gentzkow 2017

Allcott and Gentzkow [16] develop a model of supply and demand of fake news.
Under this model “Fake news arises in equilibrium because it is cheaper to
provide than precise signals, because consumers cannot costlessly infer
accuracy, and because consumers may enjoy partisan news.”

This model considers consumer decisions at the level of the news outlet (e.g.
“shall I subscribe to this newspaper or not?”) rather than at the level of the
individual story, and thus does not make direct predictions about our key
empirical outcome, the relative rate of engagement for true versus false news
within a given outlet. However, the model’s implications can be extrapolated
to our setting. On the demand side, this means that consumers/receivers either
(i) engage with fake news accidentally due to inattention or (ii) derive
benefit from engaging with fake news due to e.g. partisan preference.
Inattention alone, absent feedback between transmitter and receiver sharing
strategy, leads in the limit to equal levels of engagement with true and false
news. Similar to the case of a consumer preference for novelty discussed
above, a partisan preference for false news, absent feedback between receiver
and transmitter sharing strategy, would predict greater engagement with false
stories among both fake and mainstream sites, which is not consistent with our
empirical findings (main text Figure 3).

On the supply side, this theory posits that transmitters broadcast false news
because it is cheaper to produce than true news (i.e. the net payoff for
transmitters from receivers engaging with false news is greater than it is for
true news). By this logic, it is payoff maximizing for transmitters to produce
false news so long as the engagement advantage of true news over false news is
sufficiently small (or non-existent) – i.e. due to receivers being inattentive
and/or preference for falsehoods. This explanation for the production of fake
news, while entirely plausible, makes no prediction for the different patterns
of engagement among consumers with mainstream versus fake news sources, which
is our key empirical result (main text Figure 3).

## Model robustness

In this section we perform robustness checks on our model, and show that our
results hold as we alter our model specification. We show that i) our results
are robust to different definitions of “successful” transmitters ii) our
results are robust to different models of receiver learning, iii) our results
are robust to different receiver group sizes but require a moderate to high
degree of transmitter microtargetting, iv) our results require low levels of
receiver attention, v) our results are robust to competition between
transmitters and vi) our results are robust to changes to consumer demand for
misinformation.

### Identifying successful transmitter strategies

In order to find successful mainstream and fake news transmission strategies
we generated $10^{8}$ receiver strategies as described in Methods. We then
defined successful fake news transmitters as those which achieved i) in the
90th percentile for fake news engagement, and ii) a higher rate of
transmitting fake than true news, $v_{f}>0.5$. Similarly, we defined
successful mainstream transmitters as those in the 90th percentile for true
news engagement who transmit a higher rate of true news than fake,
$v_{t}>0.5$. These definitions were used to select the strategies show in
Figure 2 of the main text. The full, unfiltered distribution of engagement and
consumption for the generated strategies is shown in Figure S2a-b. The
distributions have a weak bias towards true news consumption ($p<0.001$)
reflecting the noisy optimization process of the receivers, who prefer to
consume true and avoid fake news.

Figure S2: Full distribution of engagement and consumption – Shown are the
distributions of the difference in engagement (a) and consumption (b) with
true vs fake stories for $10^{8}$ transmitter strategies. We define successful
fake news strategies as those with $v_{f}>0.5$ and fake news engagement,
$v_{fc}/v_{f}\geq 0.704$ which is the 90th percentile among all observed
strategies. Similarly we define successful mainstream strategies as those with
$v_{t}>0.5$ and fake news engagement, $v_{tc}/v_{t}\geq 0.758$ which is the
90th percentile among all observed strategies. c) We calculated the average
difference in engagement as a function of the threshold rate of sharing the
transmitter’s proffered news type used when to determine which transmitter
strategies are successful. We see that both mainstream (blue) and fake news
sites (red) produce an engagement difference that is insensitive to this
choice of threshold. d) However the overall consumption rate is highly
sensitive to the threshold, so that when the threshold is low the preferred
news type is consumed less frequently than the non-preferred type. This arises
because many successful sites produce high levels of engagement with their
preferred news type while sharing it rarely.

We also explored the effect of changing the definition of “successful”
transmitters of true and fake news. In the main text we look at sites in the
top 10% of engagement, who share their preferred news type greater than 50% of
the time. In Figure S2c-d we vary this threshold and look at the average
difference between engagement and consumption. We see that engagement
difference is insensitive to the threshold choice, whereas consumption
difference reverses direction when the threshold is low, indicating that there
are many news sites who are good at producing engagement with true or fake
news while also sharing such stories rarely.

### The effect of receiver learning model choice

We studied the effect of different receiver learning model choice. In
particular we explored social learning via a model of cultural evolution, as
described in Section 1.6 of this Supplement, on the level of engagement and
consumption in a population of receivers. Figure S3 shows the impact of group
size on engagement and consumption of fake and true news with a single fake
news transmitter as in Figure 4 of the main text. Assuming that the rate of
strategy mutations is $\mu=1/N$ where $N$ is the size of the group of
receivers, we see that increasing group size has no effect on consumption and
only a modest effect on engagement, tending to increase true news engagement
and reduce fake news engagement, without reversing the pattern of higher
engagement with fake stories than with true.

Figure S3: Cultural evolution of receiver strategy – a) A group of receivers
evolving in response to a single transmitter, under the same process as
described is Section 1.6 of the Supplement. We vary the size of the receiver
group (x-axis) and calculate the equilibrium level of engagement with fake
(orange) and true (purple) stories. b) We also calculate the equilibrium level
of consumption. In all cases receiver mutations were local (see Methods) and
we set receiver payoffs $B=2$ and $C=1$. Here misinformation sites use a fixed
coercive strategy $\mathbf{r}=\\{1.0,0.1,-0.9,-0.1\\}$ see Methods

### The effect of receiver group size and transmitter microtargetting

We studied the effect of relaxing the assumption, employed in the main text,
that a transmitter can directly target each receiver, in response to his or
her engagement with the previous news article. We show that the effects
observed – in particular the apparent attractiveness of fake news – hold under
a wider range of conditions in which a transmitter targets groups of
individuals.

First we studied the effect of varying the size of the group of receivers
engaging with a single transmitter, when all receivers use the same strategy
to engage with the transmitter, with optimization taking place at the level of
the group rather than at the level of the individual, as in Figure 5 of the
main text. Figure S4 shows that, in such a scenario, group size has no impact
on either engagement or consumption even as group size varies across two
orders of magnitude.

Figure S4: Homogenous receiver groups – a) A group of receivers optimizing in
response to a single transmitter, under the same process as described for main
text Figure 2, with the extension that that optimization occurs at the level
of the group rather than the individual. We vary the size of the receiver
group (x-axis) and calculate the equilibrium level of engagement with fake
(orange) and true (purple) stories. b) We also calculate the equilibrium level
of consumption. In all cases receiver mutations were local (see Methods) and
we set receiver payoffs $B=2$ and $C=1$. Here misinformation sites use a fixed
coercive strategy $\mathbf{r}=\\{1.0,0.1,-0.9,-0.1\\}$ see Methods

Third we explored the effect of microtargetting on receiver engagement with
misinformation in the face of a coercive transmitter. The engagement dynamics
between a single transmitter and single receiver corresponds to the case of
direct micro-targeting by a transmitter, in the sense that the transmitter
makes their decision about what type of news to share (e.g. by promoting
stories on social media) with a receiver based solely on the engagement (or
not) of that specific receiver. In reality, this degree of precision is
unrealistic, but as we have shown above (Figure S3-S4) the same effects also
hold when a transmitter’s audience is a group of receivers who employ similar
behavioral strategies.

In order to explore how micro-targeting impacts the engagement and consumption
habits of receivers interacting with a coercive fake news strategy, we define
the strength of micro-targeting as $M=1/G$, where $G$ is the number of
independent individuals or groups to whom the transmitter promotes the same
news. Here a “group” refers to a newsfeed, a social media page, or
distribution list to which the transmitter promotes content. A group may
contain only a single individual, or be composed of many people who share the
same interest. When a transmitter promotes the same content to many such
groups ($M\ll 1$), each group updates their engagement independently in
response, and so the strength of micro-targeting is low: a change in the
behavior of any one group has a small impact on the overall engagement with
the transmitter (Main Text Eq. 2), so the transmitter is not very responsive
to a given receiver. When the strength of micro-targeting is high (e.g. when
there is only one transmitter and one receiver, $M=1$) the transmitter is
highly responsive to the engagement pattern of a given receiver.

Figure S5 shows the effect of varying the degree of micro-targeting available
to the transmitter. We find that when micro-targeting is low the difference in
engagement between false versus true stories declines, until the apparent
attractiveness of fake news disappears entirely. Thus the ability to target
news stories at specific groups of receivers, either by news sites directly or
by social media algorithms, can substantially contribute to the apparent
attractiveness of false news stories.

Figure S5: Micro-targeting facilitates engagement with fake news a) A single
misinformation site micro-targeting multiple independent receivers. The
strength of micro-targeting $M$ (x-axis) quantifies the responsiveness of the
transmitter to each of the target groups of receivers (see main text). Each
target receiver group is assumed to independently optimize their engagement
strategy as described previously (Figure 2). The misinformation site uses the
same transmission strategy for all the groups included in their targeting. The
less precise the targeting of the transmitter (lower values of $M$), the
smaller the difference in engagement between true and false news stories. b)
However, the overall consumption rate remains unchanged with $M$. In all cases
receiver strategic mutations are local (see Methods) and receiver payoffs are
$B=2$ and $C=1$. Other parameter choices are explored in the SI. Here the
misinformation site uses a fixed coercive strategy
$\mathbf{r}=\\{1.0,0.1,-0.9,-0.1\\}$ see Methods

### The effect of different types of receiver attention

We studied the effects of varying different types of receiver attention to
news content and to transmitter behavior. In the analysis presented in Figure
2 of the main text we assume that receivers are inattentive to both prior
experience with a transmitter and prior information about the veracity of each
headline, i.e. $a_{0}=a_{1}=0$ in Eq. 1. In Figure S6 we repeat the same
analysis assuming receivers are attentive to past experience, i.e. $a_{1}=1$
in Eq. 1. We see that, as in Figure 2, the most successful fake news
strategies tend to produce greater consumption and engagement with fake
stories than with true. Conversely, the most successful mainstream strategies
produce more engagement and consumption of true stories than false.

Figure S6: Successful news transmission strategies with receiver memory – To
identify strategies that are successful at promoting engagement with true news
stories (mainstream sites) or with fake news stories (misinformation sites),
we randomly drew $10^{8}$ transmitter strategies (Eq. 2) and allowed a single
receiver, incentivized to engage with true news, to optimize their engagement
strategy (see Methods) over the course of $10^{4}$ interactions. The receiver
was assumed to be inattentive to accuracy ($a_{0}=0$) but attentive to past
stories ($a_{1}=1$) and optimization was assumed to be noisy ($\sigma=1$). a)
We identified the transmitter strategies that successfully promote engagement
with true news stories (mainstream sites, blue), i.e those that produce a true
news engagement rate within the 90th percentile from the $10^{8}$ transmitter
strategies considered, as well as $v_{f}<0.5$. Similarly, we identified the
transmitter strategies that successfully promote engagement with fake news
(misinformation sites, red), i.e those that produce a fake news engagement
rate within the 90th percentile, as well as $v_{f}>0.5$. We plot the
difference between engagement with true and fake stories,
$v_{tc}/v_{t}-v_{fc}/v_{f}$, for all mainstream and misinformation sites.
Mainstream site strategies induce engagement with accurate stories while
misinformation site strategies induce the opposite effect. b) We also report
the difference in overall consumption of true stories, $v_{tc}$, and
consumption of fake stories $v_{fc}$. c) Summary of the engagement rates per
story and overall consumption rates for mainstream and misinformation site
transmission strategies. In all cases receiver mutations were local and we set
receiver payoffs $B=2$ and $C=1$.

The distribution of engagement and consumption for all $10^{8}$ transmitter
strategies when receivers employ memory of their previous interaction
($a_{1}=1$) is shown in Figure S7. We see that the distribution of engagement
is narrower than in Figure S2, when receivers are inattentive to memory, while
the distribution of consumption follows a similar distribution in both cases.

Figure S7: Full distribution of engagement and consumption with receiver
memory – Shown are the distributions of the difference in engagement (a) and
consumption (b) with true vs fake stories for $10^{8}$ transmitter strategies.
We define successful fake news strategies as those with $v_{f}>0.5$ and fake
news engagement, $v_{fc}/v_{f}$ in the 90th percentile among all observed
strategies. Similarly we define successful mainstream strategies as those with
$v_{t}>0.5$ and fake news engagement, $v_{tc}/v_{t}$ in the 90th percentile
among all observed strategies.

We also considered (Figure S8) the effects of receivers conditioning their
behavior on memory of previous interactions beyond simply the immediately
preceding encounter. We assumed that receivers treated a transmitter as having
shared fake news with them when employing their strategy $\mathbf{p}$ (Eq. 1)
if the transmitter has shared a fake story in _any_ of the last $m$ rounds. We
show engagement and consumption as a function of such memory in Eq. S7, where
we see that longer memory of this type has only a weak effect on engagement
and consumption, and does not by itself reverse the trend of successful fake
news sites generating higher engagement and consumption of fake vs true
stories among inattentive consumers.

Figure S8: Receiver memory length – a) A single receiver optimizing in
response to a single transmitter, under the same process as described for main
text Figure 2. The receiver uses memory of preceding rounds to decide whether
the transmitter shares true stories (and so is labelled $t$) or fake stories
(and so is labelled $f$). We assume that if the receiver identifies any item
of fake news in their memory of the transmitter’s shared stories, they label
the transmitter as fake ($f$). We vary the length of this memory (x-axis) and
calculate the equilibrium level of engagement with fake (orange) and true
(purple) stories. b) We also calculate the equilibrium level of consumption.
In all cases receiver mutations were local (see Methods) and we set receiver
payoffs $B=2$ and $C=1$. Here misinformation sites use a fixed coercive
strategy $\mathbf{r}=\\{1.0,0.1,-0.9,-0.1\\}$ see Methods

We also studied the effect of varying attention to headline accuracy in Eq. 1,
against successful fake news promoting transmitter strategies. We varied
attention to accuracy, $a_{0}$, memory of past interactions, $a_{1}$ and
attention to payoffs, $\sigma$ among receivers. We also considered the case
where receivers both prefer true news (as in the main text) and the case where
they prefer fake news, i.e. derive benefit $B$ from interacting with fake news
and pay cost $C$ for interacting with true news (Figure S9).

Figure S9: Attention and news consumption – We randomly drew $10^{5}$
misinformation strategies (see Methods) and allowed receivers to optimize as
described in Figure 2 of the main text. (a-b) When receivers have no memory
$a_{1}=0$, and pay little attention to long-term payoffs, $\sigma=1$, but
prefer accurate information, we increased attention to accuracy in the
optimization process ($a_{0}$) (x-axis), and calculated a) the average rate of
engagement with true (purple) and fake (orange) stories. As attention to
accuracy increases, overall engagement with fake stories decreases to zero,
while engagement with true stories increases. b) Similarly, overall
consumption rate of fake stories decreases with increasing attention to
accuracy, while consumption of true stories increases. (c-d) When receivers
have no memory $a_{1}=0$, and no attention to accuracy, $a_{0}=0$ , but prefer
accurate information, we increased attention to long-term payoffs in the
optimization process ($\sigma$) (x-axis), and calculated c) the average rate
of engagement with true (purple) and fake (orange) stories. As attention to
payoffs increases, overall engagement with true and fake stories decreases to
zero. d) Similarly, overall consumption rate of true and fake stories
decreases with increasing attention to payoffs. (e-h) Shows the same plots as
in (a-d) except with memory of past interactions, i.e. $a_{1}=1$. (e-f) show
similar patterns as (a-b) while (g-h) shows increasing engagement and
consumption for true news (purple) with increasing attention to payoffs
($\sigma$). (i-l) Shows the same plots as in (a-d) except with receivers
preferring fake news, i.e. gaining benefit $B$ from interacting with a fake
story and paying a cost $C$ from interacting with a true story. (i-j) show
opposite patterns to (a-b) while (k-l) show k) increasing engagement for both
true and fake news engagement with attention to payoffs, but l) declining
consumption of true news. (m-p) Shows the same plots as in (e-h) except with
receivers preferring fake news, i.e. gaining benefit $B$ from interacting with
a fake story and paying a cost $C$ from interacting with a true story. (m-n)
show opposite patterns to (e-f) while (o-p) show opposite patterns to (g-h).
In all cases we set attention to payoffs $\sigma=1$ and receiver payoffs $B=2$
and $C=1$ unless otherwise stated.

We see that in many cases increasing attention not only reduces engagement and
consumption among receivers who prefer true news (Figure S9 panels a-h) it can
also reverse the apparent preference for fake news induced by coercive
transmitter strategies. However increased attention tends to have the opposite
effect on engagement and consumption when receivers prefer fake news (Figure
S9 panels i-p).

### The effect of competition between news sources

We studied the effect of competition between news sources, with a receiver
facing a single misinformation site within in a population of otherwise
mainstream sites (Figure S10). We see that the receiver tends to engage and
consume more fake news from the misinformation site as the population size of
mainstream sites grows. This makes intuitive sense, because the relative
contribution of the misinformation site to the receiver’s payoff necessarily
declines as the population grows, and so avoiding fake news becomes less
important to receiver payoff.

Figure S10 Groups of transmitters – a) A single receiver optimizing in the
face of a population of transmitters, under the same process as described for
Figure 2. The transmitter population consists of a single misinformation site
and $N-1$ mainstream sites. Increasing the number of mainstream sites
increases engagement with true (purple) and fake (orange) news stories from
the fake news transmitter as well as b) overall fake news consumption. In all
cases receiver mutations were local (see Methods) and we set receiver payoffs
$B=2$ and $C=1$. Other parameter choices are explored in the SI. Here
misinformation sites use a fixed coercive strategy
$\mathbf{r}=\\{1.0,0.1,-0.9,-0.1\\}$ and mainstream sites use a fixed coercive
strategy $\mathbf{r}=\\{0.9,0.0,0.1,0.9\\}$ see Methods

### The effect of supply and demand

We studied the effects consumer demand on transmitter supply of
misinformation. To do this we varied receiver payoff on consumption and
engagement, shown in Figure S11 and Figure S12. In Figure S11 we fix the cost
$C$ of consuming fake news, and vary the benefit $B$ of consuming true news,
for different levels of attention to payoff, $\sigma$. We see that increasing
benefits of true news often has only weak effects on consumption and
engagement, but do tend to increase overall levels of both once benefits
become sufficiently large.

Figure S11: Varying benefit of true news consumption – We randomly drew
$10^{5}$ coercive fake news strategies (see Methods) and allowed receivers to
optimize as described in Figure 2 of the main text. a) When attention to
payoff is low, ($\sigma=0.1$), cost of engaging with fake news is fixed,
($C=1$), and receivers have no memory and no attention to story accuracy,
$a_{0}=a_{1}=0$, we increased the benefit of engaging with true news stories
$B$ in the optimization process (x-axis), and calculated the average rate of
engagement with true (purple) and fake (orange) stories. As attention to
payoffs increases, overall engagement slowly increases and the difference in
engagement with true and fake stories declines. b) Similarly, consumption when
attention to payoffs is low ($\sigma=0.1$) remains close to constant. c) When
attention to payoffs is moderate ($\sigma=1$) the same pattern holds for
engagement while d) consumption slowly increases, and the difference between
true and fake news consumption also increases. e) When attention to payoffs is
strong ($\sigma=10$) there is little difference between true and fake news
engagement, and overall engagement increases with $B$ while f) overall
consumption increases, and the difference between true and fake news
consumption also increases. Unless otherwise stated parameters are the same as
in Figure 4 of the main text.

In Figure S12 we fix the benefit $B$ of consuming true news, and vary the cost
$C$ of consuming fake news, for different levels of attention to payoff,
$\sigma$. We see that increasing costs of fake news has a strong effect on
consumption and engagement, reducing both close to zero once costs become
sufficiently large. This result is consistent with our game-theoretic
expectation that higher costs require a fake news sharing strategy to include
ever higher levels of true news to make engagement worthwhile for the
receiver.

Figure S12: Varying cost of true news consumption – We randomly drew $10^{5}$
coercive fake news strategies (see Methods) and allowed receivers to optimize
as described in Figure 2 of the main text. a) When attention to payoff is low,
($\sigma=0.1$), benefit of engaging with true news is fixed, ($B=1$), and
receivers have no memory and no attention to story accuracy, $a_{0}=a_{1}=0$,
we increased the cost of engaging with fake news stories $C$ in the
optimization process (x-axis), and calculated the average rate of engagement
with true (purple) and fake (orange) stories. As attention to payoffs
increases, overall engagement slowly decreases and the difference in
engagement with true and fake stories declines. b) Similarly, consumption when
attention to payoffs is low ($\sigma=0.1$) declines as $C$ increases. c) When
attention to payoffs is moderate ($\sigma=1$) the same pattern holds for
engagement and d) consumption. e) When attention to payoffs is strong
($\sigma=10$) both engagement and f) consumption decline to zero once $C>1$.
Unless otherwise stated parameters are the same as in Figure 4 of the main
text.

## Supplementary information on experiments

Here we provided additional details and analysis of our experimental findings.

### Headline selection and engagement data

We studied 20 mainstream and 20 fake news sources, used in previous studies of
misinformation online [32]. We selected the most recent 20 unique news
headlines with engagement data available for Facebook via CrowdTangle. We
excluded stories whose headlines did not constitute news (e.g. a quiz). We
also studied the relationship between source trust and patterns of engagement
without a priori groupings of mainstream and misinormation sites (main text
Figure 3b). To do this we used trust ratings from [36].

We gathered engagement data via CrowdTangle on the most recent 20 news
headlines for each news source as available on November 22nd, 2020.This
information was gathered by generating a historical report for each news site
and gathering the 20 most recent news headlines in the database. The
engagement information recorded for each story is the number of Facebook
interactions reported by CrowdTangle at the the time. This number accounts for
likes, comments and shares of posts that contain the URL for the article
across the CrowdTangle database.

The list of headlines for each source, the headline accuracy rating, the
publication date of the story and the Facebook engagement information
recovered via CrowdTangle is available via github.

### Meta-analysis

We performed a meta-analysis on the results of linear regressions of
engagement against accuracy, for the 20 fake news and 20 mainstream sites. In
order to determine whether there was a negative correlation between engagement
and accuracy for fake news sites, we performed Fisher’s combined test, using
the p-values resulting from a single-tailed t-test from a regression of
engagement against accuracy. we used the same procedure to determine whether
there was a positive correlation between engagement and accuracy for
mainstream news sites.

To determine whether there was significant heterogeneity among fake news or
mainstream sites we calculated the between study variance $\tau^{2}_{DL}$
using the DerSimonian and Laird [34] moment-based estimate, and calculated the
proportion of total variance attributable to between study effects, $I^{2}$,
as well as the degree of homogeneity $Q_{RE}$ under a random-effects model
[35]. These results are summarized in Table S2 where we test regressions of
$\log_{10}$-engagement vs accuracy, using both unstandardadized and
standardized data, as well as regression using the rank of engagement and
accuracy for each site. In all cases we find significant negative correlation
for fake news and significant positive correlation for mainstream sites with
$p<0.01$. However we find no significant between site heterogeneity for either
$I^{2}$ or $Q_{RE}$ with $p<0.05$.

Table S2:  Meta analysis – The mean effect size $\hat{\mu}_{RE}$ for
correlations with weights from a random effect model, as well as measures
between study heterogeneity $I^{2}$ and $Q_{RE}$. Significance is indicated
with an asterisk (*).

## References

  * [1] Lazer, D. M. J. _et al._ The science of fake news.  _Science_ 359, 1094–1096 (2018).
  * [2] Pennycook, G., McPhetres, J., Zhang, Y., Lu, J. G. & Rand, D. G.  Fighting covid-19 misinformation on social media: Experimental evidence for a scalable accuracy-nudge intervention.  _Psychological Science_ 31, 770–780 (2020).
  * [3] West, J. D. & Bergstrom, C. T.  Misinformation in and about science.  _Proceedings of the National Academy of Sciences_ 118 (2021).
  * [4] Pennycook, G. & Rand, D. G.  Research note: Examining false beliefs about voter fraud in the wake of the 2020 presidential election.  _Harvard Kennedy School (HKS) Misinformation Review_ (2021).
  * [5] Guess, A., Nagler, J. & Tucker, J.  Less than you think: Prevalence and predictors of fake news dissemination on facebook.  _Science Advances_ 5 (2019).
  * [6] Guess, A. M., Nyhan, B. & Reifler, J.  Exposure to untrustworthy websites in the 2016 us election.  _Nature Human Behaviour_ 4, 472–480 (2020).
  * [7] Grinberg, N., Joseph, K., Friedland, L., Swire-Thompson, B. & Lazer, D.  Fake news on twitter during the 2016 u.s. presidential election.  _Science_ 363, 374–378 (2019).
  * [8] Pennycook, G. & Rand, D. G.  Lazy, not biased: Susceptibility to partisan fake news is better explained by lack of reasoning than by motivated reasoning.  _Cognition_ 188, 39–50 (2019).  The Cognitive Science of Political Thought.
  * [9] Bago, B., Rand, D. G. & Pennycook, G.  Fake news, fast and slow: Deliberation reduces belief in false (but not true) news headlines.  _Journal of Experimental Psychology: General_ 149, 1608–1613 (2020).
  * [10] Mosleh, M., Pennycook, G., Arechar, A. A. & Rand, D. G.  Cognitive reflection correlates with behavior on twitter.  _Nature Communications_ 12, 921 (2021).
  * [11] Pennycook, G. _et al._ Shifting attention to accuracy can reduce misinformation online.  _Nature_ 592, 590–595 (2021).
  * [12] Guess, A. M. _et al._ A digital media literacy intervention increases discernment between mainstream and false news in the united states and india.  _Proceedings of the National Academy of Sciences_ 117, 15536–15545 (2020).
  * [13] Chan, M. S., Jones, C. R., Jamieson, K. H. & Albarracín, D.  Debunking: A meta-analysis of the psychological efficacy of messages countering misinformation.  _Psychological Science_ 28, 1531–1546 (2017).  PMID: 28895452.
  * [14] Cook, J., Lewandowsky, S. & Ecker, U. K. H.  Neutralizing misinformation through inoculation: Exposing misleading argumentation techniques reduces their influence.  _PLOS ONE_ 12, 1–21 (2017).
  * [15] Gentzkow, M. A., Shapiro, J. M. & Stone, D. F.  Media bias in the marketplace: Theory.  _NBER Working Paper No. w19880_ (2014).
  * [16] Allcott, H. & Gentzkow, M.  Social media and fake news in the 2016 election.  _Journal of Economic Perspectives_ 31, 211–36 (2017).
  * [17] Bak-Coleman, J. B. _et al._ Stewardship of global collective behavior.  _Proceedings of the National Academy of Sciences_ 118 (2021).
  * [18] Stewart, A. J. _et al._ Information gerrymandering and undemocratic decisions.  _Nature_ 573, 117–121 (2019).
  * [19] Bak-Coleman, J. B. & Bergstrom, C. T.  Information gerrymandering in social networks skews collective decision-making.  _Nature_ 573, 40–41 (2019).
  * [20] Akin, E.  What you gotta know to play good in the iterated prisoner’s dilemma.  _Games_ 6, 175–190 (2015).
  * [21] Hilbe, C., Nowak, M. A. & Sigmund, K.  Evolution of extortion in iterated prisoner’s dilemma games.  _Proc Natl Acad Sci U S A_ 110, 6913–8 (2013).
  * [22] Press, W. H. & Dyson, F. J.  Iterated prisoner’s dilemma contains strategies that dominate any evolutionary opponent.  _Proc Natl Acad Sci U S A_ 109, 10409–13 (2012).
  * [23] Stewart, A. J. & Plotkin, J. B.  Extortion and cooperation in the prisoner’s dilemma.  _Proc Natl Acad Sci U S A_ 109, 10134–5 (2012).
  * [24] Stewart, A. J. & Plotkin, J. B.  From extortion to generosity, evolution in the iterated prisoner’s dilemma.  _Proc Natl Acad Sci U S A_ 110, 15348–53 (2013).
  * [25] Becks, L. & Milinski, M.  Extortion strategies resist disciplining when higher competitiveness is rewarded with extra gain.  _Nature Communications_ 10, 783 (2019).
  * [26] Bernhard, R. M. & Cushman, F.  Extortion, intuition, and the dark side of reciprocity.  _PsyArXiv_ (2021).
  * [27] Crowdtangle team (accessed 11/20).  _CrowdTangle. Facebook, Menlo Park, California, United States_ .
  * [28] Vosoughi, S., Roy, D. & Aral, S.  The spread of true and false news online.  _Science_ 359, 1146–1151 (2018).
  * [29] Fudenberg, D. & Tirole, J.  _Game Theory_ (MIT Press, Cambridge, MA, 1991).
  * [30] Friedman, J. W.  A Non-cooperative Equilibrium for Supergames.  _The Review of Economic Studies_ 38, 1–12 (1971).
  * [31] Traulsen, A., Nowak, M. A. & Pacheco, J. M.  Stochastic dynamics of invasion and fixation.  _Phys Rev E Stat Nonlin Soft Matter Phys_ 74, 011909 (2006).
  * [32] Pennycook, G. & Rand, D. G.  Fighting misinformation on social media using crowdsourced judgments of news source quality.  _Proceedings of the National Academy of Sciences_ 116, 2521–2526 (2019).
  * [33] Allen, J., Mobius, M., Rothschild, D. M. & Watts, D. J.  Research note: Examining potential bias in large-scale censored data.  _Harvard Kennedy School (HKS) Misinformation Review_ (2021).
  * [34] DerSimonian, R. & Laird, N.  Meta-analysis in clinical trials.  _Controlled Clinical Trials_ 7, 177–188 (1986).
  * [35] Higgins, J. P. T. & Thompson, S. G.  Quantifying heterogeneity in a meta-analysis.  _Statistics in Medicine_ 21, 1539–1558 (2002).
  * [36] Dias, N., Pennycook, G. & Rand, D. G.  Emphasizing publishers does not effectively reduce susceptibility to misinformation on social media.  _Harvard Kennedy School (HKS) Misinformation Review_ (2020).
  * [37] Stewart, A. J. & Plotkin, J. B.  Collapse of cooperation in evolving games.  _Proceedings of the National Academy of Sciences_ 111, 17558–17563 (2014).