[1]Niklas Carlsson

[1] Linköping University, Linköping, 581 83, Sweden

# Understanding Differences in News Article Interaction Patterns on Facebook:
Public vs. Private Sharing with Varying Bias and Reliability

Alireza Mohammadinodooshan<EMAIL_ADDRESS><EMAIL_ADDRESS>*

###### Abstract

The rapid growth of news dissemination and user engagement on social media has
raised concerns about the influence and societal impact of biased and
unreliable information. As a response to these concerns, a substantial body of
research has been dedicated to understanding how users interact with different
news. However, this research has primarily analyzed publicly shared posts.
With a significant portion of engagement taking place within Facebook’s
private sphere, it is therefore important to also consider the private posts.
In this paper, we present the first comprehensive comparison of the
interaction patterns and depth of engagement between public and private posts
of different types of news content shared on Facebook. To compare these
patterns, we gathered and analyzed two complementary datasets: the first
includes interaction data for all Facebook posts (private + public)
referencing a manually labeled collection of over 19K news articles, while the
second contains only interaction data for public posts tracked by CrowdTangle.
As part of our methodology, we introduce several carefully designed data
processing steps that address some critical aspects missed by prior works but
that (through our iterative discussions and feedback with the CrowdTangle
team) emerged as important to ensure fairness for this type of study. Our
findings highlight significant disparities in interaction patterns across
various news classes and spheres. For example, our statistical analysis
demonstrates that users engage significantly more deeply with news in the
private sphere compared to the public one, underscoring the pivotal role of
considering both the public and private spheres of Facebook in future
research. Beyond its scholarly impact, the findings of this study can benefit
a wide range of stakeholders, including Facebook content moderators,
regulators, and policymakers, contributing to a healthier online discourse.

###### keywords:

News sharing, Facebook, User engagement, Bias, Reliability, Public vs private

##  1 Introduction

Social media has become a dominant medium for news consumption and sharing,
with Facebook being the most popular and widely used platform for these
purposes [1, 2]. For example, according to a 2021 Pew Research Center survey
[2], 66% of all Americans regularly use Facebook, and almost half of them (31%
of the US population) regularly get their news via the platform. Other popular
platforms that users regularly use for news consumption include YouTube (22%
of the US population), Twitter (13%), and Instagram (11%). This significant
reach highlights the potential impact that news consumption and sharing on
Facebook (and social media in general) can have on the opinions and thoughts
of both individuals and the society [3].

Unfortunately, not all news are reliable. With an increasing amount of
misinformation being circulated on Facebook, it is crucial to understand how
users engage with news of varying reliability [4]. Improved insights and
understanding into these dynamics are expected to benefit media researchers,
journalists, content moderators, and policymakers, which choices based on
these insights, in turn, are expected to influence public opinion and the
behavior of regular users [5, 6].

Previous research has extensively examined public user engagement dynamics
with different news content and the factors affecting this engagement [7, 8,
9, 10, 11, 12]. However, with a growing privacy inclination of Facebook users
and a rise in user engagement in private spaces of Facebook [13, 14], it is
becoming increasingly important to understand these dynamics also with regards
to private sharing. Yet, the current literature predominantly focuses on
public posts, leaving gaps in our understanding of engagement differences
between these realms. Most importantly, prior research has not studied the
variations in public vs. non-public interaction with regards to (1) reliable
and unreliable news content, (2) news with different political biases, or (3)
the depth of the interactions associated with each of these types of posts and
content types. These are the primary gaps addressed in this paper.

In this paper, we examine the differences in public and private sharing of
news articles on Facebook, with an emphasis on users’ interactions with these
articles. Specifically, we investigate how news articles written with
different bias and reliability are shared and engaged with, as well as the
depth of these interactions. In this context, bias refers to a tendency for
news articles to exhibit partiality or favoritism towards particular groups or
ideas (e.g., left or right on the political spectrum), while reliability
refers to the accuracy and credibility of the information presented [15]
(e.g., fake or true). While most previous research focuses on either bias or
reliability, we consider both dimensions, as they have been found related, but
yet each has its own ability to influence news article sharing behaviors and
affect the quality and diversity of information that users encounter on social
media [16, 17]. On the other hand, by considering whether an interaction is
deep or shallow, we can identify whether there are areas (private or public)
where users engage more deeply with specific content types (e.g., fake news).
This is important since the depth of user engagement can significantly
influence the success of an article. For example, an article may have greater
success if people share and comment on the posting of the article than if they
only press Facebook’s “like” button [1].

Research questions and methodology: With the above goals in mind, we designed
our study to address the following research questions:

  1. RQ1

How do the patterns of public and private sharing of news articles on Facebook
differ for articles with varying levels of bias and reliability?

  2. RQ2

If and how does the depth of user interactions with news articles differ
within public and private sharing contexts for articles with varying levels of
bias and reliability?

We took several steps to address these previously unaddressed questions in as
controlled matter as possible. First, in contrast to most prior works that use
publisher-level labeling of news (e.g., Adelson et al. [7]; Horne et al.
[18]), we employ article-level labeling of bias and reliability. By doing so,
we capture that not all articles by a publisher have the same bias or
reliability, recognizing that the publisher (source) is only one factor [19]
in the bias/reliability. In particular, we consider the bias and reliability
of articles ($N=19,050$), meticulously selected from a substantial pool of
publishers ($N=1,121$) and 30K+ manually labeled news articles. Here, the
original labeling was provided by Ad Fontes Media [20], which rates both news
sources and individual articles based on their level of bias and reliability.
Second, through active back-and-forth discussions and feedback with the
Crowdtangle team, we developed and implemented a methodology using their
Chrome addon that allows us to obtain simultaneous interaction statistics for
a representative sample of publicly shared Facebook posts linking these
articles, as well as across all Facebook posts (including both public and
private posts) linking these articles. Finally, using this unique dataset, we
perform a comparison of interaction dynamics when users share news articles
with different levels of bias and reliability, both publicly and privately.

Empirical example findings: Our analysis reveals several key insights into how
users interact with news articles of varying bias and reliability on Facebook.
For instance, we show that users tend to engage more deeply in private
discussions than in public ones, irrespective of the news class. When
considering the news class, we highlight that users exhibit relatively higher
deep interaction levels with highly-unreliable content. Our results also show
that reliable news content has significantly lower private interaction shares
compared to the highly-reliable or even unreliable content.

Example beneficiaries: Our methodology and findings contribute to a better
understanding of the dynamics of news sharing and interaction on Facebook and
provide valuable insights for multiple stakeholders. For example, considering
the significant impact that social media interactions have on content and
platform popularity [21], our findings on how bias and reliability influence
user engagement in public vs. private spheres can aid media researchers and
journalists. These insights can inform more effective news content creation
and distribution strategies for enhanced audience engagement. For content
moderators of platforms such as Facebook, the analysis offers data-driven
guidance on where moderation efforts may be most needed to curb the spread of
problematic content. For policymakers and regulators, our findings illustrate
how user privacy settings impact news content engagement types and interaction
depth. Notably, highly unreliable news receives more private discussion
engagement share compared to the broader unreliable category. These results
underscore how platform design choices can influence selective exposure and
engagement, potentially exacerbating issues like ideological polarization and
misinformation spread.

Roadmap: Sect. 2 explains our multi-faceted research design, including news
article selection, labeling, interactions data collection, and the processing
steps taken to ensure fair comparisons between public and private engagement.
The resulting dataset is summarized in Sect. 3. Sect. 4 analyzes public vs.
private dynamics both from a high-level aggregated perspective (Sect. 4.1) and
then using a detailed statistical analysis (Sect. 4.2). In Sect. 5, we turn
our attention to differences in the depth of interactions. Sect. 6 concludes
our analysis with a correlation-based analysis between public and private
interactions. Discussion of the key findings is provided in Sect. 7. Finally,
related works (Sect. 8) and conclusions (Sect. 9) are presented.

##  2 Research Design, Data Collection and Limitations

At a high level, our research design has three parts.

  * 1.

Article selection and labeling: We first obtain the URLs to a substantial
number of news articles, which have been classified by Ad Fontes Media [20]
with regards to both their individual bias and reliability. We then decide
category thresholds and label each article into one of three bias classes and
one of four reliability classes.

  * 2.

Collection of interactive data: After careful preprocessing of the URLs, we
use the CrowdTangle browser extension to obtain interaction data for two sets
of posts linked to these URLs (1) public and (2) combined (public + private),
where the first set is a subset of the second. To address some limitations of
the API and to ensure fair comparison of the sets, we leave a four-month gap
between the latest labeled article (Nov. 1, 2022) and the primary data
collection (Mar. 2023), as well as apply some additional post-processing
(e.g., examining the actual posts), filtering (e.g., based on thresholds
determined via communication with the CrowdTangle team), and collect some
complementing data directly from CrowdTangle (e.g., to address limitations of
the extension API).

  * 3.

Data analysis: Finally, we use the final dataset (capturing the two sets) to
compare the properties of the public vs. combined (private + public) sets and
discover statistically significant differences between the interaction
patterns of public and private posts. Here, it should also be noted that the
combined set is dominated by private posts.

We next provide detailed descriptions of the initial two steps, before
presenting our analysis results (step 3) in the subsequent sections.

###  2.1 News Article Selection and Labeling

Selection of articles: There are several independent initiatives that assess
the bias and/or reliability of individual news articles and/or news sources.
Examples of such evaluation efforts include Media Bias Fact Check [22], Ad
Fontes Media, AllSides [23], and NewsGuard [24]. Among these, we opted to use
data from Ad Fontes Media for the following main reasons: (1) it has been
widely used in previous research [25, 26, 27, 12], (2) they evaluate
individual news articles (not only the news publishers), (3) each assessed
article receives a score for both bias and reliability, (4) they offer a
transparent evaluation methodology, which is published and explained in a
white paper [28], and finally (5) the dataset comprises a large set of news
contents (including news articles, podcasts and TV shows) randomly selected
from a diverse set of publishers.

We obtained all evaluated news from Ad Fontes Media as of Nov. 1, 2022. After
filtering the podcasts and TV shows, this set included 31,446 news article
URLs from 1,121 publishers. We also filtered a few URLs that (1) primarily
served as event reporting pages (e.g., https://www.nola.com/news/hurricane)
and (2) articles for which Ad Fontes Media had changed their rating
(identified through complementary documents published by Ad Fontes Media).
After excluding these articles, the dataset comprised 31,408 articles.

Labeling of articles: Each article in the Ad Fontes Media dataset is assessed
for both bias and reliability by a minimum of three human analysts,
representing a mix of right, left, and center self-reported political
perspectives. Bias scores from Ad Fontes Media range between -42 and +42,
where more negative values suggest a stronger left-ward bias, and positive
values indicate a right-leaning bias. As for reliability, scores vary from 0
to 64, with 64 representing the most-reliable news.

Figure 1: CDF of bias scores Figure 2: CDF of reliability scores

For our analysis, we categorized the bias and reliability scores into distinct
classes. While previous work mainly focuses on binary classification of news
(e.g., Left vs. Right or Fake vs. True), we opted for a more granular approach
by defining three classes of bias (Left, Right, and Center) and four classes
of reliability (Most-unreliable, Unreliable, Reliable, and Most-reliable).
Figures 1 and 2 show the Cumulative Distribution Function (CDF) of the bias
and reliability scores of the articles included in our analysis (after
additional filtering steps, explained later in this section) together with the
threshold values and class labels used to define each class.

With our labeling, anything with a bias score below -6 is called Left-biased,
and anything above 6 is called Right-biased, with the range (-6,6) capturing
the Center class. We note that Ad Fontes Media also refers to this range as
“Middle or Balanced Bias” [20]. As observed, this class accounts for most
samples in our dataset, representing 58% (11,094 out of 19,050 articles in the
final dataset). With this split, the two biased classes have similar, close to
20%, shares.

For the reliability classes, we assigned the Most-unreliable class to cover
the lowest range (0, 32). According to Ad Fontes Media’s terminology [20],
these values typically correspond to articles “which contain inaccurate and
misleading info, selective stories or have opinion and wide variation in their
reliability”. Similarly, the range (48-64), chosen for the Most-reliable
class, encompasses articles characterized by “thorough and original fact
reporting” due to Ad Fontes terminology. Finally, the threshold of 40, between
Reliable and Unreliable, matches the mid-point between these two classes as
well as where Ad Fontes make their split between “wide variation in
reliability” and “mix of fact reporting and analysis”.

###  2.2 Collection of Interaction Data

Preprocessing of URLs: Before collecting the interaction data associated with
each article (URL), we (1) replaced the link shorteners with their
corresponding URLs and (2) converted all article URLs to their canonical
forms. During the conversion process, we primarily removed all unnecessary URL
parameters (those sent after the ”?” symbol) except for the ones essential to
the canonical form (e.g., “id” parameters are sometimes part of the canonical
form). As another step in identifying the canonical forms, we replaced all the
web archive links in the Ad Fontes dataset with the original links.

Primary data collection for the two sets of posts: We used the CrowdTangle
browser extension [29] to collect the interaction data for the Facebook posts
sharing each news article of interest. In addition to interaction statistics
for the public posts that CrowdTangle (owned by Facebook) indexes and tracks
over time (their database, available for and used by researchers, contains
longitudinal interaction data for indexed posts), this extension provides
access to the life-time interaction statistics, obtained thus far, across all
posts (private + public). In this paper, we use this extension to collect
statistics about all interactions associated with these two sets of posts:
combined and public, where the combined captures all interactions of all posts
(public + private) and the public is a subset of the combined (only capturing
public posts).

To collect the data for each URL, we used the “Download” option within the
extension’s interface when browsing each article. This allowed us to easily
download a CSV file containing the interactions data for both sets. As
expected from a privacy standpoint, only the aggregated statistics are
provided for the combined set. In contrast, every post and their individual
interaction data are provided for the public set.

At a high level, the combined category aggregates the interactions (e.g.,
likes, shares, comments, etc.) of all Facebook posts referencing the article’s
URL, regardless of the post’s privacy settings. This includes interactions on
posts with limited privacy settings, such as “Friends only” or “Only Me”. In
contrast, the public category, includes all interactions with the public posts
tracked by CrowdTangle, which, due to heavy-tailed characteristics and
significant coverage, captures most interactions with public Facebook posts.
As supporting examples, all posts of all US-based public groups with 2K+
members are indexed, and in 2021 over 99% of all Facebook pages with at least
25K likes were indexed [30]. While some public posts are not indexed, we note
that their indexing does not consider any bias/reliability criteria, is
expected to capture the majority of public posts, and the majority of the
posts in the combined set are private.

Timeline and limitations: We collected the interaction data in Mar. 2023,
ensuring at least a four-month gap between the publication date of the newest
article in our dataset and our interaction data collection. This length of
interval was chosen to ensure that we capture most of the posts and
interactions for the URLs in our dataset.

As mentioned in [31] and confirmed via our communication with CrowdTangle, the
information from the CrowdTangle addon originates from two different sources:
(1) public interaction data is obtained from the CrowdTangle database, while
(2) combined interaction data is derived from Facebook’s Graph API.
Consequently, variations in data capturing times might cause some URLs to
exhibit higher public than combined interactions [31]. Furthermore, as
discussed in [31], Facebook’s aggregation data restarts the interaction count
whenever websites update their connection schemes (e.g., switching from HTTP
to HTTPS) after publishing a URL, while CrowdTangle continues accumulating
data. Although the discrepancy is not substantial (9% in our initial dataset),
we aimed to mitigate the effect of these differences where possible (described
next).

Further filtering and data processing for fairer comparisons: After consulting
with the CrowdTangle team regarding all numerical discrepancies that we
observed, we identified two additional reasons for the discrepancies. First,
as stated in [32], for privacy reasons, for the combined data (that include
private posts) “the values are intentionally not precise, but you can be
confident they accurately reflect user engagement with a URL.” Discussing the
identified discrepancy for such cases, the CrowdTangle team confirmed that
combined interaction data with counts below 100 is not reliable; however, that
data with counts above 100 provides a reliable estimate of public + private
interactions. For this reason, we removed all URLs in our dataset with
combined interactions below 100. Although this reduced our dataset to 19,505
articles, it is important to note that this group of URLs accounted for less
than 4% of public interactions in our dataset. Furthermore, we note that the
public interaction data for this group still remains reliable.

Second, the CrowdTangle team informed us that the Facebook Graph API only
considers posts with the URL attached to the post (i.e., those displaying a
preview of the URL) when counting interaction with the URL, while
CrowdTangle’s algorithm includes any post containing a linked URL, regardless
of attachment status. Taking this into account, in our final step toward
enhancing interaction data quality, we chose not to rely on the aggregated
data from the CrowdTangle extension (which sums the interactions of all posts
mentioned in the extension). Instead, we examined each post to determine
whether the related URL was attached or not, and if so, we included them in
the sum of public interactions for that URL.111The attachment status of all
the posts and URLs will be published with the camera-ready version for the
interested researchers. This resulted in a more accurate comparison between
combined and public interactions. After these processing steps, we had only
455 URLs for which public interactions exceeded combined interactions, which
were excluded from our final dataset. The primary causes of these
discrepancies were beyond the scope of our research to address (e.g., changes
in URL, publisher protocol schemes, or different data capturing times as
mentioned above).

While some of the pre- and post-processing steps described above require
significant effort, they help ensure an accurate and fair comparison between
the public and combined sets in such a way that we can provide conclusive
insights into the relative sharing patterns of public vs. private posts.

Enhancing the data with additional posts: Another limitation of the
CrowdTangle extension is that it only allows us to retrieve data for up to 500
posts per URL. To address this limitation, for the URLs with more than 500
posts in the public interaction category, we obtained additional data directly
from CrowdTangle. However, these URLs constitute only 42 instances in our
dataset and will have no significant impact on the reproducibility of our
statistical analysis.

##  3 Dataset

High-level summary: After applying the aforementioned filtering steps, we have
in total 19,050 articles from 1,121 news outlets remaining in our dataset.
These articles have been shared in 253,350 posts, which combined are
responsible for 225,282,745 interactions (out of which 70,428,316 are public).
Table 1 summarizes the dataset.

Bias data: Table 2 shows the number of articles and the total combined and
public interactions for each bias and reliability class. With the selected
bias thresholds, our dataset includes 4,516 Left-biased articles and 3,440
Right-biased articles, with the remaining 11,094 articles falling in the
Center category. These articles are in turn responsible for 79,038,246 (Left),
56,160,101 (Right), and 90,084,398 (Center) interactions. While the Center
category includes 58% of the articles, it is interesting to note that it is
responsible for a significantly smaller fraction of the total interactions
(40%). Instead, the Left-biased and Right-biased articles see relatively
higher interaction rates (studied in the next section). For example, the Left-
biased articles account for only 24% of the articles but 35% of the
interactions, and the Right-biased articles are responsible for only 18% of
the articles but 25% of the interactions.

Table 1: Summary of the final dataset Type | Articles | Publishers | Posts | Combined interactions | Public interactions
---|---|---|---|---|---
Number | 19,050 | 1,121 | 253,350 | 225,282,745 | 70,428,316
Table 2: Statistics based on Bias and Reliability | Class | Articles no | Combined interactions | Public interactions
---|---|---|---|---
Bias | Left | 4,516 | 79,038,246 | 20,307,107
| Center | 11,094 | 90,084,398 | 32,109,924
| Right | 3,440 | 56,160,101 | 18,011,285
Reliability | Most-unreliable | 3,520 | 45,299,475 | 13,206,643
| Unreliable | 3,795 | 46,712,927 | 14,199,322
| Reliable | 9,308 | 89,596,137 | 30,639,973
| Most-reliable | 2,427 | 43,674,206 | 12,382,378

Reliability data: For the reliability classes as shown in Table 2, the
relative differences are smaller. Here, the Reliable articles make up the
largest share (9,308 articles and 89,596,137 interactions), followed by
Unreliable (3,795 articles and 46,712,927 interactions), Most-unreliable
(3,520 articles and 45,299,475 interactions), and Most-reliable (2,427
articles and 43,674,206 interactions).

##  4 Public vs. Private Interactions

To better understand the differences in interactions with public vs. private
posts, we first compare the interaction rates with all public posts and with
all combined (public + private) posts. We note that this implicit comparison
approach provides more stable and reliable statistical comparisons than trying
to use only the estimated number of private interactions. The relative
strength of this approach is further compounded by the private interactions
typically dominating the combined set.

###  4.1 Aggregated (Macro) Analysis of Interactions

Figure 3: Normalized number of combined and public interactions per article
for each bias and reliability class.

Figure 3 presents the number of combined and public interactions per article
for each class. Here, two metrics are used:

  * •

Average combined interactions (left bars for each class) represents the
average number of combined (private+public) interactions per article in each
class. More specifically, considering $I_{c,i}^{\text{comb}}$ as the combined
interaction for the $i^{th}$ article in class $c$, the average combined
interactions for this class ($\bar{I}_{c}^{\text{comb}}$) is calculated as:

$\bar{I}_{c}^{\text{comb}}=\frac{\sum_{i=1}^{N_{c}}I_{c,i}^{\text{comb}}}{N_{c}},$
(1)

where the numerator ($\sum_{i=1}^{N_{c}}I_{c,i}^{\text{comb}}$) corresponds to
the “Combined interactions” column in Table 2 and the denominator (i.e., the
total number of articles $N_{c}$ for class $c$) is found in the “Articles no”
column of Table 2. For example, for the Center class in the Bias category, the
average combined interactions per article is computed by dividing the total
interactions (90,084,398) by the articles count (11,094), yielding
$\bar{I}_{\text{Center}}^{\text{comb}}$ to be 8,120 combined interactions per
article (gray bar in Figure 3).

  * •

Average public interactions per article (right bars) is defined in a similar
manner but considering only the public interactions. More specifically,
considering $I_{c,i}^{\text{public}}$ as the public interactions for the
$i^{th}$ article in a class $c$, the average public interactions for this
class ($\bar{I}_{c}^{\text{public}}$) is calculated as follows:

$\bar{I}_{c}^{\text{public}}=\frac{\sum_{i=1}^{N_{c}}I_{c,i}^{\text{public}}}{N_{c}},$
(2)

where the numerator (i.e., $\sum_{i=1}^{N_{c}}I_{c,i}^{\text{public}}$)
corresponds to the “Public Interactions” column in Table 2. Now, taking the
Center class as an example again, the average public interactions per article
is calculated by dividing the total public interactions (32,109,924) by the
article count (11,094), resulting in an
$\bar{I}_{\text{Center}}^{\text{public}}$ of 2,894 interactions per article
(striped gray bar in Figure 3).

As seen in the figure, there are some very clear and interesting differences
in the trends observed for the public vs. the combined statistics. First, for
all three bias classes and for all four reliability classes, the interaction
rates are significantly lower for the public posts than for the combined set
(which for all classes are dominated by private posts). These aggregate rate
differences suggest that users are more likely to interact with private posts,
regardless of bias and reliability class, and underscores the importance of
considering the total (combined) interactions, not only the (typically
studied) public posts. We next look closer at the relative interaction levels
of each class and the impact of using the public vs. combined sets for such
interaction comparisons.

Bias comparisons: Second, looking at the relative differences among the bias
classes, the Left bias class has the highest average combined interactions per
article (17,501), followed by the Right (16,325) and Center (8,120) classes.
For public interactions; however, the Right bias class has the highest average
public interactions per article (5,236), with the Left (4,497) and Center
(2,894) classes trailing behind. This shows that biased articles see even
greater relative interaction rates when posts are private than public.

Reliability comparisons: Third, for the reliability classes, the Most-reliable
class has the highest average combined and public interactions per article
(17,995 and 5,101). The Unreliable and Reliable classes have relatively
similar average public interactions per article (3,741 and 3,291,
respectively). Moreover, the Most-unreliable class has a higher average
combined interactions per article (12,869) and public interactions per article
(3,751). It is interesting to see that the two extreme classes (i.e., the
Most-reliable and Most-unreliable) have the highest average interactions per
article in both combined and public contexts, implying that users are more
likely to engage with articles on both ends of the reliability spectrum,
regardless if the posts are public or non-public, although the level of
interactions is different.

Key observation: Compared to the Center group, biased groups receive higher
levels of aggregated engagement per article. In terms of reliability, the
Reliable news reaches the lowest engagement levels per article.

Aggregated public interaction share: Motivated by most prior works only
studying the public posts tracked by CrowdTangle, we next look closer at how
big fraction of the total interactions this set captures and compare the
relative differences in the public interaction share of the different
categories. For this analysis, we calculate the (aggregate) public interaction
share, defined as the ratio of public interactions to the combined
interactions for each class $c$ as
$\bar{I}_{c}^{\text{public}}/\bar{I}_{c}^{\text{comb}}$.

Figure 4: Aggregated public interactions shares

Figure 4 summarizes these ratios. We note that the Center class has the
highest aggregated public interaction share, and that among the biased groups,
there is a clear gap between the left and right parties, as the Right class
has a significantly (25% extra) higher public interaction share (lower private
interaction share). When it comes to the reliability classes, the relative gap
between the classes is smaller. However, it should be noted that the two
extreme classes (Most-reliable and Most-unreliable) have the lowest (and
almost similar) aggregated public interaction shares.

Key observation: Compared to right-biased articles, left-biased articles
receive higher private interactions share. In terms of reliability, extremely
reliable/unreliable news articles receive higher levels of private
interactions share.

###  4.2 Granular Statistical (Micro) Analysis of Interactions

Having presented aggregated analysis and insights, we next present a more
detailed statistical comparison between different classes in which we give
equal consideration to every article in a class, regardless of its total
interaction count. For this analysis, we utilize the distributions of the per-
article level interactions for each class $c$:
$\mathcal{D}^{\text{comb}}_{c}=\\{I_{c,i}^{\text{comb}}\\}$ and
$\mathcal{D}^{\text{public}}_{c}=\\{I_{c,i}^{\text{public}}\\}$, where
$I_{c,i}^{\text{comb}}$ is the total number of interactions across all posts
associated with the $i^{th}$ article of class $c$ ($1\leq i\leq N_{c}$) and
$I_{c,i}^{\text{public}}$ is the corresponding number of interactions with
public posts.

The next two subsections compare the distributions of the total (combined)
interactions (Sect. 4.2.1) and study the distributions of the relative public
interaction shares (Sect. 4.2.2), respectively. In both subsections, the
distributions are compared using multiple statistical tests; statistical
significance is reported if a p-value is below 0.001, and supporting p-values
are reported for the main findings.

####  4.2.1 Distribution of Combined Interactions

Comparison of Bias Classes: Figure 5 shows the CCDFs of both the set of
combined interactions ($\mathcal{D}^{\text{comb}}_{c}$) and the set of public
interactions ($\mathcal{D}^{\text{public}}_{c}$) for each bias class $c$ on a
log-log scale. Accompanying the CCDF is a boxplot, highlighting key
statistical percentiles for the distribution of the combined interactions
($\mathcal{D}^{\text{comb}}_{c}$). Specifically, the 10th percentile (bottom
marker), 25th percentile (bottom of box), median (middle marker), 75th
percentile (top of box), 90th percentile (top marker), and the mean (i.e.,
$\bar{I}_{c}^{\text{comb}}$) (circle). To allow higher resolution, outliers
are not shown in the boxplot.

Figure 5: Empirical complementary cumulative distribution function (CCDF) of
both combined and public interactions, alongside an inset boxplot detailing
the distribution of the combined interactions for different bias classes.
While the CCDF underscores the tail characteristics of the distributions, the
boxplot provides a granular statistical summary of the combined interactions,
including medians, quartiles, and means.

Upon examining the figure, we can make several observations. First, as
articles with fewer than 100 combined interactions were excluded, the minimum
value for $\mathcal{D}^{\text{comb}}_{c}$ starts at 100. Second, the shape of
the of the CCDFs when plotted on log-log scale underscores the “heavy-tailed”
nature of the interactions ($\mathcal{D}^{\text{comb}}_{c}$ and
$\mathcal{D}^{\text{public}}_{c}$) of all bias classes $c$. This suggests that
a small subset of the articles are responsible for a most of the interactions.
Third, there are discernible variations in the distributions. For example,
articles classified as biased (both Left and Right) consistently see higher
interaction levels (i.e., right-shifted curves compared to the Center class),
echoing the trends we observed in the aggregated analysis. To better
understand these disparities and their implications, we next provide a more
rigorous statistical analysis.

Statistical Tests for Bias-related Differences: To evaluate the differences in
distributions of the set of combined interactions
$\mathcal{D}^{\text{comb.}}_{c}$ across all bias classes $c$, we employed a
multi-step statistical testing approach:

  1. •

Overall Differences: We first used the Kruskal-Wallis test to identify overall
differences between the distributions between the bias classes. This test
revealed significant disparities among the classes, with a p-value of
$1.66\cdot 10^{-65}$.

  2. •

Pairwise Comparisons: Second, we used the Dunn test to identify specific pairs
of classes that exhibited differences. This test confirmed that the two biased
classes Left and Right are different than the Center class, but the difference
between the two biased classes themselves was not significant. Here, the most
significant difference were observed between the Left and Center class (i.e.,
$\mathcal{D}^{\text{comb}}_{\text{Left}}$ and
$\mathcal{D}^{\text{comb}}_{\text{Center}}$), registering a p-value of
$2.06\cdot 10^{-55}$.

  3. •

Median Comparisons: Third, we used the Mann-Whitney U test to compare the
medians of the three classes: Left (1,440), Center (842), and Right (1,286).
Also, here, we observed statistically significant pairwise differences when
comparing each of the two biased classes (Left and Right) with the Center
class, but not between each other. Here, the two significant cases obtained
p-values of $8.4\cdot 10^{-57}$ and $9.1\cdot 10^{-28}$, respectively.

  4. •

Comparing Means: Finally, due to the violations of assumptions intrinsic to a
t-test (namely, normality and homogeneity of variances), we instead use
bootstrapping with 100K iterations and a 99% confidence interval to compare
the means. Again, the pairwise differences in the means between Left vs.
Center and between Right vs. Center are statistically significant, but not
between the two biased classes themselves (i.e., Left vs. Right).

Key observation: Statistical analysis further strengthens the observation that
biased news consistently garners higher combined engagement levels per article
than center-aligned news. Interestingly, there is no statistically significant
difference between the Left and Right class.

Comparison of Reliability Classes: Figure 6 shows the CCDFs for the different
reliability classes, broken down for both combined and public interactions,
accompanied by a boxplot offering a clearer perspective on the percentile
values for the combined interactions. The CCDFs again demonstrate heavy-tailed
distribution characteristic, and the relative shifts of the distributions are
consistent with the aggregate (average values) previously discussed (and seen
in Figure 3), with the Most-reliable class typically getting the most
interactions, followed by the unreliable classes, and subsequently by the
Reliable class. While comparing the tail parts is straightforward, comparing
the whole distributions requires some care. For example, here, the core of the
boxplot does not distinctly set apart the Most-reliable from the unreliable
classes. To derive robust conclusions, we therefore again performed a sequence
of statistical tests on the combined interactions distributions.

Figure 6: CCDF of both combined and public interactions, alongside an inset
boxplot detailing the distribution of the combined interactions for different
reliability classes. While the CCDF underscores the tail characteristics of
the distributions, the boxplot provides a granular statistical summary of the
combined interactions, including medians, quartiles, and means.

First, the Kruskal-Wallis test was applied to discern differences among the
reliability classes. While we observed statistical differences (p-value of
$7.9\cdot 10^{-20}$), the magnitude of distinction among reliability classes
is more nuanced than in the bias classes (which had a p-value of $1.66\cdot
10^{-65}$). Further analysis employing the Dunn’s test confirmed that the
Reliable class statistically diverges from the other three. The least
significant p-value here, $7.63\cdot 10^{-8}$, is attributed to the comparison
between the Reliable and Unreliable classes. The most significant, on the
other hand, emerges from the Reliable versus Most-reliable comparison with the
p-value of $7.05\cdot 10^{-15}$. Except for these distributions, none of the
other pairwise distribution comparisons resulted in statistically significant
differences. Third, the Mann-Whitney U test reinforced our findings,
particularly highlighting the statistical significance when comparing the
medians of the Reliable class (median of 891) with the others: Most-unreliable
(1,162), Unreliable (1,104), and Most-reliable (1,218). While all three
pairwise comparisons are significant, their p-values range from $8.96\cdot
10^{-9}$ (Reliable vs. Unreliable) to $8.4\cdot 10^{-16}$ (Reliable vs. Most-
reliable). Finally, we apply bootstrapping to compare the means. While only
the Reliable and Most-reliable classes comparison is significant at the 99%
confidence level, we note that all three comparisons against the Reliable
class are significant at the 95% confidence level.

Key observation: Comparing to the other reliability classes, the Reliable
articles demonstrate statistically significant lower interaction values,
especially when compared to the Most-reliable articles.

####  4.2.2 Distribution of Public Interaction Shares

The preceding analysis centered on the distribution of combined interactions.
Like the aggregated analysis, we next shift our focus to the proportion of
interactions that are public. Specifically, for each class, we analyze the
distribution of public interaction shares at the granularity of individual
articles. For this analysis, we define the set of public interaction ratios
(PIR) of class $c$ as
$\mathcal{D}^{\text{PIR}}_{c}=\left\\{\frac{I_{c,i}^{\text{public}}}{I_{c,i}^{\text{comb}}}\right\\}$,
where $1\leq i\leq N_{c}$ is the index of articles included in class $c$.

Figure 7: Distribution of public interaction ratios

Figure 7 presents the public interaction ratios using box-plots broken down
per bias and reliability category. Here, we again show the 10th percentile,
25th percentile, median, 75th percentile, and 90th percentile, as well as the
average. Like for the aggregate analysis, we observe some notable differences,
which we can now support using statistical analysis. First, the Kruskal-Wallis
test shows that there are statistically significant differences between both
the biases classes (p-values of $2.68\cdot 10^{-22}$) and reliability classes
($8.74\cdot 10^{-33}$).

Motivated by the higher significance (smaller p-value) for the reliability
classes, we consider these differences first. Using the Dunn test, we find
statistically significant differences for all pairwise distribution
comparisons except between the Most-unreliable and Most-reliable classes.
Similarly, the Mann-Whitney U test shows that all pairwise relative
differences in the medians (observed in Figure 7), except for between the
Most-unreliable and Most-reliable classes, are significant (with the largest
p-value among the these pairwise cases being $2.82\cdot 10^{-14}$). Finally,
bootstrapping confirms that all pairwise differences in the means observed in
the figure are statistically significant, except for the case of the Most-
unreliable vs. the Most-reliable class. It is also worth noting that our
observation that the two extreme classes have lower public interaction ratios
(higher private interaction ratios) aligns with our findings from the
aggregated analysis.

Key observation: The two extreme classes, Most-reliable and Most-unreliable,
exhibit similar private interaction ratios that are statistically higher
compared to the other classes.

We now shift our attention to the public interaction ratios across the bias
classes, where we make some interesting observations regarding the classes’
relative order. Specifically, we now observe the first difference in the
relative order of the classes when comparing the average (and median) public
shares calculated on a per-article basis (Figure 7) with the aggregated
results (Figure 4). More specifically, comparing with the aggregate results,
the Right class has now switched ranking with the Left class to become the
class with the noticeably smallest public interaction share. Furthermore,
comparing the distributions seen for the Right and Left class is statistically
significant: Dunn test (p-value of $3.25\times 10^{-14}$), Mann-Whitney U test
(p-value of $6.16\times 10^{-17}$) and bootstrap results being significant at
99% level.

While these results may appear surprising at first, they are due to some
interesting differences in the distributions. First, note that the aggregate
analysis only calculates the overall ratio
$\frac{\sum_{i=1}^{N_{c}}I_{c,i}^{\text{public}}}{\sum_{i=1}^{N_{c}}I_{c,i}^{\text{comb}}}$
compared to the granular analysis, where the average (for example) is
calculated as
$\frac{1}{N_{c}}\sum_{i=1}^{N_{c}}\frac{I_{c,i}^{\text{public}}}{I_{c,i}^{\text{comb}}}$.
Second, taking a closer look at the distributions, $\mathcal{D}_{Left}^{PIR}$
and $\mathcal{D}_{Right}^{PIR}$ differ substantially. Part of this can be
observed in Figure 5, where we can see that Left class demonstrates notable
disparities between public and combined interactions in the tail, while the
Right class exhibits relatively larger differences in the head of the
distribution. This suggests that articles with higher combined number of
interactions in the Right class may exhibit higher normalized public
interaction shares, and articles with few combined interactions may exhibit
relatively smaller public shares (compared to the Left class).

To confirm this conjecture and substantiate our claim that the above
differences are due to aggregated statistics being more affected by larger
public interaction shares associated with the tail, we compared the public
interaction ratios for articles associated with the tail and head of the
distributions of the combined number of interactions. For this analysis, we
split the total sets ($\mathcal{D}^{PIR}_{Left}$ and
$\mathcal{D}^{PIR}_{Right}$) into two to five equally sized subsets (based on
the total combined number of interactions $I_{c,i}^{comb}$ associated with
each entry) and then compared the medians and averages of the public
interaction ratios seen in the first and last bucket of the Right class with
the corresponding values seen for the Left class. In all cases, the Left
dominates the Right for the head and is dominated by the Right for the tail.
For example, with three equal-sized buckets, the two classes have median
ratios of 0.34 vs. 0.13 for the first bucket (head) and 0.39 vs. 0.46 for the
last bucket (tail). Furthermore, we observed statistically significant
differences (99% confidence level) as per the above conjecture with both the
Mann-Whitney U test (medians) and Bootstrap tests (averages), validating these
differences. This confirms that the articles in the Right class with the
highest interactions have elevated normalized public interaction ratios, and
those in the Left class with the lowest interactions have similarly elevated
ratios.

Key observation: In terms of bias, the Right class sees the smallest public
interaction share among the least popular content to interact with, while the
Left class sees relatively less overall public interaction share (due to its
most popular articles to interact with not seeing as big public interaction
share).

##  5 Deep vs. Shallow Interactions

In this section, we investigate the prevalence of deep vs. shallow
interactions and explore their variations among different reliability and bias
classes. For this analysis, we categorize all (emoji-based) reactions (i.e.,
likes, loves, sads, etc.) as shallow interactions while comments and shares
are considered as deep interactions. This distinction, supported by the
previous works [8, 33], is motivated by the idea that comments and shares
typically involve more cognitive effort and engagement from users, as they
require formulating thoughts, opinions, or responses in the form of comments
or actively endorsing and disseminating the content by sharing. In contrast,
reacting with an emoji is a more passive and less cognitively demanding form
of interaction.

To compare the depth of interaction seen for subsets of public and private
posts, we define the following two per-article metrics:

  * •

Deep Interactions Ratio for Combined (DIRC): For each article, this ratio
measures the proportion of the combined interactions that are classified as
deep interactions.

  * •

Deep Interactions Ratio for Public (DIRP): For each article, this ratio
measures the proportion of the public interactions that are classified as deep
interactions.

For example, consider an article with 1,000 total interactions, 500 of which
are deep interactions, resulting in a DIRC of 0.5. Now, if 400 of these
interactions are public, and 100 of them are deep interactions, the DIRP is
0.25 for the same article. We next analyze the distribution of DIRC and DIRP
values for all articles in each class.

Comparisons of bias classes: We first compare the deep interaction patterns of
the bias classes. Figure 8 shows the distributions of DIRC and DIRP for the
different classes. As a baseline, we also include the distributions for the
general population (rightmost box pair), considering all samples.

Figure 8: Deep interaction ratios (DIRC and DIRP) for different bias classes
and a baseline.

First, referring to the figure, for each bias class, we observe substantial
difference between the distributions of DIRC and DIRP. For example, for every
percentile and for the means, the DRIC distributions (combined) have
significantly larger values than the DIRP distributions (public). This again
highlights the difference between the public and combined interactions and the
importance of considering both the private and public spheres in future
research. This observation leads us to the following key insight.

Key observation: Irrespective of the news class, users tend to engage more
deeply in private discussions compared to public ones.

Now, and perhaps even more importantly, to better understand to what extent
the relative level of interaction seen for different bias classes depends on
which dataset was used we apply the same statistical testing methodology as
discussed in previous sections for the two sets independently. First, the
Kruskal-Wallis test provides evidence that the distributions differ
significantly among the bias classes, regardless if using of the DIRC and
DIRP, with p-values of $7.14\cdot 10^{-23}$ and $2.08\cdot 10^{-9}$,
respectively. Comparing the p-values, we note that the differences appear more
significant using the combined set (DIRC) than using only the public data
(DIRP). Second, consistent with visual comparison in the figure, the post-hoc
Dunn test only supports the statistically significance differences between the
Right group (which has the biggest fraction of deep interactions regardless of
using the combined or pubic sets) and the other two groups but not among the
Center and Left group. Finally, the Mann-Whitney U test and bootstrapping
support the same pattern for the median and means, with the only significant
differences again being between the Right class and the other two bias
classes. For the bias classes, the main results regarding the relative depth
of the interactions among the classes are, therefore, consistent regardless of
the dataset.

Figure 9: CDFs of DIRC (combined) and DIRP (public) for different bias
classes.

To illustrate the tendency of the posts with Right-biased articles seeing a
higher deep interaction share, regardless of whether publicly shared or not,
we include the CDFs of DIRC (combined) and DIRP (public) for all three biases
classes in Figure 9. We note that in both cases, the Right class exhibits a
noticeable shift to the right, supporting the following insight.

Key observation: Facebook users tend to engage more deeply with content from
the Right bias class in both public and combined contexts.

Comparison of reliability classes: We now shift our focus to the reliability
classes and their deep interaction ratios. Figure 10 summarizes these results.
While we again observe deeper interactions for the combined set (suggesting
shallower interactions with public posts), the relative differences among the
reliability classes themselves (given a dataset) are less apparent compared to
those observed among the bias classes. We also observe some smaller
differences in which relative differences are significant, highlighting the
value of also considering the combined data (not only the public data).

Figure 10: Deep interaction ratios (DIRC and DIRP) for different reliability
classes Figure 11: CDFs of DIRC (combined) and DIRP (public) for different
reliability classes.

First, starting with the combined dataset (DIRC), we observe presence of
significant differences among the distributions (Kruskal-Wallis test), with
the post-hoc Dunn test (distribution) and Mann-Witney U test (median)
revealing that only the Most-unreliable class differs statistically from the
others. This observation is further visually supported by the CDF plot in
Figure 11, which clearly shows a shift to the right in the DIRC distribution
of the Most-unreliable class.

Switching our focus to the DIRP distributions, containing only public
interactions, we interestingly observe a slightly different pattern, as we,
for that dataset, also observe statistical differences between some additional
classes. First, as suggested by the DIRP CDFs in Figure 11, both extreme
classes (Most-unreliable and Most-reliable) are statistically different from
the other two classes (Kruskal-Wallis test followed by Mann-Whitney test,
followed by the post-hoc Dunn test). Second, no statistical significance was
observed between the Most-reliable and Most-unreliable classes themselves or
between the Reliable and Unreliable classes. We base the following insight
based on our observations across the two datasets.

Key observation: Users exhibit higher deep interaction levels with content
from the Most-unreliable class.

##  6 Correlation-based Analysis

Given the significant differences observed in interaction rates and deep
interaction levels between public and private posts, we aim to determine if
these variations stem from specific articles within each bias/reliability
class that receive more or fewer private interactions or if certain articles
consistently exhibit higher (deep) interaction rates, regardless of public or
private sharing. To investigate this, we next examine the correlations between
public post statistics and their combined counterparts.

Public vs. combined interactions: Table 3 summarizes the correlation
statistics (between combined and public interactions) observed for each class
using two different correlation metrics: Spearman and Kendall. Given that the
normality assumption required for the Pearson correlation is not satisfied for
the different classes, we opted for non-parametric correlation metrics. These
metrics are better suited to capture monotonic relationships, account for
outliers, and take into consideration the ranks of the data points.

Table 3: Correlation between public and combined interactions. (Bold/italics indicate highest/lowest correlation.) | Class | Spearman | Kendall
---|---|---|---
Bias | Left | 0.82 | 0.66
| Center | 0.74 | 0.59
| Right | 0.74 | 0.58
Reliability | Most-unreliable | 0.73 | 0.58
| Unreliable | 0.79 | 0.63
| Reliable | 0.76 | 0.61
| Most-reliable | 0.77 | 0.62

While different trends are observed for reliability classes, a consistent
pattern emerges for the bias classes across both correlation metrics. The Left
class exhibits the strongest correlations, followed by the Center class, and
finally, the Right class with the weakest correlations. This suggests that
articles with a left-leaning bias tend to have more similar sharing and
interaction patterns in both public and private contexts, indicating higher
similarity in engagement across different articles. In contrast, articles with
a right-leaning bias show more variability in their public and private
interactions, which may reflect a more diverse range of opinions or reactions
to the content among readers. These observations are consistent with our
observations in Section 4.2.2, which showed that the Right class saw much more
diverse public interaction shares depending on the level of combined
interactions of the article (see our bucket-based analysis).

Table 4: Correlations for deep interaction shares: public vs combined. (Bold/italics indicate highest/lowest correlation.) | Class | Spearman | Kendall
---|---|---|---
Bias | Left | 0.64 | 0.48
Center | 0.62 | 0.47
Right | 0.51 | 0.39
Reliability | Most-unreliable | 0.55 | 0.42
Unreliable | 0.61 | 0.47
Reliable | 0.64 | 0.49
Most-reliable | 0.57 | 0.42

Deep interaction rates: We next consider the correlation between the deep
interactions ratio for combined (DIRC) and public (DIRP) posts. Table 4
summarizes these results.

First, when comparing the correlations for deep interaction ratios across the
bias classes, we observe that the Left and Center bias classes show higher
correlations across both metrics, while the Right class exhibits weaker
relationships. This pattern suggests that deep interactions, or more in-depth
engagements with content, are more consistent between public and private
contexts for left-leaning and centrist articles. Second, in terms of
reliability, the correlations are relatively similar across classes, with the
Reliable class showing the strongest relationships and the Most-unreliable
class having the weakest. These findings indicate that the consistency of deep
interactions across public and private contexts is less affected by articles’
perceived reliability than their political bias.

##  7 Discussion

Our research provides a comprehensive exploration of news interactions on
Facebook, with a specific focus on discerning the nuances between public and
combined interactions across diverse news classes. We developed a robust
comparison methodology, crucial for future studies in this area. Our findings,
presented in several key facets, provide valuable insights along several
dimensions (discussed next).

Reliability and Bias: Several discoveries emerged when examining the
reliability of news articles. Surprisingly, articles classified as Reliable
garnered lower levels of engagement compared to those classified as Most-
reliable or even various Unreliable categories. Furthermore, our investigation
into the impact of biases unveiled that biased news, regardless of whether it
leans Left or Right, consistently received higher levels of user interaction
than more neutral (Center) content.

Public vs. Private: One of the central aspects of our study concerned the
comparisons of the interaction patterns within the public vs. private spheres.
What became evident was that articles classified as Most-reliable or Most-
unreliable, representing the extremities of the reliability spectrum,
attracted significantly higher proportions of engagement within private spaces
compared to their public counterparts. Intriguingly, no statistically
significant differences were observed between these two extreme groups here.
This observation holds substantial implications for content moderators,
policymakers, and researchers, prompting us to ponder whether an exclusive
focus on the public facet of Facebook provides a comprehensive understanding
of user interactions, particularly concerning news content classified as Most-
unreliable.

Interaction Depth: Our research extended beyond interaction frequency to
explore the depth of interactions, revealing noteworthy variations between the
public and private spheres. Notably, statistically significant differences in
interaction depth were observed, reinforcing the importance of considering
both spheres in research endeavors. For example, the extremities of the
reliability spectrum (Most-reliable and Most-unreliable) exhibited a tendency
to elicit deeper interactions from users within the public sphere. However,
this narrative shifted when examining combined interactions, indicating that
only the Most-unreliable class triggered statistically significant deeper
engagement from Facebook users. This underscores the intricate interplay
between reliability, bias, and interaction depth, urging researchers to
explore these dynamics beyond the confines of public engagement.

Ethical Considerations: It is essential to acknowledge the ethical dimension
of our research. While our findings offer invaluable insights, they also hold
the potential for misuse. To mitigate this risk, we undertook several
precautions, such as selecting a diverse set of articles and news sources and
maintaining objectivity in our analysis. We adhered to relevant privacy
regulations, ensuring that no personally identifiable information was
collected or disclosed. Transparency is of utmost importance to us, and we
will provide all codes used in this paper, along with the list of news
articles, in a camera-ready version. Although we cannot publish raw engagement
data due to CrowdTangle terms of service, we will furnish attachment status
information for URLs and posts through a lookup table with hashed keys. This
approach enables responsible utilization of our research findings while
minimizing the likelihood of negative consequences.

Beneficiaries: Our findings hold value for a diverse range of beneficiaries.
Researchers, journalists, and policymakers can gain insights into news
engagement patterns, aiding in informed decision-making. Social media content
moderators can benefit by understanding where to focus their efforts.
Moreover, our insights have relevance for platforms and media organizations,
guiding them in shaping content strategies for more effective audience
engagement.

In conclusion, our research sheds light on complex engagement dynamics on
social media and emphasizes the responsible use of these findings for informed
decision-making.

##  8 Related Works

This research most closely relates to studies exploring user engagement
dynamics with various social media content. Our main contribution lies in the
novel analysis of interaction disparities between public and private news
article sharing on Facebook, particularly concerning articles with varying
bias and reliability. Examining interactions depth provides another level of
depth to our analysis and findings.

There exist studies that partially touch upon some dimensions highlighted in
this paper. However, as discussed in the introduction, the vast majority of
these works, particularly pertaining to Facebook, predominantly concentrate on
the platform’s public sphere, neglecting to provide a comparative analysis
with the private or combined sphere. To contextualize our study within the
context of the existing literature and to highlight the novelty of our
contribution, we next describe the most closely related works.

One line of this prior research investigates user engagement with different
classes of news, particularly regarding bias and reliability. For example,
Edelson et al. [7], while focusing on the public sphere, conducted a large-
scale study on user engagement with 7.5 million posts from 2551 publishers
(Facebook pages) across both bias and reliability dimensions. Their results
show that individual posts from non-misinformation news outlets tend to
attract lower median engagement than misinformation. Weld et al. [34] carried
out a similar investigation on Reddit rather than on Facebook. When
considering the reliability parameter, Vosoughi et al. [35] analyzed a dataset
of approximately 126,000 stories tweeted by over 3 million people more than
4.5 million times, finding that false news stories reached more people than
true news and diffused significantly farther, faster, deeper, and more broadly
than the truth across various categories of information. Notably, during the
critical period of the COVID-19 pandemic and by doing a cross-national study,
Altay et al. [36] delineated the surge in online news consumption, with
credible news outlets witnessing a significant boost. Samory et al. [37]
characterized the social media news sphere through user co-sharing practices
by focusing on 639 news sources, both credible and questionable, and divided
them into 4 clusters, and characterizing them according to the audience that
shares their articles on Twitter and how the stylometric features used by each
cluster is successful in getting users engagement. Lamot et al. [38] examined
how news headlines are remediated on Facebook and how this affects user
engagement. Finally, Boukes et al. [39] compared user-content interactivity
and audience diversity across news and satire, finding differences in online
engagement between satire, regular news, and partisan news. Considering the
bias dimension, on the other hand, Wischnewski et al. [40] discovered that
users are more inclined to share hyperpartisan news articles that coincide
with their own political views. The bias towards the right party in sharing
and engagement with news is another thesis that has been studied by previous
works on both Twitter [41] and Facebook [42].

Other dimensions of user engagement have also been considered. For example,
Aldous et al. [8] investigated the effects of topic and emotional factors on
posting across five social media platforms (Facebook, Instagram, Twitter,
YouTube, and Reddit). They found that user engagement is strongly influenced
by the content’s topic, with certain topics being more engaging on a
particular platform. By controlling the effect of the author (considering
articles by the same author) over a one-year period, Maier et al. [43], among
other things, found that highly emotional topics get a higher level of
Facebook engagement. A large body of research has also focused on studying the
role of sad and anger in social media news sharing and engagement (e.g., [44,
45]).

Finally, we note that profiling of the relation between user engagement
dynamics and different content itself has paved the way for new applications.
Examples include techniques and models that rely (partly) on user engagement
dynamics (as features) for the detection of different content types. Such
applications may benefit social media content moderators [46, 47]. Others have
profiled the temporal dynamics of user engagement with different classes of
news [12, 48].

In summary, while previous research has partly explored some dimensions of
user engagement with news articles, our study is the first to directly compare
the dynamics of public and private sharing of news articles on Facebook,
specifically examining the interactions and depth of engagement with articles
of different levels of bias and reliability.

##  9 Conclusions

In this paper, we presented the first investigation of the differences in
interaction patterns between public and private sharing of news articles on
Facebook, focusing on articles with varying bias and reliability, as well as
the depth of interactions. By analyzing a dataset of over 19K news articles
(with more than 225M interactions on Facebook), employing the CrowdTangle
browser addon for data collection, and carefully processing this data, we
provided a comprehensive analysis of user interaction with news content on
Facebook.

Our findings reveal significant differences in interaction patterns between
public and private sharing across different classes of news. Notably, users
tend to engage more deeply in private discussions compared to public ones,
irrespective of the bias or reliability class of the news. We also found that
Facebook users engage more deeply with the least reliable news and the right-
biased news, both classes exhibiting higher deep interaction ratio levels.
While this tendency may suggest that the content from these classes may be
more polarizing or controversial, leading to deeper discussions and debates,
further research (e.g., by interviewing users) is needed to determine the
underlying factors driving this difference in engagement.

Our research findings offer valuable insights with significant implications
for various stakeholders (e.g., researchers, journalists, policymakers, and
social media content moderators) and deepen our understanding of how public
and private spheres influence the dissemination and engagement with news
articles on social media, enabling more informed decision-making. For example,
our identification of content classes that generate deeper user engagement can
provide actionable guidance to Facebook moderators, enabling them to focus
their efforts effectively. Furthermore, the disparities we uncovered between
public and private interaction patterns can guide future research that
considers both spheres, promoting a more comprehensive approach. To facilitate
such analysis, we present a robust methodology developed through active
collaboration with CrowdTangle, which ensures a fair comparison between public
and private spheres. This methodology equips researchers and content
moderators with a valuable tool to navigate the evolving landscape of online
news engagement.

In conclusion, by examining the differences in public and private sharing of
news articles, this study offers valuable insights into the user engagement
patterns associated with contents belonging to different bias and reliability
classes, as well as highlights the importance of understanding the dynamics of
news sharing and interaction both in the public and private spheres of social
media platforms such as Facebook. While challenges remain, our methodology and
discoveries offer valuable data-driven guidance toward addressing both
spheres.

##  10 Statements and Declarations

###  10.1 Conflict of interest

The authors have no competing interests.

###  10.2 Author contributions

Both authors have made substantial contributions to the conceptualization,
design, data collection, analysis, and writing of the research.

###  10.3 Acknowledgment

The authors thank the CrowdTangle team for their active feedback and help
during this research.

###  10.4 Data and code availability

All the codes associated with this research will be made available with the
camera-ready version. At the same time, to facilitate replicating the results
and conducting similar research, the list of all news articles considered in
this study and also the attachment status of all related posts will be
published.

## References

  *   * Newman et al. [2023] Newman, N., Fletcher, R., Eddy, K., Robertson, C.T., Nielsen, R.K.: Digital News Report 2023. Reuters Institute for the Study of Journalism (2023). https://reutersinstitute.politics.ox.ac.uk/sites/default/files/2023-06/Digital_News_Report_2023.pdf
  * Walker and Matsa [2021] Walker, M., Matsa, K.E.: News Consumption Across Social Media in 2021. Pew Research Center (2021). https://www.pewresearch.org/journalism/2021/09/20/news-consumption-across-social-media-in-2021
  * Thompson et al. [2020] Thompson, N., Wang, X., Daya, P.: Determinants of news sharing behavior on social media. Journal of Computer Information Systems 60(6), 593–601 (2020) https://doi.org/10.1080/08874417.2019.1566803
  * Geeng et al. [2020] Geeng, C., Yee, S., Roesner, F.: Fake news on facebook and twitter: Investigating how people (don’t) investigate. In: Proceedings of the 2020 CHI Conference on Human Factors in Computing Systems. CHI ’20, pp. 1–14. Association for Computing Machinery, New York, NY, USA (2020). https://doi.org/10.1145/3313831.3376784
  * Nah and Yamamoto [2018] Nah, S., Yamamoto, M.: Communication and Citizenship Revisited: Theorizing Communication and Citizen Journalism Practice as Civic Participation. Communication Theory 29(1), 24–45 (2018) https://doi.org/10.1093/ct/qty019
  * Jones et al. [2017] Jones, J.J., Bond, R.M., Bakshy, E., Eckles, D., Fowler, J.H.: Social influence and political mobilization: Further evidence from a randomized experiment in the 2012 us presidential election. PloS one 12(4), 1–9 (2017) https://doi.org/10.1371/journal.pone.0173851
  * Edelson et al. [2021] Edelson, L., Nguyen, M.-K., Goldstein, I., Goga, O., McCoy, D., Lauinger, T.: Understanding engagement with u.s. (mis)information news sources on facebook. In: Proceedings of the 21st ACM Internet Measurement Conference. IMC ’21, pp. 444–463. Association for Computing Machinery, New York, NY, USA (2021). https://doi.org/10.1145/3487552.3487859
  * Aldous et al. [2019] Aldous, K.K., An, J., Jansen, B.J.: View, like, comment, post: Analyzing user engagement by topic at 4 levels across 5 social media platforms for 53 news organizations. In: Proceedings of the International AAAI Conference on Web and Social Media, vol. 13, pp. 47–57 (2019). https://doi.org/10.1609/icwsm.v13i01.3208
  * Brena et al. [2019] Brena, G., Brambilla, M., Ceri, S., Di Giovanni, M., Pierri, F., Ramponi, G.: News sharing user behaviour on twitter: A comprehensive data collection of news articles and social interactions. In: Proceedings of the International AAAI Conference on Web and Social Media, vol. 13, pp. 592–597 (2019). https://doi.org/10.1609/icwsm.v13i01.3256
  * Marwick [2018] Marwick, A.E.: Why do people share fake news? a sociotechnical model of media effects. Georgetown law technology review 2(2), 474–512 (2018)
  * Vitak et al. [2011] Vitak, J., Zube, P., Smock, A., Carr, C.T., Ellison, N., Lampe, C.: It’s complicated: Facebook users’ political participation in the 2008 election. CyberPsychology, behavior, and social networking 14(3), 107–114 (2011) https://doi.org/10.1089/cyber.2009.0226
  * Mohammadinodooshan and Carlsson [2023] Mohammadinodooshan, A., Carlsson, N.: Effects of political bias and reliability on temporal user engagement with news articles shared on facebook. In: International Conference on Passive and Active Network Measurement (PAM), pp. 160–187. Springer, Cham (2023). https://doi.org/10.1007/978-3-031-28486-1_8
  * Dey et al. [2012] Dey, R., Jelveh, Z., Ross, K.: Facebook users have become much more private: A large-scale study. In: 2012 IEEE International Conference on Pervasive Computing and Communications Workshops, pp. 346–352 (2012). https://doi.org/10.1109/PerComW.2012.6197508
  * Lottridge and Bentley [2018] Lottridge, D., Bentley, F.R.: Let’s hate together: How people share news in messaging, social, and public networks. In: Proceedings of the 2018 CHI Conference on Human Factors in Computing Systems. CHI ’18, pp. 1–13. Association for Computing Machinery, New York, NY, USA (2018). https://doi.org/10.1145/3173574.3173634
  * Mellado et al. [2019] Mellado, C., Moreira, S., Lagos, F., Scherman, A.: Perceived bias, reliability, and journalistic quality in news and social media sharing behavior on facebook: Exploring the mediating role of perceived usefulness. Journalism & Mass Communication Quarterly 96(3), 825–847 (2019) https://doi.org/10.1177/1077699019846412
  * Guess et al. [2018] Guess, A., Nyhan, B., Reifler, J.: Less than you think: Prevalence and predictors of fake news dissemination on facebook. Science Advances 4(1) (2018) https://doi.org/10.1126/sciadv.aau4586
  * Pennycook et al. [2018] Pennycook, G., Cannon, T.D., Rand, D.G.: Prior exposure increases perceived accuracy of fake news. Journal of Experimental Psychology: General 147(12), 1865 (2018) https://doi.org/10.1037/xge0000465
  * Horne et al. [2019] Horne, B.D., Nørregaard, J., Adalı, S.: Different spirals of sameness: A study of content sharing in mainstream and alternative media. In: Proceedings of the International AAAI Conference on Web and Social Media, vol. 13, pp. 257–266 (2019). https://doi.org/10.1609/icwsm.v13i01.3227
  * Pidikiti et al. [2020] Pidikiti, S., Zhang, J.S., Han, R., Lehman, T., Lv, Q., Mishra, S.: Understanding how readers determine the legitimacy of online news articles in the era of fake news. In: 2020 IEEE/ACM International Conference on Advances in Social Networks Analysis and Mining (ASONAM). ASONAM ’20, pp. 768–775 (2020). https://doi.org/10.1109/ASONAM49781.2020.9381451
  * Ad Fontes Media [2022] Ad Fontes Media: Ad Fontes Media Bias Chart. https://adfontesmedia.com/interactive-media-bias-chart/ (2022)
  * Rajapaksha et al. [2018] Rajapaksha, P., Farahbakhsh, R., Crespi, N., Defude, B.: Inspecting interactions: Online news media synergies in social media. In: 2018 IEEE/ACM International Conference on Advances in Social Networks Analysis and Mining (ASONAM). ASONAM ’18, pp. 535–539 (2018). https://doi.org/10.1109/ASONAM.2018.8508534
  * Media Bias Fact Check [2022] Media Bias Fact Check: Media Bias Fact Check. https://mediabiasfactcheck.com/ (2022)
  * AllSides [2023] AllSides: AllSides. https://www.allsides.com/ (2023)
  * NewsGuard [2023] NewsGuard: NewsGuard. https://www.newsguardtech.com/ (2023)
  * Haq et al. [2022] Haq, E.-U., Lu, Y.K., Hui, P.: It’s all relative! a method to counter human bias in crowdsourced stance detection of news articles. Proceedings of the ACM on Human-Computer Interaction 6(CSCW), 1–25 (2022) https://doi.org/10.1145/3555636
  * Huszár et al. [2022] Huszár, F., Ktena, S.I., O’Brien, C., Belli, L., Schlaikjer, A., Hardt, M.: Algorithmic amplification of politics on twitter. Proceedings of the National Academy of Sciences 119(1) (2022) https://doi.org/10.1073/pnas.2025334119
  * Christensen et al. [2020] Christensen, S.R., Pilling, E.B., Eyring, J., Dickerson, G., Sloan, C.D., Magnusson, B.M.: Political and personal reactions to covid-19 during initial weeks of social distancing in the united states. PloS one 15(9), 1–16 (2020) https://doi.org/10.1371/journal.pone.0239693
  * Otero [2021] Otero, V.: Ad Fontes Media’s Multi-Analyst Content Analysis White Paper (2021). https://adfontesmedia.com/white-paper-2021
  * CrowdTangle [2021] CrowdTangle: CrowdTangle link checker (2021). https://chromewebstore.google.com/detail/crowdtangle-link-checker/klakndphagmmfkpelfkgjbkimjihpmkh/reviews
  * Tess [2021] Tess: What data is CrowdTangle tracking? https://help.crowdtangle.com/en/articles/1140930-what-data-is-crowdtangle-tracking (2021)
  * Tess [2022] Tess: Chrome Extension: Why do the items add up to more than the totals? CrowdTangle Help Center (2022). https://help.crowdtangle.com/en/articles/2252105-chrome-extension-why-do-the-items-add-up-to-more-than-the-totals
  * Meta [2022] Meta: URL - Graph API. Facebook for Developers (2022). https://developers.facebook.com/docs/graph-api/reference/v16.0/url
  * Kim and Yang [2017] Kim, C., Yang, S.-U.: Like, comment, and share on facebook: How each behavior differs from the other. Public Relations Review 43(2), 441–449 (2017) https://doi.org/10.1016/j.pubrev.2017.02.006
  * Weld et al. [2021] Weld, G., Glenski, M., Althoff, T.: Political bias and factualness in news sharing across more than 100,000 online communities. In: Proceedings of the International AAAI Conference on Web and Social Media, vol. 15, pp. 796–807 (2021). https://doi.org/%****␣paper.bbl␣Line␣525␣****10.1609/icwsm.v15i1.18104
  * Vosoughi et al. [2018] Vosoughi, S., Roy, D., Aral, S.: The spread of true and false news online. Science 359(6380), 1146–1151 (2018) https://doi.org/10.1126/science.aap9559
  * Altay et al. [2022] Altay, S., Kleis Nielsen, R., Fletcher, R.: Quantifying the “infodemic”: People turned to trustworthy news outlets during the 2020 coronavirus pandemic. Journal of Quantitative Description: Digital Media 2 (2022) https://doi.org/10.51685/jqd.2022.020
  * Samory et al. [2020] Samory, M., Kesiz Abnousi, V., Mitra, T.: Characterizing the social media news sphere through user co-sharing practices. In: Proceedings of the International AAAI Conference on Web and Social Media, vol. 14, pp. 602–613 (2020). https://doi.org/10.1609/icwsm.v14i1.7327
  * Lamot et al. [2022] Lamot, K., Kreutz, T., Opgenhaffen, M.: “we rewrote this title”: How news headlines are remediated on facebook and how this affects engagement. Social Media + Society 8(3), 20563051221114827 (2022) https://doi.org/10.1177/20563051221114827
  * Boukes et al. [2022] Boukes, M., Chu, X., Noon, M.A., Liu, R., Araujo, T., Kroon, A.C.: Comparing user-content interactivity and audience diversity across news and satire: Differences in online engagement between satire, regular news and partisan news. Journal of Information Technology & Politics 19(1), 98–117 (2022) https://doi.org/10.1080/19331681.2021.1927928
  * Wischnewski et al. [2021] Wischnewski, M., Bruns, A., Keller, T.: Shareworthiness and Motivated Reasoning in Hyper-Partisan News Sharing Behavior on Twitter. Digital Journalism 9(5), 549–570 (2021) https://doi.org/10.1080/21670811.2021.1903960
  * González-Bailón et al. [2022] González-Bailón, S., d’Andrea, V., Freelon, D., De Domenico, M.: The advantage of the right in social media news sharing. PNAS Nexus 1(3), 137 (2022) https://doi.org/10.1093/pnasnexus/pgac137
  * González-Bailón et al. [2023] González-Bailón, S., Lazer, D., Barberá, P., Zhang, M., Allcott, H., Brown, T., Crespo-Tenorio, A., Freelon, D., Gentzkow, M., Guess, A.M., et al.: Asymmetric ideological segregation in exposure to political news on facebook. Science 381(6656), 392–398 (2023) https://doi.org/10.1126/science.ade713
  * Maier [2015] Maier, S.R.: Compassion fatigue and the elusive quest for journalistic impact: A content and reader-metrics analysis assessing audience response. Journalism & Mass Communication Quarterly 92(3), 700–722 (2015) https://doi.org/10.1177/1077699015599660
  * de León and Trilling [2021] León, E., Trilling, D.: A sadness bias in political news sharing? the role of discrete emotions in the engagement and dissemination of political news on facebook. Social Media + Society 7(4) (2021) https://doi.org/10.1177/20563051211059710
  * Eberl et al. [2020] Eberl, J.-M., Tolochko, P., Jost, P., Heidenreich, T., Boomgaarden, H.G.: What’s in a post? how sentiment and issue salience affect users’ emotional reactions on facebook. Journal of Information Technology & Politics 17(1), 48–65 (2020) https://doi.org/10.1080/19331681.2019.1710318
  * Dewan and Kumaraguru [2017] Dewan, P., Kumaraguru, P.: Facebook inspector (fbi): Towards automatic real-time detection of malicious content on facebook. Social Network Analysis and Mining 7, 1–25 (2017) https://doi.org/10.1007/s13278-017-0434-5
  * Uppada et al. [2022] Uppada, S.K., Manasa, K., Vidhathri, B., Harini, R., Sivaselvan, B.: Novel approaches to fake news and fake account detection in osns: user social engagement and visual content centric model. Social Network Analysis and Mining 12(1), 52 (2022) https://doi.org/10.1007/s13278-022-00878-9
  * Vassio et al. [2022] Vassio, L., Garetto, M., Leonardi, E., Chiasserini, C.F.: Mining and modelling temporal dynamics of followers’ engagement on online social networks. Social Network Analysis and Mining 12(1), 96 (2022) https://doi.org/10.1007/s13278-022-00928-2