# Does Offline Political Segregation Affect the Filter Bubble? An Empirical
Analysis of Information Diversity for Dutch and Turkish Twitter Users

E. Bozdag<EMAIL_ADDRESS>Q. Gao<EMAIL_ADDRESS>G.J. Houben
<EMAIL_ADDRESS>M.E. Warnier<EMAIL_ADDRESS>Delft
University of Technology, Department Values and Technology, P.O. Box 5015 2600
GA Delft, The Netherlands  Philips Research, Eindhoven, the Netherlands  Delft
University of Technology, Web Information Systems Group, Delft, the
Netherlands  Delft University of Technology, Section of System Engineering.
Delft, the Netherlands

###### Abstract

From a liberal perspective, pluralism and viewpoint diversity are seen as a
necessary condition for a well-functioning democracy. Recently, there have
been claims that viewpoint diversity is diminishing in online social networks,
putting users in a “bubble”, where they receive political information which
they agree with. The contributions from our investigations are fivefold: (1)
we introduce different dimensions of the highly complex value viewpoint
diversity using political theory; (2) we provide an overview of the metrics
used in the literature of viewpoint diversity analysis; (3) we operationalize
new metrics using the theory and provide a framework to analyze viewpoint
diversity in Twitter for different political cultures; (4) we share our
results for a case study on minorities we performed for Turkish and Dutch
Twitter users; (5) we show that minority users cannot reach a large percentage
of Turkish Twitter users. With the last of these contributions, using theory
from communication scholars and philosophers, we show how minority access is
missing from the typical dimensions of viewpoint diversity studied by computer
scientists and the impact it has on viewpoint diversity analysis.

###### keywords:

Twitter , diversity , polarization , politics , Turkey , Netherlands

††journal: Computers in Human Behavior (forthcoming)

*figure

##  1 Introduction

It is well known that traditional media have a bias in selecting what to
report and in choosing a perspective on a particular topic. Individual factors
such as personal judgment can play a role during the selection of news for a
newspaper. Selection bias, organizational factors, advertiser and government
influences can all affect which items will become news (Bozdag, 2013). About
37% of Americans see a great deal of political bias in news coverage and 68%
percent prefer to get political news from sources that have no particular
point of view (Pew Research, 2012). Similarly, in a survey performed before
the general elections in the UK, 96% of the population said they believe they
have seen clear bias within the UK media (Wei et al., 2013). Evidence of bias
ranges from the topic choice of the New York Times to the choice of think-
tanks that the media refer to (DellaVigna & Kaplan, 2007).

Many democracy theorists claim that modern deliberative democracy requires
citizens to have socially validated and justifiable preferences. Citizens must
be exposed to opposed preferences and viewpoints and should be able to defend
their views (Held, 2006; Offe & Preuss, 1990; Dryzek, 1994). Exposure to
biased news information can foster intolerance to opposing viewpoints, lead to
ideological segregation and antagonisms in major political and social issues
(Glynn et al., 2004; An et al., 2012; Saez-Trumper et al., 2013). Being aware
of and overcoming bias in news reporting is essential for a fair society, as
media has the power to shape voting behavior (Saez-Trumper et al., 2013).

Social information streams, i.e., status updates from social networking sites,
have emerged as a popular means of information sharing. Political discussions
on these platforms are becoming an increasingly relevant source of political
information, often also used as a source of quotes for media outlets (Jürgens
et al., 2011). Traditional media are declining in their gatekeeping role to
determine the agenda and select which issues and viewpoints reach their
audiences (Bruns, 2011). Internet users have moved from scanning traditional
media such as newspapers and television to using the Internet, in particular
social networking sites (An et al., 2012). Social networking sites are thus
now acting as gatekeepers (Bozdag, 2013).

Communication theorists argue that the traditional media are declining in
their gatekeeping role to determine what is “newsworthy” and select which
issues and viewpoints will reach their audiences Bruns (2011). It is often
argued that the Internet, by promoting equal access to diverging preferences
and opinions in society, actually increases information diversity. Many
scholars characterize the online media landscape as the “age of plenty , with
an almost infinite choice and unparalleled pluralization of voices that have
access to the public sphere (Karppinen, 2009). Some argue that social media
will disrupt the traditional elite control of media and amplify the political
voice of non-elites and minorities (Castells, 2011).Still others claim that
tools such as Twitter are neutral spaces for collaborative news coverage
operated by third parties outside the journalism industry. As a result, the
information curated through collaborative action on such social media
platforms should be expected to be drawn from a diverse, multi-perspectival
range of sources (Bruns, 2011). Some further claim that platforms such as
Twitter are neutral communication spaces, and offer a unique environment in
which journalists are free to communicate virtually anything to anyone, beyond
many of the natural constraints posed by organizational norms that are
existing in traditional media (Lasorsa et al., 2012).

On the other hand, there are skeptical voices that argue that the Internet has
not fundamentally changed the concentrated structure typical of mass media,
but reflects the previously recognized inequalities (Karppinen, 2009). It is
also argued that it has brought about new forms of exclusion and hierarchy
(Suoranta & Vadén, 2009). While it has increased some sort of political
participation, it has empowered a small set of elites and they still strongly
shape how political material is presented and accessed (Hindman, 2008). Others
have pointed out the danger of “cyberbalkanization”” caused by the
Internet(Sunstein, 2002; Pariser, 2011). They argue that the filters we choose
on the Internet, or the filters that are imposed upon us will weaken the
democratic process. This is because it will allow citizens to join into groups
that share their own views and values, and cut themselves off from any
information that might challenge their beliefs. Group deliberation among like-
minded people can create polarization; individuals may lead each other in the
direction of error and falsehood, simply because of the limited argument pool
and the operation of social influences.

It is thus very important to verify whether viewpoint diversity is diminishing
in social media and whether cyberbalkanization indeed occurs. There are
empirical studies that have observed a high level of information diversity in
Twitter and Facebook, mainly due to retweets and weak-ties (Bakshy et al.,
2012; An et al., 2011; Sun et al., 2013). While being very valuable
contributions to the literature, these studies often focus on American users
and they define information diversity either as “novelty”, or “source
diversity”. However, as we will show below, novel information does not
necessarily contribute to information diversity and highly competitive media
markets with many sources may still result in excessive sameness of media
contents. As we will argue, marginalized members of segregated groups,
structurally underprivileged actors and minorities must receive special
attention and just measuring number of available sources will not guarantee
viewpoint diversity.

In this paper, we contribute with a framework to analyze and understand the
impact of political culture in Twitter. Rather than reducing the concept
viewpoint diversity to a single quantity or metric, we introduce different
dimensions of viewpoint diversity, based on previous studies and the theory
from communication studies and political philosophy. In addition, we provide a
set of new metrics and operationalize them. Finally, we present the result of
a case study we performed for Dutch and Turkish Twitter users using this
framework. We show that minority users cannot reach a large percentage of the
studied Turkish Twitter users and political culture is making a difference.

##  2 Empirical Studies of Information Diversity in Social Media

An empirical study performed by Facebook suggests that online social networks
may increase the spread of novel information and of diverse viewpoints.
According to Bakshy (2012), even though people are more likely to consume and
share information that comes from close contacts that they interact with
frequently, the vast majority of information comes from contacts that they
interact with infrequently. These so-called “weak-ties” (Granovetter, 1981)
are also more likely to share novel information. However, there are some
concerns with this study. First, Facebook does not provide open access to
everyone, thus we can not repeat or reproduce the results using Facebook data.
Second, our weak ties give us access to new stories that we would not
otherwise have seen, but these stories might not be different ideologically
from our own general worldview. They might be novel information, but not
particularly diverse. The concepts serendipity, diversity and novelty are
different from each other (Sun et al., 2013). The Facebook research does not
indicate whether we encounter and engage with news that opposes our own
beliefs through “weak-links”.

Twitter, with its API, provides an excellent environment for information
diversity research. An et al. (2012) observe extreme polarization among media
sources in Twitter. In another study, they found that, when direct
subscription is considered alone, most Twitter users receive only biased
political views they agree with (An et al., 2011). However, they note that the
news media landscape changes dramatically under the influence of retweets,
broadening the opportunity for users to receive updates from politically
diverse media outlets. Sun et al. (2013) performed an empirical study using
statistical models to identify serendipity in Twitter and Weibo. Using
likelihood ratio test and by measuring unexpectedness and relevance, they
observe high levels of serendipity in information diffusion in microblogging
communities. Saez-Trumper et al. (2013) found that political bias is evident
in social media, in terms of the distribution of tweets that different stories
receive. Further, statement bias is evident in social media; a more
opinionated and negative language is used than the one used in traditional
media. Twitter users are more interested in what is happening directly around
them and what is happening to those around them. While communities talk about
a broad range of news, Twitter users dedicate most of their tweets to a few of
them (Saez-Trumper et al., 2013). Wei et al. (2013) found that individual
journalists have the strongest influence on Twitter for UK users. Further,
they observed that all influential British Twitter users (mainstream media,
journalists and celebrities) display some kind of bias towards a particular
political party in their tweets. Jurgens et al. (2011) show that certain
individual German Twitter users act as gatekeepers, especially in the
distribution of political information. Those users are also not neutral hubs.
They tend to curate political information and post messages that they find
important (Jürgens et al., 2011).

##  3 Theory

In this section, we first give a short overview “information diversity” and
explain why it is a vital value for a democratic society. Later, we show
different dimensions of this value and show how it can be defined.

###  3.1 Information Diversity

A cyberbalkanized Internet or “filter bubble” is not acceptable in different
models of modern democracy. Aggregative versions of democracy hold that
legitimacy lies in the fair counting of votes casted by informed voters Held
(2006). Deliberative democrats on the other hand hold that a decision is only
legitimate if it is determined by fair, informed deliberations (Fishkin, 1993;
Cohen, 2009). Because no set of values or preferences can claim to be correct
by themselves, they must be justified and tested through social encounters
which take the point of view of others into account (Held, 2006). In addition
to the normative value of discussion, information-sharing is required for many
of the practical benefits that proponents of deliberation hope deliberative
institutions will provide, such as higher quality policy, greater appreciation
of the views of the opposing side, cultural pluralism and citizen welfare
(Napoli, 1999). According to deliberative democrats, we must focus on why and
how we come to adopt our views, and whether they can be defended in a complex
social setting with people with opposed preferences. This will complement
voting, the necessary mode of participation, by a “conscious confrontation of
one’s own point of view with an opposing point of view, or of the multiplicity
of diverse viewpoints that the citizen, upon reflection, is likely to discover
within his or her own self”(Offe & Preuss, 1990). Under conditions of ideal
deliberation, ‘no force except that of the better argument is exercised’
(Habermas, 1975).

Information diversity is also an important concept in communication studies.
The freedom of media, a multiplicity of opinions and the good of society are
inextricably connected (Napoli, 1999). _Free Press theory_ , a theory of media
diversity, states that we establish and preserve conditions that provide many
alternative voices, regardless of intrinsic merit or truth, with the condition
that they emerge from those whom society is supposed to benefit its individual
members and constituent groups(van Cuilenburg & McQuail, 2003). What is good
for the members of the society can only be discovered by the free expression
of alternative goals and solutions to problems, often disseminated through
media (Napoli, 1999).

While many scholars from different disciplines agree that information
diversity is an important value that we should include in the design of
institutions, policies and online services, this value is often reduced to a
single definition, such as “source diversity”, or “hearing the opinion of the
other side”. In the next subsections, we explain that just having a
deliberation is not enough, and that a bias against arguments made by
deliberators who are in the minority in terms of their interests in the
decision being made can exist.

###  3.2 Dimension of Information Diversity

Following Napoli (1999), we may distinguish three different dimensions of
information diversity. The first dimension is source diversity, which is
diversity in terms of outlets (cables and channel owners) or program producers
(content owners). Content diversity consists of diversity in format (program-
type), demographic (in terms of racial, ethnic, and gender), and idea-
viewpoint (of social, political and cultural perspectives). The third
dimension, exposure diversity, deals with audience reach and whether users
have actually consumed a diverse set of items.

In US media policy, with the “free marketplace of ideas” theory, it is assumed
that increasing source diversity will increase content diversity and exposure
diversity will follow these two. American media policy consequently focuses on
source diversity by way of competition and antitrust regulation (van
Cuilenburg, 2002). However, whether more media competition (more sources)
really brings about more media variety is highly debated and research
addressing this relationship has not provided definitive evidence of a
systematic relationship (Napoli, 1999; van Cuilenburg, 1999; McQuail & van
Cuilenburg, 1983; Karppinen, 2013). Highly competitive media markets may still
have low content diversity and media monopolies can produce highly diverse
supply of media content (van Cuilenburg, 2002). It has also been argued that
to fulfill the objectives of the marketplace of ideas metaphor, policymakers
need to focus on exposure diversity. So, one should not look at availability
of different sources or content, but whether the public consumes a diverse set
of items (Napoli, 1999).

###  3.3 Minorities and Openness

Karppinen (2009) argues that the aim of media diversity should not be the
multiplication of genre, sources or markets, but giving voice to different
members of the society. We should not see diversity as something that can be
measured through the number of organizations or channels or just “having two
parties reach all citizens”. Karppinen holds that we should focus on
democratic distribution of communicative power in the public sphere and
whether everyone has the chance and resources to get their voices heard.
Karppinen argues: “the key task for media policy from the radical pluralist
perspective is to support and enlarge the opportunities for structurally
underprivileged actors and to create space for the critical voices and social
perspectives excluded from the systematic structures of the market or state
bureaucracy”(Karppinen, 2009). If democratic processes and public policies
exclude and marginalize members of segregated groups from political influence
to the extent that privileged groups often dominate the public policy process,
they will magnify the harms of segregation. These “minorities” must be
politically mobilized and included as equals in a process of discussing issues
(Young, 2002).

McQuail and van Cuilenburg (1983) propose to assess media diversity by
introducing two normative frameworks. The norm of reflection checks whether
“media content proportionally reflects differences in politics, religion,
culture and social conditions in a more or less proportional way . The norm of
openness checks whether media “provide perfectly equal access to their
channels for all people and all ideas in society . If the population
preferences were uniformly distributed over society, then satisfying the first
condition (reflection) would also satisfy the second condition (equal access).
However, this is seldom the case (van Cuilenburg, 1999). Often population
preferences tend toward the middle and mainstreams. In such cases, the media
will not satisfy the openness norm, and the preferences of the minorities will
not reach a larger public. This is undesirable, because “social change usually
begins with minority views and movements ($...$) asymmetric media provision of
content may challenge majority preferences and eventually may open up majority
preferences for cultural change in one direction or another” (van Cuilenburg,
1999). Van Cuilenburg (1999) argues that the Internet has to be assessed in
terms of its ability to give open access to new and creative ideas, opinions
and knowledge that the old media do not cover yet. Otherwise it will only be
“more of the same”.

##  4 Polarization in the Netherlands and Turkey

Before discussing methods and the results of our empirical study that focused
on Dutch and Turkish users, we give a short overview of political diversity
for those two countries and explain why they are interesting for a case study
on information diversity.

###  4.1 the Netherlands

Pillarization (Dutch: “verzuiling”) is a process that occurred in the
Netherlands and reached its highest point in 1950 s. During this period,
several ideological groups making up Dutch society were systematically
organized as parallel complexes that were mutually segregated and polarized
(van Doorn, 1956; Post, 1989). As part of this social apartheid dividing the
population into subcultures, political parties were used for political
mobilization of the ideologically and religiously defined groups and social
activities were concentrated within the particular categorical group
(Steininger, 1977). Few contact existed between the different groups and
internally the groups were tightly organized (Lijphart, 1968). Elites at the
‘top’ level communicated, while the ones at the ‘bottom’ did not.
Pillarization had an effect on parental choice of an elementary school for
children, the voting for political parties and the choice on which daily
newspaper to read (Kruijt, 1962). People belonging to a pillar retreated into
their own organizations and entered into a ‘voluntary’ isolation, because they
perceive that values important to them were threatened (Marsman, 1967).

Depillarization (Dutch: “ontzuiling”) started in mid 1960’s as a
democratization process and pillarization has lost much of its significance
since the 1960’s as a result of secularization and individualization. Even
though depillarization has started, many institutional legacies in present-day
Netherlands still reflect its pillarized past, for example in its public
broadcasting system or in the school system (Vink, 2007). The Netherlands
continues to be a country of minorities, which may be a main reason that
consensus seems so ingrained in the Dutch political culture (Peter van der
Hoek, 2000). The Dutch parliament currently has 12 political parties. Due to
the very low chance of any party gaining power alone, parties often form
coalitions.

The Netherlands has created several media policies to implement diversity in
the media. The Media Monitor, an independent institution, measures ownership
concentration, editorial concentration and audience preferences (Aslama et
al., 2007). It also measures diversity of television programming on the basis
of a content classification system, by categorizing program output in
categories like news and information, education, drama, sports, etc.
(Mediamonitor, 2013; van Cuilenburg, 2002).

###  4.2 Turkey

Turkey has regularly held free and competitive elections since 1946. The
country has alternated between a two-party political system and a multi-party
system. Electoral politics has often been dominated by highly ideological
rival parties and military inventions changed the political landscape several
times (Tessler & Altinoglu, 2004). Elections in 2002 led to a two-party
parliament, partially due to a ten per cent threshold. The Justice and
Development Party111Adalet ve Kalkinma Partisi (AKP) won the elections and
still is the ruling party, having an absolute majority. The parliament is
currently formed by 4 political parties. While AKP has 59% of the MP’s,
secular Republican People’s Party222Cumhuriyet Halk Partisi (CHP) has 24%.

AKP’s dominance and the despair and sense of marginalization felt by its
opponents threaten to create a political polarization. Muftuler-Bas and Keyman
(2012) argue that “many other polarizing social and political struggles remain
unresolved in Turkey, and mutually antagonistic groups remain unreconciled.
This social and political polarization remains potentially explosive and
reduces the capacity for social consensus and political compromise”. Similarly
Unver (2011, p.2) claims that “the society is pushed towards two extremes that
are independent of party politics. (…) Competing narratives and ‘realities’
clash with each other so intensely, that the resultant effect is one of
alienation and ‘otherness’ within the society.”

Some scholars argue that, the top-down imposition of concepts such as
democracy, political parties and parliament as part of westernization efforts
is causing the socio-political polarization in Turkey (Altintas, 2003).
Agirdir (2010) argues that “the system does not breed from the diverse
interests and demands of the society, but around the values and interests of a
party leader and the narrow crew around her”. Economic voting behavior,
religiosity, and modern versus traditional orientation seem to be the
strongest drivers of polarization (Yilmaz et al., 2012). Some argue that,
after 2011 polarization has increased and reached its highest points in
Turkish history (Ozturk, 2013). The difference of opinion between different
clusters about secularity, tolerance and political change issues in total
contradiction of each other, therefore a danger of absolute social
polarization is imminent (Agirdir, 2010). Kiris (2011) observes an identity-
based polarization, between secularists and islamists, between Turkish
nationalists and Kurdish Ethnic Nationalists, and between Alevis and Sunnis
(different sects of Islam).

Turkish Radio Television Supreme Council (RTUK) was established in order to
control whether Turkish language, Turkish history, historical values, Turkish
way of life, thoughts and feelings are being given a significant place in
broadcasting programs (Acar, 2004). RTUK is sometimes referred as “the Censure
Board” (Müftüler Baç, 2005) and its decisions of penalizing the broadcasters
have been criticized domestically and internationally (Baris, 2007; Demir &
Ben-Zadok, 2007). RTUK does not have a diversity policy and the lack of
diversity in programme-making is said to undermine the quality of the audio-
visual media (Baris, 2007).

###  4.3 Conclusion

In short, The Netherlands and Turkey are two different countries in terms of
political landscape and diversity policy. The Dutch society is less polarized
than it was half a century ago, while the Turkish society is thought to be
heavily polarized. The Dutch Parliament contains many political parties, no
party has absolute power to govern alone. Turkey, on the other hand has few
political parties represented in the government and the ruling party has
almost 60% of all the seats. Further, the Dutch media is regulated with a
diversity policy. While Turkey has a similar institution, it acts more as a
censor board and does not employ an active diversity policy. If the social
networking platforms mirror the society, then we can expect the Dutch users to
receive more diverse content, while the Turkish users to be more polarized and
have a less diverse newsfeed.

##  5 Method

In this section we provide our method of data collection, present our research
model and the metrics we operationalized to measure information diversity.

###  5.1 Data collection

In January 2013, over a period of more than one month we crawled microblogging
data via the Twitter’s REST and streaming APIs333https://dev.twitter.com/. We
started from a seed set of Dutch and Turkish Twitter users $U_{s}$, who mainly
publish news-related tweets. We have selected different types of users
including mainstream news media, journalists, individual bloggers and
politicians. The list of these “influential” users were picked up from
different ranking sites. For the Dutch ranking, we used
Peerreach444http://peerreach.com/lists/politics/nl,
Twittergids555http://twittergids.nl/ and Haagse Twitter-
stolp666http://alleplanten.net/twitter/site/de-resultaten/belangrijke-
personen/ . For Turkish ranking, we used
TwitterTurk777http://twitturk.com/twituser/users/turk and
TwitterTakip888http://www.twittertakip.com/.

This resulted in two lists for Turkey and the Netherlands, both containing
seed users categorized in political groups, which differed per country. We
mapped the political leaning of Dutch seed users into five groups and the
political leaning of Turkish seed users into nine groups. We did this using a
number of public sources (Krouwel, 2008; van der Eijk, 2000; Trendlight, 2012;
Carkoglu & Kalaycioglu, 2007). Later, we defined some of the seed users as a
“minority”. We did this by selecting seed users who belong to a political
party that is either not represented in the parliament, or is represented with
few MP’s. We also included MP’s of a large political party who belong to an
ethnic minority. That makes for instance the Kurdish Party BDP and its MP’s a
minority in Turkey, while we consider the Greens as a minority in the
Netherlands. See Appendix A for a list of minorities. Both user groups defined
as minorities create about 15% of the all observed tweets for both countries.

By monitoring the Twitter streams of $U_{s}$, we were able to add another set
of users $U_{n}$, who followed and retweeted at least 5 items from users in
$U_{s}$. After removing users who were involved in spam, we had 1981 Dutch
users and 1746 Turkish users. After crawling seed user tweets and identify the
retweets made by their followers, we operationalized various metrics. In the
following subsections, we explain the translation of research questions into
metrics.

###  5.2 Research questions

The main question in this research is the following: “Does offline political
segregation affect information diversity in Twitter?”. To answer this
question, we have provided some sub-questions.

  1. 1.

Q1- Seed User Interaction: Do seed users from one end of the political
spectrum ever tweet links from another category? Do they reply to each other?
The results of this question is relevant to the previously conducted studies
that studied media bias on Twitter, such as Wei et al. (2013)

  2. 2.

Q2 - Source Diversity: Is the newsfeed of social media users diverse? Are they
receiving updates from a diverse set of users? Does indirect exposure (e.g.,
via retweets or weak-links) increase diversity marginally? Result of these
questions are relevant to the previously conducted studies, such as An et
al.’s (2011).

  3. 3.

Q3- Output Diversity: Do users share items from a diverse set of users or
mainly from the same political category? This question is relevant to the
framework provided by Napoli, which we have mentioned in Section 3.2.

  4. 4.

Q4 - Openness/Minority Diversity: Can minorities reach the social media users,
so that “equal access” principle is satisfied? Or can only the ”popular” reach
a bigger public? This question is relevant to the normative theory of McQuail
& van Cuilenburg (1983) and Karppinen (2013), which we discussed in Section
3.3.

  5. 5.

Q5 - Input-Output Correlation: Do users post political messages whose
political position reflects the political position of those messages that the
users receive? Or do the messages they choose to retweet show a political
position significantly skewed from the political position of the messages
which they receive? Result of this question is relevant to the previously
conducted studies such as Jurgens et al.’s (2011).

###  5.3 Entropy

While translating the concepts introduced in the previous subsection into
metrics, we apply the following entropy formula used by van Cuilenburg (2007)
to measure traditional media diversity, which is based on the work of Shannon
(1948) :

$-(\sum(p_{i}\log p_{i})/log(n))$ (1)

In van Cuilenburg & van der Wurff (2007), $p_{i}$ represents the proportion of
items of content type category $i$. $n$ represents the number of content type
categories. We use this formula for calculating source diversity and exposure
diversity in our Twitter study. For instance in source diversity, $p_{i}$
represents incoming tweets from seed users with a specific political stance,
while $n$ represents all possible categories. As a result of this formula, the
user will have a diversity between 0 and 1, where 0 represents minimum
diversity and 1 represents maximum diversity. Figure 1 shows a user that
receives equal amount of tweets (20) from all political categories and has an
incoming diversity of 1. The user only retweets from one political category
(10 from Category 1), therefore she/he has an outgoing diversity of 0.

Figure 1: Applying entropy

###  5.4 Translating Research Questions into Metrics

Source Diversity:

For each user, we used Equation 1 to calculate his/her source diversity. For a
user A, we compare the tweets published by A’s direct followees (people A
follows) from different groups of which the political leanings have been
categorized as discussed above (See Figure 3). This gives us a user’s direct
input entropy. We then also added the tweets A gets through retweets and
investigated if A receives more diverse information through indirect media
exposure (See Figure 3). This provides us a user’s indirect input entropy. In
both of figures, arrows show the information flow.

Figure 2: Direct source diversity

Figure 3: Indirect source diversity

Figure 4: Minority access

Figure 5: Input-Output Correlation

Output Diversity:

To measure what the user is sharing after she/he was exposed to different
incoming information, we used Equation 1 to compare the retweets and replies
she/he makes for each political category.

Openness/Minority Diversity:

For this definition of diversity, we first defined all seed users who belong
to a political party that is either not represented in the parliament, or is
represented with few MP’s. We also included MP’s of a large political party
who belong to an ethnic minority. That makes for instance the Kurdish Party
BDP and its MP’s a minority in Turkey, while we consider the Greens as a
minority in the Netherlands. See Appendix A for a list of minorities. Both
users defined as minorities create about 15% of the all observed tweets for
both countries.

We then looked whether the user is receiving minority tweets directly or
indirectly. For instance, in Figure 5, User A is receiving minority tweets by
direct subscription, but also indirectly via User B. We defined two metrics to
measure minority access. We first look at the ratio of minority tweets a user
gets out of all minority tweets:

$\mbox{\\# received minority tweets}\over\mbox{\\# all published minority
tweets}$ (2)

We later calculate the ratio of minority tweets in a users’ timeline

$\mbox{\\# received minority tweets}\over\mbox{\\#received tweets from seeds}$
(3)

Input-Output Correlation:

For each user in our sample we look whether the maximum number of the
political position of the messages retweeted by a user is significantly skewed
from the political position of the messages that she/he receives.

$\max(\mbox{incoming political category})==\max(\mbox{outgoing political
category})$ (4)

For instance, Figure 5 shows a biased user which receives most items from
category 1, and also retweets mainly from category 1.

##  6 Results

This section shows the results for the defined metrics. We tested statistical
significance of our results with a two-tailed $t$-Test where the significance
level was set to $\alpha=0.01$ unless otherwise noted.

Figures 7 and 7 show the distribution of the seed users for both countries.
Figures 9 and 9 show the distribution of regular users. We see that our
selection of popular users covers the political spectrum and it is not
concentrated on a single political category. We have used several sources to
do the seed user categorization (Krouwel, 2008; van der Eijk, 2000;
Trendlight, 2012; Krouwel, 2012, 2008; Baris, 2007). We used the retweet
behavior of the users to assign them to a political category to identify their
political stance.

Figure 6: Dutch seed user distribution

Figure 7: Turkish seed user distribution

Figure 8: Dutch user distribution

Figure 9: Turkish user distribution

###  6.1 Seed User Interaction

To answer research question Q1, Tables 1a and 1b show the retweet and reply
behavior of seed users. Each row shows the category of users who retweet an
item or reply to another user. The columns show the source of their retweet or
the user they interact with. We observe that 73% of the left seed users
retweet only from left users and reply to left users, while 72% of the right
users do the same. The situation is more extreme for Turkish seed users: 93%
of left seed users only retweet from and reply to left users, while 94% of the
right seed users show the same behavior.

###  6.2 Source and Output Diversity

Table 2a shows the results for research questions Q2 and Q3. Here we see that
on a scale of 0 to 1, the diversity of the incoming tweets for an average user
is approximately 0.6 and the results are not very different for both
countries. While diversity is not perfect, we do not observe a true
cyberbalkanization and we do not observe a significant difference between two
countries. We observe that indirect communication (retweets) does increase
diversity, but not dramatically. Figure 11 and Figure 11 show the distribution
of source diversity among users. We observe that, indirect communication
decreases the number of users who have a diversity approaching 0 for both
countries. Approximately 27% of the Dutch and 29% of the Turkish users have an
indirect diversity less than 0.5.

However, if we look at the diversity of an average user’s output, we see much
lower numbers. As Table 2b shows, on a scale of 0 to 1, retweet diversity is
0.43 for the Dutch and 0.40 for the Turkish users. If we look at reply
diversity, it is 0.42 for the Dutch and 0.29 for the Turkish users. Figures 13
and 13 show the distribution of output (retweet and reply) diversity among the
population. About 55% of the Dutch and 66% of the Turkish users have a retweet
diversity lower than 0.5, and about 17% of the Dutch and 12% of the Turkish
users have a retweet diversity lower than 0.01 ($p<0.001$). About 61% of the
Dutch and 77% of the Turkish users have a reply diversity lower than 0.5, and
about 24% of the Dutch and 26% of the Turkish users have a reply diversity
lower than 0.01 ($p<0.001$). This means that the users show bias for both
their retweet and reply preferences and especially the Turkish reply diversity
is quite low.

###  6.3 Opennes/Minority Diversity

Table 2d shows the results for the research question Q4. First row, which we
call “minority reach” shows the result for Equation 2 and the second row,
which we call “minority exposure” shows the result for Equation 3. Minority
reach measures how many percent of all the produced tweets by minorities reach
an average user. Minority exposure checks the ratio of minority tweets in a
user’s newsfeed. We observe that an average Dutch Twitter users will receive
15% of the produced minority tweets, whereas an average Turkish user will only
receive 2% of them. Later, we observe that minority tweets make up 23% of an
average Dutch users’ incoming tweets from seed users, while it only makes up
2% for a Turkish user. Figures 15 and 15 show the distribution of users for
this metric. Here we observe a significant difference between two countries
($p<0.001$). About 55% of the Turkish users have a minority exposure under
0.05 and 57% of them have a minority reach under 0.05. The percentages are
much lower for the Dutch users: 14% and 23% respectively. This means that more
than half of the Turkish users are missing almost all the updates produced by
the minorities and their newsfeed contains almost no minority tweets at all.
This includes indirect minority tweets thanks to retweets done by their
friends.

###  6.4 Input-Output Correlation

Table 2c shows the results for the research question Q5. The first row shows
the number of users whose output correlates with their input. Such users make
up 33% of the Dutch and 47% of the Turkish userbase. Further, if we only
consider a bias towards a certain political category that is higher than 15%
(for both input and output), 26% of the Dutch and 36% of the Turkish users
show this behavior.

Table 1: Seed user bias

User / Source | Left | Right
---|---|---
Left | 73% | 27%
Right | 28% | 72%
(a) Netherlands

User / Source | Left | Right
---|---|---
Left | 93% | 7%
Right | 6% | 94%
(b) Turkey

Table 2: Different dimensions of diversity. NL = the Netherlands, TR = Turkey.

| NL | TR
---|---|---
Direct | 0.63 | 0.58
Indirect | 0.68 | 0.62
(a) Source Diversity (on a scale of 0 to 1)

| NL | TR
---|---|---
Retweet | 0.43 | 0.40
Reply | 0.42 | 0.29
(b) Output Diversity (on a scale of 0 to 1)

| NL | TR
---|---|---
# users | 657 | 828
% users | 33% | 47%
(c) Input-Output Correlation

| NL | TR
---|---|---
minority reach | 15% | 2%
minority exposure | 23% | 2%
% users under $<$0.05 reach | 14% | 57%
% users under $<$0.05 exposure | 23% | 55%
(d) Openness/Minority Diversity

Figure 10: Direct source diversity

Figure 11: Indirect source diversity

Figure 12: Output diversity (retweet diversity)

Figure 13: Output diversity (reply diversity)

Figure 14: Minority Reach

Figure 15: Minority Exposure

##  7 Discussion

In this study we have shown different dimensions of diversity and discussed an
additional dimension, namely minority access. This dimension has not been
previously been subjected to large-scale quantitative validation. However, as
many communication scholars and philosophers have argued, while the media
should reflect the preferences present in the society, it should also allow
equal access to everyone, including those whose common social location tends
to exclude them from political participation. Public life needs to include
differently situated voices, not just the majority.

We have shown that different definitions of diversity can be operationalized
in different metrics and the question whether “the filter bubble exists in
social media” will have different answers depending on the metric and
political segregation of the observed country. For instance, according to the
results of our study, source diversity does not differ much for Turkish and
Dutch users and we certainly cannot observe a bubble. However, if we consider
output, then we see that the diversity is much lower. Further, if we consider
the minority access as a diversity metric, we see that minorities cannot reach
a large percentage of the Turkish population.

Twitter, with its 140 character limitation, is not the ideal platform for
deliberation. However, having a diverse information stream in Twitter is still
important, as it can serve as an input for deliberation elsewhere. Information
intermediaries such as Twitter could have a considerable influence in nudging
people towards more valuable and diverse choices (Helberger, 2011). Media
diversity has been an important policy objective for the regulation of
traditional media (van Cuilenburg & McQuail, 2003). Journalism ethics requires
the newspapers to have a balanced and fair coverage of news and opinions and
editors and journalists to minimize bias in their filtering decisions. In the
abundance of digital online information and algorithmic filters to deal with
information overload, bias should also be minimized and ideas and opinions of
minorities should not be lost.

Design choices in software codes and other forms of information politics still
largely determine the way information is made available and who can speak to
whom under what condition (Karppinen, 2009). According to Karppinen (2009), it
is important to make decisions about standards, because those “can have
lasting influence on media pluralism, even if they are not necessarily
recognized as sites of media policy as such”. However, making minority voices
reach a wider public is no easy matter. While identifying minorities and their
valuable tweets is no easy task, showing these items to “challenge averse”
users is a real challenge (Munson & Resnick, 2010). For instance Munson et
al.(2013) provided people with feedback about the political lean of their
reading behaviors and found that such feedback had only a small effect on
nudging people to read more diversely. More research is needed to understand
how users’ reading behavior change and to determine the conditions that would
allow such a change.

Further, normative questions arise while making design decisions. When
designing diverse recommendation systems, it is definitely a challenge to
determine which view is “valid”. For instance, should a recommendation system
show all viewpoints in the climate change debate, if some viewpoints are not
empirically validated or simply seen as false by the majority of the experts?
Should a viewpoint get equal attention even if it provides no information or
only contains arguments with fallacies? These questions would need to be
addressed by a good ethical analysis before design decisions of such systems
are made.

##  8 Limitations

This study has several limitations. First of all, next to the accounts of
traditional media outlets on Twitter, we also selected politicians and
bloggers. While they mainly tweet political matters, it is possible that they
have shared personal and non political matters as well.

Second, while the results give us an idea on the political landscape of the
studied countries, Twitter does not represent ‘all people’. As boyd and
Crawford (2011) have stated, “many journalists and researchers refer to
‘people’ and ‘Twitter users’ as synonymous (…) Some users have multiple
accounts. Some accounts are used by multiple people. Some people never
establish an account, and simply access Twitter via the web”. Therefore we
cannot conclude that our sample represent the real population of the studied
countries.

Third, input-output correlation does not always implicate that the volume of
the content affects the items users share. Users might already be biased
before they select their sources and can therefore follow more from certain
sources and share from certain categories.

Fourth, users will make different uses of Twitter. Some might use it as its
primary news source, therefore following mainstream items, while others will
use it to be informed of the opposing political view or to find items missing
in the traditional media. Therefore, we do not know why some users only follow
sources from a specific political category. More qualitative studies are
needed.

Fifth, retweets in Twitter can be made for different purposes. There is a
difference between endorsement retweets (created by pushing the retweet
button) and informal tweets (where users include most of the same text often
prefixed by ‘RT’ or similar but also add their own comments before or after
the tweet). These two actions measure a different interaction. Informal
retweets and replies could also express disagreement and show us deliberation.
In order to make a distinction between these two types of tweets, we need
semantic analysis. We are not aware of the availability of such tools for the
Turkish language, therefore we were unable to perform such an analysis. While
information diversity is important not only for deliberative models of
democracy, it would be very useful to study deliberation on Twitter by using
such tools in the future.

Sixth, users could retweet or reply with bad intentions, such as trolling. For
retweets, we only measured users’ retweets to original tweets created by seed
users. We assume that, those powerful political actors would not take part in
trolling. Users can retweet a seed user’s tweet randomly or for trolling
purposes. Same issue can manifest itself in replies. Since we did not perform
a semantic analysis, this remains a limitation.

##  9 Conclusion and Future Work

In this paper, we have introduced a framework that lists metrics used in the
previous social media analytics studies and added new ones using the theory
from other fields. As one of the outcomes of the results from the previous
section, we showed how minority access is missing from the typical dimensions
of viewpoint diversity studied by computer scientists and the impact it has on
viewpoint diversity analysis. To our knowledge, this is the first work that
provides an overview of all the used metrics in social media analytics
literature and the first study to apply an “openness” metric. Building on this
framework, new studies can be performed for different countries and political
cultures. For instance, Belgium, a country where different languages are
spoken in different regions is an an interesting case to apply our framework.

In the recent months, Turkey experienced several political protests that
spontaneously erupted against the destruction of trees and the building of a
shopping mall at Gezi Park in Taksim Square and large scale corruptions within
the government. Twitter and Facebook played a vital role during these
movements and became the only communication medium when traditional media
performed self-censorship (Dorsey, 2013; Hammond & Angell, 2013; Oktem, 2013).
It would be very useful to see whether the political stance of our observed
users have changed. It is also challenging to identify the opinion leaders
during these movements and find whether they communicate with each other or
form their own “bubbles”. It is further valuable to see if minorities were
able to reach a wider public during those protests. A hashtag based political
communication or an extension of our methodology could bring new insights.

Our study was focused on Twitter and studied whether users have put themselves
in bubbles by following individuals from only one end of the political
spectrum and showed a biased sharing behavior. Twitter itself does not employ
a personalization algorithm in a user’s timeline. However other social
networking platforms, such as Facebook, do use a personalization algorithm and
filter certain information on user’s behalf (Bozdag, 2013). Future studies can
perform black-box testing techniques to determine whether filters used by
these platforms lead to bubbles (See Jürgens (2013)). Creating multiple
profiles while modifying certain factors, such as political affiliation, age,
location, etc. can help us detect bubbles, if they exist.

## Appendix A: List of Minorities

Dutch minorities: Keklik Yucel, SGP, Khadija Arib, Vera Bergkamp, Sadet
Karabulut, Farshad Bashir, Tanja Jadnanansing, Piratenpartij NLD, Partij van
de Dieren, Fatma Koser Kaya, ChristenUnie, Groenlinks, Marianne Thieme, Femke
Halsema.

Turkish minorities: Ayca Soylemez, Evrensel, Aydinlik, Ozgur Gundem, Pinar
Ogunc, Bianet, Sebahat Tuncel, Sol Haber Portali, Halkin Gazetesi Birgun
Yildirim Turker, BDPGenelMerkez, Ufuk Uras, Selahattin Demirtas, Sirri Sureyya
Onder, Sinan Ogan, Hasip Kaplan.

Note that both minorities create about 15% of all tweets produced by seed
users.

## References

  * Acar (2004) Acar, K. (2004).  Globalization And Language: English In Turkey.  Sosyal Bilimler, 2.
  * Agirdir (2010) Agirdir, B. (2010).  Polarization in Politics and the Society.  Technical Report Konda Research Istanbul.
  * Altintas (2003) Altintas, H. (2003).  The Impact of Political Parties and Urbanisation on the Process of Polarisation in the Political System.  Akdeniz İktisadi ve İdari Bilimler Fakültesi Dergisi,, 3, 1–31.
  * An et al. (2012) An, J., Cha, M., & Gummadi, K. (2012).  Visualizing media bias through Twitter.  AAAI Technical Report WS-12-01 The Potential of Social Media Tools and Data for Journalists in the News Media Industry, .
  * An et al. (2011) An, J., Cha, M., Gummadi, K., & Crowcroft, J. (2011).  Media Landscape in Twitter: A World of New Conventions and Political Diversity.  In Proceedings of the Fifth International AAAI Conference on Weblogs and Social Media (pp. 18–25).
  * Aslama et al. (2007) Aslama, M., de Bens, E., van Cuilenburg, J., Nordenstreng, K., Schulz, W., & van der Wurff, R. (2007).  Measuring and assessing empirical media diversity: Some European cases.  In E. de Bens (Ed.), Media between Culture and Commerce (pp. 55–98).  Bristol: Intellect.
  * Bakshy et al. (2012) Bakshy, E., Rosenn, I., Marlow, C., & Adamic, L. (2012).  The role of social networks in information diffusion.  Proceedings of the 21st international conference on World Wide Web, (pp. 519–528). URL: http://dl.acm.org/citation.cfm?id=2187907.
  * Baris (2007) Baris, R. (2007).  The Turkish Media Landscape.  In G. Terzis (Ed.), European Media Governance: National and Regional Dimensions (pp. 289–301).  Intellect Books.
  * Boyd & Crawford (2011) Boyd, D., & Crawford, K. (2011).  Six Provocations for Big Data.  In A Decade in Internet Time: Symposium on the Dynamics of the Internet and Society.  Oxford.  doi:10.2139/ssrn.1926431.
  * Bozdag (2013) Bozdag, E. (2013).  An Algorithmic Gatekeeping Model for the Social Web , Bias in Algorithmic Filtering and Personalization.  Ethics and Information Technology, 15, 209–227.
  * Bruns (2011) Bruns, A. (2011).  Gatekeeping, Gatewatching, Real-time Feedback.  Brazilian Journalism Research, 7, 117–136.
  * Carkoglu & Kalaycioglu (2007) Carkoglu, A., & Kalaycioglu, E. (2007).  Turkish Democracy Today: Elections, Protest and Stability in an Islamic Society.  New York: I.B.Tauris.
  * Castells (2011) Castells, M. (2011).  Communication Power.  Oxford University Press, USA.
  * Cohen (2009) Cohen, J. (2009).  Reflections on Deliberative Democracy.  In T. Christiano, & J. Christman (Eds.), Contemporary Debates in Political Philosophy chapter 14.  West-Sussex, UK: Blackwell.
  * van Cuilenburg (1999) van Cuilenburg, J. (1999).  On Competition, Access and Diversity in Media, Old and New Some Remarks for Communications Policy in the Information Age.  New media & society, 1, 183–207.
  * van Cuilenburg (2002) van Cuilenburg, J. (2002).  The media diversity concept and European Perspectives.  In Media Economics, Content and Diversity Seminar Finnish Academy of Sciences.  Helsinki.
  * van Cuilenburg & McQuail (2003) van Cuilenburg, J., & McQuail, D. (2003).  Media Policy Paradigm Shifts: Towards a New Communications Policy Paradigm.  European Journal of Communication, 18, 181–207.
  * van Cuilenburg & van der Wurff (2007) van Cuilenburg, J., & van der Wurff, R. (2007).  Toward Easy-to-Measure Media Diversity Indicators.  In E. D. Bens (Ed.), Media Between Culture and Commerce (p. 258).  Intellect Books.
  * DellaVigna & Kaplan (2007) DellaVigna, S., & Kaplan, E. (2007).  The Fox News Effect: Media Bias and Voting.  The Quarterly Journal of Economics, .
  * Demir & Ben-Zadok (2007) Demir, T., & Ben-Zadok, E. (2007).  Politically Driven Regulations and Enforcement: Monitoring Kurd and Fundamentalist Broadcasts in Turkey.  Administration & Society, 39, 262–293.
  * van Doorn (1956) van Doorn, J. (1956).  Verzuiling: een eigentijds systeem van social controle.  Sociologische Gids, 3, 41–49.
  * Dorsey (2013) Dorsey, S. (2013).  Turkey’s Social Media And Smartphones Key To ’Occupy Gezi’ Protests.
  * Dryzek (1994) Dryzek, J. S. (1994).  Discursive Democracy: Politics, Policy, and Political Science.  Cambridge University Press.
  * van der Eijk (2000) van der Eijk, C. (2000).  The Netherlands: Media and Politics between Segmented Pluralism and Market Forces.  In Richard Gunther, & Anthony Mughan (Eds.), Democracy and the Media: A Comparative Perspective (p. 510).  Cambridge University Press.
  * Fishkin (1993) Fishkin, J. S. (1993).  Democracy and Deliberation: New Directions for Democratic Reform.  Yale University Press.
  * Glynn et al. (2004) Glynn, C. J., Herbst, S., O’Keefe, G. J., Shapiro, R. Y., & Lindeman, M. (2004).  Public Opinion.  Westview Press.
  * Granovetter (1981) Granovetter, M. S. (1981).  The strength of weak ties : a network theory revisited.  State University of New York, Department of Sociology.
  * Habermas (1975) Habermas, J. (1975).  Legitimation Crisis.  Beacon Press.
  * Hammond & Angell (2013) Hammond, T., & Angell, E. (2013).  Is Everywhere Taksim?: Public Space and Possible Publics.  Jadaliyya, . URL: http://www.jadaliyya.com/pages/index/12143/is-everywhere-taksim_public-space-and-possible-pub.
  * Helberger (2011) Helberger, N. (2011).  Diversity by Design.  Journal of Information Policy, 1. URL: http://jip.vmhost.psu.edu/ojs/index.php/jip/article/viewArticle/59.
  * Held (2006) Held, D. (2006).  Models of Democracy, 3rd Edition.  Stanford University Press.
  * Hindman (2008) Hindman, M. (2008).  The Myth of Digital Democracy.  Princeton University Press.
  * Jürgens (2013) Jürgens, P. (2013).  Measuring Personalization: An Experimental Framework for Testing Technological Black Boxes.  In Society of the Query.  URL: http://networkcultures.org/wpmu/query/tag/pascal-jurgens/.
  * Jürgens et al. (2011) Jürgens, P., Jungherr, A., & Schoen, H. (2011).  Small worlds with a difference: New gatekeepers and the filtering of political information on twitter.  In WebSci ’11.
  * Karppinen (2009) Karppinen, K. (2009).  Rethinking media pluralism and communicative abundance.  Observatorio (OBS*), 3.
  * Karppinen (2013) Karppinen, K. (2013).  Rethinking Media Pluralism.  (onald mcga ed.).  Fordham University Press.
  * Kiris (2011) Kiris, H. M. (2011).  Parti Sisteminde Kutuplasma ve Türk Parti Örnegi.  Amme Idaresi Dergisi, 44, 33–67.
  * Krouwel (2008) Krouwel, A. (2008).  Zo links zijn de kranten niet.  Technical Report Dag.
  * Krouwel (2012) Krouwel, A. (2012).  Regeerakkoord analyse Kieskompas 2012.  URL: http://www.kieskompas.nl/page/Home/1/nl/content.html.
  * Kruijt (1962) Kruijt, J. (1962).  De begrippen verzuiling en ontzuiling.  Socialisme en Democratie, (pp. 791–797).
  * Lasorsa et al. (2012) Lasorsa, D., Lewis, S., & Holton, A. (2012).  Normalizing Twitter-Journalism practice in an emerging communication space.  Journalism Studies, 13, 19–36.
  * Lijphart (1968) Lijphart, A. (1968).  The politics of Accommodation: Pluralism and Democracy in the Netherlands.  University of California Press.
  * Marsman (1967) Marsman, G. (1967).  De katholieke dagbladpers in sociologisch perspectief.  Nimegue: Van Gorcum.
  * McQuail & van Cuilenburg (1983) McQuail, D., & van Cuilenburg, J. J. (1983).  Diversity as a Media Policy Goal: a Strategy for Evaluative Research and a Netherlands Case Study.  International Communication Gazette, 31, 145–162.
  * Mediamonitor (2013) Mediamonitor (2013).  The concept of pluralism: media diversity.  URL: http://www.mediamonitor.nl/english/the-concept-of-pluralism-media-diversity/.
  * Müftüler Baç (2005) Müftüler Baç, M. (2005).  Turkey’s Political Reforms and the Impact of the European Union.  South European Society and Politics, 10, 17–31.
  * Muftuler-Bas & Keyman (2012) Muftuler-Bas, M., & Keyman, F. E. (2012).  Turkey Under the AKP.  Journal of Democracy, 23, 85–99.
  * Munson & Resnick (2010) Munson, S., & Resnick, P. (2010).  Presenting diverse political opinions: how and how much.  In Proceedings of the SIGCHI Conference on Human Factors in Computing Systems, CHI 2010 (pp. 1457–1466).  New York, New York, USA.
  * Munson et al. (2013) Munson, S. A., Lee, S. Y., & Resnick, P. (2013).  Encouraging Reading of Diverse Political Viewpoints with a Browser Widget.  In International Conference on Weblogs and Social Media (ICWSM).  Boston.
  * Napoli (1999) Napoli, P. (1999).  Deconstructing the diversity principle.  Journal of Communication, 49, 7–34. doi:10.1111/j.1460-2466.1999.tb02815.x.
  * Offe & Preuss (1990) Offe, C., & Preuss, U. K. (1990).  Democratic Institutions and Moral Resources.
  * Oktem (2013) Oktem, K. (2013).  Why Turkey’s mainstream media chose to show penguins rather than protests.  Guardian, . URL: http://www.theguardian.com/commentisfree/2013/jun/09/turkey-mainstream-media-penguins-protests.
  * Ozturk (2013) Ozturk, M. (2013).  Agirdir: Bugunku kutuplasma 1970’lerdekinden daha vahim.  Aksiyon, . URL: http://t24.com.tr/haber/agirdir-bugunku-kutuplasma-1970lerdekinden-daha-vahim/241939.
  * Pariser (2011) Pariser, E. (2011).  The Filter Bubble: What the Internet Is Hiding from You.  New York: Penguin Press.
  * Peter van der Hoek (2000) Peter van der Hoek, M. (2000).  Does the dutch model really exist?  International Advances in Economic Research, 6, 387–403. doi:10.1007/BF02294959.
  * Pew Research (2012) Pew Research (2012).  Bias Perceived in News Coverage.  URL: http://www.pewresearch.org/daily-number/bias-perceived-in-news-coverage/.
  * Post (1989) Post, H. (1989).  Pillarization: an analysis of Dutch and Belgian society.  Vermont: Gower Publishing Company.
  * Saez-Trumper et al. (2013) Saez-Trumper, D., Castillo, C., & Lalmas, M. (2013).  Social media news communities: gatekeeping, coverage, and statement bias.  In Proceedings of the 22nd ACM international conference on Conference on information & knowledge management (pp. 1679–1684).  New York, NY, USA: ACM.
  * Shannon (1948) Shannon, C. (1948).  A Mathematical Theory of Communication.  Bell System Technical Journal, 27, 379 – 423.
  * Steininger (1977) Steininger, R. (1977).  Pillarization and Political Parties.  Sociologische Gids, (pp. 242–257).
  * Sun et al. (2013) Sun, T., Zhang, M., & Mei, Q. (2013).  Unexpected Relevance: An Empirical Study of Serendipity in Retweets.  In Seventh International AAAI Conference on Weblogs and Social Media (ICWSM’13).
  * Sunstein (2002) Sunstein, C. R. (2002).  Republic.com.  Princeton University Press.
  * Suoranta & Vadén (2009) Suoranta, J., & Vadén, T. (2009).  Wikiworld.  Pluto Press.
  * Tessler & Altinoglu (2004) Tessler, M., & Altinoglu, E. (2004).  Political culture in Turkey: Connections among attitudes toward democracy, the military and Islam.  Democratization, 11, 22–51.
  * Trendlight (2012) Trendlight (2012).  AD is de krant voor Henk en Ingrid — TrendLight.  URL: http://trendlight.eu/ad-is-de-krant-voor-henk-en-ingrid/.
  * Unver (2011) Unver, A. (2011).  Clash of Communities: Turkey’s Dormant Domestic Cold War.  Political Reflection Quarterly Journal 1, 2.
  * Vink (2007) Vink, M. (2007).  Dutch “Multiculturalism” Beyond the Pillarisation Myth.  Political Studies Review, 5, 337–350.
  * Wei et al. (2013) Wei, Z., He, Y., Gao, W., Li, B., Zhou, L., & Wong, K. (2013).  Mainstream media behavior analysis on Twitter: a case study on UK general election.  Labour, (pp. 174–178).
  * Yilmaz et al. (2012) Yilmaz, C., Aygoren, O., & Ozdemir, O. (2012).  Türkiye’de siyasi kutuplasmayi olusturan unsurlar: Secmen tercihlerinde ekonomik oy verme davranisindan toplumsal travma kuramina kadar bir dizi etkenin görece etkileri.  Iktisat Isletme ve Finans, 27, 09–39.
  * Young (2002) Young, I. M. (2002).  Inclusion and Democracy (Oxford Political Theory).  Oxford University Press, USA.


# DMTCP Checkpoint/Restart of MPI Programs via Proxies

Gregory Price

###### Abstract

MPI accomplishes portable, standardized message-passing between processes by
exposing a standard API that hides the implementation of the underlying
mechanism for message passing. Until now, checkpointing an MPI program
required knowledge of these underlying mechanisms. Through the addition of a
proxy, we demonstrate that MPI programs can be checkpointed and restarted
regardless of the MPI implementation utilized. Further, proxies may enable MPI
programs to be checkpointed on one MPI implementation, and restarted on
another.

##  1 Introduction

This work has largely been produced using examples from other DMTCP plugins
from the core dmtcp library [1]. It was implemented and tested against MPICH
[2], with examples from mpitutorial.com [3].

MPI provides the HPC community a standardized way to pass messages between
processes in parallel computation. Each MPI implementation provides an opaque,
standardized interface to send and receive messages; however, each
implementation may differ in system requirements and mechanisms used to
actually pass the message (i.e., sockets, pipes, shared memory, etc.).

This provides a challenge when checkpoint/restart is desired. Classically,
checkpoint/restart is achieved via virtualization of various resources (such
as file descriptors, sockets, shared memory, etc.), which are used
transparently by the user application. Checkpoint/restart software must then
provide support for each potential implementation of MPI; agnosticism is a
pipe dream in this reality.

Through the use of MPI proxies and a DMTCP Plugin, we present a solution that
provides simple, portable, transparent, and agnostic Checkpoint/Restart. Our
implementation requires no changes to application code, and can be deployed on
any architecture machine that supports DMTCP.

Our strategy makes one, key, novel observation about how to checkpoint MPI:
don’t. By interposing on the MPI library, and moving all MPI procedures into a
separate process, we only need to checkpoint a single, ephemeral interface
between the application program and the proxy. Then, we simply, transparently
translate MPI calls into calls to our MPI proxy.

Restart presents a separate problem, one which is fairly understood within the
realm of checkpoint/restart: How do we handle messages in flight? There are
two options for managing the situation.

  1. 1.

Log and replay of messages on restart

  2. 2.

Drain the network of in-flight messages, and cache them for restart

Logging many messages requires additional (potentially significant) overhead
throughout the lifetime of the computation, while draining the network only
incurs a cost at the time of checkpoint. We employ network draining as our
mechanism of choice, under the assumption that a one-time cost during
checkpoint is more tolerable to HPC professionals. This overhead can also be
easily controlled through changing how often a checkpoint is created.

Finally, we believe solving this checkpoint/restart issue for MPI via the
combination of library interposition (via DMTCP plugins) and a proxy provides
a compelling case for using this strategy elsewhere. Other technologies that
provide a standardized interface to an external service can also benefit from
checkpoint/restart via the use of proxies.

##  2 Background

MPI provides two main services:

  1. 1.

Creation of processes (called ranks) for distributed computation

  2. 2.

Message passing between these ranks

A coordinator is used for exchanging information about rank identification,
how to build direct communication paths between ranks, and what groups have
been created for publish-subscribe style communications (Broadcasting to a
subset of ranks). It is useful to visualize this relationship in Figure 1, as
one possible implementation of an MPI coordinator.

Figure 1: The communication structure of a simple MPI application

From here, we can see that there is a parent-child process relationship. The
Coordinator is a parent to both Rank 0 and Rank 1. Also, there are at least
three “communication interfaces” created between these three programs:
Interfaces between each rank and the coordinator, and between each rank.

Each rank has a separate “MPI Library”, which is used by the application
program to interact with the coordinator and other ranks. We call this an
Active MPI Library. An “Active” library is one whose functions are loaded and
executed, whereas a “passive” library is one whose functions may be referenced
but never used.

Figure 2: MPI library in User Application

Prior work on checkpointing MPI programs had been built on the assumption that
all external resources within the entire system must be virtualized. We can
visualize this with a “checkpoint boundary” for an arbitrary MPI computation.

Figure 3: The checkpoint boundary of existing solutions

Each relationship and communication mechanism within the checkpoint boundary
will increase the complexity of a successful checkpoint/restart approach as
the number of potential communication mechanisms used by MPI implementations
increases. (The “checkpoint boundary” encapsulates all the resources that must
be recorded and/or restored upon restart.)

##  3 Our Approach

We seek to avoid checkpointing any implementation-specific MPI code, and
instead create a network with a checkpoint boundary that looks as follows:

Figure 4: Our Checkpoint Boundary

This reduces the total number of interfaces needed to be virtualized, and
completely segregates the MPI implementation from the user application. The
MPI library used by the user application is no longer “active”, and is
replaced by a DMTCP plugin that provides the virtual interface to the “active”
library made available by the proxy.

##  4 challenges

By introducing an external proxy, a one-step message-passing mechanism is
broken into two steps. This introduces a temporal issue that was not present
in other checkpointing solutions. We must deal with two challenges that must
be solved in order to successfully checkpoint and restart:

  1. 1.

In-flight Data. Rank 0 sends Rank 1 a message, the message has been received
by Proxy 0, but has not been transferred to Proxy 1 or Rank 1. A checkpoint is
initiated, leaving the message stuck “in flight”.

  2. 2.

Proxy State. Rank 0 makes a call to receive a message, but Rank 1 has not sent
the message yet. Rank 0 must re-issue the call in order to receive on restart.

In both cases, we recognize that a potential for failure occurs when the proxy
has a “state” that is not being recorded due to our chosen checkpoint
boundary. We revert to the two basic techniques discussed previously (Drain,
and Log-and-Replay) to solve these in the simplest way possible.

For “in-flight data”, we drain the network of all in-flight messages, and
cache this information in the receiving processes’ memory space. In order to
know when all in-flight data has been successfully drained, we utilize the
DMTCP coordinator to share the number of messages that each rank has sent and
received. When draining, we continue to increment these numbers. When the
numbers are equivalent, we know that there is no longer any data in flight,
and thus we can move forward with checkpointing. This heuristic was previously
used in the PhD thesis of Jiajun Cao for checkpointing over InfiniBand [5].

For Proxy-State replication, we break messages into two basic categories:

  1. 1.

Administrative messages

  2. 2.

Message actions

Administrative messages are messages between the rank and the MPI coordinator
to either retrieve information about the current configuration of the MPI
ranks, or to create new configurations (such as new groups of ranks).
Configuration messages must be replayed to ensure that the “Active” MPI
library in the proxy is in the same state as when we initiated the checkpoint.

Message actions are calls such as MPI_IProbe and MPI_Recv, where the user
application is trying to retrieve either data, or information about the state
of the network. For example, MPI_IProbe is used to check if there is a message
waiting to be received. The relevant information for these message actions
must be cached at checkpoint time. The simplest example is the drain mechanism
for in-flight data — these messages are cached for use on restart. A call to
MPI_Recv must check the cache for a buffered message before checking the proxy
for a buffered message.

##  5 API Support

Currently only the following calls to the MPI interface are fully supported.

  * •

MPI_Init

  * •

MPI_Finalize

  * •

MPI_Comm_size

  * •

MPI_Comm_rank

  * •

MPI_Type_size

  * •

MPI_Send

  * •

MPI_Recv

  * •

MPI_Probe

  * •

MPI_Iprobe

  * •

MPI_Get_count

Future work will expand support to the following API calls:

  * •

Non-blocking Asynchronous Send / Receive

    * –

MPI_Isend

    * –

MPI_Irecv

    * –

MPI_Test

  * •

Collective Communication

    * –

MPI_Bcast

    * –

MPI_Barrier

    * –

MPI_Scatter

    * –

MPI_Gather

    * –

MPI_Allgather

    * –

MPI_Reduce

    * –

MPI_Allreduce

  * •

Communicators and Groups

    * –

MPI_Comm_group

    * –

MPI_Group_incl

    * –

MPI_Comm_create_group

    * –

MPI_Group_free

    * –

MPI_Comm_free

##  6 Current State

Presently we support checkpointing of simple Send/Receive programs that use
any of the supported API calls. We do not currently support asynchronous send
and receive (i.e., non-blocking sends and receives). Trivially, we support
non-blocking receives by supporting MPI_Iprobe, which allows user applications
to determine if a message is waiting to be received. However, support of
MPI_Isend will require the caching of additional data (not yet implemented).

With these basic building blocks supported, we can confidently say that the
remaining API calls can also be supported — it is now a simple matter of
plumbing, and no longer a question of feasibility.

##  7 Future Work

Along with adding full API support, we also intend to explore the idea of
checkpointing an application on one MPI implementation, and then restarting
the application on a different MPI implementation. This will be accomplished
via two core mechanisms. First, we will provide a virtualized interface to MPI
specific resources, such as an MPI_Request id or MPI_Status structure. Second,
we will provide a new layer to handle translations between the Passive MPI
Library in the MPI Plugin and the Active MPI Library in the MPI Proxy. We
believe this approach will make it possible to, for example, checkpoint a
program built and run on MPICH, and restart the same program on an OMPI
implementation without recompilation.

##  8 Acknowledgements

Gene Cooperman and Rohan Garg (Northeastern University) provided guidance on
DMTCP plugin creation, and in the design of the proxy architecture. The
initial idea for the proxy architecture was developed by Rohan Garg, Gene
Cooperman and Jiajun Cao [5] during the development of other
checkpoint/restart technologies. This report and project reinforces their
claim that a proxy architecture for checkpointing can be applicable much more
generally.

The members of the High Performance Computing Lab at Northeastern University
provided feedback and support during the development of this project.

## References

  * [1] DMTCP. http://github.com/dmtcp/dmtcp.
  * [2] MPICH. http://www.mpich.org.
  * [3] MPI Tutorials. http://mpitutorial.com
  * [4] Douglas Thain, Miron Livny: "Multiple Bypass: Interposition Agents for Distributed Computing. Cluster Computing" 4(1): 39-47 (2001)
  * [5] Jiajun Cao, Transparent Checkpointing over RDMA-based Networks, PhD thesis, 2017