# A Taxonomy of Knowledge Gaps for Wikimedia Projects (Second Draft)

Miriam Redi  Wikimedia Foundation ,  Martin Gerlach  Wikimedia Foundation ,
Isaac Johnson  Wikimedia Foundation ,  Jonathan Morgan  Wikimedia Foundation
and  Leila Zia  Wikimedia Foundation

††copyright: none

## Executive Summary

In January 2019, prompted by the Wikimedia Movement’s 2030 strategic direction
(strategy, 108), the Research team at the Wikimedia
Foundation111https://research.wikimedia.org/team.html identified the need to
develop a _knowledge gaps index_ —a composite index to support the decision
makers across the Wikimedia movement by providing: a framework to encourage
structured and targeted brainstorming discussions; data on the state of the
knowledge gaps across the Wikimedia projects that can inform decision making
and assist with measuring the long term impact of large scale initiatives in
the Movement.

After its first release in July 2020, the Research team has developed the
second complete draft of a taxonomy of knowledge gaps for the Wikimedia
projects, as the first step towards building the knowledge gap index. We
studied more than 250 references by scholars, researchers, practitioners,
community members and affiliates—exposing evidence of knowledge gaps in
readership, contributorship, and content of Wikimedia projects. We elaborated
the findings and compiled the taxonomy of knowledge gaps in this paper, where
we describe, group and classify knowledge gaps into a structured framework.
The taxonomy that you will learn more about in the rest of this work will
serve as a basis to operationalize and quantify knowledge equity, one of the
two 2030 strategic directions, through the knowledge gaps index.

We hope you enjoy the read and join the conversation about how we can improve
the taxonomy of knowledge gaps for the Wikimedia projects. If you have any
suggestion or feedback, please reach out to us via the Knowledge Gaps Index
Meta
page.222https://meta.wikimedia.org/wiki/Research:Knowledge_Gaps_Index/Taxonomy

###### Contents

  1. 1 Introduction
  2. 2 Methodology
    1. 2.1 Guiding Principles
    2. 2.2 Sources
    3. 2.3 Taxonomy Structure
    4. 2.4 Root Dimensions: Readers, Contributors, and Content
      1. 3 Readers
        1. 3.1 Representation Gaps
          1. 3.1.1 Gender
          2. 3.1.2 Age
          3. 3.1.3 Geography
          4. 3.1.4 Language
          5. 3.1.5 Socioeconomic Status
          6. 3.1.6 Sexual Orientation
          7. 3.1.7 Cultural Background
        2. 3.2 Interaction Gaps
          1. 3.2.1 Motivation
          2. 3.2.2 Information Depth
          3. 3.2.3 Familiarity
          4. 3.2.4 Tech Skills
          5. 3.2.5 Disabilities
        3. 3.3 Barriers
          1. 3.3.1 Censorship
          2. 3.3.2 Internet Connectivity and Cost
          3. 3.3.3 Device
          4. 3.3.4 Interface Complexity
          5. 3.3.5 Text Readability
            1. 4 Contributors
              1. 4.1 Representation Gaps
                1. 4.1.1 Gender
                2. 4.1.2 Age
                3. 4.1.3 Geography
                4. 4.1.4 Language
                5. 4.1.5 Socioeconomic Status
                6. 4.1.6 Sexual Orientation
                7. 4.1.7 Cultural Background
              2. 4.2 Interaction Gaps
                1. 4.2.1 Motivation
                2. 4.2.2 Role
                3. 4.2.3 Tech Skills
                4. 4.2.4 Disabilities
              3. 4.3 Barriers
                1. 4.3.1 Censorship
                2. 4.3.2 Internet Connectivity
                3. 4.3.3 Device
                4. 4.3.4 Interface Complexity
                5. 4.3.5 Privacy
                6. 4.3.6 Community Health
                  1. 5 Content
                    1. 5.1 Representation Gaps
                      1. 5.1.1 Gender
                      2. 5.1.2 Age (Recency)
                      3. 5.1.3 Geography
                      4. 5.1.4 Language
                      5. 5.1.5 Socioeconomic Status
                      6. 5.1.6 Sexual Orientation
                      7. 5.1.7 Important Topics
                      8. 5.1.8 Cultural Background
                    2. 5.2 Interaction Gaps
                      1. 5.2.1 Multimedia
                      2. 5.2.2 Structured Data
                      3. 5.2.3 Readability
                    3. 5.3 Barriers
                      1. 5.3.1 Verifiability
                      2. 5.3.2 Neutrality
                      3. 5.3.3 Notability
                      4. 5.3.4 Free Knowledge
                        1. 6 How to Use this Taxonomy
                          1. 7 Future Work
                            1. 7.1 Metrics
                            2. 7.2 Knowledge Gap Index
                            3. 7.3 Interactions between Gaps
                            4. 7.4 Causes and Interventions
                            5. 7.5 Expanding the Taxonomy
                              1. 8 Acknowledgments

##  1\. Introduction

With almost 56 million articles written by roughly 500,000 monthly editors
across more than 160 actively edited languages, Wikipedia is the most
important source of encyclopedic knowledge and one of the most important
knowledge resources available on the internet. Every month, the project
attracts users on more than 1.5 billion unique devices from across the globe,
for a total of more than 15 billion monthly pageviews (stats2, 203).

While global and massively popular resources, Wikipedia and its sister
projects such as Wikimedia Commons and Wikidata suffer from a wide range of
knowledge gaps, which we define as disparities in content coverage or
participation of a specific group of readers or contributors.

A typical example of a knowledge gap is the gender gap, one of the most well-
studied gaps in the Wikiverse. Researchers and practitioners have investigated
the gender gap by measuring the representation of different gender groups in
content, readers, and contributors, and found, for example, that less than 20%
of the biographies in Wikipedia are about women (whgi, 246), roughly 75% of
readers in Wikipedia are men (johnson2020global, 183), and that this disparity
becomes even more extreme when analyzing editors’ gender distribution
(2020communityinsights, 119).

Beyond the gender gap, only a handful of works have studied ways to quantify
or address other kinds of knowledge gaps—for example by analyzing readers’
(lemmerich2019world, 150) and editors’ (cho2010testing, 16) motivations for
accessing the site, studying the role of technical skills and awareness in
Wikimedia contributorship (menking2015heart, 67, 66) and readership
(shaw2018pipeline, 129), designing algorithms to identify and prioritize
missing content in different languages (wulczyn2016growing, 88), or proposing
question-and-answer facilities to satisfy readers’ information needs
(chhabra2016should, 75).

While these works made substantial progress towards measuring and identifying
directions for bridging individual gaps, they did not provide a holistic and
systematic framework to understand the scope of knowledge gaps in Wikimedia
and the relationships between them.

To see why having such a comprehensive map of the space of knowledge gaps
could be key for Wikimedia projects, consider the main directions identified
as part of the Movement Strategy (strategy, 108): by 2030, Wikimedia will
become the essential infrastructure of the ecosystem of free knowledge, and
achieve knowledge equity by focusing efforts on including all “knowledge and
communities that have been left out by structures of power and privilege”. To
support this goal, and measure the progress towards knowledge equity, the
Research team at the Wikimedia Foundation has identified the need to
operationalize knowledge equity into a _knowledge gaps index_ – a composite
index tracking the collective evolution of knowledge gaps in Wikimedia333While
it is hard to make the claim that understanding gaps is the most effective way
to understand inequalities, what we observe is that understanding gaps is a
way to spark conversations and motivate/mobilize towards reaching knowledge
equity. However, we acknowledge that inequalities are usually highly complex
and the abstraction level that gaps introduce can over-simplify the deep
understandings one can gain through ethnographic research and focus too much
on deficits and not enough on the success of various initiatives, for
example..

While extremely important, operationalizing knowledge equity by studying and
measuring its underlying factors is not a trivial task. The first step towards
this goal is to generate a systematic picture of those Wikimedia audiences,
groups, and cultures that could be underrepresented in terms of participation,
representation, and coverage.

Therefore, in this work we propose the first taxonomy of knowledge
gaps444Following strict definitions (smith2002typologies, 2), the final
product should be defined as a typology rather than a taxonomy, as
“[t]axonomies differ from typologies in that they classify items on the basis
of empirically observable and measurable characteristics”
(smith2002typologies, 2), while typologies group concepts according to
characteristics which are less tangible and might not exist in physical
reality. However, given the widespread usage of the term “taxonomy” to
classify objects according to abstract properties—e.g., emotions, clustering
algorithms, or educational objectives (tamir2016people, 84, 51, 1)—we will use
this term to define the final product of our work throughout the manuscript.
in the context of Wikimedia projects. We identify three macro-dimensions of
the Wikimedia ecosystem as the roots of the taxonomy: Readers, Contributors,
and Content. We then review studies and discussions from scholars,
researchers, practitioners and community members, and compile for each of
these dimensions a list of knowledge gaps as areas of the Wikiverse where we
found systematic evidence of inequality, together with potential barriers
which prevent us from addressing these knowledge gaps. Finally, we group these
gaps into facets containing knowledge gaps related to Representation or
Interaction aspects. The final three-layer taxonomy is illustrated in Figure
1.

Figure 1. The Knowledge Gaps Taxonomy

This taxonomy provides a theoretical framework based on which we can develop
different approaches to address knowledge gaps and represents a first step
towards the full operationalization of knowledge equity. With this taxonomy,
we aim at fostering conversations across the Wikimedia and academic
communities around the nature and composition of the “content and communities
that have been left out by structures of power and privilege” (strategy, 108).
Future work will include the development of robust metrics and indicators to
quantify gaps, which could allow to track progress towards knowledge equity
(Commission2008handbook, 9). Quantifying gaps will facilitate the
conceptualization of the interdependence between different gaps in order to
understand potential causes and barriers, and to design more effective
interventions.

In the remainder of this paper, we will first look at the methodology used to
compile the taxonomy (Section 2), then present each macro branch of the
taxonomy: Readers (Section 3), Contributors (Section 4), and Content (Section
5). We will then provide some suggestions on how different audiences can use
and benefit from this taxonomy (Section 6), and finally expose interesting
directions for future work (Section 7).

##  2\. Methodology

The Taxonomy of Knowledge Gaps emerges as the result of hours of literature
review, survey analysis, in-depth movement strategy reading, research
brainstorming and Wikimedia Foundation-wide discussions. To help understand
the methodology we used to compile this taxonomy, in this Section we explain
its primary guiding principles and structural characteristics.

###  2.1. Guiding Principles

Our task is to collect elements of Wikimedia spaces lacking knowledge equity
in a taxonomy of knowledge gaps. There are endless ways in which we could
bring order to this large pool of unstructured pieces of information. To help
in narrowing down the scope while delivering a consistent final product, we
let different themes, principles and values guide the selection of the
taxonomy classes and levels.

Driven by communities.:

The taxonomy is largely inspired by values and principles of the Wikimedia
communities and the latest recommendations of the Movement Strategy (strategy,
108). At the same time, the taxonomy is grounded in the literature produced by
the broader academic and scientific communities who study Wikimedia space from
a computational and sociological perspective (for an overview, see reviews
(Okoli2012review, 38, 37, 68) or the Wikimedia Research Newsletter (wrn,
276)).

Neutrality.:

Similar to Bloom’s taxonomy of educational objectives (krathwohl2009taxonomy,
227), one of the foundational principles of this taxonomy is its neutrality.
While defining gaps is an inherently value-laden process, our mission is to
define a classification of knowledge gaps, and identify some of the barriers
preventing people from accessing free knowledge. While doing so, we aim at
being as impartial and inclusive as possible, without expressing value
judgments on the importance of one gap or source over another.

Flexibility.:

Like any project in Wikimedia spaces, we wanted this taxonomy to be
“editable”. This is the second version of the taxonomy of knowledge gaps. We
made the first version of the taxonomy available to the community and opened
it up for feedback from researchers and community members. After one month of
consultation, we worked on summarizing and incorporating the precious
suggestions we received 555For a summary of changes, see here:
https://meta.wikimedia.org/wiki/Research:Knowledge_Gaps_Index/Taxonomy/Summary_of_Changes_for_Second_Version,
which are now embedded in this second version. Also, its inherent structure
allows the taxonomy to evolve and be expanded by experts and community members
who want to contribute to the improvement of its content.

Measurability.:

One of the end goals of this taxonomy is to help Wikimedia communities measure
the impact of their initiatives and content creation. With this in mind, when
possible, we explicitly formulated gaps and objectives to incorporate elements
that are quantifiable in a globally-consistent manner via surveys, large-scale
data analysis or other computational methods.

###  2.2. Sources

To understand relevant components of the taxonomy of knowledge gaps and make
decisions about which aspects of Wikimedia ecosystems to include in the
taxonomy, we gathered information from different sources describing or
discussing inequalities among readers, contributors, and content in Wikimedia
spaces. We explain here our main data sources and the rationale behind
choosing each of them.

Academic Literature.:

We tap into ideas from several fields of academic literature that study
knowledge gaps in Wikimedia. The majority of this research belongs to the
broad field of “computational social science” and tries to characterize and
quantify different aspects of Wikimedia communities using a computational
approach. For example, researchers have studied gender gaps across different
dimensions (Hinnosaar2019newmedia, 147, 124), quantified the usage of visual
content across different languages (He2018babel, 121) or estimated the
geographical bias of Wikipedia content (Beytia2020positioning, 168). Research
aiming to bridge knowledge gaps using tools from recommender systems and
natural language processing also largely helped the creation of this taxonomy:
these include methods to grow Wikipedia languages via recommendations
(wulczyn2016growing, 88), as well as machine learning models to score
readability of Wikipedia articles (brezar12019readability, 138), or discover
sentences needing citations (Redi2019citationneeded, 158).

Community Surveys.:

Throughout the years, the Wikimedia Foundation, Wikimedia Affiliates, and
independent organizations have run surveys to characterize Wikimedia readers
and editors. We tap into the questions asked in these surveys to define some
of the gaps and facets in our taxonomy, for example many of our
representational gaps. Most of these surveys are referenced throughout this
manuscript, and a subset of them can be found by checking the appropriate
categories on
meta.wikimedia.org.666https://meta.wikimedia.org/wiki/Category:Surveys;
https://meta.wikimedia.org/wiki/Category:Reader_surveys;
https://meta.wikimedia.org/wiki/Category:Editor_surveys

Movement Strategy.:

The third source of content for this taxonomy is a set of guidelines from the
Wikimedia Movement Strategy. Such guidelines include broad strategic
directions, namely Knowledge Equity and Knowledge as a Service (strategy,
108), as well as more specific recommendations to implement such strategic
directions (uxstrategy, 191, 189). We also borrow from the Wikimedia
Foundation’s medium-term-plan priorities (mtp, 143). Collectively, these
guidelines helped us to identify gaps, barriers, and common objectives related
to free knowledge.

Community Initiatives.:

Throughout the years, the Wikimedia community have worked on many initiatives
and discussions aiming to address knowledge gaps. These not only include
research (e.g. trying to conceptualize or measure the gap) but also cover a
wide variety of initiatives such as WikiProjects (e.g,. WikiProject Women
(wikiprojectwomen, 274)) or campaigns (e.g., Wiki Loves Monuments
(wikilovesmonuments, 272)).

###  2.3. Taxonomy Structure

Beyond the identification of potential gaps within the Wikimedia ecosystem,
one of the main challenges in building a taxonomy of knowledge gaps is to
provide a structure for how to analyze and classify the different gaps. In
this section we give an overview over the overall structure, and define and
motivate the different levels in the taxonomy.

Gap.:

A gap corresponds to an individual aspect of the Wikimedia ecosystem—for
example readers’ gender, or images in content—for which we found evidence of a
lack of diversity, or imbalanced coverage across its inner categories (for
example, proportion of readers who identify as men, women or non-binary in the
case of the reader gender gap). For each gap, we include the following
characterizing fields:

  * •:

Description: This field is the definition of the gap, and describes which type
of disparity is covered by the gap. For example, the reader gender gap is
defined as “the difference between readers of different genders in how and how
much they access the sites”.

  * •:

Sources: The decision to include a gap in the taxonomy is heavily driven by
existing sources. The source property of a gap includes all the references we
examined when characterising and describing a given gap, grouped by source
type (literature, surveys, strategy and/or community).

Facet.:

We grouped semantically-related gaps into facets, namely macro-categories
describing the general semantics of a gap. For example, gaps such as “gender”
or “age” both belong to the “Representation” facet.

Dimension.:

We identify three foundational dimensions as root nodes in our taxonomy
grouping different facets: Wikimedia readers, Wikimedia contributors, and
Wikimedia content. We will expand the rationale behind the choice of the
dimensions in the next Subsection.

Barriers.:

Defining an exhaustive list of gaps’ potential causes—i.e. the technical,
social, and economic barriers preventing the Wikimedia ecosystem from closing
knowledge gaps and achieving greater diversity—lie beyond the scope of this
manuscript. Still, throughout our literature review of knowledge gaps in
Wikipedia and sister projects, we found evidence of elements which could
potentially amplify or cause inequalities, and describe them, for each
dimension, in a Barriers section.

###  2.4. Root Dimensions: Readers, Contributors, and Content

This taxonomy is developed along three dimensions representing macro-elements
of the Wikimedia ecosystem: Readers, Contributors, and Content. The choice of
these dimensions is broadly inspired by two models summarizing the inner
mechanisms of Wikimedia communities.

A model proposed by Wikimedia Foundation as part of their medium-term plan
(mtp, 143) for how to build engagement across the Movement sketches a diagram
in the form of a flywheel describing the relationship between awareness,
consumption, contributors, content, and advocacy, as shown in Figure 2(a). The
model shows how people initially engage with the site (i.e. they become
Wikimedia readers) through spontaneous search or awareness campaigns. Some
readers then might become more involved in the values or the mission of the
Wikimedia movement, thus potentially converting into contributors or donors.
When more people engage with the site, the content production becomes greater
and more diverse, thus making the site more inclusive for new readers, and the
cycle begins again.

A complementary model by Shaw and Hargittai (shaw2018pipeline, 129) (“The
Pipeline of Online Participation Inequalities: The Case of Wikipedia Editing”)
proposes a framework in the form of a pipeline to describe how gaps form at
different levels of engagement with Wikipedia (see Figure 2(b)). Specifically,
they propose a similar series of steps through which a user must go to become
a contributor: being an internet user, having heard of the site (awareness of
Wikipedia), having visited the site (Wikipedia reader), knowing it’s possible
to contribute to the site (awareness they can edit), and then finally having
contributed (is a contributor). At each of these steps, they identify
structural barriers to participation that make different groups of people more
or less likely to drop off in participation. As a result, gaps that were
perhaps non-existent or small at the earlier step of awareness become much
larger at the later step of contribution.

The research cited within this taxonomy bears out the interconnectedness of
reader, contributor, and content gaps as described by these models. Together,
these two models demonstrate both the importance of initiatives that seek to
expand the diversity of the reader population, contributor population, and
content, and, the need to address barriers to awareness and participation at
each step of joining the Wikimedia Movement.

(a) Margeigh Novotny / CC BY-SA

(b) Aaronshaw / CC BY-SA

Figure 2. Sketch of the two models on the complex dynamics in Wikimedia
communities giving rise to the three root dimensions (readers, contributors,
content) in the taxonomy of knowledge gaps.

For each dimension of this Knowledge Gaps Taxonomy—Readership (Section 3),
Contributorship (Section 4) and, Content (Section 5)—we present a short
overview of its meaning and the corresponding taxonomy in a tabular form. For
each of the dimension’s facet, we provide a detailed explanation for the
rationale behind it.

Facet |  Gap |  Description |  Sources
---|---|---|---
Representation  |  |  |
Gender |  Difference between readers of different gender identities in how and how much they access the sites. |  literature (shaw2018pipeline, 129, 83, 147), surveys (2014globalsouthsurvey, 52, 62, 78, 99, 141, 31, 57, 85, 19), strategy (uxstrategy, 191, 108)
Age |  Difference between readers of different age in how and how much they access the sites. |  literature (shaw2018pipeline, 129, 83, 147), surveys (2014globalsouthsurvey, 52, 62, 78, 99, 141, 31, 57, 85, 19), strategy (uxstrategy, 191, 108, 184), community (simplewikiabout, 266, 247, 209)
Geography |  Differences in readership between different areas of the world |  literature (shaw2018pipeline, 129), surveys (2019survey, 141, 78, 99), community (wikiconnect, 249, 144)
Language |  Differences in readership depending on readers’ ability to read one or more languages |  surveys (2019survey, 141, 85, 52, 21), strategy (strategy, 108, 191), community (scribe, 237, 221)
Socioeconomic Status |  Differences in readership depending on readers’ education, income, wealth, or employment status |  literature (shaw2018pipeline, 129, 147) surveys (2007pewsurvey, 6, 141, 57, 85, 52, 11, 19, 31, 21, 22), community (wikiversity, 270, 251)
|  Sexual Orientation |  Difference between readers of different sexual orientations in how and how much they access the sites. |  literature (wexelbaum2015queering, 73)
|  Cultural Background |  Differences in readership among people with different ethnic, political, and religious backgrounds |  community (lgbtportal, 236, 258)
Interaction  |  |  |
Motivation |  Differences in readership depending on the reason behind readers’ visit to the site |  literature (lemmerich2019world, 150, 98), surveys (2019survey, 141)
Information Depth |  Differences in readership depending on the depth of information for which a reader is looking |  literature (lemmerich2019world, 150, 113), surveys (2019survey, 141, 146)
Familiarity |  Differences in readership depending on one’s prior familiarity with a topic |  literature (lemmerich2019world, 150, 98), surveys (2019survey, 141), community (breadthdepth, 232)
Tech Skills |  Differences in readership depending on readers’ general internet skill |  literature (shaw2018pipeline, 129), strategy (uxstrategy, 191), other (mozillaskills, 77)
|  Disabilities |  Disparities in ability to access the knowledge within Wikipedia depedning on individual disabilities |  literature (webaimscreenreadersurvey, 15), community (grahampierce, 114, 49, 209, 240, 211, 210, 268, 241)

##  3\. Readers

The readership dimension of knowledge gaps encompasses all those gaps related
to readers’ access to Wikimedia sites. We identify different areas where
readers are under-represented according to literature, surveys, and community
strategic directions and organize them around two main facets: representation
gaps and interaction gaps. We also highlight different initiatives aiming at
closing each gap. We define readers as all users who connect to the site to
consume Wikimedia content. While there exists a body of research
(alshomary2019wikipedia, 137, 104, 159, 162, 133, 117) studying how content
consumption happens outside of Wikimedia—e.g., voice assistants, search
engines, or third-party apps—we scope down our definition of readership to
readers who come directly to the projects to access content.

A large source of inspiration for this dimension is the 2020 Wikimedia
Movement Strategy, which recommends that Wikimedia platforms should be
designed to “enable everyone—irrespective of gender, culture, age,
technological background or skills, or physical abilities—to enjoy a positive
experience during both consumption and contribution to knowledge throughout
the Wikimedia ecosystem” (uxstrategy, 191).

###  3.1. Representation Gaps

Representation readership gaps have been widely discussed across the Wikimedia
movement, exposed by surveys organized by the Wikimedia Foundation and the
chapters, and analyzed by research studying the Wikimedia ecosystem. This
facet includes gaps in readership related to sociodemographics and identity
such as gender, age, language and location.

####  3.1.1. Gender

The gender gap is the difference between readers of different gender
identities in how and how much they access the sites. The gender gap has been
measured through many readership surveys organized by the Wikimedia Foundation
(2011survey, 22, 21, 52, 62, 78, 99, 141), different chapters
(2012banglasurvey, 31, 57, 85, 47), independent organizations (2007pewsurvey,
6, 19, 11) and research communities (shaw2018pipeline, 129, 83, 147) by asking
respondents for their gender identity. This makes the gender gap one of the
most heavily studied constructs with respect to readership. However, almost
all surveys have only included men and women as gender identities and have not
distinguished between cis and trans identities, making it difficult to e.g.,
draw any conclusions about readership among people with non-binary gender
identities. Furthermore, conclusions about gender gaps might differ depending
on the definition of readers. For example, the most recent survey in this
space (2019survey, 141) defined readers as people who read Wikipedia daily,
and found that, in almost every country/language surveyed (with the exception
of Romanian), there is a substantial gender gap—i.e. men are more likely to be
frequent readers and tend to generate many more page views than other gender
identities even in regions where previous surveys have shown no strong gender
gaps in occasional usage. A number of factors cause some gender identities to
be under-represented in Wikimedia’s readership population, and across the
years, different community initiatives have focused on bridging this gap
(gendergapinitiatives, 184).

####  3.1.2. Age

The age gap reflects how and how much readers of different age access
Wikimedia sites. There exists a large volume of data on readers’ age,
collected by surveys and academic literature (2011survey, 22, 21, 52, 62, 78,
99, 141, 31, 57, 85, 6, 19, 11, 129, 83, 147, 47). In cases where the age-
distribution at country level is available, the data indicates that readers
tend to be much younger than the general populace. Among other things, the
quality and presentation of content might affect this gap, and various
initiatives across the movement focus on making content more available and
readable for specific age ranges, such as Simple English Wikipedia
(simplewikiabout, 266), the Wikijuniors project from Wikibooks
(wikibooksjunior, 247), or WikiProject Accessibility (wikiprojectaccess, 209),
which, among other things, aims at making content more accessible to elderly
readers.

####  3.1.3. Geography

The geography gap refers to the different levels of readership across
different geographic regions. This gap occurs primarily across two scales:
between countries and within countries. For between countries, it is well-
documented that there are varying levels of Wikipedia readership across
different countries worldwide (wivivi, 135, 78, 99, 22, 52). Many of the
barriers discussed in Section 3.3 such as censorship or data cost are mediated
by country-level policies and resources. Initiatives such as awareness
campaigns (newreadersresearch, 144) or Wikipedia Zero (wikipediazero, 33) have
been run in the past to boost readership in various countries.

The within-country aspect of geography largely refers to an individual’s
proximity to urban areas, which tends to be a strong proxy for availability of
services. While there are no global standardized definitions for how to define
urban/rural, which makes it difficult to compare self-reported locale to
official statistics, results from Wikimedia-Foundation-led and academic
surveys (2019survey, 141, 78, 129, 99) suggest strong over-representation of
urban areas among the reader population. To help bridging this gap, projects
such as WikiConnect (wikiconnect, 249) and the Wikimedia Foundation’s “New
Readers” initiative (newreadersresearch, 144) have worked on connecting
Wikipedia with rural areas and those people who don’t have direct access to
Wikipedia.

####  3.1.4. Language

The language gap reflects the different levels of readership depending on
readers’ ability to read one or more languages. What languages an individual
can read greatly impacts what content is available to them and can introduce
greater barriers if they are forced to read content in a language that is less
familiar to them. Surveys have been conducted to estimate readers’ literacy
(2019survey, 141, 85, 52, 21) suggesting that certain languages have highly-
literate readers. For example, languages that are specific to one country show
high levels of literacy amongst readers. In contrast, other languages such as
English or French, which are more strongly associated with colonialism, have
many readers for which English / French is not their native language
(2019survey, 141). In order to address this issue, in English Simple Wikipedia
was introduced using a simpler grammar and a limited vocabulary. While
improving readability in comparison to English Wikipedia, research has shown
that its level is still not ideal for readers with limited language literacy
(lucassen2012readability, 36). Other initiatives attempting to bridge this gap
aim at making access to content in one’s local language by growing under-
represented languages such as Scribe (scribe, 237), the GapFinder tool
(gapfinder, 221), Content Translation (contenttranslation, 218), or the
Growing Local Language Content on Wikipedia initiative (glow, 223).

####  3.1.5. Socioeconomic Status

The socioeconomic status gap is the difference in how readers with different
education, income, wealth, or employment status access Wikimedia sites. All
these aspects are highly correlated and difficult to disentangle. We start by
looking at education. Education systems vary across different countries, and
one commonly-used country-agnostic proxy for education level is the number of
years of education. Surveys from researchers (shaw2018pipeline, 129, 147) and
from the Wikimedia Foundation and affiliates (2019survey, 141, 57, 85, 52, 11,
19, 19, 31, 21, 22, 47) consistently demonstrate that individuals with higher
levels of education are more likely to be readers of Wikipedia. Similar to
education, income, wealth, and employment status relate to an individual’s
means and class status. Surveys (2007pewsurvey, 6, 19, 21, 57, 22, 31, 21, 52)
and recent research (shaw2018pipeline, 129, 147) consistently indicate that
higher volumes of readership are found among people with higher income, though
the relationship between employment status and readership is less clear. The
other strong, and sometimes countervailing, trend is that students also tend
to be much heavier readers of Wikipedia. Some Wikimedia projects are trying to
close this gap, by specifically targeting readers with different levels of
education, for example Wikiversity (wikiversity, 270); relatedly, initiatives
such as “Wikimedia+Education” specifically focus on creating community around
the topic of making Wikipedia more integrated with education systems at
different levels (wikimediaeducation, 251).

####  3.1.6. Sexual Orientation

The sexual orientation gap is the difference between readers who hold various
sexual orientations and how they access the site. Cultural communities around
sexual orientation often are tied to gender identity as well as with LGBTQ+
communities.777https://meta.wikimedia.org/wiki/Wikimedia_LGBT%2B/Portal We are
unaware of empirical research that has specifically focused on sexual
orientation on-wiki within the reader community but careful handling of sexual
orientation on Wikipedia can be very important to readers who have these
identities (wexelbaum2015queering, 73). Notably, given the oppression around
sexual orientation in many countries around the world, extreme care should be
taken before including questions about sexual orientation in surveys to
readers.

####  3.1.7. Cultural Background

The cultural background gap includes differences in readership among people
with different ethnicity, race, political beliefs, or religious beliefs. These
are gaps that tend to be very contextual and specific to regions—i.e. the
relationship between any aspect of one’s cultural background and the power
they hold within society and representation on Wikimedia projects can vary
greatly between regions. Studies on readers from emerging communities found
some evidence that local socio-political constructs might impact the way in
which Wikipedia’s credibility is perceived (newreadersresearch, 144).
Wikipedia by nature aims to structure its content to be welcoming to readers
with different individual identities, perspectives and opinions. For example,
the “Neutral Point of View” policy (npov, 258) encourages editors to represent
“fairly, proportionately, and, as far as possible, without editorial bias, all
the significant views that have been published by reliable sources on a
topic”. Most friendly space policies in Wikimedia also encourage inclusiveness
at many levels, including the respect of people’s religious views. Moreover,
surveys around community health report religion as one prominent factor
creating barriers to involvement in Wikimedia communities (communityhealth,
185).

Ethnicity and race, or other proxies for caste, bring additional context to an
individual’s cultural background and often relate strongly with access to
resources. Studies in the United States have shown gaps in usage along racial
lines (2007pewsurvey, 6, 19, 129) and presumably these same trends are found
in other countries in that people who are disenfranchised due to their race or
ethnicity also would be less likely to access Wikipedia.

###  3.2. Interaction Gaps

This facet deals with all those gaps related to readers’ ability and
experience when interacting Wikimedia sites. Gaps in this facet include
reader’s motivation, familiarity with the content, and information need behind
their presence on Wikipedia and sister projects, as well as individuals’
abilities to interact with the sites. Notably, while readers’ identities
described under the Representation gaps are relatively fixed, Interaction gaps
capture how a readers’ changing context might impact access.

####  3.2.1. Motivation

The motivation gaps reflect different levels of readership depending on the
reason behind readers’ visit to the site. There is quite a bit of variability
between languages in readers’ predominant motivations (lemmerich2019world,
150, 98). The 2019 Reader Demographics Surveys (2019survey, 141) confirmed
that these differences cannot be fully explained by individual demographics.

####  3.2.2. Information Depth

The information depth gaps reflect different levels of readership depending on
what level of content a reader is looking for on the site—i.e. checking a
quick fact, looking for an overview of a topic, or an in-depth read. Analogous
to reader motivations, there is quite a bit of variability between languages
in what readers’ predominant information depths are. The 2019 Reader
Demographics Surveys (2019survey, 141) confirmed that these differences cannot
be explained by individual demographics. Still not fully understood is how
these information needs might relate to the utility of Wikipedia for a given
reader and whether readers with information needs like “fact” or even
“overview” are largely consuming Wikipedia via external re-use and are
therefore underrepresented in the existing data. Additional information about
reader information depth was gathered in past reader surveys (2019trustsurvey,
146, 150, 113).

####  3.2.3. Familiarity

The familiarity gaps reflect different levels of readership depending on one’s
prior familiarity with a topic. Similar to the motivation, someone’s prior
knowledge about a topic might affect their perception of Wikipedia as an
appropriate data source, and average readers’ prior knowledge on the topic
they read varies across Wikipedia language editions (lemmerich2019world, 150,
98). Ongoing community discussions about content “Depth vs Breadth” indirectly
reason about solutions to address these gaps (breadthdepth, 232).

####  3.2.4. Tech Skills

The tech skills gap reflects different levels of readership depending on
readers’ general internet skills. This captures one’s experience with the
internet and ability to not only find the content one is looking for but also
the ability to verify it. Academic research (shaw2018pipeline, 129) has found
that high internet skills are associated with an increase in participation and
readership on Wikipedia. External initiatives such as Mozilla’s Digital Skills
Observatory (mozillaskills, 77), a research project studying the impact of
digital skills training on confidence and agency of low-income first-time
smartphone users, provide useful insight in how to bridge this gap.

####  3.2.5. Disabilities

The disabilities gap reflects how individual disabilities might affect one’s
ability to access the knowledge within Wikipedia. While individuals who are
blind might be the most salient example, disabilities fall into many
categories: cognitive, developmental, intellectual, mental, physical, or
sensory disabilities. For some, disability can connote a cultural community as
well. Access is closely tied to what options are available for interacting
with Wikimedia projects (see Barriers below). There is scant literature on the
degree to which individuals who read Wikipedia have various disabilities
beyond anecdotal evidence—e.g., an interview with Graham Pierce, an individual
who is blind and a prolific editor (grahampierce, 114), tips for improving
accessibility (signpost20130904, 49), a survey from 2008 that indicated that
Wikipedia was a popular site for individuals who use screen readers
(webaimscreenreadersurvey, 15). A number of projects focus on improving the
accessibility of Wikimedia sites though, such as WikiProject Accessibility
(wikiprojectaccess, 209), WikiProject Usability (wikiprojectusability, 240),
WikiBlind User Group (wikiblind, 211), and Para-Wikimedians User Group
(parawikimedians, 210).

###  3.3. Barriers

We identified a few components which act as barriers preventing people from
accessing free knowledge in Wikimedia projects. While this is not an
comprehensive list, the following elements emerged during our literature
review as potential causes and barriers for readers’ knowledge gaps.

####  3.3.1. Censorship

Throughout the years, Wikipedia has been censored in various times at
different locations around the world (clark2017analyzing, 94), and as of 2020,
the access to the site is still banned in a few countries around the world
(censorshipwiki, 252). Even in countries where Wikipedia is not blocked, self-
censorship plays a role in preventing people from accessing free knowledge:
research has found that, as a result of the fear of surveillance, traffic to
privacy-sensitive Wikipedia articles has been progressively declining
(penney2016chilling, 82). Due to country-wide blocks of Wikipedia and self-
censorship, many readers therefore face difficulties accessing Wikipedia that
generally require complicated technical solutions such as VPNs.

####  3.3.2. Internet Connectivity and Cost

Even though a given Wikipedia article is relatively small in size, slow
internet connections can still make it very difficult to reach and browse
Wikipedia. In surveys linking internet connectivity to readership
(2010pewsurvey, 19, 21, 78), we see much lower readership among low-speed
internet users. Evidence from early experiments after the introduction of a
new data center in Singapore showed that lower latency correlates with higher
longer-term reader engagement (phabT222078, 157). Internet cost can also be a
huge barrier to internet access even when a country has good connectivity
(2014globalsouthsurvey, 52). The data on usage after Wikipedia Zero program
ended suggest that in places where data have indeed a high cost, it is a
significant barrier to readership. Initiatives such as Kiwix (kiwix, 225) and
other projects from the Inuka team at the Wikimedia Foundation
(newreadersresearch, 144) aim at breaking this barrier by making Wikipedia
available in offline settings.

####  3.3.3. Device

With predominantly mobile usage in countries that are only now coming online
(2016phonesurvey, 78, 52), making Wikipedia more accessible via mobile phones
is a high priority. Efforts are already underway to support
KaiOS888https://www.kaiostech.com/ (lightweight phone OS) and continued
development of the mobile interface as well as official Android and iOS apps
is important for welcoming this increasingly mobile population (kaios, 100).

####  3.3.4. Interface Complexity

One of the typical causes of interaction gaps such as Tech Skills or
Disabilities is the complexity of the interface on Wikimedia sites. In
general, providing a variety of ways to access content—e.g., text, images,
video, audio—while reducing barriers to understanding—e.g., good color
contrast, high readability—is a good approach to improving the accessibility
for readers of all abilities (universaldesignlearning, 175). The Design
Research at the Wikimedia Foundation has developed a set of Wikimedia design
guidelines (designwmf, 180) that are based on the principle of “designing for
everyone”. This is aligned with the Wikimedia Strategy Recommendation of
“improving user experience” (uxstrategy, 191) by making Wikimedia platforms
more inclusive, allowing everyone to access free knowledge. Initiatives aiming
at breaking this barrier include, for example, Wikispeech (wikispeech, 268)
which provides text-to-speech for Wikipedia articles and VideoWiki (videowiki,
241), providing a tool for collaboratively editing videos from images and
wikitext.

####  3.3.5. Text Readability

Content might be structured in such a way that it is inaccessible to readers
with various cognitive, developmental, intellectual, mental, physical, or
sensory disabilities—e.g., blindness, dyslexia. For English Wikipedia, the
Accessibility component of the Manual of Style (manualofstyle, 257) was
written to address these issues and the Accessibility Dos and Don’ts
(dosanddonts, 254) provides a quick overview of some of the more prominent
accessibility gaps in content. In particular, the following barriers are
identified (to which we add how they might be better tracked):

  * •

Do use high contrast and color-blind friendly color schemes:
{{Template:Overcolored}}, {{Template:Cleanup colors}}, and
{{Template:Overcoloured}}, all of which add articles to [[Category:Wikipedia
articles with colour accessibility problems]].

  * •

Do provide alt text and a caption for most images: {{Template:Alternative text
missing}} adds articles to [[Category:Unclassified articles missing image
alternative text]], but this should also be detectable from wikitext dumps.

  * •

Do provide a text description of any charts or diagrams: for tables, this
should be detectable via wikitext dumps.

  * •

Do nest section headings sequentially: pseudoheadings would be harder to
detect but finding articles where section headers are out of order should just
require going through parsed wikitext (or parsing the wikitext dumps with
mwparserfromhell999https://mwparserfromhell.readthedocs.io/en/latest/ or
similar libraries).

  * •

Do create correctly structured tables: it should be possible to detect lack of
headers / scope for tables but other issues such as tables for layout or
incorrectly structured tables would be much harder to detect.

  * •

Do encase non-English words or phrases in {{lang}}: {{Template:Cleanup lang}}
tracks instances where the {{lang}} template should be used and adds them to
[[Category:Pages with non-English text lacking appropriate markup]]. There are
also machine-learning models that do language identification but they are
unlikely to work well for identifying the short stretches of other-language
text for which these templates tend to be used.

Facet |  Gap |  Description |  Sources
---|---|---|---
Representation  |  |  |
Gender |  Differences between contributors of different gender identities in how and how much they contribute to the sites. |  literature (shaw2018pipeline, 129, 83, 147), surveys (2011survey, 22, 21, 23, 24, 32, 52, 99, 118, 97, 140, 142, 11, 119), strategy (uxstrategy, 191, 108), community (2012banglasurvey, 31, 57, 87, 79, 131, 69)
Age |  Differences between contributors of different ages in how and how much they contribute to the sites. |  literature (shaw2018pipeline, 129, 83, 147), surveys (2008survey, 11, 22, 21, 23, 24, 32, 52, 99, 97, 118, 119), strategy (uxstrategy, 191, 108, 184), community (2012banglasurvey, 31, 57, 87, 79, 131, 69, 269)
Geography |  Differences between contributors of different geographies in how and how much they contribute to the sites. |  literature (shaw2018pipeline, 129, 80), surveys (2017strategysurvey, 99, 119), community (2016ireland, 79, 152, 235)
Language |  Differences between contributors of different reading abilities in a language in how and how much they contribute to the sites. |  surveys (2011phonesurvey, 21, 52, 119), strategy (strategy, 108, 191), community (2016ireland, 79, 131, 69, 255)
Socioeconomic Status |  Differences between contributors of different education, income, wealth or employment status in how and how much they contribute to the sites. |  literature (Hinnosaar2019newmedia, 147, 129), surveys (2008survey, 11, 22, 21, 23, 24, 32, 52, 118, 119), community (2012banglasurvey, 31, 57, 79, 131, 69, 250)
|  Sexual Orientation |  Differences between contributors of different sexual orientations in how and how much they contribute to the sites. |  literature (genderequity2018, 130, 73), surveys (2011aprileditorsurvey, 23), strategy (lgbtsupport, 217), community (lgbtportal, 236)
|  Cultural Background |  Differences between contributors of different political, religious, or ethnic background in how and how much they contribute to the sites. |  literature (shaw2018pipeline, 129, 161, 149) surveys (evs2017, 172), community (whoseknowledge, 226, 212, 233, 187)
Interaction  |  |  |
Motivation |  Differences in contribution depending on one’s reason for contributing to the site. |  literature (algan2013cooperation, 42, 74, 91, 92, 196), surveys (2014globalsouthsurvey, 52, 23, 32, 11, 69)
Role |  Differences in contribution depending on the type of editing that one chooses to do. |  literature (arazy2015functional, 59, 89), surveys (2011aprileditorsurvey, 23, 24, 32, 52, 97), community (useraccesslevels, 263, 127, 219)
Tech Skills |  Disparities in ability to contribute to the knowledge within Wikipedia depending on one’s general internet skills |  literature (shaw2018pipeline, 129), strategy (skillsstrategy, 190, 191)
|  Disabilities |  Disparities in ability to contribute to the knowledge within Wikipedia depending on individual disabilities |  literature (buzzi2008making, 8), community (wikiblind, 211, 210, 209, 114)

##  4\. Contributors

The contributor dimension of knowledge gaps covers all gaps related to
categories of people contributing to Wikimedia sites. We define contributors
as all individuals who edit or otherwise maintain Wikimedia content. For the
purpose of this taxonomy, this definition does not include technical
contributors—i.e. the individuals who build the MediaWiki software on which
Wikimedia sites run—though the software and choices made in its design
certainly are highly impactful on what types of contributors feel supported
and what content is created (ford2017anyone, 96). It also does not
specifically address bot accounts, though the different roles these accounts
take on can have a large impact on the community and content
(halfaker2012bots, 35, 166, 80). Contributors gaps are organized into two main
facets: representation gaps—i.e. who is included in the contributor
population—and interaction gaps—i.e. are individuals supported regardless of
their context or ways in which they would like to interact with Wikimedia. We
also highlight different initiatives aiming at closing each gap and some
common barriers to greater equity in this space. Similar to the Readership
dimension, the gaps in the Contributors facet are inspired by the 2020
Movement Strategy, which invites Wikimedia communities to be safe and
inclusive (inclusivestrategy, 187) and projects to be designed so that
everyone is welcome to contribute (uxstrategy, 191).

###  4.1. Representation Gaps

Similar to their Readers counterpart, contributors’ representation gaps—i.e.
sociodemographic gaps such as gender, age, and education—have been widely
discussed by different sources.

####  4.1.1. Gender

The gender gap is the difference between individuals of different gender
identities in how likely they are to contribute to Wikimedia sites. The gender
gap has been measured through many surveys organized by the Wikimedia
Foundation (2011survey, 22, 21, 23, 24, 32, 52, 99, 118, 97, 140, 142, 119),
different chapters (2012banglasurvey, 31, 57, 87, 79, 131, 69, 47, 26),
independent organizations (2008survey, 11) and research communities
(shaw2018pipeline, 129, 83, 147, 56) by asking respondents for their gender
identity, though generally limiting the possible answers to just a binary
man/woman choice and do not specify between cis and trans identities.
Conclusions about gender gaps might differ depending on the definition of
contributors. For example, recent surveys in this space
(2019editorgendersurvey, 140) stratified editors by how many edits were
associated with their account. For all three languages surveyed—Arabic,
English, and Norwegian—the gender gap is less extreme for editors with fewer
edits. While researchers are still working on investigating the causes behind
some gender identities being under-represented in Wikimedia’s contributor
population, different community initiatives are focusing on bridging this gap
(gendergapinitiatives, 184).

####  4.1.2. Age

The age gap is the difference between individuals of different age in how
likely they are to contribute to Wikimedia sites. Contributors’ age data has
been collected by surveys and academic literature (2008survey, 11, 22, 21, 23,
24, 32, 31, 147, 57, 52, 83, 129, 87, 79, 99, 97, 118, 131, 69, 119, 47, 26,
56). These studies found that the median age of contributors varies
substantially, but the median age tracks slightly lower than country-by-
country median
age.101010https://en.wikipedia.org/wiki/List_of_countries_by_median_age This
is best captured by the 2018 Community Insights survey (2018communityinsights,
118). Median age ranges from around 40 in Western Europe (2015wikimedianl, 69,
87, 97, 118, 79) to the 20s in Eastern Europe (2018ukraine, 131, 83) and
“Global South” (2014globalsouthsurvey, 52). Globally, the median age has
increased overtime from the low 30s (2011novembereditorsurvey, 24, 32) to high
30s (2017communityinsights, 97, 118). Throughout the years, several
initiatives have been developed to bridge this gap and involve senior citizens
as contributors for Wikimedia projects (seniorsoutreach, 269).

####  4.1.3. Geography

The geography gap is the difference between where an individual lives—both
which country and where in the country—and how likely they are to contribute
to Wikimedia sites. Understanding editors’ locales is quite important to
understanding gaps as it is both relatively easy to infer with high accuracy
and also highly indicative of their cultural context and availability of
services. In particular, there is an abundance of data about which countries
editors come from because country can be inferred with high accuracy based on
an editor’s IP address.111111Which country an editor’s IP address is from is
different than where they live or are from, but generally it is assumed that
there is high overlap. While that information is not public (unless the
individual was not signed-in when they edited), the geoeditors dataset
(geoeditors, 186) provides monthly snapshots of about how many editors from
each country contributed to each project. Analyses of this and related data
have examined the relationship between what regions the content is about on
Wikipedia and where the editors who wrote that content are from
(sen2015barriers, 70, 41). Various surveys (2008survey, 11, 23, 24, 32, 97,
118, 119, 31, 69) have also been able to explore the distribution of editors
for various language editions without requiring access to the more sensitive
IP information.

####  4.1.4. Language

The language gap is the difference between an individual’s fluency in a
language and how likely they are to contribute to Wikimedia sites. Surveys
have been conducted to estimate contributors’ literacy or language skills
(2011phonesurvey, 21, 52, 79, 131, 69, 119) and the Babel system (babelsystem,
255) is widespread on user talk pages and offers an alternative to
understanding the fluency of contributors. Though it may feel intuitive that
fluency would be required to contribute, lowering the barrier to contribution
by lower-fluency individuals can be important for effective patrolling in
small wikis (patrollingwikimedia, 155), increase the diversity of
contributors, and allow for the cross-pollination of content that might
otherwise remain locked up in other languages (hale2014multilinguals, 55).
Many editors are multilingual and contribute to Wikipedia in a variety of
languages, with small wikis heavily depending on multilingual editors and
English the most common second-language outside of one’s native language
(2015wikimedianl, 69, 55, 52). While reducing language barriers is important,
it also brings risks of larger communities overshadowing the contributions of
more local contributors as happened recently with Scots Wikipedia
(scotswikirewrite, 171). Tools like Scribe (scribe, 237) have sought to
address the language gap by making it easier to contribute in one’s own
language even when there are not easier approaches to writing articles like
Content Translation (contenttranslation, 218) available.

####  4.1.5. Socioeconomic Status

The socioeconomic status gap is the difference between individuals of
different education, income, wealth, or employment and how likely they are to
contribute to Wikimedia sites. Research on contributor income
(2011phonesurvey, 21, 129) and employment (2011survey, 22, 21, 23, 24, 32, 31,
147, 57, 52, 79, 131) does not show consistent patterns in contribution by
income (shaw2018pipeline, 129) but higher contribution rates among individuals
who are employed (2014globalsouthsurvey, 52, 129). This is presumably
convoluted by the high number of student contributors to Wikipedia
(2012editorsurvey, 32). Regarding education, several surveys (2008survey, 11,
22, 21, 23, 24, 32, 31, 147, 57, 52, 129, 79, 131, 118, 69, 119, 26, 56)
consistently demonstrate that individuals with higher levels of education are
more likely to be contributors to Wikipedia. Despite this skew, many
academics, students, librarians, and other scholars do not contribute to
Wikipedia. Organizations like Wiki Education (wikiedu, 250) and the Wikipedia
Library (wikipedialibrary, 229) work to change this.

####  4.1.6. Sexual Orientation

The sexual orientation gap is the difference between contributors who hold
various sexual orientations and how they contribute to the site. Cultural
communities around sexual orientation often are tied to gender identity as
well as with LGBTQ+
communities.121212https://meta.wikimedia.org/wiki/Wikimedia_LGBT%2B/Portal
There has been less empirical research that has specifically focused on sexual
orientation on-wiki within the contributor community, but plenty of evidence
of barriers facing this community. Surveys as early as 2011
(2011aprileditorsurvey, 23) showed that editors were being harassed about
their sexual orientation and the Wikimedia LGBT+ user group (lgbtportal, 236)
was beginning to organize as early as 2006 (wexelbaum2015queering, 73). There
are WikiProjects focused on LGBT+ issues on at least 29
wikis131313https://www.wikidata.org/wiki/Q15092984 and the Wikimedia
Foundation released a statement in December 2020 detailing issues around
safety faced by these communities (lgbtsupport, 217). Notably, given the
oppression around sexual orientation in many countries around the world,
extreme care should be taken before including questions about sexual
orientation in surveys to contributors.

####  4.1.7. Cultural Background

The cultural background gap includes differences in contributorship among
people with different ethnicity, race, political beliefs, or religious
beliefs. These are gaps that tend to be very contextual and specific to
regions. Ethnicity and race, or other proxies for caste, are very contextual
as to what it means about an individual’s status and access to resources.
Studies in the United States have shown gaps in contribution along racial
lines (shaw2018pipeline, 129) and presumably these same trends are found in
other countries where groups of people who are disenfranchised due to their
race/ethnicity are also less likely to contribute to Wikipedia.

No comprehensive surveys were found that included measures of individual
beliefs related to politics, religion, or culture such as those asked by the
European Values Survey (evs2017, 172). Research (shi2019wisdom, 161, 149) has
attempted to assign political leanings to editors, however, based upon their
contribution history, showing interest in how individual beliefs of
contributors affect the process of collaboration, the degree to which they
contribute, and what articles they edit. One of the main recommendations
within the 2030 movement strategy encourage communities to bridge these gaps
by making the community more inclusive and safe (inclusivestrategy, 187). One
of the main mechanisms the community uses to ensure respect for the diversity
of identities and beliefs represented is the Friendly Space Policy, which is
in force at every community event (friendlyspace, 233). Work is ongoing
regarding a Universal Code of Conduct to establish basic expectations of
acceptable behavior and enforcement for the entire movement as well
(universalcoc, 174).

###  4.2. Interaction Gaps

Individuals contribute to Wikipedia for many different reasons and in many
different ways. Our tools need to support this diversity of motivations,
skills and types of work or we risk exacerbating gaps by only supporting
certain types of contributors. Of note, while research that seeks to reduce
barriers to contributing (e.g., socialization via Wikipedia Teahouse
(morgan2018evaluating, 127), personalized edit recommendations from SuggestBot
(cosley2007suggestbot, 5) or GapFinder (wulczyn2016growing, 88)) has been
demonstrated to increase contributions, research that has focused on playing
directly to editors motivations is a more cautionary tale and can instead
supersede existing motivations (see Gamified Onboarding (narayan2017wikipedia,
109)), or be perceived as manipulative (rolespecificrewards, 134).

####  4.2.1. Motivation

Motivation gaps reflect different levels of contributions depending on the
reason behind an individual’s desire to contribute to the site. Contributor
motivations include both the intrinsic (e.g., support of free knowledge, for
fun, curiosity, personal satisfaction) and extrinsic (e.g., fixing errors,
promoting a topic, professional or school-related reasons, learning a new
skill, money) (hanna2014motivate, 56). This wide range of motivations affect
contributors’ levels of activity (algan2013cooperation, 42, 74) and roles that
they take on (arazy2017and, 91). Contributors’ initial
motivations141414https://upload.wikimedia.org/wikipedia/commons/8/8a/Global_South_User_Survey_2014_-_Full_Analysis_Report.pdf?page=235;
https://upload.wikimedia.org/wikipedia/commons/7/76/Editor_Survey_Report_-
_April_2011.pdf?page=12 can also differ greatly from the reasons they
stay.151515https://upload.wikimedia.org/wikipedia/commons/8/81/Editor_Survey_2012_-_Wikipedia_editing_experience.pdf?page=19;
https://upload.wikimedia.org/wikipedia/commons/7/76/Editor_Survey_Report_-
_April_2011.pdf?page=14 See “why do people edit?” (Rader2020whydopeopleedit,
196), Balestra et al. (balestra2017fun, 92), various surveys (2008survey, 11,
23, 52, 69), or the detailed
results161616https://upload.wikimedia.org/wikipedia/commons/a/a1/Wikipedia_Editor_Survey_2012_-_motivation_analysis.pdf
from the 2012 Editor Survey (2012editorsurvey, 32) for additional details.

####  4.2.2. Role

Roles reflect different levels of contributions depending on what type of work
an individual does to support the wikis. Editors might be dedicated to one or
more projects, and have different levels of experience and lifetime on their
projects. There are explicit roles—i.e. user access levels (useraccesslevels,
263)—on Wikipedia through which users ascend (arazy2015functional, 59).
However, within these access levels, individuals can do very different types
of work (contributiontaxonomy, 28). Yang et al. (yang2016did, 89) identified
seven editor roles that users of all access levels undertake. These can be
expanded to more specific types of work but offer a high-level view and can be
modeled from edit activity.

  * •

Social Networker: communicate on user pages and communication namespaces

  * •

Fact Checker: removal / verification of content

  * •

Substantive Expert: content producers—adding substantive content to articles

  * •

Copy Editor: grammar, paraphrase, relocation

  * •

Wiki Gnomes: clean up content and wikitext markup issues

  * •

Vandal Fighter: reverting vandalism, warning editors, and other patrolling
work

  * •

Fact Updater: updating templated content or Wikidata

  * •

Wikipedian: working behind the scenes (non-article mainspace) to keep things
organized etc.

Additionally one might also include the following roles that do not
necessarily have public traces on Wikimedia but are highly important roles
that contributors also take: organizers, public advocates, technical
contributors, affiliates, board members. Surveys generally ask contributors
what types of work they do on-wiki (2011aprileditorsurvey, 23, 24, 32, 52,
97), but these tasks can be matched to the above roles. Initiatives such as
the Wikipedia Teahouse (morgan2018evaluating, 127) or the Newcomer Page
(newcomerhomepage, 219) help newer Wikipedians learn how to familiarize
themselves with the platform and progressively become more expert editors.

####  4.2.3. Tech Skills

The tech skills gap reflects different levels of contributions depending on an
individual’s general internet skills. While extensive effort has been put into
simplifying the process of contributing to Wikipedia, editing articles,
uploading images, and identifying sources can still pose substantial technical
challenges for individuals. Academic research (shaw2018pipeline, 129) has
found that high internet skills are associated with an increase in awareness
that Wikipedia can be edited and having edited Wikipedia. Edit-a-
thons171717https://en.wikipedia.org/wiki/Edit-a-thon in particular can help to
bridge this gap and the 2030 strategy calls for further investment in skills
and training (skillsstrategy, 190).

####  4.2.4. Disabilities

The disabilities gap reflects how individual disabilities might affect one’s
ability to access and contribute to the knowledge within Wikipedia. While
individuals who are blind might be the most salient example, disabilities fall
into many categories: cognitive, developmental, intellectual, mental,
physical, or sensory disabilities. There is scant evidence beyond the
anecdotal of the degree to which individuals with various disabilities
contribute to Wikipedia (see Userboxes in WikiProject Accessibility
(wikiprojectaccess, 209), an interview with Graham Pierce (grahampierce, 114),
and evaluation from 2008 of editing with a screenreader (buzzi2008making, 8)).
Various groups have also been established for individuals with disabilities
such as the WikiBlind User Group (wikiblind, 211) and Para-Wikimedians User
Group (parawikimedians, 210).

###  4.3. Barriers

Barriers and causes of Contributors’ knowledge gaps match most of the ones we
identified for readers in Sec. 3.3.

####  4.3.1. Censorship

Similar to the considerations around censorship and readers in Sec. 3.3, both
direct and self-imposed censorship can be important barriers preventing people
to contribute to the sites. In a study around Wikipedia censorship in China,
Zhang and Zhu (zhang2011group, 29) found that blocks to Chinese Wikipedia in
Mainland China in October 2005, beyond preventing contributions from
individuals located in Mainland China, also led to a 42.8% drop in
contributions from non-blocked individuals.

####  4.3.2. Internet Connectivity

As seen in the case of readership, the quality of internet access can also
affect one’s ability to contribute. This covers both internet speed and
internet cost. While Wikipedia content is relatively lightweight, cost and
speed have been identified as a major barriers to contributors
(2014globalsouthsurvey, 52). Graham et al. (graham2014uneven, 53) found that
removing broadband access barriers is a necessary but insufficient condition
for content generation. Initiatives like Growing Local Language Content on
Wikipedia (glow, 223) work on addressing this gap by subsidizing internet
costs for contributors.

####  4.3.3. Device

Editing Wikimedia projects classically requires a good laptop or desktop
computer in order to easily use the available editing interfaces while
potentially also gathering research in other windows. There has been a shift
to mobile that has been accompanied with huge improvements in mobile editing
interfaces (e.g., Suggested Edits (suggestededits, 230)). Additionally,
efforts have been made to provide laptops (e.g., GLOW (glowfaq, 222)) to new
editors in order to reduce this technological burden. Various surveys
(2011survey, 22, 21, 23, 24, 32, 52) have asked contributors about their
devices.

####  4.3.4. Interface Complexity

Interaction gaps such as Technical Skills and Disabilities are largely
influenced by the complexity of the interface. To reduce the technical entry
barriers for Wikipedia editing, in 2013 the Editing team at the Wikimedia
Foundation introduced the VisualEditor (visualeditor, 44), a rich-text editor
now available on almost all Wikipedias and Wikivoyages. While the visual
editor has been found to have no effect on new editor retention or
productivity, it helped creating more reliable (less reverted) edits
(visualeditoreffects, 45). Another project to reduce interface complexity is
the “Talk Pages Project”, which aims at adding functions such as direct
responses to Wikipedia discussion pages. Beyond Wikipedia, initiatives such as
Structured Data on Commons (sdc, 238) also aimed at removing technical
barriers to contribution to Wikimedia Commons, by introducing an easy
interface for image description and categorization.

####  4.3.5. Privacy

Another important barrier for contribution is the importance that different
people give to privacy. Many contributors have strong privacy concerns around
their contributions to Wikipedia. Usernames allowed much of one’s identity to
be protected, and Wikimedia’s privacy policy ensures that contributors should
not disclose “nonpublic personal information to participate in the free
knowledge movement” (privacypolicy, 181). However, some individuals have
credible reasons to contribute while obscuring their IP address, even from
private server logs. These individuals, while possibly making equally high-
quality contributions (tran2020anonymity, 201), face the challenge that Tor
and many of these proxy IP addresses (ipblockexemption, 206, 207) are blocked
due to vandals (or perceived potential for vandalism) using them to circumvent
IP blocks.

####  4.3.6. Community Health

A well-documented barrier to retaining a diverse community of contributors is
the lack of inclusivity and sometimes outright harassment (lack of safety)
that some experience in the editor community. This has been identified as a
major challenge for newcomer retention (halfaker2013rise, 46, 127) and been
raised consistently as a challenge in community surveys
(2017communityinsights, 97, 118, 63, 23, 24, 32, 119, 185). This harassment
can be disproportionately targeted at already-marginalized communities–e.g.,
women (menking2015heart, 67, 119, 96, 130). Initiatives like the Community
Health Initiative (communityhealthinitiative, 107) and Universal Code of
Conduct (universalcoc, 174) seek to address some of these challenges by
establishing clearer norms, processes, and better tools for identifying and
dealing with harassment.

Facet |  Gap |  Description |  Sources
---|---|---|---
Representation  |  |  |
Gender |  Differences in content coverage depending on the gender identity of subjects |  literature (Reagle2011britannica, 27, 50, 61, 64, 71, 86, 90, 101, 105, 116, 124, 136, 147, 160), strategy (topictrategy, 188, 143), community (counteringsystemicbias, 239, 226, 246, 216, 243, 234, 274, 273)
Age (Recency) |  Differences in coverage of topics and facts related to different times in history |  literature (samoilenko2017analysing, 111), community (wikiprojecthistorical, 224)
Geography |  Differences in coverage of topics related to geographic regions or population distribution |  literature (Hecht2009selffocus, 12, 25, 54, 168), strategy (topictrategy, 188, 143), community (counteringsystemicbias, 239, 226, 253, 271)
Language |  Differences in coverage of topics and facts depending on the content language |  literature (wulczyn2016growing, 88, 122, 30), community (sdc, 238)
|  Socioeconomic Status |  Differences in coverage of topics and facts related to different socioeconomic contexts |  literature (miquel2019wikipedia, 153, 10), strategy , community (counteringsystemicbias, 239)
|  Sexual Orientation |  Differences in coverage of topics and facts related to different sexual orientations |  literature (park2020multilingual, 193), community (wikidatalgbt, 228)
|  Important topics |  Differences in coverage of topics that are of common interest |  literature (Kittur2009whatsinwikipedia, 13, 151, 112, 198), strategy (topictrategy, 188, 143), community (counteringsystemicbias, 239, 226, 213, 231, 18)
|  Cultural Background |  Differences in coverage of topics related to the history, heritage, and characteristics of a current or former cultural group |  literature (MiquelRibe2016, 81, 125), strategy (topictrategy, 188, 143), community (counteringsystemicbias, 239, 226, 242, 215)
Interaction  |  |  |
Multimedia |  Differences in coverage with respect to the type of media used to share the content |  literature (Viegas2007, 7, 121, 195), strategy (strategy, 108, 189), community (imageclassifiers, 178, 179, 120, 145, 256, 245, 267, 272, 273)
Structured Data |  Differences in the use of information which is indexed and machine-readable |  literature (Vrandecic2014wikidata, 58, 202), strategy (strategy, 108, 143), community (wpwikidata, 265, 244, 262, 238)
Readability |  Differences in the ease with which textual content can be understood by readers of different backgrounds |  literature (Yasseri2012language, 40, 36, 88, 138), strategy (strategy, 108), community (contenttranslation, 218, 257, 254)

##  5\. Content

Wikipedia is incomplete by design. The opportunity to share new information
with the world is a major motivating factor among both new and established
Wikipedia contributors. However, when important information about a topic is
absent, incomplete, biased, or otherwise inaccessible to readers, these
content gaps can undermine Wikipedia’s ability to serve the needs of its
global audience.

The goal of this Section is to characterize gaps in content coverage. In the
widest sense, “content” is information about a topic, i.e. a piece of
knowledge that could be the focus of one or more Wikipedia articles.
“Coverage” refers to how well Wikimedia project content addresses a particular
topic. In turn, a content gap refers to differences in coverage of one or more
topics.

The gaps in this dimensions are largely inspired by the “Impactful Topics”
Movement Strategy recommendations, which suggests that Wikimedia communities
should “develop and increase access to content that has historically been left
out by structures of power and privilege” (topictrategy, 188). Furthermore,
the facets have been inspired by community discussions on systemic biases
(systemicbias, 261) as well as reviews on Wikipedia’s content (Mesgari2015sum,
68) and its biases (Hube2017bias, 102).

###  5.1. Representation Gaps

The content representation facet includes all gaps related to topic coverage
in Wikimedia projects. Developing a comprehensive, hierarchically-structured,
and canonical representation of Wikipedia content is a non-trivial if not
impossible task. Wikipedia’s own category structure for
content181818https://en.wikipedia.org/wiki/Wikipedia:Categorization is not
hierarchical, and the language and culture-specific nature of topic relevance
make the creation of an exhaustive list of topic gaps infeasible. Therefore,
our aim here is to provide a list of example topic gaps that have received
widespread attention in public media, research studies, and include topics
highly curated by community projects and discussed across different parts of
the movement, such as gender, geography, and culture gaps. Gaps in this facet
are inspired by both the Wikimedia Foundation’s Medium-term plan (mtp, 143),
which supports diverse content creation “by actively foster[ing] the inclusion
of underrepresented and marginalized knowledge, and ensur[ing] content is
locally relevant to communities”, as well as the the movement strategy
directions (topictrategy, 188) which encourages “improving coverage of
collectively-identified priority topics that impact our world and improve
people’s lives”. Community initiatives such as the Wikiproject Countering
Systemic Bias (counteringsystemicbias, 239), the Wikipedia Diversity
Observatory (miquel2019wikipedia, 153), or Whose Knowledge? (whoseknowledge,
226) aim to address imbalances in the coverage of subjects and topics.

####  5.1.1. Gender

The gender gaps refer to the differences in content coverage depending on the
gender identity of subjects. The gender gap—the fact that the majority of
Wikimedia content about people focuses on male subjects—is one of the most-
studied examples of content gaps and well-documented in the research
literature (Reagle2011britannica, 27, 50, 61, 64, 71, 86, 90, 101, 105, 116,
124, 136, 147, 160) and interactive tools such as the Wikidata Human Gender
Indicators (whgi, 246), Denelezh (denelezh, 216), or the Wikidata Concepts
Monitor (wdcm_bias, 243). A number of community initiatives such as the Gender
Gap Portal (gendergapportal, 234), the Wikiproject Women (wikiprojectwomen,
274), as well as organizations such as Whose Knowledge? (visiblewikiwomen,
273), are focusing their efforts and working hard to address the gender
content gap across Wikimedia projects.

####  5.1.2. Age (Recency)

The age (recency) gap reflects the difference in coverage depending on the
point in time a given topic, fact or event has taken place. That Wikipedia
suffers from a “recency” bias is a well-known fact: researchers have found
that, consistently across Wikipedia languages, content is skewed towards more
recent events (samoilenko2017analysing, 111). This bias seems to match content
demand. Academic research has shown that recent events attracts more views on
Wikipedia (ciglan2010wikipop, 17), especially as a result of wide media
coverage (tizzoni2020impact, 200). Recent work has also found that readers
interact more often with content within the page which refers to recent events
(piccardi2020quantifying, 194). To improve the availability of content across
different points in time, and counter the systemic recency bias, some
communities such as Wikiproject Historical Information (wikiprojecthistorical,
224) focus on identifying and fixing articles that lack historical
information.

####  5.1.3. Geography

The geographic gap captures coverage differences in topics related to
geographic regions or population distribution. Looking at geo-tagged
information on Wikipedia content, research has shown that the geographic
coverage differs substantially across language editions (Hecht2009selffocus,
12, 168) and that geographic coverage is extremely uneven and clustered with a
strong bias towards content related to the United States and Western Europe
(Graham2011geographies, 25, 54). Within countries, there also tends to be a
strong bias towards content in urban areas as opposed to rural areas
(johnson2016not, 80). To address this gap, several initiatives across
Wikimedia projects aim at increasing content coverage of under-represented
areas such as the Africa Portal in Wikipedia (africaportal, 253), Wiki Loves
Africa Contest in Wikimedia Commons (wikilovesafrica, 271), and Wiki Takes
campaign to document rural Spain (wikitakes_rural, 132).

####  5.1.4. Language

The language gap refers to the difference in content coverage across different
languages. While each Wikipedia language edition is a stand-alone project,
with different size and coverage of relevant topics (wulczyn2016growing, 88,
30), other projects such as Wikidata and Wikimedia commons are multilingual by
design. However, while Wikimedia Commons is used across many languages
(he2018the_tower_of_babel, 122), its captions and descriptions area available
mainly in English. Wikidata’s labels are also non-uniformly distributed across
languages, with only 11 languages holding almost 50% of all language knowledge
in Wikidata, English being one of the most prominent ones (kaffee2017glimpse,
103). Projects such as Structured Data on Commons (sdc, 238) and Suggested
Edits (suggestededits, 230) aimed at rehauling the projects’ interface to make
the translation efforts on Commons easier and more effective.

####  5.1.5. Socioeconomic Status

The socioeconomic status gap refers to different levels of content coverage
depending on the socioeconomic status of a location or a person. That content
in Wikipedia is skewed towards western countries with higher Human Development
Index is a well-established fact (wulczyn2016growing, 88, 10). Several
projects such as the Cultural Diversity Observatory (miquel2019wikipedia, 153)
and Wikiproject Countering Systemic Bias (counteringsystemicbias, 239) are
building tools and empowering communities to foster the inclusion of content
from marginalized regions into Wikimedia projects.

####  5.1.6. Sexual Orientation

The sexual orientation gap is the difference in content relevant to various
sexual orientations. Sexual orientation is often discussed in relation to
gender identity as with LGBTQ+
communities.191919https://meta.wikimedia.org/wiki/Wikimedia_LGBT%2B/Portal
There has been less empirical research that has specifically focused on
content and sexual orientation, but the portrayal of LGBTQ+ individuals has
been shown to vary across different language editions (park2020multilingual,
193). Wikidata WikiProject LGBT (wikidatalgbt, 228) has been working on
expanding well-sourced statements about sexual orientation in Wikidata, which
will greatly help content analyses around the representation of sexual
identities.

####  5.1.7. Important Topics

The important topics gap captures coverage differences among content deemed to
be especially impactful to readers and important to be of high quality. The
abundance of topics varies substantially across different Wikipedia language
editions (Kittur2009whatsinwikipedia, 13, 151). Certain topics may be deemed
to take on special importance such as the coverage of medical knowledge on
Wikipedia (Shafee2017evolution, 112) since “[its] health content is the most
frequently visited resource for health information on the internet”
(Smith2020health, 198) (see also the WikiProject Medicine in English Wikipedia
(wikiprojectmedicine, 231)). While the Movement Strategy indicates that the
community is still missing tools to identify which topics are most impactful
in the world, initiatives such as WikiProject Vital Articles (vitalarticles,
213) or All Human Knowledge (allhumanknowledge, 18) help in addressing this
gap by gathering lists of important and impactful topics that should be
present in all Wikipedia editions.

####  5.1.8. Cultural Background

The cultural background gap captures coverage differences related to the
history, heritage, and characteristics of different cultural groups. Cultural
identity has been shown to be a crucial part for the motivation of editors to
contribute (MiquelRibe2016, 81), and the project Wikipedia Diversity
Observatory (wcdo, 242) aims to define and quantify the extent of articles
which can be considered cultural context content (MiquelRibe2018, 125).
Initiatives such as Wikiproject French Caribbean Culture
(frenchcarribeanculture, 215) or AfroCrowd (afrocrowd, 212) aim to address
some of these cultural gaps.

###  5.2. Interaction Gaps

Content in Wikimedia projects is multimodal by nature, and can take very
different forms, such as images, text, structured data, etc., and the form of
presentation clearly affects how accessible content is to different audiences
(strategy, 108). The gaps in this facet cover various aspects of the content
available in Wikimedia spaces that mediate how people access free knowledge
and can help to break down barriers to access.

####  5.2.1. Multimedia

The multimedia gap reflects differences in coverage with respect to the type
of media used to share the content. Acknowledging the potential of different
forms of media beyond text (image, audio, video, geospatial, etc.) to convey
content to different audiences, the Movement strategy (innovationstrategy,
189) recommends building the necessary technology to make free knowledge
content accessible in various formats and support more diverse modes of
consumption and contribution to Wikimedia projects. For example, with the bulk
of a typical article made up of text, the use of images is encouraged in order
to increase readers’ understanding of the subject matter (see, for example,
English Wikipedia’s policy on image use (imageusepolicy, 256)). However, on
average, half of Wikipedia articles are missing images, and around 95% of
Wikidata items do not have a value for the image-property
($P18$)202020https://www.wikidata.org/wiki/Property:P18 (imagewikipedia, 145).
This is exacerbated by the fact that with few exceptions (e.g., (Viegas2007,
7, 121, 195)), the role of visual and multimedia aspects in Wikipedia and
other projects has largely been ignored by researchers. To help in bridging
this gap, several recent research initiatives (imageclassifiers, 178, 179,
120, 145) aim at designing smart tools for image analysis and adding structure
to the Wikimedia Commons repository. Tools built by community members also
support editors in discovering and adding images in Wikimedia projects,
including the WDFIST (wdfist, 245) and the Wikishootme (wikishootme, 267)
tools. Finally, several community initiatives are organized around adding
pictorial representations of under-represented people and topics, such as Wiki
Loves Monuments (wikilovesmonuments, 272), Visible Wiki Women
(visiblewikiwomen, 273), and Wikipedia Pages Wanting Photos (wpwp, 275).

####  5.2.2. Structured Data

The structured data gap reflects the differences in the use of structured data
across Wikimedia projects, namely the amount of information that is organized
and indexed in a machine-readable fashion. Structured data offers the
potential for managing data for Wikimedia projects on a global scale, allowing
easier indexing and offering tools for multilingual data creation and
description. One of the most prominent examples of structured data is
Wikidata, a structured data knowledge base which was designed with the goal of
increasing knowledge diversity (Vrandecic2014wikidata, 58). More structured
data is not a panacea for equity, however, and greater reliance on structured
data can concentrate power and remove important context (ford2016provenance,
76, 170). Information from Wikidata is being re-used in Wikipedia articles in
different ways, most notably in infoboxes (wpwikidata, 265) but perhaps most
commonly in metadata templates (wikidatareuse, 148). The Wikidata Concepts
Monitor (wdcm_usage, 244) provides quantitative insight into the degree of
Wikidata usage, albeit inexactness in tracking this usage makes it non-trivial
to interpret the statistics with respect to individual templates or infoboxes.
In addition, it must be noted that, beyond some established cases, the degree
to which Wikidata should be used in Wikimedia projects is part of ongoing
discussions within the respective communities (see for example the case in
English Wikipedia (useofwikidata, 262)). Given the importance and potential of
structured data, the Wikimedia Foundation’s Medium Term Plan 2019-2020
(Platform Evolution) aims at increasing its usage across Wikimedia projects
(mtp, 143). Product initiatives such as the Structured Data on Commons program
(sdc, 238), as well as new proposals for an Abstract Wikipedia
(vrandecic2020architecture, 202) aim at closing this gap and enable
communities to consume and contribute structured data across languages. Along
with structured data on Wikimedia projects, there are many semi-structured
annotations that play an important role in the maintenance and curation of
Wikimedia content. For example, the category network helps with content
discoverability and classification, various extensions such as PageAssessments
(pageassessments, 177) can be used to rate the quality and importance of
articles to help in evaluating and prioritizing content, numerous templates
are used for tracking issues to be resolved in content (maintenancetemplates,
204). While these annotations are often hidden to readers, they are crucial to
research and tools aimed at discovering and improving content.

####  5.2.3. Readability

The readability gap refers to how difficult it is to read the content of an
article or a piece of information in Wikimedia projects in comparison to the
reading abilities of their readers. Using statistical tools such as the
Gunning-Fog index,212121https://en.wikipedia.org/wiki/Gunning_fog_index
researchers have tried to quantify this gap by computing language complexity
of texts in Wikipedia articles (Yasseri2012language, 40). Different studies
(lucassen2012readability, 36, 138) have shown that the difficulty is
considerably above the reading ability of average adults even for projects
such as Simple English Wikipedia, which are explicitly aimed at closing this
gap.

An indirect approach to bridge the gap is to make content available in
different languages given the high proportion of non-native readers in some
languages (2019survey, 141). For this, researchers developed machine-learning
models to discover and prioritize articles missing in a given Wikipedia
language edition which, in turn, are recommended for creation to editors
(wulczyn2016growing, 88). Platforms such as the Content Translation Tool
(contenttranslation, 218) help in translating articles to any language
available in Wikipedia.

###  5.3. Barriers

Wikipedia content is governed by three principal core content policies—Neutral
point of view, Verifiability, and No original research—which define the scope
and the material that should exist in the online encyclopedia. These policies
shape the way in which content is added to Wikipedia, and, to some extent, to
its sister projects. While these policies ensure that the content in the
project is of high quality, they have also been identified as major barriers
for diverse content inclusion (genderequity2018, 130).

####  5.3.1. Verifiability

Wikipedia’s Verifiability core content policy (verifiability, 264) requires
every piece of information that has been challenged—or is likely to be
challenged—to be backed by a reliable source. The accessibility of a source
and its availability in a specific language play a key role in the coverage of
a piece of content in different projects (sen2015barriers, 70). Previous
research has deepened our understanding of citation usage by highlighting gaps
in terms of the types of sources used (Ford2013, 43, 70), barriers to
inclusion of relevant sources around content about marginalized communities
(2020readingtogether, 167), or their accessibility (Harder2015, 65) as well as
in developing methods for how to automatically detect the citation span of
individual sources (Fetahu2017, 95) or classify what content is missing
citations (Redi2019citationneeded, 158). Censorship outside of Wikimedia—i.e.
more general political freedom within a country as measured by organizations
like Reporters Without Borders or Freedom House—can also have a large impact
on the availability of reliable sources and balanced coverage of topics. There
exist different initiatives to help editors monitor citation quality at scale
such as the Citation Detective tool (choucitation, 214), which automatically
produces dumps of sentences requiring citations in English Wikipedia, or
Wikicite (wikicite, 248), whose goals include the “improvement of citations in
Wikimedia projects and an open, collaborative repository of bibliographic data
for innovative applications”. Furthermore, the Wikipedia Library portal
(wikipedialibrary, 229) constitutes a fundamental tool to help breaking the
verifiability barrier, as it provides reliable sources that editors can use to
improve the articles, as well as promoting community initiatives such as the
“1Lib1Ref” campaign, which encourages librarians to add references missing in
Wikipedia. Furthermore, tools like Scholia (NielsenF2017Scholia, 110), a
service that creates visual scholarly profiles for different items such as
topics, people, and organizations using bibliographic and other information in
Wikidata, can help retrieve specific sources needed to support a claim. While
citations and references are a key tool to monitor the quality of knowledge
coming from written sources, how to incorporate oral knowledge within the
verifiability framework, remains an open point for discussion (oralknowledge,
192).

####  5.3.2. Neutrality

The Neutral Point of View (NPOV) policy (npov, 258) demands that all
encyclopedic content should represent all significant views fairly,
proportionately, and without bias. To help in understanding and bridging this
gap, researchers in various fields have tried to characterize the dynamics of
NPOV disputes (hassine2005dynamics, 4, 128), as well as quantify the NPOV gap
from explicit (statements supporting a certain POV) or implicit (omission of
certain aspects) bias (Hube2017bias, 102). Examples include the use of NPOV-
templates to automatically detect biased language in the content of Wikipedia
(Recasens2013, 48, 123), or documenting biases in specific topics such as
politics (Greenstein2012bias, 34) or culture (Callahan2011culture, 20) among
others. The Neutral point of view Noticeboard (npovnoticeboard, 259) is one of
the main sub-communities dedicated to the discussion of neutrality of content
in English Wikipedia. Similar to the verifiability case, incorporating non-
written knowledge which complies with the NPOV policy is still under
discussion (oralknowledge, 192).

####  5.3.3. Notability

Wikipedia contains several other policies that constitute potential barriers
for content, most prominently the guidelines around notability (notability,
260). The criterion of notability is used to decide whether a topics warrants
its own article on Wikipedia. In its most general form it requires that “a
topic has received significant coverage in reliable sources that are
independent of the subject” for a topic to warrant its own article. The
interpretation of these guidelines is highly controversial (see, for example,
the debate on deletionism vs inclusionism (deletionism, 205)). Notability has
been claimed to play a key role in causing and amplifying systemic biases in
Wikipedia (systemicbias, 261). For example, it is hypothesized that notability
is harder to establish for certain topics due to lack of English sources. In
fact, research on discussions for deletions (lam2009wikipedia, 14, 39) has
shown that notability is the most prominent indicator for deletion of
articles. Case studies, such as on the deletion of biographies of female
scientists, have received wide media attention. For the example of gender
bias, several studies (Wagner2016glass, 86, 160) have shown that female
biographies on Wikipedia are more notable than men using external signals of
notability such as Google Search volume. However, overall notability remains
an understudied area with respect to its role in content gaps.

####  5.3.4. Free Knowledge

A major barrier to providing diverse, high-quality content is the availability
and openness of knowledge. For example, laws related to the freedom of
panorama, which allows people to freely take photos of otherwise copyrighted
work such as public art, varies greatly country to
country.222222https://en.wikipedia.org/wiki/Freedom_of_panorama Strict rules
around what licenses are permitted for use on Wikimedia Commons creates
clarity for end-users (commonsreusers, 126) but restricts what content can be
uploaded—e.g., valuable health-related content from the World Health
Organization.232323https://www.who.int/about/who-we-are/publishing-
policies/copyright These restrictions reduce the availability of content and
also can be challenging to understand and navigate for newcomers (glamcommons,
106).

##  6\. How to Use this Taxonomy

The taxonomy of knowledge gaps developed in this research and the
corresponding literature review can be utilized in a variety of ways by
different actors in the free knowledge ecosystem and the Wikimedia projects.
This includes Wikimedia community organizers, affiliates, Wikimedia Foundation
staff, contributors, researchers and partners. Below we describe the different
aspect of the taxonomy and how they can be utilized.

The taxonomy as a framework for conversations. As evidenced through the work
in developing the taxonomy of knowledge gaps, Wikimedia projects face many
different knowledge gaps. However, prior to the development of this taxonomy,
the bulk of the attention of large actors within the Movement such as the
Wikimedia Foundation has been heavily focused on specific gap types. We hope
that the more comprehensive taxonomy of gaps can provide a framework for the
decision makers to learn about the different gap types, brainstorm about their
possible relationships, and devise ways to address them.

The references that support the gaps. As part of the process of developing the
taxonomy of knowledge gaps, we conducted a major literature review. This
review of past work which is presented as sources in the tables throughout
this paper as well as in the reference section can be a valuable resource for
those who are eager to expand their knowledge of the particular gaps and learn
about their possible causes.

A first step towards measuring knowledge gaps. The definitions of the
different gap types provided through the taxonomy are a necessary step for
developing metrics to measure the different gap types. The measurement of the
gap types and understanding the relationships between them can further help to
develop the knowledge gap index, a composite index that can operationalize
knowledge equity in the Wikimedia projects and help support more data-informed
decision-making processes.

##  7\. Future Work

This taxonomy represents a first step towards understanding the underlying
mechanisms that prevent us from reaching knowledge equity, and designing
solutions to remove those barriers preventing people from accessing free
knowledge. In this Section, we identify the next steps to reach this goal by
building on top of the taxonomy presented in this manuscript.

###  7.1. Metrics

In order to track progress towards knowledge equity, it is necessary to
quantify each of the knowledge gaps described in this taxonomy. For this we
need to i) carefully define metrics reflecting the gap extent across different
categories, and, ii) identify relevant reference data sources to contextualize
the metric and measure the amplitude of the gap. The main challenges in this
endeavour are:

  * •

Operationalizing a gap: The definition of each gap must be translated into a
set of measurable quantities. For example, for the content gender gap in
Wikipedia, a common choice is to use the property P21 (sex or
gender)242424https://www.wikidata.org/wiki/Property:P21 of the corresponding
item in Wikidata. While that approach is simple, transparent, and has good
data coverage, questions can still be raised about the appropriate dimensions
to report and how to capture articles that are not explicitly about a person
(such as women’s health). Operationalizing other gaps might be even less
straightforward. The case of topical content gaps serves as an illustrative
example. Identifying content related to, e.g., medicine, is possible through
manual annotation in Wikiprojects (wikiprojectmedicine, 231) though it is
time-intensive and requires constant updating which might be unfeasible for
the many small and medium-sized projects. Automatic methods such as the ORES
topic classification (ores, 220) alleviate some of these issues, however, they
are currently limited in their general applicability as they only support a
small set of languages and a fixed set of 64 topics.

  * •

Availability of data: Some data for specific gaps is just not available or
hard to obtain for privacy or other reasons. For example, obtaining
information about readers’ ages in order to measure the age representation gap
requires careful legal frameworks (2019survey, 141). Succinctly capturing
complex and contextual social phenomena like race is largely not possible at a
global scale.

  * •

Consistency of data: seasonality, large-scale events or holidays, and edit
campaigns can have a large impact on who is reading or contributing to
Wikipedia at any given moment in a year (chelsy2019detecting, 139). This
becomes particularly salient when trying to survey a global population where
the effects of these events will vary greatly between regions and even
hemispheres of the world. These external effects will have to be taken into
account for data that is collected at irregular (or even regular) intervals.
On top of this, varying sampling methodologies or survey questions can also
greatly impact who responds and how (howmanywomeneditwikipedia, 60).

  * •

One gap, many metrics: For every individual gap, there exist multiple, equally
relevant, ways in which to measure it. Thus, it is important to note that
there are many metrics which capture different aspects of the same gap. The
content gender gap provides an illustrative example, as it constitutes one of
the most well-studied gaps. Public tools documenting the gap (such as Wikidata
Human Gender Indicators (whgi, 246) or Denelezh (denelezh, 216)) document the
number of articles on people of different gender identities, respectively,
thus capturing only the selection of content. Research has shown that it is
equally important to consider other aspects such as extent (e.g., comparing
the quality of articles on men and women (Halfaker2017keilana, 101)) or the
framing (e.g., comparing the language in the articles of men and women
(Wagner2016glass, 86)). Building on initial insights (contentgapswikipedia,
154), future research needs to understand the role of different metrics of the
same gap and how the well-explored content gender gap might inform development
of metrics for less-explored facets.

  * •

Goal: The definition of a metric for a gap allows one to make a statement
about whether the gap is small or large. Such definition thus leads to
conclusions about whether a gap is closed or not. What this goal should be
might not be as obvious as it seems. For example, in the context of the
content gender gap, recent discussions in the community revolve around what
are suitable baselines for the comparison of the number of biographies on men
and women (sizegendergapactually, 199). Future work involves curation of
external datasets such as sociodemographic data on country-level and
consultation and discussions with communities on how to define the metrics.

###  7.2. Knowledge Gap Index

One of the end goals of this taxonomy is to generate a knowledge gaps index,
namely a single indicator combining metrics from different gaps, measuring the
overall knowledge equity of Wikimedia projects. Socioeconomic indices have
been adopted by many organizations, advocacy groups and policy makers such as
the Global Innovation Index,252525https://www.globalinnovationindex.org/Home
the Human Development Index from the United
Nations,262626http://hdr.undp.org/en/content/human-development-index-hdi or
the Gender Equality Index from the European
Union.272727https://eige.europa.eu/gender-equality-index/ Such indices are a
useful tool for analysis of policies and communication with the public because
they can give a coarse-grained view on complex multi-dimensional realities.
Indices require the projection and aggregation of different metrics into a
composite index and are developed over the course of several steps, the first
being the construction of a theoretical framework (Commission2008handbook, 9).
The main challenges to reach this goal are:

  * •

Availability of metrics: An index requires the availability of validated
metrics (see the discussion in the previous subsection).

  * •

Robustness: Even if all metrics are available, the robust construction of a
composite index is non-trivial. According to recommended procedures, this
taxonomy indeed represents only the first, theoretical step towards a
comprehensive, robust and high-quality index (Commission2008handbook, 9).
Additional steps would include, for example, uncertainty and sensitivity
analysis.

###  7.3. Interactions between Gaps

The taxonomy presented here reflects a simplified version of the complex
landscape of knowledge gaps by depicting readership, contributorship, and
content as three independent dimensions. However, these dimensions are
interrelated and feed into each other, as pointed out by the community
(systemicbias, 261), research (shaw2018pipeline, 129), and Wikimedia
Foundation in their medium-term plan (mtp, 143). Apart from few examples (e.g.
(WarnckeWang2015misalignment, 72) on the misalignment between content and
readership), our understanding of how different gaps interact with each other
is still very limited, and part of our future research work includes shedding
light on these interactions.

###  7.4. Causes and Interventions

In order to bridge knowledge gaps, we not only need metrics to monitor the
efficacy of individual interventions, but also require an understanding of the
underlying causes to design effective interventions. However, causal evidence
on what drives gaps is scarce. Recent research has identified several
candidate hypothesis for individual gaps (explainingreadergendergap, 156). An
example comes from previous work analyzing knowledge gaps in readership and
contributorship (shaw2018pipeline, 129): the authors found that the gender gap
is higher in contributors than in readers, and proposed interventions
including continued efforts in female editors recruitment, as well as
awareness campaigns to increase awareness that Wikipedia is editable among
women readers.

The existence of power asymmetries in society282828For further reading, see
(d2020data, 176) are central to this taxonomy, knowledge equity (strategy,
108), and whether any particular facet is included. Wikimedia reflects the
broader world, therefore systematic inequalities in power lead to encyclopedic
knowledge being systematically biased against specific communities
(systemicbias, 261, 96, 226). Some power asymmetries are more consistent
globally—e.g., gender, urban versus rural, socioeconomic status—while others
are much more contextual and regional—e.g., race, religion. The consistency of
a power asymmetry does not make a gap more or less important, but it does
affect the ability to characterize and quantify a gap and thus its relevance
to the guiding principles of this taxonomy (Section 2.1). Addressing the gaps
listed in this manuscript is not therefore simply a matter of creating more
content or bringing in a more diverse community. Analogous to the deep body of
work discussing the relationship between societal inequalities and bias within
AI systems (west2019discriminating, 163, 93, 169, 182), addressing these gaps
in the long-term will also require addressing these societal inequalities,
along with the many, more-Wikimedia-specific barriers that drive these
gaps—see Sections 3.3, 4.3, and 5.3.

###  7.5. Expanding the Taxonomy

The taxonomy itself is also incomplete (and will always be) but it is worth
detailing a few salient gaps in the process itself:

  * •

Positionality: while we, the authors, sought to build on as wide of a range of
sources and incorporated feedback from many in the community
292929https://meta.wikimedia.org/wiki/Research_talk:Knowledge_Gaps_Index/Taxonomy,
we represent a narrow slice of backgrounds and language communities. Notably,
we generally are able-bodied, employed by the Wikimedia Foundation, and share
an academic computer science background from institutions in the United States
and Europe. Our common language is English. As such, we almost certainly
missed or misrepresented aspects that others with different backgrounds would
have incorporated differently. Addressing these gaps will require decentering
our backgrounds and centering marginalized knowledge (centeringmarginalized,
115).

  * •

Sources: while we sought to gather sources from many areas, we almost
certainly have blind spots in our literature review due to our
positionality—e.g., fields of study such as science and technology studies
(STS), languages beyond English and the others we speak. Additionally, the
research community as a whole has largely focused its study on Wikipedia, with
a growing literature about Wikidata and Wikimedia Commons, but the research is
scant about Wiktionary, Wikisource, and other projects.

  * •

Types of knowledge: while this taxonomy seeks to cover the Wikimedia projects,
there are certainly types of knowledge that sit outside of the scope of the
existing Wikimedia projects and therefore are not taken into consideration
here. For example, Wikifunctions (abstractwikipedia, 173) was established as
the newest Wikimedia project in 2020 and there are certainly other forms of
knowledge that are not covered by the existing projects – e.g., Jimmy Wales’
list of things that need to be free (thingsneedfree, 3).

  * •

Other types of contributors: the contributors gap (Section 4) is specifically
scoped to people who are editing the Wikimedia projects. There are many other
ways to contribute to Wikimedia though outside of editing content. Most
notably not included in this definition are organizers and developers.
Organizers are central to the success of the Movement and advancing equity in
participation and content through work such as campaigns or advocacy
(organizerstudy, 164)—for more information, see the Organizer Framework
(organizerframework, 208) and survey data from Community Insights
(2020communityinsights, 119). Developers build the Mediawiki infrastructure,
associated gadgets, templates, and bots that allow for editing of Wikimedia
projects. This community has a large impact on equity in the projects both
directly—e.g., bot-generated articles—and indirectly—e.g., by what power
relations they encode in the infrastructure such as acceptable or default
genders (ford2017anyone, 96). Survey data about different parts of this
community can be found in the Community Insights surveys
(2020communityinsights, 119), Wikimedia Foundation diversity report
(wmfdiversity, 165), and Cloud Services Annual Survey (developersurvey, 197).

##  8\. Acknowledgments

We would like to thank all of the affiliates, user groups, and volunteer
community members as well as the Wikimedia Foundation staff who engaged in
providing feedback to the first draft of the taxonomy through surveys,
meetings, and the Wikimedia page for the
project303030https://meta.wikimedia.org/wiki/Research_talk:Knowledge_Gaps_Index/Taxonomy.
Thanks to these contributions, this second version of the Taxonomy of
Knowledge gaps is much more solid and aligned with the spirit and strategic
directions of the Wikimedia community.

## References

  * (1) Benjamin S Bloom  “Taxonomy of educational objectives. Vol. 1: Cognitive domain”  In _New York: McKay_ , 1956, pp. 20–24
  * (2) Kevin B Smith  “Typologies, taxonomies, and the benefits of policy classification”  In _Policy Studies Journal_ 30.3 Wiley Online Library, 2002, pp. 379–395
  * (3) Wikipedia contributors  “List of things that need to be free”, 2005  URL: https://strategy.wikimedia.org/wiki/List_of_things_that_need_to_be_free
  * (4) Tsila Hassine  “The dynamics of NPOV disputes”  In _Proceedings of Wikimania_ , 2005
  * (5) Dan Cosley, Dan Frankowski, Loren Terveen and John Riedl  “SuggestBot: using intelligent task routing to help people find work in wikipedia”  In _Proceedings of the 12th international conference on Intelligent user interfaces_ , 2007, pp. 32–41
  * (6) Pew Research  “2007 Pew Research Study (US)”, 2007  URL: https://www.pewresearch.org/internet/2007/04/24/wikipedia-users/
  * (7) F B Viegas  “The Visual Side of Wikipedia”  In _2007 40th Annual Hawaii International Conference on System Sciences (HICSS’07)_ , 2007, pp. 85–85  DOI: 10.1109/HICSS.2007.559
  * (8) M Claudia Buzzi, Marina Buzzi, Barbara Leporini and Caterina Senette  “Making Wikipedia editing easier for the blind”  In _Proceedings of the 5th Nordic conference on Human-computer interaction: building bridges_ , 2008, pp. 423–426
  * (9) Joint Research Centre-European Commission and Others  “Handbook on constructing composite indicators: methodology and user guide”  OECD publishing, 2008
  * (10) Morten Rask  “The reach and richness of Wikipedia: Is Wikinomics only for rich countries?”  In _First Monday_ , 2008
  * (11) UNU-Merit  “2008 UNU-MERIT Survey (22 languages)”, 2008  URL: https://meta.wikimedia.org/wiki/Research:UNU-MERIT_Wikipedia_survey
  * (12) Brent Hecht and Darren Gergle  “Measuring self-focus bias in community-maintained knowledge repositories”  In _Proceedings of the fourth international conference on Communities and technologies - C &T ’09_ New York, New York, USA: ACM Press, 2009, pp. 11  DOI: 10.1145/1556460.1556463
  * (13) Aniket Kittur, Ed H Chi and Bongwon Suh  “What’s in Wikipedia?”  In _Proceedings of the 27th international conference on Human factors in computing systems - CHI 09_ ACM Press, 2009  DOI: 10.1145/1518701.1518930
  * (14) Shyong (Tony) K Lam and John Riedl  “Is Wikipedia growing a longer tail?”  In _Proceedings of the ACM 2009 international conference on Supporting group work_ , 2009, pp. 105–114
  * (15) WebAIM  “Survey of Preferences of Screen Readers Users”, 2009  URL: https://webaim.org/projects/screenreadersurvey/
  * (16) Hichang Cho, MeiHui Chen and Siyoung Chung  “Testing an integrative theoretical model of knowledge-sharing behavior in the context of Wikipedia”  In _Journal of the American Society for Information Science and Technology_ 61.6 Wiley Online Library, 2010, pp. 1198–1212
  * (17) Marek Ciglan and Kjetil Nørvåg  “WikiPop: personalized event detection system based on Wikipedia page view statistics”  In _Proceedings of the 19th ACM international conference on Information and knowledge management_ , 2010, pp. 1931–1932
  * (18) emijrp  “All Human Knowledge”, 2010  URL: https://en.wikipedia.org/wiki/User:Emijrp/All_Human_Knowledge
  * (19) Pew Research  “2010 Pew Research Study (US)”, 2010  URL: https://www.pewresearch.org/internet/2011/01/13/wikipedia-past-and-present/
  * (20) Ewa S Callahan and Susan C Herring  “Cultural bias in Wikipedia content on famous persons”  In _Journal of the American Society for Information Science. American Society for Information Science_ 62.10, 2011, pp. 1899–1915  DOI: 10.1002/asi.21577
  * (21) Wikimedia Foundation  “2011 Mobile Readers Survey (11 countries)”, 2011  URL: https://meta.wikimedia.org/wiki/Research:Wikipedia_Mobile_Readers_Survey_2011
  * (22) Wikimedia Foundation  “2011 Readership Survey (16 countries)”, 2011  URL: https://meta.wikimedia.org/wiki/Research:Wikipedia_Readership_Survey_2011/Results
  * (23) Wikimedia Foundation  “April 2011 Editor Surveys”, 2011  URL: https://meta.wikimedia.org/wiki/Research:Wikipedia_Editors_Survey_2011_April
  * (24) Wikimedia Foundation  “November 2011 Editor Surveys”, 2011  URL: https://meta.wikimedia.org/wiki/Research:Wikipedia_Editors_Survey_2011_November
  * (25) Mark Graham, Scott Hale and Monica Stephens  “Geographies of the World’s Knowledge”  Oxford Internet Institute, 2011
  * (26) Manuel Merz  “Research:Surveys on the gender of editors/”, 2011  URL: https://meta.wikimedia.org/wiki/Research:Projects/Wikipedia_Editor_Satisfaction_Survey
  * (27) Joseph Reagle and Lauren Rhue  “Gender Bias in Wikipedia and Britannica”  In _International Journal of Communication Systems_ 5.0, 2011, pp. 21
  * (28) Steven Walling  “Contribution Taxonomy Project”, 2011  URL: https://meta.wikimedia.org/wiki/Research:Contribution_Taxonomy_Project
  * (29) Xiaoquan Michael Zhang and Feng Zhu  “Group size and incentives to contribute: A natural experiment at Chinese Wikipedia”  In _American Economic Review_ 101.4, 2011, pp. 1601–15
  * (30) Patti Bao et al.  “Omnipedia: bridging the wikipedia language gap”  In _Proceedings of the SIGCHI Conference on Human Factors in Computing Systems_ , 2012, pp. 1075–1084
  * (31) Wikimedia Foundation  “2012 Bangla Wikipedia Survey”, 2012  URL: https://meta.wikimedia.org/wiki/Small_Wiki_Editor_Engagement_Project/Report/Monthly_(June-July_2012)
  * (32) Wikimedia Foundation  “2012 Editor Surveys”, 2012  URL: https://meta.wikimedia.org/wiki/Research:Wikipedia_Editors_Survey_2012
  * (33) Wikimedia Foundation  “Wikipedia Zero”, 2012  URL: https://en.wikipedia.org/wiki/Wikipedia_Zero
  * (34) Shane Greenstein and Feng Zhu  “Is Wikipedia Biased?”  In _The American economic review_ 102.3, 2012, pp. 343–348  DOI: 10.1257/aer.102.3.343
  * (35) Aaron Halfaker and John Riedl  “Bots and cyborgs: Wikipedia’s immune system”  In _Computer_ 45.3 IEEE, 2012, pp. 79–82
  * (36) Teun Lucassen, Roald Dijkstra and Jan Maarten Schraagen  “Readability of Wikipedia”  In _First Monday_ , 2012
  * (37) Finn Årup Nielsen  “Wikipedia Research and Tools: Review and Comments”, 2012  DOI: 10.2139/ssrn.2129874
  * (38) Chitu Okoli et al.  “The people’s encyclopedia under the gaze of the sages: A systematic review of scholarly research on Wikipedia”  In _Available at SSRN 2021326_ , 2012
  * (39) Jodi Schneider, Alexandre Passant and Stefan Decker  “Deletion discussions in Wikipedia: Decision factors and outcomes”  In _Proceedings of the Eighth Annual International Symposium on Wikis and Open Collaboration_ , 2012, pp. 1–10
  * (40) Taha Yasseri, András Kornai and János Kertész  “A Practical Approach to Language Complexity: A Wikipedia Case Study”  In _PloS one_ 7.11, 2012, pp. e48386  DOI: 10.1371/journal.pone.0048386
  * (41) Taha Yasseri, Robert Sumi and János Kertész  “Circadian patterns of wikipedia editorial activity: A demographic analysis”  In _PloS one_ 7.1 Public Library of Science, 2012, pp. e30091
  * (42) Yann Algan, Yochai Benkler, Mayo Fuster Morell and Jérôme Hergueux  “Cooperation in a Peer Production Economy Experimental Evidence from Wikipedia.”  In _Available at SSRN 2843518_ , 2013
  * (43) Heather Ford, Shilad Sen, David R Musicant and Nathaniel Miller  “Getting to the source: where does Wikipedia get its information from?”  In _Proceedings of the 9th International Symposium on Open Collaboration_ , WikiSym ’13 Article 9  New York, NY, USA: Association for Computing Machinery, 2013, pp. 1–10  DOI: 10.1145/2491055.2491064
  * (44) Wikimedia Foundation  “Visual Editor”, 2013  URL: https://www.mediawiki.org/wiki/VisualEditor
  * (45) Wikimedia Foundation  “Visual Editor Effect”, 2013  URL: https://meta.wikimedia.org/wiki/Research:VisualEditor%27s_effect_on_newly_registered_editors
  * (46) Aaron Halfaker, R Stuart Geiger, Jonathan T Morgan and John Riedl  “The rise and decline of an open collaboration system: How Wikipedia’s reaction to popularity is causing its decline”  In _American Behavioral Scientist_ 57.5 Sage Publications Sage CA: Los Angeles, CA, 2013, pp. 664–688
  * (47) Vereniging Wikimedia Nederland  “Getting to know the grassroots Vereniging Wikimedia Nederland”, 2013  URL: https://upload.wikimedia.org/wikipedia/commons/8/88/Getting_to_know_the_grassroots_Vereniging_Wikimedia_Nederland.pdf
  * (48) Marta Recasens, Cristian Danescu-Niculescu-Mizil and Dan Jurafsky  “Linguistic models for analyzing and detecting biased language”  In _Proceedings of the 51st Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers)_ , 2013, pp. 1650–1659
  * (49) The Signpost  “Making Wikipedia more accessible”, 2013  URL: https://en.wikipedia.org/wiki/Wikipedia:Wikipedia_Signpost/2013-09-04/Technology_report#Making_Wikipedia_more_accessible
  * (50) David Bamman and Noah A Smith  “Unsupervised Discovery of Biographical Structure from Text”  In _Transactions of the Association for Computational Linguistics_ 2 MIT Press, 2014, pp. 363–376  DOI: 10.1162/tacl“˙a“˙00189
  * (51) Adil Fahad et al.  “A survey of clustering algorithms for big data: Taxonomy and empirical analysis”  In _IEEE transactions on emerging topics in computing_ 2.3 IEEE, 2014, pp. 267–279
  * (52) Wikimedia Foundation  “2014 Global South User Survey (11 countries)”, 2014  URL: https://meta.wikimedia.org/wiki/Research:Global_South_User_Survey_2014
  * (53) Mark Graham, Bernie Hogan, Ralph K Straumann and Ahmed Medhat  “Uneven geographies of user-generated information: Patterns of increasing informational poverty”  In _Annals of the Association of American Geographers_ 104.4 Taylor & Francis, 2014, pp. 746–764
  * (54) Mark Graham, Bernie Hogan, Ralph K Straumann and Ahmed Medhat  “Uneven Geographies of User-Generated Information: Patterns of Increasing Informational Poverty”  In _Annals of the Association of American Geographers. Association of American Geographers_ 104.4 Routledge, 2014, pp. 746–764  DOI: 10.1080/00045608.2014.910087
  * (55) Scott A Hale  “Multilinguals and Wikipedia editing”  In _Proceedings of the 2014 ACM conference on Web science_ , 2014, pp. 99–108
  * (56) Amal Hanna  “How to motivate formal students and informal learners to participate in Open Content Educational Resources (OCER)?”  In _International Journal of Research in Open Educational Resources_ 1.1 Citeseer, 2014, pp. 1–15
  * (57) Wikimedia Pakistan  “2014 Pakistan Reader Survey”, 2014  URL: https://blog.wikimedia.org/2015/02/20/pakistani-readers-survey/
  * (58) Denny Vrandečić and Markus Krötzsch  “Wikidata: a free collaborative knowledgebase”  In _Communications of the ACM_ 57.10, 2014, pp. 78–85  DOI: 10.1145/2629489
  * (59) Ofer Arazy et al.  “Functional roles and career paths in Wikipedia”  In _Proceedings of the 18th ACM Conference on Computer Supported Cooperative Work & Social Computing_, 2015, pp. 1092–1105
  * (60) Tilman Byer  “How many women edit Wikipedia?”, 2015  URL: https://blog.wikimedia.org/2015/04/30/how-many-women-edit-wikipedia/
  * (61) Young-Ho Eom et al.  “Interactions of cultures and top people of Wikipedia from ranking of 24 language editions”  In _PloS one_ 10.3, 2015, pp. e0114825  DOI: 10.1371/journal.pone.0114825
  * (62) Wikimedia Foundation  “2015 WMF Donation Survey (US)”, 2015  URL: https://upload.wikimedia.org/wikipedia/commons/2/25/Wikimedia_Reader_Survey_November_2015.pdf
  * (63) Wikimedia Foundation  “Harassment survey 2015”, 2015  URL: https://meta.wikimedia.org/wiki/Research:Harassment_survey_2015
  * (64) Eduardo Graells-Garrido, Mounia Lalmas and Filippo Menczer  “First women, second sex: Gender bias in Wikipedia”  In _Proceedings of the 26th ACM Conference on Hypertext & Social Media_, 2015, pp. 165–174
  * (65) Reed H Harder, Alfredo J Velasco, Michael S Evans and Daniel N Rockmore  “Measuring Verifiability in Online Information”, 2015  arXiv:1509.05631 [cs.SI]
  * (66) Eszter Hargittai and Aaron Shaw  “Mind the skills gap: the role of Internet know-how and gender in differentiated contributions to Wikipedia”  In _Information, communication & society_ 18.4 Taylor & Francis, 2015, pp. 424–442
  * (67) Amanda Menking and Ingrid Erickson  “The heart work of Wikipedia: Gendered, emotional labor in the world’s largest online encyclopedia”  In _Proceedings of the 33rd annual ACM conference on human factors in computing systems_ , 2015, pp. 207–210
  * (68) Mostafa Mesgari et al.  ““The sum of all human knowledge”: A systematic review of scholarly research on the content of W ikipedia”  In _Journal of the Association for Information Science and Technology_ 66.2 Wiley Online Library, 2015, pp. 219–245
  * (69) Wikimedia NL  “2015 Wikimedia Nederland Editor Survey”, 2015  URL: https://upload.wikimedia.org/wikipedia/commons/5/5c/Report_on_survey_among_editors_of_NLWP_2015.pdf
  * (70) Shilad W Sen et al.  “Barriers to the localness of volunteered geographic information”  In _Proceedings of the 33rd Annual ACM Conference on Human Factors in Computing Systems_ , 2015, pp. 197–206
  * (71) Claudia Wagner, David Garcia, Mohsen Jadidi and Markus Strohmaier  “It’s a man’s Wikipedia? Assessing gender inequality in an online encyclopedia”  In _Ninth international AAAI conference on web and social media_ , 2015
  * (72) Morten Warncke-Wang, Vivek Ranjan, Loren Terveen and Brent Hecht  “Misalignment between supply and demand of quality content in peer production communities”  In _Ninth International AAAI Conference on Web and Social Media_ , 2015
  * (73) Rachel S Wexelbaum, Katie Herzog and Lane Rasberry  “Queering Wikipedia”, 2015
  * (74) Martina Balestra, Ofer Arazy, Coye Cheshire and Oded Nov  “Motivational determinants of participation trajectories in Wikipedia”  In _Tenth International AAAI Conference on Web and Social Media_ , 2016
  * (75) Anamika Chhabra and SRS Iyengar  “Should wikipedia and quora collaborate?”  In _2016 8th International Conference on Communication Systems and Networks (COMSNETS)_ , 2016, pp. 1–2  IEEE
  * (76) Heather Ford and Mark Graham  “Provenance, power and place: Linked data and opaque digital geographies”  In _Environment and Planning D: Society and Space_ 34.6 SAGE Publications Sage UK: London, England, 2016, pp. 957–970
  * (77) Mozilla Foundation  “Digital Skills Observatory”, 2016  URL: http://mozillafoundation.github.io/digital-skills-observatory/
  * (78) Wikimedia Foundation  “2016 New Readers Phone Surveys”, 2016  URL: https://meta.wikimedia.org/wiki/Global_Reach/Insights
  * (79) Wikimedia Community Ireland  “Wikimedia Community Ireland 2016 Editor Survey”, 2016  URL: https://meta.wikimedia.org/wiki/Research:Survey_of_user_in_Ireland
  * (80) Isaac L Johnson et al.  “Not at home on the range: Peer production and the urban/rural divide”  In _Proceedings of the 2016 CHI Conference on Human Factors in Computing Systems_ , 2016, pp. 13–25
  * (81) Marc Miquel-Ribé and David Laniado  “Cultural Identities in Wikipedias”  In _Proceedings of the 7th 2016 International Conference on Social Media & Society_, SMSociety ’16 Article 24  New York, NY, USA: Association for Computing Machinery, 2016, pp. 1–10  DOI: 10.1145/2930971.2930996
  * (82) Jonathon W Penney  “Chilling effects: Online surveillance and Wikipedia use”  In _Berkeley Tech. LJ_ 31 HeinOnline, 2016, pp. 117
  * (83) Ioannis Protonotarios, Vasiliki Sarimpei and Jahna Otterbacher  “Similar gaps, different origins? Women readers and editors at Greek Wikipedia”  In _Tenth International AAAI Conference on Web and Social Media_ , 2016
  * (84) Maya Tamir  “Why do people regulate their emotions? A taxonomy of motives in emotion regulation”  In _Personality and Social Psychology Review_ 20.3 Sage Publications Sage CA: Los Angeles, CA, 2016, pp. 199–222