# Mining the online infosphere: A survey

Sayantan Adak  Department of Computer Science and Engineering, Indian
Institute of Technology, Kharagpur, West Bengal, India – 721302  Souvic
Chakraborty  Department of Computer Science and Engineering, Indian Institute
of Technology, Kharagpur, West Bengal, India – 721302  Paramtia Das
Department of Computer Science and Engineering, Indian Institute of
Technology, Kharagpur, West Bengal, India – 721302  Mithun Das  Department of
Computer Science and Engineering, Indian Institute of Technology, Kharagpur,
West Bengal, India – 721302  Abhisek Dash  Department of Computer Science and
Engineering, Indian Institute of Technology, Kharagpur, West Bengal, India –
721302  Rima Hazra  Advanced Technology Development Center, Indian Institute
of Technology, Kharagpur, West Bengal, India – 721302††thanks: The first nine
authors have been arranged based on family names and have equal contributions.
Binny Mathew  Department of Computer Science and Engineering, Indian Institute
of Technology, Kharagpur, West Bengal, India – 721302  Punyajoy Saha
Department of Computer Science and Engineering, Indian Institute of
Technology, Kharagpur, West Bengal, India – 721302  Soumya Sarkar  Department
of Computer Science and Engineering, Indian Institute of Technology,
Kharagpur, West Bengal, India – 721302  Animesh Mukherjee  Department of
Computer Science and Engineering, Indian Institute of Technology, Kharagpur,
West Bengal, India – 721302

###### Abstract

The evolution of AI-based system and applications had pervaded everyday life
to make decisions that have momentous impact on individuals and society. With
the staggering growth of online data, often termed as the _Online Infosphere_
it has become paramount to monitor the infosphere to ensure social good as the
AI-based decisions are severely dependant on it. The goal of this survey is to
provide a comprehensive review of some of the most important research areas
related to infosphere, focusing on the technical challenges and potential
solutions. The survey also outlines some of the important future directions.
We begin by discussions focused on the collaborative systems that have emerged
within the infosphere with a special thrust on Wikipedia. In the follow up we
demonstrate how the infosphere has been instrumental in the growth of
scientific citations and collaborations thus fuelling interdisciplinary
research. Finally, we illustrate the issues related to the governance of the
infosphere such as the tackling of the (a) rising hateful and abusive
behaviour and (b) bias and discrimination in different online platforms and
news reporting.

##  1 Introduction

Online infosphere111https://en.wikipedia.org/wiki/Infosphere is the term
corresponding to the Internet becoming a virtual parallel world formed from
billions of networks of artificial life at different scales ranging from tiny
pieces of software to massive AI tools running a factory or driving a car. The
motivations for this are diverse, seeking to both help mankind and harm it.

In this article, we shall attempt to portray some of the areas that are
increasingly gaining importance in research related to the evolution of this
infosphere. In particular, we would begin with infosphere as a collaborative
platform, Wikipedia being the prime point of discussion. As a next step we
would discuss how the infosphere has influenced the evolution of scientific
citations and collaborations. Finally, we shall outline the emerging research
interest in the governance of this infosphere to eradicate discrimination,
bias, abuse and hate.

###  1.1 Infosphere as a collaborative platform

The infosphere hosts numerous collaborative platforms including question
answering sites, folksonomies, microblogging sites and above all
encyclopedias. In this survey we shall focus on Wikipedia which is one of the
largest online collaborative encyclopedia. We shall primarily discuss two of
the most important aspects of Wikipedia – (a) the quality of an article and
its indicators and (b) the collaboration dynamics of Wikipedia editors who
constitute the backbone of this massive initiative. Under the first topic we
shall identify the different features of an article like its language,
structure and stability as well as their quality [150, 53, 120]. We shall
further summarise attempts that have been made to automatically predict the
quality of an article [52, 91]. Within the second topic we shall briefly
describe various issues related to the community of editors including
anomalies, vandalism and edit wars[71, 135]. Finally, we shall talk about ways
to enabling retention of editors on the platform [54, 104, 132].

###  1.2 Infosphere shaping scientific citations and collaborations

Citations play a crucial role in shaping the evolution of a scientific
discipline. With an exponential growth of research publications in various
disciplines it has become very important for researchers and scientists to
grasp different concepts within a short period of time. We would explore how
the infosphere has influenced the growth and interaction of different
scientific disciplines over the period of last few years by investigating
several different aspects of citation networks. Our survey includes – (a) a
detailed account of how the basic sciences and the computer sciences have
interacted with each other over the years resulting in an interdisciplinary
research landscape [58, 103],(b) the temporal dynamics of citations [129], (c)
ways for assessment of article quality, and finally (d) a brief account of
anomalous citation flows.

###  1.3 Governance of the infosphere

The stupendous growth of the infosphere has resulted in the emergence of
various online communities that have massively started infusing bias,
discrimination, hatred and abuse often resulting in violence in the offline
world. In this segment, we shall primarily focus our discussion on the
following topics – (a) analysis, spread, detection and mitigation of online
hate speech and (b) biases that manifest across news media and in traditional
recommendation systems. Within the first topic we shall motivate the need to
tackle online hate speech by citing some of the adverse consequences of the
same. In particular, we shall see how unmoderated hate speech spreads in a
social network [93, 94], what are the challenges to automatically detect
online hate speech [49, 2] and the possible techniques to combat this problem
[95, 96]. Within the second topic we shall discuss two important forms of
biases. The first one corresponds to political biases that manifest due to the
massive production of unverified (and in many cases false) news generated in
the form of news/blog/tweets etc. We shall also discuss the difficulties faced
by modern machine learning techniques in preventing the infusion of such
biases. The second one narrates the idea of formation of filter bubbles [107]
in traditional recommendation systems followed by a discussion on the need to
systematically audit such systems [28, 27].

##  2 Wikipedia as a collaborative platform

Wikipedia models a hypertext collaborative platform along with an open and
free knowledge base catering a large variety of information ranging from
history, arts, culture, politics to science, technology and many more fields.
Being one of the most widely viewed sites (within top ten) in the world since
2007, it spans over 208 languages with a copious amount of articles in each
edition. For example, the English-language Wikipedia, the largest in volume,
contains more than 6 million articles as of February, 2020. Owing to the
collaborative nature and an open-access policy announced by Wikipedia as
“anyone can edit”, a number of challenges have cropped up in maintaining the
veracity-quality balance of the content. Although Wikipedia has enforced
several rules and strict administrative policies to protect the encyclopedia
from malicious activities, a lack of authoritative vigilance prohibits its
trustworthiness in academics. In contrast, Wikipedia’s structured, complete
and detailed evolution history receives increasing attention of the research
community in discovering automated solution (e.g. bots, software, API etc.) to
meet the goal of quality management.

###  2.1 Quality

The elementary purpose of Wikipedia is free, unbiased, accurate information
curation. To achieve this objective Wikimedia foundation which is the
governing body of the platform, has developed labyrinth of guidelines which
editors are expected to follow so that highest encyclopedic standards are
maintained. These guidelines also enhance accessibility of the Wikipedia
articles to a broad community of netizens. We enumerate these guidelines into
three categories as discussed below.

####  2.1.1 Article language

Expressions describing a subject should be neutral. Promotion bearing words
such as renowned, visionary, iconic, virtuoso etc. should not be used. Subject
importance should be demonstrated using facts and
attribution222https://en.wikipedia.org/wiki/Wikipedia:Manual_of_Style/Words_to_watch
Prose should have active voice. Jargon needs to be elaborated or substantiated
with reference. Any effort to propagate myth or contentious content should be
curtailed. An example for this is addition of prefix pseudo or suffix -gate
which encourages the reader to assume that the subject is factitious or
scandalous respectively. Euphemisms (e.g., passed away, collateral damage and
cliches (e.g., lion’s share, tip of the iceberg) disallows presentation of
prose directly and hence is restricted. Any unnecessary emphasis in the form
of italics, quotations etc. is discouraged. For a complete list of details of
content guidelines for the English Wikipedia we refer the reader to the
Wikipeida manual of
style333https://en.wikipedia.org/wiki/Wikipedia:Manual_of_Style.

####  2.1.2 Article structure

These guidelines include proper formatting of the Wikipedia article in terms
of section headings, infobox, article name, section organisation etc. The lead
section should not be of arbitrary length. The following sections should not
be exorbitant in size and bigger sections should be broken into coherent
smaller sections. Another requirement is proper positioning of the images with
captions and references. In order to alleviate manual labor in improving
article structure there have been some automated approaches leveraging
advances in machine learning techniques [63].

####  2.1.3 Article stability

These guidelines denote stability of the article, i.e., the respective article
should not be subject of frequent edit wars. There should not be abusive
language exchange among editors and discussions toward improving article
quality should organically reach consensus. This is the most difficult
objective in collaborative content creation and generally the onus lies in the
hands of senior level editors and moderators for smooth conflict arbitration.

####  2.1.4 Peer review framework

Although Wikipedia has grown significantly in terms of volume and veracity
over the last decade, the quality of articles is not uniform [138]. The
quality of Wikipedia articles is monitored through a rating system where each
article is assigned one of several class indicators. Some of the major article
categories are FA, GA, B, C, Start and Stub. Most complete and dependable
content is annotated by an FA (aka featured article) tag while the lowest
quality content is annotated with a Stub tag. The intention behind this
elaborate scheme is to notify editors regarding current state of the article
and extent of effort needed for escalating to encyclopedic
standards444wiki/Wikipedia:WikiProjectWikipedia/Assessment. The editors are
expected to rigorously follow the aforementioned guidelines. As has been
evident from the guidelines, they are circuitous and often require experience
for implementation. Such strict policy adherence have also been sometimes a
barrier for onboarding of new editors on Wikipedia which has led to the
decline of newcomers over the past decade [132, 55]. Since it is nontrivial to
discern qualifying differences between articles manually, it has given rise to
the emergence of automated techniques using machine learning models.

####  2.1.5 Computational methods for quality prediction

Automatic article assessment is one of the key research agendas of the
Wikimedia foundation555www.mediawiki.org/wiki/ORES. One of the preliminary
approaches [56] seeking to solve this problem extracted structural features
such as presence of infobox, references, level 2 headings etc. as indicators
of the article quality. [26] proposed the first application of deep neural
networks into quality assessment task where they employed distributional
representation of documents [80] without using manual features. The authors in
[123] introduce a hybrid approach, where textual content of the Wikipedia
articles are encoded using a BILSTM model. The hidden representation captured
by the sequence model is further augmented with handcrafted features and the
concatenated feature vector is used for final classification. [151] is an edit
history based approach where every version of an article is represented by
$17$ dimensional handcrafted features. Hence, an article with $k$ versions
will be represented by $k\times 17$ matrix. This $k$ length sequence is passed
through a stacked LSTM for final representation used in classification. [124]
proposed a multimodal information fusion approach where embeddings obtained
from both article text as well as html rendering of the article webpage is
used for final classification. [52] proposed the first approach which
incorporates information from three modes for quality assessment, i.e.,
article text, article image and article talk page. [52] obtains $8\%$
improvement over [125] approach and achieves the SOTA result. A complementary
direction of exploration has been put forward by [84, 31] where correlation
between article quality and structural properties of co-editor network and
editor-article network has been exploited. An orthogonal direction of research
looks into edit level quality prediction which is a fine-grained approach
toward article content management [120].

###  2.2 Collaboration among editors

The workhorse behind the success story of Wikipedia is the large pool of its
voluntary editors; an encouragement toward global collaboration influences
people to contribute on almost all wikipages. These group of people maintain
Wikipedia pages behind the scenes which includes creating new pages, adding
facts and graphics, citing references, keeping the wording and formatting
appropriate etc. to lead the articles to the highest level of quality. The
achievement of any open collaborative project is hinged on the continued and
active participation of its collaborators, and hence, Wikipedia needs to
manage its voluntarily contributing editor community carefully. In the days of
extreme socio-cultural polarization, algorithmically crafted filter bubbles
and fake information represented as facts, editors are highly motivated to
contribute to the largest non-biased knowledge sharing platform although their
works are not financially compensated most of the times [88]. In these lines
there have been several works [145, 104], which attempt to understand the
dynamics of interaction behaviours of the community in sustaining the health
of Wikipedia.

####  2.2.1 Anomaly

While investigating the editing behaviours of editors in general context,
researchers have found out a taxonomy of semantic intentions behind the edits,
and conflicts and controversy are inherent components of the classification.
Wikipedia owes its success for several reasons and openness is one of those
pillars. Sometimes, the very openness misguides editors to violate Wikipedia’s
strict guidelines of the neutral-point of view (NPOV), and their disruptive
edits cause various kinds of anomalies. We describe the two dominant disputes,
produced by the damaging edits as follows.

Vandalism: With the freedom of editing anything by anyone, Wikipedia has to
struggle in stopping the malicious practice of contaminating articles by bad
faith edits intentionally. The popular pages like famous celebrities,
controversial topics etc. become the frequent targets of vandalism where
vandals try to mislead the readers by addition, deletion or modification,
which can be termed as hoax. Wikipedia has enforced several strict policies
such as blocking and banning Vandals (registered / unregistered editors),
patrolling recent changes by adding watch-lists, protecting articles (ex,
semi-protected pages) from new editors, random IP addresses etc. In addition
to the administrative decisions, bots
666https://en.wikipedia.org/wiki/User:ClueBot_NG are employed to detect and
revert the vandalism automatically and finally warn the editors without human
intervention. Researchers have proposed various automated ways [131, 71],
i.e., the state-of-the-art techniques based on machine learning [75, 133] and
deep neural methods [92, 135] in preventing Wikipedia from vandalism.

Edit war: Apart from the intended malpractice of vandalism, editors often
engage themselves in disagreement which further influence them to override
each other’s contribution instead of dispute resolution. Any such actions
violating the three-revert
rule777https://en.wikipedia.org/wiki/Wikipedia:Edit_warring is coined as edit
warring in Wikipedia and it promotes a toxic environment in the community.
Ultimately in the long-term, the integrity of the encyclopedia will be
affected significantly by the damaging effects of edit wars [117]. Although
Wikipedia encourages editors to be bold, in contrast a constant refusal to get
the point is also not entertained.

####  2.2.2 Retention of editors

Historically, Wikipedia managed the numbers of its volunteers quite
successfully; however, experts [54] note that it is at the danger of sharp
decline of its active editors due to the lack of the socialization effort.
Editors may choose to leave the platform for personal reasons as well as for
their disagreement/conflict with their fellow editors. The damage is happening
in both ways - when new editors fail to inherit the rules and policies they
easily become upset and leave eventually. Experienced editors, on the other
hand, can get discouraged because of the continuous upgradation of policies to
retain newcomers, or even for the nuisances by the newbies. Two way effort are
being taken to combat with this problem – researchers are coming up with
various approaches (see [101, 102, 147] and the refereces therein) while
Wikipedia itself is running several
wikiprojects888https://en.wikipedia.org/wiki/Wikipedia:WikiProject_Editor_Retention,999https://en.wikipedia.org/wiki/Wikipedia:Expert_retention
to proactively retain its contributors.

Future directions: Due to the enormous volume of data publicly available from
various multilingual wikiprojects, several interesting future directions can
be explored. One of the directions is combating repeat abusers who add
malicious content annonymously after being blocked through sockpuppetry [90]
or collusion. A large volume of effort has been invested in understanding
editor behaviour; however, similar large scale exploration need to be done on
understanding readers. A promising start in this direction includes the
following [65, 115]. However, further work needs to explore the interplay
between editors and readers and how these two stakeholders can forge a
partnership in mutually beneficial fashion. We refer to the mediawiki research
index 101010https://meta.wikimedia.org/wiki/Research:Index for a comprehensive
take on this emerging research scope.

##  3 Recent trends in citation dynamics

Research on citation network have always remained essential in solving various
problems such as predicting emerging topics, early citation prediction,
modelling evolving citation networks. Citation network is a directed graph
where nodes could be authors/papers/journals and edges are the citation flows
(weighted/unweighted) from one node to another node. Using citation networks,
one can predict which field/topic could be the ‘most attractive ones to work
on’ in the immediate future years. The research dynamics in various fields
over the years can be analyzed with the help of the underlying citation
network. Various studies uncover the chances of the manifestation of certain
new fields/sub-fields by investigating the citation flows from papers of one
field to the papers of another field over the time. In recent years, citation
count prediction task plays an important role in fund allocations and rewards.
Researchers are also interested in building models for automatic citation
recommendation while drafting an article. Apart from these, some anomalous
practices in exchanging citations have been exposed in the late ’90s. Now,
such malpractices are becoming more common among the researchers/journals
(mostly low ranked). In the rest of this section, we shall discuss each of the
above issues in details.

###  3.1 Interdisciplinary research in terms of citation interactions

Various research questions such as “which field will collaborate with which
field in future?”, “Which field will receive more citations from recently
published papers?”, etc., can be addressed with the help of the underlying
citation networks among the articles. Nowadays, research is performed by
combining the ideas from multiple disciplines. In [58] the authors have
analyzed the interdisciplinarity among the two basic science fields –
Mathematics and Physics and one fast growing field – Computer Science. Further
they observe how the citation from papers of one discipline flows to the
papers of another discipline over the years. They observe that in initial
years huge amount of citation flows from Physics to Mathematics and vice
versa. Over the years, Computer Science started gaining citation from
Mathematics. In the recent years, both the basic science fields tend to
massively cite papers from Computer Science. They observe how popularity of
some topics decreases over the time. They found that the Computer Science
papers mostly cites the quantum physics sub-field for long time span. In late
’90s, Physics mostly cites information theory papers of Computer Science but
in recent times it mostly cites papers from machine learning and social &
information networks domain.

Further, interdisciplinarity has been studied in different fields including
biology [103], mathematics [103], cognitive science [9, 76], social science
[109], humanities [109]. Various studies [7, 121, 9] have attempted to propose
novel metrics to measure the degree of interdisciplinarity based on
researchers’ scientific impact, collaborator’s knowledge, publication history,
etc. In addition, metric for measuring interdisciplinarity of an article has
been proposed [9] where authors’ research area, publications in different
domains have been used to define the metric. A study [126] has been carried
out to analyze the interdisciplinarity nature of highly cited articles from
Thomson Reuters’ databases published in between 1900-2012 years.

###  3.2 Analyzing and modelling the citation dynamics

Several studies have been carried out in the past to model the temporal
dynamics of citation networks. In order to model the temporal dynamics of
citation networks, researchers traditionally used preferential attachment [1]
and copying based [74] models.

In [137], the authors investigated the citation behavior of older papers in
various fields. They concluded that older articles receive more citations over
the years. It is observed that in 2013, 36% of citations flowed toward the at
least ten years old papers. However, a re-investigation of this study showed
that the observations are only partly true since the authors did not take into
account the accelerating volume of publications over time. In order to tackle
the tug-of-war between obsolescence and entrenchment, recently, in [129], the
authors proposed a complex model based on the idea of relay-linking where the
older article relays a citation to a recently published article. This model
has very less number of parameters and fits with the real data much better
than the traditional models. Yet another novel citation growth model called
RefOrCite [106] have been proposed recently where the authors allow copying
from the references (out-edges) and citations (in-edges) of an article (as
opposed to only references in the traditional setup). It is observed that
RefOrCite model fits well with real compared to the previous models.

### Citation count prediction

Predicting future impact of scientific articles is important for making
decision in fund allocation (by funding agencies), recruitment etc. There are
various works [82, 143, 149, 83] that have been carried out in the past to
automatically estimate the citation count of scientific articles. In this
article, we shall mainly focus on the recent literature. In 2015, the authors
in [82] proposed Trend-based Citation Count Prediction (T-CCP) model where the
model first would first learn the type of the citation trends of the articles
and then predict the citation count for that trend. All the articles were
categorized into five citation trend categories based on the “burst” time
(“burst time” is the time when the paper gets maximum citations) – early
burst, middle burst, late burst, multi bursts, and no bursts. Two types of
features have been used – (a) publication related features like author centric
features (i.e., h-index, number of papers published, citation count, number of
collaborators), publication venue (average citation count, impact factor etc.)
and (b) reinforcement features which are the graph based features (i.e.,
PageRank, HITS etc.) calculated from weighted citation network among authors.
In their model, they mainly use SVR and SVM (LibLinear) for citation count
prediction and classification task respectively. In paper [128], the authors
found that the knowledge gathered from citation context within the article
could help to predict future citation count. Number of occurrences of the
citations for a paper within the article and the average number of words in
citation context, have been derived from citation context knowledge. Further
they categorized the articles into six citation profiles (PeakInit, PeakMul,
PeakLate, MonDec, MonIncr, Oth) and found that the above two citation context
based features are able to nicely distinguish these six categories. In [127],
the authors observed that the long term citation of an article depends on the
citations it receives in the early years (within one or two years from its
publication date). The authors who cite an article in its early years are
called early citer. Early citers based on whether they are influential or not
affect the long term citation count of the article. In most cases, influential
authors negatively affect the long term citation of an article. In [143], the
authors have proposed a novel point process method to predict the citations of
individual articles. In their approach they tried to capture two properties –
the “rich gets richer” effect and the recency effect. The authors in [149]
used four factors – intrinsic quality (citation count) of a paper, aging
effect, Mathew effect and recency effect to derive a model called long term
individual level citation count prediction (LT-CCP). In this model they mainly
use RNN with LSTM units. It is observed that LT-CPP model achieves better
performance than existing models. Authors in [83] proposed a neural model for
predicting citation count with the help of peer review text. They mainly learn
two deep features – (a) the abstract-review match mechanism (in order to learn
the abstract aware review representation) and (b) the cross review match from
peer review text.

###  3.3 Citation recommendation

Often new researchers face difficulties in finding appropriate published
research papers while exploring the domain literature and citing published
papers. Citation recommendation is a technique that recommends appropriate
published articles for the given text/sentence. The sentences present around
the reference (placeholder) are called context sentences. Citation
recommendation task can be divided into two parts – (i) local citation
recommendation, and (ii) global citation recommendation. In case of local
citation recommendation, only the context sentences are used. In case of
global citation recommendation, the whole article is used as input and the
system outputs a list of published papers as output. In cite [10] the authors
proposed a model for the global citation recommendation task where they
embedded the textual information (i.e., the title and the abstract) of the
candidate citations in a vector space and considered the nearest neighbors as
the candidate citations for the target document. Further, re-ranking of the
candidate citations was done. They used DBLP (50K articles having an average
citation of 5 per article) and PubMed (45K articles with average citation of
17 per article) datasets and also introduced a new dataset OpenCorpus (7
million articles) in the paper. They showed that their model achieved state-
of-the-art performance without using metadata (authors, publication venues,
keyphrases). In paper [64], the authors proposed a deep learning model
(consists of context encoder and citation encoder) and used a dataset [59] for
context aware citation recommendation. Pre-trained BERT [34] model has been
used in order to learn the embedding of the context sentences. GCN has been
employed to learn the citation graph embedding from the paper-paper citation
graph. They mainly revised two existing datasets – AAN and FullTextPeerRead
(revised version of PeerRead). They showed that their model performed three
times better than the SOTA approaches (CACR etc.). The authors in [110]
proposed a novel method – ConvCN – based on the citation knowledge graph
embedding.

###  3.4 Detection of anomalous citation flows

Various anomalous citation patterns have been found to emerge over the years.
Various ways of maliciously increasing one’s citation are through self-
citations, citation stacking among journals, and citation cartel. Nowadays,
authors are more concerned about their position in academia, publication
pressure etc. and this leads to most of them adopting unfair means to increase
their citation. Citation cartel is one of the anomalous citation patterns
which was first reported in late
’90s111111https://science.sciencemag.org/content/286/5437/53. Citation cartel
is formed by a group of authors/editors/journals where they cite each other
heavily for mutual benefit. The relationship in citation cartel could be
author-author, editor-author, journal-journal etc. There are a few cases found
where the journal’s impact factor increases rapidly due to this anomalous
behavior. Cell
Transplantation121212https://www.cognizantcommunication.com/journal-
titles/cell-transplantation is a medical journal whose impact factor rapidly
increased between 2006 and 2010 (3.48 to 6.20). After investigation carried
out by JCR publisher, it was found that one review article published in this
journal Medical Science Monitor131313http://www.medscimonit.com/ cited almost
91% papers published in Cell Transplantation from the time bucket 2008–2009.
It was found that the impact factor of the journal Cell Transplantation was
calculated based on this time
bucket141414https://scholarlykitchen.sspnet.org/2012/04/10/emergence-of-a-
citation-cartel/. Surprisingly, the authors (three out of four) are from the
editorial board of this journal. In cite [40] the authors tried to detect
citation cartels. They defined a citation cartel as a group of authors citing
each other excessively than they do with other authors’ works in the same
domain. They observed that there could be multiple reasons like academic
pressure, “publish or perish” concept in academia, fear of losing job,
scientific competition etc. behind establishing such citation cartels. It was
observed that such unfair means are mostly adopted by low ranked researchers
[41]. In their work, they prepared a multilayer graph where they include
paper-paper citation network (directed graph), authors’ collaboration network
and authors’ citation networks (weighted directed graph). Finally, citation
cartel has been captured from the authors’ citation network. Cartels have been
discovered by using Resource Description Framework (RDF) and RDF query
language and some threshold has been declared to identify the existence of
citation cartel among authors. The authors in [72] proposed a novel algorithm
– Citation Donors and REcipients (CIDRE) to detect the citation cartel among
the journals that cite each other disproportionately to increase the impact
factor of the journal. CIDRE algorithm first distinguishes between the normal
and malicious citation exchange with the help of few parameters. These
parameters are similarity in research areas, citation inflow and outflow. A
weighted citation network among 48K journals was constructed from the dataset
collected from MAG151515https://www.microsoft.com/en-
us/research/project/microsoft-academic-graph/. With the help of the algorithm,
more than half of the malicious journals were detected (those were actually
suspended by Thomson Reuters) in the same year. In addition, CIDRE algorithm
detected few malicious journal groups in 2019 whose journals received 30% of
its in-flow citation from the journals in the same group. Such anomalous
citations help to grow the impact factor of the journals over the years. In
[62], the authors studied how malicious journals are increasing in the Indian
research community and avoiding proper rules and regulations. The analysis has
been carried out on Indian publishing group OMICS (considered as predatory by
the research community). Surprisingly they observed that such malicious
journals share very similar characteristics with various reputed journals.

Future directions: In order to gather more citations, malpractices among the
journals are rapidly increasing. More research is required to build a
mechanism which can automatically predict those (predatory) journals
(depending on the topics of the journal). In case of citation recommendation,
there is a need for improving the recommendation system such that the system
is able to recommend papers that are conceptually similar or exhibit
conflicting claims [42]. Also, prioritizing the citation recommendation would
be another help to maintain the page limit given by many conferences [42].

##  4 Governance

As noted in the introduction, this section is laid out into two major parts.
The former part centers around the spread, automatic detection and containment
of hate speech. The latter part deals with bias in media outlets and online
recommendation platforms.

###  4.1 Hate speech

####  4.1.1 Spread

The Internet is one of the greatest innovations of mankind which has brought
together people from every race, religion, and nationality. Social media sites
such as Twitter and Facebook have connected billions of
people161616https://techcrunch.com/2018/07/25/facebook-2-5-billion-people and
allowed them to share their ideas and opinions instantly. That being said,
there are several ill consequences as well such as online harassment,
trolling, cyber-bullying, and _hate speech_.

The rise of hate speech: Hate speech has recently received a lot of research
attention with several works that focus on detecting hate speech in online
social media [29, 33, 4, 119, 73]. Even though several government and social
media sites are trying to curb all forms of hate speech, it is still plaguing
our society. With hate crimes increasing in several
states171717http://www.aaiusa.org/unprecedented_increase_expected_in_upcoming_fbi_hate_crime_report,
there is an urgent need to have a better understanding of how the users spread
hateful posts in online social media. Companies like Facebook have been
accused of instigating anti-Muslim mob violence in Sri Lanka that left three
people dead181818https://www.theguardian.com/world/2018/mar/14/facebook-
accused-by-sri-lanka-of-failing-to-control-hate-speech and a United Nations
report blamed them for playing a leading role in the possible genocide of the
Rohingya community in Myanmar by spreading hate
speech191919https://www.reuters.com/investigates/special-report/myanmar-
facebook-hate. In response to the UN report, Facebook later banned several
accounts belonging to Myanmar military
officials202020https://www.reuters.com/article/us-myanmar-facebook/facebook-
bans-myanmar-army-chief-others-in-unprecedented-move-idUSKCN1LC0R7 for
spreading hate speech. In the recent Pittsburgh synagogue shooting212121
https://en.wikipedia.org/wiki/Pittsburgh_synagogue_shooting, the sole suspect,
_Robert Gregory Bowers_ , maintained an account (@onedingo) on
GabLABEL:https://gab.com/ and posted his final message before the
shooting222222https://www.independent.co.uk/news/world/americas/pittsburgh-
synagogue-shooter-gab-robert-bowers-final-posts-online-comments-a8605721.html.
Inspection of his Gab account shows months of anti-semitic and racist posts
that were endorsed by a lot of users on Gab.

Understanding the spread of hate speech: We perform the first study which
looks into the diffusion dynamics of the posts by hateful users in Gab [93].
We choose Gab for all our analysis. This choice is primarily motivated by the
nature of Gab, which allows users to post content that may be hateful in
nature without any fear of repercussion. This provides an unique opportunity
to study how the hateful content would spread in the online medium, if there
were no restrictions. To this end, we crawl the Gab platform and acquire 21M
posts by 341K users over a period of 20 months (October, 2016 to June, 2018).
Our analysis reveals that the posts by hateful users tend to spread faster,
farther, and wider as compared to normal users. We find that the hate users in
our dataset (which constitute 0.67% of the total number of users) are very
densely connected and are responsible for 26.80% of posts generated in Gab.

We also study the temporal effect of hate speech on the users and the platform
as well [94]. To understand the temporal characteristics, we needed data from
consecutive time points in Gab. As a first step, using a heuristic [98], we
generate successive graphs which capture the different time snapshots of Gab
at one month intervals. Then, using the DeGroot model [32], we assign a hate
intensity score to every user in the temporal snapshot and categorize them
based on their degrees of hate. We then perform several linguistic and network
studies on these users across the different time snapshots. We find that the
amount of hate speech in Gab is consistently increasing. This is true for the
new users joining as well. We further find that the recently joining new users
take much less time to turn hateful as compared to those that joined at
earlier time periods. In addition, the fraction of users becoming hateful is
increasing as well. Also, we find that the language used by the community as a
whole is becoming more correlated with that of the hateful users as compared
to the non-hateful ones. The hateful users also seem to be playing a pivotal
role from the network point of view.

####  4.1.2 Detection

Due to the massive scale of online social media, methods that automatically
detect hate speech are required. In this section, we explore a few of the
methods that try to automatically detect hate speech. Some of the approaches
utilize Keyword-based techniques, machine learning models, etc. Perceiving the
right features for a classification problem can be one of the challenging
tasks when using machine learning. Though surface-level features, such as bag
of words, uni-grams, larger n-grams, etc. [18, 136] have been used for this
problem, since hate speech detection is usually applied on small pieces of
text, one may face a data sparsity problem. Lately, neural network based
distributed word/paragraph representations, also referred to as word
embeddings/paragraph embeddings [36] have been proposed. Using large
(unlabelled) text corpus, for each word or for a paragraph a vector
representation is induced [81] that can eventually be used as classification
features, replacing binary features indicating the presence or frequency of
particular words.

Hate speech detection is a task that cannot always be solved by using only
lexicon based features/ word embedding. For instance, ‘6 Million Wasn’t
Enough‘ may not be regarded as some form of hate speech when observed in
isolation. However, when the context is given that the utterance is directed
toward Jewish people who were killed in the holocaust by white supremacists
and Neo Nazis232323https://www.vice.com/en/article/y3z9aj/a-national-guard-
twitch-streamer-said-6-million-wasnt-enough-on-stream, one could infer that
this is a hate speech against Jews. The above example shows us whether a
message is hateful or not can be highly dependent on world knowledge. In [46],
the authors annotated context dependant hate speech and showed that
incorporating context information improved the overall performance of the
model.

Apart from world knowledge, meta-information (i.e., background information
about the user of a post, number of posts by a user, geographical origin) can
be used as a feature to improve the hate speech detection task. Since the data
commonly comes from the online social media platform, variety of meta-
information about the post can be collected while crawling the data. A user
who is known to post mostly hateful content, may do so in the future. Existing
research [93] has found that, high number of hateful messages are generated
from less number of users. It has been also observed that men are more likely
to spread hate speech than women [139]. Also, the number of profane words in
the post history of user has been used as a feature for hate speech
classification task [24].

Nowadays the number of posts which consists of images, audios, and video
content are getting shared more in Social media platforms. In [49], the
authors have explored the textual and visual information of images for the
hate speech detection task.

Most of the methods that have been explored earlier were supervised and
heavily dependant on the annotated data. Off-the-shelf classifiers such as
logistic regression, support vector machines have been extensively used.
Recently, deep neural models are being extensively used for the classification
task. In [2], the authors explored several models such as CNN-GRU, BERT, mBERT
for the classification task in 9 languages and observed that with increasing
training data, the classifier performance increased. Another key observation
they made was that the relative performance seemed to vary depending on the
language and the model. While better models for hate speech detection are
continually being developed, little study has been done to understand the bias
and interpretability aspect of hate speech. In [97], the authors release a
benchmark dataset which could be used to study this. In their work, they show
that the models that perform very well in classification do not score high on
explainability metrics.

####  4.1.3 Counter speech

After detection of hate speech, we need proper mitigation strategies to stop
it from becoming viral. The current methods largely depend on blocking or
suspending the users, deleting the tweets etc. This is performed mostly by
moderators which is a tedious task for them given the information rate. Many
companies like Facebook have started to automate this process but both these
methods have the risk of violating free speech. This work [69] identifies
various pitfalls with respect to trust, fairness and bias of these algorithms.
A more promising direction could be to counter with speech which are popularly
known as counter speech. Specifically, counter speech is a direct non-hostile
response/comment that counters the hateful or harmful speech [116]. The idea
that ‘more speech’ is a remedy for dangerous speech has been familiar in
liberal democratic thought at least since the U.S. Supreme Court Justice Louis
Brandeis declared it in 1927. There are several initiatives with the aim of
using counter speech to tackle hate speech. For example, UNESCO released a
study [44] titled ‘Countering Online Hate Speech’, to help countries deal with
this problem.

The frameworks for mitigating hate speech using counter speech involves two
school of thoughts. One of them will be to develop fully automatic counter
speech generation system, which can output contextually relevant counter
speech given a hate speech. Since generating contextual replies to a text is
still a nascent area in natural language processing, generating counter speech
is expected to be further difficult for AI systems due to the variety of
socio-political variables present in them. Hence, a more practical approach
could be to find a task force of moderators who can suitably edit system
generated counter speech for large scale use.

One of the earliest computational studies attempted to identify hate and
counter users on Twitter and further observe how they interact in online
social media [95]. The authors created a small dataset of hate-counter reply
pairs and observed how different communities responded differently to the hate
speech targeting them. The paper further tried to build a machine learning
model to classify a user as hate or counter. A followup study [96] on YouTube
comments was conducted further to understand how different communities
attempted to respond to hate speech. Taking the YouTube videos that contain
hateful content toward three target communities: Jews, African-American
(Blacks) and LGBT, the authors collected user comments to create a dataset
which contained counter speech. The dataset is rich since it not only has the
counter/non-counter binary classes but also a detailed fine-grained
classification of the counter class as described in [142] with a slight
modification to the ‘Tone’ Category. The authors observed that the LGBT
community usually responded to hate speech via “humour” whereas the Jew
community used messages with a “positive tone”. The work further adds a
classification model which can identify a text as counter speech and its type.
The authors in [111] generated a large scale dataset having hate and their
counter replies. They further used this dataset for counter speech generation
using seq2seq models and variational autoencoders. One of the limitations of
this paper was that the counter speech data annotated through crowd-sourcing
were very generic in nature. Hence, a later work [21] took help from experts
from and NGO to curate a dataset of counter speech toward Islamophobic
statements from the social media. While this dataset provides diverse and to
the point reply to hate speech, it largely depends on the experts
availability. These challenges were compiled in [134], where the authors
showed how data collection and counter speech generation is dependant on the
assistance from the experts. The former paper also highlighted the weakness of
the generation models with around 10% of the automatic responses being proper
response to the given hate speech. This reinstates the fact that current
generation systems are not capable of understanding the hidden nuances
required to generate proper counter speech.

Another important question that lurks around in the research community is the
“effect of counter speech”. While in the case of banning or suspension the
effect, i.e., removal of tweet/user is visible, the effect of counter speech
is rather subjective in nature. In a recent work [47], the authors used a
classifier to identify 100,000 hate-counter speech pairs and found reduction
of hate speech due to the organised counter speech. While whether this solely
was caused by the counter speech or some other reason is still an open
question to the research community.

Overall the counter speech research shows promise but has several unanswered
questions about its data collection, execution and effect. Nevertheless,
fighting hate speech in this way has some benefits: it is faster, more
flexible and responsive, capable of dealing with extremism from anywhere and
in any language and it does not form a barrier against the principle of free
and open public space for debate. We hope gradually by using counter speech,
it will be slowly possible to move toward an online world where differences
could exist but not divisions.

Future directions:  Increased polarization seems to be spreading hate speech
more. Most of the current models have been developed for English language.
There is a need for larger and better hate speech datasets for other languages
as well. Using transfer learning for improving the task is another direction.
Zero or few shot learning would allow models to be able to build models for
low resource languages. Finally, an orthogonal but very interesting direction
is to understand the complexity of the annotation task itself. Due to the
subjective nature of the task, the perception of hate speech is different for
people belonging to different demographics. Another important direction is to
integrate the detection and counter systems to build an end-to-end framework
for effective hate speech detection and countering mechamism, as shown in
figure 1.

Figure 1: An overview of the hate speech framework.

###  4.2 Bias and fairness

As outlined in the introduction, in this section we shall talk about two types
of important biases (out of many more that dwells in the online world) – bias
in news reporting and bias in online recommendation systems.

####  4.2.1 Media bias

As we hit 53.6% Internet penetration worldwide, compounded by an exponential
growth in the number of social media users in the developing countries fuelled
by cheap data-rates and smartphone-based-
accessibility242424https://economictimes.indiatimes.com/tech/internet/internet-
users-in-india-to-reach-627-million-
in-2019-report/articleshow/68288868.cms?from=mdr, the news media continues to
play a significant role in shaping political discourse and influencing
national priorities. Of late, while on one hand, it has become easier to
produce news without adequate references, on the other hand, most newsreaders
share news without any
verification252525https://www.washingtonpost.com/news/the-
intersect/wp/2016/06/16/six-in-10-of-you-will-share-this-link-without-reading-
it-according-to-a-new-and-depressing-study/ [43]. In many instances, even the
mainstream media houses have been accused of copying and distributing news
from other media houses with little or no verification
262626http://archiwum.thenews.pl/1/10/Artykul/280476. While being informed
about news from sources other than direct correspondents is a common practice,
distribution of that news without verification indeed is a worrying trend. In
many cases, this has led to fake news propagation by the most reputed media
moguls. So, quantifying media bias and defining the abstract idea of bias in
this context is an important area of research.

Genesis: In Manufacturing Consent [60], 1988, Edward Herman & Noam Chomsky saw
news media as the propagandist which will find ways to propagate the
“filtered” message of the rich and powerful to the ordinary masses. Sooner or
later in any system, they hypothesize that the news medium will get
concentrated in the hands of a few people of power and money and will get
manipulated either by ownership or by filtering out news not beneficial for
the people in power. Following this model, researchers have hypothesized
different kinds of biases like there have been numerous studies examining bias
in media especially in the US and the European context. While the term “bias”
still remains abstract, some studies have put efforts to make a distinction
between the computational sense of bias and the journalistic sense of the same
making it more scientifically definable and quantifiable. Journalistic and
linguistic studies mostly discuss selection/coverage bias,
confirmation/statement bias [78, 86, 105, 118] and psychological/cognitive
biases [15, 112]. Recently, a lot of work is being done where the researchers
are interested to formulate a computational basis for investigating bias. Some
works are focused on specific kinds of bias, such as gender [11, 89, 152], and
race [20]. Politics, in particular, is a widely studied and discussed topic.
Researchers seek to find ideological political bias of users in social
networks [22, 66, 141], news media [5, 12, 77, 79, 113] and user comments
[114, 148]. D’Alessio and Allen [25] list three kinds of media bias to be the
most widely studied: Coverage/visibility bias [37], gatekeeping
bias/selectivity [61] or selection bias [51] (sometimes referred to as agenda
bias [37]) and statement bias/tonality bias/presentation bias [37, 51].

Document level bias in reporting: A news article may choose to cover some
aspects of one news and filter other aspects to bias the sentiments of the
readers toward/against a specific political party or interest group.
Researchers have annotated such sentiment leanings of news articles at the
article level [70, 12] or the sentence level [85] building document sentiment
prediction models based on the annotated data. Following previous researches
on sentiment prediction of documents [70, 19, 16], dominated by BERT [35]
based methods, Longformer [8] is shown to be the best choice in the prediction
of document level bias.

Media bias is topic & demographics dependent: While machine learning models
can achieve high accuracy in the task of bias detection, the models are not
immune to dataset variations. Inferring media bias is shown to be highly
subjective and dependent on the political beliefs of the annotators [45].
Also, the demographic shift plays a key role. The authors in [23] showed that
simple language variations in the data due to change in point of view (a
cross-country dataset) can have a devastating effect on the media bias
classifier. In [45], the authors showed that the common belief [70] that the
bias of the source media house is the bias of any news article originated from
it, does not hold well. Also, media houses tend to show different political
bias both by magnitude and polarity for different topics (e.g., ABC News and
CNN show heavy liberal bias on Gun issues but exhibit conservative bias on
immigration issues) [23].

Source level bias: Predicting the topical/political bias of individual news
outlets is as critical to media profiling as determining factuality. With the
advent of user-generated content and exponential rise of digital media,
scaling the process of determining media-bias and factuality of reporting of
the media houses has become more and more important as everybody who shares an
article or screenshot of the same of any source is a news provider now. While
measuring the factuality or bias of each news media is a hard task and
requires world-knowledge, the prediction of aggregate factuality or bias of a
news media house is relatively straight-forward. Political bias and factuality
of reporting have a linguistic aspect (what was written) along with a social
context (who read it). So, the authors in [6] crawled relevant data from
Twitter, Facebook, YouTube & Wikipedia and studied the impact of different
metadata extracted from these sources while classifying the media sources. The
evaluation results showed that what was written matters most, and that putting
all information sources together yields huge improvements over the current
state-of-the-art. On the other hand, in [113], the authors studied the
demographics of the US population interested in a media source with a specific
political inclination, using the Facebook AdSense tool, to understand the
leanings of the audience of the news media houses and reports high accuracy by
doing just that. Along with political bias, they were also able to identify
the demographic biases in the consumer population of any media house in a
zero-shot setting (i.e., using no training data). A demo of their application
can be found here272727https://twitter-app.mpi-sws.org/media-bias-monitor/.
TIMME [144] supervises a special form of GCNs to identify the political bias
of each Twitter user on annotated data and is able to identify the geographic
distribution of Twitter users with particular bias which correlates well with
the voting pattern of American citizens. They were able to use the same
algorithm to identify the bias of each news media house by gathering their
Twitter data.

Future directions: News media is widely cited as the fourth pillar of
democracy. While the health of a democratic institution depends heavily on
fair coverage of the institution by the media houses, studies on how
computational media bias is related to the health of democracies are lacking.
Again, most of the studies in media bias are concentrated on two-party systems
and is done on American and European demographics for the English language
media while other democracies also face the same problem deserving similar
attention. Further research is needed to understand the challenges faced in a
multi-party system for other languages in different demographics. Also, event
space in media changes very rapidly. So, research in an online learning setup
is needed to further enhance the media bias prediction accuracy over time.

####  4.2.2 Fairness in recommendation

The digital platform is full of choices. To help users make intelligent
choices, different information filtering systems are deployed in online
platforms. Recommendation systems (RSs) are one such omnipresent module. From
consumers’ perspective, RSs help in finding useful contents; while from
platforms’ and producers’ perspectives RSs bring revenue and profit [50, 122].
Given the multi-stakeholder setup and importance of RSs in the livelihood of
many of the stakeholders, recommendation fairness is of utmost importance.

Filter bubble and evolution of fairness in RSs: Traditionally, RSs like other
information filtering systems are keyed to relevance [87, 130]. However, over
dependency on relevance has led to differential services to different users or
different user groups. To describe these effects succinctly, Eli Pariser
coined a term ‘filter bubble’ [107]. The filter bubble problem is a concern
that personalization technologies, including RSs, narrow and bias the topics
of information provided to people while they do not notice these facts. To
account for these effects, the field of fairness in recommendation first
evolved as ‘Information Neutrality in RSs’. Focusing on customer fairness
(information neutrality toward customers) Kamishima et. al. [67, 68] tried to
solve the unfairness issue in RSs by adding a regularization term that
enforces demographic parity282828$U_{par}=|E_{g}[y]-E_{\neg g}[y]|$ can be an
instantiation of such demographic parity based regularizers [146].. Such
objectives penalized the differences among the average predicted ratings of
user groups (based on sensitive attributes, e.g., gender). However,
demographic parity is only appropriate when preferences are unrelated to the
sensitive features. In tasks such as recommendation, user preferences are
indeed influenced by sensitive features such as gender, race, and age [17,
30]. Taking a leaf out of the progresses in fairness literature in supervised
machine learning [57, 99], Yao et. al. [146] put forward fairness notions to
bridge the gap. They formulated different customer fairness metric by taking a
leaf out of the evolution of fairness in supervised learning [57] and showed
their effectiveness in improving customer fairness in recommendation [146].
Following their footsteps a number of works focused on ‘group fairness’ in
personalized recommendations [153, 38] where first they quantified biases due
to recommendation algorithms toward socially salient groups and proposed
methodologies to mitigate such biases. However, a major drawback of many of
these works was their negligence toward one of the major stakeholder in RSs,
i.e., the producer of items/services. This led to a second school of thought
when Burke et. al. [13, 14] first advocated for fairness toward both customers
and providers in a recommendation framework. Considering RSs as a two-sided
affair many nuanced algorithms came into existence considering fairness toward
both customers and producers and thus taking a giant step toward a fair
marketplace [100, 108, 48].

Figure 2: The figure shows the genesis of notions of fairness in
recommendation frameworks. In early 2010, it started with the notion of
‘neutrality in recommendation and by the end of the decade it has evolved into
multi-sided fairness toward building a fair marketplace.

Auditing RSs: While the fairness community seems to have covered different
forms of biases, there is a lack of understanding of the existing online
recommendation systems and biases thereof. Understanding of these systems are
especially important today due to the emergence of different private label
products (and in-house products) in e-commerce (and OTT) platforms [140, 39,
3]. A private label product is often produced and sold under the retailer’s
brand name, providing enough monetary incentive to the platforms to be
discriminative against several other products (or producers) on the platform.
Note that no third party (3P) regulator can quantify such biases because of
the lack of access to the exact underlying algorithms and the exact user-item
interaction details. To enable such 3P audits, in one of our works, we
presented a novel network-based technique that enabled us to extract important
parameters for auditing RSs by considering them as black-boxes [28]. With
detailed analysis on three different existing online RSs, we first proposed
ways to quantify their induced diversity and extent of information segregation
[28]. The usefulness of such a framework is manifold: (a) it sheds light on
how recommendations are formed between items based on different item-centric
properties, (b) it can be used as a tool for quantifying and auditing for
different consumer-focused metrics e.g., relevance, diversity, information-
segregation etc., (c) it can be used to quantify different biases toward the
associated producers of different items/services, and (d) finally, it also can
be used as a tool for mitigating the existing biases by re-wiring few of the
recommendation edges. In another follow-up work, we analyzed the sponsored
recommendation on e-commerce marketplaces and showed how they are utilized as
a trojan-horse to improve sales of products having special relationship with
the marketplace (e.g., private label products) [27]. Given that sponsored
recommendations potentially have delayed impact on their organic counterparts
(by affecting the sales and visibility of products), such studies open up a
completely new avenue of research.

Future directions: While the fairness community seems to have covered
different forms of biases in recommendation frameworks, it has overlooked the
special relationships that may exist between the digital marketplace and a
subset of stakeholders, and the biases thereof. Hence, studies of unfairness
discovery and mitigation considering the special relationships of platforms
remain an under-explored avenue of research till date. The introduction of
sponsored search and recommendations complicates the scenario even further.
Policies that allow sponsored results to deviate from organic results; while
adhering to fairness of marketplace can be another interesting broad direction
for further research.

##  5 Conclusion

In this survey we have presented a critical rundown on the evolution of the
online infosphere by depicting some of the research areas that are becoming
very crucial at current times. We started our discussion with a view of the
infosphere as a collaborative platform, with a dedicated focus on Wikipedia.
Wikipedia, the freely available and one of the largest knowledge base,
containing a wide variety of information has been a primary focus of an
extensive research so far. In this survey we have presented a detailed account
of the works on article quality monitoring, editor behaviour and their
retention and malicious activities like vandalism.

In the next section we have detailed the growth of the citations and
collaborations within and across various scientific disciplines that have
their roots in the infosphere. In fact, this has resulted in the birth of many
new interdisciplinary landscapes. We also discussed how machine learning
algorithms can be used to predict future citations as well as for recommending
citations. Finally we touched upon various issues related to anomalous
citation flows and their behaviour.

Finally, we summarised the research drives for patrolling the infosphere to
suppress the rising volume of harmful content. The discussion started with
analyzing the concept of hate speech and its growth over the past few years
through the online social media platforms and the adverse impact, thereby, on
both the online and the real world. We have shown the massive efforts the
research community have put forward in detection and mitigation of such
hateful behaviour. However, a lot of issues still remain as open problems and
need immediate attention. We observed dense connectivity in the hateful users
network by crawling Gab platform and analyzing their data and found that a
significant amount of posts are generated by these hateful users in social
media platforms. Additionally, we studied the temporal effect of hate speech
on the users by using the Gab data and found the increasing rate of hateful
users in the social network. After skimming through the recent literature we
have pointed out that by incorporating knowledge based context information for
a given post improves the overall performance of hatespeech detection rather
than by analyzing only the textual information. Apart from that we have
observed that by adding the user information of a given post can be further
analyzed to improve the hatespeech detection task. The last segment of the
discussion dealt with the bias and discrimination that are becoming pervasive
across different online environments like recommendation and news media
platforms.

To conclude, the above mentioned research aspects related to the _online
infosphere_ have attracted a lot of attention across the board including
scientific communities, industry stakeholders and policy-makers. This paper
discusses the technical challenges and possible solutions in a direction that
utilizes the immense power of AI for solving real world problems but also
considers the societal implications of these solutions. As a final note, we
could see that the problems related to the anomalies in scientific
collaborations and citations, hate speech detection and mitigation in social
media and the bias and unfairness in news media and recommendation systems
have a lot of open ends thus enabling exciting opportunities for future
research.

## References

  * [1] Albert, R., and Barabási, A.-L. Statistical mechanics of complex networks.  Rev. Mod. Phys. 74 (Jan 2002), 47–97.
  * [2] Aluru, S. S., Mathew, B., Saha, P., and Mukherjee, A. Deep learning models for multilingual hate speech detection.  arXiv preprint arXiv:2004.06465 (2020).
  * [3] Amazon.  Online platforms and market power, part 2: Innovation and entrepreneurship.  https://docs.house.gov/meetings/JU/JU05/20190716/109793/HHRG-116-JU05-20190716-SD038.pdf, 2019\.
  * [4] Badjatiya, P., Gupta, S., Gupta, M., and Varma, V. Deep learning for hate speech detection in tweets.  WWW, pp. 759–760.
  * [5] Baly, R., Karadzhov, G., Alexandrov, D., Glass, J., and Nakov, P. Predicting factuality of reporting and bias of news media sources.  arXiv preprint arXiv:1810.01765 (2018).
  * [6] Baly, R., Karadzhov, G., An, J., Kwak, H., Dinkov, Y., Ali, A., Glass, J., and Nakov, P. What was written vs. who read it: News media profiling using text analysis and social media context.  In Proceedings of the 58th Annual Meeting of the Association for Computational Linguistics (Online, July 2020), Association for Computational Linguistics, pp. 3364–3374.
  * [7] Barthel, R., and Seidl, R. Interdisciplinary collaboration between natural and social sciences – status and trends exemplified in groundwater research.  PLOS ONE 12, 1 (01 2017), 1–27.
  * [8] Beltagy, I., Peters, M. E., and Cohan, A. Longformer: The long-document transformer, 2020.
  * [9] Bergmann, T., Dale, R., Sattari, N., Heit, E., and Bhat, H. S. The interdisciplinarity of collaborations in cognitive science.  Cognitive science 41, 5 (2017), 1412–1418.
  * [10] Bhagavatula, C., Feldman, S., Power, R., and Ammar, W. Content-based citation recommendation.  In Proceedings of the 2018 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, Volume 1 (Long Papers) (New Orleans, Louisiana, June 2018), Association for Computational Linguistics, pp. 238–251.
  * [11] Bolukbasi, T., Chang, K.-W., Zou, J., Saligrama, V., and Kalai, A. Man is to computer programmer as woman is to homemaker? debiasing word embeddings.  In Proceedings of the 30th International Conference on Neural Information Processing Systems (USA, 2016), NIPS’16, Curran Associates Inc., pp. 4356–4364.
  * [12] Budak, C., Goel, S., and Rao, J. M. Fair and balanced? quantifying media bias through crowdsourced content analysis.  Public Opinion Quarterly 80, S1 (2016), 250–271.
  * [13] Burke, R. Multisided fairness for recommendation.  arXiv preprint arXiv:1707.00093 (2017).
  * [14] Burke, R., Sonboli, N., and Ordonez-Gauger, A. Balanced neighborhoods for multi-sided fairness in recommendation.  In ACM FAccT (2018).
  * [15] Caliskan, A., Bryson, J. J., and Narayanan, A. Semantics derived automatically from language corpora contain human-like biases.  Science 356, 6334 (2017), 183–186.
  * [16] Chakraborty, S., Goyal, P., and Mukherjee, A. Aspect-based sentiment analysis of scientific reviews.  In Proceedings of the ACM/IEEE Joint Conference on Digital Libraries in 2020 (New York, NY, USA, 2020), JCDL ’20, Association for Computing Machinery, p. 207–216.
  * [17] Chausson, O. Who watches what?: assessing the impact of gender and personality on film preferences.  Paper published online on the MyPersonality project website http://mypersonality. org/wiki/doku. php (2010).
  * [18] Chen, Y., Zhou, Y., Zhu, S., and Xu, H. Detecting offensive language in social media to protect adolescent online safety.  pp. 71–80.
  * [19] Choi, G., Oh, S., and Kim, H. Improving document-level sentiment classification using importance of sentences.  Entropy 22, 12 (2020), 1336.
  * [20] Chouldechova, A. Fair prediction with disparate impact: A study of bias in recidivism prediction instruments.  Big data 5, 2 (2017), 153–163.
  * [21] Chung, Y.-L., Kuzmenko, E., Tekiroglu, S. S., and Guerini, M. CONAN - COunter NArratives through nichesourcing: a multilingual dataset of responses to fight online hate speech.  In Proceedings of the 57th Annual Meeting of the Association for Computational Linguistics (Florence, Italy, July 2019), Association for Computational Linguistics, pp. 2819–2829.
  * [22] Conover, M. D., Gonçalves, B., Ratkiewicz, J., Flammini, A., and Menczer, F. Predicting the political alignment of twitter users.  In 2011 IEEE third international conference on privacy, security, risk and trust and 2011 IEEE third international conference on social computing (2011), IEEE, pp. 192–199.
  * [23] Cremisini, A., Aguilar, D., and Finlayson, M. A Challenging Dataset for Bias Detection: The Case of the Crisis in the Ukraine.  06 2019, pp. 173–183.
  * [24] Dadvar, M., Trieschnigg, D., Ordelman, R., and de Jong, F. Improving cyberbullying detection with user context.  pp. pp 693–696.
  * [25] D’Alessio, D., and Allen, M. Media bias in presidential elections: A meta-analysis.  Journal of communication 50, 4 (2000), 133–156.
  * [26] Dang, Q. V., and Ignat, C.-L. Quality assessment of wikipedia articles without feature engineering.  In Proceedings of the 16th ACM/IEEE-CS on Joint Conference on Digital Libraries (2016), pp. 27–30.
  * [27] Dash, A., Chakraborty, A., Ghosh, S., Mukherjee, A., and Gummadi, K. P. When the umpire is also a player: Bias in private label product recommendations on e-commerce marketplaces.  In ACM FAccT (2021).
  * [28] Dash, A., Mukherjee, A., and Ghosh, S. A network-centric framework for auditing recommendation systems.  In IEEE INFOCOM (2019).
  * [29] Davidson, T., Warmsley, D., Macy, M., and Weber, I. Automated hate speech detection and the problem of offensive language.
  * [30] Daymont, T. N., and Andrisani, P. J. Job preferences, college major, and the gender gap in earnings.  Journal of Human Resources (1984).
  * [31] de La Robertie, B., Pitarch, Y., and Teste, O. Measuring article quality in wikipedia using the collaboration network.  In 2015 IEEE/ACM International Conference on Advances in Social Networks Analysis and Mining (ASONAM) (2015), IEEE, pp. 464–471.
  * [32] DeGroot, M. H. Reaching a consensus.  Journal of the American Statistical Association 69, 345 (1974), 118–121.
  * [33] Del Vigna, F., Cimino, A., Dell’Orletta, F., Petrocchi, M., and Tesconi, M. Hate me, hate me not: Hate speech detection on facebook.
  * [34] Devlin, J., Chang, M.-W., Lee, K., and Toutanova, K. BERT: Pre-training of deep bidirectional transformers for language understanding.  In Proceedings of the 2019 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, Volume 1 (Long and Short Papers) (Minneapolis, Minnesota, June 2019), Association for Computational Linguistics, pp. 4171–4186.
  * [35] Devlin, J., Chang, M.-W., Lee, K., and Toutanova, K. BERT: Pre-training of deep bidirectional transformers for language understanding.  In Proceedings of the 2019 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, Volume 1 (Long and Short Papers) (Minneapolis, Minnesota, June 2019), Association for Computational Linguistics, pp. 4171–4186.
  * [36] Djuric, N., Zhou, J., Morris, R., Grbovic, M., Radosavljevic, V., and Bhamidipati, N. Hate speech detection with comment embeddings.  pp. 29–30.
  * [37] Eberl, J.-M., Boomgaarden, H. G., and Wagner, M. One bias fits all? three types of media bias and their effects on party preferences.  Communication Research 44, 8 (2017), 1125–1148.
  * [38] Edizel, B., Bonchi, F., Hajian, S., Panisson, A., and Tassa, T. Fairecsys: mitigating algorithmic bias in recommender systems.  Springer International Journal of Data Science and Analytics (2019).
  * [39] Faherty, E., Huang, K., and Land, R. The amazon monopoly: Is amazon’s private label business the tipping point?  Munich Personal RePEc Archive (2017).
  * [40] Fister, I., Fister, I., and Perc, M. Toward the discovery of citation cartels in citation networks.  Frontiers in Physics 4 (2016), 49.
  * [41] Fister, I., Mlakar, U., and Brest, J. A new population-based nature-inspired algorithm every month : Is the current era coming to the end ?  In StuCoSReC: Proceedings of the 2016 3rd Student Computer Science Research Conference (2016).
  * [42] Färber, M., and Jatowt, A. Citation recommendation: approaches and datasets.  International Journal on Digital Libraries 21, 4 (Aug 2020), 375–405.
  * [43] Gabielkov, M., Ramachandran, A., Chaintreau, A., and Legout, A. Social Clicks: What and Who Gets Read on Twitter?  In IFIP Perf. (Antibes Juan-les-Pins, France, June 2016).
  * [44] Gagliardone, I., Gal, D., Alves, T., and Martinez, G. Countering online hate speech.  UNESCO Publishing, 2015.
  * [45] Ganguly, S., Kulshrestha, J., An, J., and Kwak, H. Empirical evaluation of three common assumptions in building political media bias datasets.  Proceedings of the International AAAI Conference on Web and Social Media 14, 1 (May 2020), 939–943.
  * [46] Gao, L., and Huang, R. Detecting online hate speech using context aware models.  pp. 260–266.
  * [47] Garland, J., Ghazi-Zahedi, K., Young, J.-G., Hébert-Dufresne, L., and Galesic, M. Countering hate on social media: Large scale classification of hate and counter speech.  arXiv preprint arXiv:2006.01974 (2020).
  * [48] Geyik, S. C., Ambler, S., and Kenthapadi, K. Fairness-aware ranking in search & recommendation systems with application to linkedin talent search.
  * [49] Gomez, R., Gibert, J., Gomez, L., and Karatzas, D. Exploring hate speech detection in multimodal publications.  pp. 1459–1467.
  * [50] Gomez-Uribe, C. A., and Hunt, N. The netflix recommender system: Algorithms, business value, and innovation.  ACM TMIS 6, 4 (2016).
  * [51] Groeling, T. Media bias by the numbers: Challenges and opportunities in the empirical study of partisan news.  Annual Review of Political Science 16, 1 (2013), 129–151.
  * [52] Guda, B. P. R., Seelaboyina, S. B., Sarkar, S., and Mukherjee, A. Nwqm: A neural quality assessment framework for wikipedia.  In Proceedings of the 2020 Conference on Empirical Methods in Natural Language Processing (EMNLP) (2020), pp. 8396–8406.
  * [53] Halfaker, A., and Geiger, R. S. Ores: Lowering barriers with participatory machine learning in wikipedia.  arXiv preprint arXiv:1909.05189 (2019).
  * [54] Halfaker, A., Geiger, R. S., Morgan, J. T., and Riedl, J. The rise and decline of an open collaboration system: How wikipedia’s reaction to popularity is causing its decline.  American Behavioral Scientist 57, 5 (2013), 664–688.
  * [55] Halfaker, A., Kittur, A., and Riedl, J. Don’t bite the newbies: how reverts affect the quantity and quality of wikipedia work.  In Proceedings of the 7th international symposium on wikis and open collaboration (2011), pp. 163–172.
  * [56] Halfaker, A., and Taraborelli, D. Artificial intelligence service “ores” gives wikipedians x-ray specs to see through bad edits.  Wikimedia Blog (2015).
  * [57] Hardt, M., Price, E., Srebro, N., et al. Equality of opportunity in supervised learning.  In NIPS (2016).
  * [58] Hazra, R., Singh, M., Goyal, P., Adhikari, B., and Mukherjee, A. The rise and rise of interdisciplinary research: Understanding the interaction dynamics of three major fields - physics, mathematics and computer science.  In Digital Libraries at the Crossroads of Digital Information for the Future - 21st International Conference on Asia-Pacific Digital Libraries, ICADL 2019, Kuala Lumpur, Malaysia, November 4-7, 2019, Proceedings (2019), pp. 71–77.
  * [59] He, Q., Pei, J., Kifer, D., Mitra, P., and Giles, L. Context-aware citation recommendation.  In Proceedings of the 19th International Conference on World Wide Web (New York, NY, USA, 2010), WWW ’10, Association for Computing Machinery, p. 421–430.
  * [60] Herman, E. S., and Chomsky, N. Manufacturing consent: The political economy of the mass media.  Pantheon Books, 1988.
  * [61] Hofstetter, C. R., and Buss, T. F. Bias in television news coverage of political events: A methodological analysis.  Journal of Broadcasting 22, 4 (1978), 517–530.
  * [62] Jain, N., and Singh, M. The evolving ecosystem of predatory journals: a case study in indian perspective.  In International Conference on Asian Digital Libraries (2019), Springer, pp. 78–92.
  * [63] Jana, A., Kanojiya, P., Goyal, P., and Mukherjee, A. Wikiref: Wikilinks as a route to recommending appropriate references for scientific wikipedia pages.  arXiv preprint arXiv:1806.04092 (2018).
  * [64] Jeong, C., Jang, S., Shin, H., Park, E., and Choi, S. A context-aware citation recommendation model with bert and graph convolutional networks, 2019.
  * [65] Johnson, I., Lemmerich, F., Sáez-Trumper, D., West, R., Strohmaier, M., and Zia, L. Global gender differences in wikipedia readership.  arXiv preprint arXiv:2007.10403 (2020).
  * [66] Johnson, K., and Goldwasser, D. Identifying stance by analyzing political discourse on twitter.  In Proceedings of the First Workshop on NLP and Computational Social Science (2016), pp. 66–75.
  * [67] Kamishima, T., Akaho, S., Asoh, H., and Sakuma, J. Enhancement of the neutrality in recommendation.  In Decisions@ RecSys (2012).
  * [68] Kamishima, T., Akaho, S., and Sakuma, J. Fairness-aware learning through regularization approach.  In IEEE ICDMW (2011).
  * [69] Katzenbach, C., Binns, R., and Gorwa, R. Algorithmic content moderation: Technical and political challenges in the automation of platform governance.  Big Data and Society 7, 1 (2020).
  * [70] Kiesel, J., Mestre, M., Shukla, R., Vincent, E., Adineh, P., Corney, D., Stein, B., and Potthast, M. SemEval-2019 task 4: Hyperpartisan news detection.  In Proceedings of the 13th International Workshop on Semantic Evaluation (Minneapolis, Minnesota, USA, June 2019), Association for Computational Linguistics, pp. 829–839.
  * [71] Kiesel, J., Potthast, M., Hagen, M., and Stein, B. Spatio-temporal analysis of reverted wikipedia edits.  In ICWSM (2017).
  * [72] Kojaku, S., Livan, G., and Masuda, N. Detecting citation cartels in journal networks, 2020.
  * [73] Kshirsagar, R., Cukuvac, T., McKeown, K., and McGregor, S. Predictive embeddings for hate speech detection on twitter.  In Proceedings of the 2nd Workshop on Abusive Language Online (ALW2) (2018), pp. 26–32.
  * [74] Kumar, R., Raghavan, P., Rajagopalan, S., Sivakumar, D., Tomkins, A., and Upfal, E. Stochastic models for the web graph.  In Proceedings of the 41st Annual Symposium on Foundations of Computer Science (USA, 2000), FOCS ’00, IEEE Computer Society, p. 57.
  * [75] Kumar, S., Spezzano, F., and Subrahmanian, V. Vews: A wikipedia vandal early warning system.  ArXiv abs/1507.01272 (2015).
  * [76] Kwon, S., Solomon, G. E. A., Youtie, J., and Porter, A. L. A measure of knowledge flow between specific fields: Implications of interdisciplinarity for impact and funding.  PLOS ONE 12, 10 (10 2017), 1–16.
  * [77] Laver, M., Benoit, K., and Garry, J. Extracting policy positions from political texts using words as data.  American political science review 97, 2 (2003), 311–331.
  * [78] Lazaridou, K., Krestel, R., and Naumann, F. Identifying media bias by analyzing reported speech.  In 2017 IEEE International Conference on Data Mining (ICDM) (2017), IEEE, pp. 943–948.
  * [79] Le, H. T. T., Shafiq, Z., and Srinivasan, P. Scalable news slant measurement using twitter.  In Eleventh International AAAI Conference on Web and Social Media (2017).
  * [80] Le, Q., and Mikolov, T. Distributed representations of sentences and documents.  In International conference on machine learning (2014), pp. 1188–1196.
  * [81] Le, Q., and Mikolov, T. Distributed representations of sentences and documents.  31st International Conference on Machine Learning, ICML 2014 4 (05 2014).
  * [82] Li, C.-T., Lin, Y.-J., Yan, R., and Yeh, M.-Y. Trend-based citation count prediction for research articles.  In Advances in Knowledge Discovery and Data Mining (Cham, 2015), T. Cao, E.-P. Lim, Z.-H. Zhou, T.-B. Ho, D. Cheung, and H. Motoda, Eds., Springer International Publishing, pp. 659–671.
  * [83] Li, S., Zhao, W. X., Yin, E. J., and Wen, J.-R. A neural citation count prediction model based on peer review text.  In Proceedings of the 2019 Conference on Empirical Methods in Natural Language Processing and the 9th International Joint Conference on Natural Language Processing (EMNLP-IJCNLP) (Hong Kong, China, Nov. 2019), Association for Computational Linguistics, pp. 4914–4924.
  * [84] Li, X., Tang, J., Wang, T., Luo, Z., and De Rijke, M. Automatically assessing wikipedia article quality by exploiting article–editor networks.  In European Conference on Information Retrieval (2015), Springer, pp. 574–580.
  * [85] Lim, S., Jatowt, A., and Masatoshi, Y. Creating a dataset for fine-grained bias detection in news articles.  Forum on Data Engineering and Information Management 12 (May 2020), 1–35.
  * [86] Lin, Y.-R., Bagrow, J. P., and Lazer, D. More voices than ever? quantifying media bias in networks.  In Fifth International AAAI Conference on Weblogs and Social Media (2011).
  * [87] Linden, G., Smith, B., and York, J. Amazon. com recommendations: Item-to-item collaborative filtering.  IEEE Internet computing (2003).
  * [88] Littlejohn, A., and Hood, N. Becoming an online editor: Perceived roles and responsibilities of wikipedia editors.  Information Research 23, 1 (2018).
  * [89] Madaan, N., Mehta, S., Agrawaal, T., Malhotra, V., Aggarwal, A., Gupta, Y., and Saxena, M. Analyze, detect and remove gender stereotyping from bollywood movies.  In Conference on Fairness, Accountability and Transparency (2018), pp. 92–105.
  * [90] Maity, S. K., Chakraborty, A., Goyal, P., and Mukherjee, A. Detection of sockpuppets in social media.  In Companion of the 2017 ACM Conference on Computer Supported Cooperative Work and Social Computing (2017), pp. 243–246.
  * [91] Marrese-Taylor, E., Loyola, P., and Matsuo, Y. An edit-centric approach for wikipedia article quality assessment.  arXiv preprint arXiv:1909.08880 (2019).
  * [92] Martinez-Rico, J. R., Martínez-Romo, J., and Araujo, L. Can deep learning techniques improve classification performance of vandalism detection in wikipedia?  Eng. Appl. Artif. Intell. 78 (2019), 248–259.
  * [93] Mathew, B., Dutt, R., Goyal, P., and Mukherjee, A. Spread of hate speech in online social media.  pp. 173–182.
  * [94] Mathew, B., Illendula, A., Saha, P., Sarkar, S., Goyal, P., and Mukherjee, A. Hate begets hate: A temporal study of hate speech.  Proc. ACM Hum.-Comput. Interact. 4, CSCW2 (Oct. 2020).
  * [95] Mathew, B., Kumar, N., Goyal, P., and Mukherjee, A. Interaction dynamics between hate and counter users on twitter.  In Proceedings of the 7th ACM IKDD CoDS and 25th COMAD. 2020, pp. 116–124.
  * [96] Mathew, B., Saha, P., Tharad, H., Rajgaria, S., Singhania, P., Maity, S. K., Goyal, P., and Mukherjee, A. Thou shalt not hate: Countering online hate speech.  In Proceedings of the International AAAI Conference on Web and Social Media (2019), vol. 13, pp. 369–380.
  * [97] Mathew, B., Saha, P., Yimam, S. M., Biemann, C., Goyal, P., and Mukherjee, A. Hatexplain: A benchmark dataset for explainable hate speech detection.  arXiv preprint arXiv:2012.10289 (2020).
  * [98] Meeder, B., Karrer, B., Sayedi, A., Ravi, R., Borgs, C., and Chayes, J. We know who you followed last summer: inferring social link creation times in twitter.  In Proceedings of the 20th international conference on World wide web (2011), pp. 517–526.
  * [99] Mehrabi, N., Morstatter, F., Saxena, N., Lerman, K., and Galstyan, A. A survey on bias and fairness in machine learning.  arXiv preprint arXiv:1908.09635 (2019).
  * [100] Mehrotra, R., McInerney, J., Bouchard, H., Lalmas, M., and Diaz, F. Towards a fair marketplace: Counterfactual evaluation of the trade-off between relevance, fairness & satisfaction in recommendation systems.  In ACM CIKM (2018).
  * [101] Morgan, J., Bouterse, S., Walls, H., and Stierch, S. Tea and sympathy: crafting positive new user experiences on wikipedia.  In CSCW ’13 (2013).
  * [102] Morgan, J., and Halfaker, A. Evaluating the impact of the wikipedia teahouse on newcomer socialization and retention.  Proceedings of the 14th International Symposium on Open Collaboration (2018).
  * [103] Morillo, F., Bordons, M., and Gómez, I. Interdisciplinarity in science: A tentative typology of disciplines and research areas.  J. Am. Soc. Inf. Sci. Technol. 54, 13 (Nov. 2003), 1237–1249.
  * [104] Murić, G., Abeliuk, A., Lerman, K., and Ferrara, E. Collaboration drives individual productivity.  Proceedings of the ACM on Human-Computer Interaction 3, CSCW (2019), 1–24.
  * [105] Nickerson, R. S. Confirmation bias: A ubiquitous phenomenon in many guises.  Review of general psychology 2, 2 (1998), 175–220.
  * [106] Pandey, P. K., Singh, M., Goyal, P., Mukherjee, A., and Chakrabarti, S. Analysis of reference and citation copying in evolving bibliographic networks.  Journal of Informetrics 14, 1 (2020), 101003.
  * [107] Pariser, E. The filter bubble: What the Internet is hiding from you.  Penguin UK, 2011.
  * [108] Patro, G. K., Biswas, A., Ganguly, N., Gummadi, K. P., and Chakraborty, A. Fairrec: Two-sided fairness for personalized recommendations in two-sided platforms.  In WWW (2020).
  * [109] Pedersen, D. B. Integrating social sciences and humanities in interdisciplinary research.  Palgrave Communications 2 (2016), 16036.
  * [110] Pornprasit, C., Liu, X., Kertkeidkachorn, N., Kim, K.-S., Noraset, T., and Tuarob, S. Convcn: A cnn-based citation network embedding algorithm towards citation recommendation.  In Proceedings of the ACM/IEEE Joint Conference on Digital Libraries in 2020 (New York, NY, USA, 2020), JCDL ’20, Association for Computing Machinery, p. 433–436.
  * [111] Qian, J., Bethke, A., Liu, Y., Belding, E., and Wang, W. Y. A benchmark dataset for learning to intervene in online hate speech.  In Proceedings of the 2019 Conference on Empirical Methods in Natural Language Processing and the 9th International Joint Conference on Natural Language Processing (EMNLP-IJCNLP) (Hong Kong, China, Nov. 2019), Association for Computational Linguistics, pp. 4755–4764.
  * [112] Recasens, M., Danescu-Niculescu-Mizil, C., and Jurafsky, D. Linguistic models for analyzing and detecting biased language.  In Proceedings of the 51st Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers) (2013), pp. 1650–1659.
  * [113] Ribeiro, F. N., Henrique, L., Benevenuto, F., Chakraborty, A., Kulshrestha, J., Babaei, M., and Gummadi, K. P. Media bias monitor: Quantifying biases of social media news outlets at large-scale.  In Twelfth International AAAI Conference on Web and Social Media (2018).
  * [114] Ribeiro, M. H., Calais, P. H., Almeida, V. A., and Meira Jr, W. ” everything i disagree with is# fakenews”: Correlating political polarization and spread of misinformation.  arXiv preprint arXiv:1706.05924 (2017).
  * [115] Ribeiro, M. H., Gligorić, K., Peyrard, M., Lemmerich, F., Strohmaier, M., and West, R. Sudden attention shifts on wikipedia following covid-19 mobility restrictions.  arXiv preprint arXiv:2005.08505 (2020).
  * [116] Richards, R. D., and Calvert, C. Counterspeech 2000: A new look at the old remedy for bad speech.  BYU L. Rev. (2000), 553.
  * [117] Ruprechter, T., Santos, T., and Helic, D. Relating wikipedia article quality to edit behavior and link structure.  Applied Network Science 5 (2020), 1–20.
  * [118] Saez-Trumper, D., Castillo, C., and Lalmas, M. Social media news communities: Gatekeeping, coverage, and statement bias.  In Proceedings of the 22Nd ACM International Conference on Information & Knowledge Management (New York, NY, USA, 2013), CIKM ’13, ACM, pp. 1679–1684.
  * [119] Saha, P., Mathew, B., Goyal, P., and Mukherjee, A. Hateminers: Detecting hate speech against women.  arXiv preprint arXiv:1812.06700 (2018).
  * [120] Sarkar, S., Reddy, B. P., Sikdar, S., and Mukherjee, A. Stre: Self attentive edit quality prediction in wikipedia.  In Proceedings of the 57th Annual Meeting of the Association for Computational Linguistics (2019), pp. 3962–3972.
  * [121] Sayama, H., and Akaishi, J. Characterizing interdisciplinarity of researchers and research topics using web search engines.  PLOS ONE 7, 6 (06 2012), 1–9.
  * [122] Sharma, A., Hofman, J. M., and Watts, D. J. Estimating the causal impact of recommendation systems from observational data.  In ACM EC (2015).
  * [123] Shen, A., Qi, J., and Baldwin, T. A hybrid model for quality assessment of wikipedia articles.  In Proceedings of the Australasian Language Technology Association Workshop 2017 (2017), pp. 43–52.
  * [124] Shen, A., Salehi, B., Baldwin, T., and Qi, J. A joint model for multimodal document quality assessment.  In 2019 ACM/IEEE Joint Conference on Digital Libraries (JCDL) (2019), IEEE, pp. 107–110.
  * [125] Shen, A., Salehi, B., Qi, J., and Baldwin, T. Feature-guided neural model training for supervised document representation learning.  In Proceedings of the The 17th Annual Workshop of the Australasian Language Technology Association (2019), pp. 47–51.
  * [126] Shiji, C., Yves, G., Clément, A., and Vincent, L. Interdisciplinarity patterns of highly‐cited papers: A cross‐disciplinary analysis.  Proceedings of the American Society for Information Science and Technology 51, 1 (2015), 1–4.
  * [127] Singh, M., Jaiswal, A., Shree, P., Pal, A., Mukherjee, A., and Goyal, P. Understanding the impact of early citers on long-term scientific impact.  In 2017 ACM/IEEE Joint Conference on Digital Libraries (JCDL) (2017), pp. 1–10.
  * [128] Singh, M., Patidar, V., Kumar, S., Chakraborty, T., Mukherjee, A., and Goyal, P. The role of citation context in predicting long-term citation profiles: An experimental study based on a massive bibliographic text dataset.  In Proceedings of the 24th ACM International on Conference on Information and Knowledge Management (New York, NY, USA, 2015), CIKM ’15, Association for Computing Machinery, p. 1271–1280.
  * [129] Singh, M., Sarkar, R., Goyal, P., Mukherjee, A., and Chakrabarti, S. Relay-linking models for prominence and obsolescence in evolving networks.  In Proceedings of the 23rd ACM SIGKDD International Conference on Knowledge Discovery and Data Mining (New York, NY, USA, 2017), KDD ’17, Association for Computing Machinery, p. 1077–1086.
  * [130] Smith, B., and Linden, G. Two decades of recommender systems at amazon. com.  IEEE internet computing 21, 3 (2017).
  * [131] Spezzano, F., Suyehira, K., and Gundala, L. A. Detecting pages to protect in wikipedia across multiple languages.  Social Network Analysis and Mining 9 (2019), 1–16.
  * [132] Steinmacher, I., Conte, T., Gerosa, M. A., and Redmiles, D. Social barriers faced by newcomers placing their first contribution in open source software projects.  In Proceedings of the 18th ACM conference on Computer supported cooperative work & social computing (2015), pp. 1379–1392.
  * [133] Susuri, A., Hamiti, M., and Dika, A. Machine learning based detection of vandalism in wikipedia across languages.  2016 5th Mediterranean Conference on Embedded Computing (MECO) (2016), 446–451.
  * [134] Tekiroglu, S. S., Chung, Y.-L., and Guerini, M. Generating counter narratives against online hate speech: Data and strategies.  arXiv preprint arXiv:2004.04216 (2020).
  * [135] Tran, K., and Christen, P. Cross-language learning from bots and users to detect vandalism on wikipedia.  IEEE Transactions on Knowledge and Data Engineering 27 (2015), 673–685.
  * [136] Van Hee, C., Lefever, E., Verhoeven, B., Mennes, J., Desmet, B., Pauw, G., Daelemans, W., and Hoste, V. Detection and fine-grained classification of cyberbullying events.
  * [137] Verstak, A., Acharya, A., Suzuki, H., Henderson, S., Iakhiaev, M., Lin, C. C. Y., and Shetty, N. On the shoulders of giants: The growing impact of older articles, 2014\.
  * [138] Warncke-Wang, M., Ayukaev, V. R., Hecht, B., and Terveen, L. G. The success and failure of quality improvement projects in peer production communities.  In Proceedings of the 18th ACM Conference on Computer Supported Cooperative Work & Social Computing (2015), pp. 743–756.
  * [139] Waseem, Z., and Hovy, D. Hateful symbols or hateful people? predictive features for hate speech detection on twitter.  pp. 88–93.
  * [140] Wired.  Amazon and netflix look to their own shows as the key to world domination.  https://www.wired.com/2016/12/amazon-netflix-look-shows-key-world-domination/, 2016\.
  * [141] Wong, F. M. F., Tan, C. W., Sen, S., and Chiang, M. Quantifying political leaning from tweets, retweets, and retweeters.  IEEE transactions on knowledge and data engineering 28, 8 (2016), 2158–2172.
  * [142] Wright, L., Ruths, D., Dillon, K. P., Saleem, H. M., and Benesch, S. Vectors for counterspeech on Twitter.  In Proceedings of the First Workshop on Abusive Language Online (Vancouver, BC, Canada, Aug. 2017), Association for Computational Linguistics, pp. 57–62.
  * [143] Xiao, S., Yan, J., Li, C., Jin, B., Wang, X., Yang, X., Chu, S. M., and Zhu, H. On modeling and predicting individual paper citation count over time.  In Proceedings of the Twenty-Fifth International Joint Conference on Artificial Intelligence (2016), IJCAI’16, AAAI Press, p. 2676–2682.
  * [144] Xiao, Z., Song, W., Xu, H., Ren, Z., and Sun, Y. Timme: Twitter ideology-detection via multi-task multi-relational embedding.  In Proceedings of the 26th ACM SIGKDD International Conference on Knowledge Discovery and Data Mining (New York, NY, USA, 2020), KDD ’20, Association for Computing Machinery, p. 2258–2268.
  * [145] Yang, D., Halfaker, A., Kraut, R. E., and Hovy, E. H. Who did what: Editor role identification in wikipedia.  In ICWSM (2016), pp. 446–455.
  * [146] Yao, S., and Huang, B. Beyond parity: Fairness objectives for collaborative filtering.  In NeurIPS (2017).
  * [147] Yazdanian, R., Zia, L., Morgan, J., Mansurov, B., and West, R. Eliciting new wikipedia users’ interests via automatically mined questionnaires: For a warm welcome, not a cold start.  In Proceedings of the International AAAI Conference on Web and Social Media (2019), vol. 13, pp. 537–547.
  * [148] Yigit-Sert, S., Altingovde, I. S., and Ulusoy, O. Towards detecting media bias by utilizing user comments.  In Proceedings of the 8th ACM Conference on Web Science (New York, NY, USA, 2016), WebSci ’16, ACM, pp. 374–375.
  * [149] Yuan, S., Tang, J., Zhang, Y., Wang, Y., and Xiao, T. Modeling and predicting citation count via recurrent neural network with long short-term memory.  ArXiv abs/1811.02129 (2018).
  * [150] Zhang, H., Ren, Y., and Kraut, R. E. Mining and predicting temporal patterns in the quality evolution of wikipedia articles.  In HICSS (2020), pp. 1–10.
  * [151] Zhang, S., Hu, Z., Zhang, C., and Yu, K. History-based article quality assessment on wikipedia.  In 2018 IEEE International Conference on Big Data and Smart Computing (BigComp) (2018), IEEE, pp. 1–8.
  * [152] Zhao, J., Wang, T., Yatskar, M., Ordonez, V., and Chang, K.-W. Men also like shopping: Reducing gender bias amplification using corpus-level constraints.  arXiv preprint arXiv:1707.09457 (2017).
  * [153] Zhu, Z., Hu, X., and Caverlee, J. Fairness-aware tensor-based recommendation.  In ACM CIKM (2018).