# Convergence and Inequality in Research Globalization

Saurabh Mishra
Institute for Human-Centered Artificial Intelligence (HAI)
Stanford University, Stanford, California, USA
<EMAIL_ADDRESS>
&Kuansan Wang
Microsoft Research
Microsoft Corp., Redmond Washington, USA
<EMAIL_ADDRESS>

###### Abstract

The catch-up effect and the Matthew effect offer opposing characterizations of
globalization: the former predicts an eventual convergence as the poor can
grow faster than the rich due to free exchanges of complementary resources,
while the latter, a deepening inequality between the rich and the poor. To
understand these effects on the globalization of research, we conduct an in-
depth study based on scholarly and patent publications covering STEM research
from 218 countries/regions over the past four decades, covering more than 55
million scholarly articles and 1.7 billion citations. Unique to this
investigation is the simultaneous examination of both the research output and
its impact in the same data set, using a novel machine learning based measure,
called saliency, to mitigate the intrinsic biases in quantifying the research
impact. The results show that the two effects are in fact co-occurring: there
are clear indications of convergence among the high income and upper middle
income countries across the STEM fields, but a widening gap is developing that
segregates the lower middle and low income regions from the higher income
regions. Furthermore, the rate of convergence varies notably among the STEM
sub-fields, with the highly strategic area of Artificial Intelligence (AI)
sandwiched between fields such as Medicine and Materials Science that occupy
the opposite ends of the spectrum. The data support the argument that a
leading explanation of the Matthew effect, namely, the preferential attachment
theory, can actually foster the catch-up effect when organizations from lower
income countries forge substantial research collaborations with those already
dominant. The data resoundingly show such collaborations benefit all parties
involved, and a case of role reversal can be seen in the Materials Science
field where the most advanced signs of convergence are observed.

_K_ eywords Convergence $\cdot$ Matthew Effect $\cdot$ Saliency $\cdot$
Science and Technology $\cdot$ Artificial Intelligence $\cdot$ Globalization
$\cdot$ Collaboration $\cdot$ Measurement Science $\cdot$ Research Impact
$\cdot$ US-China $\cdot$ Developing countries $\cdot$ Inequality $\cdot$
Technology Policy

##  1 Introduction

The exchange of people and ideas is the cornerstone of globalization. Research
globalization implies collaboration between countries. Multidisciplinary
scientists have extolled various forms of globalization, including economic,
financial, cultural, or political, but technology globalization, which is the
dominant driver of globalization in the $21^{st}$ century, is less studied.
Technology (or scientific) globalization creates opportunities for national
competitiveness and forms the backbone of long-run economic growth and
development [2, 35, 49]. Incomplete or incorrect measures to track technology
globalization could lead to misleading conclusions for global and national
science policy.

The central theme of this paper is to explore whether poor countries are
catching up with rich countries in the realm of scientific research or,
alternatively, the gap in technology research output and impact is widening
between rich and poor countries. In natural and social sciences, these trends
are referred to as convergence or divergence. Convergence refers to coming
together, while divergence means moving apart or an increase in inequality
[33, 18, 21]. Convergence implies that the lesser will catch up by growing
faster than the greater in terms of research output and impact. Globally,
there is a growing shift from so called “national system(s) of innovation” to
a series of globally connected networked systems of knowledge creation [46].
Technologies such as Artificial Intelligence (AI) could qualify as general-
purpose technologies (GPTs) that have many applications across a wide spectrum
of uses in production and research. Most encouragingly, recent advancements
there are propelled by open data sharing, open source software and other open
science principles that considerably lower the access barriers [54]. There is
a growing list of indicators, including composite indices that are used to
guide global and national science (more generally) and AI policy (more
specifically) [40, 22, 27, 5]. Many of them lend support to convergence as a
necessary outcome, where it can be observed that the dominant are becoming
less so. On the other hand, it is also widely observed that inequality,
characterized by the Matthew Effect [34], is deeply entrenched in the global
research ecosystem, confirmed again in a recent study that shows top-cited
scientists, often affiliated with high-ranking institutions in the western
world, are receiving ever increasing citations [38]. Similarly for AI, deep
learning’s unanticipated rise appears to have resulted in divergence, where
compute divide between large firms and non-elite universities increases
concerns of "de-democratization" of knowledge production [4]. These
observations motivate this study to view convergence and divergence as yin and
yang, namely, the twin effects of research globalization so that we can avoid
the pitfall of missing half of the puzzle by looking for indicators of only
convergence or only inequality.

The unique contribution of this paper is thus to provide a more complete view
of research globalization by accounting for research output, impact, and
collaboration to answer fundamental questions in research globalization. Is
there hope for developing countries to catch up with advanced economies in
research output and impact? Countries can generate a large amount of research
output with relative impact not increasing in a commensurate manner. How do
countries get out of this trap? What is the role of science policy, especially
for developing countries to catch up with rich countries? Our data suggest
that the example of China could provide an alternative path for developing
countries. The output and impact of technology research from China in the
1980’s was at par with other developing countries such as India. However, in
recent years, China’s technology research has been catching up with that of
rich countries, first in output and subsequently, in impact. If developing
countries build research collaborations with advanced economies as China has
been doing, could that lead to global research convergence?

The study employs a quantitative and data-driven approach, following the
similar methodologies reported in the literature for general research [62, 58,
31, 16, 48, 57, 36] and specifically for AI [40, 27, 5]. We take into account
the global corpora of journal publications, conference papers, and patents to
answer three key questions. First, is there global convergence in research
output and impact across sub-fields of technology research? Second, what are
the dynamics of convergence (or divergence) between comparable income groups?
Third and perhaps most novel, which country pairs have emerged as central,
field-specific collaborators and what are the science policy implications to
avoid the output-impact trap?

The results show that convergence is taking place, but only in the case of
Upper-Middle Income countries (UMCs) catching up with High-Income countries
(HICs), following the World Bank’s 2020 income categorization. Similar to [1],
we find that, on average, rich countries are becoming less dominant as they
are losing their share in world research output and impact. However, if we
separate Middle-Income countries (MICs) into UMCs and Lower-Middle-Income
countries (LMCs), the results are mixed. These two groups started out with a
similar low base of output and impact but, in recent years, the gap between
UMCs and LMCs is widening, supporting evidence for divergence. The results
vary by field — for example, China, an UMC, is catching up in fields such as
AI or Materials Science but not Medicine. Low-income countries (LICs) are
diverging and do not show signs of catching up. LICs are not catching up with
LMCs, and LMCs are not catching up with UMCs. The data nevertheless supports
that neither convergence nor divergence is a preordained outcome of
globalization. The contrasting trends between UMCs and LMCs appear to support
a formula for catching up by substantial research collaboration. The results
about research collaboration provide hope for science policy, especially in
developing countries. For example, STEM and AI research in China is driven by
research culture [44] and policy incentives to formulate R&D output milestones
[11, 59, 12]. It is key to recognize that China was able to reach from a small
base in the 1990’s to take over market share in STEM and AI, and collaboration
between China and G-7 countries has been identified as critical to global R&D
growth [28, 25]. We show evidence that collaboration between the US and China
in AI and STEM fields offers hope for developing countries to increase output
and impact. Conscious policy efforts to build field-specific research
collaboration can help developing countries increase their scientific
contribution and impact.

The rest of the paper is divided as follows. Section 2 presents details about
the data and methodology for the quantitative measure used in this study.
Section 3 presents the results segmenting metrics of research output, impact,
collaboration, and saliency comparing world regions between STEM and AI fields
of study. Section 4 provides discussion on policy implications with areas of
future research.

##  2 Data and Method

Following the well-established practice first proposed by Price [41], we
measure research output and impact through two proxy activities, namely, the
participation and the recognition received in the global scholarly
communications. In a nutshell, this study quantifies research output by the
number of scholarly articles published, and research impact, by the citations
they have and likely to receive. There are known issues in using publication
and citation measures, and plenty of other options have been proposed in the
scientometric, bibliometric, or science of science literature, with the most
relevant ones discussed below. However, we note that a widely accepted
measurement for convergence has yet to emerge, and the causality between
globalization and convergence is tricky to prove. Accordingly, we approach the
question of convergence by examining whether the historical elite dominance is
lessening (or the relative share share of global research output and impact is
increasing for developing countries). For this purpose, we contend it adequate
to base the analyses on the time honored metrics with straightforward
enhancements as described below.

###  2.1 Data

Similar to [16], the study is conducted with Microsoft Academic Graph (MAG)
but on the newer version dated on August 7, 2020 and validated against the
version dated October 29, 2020. These two versions of MAG allow multiple
affiliations to be associated with each author, in contrast to the data used
previously where only the primary author affiliation was available. The new
versions of MAG also provide the geolocation and ISO-3166 region code for each
affiliation, with which we classify the world into the following regions based
on 2020 World Bank income codes (included in the data in the Supplementary
Material). Within this study, the income codes used are HIC, UMC, LMC, and
LIC, which correspond to World Bank’s classification of high income, upper
middle income, lower middle income, and low income countries, respectively. No
countries simultaneously belong to more than one categories. The numbers of
countries/regions in these categories are shown in Table 1.

Since the objective here is to gain deeper understanding of the global trends
in recent decades that coincide with the accelerating globalization propelled
by the internet and digital transformation, we focus on the time horizon of
year 1980 (inclusive) through 2020 (exclusive). To avoid the data quality
issues, this study reports only the results in the STEM areas that better
align with MAG’s selection preference of including internationally oriented
articles, typically written in English, than the fields in the humanity area
where English publications do not provide an adequate characterization of the
state of research. The STEM areas, corresponding to the topmost 12 MAG fields
of study, are Medicine, Biology, Chemistry, Computer Science, Engineering,
Material Science, Physics, Psychology, Environmental Science, Geography,
Geology, and Math. The analytical scripts provided in the supplementary
material, though, can be extended to broader fields and year range in a
straightforward manner. In total, this study inspects 56,084,649 scholarly
articles and their 1,710,566,975 citations as recorded in MAG.

There are uncertainties associated with the global corpora. For example, the
class imbalance in limited number of low-income countries (LIC’s) relative to
UMC’s or HIC’s. Further, although although MAG strives to be comprehensive, it
is by no means perfect for this study in the following aspects. First and
foremost, MAG by design includes publications that are international oriented
[55], which leads to a lower representation of articles written in languages
other than English. The data set thus has a bias against non-English research
and can under-count research output of countries where local language
publications are not always translated into equivalent English versions, for
example, Japan (a HIC), Russia, Brazil and China (all UMCs). This study is
only focused on English-language research output. Secondly, the affiliation
coverage is uneven, especially for older publications, as many of its data
sources do not report this information. Although MAG uses PDF parsing to
extract such information [55], we notice such results are not error free.
Finally, MAG’s affiliation attribution may be not fine grain enough,
especially for work produced by multi-national industrial institutions for
which affiliation are attributed to the countries/regions where their
headquarters’ reside. For example, publications from Microsoft Research Asia,
India, and Cambridge labs are all counted towards the US, rather than China,
India, and UK, respectively. This attribution policy under-represents the
research activities of their host countries and inflate the international
collaboration (Sec. 2.4) when these researchers work with nearby universities.
An alternative is to use the Global Research Identifier Database (GRID), a
budding community-wide grass root effort to annotate finer grained information
of affiliations led by Digital Science111See https://digital-
science.com/products/grid for more details. Earlier investigations show the
GRID code in MAG does not lead to more accurate results than ISO-3166 code,
but the analytical software included in the Supplementary Materials can be
easily adapted for GRID should the data quality improve.

Category | Number of Countries/Regions
---|---
High income country (HIC) | 83
Upper middle income country (UMC) | 56
Lower middle income country (LMC) | 50
Low income country (LIC) | 29
Total | 218
Table 1: World Bank’s classification of the 218 countries/regions and their
sizes

###  2.2 Research Output

The annual research output is characterized by the number of articles
published in the given year of a given field with at least one author
affiliated with an institution in the target region. There is no weighting on
the authorship, e.g., one author affiliation from region A and ten from region
B leads to the equal attribution to either region. Similarly, there is no
weighting or fractional counting based on the order in which an author appears
in the author list. In the case of an author with multiple affiliations, the
order of affiliations is also ignored. Note that an article with affiliations
from multiple regions or classified as more than one field is counted multiple
times. The sum of the article counts from individual regions and fields
therefore exceeds the total number of articles published.

###  2.3 Research Impact

Three indices are considered in this study to assess research impacts:
citation counts, the saliency, and the saliency ratio.

####  2.3.1 Citation Count

The total citations received by articles in a given year with authors from a
world region are aggregated from MAG’s records as a conventional baseline to
characterize the research impact for the region. Similar to the publication
counts above, regions received citation attributed uniformly in the sense that
no author weighting is performed and all affiliations of a single author are
all treated equally when tabulating the citations.

Citation count described above is a commonly used metric of research impact,
and can either be used alone for measuring individual articles or in
aggregations such as in Journal Impact Factor (JIF) [10] for measuring
journals, h-index for authors [51], or share of highly cited articles for
countries/regions [36]. In this study, we also report citation counts
aggregated at the country or regional levels as a baseline measure of research
impact. Known issues for citation counts, however, are plenty. First,
citations exhibit a strong age bias against newer articles because it takes
time for a new publication to receive its due recognition. The estimation of a
lag ranging from 7 to 10 years, first made by Price in 1965 [41], has
withstood the test of time. Citation count is therefore a lagging indicator of
research impact that is not suitable to discover recent trends, as can be seen
in the results of this study later.

Secondly, citation behaviors are known to vary significantly from one field to
another [52], and lumping the analyses of different fields into a general
statement can over-represent the fields where the community favors more
references in the research articles. This challenge has led to the development
of field-normalized or weighted modifications on citation counts, such as Mean
Normalized Citation Score (MNCS), also known as the crown indicator [53],
Relative Citation Ratio (RCR) adopted by National Institute of Health in
PubMed [30], and the Field Weighted Citation Index (FWCI) by Elsevier that has
been empirically shown to be consistent with the MNCS and RCR [42]. However,
when citation counts of individual articles are normalized by different
factors, they pose a mathematical challenge to be aggregated into an author or
country level measure mainly because $E[X/Y]\neq E[X]/E[Y]$, namely, an order
reversal in normalization and aggregation leads to very different measurement.
Despite the successful applications, the normalization methods are often more
heuristically than mathematically motivated that may lead to unexpected and
inexplicable outcomes. For instance, FWCI has been used in numerous studies to
guide policy and identify top countries in AI research [40, 27, 48, 58, 62].
However, in these instances, the US is ranked behind Georgia, Hong Kong,
Switzerland, and Singapore, with Qatar, Palestine, Israel, and Belarus
following closely ahead of European Union (EU) countries (Figure A1.5, [40]).
These results defy the shared understanding of the AI community, and cannot be
collaborated in this study. Still, as studies like these are frequently
consulted in making consequential policy or funding decisions, the importance
of quantitative measurements based on a solid mathematical foundation cannot
be over-emphasized.

Furthermore, citations are often treated equal without considering the topical
relevance and the reputation of citing sources. This measurement weakness,
often exacerbated by the research reward system [13], has led to schemes of
“gaming the system” where abusive behaviors, such as citation cartels [19, 39]
and coercive citations [29, 56, 24], have become an ever more concerning issue
for research integrity and assessments. Research into better measurements of
impact have been active [20, 6, 61, 32, 50, 3], though widely accepted
alternatives have yet to emerge.

####  2.3.2 Saliency Measure

To address these issues, the saliency measure in MAG is used as a
supplementary metric for research impact in this study. As described in [55],
the gaming of the citation counts is treated as a link spam detection problem
[8, 23, 60], and the known effective method, namely, using the eigenvalue
centrality of the citation graph, is employed in the saliency design to combat
this problem. Popularized by Google’s PageRank and widely experimented in
bibliometric studies [50], the eigenvalue centrality uses the number of
references made to the cited articles as an indication to the relevance of the
citation, and citations from more reputable sources carry more weights. Both
are desired properties in alleviating the effects from the illicit citation
behaviors.

Additionally, the saliency measure utilizes the time stamp on each citation to
capture the temporal dynamics of the eigenvalue centrality to avoid biases
against newer articles. Rather than simply counting citations that have
already occurred, the saliency measure is designed to estimate the eigenvalue
centrality of the citation graph 5 years into the future, leveraging the
remarkable prediction power of the reinforcement learning algorithm that is
convincingly demonstrated in DeepMind’s AlphaZero [45] in anticipating the
opponent’s future moves in the game of Go. Whereas in a game setting the
learner can be programmed to play against itself to harvest the sufficient
amount of training data, MAG has over two centuries worth of citation records
for the learner to travel back in time and learn to predict. The outcome is a
measure that characterizes the likelihood of an article to be highly cited in
the next 5 years. As research is constantly built upon accumulated knowledge
and research articles tend to cite up-to-date information, empirical evidence
[55] suggests the saliency measure can effectively identify recent articles as
having higher impact potential than older ones, an observation that can also
be made in this study (Sec. 3).

Finally, the probabilistic nature of an eigenvalue centrality also provides a
mathematically rigorous framework to aggregate article-level impact from
various fields into a measure of a higher level. To be more specific, let
$s(c,v,t,f,a,o)$ denote the saliency of the article $c$ published at venue $v$
on the publication date $t$ about a topic $f$ by an author $a$ affiliated with
an organization $o$. The saliency of a region $W$ on an aggregate field $F$ at
time $t$ can be obtained by computing the marginal probability distribution
from the saliency as

$s(F,W,t)=\sum_{f\in F}\sum_{o\in W}\sum_{v}\sum_{a}\sum_{c}s(c,v,t,f,a,o)$
(1)

Similarly, the saliency of an author or an venue can be computed by summing
the saliencies of all articles by the author or in the venue. Putting it all
together, an article must receive sustained references from high saliency
articles by high saliency authors published in high saliency venues. Citation
cartels with unknown authors using articles published in low quality journals,
for example, may distort the citation count of a target article more easily
than manipulate its saliency.

As eigenvalue centrality is also used in many economic studies, most notable
the Nobel winning input-output model, it is not surprising that the properties
of saliency can be understood in the same manner. Particularly, saliency could
be viewed to bibliometric analysis akin to price indices, i.e., inflation in
economics. For example, a 1-2% inflation over one or two-year horizon could be
barely noticeable but over a long period such as a decade, could erode 20% of
the cash value. The time value of money decreases the value of a currency over
time. In similar manner, citation metrics are fraught with biases stemming
from authorship, fields of study, concepts, entities involved (sponsoring
agency, university, etc.), or time-lag of accruing citation. These biases
could erode the time value of scientific contribution. In the same manner
inflationary bias are caused [17, 43], the life cycle of research could be
biased by many factors. Saliency accounts for biases caused to citation
metrics by accounting for temporal, field-wise, geographic variations through
an explainable probabilistic measure to normalize research impact.

####  2.3.3 Saliency Ratio

In this work we study the trend of convergence by examining the signs that
suggest the elite dominance is attenuated with a specified condition, such as
within in time frame or within a region. While the saliency measure, even the
marginalized one of 1, gives an absolute measurement agnostic to the
underlying condition, its probabilistic nature affords a well motivated and
mathematically rigorous processing, namely, the conditional probability, to
quantify the performance of an entity “relative” to its peers under the
condition. Such relative measure is quite intuitive as it describes the
percentage share of the entity. For instance, to study how dominant a country
$C$ in region $W$ is in generating impact for a research area $F$ at time $t$,
we quantify the degree of dominance as the conditional probability

$R(F,C\subseteq W,t)=s(F,C,t|W)=\frac{s(F,C,t)}{s(F,W,t)}$ (2)

where $s(F,C,t)$ can be obtained by replacing the summation over $o\in W$ with
$o\in C$.

As the conditional probability, by definition, assumes the form of a ratio
between the intersection event versus the condition event, we call the
conditional saliency the saliency ratio in the following.

###  2.4 Research Collaboration

The trends in the world regions are studied in terms of their individual
research output and impact. Additionally, as the previous studies (e.g., [16])
show the heightened collaborations among the US, EU and China in recent years,
we also tease apart the outputs and impacts resulted from collaborations among
these three regions. Generally speaking, all articles are first reduced down
to the unique countries/regions of their author affiliations without
weightings, namely, multiple affiliations from the same country/region are
counted only once for the country/region. Each article associated with more
than one country/region is viewed as a product of collaborative research with
equal contributions from all its countries/regions. These articles are counted
for research output and impact of their respective regions as well as in the
trans-regional research collaboration. For example, an article has two co-
authors from the US, three from China, and another one from Europe. This
article is counted once towards the regional output and impact of US, China,
and Europe individually, and again for US-Europe, US-China, and Europe-China
collaborations, respectively. The number of authors from each region has no
effect in this study: this choice simplifies the considerations for authors
affiliated with multiple affiliations of different regions.

##  3 Results

The 2020 World Bank region codes for high-income countries (HIC’s), upper and
lower middle-income countries (UMCs and LMCs), and low-income countries
(LIC’s) are used to present the results. This study examines all STEM fields
for all territories with World Bank region codes and the data are available
for download in the Supplementary Material.

Figure 1: The global STEM research output (top) and impact as measured in
citation counts (middle) and saliencies (bottom) for the World Bank country
groups for past 4 decades.

Figure 1 shows the contribution margins in the research output and the impact,
in either citation counts or saliencies, for these four groups over the past
four decades. At first glance, these results suggest a convergence in both
research output and impact as the dominance of HICs is yielding to more
contributions from the lower income countries. Particularly, the UMCs have
accounted for more than one quarter of the world’s research output and
approximately 15% of the impact, impressive growths from low single digit
shares in the 1980’s. Additionally, a necessary condition for convergence is
lower income countries will grow much faster starting from a lower base. This
is true between HICs and UMCs but does not hold for LMCs and LICs, suggesting
a chasm may be forming between the UMCs and the LMCs. The latter can be
clearly seen in Figure 1: while LMCs and UMCs account for similar output and
impact in 1980’s, UMCs have produces almost 5 times more output and impact by
2010’s than LMCs. Most importantly, the data show that highly aggregated
results as presented in Figure 1 can be misleading for the following two
reasons: first, the trends in STEM sub-fields exhibit considerable nuances
and, second, the trends within distinct region groups also follow different
patterns: while a convergence appears to take place within HICs, the opposite
seems to be true within UMCs. To better illustrate these points, we pick a
“bellwether" country in each group, the United States (US) for the HICs and
China (CN) for the UMCs, and examine their relative dominance in the groups
they belong.

###  3.1 STEM Overall

Figure 2 shows the overall STEM research output and impact for all country
groups for the four decades straddling the turn of the century. As can be seen
in Figure 2, groups ranging from HICs through LICs have all produced more
scholarly publications growing at exponential rates, with lower income groups
seeing a higher growth rate off a lower base. For example, the three decade-
over-decade growth rates for HICs and LICs are 112.6%, 92.5%, 81.2% and
170.1%, 187.7%, 281.8%, respectively. While these growth rates may appear to
meet the theoretical requirement for a convergence, the growth rates of LICs,
nevertheless, are eclipsed by the middle income countries. In fact, most
amazingly is the output from UMCs that, in 1980’s, was more than an order of
the magnitude behind HICs but has dramatically narrowed the gap in the most
recent decade. One might take this development as an encouraging sign of
convergence, yet the data also show the catching up of UMCs is at the price of
a widening gap between UMCs and LMCs, from virtually non-existent in 1980’s to
more than half of an order of the magnitude in the most recent decade.
Consequently, the output gaps between higher and lower incoming regions show
little sign of narrowing, suggesting the catch-up effect has yet to emerge a
the global scale.

Figure 2: STEM research output, as measured by publication count (top), and
impact, measured by citation counts (middle) and saliency (bottom), for
country groups per World Bank regional codes for the past four decades.

Notably, the convergence within individual groups, which can be seen by
comparing the contribution ratios from the bellwether countries, depicts
diametrical trends between the two upper income groups. Within HICs, the ratio
of US output has reduced, suggesting the growing contributions from European
and other high income countries have lessened the dominance of US. For UMCs,
on the other hand, the opposite is true as China’s research output is
accounting for an ever higher ratio of the group’s output (reaching more than
50% in the recent two decades), making China _the_ undisputed dominant country
among the UMCs. In fact, China alone has even surpassed the combined output of
all LMCs since the first decade of the $21^{th}$ century.

A second and related question is whether there is a convergence in research
impact. Data points that may shed lights on the answer are shown in Figure 2
where the conventional citation count and the novel saliency measures are used
to quantify the community-wide recognition of impact (Sec. 2.1). Here, the
citation count measure makes it difficult to arrive at a definitive conclusion
as the citation counts of HICs, trending downwards, are confounded by the
temporal bias against newer reports and cannot support the argument that the
recent research from HICs has lower impact. It is, however, remarkable to note
that the citation measure for UMCs bucks the trend and continues to show
growth despite of the same temporal bias.

In contrast, after adjusting for the temporal bias, the saliency measure shows
that all income groups have produced research with increasing impact and the
HICs, although with a higher base, continue to dominate the growth. Notably,
the gap between HICs and UMCs has been narrowing. The impact from the United
States alone, for example, still doubles that of all of the combined work from
the UMCs even though its share is lessened considerably among its HICs peers.
The results suggest a convergence in research impact among the HICs. However,
like the case of research output, the trend in UMCs shows the increasingly
dominant role of its bellwether country, China, that in the most recent decade
has seen its saliency ratio grow to over 50%, i.e., accounting for more than
half of the research impact from the UMC group. Again, the research from lower
income countries of LMCs and LICs has yet to play a significant role in the
global research arena. The data support the argument that the lower income
countries lag even more in their impact in recent decades, with China alone
surpassing all their combined work by a wide margin. .

###  3.2 Tales of three sub-fields

The highly aggregated results of STEM run the risk of painting the picture
with too broad a brush as the data show uneven rates of convergence in the
STEM sub-fields: While there are fields seeing more pronounced convergence as
in AI, Chemistry and Materials Science at least among the higher income
categories, there are still many fields, e.g., Medicine, where the historical
leaders remain overly dominant. To illustrate this point, we selectively
discuss the results of three representative fields, AI, Medicine and Materials
Science, respectively, noting that the data in the supplementary material can
provide deeper evidence for other fields.

####  3.2.1 Artificial Intelligence (AI)

Figure 3: Research output, measured by publication count (top), and impact, by
citation count (middle) and saliency (bottom), for the field of AI that is one
of the fields where the globalization convergence in research output seems to
accelerate within HICs and UMCs in recent decades, yet the gain does not
translate proportionally to research impact.

AI, a field rooted in Computer Science but with rich history of
interdisciplinary cross-pollination, is an example in which the convergence in
research output outpaces STEM, and the sign of a convergence in research
impact is modest but encouraging. As can be seen in Figure 3, the current
landscape bears little resemblance to that of the last century where the field
was largely propelled by the HICs and, within it, dominated by the US in
research impact but not in output. During the four decades, the publication
and saliency ratios of HICs change from 93.6% and 97.0% in 1980’s to 68.8% and
80.9%, respectively. Within HICs, the quality of AI research has caught up
since the turn of the century that the US has accounted for less then half of
the impact. To be more specific, the US accounts for 56.9% of research output
and has a saliency ratio 70.8% in 1980’s, but in the most recent decade these
two indicators have lowered to 20.0% and 33.2%, respectively.

In the meantime, the lower income countries have produced more research in AI,
especially the impressive gain by the UMC group. UMCs have collectively
generated research output larger than the US alone and, by the second decade,
the bellwether country in the group, China, has matched the US in research
output. In terms of research impact, however, UMCs have made impressive
progress but still have a significant room left in order to catch up with
HICs. For the most recent decade, the impact of the US alone is approximately
twice as high as the collective impact from UMCs. As in the case of general
STEM, UMCs show scant signs of convergence as the group’s research output and
impact are largely dominated by China. The publication and saliency ratio of
China have grown from 1.0% and 0.7% in 1980’s to 20.6% and 12.6% in 2010’s,
respectively.

####  3.2.2 Medicine

Figure 4: Research output and impact for the field of Medicine, one that no
significant convergence can be observed in recent decades, either in research
output or impact.

Medicine is one of the fields where the globalization seems to have changed
little the origins of research output and impact. As shown in Figure 4, the
field remains dominated by the HICs, even though the lower incoming countries
have collectively made a substantial gains. Within the four decades, HICs’
research output has changed from 95.8% to 77.3%, and research impact from
97.7% to 87.9%, with the US alone still far exceeding the collective impact
from the lower incoming countries.

####  3.2.3 Materials Science

Figure 5: Research output and impact for the field of Materials Science, where
the UMCs have significantly narrowed the gaps with HICs in both output and
impact.

Materials Science is one of the few fields where the sign of convergence is
most observable in that no single country in the HICs has produced more
research impact than the lower incoming countries combined. As can be seen in
Figure 5, the field has seen the output and impact gains from the lower income
countries that the publication and saliency ratios from the HICs are reduced
from 91.9% and 96.0% in 1980’s to 58.9% and 67.9%, respectively. Most
remarkably, while the US alone still produced more output and impact than the
combined work of the lower income countries in the first decade of the
century, it is no longer the case in the most recent decade. In fact, China
has overtaken US in output and is quickly catching up on impact. Notably,
while the convergence within the HIC group is clear, the data suggest the same
may also be true within the UMCs as the dominance of China seems to be
loosened somewhat between the first and the second decade of the century. This
observation, however, has to be taken with a grain of salt because it is based
on a single data point, and there are limitations in our data that only
consider publications addressed to the international audience, i.e., written
in English. The latter point is further elaborated in Sec. 4.

###  3.3 The role of cross-border collaboration

At the root of globalization is collaboration across regional boundaries, and
the momentum of such collaboration may be a leading indication of an eventual
convergence. Literature abounds in reporting increasing research output as
results of collaboration (Sec. 1), and our data, as shown Figure 6, confirm
these observations. Aside from the quantity, we further examine the quality of
collaborative research, measured by the saliency ratio of such work, and find
the trends most encouraging. While the detailed data in the Supplementary
Material offer in-depth insights into individual countries, the state of the
global collaborations can be best described by contrasting the recent rise of
China, be it the transpacific collaborations with the US or the Sino-European
activities, against the traditional transatlantic research collaborations
between the US and the European countries. Either in terms of output or
impact, the trends in research collaborations bear remarkable parallels to the
regional results described above.

Figure 6: Research output of global collaborations for the most recent decade
for STEM research and three representative sub-fields. The output is measured
by the number of research articles published in the years from 2010 through
2019.

Figure 6 shows the output as a result of transregional collaborations of the
transatlantic (US-EU), the transpacific (US-CN), and the Sini-European
research in the most recent decade. Generally speaking, STEM research has seen
growing collaborations that track the growth of research output overall, and
the transpacific collaborations seems to be gaining on the traditional
transatlantic ones. However, the highly aggregated STEM results do not
highlight nuanced details varying significantly among sub-fields. For
instance, Medicine is the area where the pattern has hardly evolved from the
dominant transatlantic collaborations dated back for decades, while the
transpacific collaborations have noticeably overtaken transatlantic ones in AI
and Materials Science. The sign of the catch-up effect in research output (at
least between the HICs and the UMCs) is most and least observable in the
Materials Science and Medicine, with AI in between.

Figure 7: Research impact, measured in saliency ratio of the STEM overall
impact, for collaborative research for the past decade.

Similarly in terms of impact, the data suggest that STEM research
collaborations are playing an increasingly important role relative to the work
produced in the same year. This can be seen in Figure 7 as the saliency of
collaborative work is accounting more share of the impact of the same year.
Indeed, there are fields such as Medicine where the transatlantic
collaborations still dominate in impact, but the fields of AI and Materials
Science have seen transpacific collaborations upend the traditional norms. The
transpacific collaborations in Materials Science, the research impact from
transpacific collaborations has become more than double that of the
transatlantic ones, and the transpacific collaborations in AI have surpassed
transatlantic collaborations in research impact since 2015. In fact, the data
show most of the impact gains in China’s AI research in recent decades can be
largely attributed to research with global collaborations. The fact that
collaborative work increasingly accounts for a higher saliency ratio suggest
high impact research is likely a result of global collaboration, an
encouraging indication to support a convergence of globalized research.

The varying degrees of convergence in these fields reflect clearly in their
respective collaboration trends as shown in Figures 8 and 9 where the
percentage of each country’s published articles as a result of transregional
collaboration (left) is juxtaposed against their corresponding impact (right)
for the most recent decade. Consistently across all fields and throughout all
the years are research articles with transregional collaboration among these
regions increasing in their output shares, and the collaborative research
account for proportionally larger impact than those that are not. Take year
2019 as an example. As shown in Figure 8, 17.0% of the output from China in
all STEM research is a result of collaboration with US, and this transpacific
research collaboration accounts for 36.0% of the total impact from China of
that year, a significant growth from 5.7% and 20.4%, respectively, from a
decade earlier. In contrast, the US-EU collaboration has its research output
hovering relatively steadily between 10.5% and 15.4% percentage points for
either region’s total, with European countries seeing their collaborative
research account for a slightly larger portion of their total impact (between
20.5% and 29.6% versus 15.7% to 20.3% for the US). The fact that EU and China
are gaining more in research collaboration with the historically dominant US
is consistent with the economical model of convergence supported by the data
both in research output and impact shown previously.

Figure 8: Shares of STEM research output (left) and impact (right) as a result
of transregional collaboration with respect to individual countries of US, EU,
and China for the past decade. The share of research output and impact are
measured by the percentage of published articles and the saliency ratio,
respectively.

Figure 9: Percentages of transregional collaboration with respective to
regional totals of research output (left) and impact (right) for the fields of
Medicine (top), AI (middle) and Materials Science (bottom).

Again, the highly aggregated STEM results masquerade intricacies in its sub-
fields, especially in the intimate relationship between research collaboration
and convergence. Again, the results from Medicine, AI and Materials Science,
representing the fields of early, middle, and advanced stages in convergence,
are shown in Figure 9 to illustrate this point. The field of Medicine, with
research output and impact shown to be dominated by historical elites (Figures
4) and collaboration by transatlantic partnerships (Figures 6 and 7),
nevertheless sees significant outreach from China. Though still low in
absolute quantities, the collaborations with the US and European countries are
growing steadily, with their shares of impact reaching 33.6% and 13.6% in year
2019, out of 13.1% and 5.4% of China’s research output, respectively. These
out-sized gains in proportional impact have rivaled the European paybacks in
their collaboration with the US, and can be a sustaining driver for future
collaboration.

On the other end of the research convergence is the field of Materials Science
where, as shown in Figure 9, the trend has exhibited a remarkable role
reversal in recent years. There, in the middle of the most recent decade, the
transpacific collaboration, both in terms of research output and impact, has
overtaken the transatlantic one. Most amazingly, the US-China collaboration
has accounted for larger portion of US’s output (27.0% in year 2019) and
impact (34.3% in 2019) than of China’s in recent years (17.3% and 33.6% in
year 2019, respectively). Although at lower percentage points, the Sino-
European collaboration tells a similar story: since the onset of the decade,
such collaboration has accounted for a slightly higher portion of the research
output for Europe than China, and since the mid decade, the research impact.
Unlike the US, however, Europe still produces relatively more research output
and impact from the collaboration with US than with China.

Straddled in the middle is the field of AI where China has surpassed US in
output but still is dwarfed in impact by the US (Figure 3), and the
transpacific collaboration has lead to higher output and impact than the
transatlantic (Figures 6 and 7). These give rise to US seeing larger portions
of its output and impact from its collaboration with China in recent years
than with Europe earlier in the most recent decade, as shown in Figure 9.
Unlike the case in Materials Science, however, the transpacific collaboration
only accounts for a higher share for US than China in research output but not
for the research impact yet. Specifically, transpacific collaboration has
grown to account for 16.3% of US’s output versus China’s 15.7% in year 2019,
while, for research impact, to 17.5% of US’s and 35.0% of China’s. In
contrast, the transatlantic collaboration has steadily hovered around 10% of
US’s output and 15% of US’s impact throughout the decade. The shares of Sino-
European collaboration have also climbed for Europe and China in both output
and impact, though the percentage points are much lower than their respective
collaborations with the US.

##  4 Discussion

This paper examined output, impact, and collaboration in artificial
intelligence (AI) and technology research for more than 200 countries/regions
in the past four decades. Using a global corpora of journal publications,
conference papers, and patents that include more than 56 million articles and
1.7 billion citations, the paper examines whether developing countries are
catching up (or diverging) with advanced economies in AI and scientific
fields. The results show that convergence is accelerating, but only in the
case of UMCs catching up with HICs, and the US becoming less dominant within
HICs. The gap between UMCs and LMCs is widening, supporting evidence for
divergence. Similarly, low-income countries (LICs) are diverging from LMCs and
LMCs are diverging from UMCs. Various sub-fields show significantly different
rates of convergence. While fields like Medicine show the least visible signs
of convergence, Material Science exhibits an accelerated state of convergence.
The enabling field of AI is in the middle stage of convergence, with UMCs
catching up with HICs in research output but not yet in impact.

Since this study is a quantitative one, we pay special attention to the
measurements of research output and impact. In addition to the broadly adopted
proxies of publication counts and citation counts for output and impact,
respectively, we examine the efficacy of saliency, a research impact measure
included in MAG that employs a modern AI method to rectify several known
problems in using citation counts. We have found that the measurement
mitigates against the temporal bias in citations and its clear mathematical
foundation provides a solid analytical framework to avoid the inexplicable and
embarrassing conclusions derived from alternatives. The rigorous approach to
quantifying research lends us confidence in describing the remarkable trends
of convergence and divergence, especially the dramatic developments in recent
decades. In a way, the measurements in this study are chosen rather
conservatively to avoid unresolved research problems. For this reason, we have
not considered data sets and software tools alone as evidence of research
output or using Altmetrics for impact although these proposals have received a
lot of attentions recently [47, 14, 26, 9].

Aside from reporting observations in the data, we attempt to uncover the
potential drivers of convergence by examining the heightened inter-regional
collaborations, an outcome of globalization. The data clearly shows that
collaborations benefits all parties involved. Specifically for the case study
on the rise of China, we have found that the country’s increasing output is a
result of collaborating with HICs such as the US and the European countries,
and these collaborations always account for more than commensurate impact. A
closer look into the data suggests the rise of China is consistent with the
preferential attachment process that has been theorized as a mechanism
underlying the formation of naturally occurring networks, including social and
research collaboration networks [7, 37, 1]. The central idea here is that
newcomers can receive faster and broader recognition by associating themselves
with those already highly recognized. Producing joint research output is
certainly an effective means, and data are consistent with the catch-up effect
observed for Europe catching up with the US in earlier decades, and China with
the HICs more recently. The lack of similar collaboration patterns from LMCs
and LICs may well be a pivotal reason to explain their divergence. Note that
the preferential attachment process (or a more specialized cumulative
advantage process [15]) does not prescribe a sufficient condition for catching
up. Indeed, the theory was originally developed to explain the inequality of
the Matthew effect [34] which suggests that the highly recognized will get
more recognition, and even eclipse the attribution to the newcomers if the
same discoveries are made by both at the same time. Nevertheless, the data
suggest the same process that leads to deepening inequality between competing
parties can also be deftly employed to induce the catch-up effect through
joint work.

This finding suggests that technology progress is tightly coupled with network
effects, and therefore policymakers should view bilateral relationships as an
opportunity rather than a constraint. Historically, the US-EU were the largest
scientific collaborators. However, the US-China collaboration in AI, for
instance, has become the largest AI partnership today, heralding a new wave of
technology globalization. Developing countries can learn from China’s growth
pattern in research output and impact. The tides of “rich getting richer”
phenomenon can be turned by building high-impact collaborations and catch up.
When countries collaborate across borders, larger research impact is observed.
The power of collaboration should be explicitly included when designing
science policy.

Finally, the data show the rate of convergence is uneven across fields,
suggesting the catch-up effect may be equally achievable. As developing
countries discover the importance of collaboration in technology research,
they should use network effects to their advantage to grow in output and
impact in fields of their specialization. Policymakers have to make a
conscious effort to exercise preferential attachment to foster collaborations
with scientific leaders in fields of strategic importance.

## References

  * [1] Alireza Abbasi, Liaquat Hossain, and Loet Leydesdorff.  Betweenness centrality as a driver of preferential attachment in the evolution of research collaboration networks.  Journal of Informetrics, 6(3):403–412, 2012.
  * [2] Philippe Aghion, Paul A David, and Dominique Foray.  Science, technology and innovation for economic growth: linking policy research and practice in ‘stig systems’.  Research policy, 38(4):681–693, 2009.
  * [3] Mohammad Ahmadpoor and Benjamin F. Jones.  Decoding team and individual impact in science and invention.  Proceedings of the National Academy of Sciences of the United States of America, 116(28):13885–13890, 2019.
  * [4] Nur Ahmed and Muntasir Wahed.  The de-democratization of ai: Deep learning and the compute divide in artificial intelligence research.  arXiv preprint arXiv:2010.15581, 2020.
  * [5] OECD AI.  Oecd AI policy observatory.
  * [6] Dag W. Aksnes, Liv Langfeldt, and Paul Wouters.  Citations, citation indicators, and research quality: An overview of basic concepts and theories.  SAGE Open, 9(1), 2019.
  * [7] Reka Zsuzsanna Albert and Albert-Laszlo Barabasi.  Statistical mechanics of complex networks.  Reviews of Modern Physics, 74(1):47–97, 2001.
  * [8] A. Benczcur, Casalogany K, T. Sarlos, and M. Uher.  Spamrank – fully automatic link spam detection.  In AIRWeb, pages 25–38, 2005.
  * [9] Lutz Bornmann.  Do altmetrics point to the broader impact of research? an overview of benefits and disadvantages of altmetrics.  Journal of Informetrics, 8(4):895–903, 2014.
  * [10] Ross Cagan.  The san francisco declaration on research assessment, 2013.
  * [11] Cong Cao, Ning Li, Xia Li, and Li Liu.  Reforming china’s s&t system.  Science, 341(6145):460–462, 2013.
  * [12] Cong Cao, Richard P. Suttmeier, and Denis Fred Simon.  China’s 15-year science and technology plan.  Physics Today, 59(12):38–43, 2006.
  * [13] Colin A. Chapman, JÃºlio CÃ©sar Bicca-Marques, SÃ©bastien Calvignac-Spencer, Pengfei Fan, Peter J. Fashing, Jan Gogarten, Songtao Guo, Claire A. Hemingway, Fabian Leendertz, Baoguo Li, Ikki Matsuda, Rong Hou, Juan Carlos Serio-Silva, and Nils Chr. Stenseth.  Games academics play and their consequences: how authorship, h-index and journal impact factors are shaping the future of academia.  Proceedings of The Royal Society B: Biological Sciences, 286(1916):20192047, 2019.
  * [14] Rodrigo Costas, Zohreh Zahedi, and Paul Wouters.  Do “altmetrics” correlate with citations? extensive comparison of altmetric indicators with citations from a multidisciplinary perspective.  Journal of the Association for Information Science and Technology, 66(10):2003–2019, 2015.
  * [15] Derek J. de Solla Price.  A general theory of bibliometric and other cumulative advantage processes.  Journal of the Association for Information Science and Technology, 27(5):292–306, 1976.
  * [16] Yuxiao Dong, Hao Ma, Zhihong Shen, and Kuansan Wang.  A century of science: Globalization of scientific collaborations, citations, and innovations.  In Proceedings of the 23rd ACM SIGKDD International Conference on Knowledge Discovery and Data Mining, pages 1437–1446, 2017.
  * [17] John Driffill.  Macroeconomic policy games with incomplete information: A survey.  European Economic Review, 32:533–541, 1988.
  * [18] Steven N Durlauf.  On the convergence and divergence of growth rates.  The Economic Journal, 106(437):1016–1018, 1996.
  * [19] Iztok Fister and MatjaÅŸ Perc.  Toward the discovery of citation cartels in citation networks.  Frontiers in Physics, 4:49, 2016.
  * [20] Santo Fortunato, Carl T. Bergstrom, Katy Börner, James A. Evans, Dirk Helbing, Staša Milojević, Alexander M. Petersen, Filippo Radicchi, Roberta Sinatra, Brian Uzzi, Alessandro Vespignani, Ludo Waltman, Dashun Wang, and Albert László Barabási.  Science of science.  Science, 359(6379), 2018.
  * [21] Jon Frost, Leonardo Gambacorta, and Romina Gambacorta.  The matthew effect and modern finance: on the nexus between wealth inequality, financial development and financial technology.  Questioni di Economia e Finanza (Occasional Papers), 2020.
  * [22] Hariolf Grupp and Mary Ellen Mogee.  Indicators for national science and technology policy: how robust are composite indicators?  Research Policy, 33(9):1373–1384, 2004.
  * [23] Zoltain Gyongyi¶ngyi and Hector Garcia-Molina.  Link spam alliances.  In VLDB ’05 Proceedings of the 31st international conference on Very large data bases, pages 517–528, 2005.
  * [24] M Ryan Haley.  On the inauspicious incentives of the scholar-level h-index: an economist’s take on collusive and coercive citation.  Applied Economics Letters, 24(2):85–89, 2017.
  * [25] Xueying Han and Richard P. Appelbaum.  China’s science, technology, engineering, and mathematics (stem) research environment: A snapshot.  PLOS ONE, 13(4):1–22, 2018.
  * [26] Stefanie Haustein, Isabella Peters, Cassidy R. Sugimoto, Mike Thelwall, and Vincent Larivière.  Tweeting biomedicine: An analysis of tweets and citations in the biomedical literature.  Journal of the Association for Information Science and Technology, 65(4):656–669, 2014.
  * [27] A Haynes and L Gbedemah.  The global ai index: methodology.  Tortoise Media, 2019.
  * [28] Tianwei He.  International scientific collaboration of china with the g7 countries.  Scientometrics, 80(3):571–582, 2009.
  * [29] Claudiu Herteliu, Marcel Ausloos, Bogdan Vasile Ileanu, Giulia Rotundo, and Tudorel Andrei.  Quantitative and qualitative analysis of editor behavior through potentially coercive citations.  Publications, 5(2):15, 2017.
  * [30] B Ian Hutchins, Xin Yuan, James M Anderson, and George M Santangelo.  Relative citation ratio (rcr): A new metric that uses citation rates to measure influence at the article level.  PLoS biology, 14(9):e1002541, 2016.
  * [31] Loet Leydesdorff, Caroline S Wagner, and Lutz Bornmann.  The european union, china, and the united states in the top-1% and top-10% layers of most-frequently cited publications: competition and collaborations.  Journal of Informetrics, 8(3):606–617, 2014.
  * [32] Hao Liao, Manuel Sebastian Mariani, MatÃºÅ¡ Medo, Yi-Cheng Zhang, and Ming-Yang Zhou.  Ranking in evolving complex networks.  Physics Reports, 689:1–54, 2017.
  * [33] Robert MacArthur and Richard Levins.  The limiting similarity, convergence, and divergence of coexisting species.  The american naturalist, 101(921):377–385, 1967.
  * [34] Robert K Merton.  The matthew effect in science: The reward and communication systems of science are considered.  Science, 159(3810):56–63, 1968.
  * [35] Joel Mokyr.  Building taller ladders.  Finance and Development, 55(2):32–35, 2018.
  * [36] National Science Foundation National Science Board.  Publication output: U.S. trends and international comparisons.  In Science and Engineering Indicator 2020, 2019.
  * [37] Mark EJ Newman.  Clustering and preferential attachment in growing networks.  Physical review E, 64(2):025102, 2001.
  * [38] Mathias Wullum Nielsen and Jens Peter Andersen.  Global citation inequality is on the rise.  Proceedings of the National Academy of Sciences, 118(7), 2021.
  * [39] Oren Perez, Judit Bar-Ilan, Reuven Cohen, and Nir Schreiber.  The network of law reviews: citation cartels, scientific communities, and journal rankings.  The Modern Law Review, 82(2):240–268, 2019.
  * [40] Raymond Perrault, Yoav Shoham, Erik Brynjolfsson, Jack Clark, John Etchemendy, Barbara Grosz, Terah Lyons, James Manyika, Saurabh Mishra, and Juan Carlos Niebles.  The ai index 2019 annual report.  AI Index Steering Committee, Human-Centered AI Institute, Stanford University, Stanford, CA, 2019.
  * [41] Derek J De Solla Price.  Networks of scientific papers.  Science, pages 510–515, 1965.
  * [42] Amrita Purkayastha, Eleonora Palmaro, Holly J Falk-Krzesinski, and Jeroen Baas.  Comparison of two article-level, field-independent citation metrics: Field-weighted citation impact (fwci) and relative citation ratio (rcr).  Journal of Informetrics, 13(2):635–642, 2019.
  * [43] Pascal Seiler.  Weighting bias and inflation in the time of covid-19: evidence from swiss transaction data.  Swiss Journal of Economics and Statistics, 156(1):1–11, 2020.
  * [44] Yigong Shi and Yi Rao.  China’s research culture.  Science, 329(5996):1128–1128, 2010.
  * [45] David Silver, Julian Schrittwieser, Karen Simonyan, Ioannis Antonoglou, Aja Huang, Arthur Guez, Thomas Hubert, Lucas Baker, Matthew Lai, Adrian Bolton, Yutian Chen, Timothy P. Lillicrap, Fan Hui, Laurent Sifre, George van den Driessche, Thore Graepel, and Demis Hassabis.  Mastering the game of GO without human knowledge.  Nature, 550(7676):354–359, 2017.
  * [46] Denis Simon, Machi F Dilworth, and Alan Tonelson.  The globalization of scientific research.  Issues in Science and Technology, 28(2):10–13, 2012.
  * [47] Mike Thelwall, Stefanie Haustein, Vincent Larivière, and Cassidy R. Sugimoto.  Do altmetrics work? twitter and ten other social web services.  PLOS ONE, 8(5), 2013.
  * [48] Jeff Tollefson.  China declared world’s largest producer of scientific articles.  Nature, 553(7689):390–390, 2018.
  * [49] B Bart Verspagen.  Innovation and economic growth.  The Oxford handbook of innovation, pages 487–513, 2006.
  * [50] Ludo Waltman.  A review of the literature on citation impact indicators.  Journal of Informetrics, 10(2):365–391, 2016.
  * [51] Ludo Waltman and Nees Jan Van Eck.  The inconsistency of the h-index.  Journal of the American Society for Information Science and Technology, 63(2):406–415, 2012.
  * [52] Ludo Waltman and Nees Jan van Eck.  Field normalization of scientometric indicators.  Springer Handbook of Science and Technology Indicators, pages 281–300, 2019.
  * [53] Ludo Waltman, Nees Jan van Eck, Thed N van Leeuwen, Martijn S Visser, and Anthony FJ van Raan.  Towards a new crown indicator: Some theoretical considerations.  Journal of informetrics, 5(1):37–47, 2011.
  * [54] Kuansan Wang.  Opportunities in open science with ai.  Frontiers in Big Data, 2:26, 2019.
  * [55] Kuansan Wang, Zhihong Shen, Chiyuan Huang, Chieh-Han Wu, Darrin Eide, Yuxiao Dong, Junjie Qian, Anshul Kanakia, Alvin Chen, and Richard Rogahn.  A review of microsoft academic services for science of science studies.  Frontiers in Big Data, 2:45, 2019.
  * [56] Allen W. Wilhite and Eric A. Fong.  Coercive citation in academic publishing.  Science, 335(6068):542–543, 2012.
  * [57] Qingnan Xie and Richard B Freeman.  Bigger than you thought: China’s contribution to scientific publications and its impact on the global economy.  China & World Economy, 27(1):1–27, 2019.
  * [58] Yu Xie, Chunni Zhang, and Qing Lai.  China’s rise as a major contributor to science and technology.  Proceedings of the National Academy of Sciences of the United States of America, 111(26):9437–9442, 2014.
  * [59] Wei Yang.  Policy: Boost basic research in china.  Nature, 534(7608):467–469, 2016.
  * [60] Dengyong Zhou, Christopher J. C. Burges, and Tao Tao.  Transductive link spam detection.  In Proceedings of the 3rd international workshop on Adversarial information retrieval on the web, pages 21–28, 2007.
  * [61] Ming-Yang Zhou, Hao Liao, Wen-Man Xiong, Xiang-Yang Wu, and Zong-Wen Wei.  Connecting patterns inspire link prediction in complex networks.  Complexity, 2017, 2017.
  * [62] Ping Zhou and Loet Leydesdorff.  The emergence of china as a leading nation in science.  Research Policy, 35(1):83–104, 2006.