††thanks: Electronic address<EMAIL_ADDRESS>
# Inheritance patterns in citation networks reveal scientific memes

Tobias Kuhn  Chair of Sociology, in particular of Modeling and Simulation, ETH
Zurich, 8092 Zurich, Switzerland  Matjaž Perc  Faculty of Natural Sciences and
Mathematics, University of Maribor, Koroška cesta 160, SI-2000 Maribor,
Slovenia  CAMTP – Center for Applied Mathematics and Theoretical Physics,
University of Maribor, Krekova 2, SI-2000 Maribor, Slovenia  Dirk Helbing
Chair of Sociology, in particular of Modeling and Simulation, ETH Zurich, 8092
Zurich, Switzerland  Risk Center, ETH Zurich, 8092 Zurich, Switzerland

###### Abstract

Memes are the cultural equivalent of genes that spread across human culture by
means of imitation. What makes a meme and what distinguishes it from other
forms of information, however, is still poorly understood. Our analysis of
memes in the scientific literature reveals that they are governed by a
surprisingly simple relationship between frequency of occurrence and the
degree to which they propagate along the citation graph. We propose a simple
formalization of this pattern and we validate it with data from close to 50
million publication records from the Web of Science, PubMed Central, and the
American Physical Society. Evaluations relying on human annotators, citation
network randomizations, and comparisons with several alternative approaches
confirm that our formula is accurate and effective, without a dependence on
linguistic or ontological knowledge and without the application of arbitrary
thresholds or filters.

The evaluation of scientific output and the study of patterns of scientific
collaboration have received increasing attention by researchers. From citation
distributions 1, 2, coauthorship networks 3 and the formation of research
teams 4, 5, to the ranking of researchers 6, 7, 8 and the quantification and
prediction of scientific success 9, 10 — how we do science has become a
science in its own right. While the famous works of Derek de Solla Price 11
and Robert Merton 12 from the mid 1960s marked the beginning of a popular and
long-lasting research field, the rapid progress made in recent years is
largely due to the increasing availability of vast amounts of digitized data.
Massive publication and citation databases, also referred to as
“metaknowledge” 13, along with leaps of progress in the theory and modeling of
complex systems, fuel large-scale explorations of human culture that were
unimaginable even a decade ago 14. The “science of science” is scaling up
massively as well, with studies on global citation and collaboration networks
15, the “scientific food web” 16, and phylomemetic patterns in the evolution
of science 17, culminating in the visually compelling atlases of science 18
and knowledge 19.

Science is a key pillar of modern human culture, and the general concept of
memes has proved to be very insightful for the study of culture. The term
“meme” was coined by Richard Dawkins in his book _The Selfish Gene_ 20, where
he argues that cultural entities such as words, melodies, recipes, and ideas
evolve similarly as genes, involving replication and mutation but using human
culture instead of the gene pool as their medium of propagation. Recent
research on memes has enhanced our understanding of the dynamics of the news
cycle 21, the tracking of information epidemics in the blogspace 22, and the
political polarization on Twitter 23. It has been shown that the evolution of
memes can be exploited effectively for inferring networks of diffusion and
influence 24, and that information contained in memes is evolving as it is
being processed collectively in online social media 25. The question of how
memes compete with each other for the limited and fluctuating resource of user
attention has also amassed the attention of scientists, demonstrating that
social network structure is crucial to understand the diversity of memes 26,
which suggests that social contagion mechanisms 27 play an important role. It
has also been shown that the competition among memes can bring the network at
the brink of criticality 28, where even minute disturbances can lead to
avalanches of events that make a certain meme go viral 29.

While the study of memes in mass media and popular culture has been based
primarily on their aggregated bursty occurrence patterns, the citation network
of scientific literature allows for more sophisticated and fine-grained
analyses. _Quantum_ , _fission_ , _graphene_ , _self-organized criticality_ ,
and _traffic flow_ are examples of well-known memes from the field of physics,
but what exactly makes such memes different from other words and phrases found
in the scientific literature? As an answer to this question, we propose the
following definition that is a modified version of Dawkins’ definition of the
word “gene” 20: _A scientific meme is a short unit of text in a publication
that is replicated in citing publications and thereby distributed around in
many copies; the more likely a certain sequence of words is to be broken
apart, altered, or simply not present in citing publications, the less it
qualifies to be called a meme._ Publications that reproduce words or phrases
from cited publications are thus the analogue to offspring organisms that
inherit genes from their parents. In contrast to existing work on scientific
memes, our approach is therefore grounded in the “inheritance mechanisms” of
memes and not just their accumulated frequencies. The above definition covers
memes made up of exact words and phrases, but the same methods apply just as
well to more abstract forms of memes.

Figure 1:  Citation networks of the Web of Science and the American Physical
Society (APS) datasets reveal community structures that nicely align with
scientific disciplines, journals covering particular subfields, and
occurrences of memes. The generation of the visualizations was based on Gephi
30 and the OpenOrd plugin 31, which implements a force-directed layout
algorithm that is able to handle very large graphs.

For our analyses, we rely on 47.1 million publication records from three
sources. Due to its representative long-term coverage of a specific field of
research, we focus mainly on the titles and abstracts from the dataset of the
American Physical Society, consisting of almost half a million publications
from the Physical Review journals published between July 1893 and December
2009. We also present results for the over 46 million publications indexed by
the comprehensive Web of Science database, and for the over 0.6 million
publications from the open access subset of PubMed Central that covers
research mostly from the biomedical domain and mostly from recent years.

Fig. 1 shows visualizations of these citation graphs. The leftmost network
depicts the entire giant component of the citation graph of the Web of
Science, consisting of more than 33 million publications. Different scientific
disciplines form relatively compact communities: The physical sciences (cyan)
are close to engineering and technology (magenta) in the top right corner of
the network, but rather far from the social sciences and humanities (green) as
well as the medical and health sciences (red), which take up the majority of
the left hand side of the network, with the natural and agricultural sciences
in between (blue). Zooming in on the physical sciences and switching to the
dataset from the American Physical Society, we get the picture shown in the
middle. The colors now encode the five most important special-focus journals
of Physical Review, each covering a particular subfield of physics (general-
coverage and smaller journals are shown in gray). We see a complex structure
with many small and large clusters. Importantly, even though the employed
layout algorithm 31 did not take the scientific disciplines and the journal
information explicitly into account, the different communities can be clearly
inferred in the citation graphs. Following our general meme-centric
perspective, the rightmost network highlights the above-mentioned memes from
physics, which mostly appear in publications that form compact communities in
the citation graph. The meme _quantum_ is widely but by no means uniformly
distributed, pervading several large clusters. Publications containing the
meme _fission_ form a few connected clusters limited to an area that makes up
the journal Physical Review C covering nuclear physics. Similarly, the memes
_graphene_ , _self-organized criticality_ , and _traffic flow_ (see enlarged
area) are each concentrated in their own medium-sized or small communities.

## Results

All words and phrases that occur frequently in the literature can be
considered important memes, but many frequent words like “method” are not
particularly interesting for any given scientific field. To quantify the
degree to which a meme is interesting, we define the propagation score
$P_{m}$, which determines the alignment of the occurrences of a given meme
with the citation graph. $P_{m}$ is high for memes that frequently appear in
publications that cite meme-carrying publications (“sticking”) but rarely
appear in publications that do not cite a publication that already contains
the meme (“sparking”). Formally, we define the propagation score for a given
meme $m$ as its _sticking factor_ divided by its _sparking factor_. The
sticking factor quantifies the degree to which a meme replicates in a
publication that cites a meme-carrying publication. Concretely, it is defined
as $d_{m{\rightarrow}m}/d_{{\rightarrow}m}$, where $d_{m{\rightarrow}m}$ is
the number of publications that carry the meme and cite at least one
publication carrying the meme, while $d_{{\rightarrow}m}$ is the number of all
publications (meme-carrying or not) that cite at least one publication that
carries the meme. Similarly, the sparking factor quantifies how often a meme
appears in a publication without being present in any of the cited
publications. It is thus defined as
$d_{m{\rightarrow}\cancel{m}}/d_{{\rightarrow}\cancel{m}}$, where
$d_{m{\rightarrow}\cancel{m}}$ is the number of meme-carrying publications
that do _not_ cite publications that carry the meme, and
$d_{{\rightarrow}\cancel{m}}$ is the number of _all_ publications (meme-
carrying or not) that do _not_ cite meme-carrying publications. For the
propagation score $P_{m}$, we thus obtain

$P_{m}=\left.\frac{d_{m{\rightarrow}m}}{d_{{\rightarrow}m}}\middle/\frac{d_{m{\rightarrow}\cancel{m}}}{d_{{\rightarrow}\cancel{m}}}\right..$
(1)

Based on the propagation score $P_{m}$ and the frequency of occurrence $f_{m}$
(which is simply the ratio of publications carrying the meme) of a particular
meme $m$, we define the _meme score_ $M_{m}$ as

$M_{m}=f_{m}P_{m}.$ (2)

The propagation score, as defined in Eq. 1, can be improved by adding a small
amount of controlled noise $\delta$, thus obtaining

$P_{m}=\left.\frac{d_{m{\rightarrow}m}}{d_{{\rightarrow}m}+\delta}\middle/\frac{d_{m{\rightarrow}\cancel{m}}+\delta}{d_{{\rightarrow}\cancel{m}}+\delta}\right..$
(3)

This corrects for the fact that any of the four basic terms can be zero, and
it also prevents that phrases with a very low frequency get a high score by
chance. The controlled noise corresponds to $\delta$ fictitious publications
that carry all memes and cite none, plus another $\delta$ publications that
carry no memes and cite all. This decreases the sticking factors and increases
the sparking factors of all memes, thereby reducing all meme scores — very
slightly so for frequent memes but heavily for rare ones. Our tests show that
a small value of $\delta$ (e.g. $\delta=3$ as used throughout this work unless
stated otherwise) is sufficient. Another matter that deserves attention is the
potential “free-riding” of shorter memes on longer ones. For example, the
multi-token meme “the littlest Higgs model,” contains the specific token
“littlest” that rarely occurs otherwise. The meme “littlest” therefore gets
about the same propagation score as the long meme, yet the larger meme is
clearly more interesting. This can be addressed by discounting for free-riding
by redefining the term $d_{m{\rightarrow}m}$ in Eq. 1 to exclude publications
where the given meme appears in the publication and its cited publications
only within the same larger meme. If “littlest,” for example, is always
followed by “Higgs” in a given publication and all its cited publications,
then this publication shall not contribute to the $d_{m{\rightarrow}m}$ term
for $m=\mbox{``littlest''}$.

The meme score considers whether a meme is important ($f_{m}$) and whether it
is interesting ($P_{m}$), and it has additionally a number of desirable
properties: (i) it can be calculated exactly without the introduction of
arbitrary thresholds, such as a minimum number of occurrences, without
limiting the length of $n$-grams to consider, and without filtering out words
containing special characters; (ii) it does not depend on external resources,
such as dictionaries or other linguistic data; (iii) it does not depend on
filters, like stop-word lists, to remove the most common words and phrases;
and (iv) it is very simple with only one parameter ($\delta$).

Figure 2:  Universality in the distribution patterns of scientific memes
across datasets. Heat maps encode the density of all $n$-grams with $M\neq 0$
($N$ being the number of such $n$-grams) with respect to their propagation
score and frequency. Maps A, C and D each show a broad band with a downward
slope for the datasets from the American Physical Society (APS), the open
access subset of PubMed Central, and the Web of Science, respectively. The
99.9%-quantile with respect to the meme score distribution ($M_{0.999}$) is
depicted as a white line. Memes are located mostly around the very edge of the
top-right side of the band (in the vicinity of the 99.9%-quantile line). Heat
map B shows the results obtained with a time-preserving randomization of the
APS citation graph (see Methods for details).

Calculating the meme score for all $n$-grams in the three datasets considered
gives us the results presented in Fig. 2. Their relatively frequency and their
propagation score are plotted against each other in the form of heat maps with
logarithmic scales. There is no upper limit to the length of $n$-grams, and
the presented maps cover without exception all $n$-grams with a non-zero meme
score ($N$ being the number of such $n$-grams). Meme scores are increasing
towards the top-right and decreasing towards the bottom-left corner. Maps A, C
and D feature a broad band with a downward slope, indicating that, in general,
more frequent memes tend to propagate less via the citation graph. In the
lower half of each map, we see a wedge of very high densities that follows the
larger band on the bottom-left edge, but getting narrower towards the middle
where it ends. Though this wedge has a somewhat rounder and broader shape for
the Web of Science database, overall these patterns look remarkably similar
across all datasets despite their differences with respect to topic, coverage,
and size. This is an indication of universality in the distribution patterns
of scientific memes. The 99.9%-quantile line ($M_{0.999}$) is also
surprisingly stable, considering that the underlying values range over five
orders of magnitude or more. Localizing the previously mentioned physics memes
in the APS dataset (map A), we see that they are located on the very edge of
the top-right side of the band, where the density of $n$-grams is very low.
(Very frequent words like “of” or “the” are found in the faint spike at the
top of the plot where $P\approx 1$ and the frequency is close to 100%.)

The heat map B in Fig. 2 illustrates a typical case of what happens when the
APS citation graph is randomized but the time ordering of publications is
preserved. The number of terms with a non-zero meme score decreases
dramatically (from $\approx\,$1.4 million in map A to just 89,356 in map B),
the universal distribution pattern of scientific memes vanishes, and the top-
right part, where the top-ranked memes should be located, disappears
completely. Naturally, if the APS citation graph is randomized without
preserving the time ordering, the overlap with the original results presented
in map A is even smaller (see Supplementary Material). Statistical analysis
reveals that median values of the meme score obtained with the randomized
networks differ by more than one order of magnitude from those obtained with
the original citation graph, with very little variation between different
randomization runs. These results show that topology and time structure alone
fail to account for the reported universality in the distribution patterns,
and that thus the top memes get their high meme scores based on intricate
processes and conventions that underlie the dynamics of scientific progress
and the way credit is given to previous work.

Table 1 shows the $50$ top-ranked memes from the APS dataset, also indicating
their agreement with human annotation and whether they can be found under a
subcategory of physics in Wikipedia. Most of these memes are noun phrases
denoting real and reasonable physics concepts, which is remarkable given that
the computation of the simple meme score formula uses no linguistic or
ontological knowledge whatsoever. The dominance of noun phrases is consistent
with the finding that (scientific) concepts are typically captured by noun
phrases when represented as keywords in terminologies 32, 33. The extracted
memes consist of one, two or three words, which indicates that the meme score
does not favor short or long phrases, again without applying explicit measures
to balance $n$-gram lengths. A further observation is that chemical formulas
such as MgB2 and CuGeO3 are relatively frequent, which we investigate in more
detail below.

1. |  loop quantum cosmology+* | 14. | strange nonchaotic | 27. |  NaxCoO2+ | 38. | inspiral*
---|---|---|---|---|---|---|---
2. |  unparticle+* | 15. |  in NbSe3 | 28. |  the unparticle+ | 39. |  spin Hall effect+*
3. |  sonoluminescence+* | 16. |  spin Hall+ | 29. | black | 40. | PAMELA
4. |  MgB2+ | 17. |  elliptic flow+* | 30. | electromagnetically induced | 41. |  BaFe2As2+
5. |  stochastic resonance+* | 18. |  quantum Hall+* |  |  transparency+* | 42. |  quantum dots+*
6. |  carbon nanotubes+* | 19. |  CeCoIn5+ | 31. |  light-induced drift+ | 43. |  Bose-Einstein condensates+
7. |  NbSe3+ | 20. |  inflation+ | 32. |  proton-proton bremsstrahlung+ | 44. | X(3872)*
8. |  black hole+* | 21. |  exchange bias+* | 33. | antisymmetrized molecular | 45. |  relaxor+
9. |  nanotubes+ | 22. |  Sr2RuO4+ |  |  dynamics+ | 46. |  blue phases+
10. |  lattice Boltzmann+* | 23. |  traffic flow+* | 34. |  radiative muon capture+ | 47. |  black holes+*
11. |  dark energy+* | 24. | TiOCl | 35. |  Bose-Einstein+ | 48. |  PrOs4Sb12+
12. | Rashba | 25. |  key distribution+ | 36. |  C60+ | 49. |  the Schwinger multichannel method+
13. |  CuGeO3+ | 26. |  graphene+* | 37. |  entanglement+ | 50. |  Higgsless+
Table 1:  Top 50 memes with respect to their meme score from the APS dataset. The symbol + indicates memes where the human annotators agreed that this is an interesting and important physics concept, while the symbol * indicates memes that are also found on the list of memes extracted from Wikipedia (see Methods for details). method | main class |  annotator | (annotator | classified as main class: |  $p$-value for difference to:
---|---|---|---|---|---
|  |  | agreement) | _individually_ | _in agreement_ | _random_ | _weighted random_
| physics concept | A1 | (90.0%) | 85.3% | 81.3% | ${<}10^{-15}$* | ${<}10^{-15}$*
meme score | A2 | 87.3%
noun phrase | A1 | (93.3%) | 86.0% | 82.7% | ${<}10^{-15}$* | ${<}10^{-15}$*
| A2 | 86.0%
| physics concept | A1 | (85.3%) | 32.7% | 25.3% | – | 0.123
random | A2 | 32.7%
noun phrase | A1 | (86.0%) | 39.3% | 33.3% | – | 0.163
| A2 | 36.0%
| physics concept | A1 | (90.7%) | 20.7% | 19.3% | 0.123 | –
weighted random | A2 | 27.3%
noun phrase | A1 | (86.0%) | 28.7% | 25.3% | 0.163 | –
| A2 | 28.7%
Table 2: The two annotators (A1 and A2) classified more than 80% of the memes
with the highest meme scores as relevant physics concepts and noun phrases.
The differences involving the meme score are highly significant (*). See
Methods and Supplementary Material for details.

In Table 2, we present results of a manual annotation of terms identified by
meme score as compared to randomly selected terms (see also Supplementary
Material). Each of the annotators considered around 86% of the meme score
terms to be important physics concepts, agreeing on this in 81% of the cases.
With respect to their linguistic categories, each annotator considered 86% of
the meme score terms to be noun phrases, and the two annotators agreed on that
for 83% of the terms. The respective values are much lower for the randomly
extracted terms. Only 25% (non-weighted) and 19% (weighted) of terms were, in
agreement, found to be important physics concepts, and only 33% (non-weighted)
and 25% (weighted) to be noun phrases. The reported differences between the
meme score and the two random selection methods are highly significant
($p<10^{-15}$ using Fisher’s exact test on the number of agreed
classifications). These results confirm that the meme score strongly favors
noun phrases and important concepts.

Figure 3:  The meme score outperforms alternative metrics. The box plot shows
the agreement with the ground-truth list of physics terms extracted from
Wikipedia as achieved by the different metrics. Agreement is measured as the
area under the curve $A$, as shown for the meme score in the embedded graph on
the top right. The curves are defined as the percentage from the $x$ top-
ranked terms that also appear on the ground-truth list for the different
parameter settings ($1\leq\delta\leq 10$ for the meme score; the thick line
stands for $\delta=4$, which has the largest area $A$; see Methods for
details).

Next we compare the meme score to a number of possible alternative metrics, as
defined in the Methods section, and align the identified words and phrases
with a ground-truth list of terms extracted from physics-related Wikipedia
titles. Fig. 3 summarizes the results, showing on the top right that
$\approx\,$70% of the top $10$ memes identified by meme score correspond to
terms extracted from Wikipedia, $\approx\,$55% of the top $20$, $\approx\,$40%
of the top $50$, and $\approx\,$26% of the top $100$. The largest area under
the curve $A$ is obtained for a controlled noise level $\delta=4$, which is
highlighted by the thick blue line. The box plot compares the outcomes of
different metrics with respect to $A$, as described in the Methods section.
The isolated outlier at 32.3% on the lower end of the distribution for the
meme score originates from the parameter setting $\delta=1$. All other
parameter settings $2\leq\delta\leq 10$ lead to results in a narrow band
between 40.9% and 44.8% (meaning that the performance of the metric is not
sensitive to movements within this range of the parameter space). In contrast,
all alternative metrics score considerably worse, consistently below 22%
(including outliers).

Figure 4:  Plot A shows that phrases with a high meme score (i.e. around the
99.9%-quantile $M_{0.999}$) tend to be found as titles of Wikipedia articles
on physics, while other phrases tend not to. Plot B shows that phrases of
relatively low frequency but high meme score have the highest density of terms
containing chemical formulas (e.g. “MgB2”). The data points of the two plots
are the same as in plot A of Fig. 2, but colors showing the relative density
of the specific type of terms. Note that the colors are log scaled: white
stands for 100% terms of the given type, yellow for $\approx\,$30%, orange for
$\approx\,$10%, light red for $\approx\,$2%, dark red for $<\,$1%, and black
for $<\,$0.1%.

The two plots of Fig. 4 show the same data points as plot A of Fig. 2 but with
colors standing for the relative density of Wikipedia terms (A) and of terms
containing chemical formulas (B). Plot A confirms that phrases in the area of
a high meme score (towards the top right) tend to show up as titles of
Wikipedia articles on physics. Additionally, the plot shows that this is the
_only_ such area. There are a few scattered outliers, but the only significant
area with a high density of Wikipedia terms is found around the
99.9%-quantile. Plot B shows that phrases containing chemical formulas (such
as “BaFe2As2”) tend to have a relatively low frequency (individually) but high
propagation score. The area with the highest density can again be found along
the 99.9%-quantile, which is consistent with the expectation of chemical
compounds to be important and interesting entities for physics research. The
fact that they are standardized and compressed representations reduces
moreover their “vulnerability” to synonyms or spelling variants, making them
stronger memes on the level of pure character sequences. (We come back to the
issue of memes on different levels of abstraction below.)

Figure 5:  Time history of top physics memes based on their meme scores
obtained from the American Physical Society dataset. The time axis is scaled
by publication count. Bars and labels are shown for all memes that top the
rankings for at least ten out of the displayed $911$ points in time. The gray
area represents the second-ranked meme at a given time.

Fig. 5 shows the top memes over time, revealing bursty dynamics, akin to the
one reported previously in humans dynamics 34 and the temporal distribution of
words 35. These bursts might be a reflection of the fast rise and fall of many
scientific memes in terms of their popularity. As new scientific paradigms
emerge, the old ones seem to quickly lose their appeal, and only a few memes
manage to top the rankings over extended periods of time. The bursty dynamics
also support the idea that both the rise and fall of scientific paradigms is
driven by robust principles of self-organization 36.

## Discussion

By going back to the original analogy with genes put forward by Richard
Dawkins 20, we investigated the relation between the occurrence frequency of
scientific memes and the degree to which they propagate along the citation
graph. We found that scientific memes are indeed governed by a surprisingly
simple relationship of these two factors. This is formalized by the meme score
— a metric to characterize and identify scientific memes — defined as the
product of the frequency of occurrence and the propagation score.

We have shown that the meme score can be calculated exactly and exhaustively
without the introduction of arbitrary thresholds or filters and without
relying on any kind of linguistic or ontological knowledge. The method is fast
and reliable, and it can be applied to massive databases. We have demonstrated
the effectiveness of the meme score on more than 47 million publication
records from the Web of Science, PubMed Central, and the American Physical
Society. Moreover, we have evaluated the accuracy of the proposed meme score
by means of full and time-preserving randomizations of the citation graphs, by
means of manual annotation of publications, as well as by means of several
alternative metrics. We have provided statistical evidence for the agreement
between human annotators and the meme-score results, and we have shown that it
is superior to alternative metrics. We have also confirmed that the observed
patterns cannot be explained by topological or temporal features alone, but
are grounded in more intricate processes that determine the dynamics of the
scientific progress and the way credit is given to preceding publications.
Furthermore, the top-ranking scientific memes reveal a bursty time dynamics,
which might be a reflection of the fierce competition among memes for the
limited and fluctuating resource of scientists’ attention.

We have only considered fixed character sequences as potential memes, but it
is clear that memes do not only exist on this low level, and it is reasonable
to expect that the inclusion of additional layers of processing using
linguistic and ontological resources would lead to even better results and
would let us capture memes on a more abstract level. Such memes might consist
of sets of morphological variants, co-occurrences of words, compositions of
multiple memes, grammatical constructions, or even argumentation schemes and
rhetorical styles. We deliberately kept the meme score as simple as possible
to emphasize that it is surprisingly precise on its own. At the same time,
there are many ways to improve the metric in the future with more
sophisticated processing to capture memes on a higher level. In general, we
believe that the presented approach by allowing to study memes in a
comprehensive manner opens up the field for a wide range of future research on
topics such as information diffusion, complex systems, innovation, scientific
progress, social dynamics, ecosystems, cultural evolution, and of course the
study of memes themselves.

## Methods

### Graph randomization

The analyzed randomized networks have exactly the same topology as the
original ones but the article texts (i.e. titles and abstracts with their
memes) are randomly assigned to the nodes. Each node therefore owes its
position in the network to one particular publication but has text attached
that comes from a different one. For the time-preserving randomizations, we
shuffle only publications that were published within narrow consecutive time
windows. Concretely we use time windows of 1000 publications, meaning that —
after shuffling — no publication has moved more than 1000 positions forward or
backward from the original chronological order.

### Human annotation

For the first part of the manual annotation, we use the following two
categories: (i) the phrase is not a meaningful term or not an important
concept of physics, and (ii) the phrase is an important concept or entity of
physics — it could appear as the title of an entry of a comprehensive
encyclopedia of physics. For the second part, we defined the following
linguistic phrase types: (i) noun phrase, (ii) verb, (iii) adjective or
adverb, and (iv) other.

The set of phrases used for this evaluation consisted of the top 150 memes
with respect to their meme score, extracted from the American Physical Society
dataset, plus another two sets for comparison of 150 randomly drawn phrases
each. For the two comparison sets, we considered all phrases that appear in at
least 100 publications. From these, 150 terms were drawn randomly without
taking into account their frequency, i.e., frequent terms had the same chance
of being selected as infrequent ones, whereas the 150 terms of the second set
were drawn with a weight that corresponded to their frequency. Moreover, to
rule out effects of different $n$-gram lengths, we made sure that the two
batches of random terms followed exactly the same length distribution as the
main sample extracted based on the meme score. The resulting 450 terms were
shuffled and given to two human annotators, both PhD students with a degree in
physics, who independently annotated the terms.

### Metrics for comparison

We have used the following metrics with different parameter settings as
alternatives to the meme score (see Supplementary Material for details): (i)
frequency — the most frequent terms, optionally skipping the first $x$ terms;
(ii) maximum absolute change over time — the highest-scoring terms with
respect to maximum _absolute_ change in frequency; (iii) maximum relative
change over time — the same as (ii) but based on _relative_ changes; (iv)
maximum absolute difference across journals — the highest-scoring terms with
respect to maximum _absolute_ difference in frequency between journals; (v)
maximum relative difference across journals — the same as (iv) but based on
_relative_ changes. Metric (i) is based on the assumption that important memes
are frequent but not as frequent as the small class of general words that can
be found in all types of texts. Metrics (ii) and (iii) are based on an idea
proposed in 36, being that interesting memes exhibit trends over time. Metrics
(iv) and (v) are based on the intuition that phrases occurring mostly in
specific journals but not in others must be specific concepts of the
particular field of research.

As a ground-truth list of memes, we automatically extracted $5178$ terms from
Wikipedia. We collected the titles of all articles — and terms redirecting to
them — from the categories “physics”, “applied and interdisciplinary physics”,
“theoretical physics”, “emerging technologies”, and their direct sub-
categories, but filtering out terms that appear in less than $10$ publications
of the American Physical Society dataset. To quantify the agreement between
the top memes identified by a particular metric and the Wikipedia list, we use
the normalized area $A$ under the curve as shown on Fig. 3. The step-shaped
curved has a log-scaled $x$-axis running up to the number of terms $s$ on the
ground-truth meme list ($s=5178$ in our case) and an $y$-axis running from $0$
(no overlap) to $1$ (perfect overlap). Limiting cases are $A=1$, representing
perfect agreement, and $A=0$, representing no agreement at all between the two
compared lists.

###### Acknowledgements.

This research was supported by the European Commission through the ERC
Advanced Investigator Grant “Momentum” (Grant No. 324247) and by the Slovenian
Research Agency through the Program P5-0027. In addition, we would like to
thank Karsten Donnay, Matthias Leiss, Christian Schulz, and Olivia Woolley-
Meza for their useful feedback and help with the realization of the
evaluations.

## References

  * 1 Redner, S.  How popular is your paper? an empirical study of the citation distribution.  Eur. Phys. J. B 4, 131–134 (1998).
  * 2 Radicchi, F., Fortunato, S., and Castellano, C.  Universality of citation distributions: Toward an objective measure of scientific impact.  Proc. Natl. Acad. Sci. USA 105, 17268–17272 (2008).
  * 3 Newman, M. E. J.  Coauthorship networks and patterns of scientific collaboration.  Proc. Natl. Acad. Sci. USA 101, 5200–5205 (2004).
  * 4 Guimerà, R., Uzzi, B., Spiro, J., and Amaral, L. A. N.  Team assembly mechanisms determine collaboration network structure and team performance.  Science 308, 697–702 (2005).
  * 5 Milojević, S.  Principles of scientific research team formation and evolution.  Proc. Natl. Acad. Sci. USA 111, 3984–3989 (2014).
  * 6 Hirsch, J. E.  An index to quantify an individual’s scientific research output.  Proc. Natl. Acad. Sci. USA 104, 16569 (2005).
  * 7 Radicchi, F., Fortunato, S., Markines, B., and Vespignani, A.  Diffusion of scientific credits and the ranking of scientists.  Phys. Rev. E 80, 056103 (2009).
  * 8 Petersen, A. M., Wang, F., and Stanley, H. E.  Methods for measuring the citations and productivity of scientists across time and discipline.  Phys. Rev. E 81, 036114 (2010).
  * 9 Wang, D., Song, C., and Barabási, A.-L.  Quantifying long-term scientific impact.  Science 342(6154), 127–132 (2013).
  * 10 Penner, O., Pan, R. K., Petersen, A. M., Kaski, K., and Fortunato, S.  On the predictability of future impact in science.  Sci. Rep. 3, 3052 (2013).
  * 11 de Solla Price, D. J.  Networks of scientific papers.  Science 149, 510–515 (1965).
  * 12 Merton, R. K.  The matthew effect in science.  Science 159, 53–63 (1968).
  * 13 Evans, J. A. and Foster, J. G.  Metaknowledge.  Science 331, 721–725 (2011).
  * 14 Michel, J. B., Shen, Y. K., Presser Aiden, A., Veres, A., Gray, M. K., The Google Books Team, Pickett, J. P., Hoiberg, D., Clancy, D., Norvig, P., Orwant, J., Pinker, S., Nowak, M. A., and Lieberman Aiden, E.  Quantitative analysis of culture using millions of digitized books.  Science 331, 176–182 (2011).
  * 15 Pan, R. K., Kaski, K., and Fortunato, S.  World citation and collaboration networks: uncovering the role of geography in science.  Sci. Rep. 2, 902 (2012).
  * 16 Mazloumian, A., Helbing, D., Lozano, S., Light, R. P., and Börner, K.  Global multi-level analysis of the ‘scientific food web’.  Sci. Rep. 3, 1167 (2013).
  * 17 Chavalarias, D. and Cointet, J.-P.  Phylomemetic patterns in science evolution — the rise and fall of scientific fields.  PLoS ONE 8, e54847 (2013).
  * 18 Börner, K.  Atlas of Science.  MIT Press, Cambridge, MA, (2010).
  * 19 Börner, K.  Atlas of Knowledge.  MIT Press, Cambridge, MA, (2014).
  * 20 Dawkins, R.  The Selfish Gene.  Oxford University Press, Oxford, (1989).
  * 21 Leskovec, J., Backstrom, L., and Kleinberg, J.  Meme-tracking and the dynamics of the news cycle.  In Proceedings of ACM SIGKDD, 497–506, (2009).
  * 22 Adar, E. and Adamic, L. A.  Tracking information epidemics in blogspace.  In Proceedings of IEEE/WIC/ACM, 207–214, (2005).
  * 23 Conover, M., Ratkiewicz, J., Francisco, M., Gonçalves, B., Menczer, F., and Flammini, A.  Political polarization on Twitter.  In Proceedings of ICWSM, 89–96, (2011).
  * 24 Gomez Rodriguez, M., Leskovec, J., and Krause, A.  Inferring networks of diffusion and influence.  In Proceedings of ACM SIGKDD, 1019–1028, (2010).
  * 25 Simmons, M. P., Adamic, L. A., and Adar, E.  Memes online: Extracted, subtracted, injected, and recollected.  In Proceedings of ICWSM, 353–360, (2011).
  * 26 Weng, L., Flammini, A., Vespignani, A., and Menczer, F.  Competition among memes in a world with limited attention.  Scientific Reports 2 (2012).
  * 27 Christakis, N. A. and Fowler, J. H.  Social contagion theory: examining dynamic social networks and human behavior.  Statistics in Medicine 32(4), 556–577 (2013).
  * 28 Stanley, H. E.  Introduction to Phase Transitions and Critical Phenomena.  Clarendon Press, Oxford, (1971).
  * 29 Gleeson, J. P., Ward, J. A., O Sullivan, K. P., and Lee, W. T.  Competition-induced criticality in a model of meme popularity.  Phys. Rev. Lett. 112, 048701 (2014).
  * 30 Bastian, M., Heymann, S., and Jacomy, M.  Gephi: an open source software for exploring and manipulating networks.  In ICWSM, 361–362, (2009).
  * 31 Martin, S., Brown, W. M., Klavans, R., and Boyack, K. W.  OpenOrd: an open-source toolbox for large graph layout.  In IS&T/SPIE Electronic Imaging. International Society for Optics and Photonics, (2011).
  * 32 Bourigault, D., Gonzalez-Mullier, I., and Gros, C.  Lexter, a natural language processing tool for terminology extraction.  In Proceedings of the 7th EURALEX International Congress, 771–779, (1996).
  * 33 Hulth, A.  Improved automatic keyword extraction given more linguistic knowledge.  In Proceedings of the 2003 Conference on Empirical Methods in Natural Language Processing, 216–223. Association for Computational Linguistics, (2003).
  * 34 Barabási, A. L.  The origin of bursts and heavy tails in humans dynamics.  Nature 435, 207–211 (2005).
  * 35 Altmann, E. G., Pierrehumbert, J. B., and Motter, A. E.  Beyond word frequency: bursts, lulls, and scaling in the temporal distributions of words.  PLoS ONE 4, e7678 (2009).
  * 36 Perc, M.  Self-organization of progress across the century of physics.  Sci. Rep. 3, 1720 (2013).