Version 4 as of

Primary authors: MIT, NTU, INFN Bari

To be submitted to PRL

# Long-range near-side correlation in $e^{+}e^{-}$ Collisions at 183-209 GeV
with ALEPH Archived Data

Yu-Chen Chen  Massachusetts Institute of Technology, Cambridge, Massachusetts,
USA  Yi Chen  Massachusetts Institute of Technology, Cambridge, Massachusetts,
USA  Anthony Badea  University of Chicago, Chicago, Illinois, USA  Austin Baty
University Illinois Chicago, Illinois, USA  Gian Michele Innocenti  CERN,
Geneva, Switzerland  Marcello Maggi  INFN Sezione di Bari, Bari, Italy
Christopher McGinn  Massachusetts Institute of Technology, Cambridge,
Massachusetts, USA  Michael Peters  Massachusetts Institute of Technology,
Cambridge, Massachusetts, USA  Tzu-An Sheng  Massachusetts Institute of
Technology, Cambridge, Massachusetts, USA  Jesse Thaler  Massachusetts
Institute of Technology, Cambridge, Massachusetts, USA  Yen-Jie Lee
<EMAIL_ADDRESS>Massachusetts Institute of Technology, Cambridge,
Massachusetts, USA

###### Abstract

The first measurement of two-particle angular correlations for charged
particles produced in $e^{+}e^{-}$ annihilation up to $\sqrt{s}=209$ GeV with
LEP-II data is presented. Hadronic $e^{+}e^{-}$ data, archived at center-of-
mass energies ranging from 183 to 209 GeV, were collected using the ALEPH
detector at LEP. The angular correlation functions have been measured across a
wide range of pseudorapidities and the full azimuth in bins of charged
particle multiplicity. Results for $e^{+}e^{-}$ data at high energies, which
allow for higher event multiplicities reaching approximately 50 than LEP-I at
Z pole energy, are presented for the first time. A long-range near-side excess
in the correlation function has been identified in the analysis when
calculating particle kinematic variables with respect to the thrust axis.
Moreover, the two-particle correlation functions were decomposed using a
Fourier series, and the resulting Fourier coefficients $v_{n}$ were compared
with event generator outputs. In events with high multiplicity featuring more
than 50 particles, the extracted $v_{2}$ magnitude from the data are compared
to those from the Monte Carlo reference.

In heavy-ion collision experiments, two-particle angular correlations Adams et
al. (2005); Abelev et al. (2009); Alver et al. (2010); Chatrchyan et al.
(2012); Aamodt et al. (2012); Adam et al. (2019) are extracted for studying
the Quark-Gluon Plasma (QGP) Busza et al. (2018). In these measurements, a
long-range angular correlation, known as the ridge Abelev et al. (2009); Alver
et al. (2010), has been observed in various collision systems and at different
collision energies. Since the beginning of LHC operations, this ridge
structure has also been observed in high-multiplicity proton-proton (pp)
collisions by the CMS collaboration Khachatryan et al. (2010) and confirmed by
experiments at the LHC and RHIC using smaller collision systems than ion-ion
collisions, such as proton-proton Aad et al. (2016), proton-ion (pA)
Chatrchyan et al. (2013); Abelev et al. (2013a, b); Aad et al. (2013); Aaij et
al. (2016), and deuteron-ion Adare et al. (2013) collisions. In heavy-ion
collisions, the ridge structure is associated with the fluctuating initial
state of the ions Ollitrault (1992); Alver and Roland (2010). However, the
physical origin of the ridge structure in small systems remains under debate
Dumitru et al. (2011); Dusling and Venugopalan (2013); Bozek (2012); He et al.
(2016); Nagle and Zajc (2018). The potential correlations in the initial state
partons arising from hadronic structure make understanding pp and pA
measurements challenging. Numerous theoretical models exist to explain these
systems with high particle densities. These models incorporate various
mechanisms, from initial state correlations as suggested in Dusling and
Venugopalan (2013), through final-state interactions He et al. (2016), to
hydrodynamic effects Bozek (2012).

Lately, the focus has intensified on assessing two-particle correlations in
even smaller systems than pp and pA collisions. This includes systems like
photonuclear collisions with ultra-peripheral proton-lead and lead-lead data
as demonstrated by ATLAS and CMS Aad et al. (2021); Tumasyan et al. (2023),
electron-proton collisions reported by ZEUS Abt et al. (2020), and
$e^{+}e^{-}$ Badea et al. (2019); Chen et al. (2022, 2023a). Such studies are
invaluable complements to those done on larger collision systems, shedding
light on the bare minimum conditions required for collective behavior Nagle et
al. (2018). Electron beams, in particular, are free from issues like multiple
parton interactions and initial state correlations. Notably, no significant
ridge-like patterns have been detected in the electron-positron annihilations,
giving further clarity to the emergence of the collectivity signal as
discussed in various studies Bierlich and Rasmussen (2019); Bierlich et al.
(2021); Castorina et al. (2021); Agostini et al. (2021); Larkoski and Melia
(2021); Baty et al. (2021).

There are two potential approaches to making progress in detecting a possible
ridge-like signal. The first approach involves increasing the final state
multiplicity of the system. This is because the probability of parton-parton
scattering increases with rising parton density, which results in larger final
state multiplicity. Additionally, insights gained from pp and photonuclear
collisions suggest that a larger multiplicity decreases the magnitude of the
negative direct flow ($v_{1}$) due to momentum conservation. A diminished
$v_{1}$ could facilitate the detection of the possible ridge-like signal. The
second approach involves exploring different physics processes. As recommended
in Ref. Nagle et al. (2018), a two-string configuration simulated in AMPT
strengthens the ridge-like signal compared to a single-string configuration.
Investigating data with a two-string configuration could increase the chances
of detecting a ridge-like signal in the most elementary collisions.

This study utilizes archived data collected by the ALEPH detector at LEP-II
Decamp et al. (1990) between 1996 and 2000. To analyze these data, an MIT Open
Data format was created Tripathee et al. (2017). Unlike the 91.2 GeV sample at
LEP-I, which is dominated by $Z$-decays, the high-energy sample sees
significant contributions from various processes beyond $e^{+}e^{-}\to
q\bar{q}$ fragmentation, including a notable “radiative-return-to-$Z$” effect
due to initial-state QED radiation. Adopting the selection criteria from the
ALEPH collaboration Heister et al. (2004), we cluster the event into two jets
to determine the effective center-of-mass energy ($\sqrt{s^{\prime}}$) using
the equation

$s^{\prime}=\frac{\sin\theta_{1}+\sin\theta_{2}-|\sin(\theta_{1}+\theta_{2})|}{\sin\theta_{1}+\sin\theta_{2}+|\sin(\theta_{1}+\theta_{2})|}\times
s,$ (1)

where $\theta_{1,2}$ are the angles of these jets to the beam direction. Using
this, the visible two-jet invariant mass ($M_{\rm vis}$) is derived, aiding in
minimizing the QED radiation background. In our analysis, $\sqrt{s^{\prime}}$
must exceed $0.9\sqrt{s}$, and $M_{\rm vis}$ must surpass $0.7\sqrt{s}$.
Furthermore, adhering to the hadronic event criteria from previous LEP-I work
Badea et al. (2019), events are selected based on the event sphericity axis’s
polar angle ($7\pi/36<\theta_{\rm lab}<29\pi/36$), and those with under five
tracks or with total reconstructed charged-particle energy below $15$ GeV are
discarded.

High-quality tracks from particles are selected using requirements identical
to those in previous ALEPH analyses Barate et al. (1998). They are also
required to have a transverse momentum with respect to the beam axis ($p_{\rm
T}^{\rm lab}$) above 0.2 GeV/c and $|\cos{\theta_{\text{lab}}}|<0.94$ in the
lab frame. Secondary charged particles from neutral particle decays are
suppressed by $V^{0}$ reconstruction in the energy flow algorithm Barate et
al. (1998). We employed the Monte Carlo (MC) events from the ALEPH
collaboration for reconstruction effects and data correction. Specifically, we
relied on archived pythia 6.1 Sjostrand et al. (2001) MC simulation samples
produced by ALEPH detector conditions at LEP-II. These samples informed our
tracking efficiency and event selection corrections. The various MC
subprocesses were weighted based on cross-sections from event generators.

The analysis procedure aligns with prior two-particle correlation function
studies Chatrchyan et al. (2013); Badea et al. (2019). For each event, the
efficiency-corrected differential yield of charged-particle pairs, denoted as
$\frac{\rm d^{2}N^{\rm same}}{\rm d\Delta\eta\rm d\Delta\phi}$ (where “same”
means particles from the same event), is computed. It is then normalized by
the average corrected number of charged particles in the event, $\rm
N_{\text{trk}}^{corr}$, yielding:

$\displaystyle S(\Delta\eta,\Delta\phi)$ $\displaystyle=\frac{1}{\rm N_{\rm
trk}^{corr}}\frac{\rm d^{2}N^{\rm same}}{\rm d\Delta\eta\rm d\Delta\phi}.$ (2)

Figure 1: Two-particle correlation functions for events with the number of
charged particle tracks in hadronic $e^{+}e^{-}$ in the thrust coordinate
analysis with $\rm N_{\mathrm{trk}}\geq$ 5 (left) and $\rm
N_{\mathrm{trk}}\geq$ 50 (right). The sharp near-side peaks arise from jet
correlations and have been truncated to illustrate the structure outside that
region better.

A mixed-event background correlation, $B(\Delta\eta,\Delta\phi)$, pairs
charged particles from one event with those from 48 random events of the same
multiplicity, giving

$\displaystyle B(\Delta\eta,\Delta\phi)$ $\displaystyle=\frac{1}{\rm N_{\rm
trk}^{corr}}\frac{\rm d^{2}N^{\rm mix}}{\rm d\Delta\eta\rm d\Delta\phi}.$ (3)

Here, ${\rm N^{\rm mix}}$ is the efficiency-corrected pair count from the
mixed event. By dividing this by $B(0,0)$, computed using pairs with
$|\Delta\eta|<0.32$ and $|\Delta\phi|<\pi/20$, we obtain the detector’s pair
acceptance for uncorrelated particles. Hence, the acceptance-corrected pair
yield is:

$\displaystyle\frac{1}{\rm N_{\rm trk}^{corr}}\frac{\rm
d^{2}N^{pair}}{d\Delta\eta\rm d\Delta\phi}$
$\displaystyle=B(0,0)\times\frac{S(\Delta\eta,\Delta\phi)}{B(\Delta\eta,\Delta\phi)}.$
(4)

For multiplicity-dependent analysis, events are grouped into five intervals
based on reconstructed charged track count, $\rm N_{\text{trk}}$, with $p_{\rm
T}^{\rm lab}>0.2$ GeV/c. Details, including multiplicity ranges and average
track counts before and after correction, are in Table 1.

$\rm N_{\text{trk}}$ range | Fraction of data (%) | $\langle{\rm N_{\text{trk}}}\rangle$ | $\langle{\rm N_{\text{trk}}^{\text{corr}}}\rangle$
---|---|---|---
$\left[\mathrm{10,20}\right)$ | 58.6 | 15.2 | 17.3
$\left[\mathrm{20,30}\right)$ | 33.1 | 23.1 | 25.7
$\left[\mathrm{30,40}\right)$ | 3.7 | 32.6 | 35.9
$\left[\mathrm{40,50}\right)$ | 0.4 | 42.8 | 47.1
$\left[\mathrm{50,\infty}\right)$ | $<0.1$ | 53.0 | 58.4
Table 1: Fraction of the full event sample for each multiplicity class. The
last two columns show the observed and corrected multiplicities, respectively,
of charged particles with $p_{\rm T}^{\rm lab}>$ 0.2 GeV/c and
$|\cos{\theta_{\text{lab}}}|<0.94$.

Experimentally, the thrust axis Farhi (1977), closely related to the outgoing
$q\bar{q}$ direction, establishes the coordinate system for thrust coordinate
analysis. The thrust axis serves as the reference to address the outgoing-
state energy flow orientation in $e^{+}e^{-}$ collisions. Including an extra
particle representing the event’s unreconstructed momentum in the thrust axis
calculation mitigates the effect of detector inefficiencies on the correlation
function. All tracks meeting quality criteria then have their kinematic
variables ($p_{\text{T}}$, $\eta$, $\phi$) recalculated, with the thrust axis
substituting the beam axis, using the prescription of the LEP-I analysis Badea
et al. (2019). Kinematics are recalculated for particles in paired events
relative to the signal event’s thrust axis for the background correlation
calculation. The $\eta$ and $\phi$ distributions of charged tracks in these
paired events are reweighted to align with the distributions of the signal
events. This strategy accounts for the random pairing effect under the
detector acceptance in the thrust coordinate for different signal events.

In hadronic collision systems, the azimuthal anisotropy of charged particle
production is typically quantified with the azimuthal anisotropy coefficients
($v_{n}$), $v_{n}$ Voloshin and Zhang (1996); Poskanzer and Voloshin (1998);
Alver and Roland (2010). In particular, the second order coefficient, $v_{2}$
is sensitive to the collective behavior and the level of thermalization of the
system in relativistic heavy ion collisions Ollitrault (1992); Ackermann et
al. (2001). However, it is often difficult to make a direct quantitative
connection between the size of any associated yields and the corresponding
value of $v_{2}$ because most of the structure of the correlation functions
comes from jetlike correlations. These correlations are sometimes referred to
as “nonflow” Adler et al. (2003); Adare et al. (2008); Aamodt et al. (2010);
Sirunyan et al. (2018).

We employ the Fourier decomposition analysis used in prior studies to
investigate potential flow-like signatures. This helps us understand
anisotropy harmonics through two-particle azimuthal correlations. The non-flow
effects diminish significantly at large $|\Delta\eta|$. The long-range
azimuthal differential yields can be described by:

$Y_{l}(\Delta\phi)=\frac{1}{{\rm N}_{\rm trk}^{\rm corr}}\frac{d{\rm N}^{\rm
pair}}{d\Delta\phi}=\frac{{\rm N}^{\rm
assoc}}{2\pi}\left(1+\sum_{n=1}^{\infty}2V_{n\Delta}\cos(n\Delta\phi)\right),$

with ${\rm N}^{\rm assoc}$ representing associated track pairs in specified
$|\Delta\eta|$ and $\Delta\phi$ ranges. The long-range associated yield is a
histogram, and the Discrete Fourier Transform is used to determine Fourier
coefficients ($V_{n\Delta}$) and normalization (${\rm N}^{\rm assoc}$). These
coefficients relate to single-particle Fourier harmonics, assuming they
originate from hydrodynamic flow effects. In our approach, the trigger and
associated particles are in the same $p_{T}$ bin, leading to $v_{n}=v_{n}^{\rm
trig}=v_{n}^{\rm assoc}$.

This analysis uses Bayesian inference to assess the statistical uncertainties
for the observables of interest: correlation yields and flow coefficients
$v_{n}$. The primary rationale behind adopting the Bayesian analysis is to
offer a more detailed estimation of uncertainties, particularly when assuming
a Gaussian distribution is not ideal for a data set with a non-Gaussian
distribution. With Bayes’ theorem, we obtain the posterior probability for an
observable of interest, using a flat prior and a “weighted Poisson
distribution Bohm and Zech (2014)” as the likelihood function. Reported
central values and uncertainties for pairing yields and flow coefficients are
based on the “maximum a posteriori (MAP)” method. The comprehensive Bayesian
calculation has been documented in the note Chen et al. (2023b).

Systematic uncertainties for the long-range associated yield
$Y_{l}(\Delta\phi)$ and $v_{n}$ arise from event and track selections, the
$B(0,0)$ normalization factor, and residual MC corrections. For event
selections, variations involve altering the ISR requirements Heister et al.
(2004) on the visible two-jet invariant mass $M_{\rm vis}$ from $0.7\sqrt{s}$
to $0.65\sqrt{s}$ and adjusting the effective center-of-mass energy
$\sqrt{s^{\prime}}$ from $0.9\sqrt{s}$ to $0.87\sqrt{s}$. Meanwhile,
consistent with the LEP-I analysis approach, the hadronic event selection
criteria adjust the number of particles from 13 to 10 and the reconstructed
charged-particle energy from 15 GeV to 10 GeV. ISR selections have a more
pronounced impact on systematic uncertainties. Track selection changes involve
the number of track hits in the time projection chamber, shifting from 4 to 7.
Including the $B(0,0)$ factor as the normalization choice also introduces a
systematic uncertainty. We evaluate its impact based on the statistical
uncertainty of the $B(0,0)$ normalization factor. Generally, these systematic
uncertainties affect $\Delta\phi$ bins uniformly. Lastly, the residual MC
correction factor results in an uncorrelated uncertainty across $\Delta\phi$
bins ascertained through different fit attempts on this correction factor.
Three function types are evaluated, with half of their maximum deviation
deemed as the associated uncertainty.

The two-particle correlation functions for inclusive and high multiplicity
events are shown in Fig. 1. No significant ridge-like structure was observed
in the correlation function at low multiplicity (${\rm N}_{\rm trk}<50$). In
the highest multiplicity bin (${\rm N}_{\rm trk}>50$), an intriguing U shape
was revealed at the large $|\Delta\eta|$ and small $\Delta\phi$ phase space,
which is studied further in the later sections.

One-dimensional distributions in $\Delta\phi$ are studied by averaging the
two-particle correlation function over the region between
$1.6<|\Delta\eta|<3.2$ to investigate the long-range correlation in finer
detail. Fig. 2 shows the comparisons between data and MC on the long-range
azimuthal differential associated yields. The MC simulation aligns well with
the data for low multiplicity events with ${\rm N}_{\rm trk}<40$. However, in
the highest multiplicity class, where ${\rm N}_{\rm trk}>50$, the data reveals
a long-range near-side signal that the MC simulation does not capture.
Moreover, the data display a more significant slope when going to large
$\Delta\phi$ than predictions from MC. We also examined the correlation
functions using the pythia 8 simulation, which allows for the inclusion of
microscopic collective effects from the shoving mechanism Bierlich et al.
(2018, 2021). However, a similar long-range near-side enhancement is not seen
in the pythia 8 simulation, either with or without the inclusion of the
shoving model.

Figure 2: For the long-range region $1.6<|\Delta\eta|<3.2$, the azimuthal
associated yield is presented for $N_{\rm trk}\geq 5$ (left) and $N_{\rm
trk}\geq 50$ (right). Data is presented in red dots with statistical error
bars, while systematic uncertainties are detailed in the text. The pythia 6
model is shown in blue with its statistical error band.

The size of any potential enhancement around $\Delta\phi=0$ is calculated by
fitting this distribution from $0<\Delta\phi<\pi/2$ and then performing a zero
yield at minimum (ZYAM) subtraction procedure using the fit minimum,
$c_{\text{ZYAM}}$ Ajitanand et al. (2005). A constant, combined with a three-
term Fourier series, was used as the nominal fit function. Fits with a purely-
even quartic function and a purely-even quadratic function plus a
$\cos{2\Delta\phi}$ term were also attempted. Discrepancies resulting from
these different choices of fit function were found to be small and are
included in the systematic uncertainties of the total near-side yield
calculation. After this subtraction and correction for reconstruction effects,
the results are shown for ${\rm N}_{\rm trk}\geq 5$ and ${\rm N}_{\rm trk}\geq
50$ in Fig. 2.

Figure 3: Confidence limits on associated yield as a function of
$\left\langle{\rm N}_{\mathrm{trk}}^{\mathrm{corr}}\right\rangle$ in the
thrust axis analysis. This work (LEP-II analysis, $\sqrt{s}=183-209$ GeV) is
shown in red, overlapping with results from Belle (pale purple) Chen et al.
(2022), LEP-I (pale orange) Badea et al. (2019), and ALICE (pale gray, lab
frame) Acharya et al. (2023). The label “$>5\sigma$” indicates the $5\sigma$
confidence level upper limit.

The excess yield of particle pairs near $\Delta\phi=0$ is determined by
integrating the data up to the ZYAM fit’s minimum position in $\Delta\phi$.
For low multiplicity, a confidence limit (C.L.) on the near-side pair excess
is deduced using a bootstrap method Efron (1979). This considers the
variability in correlation function data points based on uncertainties. For
each ${\rm N}_{\rm trk}$ bin, the bootstrap samples $2\times 10^{5}$
variations. Most yield a minimum at $\Delta\phi=0$, implying zero associated
yield. If over 5% of variations exceed a yield of $1\times 10^{-7}$, a 95%
C.L. is given. Otherwise, a C.L. for variations below this threshold is
stated. This typically occurs in low multiplicity scenarios due to minor
uncertainties. At high multiplicity, the central value and the total
uncertainty are reported, and the results are shown in Fig. 3. The results are
also overlayed with the associated yield reported in other small systems:
$e^{+}e^{-}$ collisions by Belle Chen et al. (2022), ALEPH (LEP-I) Badea et
al. (2019), and low-multiplicity $pp$ collisions by ALICE Acharya et al.
(2023). Incorporating the same scaling treatment for $e^{+}e^{-}$ and $pp$
collisions as detailed in ALICE publication Acharya et al. (2023), we scale
the $x$ axis of the ALICE data by the acceptance correction coefficients
$c_{\rm ee}=0.78$ and $c_{\rm pp}=0.57$ for ALEPH and ALICE experiments,
respectively. The scaled $\left\langle{\rm
N}_{\mathrm{trk}}^{\mathrm{corr}}\right\rangle$ for ALICE data points are
displayed with uncertainty ranges from the scaling process. A half of the
maximum deviation between the correction coefficients is quoted as the
relative uncertainty. The reported thrust C.L.s are compatible or lower than
the central values of the associated yield reported by CMS and ALICE, although
the systematic uncertainties of the CMS measurements at low multiplicity are
large. These C.L.s contrast measurements of a nonzero azimuthal anisotropy
signal in lower multiplicity pp collisions Aaboud et al. (2017); Khachatryan
et al. (2017a). At a high multiplicity above 50, the results are compatible
with pp results from ALICE.

In Fig. 4, the extracted $v_{n}$ coefficients between the ALEPH data and the
archived PYTHIA6 simulation are compared as a function of $p_{T}$. The quoted
$v_{n}$ coefficients are obtained from $V_{n\Delta}$ assuming a factorization
between $v_{n}^{assoc}$ and $v_{n}^{trig}$, leading to
$v_{n}=\text{sign}(V_{n\Delta})\sqrt{|V_{n\Delta}|}$. The inclusive result,
dominated by events with lower ${\rm N}_{\rm trk}$, is presented in the left
panel. We observe a decent agreement between data and simulation. A difference
is seen for high multiplicity events with ${\rm N}_{\rm trk}\geq 50$, as shown
in the right panel. The simulation generally predicts a smaller magnitude for
$|v_{n}|$, reflecting the more complex event topologies selected by the large
particle multiplicity. The data, however, shows an intriguing trend compared
to the simulation, especially in $v_{2}$ and $v_{3}$, where the magnitude is
larger.

The $V_{2\Delta}$ for high multiplicity events with ${\rm N}_{\rm trk}\geq 50$
is also compared with measurements from the CMS collaboration in high
multiplicity proton-proton data Khachatryan et al. (2017b) across three
different collision energies, as shown in Fig. 5. To further suppress
contributions from known processes, the $V_{2\Delta}$ from the simulation is
subtracted from that of the data. There is no large dependence of
$v_{2}^{\text{sub}}\\{2\\}$ on collision energy in high multiplicity proton-
proton collisions. A remarkably similar trend is observed in the ALEPH data
compared to $v_{2}^{\rm sub}\\{2\\}$ in proton-proton collisions. Both
datasets exhibit a rising trend as a function of $p_{\text{T}}$ up to 3 GeV
with a similar magnitude.

Figure 4: $v_{n}$ as a function of the track pairs’ $p_{T}$ requirement in
different multiplicity intervals for the thrust axis analysis for the LEP-II
high-energy sample. Data’s $v_{1}$, $v_{2}$, and $v_{3}$ are shown in black,
red, and purple error bars. MC results are dashed lines with corresponding
colors. Figure 5: Excess of flow coefficient ${\rm sign}(\Delta
V_{2})\sqrt{\Delta V_{2}}$, where $\Delta V_{2}=V_{2,\rm data}-V_{2,\rm MC}$,
as a function of the track pairs’ $p_{T}$ requirement for ${\rm N}_{\rm
trk}\geq 50$ in the thrust axis analysis for LEP-II high-energy sample. The
result is overlaid with CMS subtracted flow coefficient measurements
Khachatryan et al. (2017b).

In summary, we present the first measurement of two-particle angular
correlations from $e^{+}e^{-}$ annihilation at energies $\sqrt{s}=183$–$209$
GeV using archived ALEPH LEP-II data recorded between 1996 and 2000. In
analyzing the thrust axis of these collisions between $\sqrt{s}=183$ to 209
GeV, a long-range near-side excess in the correlation function emerges. For
the first time, we decomposed two-particle correlation functions in
$e^{+}e^{-}$ collisions using a Fourier series. The resulting Fourier
coefficients $v_{n}$ from LEP-II provided a comparison to the archived MC,
especially in high multiplicity events where particle counts exceeded 50; the
magnitudes of $v_{2}$ and $v_{3}$ in data are larger than those in the Monte
Carlo reference. Highlighting these contrasts, we present the difference in
$v_{2}$ between data and the MC. The difference between data- and MC-derived
$v_{2}$ as a function of associated particle $p_{T}$ is remarkably similar to
the $v_{2}^{\rm sub}\\{2\\}$ measured in high multiplicity $pp$ collisions.
These intriguing findings fortify our understanding of the underlying
mechanisms in particle collisions and shed light on the origins of flow-like
signals in smaller collision systems.

The authors would like to thank the ALEPH Collaboration for their support and
foresight in archiving their data. We would like to thank the valuable
comments and suggestions from Roberto Tenchini, Guenther Dissertori, Wei Li,
Jiangyong Jia, Wit Busza, Néstor Armesto, Jean-Yves Ollitrault, Jürgen
Schukraft and Jan Fiete Grosse-Oetringhaus. This work has been supported by
the Department of Energy, Office of Science, under Grant No. DE-SC0011088 (to
Y.-C.C., Y.C., M.P., T.S., C.M., Y.-J.L.), Eric and Wendy Schmidt AI in
Science Postdoctoral Fellowship (A. Badea) and Grant No. DE-SC0012567 (to
J.T.).

## References

  * Adams et al. (2005) J. Adams et al. (STAR), Phys. Rev. Lett. 95, 152301 (2005), eprint nucl-ex/0501016.
  * Abelev et al. (2009) B. I. Abelev et al. (STAR), Phys. Rev. C 80, 064912 (2009), eprint 0909.0191.
  * Alver et al. (2010) B. Alver et al. (PHOBOS), Phys. Rev. Lett. 104, 062301 (2010), eprint 0903.2811.
  * Chatrchyan et al. (2012) S. Chatrchyan et al. (CMS), Eur. Phys. J. C72, 2012 (2012), eprint 1201.3158.
  * Aamodt et al. (2012) K. Aamodt et al. (ALICE), Phys. Lett. B 708, 249 (2012), eprint 1109.2501.
  * Adam et al. (2019) J. Adam et al. (STAR), Phys. Rev. Lett. 122, 172301 (2019), eprint 1901.08155.
  * Busza et al. (2018) W. Busza, K. Rajagopal, and W. van der Schee, Ann. Rev. Nucl. Part. Sci. 68, 339 (2018), eprint 1802.04801.
  * Khachatryan et al. (2010) V. Khachatryan et al. (CMS), JHEP 09, 091 (2010), eprint 1009.4122.
  * Aad et al. (2016) G. Aad et al. (ATLAS), Phys. Rev. Lett. 116, 172301 (2016), eprint 1509.04776.
  * Chatrchyan et al. (2013) S. Chatrchyan et al. (CMS), Phys. Lett. B718, 795 (2013), eprint 1210.5482.
  * Abelev et al. (2013a) B. Abelev et al. (ALICE), Phys. Lett. B 719, 29 (2013a), eprint 1212.2001.
  * Abelev et al. (2013b) B. B. Abelev et al. (ALICE), Phys. Lett. B 726, 164 (2013b), eprint 1307.3237.
  * Aad et al. (2013) G. Aad et al. (ATLAS), Phys. Rev. Lett. 110, 182302 (2013), eprint 1212.5198.
  * Aaij et al. (2016) R. Aaij et al. (LHCb), Phys. Lett. B762, 473 (2016), eprint 1512.00439.
  * Adare et al. (2013) A. Adare et al. (PHENIX), Phys. Rev. Lett. 111, 212301 (2013), eprint 1303.1794.
  * Ollitrault (1992) J.-Y. Ollitrault, Phys. Rev. D46, 229 (1992).
  * Alver and Roland (2010) B. Alver and G. Roland, Phys. Rev. C81, 054905 (2010), [Erratum: Phys. Rev.C82,039903(2010)], eprint 1003.0194.
  * Dumitru et al. (2011) A. Dumitru, K. Dusling, F. Gelis, J. Jalilian-Marian, T. Lappi, and R. Venugopalan, Phys. Lett. B 697, 21 (2011), eprint 1009.5295.
  * Dusling and Venugopalan (2013) K. Dusling and R. Venugopalan, Phys. Rev. D87, 094034 (2013), eprint 1302.7018.
  * Bozek (2012) P. Bozek, Phys. Rev. C85, 014911 (2012), eprint 1112.0915.
  * He et al. (2016) L. He, T. Edmonds, Z.-W. Lin, F. Liu, D. Molnar, and F. Wang, Phys. Lett. B753, 506 (2016), eprint 1502.05572.
  * Nagle and Zajc (2018) J. L. Nagle and W. A. Zajc, Ann. Rev. Nucl. Part. Sci. 68, 211 (2018), eprint 1801.03477.
  * Aad et al. (2021) G. Aad et al. (ATLAS), Phys. Rev. C 104, 014903 (2021), eprint 2101.10771.
  * Tumasyan et al. (2023) A. Tumasyan et al. (CMS), Phys. Lett. B 844, 137905 (2023), eprint 2204.13486.
  * Abt et al. (2020) I. Abt et al. (ZEUS), JHEP 04, 070 (2020), eprint 1912.07431.
  * Badea et al. (2019) A. Badea, A. Baty, P. Chang, G. M. Innocenti, M. Maggi, C. Mcginn, M. Peters, T.-A. Sheng, J. Thaler, and Y.-J. Lee, Phys. Rev. Lett. 123, 212002 (2019), eprint 1906.00489.
  * Chen et al. (2022) Y. C. Chen et al. (Belle), Phys. Rev. Lett. 128, 142005 (2022), eprint 2201.01694.
  * Chen et al. (2023a) Y. C. Chen et al. (Belle), JHEP 03, 171 (2023a), eprint 2206.09440.
  * Nagle et al. (2018) J. L. Nagle, R. Belmont, K. Hill, J. Orjuela Koop, D. V. Perepelitsa, P. Yin, Z.-W. Lin, and D. McGlinchey, Phys. Rev. C97, 024909 (2018), eprint 1707.02307.
  * Bierlich and Rasmussen (2019) C. Bierlich and C. O. Rasmussen, JHEP 10, 026 (2019), eprint 1907.12871.
  * Bierlich et al. (2021) C. Bierlich, S. Chakraborty, G. Gustafson, and L. Lönnblad, JHEP 03, 270 (2021), eprint 2010.07595.
  * Castorina et al. (2021) P. Castorina, D. Lanteri, and H. Satz, Eur. Phys. J. A 57, 111 (2021), eprint 2011.06966.
  * Agostini et al. (2021) P. Agostini, T. Altinoluk, and N. Armesto, Eur. Phys. J. C 81, 760 (2021), eprint 2103.08485.
  * Larkoski and Melia (2021) A. J. Larkoski and T. Melia, JHEP 10, 094 (2021), eprint 2107.04041.
  * Baty et al. (2021) A. Baty, P. Gardner, and W. Li (2021), eprint 2104.11735.
  * Decamp et al. (1990) D. Decamp et al. (ALEPH), Nucl. Instrum. Meth. A294, 121 (1990), [Erratum: Nucl. Instrum. Meth.A303,393(1991)].
  * Tripathee et al. (2017) A. Tripathee, W. Xue, A. Larkoski, S. Marzani, and J. Thaler, Phys. Rev. D96, 074003 (2017), eprint 1704.05842.
  * Heister et al. (2004) A. Heister et al. (ALEPH), Eur. Phys. J. C 35, 457 (2004).
  * Barate et al. (1998) R. Barate et al. (ALEPH), Phys. Rept. 294, 1 (1998).
  * Sjostrand et al. (2001) T. Sjostrand, P. Eden, C. Friberg, L. Lonnblad, G. Miu, S. Mrenna, and E. Norrbin, Comput. Phys. Commun. 135, 238 (2001), eprint hep-ph/0010017.
  * Farhi (1977) E. Farhi, Phys. Rev. Lett. 39, 1587 (1977).
  * Voloshin and Zhang (1996) S. Voloshin and Y. Zhang, Z. Phys. C70, 665 (1996), eprint hep-ph/9407282.
  * Poskanzer and Voloshin (1998) A. M. Poskanzer and S. A. Voloshin, Phys. Rev. C58, 1671 (1998), eprint nucl-ex/9805001.
  * Ackermann et al. (2001) K. H. Ackermann et al. (STAR), Phys. Rev. Lett. 86, 402 (2001), eprint nucl-ex/0009011.
  * Adler et al. (2003) C. Adler et al. (STAR), Phys. Rev. Lett. 90, 082302 (2003), eprint nucl-ex/0210033.
  * Adare et al. (2008) A. Adare et al. (PHENIX), Phys. Rev. C78, 014901 (2008), eprint 0801.4545.
  * Aamodt et al. (2010) K. Aamodt et al. (ALICE), Phys. Rev. Lett. 105, 252302 (2010), eprint 1011.3914.
  * Sirunyan et al. (2018) A. M. Sirunyan et al. (CMS), Phys. Lett. B776, 195 (2018), eprint 1702.00630.
  * Bohm and Zech (2014) G. Bohm and G. Zech, Nucl. Instrum. Meth. A 748, 1 (2014), eprint 1309.1287.
  * Chen et al. (2023b) Y.-C. Chen, Y.-J. Lee, Y. Chen, P. Chang, C. McGinn, T.-A. Sheng, G. M. Innocenti, and M. Maggi (2023b), eprint 2309.09874.
  * Bierlich et al. (2018) C. Bierlich, G. Gustafson, and L. Lönnblad, Phys. Lett. B 779, 58 (2018), eprint 1710.09725.
  * Ajitanand et al. (2005) N. N. Ajitanand, J. M. Alexander, P. Chung, W. G. Holzmann, M. Issah, R. A. Lacey, A. Shevel, A. Taranenko, and P. Danielewicz, Phys. Rev. C72, 011902 (2005), eprint nucl-ex/0501025.
  * Acharya et al. (2023) S. Acharya et al. (ALICE) (2023), eprint 2311.14357.
  * Efron (1979) B. Efron, Ann. Statist. 7, 1 (1979), URL https://doi.org/10.1214/aos/1176344552.
  * Aaboud et al. (2017) M. Aaboud et al. (ATLAS), Phys. Rev. C96, 024908 (2017), eprint 1609.06213.
  * Khachatryan et al. (2017a) V. Khachatryan et al. (CMS), Phys. Lett. B765, 193 (2017a), eprint 1606.06198.
  * Khachatryan et al. (2017b) V. Khachatryan et al. (CMS), Phys. Lett. B 765, 193 (2017b), eprint 1606.06198.


org100Physics Department, University of Jammu, Jammu, India

org101Physics Department, University of Rajasthan, Jaipur, India

org102Physikalisches Institut, Eberhard Karls Universität Tübingen, Tübingen,
Germany

org103Physikalisches Institut, Ruprecht-Karls-Universität Heidelberg,
Heidelberg, Germany

org104Physik Department, Technische Universität München, Munich, Germany

org105Purdue University, West Lafayette, Indiana, United States

org106Research Division and ExtreMe Matter Institute EMMI, GSI
Helmholtzzentrum für Schwerionenforschung GmbH, Darmstadt, Germany

org107Rudjer Bošković Institute, Zagreb, Croatia

org108Russian Federal Nuclear Center (VNIIEF), Sarov, Russia

org109Saha Institute of Nuclear Physics, Kolkata, India

org110School of Physics and Astronomy, University of Birmingham, Birmingham,
United Kingdom

org111Sección Física, Departamento de Ciencias, Pontificia Universidad
Católica del Perú, Lima, Peru

org112SSC IHEP of NRC Kurchatov institute, Protvino, Russia

org113Stefan Meyer Institut für Subatomare Physik (SMI), Vienna, Austria

org114SUBATECH, IMT Atlantique, Université de Nantes, CNRS-IN2P3, Nantes,
France

org115Suranaree University of Technology, Nakhon Ratchasima, Thailand

org116Technical University of Košice, Košice, Slovakia

org117Technical University of Split FESB, Split, Croatia

org118The Henryk Niewodniczanski Institute of Nuclear Physics, Polish Academy
of Sciences, Cracow, Poland

org119The University of Texas at Austin, Physics Department, Austin, Texas,
United States

org120Universidad Autónoma de Sinaloa, Culiacán, Mexico

org121Universidade de São Paulo (USP), São Paulo, Brazil

org122Universidade Estadual de Campinas (UNICAMP), Campinas, Brazil

org123Universidade Federal do ABC, Santo Andre, Brazil

org124University of Houston, Houston, Texas, United States

org125University of Jyväskylä, Jyväskylä, Finland

org126University of Liverpool, Liverpool, United Kingdom

org127University of Tennessee, Knoxville, Tennessee, United States

org128University of the Witwatersrand, Johannesburg, South Africa

org129University of Tokyo, Tokyo, Japan

org130University of Tsukuba, Tsukuba, Japan

org131Université Clermont Auvergne, CNRS/IN2P3, LPC, Clermont-Ferrand, France

org132Université de Lyon, Université Lyon 1, CNRS/IN2P3, IPN-Lyon,
Villeurbanne, Lyon, France

org133Université de Strasbourg, CNRS, IPHC UMR 7178, F-67000 Strasbourg,
France, Strasbourg, France

org134Università degli Studi di Pavia, Pavia, Italy

org135Università di Brescia, Brescia, Italy

org136V. Fock Institute for Physics, St. Petersburg State University, St.
Petersburg, Russia

org137Variable Energy Cyclotron Centre, Kolkata, India

org138Warsaw University of Technology, Warsaw, Poland

org139Wayne State University, Detroit, Michigan, United States

org140Wigner Research Centre for Physics, Hungarian Academy of Sciences,
Budapest, Hungary

org141Yale University, New Haven, Connecticut, United States

org142Yonsei University, Seoul, Republic of Korea

org143Zentrum für Technologietransfer und Telekommunikation (ZTT),
Fachhochschule Worms, Worms, Germany


# OK Google, What Is Your Ontology?
Or: Exploring Freebase Classification to Understand Google’s Knowledge Graph

Niel Chah University of Toronto, Faculty of Information
<EMAIL_ADDRESS>University of Toronto, Faculty of Information

###### Abstract

This paper reconstructs the Freebase data dumps to understand the underlying
ontology behind Google’s semantic search feature. The Freebase knowledge base
was a major Semantic Web and linked data technology that was acquired by
Google in 2010 to support the Google Knowledge Graph, the backend for Google
search results that include structured answers to queries instead of a series
of links to external resources. After its shutdown in 2016, Freebase is
contained in a data dump of 1.9 billion Resource Description Format (RDF)
triples. A recomposition of the Freebase ontology will be analyzed in relation
to concepts and insights from the literature on classification by Bowker and
Star. This paper will explore how the Freebase ontology is shaped by many of
the forces that also shape classification systems through a deep dive into the
ontology and a small correlational study. These findings will provide a
glimpse into the proprietary blackbox Knowledge Graph and what is meant by
Google’s mission to “organize the world’s information and make it universally
accessible and useful”.

###### keywords:

ontology Google Freebase Knowledge Graph classification

###### keywords:

##  1 Introduction

This paper’s title is a riff on a key phrase “OK, Google” that is uttered by
users to trigger Google’s virtual assistant services. Through products such as
the Google Home appliance and a mobile application, the Google Assistant taps
into the structured data in the proprietary Google Knowledge Graph (KG) to
answer queries like “What is the capital of Canada?”. The KG can be most
prominently seen in the structured answers to queries in the web search engine
(see Figure 1). The underlying structures in the KG that allow Google to offer
this significantly different service from its original offerings in providing
links to external resources will be explored.

This paper will consider the underlying ontology that structures the Knowledge
Graph and by extension the kinds of answers that Google Search or Assistant
can provide. By reconstructing the Freebase knowledge base that was acquired
by Google to power its Knowledge Graph, this paper will provide a glimpse into
the proprietary and blackbox system that is used by millions of users for
information retrieval and question answering. The structures found in the
Freebase/Knowledge Graph ontology will be analyzed in light of the findings on
classification systems in a key text by Bowker and Star (2000) [5].

Figure 1: A Knowledge Panel in Google Search

##  2 Background: Freebase and Google Knowledge Graph

In 2007, Freebase, available at freebase.com (see Figure 2), was launched by
Metaweb as an open and collaborative knowledge base [3]. On the website, users
could register a free account and edit the data of various entries, creating
linkages between entities in the data. This was similar to the kind of editing
and data entry that was possible on Wikipedia. A significant difference was
that while Wikipedia predominantly consists of free-form text suitable for an
article, Freebase encoded links and relationships between entities in the
knowledge base.

Figure 2: A screenshot of freebase.com on May 2, 2016 before it was shut down.

In 2010, Metaweb was acquired by Google for an undisclosed sum [13]. Freebase
data was then used to power their internal Knowledge Graph to support Google
Search features such as the Knowledge Panels (see Figure 1) [15]. On December
16, 2014, it was announced on the Freebase Google Plus page that Freebase
would be gradually shut down over the next six months, and its data would be
migrated to Wikidata [10]. In reality, the Freebase service was closed from
any further edits by its users and finally shut down on May 2, 2016 [8]. From
this date onward, all freebase.com URLs began to redirect to the Google
Developers page for the project where a data dump of Freebase data is
available.

##  3 Definition of Terms

It is important to define the concepts and vocabulary that will be used
throughout this paper. An attempt has been made to start with the most
indivisible parts, and build a vocabulary from them below. In Table 1, an
example of a thing in the real world, a conceptual counterpart, and its
implementation in Freebase have been mapped together. After each term has been
defined below, the Freebase term will be used throughout the paper for
consistency.

Table 1: Equivalent Terms in Defining Parts of an Ontology Example from the World |  Concept |  Implementation in Freebase |  Comment
---|---|---|---
Barack Obama |  Entity or Instance |  Freebase Object |  The specific U.S. president.
Person |  Class |  Freebase Type |  Obama belongs to the class of “people”.
Place of Birth |  Relation or Property or Slot |  Freebase Property |  The “place of birth” relation links Obama and Hawaii.
Place of Birth accepts object of Location type |  Facet |  Freebase Property Detail |  The “place of birth” relation links Obama and Hawaii. Hawaii is an object with type Location.
Barack Obama was born in Hawaii. |  Fact |  RDF Triple |  “Fact” is used in the same sense as “statement” or “assertion”.

Object \- A Freebase object is a globally unique identifier that is a
representation in Freebase of something in the world. This broad definition
allows arbitrary things to be represented in the data. A tangible thing, such
as the person Barack Obama, or an abstract thing, such as the number “5”, can
be represented by a Freebase object. An implementation detail is that a
Freebase object is written as a unique machineId (a mid) that is in the format
“/m/” + {alphanumeric}. For instance, “/m/abc123” is the Freebase object for
the person Barack Obama. Further interesting things can then reference the
“/m/abc123” identifier to express interesting relations or properties of
Barack Obama.

Type \- A Freebase type is used to express the concept of a class. According
to Freebase idiomatic usage, it is said that a Freebase object “has X, Y, Z
types”. This is equivalent to saying that such an object “is a member of the
X, Y, Z classes”. In the case of U.S. President Barack Obama, the Freebase
object for Obama would have the types “/people/person” and
“/government/us_president” (the exact notation of these types will be
explained later) to indicate that the object is a member of the Persons and
U.S. Presidents classes.

Property \- A Freebase property is a relation that describes how an object can
be linked to other values or objects. A property can link an object to a value
in the case of a person’s name value “Barack Obama” or a date of birth value
“1960”. A property that links an object to another object is the “place of
birth” as it links an object of People type with an object of Location type.
An implementation detail is that a Freebase property is situated in the
context of a type (i.e., the People type specifies a Place of Birth property,
while other types specify different properties). This will be briefly
explained in a short comment following these definitions.

Property Detail \- A property detail refers to the constraints on what objects
or values can be linked through a property. A property can link an object and
a value in the case of a person’s name value “Barack Obama” or a date of birth
value “1960”, or it can link an object and another object in the case of a
person’s place of birth “(the object for) Hawaii” The specifications that “the
name property links to a text value” or that “the place of birth accepts a
Location typed object” are property details in Freebase.

RDF triple \- The Resource Description Format (RDF) is a specification for
data representation in a “triples” (or tuple of N=3) format [17]. An RDF
triple is a single line of data made up of 3 ordered parts: the subject,
predicate, and object. Each part of the triple plays a role in the data
expressed. From left to right, the (1) subject, through the (2) predicate
part, expressed a relationship to the (3) object. This format intuitively
mirrors how a single triple expresses a fact through a sentence-like format:
Johnny Appleseed (subject) likes (predicate) apples (object).

Ontology \- For this paper, the Freebase ontology is the formal structure and
description of the types, properties and property details that specify how
objects can be related to one another. This definition adapts the definition
by Noy and McGuinness that “an ontology is a formal explicit description of
concepts in a domain of discourse (classes (sometimes called concepts)),
properties of each concept describing various features and attributes of the
concept (slots (sometimes called roles or properties)), and restrictions on
slots (facets (sometimes called role restrictions))…An ontology together with
a set of individual instances of classes constitutes a knowledge base” [14].
An implementation detail is that this ontology is also expressed as RDF
triples (e.g. by saying “Place of Birth - is a - property”).

Noy and McGuiness state there is a distinction between “the ontology” and “the
set of individual instances of classes”, but also advise that in practice
“there is a fine line where the ontology ends and the knowledge base
begins”[14]. For example, as Obama is an instance of the Person and US
President types (classes), the RDF triples that express this would be
considered as part of the knowledge base and not the ontology. It is not just
instances of classes that can be expressed. A Freebase object for the Honda
Civic would represent not a single instance of the automobile, but an
abstracted notion of Honda Civics in general. Depending on how this is
expressed, the ontology could include the notion of Honda Civics as a Freebase
type (a class) where Obama’s Honda could be an instance of this class.

Architecture \- In this paper, the architecture refers to the kind of general
patterns and relationships that can be found in the ontology. Does the
ontology allow for inheritance between classes (or types in Freebase
parlance)? Are there default values associated with properties? How are “zero”
or null values handled? These kinds of questions that are not necessarily
concerned about what is specifically expressed in the ontology (whether
Planes, Trains, or Automobiles) but more so about the ways in which the
ontology expresses it should be addressed by examining the architecture.

The Freeabse ontology of types and properties also has an interesting human-
readable aspect that borrows from the Unix-like use of forward slashes “/” in
directories. In the Freebase ontology, there are a number of top-level domains
on specific subject matters. For example, there is a domain for People and
another domain for Films, expressed as the “/people” and “/film” domains
respectively. Within each domain, there are “type(s)” that represent the
classes of entities in a domain. Thus, there is a Person type, expressed by a
human-readable ID as “/people/person”, and a Deceased Person type, expressed
as “/people/deceased_person” type. In the Film domain, notable types include
the Film Director “/film/film_director” and Film “/film/film”. Within each
type in turn, there are properties to capture the granular facts, such as the
Date of Birth for people “/people/person/date_of_birth”.

##  4 Methodology

The data collection and pre-processing process will be briefly explained. The
Freebase data dump of 1.9 billion triples is available for download in a
N-Triples RDF format [11].111freebase.com The data dumps were parsed using
shell scripts and command line tools run on a MacBook Pro (Early 2015 model,
2.7 GHz Intel Core i5, 8 GB RAM) and an external hard drive (Seagate 1 TB)
[7]. Open source Unix-like tools such as awk, cut, grep, less, more, parallel,
sed, sort, wc, zless, zmore, and zgrep were used to slice out portions of the
data based on the Freebase ontology.

A slice is defined as a subset of the RDF triples data, where the triple’s
predicate (the middle term or “verb” in a simple sentence) is part of a unique
domain, type, or property. For example, a slice for all of the RDF triples
that have a predicate in the form “/people/*” (where the asterisk represents a
wildcard) are put into a distinct slice. This “people” slice then expresses
all data on people-typed properties. This is an intuitive slicing method since
the predicate term is the edge or link between the subject and object nodes,
if triples are conceptualized as a graph. With the resulting cleaned data
sets, the ontology of various domains were examined in detail. A sample of the
slices is shown in Table 2.

Table 2: A Selection of Freebase Slices by Domain, ordered alphabetically Slice Name | Predicate | Number of Triples | % of All Triples
---|---|---|---
american_football | /american_football/* | 278,179 | 0.009
amusement_parks | /amusement_parks/* | 22,880 | 0.001
architecture | /architecture/* | 253,718 | 0.008
astronomy | /astronomy/* | 556,381 | 0.018
automotive | /automotive/* | 46,543 | 0.001

##  5 Freebase Ontology and Classification

Users of the Google search engine and more recently the expanding suite of
voice-assisted virtual assistant services in the form of Google Home will
probably find great utility in receiving the structured answers to certain
queries found in the Knowledge Panels (KPs) or Knowledge Cards. In contrast to
the information retrieval services that were the focus of the search engine
until only recently, KPs presumably service the user’s needs more efficiently,
by reducing the user’s hops to other external links, and pose a competitive
advantage to Google, by keeping users within Google’s virtual properties
longer. However, as “convenient” as it may be to find out at a moment’s
notice, which team won the latest MLB game or when a celebrity was born, it is
important to consider the backend infrastructure that delivers these answers.
A certain order or classification is imposed on some data somewhere to
implement this service. As Bowker and Star note, “Information infrastructure
is a tricky thing to analyze…the easier they are to use, the harder they are
to see.” [5]. What does the system make sense of? What is left out? What is
privileged and by extension what is ignored by Google?

In Sorting Things Out, Bowker and Star note how classification schemes are
inconsistent, contradictory, not natural, and not universal [5]. Although the
Freebase ontology may not immediately seem like a classification system, the
structure of types (classes) and properties is a system based on classifying
diverse kinds of things. The Freebase ontology is more complex than the list
of animals created by the Chinese emperor as recounted by Borges, but there
are still ways in which Freebase is also not “natural, eloquent, and
homogeneous” [4, 5]. As a system for ordering and classifying representations
of things in the world, the Freebase ontology will be discussed in light of
Bowker and Star’s findings on classification.

Bowker and Star make a distinction between two classical kinds of
classification systems: Aristotelian and prototype classification [5].
Although the distinction between the two is not absolute, it is useful to keep
in mind the broadly different approaches in the two systems. An Aristotelian
classification operates “according to a set of binary characteristics that the
object being classified either presents or does not present” while a prototype
classification considers “a broad picture in our minds of what a chair is; and
we extend this picture by metaphor and analogy” [5]. The two main reasons
cited by Bowker and Star as leading to the convergence and mixed usages of
Aristotelian and prototypical approaches are applicable to this paper. An
inconsistency in classification systems arises because (1) “each
classification system is tied to a particular set of coding practices” and (2)
“classification systems in general…reflect the conflicting, contradictory
motives of the sociotechnical situations that gave rise to them” [5]. The
Aristotelian and prototypical characteristics of the Freebase ontology will be
pointed out later in the discussion.

In the next sub-sections, this paper will explore a selection of the ontology
and architecture of Freebase that is most applicable to Bowker and Star’s
critiques of classification systems. A review of the Freebase type system for
expressing entities as members of classes will show how Aristotelian
classification processes can be seen. The notion of incompatible types will be
explained, followed by a reflection on the lack of inheritance in the system.
Then, the interesting implementation of Freebase’s “Has Value” and “Has No
Value” notation will be explored to see how the system is sufficient for
Google’s KG usage. An exploration of how Freebase handles data duplication and
data conflation is used to point out how the system is made to serve Google’s
particular needs. Lastly, a small correlational study is conducted to explore
a research question inspired by Bowker and Star.

###  5.1 Freebase’s Type System

An interesting observation on the Aristotelian nature of the Freebase ontology
can be found from parsing how types (classes) are referred to idiomatically in
Freebase.222Another interesting Freebase idiom that is generally confusing for
outsiders is how a Freebase object is referenced in practice. The Freebase
object that represents a real world entity is said to be a topic (i.e.,
“/m/abc123 is the object for the Barack Obama topic.”). This jargon was
avoided for this paper. A Freebase object that represents a real world entity
is said to “have certain X, Y, Z types” to indicate that it is a member of X,
Y, Z classes. Under this system, types are something that an entity “has”,
much like an entity “has” a certain characteristic like size or colour. The
film The Terminator (1984) had a director, a runtime, and a “Film-ness” (but
not “TV Program-ness”) according to this conceptualization. In this way, types
are added like binary descriptors to the entity being represented, indicating
that this is a Person, Film, or Event.

The notion of incompatibility arises in the Freebase system to express how an
object can have certain types which necessarily exclude it from having other
types. This is implemented at the RDF triples level (e.g. by saying “X - is
incompatible with - Y”) and applies to the whole system. For instance,
incompatibility is enforced so that the object representing the film The
Terminator (1984) and the object representing the Terminator film series
remain distinct. Without this rule, inaccurate relationships may be expressed
and cause embarrassing results in public KPs. For instance, in saying
“Terminator 2 (1991) - is the Sequel to - Terminator” it is important that
this Terminator is the Film. The implementation of incompatibility in the
Freebase architecture serves Google well to prevent these and potentially more
impactful mistakes from surfacing in the Knowledge Panels. When there are
failures (see Figure 3), this can likely be attributed to the lack of an
incompatibility rule.

Figure 3: An unfortunate KP that displays the country of Belarus as a book
(Aug. 2016).

Unlike many widely known classification systems or even Object Oriented
Programming (OOP) language infrastructures, Freebase does not implement
inheritance in its types. Although there is a Person type and a Film Actor
type in Freebase, the latter does not inherit from the former. One possible
explanation for the lack of inheritance in the architecture is that the
aforementioned incompatibilities serve a sufficiently robust role in ensuring
the data does not express an embarrassing, harmful, or incorrect statement
that has the potential to be served in a Google KP.

The lack of inheritance may also be a feature that allows entities to be
represented with greater flexibility. For instance, if the cast in a film is
represented through the Film Actor type, the lack of inheritance makes it
relatively easy to express that a famous non-human animal, like a golden
retriever, was a Film Actor starring in a film. In the interests of Google
Search and KPs, displaying Air Buddy (the dog) as part of the cast of Air Bud
(1997) may be more important than whether the ontology expresses the structure
that Film Actor inherits from Person, or Animal, or Thing.

A general review of Freebase’s type system has been conducted to show how
aspects of the architecture (incompatibility, inheritance) and the ontology
(how a type is determined and “had”) are largely shaped by and aligned with
Google’s interests in presenting data that would be serviceable for users of
Knowledge Panels. This observation follows Bowker and Star’s characterization
of classification systems as being formed for specific uses under specific
sociotechnical contexts.

###  5.2 Has Value or Has No Value?

It has been mentioned in passing throughout the paper that the ontology and
knowledge base are implemented through RDF triples. Triples can express a fact
about an object by linking said object to a value (e.g. by saying “Canada -
was founded in - 1867”). However, how can a triple express an estimated,
uncertain, or null value? There must be a way to express “unknown” (there is
some value, but it is either not known or uncertain) and “null” (there is
definitely not a value; it is undefined) values in Freebase and the Knowledge
Graph. These are called “Has Value” (HV) and “Has No Value” (HNV) value
notations respectively.

In a sense, the HV and HUV notations are “hacks” or workarounds to express
uncertainty and undefined values while preserving the overall implementation
of linked data via RDF triples. First, it is possible to consider a simple
implementation of HV and HNV by letting the third term in a triple express the
uncertainty as a string value (e.g. by saying “Plato - Date of Birth - Unknown
Value”). A slightly more sophisticated implementation may link to an object
representing unknown-ness, say /m/unkn0wn. The actual implementation is more
roundabout. HV and HUV are expressed by linking the property to the object
(e.g. by saying “Date of Birth - Has Value - Plato”). From the data dumps, the
descriptions for these HV and HUV properties also state “Note: this property
takes as MID as value. The object has a value for this property, but the
actual value isn’t known.”333The precise Freebase notation for these
properties are “/freebase/valuenotation/has_no_value” and
“/freebase/valuenotation/has_value”. From the data dumps, the triple that
expresses the description is /freebase/valuenotation/has_value -
/common/topic/description - ”Note: this property takes as MID as value. The
object has a value for this property, but the actual value isn’t known. (E.g.,
we know that this marriage has ended, but we don’t know the date it ended.)
This property is asserted as a ’bare’ property (without
/freebase/valuenotation being asserted as a type).”. The same description
applies for /freebase/valuenotation/has_no_value.

The interesting implementation of expressing unknowns and nulls in this way
may suggest that Freebase/KG was not initially built to support this level of
uncertainty; it was an afterthought once the basic system of triples as a
Knowledge Graph was found to work. In hindsight, it may seem like an unwieldy
workaround, but this kind of architecture may be exactly what Google finds
useful for its purposes. Google KPs are meant to display facts about entities
(the “knowledge” in Knowledge Panels) and rarely are unknowns or nulls
expressed (see Figure 4). The Freebase/KG system expresses known facts well
enough for Google to use it in its production environments in Search, Home,
and Assistant. The notion that Google’s data encodes certain uncertainties is
not exposed to the end user although it is certainly implemented in this
unique way.

Figure 4: A KP that displays “known” facts about Plato (Dec. 2017).

It is also worth considering the level at which uncertainty is expressed in
the architecture. There is the notion of Has Values and Has No Value. Values
outside of these two possibilities are not supported. A person’s Date of Birth
could be known as being either 1900 or 1901 (due to a clerical error or a lost
record). A book’s author could be a well known writer or a group of unknown
writers. A time period could have a range of arbitrarily different values
“1901-10” and “1930-35”. In all of these cases, Google’s system cannot express
these details and only presents two possible options.

###  5.3 Dealing with Doppelgangers and Chimeras

This section’s colourful title concerns how Freebase deals with “merging”
duplicate objects (doppelgangers) and “splitting” conflated objects
(chimeras). A duplicate object refers to another object, “/m/xyz123”, that
represents the same entity as an already existing object, “/m/abc123”.
Duplicates can be a pervasive problem as many data sources, editors, or human
contributors can create another unique object to represent an entity with an
existing representation in the data. These many objects should be “merged”
into one. A conflated object can occur when a single object is linked to facts
that represent more than 1 entity. This single object should be “split” into
many. Without these recommended actions, these situations have the potential
to lead to embarrassing and damaging representations in Google’s KPs.

The New York Times business reporter Rachel Abrams discovered the problems
associated with merging and splitting firsthand through a KP that displayed
her Date of Death was five years ago (see Figure 5). [1]. What occurred here
was that the Google KG had merged data on the living reporter Rachel Abrams
with another deceased Rachel Abrams. Abrams was in a position to publish an
article about her journey in addressing the issue as an end user (and also the
very subject of the KP), but the underlying merge/split operations were hidden
inside the blackbox of Google’s KG.

Figure 5: A KP colourfully marked up by NYT reporter Rachel Abrams (Dec. 2017)
[1].

Just as the “/freebase/*” part of the ontology was used to implement HV and
HNV value notations, the “/dataworld/*” ontology is used to implement
merges.444The description of the “/dataworld/*” domain from the data dumps
reads as “Dataworld is a domain for schema that deals with operational or
infrastructural information in relation to the knowledge graph. It contains
schema that captured information in relation to bulk loading or
transformations of graph data.”. Specifically, the property
“/dataworld/gardening_hint/replaced_by” is used to implement merges between
various objects (e.g. by saying “/m/xyz123 - Replaced By - /m/abc123”). The
description of this property from the data dumps states “This property is used
to point to a node that subsumes the identity of this node, usually as a
result of a merge operation.” (where node refers to the term object used in
this paper). This system of “Replaced By” arrows to point to the canonical
object is arguably quicker than the alternative of editing each triple where
the duplicate object is involved. Like the system of HV and HNV notations,
this merging and splitting system works well enough for quickly triaging any
Abrams-like mistakes.

In the backend of Freebase/KG, the mechanisms that support merging and
splitting objects are important and necessary considering how Google uses the
data in such publicly visible ways. A variety of diverse sources (some
programmatic and others manual) contribute data to the KG. In the event of a
duplicate object being created or an object becoming conflated with another
entity, the effects can be readily seen on Google’s KPs. It is in Google’s
interests to adopt an ontology and architecture that can facilitate such
fixes.

##  6 A Small Correlational Study

A small correlational study was conducted to explore the following research
question. Is there a relationship between the complexity of a domain’s
ontology (the types, properties, etc. for the People, Films, etc. domain) and
the amount of triples expressing facts (the “knowledge base”) associated with
that ontology? Phrased another way, are the ontologies for domains that Google
actively uses for its KPs, like music, films, and people domains, more
“mature” than the ontologies for domains that exist in Freebase but are not
exposed in KPs? This question is based on a hunch that actively curated and
administered domains for Google’s uses have more labour and detail put into
them than other less active domains. This aligns with Bowker and Star’s
observation that classification systems are built for specific uses according
to sociotechnical circumstances. For this study, “complexity” and “maturity”
are operationalized by considering the amount of property details (how many
descriptions, constraints, etc.) associated with a domain.

As explained earlier in the Methodology section, the data dumps were
transformed into a series of slices. Each slice contained a subset of the RDF
triples where the predicate (middle) term corresponded to the domain. The
People slice contains all triples where the predicate is of the form
“/people/*”. The resulting domains for each slices were examined and organized
into three large categories (the full table is available in Appendix A). The
domains that (1) implement Freebase features at a technical level and (2)
duplicate triples through the OWL ontology were ignored.

This left 89 slices on domains concerning specific subject matters, such as
music, films, and TV. As mentioned earlier in the paper, the ontology itself
is also expressed in the data dumps with RDF triples (e.g. by saying “Place of
Birth - is a - property”). For each of the 89 domains, the following
statistics on each domain’s ontology were obtained.

  * •

The number of types and properties in the domain

  * •

The number of descriptions for each type and property

  * •

The number of property details for each type and property

A simple complexity score was calculated by obtaining the average number of
descriptions and property details for each type and property in a domain. The
Pearson’s r correlation coefficient between the count of RDF triples and this
complexity score for all domains was positively correlated at 0.2824, with the
slope of a simple linear regression as 78,424.08 (see Figure 6). When the
outlier Music slice is excluded, the correlation and slope become 0.6680 and
33,899.53 respectively. Although further work is needed to explore this
research question, this small correlational study offers some promising
initial results for further experimentation.

Figure 6: Scatterplots of complexity score and triples count for all slices;
and without music.

##  7 Discussion

This paper has explored select parts of the Freebase ontology and architecture
to discover how they can be related to Bowker and Star’s general findings on
classification schemes. Three parts of the overall architecture were
considered: the Freebase type system and its lack of inheritance and reliance
on incompatibility, the implementation that allows representation of
uncertainty in values, and the implementation of merging and splitting
objects. In addition, a small correlational study was conducted to test a
hypothesis based on a hunch motivated by Bowker and Star. To a large extent,
many of the characteristics found in classification systems were also found in
the ontology and architecture of Freebase. There are “hacks” and workarounds
in the system that implement features to make it possible for Google to
minimize the potential damage from embarrassing KPs and promote Google’s
features as an attractive destination for the end user. These “security”
features are implemented in an environment of linked data, where users can hop
back and forth between entities and their relationships with other entities.

The Google Knowledge Graph is a proprietary system that powers the structured
answers to queries delivered in Search, Home, and Assistant. Very little of
the operations and implementation details of this KG are known to the public,
yet it is tapped into increasingly frequently as the scope of virtual
assistants and home assistants rises. By exploring these aspects of Freebase,
it may be possible to develop an understanding of the Google KG, of which
Freebase was a significant part. Specifically, the underlying structures (the
ontology and architecture) that support the overall pipeline of delivering
answers were explored rather than the particular facts that are represented in
Freebase/KG.

Google’s mission statement is to “to organize the world’s information and make
it universally accessible and useful” [12]. In the field of information
studies, the term information takes on vastly different definitions and uses
depending on who provides the definition. Information in the sense of Claude
Shannon, Michael Buckland, Peter Adriaans, or Fred Dretske take on very
different flavours from one another [16, 6, 2, 9]. What does Google mean by
information? According to the exploration of Google’s KG through Freebase that
was conducted here, information is most closely aligned with Buckland’s
conceptualization of information-as-thing, information in the form of data and
documents [6]. By organizing the world’s information, Google is expressing the
searchable world in billions of RDF triples that conform to the KG system’s
implemented ontology and architecture. Associated with this view of
information are the issues with the ontology/architecture that were mentioned
in this paper. Representations of entities are classified according to a
system of types that come with implicit incompatibilities (one cannot be X and
also be a Y). Only certain kinds of uncertainty and ambiguous values are
captured. The system must be able to readily merge and split data as more and
more of it is amassed from various sources. Ontological work likely
concentrates on subject matters that are deemed valuable for search traffic.

There is also the sense that Google talks about information, but actually
means to say knowledge as seen in the very name for their datastore: the
Knowledge Graph. Google may be trying to invoke Buckland’s information-as-
knowledge [6]. As Buckland notes, information-as-knowledge “is intangible: one
cannot touch it or measure it in any direct way. Knowledge, belief, and
opinion are personal, subjective, and conceptual. Therefore, to communicate
them, they have to be expressed, described, or represented in some physical
way…” [6]. It is likely that Google’s mission, through its Search service,
Home appliance, and virtual Assistant, is more broadly “to organize the
world’s information[-as-thing] and make it universally accessible and useful
[as information-as-knowledge]”.

##  8 Conclusion

This paper has applied concepts from Bowker and Star’s findings on
classification systems to a popular company’s once public database. Due to the
incredible diversity of domains and features that can be found in the Freebase
data dumps, only a small selection of relevant aspects could be examined here.
A small correlational study was also conducted to obtain preliminary findings.
Further research should be conducted by exploring additional aspects of the
Freebase ontology and architecture and by conducting a more thorough
experimental analysis of Freebase. This paper ends in the same way it started,
by invoking the special phrase to say: “OK Google, let’s do more research.”

## References

  * [1] Rachel Abrams.  Google thinks i’m dead.  https://www.nytimes.com/2017/12/16/business/google-thinks-im-dead.html, 2017\.
  * [2] Pieter Adriaans.  A critical analysis of floridi?s theory of semantic information.  Knowledge, Technology & Policy, 23(1-2):41–56, 2010.
  * [3] Kurt Bollacker, Colin Evans, Praveen Paritosh, Tim Sturge, and Jamie Taylor.  Freebase: a collaboratively created graph database for structuring human knowledge.  SIGMOD 08 Proceedings of the 2008 ACM SIGMOD international conference on Management of data, pages 1247–1250, 2008.
  * [4] Jorge Luis Borges.  The analytical language of john wilkins.  Borges: A reader, 1981.
  * [5] Geoffrey C Bowker and Susan Leigh Star.  Sorting things out: Classification and its consequences.  MIT press, 2000.
  * [6] Michael K Buckland.  Information as thing.  Journal of the American Society for Information Science (1986-1998), 42(5):351, 1991.
  * [7] Niel Chah.  nchah/freebase-triples v1.0.0.  https://zenodo.org/record/843599, jan 2017.
  * [8] Jason Douglas.  So long and thanks for all the data!  https://groups.google.com/d/msg/freebase-discuss/WEnyO8f7xOQ/VucJKFVhBAAJ, 2016\.
  * [9] Fred I Dretske.  Precis of knowledge and the flow of information.  Behavioral and Brain Sciences, 6(1):55–63, 1983.
  * [10] Freebase.  Freebase.  https://plus.google.com/109936836907132434202/posts/bu3z2wVqcQc, 2017.
  * [11] Google.  Data Dumps.  https://developers.google.com/freebase/, 2017.
  * [12] Google.  Our company.  https://www.google.com/intl/en/about/our-company/, 2017.
  * [13] Jack Menzel.  Deeper understanding with Metaweb.  https://googleblog.blogspot.ca/2010/07/deeper-understanding-with-metaweb.html, 2010\.
  * [14] Natalya F Noy and Deborah L McGuinness.  Ontology development 101: A guide to creating your first ontology, 2001\.
  * [15] Thomas Pellissier Tanon, Denny Vrandecic, Sebastian Schaffert, Thomas Steiner, and Lydia Pintscher.  From Freebase to Wikidata.  Proceedings of the 25th International Conference on World Wide Web - WWW ’16, pages 1419–1428, 2016.
  * [16] Claude E Shannon and Warren Weaver.  The mathematical theory of information, 1949.
  * [17] W3C.  RDF 1.1 Concepts and Abstract Syntax., 2014.

##  9 Appendix A

All of the Freebase domains have been clustered into one of three groups:
Freebase Implementation Domains, OWL Domains, and Subject Matter Domains. The
Freebase Implementation Domains include the triples that express the ontology
and other helper domains that implemented Freebase on a technical level. OWL
Domains include the triples based on the standard Web Ontology Language (OWL).
Subject Matter Domains cover various domain areas on different subject matters
in the world, from football to zoos.

Table 3: Freebase Domains No. | Name |  Domain | Triples | Total % | Group %
---|---|---|---|---|---
Freebase Implementation Domains
1 | common |  /common/* | 1,429,443,085 | 45.658% | 58.507%
2 | type |  /type/* | 788,652,672 | 25.191% | 32.280%
3 | key |  /key/* | 149,564,822 | 4.777% | 6.122%
4 | kg |  /kg/* | 30,689,453 | 0.980% | 1.256%
5 | base |  /base/* | 24,063,303 | 0.769% | 0.985%
6 | freebase |  /freebase/* | 11,259,415 | 0.360% | 0.461%
7 | dataworld |  /dataworld/* | 7,054,575 | 0.225% | 0.289%
8 | topic_server |  /topic_server/* | 1,010,720 | 0.032% | 0.041%
9 | user |  /user/* | 912,258 | 0.029% | 0.037%
10 | pipeline |  /pipeline/* | 547,896 | 0.018% | 0.022%
11 | kp_lw |  /kp_lw/* | 1,089 | 0.000% | 0.000%
OWL Domains
1 | type |  rdf-syntax-ns#type | 266,321,867 | 8.507% | 78.520%
2 | label |  rdf-schema#label | 72,698,733 | 2.322% | 21.434%
3 | domain |  rdf-schema#domain | 71,338 | 0.002% | 0.021%
4 | range |  rdf-schema#range | 71,200 | 0.002% | 0.021%
5 | inverseOf |  owl#inverseOf | 12,108 | 0.000% | 0.004%
Subject Matter Domains
1 | music |  /music/* | 209,244,812 | 6.684% | 60.062%
2 | film |  /film/* | 17,319,142 | 0.553% | 4.971%
3 | tv |  /tv/* | 16,375,388 | 0.523% | 4.700%
4 | location |  /location/* | 16,071,442 | 0.513% | 4.613%
5 | people |  /people/* | 15,936,253 | 0.509% | 4.574%
6 | measurement_unit |  /measurement _unit/* | 15,331,454 | 0.490% | 4.401%
7 | book |  /book/* | 13,627,947 | 0.435% | 3.912%
8 | media_common |  /media _common/* | 6,388,780 | 0.204% | 1.834%
9 | medicine |  /medicine/* | 5,748,466 | 0.184% | 1.650%
10 | award |  /award/* | 4,838,870 | 0.155% | 1.389%
11 | biology |  /biology/* | 3,444,611 | 0.110% | 0.989%
12 | sports |  /sports/* | 3,158,835 | 0.101% | 0.907%
13 | organization |  /organization/* | 2,778,122 | 0.089% | 0.797%
14 | education |  /education/* | 2,609,837 | 0.083% | 0.749%
15 | baseball |  /baseball/* | 2,444,241 | 0.078% | 0.702%
16 | business |  /business/* | 2,134,788 | 0.068% | 0.613%
17 | imdb |  /imdb/* | 1,020,270 | 0.033% | 0.293%
18 | government |  /government/* | 852,785 | 0.027% | 0.245%
19 | cvg |  /cvg/* | 841,398 | 0.027% | 0.242%
20 | soccer |  /soccer/* | 820,410 | 0.026% | 0.235%
21 | time |  /time/* | 791,442 | 0.025% | 0.227%
22 | astronomy |  /astronomy/* | 556,381 | 0.018% | 0.160%
23 | basketball |  /basketball/* | 519,652 | 0.017% | 0.149%
24 | american_football |  /american _football/* | 483,372 | 0.015% | 0.139%
25 | olympics |  /olympics/* | 400,927 | 0.013% | 0.115%
26 | fictional_universe |  /fictional _universe/* | 349,147 | 0.011% | 0.100%
27 | theater |  /theater/* | 320,721 | 0.010% | 0.092%
28 | visual_art |  /visual_art/* | 310,238 | 0.010% | 0.089%
29 | military |  /military/* | 292,533 | 0.009% | 0.084%
30 | protected_sites |  /protected _sites/* | 288,788 | 0.009% | 0.083%
31 | geography |  /geography/* | 256,768 | 0.008% | 0.074%
32 | broadcast |  /broadcast/* | 256,312 | 0.008% | 0.074%
33 | architecture |  /architecture/* | 253,718 | 0.008% | 0.073%
34 | food |  /food/* | 253,415 | 0.008% | 0.073%
35 | aviation |  /aviation/* | 187,187 | 0.006% | 0.054%
36 | finance |  /finance/* | 131,762 | 0.004% | 0.038%
37 | transportation |  /transportation/* | 112,099 | 0.004% | 0.032%
38 | boats |  /boats/* | 108,763 | 0.003% | 0.031%
39 | computer |  /computer/* | 106,986 | 0.003% | 0.031%
40 | royalty |  /royalty/* | 92,787 | 0.003% | 0.027%
41 | library |  /library/* | 86,249 | 0.003% | 0.025%
42 | internet |  /internet/* | 80,426 | 0.003% | 0.023%
43 | wine |  /wine/* | 79,520 | 0.003% | 0.023%
44 | projects |  /projects/* | 79,102 | 0.003% | 0.023%
45 | chemistry |  /chemistry/* | 72,698 | 0.002% | 0.021%
46 | cricket |  /cricket/* | 67,422 | 0.002% | 0.019%
47 | travel |  /travel/* | 56,297 | 0.002% | 0.016%
48 | symbols |  /symbols/* | 56,139 | 0.002% | 0.016%
49 | religion |  /religion/* | 54,887 | 0.002% | 0.016%
50 | influence |  /influence/* | 53,976 | 0.002% | 0.015%
51 | language |  /language/* | 53,588 | 0.002% | 0.015%
52 | community |  /community/* | 50,164 | 0.002% | 0.014%
53 | metropolitan_transit |  /metropolitan _transit/* | 47,777 | 0.002% | 0.014%
54 | automotive |  /automotive/* | 46,543 | 0.001% | 0.013%
55 | digicams |  /digicams/* | 42,188 | 0.001% | 0.012%
56 | law |  /law/* | 37,606 | 0.001% | 0.011%
57 | exhibitions |  /exhibitions/* | 37,434 | 0.001% | 0.011%
58 | tennis |  /tennis/* | 34,853 | 0.001% | 0.010%
59 | venture_capital |  /venture _capital/* | 27,410 | 0.001% | 0.008%
60 | opera |  /opera/* | 26,630 | 0.001% | 0.008%
61 | comic_books |  /comic_books/* | 25,529 | 0.001% | 0.007%
62 | amusement_parks |  /amusement _parks/* | 22,880 | 0.001% | 0.007%
63 | dining |  /dining/* | 21,297 | 0.001% | 0.006%
64 | ice_hockey |  /ice_hockey/* | 17,275 | 0.001% | 0.005%
65 | event |  /event/* | 14,783 | 0.000% | 0.004%
66 | spaceflight |  /spaceflight/* | 14,238 | 0.000% | 0.004%
67 | zoo |  /zoo/* | 13,226 | 0.000% | 0.004%
68 | meteorology |  /meteorology/* | 12,432 | 0.000% | 0.004%
69 | martial_arts |  /martial_arts/* | 12,065 | 0.000% | 0.003%
70 | periodicals |  /periodicals/* | 9,424 | 0.000% | 0.003%
71 | games |  /games/* | 9,024 | 0.000% | 0.003%
72 | celebrities |  /celebrities/* | 8,815 | 0.000% | 0.003%
73 | nytimes |  /nytimes/* | 7,537 | 0.000% | 0.002%
74 | rail |  /rail/* | 7,431 | 0.000% | 0.002%
75 | interests |  /interests/* | 5,345 | 0.000% | 0.002%
76 | atom |  /atom/* | 5,199 | 0.000% | 0.001%
77 | boxing |  /boxing/* | 4,282 | 0.000% | 0.001%
78 | comic_strips |  /comic_strips/* | 4,234 | 0.000% | 0.001%
79 | conferences |  /conferences/* | 2,495 | 0.000% | 0.001%
80 | skiing |  /skiing/* | 1,949 | 0.000% | 0.001%
81 | engineering |  /engineering/* | 1,546 | 0.000% | 0.000%
82 | fashion |  /fashion/* | 1,535 | 0.000% | 0.000%
83 | radio |  /radio/* | 1,385 | 0.000% | 0.000%
84 | distilled_spirits |  /distilled _spirits/* | 1,055 | 0.000% | 0.000%
85 | chess |  /chess/* | 558 | 0.000% | 0.000%
86 | physics |  /physics/* | 449 | 0.000% | 0.000%
87 | geology |  /geology/* | 353 | 0.000% | 0.000%
88 | bicycles |  /bicycles/* | 313 | 0.000% | 0.000%
89 | comedy |  /comedy/* | 120 | 0.000% | 0.000%