# Adversarial Linear-Quadratic Mean-Field Games over Multigraphs

Muhammad Aneeq uz Zaman, Sujay Bhatt, and Tamer Başar First and third authors
are affiliated with the Coordinated Science Laboratory, University of Illinois
at Urbana–Champaign Urbana, IL 61801. Emails<EMAIL_ADDRESS><EMAIL_ADDRESS><EMAIL_ADDRESS>of first and third
authors supported in part by an AFOSR Grant (FA9550-19-1-0353), and in part by
an ARO MURI Grant (AG285).

###### Abstract

In this paper, we propose a game between an exogenous adversary and a network
of agents connected via a multigraph. The multigraph is composed of (1) a
global graph structure, capturing the virtual interactions among the agents,
and (2) a local graph structure, capturing physical/local interactions among
the agents. The aim of each agent is to achieve consensus with the other
agents in a decentralized manner by minimizing a local cost associated with
its local graph and a global cost associated with the global graph. The
exogenous adversary, on the other hand, aims to maximize the average cost
incurred by all agents in the multigraph. We derive Nash equilibrium policies
for the agents and the adversary in the Mean-Field Game setting, when the
agent population in the global graph is arbitrarily large and the “homogeneous
mixing” hypothesis holds on local graphs. This equilibrium is shown to be
unique and the equilibrium Markov policies for each agent depend on the local
state of the agent, as well as the influences on the agent by the local and
global mean fields.

##  I Introduction

Mean-field games (MFGs) model large scale strategic interactions in a network
of individual rational entities (a.k.a agents), where each agent tries to
optimize its individual objective function. Because of the presence of a very
large number of agents in a MFG setting, the strategic interaction is such
that the impact of any one agent on the other agents is negligible; however,
the overall effect of the network on each agent cannot be ignored. There is
rich literature on MFGs in both continuous time and discrete-time ([1, 2, 3]),
with applications in diverse fields, ranging from cybersecurity to inter-bank
borrowing/lending ([4, 5]).

In this paper, we introduce Adversarial MFGs (A-MFGs) as a class of MFGs where
an exogenous adversary and a network of agents strategically interact to
realize self-objectives. The objective of each agent is to achieve consensus
with other agents in a decentralized manner, while the objective of the
exogenous adversary is to adversely affect the cumulative objectives of the
agents in the network. One area of application of A-MFGs is social networks,
and particularly study of evolution of opinions in adversarial settings.

Heretofore, the influence of the structure of the network on the mean-field
interactions has not been extensively studied in the literature. In this
paper, we consider MFGs on multigraphs with an exogenous adversary, where we
explicitly include the effect of the network on the multi-agent interactions.
Multigraphs ([6, 7]) are graphs representing networks where multiple edges
standing for distinct relationships between the agents are permitted. As a
representative example, see Fig. 1 where the dark edges represent
physical/local interactions and light edges represent virtual/social
interactions.

Specifically, we consider linear dynamics for the agents and incorporate the
effect of interaction over the multigraph edges via a local cost associated
with its local neighborhood and the global cost associated with the collective
behavior of the network. The exogenous adversary is able to influence the
agents through the agents’ dynamics and cost functions, and aims to maximize
the average cost over all agents. By considering the MFG (limiting case where
the number of agents tends to infinity), it is possible to propose
approximate-Nash equilibrium strategies for the agents and the adversary in
the finite population game.

Figure 1: The multigraph is composed of two underlying graphs. The global
graph (light edges) represents the social/virtual interactions among agents
and strongly connects all the agents in the multigraph. The local graph (dark
edges) represents physical/inter-personal interactions and connects an agent
to its neighborhood.

###  I-A Motivating example of adversarial MFGs on multigraphs

The framework and results of this paper can be used to analyze the evolution
of opinion dynamics on multigraphs. Consider a multigraph in which the dark
edges in Fig. 1 provide a backbone for the opinion evolution via word-of-mouth
and the light edges act as a backbone for the opinion evolution via
virtual/social media. While with global interactions (light edges) consensus
can be reached, with local interactions (dark edges) only clusters of
consensus opinions may emerge [8]. However, we would be remiss not to consider
the role of exogenous media houses acting to manipulate the opinion evolution
[9].

###  I-B Main results & organization

The contributions of this paper are as follows:

  1. 1.

We formalize the Adversarial MFG (A-MFG) over multigraphs to model the
evolution of opinion dynamics over a network of agents connected through
virtual/global and physical/local networks

  2. 2.

We establish the existence and uniqueness of the mean-field equilibrium for
Linear Quadratic Mean Field Games (LQ-MFGs) on multigraphs. Furthermore, we
show that the local and global effects display some useful linearity
properties, which can be leveraged to obtain equilibrium control policies for
the agents and the adversary. This extends the results available in the
literature by explicitly considering the network structure in the MFG
framework.

  3. 3.

We show that, for an A-MFG on a multigraph, the equilibrium strategy for a
generic agent depends linearly on the local state of the agent, the local
effect (effect of the neighborhood on the agent) and the global effect
(aggregate effect of all the agents).

###  I-C Related literature

Mean-field games (MFGs) were introduced in the seminal works of [1, 10] and,
independently, [2], to address the issue of scalability in dynamic games.
Rooted in the original MFG formulation, LQ-MFGs have been studied in the
continuous-time setting ([11, 12, 13, 14]) as well as the discrete-time
setting ([3, 15, 16]). There is also growing interest in using Reinforcement
Learning for MFGs in the setting where model parameters are unknown to the
agents ([17, 16, 18]).

MFGs with a network structure were first considered in [19], where the
underlying graph was of the Erdös–Rényi type. Recently there has been a number
of papers on Graphon MFGs ([20, 21, 22]) which utilizes graphon theory to
model infinite limits of large networks. Our work here appears to be the first
to investigate the local/physical vs global/virtual graph dichotomy which
emerges in social networks. Similar to our setting, literature on MFGs with
dominating [23] or major agents ([24, 25, 26]) considers one agent which has a
significant effect on the other players. On the contrary the significant
player (adversary) explicitly aims is to minimize the average objective of the
other agents and has no idiosyncratic noise in this work. Works on risk-
sensitive and robust MFGs ([27, 28, 29, 30]) bear semblance to this work as
they consider the presence of an adversarial player through risk-sensitivity
or robustness considerations but differ on account of the multigraph network
structure of this work.

The paper is organized as follows. Section II formulates the A-MFG over
multigraphs, and discusses the relevant solution concepts in finite population
and infinite population settings. Section III deals with characterization of
the equilibrium of the game in the infinite population setting. Section IV
provides concluding remarks and describes future directions of research. The
proofs are included in the appendix.

##  II Problem Formulation

In this section we first introduce the $N$-agent adversarial game over
multigraphs. The game is composed of an exogenous adversary strategically
interacting with $N$ rational agents connected with each other through a
multigraph. Due to the decentralized information structure solving the finite
population game becomes difficult. So we formulate the A-MFG over multigraphs,
where the exogenous adversary interacts with infinitely many players connected
via a multigraph.

###  II-A Adversarial LQ game over multigraphs

We propose a nonzero-sum Linear Quadratic game, where an exogenous adversary
interacts with $N$ agents and all the agents are connected through an
underlying multigraph structure. The agents are connected with each other
through two distinct graphs (global and local), hence the name multigraph. The
global graph structure represents strategic interactions of a global nature,
like virtual/social media interactions, as shown by the gray edged graph in
Fig. 1. The local graph structure represents the physical/real-world
interactions among subsets of agents, like neighborhoods or friendships as
shown by the dark edged graphs in Fig. 1. A set of agents connected by the
local graph (dark) is called a neighborhood. In our formulation, neighborhoods
do not overlap, and hence each agent $i$ belongs to one and only one
neighborhood, which is denoted by $\mathcal{N}(i)$.

The state of each agent $i$ at time $t$ is denoted by
$Z^{i}_{t}\in\mathbb{R}^{z}$ and its control effort by
$U^{i}_{t}\in\mathbb{R}^{u}$. The adversary’s control is denoted by
$V_{t}\in\mathbb{R}^{v}$, through which it can affect the dynamics and the
cost functions of the agents in the multigraph. Each agent (say $i$) has
linear dynamics, where the agent’s control $U^{i}_{t}$ and the adversary’s
control $V_{t}$ enter linearly, as given below.

$\displaystyle Z_{t+1}^{i}$
$\displaystyle=AZ_{t}^{i}+BU_{t}^{i}+CV_{t}+W_{t}^{i},t\in(0,\ldots,T-1)$ (1)

where $W_{t}^{i}\sim\mathcal{G}(0,\Sigma_{w})$ denotes an i.i.d. Gaussian
process noise of agent $i$ and $Z^{i}_{0}\sim\mathcal{G}(\mu_{0},\Sigma_{0})$
is the Gaussian distributed initial state of agent $i$. The initial states
$Z^{i}_{0}$ and the noise processes $W_{t}^{i}$ of all the agents are assumed
to be independent of each other. The aggregate state of a neighborhood
$\mathcal{N}(i)$ at time $t$ is the denoted by $Y_{t}^{i}$ where

$\displaystyle
Y_{t}^{i}=\sum_{j\in\mathcal{N}(i)}Z_{t}^{j}/\lvert\mathcal{N}(i)\rvert$ (2)

In this work, we assume _homogeneous mixing_ [31] of the local graph, which
says that the neighbors of an agent $i$ can be chosen independently from the
set of all $N$ agents. This will be expressed precisely for the mean-field
setting where the number of agents is assumed to be large. The individual cost
function each agent wants to minimize is:

$\displaystyle
J^{i}_{N}(\pi^{i},\pi^{-i},V)=\sum_{t=0}^{T-1}\mathbb{E}\bigg{[}\big{\lVert}Z_{t}^{i}\big{\rVert}^{2}_{Q_{t}}+\bigg{\lVert}Z_{t}^{i}-\frac{1}{N}\sum_{j=1}^{N}Z_{t}^{j}\bigg{\rVert}^{2}_{\bar{Q}_{t}}$
$\displaystyle+\big{\lVert}Z_{t}^{i}-Y_{t}^{i}\big{\rVert}^{2}_{\tilde{Q}_{t}}+\big{\lVert}U_{t}^{i}\big{\rVert}^{2}_{R_{t}}-\big{\lVert}V_{t}\big{\rVert}^{2}_{S_{t}}\bigg{]}+\mathbb{E}\bigg{[}\big{\lVert}Z_{T}^{i}\big{\rVert}^{2}_{Q_{T}}$
$\displaystyle+\bigg{\lVert}Z_{T}^{i}-\frac{1}{N}\sum_{j=1}^{N}Z_{T}^{j}\bigg{\rVert}^{2}_{\bar{Q}_{T}}+\big{\lVert}Z_{T}^{i}-Y_{T}^{i}\big{\rVert}^{2}_{\tilde{Q}_{T}}\bigg{]}.$
(3)

Each agent $i$ at time $t$ is assumed to have access to only its local history
$Z^{i}_{[0,t]}=(Z^{i}_{0},\dots,Z^{i}_{t})$. For agent $i$, the set of all
such policies $\pi^{i}:=(\pi^{i}_{0},\ldots,\pi^{i}_{T-1})$ is denoted by
$\Pi^{i}$. The agent aims to find the control policy $\pi^{i}$ which minimizes
its cost (II-A).

Note that the agent’s cost function, as in (II-A), is composed of a local term
$\big{\lVert}Z_{t}^{i}-Y_{t}^{i}\big{\rVert}^{2}_{\tilde{Q}_{t}}$ due to the
interaction in the local graph and a global term
$\big{\lVert}Z_{t}^{i}-\frac{1}{N}\sum_{j=1}^{N}Z_{t}^{j}\big{\rVert}^{2}_{\bar{Q}_{t}}$
due to the interaction in the global graph. These terms are consensus type
terms and penalize the deviation of agent $i$ from the local behavior and
global aggregate behavior, respectively. Hence, the agent aims to align itself
with local as well as global consensus. The cost function also includes terms
that penalize large values of the state of the agent and its control over the
horizon, as well as (negatively) the control effort of the adversary. The
weighting matrices $Q_{t},\bar{Q}_{t},\tilde{Q}_{t},S_{t}$ and $R_{t}$ in
(II-A) are symmetric matrices such that $Q_{t},\bar{Q}_{t},\tilde{Q}_{t}\geq
0,S_{t},R_{t}>0$ for $t\in[T]$.

The adversary is exogenous to the multigraph and can affect all the agents
through their dynamics (1) and cost functions (II-A). The adversary uses its
influence over the multigraph to disrupt consensus between agents by
maximizing the average cost of all the agents, that is minimize

$\displaystyle
J^{0}_{N}(V,\pi^{(N)})=-\frac{1}{N}\sum_{i=1}^{N}J^{i}_{N}(\pi^{i},\pi^{-i},V)$
(4)

where $\pi^{(N)}=(\pi^{1},\ldots,\pi^{N})$ is the joint policy. The adversary
at time $t$, is assumed to have access to the history of states of all the
agents up to time $t$, $Z_{[0,t]}$. The set of all possible adversary policies
under this information structure, is denoted by $\mathcal{V}$.

The solution concept used for this non-cooperative setting is that of Nash
equilibrium which is the set of policies such that if all the agents and the
adversary follow these policies, then none of the agents or the adversary have
any incentive to unilaterally deviate. This is formally introduced as follows.

######  Definition 1.

The set of agent policies $\pi^{*}=(\pi^{1*},\ldots,\pi^{*N})$ and adversary
policy $V^{*}=(V^{*}_{0},\ldots,V^{*}_{T-1})$ constitute a Nash equilibrium
if, for $i=1,\ldots,N$,

$\displaystyle J^{i}_{N}(\pi^{i*},\pi^{-i*},V^{*})$ $\displaystyle\leq
J^{i}_{N}(\pi^{i},\pi^{-i*},V^{*}),\hskip 5.69046pt\pi^{i}\in\Pi^{i},$
$\displaystyle J^{0}_{N}(\pi^{*},V^{*})$ $\displaystyle\leq
J^{0}_{N}(\pi^{*},V),\hskip 5.69046ptV\in\mathcal{V}$

Due to the local decentralized information structure, computing Nash
equilibria in the finite population setting might prove to be difficult [32],
so we turn to use of the MFG setting where $N\rightarrow\infty$. We call this
the Adversarial Mean-Field Game (A-MFG) over a multigraph, which is different
from the standard LQ-MFGs in two distinct ways: 1) there is an adversary which
aims to manipulate the agents to maximize their average cost, and 2) each
agent is also affected by a local interaction term, in addition to the global
interaction term, due to the presence of local and global graphs in the
multigraph. We assume the local effect follows the homogeneous mixing
hypothesis, as to be further clarified in the following subsection.

###  II-B Adversarial MFG over multigraphs

In the mean-field setting of this game, we consider a generic (representative)
agent interacting with infinitely many agents, where the global aggregate is
denoted by $\bar{Z}_{t}$ and the local aggregate is denoted by $Y_{t}$. The
total number of agents $N\rightarrow\infty$, but the number of agents in a
neighborhood may still be finite. The dynamics of the generic agent in the MFG
is given by

$\displaystyle Z_{t+1}$ $\displaystyle=AZ_{t}+BU_{t}+CV_{t}+W_{t}.$ (5)

We dropped the superscript for simplicity. The cost function the generic agent
aims to minimize is given by

$\displaystyle
J(\mu,\bar{Z},V)=\sum_{t=0}^{T-1}\mathbb{E}\big{[}\big{\lVert}Z_{t}\big{\rVert}^{2}_{Q_{t}}+\big{\lVert}Z_{t}-Y_{t}\big{\rVert}^{2}_{\tilde{Q}_{t}}+\big{\lVert}Z_{t}-\bar{Z}_{t}\big{\rVert}^{2}_{\bar{Q}_{t}}$
$\displaystyle+\big{\lVert}U_{t}\big{\rVert}^{2}_{R_{t}}-\big{\lVert}V_{t}\big{\rVert}^{2}_{S_{t}}\big{]}+\mathbb{E}\big{[}\big{\lVert}Z_{T}\big{\rVert}^{2}_{Q_{T}}+\lVert
Z_{T}-\bar{Z}_{T}\rVert^{2}_{\bar{Q}_{T}}$
$\displaystyle+\big{\lVert}Z_{T}-Y_{T}\big{\rVert}^{2}_{\tilde{Q}_{T}}\big{]}$
(6)

where $\bar{Z}=(\bar{Z}_{0},\ldots,\bar{Z}_{T})$ represents the global
aggregate (also called global mean-field), $Y=(Y_{0},\ldots,Y_{T})$ represents
the local aggregate (local mean-field) and adversary control policy is denoted
by $V=(V_{0},\ldots,V_{T})$. In this setting, the quantities $\bar{Z}$ and $V$
are assumed to be deterministic signals. The control $\mu_{t}$ of the generic
agent at time $t$, is adapted to the filtration generated by the state of the
agent and the local mean-field, $\mathcal{F}^{Z}_{t}$ and
$\mathcal{F}^{Y}_{T}$ respectively. We assume that the multigraph follows the
homogeneous mixing property and as a result the local mean-field is assumed to
satisfy the following property.

######  Assumption 1.

[Homogeneous Mixing] The local mean-field $Y_{t}$ is assumed to be an
exogenous noise process with mean $\mathbb{E}[Y_{t}]=\bar{Z}_{t}$.

######  Remark 1.

This assumption is justified by the homogeneous mixing hypothesis [31], which
states that an agent’s neighbors are randomly distributed among all agents in
the game. This hypothesis is quite standard in infection [31, 33] and opinion
[34] dynamics literature and is being used in mean-field games for the first
time to the best of the author’s knowledge. We notice that this hypothesis is
valid only for large population games.

The adversary aims to disrupt consensus between the agents, by maximizing the
average cost of the generic agent, given the control policy $\mu$ of the
generic agent and the mean-field trajectory,

$\displaystyle
J^{0}(V,\mu,\bar{Z})=\lim_{N\rightarrow\infty}J^{0}_{N}(V,\mu^{(N)})$

where $\mu^{(N)}=(\mu,\ldots,\mu)$ signifies that all the agents (in the
finite population setting) follows the control policy $\mu$. The mean-field
equilibrium is an analog of the Nash equilibrium for the MFG and is defined as
the $4$-tuple $(\mu,V,\bar{Z},Y)$, where $\mu=(\mu_{0},\ldots,\mu_{T-1})$ is
the control policy of the generic agent, $V=(V_{0},\ldots,V_{T-1})$ is the
control policy of the adversary, $\bar{Z}$ is the global mean-field and $Y$ is
the local mean-field of the neighborhood of the generic agent. It is formally
defined as follows.

######  Definition 2.

The tuple $(\mu^{*},V^{*},\bar{Z}^{*},Y^{*})$ is a mean-field equilibrium if
(1) Optimality:

$\displaystyle\mu^{*}=\operatorname*{argmin}_{\mu}J(\mu,\bar{Z}^{*},V^{*}),\hskip
5.69046ptV^{*}=\operatorname*{argmin}_{V}J^{0}(V,\mu^{*},\bar{Z}^{*})$

and (2) Consistency: $\bar{Z}^{*}$ is the aggregate behavior of the infinitely
many agents and $Y^{*}$ is the aggregate behavior of the neighborhood of the
generic agent, if the agents follow control policy $\mu^{*}$ and adversary
follows control policy $V^{*}$.

The quantities $Y^{*}$ and $\bar{Z}^{*}$ are called the _equilibrium_ global
and local mean-fields (MFs), respectively and the control policies $V^{*}$ and
$\mu^{*}$ are called the _equilibrium_ adversarial and generic agent control
policies, respectively. Next we characterize the mean-field equilibrium (MFE)
of the A-MFG over multigraphs, by using open-loop analysis. This is
accomplished by first developing the maximum principle for stochastic
tracking. This maximum principle is then used to obtain the form of optimal
control policies for the generic agent and the adversary. Then the equilibrium
local and global MFs are characterized under these optimal control policies
and the consistency conditions.

##  III MFE Characterization

In this section we characterize the MFE of A-MFG over a multigraph. We start
by providing the form of equilibrium control policy of the generic agent and
the adversary in section III-A. This is obtained by developing a stochastic
maximum principle (SMP) for tracking a stochastic reference signal. The
behavior of the agents and the adversary under equilibrium control policies is
aggregated to characterize the equilibrium global MF in section III-B. This
also leads to a closed-form expression for adversary’s equilibrium policy.
Finally we characterize the behavior of the equilibrium local MF and the
equilibrium policy of the generic agent.

###  III-A Equilibrium control policies

In this subsection, we obtain the form of equilibrium control policies for the
generic agent and the adversary. First we deal with the generic agent’s
equilibrium control problem for a given deterministic global MF
$\bar{Z}=(\bar{Z}_{0},\ldots,\bar{Z}_{T-1})$, adversary control policy
$V=(V_{0},\ldots,V_{T-1})$, and an exogenous stochastic local MF noise process
$Y=(Y_{0},\ldots,Y_{T-1})$. The control of the generic agent is assumed to be
adapted to filtrations generated by its state and the local MF,
$\mathcal{F}^{Z}$ and $\mathcal{F}^{Y}$, respectively. Recalling the dynamics
of the generic agent,

$\displaystyle
Z_{t+1}=AZ_{t}+BU_{t}+CV_{t}+W_{t},Z_{0}\sim\mathcal{G}(\mu_{0},\Sigma_{0}),$
(7)

and that the cost function of the generic agent is coupled with the other
agents through the global MF $\bar{Z}_{t}$ and the local MF $Y_{t}$ in the
following manner,

$\displaystyle
J=\sum_{t=0}^{T-1}\mathbb{E}\big{[}\big{\lVert}Z_{t}\big{\rVert}^{2}_{Q_{t}}+\big{\lVert}Z_{t}-Y_{t}\big{\rVert}^{2}_{\tilde{Q}_{t}}+\big{\lVert}Z_{t}-\bar{Z}_{t}\big{\rVert}^{2}_{\bar{Q}_{t}}+\big{\lVert}U_{t}\big{\rVert}^{2}_{R_{t}}$
$\displaystyle-\big{\lVert}V_{t}\big{\rVert}^{2}_{S_{t}}\big{]}+\mathbb{E}\big{[}\big{\lVert}Z_{T}\big{\rVert}^{2}_{Q_{T}}+\lVert
Z_{T}-\bar{Z}_{T}\rVert^{2}_{\bar{Q}_{T}}+\big{\lVert}Z_{T}-Y_{T}\big{\rVert}^{2}_{\tilde{Q}_{T}}\big{]}$

where $\bar{Z}_{t}$ and $V_{t}$ are deterministic signals and $\mu$ is adapted
to the state process $Z_{t}$ and _exogenous_ local MF $Y_{t}$. We obtain the
SMP for this problem by using a technique similar to [35]. This maximum
principle is novel as it solves the problem of linear quadratic tracking of a
stochastic signal.

######  Theorem 1.

The generic agent’s equilibrium control adapted to filtration
$\mathcal{F}^{Z}_{t}\vee\mathcal{F}^{Y}_{t}$ is given by

$\displaystyle U^{*}_{t}=-R^{-1}_{t}B^{T}\zeta_{t+1}$

where $\zeta_{t}$ can be constructed as

$\displaystyle\zeta_{t}=A^{T}\zeta_{t+1}+(Q_{t}+\bar{Q}_{t}+\tilde{Q}_{t})Z_{t}-\bar{Q}_{t}\bar{Z}_{t}-\tilde{Q}_{t}Y_{t}-M^{\zeta}_{t},$
$\displaystyle\zeta_{T}=(Q_{T}+\bar{Q}_{T}+\tilde{Q}_{T})Z_{T}-\bar{Q}_{T}\bar{Z}_{T}-\tilde{Q}_{T}Y_{T},$
$\displaystyle
M^{\zeta}_{t}=A^{T}\zeta_{t+1}-A^{T}\mathbb{E}\big{[}\zeta_{t+1}\mid\mathcal{F}^{Z}_{t}\vee\mathcal{F}^{Y}_{t}\big{]}.$

where $M^{\zeta}_{t}$ is a martingale difference sequence adapted to
filtration $\mathcal{F}^{Z}_{t}\vee\mathcal{F}^{Y}_{t}$.

The agent’s equilibrim policy doesn’t depend explicitly on the adversary’s
control actions but might have implicit dependence through the mean-field
trajectories. Notice that the co-state equation depends on a martingale
difference sequence adapted to a filtration. This makes the control policy
adapted to the filtration $\mathcal{F}^{Z}_{t}\vee\mathcal{F}^{Y}_{t}$. This
gives us the form of equilibrium control of the generic agent but still does
not provide its closed-form expression. Towards that end we need to first
characterize the equilibrium local and global MFs. And for this, we first
derive the form of equilibrium control of the adversary, which is shown to
depend on just the global MF of the game.

######  Theorem 2.

If the following condition is satisfied

$\displaystyle S_{t}-C^{T}\hat{P}_{t+1}C>0,$ (8)

where the matrix $\hat{P}_{t}$ is defined recursively by
$\hat{P}_{t}=Q_{t}+A^{T}\hat{P}_{t+1}A+A^{T}\hat{P}_{t+1}C(S_{t}-C^{T}\hat{P}_{t+1}C)^{-1}C^{T}\hat{P}_{t+1}A,\hskip
5.69046pt\hat{P}_{T}=Q_{T}$, then the equilibrium control policy of the
adversary is,

$\displaystyle
V^{*}_{t}=-S^{-1}_{t}C^{T}\bar{\zeta}^{0}_{t+1},\bar{\zeta}^{0}_{t}=A^{T}\bar{\zeta}^{0}_{t+1}-Q_{t}\bar{Z}_{t},\bar{\zeta}^{0}_{T}=-Q_{T}\bar{Z}_{T},$
(9)

where $\bar{\zeta}^{0}_{t}$ is the adversary’s co-state and $\bar{Z}$ is the
global MF of the agents.

The theorem provides the form of equilibrium control for the adversary. The
proof starts by obtaining a SMP for the adversary’s equilibrium control
problem in the finite population setting. Then by taking the limit
$N\rightarrow\infty$ we arrive at the conclusion. The condition on positive
definiteness of the matrix $S_{t}-C^{T}\hat{P}_{t+1}C$ is standard condition
in two-player zero sum games [32]. This arises because in essence the
adversary plays a zero sum game with the global MF.

###  III-B Equilibrium MF analysis and equilibrium policies

Having obtained the form of equilibrium control for both the adversary and the
generic agent, we now characterize the equilibrium local and global MFs. This
will allow us to compute the closed-form expressions for the equilibrium
control policies of the generic agent and the adversary. MF characterization
involves proving existence, uniqueness and some useful properties of the
equilibrium global and local MFs.

######  Theorem 3.

If condition (8) is satisfied, then equilibrium global MF follows linear
dynamics,

$\displaystyle\bar{Z}^{*}_{t+1}=E^{-1}_{t}A\bar{Z}^{*}_{t}=\bar{F}_{t}\bar{Z}^{*}_{t},$
(10)

where $E_{t}=(I+BR^{-1}_{t}B^{T}\bar{P}_{t+1}-CS^{-1}_{t}C^{T}\bar{P}_{t+1})$
and $\bar{P}_{t}$ is given by the Riccati equation,
$\bar{P}_{t}=A^{T}\bar{P}_{t+1}E^{-1}_{t}A+Q_{t},P_{T}=Q_{T}$, and the
equilibrium adversarial policy,

$\displaystyle
V^{*}_{t}=S^{-1}_{t}C^{T}\bar{P}_{t}\bar{F}_{t}\bar{Z}^{*}_{t}.$ (11)

is linear in the equilibrium global MF.

Now we need to characterize the dynamics of equilibrium local MF so as to
obtain the closed-form expression of the generic agent’s equilibrium control
policy. Let us define
$Y^{*}_{t}=\sum_{j\in\mathcal{N}}Z^{j*}_{t}/\lvert\mathcal{N}\rvert$, where
each $j$ corresponds to a generic agent chosen independently from infinitely
many agents (due to homogenous mixing property) and $\mathcal{N}$ corresponds
to the neighborhood of the generic agent. The dynamics of the equilibrium
local MF can be characterized as follows,

######  Theorem 4.

If the condition (8) is satisfied, then local equilibrium MF has linear
Gaussian dynamics driven by the equilibrium global MF:

$\displaystyle
Y^{*}_{t+1}=\tilde{F}^{1}_{t}Y^{*}_{t}+\tilde{F}^{2}_{t}\bar{Z}^{*}_{t}+\tilde{E}^{-1}_{t}\tilde{W}_{t}$
(12)

where $\tilde{W}_{t}=\sum_{j\in\mathcal{N}}W^{j}_{t}/\lvert\mathcal{N}\rvert$,

$\displaystyle\tilde{E}_{t}=$
$\displaystyle(I+BR^{-1}B^{T}\tilde{P}_{t+1}-CS^{-1}_{t}C^{T}\tilde{P}_{t+1}),\tilde{F}^{1}_{t}=\tilde{E}^{-1}_{t}A,$
$\displaystyle\tilde{F}^{2}_{t}=$
$\displaystyle\tilde{E}^{-1}_{t}(BR^{-1}_{t}B^{T}-CS^{-1}_{t}C^{T}))\sum_{i=0}^{T-t}\prod_{j=1}^{i}\tilde{H}_{t+j}\bar{Q}_{t+i}\bar{F}_{t+j}$
$\displaystyle\tilde{P}_{t}=$ $\displaystyle
A^{T}\tilde{P}_{t+1}\tilde{E}^{-1}_{t}A+Q_{t}+\bar{Q}_{t},\hskip
5.69046pt\tilde{P}_{T}=Q_{T}+\bar{Q}_{T},$

and $\tilde{H}_{t}=A^{T}(I-\tilde{P}_{t+1}\tilde{E}^{-1}_{t}BR^{-1}B^{T})$.

Having shown that the equilibrium global MF follows deterministic linear
dynamics and the equilibrium local MF follows stochastic linear dynamics
driven by the equilibrium global MF, we now turn our attention to obtaining
the expression for the equilibrium control policy for the generic agent
$\mu^{*}$. We start by concatenating the equilibrium local and global MFs with
the state of the generic agent
$X_{t}=[Z^{T}_{t},Y^{*T}_{t},\bar{Z}^{*T}_{t}]^{T}$. Now finding the generic
agent’s equilibrium control policy, turns into an LQG problem with dynamics,

$\displaystyle X_{t+1}=\bar{A}_{t}X_{t}+\bar{B}U_{t}+\bar{W}_{t}$

where,

$\displaystyle\bar{A}_{t}=\begin{pmatrix}A&0&0\\\
0&\tilde{F}^{1}_{t}&\tilde{F}^{2}_{t}\\\
0&0&\bar{F}_{t}\end{pmatrix},\bar{B}=\begin{pmatrix}B\\\ 0\\\
0\end{pmatrix},\bar{W}_{t}=\begin{pmatrix}W_{t}\\\
\tilde{E}^{-1}_{t}\tilde{W}_{t}\\\ 0\end{pmatrix}$

The reformulated cost function is

$\displaystyle
J=\sum_{t=0}^{T-1}\mathbb{E}\big{[}\big{\lVert}X_{t}\big{\rVert}^{2}_{\mathcal{Q}_{t}}+\big{\lVert}U_{t}\big{\rVert}^{2}_{R_{t}}-\big{\lVert}V_{t}\big{\rVert}^{2}_{S_{t}}\big{]}+\mathbb{E}\big{[}\big{\lVert}Z_{T}\big{\rVert}^{2}_{Q_{T}}\big{]}$

where $V_{t}$ is given by (11) and

$\displaystyle\mathcal{Q}_{t}=\begin{pmatrix}Q_{t}+\bar{Q}_{t}+\tilde{Q}_{t}&-\tilde{Q}_{t}&-\bar{Q}_{t}\\\
-\tilde{Q}_{t}&\tilde{Q}_{t}&0\\\ -\bar{Q}_{t}&0&-\bar{Q}_{t}\end{pmatrix}$

for $t=0,\ldots,T$. If condition (8) is satisfied, this is a standard time
varying linear quadratic regulator (TV-LQR) problem. Using standard techniques
[36] the equilibrium control for the generic agent is,

$\displaystyle
U^{*}_{t}=\mu^{*}_{t}(Z_{t},Y^{*}_{t},\bar{Z}^{*}_{t})=-(R_{t}+\bar{B}^{T}P^{*}_{t+1}\bar{B})^{-1}\bar{B}^{T}P^{*}_{t+1}\bar{A}_{t}X_{t}$

where $P^{*}_{t}$ satisfies the Riccati equation:
$P^{*}_{t}=\mathcal{Q}_{t}+\bar{A}^{T}_{t}P^{*}_{t+1}\bar{A}_{t}-\bar{A}^{T}_{t}P^{*}_{t+1}\bar{B}(R_{t}+\bar{B}^{T}P^{*}_{t+1}\bar{B})^{-1}\bar{B}^{T}P^{*}_{t+1}\bar{A}_{t},P^{*}_{T}=\mathcal{Q}_{T}$.

We have thus completely characterized the MFE of the A-MFG over networks. The
equilibrium control policy of the adversary (11), depends solely on the
equilibrium global MF whose dynamics are deterministic and linear (10). So the
equilibrium global MF and equilibrium adversary policy can be precomputed. The
equilibrium control policy of the generic agent, on the other hand, depends on
the equilibrium local and global MFs. Since the dynamics of the equilibrium
local MF is Gaussian (12), it cannot be computed and hence needs to be
observed by the generic agent. Thus the generic agent’s equilibrium control
policy has closed-loop dependence on the equilibrium local MF but open-loop
dependence on the equilibrium global MF.

Considering opinion dynamics over multigraphs, this means that consensus over
a global graph can be precomputed with high level of confidence (due to the
large number of agents). But consensus over the local graph (neighborhood)
must be actively measured as it is significantly perturbed by the deviations
of individual agents in the neighborhood.

##  IV Conclusion

We have introduced the model of the Adversarial Linear Quadratic Mean-Field
Games over Multigraphs, to study the strategic interactions among a network of
agents and an exogenous adversary. The network is of multigraph type which is
composed of two graphs. The global graph (representing virtual interactions)
connects together all the agents, whereas the local graph (representing
physical interactions) couples a subset of agents to form a neighborhood. The
agents aim to form consensus over the multigraph whereas the adversary aims to
disrupt that consensus.

Under homogeneous mixing hypothesis of the multigraph, we have shown that the
equilibrium control policy of the adversary depends linearly on the
equilibrium global MF whereas the equilibrium control policy of the generic
agent depends linearly on the equilibrium local and global MFs. Furthermore,
we have also shown that the equilibrium global MF can be precomputed, as
opposed to the equilibrium local MF which follows linear Gaussian dynamics. As
a result the equilibrium control policy of the adversary can be computed
offline whereas the equilibrium control policy of the generic agent has online
dependence on the state of the generic agent, the equilibrium local and global
MFs.

As one extension, our goal is to show that the MFE of the A-MFG over
multigraphs, is also an approximate Nash equilibrium for the $N$-player
Adversarial game over multigraphs. Our goal, in particular, is to investigate
the level of the approximation as a function of increasing neighborhood and
global graph size ($\lvert\mathcal{N}\rvert$ and $N$ respectively). Other
possible extensions to this work include generalizing the local graph
structure to overlapping neighborhoods with sparse interconnections.

## References

  * [1] M. Huang, R. P. Malhamé, and P. E. Caines, “Large population stochastic dynamic games: closed-loop Mckean-Vlasov systems and the Nash certainty equivalence principle,” _Communications in Information & Systems_, vol. 6, no. 3, pp. 221–252, 2006.
  * [2] J.-M. Lasry and P.-L. Lions, “Mean field games,” _Japanese Journal of Mathematics_ , vol. 2, no. 1, pp. 229–260, 2007.
  * [3] J. Moon and T. Başar, “Discrete-time LQG mean field games with unreliable communication,” in _53rd IEEE Conference on Decision and Control_. IEEE, 2014, pp. 2697–2702.
  * [4] M. A. uz Zaman, S. Bhatt, and T. Başar, “Secure discrete-time linear-quadratic mean-field games,” in _International Conference on Decision and Game Theory for Security_. Springer, 2020, pp. 203–222.
  * [5] R. Carmona, J.-P. Fouque, and L.-H. Sun, “Mean field games and systemic risk,” _Available at SSRN 2307814_ , 2013.
  * [6] M. Gjoka, C. T. Butts, M. Kurant, and A. Markopoulou, “Multigraph sampling of online social networks,” _IEEE Journal on Selected Areas in Communications_ , vol. 29, no. 9, pp. 1893–1905, 2011.
  * [7] T. Shafie, “A multigraph approach to social network analysis.” _Journal of Social Structure_ , vol. 16, 2015.
  * [8] A. K. Shrimplin, A. Revelle, S. Hurst, and K. Messner, “Contradictions and consensus—clusters of opinions on e-books,” _College & Research Llibraries_, vol. 72, no. 2, pp. 181–190, 2011.
  * [9] M. A. Baum and P. B. Potter, “The relationships between mass media, public opinion, and foreign policy: Toward a theoretical synthesis,” _Annu. Rev. Polit. Sci._ , vol. 11, pp. 39–65, 2008.
  * [10] M. Huang, P. E. Caines, and R. P. Malhamé, “Individual and mass behaviour in large population stochastic wireless power control problems: Centralized and Nash equilibrium solutions,” in _IEEE International Conference on Decision and Control_ , vol. 1. IEEE, 2003, pp. 98–103.
  * [11] ——, “Large-population cost-coupled LQG problems with nonuniform agents: Individual-mass behavior and decentralized $\varepsilon$-Nash equilibria,” _IEEE Transactions on Automatic Control_ , vol. 52, no. 9, pp. 1560–1571, 2007.
  * [12] A. Bensoussan, K. Sung, S. C. P. Yam, and S.-P. Yung, “Linear-quadratic mean field games,” _Journal of Optimization Theory and Applications_ , vol. 169, no. 2, pp. 496–529, 2016.
  * [13] M. Huang and M. Zhou, “Linear quadratic mean field games–part I: The asymptotic solvability problem,” in _23rd MTNS Internat. Symposium, Hong Kong_ , 2018.
  * [14] M. Bardi, “Explicit solutions of some linear-quadratic mean field games,” 2012\.
  * [15] M. Zaman, K. Zhang, E. Miehling, and T. Başar, “Approximate equilibrium computation for discrete-time linear-quadratic mean-field games,” in _2020 American Control Conference (ACC)_. IEEE, 2020, pp. 333–339.
  * [16] Z. Fu, Z. Yang, Y. Chen, and Z. Wang, “Actor-critic provably finds Nash equilibria of linear-quadratic mean-field games,” in _International Conference on Learning Representation_ , 2020.
  * [17] J. Subramanian and A. Mahajan, “Reinforcement learning in stationary mean-field games,” in _International Conference on Autonomous Agents and MultiAgent Systems_ , 2019, pp. 251–259.
  * [18] M. A. uz Zaman, K. Zhang, E. Miehling, and T. Bașar, “Reinforcement learning in non-stationary discrete-time linear-quadratic mean-field games,” in _2020 59th IEEE Conference on Decision and Control (CDC)_. IEEE, 2020, pp. 2278–2284.
  * [19] F. Delarue, “Mean field games: A toy model on an Erdös-Renyi graph.” _ESAIM: Proceedings and Surveys_ , vol. 60, pp. 1–26, 2017.
  * [20] P. E. Caines and M. Huang, “Graphon mean field games and the GMFG equations,” in _2018 IEEE Conference on Decision and Control (CDC)_. IEEE, 2018, pp. 4129–4134.
  * [21] S. Gao, P. E. Caines, and M. Huang, “LQG graphon mean field games,” _arXiv preprint arXiv:2004.00679_ , 2020.
  * [22] P. E. Caines and M. Huang, “Graphon mean field games and the GMFG equations: $\varepsilon$-Nash equilibria,” in _2019 IEEE 58th Conference on Decision and Control (CDC)_. IEEE, 2019, pp. 286–292.
  * [23] A. Bensoussan, M. Chau, and S. Yam, “Mean field games with a dominating player,” _Applied Mathematics & Optimization_, vol. 74, no. 1, pp. 91–128, 2016.
  * [24] J.-M. Lasry and P.-L. Lions, “Mean-field games with a major player,” _Comptes Rendus Mathematique_ , vol. 356, no. 8, pp. 886–890, 2018.
  * [25] Y. Ma and M. Huang, “Linear quadratic mean field games with a major player: The multi-scale approach,” _Automatica_ , vol. 113, p. 108774, 2020.
  * [26] M. Huang, “Large-population LQG games involving a major player: the Nash certainty equivalence principle,” _SIAM Journal on Control and Optimization_ , vol. 48, no. 5, pp. 3318–3353, 2010.
  * [27] H. Tembine, Q. Zhu, and T. Başar, “Risk-sensitive mean-field games,” _IEEE Transactions on Automatic Control_ , vol. 59, no. 4, pp. 835–850, 2013\.
  * [28] D. Bauso, H. Tembine, and T. Başar, “Robust mean field games,” _Dynamic games and applications_ , vol. 6, no. 3, pp. 277–303, 2016.
  * [29] J. Moon and T. Başar, “Linear quadratic risk-sensitive and robust mean field games,” _IEEE Transactions on Automatic Control_ , vol. 62, no. 3, pp. 1062–1077, 2016.
  * [30] D. Bauso, H. Tembine, and T. Basar, “Opinion dynamics in social networks through mean-field games,” _SIAM Journal on Control and Optimization_ , vol. 54, no. 6, pp. 3225–3257, 2016.
  * [31] P. Turnes Jr and L. Monteiro, “An epidemic model to evaluate the homogeneous mixing assumption,” _Communications in Nonlinear Science and Numerical Simulation_ , vol. 19, no. 11, pp. 4042–4047, 2014.
  * [32] T. Başar and G. J. Olsder, _Dynamic Noncooperative Game Theory_. SIAM, 1998.
  * [33] S. Y. Del Valle, J. M. Hyman, and N. Chitnis, “Mathematical models of contact patterns between age groups for predicting the spread of infectious diseases,” _Mathematical biosciences and engineering: MBE_ , vol. 10, p. 1475, 2013.
  * [34] B. O. Baumgaertner, P. A. Fetros, S. M. Krone, and R. C. Tyson, “Spatial opinion dynamics and the effects of two types of mixing,” _Physical Review E_ , vol. 98, no. 2, p. 022310, 2018.
  * [35] M. Chau, Y. Lai, and S. Yam, “Discrete-time mean field partially observable controlled systems subject to common noise,” _Applied Mathematics & Optimization_, vol. 76, no. 1, pp. 59–91, 2017.
  * [36] F. L. Lewis, D. Vrabie, and V. L. Syrmos, _Optimal Control_. John Wiley & Sons, 2012.

## Appendix

###### Proof of Theorem 1.

Consider perturbation of optimal control $U_{t}+\tau\tilde{U}_{t}$,
$\tau\in\mathbb{R}^{+}$ and the control
$\tilde{U}=(\tilde{U}_{0},\tilde{U}_{1},\ldots)$ is adapted to filtration
$\mathcal{F}^{Z}_{t}\vee\mathcal{F}^{Y}_{t}$. The original state becomes
$Z_{t}+\tau\tilde{Z}_{t}$ where

$\displaystyle\tilde{Z}_{t+1}=A\tilde{Z}_{t}+B\tilde{U}_{t},\tilde{Z}_{0}=0.$

Due to optimality of control the necessary Euler condition must be satisfied:

$\displaystyle
0=\frac{d}{d\tau}J\bigg{\rvert}_{\tau=0}=2\mathbb{E}\bigg{[}\langle
U_{0},\tilde{U}_{0}\rangle_{R_{t}}+\sum_{t=0}^{T}\big{(}\langle
Z_{t},\tilde{Z}_{t}\rangle_{Q_{t}}$ $\displaystyle+\langle
U_{t},\tilde{U}_{t}\rangle_{R_{t}}+\langle
Z_{t}-\bar{Z}_{t},\tilde{Z}_{t}\rangle_{\bar{Q}_{t}}+\langle
Z_{t}-Y_{t},\tilde{Z}_{t}\rangle_{\tilde{Q}_{t}}\big{)}$
$\displaystyle+\langle Z_{T},\tilde{Z}_{T}\rangle_{Q_{T}}+\langle
Z_{T}-\bar{Z}_{T},\tilde{Z}_{T}\rangle_{\bar{Q}_{T}}+\langle
Z_{T}-Y_{T},\tilde{Z}_{T}\rangle_{\tilde{Q}_{T}}\bigg{]}$

Let us introduce the adjoint process:

$\displaystyle\xi_{t}$
$\displaystyle=A^{T}\xi_{t+1}+(Q_{t}+\bar{Q}_{t}+\tilde{Q}_{t})Z_{t}-\bar{Q}_{t}\bar{Z}_{t}-\tilde{Q}_{t}Y_{t}-\Delta
M^{\xi}_{t},$ $\displaystyle\xi_{T}$
$\displaystyle=(Q_{T}+\bar{Q}_{T}+\tilde{Q}_{T})Z_{T}-\bar{Q}_{T}\bar{Z}_{T}-\tilde{Q}_{T}Y_{T}$

where, $\Delta
M^{\xi}_{t}=A^{T}\xi_{t+1}-A^{T}\mathbb{E}\big{[}\xi_{t+1}\mid\mathcal{F}_{t}\big{]}$.
Note that
$A^{T}\mathbb{E}[\xi_{t+1}\mid\mathcal{F}_{t}]+(Q+\bar{Q}_{t}+\tilde{Q}_{t})Z_{t}-\bar{Q}_{t}\bar{Z}_{t}-\tilde{Q}_{t}Y_{t}=\xi_{t}\in\mathcal{F}_{t}$.
Consider,

$\displaystyle\langle\tilde{Z}_{t+1},\xi_{t+1}\rangle-\langle\tilde{Z}_{t},p_{t}\rangle$
$\displaystyle=\langle\tilde{Z}_{t+1}-\tilde{Z}_{t},\xi_{t+1}\rangle-\langle\tilde{Z}_{t},\xi_{t+1}-p_{t}\rangle$
$\displaystyle=\langle
A\tilde{Z}_{t}+B\tilde{U}_{t}-\tilde{Z}_{t},\xi_{t+1}\rangle-\langle\tilde{Z}_{t},\xi_{t+1}-A^{T}\xi_{t+1}$
$\displaystyle\hskip
11.38092pt-(Q_{t}+\bar{Q}_{t}+\tilde{Q}_{t})Z_{t}+\bar{Q}_{t}\bar{Z}_{t}+\tilde{Q}_{t}Y_{t}+\Delta
M^{\xi}_{t}\rangle$
$\displaystyle=\langle\tilde{Z}_{t},-(Q+\bar{Q}_{t}+\tilde{Q}_{t})Z_{t}+\bar{Q}_{t}\bar{Z}_{t}+\tilde{Q}_{t}Y_{t}\rangle+\langle
B\tilde{U}_{t},\xi_{t+1}\rangle$ $\displaystyle\hskip
11.38092pt+\langle\tilde{Z}_{t},\Delta M^{\xi}_{t}\rangle$

Summing up for $t=0$ to $T-1$ and taking expectation,

$\displaystyle 0=$
$\displaystyle\mathbb{E}\big{[}\langle\tilde{Z}_{T},-(Q_{T}+\bar{Q}_{T}+\tilde{Q}_{T})Z_{T}+\bar{Q}_{T}\bar{Z}_{T}+\tilde{Q}_{T}Y_{T}\rangle$
$\displaystyle+\sum_{t=0}^{T-1}\langle\tilde{Z}_{t},-(Q_{t}+\bar{Q}_{t}+\tilde{Q}_{t})Z_{t}+\bar{Q}_{t}\bar{Z}_{t}+\tilde{Q}_{t}Y_{t}\rangle$
$\displaystyle+\langle
B\tilde{U}_{t},\xi_{t+1}\rangle+\langle\tilde{Z}_{t},\Delta
M^{\xi}_{t}\rangle\big{]}$

Using the Euler condition and noting that $U_{t}$ and $\tilde{U}_{t}$ are
adapted to process $\mathcal{F}^{Z}_{t}\vee\mathcal{F}^{Y}_{t}$,

$\displaystyle 0$ $\displaystyle=\mathbb{E}\big{[}\sum_{t=0}^{T-1}\langle
R_{t}U_{t}+B^{T}\xi_{t+1},\tilde{U}_{t}\rangle\big{]}$
$\displaystyle=\mathbb{E}\big{[}\sum_{t=0}^{T-1}\langle
R_{t}U_{t}+B^{T}\mathbb{E}[\xi_{t+1}\mid\mathcal{F}^{Z}_{t}\vee\mathcal{F}^{Y}_{t}],\tilde{U}_{t}\rangle\big{]}.$

Since $\tilde{U}_{t}$ is arbitrary, we get the optimal control
$U_{t}=-R^{-1}_{t}B^{T}\mathbb{E}[\xi_{t+1}\mid\mathcal{F}^{Z}_{t}\vee\mathcal{F}^{Y}_{t}]$,
$t=0,1,\ldots,T-1$. Let
$\zeta_{t}=\mathbb{E}[\xi_{t}\mid\mathcal{F}^{Z}_{t}\vee\mathcal{F}^{Y}_{t}]$
then

$\displaystyle\zeta_{t}$
$\displaystyle=\mathbb{E}[\xi_{t}\mid\mathcal{F}^{Z}_{t}\vee\mathcal{F}^{Y}_{t}]=A^{T}\mathbb{E}[\xi_{t+1}\mid\mathcal{F}^{Z}_{t}\vee\mathcal{F}^{Y}_{t}]$
$\displaystyle+(Q_{t}+\bar{Q}_{t}+\tilde{Q}_{t})Z_{t}-\bar{Q}_{t}\bar{Z}_{t}-\tilde{Q}_{t}Y_{t}$
$\displaystyle=A^{T}\zeta_{t+1}+(Q_{t}+\bar{Q}_{t}+\tilde{Q}_{t})Z_{t}-\bar{Q}_{t}\bar{Z}_{t}-\tilde{Q}_{t}Y_{t}-\Delta
M^{\zeta}_{t}$

where

$\displaystyle\Delta
M^{\zeta}_{t}=A^{T}\zeta_{t+1}-A^{T}\mathbb{E}[p_{k+1}\mid\mathcal{F}^{Z}_{t}\vee\mathcal{F}^{Y}_{t}]$
$\displaystyle=A^{T}\zeta_{t+1}-A^{T}\mathbb{E}[\mathbb{E}[p_{k+1}\mid\mathcal{F}^{Z}_{t+1}\vee\mathcal{F}^{\bar{Z}}_{t+1}]\mid\mathcal{F}^{Z}_{t}\vee\mathcal{F}^{Y}_{t}]$
$\displaystyle=A^{T}\zeta_{t+1}-A^{T}\mathbb{E}\big{[}\zeta_{t+1}\mid\mathcal{F}^{Z}_{t}\vee\mathcal{F}^{Y}_{t}\big{]}.$

Therefore the optimal control is $U_{t}=-R^{-1}_{t}B^{T}\zeta_{t+1}$. Since
the cost is convex, the optimal control is unique and the necessary condition
for optimality is also sufficient. ∎

###### Proof of Theorem 2.

First we obtain the form of the optimal control for the adversary, for the
finite population setting. Using techniques similar to the proof of Theorem 1,
we can obtain the form of optimal control of adversary:

$\displaystyle 0=$ $\displaystyle
SV_{t}+C^{T}\sum_{i=1}^{N}\zeta^{i,0}_{t+1}=SV_{t}+C^{T}\zeta^{0}_{t+1}$
$\displaystyle\implies V_{t}=-S^{-1}_{t}C^{T}\zeta^{0}_{t+1},$

and the co-state equations are

$\displaystyle\zeta^{i,0}_{t}=$
$\displaystyle-\frac{1}{N}Q_{t}Z^{i}_{t}-\frac{1}{N}\tilde{Q}_{t}\big{(}Z^{i}_{t}-Y^{i}_{t}\big{)}-\frac{1}{N}\bar{Q}_{t}\big{(}Z^{i}_{t}-\bar{Z}_{t}\big{)}$
$\displaystyle+\frac{1}{N}A^{T}\zeta^{i,0}_{t+1}-M^{\zeta,0}_{t},\hskip
11.38092ptt=1,\ldots,T-1$ $\displaystyle\zeta^{i,0}_{T}=$
$\displaystyle-\frac{1}{N}Q_{T}Z^{i}_{T}-\frac{1}{N}\tilde{Q}_{T}\big{(}Z^{i}_{T}-Y^{i}_{T}\big{)}-\frac{1}{N}\bar{Q}_{T}\big{(}Z^{i}_{T}-\bar{Z}_{T}\big{)}$

for $i=(1,\ldots,N)$, where $\zeta^{0}_{t}:=\sum_{i=1}^{N}\zeta^{i,0}_{t}$ and
$M^{\zeta,0}_{t}$ is a Martingale difference sequence adapted to the
filtration
$\mathcal{F}^{0}_{t}=\mathcal{F}^{Z^{1}}_{t}\vee\mathcal{F}^{Z^{1}}_{t}\ldots\mathcal{F}^{Z^{N}}_{t}\vee\mathcal{F}^{Z^{N}}_{t}$.

$\displaystyle
M^{\zeta,0}_{t}=\frac{1}{N}A^{T}\zeta^{i,0}_{t+1}-\frac{1}{N}A^{T}\mathbb{E}[\zeta^{i,0}_{t+1}\mid\mathcal{F}^{0}_{t}].$

For this necessary condition to be sufficient, the adversary’s cost needs to
be convex in $V$. This is satisfied by the concavity condition in 2-player
zero-sum games [32], $S_{t}-C^{T}\hat{P}_{t+1}C>0$ where the matrix
$\hat{P}_{t}$ is defined recursively,

$\displaystyle\hat{P}_{t}$ $\displaystyle=Q_{t}+A^{T}\hat{P}_{t+1}A+$
$\displaystyle
A^{T}\hat{P}_{t+1}C(S_{t}-C^{T}\hat{P}_{t+1}C)^{-1}C^{T}\hat{P}_{t+1}A,\hskip
5.69046pt\hat{P}_{T}=Q_{T}$

Now we take the limit $N\rightarrow\infty$ to arrive at the MFE of the game.
Let $\bar{Z}_{t}$ and $\bar{\zeta}^{0}_{t}$ denote the global MF and adversary
co-state at time $t$ respectively, such that

$\displaystyle\bar{Z}_{t}=\lim_{N\rightarrow\infty}\sum_{i=1}^{N}\frac{Z^{i}_{t}}{N},\hskip
5.69046pt\bar{\zeta}^{0}_{t}=\lim_{N\rightarrow\infty}\sum_{i=1}^{N}\zeta^{i,0}_{t}.$

Notice that the first equation assumes that the tracking signal $\bar{Z}_{t}$
is also the aggregate state trajectory. We can write down the dynamics of
these quantities as follows,

$\displaystyle\bar{Z}_{t+1}=$ $\displaystyle
A\bar{Z}_{t}+B\bar{U}_{t}+CV_{t},$ $\displaystyle V_{t}=$
$\displaystyle-S^{-1}_{t}C^{T}\bar{\zeta}^{0}_{t+1},\hskip
5.69046pt\bar{\zeta}^{0}_{t}=A^{T}\bar{\zeta}^{0}_{t+1}-Q_{t}\bar{Z}_{t},$
(13)

Hence, we arrive at the adversary’s equilibrium control policy. ∎

###### Proof of Theorem 3.

To characterize the equilibrium global MF we start by substituting the
equilibrium control of the generic agent from Theorem 1 into the dynamics of
the generic agent:

$\displaystyle Z_{t+1}$ $\displaystyle=AZ_{t}-BU_{t}+CV_{t}+W_{t},$
$\displaystyle U^{*}_{t}$ $\displaystyle=-R^{-1}_{t}B^{T}\zeta_{t+1}$

where $\zeta_{t}$ and $M^{\zeta}_{t}$ are defined in Theorem 1. By taking
expectation and invoking the _consistency conditions_
$\bar{Z}^{*}_{t}=\mathbb{E}[Z_{t}]$ and
$\bar{Z}^{*}_{t}=\mathbb{E}[Y^{*}_{t}]$, we get

$\displaystyle\bar{Z}^{*}_{t+1}$
$\displaystyle=A\bar{Z}^{*}_{t}+B\bar{U}^{*}_{t}+CV^{*}_{t},\hskip
2.84544pt\bar{U}^{*}_{t}=-R^{-1}_{t}B^{T}\bar{\zeta}_{t+1}$
$\displaystyle\bar{\zeta}_{t}$
$\displaystyle=A^{T}\bar{\zeta}_{t+1}+Q_{t}\bar{Z}^{*}_{t},\hskip
5.69046pt\bar{\zeta}_{T}=Q_{T}\bar{Z}^{*}_{T}$ (14)

where $\bar{\zeta}_{t}=\mathbb{E}[\zeta_{t}]$. Comparing equations (Proof of
Theorem 2.) and (Proof of Theorem 3.) we can see that
$\bar{\zeta}_{t}=-\bar{\zeta}^{0}_{t}$ so the equilibrium global MF and
costate dynamics can be written down as

$\displaystyle\bar{Z}^{*}_{t+1}$
$\displaystyle=A\bar{Z}^{*}_{t}-(BR^{-1}_{t}B^{T}-CS^{-1}_{t}C^{T})\bar{\zeta}_{t+1},$
(15) $\displaystyle\bar{\zeta}_{t}$
$\displaystyle=A^{T}\bar{\zeta}_{t+1}+Q_{t}\bar{Z}^{*}_{t},\hskip
5.69046pt\bar{\zeta}_{T}=Q_{T}\bar{Z}^{*}_{T}$ (16)

Let us assume the form of costate as
$\bar{\zeta}_{t}=\bar{P}_{t}\bar{Z}^{*}_{t}+\bar{s}_{t}$. Substituting into
(15), we get

$\displaystyle\bar{Z}^{*}_{t+1}$
$\displaystyle=A\bar{Z}^{*}_{t}-(BR^{-1}_{t}B^{T}-CS^{-1}_{t}C^{T})(\bar{P}_{t+1}\bar{Z}^{*}_{t+1}+\bar{s}_{t+1}),$
$\displaystyle=E^{-1}_{t}(A\bar{Z}^{*}_{t}-(BR^{-1}_{t}B^{T}-CS^{-1}_{t}C^{T})\bar{s}_{t+1})$
(17)

where $E_{t}=(I+BR^{-1}_{t}B^{T}\bar{P}_{t+1}-CS^{-1}_{t}C^{T}\bar{P}_{t+1})$.
Now substituting $\bar{\zeta}_{t}=\bar{P}_{t}\bar{Z}^{*}_{t}+\bar{s}_{t}$ into
(16), we get

$\displaystyle\bar{P}_{t}\bar{Z}^{*}_{t}+\bar{s}_{t}$
$\displaystyle=A^{T}(\bar{P}_{t+1}\bar{Z}^{*}_{t+1}+\bar{s}_{t+1})+Q_{t}\bar{Z}^{*}_{t},$
$\displaystyle\bar{P}_{T}\bar{Z}^{*}_{T}+\bar{s}_{T}$
$\displaystyle=Q_{T}\bar{Z}^{*}_{T}$

Substituting (17) into this equation, we arrive at

$\displaystyle\bar{P}_{t}\bar{Z}^{*}_{t}+\bar{s}_{t}=A^{T}(\bar{P}_{t+1}E^{-1}_{t}(A\bar{Z}^{*}_{t}$
(18) $\displaystyle\hskip
14.22636pt-(BR^{-1}_{t}B^{T}-CS^{-1}_{t}C^{T})\bar{s}_{t+1})+\bar{s}_{t+1})+Q_{t}\bar{Z}^{*}_{t}$

Comparing coefficients of $\bar{Z}^{*}_{t}$ we obtain the Riccati eq.:

$\displaystyle\bar{P}_{t}=A^{T}\bar{P}_{t+1}E^{-1}_{t}A+Q_{t},\hskip
5.69046ptP_{T}=Q_{T}$

The variable $\bar{s}_{t}$ can be recursively computed from (18):

$\displaystyle\bar{s}_{t}$
$\displaystyle=-A^{T}(\bar{P}E^{-1}_{t}(BR^{-1}_{t}B^{T}-CS^{-1}_{t}C^{T})+I)\bar{s}_{t+1},$

and $\bar{s}_{T}=0$. Solving these equations we get $\bar{s}_{t}\equiv 0$.
Hence, from (17) we can deduce that the equilibrium global MF follows linear
dynamics:

$\displaystyle\bar{Z}^{*}_{t+1}=E^{-1}_{t}A\bar{Z}^{*}_{t}=\bar{F}_{t}\bar{Z}^{*}_{t},$
(19)

where $E_{t}=(I+BR^{-1}_{t}B^{T}\bar{P}_{t+1}-CS^{-1}_{t}C^{T}\bar{P}_{t+1})$.
Furthermore the co-state $\bar{\zeta}_{t}$ has the form
$\bar{\zeta}_{t}=\bar{P}_{t}\bar{Z}^{*}_{t}$. Substituting this into the
equilibrium adversarial control (9) and observing that
$\bar{\zeta}_{t}=-\bar{\zeta}^{0}_{t}$ we get

$\displaystyle
V^{*}_{t}=S^{-1}_{t}C^{T}\bar{\zeta}_{t+1}=S^{-1}_{t}C^{T}\bar{P}_{t}\bar{Z}^{*}_{t+1}=S^{-1}_{t}C^{T}\bar{P}_{t}\bar{F}_{t}\bar{Z}^{*}_{t}.$
(20)

Hence we have shown that the equilibrium adversarial policy is linear in the
equilibrium global MF, which has deterministic linear dynamics. ∎

###### Proof of Theorem 4.

Let us define
$\tilde{\zeta}_{t}=\sum_{j\in\mathcal{N}}\zeta^{j}_{t}/\lvert\mathcal{N}\rvert$,
where $\zeta^{j}_{t}$ is the co-state of the $j$th agent. We can write the
dynamics of these quantities as follows:

$\displaystyle Y^{*}_{t+1}$
$\displaystyle=AY^{*}_{t}-(BR^{-1}_{t}B^{T}-CS^{-1}_{t}C^{T})\tilde{\zeta}_{t+1}+\tilde{W}_{t},$
(21) $\displaystyle\tilde{\zeta}_{t}$
$\displaystyle=A^{T}\tilde{\zeta}_{t+1}+(Q_{t}+\bar{Q}_{t})Y^{*}_{t}-\bar{Q}_{t}\bar{Z}^{*}_{t}-\tilde{M}^{\zeta}_{t},$
(22) $\displaystyle\tilde{\zeta}_{T}$
$\displaystyle=(Q_{T}+\bar{Q}_{T})Y^{*}_{T}-\bar{Q}_{T}\bar{Z}^{*}_{T},$

where $\tilde{W}_{t}=\sum_{j\in\mathcal{N}}W^{j}_{t}/\lvert\mathcal{N}\rvert$
and $\tilde{M}^{\zeta}_{t}$ is a Martingale difference sequence,
$\tilde{M}^{\zeta}_{t}=A^{T}\tilde{\zeta}_{t+1}-A^{T}\mathbb{E}\big{[}\tilde{\zeta}_{t+1}\mid\mathcal{F}^{Y}_{t}\big{]}$.
Now we assume the form of the co-state
$\tilde{\zeta}_{t}=\tilde{P}_{t}Y^{*}_{t}+\tilde{s}_{t}$. Substituting into
the dynamics of the equilibrium local MF, we get

$\displaystyle Y^{*}_{t+1}$ (23)
$\displaystyle=AY^{*}_{t}-(BR^{-1}_{t}B^{T}-CS^{-1}_{t}C^{T})(\tilde{P}_{t+1}Y^{*}_{t+1}+\tilde{s}_{t+1})+\tilde{W}_{t}$
$\displaystyle=\tilde{E}^{-1}_{t}(AY^{*}_{t}-(BR^{-1}_{t}B^{T}-CS^{-1}_{t}C^{T})\tilde{s}_{t+1}+\tilde{W}_{t})$

where
$\tilde{E}_{t}=(I+BR^{-1}B^{T}\tilde{P}_{t+1}-CS^{-1}_{t}C^{T}\tilde{P}_{t+1})$.
Now we substitute $\tilde{\zeta}_{t}=\tilde{P}_{t}Y^{*}_{t}+\tilde{s}_{t}$
into (22) to arrive at

$\displaystyle\tilde{P}_{t}Y^{*}_{t}+\tilde{s}_{t}=A^{T}\tilde{P}_{t+1}\mathbb{E}[Y^{*}_{t+1}\mid\mathcal{F}^{Y}_{t}]+A^{T}\tilde{s}_{t+1}$
(24) $\displaystyle\hskip
22.76228pt+(Q_{t}+\bar{Q}_{t})Y^{*}_{t}-\bar{Q}_{t}\bar{Z}^{*}_{t}$
$\displaystyle=A^{T}\tilde{P}_{t+1}(\tilde{E}^{-1}_{t}(AY^{*}_{t}-(BR^{-1}_{t}B^{T}-CS^{-1}_{t}C^{T})\tilde{s}_{t+1})$
$\displaystyle\hskip
22.76228pt+A^{T}\tilde{s}_{t+1}+(Q_{t}+\bar{Q}_{t})Y^{*}_{t}-\bar{Q}_{t}\bar{Z}^{*}_{t},$
$\displaystyle\tilde{P}_{T}Y^{*}_{T}+\tilde{s}_{T}=(Q_{T}+\bar{Q}_{T})Y^{*}_{T}-\bar{Q}_{T}\bar{Z}^{*}_{T}$

Comparing coefficients of $Y^{*}_{t}$, we arrive at the Riccati equation:

$\displaystyle\tilde{P}_{t}=A^{T}\tilde{P}_{t+1}\tilde{E}^{-1}_{t}A+Q_{t}+\bar{Q}_{t},\hskip
5.69046pt\tilde{P}_{T}=Q_{T}+\bar{Q}_{T}$

From (24) we can compute the process $\tilde{s}_{t}$ in a backwards-in-time
manner,

$\displaystyle\tilde{s}_{t}=\tilde{H}_{t}\tilde{s}_{t+1}-\bar{Q}_{t}\bar{Z}^{*}_{t},\hskip
5.69046pt\tilde{s}_{T}=-\bar{Q}_{T}\bar{Z}^{*}_{T},$

where $\tilde{H}=A^{T}(I-\tilde{P}_{t+1}\tilde{E}^{-1}_{t}BR^{-1}B^{T})$.
Hence, the process $\tilde{s}_{t}$ can be computed as

$\displaystyle\tilde{s}_{t}$
$\displaystyle=-\sum_{i=0}^{T-t}\prod_{j=1}^{i}\tilde{H}_{t+j}\bar{Q}_{t+i}\bar{Z}_{t+i}$
$\displaystyle=-\sum_{i=0}^{T-t}\prod_{j=1}^{i}\tilde{H}_{t+j}\bar{Q}_{t+i}\bar{F}_{t+j}\bar{Z}_{t}.$

The last equality is due to the fact that equilibrium global MF follows linear
dynamics (10). Substituting this expression into (23), we get

$\displaystyle
Y^{*}_{t+1}=\tilde{F}^{1}_{t}Y^{*}_{t}+\tilde{F}^{2}_{t}\bar{Z}^{*}_{t}+\tilde{E}^{-1}_{t}\tilde{W}_{t}$
(25)

where

$\displaystyle\tilde{F}^{1}_{t}=$ $\displaystyle\tilde{E}^{-1}A,$
$\displaystyle\tilde{F}^{2}_{t}=$
$\displaystyle\tilde{E}^{-1}_{t}(BR^{-1}_{t}B^{T}-CS^{-1}_{t}C^{T}))\sum_{i=0}^{T-t}\prod_{j=1}^{i}\tilde{H}_{t+j}\bar{Q}_{t+i}\bar{F}_{t+j}$

Hence the equilibrium local MF has linear Gaussian dynamics driven by the
equilibrium global MF. ∎


# Scientific X-ray

Qi Li,1 Luoyi Fu,2 Xinbing Wang,2∗ Chenghu Zhou3∗

1Department of Electronic Engineering, Shanghai JiaoTong University, China
2Department of Computer Science and Engineering, Shanghai JiaoTong University,
China
3State Key Laboratory of resources and environmental information system,
Institute of Geographical Sciences and Natural Resources Research,
Chinese Academy of Sciences, China
∗To whom correspondence should be addressed;
E-mail<EMAIL_ADDRESS><EMAIL_ADDRESS>
> The rapid development of modern science and technology has spawned rich
> scientific topics to research and endless production of literature in them.
> Just like X-ray imaging in medicine, can we intuitively identify the
> development limit and internal evolution pattern of scientific topic from
> the relationship of massive knowledge? This is essential for grasping
> research trends, helping policy making and even promoting social
> development. To answer this question, we collect 71431 seminal articles of
> topics that cover 16 disciplines and their citation data, analogize the
> evolution of each topic to the growth of a tree, and extracts the ”idea
> tree” of each topic by retaining the most representative citation in the
> network to ”scan” the structure of the development of 71431 topic networks
> from scratch. We define the Knowledge Entropy (KE) metric utilizing the
> proportion of subtrees led by papers in the idea tree to the original
> network, and the contribution of high knowledge entropy nodes to increase
> the depth of the idea tree is regarded as the basis for topic development.
> By observing ”X-ray images” of topics, We find two interesting phenomena:
> (1) Even though the scale of topics (number of publications) may increase
> unlimitedly, there is an insurmountable cap of STL: the depth of the idea
> tree does not exceed 6 jumps, which coincides with the classical ”Six
> Degrees of Separation”! (2) It is difficult for a single article to
> contribute more than 3 jumps to the depth of its topic, to this end, the
> continuing increase in STL needs to be motivated by the influence relay of
> multiple high knowledge entropy nodes. Through substantial statistical fits,
> we derive a unified quantitative relationship between the change in topic
> depth ${\Delta D}^{t}(v)$ and the change in knowledge entropy over time
> ${KE}^{t}\left(v\right)$ of the article $v$ driving the increase in depth in
> the topic: ${\Delta
> D}^{t}(v)\approx\log\frac{KE^{t}(v)}{\left(t-t_{0}\right)^{1.8803}}$. By
> transforming the value conditions of the formula, we can effectively portray
> the different patterns of a single topic in depth development: (1) Too small
> ${KE}^{t}\left(v\right)$ cannot advance the topic depth to achieve a new
> breakthrough (${\Delta D}^{t}(v)$ is always less than 1); (2) Time will
> continue to decay the article’s motivating effect on depth to zero. Any node
> (single article) that fails to stimulate depth for a long time will lead to
> stagnation of topic development; (3) If the depth of the topic fails to
> increase significantly for a long time because of the rapid expansion of
> ${KE}^{t}\left(v\right)$, this article takes some of the new references
> belonging to the original topic and grows uniquely into a new topic, which
> inhibits the development of the original topic. The various patterns and
> phenomena found by topic X-ray may provide a new paradigm for explaining and
> understanding the evolution of human science and technology.

## Introduction

The explosion of the scientific enterprise has boosted the increase of high-
quality scientific publication data (?, ?, ?, ?), significantly attracting
interest in utilizing data-driven methods to understand the process of
scientific evolution (?). In this process, a large number of new scientific
topics emerge continually, and others gain or lose attention (?). Therefore,
this raises a fundamental question: does the evolution and development of
these diverse scientific topics have a limit? Futurist Raymond Kurzweil, in
his futuristic book ”The Singularity Is Near”, predicts that human
technological development will reach its limit in 2045, i.e., the arrival of
the ”technological singularity” (?). At this point, the intelligence of
machines has far surpassed humans, causing it difficult for humans to
understand the technology created by machines and ultimately making it
impossible for humans to push civilization forward. In the current
semiconductor industry, as we approach the atomic scale, chip manufacturing
has become highly complex, and the increase in revenue becomes less
significant (?). At present, Moore’s Law slowdown has become the industry
consensus (?, ?, ?), and soon, the chip miniaturization process will most
likely end at 5 nm (?). By observing the citation trends over time, we can
empirically perceive that the impact of publications and researchers is
dominated by two underlying processes: birth and aging (?). We count the birth
and aging time according to citation trends of 71431 high-impact publications
(citation $\geq$ 1000, covering 16 disciplines, published from 1800 to 2019)
and find that the growth of citation of high-impact work also has a limit (?)
(Fig. 1a). Although citation-based metrics for gauging the impact of academic
entities are widely used, they fail to reveal why the topic will get ”old” and
how the topic evolve from birth to getting ”old” (?, ?, ?, ?, ?).(check
citation)

Current indicators only provide a suboptimal reference for the impact
evolution of a single academic entity. It is hard to transfer directly from
individual academic entities to the scientific topic formed by the interaction
of these entities to explore the limit and evolution of the topic. The
existence of such complex correlations leads to a complex and varied evolution
of topics, which makes it extremely difficult to identify specific patterns.
In the medical field, people use the difference in X-ray transmission
intensity of different tissues to realize the intuitive observation of human
tissue morphology and pathological changes. The emergence of medical X-rays
solves the problem of unintuitive data presentation in early human information
processing, which greatly improves the level of human health. Just like X-ray
imaging in medicine, intuitively identifying the development limit and
internal evolution pattern of scientific topic can provide constructive
guidance to the decision-making process of resource allocation (?, ?),
researcher hiring (?, ?), and direction selection (?, ?). For the government,
potential scientific topics should be funded to allocate trillions of research
funding taxed from the public efficiently (?). For the research institution,
the potential value of the hired candidate’s research topics should be a key
consideration to maximize the recruitment benefit. For a researcher, promising
scientific topics should be explored as a priority to publish more impactful
work. The depth of understanding of the limit of scientific topics arguably
determines the upper limit of mankind’s ability to promote scientific
development to achieve more advanced results (?, ?, ?). However, due to lack
of intuitive and in-depth perception of scientific topics, little is known
about the limit and evolution pattern of them. Here we ask: can we identify
whether a topic has reached the end of its development? What are the evolution
patterns of topics from birth to limit? What inner mechanisms lead to the
existence of limit and specific evolution pattern?

To answer these questions systematically, we need first to define the
scientific topic and seek a suitable way to describe it. In a specific
scientific topic, papers usually have similar research interests and are
linked by complex citation relationships. Moreover, the birth and summary of
the ideas occur among the articles (?) and diffuses through citations, making
the topic developed. Therefore, the scientific topic can be defined and
described as a citation network, where nodes represent papers, edges symbolize
citations, and the evolution of the network is regarded as the topic’s
development. High impact papers that break significant ground or create new
directions always attract numerous articles with similar research interests.
Consequently, the citation network pioneered by a non-trivial influence
article and consisting of the leading paper, the articles citing the leading
paper and all citations among them can be considered as a particular
scientific topic. We collect and integrate academic data from bibliographic
databases, including but not limit to IEEE, ACM, arXiv, Elsevier, and Spring.
We then select 71431 high-impact publications with more than 1000 citations as
pioneering works and construct topic citation networks for them. All leading
articles were published between 1800 and 2019, and their research interests
cover 294 fields in 16 disciplines: History, Computer science, Environmental
science, Geology, Psychology, Mathematics, Physics, Materials science,
Philosophy, Biology, Medicine, Sociology, Art, Economics, Chemistry, and
Political science. To intuitively perceive spatial structure for scientific
topics, we visualize all 71431 topic citation networks to construct the
spatial reviews of them, which simulates a gravity system to spatialize a
network by structurally determined attraction-repulsive force, and we call
them galaxy maps (?). Galaxy map helps us understand the current
characteristics of a scientific topic with the power of human vision and
spatial cognition by showing the structure of the topic citation network (?).
We find the topics without apparent community structure are more conclusive,
whose pioneering works are almost monographs or reviews (Fig. 1b). Since the
broad scope of discussion involved in the seminal work, the child nodes are
weakly correlated, preventing the emergence of complex community structures
over time in the topic network. Although the topic was born in 1992 and has
accumulated more than 6000 citations, the structure of the topic network has
not evolved to become more complex, suggesting that the topic has not moved
forward. However, the topics driving rich community structures’ formation are
usually more inspiring (Fig. 1c, d). At least one child article with non-
trivial influence is bred within the topic, and their citation can even exceed
the leading paper with the network evolving. The pioneering articles of this
type of topic tend to propose new methods, theories, or frameworks. When their
children nodes inherit their ideas and develop them, the topic network will
generate multiple communities, i.e., the original topic is transformed into
multiple new directions and attracts new attention, which clearly indicates a
substantive development of the topic. The emergence or otherwise of complex
community structures indicates a variation in the degree of development of
these topics. However, all of these topics stagnated at some point after their
birth, with the network structure remaining stable and citation trends
declining (Fig. 1b, c, d). This suggests that if new high-impact articles are
not born within the topics to break the stagnation, this will lead to the
topics reaching their limit. Although a galaxy map can transform abstract data
into intuitive structures, it contains much redundant information, making it
hard to reveal the evolutionary pattern of a topic from birth to limit, let
alone the inner causes of its limit. To this end, how can we extract the
backbone of knowledge flow from the intricate referential relationships of the
topic network?

The development of topics is driven by the inheritance of existing knowledge
and the creation of new knowledge through newly published work. In contrast,
the structural redundancies in the network generated by non-critical citations
have a minimal effect on the development of topics. Therefore, repetitive,
invalid inheritance relationships need to be removed to clearly and accurately
reproduce the pattern of topic evolution, which can be achieved by assessing
the similarity between papers. Ideally, we assume that any child article in
the topic except the seminal work is inspired by one of the most essential
citation so that we can get an ”idea tree” that describes the backbone of
knowledge flow within the topic and reveals the inheritance of ideas between
different articles (?). The idea tree is similar to a tree in nature in
different types of work play different roles: the seminal work is like the
root of the tree that provides the impetus for the development of the topic
with their unparalleled innovation; the non-trivial work with different values
of knowledge is like the trunks of the tree that drive the growth of their
branches; the ordinary articles in the topic are more like the leaves that
nourish the impact of the trunk work. Within the topic, when existing
knowledge flows through the nodes (publications), new knowledge is generated
after cognition, fusion, and creation by the human brain and then inspires
later generations. These newly added nodes tend to be more similar to the
creators of new knowledge than the propagators of the original knowledge. The
difference in the quality of the new knowledge leads to a difference in the
number of new child nodes attracted, which is expressed macroscopically as a
difference in the structure of the idea tree. In this way, we can answer two
fundamental questions: the structure of different idea trees characterizes the
evolutionary patterns of different topics; the causes of the limit of
scientific topics can be revealed by the path of the inheritance of ideas.
Specifically, we utilize graph embedding techniques to calculate the
difference between each pair of articles (DiffIdx) in a high dimensional
space. Based on DiffIdx, we further compute the similarity between an article
and the whole topic (ReductionIdx) to judge the importance of each citation. A
detailed description of DiffIdx and ReductionIdx can be found in Supporting
Materials. Idea tree extraction follows two guidelines (Fig. 1e): first, keep
the most valuable citation for every publication except the seminal paper, and
second, ensure the connectivity of the idea tree. The extraction algorithm
consists of 2 steps: first, cut loops in the topic citation network and
second, prune the network to retain the most valuable citation for every child
paper. Based on the topic idea tree we got, we can reproduce the evolutionary
pattern of the topic and thus explore the inner causes of the limits by
tracing the main paths of inheritance of ideas in the topic.

The emergence of new, innovative, and inspiring knowledge can breathe new life
into the topic with the attraction of additional research, which is the
critical internal factor that stimulates the evolution of inheritance
relationships. Therefore, the effective measurement of knowledge is a
fundamental question in revealing the evolution of scientific topics. Although
knowledge exists objectively, it is as invisible as air, making it difficult
to define and lacks a unified quantification. However, the impact of knowledge
can be felt all the time: the more knowledge a person has, the more complex
work he or she can do, and the more knowledge a machine can learn, the more
intelligent it can get. For publications, it is almost impossible to go
directly, effectively, and uniformly evaluate the quality of knowledge without
any experience. Nevertheless, it is still possible to perceive differences in
the quality of knowledge across articles through existing impact metrics such
as the most commonly used citation-based metrics. However, these one-
dimensional indicators consider little about the differences in knowledge
structures inspired by different qualities of knowledge, leaving out a great
deal of helpful information and thus providing only a suboptimal reference.
The idea tree preserves the most useful information within the topic through a
process of de-redundancy, and differences in the complexity of the subtree
structure led by a node characterize the difference in the quality of
knowledge at that node. Although knowledge is hard to capture, based on the
idea tree whose structure is shaped by the flow of knowledge through the topic
network, we can use structural information to measure and analyze the quality
of knowledge. Structural information has proven to be very useful in analyzing
and measuring the complexity of network structures, including biological
networks, social networks, and computer networks. Here we have been inspired
by previous work to design a knowledge entropy to measure the quality of
knowledge in the idea tree nodes. Based on this, we can describe at a fine-
grained level the contribution of different qualities of knowledge to the
development of the topic. At timestamp $t$, the topic citation network is
$G^{t}=\left(V^{t},E^{t}\right)$. The idea tree extracted from the network is
$IdeaTree\left(G^{t}\right)$. For any paper $v$ belong to
$IdeaTree\left(G^{t}\right)$ except the seminal work, we define its knowledge
entropy ${KE}^{t}\left(v\right)$ as:

${KE}^{t}\left(v\right)=\left|H^{t}\left(v\right)-\sum_{v_{i}\in
C^{t}\left(v\right)}{H^{t}\left(v_{i}\right)}+\sum_{v_{i},v_{j}\in
C^{t}\left(v\right),i\neq j}{I^{t}\left(v_{i},v_{j}\right)}\right|$ (1)

where $H^{t}\left(v\right)$ represents the subtree entropy of the subtree led
by node $v$ at $t$, $C^{t}\left(v\right)$ represents the children of $v$ in
$IdeaTree\left(G^{t}\right)$ at $t$, and $I^{t}\left(v_{i},v_{j}\right)$
represents the mutual subtree entropy of the subtree led by $v_{i}$ and
$v_{j}$ at $t$. A detailed explanation of the formula can be found in
Supporting Materials.

Knowledge changes as it is passed on. For any child node in the idea tree,
each time an idea is inherited along the connection path from the seminal work
to this paper, a difference in knowledge is accumulated. In terms of the whole
topic, when we use pioneering work as a reference, the depth of the idea tree
represents the difference between the new knowledge and the original
knowledge, which characterizes the degree of the development of the topic.
Besides, knowledge entropy helps us to identify it. The appearance of a high
knowledge entropy node at a specific level of the idea tree symbolizes a
qualitative shift in the accumulation of knowledge variation, resulting in the
emergence of knowledge of extraordinary value within the topic, thus marking a
substantial development of the scientific topic. Based on this, we call the
layer in the idea tree that contains at least one node with knowledge entropy
greater than $M$ visible layer. With the development of the topic, the deeper
layers will gradually become visible. Therefore, based on the idea tree and
knowledge entropy, at timestamp $t$, we define the number of visible layers as
the visible depth $\left(VD^{t}\right)$ of the topic to characterize the
degree of development of the topic. Further, we define the maximum visible
depth that an idea tree can achieve over time as the limit of the scientific
topic:

$STL=\lim_{t\rightarrow\infty}VD^{t}$ (2)

Through the statistics of maximum visible depth for more than 30,000 topics
(covering 16 disciplines, published from 1800 to 2019), we find that even
though the scale of the topic (the number of publications) may increase
unlimitedly, STL does exist: the visible depth of the idea tree does not
exceed six (Fig. 2a), which coincides with the “Six Degrees of Separation”
theory of psychology and the “small-world” theory of network science. We find
that the number of topics with a limit depth of 0 is the largest (43.2%, Fig.
2a), suggesting that it is difficult for topics to make the breakthrough from
0 to 1. The decrease in the number of topics with depth increasing indicates
that the later the topic is developed, the more difficult it is for valuable
knowledge to emerge. We are now convinced that there is a limit of the
development and evolution of scientific topics. Next, we will answer this
question: “What is the pattern of evolution of topics from their birth to
limit?”. We utilize idea tree and knowledge entropy to explore the
evolutionary patterns of topics and their internal cause. Among all topics, 16
representative topics have been selected to reveal depth evolutionary
dynamics. The pioneering works of these topics were published between 1959 and
2014, and their research interests fall in domains including machine learning,
wireless network, graph theory, biology, and physics. These topics range from
over 1000 articles and approximately 5000 citations to more than 31000
articles and nearly 200 thousand citations. A detailed description of the
selected topics can be found in Supporting Materials. We find that topic
evolution follows some fixed patterns through our observations and that papers
with non-trivial influence play an important role in driving topic
development.

$\bullet$ Pattern 1: The visible depth of the topic pioneered by summative
work is hard to exceed one (Fig. 2b). The scale of such topics grows over
time, but their visible depth barely increases, which leads to the nodes of
the idea tree expanding only at the first level. Pioneering work tends to
summarize existing knowledge, so it is not very inspiring for child nodes and
cannot provide new research ideas. Once the topic fails to breed new high-
impact work, it will gradually lose attention and then stagnates in
development.
$\bullet$ Pattern 2: The increase in visible depth needs to be driven by non-
trivial child nodes. In the topic led by PWGH (Abbreviations for paper titles,
all titles below use abbreviations instead) (Fig. 2c), child paper RTUE
attracts much external attention, making the subtree led by it flourish and
causing another high knowledge entropy node to be born under its subtree,
which increases the visible depth to two. This shows that RTUE creates
valuable knowledge and drives the topic forward. High knowledge entropy nodes
are particularly important for the development of topics that have been
dormant for many years. When high knowledge entropy nodes appear in the deeper
levels of the idea tree, it will breathe new life into the topic (Fig. 2e).
After a 9-year-long silence of the topic led by CCAT, the influence of SDRT
started to surge in the third level of the idea tree. It continued to breed
two high knowledge entropy nodes in the fourth and fifth levels, which greatly
boosted the development of the topic. It is important to note that even though
no high knowledge entropy work appeared in the early stage of topic
development, the previous work laid the theoretical foundation for the
emergence of the extraordinary impact work that followed.
$\bullet$ Pattern 3: The continuous increase of the visible depth of the topic
needs to be stimulated by the influence relay of multiple high knowledge
entropy nodes (Fig. 2f). We find that the knowledge entropy of extraordinary
child nodes within the topic does not keep increasing. In the topics led by
SSAT, the knowledge entropy of AERT is the first to highlight, but after 2009,
its knowledge entropy almost stopped increasing. At this time, the influence
of BNHY locating in the level behind it began to emerge, achieving multiple
inheritances of ideas and pushing the envelope, ensuring that the topic can
continue to attract attention. We also find that when the influence relay
occurs at a deeper level of the topic, it can increase the visible depth of
the topic significantly (Supporting Materials).
$\bullet$ Pattern 4: The presence of overpowered child nodes can ruin the
increase in the visible depth of the topic(Fig. 2g). In topics led by TCWL,
the knowledge entropy of TCOWN swells rapidly and goes beyond the leading
work. Although TCOWN drives the topic to reach a visible depth of 3 jumps and
showed an amazing impact, the topic’s visible depth increase has stagnated
since 2009. TCOWN has become the new authority, and by 2021, the size of
topics led by TCOWN has reached 8976, and the visible depth even reached seven
jumps, both far more than the topic led by TCWL. The extraordinary innovation
of TCOWN has led to a large gap between the new knowledge inspired by it and
the knowledge created by TCOWN. This gap makes most of the papers that cite
TCOWN do not cite TCWL, thus leaving the original topic without the addition
of new valuable knowledge and stagnating its development.
$\bullet$ Pattern 5: Stronger branches within the topic inhibit the increase
in visible depth of weaker branches (Fig. 2h). In the topic led by JAOK, child
paper HUTY and child paper FGHU were born almost simultaneously. In the early
stage of topic development, the subtree led by HUTY became prosperous first,
which bred high knowledge entropy nodes and thus promoted the topic
development. However, around 2012, the knowledge entropy of child paper FGHU
surpassed HUTY, which led to a stagnation of the knowledge entropy growth of
HUTY. Accordingly, the subtree led by child paper FGHU started to flourish and
drove the topic depth to reach four. In this process, the emerging branch
attracts outside attention, thus making the original branch neglected and miss
the golden period of development, making the original branches stagnant.
$\bullet$ Pattern 6: The increase in the visible depth of the topic can be
brought by interdisciplinary work. In the topic shown in Fig. 2i, the seminal
work POIU belongs to statistics, while the extraordinary influential child
node ASRT appearing in the topic belongs to biology. ASRT applies the theory
put forward by pioneering work to its field and promotes its field
development. This has also led to more biology articles citing seminal work,
which has contributed to an increase in the depth of the original topic.
$\bullet$ Pattern 7: Different topics evolve at different rates (Fig. 2j, k).
The topic led by JIQW shows a continuous growth trend, and after only six
years of development since its birth, its visible depth has reached six jumps.
However, the topic led by KKOP has been growing slowly more than 40 years ago
since its birth, during which it has experienced several long periods of
stagnation.

By observing the pattern of topic evolution, it is easy to see that the
development potential of scientific topics is closely related to articles’
ability to drive visible depth. When work with different knowledge values
appears in different positions of the idea tree, it will inspire different
subtree structures. Combining subtrees with different shapes can get the
skeleton structure of the topic, corresponding to the different evolution
patterns of the topic. Therefore, we conjecture that the existence of a limit
to the development of scientific topics is due to the limited ability of nodes
to drive visible depth. To verify this conjecture, we took out nodes with
knowledge entropy greater than or equal to 10 and subtrees led by them
totaling 58,970 in all topics to analyze the driving effect of nodes with
different knowledge entropy on topic development. We find that the
contribution of a single article to the depth of its topic is log positively
correlated with its knowledge entropy but hardly exceeds three jumps (Fig.
3a). This forces the continuous increase of the visible depth of the topic to
be stimulated by the influence of multiple high-knowledge-entropy nodes in
succession, which leads to the emergence of Pattern 3 in the above section.
Moreover, when the knowledge entropy of the article increases to exceed the
leading work, due to the destruction effect in Pattern 4, this shifts its
effect on the visible depth from facilitation to inhibition, fundamentally
leading to the limit of the depth that the article can drive. The existence of
this limit makes it necessary to spend additional time breeding new valuable
knowledge to drive the increase of visible depth when the topic reaches the
maximum visible depth that a single high-impact article can inspire. We
respectively counted the time interval of topic visible depth change for the
topics whose maximum visible depths are range from 1 to 6. We find that the
visible depth of all types of topics took the longest time to develop from 0
to 1, verifying that the topic depth is difficult to achieve a breakthrough
from 0 to 1 (Fig. 3b). And except for topics with the maximum visible depth of
1 and 2, all types of topics show a significant increase in the time it takes
to germinate a layer later in their development (Fig. 3b). This is because
nodes with extraordinary influence appearing early in the topic reach the
limit of their driving effect on the topic development. The topic takes longer
to germinate new valuable knowledge.

Can we quantify the driving effect of high knowledge entropy nodes on depth to
unify the different evolutionary patterns and further predict visible depth?
Although it is difficult to predict the development of science, we can still
judge and predict the vitality and development potential of the topic based on
the structure of the idea tree. As shown in several examples in Supporting
Materials, we find that nodes with small knowledge entropy are hard to
stimulate depth. When the knowledge entropy does not increase to inhibit the
development of the topic, the greater the knowledge entropy is, the stronger
the node’s ability to stimulate depth. As mentioned above, the ability of a
node to stimulate depth is positively related to its knowledge entropy.
However, there are still cases where some nodes with high knowledge entropy
fail to stimulate depth. We find that these nodes tend to have high knowledge
entropy in the early stage but fail to drive topic depth increase even though
the duration is significantly longer than the time interval of visible depth
increase. This suggests that the increase in visible depth tends to occur
within an effective duration. If a high knowledge node fails to increase the
depth for a more extended period, it will become less attractive for
additional research. It will gradually lose the opportunity to drive the topic
forward. The increase in visible depth results from the combined effect of
increased knowledge entropy and time, and we believe that time begins to decay
the node’s ability to stimulate depth from the influence of the paper
beginning to emerge. In contrast, when the effect of increasing knowledge
entropy is stronger than the attenuation effect of time, it will increase
visible depth. However, the decay effect of time is always present, similar to
how the radioactivity of isotope attenuates over time in nature. Since the
knowledge entropy of nodes will not keep increasing, in the long run, the
driving effect of high knowledge entropy nodes will end in the future, and
then the development of the topic will come to a standstill. We derive the
Idea Limit Formula (ILF) for quantitatively describing the effect of nodes on
driving topic development based on conjecture, fitting, and verification.
Assume that the minimum knowledge entropy of the nodes makes a certain level
of the idea tree visible $KE_{Threshold}=M$. Node $v$ becomes visible at
$t_{0}$, i.e., $KE^{t}\left(v\right)\geq M$. The depth ${\Delta D}^{t}(v)$
that the node $v$ can stimulate for the topic after time t satisfies:

${\Delta D}^{t}(v)\approx\log\frac{KE^{t}(v)}{\left(t-t_{0}\right)^{1.8803}}$
(3)

By transforming the value conditions of the formula, we can effectively
portray the different patterns of a single topic in depth evolution: (1) Too
small knowledge entropy cannot advance the topic depth to achieve a
breakthrough (${\Delta D}^{t}(v)$ is always less than 1), which can explain
the situation that the topic cannot breed new high knowledge entropy nodes and
then reaches the development limit. (2) If the nodes with high knowledge
entropy do not promote the increase of the visible depth within the effective
time, the topic development will stagnate. This can describe the situation
where the topic has bred high knowledge entropy nodes, but the depth has not
been increased for a long time. It also includes Pattern 4 mentioned above.
(3) If the high knowledge entropy nodes appearing in the topic are still in
the prime time to stimulate the depth of the topic, i.e., the upper bound of
${\Delta D}^{t}(v)$ is greater than 1 when specific values are brought into
the formula, we consider the topic to have a very high potential for
development. We cannot predict when a topic with extremely high knowledge
value will be born in the future, so we do not absolutely believe that there
is no development potential for topics without high knowledge nodes.

Based on the predictive power of ILF for topics with development potential, we
can identify scientific topics with potential research value, even without any
prior expertise. This has significant implications for the decision-making
process of resource allocation, researcher hiring, and direction selection. We
selected three scientific topics in computer science, including wireless
network capacity analysis, network information theory, and deep learning. Then
we utilize ILF to identify whether the scientific topics still have the
potential for development (Fig. 4a, b, c). These topics are led by ‘The
capacity of wireless networks(TCWN)’,’Network information flow(NIF)’, ’Image
Super-Resolution Using Deep Convolutional Networks(ISRUDCN)’ respectively.
These topics have produced a great deal of high-impact work and valuable
knowledge throughout their development. However, in our experience, the topics
led by TCWN and NIF have nearly reached their limits of development. Almost
all important issues have been solved in these two topics so that a complete
knowledge system has been formed. We selected all nodes having the potential
to inspire depth for the topic as candidates in the idea tree with the help of
knowledge entropy. After calculating their ${\Delta D}^{t}(v)$, we finally
obtained that all the ${\Delta D}^{t}(v)$ are less than 1, which coincides
with our cognition. Due to the passage of time, the attractiveness of the
valuable knowledge in the topic started to decline, leading to a stagnation of
the topic development. However, in the topic led by ISRUDCN, we also
calculated ${\Delta D}^{t}(v)$ for all jobs with development potential. We
find that the upper bound of ${\Delta D}^{t}(v)$ for IUYTO in the topic is
1.8769, which is greater than 1. This indicates that the subtree led by it is
in a prime period of development and deserves continued research investment,
which promises to produce impactful work and drive the depth of the topic.

To further verify the conclusions obtained in the previous section, we extract
semantic information from each node within the topic. Specifically, first, we
extract the main idea sentence from the abstract using the rule matching
method. Second, we utilize dependency syntactic parsing to obtain the subject
sentence’s core verb and direct object. Finally, we divide the publications
into two categories based on the core verbs and direct objects: summary
articles and research articles. The specific classification method is shown in
the supporting materials. Similar to the seminal paper, we find that the
summary articles within the topic do little to advance the development of the
topic, and the increase of depth is still driven by the research articles.
Among the topics led by the summary articles, the summary articles still
occupy the majority (Fig. 4d). Although these articles occupy most of the
topic, they are repetition and integration of existing knowledge, which makes
it still difficult to move the topic forward. In addition, we find that when
the development of the topic stagnated, the number and proportion of summary
articles increased significantly (Fig. 4e). This also shows that the topic’s
knowledge system has tended to be complete, and the work that can be done at
present is to summarize the previous knowledge. This also shows from the side
that it is difficult for a summary article to stimulate depth for the topic.

Thanks to the rapid development of current deep learning techniques, we can
extract structural and semantic features of articles more accurately, which
will improve the models’ ability to identify the limits and evolutionary
patterns of topics and judge their development potential. However, the
improvement in the ability to quantify knowledge relies on a deeper
understanding of how knowledge itself is generated and how it interacts with
each other. There is an obvious limitation in quantifying knowledge from the
structural perspective: we have difficulty in measuring the knowledge of
articles with extremely sparse structure. For a valuable article, due to a
short publication time or being ignored for a long time for some reason, the
quality of knowledge calculated from the structure will deviate from its own.
Yet, knowledge structure is the external expression of knowledge under complex
interaction mechanisms, and the quality of knowledge contained in an article
will not change since its publication and is not related to the complexity of
knowledge structure. Therefore, the further improvement of model’s capability
relies on how to achieve an effective measurement of knowledge quality in the
early stage of article publication.

## References

  * 1. S. Mukherjee, D. M. Romero, B. Jones, B. Uzzi, Science advances 3, e1601315 (2017).
  * 2. M. McNutt, The measure of research merit (2014).
  * 3. J. W. Weis, J. M. Jacobson, Nature Biotechnology pp. 1–8 (2021).
  * 4. S. Galiani, R. H. Gálvez, The life cycle of scholarly articles across fields of research, Tech. rep., National Bureau of Economic Research (2017).
  * 5. A. Clauset, D. B. Larremore, R. Sinatra, Science 355, 477 (2017).
  * 6. K. K. Mane, K. Börner, Proceedings of the National Academy of Sciences 101, 5287 (2004).
  * 7. R. Kurzweil, The singularity is near: When humans transcend biology (Penguin, 2005).
  * 8. M. M. Waldrop, Nature News 530, 144 (2016).
  * 9. ITRS (2015).
  * 10. J. B. Aimone, Communications of the ACM 62, 110 (2019).
  * 11. H. Esmaeilzadeh, E. Blem, R. S. Amant, K. Sankaralingam, D. Burger, 2011 38th Annual international symposium on computer architecture (ISCA) (IEEE, 2011), pp. 365–376.
  * 12. C. E. Leiserson, et al., Science 368 (2020).
  * 13. K. Börner, J. T. Maru, R. L. Goldstone, Proceedings of the National Academy of Sciences 101, 5266 (2004).
  * 14. Q. Ke, E. Ferrara, F. Radicchi, A. Flammini, Proceedings of the National Academy of Sciences 112, 7426 (2015).
  * 15. E. Garfield, Science 178, 471 (1972).
  * 16. E. Garfield, International journal of epidemiology 35, 1123 (2006).
  * 17. J. E. Hirsch, Proceedings of the National academy of Sciences 102, 16569 (2005).
  * 18. R. Van Noorden, B. Maher, R. Nuzzo, Nature News 514, 550 (2014).
  * 19. R. Sinatra, D. Wang, P. Deville, C. Song, A.-L. Barabási, Science 354 (2016).
  * 20. O. Penner, R. K. Pan, A. M. Petersen, K. Kaski, S. Fortunato, Scientific reports 3, 1 (2013).
  * 21. Y. Ma, B. Uzzi, Proceedings of the National Academy of Sciences 115, 12608 (2018).
  * 22. D. Wang, C. Song, A.-L. Barabási, Science 342, 127 (2013).
  * 23. T. Jia, D. Wang, B. K. Szymanski, Nature Human Behaviour 1, 1 (2017).
  * 24. M. Jacomy, T. Venturini, S. Heymann, M. Bastian, PloS one 9, e98679 (2014).
  * 25. L. Fu, D. Lu, Q. Li, X. Wang, C. Zhou, Plos one 16, e0244618 (2021).