#  CoCo Games: Graphical Game-Theoretic Swarm Control for Communication-Aware
Coverage

Paper ID  Malintha Fernando  Ransalu Senanayake  Martin Swany Malintha
Fernando and Martin Swany are with the Luddy School of Informatics, Computing,
and Engineering at Indiana University, Bloomngton, IN, 47401, USA. Ransalu
Senanayake is with Stanford University, CA, 94305, USA. E-mail:{ccfernan,
swany} @iu.edu<EMAIL_ADDRESS>implementations and the video
demonstrations for this paper can be found at:
https://malintha.github.io/coco.Digital Object Identifier (DOI):
10.1109/LRA.2022.3160968

(December 2019)

###### Abstract

We propose a novel framework for real-time communication-aware coverage
control in networked robot swarms. Our framework unifies the robot dynamics
with network-level message-routing to reach consensus on swarm formations in
the presence of communication uncertainties by leveraging local information.
Specifically, we formulate the communication-aware coverage as a cooperative
graphical game, and use variational inference to reach mixed strategy Nash
equilibria of the stage games. We experimentally validate the proposed
approach in a mobile ad-hoc wireless network scenario using teams of aerial
vehicles and terrestrial user equipment (UE) operating over a large geographic
region of interest. We show that our approach can provide wireless coverage to
stationary and mobile UEs under realistic network conditions. Find the video
demonstrations at https://youtu.be/kQJbc_s4ZLI.

###### Index Terms:

Distributed Robot Systems, Networked Robots, Cooperating Robots

##  I Introduction

Multi-robot systems have been gaining significant attention in
interdisciplinary research areas such as wireless networks and environmental
monitoring thanks to the recent advancements in robotics and telecommunication
sectors [1]. Specifically, mobile ad-hoc wireless networks are emerging as a
disruptive technology to accommodate on-demand coverage and capacity
enhancement for networked robot systems [2, 3]. However, deploying robots in
such applications requires overcoming numerous challenges; maintaining the
connectivity, maximizing the network coverage, and performing real-time motion
planning with limited global information, to name a few.

We propose a novel game-theoretical approach to maximize the coverage over a
geographical region of interest (ROI) under practical network constraints in
real-time. Specifically, we formulate the communication-aware coverage as a
general-sum graphical game where the robots aim to maximize two auxiliary
objectives: 1) coverage, and 2) connectivity, by leveraging local information.
The general-sum games permit each robot’s payoff to arbitrarily relate, thus
making it an ideal framework for coordinating swarms with interconnected
objectives like ours. Additionally, the graphical game-theoretic foundation
underpins using the robots’ local information to reach swarm consensus.
Compared to more conventional stochastic games framework which require one’s
payoff to depend on the joint action profile of all the others, this greatly
simplifies the game structure [4].

Figure 1: A swarm of 3 UAV robots providing wireless network coverage to a
team of 3 travelling UEs. The ROI, UAV-UAV and UAV-UE communication linkes are
represented by the ellipsoid, red and blue dashed lines, respectively. CoCo
games can coordinate the UAV swarm in real time to maximize the coverage for
the ROI independent of the UE movements.

To account for often changing communication topology caused by network
uncertainties, we routinely update the game structure with network-level
information from message-routing tables. In contrast to disk-based coverage
schemes, this renders our approach highly robust to the volatility of the
communication topology caused by the robots’ movements and signal attenuation.
While many swarm control methods require aggregating the global information
[5, 6], the local neighborhood property of graphical games allow us to
selectively integrate partial observations into decision-making, reducing the
communication overhead. Thus, we believe graphical games allow designing more
effective control paradigms for large-scale robot swarms, where the global
state aggregation is intractable due to communication limitations.

By using variational inference (VI), we substantiate the interplay between the
game and an adjoined Markov random field (MRF), whose posterior distribution
resembles the solution of the former. We experimentally validate our approach
in a mobile wireless network scenario with an Unmanned Aerial Vehicle (UAV)
team to provide coverage for a set of User Equipment (UE) over an ROI (Fig.
1).

The main contributions of our work are, 1) Formulation of the communication-
aware coverage as a graphical game, 2) Theoretical guarantees for the stage-
game’s equilibrium by leveraging variational inference, 3) A scalable
algorithm to reach the equilibrium consensus, and 4) Experimental results for
the proposed approach under realistic network conditions.

##  II Related Works

###  II-A Multi-robot Coverage

The coverage problem typically involves deploying a set of mobile nodes over a
field to maximize some objective: wireless signal coverage – in case of
wireless routers or, information gain – in case of sensors [7, 8]. Myriad
literature discusses coordinating wireless nodes over the spectrum of
communication and control methods ranging from disk-based channel models with
centralized control to stochastic models with decentralized control. A widely
known array of work uses disk graph-based methods to coordinate multiple
robots while maintaining the overall network connectivity as the nodes move
[9, 10, 11]. Further, disk-based channel models have also been used for
coverage controlling in [10, 11, 12, 13]. However, most of them overlook the
volatility of the network topology caused by the stochasticity of wireless
signals. Additionally, the “disk” assumption enforces excessively restrictive
control on the robots to maintain the local connectivity, sacrificing the
coverage gain.

In [8, 14] authors present optimization-based decentralized coverage control
for networked robot teams. The former approach mainly relies on a static
coverage function and fails to adjust to dynamic ROIs. In the latter, Kantaros
et al. proposes optimizing similar auxiliary objectives to ours, considering
message-routing and fixed UEs. However, our work differs in its ability to
cater to both fixed and moving UEs while eliminating the need to incorporate
UE positions into the optimization problem explicitly. By using the team
abstraction proposed in [15], we make our approach highly scalable in the
number of UEs. In [16, 17] the authors combine the stochastic channel models
to account for the wireless fading effect in point-to-point communication to
find the optimum router configurations under different routing algorithms.

###  II-B Graphical Games

Graphical games reflect the notion that a multi-player game can be succinctly
represented by a graph, and a player’s payoff only depends on its neighbors’
actions [4]. In [18, 19, 20] the authors established the interplay between
games’ solutions and probabilistic graphical models. Although the notion of
graphical games resembles that of collective dynamics-based approaches, only a
few attempts have been made to employ the framework in swarm coordination,
despite the success gained by the latter. In our previous work [21], we
presented an MRF-based approach to steer a robot swarm to a flocking
consensus, yet the theoretical guarantees and the connection to graphical
games were missing.

In [22, 12] the authors proposed graphical game-theoretic methods for
distributed mobile sensor coverage by conserving energy; however, the works
assume stationary neighborhoods while overlooking the robot dynamics and
network limitations.

##  III Preliminaries

###  III-A Graphical Game Theory

Consider a game involving $n$ players and $\mathcal{A}_{i}$ define the set of
actions available to any player $i\in\\{1,\dots,n\\}$. Let
$x=(x_{1},\dots,x_{n})$ denote the joint action profile of the players. We
allow the players to play mixed strategies, and the probability that $i$ is
playing the action $x_{i}$ is denoted by the mixed strategy $Q_{i}(x_{i})$.
Further, $-i$ denotes the set of all players but $i$,
$(x^{\prime}_{i},x_{-i})$ denotes an alternative action profile where $i$
plays $x^{\prime}_{i}$ instead of $x_{i}$ while $x_{-i}$ remains the same. We
define $Q(x)$ as an arbitrary joint probability distribution over the action
profile $x$ with mixed strategies $Q_{i}(x_{i})$ as the marginals.

A graphical game $\Gamma$ consists of the tuple
$\langle\mathcal{G},\mathbf{M}\rangle$ where $\mathcal{G}$ defines a graph
whose vertices correspond to the players, and $\mathbf{M}$ represents the set
of payoff functions. The graphical game-theory significantly reduce one’s
interacting agents count from $n$ to it’s local neighborhood size [4]. For
some player $i$, $\mathbf{M}_{i}\in\mathbf{M}$,
$\mathbf{M}_{i}:\mathcal{A}_{i}\xrightarrow[]{}\mathbb{R}$. Following the
definition of expectation, we define the expected utility as follows.

######  Definition 1.

The expected utility of player $i$ under a joint probability $Q$ is

$\mathbb{E}_{Q}\big{[}\mathbf{M}_{i}(x)\big{]}=\sum_{x}Q(x_{i},x_{-i})\mathbf{M}_{i}(x_{i},x_{-i}).$
(1)

######  Definition 2.

The correlated equilibrium (CE) of a graphical game is a joint distribution
$Q$ over the associated undirected graphical model, under which no player has
a unilateral incentive to deviate. Thus, for
$x_{i},x^{\prime}_{i}\in\mathcal{A}_{i}$, $\forall i$, and $x_{i}\neq
x^{\prime}_{i}$,

$\mathbb{E}_{Q}\big{[}\mathbf{M}_{i}(x_{-i},x_{i})\big{]}\geq\mathbb{E}_{Q}\big{[}\mathbf{M}_{i}(x_{-i},x^{\prime}_{i})\big{]}.$

######  Definition 3.

A mixed strategy Nash equilibrium (MSNE) is a special case of CE, where the
joint probability is a product distribution of the marginals. Thus, $Q(X)$
$=\prod_{i}Q_{i}(X_{i})$ [20].

###  III-B Variational Energy Functional

The VI casts the inference problem over an MRF as a convex optimization
problem and approximates the posterior distribution much more efficiently in
contrast to exact inference methods [23]. Let
$\mathcal{G}=(\mathcal{V},\mathcal{E})$ be an MRF, where $\mathcal{V}$ and
$\mathcal{E}$ are the set of vertices and edges, and $\mathcal{V}$ consists of
a set of discrete RVs $\\{X_{1},\dots,X_{n}\\}$.

######  Definition 4.

The joint probability distribution over an MRF is often represented as a Gibbs
distribution

$p(X=x)=\frac{1}{Z}\prod_{c\in\mathcal{C}}\phi_{c}(x_{c}),$ (2)

where, $\phi_{c}$ is a factor potential function associated with some clique
$c\in\mathcal{C}$ of $\mathcal{G}$,
$\phi_{c}:X^{|c|}\xrightarrow{}\mathbb{R}^{+}$, and $Z$ is the partition
function to normalize the distribution, where
$Z=\sum_{x_{c}}\prod_{c\in\mathcal{C}}\phi_{c}(x_{c})$.

######  Remark 1.

For any $\phi_{c}=\exp\\{\varepsilon(x_{c})\\}$, $p(x)$ defines an exponential
family distribution, where $\varepsilon:X^{|c|}\xrightarrow{}\mathbb{R}$ is
some function that maps the clique $c$ to a real number.

Let $Q$ and $P$ denote the approximating and the true posterior distributions
in VI. We consider the $I$-projection of Kullback-Leiber (KL)-divergence
between the two distributions,

$D(Q||P_{\mathcal{C}})=\mathbb{E}_{Q}\Big{[}\ln\frac{Q(X)}{P_{\mathcal{C}}(X)}\Big{]},$

where, $P_{\mathcal{C}}$ is the probability distribution over the set of
cliques $\mathcal{C}$. From (2) and, $\mathbb{H}_{Q}(X)=-\mathbb{E}_{Q}[\ln
Q(X)]$;

$D(Q||P_{\mathcal{C}})=-\mathbb{H}_{Q}(X)-\mathbb{E}_{Q}\Big{[}\sum_{c\in\mathcal{C}}\ln\phi_{c}(x_{c})\Big{]}+\mathbb{E}_{Q}[\ln
Z],$ $D(Q||P_{\mathcal{C}})=-F[\tilde{P}_{\mathcal{C}},Q]+\ln Z,$ (3)

for $\tilde{P}_{\mathcal{C}}(x)=\prod_{c\in\mathcal{C}}\phi_{c}(x_{c})$. We
identify $F[\tilde{P}_{\mathcal{C}},Q]$ as the variational energy functional.
From (3), maximum $F$ gives the minimum KL divergence between the
approximating and the true posteriors. In this work, we subsume the two
auxiliary objectives into factor potentials associated with cliques of the
underlying graphical model.

Figure 2: (a) The change of RSS, (b) the number of communicating nodes (blue)
and the average hop-count (red) against the physical distance. The $\inf$ hops
have been ignored when the link disappears for the purpose of plotting. The
parameters used are $n$=3, $\mathcal{F}$=$32(dBm)^{2}$, $T_{0}=16.02$ dBm.

##  IV Problem Formulation

To accommodate a wide range of applications, we consider providing wireless
network coverage to a team of mobile UEs using a robot (UAV) team. Note that
we only control the robots while the UEs move arbitrarily over the ROI. Our
formulation can be further generalized to other coverage control problems such
as surveillance, where the robot teams need to maintain the local connectivity
while patrolling over an ROI. To accommodate the cooperative nature of the
game, we assume that all the robots are interested in maximizing the coverage
and thus, share the same payoff function.

First, we define a concentration ellipsoid $\mathcal{R}$ to represent the ROI
to capture the UE distribution, following [15]. Such an abstraction allows us
to easily scale, and to quantify the total wireless coverage independent of
the UE team size. Let $\Sigma$ define the covariance matrix associated with
$\mathcal{R}$.

Considering the homogeneity in the robots, we populate the action space
$\mathcal{A}_{i}$ by discretizing the dynamically feasible control action
space of a robot. The state equation of a robot $i$ can be written as
$\dot{y}_{i}=Ay_{i}+Bx_{i}$, where $A$, $B$ are positive semi-definite
matrices and $x_{i}\in\mathcal{A}_{i}$. Let
$\dot{y}_{i}^{pos}\in\mathbb{R}^{3}$ be the position of $i$ as extracted from
the state vector $\dot{y}_{i}$.

We consider that each stage of the communication-aware coverage game
corresponds to a timestep $t$ and is a graphical game $\Gamma_{t}$ defined on
the local neighborhoods $\mathcal{N}_{i}^{t},\forall i$. The aim of our work
is to find a consensus UAV formation for $\mathcal{R}$ in the form of a MSNE
for $\Gamma_{t}$. In the following sections, we show that this can be achieved
by leveraging VI and iterative stage game optimization.

##  V Approach

###  V-A Communication-Aware Cooperative Coverage

In practice, the Recieved Signal Strength (RSS) between any two wireless nodes
can attenuate for multiple reasons such as path loss, shadowing, and fading.
Thus, in wireless networks it is common to model the channels in a stochastic
fashion [24]. In this work, we employ a channel model for the IEEE 802.11a
protocol to obtain the expected RSS in point-point communication. For a joint
action set we define
$f_{RSS}:\mathcal{A}_{i}\times\mathcal{A}_{j}\xrightarrow[]{}\mathbb{R}$ as

$f_{RSS}(x_{i},x_{j})=T_{0}-\big{\\{}L_{0}+10n.\log(d_{ij})+\mathcal{F},\big{\\}}$
(4)

where, $d_{ij}=\lVert\dot{y}_{i}^{pos}-\dot{y}_{j}^{pos}\rVert$,
$f_{RSS}(d_{ij})$ is the RSS measured in dBm (decibels relative to milliWatt),
$T_{0}$ is the transmission power, $L_{0}$ is the reference power loss for
free space, $n$ is a path loss exponent and $\mathcal{F}$ is a zero mean
Gaussian distribution to account for the fading effect. In this work, we used
$L_{0}$ = 46.67 dBm calculated using the $Friis$ model for open spaces.
Similar real-world experiments have also been conducted in [24], which helped
us to model the fading in the channel.

Fig. 2(a) shows the change in RSS against the inter-robot distance for the
channel model. As in Fig. 2(b), we observed the network is densely connected
at the beginning, but separated drastically as the nodes moved further away.
Specifically, the hop-count between the robots increased with distance, making
it much harder to maintain fixed neighborhoods. Here the hop-count refers to
the number of interim connections between two communicating nodes in the
network. In practice, as the ROI expands, the robots require to travel farther
in search of better coverage; however, these experimental results signify the
need to adjust the neighborhoods as the network topology changes.

Thus, we define the local neighborhood of any robot $i$ at time $t$,
$\mathcal{N}_{i}^{t}(k)$ as the list of $k$-hop nodes in the instantaneous
network topology. Therefore, for any $j\neq i$,
$\mathcal{N}_{i}^{t}(k)=\\{j\lvert\mathrm{Hops}(i,j)\leq k\\}$, where
$\mathrm{Hops}(i,j)$ is the number of hops to node $j$ according to $i$’s
routing table. With the definition of expectation, we obtain the expected RSS
$\mathbb{E}[f_{RSS}]=\psi_{R}(x_{i},x_{j})$ resulted by selecting the actions
$x_{i},x_{j}$ as

$\psi_{R}(x_{i},x_{j})=T_{0}-\big{\\{}L_{0}+10n.\log(d_{ij})\big{\\}}.$ (5)

We define function $\psi_{C}:\mathcal{A}_{i}\xrightarrow[]{}\mathbb{R}$ to
quantify the coverage provided by the neighborhood $\mathcal{N}_{i}$. We
define the communication-aware coverage as the expected “cooperative RSS
field” for the ellipsoidal $\mathcal{R}$. Therefore,

$\psi_{C}(x_{\mathcal{N}},\mathcal{R})=\sum_{r\in\mathcal{R}}\max\big{\\{}\psi_{R}(x_{i},r),\psi_{R}(x_{-i},r)\big{\\}}p(r),$
(6)

where $p(r)$ is the probability of a point $r\in\mathcal{R}$ in the ROI. In
our work, $p(r)$ denotes the probability of having a UE at $r$ under the
distribution characterized by $\Sigma$. Here, $\psi_{R}(x_{-i},r)$ denotes the
coverage imparted on $r$ by $i$’s any other neighbor. When the values assigned
to the local neighborhood of $i$, $x_{-i}$, and the ROI are fixed, we observe
that the coverage function only varies with $x_{i}$ within the time interval.
Therefore, we denote the coverage function associated with $\mathcal{N}_{i}$
as $\psi_{C}(x_{i})$. This formulation allows us to construct a more realistic
cooperative coverage field for the neighborhood, as UEs often select the
wireless node with the highest signal strength to connect in practice.

Note that for $r$ over large distances, $\psi_{C}(x_{i})$ is stationary for
some robot $j\in\mathcal{N}_{-i}$ when
$\psi_{R}(x_{i},r)\leq\psi_{R}(x_{j},r)$, as any UE at $r$ would connect to
$j$ despite the actions of $i$ due to the higher signal strength. Thus, this
introduces a partitioning in the local coverage functions. Fig. 3(a) shows a
simple network topology for 4 robot nodes and a $k=1$-hop neighborhood for the
$i$-th node. Next, we use this coverage model to introduce the sufficient
statistics for MRF and the payoff functions.

###  V-B Payoff Function for the Stage Graphical Game

Following the communication topology and the neighborhood parameter $k$, we
define the graph $\mathcal{G}_{t}=(\mathcal{V}_{t},\mathcal{E}_{t})$ for the
stage game $\Gamma_{t}$. The set of vertices
$\mathcal{V}_{t}=\\{X_{1},\dots,X_{n}\\}$ comprises the random variables for
each robot node. The set of edges $\mathcal{E}_{t}$ contains an element
$(i,j)$ if and only if $i$ and $j$ satisfies the neighborhood condition under
$k$. Thus, $\mathcal{E}_{t}=\\{(i,j)|j\in\mathcal{N}_{i}^{t}(k),\forall i\\}$.
Hereafter, we ignore the script $t$, as we are interested in finding the
equilibrium for a single-stage game. Also, for a fixed $k$, let
$\mathcal{N}_{i}^{t}(k)=\mathcal{N}_{i}^{t}$.

We consider that the payoff of a robot $i$ relies on the coverage provided by
the neighborhood and its expected RSS with the neighbors. Formally, we define,

$\mathbf{M}_{i}(x)=\alpha_{a}\psi_{C}(x_{i})+\alpha_{b}\sum_{j\in\mathcal{N}_{-i}}\psi_{R}(x_{i},x_{j}),$
(7)

where $\alpha_{a}$, $\alpha_{b}>0$ are two predefined weight parameters. In
this work, they scale the contributions from two factor potentials $\psi_{c}$,
$\psi_{p}$ in the payoff function proportionately. The first term forces the
robot to move farther to maximize the coverage, and the second term penalizes
the robot for selecting the actions that weaken the signal strength. We assume
that the expected RSS remains roughly unchanged in close proximities over
$\mathcal{R}$. Thus, considering the ROI size and inter-robot distances, we
argue that the contradicting effects of the auxiliary objectives can be
ignored for sufficiently small intervals of $t$. Therefore, a robot’s best
response action maximizes the coverage as well as the expected RSS with its
neighbors. In this work, we propose solving the game $\Gamma$ by performing
posterior inference over an appropriately tailored MRF. By using an
exponential family distribution, we establish an analogy between the
equilibrium in the game and the resulting joint distribution.

###  V-C Exponential Family Posterior Distribution for MRF

We now discuss integrating the payoff function with a probabilistic graphical
model defined on the game $\Gamma=(\mathcal{G},\mathbf{M})$. We start by
modifying and factorizing $\mathcal{G}$ into pairwise and neighborhood
subgraphs to complement the communication-aware coverage model defined in (5)
and (6). Let us first introduce an auxiliary edge $(j,h)$ between $j$ and $h$
nodes, if $j,h\in\mathcal{N}_{i}$ and $(j,h)\notin\mathcal{E}$ for all $i$. In
other words, we convert every local neighborhood of $\mathcal{G}$ into a
complete subgraph by adding a set of auxiliary edges,
$(j,h)\in\mathcal{E}_{Aux}$. We denote the derived graph as
$\mathcal{G}^{\prime}=(\mathcal{V},\mathcal{E}^{\prime})$, where
$\mathcal{E}^{\prime}=\mathcal{E}+\mathcal{E}_{Aux}$. Fig. 3(b) shows a clique
transformed neighborhood subgraph for an initial communication topology. We
define neighborhood clique as a clique that is comprised of the nodes of some
neighborhood. Next, we factorize the derived graph into a set of cliques
$\mathcal{C}=\mathcal{E}^{\prime}\cap\\{\mathcal{N}_{i}|\forall i\\}$, such
that each clique $c\in\mathcal{C}$ either represents an edge in
$\mathcal{E}^{\prime}$ or a neighborhood clique. Finally, we associate each
clique $c\in\mathcal{C}$ with a factor potential function
$\phi_{c}:X^{|c|}\xrightarrow[]{}\mathbb{R}^{+}$.

Figure 3:  (a). A visual representation of the local cooperative coverage
function of $i$ for a single hop neighborhood. The partitions covered by each
neighbor are colored differently. (b). A communication topology with the newly
added auxiliary edges (red) to a neighborhood. The dashed polygon denotes the
resulting neighborhood clique. The neighborhood robots and the communication
links are shown in blue.

This formulation allows us to define a joint distribution over the induced MRF
$\mathcal{G}^{\prime}$ using the exponential family similar to (2). Consider

$\phi_{c}(x_{c})=\begin{cases}\hskip
5.0pt\phi_{i}(x_{i})=\exp\big{\\{}\alpha_{i}.\psi_{C}\big{\\}}&c\in\\{\mathcal{N}_{i}|\forall
i\\}\\\ \hskip
5.0pt\phi_{ij}(x_{i},x_{j})=\exp\big{\\{}\alpha_{ij}.\psi_{R}\big{\\}}&c\in\mathcal{E}^{\prime},\\\
\end{cases}$ (8)

where $\phi_{i}$, $\phi_{ij}$ simply redefine the factor potentials given the
clique $c$’s type and $\alpha_{i}$, $\alpha_{ij}\geq 0$ are some weights
associated with the cliques. With this definition, we consider the following
joint probability for the MRF $\mathcal{G}^{\prime}$.

$p(x)=\frac{1}{Z}\exp\big{\\{}\alpha_{a}\sum_{i\in\mathcal{V}}\psi_{C}(x_{i})+\alpha_{b}\sum_{(ij)\in\mathcal{E}}\psi_{R}(x_{i},x_{j})\big{\\}}.$
(9)

######  Theorem 1.

The probability over the MRF $\mathcal{G}^{\prime}$, $p(x)$ defines a linear
exponential family with canonical parameters $\alpha$ and sufficient
statistics $\psi$.

###### Proof.

Define two vectors $\psi$ and $\alpha$ that contain factor potentials and
associated weights for cliques $\mathcal{C}$ of $\mathcal{G}^{\prime}$. Set
each $\alpha_{c}$ associated with the clique $c$; $\alpha_{c}=\alpha_{a}$ for
neighborhood cliques, $\alpha_{c}=\alpha_{b}$ for pairwise cliques, and
$\alpha_{c}=0$ for auxiliary pairwise cliques. Thus, the summations in (9)
become $p(x)=\exp\big{\\{}\langle\alpha,\psi\rangle-\Lambda\big{\\}}$, where
$\langle\alpha,\psi\rangle$ is the inner product $\alpha$, $\psi$ , and
$\Lambda=\ln Z$. Following Remark 1, this defines a linear exponential family.
∎

With Remark 1 and Theorem 1, we observe that the posterior $p(x)$ takes the
standard form of a joint distribution over an MRF. Given $p(x)$ and the payoff
$\mathbf{M}_{i}$, we further notice that two functions share a mutual form.
Specifically, the summation terms that pertain to a single robot $i$ inside
the exponential of the former resembles the payoff function. In the following
section, we perform posterior inference over the MRF $\mathcal{G}$ and
establish that the resulting probability distribution $Q(x)$ yields a
consensus formation of the stage game $\Gamma$.

###  V-D Stage Game Optimization with MFVI

In Theorem 1 we established that the joint posterior $p(x)$ adheres to the
form in (2), and in general, $\varepsilon(x_{c})=\psi_{c}(x_{c})$ for all
$c\in\mathcal{C}$. Thus, we start by considering the variational energy
functional $F$, associated with (9). It is clear from (3) that maximizing $F$
results in the minimum KL divergence between the true and the approximating
posteriors. Here we use MFVI to optimize $F$ and to obtain the approximating
posterior. Specifically, the mean-field assumption considers the posterior
distribution as a product of marginals each corresponding to a single RV.
Therefore, the posterior consists of a set of independently and identically
distributed (iid) exponential marginals characterized by their means [25].
This assumption improves the tractability of the inference procedure by
restricting the search space to the mean-field family instead of the entire
space of distributions. Using the mean-field assumption, we formally define
the optimization problem as

$\displaystyle\mathrm{Finding}\quad$ $\displaystyle\\{Q_{i}(X_{i})\\},$
$\displaystyle\mathrm{Maximizing}\quad$ $\displaystyle
F[\tilde{P}_{\mathcal{C}},Q],$ $\displaystyle\mathrm{Subjecting\,to}\quad$
$\displaystyle Q(X)=\prod_{i}{Q_{i}(X_{i})},$
$\displaystyle\sum_{x_{i}}{Q(x_{i})}=1,\forall i=1\dots,n.,$

where $Q_{i}(x_{i})$ denotes the marginal probability distribution of $i$ in
the posterior. The goal of the MFVI is to find an update rule for each
marginal distribution while keeping the neighboring RVs fixed. From (3), the
variational energy functional takes the form

$F[\tilde{P}_{\mathcal{C}},Q]=\mathbb{H}_{Q}(X)+\mathbb{E}_{Q}\Big{[}\sum_{c\in\mathcal{C}}\ln\phi_{c}(x_{c})\Big{]}.$

We consider the variational energy imparted on $i$ from its neighborhood
$\mathcal{N}_{i}$ under joint $Q$ expressively,

$F_{i}=\mathbb{H}_{Q}(X_{i})+\mathbb{E}_{Q}\Big{[}\sum_{j\in\mathcal{N}_{-i}}\ln\phi_{ij}(x_{i},x_{j})+\ln\phi_{i}(x_{i})\Big{]}.$
(11)

In order to maximize $F_{i}$, we write the Lagrangian $\mathcal{L}_{i}$ with
the Lagrange multiplier being $\lambda$ as

$\mathcal{L}_{i}[Q]=\mathbb{H}_{Q}(X_{i})+\sum_{c\in\mathcal{C}}\mathbb{E}_{Q}\Big{[}\ln\phi_{c}\Big{]}+\lambda\Big{\\{}\sum_{x_{i}}{Q_{i}(x_{i})}-1\Big{\\}}.$

Next, we differentiate the Lagrangian $\mathcal{L}_{i}$ w.r.t the marginal
$Q_{i}(x_{i})$ and obtain the fixed points corresponding to the maximum.

$\begin{split}\frac{d}{dQ(x)}\mathcal{L}_{i}[Q]&=-\ln
Q(x)-1+\sum_{c\in\mathcal{C}}\mathbb{E}_{Q}\big{[}\ln\phi_{c}|x_{i}\big{]}+\lambda.\end{split}$
(12)

During the differentiation step we used the facts that the derivatives of the
entropy $\mathbb{H}_{Q}(X_{i})$ and expectation $\mathbb{E}_{Q}[\ln\phi_{c}]$
terms w.r.t $Q(x_{i})$ are $-\ln Q(x_{i})-1$ and
$\mathbb{E}_{Q}[\ln\phi_{c}|x_{i}]$ respectively. Here
$\mathbb{E}_{Q}\big{[}\ln\phi_{c}|x_{i}\big{]}$ denotes the conditional
expectation of the factor potential $\phi_{c}$ given the value $x_{i}$. Notice
that the factor function $\phi_{c}$ changes according to (8) given the clique
type. Setting the derivative to 0 and taking the exponentials of the both
sides yields the update rule

$Q_{i}(x_{i})=\frac{1}{Z_{i}}\exp\Big{\\{}\mathbb{E}_{Q}\Big{[}\sum_{j\in\mathcal{N}_{-i}}\ln\phi_{ij}|x_{i}+\ln\phi_{i}\Big{]}\Big{\\}},$
(13)

where $Z_{i}$ is a typical exponential family normalization constant, as
introduced in Definition 4, and the Lagrange multiplier $\lambda$ gets dropped
in the normalization. The resulting update rule is also known as the
coordinate ascent mean-field approximation. Further, the redefinition of
$\phi_{c}$ in (8) ensures that $\ln\phi_{c}(x_{c})$ exists and reflects the
coverage model.

1Populate $\mathcal{N}_{i},\forall i$ using the routing table for $i$

2 Construct $\mathcal{G}^{\prime}$

3 for  _$i\leftarrow 1\dots r\dots n$_ do

4 $Q_{i}(X_{i})\leftarrow\frac{1}{Z_{i}}\\{\phi_{i}(x_{i})\\}$

5

6 end for

7while  _$Q_{old}(X)\neq Q(X)$_ do

8 $Q_{old}(X)=Q(X)$

9 Choose $X_{i}$ from $\\{X_{1},\dots,X_{n}\\}$

10
$\hat{Q}_{i}(x_{i},x_{j})\leftarrow\sum_{\begin{subarray}{c}j\in\mathcal{N}_{-i}\end{subarray}}\Big{\\{}\sum_{x_{j}}\ln\phi_{ij}(x_{i},x_{j})\Big{\\}}$

11 $\tilde{Q}_{i}(x_{i})\leftarrow\sum_{x_{i}}\ln\phi_{i}(x_{i})$

12
$Q_{i}(x_{i})\leftarrow\frac{1}{Z_{i}}\exp\\{\hat{Q}_{i}(x_{i},x_{j})+\tilde{Q}_{i}(x_{i})\\}$

13 $Q(X)\leftarrow\prod_{i}Q_{i}(X_{i})$

14 end while

Execute $\operatorname*{arg\,max}_{x_{i}}Q_{i}(X_{i})$ on robot $i$

Algorithm 1 Mean-Field Stage Game Optimization

The stage game optimization algorithm summarizes the key steps of MFVI into an
iterative procedure to solve the graphical game
$\Gamma=\langle\mathcal{G},\mathbf{M}\rangle$ by performing posterior
inference on MRF $\mathcal{G}^{\prime}$. Specifically, we optimize each RV of
the MRF to calculate the marginal probabilities $Q_{i}(X_{i})$ while fixing
the neighboring RVs. In the graphical game theoretical paradigm, this is
equivalent to calculating the mixed strategies profile of each player given
the neighborhood. In the next section, we next show that a resulting posterior
probability distribution $Q$ induces an equilibrium in the graphical game
$\Gamma$.

###  V-E Equilibrium in the Stage Game

Consider the modified stage game $\Gamma=\langle\mathcal{G},\mathbf{M}\rangle$
where $\mathbf{M}_{i}\in\mathbf{M}$. In this section, we discuss the interplay
between the equilibrium solutions of $\Gamma$ and the posterior probability of
the factorized MRF induced by $\mathcal{G}^{\prime}$.

######  Theorem 2.

The joint posterior distribution $Q^{*}(x)$ over the MRF induced by
$\mathcal{G}^{\prime}$ results a CE for the stage game.

###### Proof.

According to (11) and (12), each marginal $Q_{i}(X_{i})$ that comprises
$Q^{*}(X)$ defines a local maximum of the energy functional $F_{i}$.
Therefore, for some actions $x_{i},x^{\prime}_{i}\in\mathcal{A}_{i}$, under
$Q^{*}(X)$ the marginals $Q_{i}(x_{i})\geq Q_{i}(x^{\prime}_{i})$, and thus,

$\ln Q_{i}(x_{i})\geq\ln Q_{i}(x^{\prime}_{i}).$ (14)

Now consider an action profile $x=(x_{i},x_{-i})$ where $x_{-i}$ represents
the actions of $\mathcal{N}_{-i}$ and $x_{j}\in x_{-i}$ for some
$j\in\mathcal{N}_{-i}$. From (13),

$\ln
Q_{i}(x_{i})\propto\mathbb{E}_{Q^{*}}\Big{[}\sum_{j\in\mathcal{N}_{-i}}\ln\phi_{ij}(x_{i},x_{j})+\ln\phi_{i}(x_{i})\Big{]}.$

Substituting from the definition (8) for $\phi_{i},\phi_{ij}$ gives,

$\ln
Q_{i}(x_{i})\propto\mathbb{E}_{Q^{*}}\Big{[}\sum_{j\in\mathcal{N}_{-i}}\alpha_{b}\psi_{R}(x_{i},x_{j})+\alpha_{a}\psi_{C}(x_{i})\Big{]}.$

Notice that the summations inside the expectation match the definition of a
player’s payoff function (7). Therefore, substituting from the payoff
function,

$\ln
Q_{i}(x_{i})\propto\mathbb{E}_{Q^{*}}\Big{[}\mathbf{M}_{i}(x_{i},x_{-i})\Big{]}.$
(15)

From (14) and (15),

$\mathbb{E}_{Q^{*}}\big{[}\mathbf{M}_{i}(x_{i},x_{-i})\big{]}\geq\mathbb{E}_{Q^{*}}\big{[}\mathbf{M}_{i}(x^{\prime}_{i},x_{-i})\big{]}.$

Thus, according to the definition, $Q^{*}(X)$ yields a CE of the stage game
$\Gamma$. ∎

According to Definition 3, we observe that the CE resulting from MFVI is
indeed an MSNE, due to the product form of $Q^{*}(X)$. Therefore, we argue
that the posterior inference over MRF $\mathcal{G}^{\prime}$ results in a
consensus formation for the stage game $\Gamma$. However, recall that the
stage game is sub-terminal as the calculated mixed-strategy actions are the
controls of the robots’ dynamical model. Therefore, we solve the stage game in
iteratively to obtain continuous trajectories and, subsequently, a consensus
formation for the communication-aware coverage game. The distributed
configuration of our approach reduces the stage game graph into the
neighborhood clique of the derived graph $\mathcal{G}^{\prime}$ for any robot,
rendering the distributed setting much more desirable for controlling large-
scale swarms.

##  VI Experiments and Results

###  VI-A Experiments Setup

We evaluated the proposed approach using NS-3 and Robot Operating System (ROS)
environments in a mobile wireless network scenario, that consists of UAVs and
a mobile UE team.111Find the Mavswarm simulator we used for this work at:
https://github.com/malintha/multi_uav_simulator. Briefly, NS-3 is a widely
employed event simulator to design and implement network models, and we
delegate the task of network packet routing to NS-3 by accounting for wireless
signal attenuation as the nodes move. Each UAV was equipped with two wireless
interfaces that complied with the channel model to establish the 1) inter-UAV,
and 2) UAV-UE links. The action set of a UAV consists of acceleration-based
discretized control inputs, and we select the actions by optimizing the stage
games iteratively to reach a consensus formation. The UAVs communicate with
their neighbors over the inter-UAV network for optimizing the game. The UAV-UE
links were only used to measure the coverage RSS of the UEs for evaluation
purposes.

Further, to calculate the routing paths as the UAV and UE nodes move, we used
NS-3’s inbuilt Optimized Link State Routing (OLSR) algorithm [26]. The OLSR
algorithm’s ability to populate the routing tables by accounting for the
communication uncertainties helps us update the local neighborhoods in real-
time. The ROS and NS-3 platforms communicate through Remote Procedure Calls
(RPC) to update the game accordingly. For all the UAVs, we ran the stage game
optimization algorithm at a modest frequency $1/t=$ 1Hz in order to account
for communication delays. We further assume that the UAVs can observe the ROI
abstraction, characterized by its mean and the covariance matrix $\Sigma$, by
communicating with a base station. We considered a differentially flat
dynamical model for fixed altitude navigation for the UAVs. Thus, we uniformly
sampled the acceleration space of a UAV within bounds [$-3ms^{2},3ms^{2}$] to
populate $\mathcal{A}_{i}\subset\mathbb{R}^{2},\forall i$ along each X, Y
dimension similar to our previous work [21]. For the computational
feasibility, we discretized the ROI $\mathcal{R}$ into $10m\times 10m$ cells.
All the implementations and experiments are conducted using the C++
programming language.

###  VI-B Experiments

Figure 4: (a). Change of payoff according to Eq. (7) as the UAV swarm navigate
over a stationary ROI. The values are plotted by changing the swarm size and
the communication hop-count. (b). Computational time to optimize a single
stage of the game against the number of neighbors $|\mathcal{N}_{i}|$ and the
size of the ROI. Each cell represents an area of 100$m^{2}$.

We evaluated the coverage and computational performance of the proposed
approach against the number of UAVs and the size of the ROI. We accommodated
the latter scenario by allowing the UE team to travel between arbitrarily
chosen start and goal locations, resulting the ROI’s shape and size to change
over time (Fig. 5). We observed that for small ROIs, $\alpha_{b}$’s effect is
minimal, thus rendering the cooperative coverage to govern the payoff. This
reduces the tuning effort of the proposed method to a great extent. Throughout
the experiments we used the parameters $\alpha_{a}=1$ and $\alpha_{b}=0.001$
to cater to varying ROI sizes.

We first computed the average payoff of a UAV for the stationary ROI case by
varying the maximum hop-count and the number of UAV nodes in the swarm as in
Fig. 4(a). Even though the single-hop neighborhoods’ payoffs overlapped with
those of higher-order at close proximities, as the UAVs travel farther, the
stability of the formations and the payoffs deteriorated. This is mainly
caused by the densely connected network topology at close proximities; which
permits the UAVs to observe the global swarm state with fewer hops. Therefore,
it is understandable that as the network separates, the cooperative coverage
deviates from the global value due to the increased locality. By increasing
the allowed maximum hops, we show our approach can result in better payoffs
and stable UAV formations; without explicitly aggregating the global swarm
state. Thanks to the adaptive neighborhood property that complements the
network dynamics, our approach scales well with the size of the swarm and ROI.

For the performance evaluations, we used $k=3$ as the allowed communication
hops throughout the experiments. We evaluated computational time for two
crucial steps in the game: 1) calculating the cooperative coverage and 2)
mean-field approximation stages. Specifically, we performed the calculations
against the number of the neighbors in a game $|\mathcal{N}_{i}|$, and the
size of the discretized ROI. In the implementation, we calculated the fixed
values for any $r\in\mathcal{R}$ in the cooperative coverage function
$\psi_{C}$ beforehand to eliminate redundant calculations and $\max(.)$
comparisons within a single stage of the game. We observed this to improve the
computational efficiency by multitudes, especially optimizing a single stage
of the game under 50ms for most cases. Fig. 4(b) shows the computational time
for the two corresponding steps.

Fig. 5 shows the robot trajectories for $n=3,5$ and $8$ UAVs scenarios.  We
defined the ROI using the concentration ellipsoid for 5 UEs distributed around
the origin. In each scenario, we simulated the system with stationary and
moving ROIs for 40s and 80s intervals, respectively. We initialized the UAVs
from the origin for all the experiments with a fixed altitude, and maintaining
the UEs stationary at first. Fig. 5(a-c) show the equilibrium UAV formations
and the communication topologies.

Figure 5: Trajectories of robot nodes with time. The top row shows the
equilibrium communication topology for a stationary ROI and the initial UE
positions (blue). Fig. (d)-(i) represents the UAV (red) and UE (blue)
trajectories as the UE nodes move to their designated goal positions. The
green ellipsoidal fence represents the concentration ellipsoid for the ROI
$\mathcal{R}$. Note that the robot models are not to the scale for
visualization purposes.

Similarly, Fig. 5 (d-i) show the UAV and UE trajectories for the moving ROI
experiments. The UEs travels between the start and arbitrarily chosen goal
locations by morphing the ROI. The blue squares and dots represent the start
and current locations of the UEs. With the stage game optimization, the UAV
swarm repeatedly move to maximize the coverage as the ROI changes. As the UE
team members reach their destinations, the UAV swarm converge to the
equilibrium formations as showed in Fig. 5(g-i).

Figure 6: Change of the average RSS of UE nodes against time in stationary (a)
and moving (b) scenarios for different UAV swarm sizes, using the CoCo model.
For clarity, only the error for $n=8$ scenario is plotted throughout the
experiments.

For clarity, only the most recent trajectory trails of the UAVs are showed. We
repeated each experiment for 20 trials using NS-3 to perform network events
such as attenuating wireless signals, routing, and calculating RSS for each
communication link. In 6(a-b), we present the average RSS as observed by a UE
for for the experiments. Throughout the convergence process, we observed that
the UAVs to constantly increase their cooperative payoff, leading to higher
RSS in the UEs. We observed significant improvements in the average RSS as the
UAVs reached the equilibrium formations under both scenarios. Also, it’s worth
noting that decibel is a logarithmic scale, and every 3 unit increments double
the signal strength. The global connectivity of the network was maintained
throughout the experiments, as the OLSR algorithm was able to find routes
between the network nodes successfully. The final communication topologies for
the moving ROI experiments were observed to be more densely connected than the
stationary scenarios due to the relatively small ROI size.

We compare the network and optimization aspects of our framework to that of
the widely used disk-based model and the decentralized, adaptive coverage
method presented in [8]. Following our initial findings presented in Fig.
4(b), we selected the average single-hop distance $\approx 60$m as the radii
for the disk model under similar environmental constraints. We report the
average observed RSS of a UE in Fig. 7(a-b) for the same configurations. The
disk-based models resulted in RSS gains initially. However, as the UAVs moved
beyond each others’ disk radii, the algorithm failed to reach a consensus
formation. Even though increasing the communication radii seems like a trivial
solution, we emphasize that it can risk losing the global connectivity
altogether due to the highly stochastic nature of the wireless signals at
large distances.

The decentralized, adaptive coverage control [8] follows a similar approach to
partition the environment based on the observed sensor gain for each robot.
For the comparison, we implemented the coverage as a bivariate Gaussian
function defined at the center of $\mathcal{R}$ with covariance $\Sigma$. The
work, however, limits itself to static coverage functions and fails to handle
the scenarios with dynamic ROIs and stochastic communication links. In
contrast to our unified framework, it also overlooks the robots’ dynamics,
thus the calculated paths needed to be further smoothed with trajectory
optimization before deploying on the UAVs. Fig. 7(c) shows the average RSS
resulting in a UE using the decentralized, adaptive coverage control. Even
though the approach converged to the consensus faster, it failed to yield
higher RSS values in practice due to rigid connectivity and the overlooked
dynamics.

Figure 7: Comparisons: Average RSS of UE nodes in stationary (a) and moving
ROI (b) scenarios for different UAV swarm sizes using a fixed-radii model. (c)
Average RSS plots for the adaptive coverage method in the stationary scenario.

##  VII Conclusion

We have proposed a novel game-theoretic swarm coordination framework to
achieve communication-aware coverage for robot swarms by only utilizing the
local information. Our work complements the underlying network and robot
dynamic models in neighborhood selection and control, resulting a robust
coverage scheme for large-scale ROIs. We have evaluated our approach in an ad-
hoc mobile wireless network scenario to show that it can result in significant
coverage gains while maintaining the local connectivity. We have further
established that our approach achieves real-time control and consensus for
networked robot teams.

## References

  * [1] M. Mozaffari, W. Saad, M. Bennis, Y.-H. Nam, and M. Debbah, “A tutorial on uavs for wireless networks: Applications, challenges, and open problems,” IEEE communications surveys & tutorials, vol. 21, no. 3, pp. 2334–2360, 2019.
  * [2] V. Sharma, M. Bennis, and R. Kumar, “Uav-assisted heterogeneous networks for capacity enhancement,” IEEE Communications Letters, vol. 20, no. 6, pp. 1207–1210, 2016.
  * [3] G. Skorobogatov, C. Barrado, and E. Salamí, “Multiple uav systems: a survey,” Unmanned Systems, vol. 8, no. 02, pp. 149–169, 2020.
  * [4] M. J. Kearns, M. L. Littman, and S. P. Singh, “Graphical models for game theory,” in Proceedings of the 17th Conference in Uncertainty in Artificial Intelligence, UAI ’01, (San Francisco, CA, USA), p. 253–260, Morgan Kaufmann Publishers Inc., 2001.
  * [5] H. Zhu, J. Juhl, L. Ferranti, and J. Alonso-Mora, “Distributed multi-robot formation splitting and merging in dynamic environments,” in 2019 International Conference on Robotics and Automation (ICRA), pp. 9080–9086, IEEE, 2019.
  * [6] B. Şenbaşlar, W. Hönig, and N. Ayanian, “Robust trajectory execution for multi-robot teams using distributed real-time replanning,” in Distributed autonomous robotic systems, pp. 167–181, Springer, 2019.
  * [7] W. Li and C. G. Cassandras, “Distributed cooperative coverage control of sensor networks,” in Proceedings of the 44th IEEE Conference on Decision and Control, pp. 2542–2547, IEEE, 2005.
  * [8] M. Schwager, D. Rus, and J.-J. Slotine, “Decentralized, adaptive coverage control for networked robots,” The International Journal of Robotics Research, vol. 28, no. 3, pp. 357–375, 2009.
  * [9] M. Ji and M. Egerstedt, “Distributed coordination control of multiagent systems while preserving connectedness,” IEEE Transactions on Robotics, vol. 23, no. 4, pp. 693–703, 2007.
  * [10] G. Notarstefano, K. Savla, F. Bullo, and A. Jadbabaie, “Maintaining limited-range connectivity among second-order agents,” in 2006 American control conference, pp. 6–pp, IEEE, 2006.
  * [11] P. Yang, R. A. Freeman, G. J. Gordon, K. M. Lynch, S. S. Srinivasa, and R. Sukthankar, “Decentralized estimation and control of graph connectivity for mobile sensor networks,” Automatica, vol. 46, no. 2, pp. 390–396, 2010\.
  * [12] E. Paraskevas, D. Maity, and J. S. Baras, “Distributed energy-aware mobile sensor coverage: A game theoretic approach,” in 2016 American Control Conference (ACC), pp. 6259–6264, IEEE, 2016.
  * [13] J.-M. Etancelin, A. Fabbri, F. Guinand, and M. Rosalie, “Dacyclem: A decentralized algorithm for maximizing coverage and lifetime in a mobile wireless sensor network,” Ad Hoc Networks, vol. 87, pp. 174–187, 2019\.
  * [14] Y. Kantaros and M. M. Zavlanos, “Distributed communication-aware coverage control by mobile sensor networks,” Automatica, vol. 63, pp. 209–220, 2016\.
  * [15] C. Belta and V. Kumar, “Abstraction and control for groups of robots,” IEEE Transactions on robotics, vol. 20, no. 5, pp. 865–875, 2004.
  * [16] D. Mox, M. Calvo-Fullana, M. Gerasimenko, J. Fink, V. Kumar, and A. Ribeiro, “Mobile wireless network infrastructure on demand,” in 2020 IEEE International Conference on Robotics and Automation (ICRA), pp. 7726–7732, IEEE, 2020.
  * [17] Y. Yan and Y. Mostofi, “Robotic router formation in realistic communication environments,” IEEE Transactions on Robotics, vol. 28, no. 4, pp. 810–827, 2012.
  * [18] C. Daskalakis and C. H. Papadimitriou, “Computing pure nash equilibria in graphical games via markov random fields,” in Proceedings of the 7th ACM Conference on Electronic Commerce, pp. 91–99, 2006.
  * [19] S. Kakade, M. Kearns, J. Langford, and L. Ortiz, “Correlated equilibria in graphical games,” in Proceedings of the 4th ACM Conference on Electronic Commerce, pp. 42–47, 2003.
  * [20] L. E. Ortiz, B. Wang, and Z. Gong, “Correlated equilibria for approximate variational inference in mrfs,” in International Conference on Probabilistic Graphical Models, pp. 329–340, PMLR, 2020.
  * [21] M. Fernando, “Online flocking control of uavs with mean-field approximation,” in 2021 IEEE International Conference on Robotics and Automation (ICRA), pp. 8977–8983, 2021.
  * [22] X. Ai, V. Srinivasan, and C.-K. Tham, “Optimality and complexity of pure nash equilibria in the coverage game,” IEEE Journal on Selected Areas in Communications, vol. 26, no. 7, pp. 1170–1182, 2008.
  * [23] D. Koller and N. Friedman, Probabilistic graphical models: principles and techniques.  MIT press, 2009.
  * [24] J. Fink, A. Ribeiro, and V. Kumar, “Robust control for mobility and wireless communication in cyber–physical systems with application to robot teams,” Proceedings of the IEEE, vol. 100, no. 1, pp. 164–178, 2011.
  * [25] M. J. Wainwright and M. I. Jordan, Graphical models, exponential families, and variational inference.  Now Publishers Inc, 2008.
  * [26] T. Clausen and P. Jacquet, “Rfc3626: Optimized link state routing protocol (olsr),” 2003.